2024-04-25T09:49:11.274 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-25T09:49:11.275 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_neesingh@teuthology --name neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi --description fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} --archive /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043 -- /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/orig.config.yaml 2024-04-25T09:49:11.322 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T09:49:11.403 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043 branch: wip-neesingh-testing-240416 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7673043' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-neesingh-testing-240416 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds cache max size: 1000 mds debug frag: true mds debug scatterstat: true mds log max segments: 1 mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - Replacing daemon mds - Scrub error on inode - Behind on trimming - Metadata damage detected - bad backtrace on inode - overall HEALTH_ - \(MDS_TRIM\) sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec kclient: syntax: v1 subvolume_version: 1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62537 sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 owner: scheduled_neesingh@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 1124 sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec sleep_before_teardown: 0 suite: fs:functional suite_branch: wip-62537 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: a3b7a189d278a9762c1acb7de36790ef0ed7b448 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6aup13/8uXndBL7OybaWL5ojZP3xZkhfwUedxkjro7f5VMctuST/SOP6MuUCBjN1Qap8rZapUydXA7FkvMNJk= smithi172.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsm4vg/xcnI6aXvixXr/+qzZ14gpGbLouNFU4J2X0xkBFy26DfUNH5vwRcycNBuhrTI56C3KhERAQOhuhrcrnM= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_scrub_checks - tasks.cephfs.test_scrub teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-25_09:36:50 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-25T09:49:11.404 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa; will attempt to use it 2024-04-25T09:49:11.404 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa/tasks 2024-04-25T09:49:11.404 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-25T09:49:11.406 INFO:teuthology.task.internal:Checking packages... 2024-04-25T09:49:11.429 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec' 2024-04-25T09:49:11.430 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-25T09:49:11.430 INFO:teuthology.packaging:ref: None 2024-04-25T09:49:11.430 INFO:teuthology.packaging:tag: None 2024-04-25T09:49:11.430 INFO:teuthology.packaging:branch: wip-neesingh-testing-240416 2024-04-25T09:49:11.430 INFO:teuthology.packaging:sha1: 97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:49:11.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-neesingh-testing-240416 2024-04-25T09:49:11.575 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1628.g97ae6cc9 2024-04-25T09:49:11.576 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-25T09:49:11.587 INFO:teuthology.task.internal:no buildpackages task found 2024-04-25T09:49:11.587 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-25T09:49:11.607 INFO:teuthology.task.internal:Saving configuration 2024-04-25T09:49:11.624 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-25T09:49:11.641 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-25T09:49:11.665 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:44:03.174815', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6aup13/8uXndBL7OybaWL5ojZP3xZkhfwUedxkjro7f5VMctuST/SOP6MuUCBjN1Qap8rZapUydXA7FkvMNJk='} 2024-04-25T09:49:11.690 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi172.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-25 09:44:03.172789', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsm4vg/xcnI6aXvixXr/+qzZ14gpGbLouNFU4J2X0xkBFy26DfUNH5vwRcycNBuhrTI56C3KhERAQOhuhrcrnM='} 2024-04-25T09:49:11.690 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-25T09:49:11.704 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-04-25T09:49:11.704 INFO:teuthology.task.internal:roles: ubuntu@smithi172.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-04-25T09:49:11.704 INFO:teuthology.run_tasks:Running task console_log... 2024-04-25T09:49:11.786 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd8c488f160>, signals=[15]) 2024-04-25T09:49:11.786 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-25T09:49:11.793 INFO:teuthology.task.internal:Opening connections... 2024-04-25T09:49:11.794 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T09:49:11.796 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:49:11.875 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T09:49:11.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:49:11.953 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-25T09:49:11.962 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-25T09:49:11.980 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-25T09:49:11.981 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:49:12.038 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-25T09:49:12.039 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:49:12.039 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:49:12.039 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:49:12.039 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:49:12.039 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:49:12.039 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:49:12.040 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-25T09:49:12.060 DEBUG:teuthology.orchestra.run.smithi172:> uname -m 2024-04-25T09:49:12.078 INFO:teuthology.orchestra.run.smithi172.stdout:x86_64 2024-04-25T09:49:12.079 DEBUG:teuthology.orchestra.run.smithi172:> cat /etc/os-release 2024-04-25T09:49:12.135 INFO:teuthology.orchestra.run.smithi172.stdout:NAME="CentOS Stream" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION="9" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:ID="centos" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:ID_LIKE="rhel fedora" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:VERSION_ID="9" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:PLATFORM_ID="platform:el9" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:ANSI_COLOR="0;31" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:LOGO="fedora-logo-icon" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:HOME_URL="https://centos.org/" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-25T09:49:12.136 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-25T09:49:12.137 INFO:teuthology.orchestra.run.smithi172.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-25T09:49:12.137 INFO:teuthology.lock.ops:Updating smithi172.front.sepia.ceph.com on lock server 2024-04-25T09:49:12.160 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-25T09:49:12.171 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-25T09:49:12.178 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-25T09:49:12.178 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:49:12.180 DEBUG:teuthology.orchestra.run.smithi172:> test '!' -e /home/ubuntu/cephtest 2024-04-25T09:49:12.196 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-25T09:49:12.202 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-25T09:49:12.202 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:49:12.237 DEBUG:teuthology.orchestra.run.smithi172:> test -z $(ls -A /var/lib/ceph) 2024-04-25T09:49:12.341 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-25T09:49:12.396 INFO:teuthology.run_tasks:Running task kernel... 2024-04-25T09:49:12.412 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2024-04-25T09:49:12.412 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-04-25T09:49:12.412 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-04-25T09:49:12.413 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:49:12.413 DEBUG:teuthology.orchestra.run.smithi172:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-25T09:49:12.429 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:49:12.430 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T09:49:12.433 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:49:12.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-04-25T09:49:12.630 DEBUG:teuthology.task.kernel:sha1 for client.1 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:12.630 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:49:12.631 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-04-25T09:49:12.642 DEBUG:teuthology.task.kernel:sha1 for client.0 is 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:12.642 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "806ec36b21129f113ce356c007dd59694d340b37"... 2024-04-25T09:49:12.643 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-04-25T09:49:12.648 INFO:teuthology.orchestra.run.smithi172.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:49:12.648 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:12.649 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:49:12.649 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum upgrade -y linux-firmware 2024-04-25T09:49:12.660 INFO:teuthology.orchestra.run.smithi165.stdout:5.14.0-437.el9.x86_64 2024-04-25T09:49:12.660 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-437.el9.x86_64 vs 806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:12.660 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-04-25T09:49:12.660 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum upgrade -y linux-firmware 2024-04-25T09:49:15.061 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 9.7 MB/s | 8.0 MB 00:00 2024-04-25T09:49:15.365 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 7.1 MB/s | 8.0 MB 00:01 2024-04-25T09:49:17.441 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-25T09:49:18.320 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 19 MB 00:02 2024-04-25T09:49:21.583 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 16 MB/s | 6.2 MB 00:00 2024-04-25T09:49:22.862 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 6.4 MB/s | 6.2 MB 00:00 2024-04-25T09:49:23.276 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-04-25T09:49:24.404 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 42 kB/s | 16 kB 00:00 2024-04-25T09:49:26.702 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 6.3 MB/s | 21 MB 00:03 2024-04-25T09:49:31.969 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 8.6 kB/s | 1.7 kB 00:00 2024-04-25T09:49:33.982 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-04-25T09:49:33.983 INFO:teuthology.orchestra.run.smithi172.stdout:Nothing to do. 2024-04-25T09:49:33.984 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-04-25T09:49:34.094 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:49:34.095 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:34.262 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T09:49:34.262 DEBUG:teuthology.orchestra.run.smithi172:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T09:49:34.265 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi172.front.sepia.ceph.com to complete... 2024-04-25T09:49:43.947 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 1.1 MB/s | 21 MB 00:19 2024-04-25T09:49:48.919 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-04-25T09:49:50.886 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-25T09:49:50.886 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-25T09:49:50.887 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-25T09:49:51.016 INFO:teuthology.task.kernel:Downloading kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:49:51.017 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=806ec36b21129f113ce356c007dd59694d340b37 2024-04-25T09:49:51.160 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ 2024-04-25T09:49:51.161 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/ --input-file=- 2024-04-25T09:49:51.163 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi165.front.sepia.ceph.com to complete... 2024-04-25T09:49:52.594 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25 09:49:52 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T09:49:52.594 INFO:teuthology.orchestra.run.smithi172.stderr:FINISHED --2024-04-25 09:49:52-- 2024-04-25T09:49:52.594 INFO:teuthology.orchestra.run.smithi172.stderr:Total wall clock time: 18s 2024-04-25T09:49:52.594 INFO:teuthology.orchestra.run.smithi172.stderr:Downloaded: 1 files, 341M in 18s (19.0 MB/s) 2024-04-25T09:49:52.597 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.1... 2024-04-25T09:49:52.597 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T09:49:54.673 INFO:teuthology.orchestra.run.smithi172.stdout:Verifying... ######################################## 2024-04-25T09:49:54.802 INFO:teuthology.orchestra.run.smithi172.stdout:Preparing... ######################################## 2024-04-25T09:49:54.940 INFO:teuthology.orchestra.run.smithi172.stdout:Updating / installing... 2024-04-25T09:50:02.959 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T09:50:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25 09:50:03 URL:https://4.chacra.ceph.com/r/kernel/testing/806ec36b21129f113ce356c007dd59694d340b37/centos/9/flavors/default/x86_64/kernel-6.9.0_rc3_g806ec36b2112-1.x86_64.rpm [357367792/357367792] -> "/tmp/kernel.x86_64.rpm" [1] 2024-04-25T09:50:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:FINISHED --2024-04-25 09:50:03-- 2024-04-25T09:50:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:Total wall clock time: 13s 2024-04-25T09:50:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:Downloaded: 1 files, 341M in 12s (27.4 MB/s) 2024-04-25T09:50:03.944 INFO:teuthology.task.kernel:Installing kernel 806ec36b21129f113ce356c007dd59694d340b37 on client.0... 2024-04-25T09:50:03.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-04-25T09:50:06.071 INFO:teuthology.orchestra.run.smithi165.stdout:Verifying... ######################################## 2024-04-25T09:50:06.306 INFO:teuthology.orchestra.run.smithi165.stdout:Preparing... ######################################## 2024-04-25T09:50:06.433 INFO:teuthology.orchestra.run.smithi165.stdout:Updating / installing... 2024-04-25T09:50:14.491 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-6.9.0_rc3_g806ec36b2112-1 ######################################## 2024-04-25T09:50:42.049 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi172.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T09:50:42.049 DEBUG:teuthology.orchestra.run.smithi172:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:42.154 INFO:teuthology.orchestra.run.smithi172.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.154 INFO:teuthology.orchestra.run.smithi172.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.154 INFO:teuthology.orchestra.run.smithi172.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.154 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T09:50:42.154 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:50:42.155 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:50:42.156 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:50:42.157 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:50:42.158 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:50:42.159 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:50:42.160 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:50:42.161 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:50:42.162 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:50:42.163 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:50:42.164 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:50:42.165 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T09:50:42.166 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T09:50:42.167 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:50:42.168 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T09:50:42.169 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:50:42.170 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T09:50:42.171 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T09:50:42.172 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:50:42.173 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:50:42.174 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:50:42.175 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:50:42.176 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T09:50:42.177 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:50:42.178 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:50:42.179 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:50:42.180 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:50:42.181 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:50:42.182 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T09:50:42.183 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:50:42.184 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:50:42.185 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:50:42.186 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:50:42.187 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:50:42.188 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T09:50:42.189 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:50:42.190 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:50:42.191 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:50:42.192 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:50:42.193 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:50:42.194 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:50:42.195 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:50:42.196 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T09:50:42.197 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:50:42.198 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:50:42.199 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T09:50:42.200 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:50:42.201 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T09:50:42.202 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:50:42.203 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:50:42.204 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T09:50:42.205 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:50:42.206 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T09:50:42.207 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T09:50:42.208 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T09:50:42.209 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:50:42.210 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:50:42.211 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:50:42.212 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:50:42.213 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:50:42.214 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:50:42.215 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:50:42.216 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:50:42.217 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:50:42.218 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:50:42.219 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:50:42.220 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:50:42.221 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:50:42.222 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T09:50:42.223 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:50:42.224 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T09:50:42.225 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:50:42.226 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:50:42.227 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:50:42.228 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:50:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:50:42.230 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:50:42.231 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:50:42.232 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:50:42.233 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T09:50:42.234 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:50:42.235 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:50:42.236 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:50:42.237 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:50:42.238 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:50:42.239 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T09:50:42.240 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:50:42.241 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:50:42.241 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:50:42.241 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:50:42.241 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T09:50:42.242 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:50:42.243 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:50:42.244 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:50:42.245 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T09:50:42.246 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:50:42.247 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:50:42.248 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T09:50:42.249 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:50:42.250 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:50:42.251 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:50:42.252 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:50:42.253 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T09:50:42.254 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:50:42.255 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T09:50:42.256 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:50:42.257 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T09:50:42.258 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:50:42.259 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:50:42.260 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:50:42.261 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:50:42.262 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T09:50:42.263 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:50:42.264 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:50:42.265 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:50:42.266 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:50:42.267 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:50:42.268 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:50:42.269 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:50:42.270 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:50:42.271 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:50:42.272 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:50:42.273 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:50:42.274 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T09:50:42.275 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:50:42.276 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:50:42.277 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:50:42.278 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:50:42.279 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T09:50:42.280 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:50:42.281 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:50:42.282 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:50:42.283 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:50:42.284 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:50:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:50:42.286 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:50:42.287 INFO:teuthology.orchestra.run.smithi172.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:42.291 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.291 DEBUG:teuthology.orchestra.run.smithi172:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:42.316 INFO:teuthology.orchestra.run.smithi172.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:fi 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:for file in vmlinuz System.map config; do 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: fi 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:done 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:fi 2024-04-25T09:50:42.317 INFO:teuthology.orchestra.run.smithi172.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T09:50:42.318 INFO:teuthology.orchestra.run.smithi172.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.318 INFO:teuthology.orchestra.run.smithi172.stdout:fi 2024-04-25T09:50:42.318 INFO:teuthology.orchestra.run.smithi172.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T09:50:42.318 INFO:teuthology.orchestra.run.smithi172.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:42.318 INFO:teuthology.orchestra.run.smithi172.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.318 INFO:teuthology.orchestra.run.smithi172.stdout:fi 2024-04-25T09:50:42.319 DEBUG:teuthology.orchestra.run.smithi172:> sudo rpm -qi grub2-tools 2024-04-25T09:50:42.394 INFO:teuthology.orchestra.run.smithi172.stdout:Name : grub2-tools 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Epoch : 1 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Version : 2.06 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Release : 68.el9 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Architecture: x86_64 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Group : Unspecified 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Size : 8264254 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:License : GPLv3+ 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Packager : builder@centos.org 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:Vendor : CentOS 2024-04-25T09:50:42.395 INFO:teuthology.orchestra.run.smithi172.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:Summary : Support tools for GRUB. 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:Description : 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:hardware devices. 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T09:50:42.396 INFO:teuthology.orchestra.run.smithi172.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T09:50:42.397 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T09:50:42.397 INFO:teuthology.task.kernel:Updating grub on smithi172 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:42.397 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T09:50:42.982 INFO:teuthology.orchestra.run.smithi172.stderr:Generating grub configuration file ... 2024-04-25T09:50:44.464 INFO:teuthology.orchestra.run.smithi172.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T09:50:44.489 INFO:teuthology.orchestra.run.smithi172.stderr:done 2024-04-25T09:50:44.491 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-25T09:50:44.507 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.uRPKs3baVb 2024-04-25T09:50:44.507 DEBUG:teuthology.orchestra.run.smithi172:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.uRPKs3baVb 2024-04-25T09:50:44.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.uRPKs3baVb 2024-04-25T09:50:44.713 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.uRPKs3baVb is 6KB 2024-04-25T09:50:44.762 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.uRPKs3baVb 2024-04-25T09:50:44.776 DEBUG:teuthology.orchestra.run.smithi172:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T09:50:44.838 INFO:teuthology.orchestra.run.smithi172.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T09:50:44.838 INFO:teuthology.orchestra.run.smithi172.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T09:50:44.838 INFO:teuthology.orchestra.run.smithi172.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T09:50:44.838 INFO:teuthology.orchestra.run.smithi172.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T09:50:44.840 DEBUG:teuthology.orchestra.run.smithi172:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:44.982 DEBUG:teuthology.orchestra.run.smithi172:> sudo shutdown -r now 2024-04-25T09:50:53.632 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi165.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-04-25T09:50:53.632 DEBUG:teuthology.orchestra.run.smithi165:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:53.713 INFO:teuthology.orchestra.run.smithi165.stdout:/boot/System.map-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.713 INFO:teuthology.orchestra.run.smithi165.stdout:/boot/config-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/boot/vmlinuz-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/System.map 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/config 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-04-25T09:50:53.714 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crc32c-intel.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-avx2.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-04-25T09:50:53.715 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-amd.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm-intel.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/arch/x86/kvm/kvm.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ansi_cprng.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/anubis.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/arc4.ko 2024-04-25T09:50:53.716 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_memcpy.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_pq.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_raid6_recov.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_tx.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/async_xor.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/async_tx/raid6test.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authenc.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/authencesn.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blake2b_generic.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_common.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/blowfish_generic.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/camellia_generic.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast5_generic.ko 2024-04-25T09:50:53.717 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast6_generic.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cast_common.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/ccm.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cmac.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crc32_generic.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_engine.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/crypto_user.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/cts.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/des_generic.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/dh_generic.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/echainiv.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/essiv.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/fcrypt.ko 2024-04-25T09:50:53.718 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/khazad.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/lz4hc.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/md4.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/michael_mic.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcbc.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/pcrypt.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/rmd160.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/seed.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/serpent_generic.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tcrypt.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/tea.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_common.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/twofish_generic.ko 2024-04-25T09:50:53.719 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/vmac.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/wp512.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xcbc.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xor.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/crypto/xxhash_generic.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_ipmi.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/acpi_pad.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/ec_sys.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/platform_profile.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbs.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/sbshc.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/acpi/video.ko 2024-04-25T09:50:53.720 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/acard-ahci.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ahci_platform.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/ata_generic.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/libahci_platform.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_acpi.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ali.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_amd.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_artop.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atiixp.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_atp867x.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd640.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cmd64x.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_cypress.ko 2024-04-25T09:50:53.721 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_efar.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt366.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt37x.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x2n.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_hpt3x3.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it8213.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_it821x.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_jmicron.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_marvell.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_mpiix.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_netcell.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ninja32.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87410.ko 2024-04-25T09:50:53.722 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_ns87415.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_oldpiix.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_opti.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_optidma.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pcmcia.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc2027x.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_piccolo.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_rdc.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sch.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_serverworks.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sil680.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sis.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_sl82c105.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_triflex.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pata_via.ko 2024-04-25T09:50:53.723 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/pdc_adma.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_inic162x.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_mv.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_nv.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_promise.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_qstor.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sil24.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sis.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_svw.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_sx4.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_uli.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_via.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ata/sata_vsc.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/auxdisplay/ks0108.ko 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base 2024-04-25T09:50:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-i2c.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/base/regmap/regmap-mmio.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/bcma/bcma.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/aoe/aoe.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/brd.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/drbd/drbd.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/floppy.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/loop.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-04-25T09:50:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/nbd.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/null_blk/null_blk.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/pktcdvd.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/rbd.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/virtio_blk.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/xen-blkfront.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/block/zram/zram.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hangcheck-timer.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/amd-rng.ko 2024-04-25T09:50:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/intel-rng.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/via-rng.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/hw_random/virtio-rng.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_si.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/lp.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/mwave/mwave.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/ppdev.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tlclk.ko 2024-04-25T09:50:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_atmel.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_infineon.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_nsc.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/uv_mmtimer.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/char/virtio_console.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/p4-clockmod.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/powernow-k8.ko 2024-04-25T09:50:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/cpufreq/speedstep-lib.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/ccp/ccp.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-aes.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/padlock-sha.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca 2024-04-25T09:50:53.729 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dca/dca.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/dw/dw_dmac.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/dma/ioat/ioatdma.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/amd64_edac.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/e752x_edac.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/edac_mce_amd.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3000_edac.ko 2024-04-25T09:50:53.730 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i3200_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5100_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i5400_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7300_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i7core_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/i82975x_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/ie31200_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/sb_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/edac/x38_edac.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/edd.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/firmware/iscsi_ibft.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-ich.ko 2024-04-25T09:50:53.731 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/gpio/gpio-viperboard.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-appleir.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-aureal.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-axff.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-dr.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elecom.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-elo.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-emsff.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gaff.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gt683r.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-gyration.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-kbd.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtek-mouse.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-holtekff.ko 2024-04-25T09:50:53.732 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-hyperv.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-icade.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-keytouch.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-kye.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lcpower.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-led.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-lenovo.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-dj.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-multitouch.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-ortek.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-penmount.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-petalynx.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-picolcd.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-pl.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-plantronics.ko 2024-04-25T09:50:53.733 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-primax.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-rmi.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-arvo.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-common.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-isku.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kone.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-konepure.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-lua.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-pyra.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-ryos.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat-savu.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-roccat.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-saitek.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-samsung.ko 2024-04-25T09:50:53.734 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sensor-hub.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sjoy.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sony.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-speedlink.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-steelseries.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-sunplus.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-thrustmaster.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tivo.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-tmff.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-topseed.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-twinhan.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-uclogic.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-waltop.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-wiimote.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-xinmo.ko 2024-04-25T09:50:53.735 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zpff.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/hid-zydacron.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/uhid.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hid/wacom.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_balloon.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_utils.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hv/hv_vmbus.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/abituguru3.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/acpi_power_meter.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7414.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ad7418.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adc128d818.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1025.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1026.ko 2024-04-25T09:50:53.736 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1029.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm1031.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adm9240.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ads7828.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7410.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7411.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7462.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7470.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7475.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/adt7x10.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/amc6821.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/applesmc.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asb100.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asc7621.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/asus_atk0110.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/atxp1.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/coretemp.ko 2024-04-25T09:50:53.737 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/dme1737.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds1621.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ds620.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc1403.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/emc6w201.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71805f.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f71882fg.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/f75375s.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fam15h_power.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/fschmd.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g760a.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/g762.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl518sm.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/gl520sm.ko 2024-04-25T09:50:53.738 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/hwmon-vid.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5500_temp.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/i5k_amb.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmaem.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ibmpex.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina209.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ina2xx.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/it87.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k10temp.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/k8temp.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lineage-pem.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm63.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm73.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm75.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm77.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm78.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm80.ko 2024-04-25T09:50:53.739 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm83.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm85.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm87.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm90.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm92.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm93.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95234.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95241.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/lm95245.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc2945.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4151.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4215.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4222.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4245.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4260.ko 2024-04-25T09:50:53.740 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/ltc4261.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max16065.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1619.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max1668.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max197.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6639.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6650.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/max6697.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/mcp3021.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6683.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775-core.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct6775.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/nct7802.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87360.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pc87427.ko 2024-04-25T09:50:53.741 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pcf8591.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max16064.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max34440.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/max8688.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/powr1220.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5627.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch5636.ko 2024-04-25T09:50:53.742 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sch56xx-common.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht15.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sht21.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/shtc1.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/sis5595.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47b397.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m1.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/smsc47m192.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/thmc50.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp102.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp103.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp401.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/tmp421.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via-cputemp.ko 2024-04-25T09:50:53.743 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/via686a.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt1211.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/vt8231.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627ehf.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83627hf.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83781d.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83791d.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83792d.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83793.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83795.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l785ts.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/hwmon/w83l786ng.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-04-25T09:50:53.744 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-i801.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-isch.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-parport.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-04-25T09:50:53.745 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-via.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-dev.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-mux.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-smbus.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/i2c/i2c-stub.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_cm.ko 2024-04-25T09:50:53.746 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_core.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_umad.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/iw_cm.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_cm.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib 2024-04-25T09:50:53.747 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/fm801-gp.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/gameport/gameport.ko 2024-04-25T09:50:53.748 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/joydev.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/keyboard/gpio_keys.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/appletouch.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/bcm5974.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/cyapatp.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/elan_i2c.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/sermouse.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/synaptics_usb.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/mouse/vsxxxaa.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/rmi4/rmi_core.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/altera_ps2.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/arc_ps2.ko 2024-04-25T09:50:53.749 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/serio/serio_raw.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/input/sparse-keymap.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-blinkm.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lm3530.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-lp3944.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/leds-ss4200.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-audio.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-04-25T09:50:53.750 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/bcache/bcache.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-bio-prison.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache-smq.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-cache.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-crypt.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-delay.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-flakey.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-log-userspace.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-multipath.ko 2024-04-25T09:50:53.751 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-queue-length.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-raid.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-round-robin.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-service-time.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-switch.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-thin-pool.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/dm-verity.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid0.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid1.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid10.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/md/raid456.ko 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion 2024-04-25T09:50:53.752 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptbase.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptctl.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptfc.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptlan.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptsas.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptscsih.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/message/fusion/mptspi.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_ich.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/lpc_sch.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/mfd-core.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/sm501.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/viperboard.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/vx855.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/mfd/wl1273-core.ko 2024-04-25T09:50:53.753 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds9802als.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/apds990x.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/bh1770glc.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/cb710/cb710.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/at24.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/eeprom/max6875.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/enclosure.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/hpilo.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29003.ko 2024-04-25T09:50:53.754 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/isl29020.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-me.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei-txe.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/mei/mei.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-gru/gru.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xp.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpc.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_7xx1.ko 2024-04-25T09:50:53.755 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tifm_core.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/tsl2550.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_balloon.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/bonding/bonding.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/b53/b53_common.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/bcm-sf2.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6060.ko 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx 2024-04-25T09:50:53.756 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/dummy.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/eql.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/8390.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec 2024-04-25T09:50:53.757 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/agere/et131x.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-04-25T09:50:53.758 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-04-25T09:50:53.759 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-04-25T09:50:53.760 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-04-25T09:50:53.761 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/dnet.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ethoc.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/fealnx.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e100.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k 2024-04-25T09:50:53.762 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/jme.ko 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell 2024-04-25T09:50:53.763 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/skge.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi 2024-04-25T09:50:53.764 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-04-25T09:50:53.765 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/atp.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis 2024-04-25T09:50:53.766 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis190.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sis/sis900.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/cassini.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/niu.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sungem.ko 2024-04-25T09:50:53.767 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/ti/tlan.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/geneve.ko 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv 2024-04-25T09:50:53.768 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ieee802154/fakelb.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ifb.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ipvlan/ipvlan.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvlan.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/macvtap.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mdio/mdio-regmap.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/mii.ko 2024-04-25T09:50:53.769 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/net_failover.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/netconsole.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/nlmon.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs-lynx.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/amd.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm7xxx.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/bcm87xx.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/broadcom.ko 2024-04-25T09:50:53.770 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/cicada.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/davicom.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/dp83640.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/et1011c.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/icplus.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/lxt.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/marvell.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/micrel.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/national.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/phylink.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/qsemi.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/realtek.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/smsc.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/ste10Xp.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/phy/vitesse.ko 2024-04-25T09:50:53.771 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/bsd_comp.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_async.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_deflate.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_generic.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_mppe.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/ppp_synctty.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppoe.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pppox.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/ppp/pptp.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slhc.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/slip/slip.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/sungem_phy.ko 2024-04-25T09:50:53.772 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tap.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_activebackup.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_broadcast.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_random.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/tun.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/veth.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/virtio_net.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/vxlan/vxlan.ko 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback 2024-04-25T09:50:53.773 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netback/xen-netback.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/net/xen-netfront.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ntb/ntb.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme-core.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/nvme/host/nvme.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_cs.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_pc.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/parport/parport_serial.ko 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci 2024-04-25T09:50:53.774 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/pcie/aer_inject.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pci/xen-pcifront.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/i82092.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/pd6729.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pcmcia/yenta_socket.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acer-wmi.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/acerhdf.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/apple-gmux.ko 2024-04-25T09:50:53.775 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-laptop.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/asus-wmi.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/classmate-laptop.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/hdaps.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/intel_ips.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/msi-wmi.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/mxm-wmi.ko 2024-04-25T09:50:53.776 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-laptop.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/samsung-q10.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/topstar-laptop.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/toshiba_haps.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi-bmof.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/platform/x86/wmi.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_common.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/powercap/intel_rapl_msr.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-gpio.ko 2024-04-25T09:50:53.777 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps-ldisc.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/clients/pps_parport.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/pps/pps_core.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ptp/ptp_kvm.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ras/amd/fmpm.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-bq32k.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1286.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1307.ko 2024-04-25T09:50:53.778 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1374.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1511.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1553.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1672.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds1742.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds2404.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-ds3232.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-em3027.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-fm3130.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl12022.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-isl1208.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m41t80.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t35.ko 2024-04-25T09:50:53.779 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-m48t59.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-max6900.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-msm6242.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf2127.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf85063.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8523.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8563.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-pcf8583.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rp5c01.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rs5c372.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8025.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-rx8581.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-04-25T09:50:53.780 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/rtc/rtc-x1205.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-9xxx.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-sas.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/3w-xxxx.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/BusLogic.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/a100u2w.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aacraid/aacraid.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/advansys.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/am53c974.ko 2024-04-25T09:50:53.781 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/atp870u.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bfa/bfa.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ch.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/csiostor/csiostor.ko 2024-04-25T09:50:53.782 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dc395x.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/dmx3191d.ko 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r 2024-04-25T09:50:53.783 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esas2r/esas2r.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/esp_scsi.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/fcoe.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/fnic/fnic.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hpsa.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hptiop.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/hv_storvsc.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/initio.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ipr.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ips.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/isci/isci.ko 2024-04-25T09:50:53.784 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/iscsi_tcp.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libfc/libfc.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libiscsi_tcp.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/libsas/libsas.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/lpfc/lpfc.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-04-25T09:50:53.785 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvsas/mvsas.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/mvumi.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/pmcraid.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla1280.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-04-25T09:50:53.786 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/raid_class.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_debug.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_fc.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_sas.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_spi.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/scsi_transport_srp.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/ses.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/st.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/stex.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/virtio_scsi.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/vmw_pvscsi.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/scsi/wd719x.ko 2024-04-25T09:50:53.787 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ssb/ssb.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/loopback/tcm_loop.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_file.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_iblock.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_mod.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_pscsi.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/target_core_user.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-04-25T09:50:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-04-25T09:50:53.789 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_gsm.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/n_hdlc.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/nozomi.ko 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial 2024-04-25T09:50:53.790 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/arc_uart.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/tty/serial/jsm/jsm.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/core/ufshcd-core.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_aec.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_cif.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_pci_generic.ko 2024-04-25T09:50:53.791 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/uio/uio_sercos3.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-acm.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/cdc-wdm.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usblp.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/class/usbtmc.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/host/sl811-hcd.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/mdc800.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/image/microtek.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/adutux.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/appledisplay.ko 2024-04-25T09:50:53.792 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi26.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/emi62.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ezusb.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/idmouse.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/iowarrior.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/isight_firmware.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/ldusb.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/legousbtower.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/trancevibrator.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usb3503.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usblcd.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/usbsevseg.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/uss720.ko 2024-04-25T09:50:53.793 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/misc/yurex.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/phy/phy-generic.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/aircable.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ark3116.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/belkin_sa.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ch341.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cp210x.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cyberjack.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/cypress_m8.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/digi_acceleport.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/empeg.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ftdi_sio.ko 2024-04-25T09:50:53.794 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/garmin_gps.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_edgeport.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/io_ti.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipaq.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ipw.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ir-usb.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/keyspan_pda.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kl5kusb105.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/kobil_sct.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mct_u232.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7720.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/mos7840.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/navman.ko 2024-04-25T09:50:53.795 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/omninet.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/opticon.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/option.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/oti6858.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/pl2303.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcaux.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/qcserial.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/quatech2.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/safe_serial.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/sierra.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/spcp8x5.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ssu100.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/symbolserial.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-04-25T09:50:53.796 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_debug.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/usb_wwan.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/visor.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/whiteheat.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/serial/xsens_mt.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/uas.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-alauda.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-cypress.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-datafab.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-freecom.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-isd200.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-karma.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-onetouch.ko 2024-04-25T09:50:53.797 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-realtek.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr09.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-sddr55.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/ums-usbat.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/storage/usb-storage.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-core.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/usbip-host.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/pci/vfio-pci.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio.ko 2024-04-25T09:50:53.798 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_iotlb.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_net.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/vhost/vhost_scsi.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/apple_bl.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/lcd.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/video/backlight/platform_lcd.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_balloon.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_mmio.ko 2024-04-25T09:50:53.799 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/virtio/virtio_ring.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2482.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/masters/ds2490.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-04-25T09:50:53.800 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_smem.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/slaves/w1_therm.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/w1/wire.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim1535_wdt.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/alim7101_wdt.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/f71808e_wdt.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/hpwdt.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/i6300esb.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/iTCO_wdt.ko 2024-04-25T09:50:53.801 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ib700wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ibmasr.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it8712f_wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/it87_wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/machzwd.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/nv_tco.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_pci.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/pcwd_usb.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sch311x_wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/softdog.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/sp5100_tco.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/via_wdt.ko 2024-04-25T09:50:53.802 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83877f_wdt.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/w83977f_wdt.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/wdt_pci.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/watchdog/xen_wdt.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-acpi-processor.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-evtchn.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntalloc.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-gntdev.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xen-privcmd.ko 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs 2024-04-25T09:50:53.803 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/drivers/xen/xenfs/xenfs.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/9p/9p.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/affs/affs.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/befs/befs.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/binfmt_misc.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/btrfs/btrfs.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cachefiles/cachefiles.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ceph/ceph.ko 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda 2024-04-25T09:50:53.804 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/coda/coda.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/cramfs/cramfs.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/dlm/dlm.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ecryptfs/ecryptfs.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/f2fs/f2fs.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/fat.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/msdos.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fat/vfat.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/cuse.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/fuse/fuse.ko 2024-04-25T09:50:53.805 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/gfs2/gfs2.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfs/hfs.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/hfsplus/hfsplus.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/isofs/isofs.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/jfs/jfs.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/lockd/lockd.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/minix/minix.ko 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs 2024-04-25T09:50:53.806 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/netfs/netfs.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfs.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv3.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs/nfsv4.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/grace.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfs_common/nfs_acl.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nfsd/nfsd.ko 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2 2024-04-25T09:50:53.807 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nilfs2/nilfs2.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-celtic.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-centeuro.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-croatian.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-cyrillic.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-gaelic.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-greek.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-iceland.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-inuit.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-roman.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-romanian.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/mac-turkish.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1250.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1251.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp1255.ko 2024-04-25T09:50:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp737.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp775.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp850.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp852.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp855.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp857.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp860.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp861.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp862.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp863.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp864.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp865.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp866.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp869.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp874.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp932.ko 2024-04-25T09:50:53.809 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp936.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp949.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_cp950.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_euc-jp.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-1.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-13.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-14.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-15.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-2.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-3.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-4.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-5.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-6.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-7.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_iso8859-9.ko 2024-04-25T09:50:53.810 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-r.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-ru.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_koi8-u.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_ucs2_utils.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/nls/nls_utf8.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-04-25T09:50:53.811 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/overlayfs/overlay.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/pstore/ramoops.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/reiserfs/reiserfs.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/romfs/romfs.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/client/cifs.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_arc4.ko 2024-04-25T09:50:53.812 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/smb/common/cifs_md4.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/squashfs/squashfs.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/sysv/sysv.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/udf/udf.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/ufs/ufs.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/fs/xfs/xfs.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/rcu/rcutorture.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/torture.ko 2024-04-25T09:50:53.813 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/asn1_encoder.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc-itu-t.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crc8.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libarc4.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/crypto/libdes.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/libcrc32c.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lru_cache.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4_compress.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/lz4/lz4hc_compress.ko 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math 2024-04-25T09:50:53.814 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/math/cordic.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/raid6/raid6_pq.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/reed_solomon/reed_solomon.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_bm.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_fsm.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/ts_kmp.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/lib/zstd/zstd_compress.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/mm/hwpoison-inject.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/6lowpan.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_dest.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_fragment.ko 2024-04-25T09:50:53.815 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_hop.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_ipv6.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_mobility.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_routing.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/6lowpan/nhc_udp.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/garp.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/mrp.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/p8022.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/psnap.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/802/stp.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/8021q/8021q.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet.ko 2024-04-25T09:50:53.816 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_fd.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_rdma.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/9p/9pnet_virtio.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/batman-adv/batman-adv.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/br_netfilter.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/bridge.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_among.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arp.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip.ko 2024-04-25T09:50:53.817 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_limit.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_log.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_snat.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_stp.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/ebtables.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-04-25T09:50:53.818 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ceph/libceph.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/failover.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/core/pktgen.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_diag.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv4.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dccp/dccp_ipv6.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dns_resolver/dns_resolver.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/dsa_core.ko 2024-04-25T09:50:53.819 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_brcm.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_dsa.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_none.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/dsa/tag_trailer.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ieee802154/ieee802154_socket.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ah4.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/esp4.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/fou.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/gre.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/inet_diag.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_gre.ko 2024-04-25T09:50:53.820 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_tunnel.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ip_vti.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipcomp.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/ipip.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arp_tables.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-04-25T09:50:53.821 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/iptable_security.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_bic.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_dctcp.ko 2024-04-25T09:50:53.822 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_diag.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_highspeed.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_htcp.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_hybla.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_illinois.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_lp.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_scalable.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_vegas.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_veno.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_westwood.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tcp_yeah.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/tunnel4.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_diag.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/udp_tunnel.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv4/xfrm4_tunnel.ko 2024-04-25T09:50:53.823 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ah6.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/esp6.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/fou6.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_tunnel.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ip6_vti.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/ipcomp6.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-04-25T09:50:53.824 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-04-25T09:50:53.825 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/sit.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/tunnel6.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/ipv6/xfrm6_tunnel.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/key/af_key.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_core.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_debugfs.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_eth.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip.ko 2024-04-25T09:50:53.826 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ip6.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_netlink.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/l2tp/l2tp_ppp.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/llc/llc.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mac802154/mac802154.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/mpls/mpls_gso.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-04-25T09:50:53.827 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-04-25T09:50:53.828 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conncount.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-04-25T09:50:53.829 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_h323.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_irc.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sane.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_sip.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_dup_netdev.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_flow_table_inet.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_log_syslog.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat.ko 2024-04-25T09:50:53.830 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_amanda.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_ftp.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_irc.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_sip.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_nat_tftp.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_synproxy_core.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nf_tables.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_acct.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_log.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_osf.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nfnetlink_queue.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_chain_nat.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_compat.ko 2024-04-25T09:50:53.831 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_ct.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_dup_netdev.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fib.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_fwd_netdev.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_hash.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_limit.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_log.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_masq.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_nat.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_numgen.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_queue.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_quota.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_redir.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_reject_inet.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_socket.ko 2024-04-25T09:50:53.832 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/nft_tunnel.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_AUDIT.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CHECKSUM.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CLASSIFY.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_CT.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_DSCP.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HL.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_HMARK.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_IDLETIMER.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LED.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_LOG.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_MASQUERADE.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NETMAP.ko 2024-04-25T09:50:53.833 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFLOG.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_NFQUEUE.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_RATEEST.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_REDIRECT.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_SECMARK.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPMSS.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TEE.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TPROXY.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_TRACE.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_addrtype.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_bpf.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cgroup.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cluster.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_comment.ko 2024-04-25T09:50:53.834 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connbytes.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlabel.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connlimit.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_connmark.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_conntrack.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_cpu.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dccp.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_devgroup.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_dscp.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ecn.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_esp.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hashlimit.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_helper.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_hl.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipcomp.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_iprange.ko 2024-04-25T09:50:53.835 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_ipvs.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_l2tp.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_length.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_limit.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mac.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_mark.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_multiport.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nat.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_nfacct.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_osf.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_owner.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_physdev.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_pkttype.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_policy.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_quota.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_rateest.ko 2024-04-25T09:50:53.836 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_realm.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_recent.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_sctp.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_set.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_socket.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_state.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_statistic.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_string.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_tcpmss.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_time.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netfilter/xt_u32.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/netlink/netlink_diag.ko 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh 2024-04-25T09:50:53.837 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/nsh/nsh.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/openvswitch.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-geneve.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-gre.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/openvswitch/vport-vxlan.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/packet/af_packet_diag.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_rdma.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/rds/rds_tcp.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_csum.ko 2024-04-25T09:50:53.838 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_gact.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_mirred.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_nat.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_pedit.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_police.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_simple.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_skbedit.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/act_vlan.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_basic.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_bpf.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_flow.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_fw.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_route.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/cls_u32.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_cmp.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_ipset.ko 2024-04-25T09:50:53.839 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_meta.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_nbyte.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_text.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/em_u32.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_choke.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_codel.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_drr.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_fq.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_gred.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hfsc.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_hhf.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_htb.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_ingress.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_mqprio_lib.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_multiq.ko 2024-04-25T09:50:53.840 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_netem.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_pie.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_plug.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_prio.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_qfq.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_red.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfb.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_sfq.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_tbf.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sched/sch_teql.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sctp/sctp_diag.ko 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc 2024-04-25T09:50:53.841 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/sunrpc.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/tipc/tipc_diag.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/unix/unix_diag.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-04-25T09:50:53.842 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock.ko 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_diag.ko 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/vmw_vsock/vsock_loopback.ko 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/net/xfrm/xfrm_ipcomp.ko 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/kernel/security/keys/trusted-keys/trusted.ko 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.alias.bin 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin 2024-04-25T09:50:53.843 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.alias.bin 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.bin 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.builtin.modinfo 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep.bin 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.devname 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.order 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.softdep 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/modules.symbols.bin 2024-04-25T09:50:53.844 INFO:teuthology.orchestra.run.smithi165.stdout:/lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:53.848 DEBUG:teuthology.task.kernel:get_image_version: 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.848 DEBUG:teuthology.orchestra.run.smithi165:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-04-25T09:50:53.871 INFO:teuthology.orchestra.run.smithi165.stdout:postinstall scriptlet (using /bin/sh): 2024-04-25T09:50:53.871 INFO:teuthology.orchestra.run.smithi165.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: /usr/bin/kernel-install add 6.9.0-rc3-g806ec36b2112 /lib/modules/6.9.0-rc3-g806ec36b2112/vmlinuz 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout:for file in vmlinuz System.map config; do 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: if ! cmp --silent "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112"; then 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: cp "/lib/modules/6.9.0-rc3-g806ec36b2112/${file}" "/boot/${file}-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: fi 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout:done 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout:if [ -d "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" ] && \ 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: ! diff -rq "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" >/dev/null 2>&1; then 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: rm -rf "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: cp -r "/lib/modules/6.9.0-rc3-g806ec36b2112/dtb" "/boot/dtb-6.9.0-rc3-g806ec36b2112" 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout:if [ ! -e "/lib/modules/6.9.0-rc3-g806ec36b2112/modules.dep" ]; then 2024-04-25T09:50:53.872 INFO:teuthology.orchestra.run.smithi165.stdout: /usr/sbin/depmod 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.873 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-04-25T09:50:53.873 INFO:teuthology.orchestra.run.smithi165.stdout:preuninstall scriptlet (using /bin/sh): 2024-04-25T09:50:53.873 INFO:teuthology.orchestra.run.smithi165.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-04-25T09:50:53.873 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-install remove 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.873 INFO:teuthology.orchestra.run.smithi165.stdout:fi 2024-04-25T09:50:53.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo rpm -qi grub2-tools 2024-04-25T09:50:53.947 INFO:teuthology.orchestra.run.smithi165.stdout:Name : grub2-tools 2024-04-25T09:50:53.947 INFO:teuthology.orchestra.run.smithi165.stdout:Epoch : 1 2024-04-25T09:50:53.947 INFO:teuthology.orchestra.run.smithi165.stdout:Version : 2.06 2024-04-25T09:50:53.947 INFO:teuthology.orchestra.run.smithi165.stdout:Release : 68.el9 2024-04-25T09:50:53.947 INFO:teuthology.orchestra.run.smithi165.stdout:Architecture: x86_64 2024-04-25T09:50:53.947 INFO:teuthology.orchestra.run.smithi165.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Group : Unspecified 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Size : 8264254 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:License : GPLv3+ 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Packager : builder@centos.org 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Vendor : CentOS 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:URL : http://www.gnu.org/software/grub/ 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Summary : Support tools for GRUB. 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout:Description : 2024-04-25T09:50:53.948 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T09:50:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-04-25T09:50:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:customizable bootloader with modular architecture. It supports a rich 2024-04-25T09:50:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:variety of kernel formats, file systems, computer architectures and 2024-04-25T09:50:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:hardware devices. 2024-04-25T09:50:53.949 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T09:50:53.949 INFO:teuthology.orchestra.run.smithi165.stdout:This subpackage provides tools for support of all platforms. 2024-04-25T09:50:53.950 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-04-25T09:50:53.950 INFO:teuthology.task.kernel:Updating grub on smithi165 to boot 6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:53.950 DEBUG:teuthology.orchestra.run.smithi165:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-04-25T09:50:54.556 INFO:teuthology.orchestra.run.smithi165.stderr:Generating grub configuration file ... 2024-04-25T09:50:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-04-25T09:50:55.909 INFO:teuthology.orchestra.run.smithi165.stderr:done 2024-04-25T09:50:55.911 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-04-25T09:50:55.926 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.iAuVFA3dWX 2024-04-25T09:50:55.926 DEBUG:teuthology.orchestra.run.smithi165:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.iAuVFA3dWX 2024-04-25T09:50:55.992 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.iAuVFA3dWX 2024-04-25T09:50:56.135 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.iAuVFA3dWX is 6KB 2024-04-25T09:50:56.187 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.iAuVFA3dWX 2024-04-25T09:50:56.201 DEBUG:teuthology.orchestra.run.smithi165:> sudo /bin/ls /boot/loader/entries || true 2024-04-25T09:50:56.263 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-04-25T09:50:56.263 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-04-25T09:50:56.263 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-04-25T09:50:56.263 INFO:teuthology.orchestra.run.smithi165.stdout:7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112.conf 2024-04-25T09:50:56.264 DEBUG:teuthology.orchestra.run.smithi165:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-6.9.0-rc3-g806ec36b2112 2024-04-25T09:50:56.423 DEBUG:teuthology.orchestra.run.smithi165:> sudo shutdown -r now 2024-04-25T09:51:15.002 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-04-25T09:51:15.003 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:51:15.005 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:26.437 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-04-25T09:51:26.437 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:51:26.438 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:51:49.535 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-25T09:51:58.543 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:51:58.545 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:01.054 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-25T09:52:10.062 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:52:10.063 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:16.927 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-25T09:52:28.449 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-25T09:52:28.928 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:52:28.929 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:31.998 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.172 2024-04-25T09:52:40.458 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:52:40.459 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:43.518 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.165 2024-04-25T09:52:47.002 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:52:47.003 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:47.443 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-25T09:52:47.980 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:52:47.980 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T09:52:47.980 DEBUG:teuthology.orchestra.run.smithi172:> uname -r 2024-04-25T09:52:48.017 INFO:teuthology.orchestra.run.smithi172.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:48.018 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:48.018 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:52:48.018 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T09:52:49.019 DEBUG:teuthology.parallel:result is None 2024-04-25T09:52:58.519 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:52:58.520 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:52:58.933 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-25T09:52:59.543 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:52:59.543 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.9.0-rc3-g806ec36b2112"... 2024-04-25T09:52:59.543 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-04-25T09:52:59.585 INFO:teuthology.orchestra.run.smithi165.stdout:6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:59.586 DEBUG:teuthology.task.kernel:current kernel version is 6.9.0-rc3-g806ec36b2112 vs 6.9.0-rc3-g806ec36b2112 2024-04-25T09:52:59.586 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-25T09:52:59.586 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-04-25T09:53:00.589 DEBUG:teuthology.parallel:result is None 2024-04-25T09:53:00.590 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-25T09:53:00.596 INFO:teuthology.task.internal:Creating test directory... 2024-04-25T09:53:00.597 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:53:00.607 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-25T09:53:00.685 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-25T09:53:00.692 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-25T09:53:00.698 INFO:teuthology.task.internal:Creating archive directory... 2024-04-25T09:53:00.698 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:53:00.710 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-25T09:53:00.801 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-25T09:53:00.808 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-25T09:53:00.809 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:53:00.818 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-25T09:53:00.959 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:53:00.960 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:53:01.012 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:53:01.025 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-25T09:53:01.031 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-25T09:53:01.039 INFO:teuthology.task.internal:Configuring sudo... 2024-04-25T09:53:01.039 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:53:01.052 DEBUG:teuthology.orchestra.run.smithi172:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-25T09:53:01.162 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-25T09:53:01.172 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-25T09:53:01.172 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:53:01.185 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-25T09:53:01.250 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:53:01.364 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:53:01.499 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:53:01.615 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:53:01.732 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T09:53:01.732 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:53:01.852 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:53:01.941 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-25T09:53:02.023 DEBUG:teuthology.orchestra.run.smithi172:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:53:02.097 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-25T09:53:02.179 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T09:53:02.179 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-25T09:53:02.261 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-04-25T09:53:02.272 DEBUG:teuthology.orchestra.run.smithi172:> sudo service rsyslog restart 2024-04-25T09:53:02.411 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:53:02.418 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T09:53:02.756 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-25T09:53:02.764 INFO:teuthology.task.internal:Starting timer... 2024-04-25T09:53:02.764 INFO:teuthology.run_tasks:Running task pcp... 2024-04-25T09:53:02.784 INFO:teuthology.run_tasks:Running task selinux... 2024-04-25T09:53:02.796 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-04-25T09:53:02.899 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-04-25T09:53:02.905 DEBUG:teuthology.orchestra.run.smithi172:> sudo service auditd rotate 2024-04-25T09:53:03.029 INFO:teuthology.orchestra.run.smithi172.stdout:Rotating logs: 2024-04-25T09:53:03.034 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-25T09:53:03.034 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-04-25T09:53:03.081 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-04-25T09:53:03.081 DEBUG:teuthology.orchestra.run.smithi172:> /usr/sbin/getenforce 2024-04-25T09:53:03.141 INFO:teuthology.orchestra.run.smithi172.stdout:Permissive 2024-04-25T09:53:03.141 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi165.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi172.front.sepia.ceph.com': 'permissive'} 2024-04-25T09:53:03.141 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:53:03.212 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:03.213 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T09:53:03.297 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T09:53:03.298 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-25T09:53:03.298 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:53:03.373 DEBUG:teuthology.orchestra.run.smithi172:> sudo /usr/sbin/setenforce permissive 2024-04-25T09:53:03.442 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-25T09:53:03.453 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-25T09:53:03.470 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-25T09:53:03.472 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi165.front.sepia.ceph.com,smithi172.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-25T09:58:37.980 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi165.front.sepia.ceph.com'), Remote(name='ubuntu@smithi172.front.sepia.ceph.com')] 2024-04-25T09:58:37.982 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:58:37.983 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:38.095 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-25T09:58:38.251 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-25T09:58:38.252 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:58:38.252 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi172.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-25T09:58:38.362 DEBUG:teuthology.orchestra.run.smithi172:> true 2024-04-25T09:58:38.502 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi172.front.sepia.ceph.com' 2024-04-25T09:58:38.503 INFO:teuthology.run_tasks:Running task clock... 2024-04-25T09:58:38.516 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-25T09:58:38.516 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:38.516 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:38.524 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-25T09:58:38.524 DEBUG:teuthology.orchestra.run.smithi172:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T09:58:38.611 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:58:38.615 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-25T09:58:38.653 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:58:38.654 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-25T09:58:38.727 INFO:teuthology.orchestra.run.smithi172.stderr:sudo: ntpd: command not found 2024-04-25T09:58:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-04-25T09:58:38.756 INFO:teuthology.orchestra.run.smithi172.stdout:506 Cannot talk to daemon 2024-04-25T09:58:38.764 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-04-25T09:58:38.798 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:58:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-25T09:58:38.839 INFO:teuthology.orchestra.run.smithi172.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:58:38.840 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-25T09:58:38.962 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:58:38.963 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-25T09:58:38.970 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:58:38.970 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-25T09:58:38.970 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.970 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.970 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.970 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.972 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T09:58:38.972 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-25T09:58:38.972 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.972 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.972 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.972 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T09:58:38.973 INFO:teuthology.run_tasks:Running task install... 2024-04-25T09:58:38.982 DEBUG:teuthology.task.install:project ceph 2024-04-25T09:58:38.983 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}} 2024-04-25T09:58:38.983 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'} 2024-04-25T09:58:38.983 INFO:teuthology.task.install:Using flavor: default 2024-04-25T09:58:38.989 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-25T09:58:38.989 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-25T09:58:38.990 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:58:38.990 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:38.993 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'tag': None, 'wait_for_package': False} 2024-04-25T09:58:38.993 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T09:58:39.138 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:58:39.139 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:58:39.142 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/ 2024-04-25T09:58:39.142 INFO:teuthology.task.install.rpm:Package version is 19.0.0-1628.g97ae6cc9 2024-04-25T09:58:39.261 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:58:39.262 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T09:58:39.262 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:58:39.267 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-neesingh-testing-240416/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-25T09:58:39.268 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T09:58:39.268 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-25T09:58:39.354 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:58:39.354 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:58:39.360 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-25T09:58:39.360 DEBUG:teuthology.orchestra.run.smithi172:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-25T09:58:39.485 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:58:39.487 DEBUG:teuthology.orchestra.run.smithi172:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-25T09:58:39.593 DEBUG:teuthology.orchestra.run.smithi172:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:58:39.597 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-25T09:58:39.669 INFO:teuthology.orchestra.run.smithi172.stdout:check_obsoletes = 1 2024-04-25T09:58:39.673 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum clean all 2024-04-25T09:58:39.676 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-04-25T09:58:39.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-04-25T09:58:39.989 INFO:teuthology.orchestra.run.smithi172.stdout:48 files removed 2024-04-25T09:58:40.000 INFO:teuthology.orchestra.run.smithi165.stdout:48 files removed 2024-04-25T09:58:40.028 DEBUG:teuthology.orchestra.run.smithi172:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:58:40.039 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-25T09:58:40.832 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 225 kB/s | 82 kB 00:00 2024-04-25T09:58:40.867 INFO:teuthology.orchestra.run.smithi172.stdout:ceph packages for x86_64 199 kB/s | 82 kB 00:00 2024-04-25T09:58:41.316 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 26 kB/s | 10 kB 00:00 2024-04-25T09:58:41.334 INFO:teuthology.orchestra.run.smithi172.stdout:ceph noarch packages 27 kB/s | 10 kB 00:00 2024-04-25T09:58:41.775 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-25T09:58:41.776 INFO:teuthology.orchestra.run.smithi172.stdout:ceph source packages 5.2 kB/s | 1.9 kB 00:00 2024-04-25T09:58:42.374 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.0 MB 00:00 2024-04-25T09:58:42.823 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 8.3 MB/s | 8.0 MB 00:00 2024-04-25T09:58:45.192 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T09:58:45.615 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-25T09:58:49.601 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-04-25T09:58:50.408 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 6.2 MB 00:00 2024-04-25T09:58:51.538 INFO:teuthology.orchestra.run.smithi172.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 16 kB 00:00 2024-04-25T09:58:52.200 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 33 kB/s | 16 kB 00:00 2024-04-25T09:58:55.576 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 6.6 MB/s | 21 MB 00:03 2024-04-25T09:59:01.559 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 25 kB/s | 1.7 kB 00:00 2024-04-25T09:59:03.960 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:03.962 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:03.981 INFO:teuthology.orchestra.run.smithi165.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:59:03.983 INFO:teuthology.orchestra.run.smithi165.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:59:03.984 INFO:teuthology.orchestra.run.smithi165.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:59:03.985 INFO:teuthology.orchestra.run.smithi165.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:59:03.988 INFO:teuthology.orchestra.run.smithi165.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:59:03.991 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:59:03.992 INFO:teuthology.orchestra.run.smithi165.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:59:04.062 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-25T09:59:04.092 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-25T09:59:04.092 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-25T09:59:04.092 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:59:04.093 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:59:04.094 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:59:04.095 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:04.096 INFO:teuthology.orchestra.run.smithi165.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:59:04.097 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:59:04.098 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:59:04.099 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:59:04.100 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:59:04.101 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:59:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:59:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:59:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:59:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:59:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:59:04.102 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:59:04.105 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:59:04.106 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:59:04.107 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:59:04.108 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:59:04.109 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout:Install 148 Packages 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 4 Packages 2024-04-25T09:59:04.110 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T09:59:04.111 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 215 M 2024-04-25T09:59:04.111 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-25T09:59:05.535 INFO:teuthology.orchestra.run.smithi165.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 23 kB/s | 6.6 kB 00:00 2024-04-25T09:59:05.769 INFO:teuthology.orchestra.run.smithi165.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 3.7 MB/s | 887 kB 00:00 2024-04-25T09:59:05.819 INFO:teuthology.orchestra.run.smithi165.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 2.9 MB/s | 145 kB 00:00 2024-04-25T09:59:06.110 INFO:teuthology.orchestra.run.smithi172.stdout:Extra Packages for Enterprise Linux 1.5 MB/s | 21 MB 00:14 2024-04-25T09:59:06.111 INFO:teuthology.orchestra.run.smithi165.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 7.3 MB/s | 2.1 MB 00:00 2024-04-25T09:59:06.336 INFO:teuthology.orchestra.run.smithi165.stdout:(5/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 4.9 MB/s | 5.3 MB 00:01 2024-04-25T09:59:06.411 INFO:teuthology.orchestra.run.smithi165.stdout:(6/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 5.0 MB/s | 1.5 MB 00:00 2024-04-25T09:59:07.028 INFO:teuthology.orchestra.run.smithi165.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 6.8 MB/s | 4.7 MB 00:00 2024-04-25T09:59:07.628 INFO:teuthology.orchestra.run.smithi165.stdout:(8/152): ceph-common-19.0.0-1628.g97ae6cc9.el9. 8.4 MB/s | 20 MB 00:02 2024-04-25T09:59:07.687 INFO:teuthology.orchestra.run.smithi165.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 441 kB/s | 25 kB 00:00 2024-04-25T09:59:08.112 INFO:teuthology.orchestra.run.smithi165.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 9.9 MB/s | 17 MB 00:01 2024-04-25T09:59:08.212 INFO:teuthology.orchestra.run.smithi165.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.2 MB/s | 220 kB 00:00 2024-04-25T09:59:08.279 INFO:teuthology.orchestra.run.smithi165.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 409 kB/s | 27 kB 00:00 2024-04-25T09:59:08.346 INFO:teuthology.orchestra.run.smithi165.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 474 kB/s | 31 kB 00:00 2024-04-25T09:59:08.522 INFO:teuthology.orchestra.run.smithi165.stdout:(14/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el 6.5 MB/s | 9.8 MB 00:01 2024-04-25T09:59:08.580 INFO:teuthology.orchestra.run.smithi165.stdout:(15/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 3.1 MB/s | 744 kB 00:00 2024-04-25T09:59:08.614 INFO:teuthology.orchestra.run.smithi165.stdout:(16/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 1.8 MB/s | 163 kB 00:00 2024-04-25T09:59:08.647 INFO:teuthology.orchestra.run.smithi165.stdout:(17/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 127 kB 00:00 2024-04-25T09:59:08.714 INFO:teuthology.orchestra.run.smithi165.stdout:(18/152): libradosstriper1-19.0.0-1628.g97ae6cc 4.9 MB/s | 501 kB 00:00 2024-04-25T09:59:08.806 INFO:teuthology.orchestra.run.smithi165.stdout:(19/152): python3-ceph-argparse-19.0.0-1628.g97 498 kB/s | 45 kB 00:00 2024-04-25T09:59:09.107 INFO:teuthology.orchestra.run.smithi165.stdout:(20/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 11 MB/s | 5.1 MB 00:00 2024-04-25T09:59:09.173 INFO:teuthology.orchestra.run.smithi165.stdout:(21/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.4 MB/s | 162 kB 00:00 2024-04-25T09:59:09.257 INFO:teuthology.orchestra.run.smithi165.stdout:(22/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.8 MB/s | 321 kB 00:00 2024-04-25T09:59:09.307 INFO:teuthology.orchestra.run.smithi165.stdout:(23/152): python3-ceph-common-19.0.0-1628.g97ae 263 kB/s | 131 kB 00:00 2024-04-25T09:59:09.340 INFO:teuthology.orchestra.run.smithi165.stdout:(24/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.5 MB/s | 298 kB 00:00 2024-04-25T09:59:09.382 INFO:teuthology.orchestra.run.smithi165.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.3 MB/s | 99 kB 00:00 2024-04-25T09:59:09.408 INFO:teuthology.orchestra.run.smithi165.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-25T09:59:09.483 INFO:teuthology.orchestra.run.smithi165.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.2 MB/s | 170 kB 00:00 2024-04-25T09:59:09.558 INFO:teuthology.orchestra.run.smithi165.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 423 kB/s | 31 kB 00:00 2024-04-25T09:59:09.642 INFO:teuthology.orchestra.run.smithi165.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 1.6 MB/s | 138 kB 00:00 2024-04-25T09:59:09.926 INFO:teuthology.orchestra.run.smithi165.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 6.3 MB/s | 1.8 MB 00:00 2024-04-25T09:59:10.076 INFO:teuthology.orchestra.run.smithi165.stdout:(31/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 4.5 MB/s | 3.1 MB 00:00 2024-04-25T09:59:10.234 INFO:teuthology.orchestra.run.smithi165.stdout:(32/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.5 MB/s | 248 kB 00:00 2024-04-25T09:59:10.334 INFO:teuthology.orchestra.run.smithi165.stdout:(33/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 499 kB/s | 49 kB 00:00 2024-04-25T09:59:10.435 INFO:teuthology.orchestra.run.smithi165.stdout:(34/152): ceph-prometheus-alerts-19.0.0-1628.g9 155 kB/s | 15 kB 00:00 2024-04-25T09:59:10.618 INFO:teuthology.orchestra.run.smithi165.stdout:(35/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 1.4 MB/s | 264 kB 00:00 2024-04-25T09:59:11.144 INFO:teuthology.orchestra.run.smithi165.stdout:(36/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 14 MB/s | 47 MB 00:03 2024-04-25T09:59:11.285 INFO:teuthology.orchestra.run.smithi165.stdout:(37/152): ceph-mgr-diskprediction-local-19.0.0- 5.4 MB/s | 7.4 MB 00:01 2024-04-25T09:59:11.310 INFO:teuthology.orchestra.run.smithi165.stdout:(38/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 110 kB/s | 18 kB 00:00 2024-04-25T09:59:11.361 INFO:teuthology.orchestra.run.smithi165.stdout:(39/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 1.0 MB/s | 748 kB 00:00 2024-04-25T09:59:11.620 INFO:teuthology.orchestra.run.smithi165.stdout:(40/152): fuse-2.9.9-15.el9.x86_64.rpm 241 kB/s | 80 kB 00:00 2024-04-25T09:59:11.653 INFO:teuthology.orchestra.run.smithi165.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 671 kB/s | 195 kB 00:00 2024-04-25T09:59:11.712 INFO:teuthology.orchestra.run.smithi165.stdout:(42/152): libgfortran-11.4.1-3.el9.x86_64.rpm 2.0 MB/s | 803 kB 00:00 2024-04-25T09:59:11.770 INFO:teuthology.orchestra.run.smithi165.stdout:(43/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 253 kB 00:00 2024-04-25T09:59:11.804 INFO:teuthology.orchestra.run.smithi165.stdout:(44/152): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-04-25T09:59:11.871 INFO:teuthology.orchestra.run.smithi165.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 5.7 MB/s | 1.2 MB 00:00 2024-04-25T09:59:11.913 INFO:teuthology.orchestra.run.smithi165.stdout:(46/152): python3-pycparser-2.20-6.el9.noarch.r 954 kB/s | 135 kB 00:00 2024-04-25T09:59:11.938 INFO:teuthology.orchestra.run.smithi165.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 940 kB/s | 125 kB 00:00 2024-04-25T09:59:11.988 INFO:teuthology.orchestra.run.smithi165.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.8 MB/s | 215 kB 00:00 2024-04-25T09:59:12.180 INFO:teuthology.orchestra.run.smithi165.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 561 kB/s | 107 kB 00:00 2024-04-25T09:59:12.239 INFO:teuthology.orchestra.run.smithi165.stdout:(50/152): elfutils-libelf-devel-0.190-2.el9.x86 375 kB/s | 21 kB 00:00 2024-04-25T09:59:12.259 INFO:teuthology.orchestra.run.smithi172.stdout:lab-extras 19 kB/s | 1.7 kB 00:00 2024-04-25T09:59:12.280 INFO:teuthology.orchestra.run.smithi165.stdout:(51/152): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 689 kB 00:00 2024-04-25T09:59:12.322 INFO:teuthology.orchestra.run.smithi165.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.4 MB/s | 943 kB 00:00 2024-04-25T09:59:12.373 INFO:teuthology.orchestra.run.smithi165.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 359 kB/s | 33 kB 00:00 2024-04-25T09:59:12.406 INFO:teuthology.orchestra.run.smithi165.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.8 MB/s | 313 kB 00:00 2024-04-25T09:59:12.431 INFO:teuthology.orchestra.run.smithi165.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 316 kB/s | 18 kB 00:00 2024-04-25T09:59:12.465 INFO:teuthology.orchestra.run.smithi165.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 193 kB/s | 11 kB 00:00 2024-04-25T09:59:12.499 INFO:teuthology.orchestra.run.smithi165.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-04-25T09:59:12.524 INFO:teuthology.orchestra.run.smithi165.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 795 kB/s | 45 kB 00:00 2024-04-25T09:59:12.641 INFO:teuthology.orchestra.run.smithi165.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-25T09:59:12.675 INFO:teuthology.orchestra.run.smithi165.stdout:(60/152): libtool-2.4.6-45.el9.x86_64.rpm 3.8 MB/s | 582 kB 00:00 2024-04-25T09:59:12.700 INFO:teuthology.orchestra.run.smithi165.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 441 kB/s | 25 kB 00:00 2024-04-25T09:59:12.734 INFO:teuthology.orchestra.run.smithi165.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 824 kB/s | 47 kB 00:00 2024-04-25T09:59:12.817 INFO:teuthology.orchestra.run.smithi165.stdout:(63/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.1 MB/s | 3.0 MB 00:00 2024-04-25T09:59:12.851 INFO:teuthology.orchestra.run.smithi165.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-04-25T09:59:12.876 INFO:teuthology.orchestra.run.smithi165.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 610 kB/s | 35 kB 00:00 2024-04-25T09:59:12.910 INFO:teuthology.orchestra.run.smithi165.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-04-25T09:59:13.196 INFO:teuthology.orchestra.run.smithi165.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-04-25T09:59:13.326 INFO:teuthology.orchestra.run.smithi165.stdout:(68/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-04-25T09:59:13.402 INFO:teuthology.orchestra.run.smithi165.stdout:(69/152): postgresql-contrib-13.14-1.el9.x86_64 4.4 MB/s | 911 kB 00:00 2024-04-25T09:59:13.435 INFO:teuthology.orchestra.run.smithi165.stdout:(70/152): postgresql-private-libs-13.14-1.el9.x 1.2 MB/s | 133 kB 00:00 2024-04-25T09:59:13.544 INFO:teuthology.orchestra.run.smithi165.stdout:(71/152): openblas-openmp-0.3.21-2.el9.x86_64.r 6.8 MB/s | 4.7 MB 00:00 2024-04-25T09:59:13.611 INFO:teuthology.orchestra.run.smithi165.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-25T09:59:13.646 INFO:teuthology.orchestra.run.smithi165.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 875 kB/s | 87 kB 00:00 2024-04-25T09:59:13.728 INFO:teuthology.orchestra.run.smithi165.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-25T09:59:13.795 INFO:teuthology.orchestra.run.smithi165.stdout:(75/152): python3-jinja2-2.11.3-5.el9.noarch.rp 3.7 MB/s | 248 kB 00:00 2024-04-25T09:59:13.864 INFO:teuthology.orchestra.run.smithi165.stdout:(76/152): python3-jsonpatch-1.21-16.el9.noarch. 377 kB/s | 26 kB 00:00 2024-04-25T09:59:13.964 INFO:teuthology.orchestra.run.smithi165.stdout:(77/152): postgresql-server-13.14-1.el9.x86_64. 10 MB/s | 5.8 MB 00:00 2024-04-25T09:59:13.989 INFO:teuthology.orchestra.run.smithi165.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 154 kB/s | 19 kB 00:00 2024-04-25T09:59:14.031 INFO:teuthology.orchestra.run.smithi165.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 2.6 MB/s | 172 kB 00:00 2024-04-25T09:59:14.056 INFO:teuthology.orchestra.run.smithi165.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 525 kB/s | 35 kB 00:00 2024-04-25T09:59:14.182 INFO:teuthology.orchestra.run.smithi165.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 10 MB/s | 6.0 MB 00:00 2024-04-25T09:59:14.215 INFO:teuthology.orchestra.run.smithi165.stdout:(82/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.6 MB/s | 250 kB 00:00 2024-04-25T09:59:14.257 INFO:teuthology.orchestra.run.smithi165.stdout:(83/152): python3-oauthlib-3.1.1-5.el9.noarch.r 3.0 MB/s | 222 kB 00:00 2024-04-25T09:59:14.282 INFO:teuthology.orchestra.run.smithi165.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 1.2 MB/s | 77 kB 00:00 2024-04-25T09:59:14.325 INFO:teuthology.orchestra.run.smithi165.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 642 kB/s | 42 kB 00:00 2024-04-25T09:59:14.358 INFO:teuthology.orchestra.run.smithi165.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 3.5 MB/s | 269 kB 00:00 2024-04-25T09:59:14.421 INFO:teuthology.orchestra.run.smithi165.stdout:(87/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-25T09:59:14.458 INFO:teuthology.orchestra.run.smithi165.stdout:(88/152): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-04-25T09:59:14.484 INFO:teuthology.orchestra.run.smithi165.stdout:(89/152): python3-pytz-2021.1-5.el9.noarch.rpm 823 kB/s | 51 kB 00:00 2024-04-25T09:59:14.517 INFO:teuthology.orchestra.run.smithi165.stdout:(90/152): python3-requests-oauthlib-1.3.0-12.el 930 kB/s | 54 kB 00:00 2024-04-25T09:59:14.593 INFO:teuthology.orchestra.run.smithi165.stdout:(91/152): python3-toml-0.10.2-6.el9.noarch.rpm 564 kB/s | 42 kB 00:00 2024-04-25T09:59:14.685 INFO:teuthology.orchestra.run.smithi172.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:14.687 INFO:teuthology.orchestra.run.smithi172.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-25T09:59:14.701 INFO:teuthology.orchestra.run.smithi165.stdout:(92/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 7.6 MB/s | 5.1 MB 00:00 2024-04-25T09:59:14.707 INFO:teuthology.orchestra.run.smithi172.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-25T09:59:14.708 INFO:teuthology.orchestra.run.smithi172.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-25T09:59:14.710 INFO:teuthology.orchestra.run.smithi172.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-25T09:59:14.711 INFO:teuthology.orchestra.run.smithi172.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-25T09:59:14.713 INFO:teuthology.orchestra.run.smithi172.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-25T09:59:14.717 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-25T09:59:14.718 INFO:teuthology.orchestra.run.smithi172.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-25T09:59:14.726 INFO:teuthology.orchestra.run.smithi165.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 333 kB/s | 44 kB 00:00 2024-04-25T09:59:14.777 INFO:teuthology.orchestra.run.smithi165.stdout:(94/152): qatlib-24.02.0-1.el9.x86_64.rpm 2.9 MB/s | 221 kB 00:00 2024-04-25T09:59:14.787 INFO:teuthology.orchestra.run.smithi172.stdout:Dependencies resolved. 2024-04-25T09:59:14.802 INFO:teuthology.orchestra.run.smithi165.stdout:(95/152): qatlib-service-24.02.0-1.el9.x86_64.r 482 kB/s | 36 kB 00:00 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout: Package Arch Version Repository Size 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout:Installing: 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout: ceph x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 6.6 k 2024-04-25T09:59:14.803 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.3 M 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 887 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 145 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 1.5 M 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 138 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 1.8 M 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 7.4 M 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 49 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 9.8 M 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 47 M 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 264 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 748 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 220 k 2024-04-25T09:59:14.804 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 27 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 18 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 31 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 744 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 127 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 162 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 321 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 298 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 99 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 85 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 170 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrading: 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: librados2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.4 M 2024-04-25T09:59:14.806 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 3.1 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout:Installing dependencies: 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 20 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 31 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 2.1 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 248 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 4.7 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 17 M 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts noarch 2:19.0.0-1628.g97ae6cc9.el9 ceph-noarch 15 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 25 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-25T09:59:14.807 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 163 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 501 k 2024-04-25T09:59:14.808 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2 x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 5.1 M 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-25T09:59:14.809 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 45 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common x86_64 2:19.0.0-1628.g97ae6cc9.el9 ceph 131 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-25T09:59:14.810 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-25T09:59:14.811 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-25T09:59:14.812 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-25T09:59:14.813 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout:Installing weak dependencies: 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T09:59:14.814 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction Summary 2024-04-25T09:59:14.815 INFO:teuthology.orchestra.run.smithi172.stdout:======================================================================================= 2024-04-25T09:59:14.815 INFO:teuthology.orchestra.run.smithi172.stdout:Install 148 Packages 2024-04-25T09:59:14.815 INFO:teuthology.orchestra.run.smithi172.stdout:Upgrade 4 Packages 2024-04-25T09:59:14.815 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T09:59:14.815 INFO:teuthology.orchestra.run.smithi172.stdout:Total download size: 215 M 2024-04-25T09:59:14.815 INFO:teuthology.orchestra.run.smithi172.stdout:Downloading Packages: 2024-04-25T09:59:14.836 INFO:teuthology.orchestra.run.smithi165.stdout:(96/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 825 kB/s | 47 kB 00:00 2024-04-25T09:59:14.878 INFO:teuthology.orchestra.run.smithi165.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 4.0 MB/s | 305 kB 00:00 2024-04-25T09:59:14.902 INFO:teuthology.orchestra.run.smithi165.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 880 kB/s | 58 kB 00:00 2024-04-25T09:59:14.936 INFO:teuthology.orchestra.run.smithi165.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-25T09:59:14.961 INFO:teuthology.orchestra.run.smithi165.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-25T09:59:14.995 INFO:teuthology.orchestra.run.smithi165.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 787 kB/s | 45 kB 00:00 2024-04-25T09:59:15.070 INFO:teuthology.orchestra.run.smithi165.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 207 kB/s | 22 kB 00:00 2024-04-25T09:59:15.521 INFO:teuthology.orchestra.run.smithi165.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:01 2024-04-25T09:59:15.571 INFO:teuthology.orchestra.run.smithi165.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.5 MB/s | 863 kB 00:00 2024-04-25T09:59:15.961 INFO:teuthology.orchestra.run.smithi172.stdout:(1/152): ceph-19.0.0-1628.g97ae6cc9.el9.x86_64. 16 kB/s | 6.6 kB 00:00 2024-04-25T09:59:15.971 INFO:teuthology.orchestra.run.smithi165.stdout:(105/152): grpc-data-1.46.7-10.el9.noarch.rpm 49 kB/s | 19 kB 00:00 2024-04-25T09:59:16.129 INFO:teuthology.orchestra.run.smithi165.stdout:(106/152): abseil-cpp-20211102.0-3.el9.x86_64.r 518 kB/s | 548 kB 00:01 2024-04-25T09:59:16.180 INFO:teuthology.orchestra.run.smithi165.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 472 kB/s | 309 kB 00:00 2024-04-25T09:59:16.231 INFO:teuthology.orchestra.run.smithi165.stdout:(108/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 259 kB/s | 26 kB 00:00 2024-04-25T09:59:16.289 INFO:teuthology.orchestra.run.smithi165.stdout:(109/152): liboath-2.6.7-2.el9.x86_64.rpm 447 kB/s | 49 kB 00:00 2024-04-25T09:59:16.340 INFO:teuthology.orchestra.run.smithi165.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 625 kB/s | 67 kB 00:00 2024-04-25T09:59:16.414 INFO:teuthology.orchestra.run.smithi165.stdout:(111/152): luarocks-3.9.2-1.el9.noarch.rpm 1.2 MB/s | 151 kB 00:00 2024-04-25T09:59:16.461 INFO:teuthology.orchestra.run.smithi172.stdout:(2/152): ceph-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.7 MB/s | 887 kB 00:00 2024-04-25T09:59:16.562 INFO:teuthology.orchestra.run.smithi172.stdout:(3/152): ceph-immutable-object-cache-19.0.0-162 1.4 MB/s | 145 kB 00:00 2024-04-25T09:59:16.565 INFO:teuthology.orchestra.run.smithi165.stdout:(112/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.6 MB/s | 838 kB 00:00 2024-04-25T09:59:16.649 INFO:teuthology.orchestra.run.smithi165.stdout:(113/152): python3-asyncssh-2.13.2-5.el9.noarch 2.3 MB/s | 548 kB 00:00 2024-04-25T09:59:16.690 INFO:teuthology.orchestra.run.smithi165.stdout:(114/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 350 kB/s | 43 kB 00:00 2024-04-25T09:59:16.799 INFO:teuthology.orchestra.run.smithi165.stdout:(115/152): libarrow-9.0.0-10.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2024-04-25T09:59:16.824 INFO:teuthology.orchestra.run.smithi165.stdout:(116/152): python3-certifi-2021.10.8-2.el9.noar 113 kB/s | 15 kB 00:00 2024-04-25T09:59:16.849 INFO:teuthology.orchestra.run.smithi165.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 161 kB/s | 32 kB 00:00 2024-04-25T09:59:16.925 INFO:teuthology.orchestra.run.smithi165.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 1.4 MB/s | 172 kB 00:00 2024-04-25T09:59:16.967 INFO:teuthology.orchestra.run.smithi165.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 2.5 MB/s | 358 kB 00:00 2024-04-25T09:59:17.000 INFO:teuthology.orchestra.run.smithi165.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.0 MB/s | 303 kB 00:00 2024-04-25T09:59:17.026 INFO:teuthology.orchestra.run.smithi165.stdout:(121/152): python3-colorama-0.4.4-12.el9.noarch 353 kB/s | 35 kB 00:00 2024-04-25T09:59:17.076 INFO:teuthology.orchestra.run.smithi165.stdout:(122/152): python3-google-auth-2.29.0-1.el9.noa 1.9 MB/s | 206 kB 00:00 2024-04-25T09:59:17.135 INFO:teuthology.orchestra.run.smithi165.stdout:(123/152): python3-grpcio-tools-1.46.7-10.el9.x 1.3 MB/s | 144 kB 00:00 2024-04-25T09:59:17.176 INFO:teuthology.orchestra.run.smithi165.stdout:(124/152): python3-jaraco-8.2.1-3.el9.noarch.rp 108 kB/s | 11 kB 00:00 2024-04-25T09:59:17.203 INFO:teuthology.orchestra.run.smithi172.stdout:(4/152): ceph-mds-19.0.0-1628.g97ae6cc9.el9.x86 3.3 MB/s | 2.1 MB 00:00 2024-04-25T09:59:17.235 INFO:teuthology.orchestra.run.smithi165.stdout:(125/152): python3-jaraco-classes-3.2.1-5.el9.n 178 kB/s | 18 kB 00:00 2024-04-25T09:59:17.310 INFO:teuthology.orchestra.run.smithi165.stdout:(126/152): python3-grpcio-1.46.7-10.el9.x86_64. 6.6 MB/s | 2.0 MB 00:00 2024-04-25T09:59:17.335 INFO:teuthology.orchestra.run.smithi165.stdout:(127/152): python3-jaraco-collections-3.0.0-8.e 146 kB/s | 23 kB 00:00 2024-04-25T09:59:17.360 INFO:teuthology.orchestra.run.smithi165.stdout:(128/152): python3-jaraco-functools-3.5.0-2.el9 156 kB/s | 19 kB 00:00 2024-04-25T09:59:17.411 INFO:teuthology.orchestra.run.smithi165.stdout:(129/152): python3-jaraco-text-3.2.0-6.el9.noar 198 kB/s | 20 kB 00:00 2024-04-25T09:59:17.461 INFO:teuthology.orchestra.run.smithi165.stdout:(130/152): python3-logutils-0.3.5-21.el9.noarch 467 kB/s | 46 kB 00:00 2024-04-25T09:59:17.543 INFO:teuthology.orchestra.run.smithi165.stdout:(131/152): python3-kubernetes-26.1.0-2.el9.noar 5.6 MB/s | 1.0 MB 00:00 2024-04-25T09:59:17.578 INFO:teuthology.orchestra.run.smithi165.stdout:(132/152): python3-natsort-7.1.1-5.el9.noarch.r 497 kB/s | 58 kB 00:00 2024-04-25T09:59:17.603 INFO:teuthology.orchestra.run.smithi165.stdout:(133/152): python3-more-itertools-8.12.0-2.el9. 411 kB/s | 79 kB 00:00 2024-04-25T09:59:17.654 INFO:teuthology.orchestra.run.smithi165.stdout:(134/152): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-04-25T09:59:17.679 INFO:teuthology.orchestra.run.smithi165.stdout:(135/152): python3-portend-3.1.0-2.el9.noarch.r 165 kB/s | 16 kB 00:00 2024-04-25T09:59:17.679 INFO:teuthology.orchestra.run.smithi172.stdout:(5/152): ceph-mgr-19.0.0-1628.g97ae6cc9.el9.x86 3.1 MB/s | 1.5 MB 00:00 2024-04-25T09:59:17.704 INFO:teuthology.orchestra.run.smithi165.stdout:(136/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 901 kB/s | 90 kB 00:00 2024-04-25T09:59:17.754 INFO:teuthology.orchestra.run.smithi165.stdout:(137/152): python3-pyperclip-1.8.2-2.el9.noarch 283 kB/s | 28 kB 00:00 2024-04-25T09:59:17.780 INFO:teuthology.orchestra.run.smithi165.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 308 kB/s | 31 kB 00:00 2024-04-25T09:59:17.813 INFO:teuthology.orchestra.run.smithi165.stdout:(139/152): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-04-25T09:59:17.855 INFO:teuthology.orchestra.run.smithi165.stdout:(140/152): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-04-25T09:59:17.881 INFO:teuthology.orchestra.run.smithi165.stdout:(141/152): python3-tempora-5.0.0-2.el9.noarch.r 359 kB/s | 36 kB 00:00 2024-04-25T09:59:17.914 INFO:teuthology.orchestra.run.smithi165.stdout:(142/152): python3-typing-extensions-4.4.0-2.el 509 kB/s | 51 kB 00:00 2024-04-25T09:59:17.921 INFO:teuthology.orchestra.run.smithi172.stdout:(6/152): ceph-base-19.0.0-1628.g97ae6cc9.el9.x8 2.3 MB/s | 5.3 MB 00:02 2024-04-25T09:59:17.965 INFO:teuthology.orchestra.run.smithi165.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-25T09:59:17.990 INFO:teuthology.orchestra.run.smithi165.stdout:(144/152): python3-websocket-client-1.2.3-2.el9 827 kB/s | 90 kB 00:00 2024-04-25T09:59:18.031 INFO:teuthology.orchestra.run.smithi165.stdout:(145/152): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-04-25T09:59:18.065 INFO:teuthology.orchestra.run.smithi165.stdout:(146/152): python3-zc-lockfile-2.0-10.el9.noarc 200 kB/s | 20 kB 00:00 2024-04-25T09:59:18.098 INFO:teuthology.orchestra.run.smithi165.stdout:(147/152): re2-20211101-3.el9.x86_64.rpm 1.7 MB/s | 192 kB 00:00 2024-04-25T09:59:18.182 INFO:teuthology.orchestra.run.smithi165.stdout:(148/152): thrift-0.15.0-2.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-04-25T09:59:18.399 INFO:teuthology.orchestra.run.smithi165.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.3 MB/s | 1.3 MB 00:00 2024-04-25T09:59:18.599 INFO:teuthology.orchestra.run.smithi165.stdout:(150/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 12 MB/s | 2.4 MB 00:00 2024-04-25T09:59:18.741 INFO:teuthology.orchestra.run.smithi165.stdout:(151/152): librados2-19.0.0-1628.g97ae6cc9.el9. 5.0 MB/s | 3.4 MB 00:00 2024-04-25T09:59:18.787 INFO:teuthology.orchestra.run.smithi172.stdout:(7/152): ceph-mon-19.0.0-1628.g97ae6cc9.el9.x86 4.2 MB/s | 4.7 MB 00:01 2024-04-25T09:59:19.699 INFO:teuthology.orchestra.run.smithi165.stdout:(152/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 2.0 MB/s | 3.1 MB 00:01 2024-04-25T09:59:19.712 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:59:19.712 INFO:teuthology.orchestra.run.smithi165.stdout:Total 14 MB/s | 215 MB 00:15 2024-04-25T09:59:20.287 INFO:teuthology.orchestra.run.smithi172.stdout:(8/152): ceph-radosgw-19.0.0-1628.g97ae6cc9.el9 6.5 MB/s | 9.8 MB 00:01 2024-04-25T09:59:20.338 INFO:teuthology.orchestra.run.smithi172.stdout:(9/152): ceph-selinux-19.0.0-1628.g97ae6cc9.el9 515 kB/s | 25 kB 00:00 2024-04-25T09:59:22.558 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-25T09:59:22.768 INFO:teuthology.orchestra.run.smithi172.stdout:(10/152): ceph-osd-19.0.0-1628.g97ae6cc9.el9.x8 3.5 MB/s | 17 MB 00:04 2024-04-25T09:59:22.803 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-25T09:59:22.803 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-25T09:59:22.846 INFO:teuthology.orchestra.run.smithi172.stdout:(11/152): cephfs-mirror-19.0.0-1628.g97ae6cc9.e 2.8 MB/s | 220 kB 00:00 2024-04-25T09:59:22.913 INFO:teuthology.orchestra.run.smithi172.stdout:(12/152): cephfs-shell-19.0.0-1628.g97ae6cc9.el 412 kB/s | 27 kB 00:00 2024-04-25T09:59:22.980 INFO:teuthology.orchestra.run.smithi172.stdout:(13/152): libcephfs-devel-19.0.0-1628.g97ae6cc9 474 kB/s | 31 kB 00:00 2024-04-25T09:59:23.139 INFO:teuthology.orchestra.run.smithi172.stdout:(14/152): libcephfs2-19.0.0-1628.g97ae6cc9.el9. 4.6 MB/s | 744 kB 00:00 2024-04-25T09:59:23.198 INFO:teuthology.orchestra.run.smithi172.stdout:(15/152): libcephsqlite-19.0.0-1628.g97ae6cc9.e 2.8 MB/s | 163 kB 00:00 2024-04-25T09:59:23.265 INFO:teuthology.orchestra.run.smithi172.stdout:(16/152): librados-devel-19.0.0-1628.g97ae6cc9. 1.9 MB/s | 127 kB 00:00 2024-04-25T09:59:23.390 INFO:teuthology.orchestra.run.smithi172.stdout:(17/152): libradosstriper1-19.0.0-1628.g97ae6cc 3.9 MB/s | 501 kB 00:00 2024-04-25T09:59:24.657 INFO:teuthology.orchestra.run.smithi172.stdout:(18/152): ceph-test-19.0.0-1628.g97ae6cc9.el9.x 11 MB/s | 47 MB 00:04 2024-04-25T09:59:24.766 INFO:teuthology.orchestra.run.smithi172.stdout:(19/152): librgw2-19.0.0-1628.g97ae6cc9.el9.x86 3.7 MB/s | 5.1 MB 00:01 2024-04-25T09:59:24.816 INFO:teuthology.orchestra.run.smithi172.stdout:(20/152): python3-ceph-argparse-19.0.0-1628.g97 286 kB/s | 45 kB 00:00 2024-04-25T09:59:24.849 INFO:teuthology.orchestra.run.smithi172.stdout:(21/152): python3-ceph-common-19.0.0-1628.g97ae 1.6 MB/s | 131 kB 00:00 2024-04-25T09:59:24.891 INFO:teuthology.orchestra.run.smithi172.stdout:(22/152): python3-cephfs-19.0.0-1628.g97ae6cc9. 2.1 MB/s | 162 kB 00:00 2024-04-25T09:59:24.950 INFO:teuthology.orchestra.run.smithi172.stdout:(23/152): python3-rados-19.0.0-1628.g97ae6cc9.e 3.2 MB/s | 321 kB 00:00 2024-04-25T09:59:24.975 INFO:teuthology.orchestra.run.smithi172.stdout:(24/152): python3-rbd-19.0.0-1628.g97ae6cc9.el9 3.5 MB/s | 298 kB 00:00 2024-04-25T09:59:25.009 INFO:teuthology.orchestra.run.smithi172.stdout:(25/152): python3-rgw-19.0.0-1628.g97ae6cc9.el9 1.7 MB/s | 99 kB 00:00 2024-04-25T09:59:25.034 INFO:teuthology.orchestra.run.smithi172.stdout:(26/152): rbd-fuse-19.0.0-1628.g97ae6cc9.el9.x8 1.5 MB/s | 85 kB 00:00 2024-04-25T09:59:25.093 INFO:teuthology.orchestra.run.smithi172.stdout:(27/152): rbd-nbd-19.0.0-1628.g97ae6cc9.el9.x86 2.9 MB/s | 170 kB 00:00 2024-04-25T09:59:25.160 INFO:teuthology.orchestra.run.smithi172.stdout:(28/152): ceph-grafana-dashboards-19.0.0-1628.g 475 kB/s | 31 kB 00:00 2024-04-25T09:59:25.219 INFO:teuthology.orchestra.run.smithi172.stdout:(29/152): ceph-mgr-cephadm-19.0.0-1628.g97ae6cc 2.4 MB/s | 138 kB 00:00 2024-04-25T09:59:25.402 INFO:teuthology.orchestra.run.smithi172.stdout:(30/152): ceph-mgr-dashboard-19.0.0-1628.g97ae6 9.8 MB/s | 1.8 MB 00:00 2024-04-25T09:59:25.488 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-25T09:59:25.488 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-25T09:59:25.661 INFO:teuthology.orchestra.run.smithi172.stdout:(31/152): ceph-common-19.0.0-1628.g97ae6cc9.el9 2.0 MB/s | 20 MB 00:10 2024-04-25T09:59:25.839 INFO:teuthology.orchestra.run.smithi172.stdout:(32/152): rbd-mirror-19.0.0-1628.g97ae6cc9.el9. 3.7 MB/s | 3.1 MB 00:00 2024-04-25T09:59:25.877 INFO:teuthology.orchestra.run.smithi172.stdout:(33/152): ceph-mgr-modules-core-19.0.0-1628.g97 1.1 MB/s | 248 kB 00:00 2024-04-25T09:59:25.919 INFO:teuthology.orchestra.run.smithi172.stdout:(34/152): ceph-mgr-rook-19.0.0-1628.g97ae6cc9.e 622 kB/s | 49 kB 00:00 2024-04-25T09:59:25.962 INFO:teuthology.orchestra.run.smithi172.stdout:(35/152): ceph-prometheus-alerts-19.0.0-1628.g9 185 kB/s | 15 kB 00:00 2024-04-25T09:59:26.020 INFO:teuthology.orchestra.run.smithi172.stdout:(36/152): ceph-volume-19.0.0-1628.g97ae6cc9.el9 2.6 MB/s | 264 kB 00:00 2024-04-25T09:59:26.095 INFO:teuthology.orchestra.run.smithi172.stdout:(37/152): cephfs-top-19.0.0-1628.g97ae6cc9.el9. 246 kB/s | 18 kB 00:00 2024-04-25T09:59:26.196 INFO:teuthology.orchestra.run.smithi172.stdout:(38/152): cephadm-19.0.0-1628.g97ae6cc9.el9.noa 3.1 MB/s | 748 kB 00:00 2024-04-25T09:59:26.229 INFO:teuthology.orchestra.run.smithi172.stdout:(39/152): fuse-2.9.9-15.el9.x86_64.rpm 607 kB/s | 80 kB 00:00 2024-04-25T09:59:26.430 INFO:teuthology.orchestra.run.smithi172.stdout:(40/152): ceph-mgr-diskprediction-local-19.0.0- 7.2 MB/s | 7.4 MB 00:01 2024-04-25T09:59:26.488 INFO:teuthology.orchestra.run.smithi172.stdout:(41/152): libquadmath-11.4.1-3.el9.x86_64.rpm 756 kB/s | 195 kB 00:00 2024-04-25T09:59:26.572 INFO:teuthology.orchestra.run.smithi172.stdout:(42/152): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-04-25T09:59:26.647 INFO:teuthology.orchestra.run.smithi172.stdout:(43/152): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-04-25T09:59:26.748 INFO:teuthology.orchestra.run.smithi172.stdout:(44/152): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-04-25T09:59:26.798 INFO:teuthology.orchestra.run.smithi172.stdout:(45/152): python3-cryptography-36.0.1-4.el9.x86 4.0 MB/s | 1.2 MB 00:00 2024-04-25T09:59:26.882 INFO:teuthology.orchestra.run.smithi172.stdout:(46/152): libgfortran-11.4.1-3.el9.x86_64.rpm 1.1 MB/s | 803 kB 00:00 2024-04-25T09:59:26.932 INFO:teuthology.orchestra.run.smithi172.stdout:(47/152): python3-requests-2.25.1-8.el9.noarch. 682 kB/s | 125 kB 00:00 2024-04-25T09:59:26.957 INFO:teuthology.orchestra.run.smithi172.stdout:(48/152): python3-urllib3-1.26.5-5.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-04-25T09:59:27.149 INFO:teuthology.orchestra.run.smithi172.stdout:(49/152): boost-program-options-1.75.0-8.el9.x8 558 kB/s | 107 kB 00:00 2024-04-25T09:59:27.191 INFO:teuthology.orchestra.run.smithi172.stdout:(50/152): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-25T09:59:27.233 INFO:teuthology.orchestra.run.smithi172.stdout:(51/152): elfutils-libelf-devel-0.190-2.el9.x86 259 kB/s | 21 kB 00:00 2024-04-25T09:59:27.300 INFO:teuthology.orchestra.run.smithi172.stdout:(52/152): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 943 kB 00:00 2024-04-25T09:59:27.350 INFO:teuthology.orchestra.run.smithi172.stdout:(53/152): flexiblas-3.0.4-8.el9.x86_64.rpm 282 kB/s | 33 kB 00:00 2024-04-25T09:59:27.384 INFO:teuthology.orchestra.run.smithi172.stdout:(54/152): flex-2.6.4-9.el9.x86_64.rpm 1.6 MB/s | 313 kB 00:00 2024-04-25T09:59:27.442 INFO:teuthology.orchestra.run.smithi172.stdout:(55/152): flexiblas-openblas-openmp-3.0.4-8.el9 199 kB/s | 18 kB 00:00 2024-04-25T09:59:27.516 INFO:teuthology.orchestra.run.smithi172.stdout:(56/152): libaio-devel-0.3.111-13.el9.x86_64.rp 85 kB/s | 11 kB 00:00 2024-04-25T09:59:27.551 INFO:teuthology.orchestra.run.smithi172.stdout:(57/152): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.5 MB/s | 160 kB 00:00 2024-04-25T09:59:27.602 INFO:teuthology.orchestra.run.smithi172.stdout:(58/152): librabbitmq-0.11.0-7.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-04-25T09:59:27.710 INFO:teuthology.orchestra.run.smithi172.stdout:(59/152): librdkafka-1.6.1-102.el9.x86_64.rpm 4.1 MB/s | 662 kB 00:00 2024-04-25T09:59:27.785 INFO:teuthology.orchestra.run.smithi172.stdout:(60/152): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.2 MB/s | 3.0 MB 00:00 2024-04-25T09:59:27.836 INFO:teuthology.orchestra.run.smithi172.stdout:(61/152): libuuid-devel-2.37.4-18.el9.x86_64.rp 204 kB/s | 25 kB 00:00 2024-04-25T09:59:27.886 INFO:teuthology.orchestra.run.smithi172.stdout:(62/152): libzstd-devel-1.5.1-2.el9.x86_64.rpm 475 kB/s | 47 kB 00:00 2024-04-25T09:59:27.920 INFO:teuthology.orchestra.run.smithi172.stdout:(63/152): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-04-25T09:59:27.953 INFO:teuthology.orchestra.run.smithi172.stdout:(64/152): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.5 MB/s | 292 kB 00:00 2024-04-25T09:59:28.004 INFO:teuthology.orchestra.run.smithi172.stdout:(65/152): openblas-0.3.21-2.el9.x86_64.rpm 427 kB/s | 35 kB 00:00 2024-04-25T09:59:28.069 INFO:teuthology.orchestra.run.smithi172.stdout:(66/152): lua-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 188 kB 00:00 2024-04-25T09:59:28.254 INFO:teuthology.orchestra.run.smithi172.stdout:(67/152): postgresql-13.14-1.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-04-25T09:59:28.371 INFO:teuthology.orchestra.run.smithi172.stdout:(68/152): postgresql-contrib-13.14-1.el9.x86_64 7.7 MB/s | 911 kB 00:00 2024-04-25T09:59:28.438 INFO:teuthology.orchestra.run.smithi172.stdout:(69/152): postgresql-private-libs-13.14-1.el9.x 2.0 MB/s | 133 kB 00:00 2024-04-25T09:59:28.572 INFO:teuthology.orchestra.run.smithi172.stdout:(70/152): openblas-openmp-0.3.21-2.el9.x86_64.r 7.7 MB/s | 4.7 MB 00:00 2024-04-25T09:59:28.689 INFO:teuthology.orchestra.run.smithi172.stdout:(71/152): openssl-devel-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 4.4 MB 00:00 2024-04-25T09:59:28.748 INFO:teuthology.orchestra.run.smithi172.stdout:(72/152): protobuf-3.14.0-13.el9.x86_64.rpm 5.8 MB/s | 1.0 MB 00:00 2024-04-25T09:59:28.773 INFO:teuthology.orchestra.run.smithi172.stdout:(73/152): python3-attrs-20.3.0-7.el9.noarch.rpm 1.0 MB/s | 87 kB 00:00 2024-04-25T09:59:28.840 INFO:teuthology.orchestra.run.smithi172.stdout:(74/152): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-25T09:59:28.956 INFO:teuthology.orchestra.run.smithi172.stdout:(75/152): postgresql-server-13.14-1.el9.x86_64. 11 MB/s | 5.8 MB 00:00 2024-04-25T09:59:28.990 INFO:teuthology.orchestra.run.smithi172.stdout:(76/152): python3-jinja2-2.11.3-5.el9.noarch.rp 1.6 MB/s | 248 kB 00:00 2024-04-25T09:59:29.049 INFO:teuthology.orchestra.run.smithi172.stdout:(77/152): python3-jsonpatch-1.21-16.el9.noarch. 281 kB/s | 26 kB 00:00 2024-04-25T09:59:29.091 INFO:teuthology.orchestra.run.smithi172.stdout:(78/152): python3-jsonpointer-2.0-4.el9.noarch. 192 kB/s | 19 kB 00:00 2024-04-25T09:59:29.141 INFO:teuthology.orchestra.run.smithi172.stdout:(79/152): python3-mako-1.1.4-6.el9.noarch.rpm 1.8 MB/s | 172 kB 00:00 2024-04-25T09:59:29.224 INFO:teuthology.orchestra.run.smithi172.stdout:(80/152): python3-markupsafe-1.1.1-12.el9.x86_6 262 kB/s | 35 kB 00:00 2024-04-25T09:59:29.433 INFO:teuthology.orchestra.run.smithi172.stdout:(81/152): python3-babel-2.9.1-2.el9.noarch.rpm 8.7 MB/s | 6.0 MB 00:00 2024-04-25T09:59:29.517 INFO:teuthology.orchestra.run.smithi172.stdout:(82/152): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-04-25T09:59:29.659 INFO:teuthology.orchestra.run.smithi172.stdout:(83/152): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.9 MB/s | 5.1 MB 00:00 2024-04-25T09:59:29.684 INFO:teuthology.orchestra.run.smithi172.stdout:(84/152): python3-packaging-20.9-5.el9.noarch.r 464 kB/s | 77 kB 00:00 2024-04-25T09:59:29.734 INFO:teuthology.orchestra.run.smithi172.stdout:(85/152): python3-prettytable-0.7.2-27.el9.noar 569 kB/s | 42 kB 00:00 2024-04-25T09:59:29.801 INFO:teuthology.orchestra.run.smithi172.stdout:(86/152): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-04-25T09:59:29.826 INFO:teuthology.orchestra.run.smithi172.stdout:(87/152): python3-numpy-f2py-1.20.1-5.el9.x86_6 417 kB/s | 250 kB 00:00 2024-04-25T09:59:29.852 INFO:teuthology.orchestra.run.smithi172.stdout:(88/152): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-04-25T09:59:29.885 INFO:teuthology.orchestra.run.smithi172.stdout:(89/152): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-04-25T09:59:29.927 INFO:teuthology.orchestra.run.smithi172.stdout:(90/152): python3-pytz-2021.1-5.el9.noarch.rpm 509 kB/s | 51 kB 00:00 2024-04-25T09:59:29.969 INFO:teuthology.orchestra.run.smithi172.stdout:(91/152): python3-requests-oauthlib-1.3.0-12.el 461 kB/s | 54 kB 00:00 2024-04-25T09:59:30.011 INFO:teuthology.orchestra.run.smithi172.stdout:(92/152): python3-toml-0.10.2-6.el9.noarch.rpm 503 kB/s | 42 kB 00:00 2024-04-25T09:59:30.054 INFO:teuthology.orchestra.run.smithi172.stdout:(93/152): python3-wcwidth-0.2.5-8.el9.noarch.rp 533 kB/s | 44 kB 00:00 2024-04-25T09:59:30.129 INFO:teuthology.orchestra.run.smithi172.stdout:(94/152): qatlib-service-24.02.0-1.el9.x86_64.r 483 kB/s | 36 kB 00:00 2024-04-25T09:59:30.212 INFO:teuthology.orchestra.run.smithi172.stdout:(95/152): qatzip-libs-1.2.0-1.el9.x86_64.rpm 574 kB/s | 47 kB 00:00 2024-04-25T09:59:30.246 INFO:teuthology.orchestra.run.smithi172.stdout:(96/152): qatlib-24.02.0-1.el9.x86_64.rpm 945 kB/s | 221 kB 00:00 2024-04-25T09:59:30.296 INFO:teuthology.orchestra.run.smithi172.stdout:(97/152): socat-1.7.4.1-5.el9.x86_64.rpm 3.6 MB/s | 305 kB 00:00 2024-04-25T09:59:30.338 INFO:teuthology.orchestra.run.smithi172.stdout:(98/152): uuid-1.6.2-55.el9.x86_64.rpm 634 kB/s | 58 kB 00:00 2024-04-25T09:59:30.380 INFO:teuthology.orchestra.run.smithi172.stdout:(99/152): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-04-25T09:59:30.422 INFO:teuthology.orchestra.run.smithi172.stdout:(100/152): xmlstarlet-1.6.1-20.el9.x86_64.rpm 766 kB/s | 64 kB 00:00 2024-04-25T09:59:30.474 INFO:teuthology.orchestra.run.smithi172.stdout:(101/152): zlib-devel-1.2.11-41.el9.x86_64.rpm 481 kB/s | 45 kB 00:00 2024-04-25T09:59:30.591 INFO:teuthology.orchestra.run.smithi172.stdout:(102/152): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-04-25T09:59:30.884 INFO:teuthology.orchestra.run.smithi172.stdout:(103/152): python3-scipy-1.6.2-8.el9.x86_64.rpm 16 MB/s | 16 MB 00:00 2024-04-25T09:59:31.076 INFO:teuthology.orchestra.run.smithi172.stdout:(104/152): protobuf-compiler-3.14.0-13.el9.x86_ 1.4 MB/s | 863 kB 00:00 2024-04-25T09:59:31.143 INFO:teuthology.orchestra.run.smithi172.stdout:(105/152): abseil-cpp-20211102.0-3.el9.x86_64.r 996 kB/s | 548 kB 00:00 2024-04-25T09:59:31.401 INFO:teuthology.orchestra.run.smithi172.stdout:(106/152): grpc-data-1.46.7-10.el9.noarch.rpm 60 kB/s | 19 kB 00:00 2024-04-25T09:59:31.701 INFO:teuthology.orchestra.run.smithi172.stdout:(107/152): gperftools-libs-2.9.1-2.el9.x86_64.r 378 kB/s | 309 kB 00:00 2024-04-25T09:59:31.756 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-25T09:59:31.847 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:59:31.943 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:59:32.110 INFO:teuthology.orchestra.run.smithi172.stdout:(108/152): liboath-2.6.7-2.el9.x86_64.rpm 119 kB/s | 49 kB 00:00 2024-04-25T09:59:32.152 INFO:teuthology.orchestra.run.smithi172.stdout:(109/152): libarrow-doc-9.0.0-10.el9.noarch.rpm 35 kB/s | 26 kB 00:00 2024-04-25T09:59:32.177 INFO:teuthology.orchestra.run.smithi172.stdout:(110/152): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-04-25T09:59:32.210 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:59:32.366 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:59:32.409 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:59:32.527 INFO:teuthology.orchestra.run.smithi172.stdout:(111/152): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.3 MB/s | 838 kB 00:00 2024-04-25T09:59:32.543 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:59:32.969 INFO:teuthology.orchestra.run.smithi172.stdout:(112/152): libarrow-9.0.0-10.el9.x86_64.rpm 2.4 MB/s | 4.4 MB 00:01 2024-04-25T09:59:32.986 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:59:33.028 INFO:teuthology.orchestra.run.smithi172.stdout:(113/152): python3-bcrypt-3.2.2-1.el9.x86_64.rp 749 kB/s | 43 kB 00:00 2024-04-25T09:59:33.105 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:33.120 INFO:teuthology.orchestra.run.smithi172.stdout:(114/152): luarocks-3.9.2-1.el9.noarch.rpm 156 kB/s | 151 kB 00:00 2024-04-25T09:59:33.284 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:33.306 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:33.432 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:33.463 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:59:33.562 INFO:teuthology.orchestra.run.smithi172.stdout:(115/152): python3-certifi-2021.10.8-2.el9.noar 34 kB/s | 15 kB 00:00 2024-04-25T09:59:33.612 INFO:teuthology.orchestra.run.smithi172.stdout:(116/152): python3-asyncssh-2.13.2-5.el9.noarch 506 kB/s | 548 kB 00:01 2024-04-25T09:59:33.852 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:59:33.993 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:59:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:34.155 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:34.185 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:34.271 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:34.303 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:34.367 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:34.666 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:34.706 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:34.879 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:59:34.996 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:59:35.037 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:59:35.083 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:59:35.172 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:59:35.223 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:59:35.285 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:59:35.335 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:59:35.368 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:59:35.481 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:59:35.555 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:59:35.587 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:59:35.643 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:59:35.747 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:59:35.782 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:59:35.831 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:59:35.863 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:59:35.887 INFO:teuthology.orchestra.run.smithi172.stdout:(117/152): python3-cachetools-4.2.4-1.el9.noarc 11 kB/s | 32 kB 00:02 2024-04-25T09:59:35.957 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:59:36.023 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:59:36.121 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:59:36.289 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:59:36.351 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T09:59:37.006 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T09:59:37.108 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T09:59:37.153 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T09:59:37.198 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T09:59:37.239 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T09:59:37.309 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T09:59:37.559 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T09:59:37.587 INFO:teuthology.orchestra.run.smithi172.stdout:(118/152): python3-cheroot-8.6.0-4.el9.noarch.r 43 kB/s | 172 kB 00:04 2024-04-25T09:59:37.614 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T09:59:37.629 INFO:teuthology.orchestra.run.smithi172.stdout:(119/152): python3-cherrypy-18.6.1-2.el9.noarch 89 kB/s | 358 kB 00:04 2024-04-25T09:59:37.685 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T09:59:37.773 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T09:59:37.796 INFO:teuthology.orchestra.run.smithi172.stdout:(120/152): python3-cmd2-2.4.2-2.el9.noarch.rpm 159 kB/s | 303 kB 00:01 2024-04-25T09:59:37.813 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T09:59:37.851 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T09:59:37.916 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T09:59:37.960 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T09:59:38.019 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T09:59:38.086 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T09:59:38.284 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T09:59:38.332 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T09:59:38.370 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T09:59:38.425 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T09:59:38.552 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T09:59:38.594 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T09:59:38.630 INFO:teuthology.orchestra.run.smithi172.stdout:(121/152): python3-google-auth-2.29.0-1.el9.noa 206 kB/s | 206 kB 00:01 2024-04-25T09:59:38.641 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T09:59:38.655 INFO:teuthology.orchestra.run.smithi172.stdout:(122/152): python3-colorama-0.4.4-12.el9.noarch 33 kB/s | 35 kB 00:01 2024-04-25T09:59:38.676 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T09:59:38.709 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T09:59:38.722 INFO:teuthology.orchestra.run.smithi172.stdout:(123/152): python3-jaraco-8.2.1-3.el9.noarch.rp 162 kB/s | 11 kB 00:00 2024-04-25T09:59:38.780 INFO:teuthology.orchestra.run.smithi172.stdout:(124/152): python3-jaraco-classes-3.2.1-5.el9.n 306 kB/s | 18 kB 00:00 2024-04-25T09:59:38.839 INFO:teuthology.orchestra.run.smithi172.stdout:(125/152): python3-jaraco-collections-3.0.0-8.e 401 kB/s | 23 kB 00:00 2024-04-25T09:59:38.908 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T09:59:38.914 INFO:teuthology.orchestra.run.smithi172.stdout:(126/152): python3-jaraco-functools-3.5.0-2.el9 261 kB/s | 19 kB 00:00 2024-04-25T09:59:39.013 INFO:teuthology.orchestra.run.smithi172.stdout:(127/152): python3-jaraco-text-3.2.0-6.el9.noar 201 kB/s | 20 kB 00:00 2024-04-25T09:59:39.088 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T09:59:39.132 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T09:59:39.170 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T09:59:39.225 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T09:59:39.273 INFO:teuthology.orchestra.run.smithi172.stdout:(128/152): python3-grpcio-tools-1.46.7-10.el9.x 225 kB/s | 144 kB 00:00 2024-04-25T09:59:39.285 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T09:59:39.332 INFO:teuthology.orchestra.run.smithi172.stdout:(129/152): python3-logutils-0.3.5-21.el9.noarch 799 kB/s | 46 kB 00:00 2024-04-25T09:59:39.337 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T09:59:39.385 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T09:59:39.400 INFO:teuthology.orchestra.run.smithi172.stdout:(130/152): python3-grpcio-1.46.7-10.el9.x86_64. 1.3 MB/s | 2.0 MB 00:01 2024-04-25T09:59:39.425 INFO:teuthology.orchestra.run.smithi172.stdout:(131/152): python3-more-itertools-8.12.0-2.el9. 871 kB/s | 79 kB 00:00 2024-04-25T09:59:39.431 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T09:59:39.488 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T09:59:39.566 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T09:59:39.614 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T09:59:39.756 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T09:59:39.817 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T09:59:39.861 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T09:59:39.930 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T09:59:40.078 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T09:59:40.140 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T09:59:40.183 INFO:teuthology.orchestra.run.smithi172.stdout:(132/152): python3-pecan-1.4.2-3.el9.noarch.rpm 359 kB/s | 272 kB 00:00 2024-04-25T09:59:40.233 INFO:teuthology.orchestra.run.smithi172.stdout:(133/152): python3-kubernetes-26.1.0-2.el9.noar 858 kB/s | 1.0 MB 00:01 2024-04-25T09:59:40.258 INFO:teuthology.orchestra.run.smithi172.stdout:(134/152): python3-portend-3.1.0-2.el9.noarch.r 221 kB/s | 16 kB 00:00 2024-04-25T09:59:40.762 INFO:teuthology.orchestra.run.smithi172.stdout:(135/152): python3-pyperclip-1.8.2-2.el9.noarch 56 kB/s | 28 kB 00:00 2024-04-25T09:59:40.792 INFO:teuthology.orchestra.run.smithi172.stdout:(136/152): python3-natsort-7.1.1-5.el9.noarch.r 41 kB/s | 58 kB 00:01 2024-04-25T09:59:40.951 INFO:teuthology.orchestra.run.smithi172.stdout:(137/152): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-04-25T09:59:40.977 INFO:teuthology.orchestra.run.smithi172.stdout:(138/152): python3-repoze-lru-0.7-16.el9.noarch 144 kB/s | 31 kB 00:00 2024-04-25T09:59:41.001 INFO:teuthology.orchestra.run.smithi172.stdout:(139/152): python3-pyOpenSSL-21.0.0-1.el9.noarc 117 kB/s | 90 kB 00:00 2024-04-25T09:59:41.035 INFO:teuthology.orchestra.run.smithi172.stdout:(140/152): python3-tempora-5.0.0-2.el9.noarch.r 617 kB/s | 36 kB 00:00 2024-04-25T09:59:41.097 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T09:59:41.148 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T09:59:41.182 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T09:59:41.303 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T09:59:41.330 INFO:teuthology.orchestra.run.smithi172.stdout:(141/152): python3-typing-extensions-4.4.0-2.el 155 kB/s | 51 kB 00:00 2024-04-25T09:59:41.361 INFO:teuthology.orchestra.run.smithi172.stdout:(142/152): python3-rsa-4.9-2.el9.noarch.rpm 144 kB/s | 59 kB 00:00 2024-04-25T09:59:41.367 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T09:59:41.394 INFO:teuthology.orchestra.run.smithi172.stdout:(143/152): python3-webob-1.8.7-6.el9.noarch.rpm 641 kB/s | 230 kB 00:00 2024-04-25T09:59:41.453 INFO:teuthology.orchestra.run.smithi172.stdout:(144/152): python3-zc-lockfile-2.0-10.el9.noarc 346 kB/s | 20 kB 00:00 2024-04-25T09:59:41.462 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T09:59:41.516 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T09:59:41.620 INFO:teuthology.orchestra.run.smithi172.stdout:(145/152): re2-20211101-3.el9.x86_64.rpm 1.1 MB/s | 192 kB 00:00 2024-04-25T09:59:41.653 INFO:teuthology.orchestra.run.smithi172.stdout:(146/152): python3-werkzeug-2.0.3-3.el9.1.noarc 1.4 MB/s | 427 kB 00:00 2024-04-25T09:59:41.746 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T09:59:41.778 INFO:teuthology.orchestra.run.smithi172.stdout:(147/152): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-04-25T09:59:41.796 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T09:59:41.916 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T09:59:42.029 INFO:teuthology.orchestra.run.smithi172.stdout:(148/152): librados2-19.0.0-1628.g97ae6cc9.el9. 9.0 MB/s | 3.4 MB 00:00 2024-04-25T09:59:42.229 INFO:teuthology.orchestra.run.smithi172.stdout:(149/152): openssl-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 1.3 MB 00:00 2024-04-25T09:59:42.380 INFO:teuthology.orchestra.run.smithi172.stdout:(150/152): librbd1-19.0.0-1628.g97ae6cc9.el9.x8 5.2 MB/s | 3.1 MB 00:00 2024-04-25T09:59:42.563 INFO:teuthology.orchestra.run.smithi172.stdout:(151/152): openssl-libs-3.2.1-1.el9.x86_64.rpm 7.2 MB/s | 2.4 MB 00:00 2024-04-25T09:59:42.738 INFO:teuthology.orchestra.run.smithi172.stdout:(152/152): python3-websocket-client-1.2.3-2.el9 64 kB/s | 90 kB 00:01 2024-04-25T09:59:42.744 INFO:teuthology.orchestra.run.smithi172.stdout:-------------------------------------------------------------------------------- 2024-04-25T09:59:42.745 INFO:teuthology.orchestra.run.smithi172.stdout:Total 7.7 MB/s | 215 MB 00:27 2024-04-25T09:59:42.753 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T09:59:42.894 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T09:59:44.539 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T09:59:44.634 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T09:59:44.681 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T09:59:44.727 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T09:59:45.102 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T09:59:45.147 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:59:45.269 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T09:59:45.304 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T09:59:45.360 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T09:59:45.613 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction check 2024-04-25T09:59:45.862 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction check succeeded. 2024-04-25T09:59:45.862 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction test 2024-04-25T09:59:45.878 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T09:59:45.908 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:59:46.008 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T09:59:46.034 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T09:59:48.470 INFO:teuthology.orchestra.run.smithi172.stdout:Transaction test succeeded. 2024-04-25T09:59:48.471 INFO:teuthology.orchestra.run.smithi172.stdout:Running transaction 2024-04-25T09:59:48.564 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:59:48.606 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:59:49.267 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T09:59:49.326 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:59:49.511 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T09:59:49.511 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T09:59:49.511 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T09:59:49.511 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T09:59:49.551 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T09:59:54.783 INFO:teuthology.orchestra.run.smithi172.stdout: Preparing : 1/1 2024-04-25T09:59:54.887 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/156 2024-04-25T09:59:54.985 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/156 2024-04-25T09:59:55.082 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/156 2024-04-25T09:59:55.163 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/156 2024-04-25T09:59:55.205 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/156 2024-04-25T09:59:55.349 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/156 2024-04-25T09:59:55.800 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/156 2024-04-25T09:59:55.911 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:56.089 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T09:59:56.112 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:56.297 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T09:59:56.345 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T09:59:56.695 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/156 2024-04-25T09:59:56.752 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/156 2024-04-25T09:59:56.783 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:56.993 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T09:59:57.026 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:57.147 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 14/156 2024-04-25T09:59:57.177 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:57.254 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/156 2024-04-25T09:59:57.681 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:57.737 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/156 2024-04-25T09:59:57.932 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : re2-1:20211101-3.el9.x86_64 17/156 2024-04-25T09:59:58.066 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 18/156 2024-04-25T09:59:58.119 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/156 2024-04-25T09:59:58.169 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/156 2024-04-25T09:59:58.273 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 21/156 2024-04-25T09:59:58.323 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 22/156 2024-04-25T09:59:58.388 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 23/156 2024-04-25T09:59:58.438 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 24/156 2024-04-25T09:59:58.476 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-5.4.4-4.el9.x86_64 25/156 2024-04-25T09:59:58.592 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 26/156 2024-04-25T09:59:58.662 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 27/156 2024-04-25T09:59:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 28/156 2024-04-25T09:59:58.761 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 29/156 2024-04-25T09:59:58.864 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 30/156 2024-04-25T09:59:58.898 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 31/156 2024-04-25T09:59:58.950 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 32/156 2024-04-25T09:59:58.979 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 33/156 2024-04-25T09:59:59.092 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 34/156 2024-04-25T09:59:59.150 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : luarocks-3.9.2-1.el9.noarch 35/156 2024-04-25T09:59:59.258 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 36/156 2024-04-25T09:59:59.433 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 37/156 2024-04-25T09:59:59.501 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 38/156 2024-04-25T10:00:00.177 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rsa-4.9-2.el9.noarch 39/156 2024-04-25T10:00:00.282 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 40/156 2024-04-25T10:00:00.352 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 41/156 2024-04-25T10:00:00.410 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 42/156 2024-04-25T10:00:00.459 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 43/156 2024-04-25T10:00:00.534 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 44/156 2024-04-25T10:00:00.793 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 45/156 2024-04-25T10:00:00.856 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 46/156 2024-04-25T10:00:00.965 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 47/156 2024-04-25T10:00:01.089 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 48/156 2024-04-25T10:00:01.131 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 49/156 2024-04-25T10:00:01.170 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 50/156 2024-04-25T10:00:01.237 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 51/156 2024-04-25T10:00:01.278 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 52/156 2024-04-25T10:00:01.340 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 53/156 2024-04-25T10:00:01.404 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 54/156 2024-04-25T10:00:01.604 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 55/156 2024-04-25T10:00:01.659 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 56/156 2024-04-25T10:00:01.696 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 57/156 2024-04-25T10:00:01.743 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 58/156 2024-04-25T10:00:01.872 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 59/156 2024-04-25T10:00:01.912 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 60/156 2024-04-25T10:00:01.960 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 61/156 2024-04-25T10:00:01.994 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 62/156 2024-04-25T10:00:02.054 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 63/156 2024-04-25T10:00:02.235 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 64/156 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-04-25T10:00:02.314 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:02.409 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 65/156 2024-04-25T10:00:02.451 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 66/156 2024-04-25T10:00:02.489 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 67/156 2024-04-25T10:00:02.529 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:02.544 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 68/156 2024-04-25T10:00:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:02.574 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:02.574 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:00:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:02.611 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : uuid-1.6.2-55.el9.x86_64 69/156 2024-04-25T10:00:02.706 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 70/156 2024-04-25T10:00:02.766 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 71/156 2024-04-25T10:00:02.838 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 72/156 2024-04-25T10:00:02.912 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 73/156 2024-04-25T10:00:03.014 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 74/156 2024-04-25T10:00:03.091 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 75/156 2024-04-25T10:00:03.257 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 76/156 2024-04-25T10:00:03.349 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 77/156 2024-04-25T10:00:03.417 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 78/156 2024-04-25T10:00:03.509 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 79/156 2024-04-25T10:00:03.681 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-packaging-20.9-5.el9.noarch 80/156 2024-04-25T10:00:03.777 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 81/156 2024-04-25T10:00:04.189 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:04.266 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:04.267 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:04.267 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:00:04.267 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:04.267 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:04.267 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:04.497 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:04.565 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:04.566 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:04.566 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:00:04.566 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:04.566 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:04.566 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:04.768 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 82/156 2024-04-25T10:00:04.843 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 83/156 2024-04-25T10:00:04.893 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 84/156 2024-04-25T10:00:05.063 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 85/156 2024-04-25T10:00:05.073 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:05.140 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:05.146 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 86/156 2024-04-25T10:00:05.272 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 87/156 2024-04-25T10:00:05.282 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:00:05.329 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 88/156 2024-04-25T10:00:05.345 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:00:05.427 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:00:05.591 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-0.3.21-2.el9.x86_64 89/156 2024-04-25T10:00:05.664 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:00:05.667 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 90/156 2024-04-25T10:00:05.727 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:00:05.817 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:00:05.817 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 91/156 2024-04-25T10:00:05.951 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:06.121 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:06.177 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:00:06.217 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:00:06.258 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:00:06.750 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 92/156 2024-04-25T10:00:06.902 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:06.916 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 93/156 2024-04-25T10:00:06.945 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:06.998 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:00:07.111 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:00:07.146 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:07.473 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:07.505 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:08.496 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 94/156 2024-04-25T10:00:08.599 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 95/156 2024-04-25T10:00:08.635 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 96/156 2024-04-25T10:00:08.651 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:08.658 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 97/156 2024-04-25T10:00:08.681 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:08.828 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:09.012 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:00:09.022 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/156 2024-04-25T10:00:09.041 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:09.070 INFO:teuthology.orchestra.run.smithi172.stdout: Upgrading : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:00:09.137 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:09.137 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:09.137 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:00:09.137 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:09.137 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:09.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:09.183 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 99/156 2024-04-25T10:00:09.200 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:09.210 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 100/156 2024-04-25T10:00:09.239 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:09.266 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 101/156 2024-04-25T10:00:09.350 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:00:09.423 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:00:09.780 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 102/156 2024-04-25T10:00:09.815 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:00:09.939 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 103/156 2024-04-25T10:00:09.967 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 104/156 2024-04-25T10:00:10.354 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:00:10.385 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:10.458 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:10.459 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:10.459 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:00:10.459 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:10.459 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:10.460 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:10.503 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:10.588 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:10.588 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:10.588 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:00:10.589 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:10.653 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:10.747 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:10.747 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:10.747 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:00:10.747 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:10.747 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:10.747 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:11.104 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:11.178 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:11.179 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:11.179 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:00:11.179 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:11.179 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:11.179 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:12.643 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:00:12.682 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:00:13.339 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 105/156 2024-04-25T10:00:13.385 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:00:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 106/156 2024-04-25T10:00:13.543 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-25T10:00:13.543 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-25T10:00:13.543 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:13.594 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:00:16.025 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:00:16.079 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:00:16.189 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:00:16.239 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:00:16.288 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:00:16.614 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:00:16.671 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:00:16.699 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:00:17.577 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:17.621 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:17.687 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:19.059 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:00:19.185 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:00:19.268 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:00:19.435 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:00:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:00:19.474 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:19.599 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:19.648 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:19.789 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:19.920 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:20.198 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:20.198 INFO:teuthology.orchestra.run.smithi165.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T10:00:20.199 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:00:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:00:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:00:25.285 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:00:25.286 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:00:25.287 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:00:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:00:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:00:25.299 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:00:25.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:00:25.301 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:00:25.302 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:00:25.303 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:00:25.304 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:00:25.305 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:00:25.306 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:00:25.307 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:25.308 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:00:26.538 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 107/156 2024-04-25T10:00:26.538 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /sys 2024-04-25T10:00:26.538 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /proc 2024-04-25T10:00:26.539 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /mnt 2024-04-25T10:00:26.539 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /var/tmp 2024-04-25T10:00:26.539 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /home 2024-04-25T10:00:26.539 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /root 2024-04-25T10:00:26.539 INFO:teuthology.orchestra.run.smithi172.stdout:skipping the directory /tmp 2024-04-25T10:00:26.539 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:26.775 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:26.822 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 108/156 2024-04-25T10:00:26.822 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:26.822 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-25T10:00:26.822 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:27.657 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:27.657 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:27.657 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-04-25T10:00:27.657 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.658 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.659 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:00:27.660 INFO:teuthology.orchestra.run.smithi165.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:00:27.661 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:00:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:00:27.663 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.664 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:00:27.665 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:00:27.666 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:00:27.667 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:00:27.668 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.669 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:00:27.670 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:00:27.671 INFO:teuthology.orchestra.run.smithi165.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:00:27.671 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:27.671 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-25T10:00:28.231 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:28.469 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:28.559 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 109/156 2024-04-25T10:00:28.559 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:28.559 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-25T10:00:28.559 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:28.559 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-25T10:00:28.559 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:28.785 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:28.883 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 110/156 2024-04-25T10:00:28.883 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:28.883 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-25T10:00:28.883 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:28.883 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-25T10:00:28.883 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:29.378 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:29.473 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 111/156 2024-04-25T10:00:29.473 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:29.473 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-25T10:00:29.473 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:29.474 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-25T10:00:29.474 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:29.621 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : automake-1.16.2-8.el9.noarch 112/156 2024-04-25T10:00:29.685 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-ply-3.11-14.el9.noarch 113/156 2024-04-25T10:00:29.764 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 114/156 2024-04-25T10:00:30.008 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 115/156 2024-04-25T10:00:30.066 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 116/156 2024-04-25T10:00:30.150 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 117/156 2024-04-25T10:00:30.250 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:30.429 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:30.508 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 120/156 2024-04-25T10:00:30.549 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 121/156 2024-04-25T10:00:30.589 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : fuse-2.9.9-15.el9.x86_64 122/156 2024-04-25T10:00:31.237 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:31.264 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 123/156 2024-04-25T10:00:31.308 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 124/156 2024-04-25T10:00:31.408 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 125/156 2024-04-25T10:00:31.444 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:31.773 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 126/156 2024-04-25T10:00:31.803 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:32.939 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 127/156 2024-04-25T10:00:32.962 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:33.107 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 128/156 2024-04-25T10:00:33.303 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 129/156 2024-04-25T10:00:33.381 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 130/156 2024-04-25T10:00:33.452 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:33.452 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-25T10:00:33.452 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:33.452 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-25T10:00:33.452 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:33.540 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:33.582 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 131/156 2024-04-25T10:00:33.734 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 132/156 2024-04-25T10:00:33.810 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 133/156 2024-04-25T10:00:34.799 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libtool-2.4.6-45.el9.x86_64 134/156 2024-04-25T10:00:34.834 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:34.903 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 135/156 2024-04-25T10:00:34.904 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:34.904 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-25T10:00:34.904 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:34.904 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-25T10:00:34.904 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:34.960 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:35.032 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 136/156 2024-04-25T10:00:35.032 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:35.032 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-25T10:00:35.032 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:35.110 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:35.190 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 137/156 2024-04-25T10:00:35.190 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:35.190 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-25T10:00:35.190 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:35.190 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-25T10:00:35.190 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:35.594 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:35.662 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 138/156 2024-04-25T10:00:35.662 INFO:teuthology.orchestra.run.smithi172.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-25T10:00:35.662 INFO:teuthology.orchestra.run.smithi172.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-25T10:00:35.662 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:35.662 INFO:teuthology.orchestra.run.smithi172.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-25T10:00:35.662 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:40.550 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 139/156 2024-04-25T10:00:40.605 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 140/156 2024-04-25T10:00:40.704 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 141/156 2024-04-25T10:00:40.756 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 142/156 2024-04-25T10:00:40.803 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 143/156 2024-04-25T10:00:41.169 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 144/156 2024-04-25T10:00:41.221 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 145/156 2024-04-25T10:00:41.249 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 146/156 2024-04-25T10:00:42.537 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:42.589 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:42.648 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 147/156 2024-04-25T10:00:43.981 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 148/156 2024-04-25T10:00:44.096 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 149/156 2024-04-25T10:00:44.177 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 150/156 2024-04-25T10:00:44.343 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : flex-2.6.4-9.el9.x86_64 151/156 2024-04-25T10:00:44.385 INFO:teuthology.orchestra.run.smithi172.stdout: Installing : bison-3.7.4-5.el9.x86_64 152/156 2024-04-25T10:00:44.386 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:44.511 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 153/156 2024-04-25T10:00:44.568 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:44.568 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:44.708 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 155/156 2024-04-25T10:00:44.838 INFO:teuthology.orchestra.run.smithi172.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:45.190 INFO:teuthology.orchestra.run.smithi172.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:45.190 INFO:teuthology.orchestra.run.smithi172.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-04-25T10:00:45.190 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:50.288 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 1/156 2024-04-25T10:00:50.288 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2/156 2024-04-25T10:00:50.288 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 3/156 2024-04-25T10:00:50.288 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 4/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-1628.g97ae6 5/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 6/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 7/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 8/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 9/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 10/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 11/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 12/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 13/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 14/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_ 15/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 16/156 2024-04-25T10:00:50.289 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 17/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_6 18/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86 19/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 20/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el 21/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9. 22/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_6 23/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 24/156 2024-04-25T10:00:50.290 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 25/156 2024-04-25T10:00:50.302 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 26/156 2024-04-25T10:00:50.302 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 27/156 2024-04-25T10:00:50.302 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 28/156 2024-04-25T10:00:50.302 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 29/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9. 30/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noa 31/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.n 32/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-1628.g97a 33/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el 34/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 35/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.e 36/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 37/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 38/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 39/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/156 2024-04-25T10:00:50.303 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/156 2024-04-25T10:00:50.304 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/156 2024-04-25T10:00:50.305 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/156 2024-04-25T10:00:50.306 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/156 2024-04-25T10:00:50.307 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/156 2024-04-25T10:00:50.308 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/156 2024-04-25T10:00:50.309 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/156 2024-04-25T10:00:50.310 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 149/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 151/156 2024-04-25T10:00:50.311 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/156 2024-04-25T10:00:50.312 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 153/156 2024-04-25T10:00:50.312 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 154/156 2024-04-25T10:00:50.312 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 155/156 2024-04-25T10:00:52.786 INFO:teuthology.orchestra.run.smithi172.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 156/156 2024-04-25T10:00:52.786 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:52.786 INFO:teuthology.orchestra.run.smithi172.stdout:Upgraded: 2024-04-25T10:00:52.787 INFO:teuthology.orchestra.run.smithi172.stdout: librados2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.787 INFO:teuthology.orchestra.run.smithi172.stdout: librbd1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.787 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:52.787 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:52.787 INFO:teuthology.orchestra.run.smithi172.stdout:Installed: 2024-04-25T10:00:52.787 INFO:teuthology.orchestra.run.smithi172.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-25T10:00:52.788 INFO:teuthology.orchestra.run.smithi172.stdout: automake-1.16.2-8.el9.noarch 2024-04-25T10:00:52.788 INFO:teuthology.orchestra.run.smithi172.stdout: bison-3.7.4-5.el9.x86_64 2024-04-25T10:00:52.788 INFO:teuthology.orchestra.run.smithi172.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-25T10:00:52.788 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.788 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-base-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.789 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.789 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.789 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-grafana-dashboards-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.789 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-immutable-object-cache-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.789 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mds-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.789 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.790 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.790 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-dashboard-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.790 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-diskprediction-local-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.790 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-modules-core-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.791 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mgr-rook-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.791 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-mon-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.791 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-osd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.791 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-prometheus-alerts-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.791 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-radosgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.791 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-selinux-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.792 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-test-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.792 INFO:teuthology.orchestra.run.smithi172.stdout: ceph-volume-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.792 INFO:teuthology.orchestra.run.smithi172.stdout: cephadm-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.792 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.792 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-shell-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.792 INFO:teuthology.orchestra.run.smithi172.stdout: cephfs-top-2:19.0.0-1628.g97ae6cc9.el9.noarch 2024-04-25T10:00:52.793 INFO:teuthology.orchestra.run.smithi172.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-25T10:00:52.793 INFO:teuthology.orchestra.run.smithi172.stdout: flex-2.6.4-9.el9.x86_64 2024-04-25T10:00:52.793 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-25T10:00:52.793 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-25T10:00:52.793 INFO:teuthology.orchestra.run.smithi172.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-25T10:00:52.794 INFO:teuthology.orchestra.run.smithi172.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-25T10:00:52.794 INFO:teuthology.orchestra.run.smithi172.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-25T10:00:52.794 INFO:teuthology.orchestra.run.smithi172.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-25T10:00:52.794 INFO:teuthology.orchestra.run.smithi172.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-25T10:00:52.794 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-25T10:00:52.794 INFO:teuthology.orchestra.run.smithi172.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-25T10:00:52.795 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.795 INFO:teuthology.orchestra.run.smithi172.stdout: libcephfs2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.795 INFO:teuthology.orchestra.run.smithi172.stdout: libcephsqlite-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.795 INFO:teuthology.orchestra.run.smithi172.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-25T10:00:52.795 INFO:teuthology.orchestra.run.smithi172.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-25T10:00:52.796 INFO:teuthology.orchestra.run.smithi172.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-25T10:00:52.796 INFO:teuthology.orchestra.run.smithi172.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-25T10:00:52.796 INFO:teuthology.orchestra.run.smithi172.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-25T10:00:52.796 INFO:teuthology.orchestra.run.smithi172.stdout: librados-devel-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.796 INFO:teuthology.orchestra.run.smithi172.stdout: libradosstriper1-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.797 INFO:teuthology.orchestra.run.smithi172.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-25T10:00:52.797 INFO:teuthology.orchestra.run.smithi172.stdout: librgw2-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.797 INFO:teuthology.orchestra.run.smithi172.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-25T10:00:52.797 INFO:teuthology.orchestra.run.smithi172.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-25T10:00:52.797 INFO:teuthology.orchestra.run.smithi172.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-25T10:00:52.798 INFO:teuthology.orchestra.run.smithi172.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-25T10:00:52.798 INFO:teuthology.orchestra.run.smithi172.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-25T10:00:52.798 INFO:teuthology.orchestra.run.smithi172.stdout: lua-5.4.4-4.el9.x86_64 2024-04-25T10:00:52.798 INFO:teuthology.orchestra.run.smithi172.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-25T10:00:52.798 INFO:teuthology.orchestra.run.smithi172.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-25T10:00:52.799 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-25T10:00:52.799 INFO:teuthology.orchestra.run.smithi172.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-25T10:00:52.799 INFO:teuthology.orchestra.run.smithi172.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-04-25T10:00:52.799 INFO:teuthology.orchestra.run.smithi172.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-25T10:00:52.799 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-13.14-1.el9.x86_64 2024-04-25T10:00:52.799 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-25T10:00:52.800 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-25T10:00:52.800 INFO:teuthology.orchestra.run.smithi172.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-25T10:00:52.800 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-25T10:00:52.800 INFO:teuthology.orchestra.run.smithi172.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-25T10:00:52.800 INFO:teuthology.orchestra.run.smithi172.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-25T10:00:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-25T10:00:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-25T10:00:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-25T10:00:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-25T10:00:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-argparse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.801 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ceph-common-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.802 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cephfs-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.802 INFO:teuthology.orchestra.run.smithi172.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-25T10:00:52.802 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-25T10:00:52.802 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-25T10:00:52.803 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-25T10:00:52.804 INFO:teuthology.orchestra.run.smithi172.stdout: python3-ply-3.11-14.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rados-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-25T10:00:52.805 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rgw-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-25T10:00:52.806 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-fuse-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-mirror-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: rbd-nbd-2:19.0.0-1628.g97ae6cc9.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: re2-1:20211101-3.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-25T10:00:52.807 INFO:teuthology.orchestra.run.smithi172.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi172.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:52.808 INFO:teuthology.orchestra.run.smithi172.stdout:Complete! 2024-04-25T10:00:53.365 DEBUG:teuthology.parallel:result is None 2024-04-25T10:00:53.366 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:00:53.509 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:00:53.581 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:53.582 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:53.582 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:00:53.584 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec 2024-04-25T10:00:53.721 DEBUG:teuthology.orchestra.run.smithi172:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-25T10:00:53.776 INFO:teuthology.orchestra.run.smithi172.stdout:19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:53.777 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1628.g97ae6cc9.el9 2024-04-25T10:00:53.777 INFO:teuthology.task.install:The correct ceph version 19.0.0-1628.g97ae6cc9 is installed. 2024-04-25T10:00:53.779 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-25T10:00:53.779 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:53.779 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:00:53.871 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:53.871 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-25T10:00:53.938 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-25T10:00:53.939 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:53.939 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:00:54.031 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:00:54.161 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:54.161 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/daemon-helper 2024-04-25T10:00:54.243 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-25T10:00:54.355 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-25T10:00:54.355 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:54.355 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:00:54.444 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:00:54.573 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:54.573 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-25T10:00:54.643 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-25T10:00:54.750 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-25T10:00:54.750 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:54.750 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:00:54.837 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:00:54.914 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:54.915 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/usr/bin/stdin-killer 2024-04-25T10:00:54.988 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-25T10:00:55.094 INFO:teuthology.run_tasks:Running task ceph... 2024-04-25T10:00:55.192 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-25T10:00:55.192 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2024-04-25T10:00:55.200 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 777 /var/log/ceph 2024-04-25T10:00:55.274 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-25T10:00:55.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:00:55.282 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-25T10:00:55.355 INFO:tasks.ceph:Creating extra log directories... 2024-04-25T10:00:55.355 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:00:55.360 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-25T10:00:55.446 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-04-25T10:00:55.447 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-04-25T10:00:55.447 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-04-25T10:00:55.447 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:55.447 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:00:55.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:55.716 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:55.801 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:55.801 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-04-25T10:00:55.845 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:56.029 DEBUG:teuthology.orchestra.run.smithi172:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-04-25T10:00:56.099 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-25T10:00:56.099 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-25T10:00:56.099 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043', 'branch': 'wip-neesingh-testing-240416', 'description': 'fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7673043', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-240416'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, 'kclient': {'syntax': 'v1'}, 'subvolume_version': 1, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-62537', 'sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 1124, 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'sleep_before_teardown': 0, 'suite': 'fs:functional', 'suite_branch': 'wip-62537', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_a3b7a189d278a9762c1acb7de36790ef0ed7b448/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'a3b7a189d278a9762c1acb7de36790ef0ed7b448', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC6aup13/8uXndBL7OybaWL5ojZP3xZkhfwUedxkjro7f5VMctuST/SOP6MuUCBjN1Qap8rZapUydXA7FkvMNJk=', 'smithi172.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKsm4vg/xcnI6aXvixXr/+qzZ14gpGbLouNFU4J2X0xkBFy26DfUNH5vwRcycNBuhrTI56C3KhERAQOhuhrcrnM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'Replacing daemon mds', 'Scrub error on inode', 'Behind on trimming', 'Metadata damage detected', 'bad backtrace on inode', 'overall HEALTH_', '\\(MDS_TRIM\\)'], 'sha1': '97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_scrub_checks', 'tasks.cephfs.test_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-25_09:36:50', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-25T10:00:56.100 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:00:56.107 DEBUG:teuthology.orchestra.run.smithi172:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-25T10:00:56.159 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:00:56.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo install -d -m0777 -- /var/run/ceph 2024-04-25T10:00:56.251 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:56.251 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:00:56.304 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:00:56.305 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-25 10:00:20.636851179 +0000 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-25 09:57:40.836834939 +0000 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-25 09:57:40.836834939 +0000 2024-04-25T10:00:56.357 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-25 09:57:40.836834939 +0000 2024-04-25T10:00:56.358 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:00:56.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:00:56.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:00:56.438 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000379971 s, 1.3 MB/s 2024-04-25T10:00:56.441 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:00:56.501 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-04-25T10:00:56.550 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:00:56.550 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:56.550 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 651 Links: 1 2024-04-25T10:00:56.551 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:56.551 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:56.551 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-25 10:00:20.637851164 +0000 2024-04-25T10:00:56.551 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-25 09:57:41.407826077 +0000 2024-04-25T10:00:56.551 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-25 09:57:41.407826077 +0000 2024-04-25T10:00:56.551 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-25 09:57:41.407826077 +0000 2024-04-25T10:00:56.552 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:00:56.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:00:56.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:00:56.627 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000328806 s, 1.6 MB/s 2024-04-25T10:00:56.630 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:00:56.681 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-04-25T10:00:56.732 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:00:56.732 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:56.732 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 669 Links: 1 2024-04-25T10:00:56.733 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:56.733 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:56.733 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-25 10:00:20.637851164 +0000 2024-04-25T10:00:56.733 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-25 09:57:42.010816719 +0000 2024-04-25T10:00:56.733 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-25 09:57:42.010816719 +0000 2024-04-25T10:00:56.733 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-25 09:57:42.010816719 +0000 2024-04-25T10:00:56.733 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:00:56.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:00:56.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:00:56.809 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000342668 s, 1.5 MB/s 2024-04-25T10:00:56.812 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:00:56.866 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-04-25T10:00:56.915 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:00:56.915 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:56.915 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 6h/6d Inode: 690 Links: 1 2024-04-25T10:00:56.915 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:56.916 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:56.916 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-25 10:00:20.638851149 +0000 2024-04-25T10:00:56.916 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-25 09:57:42.600807562 +0000 2024-04-25T10:00:56.916 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-25 09:57:42.600807562 +0000 2024-04-25T10:00:56.916 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-25 09:57:42.600807562 +0000 2024-04-25T10:00:56.916 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:00:56.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:00:56.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:00:56.986 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000359005 s, 1.4 MB/s 2024-04-25T10:00:56.989 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:00:57.042 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:57.043 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:57.043 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/scratch_devs of=/dev/stdout 2024-04-25T10:00:57.082 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-25T10:00:57.082 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_1 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 633 Links: 1 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-25 10:00:45.628278732 +0000 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-25 09:58:11.095024565 +0000 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-25 09:58:11.095024565 +0000 2024-04-25T10:00:57.162 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-25 09:58:11.095024565 +0000 2024-04-25T10:00:57.162 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-25T10:00:57.264 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-25T10:00:57.265 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-25T10:00:57.265 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.00039911 s, 1.3 MB/s 2024-04-25T10:00:57.269 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-25T10:00:57.310 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_2 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-25 10:00:45.628278732 +0000 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-25 09:58:11.699009801 +0000 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-25 09:58:11.699009801 +0000 2024-04-25T10:00:57.391 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-25 09:58:11.699009801 +0000 2024-04-25T10:00:57.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-25T10:00:57.492 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-25T10:00:57.492 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-25T10:00:57.492 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000324847 s, 1.6 MB/s 2024-04-25T10:00:57.495 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-25T10:00:57.534 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_3 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 670 Links: 1 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-25 10:00:45.629278707 +0000 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-25 09:58:12.304994989 +0000 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-25 09:58:12.304994989 +0000 2024-04-25T10:00:57.614 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-25 09:58:12.304994989 +0000 2024-04-25T10:00:57.615 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-25T10:00:57.714 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000692667 s, 739 kB/s 2024-04-25T10:00:57.716 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-25T10:00:57.757 DEBUG:teuthology.orchestra.run.smithi172:> stat /dev/vg_nvme/lv_4 2024-04-25T10:00:57.839 INFO:teuthology.orchestra.run.smithi172.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout:Device: 6h/6d Inode: 691 Links: 1 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout:Context: system_u:object_r:device_t:s0 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout:Access: 2024-04-25 10:00:45.630278683 +0000 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout:Modify: 2024-04-25 09:58:12.878980959 +0000 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout:Change: 2024-04-25 09:58:12.878980959 +0000 2024-04-25T10:00:57.840 INFO:teuthology.orchestra.run.smithi172.stdout: Birth: 2024-04-25 09:58:12.878980959 +0000 2024-04-25T10:00:57.840 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-25T10:00:57.936 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records in 2024-04-25T10:00:57.937 INFO:teuthology.orchestra.run.smithi172.stderr:1+0 records out 2024-04-25T10:00:57.937 INFO:teuthology.orchestra.run.smithi172.stderr:512 bytes copied, 0.000287416 s, 1.8 MB/s 2024-04-25T10:00:57.939 DEBUG:teuthology.orchestra.run.smithi172:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-25T10:00:57.978 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:00:57.979 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:00:57.979 INFO:tasks.ceph:Generating config... 2024-04-25T10:00:57.980 INFO:tasks.ceph:[client] client mount timeout = 600 2024-04-25T10:00:57.980 INFO:tasks.ceph:[client] debug client = 20 2024-04-25T10:00:57.980 INFO:tasks.ceph:[client] debug ms = 1 2024-04-25T10:00:57.980 INFO:tasks.ceph:[client] pid file = 2024-04-25T10:00:57.980 INFO:tasks.ceph:[client] rados mon op timeout = 900 2024-04-25T10:00:57.980 INFO:tasks.ceph:[client] rados osd op timeout = 900 2024-04-25T10:00:57.980 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-04-25T10:00:57.980 INFO:tasks.ceph:[mds] debug mds = 20 2024-04-25T10:00:57.980 INFO:tasks.ceph:[mds] debug mds balancer = 20 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] debug ms = 1 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] mds cache max size = 1000 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] mds debug frag = True 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] mds log max segments = 1 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] mds op complaint time = 180 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] mds verify scatter = True 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] osd op complaint time = 180 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-25T10:00:57.981 INFO:tasks.ceph:[mon] mon op complaint time = 120 2024-04-25T10:00:57.981 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-25T10:00:57.982 INFO:tasks.ceph:[osd] osd op complaint time = 180 2024-04-25T10:00:57.986 INFO:tasks.ceph:Setting up mon.a... 2024-04-25T10:00:57.986 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-25T10:00:58.119 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.keyring 2024-04-25T10:00:58.128 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-25T10:00:58.247 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:58.330 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.165'), ('mon.b', '172.21.15.172'), ('mon.c', '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]')] 2024-04-25T10:00:58.330 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds cache max size': 1000, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds log max segments': 1, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2024-04-25T10:00:58.331 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:58.331 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:00:58.383 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.165 --add b 172.21.15.172 --addv c '[v2:172.21.15.172:3301,v1:172.21.15.172:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:58.535 INFO:teuthology.orchestra.run.smithi165.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:58.535 INFO:teuthology.orchestra.run.smithi165.stdout:monmaptool: generated fsid 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:00:58.535 INFO:teuthology.orchestra.run.smithi165.stdout:setting min_mon_release = quincy 2024-04-25T10:00:58.535 INFO:teuthology.orchestra.run.smithi165.stdout:epoch 0 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:fsid 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:last_changed 2024-04-25T10:00:58.532352+0000 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:created 2024-04-25T10:00:58.532352+0000 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:min_mon_release 17 (quincy) 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:election_strategy: 1 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:0: [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] mon.a 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:1: [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] mon.b 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:2: [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] mon.c 2024-04-25T10:00:58.536 INFO:teuthology.orchestra.run.smithi165.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-25T10:00:58.539 DEBUG:teuthology.orchestra.run.smithi165:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-25T10:00:58.587 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 55d22d87-b25b-4ccd-bc45-d797a973b3c0... 2024-04-25T10:00:58.588 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:00:58.593 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.700 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.701 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.702 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-25T10:00:58.703 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-25T10:00:58.704 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.705 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-25T10:00:58.706 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.707 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:00:58.708 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-25T10:00:58.709 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-25T10:00:58.710 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: mds cache max size = 1000 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: mds log max segments = 1 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = True 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:00:58.711 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:00:58.727 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-25T10:00:58.728 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-25T10:00:58.729 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-25T10:00:58.730 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on skipped message = False 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-25T10:00:58.731 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:00:58.732 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 20 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 20 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 10 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:00:58.733 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:00:58.734 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:00:58.735 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: mon op complaint time = 120 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:00:58.736 INFO:teuthology.orchestra.run.smithi165.stdout: client mount timeout = 600 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: debug client = 20 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = "" 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.d] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout:[mds] 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds = 20 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds balancer = 20 2024-04-25T10:00:58.737 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: mds cache max size = 1000 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug frag = True 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug scatterstat = True 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: mds log max segments = 1 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: mds op complaint time = 180 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: mds verify scatter = True 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:00:58.738 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:00:58.763 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-25T10:00:58.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-25T10:00:58.899 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-25T10:00:58.899 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:58.899 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-25T10:00:58.950 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:58.950 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-25T10:00:58.998 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:00:58.998 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:58.998 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:00:58.998 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:59.170 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:00:59.170 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:59.267 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:00:59.267 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:59.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-25T10:00:59.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-25T10:00:59.366 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:00:59.366 DEBUG:teuthology.orchestra.run.smithi172:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:00:59.449 INFO:tasks.ceph:Setting up mon nodes... 2024-04-25T10:00:59.450 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-25T10:00:59.450 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:00:59.617 INFO:teuthology.orchestra.run.smithi165.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-25T10:00:59.627 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:00:59.751 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-25T10:00:59.759 INFO:tasks.ceph:Setting up mds nodes... 2024-04-25T10:00:59.760 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:00:59.925 INFO:teuthology.orchestra.run.smithi165.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-04-25T10:00:59.935 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-04-25T10:01:00.020 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:01:00.176 INFO:teuthology.orchestra.run.smithi165.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-04-25T10:01:00.185 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-04-25T10:01:00.264 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:01:00.411 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-04-25T10:01:00.418 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-04-25T10:01:00.483 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:01:00.598 INFO:teuthology.orchestra.run.smithi172.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-04-25T10:01:00.605 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-04-25T10:01:00.668 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-25T10:01:00.668 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-25T10:01:00.792 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-25T10:01:00.835 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-04-25T10:01:00.948 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-04-25T10:01:01.001 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-25T10:01:01.121 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-25T10:01:01.154 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-04-25T10:01:01.263 INFO:teuthology.orchestra.run.smithi172.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-04-25T10:01:01.300 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-25T10:01:01.301 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi172.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-25T10:01:01.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:01.374 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:01.374 INFO:tasks.ceph:role: osd.0 2024-04-25T10:01:01.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:01:01.375 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:01.446 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:01.869 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-25T10:01:01.878 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-25T10:01:01.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:02.057 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:02.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:02.238 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:02.238 INFO:tasks.ceph:role: osd.1 2024-04-25T10:01:02.238 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:01:02.238 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:01:02.321 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:02.322 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:02.676 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-25T10:01:02.682 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-25T10:01:02.682 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:02.796 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:02.882 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:02.960 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:02.960 INFO:tasks.ceph:role: osd.2 2024-04-25T10:01:02.960 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:01:02.960 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:03.049 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:03.438 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-25T10:01:03.447 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-25T10:01:03.447 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:03.606 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:03.700 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:03.784 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:03.784 INFO:tasks.ceph:role: osd.3 2024-04-25T10:01:03.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:01:03.784 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:01:03.870 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:03.871 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:04.282 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-25T10:01:04.289 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-25T10:01:04.290 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:04.424 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:04.502 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:04.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:04.631+0000 7f6126010640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-25T10:01:04.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:04.631+0000 7f6126010640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-25T10:01:04.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:04.631+0000 7f6126010640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:01:04.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:04.632+0000 7f6126010640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:01:04.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:04.632+0000 7f6126010640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-25T10:01:04.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:04.635+0000 7f6126010640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-25T10:01:07.840 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-25T10:01:07.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:08.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:08.050+0000 7fb87f61b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-25T10:01:08.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:08.051+0000 7fb87f61b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-25T10:01:08.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:08.051+0000 7fb87f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:08.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:08.051+0000 7fb87f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:08.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:08.051+0000 7fb87f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-25T10:01:08.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:08.053+0000 7fb87f61b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-25T10:01:11.032 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-25T10:01:11.115 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:11.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:11.237+0000 7fd05d61c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-25T10:01:11.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:11.237+0000 7fd05d61c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-25T10:01:11.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:11.237+0000 7fd05d61c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:11.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:11.237+0000 7fd05d61c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:11.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:11.237+0000 7fd05d61c640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-25T10:01:11.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:11.239+0000 7fd05d61c640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-25T10:01:14.221 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-25T10:01:14.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:14.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:14.417+0000 7ff538f33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-25T10:01:14.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:14.417+0000 7ff538f33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-25T10:01:14.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:14.417+0000 7ff538f33640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:14.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:14.417+0000 7ff538f33640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:14.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:14.417+0000 7ff538f33640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-25T10:01:14.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:14.420+0000 7ff538f33640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-25T10:01:17.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-25T10:01:17.703 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:17.799 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:17.799 INFO:tasks.ceph:role: osd.4 2024-04-25T10:01:17.799 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:01:17.799 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:17.887 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:18.291 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-25T10:01:18.299 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-25T10:01:18.299 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:18.445 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:18.528 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:18.599 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:18.599 INFO:tasks.ceph:role: osd.5 2024-04-25T10:01:18.599 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:01:18.599 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-25T10:01:18.673 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:18.673 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:18.673 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:18.673 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:18.674 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:19.028 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-25T10:01:19.033 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-25T10:01:19.033 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:19.124 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:19.191 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:19.253 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:19.253 INFO:tasks.ceph:role: osd.6 2024-04-25T10:01:19.253 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:01:19.253 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-25T10:01:19.327 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:19.328 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:19.712 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-25T10:01:19.718 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-25T10:01:19.719 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:19.838 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:19.918 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:19.995 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-25T10:01:19.995 INFO:tasks.ceph:role: osd.7 2024-04-25T10:01:19.995 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:01:19.996 DEBUG:teuthology.orchestra.run.smithi172:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-25T10:01:20.081 INFO:teuthology.orchestra.run.smithi172.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout: = sunit=0 swidth=0 blks 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-25T10:01:20.082 INFO:teuthology.orchestra.run.smithi172.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-25T10:01:20.499 INFO:teuthology.orchestra.run.smithi172.stdout:Discarding blocks...Done. 2024-04-25T10:01:20.505 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi172.front.sepia.ceph.com -o noatime 2024-04-25T10:01:20.505 DEBUG:teuthology.orchestra.run.smithi172:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:20.673 DEBUG:teuthology.orchestra.run.smithi172:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:20.764 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:20.905 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:20.900+0000 7f6274632640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-25T10:01:20.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:20.901+0000 7f6274632640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-25T10:01:20.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:20.901+0000 7f6274632640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:20.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:20.901+0000 7f6274632640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:20.906 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:20.901+0000 7f6274632640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-25T10:01:20.909 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:20.905+0000 7f6274632640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-25T10:01:23.873 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-25T10:01:23.962 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:24.096 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:24.091+0000 7f9c24a13640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-25T10:01:24.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:24.092+0000 7f9c24a13640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-25T10:01:24.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:24.092+0000 7f9c24a13640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:24.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:24.092+0000 7f9c24a13640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:24.097 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:24.092+0000 7f9c24a13640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-25T10:01:24.099 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:24.094+0000 7f9c24a13640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-25T10:01:26.101 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:26.109 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:27.079 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-25T10:01:27.164 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:27.290 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:27.286+0000 7f034c617640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-25T10:01:27.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:27.286+0000 7f034c617640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-25T10:01:27.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:27.286+0000 7f034c617640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:27.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:27.286+0000 7f034c617640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:27.291 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:27.286+0000 7f034c617640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-25T10:01:27.293 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:27.289+0000 7f034c617640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-25T10:01:30.518 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-25T10:01:30.604 DEBUG:teuthology.orchestra.run.smithi172:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:30.729 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:30.726+0000 7f15ae523640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-25T10:01:30.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:30.726+0000 7f15ae523640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-25T10:01:30.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:30.726+0000 7f15ae523640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:30.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:30.726+0000 7f15ae523640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:30.730 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:30.726+0000 7f15ae523640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-25T10:01:30.732 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:30.729+0000 7f15ae523640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-25T10:01:33.670 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-25T10:01:33.759 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-25T10:01:33.759 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:33.759 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-25T10:01:33.847 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:33.847 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-04-25T10:01:33.934 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:33.934 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-04-25T10:01:34.018 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:34.018 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-25T10:01:34.094 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:34.094 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-25T10:01:34.221 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:34.221 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-25T10:01:34.304 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:34.304 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-25T10:01:34.385 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.385 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-25T10:01:34.474 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.474 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-04-25T10:01:34.561 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.561 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-04-25T10:01:34.646 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.646 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-25T10:01:34.725 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-25T10:01:34.804 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.804 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-25T10:01:34.883 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:34.883 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-25T10:01:34.962 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:34.962 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-25T10:01:35.019 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:35.020 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-04-25T10:01:35.071 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:35.071 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-25T10:01:35.131 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:35.131 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-04-25T10:01:35.182 INFO:tasks.ceph:Adding keys to all mons... 2024-04-25T10:01:35.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:35.189 DEBUG:teuthology.orchestra.run.smithi172:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr.y] 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw== 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ== 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w== 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.0] 2024-04-25T10:01:35.266 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.1] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.2] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.3] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr.x] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.d] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.4] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ== 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.5] 2024-04-25T10:01:35.267 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g== 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.6] 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA== 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout:[osd.7] 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw== 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout:[client.2] 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg== 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout:[client.1] 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw== 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout:[client.3] 2024-04-25T10:01:35.268 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.y] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.0] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.1] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.2] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g== 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.3] 2024-04-25T10:01:35.269 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr.x] 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.4] 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.5] 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.6] 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA== 2024-04-25T10:01:35.270 INFO:teuthology.orchestra.run.smithi172.stdout:[osd.7] 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw== 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout:[client.0] 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout:[client.2] 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg== 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout:[client.1] 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw== 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout:[client.3] 2024-04-25T10:01:35.271 INFO:teuthology.orchestra.run.smithi172.stdout: key = AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ== 2024-04-25T10:01:35.273 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:35.321 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:35.457 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:35.463 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:35.599 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:35.606 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:35.741 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:35.746 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:35.874 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:35.879 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.012 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.017 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.149 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.154 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.289 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:36.295 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-25T10:01:36.432 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:36.439 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:36.575 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:36.582 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-04-25T10:01:36.718 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.855 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.861 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.993 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:36.998 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:37.129 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:37.137 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-25T10:01:37.267 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.273 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.407 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.414 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.548 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.556 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.689 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.694 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-25T10:01:37.836 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-25T10:01:37.836 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:37.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:38.076 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-25T10:01:38.148 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:38.232 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:38.371 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-25T10:01:38.450 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:38.536 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-25T10:01:38.673 DEBUG:teuthology.orchestra.run.smithi172:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-25T10:01:38.749 DEBUG:teuthology.orchestra.run.smithi165:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:38.756 DEBUG:teuthology.orchestra.run.smithi172:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-25T10:01:38.807 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-25T10:01:38.808 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-25T10:01:38.808 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-25T10:01:38.813 INFO:tasks.ceph.mon.a:Started 2024-04-25T10:01:38.813 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-25T10:01:38.813 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-25T10:01:38.818 INFO:tasks.ceph.mon.b:Started 2024-04-25T10:01:38.818 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-25T10:01:38.818 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-25T10:01:38.823 INFO:tasks.ceph.mon.c:Started 2024-04-25T10:01:38.823 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-25T10:01:38.823 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-25T10:01:38.823 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-25T10:01:38.828 INFO:tasks.ceph.mgr.y:Started 2024-04-25T10:01:38.828 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-25T10:01:38.828 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-25T10:01:38.832 INFO:tasks.ceph.mgr.x:Started 2024-04-25T10:01:38.833 INFO:tasks.ceph:Setting crush tunables to default 2024-04-25T10:01:38.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd crush tunables default 2024-04-25T10:01:39.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:38.999+0000 7f9143800640 1 Processor -- start 2024-04-25T10:01:39.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:38.999+0000 7f9143800640 1 -- start start 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f9143800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 0x7f913c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f9143800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c06cda0 con 0x7f913c0d32f0 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f9143800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f913c15e000 con 0x7f913c161710 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f9143800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f913c0d1de0 con 0x7f913c15dc20 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f913be00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f913c161710 0x7f913c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:33544/0 (socket says 172.21.15.165:33544) 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f913be00640 1 -- 172.21.15.165:0/3195662052 learned_addr learned my addr 172.21.15.165:0/3195662052 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.001+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 0x7f913c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.002+0000 7f9140e00640 1 -- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 msgr2=0x7f913c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.002+0000 7f9140e00640 1 -- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 msgr2=0x7f913c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.002+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 0x7f913c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-25T10:01:39.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.002+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 0x7f913c0ce670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-25T10:01:39.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1600253719 0 0) 0x7f913c06cda0 con 0x7f913c0d32f0 2024-04-25T10:01:39.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9124003520 con 0x7f913c0d32f0 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613486365 0 0) 0x7f9124003520 con 0x7f913c0d32f0 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 msgr2=0x7f913c0ce670 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 --2- 172.21.15.165:0/3195662052 >> v2:172.21.15.172:3300/0 conn(0x7f913c15dc20 0x7f913c0ce670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 >> v1:172.21.15.172:6789/0 conn(0x7f913c161710 legacy=0x7f913c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c161c60 con 0x7f913c0d32f0 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f91340019f0 con 0x7f913c0d32f0 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 >> v1:172.21.15.165:6789/0 conn(0x7f913c0d32f0 legacy=0x7f913c06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f9124003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9124004800 0x7f9124004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9124005210 0x7f912400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0d1de0 con 0x7f9124004800 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912400c160 con 0x7f9124005210 2024-04-25T10:01:39.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912400c360 con 0x7f9124003a00 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.020+0000 7f9141800640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9124005210 0x7f912400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f9124003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f913be00640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9124004800 0x7f9124004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9140e00640 1 -- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9124005210 msgr2=0x7f912400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9124005210 0x7f912400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9140e00640 1 -- 172.21.15.165:0/3195662052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9124004800 msgr2=0x7f9124004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9124004800 0x7f9124004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9140e00640 1 -- 172.21.15.165:0/3195662052 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9124003520 con 0x7f9124003a00 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f913be00640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9124004800 0x7f9124004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9141800640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9124005210 0x7f912400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:39.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.021+0000 7f9141800640 1 -- 172.21.15.165:0/3195662052 reap_dead start 2024-04-25T10:01:39.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.022+0000 7f9140e00640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f9124003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f913000b9e0 tx=0x7f9124005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6b317f6550b8de2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.023+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9130030070 con 0x7f9124003a00 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.023+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913002ada0 con 0x7f9124003a00 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.023+0000 7f913b400640 1 -- 172.21.15.165:0/3195662052 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913000b5e0 con 0x7f9124003a00 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 -- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 msgr2=0x7f9124003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f9124003e20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f913000b9e0 tx=0x7f9124005830 comp rx=0 tx=0).stop 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 -- 172.21.15.165:0/3195662052 shutdown_connections 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 --2- 172.21.15.165:0/3195662052 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f9124003e20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 -- 172.21.15.165:0/3195662052 >> 172.21.15.165:0/3195662052 conn(0x7f913c0d0420 msgr2=0x7f913c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 -- 172.21.15.165:0/3195662052 shutdown_connections 2024-04-25T10:01:39.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.024+0000 7f9143800640 1 -- 172.21.15.165:0/3195662052 wait complete. 2024-04-25T10:01:39.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.027+0000 7f9143800640 1 Processor -- start 2024-04-25T10:01:39.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.027+0000 7f9143800640 1 -- start start 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9143800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f913c160aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9143800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c152f30 0x7f913c1f7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9143800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 0x7f913c15a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9143800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c162500 con 0x7f913c1f7590 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9143800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913c162220 con 0x7f913c152f30 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9143800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913c0b5e50 con 0x7f9124003a00 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 0x7f913c15a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 0x7f913c15a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37798/0 (socket says 172.21.15.165:37798) 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 -- 172.21.15.165:0/3044215279 learned_addr learned my addr 172.21.15.165:0/3044215279 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f913be00640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c152f30 0x7f913c1f7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9140e00640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f913c160aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:39.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 -- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 msgr2=0x7f913c160aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f913c160aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 -- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c152f30 msgr2=0x7f913c1f7050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c152f30 0x7f913c1f7050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.028+0000 7f9141800640 1 -- 172.21.15.165:0/3044215279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f913c1f7bc0 con 0x7f913c1f7590 2024-04-25T10:01:39.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.029+0000 7f9141800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 0x7f913c15a930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b6570 tx=0x7f913400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134017030 con 0x7f913c1f7590 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913400fba0 con 0x7f913c1f7590 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913400c0e0 con 0x7f913c1f7590 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f913c15ffc0 con 0x7f913c1f7590 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f913c160580 con 0x7f913c1f7590 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f913401a070 con 0x7f913c1f7590 2024-04-25T10:01:39.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.030+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9134005410 con 0x7f913c1f7590 2024-04-25T10:01:39.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.031+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f913c0d16b0 con 0x7f913c1f7590 2024-04-25T10:01:39.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.034+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f9134022020 con 0x7f913c1f7590 2024-04-25T10:01:39.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.106+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f913c0739e0 con 0x7f913c1f7590 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.109+0000 7f9138c00640 1 -- 172.21.15.165:0/3044215279 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f9134003e20 con 0x7f913c1f7590 2024-04-25T10:01:39.113 INFO:teuthology.orchestra.run.smithi165.stderr:adjusted tunables profile to default 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.110+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 msgr2=0x7f913c15a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.110+0000 7f9143800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 0x7f913c15a930 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f913c0b6570 tx=0x7f913400f5a0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.110+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 shutdown_connections 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.111+0000 7f9143800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f913c1f7590 0x7f913c15a930 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.111+0000 7f9143800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f913c152f30 0x7f913c1f7050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.111+0000 7f9143800640 1 --2- 172.21.15.165:0/3044215279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9124003a00 0x7f913c160aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.111+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 >> 172.21.15.165:0/3044215279 conn(0x7f913c0d0420 msgr2=0x7f913c0b7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.111+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 shutdown_connections 2024-04-25T10:01:39.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:39.111+0000 7f9143800640 1 -- 172.21.15.165:0/3044215279 wait complete. 2024-04-25T10:01:39.132 INFO:tasks.ceph:check_enable_crimson: False 2024-04-25T10:01:39.133 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-25T10:01:39.133 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:39.133 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-25T10:01:39.205 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:39.205 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-25T10:01:39.313 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:39.314 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-25T10:01:39.317 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:39.313+0000 7f64d7b30040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:39.332 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:39.328+0000 7fae53b61040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-25T10:01:39.379 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:01:39.379 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-25T10:01:39.490 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:39.490 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-25T10:01:39.511 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:39.506+0000 7f64d7b30040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:39.539 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:39.534+0000 7fae53b61040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:39.567 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:39.567 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-25T10:01:39.571 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:39.567+0000 7f64d7b30040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:39.608 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:39.602+0000 7fae53b61040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-25T10:01:39.688 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:39.688 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-25T10:01:39.800 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:01:39.800 DEBUG:teuthology.orchestra.run.smithi172:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-25T10:01:39.856 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:39.852+0000 7f64d7b30040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:39.909 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:39.905+0000 7f64d7b30040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:39.918 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:39.914+0000 7fae53b61040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-25T10:01:39.920 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 1aedd4b5-8325-4232-9d73-f008e988fa3a 0 2024-04-25T10:01:39.975 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:39.971+0000 7fae53b61040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-25T10:01:40.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.140+0000 7efe4ac00640 1 Processor -- start 2024-04-25T10:01:40.145 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.140+0000 7efe4ac00640 1 -- start start 2024-04-25T10:01:40.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe4ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe440cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe4ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efe4406bff0 0x7efe440b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe4ac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efe440b5970 0x7efe44161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe4ac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4406d000 con 0x7efe440b5970 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe4ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4415e110 con 0x7efe4406bff0 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe4ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440b2fd0 con 0x7efe4415dd30 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe48e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efe4406bff0 0x7efe440b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe49800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe440cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe48e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efe4406bff0 0x7efe440b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53388/0 (socket says 172.21.15.172:53388) 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe48e00640 1 -- 172.21.15.172:0/3662157062 learned_addr learned my addr 172.21.15.172:0/3662157062 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.142+0000 7efe49800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe440cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:44566/0 (socket says 172.21.15.172:44566) 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe4a200640 1 --2- 172.21.15.172:0/3662157062 >> v2:172.21.15.165:3300/0 conn(0x7efe440b5970 0x7efe44161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe48e00640 1 -- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 msgr2=0x7efe440cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe48e00640 1 --2- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe440cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe48e00640 1 -- 172.21.15.172:0/3662157062 >> v2:172.21.15.165:3300/0 conn(0x7efe440b5970 msgr2=0x7efe44161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe48e00640 1 --2- 172.21.15.172:0/3662157062 >> v2:172.21.15.165:3300/0 conn(0x7efe440b5970 0x7efe44161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe48e00640 1 -- 172.21.15.172:0/3662157062 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4406c480 con 0x7efe4406bff0 2024-04-25T10:01:40.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.143+0000 7efe49800640 1 --2- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe440cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.144+0000 7efe48e00640 1 --2- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4406bff0 0x7efe440b5430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe4408bd20 tx=0x7efe3002a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=627b4bd58f73d97a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.144+0000 7efe43e00640 1 -- 172.21.15.172:0/3662157062 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30037070 con 0x7efe4406bff0 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.144+0000 7efe43e00640 1 -- 172.21.15.172:0/3662157062 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe30032440 con 0x7efe4406bff0 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.144+0000 7efe43e00640 1 -- 172.21.15.172:0/3662157062 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30032810 con 0x7efe4406bff0 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 -- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4406bff0 msgr2=0x7efe440b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4406bff0 0x7efe440b5430 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe4408bd20 tx=0x7efe3002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 -- 172.21.15.172:0/3662157062 shutdown_connections 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3662157062 >> v2:172.21.15.165:3300/0 conn(0x7efe440b5970 0x7efe44161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.150 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe4406bff0 0x7efe440b5430 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3662157062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe440cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 -- 172.21.15.172:0/3662157062 >> 172.21.15.172:0/3662157062 conn(0x7efe440d0610 msgr2=0x7efe440d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 -- 172.21.15.172:0/3662157062 shutdown_connections 2024-04-25T10:01:40.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.145+0000 7efe4ac00640 1 -- 172.21.15.172:0/3662157062 wait complete. 2024-04-25T10:01:40.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.146+0000 7efe4ac00640 1 Processor -- start 2024-04-25T10:01:40.151 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.147+0000 7efe4ac00640 1 -- start start 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.147+0000 7efe4ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe440b5970 0x7efe44157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.147+0000 7efe4ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe44152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe4ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe441537f0 0x7efe440b6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe4ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4406cd20 con 0x7efe440b5970 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe4ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4406ca40 con 0x7efe441537f0 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe4ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe440cf110 con 0x7efe4415dd30 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe44152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe4a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe441537f0 0x7efe440b6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe44152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:44572/0 (socket says 172.21.15.172:44572) 2024-04-25T10:01:40.153 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 -- 172.21.15.172:0/3113137747 learned_addr learned my addr 172.21.15.172:0/3113137747 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe49800640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe440b5970 0x7efe44157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 -- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe441537f0 msgr2=0x7efe440b6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe441537f0 0x7efe440b6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 -- 172.21.15.172:0/3113137747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe440b5970 msgr2=0x7efe44157610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe440b5970 0x7efe44157610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe48e00640 1 -- 172.21.15.172:0/3113137747 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44157d80 con 0x7efe4415dd30 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.148+0000 7efe49800640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe440b5970 0x7efe44157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:40.154 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe48e00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe44152fd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe300046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30041070 con 0x7efe4415dd30 2024-04-25T10:01:40.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3003c870 con 0x7efe4415dd30 2024-04-25T10:01:40.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe44158060 con 0x7efe4415dd30 2024-04-25T10:01:40.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe30037040 con 0x7efe4415dd30 2024-04-25T10:01:40.155 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe44158620 con 0x7efe4415dd30 2024-04-25T10:01:40.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7efe30044070 con 0x7efe4415dd30 2024-04-25T10:01:40.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.149+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7efe30046d60 con 0x7efe4415dd30 2024-04-25T10:01:40.156 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.150+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe44073990 con 0x7efe4415dd30 2024-04-25T10:01:40.159 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.155+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efe30046450 con 0x7efe4415dd30 2024-04-25T10:01:40.238 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.234+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1aedd4b5-8325-4232-9d73-f008e988fa3a", "id": 0} v 0) v1 -- 0x7efe440d16c0 con 0x7efe4415dd30 2024-04-25T10:01:40.245 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.241+0000 7efe41600640 1 -- 172.21.15.172:0/3113137747 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1aedd4b5-8325-4232-9d73-f008e988fa3a", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7efe3003cc60 con 0x7efe4415dd30 2024-04-25T10:01:40.246 INFO:teuthology.orchestra.run.smithi172.stdout:0 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 msgr2=0x7efe44152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe44152fd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe30000c00 tx=0x7efe300046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 shutdown_connections 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe441537f0 0x7efe440b6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe4415dd30 0x7efe44152fd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 --2- 172.21.15.172:0/3113137747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe440b5970 0x7efe44157610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 >> 172.21.15.172:0/3113137747 conn(0x7efe440d0610 msgr2=0x7efe440b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 shutdown_connections 2024-04-25T10:01:40.248 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.243+0000 7efe4ac00640 1 -- 172.21.15.172:0/3113137747 wait complete. 2024-04-25T10:01:40.263 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new e91c7f9f-4fa9-468f-b6e2-afd7c09b8886 1 2024-04-25T10:01:40.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.479+0000 7f32eb200640 1 Processor -- start 2024-04-25T10:01:40.484 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.480+0000 7f32eb200640 1 -- start start 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32eb200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f32ec099cb0 0x7f32ec06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32eb200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32ec0d1120 0x7f32ec0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32eb200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ec0d0080 con 0x7f32ec09a090 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32eb200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec06c4b0 con 0x7f32ec099cb0 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32eb200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ec0cfe30 con 0x7f32ec0d1120 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32e3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32ec0d1120 0x7f32ec0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32e3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32ec0d1120 0x7f32ec0cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53402/0 (socket says 172.21.15.172:53402) 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.481+0000 7f32e3e00640 1 -- 172.21.15.172:0/1811063095 learned_addr learned my addr 172.21.15.172:0/1811063095 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.482+0000 7f32e9e00640 1 --2- 172.21.15.172:0/1811063095 >> v2:172.21.15.165:3300/0 conn(0x7f32ec099cb0 0x7f32ec06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.482+0000 7f32e3e00640 1 -- 172.21.15.172:0/1811063095 >> v2:172.21.15.165:3300/0 conn(0x7f32ec099cb0 msgr2=0x7f32ec06bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.482+0000 7f32e3e00640 1 --2- 172.21.15.172:0/1811063095 >> v2:172.21.15.165:3300/0 conn(0x7f32ec099cb0 0x7f32ec06bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.487 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.482+0000 7f32e3e00640 1 -- 172.21.15.172:0/1811063095 >> v1:172.21.15.172:6789/0 conn(0x7f32ec09a090 legacy=0x7f32ec15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.482+0000 7f32e3e00640 1 -- 172.21.15.172:0/1811063095 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0d0360 con 0x7f32ec0d1120 2024-04-25T10:01:40.488 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.482+0000 7f32e9e00640 1 --2- 172.21.15.172:0/1811063095 >> v2:172.21.15.165:3300/0 conn(0x7f32ec099cb0 0x7f32ec06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.483+0000 7f32e3e00640 1 --2- 172.21.15.172:0/1811063095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec0d1120 0x7f32ec0cf8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32ec072830 tx=0x7f32d402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69df0e85beea37ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.483+0000 7f32e9400640 1 -- 172.21.15.172:0/1811063095 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4037070 con 0x7f32ec0d1120 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.483+0000 7f32e9400640 1 -- 172.21.15.172:0/1811063095 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d402ae50 con 0x7f32ec0d1120 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.483+0000 7f32e9400640 1 -- 172.21.15.172:0/1811063095 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4032860 con 0x7f32ec0d1120 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 -- 172.21.15.172:0/1811063095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec0d1120 msgr2=0x7f32ec0cf8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 --2- 172.21.15.172:0/1811063095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec0d1120 0x7f32ec0cf8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32ec072830 tx=0x7f32d402a300 comp rx=0 tx=0).stop 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 -- 172.21.15.172:0/1811063095 shutdown_connections 2024-04-25T10:01:40.489 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 --2- 172.21.15.172:0/1811063095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec0d1120 0x7f32ec0cf8f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 --2- 172.21.15.172:0/1811063095 >> v2:172.21.15.165:3300/0 conn(0x7f32ec099cb0 0x7f32ec06bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 -- 172.21.15.172:0/1811063095 >> 172.21.15.172:0/1811063095 conn(0x7f32ec0b3250 msgr2=0x7f32ec0b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 -- 172.21.15.172:0/1811063095 shutdown_connections 2024-04-25T10:01:40.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.484+0000 7f32eb200640 1 -- 172.21.15.172:0/1811063095 wait complete. 2024-04-25T10:01:40.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.486+0000 7f32eb200640 1 Processor -- start 2024-04-25T10:01:40.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.486+0000 7f32eb200640 1 -- start start 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32eb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ec099cb0 0x7f32ec0cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32eb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 0x7f32ec0c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32eb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ec0d1120 0x7f32ec0ca350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32eb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0d0c00 con 0x7f32ec0d1120 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32eb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec0d0920 con 0x7f32ec09a090 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32eb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ec15de10 con 0x7f32ec099cb0 2024-04-25T10:01:40.493 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 0x7f32ec0c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e9e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ec099cb0 0x7f32ec0cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 0x7f32ec0c9e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53416/0 (socket says 172.21.15.172:53416) 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 -- 172.21.15.172:0/2413281606 learned_addr learned my addr 172.21.15.172:0/2413281606 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 -- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ec099cb0 msgr2=0x7f32ec0cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32ea800640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ec0d1120 0x7f32ec0ca350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ec099cb0 0x7f32ec0cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 -- 172.21.15.172:0/2413281606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ec0d1120 msgr2=0x7f32ec0ca350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ec0d1120 0x7f32ec0ca350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32e3e00640 1 -- 172.21.15.172:0/2413281606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ec0cc170 con 0x7f32ec09a090 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.487+0000 7f32ea800640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ec0d1120 0x7f32ec0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32e3e00640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 0x7f32ec0c9e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f32d402a340 tx=0x7f32d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4041070 con 0x7f32ec09a090 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d403c890 con 0x7f32ec09a090 2024-04-25T10:01:40.494 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d4037040 con 0x7f32ec09a090 2024-04-25T10:01:40.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ec0cc450 con 0x7f32ec09a090 2024-04-25T10:01:40.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ec0cca10 con 0x7f32ec09a090 2024-04-25T10:01:40.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f32d4044070 con 0x7f32ec09a090 2024-04-25T10:01:40.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.488+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f32d403d7a0 con 0x7f32ec09a090 2024-04-25T10:01:40.495 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.489+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32ec0b4010 con 0x7f32ec09a090 2024-04-25T10:01:40.496 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.492+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f32d403cc80 con 0x7f32ec09a090 2024-04-25T10:01:40.575 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.570+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e91c7f9f-4fa9-468f-b6e2-afd7c09b8886", "id": 1} v 0) v1 -- 0x7f32ec0cf8b0 con 0x7f32ec09a090 2024-04-25T10:01:40.582 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.577+0000 7f32e2000640 1 -- 172.21.15.172:0/2413281606 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e91c7f9f-4fa9-468f-b6e2-afd7c09b8886", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f32d4046c20 con 0x7f32ec09a090 2024-04-25T10:01:40.582 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 msgr2=0x7f32ec0c9e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 0x7f32ec0c9e10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f32d402a340 tx=0x7f32d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 shutdown_connections 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ec0d1120 0x7f32ec0ca350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ec09a090 0x7f32ec0c9e10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 --2- 172.21.15.172:0/2413281606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ec099cb0 0x7f32ec0cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.583 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 >> 172.21.15.172:0/2413281606 conn(0x7f32ec0b3250 msgr2=0x7f32ec06cae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 shutdown_connections 2024-04-25T10:01:40.584 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.578+0000 7f32eb200640 1 -- 172.21.15.172:0/2413281606 wait complete. 2024-04-25T10:01:40.600 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new ff3dc377-50c0-4312-a25f-e4148ffbc34f 2 2024-04-25T10:01:40.777 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.772+0000 7f5c6fe00640 1 Processor -- start 2024-04-25T10:01:40.778 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.772+0000 7f5c6fe00640 1 -- start start 2024-04-25T10:01:40.779 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5c680ced80 0x7f5c680d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c6815e000 con 0x7f5c6815dc20 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c680d1de0 con 0x7f5c68161710 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c68161c60 con 0x7f5c680ced80 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6de00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f5c68161710 0x7f5c680d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43178/0 (socket says 172.21.15.172:43178) 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5c680ced80 0x7f5c680d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c6de00640 1 -- 172.21.15.172:0/3749165443 learned_addr learned my addr 172.21.15.172:0/3749165443 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.775+0000 7f5c6ca00640 1 -- 172.21.15.172:0/3749165443 >> v1:172.21.15.172:6789/0 conn(0x7f5c68161710 legacy=0x7f5c680d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.774+0000 7f5c67e00640 1 -- 172.21.15.172:0/3749165443 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439740405 0 0) 0x7f5c680d1de0 con 0x7f5c68161710 2024-04-25T10:01:40.780 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.775+0000 7f5c6ca00640 1 -- 172.21.15.172:0/3749165443 >> v1:172.21.15.165:6789/0 conn(0x7f5c6815dc20 legacy=0x7f5c680ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.781 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.775+0000 7f5c6ca00640 1 -- 172.21.15.172:0/3749165443 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68161f40 con 0x7f5c680ced80 2024-04-25T10:01:40.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.776+0000 7f5c6ca00640 1 --2- 172.21.15.172:0/3749165443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c680d22e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097a0 tx=0x7f5c5402a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95890deadbbfabdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.783 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.776+0000 7f5c67e00640 1 -- 172.21.15.172:0/3749165443 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54037070 con 0x7f5c680ced80 2024-04-25T10:01:40.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.776+0000 7f5c67e00640 1 -- 172.21.15.172:0/3749165443 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c54032440 con 0x7f5c680ced80 2024-04-25T10:01:40.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.776+0000 7f5c67e00640 1 -- 172.21.15.172:0/3749165443 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c54032810 con 0x7f5c680ced80 2024-04-25T10:01:40.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3749165443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 msgr2=0x7f5c680d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.784 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 --2- 172.21.15.172:0/3749165443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c680d22e0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5c540097a0 tx=0x7f5c5402a230 comp rx=0 tx=0).stop 2024-04-25T10:01:40.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3749165443 shutdown_connections 2024-04-25T10:01:40.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 --2- 172.21.15.172:0/3749165443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c680d22e0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3749165443 >> 172.21.15.172:0/3749165443 conn(0x7f5c680d0420 msgr2=0x7f5c680d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3749165443 shutdown_connections 2024-04-25T10:01:40.785 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.777+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3749165443 wait complete. 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.779+0000 7f5c6fe00640 1 Processor -- start 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.779+0000 7f5c6fe00640 1 -- start start 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c681f4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6815dc20 0x7f5c6815fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c68161710 0x7f5c68154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:40.791 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b5e50 con 0x7f5c68161710 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c68162500 con 0x7f5c680ced80 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c680b5fd0 con 0x7f5c6815dc20 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c681f4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c681f4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53438/0 (socket says 172.21.15.172:53438) 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6d400640 1 -- 172.21.15.172:0/3217098429 learned_addr learned my addr 172.21.15.172:0/3217098429 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.780+0000 7f5c6ca00640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6815dc20 0x7f5c6815fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6de00640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c68161710 0x7f5c68154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6d400640 1 -- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6815dc20 msgr2=0x7f5c6815fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6d400640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6815dc20 0x7f5c6815fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6d400640 1 -- 172.21.15.172:0/3217098429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c68161710 msgr2=0x7f5c68154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6d400640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c68161710 0x7f5c68154a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6d400640 1 -- 172.21.15.172:0/3217098429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c681603f0 con 0x7f5c680ced80 2024-04-25T10:01:40.792 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6d400640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c681f4350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c480 tx=0x7f5c5c00c940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0052d0 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c5c005470 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c681606d0 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.781+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c012870 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.782+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c68160c90 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.782+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f5c5c012a10 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.782+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f5c5c012d00 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.783+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c68073a00 con 0x7f5c680ced80 2024-04-25T10:01:40.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.785+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f5c5c0035f0 con 0x7f5c680ced80 2024-04-25T10:01:40.863 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:40.858+0000 7f64d7b30040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:40.868 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.864+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ff3dc377-50c0-4312-a25f-e4148ffbc34f", "id": 2} v 0) v1 -- 0x7f5c680d16b0 con 0x7f5c680ced80 2024-04-25T10:01:40.875 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.871+0000 7f5c65600640 1 -- 172.21.15.172:0/3217098429 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ff3dc377-50c0-4312-a25f-e4148ffbc34f", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5c5c0038b0 con 0x7f5c680ced80 2024-04-25T10:01:40.876 INFO:teuthology.orchestra.run.smithi172.stdout:2 2024-04-25T10:01:40.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 msgr2=0x7f5c681f4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:40.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c681f4350 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c00c480 tx=0x7f5c5c00c940 comp rx=0 tx=0).stop 2024-04-25T10:01:40.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 shutdown_connections 2024-04-25T10:01:40.877 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c68161710 0x7f5c68154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c6815dc20 0x7f5c6815fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 --2- 172.21.15.172:0/3217098429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c680ced80 0x7f5c681f4350 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:40.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.872+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 >> 172.21.15.172:0/3217098429 conn(0x7f5c680d0420 msgr2=0x7f5c680d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:40.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.873+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 shutdown_connections 2024-04-25T10:01:40.878 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:40.873+0000 7f5c6fe00640 1 -- 172.21.15.172:0/3217098429 wait complete. 2024-04-25T10:01:40.894 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new f8d3e5bf-8745-4d91-9248-9e088389dc1a 3 2024-04-25T10:01:40.930 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:40.925+0000 7fae53b61040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-25T10:01:40.968 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:40.964+0000 7f64d7b30040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:41.019 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:41.015+0000 7f64d7b30040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:41.041 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:41.036+0000 7fae53b61040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:41.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.060+0000 7fafcfe00640 1 Processor -- start 2024-04-25T10:01:41.065 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.061+0000 7fafcfe00640 1 -- start start 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.062+0000 7fafcfe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fafd00b95c0 0x7fafd00b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.062+0000 7fafcfe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd0161e70 con 0x7fafd015dbd0 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.062+0000 7fafcfe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafd0162150 con 0x7fafd00b5910 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.062+0000 7fafcfe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafd00d1ce0 con 0x7fafd00b95c0 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.062+0000 7fafcf400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fafd00b5910 0x7fafd00d2170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43190/0 (socket says 172.21.15.172:43190) 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.062+0000 7fafcf400640 1 -- 172.21.15.172:0/263348201 learned_addr learned my addr 172.21.15.172:0/263348201 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-25T10:01:41.067 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafc7e00640 1 --2- 172.21.15.172:0/263348201 >> v2:172.21.15.165:3300/0 conn(0x7fafd00b95c0 0x7fafd00b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafce000640 1 -- 172.21.15.172:0/263348201 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3706221216 0 0) 0x7fafd0162150 con 0x7fafd00b5910 2024-04-25T10:01:41.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafce000640 1 -- 172.21.15.172:0/263348201 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafb4003520 con 0x7fafd00b5910 2024-04-25T10:01:41.068 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafce000640 1 -- 172.21.15.172:0/263348201 <== mon.1 v1:172.21.15.172:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4148597107 0 0) 0x7fafb4003520 con 0x7fafd00b5910 2024-04-25T10:01:41.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafc7e00640 1 -- 172.21.15.172:0/263348201 >> v1:172.21.15.172:6789/0 conn(0x7fafd00b5910 legacy=0x7fafd00d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafc7e00640 1 -- 172.21.15.172:0/263348201 >> v1:172.21.15.165:6789/0 conn(0x7fafd015dbd0 legacy=0x7fafd0161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.063+0000 7fafc7e00640 1 -- 172.21.15.172:0/263348201 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0162430 con 0x7fafd00b95c0 2024-04-25T10:01:41.069 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.064+0000 7fafc7e00640 1 --2- 172.21.15.172:0/263348201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd00b53d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009720 tx=0x7fafb802a150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23b2f45664c34ac6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.065+0000 7fafce000640 1 -- 172.21.15.172:0/263348201 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8037070 con 0x7fafd00b95c0 2024-04-25T10:01:41.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.065+0000 7fafce000640 1 -- 172.21.15.172:0/263348201 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb802ac60 con 0x7fafd00b95c0 2024-04-25T10:01:41.070 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.065+0000 7fafce000640 1 -- 172.21.15.172:0/263348201 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc80019f0 con 0x7fafd00b95c0 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.065+0000 7fafcfe00640 1 -- 172.21.15.172:0/263348201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 msgr2=0x7fafd00b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.065+0000 7fafcfe00640 1 --2- 172.21.15.172:0/263348201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd00b53d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fafb8009720 tx=0x7fafb802a150 comp rx=0 tx=0).stop 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.066+0000 7fafcfe00640 1 -- 172.21.15.172:0/263348201 shutdown_connections 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.066+0000 7fafcfe00640 1 --2- 172.21.15.172:0/263348201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd00b53d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.066+0000 7fafcfe00640 1 -- 172.21.15.172:0/263348201 >> 172.21.15.172:0/263348201 conn(0x7fafd00d03d0 msgr2=0x7fafd00d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.066+0000 7fafcfe00640 1 -- 172.21.15.172:0/263348201 shutdown_connections 2024-04-25T10:01:41.071 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.066+0000 7fafcfe00640 1 -- 172.21.15.172:0/263348201 wait complete. 2024-04-25T10:01:41.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.067+0000 7fafcfe00640 1 Processor -- start 2024-04-25T10:01:41.072 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.068+0000 7fafcfe00640 1 -- start start 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.068+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 0x7fafd0156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.068+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd0152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafd015dbd0 0x7fafd00c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00d2a00 con 0x7fafd00b95c0 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00d2880 con 0x7fafd00b5910 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafd00d2b80 con 0x7fafd015dbd0 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 0x7fafd0156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.075 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafd015dbd0 0x7fafd00c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 0x7fafd0156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53444/0 (socket says 172.21.15.172:53444) 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 -- 172.21.15.172:0/4245253304 learned_addr learned my addr 172.21.15.172:0/4245253304 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafc7e00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd0152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 -- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafd015dbd0 msgr2=0x7fafd00c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafd015dbd0 0x7fafd00c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 -- 172.21.15.172:0/4245253304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 msgr2=0x7fafd0152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd0152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.069+0000 7fafcea00640 1 -- 172.21.15.172:0/4245253304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd0161080 con 0x7fafd00b5910 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafcea00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 0x7fafd0156040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c3c0 tx=0x7fafc000c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc00052d0 con 0x7fafd00b5910 2024-04-25T10:01:41.076 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc0005490 con 0x7fafd00b5910 2024-04-25T10:01:41.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc0007260 con 0x7fafd00b5910 2024-04-25T10:01:41.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd01568d0 con 0x7fafd00b5910 2024-04-25T10:01:41.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd0160880 con 0x7fafd00b5910 2024-04-25T10:01:41.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fafc0007400 con 0x7fafd00b5910 2024-04-25T10:01:41.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.070+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fafc0011590 con 0x7fafd00b5910 2024-04-25T10:01:41.077 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.071+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd00d1520 con 0x7fafd00b5910 2024-04-25T10:01:41.078 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.073+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fafc00039f0 con 0x7fafd00b5910 2024-04-25T10:01:41.095 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:41.091+0000 7fae53b61040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-25T10:01:41.157 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.153+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f8d3e5bf-8745-4d91-9248-9e088389dc1a", "id": 3} v 0) v1 -- 0x7fafd00b8db0 con 0x7fafd00b5910 2024-04-25T10:01:41.164 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.160+0000 7fafc6a00640 1 -- 172.21.15.172:0/4245253304 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f8d3e5bf-8745-4d91-9248-9e088389dc1a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fafc0003c50 con 0x7fafd00b5910 2024-04-25T10:01:41.165 INFO:teuthology.orchestra.run.smithi172.stdout:3 2024-04-25T10:01:41.166 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 msgr2=0x7fafd0156040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 0x7fafd0156040 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fafc000c3c0 tx=0x7fafc000c880 comp rx=0 tx=0).stop 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 shutdown_connections 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafd015dbd0 0x7fafd00c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fafd00b95c0 0x7fafd0152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 --2- 172.21.15.172:0/4245253304 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafd00b5910 0x7fafd0156040 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.161+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 >> 172.21.15.172:0/4245253304 conn(0x7fafd00d03d0 msgr2=0x7fafd006bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.162+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 shutdown_connections 2024-04-25T10:01:41.167 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.162+0000 7fafcfe00640 1 -- 172.21.15.172:0/4245253304 wait complete. 2024-04-25T10:01:41.182 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 8fc4af7f-9417-4389-8c94-5a457e3ab6d4 4 2024-04-25T10:01:41.214 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:41.210+0000 7f64d7b30040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:41.270 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:41.266+0000 7f64d7b30040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:41.296 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:41.292+0000 7fae53b61040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-25T10:01:41.331 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:41.327+0000 7f64d7b30040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:41.355 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:41.350+0000 7fae53b61040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-25T10:01:41.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.394+0000 7f2447200640 1 Processor -- start 2024-04-25T10:01:41.402 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.394+0000 7f2447200640 1 -- start start 2024-04-25T10:01:41.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2447200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f244015dbd0 0x7f2440161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.404 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2447200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24400b5ae0 0x7f24400d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2447200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2440161e70 con 0x7f24400b95c0 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2447200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24400d1ce0 con 0x7f244015dbd0 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2447200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2440162150 con 0x7f24400b5ae0 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2445e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f244015dbd0 0x7f2440161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2446800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24400b5ae0 0x7f24400d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2445e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f244015dbd0 0x7f2440161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53452/0 (socket says 172.21.15.172:53452) 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2445400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f24400b95c0 0x7f24400b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43204/0 (socket says 172.21.15.172:43204) 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2445400640 1 -- 172.21.15.172:0/1841016565 learned_addr learned my addr 172.21.15.172:0/1841016565 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.396+0000 7f2446800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24400b5ae0 0x7f24400d2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:44602/0 (socket says 172.21.15.172:44602) 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.397+0000 7f2445e00640 1 -- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24400b5ae0 msgr2=0x7f24400d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.397+0000 7f2445e00640 1 --2- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24400b5ae0 0x7f24400d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.405 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.397+0000 7f2445e00640 1 -- 172.21.15.172:0/1841016565 >> v1:172.21.15.172:6789/0 conn(0x7f24400b95c0 legacy=0x7f24400b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.397+0000 7f2444a00640 1 -- 172.21.15.172:0/1841016565 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4198963377 0 0) 0x7f2440161e70 con 0x7f24400b95c0 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.397+0000 7f2445e00640 1 -- 172.21.15.172:0/1841016565 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2440162430 con 0x7f244015dbd0 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.398+0000 7f2445e00640 1 --2- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f244015dbd0 0x7f2440161620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24340097a0 tx=0x7f243402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b065cccf1561468 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.398+0000 7f2444a00640 1 -- 172.21.15.172:0/1841016565 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434037070 con 0x7f244015dbd0 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.398+0000 7f2444a00640 1 -- 172.21.15.172:0/1841016565 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2434032440 con 0x7f244015dbd0 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.398+0000 7f2444a00640 1 -- 172.21.15.172:0/1841016565 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2434032810 con 0x7f244015dbd0 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 -- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f244015dbd0 msgr2=0x7f2440161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 --2- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f244015dbd0 0x7f2440161620 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f24340097a0 tx=0x7f243402a180 comp rx=0 tx=0).stop 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 -- 172.21.15.172:0/1841016565 shutdown_connections 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 --2- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24400b5ae0 0x7f24400d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 --2- 172.21.15.172:0/1841016565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f244015dbd0 0x7f2440161620 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 -- 172.21.15.172:0/1841016565 >> 172.21.15.172:0/1841016565 conn(0x7f24400d03d0 msgr2=0x7f24400d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.406 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 -- 172.21.15.172:0/1841016565 shutdown_connections 2024-04-25T10:01:41.407 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.399+0000 7f2447200640 1 -- 172.21.15.172:0/1841016565 wait complete. 2024-04-25T10:01:41.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.401+0000 7f2447200640 1 Processor -- start 2024-04-25T10:01:41.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.401+0000 7f2447200640 1 -- start start 2024-04-25T10:01:41.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2447200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24400b5ae0 0x7f2440151aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.409 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2447200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 0x7f24401604e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2447200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f244015dbd0 0x7f2440160a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2447200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24400d26b0 con 0x7f24400b5ae0 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2447200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2440161b60 con 0x7f24400b95c0 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2447200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24400d2830 con 0x7f244015dbd0 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 0x7f24401604e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 0x7f24401604e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53456/0 (socket says 172.21.15.172:53456) 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 -- 172.21.15.172:0/3069051563 learned_addr learned my addr 172.21.15.172:0/3069051563 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2446800640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f244015dbd0 0x7f2440160a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445e00640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24400b5ae0 0x7f2440151aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 -- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f244015dbd0 msgr2=0x7f2440160a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f244015dbd0 0x7f2440160a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 -- 172.21.15.172:0/3069051563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24400b5ae0 msgr2=0x7f2440151aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.410 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24400b5ae0 0x7f2440151aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445400640 1 -- 172.21.15.172:0/3069051563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24400c8350 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.402+0000 7f2445e00640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24400b5ae0 0x7f2440151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f2445400640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 0x7f24401604e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f243000c4d0 tx=0x7f243000c990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430018070 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2430007500 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24400c8630 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2430004a20 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24400c8bf0 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f243001b070 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.403+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f2430004bc0 con 0x7f24400b95c0 2024-04-25T10:01:41.411 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.404+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24400b6020 con 0x7f24400b95c0 2024-04-25T10:01:41.412 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.406+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f24400b6020 con 0x7f24400b95c0 2024-04-25T10:01:41.421 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:41.416+0000 7fae53b61040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-25T10:01:41.457 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:41.452+0000 7f64d7b30040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:41.490 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.485+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "8fc4af7f-9417-4389-8c94-5a457e3ab6d4", "id": 4} v 0) v1 -- 0x7f24400d1520 con 0x7f24400b95c0 2024-04-25T10:01:41.498 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.494+0000 7f243a000640 1 -- 172.21.15.172:0/3069051563 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8fc4af7f-9417-4389-8c94-5a457e3ab6d4", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f243001d440 con 0x7f24400b95c0 2024-04-25T10:01:41.499 INFO:teuthology.orchestra.run.smithi172.stdout:4 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 msgr2=0x7f24401604e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 0x7f24401604e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f243000c4d0 tx=0x7f243000c990 comp rx=0 tx=0).stop 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 shutdown_connections 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f244015dbd0 0x7f2440160a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24400b95c0 0x7f24401604e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 --2- 172.21.15.172:0/3069051563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24400b5ae0 0x7f2440151aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 >> 172.21.15.172:0/3069051563 conn(0x7f24400d03d0 msgr2=0x7f24400d3950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 shutdown_connections 2024-04-25T10:01:41.500 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.495+0000 7f2447200640 1 -- 172.21.15.172:0/3069051563 wait complete. 2024-04-25T10:01:41.516 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 9c66e050-67b9-48ca-b29b-4f3620e3cd10 5 2024-04-25T10:01:41.554 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:41.550+0000 7fae53b61040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-25T10:01:41.701 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.697+0000 7f7043e00640 1 Processor -- start 2024-04-25T10:01:41.702 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.697+0000 7f7043e00640 1 -- start start 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.698+0000 7f7043e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f704415dbd0 0x7f7044161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.698+0000 7f7043e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 0x7f70440b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.698+0000 7f7043e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7044161e70 con 0x7f70440b5910 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.698+0000 7f7043e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70440d1ce0 con 0x7f704415dbd0 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.698+0000 7f7043e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7044162150 con 0x7f70440b95c0 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 0x7f70440b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 0x7f70440b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:44614/0 (socket says 172.21.15.172:44614) 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 -- 172.21.15.172:0/1427915645 learned_addr learned my addr 172.21.15.172:0/1427915645 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042a00640 1 --2- 172.21.15.172:0/1427915645 >> v2:172.21.15.165:3300/0 conn(0x7f704415dbd0 0x7f7044161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 -- 172.21.15.172:0/1427915645 >> v2:172.21.15.165:3300/0 conn(0x7f704415dbd0 msgr2=0x7f7044161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 --2- 172.21.15.172:0/1427915645 >> v2:172.21.15.165:3300/0 conn(0x7f704415dbd0 0x7f7044161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 -- 172.21.15.172:0/1427915645 >> v1:172.21.15.165:6789/0 conn(0x7f70440b5910 legacy=0x7f704406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:41.704 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.699+0000 7f7042000640 1 -- 172.21.15.172:0/1427915645 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7044162430 con 0x7f70440b95c0 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.700+0000 7f7042000640 1 --2- 172.21.15.172:0/1427915645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 0x7f70440b53d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f702c009870 tx=0x7f702c02a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc1349467c15b41c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.701+0000 7f7041600640 1 -- 172.21.15.172:0/1427915645 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c037070 con 0x7f70440b95c0 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.701+0000 7f7041600640 1 -- 172.21.15.172:0/1427915645 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c032440 con 0x7f70440b95c0 2024-04-25T10:01:41.705 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.701+0000 7f7041600640 1 -- 172.21.15.172:0/1427915645 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c032810 con 0x7f70440b95c0 2024-04-25T10:01:41.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.701+0000 7f7043e00640 1 -- 172.21.15.172:0/1427915645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 msgr2=0x7f70440b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.701+0000 7f7043e00640 1 --2- 172.21.15.172:0/1427915645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 0x7f70440b53d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f702c009870 tx=0x7f702c02a200 comp rx=0 tx=0).stop 2024-04-25T10:01:41.706 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.702+0000 7f7043e00640 1 -- 172.21.15.172:0/1427915645 shutdown_connections 2024-04-25T10:01:41.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.702+0000 7f7043e00640 1 --2- 172.21.15.172:0/1427915645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b95c0 0x7f70440b53d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.702+0000 7f7043e00640 1 --2- 172.21.15.172:0/1427915645 >> v2:172.21.15.165:3300/0 conn(0x7f704415dbd0 0x7f7044161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.702+0000 7f7043e00640 1 -- 172.21.15.172:0/1427915645 >> 172.21.15.172:0/1427915645 conn(0x7f70440d03d0 msgr2=0x7f70440d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.702+0000 7f7043e00640 1 -- 172.21.15.172:0/1427915645 shutdown_connections 2024-04-25T10:01:41.707 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.702+0000 7f7043e00640 1 -- 172.21.15.172:0/1427915645 wait complete. 2024-04-25T10:01:41.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.703+0000 7f7043e00640 1 Processor -- start 2024-04-25T10:01:41.708 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.703+0000 7f7043e00640 1 -- start start 2024-04-25T10:01:41.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7043e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 0x7f70441f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.710 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7043e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70440b95c0 0x7f70440c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7043e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f704415dbd0 0x7f70440c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7043e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70440d3180 con 0x7f704415dbd0 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7043e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7044161b60 con 0x7f70440b95c0 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7043e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70440d3300 con 0x7f70440b5910 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.704+0000 7f7042a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 0x7f70441f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 0x7f70441f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:44624/0 (socket says 172.21.15.172:44624) 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 -- 172.21.15.172:0/3593757749 learned_addr learned my addr 172.21.15.172:0/3593757749 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042000640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70440b95c0 0x7f70440c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7043400640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f704415dbd0 0x7f70440c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 -- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70440b95c0 msgr2=0x7f70440c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70440b95c0 0x7f70440c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 -- 172.21.15.172:0/3593757749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f704415dbd0 msgr2=0x7f70440c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.711 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f704415dbd0 0x7f70440c9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 -- 172.21.15.172:0/3593757749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70441608f0 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7043400640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f704415dbd0 0x7f70440c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.705+0000 7f7042a00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 0x7f70441f2e30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f703400d800 tx=0x7f703400dcc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400b6c0 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7034019070 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70440cc630 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f703400ba80 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70440ccbf0 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f703400bc20 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.706+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f703401e9e0 con 0x7f70440b5910 2024-04-25T10:01:41.712 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.707+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70440d1520 con 0x7f70440b5910 2024-04-25T10:01:41.714 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.710+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f703401ebe0 con 0x7f70440b5910 2024-04-25T10:01:41.793 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.788+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c66e050-67b9-48ca-b29b-4f3620e3cd10", "id": 5} v 0) v1 -- 0x7f704415e000 con 0x7f70440b5910 2024-04-25T10:01:41.800 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.796+0000 7f703aa00640 1 -- 172.21.15.172:0/3593757749 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c66e050-67b9-48ca-b29b-4f3620e3cd10", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f703401e460 con 0x7f70440b5910 2024-04-25T10:01:41.801 INFO:teuthology.orchestra.run.smithi172.stdout:5 2024-04-25T10:01:41.802 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.797+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 msgr2=0x7f70441f2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.797+0000 7f7043e00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 0x7f70441f2e30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f703400d800 tx=0x7f703400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 shutdown_connections 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f704415dbd0 0x7f70440c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70440b95c0 0x7f70440c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 --2- 172.21.15.172:0/3593757749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70440b5910 0x7f70441f2e30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 >> 172.21.15.172:0/3593757749 conn(0x7f70440d03d0 msgr2=0x7f70440d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 shutdown_connections 2024-04-25T10:01:41.803 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:41.798+0000 7f7043e00640 1 -- 172.21.15.172:0/3593757749 wait complete. 2024-04-25T10:01:41.819 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new 640aa33e-e6c9-4b58-8101-5bd88c38fbc8 6 2024-04-25T10:01:41.914 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:41.910+0000 7f64d7b30040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:01:42.040 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:42.036+0000 7fae53b61040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-25T10:01:42.048 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.044+0000 7f4974a00640 1 Processor -- start 2024-04-25T10:01:42.049 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.044+0000 7f4974a00640 1 -- start start 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.045+0000 7f4974a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f497015dc20 0x7f49700ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.045+0000 7f4974a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49700d32f0 0x7f49700cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.045+0000 7f4974a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f497015e000 con 0x7f4970161710 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.045+0000 7f4974a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49700d1de0 con 0x7f497015dc20 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.045+0000 7f4974a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4970161c60 con 0x7f49700d32f0 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f4967e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f4970161710 0x7f497006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.172:43218/0 (socket says 172.21.15.172:43218) 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f4967e00640 1 -- 172.21.15.172:0/1946290301 learned_addr learned my addr 172.21.15.172:0/1946290301 (peer_addr_for_me v1:172.21.15.172:0/0) 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496e800640 1 --2- 172.21.15.172:0/1946290301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49700d32f0 0x7f49700cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496de00640 1 --2- 172.21.15.172:0/1946290301 >> v2:172.21.15.165:3300/0 conn(0x7f497015dc20 0x7f49700ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496d400640 1 -- 172.21.15.172:0/1946290301 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959269193 0 0) 0x7f497015e000 con 0x7f4970161710 2024-04-25T10:01:42.051 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496e800640 1 -- 172.21.15.172:0/1946290301 >> v2:172.21.15.165:3300/0 conn(0x7f497015dc20 msgr2=0x7f49700ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496e800640 1 --2- 172.21.15.172:0/1946290301 >> v2:172.21.15.165:3300/0 conn(0x7f497015dc20 0x7f49700ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496e800640 1 -- 172.21.15.172:0/1946290301 >> v1:172.21.15.172:6789/0 conn(0x7f4970161710 legacy=0x7f497006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.052 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.046+0000 7f496e800640 1 -- 172.21.15.172:0/1946290301 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970161f40 con 0x7f49700d32f0 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.047+0000 7f496e800640 1 --2- 172.21.15.172:0/1946290301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49700d32f0 0x7f49700cebb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4968009d90 tx=0x7f496802a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ec5a50608956f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.047+0000 7f496d400640 1 -- 172.21.15.172:0/1946290301 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968007740 con 0x7f49700d32f0 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.047+0000 7f496d400640 1 -- 172.21.15.172:0/1946290301 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49680078e0 con 0x7f49700d32f0 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f496d400640 1 -- 172.21.15.172:0/1946290301 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968007cb0 con 0x7f49700d32f0 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 -- 172.21.15.172:0/1946290301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49700d32f0 msgr2=0x7f49700cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.053 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 --2- 172.21.15.172:0/1946290301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49700d32f0 0x7f49700cebb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4968009d90 tx=0x7f496802a660 comp rx=0 tx=0).stop 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 -- 172.21.15.172:0/1946290301 shutdown_connections 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 --2- 172.21.15.172:0/1946290301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49700d32f0 0x7f49700cebb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 --2- 172.21.15.172:0/1946290301 >> v2:172.21.15.165:3300/0 conn(0x7f497015dc20 0x7f49700ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 -- 172.21.15.172:0/1946290301 >> 172.21.15.172:0/1946290301 conn(0x7f49700d0420 msgr2=0x7f49700d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 -- 172.21.15.172:0/1946290301 shutdown_connections 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.048+0000 7f4974a00640 1 -- 172.21.15.172:0/1946290301 wait complete. 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.050+0000 7f4974a00640 1 Processor -- start 2024-04-25T10:01:42.054 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.050+0000 7f4974a00640 1 -- start start 2024-04-25T10:01:42.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4974a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 0x7f4970160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4974a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f497015dc20 0x7f49700d23a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4974a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4970161710 0x7f497014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4974a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b5e50 con 0x7f4970161710 2024-04-25T10:01:42.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4974a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970162500 con 0x7f49700d32f0 2024-04-25T10:01:42.057 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4974a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49700b5fd0 con 0x7f497015dc20 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f496de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 0x7f4970160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f4967e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f497015dc20 0x7f49700d23a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f496de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 0x7f4970160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53468/0 (socket says 172.21.15.172:53468) 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.051+0000 7f496de00640 1 -- 172.21.15.172:0/368777281 learned_addr learned my addr 172.21.15.172:0/368777281 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496e800640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4970161710 0x7f497014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496de00640 1 -- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f497015dc20 msgr2=0x7f49700d23a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496de00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f497015dc20 0x7f49700d23a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496de00640 1 -- 172.21.15.172:0/368777281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4970161710 msgr2=0x7f497014e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496de00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4970161710 0x7f497014e850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496de00640 1 -- 172.21.15.172:0/368777281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49701f7260 con 0x7f49700d32f0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496e800640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4970161710 0x7f497014e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f496de00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 0x7f4970160b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f496000d5f0 tx=0x7f496000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49600125f0 con 0x7f49700d32f0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960019070 con 0x7f49700d32f0 2024-04-25T10:01:42.058 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49600129d0 con 0x7f49700d32f0 2024-04-25T10:01:42.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49701f7540 con 0x7f49700d32f0 2024-04-25T10:01:42.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.052+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49701f7b00 con 0x7f49700d32f0 2024-04-25T10:01:42.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.053+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f4960012b70 con 0x7f49700d32f0 2024-04-25T10:01:42.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.053+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f496001ea50 con 0x7f49700d32f0 2024-04-25T10:01:42.059 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.054+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49700cfc20 con 0x7f49700d32f0 2024-04-25T10:01:42.060 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.056+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f49700cfc20 con 0x7f49700d32f0 2024-04-25T10:01:42.139 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.134+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "640aa33e-e6c9-4b58-8101-5bd88c38fbc8", "id": 6} v 0) v1 -- 0x7f4970073a00 con 0x7f49700d32f0 2024-04-25T10:01:42.147 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.142+0000 7f4966000640 1 -- 172.21.15.172:0/368777281 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "640aa33e-e6c9-4b58-8101-5bd88c38fbc8", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f496001e460 con 0x7f49700d32f0 2024-04-25T10:01:42.147 INFO:teuthology.orchestra.run.smithi172.stdout:6 2024-04-25T10:01:42.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.143+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 msgr2=0x7f4970160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.143+0000 7f4974a00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 0x7f4970160b40 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f496000d5f0 tx=0x7f496000dab0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.148 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 shutdown_connections 2024-04-25T10:01:42.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4970161710 0x7f497014e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f497015dc20 0x7f49700d23a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 --2- 172.21.15.172:0/368777281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49700d32f0 0x7f4970160b40 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 >> 172.21.15.172:0/368777281 conn(0x7f49700d0420 msgr2=0x7f49700b4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 shutdown_connections 2024-04-25T10:01:42.149 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.144+0000 7f4974a00640 1 -- 172.21.15.172:0/368777281 wait complete. 2024-04-25T10:01:42.165 DEBUG:teuthology.orchestra.run.smithi172:> sudo ceph --cluster ceph osd new d1f4680d-75ad-4400-822c-95b0e6898501 7 2024-04-25T10:01:42.210 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:42.206+0000 7f64d7b30040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:42.271 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:42.267+0000 7f64d7b30040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:42.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.335+0000 7f00a5000640 1 Processor -- start 2024-04-25T10:01:42.340 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.336+0000 7f00a5000640 1 -- start start 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f00a5000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f00a0161890 0x7f00a00b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f00a5000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00a00b54a0 0x7f00a00b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f00a5000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a00cf1d0 con 0x7f00a0161240 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f00a5000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a00b7120 con 0x7f00a0161890 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f00a5000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a00b5e30 con 0x7f00a00b54a0 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f009e800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00a00b54a0 0x7f00a00b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f009e800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00a00b54a0 0x7f00a00b58f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53472/0 (socket says 172.21.15.172:53472) 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.337+0000 7f009e800640 1 -- 172.21.15.172:0/3826872822 learned_addr learned my addr 172.21.15.172:0/3826872822 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:42.342 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.338+0000 7f0097e00640 1 --2- 172.21.15.172:0/3826872822 >> v2:172.21.15.165:3300/0 conn(0x7f00a0161890 0x7f00a00b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.338+0000 7f009e800640 1 -- 172.21.15.172:0/3826872822 >> v2:172.21.15.165:3300/0 conn(0x7f00a0161890 msgr2=0x7f00a00b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.338+0000 7f009e800640 1 --2- 172.21.15.172:0/3826872822 >> v2:172.21.15.165:3300/0 conn(0x7f00a0161890 0x7f00a00b6be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.338+0000 7f009e800640 1 -- 172.21.15.172:0/3826872822 >> v1:172.21.15.165:6789/0 conn(0x7f00a0161240 legacy=0x7f00a00ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.338+0000 7f009e800640 1 -- 172.21.15.172:0/3826872822 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a0161620 con 0x7f00a00b54a0 2024-04-25T10:01:42.343 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.339+0000 7f009e800640 1 --2- 172.21.15.172:0/3826872822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a00b54a0 0x7f00a00b58f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00a008ce10 tx=0x7f009802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e772630e6fdd6b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.339+0000 7f009d400640 1 -- 172.21.15.172:0/3826872822 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0098036070 con 0x7f00a00b54a0 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.339+0000 7f009d400640 1 -- 172.21.15.172:0/3826872822 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0098031600 con 0x7f00a00b54a0 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.339+0000 7f009d400640 1 -- 172.21.15.172:0/3826872822 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00980319d0 con 0x7f00a00b54a0 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 -- 172.21.15.172:0/3826872822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a00b54a0 msgr2=0x7f00a00b58f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 --2- 172.21.15.172:0/3826872822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a00b54a0 0x7f00a00b58f0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f00a008ce10 tx=0x7f009802a730 comp rx=0 tx=0).stop 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 -- 172.21.15.172:0/3826872822 shutdown_connections 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 --2- 172.21.15.172:0/3826872822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a00b54a0 0x7f00a00b58f0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 --2- 172.21.15.172:0/3826872822 >> v2:172.21.15.165:3300/0 conn(0x7f00a0161890 0x7f00a00b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 -- 172.21.15.172:0/3826872822 >> 172.21.15.172:0/3826872822 conn(0x7f00a00d04a0 msgr2=0x7f00a00d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 -- 172.21.15.172:0/3826872822 shutdown_connections 2024-04-25T10:01:42.345 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.340+0000 7f00a5000640 1 -- 172.21.15.172:0/3826872822 wait complete. 2024-04-25T10:01:42.346 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.342+0000 7f00a5000640 1 Processor -- start 2024-04-25T10:01:42.347 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.342+0000 7f00a5000640 1 -- start start 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f00a5000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00a00b54a0 0x7f00a014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f00a5000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 0x7f00a00c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.349 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f00a5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a0161890 0x7f00a00c9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f00a5000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a01623c0 con 0x7f00a0161890 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f00a5000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a01620e0 con 0x7f00a0161240 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f00a5000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a01626a0 con 0x7f00a00b54a0 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 0x7f00a00c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f009de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00a00b54a0 0x7f00a014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 0x7f00a00c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.172:53478/0 (socket says 172.21.15.172:53478) 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f009de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00a00b54a0 0x7f00a014ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.172:44654/0 (socket says 172.21.15.172:44654) 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 -- 172.21.15.172:0/1335484580 learned_addr learned my addr 172.21.15.172:0/1335484580 (peer_addr_for_me v2:172.21.15.172:0/0) 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f009e800640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a0161890 0x7f00a00c9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 -- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00a00b54a0 msgr2=0x7f00a014ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00a00b54a0 0x7f00a014ed40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 -- 172.21.15.172:0/1335484580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a0161890 msgr2=0x7f00a00c9640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.350 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a0161890 0x7f00a00c9640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.343+0000 7f0097e00640 1 -- 172.21.15.172:0/1335484580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a00c7f80 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f0097e00640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 0x7f00a00c7550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f008800e870 tx=0x7f008800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0088017070 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f008800c940 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a00c8260 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008800cd30 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a00c8820 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.344+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f008801a070 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.345+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f0088006020 con 0x7f00a0161240 2024-04-25T10:01:42.351 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.345+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a008b580 con 0x7f00a0161240 2024-04-25T10:01:42.352 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.347+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f00a008b580 con 0x7f00a0161240 2024-04-25T10:01:42.353 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:42.349+0000 7fae53b61040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-25T10:01:42.403 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:42.399+0000 7f64d7b30040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:01:42.416 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:42.412+0000 7fae53b61040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-25T10:01:42.432 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.428+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd new", "uuid": "d1f4680d-75ad-4400-822c-95b0e6898501", "id": 7} v 0) v1 -- 0x7f00a0073a00 con 0x7f00a0161240 2024-04-25T10:01:42.440 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.435+0000 7f0096000640 1 -- 172.21.15.172:0/1335484580 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "d1f4680d-75ad-4400-822c-95b0e6898501", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f00880064d0 con 0x7f00a0161240 2024-04-25T10:01:42.440 INFO:teuthology.orchestra.run.smithi172.stdout:7 2024-04-25T10:01:42.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 msgr2=0x7f00a00c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 0x7f00a00c7550 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f008800e870 tx=0x7f008800ed30 comp rx=0 tx=0).stop 2024-04-25T10:01:42.441 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 shutdown_connections 2024-04-25T10:01:42.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00a0161890 0x7f00a00c9640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00a0161240 0x7f00a00c7550 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 --2- 172.21.15.172:0/1335484580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00a00b54a0 0x7f00a014ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 >> 172.21.15.172:0/1335484580 conn(0x7f00a00d04a0 msgr2=0x7f00a00d1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 shutdown_connections 2024-04-25T10:01:42.442 INFO:teuthology.orchestra.run.smithi172.stderr:2024-04-25T10:01:42.437+0000 7f00a5000640 1 -- 172.21.15.172:0/1335484580 wait complete. 2024-04-25T10:01:42.459 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-25T10:01:42.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-25T10:01:42.467 INFO:tasks.ceph.osd.0:Started 2024-04-25T10:01:42.467 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-25T10:01:42.467 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-25T10:01:42.474 INFO:tasks.ceph.osd.1:Started 2024-04-25T10:01:42.474 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-25T10:01:42.474 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-25T10:01:42.481 INFO:tasks.ceph.osd.2:Started 2024-04-25T10:01:42.481 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-25T10:01:42.482 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-25T10:01:42.488 INFO:tasks.ceph.osd.3:Started 2024-04-25T10:01:42.488 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-25T10:01:42.488 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-25T10:01:42.495 INFO:tasks.ceph.osd.4:Started 2024-04-25T10:01:42.495 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-25T10:01:42.495 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-25T10:01:42.502 INFO:tasks.ceph.osd.5:Started 2024-04-25T10:01:42.502 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-25T10:01:42.502 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-25T10:01:42.508 INFO:tasks.ceph.osd.6:Started 2024-04-25T10:01:42.508 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-25T10:01:42.508 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-25T10:01:42.513 INFO:tasks.ceph.osd.7:Started 2024-04-25T10:01:42.514 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:42.573 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:42.568+0000 7fae53b61040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-25T10:01:42.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.733+0000 7f3503a00640 1 Processor -- start 2024-04-25T10:01:42.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.733+0000 7f3503a00640 1 -- start start 2024-04-25T10:01:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.734+0000 7f3503a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f34fc154ee0 0x7f34fc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.734+0000 7f3503a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34fc0cfcf0 0x7f34fc0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.734+0000 7f3503a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34fc071480 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.734+0000 7f3503a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0ce830 con 0x7f34fc0cfcf0 2024-04-25T10:01:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.734+0000 7f3503a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc072090 con 0x7f34fc154ee0 2024-04-25T10:01:42.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34fc0cfcf0 0x7f34fc0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f3501000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f34fc154ee0 0x7f34fc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34fc0cfcf0 0x7f34fc0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37834/0 (socket says 172.21.15.165:37834) 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 -- 172.21.15.165:0/2582277081 learned_addr learned my addr 172.21.15.165:0/2582277081 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 -- 172.21.15.165:0/2582277081 >> v2:172.21.15.172:3300/0 conn(0x7f34fc154ee0 msgr2=0x7f34fc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 --2- 172.21.15.165:0/2582277081 >> v2:172.21.15.172:3300/0 conn(0x7f34fc154ee0 0x7f34fc0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 -- 172.21.15.165:0/2582277081 >> v1:172.21.15.172:6789/0 conn(0x7f34fc0ce3c0 legacy=0x7f34fc15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.735+0000 7f34fbe00640 1 -- 172.21.15.165:0/2582277081 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc071760 con 0x7f34fc0cfcf0 2024-04-25T10:01:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.737+0000 7f34fbe00640 1 --2- 172.21.15.165:0/2582277081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0cfcf0 0x7f34fc0cde80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0d02e0 tx=0x7f34e802c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d32e6593cd19c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.737+0000 7f34fb400640 1 -- 172.21.15.165:0/2582277081 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8036020 con 0x7f34fc0cfcf0 2024-04-25T10:01:42.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.737+0000 7f34fb400640 1 -- 172.21.15.165:0/2582277081 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e8032640 con 0x7f34fc0cfcf0 2024-04-25T10:01:42.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.737+0000 7f34fb400640 1 -- 172.21.15.165:0/2582277081 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e8032a10 con 0x7f34fc0cfcf0 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 -- 172.21.15.165:0/2582277081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0cfcf0 msgr2=0x7f34fc0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 --2- 172.21.15.165:0/2582277081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0cfcf0 0x7f34fc0cde80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f34fc0d02e0 tx=0x7f34e802c940 comp rx=0 tx=0).stop 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 -- 172.21.15.165:0/2582277081 shutdown_connections 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 --2- 172.21.15.165:0/2582277081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0cfcf0 0x7f34fc0cde80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 --2- 172.21.15.165:0/2582277081 >> v2:172.21.15.172:3300/0 conn(0x7f34fc154ee0 0x7f34fc0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 -- 172.21.15.165:0/2582277081 >> 172.21.15.165:0/2582277081 conn(0x7f34fc06abe0 msgr2=0x7f34fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 -- 172.21.15.165:0/2582277081 shutdown_connections 2024-04-25T10:01:42.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.738+0000 7f3503a00640 1 -- 172.21.15.165:0/2582277081 wait complete. 2024-04-25T10:01:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.741+0000 7f3503a00640 1 Processor -- start 2024-04-25T10:01:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.741+0000 7f3503a00640 1 -- start start 2024-04-25T10:01:42.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3503a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 0x7f34fc14e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3503a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0cfcf0 0x7f34fc1504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3503a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc154ee0 0x7f34fc150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:42.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3503a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc071d20 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3503a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc071230 con 0x7f34fc154ee0 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3503a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc1552c0 con 0x7f34fc0cfcf0 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.742+0000 7f3501000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 0x7f34fc14e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 0x7f34fc14e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37836/0 (socket says 172.21.15.165:37836) 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 -- 172.21.15.165:0/541644734 learned_addr learned my addr 172.21.15.165:0/541644734 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501a00640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc154ee0 0x7f34fc150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f34fbe00640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0cfcf0 0x7f34fc1504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 -- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0cfcf0 msgr2=0x7f34fc1504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0cfcf0 0x7f34fc1504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 -- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc154ee0 msgr2=0x7f34fc150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc154ee0 0x7f34fc150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 -- 172.21.15.165:0/541644734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc1ef560 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.743+0000 7f3501000640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 0x7f34fc14e320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e870 tx=0x7f34f000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0017070 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f000c940 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000cd30 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc1ef820 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1efde0 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f34f001a070 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.744+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f34f0013b40 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.745+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc073990 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.747+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f34f0013d90 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.799 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:42.795+0000 7f64d7b30040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:42.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.811+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34fc06d360 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.812+0000 7f34f8c00640 1 -- 172.21.15.165:0/541644734 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f34f0006520 con 0x7f34fc0ce3c0 2024-04-25T10:01:42.817 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:01:42.817 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":10,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:01:42.431260+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 msgr2=0x7f34fc14e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 0x7f34fc14e320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f34f000e870 tx=0x7f34f000ed30 comp rx=0 tx=0).stop 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 shutdown_connections 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc154ee0 0x7f34fc150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0cfcf0 0x7f34fc1504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 --2- 172.21.15.165:0/541644734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc0ce3c0 0x7f34fc14e320 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 >> 172.21.15.165:0/541644734 conn(0x7f34fc06abe0 msgr2=0x7f34fc06de10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 shutdown_connections 2024-04-25T10:01:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:42.813+0000 7f3503a00640 1 -- 172.21.15.165:0/541644734 wait complete. 2024-04-25T10:01:42.834 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-25T10:01:42.834 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-25T10:01:42.859 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:42.855+0000 7f64d7b30040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:42.922 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:42.917+0000 7fae53b61040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-25T10:01:42.980 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:42.975+0000 7fae53b61040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-25T10:01:43.047 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:43.043+0000 7f64d7b30040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:43.157 INFO:tasks.ceph.osd.0.smithi165.stderr:2024-04-25T10:01:43.153+0000 7f87ee615640 -1 Falling back to public interface 2024-04-25T10:01:43.162 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:43.158+0000 7fae53b61040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-25T10:01:43.164 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-25T10:01:43.160+0000 7fcc63814640 -1 Falling back to public interface 2024-04-25T10:01:43.184 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-04-25T10:01:43.180+0000 7f85d2553640 -1 Falling back to public interface 2024-04-25T10:01:43.186 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-25T10:01:43.182+0000 7fa083e0d640 -1 Falling back to public interface 2024-04-25T10:01:43.190 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-04-25T10:01:43.185+0000 7fd66f1bc640 -1 Falling back to public interface 2024-04-25T10:01:43.210 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-25T10:01:43.206+0000 7fd3ae5fc640 -1 Falling back to public interface 2024-04-25T10:01:43.212 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-04-25T10:01:43.207+0000 7f4bd852c640 -1 Falling back to public interface 2024-04-25T10:01:43.212 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-04-25T10:01:43.208+0000 7fc568c1e640 -1 Falling back to public interface 2024-04-25T10:01:43.218 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:43.214+0000 7f64d7b30040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:43.279 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:43.274+0000 7f64d7b30040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:43.338 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:43.333+0000 7fae53b61040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-25T10:01:43.360 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:43.356+0000 7f64d7b30040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:43.396 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:43.391+0000 7fae53b61040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-25T10:01:43.413 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:43.409+0000 7f64d7b30040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:43.477 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:43.472+0000 7fae53b61040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-25T10:01:43.532 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:43.528+0000 7fae53b61040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-25T10:01:43.639 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:43.811 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.806+0000 7fbd95600640 1 Processor -- start 2024-04-25T10:01:43.811 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.807+0000 7fbd95600640 1 -- start start 2024-04-25T10:01:43.812 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd95600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbd900d2ea0 0x7fbd900d03d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.812 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd95600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbd900d0910 0x7fbd900b6a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd95600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd900b9950 con 0x7fbd9015dd40 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd95600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900d0d50 con 0x7fbd900d0910 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd95600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd900d1e00 con 0x7fbd900d2ea0 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbd900d0910 0x7fbd900b6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbd900d0910 0x7fbd900b6a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37840/0 (socket says 172.21.15.165:37840) 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 -- 172.21.15.165:0/3654070379 learned_addr learned my addr 172.21.15.165:0/3654070379 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbd9015dd40 0x7fbd900b9240 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34356/0 (socket says 172.21.15.165:34356) 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd87e00640 1 --2- 172.21.15.165:0/3654070379 >> v2:172.21.15.172:3300/0 conn(0x7fbd900d2ea0 0x7fbd900d03d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 -- 172.21.15.165:0/3654070379 >> v2:172.21.15.172:3300/0 conn(0x7fbd900d2ea0 msgr2=0x7fbd900d03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 --2- 172.21.15.165:0/3654070379 >> v2:172.21.15.172:3300/0 conn(0x7fbd900d2ea0 0x7fbd900d03d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 -- 172.21.15.165:0/3654070379 >> v1:172.21.15.165:6789/0 conn(0x7fbd9015dd40 legacy=0x7fbd900b9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.813 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.808+0000 7fbd8f400640 1 -- 172.21.15.165:0/3654070379 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd900d20e0 con 0x7fbd900d0910 2024-04-25T10:01:43.814 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.809+0000 7fbd8f400640 1 --2- 172.21.15.165:0/3654070379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b6a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd900b6550 tx=0x7fbd8802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29d7fc959eb031ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd8e000640 1 -- 172.21.15.165:0/3654070379 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd88036070 con 0x7fbd900d0910 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd8e000640 1 -- 172.21.15.165:0/3654070379 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd88031600 con 0x7fbd900d0910 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd8e000640 1 -- 172.21.15.165:0/3654070379 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd880319d0 con 0x7fbd900d0910 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 -- 172.21.15.165:0/3654070379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 msgr2=0x7fbd900b6a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 --2- 172.21.15.165:0/3654070379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b6a50 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbd900b6550 tx=0x7fbd8802a610 comp rx=0 tx=0).stop 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 -- 172.21.15.165:0/3654070379 shutdown_connections 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 --2- 172.21.15.165:0/3654070379 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b6a50 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 --2- 172.21.15.165:0/3654070379 >> v2:172.21.15.172:3300/0 conn(0x7fbd900d2ea0 0x7fbd900d03d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 -- 172.21.15.165:0/3654070379 >> 172.21.15.165:0/3654070379 conn(0x7fbd900728f0 msgr2=0x7fbd9006c170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 -- 172.21.15.165:0/3654070379 shutdown_connections 2024-04-25T10:01:43.815 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.810+0000 7fbd95600640 1 -- 172.21.15.165:0/3654070379 wait complete. 2024-04-25T10:01:43.816 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.812+0000 7fbd95600640 1 Processor -- start 2024-04-25T10:01:43.816 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.812+0000 7fbd95600640 1 -- start start 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd95600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd95600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd900d2ea0 0x7fbd9014af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd95600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd9015dd40 0x7fbd901f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd95600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900d2980 con 0x7fbd900d0910 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd95600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900d26a0 con 0x7fbd900d2ea0 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd95600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd900d1a20 con 0x7fbd9015dd40 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd8ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd8ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37842/0 (socket says 172.21.15.165:37842) 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd8ea00640 1 -- 172.21.15.165:0/3727953527 learned_addr learned my addr 172.21.15.165:0/3727953527 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd87e00640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd900d2ea0 0x7fbd9014af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.819 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.813+0000 7fbd8f400640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd9015dd40 0x7fbd901f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd8ea00640 1 -- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd9015dd40 msgr2=0x7fbd901f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd8ea00640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd9015dd40 0x7fbd901f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd8ea00640 1 -- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd900d2ea0 msgr2=0x7fbd9014af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd8ea00640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd900d2ea0 0x7fbd9014af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd8ea00640 1 -- 172.21.15.165:0/3727953527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd901509c0 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd8ea00640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b5e20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c3f0 tx=0x7fbd8000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd800074e0 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd80007680 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd80004a00 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd90150ca0 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.814+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd900b3200 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.815+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fbd8001a020 con 0x7fbd900d0910 2024-04-25T10:01:43.820 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.815+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fbd80013ad0 con 0x7fbd900d0910 2024-04-25T10:01:43.821 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.815+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9004ef20 con 0x7fbd900d0910 2024-04-25T10:01:43.822 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.818+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fbd80021070 con 0x7fbd900d0910 2024-04-25T10:01:43.890 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.886+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd900c7180 con 0x7fbd900d0910 2024-04-25T10:01:43.891 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.887+0000 7fbd86a00640 1 -- 172.21.15.165:0/3727953527 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7058 (secure 0 0 0) 0x7fbd80013400 con 0x7fbd900d0910 2024-04-25T10:01:43.891 INFO:teuthology.misc.health.smithi165.stdout: 2024-04-25T10:01:43.891 INFO:teuthology.misc.health.smithi165.stdout:{"epoch":10,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:01:42.431260+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 msgr2=0x7fbd900b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b5e20 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbd8000c3f0 tx=0x7fbd8000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 shutdown_connections 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd9015dd40 0x7fbd901f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd900d2ea0 0x7fbd9014af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 --2- 172.21.15.165:0/3727953527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd900d0910 0x7fbd900b5e20 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:43.892 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 >> 172.21.15.165:0/3727953527 conn(0x7fbd900728f0 msgr2=0x7fbd900c8f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:43.893 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 shutdown_connections 2024-04-25T10:01:43.893 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:43.888+0000 7fbd95600640 1 -- 172.21.15.165:0/3727953527 wait complete. 2024-04-25T10:01:43.905 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:01:43.967 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:43.963+0000 7f64d7b30040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:44.074 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:44.070+0000 7fae53b61040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-25T10:01:44.103 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:44.098+0000 7f64d7b30040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:44.216 INFO:tasks.ceph.mgr.x.smithi172.stderr:2024-04-25T10:01:44.211+0000 7fae53b61040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-25T10:01:45.141 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:45.136+0000 7f6478c00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-25T10:01:45.141 INFO:tasks.ceph.mgr.y.smithi165.stderr:2024-04-25T10:01:45.137+0000 7f6478c00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-25T10:01:46.539 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-25T10:01:46.534+0000 7fa083e0d640 -1 osd.3 0 log_to_monitors true 2024-04-25T10:01:46.598 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-25T10:01:46.593+0000 7fd3ae5fc640 -1 osd.2 0 log_to_monitors true 2024-04-25T10:01:46.600 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-04-25T10:01:46.595+0000 7fd66f1bc640 -1 osd.6 0 log_to_monitors true 2024-04-25T10:01:46.675 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-04-25T10:01:46.670+0000 7f4bd852c640 -1 osd.7 0 log_to_monitors true 2024-04-25T10:01:46.788 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-04-25T10:01:46.783+0000 7f85d2553640 -1 osd.5 0 log_to_monitors true 2024-04-25T10:01:46.811 INFO:tasks.ceph.osd.0.smithi165.stderr:2024-04-25T10:01:46.807+0000 7f87ee615640 -1 osd.0 0 log_to_monitors true 2024-04-25T10:01:46.825 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-25T10:01:46.821+0000 7fcc63814640 -1 osd.1 0 log_to_monitors true 2024-04-25T10:01:46.875 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-04-25T10:01:46.870+0000 7fc568c1e640 -1 osd.4 0 log_to_monitors true 2024-04-25T10:01:50.515 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-04-25T10:01:50.511+0000 7f85cde00640 -1 osd.5 0 waiting for initial osdmap 2024-04-25T10:01:50.521 INFO:tasks.ceph.osd.5.smithi172.stderr:2024-04-25T10:01:50.517+0000 7f85c3400640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.529 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-25T10:01:50.525+0000 7fcc5e200640 -1 osd.1 0 waiting for initial osdmap 2024-04-25T10:01:50.539 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-25T10:01:50.534+0000 7fcc54200640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.546 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-04-25T10:01:50.542+0000 7fd66a000640 -1 osd.6 0 waiting for initial osdmap 2024-04-25T10:01:50.550 INFO:tasks.ceph.osd.6.smithi172.stderr:2024-04-25T10:01:50.545+0000 7fd660000640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.552 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-04-25T10:01:50.547+0000 7f4bd3e00640 -1 osd.7 0 waiting for initial osdmap 2024-04-25T10:01:50.559 INFO:tasks.ceph.osd.7.smithi172.stderr:2024-04-25T10:01:50.554+0000 7f4bc9400640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.560 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-25T10:01:50.556+0000 7fd3a9e00640 -1 osd.2 0 waiting for initial osdmap 2024-04-25T10:01:50.563 INFO:tasks.ceph.osd.0.smithi165.stderr:2024-04-25T10:01:50.559+0000 7f87e8e00640 -1 osd.0 0 waiting for initial osdmap 2024-04-25T10:01:50.565 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-25T10:01:50.561+0000 7fd39f400640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.570 INFO:tasks.ceph.osd.0.smithi165.stderr:2024-04-25T10:01:50.566+0000 7f87dee00640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.602 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-04-25T10:01:50.598+0000 7fc563600640 -1 osd.4 0 waiting for initial osdmap 2024-04-25T10:01:50.610 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-25T10:01:50.606+0000 7fa07f000640 -1 osd.3 0 waiting for initial osdmap 2024-04-25T10:01:50.614 INFO:tasks.ceph.osd.4.smithi172.stderr:2024-04-25T10:01:50.609+0000 7fc559600640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.616 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-25T10:01:50.612+0000 7fa074600640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-25T10:01:50.715 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:50.874 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.869+0000 7f89b3e00640 1 Processor -- start 2024-04-25T10:01:50.874 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.869+0000 7f89b3e00640 1 -- start start 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f89b415dc50 0x7f89b406d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89b40cb9b0 0x7f89b406d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89b415e030 con 0x7f89b40b1bc0 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b3e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b4071e00 con 0x7f89b415dc50 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89b40caed0 con 0x7f89b40cb9b0 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f89b415dc50 0x7f89b406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f89b415dc50 0x7f89b406d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50566/0 (socket says 172.21.15.165:50566) 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b2a00640 1 -- 172.21.15.165:0/3399047688 learned_addr learned my addr 172.21.15.165:0/3399047688 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.871+0000 7f89b3400640 1 --2- 172.21.15.165:0/3399047688 >> v2:172.21.15.172:3300/0 conn(0x7f89b40cb9b0 0x7f89b406d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.872+0000 7f89b2a00640 1 -- 172.21.15.165:0/3399047688 >> v2:172.21.15.172:3300/0 conn(0x7f89b40cb9b0 msgr2=0x7f89b406d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.876 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.872+0000 7f89b2a00640 1 --2- 172.21.15.165:0/3399047688 >> v2:172.21.15.172:3300/0 conn(0x7f89b40cb9b0 0x7f89b406d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.877 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.872+0000 7f89b2a00640 1 -- 172.21.15.165:0/3399047688 >> v1:172.21.15.172:6789/0 conn(0x7f89b40b1bc0 legacy=0x7f89b40b3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:50.877 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.872+0000 7f89b2a00640 1 -- 172.21.15.165:0/3399047688 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b40cb1b0 con 0x7f89b415dc50 2024-04-25T10:01:50.877 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.873+0000 7f89b2a00640 1 --2- 172.21.15.165:0/3399047688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b415dc50 0x7f89b406d210 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89b40cd4b0 tx=0x7f89a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7627ece595c5ac48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.877 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.873+0000 7f89b2000640 1 -- 172.21.15.165:0/3399047688 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4037070 con 0x7f89b415dc50 2024-04-25T10:01:50.877 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.873+0000 7f89b2000640 1 -- 172.21.15.165:0/3399047688 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89a4032440 con 0x7f89b415dc50 2024-04-25T10:01:50.877 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.873+0000 7f89b2000640 1 -- 172.21.15.165:0/3399047688 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89a4032810 con 0x7f89b415dc50 2024-04-25T10:01:50.878 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 -- 172.21.15.165:0/3399047688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b415dc50 msgr2=0x7f89b406d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.878 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 --2- 172.21.15.165:0/3399047688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b415dc50 0x7f89b406d210 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f89b40cd4b0 tx=0x7f89a402a270 comp rx=0 tx=0).stop 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 -- 172.21.15.165:0/3399047688 shutdown_connections 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 --2- 172.21.15.165:0/3399047688 >> v2:172.21.15.172:3300/0 conn(0x7f89b40cb9b0 0x7f89b406d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 --2- 172.21.15.165:0/3399047688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b415dc50 0x7f89b406d210 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 -- 172.21.15.165:0/3399047688 >> 172.21.15.165:0/3399047688 conn(0x7f89b4067070 msgr2=0x7f89b40b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 -- 172.21.15.165:0/3399047688 shutdown_connections 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.874+0000 7f89b3e00640 1 -- 172.21.15.165:0/3399047688 wait complete. 2024-04-25T10:01:50.879 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.875+0000 7f89b3e00640 1 Processor -- start 2024-04-25T10:01:50.880 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 -- start start 2024-04-25T10:01:50.882 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89b40b1bc0 0x7f89b4141c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.882 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 0x7f89b413e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.882 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89b415dc50 0x7f89b415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.882 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40714e0 con 0x7f89b40cb9b0 2024-04-25T10:01:50.882 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89b4071360 con 0x7f89b415dc50 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.876+0000 7f89b3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89b40718b0 con 0x7f89b40b1bc0 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 0x7f89b413e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 0x7f89b413e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50572/0 (socket says 172.21.15.165:50572) 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 -- 172.21.15.165:0/2487399225 learned_addr learned my addr 172.21.15.165:0/2487399225 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89b2a00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89b40b1bc0 0x7f89b4141c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89b3400640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89b415dc50 0x7f89b415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 -- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89b40b1bc0 msgr2=0x7f89b4141c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89b40b1bc0 0x7f89b4141c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 -- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89b415dc50 msgr2=0x7f89b415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89b415dc50 0x7f89b415f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89abe00640 1 -- 172.21.15.165:0/2487399225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89b41421c0 con 0x7f89b40cb9b0 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89b3400640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89b415dc50 0x7f89b415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.877+0000 7f89b2a00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89b40b1bc0 0x7f89b4141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.878+0000 7f89abe00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 0x7f89b413e950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c440 tx=0x7f899c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.883 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.878+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c0052d0 con 0x7f89b40cb9b0 2024-04-25T10:01:50.884 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.878+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899c005470 con 0x7f89b40cb9b0 2024-04-25T10:01:50.884 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.878+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89b41f7850 con 0x7f89b40cb9b0 2024-04-25T10:01:50.884 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.878+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899c007260 con 0x7f89b40cb9b0 2024-04-25T10:01:50.884 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.878+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89b41f7e10 con 0x7f89b40cb9b0 2024-04-25T10:01:50.884 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.879+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89b4072af0 con 0x7f89b40cb9b0 2024-04-25T10:01:50.884 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.880+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 4) v1 ==== 89724+0+0 (secure 0 0 0) 0x7f899c007400 con 0x7f89b40cb9b0 2024-04-25T10:01:50.885 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.881+0000 7f89aaa00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f899006b470 0x7f899006d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:50.885 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.881+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f899c088750 con 0x7f89b40cb9b0 2024-04-25T10:01:50.885 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.881+0000 7f89b2a00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f899006b470 0x7f899006d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:50.885 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.882+0000 7f89b2a00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f899006b470 0x7f899006d930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f89a4034000 tx=0x7f89a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:50.890 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:50.886+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f899c0584d0 con 0x7f89b40cb9b0 2024-04-25T10:01:51.070 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.066+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f89b4074ed0 con 0x7f89b40cb9b0 2024-04-25T10:01:51.072 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.067+0000 7f89aaa00640 1 -- 172.21.15.165:0/2487399225 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7058 (secure 0 0 0) 0x7f899c057dd0 con 0x7f89b40cb9b0 2024-04-25T10:01:51.072 INFO:teuthology.misc.health.smithi165.stdout: 2024-04-25T10:01:51.072 INFO:teuthology.misc.health.smithi165.stdout:{"epoch":12,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:01:48.134391+0000","last_up_change":"0.000000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.070+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f899006b470 msgr2=0x7f899006d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.070+0000 7f89b3e00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f899006b470 0x7f899006d930 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f89a4034000 tx=0x7f89a402e000 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.070+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 msgr2=0x7f89b413e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.070+0000 7f89b3e00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 0x7f89b413e950 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f899c00c440 tx=0x7f899c001e70 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 shutdown_connections 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f899006b470 0x7f899006d930 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89b415dc50 0x7f89b415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89b40cb9b0 0x7f89b413e950 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 --2- 172.21.15.165:0/2487399225 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89b40b1bc0 0x7f89b4141c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:51.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 >> 172.21.15.165:0/2487399225 conn(0x7f89b4067070 msgr2=0x7f89b40c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:51.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 shutdown_connections 2024-04-25T10:01:51.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:51.071+0000 7f89b3e00640 1 -- 172.21.15.165:0/2487399225 wait complete. 2024-04-25T10:01:51.088 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-25T10:01:56.245 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:56.253 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:01:57.896 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:58.074 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.070+0000 7f022ec00640 1 Processor -- start 2024-04-25T10:01:58.075 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.070+0000 7f022ec00640 1 -- start start 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.071+0000 7f022ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022806d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.071+0000 7f022ec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f02280cc1a0 0x7f02280b4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.071+0000 7f022ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f02280b45b0 0x7f0228142840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.071+0000 7f022ec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02280b3be0 con 0x7f02280cc1a0 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.071+0000 7f022ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02280b38b0 con 0x7f02280b45b0 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.071+0000 7f022ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0228142500 con 0x7f0228142120 2024-04-25T10:01:58.076 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f02280cc1a0 0x7f02280b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.077 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f02280cc1a0 0x7f02280b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50732/0 (socket says 172.21.15.165:50732) 2024-04-25T10:01:58.077 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 -- 172.21.15.165:0/3778040434 learned_addr learned my addr 172.21.15.165:0/3778040434 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:58.077 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022d800640 1 --2- 172.21.15.165:0/3778040434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022806d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.077 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022e200640 1 --2- 172.21.15.165:0/3778040434 >> v2:172.21.15.172:3300/0 conn(0x7f02280b45b0 0x7f0228142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.077 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 -- 172.21.15.165:0/3778040434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 msgr2=0x7f022806d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.077 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 --2- 172.21.15.165:0/3778040434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022806d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.078 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 -- 172.21.15.165:0/3778040434 >> v2:172.21.15.172:3300/0 conn(0x7f02280b45b0 msgr2=0x7f0228142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.078 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 --2- 172.21.15.165:0/3778040434 >> v2:172.21.15.172:3300/0 conn(0x7f02280b45b0 0x7f0228142840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.078 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.072+0000 7f022ce00640 1 -- 172.21.15.165:0/3778040434 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022806da60 con 0x7f02280cc1a0 2024-04-25T10:01:58.078 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.074+0000 7f022ce00640 1 --2- 172.21.15.165:0/3778040434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f02280b4070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f022806e330 tx=0x7f021402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e569c95d72ac89a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.078 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.074+0000 7f0227e00640 1 -- 172.21.15.165:0/3778040434 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214037070 con 0x7f02280cc1a0 2024-04-25T10:01:58.079 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.074+0000 7f0227e00640 1 -- 172.21.15.165:0/3778040434 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0214032440 con 0x7f02280cc1a0 2024-04-25T10:01:58.079 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.074+0000 7f0227e00640 1 -- 172.21.15.165:0/3778040434 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214032810 con 0x7f02280cc1a0 2024-04-25T10:01:58.079 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 -- 172.21.15.165:0/3778040434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 msgr2=0x7f02280b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.079 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 --2- 172.21.15.165:0/3778040434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f02280b4070 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f022806e330 tx=0x7f021402a140 comp rx=0 tx=0).stop 2024-04-25T10:01:58.079 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 -- 172.21.15.165:0/3778040434 shutdown_connections 2024-04-25T10:01:58.079 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 --2- 172.21.15.165:0/3778040434 >> v2:172.21.15.172:3300/0 conn(0x7f02280b45b0 0x7f0228142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.080 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 --2- 172.21.15.165:0/3778040434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f02280b4070 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.080 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 --2- 172.21.15.165:0/3778040434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022806d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.080 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 -- 172.21.15.165:0/3778040434 >> 172.21.15.165:0/3778040434 conn(0x7f0228067070 msgr2=0x7f02280b2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.080 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 -- 172.21.15.165:0/3778040434 shutdown_connections 2024-04-25T10:01:58.080 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.075+0000 7f022ec00640 1 -- 172.21.15.165:0/3778040434 wait complete. 2024-04-25T10:01:58.080 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.077+0000 7f022ec00640 1 Processor -- start 2024-04-25T10:01:58.081 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.077+0000 7f022ec00640 1 -- start start 2024-04-25T10:01:58.083 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.077+0000 7f022ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02280b45b0 0x7f022815e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f022815a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022806d750 con 0x7f02280cc1a0 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022806e020 con 0x7f02280b45b0 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0228142d80 con 0x7f0228142120 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f022815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f022815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50744/0 (socket says 172.21.15.165:50744) 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 -- 172.21.15.165:0/234496439 learned_addr learned my addr 172.21.15.165:0/234496439 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022d800640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02280b45b0 0x7f022815e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022e200640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 -- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 msgr2=0x7f022815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.084 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 -- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02280b45b0 msgr2=0x7f022815e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02280b45b0 0x7f022815e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022ce00640 1 -- 172.21.15.165:0/234496439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02281f2b30 con 0x7f02280cc1a0 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.078+0000 7f022e200640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.079+0000 7f022ce00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f022815a830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0214000c00 tx=0x7f02140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.079+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214041020 con 0x7f02280cc1a0 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.079+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02140306c0 con 0x7f02280cc1a0 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.079+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02281f2e10 con 0x7f02280cc1a0 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.079+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214037040 con 0x7f02280cc1a0 2024-04-25T10:01:58.085 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.079+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02281f33d0 con 0x7f02280cc1a0 2024-04-25T10:01:58.086 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.081+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0228072af0 con 0x7f02280cc1a0 2024-04-25T10:01:58.086 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.082+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0214044020 con 0x7f02280cc1a0 2024-04-25T10:01:58.087 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.083+0000 7f0225600640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.087 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.083+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f02140aa5e0 con 0x7f02280cc1a0 2024-04-25T10:01:58.087 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.083+0000 7f022d800640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.087 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.083+0000 7f022d800640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f020806b4d0 0x7f020806d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.092 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.088+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f021404b030 con 0x7f02280cc1a0 2024-04-25T10:01:58.290 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.286+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f02280ce250 con 0x7f02280cc1a0 2024-04-25T10:01:58.292 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.287+0000 7f0225600640 1 -- 172.21.15.165:0/234496439 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f021407f020 con 0x7f02280cc1a0 2024-04-25T10:01:58.292 INFO:teuthology.misc.health.smithi165.stdout: 2024-04-25T10:01:58.292 INFO:teuthology.misc.health.smithi165.stdout:{"epoch":16,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:01:54.566664+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f020806b4d0 msgr2=0x7f020806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f020806b4d0 0x7f020806d990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f021c006fd0 tx=0x7f021c008040 comp rx=0 tx=0).stop 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 msgr2=0x7f022815a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f022815a830 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0214000c00 tx=0x7f02140046c0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 shutdown_connections 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f020806b4d0 0x7f020806d990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0228142120 0x7f022815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02280cc1a0 0x7f022815a830 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 --2- 172.21.15.165:0/234496439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02280b45b0 0x7f022815e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 >> 172.21.15.165:0/234496439 conn(0x7f0228067070 msgr2=0x7f02280cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 shutdown_connections 2024-04-25T10:01:58.295 INFO:teuthology.misc.health.smithi165.stderr:2024-04-25T10:01:58.290+0000 7f022ec00640 1 -- 172.21.15.165:0/234496439 wait complete. 2024-04-25T10:01:58.309 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-25T10:01:58.309 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-25T10:01:58.309 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:01:58.309 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:01:58.316 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:01:58.316 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:01:58.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:01:58.323 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:01:58.323 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:01:58.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:01:58.331 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:01:58.331 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:01:58.331 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:01:58.338 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:01:58.338 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-04-25T10:01:58.341 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:58.459 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:01:58.474 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:01:58.486 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:01:58.486 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:01:58.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.515+0000 7fbf17200640 1 Processor -- start 2024-04-25T10:01:58.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.515+0000 7fbf17200640 1 -- start start 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.516+0000 7fbf17200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf100b93c0 0x7fbf100cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.516+0000 7fbf17200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100d04f0 0x7fbf100d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.516+0000 7fbf17200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf10161df0 con 0x7fbf1015db90 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.516+0000 7fbf17200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf100d0980 con 0x7fbf100b93c0 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.516+0000 7fbf17200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100d1f50 con 0x7fbf100d04f0 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf100b93c0 0x7fbf100cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf16800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100d04f0 0x7fbf100d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf100b93c0 0x7fbf100cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:53404/0 (socket says 172.21.15.165:53404) 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 -- 172.21.15.165:0/3350603617 learned_addr learned my addr 172.21.15.165:0/3350603617 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 -- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100d04f0 msgr2=0x7fbf100d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 --2- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100d04f0 0x7fbf100d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 -- 172.21.15.165:0/3350603617 >> v1:172.21.15.172:6789/0 conn(0x7fbf1015db90 legacy=0x7fbf101615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf15400640 1 -- 172.21.15.165:0/3350603617 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf101620d0 con 0x7fbf100b93c0 2024-04-25T10:01:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.517+0000 7fbf16800640 1 --2- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100d04f0 0x7fbf100d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:58.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.518+0000 7fbf15400640 1 --2- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100b93c0 0x7fbf100cffb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009870 tx=0x7fbf0002a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1635fac20b27575 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.518+0000 7fbf14a00640 1 -- 172.21.15.165:0/3350603617 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00037070 con 0x7fbf100b93c0 2024-04-25T10:01:58.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.518+0000 7fbf14a00640 1 -- 172.21.15.165:0/3350603617 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00032440 con 0x7fbf100b93c0 2024-04-25T10:01:58.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.518+0000 7fbf14a00640 1 -- 172.21.15.165:0/3350603617 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00032810 con 0x7fbf100b93c0 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 -- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100b93c0 msgr2=0x7fbf100cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 --2- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100b93c0 0x7fbf100cffb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbf00009870 tx=0x7fbf0002a250 comp rx=0 tx=0).stop 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 -- 172.21.15.165:0/3350603617 shutdown_connections 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 --2- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100d04f0 0x7fbf100d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 --2- 172.21.15.165:0/3350603617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100b93c0 0x7fbf100cffb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 -- 172.21.15.165:0/3350603617 >> 172.21.15.165:0/3350603617 conn(0x7fbf100b52b0 msgr2=0x7fbf100b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 -- 172.21.15.165:0/3350603617 shutdown_connections 2024-04-25T10:01:58.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.519+0000 7fbf17200640 1 -- 172.21.15.165:0/3350603617 wait complete. 2024-04-25T10:01:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.521+0000 7fbf17200640 1 Processor -- start 2024-04-25T10:01:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.521+0000 7fbf17200640 1 -- start start 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf17200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 0x7fbf1014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf17200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100d04f0 0x7fbf1015b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf17200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 0x7fbf1015bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf17200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100b68d0 con 0x7fbf1015db90 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf17200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100d1be0 con 0x7fbf100d04f0 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf17200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100b6a50 con 0x7fbf100b93c0 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf15e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 0x7fbf1014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf16800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 0x7fbf1015bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf15400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100d04f0 0x7fbf1015b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf16800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 0x7fbf1015bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50798/0 (socket says 172.21.15.165:50798) 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf16800640 1 -- 172.21.15.165:0/2752840939 learned_addr learned my addr 172.21.15.165:0/2752840939 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.522+0000 7fbf15e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 0x7fbf1014f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:59612/0 (socket says 172.21.15.165:59612) 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf16800640 1 -- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 msgr2=0x7fbf1014f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf16800640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 0x7fbf1014f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf16800640 1 -- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100d04f0 msgr2=0x7fbf1015b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf16800640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100d04f0 0x7fbf1015b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf16800640 1 -- 172.21.15.165:0/2752840939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100cbf20 con 0x7fbf1015db90 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf15400640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100d04f0 0x7fbf1015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf15e00640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 0x7fbf1014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:01:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf16800640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 0x7fbf1015bcf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009e50 tx=0x7fbf0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c018030 con 0x7fbf1015db90 2024-04-25T10:01:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c00cd90 con 0x7fbf1015db90 2024-04-25T10:01:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c003bb0 con 0x7fbf1015db90 2024-04-25T10:01:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf10159ae0 con 0x7fbf1015db90 2024-04-25T10:01:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.523+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1014f580 con 0x7fbf1015db90 2024-04-25T10:01:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.524+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf10073990 con 0x7fbf1015db90 2024-04-25T10:01:58.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.525+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf0c01b070 con 0x7fbf1015db90 2024-04-25T10:01:58.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.526+0000 7fbf0a000640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbef406b4d0 0x7fbef406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.526+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbf0c08a6f0 con 0x7fbf1015db90 2024-04-25T10:01:58.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.526+0000 7fbf15e00640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbef406b4d0 0x7fbef406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.527+0000 7fbf15e00640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbef406b4d0 0x7fbef406d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf04001ad0 tx=0x7fbf04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.531+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf0c010040 con 0x7fbf1015db90 2024-04-25T10:01:58.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.721+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf100d14b0 con 0x7fbf1015db90 2024-04-25T10:01:58.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.723+0000 7fbf0a000640 1 -- 172.21.15.165:0/2752840939 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12920 (secure 0 0 0) 0x7fbf0c059260 con 0x7fbf1015db90 2024-04-25T10:01:58.727 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:01:58.727 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":16,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:01:54.566664+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:01:58.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.725+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbef406b4d0 msgr2=0x7fbef406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.725+0000 7fbf17200640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbef406b4d0 0x7fbef406d990 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbf04001ad0 tx=0x7fbf04009040 comp rx=0 tx=0).stop 2024-04-25T10:01:58.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.725+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 msgr2=0x7fbf1015bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.725+0000 7fbf17200640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 0x7fbf1015bcf0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c009e50 tx=0x7fbf0c00c490 comp rx=0 tx=0).stop 2024-04-25T10:01:58.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 shutdown_connections 2024-04-25T10:01:58.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbef406b4d0 0x7fbef406d990 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf1015db90 0x7fbf1015bcf0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf100d04f0 0x7fbf1015b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 --2- 172.21.15.165:0/2752840939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf100b93c0 0x7fbf1014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 >> 172.21.15.165:0/2752840939 conn(0x7fbf100b52b0 msgr2=0x7fbf100c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 shutdown_connections 2024-04-25T10:01:58.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.726+0000 7fbf17200640 1 -- 172.21.15.165:0/2752840939 wait complete. 2024-04-25T10:01:58.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:01:58.747 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:01:58.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.944+0000 7f2586a00640 1 Processor -- start 2024-04-25T10:01:58.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.944+0000 7f2586a00640 1 -- start start 2024-04-25T10:01:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.945+0000 7f2586a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25801446d0 0x7f25800b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f2586a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800cccc0 con 0x7f258006e8a0 2024-04-25T10:01:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f2586a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25800d0460 con 0x7f2580144d50 2024-04-25T10:01:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f2586a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25800b2b20 con 0x7f25801446d0 2024-04-25T10:01:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f2584a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f258006e8a0 0x7f25800cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54042/0 (socket says 172.21.15.165:54042) 2024-04-25T10:01:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f2584a00640 1 -- 172.21.15.165:0/2626068440 learned_addr learned my addr 172.21.15.165:0/2626068440 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f257fe00640 1 --2- 172.21.15.165:0/2626068440 >> v2:172.21.15.172:3300/0 conn(0x7f25801446d0 0x7f25800b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050900698 0 0) 0x7f25800cccc0 con 0x7f258006e8a0 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.946+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2564003520 con 0x7f258006e8a0 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 709549057 0 0) 0x7f2564003520 con 0x7f258006e8a0 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 >> v2:172.21.15.172:3300/0 conn(0x7f25801446d0 msgr2=0x7f25800b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 --2- 172.21.15.165:0/2626068440 >> v2:172.21.15.172:3300/0 conn(0x7f25801446d0 0x7f25800b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 >> v1:172.21.15.172:6789/0 conn(0x7f2580144d50 legacy=0x7f25800cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800cc140 con 0x7f258006e8a0 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f25780019f0 con 0x7f258006e8a0 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 >> v1:172.21.15.165:6789/0 conn(0x7f258006e8a0 legacy=0x7f25800cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25640039b0 0x7f2564003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2564004800 0x7f2564004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f256400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800b2b20 con 0x7f25640051c0 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256400c060 con 0x7f2564004800 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f256400c260 con 0x7f25640039b0 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f2584a00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f256400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f257fe00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25640039b0 0x7f2564003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f2584a00640 1 -- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25640039b0 msgr2=0x7f2564003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f2584a00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25640039b0 0x7f2564003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f2584a00640 1 -- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2564004800 msgr2=0x7f2564004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f2584a00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2564004800 0x7f2564004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.947+0000 7f2584a00640 1 -- 172.21.15.165:0/2626068440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564003520 con 0x7f25640051c0 2024-04-25T10:01:58.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.948+0000 7f2584a00640 1 -- 172.21.15.165:0/2626068440 reap_dead start 2024-04-25T10:01:58.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.948+0000 7f2584a00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f256400b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f257800a860 tx=0x7f2564005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=996aa97e6d9322fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257800a480 con 0x7f25640051c0 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257802f070 con 0x7f25640051c0 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f257ea00640 1 -- 172.21.15.165:0/2626068440 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257802bca0 con 0x7f25640051c0 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 -- 172.21.15.165:0/2626068440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 msgr2=0x7f256400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f256400b9c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f257800a860 tx=0x7f2564005830 comp rx=0 tx=0).stop 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 -- 172.21.15.165:0/2626068440 shutdown_connections 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 --2- 172.21.15.165:0/2626068440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f256400b9c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 -- 172.21.15.165:0/2626068440 >> 172.21.15.165:0/2626068440 conn(0x7f258006abe0 msgr2=0x7f258006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 -- 172.21.15.165:0/2626068440 shutdown_connections 2024-04-25T10:01:58.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.949+0000 7f2586a00640 1 -- 172.21.15.165:0/2626068440 wait complete. 2024-04-25T10:01:58.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.951+0000 7f2586a00640 1 Processor -- start 2024-04-25T10:01:58.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.951+0000 7f2586a00640 1 -- start start 2024-04-25T10:01:58.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f2586a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f25800cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f2586a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25800c46d0 0x7f2580156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f2586a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25800ad900 0x7f25800add40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f2586a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25800cc700 con 0x7f25640051c0 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f2586a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258006d400 con 0x7f25800ad900 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f2586a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25800cc9e0 con 0x7f25800c46d0 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f257fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f25800cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f257fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f25800cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50810/0 (socket says 172.21.15.165:50810) 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.952+0000 7f257fe00640 1 -- 172.21.15.165:0/3104385866 learned_addr learned my addr 172.21.15.165:0/3104385866 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f257fe00640 1 -- 172.21.15.165:0/3104385866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25800c46d0 msgr2=0x7f2580156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f257fe00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25800c46d0 0x7f2580156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f257fe00640 1 -- 172.21.15.165:0/3104385866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25800ad900 msgr2=0x7f25800add40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f257fe00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25800ad900 0x7f25800add40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f257fe00640 1 -- 172.21.15.165:0/3104385866 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25800cf2f0 con 0x7f25640051c0 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f257fe00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f25800cedb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2570010890 tx=0x7f2570010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257000f040 con 0x7f25640051c0 2024-04-25T10:01:58.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257000eb60 con 0x7f25640051c0 2024-04-25T10:01:58.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25800c4c90 con 0x7f25640051c0 2024-04-25T10:01:58.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257001c400 con 0x7f25640051c0 2024-04-25T10:01:58.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.953+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25800c5250 con 0x7f25640051c0 2024-04-25T10:01:58.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.954+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2580073a00 con 0x7f25640051c0 2024-04-25T10:01:58.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.955+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f257001c5a0 con 0x7f25640051c0 2024-04-25T10:01:58.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.956+0000 7f2577e00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f256006b500 0x7f256006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:58.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.956+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f257008a5d0 con 0x7f25640051c0 2024-04-25T10:01:58.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.956+0000 7f257f400640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f256006b500 0x7f256006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:58.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.957+0000 7f257f400640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f256006b500 0x7f256006d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25800b23e0 tx=0x7f256c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:58.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:58.961+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2570018030 con 0x7f25640051c0 2024-04-25T10:01:59.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.161+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f25800cccc0 con 0x7f25640051c0 2024-04-25T10:01:59.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.162+0000 7f2577e00640 1 -- 172.21.15.165:0/3104385866 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2570059670 con 0x7f25640051c0 2024-04-25T10:01:59.167 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:01:59.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f256006b500 msgr2=0x7f256006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f256006b500 0x7f256006d9c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f25800b23e0 tx=0x7f256c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 msgr2=0x7f25800cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f25800cedb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2570010890 tx=0x7f2570010d50 comp rx=0 tx=0).stop 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 shutdown_connections 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f256006b500 0x7f256006d9c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25800ad900 0x7f25800add40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25800c46d0 0x7f2580156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 --2- 172.21.15.165:0/3104385866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25640051c0 0x7f25800cedb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 >> 172.21.15.165:0/3104385866 conn(0x7f258006abe0 msgr2=0x7f25800af440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 shutdown_connections 2024-04-25T10:01:59.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.165+0000 7f2586a00640 1 -- 172.21.15.165:0/3104385866 wait complete. 2024-04-25T10:01:59.188 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:01:59.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.379+0000 7f29c8400640 1 Processor -- start 2024-04-25T10:01:59.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.380+0000 7f29c8400640 1 -- start start 2024-04-25T10:01:59.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.381+0000 7f29c8400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f29c0144750 0x7f29c006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.381+0000 7f29c8400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29c00cf030 0x7f29c006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.381+0000 7f29c8400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29c0144b30 con 0x7f29c006eb10 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.381+0000 7f29c8400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00712d0 con 0x7f29c00cf030 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.381+0000 7f29c8400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29c00715b0 con 0x7f29c0144750 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29c00cf030 0x7f29c006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29c00cf030 0x7f29c006d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50820/0 (socket says 172.21.15.165:50820) 2024-04-25T10:01:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 -- 172.21.15.165:0/4241299175 learned_addr learned my addr 172.21.15.165:0/4241299175 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:59.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c5a00640 1 --2- 172.21.15.165:0/4241299175 >> v2:172.21.15.172:3300/0 conn(0x7f29c0144750 0x7f29c006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 -- 172.21.15.165:0/4241299175 >> v2:172.21.15.172:3300/0 conn(0x7f29c0144750 msgr2=0x7f29c006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 --2- 172.21.15.165:0/4241299175 >> v2:172.21.15.172:3300/0 conn(0x7f29c0144750 0x7f29c006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 -- 172.21.15.165:0/4241299175 >> v1:172.21.15.172:6789/0 conn(0x7f29c006eb10 legacy=0x7f29c00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.382+0000 7f29c6400640 1 -- 172.21.15.165:0/4241299175 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c0071890 con 0x7f29c00cf030 2024-04-25T10:01:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.383+0000 7f29c6400640 1 --2- 172.21.15.165:0/4241299175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c006d290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29c006f5d0 tx=0x7f29b802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb5aba0c159c0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.383+0000 7f29bfe00640 1 -- 172.21.15.165:0/4241299175 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8036070 con 0x7f29c00cf030 2024-04-25T10:01:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29bfe00640 1 -- 172.21.15.165:0/4241299175 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b8031600 con 0x7f29c00cf030 2024-04-25T10:01:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29bfe00640 1 -- 172.21.15.165:0/4241299175 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b80319d0 con 0x7f29c00cf030 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 -- 172.21.15.165:0/4241299175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 msgr2=0x7f29c006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 --2- 172.21.15.165:0/4241299175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c006d290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f29c006f5d0 tx=0x7f29b802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 -- 172.21.15.165:0/4241299175 shutdown_connections 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 --2- 172.21.15.165:0/4241299175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c006d290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 --2- 172.21.15.165:0/4241299175 >> v2:172.21.15.172:3300/0 conn(0x7f29c0144750 0x7f29c006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 -- 172.21.15.165:0/4241299175 >> 172.21.15.165:0/4241299175 conn(0x7f29c006abe0 msgr2=0x7f29c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 -- 172.21.15.165:0/4241299175 shutdown_connections 2024-04-25T10:01:59.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.384+0000 7f29c8400640 1 -- 172.21.15.165:0/4241299175 wait complete. 2024-04-25T10:01:59.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.386+0000 7f29c8400640 1 Processor -- start 2024-04-25T10:01:59.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.386+0000 7f29c8400640 1 -- start start 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c8400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29c006eb10 0x7f29c013e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c8400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c00af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c8400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29c0144750 0x7f29c0140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c8400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c01455d0 con 0x7f29c00cf030 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c8400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29c0145450 con 0x7f29c0144750 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c8400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29c015c000 con 0x7f29c006eb10 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c00af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c00af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50828/0 (socket says 172.21.15.165:50828) 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.387+0000 7f29c5000640 1 -- 172.21.15.165:0/777746350 learned_addr learned my addr 172.21.15.165:0/777746350 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c6400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29c0144750 0x7f29c0140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5a00640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29c006eb10 0x7f29c013e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5000640 1 -- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29c006eb10 msgr2=0x7f29c013e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5000640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29c006eb10 0x7f29c013e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5000640 1 -- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29c0144750 msgr2=0x7f29c0140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5000640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29c0144750 0x7f29c0140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5000640 1 -- 172.21.15.165:0/777746350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29c00afbc0 con 0x7f29c00cf030 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c6400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29c0144750 0x7f29c0140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c5000640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c00af2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b420 tx=0x7f29ac00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac011830 con 0x7f29c00cf030 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac016070 con 0x7f29c00cf030 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29c00afea0 con 0x7f29c00cf030 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac011c10 con 0x7f29c00cf030 2024-04-25T10:01:59.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.388+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29c00af800 con 0x7f29c00cf030 2024-04-25T10:01:59.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.390+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29c0073990 con 0x7f29c00cf030 2024-04-25T10:01:59.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.390+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29ac01b460 con 0x7f29c00cf030 2024-04-25T10:01:59.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.391+0000 7f29bd600640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.391+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f29ac089a50 con 0x7f29c00cf030 2024-04-25T10:01:59.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.392+0000 7f29c5a00640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.392+0000 7f29c5a00640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29a006b4d0 0x7f29a006d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006fd0 tx=0x7f29b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.396+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29ac020020 con 0x7f29c00cf030 2024-04-25T10:01:59.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.635+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29c00d0510 con 0x7f29c00cf030 2024-04-25T10:01:59.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.636+0000 7f29bd600640 1 -- 172.21.15.165:0/777746350 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3706 (secure 0 0 0) 0x7f29ac058b80 con 0x7f29c00cf030 2024-04-25T10:01:59.641 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:01:59.641 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4288,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2024-04-25T10:01:59.641 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 3 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29a006b4d0 msgr2=0x7f29a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29a006b4d0 0x7f29a006d990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f29b4006fd0 tx=0x7f29b4008040 comp rx=0 tx=0).stop 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 msgr2=0x7f29c00af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c00af2c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f29ac00b420 tx=0x7f29ac00b8e0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 shutdown_connections 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29a006b4d0 0x7f29a006d990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29c0144750 0x7f29c0140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29c00cf030 0x7f29c00af2c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 --2- 172.21.15.165:0/777746350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29c006eb10 0x7f29c013e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 >> 172.21.15.165:0/777746350 conn(0x7f29c006abe0 msgr2=0x7f29c0142830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 shutdown_connections 2024-04-25T10:01:59.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.639+0000 7f29c8400640 1 -- 172.21.15.165:0/777746350 wait complete. 2024-04-25T10:01:59.662 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:01:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.860+0000 7f6e38a00640 1 Processor -- start 2024-04-25T10:01:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.860+0000 7f6e38a00640 1 -- start start 2024-04-25T10:01:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.862+0000 7f6e38a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 0x7f6e340cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.862+0000 7f6e38a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e34144240 con 0x7f6e34143e60 2024-04-25T10:01:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.862+0000 7f6e38a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415e220 con 0x7f6e340cc840 2024-04-25T10:01:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.862+0000 7f6e38a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415e500 con 0x7f6e3406e870 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e32800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6e34143e60 0x7f6e3406e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54048/0 (socket says 172.21.15.165:54048) 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e32800640 1 -- 172.21.15.165:0/2556122240 learned_addr learned my addr 172.21.15.165:0/2556122240 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e29e00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 0x7f6e340cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992525282 0 0) 0x7f6e34144240 con 0x7f6e34143e60 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003520 con 0x7f6e34143e60 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339613797 0 0) 0x7f6e18003520 con 0x7f6e34143e60 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 msgr2=0x7f6e340cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 0x7f6e340cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 >> v1:172.21.15.172:6789/0 conn(0x7f6e340cc840 legacy=0x7f6e3415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415e7e0 con 0x7f6e34143e60 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.863+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6e24001670 con 0x7f6e34143e60 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 >> v1:172.21.15.165:6789/0 conn(0x7f6e34143e60 legacy=0x7f6e3406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e180039e0 0x7f6e18003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 0x7f6e18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e18004d40 0x7f6e180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3415e500 con 0x7f6e18004d40 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800ba70 con 0x7f6e180039e0 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e1800bc70 con 0x7f6e3406e870 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e33200640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e18004d40 0x7f6e180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e32800640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e180039e0 0x7f6e18003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e29e00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 0x7f6e18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e33200640 1 -- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 msgr2=0x7f6e18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e33200640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3406e870 0x7f6e18004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e33200640 1 -- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e180039e0 msgr2=0x7f6e18003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:01:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e33200640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e180039e0 0x7f6e18003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.864+0000 7f6e33200640 1 -- 172.21.15.165:0/2556122240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18003520 con 0x7f6e18004d40 2024-04-25T10:01:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.865+0000 7f6e33200640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e18004d40 0x7f6e180051c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c001e20 tx=0x7f6e18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9bcfc70c5fa0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c001a70 con 0x7f6e18004d40 2024-04-25T10:01:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e2c02ae40 con 0x7f6e18004d40 2024-04-25T10:01:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e31e00640 1 -- 172.21.15.165:0/2556122240 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e2c003bb0 con 0x7f6e18004d40 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e38a00640 1 -- 172.21.15.165:0/2556122240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e18004d40 msgr2=0x7f6e180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e38a00640 1 --2- 172.21.15.165:0/2556122240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e18004d40 0x7f6e180051c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6e2c001e20 tx=0x7f6e18005830 comp rx=0 tx=0).stop 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e33200640 1 -- 172.21.15.165:0/2556122240 reap_dead start 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e38a00640 1 -- 172.21.15.165:0/2556122240 shutdown_connections 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e38a00640 1 -- 172.21.15.165:0/2556122240 >> 172.21.15.165:0/2556122240 conn(0x7f6e3406abe0 msgr2=0x7f6e3406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e38a00640 1 -- 172.21.15.165:0/2556122240 shutdown_connections 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.866+0000 7f6e38a00640 1 -- 172.21.15.165:0/2556122240 wait complete. 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.868+0000 7f6e38a00640 1 Processor -- start 2024-04-25T10:01:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.868+0000 7f6e38a00640 1 -- start start 2024-04-25T10:01:59.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e38a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e18004d40 0x7f6e34139e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e38a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 0x7f6e3415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e38a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e341f12b0 0x7f6e341f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e38a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b2d30 con 0x7f6e3415c110 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e38a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340cccf0 con 0x7f6e18004d40 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e38a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340b2eb0 con 0x7f6e341f12b0 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e29e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 0x7f6e3415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e29e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 0x7f6e3415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50840/0 (socket says 172.21.15.165:50840) 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.869+0000 7f6e29e00640 1 -- 172.21.15.165:0/487631945 learned_addr learned my addr 172.21.15.165:0/487631945 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e32800640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e18004d40 0x7f6e34139e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e33200640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e341f12b0 0x7f6e341f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e29e00640 1 -- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e341f12b0 msgr2=0x7f6e341f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e29e00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e341f12b0 0x7f6e341f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e29e00640 1 -- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e18004d40 msgr2=0x7f6e34139e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e29e00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e18004d40 0x7f6e34139e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e29e00640 1 -- 172.21.15.165:0/487631945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e3415d950 con 0x7f6e3415c110 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e32800640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e18004d40 0x7f6e34139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e33200640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e341f12b0 0x7f6e341f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:01:59.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e29e00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 0x7f6e3415c590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0108c0 tx=0x7f6e1c010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c00f040 con 0x7f6e3415c110 2024-04-25T10:01:59.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e1c00eb20 con 0x7f6e3415c110 2024-04-25T10:01:59.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e1c01c400 con 0x7f6e3415c110 2024-04-25T10:01:59.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.870+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e341f1c70 con 0x7f6e3415c110 2024-04-25T10:01:59.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.871+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e341f21e0 con 0x7f6e3415c110 2024-04-25T10:01:59.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.872+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340ce610 con 0x7f6e3415c110 2024-04-25T10:01:59.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.873+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e1c0143f0 con 0x7f6e3415c110 2024-04-25T10:01:59.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.873+0000 7f6e2b400640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1006b4d0 0x7f6e1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:01:59.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.874+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6e1c08a160 con 0x7f6e3415c110 2024-04-25T10:01:59.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.874+0000 7f6e32800640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1006b4d0 0x7f6e1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:01:59.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.874+0000 7f6e32800640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1006b4d0 0x7f6e1006d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e24001cc0 tx=0x7f6e24001db0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:01:59.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:01:59.878+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e340ce610 con 0x7f6e3415c110 2024-04-25T10:02:00.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.074+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e34073990 con 0x7f6e3415c110 2024-04-25T10:02:00.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.075+0000 7f6e2b400640 1 -- 172.21.15.165:0/487631945 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12920 (secure 0 0 0) 0x7f6e1c018030 con 0x7f6e3415c110 2024-04-25T10:02:00.080 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:00.080 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":16,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:01:54.566664+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1006b4d0 msgr2=0x7f6e1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1006b4d0 0x7f6e1006d990 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6e24001cc0 tx=0x7f6e24001db0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 msgr2=0x7f6e3415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 0x7f6e3415c590 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c0108c0 tx=0x7f6e1c010d80 comp rx=0 tx=0).stop 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 shutdown_connections 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1006b4d0 0x7f6e1006d990 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e341f12b0 0x7f6e341f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e3415c110 0x7f6e3415c590 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 --2- 172.21.15.165:0/487631945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e18004d40 0x7f6e34139e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 >> 172.21.15.165:0/487631945 conn(0x7f6e3406abe0 msgr2=0x7f6e340b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 shutdown_connections 2024-04-25T10:02:00.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.078+0000 7f6e38a00640 1 -- 172.21.15.165:0/487631945 wait complete. 2024-04-25T10:02:00.101 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:02:00.101 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:00.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.286+0000 7f7924e00640 1 Processor -- start 2024-04-25T10:02:00.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.286+0000 7f7924e00640 1 -- start start 2024-04-25T10:02:00.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.288+0000 7f7924e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7920144d50 0x7f792006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.288+0000 7f7924e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79201452a0 0x7f792006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.288+0000 7f7924e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79200cccc0 con 0x7f79201446d0 2024-04-25T10:02:00.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.288+0000 7f7924e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79200b2b20 con 0x7f7920144d50 2024-04-25T10:02:00.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.288+0000 7f7924e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f792006d400 con 0x7f79201452a0 2024-04-25T10:02:00.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.288+0000 7f791d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7920144d50 0x7f792006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7920144d50 0x7f792006d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:53484/0 (socket says 172.21.15.165:53484) 2024-04-25T10:02:00.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791d400640 1 -- 172.21.15.165:0/1967215090 learned_addr learned my addr 172.21.15.165:0/1967215090 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:00.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791e800640 1 --2- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79201452a0 0x7f792006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791d400640 1 -- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79201452a0 msgr2=0x7f792006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791d400640 1 --2- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79201452a0 0x7f792006ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791d400640 1 -- 172.21.15.165:0/1967215090 >> v1:172.21.15.172:6789/0 conn(0x7f79201446d0 legacy=0x7f79200b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.289+0000 7f791d400640 1 -- 172.21.15.165:0/1967215090 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79200cc140 con 0x7f7920144d50 2024-04-25T10:02:00.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.290+0000 7f791d400640 1 --2- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920144d50 0x7f792006d890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7908009870 tx=0x7f790802a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8677a29c16cd2a91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.291+0000 7f791ca00640 1 -- 172.21.15.165:0/1967215090 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908037070 con 0x7f7920144d50 2024-04-25T10:02:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.291+0000 7f791ca00640 1 -- 172.21.15.165:0/1967215090 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7908032440 con 0x7f7920144d50 2024-04-25T10:02:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.291+0000 7f791ca00640 1 -- 172.21.15.165:0/1967215090 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7908032810 con 0x7f7920144d50 2024-04-25T10:02:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 -- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920144d50 msgr2=0x7f792006d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 --2- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920144d50 0x7f792006d890 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7908009870 tx=0x7f790802a250 comp rx=0 tx=0).stop 2024-04-25T10:02:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 -- 172.21.15.165:0/1967215090 shutdown_connections 2024-04-25T10:02:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 --2- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79201452a0 0x7f792006ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 --2- 172.21.15.165:0/1967215090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7920144d50 0x7f792006d890 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 -- 172.21.15.165:0/1967215090 >> 172.21.15.165:0/1967215090 conn(0x7f792006abe0 msgr2=0x7f792006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 -- 172.21.15.165:0/1967215090 shutdown_connections 2024-04-25T10:02:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.292+0000 7f7924e00640 1 -- 172.21.15.165:0/1967215090 wait complete. 2024-04-25T10:02:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.294+0000 7f7924e00640 1 Processor -- start 2024-04-25T10:02:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.294+0000 7f7924e00640 1 -- start start 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f7924e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79201446d0 0x7f79200cef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f7924e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7920144d50 0x7f79201f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f7924e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 0x7f79200ce420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f7924e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200cc9e0 con 0x7f79201452a0 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f7924e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79200cc700 con 0x7f79201446d0 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f7924e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7920144ab0 con 0x7f7920144d50 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 0x7f79200ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 0x7f79200ce420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50852/0 (socket says 172.21.15.165:50852) 2024-04-25T10:02:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791e800640 1 -- 172.21.15.165:0/2376530665 learned_addr learned my addr 172.21.15.165:0/2376530665 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791de00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79201446d0 0x7f79200cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791d400640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7920144d50 0x7f79201f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791e800640 1 -- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7920144d50 msgr2=0x7f79201f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.295+0000 7f791e800640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7920144d50 0x7f79201f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f791e800640 1 -- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79201446d0 msgr2=0x7f79200cef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f791e800640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79201446d0 0x7f79200cef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f791e800640 1 -- 172.21.15.165:0/2376530665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79201f4720 con 0x7f79201452a0 2024-04-25T10:02:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f791e800640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 0x7f79200ce420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7918009dd0 tx=0x7f791800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918018020 con 0x7f79201452a0 2024-04-25T10:02:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7918014480 con 0x7f79201452a0 2024-04-25T10:02:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79200cd380 con 0x7f79201452a0 2024-04-25T10:02:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7918014870 con 0x7f79201452a0 2024-04-25T10:02:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.296+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79200cd940 con 0x7f79201452a0 2024-04-25T10:02:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.297+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7920073a00 con 0x7f79201452a0 2024-04-25T10:02:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.298+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7918005d10 con 0x7f79201452a0 2024-04-25T10:02:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.299+0000 7f7916000640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.299+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f791808ac10 con 0x7f79201452a0 2024-04-25T10:02:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.299+0000 7f791de00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.300+0000 7f791de00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78fc06b4d0 0x7f78fc06d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7910000f60 tx=0x7f7910009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.303+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7918022030 con 0x7f79201452a0 2024-04-25T10:02:00.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.498+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f79200b1b70 con 0x7f79201452a0 2024-04-25T10:02:00.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.499+0000 7f7916000640 1 -- 172.21.15.165:0/2376530665 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f79201f4720 con 0x7f79201452a0 2024-04-25T10:02:00.504 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:00.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78fc06b4d0 msgr2=0x7f78fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78fc06b4d0 0x7f78fc06d990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7910000f60 tx=0x7f7910009040 comp rx=0 tx=0).stop 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 msgr2=0x7f79200ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 0x7f79200ce420 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f7918009dd0 tx=0x7f791800c490 comp rx=0 tx=0).stop 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 shutdown_connections 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78fc06b4d0 0x7f78fc06d990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79201452a0 0x7f79200ce420 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7920144d50 0x7f79201f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 --2- 172.21.15.165:0/2376530665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79201446d0 0x7f79200cef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 >> 172.21.15.165:0/2376530665 conn(0x7f792006abe0 msgr2=0x7f79201431e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 shutdown_connections 2024-04-25T10:02:00.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.502+0000 7f7924e00640 1 -- 172.21.15.165:0/2376530665 wait complete. 2024-04-25T10:02:00.525 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:02:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.713+0000 7f02f5c00640 1 Processor -- start 2024-04-25T10:02:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.713+0000 7f02f5c00640 1 -- start start 2024-04-25T10:02:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02f5c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f02f015af20 0x7f02f00b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02f5c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f02f015b300 0x7f02f015b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02f5c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02f00b4c10 con 0x7f02f006dfb0 2024-04-25T10:02:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02f5c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f00b2800 con 0x7f02f015af20 2024-04-25T10:02:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02f5c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02f00b2ae0 con 0x7f02f015b300 2024-04-25T10:02:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f02f015af20 0x7f02f00b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f02f015af20 0x7f02f00b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50868/0 (socket says 172.21.15.165:50868) 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.715+0000 7f02eea00640 1 -- 172.21.15.165:0/3322604708 learned_addr learned my addr 172.21.15.165:0/3322604708 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.716+0000 7f02ef400640 1 --2- 172.21.15.165:0/3322604708 >> v2:172.21.15.172:3300/0 conn(0x7f02f015b300 0x7f02f015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.716+0000 7f02eea00640 1 -- 172.21.15.165:0/3322604708 >> v2:172.21.15.172:3300/0 conn(0x7f02f015b300 msgr2=0x7f02f015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.716+0000 7f02eea00640 1 --2- 172.21.15.165:0/3322604708 >> v2:172.21.15.172:3300/0 conn(0x7f02f015b300 0x7f02f015b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.716+0000 7f02eea00640 1 -- 172.21.15.165:0/3322604708 >> v1:172.21.15.172:6789/0 conn(0x7f02f006dfb0 legacy=0x7f02f00b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.716+0000 7f02eea00640 1 -- 172.21.15.165:0/3322604708 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f00b2dc0 con 0x7f02f015af20 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.717+0000 7f02eea00640 1 --2- 172.21.15.165:0/3322604708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015af20 0x7f02f00b35c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02f00cd0d0 tx=0x7f02e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf7bd1c0b6233665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.717+0000 7f02ee000640 1 -- 172.21.15.165:0/3322604708 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0037070 con 0x7f02f015af20 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.717+0000 7f02ee000640 1 -- 172.21.15.165:0/3322604708 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e0032440 con 0x7f02f015af20 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.717+0000 7f02ee000640 1 -- 172.21.15.165:0/3322604708 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e0032810 con 0x7f02f015af20 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 -- 172.21.15.165:0/3322604708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015af20 msgr2=0x7f02f00b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 --2- 172.21.15.165:0/3322604708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015af20 0x7f02f00b35c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f02f00cd0d0 tx=0x7f02e002a230 comp rx=0 tx=0).stop 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 -- 172.21.15.165:0/3322604708 shutdown_connections 2024-04-25T10:02:00.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 --2- 172.21.15.165:0/3322604708 >> v2:172.21.15.172:3300/0 conn(0x7f02f015b300 0x7f02f015b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 --2- 172.21.15.165:0/3322604708 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015af20 0x7f02f00b35c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 -- 172.21.15.165:0/3322604708 >> 172.21.15.165:0/3322604708 conn(0x7f02f006abe0 msgr2=0x7f02f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 -- 172.21.15.165:0/3322604708 shutdown_connections 2024-04-25T10:02:00.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.718+0000 7f02f5c00640 1 -- 172.21.15.165:0/3322604708 wait complete. 2024-04-25T10:02:00.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.720+0000 7f02f5c00640 1 Processor -- start 2024-04-25T10:02:00.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.720+0000 7f02f5c00640 1 -- start start 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02f006dfb0 0x7f02f0158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02f015af20 0x7f02f0156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02f5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 0x7f02f00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02f5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015c070 con 0x7f02f015b300 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02f5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015bcd0 con 0x7f02f006dfb0 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02f5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02f015c1f0 con 0x7f02f015af20 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 0x7f02f00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 0x7f02f00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50876/0 (socket says 172.21.15.165:50876) 2024-04-25T10:02:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 -- 172.21.15.165:0/2027112492 learned_addr learned my addr 172.21.15.165:0/2027112492 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02eea00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02f006dfb0 0x7f02f0158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02e6000640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02f015af20 0x7f02f0156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 -- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02f015af20 msgr2=0x7f02f0156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02f015af20 0x7f02f0156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 -- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02f006dfb0 msgr2=0x7f02f0158ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02f006dfb0 0x7f02f0158ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02ef400640 1 -- 172.21.15.165:0/2027112492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f01e4450 con 0x7f02f015b300 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.721+0000 7f02e6000640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02f015af20 0x7f02f0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.722+0000 7f02ef400640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 0x7f02f00ac8b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02e800cc30 tx=0x7f02e800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.722+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e800ce40 con 0x7f02f015b300 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.722+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02e8003990 con 0x7f02f015b300 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.722+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e8003d40 con 0x7f02f015b300 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.722+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02f01e4730 con 0x7f02f015b300 2024-04-25T10:02:00.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.722+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02f01e4cf0 con 0x7f02f015b300 2024-04-25T10:02:00.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.723+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02f01e4ef0 con 0x7f02f015b300 2024-04-25T10:02:00.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.724+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f02e8003ee0 con 0x7f02f015b300 2024-04-25T10:02:00.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.725+0000 7f02e7400640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f02cc06b4d0 0x7f02cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:00.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.725+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f02e808a370 con 0x7f02f015b300 2024-04-25T10:02:00.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.725+0000 7f02eea00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f02cc06b4d0 0x7f02cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:00.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.726+0000 7f02eea00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f02cc06b4d0 0x7f02cc06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02e0034000 tx=0x7f02e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:00.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.729+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f02f01e4ef0 con 0x7f02f015b300 2024-04-25T10:02:00.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.912+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f02f0073a00 con 0x7f02f015b300 2024-04-25T10:02:00.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.913+0000 7f02e7400640 1 -- 172.21.15.165:0/2027112492 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f02e8014030 con 0x7f02f015b300 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:02:00.918 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:02:00.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f02cc06b4d0 msgr2=0x7f02cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f02cc06b4d0 0x7f02cc06d990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f02e0034000 tx=0x7f02e002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:00.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 msgr2=0x7f02f00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 0x7f02f00ac8b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f02e800cc30 tx=0x7f02e800a5c0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 shutdown_connections 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f02cc06b4d0 0x7f02cc06d990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f02f015b300 0x7f02f00ac8b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f02f015af20 0x7f02f0156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 --2- 172.21.15.165:0/2027112492 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f02f006dfb0 0x7f02f0158ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 >> 172.21.15.165:0/2027112492 conn(0x7f02f006abe0 msgr2=0x7f02f0158300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 shutdown_connections 2024-04-25T10:02:00.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:00.916+0000 7f02f5c00640 1 -- 172.21.15.165:0/2027112492 wait complete. 2024-04-25T10:02:00.939 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:02:00.939 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:02:01.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.138+0000 7f1cde000640 1 Processor -- start 2024-04-25T10:02:01.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.139+0000 7f1cde000640 1 -- start start 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.140+0000 7f1cde000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1cd81446d0 0x7f1cd80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.140+0000 7f1cde000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd80cccc0 con 0x7f1cd8144d50 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.140+0000 7f1cde000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd80d0460 con 0x7f1cd806e8a0 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.140+0000 7f1cde000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd80b2b20 con 0x7f1cd81446d0 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1cd7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1cd81446d0 0x7f1cd80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1ccfe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1cd8144d50 0x7f1cd80cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54052/0 (socket says 172.21.15.165:54052) 2024-04-25T10:02:01.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1ccfe00640 1 -- 172.21.15.165:0/3499402021 learned_addr learned my addr 172.21.15.165:0/3499402021 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:01.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1cd7400640 1 -- 172.21.15.165:0/3499402021 >> v1:172.21.15.172:6789/0 conn(0x7f1cd806e8a0 legacy=0x7f1cd80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:01.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1cd7400640 1 -- 172.21.15.165:0/3499402021 >> v1:172.21.15.165:6789/0 conn(0x7f1cd8144d50 legacy=0x7f1cd80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1cd7400640 1 -- 172.21.15.165:0/3499402021 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd80cc140 con 0x7f1cd81446d0 2024-04-25T10:02:01.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.141+0000 7f1cd6a00640 1 -- 172.21.15.165:0/3499402021 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 337510799 0 0) 0x7f1cd80cccc0 con 0x7f1cd8144d50 2024-04-25T10:02:01.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.142+0000 7f1cd7400640 1 --2- 172.21.15.165:0/3499402021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd81446d0 0x7f1cd80b3f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097a0 tx=0x7f1cc802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42530c0bab2ca874 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cd6a00640 1 -- 172.21.15.165:0/3499402021 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8037070 con 0x7f1cd81446d0 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cd6a00640 1 -- 172.21.15.165:0/3499402021 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc8032440 con 0x7f1cd81446d0 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cd6a00640 1 -- 172.21.15.165:0/3499402021 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8032810 con 0x7f1cd81446d0 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 -- 172.21.15.165:0/3499402021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd81446d0 msgr2=0x7f1cd80b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 --2- 172.21.15.165:0/3499402021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd81446d0 0x7f1cd80b3f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097a0 tx=0x7f1cc802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 -- 172.21.15.165:0/3499402021 shutdown_connections 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 --2- 172.21.15.165:0/3499402021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd81446d0 0x7f1cd80b3f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 -- 172.21.15.165:0/3499402021 >> 172.21.15.165:0/3499402021 conn(0x7f1cd806abe0 msgr2=0x7f1cd806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 -- 172.21.15.165:0/3499402021 shutdown_connections 2024-04-25T10:02:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.143+0000 7f1cde000640 1 -- 172.21.15.165:0/3499402021 wait complete. 2024-04-25T10:02:01.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.145+0000 7f1cde000640 1 Processor -- start 2024-04-25T10:02:01.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.145+0000 7f1cde000640 1 -- start start 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cde000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 0x7f1cd8156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cde000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd81446d0 0x7f1cd8159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cde000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cd8144d50 0x7f1cd81597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cde000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80cc700 con 0x7f1cd806e8a0 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cde000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd806d400 con 0x7f1cd81446d0 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cde000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd80cc9e0 con 0x7f1cd8144d50 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cd7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 0x7f1cd8156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cd7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 0x7f1cd8156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50892/0 (socket says 172.21.15.165:50892) 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.146+0000 7f1cd7400640 1 -- 172.21.15.165:0/646047845 learned_addr learned my addr 172.21.15.165:0/646047845 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:01.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1ccfe00640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd81446d0 0x7f1cd8159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cd7400640 1 -- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cd8144d50 msgr2=0x7f1cd81597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cd7400640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cd8144d50 0x7f1cd81597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cd7400640 1 -- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd81446d0 msgr2=0x7f1cd8159290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:01.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cd7400640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd81446d0 0x7f1cd8159290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:01.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cd7400640 1 -- 172.21.15.165:0/646047845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd81e0d40 con 0x7f1cd806e8a0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cd7400640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 0x7f1cd8156220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097a0 tx=0x7f1cc80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8041070 con 0x7f1cd806e8a0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc803c870 con 0x7f1cd806e8a0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc8037040 con 0x7f1cd806e8a0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.147+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd81f27b0 con 0x7f1cd806e8a0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.148+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd81f2d70 con 0x7f1cd806e8a0 2024-04-25T10:02:01.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.149+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd8073a00 con 0x7f1cd806e8a0 2024-04-25T10:02:01.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.150+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1cc8044070 con 0x7f1cd806e8a0 2024-04-25T10:02:01.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.150+0000 7f1ccf400640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1cb406b4d0 0x7f1cb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:01.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.151+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1cc80aaba0 con 0x7f1cd806e8a0 2024-04-25T10:02:01.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.151+0000 7f1ccfe00640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1cb406b4d0 0x7f1cb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:01.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.151+0000 7f1ccfe00640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1cb406b4d0 0x7f1cb406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80b36f0 tx=0x7f1cc0007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:01.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.155+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1cc8079df0 con 0x7f1cd806e8a0 2024-04-25T10:02:01.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:01.350+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1cd806b340 con 0x7f1cd806e8a0 2024-04-25T10:02:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.086+0000 7f1ccf400640 1 -- 172.21.15.165:0/646047845 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f1cc80796f0 con 0x7f1cd806e8a0 2024-04-25T10:02:02.091 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.089+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1cb406b4d0 msgr2=0x7f1cb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.089+0000 7f1cde000640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1cb406b4d0 0x7f1cb406d990 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1cd80b36f0 tx=0x7f1cc0007c00 comp rx=0 tx=0).stop 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.089+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 msgr2=0x7f1cd8156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.089+0000 7f1cde000640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 0x7f1cd8156220 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1cc80097a0 tx=0x7f1cc80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 shutdown_connections 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1cb406b4d0 0x7f1cb406d990 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1cd8144d50 0x7f1cd81597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1cd81446d0 0x7f1cd8159290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 --2- 172.21.15.165:0/646047845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1cd806e8a0 0x7f1cd8156220 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 >> 172.21.15.165:0/646047845 conn(0x7f1cd806abe0 msgr2=0x7f1cd80af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 shutdown_connections 2024-04-25T10:02:02.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.090+0000 7f1cde000640 1 -- 172.21.15.165:0/646047845 wait complete. 2024-04-25T10:02:02.122 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-25T10:02:02.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.313+0000 7fd9cfe00640 1 Processor -- start 2024-04-25T10:02:02.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.314+0000 7fd9cfe00640 1 -- start start 2024-04-25T10:02:02.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.315+0000 7fd9cfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d006bff0 0x7fd9d00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.315+0000 7fd9cfe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d015e110 con 0x7fd9d00b5970 2024-04-25T10:02:02.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.315+0000 7fd9cfe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9d00b2fd0 con 0x7fd9d015dd30 2024-04-25T10:02:02.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.315+0000 7fd9cfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d006d000 con 0x7fd9d006bff0 2024-04-25T10:02:02.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.316+0000 7fd9cf400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd9d00b5970 0x7fd9d00d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54068/0 (socket says 172.21.15.165:54068) 2024-04-25T10:02:02.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.316+0000 7fd9cf400640 1 -- 172.21.15.165:0/3887715865 learned_addr learned my addr 172.21.15.165:0/3887715865 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:02.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.316+0000 7fd9ce000640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d006bff0 0x7fd9d00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.316+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285347530 0 0) 0x7fd9d015e110 con 0x7fd9d00b5970 2024-04-25T10:02:02.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.316+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9b4003520 con 0x7fd9d00b5970 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1965579334 0 0) 0x7fd9b4003520 con 0x7fd9d00b5970 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d006bff0 msgr2=0x7fd9d00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d006bff0 0x7fd9d00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 >> v1:172.21.15.172:6789/0 conn(0x7fd9d015dd30 legacy=0x7fd9d00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d006c480 con 0x7fd9d00b5970 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd9c80019f0 con 0x7fd9d00b5970 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 >> v1:172.21.15.165:6789/0 conn(0x7fd9d00b5970 legacy=0x7fd9d00d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b40039e0 0x7fd9b4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9b4004800 0x7fd9b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9b40051c0 0x7fd9b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d006d000 con 0x7fd9b4004800 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b400bf00 con 0x7fd9b40039e0 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b400c080 con 0x7fd9b40051c0 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9ce000640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9b4004800 0x7fd9b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cea00640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b40039e0 0x7fd9b4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cf400640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9b40051c0 0x7fd9b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9ce000640 1 -- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9b40051c0 msgr2=0x7fd9b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9ce000640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9b40051c0 0x7fd9b400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9ce000640 1 -- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b40039e0 msgr2=0x7fd9b4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9ce000640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b40039e0 0x7fd9b4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9ce000640 1 -- 172.21.15.165:0/3887715865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b4003520 con 0x7fd9b4004800 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.317+0000 7fd9cf400640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9b40051c0 0x7fd9b400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.318+0000 7fd9ce000640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9b4004800 0x7fd9b4004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009ac0 tx=0x7fd9b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1c4ac12e6f00f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc036070 con 0x7fd9b4004800 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9bc031400 con 0x7fd9b4004800 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cd600640 1 -- 172.21.15.165:0/3887715865 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc009690 con 0x7fd9b4004800 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cfe00640 1 -- 172.21.15.165:0/3887715865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9b4004800 msgr2=0x7fd9b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/3887715865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9b4004800 0x7fd9b4004c80 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc009ac0 tx=0x7fd9b4005830 comp rx=0 tx=0).stop 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cf400640 1 -- 172.21.15.165:0/3887715865 reap_dead start 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cfe00640 1 -- 172.21.15.165:0/3887715865 shutdown_connections 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.319+0000 7fd9cfe00640 1 -- 172.21.15.165:0/3887715865 >> 172.21.15.165:0/3887715865 conn(0x7fd9d00d0610 msgr2=0x7fd9d00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.320+0000 7fd9cfe00640 1 -- 172.21.15.165:0/3887715865 shutdown_connections 2024-04-25T10:02:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.320+0000 7fd9cfe00640 1 -- 172.21.15.165:0/3887715865 wait complete. 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.321+0000 7fd9cfe00640 1 Processor -- start 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.322+0000 7fd9cfe00640 1 -- start start 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.322+0000 7fd9cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b4004800 0x7fd9d00cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.322+0000 7fd9cfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d01f6390 0x7fd9d01f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 0x7fd9d00caf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d006cd20 con 0x7fd9d00caa80 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d006ca40 con 0x7fd9b4004800 2024-04-25T10:02:02.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9d00b5430 con 0x7fd9d01f6390 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 0x7fd9d00caf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 0x7fd9d00caf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50904/0 (socket says 172.21.15.165:50904) 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 -- 172.21.15.165:0/2571810941 learned_addr learned my addr 172.21.15.165:0/2571810941 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cea00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b4004800 0x7fd9d00cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9ce000640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d01f6390 0x7fd9d01f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 -- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d01f6390 msgr2=0x7fd9d01f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d01f6390 0x7fd9d01f6810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 -- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b4004800 msgr2=0x7fd9d00cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b4004800 0x7fd9d00cd3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cf400640 1 -- 172.21.15.165:0/2571810941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9d00cb720 con 0x7fd9d00caa80 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.323+0000 7fd9cea00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b4004800 0x7fd9d00cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.324+0000 7fd9cf400640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 0x7fd9d00caf00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8000f80 tx=0x7fd9c800b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.324+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c800bb80 con 0x7fd9d00caa80 2024-04-25T10:02:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.324+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9c800bd40 con 0x7fd9d00caa80 2024-04-25T10:02:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.324+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9c80072b0 con 0x7fd9d00caa80 2024-04-25T10:02:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.324+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9d00cba00 con 0x7fd9d00caa80 2024-04-25T10:02:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.324+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9d00cbfc0 con 0x7fd9d00caa80 2024-04-25T10:02:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.325+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9d0073990 con 0x7fd9d00caa80 2024-04-25T10:02:02.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.326+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9c801b020 con 0x7fd9d00caa80 2024-04-25T10:02:02.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.327+0000 7fd9c7400640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd9b006b500 0x7fd9b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:02.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.327+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fd9c808a290 con 0x7fd9d00caa80 2024-04-25T10:02:02.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.328+0000 7fd9cea00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd9b006b500 0x7fd9b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:02.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.328+0000 7fd9cea00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd9b006b500 0x7fd9b006d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00016b0 tx=0x7fd9c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:02.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.332+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9c80592f0 con 0x7fd9d00caa80 2024-04-25T10:02:02.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:02.531+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7fd9d00d16c0 con 0x7fd9d00caa80 2024-04-25T10:02:03.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.102+0000 7fd9c7400640 1 -- 172.21.15.165:0/2571810941 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v18) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd9c8058bf0 con 0x7fd9d00caa80 2024-04-25T10:02:03.107 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:02:03.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd9b006b500 msgr2=0x7fd9b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd9b006b500 0x7fd9b006d9c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00016b0 tx=0x7fd9c0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:03.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 msgr2=0x7fd9d00caf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 0x7fd9d00caf00 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fd9c8000f80 tx=0x7fd9c800b4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 shutdown_connections 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd9b006b500 0x7fd9b006d9c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9d00caa80 0x7fd9d00caf00 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9d01f6390 0x7fd9d01f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 --2- 172.21.15.165:0/2571810941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9b4004800 0x7fd9d00cd3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.107+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 >> 172.21.15.165:0/2571810941 conn(0x7fd9d00d0610 msgr2=0x7fd9d015cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.108+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 shutdown_connections 2024-04-25T10:02:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.108+0000 7fd9cfe00640 1 -- 172.21.15.165:0/2571810941 wait complete. 2024-04-25T10:02:03.144 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:02:03.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.316+0000 7fa7a6c00640 1 Processor -- start 2024-04-25T10:02:03.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.316+0000 7fa7a6c00640 1 -- start start 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 0x7fa7a00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a6c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa7a0161710 0x7fa7a00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a6c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7a015e000 con 0x7fa7a00b9330 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a6c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7a00d1de0 con 0x7fa7a0161710 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0161c60 con 0x7fa7a015dc20 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a4c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa7a00b9330 0x7fa7a006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54072/0 (socket says 172.21.15.165:54072) 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2885309876 learned_addr learned my addr 172.21.15.165:0/2885309876 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.318+0000 7fa79fe00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 0x7fa7a00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79f400640 1 --2- 172.21.15.165:0/2885309876 >> v2:172.21.15.172:3300/0 conn(0x7fa7a0161710 0x7fa7a00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455094812 0 0) 0x7fa7a015e000 con 0x7fa7a00b9330 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa780003520 con 0x7fa7a00b9330 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576412044 0 0) 0x7fa780003520 con 0x7fa7a00b9330 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 msgr2=0x7fa7a00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 0x7fa7a00ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 >> v2:172.21.15.172:3300/0 conn(0x7fa7a0161710 msgr2=0x7fa7a00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 --2- 172.21.15.165:0/2885309876 >> v2:172.21.15.172:3300/0 conn(0x7fa7a0161710 0x7fa7a00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a0161f40 con 0x7fa7a00b9330 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa798001670 con 0x7fa7a00b9330 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 >> v1:172.21.15.165:6789/0 conn(0x7fa7a00b9330 legacy=0x7fa7a006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 0x7fa780004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa780004d40 0x7fa7800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00d1de0 con 0x7fa780004d40 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0161c60 con 0x7fa7800039b0 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78000b9c0 con 0x7fa7a015dc20 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.319+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa780004d40 0x7fa7800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa79f400640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 0x7fa780004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa79fe00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 msgr2=0x7fa780004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7a015dc20 0x7fa780004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7800039b0 msgr2=0x7fa780003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2885309876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa780003520 con 0x7fa780004d40 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.320+0000 7fa79fe00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7800039b0 0x7fa780003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.321+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa780004d40 0x7fa7800051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c290 tx=0x7fa780005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1124171bec425554 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.321+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798007d50 con 0x7fa780004d40 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.321+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798005af0 con 0x7fa780004d40 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.321+0000 7fa79ea00640 1 -- 172.21.15.165:0/2885309876 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798007070 con 0x7fa780004d40 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2885309876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa780004d40 msgr2=0x7fa7800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2885309876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa780004d40 0x7fa7800051c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa79800c290 tx=0x7fa780005830 comp rx=0 tx=0).stop 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2885309876 reap_dead start 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2885309876 shutdown_connections 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2885309876 >> 172.21.15.165:0/2885309876 conn(0x7fa7a00d0420 msgr2=0x7fa7a00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2885309876 shutdown_connections 2024-04-25T10:02:03.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.322+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2885309876 wait complete. 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.324+0000 7fa7a6c00640 1 Processor -- start 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.324+0000 7fa7a6c00640 1 -- start start 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa780004d40 0x7fa7a00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a6c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a00caa70 0x7fa7a0150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 0x7fa7a00d3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00b5e50 con 0x7fa7a00d2b80 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a6c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a0162500 con 0x7fa7a00caa70 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a00b5fd0 con 0x7fa780004d40 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 0x7fa7a00d3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 0x7fa7a00d3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50922/0 (socket says 172.21.15.165:50922) 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.325+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2000248190 learned_addr learned my addr 172.21.15.165:0/2000248190 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa79fe00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa780004d40 0x7fa7a00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa79f400640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a00caa70 0x7fa7a0150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa780004d40 msgr2=0x7fa7a00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa780004d40 0x7fa7a00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a00caa70 msgr2=0x7fa7a0150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a00caa70 0x7fa7a0150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a4c00640 1 -- 172.21.15.165:0/2000248190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a00caef0 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa79f400640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a00caa70 0x7fa7a0150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a4c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 0x7fa7a00d3000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa79802ae50 tx=0x7fa7a0162110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa798001c80 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa798044070 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a01f7410 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7980076a0 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.326+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a01f79d0 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.327+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a00d3540 con 0x7fa7a00d2b80 2024-04-25T10:02:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.329+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa79803e480 con 0x7fa7a00d2b80 2024-04-25T10:02:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.330+0000 7fa797e00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa77c06b4d0 0x7fa77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.330+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fa7980aaaf0 con 0x7fa7a00d2b80 2024-04-25T10:02:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.330+0000 7fa79fe00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa77c06b4d0 0x7fa77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.331+0000 7fa79fe00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa77c06b4d0 0x7fa77c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa790009aa0 tx=0x7fa790008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:03.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.335+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7a00caef0 con 0x7fa7a00d2b80 2024-04-25T10:02:03.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:03.568+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa7a0073a00 con 0x7fa7a00d2b80 2024-04-25T10:02:04.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.115+0000 7fa797e00640 1 -- 172.21.15.165:0/2000248190 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa7980792e0 con 0x7fa7a00d2b80 2024-04-25T10:02:04.121 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 2 and data pool 3 2024-04-25T10:02:04.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.127+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa77c06b4d0 msgr2=0x7fa77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.127+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa77c06b4d0 0x7fa77c06d990 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa790009aa0 tx=0x7fa790008040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.127+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 msgr2=0x7fa7a00d3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.127+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 0x7fa7a00d3000 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa79802ae50 tx=0x7fa7a0162110 comp rx=0 tx=0).stop 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.128+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 shutdown_connections 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.128+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa77c06b4d0 0x7fa77c06d990 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa790009aa0 tx=0x7fa790008040 comp rx=0 tx=0).stop 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.128+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7a00d2b80 0x7fa7a00d3000 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.128+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7a00caa70 0x7fa7a0150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.128+0000 7fa7a6c00640 1 --2- 172.21.15.165:0/2000248190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa780004d40 0x7fa7a00cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.128+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 >> 172.21.15.165:0/2000248190 conn(0x7fa7a00d0420 msgr2=0x7fa7a00b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.129+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 shutdown_connections 2024-04-25T10:02:04.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.130+0000 7fa7a6c00640 1 -- 172.21.15.165:0/2000248190 wait complete. 2024-04-25T10:02:04.167 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-25T10:02:04.167 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-25T10:02:04.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.341+0000 7f2102e00640 1 Processor -- start 2024-04-25T10:02:04.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.341+0000 7f2102e00640 1 -- start start 2024-04-25T10:02:04.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.342+0000 7f2102e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f20fc15b7a0 0x7f20fc0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f2102e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0b0b80 con 0x7f20fc0cc150 2024-04-25T10:02:04.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f2102e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20fc0cc800 con 0x7f20fc15b3c0 2024-04-25T10:02:04.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f2102e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20fc0cc5e0 con 0x7f20fc15b7a0 2024-04-25T10:02:04.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f2102400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f20fc0cc150 0x7f20fc15e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54082/0 (socket says 172.21.15.165:54082) 2024-04-25T10:02:04.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f2102400640 1 -- 172.21.15.165:0/627356160 learned_addr learned my addr 172.21.15.165:0/627356160 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2435415536 0 0) 0x7f20f4001490 con 0x7f20fc0cc150 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20e4003520 con 0x7f20fc0cc150 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.343+0000 7f2101000640 1 --2- 172.21.15.165:0/627356160 >> v2:172.21.15.172:3300/0 conn(0x7f20fc15b7a0 0x7f20fc0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3935264757 0 0) 0x7f20e4003520 con 0x7f20fc0cc150 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 >> v2:172.21.15.172:3300/0 conn(0x7f20fc15b7a0 msgr2=0x7f20fc0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 --2- 172.21.15.165:0/627356160 >> v2:172.21.15.172:3300/0 conn(0x7f20fc15b7a0 0x7f20fc0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 >> v1:172.21.15.172:6789/0 conn(0x7f20fc15b3c0 legacy=0x7f20fc0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0ccae0 con 0x7f20fc0cc150 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f20f4001bc0 con 0x7f20fc0cc150 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 >> v1:172.21.15.165:6789/0 conn(0x7f20fc0cc150 legacy=0x7f20fc15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e4004800 0x7f20e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e40052e0 0x7f20e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0cc5e0 con 0x7f20e40039b0 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20e400c1b0 con 0x7f20e40052e0 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20e400c3b0 con 0x7f20e4004800 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f2101000640 1 --2- 172.21.15.165:0/627356160 >> v2:172.21.15.172:3300/0 conn(0x7f20fc15b7a0 0x7f20fc0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f2101a00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.344+0000 7f2102400640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e40052e0 0x7f20e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101000640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e4004800 0x7f20e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101a00640 1 -- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e4004800 msgr2=0x7f20e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101a00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e4004800 0x7f20e4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101a00640 1 -- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e40052e0 msgr2=0x7f20e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101a00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20e40052e0 0x7f20e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2102400640 1 -- 172.21.15.165:0/627356160 reap_dead start 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101a00640 1 -- 172.21.15.165:0/627356160 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20e4003520 con 0x7f20e40039b0 2024-04-25T10:02:04.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.345+0000 7f2101000640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20e4004800 0x7f20e4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.346+0000 7f2101a00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20e4003dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20f0001720 tx=0x7f20e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff0ad1804281299 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.346+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0001370 con 0x7f20e40039b0 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.346+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f00052d0 con 0x7f20e40039b0 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.346+0000 7f20fbe00640 1 -- 172.21.15.165:0/627356160 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f002ab90 con 0x7f20e40039b0 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 -- 172.21.15.165:0/627356160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 msgr2=0x7f20e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20e4003dd0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20f0001720 tx=0x7f20e4005830 comp rx=0 tx=0).stop 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 -- 172.21.15.165:0/627356160 shutdown_connections 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 --2- 172.21.15.165:0/627356160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20e4003dd0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 -- 172.21.15.165:0/627356160 >> 172.21.15.165:0/627356160 conn(0x7f20fc061cc0 msgr2=0x7f20fc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 -- 172.21.15.165:0/627356160 shutdown_connections 2024-04-25T10:02:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.347+0000 7f2102e00640 1 -- 172.21.15.165:0/627356160 wait complete. 2024-04-25T10:02:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.349+0000 7f2102e00640 1 Processor -- start 2024-04-25T10:02:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.349+0000 7f2102e00640 1 -- start start 2024-04-25T10:02:04.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2102e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20fc0c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2102e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc0c93d0 0x7f20fc145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2102e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc145ec0 0x7f20fc146370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2102e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0b0f60 con 0x7f20e40039b0 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2102e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0cd650 con 0x7f20fc0c93d0 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2102e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20fc0b10e0 con 0x7f20fc145ec0 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20fc0c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20fc0c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50936/0 (socket says 172.21.15.165:50936) 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 -- 172.21.15.165:0/3451845711 learned_addr learned my addr 172.21.15.165:0/3451845711 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 -- 172.21.15.165:0/3451845711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc145ec0 msgr2=0x7f20fc146370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc145ec0 0x7f20fc146370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 -- 172.21.15.165:0/3451845711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc0c93d0 msgr2=0x7f20fc145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc0c93d0 0x7f20fc145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.350+0000 7f2101a00640 1 -- 172.21.15.165:0/3451845711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc0c9c00 con 0x7f20e40039b0 2024-04-25T10:02:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.351+0000 7f2101a00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20fc0c8e90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20f0001840 tx=0x7f20f0001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.351+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0037040 con 0x7f20e40039b0 2024-04-25T10:02:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.351+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f0040070 con 0x7f20e40039b0 2024-04-25T10:02:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.351+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20fc0c9ee0 con 0x7f20e40039b0 2024-04-25T10:02:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.351+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f0007a30 con 0x7f20e40039b0 2024-04-25T10:02:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.351+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20fc0ca4a0 con 0x7f20e40039b0 2024-04-25T10:02:04.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.352+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20fc073990 con 0x7f20e40039b0 2024-04-25T10:02:04.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.353+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20f00052d0 con 0x7f20e40039b0 2024-04-25T10:02:04.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.354+0000 7f20f9600640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20dc06b4d0 0x7f20dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:04.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.354+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f20f00a9e40 con 0x7f20e40039b0 2024-04-25T10:02:04.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.354+0000 7f2101000640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20dc06b4d0 0x7f20dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:04.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.355+0000 7f2101000640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20dc06b4d0 0x7f20dc06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20e8006e80 tx=0x7f20e8006e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:04.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.359+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20f0078cf0 con 0x7f20e40039b0 2024-04-25T10:02:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:04.560+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f20fc072360 con 0x7f20e40039b0 2024-04-25T10:02:05.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.124+0000 7f20f9600640 1 -- 172.21.15.165:0/3451845711 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v20) v1 ==== 157+0+0 (secure 0 0 0) 0x7f20f00785f0 con 0x7f20e40039b0 2024-04-25T10:02:05.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20dc06b4d0 msgr2=0x7f20dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20dc06b4d0 0x7f20dc06d990 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f20e8006e80 tx=0x7f20e8006e10 comp rx=0 tx=0).stop 2024-04-25T10:02:05.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 msgr2=0x7f20fc0c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20fc0c8e90 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20f0001840 tx=0x7f20f0001120 comp rx=0 tx=0).stop 2024-04-25T10:02:05.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 shutdown_connections 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20dc06b4d0 0x7f20dc06d990 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20fc145ec0 0x7f20fc146370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20fc0c93d0 0x7f20fc145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 --2- 172.21.15.165:0/3451845711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20e40039b0 0x7f20fc0c8e90 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.128+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 >> 172.21.15.165:0/3451845711 conn(0x7f20fc061cc0 msgr2=0x7f20fc15f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.129+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 shutdown_connections 2024-04-25T10:02:05.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.129+0000 7f2102e00640 1 -- 172.21.15.165:0/3451845711 wait complete. 2024-04-25T10:02:05.155 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:02:05.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.354+0000 7ff94d400640 1 Processor -- start 2024-04-25T10:02:05.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.354+0000 7ff94d400640 1 -- start start 2024-04-25T10:02:05.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff9480cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff94806bff0 0x7ff9480b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94d400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94815e110 con 0x7ff9480b5970 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94806d000 con 0x7ff94806bff0 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480b2fd0 con 0x7ff94815dd30 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94ca00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff9480b5970 0x7ff948161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54096/0 (socket says 172.21.15.165:54096) 2024-04-25T10:02:05.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff94ca00640 1 -- 172.21.15.165:0/3609548847 learned_addr learned my addr 172.21.15.165:0/3609548847 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:05.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff947400640 1 --2- 172.21.15.165:0/3609548847 >> v2:172.21.15.172:3300/0 conn(0x7ff94806bff0 0x7ff9480b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.356+0000 7ff947e00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff9480cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645414663 0 0) 0x7ff94815e110 con 0x7ff9480b5970 2024-04-25T10:02:05.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff92c003520 con 0x7ff9480b5970 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 761393785 0 0) 0x7ff92c003520 con 0x7ff9480b5970 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 msgr2=0x7ff9480cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff9480cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 >> v2:172.21.15.172:3300/0 conn(0x7ff94806bff0 msgr2=0x7ff9480b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 --2- 172.21.15.165:0/3609548847 >> v2:172.21.15.172:3300/0 conn(0x7ff94806bff0 0x7ff9480b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94806c480 con 0x7ff9480b5970 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff9400019f0 con 0x7ff9480b5970 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 >> v1:172.21.15.165:6789/0 conn(0x7ff9480b5970 legacy=0x7ff948161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0039b0 0x7ff92c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff92c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff92c004d40 0x7ff92c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94806d000 con 0x7ff92c004d40 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480b2fd0 con 0x7ff92c0039b0 2024-04-25T10:02:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c00b9c0 con 0x7ff94815dd30 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.357+0000 7ff94ca00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff92c004d40 0x7ff92c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff947e00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0039b0 0x7ff92c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff947400640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff92c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff94ca00640 1 -- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 msgr2=0x7ff92c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff94ca00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff92c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff94ca00640 1 -- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0039b0 msgr2=0x7ff92c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff94ca00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff92c0039b0 0x7ff92c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff94ca00640 1 -- 172.21.15.165:0/3609548847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92c003520 con 0x7ff92c004d40 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.358+0000 7ff947400640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff94815dd30 0x7ff92c004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.359+0000 7ff94ca00640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff92c004d40 0x7ff92c0051c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff94000a7d0 tx=0x7ff92c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d7c64452d45f315 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.359+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94000a3f0 con 0x7ff92c004d40 2024-04-25T10:02:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.359+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff94002f070 con 0x7ff92c004d40 2024-04-25T10:02:05.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.359+0000 7ff946a00640 1 -- 172.21.15.165:0/3609548847 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff94002bc50 con 0x7ff92c004d40 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94d400640 1 -- 172.21.15.165:0/3609548847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff92c004d40 msgr2=0x7ff92c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94d400640 1 --2- 172.21.15.165:0/3609548847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff92c004d40 0x7ff92c0051c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff94000a7d0 tx=0x7ff92c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94ca00640 1 -- 172.21.15.165:0/3609548847 reap_dead start 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94d400640 1 -- 172.21.15.165:0/3609548847 shutdown_connections 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94d400640 1 -- 172.21.15.165:0/3609548847 >> 172.21.15.165:0/3609548847 conn(0x7ff9480d0610 msgr2=0x7ff9480d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94d400640 1 -- 172.21.15.165:0/3609548847 shutdown_connections 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.360+0000 7ff94d400640 1 -- 172.21.15.165:0/3609548847 wait complete. 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.362+0000 7ff94d400640 1 Processor -- start 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.362+0000 7ff94d400640 1 -- start start 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c004d40 0x7ff948152e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 0x7ff9480cc080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948153c00 0x7ff9480cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94806cd20 con 0x7ff948153780 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94806ca40 con 0x7ff948153c00 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480d3170 con 0x7ff92c004d40 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 0x7ff9480cc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 0x7ff9480cc080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50958/0 (socket says 172.21.15.165:50958) 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 -- 172.21.15.165:0/535431527 learned_addr learned my addr 172.21.15.165:0/535431527 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff94ca00640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948153c00 0x7ff9480cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 -- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c004d40 msgr2=0x7ff948152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c004d40 0x7ff948152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 -- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948153c00 msgr2=0x7ff9480cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948153c00 0x7ff9480cc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.363+0000 7ff947400640 1 -- 172.21.15.165:0/535431527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9481541d0 con 0x7ff948153780 2024-04-25T10:02:05.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.364+0000 7ff947400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 0x7ff9480cc080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff934010a00 tx=0x7ff934010ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.364+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93400f040 con 0x7ff948153780 2024-04-25T10:02:05.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.364+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93400ec20 con 0x7ff948153780 2024-04-25T10:02:05.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.364+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9481544b0 con 0x7ff948153780 2024-04-25T10:02:05.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.364+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93401c400 con 0x7ff948153780 2024-04-25T10:02:05.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.364+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff948154a70 con 0x7ff948153780 2024-04-25T10:02:05.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.365+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff948073990 con 0x7ff948153780 2024-04-25T10:02:05.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.366+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff93401c5a0 con 0x7ff948153780 2024-04-25T10:02:05.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.367+0000 7ff93fe00640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92806b4d0 0x7ff92806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:05.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.367+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6007+0+0 (secure 0 0 0) 0x7ff93408ab60 con 0x7ff948153780 2024-04-25T10:02:05.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.367+0000 7ff947e00640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92806b4d0 0x7ff92806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:05.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.368+0000 7ff947e00640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92806b4d0 0x7ff92806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff938009890 tx=0x7ff938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:05.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.371+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff934018030 con 0x7ff948153780 2024-04-25T10:02:05.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:05.572+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff9480d16c0 con 0x7ff948153780 2024-04-25T10:02:07.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.146+0000 7ff93fe00640 1 -- 172.21.15.165:0/535431527 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v22) v1 ==== 228+0+0 (secure 0 0 0) 0x7ff934059810 con 0x7ff948153780 2024-04-25T10:02:07.151 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data_ec' created 2024-04-25T10:02:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.150+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92806b4d0 msgr2=0x7ff92806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.150+0000 7ff94d400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92806b4d0 0x7ff92806d990 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff938009890 tx=0x7ff938008040 comp rx=0 tx=0).stop 2024-04-25T10:02:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 msgr2=0x7ff9480cc080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 0x7ff9480cc080 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff934010a00 tx=0x7ff934010ec0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 shutdown_connections 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92806b4d0 0x7ff92806d990 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948153c00 0x7ff9480cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948153780 0x7ff9480cc080 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 --2- 172.21.15.165:0/535431527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff92c004d40 0x7ff948152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 >> 172.21.15.165:0/535431527 conn(0x7ff9480d0610 msgr2=0x7ff9480d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 shutdown_connections 2024-04-25T10:02:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.151+0000 7ff94d400640 1 -- 172.21.15.165:0/535431527 wait complete. 2024-04-25T10:02:07.186 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-25T10:02:07.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.359+0000 7fd1c7200640 1 Processor -- start 2024-04-25T10:02:07.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.359+0000 7fd1c7200640 1 -- start start 2024-04-25T10:02:07.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.360+0000 7fd1c7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd1c006ea90 0x7fd1c0071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.360+0000 7fd1c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c006d2d0 0x7fd1c006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.360+0000 7fd1c7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c0162ba0 con 0x7fd1c0161fc0 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.360+0000 7fd1c7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c006dc80 con 0x7fd1c006ea90 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.360+0000 7fd1c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0165be0 con 0x7fd1c006d2d0 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd1c006ea90 0x7fd1c0071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd1c006ea90 0x7fd1c0071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52666/0 (socket says 172.21.15.165:52666) 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 -- 172.21.15.165:0/1388275283 learned_addr learned my addr 172.21.15.165:0/1388275283 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 -- 172.21.15.165:0/1388275283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c006d2d0 msgr2=0x7fd1c006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 --2- 172.21.15.165:0/1388275283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c006d2d0 0x7fd1c006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 -- 172.21.15.165:0/1388275283 >> v1:172.21.15.172:6789/0 conn(0x7fd1c0161fc0 legacy=0x7fd1c01623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:07.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.361+0000 7fd1bf400640 1 -- 172.21.15.165:0/1388275283 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0162e80 con 0x7fd1c006ea90 2024-04-25T10:02:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.362+0000 7fd1bf400640 1 --2- 172.21.15.165:0/1388275283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c0071480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098a0 tx=0x7fd1a802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9659b9435ff13da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.363+0000 7fd1bea00640 1 -- 172.21.15.165:0/1388275283 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8037070 con 0x7fd1c006ea90 2024-04-25T10:02:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.363+0000 7fd1bea00640 1 -- 172.21.15.165:0/1388275283 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a8032440 con 0x7fd1c006ea90 2024-04-25T10:02:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.363+0000 7fd1bea00640 1 -- 172.21.15.165:0/1388275283 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8032810 con 0x7fd1c006ea90 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.363+0000 7fd1c7200640 1 -- 172.21.15.165:0/1388275283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 msgr2=0x7fd1c0071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.363+0000 7fd1c7200640 1 --2- 172.21.15.165:0/1388275283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c0071480 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd1a80098a0 tx=0x7fd1a802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.364+0000 7fd1c7200640 1 -- 172.21.15.165:0/1388275283 shutdown_connections 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.364+0000 7fd1c7200640 1 --2- 172.21.15.165:0/1388275283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c006d2d0 0x7fd1c006d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.364+0000 7fd1c7200640 1 --2- 172.21.15.165:0/1388275283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c0071480 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.364+0000 7fd1c7200640 1 -- 172.21.15.165:0/1388275283 >> 172.21.15.165:0/1388275283 conn(0x7fd1c006abe0 msgr2=0x7fd1c006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.364+0000 7fd1c7200640 1 -- 172.21.15.165:0/1388275283 shutdown_connections 2024-04-25T10:02:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.364+0000 7fd1c7200640 1 -- 172.21.15.165:0/1388275283 wait complete. 2024-04-25T10:02:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.366+0000 7fd1c7200640 1 Processor -- start 2024-04-25T10:02:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.366+0000 7fd1c7200640 1 -- start start 2024-04-25T10:02:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1c7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c006d2d0 0x7fd1c014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1c7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c014cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c0161fc0 0x7fd1c01499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1c7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c00719c0 con 0x7fd1c006ea90 2024-04-25T10:02:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1c7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0163440 con 0x7fd1c006d2d0 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c0071b40 con 0x7fd1c0161fc0 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.367+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c014cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c014cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52670/0 (socket says 172.21.15.165:52670) 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 -- 172.21.15.165:0/3663967958 learned_addr learned my addr 172.21.15.165:0/3663967958 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bfe00640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c006d2d0 0x7fd1c014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 -- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c0161fc0 msgr2=0x7fd1c01499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1c5200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c0161fc0 0x7fd1c01499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c0161fc0 0x7fd1c01499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 -- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c006d2d0 msgr2=0x7fd1c014a1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c006d2d0 0x7fd1c014a1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 -- 172.21.15.165:0/3663967958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c0149f10 con 0x7fd1c006ea90 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1c5200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c0161fc0 0x7fd1c01499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.368+0000 7fd1bf400640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c014cd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.369+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a8037040 con 0x7fd1c006ea90 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.369+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1a8004230 con 0x7fd1c006ea90 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.369+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1a803c730 con 0x7fd1c006ea90 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.369+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c014a920 con 0x7fd1c006ea90 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.369+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c014aee0 con 0x7fd1c006ea90 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.370+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c0073a00 con 0x7fd1c006ea90 2024-04-25T10:02:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.371+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1a803a070 con 0x7fd1c006ea90 2024-04-25T10:02:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.372+0000 7fd1b7e00640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.372+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6500+0+0 (secure 0 0 0) 0x7fd1a80aaec0 con 0x7fd1c006ea90 2024-04-25T10:02:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.373+0000 7fd1bfe00640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.373+0000 7fd1bfe00640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd19c06b4d0 0x7fd19c06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006fd0 tx=0x7fd1b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.377+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1c0149f10 con 0x7fd1c006ea90 2024-04-25T10:02:07.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:07.569+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7fd1c006b160 con 0x7fd1c006ea90 2024-04-25T10:02:08.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.151+0000 7fd1b7e00640 1 -- 172.21.15.165:0/3663967958 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v23) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd1a8041020 con 0x7fd1c006ea90 2024-04-25T10:02:08.156 INFO:teuthology.orchestra.run.smithi165.stderr:set pool 4 allow_ec_overwrites to true 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd19c06b4d0 msgr2=0x7fd19c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd19c06b4d0 0x7fd19c06d990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fd1b0006fd0 tx=0x7fd1b0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 msgr2=0x7fd1c014cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c014cd20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd1a8000c00 tx=0x7fd1a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 shutdown_connections 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd19c06b4d0 0x7fd19c06d990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c0161fc0 0x7fd1c01499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c006ea90 0x7fd1c014cd20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.154+0000 7fd1c7200640 1 --2- 172.21.15.165:0/3663967958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c006d2d0 0x7fd1c014a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.155+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 >> 172.21.15.165:0/3663967958 conn(0x7fd1c006abe0 msgr2=0x7fd1c015dac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:08.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.155+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 shutdown_connections 2024-04-25T10:02:08.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.156+0000 7fd1c7200640 1 -- 172.21.15.165:0/3663967958 wait complete. 2024-04-25T10:02:08.201 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-25T10:02:08.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.368+0000 7f8304000640 1 Processor -- start 2024-04-25T10:02:08.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.368+0000 7f8304000640 1 -- start start 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8304000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f82fc161fc0 0x7f82fc1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8304000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f82fc06ea90 0x7f82fc071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8304000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82fc1628e0 con 0x7f82fc06d2d0 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8304000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc165be0 con 0x7f82fc06ea90 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8304000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82fc162ba0 con 0x7f82fc161fc0 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8302000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f82fc06d2d0 0x7f82fc06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58784/0 (socket says 172.21.15.165:58784) 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8300c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f82fc06ea90 0x7f82fc071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.370+0000 7f8302000640 1 -- 172.21.15.165:0/503893317 learned_addr learned my addr 172.21.15.165:0/503893317 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:08.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.371+0000 7f8301600640 1 --2- 172.21.15.165:0/503893317 >> v2:172.21.15.172:3300/0 conn(0x7f82fc161fc0 0x7f82fc1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.371+0000 7f8300c00640 1 -- 172.21.15.165:0/503893317 >> v2:172.21.15.172:3300/0 conn(0x7f82fc161fc0 msgr2=0x7f82fc1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:08.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.371+0000 7f8300c00640 1 --2- 172.21.15.165:0/503893317 >> v2:172.21.15.172:3300/0 conn(0x7f82fc161fc0 0x7f82fc1623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.371+0000 7f8300c00640 1 -- 172.21.15.165:0/503893317 >> v1:172.21.15.165:6789/0 conn(0x7f82fc06d2d0 legacy=0x7f82fc06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.371+0000 7f8300c00640 1 -- 172.21.15.165:0/503893317 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc162e80 con 0x7f82fc06ea90 2024-04-25T10:02:08.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.371+0000 7f82fbe00640 1 -- 172.21.15.165:0/503893317 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1244647421 0 0) 0x7f82fc1628e0 con 0x7f82fc06d2d0 2024-04-25T10:02:08.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.372+0000 7f8300c00640 1 --2- 172.21.15.165:0/503893317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc071480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82fc06ba40 tx=0x7f82e802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f896a9f0f48743ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.372+0000 7f82fbe00640 1 -- 172.21.15.165:0/503893317 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8037070 con 0x7f82fc06ea90 2024-04-25T10:02:08.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.372+0000 7f82fbe00640 1 -- 172.21.15.165:0/503893317 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8032440 con 0x7f82fc06ea90 2024-04-25T10:02:08.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.372+0000 7f82fbe00640 1 -- 172.21.15.165:0/503893317 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e802ae60 con 0x7f82fc06ea90 2024-04-25T10:02:08.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 -- 172.21.15.165:0/503893317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 msgr2=0x7f82fc071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 --2- 172.21.15.165:0/503893317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc071480 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f82fc06ba40 tx=0x7f82e802a110 comp rx=0 tx=0).stop 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 -- 172.21.15.165:0/503893317 shutdown_connections 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 --2- 172.21.15.165:0/503893317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc071480 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 --2- 172.21.15.165:0/503893317 >> v2:172.21.15.172:3300/0 conn(0x7f82fc161fc0 0x7f82fc1623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 -- 172.21.15.165:0/503893317 >> 172.21.15.165:0/503893317 conn(0x7f82fc06abe0 msgr2=0x7f82fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 -- 172.21.15.165:0/503893317 shutdown_connections 2024-04-25T10:02:08.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.373+0000 7f8304000640 1 -- 172.21.15.165:0/503893317 wait complete. 2024-04-25T10:02:08.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.375+0000 7f8304000640 1 Processor -- start 2024-04-25T10:02:08.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.375+0000 7f8304000640 1 -- start start 2024-04-25T10:02:08.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8304000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc06d2d0 0x7f82fc1f7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8304000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc0af440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8304000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc161fc0 0x7f82fc0af980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8304000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc0719c0 con 0x7f82fc06ea90 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8304000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc163440 con 0x7f82fc161fc0 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8304000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc071b40 con 0x7f82fc06d2d0 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc0af440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc0af440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52686/0 (socket says 172.21.15.165:52686) 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 -- 172.21.15.165:0/2884631283 learned_addr learned my addr 172.21.15.165:0/2884631283 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8301600640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc06d2d0 0x7f82fc1f7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8302000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc161fc0 0x7f82fc0af980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 -- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc06d2d0 msgr2=0x7f82fc1f7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc06d2d0 0x7f82fc1f7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 -- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc161fc0 msgr2=0x7f82fc0af980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc161fc0 0x7f82fc0af980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8300c00640 1 -- 172.21.15.165:0/2884631283 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc14f6a0 con 0x7f82fc06ea90 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.376+0000 7f8301600640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc06d2d0 0x7f82fc1f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.377+0000 7f8300c00640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc0af440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000c00 tx=0x7f82e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.377+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8037050 con 0x7f82fc06ea90 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.377+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e8038040 con 0x7f82fc06ea90 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.377+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8004470 con 0x7f82fc06ea90 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.377+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82fc1fbfe0 con 0x7f82fc06ea90 2024-04-25T10:02:08.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.377+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82fc14fc50 con 0x7f82fc06ea90 2024-04-25T10:02:08.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.378+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82fc073a00 con 0x7f82fc06ea90 2024-04-25T10:02:08.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.379+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f82e8030900 con 0x7f82fc06ea90 2024-04-25T10:02:08.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.381+0000 7f82f9600640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82dc06b4d0 0x7f82dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:08.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.381+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6500+0+0 (secure 0 0 0) 0x7f82e80aaf10 con 0x7f82fc06ea90 2024-04-25T10:02:08.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.381+0000 7f8301600640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82dc06b4d0 0x7f82dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:08.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.381+0000 7f8301600640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82dc06b4d0 0x7f82dc06d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f82f0006fd0 tx=0x7f82f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:08.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.385+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f82e8079c20 con 0x7f82fc06ea90 2024-04-25T10:02:08.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:08.621+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f82fc163c40 con 0x7f82fc06ea90 2024-04-25T10:02:09.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.187+0000 7f82f9600640 1 -- 172.21.15.165:0/2884631283 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f82e8079520 con 0x7f82fc06ea90 2024-04-25T10:02:09.192 INFO:teuthology.orchestra.run.smithi165.stderr:added data pool 4 to fsmap 2024-04-25T10:02:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.190+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82dc06b4d0 msgr2=0x7f82dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.190+0000 7f8304000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82dc06b4d0 0x7f82dc06d990 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f82f0006fd0 tx=0x7f82f0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.190+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 msgr2=0x7f82fc0af440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.190+0000 7f8304000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc0af440 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f82e8000c00 tx=0x7f82e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 shutdown_connections 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82dc06b4d0 0x7f82dc06d990 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc161fc0 0x7f82fc0af980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc06ea90 0x7f82fc0af440 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 --2- 172.21.15.165:0/2884631283 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc06d2d0 0x7f82fc1f7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 >> 172.21.15.165:0/2884631283 conn(0x7f82fc06abe0 msgr2=0x7f82fc161060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 shutdown_connections 2024-04-25T10:02:09.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.191+0000 7f8304000640 1 -- 172.21.15.165:0/2884631283 wait complete. 2024-04-25T10:02:09.214 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:09.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.406+0000 7f7275400640 1 Processor -- start 2024-04-25T10:02:09.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.406+0000 7f7275400640 1 -- start start 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.407+0000 7f7275400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015db90 0x7f72700b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7275400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f72700d06c0 0x7f72700d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7275400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7270161df0 con 0x7f72701615a0 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7275400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72700d1f50 con 0x7f72700d06c0 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7275400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72701620d0 con 0x7f727015db90 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7267400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f72701615a0 0x7f72700cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58786/0 (socket says 172.21.15.165:58786) 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7267400640 1 -- 172.21.15.165:0/1819744615 learned_addr learned my addr 172.21.15.165:0/1819744615 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008295734 0 0) 0x7f7270161df0 con 0x7f72701615a0 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f726fe00640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015db90 0x7f72700b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7254003520 con 0x7f72701615a0 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.408+0000 7f7274a00640 1 --2- 172.21.15.165:0/1819744615 >> v2:172.21.15.172:3300/0 conn(0x7f72700d06c0 0x7f72700d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3699557014 0 0) 0x7f7254003520 con 0x7f72701615a0 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015db90 msgr2=0x7f72700b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015db90 0x7f72700b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 >> v2:172.21.15.172:3300/0 conn(0x7f72700d06c0 msgr2=0x7f72700d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 --2- 172.21.15.165:0/1819744615 >> v2:172.21.15.172:3300/0 conn(0x7f72700d06c0 0x7f72700d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72701623b0 con 0x7f72701615a0 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7258001670 con 0x7f72701615a0 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 >> v1:172.21.15.165:6789/0 conn(0x7f72701615a0 legacy=0x7f72700cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72540039b0 0x7f7254003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f7254004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72540051c0 0x7f725400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72700d1f50 con 0x7f7254004800 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72701620d0 con 0x7f72540039b0 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f725400c000 con 0x7f72540051c0 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7274a00640 1 --2- 172.21.15.165:0/1819744615 >> v2:172.21.15.172:3300/0 conn(0x7f72700d06c0 0x7f72700d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f726fe00640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015db90 0x7f72700b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7267400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f7254004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7267400640 1 -- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72540051c0 msgr2=0x7f725400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7267400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72540051c0 0x7f725400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7267400640 1 -- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72540039b0 msgr2=0x7f7254003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7267400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72540039b0 0x7f7254003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.409+0000 7f7267400640 1 -- 172.21.15.165:0/1819744615 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7254003520 con 0x7f7254004800 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.410+0000 7f7267400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f7254004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f725800c240 tx=0x7f7254005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233cb59240aec2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258007d50 con 0x7f7254004800 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725802e070 con 0x7f7254004800 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f726f400640 1 -- 172.21.15.165:0/1819744615 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7258005d30 con 0x7f7254004800 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7275400640 1 -- 172.21.15.165:0/1819744615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 msgr2=0x7f7254004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7275400640 1 --2- 172.21.15.165:0/1819744615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f7254004c80 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f725800c240 tx=0x7f7254005830 comp rx=0 tx=0).stop 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7274a00640 1 -- 172.21.15.165:0/1819744615 reap_dead start 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7275400640 1 -- 172.21.15.165:0/1819744615 shutdown_connections 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7275400640 1 -- 172.21.15.165:0/1819744615 >> 172.21.15.165:0/1819744615 conn(0x7f72700b52b0 msgr2=0x7f72700b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7275400640 1 -- 172.21.15.165:0/1819744615 shutdown_connections 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.411+0000 7f7275400640 1 -- 172.21.15.165:0/1819744615 wait complete. 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.413+0000 7f7275400640 1 Processor -- start 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.413+0000 7f7275400640 1 -- start start 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f7275400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f727014e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f7275400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72700cc4f0 0x7f72700cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f7275400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015ff50 0x7f72701603d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f7275400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7270161ba0 con 0x7f7254004800 2024-04-25T10:02:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f7275400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7270161a20 con 0x7f72700cc4f0 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f7275400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72700b68d0 con 0x7f727015ff50 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f726fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f727014e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f726fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f727014e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52708/0 (socket says 172.21.15.165:52708) 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.414+0000 7f726fe00640 1 -- 172.21.15.165:0/1756841665 learned_addr learned my addr 172.21.15.165:0/1756841665 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f7267400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72700cc4f0 0x7f72700cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726fe00640 1 -- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015ff50 msgr2=0x7f72701603d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726fe00640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015ff50 0x7f72701603d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726fe00640 1 -- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72700cc4f0 msgr2=0x7f72700cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726fe00640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72700cc4f0 0x7f72700cc970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726fe00640 1 -- 172.21.15.165:0/1756841665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72700cb450 con 0x7f7254004800 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f7267400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72700cc4f0 0x7f72700cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:09.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726fe00640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f727014e9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7260007e20 tx=0x7f726000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726001a070 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7260003b90 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726000de20 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.415+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72700cb730 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.416+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72700cbcf0 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.417+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7270073990 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.418+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f726001d070 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.418+0000 7f726cc00640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f724c06b4d0 0x7f724c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.419+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f726008a770 con 0x7f7254004800 2024-04-25T10:02:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.419+0000 7f7267400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f724c06b4d0 0x7f724c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.419+0000 7f7267400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f724c06b4d0 0x7f724c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7258033000 tx=0x7f7258009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:09.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.423+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7260010070 con 0x7f7254004800 2024-04-25T10:02:09.488 INFO:tasks.ceph.mds.b.smithi172.stderr:2024-04-25T10:02:09.484+0000 7f8927c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:02:09.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.655+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72700d14b0 con 0x7f7254004800 2024-04-25T10:02:09.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.656+0000 7f726cc00640 1 -- 172.21.15.165:0/1756841665 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f7260059210 con 0x7f7254004800 2024-04-25T10:02:09.661 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:09.661 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:08.622677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:02:09.661 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 7 2024-04-25T10:02:09.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.658+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f724c06b4d0 msgr2=0x7f724c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.658+0000 7f7275400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f724c06b4d0 0x7f724c06d990 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7258033000 tx=0x7f7258009030 comp rx=0 tx=0).stop 2024-04-25T10:02:09.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.658+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 msgr2=0x7f727014e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.658+0000 7f7275400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f727014e9c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7260007e20 tx=0x7f726000d4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 shutdown_connections 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f724c06b4d0 0x7f724c06d990 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f727015ff50 0x7f72701603d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72700cc4f0 0x7f72700cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 --2- 172.21.15.165:0/1756841665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7254004800 0x7f727014e9c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 >> 172.21.15.165:0/1756841665 conn(0x7f72700b52b0 msgr2=0x7f7270157a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 shutdown_connections 2024-04-25T10:02:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.659+0000 7f7275400640 1 -- 172.21.15.165:0/1756841665 wait complete. 2024-04-25T10:02:09.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:09.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.870+0000 7f0d77800640 1 Processor -- start 2024-04-25T10:02:09.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.871+0000 7f0d77800640 1 -- start start 2024-04-25T10:02:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d77800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0d70154ee0 0x7f0d700cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d77800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d700cfcf0 0x7f0d700cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d77800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700ce3c0 0x7f0d7015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d77800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d700ce830 con 0x7f0d700cfcf0 2024-04-25T10:02:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d77800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d70072090 con 0x7f0d70154ee0 2024-04-25T10:02:09.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d77800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70071230 con 0x7f0d700ce3c0 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.872+0000 7f0d6fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d700cfcf0 0x7f0d700cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d700cfcf0 0x7f0d700cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52724/0 (socket says 172.21.15.165:52724) 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 -- 172.21.15.165:0/1990049566 learned_addr learned my addr 172.21.15.165:0/1990049566 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d74e00640 1 --2- 172.21.15.165:0/1990049566 >> v2:172.21.15.172:3300/0 conn(0x7f0d70154ee0 0x7f0d700cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d75800640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700ce3c0 0x7f0d7015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 -- 172.21.15.165:0/1990049566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700ce3c0 msgr2=0x7f0d7015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700ce3c0 0x7f0d7015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 -- 172.21.15.165:0/1990049566 >> v2:172.21.15.172:3300/0 conn(0x7f0d70154ee0 msgr2=0x7f0d700cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 --2- 172.21.15.165:0/1990049566 >> v2:172.21.15.172:3300/0 conn(0x7f0d70154ee0 0x7f0d700cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d6fe00640 1 -- 172.21.15.165:0/1990049566 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70071480 con 0x7f0d700cfcf0 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d75800640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700ce3c0 0x7f0d7015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:09.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.873+0000 7f0d74e00640 1 --2- 172.21.15.165:0/1990049566 >> v2:172.21.15.172:3300/0 conn(0x7f0d70154ee0 0x7f0d700cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:09.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.874+0000 7f0d6fe00640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700cfcf0 0x7f0d700cde80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d700d02e0 tx=0x7f0d5c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491892e5b447386a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:09.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.874+0000 7f0d6f400640 1 -- 172.21.15.165:0/1990049566 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c032790 con 0x7f0d700cfcf0 2024-04-25T10:02:09.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.874+0000 7f0d6f400640 1 -- 172.21.15.165:0/1990049566 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c038070 con 0x7f0d700cfcf0 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.874+0000 7f0d6f400640 1 -- 172.21.15.165:0/1990049566 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c032b50 con 0x7f0d700cfcf0 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 -- 172.21.15.165:0/1990049566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700cfcf0 msgr2=0x7f0d700cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700cfcf0 0x7f0d700cde80 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f0d700d02e0 tx=0x7f0d5c02c890 comp rx=0 tx=0).stop 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 -- 172.21.15.165:0/1990049566 shutdown_connections 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700ce3c0 0x7f0d7015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 --2- 172.21.15.165:0/1990049566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700cfcf0 0x7f0d700cde80 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 --2- 172.21.15.165:0/1990049566 >> v2:172.21.15.172:3300/0 conn(0x7f0d70154ee0 0x7f0d700cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 -- 172.21.15.165:0/1990049566 >> 172.21.15.165:0/1990049566 conn(0x7f0d7006abe0 msgr2=0x7f0d7006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 -- 172.21.15.165:0/1990049566 shutdown_connections 2024-04-25T10:02:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.875+0000 7f0d77800640 1 -- 172.21.15.165:0/1990049566 wait complete. 2024-04-25T10:02:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.877+0000 7f0d77800640 1 Processor -- start 2024-04-25T10:02:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.877+0000 7f0d77800640 1 -- start start 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d77800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 0x7f0d700a9640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d77800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700cfcf0 0x7f0d701489e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d77800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d70154ee0 0x7f0d70148040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d77800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70071d20 con 0x7f0d700ce3c0 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d77800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d70071a40 con 0x7f0d70154ee0 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d77800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d701552c0 con 0x7f0d700cfcf0 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 0x7f0d700a9640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 0x7f0d700a9640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52740/0 (socket says 172.21.15.165:52740) 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 -- 172.21.15.165:0/1982216607 learned_addr learned my addr 172.21.15.165:0/1982216607 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d6fe00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700cfcf0 0x7f0d701489e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 -- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700cfcf0 msgr2=0x7f0d701489e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700cfcf0 0x7f0d701489e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 -- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d70154ee0 msgr2=0x7f0d70148040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:09.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d70154ee0 0x7f0d70148040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:09.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.878+0000 7f0d74e00640 1 -- 172.21.15.165:0/1982216607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d70148f20 con 0x7f0d700ce3c0 2024-04-25T10:02:09.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.879+0000 7f0d74e00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 0x7f0d700a9640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c7f0 tx=0x7f0d6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:09.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.879+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64015070 con 0x7f0d700ce3c0 2024-04-25T10:02:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.879+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64007820 con 0x7f0d700ce3c0 2024-04-25T10:02:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.879+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64007bd0 con 0x7f0d700ce3c0 2024-04-25T10:02:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.879+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d700ae310 con 0x7f0d700ce3c0 2024-04-25T10:02:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.879+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d700a9fe0 con 0x7f0d700ce3c0 2024-04-25T10:02:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.880+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d70073990 con 0x7f0d700ce3c0 2024-04-25T10:02:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.881+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d64018070 con 0x7f0d700ce3c0 2024-04-25T10:02:09.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.882+0000 7f0d6cc00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5006b500 0x7f0d5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:09.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.883+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0d6408a690 con 0x7f0d700ce3c0 2024-04-25T10:02:09.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.883+0000 7f0d6fe00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5006b500 0x7f0d5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:09.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.883+0000 7f0d6fe00640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5006b500 0x7f0d5006d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c037000 tx=0x7f0d5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:09.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:09.887+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d64059130 con 0x7f0d700ce3c0 2024-04-25T10:02:10.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.066+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d70148580 con 0x7f0d700ce3c0 2024-04-25T10:02:10.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.068+0000 7f0d6cc00640 1 -- 172.21.15.165:0/1982216607 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f0d64058a30 con 0x7f0d700ce3c0 2024-04-25T10:02:10.073 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:10.073 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5006b500 msgr2=0x7f0d5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5006b500 0x7f0d5006d9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c037000 tx=0x7f0d5c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 msgr2=0x7f0d700a9640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 0x7f0d700a9640 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f0d6400c7f0 tx=0x7f0d6400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 shutdown_connections 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5006b500 0x7f0d5006d9c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d70154ee0 0x7f0d70148040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d700cfcf0 0x7f0d701489e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 --2- 172.21.15.165:0/1982216607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d700ce3c0 0x7f0d700a9640 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 >> 172.21.15.165:0/1982216607 conn(0x7f0d7006abe0 msgr2=0x7f0d7015d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 shutdown_connections 2024-04-25T10:02:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.071+0000 7f0d77800640 1 -- 172.21.15.165:0/1982216607 wait complete. 2024-04-25T10:02:10.096 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:10.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.286+0000 7f692fe00640 1 Processor -- start 2024-04-25T10:02:10.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.286+0000 7f692fe00640 1 -- start start 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.287+0000 7f692fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6930071700 0x7f69300cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.287+0000 7f692fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f69300cd860 0x7f693015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.287+0000 7f692fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f693015bf30 con 0x7f6930144440 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.287+0000 7f692fe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693015c860 con 0x7f69300cd860 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.287+0000 7f692fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f693015c210 con 0x7f6930071700 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6930144440 0x7f6930070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58798/0 (socket says 172.21.15.165:58798) 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692ea00640 1 -- 172.21.15.165:0/2323886807 learned_addr learned my addr 172.21.15.165:0/2323886807 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:10.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692f400640 1 --2- 172.21.15.165:0/2323886807 >> v2:172.21.15.165:3300/0 conn(0x7f69300cd860 0x7f693015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692d600640 1 -- 172.21.15.165:0/2323886807 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278563702 0 0) 0x7f693015bf30 con 0x7f6930144440 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692e000640 1 --2- 172.21.15.165:0/2323886807 >> v2:172.21.15.172:3300/0 conn(0x7f6930071700 0x7f69300cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692d600640 1 -- 172.21.15.165:0/2323886807 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6914003520 con 0x7f6930144440 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692f400640 1 -- 172.21.15.165:0/2323886807 >> v2:172.21.15.172:3300/0 conn(0x7f6930071700 msgr2=0x7f69300cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692f400640 1 --2- 172.21.15.165:0/2323886807 >> v2:172.21.15.172:3300/0 conn(0x7f6930071700 0x7f69300cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692f400640 1 -- 172.21.15.165:0/2323886807 >> v1:172.21.15.165:6789/0 conn(0x7f6930144440 legacy=0x7f6930070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692f400640 1 -- 172.21.15.165:0/2323886807 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f693015c4f0 con 0x7f69300cd860 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.288+0000 7f692e000640 1 --2- 172.21.15.165:0/2323886807 >> v2:172.21.15.172:3300/0 conn(0x7f6930071700 0x7f69300cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.289+0000 7f692f400640 1 --2- 172.21.15.165:0/2323886807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69300cd860 0x7f693015ccf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f693006b970 tx=0x7f692802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9404b2bf30d60d3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.289+0000 7f692d600640 1 -- 172.21.15.165:0/2323886807 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928007740 con 0x7f69300cd860 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.289+0000 7f692d600640 1 -- 172.21.15.165:0/2323886807 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69280078e0 con 0x7f69300cd860 2024-04-25T10:02:10.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.289+0000 7f692d600640 1 -- 172.21.15.165:0/2323886807 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928007cb0 con 0x7f69300cd860 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 -- 172.21.15.165:0/2323886807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69300cd860 msgr2=0x7f693015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 --2- 172.21.15.165:0/2323886807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69300cd860 0x7f693015ccf0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f693006b970 tx=0x7f692802a710 comp rx=0 tx=0).stop 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 -- 172.21.15.165:0/2323886807 shutdown_connections 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 --2- 172.21.15.165:0/2323886807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69300cd860 0x7f693015ccf0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 --2- 172.21.15.165:0/2323886807 >> v2:172.21.15.172:3300/0 conn(0x7f6930071700 0x7f69300cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 -- 172.21.15.165:0/2323886807 >> 172.21.15.165:0/2323886807 conn(0x7f693006abe0 msgr2=0x7f693006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 -- 172.21.15.165:0/2323886807 shutdown_connections 2024-04-25T10:02:10.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.290+0000 7f692fe00640 1 -- 172.21.15.165:0/2323886807 wait complete. 2024-04-25T10:02:10.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.292+0000 7f692fe00640 1 Processor -- start 2024-04-25T10:02:10.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.292+0000 7f692fe00640 1 -- start start 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6930071700 0x7f6930140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69300cd860 0x7f69300cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 0x7f69300cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69300cde70 con 0x7f6930144440 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69300cdcf0 con 0x7f69300cd860 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f693015d230 con 0x7f6930071700 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 0x7f69300cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 0x7f69300cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52766/0 (socket says 172.21.15.165:52766) 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692f400640 1 -- 172.21.15.165:0/4277720382 learned_addr learned my addr 172.21.15.165:0/4277720382 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692e000640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69300cd860 0x7f69300cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.293+0000 7f692ea00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6930071700 0x7f6930140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692f400640 1 -- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6930071700 msgr2=0x7f6930140040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692f400640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6930071700 0x7f6930140040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692f400640 1 -- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69300cd860 msgr2=0x7f69300cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692f400640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69300cd860 0x7f69300cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692f400640 1 -- 172.21.15.165:0/4277720382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69300c7bc0 con 0x7f6930144440 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692e000640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69300cd860 0x7f69300cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692ea00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6930071700 0x7f6930140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692f400640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 0x7f69300cb550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6928009e50 tx=0x7f6928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928040070 con 0x7f6930144440 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928031bf0 con 0x7f6930144440 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928009450 con 0x7f6930144440 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69301f1f60 con 0x7f6930144440 2024-04-25T10:02:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.294+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69301f2520 con 0x7f6930144440 2024-04-25T10:02:10.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.295+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6930073990 con 0x7f6930144440 2024-04-25T10:02:10.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.296+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6928039040 con 0x7f6930144440 2024-04-25T10:02:10.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.298+0000 7f6926a00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f691006b4d0 0x7f691006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.298+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f69280ab7a0 con 0x7f6930144440 2024-04-25T10:02:10.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.298+0000 7f692ea00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f691006b4d0 0x7f691006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.298+0000 7f692ea00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f691006b4d0 0x7f691006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6920005880 tx=0x7f6920009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.302+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f692804c070 con 0x7f6930144440 2024-04-25T10:02:10.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.494+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69301e2350 con 0x7f6930144440 2024-04-25T10:02:10.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.496+0000 7f6926a00640 1 -- 172.21.15.165:0/4277720382 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f692807a240 con 0x7f6930144440 2024-04-25T10:02:10.501 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:10.501 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.498+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f691006b4d0 msgr2=0x7f691006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.498+0000 7f692fe00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f691006b4d0 0x7f691006d990 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6920005880 tx=0x7f6920009040 comp rx=0 tx=0).stop 2024-04-25T10:02:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 msgr2=0x7f69300cb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 0x7f69300cb550 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6928009e50 tx=0x7f6928004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 shutdown_connections 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f691006b4d0 0x7f691006d990 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6930144440 0x7f69300cb550 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69300cd860 0x7f69300cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 --2- 172.21.15.165:0/4277720382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6930071700 0x7f6930140040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 >> 172.21.15.165:0/4277720382 conn(0x7f693006abe0 msgr2=0x7f69300b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 shutdown_connections 2024-04-25T10:02:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.499+0000 7f692fe00640 1 -- 172.21.15.165:0/4277720382 wait complete. 2024-04-25T10:02:10.535 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:10.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.732+0000 7feb36800640 1 Processor -- start 2024-04-25T10:02:10.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.732+0000 7feb36800640 1 -- start start 2024-04-25T10:02:10.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb36800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7feb30144440 0x7feb30070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb36800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300cda30 0x7feb3015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb36800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3015bf30 con 0x7feb30071530 2024-04-25T10:02:10.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb36800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb300719a0 con 0x7feb30144440 2024-04-25T10:02:10.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb36800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3015c860 con 0x7feb300cda30 2024-04-25T10:02:10.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb35400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7feb30144440 0x7feb30070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb34a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7feb30071530 0x7feb300cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58800/0 (socket says 172.21.15.165:58800) 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb34a00640 1 -- 172.21.15.165:0/1703485764 learned_addr learned my addr 172.21.15.165:0/1703485764 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.734+0000 7feb35e00640 1 --2- 172.21.15.165:0/1703485764 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300cda30 0x7feb3015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.735+0000 7feb35400640 1 -- 172.21.15.165:0/1703485764 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300cda30 msgr2=0x7feb3015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.735+0000 7feb35400640 1 --2- 172.21.15.165:0/1703485764 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300cda30 0x7feb3015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.735+0000 7feb35400640 1 -- 172.21.15.165:0/1703485764 >> v1:172.21.15.165:6789/0 conn(0x7feb30071530 legacy=0x7feb300cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.735+0000 7feb35400640 1 -- 172.21.15.165:0/1703485764 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3015c210 con 0x7feb30144440 2024-04-25T10:02:10.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.735+0000 7feb2fe00640 1 -- 172.21.15.165:0/1703485764 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458154615 0 0) 0x7feb3015bf30 con 0x7feb30071530 2024-04-25T10:02:10.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.736+0000 7feb35400640 1 --2- 172.21.15.165:0/1703485764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb30144440 0x7feb30070ff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb240097a0 tx=0x7feb2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=780fa748372237db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.736+0000 7feb2fe00640 1 -- 172.21.15.165:0/1703485764 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24037070 con 0x7feb30144440 2024-04-25T10:02:10.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.736+0000 7feb2fe00640 1 -- 172.21.15.165:0/1703485764 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb24032440 con 0x7feb30144440 2024-04-25T10:02:10.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.736+0000 7feb2fe00640 1 -- 172.21.15.165:0/1703485764 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb24032810 con 0x7feb30144440 2024-04-25T10:02:10.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.736+0000 7feb36800640 1 -- 172.21.15.165:0/1703485764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb30144440 msgr2=0x7feb30070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.736+0000 7feb36800640 1 --2- 172.21.15.165:0/1703485764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb30144440 0x7feb30070ff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7feb240097a0 tx=0x7feb2402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.737+0000 7feb36800640 1 -- 172.21.15.165:0/1703485764 shutdown_connections 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.737+0000 7feb36800640 1 --2- 172.21.15.165:0/1703485764 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb300cda30 0x7feb3015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.737+0000 7feb36800640 1 --2- 172.21.15.165:0/1703485764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb30144440 0x7feb30070ff0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.737+0000 7feb36800640 1 -- 172.21.15.165:0/1703485764 >> 172.21.15.165:0/1703485764 conn(0x7feb3006abe0 msgr2=0x7feb3006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.737+0000 7feb36800640 1 -- 172.21.15.165:0/1703485764 shutdown_connections 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.737+0000 7feb36800640 1 -- 172.21.15.165:0/1703485764 wait complete. 2024-04-25T10:02:10.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.738+0000 7feb36800640 1 Processor -- start 2024-04-25T10:02:10.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.739+0000 7feb36800640 1 -- start start 2024-04-25T10:02:10.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.739+0000 7feb36800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30071530 0x7feb300b38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb36800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 0x7feb300cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb36800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb30144440 0x7feb300cca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb36800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3015bc30 con 0x7feb300cda30 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb36800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb300cdea0 con 0x7feb30071530 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb36800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3015d230 con 0x7feb30144440 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 0x7feb300cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 0x7feb300cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52788/0 (socket says 172.21.15.165:52788) 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 -- 172.21.15.165:0/4259314407 learned_addr learned my addr 172.21.15.165:0/4259314407 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 -- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb30144440 msgr2=0x7feb300cca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb35e00640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb30144440 0x7feb300cca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb30144440 0x7feb300cca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb35400640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30071530 0x7feb300b38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 -- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30071530 msgr2=0x7feb300b38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30071530 0x7feb300b38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.740+0000 7feb34a00640 1 -- 172.21.15.165:0/4259314407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb300b2830 con 0x7feb300cda30 2024-04-25T10:02:10.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.741+0000 7feb34a00640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 0x7feb300cc4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c4a0 tx=0x7feb1c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.741+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c018070 con 0x7feb300cda30 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.741+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb1c0074e0 con 0x7feb300cda30 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.741+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb1c004a20 con 0x7feb300cda30 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.741+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb300b2b10 con 0x7feb300cda30 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.741+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb300b30d0 con 0x7feb300cda30 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.742+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb30073990 con 0x7feb300cda30 2024-04-25T10:02:10.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.743+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb1c004bc0 con 0x7feb300cda30 2024-04-25T10:02:10.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.744+0000 7feb2d600640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1006b4d0 0x7feb1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:10.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.744+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feb1c089930 con 0x7feb300cda30 2024-04-25T10:02:10.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.744+0000 7feb35400640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1006b4d0 0x7feb1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:10.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.745+0000 7feb35400640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1006b4d0 0x7feb1006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb24034000 tx=0x7feb2402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:10.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.748+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb1c0583d0 con 0x7feb300cda30 2024-04-25T10:02:10.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.980+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb3006e4b0 con 0x7feb300cda30 2024-04-25T10:02:10.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.981+0000 7feb2d600640 1 -- 172.21.15.165:0/4259314407 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4946 (secure 0 0 0) 0x7feb1c057cd0 con 0x7feb300cda30 2024-04-25T10:02:10.986 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:10.986 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:08.622677+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-25T10:02:10.986 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 7 2024-04-25T10:02:10.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1006b4d0 msgr2=0x7feb1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1006b4d0 0x7feb1006d990 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7feb24034000 tx=0x7feb2402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:10.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 msgr2=0x7feb300cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:10.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 0x7feb300cc4e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7feb1c00c4a0 tx=0x7feb1c00c960 comp rx=0 tx=0).stop 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 shutdown_connections 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1006b4d0 0x7feb1006d990 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb30144440 0x7feb300cca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb300cda30 0x7feb300cc4e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 --2- 172.21.15.165:0/4259314407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb30071530 0x7feb300b38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 >> 172.21.15.165:0/4259314407 conn(0x7feb3006abe0 msgr2=0x7feb30140d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:10.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 shutdown_connections 2024-04-25T10:02:10.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:10.984+0000 7feb36800640 1 -- 172.21.15.165:0/4259314407 wait complete. 2024-04-25T10:02:11.007 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:11.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.206+0000 7f4906000640 1 Processor -- start 2024-04-25T10:02:11.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.206+0000 7f4906000640 1 -- start start 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.207+0000 7f4906000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f490015db90 0x7f49000b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f4906000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49001615a0 0x7f49000cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f4906000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49000d33f0 con 0x7f49000d04f0 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f4906000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49000d0980 con 0x7f49001615a0 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f4906000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49000d1f50 con 0x7f490015db90 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49001615a0 0x7f49000cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49001615a0 0x7f49000cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52798/0 (socket says 172.21.15.165:52798) 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 -- 172.21.15.165:0/1249853933 learned_addr learned my addr 172.21.15.165:0/1249853933 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f4904c00640 1 --2- 172.21.15.165:0/1249853933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f490015db90 0x7f49000b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 -- 172.21.15.165:0/1249853933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f490015db90 msgr2=0x7f49000b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 --2- 172.21.15.165:0/1249853933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f490015db90 0x7f49000b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 -- 172.21.15.165:0/1249853933 >> v1:172.21.15.172:6789/0 conn(0x7f49000d04f0 legacy=0x7f4900061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:11.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.208+0000 7f48ffe00640 1 -- 172.21.15.165:0/1249853933 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4900161df0 con 0x7f49001615a0 2024-04-25T10:02:11.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.209+0000 7f48ffe00640 1 --2- 172.21.15.165:0/1249853933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49001615a0 0x7f49000cffb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009e50 tx=0x7f48e802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f2890a4a3f77de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.210+0000 7f48ff400640 1 -- 172.21.15.165:0/1249853933 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8036070 con 0x7f49001615a0 2024-04-25T10:02:11.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.210+0000 7f48ff400640 1 -- 172.21.15.165:0/1249853933 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8031600 con 0x7f49001615a0 2024-04-25T10:02:11.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.210+0000 7f48ff400640 1 -- 172.21.15.165:0/1249853933 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e80319d0 con 0x7f49001615a0 2024-04-25T10:02:11.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.210+0000 7f4906000640 1 -- 172.21.15.165:0/1249853933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49001615a0 msgr2=0x7f49000cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.210+0000 7f4906000640 1 --2- 172.21.15.165:0/1249853933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49001615a0 0x7f49000cffb0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009e50 tx=0x7f48e802a710 comp rx=0 tx=0).stop 2024-04-25T10:02:11.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.211+0000 7f4906000640 1 -- 172.21.15.165:0/1249853933 shutdown_connections 2024-04-25T10:02:11.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.211+0000 7f4906000640 1 --2- 172.21.15.165:0/1249853933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49001615a0 0x7f49000cffb0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.211+0000 7f4906000640 1 --2- 172.21.15.165:0/1249853933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f490015db90 0x7f49000b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.211+0000 7f4906000640 1 -- 172.21.15.165:0/1249853933 >> 172.21.15.165:0/1249853933 conn(0x7f49000b52b0 msgr2=0x7f49000b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.211+0000 7f4906000640 1 -- 172.21.15.165:0/1249853933 shutdown_connections 2024-04-25T10:02:11.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.211+0000 7f4906000640 1 -- 172.21.15.165:0/1249853933 wait complete. 2024-04-25T10:02:11.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.212+0000 7f4906000640 1 Processor -- start 2024-04-25T10:02:11.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.212+0000 7f4906000640 1 -- start start 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.213+0000 7f4906000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49000d04f0 0x7f4900155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.213+0000 7f4906000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 0x7f49000b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.213+0000 7f4906000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49001615a0 0x7f49000b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.213+0000 7f4906000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49000d1be0 con 0x7f490015db90 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.213+0000 7f4906000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49001623b0 con 0x7f49000d04f0 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.213+0000 7f4906000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4900161a20 con 0x7f49001615a0 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 0x7f49000b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 0x7f49000b7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52800/0 (socket says 172.21.15.165:52800) 2024-04-25T10:02:11.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 -- 172.21.15.165:0/2496922110 learned_addr learned my addr 172.21.15.165:0/2496922110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f4904c00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49000d04f0 0x7f4900155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f4905600640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49001615a0 0x7f49000b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 -- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49001615a0 msgr2=0x7f49000b81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49001615a0 0x7f49000b81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 -- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49000d04f0 msgr2=0x7f4900155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49000d04f0 0x7f4900155620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 -- 172.21.15.165:0/2496922110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49000cbc60 con 0x7f490015db90 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f4905600640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49001615a0 0x7f49000b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.214+0000 7f48ffe00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 0x7f49000b7c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000f80 tx=0x7f48e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.215+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8036050 con 0x7f490015db90 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.215+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48e8042070 con 0x7f490015db90 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.215+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48e8009510 con 0x7f490015db90 2024-04-25T10:02:11.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.215+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49000cbf40 con 0x7f490015db90 2024-04-25T10:02:11.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.215+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49000cc500 con 0x7f490015db90 2024-04-25T10:02:11.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.216+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4900073990 con 0x7f490015db90 2024-04-25T10:02:11.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.217+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48e8007c70 con 0x7f490015db90 2024-04-25T10:02:11.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.218+0000 7f48fcc00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f48dc06b500 0x7f48dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:11.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.218+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f48e80ab4d0 con 0x7f490015db90 2024-04-25T10:02:11.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.219+0000 7f4904c00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f48dc06b500 0x7f48dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:11.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.219+0000 7f4904c00640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f48dc06b500 0x7f48dc06d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:11.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.223+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48e8079f70 con 0x7f490015db90 2024-04-25T10:02:11.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.404+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f49000d14b0 con 0x7f490015db90 2024-04-25T10:02:11.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.407+0000 7f48fcc00640 1 -- 172.21.15.165:0/2496922110 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f48e8079870 con 0x7f490015db90 2024-04-25T10:02:11.412 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:11.412 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.409+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f48dc06b500 msgr2=0x7f48dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.409+0000 7f4906000640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f48dc06b500 0x7f48dc06d9c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f48f0006fd0 tx=0x7f48f0008040 comp rx=0 tx=0).stop 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.409+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 msgr2=0x7f49000b7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.409+0000 7f4906000640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 0x7f49000b7c80 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f48e8000f80 tx=0x7f48e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:11.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 shutdown_connections 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f48dc06b500 0x7f48dc06d9c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49001615a0 0x7f49000b81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f490015db90 0x7f49000b7c80 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 --2- 172.21.15.165:0/2496922110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49000d04f0 0x7f4900155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 >> 172.21.15.165:0/2496922110 conn(0x7f49000b52b0 msgr2=0x7f49000c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 shutdown_connections 2024-04-25T10:02:11.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:11.410+0000 7f4906000640 1 -- 172.21.15.165:0/2496922110 wait complete. 2024-04-25T10:02:11.833 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:03.570541+0000', 'modified': '2024-04-25T10:02:08.622677+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6835/2597521026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6833', 'nonce': 2597521026}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2597521026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2024-04-25T10:02:11.834 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:11.834 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:02:12.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.040+0000 7fb389a00640 1 Processor -- start 2024-04-25T10:02:12.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.040+0000 7fb389a00640 1 -- start start 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb389a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb38415b710 0x7fb3840cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb389a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb38406ec50 0x7fb38415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb389a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38406f0e0 con 0x7fb38415ee60 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb389a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38415e490 con 0x7fb38415b710 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb389a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3840cfd80 con 0x7fb38406ec50 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb382a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb38415b710 0x7fb3840cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb383400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb38415ee60 0x7fb3840ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58812/0 (socket says 172.21.15.165:58812) 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb383400640 1 -- 172.21.15.165:0/3215035612 learned_addr learned my addr 172.21.15.165:0/3215035612 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb382000640 1 --2- 172.21.15.165:0/3215035612 >> v2:172.21.15.172:3300/0 conn(0x7fb38406ec50 0x7fb38415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb382a00640 1 -- 172.21.15.165:0/3215035612 >> v2:172.21.15.172:3300/0 conn(0x7fb38406ec50 msgr2=0x7fb38415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb382a00640 1 --2- 172.21.15.165:0/3215035612 >> v2:172.21.15.172:3300/0 conn(0x7fb38406ec50 0x7fb38415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb382a00640 1 -- 172.21.15.165:0/3215035612 >> v1:172.21.15.165:6789/0 conn(0x7fb38415ee60 legacy=0x7fb3840ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.042+0000 7fb382a00640 1 -- 172.21.15.165:0/3215035612 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3840d0060 con 0x7fb38415b710 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.043+0000 7fb382a00640 1 --2- 172.21.15.165:0/3215035612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415b710 0x7fb3840cf640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb38406ba10 tx=0x7fb37402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3ac5a5e822d99b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.044+0000 7fb381600640 1 -- 172.21.15.165:0/3215035612 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374037070 con 0x7fb38415b710 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.044+0000 7fb381600640 1 -- 172.21.15.165:0/3215035612 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb374032440 con 0x7fb38415b710 2024-04-25T10:02:12.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.044+0000 7fb381600640 1 -- 172.21.15.165:0/3215035612 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374032810 con 0x7fb38415b710 2024-04-25T10:02:12.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.044+0000 7fb389a00640 1 -- 172.21.15.165:0/3215035612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415b710 msgr2=0x7fb3840cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.044+0000 7fb389a00640 1 --2- 172.21.15.165:0/3215035612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415b710 0x7fb3840cf640 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fb38406ba10 tx=0x7fb37402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.045+0000 7fb389a00640 1 -- 172.21.15.165:0/3215035612 shutdown_connections 2024-04-25T10:02:12.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.045+0000 7fb389a00640 1 --2- 172.21.15.165:0/3215035612 >> v2:172.21.15.172:3300/0 conn(0x7fb38406ec50 0x7fb38415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.045+0000 7fb389a00640 1 --2- 172.21.15.165:0/3215035612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415b710 0x7fb3840cf640 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.045+0000 7fb389a00640 1 -- 172.21.15.165:0/3215035612 >> 172.21.15.165:0/3215035612 conn(0x7fb38406abe0 msgr2=0x7fb38406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.045+0000 7fb389a00640 1 -- 172.21.15.165:0/3215035612 shutdown_connections 2024-04-25T10:02:12.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.045+0000 7fb389a00640 1 -- 172.21.15.165:0/3215035612 wait complete. 2024-04-25T10:02:12.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.046+0000 7fb389a00640 1 Processor -- start 2024-04-25T10:02:12.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 -- start start 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb38406ec50 0x7fb384157200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb38415b710 0x7fb384146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 0x7fb3840afd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415bc70 con 0x7fb38415ee60 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415baf0 con 0x7fb38406ec50 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.047+0000 7fb389a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3840acfc0 con 0x7fb38415b710 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 0x7fb3840afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 0x7fb3840afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52820/0 (socket says 172.21.15.165:52820) 2024-04-25T10:02:12.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 -- 172.21.15.165:0/120074020 learned_addr learned my addr 172.21.15.165:0/120074020 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:12.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb382a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb38406ec50 0x7fb384157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb382000640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb38415b710 0x7fb384146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 -- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb38415b710 msgr2=0x7fb384146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb38415b710 0x7fb384146d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 -- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb38406ec50 msgr2=0x7fb384157200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb38406ec50 0x7fb384157200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb383400640 1 -- 172.21.15.165:0/120074020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb384157740 con 0x7fb38415ee60 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.048+0000 7fb382a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb38406ec50 0x7fb384157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.049+0000 7fb383400640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 0x7fb3840afd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00cab0 tx=0x7fb37c00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.049+0000 7fb37aa00640 1 -- 172.21.15.165:0/120074020 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c017070 con 0x7fb38415ee60 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.049+0000 7fb37aa00640 1 -- 172.21.15.165:0/120074020 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37c00ac30 con 0x7fb38415ee60 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.049+0000 7fb37aa00640 1 -- 172.21.15.165:0/120074020 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c012400 con 0x7fb38415ee60 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.049+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3841e6950 con 0x7fb38415ee60 2024-04-25T10:02:12.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.049+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3841447c0 con 0x7fb38415ee60 2024-04-25T10:02:12.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.051+0000 7fb37aa00640 1 -- 172.21.15.165:0/120074020 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb37c01a070 con 0x7fb38415ee60 2024-04-25T10:02:12.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.052+0000 7fb37aa00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb36406b4d0 0x7fb36406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.052+0000 7fb37aa00640 1 -- 172.21.15.165:0/120074020 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb37c08a170 con 0x7fb38415ee60 2024-04-25T10:02:12.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.052+0000 7fb382a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb36406b4d0 0x7fb36406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.053+0000 7fb38c99b740 1 Processor -- start 2024-04-25T10:02:12.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.053+0000 7fb38c99b740 1 -- start start 2024-04-25T10:02:12.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.053+0000 7fb382a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb36406b4d0 0x7fb36406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb374034000 tx=0x7fb37402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560354785950 0x5603547ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x560354785d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x560354784110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb38c99b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603546e9620 con 0x5603547adcd0 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560354692ca0 con 0x560354785950 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603546e56c0 con 0x5603547ce5c0 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x560354785d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x560354785d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52834/0 (socket says 172.21.15.165:52834) 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 -- 172.21.15.165:0/474342144 learned_addr learned my addr 172.21.15.165:0/474342144 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 -- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 msgr2=0x560354784110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x560354784110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 -- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560354785950 msgr2=0x5603547ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382a00640 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560354785950 0x5603547ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560354785950 0x5603547ad790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.054+0000 7fb382000640 1 -- 172.21.15.165:0/474342144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560354699f40 con 0x5603547adcd0 2024-04-25T10:02:12.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.055+0000 7fb382000640 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x560354785d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb370006fd0 tx=0x7fb37000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae976d29f4b63c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.055+0000 7fb36f400640 1 -- 172.21.15.165:0/474342144 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370018030 con 0x5603547adcd0 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.055+0000 7fb36f400640 1 -- 172.21.15.165:0/474342144 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370004020 con 0x5603547adcd0 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.055+0000 7fb36f400640 1 -- 172.21.15.165:0/474342144 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3700043d0 con 0x5603547adcd0 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 -- 172.21.15.165:0/474342144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 msgr2=0x560354785d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x560354785d30 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb370006fd0 tx=0x7fb37000bd00 comp rx=0 tx=0).stop 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 -- 172.21.15.165:0/474342144 shutdown_connections 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x560354784110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x560354785d30 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 --2- 172.21.15.165:0/474342144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560354785950 0x5603547ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 -- 172.21.15.165:0/474342144 >> 172.21.15.165:0/474342144 conn(0x560354782b70 msgr2=0x5603547cb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 -- 172.21.15.165:0/474342144 shutdown_connections 2024-04-25T10:02:12.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.056+0000 7fb38c99b740 1 -- 172.21.15.165:0/474342144 wait complete. 2024-04-25T10:02:12.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.057+0000 7fb38c99b740 1 -- start start 2024-04-25T10:02:12.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.057+0000 7fb38c99b740 1 Processor -- start 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x5603547caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x5603547b0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 0x5603547b1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603547ad300 con 0x5603547ae4f0 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603546881f0 con 0x5603547adcd0 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603547accf0 con 0x5603547ce5c0 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb38c99b740 10 client.0 _ll_register_callbacks cb 0x5603546a5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 0x5603547b1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 0x5603547b1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52842/0 (socket says 172.21.15.165:52842) 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 -- 172.21.15.165:0/555265395 learned_addr learned my addr 172.21.15.165:0/555265395 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:12.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb382000640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x5603547b0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb382a00640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x5603547caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 -- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 msgr2=0x5603547b0170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x5603547b0170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 -- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 msgr2=0x5603547caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x5603547caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.058+0000 7fb383400640 1 -- 172.21.15.165:0/555265395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603546a7c50 con 0x5603547ae4f0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb383400640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 0x5603547b1dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb37c054a20 tx=0x7fb37c05c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3d98d06504d28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb363e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb363e00640 1 -- 172.21.15.165:0/555265395 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37c004b60 con 0x5603547ae4f0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb363e00640 1 -- 172.21.15.165:0/555265395 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb37c004d00 con 0x5603547ae4f0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb370004020 con 0x5603547ae4f0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb363e00640 1 -- 172.21.15.165:0/555265395 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb37c086050 con 0x5603547ae4f0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb38c99b740 10 client.4401 fetch_fsmap learned FSMap version 7 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5603543fe2d0 con 0x5603547ae4f0 2024-04-25T10:02:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb363e00640 1 -- 172.21.15.165:0/555265395 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370004020 con 0x5603547ae4f0 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.059+0000 7fb363e00640 1 -- 172.21.15.165:0/555265395 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb37c057a10 con 0x5603547ae4f0 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb38c99b740 10 client.4401 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb38c99b740 10 client.4401 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb38c99b740 20 client.4401 populate_metadata read hostname 'smithi165' 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb38c99b740 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547a9840 0x5603547a9c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb38c99b740 4 client.4401 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 --> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5603546f6b60 con 0x5603547a9840 2024-04-25T10:02:12.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.060+0000 7fb382a00640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547a9840 0x5603547a9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.061+0000 7fb382a00640 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547a9840 0x5603547a9c20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a4e0195657314b server_cookie=9c4ae758020b534c in_seq=0 out_seq=0 2024-04-25T10:02:12.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.061+0000 7fb363e00640 10 client.4401 ms_handle_connect on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:12.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb363e00640 1 -- 172.21.15.165:0/555265395 <== mds.0 v2:172.21.15.172:6833/2597521026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5603546f6b60 con 0x5603547a9840 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb363e00640 10 client.4401 handle_command_reply: tid=0 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 client.4401 shutdown 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 msgr2=0x5603547b1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 0x5603547b1dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb37c054a20 tx=0x7fb37c05c7a0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 shutdown_connections 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547a9840 0x5603547a9c20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547ae4f0 0x5603547b1dc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb363e00640 0 client.4401 ms_handle_reset on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ce5c0 0x5603547b0170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 --2- 172.21.15.165:0/555265395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x5603547caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 >> 172.21.15.165:0/555265395 conn(0x560354785950 msgr2=0x56035477f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 shutdown_connections 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 1 -- 172.21.15.165:0/555265395 wait complete. 2024-04-25T10:02:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.063+0000 7fb38c99b740 20 client.4401 trim_cache size 0 max 16384 2024-04-25T10:02:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.080+0000 7fb38c99b740 1 Processor -- start 2024-04-25T10:02:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 -- start start 2024-04-25T10:02:12.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547a9840 0x5603547a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x5603547a7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603547a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384200450 con 0x5603547adcd0 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38415a0b0 con 0x5603547a9840 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.081+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3841ffe50 con 0x5603547ae4f0 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x5603547a7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x5603547a7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52858/0 (socket says 172.21.15.165:52858) 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 -- 172.21.15.165:0/2917673417 learned_addr learned my addr 172.21.15.165:0/2917673417 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382a00640 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547a9840 0x5603547a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb383400640 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603547a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 -- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 msgr2=0x5603547a8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603547a8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 -- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547a9840 msgr2=0x5603547a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547a9840 0x5603547a4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.082+0000 7fb382000640 1 -- 172.21.15.165:0/2917673417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603544c86b0 con 0x5603547adcd0 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb382000640 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x5603547a7c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb370016000 tx=0x7fb370009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf7dffd57462934 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb36ea00640 1 -- 172.21.15.165:0/2917673417 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb370027070 con 0x5603547adcd0 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb36ea00640 1 -- 172.21.15.165:0/2917673417 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370004380 con 0x5603547adcd0 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb36ea00640 1 -- 172.21.15.165:0/2917673417 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000fa00 con 0x5603547adcd0 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 -- 172.21.15.165:0/2917673417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 msgr2=0x5603547a7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x5603547a7c20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb370016000 tx=0x7fb370009000 comp rx=0 tx=0).stop 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 -- 172.21.15.165:0/2917673417 shutdown_connections 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603547a8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547adcd0 0x5603547a7c20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2917673417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547a9840 0x5603547a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.083+0000 7fb38c99b740 1 -- 172.21.15.165:0/2917673417 >> 172.21.15.165:0/2917673417 conn(0x560354785950 msgr2=0x5603547ca8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.084+0000 7fb38c99b740 1 -- 172.21.15.165:0/2917673417 shutdown_connections 2024-04-25T10:02:12.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.084+0000 7fb38c99b740 1 -- 172.21.15.165:0/2917673417 wait complete. 2024-04-25T10:02:12.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.084+0000 7fb38c99b740 1 -- start start 2024-04-25T10:02:12.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.084+0000 7fb38c99b740 1 Processor -- start 2024-04-25T10:02:12.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.085+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 0x5603547e8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.085+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x560354836e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.085+0000 7fb38c99b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603548337a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.085+0000 7fb38c99b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603547ae970 con 0x5603547a9840 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.085+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603547af970 con 0x5603547adcd0 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.085+0000 7fb38c99b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603547abaf0 con 0x5603547ae4f0 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb38c99b740 10 client.0 _ll_register_callbacks cb 0x7fb37c001c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 0x5603547e8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 0x5603547e8ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52872/0 (socket says 172.21.15.165:52872) 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 -- 172.21.15.165:0/2105563773 learned_addr learned my addr 172.21.15.165:0/2105563773 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 -- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 msgr2=0x5603548337a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603548337a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 -- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 msgr2=0x560354836e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x560354836e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 -- 172.21.15.165:0/2105563773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603547a7f00 con 0x5603547a9840 2024-04-25T10:02:12.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb382a00640 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 0x5603547e8ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb374009890 tx=0x7fb374004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6870ee8dfaf595d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.086+0000 7fb363e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb363e00640 1 -- 172.21.15.165:0/2105563773 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374037040 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb363e00640 1 -- 172.21.15.165:0/2105563773 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb37404c2b0 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb370004380 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb363e00640 1 -- 172.21.15.165:0/2105563773 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb37405d050 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 10 client.4407 fetch_fsmap learned FSMap version 7 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x560354833ce0 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb363e00640 1 -- 172.21.15.165:0/2105563773 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb370004380 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb363e00640 1 -- 172.21.15.165:0/2105563773 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 7) v1 ==== 3302+0+0 (secure 0 0 0) 0x7fb374060020 con 0x5603547a9840 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 10 client.4407 fetch_fsmap finished waiting for FSMap version 7 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 10 client.4407 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 20 client.4407 populate_metadata read hostname 'smithi165' 2024-04-25T10:02:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547ce5c0 0x560354841c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 4 client.4407 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.087+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 --> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5603547b1930 con 0x5603547ce5c0 2024-04-25T10:02:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.088+0000 7fb382000640 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547ce5c0 0x560354841c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb382000640 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547ce5c0 0x560354841c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d983cbce516704cc server_cookie=77708a361d43585e in_seq=0 out_seq=0 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb363e00640 10 client.4407 ms_handle_connect on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb363e00640 1 -- 172.21.15.165:0/2105563773 <== mds.0 v2:172.21.15.172:6833/2597521026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+322 (crc 0 0 0) 0x5603547b1930 con 0x5603547ce5c0 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb363e00640 10 client.4407 handle_command_reply: tid=0 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 client.4407 shutdown 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 msgr2=0x5603547e8ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 0x5603547e8ad0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb374009890 tx=0x7fb374004730 comp rx=0 tx=0).stop 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 shutdown_connections 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5603547ae4f0 0x5603548337a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5603547adcd0 0x560354836e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x5603547ce5c0 0x560354841c60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 --2- 172.21.15.165:0/2105563773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5603547a9840 0x5603547e8ad0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 >> 172.21.15.165:0/2105563773 conn(0x560354785950 msgr2=0x5603547843c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.089+0000 7fb363e00640 0 client.4407 ms_handle_reset on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 shutdown_connections 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb38c99b740 1 -- 172.21.15.165:0/2105563773 wait complete. 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb38c99b740 20 client.4407 trim_cache size 0 max 16384 2024-04-25T10:02:12.097 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 4288, 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 7.9672890369999996, 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 7, 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 24, 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 13.6029556 2024-04-25T10:02:12.098 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:02:12.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb36406b4d0 msgr2=0x7fb36406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb36406b4d0 0x7fb36406d990 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb374034000 tx=0x7fb37402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 msgr2=0x7fb3840afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 0x7fb3840afd60 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fb37c00cab0 tx=0x7fb37c00cf70 comp rx=0 tx=0).stop 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 shutdown_connections 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb36406b4d0 0x7fb36406d990 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb38415ee60 0x7fb3840afd60 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb38415b710 0x7fb384146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 --2- 172.21.15.165:0/120074020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb38406ec50 0x7fb384157200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.090+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 >> 172.21.15.165:0/120074020 conn(0x7fb38406abe0 msgr2=0x7fb3840ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.091+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 shutdown_connections 2024-04-25T10:02:12.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.091+0000 7fb389a00640 1 -- 172.21.15.165:0/120074020 wait complete. 2024-04-25T10:02:12.112 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:12.112 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:12.113 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.302+0000 7f38a6000640 1 Processor -- start 2024-04-25T10:02:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.302+0000 7f38a6000640 1 -- start start 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.303+0000 7f38a6000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38a0071700 0x7f38a00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.303+0000 7f38a6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 0x7f38a015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.303+0000 7f38a6000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a015bf30 con 0x7f38a0144440 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.303+0000 7f38a6000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a015c860 con 0x7f38a0071700 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f38a6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015c210 con 0x7f38a00cd860 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f38a4c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f38a0144440 0x7f38a0070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58824/0 (socket says 172.21.15.165:58824) 2024-04-25T10:02:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f38a4c00640 1 -- 172.21.15.165:0/27792576 learned_addr learned my addr 172.21.15.165:0/27792576 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f3897e00640 1 --2- 172.21.15.165:0/27792576 >> v2:172.21.15.172:3300/0 conn(0x7f38a0071700 0x7f38a00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f38a5600640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 0x7f38a015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082340792 0 0) 0x7f38a015bf30 con 0x7f38a0144440 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.304+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3884003520 con 0x7f38a0144440 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4101878407 0 0) 0x7f3884003520 con 0x7f38a0144440 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 msgr2=0x7f38a015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 0x7f38a015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 >> v2:172.21.15.172:3300/0 conn(0x7f38a0071700 msgr2=0x7f38a00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 --2- 172.21.15.165:0/27792576 >> v2:172.21.15.172:3300/0 conn(0x7f38a0071700 0x7f38a00cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a015c4f0 con 0x7f38a0144440 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3890001670 con 0x7f38a0144440 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 >> v1:172.21.15.165:6789/0 conn(0x7f38a0144440 legacy=0x7f38a0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 0x7f38840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884003f30 0x7f3884004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3884004d40 0x7f38840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015c860 con 0x7f3884003f30 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015c210 con 0x7f3884004d40 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388400b9f0 con 0x7f38a00cd860 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f3897e00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884003f30 0x7f3884004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.305+0000 7f38a4c00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 0x7f38840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f38a5600640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3884004d40 0x7f38840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f3897e00640 1 -- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 msgr2=0x7f38840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f3897e00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38a00cd860 0x7f38840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f3897e00640 1 -- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3884004d40 msgr2=0x7f38840051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f3897e00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3884004d40 0x7f38840051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f3897e00640 1 -- 172.21.15.165:0/27792576 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884003520 con 0x7f3884003f30 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.306+0000 7f38a5600640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3884004d40 0x7f38840051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f3897e00640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884003f30 0x7f3884004800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3888009ac0 tx=0x7f3884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7031e26df5829b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888036070 con 0x7f3884003f30 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3888031400 con 0x7f3884003f30 2024-04-25T10:02:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f389fe00640 1 -- 172.21.15.165:0/27792576 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3888009690 con 0x7f3884003f30 2024-04-25T10:02:12.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f38a6000640 1 -- 172.21.15.165:0/27792576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884003f30 msgr2=0x7f3884004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f38a6000640 1 --2- 172.21.15.165:0/27792576 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884003f30 0x7f3884004800 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f3888009ac0 tx=0x7f3884005830 comp rx=0 tx=0).stop 2024-04-25T10:02:12.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.307+0000 7f38a5600640 1 -- 172.21.15.165:0/27792576 reap_dead start 2024-04-25T10:02:12.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.308+0000 7f38a6000640 1 -- 172.21.15.165:0/27792576 shutdown_connections 2024-04-25T10:02:12.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.308+0000 7f38a6000640 1 -- 172.21.15.165:0/27792576 >> 172.21.15.165:0/27792576 conn(0x7f38a006abe0 msgr2=0x7f38a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.308+0000 7f38a6000640 1 -- 172.21.15.165:0/27792576 shutdown_connections 2024-04-25T10:02:12.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.308+0000 7f38a6000640 1 -- 172.21.15.165:0/27792576 wait complete. 2024-04-25T10:02:12.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.309+0000 7f38a6000640 1 Processor -- start 2024-04-25T10:02:12.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.310+0000 7f38a6000640 1 -- start start 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.310+0000 7f38a6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3884003f30 0x7f38a0138d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.310+0000 7f38a6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a0144820 0x7f38a0144c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.310+0000 7f38a6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 0x7f38a00cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00cde40 con 0x7f38a00cc200 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00cdcc0 con 0x7f38a0144820 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a015ccf0 con 0x7f3884003f30 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 0x7f38a00cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 0x7f38a00cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52880/0 (socket says 172.21.15.165:52880) 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 -- 172.21.15.165:0/3789525756 learned_addr learned my addr 172.21.15.165:0/3789525756 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a4c00640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3884003f30 0x7f38a0138d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f3897e00640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a0144820 0x7f38a0144c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 -- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3884003f30 msgr2=0x7f38a0138d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3884003f30 0x7f38a0138d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 -- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a0144820 msgr2=0x7f38a0144c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a0144820 0x7f38a0144c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.311+0000 7f38a5600640 1 -- 172.21.15.165:0/3789525756 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a00ccf80 con 0x7f38a00cc200 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.312+0000 7f38a5600640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 0x7f38a00cc680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38a00b2250 tx=0x7f389800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.312+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389801c070 con 0x7f38a00cc200 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.312+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3898017400 con 0x7f38a00cc200 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.312+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38980177f0 con 0x7f38a00cc200 2024-04-25T10:02:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.312+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00cd260 con 0x7f38a00cc200 2024-04-25T10:02:12.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.312+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a00c7770 con 0x7f38a00cc200 2024-04-25T10:02:12.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.313+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a00cece0 con 0x7f38a00cc200 2024-04-25T10:02:12.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.314+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f389801f070 con 0x7f38a00cc200 2024-04-25T10:02:12.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.315+0000 7f389d600640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f387c06b4d0 0x7f387c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:12.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.315+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f389808a640 con 0x7f38a00cc200 2024-04-25T10:02:12.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.315+0000 7f38a4c00640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f387c06b4d0 0x7f387c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:12.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.316+0000 7f38a4c00640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f387c06b4d0 0x7f387c06d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3890000fb0 tx=0x7f3890009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:12.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.319+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f38a00cece0 con 0x7f38a00cc200 2024-04-25T10:02:12.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.561+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f38a0073990 con 0x7f38a00cc200 2024-04-25T10:02:12.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.562+0000 7f389d600640 1 -- 172.21.15.165:0/3789525756 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7f3898012070 con 0x7f38a00cc200 2024-04-25T10:02:12.566 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.564+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f387c06b4d0 msgr2=0x7f387c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.564+0000 7f38a6000640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f387c06b4d0 0x7f387c06d990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f3890000fb0 tx=0x7f3890009040 comp rx=0 tx=0).stop 2024-04-25T10:02:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 msgr2=0x7f38a00cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 0x7f38a00cc680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f38a00b2250 tx=0x7f389800f4a0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 shutdown_connections 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f387c06b4d0 0x7f387c06d990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38a00cc200 0x7f38a00cc680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38a0144820 0x7f38a0144c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 --2- 172.21.15.165:0/3789525756 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3884003f30 0x7f38a0138d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 >> 172.21.15.165:0/3789525756 conn(0x7f38a006abe0 msgr2=0x7f38a01395e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 shutdown_connections 2024-04-25T10:02:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:12.565+0000 7f38a6000640 1 -- 172.21.15.165:0/3789525756 wait complete. 2024-04-25T10:02:12.588 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:12.589 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:12.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.589 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:02:12.641 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:12.642 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42633sec preferred_lft 42633sec 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:02:12.643 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:12.645 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:12.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:12.645 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:12.645 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:12.645 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:12.645 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:12.645 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set ceph-brx up 2024-04-25T10:02:12.645 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:12.646 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:13.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.049 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:13.115 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:02:13.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.118 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:02:13.172 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:02:13.214 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:13.214 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:13.215 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:13.215 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:13.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.215 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:13.215 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:13.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:13.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:13.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:13.216 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:13.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.894 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:02:13.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.945 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list-id 2024-04-25T10:02:13.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:13.990 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:13.990 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:13.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:13.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-25T10:02:13.990 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:14.152 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-25T10:02:14.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:14.152 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:14.152 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:14.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-25T10:02:14.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:14.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-25T10:02:14.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-25T10:02:14.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-25T10:02:14.153 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:15.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.031 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:15.031 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:15.031 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:02:15.031 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:15.031 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:15.170 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-25T10:02:15.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:15.170 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:15.217 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-25T10:02:15.217 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:15.217 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:15.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo modprobe fuse 2024-04-25T10:02:15.392 DEBUG:teuthology.orchestra.run.smithi165:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:15.447 INFO:teuthology.orchestra.run.smithi165.stdout:/proc 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/sys 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/dev 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/security 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/shm 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/pts 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/run 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/cgroup 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/pstore 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/bpf 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/ 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/selinux 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:15.448 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/hugepages 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/mqueue 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/debug 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/tracing 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/config 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/run/user/1000 2024-04-25T10:02:15.449 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/run/netns 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.450 INFO:teuthology.orchestra.run.smithi165.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.451 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.451 DEBUG:teuthology.orchestra.run.smithi165:> ls /sys/fs/fuse/connections 2024-04-25T10:02:15.500 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-25T10:02:15.501 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-25T10:02:15.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo modprobe fuse 2024-04-25T10:02:15.570 DEBUG:teuthology.orchestra.run.smithi165:> cat /proc/self/mounts | awk '{print $2}' 2024-04-25T10:02:15.611 INFO:teuthology.orchestra.run.smithi165.stdout:/proc 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/sys 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/dev 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/security 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/shm 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/pts 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/run 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/cgroup 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/pstore 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/bpf 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/ 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/selinux 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/proc/sys/fs/binfmt_misc 2024-04-25T10:02:15.612 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/hugepages 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/dev/mqueue 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/debug 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/tracing 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/fs/fuse/connections 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/sys/kernel/config 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/run/credentials/systemd-sysctl.service 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/nfs/rpc_pipefs 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/run/user/1000 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-0 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-1 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-2 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/var/lib/ceph/osd/ceph-3 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/run/netns 2024-04-25T10:02:15.613 INFO:teuthology.orchestra.run.smithi165.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.614 INFO:teuthology.orchestra.run.smithi165.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:15.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.615 DEBUG:teuthology.orchestra.run.smithi165:> ls /sys/fs/fuse/connections 2024-04-25T10:02:15.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi165.stderr:2024-04-25T10:02:15.675+0000 7f54b1088480 -1 init, newargv = 0x55f7a21ec810 newargc=15 2024-04-25T10:02:15.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi165.stderr:ceph-fuse[40435]: starting ceph client 2024-04-25T10:02:15.694 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi165.stderr:ceph-fuse[40435]: starting fuse 2024-04-25T10:02:15.698 INFO:teuthology.orchestra.run.smithi165.stdout:40 2024-04-25T10:02:15.698 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> sudo stdin-killer -- python3 -c ' 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> import glob 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> import re 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> import subprocess 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> def _find_admin_socket(client_name): 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> files = glob.glob(asok_path) 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> # Given a non-glob path, it better be there 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> if "*" not in asok_path: 2024-04-25T10:02:15.699 DEBUG:teuthology.orchestra.run.smithi165:> assert(len(files) == 1) 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> return files[0] 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> for f in files: 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> contents = proc_f.read() 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> if mountpoint in contents: 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> return f 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> print(_find_admin_socket("client.admin")) 2024-04-25T10:02:15.700 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:02:15.892 INFO:teuthology.orchestra.run.smithi165.stdout:/var/run/ceph/ceph-client.admin.40435.asok 2024-04-25T10:02:15.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:02:15.909 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.40435.asok 2024-04-25T10:02:15.910 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:15.910 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.40435.asok status 2024-04-25T10:02:16.137 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:02:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "metadata": { 2024-04-25T10:02:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_sha1": "97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec", 2024-04-25T10:02:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_version": "ceph version 19.0.0-1628-g97ae6cc9 (97ae6cc94daeb493fb1bfb6c82833f1485dfb1ec) squid (dev)", 2024-04-25T10:02:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "entity_id": "admin", 2024-04-25T10:02:16.137 INFO:teuthology.orchestra.run.smithi165.stdout: "hostname": "smithi165", 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "pid": "40435", 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "root": "/" 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "dentry_count": 0, 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "dentry_pinned_count": 0, 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 4419, 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "inst": { 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "name": { 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "type": "client", 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: "num": 4419 2024-04-25T10:02:16.138 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "addr": { 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "type": "v1", 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "nonce": 669194481 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "addr": { 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "type": "v1", 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "addr": "192.168.0.1:0", 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "nonce": 669194481 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:02:16.139 INFO:teuthology.orchestra.run.smithi165.stdout: "inst_str": "client.4419 192.168.0.1:0/669194481", 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "addr_str": "192.168.0.1:0/669194481", 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "inode_count": 1, 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_epoch": 7, 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_epoch": 24, 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_epoch_barrier": 0, 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "blocklisted": false, 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs" 2024-04-25T10:02:16.140 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:02:16.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:16.151 DEBUG:teuthology.orchestra.run.smithi165:> stat --file-system '--printf=%T 2024-04-25T10:02:16.151 DEBUG:teuthology.orchestra.run.smithi165:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:16.205 INFO:teuthology.orchestra.run.smithi165.stdout:fuseblk 2024-04-25T10:02:16.205 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:16.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:16.206 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:16.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:16.276 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-25T10:02:16.364 INFO:teuthology.orchestra.run.smithi165.stdout:# file: . 2024-04-25T10:02:16.364 INFO:teuthology.orchestra.run.smithi165.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-25T10:02:16.364 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:16.366 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:02:16.366 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:16.421 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:16.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.613+0000 7fb2bfe00640 1 Processor -- start 2024-04-25T10:02:16.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.613+0000 7fb2bfe00640 1 -- start start 2024-04-25T10:02:16.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.614+0000 7fb2bfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2b8144750 0x7fb2b806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.614+0000 7fb2bfe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2b80cf030 0x7fb2b806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.614+0000 7fb2bfe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b8144b30 con 0x7fb2b806eb10 2024-04-25T10:02:16.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.614+0000 7fb2bfe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80712d0 con 0x7fb2b80cf030 2024-04-25T10:02:16.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bfe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80715b0 con 0x7fb2b8144750 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bca00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb2b806eb10 0x7fb2b80ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58832/0 (socket says 172.21.15.165:58832) 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bca00640 1 -- 172.21.15.165:0/2379293467 learned_addr learned my addr 172.21.15.165:0/2379293467 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bde00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2b80cf030 0x7fb2b806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bd400640 1 --2- 172.21.15.165:0/2379293467 >> v2:172.21.15.172:3300/0 conn(0x7fb2b8144750 0x7fb2b806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2b7e00640 1 -- 172.21.15.165:0/2379293467 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559236224 0 0) 0x7fb2b8144b30 con 0x7fb2b806eb10 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2b7e00640 1 -- 172.21.15.165:0/2379293467 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a0003520 con 0x7fb2b806eb10 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bde00640 1 -- 172.21.15.165:0/2379293467 >> v2:172.21.15.172:3300/0 conn(0x7fb2b8144750 msgr2=0x7fb2b806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bde00640 1 --2- 172.21.15.165:0/2379293467 >> v2:172.21.15.172:3300/0 conn(0x7fb2b8144750 0x7fb2b806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.615+0000 7fb2bde00640 1 -- 172.21.15.165:0/2379293467 >> v1:172.21.15.165:6789/0 conn(0x7fb2b806eb10 legacy=0x7fb2b80ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.616+0000 7fb2bde00640 1 -- 172.21.15.165:0/2379293467 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b8071890 con 0x7fb2b80cf030 2024-04-25T10:02:16.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.616+0000 7fb2bd400640 1 --2- 172.21.15.165:0/2379293467 >> v2:172.21.15.172:3300/0 conn(0x7fb2b8144750 0x7fb2b806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2bde00640 1 --2- 172.21.15.165:0/2379293467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b80cf030 0x7fb2b806d290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2b806f5d0 tx=0x7fb2b002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3b51cc44051f7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2b7e00640 1 -- 172.21.15.165:0/2379293467 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0007740 con 0x7fb2b80cf030 2024-04-25T10:02:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2b7e00640 1 -- 172.21.15.165:0/2379293467 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b00078e0 con 0x7fb2b80cf030 2024-04-25T10:02:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2b7e00640 1 -- 172.21.15.165:0/2379293467 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0007cb0 con 0x7fb2b80cf030 2024-04-25T10:02:16.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2bfe00640 1 -- 172.21.15.165:0/2379293467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b80cf030 msgr2=0x7fb2b806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/2379293467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b80cf030 0x7fb2b806d290 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb2b806f5d0 tx=0x7fb2b002a680 comp rx=0 tx=0).stop 2024-04-25T10:02:16.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2bfe00640 1 -- 172.21.15.165:0/2379293467 shutdown_connections 2024-04-25T10:02:16.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/2379293467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b80cf030 0x7fb2b806d290 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.617+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/2379293467 >> v2:172.21.15.172:3300/0 conn(0x7fb2b8144750 0x7fb2b806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.618+0000 7fb2bfe00640 1 -- 172.21.15.165:0/2379293467 >> 172.21.15.165:0/2379293467 conn(0x7fb2b806abe0 msgr2=0x7fb2b806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.618+0000 7fb2bfe00640 1 -- 172.21.15.165:0/2379293467 shutdown_connections 2024-04-25T10:02:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.618+0000 7fb2bfe00640 1 -- 172.21.15.165:0/2379293467 wait complete. 2024-04-25T10:02:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.619+0000 7fb2bfe00640 1 Processor -- start 2024-04-25T10:02:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.619+0000 7fb2bfe00640 1 -- start start 2024-04-25T10:02:16.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.620+0000 7fb2bfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b806eb10 0x7fb2b813dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.620+0000 7fb2bfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80cf030 0x7fb2b815c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.620+0000 7fb2bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 0x7fb2b80b1990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.620+0000 7fb2bfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b81455d0 con 0x7fb2b8144750 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.620+0000 7fb2bfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8145450 con 0x7fb2b806eb10 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.620+0000 7fb2bfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b815c000 con 0x7fb2b80cf030 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 0x7fb2b80b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 0x7fb2b80b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52900/0 (socket says 172.21.15.165:52900) 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 -- 172.21.15.165:0/3140172334 learned_addr learned my addr 172.21.15.165:0/3140172334 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bd400640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b806eb10 0x7fb2b813dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bca00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80cf030 0x7fb2b815c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 -- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80cf030 msgr2=0x7fb2b815c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80cf030 0x7fb2b815c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 -- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b806eb10 msgr2=0x7fb2b813dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b806eb10 0x7fb2b813dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 -- 172.21.15.165:0/3140172334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80ae2d0 con 0x7fb2b8144750 2024-04-25T10:02:16.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.621+0000 7fb2bde00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 0x7fb2b80b1990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009e50 tx=0x7fb2b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.622+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b003f070 con 0x7fb2b8144750 2024-04-25T10:02:16.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.622+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b0031bf0 con 0x7fb2b8144750 2024-04-25T10:02:16.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.622+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0009450 con 0x7fb2b8144750 2024-04-25T10:02:16.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.622+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80add10 con 0x7fb2b8144750 2024-04-25T10:02:16.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.622+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b80b03b0 con 0x7fb2b8144750 2024-04-25T10:02:16.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.623+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b8073990 con 0x7fb2b8144750 2024-04-25T10:02:16.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.624+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2b003b410 con 0x7fb2b8144750 2024-04-25T10:02:16.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.625+0000 7fb2b5600640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb29806b4d0 0x7fb29806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:16.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.625+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb2b00abd00 con 0x7fb2b8144750 2024-04-25T10:02:16.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.625+0000 7fb2bd400640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb29806b4d0 0x7fb29806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:16.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.626+0000 7fb2bd400640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb29806b4d0 0x7fb29806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.630+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2b007a150 con 0x7fb2b8144750 2024-04-25T10:02:16.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.825+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2b80d0510 con 0x7fb2b8144750 2024-04-25T10:02:16.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.827+0000 7fb2b5600640 1 -- 172.21.15.165:0/3140172334 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fb2b007e1f0 con 0x7fb2b8144750 2024-04-25T10:02:16.832 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:16.832 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:16.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.829+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb29806b4d0 msgr2=0x7fb29806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.829+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb29806b4d0 0x7fb29806d990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac006fd0 tx=0x7fb2ac008040 comp rx=0 tx=0).stop 2024-04-25T10:02:16.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.829+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 msgr2=0x7fb2b80b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:16.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.829+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 0x7fb2b80b1990 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009e50 tx=0x7fb2b0004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 shutdown_connections 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb29806b4d0 0x7fb29806d990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2b8144750 0x7fb2b80b1990 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2b80cf030 0x7fb2b815c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 --2- 172.21.15.165:0/3140172334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2b806eb10 0x7fb2b813dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 >> 172.21.15.165:0/3140172334 conn(0x7fb2b806abe0 msgr2=0x7fb2b8140aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 shutdown_connections 2024-04-25T10:02:16.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:16.830+0000 7fb2bfe00640 1 -- 172.21.15.165:0/3140172334 wait complete. 2024-04-25T10:02:16.852 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:16.852 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:17.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.043+0000 7fa375600640 1 Processor -- start 2024-04-25T10:02:17.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.043+0000 7fa375600640 1 -- start start 2024-04-25T10:02:17.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa375600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa3700d3320 0x7fa3701689f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa375600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa370161d70 con 0x7fa3700b9750 2024-04-25T10:02:17.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa375600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa370162040 con 0x7fa37015da90 2024-04-25T10:02:17.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa375600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa370162600 con 0x7fa3700d3320 2024-04-25T10:02:17.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa367e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa3700b9750 0x7fa3700ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45754/0 (socket says 172.21.15.165:45754) 2024-04-25T10:02:17.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa36f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa3700d3320 0x7fa3701689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.045+0000 7fa367e00640 1 -- 172.21.15.165:0/204048513 learned_addr learned my addr 172.21.15.165:0/204048513 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:17.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.046+0000 7fa36f400640 1 -- 172.21.15.165:0/204048513 >> v1:172.21.15.172:6789/0 conn(0x7fa37015da90 legacy=0x7fa370161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.046+0000 7fa36f400640 1 -- 172.21.15.165:0/204048513 >> v1:172.21.15.165:6789/0 conn(0x7fa3700b9750 legacy=0x7fa3700ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.046+0000 7fa36f400640 1 -- 172.21.15.165:0/204048513 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa370162320 con 0x7fa3700d3320 2024-04-25T10:02:17.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.047+0000 7fa36f400640 1 --2- 172.21.15.165:0/204048513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700d3320 0x7fa3701689f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d90 tx=0x7fa36802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8b8562db99261 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.047+0000 7fa36e000640 1 -- 172.21.15.165:0/204048513 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368031600 con 0x7fa3700d3320 2024-04-25T10:02:17.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.047+0000 7fa36e000640 1 -- 172.21.15.165:0/204048513 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3680317a0 con 0x7fa3700d3320 2024-04-25T10:02:17.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.047+0000 7fa36e000640 1 -- 172.21.15.165:0/204048513 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368031b70 con 0x7fa3700d3320 2024-04-25T10:02:17.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 -- 172.21.15.165:0/204048513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700d3320 msgr2=0x7fa3701689f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 --2- 172.21.15.165:0/204048513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700d3320 0x7fa3701689f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d90 tx=0x7fa36802a610 comp rx=0 tx=0).stop 2024-04-25T10:02:17.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 -- 172.21.15.165:0/204048513 shutdown_connections 2024-04-25T10:02:17.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 --2- 172.21.15.165:0/204048513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700d3320 0x7fa3701689f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 -- 172.21.15.165:0/204048513 >> 172.21.15.165:0/204048513 conn(0x7fa370061a20 msgr2=0x7fa370061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 -- 172.21.15.165:0/204048513 shutdown_connections 2024-04-25T10:02:17.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.048+0000 7fa375600640 1 -- 172.21.15.165:0/204048513 wait complete. 2024-04-25T10:02:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.050+0000 7fa375600640 1 Processor -- start 2024-04-25T10:02:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.050+0000 7fa375600640 1 -- start start 2024-04-25T10:02:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa375600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3700d3320 0x7fa37014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa375600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa37015da90 0x7fa3700c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa375600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 0x7fa3700c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa375600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37015de70 con 0x7fa3700c8000 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa375600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700d1df0 con 0x7fa3700d3320 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa375600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3700ced80 con 0x7fa37015da90 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa36f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 0x7fa3700c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa36f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 0x7fa3700c9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33292/0 (socket says 172.21.15.165:33292) 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.051+0000 7fa36f400640 1 -- 172.21.15.165:0/1402687151 learned_addr learned my addr 172.21.15.165:0/1402687151 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36ea00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3700d3320 0x7fa37014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa367e00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa37015da90 0x7fa3700c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36f400640 1 -- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa37015da90 msgr2=0x7fa3700c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36f400640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa37015da90 0x7fa3700c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36f400640 1 -- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3700d3320 msgr2=0x7fa37014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36f400640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3700d3320 0x7fa37014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36f400640 1 -- 172.21.15.165:0/1402687151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3701502d0 con 0x7fa3700c8000 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36ea00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3700d3320 0x7fa37014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa367e00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa37015da90 0x7fa3700c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa36f400640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 0x7fa3700c9bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d90 tx=0x7fa368004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa368040070 con 0x7fa3700c8000 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa368030b60 con 0x7fa3700c8000 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3700c8530 con 0x7fa3700c8000 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36803c420 con 0x7fa3700c8000 2024-04-25T10:02:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.052+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3700c8af0 con 0x7fa3700c8000 2024-04-25T10:02:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.054+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa370073a00 con 0x7fa3700c8000 2024-04-25T10:02:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.054+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa368039040 con 0x7fa3700c8000 2024-04-25T10:02:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.055+0000 7fa366a00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa34c06b4d0 0x7fa34c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.055+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa3680abad0 con 0x7fa3700c8000 2024-04-25T10:02:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.056+0000 7fa36ea00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa34c06b4d0 0x7fa34c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.056+0000 7fa36ea00640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa34c06b4d0 0x7fa34c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa36000a830 tx=0x7fa360008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.060+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa368079f50 con 0x7fa3700c8000 2024-04-25T10:02:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.258+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa3700b5db0 con 0x7fa3700c8000 2024-04-25T10:02:17.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.259+0000 7fa366a00640 1 -- 172.21.15.165:0/1402687151 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa36807dff0 con 0x7fa3700c8000 2024-04-25T10:02:17.263 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa34c06b4d0 msgr2=0x7fa34c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa34c06b4d0 0x7fa34c06d990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa36000a830 tx=0x7fa360008040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 msgr2=0x7fa3700c9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 0x7fa3700c9bf0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa368009d90 tx=0x7fa368004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 shutdown_connections 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa34c06b4d0 0x7fa34c06d990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3700c8000 0x7fa3700c9bf0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa37015da90 0x7fa3700c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 --2- 172.21.15.165:0/1402687151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3700d3320 0x7fa37014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.261+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 >> 172.21.15.165:0/1402687151 conn(0x7fa370061a20 msgr2=0x7fa370061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.262+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 shutdown_connections 2024-04-25T10:02:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.262+0000 7fa375600640 1 -- 172.21.15.165:0/1402687151 wait complete. 2024-04-25T10:02:17.284 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:17.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.472+0000 7fe42ae00640 1 Processor -- start 2024-04-25T10:02:17.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.472+0000 7fe42ae00640 1 -- start start 2024-04-25T10:02:17.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe42ae00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe42406ec50 0x7fe424071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe42ae00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4240cfad0 con 0x7fe42415b710 2024-04-25T10:02:17.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe42ae00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4240cfd80 con 0x7fe4240cf640 2024-04-25T10:02:17.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe42ae00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe42415e490 con 0x7fe42406ec50 2024-04-25T10:02:17.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe423e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe42415b710 0x7fe42415e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45760/0 (socket says 172.21.15.165:45760) 2024-04-25T10:02:17.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe428e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe42406ec50 0x7fe424071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.474+0000 7fe423e00640 1 -- 172.21.15.165:0/2637027678 learned_addr learned my addr 172.21.15.165:0/2637027678 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:17.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.475+0000 7fe428e00640 1 -- 172.21.15.165:0/2637027678 >> v1:172.21.15.172:6789/0 conn(0x7fe4240cf640 legacy=0x7fe4240ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.475+0000 7fe428e00640 1 -- 172.21.15.165:0/2637027678 >> v1:172.21.15.165:6789/0 conn(0x7fe42415b710 legacy=0x7fe42415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.475+0000 7fe428e00640 1 -- 172.21.15.165:0/2637027678 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4240d0060 con 0x7fe42406ec50 2024-04-25T10:02:17.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.476+0000 7fe428e00640 1 --2- 172.21.15.165:0/2637027678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42406ec50 0x7fe424071bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009d90 tx=0x7fe41c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b48aafb7282799d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.476+0000 7fe422a00640 1 -- 172.21.15.165:0/2637027678 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c007740 con 0x7fe42406ec50 2024-04-25T10:02:17.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.476+0000 7fe422a00640 1 -- 172.21.15.165:0/2637027678 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c0078e0 con 0x7fe42406ec50 2024-04-25T10:02:17.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe422a00640 1 -- 172.21.15.165:0/2637027678 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c007cb0 con 0x7fe42406ec50 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 -- 172.21.15.165:0/2637027678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42406ec50 msgr2=0x7fe424071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2637027678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42406ec50 0x7fe424071bc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fe41c009d90 tx=0x7fe41c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 -- 172.21.15.165:0/2637027678 shutdown_connections 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2637027678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42406ec50 0x7fe424071bc0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 -- 172.21.15.165:0/2637027678 >> 172.21.15.165:0/2637027678 conn(0x7fe42406abe0 msgr2=0x7fe42406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 -- 172.21.15.165:0/2637027678 shutdown_connections 2024-04-25T10:02:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.477+0000 7fe42ae00640 1 -- 172.21.15.165:0/2637027678 wait complete. 2024-04-25T10:02:17.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.479+0000 7fe42ae00640 1 Processor -- start 2024-04-25T10:02:17.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.479+0000 7fe42ae00640 1 -- start start 2024-04-25T10:02:17.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe42ae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe42406ec50 0x7fe4241524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe42ae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4240cf640 0x7fe424151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe42ae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 0x7fe424154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe42ae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42415baf0 con 0x7fe42415b710 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe42ae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe42406f0a0 con 0x7fe42406ec50 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe42ae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe42415bc70 con 0x7fe4240cf640 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe428e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 0x7fe424154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe428e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 0x7fe424154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33304/0 (socket says 172.21.15.165:33304) 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.480+0000 7fe428e00640 1 -- 172.21.15.165:0/2684331050 learned_addr learned my addr 172.21.15.165:0/2684331050 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe423e00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe42406ec50 0x7fe4241524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe423400640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4240cf640 0x7fe424151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe428e00640 1 -- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4240cf640 msgr2=0x7fe424151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe428e00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4240cf640 0x7fe424151de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe428e00640 1 -- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe42406ec50 msgr2=0x7fe4241524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe428e00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe42406ec50 0x7fe4241524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe428e00640 1 -- 172.21.15.165:0/2684331050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe424154d30 con 0x7fe42415b710 2024-04-25T10:02:17.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.481+0000 7fe428e00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 0x7fe424154290 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe41c02ace0 tx=0x7fe41c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.482+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c040070 con 0x7fe42415b710 2024-04-25T10:02:17.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.482+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe41c031c90 con 0x7fe42415b710 2024-04-25T10:02:17.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.482+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe424155010 con 0x7fe42415b710 2024-04-25T10:02:17.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.482+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41c045680 con 0x7fe42415b710 2024-04-25T10:02:17.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.482+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4241555d0 con 0x7fe42415b710 2024-04-25T10:02:17.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.483+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4241557d0 con 0x7fe42415b710 2024-04-25T10:02:17.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.484+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe41c039040 con 0x7fe42415b710 2024-04-25T10:02:17.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.485+0000 7fe41be00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe40406b500 0x7fe40406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.485+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe41c0abff0 con 0x7fe42415b710 2024-04-25T10:02:17.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.485+0000 7fe423e00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe40406b500 0x7fe40406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.486+0000 7fe423e00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe40406b500 0x7fe40406d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe414005880 tx=0x7fe414009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.489+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4241557d0 con 0x7fe42415b710 2024-04-25T10:02:17.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.684+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe424073a00 con 0x7fe42415b710 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.686+0000 7fe41be00640 1 -- 172.21.15.165:0/2684331050 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe41c043360 con 0x7fe42415b710 2024-04-25T10:02:17.690 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe40406b500 msgr2=0x7fe40406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe40406b500 0x7fe40406d9c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe414005880 tx=0x7fe414009040 comp rx=0 tx=0).stop 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 msgr2=0x7fe424154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 0x7fe424154290 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fe41c02ace0 tx=0x7fe41c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 shutdown_connections 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe40406b500 0x7fe40406d9c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe42415b710 0x7fe424154290 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4240cf640 0x7fe424151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.688+0000 7fe42ae00640 1 --2- 172.21.15.165:0/2684331050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe42406ec50 0x7fe4241524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.689+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 >> 172.21.15.165:0/2684331050 conn(0x7fe42406abe0 msgr2=0x7fe42406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.689+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 shutdown_connections 2024-04-25T10:02:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.689+0000 7fe42ae00640 1 -- 172.21.15.165:0/2684331050 wait complete. 2024-04-25T10:02:17.711 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.906+0000 7fc673200640 1 Processor -- start 2024-04-25T10:02:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.906+0000 7fc673200640 1 -- start start 2024-04-25T10:02:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc673200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc66c15dd30 0x7fc66c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc673200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc66c0b5b40 0x7fc66c0d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc673200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c15e110 con 0x7fc66c06bff0 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc673200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c06d000 con 0x7fc66c0b5b40 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc673200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0b2fd0 con 0x7fc66c15dd30 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc672800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc66c0b5b40 0x7fc66c0d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc671400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc66c06bff0 0x7fc66c0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45774/0 (socket says 172.21.15.165:45774) 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.908+0000 7fc671400640 1 -- 172.21.15.165:0/2446425397 learned_addr learned my addr 172.21.15.165:0/2446425397 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.909+0000 7fc671e00640 1 --2- 172.21.15.165:0/2446425397 >> v2:172.21.15.172:3300/0 conn(0x7fc66c15dd30 0x7fc66c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.909+0000 7fc672800640 1 -- 172.21.15.165:0/2446425397 >> v2:172.21.15.172:3300/0 conn(0x7fc66c15dd30 msgr2=0x7fc66c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:17.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.909+0000 7fc672800640 1 --2- 172.21.15.165:0/2446425397 >> v2:172.21.15.172:3300/0 conn(0x7fc66c15dd30 0x7fc66c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.909+0000 7fc672800640 1 -- 172.21.15.165:0/2446425397 >> v1:172.21.15.165:6789/0 conn(0x7fc66c06bff0 legacy=0x7fc66c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.909+0000 7fc672800640 1 -- 172.21.15.165:0/2446425397 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c06c480 con 0x7fc66c0b5b40 2024-04-25T10:02:17.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.910+0000 7fc672800640 1 --2- 172.21.15.165:0/2446425397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0b5b40 0x7fc66c0d2160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc66c08bfe0 tx=0x7fc66802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf167ab85d87f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.910+0000 7fc670a00640 1 -- 172.21.15.165:0/2446425397 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668007740 con 0x7fc66c0b5b40 2024-04-25T10:02:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.910+0000 7fc670a00640 1 -- 172.21.15.165:0/2446425397 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6680078e0 con 0x7fc66c0b5b40 2024-04-25T10:02:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.910+0000 7fc670a00640 1 -- 172.21.15.165:0/2446425397 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668007cb0 con 0x7fc66c0b5b40 2024-04-25T10:02:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 -- 172.21.15.165:0/2446425397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0b5b40 msgr2=0x7fc66c0d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 --2- 172.21.15.165:0/2446425397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0b5b40 0x7fc66c0d2160 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc66c08bfe0 tx=0x7fc66802a680 comp rx=0 tx=0).stop 2024-04-25T10:02:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 -- 172.21.15.165:0/2446425397 shutdown_connections 2024-04-25T10:02:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 --2- 172.21.15.165:0/2446425397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0b5b40 0x7fc66c0d2160 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 --2- 172.21.15.165:0/2446425397 >> v2:172.21.15.172:3300/0 conn(0x7fc66c15dd30 0x7fc66c0cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 -- 172.21.15.165:0/2446425397 >> 172.21.15.165:0/2446425397 conn(0x7fc66c0d0610 msgr2=0x7fc66c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 -- 172.21.15.165:0/2446425397 shutdown_connections 2024-04-25T10:02:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.911+0000 7fc673200640 1 -- 172.21.15.165:0/2446425397 wait complete. 2024-04-25T10:02:17.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.913+0000 7fc673200640 1 Processor -- start 2024-04-25T10:02:17.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.913+0000 7fc673200640 1 -- start start 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc673200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66c0b5b40 0x7fc66c15b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc673200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66c15dd30 0x7fc66c1606d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc673200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 0x7fc66c160c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc673200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c06cd20 con 0x7fc66c0cbad0 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc673200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c06ca40 con 0x7fc66c0b5b40 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc673200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c0cf110 con 0x7fc66c15dd30 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc672800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 0x7fc66c160c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc672800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 0x7fc66c160c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33326/0 (socket says 172.21.15.165:33326) 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc672800640 1 -- 172.21.15.165:0/127734707 learned_addr learned my addr 172.21.15.165:0/127734707 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc671e00640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66c0b5b40 0x7fc66c15b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.914+0000 7fc671400640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66c15dd30 0x7fc66c1606d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc672800640 1 -- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66c15dd30 msgr2=0x7fc66c1606d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc672800640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66c15dd30 0x7fc66c1606d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc672800640 1 -- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66c0b5b40 msgr2=0x7fc66c15b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:17.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc672800640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66c0b5b40 0x7fc66c15b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:17.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc672800640 1 -- 172.21.15.165:0/127734707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c1573d0 con 0x7fc66c0cbad0 2024-04-25T10:02:17.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc671e00640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66c0b5b40 0x7fc66c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:17.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc672800640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 0x7fc66c160c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc66802ac70 tx=0x7fc668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668040070 con 0x7fc66c0cbad0 2024-04-25T10:02:17.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc668031bc0 con 0x7fc66c0cbad0 2024-04-25T10:02:17.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc668045460 con 0x7fc66c0cbad0 2024-04-25T10:02:17.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.915+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c1f6c50 con 0x7fc66c0cbad0 2024-04-25T10:02:17.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.916+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c1f7210 con 0x7fc66c0cbad0 2024-04-25T10:02:17.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.917+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c0b78b0 con 0x7fc66c0cbad0 2024-04-25T10:02:17.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.918+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc668039040 con 0x7fc66c0cbad0 2024-04-25T10:02:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.918+0000 7fc666000640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc65006b4d0 0x7fc65006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.919+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc6680abe50 con 0x7fc66c0cbad0 2024-04-25T10:02:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.919+0000 7fc671e00640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc65006b4d0 0x7fc65006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.919+0000 7fc671e00640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc65006b4d0 0x7fc65006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc66c1f1120 tx=0x7fc660008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:17.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:17.923+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc66c0b78b0 con 0x7fc66c0cbad0 2024-04-25T10:02:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.119+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc66c0cdda0 con 0x7fc66c0cbad0 2024-04-25T10:02:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.120+0000 7fc666000640 1 -- 172.21.15.165:0/127734707 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc66807a980 con 0x7fc66c0cbad0 2024-04-25T10:02:18.124 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.122+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc65006b4d0 msgr2=0x7fc65006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.122+0000 7fc673200640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc65006b4d0 0x7fc65006d990 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc66c1f1120 tx=0x7fc660008040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 msgr2=0x7fc66c160c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 0x7fc66c160c10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc66802ac70 tx=0x7fc668004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 shutdown_connections 2024-04-25T10:02:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc65006b4d0 0x7fc65006d990 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc66c0cbad0 0x7fc66c160c10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc66c15dd30 0x7fc66c1606d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 --2- 172.21.15.165:0/127734707 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc66c0b5b40 0x7fc66c15b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 >> 172.21.15.165:0/127734707 conn(0x7fc66c0d0610 msgr2=0x7fc66c0b71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 shutdown_connections 2024-04-25T10:02:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.123+0000 7fc673200640 1 -- 172.21.15.165:0/127734707 wait complete. 2024-04-25T10:02:18.146 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:18.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.337+0000 7f00f1000640 1 Processor -- start 2024-04-25T10:02:18.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.338+0000 7f00f1000640 1 -- start start 2024-04-25T10:02:18.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00f1000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f00ec15dc20 0x7f00ec0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00f1000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f00ec0d32f0 0x7f00ec0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00f1000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ec15e000 con 0x7f00ec161710 2024-04-25T10:02:18.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00f1000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ec0d1de0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00f1000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ec161c60 con 0x7f00ec15dc20 2024-04-25T10:02:18.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00e9400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f00ec161710 0x7f00ec06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45790/0 (socket says 172.21.15.165:45790) 2024-04-25T10:02:18.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00e9400640 1 -- 172.21.15.165:0/314985466 learned_addr learned my addr 172.21.15.165:0/314985466 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:18.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00ea800640 1 --2- 172.21.15.165:0/314985466 >> v2:172.21.15.165:3300/0 conn(0x7f00ec0d32f0 0x7f00ec0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.339+0000 7f00e9e00640 1 --2- 172.21.15.165:0/314985466 >> v2:172.21.15.172:3300/0 conn(0x7f00ec15dc20 0x7f00ec0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.340+0000 7f00ea800640 1 -- 172.21.15.165:0/314985466 >> v2:172.21.15.172:3300/0 conn(0x7f00ec15dc20 msgr2=0x7f00ec0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.340+0000 7f00e8a00640 1 -- 172.21.15.165:0/314985466 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3569414033 0 0) 0x7f00ec15e000 con 0x7f00ec161710 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.340+0000 7f00ea800640 1 --2- 172.21.15.165:0/314985466 >> v2:172.21.15.172:3300/0 conn(0x7f00ec15dc20 0x7f00ec0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.340+0000 7f00ea800640 1 -- 172.21.15.165:0/314985466 >> v1:172.21.15.165:6789/0 conn(0x7f00ec161710 legacy=0x7f00ec06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.340+0000 7f00ea800640 1 -- 172.21.15.165:0/314985466 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec161f40 con 0x7f00ec0d32f0 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.340+0000 7f00e9e00640 1 --2- 172.21.15.165:0/314985466 >> v2:172.21.15.172:3300/0 conn(0x7f00ec15dc20 0x7f00ec0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.341+0000 7f00ea800640 1 --2- 172.21.15.165:0/314985466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec0cebb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0d1520 tx=0x7f00e402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a10635bb394ceb6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.341+0000 7f00e8a00640 1 -- 172.21.15.165:0/314985466 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4037070 con 0x7f00ec0d32f0 2024-04-25T10:02:18.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.341+0000 7f00e8a00640 1 -- 172.21.15.165:0/314985466 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e4032440 con 0x7f00ec0d32f0 2024-04-25T10:02:18.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.341+0000 7f00e8a00640 1 -- 172.21.15.165:0/314985466 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e4032810 con 0x7f00ec0d32f0 2024-04-25T10:02:18.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.341+0000 7f00f1000640 1 -- 172.21.15.165:0/314985466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 msgr2=0x7f00ec0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.341+0000 7f00f1000640 1 --2- 172.21.15.165:0/314985466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec0cebb0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f00ec0d1520 tx=0x7f00e402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.342+0000 7f00f1000640 1 -- 172.21.15.165:0/314985466 shutdown_connections 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.342+0000 7f00f1000640 1 --2- 172.21.15.165:0/314985466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec0cebb0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.342+0000 7f00f1000640 1 --2- 172.21.15.165:0/314985466 >> v2:172.21.15.172:3300/0 conn(0x7f00ec15dc20 0x7f00ec0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.342+0000 7f00f1000640 1 -- 172.21.15.165:0/314985466 >> 172.21.15.165:0/314985466 conn(0x7f00ec0d0420 msgr2=0x7f00ec0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.342+0000 7f00f1000640 1 -- 172.21.15.165:0/314985466 shutdown_connections 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.342+0000 7f00f1000640 1 -- 172.21.15.165:0/314985466 wait complete. 2024-04-25T10:02:18.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.343+0000 7f00f1000640 1 Processor -- start 2024-04-25T10:02:18.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.343+0000 7f00f1000640 1 -- start start 2024-04-25T10:02:18.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.344+0000 7f00f1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.344+0000 7f00f1000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00ec15dc20 0x7f00ec0cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.344+0000 7f00f1000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00ec161710 0x7f00ec0cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.344+0000 7f00f1000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec0b5e50 con 0x7f00ec0d32f0 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.344+0000 7f00f1000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec162500 con 0x7f00ec161710 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.344+0000 7f00f1000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ec0b5fd0 con 0x7f00ec15dc20 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec158130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33342/0 (socket says 172.21.15.165:33342) 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 -- 172.21.15.165:0/3652469182 learned_addr learned my addr 172.21.15.165:0/3652469182 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 -- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00ec15dc20 msgr2=0x7f00ec0cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9400640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00ec15dc20 0x7f00ec0cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00ec15dc20 0x7f00ec0cd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00ea800640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00ec161710 0x7f00ec0cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 -- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00ec161710 msgr2=0x7f00ec0cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00ec161710 0x7f00ec0cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9e00640 1 -- 172.21.15.165:0/3652469182 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec15e180 con 0x7f00ec0d32f0 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.345+0000 7f00e9400640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00ec15dc20 0x7f00ec0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.346+0000 7f00e9e00640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec158130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c7f0 tx=0x7f00dc00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.346+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc015070 con 0x7f00ec0d32f0 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.346+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00dc007820 con 0x7f00ec0d32f0 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.346+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00dc007bd0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.346+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ec0cdcf0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.346+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ec15e420 con 0x7f00ec0d32f0 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.347+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ec073a00 con 0x7f00ec0d32f0 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.348+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f00dc018070 con 0x7f00ec0d32f0 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.349+0000 7f00e2000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f00c806b4d0 0x7f00c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.349+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f00dc08a6c0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.349+0000 7f00e9400640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f00c806b4d0 0x7f00c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.350+0000 7f00e9400640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f00c806b4d0 0x7f00c806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f00d4007fd0 tx=0x7f00d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.353+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f00dc0591f0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.549+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f00ec0d16b0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.550+0000 7f00e2000640 1 -- 172.21.15.165:0/3652469182 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f00dc058af0 con 0x7f00ec0d32f0 2024-04-25T10:02:18.554 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.552+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f00c806b4d0 msgr2=0x7f00c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.552+0000 7f00f1000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f00c806b4d0 0x7f00c806d990 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f00d4007fd0 tx=0x7f00d4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.552+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 msgr2=0x7f00ec158130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.552+0000 7f00f1000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec158130 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f00dc00c7f0 tx=0x7f00dc00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 shutdown_connections 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f00c806b4d0 0x7f00c806d990 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f00ec161710 0x7f00ec0cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f00ec15dc20 0x7f00ec0cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 --2- 172.21.15.165:0/3652469182 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00ec0d32f0 0x7f00ec158130 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 >> 172.21.15.165:0/3652469182 conn(0x7f00ec0d0420 msgr2=0x7f00ec15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 shutdown_connections 2024-04-25T10:02:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.553+0000 7f00f1000640 1 -- 172.21.15.165:0/3652469182 wait complete. 2024-04-25T10:02:18.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.779+0000 7fbe7e200640 1 Processor -- start 2024-04-25T10:02:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.779+0000 7fbe7e200640 1 -- start start 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.780+0000 7fbe7e200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe781452a0 0x7fbe7806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe7e200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780cccc0 con 0x7fbe781446d0 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe7e200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe780cc140 con 0x7fbe78144d50 2024-04-25T10:02:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe7e200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe780b2b20 con 0x7fbe781452a0 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe77400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbe781446d0 0x7fbe780b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45800/0 (socket says 172.21.15.165:45800) 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe77400640 1 -- 172.21.15.165:0/3054650974 learned_addr learned my addr 172.21.15.165:0/3054650974 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:18.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe77e00640 1 --2- 172.21.15.165:0/3054650974 >> v2:172.21.15.172:3300/0 conn(0x7fbe781452a0 0x7fbe7806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.781+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4202910408 0 0) 0x7fbe780cccc0 con 0x7fbe781446d0 2024-04-25T10:02:18.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe5c003520 con 0x7fbe781446d0 2024-04-25T10:02:18.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2199076382 0 0) 0x7fbe5c003520 con 0x7fbe781446d0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 >> v2:172.21.15.172:3300/0 conn(0x7fbe781452a0 msgr2=0x7fbe7806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 --2- 172.21.15.165:0/3054650974 >> v2:172.21.15.172:3300/0 conn(0x7fbe781452a0 0x7fbe7806dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 >> v1:172.21.15.172:6789/0 conn(0x7fbe78144d50 legacy=0x7fbe7806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe780cc420 con 0x7fbe781446d0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fbe68001670 con 0x7fbe781446d0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 >> v1:172.21.15.165:6789/0 conn(0x7fbe781446d0 legacy=0x7fbe780b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe5c0039b0 0x7fbe5c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c004800 0x7fbe5c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe5c0051c0 0x7fbe5c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780b2b20 con 0x7fbe5c0051c0 2024-04-25T10:02:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c00c060 con 0x7fbe5c004800 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5c00c260 con 0x7fbe5c0039b0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.782+0000 7fbe77e00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe5c0051c0 0x7fbe5c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe6fe00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c004800 0x7fbe5c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe77400640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe5c0039b0 0x7fbe5c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe77e00640 1 -- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe5c0039b0 msgr2=0x7fbe5c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe77e00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe5c0039b0 0x7fbe5c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe77e00640 1 -- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c004800 msgr2=0x7fbe5c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe77e00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c004800 0x7fbe5c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe77e00640 1 -- 172.21.15.165:0/3054650974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe5c003520 con 0x7fbe5c0051c0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.783+0000 7fbe6fe00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c004800 0x7fbe5c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.784+0000 7fbe77e00640 1 -- 172.21.15.165:0/3054650974 reap_dead start 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.784+0000 7fbe77e00640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe5c0051c0 0x7fbe5c00b9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009fc0 tx=0x7fbe5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36ac983c1fcb70b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.784+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70035020 con 0x7fbe5c0051c0 2024-04-25T10:02:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.784+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70003b90 con 0x7fbe5c0051c0 2024-04-25T10:02:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.784+0000 7fbe76a00640 1 -- 172.21.15.165:0/3054650974 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70009c10 con 0x7fbe5c0051c0 2024-04-25T10:02:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 -- 172.21.15.165:0/3054650974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe5c0051c0 msgr2=0x7fbe5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe5c0051c0 0x7fbe5c00b9c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbe70009fc0 tx=0x7fbe5c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 -- 172.21.15.165:0/3054650974 shutdown_connections 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 --2- 172.21.15.165:0/3054650974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe5c0051c0 0x7fbe5c00b9c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 -- 172.21.15.165:0/3054650974 >> 172.21.15.165:0/3054650974 conn(0x7fbe7806abe0 msgr2=0x7fbe7806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 -- 172.21.15.165:0/3054650974 shutdown_connections 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.785+0000 7fbe7e200640 1 -- 172.21.15.165:0/3054650974 wait complete. 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.787+0000 7fbe7e200640 1 Processor -- start 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.787+0000 7fbe7e200640 1 -- start start 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c0051c0 0x7fbe7813b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe7813bf90 0x7fbe780ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe7e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 0x7fbe7813d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe7e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe780cc9e0 con 0x7fbe780acdf0 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe7e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe7806d400 con 0x7fbe5c0051c0 2024-04-25T10:02:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe7e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe78144ab0 con 0x7fbe7813bf90 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe77e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 0x7fbe7813d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe77e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 0x7fbe7813d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33368/0 (socket says 172.21.15.165:33368) 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe77e00640 1 -- 172.21.15.165:0/2915852199 learned_addr learned my addr 172.21.15.165:0/2915852199 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe77400640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c0051c0 0x7fbe7813b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe6fe00640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe7813bf90 0x7fbe780ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.788+0000 7fbe77e00640 1 -- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe7813bf90 msgr2=0x7fbe780ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe77e00640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe7813bf90 0x7fbe780ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe77e00640 1 -- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c0051c0 msgr2=0x7fbe7813b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe77e00640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c0051c0 0x7fbe7813b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe77e00640 1 -- 172.21.15.165:0/2915852199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe7813b850 con 0x7fbe780acdf0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe77e00640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 0x7fbe7813d610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbe7002af10 tx=0x7fbe70009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe7003f030 con 0x7fbe780acdf0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe70031e50 con 0x7fbe780acdf0 2024-04-25T10:02:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe70035070 con 0x7fbe780acdf0 2024-04-25T10:02:18.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe780ad290 con 0x7fbe780acdf0 2024-04-25T10:02:18.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.789+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe780ad850 con 0x7fbe780acdf0 2024-04-25T10:02:18.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.790+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe78073a00 con 0x7fbe780acdf0 2024-04-25T10:02:18.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.791+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe70042030 con 0x7fbe780acdf0 2024-04-25T10:02:18.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.792+0000 7fbe6f400640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe5406b4d0 0x7fbe5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:18.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.792+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fbe700aac10 con 0x7fbe780acdf0 2024-04-25T10:02:18.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.792+0000 7fbe77400640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe5406b4d0 0x7fbe5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:18.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.793+0000 7fbe77400640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe5406b4d0 0x7fbe5406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000ff0 tx=0x7fbe68009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:18.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:18.796+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe70079100 con 0x7fbe780acdf0 2024-04-25T10:02:19.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.005+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbe78142400 con 0x7fbe780acdf0 2024-04-25T10:02:19.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.006+0000 7fbe6f400640 1 -- 172.21.15.165:0/2915852199 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbe700ae070 con 0x7fbe780acdf0 2024-04-25T10:02:19.011 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:02:19.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe5406b4d0 msgr2=0x7fbe5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe5406b4d0 0x7fbe5406d990 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbe68000ff0 tx=0x7fbe68009040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 msgr2=0x7fbe7813d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 0x7fbe7813d610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fbe7002af10 tx=0x7fbe70009750 comp rx=0 tx=0).stop 2024-04-25T10:02:19.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 shutdown_connections 2024-04-25T10:02:19.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe5406b4d0 0x7fbe5406d990 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe780acdf0 0x7fbe7813d610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe7813bf90 0x7fbe780ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 --2- 172.21.15.165:0/2915852199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe5c0051c0 0x7fbe7813b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 >> 172.21.15.165:0/2915852199 conn(0x7fbe7806abe0 msgr2=0x7fbe780b1bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 shutdown_connections 2024-04-25T10:02:19.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.009+0000 7fbe7e200640 1 -- 172.21.15.165:0/2915852199 wait complete. 2024-04-25T10:02:19.032 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:02:19.032 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:19.082 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:19.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.310+0000 7fd1cac00640 1 Processor -- start 2024-04-25T10:02:19.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.310+0000 7fd1cac00640 1 -- start start 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1cac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd1c4144750 0x7fd1c406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1cac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd1c40cf030 0x7fd1c406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1cac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1c4144b30 con 0x7fd1c406eb10 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1cac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c40712d0 con 0x7fd1c4144750 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1cac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1c40715b0 con 0x7fd1c40cf030 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1c3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd1c4144750 0x7fd1c406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1c3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd1c4144750 0x7fd1c406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33382/0 (socket says 172.21.15.165:33382) 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.312+0000 7fd1c3e00640 1 -- 172.21.15.165:0/727140129 learned_addr learned my addr 172.21.15.165:0/727140129 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:19.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.313+0000 7fd1c8c00640 1 --2- 172.21.15.165:0/727140129 >> v2:172.21.15.172:3300/0 conn(0x7fd1c40cf030 0x7fd1c406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.313+0000 7fd1c3e00640 1 -- 172.21.15.165:0/727140129 >> v2:172.21.15.172:3300/0 conn(0x7fd1c40cf030 msgr2=0x7fd1c406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.313+0000 7fd1c3e00640 1 --2- 172.21.15.165:0/727140129 >> v2:172.21.15.172:3300/0 conn(0x7fd1c40cf030 0x7fd1c406d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.313+0000 7fd1c3e00640 1 -- 172.21.15.165:0/727140129 >> v1:172.21.15.172:6789/0 conn(0x7fd1c406eb10 legacy=0x7fd1c40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.313+0000 7fd1c3e00640 1 -- 172.21.15.165:0/727140129 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c4071890 con 0x7fd1c4144750 2024-04-25T10:02:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.314+0000 7fd1c3e00640 1 --2- 172.21.15.165:0/727140129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c4144750 0x7fd1c406e5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd1c406f720 tx=0x7fd1b402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=799e38fa9ee6ae8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.314+0000 7fd1c2a00640 1 -- 172.21.15.165:0/727140129 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4037070 con 0x7fd1c4144750 2024-04-25T10:02:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.314+0000 7fd1c2a00640 1 -- 172.21.15.165:0/727140129 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b4032440 con 0x7fd1c4144750 2024-04-25T10:02:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.314+0000 7fd1c2a00640 1 -- 172.21.15.165:0/727140129 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b4032810 con 0x7fd1c4144750 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 -- 172.21.15.165:0/727140129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c4144750 msgr2=0x7fd1c406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 --2- 172.21.15.165:0/727140129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c4144750 0x7fd1c406e5d0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd1c406f720 tx=0x7fd1b402a230 comp rx=0 tx=0).stop 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 -- 172.21.15.165:0/727140129 shutdown_connections 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 --2- 172.21.15.165:0/727140129 >> v2:172.21.15.172:3300/0 conn(0x7fd1c40cf030 0x7fd1c406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 --2- 172.21.15.165:0/727140129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c4144750 0x7fd1c406e5d0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 -- 172.21.15.165:0/727140129 >> 172.21.15.165:0/727140129 conn(0x7fd1c406abe0 msgr2=0x7fd1c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 -- 172.21.15.165:0/727140129 shutdown_connections 2024-04-25T10:02:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.315+0000 7fd1cac00640 1 -- 172.21.15.165:0/727140129 wait complete. 2024-04-25T10:02:19.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.317+0000 7fd1cac00640 1 Processor -- start 2024-04-25T10:02:19.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.317+0000 7fd1cac00640 1 -- start start 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1cac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c406eb10 0x7fd1c4142a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1cac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 0x7fd1c413fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1cac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c4144750 0x7fd1c41400d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1cac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c41455d0 con 0x7fd1c40cf030 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1cac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c4145450 con 0x7fd1c4144750 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1cac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1c415c000 con 0x7fd1c406eb10 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1c3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 0x7fd1c413fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1c3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 0x7fd1c413fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33390/0 (socket says 172.21.15.165:33390) 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.318+0000 7fd1c3400640 1 -- 172.21.15.165:0/3502790296 learned_addr learned my addr 172.21.15.165:0/3502790296 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:19.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c8c00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c4144750 0x7fd1c41400d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3e00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c406eb10 0x7fd1c4142a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3400640 1 -- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c406eb10 msgr2=0x7fd1c4142a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3400640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c406eb10 0x7fd1c4142a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3400640 1 -- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c4144750 msgr2=0x7fd1c41400d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3400640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c4144750 0x7fd1c41400d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3400640 1 -- 172.21.15.165:0/3502790296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1c41f25e0 con 0x7fd1c40cf030 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1c3400640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 0x7fd1c413fb90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000b3f0 tx=0x7fd1b000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0015020 con 0x7fd1c40cf030 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1b0011830 con 0x7fd1c40cf030 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1b0011c20 con 0x7fd1c40cf030 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.319+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1c41f2bc0 con 0x7fd1c40cf030 2024-04-25T10:02:19.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.320+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1c41f28c0 con 0x7fd1c40cf030 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.321+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1c4073990 con 0x7fd1c40cf030 2024-04-25T10:02:19.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.322+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1b0018020 con 0x7fd1c40cf030 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.323+0000 7fd1bbe00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1a406b520 0x7fd1a406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.323+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd1b008a140 con 0x7fd1c40cf030 2024-04-25T10:02:19.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.323+0000 7fd1c3e00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1a406b520 0x7fd1a406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.324+0000 7fd1c3e00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1a406b520 0x7fd1a406d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4034000 tx=0x7fd1b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.327+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd1b001f030 con 0x7fd1c40cf030 2024-04-25T10:02:19.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.528+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd1c40d0510 con 0x7fd1c40cf030 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.531+0000 7fd1bbe00640 1 -- 172.21.15.165:0/3502790296 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fd1b00172c0 con 0x7fd1c40cf030 2024-04-25T10:02:19.535 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:19.536 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1a406b520 msgr2=0x7fd1a406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1a406b520 0x7fd1a406d9e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4034000 tx=0x7fd1b402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 msgr2=0x7fd1c413fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 0x7fd1c413fb90 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd1b000b3f0 tx=0x7fd1b000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 shutdown_connections 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1a406b520 0x7fd1a406d9e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd1c4144750 0x7fd1c41400d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1c40cf030 0x7fd1c413fb90 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 --2- 172.21.15.165:0/3502790296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1c406eb10 0x7fd1c4142a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 >> 172.21.15.165:0/3502790296 conn(0x7fd1c406abe0 msgr2=0x7fd1c4143a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 shutdown_connections 2024-04-25T10:02:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.534+0000 7fd1cac00640 1 -- 172.21.15.165:0/3502790296 wait complete. 2024-04-25T10:02:19.558 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:19.558 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.749+0000 7fd216e00640 1 Processor -- start 2024-04-25T10:02:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.750+0000 7fd216e00640 1 -- start start 2024-04-25T10:02:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.751+0000 7fd216e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd210144d50 0x7fd21006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.751+0000 7fd216e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2101452a0 0x7fd21006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.751+0000 7fd216e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2100cccc0 con 0x7fd2101446d0 2024-04-25T10:02:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.751+0000 7fd216e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2100b2b20 con 0x7fd2101452a0 2024-04-25T10:02:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.751+0000 7fd216e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21006d400 con 0x7fd210144d50 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2101452a0 0x7fd21006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd2101452a0 0x7fd21006ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:54470/0 (socket says 172.21.15.165:54470) 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 -- 172.21.15.165:0/343856334 learned_addr learned my addr 172.21.15.165:0/343856334 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd20f400640 1 --2- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd210144d50 0x7fd21006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 -- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd210144d50 msgr2=0x7fd21006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 --2- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd210144d50 0x7fd21006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 -- 172.21.15.165:0/343856334 >> v1:172.21.15.172:6789/0 conn(0x7fd2101446d0 legacy=0x7fd2100b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.752+0000 7fd214e00640 1 -- 172.21.15.165:0/343856334 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2100cc140 con 0x7fd2101452a0 2024-04-25T10:02:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.754+0000 7fd214e00640 1 --2- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2101452a0 0x7fd21006ddd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd208009e50 tx=0x7fd20802a730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=402e8689dc4c88ad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.754+0000 7fd20ea00640 1 -- 172.21.15.165:0/343856334 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208036070 con 0x7fd2101452a0 2024-04-25T10:02:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.754+0000 7fd20ea00640 1 -- 172.21.15.165:0/343856334 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208031600 con 0x7fd2101452a0 2024-04-25T10:02:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.754+0000 7fd20ea00640 1 -- 172.21.15.165:0/343856334 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080319d0 con 0x7fd2101452a0 2024-04-25T10:02:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 -- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2101452a0 msgr2=0x7fd21006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 --2- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2101452a0 0x7fd21006ddd0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fd208009e50 tx=0x7fd20802a730 comp rx=0 tx=0).stop 2024-04-25T10:02:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 -- 172.21.15.165:0/343856334 shutdown_connections 2024-04-25T10:02:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 --2- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd210144d50 0x7fd21006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 --2- 172.21.15.165:0/343856334 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd2101452a0 0x7fd21006ddd0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 -- 172.21.15.165:0/343856334 >> 172.21.15.165:0/343856334 conn(0x7fd21006abe0 msgr2=0x7fd21006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 -- 172.21.15.165:0/343856334 shutdown_connections 2024-04-25T10:02:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.755+0000 7fd216e00640 1 -- 172.21.15.165:0/343856334 wait complete. 2024-04-25T10:02:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.757+0000 7fd216e00640 1 Processor -- start 2024-04-25T10:02:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.757+0000 7fd216e00640 1 -- start start 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd216e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2101446d0 0x7fd2101549a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd216e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd210144d50 0x7fd210154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd216e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 0x7fd210155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd216e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100cc9e0 con 0x7fd2101452a0 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd216e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2100cc700 con 0x7fd210144d50 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd216e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd210144ab0 con 0x7fd2101446d0 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 0x7fd210155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 0x7fd210155420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33398/0 (socket says 172.21.15.165:33398) 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 -- 172.21.15.165:0/3188626114 learned_addr learned my addr 172.21.15.165:0/3188626114 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd20f400640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd210144d50 0x7fd210154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd20fe00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2101446d0 0x7fd2101549a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 -- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2101446d0 msgr2=0x7fd2101549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2101446d0 0x7fd2101549a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 -- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd210144d50 msgr2=0x7fd210154ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd210144d50 0x7fd210154ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd214e00640 1 -- 172.21.15.165:0/3188626114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2101579c0 con 0x7fd2101452a0 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.758+0000 7fd20fe00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2101446d0 0x7fd2101549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.759+0000 7fd214e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 0x7fd210155420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd208034040 tx=0x7fd208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.759+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208036050 con 0x7fd2101452a0 2024-04-25T10:02:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.759+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd208042070 con 0x7fd2101452a0 2024-04-25T10:02:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.759+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd208009510 con 0x7fd2101452a0 2024-04-25T10:02:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.759+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21013c0e0 con 0x7fd2101452a0 2024-04-25T10:02:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.759+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd210142320 con 0x7fd2101452a0 2024-04-25T10:02:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.761+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd210073a00 con 0x7fd2101452a0 2024-04-25T10:02:19.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.761+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd208007bf0 con 0x7fd2101452a0 2024-04-25T10:02:19.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.762+0000 7fd207e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1f006b4d0 0x7fd1f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:19.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.762+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd2080ab320 con 0x7fd2101452a0 2024-04-25T10:02:19.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.763+0000 7fd20fe00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1f006b4d0 0x7fd1f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:19.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.763+0000 7fd20fe00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1f006b4d0 0x7fd1f006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd200001680 tx=0x7fd200009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.767+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd208045360 con 0x7fd2101452a0 2024-04-25T10:02:19.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.959+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd21006eb80 con 0x7fd2101452a0 2024-04-25T10:02:19.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.960+0000 7fd207e00640 1 -- 172.21.15.165:0/3188626114 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd208079e60 con 0x7fd2101452a0 2024-04-25T10:02:19.964 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1f006b4d0 msgr2=0x7fd1f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1f006b4d0 0x7fd1f006d990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fd200001680 tx=0x7fd200009040 comp rx=0 tx=0).stop 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 msgr2=0x7fd210155420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 0x7fd210155420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fd208034040 tx=0x7fd208004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:19.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 shutdown_connections 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd1f006b4d0 0x7fd1f006d990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd2101452a0 0x7fd210155420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd210144d50 0x7fd210154ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 --2- 172.21.15.165:0/3188626114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd2101446d0 0x7fd2101549a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 >> 172.21.15.165:0/3188626114 conn(0x7fd21006abe0 msgr2=0x7fd210140260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 shutdown_connections 2024-04-25T10:02:19.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:19.963+0000 7fd216e00640 1 -- 172.21.15.165:0/3188626114 wait complete. 2024-04-25T10:02:19.985 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:20.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.175+0000 7fcc5e600640 1 Processor -- start 2024-04-25T10:02:20.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.175+0000 7fcc5e600640 1 -- start start 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.176+0000 7fcc5e600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc5815bb20 0x7fcc580cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.176+0000 7fcc5e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580d00e0 0x7fcc580ad600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.176+0000 7fcc5e600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc580cb190 con 0x7fcc580ccdd0 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.176+0000 7fcc5e600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc580cb470 con 0x7fcc5815bb20 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc5e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580cb750 con 0x7fcc580d00e0 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc5815bb20 0x7fcc580cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc5815bb20 0x7fcc580cc230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33408/0 (socket says 172.21.15.165:33408) 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 -- 172.21.15.165:0/3254129238 learned_addr learned my addr 172.21.15.165:0/3254129238 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc56a00640 1 --1- 172.21.15.165:0/3254129238 >> v1:172.21.15.165:6789/0 conn(0x7fcc580ccdd0 0x7fcc580cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45810/0 (socket says 172.21.15.165:45810) 2024-04-25T10:02:20.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57e00640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580d00e0 0x7fcc580ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 -- 172.21.15.165:0/3254129238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580d00e0 msgr2=0x7fcc580ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580d00e0 0x7fcc580ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 -- 172.21.15.165:0/3254129238 >> v1:172.21.15.165:6789/0 conn(0x7fcc580ccdd0 legacy=0x7fcc580cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57400640 1 -- 172.21.15.165:0/3254129238 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580cba30 con 0x7fcc5815bb20 2024-04-25T10:02:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc56000640 1 -- 172.21.15.165:0/3254129238 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078643358 0 0) 0x7fcc580cb190 con 0x7fcc580ccdd0 2024-04-25T10:02:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.177+0000 7fcc57e00640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580d00e0 0x7fcc580ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.178+0000 7fcc57400640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580cc230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009870 tx=0x7fcc4802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=396d13f36ac404 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc56000640 1 -- 172.21.15.165:0/3254129238 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48037070 con 0x7fcc5815bb20 2024-04-25T10:02:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc56000640 1 -- 172.21.15.165:0/3254129238 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc48032440 con 0x7fcc5815bb20 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc56000640 1 -- 172.21.15.165:0/3254129238 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc48032810 con 0x7fcc5815bb20 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc5e600640 1 -- 172.21.15.165:0/3254129238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 msgr2=0x7fcc580cc230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc5e600640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580cc230 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fcc48009870 tx=0x7fcc4802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc5e600640 1 -- 172.21.15.165:0/3254129238 shutdown_connections 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc5e600640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580d00e0 0x7fcc580ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc5e600640 1 --2- 172.21.15.165:0/3254129238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580cc230 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.179+0000 7fcc5e600640 1 -- 172.21.15.165:0/3254129238 >> 172.21.15.165:0/3254129238 conn(0x7fcc580b2390 msgr2=0x7fcc580b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.180+0000 7fcc5e600640 1 -- 172.21.15.165:0/3254129238 shutdown_connections 2024-04-25T10:02:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.180+0000 7fcc5e600640 1 -- 172.21.15.165:0/3254129238 wait complete. 2024-04-25T10:02:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.181+0000 7fcc5e600640 1 Processor -- start 2024-04-25T10:02:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.181+0000 7fcc5e600640 1 -- start start 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.182+0000 7fcc5e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580ccdd0 0x7fcc58148b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.182+0000 7fcc5e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc580d00e0 0x7fcc58149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.182+0000 7fcc5e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.182+0000 7fcc5e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5815bf00 con 0x7fcc5815bb20 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.182+0000 7fcc5e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc580cc770 con 0x7fcc580d00e0 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.182+0000 7fcc5e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5815c080 con 0x7fcc580ccdd0 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33424/0 (socket says 172.21.15.165:33424) 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 -- 172.21.15.165:0/2167215012 learned_addr learned my addr 172.21.15.165:0/2167215012 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc56a00640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc580d00e0 0x7fcc58149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57400640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580ccdd0 0x7fcc58148b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 -- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580ccdd0 msgr2=0x7fcc58148b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580ccdd0 0x7fcc58148b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 -- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc580d00e0 msgr2=0x7fcc58149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc580d00e0 0x7fcc58149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.183+0000 7fcc57e00640 1 -- 172.21.15.165:0/2167215012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc580b4b60 con 0x7fcc5815bb20 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.184+0000 7fcc57e00640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580c9fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000cef0 tx=0x7fcc5000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.184+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5001a070 con 0x7fcc5815bb20 2024-04-25T10:02:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.184+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5000d860 con 0x7fcc5815bb20 2024-04-25T10:02:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.184+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5000dc50 con 0x7fcc5815bb20 2024-04-25T10:02:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.184+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc580b4e40 con 0x7fcc5815bb20 2024-04-25T10:02:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.184+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc580b5400 con 0x7fcc5815bb20 2024-04-25T10:02:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.185+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc58073a00 con 0x7fcc5815bb20 2024-04-25T10:02:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.186+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc5001d070 con 0x7fcc5815bb20 2024-04-25T10:02:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.187+0000 7fcc4f400640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc3806b4d0 0x7fcc3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.187+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fcc5008aeb0 con 0x7fcc5815bb20 2024-04-25T10:02:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.187+0000 7fcc57400640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc3806b4d0 0x7fcc3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.188+0000 7fcc57400640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc3806b4d0 0x7fcc3806d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc48034000 tx=0x7fcc4802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.191+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc50059a10 con 0x7fcc5815bb20 2024-04-25T10:02:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.383+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc5806d330 con 0x7fcc5815bb20 2024-04-25T10:02:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.384+0000 7fcc4f400640 1 -- 172.21.15.165:0/2167215012 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc50059310 con 0x7fcc5815bb20 2024-04-25T10:02:20.389 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:20.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.387+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc3806b4d0 msgr2=0x7fcc3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.387+0000 7fcc5e600640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc3806b4d0 0x7fcc3806d990 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcc48034000 tx=0x7fcc4802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.387+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 msgr2=0x7fcc580c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.387+0000 7fcc5e600640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580c9fd0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fcc5000cef0 tx=0x7fcc5000a710 comp rx=0 tx=0).stop 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 shutdown_connections 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc3806b4d0 0x7fcc3806d990 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc5815bb20 0x7fcc580c9fd0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc580d00e0 0x7fcc58149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 --2- 172.21.15.165:0/2167215012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc580ccdd0 0x7fcc58148b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 >> 172.21.15.165:0/2167215012 conn(0x7fcc580b2390 msgr2=0x7fcc580aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 shutdown_connections 2024-04-25T10:02:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.388+0000 7fcc5e600640 1 -- 172.21.15.165:0/2167215012 wait complete. 2024-04-25T10:02:20.410 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:20.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.593+0000 7f0701400640 1 Processor -- start 2024-04-25T10:02:20.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.593+0000 7f0701400640 1 -- start start 2024-04-25T10:02:20.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.595+0000 7f0701400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f06fc0b95c0 0x7f06fc0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.595+0000 7f0701400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.595+0000 7f0701400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06fc161e70 con 0x7f06fc15dbd0 2024-04-25T10:02:20.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.595+0000 7f0701400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06fc0d1ce0 con 0x7f06fc0b95c0 2024-04-25T10:02:20.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.595+0000 7f0701400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc162150 con 0x7f06fc0b5910 2024-04-25T10:02:20.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.595+0000 7f06f3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f06fc0b95c0 0x7f06fc0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f0700a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f06f3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f06fc0b95c0 0x7f06fc0b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:54490/0 (socket says 172.21.15.165:54490) 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f06f3e00640 1 -- 172.21.15.165:0/1191517172 learned_addr learned my addr 172.21.15.165:0/1191517172 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f06f3e00640 1 -- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 msgr2=0x7f06fc0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f06f3e00640 1 --2- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f06f3e00640 1 -- 172.21.15.165:0/1191517172 >> v1:172.21.15.172:6789/0 conn(0x7f06fc15dbd0 legacy=0x7f06fc161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.596+0000 7f06f3e00640 1 -- 172.21.15.165:0/1191517172 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc162430 con 0x7f06fc0b95c0 2024-04-25T10:02:20.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.598+0000 7f06f3e00640 1 --2- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc0b53d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009870 tx=0x7f06e402a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6028585870e30e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.598+0000 7f06fb400640 1 -- 172.21.15.165:0/1191517172 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4037070 con 0x7f06fc0b95c0 2024-04-25T10:02:20.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.598+0000 7f06fb400640 1 -- 172.21.15.165:0/1191517172 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e4032440 con 0x7f06fc0b95c0 2024-04-25T10:02:20.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.598+0000 7f06fb400640 1 -- 172.21.15.165:0/1191517172 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e4032810 con 0x7f06fc0b95c0 2024-04-25T10:02:20.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 -- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 msgr2=0x7f06fc0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 --2- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc0b53d0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f06e4009870 tx=0x7f06e402a270 comp rx=0 tx=0).stop 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 -- 172.21.15.165:0/1191517172 shutdown_connections 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 --2- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 --2- 172.21.15.165:0/1191517172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc0b53d0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 -- 172.21.15.165:0/1191517172 >> 172.21.15.165:0/1191517172 conn(0x7f06fc0d03d0 msgr2=0x7f06fc0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 -- 172.21.15.165:0/1191517172 shutdown_connections 2024-04-25T10:02:20.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.599+0000 7f0701400640 1 -- 172.21.15.165:0/1191517172 wait complete. 2024-04-25T10:02:20.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.601+0000 7f0701400640 1 Processor -- start 2024-04-25T10:02:20.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.602+0000 7f0701400640 1 -- start start 2024-04-25T10:02:20.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0701400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0701400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc160670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0701400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 0x7f06fc160bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0701400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0d2830 con 0x7f06fc15dbd0 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0701400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0d26b0 con 0x7f06fc0b95c0 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0701400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06fc0d29b0 con 0x7f06fc0b5910 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0700a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 0x7f06fc160bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0700a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 0x7f06fc160bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33432/0 (socket says 172.21.15.165:33432) 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f0700a00640 1 -- 172.21.15.165:0/2970311038 learned_addr learned my addr 172.21.15.165:0/2970311038 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f06f3e00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc160670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.603+0000 7f06fbe00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0700a00640 1 -- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 msgr2=0x7f06fc0b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0700a00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0b81b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0700a00640 1 -- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 msgr2=0x7f06fc160670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0700a00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc160670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0700a00640 1 -- 172.21.15.165:0/2970311038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06fc159f90 con 0x7f06fc15dbd0 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f06f3e00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0700a00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 0x7f06fc160bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06f400efc0 tx=0x7f06f400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f4018020 con 0x7f06fc15dbd0 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06f4009b10 con 0x7f06fc15dbd0 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06fc15a270 con 0x7f06fc15dbd0 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f40146c0 con 0x7f06fc15dbd0 2024-04-25T10:02:20.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.604+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06fc15a830 con 0x7f06fc15dbd0 2024-04-25T10:02:20.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.606+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06fc0d1520 con 0x7f06fc15dbd0 2024-04-25T10:02:20.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.606+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06f401b020 con 0x7f06fc15dbd0 2024-04-25T10:02:20.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.608+0000 7f06f8c00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06d806b4d0 0x7f06d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:20.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.608+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f06f408aff0 con 0x7f06fc15dbd0 2024-04-25T10:02:20.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.608+0000 7f06fbe00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06d806b4d0 0x7f06d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:20.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.609+0000 7f06fbe00640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06d806b4d0 0x7f06d806d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06ec001500 tx=0x7f06ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:20.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.613+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06f4022030 con 0x7f06fc15dbd0 2024-04-25T10:02:20.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.804+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06fc0b3ff0 con 0x7f06fc15dbd0 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.805+0000 7f06f8c00640 1 -- 172.21.15.165:0/2970311038 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06f4010040 con 0x7f06fc15dbd0 2024-04-25T10:02:20.810 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06d806b4d0 msgr2=0x7f06d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06d806b4d0 0x7f06d806d990 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06ec001500 tx=0x7f06ec009040 comp rx=0 tx=0).stop 2024-04-25T10:02:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 msgr2=0x7f06fc160bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 0x7f06fc160bb0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f06f400efc0 tx=0x7f06f400c490 comp rx=0 tx=0).stop 2024-04-25T10:02:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 shutdown_connections 2024-04-25T10:02:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06d806b4d0 0x7f06d806d990 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06fc15dbd0 0x7f06fc160bb0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06fc0b95c0 0x7f06fc160670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 --2- 172.21.15.165:0/2970311038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06fc0b5910 0x7f06fc0b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:20.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 >> 172.21.15.165:0/2970311038 conn(0x7f06fc0d03d0 msgr2=0x7f06fc06c5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:20.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 shutdown_connections 2024-04-25T10:02:20.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:20.808+0000 7f0701400640 1 -- 172.21.15.165:0/2970311038 wait complete. 2024-04-25T10:02:20.832 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:21.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.031+0000 7f70f3c00640 1 Processor -- start 2024-04-25T10:02:21.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.031+0000 7f70f3c00640 1 -- start start 2024-04-25T10:02:21.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.032+0000 7f70f3c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f70ec1446d0 0x7f70ec0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f3c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70ec06e8a0 0x7f70ec0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f3c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70ec0cccc0 con 0x7f70ec144d50 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f3c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ec0b2b20 con 0x7f70ec1446d0 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f3c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70ec06d400 con 0x7f70ec06e8a0 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f1200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f70ec1446d0 0x7f70ec0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70ebe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f70ec144d50 0x7f70ec0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45822/0 (socket says 172.21.15.165:45822) 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70ebe00640 1 -- 172.21.15.165:0/707425287 learned_addr learned my addr 172.21.15.165:0/707425287 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f1c00640 1 --2- 172.21.15.165:0/707425287 >> v2:172.21.15.172:3300/0 conn(0x7f70ec06e8a0 0x7f70ec0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f1200640 1 -- 172.21.15.165:0/707425287 >> v2:172.21.15.172:3300/0 conn(0x7f70ec06e8a0 msgr2=0x7f70ec0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f1200640 1 --2- 172.21.15.165:0/707425287 >> v2:172.21.15.172:3300/0 conn(0x7f70ec06e8a0 0x7f70ec0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.033+0000 7f70f1200640 1 -- 172.21.15.165:0/707425287 >> v1:172.21.15.165:6789/0 conn(0x7f70ec144d50 legacy=0x7f70ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.034+0000 7f70f1200640 1 -- 172.21.15.165:0/707425287 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec0cc140 con 0x7f70ec1446d0 2024-04-25T10:02:21.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.034+0000 7f70f1c00640 1 --2- 172.21.15.165:0/707425287 >> v2:172.21.15.172:3300/0 conn(0x7f70ec06e8a0 0x7f70ec0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.035+0000 7f70f1200640 1 --2- 172.21.15.165:0/707425287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec1446d0 0x7f70ec0b3f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70ec071200 tx=0x7f70e002a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d6c060fc8545075 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.035+0000 7f70eb400640 1 -- 172.21.15.165:0/707425287 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0037070 con 0x7f70ec1446d0 2024-04-25T10:02:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.035+0000 7f70eb400640 1 -- 172.21.15.165:0/707425287 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e002ad30 con 0x7f70ec1446d0 2024-04-25T10:02:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.035+0000 7f70eb400640 1 -- 172.21.15.165:0/707425287 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e0032660 con 0x7f70ec1446d0 2024-04-25T10:02:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 -- 172.21.15.165:0/707425287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec1446d0 msgr2=0x7f70ec0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 --2- 172.21.15.165:0/707425287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec1446d0 0x7f70ec0b3f20 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f70ec071200 tx=0x7f70e002a0a0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 -- 172.21.15.165:0/707425287 shutdown_connections 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 --2- 172.21.15.165:0/707425287 >> v2:172.21.15.172:3300/0 conn(0x7f70ec06e8a0 0x7f70ec0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 --2- 172.21.15.165:0/707425287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec1446d0 0x7f70ec0b3f20 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 -- 172.21.15.165:0/707425287 >> 172.21.15.165:0/707425287 conn(0x7f70ec06abe0 msgr2=0x7f70ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 -- 172.21.15.165:0/707425287 shutdown_connections 2024-04-25T10:02:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.036+0000 7f70f3c00640 1 -- 172.21.15.165:0/707425287 wait complete. 2024-04-25T10:02:21.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.038+0000 7f70f3c00640 1 Processor -- start 2024-04-25T10:02:21.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.038+0000 7f70f3c00640 1 -- start start 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f3c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec06e8a0 0x7f70ec1398c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f3c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70ec1446d0 0x7f70ec140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f3c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 0x7f70ec140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f3c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0cc700 con 0x7f70ec144d50 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f3c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0b49a0 con 0x7f70ec06e8a0 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f3c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70ec0cc9e0 con 0x7f70ec1446d0 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 0x7f70ec140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 0x7f70ec140aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33450/0 (socket says 172.21.15.165:33450) 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f1c00640 1 -- 172.21.15.165:0/2961201533 learned_addr learned my addr 172.21.15.165:0/2961201533 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:21.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f1200640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec06e8a0 0x7f70ec1398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f1c00640 1 -- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70ec1446d0 msgr2=0x7f70ec140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.039+0000 7f70f1c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70ec1446d0 0x7f70ec140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f1c00640 1 -- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec06e8a0 msgr2=0x7f70ec1398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f1c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec06e8a0 0x7f70ec1398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f1c00640 1 -- 172.21.15.165:0/2961201533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70ec0ac3c0 con 0x7f70ec144d50 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f1200640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec06e8a0 0x7f70ec1398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f1c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 0x7f70ec140aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f70e400cef0 tx=0x7f70e400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e401a070 con 0x7f70ec144d50 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70e400d860 con 0x7f70ec144d50 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70e400dc50 con 0x7f70ec144d50 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70ec0acf70 con 0x7f70ec144d50 2024-04-25T10:02:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.040+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70ec0ad530 con 0x7f70ec144d50 2024-04-25T10:02:21.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.041+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70ec073a00 con 0x7f70ec144d50 2024-04-25T10:02:21.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.042+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70e401d070 con 0x7f70ec144d50 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.043+0000 7f70e8c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70cc06b500 0x7f70cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.043+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f70e408ae60 con 0x7f70ec144d50 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.043+0000 7f70f1200640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70cc06b500 0x7f70cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.044+0000 7f70f1200640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70cc06b500 0x7f70cc06d9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70e0034000 tx=0x7f70e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.048+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70e40599c0 con 0x7f70ec144d50 2024-04-25T10:02:21.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.245+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f70ec141e40 con 0x7f70ec144d50 2024-04-25T10:02:21.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.246+0000 7f70e8c00640 1 -- 172.21.15.165:0/2961201533 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f70e40592c0 con 0x7f70ec144d50 2024-04-25T10:02:21.252 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:21.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.248+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70cc06b500 msgr2=0x7f70cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.248+0000 7f70f3c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70cc06b500 0x7f70cc06d9c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f70e0034000 tx=0x7f70e002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:21.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.248+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 msgr2=0x7f70ec140aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.248+0000 7f70f3c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 0x7f70ec140aa0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f70e400cef0 tx=0x7f70e400a710 comp rx=0 tx=0).stop 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 shutdown_connections 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70cc06b500 0x7f70cc06d9c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70ec144d50 0x7f70ec140aa0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70ec1446d0 0x7f70ec140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 --2- 172.21.15.165:0/2961201533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70ec06e8a0 0x7f70ec1398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 >> 172.21.15.165:0/2961201533 conn(0x7f70ec06abe0 msgr2=0x7f70ec0cf4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 shutdown_connections 2024-04-25T10:02:21.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.249+0000 7f70f3c00640 1 -- 172.21.15.165:0/2961201533 wait complete. 2024-04-25T10:02:21.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:02:21.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.471+0000 7fe34f200640 1 Processor -- start 2024-04-25T10:02:21.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.472+0000 7fe34f200640 1 -- start start 2024-04-25T10:02:21.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.473+0000 7fe34f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 0x7fe34806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.473+0000 7fe34f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480cfd80 con 0x7fe34815b710 2024-04-25T10:02:21.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.473+0000 7fe34f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3480d0060 con 0x7fe3480ac8b0 2024-04-25T10:02:21.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.473+0000 7fe34f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34806f190 con 0x7fe3480cf640 2024-04-25T10:02:21.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe34815b710 0x7fe34815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45832/0 (socket says 172.21.15.165:45832) 2024-04-25T10:02:21.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347e00640 1 -- 172.21.15.165:0/1650250600 learned_addr learned my addr 172.21.15.165:0/1650250600 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076210589 0 0) 0x7fe3480cfd80 con 0x7fe34815b710 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe32c003520 con 0x7fe34815b710 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe33f400640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 0x7fe34806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1522281221 0 0) 0x7fe32c003520 con 0x7fe34815b710 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 msgr2=0x7fe34806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 0x7fe34806ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 >> v1:172.21.15.172:6789/0 conn(0x7fe3480ac8b0 legacy=0x7fe348071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3480d0340 con 0x7fe34815b710 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe338001670 con 0x7fe34815b710 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.474+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 >> v1:172.21.15.165:6789/0 conn(0x7fe34815b710 legacy=0x7fe34815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347400640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 0x7fe32c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347400640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004010 0x7fe32c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347400640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe32c004d40 0x7fe32c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34806f190 con 0x7fe32c004d40 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c00bb80 con 0x7fe32c004010 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c00bd80 con 0x7fe3480cf640 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe34d200640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe32c004d40 0x7fe32c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe347e00640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 0x7fe32c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe33f400640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004010 0x7fe32c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe34d200640 1 -- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 msgr2=0x7fe32c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe34d200640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3480cf640 0x7fe32c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe34d200640 1 -- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004010 msgr2=0x7fe32c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:21.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe34d200640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004010 0x7fe32c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.475+0000 7fe34d200640 1 -- 172.21.15.165:0/1650250600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c003520 con 0x7fe32c004d40 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.476+0000 7fe34d200640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe32c004d40 0x7fe32c0051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe340001e50 tx=0x7fe32c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a33995b3a6bd28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340001a70 con 0x7fe32c004d40 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe340003970 con 0x7fe32c004d40 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe347400640 1 -- 172.21.15.165:0/1650250600 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340003d60 con 0x7fe32c004d40 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34f200640 1 -- 172.21.15.165:0/1650250600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe32c004d40 msgr2=0x7fe32c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34f200640 1 --2- 172.21.15.165:0/1650250600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe32c004d40 0x7fe32c0051c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe340001e50 tx=0x7fe32c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34d200640 1 -- 172.21.15.165:0/1650250600 reap_dead start 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34f200640 1 -- 172.21.15.165:0/1650250600 shutdown_connections 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34f200640 1 -- 172.21.15.165:0/1650250600 >> 172.21.15.165:0/1650250600 conn(0x7fe34806abe0 msgr2=0x7fe34806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34f200640 1 -- 172.21.15.165:0/1650250600 shutdown_connections 2024-04-25T10:02:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.477+0000 7fe34f200640 1 -- 172.21.15.165:0/1650250600 wait complete. 2024-04-25T10:02:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.479+0000 7fe34f200640 1 Processor -- start 2024-04-25T10:02:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.479+0000 7fe34f200640 1 -- start start 2024-04-25T10:02:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004d40 0x7fe348158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe348159200 0x7fe348159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 0x7fe3481560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34815baf0 con 0x7fe348155c00 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34815e490 con 0x7fe32c004d40 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34815bc70 con 0x7fe348159200 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 0x7fe3481560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 0x7fe3481560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33474/0 (socket says 172.21.15.165:33474) 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.480+0000 7fe34d200640 1 -- 172.21.15.165:0/87265636 learned_addr learned my addr 172.21.15.165:0/87265636 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe347e00640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004d40 0x7fe348158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe34d200640 1 -- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe348159200 msgr2=0x7fe348159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe34d200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe348159200 0x7fe348159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe34d200640 1 -- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004d40 msgr2=0x7fe348158cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe34d200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004d40 0x7fe348158cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe34d200640 1 -- 172.21.15.165:0/87265636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe348159fd0 con 0x7fe348155c00 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe347e00640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004d40 0x7fe348158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe34d200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 0x7fe3481560b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe34002a920 tx=0x7fe3400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.481+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340037040 con 0x7fe348155c00 2024-04-25T10:02:21.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.482+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe340003e50 con 0x7fe348155c00 2024-04-25T10:02:21.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.482+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340001a70 con 0x7fe348155c00 2024-04-25T10:02:21.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.482+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34815a2b0 con 0x7fe348155c00 2024-04-25T10:02:21.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.482+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3481532d0 con 0x7fe348155c00 2024-04-25T10:02:21.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.483+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe348073a00 con 0x7fe348155c00 2024-04-25T10:02:21.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.484+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe340042020 con 0x7fe348155c00 2024-04-25T10:02:21.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.485+0000 7fe344c00640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe32406b4d0 0x7fe32406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:21.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.485+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fe3400a9bd0 con 0x7fe348155c00 2024-04-25T10:02:21.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.485+0000 7fe347e00640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe32406b4d0 0x7fe32406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:21.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.486+0000 7fe347e00640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe32406b4d0 0x7fe32406d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe338001050 tx=0x7fe338001500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:21.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.489+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe340078780 con 0x7fe348155c00 2024-04-25T10:02:21.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.695+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe348160930 con 0x7fe348155c00 2024-04-25T10:02:21.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.696+0000 7fe344c00640 1 -- 172.21.15.165:0/87265636 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v24) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe340078080 con 0x7fe348155c00 2024-04-25T10:02:21.702 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:02:21.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.698+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe32406b4d0 msgr2=0x7fe32406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.698+0000 7fe34f200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe32406b4d0 0x7fe32406d990 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe338001050 tx=0x7fe338001500 comp rx=0 tx=0).stop 2024-04-25T10:02:21.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.698+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 msgr2=0x7fe3481560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.698+0000 7fe34f200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 0x7fe3481560b0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fe34002a920 tx=0x7fe3400014a0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 shutdown_connections 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe32406b4d0 0x7fe32406d990 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe348155c00 0x7fe3481560b0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe348159200 0x7fe348159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 --2- 172.21.15.165:0/87265636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe32c004d40 0x7fe348158cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 >> 172.21.15.165:0/87265636 conn(0x7fe34806abe0 msgr2=0x7fe3480acc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 shutdown_connections 2024-04-25T10:02:21.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:21.699+0000 7fe34f200640 1 -- 172.21.15.165:0/87265636 wait complete. 2024-04-25T10:02:21.723 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi165.front.sepia.ceph.com... 2024-04-25T10:02:21.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:21.723 DEBUG:teuthology.orchestra.run.smithi165:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-25T10:02:21.799 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi165.stderr:ceph-fuse[40435]: fuse finished with error 0 and tester_r 0 2024-04-25T10:02:21.805 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:02:21.879 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:02:21.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:21.879 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-25T10:02:21.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.140+0000 7f431b200640 1 Processor -- start 2024-04-25T10:02:22.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.141+0000 7f431b200640 1 -- start start 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.142+0000 7f431b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f431c15dd00 0x7f431c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.142+0000 7f431b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c15bf30 con 0x7f431c144440 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.142+0000 7f431b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f431c15c210 con 0x7f431c071700 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.142+0000 7f431b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f431c15c860 con 0x7f431c15dd00 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.142+0000 7f4319e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f431c144440 0x7f431c070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45840/0 (socket says 172.21.15.165:45840) 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.142+0000 7f4319e00640 1 -- 172.21.15.165:0/3942539007 learned_addr learned my addr 172.21.15.165:0/3942539007 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f431a800640 1 --2- 172.21.15.165:0/3942539007 >> v2:172.21.15.172:3300/0 conn(0x7f431c15dd00 0x7f431c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 833455615 0 0) 0x7f431c15bf30 con 0x7f431c144440 2024-04-25T10:02:22.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4300003520 con 0x7f431c144440 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3361121572 0 0) 0x7f4300003520 con 0x7f431c144440 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 >> v2:172.21.15.172:3300/0 conn(0x7f431c15dd00 msgr2=0x7f431c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 --2- 172.21.15.165:0/3942539007 >> v2:172.21.15.172:3300/0 conn(0x7f431c15dd00 0x7f431c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 >> v1:172.21.15.172:6789/0 conn(0x7f431c071700 legacy=0x7f431c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c15c4f0 con 0x7f431c144440 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f430c001670 con 0x7f431c144440 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 >> v1:172.21.15.165:6789/0 conn(0x7f431c144440 legacy=0x7f431c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43000039b0 0x7f4300003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c15c860 con 0x7f4300004800 2024-04-25T10:02:22.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430000c070 con 0x7f43000051c0 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.143+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430000c270 con 0x7f43000039b0 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319400640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f431a800640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319e00640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43000039b0 0x7f4300003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319400640 1 -- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43000039b0 msgr2=0x7f4300003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319400640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f43000039b0 0x7f4300003df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319400640 1 -- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43000051c0 msgr2=0x7f430000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319400640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f43000051c0 0x7f430000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f4319400640 1 -- 172.21.15.165:0/3942539007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4300003520 con 0x7f4300004800 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.144+0000 7f431a800640 1 -- 172.21.15.165:0/3942539007 reap_dead start 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.145+0000 7f4319400640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4300004800 0x7f4300004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4304001750 tx=0x7f4300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3cf7afba5ec0b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.145+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4304001370 con 0x7f4300004800 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.145+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43040052d0 con 0x7f4300004800 2024-04-25T10:02:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.145+0000 7f4318a00640 1 -- 172.21.15.165:0/3942539007 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430402abc0 con 0x7f4300004800 2024-04-25T10:02:22.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 -- 172.21.15.165:0/3942539007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4300004800 msgr2=0x7f4300004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4300004800 0x7f4300004c80 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4304001750 tx=0x7f4300005830 comp rx=0 tx=0).stop 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 -- 172.21.15.165:0/3942539007 shutdown_connections 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 --2- 172.21.15.165:0/3942539007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4300004800 0x7f4300004c80 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 -- 172.21.15.165:0/3942539007 >> 172.21.15.165:0/3942539007 conn(0x7f431c06abe0 msgr2=0x7f431c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 -- 172.21.15.165:0/3942539007 shutdown_connections 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.146+0000 7f431b200640 1 -- 172.21.15.165:0/3942539007 wait complete. 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.148+0000 7f431b200640 1 Processor -- start 2024-04-25T10:02:22.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.148+0000 7f431b200640 1 -- start start 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4300004800 0x7f431c138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f431c0ca4c0 0x7f431c15ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 0x7f431c13b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c071170 con 0x7f431c13ae70 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f431c070ff0 con 0x7f4300004800 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431c0712f0 con 0x7f431c0ca4c0 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 0x7f431c13b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 0x7f431c13b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33492/0 (socket says 172.21.15.165:33492) 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.149+0000 7f431a800640 1 -- 172.21.15.165:0/824430556 learned_addr learned my addr 172.21.15.165:0/824430556 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:22.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f4319e00640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4300004800 0x7f431c138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f4319400640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f431c0ca4c0 0x7f431c15ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431a800640 1 -- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f431c0ca4c0 msgr2=0x7f431c15ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431a800640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f431c0ca4c0 0x7f431c15ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431a800640 1 -- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4300004800 msgr2=0x7f431c138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431a800640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4300004800 0x7f431c138860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431a800640 1 -- 172.21.15.165:0/824430556 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f431c1f4000 con 0x7f431c13ae70 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431a800640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 0x7f431c13b320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f431c15e190 tx=0x7f43140115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314011e40 con 0x7f431c13ae70 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431401b070 con 0x7f431c13ae70 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4314003990 con 0x7f431c13ae70 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.150+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f431c1f42e0 con 0x7f431c13ae70 2024-04-25T10:02:22.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.151+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f431c1f48a0 con 0x7f431c13ae70 2024-04-25T10:02:22.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.152+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431c073990 con 0x7f431c13ae70 2024-04-25T10:02:22.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.152+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4314005d10 con 0x7f431c13ae70 2024-04-25T10:02:22.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.153+0000 7f4312000640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42f806b4d0 0x7f42f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.153+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f431408acc0 con 0x7f431c13ae70 2024-04-25T10:02:22.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.154+0000 7f4319e00640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42f806b4d0 0x7f42f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.154+0000 7f4319e00640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42f806b4d0 0x7f42f806d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f430c001eb0 tx=0x7f430c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.157+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4314091080 con 0x7f431c13ae70 2024-04-25T10:02:22.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.357+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f431c1f4a70 con 0x7f431c13ae70 2024-04-25T10:02:22.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.360+0000 7f4312000640 1 -- 172.21.15.165:0/824430556 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f4314025020 con 0x7f431c13ae70 2024-04-25T10:02:22.366 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:22.366 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.362+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42f806b4d0 msgr2=0x7f42f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.362+0000 7f431b200640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42f806b4d0 0x7f42f806d990 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f430c001eb0 tx=0x7f430c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.362+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 msgr2=0x7f431c13b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.362+0000 7f431b200640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 0x7f431c13b320 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f431c15e190 tx=0x7f43140115a0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 shutdown_connections 2024-04-25T10:02:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42f806b4d0 0x7f42f806d990 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f431c13ae70 0x7f431c13b320 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f431c0ca4c0 0x7f431c15ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 --2- 172.21.15.165:0/824430556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4300004800 0x7f431c138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 >> 172.21.15.165:0/824430556 conn(0x7f431c06abe0 msgr2=0x7f431c142e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 shutdown_connections 2024-04-25T10:02:22.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.363+0000 7f431b200640 1 -- 172.21.15.165:0/824430556 wait complete. 2024-04-25T10:02:22.385 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:22.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.572+0000 7feb00a00640 1 Processor -- start 2024-04-25T10:02:22.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.572+0000 7feb00a00640 1 -- start start 2024-04-25T10:02:22.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feb00a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 0x7feafc0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feb00a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feafc15df50 0x7feafc0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feb00a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feafc071650 con 0x7feafc0cec70 2024-04-25T10:02:22.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feb00a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feafc0cbd80 con 0x7feafc15df50 2024-04-25T10:02:22.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feb00a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc161910 con 0x7feafc15db70 2024-04-25T10:02:22.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feafa800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7feafc0cec70 0x7feafc0cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45852/0 (socket says 172.21.15.165:45852) 2024-04-25T10:02:22.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feafa800640 1 -- 172.21.15.165:0/423075927 learned_addr learned my addr 172.21.15.165:0/423075927 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feaf9e00640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 0x7feafc0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.574+0000 7feaf3e00640 1 --2- 172.21.15.165:0/423075927 >> v2:172.21.15.172:3300/0 conn(0x7feafc15df50 0x7feafc0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983849742 0 0) 0x7feaf4001490 con 0x7feafc0cec70 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feae0003520 con 0x7feafc0cec70 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453354997 0 0) 0x7feae0003520 con 0x7feafc0cec70 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 msgr2=0x7feafc0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 0x7feafc0cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 >> v2:172.21.15.172:3300/0 conn(0x7feafc15df50 msgr2=0x7feafc0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 --2- 172.21.15.165:0/423075927 >> v2:172.21.15.172:3300/0 conn(0x7feafc15df50 0x7feafc0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc0cf230 con 0x7feafc0cec70 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7feaf4001bc0 con 0x7feafc0cec70 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 >> v1:172.21.15.165:6789/0 conn(0x7feafc0cec70 legacy=0x7feafc0cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feae00039b0 0x7feae0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 0x7feae0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae0004d40 0x7feae00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc0cbd80 con 0x7feae0004d40 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc161910 con 0x7feae00039b0 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.575+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feae000b9c0 con 0x7feafc15db70 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feafa800640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae0004d40 0x7feae00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feaf9e00640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feae00039b0 0x7feae0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feaf3e00640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 0x7feae0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feafa800640 1 -- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 msgr2=0x7feae0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feafa800640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feafc15db70 0x7feae0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feafa800640 1 -- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feae00039b0 msgr2=0x7feae0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:22.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feafa800640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feae00039b0 0x7feae0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.576+0000 7feafa800640 1 -- 172.21.15.165:0/423075927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feae0003520 con 0x7feae0004d40 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.577+0000 7feafa800640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae0004d40 0x7feae00051c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feaf400a9a0 tx=0x7feae0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eba3d60ef402b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.577+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf400a5c0 con 0x7feae0004d40 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.577+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf402f070 con 0x7feae0004d40 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.577+0000 7feaf9400640 1 -- 172.21.15.165:0/423075927 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf402be10 con 0x7feae0004d40 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feb00a00640 1 -- 172.21.15.165:0/423075927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae0004d40 msgr2=0x7feae00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feb00a00640 1 --2- 172.21.15.165:0/423075927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feae0004d40 0x7feae00051c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7feaf400a9a0 tx=0x7feae0005830 comp rx=0 tx=0).stop 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feafa800640 1 -- 172.21.15.165:0/423075927 reap_dead start 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feb00a00640 1 -- 172.21.15.165:0/423075927 shutdown_connections 2024-04-25T10:02:22.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feb00a00640 1 -- 172.21.15.165:0/423075927 >> 172.21.15.165:0/423075927 conn(0x7feafc06abe0 msgr2=0x7feafc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feb00a00640 1 -- 172.21.15.165:0/423075927 shutdown_connections 2024-04-25T10:02:22.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.578+0000 7feb00a00640 1 -- 172.21.15.165:0/423075927 wait complete. 2024-04-25T10:02:22.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.580+0000 7feb00a00640 1 Processor -- start 2024-04-25T10:02:22.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.580+0000 7feb00a00640 1 -- start start 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feb00a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feae0004d40 0x7feafc0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feb00a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 0x7feafc0b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feb00a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feafc14e3c0 0x7feafc0b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feb00a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc162120 con 0x7feafc0af2c0 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feb00a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feafc161e40 con 0x7feafc14e3c0 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feb00a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feafc162640 con 0x7feae0004d40 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 0x7feafc0b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feaf3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 0x7feafc0b15d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33506/0 (socket says 172.21.15.165:33506) 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feaf3e00640 1 -- 172.21.15.165:0/1288845440 learned_addr learned my addr 172.21.15.165:0/1288845440 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.581+0000 7feaf9e00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feae0004d40 0x7feafc0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feafa800640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feafc14e3c0 0x7feafc0b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf3e00640 1 -- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feae0004d40 msgr2=0x7feafc0b1090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf3e00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feae0004d40 0x7feafc0b1090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf3e00640 1 -- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feafc14e3c0 msgr2=0x7feafc0b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf3e00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feafc14e3c0 0x7feafc0b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf3e00640 1 -- 172.21.15.165:0/1288845440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feafc160890 con 0x7feafc0af2c0 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf3e00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 0x7feafc0b15d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feae4010910 tx=0x7feae4010dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae400f040 con 0x7feafc0af2c0 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feae400ec20 con 0x7feafc0af2c0 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feafc160b70 con 0x7feafc0af2c0 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feae401c400 con 0x7feafc0af2c0 2024-04-25T10:02:22.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.582+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feafc15e5f0 con 0x7feafc0af2c0 2024-04-25T10:02:22.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.584+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feafc073990 con 0x7feafc0af2c0 2024-04-25T10:02:22.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.585+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feae401c5a0 con 0x7feafc0af2c0 2024-04-25T10:02:22.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.586+0000 7feaf2000640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fead806b500 0x7fead806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:22.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.586+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7feae408ac10 con 0x7feafc0af2c0 2024-04-25T10:02:22.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.586+0000 7feaf9e00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fead806b500 0x7fead806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:22.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.587+0000 7feaf9e00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fead806b500 0x7fead806d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feaec009990 tx=0x7feaec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:22.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.591+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feae4018030 con 0x7feafc0af2c0 2024-04-25T10:02:22.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.787+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feafc068070 con 0x7feafc0af2c0 2024-04-25T10:02:22.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.790+0000 7feaf2000640 1 -- 172.21.15.165:0/1288845440 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7feae4059740 con 0x7feafc0af2c0 2024-04-25T10:02:22.796 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:22.796 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fead806b500 msgr2=0x7fead806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fead806b500 0x7fead806d9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7feaec009990 tx=0x7feaec008040 comp rx=0 tx=0).stop 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 msgr2=0x7feafc0b15d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 0x7feafc0b15d0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feae4010910 tx=0x7feae4010dd0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 shutdown_connections 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fead806b500 0x7fead806d9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feafc14e3c0 0x7feafc0b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feafc0af2c0 0x7feafc0b15d0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 --2- 172.21.15.165:0/1288845440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feae0004d40 0x7feafc0b1090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 >> 172.21.15.165:0/1288845440 conn(0x7feafc06abe0 msgr2=0x7feafc0ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 shutdown_connections 2024-04-25T10:02:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:22.793+0000 7feb00a00640 1 -- 172.21.15.165:0/1288845440 wait complete. 2024-04-25T10:02:22.818 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:02:23.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.017+0000 7f268b200640 1 Processor -- start 2024-04-25T10:02:23.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.017+0000 7f268b200640 1 -- start start 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f268b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f268c145140 0x7f268c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f268b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f268b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268c0c5650 con 0x7f268c0ce950 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f268b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f268c0ced90 con 0x7f268c145140 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f268b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c145520 con 0x7f268c0c5200 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f2689e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f268c145140 0x7f268c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f2689e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f268c145140 0x7f268c0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33510/0 (socket says 172.21.15.165:33510) 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f2689e00640 1 -- 172.21.15.165:0/2192071481 learned_addr learned my addr 172.21.15.165:0/2192071481 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.019+0000 7f268a800640 1 --2- 172.21.15.165:0/2192071481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.020+0000 7f2689e00640 1 -- 172.21.15.165:0/2192071481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 msgr2=0x7f268c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.020+0000 7f2689e00640 1 --2- 172.21.15.165:0/2192071481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c15bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.020+0000 7f2689e00640 1 -- 172.21.15.165:0/2192071481 >> v1:172.21.15.172:6789/0 conn(0x7f268c0ce950 legacy=0x7f268c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:23.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.020+0000 7f2689e00640 1 -- 172.21.15.165:0/2192071481 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c071270 con 0x7f268c145140 2024-04-25T10:02:23.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.021+0000 7f2689e00640 1 --2- 172.21.15.165:0/2192071481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0ce410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f267c009870 tx=0x7f267c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4552c8e1e6bac363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.021+0000 7f2688a00640 1 -- 172.21.15.165:0/2192071481 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c037070 con 0x7f268c145140 2024-04-25T10:02:23.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.021+0000 7f2688a00640 1 -- 172.21.15.165:0/2192071481 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f267c032440 con 0x7f268c145140 2024-04-25T10:02:23.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.021+0000 7f2688a00640 1 -- 172.21.15.165:0/2192071481 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f267c032810 con 0x7f268c145140 2024-04-25T10:02:23.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 -- 172.21.15.165:0/2192071481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 msgr2=0x7f268c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 --2- 172.21.15.165:0/2192071481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0ce410 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f267c009870 tx=0x7f267c02a180 comp rx=0 tx=0).stop 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 -- 172.21.15.165:0/2192071481 shutdown_connections 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 --2- 172.21.15.165:0/2192071481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c15bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 --2- 172.21.15.165:0/2192071481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0ce410 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 -- 172.21.15.165:0/2192071481 >> 172.21.15.165:0/2192071481 conn(0x7f268c061cc0 msgr2=0x7f268c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 -- 172.21.15.165:0/2192071481 shutdown_connections 2024-04-25T10:02:23.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.022+0000 7f268b200640 1 -- 172.21.15.165:0/2192071481 wait complete. 2024-04-25T10:02:23.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.024+0000 7f268b200640 1 Processor -- start 2024-04-25T10:02:23.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.024+0000 7f268b200640 1 -- start start 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c13ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f268c0ce950 0x7f268c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c071b10 con 0x7f268c145140 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f268c071830 con 0x7f268c0ce950 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268c15c4f0 con 0x7f268c0c5200 2024-04-25T10:02:23.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0cb250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33522/0 (socket says 172.21.15.165:33522) 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f268a800640 1 -- 172.21.15.165:0/304763846 learned_addr learned my addr 172.21.15.165:0/304763846 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.025+0000 7f2689e00640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c13ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f2689400640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f268c0ce950 0x7f268c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268a800640 1 -- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 msgr2=0x7f268c13ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268a800640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c13ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268a800640 1 -- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f268c0ce950 msgr2=0x7f268c15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268a800640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f268c0ce950 0x7f268c15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:23.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268a800640 1 -- 172.21.15.165:0/304763846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268c13f2f0 con 0x7f268c145140 2024-04-25T10:02:23.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268a800640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0cb250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2684007e70 tx=0x7f268400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684014620 con 0x7f268c145140 2024-04-25T10:02:23.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2684019070 con 0x7f268c145140 2024-04-25T10:02:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f268c13f5d0 con 0x7f268c145140 2024-04-25T10:02:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.026+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2684014a20 con 0x7f268c145140 2024-04-25T10:02:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.027+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f268c13cd70 con 0x7f268c145140 2024-04-25T10:02:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.028+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f268c13cf70 con 0x7f268c145140 2024-04-25T10:02:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.029+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2684014bc0 con 0x7f268c145140 2024-04-25T10:02:23.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.030+0000 7f2682000640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f266c06b500 0x7f266c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:23.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.030+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f268408ade0 con 0x7f268c145140 2024-04-25T10:02:23.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.031+0000 7f2689e00640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f266c06b500 0x7f266c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:23.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.031+0000 7f2689e00640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f266c06b500 0x7f266c06d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f267c034000 tx=0x7f267c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:23.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.035+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f268c13cf70 con 0x7f268c145140 2024-04-25T10:02:23.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:23.269+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f268c073990 con 0x7f268c145140 2024-04-25T10:02:24.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.235+0000 7f2682000640 1 -- 172.21.15.165:0/304763846 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2684059910 con 0x7f268c145140 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f266c06b500 msgr2=0x7f266c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f266c06b500 0x7f266c06d9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f267c034000 tx=0x7f267c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 msgr2=0x7f268c0cb250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0cb250 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2684007e70 tx=0x7f268400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 shutdown_connections 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f266c06b500 0x7f266c06d9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f268c145140 0x7f268c0cb250 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f268c0ce950 0x7f268c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 --2- 172.21.15.165:0/304763846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f268c0c5200 0x7f268c13ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.239+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 >> 172.21.15.165:0/304763846 conn(0x7f268c061cc0 msgr2=0x7f268c15c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.240+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 shutdown_connections 2024-04-25T10:02:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.240+0000 7f268b200640 1 -- 172.21.15.165:0/304763846 wait complete. 2024-04-25T10:02:24.268 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2024-04-25T10:02:24.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:02:24.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.470+0000 7f0ade800640 1 Processor -- start 2024-04-25T10:02:24.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.471+0000 7f0ade800640 1 -- start start 2024-04-25T10:02:24.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ade800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ad815dc20 0x7f0ad80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ade800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ad8161710 0x7f0ad80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ade800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad815e000 con 0x7f0ad80b9330 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ade800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad80d1de0 con 0x7f0ad8161710 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ade800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad8161c60 con 0x7f0ad815dc20 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ad6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ad8161710 0x7f0ad80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ad6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ad8161710 0x7f0ad80cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33526/0 (socket says 172.21.15.165:33526) 2024-04-25T10:02:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.472+0000 7f0ad6a00640 1 -- 172.21.15.165:0/533310022 learned_addr learned my addr 172.21.15.165:0/533310022 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.473+0000 7f0ad7400640 1 --2- 172.21.15.165:0/533310022 >> v2:172.21.15.172:3300/0 conn(0x7f0ad815dc20 0x7f0ad80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.473+0000 7f0ad6a00640 1 -- 172.21.15.165:0/533310022 >> v2:172.21.15.172:3300/0 conn(0x7f0ad815dc20 msgr2=0x7f0ad80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.473+0000 7f0ad6a00640 1 --2- 172.21.15.165:0/533310022 >> v2:172.21.15.172:3300/0 conn(0x7f0ad815dc20 0x7f0ad80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.473+0000 7f0ad6a00640 1 -- 172.21.15.165:0/533310022 >> v1:172.21.15.172:6789/0 conn(0x7f0ad80b9330 legacy=0x7f0ad806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.473+0000 7f0ad6a00640 1 -- 172.21.15.165:0/533310022 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad8161f40 con 0x7f0ad8161710 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.473+0000 7f0ad7400640 1 --2- 172.21.15.165:0/533310022 >> v2:172.21.15.172:3300/0 conn(0x7f0ad815dc20 0x7f0ad80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.474+0000 7f0ad6a00640 1 --2- 172.21.15.165:0/533310022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad80cebb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80d1520 tx=0x7f0ac002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3f5fc72257957b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.474+0000 7f0ad6000640 1 -- 172.21.15.165:0/533310022 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0036070 con 0x7f0ad8161710 2024-04-25T10:02:24.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.474+0000 7f0ad6000640 1 -- 172.21.15.165:0/533310022 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0031600 con 0x7f0ad8161710 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.474+0000 7f0ad6000640 1 -- 172.21.15.165:0/533310022 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac00319d0 con 0x7f0ad8161710 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 -- 172.21.15.165:0/533310022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 msgr2=0x7f0ad80cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 --2- 172.21.15.165:0/533310022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad80cebb0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80d1520 tx=0x7f0ac002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 -- 172.21.15.165:0/533310022 shutdown_connections 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 --2- 172.21.15.165:0/533310022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad80cebb0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 --2- 172.21.15.165:0/533310022 >> v2:172.21.15.172:3300/0 conn(0x7f0ad815dc20 0x7f0ad80ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 -- 172.21.15.165:0/533310022 >> 172.21.15.165:0/533310022 conn(0x7f0ad80d0420 msgr2=0x7f0ad80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 -- 172.21.15.165:0/533310022 shutdown_connections 2024-04-25T10:02:24.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.475+0000 7f0ade800640 1 -- 172.21.15.165:0/533310022 wait complete. 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.476+0000 7f0ade800640 1 Processor -- start 2024-04-25T10:02:24.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.477+0000 7f0ade800640 1 -- start start 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ade800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad80b9330 0x7f0ad81e8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ade800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad815dc20 0x7f0ad81e7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ade800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad81f4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ade800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80b5e50 con 0x7f0ad8161710 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ade800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad8162500 con 0x7f0ad815dc20 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ade800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad80b5fd0 con 0x7f0ad80b9330 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad81f4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad81f4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33534/0 (socket says 172.21.15.165:33534) 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 -- 172.21.15.165:0/387150656 learned_addr learned my addr 172.21.15.165:0/387150656 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad6a00640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad815dc20 0x7f0ad81e7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 -- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad80b9330 msgr2=0x7f0ad81e8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7400640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad80b9330 0x7f0ad81e8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad80b9330 0x7f0ad81e8bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 -- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad815dc20 msgr2=0x7f0ad81e7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad815dc20 0x7f0ad81e7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad7e00640 1 -- 172.21.15.165:0/387150656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad81e9110 con 0x7f0ad8161710 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.478+0000 7f0ad6a00640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad815dc20 0x7f0ad81e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:24.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.479+0000 7f0ad7e00640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad81f4d00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b3e0 tx=0x7f0ad000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.479+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0015020 con 0x7f0ad8161710 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.479+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad0011830 con 0x7f0ad8161710 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.479+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0011c20 con 0x7f0ad8161710 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.479+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad80cbc90 con 0x7f0ad8161710 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.479+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad80cc250 con 0x7f0ad8161710 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.480+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad81e9340 con 0x7f0ad8161710 2024-04-25T10:02:24.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.481+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ad0018020 con 0x7f0ad8161710 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.482+0000 7f0acf400640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ab406b500 0x7f0ab406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.483+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0ad008a380 con 0x7f0ad8161710 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.483+0000 7f0ad7400640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ab406b500 0x7f0ab406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:24.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.483+0000 7f0ad7400640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ab406b500 0x7f0ab406d9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80b5580 tx=0x7f0ac8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:24.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.487+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ad001f020 con 0x7f0ad8161710 2024-04-25T10:02:24.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:24.720+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f0ad80739e0 con 0x7f0ad8161710 2024-04-25T10:02:25.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.239+0000 7f0acf400640 1 -- 172.21.15.165:0/387150656 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0ad00172c0 con 0x7f0ad8161710 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ab406b500 msgr2=0x7f0ab406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ab406b500 0x7f0ab406d9c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f0ad80b5580 tx=0x7f0ac8008040 comp rx=0 tx=0).stop 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 msgr2=0x7f0ad81f4d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad81f4d00 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0ad000b3e0 tx=0x7f0ad000b8a0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 shutdown_connections 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ab406b500 0x7f0ab406d9c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ad8161710 0x7f0ad81f4d00 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ad815dc20 0x7f0ad81e7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 --2- 172.21.15.165:0/387150656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ad80b9330 0x7f0ad81e8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 >> 172.21.15.165:0/387150656 conn(0x7f0ad80d0420 msgr2=0x7f0ad80b6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 shutdown_connections 2024-04-25T10:02:25.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.243+0000 7f0ade800640 1 -- 172.21.15.165:0/387150656 wait complete. 2024-04-25T10:02:25.270 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:02:25.271 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:25.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.463+0000 7f687be00640 1 Processor -- start 2024-04-25T10:02:25.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.464+0000 7f687be00640 1 -- start start 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f687be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c15db90 0x7f687c0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f687be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f687c1615a0 0x7f687c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f687be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f687c0d04f0 0x7f687c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f687be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c0d0980 con 0x7f687c1615a0 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f687be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c0d1f50 con 0x7f687c0d04f0 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f687be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c161df0 con 0x7f687c15db90 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f6872000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f687c1615a0 0x7f687c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f6872000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f687c1615a0 0x7f687c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33542/0 (socket says 172.21.15.165:33542) 2024-04-25T10:02:25.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.465+0000 7f6872000640 1 -- 172.21.15.165:0/4175678557 learned_addr learned my addr 172.21.15.165:0/4175678557 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f6872000640 1 -- 172.21.15.165:0/4175678557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c15db90 msgr2=0x7f687c0b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f6872000640 1 --2- 172.21.15.165:0/4175678557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c15db90 0x7f687c0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f6872000640 1 -- 172.21.15.165:0/4175678557 >> v2:172.21.15.172:3300/0 conn(0x7f687c0d04f0 msgr2=0x7f687c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f687b400640 1 --2- 172.21.15.165:0/4175678557 >> v2:172.21.15.172:3300/0 conn(0x7f687c0d04f0 0x7f687c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f6872000640 1 --2- 172.21.15.165:0/4175678557 >> v2:172.21.15.172:3300/0 conn(0x7f687c0d04f0 0x7f687c061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f6872000640 1 -- 172.21.15.165:0/4175678557 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c1620d0 con 0x7f687c1615a0 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.466+0000 7f687b400640 1 --2- 172.21.15.165:0/4175678557 >> v2:172.21.15.172:3300/0 conn(0x7f687c0d04f0 0x7f687c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.467+0000 7f6872000640 1 --2- 172.21.15.165:0/4175678557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c1615a0 0x7f687c0cffb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f687c08bcb0 tx=0x7f686402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667347441acc16fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.467+0000 7f687a000640 1 -- 172.21.15.165:0/4175678557 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864037070 con 0x7f687c1615a0 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.467+0000 7f687a000640 1 -- 172.21.15.165:0/4175678557 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864032440 con 0x7f687c1615a0 2024-04-25T10:02:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.467+0000 7f687a000640 1 -- 172.21.15.165:0/4175678557 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864032810 con 0x7f687c1615a0 2024-04-25T10:02:25.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 -- 172.21.15.165:0/4175678557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c1615a0 msgr2=0x7f687c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 --2- 172.21.15.165:0/4175678557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c1615a0 0x7f687c0cffb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f687c08bcb0 tx=0x7f686402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 -- 172.21.15.165:0/4175678557 shutdown_connections 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 --2- 172.21.15.165:0/4175678557 >> v2:172.21.15.172:3300/0 conn(0x7f687c0d04f0 0x7f687c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 --2- 172.21.15.165:0/4175678557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c1615a0 0x7f687c0cffb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 --2- 172.21.15.165:0/4175678557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c15db90 0x7f687c0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 -- 172.21.15.165:0/4175678557 >> 172.21.15.165:0/4175678557 conn(0x7f687c0b52b0 msgr2=0x7f687c0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 -- 172.21.15.165:0/4175678557 shutdown_connections 2024-04-25T10:02:25.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.468+0000 7f687be00640 1 -- 172.21.15.165:0/4175678557 wait complete. 2024-04-25T10:02:25.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.470+0000 7f687be00640 1 Processor -- start 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.470+0000 7f687be00640 1 -- start start 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f687be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c0d04f0 0x7f687c0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f687be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 0x7f687c0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f687be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f687c1615a0 0x7f687c1463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f687be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c161a20 con 0x7f687c15db90 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f687be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c0d1be0 con 0x7f687c1615a0 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f687be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c161ba0 con 0x7f687c0d04f0 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f6872000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 0x7f687c0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f6872000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 0x7f687c0cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33550/0 (socket says 172.21.15.165:33550) 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.471+0000 7f6872000640 1 -- 172.21.15.165:0/859066146 learned_addr learned my addr 172.21.15.165:0/859066146 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:25.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f687aa00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c0d04f0 0x7f687c0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f687b400640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f687c1615a0 0x7f687c1463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6872000640 1 -- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c0d04f0 msgr2=0x7f687c0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6872000640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c0d04f0 0x7f687c0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6872000640 1 -- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f687c1615a0 msgr2=0x7f687c1463f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6872000640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f687c1615a0 0x7f687c1463f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6872000640 1 -- 172.21.15.165:0/859066146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c1e7530 con 0x7f687c15db90 2024-04-25T10:02:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6872000640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 0x7f687c0cccb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f686402a220 tx=0x7f68640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864037040 con 0x7f687c15db90 2024-04-25T10:02:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6864042070 con 0x7f687c15db90 2024-04-25T10:02:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c1e7810 con 0x7f687c15db90 2024-04-25T10:02:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.472+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864030bb0 con 0x7f687c15db90 2024-04-25T10:02:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.473+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c144d70 con 0x7f687c15db90 2024-04-25T10:02:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.474+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c073990 con 0x7f687c15db90 2024-04-25T10:02:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.475+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6864047460 con 0x7f687c15db90 2024-04-25T10:02:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.476+0000 7f6873e00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f685806b4d0 0x7f685806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.476+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f68640ab1a0 con 0x7f687c15db90 2024-04-25T10:02:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.476+0000 7f687aa00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f685806b4d0 0x7f685806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.477+0000 7f687aa00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f685806b4d0 0x7f685806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f687c0c69d0 tx=0x7f686c0094f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.480+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6864045360 con 0x7f687c15db90 2024-04-25T10:02:25.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.712+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f687c0d14b0 con 0x7f687c15db90 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.713+0000 7f6873e00640 1 -- 172.21.15.165:0/859066146 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f6864079cd0 con 0x7f687c15db90 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:25.719 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f685806b4d0 msgr2=0x7f685806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f685806b4d0 0x7f685806d990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f687c0c69d0 tx=0x7f686c0094f0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 msgr2=0x7f687c0cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 0x7f687c0cccb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f686402a220 tx=0x7f68640046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 shutdown_connections 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f685806b4d0 0x7f685806d990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f687c1615a0 0x7f687c1463f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f687c15db90 0x7f687c0cccb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 --2- 172.21.15.165:0/859066146 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f687c0d04f0 0x7f687c0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 >> 172.21.15.165:0/859066146 conn(0x7f687c0b52b0 msgr2=0x7f687c0b8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 shutdown_connections 2024-04-25T10:02:25.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.716+0000 7f687be00640 1 -- 172.21.15.165:0/859066146 wait complete. 2024-04-25T10:02:25.740 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:25.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.935+0000 7fc3db200640 1 Processor -- start 2024-04-25T10:02:25.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.935+0000 7fc3db200640 1 -- start start 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3db200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3d4144440 0x7fc3d4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d40cda30 0x7fc3d415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3db200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3d415bf30 con 0x7fc3d4071530 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3db200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3d40719a0 con 0x7fc3d4144440 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d415c860 con 0x7fc3d40cda30 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3d9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3d4144440 0x7fc3d4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3d9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3d4144440 0x7fc3d4070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33556/0 (socket says 172.21.15.165:33556) 2024-04-25T10:02:25.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3d9e00640 1 -- 172.21.15.165:0/3593007675 learned_addr learned my addr 172.21.15.165:0/3593007675 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.937+0000 7fc3da800640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d40cda30 0x7fc3d415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.938+0000 7fc3d9e00640 1 -- 172.21.15.165:0/3593007675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d40cda30 msgr2=0x7fc3d415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.938+0000 7fc3d9e00640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d40cda30 0x7fc3d415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.938+0000 7fc3d9e00640 1 -- 172.21.15.165:0/3593007675 >> v1:172.21.15.165:6789/0 conn(0x7fc3d4071530 legacy=0x7fc3d40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.938+0000 7fc3d9e00640 1 -- 172.21.15.165:0/3593007675 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d415c210 con 0x7fc3d4144440 2024-04-25T10:02:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.938+0000 7fc3da800640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d40cda30 0x7fc3d415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3d9e00640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d4144440 0x7fc3d4070ff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8ce5326605d43b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3d8a00640 1 -- 172.21.15.165:0/3593007675 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8037070 con 0x7fc3d4144440 2024-04-25T10:02:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3d8a00640 1 -- 172.21.15.165:0/3593007675 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c802ae50 con 0x7fc3d4144440 2024-04-25T10:02:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3d8a00640 1 -- 172.21.15.165:0/3593007675 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c8032860 con 0x7fc3d4144440 2024-04-25T10:02:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3db200640 1 -- 172.21.15.165:0/3593007675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d4144440 msgr2=0x7fc3d4070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3db200640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d4144440 0x7fc3d4070ff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8009870 tx=0x7fc3c802a300 comp rx=0 tx=0).stop 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3db200640 1 -- 172.21.15.165:0/3593007675 shutdown_connections 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3db200640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d40cda30 0x7fc3d415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.939+0000 7fc3db200640 1 --2- 172.21.15.165:0/3593007675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d4144440 0x7fc3d4070ff0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.940+0000 7fc3db200640 1 -- 172.21.15.165:0/3593007675 >> 172.21.15.165:0/3593007675 conn(0x7fc3d406abe0 msgr2=0x7fc3d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.940+0000 7fc3db200640 1 -- 172.21.15.165:0/3593007675 shutdown_connections 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.940+0000 7fc3db200640 1 -- 172.21.15.165:0/3593007675 wait complete. 2024-04-25T10:02:25.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.941+0000 7fc3db200640 1 Processor -- start 2024-04-25T10:02:25.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.941+0000 7fc3db200640 1 -- start start 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.942+0000 7fc3db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d4071530 0x7fc3d40a8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.942+0000 7fc3db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 0x7fc3d41e6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.942+0000 7fc3db200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d4144440 0x7fc3d40cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.942+0000 7fc3db200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d415bc30 con 0x7fc3d40cda30 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.942+0000 7fc3db200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d40cdea0 con 0x7fc3d4144440 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.942+0000 7fc3db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d415d230 con 0x7fc3d4071530 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 0x7fc3d41e6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 0x7fc3d41e6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33560/0 (socket says 172.21.15.165:33560) 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 -- 172.21.15.165:0/4069870088 learned_addr learned my addr 172.21.15.165:0/4069870088 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:25.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3da800640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d4144440 0x7fc3d40cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 -- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d4071530 msgr2=0x7fc3d40a8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d4071530 0x7fc3d40a8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 -- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d4144440 msgr2=0x7fc3d40cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d4144440 0x7fc3d40cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 -- 172.21.15.165:0/4069870088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d40a97c0 con 0x7fc3d40cda30 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3d9400640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 0x7fc3d41e6af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400c400 tx=0x7fc3c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.943+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c40052d0 con 0x7fc3d40cda30 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.944+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c4005490 con 0x7fc3d40cda30 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.944+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4007260 con 0x7fc3d40cda30 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.944+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3d40a9d50 con 0x7fc3d40cda30 2024-04-25T10:02:25.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.944+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3d40aa310 con 0x7fc3d40cda30 2024-04-25T10:02:25.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.945+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3d4073990 con 0x7fc3d40cda30 2024-04-25T10:02:25.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.946+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3c4007400 con 0x7fc3d40cda30 2024-04-25T10:02:25.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.947+0000 7fc3ce000640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3b806b4d0 0x7fc3b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:25.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.947+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc3c4089b30 con 0x7fc3d40cda30 2024-04-25T10:02:25.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.947+0000 7fc3d9e00640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3b806b4d0 0x7fc3b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:25.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.948+0000 7fc3d9e00640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3b806b4d0 0x7fc3b806d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8034000 tx=0x7fc3c802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:25.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:25.952+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3c40585d0 con 0x7fc3d40cda30 2024-04-25T10:02:26.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.149+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3d40a9b80 con 0x7fc3d40cda30 2024-04-25T10:02:26.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.151+0000 7fc3ce000640 1 -- 172.21.15.165:0/4069870088 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fc3c4057ed0 con 0x7fc3d40cda30 2024-04-25T10:02:26.157 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:26.157 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:26.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3b806b4d0 msgr2=0x7fc3b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3b806b4d0 0x7fc3b806d990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc3c8034000 tx=0x7fc3c802c000 comp rx=0 tx=0).stop 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 msgr2=0x7fc3d41e6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 0x7fc3d41e6af0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400c400 tx=0x7fc3c400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 shutdown_connections 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3b806b4d0 0x7fc3b806d990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d4144440 0x7fc3d40cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d40cda30 0x7fc3d41e6af0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 --2- 172.21.15.165:0/4069870088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d4071530 0x7fc3d40a8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 >> 172.21.15.165:0/4069870088 conn(0x7fc3d406abe0 msgr2=0x7fc3d41e7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 shutdown_connections 2024-04-25T10:02:26.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.154+0000 7fc3db200640 1 -- 172.21.15.165:0/4069870088 wait complete. 2024-04-25T10:02:26.179 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:02:26.179 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:26.370 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:26.377 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.373+0000 7fd6e6e00640 1 Processor -- start 2024-04-25T10:02:26.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.373+0000 7fd6e6e00640 1 -- start start 2024-04-25T10:02:26.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6e6e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6e006bbf0 0x7fd6e006c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6e6e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00b3760 con 0x7fd6e00ad5a0 2024-04-25T10:02:26.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6e6e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00b3a40 con 0x7fd6e015da80 2024-04-25T10:02:26.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6e6e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00b3520 con 0x7fd6e006bbf0 2024-04-25T10:02:26.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6df400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd6e00ad5a0 0x7fd6e006da10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45866/0 (socket says 172.21.15.165:45866) 2024-04-25T10:02:26.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6df400640 1 -- 172.21.15.165:0/2134205289 learned_addr learned my addr 172.21.15.165:0/2134205289 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:26.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.375+0000 7fd6e4e00640 1 --2- 172.21.15.165:0/2134205289 >> v2:172.21.15.172:3300/0 conn(0x7fd6e006bbf0 0x7fd6e006c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.376+0000 7fd6dea00640 1 -- 172.21.15.165:0/2134205289 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1810784903 0 0) 0x7fd6e00b3760 con 0x7fd6e00ad5a0 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.376+0000 7fd6dea00640 1 -- 172.21.15.165:0/2134205289 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6c4003520 con 0x7fd6e00ad5a0 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.376+0000 7fd6dea00640 1 -- 172.21.15.165:0/2134205289 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3184772764 0 0) 0x7fd6c4003520 con 0x7fd6e00ad5a0 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.376+0000 7fd6e4e00640 1 -- 172.21.15.165:0/2134205289 >> v1:172.21.15.172:6789/0 conn(0x7fd6e015da80 legacy=0x7fd6e00b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.376+0000 7fd6e4e00640 1 -- 172.21.15.165:0/2134205289 >> v1:172.21.15.165:6789/0 conn(0x7fd6e00ad5a0 legacy=0x7fd6e006da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.376+0000 7fd6e4e00640 1 -- 172.21.15.165:0/2134205289 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00b3d20 con 0x7fd6e006bbf0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.377+0000 7fd6e4e00640 1 --2- 172.21.15.165:0/2134205289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e006bbf0 0x7fd6e006c0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d802a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94aed92c73aef06b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.378+0000 7fd6dea00640 1 -- 172.21.15.165:0/2134205289 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc001670 con 0x7fd6e006bbf0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.378+0000 7fd6dea00640 1 -- 172.21.15.165:0/2134205289 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d8007540 con 0x7fd6e006bbf0 2024-04-25T10:02:26.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.378+0000 7fd6dea00640 1 -- 172.21.15.165:0/2134205289 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8007910 con 0x7fd6e006bbf0 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 -- 172.21.15.165:0/2134205289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e006bbf0 msgr2=0x7fd6e006c0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/2134205289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e006bbf0 0x7fd6e006c0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d802a710 comp rx=0 tx=0).stop 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 -- 172.21.15.165:0/2134205289 shutdown_connections 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/2134205289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e006bbf0 0x7fd6e006c0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 -- 172.21.15.165:0/2134205289 >> 172.21.15.165:0/2134205289 conn(0x7fd6e0071b00 msgr2=0x7fd6e0071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 -- 172.21.15.165:0/2134205289 shutdown_connections 2024-04-25T10:02:26.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.379+0000 7fd6e6e00640 1 -- 172.21.15.165:0/2134205289 wait complete. 2024-04-25T10:02:26.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.380+0000 7fd6e6e00640 1 Processor -- start 2024-04-25T10:02:26.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.381+0000 7fd6e6e00640 1 -- start start 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.381+0000 7fd6e6e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6e006bbf0 0x7fd6e0147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.381+0000 7fd6e6e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e00ad5a0 0x7fd6e01540e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e6e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 0x7fd6e0154620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e6e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e015de60 con 0x7fd6e015da80 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e6e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00b4370 con 0x7fd6e00ad5a0 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e6e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e006e120 con 0x7fd6e006bbf0 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 0x7fd6e0154620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 0x7fd6e0154620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33572/0 (socket says 172.21.15.165:33572) 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 -- 172.21.15.165:0/1628593695 learned_addr learned my addr 172.21.15.165:0/1628593695 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6df400640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e00ad5a0 0x7fd6e01540e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6dfe00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6e006bbf0 0x7fd6e0147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 -- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6e006bbf0 msgr2=0x7fd6e0147af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6e006bbf0 0x7fd6e0147af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 -- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e00ad5a0 msgr2=0x7fd6e01540e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e00ad5a0 0x7fd6e01540e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6e4e00640 1 -- 172.21.15.165:0/1628593695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00cc770 con 0x7fd6e015da80 2024-04-25T10:02:26.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.382+0000 7fd6df400640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e00ad5a0 0x7fd6e01540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.383+0000 7fd6e4e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 0x7fd6e0154620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.383+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8040070 con 0x7fd6e015da80 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.383+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d8031bc0 con 0x7fd6e015da80 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.383+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8009450 con 0x7fd6e015da80 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.383+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e0154e50 con 0x7fd6e015da80 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.383+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e0155410 con 0x7fd6e015da80 2024-04-25T10:02:26.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.385+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0068500 con 0x7fd6e015da80 2024-04-25T10:02:26.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.385+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6d8039040 con 0x7fd6e015da80 2024-04-25T10:02:26.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.386+0000 7fd6d7e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6c006b500 0x7fd6c006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.386+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd6d80ab970 con 0x7fd6e015da80 2024-04-25T10:02:26.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.387+0000 7fd6dfe00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6c006b500 0x7fd6c006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.387+0000 7fd6dfe00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6c006b500 0x7fd6c006d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0000f60 tx=0x7fd6d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.391+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6e0068500 con 0x7fd6e015da80 2024-04-25T10:02:26.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.586+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6e0073990 con 0x7fd6c006b500 2024-04-25T10:02:26.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.616+0000 7fd6d7e00640 1 -- 172.21.15.165:0/1628593695 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332574 (secure 0 0 0) 0x7fd6e0073990 con 0x7fd6c006b500 2024-04-25T10:02:26.624 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:26.629 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6c006b500 msgr2=0x7fd6c006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6c006b500 0x7fd6c006d9c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6d0000f60 tx=0x7fd6d0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 msgr2=0x7fd6e0154620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 0x7fd6e0154620 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009d90 tx=0x7fd6d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 shutdown_connections 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6c006b500 0x7fd6c006d9c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6e015da80 0x7fd6e0154620 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6e00ad5a0 0x7fd6e01540e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 --2- 172.21.15.165:0/1628593695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6e006bbf0 0x7fd6e0147af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 >> 172.21.15.165:0/1628593695 conn(0x7fd6e0071b00 msgr2=0x7fd6e00ad9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.625+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 shutdown_connections 2024-04-25T10:02:26.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.626+0000 7fd6e6e00640 1 -- 172.21.15.165:0/1628593695 wait complete. 2024-04-25T10:02:26.651 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T10:02:26.130500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219420,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755300,"statfs":{"total":773094113280,"available":772869427200,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004471"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513776+0000","last_change":"2024-04-25T10:02:03.142252+0000","last_active":"2024-04-25T10:02:09.513776+0000","last_peered":"2024-04-25T10:02:09.513776+0000","last_clean":"2024-04-25T10:02:09.513776+0000","last_became_active":"2024-04-25T10:02:03.141984+0000","last_became_peered":"2024-04-25T10:02:03.141984+0000","last_unstale":"2024-04-25T10:02:09.513776+0000","last_undegraded":"2024-04-25T10:02:09.513776+0000","last_fullsized":"2024-04-25T10:02:09.513776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520621+0000","last_change":"2024-04-25T10:02:04.156786+0000","last_active":"2024-04-25T10:02:09.520621+0000","last_peered":"2024-04-25T10:02:09.520621+0000","last_clean":"2024-04-25T10:02:09.520621+0000","last_became_active":"2024-04-25T10:02:04.156522+0000","last_became_peered":"2024-04-25T10:02:04.156522+0000","last_unstale":"2024-04-25T10:02:09.520621+0000","last_undegraded":"2024-04-25T10:02:09.520621+0000","last_fullsized":"2024-04-25T10:02:09.520621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519839+0000","last_change":"2024-04-25T10:02:08.409422+0000","last_active":"2024-04-25T10:02:09.519839+0000","last_peered":"2024-04-25T10:02:09.519839+0000","last_clean":"2024-04-25T10:02:09.519839+0000","last_became_active":"2024-04-25T10:02:08.408455+0000","last_became_peered":"2024-04-25T10:02:08.408455+0000","last_unstale":"2024-04-25T10:02:09.519839+0000","last_undegraded":"2024-04-25T10:02:09.519839+0000","last_fullsized":"2024-04-25T10:02:09.519839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:32.901672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520971+0000","last_change":"2024-04-25T10:02:03.131043+0000","last_active":"2024-04-25T10:02:09.520971+0000","last_peered":"2024-04-25T10:02:09.520971+0000","last_clean":"2024-04-25T10:02:09.520971+0000","last_became_active":"2024-04-25T10:02:03.130682+0000","last_became_peered":"2024-04-25T10:02:03.130682+0000","last_unstale":"2024-04-25T10:02:09.520971+0000","last_undegraded":"2024-04-25T10:02:09.520971+0000","last_fullsized":"2024-04-25T10:02:09.520971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513286+0000","last_change":"2024-04-25T10:02:04.142528+0000","last_active":"2024-04-25T10:02:09.513286+0000","last_peered":"2024-04-25T10:02:09.513286+0000","last_clean":"2024-04-25T10:02:09.513286+0000","last_became_active":"2024-04-25T10:02:04.142188+0000","last_became_peered":"2024-04-25T10:02:04.142188+0000","last_unstale":"2024-04-25T10:02:09.513286+0000","last_undegraded":"2024-04-25T10:02:09.513286+0000","last_fullsized":"2024-04-25T10:02:09.513286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183112+0000","last_change":"2024-04-25T10:02:08.192140+0000","last_active":"2024-04-25T10:02:09.183112+0000","last_peered":"2024-04-25T10:02:09.183112+0000","last_clean":"2024-04-25T10:02:09.183112+0000","last_became_active":"2024-04-25T10:02:08.191658+0000","last_became_peered":"2024-04-25T10:02:08.191658+0000","last_unstale":"2024-04-25T10:02:09.183112+0000","last_undegraded":"2024-04-25T10:02:09.183112+0000","last_fullsized":"2024-04-25T10:02:09.183112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:45.834272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514852+0000","last_change":"2024-04-25T10:02:03.147520+0000","last_active":"2024-04-25T10:02:09.514852+0000","last_peered":"2024-04-25T10:02:09.514852+0000","last_clean":"2024-04-25T10:02:09.514852+0000","last_became_active":"2024-04-25T10:02:03.147274+0000","last_became_peered":"2024-04-25T10:02:03.147274+0000","last_unstale":"2024-04-25T10:02:09.514852+0000","last_undegraded":"2024-04-25T10:02:09.514852+0000","last_fullsized":"2024-04-25T10:02:09.514852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516102+0000","last_change":"2024-04-25T10:02:04.137351+0000","last_active":"2024-04-25T10:02:09.516102+0000","last_peered":"2024-04-25T10:02:09.516102+0000","last_clean":"2024-04-25T10:02:09.516102+0000","last_became_active":"2024-04-25T10:02:04.136927+0000","last_became_peered":"2024-04-25T10:02:04.136927+0000","last_unstale":"2024-04-25T10:02:09.516102+0000","last_undegraded":"2024-04-25T10:02:09.516102+0000","last_fullsized":"2024-04-25T10:02:09.516102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185918+0000","last_change":"2024-04-25T10:02:08.198068+0000","last_active":"2024-04-25T10:02:09.185918+0000","last_peered":"2024-04-25T10:02:09.185918+0000","last_clean":"2024-04-25T10:02:09.185918+0000","last_became_active":"2024-04-25T10:02:08.196209+0000","last_became_peered":"2024-04-25T10:02:08.196209+0000","last_unstale":"2024-04-25T10:02:09.185918+0000","last_undegraded":"2024-04-25T10:02:09.185918+0000","last_fullsized":"2024-04-25T10:02:09.185918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:48.568415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514318+0000","last_change":"2024-04-25T10:02:03.157531+0000","last_active":"2024-04-25T10:02:09.514318+0000","last_peered":"2024-04-25T10:02:09.514318+0000","last_clean":"2024-04-25T10:02:09.514318+0000","last_became_active":"2024-04-25T10:02:03.157216+0000","last_became_peered":"2024-04-25T10:02:03.157216+0000","last_unstale":"2024-04-25T10:02:09.514318+0000","last_undegraded":"2024-04-25T10:02:09.514318+0000","last_fullsized":"2024-04-25T10:02:09.514318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516612+0000","last_change":"2024-04-25T10:02:04.140082+0000","last_active":"2024-04-25T10:02:09.516612+0000","last_peered":"2024-04-25T10:02:09.516612+0000","last_clean":"2024-04-25T10:02:09.516612+0000","last_became_active":"2024-04-25T10:02:04.136765+0000","last_became_peered":"2024-04-25T10:02:04.136765+0000","last_unstale":"2024-04-25T10:02:09.516612+0000","last_undegraded":"2024-04-25T10:02:09.516612+0000","last_fullsized":"2024-04-25T10:02:09.516612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185629+0000","last_change":"2024-04-25T10:02:08.189048+0000","last_active":"2024-04-25T10:02:09.185629+0000","last_peered":"2024-04-25T10:02:09.185629+0000","last_clean":"2024-04-25T10:02:09.185629+0000","last_became_active":"2024-04-25T10:02:08.188475+0000","last_became_peered":"2024-04-25T10:02:08.188475+0000","last_unstale":"2024-04-25T10:02:09.185629+0000","last_undegraded":"2024-04-25T10:02:09.185629+0000","last_fullsized":"2024-04-25T10:02:09.185629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:08.343719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182619+0000","last_change":"2024-04-25T10:02:04.132020+0000","last_active":"2024-04-25T10:02:09.182619+0000","last_peered":"2024-04-25T10:02:09.182619+0000","last_clean":"2024-04-25T10:02:09.182619+0000","last_became_active":"2024-04-25T10:02:04.131018+0000","last_became_peered":"2024-04-25T10:02:04.131018+0000","last_unstale":"2024-04-25T10:02:09.182619+0000","last_undegraded":"2024-04-25T10:02:09.182619+0000","last_fullsized":"2024-04-25T10:02:09.182619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514507+0000","last_change":"2024-04-25T10:02:03.147595+0000","last_active":"2024-04-25T10:02:09.514507+0000","last_peered":"2024-04-25T10:02:09.514507+0000","last_clean":"2024-04-25T10:02:09.514507+0000","last_became_active":"2024-04-25T10:02:03.147038+0000","last_became_peered":"2024-04-25T10:02:03.147038+0000","last_unstale":"2024-04-25T10:02:09.514507+0000","last_undegraded":"2024-04-25T10:02:09.514507+0000","last_fullsized":"2024-04-25T10:02:09.514507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:50.278310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188671+0000","last_change":"2024-04-25T10:02:08.186691+0000","last_active":"2024-04-25T10:02:09.188671+0000","last_peered":"2024-04-25T10:02:09.188671+0000","last_clean":"2024-04-25T10:02:09.188671+0000","last_became_active":"2024-04-25T10:02:08.186276+0000","last_became_peered":"2024-04-25T10:02:08.186276+0000","last_unstale":"2024-04-25T10:02:09.188671+0000","last_undegraded":"2024-04-25T10:02:09.188671+0000","last_fullsized":"2024-04-25T10:02:09.188671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520343+0000","last_change":"2024-04-25T10:02:04.139116+0000","last_active":"2024-04-25T10:02:09.520343+0000","last_peered":"2024-04-25T10:02:09.520343+0000","last_clean":"2024-04-25T10:02:09.520343+0000","last_became_active":"2024-04-25T10:02:04.138565+0000","last_became_peered":"2024-04-25T10:02:04.138565+0000","last_unstale":"2024-04-25T10:02:09.520343+0000","last_undegraded":"2024-04-25T10:02:09.520343+0000","last_fullsized":"2024-04-25T10:02:09.520343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518432+0000","last_change":"2024-04-25T10:02:03.132968+0000","last_active":"2024-04-25T10:02:09.518432+0000","last_peered":"2024-04-25T10:02:09.518432+0000","last_clean":"2024-04-25T10:02:09.518432+0000","last_became_active":"2024-04-25T10:02:03.132624+0000","last_became_peered":"2024-04-25T10:02:03.132624+0000","last_unstale":"2024-04-25T10:02:09.518432+0000","last_undegraded":"2024-04-25T10:02:09.518432+0000","last_fullsized":"2024-04-25T10:02:09.518432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187996+0000","last_change":"2024-04-25T10:02:08.193709+0000","last_active":"2024-04-25T10:02:09.187996+0000","last_peered":"2024-04-25T10:02:09.187996+0000","last_clean":"2024-04-25T10:02:09.187996+0000","last_became_active":"2024-04-25T10:02:08.193240+0000","last_became_peered":"2024-04-25T10:02:08.193240+0000","last_unstale":"2024-04-25T10:02:09.187996+0000","last_undegraded":"2024-04-25T10:02:09.187996+0000","last_fullsized":"2024-04-25T10:02:09.187996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519675+0000","last_change":"2024-04-25T10:02:04.139077+0000","last_active":"2024-04-25T10:02:09.519675+0000","last_peered":"2024-04-25T10:02:09.519675+0000","last_clean":"2024-04-25T10:02:09.519675+0000","last_became_active":"2024-04-25T10:02:04.138730+0000","last_became_peered":"2024-04-25T10:02:04.138730+0000","last_unstale":"2024-04-25T10:02:09.519675+0000","last_undegraded":"2024-04-25T10:02:09.519675+0000","last_fullsized":"2024-04-25T10:02:09.519675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514609+0000","last_change":"2024-04-25T10:02:03.137563+0000","last_active":"2024-04-25T10:02:09.514609+0000","last_peered":"2024-04-25T10:02:09.514609+0000","last_clean":"2024-04-25T10:02:09.514609+0000","last_became_active":"2024-04-25T10:02:03.137080+0000","last_became_peered":"2024-04-25T10:02:03.137080+0000","last_unstale":"2024-04-25T10:02:09.514609+0000","last_undegraded":"2024-04-25T10:02:09.514609+0000","last_fullsized":"2024-04-25T10:02:09.514609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517371+0000","last_change":"2024-04-25T10:02:08.404747+0000","last_active":"2024-04-25T10:02:09.517371+0000","last_peered":"2024-04-25T10:02:09.517371+0000","last_clean":"2024-04-25T10:02:09.517371+0000","last_became_active":"2024-04-25T10:02:08.403706+0000","last_became_peered":"2024-04-25T10:02:08.403706+0000","last_unstale":"2024-04-25T10:02:09.517371+0000","last_undegraded":"2024-04-25T10:02:09.517371+0000","last_fullsized":"2024-04-25T10:02:09.517371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185839+0000","last_change":"2024-04-25T10:02:04.125617+0000","last_active":"2024-04-25T10:02:09.185839+0000","last_peered":"2024-04-25T10:02:09.185839+0000","last_clean":"2024-04-25T10:02:09.185839+0000","last_became_active":"2024-04-25T10:02:04.125319+0000","last_became_peered":"2024-04-25T10:02:04.125319+0000","last_unstale":"2024-04-25T10:02:09.185839+0000","last_undegraded":"2024-04-25T10:02:09.185839+0000","last_fullsized":"2024-04-25T10:02:09.185839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.271382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512743+0000","last_change":"2024-04-25T10:02:03.145754+0000","last_active":"2024-04-25T10:02:09.512743+0000","last_peered":"2024-04-25T10:02:09.512743+0000","last_clean":"2024-04-25T10:02:09.512743+0000","last_became_active":"2024-04-25T10:02:03.145400+0000","last_became_peered":"2024-04-25T10:02:03.145400+0000","last_unstale":"2024-04-25T10:02:09.512743+0000","last_undegraded":"2024-04-25T10:02:09.512743+0000","last_fullsized":"2024-04-25T10:02:09.512743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188721+0000","last_change":"2024-04-25T10:02:08.197346+0000","last_active":"2024-04-25T10:02:09.188721+0000","last_peered":"2024-04-25T10:02:09.188721+0000","last_clean":"2024-04-25T10:02:09.188721+0000","last_became_active":"2024-04-25T10:02:08.196902+0000","last_became_peered":"2024-04-25T10:02:08.196902+0000","last_unstale":"2024-04-25T10:02:09.188721+0000","last_undegraded":"2024-04-25T10:02:09.188721+0000","last_fullsized":"2024-04-25T10:02:09.188721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:43.992278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515041+0000","last_change":"2024-04-25T10:02:04.123238+0000","last_active":"2024-04-25T10:02:09.515041+0000","last_peered":"2024-04-25T10:02:09.515041+0000","last_clean":"2024-04-25T10:02:09.515041+0000","last_became_active":"2024-04-25T10:02:04.122886+0000","last_became_peered":"2024-04-25T10:02:04.122886+0000","last_unstale":"2024-04-25T10:02:09.515041+0000","last_undegraded":"2024-04-25T10:02:09.515041+0000","last_fullsized":"2024-04-25T10:02:09.515041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187935+0000","last_change":"2024-04-25T10:02:03.138021+0000","last_active":"2024-04-25T10:02:09.187935+0000","last_peered":"2024-04-25T10:02:09.187935+0000","last_clean":"2024-04-25T10:02:09.187935+0000","last_became_active":"2024-04-25T10:02:03.137531+0000","last_became_peered":"2024-04-25T10:02:03.137531+0000","last_unstale":"2024-04-25T10:02:09.187935+0000","last_undegraded":"2024-04-25T10:02:09.187935+0000","last_fullsized":"2024-04-25T10:02:09.187935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185644+0000","last_change":"2024-04-25T10:02:08.404604+0000","last_active":"2024-04-25T10:02:09.185644+0000","last_peered":"2024-04-25T10:02:09.185644+0000","last_clean":"2024-04-25T10:02:09.185644+0000","last_became_active":"2024-04-25T10:02:08.403392+0000","last_became_peered":"2024-04-25T10:02:08.403392+0000","last_unstale":"2024-04-25T10:02:09.185644+0000","last_undegraded":"2024-04-25T10:02:09.185644+0000","last_fullsized":"2024-04-25T10:02:09.185644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:25.532310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521257+0000","last_change":"2024-04-25T10:02:04.144042+0000","last_active":"2024-04-25T10:02:09.521257+0000","last_peered":"2024-04-25T10:02:09.521257+0000","last_clean":"2024-04-25T10:02:09.521257+0000","last_became_active":"2024-04-25T10:02:04.143734+0000","last_became_peered":"2024-04-25T10:02:04.143734+0000","last_unstale":"2024-04-25T10:02:09.521257+0000","last_undegraded":"2024-04-25T10:02:09.521257+0000","last_fullsized":"2024-04-25T10:02:09.521257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516285+0000","last_change":"2024-04-25T10:02:03.138015+0000","last_active":"2024-04-25T10:02:09.516285+0000","last_peered":"2024-04-25T10:02:09.516285+0000","last_clean":"2024-04-25T10:02:09.516285+0000","last_became_active":"2024-04-25T10:02:03.137487+0000","last_became_peered":"2024-04-25T10:02:03.137487+0000","last_unstale":"2024-04-25T10:02:09.516285+0000","last_undegraded":"2024-04-25T10:02:09.516285+0000","last_fullsized":"2024-04-25T10:02:09.516285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515852+0000","last_change":"2024-04-25T10:02:08.402309+0000","last_active":"2024-04-25T10:02:09.515852+0000","last_peered":"2024-04-25T10:02:09.515852+0000","last_clean":"2024-04-25T10:02:09.515852+0000","last_became_active":"2024-04-25T10:02:08.401865+0000","last_became_peered":"2024-04-25T10:02:08.401865+0000","last_unstale":"2024-04-25T10:02:09.515852+0000","last_undegraded":"2024-04-25T10:02:09.515852+0000","last_fullsized":"2024-04-25T10:02:09.515852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519191+0000","last_change":"2024-04-25T10:02:04.129398+0000","last_active":"2024-04-25T10:02:09.519191+0000","last_peered":"2024-04-25T10:02:09.519191+0000","last_clean":"2024-04-25T10:02:09.519191+0000","last_became_active":"2024-04-25T10:02:04.128879+0000","last_became_peered":"2024-04-25T10:02:04.128879+0000","last_unstale":"2024-04-25T10:02:09.519191+0000","last_undegraded":"2024-04-25T10:02:09.519191+0000","last_fullsized":"2024-04-25T10:02:09.519191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515418+0000","last_change":"2024-04-25T10:02:03.146769+0000","last_active":"2024-04-25T10:02:09.515418+0000","last_peered":"2024-04-25T10:02:09.515418+0000","last_clean":"2024-04-25T10:02:09.515418+0000","last_became_active":"2024-04-25T10:02:03.146576+0000","last_became_peered":"2024-04-25T10:02:03.146576+0000","last_unstale":"2024-04-25T10:02:09.515418+0000","last_undegraded":"2024-04-25T10:02:09.515418+0000","last_fullsized":"2024-04-25T10:02:09.515418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513472+0000","last_change":"2024-04-25T10:02:08.393230+0000","last_active":"2024-04-25T10:02:09.513472+0000","last_peered":"2024-04-25T10:02:09.513472+0000","last_clean":"2024-04-25T10:02:09.513472+0000","last_became_active":"2024-04-25T10:02:08.392691+0000","last_became_peered":"2024-04-25T10:02:08.392691+0000","last_unstale":"2024-04-25T10:02:09.513472+0000","last_undegraded":"2024-04-25T10:02:09.513472+0000","last_fullsized":"2024-04-25T10:02:09.513472+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516864+0000","last_change":"2024-04-25T10:02:04.140436+0000","last_active":"2024-04-25T10:02:09.516864+0000","last_peered":"2024-04-25T10:02:09.516864+0000","last_clean":"2024-04-25T10:02:09.516864+0000","last_became_active":"2024-04-25T10:02:04.140075+0000","last_became_peered":"2024-04-25T10:02:04.140075+0000","last_unstale":"2024-04-25T10:02:09.516864+0000","last_undegraded":"2024-04-25T10:02:09.516864+0000","last_fullsized":"2024-04-25T10:02:09.516864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519440+0000","last_change":"2024-04-25T10:02:03.132553+0000","last_active":"2024-04-25T10:02:09.519440+0000","last_peered":"2024-04-25T10:02:09.519440+0000","last_clean":"2024-04-25T10:02:09.519440+0000","last_became_active":"2024-04-25T10:02:03.132241+0000","last_became_peered":"2024-04-25T10:02:03.132241+0000","last_unstale":"2024-04-25T10:02:09.519440+0000","last_undegraded":"2024-04-25T10:02:09.519440+0000","last_fullsized":"2024-04-25T10:02:09.519440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518024+0000","last_change":"2024-04-25T10:02:08.186096+0000","last_active":"2024-04-25T10:02:09.518024+0000","last_peered":"2024-04-25T10:02:09.518024+0000","last_clean":"2024-04-25T10:02:09.518024+0000","last_became_active":"2024-04-25T10:02:08.185660+0000","last_became_peered":"2024-04-25T10:02:08.185660+0000","last_unstale":"2024-04-25T10:02:09.518024+0000","last_undegraded":"2024-04-25T10:02:09.518024+0000","last_fullsized":"2024-04-25T10:02:09.518024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188203+0000","last_change":"2024-04-25T10:02:04.126935+0000","last_active":"2024-04-25T10:02:09.188203+0000","last_peered":"2024-04-25T10:02:09.188203+0000","last_clean":"2024-04-25T10:02:09.188203+0000","last_became_active":"2024-04-25T10:02:04.126268+0000","last_became_peered":"2024-04-25T10:02:04.126268+0000","last_unstale":"2024-04-25T10:02:09.188203+0000","last_undegraded":"2024-04-25T10:02:09.188203+0000","last_fullsized":"2024-04-25T10:02:09.188203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515075+0000","last_change":"2024-04-25T10:02:03.143964+0000","last_active":"2024-04-25T10:02:09.515075+0000","last_peered":"2024-04-25T10:02:09.515075+0000","last_clean":"2024-04-25T10:02:09.515075+0000","last_became_active":"2024-04-25T10:02:03.143548+0000","last_became_peered":"2024-04-25T10:02:03.143548+0000","last_unstale":"2024-04-25T10:02:09.515075+0000","last_undegraded":"2024-04-25T10:02:09.515075+0000","last_fullsized":"2024-04-25T10:02:09.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519212+0000","last_change":"2024-04-25T10:02:08.196544+0000","last_active":"2024-04-25T10:02:09.519212+0000","last_peered":"2024-04-25T10:02:09.519212+0000","last_clean":"2024-04-25T10:02:09.519212+0000","last_became_active":"2024-04-25T10:02:08.196131+0000","last_became_peered":"2024-04-25T10:02:08.196131+0000","last_unstale":"2024-04-25T10:02:09.519212+0000","last_undegraded":"2024-04-25T10:02:09.519212+0000","last_fullsized":"2024-04-25T10:02:09.519212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513146+0000","last_change":"2024-04-25T10:02:04.124338+0000","last_active":"2024-04-25T10:02:09.513146+0000","last_peered":"2024-04-25T10:02:09.513146+0000","last_clean":"2024-04-25T10:02:09.513146+0000","last_became_active":"2024-04-25T10:02:04.123720+0000","last_became_peered":"2024-04-25T10:02:04.123720+0000","last_unstale":"2024-04-25T10:02:09.513146+0000","last_undegraded":"2024-04-25T10:02:09.513146+0000","last_fullsized":"2024-04-25T10:02:09.513146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181759+0000","last_change":"2024-04-25T10:02:03.131111+0000","last_active":"2024-04-25T10:02:09.181759+0000","last_peered":"2024-04-25T10:02:09.181759+0000","last_clean":"2024-04-25T10:02:09.181759+0000","last_became_active":"2024-04-25T10:02:03.130828+0000","last_became_peered":"2024-04-25T10:02:03.130828+0000","last_unstale":"2024-04-25T10:02:09.181759+0000","last_undegraded":"2024-04-25T10:02:09.181759+0000","last_fullsized":"2024-04-25T10:02:09.181759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517914+0000","last_change":"2024-04-25T10:02:08.193216+0000","last_active":"2024-04-25T10:02:09.517914+0000","last_peered":"2024-04-25T10:02:09.517914+0000","last_clean":"2024-04-25T10:02:09.517914+0000","last_became_active":"2024-04-25T10:02:08.192717+0000","last_became_peered":"2024-04-25T10:02:08.192717+0000","last_unstale":"2024-04-25T10:02:09.517914+0000","last_undegraded":"2024-04-25T10:02:09.517914+0000","last_fullsized":"2024-04-25T10:02:09.517914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513507+0000","last_change":"2024-04-25T10:02:04.131883+0000","last_active":"2024-04-25T10:02:09.513507+0000","last_peered":"2024-04-25T10:02:09.513507+0000","last_clean":"2024-04-25T10:02:09.513507+0000","last_became_active":"2024-04-25T10:02:04.131591+0000","last_became_peered":"2024-04-25T10:02:04.131591+0000","last_unstale":"2024-04-25T10:02:09.513507+0000","last_undegraded":"2024-04-25T10:02:09.513507+0000","last_fullsized":"2024-04-25T10:02:09.513507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513031+0000","last_change":"2024-04-25T10:02:03.146201+0000","last_active":"2024-04-25T10:02:09.513031+0000","last_peered":"2024-04-25T10:02:09.513031+0000","last_clean":"2024-04-25T10:02:09.513031+0000","last_became_active":"2024-04-25T10:02:03.145888+0000","last_became_peered":"2024-04-25T10:02:03.145888+0000","last_unstale":"2024-04-25T10:02:09.513031+0000","last_undegraded":"2024-04-25T10:02:09.513031+0000","last_fullsized":"2024-04-25T10:02:09.513031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514736+0000","last_change":"2024-04-25T10:02:08.403944+0000","last_active":"2024-04-25T10:02:09.514736+0000","last_peered":"2024-04-25T10:02:09.514736+0000","last_clean":"2024-04-25T10:02:09.514736+0000","last_became_active":"2024-04-25T10:02:08.403447+0000","last_became_peered":"2024-04-25T10:02:08.403447+0000","last_unstale":"2024-04-25T10:02:09.514736+0000","last_undegraded":"2024-04-25T10:02:09.514736+0000","last_fullsized":"2024-04-25T10:02:09.514736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520189+0000","last_change":"2024-04-25T10:02:04.145181+0000","last_active":"2024-04-25T10:02:09.520189+0000","last_peered":"2024-04-25T10:02:09.520189+0000","last_clean":"2024-04-25T10:02:09.520189+0000","last_became_active":"2024-04-25T10:02:04.145007+0000","last_became_peered":"2024-04-25T10:02:04.145007+0000","last_unstale":"2024-04-25T10:02:09.520189+0000","last_undegraded":"2024-04-25T10:02:09.520189+0000","last_fullsized":"2024-04-25T10:02:09.520189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515574+0000","last_change":"2024-04-25T10:02:03.149716+0000","last_active":"2024-04-25T10:02:09.515574+0000","last_peered":"2024-04-25T10:02:09.515574+0000","last_clean":"2024-04-25T10:02:09.515574+0000","last_became_active":"2024-04-25T10:02:03.149427+0000","last_became_peered":"2024-04-25T10:02:03.149427+0000","last_unstale":"2024-04-25T10:02:09.515574+0000","last_undegraded":"2024-04-25T10:02:09.515574+0000","last_fullsized":"2024-04-25T10:02:09.515574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513340+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:02:09.513340+0000","last_peered":"2024-04-25T10:02:09.513340+0000","last_clean":"2024-04-25T10:02:09.513340+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:02:09.513340+0000","last_undegraded":"2024-04-25T10:02:09.513340+0000","last_fullsized":"2024-04-25T10:02:09.513340+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183489+0000","last_change":"2024-04-25T10:02:08.405095+0000","last_active":"2024-04-25T10:02:09.183489+0000","last_peered":"2024-04-25T10:02:09.183489+0000","last_clean":"2024-04-25T10:02:09.183489+0000","last_became_active":"2024-04-25T10:02:08.403767+0000","last_became_peered":"2024-04-25T10:02:08.403767+0000","last_unstale":"2024-04-25T10:02:09.183489+0000","last_undegraded":"2024-04-25T10:02:09.183489+0000","last_fullsized":"2024-04-25T10:02:09.183489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514594+0000","last_change":"2024-04-25T10:02:04.151648+0000","last_active":"2024-04-25T10:02:09.514594+0000","last_peered":"2024-04-25T10:02:09.514594+0000","last_clean":"2024-04-25T10:02:09.514594+0000","last_became_active":"2024-04-25T10:02:04.150680+0000","last_became_peered":"2024-04-25T10:02:04.150680+0000","last_unstale":"2024-04-25T10:02:09.514594+0000","last_undegraded":"2024-04-25T10:02:09.514594+0000","last_fullsized":"2024-04-25T10:02:09.514594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182505+0000","last_change":"2024-04-25T10:02:03.154627+0000","last_active":"2024-04-25T10:02:09.182505+0000","last_peered":"2024-04-25T10:02:09.182505+0000","last_clean":"2024-04-25T10:02:09.182505+0000","last_became_active":"2024-04-25T10:02:03.154346+0000","last_became_peered":"2024-04-25T10:02:03.154346+0000","last_unstale":"2024-04-25T10:02:09.182505+0000","last_undegraded":"2024-04-25T10:02:09.182505+0000","last_fullsized":"2024-04-25T10:02:09.182505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184009+0000","last_change":"2024-04-25T10:02:08.394382+0000","last_active":"2024-04-25T10:02:09.184009+0000","last_peered":"2024-04-25T10:02:09.184009+0000","last_clean":"2024-04-25T10:02:09.184009+0000","last_became_active":"2024-04-25T10:02:08.393222+0000","last_became_peered":"2024-04-25T10:02:08.393222+0000","last_unstale":"2024-04-25T10:02:09.184009+0000","last_undegraded":"2024-04-25T10:02:09.184009+0000","last_fullsized":"2024-04-25T10:02:09.184009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516304+0000","last_change":"2024-04-25T10:02:04.144851+0000","last_active":"2024-04-25T10:02:09.516304+0000","last_peered":"2024-04-25T10:02:09.516304+0000","last_clean":"2024-04-25T10:02:09.516304+0000","last_became_active":"2024-04-25T10:02:04.144530+0000","last_became_peered":"2024-04-25T10:02:04.144530+0000","last_unstale":"2024-04-25T10:02:09.516304+0000","last_undegraded":"2024-04-25T10:02:09.516304+0000","last_fullsized":"2024-04-25T10:02:09.516304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515591+0000","last_change":"2024-04-25T10:02:03.133061+0000","last_active":"2024-04-25T10:02:09.515591+0000","last_peered":"2024-04-25T10:02:09.515591+0000","last_clean":"2024-04-25T10:02:09.515591+0000","last_became_active":"2024-04-25T10:02:03.132762+0000","last_became_peered":"2024-04-25T10:02:03.132762+0000","last_unstale":"2024-04-25T10:02:09.515591+0000","last_undegraded":"2024-04-25T10:02:09.515591+0000","last_fullsized":"2024-04-25T10:02:09.515591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513927+0000","last_change":"2024-04-25T10:02:08.186615+0000","last_active":"2024-04-25T10:02:09.513927+0000","last_peered":"2024-04-25T10:02:09.513927+0000","last_clean":"2024-04-25T10:02:09.513927+0000","last_became_active":"2024-04-25T10:02:08.186216+0000","last_became_peered":"2024-04-25T10:02:08.186216+0000","last_unstale":"2024-04-25T10:02:09.513927+0000","last_undegraded":"2024-04-25T10:02:09.513927+0000","last_fullsized":"2024-04-25T10:02:09.513927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515930+0000","last_change":"2024-04-25T10:02:04.153131+0000","last_active":"2024-04-25T10:02:09.515930+0000","last_peered":"2024-04-25T10:02:09.515930+0000","last_clean":"2024-04-25T10:02:09.515930+0000","last_became_active":"2024-04-25T10:02:04.152415+0000","last_became_peered":"2024-04-25T10:02:04.152415+0000","last_unstale":"2024-04-25T10:02:09.515930+0000","last_undegraded":"2024-04-25T10:02:09.515930+0000","last_fullsized":"2024-04-25T10:02:09.515930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:54.646358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182568+0000","last_change":"2024-04-25T10:02:03.134495+0000","last_active":"2024-04-25T10:02:09.182568+0000","last_peered":"2024-04-25T10:02:09.182568+0000","last_clean":"2024-04-25T10:02:09.182568+0000","last_became_active":"2024-04-25T10:02:03.134157+0000","last_became_peered":"2024-04-25T10:02:03.134157+0000","last_unstale":"2024-04-25T10:02:09.182568+0000","last_undegraded":"2024-04-25T10:02:09.182568+0000","last_fullsized":"2024-04-25T10:02:09.182568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512930+0000","last_change":"2024-04-25T10:02:08.406412+0000","last_active":"2024-04-25T10:02:09.512930+0000","last_peered":"2024-04-25T10:02:09.512930+0000","last_clean":"2024-04-25T10:02:09.512930+0000","last_became_active":"2024-04-25T10:02:08.402979+0000","last_became_peered":"2024-04-25T10:02:08.402979+0000","last_unstale":"2024-04-25T10:02:09.512930+0000","last_undegraded":"2024-04-25T10:02:09.512930+0000","last_fullsized":"2024-04-25T10:02:09.512930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515530+0000","last_change":"2024-04-25T10:02:04.150503+0000","last_active":"2024-04-25T10:02:09.515530+0000","last_peered":"2024-04-25T10:02:09.515530+0000","last_clean":"2024-04-25T10:02:09.515530+0000","last_became_active":"2024-04-25T10:02:04.150202+0000","last_became_peered":"2024-04-25T10:02:04.150202+0000","last_unstale":"2024-04-25T10:02:09.515530+0000","last_undegraded":"2024-04-25T10:02:09.515530+0000","last_fullsized":"2024-04-25T10:02:09.515530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188973+0000","last_change":"2024-04-25T10:02:03.132517+0000","last_active":"2024-04-25T10:02:09.188973+0000","last_peered":"2024-04-25T10:02:09.188973+0000","last_clean":"2024-04-25T10:02:09.188973+0000","last_became_active":"2024-04-25T10:02:03.132128+0000","last_became_peered":"2024-04-25T10:02:03.132128+0000","last_unstale":"2024-04-25T10:02:09.188973+0000","last_undegraded":"2024-04-25T10:02:09.188973+0000","last_fullsized":"2024-04-25T10:02:09.188973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184870+0000","last_change":"2024-04-25T10:02:08.197632+0000","last_active":"2024-04-25T10:02:09.184870+0000","last_peered":"2024-04-25T10:02:09.184870+0000","last_clean":"2024-04-25T10:02:09.184870+0000","last_became_active":"2024-04-25T10:02:08.196287+0000","last_became_peered":"2024-04-25T10:02:08.196287+0000","last_unstale":"2024-04-25T10:02:09.184870+0000","last_undegraded":"2024-04-25T10:02:09.184870+0000","last_fullsized":"2024-04-25T10:02:09.184870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.959507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513645+0000","last_change":"2024-04-25T10:02:04.148290+0000","last_active":"2024-04-25T10:02:09.513645+0000","last_peered":"2024-04-25T10:02:09.513645+0000","last_clean":"2024-04-25T10:02:09.513645+0000","last_became_active":"2024-04-25T10:02:04.147943+0000","last_became_peered":"2024-04-25T10:02:04.147943+0000","last_unstale":"2024-04-25T10:02:09.513645+0000","last_undegraded":"2024-04-25T10:02:09.513645+0000","last_fullsized":"2024-04-25T10:02:09.513645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181446+0000","last_change":"2024-04-25T10:02:03.125338+0000","last_active":"2024-04-25T10:02:09.181446+0000","last_peered":"2024-04-25T10:02:09.181446+0000","last_clean":"2024-04-25T10:02:09.181446+0000","last_became_active":"2024-04-25T10:02:03.123509+0000","last_became_peered":"2024-04-25T10:02:03.123509+0000","last_unstale":"2024-04-25T10:02:09.181446+0000","last_undegraded":"2024-04-25T10:02:09.181446+0000","last_fullsized":"2024-04-25T10:02:09.181446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514606+0000","last_change":"2024-04-25T10:02:08.181642+0000","last_active":"2024-04-25T10:02:09.514606+0000","last_peered":"2024-04-25T10:02:09.514606+0000","last_clean":"2024-04-25T10:02:09.514606+0000","last_became_active":"2024-04-25T10:02:08.181231+0000","last_became_peered":"2024-04-25T10:02:08.181231+0000","last_unstale":"2024-04-25T10:02:09.514606+0000","last_undegraded":"2024-04-25T10:02:09.514606+0000","last_fullsized":"2024-04-25T10:02:09.514606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182787+0000","last_change":"2024-04-25T10:02:04.125758+0000","last_active":"2024-04-25T10:02:09.182787+0000","last_peered":"2024-04-25T10:02:09.182787+0000","last_clean":"2024-04-25T10:02:09.182787+0000","last_became_active":"2024-04-25T10:02:04.124948+0000","last_became_peered":"2024-04-25T10:02:04.124948+0000","last_unstale":"2024-04-25T10:02:09.182787+0000","last_undegraded":"2024-04-25T10:02:09.182787+0000","last_fullsized":"2024-04-25T10:02:09.182787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184171+0000","last_change":"2024-04-25T10:02:03.136435+0000","last_active":"2024-04-25T10:02:09.184171+0000","last_peered":"2024-04-25T10:02:09.184171+0000","last_clean":"2024-04-25T10:02:09.184171+0000","last_became_active":"2024-04-25T10:02:03.134440+0000","last_became_peered":"2024-04-25T10:02:03.134440+0000","last_unstale":"2024-04-25T10:02:09.184171+0000","last_undegraded":"2024-04-25T10:02:09.184171+0000","last_fullsized":"2024-04-25T10:02:09.184171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520993+0000","last_change":"2024-04-25T10:02:08.393430+0000","last_active":"2024-04-25T10:02:09.520993+0000","last_peered":"2024-04-25T10:02:09.520993+0000","last_clean":"2024-04-25T10:02:09.520993+0000","last_became_active":"2024-04-25T10:02:08.392682+0000","last_became_peered":"2024-04-25T10:02:08.392682+0000","last_unstale":"2024-04-25T10:02:09.520993+0000","last_undegraded":"2024-04-25T10:02:09.520993+0000","last_fullsized":"2024-04-25T10:02:09.520993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.626796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514273+0000","last_change":"2024-04-25T10:02:04.145690+0000","last_active":"2024-04-25T10:02:09.514273+0000","last_peered":"2024-04-25T10:02:09.514273+0000","last_clean":"2024-04-25T10:02:09.514273+0000","last_became_active":"2024-04-25T10:02:04.145414+0000","last_became_peered":"2024-04-25T10:02:04.145414+0000","last_unstale":"2024-04-25T10:02:09.514273+0000","last_undegraded":"2024-04-25T10:02:09.514273+0000","last_fullsized":"2024-04-25T10:02:09.514273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514810+0000","last_change":"2024-04-25T10:02:03.151971+0000","last_active":"2024-04-25T10:02:09.514810+0000","last_peered":"2024-04-25T10:02:09.514810+0000","last_clean":"2024-04-25T10:02:09.514810+0000","last_became_active":"2024-04-25T10:02:03.151708+0000","last_became_peered":"2024-04-25T10:02:03.151708+0000","last_unstale":"2024-04-25T10:02:09.514810+0000","last_undegraded":"2024-04-25T10:02:09.514810+0000","last_fullsized":"2024-04-25T10:02:09.514810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520552+0000","last_change":"2024-04-25T10:02:08.393020+0000","last_active":"2024-04-25T10:02:09.520552+0000","last_peered":"2024-04-25T10:02:09.520552+0000","last_clean":"2024-04-25T10:02:09.520552+0000","last_became_active":"2024-04-25T10:02:08.392543+0000","last_became_peered":"2024-04-25T10:02:08.392543+0000","last_unstale":"2024-04-25T10:02:09.520552+0000","last_undegraded":"2024-04-25T10:02:09.520552+0000","last_fullsized":"2024-04-25T10:02:09.520552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:32.205527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512848+0000","last_change":"2024-04-25T10:02:04.137453+0000","last_active":"2024-04-25T10:02:09.512848+0000","last_peered":"2024-04-25T10:02:09.512848+0000","last_clean":"2024-04-25T10:02:09.512848+0000","last_became_active":"2024-04-25T10:02:04.136751+0000","last_became_peered":"2024-04-25T10:02:04.136751+0000","last_unstale":"2024-04-25T10:02:09.512848+0000","last_undegraded":"2024-04-25T10:02:09.512848+0000","last_fullsized":"2024-04-25T10:02:09.512848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514252+0000","last_change":"2024-04-25T10:02:03.134138+0000","last_active":"2024-04-25T10:02:09.514252+0000","last_peered":"2024-04-25T10:02:09.514252+0000","last_clean":"2024-04-25T10:02:09.514252+0000","last_became_active":"2024-04-25T10:02:03.133331+0000","last_became_peered":"2024-04-25T10:02:03.133331+0000","last_unstale":"2024-04-25T10:02:09.514252+0000","last_undegraded":"2024-04-25T10:02:09.514252+0000","last_fullsized":"2024-04-25T10:02:09.514252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516833+0000","last_change":"2024-04-25T10:02:08.399232+0000","last_active":"2024-04-25T10:02:09.516833+0000","last_peered":"2024-04-25T10:02:09.516833+0000","last_clean":"2024-04-25T10:02:09.516833+0000","last_became_active":"2024-04-25T10:02:08.393743+0000","last_became_peered":"2024-04-25T10:02:08.393743+0000","last_unstale":"2024-04-25T10:02:09.516833+0000","last_undegraded":"2024-04-25T10:02:09.516833+0000","last_fullsized":"2024-04-25T10:02:09.516833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514339+0000","last_change":"2024-04-25T10:02:04.145573+0000","last_active":"2024-04-25T10:02:09.514339+0000","last_peered":"2024-04-25T10:02:09.514339+0000","last_clean":"2024-04-25T10:02:09.514339+0000","last_became_active":"2024-04-25T10:02:04.140591+0000","last_became_peered":"2024-04-25T10:02:04.140591+0000","last_unstale":"2024-04-25T10:02:09.514339+0000","last_undegraded":"2024-04-25T10:02:09.514339+0000","last_fullsized":"2024-04-25T10:02:09.514339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:21.801655+0000","last_change":"2024-04-25T10:02:03.131213+0000","last_active":"2024-04-25T10:02:21.801655+0000","last_peered":"2024-04-25T10:02:21.801655+0000","last_clean":"2024-04-25T10:02:21.801655+0000","last_became_active":"2024-04-25T10:02:03.130876+0000","last_became_peered":"2024-04-25T10:02:03.130876+0000","last_unstale":"2024-04-25T10:02:21.801655+0000","last_undegraded":"2024-04-25T10:02:21.801655+0000","last_fullsized":"2024-04-25T10:02:21.801655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518147+0000","last_change":"2024-04-25T10:02:08.197739+0000","last_active":"2024-04-25T10:02:09.518147+0000","last_peered":"2024-04-25T10:02:09.518147+0000","last_clean":"2024-04-25T10:02:09.518147+0000","last_became_active":"2024-04-25T10:02:08.195924+0000","last_became_peered":"2024-04-25T10:02:08.195924+0000","last_unstale":"2024-04-25T10:02:09.518147+0000","last_undegraded":"2024-04-25T10:02:09.518147+0000","last_fullsized":"2024-04-25T10:02:09.518147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188422+0000","last_change":"2024-04-25T10:02:04.147813+0000","last_active":"2024-04-25T10:02:09.188422+0000","last_peered":"2024-04-25T10:02:09.188422+0000","last_clean":"2024-04-25T10:02:09.188422+0000","last_became_active":"2024-04-25T10:02:04.140229+0000","last_became_peered":"2024-04-25T10:02:04.140229+0000","last_unstale":"2024-04-25T10:02:09.188422+0000","last_undegraded":"2024-04-25T10:02:09.188422+0000","last_fullsized":"2024-04-25T10:02:09.188422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:19.761376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515518+0000","last_change":"2024-04-25T10:02:03.154687+0000","last_active":"2024-04-25T10:02:09.515518+0000","last_peered":"2024-04-25T10:02:09.515518+0000","last_clean":"2024-04-25T10:02:09.515518+0000","last_became_active":"2024-04-25T10:02:03.154323+0000","last_became_peered":"2024-04-25T10:02:03.154323+0000","last_unstale":"2024-04-25T10:02:09.515518+0000","last_undegraded":"2024-04-25T10:02:09.515518+0000","last_fullsized":"2024-04-25T10:02:09.515518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187851+0000","last_change":"2024-04-25T10:02:08.193520+0000","last_active":"2024-04-25T10:02:09.187851+0000","last_peered":"2024-04-25T10:02:09.187851+0000","last_clean":"2024-04-25T10:02:09.187851+0000","last_became_active":"2024-04-25T10:02:08.193040+0000","last_became_peered":"2024-04-25T10:02:08.193040+0000","last_unstale":"2024-04-25T10:02:09.187851+0000","last_undegraded":"2024-04-25T10:02:09.187851+0000","last_fullsized":"2024-04-25T10:02:09.187851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513593+0000","last_change":"2024-04-25T10:02:04.137131+0000","last_active":"2024-04-25T10:02:09.513593+0000","last_peered":"2024-04-25T10:02:09.513593+0000","last_clean":"2024-04-25T10:02:09.513593+0000","last_became_active":"2024-04-25T10:02:04.136832+0000","last_became_peered":"2024-04-25T10:02:04.136832+0000","last_unstale":"2024-04-25T10:02:09.513593+0000","last_undegraded":"2024-04-25T10:02:09.513593+0000","last_fullsized":"2024-04-25T10:02:09.513593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516824+0000","last_change":"2024-04-25T10:02:03.154418+0000","last_active":"2024-04-25T10:02:09.516824+0000","last_peered":"2024-04-25T10:02:09.516824+0000","last_clean":"2024-04-25T10:02:09.516824+0000","last_became_active":"2024-04-25T10:02:03.154147+0000","last_became_peered":"2024-04-25T10:02:03.154147+0000","last_unstale":"2024-04-25T10:02:09.516824+0000","last_undegraded":"2024-04-25T10:02:09.516824+0000","last_fullsized":"2024-04-25T10:02:09.516824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519420+0000","last_change":"2024-04-25T10:02:08.192052+0000","last_active":"2024-04-25T10:02:09.519420+0000","last_peered":"2024-04-25T10:02:09.519420+0000","last_clean":"2024-04-25T10:02:09.519420+0000","last_became_active":"2024-04-25T10:02:08.190628+0000","last_became_peered":"2024-04-25T10:02:08.190628+0000","last_unstale":"2024-04-25T10:02:09.519420+0000","last_undegraded":"2024-04-25T10:02:09.519420+0000","last_fullsized":"2024-04-25T10:02:09.519420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516671+0000","last_change":"2024-04-25T10:02:04.149114+0000","last_active":"2024-04-25T10:02:09.516671+0000","last_peered":"2024-04-25T10:02:09.516671+0000","last_clean":"2024-04-25T10:02:09.516671+0000","last_became_active":"2024-04-25T10:02:04.147309+0000","last_became_peered":"2024-04-25T10:02:04.147309+0000","last_unstale":"2024-04-25T10:02:09.516671+0000","last_undegraded":"2024-04-25T10:02:09.516671+0000","last_fullsized":"2024-04-25T10:02:09.516671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187650+0000","last_change":"2024-04-25T10:02:03.150915+0000","last_active":"2024-04-25T10:02:09.187650+0000","last_peered":"2024-04-25T10:02:09.187650+0000","last_clean":"2024-04-25T10:02:09.187650+0000","last_became_active":"2024-04-25T10:02:03.150382+0000","last_became_peered":"2024-04-25T10:02:03.150382+0000","last_unstale":"2024-04-25T10:02:09.187650+0000","last_undegraded":"2024-04-25T10:02:09.187650+0000","last_fullsized":"2024-04-25T10:02:09.187650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520143+0000","last_change":"2024-04-25T10:02:08.409717+0000","last_active":"2024-04-25T10:02:09.520143+0000","last_peered":"2024-04-25T10:02:09.520143+0000","last_clean":"2024-04-25T10:02:09.520143+0000","last_became_active":"2024-04-25T10:02:08.409153+0000","last_became_peered":"2024-04-25T10:02:08.409153+0000","last_unstale":"2024-04-25T10:02:09.520143+0000","last_undegraded":"2024-04-25T10:02:09.520143+0000","last_fullsized":"2024-04-25T10:02:09.520143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:59.321979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514307+0000","last_change":"2024-04-25T10:02:08.402841+0000","last_active":"2024-04-25T10:02:09.514307+0000","last_peered":"2024-04-25T10:02:09.514307+0000","last_clean":"2024-04-25T10:02:09.514307+0000","last_became_active":"2024-04-25T10:02:08.402205+0000","last_became_peered":"2024-04-25T10:02:08.402205+0000","last_unstale":"2024-04-25T10:02:09.514307+0000","last_undegraded":"2024-04-25T10:02:09.514307+0000","last_fullsized":"2024-04-25T10:02:09.514307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514163+0000","last_change":"2024-04-25T10:02:04.136408+0000","last_active":"2024-04-25T10:02:09.514163+0000","last_peered":"2024-04-25T10:02:09.514163+0000","last_clean":"2024-04-25T10:02:09.514163+0000","last_became_active":"2024-04-25T10:02:04.136094+0000","last_became_peered":"2024-04-25T10:02:04.136094+0000","last_unstale":"2024-04-25T10:02:09.514163+0000","last_undegraded":"2024-04-25T10:02:09.514163+0000","last_fullsized":"2024-04-25T10:02:09.514163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513538+0000","last_change":"2024-04-25T10:02:03.154541+0000","last_active":"2024-04-25T10:02:09.513538+0000","last_peered":"2024-04-25T10:02:09.513538+0000","last_clean":"2024-04-25T10:02:09.513538+0000","last_became_active":"2024-04-25T10:02:03.154193+0000","last_became_peered":"2024-04-25T10:02:03.154193+0000","last_unstale":"2024-04-25T10:02:09.513538+0000","last_undegraded":"2024-04-25T10:02:09.513538+0000","last_fullsized":"2024-04-25T10:02:09.513538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184564+0000","last_change":"2024-04-25T10:02:08.192506+0000","last_active":"2024-04-25T10:02:09.184564+0000","last_peered":"2024-04-25T10:02:09.184564+0000","last_clean":"2024-04-25T10:02:09.184564+0000","last_became_active":"2024-04-25T10:02:08.190910+0000","last_became_peered":"2024-04-25T10:02:08.190910+0000","last_unstale":"2024-04-25T10:02:09.184564+0000","last_undegraded":"2024-04-25T10:02:09.184564+0000","last_fullsized":"2024-04-25T10:02:09.184564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519465+0000","last_change":"2024-04-25T10:02:04.138218+0000","last_active":"2024-04-25T10:02:09.519465+0000","last_peered":"2024-04-25T10:02:09.519465+0000","last_clean":"2024-04-25T10:02:09.519465+0000","last_became_active":"2024-04-25T10:02:04.137919+0000","last_became_peered":"2024-04-25T10:02:04.137919+0000","last_unstale":"2024-04-25T10:02:09.519465+0000","last_undegraded":"2024-04-25T10:02:09.519465+0000","last_fullsized":"2024-04-25T10:02:09.519465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520192+0000","last_change":"2024-04-25T10:02:03.133971+0000","last_active":"2024-04-25T10:02:09.520192+0000","last_peered":"2024-04-25T10:02:09.520192+0000","last_clean":"2024-04-25T10:02:09.520192+0000","last_became_active":"2024-04-25T10:02:03.133661+0000","last_became_peered":"2024-04-25T10:02:03.133661+0000","last_unstale":"2024-04-25T10:02:09.520192+0000","last_undegraded":"2024-04-25T10:02:09.520192+0000","last_fullsized":"2024-04-25T10:02:09.520192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183101+0000","last_change":"2024-04-25T10:02:03.137884+0000","last_active":"2024-04-25T10:02:09.183101+0000","last_peered":"2024-04-25T10:02:09.183101+0000","last_clean":"2024-04-25T10:02:09.183101+0000","last_became_active":"2024-04-25T10:02:03.137196+0000","last_became_peered":"2024-04-25T10:02:03.137196+0000","last_unstale":"2024-04-25T10:02:09.183101+0000","last_undegraded":"2024-04-25T10:02:09.183101+0000","last_fullsized":"2024-04-25T10:02:09.183101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514125+0000","last_change":"2024-04-25T10:02:04.130491+0000","last_active":"2024-04-25T10:02:09.514125+0000","last_peered":"2024-04-25T10:02:09.514125+0000","last_clean":"2024-04-25T10:02:09.514125+0000","last_became_active":"2024-04-25T10:02:04.129963+0000","last_became_peered":"2024-04-25T10:02:04.129963+0000","last_unstale":"2024-04-25T10:02:09.514125+0000","last_undegraded":"2024-04-25T10:02:09.514125+0000","last_fullsized":"2024-04-25T10:02:09.514125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182210+0000","last_change":"2024-04-25T10:02:08.186410+0000","last_active":"2024-04-25T10:02:09.182210+0000","last_peered":"2024-04-25T10:02:09.182210+0000","last_clean":"2024-04-25T10:02:09.182210+0000","last_became_active":"2024-04-25T10:02:08.185964+0000","last_became_peered":"2024-04-25T10:02:08.185964+0000","last_unstale":"2024-04-25T10:02:09.182210+0000","last_undegraded":"2024-04-25T10:02:09.182210+0000","last_fullsized":"2024-04-25T10:02:09.182210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:17.471287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520398+0000","last_change":"2024-04-25T10:02:03.134295+0000","last_active":"2024-04-25T10:02:09.520398+0000","last_peered":"2024-04-25T10:02:09.520398+0000","last_clean":"2024-04-25T10:02:09.520398+0000","last_became_active":"2024-04-25T10:02:03.134033+0000","last_became_peered":"2024-04-25T10:02:03.134033+0000","last_unstale":"2024-04-25T10:02:09.520398+0000","last_undegraded":"2024-04-25T10:02:09.520398+0000","last_fullsized":"2024-04-25T10:02:09.520398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.186180+0000","last_change":"2024-04-25T10:02:04.137055+0000","last_active":"2024-04-25T10:02:09.186180+0000","last_peered":"2024-04-25T10:02:09.186180+0000","last_clean":"2024-04-25T10:02:09.186180+0000","last_became_active":"2024-04-25T10:02:04.131629+0000","last_became_peered":"2024-04-25T10:02:04.131629+0000","last_unstale":"2024-04-25T10:02:09.186180+0000","last_undegraded":"2024-04-25T10:02:09.186180+0000","last_fullsized":"2024-04-25T10:02:09.186180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521270+0000","last_change":"2024-04-25T10:02:08.187719+0000","last_active":"2024-04-25T10:02:09.521270+0000","last_peered":"2024-04-25T10:02:09.521270+0000","last_clean":"2024-04-25T10:02:09.521270+0000","last_became_active":"2024-04-25T10:02:08.187298+0000","last_became_peered":"2024-04-25T10:02:08.187298+0000","last_unstale":"2024-04-25T10:02:09.521270+0000","last_undegraded":"2024-04-25T10:02:09.521270+0000","last_fullsized":"2024-04-25T10:02:09.521270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:15.995769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182212+0000","last_change":"2024-04-25T10:02:04.125579+0000","last_active":"2024-04-25T10:02:09.182212+0000","last_peered":"2024-04-25T10:02:09.182212+0000","last_clean":"2024-04-25T10:02:09.182212+0000","last_became_active":"2024-04-25T10:02:04.124972+0000","last_became_peered":"2024-04-25T10:02:04.124972+0000","last_unstale":"2024-04-25T10:02:09.182212+0000","last_undegraded":"2024-04-25T10:02:09.182212+0000","last_fullsized":"2024-04-25T10:02:09.182212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181677+0000","last_change":"2024-04-25T10:02:04.145556+0000","last_active":"2024-04-25T10:02:09.181677+0000","last_peered":"2024-04-25T10:02:09.181677+0000","last_clean":"2024-04-25T10:02:09.181677+0000","last_became_active":"2024-04-25T10:02:04.144887+0000","last_became_peered":"2024-04-25T10:02:04.144887+0000","last_unstale":"2024-04-25T10:02:09.181677+0000","last_undegraded":"2024-04-25T10:02:09.181677+0000","last_fullsized":"2024-04-25T10:02:09.181677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514803+0000","last_change":"2024-04-25T10:02:04.142127+0000","last_active":"2024-04-25T10:02:09.514803+0000","last_peered":"2024-04-25T10:02:09.514803+0000","last_clean":"2024-04-25T10:02:09.514803+0000","last_became_active":"2024-04-25T10:02:04.140799+0000","last_became_peered":"2024-04-25T10:02:04.140799+0000","last_unstale":"2024-04-25T10:02:09.514803+0000","last_undegraded":"2024-04-25T10:02:09.514803+0000","last_fullsized":"2024-04-25T10:02:09.514803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514188+0000","last_change":"2024-04-25T10:02:04.141010+0000","last_active":"2024-04-25T10:02:09.514188+0000","last_peered":"2024-04-25T10:02:09.514188+0000","last_clean":"2024-04-25T10:02:09.514188+0000","last_became_active":"2024-04-25T10:02:04.140201+0000","last_became_peered":"2024-04-25T10:02:04.140201+0000","last_unstale":"2024-04-25T10:02:09.514188+0000","last_undegraded":"2024-04-25T10:02:09.514188+0000","last_fullsized":"2024-04-25T10:02:09.514188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521086+0000","last_change":"2024-04-25T10:02:04.137613+0000","last_active":"2024-04-25T10:02:09.521086+0000","last_peered":"2024-04-25T10:02:09.521086+0000","last_clean":"2024-04-25T10:02:09.521086+0000","last_became_active":"2024-04-25T10:02:04.136833+0000","last_became_peered":"2024-04-25T10:02:04.136833+0000","last_unstale":"2024-04-25T10:02:09.521086+0000","last_undegraded":"2024-04-25T10:02:09.521086+0000","last_fullsized":"2024-04-25T10:02:09.521086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514990+0000","last_change":"2024-04-25T10:02:04.135708+0000","last_active":"2024-04-25T10:02:09.514990+0000","last_peered":"2024-04-25T10:02:09.514990+0000","last_clean":"2024-04-25T10:02:09.514990+0000","last_became_active":"2024-04-25T10:02:04.135383+0000","last_became_peered":"2024-04-25T10:02:04.135383+0000","last_unstale":"2024-04-25T10:02:09.514990+0000","last_undegraded":"2024-04-25T10:02:09.514990+0000","last_fullsized":"2024-04-25T10:02:09.514990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187243+0000","last_change":"2024-04-25T10:02:04.151491+0000","last_active":"2024-04-25T10:02:09.187243+0000","last_peered":"2024-04-25T10:02:09.187243+0000","last_clean":"2024-04-25T10:02:09.187243+0000","last_became_active":"2024-04-25T10:02:04.151181+0000","last_became_peered":"2024-04-25T10:02:04.151181+0000","last_unstale":"2024-04-25T10:02:09.187243+0000","last_undegraded":"2024-04-25T10:02:09.187243+0000","last_fullsized":"2024-04-25T10:02:09.187243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521012+0000","last_change":"2024-04-25T10:02:04.133889+0000","last_active":"2024-04-25T10:02:09.521012+0000","last_peered":"2024-04-25T10:02:09.521012+0000","last_clean":"2024-04-25T10:02:09.521012+0000","last_became_active":"2024-04-25T10:02:04.132937+0000","last_became_peered":"2024-04-25T10:02:04.132937+0000","last_unstale":"2024-04-25T10:02:09.521012+0000","last_undegraded":"2024-04-25T10:02:09.521012+0000","last_fullsized":"2024-04-25T10:02:09.521012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513229+0000","last_change":"2024-04-25T10:02:04.142217+0000","last_active":"2024-04-25T10:02:09.513229+0000","last_peered":"2024-04-25T10:02:09.513229+0000","last_clean":"2024-04-25T10:02:09.513229+0000","last_became_active":"2024-04-25T10:02:04.141431+0000","last_became_peered":"2024-04-25T10:02:04.141431+0000","last_unstale":"2024-04-25T10:02:09.513229+0000","last_undegraded":"2024-04-25T10:02:09.513229+0000","last_fullsized":"2024-04-25T10:02:09.513229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520025+0000","last_change":"2024-04-25T10:02:04.148470+0000","last_active":"2024-04-25T10:02:09.520025+0000","last_peered":"2024-04-25T10:02:09.520025+0000","last_clean":"2024-04-25T10:02:09.520025+0000","last_became_active":"2024-04-25T10:02:04.147993+0000","last_became_peered":"2024-04-25T10:02:04.147993+0000","last_unstale":"2024-04-25T10:02:09.520025+0000","last_undegraded":"2024-04-25T10:02:09.520025+0000","last_fullsized":"2024-04-25T10:02:09.520025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513946+0000","last_change":"2024-04-25T10:02:04.148427+0000","last_active":"2024-04-25T10:02:09.513946+0000","last_peered":"2024-04-25T10:02:09.513946+0000","last_clean":"2024-04-25T10:02:09.513946+0000","last_became_active":"2024-04-25T10:02:04.147301+0000","last_became_peered":"2024-04-25T10:02:04.147301+0000","last_unstale":"2024-04-25T10:02:09.513946+0000","last_undegraded":"2024-04-25T10:02:09.513946+0000","last_fullsized":"2024-04-25T10:02:09.513946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513726+0000","last_change":"2024-04-25T10:02:04.136324+0000","last_active":"2024-04-25T10:02:09.513726+0000","last_peered":"2024-04-25T10:02:09.513726+0000","last_clean":"2024-04-25T10:02:09.513726+0000","last_became_active":"2024-04-25T10:02:04.136035+0000","last_became_peered":"2024-04-25T10:02:04.136035+0000","last_unstale":"2024-04-25T10:02:09.513726+0000","last_undegraded":"2024-04-25T10:02:09.513726+0000","last_fullsized":"2024-04-25T10:02:09.513726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184020+0000","last_change":"2024-04-25T10:02:04.134441+0000","last_active":"2024-04-25T10:02:09.184020+0000","last_peered":"2024-04-25T10:02:09.184020+0000","last_clean":"2024-04-25T10:02:09.184020+0000","last_became_active":"2024-04-25T10:02:04.130052+0000","last_became_peered":"2024-04-25T10:02:04.130052+0000","last_unstale":"2024-04-25T10:02:09.184020+0000","last_undegraded":"2024-04-25T10:02:09.184020+0000","last_fullsized":"2024-04-25T10:02:09.184020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515111+0000","last_change":"2024-04-25T10:02:04.146376+0000","last_active":"2024-04-25T10:02:09.515111+0000","last_peered":"2024-04-25T10:02:09.515111+0000","last_clean":"2024-04-25T10:02:09.515111+0000","last_became_active":"2024-04-25T10:02:04.146080+0000","last_became_peered":"2024-04-25T10:02:04.146080+0000","last_unstale":"2024-04-25T10:02:09.515111+0000","last_undegraded":"2024-04-25T10:02:09.515111+0000","last_fullsized":"2024-04-25T10:02:09.515111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182337+0000","last_change":"2024-04-25T10:02:04.135427+0000","last_active":"2024-04-25T10:02:09.182337+0000","last_peered":"2024-04-25T10:02:09.182337+0000","last_clean":"2024-04-25T10:02:09.182337+0000","last_became_active":"2024-04-25T10:02:04.134990+0000","last_became_peered":"2024-04-25T10:02:04.134990+0000","last_unstale":"2024-04-25T10:02:09.182337+0000","last_undegraded":"2024-04-25T10:02:09.182337+0000","last_fullsized":"2024-04-25T10:02:09.182337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517176+0000","last_change":"2024-04-25T10:02:04.140449+0000","last_active":"2024-04-25T10:02:09.517176+0000","last_peered":"2024-04-25T10:02:09.517176+0000","last_clean":"2024-04-25T10:02:09.517176+0000","last_became_active":"2024-04-25T10:02:04.140129+0000","last_became_peered":"2024-04-25T10:02:04.140129+0000","last_unstale":"2024-04-25T10:02:09.517176+0000","last_undegraded":"2024-04-25T10:02:09.517176+0000","last_fullsized":"2024-04-25T10:02:09.517176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513015+0000","last_change":"2024-04-25T10:02:04.137099+0000","last_active":"2024-04-25T10:02:09.513015+0000","last_peered":"2024-04-25T10:02:09.513015+0000","last_clean":"2024-04-25T10:02:09.513015+0000","last_became_active":"2024-04-25T10:02:04.136845+0000","last_became_peered":"2024-04-25T10:02:04.136845+0000","last_unstale":"2024-04-25T10:02:09.513015+0000","last_undegraded":"2024-04-25T10:02:09.513015+0000","last_fullsized":"2024-04-25T10:02:09.513015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514467+0000","last_change":"2024-04-25T10:02:04.133727+0000","last_active":"2024-04-25T10:02:09.514467+0000","last_peered":"2024-04-25T10:02:09.514467+0000","last_clean":"2024-04-25T10:02:09.514467+0000","last_became_active":"2024-04-25T10:02:04.132995+0000","last_became_peered":"2024-04-25T10:02:04.132995+0000","last_unstale":"2024-04-25T10:02:09.514467+0000","last_undegraded":"2024-04-25T10:02:09.514467+0000","last_fullsized":"2024-04-25T10:02:09.514467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512646+0000","last_change":"2024-04-25T10:02:04.128013+0000","last_active":"2024-04-25T10:02:09.512646+0000","last_peered":"2024-04-25T10:02:09.512646+0000","last_clean":"2024-04-25T10:02:09.512646+0000","last_became_active":"2024-04-25T10:02:04.123162+0000","last_became_peered":"2024-04-25T10:02:04.123162+0000","last_unstale":"2024-04-25T10:02:09.512646+0000","last_undegraded":"2024-04-25T10:02:09.512646+0000","last_fullsized":"2024-04-25T10:02:09.512646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:26.232712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520128+0000","last_change":"2024-04-25T10:02:04.137865+0000","last_active":"2024-04-25T10:02:09.520128+0000","last_peered":"2024-04-25T10:02:09.520128+0000","last_clean":"2024-04-25T10:02:09.520128+0000","last_became_active":"2024-04-25T10:02:04.137413+0000","last_became_peered":"2024-04-25T10:02:04.137413+0000","last_unstale":"2024-04-25T10:02:09.520128+0000","last_undegraded":"2024-04-25T10:02:09.520128+0000","last_fullsized":"2024-04-25T10:02:09.520128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188286+0000","last_change":"2024-04-25T10:02:04.148039+0000","last_active":"2024-04-25T10:02:09.188286+0000","last_peered":"2024-04-25T10:02:09.188286+0000","last_clean":"2024-04-25T10:02:09.188286+0000","last_became_active":"2024-04-25T10:02:04.146104+0000","last_became_peered":"2024-04-25T10:02:04.146104+0000","last_unstale":"2024-04-25T10:02:09.188286+0000","last_undegraded":"2024-04-25T10:02:09.188286+0000","last_fullsized":"2024-04-25T10:02:09.188286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:15.812709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183940+0000","last_change":"2024-04-25T10:02:04.149357+0000","last_active":"2024-04-25T10:02:09.183940+0000","last_peered":"2024-04-25T10:02:09.183940+0000","last_clean":"2024-04-25T10:02:09.183940+0000","last_became_active":"2024-04-25T10:02:04.148994+0000","last_became_peered":"2024-04-25T10:02:04.148994+0000","last_unstale":"2024-04-25T10:02:09.183940+0000","last_undegraded":"2024-04-25T10:02:09.183940+0000","last_fullsized":"2024-04-25T10:02:09.183940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187107+0000","last_change":"2024-04-25T10:02:04.144119+0000","last_active":"2024-04-25T10:02:09.187107+0000","last_peered":"2024-04-25T10:02:09.187107+0000","last_clean":"2024-04-25T10:02:09.187107+0000","last_became_active":"2024-04-25T10:02:04.143818+0000","last_became_peered":"2024-04-25T10:02:04.143818+0000","last_unstale":"2024-04-25T10:02:09.187107+0000","last_undegraded":"2024-04-25T10:02:09.187107+0000","last_fullsized":"2024-04-25T10:02:09.187107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515823+0000","last_change":"2024-04-25T10:02:04.152842+0000","last_active":"2024-04-25T10:02:09.515823+0000","last_peered":"2024-04-25T10:02:09.515823+0000","last_clean":"2024-04-25T10:02:09.515823+0000","last_became_active":"2024-04-25T10:02:04.142957+0000","last_became_peered":"2024-04-25T10:02:04.142957+0000","last_unstale":"2024-04-25T10:02:09.515823+0000","last_undegraded":"2024-04-25T10:02:09.515823+0000","last_fullsized":"2024-04-25T10:02:09.515823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519170+0000","last_change":"2024-04-25T10:02:04.143950+0000","last_active":"2024-04-25T10:02:09.519170+0000","last_peered":"2024-04-25T10:02:09.519170+0000","last_clean":"2024-04-25T10:02:09.519170+0000","last_became_active":"2024-04-25T10:02:04.143615+0000","last_became_peered":"2024-04-25T10:02:04.143615+0000","last_unstale":"2024-04-25T10:02:09.519170+0000","last_undegraded":"2024-04-25T10:02:09.519170+0000","last_fullsized":"2024-04-25T10:02:09.519170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513367+0000","last_change":"2024-04-25T10:02:04.140815+0000","last_active":"2024-04-25T10:02:09.513367+0000","last_peered":"2024-04-25T10:02:09.513367+0000","last_clean":"2024-04-25T10:02:09.513367+0000","last_became_active":"2024-04-25T10:02:04.140458+0000","last_became_peered":"2024-04-25T10:02:04.140458+0000","last_unstale":"2024-04-25T10:02:09.513367+0000","last_undegraded":"2024-04-25T10:02:09.513367+0000","last_fullsized":"2024-04-25T10:02:09.513367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513577+0000","last_change":"2024-04-25T10:02:04.144469+0000","last_active":"2024-04-25T10:02:09.513577+0000","last_peered":"2024-04-25T10:02:09.513577+0000","last_clean":"2024-04-25T10:02:09.513577+0000","last_became_active":"2024-04-25T10:02:04.143382+0000","last_became_peered":"2024-04-25T10:02:04.143382+0000","last_unstale":"2024-04-25T10:02:09.513577+0000","last_undegraded":"2024-04-25T10:02:09.513577+0000","last_fullsized":"2024-04-25T10:02:09.513577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514018+0000","last_change":"2024-04-25T10:02:04.138676+0000","last_active":"2024-04-25T10:02:09.514018+0000","last_peered":"2024-04-25T10:02:09.514018+0000","last_clean":"2024-04-25T10:02:09.514018+0000","last_became_active":"2024-04-25T10:02:04.138397+0000","last_became_peered":"2024-04-25T10:02:04.138397+0000","last_unstale":"2024-04-25T10:02:09.514018+0000","last_undegraded":"2024-04-25T10:02:09.514018+0000","last_fullsized":"2024-04-25T10:02:09.514018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516016+0000","last_change":"2024-04-25T10:02:04.145097+0000","last_active":"2024-04-25T10:02:09.516016+0000","last_peered":"2024-04-25T10:02:09.516016+0000","last_clean":"2024-04-25T10:02:09.516016+0000","last_became_active":"2024-04-25T10:02:04.142128+0000","last_became_peered":"2024-04-25T10:02:04.142128+0000","last_unstale":"2024-04-25T10:02:09.516016+0000","last_undegraded":"2024-04-25T10:02:09.516016+0000","last_fullsized":"2024-04-25T10:02:09.516016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513292+0000","last_change":"2024-04-25T10:02:04.124760+0000","last_active":"2024-04-25T10:02:09.513292+0000","last_peered":"2024-04-25T10:02:09.513292+0000","last_clean":"2024-04-25T10:02:09.513292+0000","last_became_active":"2024-04-25T10:02:04.124423+0000","last_became_peered":"2024-04-25T10:02:04.124423+0000","last_unstale":"2024-04-25T10:02:09.513292+0000","last_undegraded":"2024-04-25T10:02:09.513292+0000","last_fullsized":"2024-04-25T10:02:09.513292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515984+0000","last_change":"2024-04-25T10:02:04.129218+0000","last_active":"2024-04-25T10:02:09.515984+0000","last_peered":"2024-04-25T10:02:09.515984+0000","last_clean":"2024-04-25T10:02:09.515984+0000","last_became_active":"2024-04-25T10:02:04.126046+0000","last_became_peered":"2024-04-25T10:02:04.126046+0000","last_unstale":"2024-04-25T10:02:09.515984+0000","last_undegraded":"2024-04-25T10:02:09.515984+0000","last_fullsized":"2024-04-25T10:02:09.515984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515345+0000","last_change":"2024-04-25T10:02:04.137569+0000","last_active":"2024-04-25T10:02:09.515345+0000","last_peered":"2024-04-25T10:02:09.515345+0000","last_clean":"2024-04-25T10:02:09.515345+0000","last_became_active":"2024-04-25T10:02:04.137205+0000","last_became_peered":"2024-04-25T10:02:04.137205+0000","last_unstale":"2024-04-25T10:02:09.515345+0000","last_undegraded":"2024-04-25T10:02:09.515345+0000","last_fullsized":"2024-04-25T10:02:09.515345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:26.657 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:26.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.884+0000 7fc86a400640 1 Processor -- start 2024-04-25T10:02:26.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.884+0000 7fc86a400640 1 -- start start 2024-04-25T10:02:26.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc86a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc864164970 0x7fc864166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc86a400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc864144b30 con 0x7fc864144750 2024-04-25T10:02:26.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc86a400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640712d0 con 0x7fc8640ce920 2024-04-25T10:02:26.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc86a400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8640715b0 con 0x7fc864164970 2024-04-25T10:02:26.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc863e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc864164970 0x7fc864166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc863400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc864144750 0x7fc86406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:57982/0 (socket says 172.21.15.165:57982) 2024-04-25T10:02:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.886+0000 7fc863400640 1 -- 172.21.15.165:0/4016037692 learned_addr learned my addr 172.21.15.165:0/4016037692 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.887+0000 7fc863e00640 1 -- 172.21.15.165:0/4016037692 >> v1:172.21.15.172:6789/0 conn(0x7fc8640ce920 legacy=0x7fc8640cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.887+0000 7fc862000640 1 -- 172.21.15.165:0/4016037692 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4159192729 0 0) 0x7fc864144b30 con 0x7fc864144750 2024-04-25T10:02:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.887+0000 7fc863e00640 1 -- 172.21.15.165:0/4016037692 >> v1:172.21.15.165:6789/0 conn(0x7fc864144750 legacy=0x7fc86406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.887+0000 7fc863e00640 1 -- 172.21.15.165:0/4016037692 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864071890 con 0x7fc864164970 2024-04-25T10:02:26.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.888+0000 7fc863e00640 1 --2- 172.21.15.165:0/4016037692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc864164970 0x7fc864166d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009d90 tx=0x7fc85c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f599381e66a8bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.888+0000 7fc862000640 1 -- 172.21.15.165:0/4016037692 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c007740 con 0x7fc864164970 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.888+0000 7fc862000640 1 -- 172.21.15.165:0/4016037692 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0078e0 con 0x7fc864164970 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.888+0000 7fc862000640 1 -- 172.21.15.165:0/4016037692 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c007cb0 con 0x7fc864164970 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 -- 172.21.15.165:0/4016037692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc864164970 msgr2=0x7fc864166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 --2- 172.21.15.165:0/4016037692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc864164970 0x7fc864166d60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fc85c009d90 tx=0x7fc85c02a660 comp rx=0 tx=0).stop 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 -- 172.21.15.165:0/4016037692 shutdown_connections 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 --2- 172.21.15.165:0/4016037692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc864164970 0x7fc864166d60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 -- 172.21.15.165:0/4016037692 >> 172.21.15.165:0/4016037692 conn(0x7fc86406abe0 msgr2=0x7fc86406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 -- 172.21.15.165:0/4016037692 shutdown_connections 2024-04-25T10:02:26.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.889+0000 7fc86a400640 1 -- 172.21.15.165:0/4016037692 wait complete. 2024-04-25T10:02:26.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.891+0000 7fc86a400640 1 Processor -- start 2024-04-25T10:02:26.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.891+0000 7fc86a400640 1 -- start start 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc86a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 0x7fc8640b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc86a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc864144750 0x7fc8640add50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc86a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc864164970 0x7fc8640b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc86a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864145450 con 0x7fc8640ce920 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc86a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc86406f020 con 0x7fc864164970 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc86a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641455d0 con 0x7fc864144750 2024-04-25T10:02:26.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc863400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 0x7fc8640b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc863400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 0x7fc8640b1ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50264/0 (socket says 172.21.15.165:50264) 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.892+0000 7fc863400640 1 -- 172.21.15.165:0/2027743506 learned_addr learned my addr 172.21.15.165:0/2027743506 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863e00640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc864164970 0x7fc8640b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863400640 1 -- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc864144750 msgr2=0x7fc8640add50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc862a00640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc864144750 0x7fc8640add50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc864144750 0x7fc8640add50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863400640 1 -- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc864164970 msgr2=0x7fc8640b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc864164970 0x7fc8640b03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863400640 1 -- 172.21.15.165:0/2027743506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640b0c40 con 0x7fc8640ce920 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863e00640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc864164970 0x7fc8640b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc862a00640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc864144750 0x7fc8640add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:26.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc863400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 0x7fc8640b1ce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc85400c4a0 tx=0x7fc85400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854018070 con 0x7fc8640ce920 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8540074e0 con 0x7fc8640ce920 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc854004a20 con 0x7fc8640ce920 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8640b0f20 con 0x7fc8640ce920 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.893+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8640b14e0 con 0x7fc8640ce920 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.895+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc864073990 con 0x7fc8640ce920 2024-04-25T10:02:26.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.895+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc85401b070 con 0x7fc8640ce920 2024-04-25T10:02:26.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.897+0000 7fc85b400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:26.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.897+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc854089950 con 0x7fc8640ce920 2024-04-25T10:02:26.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.897+0000 7fc862a00640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:26.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.898+0000 7fc862a00640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b4d0 0x7fc84406d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc86415b2f0 tx=0x7fc850009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:26.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:26.901+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8540583f0 con 0x7fc8640ce920 2024-04-25T10:02:27.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.093+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc8640ae290 con 0x7fc84406b4d0 2024-04-25T10:02:27.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.102+0000 7fc85b400640 1 -- 172.21.15.165:0/2027743506 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332574 (secure 0 0 0) 0x7fc8640ae290 con 0x7fc84406b4d0 2024-04-25T10:02:27.109 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:27.113 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b4d0 msgr2=0x7fc84406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b4d0 0x7fc84406d990 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc86415b2f0 tx=0x7fc850009040 comp rx=0 tx=0).stop 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 msgr2=0x7fc8640b1ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 0x7fc8640b1ce0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fc85400c4a0 tx=0x7fc85400c960 comp rx=0 tx=0).stop 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 shutdown_connections 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b4d0 0x7fc84406d990 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc864164970 0x7fc8640b03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc864144750 0x7fc8640add50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 --2- 172.21.15.165:0/2027743506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8640ce920 0x7fc8640b1ce0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.109+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 >> 172.21.15.165:0/2027743506 conn(0x7fc86406abe0 msgr2=0x7fc8640cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.110+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 shutdown_connections 2024-04-25T10:02:27.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.110+0000 7fc86a400640 1 -- 172.21.15.165:0/2027743506 wait complete. 2024-04-25T10:02:27.133 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2024-04-25T10:02:26.130500+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219420,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755300,"statfs":{"total":773094113280,"available":772869427200,"internally_reserved":0,"allocated":3284992,"data_stored":1991782,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2230,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004471"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513776+0000","last_change":"2024-04-25T10:02:03.142252+0000","last_active":"2024-04-25T10:02:09.513776+0000","last_peered":"2024-04-25T10:02:09.513776+0000","last_clean":"2024-04-25T10:02:09.513776+0000","last_became_active":"2024-04-25T10:02:03.141984+0000","last_became_peered":"2024-04-25T10:02:03.141984+0000","last_unstale":"2024-04-25T10:02:09.513776+0000","last_undegraded":"2024-04-25T10:02:09.513776+0000","last_fullsized":"2024-04-25T10:02:09.513776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520621+0000","last_change":"2024-04-25T10:02:04.156786+0000","last_active":"2024-04-25T10:02:09.520621+0000","last_peered":"2024-04-25T10:02:09.520621+0000","last_clean":"2024-04-25T10:02:09.520621+0000","last_became_active":"2024-04-25T10:02:04.156522+0000","last_became_peered":"2024-04-25T10:02:04.156522+0000","last_unstale":"2024-04-25T10:02:09.520621+0000","last_undegraded":"2024-04-25T10:02:09.520621+0000","last_fullsized":"2024-04-25T10:02:09.520621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519839+0000","last_change":"2024-04-25T10:02:08.409422+0000","last_active":"2024-04-25T10:02:09.519839+0000","last_peered":"2024-04-25T10:02:09.519839+0000","last_clean":"2024-04-25T10:02:09.519839+0000","last_became_active":"2024-04-25T10:02:08.408455+0000","last_became_peered":"2024-04-25T10:02:08.408455+0000","last_unstale":"2024-04-25T10:02:09.519839+0000","last_undegraded":"2024-04-25T10:02:09.519839+0000","last_fullsized":"2024-04-25T10:02:09.519839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:32.901672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520971+0000","last_change":"2024-04-25T10:02:03.131043+0000","last_active":"2024-04-25T10:02:09.520971+0000","last_peered":"2024-04-25T10:02:09.520971+0000","last_clean":"2024-04-25T10:02:09.520971+0000","last_became_active":"2024-04-25T10:02:03.130682+0000","last_became_peered":"2024-04-25T10:02:03.130682+0000","last_unstale":"2024-04-25T10:02:09.520971+0000","last_undegraded":"2024-04-25T10:02:09.520971+0000","last_fullsized":"2024-04-25T10:02:09.520971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513286+0000","last_change":"2024-04-25T10:02:04.142528+0000","last_active":"2024-04-25T10:02:09.513286+0000","last_peered":"2024-04-25T10:02:09.513286+0000","last_clean":"2024-04-25T10:02:09.513286+0000","last_became_active":"2024-04-25T10:02:04.142188+0000","last_became_peered":"2024-04-25T10:02:04.142188+0000","last_unstale":"2024-04-25T10:02:09.513286+0000","last_undegraded":"2024-04-25T10:02:09.513286+0000","last_fullsized":"2024-04-25T10:02:09.513286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183112+0000","last_change":"2024-04-25T10:02:08.192140+0000","last_active":"2024-04-25T10:02:09.183112+0000","last_peered":"2024-04-25T10:02:09.183112+0000","last_clean":"2024-04-25T10:02:09.183112+0000","last_became_active":"2024-04-25T10:02:08.191658+0000","last_became_peered":"2024-04-25T10:02:08.191658+0000","last_unstale":"2024-04-25T10:02:09.183112+0000","last_undegraded":"2024-04-25T10:02:09.183112+0000","last_fullsized":"2024-04-25T10:02:09.183112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:45.834272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514852+0000","last_change":"2024-04-25T10:02:03.147520+0000","last_active":"2024-04-25T10:02:09.514852+0000","last_peered":"2024-04-25T10:02:09.514852+0000","last_clean":"2024-04-25T10:02:09.514852+0000","last_became_active":"2024-04-25T10:02:03.147274+0000","last_became_peered":"2024-04-25T10:02:03.147274+0000","last_unstale":"2024-04-25T10:02:09.514852+0000","last_undegraded":"2024-04-25T10:02:09.514852+0000","last_fullsized":"2024-04-25T10:02:09.514852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516102+0000","last_change":"2024-04-25T10:02:04.137351+0000","last_active":"2024-04-25T10:02:09.516102+0000","last_peered":"2024-04-25T10:02:09.516102+0000","last_clean":"2024-04-25T10:02:09.516102+0000","last_became_active":"2024-04-25T10:02:04.136927+0000","last_became_peered":"2024-04-25T10:02:04.136927+0000","last_unstale":"2024-04-25T10:02:09.516102+0000","last_undegraded":"2024-04-25T10:02:09.516102+0000","last_fullsized":"2024-04-25T10:02:09.516102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185918+0000","last_change":"2024-04-25T10:02:08.198068+0000","last_active":"2024-04-25T10:02:09.185918+0000","last_peered":"2024-04-25T10:02:09.185918+0000","last_clean":"2024-04-25T10:02:09.185918+0000","last_became_active":"2024-04-25T10:02:08.196209+0000","last_became_peered":"2024-04-25T10:02:08.196209+0000","last_unstale":"2024-04-25T10:02:09.185918+0000","last_undegraded":"2024-04-25T10:02:09.185918+0000","last_fullsized":"2024-04-25T10:02:09.185918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:48.568415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514318+0000","last_change":"2024-04-25T10:02:03.157531+0000","last_active":"2024-04-25T10:02:09.514318+0000","last_peered":"2024-04-25T10:02:09.514318+0000","last_clean":"2024-04-25T10:02:09.514318+0000","last_became_active":"2024-04-25T10:02:03.157216+0000","last_became_peered":"2024-04-25T10:02:03.157216+0000","last_unstale":"2024-04-25T10:02:09.514318+0000","last_undegraded":"2024-04-25T10:02:09.514318+0000","last_fullsized":"2024-04-25T10:02:09.514318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516612+0000","last_change":"2024-04-25T10:02:04.140082+0000","last_active":"2024-04-25T10:02:09.516612+0000","last_peered":"2024-04-25T10:02:09.516612+0000","last_clean":"2024-04-25T10:02:09.516612+0000","last_became_active":"2024-04-25T10:02:04.136765+0000","last_became_peered":"2024-04-25T10:02:04.136765+0000","last_unstale":"2024-04-25T10:02:09.516612+0000","last_undegraded":"2024-04-25T10:02:09.516612+0000","last_fullsized":"2024-04-25T10:02:09.516612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185629+0000","last_change":"2024-04-25T10:02:08.189048+0000","last_active":"2024-04-25T10:02:09.185629+0000","last_peered":"2024-04-25T10:02:09.185629+0000","last_clean":"2024-04-25T10:02:09.185629+0000","last_became_active":"2024-04-25T10:02:08.188475+0000","last_became_peered":"2024-04-25T10:02:08.188475+0000","last_unstale":"2024-04-25T10:02:09.185629+0000","last_undegraded":"2024-04-25T10:02:09.185629+0000","last_fullsized":"2024-04-25T10:02:09.185629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:08.343719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182619+0000","last_change":"2024-04-25T10:02:04.132020+0000","last_active":"2024-04-25T10:02:09.182619+0000","last_peered":"2024-04-25T10:02:09.182619+0000","last_clean":"2024-04-25T10:02:09.182619+0000","last_became_active":"2024-04-25T10:02:04.131018+0000","last_became_peered":"2024-04-25T10:02:04.131018+0000","last_unstale":"2024-04-25T10:02:09.182619+0000","last_undegraded":"2024-04-25T10:02:09.182619+0000","last_fullsized":"2024-04-25T10:02:09.182619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514507+0000","last_change":"2024-04-25T10:02:03.147595+0000","last_active":"2024-04-25T10:02:09.514507+0000","last_peered":"2024-04-25T10:02:09.514507+0000","last_clean":"2024-04-25T10:02:09.514507+0000","last_became_active":"2024-04-25T10:02:03.147038+0000","last_became_peered":"2024-04-25T10:02:03.147038+0000","last_unstale":"2024-04-25T10:02:09.514507+0000","last_undegraded":"2024-04-25T10:02:09.514507+0000","last_fullsized":"2024-04-25T10:02:09.514507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:50.278310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188671+0000","last_change":"2024-04-25T10:02:08.186691+0000","last_active":"2024-04-25T10:02:09.188671+0000","last_peered":"2024-04-25T10:02:09.188671+0000","last_clean":"2024-04-25T10:02:09.188671+0000","last_became_active":"2024-04-25T10:02:08.186276+0000","last_became_peered":"2024-04-25T10:02:08.186276+0000","last_unstale":"2024-04-25T10:02:09.188671+0000","last_undegraded":"2024-04-25T10:02:09.188671+0000","last_fullsized":"2024-04-25T10:02:09.188671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520343+0000","last_change":"2024-04-25T10:02:04.139116+0000","last_active":"2024-04-25T10:02:09.520343+0000","last_peered":"2024-04-25T10:02:09.520343+0000","last_clean":"2024-04-25T10:02:09.520343+0000","last_became_active":"2024-04-25T10:02:04.138565+0000","last_became_peered":"2024-04-25T10:02:04.138565+0000","last_unstale":"2024-04-25T10:02:09.520343+0000","last_undegraded":"2024-04-25T10:02:09.520343+0000","last_fullsized":"2024-04-25T10:02:09.520343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518432+0000","last_change":"2024-04-25T10:02:03.132968+0000","last_active":"2024-04-25T10:02:09.518432+0000","last_peered":"2024-04-25T10:02:09.518432+0000","last_clean":"2024-04-25T10:02:09.518432+0000","last_became_active":"2024-04-25T10:02:03.132624+0000","last_became_peered":"2024-04-25T10:02:03.132624+0000","last_unstale":"2024-04-25T10:02:09.518432+0000","last_undegraded":"2024-04-25T10:02:09.518432+0000","last_fullsized":"2024-04-25T10:02:09.518432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187996+0000","last_change":"2024-04-25T10:02:08.193709+0000","last_active":"2024-04-25T10:02:09.187996+0000","last_peered":"2024-04-25T10:02:09.187996+0000","last_clean":"2024-04-25T10:02:09.187996+0000","last_became_active":"2024-04-25T10:02:08.193240+0000","last_became_peered":"2024-04-25T10:02:08.193240+0000","last_unstale":"2024-04-25T10:02:09.187996+0000","last_undegraded":"2024-04-25T10:02:09.187996+0000","last_fullsized":"2024-04-25T10:02:09.187996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519675+0000","last_change":"2024-04-25T10:02:04.139077+0000","last_active":"2024-04-25T10:02:09.519675+0000","last_peered":"2024-04-25T10:02:09.519675+0000","last_clean":"2024-04-25T10:02:09.519675+0000","last_became_active":"2024-04-25T10:02:04.138730+0000","last_became_peered":"2024-04-25T10:02:04.138730+0000","last_unstale":"2024-04-25T10:02:09.519675+0000","last_undegraded":"2024-04-25T10:02:09.519675+0000","last_fullsized":"2024-04-25T10:02:09.519675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514609+0000","last_change":"2024-04-25T10:02:03.137563+0000","last_active":"2024-04-25T10:02:09.514609+0000","last_peered":"2024-04-25T10:02:09.514609+0000","last_clean":"2024-04-25T10:02:09.514609+0000","last_became_active":"2024-04-25T10:02:03.137080+0000","last_became_peered":"2024-04-25T10:02:03.137080+0000","last_unstale":"2024-04-25T10:02:09.514609+0000","last_undegraded":"2024-04-25T10:02:09.514609+0000","last_fullsized":"2024-04-25T10:02:09.514609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517371+0000","last_change":"2024-04-25T10:02:08.404747+0000","last_active":"2024-04-25T10:02:09.517371+0000","last_peered":"2024-04-25T10:02:09.517371+0000","last_clean":"2024-04-25T10:02:09.517371+0000","last_became_active":"2024-04-25T10:02:08.403706+0000","last_became_peered":"2024-04-25T10:02:08.403706+0000","last_unstale":"2024-04-25T10:02:09.517371+0000","last_undegraded":"2024-04-25T10:02:09.517371+0000","last_fullsized":"2024-04-25T10:02:09.517371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185839+0000","last_change":"2024-04-25T10:02:04.125617+0000","last_active":"2024-04-25T10:02:09.185839+0000","last_peered":"2024-04-25T10:02:09.185839+0000","last_clean":"2024-04-25T10:02:09.185839+0000","last_became_active":"2024-04-25T10:02:04.125319+0000","last_became_peered":"2024-04-25T10:02:04.125319+0000","last_unstale":"2024-04-25T10:02:09.185839+0000","last_undegraded":"2024-04-25T10:02:09.185839+0000","last_fullsized":"2024-04-25T10:02:09.185839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.271382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512743+0000","last_change":"2024-04-25T10:02:03.145754+0000","last_active":"2024-04-25T10:02:09.512743+0000","last_peered":"2024-04-25T10:02:09.512743+0000","last_clean":"2024-04-25T10:02:09.512743+0000","last_became_active":"2024-04-25T10:02:03.145400+0000","last_became_peered":"2024-04-25T10:02:03.145400+0000","last_unstale":"2024-04-25T10:02:09.512743+0000","last_undegraded":"2024-04-25T10:02:09.512743+0000","last_fullsized":"2024-04-25T10:02:09.512743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188721+0000","last_change":"2024-04-25T10:02:08.197346+0000","last_active":"2024-04-25T10:02:09.188721+0000","last_peered":"2024-04-25T10:02:09.188721+0000","last_clean":"2024-04-25T10:02:09.188721+0000","last_became_active":"2024-04-25T10:02:08.196902+0000","last_became_peered":"2024-04-25T10:02:08.196902+0000","last_unstale":"2024-04-25T10:02:09.188721+0000","last_undegraded":"2024-04-25T10:02:09.188721+0000","last_fullsized":"2024-04-25T10:02:09.188721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:43.992278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515041+0000","last_change":"2024-04-25T10:02:04.123238+0000","last_active":"2024-04-25T10:02:09.515041+0000","last_peered":"2024-04-25T10:02:09.515041+0000","last_clean":"2024-04-25T10:02:09.515041+0000","last_became_active":"2024-04-25T10:02:04.122886+0000","last_became_peered":"2024-04-25T10:02:04.122886+0000","last_unstale":"2024-04-25T10:02:09.515041+0000","last_undegraded":"2024-04-25T10:02:09.515041+0000","last_fullsized":"2024-04-25T10:02:09.515041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187935+0000","last_change":"2024-04-25T10:02:03.138021+0000","last_active":"2024-04-25T10:02:09.187935+0000","last_peered":"2024-04-25T10:02:09.187935+0000","last_clean":"2024-04-25T10:02:09.187935+0000","last_became_active":"2024-04-25T10:02:03.137531+0000","last_became_peered":"2024-04-25T10:02:03.137531+0000","last_unstale":"2024-04-25T10:02:09.187935+0000","last_undegraded":"2024-04-25T10:02:09.187935+0000","last_fullsized":"2024-04-25T10:02:09.187935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185644+0000","last_change":"2024-04-25T10:02:08.404604+0000","last_active":"2024-04-25T10:02:09.185644+0000","last_peered":"2024-04-25T10:02:09.185644+0000","last_clean":"2024-04-25T10:02:09.185644+0000","last_became_active":"2024-04-25T10:02:08.403392+0000","last_became_peered":"2024-04-25T10:02:08.403392+0000","last_unstale":"2024-04-25T10:02:09.185644+0000","last_undegraded":"2024-04-25T10:02:09.185644+0000","last_fullsized":"2024-04-25T10:02:09.185644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:25.532310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521257+0000","last_change":"2024-04-25T10:02:04.144042+0000","last_active":"2024-04-25T10:02:09.521257+0000","last_peered":"2024-04-25T10:02:09.521257+0000","last_clean":"2024-04-25T10:02:09.521257+0000","last_became_active":"2024-04-25T10:02:04.143734+0000","last_became_peered":"2024-04-25T10:02:04.143734+0000","last_unstale":"2024-04-25T10:02:09.521257+0000","last_undegraded":"2024-04-25T10:02:09.521257+0000","last_fullsized":"2024-04-25T10:02:09.521257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516285+0000","last_change":"2024-04-25T10:02:03.138015+0000","last_active":"2024-04-25T10:02:09.516285+0000","last_peered":"2024-04-25T10:02:09.516285+0000","last_clean":"2024-04-25T10:02:09.516285+0000","last_became_active":"2024-04-25T10:02:03.137487+0000","last_became_peered":"2024-04-25T10:02:03.137487+0000","last_unstale":"2024-04-25T10:02:09.516285+0000","last_undegraded":"2024-04-25T10:02:09.516285+0000","last_fullsized":"2024-04-25T10:02:09.516285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515852+0000","last_change":"2024-04-25T10:02:08.402309+0000","last_active":"2024-04-25T10:02:09.515852+0000","last_peered":"2024-04-25T10:02:09.515852+0000","last_clean":"2024-04-25T10:02:09.515852+0000","last_became_active":"2024-04-25T10:02:08.401865+0000","last_became_peered":"2024-04-25T10:02:08.401865+0000","last_unstale":"2024-04-25T10:02:09.515852+0000","last_undegraded":"2024-04-25T10:02:09.515852+0000","last_fullsized":"2024-04-25T10:02:09.515852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519191+0000","last_change":"2024-04-25T10:02:04.129398+0000","last_active":"2024-04-25T10:02:09.519191+0000","last_peered":"2024-04-25T10:02:09.519191+0000","last_clean":"2024-04-25T10:02:09.519191+0000","last_became_active":"2024-04-25T10:02:04.128879+0000","last_became_peered":"2024-04-25T10:02:04.128879+0000","last_unstale":"2024-04-25T10:02:09.519191+0000","last_undegraded":"2024-04-25T10:02:09.519191+0000","last_fullsized":"2024-04-25T10:02:09.519191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515418+0000","last_change":"2024-04-25T10:02:03.146769+0000","last_active":"2024-04-25T10:02:09.515418+0000","last_peered":"2024-04-25T10:02:09.515418+0000","last_clean":"2024-04-25T10:02:09.515418+0000","last_became_active":"2024-04-25T10:02:03.146576+0000","last_became_peered":"2024-04-25T10:02:03.146576+0000","last_unstale":"2024-04-25T10:02:09.515418+0000","last_undegraded":"2024-04-25T10:02:09.515418+0000","last_fullsized":"2024-04-25T10:02:09.515418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513472+0000","last_change":"2024-04-25T10:02:08.393230+0000","last_active":"2024-04-25T10:02:09.513472+0000","last_peered":"2024-04-25T10:02:09.513472+0000","last_clean":"2024-04-25T10:02:09.513472+0000","last_became_active":"2024-04-25T10:02:08.392691+0000","last_became_peered":"2024-04-25T10:02:08.392691+0000","last_unstale":"2024-04-25T10:02:09.513472+0000","last_undegraded":"2024-04-25T10:02:09.513472+0000","last_fullsized":"2024-04-25T10:02:09.513472+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516864+0000","last_change":"2024-04-25T10:02:04.140436+0000","last_active":"2024-04-25T10:02:09.516864+0000","last_peered":"2024-04-25T10:02:09.516864+0000","last_clean":"2024-04-25T10:02:09.516864+0000","last_became_active":"2024-04-25T10:02:04.140075+0000","last_became_peered":"2024-04-25T10:02:04.140075+0000","last_unstale":"2024-04-25T10:02:09.516864+0000","last_undegraded":"2024-04-25T10:02:09.516864+0000","last_fullsized":"2024-04-25T10:02:09.516864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519440+0000","last_change":"2024-04-25T10:02:03.132553+0000","last_active":"2024-04-25T10:02:09.519440+0000","last_peered":"2024-04-25T10:02:09.519440+0000","last_clean":"2024-04-25T10:02:09.519440+0000","last_became_active":"2024-04-25T10:02:03.132241+0000","last_became_peered":"2024-04-25T10:02:03.132241+0000","last_unstale":"2024-04-25T10:02:09.519440+0000","last_undegraded":"2024-04-25T10:02:09.519440+0000","last_fullsized":"2024-04-25T10:02:09.519440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518024+0000","last_change":"2024-04-25T10:02:08.186096+0000","last_active":"2024-04-25T10:02:09.518024+0000","last_peered":"2024-04-25T10:02:09.518024+0000","last_clean":"2024-04-25T10:02:09.518024+0000","last_became_active":"2024-04-25T10:02:08.185660+0000","last_became_peered":"2024-04-25T10:02:08.185660+0000","last_unstale":"2024-04-25T10:02:09.518024+0000","last_undegraded":"2024-04-25T10:02:09.518024+0000","last_fullsized":"2024-04-25T10:02:09.518024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188203+0000","last_change":"2024-04-25T10:02:04.126935+0000","last_active":"2024-04-25T10:02:09.188203+0000","last_peered":"2024-04-25T10:02:09.188203+0000","last_clean":"2024-04-25T10:02:09.188203+0000","last_became_active":"2024-04-25T10:02:04.126268+0000","last_became_peered":"2024-04-25T10:02:04.126268+0000","last_unstale":"2024-04-25T10:02:09.188203+0000","last_undegraded":"2024-04-25T10:02:09.188203+0000","last_fullsized":"2024-04-25T10:02:09.188203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515075+0000","last_change":"2024-04-25T10:02:03.143964+0000","last_active":"2024-04-25T10:02:09.515075+0000","last_peered":"2024-04-25T10:02:09.515075+0000","last_clean":"2024-04-25T10:02:09.515075+0000","last_became_active":"2024-04-25T10:02:03.143548+0000","last_became_peered":"2024-04-25T10:02:03.143548+0000","last_unstale":"2024-04-25T10:02:09.515075+0000","last_undegraded":"2024-04-25T10:02:09.515075+0000","last_fullsized":"2024-04-25T10:02:09.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519212+0000","last_change":"2024-04-25T10:02:08.196544+0000","last_active":"2024-04-25T10:02:09.519212+0000","last_peered":"2024-04-25T10:02:09.519212+0000","last_clean":"2024-04-25T10:02:09.519212+0000","last_became_active":"2024-04-25T10:02:08.196131+0000","last_became_peered":"2024-04-25T10:02:08.196131+0000","last_unstale":"2024-04-25T10:02:09.519212+0000","last_undegraded":"2024-04-25T10:02:09.519212+0000","last_fullsized":"2024-04-25T10:02:09.519212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513146+0000","last_change":"2024-04-25T10:02:04.124338+0000","last_active":"2024-04-25T10:02:09.513146+0000","last_peered":"2024-04-25T10:02:09.513146+0000","last_clean":"2024-04-25T10:02:09.513146+0000","last_became_active":"2024-04-25T10:02:04.123720+0000","last_became_peered":"2024-04-25T10:02:04.123720+0000","last_unstale":"2024-04-25T10:02:09.513146+0000","last_undegraded":"2024-04-25T10:02:09.513146+0000","last_fullsized":"2024-04-25T10:02:09.513146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181759+0000","last_change":"2024-04-25T10:02:03.131111+0000","last_active":"2024-04-25T10:02:09.181759+0000","last_peered":"2024-04-25T10:02:09.181759+0000","last_clean":"2024-04-25T10:02:09.181759+0000","last_became_active":"2024-04-25T10:02:03.130828+0000","last_became_peered":"2024-04-25T10:02:03.130828+0000","last_unstale":"2024-04-25T10:02:09.181759+0000","last_undegraded":"2024-04-25T10:02:09.181759+0000","last_fullsized":"2024-04-25T10:02:09.181759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517914+0000","last_change":"2024-04-25T10:02:08.193216+0000","last_active":"2024-04-25T10:02:09.517914+0000","last_peered":"2024-04-25T10:02:09.517914+0000","last_clean":"2024-04-25T10:02:09.517914+0000","last_became_active":"2024-04-25T10:02:08.192717+0000","last_became_peered":"2024-04-25T10:02:08.192717+0000","last_unstale":"2024-04-25T10:02:09.517914+0000","last_undegraded":"2024-04-25T10:02:09.517914+0000","last_fullsized":"2024-04-25T10:02:09.517914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513507+0000","last_change":"2024-04-25T10:02:04.131883+0000","last_active":"2024-04-25T10:02:09.513507+0000","last_peered":"2024-04-25T10:02:09.513507+0000","last_clean":"2024-04-25T10:02:09.513507+0000","last_became_active":"2024-04-25T10:02:04.131591+0000","last_became_peered":"2024-04-25T10:02:04.131591+0000","last_unstale":"2024-04-25T10:02:09.513507+0000","last_undegraded":"2024-04-25T10:02:09.513507+0000","last_fullsized":"2024-04-25T10:02:09.513507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513031+0000","last_change":"2024-04-25T10:02:03.146201+0000","last_active":"2024-04-25T10:02:09.513031+0000","last_peered":"2024-04-25T10:02:09.513031+0000","last_clean":"2024-04-25T10:02:09.513031+0000","last_became_active":"2024-04-25T10:02:03.145888+0000","last_became_peered":"2024-04-25T10:02:03.145888+0000","last_unstale":"2024-04-25T10:02:09.513031+0000","last_undegraded":"2024-04-25T10:02:09.513031+0000","last_fullsized":"2024-04-25T10:02:09.513031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514736+0000","last_change":"2024-04-25T10:02:08.403944+0000","last_active":"2024-04-25T10:02:09.514736+0000","last_peered":"2024-04-25T10:02:09.514736+0000","last_clean":"2024-04-25T10:02:09.514736+0000","last_became_active":"2024-04-25T10:02:08.403447+0000","last_became_peered":"2024-04-25T10:02:08.403447+0000","last_unstale":"2024-04-25T10:02:09.514736+0000","last_undegraded":"2024-04-25T10:02:09.514736+0000","last_fullsized":"2024-04-25T10:02:09.514736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520189+0000","last_change":"2024-04-25T10:02:04.145181+0000","last_active":"2024-04-25T10:02:09.520189+0000","last_peered":"2024-04-25T10:02:09.520189+0000","last_clean":"2024-04-25T10:02:09.520189+0000","last_became_active":"2024-04-25T10:02:04.145007+0000","last_became_peered":"2024-04-25T10:02:04.145007+0000","last_unstale":"2024-04-25T10:02:09.520189+0000","last_undegraded":"2024-04-25T10:02:09.520189+0000","last_fullsized":"2024-04-25T10:02:09.520189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515574+0000","last_change":"2024-04-25T10:02:03.149716+0000","last_active":"2024-04-25T10:02:09.515574+0000","last_peered":"2024-04-25T10:02:09.515574+0000","last_clean":"2024-04-25T10:02:09.515574+0000","last_became_active":"2024-04-25T10:02:03.149427+0000","last_became_peered":"2024-04-25T10:02:03.149427+0000","last_unstale":"2024-04-25T10:02:09.515574+0000","last_undegraded":"2024-04-25T10:02:09.515574+0000","last_fullsized":"2024-04-25T10:02:09.515574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513340+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:02:09.513340+0000","last_peered":"2024-04-25T10:02:09.513340+0000","last_clean":"2024-04-25T10:02:09.513340+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:02:09.513340+0000","last_undegraded":"2024-04-25T10:02:09.513340+0000","last_fullsized":"2024-04-25T10:02:09.513340+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183489+0000","last_change":"2024-04-25T10:02:08.405095+0000","last_active":"2024-04-25T10:02:09.183489+0000","last_peered":"2024-04-25T10:02:09.183489+0000","last_clean":"2024-04-25T10:02:09.183489+0000","last_became_active":"2024-04-25T10:02:08.403767+0000","last_became_peered":"2024-04-25T10:02:08.403767+0000","last_unstale":"2024-04-25T10:02:09.183489+0000","last_undegraded":"2024-04-25T10:02:09.183489+0000","last_fullsized":"2024-04-25T10:02:09.183489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514594+0000","last_change":"2024-04-25T10:02:04.151648+0000","last_active":"2024-04-25T10:02:09.514594+0000","last_peered":"2024-04-25T10:02:09.514594+0000","last_clean":"2024-04-25T10:02:09.514594+0000","last_became_active":"2024-04-25T10:02:04.150680+0000","last_became_peered":"2024-04-25T10:02:04.150680+0000","last_unstale":"2024-04-25T10:02:09.514594+0000","last_undegraded":"2024-04-25T10:02:09.514594+0000","last_fullsized":"2024-04-25T10:02:09.514594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182505+0000","last_change":"2024-04-25T10:02:03.154627+0000","last_active":"2024-04-25T10:02:09.182505+0000","last_peered":"2024-04-25T10:02:09.182505+0000","last_clean":"2024-04-25T10:02:09.182505+0000","last_became_active":"2024-04-25T10:02:03.154346+0000","last_became_peered":"2024-04-25T10:02:03.154346+0000","last_unstale":"2024-04-25T10:02:09.182505+0000","last_undegraded":"2024-04-25T10:02:09.182505+0000","last_fullsized":"2024-04-25T10:02:09.182505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184009+0000","last_change":"2024-04-25T10:02:08.394382+0000","last_active":"2024-04-25T10:02:09.184009+0000","last_peered":"2024-04-25T10:02:09.184009+0000","last_clean":"2024-04-25T10:02:09.184009+0000","last_became_active":"2024-04-25T10:02:08.393222+0000","last_became_peered":"2024-04-25T10:02:08.393222+0000","last_unstale":"2024-04-25T10:02:09.184009+0000","last_undegraded":"2024-04-25T10:02:09.184009+0000","last_fullsized":"2024-04-25T10:02:09.184009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516304+0000","last_change":"2024-04-25T10:02:04.144851+0000","last_active":"2024-04-25T10:02:09.516304+0000","last_peered":"2024-04-25T10:02:09.516304+0000","last_clean":"2024-04-25T10:02:09.516304+0000","last_became_active":"2024-04-25T10:02:04.144530+0000","last_became_peered":"2024-04-25T10:02:04.144530+0000","last_unstale":"2024-04-25T10:02:09.516304+0000","last_undegraded":"2024-04-25T10:02:09.516304+0000","last_fullsized":"2024-04-25T10:02:09.516304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515591+0000","last_change":"2024-04-25T10:02:03.133061+0000","last_active":"2024-04-25T10:02:09.515591+0000","last_peered":"2024-04-25T10:02:09.515591+0000","last_clean":"2024-04-25T10:02:09.515591+0000","last_became_active":"2024-04-25T10:02:03.132762+0000","last_became_peered":"2024-04-25T10:02:03.132762+0000","last_unstale":"2024-04-25T10:02:09.515591+0000","last_undegraded":"2024-04-25T10:02:09.515591+0000","last_fullsized":"2024-04-25T10:02:09.515591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513927+0000","last_change":"2024-04-25T10:02:08.186615+0000","last_active":"2024-04-25T10:02:09.513927+0000","last_peered":"2024-04-25T10:02:09.513927+0000","last_clean":"2024-04-25T10:02:09.513927+0000","last_became_active":"2024-04-25T10:02:08.186216+0000","last_became_peered":"2024-04-25T10:02:08.186216+0000","last_unstale":"2024-04-25T10:02:09.513927+0000","last_undegraded":"2024-04-25T10:02:09.513927+0000","last_fullsized":"2024-04-25T10:02:09.513927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515930+0000","last_change":"2024-04-25T10:02:04.153131+0000","last_active":"2024-04-25T10:02:09.515930+0000","last_peered":"2024-04-25T10:02:09.515930+0000","last_clean":"2024-04-25T10:02:09.515930+0000","last_became_active":"2024-04-25T10:02:04.152415+0000","last_became_peered":"2024-04-25T10:02:04.152415+0000","last_unstale":"2024-04-25T10:02:09.515930+0000","last_undegraded":"2024-04-25T10:02:09.515930+0000","last_fullsized":"2024-04-25T10:02:09.515930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:54.646358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182568+0000","last_change":"2024-04-25T10:02:03.134495+0000","last_active":"2024-04-25T10:02:09.182568+0000","last_peered":"2024-04-25T10:02:09.182568+0000","last_clean":"2024-04-25T10:02:09.182568+0000","last_became_active":"2024-04-25T10:02:03.134157+0000","last_became_peered":"2024-04-25T10:02:03.134157+0000","last_unstale":"2024-04-25T10:02:09.182568+0000","last_undegraded":"2024-04-25T10:02:09.182568+0000","last_fullsized":"2024-04-25T10:02:09.182568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512930+0000","last_change":"2024-04-25T10:02:08.406412+0000","last_active":"2024-04-25T10:02:09.512930+0000","last_peered":"2024-04-25T10:02:09.512930+0000","last_clean":"2024-04-25T10:02:09.512930+0000","last_became_active":"2024-04-25T10:02:08.402979+0000","last_became_peered":"2024-04-25T10:02:08.402979+0000","last_unstale":"2024-04-25T10:02:09.512930+0000","last_undegraded":"2024-04-25T10:02:09.512930+0000","last_fullsized":"2024-04-25T10:02:09.512930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515530+0000","last_change":"2024-04-25T10:02:04.150503+0000","last_active":"2024-04-25T10:02:09.515530+0000","last_peered":"2024-04-25T10:02:09.515530+0000","last_clean":"2024-04-25T10:02:09.515530+0000","last_became_active":"2024-04-25T10:02:04.150202+0000","last_became_peered":"2024-04-25T10:02:04.150202+0000","last_unstale":"2024-04-25T10:02:09.515530+0000","last_undegraded":"2024-04-25T10:02:09.515530+0000","last_fullsized":"2024-04-25T10:02:09.515530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188973+0000","last_change":"2024-04-25T10:02:03.132517+0000","last_active":"2024-04-25T10:02:09.188973+0000","last_peered":"2024-04-25T10:02:09.188973+0000","last_clean":"2024-04-25T10:02:09.188973+0000","last_became_active":"2024-04-25T10:02:03.132128+0000","last_became_peered":"2024-04-25T10:02:03.132128+0000","last_unstale":"2024-04-25T10:02:09.188973+0000","last_undegraded":"2024-04-25T10:02:09.188973+0000","last_fullsized":"2024-04-25T10:02:09.188973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184870+0000","last_change":"2024-04-25T10:02:08.197632+0000","last_active":"2024-04-25T10:02:09.184870+0000","last_peered":"2024-04-25T10:02:09.184870+0000","last_clean":"2024-04-25T10:02:09.184870+0000","last_became_active":"2024-04-25T10:02:08.196287+0000","last_became_peered":"2024-04-25T10:02:08.196287+0000","last_unstale":"2024-04-25T10:02:09.184870+0000","last_undegraded":"2024-04-25T10:02:09.184870+0000","last_fullsized":"2024-04-25T10:02:09.184870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.959507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513645+0000","last_change":"2024-04-25T10:02:04.148290+0000","last_active":"2024-04-25T10:02:09.513645+0000","last_peered":"2024-04-25T10:02:09.513645+0000","last_clean":"2024-04-25T10:02:09.513645+0000","last_became_active":"2024-04-25T10:02:04.147943+0000","last_became_peered":"2024-04-25T10:02:04.147943+0000","last_unstale":"2024-04-25T10:02:09.513645+0000","last_undegraded":"2024-04-25T10:02:09.513645+0000","last_fullsized":"2024-04-25T10:02:09.513645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181446+0000","last_change":"2024-04-25T10:02:03.125338+0000","last_active":"2024-04-25T10:02:09.181446+0000","last_peered":"2024-04-25T10:02:09.181446+0000","last_clean":"2024-04-25T10:02:09.181446+0000","last_became_active":"2024-04-25T10:02:03.123509+0000","last_became_peered":"2024-04-25T10:02:03.123509+0000","last_unstale":"2024-04-25T10:02:09.181446+0000","last_undegraded":"2024-04-25T10:02:09.181446+0000","last_fullsized":"2024-04-25T10:02:09.181446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514606+0000","last_change":"2024-04-25T10:02:08.181642+0000","last_active":"2024-04-25T10:02:09.514606+0000","last_peered":"2024-04-25T10:02:09.514606+0000","last_clean":"2024-04-25T10:02:09.514606+0000","last_became_active":"2024-04-25T10:02:08.181231+0000","last_became_peered":"2024-04-25T10:02:08.181231+0000","last_unstale":"2024-04-25T10:02:09.514606+0000","last_undegraded":"2024-04-25T10:02:09.514606+0000","last_fullsized":"2024-04-25T10:02:09.514606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182787+0000","last_change":"2024-04-25T10:02:04.125758+0000","last_active":"2024-04-25T10:02:09.182787+0000","last_peered":"2024-04-25T10:02:09.182787+0000","last_clean":"2024-04-25T10:02:09.182787+0000","last_became_active":"2024-04-25T10:02:04.124948+0000","last_became_peered":"2024-04-25T10:02:04.124948+0000","last_unstale":"2024-04-25T10:02:09.182787+0000","last_undegraded":"2024-04-25T10:02:09.182787+0000","last_fullsized":"2024-04-25T10:02:09.182787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184171+0000","last_change":"2024-04-25T10:02:03.136435+0000","last_active":"2024-04-25T10:02:09.184171+0000","last_peered":"2024-04-25T10:02:09.184171+0000","last_clean":"2024-04-25T10:02:09.184171+0000","last_became_active":"2024-04-25T10:02:03.134440+0000","last_became_peered":"2024-04-25T10:02:03.134440+0000","last_unstale":"2024-04-25T10:02:09.184171+0000","last_undegraded":"2024-04-25T10:02:09.184171+0000","last_fullsized":"2024-04-25T10:02:09.184171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520993+0000","last_change":"2024-04-25T10:02:08.393430+0000","last_active":"2024-04-25T10:02:09.520993+0000","last_peered":"2024-04-25T10:02:09.520993+0000","last_clean":"2024-04-25T10:02:09.520993+0000","last_became_active":"2024-04-25T10:02:08.392682+0000","last_became_peered":"2024-04-25T10:02:08.392682+0000","last_unstale":"2024-04-25T10:02:09.520993+0000","last_undegraded":"2024-04-25T10:02:09.520993+0000","last_fullsized":"2024-04-25T10:02:09.520993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.626796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514273+0000","last_change":"2024-04-25T10:02:04.145690+0000","last_active":"2024-04-25T10:02:09.514273+0000","last_peered":"2024-04-25T10:02:09.514273+0000","last_clean":"2024-04-25T10:02:09.514273+0000","last_became_active":"2024-04-25T10:02:04.145414+0000","last_became_peered":"2024-04-25T10:02:04.145414+0000","last_unstale":"2024-04-25T10:02:09.514273+0000","last_undegraded":"2024-04-25T10:02:09.514273+0000","last_fullsized":"2024-04-25T10:02:09.514273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514810+0000","last_change":"2024-04-25T10:02:03.151971+0000","last_active":"2024-04-25T10:02:09.514810+0000","last_peered":"2024-04-25T10:02:09.514810+0000","last_clean":"2024-04-25T10:02:09.514810+0000","last_became_active":"2024-04-25T10:02:03.151708+0000","last_became_peered":"2024-04-25T10:02:03.151708+0000","last_unstale":"2024-04-25T10:02:09.514810+0000","last_undegraded":"2024-04-25T10:02:09.514810+0000","last_fullsized":"2024-04-25T10:02:09.514810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520552+0000","last_change":"2024-04-25T10:02:08.393020+0000","last_active":"2024-04-25T10:02:09.520552+0000","last_peered":"2024-04-25T10:02:09.520552+0000","last_clean":"2024-04-25T10:02:09.520552+0000","last_became_active":"2024-04-25T10:02:08.392543+0000","last_became_peered":"2024-04-25T10:02:08.392543+0000","last_unstale":"2024-04-25T10:02:09.520552+0000","last_undegraded":"2024-04-25T10:02:09.520552+0000","last_fullsized":"2024-04-25T10:02:09.520552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:32.205527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512848+0000","last_change":"2024-04-25T10:02:04.137453+0000","last_active":"2024-04-25T10:02:09.512848+0000","last_peered":"2024-04-25T10:02:09.512848+0000","last_clean":"2024-04-25T10:02:09.512848+0000","last_became_active":"2024-04-25T10:02:04.136751+0000","last_became_peered":"2024-04-25T10:02:04.136751+0000","last_unstale":"2024-04-25T10:02:09.512848+0000","last_undegraded":"2024-04-25T10:02:09.512848+0000","last_fullsized":"2024-04-25T10:02:09.512848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514252+0000","last_change":"2024-04-25T10:02:03.134138+0000","last_active":"2024-04-25T10:02:09.514252+0000","last_peered":"2024-04-25T10:02:09.514252+0000","last_clean":"2024-04-25T10:02:09.514252+0000","last_became_active":"2024-04-25T10:02:03.133331+0000","last_became_peered":"2024-04-25T10:02:03.133331+0000","last_unstale":"2024-04-25T10:02:09.514252+0000","last_undegraded":"2024-04-25T10:02:09.514252+0000","last_fullsized":"2024-04-25T10:02:09.514252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516833+0000","last_change":"2024-04-25T10:02:08.399232+0000","last_active":"2024-04-25T10:02:09.516833+0000","last_peered":"2024-04-25T10:02:09.516833+0000","last_clean":"2024-04-25T10:02:09.516833+0000","last_became_active":"2024-04-25T10:02:08.393743+0000","last_became_peered":"2024-04-25T10:02:08.393743+0000","last_unstale":"2024-04-25T10:02:09.516833+0000","last_undegraded":"2024-04-25T10:02:09.516833+0000","last_fullsized":"2024-04-25T10:02:09.516833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514339+0000","last_change":"2024-04-25T10:02:04.145573+0000","last_active":"2024-04-25T10:02:09.514339+0000","last_peered":"2024-04-25T10:02:09.514339+0000","last_clean":"2024-04-25T10:02:09.514339+0000","last_became_active":"2024-04-25T10:02:04.140591+0000","last_became_peered":"2024-04-25T10:02:04.140591+0000","last_unstale":"2024-04-25T10:02:09.514339+0000","last_undegraded":"2024-04-25T10:02:09.514339+0000","last_fullsized":"2024-04-25T10:02:09.514339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:21.801655+0000","last_change":"2024-04-25T10:02:03.131213+0000","last_active":"2024-04-25T10:02:21.801655+0000","last_peered":"2024-04-25T10:02:21.801655+0000","last_clean":"2024-04-25T10:02:21.801655+0000","last_became_active":"2024-04-25T10:02:03.130876+0000","last_became_peered":"2024-04-25T10:02:03.130876+0000","last_unstale":"2024-04-25T10:02:21.801655+0000","last_undegraded":"2024-04-25T10:02:21.801655+0000","last_fullsized":"2024-04-25T10:02:21.801655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518147+0000","last_change":"2024-04-25T10:02:08.197739+0000","last_active":"2024-04-25T10:02:09.518147+0000","last_peered":"2024-04-25T10:02:09.518147+0000","last_clean":"2024-04-25T10:02:09.518147+0000","last_became_active":"2024-04-25T10:02:08.195924+0000","last_became_peered":"2024-04-25T10:02:08.195924+0000","last_unstale":"2024-04-25T10:02:09.518147+0000","last_undegraded":"2024-04-25T10:02:09.518147+0000","last_fullsized":"2024-04-25T10:02:09.518147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188422+0000","last_change":"2024-04-25T10:02:04.147813+0000","last_active":"2024-04-25T10:02:09.188422+0000","last_peered":"2024-04-25T10:02:09.188422+0000","last_clean":"2024-04-25T10:02:09.188422+0000","last_became_active":"2024-04-25T10:02:04.140229+0000","last_became_peered":"2024-04-25T10:02:04.140229+0000","last_unstale":"2024-04-25T10:02:09.188422+0000","last_undegraded":"2024-04-25T10:02:09.188422+0000","last_fullsized":"2024-04-25T10:02:09.188422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:19.761376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515518+0000","last_change":"2024-04-25T10:02:03.154687+0000","last_active":"2024-04-25T10:02:09.515518+0000","last_peered":"2024-04-25T10:02:09.515518+0000","last_clean":"2024-04-25T10:02:09.515518+0000","last_became_active":"2024-04-25T10:02:03.154323+0000","last_became_peered":"2024-04-25T10:02:03.154323+0000","last_unstale":"2024-04-25T10:02:09.515518+0000","last_undegraded":"2024-04-25T10:02:09.515518+0000","last_fullsized":"2024-04-25T10:02:09.515518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187851+0000","last_change":"2024-04-25T10:02:08.193520+0000","last_active":"2024-04-25T10:02:09.187851+0000","last_peered":"2024-04-25T10:02:09.187851+0000","last_clean":"2024-04-25T10:02:09.187851+0000","last_became_active":"2024-04-25T10:02:08.193040+0000","last_became_peered":"2024-04-25T10:02:08.193040+0000","last_unstale":"2024-04-25T10:02:09.187851+0000","last_undegraded":"2024-04-25T10:02:09.187851+0000","last_fullsized":"2024-04-25T10:02:09.187851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513593+0000","last_change":"2024-04-25T10:02:04.137131+0000","last_active":"2024-04-25T10:02:09.513593+0000","last_peered":"2024-04-25T10:02:09.513593+0000","last_clean":"2024-04-25T10:02:09.513593+0000","last_became_active":"2024-04-25T10:02:04.136832+0000","last_became_peered":"2024-04-25T10:02:04.136832+0000","last_unstale":"2024-04-25T10:02:09.513593+0000","last_undegraded":"2024-04-25T10:02:09.513593+0000","last_fullsized":"2024-04-25T10:02:09.513593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516824+0000","last_change":"2024-04-25T10:02:03.154418+0000","last_active":"2024-04-25T10:02:09.516824+0000","last_peered":"2024-04-25T10:02:09.516824+0000","last_clean":"2024-04-25T10:02:09.516824+0000","last_became_active":"2024-04-25T10:02:03.154147+0000","last_became_peered":"2024-04-25T10:02:03.154147+0000","last_unstale":"2024-04-25T10:02:09.516824+0000","last_undegraded":"2024-04-25T10:02:09.516824+0000","last_fullsized":"2024-04-25T10:02:09.516824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519420+0000","last_change":"2024-04-25T10:02:08.192052+0000","last_active":"2024-04-25T10:02:09.519420+0000","last_peered":"2024-04-25T10:02:09.519420+0000","last_clean":"2024-04-25T10:02:09.519420+0000","last_became_active":"2024-04-25T10:02:08.190628+0000","last_became_peered":"2024-04-25T10:02:08.190628+0000","last_unstale":"2024-04-25T10:02:09.519420+0000","last_undegraded":"2024-04-25T10:02:09.519420+0000","last_fullsized":"2024-04-25T10:02:09.519420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516671+0000","last_change":"2024-04-25T10:02:04.149114+0000","last_active":"2024-04-25T10:02:09.516671+0000","last_peered":"2024-04-25T10:02:09.516671+0000","last_clean":"2024-04-25T10:02:09.516671+0000","last_became_active":"2024-04-25T10:02:04.147309+0000","last_became_peered":"2024-04-25T10:02:04.147309+0000","last_unstale":"2024-04-25T10:02:09.516671+0000","last_undegraded":"2024-04-25T10:02:09.516671+0000","last_fullsized":"2024-04-25T10:02:09.516671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187650+0000","last_change":"2024-04-25T10:02:03.150915+0000","last_active":"2024-04-25T10:02:09.187650+0000","last_peered":"2024-04-25T10:02:09.187650+0000","last_clean":"2024-04-25T10:02:09.187650+0000","last_became_active":"2024-04-25T10:02:03.150382+0000","last_became_peered":"2024-04-25T10:02:03.150382+0000","last_unstale":"2024-04-25T10:02:09.187650+0000","last_undegraded":"2024-04-25T10:02:09.187650+0000","last_fullsized":"2024-04-25T10:02:09.187650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520143+0000","last_change":"2024-04-25T10:02:08.409717+0000","last_active":"2024-04-25T10:02:09.520143+0000","last_peered":"2024-04-25T10:02:09.520143+0000","last_clean":"2024-04-25T10:02:09.520143+0000","last_became_active":"2024-04-25T10:02:08.409153+0000","last_became_peered":"2024-04-25T10:02:08.409153+0000","last_unstale":"2024-04-25T10:02:09.520143+0000","last_undegraded":"2024-04-25T10:02:09.520143+0000","last_fullsized":"2024-04-25T10:02:09.520143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:59.321979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514307+0000","last_change":"2024-04-25T10:02:08.402841+0000","last_active":"2024-04-25T10:02:09.514307+0000","last_peered":"2024-04-25T10:02:09.514307+0000","last_clean":"2024-04-25T10:02:09.514307+0000","last_became_active":"2024-04-25T10:02:08.402205+0000","last_became_peered":"2024-04-25T10:02:08.402205+0000","last_unstale":"2024-04-25T10:02:09.514307+0000","last_undegraded":"2024-04-25T10:02:09.514307+0000","last_fullsized":"2024-04-25T10:02:09.514307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514163+0000","last_change":"2024-04-25T10:02:04.136408+0000","last_active":"2024-04-25T10:02:09.514163+0000","last_peered":"2024-04-25T10:02:09.514163+0000","last_clean":"2024-04-25T10:02:09.514163+0000","last_became_active":"2024-04-25T10:02:04.136094+0000","last_became_peered":"2024-04-25T10:02:04.136094+0000","last_unstale":"2024-04-25T10:02:09.514163+0000","last_undegraded":"2024-04-25T10:02:09.514163+0000","last_fullsized":"2024-04-25T10:02:09.514163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513538+0000","last_change":"2024-04-25T10:02:03.154541+0000","last_active":"2024-04-25T10:02:09.513538+0000","last_peered":"2024-04-25T10:02:09.513538+0000","last_clean":"2024-04-25T10:02:09.513538+0000","last_became_active":"2024-04-25T10:02:03.154193+0000","last_became_peered":"2024-04-25T10:02:03.154193+0000","last_unstale":"2024-04-25T10:02:09.513538+0000","last_undegraded":"2024-04-25T10:02:09.513538+0000","last_fullsized":"2024-04-25T10:02:09.513538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184564+0000","last_change":"2024-04-25T10:02:08.192506+0000","last_active":"2024-04-25T10:02:09.184564+0000","last_peered":"2024-04-25T10:02:09.184564+0000","last_clean":"2024-04-25T10:02:09.184564+0000","last_became_active":"2024-04-25T10:02:08.190910+0000","last_became_peered":"2024-04-25T10:02:08.190910+0000","last_unstale":"2024-04-25T10:02:09.184564+0000","last_undegraded":"2024-04-25T10:02:09.184564+0000","last_fullsized":"2024-04-25T10:02:09.184564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519465+0000","last_change":"2024-04-25T10:02:04.138218+0000","last_active":"2024-04-25T10:02:09.519465+0000","last_peered":"2024-04-25T10:02:09.519465+0000","last_clean":"2024-04-25T10:02:09.519465+0000","last_became_active":"2024-04-25T10:02:04.137919+0000","last_became_peered":"2024-04-25T10:02:04.137919+0000","last_unstale":"2024-04-25T10:02:09.519465+0000","last_undegraded":"2024-04-25T10:02:09.519465+0000","last_fullsized":"2024-04-25T10:02:09.519465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520192+0000","last_change":"2024-04-25T10:02:03.133971+0000","last_active":"2024-04-25T10:02:09.520192+0000","last_peered":"2024-04-25T10:02:09.520192+0000","last_clean":"2024-04-25T10:02:09.520192+0000","last_became_active":"2024-04-25T10:02:03.133661+0000","last_became_peered":"2024-04-25T10:02:03.133661+0000","last_unstale":"2024-04-25T10:02:09.520192+0000","last_undegraded":"2024-04-25T10:02:09.520192+0000","last_fullsized":"2024-04-25T10:02:09.520192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183101+0000","last_change":"2024-04-25T10:02:03.137884+0000","last_active":"2024-04-25T10:02:09.183101+0000","last_peered":"2024-04-25T10:02:09.183101+0000","last_clean":"2024-04-25T10:02:09.183101+0000","last_became_active":"2024-04-25T10:02:03.137196+0000","last_became_peered":"2024-04-25T10:02:03.137196+0000","last_unstale":"2024-04-25T10:02:09.183101+0000","last_undegraded":"2024-04-25T10:02:09.183101+0000","last_fullsized":"2024-04-25T10:02:09.183101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514125+0000","last_change":"2024-04-25T10:02:04.130491+0000","last_active":"2024-04-25T10:02:09.514125+0000","last_peered":"2024-04-25T10:02:09.514125+0000","last_clean":"2024-04-25T10:02:09.514125+0000","last_became_active":"2024-04-25T10:02:04.129963+0000","last_became_peered":"2024-04-25T10:02:04.129963+0000","last_unstale":"2024-04-25T10:02:09.514125+0000","last_undegraded":"2024-04-25T10:02:09.514125+0000","last_fullsized":"2024-04-25T10:02:09.514125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182210+0000","last_change":"2024-04-25T10:02:08.186410+0000","last_active":"2024-04-25T10:02:09.182210+0000","last_peered":"2024-04-25T10:02:09.182210+0000","last_clean":"2024-04-25T10:02:09.182210+0000","last_became_active":"2024-04-25T10:02:08.185964+0000","last_became_peered":"2024-04-25T10:02:08.185964+0000","last_unstale":"2024-04-25T10:02:09.182210+0000","last_undegraded":"2024-04-25T10:02:09.182210+0000","last_fullsized":"2024-04-25T10:02:09.182210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:17.471287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520398+0000","last_change":"2024-04-25T10:02:03.134295+0000","last_active":"2024-04-25T10:02:09.520398+0000","last_peered":"2024-04-25T10:02:09.520398+0000","last_clean":"2024-04-25T10:02:09.520398+0000","last_became_active":"2024-04-25T10:02:03.134033+0000","last_became_peered":"2024-04-25T10:02:03.134033+0000","last_unstale":"2024-04-25T10:02:09.520398+0000","last_undegraded":"2024-04-25T10:02:09.520398+0000","last_fullsized":"2024-04-25T10:02:09.520398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.186180+0000","last_change":"2024-04-25T10:02:04.137055+0000","last_active":"2024-04-25T10:02:09.186180+0000","last_peered":"2024-04-25T10:02:09.186180+0000","last_clean":"2024-04-25T10:02:09.186180+0000","last_became_active":"2024-04-25T10:02:04.131629+0000","last_became_peered":"2024-04-25T10:02:04.131629+0000","last_unstale":"2024-04-25T10:02:09.186180+0000","last_undegraded":"2024-04-25T10:02:09.186180+0000","last_fullsized":"2024-04-25T10:02:09.186180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521270+0000","last_change":"2024-04-25T10:02:08.187719+0000","last_active":"2024-04-25T10:02:09.521270+0000","last_peered":"2024-04-25T10:02:09.521270+0000","last_clean":"2024-04-25T10:02:09.521270+0000","last_became_active":"2024-04-25T10:02:08.187298+0000","last_became_peered":"2024-04-25T10:02:08.187298+0000","last_unstale":"2024-04-25T10:02:09.521270+0000","last_undegraded":"2024-04-25T10:02:09.521270+0000","last_fullsized":"2024-04-25T10:02:09.521270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:15.995769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182212+0000","last_change":"2024-04-25T10:02:04.125579+0000","last_active":"2024-04-25T10:02:09.182212+0000","last_peered":"2024-04-25T10:02:09.182212+0000","last_clean":"2024-04-25T10:02:09.182212+0000","last_became_active":"2024-04-25T10:02:04.124972+0000","last_became_peered":"2024-04-25T10:02:04.124972+0000","last_unstale":"2024-04-25T10:02:09.182212+0000","last_undegraded":"2024-04-25T10:02:09.182212+0000","last_fullsized":"2024-04-25T10:02:09.182212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181677+0000","last_change":"2024-04-25T10:02:04.145556+0000","last_active":"2024-04-25T10:02:09.181677+0000","last_peered":"2024-04-25T10:02:09.181677+0000","last_clean":"2024-04-25T10:02:09.181677+0000","last_became_active":"2024-04-25T10:02:04.144887+0000","last_became_peered":"2024-04-25T10:02:04.144887+0000","last_unstale":"2024-04-25T10:02:09.181677+0000","last_undegraded":"2024-04-25T10:02:09.181677+0000","last_fullsized":"2024-04-25T10:02:09.181677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514803+0000","last_change":"2024-04-25T10:02:04.142127+0000","last_active":"2024-04-25T10:02:09.514803+0000","last_peered":"2024-04-25T10:02:09.514803+0000","last_clean":"2024-04-25T10:02:09.514803+0000","last_became_active":"2024-04-25T10:02:04.140799+0000","last_became_peered":"2024-04-25T10:02:04.140799+0000","last_unstale":"2024-04-25T10:02:09.514803+0000","last_undegraded":"2024-04-25T10:02:09.514803+0000","last_fullsized":"2024-04-25T10:02:09.514803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514188+0000","last_change":"2024-04-25T10:02:04.141010+0000","last_active":"2024-04-25T10:02:09.514188+0000","last_peered":"2024-04-25T10:02:09.514188+0000","last_clean":"2024-04-25T10:02:09.514188+0000","last_became_active":"2024-04-25T10:02:04.140201+0000","last_became_peered":"2024-04-25T10:02:04.140201+0000","last_unstale":"2024-04-25T10:02:09.514188+0000","last_undegraded":"2024-04-25T10:02:09.514188+0000","last_fullsized":"2024-04-25T10:02:09.514188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521086+0000","last_change":"2024-04-25T10:02:04.137613+0000","last_active":"2024-04-25T10:02:09.521086+0000","last_peered":"2024-04-25T10:02:09.521086+0000","last_clean":"2024-04-25T10:02:09.521086+0000","last_became_active":"2024-04-25T10:02:04.136833+0000","last_became_peered":"2024-04-25T10:02:04.136833+0000","last_unstale":"2024-04-25T10:02:09.521086+0000","last_undegraded":"2024-04-25T10:02:09.521086+0000","last_fullsized":"2024-04-25T10:02:09.521086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514990+0000","last_change":"2024-04-25T10:02:04.135708+0000","last_active":"2024-04-25T10:02:09.514990+0000","last_peered":"2024-04-25T10:02:09.514990+0000","last_clean":"2024-04-25T10:02:09.514990+0000","last_became_active":"2024-04-25T10:02:04.135383+0000","last_became_peered":"2024-04-25T10:02:04.135383+0000","last_unstale":"2024-04-25T10:02:09.514990+0000","last_undegraded":"2024-04-25T10:02:09.514990+0000","last_fullsized":"2024-04-25T10:02:09.514990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187243+0000","last_change":"2024-04-25T10:02:04.151491+0000","last_active":"2024-04-25T10:02:09.187243+0000","last_peered":"2024-04-25T10:02:09.187243+0000","last_clean":"2024-04-25T10:02:09.187243+0000","last_became_active":"2024-04-25T10:02:04.151181+0000","last_became_peered":"2024-04-25T10:02:04.151181+0000","last_unstale":"2024-04-25T10:02:09.187243+0000","last_undegraded":"2024-04-25T10:02:09.187243+0000","last_fullsized":"2024-04-25T10:02:09.187243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521012+0000","last_change":"2024-04-25T10:02:04.133889+0000","last_active":"2024-04-25T10:02:09.521012+0000","last_peered":"2024-04-25T10:02:09.521012+0000","last_clean":"2024-04-25T10:02:09.521012+0000","last_became_active":"2024-04-25T10:02:04.132937+0000","last_became_peered":"2024-04-25T10:02:04.132937+0000","last_unstale":"2024-04-25T10:02:09.521012+0000","last_undegraded":"2024-04-25T10:02:09.521012+0000","last_fullsized":"2024-04-25T10:02:09.521012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513229+0000","last_change":"2024-04-25T10:02:04.142217+0000","last_active":"2024-04-25T10:02:09.513229+0000","last_peered":"2024-04-25T10:02:09.513229+0000","last_clean":"2024-04-25T10:02:09.513229+0000","last_became_active":"2024-04-25T10:02:04.141431+0000","last_became_peered":"2024-04-25T10:02:04.141431+0000","last_unstale":"2024-04-25T10:02:09.513229+0000","last_undegraded":"2024-04-25T10:02:09.513229+0000","last_fullsized":"2024-04-25T10:02:09.513229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520025+0000","last_change":"2024-04-25T10:02:04.148470+0000","last_active":"2024-04-25T10:02:09.520025+0000","last_peered":"2024-04-25T10:02:09.520025+0000","last_clean":"2024-04-25T10:02:09.520025+0000","last_became_active":"2024-04-25T10:02:04.147993+0000","last_became_peered":"2024-04-25T10:02:04.147993+0000","last_unstale":"2024-04-25T10:02:09.520025+0000","last_undegraded":"2024-04-25T10:02:09.520025+0000","last_fullsized":"2024-04-25T10:02:09.520025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513946+0000","last_change":"2024-04-25T10:02:04.148427+0000","last_active":"2024-04-25T10:02:09.513946+0000","last_peered":"2024-04-25T10:02:09.513946+0000","last_clean":"2024-04-25T10:02:09.513946+0000","last_became_active":"2024-04-25T10:02:04.147301+0000","last_became_peered":"2024-04-25T10:02:04.147301+0000","last_unstale":"2024-04-25T10:02:09.513946+0000","last_undegraded":"2024-04-25T10:02:09.513946+0000","last_fullsized":"2024-04-25T10:02:09.513946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513726+0000","last_change":"2024-04-25T10:02:04.136324+0000","last_active":"2024-04-25T10:02:09.513726+0000","last_peered":"2024-04-25T10:02:09.513726+0000","last_clean":"2024-04-25T10:02:09.513726+0000","last_became_active":"2024-04-25T10:02:04.136035+0000","last_became_peered":"2024-04-25T10:02:04.136035+0000","last_unstale":"2024-04-25T10:02:09.513726+0000","last_undegraded":"2024-04-25T10:02:09.513726+0000","last_fullsized":"2024-04-25T10:02:09.513726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184020+0000","last_change":"2024-04-25T10:02:04.134441+0000","last_active":"2024-04-25T10:02:09.184020+0000","last_peered":"2024-04-25T10:02:09.184020+0000","last_clean":"2024-04-25T10:02:09.184020+0000","last_became_active":"2024-04-25T10:02:04.130052+0000","last_became_peered":"2024-04-25T10:02:04.130052+0000","last_unstale":"2024-04-25T10:02:09.184020+0000","last_undegraded":"2024-04-25T10:02:09.184020+0000","last_fullsized":"2024-04-25T10:02:09.184020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515111+0000","last_change":"2024-04-25T10:02:04.146376+0000","last_active":"2024-04-25T10:02:09.515111+0000","last_peered":"2024-04-25T10:02:09.515111+0000","last_clean":"2024-04-25T10:02:09.515111+0000","last_became_active":"2024-04-25T10:02:04.146080+0000","last_became_peered":"2024-04-25T10:02:04.146080+0000","last_unstale":"2024-04-25T10:02:09.515111+0000","last_undegraded":"2024-04-25T10:02:09.515111+0000","last_fullsized":"2024-04-25T10:02:09.515111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182337+0000","last_change":"2024-04-25T10:02:04.135427+0000","last_active":"2024-04-25T10:02:09.182337+0000","last_peered":"2024-04-25T10:02:09.182337+0000","last_clean":"2024-04-25T10:02:09.182337+0000","last_became_active":"2024-04-25T10:02:04.134990+0000","last_became_peered":"2024-04-25T10:02:04.134990+0000","last_unstale":"2024-04-25T10:02:09.182337+0000","last_undegraded":"2024-04-25T10:02:09.182337+0000","last_fullsized":"2024-04-25T10:02:09.182337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517176+0000","last_change":"2024-04-25T10:02:04.140449+0000","last_active":"2024-04-25T10:02:09.517176+0000","last_peered":"2024-04-25T10:02:09.517176+0000","last_clean":"2024-04-25T10:02:09.517176+0000","last_became_active":"2024-04-25T10:02:04.140129+0000","last_became_peered":"2024-04-25T10:02:04.140129+0000","last_unstale":"2024-04-25T10:02:09.517176+0000","last_undegraded":"2024-04-25T10:02:09.517176+0000","last_fullsized":"2024-04-25T10:02:09.517176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513015+0000","last_change":"2024-04-25T10:02:04.137099+0000","last_active":"2024-04-25T10:02:09.513015+0000","last_peered":"2024-04-25T10:02:09.513015+0000","last_clean":"2024-04-25T10:02:09.513015+0000","last_became_active":"2024-04-25T10:02:04.136845+0000","last_became_peered":"2024-04-25T10:02:04.136845+0000","last_unstale":"2024-04-25T10:02:09.513015+0000","last_undegraded":"2024-04-25T10:02:09.513015+0000","last_fullsized":"2024-04-25T10:02:09.513015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514467+0000","last_change":"2024-04-25T10:02:04.133727+0000","last_active":"2024-04-25T10:02:09.514467+0000","last_peered":"2024-04-25T10:02:09.514467+0000","last_clean":"2024-04-25T10:02:09.514467+0000","last_became_active":"2024-04-25T10:02:04.132995+0000","last_became_peered":"2024-04-25T10:02:04.132995+0000","last_unstale":"2024-04-25T10:02:09.514467+0000","last_undegraded":"2024-04-25T10:02:09.514467+0000","last_fullsized":"2024-04-25T10:02:09.514467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512646+0000","last_change":"2024-04-25T10:02:04.128013+0000","last_active":"2024-04-25T10:02:09.512646+0000","last_peered":"2024-04-25T10:02:09.512646+0000","last_clean":"2024-04-25T10:02:09.512646+0000","last_became_active":"2024-04-25T10:02:04.123162+0000","last_became_peered":"2024-04-25T10:02:04.123162+0000","last_unstale":"2024-04-25T10:02:09.512646+0000","last_undegraded":"2024-04-25T10:02:09.512646+0000","last_fullsized":"2024-04-25T10:02:09.512646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:26.232712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520128+0000","last_change":"2024-04-25T10:02:04.137865+0000","last_active":"2024-04-25T10:02:09.520128+0000","last_peered":"2024-04-25T10:02:09.520128+0000","last_clean":"2024-04-25T10:02:09.520128+0000","last_became_active":"2024-04-25T10:02:04.137413+0000","last_became_peered":"2024-04-25T10:02:04.137413+0000","last_unstale":"2024-04-25T10:02:09.520128+0000","last_undegraded":"2024-04-25T10:02:09.520128+0000","last_fullsized":"2024-04-25T10:02:09.520128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188286+0000","last_change":"2024-04-25T10:02:04.148039+0000","last_active":"2024-04-25T10:02:09.188286+0000","last_peered":"2024-04-25T10:02:09.188286+0000","last_clean":"2024-04-25T10:02:09.188286+0000","last_became_active":"2024-04-25T10:02:04.146104+0000","last_became_peered":"2024-04-25T10:02:04.146104+0000","last_unstale":"2024-04-25T10:02:09.188286+0000","last_undegraded":"2024-04-25T10:02:09.188286+0000","last_fullsized":"2024-04-25T10:02:09.188286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:15.812709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183940+0000","last_change":"2024-04-25T10:02:04.149357+0000","last_active":"2024-04-25T10:02:09.183940+0000","last_peered":"2024-04-25T10:02:09.183940+0000","last_clean":"2024-04-25T10:02:09.183940+0000","last_became_active":"2024-04-25T10:02:04.148994+0000","last_became_peered":"2024-04-25T10:02:04.148994+0000","last_unstale":"2024-04-25T10:02:09.183940+0000","last_undegraded":"2024-04-25T10:02:09.183940+0000","last_fullsized":"2024-04-25T10:02:09.183940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187107+0000","last_change":"2024-04-25T10:02:04.144119+0000","last_active":"2024-04-25T10:02:09.187107+0000","last_peered":"2024-04-25T10:02:09.187107+0000","last_clean":"2024-04-25T10:02:09.187107+0000","last_became_active":"2024-04-25T10:02:04.143818+0000","last_became_peered":"2024-04-25T10:02:04.143818+0000","last_unstale":"2024-04-25T10:02:09.187107+0000","last_undegraded":"2024-04-25T10:02:09.187107+0000","last_fullsized":"2024-04-25T10:02:09.187107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515823+0000","last_change":"2024-04-25T10:02:04.152842+0000","last_active":"2024-04-25T10:02:09.515823+0000","last_peered":"2024-04-25T10:02:09.515823+0000","last_clean":"2024-04-25T10:02:09.515823+0000","last_became_active":"2024-04-25T10:02:04.142957+0000","last_became_peered":"2024-04-25T10:02:04.142957+0000","last_unstale":"2024-04-25T10:02:09.515823+0000","last_undegraded":"2024-04-25T10:02:09.515823+0000","last_fullsized":"2024-04-25T10:02:09.515823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519170+0000","last_change":"2024-04-25T10:02:04.143950+0000","last_active":"2024-04-25T10:02:09.519170+0000","last_peered":"2024-04-25T10:02:09.519170+0000","last_clean":"2024-04-25T10:02:09.519170+0000","last_became_active":"2024-04-25T10:02:04.143615+0000","last_became_peered":"2024-04-25T10:02:04.143615+0000","last_unstale":"2024-04-25T10:02:09.519170+0000","last_undegraded":"2024-04-25T10:02:09.519170+0000","last_fullsized":"2024-04-25T10:02:09.519170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513367+0000","last_change":"2024-04-25T10:02:04.140815+0000","last_active":"2024-04-25T10:02:09.513367+0000","last_peered":"2024-04-25T10:02:09.513367+0000","last_clean":"2024-04-25T10:02:09.513367+0000","last_became_active":"2024-04-25T10:02:04.140458+0000","last_became_peered":"2024-04-25T10:02:04.140458+0000","last_unstale":"2024-04-25T10:02:09.513367+0000","last_undegraded":"2024-04-25T10:02:09.513367+0000","last_fullsized":"2024-04-25T10:02:09.513367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513577+0000","last_change":"2024-04-25T10:02:04.144469+0000","last_active":"2024-04-25T10:02:09.513577+0000","last_peered":"2024-04-25T10:02:09.513577+0000","last_clean":"2024-04-25T10:02:09.513577+0000","last_became_active":"2024-04-25T10:02:04.143382+0000","last_became_peered":"2024-04-25T10:02:04.143382+0000","last_unstale":"2024-04-25T10:02:09.513577+0000","last_undegraded":"2024-04-25T10:02:09.513577+0000","last_fullsized":"2024-04-25T10:02:09.513577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514018+0000","last_change":"2024-04-25T10:02:04.138676+0000","last_active":"2024-04-25T10:02:09.514018+0000","last_peered":"2024-04-25T10:02:09.514018+0000","last_clean":"2024-04-25T10:02:09.514018+0000","last_became_active":"2024-04-25T10:02:04.138397+0000","last_became_peered":"2024-04-25T10:02:04.138397+0000","last_unstale":"2024-04-25T10:02:09.514018+0000","last_undegraded":"2024-04-25T10:02:09.514018+0000","last_fullsized":"2024-04-25T10:02:09.514018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516016+0000","last_change":"2024-04-25T10:02:04.145097+0000","last_active":"2024-04-25T10:02:09.516016+0000","last_peered":"2024-04-25T10:02:09.516016+0000","last_clean":"2024-04-25T10:02:09.516016+0000","last_became_active":"2024-04-25T10:02:04.142128+0000","last_became_peered":"2024-04-25T10:02:04.142128+0000","last_unstale":"2024-04-25T10:02:09.516016+0000","last_undegraded":"2024-04-25T10:02:09.516016+0000","last_fullsized":"2024-04-25T10:02:09.516016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513292+0000","last_change":"2024-04-25T10:02:04.124760+0000","last_active":"2024-04-25T10:02:09.513292+0000","last_peered":"2024-04-25T10:02:09.513292+0000","last_clean":"2024-04-25T10:02:09.513292+0000","last_became_active":"2024-04-25T10:02:04.124423+0000","last_became_peered":"2024-04-25T10:02:04.124423+0000","last_unstale":"2024-04-25T10:02:09.513292+0000","last_undegraded":"2024-04-25T10:02:09.513292+0000","last_fullsized":"2024-04-25T10:02:09.513292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515984+0000","last_change":"2024-04-25T10:02:04.129218+0000","last_active":"2024-04-25T10:02:09.515984+0000","last_peered":"2024-04-25T10:02:09.515984+0000","last_clean":"2024-04-25T10:02:09.515984+0000","last_became_active":"2024-04-25T10:02:04.126046+0000","last_became_peered":"2024-04-25T10:02:04.126046+0000","last_unstale":"2024-04-25T10:02:09.515984+0000","last_undegraded":"2024-04-25T10:02:09.515984+0000","last_fullsized":"2024-04-25T10:02:09.515984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515345+0000","last_change":"2024-04-25T10:02:04.137569+0000","last_active":"2024-04-25T10:02:09.515345+0000","last_peered":"2024-04-25T10:02:09.515345+0000","last_clean":"2024-04-25T10:02:09.515345+0000","last_became_active":"2024-04-25T10:02:04.137205+0000","last_became_peered":"2024-04-25T10:02:04.137205+0000","last_unstale":"2024-04-25T10:02:09.515345+0000","last_undegraded":"2024-04-25T10:02:09.515345+0000","last_fullsized":"2024-04-25T10:02:09.515345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:27.138 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:02:27.138 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-25T10:02:27.138 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-25T10:02:27.138 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-25T10:02:27.139 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-25T10:02:27.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.368+0000 7f0d7b200640 1 Processor -- start 2024-04-25T10:02:27.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.369+0000 7f0d7b200640 1 -- start start 2024-04-25T10:02:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.370+0000 7f0d7b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d7c071700 0x7f0d7c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.370+0000 7f0d7b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c15bf30 con 0x7f0d7c0cd860 2024-04-25T10:02:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.370+0000 7f0d7b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d7c15c210 con 0x7f0d7c144440 2024-04-25T10:02:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.370+0000 7f0d7b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d7c15c860 con 0x7f0d7c071700 2024-04-25T10:02:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d71400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d7c071700 0x7f0d7c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d7a800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0d7c0cd860 0x7f0d7c15ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:57994/0 (socket says 172.21.15.165:57994) 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d7a800640 1 -- 172.21.15.165:0/2734239863 learned_addr learned my addr 172.21.15.165:0/2734239863 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d71400640 1 -- 172.21.15.165:0/2734239863 >> v1:172.21.15.172:6789/0 conn(0x7f0d7c144440 legacy=0x7f0d7c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d71400640 1 -- 172.21.15.165:0/2734239863 >> v1:172.21.15.165:6789/0 conn(0x7f0d7c0cd860 legacy=0x7f0d7c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d71400640 1 -- 172.21.15.165:0/2734239863 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c15c4f0 con 0x7f0d7c071700 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.371+0000 7f0d79400640 1 -- 172.21.15.165:0/2734239863 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 875695533 0 0) 0x7f0d7c15bf30 con 0x7f0d7c0cd860 2024-04-25T10:02:27.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.372+0000 7f0d71400640 1 --2- 172.21.15.165:0/2734239863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c071700 0x7f0d7c0cd320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0d640097a0 tx=0x7f0d6402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42132c73694afc9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.373+0000 7f0d79400640 1 -- 172.21.15.165:0/2734239863 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64037070 con 0x7f0d7c071700 2024-04-25T10:02:27.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.373+0000 7f0d79400640 1 -- 172.21.15.165:0/2734239863 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d64032640 con 0x7f0d7c071700 2024-04-25T10:02:27.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.373+0000 7f0d79400640 1 -- 172.21.15.165:0/2734239863 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64032a10 con 0x7f0d7c071700 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.373+0000 7f0d7b200640 1 -- 172.21.15.165:0/2734239863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c071700 msgr2=0x7f0d7c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.373+0000 7f0d7b200640 1 --2- 172.21.15.165:0/2734239863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c071700 0x7f0d7c0cd320 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0d640097a0 tx=0x7f0d6402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.374+0000 7f0d7b200640 1 -- 172.21.15.165:0/2734239863 shutdown_connections 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.374+0000 7f0d7b200640 1 --2- 172.21.15.165:0/2734239863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c071700 0x7f0d7c0cd320 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.374+0000 7f0d7b200640 1 -- 172.21.15.165:0/2734239863 >> 172.21.15.165:0/2734239863 conn(0x7f0d7c06abe0 msgr2=0x7f0d7c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.374+0000 7f0d7b200640 1 -- 172.21.15.165:0/2734239863 shutdown_connections 2024-04-25T10:02:27.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.374+0000 7f0d7b200640 1 -- 172.21.15.165:0/2734239863 wait complete. 2024-04-25T10:02:27.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.375+0000 7f0d7b200640 1 Processor -- start 2024-04-25T10:02:27.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.375+0000 7f0d7b200640 1 -- start start 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.376+0000 7f0d7b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c071700 0x7f0d7c158ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.376+0000 7f0d7b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c0cd860 0x7f0d7c0b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.376+0000 7f0d7b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 0x7f0d7c13b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.376+0000 7f0d7b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0cde70 con 0x7f0d7c144440 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.376+0000 7f0d7b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c0cdcf0 con 0x7f0d7c0cd860 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.376+0000 7f0d7b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7c15d400 con 0x7f0d7c071700 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 0x7f0d7c13b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 0x7f0d7c13b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50282/0 (socket says 172.21.15.165:50282) 2024-04-25T10:02:27.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 -- 172.21.15.165:0/277901281 learned_addr learned my addr 172.21.15.165:0/277901281 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d71400640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c0cd860 0x7f0d7c0b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 -- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c071700 msgr2=0x7f0d7c158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d79e00640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c071700 0x7f0d7c158ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c071700 0x7f0d7c158ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 -- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c0cd860 msgr2=0x7f0d7c0b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c0cd860 0x7f0d7c0b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.377+0000 7f0d7a800640 1 -- 172.21.15.165:0/277901281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0ca880 con 0x7f0d7c144440 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.378+0000 7f0d7a800640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 0x7f0d7c13b490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c7a0 tx=0x7f0d7400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.378+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d74018070 con 0x7f0d7c144440 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.378+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7400a640 con 0x7f0d7c144440 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.378+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7c139be0 con 0x7f0d7c144440 2024-04-25T10:02:27.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.378+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7400aa70 con 0x7f0d7c144440 2024-04-25T10:02:27.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.378+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7c0cadd0 con 0x7f0d7c144440 2024-04-25T10:02:27.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.379+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7c073990 con 0x7f0d7c144440 2024-04-25T10:02:27.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.380+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d7401b070 con 0x7f0d7c144440 2024-04-25T10:02:27.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.381+0000 7f0d72a00640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5806b4d0 0x7f0d5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.381+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0d74089b90 con 0x7f0d7c144440 2024-04-25T10:02:27.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.381+0000 7f0d79e00640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5806b4d0 0x7f0d5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.382+0000 7f0d79e00640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5806b4d0 0x7f0d5806d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0ca740 tx=0x7f0d6c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.386+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d74058630 con 0x7f0d7c144440 2024-04-25T10:02:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.633+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0d7c1427d0 con 0x7f0d7c144440 2024-04-25T10:02:27.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.643+0000 7f0d72a00640 1 -- 172.21.15.165:0/277901281 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+171791 (secure 0 0 0) 0x7f0d74057f30 con 0x7f0d7c144440 2024-04-25T10:02:27.651 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5806b4d0 msgr2=0x7f0d5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5806b4d0 0x7f0d5806d990 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c0ca740 tx=0x7f0d6c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 msgr2=0x7f0d7c13b490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 0x7f0d7c13b490 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0d7400c7a0 tx=0x7f0d7400cc60 comp rx=0 tx=0).stop 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 shutdown_connections 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d5806b4d0 0x7f0d5806d990 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d7c144440 0x7f0d7c13b490 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d7c0cd860 0x7f0d7c0b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 --2- 172.21.15.165:0/277901281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d7c071700 0x7f0d7c158ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 >> 172.21.15.165:0/277901281 conn(0x7f0d7c06abe0 msgr2=0x7f0d7c141760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.647+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 shutdown_connections 2024-04-25T10:02:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.648+0000 7f0d7b200640 1 -- 172.21.15.165:0/277901281 wait complete. 2024-04-25T10:02:27.670 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":5,"flags":0,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6832","nonce":3683600865},{"type":"v1","addr":"172.21.15.165:6833","nonce":3683600865}]},"active_addr":"172.21.15.165:6833/3683600865","active_change":"2024-04-25T10:01:44.102224+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4117,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:latest","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":3693307149}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":510082405}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":3803132300}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":2682639243}]}]} 2024-04-25T10:02:27.673 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-25T10:02:27.673 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-25T10:02:27.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:27.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.867+0000 7fb187800640 1 Processor -- start 2024-04-25T10:02:27.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.867+0000 7fb187800640 1 -- start start 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb187800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb180144750 0x7fb18006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb187800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb180144b30 con 0x7fb1800cf030 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb187800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1800712d0 con 0x7fb18006eb10 2024-04-25T10:02:27.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb187800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1800715b0 con 0x7fb180144750 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb185800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb1800cf030 0x7fb18006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:57996/0 (socket says 172.21.15.165:57996) 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb185800640 1 -- 172.21.15.165:0/1251757787 learned_addr learned my addr 172.21.15.165:0/1251757787 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.869+0000 7fb184e00640 1 --2- 172.21.15.165:0/1251757787 >> v2:172.21.15.165:3300/0 conn(0x7fb180144750 0x7fb18006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.870+0000 7fb184e00640 1 -- 172.21.15.165:0/1251757787 >> v1:172.21.15.172:6789/0 conn(0x7fb18006eb10 legacy=0x7fb1800ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.870+0000 7fb184e00640 1 -- 172.21.15.165:0/1251757787 >> v1:172.21.15.165:6789/0 conn(0x7fb1800cf030 legacy=0x7fb18006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.870+0000 7fb184e00640 1 -- 172.21.15.165:0/1251757787 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180071890 con 0x7fb180144750 2024-04-25T10:02:27.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.871+0000 7fb184e00640 1 --2- 172.21.15.165:0/1251757787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb180144750 0x7fb18006e5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1700097a0 tx=0x7fb17002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c876595081efafc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.871+0000 7fb17fe00640 1 -- 172.21.15.165:0/1251757787 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170037070 con 0x7fb180144750 2024-04-25T10:02:27.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.871+0000 7fb17fe00640 1 -- 172.21.15.165:0/1251757787 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb170032440 con 0x7fb180144750 2024-04-25T10:02:27.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.871+0000 7fb17fe00640 1 -- 172.21.15.165:0/1251757787 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170032810 con 0x7fb180144750 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 -- 172.21.15.165:0/1251757787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb180144750 msgr2=0x7fb18006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 --2- 172.21.15.165:0/1251757787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb180144750 0x7fb18006e5d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb1700097a0 tx=0x7fb17002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 -- 172.21.15.165:0/1251757787 shutdown_connections 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 --2- 172.21.15.165:0/1251757787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb180144750 0x7fb18006e5d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 -- 172.21.15.165:0/1251757787 >> 172.21.15.165:0/1251757787 conn(0x7fb18006abe0 msgr2=0x7fb18006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 -- 172.21.15.165:0/1251757787 shutdown_connections 2024-04-25T10:02:27.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.872+0000 7fb187800640 1 -- 172.21.15.165:0/1251757787 wait complete. 2024-04-25T10:02:27.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.874+0000 7fb187800640 1 Processor -- start 2024-04-25T10:02:27.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.874+0000 7fb187800640 1 -- start start 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb187800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 0x7fb1800adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb187800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1800cf030 0x7fb180150780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb187800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb180144750 0x7fb1800b0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb187800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1801455d0 con 0x7fb18006eb10 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb187800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb180145450 con 0x7fb180144750 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb187800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18015c000 con 0x7fb1800cf030 2024-04-25T10:02:27.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb184e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 0x7fb1800adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb184e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 0x7fb1800adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50294/0 (socket says 172.21.15.165:50294) 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.875+0000 7fb184e00640 1 -- 172.21.15.165:0/3055106540 learned_addr learned my addr 172.21.15.165:0/3055106540 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb185800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb180144750 0x7fb1800b0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb184e00640 1 -- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1800cf030 msgr2=0x7fb180150780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb184e00640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1800cf030 0x7fb180150780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb184e00640 1 -- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb180144750 msgr2=0x7fb1800b0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb184e00640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb180144750 0x7fb1800b0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb184e00640 1 -- 172.21.15.165:0/3055106540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1800aca30 con 0x7fb18006eb10 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb185800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb180144750 0x7fb1800b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:27.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb184e00640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 0x7fb1800adfb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb170000c00 tx=0x7fb1700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170037040 con 0x7fb18006eb10 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb170042070 con 0x7fb18006eb10 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb170030a70 con 0x7fb18006eb10 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.876+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb180150ef0 con 0x7fb18006eb10 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.877+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1801514b0 con 0x7fb18006eb10 2024-04-25T10:02:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.878+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb180073990 con 0x7fb18006eb10 2024-04-25T10:02:27.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.879+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb170030c10 con 0x7fb18006eb10 2024-04-25T10:02:27.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.880+0000 7fb17d600640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb15c06b4d0 0x7fb15c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:27.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.880+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb1700aaff0 con 0x7fb18006eb10 2024-04-25T10:02:27.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.880+0000 7fb177e00640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb15c06b4d0 0x7fb15c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:27.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.881+0000 7fb177e00640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb15c06b4d0 0x7fb15c06d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb168006fd0 tx=0x7fb168006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:27.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:27.884+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb170079af0 con 0x7fb18006eb10 2024-04-25T10:02:28.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.084+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1800d0510 con 0x7fb18006eb10 2024-04-25T10:02:28.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.087+0000 7fb17d600640 1 -- 172.21.15.165:0/3055106540 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fb1700793f0 con 0x7fb18006eb10 2024-04-25T10:02:28.093 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:28.093 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:28.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb15c06b4d0 msgr2=0x7fb15c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb15c06b4d0 0x7fb15c06d990 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb168006fd0 tx=0x7fb168006bc0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 msgr2=0x7fb1800adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 0x7fb1800adfb0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb170000c00 tx=0x7fb1700046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 shutdown_connections 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb15c06b4d0 0x7fb15c06d990 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb180144750 0x7fb1800b0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1800cf030 0x7fb180150780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 --2- 172.21.15.165:0/3055106540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb18006eb10 0x7fb1800adfb0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 >> 172.21.15.165:0/3055106540 conn(0x7fb18006abe0 msgr2=0x7fb1800b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 shutdown_connections 2024-04-25T10:02:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.090+0000 7fb187800640 1 -- 172.21.15.165:0/3055106540 wait complete. 2024-04-25T10:02:28.113 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-25T10:02:28.114 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:28.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.295+0000 7f3dbaa00640 1 Processor -- start 2024-04-25T10:02:28.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.295+0000 7f3dbaa00640 1 -- start start 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3dbaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3db4144440 0x7f3db4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db40cda30 0x7f3db415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3dbaa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3db415bf30 con 0x7f3db4071530 2024-04-25T10:02:28.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3dbaa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3db40719a0 con 0x7f3db4144440 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3dbaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db415c860 con 0x7f3db40cda30 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3db9600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3db4144440 0x7f3db4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3db9600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3db4144440 0x7f3db4070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50298/0 (socket says 172.21.15.165:50298) 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.297+0000 7f3db9600640 1 -- 172.21.15.165:0/844369987 learned_addr learned my addr 172.21.15.165:0/844369987 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.298+0000 7f3dba000640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db40cda30 0x7f3db415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.298+0000 7f3db9600640 1 -- 172.21.15.165:0/844369987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db40cda30 msgr2=0x7f3db415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.298+0000 7f3db9600640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db40cda30 0x7f3db415ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.298+0000 7f3db9600640 1 -- 172.21.15.165:0/844369987 >> v1:172.21.15.172:6789/0 conn(0x7f3db4071530 legacy=0x7f3db40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.298+0000 7f3db9600640 1 -- 172.21.15.165:0/844369987 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db415c210 con 0x7f3db4144440 2024-04-25T10:02:28.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.298+0000 7f3dba000640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db40cda30 0x7f3db415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.299+0000 7f3db9600640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db4070ff0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009870 tx=0x7f3da802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18df91ee984d959b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.299+0000 7f3db3e00640 1 -- 172.21.15.165:0/844369987 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8037070 con 0x7f3db4144440 2024-04-25T10:02:28.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.299+0000 7f3db3e00640 1 -- 172.21.15.165:0/844369987 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3da8032440 con 0x7f3db4144440 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.299+0000 7f3db3e00640 1 -- 172.21.15.165:0/844369987 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3da8032810 con 0x7f3db4144440 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 -- 172.21.15.165:0/844369987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 msgr2=0x7f3db4070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db4070ff0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3da8009870 tx=0x7f3da802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 -- 172.21.15.165:0/844369987 shutdown_connections 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db40cda30 0x7f3db415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/844369987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db4070ff0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 -- 172.21.15.165:0/844369987 >> 172.21.15.165:0/844369987 conn(0x7f3db406abe0 msgr2=0x7f3db406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 -- 172.21.15.165:0/844369987 shutdown_connections 2024-04-25T10:02:28.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.300+0000 7f3dbaa00640 1 -- 172.21.15.165:0/844369987 wait complete. 2024-04-25T10:02:28.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.301+0000 7f3dbaa00640 1 Processor -- start 2024-04-25T10:02:28.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 -- start start 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db4071530 0x7f3db413dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3db40cda30 0x7f3db41f3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db40c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db415bc30 con 0x7f3db4144440 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db40cdea0 con 0x7f3db40cda30 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.302+0000 7f3dbaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db415d230 con 0x7f3db4071530 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db40c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db40c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50310/0 (socket says 172.21.15.165:50310) 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 -- 172.21.15.165:0/3222248767 learned_addr learned my addr 172.21.15.165:0/3222248767 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3db9600640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db4071530 0x7f3db413dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3db8c00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3db40cda30 0x7f3db41f3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 -- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db4071530 msgr2=0x7f3db413dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db4071530 0x7f3db413dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 -- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3db40cda30 msgr2=0x7f3db41f3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3db40cda30 0x7f3db41f3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 -- 172.21.15.165:0/3222248767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db413e2b0 con 0x7f3db4144440 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.303+0000 7f3dba000640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db40c49f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00cef0 tx=0x7f3dac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.304+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac01a070 con 0x7f3db4144440 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.304+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dac00d860 con 0x7f3db4144440 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.304+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac00dc50 con 0x7f3db4144440 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.304+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db40b30a0 con 0x7f3db4144440 2024-04-25T10:02:28.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.304+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db40b3660 con 0x7f3db4144440 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.305+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3db4073990 con 0x7f3db4144440 2024-04-25T10:02:28.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.306+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3dac01d070 con 0x7f3db4144440 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.307+0000 7f3db1600640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d9406b4d0 0x7f3d9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.307+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3dac08adb0 con 0x7f3db4144440 2024-04-25T10:02:28.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.307+0000 7f3db9600640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d9406b4d0 0x7f3d9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.308+0000 7f3db9600640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d9406b4d0 0x7f3d9406d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3da8034000 tx=0x7f3da802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.311+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3db413e2b0 con 0x7f3db4144440 2024-04-25T10:02:28.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.492+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3db406bff0 con 0x7f3db4144440 2024-04-25T10:02:28.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.493+0000 7f3db1600640 1 -- 172.21.15.165:0/3222248767 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f3dac0599b0 con 0x7f3db4144440 2024-04-25T10:02:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:28.500 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d9406b4d0 msgr2=0x7f3d9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d9406b4d0 0x7f3d9406d990 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3da8034000 tx=0x7f3da802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 msgr2=0x7f3db40c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db40c49f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00cef0 tx=0x7f3dac00a710 comp rx=0 tx=0).stop 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 shutdown_connections 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d9406b4d0 0x7f3d9406d990 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3db4144440 0x7f3db40c49f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3db40cda30 0x7f3db41f3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 --2- 172.21.15.165:0/3222248767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3db4071530 0x7f3db413dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.496+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 >> 172.21.15.165:0/3222248767 conn(0x7f3db406abe0 msgr2=0x7f3db413fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.497+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 shutdown_connections 2024-04-25T10:02:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.497+0000 7f3dbaa00640 1 -- 172.21.15.165:0/3222248767 wait complete. 2024-04-25T10:02:28.520 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-25T10:02:28.521 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-25T10:02:28.521 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-25T10:02:28.521 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-25T10:02:28.522 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-25T10:02:28.522 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-25T10:02:28.523 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-25T10:02:28.523 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-25T10:02:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.794+0000 7fc3cba00640 1 Processor -- start 2024-04-25T10:02:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.794+0000 7fc3cba00640 1 -- start start 2024-04-25T10:02:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.796+0000 7fc3cba00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3bc04f280 0x7fc3bc007180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.796+0000 7fc3cba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc037c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.796+0000 7fc3cba00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3bc038170 0x7fc3bc04f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.796+0000 7fc3cba00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3bc0543b0 con 0x7fc3bc04f280 2024-04-25T10:02:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.796+0000 7fc3cba00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3bc0069c0 con 0x7fc3bc038170 2024-04-25T10:02:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.796+0000 7fc3cba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc053df0 con 0x7fc3bc033980 2024-04-25T10:02:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.797+0000 7fc3c9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3bc04f280 0x7fc3bc007180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.797+0000 7fc3c9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3bc038170 0x7fc3bc04f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.798+0000 7fc3c9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3bc038170 0x7fc3bc04f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51368/0 (socket says 172.21.15.165:51368) 2024-04-25T10:02:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.798+0000 7fc3c9a00640 1 -- 172.21.15.165:0/3266763042 learned_addr learned my addr 172.21.15.165:0/3266763042 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.798+0000 7fc3c3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc037c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.798+0000 7f6efe800640 1 Processor -- start 2024-04-25T10:02:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.799+0000 7fc3c9000640 1 -- 172.21.15.165:0/3266763042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 msgr2=0x7fc3bc037c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.799+0000 7f6efe800640 1 -- start start 2024-04-25T10:02:28.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.799+0000 7fc3c9000640 1 --2- 172.21.15.165:0/3266763042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc037c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.799+0000 7fc3c9000640 1 -- 172.21.15.165:0/3266763042 >> v2:172.21.15.172:3300/0 conn(0x7fc3bc038170 msgr2=0x7fc3bc04f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.799+0000 7fc3c9000640 1 --2- 172.21.15.165:0/3266763042 >> v2:172.21.15.172:3300/0 conn(0x7fc3bc038170 0x7fc3bc04f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.799+0000 7fc3c9000640 1 -- 172.21.15.165:0/3266763042 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0540d0 con 0x7fc3bc04f280 2024-04-25T10:02:28.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.800+0000 7f6efe800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ef00557b0 0x7f6ef00df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efe800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef0053220 0x7f6ef0054080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efe800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ef0006060 con 0x7f6ef00073e0 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efe800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ef0006340 con 0x7f6ef00557b0 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efe800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef0006620 con 0x7f6ef0053220 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ef00557b0 0x7f6ef00df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ef00557b0 0x7f6ef00df3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51374/0 (socket says 172.21.15.165:51374) 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.801+0000 7f6efca00640 1 -- 172.21.15.165:0/951901451 learned_addr learned my addr 172.21.15.165:0/951901451 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7f6ef7e00640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef0053220 0x7f6ef0054080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7fc3c9000640 1 --2- 172.21.15.165:0/3266763042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc04f280 0x7fc3bc007180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc051460 tx=0x7fc3b802c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4783f54332d523c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7fc3c3400640 1 -- 172.21.15.165:0/3266763042 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b8038070 con 0x7fc3bc04f280 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7f6efca00640 1 -- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef0053220 msgr2=0x7f6ef0054080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7f6efca00640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef0053220 0x7f6ef0054080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7f6efca00640 1 -- 172.21.15.165:0/951901451 >> v1:172.21.15.165:6789/0 conn(0x7f6ef00073e0 legacy=0x7f6ef00381a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.802+0000 7f6efca00640 1 -- 172.21.15.165:0/951901451 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef0006900 con 0x7f6ef00557b0 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.803+0000 7f6ef7e00640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef0053220 0x7f6ef0054080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.803+0000 7fc3c3400640 1 -- 172.21.15.165:0/3266763042 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b8008df0 con 0x7fc3bc04f280 2024-04-25T10:02:28.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.803+0000 7f8fbaa00640 1 Processor -- start 2024-04-25T10:02:28.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.803+0000 7f8fbaa00640 1 -- start start 2024-04-25T10:02:28.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.804+0000 7f6efca00640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef00557b0 0x7f6ef00df3f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8069d80 tx=0x7f6ef80a72c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9b0656390e298a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.804+0000 7fc3cba00640 1 -- 172.21.15.165:0/3266763042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc04f280 msgr2=0x7fc3bc007180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.804+0000 7fc3cba00640 1 --2- 172.21.15.165:0/3266763042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc04f280 0x7fc3bc007180 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc051460 tx=0x7fc3b802c410 comp rx=0 tx=0).stop 2024-04-25T10:02:28.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.804+0000 7f6effe00640 1 -- 172.21.15.165:0/951901451 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80ad070 con 0x7f6ef00557b0 2024-04-25T10:02:28.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.804+0000 7f6effe00640 1 -- 172.21.15.165:0/951901451 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef80b1600 con 0x7f6ef00557b0 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7fc3cba00640 1 -- 172.21.15.165:0/3266763042 shutdown_connections 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7fc3cba00640 1 --2- 172.21.15.165:0/3266763042 >> v2:172.21.15.172:3300/0 conn(0x7fc3bc038170 0x7fc3bc04f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7fc3cba00640 1 --2- 172.21.15.165:0/3266763042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc037c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7fc3cba00640 1 --2- 172.21.15.165:0/3266763042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc04f280 0x7fc3bc007180 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7fc3cba00640 1 -- 172.21.15.165:0/3266763042 >> 172.21.15.165:0/3266763042 conn(0x7fc3bc009fa0 msgr2=0x7fc3bc00a380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7f8fbaa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8fb415dbd0 0x7f8fb4161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7fc3cba00640 1 -- 172.21.15.165:0/3266763042 shutdown_connections 2024-04-25T10:02:28.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.806+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 0x7f8fb40b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.806+0000 7f8fbaa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb4161e70 con 0x7f8fb40b5910 2024-04-25T10:02:28.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.806+0000 7fc3cba00640 1 -- 172.21.15.165:0/3266763042 wait complete. 2024-04-25T10:02:28.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7f6efe800640 1 -- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef00557b0 msgr2=0x7f6ef00df3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.805+0000 7f6efe800640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef00557b0 0x7f6ef00df3f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f6ef8069d80 tx=0x7f6ef80a72c0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.806+0000 7f8fbaa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb40d1ce0 con 0x7f8fb415dbd0 2024-04-25T10:02:28.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.806+0000 7f8fbaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4162150 con 0x7f8fb40b95c0 2024-04-25T10:02:28.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.806+0000 7f8fb8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 0x7f8fb40b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f6efe800640 1 -- 172.21.15.165:0/951901451 shutdown_connections 2024-04-25T10:02:28.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f6efe800640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef0053220 0x7f6ef0054080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f6efe800640 1 --2- 172.21.15.165:0/951901451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef00557b0 0x7f6ef00df3f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f6efe800640 1 -- 172.21.15.165:0/951901451 >> 172.21.15.165:0/951901451 conn(0x7f6ef0054c90 msgr2=0x7f6ef00550a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f8fb8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 0x7f8fb40b53d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37660/0 (socket says 172.21.15.165:37660) 2024-04-25T10:02:28.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2935740011 learned_addr learned my addr 172.21.15.165:0/2935740011 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f6efe800640 1 -- 172.21.15.165:0/951901451 shutdown_connections 2024-04-25T10:02:28.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f6efe800640 1 -- 172.21.15.165:0/951901451 wait complete. 2024-04-25T10:02:28.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2935740011 >> v2:172.21.15.172:3300/0 conn(0x7f8fb415dbd0 msgr2=0x7f8fb4161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f8fb8c00640 1 --2- 172.21.15.165:0/2935740011 >> v2:172.21.15.172:3300/0 conn(0x7f8fb415dbd0 0x7f8fb4161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2935740011 >> v1:172.21.15.165:6789/0 conn(0x7f8fb40b5910 legacy=0x7f8fb406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.807+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2935740011 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb4162430 con 0x7f8fb40b95c0 2024-04-25T10:02:28.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.809+0000 7fc3cba00640 1 Processor -- start 2024-04-25T10:02:28.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.810+0000 7fc3cba00640 1 -- start start 2024-04-25T10:02:28.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.810+0000 7f8fb8c00640 1 --2- 172.21.15.165:0/2935740011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 0x7f8fb40b53d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009870 tx=0x7f8fa802a290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18e55fcb92532e83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.810+0000 7f6efe800640 1 Processor -- start 2024-04-25T10:02:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.811+0000 7f8fb3e00640 1 -- 172.21.15.165:0/2935740011 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8037070 con 0x7f8fb40b95c0 2024-04-25T10:02:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.811+0000 7f8fb3e00640 1 -- 172.21.15.165:0/2935740011 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8032440 con 0x7f8fb40b95c0 2024-04-25T10:02:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.811+0000 7f8fb3e00640 1 -- 172.21.15.165:0/2935740011 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8032810 con 0x7f8fb40b95c0 2024-04-25T10:02:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.810+0000 7f6efe800640 1 -- start start 2024-04-25T10:02:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7fc3cba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f6efe800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef00073e0 0x7f6ef00df3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f5918a00640 1 Processor -- start 2024-04-25T10:02:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f5918a00640 1 -- start start 2024-04-25T10:02:28.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7fc3cba00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc038170 0x7fc3bc0fad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f6efe800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef0053220 0x7f6ef002b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f6efe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef00557b0 0x7f6ef004d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f6efe800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef0007870 con 0x7f6ef00557b0 2024-04-25T10:02:28.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f6efe800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef0006ec0 con 0x7f6ef0053220 2024-04-25T10:02:28.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7f6efe800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ef00079f0 con 0x7f6ef00073e0 2024-04-25T10:02:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7fc3cba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3bc04f280 0x7fc3bc19f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7fc3cba00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0076c0 con 0x7fc3bc038170 2024-04-25T10:02:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7fc3cba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc006c60 con 0x7fc3bc04f280 2024-04-25T10:02:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.812+0000 7fc3cba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3bc0385e0 con 0x7fc3bc033980 2024-04-25T10:02:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fb242a00640 1 Processor -- start 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fc3c9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fc3c9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3bc04f280 0x7fc3bc19f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7f6efd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef00557b0 0x7f6ef004d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7f6efca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef00073e0 0x7f6ef00df3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7f6efd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef00557b0 0x7f6ef004d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50318/0 (socket says 172.21.15.165:50318) 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7f6efd400640 1 -- 172.21.15.165:0/4939479 learned_addr learned my addr 172.21.15.165:0/4939479 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7f6ef7e00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef0053220 0x7f6ef002b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fc3c9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3bc04f280 0x7fc3bc19f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51416/0 (socket says 172.21.15.165:51416) 2024-04-25T10:02:28.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fc3c9a00640 1 -- 172.21.15.165:0/1852882350 learned_addr learned my addr 172.21.15.165:0/1852882350 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7fc3c3e00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc038170 0x7fc3bc0fad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fb242a00640 1 -- start start 2024-04-25T10:02:28.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2935740011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 msgr2=0x7f8fb40b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f8fbaa00640 1 --2- 172.21.15.165:0/2935740011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 0x7f8fb40b53d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8fa8009870 tx=0x7f8fa802a290 comp rx=0 tx=0).stop 2024-04-25T10:02:28.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f5918a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591406bff0 0x7f59140b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f5918a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59140b5970 0x7f59140d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.813+0000 7fb3aa200640 1 Processor -- start 2024-04-25T10:02:28.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7fb3aa200640 1 -- start start 2024-04-25T10:02:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f5918a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f591415e110 con 0x7f591415dd30 2024-04-25T10:02:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f6efca00640 1 -- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef0053220 msgr2=0x7f6ef002b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f6efca00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef0053220 0x7f6ef002b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f6efca00640 1 -- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef00557b0 msgr2=0x7f6ef004d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f6efca00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ef00557b0 0x7f6ef004d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f6efca00640 1 -- 172.21.15.165:0/4939479 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ef00df9d0 con 0x7f6ef00073e0 2024-04-25T10:02:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f5918a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f591406d000 con 0x7f59140b5970 2024-04-25T10:02:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f5918a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59140b2fd0 con 0x7f591406bff0 2024-04-25T10:02:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2935740011 shutdown_connections 2024-04-25T10:02:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f8fbaa00640 1 --2- 172.21.15.165:0/2935740011 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb40b95c0 0x7f8fb40b53d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f8fbaa00640 1 --2- 172.21.15.165:0/2935740011 >> v2:172.21.15.172:3300/0 conn(0x7f8fb415dbd0 0x7f8fb4161620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.814+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2935740011 >> 172.21.15.165:0/2935740011 conn(0x7f8fb40d03d0 msgr2=0x7f8fb40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f5912a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591406bff0 0x7f59140b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2935740011 shutdown_connections 2024-04-25T10:02:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f6ef7e00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ef0053220 0x7f6ef002b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f5913e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59140b5970 0x7f59140d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f5913e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59140b5970 0x7f59140d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51422/0 (socket says 172.21.15.165:51422) 2024-04-25T10:02:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f5913e00640 1 -- 172.21.15.165:0/1431386085 learned_addr learned my addr 172.21.15.165:0/1431386085 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f6efca00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef00073e0 0x7f6ef00df3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6ef806a560 tx=0x7f6ef8068960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7fc3c9a00640 1 -- 172.21.15.165:0/1852882350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 msgr2=0x7fc3bc195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7fc3c9a00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3bc033980 0x7fc3bc195b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7fc3c9a00640 1 -- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc038170 msgr2=0x7fc3bc0fad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7fc3c9a00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3bc038170 0x7fc3bc0fad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7fc3c9a00640 1 -- 172.21.15.165:0/1852882350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3bc0fa2d0 con 0x7fc3bc04f280 2024-04-25T10:02:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7fb3aa200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a41620f0 0x7fb3a4163100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.815+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2935740011 wait complete. 2024-04-25T10:02:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80ba640 con 0x7f6ef00073e0 2024-04-25T10:02:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef80ba7e0 con 0x7f6ef00073e0 2024-04-25T10:02:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80bab90 con 0x7f6ef00073e0 2024-04-25T10:02:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb3a3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb3a41666b0 0x7fb3a40b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58048/0 (socket says 172.21.15.165:58048) 2024-04-25T10:02:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb3a3e00640 1 -- 172.21.15.165:0/3576942670 learned_addr learned my addr 172.21.15.165:0/3576942670 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb3aa200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a4166b10 con 0x7fb3a41666b0 2024-04-25T10:02:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f5913e00640 1 -- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591406bff0 msgr2=0x7f59140b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f5913e00640 1 --2- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591406bff0 0x7f59140b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f5913e00640 1 -- 172.21.15.165:0/1431386085 >> v1:172.21.15.165:6789/0 conn(0x7f591415dd30 legacy=0x7f59140cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f5913e00640 1 -- 172.21.15.165:0/1431386085 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591406c480 con 0x7f59140b5970 2024-04-25T10:02:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fc3c9a00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3bc04f280 0x7fc3bc19f1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3c404f440 tx=0x7fc3c4067fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.817+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4074530 con 0x7fc3bc04f280 2024-04-25T10:02:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.817+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3c40746d0 con 0x7fc3bc04f280 2024-04-25T10:02:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.817+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4083640 con 0x7fc3bc04f280 2024-04-25T10:02:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef00dfcb0 con 0x7f6ef00073e0 2024-04-25T10:02:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ef00e0270 con 0x7f6ef00073e0 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb3a8e00640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a41620f0 0x7fb3a4163100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb242a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb23c159b20 0x7fb23c0b38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb242a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c06e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb242a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c159f00 con 0x7fb23c06dbc0 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb242a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0b3460 con 0x7fb23c159b20 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.816+0000 7fb242a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0b2850 con 0x7fb23c0cfa10 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.817+0000 7fb23b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb23c06dbc0 0x7fb23c0cf300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58056/0 (socket says 172.21.15.165:58056) 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.817+0000 7fb23b400640 1 -- 172.21.15.165:0/1243837368 learned_addr learned my addr 172.21.15.165:0/1243837368 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.817+0000 7fb23be00640 1 --2- 172.21.15.165:0/1243837368 >> v2:172.21.15.165:3300/0 conn(0x7fb23c159b20 0x7fb23c0b38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6ef0005560 con 0x7f6ef00073e0 2024-04-25T10:02:28.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb240a00640 1 --2- 172.21.15.165:0/1243837368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c06e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7f8fbaa00640 1 Processor -- start 2024-04-25T10:02:28.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3bc0fb2d0 con 0x7fc3bc04f280 2024-04-25T10:02:28.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3bc19ff80 con 0x7fc3bc04f280 2024-04-25T10:02:28.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb23be00640 1 -- 172.21.15.165:0/1243837368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 msgr2=0x7fb23c06e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb23be00640 1 --2- 172.21.15.165:0/1243837368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c06e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb23be00640 1 -- 172.21.15.165:0/1243837368 >> v1:172.21.15.165:6789/0 conn(0x7fb23c06dbc0 legacy=0x7fb23c0cf300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb23be00640 1 -- 172.21.15.165:0/1243837368 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c0b2b30 con 0x7fb23c159b20 2024-04-25T10:02:28.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3a40b5b90 con 0x7fb3a40cdfa0 2024-04-25T10:02:28.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4162c70 con 0x7fb3a41620f0 2024-04-25T10:02:28.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 408588052 0 0) 0x7fb394001110 con 0x7fb3a41666b0 2024-04-25T10:02:28.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb388003520 con 0x7fb3a41666b0 2024-04-25T10:02:28.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7f5913e00640 1 --2- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 0x7f59140d2160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5904009870 tx=0x7f590402a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b82c8f040dcf2f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.818+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 158726709 0 0) 0x7fb3a40b5b90 con 0x7fb3a40cdfa0 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb394001110 con 0x7fb3a40cdfa0 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 7507399 0 0) 0x7fb388003520 con 0x7fb3a41666b0 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a41620f0 msgr2=0x7fb3a4163100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a41620f0 0x7fb3a4163100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 >> v1:172.21.15.172:6789/0 conn(0x7fb3a40cdfa0 legacy=0x7fb3a40d1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40b5e70 con 0x7fb3a41666b0 2024-04-25T10:02:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb394001840 con 0x7fb3a41666b0 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 >> v1:172.21.15.165:6789/0 conn(0x7fb3a41666b0 legacy=0x7fb3a40b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb388003b70 0x7fb388003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3880051c0 0x7fb38800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4162c70 con 0x7fb3880051c0 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38800bfb0 con 0x7fb388004800 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38800c1b0 con 0x7fb388003b70 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3880051c0 0x7fb38800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.819+0000 7f8fbaa00640 1 -- start start 2024-04-25T10:02:28.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7f5912000640 1 -- 172.21.15.165:0/1431386085 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904037070 con 0x7f59140b5970 2024-04-25T10:02:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb3a3e00640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3bc005380 con 0x7fc3bc04f280 2024-04-25T10:02:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7f5912000640 1 -- 172.21.15.165:0/1431386085 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904032440 con 0x7f59140b5970 2024-04-25T10:02:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7f5912000640 1 -- 172.21.15.165:0/1431386085 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904032810 con 0x7f59140b5970 2024-04-25T10:02:28.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb23be00640 1 --2- 172.21.15.165:0/1243837368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c0b38f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2280090e0 tx=0x7fb22802b360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=517be7acf55fd930 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb3a8e00640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb388003b70 0x7fb388003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb23aa00640 1 -- 172.21.15.165:0/1243837368 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228038070 con 0x7fb23c159b20 2024-04-25T10:02:28.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb23aa00640 1 -- 172.21.15.165:0/1243837368 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb228033640 con 0x7fb23c159b20 2024-04-25T10:02:28.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb23aa00640 1 -- 172.21.15.165:0/1243837368 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb228033a10 con 0x7fb23c159b20 2024-04-25T10:02:28.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.820+0000 7fb3a9800640 1 -- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb388003b70 msgr2=0x7fb388003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb40b5910 0x7f8fb41ffa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb40b95c0 0x7f8fb40d4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f8fbaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb415dbd0 0x7f8fb415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f8fbaa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40d3180 con 0x7f8fb40b95c0 2024-04-25T10:02:28.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f8fbaa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb4161b60 con 0x7f8fb40b5910 2024-04-25T10:02:28.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f8fbaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb40d3300 con 0x7f8fb415dbd0 2024-04-25T10:02:28.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7fb242a00640 1 -- 172.21.15.165:0/1243837368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 msgr2=0x7fb23c0b38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7fb242a00640 1 --2- 172.21.15.165:0/1243837368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c0b38f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2280090e0 tx=0x7fb22802b360 comp rx=0 tx=0).stop 2024-04-25T10:02:28.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f5918a00640 1 -- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 msgr2=0x7f59140d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ef80b1600 con 0x7f6ef00073e0 2024-04-25T10:02:28.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7f5918a00640 1 --2- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 0x7f59140d2160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5904009870 tx=0x7f590402a210 comp rx=0 tx=0).stop 2024-04-25T10:02:28.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb242a00640 1 -- 172.21.15.165:0/1243837368 shutdown_connections 2024-04-25T10:02:28.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb242a00640 1 --2- 172.21.15.165:0/1243837368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c06e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb242a00640 1 --2- 172.21.15.165:0/1243837368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c0b38f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb242a00640 1 -- 172.21.15.165:0/1243837368 >> 172.21.15.165:0/1243837368 conn(0x7fb23c06abe0 msgr2=0x7fb23c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7f8fb8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb40b95c0 0x7f8fb40d4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7f8fb8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb40b95c0 0x7f8fb40d4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50348/0 (socket says 172.21.15.165:50348) 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2821859840 learned_addr learned my addr 172.21.15.165:0/2821859840 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb388003b70 0x7fb388003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7fb3a9800640 1 -- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb388004800 msgr2=0x7fb388004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.821+0000 7fb3a9800640 1 -- 172.21.15.165:0/3576942670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40b5e70 con 0x7fb3880051c0 2024-04-25T10:02:28.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb3a3e00640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb388004800 0x7fb388004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb242a00640 1 -- 172.21.15.165:0/1243837368 shutdown_connections 2024-04-25T10:02:28.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7f8fba000640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb415dbd0 0x7f8fb415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.822+0000 7fb242a00640 1 -- 172.21.15.165:0/1243837368 wait complete. 2024-04-25T10:02:28.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.823+0000 7f5918a00640 1 -- 172.21.15.165:0/1431386085 shutdown_connections 2024-04-25T10:02:28.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.823+0000 7f5918a00640 1 --2- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 0x7f59140d2160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.823+0000 7f5918a00640 1 --2- 172.21.15.165:0/1431386085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591406bff0 0x7f59140b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.823+0000 7f5918a00640 1 -- 172.21.15.165:0/1431386085 >> 172.21.15.165:0/1431386085 conn(0x7f59140d0610 msgr2=0x7f59140d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.823+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3880051c0 0x7fb38800b9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0018c0 tx=0x7fb388005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b596d6582d4c068a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.823+0000 7f8fb9600640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb40b5910 0x7f8fb41ffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0014e0 con 0x7fb3880051c0 2024-04-25T10:02:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f6ef5600640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ed806b5a0 0x7f6ed806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6ef8129280 con 0x7f6ef00073e0 2024-04-25T10:02:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f6ef7e00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ed806b5a0 0x7f6ed806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f6ef5600640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7f6ed8074d20 0x7f6ed8077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 --> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6ef80c8ea0 con 0x7f6ed8074d20 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ef8129670 con 0x7f6ef00073e0 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f5918a00640 1 -- 172.21.15.165:0/1431386085 shutdown_connections 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2821859840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb415dbd0 msgr2=0x7f8fb415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f8fb8c00640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb415dbd0 0x7f8fb415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2821859840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb40b5910 msgr2=0x7f8fb41ffa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f8fb8c00640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb40b5910 0x7f8fb41ffa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f8fb8c00640 1 -- 172.21.15.165:0/2821859840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb415b680 con 0x7f8fb40b95c0 2024-04-25T10:02:28.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c0052f0 con 0x7fb3880051c0 2024-04-25T10:02:28.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7fb3a3400640 1 -- 172.21.15.165:0/3576942670 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c02acc0 con 0x7fb3880051c0 2024-04-25T10:02:28.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3c408c3f0 con 0x7fc3bc04f280 2024-04-25T10:02:28.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f5918a00640 1 -- 172.21.15.165:0/1431386085 wait complete. 2024-04-25T10:02:28.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f6efd400640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7f6ed8074d20 0x7f6ed8077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f280c000640 1 Processor -- start 2024-04-25T10:02:28.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.824+0000 7f280c000640 1 -- start start 2024-04-25T10:02:28.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7fb242a00640 1 Processor -- start 2024-04-25T10:02:28.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7fb242a00640 1 -- start start 2024-04-25T10:02:28.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f280c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f280416a4b0 0x7f280416a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f280c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f2804071f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f280c000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280416e840 con 0x7f2804071b10 2024-04-25T10:02:28.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f280c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280416f170 con 0x7f280416a4b0 2024-04-25T10:02:28.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7f280c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2804072490 con 0x7f280406ee00 2024-04-25T10:02:28.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7fc3c0c00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3a406b5a0 0x7fc3a406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7f6efd400640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7f6ed8074d20 0x7f6ed8077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7f2809600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f280416a4b0 0x7f280416a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7f2809600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f280416a4b0 0x7f280416a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51452/0 (socket says 172.21.15.165:51452) 2024-04-25T10:02:28.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7f2809600640 1 -- 172.21.15.165:0/3761514721 learned_addr learned my addr 172.21.15.165:0/3761514721 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7f6ef7e00640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ed806b5a0 0x7f6ed806da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00489e0 tx=0x7f6ee4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3880051c0 msgr2=0x7fb38800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7fb3aa200640 1 --2- 172.21.15.165:0/3576942670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3880051c0 0x7fb38800b9c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb38c0018c0 tx=0x7fb388005830 comp rx=0 tx=0).stop 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.825+0000 7fb3a9800640 1 -- 172.21.15.165:0/3576942670 reap_dead start 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 shutdown_connections 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 >> 172.21.15.165:0/3576942670 conn(0x7fb3a40d03d0 msgr2=0x7fb3a40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc3c40f42d0 con 0x7fc3bc04f280 2024-04-25T10:02:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.826+0000 7fc3c9000640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3a406b5a0 0x7fc3a406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7f2809600640 1 -- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 msgr2=0x7f2804071f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7f2809600640 1 --2- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f2804071f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7f2809600640 1 -- 172.21.15.165:0/3761514721 >> v1:172.21.15.165:6789/0 conn(0x7f2804071b10 legacy=0x7f280406e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7f2809600640 1 -- 172.21.15.165:0/3761514721 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280416eb20 con 0x7f280416a4b0 2024-04-25T10:02:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7fc3c0c00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fc3a4074d10 0x7fc3a4077150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 --> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3a4077820 con 0x7fc3a4074d10 2024-04-25T10:02:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3c40f46c0 con 0x7fc3bc04f280 2024-04-25T10:02:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.827+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== osd.1 v2:172.21.15.165:6808/4150887630 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f6ef80c8ea0 con 0x7f6ed8074d20 2024-04-25T10:02:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fc3c9000640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3a406b5a0 0x7fc3a406da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8035000 tx=0x7fc3b800c040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f8fb8c00640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb40b95c0 0x7f8fb40d4110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80090b0 tx=0x7f8fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f5918a00640 1 Processor -- start 2024-04-25T10:02:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 shutdown_connections 2024-04-25T10:02:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fb242a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb23c06dbc0 0x7fb23c1e4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8041020 con 0x7f8fb40b95c0 2024-04-25T10:02:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa8030760 con 0x7f8fb40b95c0 2024-04-25T10:02:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa8037040 con 0x7f8fb40b95c0 2024-04-25T10:02:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb415b920 con 0x7f8fb40b95c0 2024-04-25T10:02:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fc3c3e00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fc3a4074d10 0x7fc3a4077150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7fc3c3e00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fc3a4074d10 0x7fc3a4077150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb415bee0 con 0x7f8fb40b95c0 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fb242a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c1f4d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fb242a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c1fafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.829+0000 7fb240a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c1fafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.829+0000 7fb240a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c1fafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50352/0 (socket says 172.21.15.165:50352) 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.829+0000 7fb23b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c1f4d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.829+0000 7fb240a00640 1 -- 172.21.15.165:0/3598210879 learned_addr learned my addr 172.21.15.165:0/3598210879 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0b2470 con 0x7fb23c159b20 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0b30f0 con 0x7fb23c06dbc0 2024-04-25T10:02:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0b25f0 con 0x7fb23c0cfa10 2024-04-25T10:02:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7f2809600640 1 --2- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f280416a890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd41621a2a1dc113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.829+0000 7f5918a00640 1 -- start start 2024-04-25T10:02:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.828+0000 7fb3aa200640 1 -- 172.21.15.165:0/3576942670 wait complete. 2024-04-25T10:02:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb3aa200640 1 Processor -- start 2024-04-25T10:02:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5918a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 0x7f59141595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb23b400640 1 -- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb23c06dbc0 msgr2=0x7fb23c1e4d70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb23b400640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb23c06dbc0 0x7fb23c1e4d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb23b400640 1 -- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 msgr2=0x7fb23c1fafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb23b400640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c1fafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb3aa200640 1 -- start start 2024-04-25T10:02:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb23b400640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c1e3ba0 con 0x7fb23c0cfa10 2024-04-25T10:02:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7fb240a00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb23c159b20 0x7fb23c1fafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f2803e00640 1 -- 172.21.15.165:0/3761514721 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8037070 con 0x7f280416a4b0 2024-04-25T10:02:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f2803e00640 1 -- 172.21.15.165:0/3761514721 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f802ae50 con 0x7f280416a4b0 2024-04-25T10:02:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f2803e00640 1 -- 172.21.15.165:0/3761514721 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8032860 con 0x7f280416a4b0 2024-04-25T10:02:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5918a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591415dd30 0x7f5914159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== osd.0 v2:172.21.15.165:6800/4162285384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc3a4077820 con 0x7fc3a4074d10 2024-04-25T10:02:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5918a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59141fb660 0x7f59141fbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5918a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591406cd20 con 0x7f59141fb660 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7fb23b400640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c1f4d50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c3e0 tx=0x7fb22c00c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5918a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591406ca40 con 0x7f59140b5970 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5918a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59140d26a0 con 0x7f591415dd30 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.830+0000 7f5913e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59141fb660 0x7f59141fbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59141fb660 0x7f59141fbb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50354/0 (socket says 172.21.15.165:50354) 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 -- 172.21.15.165:0/2976109546 learned_addr learned my addr 172.21.15.165:0/2976109546 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 -- 172.21.15.165:0/2976109546 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591415dd30 msgr2=0x7f5914159b10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f591415dd30 0x7f5914159b10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 -- 172.21.15.165:0/2976109546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 msgr2=0x7f59141595d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59140b5970 0x7f59141595d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.831+0000 7f5913e00640 1 -- 172.21.15.165:0/2976109546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59141f9320 con 0x7f59141fb660 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7f5913e00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59141fb660 0x7f59141fbb10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5904035040 tx=0x7f59040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5904037040 con 0x7f59141fb660 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5904042070 con 0x7f59141fb660 2024-04-25T10:02:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f590403b8b0 con 0x7f59141fb660 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7f280c000640 1 -- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 msgr2=0x7f280416a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.832+0000 7f280c000640 1 --2- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f280416a890 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f27f8009870 tx=0x7f27f802a300 comp rx=0 tx=0).stop 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f280c000640 1 -- 172.21.15.165:0/3761514721 shutdown_connections 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f280c000640 1 --2- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f2804071f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f280c000640 1 --2- 172.21.15.165:0/3761514721 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f280416a890 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f280c000640 1 -- 172.21.15.165:0/3761514721 >> 172.21.15.165:0/3761514721 conn(0x7f280406abe0 msgr2=0x7f280406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7fb3aa200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3880051c0 0x7fb3a40d1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f280c000640 1 -- 172.21.15.165:0/3761514721 shutdown_connections 2024-04-25T10:02:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59141610f0 con 0x7f59141fb660 2024-04-25T10:02:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7fb3aa200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a415e720 0x7fb3a40d2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591415a420 con 0x7f59141fb660 2024-04-25T10:02:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3aa200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a40d2e80 0x7fb3a4201060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3880051c0 0x7fb3a40d1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3a8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3880051c0 0x7fb3a40d1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51474/0 (socket says 172.21.15.165:51474) 2024-04-25T10:02:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a415e720 0x7fb3a40d2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a415e720 0x7fb3a40d2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50364/0 (socket says 172.21.15.165:50364) 2024-04-25T10:02:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c017070 con 0x7fb23c0cfa10 2024-04-25T10:02:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.833+0000 7f280c000640 1 -- 172.21.15.165:0/3761514721 wait complete. 2024-04-25T10:02:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3a8e00640 1 -- 172.21.15.165:0/3374230570 learned_addr learned my addr 172.21.15.165:0/3374230570 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c005ce0 con 0x7fb23c0cfa10 2024-04-25T10:02:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c007310 con 0x7fb23c0cfa10 2024-04-25T10:02:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3aa200640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4162650 con 0x7fb3a415e720 2024-04-25T10:02:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3aa200640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a41624d0 con 0x7fb3880051c0 2024-04-25T10:02:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb3aa200640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a4163100 con 0x7fb3a40d2e80 2024-04-25T10:02:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.835+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a40d2e80 0x7fb3a4201060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c1e2b00 con 0x7fb23c0cfa10 2024-04-25T10:02:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.834+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c1e30c0 con 0x7fb23c0cfa10 2024-04-25T10:02:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.836+0000 7f280c000640 1 Processor -- start 2024-04-25T10:02:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.835+0000 7fb3a9800640 1 -- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3880051c0 msgr2=0x7fb3a40d1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.835+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3880051c0 0x7fb3a40d1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.835+0000 7fb3a9800640 1 -- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a415e720 msgr2=0x7fb3a40d2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.835+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a415e720 0x7fb3a40d2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.835+0000 7fb3a9800640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a40d24f0 con 0x7fb3a40d2e80 2024-04-25T10:02:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.836+0000 7fb3a3e00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a415e720 0x7fb3a40d2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.836+0000 7fb3a9800640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a40d2e80 0x7fb3a4201060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb38c004000 tx=0x7fb38c0012d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.836+0000 7f280c000640 1 -- start start 2024-04-25T10:02:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.836+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fa8044020 con 0x7f8fb40b95c0 2024-04-25T10:02:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.836+0000 7f8fa6a00640 1 -- 172.21.15.165:0/2821859840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8f88001030 con 0x7f8fb40b95c0 2024-04-25T10:02:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.837+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f591415a210 con 0x7f59141fb660 2024-04-25T10:02:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.837+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f590403ba50 con 0x7f59141fb660 2024-04-25T10:02:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.837+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb214001030 con 0x7fb23c0cfa10 2024-04-25T10:02:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f8fb1600640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9406b4d0 0x7f8f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f280c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f28040b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f280c000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2804071b10 0x7f280414d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0014e0 con 0x7fb3a40d2e80 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7fb3aa200640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3a40b8a80 con 0x7fb3a40d2e80 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7fb3aa200640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a4200580 con 0x7fb3a40d2e80 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7f280c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f2804161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7f2809600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f28040b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f8fa80aad50 con 0x7f8fb40b95c0 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f8fb1600640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f8f94074c40 0x7f8f94077080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 --> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8fa8032440 con 0x7f8f94074c40 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.838+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fa80b0320 con 0x7f8fb40b95c0 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7f8fb9600640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9406b4d0 0x7f8f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.839+0000 7f2809600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f28040b26d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37766/0 (socket says 172.21.15.165:37766) 2024-04-25T10:02:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f280c000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040cee40 con 0x7f2804071b10 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f280c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28040cecc0 con 0x7f280416a4b0 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f280c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040cefc0 con 0x7f280406ee00 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f280a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f2804161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f280a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f2804161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51488/0 (socket says 172.21.15.165:51488) 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f280a000640 1 -- 172.21.15.165:0/420419415 learned_addr learned my addr 172.21.15.165:0/420419415 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb22c01a070 con 0x7fb23c0cfa10 2024-04-25T10:02:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb37c001030 con 0x7fb3a40d2e80 2024-04-25T10:02:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f2808c00640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2804071b10 0x7f280414d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f8fba000640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f8f94074c40 0x7f8f94077080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.840+0000 7f8fb9600640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9406b4d0 0x7f8f9406d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009990 tx=0x7f8fa0009110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.841+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb38c040070 con 0x7fb3a40d2e80 2024-04-25T10:02:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.841+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb38c0054a0 con 0x7fb3a40d2e80 2024-04-25T10:02:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.841+0000 7f2808c00640 1 -- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 msgr2=0x7f28040b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.841+0000 7f590b400640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58f406b5d0 0x7f58f406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.841+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f59040aada0 con 0x7f59141fb660 2024-04-25T10:02:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.841+0000 7f8fba000640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f8f94074c40 0x7f8f94077080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f590b400640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f58f4074d50 0x7f58f4077190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f5913400640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58f406b5d0 0x7f58f406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f5912a00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f58f4074d50 0x7f58f4077190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 --> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5904001d30 con 0x7f58f4074d50 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f2808c00640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f28040b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f2808c00640 1 -- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 msgr2=0x7f2804161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f2808c00640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f2804161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f2808c00640 1 -- 172.21.15.165:0/420419415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280414db20 con 0x7f2804071b10 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.842+0000 7f280a000640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280416a4b0 0x7f2804161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f2809600640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280406ee00 0x7f28040b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f2808c00640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2804071b10 0x7f280414d5e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c340 tx=0x7f27f000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59040ad400 con 0x7f59141fb660 2024-04-25T10:02:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f00052d0 con 0x7f2804071b10 2024-04-25T10:02:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f0005470 con 0x7f2804071b10 2024-04-25T10:02:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f0007240 con 0x7f2804071b10 2024-04-25T10:02:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f5913400640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58f406b5d0 0x7f58f406da90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59140b6ee0 tx=0x7f590c009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.844+0000 7f5912a00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f58f4074d50 0x7f58f4077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.843+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28040b2cb0 con 0x7f2804071b10 2024-04-25T10:02:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.844+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28040b3270 con 0x7f2804071b10 2024-04-25T10:02:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.844+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== osd.2 v2:172.21.15.165:6824/3063902068 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8fa8032440 con 0x7f8f94074c40 2024-04-25T10:02:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.845+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2804073a00 con 0x7f2804071b10 2024-04-25T10:02:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.845+0000 7f2556c00640 1 Processor -- start 2024-04-25T10:02:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.846+0000 7fb233e00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb21c06b4d0 0x7fb21c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.845+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== osd.3 v2:172.21.15.165:6816/995850359 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5904001d30 con 0x7f58f4074d50 2024-04-25T10:02:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.846+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb22c089f80 con 0x7fb23c0cfa10 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.846+0000 7fb233e00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fb21c074c50 0x7fb21c077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.846+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb21c077760 con 0x7fb21c074c50 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.846+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb22c08a370 con 0x7fb23c0cfa10 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.847+0000 7fb240a00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fb21c074c50 0x7fb21c077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.847+0000 7fb23be00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb21c06b4d0 0x7fb21c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.845+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb38c0067d0 con 0x7fb3a40d2e80 2024-04-25T10:02:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.847+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27f00073e0 con 0x7f2804071b10 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.846+0000 7f2556c00640 1 -- start start 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7fb3a0c00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb38006b6c0 0x7fb38006db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f2556c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2550161fc0 0x7f25501623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb38c0aa690 con 0x7fb3a40d2e80 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7fb23be00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb21c06b4d0 0x7fb21c06d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb228035000 tx=0x7fb22802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f2556c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255006d2d0 0x7f255006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f2556c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25501628e0 con 0x7f255006ea90 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f2556c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2550071b90 con 0x7f2550161fc0 2024-04-25T10:02:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f2556c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255006dc80 con 0x7f255006d2d0 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f254fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2550161fc0 0x7f25501623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f254fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2550161fc0 0x7f25501623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50378/0 (socket says 172.21.15.165:50378) 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7f254fe00640 1 -- 172.21.15.165:0/3063797751 learned_addr learned my addr 172.21.15.165:0/3063797751 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7f2554c00640 1 --2- 172.21.15.165:0/3063797751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255006d2d0 0x7f255006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7fb3a0c00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fb380074e40 0x7fb380077280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.848+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb38c0052f0 con 0x7fb380074e40 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7f254fe00640 1 -- 172.21.15.165:0/3063797751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255006d2d0 msgr2=0x7f255006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7f254fe00640 1 --2- 172.21.15.165:0/3063797751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255006d2d0 0x7f255006d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7f254fe00640 1 -- 172.21.15.165:0/3063797751 >> v1:172.21.15.165:6789/0 conn(0x7f255006ea90 legacy=0x7f2550071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb38c0aab70 con 0x7fb3a40d2e80 2024-04-25T10:02:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7fb240a00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fb21c074c50 0x7fb21c077090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7f2801600640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f27f00899d0 con 0x7f2804071b10 2024-04-25T10:02:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7f2801600640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f27e4074c50 0x7f27e4077090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f27f001dec0 con 0x7f27e4074c50 2024-04-25T10:02:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27f008e2a0 con 0x7f2804071b10 2024-04-25T10:02:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.849+0000 7f254fe00640 1 -- 172.21.15.165:0/3063797751 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550162ba0 con 0x7f2550161fc0 2024-04-25T10:02:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7fb3a8e00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb38006b6c0 0x7fb38006db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.850+0000 7fb3a3e00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fb380074e40 0x7fb380077280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.851+0000 7f2809600640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.851+0000 7f280a000640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f27e4074c50 0x7f27e4077090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.852+0000 7fb3a8e00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb38006b6c0 0x7fb38006db80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb39c003c50 tx=0x7fb39c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.852+0000 7fb3a3e00640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fb380074e40 0x7fb380077280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.852+0000 7f2809600640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27f8034000 tx=0x7f27f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.852+0000 7f254fe00640 1 --2- 172.21.15.165:0/3063797751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550161fc0 0x7f25501623a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2540009870 tx=0x7f254002a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=162cbdad4dfef50d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.852+0000 7f254f400640 1 -- 172.21.15.165:0/3063797751 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540037070 con 0x7f2550161fc0 2024-04-25T10:02:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.853+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== osd.5 v2:172.21.15.172:6800/4199573342 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb21c077760 con 0x7fb21c074c50 2024-04-25T10:02:28.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.853+0000 7f254f400640 1 -- 172.21.15.165:0/3063797751 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2540032440 con 0x7f2550161fc0 2024-04-25T10:02:28.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.853+0000 7f254f400640 1 -- 172.21.15.165:0/3063797751 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540032810 con 0x7f2550161fc0 2024-04-25T10:02:28.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.853+0000 7f280a000640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f27e4074c50 0x7f27e4077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 --> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6ef0052c50 con 0x7f6ed8074d20 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f2556c00640 1 -- 172.21.15.165:0/3063797751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550161fc0 msgr2=0x7f25501623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f2556c00640 1 --2- 172.21.15.165:0/3063797751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550161fc0 0x7f25501623a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2540009870 tx=0x7f254002a1d0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f2556c00640 1 -- 172.21.15.165:0/3063797751 shutdown_connections 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f2556c00640 1 --2- 172.21.15.165:0/3063797751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f255006d2d0 0x7f255006d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f2556c00640 1 --2- 172.21.15.165:0/3063797751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2550161fc0 0x7f25501623a0 secure :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2540009870 tx=0x7f254002a1d0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7f2556c00640 1 -- 172.21.15.165:0/3063797751 >> 172.21.15.165:0/3063797751 conn(0x7f255006abe0 msgr2=0x7f255006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.854+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== osd.4 v2:172.21.15.172:6817/928908638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb38c0052f0 con 0x7fb380074e40 2024-04-25T10:02:28.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.855+0000 7f6ef5600640 1 -- 172.21.15.165:0/4939479 <== osd.1 v2:172.21.15.165:6808/4150887630 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6ef0052c50 con 0x7f6ed8074d20 2024-04-25T10:02:28.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.855+0000 7f2556c00640 1 -- 172.21.15.165:0/3063797751 shutdown_connections 2024-04-25T10:02:28.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.855+0000 7f2556c00640 1 -- 172.21.15.165:0/3063797751 wait complete. 2024-04-25T10:02:28.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.855+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== osd.6 v2:172.21.15.172:6808/4001543510 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f27f001dec0 con 0x7f27e4074c50 2024-04-25T10:02:28.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7f6ed8074d20 msgr2=0x7f6ed8077160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7f6ed8074d20 0x7f6ed8077160 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f2556c00640 1 Processor -- start 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f2556c00640 1 -- start start 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ed806b5a0 msgr2=0x7f6ed806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ed806b5a0 0x7f6ed806da60 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ef00489e0 tx=0x7f6ee4008040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef00073e0 msgr2=0x7f6ef00df3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 --2- 172.21.15.165:0/4939479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ef00073e0 0x7f6ef00df3f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6ef806a560 tx=0x7f6ef8068960 comp rx=0 tx=0).stop 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efd400640 1 -- 172.21.15.165:0/4939479 reap_dead start 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 shutdown_connections 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.858+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 >> 172.21.15.165:0/4939479 conn(0x7f6ef0054c90 msgr2=0x7f6ef004a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.859+0000 7f2556c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255006d2d0 0x7f25501f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.859+0000 7f2556c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255006ea90 0x7f2550155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.859+0000 7f2556c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25501fb980 0x7f25501fbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.860+0000 7f2547400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255006ea90 0x7f2550155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.860+0000 7f254fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255006d2d0 0x7f25501f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.860+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 shutdown_connections 2024-04-25T10:02:28.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.860+0000 7f2547400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255006ea90 0x7f2550155580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50394/0 (socket says 172.21.15.165:50394) 2024-04-25T10:02:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.860+0000 7f2547400640 1 -- 172.21.15.165:0/3640173828 learned_addr learned my addr 172.21.15.165:0/3640173828 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.860+0000 7f2554c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25501fb980 0x7f25501fbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.861+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550163160 con 0x7f255006ea90 2024-04-25T10:02:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.861+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2550163440 con 0x7f255006d2d0 2024-04-25T10:02:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.861+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f255006ef40 con 0x7f25501fb980 2024-04-25T10:02:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.861+0000 7f6efe800640 1 -- 172.21.15.165:0/4939479 wait complete. 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.862+0000 7f2547400640 1 -- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25501fb980 msgr2=0x7f25501fbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.862+0000 7f2547400640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25501fb980 0x7f25501fbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.862+0000 7f2547400640 1 -- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255006d2d0 msgr2=0x7f25501f76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.862+0000 7f2547400640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255006d2d0 0x7f25501f76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.862+0000 7f2547400640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2550158240 con 0x7f255006ea90 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.862+0000 7f2554c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25501fb980 0x7f25501fbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.864+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 --> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3bc033330 con 0x7fc3a4074d10 2024-04-25T10:02:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.865+0000 7fc3c0c00640 1 -- 172.21.15.165:0/1852882350 <== osd.0 v2:172.21.15.165:6800/4162285384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc3bc033330 con 0x7fc3a4074d10 2024-04-25T10:02:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fc3a4074d10 msgr2=0x7fc3a4077150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3cba00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fc3a4074d10 0x7fc3a4077150 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3a406b5a0 msgr2=0x7fc3a406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3cba00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3a406b5a0 0x7fc3a406da60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc3b8035000 tx=0x7fc3b800c040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3bc04f280 msgr2=0x7fc3bc19f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3cba00640 1 --2- 172.21.15.165:0/1852882350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3bc04f280 0x7fc3bc19f1e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3c404f440 tx=0x7fc3c4067fb0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.867+0000 7fc3c9a00640 1 -- 172.21.15.165:0/1852882350 reap_dead start 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.868+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 shutdown_connections 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.868+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 >> 172.21.15.165:0/1852882350 conn(0x7fc3bc009fa0 msgr2=0x7fc3bc04d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.865+0000 7f2547400640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255006ea90 0x7f2550155580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f253800c7c0 tx=0x7f253800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.866+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538003b90 con 0x7f255006ea90 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.866+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25501584e0 con 0x7f255006ea90 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.866+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2538003d30 con 0x7f255006ea90 2024-04-25T10:02:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.866+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2538003b90 con 0x7f255006ea90 2024-04-25T10:02:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.866+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2550158aa0 con 0x7f255006ea90 2024-04-25T10:02:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.868+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2550073a00 con 0x7f255006ea90 2024-04-25T10:02:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.868+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 shutdown_connections 2024-04-25T10:02:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.868+0000 7fc3cba00640 1 -- 172.21.15.165:0/1852882350 wait complete. 2024-04-25T10:02:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.876+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f253800a850 con 0x7f255006ea90 2024-04-25T10:02:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.880+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f28040cdc40 con 0x7f27e4074c50 2024-04-25T10:02:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.880+0000 7f254cc00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252c06b700 0x7f252c06dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.881+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f253808aa60 con 0x7f255006ea90 2024-04-25T10:02:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.881+0000 7f254cc00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7f252c074e80 0x7f252c0772c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.881+0000 7f8fa6a00640 1 -- 172.21.15.165:0/2821859840 --> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8f88002dd0 con 0x7f8f94074c40 2024-04-25T10:02:28.906 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.881+0000 7f2801600640 1 -- 172.21.15.165:0/420419415 <== osd.6 v2:172.21.15.172:6808/4001543510 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f28040cdc40 con 0x7f27e4074c50 2024-04-25T10:02:28.907 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.881+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f252c077990 con 0x7f252c074e80 2024-04-25T10:02:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.881+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f253808ae50 con 0x7f255006ea90 2024-04-25T10:02:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.882+0000 7f254fe00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252c06b700 0x7f252c06dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.882+0000 7f2554c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7f252c074e80 0x7f252c0772c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.882+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f27e4074c50 msgr2=0x7f27e4077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.882+0000 7f280c000640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f27e4074c50 0x7f27e4077090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.882+0000 7f8fb1600640 1 -- 172.21.15.165:0/2821859840 <== osd.2 v2:172.21.15.165:6824/3063902068 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8f88002dd0 con 0x7f8f94074c40 2024-04-25T10:02:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 msgr2=0x7f27e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280c000640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f27f8034000 tx=0x7f27f802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2804071b10 msgr2=0x7f280414d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280c000640 1 --2- 172.21.15.165:0/420419415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2804071b10 0x7f280414d5e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f27f000c340 tx=0x7f27f000c800 comp rx=0 tx=0).stop 2024-04-25T10:02:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280a000640 1 -- 172.21.15.165:0/420419415 reap_dead start 2024-04-25T10:02:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 shutdown_connections 2024-04-25T10:02:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f2554c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7f252c074e80 0x7f252c0772c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 >> 172.21.15.165:0/420419415 conn(0x7f280406abe0 msgr2=0x7f2804167930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f254fe00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252c06b700 0x7f252c06dbc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2540034000 tx=0x7f254002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f8f94074c40 msgr2=0x7f8f94077080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.883+0000 7f8fbaa00640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f8f94074c40 0x7f8f94077080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9406b4d0 msgr2=0x7f8f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9406b4d0 0x7f8f9406d990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f8fa0009990 tx=0x7f8fa0009110 comp rx=0 tx=0).stop 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb40b95c0 msgr2=0x7f8fb40d4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 --2- 172.21.15.165:0/2821859840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb40b95c0 0x7f8fb40d4110 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8fa80090b0 tx=0x7f8fa80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fba000640 1 -- 172.21.15.165:0/2821859840 reap_dead start 2024-04-25T10:02:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 --> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5914073990 con 0x7f58f4074d50 2024-04-25T10:02:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 shutdown_connections 2024-04-25T10:02:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 >> 172.21.15.165:0/2821859840 conn(0x7f8fb40d03d0 msgr2=0x7f8fb40d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.885+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 shutdown_connections 2024-04-25T10:02:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.885+0000 7f280c000640 1 -- 172.21.15.165:0/420419415 wait complete. 2024-04-25T10:02:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 shutdown_connections 2024-04-25T10:02:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.884+0000 7f8fbaa00640 1 -- 172.21.15.165:0/2821859840 wait complete. 2024-04-25T10:02:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.885+0000 7f590b400640 1 -- 172.21.15.165:0/2976109546 <== osd.3 v2:172.21.15.165:6816/995850359 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5914073990 con 0x7f58f4074d50 2024-04-25T10:02:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.886+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== osd.7 v2:172.21.15.172:6816/3337160074 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f252c077990 con 0x7f252c074e80 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.886+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f58f4074d50 msgr2=0x7f58f4077190 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.886+0000 7f5918a00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f58f4074d50 0x7f58f4077190 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.886+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58f406b5d0 msgr2=0x7f58f406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.886+0000 7f5918a00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58f406b5d0 0x7f58f406da90 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59140b6ee0 tx=0x7f590c009250 comp rx=0 tx=0).stop 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.887+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59141fb660 msgr2=0x7f59141fbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.887+0000 7f5918a00640 1 --2- 172.21.15.165:0/2976109546 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59141fb660 0x7f59141fbb10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5904035040 tx=0x7f59040046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.887+0000 7f5913e00640 1 -- 172.21.15.165:0/2976109546 reap_dead start 2024-04-25T10:02:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.888+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 shutdown_connections 2024-04-25T10:02:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.888+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 --> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb37c002dd0 con 0x7fb380074e40 2024-04-25T10:02:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.888+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 >> 172.21.15.165:0/2976109546 conn(0x7f59140d0610 msgr2=0x7f59140b4d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.888+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 shutdown_connections 2024-04-25T10:02:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.888+0000 7f5918a00640 1 -- 172.21.15.165:0/2976109546 wait complete. 2024-04-25T10:02:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.889+0000 7fb3a0c00640 1 -- 172.21.15.165:0/3374230570 <== osd.4 v2:172.21.15.172:6817/928908638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb37c002dd0 con 0x7fb380074e40 2024-04-25T10:02:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fb380074e40 msgr2=0x7fb380077280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fb380074e40 0x7fb380077280 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb38006b6c0 msgr2=0x7fb38006db80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb38006b6c0 0x7fb38006db80 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fb39c003c50 tx=0x7fb39c00b040 comp rx=0 tx=0).stop 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a40d2e80 msgr2=0x7fb3a4201060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 --2- 172.21.15.165:0/3374230570 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a40d2e80 0x7fb3a4201060 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb38c004000 tx=0x7fb38c0012d0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb3a9800640 1 -- 172.21.15.165:0/3374230570 reap_dead start 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 shutdown_connections 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.890+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 >> 172.21.15.165:0/3374230570 conn(0x7fb3a40d03d0 msgr2=0x7fb3a40cac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.892+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 shutdown_connections 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.892+0000 7fb39a000640 1 -- 172.21.15.165:0/3374230570 wait complete. 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.893+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 --> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb214002dd0 con 0x7fb21c074c50 2024-04-25T10:02:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.895+0000 7fb233e00640 1 -- 172.21.15.165:0/3598210879 <== osd.5 v2:172.21.15.172:6800/4199573342 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb214002dd0 con 0x7fb21c074c50 2024-04-25T10:02:28.918 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fb21c074c50 msgr2=0x7fb21c077090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb242a00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fb21c074c50 0x7fb21c077090 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb21c06b4d0 msgr2=0x7fb21c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb242a00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb21c06b4d0 0x7fb21c06d990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb228035000 tx=0x7fb22802f000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 msgr2=0x7fb23c1f4d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb242a00640 1 --2- 172.21.15.165:0/3598210879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb23c0cfa10 0x7fb23c1f4d50 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c3e0 tx=0x7fb22c00c8a0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.896+0000 7fb240a00640 1 -- 172.21.15.165:0/3598210879 reap_dead start 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.897+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 shutdown_connections 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.897+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 >> 172.21.15.165:0/3598210879 conn(0x7fb23c06abe0 msgr2=0x7fb23c0cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.897+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 shutdown_connections 2024-04-25T10:02:28.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.897+0000 7fb242a00640 1 -- 172.21.15.165:0/3598210879 wait complete. 2024-04-25T10:02:28.921 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:28.922 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:28.922 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.923 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:02:28.923 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.924 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:28.925 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.925 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:28.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.916+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 --> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25501fb2c0 con 0x7f252c074e80 2024-04-25T10:02:28.926 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.917+0000 7f254cc00640 1 -- 172.21.15.165:0/3640173828 <== osd.7 v2:172.21.15.172:6816/3337160074 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f25501fb2c0 con 0x7f252c074e80 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7f252c074e80 msgr2=0x7f252c0772c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7f252c074e80 0x7f252c0772c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252c06b700 msgr2=0x7f252c06dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252c06b700 0x7f252c06dbc0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2540034000 tx=0x7f254002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255006ea90 msgr2=0x7f2550155580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 --2- 172.21.15.165:0/3640173828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f255006ea90 0x7f2550155580 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f253800c7c0 tx=0x7f253800cc80 comp rx=0 tx=0).stop 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2554c00640 1 -- 172.21.15.165:0/3640173828 reap_dead start 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 shutdown_connections 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 >> 172.21.15.165:0/3640173828 conn(0x7f255006abe0 msgr2=0x7f25500b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 shutdown_connections 2024-04-25T10:02:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:28.918+0000 7f2556c00640 1 -- 172.21.15.165:0/3640173828 wait complete. 2024-04-25T10:02:28.928 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:28.939 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:28.939 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.177+0000 7f8052a00640 1 Processor -- start 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.178+0000 7f8052a00640 1 -- start start 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.180+0000 7f8052a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8044034400 0x7f8044053630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.180+0000 7f8052a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8044034be0 con 0x7f8044053b70 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.180+0000 7f8052a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8044034900 con 0x7f80440e76e0 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.180+0000 7f8052a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8044054040 con 0x7f8044034400 2024-04-25T10:02:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.180+0000 7f8050a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8044053b70 0x7f80440e7ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58076/0 (socket says 172.21.15.165:58076) 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.180+0000 7f8050a00640 1 -- 172.21.15.165:0/2907244766 learned_addr learned my addr 172.21.15.165:0/2907244766 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.181+0000 7f804b400640 1 --2- 172.21.15.165:0/2907244766 >> v2:172.21.15.165:3300/0 conn(0x7f8044034400 0x7f8044053630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.181+0000 7f804aa00640 1 -- 172.21.15.165:0/2907244766 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970601579 0 0) 0x7f8044034be0 con 0x7f8044053b70 2024-04-25T10:02:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.182+0000 7f804aa00640 1 -- 172.21.15.165:0/2907244766 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8034003520 con 0x7f8044053b70 2024-04-25T10:02:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.183+0000 7f804b400640 1 -- 172.21.15.165:0/2907244766 >> v1:172.21.15.172:6789/0 conn(0x7f80440e76e0 legacy=0x7f8044052f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.183+0000 7f804aa00640 1 -- 172.21.15.165:0/2907244766 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228907023 0 0) 0x7f8044034900 con 0x7f80440e76e0 2024-04-25T10:02:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.183+0000 7f804b400640 1 -- 172.21.15.165:0/2907244766 >> v1:172.21.15.165:6789/0 conn(0x7f8044053b70 legacy=0x7f80440e7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.184+0000 7f804b400640 1 -- 172.21.15.165:0/2907244766 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8044054320 con 0x7f8044034400 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.186+0000 7f804b400640 1 --2- 172.21.15.165:0/2907244766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044034400 0x7f8044053630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f804c02d370 tx=0x7f804c0a5110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaba6bb1bf02b8b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.186+0000 7f804aa00640 1 -- 172.21.15.165:0/2907244766 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c0b2070 con 0x7f8044034400 2024-04-25T10:02:29.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.186+0000 7f804aa00640 1 -- 172.21.15.165:0/2907244766 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c0a5c60 con 0x7f8044034400 2024-04-25T10:02:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.187+0000 7f8052a00640 1 -- 172.21.15.165:0/2907244766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044034400 msgr2=0x7f8044053630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.187+0000 7f8052a00640 1 --2- 172.21.15.165:0/2907244766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044034400 0x7f8044053630 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f804c02d370 tx=0x7f804c0a5110 comp rx=0 tx=0).stop 2024-04-25T10:02:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.188+0000 7f8052a00640 1 -- 172.21.15.165:0/2907244766 shutdown_connections 2024-04-25T10:02:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.188+0000 7f8052a00640 1 --2- 172.21.15.165:0/2907244766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044034400 0x7f8044053630 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.189+0000 7f8052a00640 1 -- 172.21.15.165:0/2907244766 >> 172.21.15.165:0/2907244766 conn(0x7f8044055350 msgr2=0x7f8044055760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.189+0000 7f8052a00640 1 -- 172.21.15.165:0/2907244766 shutdown_connections 2024-04-25T10:02:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.189+0000 7f8052a00640 1 -- 172.21.15.165:0/2907244766 wait complete. 2024-04-25T10:02:29.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.193+0000 7f8052a00640 1 Processor -- start 2024-04-25T10:02:29.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.194+0000 7f8052a00640 1 -- start start 2024-04-25T10:02:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.195+0000 7f8052a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8044034400 0x7f8044184fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.195+0000 7f8052a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 0x7f8044052f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.195+0000 7f8052a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80440e76e0 0x7f8044053440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.195+0000 7f8052a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044050c30 con 0x7f8044053b70 2024-04-25T10:02:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.195+0000 7f8052a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044050ab0 con 0x7f8044034400 2024-04-25T10:02:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.195+0000 7f8052a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8044050db0 con 0x7f80440e76e0 2024-04-25T10:02:29.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.196+0000 7f804b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 0x7f8044052f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.197+0000 7f804b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 0x7f8044052f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50412/0 (socket says 172.21.15.165:50412) 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.197+0000 7f8050a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80440e76e0 0x7f8044053440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.197+0000 7f8050a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80440e76e0 0x7f8044053440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37782/0 (socket says 172.21.15.165:37782) 2024-04-25T10:02:29.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.197+0000 7f8050a00640 1 -- 172.21.15.165:0/1575045446 learned_addr learned my addr 172.21.15.165:0/1575045446 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.198+0000 7f804b400640 1 -- 172.21.15.165:0/1575045446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80440e76e0 msgr2=0x7f8044053440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.198+0000 7f804b400640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80440e76e0 0x7f8044053440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.198+0000 7f804b400640 1 -- 172.21.15.165:0/1575045446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8044034400 msgr2=0x7f8044184fc0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.198+0000 7f804b400640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8044034400 0x7f8044184fc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.198+0000 7f804b400640 1 -- 172.21.15.165:0/1575045446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804403a5e0 con 0x7f8044053b70 2024-04-25T10:02:29.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.199+0000 7f804b400640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 0x7f8044052f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f804c06a560 tx=0x7f804c073660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.199+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c0b2050 con 0x7f8044053b70 2024-04-25T10:02:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.201+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804c0b3040 con 0x7f8044053b70 2024-04-25T10:02:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.201+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804c0acc50 con 0x7f8044053b70 2024-04-25T10:02:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.201+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804403a8c0 con 0x7f8044053b70 2024-04-25T10:02:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.201+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8044035160 con 0x7f8044053b70 2024-04-25T10:02:29.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.203+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80440057e0 con 0x7f8044053b70 2024-04-25T10:02:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.205+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f804c066870 con 0x7f8044053b70 2024-04-25T10:02:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.207+0000 7f803fe00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802c06b520 0x7f802c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.207+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f804c125080 con 0x7f8044053b70 2024-04-25T10:02:29.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.208+0000 7f804be00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802c06b520 0x7f802c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.209+0000 7f804be00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802c06b520 0x7f802c06d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040007950 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.210+0000 7f462a400640 1 Processor -- start 2024-04-25T10:02:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.211+0000 7f462a400640 1 -- start start 2024-04-25T10:02:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.213+0000 7f462a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f46240b6aa0 0x7f46240d0240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.213+0000 7f462a400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46240d0780 0x7f46240d25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.213+0000 7f462a400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46240ce7e0 con 0x7f462415db10 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.213+0000 7f462a400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240d0bb0 con 0x7f46240b6aa0 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.213+0000 7f462a400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46240ceab0 con 0x7f46240d0780 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f4623400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f462415db10 0x7f46240ce0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59494/0 (socket says 172.21.15.165:59494) 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f4623400640 1 -- 172.21.15.165:0/3000209819 learned_addr learned my addr 172.21.15.165:0/3000209819 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f4622a00640 1 --2- 172.21.15.165:0/3000209819 >> v2:172.21.15.165:3300/0 conn(0x7f46240b6aa0 0x7f46240d0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f4623e00640 1 --2- 172.21.15.165:0/3000209819 >> v2:172.21.15.172:3300/0 conn(0x7f46240d0780 0x7f46240d25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f75dde00640 1 Processor -- start 2024-04-25T10:02:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f75dde00640 1 -- start start 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f4622000640 1 -- 172.21.15.165:0/3000209819 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3816928475 0 0) 0x7f46240ce7e0 con 0x7f462415db10 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.214+0000 7f4622000640 1 -- 172.21.15.165:0/3000209819 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4608003520 con 0x7f462415db10 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.215+0000 7f4622a00640 1 -- 172.21.15.165:0/3000209819 >> v2:172.21.15.172:3300/0 conn(0x7f46240d0780 msgr2=0x7f46240d25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.215+0000 7f4622a00640 1 --2- 172.21.15.165:0/3000209819 >> v2:172.21.15.172:3300/0 conn(0x7f46240d0780 0x7f46240d25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.215+0000 7f4622a00640 1 -- 172.21.15.165:0/3000209819 >> v1:172.21.15.172:6789/0 conn(0x7f462415db10 legacy=0x7f46240ce0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.215+0000 7f4622a00640 1 -- 172.21.15.165:0/3000209819 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46240ced90 con 0x7f46240b6aa0 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.215+0000 7f763de00640 1 Processor -- start 2024-04-25T10:02:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.216+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f804c0f3ba0 con 0x7f8044053b70 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.216+0000 7f763de00640 1 -- start start 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.216+0000 7f4622a00640 1 --2- 172.21.15.165:0/3000209819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240b6aa0 0x7f46240d0240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46240ca8f0 tx=0x7f461002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40688e249ef70cb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f75dde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d80ce6d0 0x7f75d80ceb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f75dde00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75d80d2370 0x7f75d80d2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f75dde00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75d8162010 con 0x7f75d815d9d0 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f75dde00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75d80cf070 con 0x7f75d80d2370 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f75dde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80d1ee0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f4622000640 1 -- 172.21.15.165:0/3000209819 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610037070 con 0x7f46240b6aa0 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f4622000640 1 -- 172.21.15.165:0/3000209819 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4610032440 con 0x7f46240b6aa0 2024-04-25T10:02:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f4622000640 1 -- 172.21.15.165:0/3000209819 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461002ae60 con 0x7f46240b6aa0 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f763de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7638161240 0x7f76380b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f763de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76380b54a0 0x7f76380b5930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f763de00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76380cf1d0 con 0x7f7638161890 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f763de00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76380b7120 con 0x7f7638161240 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f763de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7638161620 con 0x7f76380b54a0 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.217+0000 7f75dca00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f75d815d9d0 0x7f75d8161590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59504/0 (socket says 172.21.15.165:59504) 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.218+0000 7f75dd400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75d80d2370 0x7f75d80d2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.218+0000 7f75d7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d80ce6d0 0x7f75d80ceb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.218+0000 7f7636000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7638161890 0x7f76380ceac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59516/0 (socket says 172.21.15.165:59516) 2024-04-25T10:02:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f7636000640 1 -- 172.21.15.165:0/71000311 learned_addr learned my addr 172.21.15.165:0/71000311 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f7637400640 1 --2- 172.21.15.165:0/71000311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76380b54a0 0x7f76380b5930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.218+0000 7f75dca00640 1 -- 172.21.15.165:0/1203348706 learned_addr learned my addr 172.21.15.165:0/1203348706 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f75d7e00640 1 -- 172.21.15.165:0/1203348706 >> v2:172.21.15.172:3300/0 conn(0x7f75d80d2370 msgr2=0x7f75d80d2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f75d7e00640 1 --2- 172.21.15.165:0/1203348706 >> v2:172.21.15.172:3300/0 conn(0x7f75d80d2370 0x7f75d80d2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f75d7e00640 1 -- 172.21.15.165:0/1203348706 >> v1:172.21.15.172:6789/0 conn(0x7f75d815d9d0 legacy=0x7f75d8161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f75d7e00640 1 -- 172.21.15.165:0/1203348706 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d81622f0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f462a400640 1 -- 172.21.15.165:0/3000209819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240b6aa0 msgr2=0x7f46240d0240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f462a400640 1 --2- 172.21.15.165:0/3000209819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240b6aa0 0x7f46240d0240 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f46240ca8f0 tx=0x7f461002a110 comp rx=0 tx=0).stop 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f462a400640 1 -- 172.21.15.165:0/3000209819 shutdown_connections 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f462a400640 1 --2- 172.21.15.165:0/3000209819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240b6aa0 0x7f46240d0240 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f462a400640 1 --2- 172.21.15.165:0/3000209819 >> v2:172.21.15.172:3300/0 conn(0x7f46240d0780 0x7f46240d25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f462a400640 1 -- 172.21.15.165:0/3000209819 >> 172.21.15.165:0/3000209819 conn(0x7f462406bef0 msgr2=0x7f462406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.220+0000 7f462a400640 1 -- 172.21.15.165:0/3000209819 shutdown_connections 2024-04-25T10:02:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f7637400640 1 -- 172.21.15.165:0/71000311 >> v2:172.21.15.165:3300/0 conn(0x7f7638161240 msgr2=0x7f76380b6be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.220+0000 7f6c67600640 1 Processor -- start 2024-04-25T10:02:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.220+0000 7f6c67600640 1 -- start start 2024-04-25T10:02:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.220+0000 7f75d7e00640 1 --2- 172.21.15.165:0/1203348706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d80ce6d0 0x7f75d80ceb30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f75c40097a0 tx=0x7f75c402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=711aaeb2591d935c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f7637400640 1 --2- 172.21.15.165:0/71000311 >> v2:172.21.15.165:3300/0 conn(0x7f7638161240 0x7f76380b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f7637400640 1 -- 172.21.15.165:0/71000311 >> v1:172.21.15.172:6789/0 conn(0x7f7638161890 legacy=0x7f76380ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.219+0000 7f7637400640 1 -- 172.21.15.165:0/71000311 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7638161e00 con 0x7f76380b54a0 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.220+0000 7f462a400640 1 -- 172.21.15.165:0/3000209819 wait complete. 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.221+0000 7f75d7400640 1 -- 172.21.15.165:0/1203348706 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4037070 con 0x7f75d80ce6d0 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.221+0000 7f75d7400640 1 -- 172.21.15.165:0/1203348706 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c4032440 con 0x7f75d80ce6d0 2024-04-25T10:02:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.221+0000 7f75d7400640 1 -- 172.21.15.165:0/1203348706 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4032810 con 0x7f75d80ce6d0 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.222+0000 7f7637400640 1 --2- 172.21.15.165:0/71000311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76380b54a0 0x7f76380b5930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76240097a0 tx=0x7f762402a1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77e2ce83373ebfbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.222+0000 7f7635600640 1 -- 172.21.15.165:0/71000311 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624037070 con 0x7f76380b54a0 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.222+0000 7f75dde00640 1 -- 172.21.15.165:0/1203348706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d80ce6d0 msgr2=0x7f75d80ceb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.222+0000 7f75dde00640 1 --2- 172.21.15.165:0/1203348706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d80ce6d0 0x7f75d80ceb30 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f75c40097a0 tx=0x7f75c402a180 comp rx=0 tx=0).stop 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f75dde00640 1 -- 172.21.15.165:0/1203348706 shutdown_connections 2024-04-25T10:02:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f75dde00640 1 --2- 172.21.15.165:0/1203348706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d80ce6d0 0x7f75d80ceb30 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f75dde00640 1 --2- 172.21.15.165:0/1203348706 >> v2:172.21.15.172:3300/0 conn(0x7f75d80d2370 0x7f75d80d2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f75dde00640 1 -- 172.21.15.165:0/1203348706 >> 172.21.15.165:0/1203348706 conn(0x7f75d80d0380 msgr2=0x7f75d80d0790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f462a400640 1 Processor -- start 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f6c67600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c60148570 0x7f6c600cbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f6c67600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6c6014b780 0x7f6c600cc350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f6c67600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c60166850 con 0x7f6c600d24f0 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f6c67600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c600cf390 con 0x7f6c6014b780 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f6c67600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600cf670 con 0x7f6c60148570 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f462a400640 1 -- start start 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f7635600640 1 -- 172.21.15.165:0/71000311 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624032440 con 0x7f76380b54a0 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.224+0000 7f7635600640 1 -- 172.21.15.165:0/71000311 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624032850 con 0x7f76380b54a0 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.224+0000 7f6c65600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6c6014b780 0x7f6c600cc350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.224+0000 7f6c65600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6c6014b780 0x7f6c600cc350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50426/0 (socket says 172.21.15.165:50426) 2024-04-25T10:02:29.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.224+0000 7f6c65600640 1 -- 172.21.15.165:0/3573292068 learned_addr learned my addr 172.21.15.165:0/3573292068 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f2110600640 1 Processor -- start 2024-04-25T10:02:29.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.224+0000 7f2110600640 1 -- start start 2024-04-25T10:02:29.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.223+0000 7f75dde00640 1 -- 172.21.15.165:0/1203348706 shutdown_connections 2024-04-25T10:02:29.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.224+0000 7f75dde00640 1 -- 172.21.15.165:0/1203348706 wait complete. 2024-04-25T10:02:29.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f462a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46240b6aa0 0x7f4624152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f462a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 0x7f4624149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f462a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f462415db10 0x7f4624149990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f763de00640 1 -- 172.21.15.165:0/71000311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76380b54a0 msgr2=0x7f76380b5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f462a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240cf070 con 0x7f46240d0780 2024-04-25T10:02:29.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f763de00640 1 --2- 172.21.15.165:0/71000311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76380b54a0 0x7f76380b5930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f76240097a0 tx=0x7f762402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f462a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46240d20c0 con 0x7f46240b6aa0 2024-04-25T10:02:29.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f462a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f462415def0 con 0x7f462415db10 2024-04-25T10:02:29.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f6c5f400640 1 -- 172.21.15.165:0/3573292068 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2587672567 0 0) 0x7f6c60166850 con 0x7f6c600d24f0 2024-04-25T10:02:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.225+0000 7f4622a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 0x7f4624149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 0x7f4624149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50428/0 (socket says 172.21.15.165:50428) 2024-04-25T10:02:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 -- 172.21.15.165:0/3541114780 learned_addr learned my addr 172.21.15.165:0/3541114780 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4623400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46240b6aa0 0x7f4624152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f763de00640 1 -- 172.21.15.165:0/71000311 shutdown_connections 2024-04-25T10:02:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f2110600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f210806d2d0 0x7f210806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4623e00640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f462415db10 0x7f4624149990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f2110600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2108162ba0 con 0x7f2108161fc0 2024-04-25T10:02:29.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f2110600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2108162e80 con 0x7f210806ea90 2024-04-25T10:02:29.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f763de00640 1 --2- 172.21.15.165:0/71000311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76380b54a0 0x7f76380b5930 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f2110600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2108071b90 con 0x7f210806d2d0 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f763de00640 1 --2- 172.21.15.165:0/71000311 >> v2:172.21.15.165:3300/0 conn(0x7f7638161240 0x7f76380b6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f763de00640 1 -- 172.21.15.165:0/71000311 >> 172.21.15.165:0/71000311 conn(0x7f76380d04a0 msgr2=0x7f76380d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 -- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f462415db10 msgr2=0x7f4624149990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f462415db10 0x7f4624149990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 -- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46240b6aa0 msgr2=0x7f4624152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46240b6aa0 0x7f4624152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4622a00640 1 -- 172.21.15.165:0/3541114780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4624160520 con 0x7f46240d0780 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f4623400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46240b6aa0 0x7f4624152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f6c5f400640 1 -- 172.21.15.165:0/3573292068 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c48003520 con 0x7f6c600d24f0 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f6c64c00640 1 --2- 172.21.15.165:0/3573292068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c60148570 0x7f6c600cbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f6c65600640 1 -- 172.21.15.165:0/3573292068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c60148570 msgr2=0x7f6c600cbe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f6c65600640 1 --2- 172.21.15.165:0/3573292068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c60148570 0x7f6c600cbe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f6c65600640 1 -- 172.21.15.165:0/3573292068 >> v1:172.21.15.172:6789/0 conn(0x7f6c600d24f0 legacy=0x7f6c60070c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.226+0000 7f6c65600640 1 -- 172.21.15.165:0/3573292068 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c600cf950 con 0x7f6c6014b780 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f210dc00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2108161fc0 0x7f21081623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58082/0 (socket says 172.21.15.165:58082) 2024-04-25T10:02:29.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f210dc00640 1 -- 172.21.15.165:0/1391543880 learned_addr learned my addr 172.21.15.165:0/1391543880 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:29.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f4622a00640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 0x7f4624149450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f46100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f763de00640 1 -- 172.21.15.165:0/71000311 shutdown_connections 2024-04-25T10:02:29.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f210e600640 1 --2- 172.21.15.165:0/1391543880 >> v2:172.21.15.172:3300/0 conn(0x7f210806d2d0 0x7f210806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f75dde00640 1 Processor -- start 2024-04-25T10:02:29.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610041020 con 0x7f46240d0780 2024-04-25T10:02:29.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f461003d700 con 0x7f46240d0780 2024-04-25T10:02:29.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4610037040 con 0x7f46240d0780 2024-04-25T10:02:29.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.227+0000 7f75dde00640 1 -- start start 2024-04-25T10:02:29.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f763de00640 1 -- 172.21.15.165:0/71000311 wait complete. 2024-04-25T10:02:29.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f6e59000640 1 Processor -- start 2024-04-25T10:02:29.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4624160800 con 0x7f46240d0780 2024-04-25T10:02:29.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f2107e00640 1 -- 172.21.15.165:0/1391543880 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678162381 0 0) 0x7f2108162ba0 con 0x7f2108161fc0 2024-04-25T10:02:29.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f2107e00640 1 -- 172.21.15.165:0/1391543880 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20f0003520 con 0x7f2108161fc0 2024-04-25T10:02:29.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f6e59000640 1 -- start start 2024-04-25T10:02:29.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4624160210 con 0x7f46240d0780 2024-04-25T10:02:29.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f75dde00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 0x7f75d80c71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f75dde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75d80d2370 0x7f75d815a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f75dde00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d815d9d0 0x7f75d80c8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f75dde00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d8161ca0 con 0x7f75d80d2370 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f75dde00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d815ddb0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f75dde00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d80b94b0 con 0x7f75d815d9d0 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.229+0000 7f6e59000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e5415dbd0 0x7f6e54161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f6e59000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e540b95c0 0x7f6e540b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f6e59000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e54161e70 con 0x7f6e540b5910 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f6e59000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e540d1ce0 con 0x7f6e5415dbd0 2024-04-25T10:02:29.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f6e59000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e54162150 con 0x7f6e540b95c0 2024-04-25T10:02:29.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f75dca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 0x7f75d80c71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f75dca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 0x7f75d80c71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51530/0 (socket says 172.21.15.165:51530) 2024-04-25T10:02:29.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f75dca00640 1 -- 172.21.15.165:0/883595566 learned_addr learned my addr 172.21.15.165:0/883595566 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f6c65600640 1 --2- 172.21.15.165:0/3573292068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c6014b780 0x7f6c600cc350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009ad0 tx=0x7f6c5802a4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84b1468f1290fb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.230+0000 7f75dd400640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d815d9d0 0x7f75d80c8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f763de00640 1 Processor -- start 2024-04-25T10:02:29.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7fa291c00640 1 Processor -- start 2024-04-25T10:02:29.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f210e600640 1 -- 172.21.15.165:0/1391543880 >> v1:172.21.15.172:6789/0 conn(0x7f210806ea90 legacy=0x7f2108071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f210e600640 1 -- 172.21.15.165:0/1391543880 >> v1:172.21.15.165:6789/0 conn(0x7f2108161fc0 legacy=0x7f21081623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.228+0000 7f210e600640 1 -- 172.21.15.165:0/1391543880 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2108163160 con 0x7f210806d2d0 2024-04-25T10:02:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6e53e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6e540b5910 0x7f6e5406bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59550/0 (socket says 172.21.15.165:59550) 2024-04-25T10:02:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6e53e00640 1 -- 172.21.15.165:0/455388718 learned_addr learned my addr 172.21.15.165:0/455388718 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4624073990 con 0x7f46240d0780 2024-04-25T10:02:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7fa291c00640 1 -- start start 2024-04-25T10:02:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6c5f400640 1 -- 172.21.15.165:0/3573292068 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58036070 con 0x7f6c6014b780 2024-04-25T10:02:29.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6e53400640 1 --2- 172.21.15.165:0/455388718 >> v2:172.21.15.165:3300/0 conn(0x7f6e5415dbd0 0x7f6e54161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f75d7e00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75d80d2370 0x7f75d815a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f75dca00640 1 -- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d815d9d0 msgr2=0x7f75d80c8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f75dca00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d815d9d0 0x7f75d80c8900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f75dca00640 1 -- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75d80d2370 msgr2=0x7f75d815a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6c5f400640 1 -- 172.21.15.165:0/3573292068 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58031600 con 0x7f6c6014b780 2024-04-25T10:02:29.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6c5f400640 1 -- 172.21.15.165:0/3573292068 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c580319d0 con 0x7f6c6014b780 2024-04-25T10:02:29.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f763de00640 1 -- start start 2024-04-25T10:02:29.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f210e600640 1 --2- 172.21.15.165:0/1391543880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806d2d0 0x7f210806d700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2100009d90 tx=0x7f210002a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1257f3174ed19fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6e52a00640 1 --2- 172.21.15.165:0/455388718 >> v2:172.21.15.172:3300/0 conn(0x7f6e540b95c0 0x7f6e540b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6e53400640 1 -- 172.21.15.165:0/455388718 >> v2:172.21.15.172:3300/0 conn(0x7f6e540b95c0 msgr2=0x7f6e540b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.231+0000 7f6e53400640 1 --2- 172.21.15.165:0/455388718 >> v2:172.21.15.172:3300/0 conn(0x7f6e540b95c0 0x7f6e540b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f6e53400640 1 -- 172.21.15.165:0/455388718 >> v1:172.21.15.172:6789/0 conn(0x7f6e540b5910 legacy=0x7f6e5406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f6e52000640 1 -- 172.21.15.165:0/455388718 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001454101 0 0) 0x7f6e54161e70 con 0x7f6e540b5910 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f75dca00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75d80d2370 0x7f75d815a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f75dca00640 1 -- 172.21.15.165:0/883595566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80c7860 con 0x7f75d80ce6d0 2024-04-25T10:02:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f6e53400640 1 -- 172.21.15.165:0/455388718 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54162430 con 0x7f6e5415dbd0 2024-04-25T10:02:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f2107e00640 1 -- 172.21.15.165:0/1391543880 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100007740 con 0x7f210806d2d0 2024-04-25T10:02:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f2107e00640 1 -- 172.21.15.165:0/1391543880 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21000078e0 con 0x7f210806d2d0 2024-04-25T10:02:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f2107e00640 1 -- 172.21.15.165:0/1391543880 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2100007cb0 con 0x7f210806d2d0 2024-04-25T10:02:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f75d7e00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75d80d2370 0x7f75d815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f6c67600640 1 -- 172.21.15.165:0/3573292068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c6014b780 msgr2=0x7f6c600cc350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.232+0000 7f6c67600640 1 --2- 172.21.15.165:0/3573292068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c6014b780 0x7f6c600cc350 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f6c58009ad0 tx=0x7f6c5802a4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f763de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76380b54a0 0x7f76380c9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f763de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7638161240 0x7f763815faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f763de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 0x7f7638203f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f75dca00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 0x7f75d80c71d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c4b0 tx=0x7f75cc00c970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f2110600640 1 -- 172.21.15.165:0/1391543880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806d2d0 msgr2=0x7f210806d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f2110600640 1 --2- 172.21.15.165:0/1391543880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806d2d0 0x7f210806d700 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2100009d90 tx=0x7f210002a510 comp rx=0 tx=0).stop 2024-04-25T10:02:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7fa291c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0afda0 0x7fa28c0b5ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7fa291c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa28c0b6010 0x7fa28c0d04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7fa291c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa28c1621d0 con 0x7fa28c161df0 2024-04-25T10:02:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f763de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76381626a0 con 0x7f7638161240 2024-04-25T10:02:29.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f6c67600640 1 -- 172.21.15.165:0/3573292068 shutdown_connections 2024-04-25T10:02:29.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f6c67600640 1 --2- 172.21.15.165:0/3573292068 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c6014b780 0x7f6c600cc350 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f6c67600640 1 --2- 172.21.15.165:0/3573292068 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c60148570 0x7f6c600cbe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f6c67600640 1 -- 172.21.15.165:0/3573292068 >> 172.21.15.165:0/3573292068 conn(0x7f6c600d17c0 msgr2=0x7f6c600d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f763de00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76381623c0 con 0x7f76380b54a0 2024-04-25T10:02:29.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f763de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76380d15f0 con 0x7f7638161890 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7fa291c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28c1658d0 con 0x7fa28c0b6010 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7fa291c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c0cdaf0 con 0x7fa28c0afda0 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f2110600640 1 -- 172.21.15.165:0/1391543880 shutdown_connections 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f2110600640 1 --2- 172.21.15.165:0/1391543880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806d2d0 0x7f210806d700 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f2110600640 1 -- 172.21.15.165:0/1391543880 >> 172.21.15.165:0/1391543880 conn(0x7f210806abe0 msgr2=0x7f210806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f2110600640 1 -- 172.21.15.165:0/1391543880 shutdown_connections 2024-04-25T10:02:29.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f2110600640 1 -- 172.21.15.165:0/1391543880 wait complete. 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7fa291200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa28c0b6010 0x7fa28c0d04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7fa291200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa28c0b6010 0x7fa28c0d04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51550/0 (socket says 172.21.15.165:51550) 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f6c67600640 1 -- 172.21.15.165:0/3573292068 shutdown_connections 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc017070 con 0x7f75d80ce6d0 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc005ce0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc007270 con 0x7f75d80ce6d0 2024-04-25T10:02:29.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7fa291200640 1 -- 172.21.15.165:0/3799689467 learned_addr learned my addr 172.21.15.165:0/3799689467 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d80c7ba0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f7636a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76380b54a0 0x7f76380c9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f7637400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 0x7f7638203f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f7637400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 0x7f7638203f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37828/0 (socket says 172.21.15.165:37828) 2024-04-25T10:02:29.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f7637400640 1 -- 172.21.15.165:0/470733746 learned_addr learned my addr 172.21.15.165:0/470733746 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.233+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4610044020 con 0x7f46240d0780 2024-04-25T10:02:29.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f461b400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f460406b5d0 0x7f460406da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f46100aac80 con 0x7f46240d0780 2024-04-25T10:02:29.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7fa291200640 1 -- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0afda0 msgr2=0x7fa28c0b5ad0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:02:29.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7fa291200640 1 --2- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0afda0 0x7fa28c0b5ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7fa291200640 1 -- 172.21.15.165:0/3799689467 >> v1:172.21.15.165:6789/0 conn(0x7fa28c161df0 legacy=0x7fa28c0cd3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:29.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7fa291200640 1 -- 172.21.15.165:0/3799689467 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c0cdd50 con 0x7fa28c0b6010 2024-04-25T10:02:29.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.236+0000 7f4623400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f460406b5d0 0x7f460406da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.234+0000 7f6c67600640 1 -- 172.21.15.165:0/3573292068 wait complete. 2024-04-25T10:02:29.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d80c83e0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f7637400640 1 -- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76380b54a0 msgr2=0x7f76380c9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f7637400640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76380b54a0 0x7f76380c9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f7637400640 1 -- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7638161240 msgr2=0x7f763815faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f7637400640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7638161240 0x7f763815faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f7637400640 1 -- 172.21.15.165:0/470733746 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380ca0c0 con 0x7f7638161890 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.235+0000 7f7637400640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 0x7f7638203f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f76240097a0 tx=0x7f76240046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.236+0000 7f6e53400640 1 --2- 172.21.15.165:0/455388718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e5415dbd0 0x7f6e54161620 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e54161580 tx=0x7f6e4c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ba5771969c840f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.236+0000 7f2110600640 1 Processor -- start 2024-04-25T10:02:29.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.236+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624041020 con 0x7f7638161890 2024-04-25T10:02:29.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.236+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76380ca3a0 con 0x7f7638161890 2024-04-25T10:02:29.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.237+0000 7f4623400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f460406b5d0 0x7f460406da90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4614007fd0 tx=0x7f4614009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.237+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76380c9cb0 con 0x7f7638161890 2024-04-25T10:02:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.237+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7624030910 con 0x7f7638161890 2024-04-25T10:02:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.237+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7624037040 con 0x7f7638161890 2024-04-25T10:02:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.237+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d8073990 con 0x7f75d80ce6d0 2024-04-25T10:02:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.237+0000 7f6c67600640 1 Processor -- start 2024-04-25T10:02:29.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.236+0000 7f2110600640 1 -- start start 2024-04-25T10:02:29.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.238+0000 7f6c67600640 1 -- start start 2024-04-25T10:02:29.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.238+0000 7f2110600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 0x7f21080a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.238+0000 7f2110600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806ea90 0x7f2108160c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.238+0000 7f210dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 0x7f21080a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7638073a00 con 0x7f7638161890 2024-04-25T10:02:29.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.238+0000 7fa291200640 1 --2- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c0b6010 0x7fa28c0d04f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa284009e50 tx=0x7fa28402a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1a698eada97f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f210dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 0x7f21080a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50460/0 (socket says 172.21.15.165:50460) 2024-04-25T10:02:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f210dc00640 1 -- 172.21.15.165:0/3283740199 learned_addr learned my addr 172.21.15.165:0/3283740199 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f6c67600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c600d24f0 0x7f6c601ec360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f6c67600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 0x7f6c6015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2108161fc0 0x7f21080aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 0x7f6c6015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210806dc40 con 0x7f210806d2d0 2024-04-25T10:02:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.239+0000 7f210d200640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806ea90 0x7f2108160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c67600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6014b780 0x7f6c6015e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c67600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6006ba90 con 0x7f6c60148570 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c64c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c600d24f0 0x7f6c601ec360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 0x7f6c6015e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50476/0 (socket says 172.21.15.165:50476) 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c5fe00640 1 -- 172.21.15.165:0/2733830925 learned_addr learned my addr 172.21.15.165:0/2733830925 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c600d1160 con 0x7f6c600d24f0 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c60166470 con 0x7f6c6014b780 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f210e600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2108161fc0 0x7f21080aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210806ef40 con 0x7f210806ea90 2024-04-25T10:02:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210806f0c0 con 0x7f2108161fc0 2024-04-25T10:02:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75cc01a070 con 0x7f75d80ce6d0 2024-04-25T10:02:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.240+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7624044020 con 0x7f7638161890 2024-04-25T10:02:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c65600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6014b780 0x7f6c6015e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c5fe00640 1 -- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6014b780 msgr2=0x7f6c6015e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c5fe00640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6014b780 0x7f6c6015e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c5fe00640 1 -- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c600d24f0 msgr2=0x7f6c601ec360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c5fe00640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c600d24f0 0x7f6c601ec360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c5fe00640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6014a1d0 con 0x7f6c60148570 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6c65600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6014b780 0x7f6c6015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f210dc00640 1 -- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2108161fc0 msgr2=0x7f21080aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f210dc00640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2108161fc0 0x7f21080aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f210dc00640 1 -- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806ea90 msgr2=0x7f2108160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.241+0000 7f6e52000640 1 -- 172.21.15.165:0/455388718 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c007740 con 0x7f6e5415dbd0 2024-04-25T10:02:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f210dc00640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806ea90 0x7f2108160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f210dc00640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21080a8810 con 0x7f210806d2d0 2024-04-25T10:02:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f6e52000640 1 -- 172.21.15.165:0/455388718 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c0078e0 con 0x7f6e5415dbd0 2024-04-25T10:02:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f6e52000640 1 -- 172.21.15.165:0/455388718 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c007cb0 con 0x7f6e5415dbd0 2024-04-25T10:02:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f75d4c00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f46100b1080 con 0x7f46240d0780 2024-04-25T10:02:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f6c5fe00640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 0x7f6c6015e450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c430 tx=0x7f6c4c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f75cc089c90 con 0x7f75d80ce6d0 2024-04-25T10:02:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f75d7e00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f210d200640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806ea90 0x7f2108160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c017070 con 0x7f6c60148570 2024-04-25T10:02:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.242+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6014a4b0 con 0x7f6c60148570 2024-04-25T10:02:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f762f400640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f761806b4d0 0x7f761806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f6e59000640 1 -- 172.21.15.165:0/455388718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e5415dbd0 msgr2=0x7f6e54161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f6e59000640 1 --2- 172.21.15.165:0/455388718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e5415dbd0 0x7f6e54161620 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6e54161580 tx=0x7f6e4c02a680 comp rx=0 tx=0).stop 2024-04-25T10:02:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c60146420 con 0x7f6c60148570 2024-04-25T10:02:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c0052d0 con 0x7f6c60148570 2024-04-25T10:02:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c017070 con 0x7f6c60148570 2024-04-25T10:02:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f210dc00640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 0x7f21080a98b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c500 tx=0x7f20f400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4018070 con 0x7f210806d2d0 2024-04-25T10:02:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21080a8af0 con 0x7f210806d2d0 2024-04-25T10:02:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21080a90b0 con 0x7f210806d2d0 2024-04-25T10:02:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f75d7e00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75b806b4d0 0x7f75b806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f4007560 con 0x7f210806d2d0 2024-04-25T10:02:29.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f4022060 con 0x7f210806d2d0 2024-04-25T10:02:29.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.243+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f76240abb50 con 0x7f7638161890 2024-04-25T10:02:29.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f7636a00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f761806b4d0 0x7f761806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f6e59000640 1 -- 172.21.15.165:0/455388718 shutdown_connections 2024-04-25T10:02:29.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f6e59000640 1 --2- 172.21.15.165:0/455388718 >> v2:172.21.15.172:3300/0 conn(0x7f6e540b95c0 0x7f6e540b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.244+0000 7f6e59000640 1 --2- 172.21.15.165:0/455388718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e5415dbd0 0x7f6e54161620 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.245+0000 7f6e59000640 1 -- 172.21.15.165:0/455388718 >> 172.21.15.165:0/455388718 conn(0x7f6e540d03d0 msgr2=0x7f6e540d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.245+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c3c005290 con 0x7f6c60148570 2024-04-25T10:02:29.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.245+0000 7f6e59000640 1 -- 172.21.15.165:0/455388718 shutdown_connections 2024-04-25T10:02:29.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.245+0000 7f6e59000640 1 -- 172.21.15.165:0/455388718 wait complete. 2024-04-25T10:02:29.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.246+0000 7f6e59000640 1 Processor -- start 2024-04-25T10:02:29.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.247+0000 7f6e59000640 1 -- start start 2024-04-25T10:02:29.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.245+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20e4005290 con 0x7f210806d2d0 2024-04-25T10:02:29.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.247+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c4c01a070 con 0x7f6c60148570 2024-04-25T10:02:29.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.247+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20f401b070 con 0x7f210806d2d0 2024-04-25T10:02:29.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e59000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 0x7f6e541f62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.248+0000 7f7636a00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f761806b4d0 0x7f761806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7630009f20 tx=0x7f76300094f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e59000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e540b95c0 0x7f6e541f3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.248+0000 7f2105600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20e806b4d0 0x7f20e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.248+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f20f40898a0 con 0x7f210806d2d0 2024-04-25T10:02:29.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f210d200640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20e806b4d0 0x7f20e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 0x7f6e541f62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 0x7f6e541f62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50490/0 (socket says 172.21.15.165:50490) 2024-04-25T10:02:29.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e53400640 1 -- 172.21.15.165:0/403566358 learned_addr learned my addr 172.21.15.165:0/403566358 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e59000640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5415dbd0 0x7f6e54160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f210d200640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20e806b4d0 0x7f20e806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f21081ec1c0 tx=0x7f20fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e59000640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540d3180 con 0x7f6e540b5910 2024-04-25T10:02:29.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e59000640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e54161b60 con 0x7f6e540b95c0 2024-04-25T10:02:29.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.249+0000 7f6e59000640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e540d3300 con 0x7f6e5415dbd0 2024-04-25T10:02:29.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6c5cc00640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c4006b6d0 0x7f6c4006db90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6c4c089850 con 0x7f6c60148570 2024-04-25T10:02:29.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6c64c00640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c4006b6d0 0x7f6c4006db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e53e00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5415dbd0 0x7f6e54160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e53400640 1 -- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5415dbd0 msgr2=0x7f6e54160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e53400640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5415dbd0 0x7f6e54160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e52a00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e540b95c0 0x7f6e541f3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e53400640 1 -- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e540b95c0 msgr2=0x7f6e541f3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e53400640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e540b95c0 0x7f6e541f3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.250+0000 7f6e53400640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e54160aa0 con 0x7f6e540b5910 2024-04-25T10:02:29.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.251+0000 7f6c64c00640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c4006b6d0 0x7f6c4006db90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c54003aa0 tx=0x7f6c54009830 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.251+0000 7f6e53400640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 0x7f6e541f62c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e80 tx=0x7f6e4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.255+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c4c057db0 con 0x7f6c60148570 2024-04-25T10:02:29.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.256+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75d80c7ba0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.256+0000 7fa28aa00640 1 -- 172.21.15.165:0/3799689467 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa284036070 con 0x7fa28c0b6010 2024-04-25T10:02:29.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.256+0000 7fa28aa00640 1 -- 172.21.15.165:0/3799689467 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa284031600 con 0x7fa28c0b6010 2024-04-25T10:02:29.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.256+0000 7fa28aa00640 1 -- 172.21.15.165:0/3799689467 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2840319d0 con 0x7fa28c0b6010 2024-04-25T10:02:29.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.256+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f762404b030 con 0x7f7638161890 2024-04-25T10:02:29.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.257+0000 7fa291c00640 1 -- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c0b6010 msgr2=0x7fa28c0d04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.257+0000 7fa291c00640 1 --2- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c0b6010 0x7fa28c0d04f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fa284009e50 tx=0x7fa28402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.258+0000 7fa291c00640 1 -- 172.21.15.165:0/3799689467 shutdown_connections 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.258+0000 7fa291c00640 1 --2- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c0b6010 0x7fa28c0d04f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.258+0000 7fa291c00640 1 --2- 172.21.15.165:0/3799689467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0afda0 0x7fa28c0b5ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.258+0000 7fa291c00640 1 -- 172.21.15.165:0/3799689467 >> 172.21.15.165:0/3799689467 conn(0x7fa28c0b53c0 msgr2=0x7fa28c099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.258+0000 7fa291c00640 1 -- 172.21.15.165:0/3799689467 shutdown_connections 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.258+0000 7fa291c00640 1 -- 172.21.15.165:0/3799689467 wait complete. 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.261+0000 7fa291c00640 1 Processor -- start 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.261+0000 7fa291c00640 1 -- start start 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.262+0000 7fa291c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 0x7fa28c1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.262+0000 7fa291c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0b6010 0x7fa28c06f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa291c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c161df0 0x7fa28c160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa291c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c1660e0 con 0x7fa28c0afda0 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa291c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c165e00 con 0x7fa28c161df0 2024-04-25T10:02:29.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa291c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa28c166600 con 0x7fa28c0b6010 2024-04-25T10:02:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa28be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 0x7fa28c1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa28be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 0x7fa28c1623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50496/0 (socket says 172.21.15.165:50496) 2024-04-25T10:02:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.263+0000 7fa28be00640 1 -- 172.21.15.165:0/1330428918 learned_addr learned my addr 172.21.15.165:0/1330428918 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa28b400640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0b6010 0x7fa28c06f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c040070 con 0x7f6e540b5910 2024-04-25T10:02:29.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f20f4057d80 con 0x7f210806d2d0 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa291200640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c161df0 0x7fa28c160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa28be00640 1 -- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0b6010 msgr2=0x7fa28c06f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa28be00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0b6010 0x7fa28c06f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa28be00640 1 -- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c161df0 msgr2=0x7fa28c160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa28be00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c161df0 0x7fa28c160430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.264+0000 7fa28be00640 1 -- 172.21.15.165:0/1330428918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28c162a60 con 0x7fa28c0afda0 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.265+0000 7fa28be00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 0x7fa28c1623a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00b410 tx=0x7fa27c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7f6e59000640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e54159dd0 con 0x7f6e540b5910 2024-04-25T10:02:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c015020 con 0x7fa28c0afda0 2024-04-25T10:02:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c011830 con 0x7fa28c0afda0 2024-04-25T10:02:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c011c00 con 0x7fa28c0afda0 2024-04-25T10:02:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa28c164b70 con 0x7fa28c0afda0 2024-04-25T10:02:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa28c165130 con 0x7fa28c0afda0 2024-04-25T10:02:29.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.266+0000 7f6e59000640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5415a390 con 0x7f6e540b5910 2024-04-25T10:02:29.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.268+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa28c073990 con 0x7fa28c0afda0 2024-04-25T10:02:29.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.268+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e540d1520 con 0x7f6e540b5910 2024-04-25T10:02:29.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.271+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c007840 con 0x7f6e540b5910 2024-04-25T10:02:29.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.271+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c031e40 con 0x7f6e540b5910 2024-04-25T10:02:29.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.271+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e4c039040 con 0x7f6e540b5910 2024-04-25T10:02:29.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.270+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa27c018020 con 0x7fa28c0afda0 2024-04-25T10:02:29.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.273+0000 7f6e4b400640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e3406b500 0x7f6e3406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.272+0000 7fa283e00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa26c06b5a0 0x7fa26c06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:29.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.272+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fa27c08a420 con 0x7fa28c0afda0 2024-04-25T10:02:29.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.273+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6e4c0abaf0 con 0x7f6e540b5910 2024-04-25T10:02:29.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.273+0000 7f6e52a00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e3406b500 0x7f6e3406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.274+0000 7fa28b400640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa26c06b5a0 0x7fa26c06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:29.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.275+0000 7f6e52a00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e3406b500 0x7f6e3406d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e540ca4a0 tx=0x7f6e40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.275+0000 7fa28b400640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa26c06b5a0 0x7fa26c06da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa278006fd0 tx=0x7fa278008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:29.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.280+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa27c01f020 con 0x7fa28c0afda0 2024-04-25T10:02:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.292+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e4c07a770 con 0x7f6e540b5910 2024-04-25T10:02:29.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.637+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f804404f6c0 con 0x7f8044053b70 2024-04-25T10:02:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.638+0000 7f803fe00640 1 -- 172.21.15.165:0/1575045446 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f804c0f34a0 con 0x7f8044053b70 2024-04-25T10:02:29.646 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.642+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802c06b520 msgr2=0x7f802c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.642+0000 7f8052a00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802c06b520 0x7f802c06d9e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040007950 comp rx=0 tx=0).stop 2024-04-25T10:02:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.642+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 msgr2=0x7f8044052f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.642+0000 7f8052a00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 0x7f8044052f00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f804c06a560 tx=0x7f804c073660 comp rx=0 tx=0).stop 2024-04-25T10:02:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.643+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 shutdown_connections 2024-04-25T10:02:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.643+0000 7f8052a00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802c06b520 0x7f802c06d9e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.643+0000 7f8052a00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80440e76e0 0x7f8044053440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.643+0000 7f8052a00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8044053b70 0x7f8044052f00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.643+0000 7f8052a00640 1 --2- 172.21.15.165:0/1575045446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8044034400 0x7f8044184fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.643+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 >> 172.21.15.165:0/1575045446 conn(0x7f8044055350 msgr2=0x7f804404d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.644+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 shutdown_connections 2024-04-25T10:02:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.644+0000 7f8052a00640 1 -- 172.21.15.165:0/1575045446 wait complete. 2024-04-25T10:02:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.657+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f46240d1580 con 0x7f46240d0780 2024-04-25T10:02:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.658+0000 7f461b400640 1 -- 172.21.15.165:0/3541114780 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f461004b030 con 0x7f46240d0780 2024-04-25T10:02:29.666 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.662+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f460406b5d0 msgr2=0x7f460406da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.662+0000 7f462a400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f460406b5d0 0x7f460406da90 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4614007fd0 tx=0x7f4614009040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.662+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 msgr2=0x7f4624149450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.662+0000 7f462a400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 0x7f4624149450 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4610000c00 tx=0x7f46100046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.663+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 shutdown_connections 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.663+0000 7f462a400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f460406b5d0 0x7f460406da90 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.663+0000 7f462a400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f462415db10 0x7f4624149990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.663+0000 7f462a400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46240d0780 0x7f4624149450 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.663+0000 7f462a400640 1 --2- 172.21.15.165:0/3541114780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46240b6aa0 0x7f4624152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.663+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 >> 172.21.15.165:0/3541114780 conn(0x7f462406bef0 msgr2=0x7f462406c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.664+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 shutdown_connections 2024-04-25T10:02:29.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.664+0000 7f462a400640 1 -- 172.21.15.165:0/3541114780 wait complete. 2024-04-25T10:02:29.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.671+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f6c3c005850 con 0x7f6c60148570 2024-04-25T10:02:29.677 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.1 2024-04-25T10:02:29.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.671+0000 7f6c5cc00640 1 -- 172.21.15.165:0/2733830925 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c4c05be50 con 0x7f6c60148570 2024-04-25T10:02:29.677 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.675+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c4006b6d0 msgr2=0x7f6c4006db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.675+0000 7f6c67600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c4006b6d0 0x7f6c4006db90 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c54003aa0 tx=0x7f6c54009830 comp rx=0 tx=0).stop 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.675+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 msgr2=0x7f6c6015e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.675+0000 7f6c67600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 0x7f6c6015e450 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c00c430 tx=0x7f6c4c00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 shutdown_connections 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c4006b6d0 0x7f6c4006db90 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c6014b780 0x7f6c6015e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c60148570 0x7f6c6015e450 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 --2- 172.21.15.165:0/2733830925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c600d24f0 0x7f6c601ec360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 >> 172.21.15.165:0/2733830925 conn(0x7f6c600d17c0 msgr2=0x7f6c600b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 shutdown_connections 2024-04-25T10:02:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.676+0000 7f6c67600640 1 -- 172.21.15.165:0/2733830925 wait complete. 2024-04-25T10:02:29.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.680+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f75d80d15a0 con 0x7f75d80ce6d0 2024-04-25T10:02:29.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.681+0000 7f75d4c00640 1 -- 172.21.15.165:0/883595566 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f75cc05e030 con 0x7f75d80ce6d0 2024-04-25T10:02:29.687 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75b806b4d0 msgr2=0x7f75b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75b806b4d0 0x7f75b806d990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f75c4034000 tx=0x7f75c402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 msgr2=0x7f75d80c71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 0x7f75d80c71d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c4b0 tx=0x7f75cc00c970 comp rx=0 tx=0).stop 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 shutdown_connections 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75b806b4d0 0x7f75b806d990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75d815d9d0 0x7f75d80c8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75d80d2370 0x7f75d815a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 --2- 172.21.15.165:0/883595566 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75d80ce6d0 0x7f75d80c71d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 >> 172.21.15.165:0/883595566 conn(0x7f75d80d0380 msgr2=0x7f75d80cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.684+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 shutdown_connections 2024-04-25T10:02:29.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.685+0000 7f75dde00640 1 -- 172.21.15.165:0/883595566 wait complete. 2024-04-25T10:02:29.696 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.0 2024-04-25T10:02:29.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.698+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fa28c06fce0 con 0x7fa28c0afda0 2024-04-25T10:02:29.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.698+0000 7fa283e00640 1 -- 172.21.15.165:0/1330428918 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa27c058f40 con 0x7fa28c0afda0 2024-04-25T10:02:29.704 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.700+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f763806d240 con 0x7f7638161890 2024-04-25T10:02:29.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7f762f400640 1 -- 172.21.15.165:0/470733746 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f762407a700 con 0x7f7638161890 2024-04-25T10:02:29.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa26c06b5a0 msgr2=0x7fa26c06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa26c06b5a0 0x7fa26c06da60 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa278006fd0 tx=0x7fa278008040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 msgr2=0x7fa28c1623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 0x7fa28c1623a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00b410 tx=0x7fa27c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.707 INFO:teuthology.orchestra.run.smithi165.stdout:55834574857 2024-04-25T10:02:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 shutdown_connections 2024-04-25T10:02:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa26c06b5a0 0x7fa26c06da60 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa28c161df0 0x7fa28c160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa28c0b6010 0x7fa28c06f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 --2- 172.21.15.165:0/1330428918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa28c0afda0 0x7fa28c1623a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 >> 172.21.15.165:0/1330428918 conn(0x7fa28c0b53c0 msgr2=0x7fa28c0b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.701+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f20e4005580 con 0x7f210806d2d0 2024-04-25T10:02:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.702+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 shutdown_connections 2024-04-25T10:02:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.702+0000 7fa291c00640 1 -- 172.21.15.165:0/1330428918 wait complete. 2024-04-25T10:02:29.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.702+0000 7f2105600640 1 -- 172.21.15.165:0/3283740199 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f20f405be20 con 0x7f210806d2d0 2024-04-25T10:02:29.709 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.710 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.6 2024-04-25T10:02:29.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f761806b4d0 msgr2=0x7f761806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.711 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.4 2024-04-25T10:02:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f761806b4d0 0x7f761806d990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7630009f20 tx=0x7f76300094f0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 msgr2=0x7f7638203f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 0x7f7638203f90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f76240097a0 tx=0x7f76240046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 shutdown_connections 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f761806b4d0 0x7f761806d990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7638161890 0x7f7638203f90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7638161240 0x7f763815faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 --2- 172.21.15.165:0/470733746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76380b54a0 0x7f76380c9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.704+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 >> 172.21.15.165:0/470733746 conn(0x7f76380d04a0 msgr2=0x7f76380b37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 shutdown_connections 2024-04-25T10:02:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f763de00640 1 -- 172.21.15.165:0/470733746 wait complete. 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20e806b4d0 msgr2=0x7f20e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20e806b4d0 0x7f20e806d990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f21081ec1c0 tx=0x7f20fc009040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 msgr2=0x7f21080a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 0x7f21080a98b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f20f400c500 tx=0x7f20f400c9c0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 shutdown_connections 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f20e806b4d0 0x7f20e806d990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2108161fc0 0x7f21080aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f210806ea90 0x7f2108160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 --2- 172.21.15.165:0/3283740199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f210806d2d0 0x7f21080a98b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 >> 172.21.15.165:0/3283740199 conn(0x7f210806abe0 msgr2=0x7f210815d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 shutdown_connections 2024-04-25T10:02:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.705+0000 7f2110600640 1 -- 172.21.15.165:0/3283740199 wait complete. 2024-04-25T10:02:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.706+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6e540cea50 con 0x7f6e540b5910 2024-04-25T10:02:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.706+0000 7f6e4b400640 1 -- 172.21.15.165:0/403566358 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6e4c07a070 con 0x7f6e540b5910 2024-04-25T10:02:29.714 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:29.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e3406b500 msgr2=0x7f6e3406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e3406b500 0x7f6e3406d9c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6e540ca4a0 tx=0x7f6e40008040 comp rx=0 tx=0).stop 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 msgr2=0x7f6e541f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 0x7f6e541f62c0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c009e80 tx=0x7f6e4c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 shutdown_connections 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e3406b500 0x7f6e3406d9c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e5415dbd0 0x7f6e54160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e540b95c0 0x7f6e541f3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 --2- 172.21.15.165:0/403566358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e540b5910 0x7f6e541f62c0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 >> 172.21.15.165:0/403566358 conn(0x7f6e540d03d0 msgr2=0x7f6e540b7300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 shutdown_connections 2024-04-25T10:02:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:29.710+0000 7f6e48c00640 1 -- 172.21.15.165:0/403566358 wait complete. 2024-04-25T10:02:29.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.7 2024-04-25T10:02:29.731 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.5 2024-04-25T10:02:29.732 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574857 for osd.2 2024-04-25T10:02:29.734 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574858 for osd.3 2024-04-25T10:02:30.677 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-25T10:02:30.696 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-25T10:02:30.710 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-25T10:02:30.712 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-25T10:02:30.730 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-25T10:02:30.732 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-25T10:02:30.733 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-25T10:02:30.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-25T10:02:30.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.923+0000 7f7974400640 1 Processor -- start 2024-04-25T10:02:30.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.923+0000 7f7974400640 1 -- start start 2024-04-25T10:02:30.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.925+0000 7f7974400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 0x7f796c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.925+0000 7f7974400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f796c161710 0x7f796c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.925+0000 7f7974400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f796c15e000 con 0x7f796c15dc20 2024-04-25T10:02:30.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.925+0000 7f7974400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f796c0d1de0 con 0x7f796c161710 2024-04-25T10:02:30.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.925+0000 7f7974400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c161c60 con 0x7f796c0ced80 2024-04-25T10:02:30.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.927+0000 7f7971a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f796c15dc20 0x7f796c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58098/0 (socket says 172.21.15.165:58098) 2024-04-25T10:02:30.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.927+0000 7f7971a00640 1 -- 172.21.15.165:0/3748708562 learned_addr learned my addr 172.21.15.165:0/3748708562 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:30.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.927+0000 7f7972400640 1 --2- 172.21.15.165:0/3748708562 >> v2:172.21.15.172:3300/0 conn(0x7f796c161710 0x7f796c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.928+0000 7f7971000640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 0x7f796c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.928+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2396591735 0 0) 0x7f796c15e000 con 0x7f796c15dc20 2024-04-25T10:02:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.928+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7954003520 con 0x7f796c15dc20 2024-04-25T10:02:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.929+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 914529430 0 0) 0x7f7954003520 con 0x7f796c15dc20 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.929+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 msgr2=0x7f796c0d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.929+0000 7f796be00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 0x7f796c0d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.929+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 >> v2:172.21.15.172:3300/0 conn(0x7f796c161710 msgr2=0x7f796c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.929+0000 7f796be00640 1 --2- 172.21.15.165:0/3748708562 >> v2:172.21.15.172:3300/0 conn(0x7f796c161710 0x7f796c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c161f40 con 0x7f796c15dc20 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7958001670 con 0x7f796c15dc20 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 >> v1:172.21.15.165:6789/0 conn(0x7f796c15dc20 legacy=0x7f796c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79540039b0 0x7f7954003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 0x7f7954004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7954004d40 0x7f79540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0d1de0 con 0x7f79540039b0 2024-04-25T10:02:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c161c60 con 0x7f7954004d40 2024-04-25T10:02:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.930+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795400b9c0 con 0x7f796c0ced80 2024-04-25T10:02:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.931+0000 7f7972400640 1 --2- 172.21.15.165:0/3748708562 >> v2:172.21.15.172:3300/0 conn(0x7f796c161710 0x7f796c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.931+0000 7f7971a00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79540039b0 0x7f7954003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.931+0000 7f7971000640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 0x7f7954004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.932+0000 7f7972400640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7954004d40 0x7f79540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.933+0000 7f7971a00640 1 -- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 msgr2=0x7f7954004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.933+0000 7f7971a00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0ced80 0x7f7954004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.933+0000 7f7971a00640 1 -- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7954004d40 msgr2=0x7f79540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.933+0000 7f7971a00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7954004d40 0x7f79540051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.933+0000 7f7971a00640 1 -- 172.21.15.165:0/3748708562 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c161f40 con 0x7f79540039b0 2024-04-25T10:02:30.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.935+0000 7f7971a00640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79540039b0 0x7f7954003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f795800c290 tx=0x7f7954005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d78b36b90b41ea0f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.935+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958036070 con 0x7f79540039b0 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.936+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7958005ad0 con 0x7f79540039b0 2024-04-25T10:02:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.937+0000 7f7974400640 1 -- 172.21.15.165:0/3748708562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79540039b0 msgr2=0x7f7954003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.937+0000 7f796be00640 1 -- 172.21.15.165:0/3748708562 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958007d70 con 0x7f79540039b0 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.937+0000 7f7974400640 1 --2- 172.21.15.165:0/3748708562 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79540039b0 0x7f7954003dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f795800c290 tx=0x7f7954005830 comp rx=0 tx=0).stop 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.937+0000 7f7972400640 1 -- 172.21.15.165:0/3748708562 reap_dead start 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.938+0000 7f7974400640 1 -- 172.21.15.165:0/3748708562 shutdown_connections 2024-04-25T10:02:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.938+0000 7f7974400640 1 -- 172.21.15.165:0/3748708562 >> 172.21.15.165:0/3748708562 conn(0x7f796c0d0420 msgr2=0x7f796c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.938+0000 7f7974400640 1 -- 172.21.15.165:0/3748708562 shutdown_connections 2024-04-25T10:02:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.939+0000 7f7974400640 1 -- 172.21.15.165:0/3748708562 wait complete. 2024-04-25T10:02:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.942+0000 7f7974400640 1 Processor -- start 2024-04-25T10:02:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.943+0000 7f7974400640 1 -- start start 2024-04-25T10:02:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.944+0000 7f7974400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c0b49e0 0x7f796c1609f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.944+0000 7f7974400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0c95d0 0x7f796c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.945+0000 7f7971a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c0b49e0 0x7f796c1609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.945+0000 7f7971a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c0b49e0 0x7f796c1609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50516/0 (socket says 172.21.15.165:50516) 2024-04-25T10:02:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.946+0000 7f7971a00640 1 -- 172.21.15.165:0/1244780031 learned_addr learned my addr 172.21.15.165:0/1244780031 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.946+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f796c15e1b0 0x7f796c15e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.946+0000 7f7971000640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0c95d0 0x7f796c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.947+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0b5e50 con 0x7f796c0b49e0 2024-04-25T10:02:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.947+0000 7f7972400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f796c15e1b0 0x7f796c15e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.947+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f796c162500 con 0x7f796c15e1b0 2024-04-25T10:02:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.947+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f796c0b5fd0 con 0x7f796c0c95d0 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.949+0000 7f7972400640 1 -- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0c95d0 msgr2=0x7f796c0b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.949+0000 7f7972400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0c95d0 0x7f796c0b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.949+0000 7f7972400640 1 -- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c0b49e0 msgr2=0x7f796c1609f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.949+0000 7f7972400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c0b49e0 0x7f796c1609f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.949+0000 7f7972400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f796c0cab10 con 0x7f796c15e1b0 2024-04-25T10:02:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.950+0000 7f7972400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f796c15e1b0 0x7f796c15e630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f796c0b33b0 tx=0x7f796400f520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.951+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964016620 con 0x7f796c15e1b0 2024-04-25T10:02:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.951+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796401b070 con 0x7f796c15e1b0 2024-04-25T10:02:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.951+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f796c0b5300 con 0x7f796c15e1b0 2024-04-25T10:02:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.951+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f796c0cb060 con 0x7f796c15e1b0 2024-04-25T10:02:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.952+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7964016a20 con 0x7f796c15e1b0 2024-04-25T10:02:30.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.953+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7948005290 con 0x7f796c15e1b0 2024-04-25T10:02:30.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.956+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7964016bc0 con 0x7f796c15e1b0 2024-04-25T10:02:30.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.958+0000 7f7969600640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f794c06b500 0x7f794c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.959+0000 7f7971a00640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f794c06b500 0x7f794c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.959+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f796408bbb0 con 0x7f796c15e1b0 2024-04-25T10:02:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.961+0000 7f7971a00640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f794c06b500 0x7f794c06d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7958007d00 tx=0x7f795802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.968+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f796405a6d0 con 0x7f796c15e1b0 2024-04-25T10:02:30.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.971+0000 7f7130e00640 1 Processor -- start 2024-04-25T10:02:30.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.972+0000 7f7130e00640 1 -- start start 2024-04-25T10:02:30.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.974+0000 7f7130e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f712c15baf0 0x7f712c071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.974+0000 7f7130e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f712c0d2400 0x7f712c0cc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.975+0000 7f7129400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f712c0d2400 0x7f712c0cc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.975+0000 7f7129400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f712c0d2400 0x7f712c0cc020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50528/0 (socket says 172.21.15.165:50528) 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.975+0000 7f7129400640 1 -- 172.21.15.165:0/109813729 learned_addr learned my addr 172.21.15.165:0/109813729 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.975+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c0cf430 con 0x7f712c15d160 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.975+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c15cee0 con 0x7f712c0d2400 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.975+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c0d0040 con 0x7f712c15baf0 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.976+0000 7f7129e00640 1 --2- 172.21.15.165:0/109813729 >> v2:172.21.15.172:3300/0 conn(0x7f712c15baf0 0x7f712c071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.976+0000 7f7129400640 1 -- 172.21.15.165:0/109813729 >> v2:172.21.15.172:3300/0 conn(0x7f712c15baf0 msgr2=0x7f712c071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.976+0000 7f7129400640 1 --2- 172.21.15.165:0/109813729 >> v2:172.21.15.172:3300/0 conn(0x7f712c15baf0 0x7f712c071690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.976+0000 7f7129400640 1 -- 172.21.15.165:0/109813729 >> v1:172.21.15.165:6789/0 conn(0x7f712c15d160 legacy=0x7f712c0d04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.976+0000 7f7129400640 1 -- 172.21.15.165:0/109813729 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c0cf710 con 0x7f712c0d2400 2024-04-25T10:02:30.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.978+0000 7f7129400640 1 --2- 172.21.15.165:0/109813729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c0d2400 0x7f712c0cc020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f712c0731d0 tx=0x7f711802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cdff43e12b90540 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.979+0000 7f99c5c00640 1 Processor -- start 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.979+0000 7f7128a00640 1 -- 172.21.15.165:0/109813729 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118037070 con 0x7f712c0d2400 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.979+0000 7f7128a00640 1 -- 172.21.15.165:0/109813729 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711802ae20 con 0x7f712c0d2400 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.979+0000 7f7128a00640 1 -- 172.21.15.165:0/109813729 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118032660 con 0x7f712c0d2400 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.980+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c0d2400 msgr2=0x7f712c0cc020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.980+0000 7f7130e00640 1 --2- 172.21.15.165:0/109813729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c0d2400 0x7f712c0cc020 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f712c0731d0 tx=0x7f711802a110 comp rx=0 tx=0).stop 2024-04-25T10:02:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.980+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 shutdown_connections 2024-04-25T10:02:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.981+0000 7f7130e00640 1 --2- 172.21.15.165:0/109813729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c0d2400 0x7f712c0cc020 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.981+0000 7f7130e00640 1 --2- 172.21.15.165:0/109813729 >> v2:172.21.15.172:3300/0 conn(0x7f712c15baf0 0x7f712c071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.981+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 >> 172.21.15.165:0/109813729 conn(0x7f712c0d17c0 msgr2=0x7f712c0d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.981+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 shutdown_connections 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.981+0000 7f99c5c00640 1 -- start start 2024-04-25T10:02:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.982+0000 7f7130e00640 1 -- 172.21.15.165:0/109813729 wait complete. 2024-04-25T10:02:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.983+0000 7f99c5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 0x7f99c00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.983+0000 7f99c5c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 0x7f99c00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.983+0000 7f99c5c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c015e000 con 0x7f99c0161710 2024-04-25T10:02:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.983+0000 7f99c5c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00d1de0 con 0x7f99c00d32f0 2024-04-25T10:02:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.983+0000 7f99c5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0161c60 con 0x7f99c015dc20 2024-04-25T10:02:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.984+0000 7f653da00640 1 Processor -- start 2024-04-25T10:02:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.984+0000 7f99bf400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 0x7f99c00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.984+0000 7f653da00640 1 -- start start 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.985+0000 7f99bea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 0x7f99c00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.985+0000 7f99bea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 0x7f99c00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37946/0 (socket says 172.21.15.165:37946) 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.985+0000 7f99bea00640 1 -- 172.21.15.165:0/1848192194 learned_addr learned my addr 172.21.15.165:0/1848192194 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.985+0000 7f99bf400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 0x7f99c00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51642/0 (socket says 172.21.15.165:51642) 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.985+0000 7f99be000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f99c0161710 0x7f99c006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59554/0 (socket says 172.21.15.165:59554) 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.985+0000 7f7130e00640 1 Processor -- start 2024-04-25T10:02:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f7130e00640 1 -- start start 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f99bea00640 1 -- 172.21.15.165:0/1848192194 >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 msgr2=0x7f99c00cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f99bd600640 1 -- 172.21.15.165:0/1848192194 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278925431 0 0) 0x7f99c015e000 con 0x7f99c0161710 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f653da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f653da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f65380d32f0 0x7f65380cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f653da00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f653815e000 con 0x7f6538161710 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f653da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65380d1de0 con 0x7f65380d32f0 2024-04-25T10:02:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f653da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6538161c60 con 0x7f653815dc20 2024-04-25T10:02:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.987+0000 7f6536000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6538161710 0x7f653806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58126/0 (socket says 172.21.15.165:58126) 2024-04-25T10:02:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.987+0000 7f6536000640 1 -- 172.21.15.165:0/2331643076 learned_addr learned my addr 172.21.15.165:0/2331643076 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.988+0000 7f6536a00640 1 --2- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.988+0000 7f6535600640 1 -- 172.21.15.165:0/2331643076 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737380822 0 0) 0x7f653815e000 con 0x7f6538161710 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.988+0000 7f6535600640 1 -- 172.21.15.165:0/2331643076 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f651c003520 con 0x7f6538161710 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.988+0000 7f6537400640 1 --2- 172.21.15.165:0/2331643076 >> v2:172.21.15.172:3300/0 conn(0x7f65380d32f0 0x7f65380cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.988+0000 7f7130e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 0x7f712c1ebc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7130e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712c15d160 0x7f712c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7130e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1513f0 0x7f712c1ed170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7130e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c071bd0 con 0x7f712c1513f0 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7130e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0cfcd0 con 0x7f712c15d160 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7130e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c071d50 con 0x7f712c15baf0 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f99bea00640 1 --2- 172.21.15.165:0/1848192194 >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 0x7f99c00cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f99bea00640 1 -- 172.21.15.165:0/1848192194 >> v1:172.21.15.172:6789/0 conn(0x7f99c0161710 legacy=0x7f99c006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f99bea00640 1 -- 172.21.15.165:0/1848192194 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c0161f40 con 0x7f99c015dc20 2024-04-25T10:02:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.986+0000 7f99bf400640 1 --2- 172.21.15.165:0/1848192194 >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 0x7f99c00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7129e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 0x7f712c1ebc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f712a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1513f0 0x7f712c1ed170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f712a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1513f0 0x7f712c1ed170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50544/0 (socket says 172.21.15.165:50544) 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f712a800640 1 -- 172.21.15.165:0/3229833781 learned_addr learned my addr 172.21.15.165:0/3229833781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f7129e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 0x7f712c1ebc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37966/0 (socket says 172.21.15.165:37966) 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f7129400640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712c15d160 0x7f712c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.989+0000 7f6535600640 1 -- 172.21.15.165:0/2331643076 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2931162897 0 0) 0x7f651c003520 con 0x7f6538161710 2024-04-25T10:02:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f6537400640 1 -- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 msgr2=0x7f65380ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f6537400640 1 --2- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f6537400640 1 -- 172.21.15.165:0/2331643076 >> v1:172.21.15.165:6789/0 conn(0x7f6538161710 legacy=0x7f653806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f6537400640 1 -- 172.21.15.165:0/2331643076 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6538161f40 con 0x7f65380d32f0 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f7129e00640 1 -- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712c15d160 msgr2=0x7f712c0b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f7129e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712c15d160 0x7f712c0b60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f7129e00640 1 -- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1513f0 msgr2=0x7f712c1ed170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f7129e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1513f0 0x7f712c1ed170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.990+0000 7f7129e00640 1 -- 172.21.15.165:0/3229833781 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c1ea7f0 con 0x7f712c15baf0 2024-04-25T10:02:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.991+0000 7f99bea00640 1 --2- 172.21.15.165:0/1848192194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 0x7f99c00ce670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009870 tx=0x7f99ac02a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55a0c87987117e0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.991+0000 7f7129e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 0x7f712c1ebc80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f712400cd90 tx=0x7f712400a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.991+0000 7f99bd600640 1 -- 172.21.15.165:0/1848192194 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac037070 con 0x7f99c015dc20 2024-04-25T10:02:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.991+0000 7f99bd600640 1 -- 172.21.15.165:0/1848192194 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ac032440 con 0x7f99c015dc20 2024-04-25T10:02:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.991+0000 7f99bd600640 1 -- 172.21.15.165:0/1848192194 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ac032810 con 0x7f99c015dc20 2024-04-25T10:02:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.991+0000 7f6537400640 1 --2- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380d32f0 0x7f65380cebb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6530009d90 tx=0x7f653002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c70a88ca6d0e86b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.992+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124003b90 con 0x7f712c15baf0 2024-04-25T10:02:31.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.992+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7124003d30 con 0x7f712c15baf0 2024-04-25T10:02:31.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.992+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124004240 con 0x7f712c15baf0 2024-04-25T10:02:31.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.992+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f712c0b6850 con 0x7f712c15baf0 2024-04-25T10:02:31.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.992+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c0b6e10 con 0x7f712c15baf0 2024-04-25T10:02:31.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 -- 172.21.15.165:0/1848192194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 msgr2=0x7f99c00ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 --2- 172.21.15.165:0/1848192194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 0x7f99c00ce670 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f99ac009870 tx=0x7f99ac02a250 comp rx=0 tx=0).stop 2024-04-25T10:02:31.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 -- 172.21.15.165:0/1848192194 shutdown_connections 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 --2- 172.21.15.165:0/1848192194 >> v2:172.21.15.172:3300/0 conn(0x7f99c00d32f0 0x7f99c00cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 --2- 172.21.15.165:0/1848192194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c015dc20 0x7f99c00ce670 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 -- 172.21.15.165:0/1848192194 >> 172.21.15.165:0/1848192194 conn(0x7f99c00d0420 msgr2=0x7f99c00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 -- 172.21.15.165:0/1848192194 shutdown_connections 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f99c5c00640 1 -- 172.21.15.165:0/1848192194 wait complete. 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f6535600640 1 -- 172.21.15.165:0/2331643076 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6524001670 con 0x7f65380d32f0 2024-04-25T10:02:31.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f6535600640 1 -- 172.21.15.165:0/2331643076 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6530007540 con 0x7f65380d32f0 2024-04-25T10:02:31.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.993+0000 7f6535600640 1 -- 172.21.15.165:0/2331643076 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6530007910 con 0x7f65380d32f0 2024-04-25T10:02:31.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.994+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c073a00 con 0x7f712c15baf0 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.995+0000 7f653da00640 1 -- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380d32f0 msgr2=0x7f65380cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.995+0000 7f653da00640 1 --2- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380d32f0 0x7f65380cebb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6530009d90 tx=0x7f653002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.995+0000 7f653da00640 1 -- 172.21.15.165:0/2331643076 shutdown_connections 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.995+0000 7f653da00640 1 --2- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65380d32f0 0x7f65380cebb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.995+0000 7f653da00640 1 --2- 172.21.15.165:0/2331643076 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65380ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.996+0000 7f653da00640 1 -- 172.21.15.165:0/2331643076 >> 172.21.15.165:0/2331643076 conn(0x7f65380d0420 msgr2=0x7f65380d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.996+0000 7f653da00640 1 -- 172.21.15.165:0/2331643076 shutdown_connections 2024-04-25T10:02:31.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.996+0000 7f653da00640 1 -- 172.21.15.165:0/2331643076 wait complete. 2024-04-25T10:02:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.996+0000 7f99c5c00640 1 Processor -- start 2024-04-25T10:02:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.996+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71240043e0 con 0x7f712c15baf0 2024-04-25T10:02:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.998+0000 7f99c5c00640 1 -- start start 2024-04-25T10:02:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.998+0000 7f7122000640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f710c06b550 0x7f710c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f653da00640 1 Processor -- start 2024-04-25T10:02:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f653da00640 1 -- start start 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f99c5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99c00d32f0 0x7f99c014fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f99c5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99c015dc20 0x7f99c00aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f99c5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c0161710 0x7f99c00d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f99c5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00b5e50 con 0x7f99c00d32f0 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f99c5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0162500 con 0x7f99c015dc20 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f99c5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00b5fd0 con 0x7f99c0161710 2024-04-25T10:02:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.000+0000 7f99bf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c0161710 0x7f99c00d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.000+0000 7f99be000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99c015dc20 0x7f99c00aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.000+0000 7f99be000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99c015dc20 0x7f99c00aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51664/0 (socket says 172.21.15.165:51664) 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:30.999+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f712408ae60 con 0x7f712c15baf0 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.000+0000 7f99be000640 1 -- 172.21.15.165:0/3585572144 learned_addr learned my addr 172.21.15.165:0/3585572144 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.000+0000 7f99bea00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99c00d32f0 0x7f99c014fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f7129400640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f710c06b550 0x7f710c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f653da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 0x7f653815ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f653da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65381f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f6536a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 0x7f653815ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f653da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6538161710 0x7f65380ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f653da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65380b5e50 con 0x7f65380d32f0 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f653da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6538162500 con 0x7f6538161710 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f653da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65380b5fd0 con 0x7f653815dc20 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f99bf400640 1 -- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99c015dc20 msgr2=0x7f99c00aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f99bf400640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99c015dc20 0x7f99c00aa380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 0x7f653815ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50552/0 (socket says 172.21.15.165:50552) 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 -- 172.21.15.165:0/3473628849 learned_addr learned my addr 172.21.15.165:0/3473628849 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f99bf400640 1 -- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99c00d32f0 msgr2=0x7f99c014fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f99bf400640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99c00d32f0 0x7f99c014fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 -- 172.21.15.165:0/3473628849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 msgr2=0x7f65381f4820 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65381f4820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.001+0000 7f99bf400640 1 -- 172.21.15.165:0/3585572144 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c014cfa0 con 0x7f99c0161710 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 -- 172.21.15.165:0/3473628849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6538161710 msgr2=0x7f65380ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6538161710 0x7f65380ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.002+0000 7f6536a00640 1 -- 172.21.15.165:0/3473628849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65380cbc40 con 0x7f65380d32f0 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.003+0000 7f6536a00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 0x7f653815ced0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f652800e870 tx=0x7f652800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.003+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528017070 con 0x7f65380d32f0 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.003+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65380cbf20 con 0x7f65380d32f0 2024-04-25T10:02:31.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.004+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65380cc4e0 con 0x7f65380d32f0 2024-04-25T10:02:31.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.004+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652800c940 con 0x7f65380d32f0 2024-04-25T10:02:31.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.004+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6528017070 con 0x7f65380d32f0 2024-04-25T10:02:31.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.005+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6510005290 con 0x7f65380d32f0 2024-04-25T10:02:31.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.005+0000 7f7129400640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f710c06b550 0x7f710c06da10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7118034000 tx=0x7f711802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.005+0000 7f99bf400640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c0161710 0x7f99c00d3980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f99b000e7f0 tx=0x7f99b000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.008+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f652801a070 con 0x7f65380d32f0 2024-04-25T10:02:31.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.009+0000 7f652ea00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f651806b700 0x7f651806dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.009+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f652808a420 con 0x7f65380d32f0 2024-04-25T10:02:31.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.010+0000 7f6536000640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f651806b700 0x7f651806dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.011+0000 7f6536000640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f651806b700 0x7f651806dbc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6524001380 tx=0x7f6524001040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.013+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b0017070 con 0x7f99c0161710 2024-04-25T10:02:31.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.013+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b000c940 con 0x7f99c0161710 2024-04-25T10:02:31.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.013+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b000eea0 con 0x7f99c0161710 2024-04-25T10:02:31.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.013+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7124059a80 con 0x7f712c15baf0 2024-04-25T10:02:31.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.013+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c014d240 con 0x7f99c0161710 2024-04-25T10:02:31.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.013+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c014d800 con 0x7f99c0161710 2024-04-25T10:02:31.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.015+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c0073a00 con 0x7f99c0161710 2024-04-25T10:02:31.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.016+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6528058ec0 con 0x7f65380d32f0 2024-04-25T10:02:31.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.021+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99b001a070 con 0x7f99c0161710 2024-04-25T10:02:31.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.022+0000 7f99b6a00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99a006b5a0 0x7f99a006da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.022+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f99b008a950 con 0x7f99c0161710 2024-04-25T10:02:31.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.024+0000 7f99bea00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99a006b5a0 0x7f99a006da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.025+0000 7f99bea00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99a006b5a0 0x7f99a006da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99ac034000 tx=0x7f99ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.030+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99b0058e20 con 0x7f99c0161710 2024-04-25T10:02:31.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.057+0000 7fdc4d400640 1 Processor -- start 2024-04-25T10:02:31.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.057+0000 7fdc4d400640 1 -- start start 2024-04-25T10:02:31.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.059+0000 7fdc4d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdc480b5b40 0x7fdc48168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.059+0000 7fdc4d400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4815e110 con 0x7fdc4806bff0 2024-04-25T10:02:31.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.059+0000 7fdc4d400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc480b2fd0 con 0x7fdc4815dd30 2024-04-25T10:02:31.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.059+0000 7fdc4d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4806d000 con 0x7fdc480b5b40 2024-04-25T10:02:31.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.060+0000 7fdc47400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fdc4806bff0 0x7fdc480b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58140/0 (socket says 172.21.15.165:58140) 2024-04-25T10:02:31.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.060+0000 7fdc47400640 1 -- 172.21.15.165:0/483500218 learned_addr learned my addr 172.21.15.165:0/483500218 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:31.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.060+0000 7fdc4ca00640 1 --2- 172.21.15.165:0/483500218 >> v2:172.21.15.172:3300/0 conn(0x7fdc480b5b40 0x7fdc48168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.060+0000 7f32b3200640 1 Processor -- start 2024-04-25T10:02:31.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.060+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774896158 0 0) 0x7fdc4815e110 con 0x7fdc4806bff0 2024-04-25T10:02:31.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.060+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc2c003520 con 0x7fdc4806bff0 2024-04-25T10:02:31.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7f32b3200640 1 -- start start 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1600588835 0 0) 0x7fdc2c003520 con 0x7fdc4806bff0 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 >> v2:172.21.15.172:3300/0 conn(0x7fdc480b5b40 msgr2=0x7fdc48168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 --2- 172.21.15.165:0/483500218 >> v2:172.21.15.172:3300/0 conn(0x7fdc480b5b40 0x7fdc48168b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 >> v1:172.21.15.172:6789/0 conn(0x7fdc4815dd30 legacy=0x7fdc480cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4806c480 con 0x7fdc4806bff0 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fdc400019f0 con 0x7fdc4806bff0 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.061+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 >> v1:172.21.15.165:6789/0 conn(0x7fdc4806bff0 legacy=0x7fdc480b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc46a00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c003a00 0x7fdc2c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc46a00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c004800 0x7fdc2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc46a00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2c0053c0 0x7fdc2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4806d000 con 0x7fdc2c003a00 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c00c230 con 0x7fdc2c0053c0 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2c00c430 con 0x7fdc2c004800 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc47e00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c003a00 0x7fdc2c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7f32b3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac0ead00 0x7f32ac04fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7f32b3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f32ac006ea0 0x7f32ac007330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b3200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32ac053720 con 0x7f32ac04ffd0 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32ac054330 con 0x7f32ac006ea0 2024-04-25T10:02:31.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac053a00 con 0x7f32ac0ead00 2024-04-25T10:02:31.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.062+0000 7fdc47400640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c004800 0x7fdc2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b1400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f32ac04ffd0 0x7f32ac037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58148/0 (socket says 172.21.15.165:58148) 2024-04-25T10:02:31.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b1400640 1 -- 172.21.15.165:0/1877853442 learned_addr learned my addr 172.21.15.165:0/1877853442 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:31.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b2800640 1 --2- 172.21.15.165:0/1877853442 >> v2:172.21.15.172:3300/0 conn(0x7f32ac006ea0 0x7f32ac007330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7f32b1e00640 1 --2- 172.21.15.165:0/1877853442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac0ead00 0x7f32ac04fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.064+0000 7f7171600640 1 Processor -- start 2024-04-25T10:02:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7fdc47e00640 1 -- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c004800 msgr2=0x7fdc2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7fdc47e00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c004800 0x7fdc2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7fdc47e00640 1 -- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2c0053c0 msgr2=0x7fdc2c00b9c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-25T10:02:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7fdc47e00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc2c0053c0 0x7fdc2c00b9c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.063+0000 7fdc47e00640 1 -- 172.21.15.165:0/483500218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc2c003520 con 0x7fdc2c003a00 2024-04-25T10:02:31.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.064+0000 7fdc4ca00640 1 -- 172.21.15.165:0/483500218 reap_dead start 2024-04-25T10:02:31.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.064+0000 7f7171600640 1 -- start start 2024-04-25T10:02:31.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.065+0000 7f32b1e00640 1 -- 172.21.15.165:0/1877853442 >> v2:172.21.15.172:3300/0 conn(0x7f32ac006ea0 msgr2=0x7f32ac007330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.065+0000 7f32b1e00640 1 --2- 172.21.15.165:0/1877853442 >> v2:172.21.15.172:3300/0 conn(0x7f32ac006ea0 0x7f32ac007330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.065+0000 7f32b1e00640 1 -- 172.21.15.165:0/1877853442 >> v1:172.21.15.165:6789/0 conn(0x7f32ac04ffd0 legacy=0x7f32ac037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.065+0000 7f32b1e00640 1 -- 172.21.15.165:0/1877853442 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac053ce0 con 0x7f32ac0ead00 2024-04-25T10:02:31.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.065+0000 7fdc47e00640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c003a00 0x7fdc2c003e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc38001750 tx=0x7fdc2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6286e3b30184180d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.066+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38001370 con 0x7fdc2c003a00 2024-04-25T10:02:31.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.066+0000 7f32b1e00640 1 --2- 172.21.15.165:0/1877853442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac0ead00 0x7f32ac04fa90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32b4000b60 tx=0x7f32b40a50e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=683a247c617bf4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7f32b0a00640 1 -- 172.21.15.165:0/1877853442 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40b2070 con 0x7f32ac0ead00 2024-04-25T10:02:31.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.066+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc380052d0 con 0x7fdc2c003a00 2024-04-25T10:02:31.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7fdc46a00640 1 -- 172.21.15.165:0/483500218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc3802abc0 con 0x7fdc2c003a00 2024-04-25T10:02:31.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7f32b0a00640 1 -- 172.21.15.165:0/1877853442 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b40ad440 con 0x7f32ac0ead00 2024-04-25T10:02:31.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7f32b0a00640 1 -- 172.21.15.165:0/1877853442 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40ad8b0 con 0x7f32ac0ead00 2024-04-25T10:02:31.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7fdc4d400640 1 -- 172.21.15.165:0/483500218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c003a00 msgr2=0x7fdc2c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7fdc4d400640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c003a00 0x7fdc2c003e20 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fdc38001750 tx=0x7fdc2c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7fdc4d400640 1 -- 172.21.15.165:0/483500218 shutdown_connections 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7fdc4d400640 1 --2- 172.21.15.165:0/483500218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc2c003a00 0x7fdc2c003e20 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.067+0000 7fdc4d400640 1 -- 172.21.15.165:0/483500218 >> 172.21.15.165:0/483500218 conn(0x7fdc480d0610 msgr2=0x7fdc480d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7fdc4d400640 1 -- 172.21.15.165:0/483500218 shutdown_connections 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f32b3200640 1 -- 172.21.15.165:0/1877853442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac0ead00 msgr2=0x7f32ac04fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f7171600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f716c161fc0 0x7f716c1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f32b3200640 1 --2- 172.21.15.165:0/1877853442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac0ead00 0x7f32ac04fa90 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f32b4000b60 tx=0x7f32b40a50e0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f7171600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f716c06ea90 0x7f716c071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f7171600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f716c1628e0 con 0x7f716c06d2d0 2024-04-25T10:02:31.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f7171600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c165be0 con 0x7f716c161fc0 2024-04-25T10:02:31.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.068+0000 7f7171600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f716c162ba0 con 0x7f716c06ea90 2024-04-25T10:02:31.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f32b3200640 1 -- 172.21.15.165:0/1877853442 shutdown_connections 2024-04-25T10:02:31.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f32b3200640 1 --2- 172.21.15.165:0/1877853442 >> v2:172.21.15.172:3300/0 conn(0x7f32ac006ea0 0x7f32ac007330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f32b3200640 1 --2- 172.21.15.165:0/1877853442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac0ead00 0x7f32ac04fa90 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f32b3200640 1 -- 172.21.15.165:0/1877853442 >> 172.21.15.165:0/1877853442 conn(0x7f32ac009fb0 msgr2=0x7f32ac00a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f3c95000640 1 Processor -- start 2024-04-25T10:02:31.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f32b3200640 1 -- 172.21.15.165:0/1877853442 shutdown_connections 2024-04-25T10:02:31.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f7162000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f716c06ea90 0x7f716c071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f716b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f716c06d2d0 0x7f716c06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58152/0 (socket says 172.21.15.165:58152) 2024-04-25T10:02:31.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f716b400640 1 -- 172.21.15.165:0/1471188814 learned_addr learned my addr 172.21.15.165:0/1471188814 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:31.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f32b3200640 1 -- 172.21.15.165:0/1877853442 wait complete. 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.070+0000 7f7162000640 1 -- 172.21.15.165:0/1471188814 >> v2:172.21.15.165:3300/0 conn(0x7f716c161fc0 msgr2=0x7f716c1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.070+0000 7f716aa00640 1 --2- 172.21.15.165:0/1471188814 >> v2:172.21.15.165:3300/0 conn(0x7f716c161fc0 0x7f716c1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.070+0000 7f7162000640 1 --2- 172.21.15.165:0/1471188814 >> v2:172.21.15.165:3300/0 conn(0x7f716c161fc0 0x7f716c1623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f7162000640 1 -- 172.21.15.165:0/1471188814 >> v1:172.21.15.165:6789/0 conn(0x7f716c06d2d0 legacy=0x7f716c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f7162000640 1 -- 172.21.15.165:0/1471188814 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c162e80 con 0x7f716c06ea90 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f716aa00640 1 --2- 172.21.15.165:0/1471188814 >> v2:172.21.15.165:3300/0 conn(0x7f716c161fc0 0x7f716c1623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:31.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7fdc4d400640 1 -- 172.21.15.165:0/483500218 wait complete. 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7fdc4d400640 1 Processor -- start 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7fdc4d400640 1 -- start start 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.069+0000 7f3c95000640 1 -- start start 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f3c95000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 0x7f3c8804fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f3c95000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c88053720 con 0x7f3c88006ea0 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f3c95000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c88053a00 con 0x7f3c8804ffd0 2024-04-25T10:02:31.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.071+0000 7f3c95000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88054330 con 0x7f3c880ead00 2024-04-25T10:02:31.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7f32b3200640 1 Processor -- start 2024-04-25T10:02:31.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7f32b3200640 1 -- start start 2024-04-25T10:02:31.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7f3c8f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 0x7f3c8804fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7f3c8ea00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f3c8804ffd0 0x7f3c88037bd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59566/0 (socket says 172.21.15.165:59566) 2024-04-25T10:02:31.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7f3c8ea00640 1 -- 172.21.15.165:0/1286724826 learned_addr learned my addr 172.21.15.165:0/1286724826 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:31.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.072+0000 7f3c8fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3c88006ea0 0x7f3c880382e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58168/0 (socket says 172.21.15.165:58168) 2024-04-25T10:02:31.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.073+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c003a00 0x7fdc480ccc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.073+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725211401 0 0) 0x7f3c88053a00 con 0x7f3c8804ffd0 2024-04-25T10:02:31.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.073+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c78003520 con 0x7f3c8804ffd0 2024-04-25T10:02:31.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.073+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1773606401 0 0) 0x7f3c88053720 con 0x7f3c88006ea0 2024-04-25T10:02:31.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.073+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c88053a00 con 0x7f3c88006ea0 2024-04-25T10:02:31.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.073+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 0x7fdc480cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1031089906 0 0) 0x7f3c88053a00 con 0x7f3c88006ea0 2024-04-25T10:02:31.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 msgr2=0x7f3c8804fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f3c8e000640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 0x7f3c8804fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 >> v1:172.21.15.172:6789/0 conn(0x7f3c8804ffd0 legacy=0x7f3c88037bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc4d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc480cd4b0 0x7fdc48071540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc47400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 0x7fdc480cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc47e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c003a00 0x7fdc480ccc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc47400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 0x7fdc480cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50584/0 (socket says 172.21.15.165:50584) 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc47e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c003a00 0x7fdc480ccc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:38004/0 (socket says 172.21.15.165:38004) 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc47400640 1 -- 172.21.15.165:0/2679315451 learned_addr learned my addr 172.21.15.165:0/2679315451 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f32b3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 0x7f32ac0355e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f7162000640 1 --2- 172.21.15.165:0/1471188814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c071480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f716c0713b0 tx=0x7f715402a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bcbdd80bb492395 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7f32b3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac04ffd0 0x7f32ac035b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac0ead00 0x7f32ac0d9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f716a000640 1 -- 172.21.15.165:0/1471188814 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7154037070 con 0x7f716c06ea90 2024-04-25T10:02:31.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4806cd20 con 0x7fdc481f55d0 2024-04-25T10:02:31.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4806ca40 con 0x7fdc480cd4b0 2024-04-25T10:02:31.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc480b5430 con 0x7fdc2c003a00 2024-04-25T10:02:31.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.074+0000 7fdc4ca00640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc480cd4b0 0x7fdc48071540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac054940 con 0x7f32ac006ea0 2024-04-25T10:02:31.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac0547c0 con 0x7f32ac0ead00 2024-04-25T10:02:31.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f716a000640 1 -- 172.21.15.165:0/1471188814 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7154032440 con 0x7f716c06ea90 2024-04-25T10:02:31.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f716a000640 1 -- 172.21.15.165:0/1471188814 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715402ae60 con 0x7f716c06ea90 2024-04-25T10:02:31.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32ac04f550 con 0x7f32ac04ffd0 2024-04-25T10:02:31.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 0x7f32ac0355e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 0x7f32ac0355e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50590/0 (socket says 172.21.15.165:50590) 2024-04-25T10:02:31.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7f32b1e00640 1 -- 172.21.15.165:0/3648217244 learned_addr learned my addr 172.21.15.165:0/3648217244 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88053ce0 con 0x7f3c88006ea0 2024-04-25T10:02:31.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f32b2800640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac0ead00 0x7f32ac0d9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3c9004f770 con 0x7f3c88006ea0 2024-04-25T10:02:31.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 >> v1:172.21.15.165:6789/0 conn(0x7f3c88006ea0 legacy=0x7f3c880382e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f3c8e000640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c78004c00 0x7f3c78004fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f3c8e000640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 0x7f3c780037a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f32b1400640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac04ffd0 0x7f32ac035b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f7171600640 1 -- 172.21.15.165:0/1471188814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 msgr2=0x7f716c071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f7171600640 1 --2- 172.21.15.165:0/1471188814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c071480 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f716c0713b0 tx=0x7f715402a110 comp rx=0 tx=0).stop 2024-04-25T10:02:31.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7fdc47400640 1 -- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c003a00 msgr2=0x7fdc480ccc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7fdc47400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c003a00 0x7fdc480ccc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7fdc47400640 1 -- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc480cd4b0 msgr2=0x7fdc48071540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7fdc47400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc480cd4b0 0x7fdc48071540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.075+0000 7fdc47400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc481f5a10 con 0x7fdc481f55d0 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f3c8e000640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c78003ce0 0x7f3c78004160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f7171600640 1 -- 172.21.15.165:0/1471188814 shutdown_connections 2024-04-25T10:02:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f7171600640 1 --2- 172.21.15.165:0/1471188814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c071480 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f7171600640 1 --2- 172.21.15.165:0/1471188814 >> v2:172.21.15.165:3300/0 conn(0x7f716c161fc0 0x7f716c1623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.076+0000 7f7171600640 1 -- 172.21.15.165:0/1471188814 >> 172.21.15.165:0/1471188814 conn(0x7f716c06abe0 msgr2=0x7f716c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7fdc47400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 0x7fdc480cb490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc40000f80 tx=0x7fdc4000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f3c8f400640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c78004c00 0x7f3c78004fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f7171600640 1 -- 172.21.15.165:0/1471188814 shutdown_connections 2024-04-25T10:02:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f3c8fe00640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c78003ce0 0x7f3c78004160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88054330 con 0x7f3c78003ce0 2024-04-25T10:02:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7800b9c0 con 0x7f3c78004c00 2024-04-25T10:02:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7800bb40 con 0x7f3c880ead00 2024-04-25T10:02:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f7171600640 1 -- 172.21.15.165:0/1471188814 wait complete. 2024-04-25T10:02:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc40017020 con 0x7fdc481f55d0 2024-04-25T10:02:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7f32b1e00640 1 -- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac04ffd0 msgr2=0x7f32ac035b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f3c8ea00640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 0x7f3c780037a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f32b1e00640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac04ffd0 0x7f32ac035b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f32b1e00640 1 -- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac0ead00 msgr2=0x7f32ac0d9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f32b1e00640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac0ead00 0x7f32ac0d9980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f32b1e00640 1 -- 172.21.15.165:0/3648217244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32ac036280 con 0x7f32ac006ea0 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc480cbbe0 con 0x7fdc481f55d0 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.077+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc400133f0 con 0x7fdc481f55d0 2024-04-25T10:02:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc400137e0 con 0x7fdc481f55d0 2024-04-25T10:02:31.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f32b1400640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac04ffd0 0x7f32ac035b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:31.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.078+0000 7f32b1e00640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 0x7f32ac0355e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32b402d370 tx=0x7f32b4073660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.079+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40b2050 con 0x7f32ac006ea0 2024-04-25T10:02:31.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.079+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32b40b3040 con 0x7f32ac006ea0 2024-04-25T10:02:31.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.079+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32ac036560 con 0x7f32ac006ea0 2024-04-25T10:02:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc480cc1a0 con 0x7fdc481f55d0 2024-04-25T10:02:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32ac036b20 con 0x7f32ac006ea0 2024-04-25T10:02:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7f7171600640 1 Processor -- start 2024-04-25T10:02:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7f7171600640 1 -- start start 2024-04-25T10:02:31.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32b40b78d0 con 0x7f32ac006ea0 2024-04-25T10:02:31.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7f3c8f400640 1 -- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 msgr2=0x7f3c780037a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.080+0000 7f3c8f400640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c880ead00 0x7f3c780037a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.081+0000 7f3c8f400640 1 -- 172.21.15.165:0/1286724826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c78003ce0 msgr2=0x7f3c78004160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.081+0000 7f3c8f400640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c78003ce0 0x7f3c78004160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.081+0000 7f3c8f400640 1 -- 172.21.15.165:0/1286724826 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88053ce0 con 0x7f3c78004c00 2024-04-25T10:02:31.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.081+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f328c005290 con 0x7f32ac006ea0 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.082+0000 7f7171600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 0x7f716c14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7f7171600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c163680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7f7171600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f716c161fc0 0x7f716c163bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7f716aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 0x7f716c14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7f716aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 0x7f716c14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:38024/0 (socket says 172.21.15.165:38024) 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7f7162000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c163680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7f716aa00640 1 -- 172.21.15.165:0/1074902038 learned_addr learned my addr 172.21.15.165:0/1074902038 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.084+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c0719c0 con 0x7f716c161fc0 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.084+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f716c163440 con 0x7f716c06ea90 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.084+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f716c071b40 con 0x7f716c06d2d0 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.084+0000 7f716aa00640 1 -- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 msgr2=0x7f716c163680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7f716aa00640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c163680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7f716aa00640 1 -- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f716c161fc0 msgr2=0x7f716c163bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7f716aa00640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f716c161fc0 0x7f716c163bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7f716aa00640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f716c1f7970 con 0x7f716c06d2d0 2024-04-25T10:02:31.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.083+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc4001a020 con 0x7fdc481f55d0 2024-04-25T10:02:31.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7fdc3fe00640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc2806b520 0x7fdc2806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc48073990 con 0x7fdc481f55d0 2024-04-25T10:02:31.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7fdc47e00640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc2806b520 0x7fdc2806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.085+0000 7f716aa00640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 0x7f716c14fff0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f715c00ce30 tx=0x7f715c00a650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.086+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00f890 con 0x7f716c06d2d0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.086+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f715c00fa30 con 0x7f716c06d2d0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.086+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f715c00fe00 con 0x7f716c06d2d0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.086+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c150530 con 0x7f716c06d2d0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.086+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f716c1604a0 con 0x7f716c06d2d0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f716c073a00 con 0x7f716c06d2d0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7fdc47e00640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc2806b520 0x7fdc2806d9e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc38032000 tx=0x7fdc3802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.086+0000 7f3c8f400640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c78004c00 0x7f3c78004fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009ac0 tx=0x7f3c78005830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95a2b6768469c6c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.087+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84036070 con 0x7f3c78004c00 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.087+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c84031400 con 0x7f3c78004c00 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.087+0000 7f3c8e000640 1 -- 172.21.15.165:0/1286724826 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c84009690 con 0x7f3c78004c00 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f3c95000640 1 -- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c78004c00 msgr2=0x7f3c78004fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f3c95000640 1 --2- 172.21.15.165:0/1286724826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c78004c00 0x7f3c78004fe0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3c84009ac0 tx=0x7f3c78005830 comp rx=0 tx=0).stop 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f3c8fe00640 1 -- 172.21.15.165:0/1286724826 reap_dead start 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f3c95000640 1 -- 172.21.15.165:0/1286724826 shutdown_connections 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f3c95000640 1 -- 172.21.15.165:0/1286724826 >> 172.21.15.165:0/1286724826 conn(0x7f3c88009fb0 msgr2=0x7f3c8800a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.088+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32b40c0070 con 0x7f32ac006ea0 2024-04-25T10:02:31.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.089+0000 7f3c95000640 1 -- 172.21.15.165:0/1286724826 shutdown_connections 2024-04-25T10:02:31.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.089+0000 7f3c95000640 1 -- 172.21.15.165:0/1286724826 wait complete. 2024-04-25T10:02:31.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.090+0000 7f32aa000640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f329406b5a0 0x7f329406da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.090+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f32b4124e80 con 0x7f32ac006ea0 2024-04-25T10:02:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.090+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f715c018400 con 0x7f716c06d2d0 2024-04-25T10:02:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.091+0000 7f32b1400640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f329406b5a0 0x7f329406da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.092+0000 7f7163400640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f714806b4d0 0x7f714806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.092+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f715c08b2e0 con 0x7f716c06d2d0 2024-04-25T10:02:31.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.093+0000 7f32b1400640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f329406b5a0 0x7f329406da60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0396b0 tx=0x7f32a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.092+0000 7f7162000640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f714806b4d0 0x7f714806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.093+0000 7f3c95000640 1 Processor -- start 2024-04-25T10:02:31.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.093+0000 7f3c95000640 1 -- start start 2024-04-25T10:02:31.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.094+0000 7f7162000640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f714806b4d0 0x7f714806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7154034000 tx=0x7f715402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.095+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fdc400565e0 con 0x7fdc481f55d0 2024-04-25T10:02:31.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.095+0000 7f3c95000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c8802d9c0 0x7f3c88038b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.095+0000 7f3c95000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 0x7f3c88039090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.095+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f32b40f33e0 con 0x7f32ac006ea0 2024-04-25T10:02:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.095+0000 7f3c95000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c880e8c00 0x7f3c880e90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c8f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c8802d9c0 0x7f3c88038b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c8ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 0x7f3c88039090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c8ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 0x7f3c88039090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:38026/0 (socket says 172.21.15.165:38026) 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c8ea00640 1 -- 172.21.15.165:0/2608907927 learned_addr learned my addr 172.21.15.165:0/2608907927 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c8fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c880e8c00 0x7f3c880e90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88054940 con 0x7f3c880e8c00 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c880547c0 con 0x7f3c8802d9c0 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.096+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8804f550 con 0x7f3c8802dda0 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.097+0000 7f3c8ea00640 1 -- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c8802d9c0 msgr2=0x7f3c88038b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.098+0000 7f3c8ea00640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c8802d9c0 0x7f3c88038b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.098+0000 7f3c8ea00640 1 -- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c880e8c00 msgr2=0x7f3c880e90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.098+0000 7f3c8ea00640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c880e8c00 0x7f3c880e90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.098+0000 7f3c8ea00640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88185880 con 0x7f3c8802dda0 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.098+0000 7f3c8f400640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c8802d9c0 0x7f3c88038b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.099+0000 7f3c8ea00640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 0x7f3c88039090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c004f60 tx=0x7f3c7c004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.104+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc40058c40 con 0x7fdc481f55d0 2024-04-25T10:02:31.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.105+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c01a070 con 0x7f3c8802dda0 2024-04-25T10:02:31.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.105+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7c0076d0 con 0x7f3c8802dda0 2024-04-25T10:02:31.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.105+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7c00fc00 con 0x7f3c8802dda0 2024-04-25T10:02:31.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.105+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f715c014030 con 0x7f716c06d2d0 2024-04-25T10:02:31.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.105+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c88185b60 con 0x7f3c8802dda0 2024-04-25T10:02:31.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.106+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c88186120 con 0x7f3c8802dda0 2024-04-25T10:02:31.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.107+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c88186320 con 0x7f3c8802dda0 2024-04-25T10:02:31.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.110+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c7c01d070 con 0x7f3c8802dda0 2024-04-25T10:02:31.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.111+0000 7f3c83400640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c7006b5d0 0x7f3c7006da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.112+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3c7c089f00 con 0x7f3c8802dda0 2024-04-25T10:02:31.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.112+0000 7f3c8f400640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c7006b5d0 0x7f3c7006da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.114+0000 7f3c8f400640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c7006b5d0 0x7f3c7006da90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c84033000 tx=0x7f3c84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.124+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c88186320 con 0x7f3c8802dda0 2024-04-25T10:02:31.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.372+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f7948005850 con 0x7f796c15e1b0 2024-04-25T10:02:31.381 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.374+0000 7f7969600640 1 -- 172.21.15.165:0/1244780031 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7964059fd0 con 0x7f796c15e1b0 2024-04-25T10:02:31.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.377+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f794c06b500 msgr2=0x7f794c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.377+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f794c06b500 0x7f794c06d9c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7958007d00 tx=0x7f795802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.377+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f796c15e1b0 msgr2=0x7f796c15e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f796c15e1b0 0x7f796c15e630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f796c0b33b0 tx=0x7f796400f520 comp rx=0 tx=0).stop 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 shutdown_connections 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f794c06b500 0x7f794c06d9c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f796c15e1b0 0x7f796c15e630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f796c0c95d0 0x7f796c0b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 --2- 172.21.15.165:0/1244780031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f796c0b49e0 0x7f796c1609f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.378+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 >> 172.21.15.165:0/1244780031 conn(0x7f796c0d0420 msgr2=0x7f796c0d2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.379+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 shutdown_connections 2024-04-25T10:02:31.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.379+0000 7f7974400640 1 -- 172.21.15.165:0/1244780031 wait complete. 2024-04-25T10:02:31.411 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.1 2024-04-25T10:02:31.411 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.418+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f712c15ae40 con 0x7f712c15baf0 2024-04-25T10:02:31.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.419+0000 7f7122000640 1 -- 172.21.15.165:0/3229833781 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7124059380 con 0x7f712c15baf0 2024-04-25T10:02:31.425 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.423+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f710c06b550 msgr2=0x7f710c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.423+0000 7f7130e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f710c06b550 0x7f710c06da10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f7118034000 tx=0x7f711802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.423+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 msgr2=0x7f712c1ebc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.423+0000 7f7130e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 0x7f712c1ebc80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f712400cd90 tx=0x7f712400a660 comp rx=0 tx=0).stop 2024-04-25T10:02:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 shutdown_connections 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f710c06b550 0x7f710c06da10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712c1513f0 0x7f712c1ed170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f712c15d160 0x7f712c0b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 --2- 172.21.15.165:0/3229833781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f712c15baf0 0x7f712c1ebc80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 >> 172.21.15.165:0/3229833781 conn(0x7f712c0d17c0 msgr2=0x7f712c071400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 shutdown_connections 2024-04-25T10:02:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.424+0000 7f7130e00640 1 -- 172.21.15.165:0/3229833781 wait complete. 2024-04-25T10:02:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.429+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6510005580 con 0x7f65380d32f0 2024-04-25T10:02:31.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.430+0000 7f652ea00640 1 -- 172.21.15.165:0/3473628849 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f65280587c0 con 0x7f65380d32f0 2024-04-25T10:02:31.437 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f651806b700 msgr2=0x7f651806dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f651806b700 0x7f651806dbc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6524001380 tx=0x7f6524001040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 msgr2=0x7f653815ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 0x7f653815ced0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f652800e870 tx=0x7f652800ed30 comp rx=0 tx=0).stop 2024-04-25T10:02:31.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 shutdown_connections 2024-04-25T10:02:31.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f651806b700 0x7f651806dbc0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6524001380 tx=0x7f6524001040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6538161710 0x7f65380ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f653815dc20 0x7f65381f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 --2- 172.21.15.165:0/3473628849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65380d32f0 0x7f653815ced0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 >> 172.21.15.165:0/3473628849 conn(0x7f65380d0420 msgr2=0x7f65380d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 shutdown_connections 2024-04-25T10:02:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.434+0000 7f653da00640 1 -- 172.21.15.165:0/3473628849 wait complete. 2024-04-25T10:02:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.437+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f99c00d16b0 con 0x7f99c0161710 2024-04-25T10:02:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.439+0000 7f99b6a00640 1 -- 172.21.15.165:0/3585572144 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f99b005cec0 con 0x7f99c0161710 2024-04-25T10:02:31.445 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.443+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99a006b5a0 msgr2=0x7f99a006da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.443+0000 7f99c5c00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99a006b5a0 0x7f99a006da60 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f99ac034000 tx=0x7f99ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.443+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c0161710 msgr2=0x7f99c00d3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.443+0000 7f99c5c00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c0161710 0x7f99c00d3980 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f99b000e7f0 tx=0x7f99b000ecb0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.444+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 shutdown_connections 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.444+0000 7f99c5c00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99a006b5a0 0x7f99a006da60 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.444+0000 7f99c5c00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99c0161710 0x7f99c00d3980 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.444+0000 7f99c5c00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99c015dc20 0x7f99c00aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.444+0000 7f99c5c00640 1 --2- 172.21.15.165:0/3585572144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99c00d32f0 0x7f99c014fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.444+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 >> 172.21.15.165:0/3585572144 conn(0x7f99c00d0420 msgr2=0x7f99c00cca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.445+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 shutdown_connections 2024-04-25T10:02:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.445+0000 7f99c5c00640 1 -- 172.21.15.165:0/3585572144 wait complete. 2024-04-25T10:02:31.457 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.6 2024-04-25T10:02:31.457 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.468 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.0 2024-04-25T10:02:31.468 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.476 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.4 2024-04-25T10:02:31.476 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.473+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3c8804a790 con 0x7f3c8802dda0 2024-04-25T10:02:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.473+0000 7f3c83400640 1 -- 172.21.15.165:0/2608907927 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3c7c0583d0 con 0x7f3c8802dda0 2024-04-25T10:02:31.479 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c7006b5d0 msgr2=0x7f3c7006da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c7006b5d0 0x7f3c7006da90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3c84033000 tx=0x7f3c84008030 comp rx=0 tx=0).stop 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 msgr2=0x7f3c88039090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 0x7f3c88039090 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f3c7c004f60 tx=0x7f3c7c004f90 comp rx=0 tx=0).stop 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 shutdown_connections 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c7006b5d0 0x7f3c7006da90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c880e8c00 0x7f3c880e90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c8802dda0 0x7f3c88039090 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 --2- 172.21.15.165:0/2608907927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c8802d9c0 0x7f3c88038b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 >> 172.21.15.165:0/2608907927 conn(0x7f3c88009fb0 msgr2=0x7f3c8804af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 shutdown_connections 2024-04-25T10:02:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.476+0000 7f3c95000640 1 -- 172.21.15.165:0/2608907927 wait complete. 2024-04-25T10:02:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.481+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f328c005850 con 0x7f32ac006ea0 2024-04-25T10:02:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.481+0000 7f32aa000640 1 -- 172.21.15.165:0/3648217244 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f32b40f7480 con 0x7f32ac006ea0 2024-04-25T10:02:31.490 INFO:teuthology.orchestra.run.smithi165.stdout:55834574858 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f329406b5a0 msgr2=0x7f329406da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f329406b5a0 0x7f329406da60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0396b0 tx=0x7f32a4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 msgr2=0x7f32ac0355e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 0x7f32ac0355e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f32b402d370 tx=0x7f32b4073660 comp rx=0 tx=0).stop 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 shutdown_connections 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f329406b5a0 0x7f329406da60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32ac0ead00 0x7f32ac0d9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32ac04ffd0 0x7f32ac035b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 --2- 172.21.15.165:0/3648217244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32ac006ea0 0x7f32ac0355e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 >> 172.21.15.165:0/3648217244 conn(0x7f32ac009fb0 msgr2=0x7f32ac037b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 shutdown_connections 2024-04-25T10:02:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.484+0000 7f32b3200640 1 -- 172.21.15.165:0/3648217244 wait complete. 2024-04-25T10:02:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.490+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fdc480ca120 con 0x7fdc481f55d0 2024-04-25T10:02:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.491+0000 7fdc3fe00640 1 -- 172.21.15.165:0/2679315451 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdc40058540 con 0x7fdc481f55d0 2024-04-25T10:02:31.496 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.492+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f716c06d9d0 con 0x7f716c06d2d0 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc2806b520 msgr2=0x7fdc2806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc2806b520 0x7fdc2806d9e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc38032000 tx=0x7fdc3802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 msgr2=0x7fdc480cb490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 0x7fdc480cb490 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdc40000f80 tx=0x7fdc4000a660 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 shutdown_connections 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc2806b520 0x7fdc2806d9e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc480cd4b0 0x7fdc48071540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7f7163400640 1 -- 172.21.15.165:0/1074902038 <== mon.2 v2:172.21.15.172:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f715c00ae20 con 0x7f716c06d2d0 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc481f55d0 0x7fdc480cb490 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 --2- 172.21.15.165:0/2679315451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc2c003a00 0x7fdc480ccc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 >> 172.21.15.165:0/2679315451 conn(0x7fdc480d0610 msgr2=0x7fdc480b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 shutdown_connections 2024-04-25T10:02:31.500 INFO:teuthology.orchestra.run.smithi165.stdout:55834574859 2024-04-25T10:02:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.493+0000 7fdc4d400640 1 -- 172.21.15.165:0/2679315451 wait complete. 2024-04-25T10:02:31.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.3 2024-04-25T10:02:31.501 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f714806b4d0 msgr2=0x7f714806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f714806b4d0 0x7f714806d990 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7154034000 tx=0x7f715402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 msgr2=0x7f716c14fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 0x7f716c14fff0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f715c00ce30 tx=0x7f715c00a650 comp rx=0 tx=0).stop 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 shutdown_connections 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f714806b4d0 0x7f714806d990 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f716c161fc0 0x7f716c163bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f716c06ea90 0x7f716c163680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 --2- 172.21.15.165:0/1074902038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f716c06d2d0 0x7f716c14fff0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 >> 172.21.15.165:0/1074902038 conn(0x7f716c06abe0 msgr2=0x7f716c15c810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 shutdown_connections 2024-04-25T10:02:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.496+0000 7f7171600640 1 -- 172.21.15.165:0/1074902038 wait complete. 2024-04-25T10:02:31.508 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574858 got 55834574858 for osd.2 2024-04-25T10:02:31.509 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.516 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.5 2024-04-25T10:02:31.516 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574859 got 55834574859 for osd.7 2024-04-25T10:02:31.518 DEBUG:teuthology.parallel:result is None 2024-04-25T10:02:31.518 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:02:31.519 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:31.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.741+0000 7ff408a00640 1 Processor -- start 2024-04-25T10:02:31.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.742+0000 7ff408a00640 1 -- start start 2024-04-25T10:02:31.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.743+0000 7ff408a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4040ce920 0x7ff4040ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.743+0000 7ff408a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff40406d290 0x7ff40406d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.743+0000 7ff408a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff404144b30 con 0x7ff404144750 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.743+0000 7ff408a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4040712d0 con 0x7ff40406d290 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.743+0000 7ff408a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4040715b0 con 0x7ff4040ce920 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff401e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff404144750 0x7ff40406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58174/0 (socket says 172.21.15.165:58174) 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff401e00640 1 -- 172.21.15.165:0/3134990936 learned_addr learned my addr 172.21.15.165:0/3134990936 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff402800640 1 --2- 172.21.15.165:0/3134990936 >> v2:172.21.15.165:3300/0 conn(0x7ff40406d290 0x7ff40406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff3fbe00640 1 --2- 172.21.15.165:0/3134990936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4040ce920 0x7ff4040ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff401400640 1 -- 172.21.15.165:0/3134990936 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3052665844 0 0) 0x7ff404144b30 con 0x7ff404144750 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff401400640 1 -- 172.21.15.165:0/3134990936 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3e8003520 con 0x7ff404144750 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff402800640 1 -- 172.21.15.165:0/3134990936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4040ce920 msgr2=0x7ff4040ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff402800640 1 --2- 172.21.15.165:0/3134990936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4040ce920 0x7ff4040ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff402800640 1 -- 172.21.15.165:0/3134990936 >> v1:172.21.15.165:6789/0 conn(0x7ff404144750 legacy=0x7ff40406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.744+0000 7ff402800640 1 -- 172.21.15.165:0/3134990936 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff404071890 con 0x7ff40406d290 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.745+0000 7ff402800640 1 --2- 172.21.15.165:0/3134990936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff40406d720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009d90 tx=0x7ff3fc02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cab4def726881fa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.745+0000 7ff401400640 1 -- 172.21.15.165:0/3134990936 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc007740 con 0x7ff40406d290 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.745+0000 7ff401400640 1 -- 172.21.15.165:0/3134990936 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3fc0078e0 con 0x7ff40406d290 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.745+0000 7ff401400640 1 -- 172.21.15.165:0/3134990936 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3fc007cb0 con 0x7ff40406d290 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 -- 172.21.15.165:0/3134990936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 msgr2=0x7ff40406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 --2- 172.21.15.165:0/3134990936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff40406d720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009d90 tx=0x7ff3fc02a660 comp rx=0 tx=0).stop 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 -- 172.21.15.165:0/3134990936 shutdown_connections 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 --2- 172.21.15.165:0/3134990936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff40406d720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 --2- 172.21.15.165:0/3134990936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4040ce920 0x7ff4040ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 -- 172.21.15.165:0/3134990936 >> 172.21.15.165:0/3134990936 conn(0x7ff40406abe0 msgr2=0x7ff40406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 -- 172.21.15.165:0/3134990936 shutdown_connections 2024-04-25T10:02:31.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.746+0000 7ff408a00640 1 -- 172.21.15.165:0/3134990936 wait complete. 2024-04-25T10:02:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.748+0000 7ff408a00640 1 Processor -- start 2024-04-25T10:02:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.748+0000 7ff408a00640 1 -- start start 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff408a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff4041e7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff408a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040ce920 0x7ff4041e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff408a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff404144750 0x7ff404141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff408a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40406f020 con 0x7ff40406d290 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff408a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40406ece0 con 0x7ff4040ce920 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff408a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff404145450 con 0x7ff404144750 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff4041e7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff4041e7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50628/0 (socket says 172.21.15.165:50628) 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 -- 172.21.15.165:0/2020157120 learned_addr learned my addr 172.21.15.165:0/2020157120 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 -- 172.21.15.165:0/2020157120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff404144750 msgr2=0x7ff404141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff404144750 0x7ff404141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 -- 172.21.15.165:0/2020157120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040ce920 msgr2=0x7ff4041e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040ce920 0x7ff4041e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.749+0000 7ff401e00640 1 -- 172.21.15.165:0/2020157120 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4041e5ea0 con 0x7ff40406d290 2024-04-25T10:02:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.750+0000 7ff401e00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff4041e7b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400c460 tx=0x7ff3f400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.750+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4017070 con 0x7ff40406d290 2024-04-25T10:02:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.750+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3f4005ce0 con 0x7ff40406d290 2024-04-25T10:02:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.750+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4007290 con 0x7ff40406d290 2024-04-25T10:02:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.750+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4041e6180 con 0x7ff40406d290 2024-04-25T10:02:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.750+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4041e6740 con 0x7ff40406d290 2024-04-25T10:02:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.751+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff404071b70 con 0x7ff40406d290 2024-04-25T10:02:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.752+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3f401a070 con 0x7ff40406d290 2024-04-25T10:02:31.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.753+0000 7ff3fa000640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:31.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.753+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff3f4089940 con 0x7ff40406d290 2024-04-25T10:02:31.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.753+0000 7ff3fbe00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:31.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.754+0000 7ff3fbe00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3e006b4d0 0x7ff3e006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff404144020 tx=0x7ff3ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:31.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.757+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff404071b70 con 0x7ff40406d290 2024-04-25T10:02:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.933+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4040d0510 con 0x7ff3e006b4d0 2024-04-25T10:02:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.942+0000 7ff3fa000640 1 -- 172.21.15.165:0/2020157120 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332537 (secure 0 0 0) 0x7ff4040d0510 con 0x7ff3e006b4d0 2024-04-25T10:02:31.951 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3e006b4d0 msgr2=0x7ff3e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3e006b4d0 0x7ff3e006d990 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff404144020 tx=0x7ff3ec008040 comp rx=0 tx=0).stop 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 msgr2=0x7ff4041e7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff4041e7b00 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff3f400c460 tx=0x7ff3f400c920 comp rx=0 tx=0).stop 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 shutdown_connections 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3e006b4d0 0x7ff3e006d990 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff404144750 0x7ff404141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4040ce920 0x7ff4041e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 --2- 172.21.15.165:0/2020157120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff40406d290 0x7ff4041e7b00 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 >> 172.21.15.165:0/2020157120 conn(0x7ff40406abe0 msgr2=0x7ff40406d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 shutdown_connections 2024-04-25T10:02:31.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:31.948+0000 7ff408a00640 1 -- 172.21.15.165:0/2020157120 wait complete. 2024-04-25T10:02:31.971 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":36,"stamp":"2024-04-25T10:02:30.131636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219420,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755300,"statfs":{"total":773094113280,"available":772869427200,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004620"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513776+0000","last_change":"2024-04-25T10:02:03.142252+0000","last_active":"2024-04-25T10:02:09.513776+0000","last_peered":"2024-04-25T10:02:09.513776+0000","last_clean":"2024-04-25T10:02:09.513776+0000","last_became_active":"2024-04-25T10:02:03.141984+0000","last_became_peered":"2024-04-25T10:02:03.141984+0000","last_unstale":"2024-04-25T10:02:09.513776+0000","last_undegraded":"2024-04-25T10:02:09.513776+0000","last_fullsized":"2024-04-25T10:02:09.513776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520621+0000","last_change":"2024-04-25T10:02:04.156786+0000","last_active":"2024-04-25T10:02:09.520621+0000","last_peered":"2024-04-25T10:02:09.520621+0000","last_clean":"2024-04-25T10:02:09.520621+0000","last_became_active":"2024-04-25T10:02:04.156522+0000","last_became_peered":"2024-04-25T10:02:04.156522+0000","last_unstale":"2024-04-25T10:02:09.520621+0000","last_undegraded":"2024-04-25T10:02:09.520621+0000","last_fullsized":"2024-04-25T10:02:09.520621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519839+0000","last_change":"2024-04-25T10:02:08.409422+0000","last_active":"2024-04-25T10:02:09.519839+0000","last_peered":"2024-04-25T10:02:09.519839+0000","last_clean":"2024-04-25T10:02:09.519839+0000","last_became_active":"2024-04-25T10:02:08.408455+0000","last_became_peered":"2024-04-25T10:02:08.408455+0000","last_unstale":"2024-04-25T10:02:09.519839+0000","last_undegraded":"2024-04-25T10:02:09.519839+0000","last_fullsized":"2024-04-25T10:02:09.519839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:32.901672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520971+0000","last_change":"2024-04-25T10:02:03.131043+0000","last_active":"2024-04-25T10:02:09.520971+0000","last_peered":"2024-04-25T10:02:09.520971+0000","last_clean":"2024-04-25T10:02:09.520971+0000","last_became_active":"2024-04-25T10:02:03.130682+0000","last_became_peered":"2024-04-25T10:02:03.130682+0000","last_unstale":"2024-04-25T10:02:09.520971+0000","last_undegraded":"2024-04-25T10:02:09.520971+0000","last_fullsized":"2024-04-25T10:02:09.520971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513286+0000","last_change":"2024-04-25T10:02:04.142528+0000","last_active":"2024-04-25T10:02:09.513286+0000","last_peered":"2024-04-25T10:02:09.513286+0000","last_clean":"2024-04-25T10:02:09.513286+0000","last_became_active":"2024-04-25T10:02:04.142188+0000","last_became_peered":"2024-04-25T10:02:04.142188+0000","last_unstale":"2024-04-25T10:02:09.513286+0000","last_undegraded":"2024-04-25T10:02:09.513286+0000","last_fullsized":"2024-04-25T10:02:09.513286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183112+0000","last_change":"2024-04-25T10:02:08.192140+0000","last_active":"2024-04-25T10:02:09.183112+0000","last_peered":"2024-04-25T10:02:09.183112+0000","last_clean":"2024-04-25T10:02:09.183112+0000","last_became_active":"2024-04-25T10:02:08.191658+0000","last_became_peered":"2024-04-25T10:02:08.191658+0000","last_unstale":"2024-04-25T10:02:09.183112+0000","last_undegraded":"2024-04-25T10:02:09.183112+0000","last_fullsized":"2024-04-25T10:02:09.183112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:45.834272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514852+0000","last_change":"2024-04-25T10:02:03.147520+0000","last_active":"2024-04-25T10:02:09.514852+0000","last_peered":"2024-04-25T10:02:09.514852+0000","last_clean":"2024-04-25T10:02:09.514852+0000","last_became_active":"2024-04-25T10:02:03.147274+0000","last_became_peered":"2024-04-25T10:02:03.147274+0000","last_unstale":"2024-04-25T10:02:09.514852+0000","last_undegraded":"2024-04-25T10:02:09.514852+0000","last_fullsized":"2024-04-25T10:02:09.514852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516102+0000","last_change":"2024-04-25T10:02:04.137351+0000","last_active":"2024-04-25T10:02:09.516102+0000","last_peered":"2024-04-25T10:02:09.516102+0000","last_clean":"2024-04-25T10:02:09.516102+0000","last_became_active":"2024-04-25T10:02:04.136927+0000","last_became_peered":"2024-04-25T10:02:04.136927+0000","last_unstale":"2024-04-25T10:02:09.516102+0000","last_undegraded":"2024-04-25T10:02:09.516102+0000","last_fullsized":"2024-04-25T10:02:09.516102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185918+0000","last_change":"2024-04-25T10:02:08.198068+0000","last_active":"2024-04-25T10:02:09.185918+0000","last_peered":"2024-04-25T10:02:09.185918+0000","last_clean":"2024-04-25T10:02:09.185918+0000","last_became_active":"2024-04-25T10:02:08.196209+0000","last_became_peered":"2024-04-25T10:02:08.196209+0000","last_unstale":"2024-04-25T10:02:09.185918+0000","last_undegraded":"2024-04-25T10:02:09.185918+0000","last_fullsized":"2024-04-25T10:02:09.185918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:48.568415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514318+0000","last_change":"2024-04-25T10:02:03.157531+0000","last_active":"2024-04-25T10:02:09.514318+0000","last_peered":"2024-04-25T10:02:09.514318+0000","last_clean":"2024-04-25T10:02:09.514318+0000","last_became_active":"2024-04-25T10:02:03.157216+0000","last_became_peered":"2024-04-25T10:02:03.157216+0000","last_unstale":"2024-04-25T10:02:09.514318+0000","last_undegraded":"2024-04-25T10:02:09.514318+0000","last_fullsized":"2024-04-25T10:02:09.514318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516612+0000","last_change":"2024-04-25T10:02:04.140082+0000","last_active":"2024-04-25T10:02:09.516612+0000","last_peered":"2024-04-25T10:02:09.516612+0000","last_clean":"2024-04-25T10:02:09.516612+0000","last_became_active":"2024-04-25T10:02:04.136765+0000","last_became_peered":"2024-04-25T10:02:04.136765+0000","last_unstale":"2024-04-25T10:02:09.516612+0000","last_undegraded":"2024-04-25T10:02:09.516612+0000","last_fullsized":"2024-04-25T10:02:09.516612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185629+0000","last_change":"2024-04-25T10:02:08.189048+0000","last_active":"2024-04-25T10:02:09.185629+0000","last_peered":"2024-04-25T10:02:09.185629+0000","last_clean":"2024-04-25T10:02:09.185629+0000","last_became_active":"2024-04-25T10:02:08.188475+0000","last_became_peered":"2024-04-25T10:02:08.188475+0000","last_unstale":"2024-04-25T10:02:09.185629+0000","last_undegraded":"2024-04-25T10:02:09.185629+0000","last_fullsized":"2024-04-25T10:02:09.185629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:08.343719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182619+0000","last_change":"2024-04-25T10:02:04.132020+0000","last_active":"2024-04-25T10:02:09.182619+0000","last_peered":"2024-04-25T10:02:09.182619+0000","last_clean":"2024-04-25T10:02:09.182619+0000","last_became_active":"2024-04-25T10:02:04.131018+0000","last_became_peered":"2024-04-25T10:02:04.131018+0000","last_unstale":"2024-04-25T10:02:09.182619+0000","last_undegraded":"2024-04-25T10:02:09.182619+0000","last_fullsized":"2024-04-25T10:02:09.182619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514507+0000","last_change":"2024-04-25T10:02:03.147595+0000","last_active":"2024-04-25T10:02:09.514507+0000","last_peered":"2024-04-25T10:02:09.514507+0000","last_clean":"2024-04-25T10:02:09.514507+0000","last_became_active":"2024-04-25T10:02:03.147038+0000","last_became_peered":"2024-04-25T10:02:03.147038+0000","last_unstale":"2024-04-25T10:02:09.514507+0000","last_undegraded":"2024-04-25T10:02:09.514507+0000","last_fullsized":"2024-04-25T10:02:09.514507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:50.278310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188671+0000","last_change":"2024-04-25T10:02:08.186691+0000","last_active":"2024-04-25T10:02:09.188671+0000","last_peered":"2024-04-25T10:02:09.188671+0000","last_clean":"2024-04-25T10:02:09.188671+0000","last_became_active":"2024-04-25T10:02:08.186276+0000","last_became_peered":"2024-04-25T10:02:08.186276+0000","last_unstale":"2024-04-25T10:02:09.188671+0000","last_undegraded":"2024-04-25T10:02:09.188671+0000","last_fullsized":"2024-04-25T10:02:09.188671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520343+0000","last_change":"2024-04-25T10:02:04.139116+0000","last_active":"2024-04-25T10:02:09.520343+0000","last_peered":"2024-04-25T10:02:09.520343+0000","last_clean":"2024-04-25T10:02:09.520343+0000","last_became_active":"2024-04-25T10:02:04.138565+0000","last_became_peered":"2024-04-25T10:02:04.138565+0000","last_unstale":"2024-04-25T10:02:09.520343+0000","last_undegraded":"2024-04-25T10:02:09.520343+0000","last_fullsized":"2024-04-25T10:02:09.520343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518432+0000","last_change":"2024-04-25T10:02:03.132968+0000","last_active":"2024-04-25T10:02:09.518432+0000","last_peered":"2024-04-25T10:02:09.518432+0000","last_clean":"2024-04-25T10:02:09.518432+0000","last_became_active":"2024-04-25T10:02:03.132624+0000","last_became_peered":"2024-04-25T10:02:03.132624+0000","last_unstale":"2024-04-25T10:02:09.518432+0000","last_undegraded":"2024-04-25T10:02:09.518432+0000","last_fullsized":"2024-04-25T10:02:09.518432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187996+0000","last_change":"2024-04-25T10:02:08.193709+0000","last_active":"2024-04-25T10:02:09.187996+0000","last_peered":"2024-04-25T10:02:09.187996+0000","last_clean":"2024-04-25T10:02:09.187996+0000","last_became_active":"2024-04-25T10:02:08.193240+0000","last_became_peered":"2024-04-25T10:02:08.193240+0000","last_unstale":"2024-04-25T10:02:09.187996+0000","last_undegraded":"2024-04-25T10:02:09.187996+0000","last_fullsized":"2024-04-25T10:02:09.187996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519675+0000","last_change":"2024-04-25T10:02:04.139077+0000","last_active":"2024-04-25T10:02:09.519675+0000","last_peered":"2024-04-25T10:02:09.519675+0000","last_clean":"2024-04-25T10:02:09.519675+0000","last_became_active":"2024-04-25T10:02:04.138730+0000","last_became_peered":"2024-04-25T10:02:04.138730+0000","last_unstale":"2024-04-25T10:02:09.519675+0000","last_undegraded":"2024-04-25T10:02:09.519675+0000","last_fullsized":"2024-04-25T10:02:09.519675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514609+0000","last_change":"2024-04-25T10:02:03.137563+0000","last_active":"2024-04-25T10:02:09.514609+0000","last_peered":"2024-04-25T10:02:09.514609+0000","last_clean":"2024-04-25T10:02:09.514609+0000","last_became_active":"2024-04-25T10:02:03.137080+0000","last_became_peered":"2024-04-25T10:02:03.137080+0000","last_unstale":"2024-04-25T10:02:09.514609+0000","last_undegraded":"2024-04-25T10:02:09.514609+0000","last_fullsized":"2024-04-25T10:02:09.514609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517371+0000","last_change":"2024-04-25T10:02:08.404747+0000","last_active":"2024-04-25T10:02:09.517371+0000","last_peered":"2024-04-25T10:02:09.517371+0000","last_clean":"2024-04-25T10:02:09.517371+0000","last_became_active":"2024-04-25T10:02:08.403706+0000","last_became_peered":"2024-04-25T10:02:08.403706+0000","last_unstale":"2024-04-25T10:02:09.517371+0000","last_undegraded":"2024-04-25T10:02:09.517371+0000","last_fullsized":"2024-04-25T10:02:09.517371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185839+0000","last_change":"2024-04-25T10:02:04.125617+0000","last_active":"2024-04-25T10:02:09.185839+0000","last_peered":"2024-04-25T10:02:09.185839+0000","last_clean":"2024-04-25T10:02:09.185839+0000","last_became_active":"2024-04-25T10:02:04.125319+0000","last_became_peered":"2024-04-25T10:02:04.125319+0000","last_unstale":"2024-04-25T10:02:09.185839+0000","last_undegraded":"2024-04-25T10:02:09.185839+0000","last_fullsized":"2024-04-25T10:02:09.185839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.271382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512743+0000","last_change":"2024-04-25T10:02:03.145754+0000","last_active":"2024-04-25T10:02:09.512743+0000","last_peered":"2024-04-25T10:02:09.512743+0000","last_clean":"2024-04-25T10:02:09.512743+0000","last_became_active":"2024-04-25T10:02:03.145400+0000","last_became_peered":"2024-04-25T10:02:03.145400+0000","last_unstale":"2024-04-25T10:02:09.512743+0000","last_undegraded":"2024-04-25T10:02:09.512743+0000","last_fullsized":"2024-04-25T10:02:09.512743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188721+0000","last_change":"2024-04-25T10:02:08.197346+0000","last_active":"2024-04-25T10:02:09.188721+0000","last_peered":"2024-04-25T10:02:09.188721+0000","last_clean":"2024-04-25T10:02:09.188721+0000","last_became_active":"2024-04-25T10:02:08.196902+0000","last_became_peered":"2024-04-25T10:02:08.196902+0000","last_unstale":"2024-04-25T10:02:09.188721+0000","last_undegraded":"2024-04-25T10:02:09.188721+0000","last_fullsized":"2024-04-25T10:02:09.188721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:43.992278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515041+0000","last_change":"2024-04-25T10:02:04.123238+0000","last_active":"2024-04-25T10:02:09.515041+0000","last_peered":"2024-04-25T10:02:09.515041+0000","last_clean":"2024-04-25T10:02:09.515041+0000","last_became_active":"2024-04-25T10:02:04.122886+0000","last_became_peered":"2024-04-25T10:02:04.122886+0000","last_unstale":"2024-04-25T10:02:09.515041+0000","last_undegraded":"2024-04-25T10:02:09.515041+0000","last_fullsized":"2024-04-25T10:02:09.515041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187935+0000","last_change":"2024-04-25T10:02:03.138021+0000","last_active":"2024-04-25T10:02:09.187935+0000","last_peered":"2024-04-25T10:02:09.187935+0000","last_clean":"2024-04-25T10:02:09.187935+0000","last_became_active":"2024-04-25T10:02:03.137531+0000","last_became_peered":"2024-04-25T10:02:03.137531+0000","last_unstale":"2024-04-25T10:02:09.187935+0000","last_undegraded":"2024-04-25T10:02:09.187935+0000","last_fullsized":"2024-04-25T10:02:09.187935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185644+0000","last_change":"2024-04-25T10:02:08.404604+0000","last_active":"2024-04-25T10:02:09.185644+0000","last_peered":"2024-04-25T10:02:09.185644+0000","last_clean":"2024-04-25T10:02:09.185644+0000","last_became_active":"2024-04-25T10:02:08.403392+0000","last_became_peered":"2024-04-25T10:02:08.403392+0000","last_unstale":"2024-04-25T10:02:09.185644+0000","last_undegraded":"2024-04-25T10:02:09.185644+0000","last_fullsized":"2024-04-25T10:02:09.185644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:25.532310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521257+0000","last_change":"2024-04-25T10:02:04.144042+0000","last_active":"2024-04-25T10:02:09.521257+0000","last_peered":"2024-04-25T10:02:09.521257+0000","last_clean":"2024-04-25T10:02:09.521257+0000","last_became_active":"2024-04-25T10:02:04.143734+0000","last_became_peered":"2024-04-25T10:02:04.143734+0000","last_unstale":"2024-04-25T10:02:09.521257+0000","last_undegraded":"2024-04-25T10:02:09.521257+0000","last_fullsized":"2024-04-25T10:02:09.521257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516285+0000","last_change":"2024-04-25T10:02:03.138015+0000","last_active":"2024-04-25T10:02:09.516285+0000","last_peered":"2024-04-25T10:02:09.516285+0000","last_clean":"2024-04-25T10:02:09.516285+0000","last_became_active":"2024-04-25T10:02:03.137487+0000","last_became_peered":"2024-04-25T10:02:03.137487+0000","last_unstale":"2024-04-25T10:02:09.516285+0000","last_undegraded":"2024-04-25T10:02:09.516285+0000","last_fullsized":"2024-04-25T10:02:09.516285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515852+0000","last_change":"2024-04-25T10:02:08.402309+0000","last_active":"2024-04-25T10:02:09.515852+0000","last_peered":"2024-04-25T10:02:09.515852+0000","last_clean":"2024-04-25T10:02:09.515852+0000","last_became_active":"2024-04-25T10:02:08.401865+0000","last_became_peered":"2024-04-25T10:02:08.401865+0000","last_unstale":"2024-04-25T10:02:09.515852+0000","last_undegraded":"2024-04-25T10:02:09.515852+0000","last_fullsized":"2024-04-25T10:02:09.515852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519191+0000","last_change":"2024-04-25T10:02:04.129398+0000","last_active":"2024-04-25T10:02:09.519191+0000","last_peered":"2024-04-25T10:02:09.519191+0000","last_clean":"2024-04-25T10:02:09.519191+0000","last_became_active":"2024-04-25T10:02:04.128879+0000","last_became_peered":"2024-04-25T10:02:04.128879+0000","last_unstale":"2024-04-25T10:02:09.519191+0000","last_undegraded":"2024-04-25T10:02:09.519191+0000","last_fullsized":"2024-04-25T10:02:09.519191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515418+0000","last_change":"2024-04-25T10:02:03.146769+0000","last_active":"2024-04-25T10:02:09.515418+0000","last_peered":"2024-04-25T10:02:09.515418+0000","last_clean":"2024-04-25T10:02:09.515418+0000","last_became_active":"2024-04-25T10:02:03.146576+0000","last_became_peered":"2024-04-25T10:02:03.146576+0000","last_unstale":"2024-04-25T10:02:09.515418+0000","last_undegraded":"2024-04-25T10:02:09.515418+0000","last_fullsized":"2024-04-25T10:02:09.515418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513472+0000","last_change":"2024-04-25T10:02:08.393230+0000","last_active":"2024-04-25T10:02:09.513472+0000","last_peered":"2024-04-25T10:02:09.513472+0000","last_clean":"2024-04-25T10:02:09.513472+0000","last_became_active":"2024-04-25T10:02:08.392691+0000","last_became_peered":"2024-04-25T10:02:08.392691+0000","last_unstale":"2024-04-25T10:02:09.513472+0000","last_undegraded":"2024-04-25T10:02:09.513472+0000","last_fullsized":"2024-04-25T10:02:09.513472+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516864+0000","last_change":"2024-04-25T10:02:04.140436+0000","last_active":"2024-04-25T10:02:09.516864+0000","last_peered":"2024-04-25T10:02:09.516864+0000","last_clean":"2024-04-25T10:02:09.516864+0000","last_became_active":"2024-04-25T10:02:04.140075+0000","last_became_peered":"2024-04-25T10:02:04.140075+0000","last_unstale":"2024-04-25T10:02:09.516864+0000","last_undegraded":"2024-04-25T10:02:09.516864+0000","last_fullsized":"2024-04-25T10:02:09.516864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519440+0000","last_change":"2024-04-25T10:02:03.132553+0000","last_active":"2024-04-25T10:02:09.519440+0000","last_peered":"2024-04-25T10:02:09.519440+0000","last_clean":"2024-04-25T10:02:09.519440+0000","last_became_active":"2024-04-25T10:02:03.132241+0000","last_became_peered":"2024-04-25T10:02:03.132241+0000","last_unstale":"2024-04-25T10:02:09.519440+0000","last_undegraded":"2024-04-25T10:02:09.519440+0000","last_fullsized":"2024-04-25T10:02:09.519440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518024+0000","last_change":"2024-04-25T10:02:08.186096+0000","last_active":"2024-04-25T10:02:09.518024+0000","last_peered":"2024-04-25T10:02:09.518024+0000","last_clean":"2024-04-25T10:02:09.518024+0000","last_became_active":"2024-04-25T10:02:08.185660+0000","last_became_peered":"2024-04-25T10:02:08.185660+0000","last_unstale":"2024-04-25T10:02:09.518024+0000","last_undegraded":"2024-04-25T10:02:09.518024+0000","last_fullsized":"2024-04-25T10:02:09.518024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188203+0000","last_change":"2024-04-25T10:02:04.126935+0000","last_active":"2024-04-25T10:02:09.188203+0000","last_peered":"2024-04-25T10:02:09.188203+0000","last_clean":"2024-04-25T10:02:09.188203+0000","last_became_active":"2024-04-25T10:02:04.126268+0000","last_became_peered":"2024-04-25T10:02:04.126268+0000","last_unstale":"2024-04-25T10:02:09.188203+0000","last_undegraded":"2024-04-25T10:02:09.188203+0000","last_fullsized":"2024-04-25T10:02:09.188203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515075+0000","last_change":"2024-04-25T10:02:03.143964+0000","last_active":"2024-04-25T10:02:09.515075+0000","last_peered":"2024-04-25T10:02:09.515075+0000","last_clean":"2024-04-25T10:02:09.515075+0000","last_became_active":"2024-04-25T10:02:03.143548+0000","last_became_peered":"2024-04-25T10:02:03.143548+0000","last_unstale":"2024-04-25T10:02:09.515075+0000","last_undegraded":"2024-04-25T10:02:09.515075+0000","last_fullsized":"2024-04-25T10:02:09.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519212+0000","last_change":"2024-04-25T10:02:08.196544+0000","last_active":"2024-04-25T10:02:09.519212+0000","last_peered":"2024-04-25T10:02:09.519212+0000","last_clean":"2024-04-25T10:02:09.519212+0000","last_became_active":"2024-04-25T10:02:08.196131+0000","last_became_peered":"2024-04-25T10:02:08.196131+0000","last_unstale":"2024-04-25T10:02:09.519212+0000","last_undegraded":"2024-04-25T10:02:09.519212+0000","last_fullsized":"2024-04-25T10:02:09.519212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513146+0000","last_change":"2024-04-25T10:02:04.124338+0000","last_active":"2024-04-25T10:02:09.513146+0000","last_peered":"2024-04-25T10:02:09.513146+0000","last_clean":"2024-04-25T10:02:09.513146+0000","last_became_active":"2024-04-25T10:02:04.123720+0000","last_became_peered":"2024-04-25T10:02:04.123720+0000","last_unstale":"2024-04-25T10:02:09.513146+0000","last_undegraded":"2024-04-25T10:02:09.513146+0000","last_fullsized":"2024-04-25T10:02:09.513146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181759+0000","last_change":"2024-04-25T10:02:03.131111+0000","last_active":"2024-04-25T10:02:09.181759+0000","last_peered":"2024-04-25T10:02:09.181759+0000","last_clean":"2024-04-25T10:02:09.181759+0000","last_became_active":"2024-04-25T10:02:03.130828+0000","last_became_peered":"2024-04-25T10:02:03.130828+0000","last_unstale":"2024-04-25T10:02:09.181759+0000","last_undegraded":"2024-04-25T10:02:09.181759+0000","last_fullsized":"2024-04-25T10:02:09.181759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517914+0000","last_change":"2024-04-25T10:02:08.193216+0000","last_active":"2024-04-25T10:02:09.517914+0000","last_peered":"2024-04-25T10:02:09.517914+0000","last_clean":"2024-04-25T10:02:09.517914+0000","last_became_active":"2024-04-25T10:02:08.192717+0000","last_became_peered":"2024-04-25T10:02:08.192717+0000","last_unstale":"2024-04-25T10:02:09.517914+0000","last_undegraded":"2024-04-25T10:02:09.517914+0000","last_fullsized":"2024-04-25T10:02:09.517914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513507+0000","last_change":"2024-04-25T10:02:04.131883+0000","last_active":"2024-04-25T10:02:09.513507+0000","last_peered":"2024-04-25T10:02:09.513507+0000","last_clean":"2024-04-25T10:02:09.513507+0000","last_became_active":"2024-04-25T10:02:04.131591+0000","last_became_peered":"2024-04-25T10:02:04.131591+0000","last_unstale":"2024-04-25T10:02:09.513507+0000","last_undegraded":"2024-04-25T10:02:09.513507+0000","last_fullsized":"2024-04-25T10:02:09.513507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513031+0000","last_change":"2024-04-25T10:02:03.146201+0000","last_active":"2024-04-25T10:02:09.513031+0000","last_peered":"2024-04-25T10:02:09.513031+0000","last_clean":"2024-04-25T10:02:09.513031+0000","last_became_active":"2024-04-25T10:02:03.145888+0000","last_became_peered":"2024-04-25T10:02:03.145888+0000","last_unstale":"2024-04-25T10:02:09.513031+0000","last_undegraded":"2024-04-25T10:02:09.513031+0000","last_fullsized":"2024-04-25T10:02:09.513031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514736+0000","last_change":"2024-04-25T10:02:08.403944+0000","last_active":"2024-04-25T10:02:09.514736+0000","last_peered":"2024-04-25T10:02:09.514736+0000","last_clean":"2024-04-25T10:02:09.514736+0000","last_became_active":"2024-04-25T10:02:08.403447+0000","last_became_peered":"2024-04-25T10:02:08.403447+0000","last_unstale":"2024-04-25T10:02:09.514736+0000","last_undegraded":"2024-04-25T10:02:09.514736+0000","last_fullsized":"2024-04-25T10:02:09.514736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520189+0000","last_change":"2024-04-25T10:02:04.145181+0000","last_active":"2024-04-25T10:02:09.520189+0000","last_peered":"2024-04-25T10:02:09.520189+0000","last_clean":"2024-04-25T10:02:09.520189+0000","last_became_active":"2024-04-25T10:02:04.145007+0000","last_became_peered":"2024-04-25T10:02:04.145007+0000","last_unstale":"2024-04-25T10:02:09.520189+0000","last_undegraded":"2024-04-25T10:02:09.520189+0000","last_fullsized":"2024-04-25T10:02:09.520189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515574+0000","last_change":"2024-04-25T10:02:03.149716+0000","last_active":"2024-04-25T10:02:09.515574+0000","last_peered":"2024-04-25T10:02:09.515574+0000","last_clean":"2024-04-25T10:02:09.515574+0000","last_became_active":"2024-04-25T10:02:03.149427+0000","last_became_peered":"2024-04-25T10:02:03.149427+0000","last_unstale":"2024-04-25T10:02:09.515574+0000","last_undegraded":"2024-04-25T10:02:09.515574+0000","last_fullsized":"2024-04-25T10:02:09.515574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513340+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:02:09.513340+0000","last_peered":"2024-04-25T10:02:09.513340+0000","last_clean":"2024-04-25T10:02:09.513340+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:02:09.513340+0000","last_undegraded":"2024-04-25T10:02:09.513340+0000","last_fullsized":"2024-04-25T10:02:09.513340+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183489+0000","last_change":"2024-04-25T10:02:08.405095+0000","last_active":"2024-04-25T10:02:09.183489+0000","last_peered":"2024-04-25T10:02:09.183489+0000","last_clean":"2024-04-25T10:02:09.183489+0000","last_became_active":"2024-04-25T10:02:08.403767+0000","last_became_peered":"2024-04-25T10:02:08.403767+0000","last_unstale":"2024-04-25T10:02:09.183489+0000","last_undegraded":"2024-04-25T10:02:09.183489+0000","last_fullsized":"2024-04-25T10:02:09.183489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514594+0000","last_change":"2024-04-25T10:02:04.151648+0000","last_active":"2024-04-25T10:02:09.514594+0000","last_peered":"2024-04-25T10:02:09.514594+0000","last_clean":"2024-04-25T10:02:09.514594+0000","last_became_active":"2024-04-25T10:02:04.150680+0000","last_became_peered":"2024-04-25T10:02:04.150680+0000","last_unstale":"2024-04-25T10:02:09.514594+0000","last_undegraded":"2024-04-25T10:02:09.514594+0000","last_fullsized":"2024-04-25T10:02:09.514594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182505+0000","last_change":"2024-04-25T10:02:03.154627+0000","last_active":"2024-04-25T10:02:09.182505+0000","last_peered":"2024-04-25T10:02:09.182505+0000","last_clean":"2024-04-25T10:02:09.182505+0000","last_became_active":"2024-04-25T10:02:03.154346+0000","last_became_peered":"2024-04-25T10:02:03.154346+0000","last_unstale":"2024-04-25T10:02:09.182505+0000","last_undegraded":"2024-04-25T10:02:09.182505+0000","last_fullsized":"2024-04-25T10:02:09.182505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184009+0000","last_change":"2024-04-25T10:02:08.394382+0000","last_active":"2024-04-25T10:02:09.184009+0000","last_peered":"2024-04-25T10:02:09.184009+0000","last_clean":"2024-04-25T10:02:09.184009+0000","last_became_active":"2024-04-25T10:02:08.393222+0000","last_became_peered":"2024-04-25T10:02:08.393222+0000","last_unstale":"2024-04-25T10:02:09.184009+0000","last_undegraded":"2024-04-25T10:02:09.184009+0000","last_fullsized":"2024-04-25T10:02:09.184009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516304+0000","last_change":"2024-04-25T10:02:04.144851+0000","last_active":"2024-04-25T10:02:09.516304+0000","last_peered":"2024-04-25T10:02:09.516304+0000","last_clean":"2024-04-25T10:02:09.516304+0000","last_became_active":"2024-04-25T10:02:04.144530+0000","last_became_peered":"2024-04-25T10:02:04.144530+0000","last_unstale":"2024-04-25T10:02:09.516304+0000","last_undegraded":"2024-04-25T10:02:09.516304+0000","last_fullsized":"2024-04-25T10:02:09.516304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515591+0000","last_change":"2024-04-25T10:02:03.133061+0000","last_active":"2024-04-25T10:02:09.515591+0000","last_peered":"2024-04-25T10:02:09.515591+0000","last_clean":"2024-04-25T10:02:09.515591+0000","last_became_active":"2024-04-25T10:02:03.132762+0000","last_became_peered":"2024-04-25T10:02:03.132762+0000","last_unstale":"2024-04-25T10:02:09.515591+0000","last_undegraded":"2024-04-25T10:02:09.515591+0000","last_fullsized":"2024-04-25T10:02:09.515591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513927+0000","last_change":"2024-04-25T10:02:08.186615+0000","last_active":"2024-04-25T10:02:09.513927+0000","last_peered":"2024-04-25T10:02:09.513927+0000","last_clean":"2024-04-25T10:02:09.513927+0000","last_became_active":"2024-04-25T10:02:08.186216+0000","last_became_peered":"2024-04-25T10:02:08.186216+0000","last_unstale":"2024-04-25T10:02:09.513927+0000","last_undegraded":"2024-04-25T10:02:09.513927+0000","last_fullsized":"2024-04-25T10:02:09.513927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515930+0000","last_change":"2024-04-25T10:02:04.153131+0000","last_active":"2024-04-25T10:02:09.515930+0000","last_peered":"2024-04-25T10:02:09.515930+0000","last_clean":"2024-04-25T10:02:09.515930+0000","last_became_active":"2024-04-25T10:02:04.152415+0000","last_became_peered":"2024-04-25T10:02:04.152415+0000","last_unstale":"2024-04-25T10:02:09.515930+0000","last_undegraded":"2024-04-25T10:02:09.515930+0000","last_fullsized":"2024-04-25T10:02:09.515930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:54.646358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182568+0000","last_change":"2024-04-25T10:02:03.134495+0000","last_active":"2024-04-25T10:02:09.182568+0000","last_peered":"2024-04-25T10:02:09.182568+0000","last_clean":"2024-04-25T10:02:09.182568+0000","last_became_active":"2024-04-25T10:02:03.134157+0000","last_became_peered":"2024-04-25T10:02:03.134157+0000","last_unstale":"2024-04-25T10:02:09.182568+0000","last_undegraded":"2024-04-25T10:02:09.182568+0000","last_fullsized":"2024-04-25T10:02:09.182568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512930+0000","last_change":"2024-04-25T10:02:08.406412+0000","last_active":"2024-04-25T10:02:09.512930+0000","last_peered":"2024-04-25T10:02:09.512930+0000","last_clean":"2024-04-25T10:02:09.512930+0000","last_became_active":"2024-04-25T10:02:08.402979+0000","last_became_peered":"2024-04-25T10:02:08.402979+0000","last_unstale":"2024-04-25T10:02:09.512930+0000","last_undegraded":"2024-04-25T10:02:09.512930+0000","last_fullsized":"2024-04-25T10:02:09.512930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515530+0000","last_change":"2024-04-25T10:02:04.150503+0000","last_active":"2024-04-25T10:02:09.515530+0000","last_peered":"2024-04-25T10:02:09.515530+0000","last_clean":"2024-04-25T10:02:09.515530+0000","last_became_active":"2024-04-25T10:02:04.150202+0000","last_became_peered":"2024-04-25T10:02:04.150202+0000","last_unstale":"2024-04-25T10:02:09.515530+0000","last_undegraded":"2024-04-25T10:02:09.515530+0000","last_fullsized":"2024-04-25T10:02:09.515530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188973+0000","last_change":"2024-04-25T10:02:03.132517+0000","last_active":"2024-04-25T10:02:09.188973+0000","last_peered":"2024-04-25T10:02:09.188973+0000","last_clean":"2024-04-25T10:02:09.188973+0000","last_became_active":"2024-04-25T10:02:03.132128+0000","last_became_peered":"2024-04-25T10:02:03.132128+0000","last_unstale":"2024-04-25T10:02:09.188973+0000","last_undegraded":"2024-04-25T10:02:09.188973+0000","last_fullsized":"2024-04-25T10:02:09.188973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184870+0000","last_change":"2024-04-25T10:02:08.197632+0000","last_active":"2024-04-25T10:02:09.184870+0000","last_peered":"2024-04-25T10:02:09.184870+0000","last_clean":"2024-04-25T10:02:09.184870+0000","last_became_active":"2024-04-25T10:02:08.196287+0000","last_became_peered":"2024-04-25T10:02:08.196287+0000","last_unstale":"2024-04-25T10:02:09.184870+0000","last_undegraded":"2024-04-25T10:02:09.184870+0000","last_fullsized":"2024-04-25T10:02:09.184870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.959507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513645+0000","last_change":"2024-04-25T10:02:04.148290+0000","last_active":"2024-04-25T10:02:09.513645+0000","last_peered":"2024-04-25T10:02:09.513645+0000","last_clean":"2024-04-25T10:02:09.513645+0000","last_became_active":"2024-04-25T10:02:04.147943+0000","last_became_peered":"2024-04-25T10:02:04.147943+0000","last_unstale":"2024-04-25T10:02:09.513645+0000","last_undegraded":"2024-04-25T10:02:09.513645+0000","last_fullsized":"2024-04-25T10:02:09.513645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181446+0000","last_change":"2024-04-25T10:02:03.125338+0000","last_active":"2024-04-25T10:02:09.181446+0000","last_peered":"2024-04-25T10:02:09.181446+0000","last_clean":"2024-04-25T10:02:09.181446+0000","last_became_active":"2024-04-25T10:02:03.123509+0000","last_became_peered":"2024-04-25T10:02:03.123509+0000","last_unstale":"2024-04-25T10:02:09.181446+0000","last_undegraded":"2024-04-25T10:02:09.181446+0000","last_fullsized":"2024-04-25T10:02:09.181446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514606+0000","last_change":"2024-04-25T10:02:08.181642+0000","last_active":"2024-04-25T10:02:09.514606+0000","last_peered":"2024-04-25T10:02:09.514606+0000","last_clean":"2024-04-25T10:02:09.514606+0000","last_became_active":"2024-04-25T10:02:08.181231+0000","last_became_peered":"2024-04-25T10:02:08.181231+0000","last_unstale":"2024-04-25T10:02:09.514606+0000","last_undegraded":"2024-04-25T10:02:09.514606+0000","last_fullsized":"2024-04-25T10:02:09.514606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182787+0000","last_change":"2024-04-25T10:02:04.125758+0000","last_active":"2024-04-25T10:02:09.182787+0000","last_peered":"2024-04-25T10:02:09.182787+0000","last_clean":"2024-04-25T10:02:09.182787+0000","last_became_active":"2024-04-25T10:02:04.124948+0000","last_became_peered":"2024-04-25T10:02:04.124948+0000","last_unstale":"2024-04-25T10:02:09.182787+0000","last_undegraded":"2024-04-25T10:02:09.182787+0000","last_fullsized":"2024-04-25T10:02:09.182787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184171+0000","last_change":"2024-04-25T10:02:03.136435+0000","last_active":"2024-04-25T10:02:09.184171+0000","last_peered":"2024-04-25T10:02:09.184171+0000","last_clean":"2024-04-25T10:02:09.184171+0000","last_became_active":"2024-04-25T10:02:03.134440+0000","last_became_peered":"2024-04-25T10:02:03.134440+0000","last_unstale":"2024-04-25T10:02:09.184171+0000","last_undegraded":"2024-04-25T10:02:09.184171+0000","last_fullsized":"2024-04-25T10:02:09.184171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520993+0000","last_change":"2024-04-25T10:02:08.393430+0000","last_active":"2024-04-25T10:02:09.520993+0000","last_peered":"2024-04-25T10:02:09.520993+0000","last_clean":"2024-04-25T10:02:09.520993+0000","last_became_active":"2024-04-25T10:02:08.392682+0000","last_became_peered":"2024-04-25T10:02:08.392682+0000","last_unstale":"2024-04-25T10:02:09.520993+0000","last_undegraded":"2024-04-25T10:02:09.520993+0000","last_fullsized":"2024-04-25T10:02:09.520993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.626796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514273+0000","last_change":"2024-04-25T10:02:04.145690+0000","last_active":"2024-04-25T10:02:09.514273+0000","last_peered":"2024-04-25T10:02:09.514273+0000","last_clean":"2024-04-25T10:02:09.514273+0000","last_became_active":"2024-04-25T10:02:04.145414+0000","last_became_peered":"2024-04-25T10:02:04.145414+0000","last_unstale":"2024-04-25T10:02:09.514273+0000","last_undegraded":"2024-04-25T10:02:09.514273+0000","last_fullsized":"2024-04-25T10:02:09.514273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514810+0000","last_change":"2024-04-25T10:02:03.151971+0000","last_active":"2024-04-25T10:02:09.514810+0000","last_peered":"2024-04-25T10:02:09.514810+0000","last_clean":"2024-04-25T10:02:09.514810+0000","last_became_active":"2024-04-25T10:02:03.151708+0000","last_became_peered":"2024-04-25T10:02:03.151708+0000","last_unstale":"2024-04-25T10:02:09.514810+0000","last_undegraded":"2024-04-25T10:02:09.514810+0000","last_fullsized":"2024-04-25T10:02:09.514810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520552+0000","last_change":"2024-04-25T10:02:08.393020+0000","last_active":"2024-04-25T10:02:09.520552+0000","last_peered":"2024-04-25T10:02:09.520552+0000","last_clean":"2024-04-25T10:02:09.520552+0000","last_became_active":"2024-04-25T10:02:08.392543+0000","last_became_peered":"2024-04-25T10:02:08.392543+0000","last_unstale":"2024-04-25T10:02:09.520552+0000","last_undegraded":"2024-04-25T10:02:09.520552+0000","last_fullsized":"2024-04-25T10:02:09.520552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:32.205527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512848+0000","last_change":"2024-04-25T10:02:04.137453+0000","last_active":"2024-04-25T10:02:09.512848+0000","last_peered":"2024-04-25T10:02:09.512848+0000","last_clean":"2024-04-25T10:02:09.512848+0000","last_became_active":"2024-04-25T10:02:04.136751+0000","last_became_peered":"2024-04-25T10:02:04.136751+0000","last_unstale":"2024-04-25T10:02:09.512848+0000","last_undegraded":"2024-04-25T10:02:09.512848+0000","last_fullsized":"2024-04-25T10:02:09.512848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514252+0000","last_change":"2024-04-25T10:02:03.134138+0000","last_active":"2024-04-25T10:02:09.514252+0000","last_peered":"2024-04-25T10:02:09.514252+0000","last_clean":"2024-04-25T10:02:09.514252+0000","last_became_active":"2024-04-25T10:02:03.133331+0000","last_became_peered":"2024-04-25T10:02:03.133331+0000","last_unstale":"2024-04-25T10:02:09.514252+0000","last_undegraded":"2024-04-25T10:02:09.514252+0000","last_fullsized":"2024-04-25T10:02:09.514252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516833+0000","last_change":"2024-04-25T10:02:08.399232+0000","last_active":"2024-04-25T10:02:09.516833+0000","last_peered":"2024-04-25T10:02:09.516833+0000","last_clean":"2024-04-25T10:02:09.516833+0000","last_became_active":"2024-04-25T10:02:08.393743+0000","last_became_peered":"2024-04-25T10:02:08.393743+0000","last_unstale":"2024-04-25T10:02:09.516833+0000","last_undegraded":"2024-04-25T10:02:09.516833+0000","last_fullsized":"2024-04-25T10:02:09.516833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514339+0000","last_change":"2024-04-25T10:02:04.145573+0000","last_active":"2024-04-25T10:02:09.514339+0000","last_peered":"2024-04-25T10:02:09.514339+0000","last_clean":"2024-04-25T10:02:09.514339+0000","last_became_active":"2024-04-25T10:02:04.140591+0000","last_became_peered":"2024-04-25T10:02:04.140591+0000","last_unstale":"2024-04-25T10:02:09.514339+0000","last_undegraded":"2024-04-25T10:02:09.514339+0000","last_fullsized":"2024-04-25T10:02:09.514339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:21.801655+0000","last_change":"2024-04-25T10:02:03.131213+0000","last_active":"2024-04-25T10:02:21.801655+0000","last_peered":"2024-04-25T10:02:21.801655+0000","last_clean":"2024-04-25T10:02:21.801655+0000","last_became_active":"2024-04-25T10:02:03.130876+0000","last_became_peered":"2024-04-25T10:02:03.130876+0000","last_unstale":"2024-04-25T10:02:21.801655+0000","last_undegraded":"2024-04-25T10:02:21.801655+0000","last_fullsized":"2024-04-25T10:02:21.801655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518147+0000","last_change":"2024-04-25T10:02:08.197739+0000","last_active":"2024-04-25T10:02:09.518147+0000","last_peered":"2024-04-25T10:02:09.518147+0000","last_clean":"2024-04-25T10:02:09.518147+0000","last_became_active":"2024-04-25T10:02:08.195924+0000","last_became_peered":"2024-04-25T10:02:08.195924+0000","last_unstale":"2024-04-25T10:02:09.518147+0000","last_undegraded":"2024-04-25T10:02:09.518147+0000","last_fullsized":"2024-04-25T10:02:09.518147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188422+0000","last_change":"2024-04-25T10:02:04.147813+0000","last_active":"2024-04-25T10:02:09.188422+0000","last_peered":"2024-04-25T10:02:09.188422+0000","last_clean":"2024-04-25T10:02:09.188422+0000","last_became_active":"2024-04-25T10:02:04.140229+0000","last_became_peered":"2024-04-25T10:02:04.140229+0000","last_unstale":"2024-04-25T10:02:09.188422+0000","last_undegraded":"2024-04-25T10:02:09.188422+0000","last_fullsized":"2024-04-25T10:02:09.188422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:19.761376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515518+0000","last_change":"2024-04-25T10:02:03.154687+0000","last_active":"2024-04-25T10:02:09.515518+0000","last_peered":"2024-04-25T10:02:09.515518+0000","last_clean":"2024-04-25T10:02:09.515518+0000","last_became_active":"2024-04-25T10:02:03.154323+0000","last_became_peered":"2024-04-25T10:02:03.154323+0000","last_unstale":"2024-04-25T10:02:09.515518+0000","last_undegraded":"2024-04-25T10:02:09.515518+0000","last_fullsized":"2024-04-25T10:02:09.515518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187851+0000","last_change":"2024-04-25T10:02:08.193520+0000","last_active":"2024-04-25T10:02:09.187851+0000","last_peered":"2024-04-25T10:02:09.187851+0000","last_clean":"2024-04-25T10:02:09.187851+0000","last_became_active":"2024-04-25T10:02:08.193040+0000","last_became_peered":"2024-04-25T10:02:08.193040+0000","last_unstale":"2024-04-25T10:02:09.187851+0000","last_undegraded":"2024-04-25T10:02:09.187851+0000","last_fullsized":"2024-04-25T10:02:09.187851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513593+0000","last_change":"2024-04-25T10:02:04.137131+0000","last_active":"2024-04-25T10:02:09.513593+0000","last_peered":"2024-04-25T10:02:09.513593+0000","last_clean":"2024-04-25T10:02:09.513593+0000","last_became_active":"2024-04-25T10:02:04.136832+0000","last_became_peered":"2024-04-25T10:02:04.136832+0000","last_unstale":"2024-04-25T10:02:09.513593+0000","last_undegraded":"2024-04-25T10:02:09.513593+0000","last_fullsized":"2024-04-25T10:02:09.513593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516824+0000","last_change":"2024-04-25T10:02:03.154418+0000","last_active":"2024-04-25T10:02:09.516824+0000","last_peered":"2024-04-25T10:02:09.516824+0000","last_clean":"2024-04-25T10:02:09.516824+0000","last_became_active":"2024-04-25T10:02:03.154147+0000","last_became_peered":"2024-04-25T10:02:03.154147+0000","last_unstale":"2024-04-25T10:02:09.516824+0000","last_undegraded":"2024-04-25T10:02:09.516824+0000","last_fullsized":"2024-04-25T10:02:09.516824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519420+0000","last_change":"2024-04-25T10:02:08.192052+0000","last_active":"2024-04-25T10:02:09.519420+0000","last_peered":"2024-04-25T10:02:09.519420+0000","last_clean":"2024-04-25T10:02:09.519420+0000","last_became_active":"2024-04-25T10:02:08.190628+0000","last_became_peered":"2024-04-25T10:02:08.190628+0000","last_unstale":"2024-04-25T10:02:09.519420+0000","last_undegraded":"2024-04-25T10:02:09.519420+0000","last_fullsized":"2024-04-25T10:02:09.519420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516671+0000","last_change":"2024-04-25T10:02:04.149114+0000","last_active":"2024-04-25T10:02:09.516671+0000","last_peered":"2024-04-25T10:02:09.516671+0000","last_clean":"2024-04-25T10:02:09.516671+0000","last_became_active":"2024-04-25T10:02:04.147309+0000","last_became_peered":"2024-04-25T10:02:04.147309+0000","last_unstale":"2024-04-25T10:02:09.516671+0000","last_undegraded":"2024-04-25T10:02:09.516671+0000","last_fullsized":"2024-04-25T10:02:09.516671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187650+0000","last_change":"2024-04-25T10:02:03.150915+0000","last_active":"2024-04-25T10:02:09.187650+0000","last_peered":"2024-04-25T10:02:09.187650+0000","last_clean":"2024-04-25T10:02:09.187650+0000","last_became_active":"2024-04-25T10:02:03.150382+0000","last_became_peered":"2024-04-25T10:02:03.150382+0000","last_unstale":"2024-04-25T10:02:09.187650+0000","last_undegraded":"2024-04-25T10:02:09.187650+0000","last_fullsized":"2024-04-25T10:02:09.187650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520143+0000","last_change":"2024-04-25T10:02:08.409717+0000","last_active":"2024-04-25T10:02:09.520143+0000","last_peered":"2024-04-25T10:02:09.520143+0000","last_clean":"2024-04-25T10:02:09.520143+0000","last_became_active":"2024-04-25T10:02:08.409153+0000","last_became_peered":"2024-04-25T10:02:08.409153+0000","last_unstale":"2024-04-25T10:02:09.520143+0000","last_undegraded":"2024-04-25T10:02:09.520143+0000","last_fullsized":"2024-04-25T10:02:09.520143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:59.321979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514307+0000","last_change":"2024-04-25T10:02:08.402841+0000","last_active":"2024-04-25T10:02:09.514307+0000","last_peered":"2024-04-25T10:02:09.514307+0000","last_clean":"2024-04-25T10:02:09.514307+0000","last_became_active":"2024-04-25T10:02:08.402205+0000","last_became_peered":"2024-04-25T10:02:08.402205+0000","last_unstale":"2024-04-25T10:02:09.514307+0000","last_undegraded":"2024-04-25T10:02:09.514307+0000","last_fullsized":"2024-04-25T10:02:09.514307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514163+0000","last_change":"2024-04-25T10:02:04.136408+0000","last_active":"2024-04-25T10:02:09.514163+0000","last_peered":"2024-04-25T10:02:09.514163+0000","last_clean":"2024-04-25T10:02:09.514163+0000","last_became_active":"2024-04-25T10:02:04.136094+0000","last_became_peered":"2024-04-25T10:02:04.136094+0000","last_unstale":"2024-04-25T10:02:09.514163+0000","last_undegraded":"2024-04-25T10:02:09.514163+0000","last_fullsized":"2024-04-25T10:02:09.514163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513538+0000","last_change":"2024-04-25T10:02:03.154541+0000","last_active":"2024-04-25T10:02:09.513538+0000","last_peered":"2024-04-25T10:02:09.513538+0000","last_clean":"2024-04-25T10:02:09.513538+0000","last_became_active":"2024-04-25T10:02:03.154193+0000","last_became_peered":"2024-04-25T10:02:03.154193+0000","last_unstale":"2024-04-25T10:02:09.513538+0000","last_undegraded":"2024-04-25T10:02:09.513538+0000","last_fullsized":"2024-04-25T10:02:09.513538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184564+0000","last_change":"2024-04-25T10:02:08.192506+0000","last_active":"2024-04-25T10:02:09.184564+0000","last_peered":"2024-04-25T10:02:09.184564+0000","last_clean":"2024-04-25T10:02:09.184564+0000","last_became_active":"2024-04-25T10:02:08.190910+0000","last_became_peered":"2024-04-25T10:02:08.190910+0000","last_unstale":"2024-04-25T10:02:09.184564+0000","last_undegraded":"2024-04-25T10:02:09.184564+0000","last_fullsized":"2024-04-25T10:02:09.184564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519465+0000","last_change":"2024-04-25T10:02:04.138218+0000","last_active":"2024-04-25T10:02:09.519465+0000","last_peered":"2024-04-25T10:02:09.519465+0000","last_clean":"2024-04-25T10:02:09.519465+0000","last_became_active":"2024-04-25T10:02:04.137919+0000","last_became_peered":"2024-04-25T10:02:04.137919+0000","last_unstale":"2024-04-25T10:02:09.519465+0000","last_undegraded":"2024-04-25T10:02:09.519465+0000","last_fullsized":"2024-04-25T10:02:09.519465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520192+0000","last_change":"2024-04-25T10:02:03.133971+0000","last_active":"2024-04-25T10:02:09.520192+0000","last_peered":"2024-04-25T10:02:09.520192+0000","last_clean":"2024-04-25T10:02:09.520192+0000","last_became_active":"2024-04-25T10:02:03.133661+0000","last_became_peered":"2024-04-25T10:02:03.133661+0000","last_unstale":"2024-04-25T10:02:09.520192+0000","last_undegraded":"2024-04-25T10:02:09.520192+0000","last_fullsized":"2024-04-25T10:02:09.520192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183101+0000","last_change":"2024-04-25T10:02:03.137884+0000","last_active":"2024-04-25T10:02:09.183101+0000","last_peered":"2024-04-25T10:02:09.183101+0000","last_clean":"2024-04-25T10:02:09.183101+0000","last_became_active":"2024-04-25T10:02:03.137196+0000","last_became_peered":"2024-04-25T10:02:03.137196+0000","last_unstale":"2024-04-25T10:02:09.183101+0000","last_undegraded":"2024-04-25T10:02:09.183101+0000","last_fullsized":"2024-04-25T10:02:09.183101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514125+0000","last_change":"2024-04-25T10:02:04.130491+0000","last_active":"2024-04-25T10:02:09.514125+0000","last_peered":"2024-04-25T10:02:09.514125+0000","last_clean":"2024-04-25T10:02:09.514125+0000","last_became_active":"2024-04-25T10:02:04.129963+0000","last_became_peered":"2024-04-25T10:02:04.129963+0000","last_unstale":"2024-04-25T10:02:09.514125+0000","last_undegraded":"2024-04-25T10:02:09.514125+0000","last_fullsized":"2024-04-25T10:02:09.514125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182210+0000","last_change":"2024-04-25T10:02:08.186410+0000","last_active":"2024-04-25T10:02:09.182210+0000","last_peered":"2024-04-25T10:02:09.182210+0000","last_clean":"2024-04-25T10:02:09.182210+0000","last_became_active":"2024-04-25T10:02:08.185964+0000","last_became_peered":"2024-04-25T10:02:08.185964+0000","last_unstale":"2024-04-25T10:02:09.182210+0000","last_undegraded":"2024-04-25T10:02:09.182210+0000","last_fullsized":"2024-04-25T10:02:09.182210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:17.471287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520398+0000","last_change":"2024-04-25T10:02:03.134295+0000","last_active":"2024-04-25T10:02:09.520398+0000","last_peered":"2024-04-25T10:02:09.520398+0000","last_clean":"2024-04-25T10:02:09.520398+0000","last_became_active":"2024-04-25T10:02:03.134033+0000","last_became_peered":"2024-04-25T10:02:03.134033+0000","last_unstale":"2024-04-25T10:02:09.520398+0000","last_undegraded":"2024-04-25T10:02:09.520398+0000","last_fullsized":"2024-04-25T10:02:09.520398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.186180+0000","last_change":"2024-04-25T10:02:04.137055+0000","last_active":"2024-04-25T10:02:09.186180+0000","last_peered":"2024-04-25T10:02:09.186180+0000","last_clean":"2024-04-25T10:02:09.186180+0000","last_became_active":"2024-04-25T10:02:04.131629+0000","last_became_peered":"2024-04-25T10:02:04.131629+0000","last_unstale":"2024-04-25T10:02:09.186180+0000","last_undegraded":"2024-04-25T10:02:09.186180+0000","last_fullsized":"2024-04-25T10:02:09.186180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521270+0000","last_change":"2024-04-25T10:02:08.187719+0000","last_active":"2024-04-25T10:02:09.521270+0000","last_peered":"2024-04-25T10:02:09.521270+0000","last_clean":"2024-04-25T10:02:09.521270+0000","last_became_active":"2024-04-25T10:02:08.187298+0000","last_became_peered":"2024-04-25T10:02:08.187298+0000","last_unstale":"2024-04-25T10:02:09.521270+0000","last_undegraded":"2024-04-25T10:02:09.521270+0000","last_fullsized":"2024-04-25T10:02:09.521270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:15.995769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182212+0000","last_change":"2024-04-25T10:02:04.125579+0000","last_active":"2024-04-25T10:02:09.182212+0000","last_peered":"2024-04-25T10:02:09.182212+0000","last_clean":"2024-04-25T10:02:09.182212+0000","last_became_active":"2024-04-25T10:02:04.124972+0000","last_became_peered":"2024-04-25T10:02:04.124972+0000","last_unstale":"2024-04-25T10:02:09.182212+0000","last_undegraded":"2024-04-25T10:02:09.182212+0000","last_fullsized":"2024-04-25T10:02:09.182212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181677+0000","last_change":"2024-04-25T10:02:04.145556+0000","last_active":"2024-04-25T10:02:09.181677+0000","last_peered":"2024-04-25T10:02:09.181677+0000","last_clean":"2024-04-25T10:02:09.181677+0000","last_became_active":"2024-04-25T10:02:04.144887+0000","last_became_peered":"2024-04-25T10:02:04.144887+0000","last_unstale":"2024-04-25T10:02:09.181677+0000","last_undegraded":"2024-04-25T10:02:09.181677+0000","last_fullsized":"2024-04-25T10:02:09.181677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514803+0000","last_change":"2024-04-25T10:02:04.142127+0000","last_active":"2024-04-25T10:02:09.514803+0000","last_peered":"2024-04-25T10:02:09.514803+0000","last_clean":"2024-04-25T10:02:09.514803+0000","last_became_active":"2024-04-25T10:02:04.140799+0000","last_became_peered":"2024-04-25T10:02:04.140799+0000","last_unstale":"2024-04-25T10:02:09.514803+0000","last_undegraded":"2024-04-25T10:02:09.514803+0000","last_fullsized":"2024-04-25T10:02:09.514803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514188+0000","last_change":"2024-04-25T10:02:04.141010+0000","last_active":"2024-04-25T10:02:09.514188+0000","last_peered":"2024-04-25T10:02:09.514188+0000","last_clean":"2024-04-25T10:02:09.514188+0000","last_became_active":"2024-04-25T10:02:04.140201+0000","last_became_peered":"2024-04-25T10:02:04.140201+0000","last_unstale":"2024-04-25T10:02:09.514188+0000","last_undegraded":"2024-04-25T10:02:09.514188+0000","last_fullsized":"2024-04-25T10:02:09.514188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521086+0000","last_change":"2024-04-25T10:02:04.137613+0000","last_active":"2024-04-25T10:02:09.521086+0000","last_peered":"2024-04-25T10:02:09.521086+0000","last_clean":"2024-04-25T10:02:09.521086+0000","last_became_active":"2024-04-25T10:02:04.136833+0000","last_became_peered":"2024-04-25T10:02:04.136833+0000","last_unstale":"2024-04-25T10:02:09.521086+0000","last_undegraded":"2024-04-25T10:02:09.521086+0000","last_fullsized":"2024-04-25T10:02:09.521086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514990+0000","last_change":"2024-04-25T10:02:04.135708+0000","last_active":"2024-04-25T10:02:09.514990+0000","last_peered":"2024-04-25T10:02:09.514990+0000","last_clean":"2024-04-25T10:02:09.514990+0000","last_became_active":"2024-04-25T10:02:04.135383+0000","last_became_peered":"2024-04-25T10:02:04.135383+0000","last_unstale":"2024-04-25T10:02:09.514990+0000","last_undegraded":"2024-04-25T10:02:09.514990+0000","last_fullsized":"2024-04-25T10:02:09.514990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187243+0000","last_change":"2024-04-25T10:02:04.151491+0000","last_active":"2024-04-25T10:02:09.187243+0000","last_peered":"2024-04-25T10:02:09.187243+0000","last_clean":"2024-04-25T10:02:09.187243+0000","last_became_active":"2024-04-25T10:02:04.151181+0000","last_became_peered":"2024-04-25T10:02:04.151181+0000","last_unstale":"2024-04-25T10:02:09.187243+0000","last_undegraded":"2024-04-25T10:02:09.187243+0000","last_fullsized":"2024-04-25T10:02:09.187243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521012+0000","last_change":"2024-04-25T10:02:04.133889+0000","last_active":"2024-04-25T10:02:09.521012+0000","last_peered":"2024-04-25T10:02:09.521012+0000","last_clean":"2024-04-25T10:02:09.521012+0000","last_became_active":"2024-04-25T10:02:04.132937+0000","last_became_peered":"2024-04-25T10:02:04.132937+0000","last_unstale":"2024-04-25T10:02:09.521012+0000","last_undegraded":"2024-04-25T10:02:09.521012+0000","last_fullsized":"2024-04-25T10:02:09.521012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513229+0000","last_change":"2024-04-25T10:02:04.142217+0000","last_active":"2024-04-25T10:02:09.513229+0000","last_peered":"2024-04-25T10:02:09.513229+0000","last_clean":"2024-04-25T10:02:09.513229+0000","last_became_active":"2024-04-25T10:02:04.141431+0000","last_became_peered":"2024-04-25T10:02:04.141431+0000","last_unstale":"2024-04-25T10:02:09.513229+0000","last_undegraded":"2024-04-25T10:02:09.513229+0000","last_fullsized":"2024-04-25T10:02:09.513229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520025+0000","last_change":"2024-04-25T10:02:04.148470+0000","last_active":"2024-04-25T10:02:09.520025+0000","last_peered":"2024-04-25T10:02:09.520025+0000","last_clean":"2024-04-25T10:02:09.520025+0000","last_became_active":"2024-04-25T10:02:04.147993+0000","last_became_peered":"2024-04-25T10:02:04.147993+0000","last_unstale":"2024-04-25T10:02:09.520025+0000","last_undegraded":"2024-04-25T10:02:09.520025+0000","last_fullsized":"2024-04-25T10:02:09.520025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513946+0000","last_change":"2024-04-25T10:02:04.148427+0000","last_active":"2024-04-25T10:02:09.513946+0000","last_peered":"2024-04-25T10:02:09.513946+0000","last_clean":"2024-04-25T10:02:09.513946+0000","last_became_active":"2024-04-25T10:02:04.147301+0000","last_became_peered":"2024-04-25T10:02:04.147301+0000","last_unstale":"2024-04-25T10:02:09.513946+0000","last_undegraded":"2024-04-25T10:02:09.513946+0000","last_fullsized":"2024-04-25T10:02:09.513946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513726+0000","last_change":"2024-04-25T10:02:04.136324+0000","last_active":"2024-04-25T10:02:09.513726+0000","last_peered":"2024-04-25T10:02:09.513726+0000","last_clean":"2024-04-25T10:02:09.513726+0000","last_became_active":"2024-04-25T10:02:04.136035+0000","last_became_peered":"2024-04-25T10:02:04.136035+0000","last_unstale":"2024-04-25T10:02:09.513726+0000","last_undegraded":"2024-04-25T10:02:09.513726+0000","last_fullsized":"2024-04-25T10:02:09.513726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184020+0000","last_change":"2024-04-25T10:02:04.134441+0000","last_active":"2024-04-25T10:02:09.184020+0000","last_peered":"2024-04-25T10:02:09.184020+0000","last_clean":"2024-04-25T10:02:09.184020+0000","last_became_active":"2024-04-25T10:02:04.130052+0000","last_became_peered":"2024-04-25T10:02:04.130052+0000","last_unstale":"2024-04-25T10:02:09.184020+0000","last_undegraded":"2024-04-25T10:02:09.184020+0000","last_fullsized":"2024-04-25T10:02:09.184020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515111+0000","last_change":"2024-04-25T10:02:04.146376+0000","last_active":"2024-04-25T10:02:09.515111+0000","last_peered":"2024-04-25T10:02:09.515111+0000","last_clean":"2024-04-25T10:02:09.515111+0000","last_became_active":"2024-04-25T10:02:04.146080+0000","last_became_peered":"2024-04-25T10:02:04.146080+0000","last_unstale":"2024-04-25T10:02:09.515111+0000","last_undegraded":"2024-04-25T10:02:09.515111+0000","last_fullsized":"2024-04-25T10:02:09.515111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182337+0000","last_change":"2024-04-25T10:02:04.135427+0000","last_active":"2024-04-25T10:02:09.182337+0000","last_peered":"2024-04-25T10:02:09.182337+0000","last_clean":"2024-04-25T10:02:09.182337+0000","last_became_active":"2024-04-25T10:02:04.134990+0000","last_became_peered":"2024-04-25T10:02:04.134990+0000","last_unstale":"2024-04-25T10:02:09.182337+0000","last_undegraded":"2024-04-25T10:02:09.182337+0000","last_fullsized":"2024-04-25T10:02:09.182337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517176+0000","last_change":"2024-04-25T10:02:04.140449+0000","last_active":"2024-04-25T10:02:09.517176+0000","last_peered":"2024-04-25T10:02:09.517176+0000","last_clean":"2024-04-25T10:02:09.517176+0000","last_became_active":"2024-04-25T10:02:04.140129+0000","last_became_peered":"2024-04-25T10:02:04.140129+0000","last_unstale":"2024-04-25T10:02:09.517176+0000","last_undegraded":"2024-04-25T10:02:09.517176+0000","last_fullsized":"2024-04-25T10:02:09.517176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513015+0000","last_change":"2024-04-25T10:02:04.137099+0000","last_active":"2024-04-25T10:02:09.513015+0000","last_peered":"2024-04-25T10:02:09.513015+0000","last_clean":"2024-04-25T10:02:09.513015+0000","last_became_active":"2024-04-25T10:02:04.136845+0000","last_became_peered":"2024-04-25T10:02:04.136845+0000","last_unstale":"2024-04-25T10:02:09.513015+0000","last_undegraded":"2024-04-25T10:02:09.513015+0000","last_fullsized":"2024-04-25T10:02:09.513015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514467+0000","last_change":"2024-04-25T10:02:04.133727+0000","last_active":"2024-04-25T10:02:09.514467+0000","last_peered":"2024-04-25T10:02:09.514467+0000","last_clean":"2024-04-25T10:02:09.514467+0000","last_became_active":"2024-04-25T10:02:04.132995+0000","last_became_peered":"2024-04-25T10:02:04.132995+0000","last_unstale":"2024-04-25T10:02:09.514467+0000","last_undegraded":"2024-04-25T10:02:09.514467+0000","last_fullsized":"2024-04-25T10:02:09.514467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512646+0000","last_change":"2024-04-25T10:02:04.128013+0000","last_active":"2024-04-25T10:02:09.512646+0000","last_peered":"2024-04-25T10:02:09.512646+0000","last_clean":"2024-04-25T10:02:09.512646+0000","last_became_active":"2024-04-25T10:02:04.123162+0000","last_became_peered":"2024-04-25T10:02:04.123162+0000","last_unstale":"2024-04-25T10:02:09.512646+0000","last_undegraded":"2024-04-25T10:02:09.512646+0000","last_fullsized":"2024-04-25T10:02:09.512646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:26.232712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520128+0000","last_change":"2024-04-25T10:02:04.137865+0000","last_active":"2024-04-25T10:02:09.520128+0000","last_peered":"2024-04-25T10:02:09.520128+0000","last_clean":"2024-04-25T10:02:09.520128+0000","last_became_active":"2024-04-25T10:02:04.137413+0000","last_became_peered":"2024-04-25T10:02:04.137413+0000","last_unstale":"2024-04-25T10:02:09.520128+0000","last_undegraded":"2024-04-25T10:02:09.520128+0000","last_fullsized":"2024-04-25T10:02:09.520128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188286+0000","last_change":"2024-04-25T10:02:04.148039+0000","last_active":"2024-04-25T10:02:09.188286+0000","last_peered":"2024-04-25T10:02:09.188286+0000","last_clean":"2024-04-25T10:02:09.188286+0000","last_became_active":"2024-04-25T10:02:04.146104+0000","last_became_peered":"2024-04-25T10:02:04.146104+0000","last_unstale":"2024-04-25T10:02:09.188286+0000","last_undegraded":"2024-04-25T10:02:09.188286+0000","last_fullsized":"2024-04-25T10:02:09.188286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:15.812709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183940+0000","last_change":"2024-04-25T10:02:04.149357+0000","last_active":"2024-04-25T10:02:09.183940+0000","last_peered":"2024-04-25T10:02:09.183940+0000","last_clean":"2024-04-25T10:02:09.183940+0000","last_became_active":"2024-04-25T10:02:04.148994+0000","last_became_peered":"2024-04-25T10:02:04.148994+0000","last_unstale":"2024-04-25T10:02:09.183940+0000","last_undegraded":"2024-04-25T10:02:09.183940+0000","last_fullsized":"2024-04-25T10:02:09.183940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187107+0000","last_change":"2024-04-25T10:02:04.144119+0000","last_active":"2024-04-25T10:02:09.187107+0000","last_peered":"2024-04-25T10:02:09.187107+0000","last_clean":"2024-04-25T10:02:09.187107+0000","last_became_active":"2024-04-25T10:02:04.143818+0000","last_became_peered":"2024-04-25T10:02:04.143818+0000","last_unstale":"2024-04-25T10:02:09.187107+0000","last_undegraded":"2024-04-25T10:02:09.187107+0000","last_fullsized":"2024-04-25T10:02:09.187107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515823+0000","last_change":"2024-04-25T10:02:04.152842+0000","last_active":"2024-04-25T10:02:09.515823+0000","last_peered":"2024-04-25T10:02:09.515823+0000","last_clean":"2024-04-25T10:02:09.515823+0000","last_became_active":"2024-04-25T10:02:04.142957+0000","last_became_peered":"2024-04-25T10:02:04.142957+0000","last_unstale":"2024-04-25T10:02:09.515823+0000","last_undegraded":"2024-04-25T10:02:09.515823+0000","last_fullsized":"2024-04-25T10:02:09.515823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519170+0000","last_change":"2024-04-25T10:02:04.143950+0000","last_active":"2024-04-25T10:02:09.519170+0000","last_peered":"2024-04-25T10:02:09.519170+0000","last_clean":"2024-04-25T10:02:09.519170+0000","last_became_active":"2024-04-25T10:02:04.143615+0000","last_became_peered":"2024-04-25T10:02:04.143615+0000","last_unstale":"2024-04-25T10:02:09.519170+0000","last_undegraded":"2024-04-25T10:02:09.519170+0000","last_fullsized":"2024-04-25T10:02:09.519170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513367+0000","last_change":"2024-04-25T10:02:04.140815+0000","last_active":"2024-04-25T10:02:09.513367+0000","last_peered":"2024-04-25T10:02:09.513367+0000","last_clean":"2024-04-25T10:02:09.513367+0000","last_became_active":"2024-04-25T10:02:04.140458+0000","last_became_peered":"2024-04-25T10:02:04.140458+0000","last_unstale":"2024-04-25T10:02:09.513367+0000","last_undegraded":"2024-04-25T10:02:09.513367+0000","last_fullsized":"2024-04-25T10:02:09.513367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513577+0000","last_change":"2024-04-25T10:02:04.144469+0000","last_active":"2024-04-25T10:02:09.513577+0000","last_peered":"2024-04-25T10:02:09.513577+0000","last_clean":"2024-04-25T10:02:09.513577+0000","last_became_active":"2024-04-25T10:02:04.143382+0000","last_became_peered":"2024-04-25T10:02:04.143382+0000","last_unstale":"2024-04-25T10:02:09.513577+0000","last_undegraded":"2024-04-25T10:02:09.513577+0000","last_fullsized":"2024-04-25T10:02:09.513577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514018+0000","last_change":"2024-04-25T10:02:04.138676+0000","last_active":"2024-04-25T10:02:09.514018+0000","last_peered":"2024-04-25T10:02:09.514018+0000","last_clean":"2024-04-25T10:02:09.514018+0000","last_became_active":"2024-04-25T10:02:04.138397+0000","last_became_peered":"2024-04-25T10:02:04.138397+0000","last_unstale":"2024-04-25T10:02:09.514018+0000","last_undegraded":"2024-04-25T10:02:09.514018+0000","last_fullsized":"2024-04-25T10:02:09.514018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516016+0000","last_change":"2024-04-25T10:02:04.145097+0000","last_active":"2024-04-25T10:02:09.516016+0000","last_peered":"2024-04-25T10:02:09.516016+0000","last_clean":"2024-04-25T10:02:09.516016+0000","last_became_active":"2024-04-25T10:02:04.142128+0000","last_became_peered":"2024-04-25T10:02:04.142128+0000","last_unstale":"2024-04-25T10:02:09.516016+0000","last_undegraded":"2024-04-25T10:02:09.516016+0000","last_fullsized":"2024-04-25T10:02:09.516016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513292+0000","last_change":"2024-04-25T10:02:04.124760+0000","last_active":"2024-04-25T10:02:09.513292+0000","last_peered":"2024-04-25T10:02:09.513292+0000","last_clean":"2024-04-25T10:02:09.513292+0000","last_became_active":"2024-04-25T10:02:04.124423+0000","last_became_peered":"2024-04-25T10:02:04.124423+0000","last_unstale":"2024-04-25T10:02:09.513292+0000","last_undegraded":"2024-04-25T10:02:09.513292+0000","last_fullsized":"2024-04-25T10:02:09.513292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515984+0000","last_change":"2024-04-25T10:02:04.129218+0000","last_active":"2024-04-25T10:02:09.515984+0000","last_peered":"2024-04-25T10:02:09.515984+0000","last_clean":"2024-04-25T10:02:09.515984+0000","last_became_active":"2024-04-25T10:02:04.126046+0000","last_became_peered":"2024-04-25T10:02:04.126046+0000","last_unstale":"2024-04-25T10:02:09.515984+0000","last_undegraded":"2024-04-25T10:02:09.515984+0000","last_fullsized":"2024-04-25T10:02:09.515984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515345+0000","last_change":"2024-04-25T10:02:04.137569+0000","last_active":"2024-04-25T10:02:09.515345+0000","last_peered":"2024-04-25T10:02:09.515345+0000","last_clean":"2024-04-25T10:02:09.515345+0000","last_became_active":"2024-04-25T10:02:04.137205+0000","last_became_peered":"2024-04-25T10:02:04.137205+0000","last_unstale":"2024-04-25T10:02:09.515345+0000","last_undegraded":"2024-04-25T10:02:09.515345+0000","last_fullsized":"2024-04-25T10:02:09.515345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:31.977 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:02:32.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.207+0000 7f0ec6000640 1 Processor -- start 2024-04-25T10:02:32.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.207+0000 7f0ec6000640 1 -- start start 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ec6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ec6000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ec00ce3c0 0x7f0ec006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ec6000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ec0071480 con 0x7f0ec0154ee0 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ec6000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ec0072090 con 0x7f0ec00ce3c0 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ec6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0071230 con 0x7f0ec00cfec0 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ebf400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0ec0154ee0 0x7f0ec00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58180/0 (socket says 172.21.15.165:58180) 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ebf400640 1 -- 172.21.15.165:0/1880172624 learned_addr learned my addr 172.21.15.165:0/1880172624 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.209+0000 7f0ebfe00640 1 --2- 172.21.15.165:0/1880172624 >> v2:172.21.15.165:3300/0 conn(0x7f0ec00ce3c0 0x7f0ec006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebea00640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebe000640 1 -- 172.21.15.165:0/1880172624 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3637142440 0 0) 0x7f0ec0071480 con 0x7f0ec0154ee0 2024-04-25T10:02:32.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebe000640 1 -- 172.21.15.165:0/1880172624 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ea4005730 con 0x7f0ec0154ee0 2024-04-25T10:02:32.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebfe00640 1 -- 172.21.15.165:0/1880172624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 msgr2=0x7f0ec00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebfe00640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebfe00640 1 -- 172.21.15.165:0/1880172624 >> v1:172.21.15.165:6789/0 conn(0x7f0ec0154ee0 legacy=0x7f0ec00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebfe00640 1 -- 172.21.15.165:0/1880172624 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec0071760 con 0x7f0ec00ce3c0 2024-04-25T10:02:32.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.210+0000 7f0ebea00640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.211+0000 7f0ebfe00640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec006d6f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009d90 tx=0x7f0eb802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb63589fe11541d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.211+0000 7f0ebe000640 1 -- 172.21.15.165:0/1880172624 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8007740 con 0x7f0ec00ce3c0 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.211+0000 7f0ebe000640 1 -- 172.21.15.165:0/1880172624 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb80078e0 con 0x7f0ec00ce3c0 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.211+0000 7f0ebe000640 1 -- 172.21.15.165:0/1880172624 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8007cb0 con 0x7f0ec00ce3c0 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 -- 172.21.15.165:0/1880172624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec00ce3c0 msgr2=0x7f0ec006d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec006d6f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009d90 tx=0x7f0eb802a660 comp rx=0 tx=0).stop 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 -- 172.21.15.165:0/1880172624 shutdown_connections 2024-04-25T10:02:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec006d6f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 --2- 172.21.15.165:0/1880172624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 -- 172.21.15.165:0/1880172624 >> 172.21.15.165:0/1880172624 conn(0x7f0ec006abe0 msgr2=0x7f0ec006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 -- 172.21.15.165:0/1880172624 shutdown_connections 2024-04-25T10:02:32.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.212+0000 7f0ec6000640 1 -- 172.21.15.165:0/1880172624 wait complete. 2024-04-25T10:02:32.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.213+0000 7f0ec6000640 1 Processor -- start 2024-04-25T10:02:32.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.214+0000 7f0ec6000640 1 -- start start 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ec6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec00add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ec6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec0159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ec6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 0x7f0ec01f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ec6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec0071d20 con 0x7f0ec0154ee0 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ec6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec00ce800 con 0x7f0ec00ce3c0 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ec6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ec01552c0 con 0x7f0ec00cfec0 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 0x7f0ec01f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 0x7f0ec01f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50650/0 (socket says 172.21.15.165:50650) 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 -- 172.21.15.165:0/3878718685 learned_addr learned my addr 172.21.15.165:0/3878718685 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 -- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 msgr2=0x7f0ec0159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec0159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebf400640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec00add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 -- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec00ce3c0 msgr2=0x7f0ec00add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec00add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.215+0000 7f0ebfe00640 1 -- 172.21.15.165:0/3878718685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ec00b04b0 con 0x7f0ec0154ee0 2024-04-25T10:02:32.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.216+0000 7f0ebfe00640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 0x7f0ec01f3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009eb0 tx=0x7f0eb8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.216+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8040070 con 0x7f0ec0154ee0 2024-04-25T10:02:32.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.216+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eb8031bc0 con 0x7f0ec0154ee0 2024-04-25T10:02:32.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.216+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eb8009450 con 0x7f0ec0154ee0 2024-04-25T10:02:32.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.216+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ec00a9ba0 con 0x7f0ec0154ee0 2024-04-25T10:02:32.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.216+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ec00aa3e0 con 0x7f0ec0154ee0 2024-04-25T10:02:32.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.217+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ec0073990 con 0x7f0ec0154ee0 2024-04-25T10:02:32.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.218+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0eb8039040 con 0x7f0ec0154ee0 2024-04-25T10:02:32.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.219+0000 7f0eb7400640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ea006b4d0 0x7f0ea006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.219+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f0eb80abe20 con 0x7f0ec0154ee0 2024-04-25T10:02:32.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.219+0000 7f0ebf400640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ea006b4d0 0x7f0ea006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.220+0000 7f0ebf400640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ea006b4d0 0x7f0ea006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0000f60 tx=0x7f0eb0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.223+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ec00a9ba0 con 0x7f0ec0154ee0 2024-04-25T10:02:32.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.405+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0ec00b1e30 con 0x7f0ea006b4d0 2024-04-25T10:02:32.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.430+0000 7f0eb7400640 1 -- 172.21.15.165:0/3878718685 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+332537 (secure 0 0 0) 0x7f0ec00b1e30 con 0x7f0ea006b4d0 2024-04-25T10:02:32.437 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:32.442 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ea006b4d0 msgr2=0x7f0ea006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ea006b4d0 0x7f0ea006d990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0eb0000f60 tx=0x7f0eb0009040 comp rx=0 tx=0).stop 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 msgr2=0x7f0ec01f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 0x7f0ec01f3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0eb8009eb0 tx=0x7f0eb8004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 shutdown_connections 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0ea006b4d0 0x7f0ea006d990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ec0154ee0 0x7f0ec01f3750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ec00cfec0 0x7f0ec0159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 --2- 172.21.15.165:0/3878718685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ec00ce3c0 0x7f0ec00add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 >> 172.21.15.165:0/3878718685 conn(0x7f0ec006abe0 msgr2=0x7f0ec006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.439+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 shutdown_connections 2024-04-25T10:02:32.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.440+0000 7f0ec6000640 1 -- 172.21.15.165:0/3878718685 wait complete. 2024-04-25T10:02:32.465 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":37,"stamp":"2024-04-25T10:02:32.132946+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595008,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":208,"num_write_kb":2435,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":160,"ondisk_log_size":160,"up":322,"acting":322,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":322,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219420,"kb_used_data":3208,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755300,"statfs":{"total":773094113280,"available":772869427200,"internally_reserved":0,"allocated":3284992,"data_stored":1992064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.005089"},"pg_stats":[{"pgid":"2.1f","version":"19'4","reported_seq":26,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513776+0000","last_change":"2024-04-25T10:02:03.142252+0000","last_active":"2024-04-25T10:02:09.513776+0000","last_peered":"2024-04-25T10:02:09.513776+0000","last_clean":"2024-04-25T10:02:09.513776+0000","last_became_active":"2024-04-25T10:02:03.141984+0000","last_became_peered":"2024-04-25T10:02:03.141984+0000","last_unstale":"2024-04-25T10:02:09.513776+0000","last_undegraded":"2024-04-25T10:02:09.513776+0000","last_fullsized":"2024-04-25T10:02:09.513776+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520621+0000","last_change":"2024-04-25T10:02:04.156786+0000","last_active":"2024-04-25T10:02:09.520621+0000","last_peered":"2024-04-25T10:02:09.520621+0000","last_clean":"2024-04-25T10:02:09.520621+0000","last_became_active":"2024-04-25T10:02:04.156522+0000","last_became_peered":"2024-04-25T10:02:04.156522+0000","last_unstale":"2024-04-25T10:02:09.520621+0000","last_undegraded":"2024-04-25T10:02:09.520621+0000","last_fullsized":"2024-04-25T10:02:09.520621+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519839+0000","last_change":"2024-04-25T10:02:08.409422+0000","last_active":"2024-04-25T10:02:09.519839+0000","last_peered":"2024-04-25T10:02:09.519839+0000","last_clean":"2024-04-25T10:02:09.519839+0000","last_became_active":"2024-04-25T10:02:08.408455+0000","last_became_peered":"2024-04-25T10:02:08.408455+0000","last_unstale":"2024-04-25T10:02:09.519839+0000","last_undegraded":"2024-04-25T10:02:09.519839+0000","last_fullsized":"2024-04-25T10:02:09.519839+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:32.901672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,7,0],"acting":[3,2,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520971+0000","last_change":"2024-04-25T10:02:03.131043+0000","last_active":"2024-04-25T10:02:09.520971+0000","last_peered":"2024-04-25T10:02:09.520971+0000","last_clean":"2024-04-25T10:02:09.520971+0000","last_became_active":"2024-04-25T10:02:03.130682+0000","last_became_peered":"2024-04-25T10:02:03.130682+0000","last_unstale":"2024-04-25T10:02:09.520971+0000","last_undegraded":"2024-04-25T10:02:09.520971+0000","last_fullsized":"2024-04-25T10:02:09.520971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513286+0000","last_change":"2024-04-25T10:02:04.142528+0000","last_active":"2024-04-25T10:02:09.513286+0000","last_peered":"2024-04-25T10:02:09.513286+0000","last_clean":"2024-04-25T10:02:09.513286+0000","last_became_active":"2024-04-25T10:02:04.142188+0000","last_became_peered":"2024-04-25T10:02:04.142188+0000","last_unstale":"2024-04-25T10:02:09.513286+0000","last_undegraded":"2024-04-25T10:02:09.513286+0000","last_fullsized":"2024-04-25T10:02:09.513286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183112+0000","last_change":"2024-04-25T10:02:08.192140+0000","last_active":"2024-04-25T10:02:09.183112+0000","last_peered":"2024-04-25T10:02:09.183112+0000","last_clean":"2024-04-25T10:02:09.183112+0000","last_became_active":"2024-04-25T10:02:08.191658+0000","last_became_peered":"2024-04-25T10:02:08.191658+0000","last_unstale":"2024-04-25T10:02:09.183112+0000","last_undegraded":"2024-04-25T10:02:09.183112+0000","last_fullsized":"2024-04-25T10:02:09.183112+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:45.834272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,3],"acting":[4,6,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514852+0000","last_change":"2024-04-25T10:02:03.147520+0000","last_active":"2024-04-25T10:02:09.514852+0000","last_peered":"2024-04-25T10:02:09.514852+0000","last_clean":"2024-04-25T10:02:09.514852+0000","last_became_active":"2024-04-25T10:02:03.147274+0000","last_became_peered":"2024-04-25T10:02:03.147274+0000","last_unstale":"2024-04-25T10:02:09.514852+0000","last_undegraded":"2024-04-25T10:02:09.514852+0000","last_fullsized":"2024-04-25T10:02:09.514852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516102+0000","last_change":"2024-04-25T10:02:04.137351+0000","last_active":"2024-04-25T10:02:09.516102+0000","last_peered":"2024-04-25T10:02:09.516102+0000","last_clean":"2024-04-25T10:02:09.516102+0000","last_became_active":"2024-04-25T10:02:04.136927+0000","last_became_peered":"2024-04-25T10:02:04.136927+0000","last_unstale":"2024-04-25T10:02:09.516102+0000","last_undegraded":"2024-04-25T10:02:09.516102+0000","last_fullsized":"2024-04-25T10:02:09.516102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185918+0000","last_change":"2024-04-25T10:02:08.198068+0000","last_active":"2024-04-25T10:02:09.185918+0000","last_peered":"2024-04-25T10:02:09.185918+0000","last_clean":"2024-04-25T10:02:09.185918+0000","last_became_active":"2024-04-25T10:02:08.196209+0000","last_became_peered":"2024-04-25T10:02:08.196209+0000","last_unstale":"2024-04-25T10:02:09.185918+0000","last_undegraded":"2024-04-25T10:02:09.185918+0000","last_fullsized":"2024-04-25T10:02:09.185918+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:48.568415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514318+0000","last_change":"2024-04-25T10:02:03.157531+0000","last_active":"2024-04-25T10:02:09.514318+0000","last_peered":"2024-04-25T10:02:09.514318+0000","last_clean":"2024-04-25T10:02:09.514318+0000","last_became_active":"2024-04-25T10:02:03.157216+0000","last_became_peered":"2024-04-25T10:02:03.157216+0000","last_unstale":"2024-04-25T10:02:09.514318+0000","last_undegraded":"2024-04-25T10:02:09.514318+0000","last_fullsized":"2024-04-25T10:02:09.514318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516612+0000","last_change":"2024-04-25T10:02:04.140082+0000","last_active":"2024-04-25T10:02:09.516612+0000","last_peered":"2024-04-25T10:02:09.516612+0000","last_clean":"2024-04-25T10:02:09.516612+0000","last_became_active":"2024-04-25T10:02:04.136765+0000","last_became_peered":"2024-04-25T10:02:04.136765+0000","last_unstale":"2024-04-25T10:02:09.516612+0000","last_undegraded":"2024-04-25T10:02:09.516612+0000","last_fullsized":"2024-04-25T10:02:09.516612+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185629+0000","last_change":"2024-04-25T10:02:08.189048+0000","last_active":"2024-04-25T10:02:09.185629+0000","last_peered":"2024-04-25T10:02:09.185629+0000","last_clean":"2024-04-25T10:02:09.185629+0000","last_became_active":"2024-04-25T10:02:08.188475+0000","last_became_peered":"2024-04-25T10:02:08.188475+0000","last_unstale":"2024-04-25T10:02:09.185629+0000","last_undegraded":"2024-04-25T10:02:09.185629+0000","last_fullsized":"2024-04-25T10:02:09.185629+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:08.343719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0,5],"acting":[4,3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182619+0000","last_change":"2024-04-25T10:02:04.132020+0000","last_active":"2024-04-25T10:02:09.182619+0000","last_peered":"2024-04-25T10:02:09.182619+0000","last_clean":"2024-04-25T10:02:09.182619+0000","last_became_active":"2024-04-25T10:02:04.131018+0000","last_became_peered":"2024-04-25T10:02:04.131018+0000","last_unstale":"2024-04-25T10:02:09.182619+0000","last_undegraded":"2024-04-25T10:02:09.182619+0000","last_fullsized":"2024-04-25T10:02:09.182619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514507+0000","last_change":"2024-04-25T10:02:03.147595+0000","last_active":"2024-04-25T10:02:09.514507+0000","last_peered":"2024-04-25T10:02:09.514507+0000","last_clean":"2024-04-25T10:02:09.514507+0000","last_became_active":"2024-04-25T10:02:03.147038+0000","last_became_peered":"2024-04-25T10:02:03.147038+0000","last_unstale":"2024-04-25T10:02:09.514507+0000","last_undegraded":"2024-04-25T10:02:09.514507+0000","last_fullsized":"2024-04-25T10:02:09.514507+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:50.278310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188671+0000","last_change":"2024-04-25T10:02:08.186691+0000","last_active":"2024-04-25T10:02:09.188671+0000","last_peered":"2024-04-25T10:02:09.188671+0000","last_clean":"2024-04-25T10:02:09.188671+0000","last_became_active":"2024-04-25T10:02:08.186276+0000","last_became_peered":"2024-04-25T10:02:08.186276+0000","last_unstale":"2024-04-25T10:02:09.188671+0000","last_undegraded":"2024-04-25T10:02:09.188671+0000","last_fullsized":"2024-04-25T10:02:09.188671+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,2,1],"acting":[4,0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520343+0000","last_change":"2024-04-25T10:02:04.139116+0000","last_active":"2024-04-25T10:02:09.520343+0000","last_peered":"2024-04-25T10:02:09.520343+0000","last_clean":"2024-04-25T10:02:09.520343+0000","last_became_active":"2024-04-25T10:02:04.138565+0000","last_became_peered":"2024-04-25T10:02:04.138565+0000","last_unstale":"2024-04-25T10:02:09.520343+0000","last_undegraded":"2024-04-25T10:02:09.520343+0000","last_fullsized":"2024-04-25T10:02:09.520343+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518432+0000","last_change":"2024-04-25T10:02:03.132968+0000","last_active":"2024-04-25T10:02:09.518432+0000","last_peered":"2024-04-25T10:02:09.518432+0000","last_clean":"2024-04-25T10:02:09.518432+0000","last_became_active":"2024-04-25T10:02:03.132624+0000","last_became_peered":"2024-04-25T10:02:03.132624+0000","last_unstale":"2024-04-25T10:02:09.518432+0000","last_undegraded":"2024-04-25T10:02:09.518432+0000","last_fullsized":"2024-04-25T10:02:09.518432+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187996+0000","last_change":"2024-04-25T10:02:08.193709+0000","last_active":"2024-04-25T10:02:09.187996+0000","last_peered":"2024-04-25T10:02:09.187996+0000","last_clean":"2024-04-25T10:02:09.187996+0000","last_became_active":"2024-04-25T10:02:08.193240+0000","last_became_peered":"2024-04-25T10:02:08.193240+0000","last_unstale":"2024-04-25T10:02:09.187996+0000","last_undegraded":"2024-04-25T10:02:09.187996+0000","last_fullsized":"2024-04-25T10:02:09.187996+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519675+0000","last_change":"2024-04-25T10:02:04.139077+0000","last_active":"2024-04-25T10:02:09.519675+0000","last_peered":"2024-04-25T10:02:09.519675+0000","last_clean":"2024-04-25T10:02:09.519675+0000","last_became_active":"2024-04-25T10:02:04.138730+0000","last_became_peered":"2024-04-25T10:02:04.138730+0000","last_unstale":"2024-04-25T10:02:09.519675+0000","last_undegraded":"2024-04-25T10:02:09.519675+0000","last_fullsized":"2024-04-25T10:02:09.519675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514609+0000","last_change":"2024-04-25T10:02:03.137563+0000","last_active":"2024-04-25T10:02:09.514609+0000","last_peered":"2024-04-25T10:02:09.514609+0000","last_clean":"2024-04-25T10:02:09.514609+0000","last_became_active":"2024-04-25T10:02:03.137080+0000","last_became_peered":"2024-04-25T10:02:03.137080+0000","last_unstale":"2024-04-25T10:02:09.514609+0000","last_undegraded":"2024-04-25T10:02:09.514609+0000","last_fullsized":"2024-04-25T10:02:09.514609+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517371+0000","last_change":"2024-04-25T10:02:08.404747+0000","last_active":"2024-04-25T10:02:09.517371+0000","last_peered":"2024-04-25T10:02:09.517371+0000","last_clean":"2024-04-25T10:02:09.517371+0000","last_became_active":"2024-04-25T10:02:08.403706+0000","last_became_peered":"2024-04-25T10:02:08.403706+0000","last_unstale":"2024-04-25T10:02:09.517371+0000","last_undegraded":"2024-04-25T10:02:09.517371+0000","last_fullsized":"2024-04-25T10:02:09.517371+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,7],"acting":[1,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185839+0000","last_change":"2024-04-25T10:02:04.125617+0000","last_active":"2024-04-25T10:02:09.185839+0000","last_peered":"2024-04-25T10:02:09.185839+0000","last_clean":"2024-04-25T10:02:09.185839+0000","last_became_active":"2024-04-25T10:02:04.125319+0000","last_became_peered":"2024-04-25T10:02:04.125319+0000","last_unstale":"2024-04-25T10:02:09.185839+0000","last_undegraded":"2024-04-25T10:02:09.185839+0000","last_fullsized":"2024-04-25T10:02:09.185839+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.271382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512743+0000","last_change":"2024-04-25T10:02:03.145754+0000","last_active":"2024-04-25T10:02:09.512743+0000","last_peered":"2024-04-25T10:02:09.512743+0000","last_clean":"2024-04-25T10:02:09.512743+0000","last_became_active":"2024-04-25T10:02:03.145400+0000","last_became_peered":"2024-04-25T10:02:03.145400+0000","last_unstale":"2024-04-25T10:02:09.512743+0000","last_undegraded":"2024-04-25T10:02:09.512743+0000","last_fullsized":"2024-04-25T10:02:09.512743+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188721+0000","last_change":"2024-04-25T10:02:08.197346+0000","last_active":"2024-04-25T10:02:09.188721+0000","last_peered":"2024-04-25T10:02:09.188721+0000","last_clean":"2024-04-25T10:02:09.188721+0000","last_became_active":"2024-04-25T10:02:08.196902+0000","last_became_peered":"2024-04-25T10:02:08.196902+0000","last_unstale":"2024-04-25T10:02:09.188721+0000","last_undegraded":"2024-04-25T10:02:09.188721+0000","last_fullsized":"2024-04-25T10:02:09.188721+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:43.992278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6,1],"acting":[4,3,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515041+0000","last_change":"2024-04-25T10:02:04.123238+0000","last_active":"2024-04-25T10:02:09.515041+0000","last_peered":"2024-04-25T10:02:09.515041+0000","last_clean":"2024-04-25T10:02:09.515041+0000","last_became_active":"2024-04-25T10:02:04.122886+0000","last_became_peered":"2024-04-25T10:02:04.122886+0000","last_unstale":"2024-04-25T10:02:09.515041+0000","last_undegraded":"2024-04-25T10:02:09.515041+0000","last_fullsized":"2024-04-25T10:02:09.515041+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187935+0000","last_change":"2024-04-25T10:02:03.138021+0000","last_active":"2024-04-25T10:02:09.187935+0000","last_peered":"2024-04-25T10:02:09.187935+0000","last_clean":"2024-04-25T10:02:09.187935+0000","last_became_active":"2024-04-25T10:02:03.137531+0000","last_became_peered":"2024-04-25T10:02:03.137531+0000","last_unstale":"2024-04-25T10:02:09.187935+0000","last_undegraded":"2024-04-25T10:02:09.187935+0000","last_fullsized":"2024-04-25T10:02:09.187935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.185644+0000","last_change":"2024-04-25T10:02:08.404604+0000","last_active":"2024-04-25T10:02:09.185644+0000","last_peered":"2024-04-25T10:02:09.185644+0000","last_clean":"2024-04-25T10:02:09.185644+0000","last_became_active":"2024-04-25T10:02:08.403392+0000","last_became_peered":"2024-04-25T10:02:08.403392+0000","last_unstale":"2024-04-25T10:02:09.185644+0000","last_undegraded":"2024-04-25T10:02:09.185644+0000","last_fullsized":"2024-04-25T10:02:09.185644+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:25.532310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,7],"acting":[4,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521257+0000","last_change":"2024-04-25T10:02:04.144042+0000","last_active":"2024-04-25T10:02:09.521257+0000","last_peered":"2024-04-25T10:02:09.521257+0000","last_clean":"2024-04-25T10:02:09.521257+0000","last_became_active":"2024-04-25T10:02:04.143734+0000","last_became_peered":"2024-04-25T10:02:04.143734+0000","last_unstale":"2024-04-25T10:02:09.521257+0000","last_undegraded":"2024-04-25T10:02:09.521257+0000","last_fullsized":"2024-04-25T10:02:09.521257+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516285+0000","last_change":"2024-04-25T10:02:03.138015+0000","last_active":"2024-04-25T10:02:09.516285+0000","last_peered":"2024-04-25T10:02:09.516285+0000","last_clean":"2024-04-25T10:02:09.516285+0000","last_became_active":"2024-04-25T10:02:03.137487+0000","last_became_peered":"2024-04-25T10:02:03.137487+0000","last_unstale":"2024-04-25T10:02:09.516285+0000","last_undegraded":"2024-04-25T10:02:09.516285+0000","last_fullsized":"2024-04-25T10:02:09.516285+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515852+0000","last_change":"2024-04-25T10:02:08.402309+0000","last_active":"2024-04-25T10:02:09.515852+0000","last_peered":"2024-04-25T10:02:09.515852+0000","last_clean":"2024-04-25T10:02:09.515852+0000","last_became_active":"2024-04-25T10:02:08.401865+0000","last_became_peered":"2024-04-25T10:02:08.401865+0000","last_unstale":"2024-04-25T10:02:09.515852+0000","last_undegraded":"2024-04-25T10:02:09.515852+0000","last_fullsized":"2024-04-25T10:02:09.515852+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0,1],"acting":[3,7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519191+0000","last_change":"2024-04-25T10:02:04.129398+0000","last_active":"2024-04-25T10:02:09.519191+0000","last_peered":"2024-04-25T10:02:09.519191+0000","last_clean":"2024-04-25T10:02:09.519191+0000","last_became_active":"2024-04-25T10:02:04.128879+0000","last_became_peered":"2024-04-25T10:02:04.128879+0000","last_unstale":"2024-04-25T10:02:09.519191+0000","last_undegraded":"2024-04-25T10:02:09.519191+0000","last_fullsized":"2024-04-25T10:02:09.519191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515418+0000","last_change":"2024-04-25T10:02:03.146769+0000","last_active":"2024-04-25T10:02:09.515418+0000","last_peered":"2024-04-25T10:02:09.515418+0000","last_clean":"2024-04-25T10:02:09.515418+0000","last_became_active":"2024-04-25T10:02:03.146576+0000","last_became_peered":"2024-04-25T10:02:03.146576+0000","last_unstale":"2024-04-25T10:02:09.515418+0000","last_undegraded":"2024-04-25T10:02:09.515418+0000","last_fullsized":"2024-04-25T10:02:09.515418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:18.746885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513472+0000","last_change":"2024-04-25T10:02:08.393230+0000","last_active":"2024-04-25T10:02:09.513472+0000","last_peered":"2024-04-25T10:02:09.513472+0000","last_clean":"2024-04-25T10:02:09.513472+0000","last_became_active":"2024-04-25T10:02:08.392691+0000","last_became_peered":"2024-04-25T10:02:08.392691+0000","last_unstale":"2024-04-25T10:02:09.513472+0000","last_undegraded":"2024-04-25T10:02:09.513472+0000","last_fullsized":"2024-04-25T10:02:09.513472+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:07.300642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7,1],"acting":[0,5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516864+0000","last_change":"2024-04-25T10:02:04.140436+0000","last_active":"2024-04-25T10:02:09.516864+0000","last_peered":"2024-04-25T10:02:09.516864+0000","last_clean":"2024-04-25T10:02:09.516864+0000","last_became_active":"2024-04-25T10:02:04.140075+0000","last_became_peered":"2024-04-25T10:02:04.140075+0000","last_unstale":"2024-04-25T10:02:09.516864+0000","last_undegraded":"2024-04-25T10:02:09.516864+0000","last_fullsized":"2024-04-25T10:02:09.516864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519440+0000","last_change":"2024-04-25T10:02:03.132553+0000","last_active":"2024-04-25T10:02:09.519440+0000","last_peered":"2024-04-25T10:02:09.519440+0000","last_clean":"2024-04-25T10:02:09.519440+0000","last_became_active":"2024-04-25T10:02:03.132241+0000","last_became_peered":"2024-04-25T10:02:03.132241+0000","last_unstale":"2024-04-25T10:02:09.519440+0000","last_undegraded":"2024-04-25T10:02:09.519440+0000","last_fullsized":"2024-04-25T10:02:09.519440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518024+0000","last_change":"2024-04-25T10:02:08.186096+0000","last_active":"2024-04-25T10:02:09.518024+0000","last_peered":"2024-04-25T10:02:09.518024+0000","last_clean":"2024-04-25T10:02:09.518024+0000","last_became_active":"2024-04-25T10:02:08.185660+0000","last_became_peered":"2024-04-25T10:02:08.185660+0000","last_unstale":"2024-04-25T10:02:09.518024+0000","last_undegraded":"2024-04-25T10:02:09.518024+0000","last_fullsized":"2024-04-25T10:02:09.518024+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4,0],"acting":[1,5,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188203+0000","last_change":"2024-04-25T10:02:04.126935+0000","last_active":"2024-04-25T10:02:09.188203+0000","last_peered":"2024-04-25T10:02:09.188203+0000","last_clean":"2024-04-25T10:02:09.188203+0000","last_became_active":"2024-04-25T10:02:04.126268+0000","last_became_peered":"2024-04-25T10:02:04.126268+0000","last_unstale":"2024-04-25T10:02:09.188203+0000","last_undegraded":"2024-04-25T10:02:09.188203+0000","last_fullsized":"2024-04-25T10:02:09.188203+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515075+0000","last_change":"2024-04-25T10:02:03.143964+0000","last_active":"2024-04-25T10:02:09.515075+0000","last_peered":"2024-04-25T10:02:09.515075+0000","last_clean":"2024-04-25T10:02:09.515075+0000","last_became_active":"2024-04-25T10:02:03.143548+0000","last_became_peered":"2024-04-25T10:02:03.143548+0000","last_unstale":"2024-04-25T10:02:09.515075+0000","last_undegraded":"2024-04-25T10:02:09.515075+0000","last_fullsized":"2024-04-25T10:02:09.515075+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519212+0000","last_change":"2024-04-25T10:02:08.196544+0000","last_active":"2024-04-25T10:02:09.519212+0000","last_peered":"2024-04-25T10:02:09.519212+0000","last_clean":"2024-04-25T10:02:09.519212+0000","last_became_active":"2024-04-25T10:02:08.196131+0000","last_became_peered":"2024-04-25T10:02:08.196131+0000","last_unstale":"2024-04-25T10:02:09.519212+0000","last_undegraded":"2024-04-25T10:02:09.519212+0000","last_fullsized":"2024-04-25T10:02:09.519212+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3,6],"acting":[1,2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513146+0000","last_change":"2024-04-25T10:02:04.124338+0000","last_active":"2024-04-25T10:02:09.513146+0000","last_peered":"2024-04-25T10:02:09.513146+0000","last_clean":"2024-04-25T10:02:09.513146+0000","last_became_active":"2024-04-25T10:02:04.123720+0000","last_became_peered":"2024-04-25T10:02:04.123720+0000","last_unstale":"2024-04-25T10:02:09.513146+0000","last_undegraded":"2024-04-25T10:02:09.513146+0000","last_fullsized":"2024-04-25T10:02:09.513146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181759+0000","last_change":"2024-04-25T10:02:03.131111+0000","last_active":"2024-04-25T10:02:09.181759+0000","last_peered":"2024-04-25T10:02:09.181759+0000","last_clean":"2024-04-25T10:02:09.181759+0000","last_became_active":"2024-04-25T10:02:03.130828+0000","last_became_peered":"2024-04-25T10:02:03.130828+0000","last_unstale":"2024-04-25T10:02:09.181759+0000","last_undegraded":"2024-04-25T10:02:09.181759+0000","last_fullsized":"2024-04-25T10:02:09.181759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517914+0000","last_change":"2024-04-25T10:02:08.193216+0000","last_active":"2024-04-25T10:02:09.517914+0000","last_peered":"2024-04-25T10:02:09.517914+0000","last_clean":"2024-04-25T10:02:09.517914+0000","last_became_active":"2024-04-25T10:02:08.192717+0000","last_became_peered":"2024-04-25T10:02:08.192717+0000","last_unstale":"2024-04-25T10:02:09.517914+0000","last_undegraded":"2024-04-25T10:02:09.517914+0000","last_fullsized":"2024-04-25T10:02:09.517914+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0,3],"acting":[1,5,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513507+0000","last_change":"2024-04-25T10:02:04.131883+0000","last_active":"2024-04-25T10:02:09.513507+0000","last_peered":"2024-04-25T10:02:09.513507+0000","last_clean":"2024-04-25T10:02:09.513507+0000","last_became_active":"2024-04-25T10:02:04.131591+0000","last_became_peered":"2024-04-25T10:02:04.131591+0000","last_unstale":"2024-04-25T10:02:09.513507+0000","last_undegraded":"2024-04-25T10:02:09.513507+0000","last_fullsized":"2024-04-25T10:02:09.513507+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513031+0000","last_change":"2024-04-25T10:02:03.146201+0000","last_active":"2024-04-25T10:02:09.513031+0000","last_peered":"2024-04-25T10:02:09.513031+0000","last_clean":"2024-04-25T10:02:09.513031+0000","last_became_active":"2024-04-25T10:02:03.145888+0000","last_became_peered":"2024-04-25T10:02:03.145888+0000","last_unstale":"2024-04-25T10:02:09.513031+0000","last_undegraded":"2024-04-25T10:02:09.513031+0000","last_fullsized":"2024-04-25T10:02:09.513031+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514736+0000","last_change":"2024-04-25T10:02:08.403944+0000","last_active":"2024-04-25T10:02:09.514736+0000","last_peered":"2024-04-25T10:02:09.514736+0000","last_clean":"2024-04-25T10:02:09.514736+0000","last_became_active":"2024-04-25T10:02:08.403447+0000","last_became_peered":"2024-04-25T10:02:08.403447+0000","last_unstale":"2024-04-25T10:02:09.514736+0000","last_undegraded":"2024-04-25T10:02:09.514736+0000","last_fullsized":"2024-04-25T10:02:09.514736+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:43.717856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2,7],"acting":[0,1,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520189+0000","last_change":"2024-04-25T10:02:04.145181+0000","last_active":"2024-04-25T10:02:09.520189+0000","last_peered":"2024-04-25T10:02:09.520189+0000","last_clean":"2024-04-25T10:02:09.520189+0000","last_became_active":"2024-04-25T10:02:04.145007+0000","last_became_peered":"2024-04-25T10:02:04.145007+0000","last_unstale":"2024-04-25T10:02:09.520189+0000","last_undegraded":"2024-04-25T10:02:09.520189+0000","last_fullsized":"2024-04-25T10:02:09.520189+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515574+0000","last_change":"2024-04-25T10:02:03.149716+0000","last_active":"2024-04-25T10:02:09.515574+0000","last_peered":"2024-04-25T10:02:09.515574+0000","last_clean":"2024-04-25T10:02:09.515574+0000","last_became_active":"2024-04-25T10:02:03.149427+0000","last_became_peered":"2024-04-25T10:02:03.149427+0000","last_unstale":"2024-04-25T10:02:09.515574+0000","last_undegraded":"2024-04-25T10:02:09.515574+0000","last_fullsized":"2024-04-25T10:02:09.515574+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:34.450282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":178,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513340+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:02:09.513340+0000","last_peered":"2024-04-25T10:02:09.513340+0000","last_clean":"2024-04-25T10:02:09.513340+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:02:09.513340+0000","last_undegraded":"2024-04-25T10:02:09.513340+0000","last_fullsized":"2024-04-25T10:02:09.513340+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183489+0000","last_change":"2024-04-25T10:02:08.405095+0000","last_active":"2024-04-25T10:02:09.183489+0000","last_peered":"2024-04-25T10:02:09.183489+0000","last_clean":"2024-04-25T10:02:09.183489+0000","last_became_active":"2024-04-25T10:02:08.403767+0000","last_became_peered":"2024-04-25T10:02:08.403767+0000","last_unstale":"2024-04-25T10:02:09.183489+0000","last_undegraded":"2024-04-25T10:02:09.183489+0000","last_fullsized":"2024-04-25T10:02:09.183489+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4,7],"acting":[6,3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514594+0000","last_change":"2024-04-25T10:02:04.151648+0000","last_active":"2024-04-25T10:02:09.514594+0000","last_peered":"2024-04-25T10:02:09.514594+0000","last_clean":"2024-04-25T10:02:09.514594+0000","last_became_active":"2024-04-25T10:02:04.150680+0000","last_became_peered":"2024-04-25T10:02:04.150680+0000","last_unstale":"2024-04-25T10:02:09.514594+0000","last_undegraded":"2024-04-25T10:02:09.514594+0000","last_fullsized":"2024-04-25T10:02:09.514594+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:43.902829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182505+0000","last_change":"2024-04-25T10:02:03.154627+0000","last_active":"2024-04-25T10:02:09.182505+0000","last_peered":"2024-04-25T10:02:09.182505+0000","last_clean":"2024-04-25T10:02:09.182505+0000","last_became_active":"2024-04-25T10:02:03.154346+0000","last_became_peered":"2024-04-25T10:02:03.154346+0000","last_unstale":"2024-04-25T10:02:09.182505+0000","last_undegraded":"2024-04-25T10:02:09.182505+0000","last_fullsized":"2024-04-25T10:02:09.182505+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184009+0000","last_change":"2024-04-25T10:02:08.394382+0000","last_active":"2024-04-25T10:02:09.184009+0000","last_peered":"2024-04-25T10:02:09.184009+0000","last_clean":"2024-04-25T10:02:09.184009+0000","last_became_active":"2024-04-25T10:02:08.393222+0000","last_became_peered":"2024-04-25T10:02:08.393222+0000","last_unstale":"2024-04-25T10:02:09.184009+0000","last_undegraded":"2024-04-25T10:02:09.184009+0000","last_fullsized":"2024-04-25T10:02:09.184009+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:50.039943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,1,0],"acting":[6,7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516304+0000","last_change":"2024-04-25T10:02:04.144851+0000","last_active":"2024-04-25T10:02:09.516304+0000","last_peered":"2024-04-25T10:02:09.516304+0000","last_clean":"2024-04-25T10:02:09.516304+0000","last_became_active":"2024-04-25T10:02:04.144530+0000","last_became_peered":"2024-04-25T10:02:04.144530+0000","last_unstale":"2024-04-25T10:02:09.516304+0000","last_undegraded":"2024-04-25T10:02:09.516304+0000","last_fullsized":"2024-04-25T10:02:09.516304+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515591+0000","last_change":"2024-04-25T10:02:03.133061+0000","last_active":"2024-04-25T10:02:09.515591+0000","last_peered":"2024-04-25T10:02:09.515591+0000","last_clean":"2024-04-25T10:02:09.515591+0000","last_became_active":"2024-04-25T10:02:03.132762+0000","last_became_peered":"2024-04-25T10:02:03.132762+0000","last_unstale":"2024-04-25T10:02:09.515591+0000","last_undegraded":"2024-04-25T10:02:09.515591+0000","last_fullsized":"2024-04-25T10:02:09.515591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:14.798218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513927+0000","last_change":"2024-04-25T10:02:08.186615+0000","last_active":"2024-04-25T10:02:09.513927+0000","last_peered":"2024-04-25T10:02:09.513927+0000","last_clean":"2024-04-25T10:02:09.513927+0000","last_became_active":"2024-04-25T10:02:08.186216+0000","last_became_peered":"2024-04-25T10:02:08.186216+0000","last_unstale":"2024-04-25T10:02:09.513927+0000","last_undegraded":"2024-04-25T10:02:09.513927+0000","last_fullsized":"2024-04-25T10:02:09.513927+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:11.340648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,5],"acting":[0,1,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515930+0000","last_change":"2024-04-25T10:02:04.153131+0000","last_active":"2024-04-25T10:02:09.515930+0000","last_peered":"2024-04-25T10:02:09.515930+0000","last_clean":"2024-04-25T10:02:09.515930+0000","last_became_active":"2024-04-25T10:02:04.152415+0000","last_became_peered":"2024-04-25T10:02:04.152415+0000","last_unstale":"2024-04-25T10:02:09.515930+0000","last_undegraded":"2024-04-25T10:02:09.515930+0000","last_fullsized":"2024-04-25T10:02:09.515930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:54.646358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182568+0000","last_change":"2024-04-25T10:02:03.134495+0000","last_active":"2024-04-25T10:02:09.182568+0000","last_peered":"2024-04-25T10:02:09.182568+0000","last_clean":"2024-04-25T10:02:09.182568+0000","last_became_active":"2024-04-25T10:02:03.134157+0000","last_became_peered":"2024-04-25T10:02:03.134157+0000","last_unstale":"2024-04-25T10:02:09.182568+0000","last_undegraded":"2024-04-25T10:02:09.182568+0000","last_fullsized":"2024-04-25T10:02:09.182568+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512930+0000","last_change":"2024-04-25T10:02:08.406412+0000","last_active":"2024-04-25T10:02:09.512930+0000","last_peered":"2024-04-25T10:02:09.512930+0000","last_clean":"2024-04-25T10:02:09.512930+0000","last_became_active":"2024-04-25T10:02:08.402979+0000","last_became_peered":"2024-04-25T10:02:08.402979+0000","last_unstale":"2024-04-25T10:02:09.512930+0000","last_undegraded":"2024-04-25T10:02:09.512930+0000","last_fullsized":"2024-04-25T10:02:09.512930+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:06:17.199032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7,6],"acting":[5,4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515530+0000","last_change":"2024-04-25T10:02:04.150503+0000","last_active":"2024-04-25T10:02:09.515530+0000","last_peered":"2024-04-25T10:02:09.515530+0000","last_clean":"2024-04-25T10:02:09.515530+0000","last_became_active":"2024-04-25T10:02:04.150202+0000","last_became_peered":"2024-04-25T10:02:04.150202+0000","last_unstale":"2024-04-25T10:02:09.515530+0000","last_undegraded":"2024-04-25T10:02:09.515530+0000","last_fullsized":"2024-04-25T10:02:09.515530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188973+0000","last_change":"2024-04-25T10:02:03.132517+0000","last_active":"2024-04-25T10:02:09.188973+0000","last_peered":"2024-04-25T10:02:09.188973+0000","last_clean":"2024-04-25T10:02:09.188973+0000","last_became_active":"2024-04-25T10:02:03.132128+0000","last_became_peered":"2024-04-25T10:02:03.132128+0000","last_unstale":"2024-04-25T10:02:09.188973+0000","last_undegraded":"2024-04-25T10:02:09.188973+0000","last_fullsized":"2024-04-25T10:02:09.188973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184870+0000","last_change":"2024-04-25T10:02:08.197632+0000","last_active":"2024-04-25T10:02:09.184870+0000","last_peered":"2024-04-25T10:02:09.184870+0000","last_clean":"2024-04-25T10:02:09.184870+0000","last_became_active":"2024-04-25T10:02:08.196287+0000","last_became_peered":"2024-04-25T10:02:08.196287+0000","last_unstale":"2024-04-25T10:02:09.184870+0000","last_undegraded":"2024-04-25T10:02:09.184870+0000","last_fullsized":"2024-04-25T10:02:09.184870+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:57.959507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3,6],"acting":[4,1,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513645+0000","last_change":"2024-04-25T10:02:04.148290+0000","last_active":"2024-04-25T10:02:09.513645+0000","last_peered":"2024-04-25T10:02:09.513645+0000","last_clean":"2024-04-25T10:02:09.513645+0000","last_became_active":"2024-04-25T10:02:04.147943+0000","last_became_peered":"2024-04-25T10:02:04.147943+0000","last_unstale":"2024-04-25T10:02:09.513645+0000","last_undegraded":"2024-04-25T10:02:09.513645+0000","last_fullsized":"2024-04-25T10:02:09.513645+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":30,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181446+0000","last_change":"2024-04-25T10:02:03.125338+0000","last_active":"2024-04-25T10:02:09.181446+0000","last_peered":"2024-04-25T10:02:09.181446+0000","last_clean":"2024-04-25T10:02:09.181446+0000","last_became_active":"2024-04-25T10:02:03.123509+0000","last_became_peered":"2024-04-25T10:02:03.123509+0000","last_unstale":"2024-04-25T10:02:09.181446+0000","last_undegraded":"2024-04-25T10:02:09.181446+0000","last_fullsized":"2024-04-25T10:02:09.181446+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514606+0000","last_change":"2024-04-25T10:02:08.181642+0000","last_active":"2024-04-25T10:02:09.514606+0000","last_peered":"2024-04-25T10:02:09.514606+0000","last_clean":"2024-04-25T10:02:09.514606+0000","last_became_active":"2024-04-25T10:02:08.181231+0000","last_became_peered":"2024-04-25T10:02:08.181231+0000","last_unstale":"2024-04-25T10:02:09.514606+0000","last_undegraded":"2024-04-25T10:02:09.514606+0000","last_fullsized":"2024-04-25T10:02:09.514606+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2,6],"acting":[0,3,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182787+0000","last_change":"2024-04-25T10:02:04.125758+0000","last_active":"2024-04-25T10:02:09.182787+0000","last_peered":"2024-04-25T10:02:09.182787+0000","last_clean":"2024-04-25T10:02:09.182787+0000","last_became_active":"2024-04-25T10:02:04.124948+0000","last_became_peered":"2024-04-25T10:02:04.124948+0000","last_unstale":"2024-04-25T10:02:09.182787+0000","last_undegraded":"2024-04-25T10:02:09.182787+0000","last_fullsized":"2024-04-25T10:02:09.182787+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:13.264948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184171+0000","last_change":"2024-04-25T10:02:03.136435+0000","last_active":"2024-04-25T10:02:09.184171+0000","last_peered":"2024-04-25T10:02:09.184171+0000","last_clean":"2024-04-25T10:02:09.184171+0000","last_became_active":"2024-04-25T10:02:03.134440+0000","last_became_peered":"2024-04-25T10:02:03.134440+0000","last_unstale":"2024-04-25T10:02:09.184171+0000","last_undegraded":"2024-04-25T10:02:09.184171+0000","last_fullsized":"2024-04-25T10:02:09.184171+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520993+0000","last_change":"2024-04-25T10:02:08.393430+0000","last_active":"2024-04-25T10:02:09.520993+0000","last_peered":"2024-04-25T10:02:09.520993+0000","last_clean":"2024-04-25T10:02:09.520993+0000","last_became_active":"2024-04-25T10:02:08.392682+0000","last_became_peered":"2024-04-25T10:02:08.392682+0000","last_unstale":"2024-04-25T10:02:09.520993+0000","last_undegraded":"2024-04-25T10:02:09.520993+0000","last_fullsized":"2024-04-25T10:02:09.520993+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:45.626796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,6],"acting":[3,7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514273+0000","last_change":"2024-04-25T10:02:04.145690+0000","last_active":"2024-04-25T10:02:09.514273+0000","last_peered":"2024-04-25T10:02:09.514273+0000","last_clean":"2024-04-25T10:02:09.514273+0000","last_became_active":"2024-04-25T10:02:04.145414+0000","last_became_peered":"2024-04-25T10:02:04.145414+0000","last_unstale":"2024-04-25T10:02:09.514273+0000","last_undegraded":"2024-04-25T10:02:09.514273+0000","last_fullsized":"2024-04-25T10:02:09.514273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514810+0000","last_change":"2024-04-25T10:02:03.151971+0000","last_active":"2024-04-25T10:02:09.514810+0000","last_peered":"2024-04-25T10:02:09.514810+0000","last_clean":"2024-04-25T10:02:09.514810+0000","last_became_active":"2024-04-25T10:02:03.151708+0000","last_became_peered":"2024-04-25T10:02:03.151708+0000","last_unstale":"2024-04-25T10:02:09.514810+0000","last_undegraded":"2024-04-25T10:02:09.514810+0000","last_fullsized":"2024-04-25T10:02:09.514810+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520552+0000","last_change":"2024-04-25T10:02:08.393020+0000","last_active":"2024-04-25T10:02:09.520552+0000","last_peered":"2024-04-25T10:02:09.520552+0000","last_clean":"2024-04-25T10:02:09.520552+0000","last_became_active":"2024-04-25T10:02:08.392543+0000","last_became_peered":"2024-04-25T10:02:08.392543+0000","last_unstale":"2024-04-25T10:02:09.520552+0000","last_undegraded":"2024-04-25T10:02:09.520552+0000","last_fullsized":"2024-04-25T10:02:09.520552+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:32.205527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,2],"acting":[3,1,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512848+0000","last_change":"2024-04-25T10:02:04.137453+0000","last_active":"2024-04-25T10:02:09.512848+0000","last_peered":"2024-04-25T10:02:09.512848+0000","last_clean":"2024-04-25T10:02:09.512848+0000","last_became_active":"2024-04-25T10:02:04.136751+0000","last_became_peered":"2024-04-25T10:02:04.136751+0000","last_unstale":"2024-04-25T10:02:09.512848+0000","last_undegraded":"2024-04-25T10:02:09.512848+0000","last_fullsized":"2024-04-25T10:02:09.512848+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":24,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514252+0000","last_change":"2024-04-25T10:02:03.134138+0000","last_active":"2024-04-25T10:02:09.514252+0000","last_peered":"2024-04-25T10:02:09.514252+0000","last_clean":"2024-04-25T10:02:09.514252+0000","last_became_active":"2024-04-25T10:02:03.133331+0000","last_became_peered":"2024-04-25T10:02:03.133331+0000","last_unstale":"2024-04-25T10:02:09.514252+0000","last_undegraded":"2024-04-25T10:02:09.514252+0000","last_fullsized":"2024-04-25T10:02:09.514252+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:15:06.836818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516833+0000","last_change":"2024-04-25T10:02:08.399232+0000","last_active":"2024-04-25T10:02:09.516833+0000","last_peered":"2024-04-25T10:02:09.516833+0000","last_clean":"2024-04-25T10:02:09.516833+0000","last_became_active":"2024-04-25T10:02:08.393743+0000","last_became_peered":"2024-04-25T10:02:08.393743+0000","last_unstale":"2024-04-25T10:02:09.516833+0000","last_undegraded":"2024-04-25T10:02:09.516833+0000","last_fullsized":"2024-04-25T10:02:09.516833+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3,2],"acting":[5,7,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514339+0000","last_change":"2024-04-25T10:02:04.145573+0000","last_active":"2024-04-25T10:02:09.514339+0000","last_peered":"2024-04-25T10:02:09.514339+0000","last_clean":"2024-04-25T10:02:09.514339+0000","last_became_active":"2024-04-25T10:02:04.140591+0000","last_became_peered":"2024-04-25T10:02:04.140591+0000","last_unstale":"2024-04-25T10:02:09.514339+0000","last_undegraded":"2024-04-25T10:02:09.514339+0000","last_fullsized":"2024-04-25T10:02:09.514339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:45.996784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.14","version":"24'10","reported_seq":32,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:21.801655+0000","last_change":"2024-04-25T10:02:03.131213+0000","last_active":"2024-04-25T10:02:21.801655+0000","last_peered":"2024-04-25T10:02:21.801655+0000","last_clean":"2024-04-25T10:02:21.801655+0000","last_became_active":"2024-04-25T10:02:03.130876+0000","last_became_peered":"2024-04-25T10:02:03.130876+0000","last_unstale":"2024-04-25T10:02:21.801655+0000","last_undegraded":"2024-04-25T10:02:21.801655+0000","last_fullsized":"2024-04-25T10:02:21.801655+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:51.256891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.518147+0000","last_change":"2024-04-25T10:02:08.197739+0000","last_active":"2024-04-25T10:02:09.518147+0000","last_peered":"2024-04-25T10:02:09.518147+0000","last_clean":"2024-04-25T10:02:09.518147+0000","last_became_active":"2024-04-25T10:02:08.195924+0000","last_became_peered":"2024-04-25T10:02:08.195924+0000","last_unstale":"2024-04-25T10:02:09.518147+0000","last_undegraded":"2024-04-25T10:02:09.518147+0000","last_fullsized":"2024-04-25T10:02:09.518147+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:30.888538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2,0],"acting":[1,6,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188422+0000","last_change":"2024-04-25T10:02:04.147813+0000","last_active":"2024-04-25T10:02:09.188422+0000","last_peered":"2024-04-25T10:02:09.188422+0000","last_clean":"2024-04-25T10:02:09.188422+0000","last_became_active":"2024-04-25T10:02:04.140229+0000","last_became_peered":"2024-04-25T10:02:04.140229+0000","last_unstale":"2024-04-25T10:02:09.188422+0000","last_undegraded":"2024-04-25T10:02:09.188422+0000","last_fullsized":"2024-04-25T10:02:09.188422+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:42:19.761376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":25,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515518+0000","last_change":"2024-04-25T10:02:03.154687+0000","last_active":"2024-04-25T10:02:09.515518+0000","last_peered":"2024-04-25T10:02:09.515518+0000","last_clean":"2024-04-25T10:02:09.515518+0000","last_became_active":"2024-04-25T10:02:03.154323+0000","last_became_peered":"2024-04-25T10:02:03.154323+0000","last_unstale":"2024-04-25T10:02:09.515518+0000","last_undegraded":"2024-04-25T10:02:09.515518+0000","last_fullsized":"2024-04-25T10:02:09.515518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:44.196001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187851+0000","last_change":"2024-04-25T10:02:08.193520+0000","last_active":"2024-04-25T10:02:09.187851+0000","last_peered":"2024-04-25T10:02:09.187851+0000","last_clean":"2024-04-25T10:02:09.187851+0000","last_became_active":"2024-04-25T10:02:08.193040+0000","last_became_peered":"2024-04-25T10:02:08.193040+0000","last_unstale":"2024-04-25T10:02:09.187851+0000","last_undegraded":"2024-04-25T10:02:09.187851+0000","last_fullsized":"2024-04-25T10:02:09.187851+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:45.384546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6,1],"acting":[4,5,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513593+0000","last_change":"2024-04-25T10:02:04.137131+0000","last_active":"2024-04-25T10:02:09.513593+0000","last_peered":"2024-04-25T10:02:09.513593+0000","last_clean":"2024-04-25T10:02:09.513593+0000","last_became_active":"2024-04-25T10:02:04.136832+0000","last_became_peered":"2024-04-25T10:02:04.136832+0000","last_unstale":"2024-04-25T10:02:09.513593+0000","last_undegraded":"2024-04-25T10:02:09.513593+0000","last_fullsized":"2024-04-25T10:02:09.513593+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516824+0000","last_change":"2024-04-25T10:02:03.154418+0000","last_active":"2024-04-25T10:02:09.516824+0000","last_peered":"2024-04-25T10:02:09.516824+0000","last_clean":"2024-04-25T10:02:09.516824+0000","last_became_active":"2024-04-25T10:02:03.154147+0000","last_became_peered":"2024-04-25T10:02:03.154147+0000","last_unstale":"2024-04-25T10:02:09.516824+0000","last_undegraded":"2024-04-25T10:02:09.516824+0000","last_fullsized":"2024-04-25T10:02:09.516824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:20.582610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519420+0000","last_change":"2024-04-25T10:02:08.192052+0000","last_active":"2024-04-25T10:02:09.519420+0000","last_peered":"2024-04-25T10:02:09.519420+0000","last_clean":"2024-04-25T10:02:09.519420+0000","last_became_active":"2024-04-25T10:02:08.190628+0000","last_became_peered":"2024-04-25T10:02:08.190628+0000","last_unstale":"2024-04-25T10:02:09.519420+0000","last_undegraded":"2024-04-25T10:02:09.519420+0000","last_fullsized":"2024-04-25T10:02:09.519420+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:44.000271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6,4],"acting":[3,1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516671+0000","last_change":"2024-04-25T10:02:04.149114+0000","last_active":"2024-04-25T10:02:09.516671+0000","last_peered":"2024-04-25T10:02:09.516671+0000","last_clean":"2024-04-25T10:02:09.516671+0000","last_became_active":"2024-04-25T10:02:04.147309+0000","last_became_peered":"2024-04-25T10:02:04.147309+0000","last_unstale":"2024-04-25T10:02:09.516671+0000","last_undegraded":"2024-04-25T10:02:09.516671+0000","last_fullsized":"2024-04-25T10:02:09.516671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:49.882635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187650+0000","last_change":"2024-04-25T10:02:03.150915+0000","last_active":"2024-04-25T10:02:09.187650+0000","last_peered":"2024-04-25T10:02:09.187650+0000","last_clean":"2024-04-25T10:02:09.187650+0000","last_became_active":"2024-04-25T10:02:03.150382+0000","last_became_peered":"2024-04-25T10:02:03.150382+0000","last_unstale":"2024-04-25T10:02:09.187650+0000","last_undegraded":"2024-04-25T10:02:09.187650+0000","last_fullsized":"2024-04-25T10:02:09.187650+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520143+0000","last_change":"2024-04-25T10:02:08.409717+0000","last_active":"2024-04-25T10:02:09.520143+0000","last_peered":"2024-04-25T10:02:09.520143+0000","last_clean":"2024-04-25T10:02:09.520143+0000","last_became_active":"2024-04-25T10:02:08.409153+0000","last_became_peered":"2024-04-25T10:02:08.409153+0000","last_unstale":"2024-04-25T10:02:09.520143+0000","last_undegraded":"2024-04-25T10:02:09.520143+0000","last_fullsized":"2024-04-25T10:02:09.520143+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:59.321979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6,5],"acting":[3,7,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514307+0000","last_change":"2024-04-25T10:02:08.402841+0000","last_active":"2024-04-25T10:02:09.514307+0000","last_peered":"2024-04-25T10:02:09.514307+0000","last_clean":"2024-04-25T10:02:09.514307+0000","last_became_active":"2024-04-25T10:02:08.402205+0000","last_became_peered":"2024-04-25T10:02:08.402205+0000","last_unstale":"2024-04-25T10:02:09.514307+0000","last_undegraded":"2024-04-25T10:02:09.514307+0000","last_fullsized":"2024-04-25T10:02:09.514307+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:58.689517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3,1],"acting":[0,7,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514163+0000","last_change":"2024-04-25T10:02:04.136408+0000","last_active":"2024-04-25T10:02:09.514163+0000","last_peered":"2024-04-25T10:02:09.514163+0000","last_clean":"2024-04-25T10:02:09.514163+0000","last_became_active":"2024-04-25T10:02:04.136094+0000","last_became_peered":"2024-04-25T10:02:04.136094+0000","last_unstale":"2024-04-25T10:02:09.514163+0000","last_undegraded":"2024-04-25T10:02:09.514163+0000","last_fullsized":"2024-04-25T10:02:09.514163+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513538+0000","last_change":"2024-04-25T10:02:03.154541+0000","last_active":"2024-04-25T10:02:09.513538+0000","last_peered":"2024-04-25T10:02:09.513538+0000","last_clean":"2024-04-25T10:02:09.513538+0000","last_became_active":"2024-04-25T10:02:03.154193+0000","last_became_peered":"2024-04-25T10:02:03.154193+0000","last_unstale":"2024-04-25T10:02:09.513538+0000","last_undegraded":"2024-04-25T10:02:09.513538+0000","last_fullsized":"2024-04-25T10:02:09.513538+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184564+0000","last_change":"2024-04-25T10:02:08.192506+0000","last_active":"2024-04-25T10:02:09.184564+0000","last_peered":"2024-04-25T10:02:09.184564+0000","last_clean":"2024-04-25T10:02:09.184564+0000","last_became_active":"2024-04-25T10:02:08.190910+0000","last_became_peered":"2024-04-25T10:02:08.190910+0000","last_unstale":"2024-04-25T10:02:09.184564+0000","last_undegraded":"2024-04-25T10:02:09.184564+0000","last_fullsized":"2024-04-25T10:02:09.184564+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:13:47.945987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1,0],"acting":[6,5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519465+0000","last_change":"2024-04-25T10:02:04.138218+0000","last_active":"2024-04-25T10:02:09.519465+0000","last_peered":"2024-04-25T10:02:09.519465+0000","last_clean":"2024-04-25T10:02:09.519465+0000","last_became_active":"2024-04-25T10:02:04.137919+0000","last_became_peered":"2024-04-25T10:02:04.137919+0000","last_unstale":"2024-04-25T10:02:09.519465+0000","last_undegraded":"2024-04-25T10:02:09.519465+0000","last_fullsized":"2024-04-25T10:02:09.519465+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520192+0000","last_change":"2024-04-25T10:02:03.133971+0000","last_active":"2024-04-25T10:02:09.520192+0000","last_peered":"2024-04-25T10:02:09.520192+0000","last_clean":"2024-04-25T10:02:09.520192+0000","last_became_active":"2024-04-25T10:02:03.133661+0000","last_became_peered":"2024-04-25T10:02:03.133661+0000","last_unstale":"2024-04-25T10:02:09.520192+0000","last_undegraded":"2024-04-25T10:02:09.520192+0000","last_fullsized":"2024-04-25T10:02:09.520192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:29.421464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183101+0000","last_change":"2024-04-25T10:02:03.137884+0000","last_active":"2024-04-25T10:02:09.183101+0000","last_peered":"2024-04-25T10:02:09.183101+0000","last_clean":"2024-04-25T10:02:09.183101+0000","last_became_active":"2024-04-25T10:02:03.137196+0000","last_became_peered":"2024-04-25T10:02:03.137196+0000","last_unstale":"2024-04-25T10:02:09.183101+0000","last_undegraded":"2024-04-25T10:02:09.183101+0000","last_fullsized":"2024-04-25T10:02:09.183101+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:37:15.184808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514125+0000","last_change":"2024-04-25T10:02:04.130491+0000","last_active":"2024-04-25T10:02:09.514125+0000","last_peered":"2024-04-25T10:02:09.514125+0000","last_clean":"2024-04-25T10:02:09.514125+0000","last_became_active":"2024-04-25T10:02:04.129963+0000","last_became_peered":"2024-04-25T10:02:04.129963+0000","last_unstale":"2024-04-25T10:02:09.514125+0000","last_undegraded":"2024-04-25T10:02:09.514125+0000","last_fullsized":"2024-04-25T10:02:09.514125+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182210+0000","last_change":"2024-04-25T10:02:08.186410+0000","last_active":"2024-04-25T10:02:09.182210+0000","last_peered":"2024-04-25T10:02:09.182210+0000","last_clean":"2024-04-25T10:02:09.182210+0000","last_became_active":"2024-04-25T10:02:08.185964+0000","last_became_peered":"2024-04-25T10:02:08.185964+0000","last_unstale":"2024-04-25T10:02:09.182210+0000","last_undegraded":"2024-04-25T10:02:09.182210+0000","last_fullsized":"2024-04-25T10:02:09.182210+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:28:17.471287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3,5],"acting":[2,1,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":23,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520398+0000","last_change":"2024-04-25T10:02:03.134295+0000","last_active":"2024-04-25T10:02:09.520398+0000","last_peered":"2024-04-25T10:02:09.520398+0000","last_clean":"2024-04-25T10:02:09.520398+0000","last_became_active":"2024-04-25T10:02:03.134033+0000","last_became_peered":"2024-04-25T10:02:03.134033+0000","last_unstale":"2024-04-25T10:02:09.520398+0000","last_undegraded":"2024-04-25T10:02:09.520398+0000","last_fullsized":"2024-04-25T10:02:09.520398+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:02.073074+0000","last_clean_scrub_stamp":"2024-04-25T10:02:02.073074+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:48:25.660990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.186180+0000","last_change":"2024-04-25T10:02:04.137055+0000","last_active":"2024-04-25T10:02:09.186180+0000","last_peered":"2024-04-25T10:02:09.186180+0000","last_clean":"2024-04-25T10:02:09.186180+0000","last_became_active":"2024-04-25T10:02:04.131629+0000","last_became_peered":"2024-04-25T10:02:04.131629+0000","last_unstale":"2024-04-25T10:02:09.186180+0000","last_undegraded":"2024-04-25T10:02:09.186180+0000","last_fullsized":"2024-04-25T10:02:09.186180+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521270+0000","last_change":"2024-04-25T10:02:08.187719+0000","last_active":"2024-04-25T10:02:09.521270+0000","last_peered":"2024-04-25T10:02:09.521270+0000","last_clean":"2024-04-25T10:02:09.521270+0000","last_became_active":"2024-04-25T10:02:08.187298+0000","last_became_peered":"2024-04-25T10:02:08.187298+0000","last_unstale":"2024-04-25T10:02:09.521270+0000","last_undegraded":"2024-04-25T10:02:09.521270+0000","last_fullsized":"2024-04-25T10:02:09.521270+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:07.130724+0000","last_clean_scrub_stamp":"2024-04-25T10:02:07.130724+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:15.995769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,2],"acting":[3,1,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182212+0000","last_change":"2024-04-25T10:02:04.125579+0000","last_active":"2024-04-25T10:02:09.182212+0000","last_peered":"2024-04-25T10:02:09.182212+0000","last_clean":"2024-04-25T10:02:09.182212+0000","last_became_active":"2024-04-25T10:02:04.124972+0000","last_became_peered":"2024-04-25T10:02:04.124972+0000","last_unstale":"2024-04-25T10:02:09.182212+0000","last_undegraded":"2024-04-25T10:02:09.182212+0000","last_fullsized":"2024-04-25T10:02:09.182212+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:57.854823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.181677+0000","last_change":"2024-04-25T10:02:04.145556+0000","last_active":"2024-04-25T10:02:09.181677+0000","last_peered":"2024-04-25T10:02:09.181677+0000","last_clean":"2024-04-25T10:02:09.181677+0000","last_became_active":"2024-04-25T10:02:04.144887+0000","last_became_peered":"2024-04-25T10:02:04.144887+0000","last_unstale":"2024-04-25T10:02:09.181677+0000","last_undegraded":"2024-04-25T10:02:09.181677+0000","last_fullsized":"2024-04-25T10:02:09.181677+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:45.210492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514803+0000","last_change":"2024-04-25T10:02:04.142127+0000","last_active":"2024-04-25T10:02:09.514803+0000","last_peered":"2024-04-25T10:02:09.514803+0000","last_clean":"2024-04-25T10:02:09.514803+0000","last_became_active":"2024-04-25T10:02:04.140799+0000","last_became_peered":"2024-04-25T10:02:04.140799+0000","last_unstale":"2024-04-25T10:02:09.514803+0000","last_undegraded":"2024-04-25T10:02:09.514803+0000","last_fullsized":"2024-04-25T10:02:09.514803+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514188+0000","last_change":"2024-04-25T10:02:04.141010+0000","last_active":"2024-04-25T10:02:09.514188+0000","last_peered":"2024-04-25T10:02:09.514188+0000","last_clean":"2024-04-25T10:02:09.514188+0000","last_became_active":"2024-04-25T10:02:04.140201+0000","last_became_peered":"2024-04-25T10:02:04.140201+0000","last_unstale":"2024-04-25T10:02:09.514188+0000","last_undegraded":"2024-04-25T10:02:09.514188+0000","last_fullsized":"2024-04-25T10:02:09.514188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:45.051893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521086+0000","last_change":"2024-04-25T10:02:04.137613+0000","last_active":"2024-04-25T10:02:09.521086+0000","last_peered":"2024-04-25T10:02:09.521086+0000","last_clean":"2024-04-25T10:02:09.521086+0000","last_became_active":"2024-04-25T10:02:04.136833+0000","last_became_peered":"2024-04-25T10:02:04.136833+0000","last_unstale":"2024-04-25T10:02:09.521086+0000","last_undegraded":"2024-04-25T10:02:09.521086+0000","last_fullsized":"2024-04-25T10:02:09.521086+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514990+0000","last_change":"2024-04-25T10:02:04.135708+0000","last_active":"2024-04-25T10:02:09.514990+0000","last_peered":"2024-04-25T10:02:09.514990+0000","last_clean":"2024-04-25T10:02:09.514990+0000","last_became_active":"2024-04-25T10:02:04.135383+0000","last_became_peered":"2024-04-25T10:02:04.135383+0000","last_unstale":"2024-04-25T10:02:09.514990+0000","last_undegraded":"2024-04-25T10:02:09.514990+0000","last_fullsized":"2024-04-25T10:02:09.514990+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187243+0000","last_change":"2024-04-25T10:02:04.151491+0000","last_active":"2024-04-25T10:02:09.187243+0000","last_peered":"2024-04-25T10:02:09.187243+0000","last_clean":"2024-04-25T10:02:09.187243+0000","last_became_active":"2024-04-25T10:02:04.151181+0000","last_became_peered":"2024-04-25T10:02:04.151181+0000","last_unstale":"2024-04-25T10:02:09.187243+0000","last_undegraded":"2024-04-25T10:02:09.187243+0000","last_fullsized":"2024-04-25T10:02:09.187243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.521012+0000","last_change":"2024-04-25T10:02:04.133889+0000","last_active":"2024-04-25T10:02:09.521012+0000","last_peered":"2024-04-25T10:02:09.521012+0000","last_clean":"2024-04-25T10:02:09.521012+0000","last_became_active":"2024-04-25T10:02:04.132937+0000","last_became_peered":"2024-04-25T10:02:04.132937+0000","last_unstale":"2024-04-25T10:02:09.521012+0000","last_undegraded":"2024-04-25T10:02:09.521012+0000","last_fullsized":"2024-04-25T10:02:09.521012+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:21.489151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513229+0000","last_change":"2024-04-25T10:02:04.142217+0000","last_active":"2024-04-25T10:02:09.513229+0000","last_peered":"2024-04-25T10:02:09.513229+0000","last_clean":"2024-04-25T10:02:09.513229+0000","last_became_active":"2024-04-25T10:02:04.141431+0000","last_became_peered":"2024-04-25T10:02:04.141431+0000","last_unstale":"2024-04-25T10:02:09.513229+0000","last_undegraded":"2024-04-25T10:02:09.513229+0000","last_fullsized":"2024-04-25T10:02:09.513229+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520025+0000","last_change":"2024-04-25T10:02:04.148470+0000","last_active":"2024-04-25T10:02:09.520025+0000","last_peered":"2024-04-25T10:02:09.520025+0000","last_clean":"2024-04-25T10:02:09.520025+0000","last_became_active":"2024-04-25T10:02:04.147993+0000","last_became_peered":"2024-04-25T10:02:04.147993+0000","last_unstale":"2024-04-25T10:02:09.520025+0000","last_undegraded":"2024-04-25T10:02:09.520025+0000","last_fullsized":"2024-04-25T10:02:09.520025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513946+0000","last_change":"2024-04-25T10:02:04.148427+0000","last_active":"2024-04-25T10:02:09.513946+0000","last_peered":"2024-04-25T10:02:09.513946+0000","last_clean":"2024-04-25T10:02:09.513946+0000","last_became_active":"2024-04-25T10:02:04.147301+0000","last_became_peered":"2024-04-25T10:02:04.147301+0000","last_unstale":"2024-04-25T10:02:09.513946+0000","last_undegraded":"2024-04-25T10:02:09.513946+0000","last_fullsized":"2024-04-25T10:02:09.513946+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513726+0000","last_change":"2024-04-25T10:02:04.136324+0000","last_active":"2024-04-25T10:02:09.513726+0000","last_peered":"2024-04-25T10:02:09.513726+0000","last_clean":"2024-04-25T10:02:09.513726+0000","last_became_active":"2024-04-25T10:02:04.136035+0000","last_became_peered":"2024-04-25T10:02:04.136035+0000","last_unstale":"2024-04-25T10:02:09.513726+0000","last_undegraded":"2024-04-25T10:02:09.513726+0000","last_fullsized":"2024-04-25T10:02:09.513726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.184020+0000","last_change":"2024-04-25T10:02:04.134441+0000","last_active":"2024-04-25T10:02:09.184020+0000","last_peered":"2024-04-25T10:02:09.184020+0000","last_clean":"2024-04-25T10:02:09.184020+0000","last_became_active":"2024-04-25T10:02:04.130052+0000","last_became_peered":"2024-04-25T10:02:04.130052+0000","last_unstale":"2024-04-25T10:02:09.184020+0000","last_undegraded":"2024-04-25T10:02:09.184020+0000","last_fullsized":"2024-04-25T10:02:09.184020+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515111+0000","last_change":"2024-04-25T10:02:04.146376+0000","last_active":"2024-04-25T10:02:09.515111+0000","last_peered":"2024-04-25T10:02:09.515111+0000","last_clean":"2024-04-25T10:02:09.515111+0000","last_became_active":"2024-04-25T10:02:04.146080+0000","last_became_peered":"2024-04-25T10:02:04.146080+0000","last_unstale":"2024-04-25T10:02:09.515111+0000","last_undegraded":"2024-04-25T10:02:09.515111+0000","last_fullsized":"2024-04-25T10:02:09.515111+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:39.949120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.182337+0000","last_change":"2024-04-25T10:02:04.135427+0000","last_active":"2024-04-25T10:02:09.182337+0000","last_peered":"2024-04-25T10:02:09.182337+0000","last_clean":"2024-04-25T10:02:09.182337+0000","last_became_active":"2024-04-25T10:02:04.134990+0000","last_became_peered":"2024-04-25T10:02:04.134990+0000","last_unstale":"2024-04-25T10:02:09.182337+0000","last_undegraded":"2024-04-25T10:02:09.182337+0000","last_fullsized":"2024-04-25T10:02:09.182337+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:18:51.292801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.517176+0000","last_change":"2024-04-25T10:02:04.140449+0000","last_active":"2024-04-25T10:02:09.517176+0000","last_peered":"2024-04-25T10:02:09.517176+0000","last_clean":"2024-04-25T10:02:09.517176+0000","last_became_active":"2024-04-25T10:02:04.140129+0000","last_became_peered":"2024-04-25T10:02:04.140129+0000","last_unstale":"2024-04-25T10:02:09.517176+0000","last_undegraded":"2024-04-25T10:02:09.517176+0000","last_fullsized":"2024-04-25T10:02:09.517176+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513015+0000","last_change":"2024-04-25T10:02:04.137099+0000","last_active":"2024-04-25T10:02:09.513015+0000","last_peered":"2024-04-25T10:02:09.513015+0000","last_clean":"2024-04-25T10:02:09.513015+0000","last_became_active":"2024-04-25T10:02:04.136845+0000","last_became_peered":"2024-04-25T10:02:04.136845+0000","last_unstale":"2024-04-25T10:02:09.513015+0000","last_undegraded":"2024-04-25T10:02:09.513015+0000","last_fullsized":"2024-04-25T10:02:09.513015+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:53.916349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514467+0000","last_change":"2024-04-25T10:02:04.133727+0000","last_active":"2024-04-25T10:02:09.514467+0000","last_peered":"2024-04-25T10:02:09.514467+0000","last_clean":"2024-04-25T10:02:09.514467+0000","last_became_active":"2024-04-25T10:02:04.132995+0000","last_became_peered":"2024-04-25T10:02:04.132995+0000","last_unstale":"2024-04-25T10:02:09.514467+0000","last_undegraded":"2024-04-25T10:02:09.514467+0000","last_fullsized":"2024-04-25T10:02:09.514467+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.512646+0000","last_change":"2024-04-25T10:02:04.128013+0000","last_active":"2024-04-25T10:02:09.512646+0000","last_peered":"2024-04-25T10:02:09.512646+0000","last_clean":"2024-04-25T10:02:09.512646+0000","last_became_active":"2024-04-25T10:02:04.123162+0000","last_became_peered":"2024-04-25T10:02:04.123162+0000","last_unstale":"2024-04-25T10:02:09.512646+0000","last_undegraded":"2024-04-25T10:02:09.512646+0000","last_fullsized":"2024-04-25T10:02:09.512646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:22:26.232712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.520128+0000","last_change":"2024-04-25T10:02:04.137865+0000","last_active":"2024-04-25T10:02:09.520128+0000","last_peered":"2024-04-25T10:02:09.520128+0000","last_clean":"2024-04-25T10:02:09.520128+0000","last_became_active":"2024-04-25T10:02:04.137413+0000","last_became_peered":"2024-04-25T10:02:04.137413+0000","last_unstale":"2024-04-25T10:02:09.520128+0000","last_undegraded":"2024-04-25T10:02:09.520128+0000","last_fullsized":"2024-04-25T10:02:09.520128+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:41.791114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.188286+0000","last_change":"2024-04-25T10:02:04.148039+0000","last_active":"2024-04-25T10:02:09.188286+0000","last_peered":"2024-04-25T10:02:09.188286+0000","last_clean":"2024-04-25T10:02:09.188286+0000","last_became_active":"2024-04-25T10:02:04.146104+0000","last_became_peered":"2024-04-25T10:02:04.146104+0000","last_unstale":"2024-04-25T10:02:09.188286+0000","last_undegraded":"2024-04-25T10:02:09.188286+0000","last_fullsized":"2024-04-25T10:02:09.188286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:15.812709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.183940+0000","last_change":"2024-04-25T10:02:04.149357+0000","last_active":"2024-04-25T10:02:09.183940+0000","last_peered":"2024-04-25T10:02:09.183940+0000","last_clean":"2024-04-25T10:02:09.183940+0000","last_became_active":"2024-04-25T10:02:04.148994+0000","last_became_peered":"2024-04-25T10:02:04.148994+0000","last_unstale":"2024-04-25T10:02:09.183940+0000","last_undegraded":"2024-04-25T10:02:09.183940+0000","last_fullsized":"2024-04-25T10:02:09.183940+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:33:14.538450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.187107+0000","last_change":"2024-04-25T10:02:04.144119+0000","last_active":"2024-04-25T10:02:09.187107+0000","last_peered":"2024-04-25T10:02:09.187107+0000","last_clean":"2024-04-25T10:02:09.187107+0000","last_became_active":"2024-04-25T10:02:04.143818+0000","last_became_peered":"2024-04-25T10:02:04.143818+0000","last_unstale":"2024-04-25T10:02:09.187107+0000","last_undegraded":"2024-04-25T10:02:09.187107+0000","last_fullsized":"2024-04-25T10:02:09.187107+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515823+0000","last_change":"2024-04-25T10:02:04.152842+0000","last_active":"2024-04-25T10:02:09.515823+0000","last_peered":"2024-04-25T10:02:09.515823+0000","last_clean":"2024-04-25T10:02:09.515823+0000","last_became_active":"2024-04-25T10:02:04.142957+0000","last_became_peered":"2024-04-25T10:02:04.142957+0000","last_unstale":"2024-04-25T10:02:09.515823+0000","last_undegraded":"2024-04-25T10:02:09.515823+0000","last_fullsized":"2024-04-25T10:02:09.515823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:34:07.297489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.519170+0000","last_change":"2024-04-25T10:02:04.143950+0000","last_active":"2024-04-25T10:02:09.519170+0000","last_peered":"2024-04-25T10:02:09.519170+0000","last_clean":"2024-04-25T10:02:09.519170+0000","last_became_active":"2024-04-25T10:02:04.143615+0000","last_became_peered":"2024-04-25T10:02:04.143615+0000","last_unstale":"2024-04-25T10:02:09.519170+0000","last_undegraded":"2024-04-25T10:02:09.519170+0000","last_fullsized":"2024-04-25T10:02:09.519170+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513367+0000","last_change":"2024-04-25T10:02:04.140815+0000","last_active":"2024-04-25T10:02:09.513367+0000","last_peered":"2024-04-25T10:02:09.513367+0000","last_clean":"2024-04-25T10:02:09.513367+0000","last_became_active":"2024-04-25T10:02:04.140458+0000","last_became_peered":"2024-04-25T10:02:04.140458+0000","last_unstale":"2024-04-25T10:02:09.513367+0000","last_undegraded":"2024-04-25T10:02:09.513367+0000","last_fullsized":"2024-04-25T10:02:09.513367+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:04.300560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513577+0000","last_change":"2024-04-25T10:02:04.144469+0000","last_active":"2024-04-25T10:02:09.513577+0000","last_peered":"2024-04-25T10:02:09.513577+0000","last_clean":"2024-04-25T10:02:09.513577+0000","last_became_active":"2024-04-25T10:02:04.143382+0000","last_became_peered":"2024-04-25T10:02:04.143382+0000","last_unstale":"2024-04-25T10:02:09.513577+0000","last_undegraded":"2024-04-25T10:02:09.513577+0000","last_fullsized":"2024-04-25T10:02:09.513577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.514018+0000","last_change":"2024-04-25T10:02:04.138676+0000","last_active":"2024-04-25T10:02:09.514018+0000","last_peered":"2024-04-25T10:02:09.514018+0000","last_clean":"2024-04-25T10:02:09.514018+0000","last_became_active":"2024-04-25T10:02:04.138397+0000","last_became_peered":"2024-04-25T10:02:04.138397+0000","last_unstale":"2024-04-25T10:02:09.514018+0000","last_undegraded":"2024-04-25T10:02:09.514018+0000","last_fullsized":"2024-04-25T10:02:09.514018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:01:44.525256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.516016+0000","last_change":"2024-04-25T10:02:04.145097+0000","last_active":"2024-04-25T10:02:09.516016+0000","last_peered":"2024-04-25T10:02:09.516016+0000","last_clean":"2024-04-25T10:02:09.516016+0000","last_became_active":"2024-04-25T10:02:04.142128+0000","last_became_peered":"2024-04-25T10:02:04.142128+0000","last_unstale":"2024-04-25T10:02:09.516016+0000","last_undegraded":"2024-04-25T10:02:09.516016+0000","last_fullsized":"2024-04-25T10:02:09.516016+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:41.341387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.513292+0000","last_change":"2024-04-25T10:02:04.124760+0000","last_active":"2024-04-25T10:02:09.513292+0000","last_peered":"2024-04-25T10:02:09.513292+0000","last_clean":"2024-04-25T10:02:09.513292+0000","last_became_active":"2024-04-25T10:02:04.124423+0000","last_became_peered":"2024-04-25T10:02:04.124423+0000","last_unstale":"2024-04-25T10:02:09.513292+0000","last_undegraded":"2024-04-25T10:02:09.513292+0000","last_fullsized":"2024-04-25T10:02:09.513292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:43:46.650573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515984+0000","last_change":"2024-04-25T10:02:04.129218+0000","last_active":"2024-04-25T10:02:09.515984+0000","last_peered":"2024-04-25T10:02:09.515984+0000","last_clean":"2024-04-25T10:02:09.515984+0000","last_became_active":"2024-04-25T10:02:04.126046+0000","last_became_peered":"2024-04-25T10:02:04.126046+0000","last_unstale":"2024-04-25T10:02:09.515984+0000","last_undegraded":"2024-04-25T10:02:09.515984+0000","last_fullsized":"2024-04-25T10:02:09.515984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:17.332297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":24,"state":"active+clean","last_fresh":"2024-04-25T10:02:09.515345+0000","last_change":"2024-04-25T10:02:04.137569+0000","last_active":"2024-04-25T10:02:09.515345+0000","last_peered":"2024-04-25T10:02:09.515345+0000","last_clean":"2024-04-25T10:02:09.515345+0000","last_became_active":"2024-04-25T10:02:04.137205+0000","last_became_peered":"2024-04-25T10:02:04.137205+0000","last_unstale":"2024-04-25T10:02:09.515345+0000","last_undegraded":"2024-04-25T10:02:09.515345+0000","last_fullsized":"2024-04-25T10:02:09.515345+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:02:03.087566+0000","last_clean_scrub_stamp":"2024-04-25T10:02:03.087566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:03.257484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4640,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":49,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":43,"ondisk_log_size":43,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574859,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343976,"statfs":{"total":96636764160,"available":96608231424,"internally_reserved":0,"allocated":851968,"data_stored":691206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574859,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":252,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":258048,"data_stored":100346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574859,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":100382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574859,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":104098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27280,"kb_used_data":248,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":253952,"data_stored":100256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574859,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27284,"kb_used_data":256,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":262144,"data_stored":100416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574859,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27868,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343972,"statfs":{"total":96636764160,"available":96608227328,"internally_reserved":0,"allocated":860160,"data_stored":691262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:02:32.470 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:02:32.470 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-25T10:02:32.470 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-25T10:02:32.470 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:02:32.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.700+0000 7ff4ecc00640 1 Processor -- start 2024-04-25T10:02:32.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.701+0000 7ff4ecc00640 1 -- start start 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4ecc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff4e815dbe0 0x7ff4e80d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4ecc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff4e80b2cb0 0x7ff4e80d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4ecc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e80b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4ecc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e815dfc0 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4ecc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4e80d1a00 con 0x7ff4e815dbe0 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4ecc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80b97a0 con 0x7ff4e80b92a0 2024-04-25T10:02:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff4e80b2cb0 0x7ff4e80d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff4e80b2cb0 0x7ff4e80d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50662/0 (socket says 172.21.15.165:50662) 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 -- 172.21.15.165:0/408950853 learned_addr learned my addr 172.21.15.165:0/408950853 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5e00640 1 --2- 172.21.15.165:0/408950853 >> v2:172.21.15.172:3300/0 conn(0x7ff4e815dbe0 0x7ff4e80d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e6800640 1 --2- 172.21.15.165:0/408950853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e80b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 -- 172.21.15.165:0/408950853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 msgr2=0x7ff4e80b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 --2- 172.21.15.165:0/408950853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e80b5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 -- 172.21.15.165:0/408950853 >> v2:172.21.15.172:3300/0 conn(0x7ff4e815dbe0 msgr2=0x7ff4e80d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 --2- 172.21.15.165:0/408950853 >> v2:172.21.15.172:3300/0 conn(0x7ff4e815dbe0 0x7ff4e80d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.703+0000 7ff4e5400640 1 -- 172.21.15.165:0/408950853 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80d1c50 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.704+0000 7ff4e5400640 1 --2- 172.21.15.165:0/408950853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80d0e50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4e806c140 tx=0x7ff4d002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fbb825effecab70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.704+0000 7ff4e4a00640 1 -- 172.21.15.165:0/408950853 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d0036070 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.704+0000 7ff4e4a00640 1 -- 172.21.15.165:0/408950853 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d0031600 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.704+0000 7ff4e4a00640 1 -- 172.21.15.165:0/408950853 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d00319d0 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 -- 172.21.15.165:0/408950853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 msgr2=0x7ff4e80d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/408950853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80d0e50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7ff4e806c140 tx=0x7ff4d002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 -- 172.21.15.165:0/408950853 shutdown_connections 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/408950853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e80b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/408950853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80d0e50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/408950853 >> v2:172.21.15.172:3300/0 conn(0x7ff4e815dbe0 0x7ff4e80d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 -- 172.21.15.165:0/408950853 >> 172.21.15.165:0/408950853 conn(0x7ff4e8061a20 msgr2=0x7ff4e8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 -- 172.21.15.165:0/408950853 shutdown_connections 2024-04-25T10:02:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.705+0000 7ff4ecc00640 1 -- 172.21.15.165:0/408950853 wait complete. 2024-04-25T10:02:32.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.707+0000 7ff4ecc00640 1 Processor -- start 2024-04-25T10:02:32.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.707+0000 7ff4ecc00640 1 -- start start 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.708+0000 7ff4ecc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.708+0000 7ff4ecc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e81607b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4ecc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e815dbe0 0x7ff4e8151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4ecc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80b6ab0 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4ecc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80d2210 con 0x7ff4e815dbe0 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4ecc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4e80b6c30 con 0x7ff4e80b92a0 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80cd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50676/0 (socket says 172.21.15.165:50676) 2024-04-25T10:02:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 -- 172.21.15.165:0/1093856753 learned_addr learned my addr 172.21.15.165:0/1093856753 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e6800640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e815dbe0 0x7ff4e8151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5400640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e81607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 -- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 msgr2=0x7ff4e81607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e81607b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 -- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e815dbe0 msgr2=0x7ff4e8151b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e815dbe0 0x7ff4e8151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.709+0000 7ff4e5e00640 1 -- 172.21.15.165:0/1093856753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e80cc3f0 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.710+0000 7ff4e5e00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80cd170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800d5f0 tx=0x7ff4d800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.710+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80125f0 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.710+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4d8019070 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.710+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4d80129d0 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.710+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4e80cc690 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.710+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4e80ccc50 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.711+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4e8073990 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.712+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff4d8012b70 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.713+0000 7ff4de000640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.713+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff4d808a920 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.713+0000 7ff4e5400640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.714+0000 7ff4e5400640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff4c406b4d0 0x7ff4c406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0033000 tx=0x7ff4d0009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.717+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff4d8059450 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.951+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff4e8154880 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.952+0000 7ff4de000640 1 -- 172.21.15.165:0/1093856753 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff4d8058d50 con 0x7ff4e80b2cb0 2024-04-25T10:02:32.957 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:32.957 INFO:teuthology.orchestra.run.smithi165.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff4c406b4d0 msgr2=0x7ff4c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff4c406b4d0 0x7ff4c406d990 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff4d0033000 tx=0x7ff4d0009000 comp rx=0 tx=0).stop 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 msgr2=0x7ff4e80cd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80cd170 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7ff4d800d5f0 tx=0x7ff4d800dab0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 shutdown_connections 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff4c406b4d0 0x7ff4c406d990 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff4e815dbe0 0x7ff4e8151b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff4e80b92a0 0x7ff4e81607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 --2- 172.21.15.165:0/1093856753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff4e80b2cb0 0x7ff4e80cd170 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 >> 172.21.15.165:0/1093856753 conn(0x7ff4e8061a20 msgr2=0x7ff4e80d2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 shutdown_connections 2024-04-25T10:02:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:32.955+0000 7ff4ecc00640 1 -- 172.21.15.165:0/1093856753 wait complete. 2024-04-25T10:02:32.979 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-25T10:02:32.979 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.156+0000 7f2f51a00640 1 Processor -- start 2024-04-25T10:02:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.156+0000 7f2f51a00640 1 -- start start 2024-04-25T10:02:33.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f51a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f4c144440 0x7f2f4c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f51a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2f4c071530 0x7f2f4c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f51a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c15bf30 con 0x7f2f4c0cd860 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f51a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c0719a0 con 0x7f2f4c071530 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f51a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c15c860 con 0x7f2f4c144440 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f43400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2f4c071530 0x7f2f4c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f43400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2f4c071530 0x7f2f4c0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50684/0 (socket says 172.21.15.165:50684) 2024-04-25T10:02:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.158+0000 7f2f43400640 1 -- 172.21.15.165:0/884145509 learned_addr learned my addr 172.21.15.165:0/884145509 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.159+0000 7f2f4be00640 1 --2- 172.21.15.165:0/884145509 >> v2:172.21.15.172:3300/0 conn(0x7f2f4c144440 0x7f2f4c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.159+0000 7f2f43400640 1 -- 172.21.15.165:0/884145509 >> v2:172.21.15.172:3300/0 conn(0x7f2f4c144440 msgr2=0x7f2f4c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.159+0000 7f2f43400640 1 --2- 172.21.15.165:0/884145509 >> v2:172.21.15.172:3300/0 conn(0x7f2f4c144440 0x7f2f4c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.159+0000 7f2f43400640 1 -- 172.21.15.165:0/884145509 >> v1:172.21.15.172:6789/0 conn(0x7f2f4c0cd860 legacy=0x7f2f4c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.159+0000 7f2f43400640 1 -- 172.21.15.165:0/884145509 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c15c210 con 0x7f2f4c071530 2024-04-25T10:02:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.160+0000 7f2f43400640 1 --2- 172.21.15.165:0/884145509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c071530 0x7f2f4c0cd320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c06f470 tx=0x7f2f3402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed61df75b3aa0ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.160+0000 7f2f4b400640 1 -- 172.21.15.165:0/884145509 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34037070 con 0x7f2f4c071530 2024-04-25T10:02:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.160+0000 7f2f4b400640 1 -- 172.21.15.165:0/884145509 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f34032440 con 0x7f2f4c071530 2024-04-25T10:02:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.160+0000 7f2f4b400640 1 -- 172.21.15.165:0/884145509 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34032810 con 0x7f2f4c071530 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 -- 172.21.15.165:0/884145509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c071530 msgr2=0x7f2f4c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 --2- 172.21.15.165:0/884145509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c071530 0x7f2f4c0cd320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c06f470 tx=0x7f2f3402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 -- 172.21.15.165:0/884145509 shutdown_connections 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 --2- 172.21.15.165:0/884145509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c071530 0x7f2f4c0cd320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 --2- 172.21.15.165:0/884145509 >> v2:172.21.15.172:3300/0 conn(0x7f2f4c144440 0x7f2f4c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 -- 172.21.15.165:0/884145509 >> 172.21.15.165:0/884145509 conn(0x7f2f4c06abe0 msgr2=0x7f2f4c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 -- 172.21.15.165:0/884145509 shutdown_connections 2024-04-25T10:02:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.161+0000 7f2f51a00640 1 -- 172.21.15.165:0/884145509 wait complete. 2024-04-25T10:02:33.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.163+0000 7f2f51a00640 1 Processor -- start 2024-04-25T10:02:33.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.164+0000 7f2f51a00640 1 -- start start 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f4c071530 0x7f2f4c1503a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4c0cd860 0x7f2f4c0c9230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 0x7f2f4c151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0cdcd0 con 0x7f2f4c144440 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15bc30 con 0x7f2f4c0cd860 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0cde50 con 0x7f2f4c071530 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 0x7f2f4c151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 0x7f2f4c151a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50692/0 (socket says 172.21.15.165:50692) 2024-04-25T10:02:33.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 -- 172.21.15.165:0/1324486611 learned_addr learned my addr 172.21.15.165:0/1324486611 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f4be00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f4c071530 0x7f2f4c1503a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f43400640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4c0cd860 0x7f2f4c0c9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 -- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f4c071530 msgr2=0x7f2f4c1503a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f4c071530 0x7f2f4c1503a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 -- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4c0cd860 msgr2=0x7f2f4c0c9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4c0cd860 0x7f2f4c0c9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f51000640 1 -- 172.21.15.165:0/1324486611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c150980 con 0x7f2f4c144440 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.165+0000 7f2f43400640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4c0cd860 0x7f2f4c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.166+0000 7f2f51000640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 0x7f2f4c151a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c7b0 tx=0x7f2f4400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.166+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004b60 con 0x7f2f4c144440 2024-04-25T10:02:33.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.166+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44004d00 con 0x7f2f4c144440 2024-04-25T10:02:33.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.166+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400aa90 con 0x7f2f4c144440 2024-04-25T10:02:33.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.166+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4c150c60 con 0x7f2f4c144440 2024-04-25T10:02:33.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.166+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c151500 con 0x7f2f4c144440 2024-04-25T10:02:33.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.167+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f4c150e30 con 0x7f2f4c144440 2024-04-25T10:02:33.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.168+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f4400ac30 con 0x7f2f4c144440 2024-04-25T10:02:33.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.170+0000 7f2f48c00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.170+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f2f4408a8b0 con 0x7f2f4c144440 2024-04-25T10:02:33.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.170+0000 7f2f4be00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.171+0000 7f2f4be00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b4d0 0x7f2f2806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c006fd0 tx=0x7f2f3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.174+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f44020070 con 0x7f2f4c144440 2024-04-25T10:02:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.356+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f4c073990 con 0x7f2f4c144440 2024-04-25T10:02:33.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.359+0000 7f2f48c00640 1 -- 172.21.15.165:0/1324486611 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f2f44059350 con 0x7f2f4c144440 2024-04-25T10:02:33.365 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:33.365 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b4d0 msgr2=0x7f2f2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b4d0 0x7f2f2806d990 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c006fd0 tx=0x7f2f3c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 msgr2=0x7f2f4c151a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 0x7f2f4c151a20 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400c7b0 tx=0x7f2f4400cc70 comp rx=0 tx=0).stop 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 shutdown_connections 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b4d0 0x7f2f2806d990 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f4c144440 0x7f2f4c151a20 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f4c0cd860 0x7f2f4c0c9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 --2- 172.21.15.165:0/1324486611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f4c071530 0x7f2f4c1503a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 >> 172.21.15.165:0/1324486611 conn(0x7f2f4c06abe0 msgr2=0x7f2f4c142ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 shutdown_connections 2024-04-25T10:02:33.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.361+0000 7f2f51a00640 1 -- 172.21.15.165:0/1324486611 wait complete. 2024-04-25T10:02:33.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:33.386 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 Processor -- start 2024-04-25T10:02:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 -- start start 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb5400cc4f0 0x7fb5400cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5400cdb20 0x7fb5400cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54015b270 0x7fb54015b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb540071270 con 0x7fb5400cdb20 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb540072280 con 0x7fb5400cc4f0 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.558+0000 7fb544a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5400714c0 con 0x7fb54015b270 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5400cdb20 0x7fb5400cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5400cdb20 0x7fb5400cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50700/0 (socket says 172.21.15.165:50700) 2024-04-25T10:02:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 -- 172.21.15.165:0/2795071763 learned_addr learned my addr 172.21.15.165:0/2795071763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb53de00640 1 --2- 172.21.15.165:0/2795071763 >> v2:172.21.15.172:3300/0 conn(0x7fb5400cc4f0 0x7fb5400cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb53e800640 1 --2- 172.21.15.165:0/2795071763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54015b270 0x7fb54015b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 -- 172.21.15.165:0/2795071763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54015b270 msgr2=0x7fb54015b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 --2- 172.21.15.165:0/2795071763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54015b270 0x7fb54015b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 -- 172.21.15.165:0/2795071763 >> v2:172.21.15.172:3300/0 conn(0x7fb5400cc4f0 msgr2=0x7fb5400cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 --2- 172.21.15.165:0/2795071763 >> v2:172.21.15.172:3300/0 conn(0x7fb5400cc4f0 0x7fb5400cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.559+0000 7fb535400640 1 -- 172.21.15.165:0/2795071763 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5400717a0 con 0x7fb5400cdb20 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.560+0000 7fb535400640 1 --2- 172.21.15.165:0/2795071763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5400cdb20 0x7fb5400cdfb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5400cd290 tx=0x7fb52802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5417daf703f30d41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.560+0000 7fb53d400640 1 -- 172.21.15.165:0/2795071763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528037070 con 0x7fb5400cdb20 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.560+0000 7fb53d400640 1 -- 172.21.15.165:0/2795071763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb528032440 con 0x7fb5400cdb20 2024-04-25T10:02:33.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.560+0000 7fb53d400640 1 -- 172.21.15.165:0/2795071763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb528032810 con 0x7fb5400cdb20 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 -- 172.21.15.165:0/2795071763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5400cdb20 msgr2=0x7fb5400cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 --2- 172.21.15.165:0/2795071763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5400cdb20 0x7fb5400cdfb0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb5400cd290 tx=0x7fb52802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 -- 172.21.15.165:0/2795071763 shutdown_connections 2024-04-25T10:02:33.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 --2- 172.21.15.165:0/2795071763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54015b270 0x7fb54015b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 --2- 172.21.15.165:0/2795071763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5400cdb20 0x7fb5400cdfb0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 --2- 172.21.15.165:0/2795071763 >> v2:172.21.15.172:3300/0 conn(0x7fb5400cc4f0 0x7fb5400cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 -- 172.21.15.165:0/2795071763 >> 172.21.15.165:0/2795071763 conn(0x7fb54006abe0 msgr2=0x7fb54006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 -- 172.21.15.165:0/2795071763 shutdown_connections 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.561+0000 7fb544a00640 1 -- 172.21.15.165:0/2795071763 wait complete. 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.563+0000 7fb544a00640 1 Processor -- start 2024-04-25T10:02:33.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.563+0000 7fb544a00640 1 -- start start 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb544a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5400cc4f0 0x7fb540146210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb544a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5400cdb20 0x7fb5400c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb544a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 0x7fb54015d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb544a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54015bc00 con 0x7fb54015b270 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb544a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540071d60 con 0x7fb5400cc4f0 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb544a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54015bd80 con 0x7fb5400cdb20 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 0x7fb54015d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 0x7fb54015d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50710/0 (socket says 172.21.15.165:50710) 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 -- 172.21.15.165:0/3171268369 learned_addr learned my addr 172.21.15.165:0/3171268369 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 -- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5400cdb20 msgr2=0x7fb5400c8ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53de00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5400cc4f0 0x7fb540146210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5400cdb20 0x7fb5400c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 -- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5400cc4f0 msgr2=0x7fb540146210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5400cc4f0 0x7fb540146210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.564+0000 7fb53e800640 1 -- 172.21.15.165:0/3171268369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb540146750 con 0x7fb54015b270 2024-04-25T10:02:33.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.565+0000 7fb53e800640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 0x7fb54015d0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb53800cd70 tx=0x7fb53800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.565+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538003b90 con 0x7fb54015b270 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.565+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb538003d30 con 0x7fb54015b270 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.565+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5400c94c0 con 0x7fb54015b270 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.565+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb538004240 con 0x7fb54015b270 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.565+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5400c9a80 con 0x7fb54015b270 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.566+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb540073a00 con 0x7fb54015b270 2024-04-25T10:02:33.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.567+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb5380043e0 con 0x7fb54015b270 2024-04-25T10:02:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.568+0000 7fb536a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb51c06b500 0x7fb51c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.568+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb53808b0a0 con 0x7fb54015b270 2024-04-25T10:02:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.568+0000 7fb53de00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb51c06b500 0x7fb51c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.569+0000 7fb53de00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb51c06b500 0x7fb51c06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb530006fd0 tx=0x7fb530008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.572+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb538059b40 con 0x7fb54015b270 2024-04-25T10:02:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.750+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb540151850 con 0x7fb54015b270 2024-04-25T10:02:33.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.751+0000 7fb536a00640 1 -- 172.21.15.165:0/3171268369 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb538059440 con 0x7fb54015b270 2024-04-25T10:02:33.755 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb51c06b500 msgr2=0x7fb51c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb51c06b500 0x7fb51c06d9c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb530006fd0 tx=0x7fb530008040 comp rx=0 tx=0).stop 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 msgr2=0x7fb54015d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 0x7fb54015d0e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fb53800cd70 tx=0x7fb53800a660 comp rx=0 tx=0).stop 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 shutdown_connections 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb51c06b500 0x7fb51c06d9c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54015b270 0x7fb54015d0e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5400cdb20 0x7fb5400c8ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 --2- 172.21.15.165:0/3171268369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5400cc4f0 0x7fb540146210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 >> 172.21.15.165:0/3171268369 conn(0x7fb54006abe0 msgr2=0x7fb5400cdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.753+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 shutdown_connections 2024-04-25T10:02:33.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.754+0000 7fb544a00640 1 -- 172.21.15.165:0/3171268369 wait complete. 2024-04-25T10:02:33.774 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:33.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.954+0000 7fde1d200640 1 Processor -- start 2024-04-25T10:02:33.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.954+0000 7fde1d200640 1 -- start start 2024-04-25T10:02:33.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.955+0000 7fde1d200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fde180ceb20 0x7fde180c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.955+0000 7fde1d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180c5030 0x7fde1815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.955+0000 7fde1d200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde18145520 con 0x7fde18145140 2024-04-25T10:02:33.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.955+0000 7fde1d200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde18071270 con 0x7fde180ceb20 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.955+0000 7fde1d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde18071550 con 0x7fde180c5030 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fde180ceb20 0x7fde180c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fde180ceb20 0x7fde180c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50722/0 (socket says 172.21.15.165:50722) 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 -- 172.21.15.165:0/3120000946 learned_addr learned my addr 172.21.15.165:0/3120000946 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 -- 172.21.15.165:0/3120000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180c5030 msgr2=0x7fde1815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 --2- 172.21.15.165:0/3120000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180c5030 0x7fde1815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 -- 172.21.15.165:0/3120000946 >> v1:172.21.15.172:6789/0 conn(0x7fde18145140 legacy=0x7fde180ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:33.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.956+0000 7fde0fe00640 1 -- 172.21.15.165:0/3120000946 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde18071830 con 0x7fde180ceb20 2024-04-25T10:02:33.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.957+0000 7fde0fe00640 1 --2- 172.21.15.165:0/3120000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180ceb20 0x7fde180c4af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fde00009870 tx=0x7fde0002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6547d6ee89c1dc93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.957+0000 7fde16a00640 1 -- 172.21.15.165:0/3120000946 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00037070 con 0x7fde180ceb20 2024-04-25T10:02:33.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.957+0000 7fde16a00640 1 -- 172.21.15.165:0/3120000946 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde00032440 con 0x7fde180ceb20 2024-04-25T10:02:33.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.957+0000 7fde16a00640 1 -- 172.21.15.165:0/3120000946 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde00032810 con 0x7fde180ceb20 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 -- 172.21.15.165:0/3120000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180ceb20 msgr2=0x7fde180c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 --2- 172.21.15.165:0/3120000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180ceb20 0x7fde180c4af0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fde00009870 tx=0x7fde0002a180 comp rx=0 tx=0).stop 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 -- 172.21.15.165:0/3120000946 shutdown_connections 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 --2- 172.21.15.165:0/3120000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180c5030 0x7fde1815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 --2- 172.21.15.165:0/3120000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180ceb20 0x7fde180c4af0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 -- 172.21.15.165:0/3120000946 >> 172.21.15.165:0/3120000946 conn(0x7fde18061cc0 msgr2=0x7fde1806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 -- 172.21.15.165:0/3120000946 shutdown_connections 2024-04-25T10:02:33.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.958+0000 7fde1d200640 1 -- 172.21.15.165:0/3120000946 wait complete. 2024-04-25T10:02:33.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.960+0000 7fde1d200640 1 Processor -- start 2024-04-25T10:02:33.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.960+0000 7fde1d200640 1 -- start start 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde1d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 0x7fde181554f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde1d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180ceb20 0x7fde180cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde1d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde18145140 0x7fde180cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde1d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180c5620 con 0x7fde180c5030 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde1d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde180c54a0 con 0x7fde18145140 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde1d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde1815c4f0 con 0x7fde180ceb20 2024-04-25T10:02:33.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde17400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 0x7fde181554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde17400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 0x7fde181554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50724/0 (socket says 172.21.15.165:50724) 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.961+0000 7fde17400640 1 -- 172.21.15.165:0/3831803395 learned_addr learned my addr 172.21.15.165:0/3831803395 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17e00640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde18145140 0x7fde180cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17400640 1 -- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180ceb20 msgr2=0x7fde180cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde0fe00640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180ceb20 0x7fde180cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17400640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180ceb20 0x7fde180cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17400640 1 -- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde18145140 msgr2=0x7fde180cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17400640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde18145140 0x7fde180cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17400640 1 -- 172.21.15.165:0/3831803395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1815b070 con 0x7fde180c5030 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17e00640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde18145140 0x7fde180cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:33.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde17400640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 0x7fde181554f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde0800b420 tx=0x7fde0800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08011830 con 0x7fde180c5030 2024-04-25T10:02:33.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde08016070 con 0x7fde180c5030 2024-04-25T10:02:33.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde08011c10 con 0x7fde180c5030 2024-04-25T10:02:33.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde1815b350 con 0x7fde180c5030 2024-04-25T10:02:33.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.962+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde1815b910 con 0x7fde180c5030 2024-04-25T10:02:33.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.963+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde1815d0e0 con 0x7fde180c5030 2024-04-25T10:02:33.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.964+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde0801b460 con 0x7fde180c5030 2024-04-25T10:02:33.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.965+0000 7fde0f400640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddf406b4d0 0x7fddf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:33.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.965+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fde0808a070 con 0x7fde180c5030 2024-04-25T10:02:33.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.965+0000 7fde0fe00640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddf406b4d0 0x7fddf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:33.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.966+0000 7fde0fe00640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddf406b4d0 0x7fddf406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde00034000 tx=0x7fde0002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:33.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:33.969+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde1815d0e0 con 0x7fde180c5030 2024-04-25T10:02:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.162+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde18073990 con 0x7fde180c5030 2024-04-25T10:02:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.163+0000 7fde0f400640 1 -- 172.21.15.165:0/3831803395 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde08020020 con 0x7fde180c5030 2024-04-25T10:02:34.168 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddf406b4d0 msgr2=0x7fddf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddf406b4d0 0x7fddf406d990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fde00034000 tx=0x7fde0002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 msgr2=0x7fde181554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 0x7fde181554f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde0800b420 tx=0x7fde0800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 shutdown_connections 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddf406b4d0 0x7fddf406d990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde18145140 0x7fde180cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde180ceb20 0x7fde180cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 --2- 172.21.15.165:0/3831803395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde180c5030 0x7fde181554f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 >> 172.21.15.165:0/3831803395 conn(0x7fde18061cc0 msgr2=0x7fde180b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 shutdown_connections 2024-04-25T10:02:34.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.166+0000 7fde1d200640 1 -- 172.21.15.165:0/3831803395 wait complete. 2024-04-25T10:02:34.190 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:34.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.386+0000 7f4cb9a00640 1 Processor -- start 2024-04-25T10:02:34.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.386+0000 7f4cb9a00640 1 -- start start 2024-04-25T10:02:34.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 0x7f4cb4168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb9a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb415e110 con 0x7f4cb415dd30 2024-04-25T10:02:34.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb9a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4cb40b2fd0 con 0x7f4cb406bff0 2024-04-25T10:02:34.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb406d000 con 0x7f4cb40b5b40 2024-04-25T10:02:34.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4cb415dd30 0x7f4cb40cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58182/0 (socket says 172.21.15.165:58182) 2024-04-25T10:02:34.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb3e00640 1 -- 172.21.15.165:0/641097007 learned_addr learned my addr 172.21.15.165:0/641097007 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb9000640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 0x7f4cb4168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007239281 0 0) 0x7f4cb415e110 con 0x7f4cb415dd30 2024-04-25T10:02:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.388+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c98003520 con 0x7f4cb415dd30 2024-04-25T10:02:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774508570 0 0) 0x7f4c98003520 con 0x7f4cb415dd30 2024-04-25T10:02:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 msgr2=0x7f4cb4168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 0x7f4cb4168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 >> v1:172.21.15.172:6789/0 conn(0x7f4cb406bff0 legacy=0x7f4cb40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb406c480 con 0x7f4cb415dd30 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4ca4001670 con 0x7f4cb415dd30 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 >> v1:172.21.15.165:6789/0 conn(0x7f4cb415dd30 legacy=0x7f4cb40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c980039e0 0x7f4c98003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c98004800 0x7f4c98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 0x7f4c980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb406d000 con 0x7f4c980039e0 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9800baa0 con 0x7f4c98004800 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9800bca0 con 0x7f4cb40b5b40 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c980039e0 0x7f4c98003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3400640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c98004800 0x7f4c98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3e00640 1 -- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 msgr2=0x7f4c980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4cb40b5b40 0x7f4c980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3e00640 1 -- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c98004800 msgr2=0x7f4c98004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c98004800 0x7f4c98004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.389+0000 7f4cb3e00640 1 -- 172.21.15.165:0/641097007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c98003520 con 0x7f4c980039e0 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.390+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c980039e0 0x7f4c98003e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4001eb0 tx=0x7f4c98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb77052f9812e3f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4007d30 con 0x7f4c980039e0 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ca4005af0 con 0x7f4c980039e0 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb2a00640 1 -- 172.21.15.165:0/641097007 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4007070 con 0x7f4c980039e0 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9a00640 1 -- 172.21.15.165:0/641097007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c980039e0 msgr2=0x7f4c98003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/641097007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c980039e0 0x7f4c98003e00 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4001eb0 tx=0x7f4c98005830 comp rx=0 tx=0).stop 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9000640 1 -- 172.21.15.165:0/641097007 reap_dead start 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9a00640 1 -- 172.21.15.165:0/641097007 shutdown_connections 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9a00640 1 -- 172.21.15.165:0/641097007 >> 172.21.15.165:0/641097007 conn(0x7f4cb40d0610 msgr2=0x7f4cb40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9a00640 1 -- 172.21.15.165:0/641097007 shutdown_connections 2024-04-25T10:02:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.391+0000 7f4cb9a00640 1 -- 172.21.15.165:0/641097007 wait complete. 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.393+0000 7f4cb9a00640 1 Processor -- start 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.393+0000 7f4cb9a00640 1 -- start start 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c980039e0 0x7f4cb41f81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cb415f980 0x7f4cb40cdb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 0x7f4cb41597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb406cd20 con 0x7f4cb4157d40 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb406ca40 con 0x7f4cb415f980 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cb40d3170 con 0x7f4c980039e0 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 0x7f4cb41597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 0x7f4cb41597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50754/0 (socket says 172.21.15.165:50754) 2024-04-25T10:02:34.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.394+0000 7f4cb9000640 1 -- 172.21.15.165:0/2048768042 learned_addr learned my addr 172.21.15.165:0/2048768042 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c980039e0 0x7f4cb41f81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb3400640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cb415f980 0x7f4cb40cdb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9000640 1 -- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c980039e0 msgr2=0x7f4cb41f81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9000640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c980039e0 0x7f4cb41f81b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9000640 1 -- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cb415f980 msgr2=0x7f4cb40cdb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9000640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cb415f980 0x7f4cb40cdb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9000640 1 -- 172.21.15.165:0/2048768042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cb41f7710 con 0x7f4cb4157d40 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb3400640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cb415f980 0x7f4cb40cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9000640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 0x7f4cb41597b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4cb408c1f0 tx=0x7f4cac00b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac00bd10 con 0x7f4cb4157d40 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cac01a070 con 0x7f4cb4157d40 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cac014640 con 0x7f4cb4157d40 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.395+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cb4160040 con 0x7f4cb4157d40 2024-04-25T10:02:34.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.396+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cb4160320 con 0x7f4cb4157d40 2024-04-25T10:02:34.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.397+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cb4073990 con 0x7f4cb4157d40 2024-04-25T10:02:34.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.397+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4cac005d10 con 0x7f4cb4157d40 2024-04-25T10:02:34.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.398+0000 7f4cabe00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c9406b500 0x7f4c9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.398+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f4cac08a690 con 0x7f4cb4157d40 2024-04-25T10:02:34.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.398+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c9406b500 0x7f4c9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.399+0000 7f4cb3e00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c9406b500 0x7f4c9406d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4032000 tx=0x7f4ca402d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.403+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4cac059130 con 0x7f4cb4157d40 2024-04-25T10:02:34.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.595+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4cb40d16c0 con 0x7f4cb4157d40 2024-04-25T10:02:34.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.596+0000 7f4cabe00640 1 -- 172.21.15.165:0/2048768042 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4cac058a30 con 0x7f4cb4157d40 2024-04-25T10:02:34.601 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:34.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.598+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c9406b500 msgr2=0x7f4c9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.598+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c9406b500 0x7f4c9406d9c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4032000 tx=0x7f4ca402d000 comp rx=0 tx=0).stop 2024-04-25T10:02:34.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.598+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 msgr2=0x7f4cb41597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 0x7f4cb41597b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4cb408c1f0 tx=0x7f4cac00b480 comp rx=0 tx=0).stop 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 shutdown_connections 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c9406b500 0x7f4c9406d9c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4cb4157d40 0x7f4cb41597b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4cb415f980 0x7f4cb40cdb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 --2- 172.21.15.165:0/2048768042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c980039e0 0x7f4cb41f81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 >> 172.21.15.165:0/2048768042 conn(0x7f4cb40d0610 msgr2=0x7f4cb415cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 shutdown_connections 2024-04-25T10:02:34.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.599+0000 7f4cb9a00640 1 -- 172.21.15.165:0/2048768042 wait complete. 2024-04-25T10:02:34.620 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.807+0000 7f6e98a00640 1 Processor -- start 2024-04-25T10:02:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.807+0000 7f6e98a00640 1 -- start start 2024-04-25T10:02:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.808+0000 7f6e98a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e9406bff0 0x7f6e940b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.808+0000 7f6e98a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9415e110 con 0x7f6e940b5970 2024-04-25T10:02:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e98a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e940b2fd0 con 0x7f6e9415dd30 2024-04-25T10:02:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e98a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9406d000 con 0x7f6e9406bff0 2024-04-25T10:02:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e93200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6e940b5970 0x7f6e940d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58192/0 (socket says 172.21.15.165:58192) 2024-04-25T10:02:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e93200640 1 -- 172.21.15.165:0/2258791823 learned_addr learned my addr 172.21.15.165:0/2258791823 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e89e00640 1 --2- 172.21.15.165:0/2258791823 >> v2:172.21.15.172:3300/0 conn(0x7f6e9406bff0 0x7f6e940b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 573380737 0 0) 0x7f6e9415e110 con 0x7f6e940b5970 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.809+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e78003520 con 0x7f6e940b5970 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247614889 0 0) 0x7f6e78003520 con 0x7f6e940b5970 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 >> v2:172.21.15.172:3300/0 conn(0x7f6e9406bff0 msgr2=0x7f6e940b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 --2- 172.21.15.165:0/2258791823 >> v2:172.21.15.172:3300/0 conn(0x7f6e9406bff0 0x7f6e940b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 >> v1:172.21.15.172:6789/0 conn(0x7f6e9415dd30 legacy=0x7f6e940cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9406c480 con 0x7f6e940b5970 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6e8c0019f0 con 0x7f6e940b5970 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 >> v1:172.21.15.165:6789/0 conn(0x7f6e940b5970 legacy=0x7f6e940d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e78003a00 0x7f6e78003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e78004800 0x7f6e78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e7800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9406d000 con 0x7f6e780051c0 2024-04-25T10:02:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7800c0c0 con 0x7f6e78003a00 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7800c2c0 con 0x7f6e78004800 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e93200640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e7800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e92800640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e78003a00 0x7f6e78003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.810+0000 7f6e89e00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e78004800 0x7f6e78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e93200640 1 -- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e78004800 msgr2=0x7f6e78004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e93200640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e78004800 0x7f6e78004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e93200640 1 -- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e78003a00 msgr2=0x7f6e78003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e93200640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e78003a00 0x7f6e78003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e93200640 1 -- 172.21.15.165:0/2258791823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e78003520 con 0x7f6e780051c0 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e89e00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e78004800 0x7f6e78004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e92800640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e78003a00 0x7f6e78003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.811+0000 7f6e93200640 1 -- 172.21.15.165:0/2258791823 reap_dead start 2024-04-25T10:02:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.812+0000 7f6e93200640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e7800b9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00a7b0 tx=0x7f6e78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410fe8bcf2b30552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.812+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00a3d0 con 0x7f6e780051c0 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.812+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c02f070 con 0x7f6e780051c0 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.812+0000 7f6e91e00640 1 -- 172.21.15.165:0/2258791823 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c02bc30 con 0x7f6e780051c0 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.812+0000 7f6e98a00640 1 -- 172.21.15.165:0/2258791823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 msgr2=0x7f6e7800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.812+0000 7f6e98a00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e7800b9c0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00a7b0 tx=0x7f6e78005830 comp rx=0 tx=0).stop 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.813+0000 7f6e98a00640 1 -- 172.21.15.165:0/2258791823 shutdown_connections 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.813+0000 7f6e98a00640 1 --2- 172.21.15.165:0/2258791823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e7800b9c0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.813+0000 7f6e98a00640 1 -- 172.21.15.165:0/2258791823 >> 172.21.15.165:0/2258791823 conn(0x7f6e940d0610 msgr2=0x7f6e940d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.813+0000 7f6e98a00640 1 -- 172.21.15.165:0/2258791823 shutdown_connections 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.813+0000 7f6e98a00640 1 -- 172.21.15.165:0/2258791823 wait complete. 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.814+0000 7f6e98a00640 1 Processor -- start 2024-04-25T10:02:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.815+0000 7f6e98a00640 1 -- start start 2024-04-25T10:02:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.815+0000 7f6e98a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e94156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.815+0000 7f6e98a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e94156fa0 0x7f6e940b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.815+0000 7f6e98a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e941569e0 0x7f6e940b4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e98a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9406cd20 con 0x7f6e780051c0 2024-04-25T10:02:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e98a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9406ca40 con 0x7f6e94156fa0 2024-04-25T10:02:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e98a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940b5430 con 0x7f6e941569e0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e94156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e94156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50758/0 (socket says 172.21.15.165:50758) 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 -- 172.21.15.165:0/4128763879 learned_addr learned my addr 172.21.15.165:0/4128763879 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e89e00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e94156fa0 0x7f6e940b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 -- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e941569e0 msgr2=0x7f6e940b4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e941569e0 0x7f6e940b4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 -- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e94156fa0 msgr2=0x7f6e940b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e94156fa0 0x7f6e940b4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 -- 172.21.15.165:0/4128763879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940cae60 con 0x7f6e780051c0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.816+0000 7f6e92800640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e94156200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400d7d0 tx=0x7f6e8400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.817+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84005270 con 0x7f6e780051c0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.817+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e84017070 con 0x7f6e780051c0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.817+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8400b650 con 0x7f6e780051c0 2024-04-25T10:02:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.817+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e940cb140 con 0x7f6e780051c0 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.817+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e940cb700 con 0x7f6e780051c0 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.818+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e94073990 con 0x7f6e780051c0 2024-04-25T10:02:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.819+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e8400b7f0 con 0x7f6e780051c0 2024-04-25T10:02:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.820+0000 7f6e8b400640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e7006b500 0x7f6e7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.820+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f6e8408a300 con 0x7f6e780051c0 2024-04-25T10:02:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.820+0000 7f6e89e00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e7006b500 0x7f6e7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.821+0000 7f6e89e00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e7006b500 0x7f6e7006d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e940cc620 tx=0x7f6e7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:34.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:34.824+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e84058da0 con 0x7f6e780051c0 2024-04-25T10:02:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.019+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6e940d16c0 con 0x7f6e780051c0 2024-04-25T10:02:35.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.020+0000 7f6e8b400640 1 -- 172.21.15.165:0/4128763879 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6e84001230 con 0x7f6e780051c0 2024-04-25T10:02:35.025 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e7006b500 msgr2=0x7f6e7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e7006b500 0x7f6e7006d9c0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6e940cc620 tx=0x7f6e7c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 msgr2=0x7f6e94156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e94156200 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400d7d0 tx=0x7f6e8400dc90 comp rx=0 tx=0).stop 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 shutdown_connections 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e7006b500 0x7f6e7006d9c0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e941569e0 0x7f6e940b4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e94156fa0 0x7f6e940b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 --2- 172.21.15.165:0/4128763879 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e780051c0 0x7f6e94156200 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 >> 172.21.15.165:0/4128763879 conn(0x7f6e940d0610 msgr2=0x7f6e940b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 shutdown_connections 2024-04-25T10:02:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.023+0000 7f6e98a00640 1 -- 172.21.15.165:0/4128763879 wait complete. 2024-04-25T10:02:35.046 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:35.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.237+0000 7fb332800640 1 Processor -- start 2024-04-25T10:02:35.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.237+0000 7fb332800640 1 -- start start 2024-04-25T10:02:35.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb332800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb32c0ce920 0x7fb32c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb332800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb32c06d290 0x7fb32c06d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb332800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb32c144b30 con 0x7fb32c144750 2024-04-25T10:02:35.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb332800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0712d0 con 0x7fb32c06d290 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb332800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb32c0715b0 con 0x7fb32c0ce920 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb32be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb32c06d290 0x7fb32c06d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb32be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb32c06d290 0x7fb32c06d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50764/0 (socket says 172.21.15.165:50764) 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb32be00640 1 -- 172.21.15.165:0/53553854 learned_addr learned my addr 172.21.15.165:0/53553854 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.239+0000 7fb32aa00640 1 --2- 172.21.15.165:0/53553854 >> v2:172.21.15.172:3300/0 conn(0x7fb32c0ce920 0x7fb32c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.240+0000 7fb32be00640 1 -- 172.21.15.165:0/53553854 >> v2:172.21.15.172:3300/0 conn(0x7fb32c0ce920 msgr2=0x7fb32c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.240+0000 7fb32be00640 1 --2- 172.21.15.165:0/53553854 >> v2:172.21.15.172:3300/0 conn(0x7fb32c0ce920 0x7fb32c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.240+0000 7fb32be00640 1 -- 172.21.15.165:0/53553854 >> v1:172.21.15.172:6789/0 conn(0x7fb32c144750 legacy=0x7fb32c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.240+0000 7fb32be00640 1 -- 172.21.15.165:0/53553854 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c071890 con 0x7fb32c06d290 2024-04-25T10:02:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.241+0000 7fb32be00640 1 --2- 172.21.15.165:0/53553854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c06d290 0x7fb32c06d6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb32c1606b0 tx=0x7fb32402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beae20fc39398f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.241+0000 7fb32a000640 1 -- 172.21.15.165:0/53553854 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324036070 con 0x7fb32c06d290 2024-04-25T10:02:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.241+0000 7fb32a000640 1 -- 172.21.15.165:0/53553854 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324031600 con 0x7fb32c06d290 2024-04-25T10:02:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.241+0000 7fb32a000640 1 -- 172.21.15.165:0/53553854 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3240319d0 con 0x7fb32c06d290 2024-04-25T10:02:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.241+0000 7fb332800640 1 -- 172.21.15.165:0/53553854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c06d290 msgr2=0x7fb32c06d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.241+0000 7fb332800640 1 --2- 172.21.15.165:0/53553854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c06d290 0x7fb32c06d6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb32c1606b0 tx=0x7fb32402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.242+0000 7fb332800640 1 -- 172.21.15.165:0/53553854 shutdown_connections 2024-04-25T10:02:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.242+0000 7fb332800640 1 --2- 172.21.15.165:0/53553854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c06d290 0x7fb32c06d6c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.242+0000 7fb332800640 1 --2- 172.21.15.165:0/53553854 >> v2:172.21.15.172:3300/0 conn(0x7fb32c0ce920 0x7fb32c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.242+0000 7fb332800640 1 -- 172.21.15.165:0/53553854 >> 172.21.15.165:0/53553854 conn(0x7fb32c06abe0 msgr2=0x7fb32c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.242+0000 7fb332800640 1 -- 172.21.15.165:0/53553854 shutdown_connections 2024-04-25T10:02:35.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.242+0000 7fb332800640 1 -- 172.21.15.165:0/53553854 wait complete. 2024-04-25T10:02:35.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.243+0000 7fb332800640 1 Processor -- start 2024-04-25T10:02:35.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 -- start start 2024-04-25T10:02:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32c06d290 0x7fb32c1e6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32c0ce920 0x7fb32c13d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 0x7fb32c1e6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c06f020 con 0x7fb32c144750 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c06ece0 con 0x7fb32c0ce920 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.244+0000 7fb332800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32c145450 con 0x7fb32c06d290 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 0x7fb32c1e6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 0x7fb32c1e6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50778/0 (socket says 172.21.15.165:50778) 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 -- 172.21.15.165:0/3377613471 learned_addr learned my addr 172.21.15.165:0/3377613471 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32b400640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32c06d290 0x7fb32c1e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32aa00640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32c0ce920 0x7fb32c13d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 -- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32c06d290 msgr2=0x7fb32c1e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32c06d290 0x7fb32c1e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 -- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32c0ce920 msgr2=0x7fb32c13d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32c0ce920 0x7fb32c13d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.245+0000 7fb32be00640 1 -- 172.21.15.165:0/3377613471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32c1e65a0 con 0x7fb32c144750 2024-04-25T10:02:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.246+0000 7fb32be00640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 0x7fb32c1e6940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb324009f40 tx=0x7fb324004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.246+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324036050 con 0x7fb32c144750 2024-04-25T10:02:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.246+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb324042070 con 0x7fb32c144750 2024-04-25T10:02:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.246+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb324009510 con 0x7fb32c144750 2024-04-25T10:02:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.246+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb32c1e4fc0 con 0x7fb32c144750 2024-04-25T10:02:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.246+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb32c1e5580 con 0x7fb32c144750 2024-04-25T10:02:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.247+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb32c073990 con 0x7fb32c144750 2024-04-25T10:02:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.248+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb324007bf0 con 0x7fb32c144750 2024-04-25T10:02:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.249+0000 7fb323400640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30c06b520 0x7fb30c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.249+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb3240ab3b0 con 0x7fb32c144750 2024-04-25T10:02:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.249+0000 7fb32b400640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30c06b520 0x7fb30c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.250+0000 7fb32b400640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30c06b520 0x7fb30c06d9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb32c15df30 tx=0x7fb31c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.253+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb324045360 con 0x7fb32c144750 2024-04-25T10:02:35.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.485+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb32c0ccde0 con 0x7fb32c144750 2024-04-25T10:02:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.486+0000 7fb323400640 1 -- 172.21.15.165:0/3377613471 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fb324079ed0 con 0x7fb32c144750 2024-04-25T10:02:35.490 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:35.491 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:02:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.488+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30c06b520 msgr2=0x7fb30c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30c06b520 0x7fb30c06d9e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fb32c15df30 tx=0x7fb31c009040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 msgr2=0x7fb32c1e6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 0x7fb32c1e6940 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb324009f40 tx=0x7fb324004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 shutdown_connections 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30c06b520 0x7fb30c06d9e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32c144750 0x7fb32c1e6940 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32c0ce920 0x7fb32c13d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 --2- 172.21.15.165:0/3377613471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32c06d290 0x7fb32c1e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 >> 172.21.15.165:0/3377613471 conn(0x7fb32c06abe0 msgr2=0x7fb32c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 shutdown_connections 2024-04-25T10:02:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.489+0000 7fb332800640 1 -- 172.21.15.165:0/3377613471 wait complete. 2024-04-25T10:02:35.513 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:35.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.700+0000 7fba87e00640 1 Processor -- start 2024-04-25T10:02:35.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.700+0000 7fba87e00640 1 -- start start 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.702+0000 7fba87e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fba800b3af0 0x7fba800c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.702+0000 7fba87e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba800cf340 0x7fba800cbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.702+0000 7fba87e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba800c7380 con 0x7fba8015b8f0 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.702+0000 7fba87e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800d03c0 con 0x7fba800b3af0 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.702+0000 7fba87e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba800c7150 con 0x7fba800cf340 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.702+0000 7fba84a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fba800b3af0 0x7fba800c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba84a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fba800b3af0 0x7fba800c6c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50784/0 (socket says 172.21.15.165:50784) 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba84a00640 1 -- 172.21.15.165:0/4291886756 learned_addr learned my addr 172.21.15.165:0/4291886756 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:35.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba85e00640 1 --2- 172.21.15.165:0/4291886756 >> v2:172.21.15.172:3300/0 conn(0x7fba800cf340 0x7fba800cbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba84a00640 1 -- 172.21.15.165:0/4291886756 >> v2:172.21.15.172:3300/0 conn(0x7fba800cf340 msgr2=0x7fba800cbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba84a00640 1 --2- 172.21.15.165:0/4291886756 >> v2:172.21.15.172:3300/0 conn(0x7fba800cf340 0x7fba800cbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba84a00640 1 -- 172.21.15.165:0/4291886756 >> v1:172.21.15.172:6789/0 conn(0x7fba8015b8f0 legacy=0x7fba800b33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:35.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba84a00640 1 -- 172.21.15.165:0/4291886756 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800c7660 con 0x7fba800b3af0 2024-04-25T10:02:35.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.703+0000 7fba85e00640 1 --2- 172.21.15.165:0/4291886756 >> v2:172.21.15.172:3300/0 conn(0x7fba800cf340 0x7fba800cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:35.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.704+0000 7fba84a00640 1 --2- 172.21.15.165:0/4291886756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba800b3af0 0x7fba800c6c10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba800d1110 tx=0x7fba6c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b43229d341ea17aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.704+0000 7fba7fe00640 1 -- 172.21.15.165:0/4291886756 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c037070 con 0x7fba800b3af0 2024-04-25T10:02:35.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.704+0000 7fba7fe00640 1 -- 172.21.15.165:0/4291886756 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c032440 con 0x7fba800b3af0 2024-04-25T10:02:35.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.704+0000 7fba7fe00640 1 -- 172.21.15.165:0/4291886756 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c032810 con 0x7fba800b3af0 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 -- 172.21.15.165:0/4291886756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba800b3af0 msgr2=0x7fba800c6c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 --2- 172.21.15.165:0/4291886756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba800b3af0 0x7fba800c6c10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fba800d1110 tx=0x7fba6c02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 -- 172.21.15.165:0/4291886756 shutdown_connections 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 --2- 172.21.15.165:0/4291886756 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba800b3af0 0x7fba800c6c10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 --2- 172.21.15.165:0/4291886756 >> v2:172.21.15.172:3300/0 conn(0x7fba800cf340 0x7fba800cbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 -- 172.21.15.165:0/4291886756 >> 172.21.15.165:0/4291886756 conn(0x7fba800d1800 msgr2=0x7fba800d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 -- 172.21.15.165:0/4291886756 shutdown_connections 2024-04-25T10:02:35.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.705+0000 7fba87e00640 1 -- 172.21.15.165:0/4291886756 wait complete. 2024-04-25T10:02:35.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.707+0000 7fba87e00640 1 Processor -- start 2024-04-25T10:02:35.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.707+0000 7fba87e00640 1 -- start start 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.708+0000 7fba87e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba800b3af0 0x7fba80157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.708+0000 7fba87e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba800cf340 0x7fba800cb010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.708+0000 7fba87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 0x7fba800ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.708+0000 7fba87e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800c7c20 con 0x7fba8015b8f0 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.708+0000 7fba87e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba800c7f90 con 0x7fba800cf340 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.708+0000 7fba87e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba8015bcd0 con 0x7fba800b3af0 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 0x7fba800ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 0x7fba800ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50786/0 (socket says 172.21.15.165:50786) 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 -- 172.21.15.165:0/1723868245 learned_addr learned my addr 172.21.15.165:0/1723868245 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85400640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba800b3af0 0x7fba80157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba84a00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba800cf340 0x7fba800cb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 -- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba800b3af0 msgr2=0x7fba80157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba800b3af0 0x7fba80157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 -- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba800cf340 msgr2=0x7fba800cb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba800cf340 0x7fba800cb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 -- 172.21.15.165:0/1723868245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba800cd1f0 con 0x7fba8015b8f0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba84a00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba800cf340 0x7fba800cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.709+0000 7fba85e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 0x7fba800ce0b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba78009fa0 tx=0x7fba7800d560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.710+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba7801a070 con 0x7fba8015b8f0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.710+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba78003bb0 con 0x7fba8015b8f0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.710+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba800c95c0 con 0x7fba8015b8f0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.710+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba78004030 con 0x7fba8015b8f0 2024-04-25T10:02:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.710+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba800c9b80 con 0x7fba8015b8f0 2024-04-25T10:02:35.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.711+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba800b3600 con 0x7fba8015b8f0 2024-04-25T10:02:35.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.712+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba7801d070 con 0x7fba8015b8f0 2024-04-25T10:02:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.713+0000 7fba7d600640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba6006b4d0 0x7fba6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.713+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fba7808a8c0 con 0x7fba8015b8f0 2024-04-25T10:02:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.713+0000 7fba85400640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba6006b4d0 0x7fba6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:35.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.714+0000 7fba85400640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba6006b4d0 0x7fba6006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba74006fd0 tx=0x7fba74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:35.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.717+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba800b3600 con 0x7fba8015b8f0 2024-04-25T10:02:35.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.916+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba80073990 con 0x7fba8015b8f0 2024-04-25T10:02:35.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.918+0000 7fba7d600640 1 -- 172.21.15.165:0/1723868245 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7fba78010070 con 0x7fba8015b8f0 2024-04-25T10:02:35.923 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:35.923 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.920+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba6006b4d0 msgr2=0x7fba6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.920+0000 7fba87e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba6006b4d0 0x7fba6006d990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fba74006fd0 tx=0x7fba74008040 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.920+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 msgr2=0x7fba800ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.920+0000 7fba87e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 0x7fba800ce0b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fba78009fa0 tx=0x7fba7800d560 comp rx=0 tx=0).stop 2024-04-25T10:02:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 shutdown_connections 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba6006b4d0 0x7fba6006d990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba8015b8f0 0x7fba800ce0b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba800cf340 0x7fba800cb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 --2- 172.21.15.165:0/1723868245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba800b3af0 0x7fba80157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 >> 172.21.15.165:0/1723868245 conn(0x7fba800d1800 msgr2=0x7fba800b1860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 shutdown_connections 2024-04-25T10:02:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:35.921+0000 7fba87e00640 1 -- 172.21.15.165:0/1723868245 wait complete. 2024-04-25T10:02:35.944 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:35.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.144+0000 7f3d29c00640 1 Processor -- start 2024-04-25T10:02:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.144+0000 7f3d29c00640 1 -- start start 2024-04-25T10:02:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.145+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d2415dd30 0x7f3d240cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.145+0000 7f3d29c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3d2406bff0 0x7f3d240b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d29c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d2415e110 con 0x7f3d240b5970 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d29c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d2406d000 con 0x7f3d2406bff0 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d29c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240b2fd0 con 0x7f3d2415dd30 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3d2406bff0 0x7f3d240b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3d2406bff0 0x7f3d240b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50800/0 (socket says 172.21.15.165:50800) 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 -- 172.21.15.165:0/2589506590 learned_addr learned my addr 172.21.15.165:0/2589506590 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 -- 172.21.15.165:0/2589506590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d2415dd30 msgr2=0x7f3d240cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23e00640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d2415dd30 0x7f3d240cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d2415dd30 0x7f3d240cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 -- 172.21.15.165:0/2589506590 >> v1:172.21.15.172:6789/0 conn(0x7f3d240b5970 legacy=0x7f3d24161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23400640 1 -- 172.21.15.165:0/2589506590 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2406c480 con 0x7f3d2406bff0 2024-04-25T10:02:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.146+0000 7f3d23e00640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d2415dd30 0x7f3d240cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.147+0000 7f3d23400640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d2406bff0 0x7f3d240b5430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009870 tx=0x7f3d1002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f9bbae6d405b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d22a00640 1 -- 172.21.15.165:0/2589506590 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10037070 con 0x7f3d2406bff0 2024-04-25T10:02:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d22a00640 1 -- 172.21.15.165:0/2589506590 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d10032440 con 0x7f3d2406bff0 2024-04-25T10:02:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d22a00640 1 -- 172.21.15.165:0/2589506590 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d10032810 con 0x7f3d2406bff0 2024-04-25T10:02:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d29c00640 1 -- 172.21.15.165:0/2589506590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d2406bff0 msgr2=0x7f3d240b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d29c00640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d2406bff0 0x7f3d240b5430 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f3d10009870 tx=0x7f3d1002a200 comp rx=0 tx=0).stop 2024-04-25T10:02:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d29c00640 1 -- 172.21.15.165:0/2589506590 shutdown_connections 2024-04-25T10:02:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d29c00640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d2406bff0 0x7f3d240b5430 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d29c00640 1 --2- 172.21.15.165:0/2589506590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d2415dd30 0x7f3d240cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.148+0000 7f3d29c00640 1 -- 172.21.15.165:0/2589506590 >> 172.21.15.165:0/2589506590 conn(0x7f3d240d0610 msgr2=0x7f3d240d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.149+0000 7f3d29c00640 1 -- 172.21.15.165:0/2589506590 shutdown_connections 2024-04-25T10:02:36.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.149+0000 7f3d29c00640 1 -- 172.21.15.165:0/2589506590 wait complete. 2024-04-25T10:02:36.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.150+0000 7f3d29c00640 1 Processor -- start 2024-04-25T10:02:36.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.150+0000 7f3d29c00640 1 -- start start 2024-04-25T10:02:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d240b5970 0x7f3d24158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d2415dd30 0x7f3d241f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 0x7f3d241f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2406cd20 con 0x7f3d241f6cf0 2024-04-25T10:02:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d2406ca40 con 0x7f3d2415dd30 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d240cf110 con 0x7f3d240b5970 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 0x7f3d241f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 0x7f3d241f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50802/0 (socket says 172.21.15.165:50802) 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.151+0000 7f3d29200640 1 -- 172.21.15.165:0/3665880398 learned_addr learned my addr 172.21.15.165:0/3665880398 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d23e00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d240b5970 0x7f3d24158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d23400640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d2415dd30 0x7f3d241f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29200640 1 -- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d240b5970 msgr2=0x7f3d24158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29200640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d240b5970 0x7f3d24158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29200640 1 -- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d2415dd30 msgr2=0x7f3d241f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29200640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d2415dd30 0x7f3d241f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29200640 1 -- 172.21.15.165:0/3665880398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d24159540 con 0x7f3d241f6cf0 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d23400640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d2415dd30 0x7f3d241f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d23e00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d240b5970 0x7f3d24158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29200640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 0x7f3d241f7180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c410 tx=0x7f3d1c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c011790 con 0x7f3d241f6cf0 2024-04-25T10:02:36.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d1c017070 con 0x7f3d241f6cf0 2024-04-25T10:02:36.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d24159820 con 0x7f3d241f6cf0 2024-04-25T10:02:36.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.152+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d1c011bb0 con 0x7f3d241f6cf0 2024-04-25T10:02:36.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.153+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d24159de0 con 0x7f3d241f6cf0 2024-04-25T10:02:36.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.154+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d24073990 con 0x7f3d241f6cf0 2024-04-25T10:02:36.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.155+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d1c01c460 con 0x7f3d241f6cf0 2024-04-25T10:02:36.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.155+0000 7f3d1be00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d0406b4d0 0x7f3d0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.156+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3d1c08a0b0 con 0x7f3d241f6cf0 2024-04-25T10:02:36.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.156+0000 7f3d23e00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d0406b4d0 0x7f3d0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.156+0000 7f3d23e00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d0406b4d0 0x7f3d0406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d240cd6a0 tx=0x7f3d140074b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.160+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d1c021020 con 0x7f3d241f6cf0 2024-04-25T10:02:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.339+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3d240d16c0 con 0x7f3d241f6cf0 2024-04-25T10:02:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.340+0000 7f3d1be00640 1 -- 172.21.15.165:0/3665880398 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v24) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3d1c058bf0 con 0x7f3d241f6cf0 2024-04-25T10:02:36.344 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:36.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.342+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d0406b4d0 msgr2=0x7f3d0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.342+0000 7f3d29c00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d0406b4d0 0x7f3d0406d990 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3d240cd6a0 tx=0x7f3d140074b0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.342+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 msgr2=0x7f3d241f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.342+0000 7f3d29c00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 0x7f3d241f7180 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3d1c00c410 tx=0x7f3d1c00c8d0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 shutdown_connections 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d0406b4d0 0x7f3d0406d990 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d241f6cf0 0x7f3d241f7180 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d2415dd30 0x7f3d241f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 --2- 172.21.15.165:0/3665880398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d240b5970 0x7f3d24158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 >> 172.21.15.165:0/3665880398 conn(0x7f3d240d0610 msgr2=0x7f3d240b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 shutdown_connections 2024-04-25T10:02:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.343+0000 7f3d29c00640 1 -- 172.21.15.165:0/3665880398 wait complete. 2024-04-25T10:02:36.365 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:36.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.550+0000 7f282d400640 1 Processor -- start 2024-04-25T10:02:36.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.551+0000 7f282d400640 1 -- start start 2024-04-25T10:02:36.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.552+0000 7f282d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f282815b710 0x7f28280cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.552+0000 7f282d400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f28280ac8b0 0x7f28280acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.552+0000 7f282d400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f282806f0e0 con 0x7f282806ec50 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.552+0000 7f282d400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f282815e490 con 0x7f28280ac8b0 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.552+0000 7f282d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28280cfd80 con 0x7f282815b710 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f28280ac8b0 0x7f28280acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f28280ac8b0 0x7f28280acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50806/0 (socket says 172.21.15.165:50806) 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 -- 172.21.15.165:0/1896805428 learned_addr learned my addr 172.21.15.165:0/1896805428 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2826800640 1 --2- 172.21.15.165:0/1896805428 >> v2:172.21.15.172:3300/0 conn(0x7f282815b710 0x7f28280cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 -- 172.21.15.165:0/1896805428 >> v2:172.21.15.172:3300/0 conn(0x7f282815b710 msgr2=0x7f28280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 --2- 172.21.15.165:0/1896805428 >> v2:172.21.15.172:3300/0 conn(0x7f282815b710 0x7f28280cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 -- 172.21.15.165:0/1896805428 >> v1:172.21.15.172:6789/0 conn(0x7f282806ec50 legacy=0x7f282815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2827200640 1 -- 172.21.15.165:0/1896805428 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280d0060 con 0x7f28280ac8b0 2024-04-25T10:02:36.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.553+0000 7f2826800640 1 --2- 172.21.15.165:0/1896805428 >> v2:172.21.15.172:3300/0 conn(0x7f282815b710 0x7f28280cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.554+0000 7f2827200640 1 --2- 172.21.15.165:0/1896805428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28280acd00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f282806b950 tx=0x7f282002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e6f3de458e42a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f2825e00640 1 -- 172.21.15.165:0/1896805428 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2820036070 con 0x7f28280ac8b0 2024-04-25T10:02:36.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f2825e00640 1 -- 172.21.15.165:0/1896805428 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2820031600 con 0x7f28280ac8b0 2024-04-25T10:02:36.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f2825e00640 1 -- 172.21.15.165:0/1896805428 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28200319d0 con 0x7f28280ac8b0 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 -- 172.21.15.165:0/1896805428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 msgr2=0x7f28280acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 --2- 172.21.15.165:0/1896805428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28280acd00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f282806b950 tx=0x7f282002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 -- 172.21.15.165:0/1896805428 shutdown_connections 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 --2- 172.21.15.165:0/1896805428 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28280acd00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 --2- 172.21.15.165:0/1896805428 >> v2:172.21.15.172:3300/0 conn(0x7f282815b710 0x7f28280cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 -- 172.21.15.165:0/1896805428 >> 172.21.15.165:0/1896805428 conn(0x7f282806abe0 msgr2=0x7f282806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 -- 172.21.15.165:0/1896805428 shutdown_connections 2024-04-25T10:02:36.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.555+0000 7f282d400640 1 -- 172.21.15.165:0/1896805428 wait complete. 2024-04-25T10:02:36.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.557+0000 7f282d400640 1 Processor -- start 2024-04-25T10:02:36.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.557+0000 7f282d400640 1 -- start start 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.558+0000 7f282d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f282806ec50 0x7f28280c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.558+0000 7f282d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28281f2970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.558+0000 7f282d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f282815b710 0x7f28281499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.558+0000 7f282d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282815bc70 con 0x7f28280ac8b0 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.558+0000 7f282d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f282815baf0 con 0x7f282806ec50 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.558+0000 7f282d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f282815f030 con 0x7f282815b710 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28281f2970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28281f2970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50818/0 (socket says 172.21.15.165:50818) 2024-04-25T10:02:36.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 -- 172.21.15.165:0/544536058 learned_addr learned my addr 172.21.15.165:0/544536058 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f2826800640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f282806ec50 0x7f28280c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f2827200640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f282815b710 0x7f28281499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 -- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f282815b710 msgr2=0x7f28281499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f282815b710 0x7f28281499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 -- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f282806ec50 msgr2=0x7f28280c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f282806ec50 0x7f28280c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 -- 172.21.15.165:0/544536058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28280c4c10 con 0x7f28280ac8b0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f2826800640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f282806ec50 0x7f28280c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.559+0000 7f281fe00640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28281f2970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f281000c430 tx=0x7f281000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.560+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810011790 con 0x7f28280ac8b0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.560+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2810017070 con 0x7f28280ac8b0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.560+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28280b0440 con 0x7f28280ac8b0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.560+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2810011bb0 con 0x7f28280ac8b0 2024-04-25T10:02:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.560+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28280b0a00 con 0x7f28280ac8b0 2024-04-25T10:02:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.561+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2828073a00 con 0x7f28280ac8b0 2024-04-25T10:02:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.562+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f281001c460 con 0x7f28280ac8b0 2024-04-25T10:02:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.563+0000 7f281ea00640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f280406b4d0 0x7f280406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.563+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f281008a410 con 0x7f28280ac8b0 2024-04-25T10:02:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.563+0000 7f2826800640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f280406b4d0 0x7f280406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.564+0000 7f2826800640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f280406b4d0 0x7f280406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2818006fd0 tx=0x7f2818008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.567+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2810058f40 con 0x7f28280ac8b0 2024-04-25T10:02:36.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.744+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28280ae370 con 0x7f28280ac8b0 2024-04-25T10:02:36.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.745+0000 7f281ea00640 1 -- 172.21.15.165:0/544536058 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v24) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2810058840 con 0x7f28280ac8b0 2024-04-25T10:02:36.750 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f280406b4d0 msgr2=0x7f280406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f280406b4d0 0x7f280406d990 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2818006fd0 tx=0x7f2818008040 comp rx=0 tx=0).stop 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 msgr2=0x7f28281f2970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28281f2970 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f281000c430 tx=0x7f281000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 shutdown_connections 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f280406b4d0 0x7f280406d990 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f282815b710 0x7f28281499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28280ac8b0 0x7f28281f2970 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 --2- 172.21.15.165:0/544536058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f282806ec50 0x7f28280c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 >> 172.21.15.165:0/544536058 conn(0x7f282806abe0 msgr2=0x7f2828160990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 shutdown_connections 2024-04-25T10:02:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.748+0000 7f282d400640 1 -- 172.21.15.165:0/544536058 wait complete. 2024-04-25T10:02:36.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.960+0000 7fd43fe00640 1 Processor -- start 2024-04-25T10:02:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.960+0000 7fd43fe00640 1 -- start start 2024-04-25T10:02:36.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.961+0000 7fd43fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd44015e720 0x7fd4400b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4400b0b80 con 0x7fd44015b7a0 2024-04-25T10:02:36.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4400cc800 con 0x7fd44015b3c0 2024-04-25T10:02:36.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43fe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4400cd650 con 0x7fd44015e720 2024-04-25T10:02:36.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd436000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd44015b7a0 0x7fd44015e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:46212/0 (socket says 172.21.15.165:46212) 2024-04-25T10:02:36.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd436000640 1 -- 172.21.15.165:0/2453567849 learned_addr learned my addr 172.21.15.165:0/2453567849 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43f400640 1 --2- 172.21.15.165:0/2453567849 >> v2:172.21.15.165:3300/0 conn(0x7fd44015e720 0x7fd4400b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43e000640 1 -- 172.21.15.165:0/2453567849 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3418673933 0 0) 0x7fd428001110 con 0x7fd44015b7a0 2024-04-25T10:02:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43e000640 1 -- 172.21.15.165:0/2453567849 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd424003520 con 0x7fd44015b7a0 2024-04-25T10:02:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43f400640 1 -- 172.21.15.165:0/2453567849 >> v1:172.21.15.172:6789/0 conn(0x7fd44015b3c0 legacy=0x7fd4400cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43f400640 1 -- 172.21.15.165:0/2453567849 >> v1:172.21.15.165:6789/0 conn(0x7fd44015b7a0 legacy=0x7fd44015e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.962+0000 7fd43f400640 1 -- 172.21.15.165:0/2453567849 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4400ccae0 con 0x7fd44015e720 2024-04-25T10:02:36.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.963+0000 7fd43f400640 1 --2- 172.21.15.165:0/2453567849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015e720 0x7fd4400b0f60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd438009d90 tx=0x7fd43802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9c706390bcbf85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43e000640 1 -- 172.21.15.165:0/2453567849 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438007740 con 0x7fd44015e720 2024-04-25T10:02:36.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43e000640 1 -- 172.21.15.165:0/2453567849 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4380078e0 con 0x7fd44015e720 2024-04-25T10:02:36.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43e000640 1 -- 172.21.15.165:0/2453567849 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd438007c90 con 0x7fd44015e720 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 -- 172.21.15.165:0/2453567849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015e720 msgr2=0x7fd4400b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 --2- 172.21.15.165:0/2453567849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015e720 0x7fd4400b0f60 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fd438009d90 tx=0x7fd43802a660 comp rx=0 tx=0).stop 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 -- 172.21.15.165:0/2453567849 shutdown_connections 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 --2- 172.21.15.165:0/2453567849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015e720 0x7fd4400b0f60 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 -- 172.21.15.165:0/2453567849 >> 172.21.15.165:0/2453567849 conn(0x7fd440061cc0 msgr2=0x7fd44006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 -- 172.21.15.165:0/2453567849 shutdown_connections 2024-04-25T10:02:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.964+0000 7fd43fe00640 1 -- 172.21.15.165:0/2453567849 wait complete. 2024-04-25T10:02:36.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.966+0000 7fd43fe00640 1 Processor -- start 2024-04-25T10:02:36.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.966+0000 7fd43fe00640 1 -- start start 2024-04-25T10:02:36.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 0x7fd44015c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd44015b7a0 0x7fd440158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd44015e720 0x7fd440156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400ce370 con 0x7fd44015b3c0 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400ce1f0 con 0x7fd44015b7a0 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4400cc150 con 0x7fd44015e720 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 0x7fd44015c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 0x7fd44015c9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45558/0 (socket says 172.21.15.165:45558) 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.967+0000 7fd43ea00640 1 -- 172.21.15.165:0/1105881951 learned_addr learned my addr 172.21.15.165:0/1105881951 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd436000640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd44015b7a0 0x7fd440158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43f400640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd44015e720 0x7fd440156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43ea00640 1 -- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd44015e720 msgr2=0x7fd440156a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43ea00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd44015e720 0x7fd440156a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43ea00640 1 -- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd44015b7a0 msgr2=0x7fd440158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43ea00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd44015b7a0 0x7fd440158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43ea00640 1 -- 172.21.15.165:0/1105881951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd440155c60 con 0x7fd44015b3c0 2024-04-25T10:02:36.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd43ea00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 0x7fd44015c9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c450 tx=0x7fd43000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4300052d0 con 0x7fd44015b3c0 2024-04-25T10:02:36.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.968+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd430005470 con 0x7fd44015b3c0 2024-04-25T10:02:36.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.969+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd430012870 con 0x7fd44015b3c0 2024-04-25T10:02:36.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.969+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd440155f40 con 0x7fd44015b3c0 2024-04-25T10:02:36.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.969+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd440156500 con 0x7fd44015b3c0 2024-04-25T10:02:36.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.970+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd440156700 con 0x7fd44015b3c0 2024-04-25T10:02:36.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.971+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd430012a10 con 0x7fd44015b3c0 2024-04-25T10:02:36.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.971+0000 7fd437400640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd41c06b4d0 0x7fd41c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:36.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.972+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fd43008a220 con 0x7fd44015b3c0 2024-04-25T10:02:36.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.972+0000 7fd436000640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd41c06b4d0 0x7fd41c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:36.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.972+0000 7fd436000640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd41c06b4d0 0x7fd41c06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd440071fa0 tx=0x7fd428009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:36.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:36.976+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd430058cc0 con 0x7fd44015b3c0 2024-04-25T10:02:37.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.172+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd440073990 con 0x7fd44015b3c0 2024-04-25T10:02:37.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.173+0000 7fd437400640 1 -- 172.21.15.165:0/1105881951 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v24) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd4300585c0 con 0x7fd44015b3c0 2024-04-25T10:02:37.178 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:37.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd41c06b4d0 msgr2=0x7fd41c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd41c06b4d0 0x7fd41c06d990 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fd440071fa0 tx=0x7fd428009040 comp rx=0 tx=0).stop 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 msgr2=0x7fd44015c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 0x7fd44015c9f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd43000c450 tx=0x7fd43000c910 comp rx=0 tx=0).stop 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 shutdown_connections 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd41c06b4d0 0x7fd41c06d990 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd44015e720 0x7fd440156a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd44015b7a0 0x7fd440158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 --2- 172.21.15.165:0/1105881951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd44015b3c0 0x7fd44015c9f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 >> 172.21.15.165:0/1105881951 conn(0x7fd440061cc0 msgr2=0x7fd440160300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 shutdown_connections 2024-04-25T10:02:37.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.176+0000 7fd43fe00640 1 -- 172.21.15.165:0/1105881951 wait complete. 2024-04-25T10:02:37.199 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:37.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.390+0000 7f3b54c00640 1 Processor -- start 2024-04-25T10:02:37.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.390+0000 7f3b54c00640 1 -- start start 2024-04-25T10:02:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b54c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3b50145140 0x7f3b500ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b54c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b500c5650 con 0x7f3b500ce950 2024-04-25T10:02:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b54c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b50145520 con 0x7f3b500c5200 2024-04-25T10:02:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b54c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b500ced90 con 0x7f3b50145140 2024-04-25T10:02:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b4ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3b500ce950 0x7f3b500c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:46228/0 (socket says 172.21.15.165:46228) 2024-04-25T10:02:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b4ea00640 1 -- 172.21.15.165:0/3347353976 learned_addr learned my addr 172.21.15.165:0/3347353976 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.392+0000 7f3b4f400640 1 --2- 172.21.15.165:0/3347353976 >> v2:172.21.15.172:3300/0 conn(0x7f3b50145140 0x7f3b500ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3236201625 0 0) 0x7f3b500c5650 con 0x7f3b500ce950 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b34003520 con 0x7f3b500ce950 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3752804632 0 0) 0x7f3b34003520 con 0x7f3b500ce950 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 >> v2:172.21.15.172:3300/0 conn(0x7f3b50145140 msgr2=0x7f3b500ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 --2- 172.21.15.165:0/3347353976 >> v2:172.21.15.172:3300/0 conn(0x7f3b50145140 0x7f3b500ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 >> v1:172.21.15.172:6789/0 conn(0x7f3b500c5200 legacy=0x7f3b5015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b50071270 con 0x7f3b500ce950 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3b3c001670 con 0x7f3b500ce950 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 >> v1:172.21.15.165:6789/0 conn(0x7f3b500ce950 legacy=0x7f3b500c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340039b0 0x7f3b34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34004800 0x7f3b34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340051c0 0x7f3b3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500ced90 con 0x7f3b340039b0 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3400c160 con 0x7f3b340051c0 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.393+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3400c360 con 0x7f3b34004800 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4f400640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340039b0 0x7f3b34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4fe00640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340051c0 0x7f3b3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4ea00640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34004800 0x7f3b34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4f400640 1 -- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34004800 msgr2=0x7f3b34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4f400640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34004800 0x7f3b34004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4f400640 1 -- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340051c0 msgr2=0x7f3b3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4f400640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340051c0 0x7f3b3400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4f400640 1 -- 172.21.15.165:0/3347353976 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b34003520 con 0x7f3b340039b0 2024-04-25T10:02:37.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.394+0000 7f3b4fe00640 1 -- 172.21.15.165:0/3347353976 reap_dead start 2024-04-25T10:02:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.395+0000 7f3b4f400640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340039b0 0x7f3b34003dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009ac0 tx=0x7f3b34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf9df722795b331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.395+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40036070 con 0x7f3b340039b0 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.395+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b40031400 con 0x7f3b340039b0 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.395+0000 7f3b4e000640 1 -- 172.21.15.165:0/3347353976 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b40009690 con 0x7f3b340039b0 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 -- 172.21.15.165:0/3347353976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340039b0 msgr2=0x7f3b34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340039b0 0x7f3b34003dd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3b40009ac0 tx=0x7f3b34005830 comp rx=0 tx=0).stop 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 -- 172.21.15.165:0/3347353976 shutdown_connections 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 --2- 172.21.15.165:0/3347353976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340039b0 0x7f3b34003dd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 -- 172.21.15.165:0/3347353976 >> 172.21.15.165:0/3347353976 conn(0x7f3b50061cc0 msgr2=0x7f3b5006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 -- 172.21.15.165:0/3347353976 shutdown_connections 2024-04-25T10:02:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.396+0000 7f3b54c00640 1 -- 172.21.15.165:0/3347353976 wait complete. 2024-04-25T10:02:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.398+0000 7f3b54c00640 1 Processor -- start 2024-04-25T10:02:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.398+0000 7f3b54c00640 1 -- start start 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340039b0 0x7f3b501ef550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b5013e8b0 0x7f3b5015db80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b54c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 0x7f3b5015e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b54c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b50071b10 con 0x7f3b5013ed30 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b54c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b50071830 con 0x7f3b340039b0 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b54c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c4af0 con 0x7f3b5013e8b0 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 0x7f3b5015e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 0x7f3b5015e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45584/0 (socket says 172.21.15.165:45584) 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.399+0000 7f3b4fe00640 1 -- 172.21.15.165:0/2105075792 learned_addr learned my addr 172.21.15.165:0/2105075792 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4ea00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b5013e8b0 0x7f3b5015db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4f400640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340039b0 0x7f3b501ef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4fe00640 1 -- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b5013e8b0 msgr2=0x7f3b5015db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4fe00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b5013e8b0 0x7f3b5015db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4fe00640 1 -- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340039b0 msgr2=0x7f3b501ef550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4fe00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340039b0 0x7f3b501ef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4fe00640 1 -- 172.21.15.165:0/2105075792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b500c73a0 con 0x7f3b5013ed30 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4ea00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b5013e8b0 0x7f3b5015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b4fe00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 0x7f3b5015e0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3b48007fd0 tx=0x7f3b4800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48019070 con 0x7f3b5013ed30 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b48006d60 con 0x7f3b5013ed30 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b48003b90 con 0x7f3b5013ed30 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b500c7680 con 0x7f3b5013ed30 2024-04-25T10:02:37.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.400+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b5015cf30 con 0x7f3b5013ed30 2024-04-25T10:02:37.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.401+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b50073990 con 0x7f3b5013ed30 2024-04-25T10:02:37.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.402+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b4801c070 con 0x7f3b5013ed30 2024-04-25T10:02:37.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.403+0000 7f3b47400640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b3006b4d0 0x7f3b3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.403+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f3b48089bc0 con 0x7f3b5013ed30 2024-04-25T10:02:37.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.404+0000 7f3b4f400640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b3006b4d0 0x7f3b3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.404+0000 7f3b4f400640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b3006b4d0 0x7f3b3006d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b40033000 tx=0x7f3b40008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.407+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b480586f0 con 0x7f3b5013ed30 2024-04-25T10:02:37.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.604+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f3b500b3d90 con 0x7f3b5013ed30 2024-04-25T10:02:37.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.606+0000 7f3b47400640 1 -- 172.21.15.165:0/2105075792 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v24) v1 ==== 103+0+11 (secure 0 0 0) 0x7f3b48057ff0 con 0x7f3b5013ed30 2024-04-25T10:02:37.610 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b3006b4d0 msgr2=0x7f3b3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b3006b4d0 0x7f3b3006d990 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f3b40033000 tx=0x7f3b40008030 comp rx=0 tx=0).stop 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 msgr2=0x7f3b5015e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 0x7f3b5015e0c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3b48007fd0 tx=0x7f3b4800ea00 comp rx=0 tx=0).stop 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 shutdown_connections 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b3006b4d0 0x7f3b3006d990 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b5013ed30 0x7f3b5015e0c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b5013e8b0 0x7f3b5015db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 --2- 172.21.15.165:0/2105075792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340039b0 0x7f3b501ef550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 >> 172.21.15.165:0/2105075792 conn(0x7f3b50061cc0 msgr2=0x7f3b5015beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 shutdown_connections 2024-04-25T10:02:37.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.608+0000 7f3b54c00640 1 -- 172.21.15.165:0/2105075792 wait complete. 2024-04-25T10:02:37.631 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:37.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.820+0000 7f7d13e00640 1 Processor -- start 2024-04-25T10:02:37.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.820+0000 7f7d13e00640 1 -- start start 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d13e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7d140b93c0 0x7f7d140cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d13e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7d140d04f0 0x7f7d140d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d13e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d14161df0 con 0x7f7d1415db90 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d13e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140d1f50 con 0x7f7d140d04f0 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d13e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d140d0940 con 0x7f7d140b93c0 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d12a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7d1415db90 0x7f7d141615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:46238/0 (socket says 172.21.15.165:46238) 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d12a00640 1 -- 172.21.15.165:0/524430443 learned_addr learned my addr 172.21.15.165:0/524430443 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.822+0000 7f7d13400640 1 --2- 172.21.15.165:0/524430443 >> v2:172.21.15.165:3300/0 conn(0x7f7d140d04f0 0x7f7d140d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.823+0000 7f7d0a000640 1 --2- 172.21.15.165:0/524430443 >> v2:172.21.15.172:3300/0 conn(0x7f7d140b93c0 0x7f7d140cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.823+0000 7f7d13400640 1 -- 172.21.15.165:0/524430443 >> v2:172.21.15.172:3300/0 conn(0x7f7d140b93c0 msgr2=0x7f7d140cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:37.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.823+0000 7f7d13400640 1 --2- 172.21.15.165:0/524430443 >> v2:172.21.15.172:3300/0 conn(0x7f7d140b93c0 0x7f7d140cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.823+0000 7f7d13400640 1 -- 172.21.15.165:0/524430443 >> v1:172.21.15.165:6789/0 conn(0x7f7d1415db90 legacy=0x7f7d141615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.823+0000 7f7d13400640 1 -- 172.21.15.165:0/524430443 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d141620d0 con 0x7f7d140d04f0 2024-04-25T10:02:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.823+0000 7f7d12000640 1 -- 172.21.15.165:0/524430443 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834423746 0 0) 0x7f7d14161df0 con 0x7f7d1415db90 2024-04-25T10:02:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.824+0000 7f7d13400640 1 --2- 172.21.15.165:0/524430443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d140d04f0 0x7f7d140d23e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7d140d1b30 tx=0x7f7d0c02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe4213304eef3476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.824+0000 7f7d12000640 1 -- 172.21.15.165:0/524430443 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c007740 con 0x7f7d140d04f0 2024-04-25T10:02:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.824+0000 7f7d12000640 1 -- 172.21.15.165:0/524430443 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c0078e0 con 0x7f7d140d04f0 2024-04-25T10:02:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.824+0000 7f7d12000640 1 -- 172.21.15.165:0/524430443 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c007cb0 con 0x7f7d140d04f0 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 -- 172.21.15.165:0/524430443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d140d04f0 msgr2=0x7f7d140d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 --2- 172.21.15.165:0/524430443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d140d04f0 0x7f7d140d23e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f7d140d1b30 tx=0x7f7d0c02a680 comp rx=0 tx=0).stop 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 -- 172.21.15.165:0/524430443 shutdown_connections 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 --2- 172.21.15.165:0/524430443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d140d04f0 0x7f7d140d23e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 --2- 172.21.15.165:0/524430443 >> v2:172.21.15.172:3300/0 conn(0x7f7d140b93c0 0x7f7d140cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 -- 172.21.15.165:0/524430443 >> 172.21.15.165:0/524430443 conn(0x7f7d140b52b0 msgr2=0x7f7d140b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 -- 172.21.15.165:0/524430443 shutdown_connections 2024-04-25T10:02:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.825+0000 7f7d13e00640 1 -- 172.21.15.165:0/524430443 wait complete. 2024-04-25T10:02:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.827+0000 7f7d13e00640 1 Processor -- start 2024-04-25T10:02:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.827+0000 7f7d13e00640 1 -- start start 2024-04-25T10:02:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d140b93c0 0x7f7d141f38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d140d04f0 0x7f7d141525b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 0x7f7d14160150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b68d0 con 0x7f7d1415db90 2024-04-25T10:02:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140d1be0 con 0x7f7d140b93c0 2024-04-25T10:02:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d140b6a50 con 0x7f7d140d04f0 2024-04-25T10:02:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 0x7f7d14160150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 0x7f7d14160150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45598/0 (socket says 172.21.15.165:45598) 2024-04-25T10:02:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 -- 172.21.15.165:0/62341067 learned_addr learned my addr 172.21.15.165:0/62341067 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:37.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d0a000640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d140d04f0 0x7f7d141525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d12a00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d140b93c0 0x7f7d141f38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 -- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d140d04f0 msgr2=0x7f7d141525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d140d04f0 0x7f7d141525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 -- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d140b93c0 msgr2=0x7f7d141f38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d140b93c0 0x7f7d141f38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d13400640 1 -- 172.21.15.165:0/62341067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1415f3d0 con 0x7f7d1415db90 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.828+0000 7f7d12a00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d140b93c0 0x7f7d141f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.829+0000 7f7d13400640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 0x7f7d14160150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c02ac70 tx=0x7f7d0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.829+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c040070 con 0x7f7d1415db90 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.829+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0c031bc0 con 0x7f7d1415db90 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.829+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d141e8680 con 0x7f7d1415db90 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.829+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0c009450 con 0x7f7d1415db90 2024-04-25T10:02:37.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.829+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1415f950 con 0x7f7d1415db90 2024-04-25T10:02:37.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.831+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d14153580 con 0x7f7d1415db90 2024-04-25T10:02:37.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.831+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d0c039040 con 0x7f7d1415db90 2024-04-25T10:02:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.832+0000 7f7d0b400640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7cf006b4d0 0x7f7cf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.833+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f7d0c0abe10 con 0x7f7d1415db90 2024-04-25T10:02:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.833+0000 7f7d12a00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7cf006b4d0 0x7f7cf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.833+0000 7f7d12a00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7cf006b4d0 0x7f7cf006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f60 tx=0x7f7d04009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:37.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:37.837+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d0c0af030 con 0x7f7d1415db90 2024-04-25T10:02:38.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.067+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d140d14b0 con 0x7f7d1415db90 2024-04-25T10:02:38.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.068+0000 7f7d0b400640 1 -- 172.21.15.165:0/62341067 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f7d0c07a940 con 0x7f7d1415db90 2024-04-25T10:02:38.073 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:38.073 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:38.073 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:02:38.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.070+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7cf006b4d0 msgr2=0x7f7cf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.070+0000 7f7d13e00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7cf006b4d0 0x7f7cf006d990 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d04000f60 tx=0x7f7d04009040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.070+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 msgr2=0x7f7d14160150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.070+0000 7f7d13e00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 0x7f7d14160150 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7d0c02ac70 tx=0x7f7d0c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 shutdown_connections 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7cf006b4d0 0x7f7cf006d990 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d1415db90 0x7f7d14160150 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d140d04f0 0x7f7d141525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 --2- 172.21.15.165:0/62341067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d140b93c0 0x7f7d141f38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 >> 172.21.15.165:0/62341067 conn(0x7f7d140b52b0 msgr2=0x7f7d140ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 shutdown_connections 2024-04-25T10:02:38.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.071+0000 7f7d13e00640 1 -- 172.21.15.165:0/62341067 wait complete. 2024-04-25T10:02:38.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:38.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.287+0000 7f4444200640 1 Processor -- start 2024-04-25T10:02:38.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.287+0000 7f4444200640 1 -- start start 2024-04-25T10:02:38.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.288+0000 7f4444200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f443c144750 0x7f443c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.288+0000 7f4444200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c06eb10 0x7f443c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.288+0000 7f4444200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f443c0cee60 0x7f443c15f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.288+0000 7f4444200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443c144b30 con 0x7f443c144750 2024-04-25T10:02:38.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.288+0000 7f4444200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f443c0712d0 con 0x7f443c0cee60 2024-04-25T10:02:38.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.288+0000 7f4444200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c0715b0 con 0x7f443c06eb10 2024-04-25T10:02:38.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f443c144750 0x7f443c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f443c144750 0x7f443c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45600/0 (socket says 172.21.15.165:45600) 2024-04-25T10:02:38.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 -- 172.21.15.165:0/1171525334 learned_addr learned my addr 172.21.15.165:0/1171525334 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4440e00640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c06eb10 0x7f443c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 -- 172.21.15.165:0/1171525334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c06eb10 msgr2=0x7f443c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4442200640 1 --2- 172.21.15.165:0/1171525334 >> v2:172.21.15.172:3300/0 conn(0x7f443c0cee60 0x7f443c15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c06eb10 0x7f443c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 -- 172.21.15.165:0/1171525334 >> v2:172.21.15.172:3300/0 conn(0x7f443c0cee60 msgr2=0x7f443c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 --2- 172.21.15.165:0/1171525334 >> v2:172.21.15.172:3300/0 conn(0x7f443c0cee60 0x7f443c15f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4441800640 1 -- 172.21.15.165:0/1171525334 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c071890 con 0x7f443c144750 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.289+0000 7f4440e00640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c06eb10 0x7f443c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.290+0000 7f4441800640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c06e5d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f443c0b4aa0 tx=0x7f443002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14b097295866455f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f443be00640 1 -- 172.21.15.165:0/1171525334 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430037070 con 0x7f443c144750 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f443be00640 1 -- 172.21.15.165:0/1171525334 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4430032440 con 0x7f443c144750 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f443be00640 1 -- 172.21.15.165:0/1171525334 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4430032810 con 0x7f443c144750 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 -- 172.21.15.165:0/1171525334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 msgr2=0x7f443c06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c06e5d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f443c0b4aa0 tx=0x7f443002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 -- 172.21.15.165:0/1171525334 shutdown_connections 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 --2- 172.21.15.165:0/1171525334 >> v2:172.21.15.172:3300/0 conn(0x7f443c0cee60 0x7f443c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c06eb10 0x7f443c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 --2- 172.21.15.165:0/1171525334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c06e5d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.291+0000 7f4444200640 1 -- 172.21.15.165:0/1171525334 >> 172.21.15.165:0/1171525334 conn(0x7f443c06abe0 msgr2=0x7f443c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.292+0000 7f4444200640 1 -- 172.21.15.165:0/1171525334 shutdown_connections 2024-04-25T10:02:38.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.292+0000 7f4444200640 1 -- 172.21.15.165:0/1171525334 wait complete. 2024-04-25T10:02:38.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.293+0000 7f4444200640 1 Processor -- start 2024-04-25T10:02:38.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.293+0000 7f4444200640 1 -- start start 2024-04-25T10:02:38.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.294+0000 7f4444200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c06eb10 0x7f443c13b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.294+0000 7f4444200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c0cee60 0x7f443c137130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.294+0000 7f4444200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c1f41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.294+0000 7f4444200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c145450 con 0x7f443c144750 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.294+0000 7f4444200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f443c06f020 con 0x7f443c06eb10 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.294+0000 7f4444200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f443c1455d0 con 0x7f443c0cee60 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c1f41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c1f41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45604/0 (socket says 172.21.15.165:45604) 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 -- 172.21.15.165:0/2458987694 learned_addr learned my addr 172.21.15.165:0/2458987694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4441800640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c06eb10 0x7f443c13b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4440e00640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c0cee60 0x7f443c137130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 -- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c0cee60 msgr2=0x7f443c137130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c0cee60 0x7f443c137130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 -- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c06eb10 msgr2=0x7f443c13b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c06eb10 0x7f443c13b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4442200640 1 -- 172.21.15.165:0/2458987694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f443c137bf0 con 0x7f443c144750 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.295+0000 7f4440e00640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c0cee60 0x7f443c137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.296+0000 7f4442200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c1f41c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f443400cd70 tx=0x7f443400a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.296+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434003b90 con 0x7f443c144750 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.296+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4434003d30 con 0x7f443c144750 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.296+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f443c1397b0 con 0x7f443c144750 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.296+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4434004240 con 0x7f443c144750 2024-04-25T10:02:38.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.296+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f443c139d70 con 0x7f443c144750 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.297+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f443c073990 con 0x7f443c144750 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.298+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44340043e0 con 0x7f443c144750 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.300+0000 7f4439600640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f441c06b4d0 0x7f441c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.300+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f443408b0c0 con 0x7f443c144750 2024-04-25T10:02:38.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.300+0000 7f4441800640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f441c06b4d0 0x7f441c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.301+0000 7f4441800640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f441c06b4d0 0x7f441c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4430034000 tx=0x7f443002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.304+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f443401d340 con 0x7f443c144750 2024-04-25T10:02:38.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.500+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f443c0d0510 con 0x7f443c144750 2024-04-25T10:02:38.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.502+0000 7f4439600640 1 -- 172.21.15.165:0/2458987694 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v24) v1 ==== 74+0+18550 (secure 0 0 0) 0x7f4434059c00 con 0x7f443c144750 2024-04-25T10:02:38.507 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:38.507 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":24,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:08.622462+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:38.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f441c06b4d0 msgr2=0x7f441c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f441c06b4d0 0x7f441c06d990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4430034000 tx=0x7f443002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 msgr2=0x7f443c1f41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c1f41c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f443400cd70 tx=0x7f443400a660 comp rx=0 tx=0).stop 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 shutdown_connections 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f441c06b4d0 0x7f441c06d990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f443c144750 0x7f443c1f41c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f443c0cee60 0x7f443c137130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 --2- 172.21.15.165:0/2458987694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f443c06eb10 0x7f443c13b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 >> 172.21.15.165:0/2458987694 conn(0x7f443c06abe0 msgr2=0x7f443c140ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 shutdown_connections 2024-04-25T10:02:38.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.505+0000 7f4444200640 1 -- 172.21.15.165:0/2458987694 wait complete. 2024-04-25T10:02:38.528 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:02:38.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.724+0000 7fc856a00640 1 Processor -- start 2024-04-25T10:02:38.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.724+0000 7fc856a00640 1 -- start start 2024-04-25T10:02:38.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc850144440 0x7fc850070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc850071530 0x7fc8500cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc85015bf30 con 0x7fc8500cd860 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc85015c860 con 0x7fc850144440 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc850071970 con 0x7fc850071530 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc8500cd860 0x7fc85006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:46250/0 (socket says 172.21.15.165:46250) 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc855600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc850144440 0x7fc850070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc856000640 1 -- 172.21.15.165:0/165820621 learned_addr learned my addr 172.21.15.165:0/165820621 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.726+0000 7fc854c00640 1 --2- 172.21.15.165:0/165820621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc850071530 0x7fc8500cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.727+0000 7fc855600640 1 -- 172.21.15.165:0/165820621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc850071530 msgr2=0x7fc8500cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:38.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.727+0000 7fc855600640 1 --2- 172.21.15.165:0/165820621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc850071530 0x7fc8500cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.727+0000 7fc855600640 1 -- 172.21.15.165:0/165820621 >> v1:172.21.15.165:6789/0 conn(0x7fc8500cd860 legacy=0x7fc85006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.727+0000 7fc855600640 1 -- 172.21.15.165:0/165820621 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85015c210 con 0x7fc850144440 2024-04-25T10:02:38.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.728+0000 7fc855600640 1 --2- 172.21.15.165:0/165820621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850144440 0x7fc850070ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc844009d90 tx=0x7fc84402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504bf45213fbeeaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.728+0000 7fc84fe00640 1 -- 172.21.15.165:0/165820621 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844007740 con 0x7fc850144440 2024-04-25T10:02:38.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.728+0000 7fc84fe00640 1 -- 172.21.15.165:0/165820621 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8440078e0 con 0x7fc850144440 2024-04-25T10:02:38.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.728+0000 7fc84fe00640 1 -- 172.21.15.165:0/165820621 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844007cb0 con 0x7fc850144440 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 -- 172.21.15.165:0/165820621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850144440 msgr2=0x7fc850070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 --2- 172.21.15.165:0/165820621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850144440 0x7fc850070ff0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fc844009d90 tx=0x7fc84402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 -- 172.21.15.165:0/165820621 shutdown_connections 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 --2- 172.21.15.165:0/165820621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc850071530 0x7fc8500cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 --2- 172.21.15.165:0/165820621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850144440 0x7fc850070ff0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 -- 172.21.15.165:0/165820621 >> 172.21.15.165:0/165820621 conn(0x7fc85006abe0 msgr2=0x7fc85006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 -- 172.21.15.165:0/165820621 shutdown_connections 2024-04-25T10:02:38.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.729+0000 7fc856a00640 1 -- 172.21.15.165:0/165820621 wait complete. 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.732+0000 7fc856a00640 1 Processor -- start 2024-04-25T10:02:38.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.732+0000 7fc856a00640 1 -- start start 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc856a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 0x7fc8501e9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc856a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8500cd860 0x7fc8501e2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc856a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc850144440 0x7fc85013e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc856a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500cdcd0 con 0x7fc850071530 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc856a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc85015bc30 con 0x7fc850144440 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc856a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8500cde50 con 0x7fc8500cd860 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc855600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 0x7fc8501e9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc855600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 0x7fc8501e9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45616/0 (socket says 172.21.15.165:45616) 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.733+0000 7fc855600640 1 -- 172.21.15.165:0/383993661 learned_addr learned my addr 172.21.15.165:0/383993661 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc854c00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8500cd860 0x7fc8501e2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc856000640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc850144440 0x7fc85013e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc855600640 1 -- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8500cd860 msgr2=0x7fc8501e2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc855600640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8500cd860 0x7fc8501e2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc855600640 1 -- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc850144440 msgr2=0x7fc85013e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc855600640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc850144440 0x7fc85013e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc855600640 1 -- 172.21.15.165:0/383993661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8501e1de0 con 0x7fc850071530 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc854c00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8500cd860 0x7fc8501e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc855600640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 0x7fc8501e9470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc844009d90 tx=0x7fc844004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.734+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84403f070 con 0x7fc850071530 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.735+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc844031c40 con 0x7fc850071530 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.735+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844044460 con 0x7fc850071530 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.735+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8501e20c0 con 0x7fc850071530 2024-04-25T10:02:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.735+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8501e1a00 con 0x7fc850071530 2024-04-25T10:02:38.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.736+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8501e24b0 con 0x7fc850071530 2024-04-25T10:02:38.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.737+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc844044600 con 0x7fc850071530 2024-04-25T10:02:38.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.738+0000 7fc84d600640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc83006b4d0 0x7fc83006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:38.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.738+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fc8440abdf0 con 0x7fc850071530 2024-04-25T10:02:38.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.738+0000 7fc854c00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc83006b4d0 0x7fc83006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:38.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.739+0000 7fc854c00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc83006b4d0 0x7fc83006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc83c006fd0 tx=0x7fc83c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:38.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.742+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc84407aae0 con 0x7fc850071530 2024-04-25T10:02:38.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.959+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8501e2290 con 0x7fc850071530 2024-04-25T10:02:38.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.960+0000 7fc84d600640 1 -- 172.21.15.165:0/383993661 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fc84407a3e0 con 0x7fc850071530 2024-04-25T10:02:38.964 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:38.964 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:02:38.965 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc83006b4d0 msgr2=0x7fc83006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc83006b4d0 0x7fc83006d990 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc83c006fd0 tx=0x7fc83c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 msgr2=0x7fc8501e9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 0x7fc8501e9470 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc844009d90 tx=0x7fc844004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 shutdown_connections 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc83006b4d0 0x7fc83006d990 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc850144440 0x7fc85013e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8500cd860 0x7fc8501e2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 --2- 172.21.15.165:0/383993661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc850071530 0x7fc8501e9470 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:38.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.962+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 >> 172.21.15.165:0/383993661 conn(0x7fc85006abe0 msgr2=0x7fc8500ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:38.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.963+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 shutdown_connections 2024-04-25T10:02:38.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:38.963+0000 7fc856a00640 1 -- 172.21.15.165:0/383993661 wait complete. 2024-04-25T10:02:39.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:02:03.570541+0000', 'modified': '2024-04-25T10:02:24.722564+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4288}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4288': {'gid': 4288, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6835/2597521026', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6833', 'nonce': 2597521026}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2597521026}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:02:39.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:02:39.388 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:02:39.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.589+0000 7ff53ac00640 1 Processor -- start 2024-04-25T10:02:39.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.590+0000 7ff53ac00640 1 -- start start 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.591+0000 7ff53ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5340cc4f0 0x7ff5340cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.591+0000 7ff53ac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5340cdb20 0x7ff5340cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.591+0000 7ff53ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff53415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.591+0000 7ff53ac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff534071270 con 0x7ff5340cdb20 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.591+0000 7ff53ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff534072280 con 0x7ff5340cc4f0 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.591+0000 7ff53ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5340714c0 con 0x7ff53415b270 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5340cdb20 0x7ff5340cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5340cdb20 0x7ff5340cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45624/0 (socket says 172.21.15.165:45624) 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 -- 172.21.15.165:0/1686107771 learned_addr learned my addr 172.21.15.165:0/1686107771 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff538c00640 1 --2- 172.21.15.165:0/1686107771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff53415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff533e00640 1 --2- 172.21.15.165:0/1686107771 >> v2:172.21.15.172:3300/0 conn(0x7ff5340cc4f0 0x7ff5340cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 -- 172.21.15.165:0/1686107771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 msgr2=0x7ff53415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 --2- 172.21.15.165:0/1686107771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff53415b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 -- 172.21.15.165:0/1686107771 >> v2:172.21.15.172:3300/0 conn(0x7ff5340cc4f0 msgr2=0x7ff5340cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 --2- 172.21.15.165:0/1686107771 >> v2:172.21.15.172:3300/0 conn(0x7ff5340cc4f0 0x7ff5340cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.592+0000 7ff52b400640 1 -- 172.21.15.165:0/1686107771 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340717a0 con 0x7ff5340cdb20 2024-04-25T10:02:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.593+0000 7ff52b400640 1 --2- 172.21.15.165:0/1686107771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cdb20 0x7ff5340cdfb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff5340cd290 tx=0x7ff51c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c93a1464e5bd06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff533400640 1 -- 172.21.15.165:0/1686107771 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c037070 con 0x7ff5340cdb20 2024-04-25T10:02:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff533400640 1 -- 172.21.15.165:0/1686107771 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51c032440 con 0x7ff5340cdb20 2024-04-25T10:02:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff533400640 1 -- 172.21.15.165:0/1686107771 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c032810 con 0x7ff5340cdb20 2024-04-25T10:02:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 -- 172.21.15.165:0/1686107771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cdb20 msgr2=0x7ff5340cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 --2- 172.21.15.165:0/1686107771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cdb20 0x7ff5340cdfb0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7ff5340cd290 tx=0x7ff51c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 -- 172.21.15.165:0/1686107771 shutdown_connections 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 --2- 172.21.15.165:0/1686107771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff53415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 --2- 172.21.15.165:0/1686107771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cdb20 0x7ff5340cdfb0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 --2- 172.21.15.165:0/1686107771 >> v2:172.21.15.172:3300/0 conn(0x7ff5340cc4f0 0x7ff5340cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 -- 172.21.15.165:0/1686107771 >> 172.21.15.165:0/1686107771 conn(0x7ff53406abe0 msgr2=0x7ff53406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 -- 172.21.15.165:0/1686107771 shutdown_connections 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.594+0000 7ff53ac00640 1 -- 172.21.15.165:0/1686107771 wait complete. 2024-04-25T10:02:39.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.596+0000 7ff53ac00640 1 Processor -- start 2024-04-25T10:02:39.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.596+0000 7ff53ac00640 1 -- start start 2024-04-25T10:02:39.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.597+0000 7ff53ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 0x7ff5341547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.597+0000 7ff53ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5340cdb20 0x7ff5341455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.597+0000 7ff53ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff534145b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.597+0000 7ff53ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415bc00 con 0x7ff5340cc4f0 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.597+0000 7ff53ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff534071d60 con 0x7ff5340cdb20 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.597+0000 7ff53ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53415bd80 con 0x7ff53415b270 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 0x7ff5341547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 0x7ff5341547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45626/0 (socket says 172.21.15.165:45626) 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 -- 172.21.15.165:0/60929212 learned_addr learned my addr 172.21.15.165:0/60929212 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff52b400640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5340cdb20 0x7ff5341455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff538c00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff534145b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 -- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 msgr2=0x7ff534145b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff534145b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 -- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5340cdb20 msgr2=0x7ff5341455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5340cdb20 0x7ff5341455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 -- 172.21.15.165:0/60929212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff534157c20 con 0x7ff5340cc4f0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff52b400640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5340cdb20 0x7ff5341455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff538c00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff534145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.598+0000 7ff533e00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 0x7ff5341547f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff52400d830 tx=0x7ff52400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.599+0000 7ff530c00640 1 -- 172.21.15.165:0/60929212 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff524018030 con 0x7ff5340cc4f0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.599+0000 7ff530c00640 1 -- 172.21.15.165:0/60929212 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52400be10 con 0x7ff5340cc4f0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.599+0000 7ff530c00640 1 -- 172.21.15.165:0/60929212 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5240098b0 con 0x7ff5340cc4f0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.599+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff534157f00 con 0x7ff5340cc4f0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.599+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff534154d30 con 0x7ff5340cc4f0 2024-04-25T10:02:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.601+0000 7ff530c00640 1 -- 172.21.15.165:0/60929212 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff52401b070 con 0x7ff5340cc4f0 2024-04-25T10:02:39.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.602+0000 7ff530c00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff51006b4d0 0x7ff51006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.602+0000 7ff530c00640 1 -- 172.21.15.165:0/60929212 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff52408a760 con 0x7ff5340cc4f0 2024-04-25T10:02:39.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.602+0000 7ff52b400640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff51006b4d0 0x7ff51006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.603+0000 7ff53d9e9740 1 Processor -- start 2024-04-25T10:02:39.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.603+0000 7ff52b400640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff51006b4d0 0x7ff51006d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff51c034000 tx=0x7ff51c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.603+0000 7ff53d9e9740 1 -- start start 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9e5790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0f9bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f921620 con 0x55fb0fa065c0 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f8caca0 con 0x55fb0f9bd950 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f91d6c0 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff538c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0f9bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff538c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0f9bc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45628/0 (socket says 172.21.15.165:45628) 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.604+0000 7ff538c00640 1 -- 172.21.15.165:0/809183097 learned_addr learned my addr 172.21.15.165:0/809183097 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:39.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff533e00640 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9e5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff52b400640 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff538c00640 1 -- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 msgr2=0x55fb0f9bdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff538c00640 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9bdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff538c00640 1 -- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 msgr2=0x55fb0f9e5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff538c00640 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9e5790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff538c00640 1 -- 172.21.15.165:0/809183097 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb0f8d1f40 con 0x55fb0fa065c0 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff538c00640 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0f9bc110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff52c007650 tx=0x7ff52c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e585bfb249c53d05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff523e00640 1 -- 172.21.15.165:0/809183097 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0147e0 con 0x55fb0fa065c0 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff523e00640 1 -- 172.21.15.165:0/809183097 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c01a070 con 0x55fb0fa065c0 2024-04-25T10:02:39.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.605+0000 7ff523e00640 1 -- 172.21.15.165:0/809183097 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c014be0 con 0x55fb0fa065c0 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 -- 172.21.15.165:0/809183097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 msgr2=0x55fb0f9bc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0f9bc110 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff52c007650 tx=0x7ff52c009750 comp rx=0 tx=0).stop 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 -- 172.21.15.165:0/809183097 shutdown_connections 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0f9bc110 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9bdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/809183097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9e5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 -- 172.21.15.165:0/809183097 >> 172.21.15.165:0/809183097 conn(0x55fb0f9bab70 msgr2=0x55fb0fa03550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 -- 172.21.15.165:0/809183097 shutdown_connections 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.606+0000 7ff53d9e9740 1 -- 172.21.15.165:0/809183097 wait complete. 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.607+0000 7ff53d9e9740 1 -- start start 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.607+0000 7ff53d9e9740 1 Processor -- start 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9e5cd0 0x55fb0f9d0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0fa05480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f9e5300 con 0x55fb0fa065c0 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f8c01f0 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f9e4cf0 con 0x55fb0f9d6250 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff53d9e9740 10 client.0 _ll_register_callbacks cb 0x55fb0f8ddad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.608+0000 7ff52b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0fa05480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0fa05480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45638/0 (socket says 172.21.15.165:45638) 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 -- 172.21.15.165:0/407652307 learned_addr learned my addr 172.21.15.165:0/407652307 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff533e00640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9e5cd0 0x55fb0f9d0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff538c00640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 -- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 msgr2=0x55fb0fa059c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa059c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 -- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9e5cd0 msgr2=0x55fb0f9d0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9e5cd0 0x55fb0f9d0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 -- 172.21.15.165:0/407652307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb0f8dfc50 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff52b400640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0fa05480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff51c02a7d0 tx=0x7ff51c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818ab83dee630ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.609+0000 7ff520c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff520c00640 1 -- 172.21.15.165:0/407652307 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c037040 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff520c00640 1 -- 172.21.15.165:0/407652307 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff51c03ca70 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52c01a070 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff520c00640 1 -- 172.21.15.165:0/407652307 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff51c055050 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 10 client.4833 fetch_fsmap learned FSMap version 9 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55fb0f6362d0 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff520c00640 1 -- 172.21.15.165:0/407652307 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c01a070 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff520c00640 1 -- 172.21.15.165:0/407652307 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff51c058020 con 0x55fb0fa065c0 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 10 client.4833 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T10:02:39.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 10 client.4833 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:39.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 20 client.4833 populate_metadata read hostname 'smithi165' 2024-04-25T10:02:39.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0f9d3b90 0x55fb0f9d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 4 client.4833 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:02:39.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.610+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 --> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb0f92eb60 con 0x55fb0f9d3b90 2024-04-25T10:02:39.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.611+0000 7ff533e00640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0f9d3b90 0x55fb0f9d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.612+0000 7ff533e00640 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0f9d3b90 0x55fb0f9d3f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e831b3b80d58db7f server_cookie=6b1a1cca5249667e in_seq=0 out_seq=0 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.612+0000 7ff520c00640 10 client.4833 ms_handle_connect on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff520c00640 1 -- 172.21.15.165:0/407652307 <== mds.0 v2:172.21.15.172:6833/2597521026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fb0f92eb60 con 0x55fb0f9d3b90 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff520c00640 10 client.4833 handle_command_reply: tid=0 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 client.4833 shutdown 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 msgr2=0x55fb0fa05480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0fa05480 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7ff51c02a7d0 tx=0x7ff51c02c070 comp rx=0 tx=0).stop 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 shutdown_connections 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0f9d3b90 0x55fb0f9d3f70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0fa065c0 0x55fb0fa05480 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/407652307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9e5cd0 0x55fb0f9d0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff520c00640 0 client.4833 ms_handle_reset on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.614+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 >> 172.21.15.165:0/407652307 conn(0x55fb0f9bd950 msgr2=0x55fb0f9bc110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.615+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 shutdown_connections 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.615+0000 7ff53d9e9740 1 -- 172.21.15.165:0/407652307 wait complete. 2024-04-25T10:02:39.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.615+0000 7ff53d9e9740 20 client.4833 trim_cache size 0 max 16384 2024-04-25T10:02:39.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.632+0000 7ff53d9e9740 1 Processor -- start 2024-04-25T10:02:39.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.632+0000 7ff53d9e9740 1 -- start start 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.632+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9eb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.632+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa11c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 0x55fb0fa121b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff534200530 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341ff960 con 0x55fb0f9bd950 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5341fff60 con 0x55fb0f9d6250 2024-04-25T10:02:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 0x55fb0fa121b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 0x55fb0fa121b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45644/0 (socket says 172.21.15.165:45644) 2024-04-25T10:02:39.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 -- 172.21.15.165:0/420496825 learned_addr learned my addr 172.21.15.165:0/420496825 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff52b400640 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa11c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff533e00640 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9eb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 -- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 msgr2=0x55fb0fa11c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa11c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 -- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 msgr2=0x55fb0f9eb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9eb1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.633+0000 7ff538c00640 1 -- 172.21.15.165:0/420496825 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb0f7006b0 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.634+0000 7ff538c00640 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 0x55fb0fa121b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff52c007620 tx=0x7ff52c00d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b171668452403301 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.634+0000 7ff523400640 1 -- 172.21.15.165:0/420496825 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c009410 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.634+0000 7ff523400640 1 -- 172.21.15.165:0/420496825 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c01a040 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.634+0000 7ff523400640 1 -- 172.21.15.165:0/420496825 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52c0147e0 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.634+0000 7ff53d9e9740 1 -- 172.21.15.165:0/420496825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 msgr2=0x55fb0fa121b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.634+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 0x55fb0fa121b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7ff52c007620 tx=0x7ff52c00d000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 -- 172.21.15.165:0/420496825 shutdown_connections 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9e5cd0 0x55fb0fa121b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9d6250 0x55fb0fa11c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/420496825 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0f9eb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 -- 172.21.15.165:0/420496825 >> 172.21.15.165:0/420496825 conn(0x55fb0f9b7ec0 msgr2=0x55fb0fa26090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 -- 172.21.15.165:0/420496825 shutdown_connections 2024-04-25T10:02:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 -- 172.21.15.165:0/420496825 wait complete. 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 -- start start 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.635+0000 7ff53d9e9740 1 Processor -- start 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.636+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0fa65540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.636+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 0x55fb0f9bdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.636+0000 7ff53d9e9740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9d4550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.636+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f9cfb60 con 0x55fb0f9d6250 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.636+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f9d1db0 con 0x55fb0f9bd950 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.636+0000 7ff53d9e9740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fb0f9d27d0 con 0x55fb0f9e5cd0 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff53d9e9740 10 client.0 _ll_register_callbacks cb 0x7ff51c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 0x55fb0f9bdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 0x55fb0f9bdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45646/0 (socket says 172.21.15.165:45646) 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 -- 172.21.15.165:0/2963656900 learned_addr learned my addr 172.21.15.165:0/2963656900 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 -- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 msgr2=0x55fb0f9d4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9d4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 -- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 msgr2=0x55fb0fa65540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0fa65540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 -- 172.21.15.165:0/2963656900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb0f9bc240 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff52b400640 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 0x55fb0f9bdd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff51c02a7d0 tx=0x7ff51c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a833198fc4dc8bb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.637+0000 7ff520c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff520c00640 1 -- 172.21.15.165:0/2963656900 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c037040 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff520c00640 1 -- 172.21.15.165:0/2963656900 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7ff51c004230 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52c01a040 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff520c00640 1 -- 172.21.15.165:0/2963656900 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff51c066050 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 10 client.4839 fetch_fsmap learned FSMap version 9 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55fb0f9d15d0 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff520c00640 1 -- 172.21.15.165:0/2963656900 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff52c01a040 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff520c00640 1 -- 172.21.15.165:0/2963656900 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 9) v1 ==== 3302+0+0 (secure 0 0 0) 0x7ff51c069020 con 0x55fb0f9d6250 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 10 client.4839 fetch_fsmap finished waiting for FSMap version 9 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 10 client.4839 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:02:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 20 client.4839 populate_metadata read hostname 'smithi165' 2024-04-25T10:02:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0fa065c0 0x55fb0fa156d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 4 client.4839 mds_command: new command op to 4288 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:02:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.638+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 --> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fb0f928eb0 con 0x55fb0fa065c0 2024-04-25T10:02:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.639+0000 7ff533e00640 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0fa065c0 0x55fb0fa156d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:39.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.640+0000 7ff533e00640 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0fa065c0 0x55fb0fa156d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72cda508eae70d9b server_cookie=eb9894e12e4defd6 in_seq=0 out_seq=0 2024-04-25T10:02:39.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.640+0000 7ff520c00640 10 client.4839 ms_handle_connect on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.640+0000 7ff520c00640 1 -- 172.21.15.165:0/2963656900 <== mds.0 v2:172.21.15.172:6833/2597521026 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x55fb0f928eb0 con 0x55fb0fa065c0 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.640+0000 7ff520c00640 10 client.4839 handle_command_reply: tid=0 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 client.4839 shutdown 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 msgr2=0x55fb0f9bdd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 0x55fb0f9bdd30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7ff51c02a7d0 tx=0x7ff51c004730 comp rx=0 tx=0).stop 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 shutdown_connections 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fb0f9e5cd0 0x55fb0f9d4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:6833/2597521026,v1:172.21.15.172:6835/2597521026] conn(0x55fb0fa065c0 0x55fb0fa156d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fb0f9d6250 0x55fb0f9bdd30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 --2- 172.21.15.165:0/2963656900 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fb0f9bd950 0x55fb0fa65540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff520c00640 0 client.4839 ms_handle_reset on v2:172.21.15.172:6833/2597521026 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 >> 172.21.15.165:0/2963656900 conn(0x55fb0f9b7ec0 msgr2=0x55fb0f9eb1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 shutdown_connections 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 1 -- 172.21.15.165:0/2963656900 wait complete. 2024-04-25T10:02:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.641+0000 7ff53d9e9740 20 client.4839 trim_cache size 0 max 16384 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 4288, 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:02:39.649 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 35.518517971999998, 2024-04-25T10:02:39.650 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 9, 2024-04-25T10:02:39.650 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 24, 2024-04-25T10:02:39.650 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 24, 2024-04-25T10:02:39.650 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 41.154186641999999 2024-04-25T10:02:39.650 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff51006b4d0 msgr2=0x7ff51006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff51006b4d0 0x7ff51006d990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff51c034000 tx=0x7ff51c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 msgr2=0x7ff5341547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 0x7ff5341547f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7ff52400d830 tx=0x7ff52400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 shutdown_connections 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff51006b4d0 0x7ff51006d990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff53415b270 0x7ff534145b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5340cdb20 0x7ff5341455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 --2- 172.21.15.165:0/60929212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5340cc4f0 0x7ff5341547f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 >> 172.21.15.165:0/60929212 conn(0x7ff53406abe0 msgr2=0x7ff53406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 shutdown_connections 2024-04-25T10:02:39.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:39.642+0000 7ff53ac00640 1 -- 172.21.15.165:0/60929212 wait complete. 2024-04-25T10:02:39.665 INFO:teuthology.run_tasks:Running task kclient... 2024-04-25T10:02:39.682 INFO:tasks.kclient:Mounting kernel clients... 2024-04-25T10:02:39.682 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-04-25T10:02:39.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.683 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:02:39.734 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-25T10:02:39.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-25T10:02:39.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.817 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link delete ceph-brx 2024-04-25T10:02:39.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.944 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:02:39.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:39.988 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:40.121 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:40.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.122 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-25T10:02:40.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.180 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.272 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:40.275 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:40.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.276 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-25T10:02:40.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.327 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-25T10:02:40.414 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-04-25T10:02:40.418 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:02:40.418 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-25T10:02:40.419 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:40.419 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:40.419 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:40.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.621+0000 7f212ac00640 1 Processor -- start 2024-04-25T10:02:40.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.621+0000 7f212ac00640 1 -- start start 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f212415e520 0x7f212415e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212ac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2124070ff0 0x7f2124071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212ac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21240ae590 con 0x7f212415f040 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212ac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21240aee30 con 0x7f2124070ff0 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2124071980 con 0x7f212415e520 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2124070ff0 0x7f2124071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2124070ff0 0x7f2124071440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45656/0 (socket says 172.21.15.165:45656) 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f212a200640 1 -- 172.21.15.165:0/1075032781 learned_addr learned my addr 172.21.15.165:0/1075032781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.623+0000 7f2129800640 1 --2- 172.21.15.165:0/1075032781 >> v2:172.21.15.172:3300/0 conn(0x7f212415e520 0x7f212415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.624+0000 7f212a200640 1 -- 172.21.15.165:0/1075032781 >> v2:172.21.15.172:3300/0 conn(0x7f212415e520 msgr2=0x7f212415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.624+0000 7f212a200640 1 --2- 172.21.15.165:0/1075032781 >> v2:172.21.15.172:3300/0 conn(0x7f212415e520 0x7f212415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.624+0000 7f212a200640 1 -- 172.21.15.165:0/1075032781 >> v1:172.21.15.172:6789/0 conn(0x7f212415f040 legacy=0x7f21240adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.624+0000 7f212a200640 1 -- 172.21.15.165:0/1075032781 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21240ae870 con 0x7f2124070ff0 2024-04-25T10:02:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.625+0000 7f212a200640 1 --2- 172.21.15.165:0/1075032781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124070ff0 0x7f2124071440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f212406ba10 tx=0x7f211c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a939a4dd6a23a1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.625+0000 7f2123e00640 1 -- 172.21.15.165:0/1075032781 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c036070 con 0x7f2124070ff0 2024-04-25T10:02:40.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.625+0000 7f2123e00640 1 -- 172.21.15.165:0/1075032781 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c031600 con 0x7f2124070ff0 2024-04-25T10:02:40.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.625+0000 7f2123e00640 1 -- 172.21.15.165:0/1075032781 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c0319d0 con 0x7f2124070ff0 2024-04-25T10:02:40.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 -- 172.21.15.165:0/1075032781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124070ff0 msgr2=0x7f2124071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 --2- 172.21.15.165:0/1075032781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124070ff0 0x7f2124071440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f212406ba10 tx=0x7f211c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 -- 172.21.15.165:0/1075032781 shutdown_connections 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 --2- 172.21.15.165:0/1075032781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2124070ff0 0x7f2124071440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 --2- 172.21.15.165:0/1075032781 >> v2:172.21.15.172:3300/0 conn(0x7f212415e520 0x7f212415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 -- 172.21.15.165:0/1075032781 >> 172.21.15.165:0/1075032781 conn(0x7f212406abe0 msgr2=0x7f212406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 -- 172.21.15.165:0/1075032781 shutdown_connections 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.626+0000 7f212ac00640 1 -- 172.21.15.165:0/1075032781 wait complete. 2024-04-25T10:02:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.627+0000 7f212ac00640 1 Processor -- start 2024-04-25T10:02:40.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.628+0000 7f212ac00640 1 -- start start 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2124070ff0 0x7f2124158540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212415e520 0x7f2124137d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 0x7f212415c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21240ae2a0 con 0x7f212415f040 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f212406d1f0 con 0x7f212415e520 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f212415f4d0 con 0x7f2124070ff0 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 0x7f212415c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 0x7f212415c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45664/0 (socket says 172.21.15.165:45664) 2024-04-25T10:02:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 -- 172.21.15.165:0/1708016258 learned_addr learned my addr 172.21.15.165:0/1708016258 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f2129800640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2124070ff0 0x7f2124158540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 -- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2124070ff0 msgr2=0x7f2124158540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2124070ff0 0x7f2124158540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 -- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212415e520 msgr2=0x7f2124137d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f2128e00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212415e520 0x7f2124137d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212415e520 0x7f2124137d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f212a200640 1 -- 172.21.15.165:0/1708016258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2124138250 con 0x7f212415f040 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.629+0000 7f2129800640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2124070ff0 0x7f2124158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.630+0000 7f212a200640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 0x7f212415c460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f211c009f40 tx=0x7f211c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.630+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c036050 con 0x7f212415f040 2024-04-25T10:02:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.630+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211c042070 con 0x7f212415f040 2024-04-25T10:02:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.630+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f211c009510 con 0x7f212415f040 2024-04-25T10:02:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.630+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21241f6f90 con 0x7f212415f040 2024-04-25T10:02:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.630+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21241f7550 con 0x7f212415f040 2024-04-25T10:02:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.631+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21241f7c10 con 0x7f212415f040 2024-04-25T10:02:40.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.632+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f211c007bd0 con 0x7f212415f040 2024-04-25T10:02:40.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.633+0000 7f2121600640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:40.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.633+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7f211c0ab4b0 con 0x7f212415f040 2024-04-25T10:02:40.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.633+0000 7f2129800640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:40.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.634+0000 7f2129800640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2118006fd0 tx=0x7f2118008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:40.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.638+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f21241f7c10 con 0x7f212415f040 2024-04-25T10:02:40.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.871+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2124073990 con 0x7f212415f040 2024-04-25T10:02:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.872+0000 7f2121600640 1 -- 172.21.15.165:0/1708016258 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f211c045360 con 0x7f212415f040 2024-04-25T10:02:40.876 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 msgr2=0x7f210406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2118006fd0 tx=0x7f2118008040 comp rx=0 tx=0).stop 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 msgr2=0x7f212415c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 0x7f212415c460 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f211c009f40 tx=0x7f211c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 shutdown_connections 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212415f040 0x7f212415c460 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212415e520 0x7f2124137d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 --2- 172.21.15.165:0/1708016258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2124070ff0 0x7f2124158540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 >> 172.21.15.165:0/1708016258 conn(0x7f212406abe0 msgr2=0x7f21240adb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.874+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 shutdown_connections 2024-04-25T10:02:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:40.875+0000 7f212ac00640 1 -- 172.21.15.165:0/1708016258 wait complete. 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:40.899 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:40.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.899 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:02:40.949 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:40.949 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:40.949 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:40.949 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.949 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42605sec preferred_lft 42605sec 2024-04-25T10:02:40.950 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:02:40.951 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:40.951 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:40.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set ceph-brx up 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:40.952 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:41.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.139 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:41.209 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:02:41.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.213 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:02:41.262 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:02:41.262 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:41.262 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:41.262 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:41.262 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:41.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.263 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:41.263 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:41.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:41.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:41.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:41.263 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:41.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.403 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:02:41.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.452 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list-id 2024-04-25T10:02:41.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.506 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:41.506 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:41.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:41.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-25T10:02:41.506 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:41.630 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-25T10:02:41.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-25T10:02:41.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:41.631 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-25T10:02:41.631 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-25T10:02:41.631 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-25T10:02:41.631 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:41.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:41.836 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:41.836 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:41.836 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:02:41.836 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:41.836 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:41.972 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:41.973 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.016 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:02:42.016 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:42.016 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:42.111 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:42.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.111 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:42.236 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:02:42.890 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:42.891 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:42.891 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:02:42.891 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:42.891 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:02:42.928 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:02:42.929 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:02:42.933 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:02:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:42.933 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:42.938 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:42.939 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:42.939 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:02:43.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.011 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:43.060 INFO:teuthology.orchestra.run.smithi165.stdout:client4639 2024-04-25T10:02:43.061 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:02:43.061 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:43.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.107 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:43.202 INFO:teuthology.orchestra.run.smithi165.stdout:client4639 2024-04-25T10:02:43.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.202 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:43.251 INFO:teuthology.orchestra.run.smithi165.stdout:client4639 2024-04-25T10:02:43.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.251 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/55d22d87-b25b-4ccd-bc45-d797a973b3c0.client4639/status') 2024-04-25T10:02:43.319 INFO:teuthology.orchestra.run.smithi165.stdout:instance: client.4639 (3)172.21.15.165:0/2872529924 2024-04-25T10:02:43.320 INFO:teuthology.orchestra.run.smithi165.stdout:blocklisted: false 2024-04-25T10:02:43.320 INFO:teuthology.orchestra.run.smithi165.stderr:0+1 records in 2024-04-25T10:02:43.320 INFO:teuthology.orchestra.run.smithi165.stderr:0+1 records out 2024-04-25T10:02:43.320 INFO:teuthology.orchestra.run.smithi165.stderr:71 bytes copied, 6.4104e-05 s, 1.1 MB/s 2024-04-25T10:02:43.323 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-04-25T10:02:43.323 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:43.323 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:43.324 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:43.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.502+0000 7fb3c5800640 1 Processor -- start 2024-04-25T10:02:43.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.502+0000 7fb3c5800640 1 -- start start 2024-04-25T10:02:43.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.504+0000 7fb3c5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 0x7fb3c00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.504+0000 7fb3c5800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c0071480 con 0x7fb3c0154ee0 2024-04-25T10:02:43.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.504+0000 7fb3c5800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3c0071760 con 0x7fb3c00ce3c0 2024-04-25T10:02:43.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.504+0000 7fb3c5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0072090 con 0x7fb3c00cfec0 2024-04-25T10:02:43.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.504+0000 7fb3bea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb3c0154ee0 0x7fb3c00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:46260/0 (socket says 172.21.15.165:46260) 2024-04-25T10:02:43.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.504+0000 7fb3bea00640 1 -- 172.21.15.165:0/1058706464 learned_addr learned my addr 172.21.15.165:0/1058706464 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:43.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3be000640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 0x7fb3c00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584292721 0 0) 0x7fb3c0071480 con 0x7fb3c0154ee0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3a4005730 con 0x7fb3c0154ee0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2893780833 0 0) 0x7fb3a4005730 con 0x7fb3c0154ee0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 msgr2=0x7fb3c00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 0x7fb3c00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 >> v1:172.21.15.172:6789/0 conn(0x7fb3c00ce3c0 legacy=0x7fb3c006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c0071a40 con 0x7fb3c0154ee0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb3b0001670 con 0x7fb3c0154ee0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 >> v1:172.21.15.165:6789/0 conn(0x7fb3c0154ee0 legacy=0x7fb3c00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a4005bf0 0x7fb3a4006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 0x7fb3a4006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a4006f50 0x7fb3a40073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a400dc80 con 0x7fb3a4005bf0 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.505+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a400de00 con 0x7fb3a4006f50 2024-04-25T10:02:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.506+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3a400e000 con 0x7fb3c00cfec0 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.506+0000 7fb3bea00640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a4005bf0 0x7fb3a4006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.506+0000 7fb3bf400640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a4006f50 0x7fb3a40073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.506+0000 7fb3be000640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 0x7fb3a4006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.506+0000 7fb3bea00640 1 -- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 msgr2=0x7fb3a4006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.506+0000 7fb3bea00640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c00cfec0 0x7fb3a4006a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.507+0000 7fb3bea00640 1 -- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a4006f50 msgr2=0x7fb3a40073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:43.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.507+0000 7fb3bea00640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a4006f50 0x7fb3a40073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.507+0000 7fb3bea00640 1 -- 172.21.15.165:0/1058706464 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a4005730 con 0x7fb3a4005bf0 2024-04-25T10:02:43.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.507+0000 7fb3bf400640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3a4006f50 0x7fb3a40073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:43.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.509+0000 7fb3bea00640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a4005bf0 0x7fb3a4006010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c210 tx=0x7fb3a4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=984ba23339dfa526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.509+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0007d10 con 0x7fb3a4005bf0 2024-04-25T10:02:43.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.509+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3b00058d0 con 0x7fb3a4005bf0 2024-04-25T10:02:43.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.509+0000 7fb3bd600640 1 -- 172.21.15.165:0/1058706464 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3b0005ce0 con 0x7fb3a4005bf0 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3c5800640 1 -- 172.21.15.165:0/1058706464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a4005bf0 msgr2=0x7fb3a4006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3c5800640 1 --2- 172.21.15.165:0/1058706464 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3a4005bf0 0x7fb3a4006010 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb3b000c210 tx=0x7fb3a4007a40 comp rx=0 tx=0).stop 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3bf400640 1 -- 172.21.15.165:0/1058706464 reap_dead start 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3c5800640 1 -- 172.21.15.165:0/1058706464 shutdown_connections 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3c5800640 1 -- 172.21.15.165:0/1058706464 >> 172.21.15.165:0/1058706464 conn(0x7fb3c006abe0 msgr2=0x7fb3c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3c5800640 1 -- 172.21.15.165:0/1058706464 shutdown_connections 2024-04-25T10:02:43.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.510+0000 7fb3c5800640 1 -- 172.21.15.165:0/1058706464 wait complete. 2024-04-25T10:02:43.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.512+0000 7fb3c5800640 1 Processor -- start 2024-04-25T10:02:43.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.512+0000 7fb3c5800640 1 -- start start 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.513+0000 7fb3c5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a4005bf0 0x7fb3c014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.513+0000 7fb3c5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c01f4710 0x7fb3c0159a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.513+0000 7fb3c5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0159f50 0x7fb3c014cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.513+0000 7fb3c5800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c00ce800 con 0x7fb3c01f4710 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.513+0000 7fb3c5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c0071230 con 0x7fb3c0159f50 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.513+0000 7fb3c5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c01552c0 con 0x7fb3a4005bf0 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3bea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a4005bf0 0x7fb3c014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3be000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c01f4710 0x7fb3c0159a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3bea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a4005bf0 0x7fb3c014f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:33438/0 (socket says 172.21.15.165:33438) 2024-04-25T10:02:43.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3bea00640 1 -- 172.21.15.165:0/2330189463 learned_addr learned my addr 172.21.15.165:0/2330189463 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:43.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3bf400640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0159f50 0x7fb3c014cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3be000640 1 -- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a4005bf0 msgr2=0x7fb3c014f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3be000640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a4005bf0 0x7fb3c014f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3be000640 1 -- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0159f50 msgr2=0x7fb3c014cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3be000640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0159f50 0x7fb3c014cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.514+0000 7fb3be000640 1 -- 172.21.15.165:0/2330189463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c01f3c50 con 0x7fb3c01f4710 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.515+0000 7fb3be000640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c01f4710 0x7fb3c0159a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac011960 tx=0x7fb3ac011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.515+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac019070 con 0x7fb3c01f4710 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.515+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac004240 con 0x7fb3c01f4710 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.515+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3c014d9e0 con 0x7fb3c01f4710 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.515+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00f7c0 con 0x7fb3c01f4710 2024-04-25T10:02:43.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.515+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3c014fbf0 con 0x7fb3c01f4710 2024-04-25T10:02:43.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.516+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3c0073990 con 0x7fb3c01f4710 2024-04-25T10:02:43.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.517+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3ac01c070 con 0x7fb3c01f4710 2024-04-25T10:02:43.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.519+0000 7fb3b6a00640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3a006b500 0x7fb3a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:43.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.519+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6532+0+0 (secure 0 0 0) 0x7fb3ac08a730 con 0x7fb3c01f4710 2024-04-25T10:02:43.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.519+0000 7fb3bea00640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3a006b500 0x7fb3a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:43.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.520+0000 7fb3bea00640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3a006b500 0x7fb3a006d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0007f60 tx=0x7fb3b002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:43.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.523+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3ac058c20 con 0x7fb3c01f4710 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.767+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb3c015a330 con 0x7fb3c01f4710 2024-04-25T10:02:43.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.768+0000 7fb3b6a00640 1 -- 172.21.15.165:0/2330189463 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb3ac05ccc0 con 0x7fb3c01f4710 2024-04-25T10:02:43.773 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:43.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3a006b500 msgr2=0x7fb3a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3a006b500 0x7fb3a006d9c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb3b0007f60 tx=0x7fb3b002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:43.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c01f4710 msgr2=0x7fb3c0159a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:43.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c01f4710 0x7fb3c0159a10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac011960 tx=0x7fb3ac011e20 comp rx=0 tx=0).stop 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 shutdown_connections 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3a006b500 0x7fb3a006d9c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c0159f50 0x7fb3c014cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c01f4710 0x7fb3c0159a10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 --2- 172.21.15.165:0/2330189463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3a4005bf0 0x7fb3c014f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 >> 172.21.15.165:0/2330189463 conn(0x7fb3c006abe0 msgr2=0x7fb3c00b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 shutdown_connections 2024-04-25T10:02:43.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:43.771+0000 7fb3c5800640 1 -- 172.21.15.165:0/2330189463 wait complete. 2024-04-25T10:02:43.793 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:43.793 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:43.793 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:02:43.794 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-04-25T10:02:43.794 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:43.794 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:43.794 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:43.794 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:43.794 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:43.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.794 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:02:43.842 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:43.842 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:43.843 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42602sec preferred_lft 42602sec 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:02:43.844 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:43.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.845 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:43.916 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:02:43.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.948 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:02:43.993 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:02:43.994 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:43.994 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:43.994 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:43.994 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:43.994 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:43.995 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:43.995 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:43.995 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:43.995 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:43.995 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:43.995 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:44.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.175 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:02:44.218 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:02:44.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.218 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list-id 2024-04-25T10:02:44.305 INFO:teuthology.orchestra.run.smithi165.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-04-25T10:02:44.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.307 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:44.307 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:44.307 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:02:44.307 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-04-25T10:02:44.307 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:44.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.416 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:44.486 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 32:f9:38:ab:3e:c8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::30f9:38ff:feab:3ec8/64 scope link 2024-04-25T10:02:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.489 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:44.560 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 32:f9:38:ab:3e:c8 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::30f9:38ff:feab:3ec8/64 scope link 2024-04-25T10:02:44.561 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:44.563 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-04-25T10:02:44.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.563 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-04-25T10:02:44.564 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:44.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:44.766 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:02:44.766 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:02:44.766 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.1 up 2024-04-25T10:02:44.766 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:44.766 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:02:44.913 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:44.913 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:44.952 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-04-25T10:02:44.952 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:44.952 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:45.043 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:45.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.043 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.217 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.217 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:02:45.217 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:02:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:02:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:02:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:45.266 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-04-25T10:02:45.266 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-04-25T10:02:45.266 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:02:45.266 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:45.267 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:45.267 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:02:45.267 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:02:45.267 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:02:45.269 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-04-25T10:02:45.269 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:45.269 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:45.269 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:45.269 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:45.273 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:45.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.273 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-04-25T10:02:45.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.352 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:45.407 INFO:teuthology.orchestra.run.smithi165.stdout:client4469 2024-04-25T10:02:45.408 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:02:45.408 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:45.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.458 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:45.512 INFO:teuthology.orchestra.run.smithi165.stdout:client4469 2024-04-25T10:02:45.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.512 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:45.565 INFO:teuthology.orchestra.run.smithi165.stdout:client4469 2024-04-25T10:02:45.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:45.566 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/55d22d87-b25b-4ccd-bc45-d797a973b3c0.client4469/status') 2024-04-25T10:02:45.650 INFO:teuthology.orchestra.run.smithi165.stdout:instance: client.4469 (3)172.21.15.165:0/2160807288 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi165.stdout:blocklisted: false 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi165.stderr:0+1 records in 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi165.stderr:0+1 records out 2024-04-25T10:02:45.651 INFO:teuthology.orchestra.run.smithi165.stderr:71 bytes copied, 0.000107101 s, 663 kB/s 2024-04-25T10:02:45.655 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-25T10:02:45.655 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:45.655 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:45.655 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:45.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.854+0000 7fb55b000640 1 Processor -- start 2024-04-25T10:02:45.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.854+0000 7fb55b000640 1 -- start start 2024-04-25T10:02:45.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.855+0000 7fb55b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb5540cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.855+0000 7fb55b000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5540cfcf0 0x7fb5540cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.855+0000 7fb55b000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb554071480 con 0x7fb5540ce3c0 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.855+0000 7fb55b000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540ce830 con 0x7fb5540cfcf0 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.855+0000 7fb55b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554072090 con 0x7fb554154ee0 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5540cfcf0 0x7fb5540cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5540cfcf0 0x7fb5540cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45692/0 (socket says 172.21.15.165:45692) 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 -- 172.21.15.165:0/2206782047 learned_addr learned my addr 172.21.15.165:0/2206782047 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb553e00640 1 --2- 172.21.15.165:0/2206782047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb5540cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 -- 172.21.15.165:0/2206782047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 msgr2=0x7fb5540cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 --2- 172.21.15.165:0/2206782047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb5540cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 -- 172.21.15.165:0/2206782047 >> v1:172.21.15.172:6789/0 conn(0x7fb5540ce3c0 legacy=0x7fb55415e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.856+0000 7fb54b400640 1 -- 172.21.15.165:0/2206782047 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb554071760 con 0x7fb5540cfcf0 2024-04-25T10:02:45.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.857+0000 7fb54b400640 1 --2- 172.21.15.165:0/2206782047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540cfcf0 0x7fb5540cde80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009870 tx=0x7fb53c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba404125b7ec1339 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.858+0000 7fb553400640 1 -- 172.21.15.165:0/2206782047 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c037070 con 0x7fb5540cfcf0 2024-04-25T10:02:45.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.858+0000 7fb553400640 1 -- 172.21.15.165:0/2206782047 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb53c032440 con 0x7fb5540cfcf0 2024-04-25T10:02:45.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.858+0000 7fb553400640 1 -- 172.21.15.165:0/2206782047 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb53c032810 con 0x7fb5540cfcf0 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.858+0000 7fb55b000640 1 -- 172.21.15.165:0/2206782047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540cfcf0 msgr2=0x7fb5540cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.858+0000 7fb55b000640 1 --2- 172.21.15.165:0/2206782047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540cfcf0 0x7fb5540cde80 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fb53c009870 tx=0x7fb53c02a210 comp rx=0 tx=0).stop 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.859+0000 7fb55b000640 1 -- 172.21.15.165:0/2206782047 shutdown_connections 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.859+0000 7fb55b000640 1 --2- 172.21.15.165:0/2206782047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540cfcf0 0x7fb5540cde80 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.859+0000 7fb55b000640 1 --2- 172.21.15.165:0/2206782047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb5540cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.859+0000 7fb55b000640 1 -- 172.21.15.165:0/2206782047 >> 172.21.15.165:0/2206782047 conn(0x7fb55406abe0 msgr2=0x7fb55406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:45.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.859+0000 7fb55b000640 1 -- 172.21.15.165:0/2206782047 shutdown_connections 2024-04-25T10:02:45.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.859+0000 7fb55b000640 1 -- 172.21.15.165:0/2206782047 wait complete. 2024-04-25T10:02:45.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.861+0000 7fb55b000640 1 Processor -- start 2024-04-25T10:02:45.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.861+0000 7fb55b000640 1 -- start start 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb55b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 0x7fb554149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb55b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5540cfcf0 0x7fb554151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb55b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb554151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb55b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554071230 con 0x7fb5540ce3c0 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb55b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554071d20 con 0x7fb5540cfcf0 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb55b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5541552c0 con 0x7fb554154ee0 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb553e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 0x7fb554149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb553e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 0x7fb554149e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45698/0 (socket says 172.21.15.165:45698) 2024-04-25T10:02:45.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.862+0000 7fb553e00640 1 -- 172.21.15.165:0/796359947 learned_addr learned my addr 172.21.15.165:0/796359947 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb559000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb554151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb54b400640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5540cfcf0 0x7fb554151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb553e00640 1 -- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 msgr2=0x7fb554151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb553e00640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb554151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb553e00640 1 -- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5540cfcf0 msgr2=0x7fb554151410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb553e00640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5540cfcf0 0x7fb554151410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb553e00640 1 -- 172.21.15.165:0/796359947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55414a340 con 0x7fb5540ce3c0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb54b400640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5540cfcf0 0x7fb554151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb553e00640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 0x7fb554149e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb544007e40 tx=0x7fb54400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544014620 con 0x7fb5540ce3c0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.863+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544019070 con 0x7fb5540ce3c0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.864+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb554133e90 con 0x7fb5540ce3c0 2024-04-25T10:02:45.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.864+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544014a00 con 0x7fb5540ce3c0 2024-04-25T10:02:45.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.864+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb554154390 con 0x7fb5540ce3c0 2024-04-25T10:02:45.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.865+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554073990 con 0x7fb5540ce3c0 2024-04-25T10:02:45.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.866+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb544014ba0 con 0x7fb5540ce3c0 2024-04-25T10:02:45.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.866+0000 7fb550c00640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb53006b4d0 0x7fb53006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:45.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.867+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb54408af60 con 0x7fb5540ce3c0 2024-04-25T10:02:45.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.867+0000 7fb54b400640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb53006b4d0 0x7fb53006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:45.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.868+0000 7fb54b400640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb53006b4d0 0x7fb53006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb53c034000 tx=0x7fb53c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:45.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:45.871+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb554133e90 con 0x7fb5540ce3c0 2024-04-25T10:02:46.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.115+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb55406d360 con 0x7fb5540ce3c0 2024-04-25T10:02:46.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.116+0000 7fb550c00640 1 -- 172.21.15.165:0/796359947 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fb544059940 con 0x7fb5540ce3c0 2024-04-25T10:02:46.121 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:46.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb53006b4d0 msgr2=0x7fb53006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb53006b4d0 0x7fb53006d990 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb53c034000 tx=0x7fb53c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 msgr2=0x7fb554149e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 0x7fb554149e00 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fb544007e40 tx=0x7fb54400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 shutdown_connections 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb53006b4d0 0x7fb53006d990 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb554154ee0 0x7fb554151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5540cfcf0 0x7fb554151410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 --2- 172.21.15.165:0/796359947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5540ce3c0 0x7fb554149e00 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 >> 172.21.15.165:0/796359947 conn(0x7fb55406abe0 msgr2=0x7fb55414cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.119+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 shutdown_connections 2024-04-25T10:02:46.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:46.120+0000 7fb55b000640 1 -- 172.21.15.165:0/796359947 wait complete. 2024-04-25T10:02:46.142 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:46.142 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:46.142 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-25T10:02:46.143 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-25T10:02:46.143 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:46.143 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:46.143 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:46.143 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:46.143 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:46.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.143 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-25T10:02:46.201 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42590sec preferred_lft 42590sec 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-25T10:02:46.202 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:46.203 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-25T10:02:46.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.203 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:46.203 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:46.203 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add name ceph-brx type bridge 2024-04-25T10:02:46.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr flush dev ceph-brx 2024-04-25T10:02:46.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set ceph-brx up 2024-04-25T10:02:46.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-25T10:02:46.204 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:46.628 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.629 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:46.696 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-25T10:02:46.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.700 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-25T10:02:46.772 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:46.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:46.773 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:46.773 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:46.773 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:46.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:46.774 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:46.774 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:47.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.471 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-25T10:02:47.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.528 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-04-25T10:02:47.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.581 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:47.581 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:47.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:47.581 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-25T10:02:47.581 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:47.703 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-25T10:02:47.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-25T10:02:47.704 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:47.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:47.929 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:47.929 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:47.929 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.0 up 2024-04-25T10:02:47.929 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.0 master ceph-brx 2024-04-25T10:02:47.929 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:48.056 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:48.056 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:48.097 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-25T10:02:48.098 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:48.098 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:48.191 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:48.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:48.192 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:48.327 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:48.327 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:02:48.327 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-25T10:02:49.058 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:49.058 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:02:49.058 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-25T10:02:49.059 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:49.059 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:49.094 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-25T10:02:49.094 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-25T10:02:49.094 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-25T10:02:49.094 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:49.094 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:49.094 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:02:49.095 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:02:49.095 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-25T10:02:49.097 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:49.098 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:49.098 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:49.098 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:49.103 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:49.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.104 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-25T10:02:49.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.185 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:49.241 INFO:teuthology.orchestra.run.smithi172.stdout:client4860 2024-04-25T10:02:49.241 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:02:49.242 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:49.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.293 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:49.350 INFO:teuthology.orchestra.run.smithi172.stdout:client4860 2024-04-25T10:02:49.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.351 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:49.408 INFO:teuthology.orchestra.run.smithi172.stdout:client4860 2024-04-25T10:02:49.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.409 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/55d22d87-b25b-4ccd-bc45-d797a973b3c0.client4860/status') 2024-04-25T10:02:49.483 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4860 (3)172.21.15.172:0/4278557719 2024-04-25T10:02:49.484 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2024-04-25T10:02:49.484 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2024-04-25T10:02:49.484 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2024-04-25T10:02:49.484 INFO:teuthology.orchestra.run.smithi172.stderr:71 bytes copied, 0.000109518 s, 648 kB/s 2024-04-25T10:02:49.487 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-04-25T10:02:49.487 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-25T10:02:49.487 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:49.488 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:02:49.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.694+0000 7f39baa00640 1 Processor -- start 2024-04-25T10:02:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.694+0000 7f39baa00640 1 -- start start 2024-04-25T10:02:49.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39baa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f39b40ce920 0x7f39b40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39baa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b4144b30 con 0x7f39b406d290 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39baa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39b40712d0 con 0x7f39b4144750 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39baa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39b40715b0 con 0x7f39b40ce920 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39b3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f39b40ce920 0x7f39b40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39b3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f39b40ce920 0x7f39b40cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56516/0 (socket says 172.21.15.165:56516) 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39b8a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f39b406d290 0x7f39b406d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37086/0 (socket says 172.21.15.165:37086) 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.696+0000 7f39b3400640 1 -- 172.21.15.165:0/3787175595 learned_addr learned my addr 172.21.15.165:0/3787175595 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.697+0000 7f39b3400640 1 -- 172.21.15.165:0/3787175595 >> v1:172.21.15.172:6789/0 conn(0x7f39b4144750 legacy=0x7f39b406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.697+0000 7f39b3400640 1 -- 172.21.15.165:0/3787175595 >> v1:172.21.15.165:6789/0 conn(0x7f39b406d290 legacy=0x7f39b406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.697+0000 7f39b3400640 1 -- 172.21.15.165:0/3787175595 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b4071890 con 0x7f39b40ce920 2024-04-25T10:02:49.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.698+0000 7f39b3400640 1 --2- 172.21.15.165:0/3787175595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b40ce920 0x7f39b40cedb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009870 tx=0x7f39a002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96a7a26ed496272a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.698+0000 7f39b2a00640 1 -- 172.21.15.165:0/3787175595 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0037070 con 0x7f39b40ce920 2024-04-25T10:02:49.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.698+0000 7f39b2a00640 1 -- 172.21.15.165:0/3787175595 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a0032440 con 0x7f39b40ce920 2024-04-25T10:02:49.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.698+0000 7f39b2a00640 1 -- 172.21.15.165:0/3787175595 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0032810 con 0x7f39b40ce920 2024-04-25T10:02:49.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 -- 172.21.15.165:0/3787175595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b40ce920 msgr2=0x7f39b40cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 --2- 172.21.15.165:0/3787175595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b40ce920 0x7f39b40cedb0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009870 tx=0x7f39a002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 -- 172.21.15.165:0/3787175595 shutdown_connections 2024-04-25T10:02:49.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 --2- 172.21.15.165:0/3787175595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b40ce920 0x7f39b40cedb0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 -- 172.21.15.165:0/3787175595 >> 172.21.15.165:0/3787175595 conn(0x7f39b406abe0 msgr2=0x7f39b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 -- 172.21.15.165:0/3787175595 shutdown_connections 2024-04-25T10:02:49.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.699+0000 7f39baa00640 1 -- 172.21.15.165:0/3787175595 wait complete. 2024-04-25T10:02:49.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.700+0000 7f39baa00640 1 Processor -- start 2024-04-25T10:02:49.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.701+0000 7f39baa00640 1 -- start start 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 0x7f39b40af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39baa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b40ce920 0x7f39b413d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39baa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b4144750 0x7f39b40b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39baa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b406f020 con 0x7f39b406d290 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39baa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b406ece0 con 0x7f39b40ce920 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39baa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4145450 con 0x7f39b4144750 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 0x7f39b40af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 0x7f39b40af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56528/0 (socket says 172.21.15.165:56528) 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 -- 172.21.15.165:0/2867023469 learned_addr learned my addr 172.21.15.165:0/2867023469 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 -- 172.21.15.165:0/2867023469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b4144750 msgr2=0x7f39b40b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b4144750 0x7f39b40b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 -- 172.21.15.165:0/2867023469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b40ce920 msgr2=0x7f39b413d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b40ce920 0x7f39b413d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.702+0000 7f39b3e00640 1 -- 172.21.15.165:0/2867023469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39b40b4480 con 0x7f39b406d290 2024-04-25T10:02:49.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.703+0000 7f39b3e00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 0x7f39b40af2c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c430 tx=0x7f39a400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.703+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4011790 con 0x7f39b406d290 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.703+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39a4017070 con 0x7f39b406d290 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.703+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a4011bb0 con 0x7f39b406d290 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.703+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b40acac0 con 0x7f39b406d290 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.703+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b40ad080 con 0x7f39b406d290 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.704+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39b40ac8b0 con 0x7f39b406d290 2024-04-25T10:02:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.705+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f39a401c460 con 0x7f39b406d290 2024-04-25T10:02:49.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.706+0000 7f39abe00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f399406b4d0 0x7f399406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:49.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.706+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f39a408a470 con 0x7f39b406d290 2024-04-25T10:02:49.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.706+0000 7f39b3400640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f399406b4d0 0x7f399406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.707+0000 7f39b3400640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f399406b4d0 0x7f399406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39a0034000 tx=0x7f39a002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:49.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.711+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f39a4058ef0 con 0x7f39b406d290 2024-04-25T10:02:49.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.949+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f39b4073990 con 0x7f39b406d290 2024-04-25T10:02:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.950+0000 7f39abe00640 1 -- 172.21.15.165:0/2867023469 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f39a40587f0 con 0x7f39b406d290 2024-04-25T10:02:49.954 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-25T10:02:49.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.952+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f399406b4d0 msgr2=0x7f399406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.952+0000 7f39baa00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f399406b4d0 0x7f399406d990 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f39a0034000 tx=0x7f39a002c000 comp rx=0 tx=0).stop 2024-04-25T10:02:49.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.952+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 msgr2=0x7f39b40af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:49.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.952+0000 7f39baa00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 0x7f39b40af2c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c430 tx=0x7f39a400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 shutdown_connections 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f399406b4d0 0x7f399406d990 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f39b4144750 0x7f39b40b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f39b40ce920 0x7f39b413d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 --2- 172.21.15.165:0/2867023469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f39b406d290 0x7f39b40af2c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 >> 172.21.15.165:0/2867023469 conn(0x7f39b406abe0 msgr2=0x7f39b406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 shutdown_connections 2024-04-25T10:02:49.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:49.953+0000 7f39baa00640 1 -- 172.21.15.165:0/2867023469 wait complete. 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi172.front.sepia.ceph.com 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:02:49.977 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:49.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:49.977 DEBUG:teuthology.orchestra.run.smithi172:> ip addr 2024-04-25T10:02:50.035 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:50.035 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:50.035 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:50.035 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.035 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2c brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f0 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:d9:17:2d brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: altname enp7s0f1 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:90 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 0c:c4:7a:88:7f:91 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: inet 172.21.15.172/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft 42586sec preferred_lft 42586sec 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::ec4:7aff:fe88:7f91/64 scope link 2024-04-25T10:02:50.036 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5a:ae:a0:38:c9:74 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::40d2:3bff:fe0a:3cfb/64 scope link 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether 5a:ae:a0:38:c9:74 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::58ae:a0ff:fe38:c974/64 scope link 2024-04-25T10:02:50.037 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.038 DEBUG:teuthology.orchestra.run.smithi172:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:02:50.114 INFO:teuthology.orchestra.run.smithi172.stdout:1 2024-04-25T10:02:50.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.117 DEBUG:teuthology.orchestra.run.smithi172:> route 2024-04-25T10:02:50.166 INFO:teuthology.orchestra.run.smithi172.stdout:Kernel IP routing table 2024-04-25T10:02:50.166 INFO:teuthology.orchestra.run.smithi172.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:02:50.166 INFO:teuthology.orchestra.run.smithi172.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:02:50.166 INFO:teuthology.orchestra.run.smithi172.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:02:50.166 INFO:teuthology.orchestra.run.smithi172.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:02:50.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.167 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:50.167 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:50.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:02:50.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:02:50.168 DEBUG:teuthology.orchestra.run.smithi172:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:02:50.168 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:50.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.320 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-25T10:02:50.370 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:02:50.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.371 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list-id 2024-04-25T10:02:50.421 INFO:teuthology.orchestra.run.smithi172.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-04-25T10:02:50.422 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.422 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:50.422 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:50.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:02:50.423 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-04-25T10:02:50.423 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:50.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.537 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:50.611 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether b6:7a:4b:03:62:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::b47a:4bff:fe03:620f/64 scope link 2024-04-25T10:02:50.612 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.617 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-04-25T10:02:50.700 INFO:teuthology.orchestra.run.smithi172.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:02:50.700 INFO:teuthology.orchestra.run.smithi172.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:02:50.700 INFO:teuthology.orchestra.run.smithi172.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:02:50.700 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.700 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 ::1/128 scope host 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout: link/ether b6:7a:4b:03:62:0f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout: inet6 fe80::b47a:4bff:fe03:620f/64 scope link 2024-04-25T10:02:50.701 INFO:teuthology.orchestra.run.smithi172.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:02:50.705 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-04-25T10:02:50.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.705 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:50.705 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:50.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-04-25T10:02:50.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-04-25T10:02:50.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-04-25T10:02:50.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-04-25T10:02:50.706 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-04-25T10:02:50.707 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:50.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:50.925 DEBUG:teuthology.orchestra.run.smithi172:> (cd / && exec bash -c ' 2024-04-25T10:02:50.925 DEBUG:teuthology.orchestra.run.smithi172:> set -e 2024-04-25T10:02:50.925 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set brx.1 up 2024-04-25T10:02:50.926 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link set dev brx.1 master ceph-brx 2024-04-25T10:02:50.926 DEBUG:teuthology.orchestra.run.smithi172:> ') 2024-04-25T10:02:51.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:51.074 DEBUG:teuthology.orchestra.run.smithi172:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run.smithi172.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-04-25T10:02:51.121 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:02:51.121 DEBUG:teuthology.orchestra.run.smithi172:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.167 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:02:51.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.168 DEBUG:teuthology.orchestra.run.smithi172:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.338 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.338 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:02:51.338 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-25T10:02:51.356 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.356 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:02:51.356 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-25T10:02:51.356 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:51.356 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:invalid new device string format 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:02:51.383 INFO:teuthology.orchestra.run.smithi172.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:02:51.385 INFO:teuthology.orchestra.run.smithi172.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-04-25T10:02:51.385 INFO:teuthology.orchestra.run.smithi172.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:02:51.385 INFO:teuthology.orchestra.run.smithi172.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:02:51.386 INFO:teuthology.orchestra.run.smithi172.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:02:51.386 INFO:teuthology.orchestra.run.smithi172.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:02:51.391 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:02:51.391 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.391 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-04-25T10:02:51.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.472 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:51.529 INFO:teuthology.orchestra.run.smithi172.stdout:client4872 2024-04-25T10:02:51.529 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:02:51.529 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:02:51.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.581 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:51.635 INFO:teuthology.orchestra.run.smithi172.stdout:client4872 2024-04-25T10:02:51.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.635 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:02:51.686 INFO:teuthology.orchestra.run.smithi172.stdout:client4872 2024-04-25T10:02:51.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:02:51.687 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/55d22d87-b25b-4ccd-bc45-d797a973b3c0.client4872/status') 2024-04-25T10:02:51.761 INFO:teuthology.orchestra.run.smithi172.stdout:instance: client.4872 (3)172.21.15.172:0/1500700373 2024-04-25T10:02:51.762 INFO:teuthology.orchestra.run.smithi172.stdout:blocklisted: false 2024-04-25T10:02:51.762 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records in 2024-04-25T10:02:51.762 INFO:teuthology.orchestra.run.smithi172.stderr:0+1 records out 2024-04-25T10:02:51.762 INFO:teuthology.orchestra.run.smithi172.stderr:71 bytes copied, 7.9587e-05 s, 892 kB/s 2024-04-25T10:02:51.765 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-04-25T10:02:51.784 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:52.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.995+0000 7f4484c00640 1 Processor -- start 2024-04-25T10:02:52.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.995+0000 7f4484c00640 1 -- start start 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.997+0000 7f4484c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f4480070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.997+0000 7f4484c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448015bf30 con 0x7f44800cda30 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.997+0000 7f4484c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f448015c210 con 0x7f4480071530 2024-04-25T10:02:52.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.997+0000 7f4484c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f448015c860 con 0x7f4480144440 2024-04-25T10:02:52.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.997+0000 7f447fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f44800cda30 0x7f448015ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37102/0 (socket says 172.21.15.165:37102) 2024-04-25T10:02:52.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.997+0000 7f447fe00640 1 -- 172.21.15.165:0/666847179 learned_addr learned my addr 172.21.15.165:0/666847179 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:52.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447f400640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f4480070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262298602 0 0) 0x7f448015bf30 con 0x7f44800cda30 2024-04-25T10:02:52.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4464003520 con 0x7f44800cda30 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2120818567 0 0) 0x7f4464003520 con 0x7f44800cda30 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 msgr2=0x7f4480070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f4480070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 >> v1:172.21.15.172:6789/0 conn(0x7f4480071530 legacy=0x7f44800cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f448015c4f0 con 0x7f44800cda30 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f44780019f0 con 0x7f44800cda30 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 >> v1:172.21.15.165:6789/0 conn(0x7f44800cda30 legacy=0x7f448015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44640039e0 0x7f4464003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.998+0000 7f447e000640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4464004800 0x7f4464004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447e000640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f44640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f448015c860 con 0x7f4464004800 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f446400bb30 con 0x7f44640039e0 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f446400bd30 con 0x7f4480144440 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447ea00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4464004800 0x7f4464004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447f400640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44640039e0 0x7f4464003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447fe00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f44640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447ea00640 1 -- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 msgr2=0x7f44640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447ea00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f44640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447ea00640 1 -- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44640039e0 msgr2=0x7f4464003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447ea00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f44640039e0 0x7f4464003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447ea00640 1 -- 172.21.15.165:0/666847179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4464003520 con 0x7f4464004800 2024-04-25T10:02:52.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:51.999+0000 7f447fe00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4480144440 0x7f44640051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:52.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.000+0000 7f447ea00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4464004800 0x7f4464004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f446c001720 tx=0x7f4464005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afc6c6df873c9c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c001370 con 0x7f4464004800 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c0052d0 con 0x7f4464004800 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f447e000640 1 -- 172.21.15.165:0/666847179 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c02ab90 con 0x7f4464004800 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f4484c00640 1 -- 172.21.15.165:0/666847179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4464004800 msgr2=0x7f4464004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f4484c00640 1 --2- 172.21.15.165:0/666847179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4464004800 0x7f4464004c80 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f446c001720 tx=0x7f4464005830 comp rx=0 tx=0).stop 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f447fe00640 1 -- 172.21.15.165:0/666847179 reap_dead start 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f4484c00640 1 -- 172.21.15.165:0/666847179 shutdown_connections 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.001+0000 7f4484c00640 1 -- 172.21.15.165:0/666847179 >> 172.21.15.165:0/666847179 conn(0x7f448006abe0 msgr2=0x7f448006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.002+0000 7f4484c00640 1 -- 172.21.15.165:0/666847179 shutdown_connections 2024-04-25T10:02:52.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.002+0000 7f4484c00640 1 -- 172.21.15.165:0/666847179 wait complete. 2024-04-25T10:02:52.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.004+0000 7f4484c00640 1 Processor -- start 2024-04-25T10:02:52.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.004+0000 7f4484c00640 1 -- start start 2024-04-25T10:02:52.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.005+0000 7f4484c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4464004800 0x7f44801f7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.005+0000 7f4484c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 0x7f44801f3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.005+0000 7f4484c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44800c9650 0x7f44800c9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.005+0000 7f4484c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44800cdea0 con 0x7f44801336e0 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.005+0000 7f4484c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4480071970 con 0x7f4464004800 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.005+0000 7f4484c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4480070ff0 con 0x7f44800c9650 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 0x7f44801f3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 0x7f44801f3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56550/0 (socket says 172.21.15.165:56550) 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 -- 172.21.15.165:0/497111231 learned_addr learned my addr 172.21.15.165:0/497111231 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447fe00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44800c9650 0x7f44800c9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447f400640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4464004800 0x7f44801f7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 -- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44800c9650 msgr2=0x7f44800c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44800c9650 0x7f44800c9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 -- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4464004800 msgr2=0x7f44801f7970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4464004800 0x7f44801f7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 -- 172.21.15.165:0/497111231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4480133bf0 con 0x7f44801336e0 2024-04-25T10:02:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447fe00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44800c9650 0x7f44800c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.006+0000 7f447ea00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 0x7f44801f3370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f446c001840 tx=0x7f446c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.007+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c037040 con 0x7f44801336e0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.007+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f446c005480 con 0x7f44801336e0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.007+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4480133ed0 con 0x7f44801336e0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.007+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f446c007470 con 0x7f44801336e0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.007+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4480134490 con 0x7f44801336e0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.008+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4480073990 con 0x7f44801336e0 2024-04-25T10:02:52.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.009+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f446c044020 con 0x7f44801336e0 2024-04-25T10:02:52.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.011+0000 7f4477400640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f446006b500 0x7f446006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.011+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f446c0aa2a0 con 0x7f44801336e0 2024-04-25T10:02:52.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.011+0000 7f447f400640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f446006b500 0x7f446006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.012+0000 7f447f400640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f446006b500 0x7f446006d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4470009950 tx=0x7f4470008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.016+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f446c078d20 con 0x7f44801336e0 2024-04-25T10:02:52.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.220+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44800cea70 con 0x7f44801336e0 2024-04-25T10:02:52.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.222+0000 7f4477400640 1 -- 172.21.15.165:0/497111231 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f446c078620 con 0x7f44801336e0 2024-04-25T10:02:52.227 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:52.227 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:52.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f446006b500 msgr2=0x7f446006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f446006b500 0x7f446006d9c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4470009950 tx=0x7f4470008040 comp rx=0 tx=0).stop 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 msgr2=0x7f44801f3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 0x7f44801f3370 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f446c001840 tx=0x7f446c001120 comp rx=0 tx=0).stop 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 shutdown_connections 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f446006b500 0x7f446006d9c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f44800c9650 0x7f44800c9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f44801336e0 0x7f44801f3370 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 --2- 172.21.15.165:0/497111231 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4464004800 0x7f44801f7970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 >> 172.21.15.165:0/497111231 conn(0x7f448006abe0 msgr2=0x7f448015adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 shutdown_connections 2024-04-25T10:02:52.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.225+0000 7f4484c00640 1 -- 172.21.15.165:0/497111231 wait complete. 2024-04-25T10:02:52.247 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:52.248 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:52.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.435+0000 7f5192200640 1 Processor -- start 2024-04-25T10:02:52.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.435+0000 7f5192200640 1 -- start start 2024-04-25T10:02:52.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.436+0000 7f5192200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f518c144d50 0x7f518c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.436+0000 7f5192200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518c1452a0 0x7f518c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.436+0000 7f5192200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f518c0cccc0 con 0x7f518c1446d0 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.436+0000 7f5192200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c0b2b20 con 0x7f518c144d50 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.436+0000 7f5192200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f518c06d400 con 0x7f518c1452a0 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f518c144d50 0x7f518c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f518c144d50 0x7f518c06d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56558/0 (socket says 172.21.15.165:56558) 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 -- 172.21.15.165:0/3401220580 learned_addr learned my addr 172.21.15.165:0/3401220580 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f518be00640 1 --2- 172.21.15.165:0/3401220580 >> v2:172.21.15.172:3300/0 conn(0x7f518c1452a0 0x7f518c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 -- 172.21.15.165:0/3401220580 >> v2:172.21.15.172:3300/0 conn(0x7f518c1452a0 msgr2=0x7f518c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 --2- 172.21.15.165:0/3401220580 >> v2:172.21.15.172:3300/0 conn(0x7f518c1452a0 0x7f518c06ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 -- 172.21.15.165:0/3401220580 >> v1:172.21.15.172:6789/0 conn(0x7f518c1446d0 legacy=0x7f518c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.437+0000 7f5182a00640 1 -- 172.21.15.165:0/3401220580 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0cc140 con 0x7f518c144d50 2024-04-25T10:02:52.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.438+0000 7f5182a00640 1 --2- 172.21.15.165:0/3401220580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c144d50 0x7f518c06d890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f518c15bfe0 tx=0x7f517402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5936699aa1fe215 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f518aa00640 1 -- 172.21.15.165:0/3401220580 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174036070 con 0x7f518c144d50 2024-04-25T10:02:52.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f518aa00640 1 -- 172.21.15.165:0/3401220580 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174031600 con 0x7f518c144d50 2024-04-25T10:02:52.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f518aa00640 1 -- 172.21.15.165:0/3401220580 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740319d0 con 0x7f518c144d50 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 -- 172.21.15.165:0/3401220580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c144d50 msgr2=0x7f518c06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 --2- 172.21.15.165:0/3401220580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c144d50 0x7f518c06d890 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f518c15bfe0 tx=0x7f517402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 -- 172.21.15.165:0/3401220580 shutdown_connections 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 --2- 172.21.15.165:0/3401220580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c144d50 0x7f518c06d890 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 --2- 172.21.15.165:0/3401220580 >> v2:172.21.15.172:3300/0 conn(0x7f518c1452a0 0x7f518c06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 -- 172.21.15.165:0/3401220580 >> 172.21.15.165:0/3401220580 conn(0x7f518c06abe0 msgr2=0x7f518c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 -- 172.21.15.165:0/3401220580 shutdown_connections 2024-04-25T10:02:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.439+0000 7f5192200640 1 -- 172.21.15.165:0/3401220580 wait complete. 2024-04-25T10:02:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.441+0000 7f5192200640 1 Processor -- start 2024-04-25T10:02:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.441+0000 7f5192200640 1 -- start start 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f5192200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c1446d0 0x7f518c13ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f5192200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c144d50 0x7f518c13cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f5192200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 0x7f518c13d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f5192200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cc9e0 con 0x7f518c1452a0 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f5192200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518c0cc700 con 0x7f518c144d50 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f5192200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f518c144ab0 con 0x7f518c1446d0 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.442+0000 7f518be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 0x7f518c13d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 0x7f518c13d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56564/0 (socket says 172.21.15.165:56564) 2024-04-25T10:02:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 -- 172.21.15.165:0/2091387073 learned_addr learned my addr 172.21.15.165:0/2091387073 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518b400640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c1446d0 0x7f518c13ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f5182a00640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c144d50 0x7f518c13cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 -- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c1446d0 msgr2=0x7f518c13ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c1446d0 0x7f518c13ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 -- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c144d50 msgr2=0x7f518c13cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c144d50 0x7f518c13cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 -- 172.21.15.165:0/2091387073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c0b1040 con 0x7f518c1452a0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.443+0000 7f518be00640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 0x7f518c13d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f518400c820 tx=0x7f518400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.444+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184015070 con 0x7f518c1452a0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.444+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5184004020 con 0x7f518c1452a0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.444+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5184004410 con 0x7f518c1452a0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.444+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f518c0b1320 con 0x7f518c1452a0 2024-04-25T10:02:52.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.444+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f518c0b18e0 con 0x7f518c1452a0 2024-04-25T10:02:52.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.445+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518c073a00 con 0x7f518c1452a0 2024-04-25T10:02:52.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.446+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51840077d0 con 0x7f518c1452a0 2024-04-25T10:02:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.447+0000 7f5183e00640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516806b4d0 0x7f516806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.447+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f518408a3c0 con 0x7f518c1452a0 2024-04-25T10:02:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.447+0000 7f518b400640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516806b4d0 0x7f516806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.448+0000 7f518b400640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516806b4d0 0x7f516806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f517c006fd0 tx=0x7f517c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.452+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5184058e30 con 0x7f518c1452a0 2024-04-25T10:02:52.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.648+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f518c0cd740 con 0x7f518c1452a0 2024-04-25T10:02:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.649+0000 7f5183e00640 1 -- 172.21.15.165:0/2091387073 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5184058730 con 0x7f518c1452a0 2024-04-25T10:02:52.654 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516806b4d0 msgr2=0x7f516806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516806b4d0 0x7f516806d990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f517c006fd0 tx=0x7f517c008040 comp rx=0 tx=0).stop 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 msgr2=0x7f518c13d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 0x7f518c13d300 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f518400c820 tx=0x7f518400cce0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 shutdown_connections 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516806b4d0 0x7f516806d990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f518c1452a0 0x7f518c13d300 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518c144d50 0x7f518c13cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 --2- 172.21.15.165:0/2091387073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518c1446d0 0x7f518c13ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 >> 172.21.15.165:0/2091387073 conn(0x7f518c06abe0 msgr2=0x7f518c15b0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 shutdown_connections 2024-04-25T10:02:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.652+0000 7f5192200640 1 -- 172.21.15.165:0/2091387073 wait complete. 2024-04-25T10:02:52.676 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:52.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.866+0000 7f7d90a00640 1 Processor -- start 2024-04-25T10:02:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.866+0000 7f7d90a00640 1 -- start start 2024-04-25T10:02:52.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.867+0000 7f7d90a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7d8c145140 0x7f7d8c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d90a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8c0c5650 con 0x7f7d8c0ce950 2024-04-25T10:02:52.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d90a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d8c145520 con 0x7f7d8c0c5200 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d90a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d8c0ced90 con 0x7f7d8c145140 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d82a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7d8c0ce950 0x7f7d8c0c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37104/0 (socket says 172.21.15.165:37104) 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d82a00640 1 -- 172.21.15.165:0/1847185834 learned_addr learned my addr 172.21.15.165:0/1847185834 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1143041725 0 0) 0x7f7d8c0c5650 con 0x7f7d8c0ce950 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d8b400640 1 --2- 172.21.15.165:0/1847185834 >> v2:172.21.15.172:3300/0 conn(0x7f7d8c145140 0x7f7d8c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.868+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d70003520 con 0x7f7d8c0ce950 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3737590216 0 0) 0x7f7d70003520 con 0x7f7d8c0ce950 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 >> v2:172.21.15.172:3300/0 conn(0x7f7d8c145140 msgr2=0x7f7d8c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 --2- 172.21.15.165:0/1847185834 >> v2:172.21.15.172:3300/0 conn(0x7f7d8c145140 0x7f7d8c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 >> v1:172.21.15.172:6789/0 conn(0x7f7d8c0c5200 legacy=0x7f7d8c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c071270 con 0x7f7d8c0ce950 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7d74001670 con 0x7f7d8c0ce950 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 >> v1:172.21.15.165:6789/0 conn(0x7f7d8c0ce950 legacy=0x7f7d8c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d70003a00 0x7f7d70003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d70005290 0x7f7d7000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c0ced90 con 0x7f7d70004800 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7000c1e0 con 0x7f7d70003a00 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7000c3e0 con 0x7f7d70005290 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.869+0000 7f7d82a00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d8b400640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d70003a00 0x7f7d70003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d82a00640 1 -- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d70005290 msgr2=0x7f7d7000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d82a00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d70005290 0x7f7d7000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d82a00640 1 -- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d70003a00 msgr2=0x7f7d70003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d82a00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d70003a00 0x7f7d70003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d82a00640 1 -- 172.21.15.165:0/1847185834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d70003520 con 0x7f7d70004800 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d8b400640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d70003a00 0x7f7d70003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.870+0000 7f7d8be00640 1 -- 172.21.15.165:0/1847185834 reap_dead start 2024-04-25T10:02:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.871+0000 7f7d82a00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d70004c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c110 tx=0x7f7d70005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29975c25f9e8c2f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.871+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74007c40 con 0x7f7d70004800 2024-04-25T10:02:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.871+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d74005ad0 con 0x7f7d70004800 2024-04-25T10:02:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.871+0000 7f7d8aa00640 1 -- 172.21.15.165:0/1847185834 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74007070 con 0x7f7d70004800 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.871+0000 7f7d90a00640 1 -- 172.21.15.165:0/1847185834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 msgr2=0x7f7d70004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.871+0000 7f7d90a00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d70004c80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400c110 tx=0x7f7d70005830 comp rx=0 tx=0).stop 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.872+0000 7f7d90a00640 1 -- 172.21.15.165:0/1847185834 shutdown_connections 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.872+0000 7f7d90a00640 1 --2- 172.21.15.165:0/1847185834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d70004c80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.872+0000 7f7d90a00640 1 -- 172.21.15.165:0/1847185834 >> 172.21.15.165:0/1847185834 conn(0x7f7d8c061cc0 msgr2=0x7f7d8c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.872+0000 7f7d90a00640 1 -- 172.21.15.165:0/1847185834 shutdown_connections 2024-04-25T10:02:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.872+0000 7f7d90a00640 1 -- 172.21.15.165:0/1847185834 wait complete. 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.873+0000 7f7d90a00640 1 Processor -- start 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.874+0000 7f7d90a00640 1 -- start start 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.874+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d8c1e68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d8c1e89e0 0x7f7d8c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d90a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d8c1e8030 0x7f7d8c0caca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d90a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c071b10 con 0x7f7d70004800 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d90a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c071830 con 0x7f7d8c1e8030 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d90a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8c0ce410 con 0x7f7d8c1e89e0 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d8c1e68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d8c1e68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56580/0 (socket says 172.21.15.165:56580) 2024-04-25T10:02:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 -- 172.21.15.165:0/3091091046 learned_addr learned my addr 172.21.15.165:0/3091091046 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:52.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d82a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d8c1e89e0 0x7f7d8c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8be00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d8c1e8030 0x7f7d8c0caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 -- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d8c1e89e0 msgr2=0x7f7d8c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d8c1e89e0 0x7f7d8c15ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 -- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d8c1e8030 msgr2=0x7f7d8c0caca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d8c1e8030 0x7f7d8c0caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.875+0000 7f7d8b400640 1 -- 172.21.15.165:0/3091091046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d8c15b290 con 0x7f7d70004800 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.876+0000 7f7d8b400640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d8c1e68b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e590 tx=0x7f7d7c00ea50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.876+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c017070 con 0x7f7d70004800 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.876+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d7c013610 con 0x7f7d70004800 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.876+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8c1e6f90 con 0x7f7d70004800 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.876+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7c00f040 con 0x7f7d70004800 2024-04-25T10:02:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.876+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8c1e7550 con 0x7f7d70004800 2024-04-25T10:02:52.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.877+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d8c073990 con 0x7f7d70004800 2024-04-25T10:02:52.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.878+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d7c01a070 con 0x7f7d70004800 2024-04-25T10:02:52.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.879+0000 7f7d83e00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d6806b4d0 0x7f7d6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:52.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.879+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7d7c08a130 con 0x7f7d70004800 2024-04-25T10:02:52.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.879+0000 7f7d82a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d6806b4d0 0x7f7d6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.880+0000 7f7d82a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d6806b4d0 0x7f7d6806d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d74032000 tx=0x7f7d74034040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:52.884+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d8c15b290 con 0x7f7d70004800 2024-04-25T10:02:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.080+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d8c15bbe0 con 0x7f7d70004800 2024-04-25T10:02:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.080+0000 7f7d83e00640 1 -- 172.21.15.165:0/3091091046 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d7c058b10 con 0x7f7d70004800 2024-04-25T10:02:53.085 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d6806b4d0 msgr2=0x7f7d6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d6806b4d0 0x7f7d6806d990 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7d74032000 tx=0x7f7d74034040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 msgr2=0x7f7d8c1e68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d8c1e68b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c00e590 tx=0x7f7d7c00ea50 comp rx=0 tx=0).stop 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 shutdown_connections 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d6806b4d0 0x7f7d6806d990 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d8c1e8030 0x7f7d8c0caca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d8c1e89e0 0x7f7d8c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 --2- 172.21.15.165:0/3091091046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d70004800 0x7f7d8c1e68b0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 >> 172.21.15.165:0/3091091046 conn(0x7f7d8c061cc0 msgr2=0x7f7d8c15da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 shutdown_connections 2024-04-25T10:02:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.083+0000 7f7d90a00640 1 -- 172.21.15.165:0/3091091046 wait complete. 2024-04-25T10:02:53.106 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.311+0000 7fa0ae000640 1 Processor -- start 2024-04-25T10:02:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.311+0000 7fa0ae000640 1 -- start start 2024-04-25T10:02:53.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0ae000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0a815b980 0x7fa0a80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0ae000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a8072280 con 0x7fa0a80ce230 2024-04-25T10:02:53.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0ae000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0a80714c0 con 0x7fa0a80cc4f0 2024-04-25T10:02:53.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0ae000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0a8071270 con 0x7fa0a815b980 2024-04-25T10:02:53.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0a6a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa0a80ce230 0x7fa0a815b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37120/0 (socket says 172.21.15.165:37120) 2024-04-25T10:02:53.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0a6a00640 1 -- 172.21.15.165:0/114246951 learned_addr learned my addr 172.21.15.165:0/114246951 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.313+0000 7fa0a7e00640 1 --2- 172.21.15.165:0/114246951 >> v2:172.21.15.172:3300/0 conn(0x7fa0a815b980 0x7fa0a80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495542183 0 0) 0x7fa0a8072280 con 0x7fa0a80ce230 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa08c003520 con 0x7fa0a80ce230 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1247677141 0 0) 0x7fa08c003520 con 0x7fa0a80ce230 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 >> v2:172.21.15.172:3300/0 conn(0x7fa0a815b980 msgr2=0x7fa0a80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 --2- 172.21.15.165:0/114246951 >> v2:172.21.15.172:3300/0 conn(0x7fa0a815b980 0x7fa0a80cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 >> v1:172.21.15.172:6789/0 conn(0x7fa0a80cc4f0 legacy=0x7fa0a80cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a80717a0 con 0x7fa0a80ce230 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa094001670 con 0x7fa0a80ce230 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 >> v1:172.21.15.165:6789/0 conn(0x7fa0a80ce230 legacy=0x7fa0a815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08c0039b0 0x7fa08c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c004800 0x7fa08c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8071270 con 0x7fa08c0051c0 2024-04-25T10:02:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c00c060 con 0x7fa08c0039b0 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.314+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08c00c260 con 0x7fa08c004800 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7e00640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7400640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08c0039b0 0x7fa08c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7e00640 1 -- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c004800 msgr2=0x7fa08c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7e00640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c004800 0x7fa08c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7e00640 1 -- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08c0039b0 msgr2=0x7fa08c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7e00640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa08c0039b0 0x7fa08c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.315+0000 7fa0a7e00640 1 -- 172.21.15.165:0/114246951 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c003520 con 0x7fa08c0051c0 2024-04-25T10:02:53.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.316+0000 7fa0a7e00640 1 -- 172.21.15.165:0/114246951 reap_dead start 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.316+0000 7fa0a7e00640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009fc0 tx=0x7fa08c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19c3688471366ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.316+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0009c10 con 0x7fa08c0051c0 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.316+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0a0003b90 con 0x7fa08c0051c0 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.316+0000 7fa0a6000640 1 -- 172.21.15.165:0/114246951 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0a0004020 con 0x7fa08c0051c0 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 -- 172.21.15.165:0/114246951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa08c0051c0 msgr2=0x7fa08c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0009fc0 tx=0x7fa08c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 -- 172.21.15.165:0/114246951 shutdown_connections 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 --2- 172.21.15.165:0/114246951 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa08c0051c0 0x7fa08c00b9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 -- 172.21.15.165:0/114246951 >> 172.21.15.165:0/114246951 conn(0x7fa0a806abe0 msgr2=0x7fa0a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 -- 172.21.15.165:0/114246951 shutdown_connections 2024-04-25T10:02:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.317+0000 7fa0ae000640 1 -- 172.21.15.165:0/114246951 wait complete. 2024-04-25T10:02:53.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.319+0000 7fa0ae000640 1 Processor -- start 2024-04-25T10:02:53.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.319+0000 7fa0ae000640 1 -- start start 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0ae000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c0051c0 0x7fa0a814f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0ae000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 0x7fa0a81f4380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0ae000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a81f6390 0x7fa0a81e72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0ae000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80cc8d0 con 0x7fa0a81f5f10 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0ae000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a8071d60 con 0x7fa0a81f6390 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0ae000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a80cca50 con 0x7fa08c0051c0 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 0x7fa0a81f4380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 0x7fa0a81f4380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56584/0 (socket says 172.21.15.165:56584) 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.320+0000 7fa0a6a00640 1 -- 172.21.15.165:0/1426148095 learned_addr learned my addr 172.21.15.165:0/1426148095 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a7e00640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a81f6390 0x7fa0a81e72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a7400640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c0051c0 0x7fa0a814f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a6a00640 1 -- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c0051c0 msgr2=0x7fa0a814f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a6a00640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c0051c0 0x7fa0a814f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a6a00640 1 -- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a81f6390 msgr2=0x7fa0a81e72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a6a00640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a81f6390 0x7fa0a81e72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a6a00640 1 -- 172.21.15.165:0/1426148095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0a814fb90 con 0x7fa0a81f5f10 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0a6a00640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 0x7fa0a81f4380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c700 tx=0x7fa09400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa094018070 con 0x7fa0a81f5f10 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0940072d0 con 0x7fa0a81f5f10 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0940047e0 con 0x7fa0a81f5f10 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.321+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a81f4e70 con 0x7fa0a81f5f10 2024-04-25T10:02:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.322+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a81f5430 con 0x7fa0a81f5f10 2024-04-25T10:02:53.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.323+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a8073a00 con 0x7fa0a81f5f10 2024-04-25T10:02:53.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.324+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa094004980 con 0x7fa0a81f5f10 2024-04-25T10:02:53.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.325+0000 7fa09f400640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa08806b500 0x7fa08806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.325+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa094089940 con 0x7fa0a81f5f10 2024-04-25T10:02:53.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.325+0000 7fa0a7400640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa08806b500 0x7fa08806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.325+0000 7fa0a7400640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa08806b500 0x7fa08806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa098007fd0 tx=0x7fa098009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.329+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0940583c0 con 0x7fa0a81f5f10 2024-04-25T10:02:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.519+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa0a806d3a0 con 0x7fa0a81f5f10 2024-04-25T10:02:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.521+0000 7fa09f400640 1 -- 172.21.15.165:0/1426148095 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa094057cc0 con 0x7fa0a81f5f10 2024-04-25T10:02:53.525 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.523+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa08806b500 msgr2=0x7fa08806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.523+0000 7fa0ae000640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa08806b500 0x7fa08806d9c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fa098007fd0 tx=0x7fa098009040 comp rx=0 tx=0).stop 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.523+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 msgr2=0x7fa0a81f4380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.523+0000 7fa0ae000640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 0x7fa0a81f4380 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa09400c700 tx=0x7fa09400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 shutdown_connections 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa08806b500 0x7fa08806d9c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0a81f6390 0x7fa0a81e72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0a81f5f10 0x7fa0a81f4380 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 --2- 172.21.15.165:0/1426148095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa08c0051c0 0x7fa0a814f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 >> 172.21.15.165:0/1426148095 conn(0x7fa0a806abe0 msgr2=0x7fa0a80cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 shutdown_connections 2024-04-25T10:02:53.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.524+0000 7fa0ae000640 1 -- 172.21.15.165:0/1426148095 wait complete. 2024-04-25T10:02:53.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:53.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.738+0000 7f6eb2200640 1 Processor -- start 2024-04-25T10:02:53.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.739+0000 7f6eb2200640 1 -- start start 2024-04-25T10:02:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb2200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6eac15dd30 0x7f6eac0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb2200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac15e110 con 0x7f6eac0b5b40 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb2200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac0b2fd0 con 0x7f6eac06bff0 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb2200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6eac06d000 con 0x7f6eac15dd30 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6eac15dd30 0x7f6eac0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6eac15dd30 0x7f6eac0cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56600/0 (socket says 172.21.15.165:56600) 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb1800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6eac0b5b40 0x7f6eac0d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37128/0 (socket says 172.21.15.165:37128) 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.740+0000 7f6eb0e00640 1 -- 172.21.15.165:0/2126601024 learned_addr learned my addr 172.21.15.165:0/2126601024 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:53.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.741+0000 7f6eb0e00640 1 -- 172.21.15.165:0/2126601024 >> v1:172.21.15.172:6789/0 conn(0x7f6eac06bff0 legacy=0x7f6eac0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:53.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.741+0000 7f6eb0e00640 1 -- 172.21.15.165:0/2126601024 >> v1:172.21.15.165:6789/0 conn(0x7f6eac0b5b40 legacy=0x7f6eac0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.741+0000 7f6eb0e00640 1 -- 172.21.15.165:0/2126601024 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac06c480 con 0x7f6eac15dd30 2024-04-25T10:02:53.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.741+0000 7f6eab400640 1 -- 172.21.15.165:0/2126601024 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997515773 0 0) 0x7f6eac15e110 con 0x7f6eac0b5b40 2024-04-25T10:02:53.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.742+0000 7f6eb0e00640 1 --2- 172.21.15.165:0/2126601024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac15dd30 0x7f6eac0cebd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009870 tx=0x7f6e9c02a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6a6074dc1c1e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.742+0000 7f6eab400640 1 -- 172.21.15.165:0/2126601024 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037070 con 0x7f6eac15dd30 2024-04-25T10:02:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.742+0000 7f6eab400640 1 -- 172.21.15.165:0/2126601024 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c032440 con 0x7f6eac15dd30 2024-04-25T10:02:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.742+0000 7f6eab400640 1 -- 172.21.15.165:0/2126601024 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c032810 con 0x7f6eac15dd30 2024-04-25T10:02:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.742+0000 7f6eb2200640 1 -- 172.21.15.165:0/2126601024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac15dd30 msgr2=0x7f6eac0cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.742+0000 7f6eb2200640 1 --2- 172.21.15.165:0/2126601024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac15dd30 0x7f6eac0cebd0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c009870 tx=0x7f6e9c02a2c0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.743+0000 7f6eb2200640 1 -- 172.21.15.165:0/2126601024 shutdown_connections 2024-04-25T10:02:53.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.743+0000 7f6eb2200640 1 --2- 172.21.15.165:0/2126601024 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac15dd30 0x7f6eac0cebd0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.743+0000 7f6eb2200640 1 -- 172.21.15.165:0/2126601024 >> 172.21.15.165:0/2126601024 conn(0x7f6eac0d0610 msgr2=0x7f6eac0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.743+0000 7f6eb2200640 1 -- 172.21.15.165:0/2126601024 shutdown_connections 2024-04-25T10:02:53.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.743+0000 7f6eb2200640 1 -- 172.21.15.165:0/2126601024 wait complete. 2024-04-25T10:02:53.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.744+0000 7f6eb2200640 1 Processor -- start 2024-04-25T10:02:53.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.744+0000 7f6eb2200640 1 -- start start 2024-04-25T10:02:53.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.745+0000 7f6eb2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6eac0b5b40 0x7f6eac156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.745+0000 7f6eb2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac15dd30 0x7f6eac0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.745+0000 7f6eb2200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 0x7f6eac159ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.745+0000 7f6eb2200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac06cd20 con 0x7f6eac157850 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.745+0000 7f6eb2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac06ca40 con 0x7f6eac0b5b40 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.745+0000 7f6eb2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0cf110 con 0x7f6eac15dd30 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 0x7f6eac159ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 0x7f6eac159ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56604/0 (socket says 172.21.15.165:56604) 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 -- 172.21.15.165:0/3405324426 learned_addr learned my addr 172.21.15.165:0/3405324426 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb0e00640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6eac0b5b40 0x7f6eac156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eabe00640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac15dd30 0x7f6eac0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 -- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac15dd30 msgr2=0x7f6eac0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac15dd30 0x7f6eac0cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 -- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6eac0b5b40 msgr2=0x7f6eac156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6eac0b5b40 0x7f6eac156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 -- 172.21.15.165:0/3405324426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac0cc440 con 0x7f6eac157850 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb0e00640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6eac0b5b40 0x7f6eac156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.746+0000 7f6eb1800640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 0x7f6eac159ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c700 tx=0x7f6ea400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.747+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4018070 con 0x7f6eac157850 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.747+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea400ce10 con 0x7f6eac157850 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.747+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac0cc720 con 0x7f6eac157850 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.747+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400a880 con 0x7f6eac157850 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.747+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac0ccce0 con 0x7f6eac157850 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.748+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac073990 con 0x7f6eac157850 2024-04-25T10:02:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.749+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ea401b070 con 0x7f6eac157850 2024-04-25T10:02:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.750+0000 7f6ea8c00640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.750+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f6ea4089b60 con 0x7f6eac157850 2024-04-25T10:02:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.750+0000 7f6eb0e00640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:53.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.751+0000 7f6eb0e00640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:53.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.754+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6ea40585d0 con 0x7f6eac157850 2024-04-25T10:02:53.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.951+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f6eac0d16c0 con 0x7f6eac157850 2024-04-25T10:02:53.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.952+0000 7f6ea8c00640 1 -- 172.21.15.165:0/3405324426 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6ea4057ed0 con 0x7f6eac157850 2024-04-25T10:02:53.957 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:53.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.955+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 msgr2=0x7f6e8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.955+0000 7f6eb2200640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:02:53.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.955+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 msgr2=0x7f6eac159ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:53.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.955+0000 7f6eb2200640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 0x7f6eac159ba0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6ea400c700 tx=0x7f6ea400cbc0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 shutdown_connections 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac157850 0x7f6eac159ba0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac15dd30 0x7f6eac0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 --2- 172.21.15.165:0/3405324426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6eac0b5b40 0x7f6eac156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 >> 172.21.15.165:0/3405324426 conn(0x7f6eac0d0610 msgr2=0x7f6eac15d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 shutdown_connections 2024-04-25T10:02:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:53.956+0000 7f6eb2200640 1 -- 172.21.15.165:0/3405324426 wait complete. 2024-04-25T10:02:53.978 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.164+0000 7f4721a00640 1 Processor -- start 2024-04-25T10:02:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.164+0000 7f4721a00640 1 -- start start 2024-04-25T10:02:54.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.166+0000 7f4721a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f471c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.166+0000 7f4721a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c071480 con 0x7f471c154ee0 2024-04-25T10:02:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.166+0000 7f4721a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471c071760 con 0x7f471c0ce3c0 2024-04-25T10:02:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.166+0000 7f4721a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c072090 con 0x7f471c0cfec0 2024-04-25T10:02:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.166+0000 7f471aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f471c154ee0 0x7f471c0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37144/0 (socket says 172.21.15.165:37144) 2024-04-25T10:02:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.166+0000 7f471aa00640 1 -- 172.21.15.165:0/1701123063 learned_addr learned my addr 172.21.15.165:0/1701123063 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f471a000640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f471c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1082798767 0 0) 0x7f471c071480 con 0x7f471c154ee0 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4700003520 con 0x7f471c154ee0 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3487773400 0 0) 0x7f4700003520 con 0x7f471c154ee0 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 msgr2=0x7f471c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f471c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 >> v1:172.21.15.172:6789/0 conn(0x7f471c0ce3c0 legacy=0x7f471c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c071a40 con 0x7f471c154ee0 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f470c0019f0 con 0x7f471c154ee0 2024-04-25T10:02:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 >> v1:172.21.15.165:6789/0 conn(0x7f471c154ee0 legacy=0x7f471c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47000039e0 0x7f4700003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f4700004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4700004d40 0x7f47000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470000bb50 con 0x7f47000039e0 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f470000bcd0 con 0x7f4700004d40 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.167+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470000bea0 con 0x7f471c0cfec0 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471aa00640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47000039e0 0x7f4700003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471b400640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4700004d40 0x7f47000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471a000640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f4700004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471aa00640 1 -- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 msgr2=0x7f4700004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471aa00640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f4700004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471aa00640 1 -- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4700004d40 msgr2=0x7f47000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471aa00640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4700004d40 0x7f47000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471aa00640 1 -- 172.21.15.165:0/1701123063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4700003520 con 0x7f47000039e0 2024-04-25T10:02:54.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.168+0000 7f471a000640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0cfec0 0x7f4700004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.169+0000 7f471aa00640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47000039e0 0x7f4700003e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f470c00a890 tx=0x7f4700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=115a449ab533379c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.169+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c00a480 con 0x7f47000039e0 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.169+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c02f070 con 0x7f47000039e0 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.169+0000 7f4719600640 1 -- 172.21.15.165:0/1701123063 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c02bce0 con 0x7f47000039e0 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f4721a00640 1 -- 172.21.15.165:0/1701123063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47000039e0 msgr2=0x7f4700003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f4721a00640 1 --2- 172.21.15.165:0/1701123063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47000039e0 0x7f4700003e00 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f470c00a890 tx=0x7f4700005830 comp rx=0 tx=0).stop 2024-04-25T10:02:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f471b400640 1 -- 172.21.15.165:0/1701123063 reap_dead start 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f4721a00640 1 -- 172.21.15.165:0/1701123063 shutdown_connections 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f4721a00640 1 -- 172.21.15.165:0/1701123063 >> 172.21.15.165:0/1701123063 conn(0x7f471c06abe0 msgr2=0x7f471c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f4721a00640 1 -- 172.21.15.165:0/1701123063 shutdown_connections 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.170+0000 7f4721a00640 1 -- 172.21.15.165:0/1701123063 wait complete. 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.172+0000 7f4721a00640 1 Processor -- start 2024-04-25T10:02:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.172+0000 7f4721a00640 1 -- start start 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f4721a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47000039e0 0x7f471c14c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f4721a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 0x7f471c0a9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f4721a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0a91d0 0x7f471c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f4721a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c0ce800 con 0x7f471c15de10 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f4721a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471c071230 con 0x7f47000039e0 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f4721a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471c1552c0 con 0x7f471c0a91d0 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 0x7f471c0a9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 0x7f471c0a9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56628/0 (socket says 172.21.15.165:56628) 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 -- 172.21.15.165:0/1765831892 learned_addr learned my addr 172.21.15.165:0/1765831892 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 -- 172.21.15.165:0/1765831892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0a91d0 msgr2=0x7f471c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0a91d0 0x7f471c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 -- 172.21.15.165:0/1765831892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47000039e0 msgr2=0x7f471c14c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47000039e0 0x7f471c14c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.173+0000 7f471a000640 1 -- 172.21.15.165:0/1765831892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471c0aa5a0 con 0x7f471c15de10 2024-04-25T10:02:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.174+0000 7f471a000640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 0x7f471c0a9c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f470800e7e0 tx=0x7f470800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.174+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708017070 con 0x7f471c15de10 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.174+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470800c670 con 0x7f471c15de10 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.174+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470800f040 con 0x7f471c15de10 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.174+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471c0a86f0 con 0x7f471c15de10 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.174+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471c0a8cb0 con 0x7f471c15de10 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.175+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471c15a880 con 0x7f471c15de10 2024-04-25T10:02:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.176+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f470801a070 con 0x7f471c15de10 2024-04-25T10:02:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.177+0000 7f4712a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f46fc06b4d0 0x7f46fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.178+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f470808a130 con 0x7f471c15de10 2024-04-25T10:02:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.178+0000 7f471aa00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f46fc06b4d0 0x7f46fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.178+0000 7f471aa00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f46fc06b4d0 0x7f46fc06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f470c033000 tx=0x7f470c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.182+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f471c15a880 con 0x7f471c15de10 2024-04-25T10:02:54.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.378+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f471c0685e0 con 0x7f471c15de10 2024-04-25T10:02:54.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.381+0000 7f4712a00640 1 -- 172.21.15.165:0/1765831892 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f471c0685e0 con 0x7f471c15de10 2024-04-25T10:02:54.386 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:54.387 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:54.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f46fc06b4d0 msgr2=0x7f46fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f46fc06b4d0 0x7f46fc06d990 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f470c033000 tx=0x7f470c007000 comp rx=0 tx=0).stop 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 msgr2=0x7f471c0a9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 0x7f471c0a9c50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f470800e7e0 tx=0x7f470800eca0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 shutdown_connections 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f46fc06b4d0 0x7f46fc06d990 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f471c0a91d0 0x7f471c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f471c15de10 0x7f471c0a9c50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 --2- 172.21.15.165:0/1765831892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47000039e0 0x7f471c14c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.384+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 >> 172.21.15.165:0/1765831892 conn(0x7f471c06abe0 msgr2=0x7f471c0b4160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.385+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 shutdown_connections 2024-04-25T10:02:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.385+0000 7f4721a00640 1 -- 172.21.15.165:0/1765831892 wait complete. 2024-04-25T10:02:54.408 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:54.409 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.613+0000 7fc193600640 1 Processor -- start 2024-04-25T10:02:54.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.614+0000 7fc193600640 1 -- start start 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.615+0000 7fc193600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c15dc20 0x7fc18c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.615+0000 7fc193600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc18c0d32f0 0x7fc18c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.615+0000 7fc193600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc18c15e000 con 0x7fc18c161710 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.615+0000 7fc193600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc18c0d1de0 con 0x7fc18c0d32f0 2024-04-25T10:02:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.615+0000 7fc193600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c161c60 con 0x7fc18c15dc20 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc18be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc18c161710 0x7fc18c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37148/0 (socket says 172.21.15.165:37148) 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc18be00640 1 -- 172.21.15.165:0/4112285681 learned_addr learned my addr 172.21.15.165:0/4112285681 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc191600640 1 --2- 172.21.15.165:0/4112285681 >> v2:172.21.15.165:3300/0 conn(0x7fc18c0d32f0 0x7fc18c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc18b400640 1 -- 172.21.15.165:0/4112285681 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 448951321 0 0) 0x7fc18c15e000 con 0x7fc18c161710 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc190c00640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c15dc20 0x7fc18c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc18b400640 1 -- 172.21.15.165:0/4112285681 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc174003520 con 0x7fc18c161710 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc191600640 1 -- 172.21.15.165:0/4112285681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c15dc20 msgr2=0x7fc18c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc191600640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c15dc20 0x7fc18c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc191600640 1 -- 172.21.15.165:0/4112285681 >> v1:172.21.15.165:6789/0 conn(0x7fc18c161710 legacy=0x7fc18c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc191600640 1 -- 172.21.15.165:0/4112285681 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c161f40 con 0x7fc18c0d32f0 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.616+0000 7fc190c00640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c15dc20 0x7fc18c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.617+0000 7fc191600640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c0d32f0 0x7fc18c0cebb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc184009d90 tx=0x7fc18402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790b814ee080923e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.618+0000 7fc18b400640 1 -- 172.21.15.165:0/4112285681 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184007740 con 0x7fc18c0d32f0 2024-04-25T10:02:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.618+0000 7fc18b400640 1 -- 172.21.15.165:0/4112285681 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1840078e0 con 0x7fc18c0d32f0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.618+0000 7fc18b400640 1 -- 172.21.15.165:0/4112285681 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc184007cb0 con 0x7fc18c0d32f0 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.618+0000 7fc193600640 1 -- 172.21.15.165:0/4112285681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c0d32f0 msgr2=0x7fc18c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.618+0000 7fc193600640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c0d32f0 0x7fc18c0cebb0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fc184009d90 tx=0x7fc18402a660 comp rx=0 tx=0).stop 2024-04-25T10:02:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.619+0000 7fc193600640 1 -- 172.21.15.165:0/4112285681 shutdown_connections 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.619+0000 7fc193600640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c0d32f0 0x7fc18c0cebb0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.619+0000 7fc193600640 1 --2- 172.21.15.165:0/4112285681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c15dc20 0x7fc18c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.619+0000 7fc193600640 1 -- 172.21.15.165:0/4112285681 >> 172.21.15.165:0/4112285681 conn(0x7fc18c0d0420 msgr2=0x7fc18c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.619+0000 7fc193600640 1 -- 172.21.15.165:0/4112285681 shutdown_connections 2024-04-25T10:02:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.619+0000 7fc193600640 1 -- 172.21.15.165:0/4112285681 wait complete. 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.620+0000 7fc193600640 1 Processor -- start 2024-04-25T10:02:54.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.621+0000 7fc193600640 1 -- start start 2024-04-25T10:02:54.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.621+0000 7fc193600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc18c0d32f0 0x7fc18c1f3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc193600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 0x7fc18c0aac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc193600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c161710 0x7fc18c1f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc193600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0b5e50 con 0x7fc18c15dc20 2024-04-25T10:02:54.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc193600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c162500 con 0x7fc18c0d32f0 2024-04-25T10:02:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc193600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc18c0b5fd0 con 0x7fc18c161710 2024-04-25T10:02:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 0x7fc18c0aac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 0x7fc18c0aac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56636/0 (socket says 172.21.15.165:56636) 2024-04-25T10:02:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 -- 172.21.15.165:0/2945385748 learned_addr learned my addr 172.21.15.165:0/2945385748 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc191600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c161710 0x7fc18c1f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc190c00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc18c0d32f0 0x7fc18c1f3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 -- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c161710 msgr2=0x7fc18c1f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c161710 0x7fc18c1f0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 -- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc18c0d32f0 msgr2=0x7fc18c1f3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc18c0d32f0 0x7fc18c1f3850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc18be00640 1 -- 172.21.15.165:0/2945385748 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc18c1f3e30 con 0x7fc18c15dc20 2024-04-25T10:02:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.622+0000 7fc190c00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc18c0d32f0 0x7fc18c1f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.623+0000 7fc18be00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 0x7fc18c0aac00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c390 tx=0x7fc17800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.623+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc178017070 con 0x7fc18c15dc20 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.623+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc178005ce0 con 0x7fc18c15dc20 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.623+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc18c1f4110 con 0x7fc18c15dc20 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.623+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1780072f0 con 0x7fc18c15dc20 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.623+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc18c1f46d0 con 0x7fc18c15dc20 2024-04-25T10:02:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.625+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc18c0ca730 con 0x7fc18c15dc20 2024-04-25T10:02:54.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.627+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc17801a070 con 0x7fc18c15dc20 2024-04-25T10:02:54.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.627+0000 7fc188c00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc16c06b4d0 0x7fc16c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:54.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.628+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc178089a10 con 0x7fc18c15dc20 2024-04-25T10:02:54.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.628+0000 7fc190c00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc16c06b4d0 0x7fc16c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.629+0000 7fc190c00640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc16c06b4d0 0x7fc16c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc180006fd0 tx=0x7fc180008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.632+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc18c0ca730 con 0x7fc18c15dc20 2024-04-25T10:02:54.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.833+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc18c073a00 con 0x7fc18c15dc20 2024-04-25T10:02:54.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.834+0000 7fc188c00640 1 -- 172.21.15.165:0/2945385748 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc178058490 con 0x7fc18c15dc20 2024-04-25T10:02:54.838 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:54.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.836+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc16c06b4d0 msgr2=0x7fc16c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.836+0000 7fc193600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc16c06b4d0 0x7fc16c06d990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc180006fd0 tx=0x7fc180008040 comp rx=0 tx=0).stop 2024-04-25T10:02:54.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.836+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 msgr2=0x7fc18c0aac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.836+0000 7fc193600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 0x7fc18c0aac00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fc17800c390 tx=0x7fc17800c850 comp rx=0 tx=0).stop 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 shutdown_connections 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc16c06b4d0 0x7fc16c06d990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc18c161710 0x7fc18c1f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc18c15dc20 0x7fc18c0aac00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 --2- 172.21.15.165:0/2945385748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc18c0d32f0 0x7fc18c1f3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 >> 172.21.15.165:0/2945385748 conn(0x7fc18c0d0420 msgr2=0x7fc18c0b4440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 shutdown_connections 2024-04-25T10:02:54.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:54.837+0000 7fc193600640 1 -- 172.21.15.165:0/2945385748 wait complete. 2024-04-25T10:02:54.860 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.061+0000 7fcc2f000640 1 Processor -- start 2024-04-25T10:02:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.062+0000 7fcc2f000640 1 -- start start 2024-04-25T10:02:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.063+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 0x7fcc28161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.063+0000 7fcc2f000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc28162040 con 0x7fcc280b9750 2024-04-25T10:02:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.063+0000 7fcc2f000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc28162320 con 0x7fcc28161ba0 2024-04-25T10:02:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.063+0000 7fcc2f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28162600 con 0x7fcc2815da90 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc27400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcc280b9750 0x7fcc280ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37160/0 (socket says 172.21.15.165:37160) 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc27400640 1 -- 172.21.15.165:0/653004258 learned_addr learned my addr 172.21.15.165:0/653004258 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc27e00640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 0x7fcc28161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912724583 0 0) 0x7fcc28162040 con 0x7fcc280b9750 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc08003520 con 0x7fcc280b9750 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 974774875 0 0) 0x7fcc08003520 con 0x7fcc280b9750 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 msgr2=0x7fcc28161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 0x7fcc28161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 >> v1:172.21.15.172:6789/0 conn(0x7fcc28161ba0 legacy=0x7fcc280d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc280d1b10 con 0x7fcc280b9750 2024-04-25T10:02:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.064+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fcc100019f0 con 0x7fcc280b9750 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 >> v1:172.21.15.165:6789/0 conn(0x7fcc280b9750 legacy=0x7fcc280ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080039e0 0x7fcc08003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc080043b0 0x7fcc08004880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 0x7fcc08005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc28162600 con 0x7fcc080043b0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0800ba70 con 0x7fcc080039e0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0800bc70 con 0x7fcc2815da90 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27400640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc080043b0 0x7fcc08004880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27e00640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080039e0 0x7fcc08003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27400640 1 -- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 msgr2=0x7fcc08005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27400640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815da90 0x7fcc08005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27400640 1 -- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080039e0 msgr2=0x7fcc08003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27400640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080039e0 0x7fcc08003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.065+0000 7fcc27400640 1 -- 172.21.15.165:0/653004258 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc08003520 con 0x7fcc080043b0 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.066+0000 7fcc27400640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc080043b0 0x7fcc08004880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000a880 tx=0x7fcc08005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee6318fd9e9c438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000a4a0 con 0x7fcc080043b0 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1002f070 con 0x7fcc080043b0 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc26a00640 1 -- 172.21.15.165:0/653004258 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1002bce0 con 0x7fcc080043b0 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2f000640 1 -- 172.21.15.165:0/653004258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc080043b0 msgr2=0x7fcc08004880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2f000640 1 --2- 172.21.15.165:0/653004258 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc080043b0 0x7fcc08004880 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fcc1000a880 tx=0x7fcc08005830 comp rx=0 tx=0).stop 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2d000640 1 -- 172.21.15.165:0/653004258 reap_dead start 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2f000640 1 -- 172.21.15.165:0/653004258 shutdown_connections 2024-04-25T10:02:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2f000640 1 -- 172.21.15.165:0/653004258 >> 172.21.15.165:0/653004258 conn(0x7fcc28061a20 msgr2=0x7fcc28061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2f000640 1 -- 172.21.15.165:0/653004258 shutdown_connections 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.067+0000 7fcc2f000640 1 -- 172.21.15.165:0/653004258 wait complete. 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.069+0000 7fcc2f000640 1 Processor -- start 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.069+0000 7fcc2f000640 1 -- start start 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080043b0 0x7fcc280cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 0x7fcc280cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc2f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815f020 0x7fcc2815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc2f000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280ced80 con 0x7fcc280caa70 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc2f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc2815de70 con 0x7fcc080043b0 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc2f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc280cef00 con 0x7fcc2815f020 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc27400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 0x7fcc280cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc27400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 0x7fcc280cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56656/0 (socket says 172.21.15.165:56656) 2024-04-25T10:02:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.070+0000 7fcc27400640 1 -- 172.21.15.165:0/3160196896 learned_addr learned my addr 172.21.15.165:0/3160196896 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27e00640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080043b0 0x7fcc280cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc2d000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815f020 0x7fcc2815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27400640 1 -- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815f020 msgr2=0x7fcc2815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27400640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815f020 0x7fcc2815f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27400640 1 -- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080043b0 msgr2=0x7fcc280cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27400640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080043b0 0x7fcc280cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27400640 1 -- 172.21.15.165:0/3160196896 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc2815b110 con 0x7fcc280caa70 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc27400640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 0x7fcc280cd2e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc10001cf0 tx=0x7fcc281624f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc10044070 con 0x7fcc280caa70 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10034670 con 0x7fcc280caa70 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc280cb910 con 0x7fcc280caa70 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.071+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1000a4a0 con 0x7fcc280caa70 2024-04-25T10:02:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.072+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc280c82f0 con 0x7fcc280caa70 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.073+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc28073a00 con 0x7fcc280caa70 2024-04-25T10:02:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.074+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc10047070 con 0x7fcc280caa70 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.075+0000 7fcc1fe00640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc0406b500 0x7fcc0406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.075+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fcc100aaaa0 con 0x7fcc280caa70 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.076+0000 7fcc27e00640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc0406b500 0x7fcc0406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.076+0000 7fcc27e00640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc0406b500 0x7fcc0406d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009a30 tx=0x7fcc18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.080+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc100793f0 con 0x7fcc280caa70 2024-04-25T10:02:55.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.274+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc280d1590 con 0x7fcc280caa70 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.275+0000 7fcc1fe00640 1 -- 172.21.15.165:0/3160196896 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc10078cf0 con 0x7fcc280caa70 2024-04-25T10:02:55.280 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc0406b500 msgr2=0x7fcc0406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc0406b500 0x7fcc0406d9c0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcc18009a30 tx=0x7fcc18008040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 msgr2=0x7fcc280cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 0x7fcc280cd2e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fcc10001cf0 tx=0x7fcc281624f0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 shutdown_connections 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc0406b500 0x7fcc0406d9c0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc2815f020 0x7fcc2815f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc280caa70 0x7fcc280cd2e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 --2- 172.21.15.165:0/3160196896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc080043b0 0x7fcc280cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 >> 172.21.15.165:0/3160196896 conn(0x7fcc28061a20 msgr2=0x7fcc280ce070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 shutdown_connections 2024-04-25T10:02:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.278+0000 7fcc2f000640 1 -- 172.21.15.165:0/3160196896 wait complete. 2024-04-25T10:02:55.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.502+0000 7ff6d2e00640 1 Processor -- start 2024-04-25T10:02:55.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.503+0000 7ff6d2e00640 1 -- start start 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc15da90 0x7ff6cc161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6d2e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff6cc161ba0 0x7ff6cc0d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6d2e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6cc162040 con 0x7ff6cc0b9750 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6d2e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6cc162600 con 0x7ff6cc161ba0 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc162320 con 0x7ff6cc15da90 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6d0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff6cc161ba0 0x7ff6cc0d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6cb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff6cc0b9750 0x7ff6cc0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37162/0 (socket says 172.21.15.165:37162) 2024-04-25T10:02:55.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.504+0000 7ff6cb400640 1 -- 172.21.15.165:0/2241781281 learned_addr learned my addr 172.21.15.165:0/2241781281 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6cbe00640 1 --2- 172.21.15.165:0/2241781281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc15da90 0x7ff6cc161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6caa00640 1 -- 172.21.15.165:0/2241781281 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359821296 0 0) 0x7ff6cc162040 con 0x7ff6cc0b9750 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6caa00640 1 -- 172.21.15.165:0/2241781281 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6ac003520 con 0x7ff6cc0b9750 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6d0e00640 1 -- 172.21.15.165:0/2241781281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc15da90 msgr2=0x7ff6cc161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6d0e00640 1 --2- 172.21.15.165:0/2241781281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc15da90 0x7ff6cc161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6d0e00640 1 -- 172.21.15.165:0/2241781281 >> v1:172.21.15.165:6789/0 conn(0x7ff6cc0b9750 legacy=0x7ff6cc0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.505+0000 7ff6d0e00640 1 -- 172.21.15.165:0/2241781281 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc0d1b10 con 0x7ff6cc161ba0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.506+0000 7ff6d0e00640 1 --2- 172.21.15.165:0/2241781281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc161ba0 0x7ff6cc0d2310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40097a0 tx=0x7ff6c402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da1d69299954545d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6caa00640 1 -- 172.21.15.165:0/2241781281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4037070 con 0x7ff6cc161ba0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6caa00640 1 -- 172.21.15.165:0/2241781281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c402ae50 con 0x7ff6cc161ba0 2024-04-25T10:02:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6caa00640 1 -- 172.21.15.165:0/2241781281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c4032660 con 0x7ff6cc161ba0 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6d2e00640 1 -- 172.21.15.165:0/2241781281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc161ba0 msgr2=0x7ff6cc0d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/2241781281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc161ba0 0x7ff6cc0d2310 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7ff6c40097a0 tx=0x7ff6c402a100 comp rx=0 tx=0).stop 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6d2e00640 1 -- 172.21.15.165:0/2241781281 shutdown_connections 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/2241781281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc161ba0 0x7ff6cc0d2310 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/2241781281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc15da90 0x7ff6cc161660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.507+0000 7ff6d2e00640 1 -- 172.21.15.165:0/2241781281 >> 172.21.15.165:0/2241781281 conn(0x7ff6cc061a20 msgr2=0x7ff6cc061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.508+0000 7ff6d2e00640 1 -- 172.21.15.165:0/2241781281 shutdown_connections 2024-04-25T10:02:55.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.508+0000 7ff6d2e00640 1 -- 172.21.15.165:0/2241781281 wait complete. 2024-04-25T10:02:55.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.509+0000 7ff6d2e00640 1 Processor -- start 2024-04-25T10:02:55.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.509+0000 7ff6d2e00640 1 -- start start 2024-04-25T10:02:55.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.510+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 0x7ff6cc14f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.510+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc161ba0 0x7ff6cc1e8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.510+0000 7ff6d2e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc151fd0 0x7ff6cc15ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.510+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0ced80 con 0x7ff6cc15da90 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.510+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc15de70 con 0x7ff6cc151fd0 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.510+0000 7ff6d2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6cc0cef00 con 0x7ff6cc161ba0 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 0x7ff6cc14f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 0x7ff6cc14f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56680/0 (socket says 172.21.15.165:56680) 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 -- 172.21.15.165:0/3360266661 learned_addr learned my addr 172.21.15.165:0/3360266661 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cb400640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc161ba0 0x7ff6cc1e8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 -- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc161ba0 msgr2=0x7ff6cc1e8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc161ba0 0x7ff6cc1e8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 -- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc151fd0 msgr2=0x7ff6cc15ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6d0e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc151fd0 0x7ff6cc15ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc151fd0 0x7ff6cc15ba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.511+0000 7ff6cbe00640 1 -- 172.21.15.165:0/3360266661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6cc1e8a50 con 0x7ff6cc15da90 2024-04-25T10:02:55.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.512+0000 7ff6cbe00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 0x7ff6cc14f340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d800 tx=0x7ff6bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.512+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00b660 con 0x7ff6cc15da90 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.512+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6bc019070 con 0x7ff6cc15da90 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.512+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6bc00ba20 con 0x7ff6cc15da90 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.512+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6cc0c8a00 con 0x7ff6cc15da90 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.512+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6cc0c8fc0 con 0x7ff6cc15da90 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.513+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6cc073a00 con 0x7ff6cc15da90 2024-04-25T10:02:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.514+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff6bc00bbc0 con 0x7ff6cc15da90 2024-04-25T10:02:55.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.515+0000 7ff6c3e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff6a806b4d0 0x7ff6a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.515+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff6bc08aa90 con 0x7ff6cc15da90 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.515+0000 7ff6cb400640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff6a806b4d0 0x7ff6a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.516+0000 7ff6cb400640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff6a806b4d0 0x7ff6a806d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc152440 tx=0x7ff6b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.519+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6bc01c360 con 0x7ff6cc15da90 2024-04-25T10:02:55.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.717+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6cc068080 con 0x7ff6cc15da90 2024-04-25T10:02:55.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.718+0000 7ff6c3e00640 1 -- 172.21.15.165:0/3360266661 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6bc059510 con 0x7ff6cc15da90 2024-04-25T10:02:55.723 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:55.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.720+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff6a806b4d0 msgr2=0x7ff6a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.720+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff6a806b4d0 0x7ff6a806d990 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff6cc152440 tx=0x7ff6b4009040 comp rx=0 tx=0).stop 2024-04-25T10:02:55.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.720+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 msgr2=0x7ff6cc14f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.720+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 0x7ff6cc14f340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ff6bc00d800 tx=0x7ff6bc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 shutdown_connections 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff6a806b4d0 0x7ff6a806d990 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6cc151fd0 0x7ff6cc15ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff6cc161ba0 0x7ff6cc1e8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 --2- 172.21.15.165:0/3360266661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6cc15da90 0x7ff6cc14f340 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 >> 172.21.15.165:0/3360266661 conn(0x7ff6cc061a20 msgr2=0x7ff6cc15d080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 shutdown_connections 2024-04-25T10:02:55.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.721+0000 7ff6d2e00640 1 -- 172.21.15.165:0/3360266661 wait complete. 2024-04-25T10:02:55.745 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:55.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.945+0000 7f7b5d400640 1 Processor -- start 2024-04-25T10:02:55.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.945+0000 7f7b5d400640 1 -- start start 2024-04-25T10:02:55.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.946+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 0x7f7b5806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.946+0000 7f7b5d400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580cccc0 con 0x7f7b58144d50 2024-04-25T10:02:55.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.946+0000 7f7b5d400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b580cc140 con 0x7f7b581446d0 2024-04-25T10:02:55.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.946+0000 7f7b5d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580b2b20 con 0x7f7b581452a0 2024-04-25T10:02:55.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.947+0000 7f7b4fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7b58144d50 0x7f7b5806d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37172/0 (socket says 172.21.15.165:37172) 2024-04-25T10:02:55.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.947+0000 7f7b4fe00640 1 -- 172.21.15.165:0/3301423260 learned_addr learned my addr 172.21.15.165:0/3301423260 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:55.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.947+0000 7f7b57200640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 0x7f7b5806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.947+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983785405 0 0) 0x7f7b580cccc0 con 0x7f7b58144d50 2024-04-25T10:02:55.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.947+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b3c003520 con 0x7f7b58144d50 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1391730259 0 0) 0x7f7b3c003520 con 0x7f7b58144d50 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 msgr2=0x7f7b5806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 0x7f7b5806dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 >> v1:172.21.15.172:6789/0 conn(0x7f7b581446d0 legacy=0x7f7b580b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580cc420 con 0x7f7b58144d50 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7b40001670 con 0x7f7b58144d50 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 >> v1:172.21.15.165:6789/0 conn(0x7f7b58144d50 legacy=0x7f7b5806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b3c0039e0 0x7f7b3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 0x7f7b3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b3c004d40 0x7f7b3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580b2b20 con 0x7f7b3c0039e0 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c00baa0 con 0x7f7b3c004d40 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c00bca0 con 0x7f7b581452a0 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.948+0000 7f7b56800640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b3c0039e0 0x7f7b3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b57200640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b3c004d40 0x7f7b3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b4fe00640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 0x7f7b3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b56800640 1 -- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 msgr2=0x7f7b3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b56800640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b581452a0 0x7f7b3c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b56800640 1 -- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b3c004d40 msgr2=0x7f7b3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b56800640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b3c004d40 0x7f7b3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b56800640 1 -- 172.21.15.165:0/3301423260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b3c003520 con 0x7f7b3c0039e0 2024-04-25T10:02:55.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.949+0000 7f7b57200640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b3c004d40 0x7f7b3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.950+0000 7f7b56800640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b3c0039e0 0x7f7b3c003e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b48001720 tx=0x7f7b3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9283f721619df604 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.950+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48001370 con 0x7f7b3c0039e0 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.950+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b480052d0 con 0x7f7b3c0039e0 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.950+0000 7f7b55e00640 1 -- 172.21.15.165:0/3301423260 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4802abc0 con 0x7f7b3c0039e0 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b5d400640 1 -- 172.21.15.165:0/3301423260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b3c0039e0 msgr2=0x7f7b3c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b5d400640 1 --2- 172.21.15.165:0/3301423260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b3c0039e0 0x7f7b3c003e00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7b48001720 tx=0x7f7b3c005830 comp rx=0 tx=0).stop 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b57200640 1 -- 172.21.15.165:0/3301423260 reap_dead start 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b5d400640 1 -- 172.21.15.165:0/3301423260 shutdown_connections 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b5d400640 1 -- 172.21.15.165:0/3301423260 >> 172.21.15.165:0/3301423260 conn(0x7f7b5806abe0 msgr2=0x7f7b5806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b5d400640 1 -- 172.21.15.165:0/3301423260 shutdown_connections 2024-04-25T10:02:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.951+0000 7f7b5d400640 1 -- 172.21.15.165:0/3301423260 wait complete. 2024-04-25T10:02:55.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.953+0000 7f7b5d400640 1 Processor -- start 2024-04-25T10:02:55.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.953+0000 7f7b5d400640 1 -- start start 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b3c0039e0 0x7f7b580c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 0x7f7b580b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b5d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b580b10d0 0x7f7b580ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b5d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b580cc9e0 con 0x7f7b5813e120 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b5d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5806d400 con 0x7f7b580b10d0 2024-04-25T10:02:55.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b5d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b58144ab0 con 0x7f7b3c0039e0 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 0x7f7b580b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 0x7f7b580b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56696/0 (socket says 172.21.15.165:56696) 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.954+0000 7f7b4fe00640 1 -- 172.21.15.165:0/2162973635 learned_addr learned my addr 172.21.15.165:0/2162973635 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b56800640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b3c0039e0 0x7f7b580c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b57200640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b580b10d0 0x7f7b580ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4fe00640 1 -- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b3c0039e0 msgr2=0x7f7b580c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4fe00640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b3c0039e0 0x7f7b580c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4fe00640 1 -- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b580b10d0 msgr2=0x7f7b580ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4fe00640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b580b10d0 0x7f7b580ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4fe00640 1 -- 172.21.15.165:0/2162973635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580cd6a0 con 0x7f7b5813e120 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b56800640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b3c0039e0 0x7f7b580c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:55.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4fe00640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 0x7f7b580b0b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000e730 tx=0x7f7b4000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4000c6c0 con 0x7f7b5813e120 2024-04-25T10:02:55.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b40018070 con 0x7f7b5813e120 2024-04-25T10:02:55.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b580cd980 con 0x7f7b5813e120 2024-04-25T10:02:55.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b4000caa0 con 0x7f7b5813e120 2024-04-25T10:02:55.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.955+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b580cdf40 con 0x7f7b5813e120 2024-04-25T10:02:55.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.957+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b58073a00 con 0x7f7b5813e120 2024-04-25T10:02:55.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.958+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b4001d460 con 0x7f7b5813e120 2024-04-25T10:02:55.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.959+0000 7f7b4ea00640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b3406b4d0 0x7f7b3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:55.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.959+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7b4008a5e0 con 0x7f7b5813e120 2024-04-25T10:02:55.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.959+0000 7f7b56800640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b3406b4d0 0x7f7b3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:55.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.960+0000 7f7b56800640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b3406b4d0 0x7f7b3406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b48032000 tx=0x7f7b4802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:55.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:55.963+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b40059050 con 0x7f7b5813e120 2024-04-25T10:02:56.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.158+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7b580b14b0 con 0x7f7b5813e120 2024-04-25T10:02:56.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.159+0000 7f7b4ea00640 1 -- 172.21.15.165:0/2162973635 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7b40058950 con 0x7f7b5813e120 2024-04-25T10:02:56.164 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b3406b4d0 msgr2=0x7f7b3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b3406b4d0 0x7f7b3406d990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f7b48032000 tx=0x7f7b4802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 msgr2=0x7f7b580b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 0x7f7b580b0b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7b4000e730 tx=0x7f7b4000ebf0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 shutdown_connections 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b3406b4d0 0x7f7b3406d990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b580b10d0 0x7f7b580ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b5813e120 0x7f7b580b0b90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 --2- 172.21.15.165:0/2162973635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b3c0039e0 0x7f7b580c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 >> 172.21.15.165:0/2162973635 conn(0x7f7b5806abe0 msgr2=0x7f7b58071ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 shutdown_connections 2024-04-25T10:02:56.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.162+0000 7f7b5d400640 1 -- 172.21.15.165:0/2162973635 wait complete. 2024-04-25T10:02:56.185 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.375+0000 7f769c400640 1 Processor -- start 2024-04-25T10:02:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.375+0000 7f769c400640 1 -- start start 2024-04-25T10:02:56.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769c400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7694144750 0x7f769406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769c400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f76940cf030 0x7f769406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769c400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7694144b30 con 0x7f769406eb10 2024-04-25T10:02:56.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769c400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940712d0 con 0x7f76940cf030 2024-04-25T10:02:56.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769c400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940715b0 con 0x7f7694144750 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f76940cf030 0x7f769406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f76940cf030 0x7f769406d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56702/0 (socket says 172.21.15.165:56702) 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 -- 172.21.15.165:0/3380724726 learned_addr learned my addr 172.21.15.165:0/3380724726 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 -- 172.21.15.165:0/3380724726 >> v2:172.21.15.172:3300/0 conn(0x7f7694144750 msgr2=0x7f769406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 --2- 172.21.15.165:0/3380724726 >> v2:172.21.15.172:3300/0 conn(0x7f7694144750 0x7f769406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 -- 172.21.15.165:0/3380724726 >> v1:172.21.15.172:6789/0 conn(0x7f769406eb10 legacy=0x7f76940ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.377+0000 7f769a400640 1 -- 172.21.15.165:0/3380724726 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7694071890 con 0x7f76940cf030 2024-04-25T10:02:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.378+0000 7f769a400640 1 --2- 172.21.15.165:0/3380724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76940cf030 0x7f769406d290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f769406f5d0 tx=0x7f768c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34bd7cd9b5aa5e6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.379+0000 7f7693e00640 1 -- 172.21.15.165:0/3380724726 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c036070 con 0x7f76940cf030 2024-04-25T10:02:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.379+0000 7f7693e00640 1 -- 172.21.15.165:0/3380724726 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c031600 con 0x7f76940cf030 2024-04-25T10:02:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.379+0000 7f7693e00640 1 -- 172.21.15.165:0/3380724726 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c0319d0 con 0x7f76940cf030 2024-04-25T10:02:56.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.379+0000 7f769c400640 1 -- 172.21.15.165:0/3380724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76940cf030 msgr2=0x7f769406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.379+0000 7f769c400640 1 --2- 172.21.15.165:0/3380724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76940cf030 0x7f769406d290 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f769406f5d0 tx=0x7f768c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.380+0000 7f769c400640 1 -- 172.21.15.165:0/3380724726 shutdown_connections 2024-04-25T10:02:56.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.380+0000 7f769c400640 1 --2- 172.21.15.165:0/3380724726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76940cf030 0x7f769406d290 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.380+0000 7f769c400640 1 --2- 172.21.15.165:0/3380724726 >> v2:172.21.15.172:3300/0 conn(0x7f7694144750 0x7f769406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.380+0000 7f769c400640 1 -- 172.21.15.165:0/3380724726 >> 172.21.15.165:0/3380724726 conn(0x7f769406abe0 msgr2=0x7f769406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.380+0000 7f769c400640 1 -- 172.21.15.165:0/3380724726 shutdown_connections 2024-04-25T10:02:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.380+0000 7f769c400640 1 -- 172.21.15.165:0/3380724726 wait complete. 2024-04-25T10:02:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.381+0000 7f769c400640 1 Processor -- start 2024-04-25T10:02:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.382+0000 7f769c400640 1 -- start start 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.382+0000 7f769c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769406eb10 0x7f76940b2750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76940cf030 0x7f769413dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769c400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 0x7f7694141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769c400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76941455d0 con 0x7f7694144750 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7694145450 con 0x7f76940cf030 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769415c000 con 0x7f769406eb10 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 0x7f7694141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 0x7f7694141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56704/0 (socket says 172.21.15.165:56704) 2024-04-25T10:02:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 -- 172.21.15.165:0/258470962 learned_addr learned my addr 172.21.15.165:0/258470962 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f7699a00640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769406eb10 0x7f76940b2750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 -- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769406eb10 msgr2=0x7f76940b2750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769406eb10 0x7f76940b2750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 -- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76940cf030 msgr2=0x7f769413dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76940cf030 0x7f769413dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.383+0000 7f769a400640 1 -- 172.21.15.165:0/258470962 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76940ad2c0 con 0x7f7694144750 2024-04-25T10:02:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.384+0000 7f769a400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 0x7f7694141d40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f768c034040 tx=0x7f768c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.384+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c041020 con 0x7f7694144750 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.384+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c007780 con 0x7f7694144750 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.384+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c036040 con 0x7f7694144750 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.384+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76940ad5a0 con 0x7f7694144750 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.384+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76940b41e0 con 0x7f7694144750 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.385+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7694073990 con 0x7f7694144750 2024-04-25T10:02:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.386+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f768c044020 con 0x7f7694144750 2024-04-25T10:02:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.387+0000 7f7691600640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f767406b4d0 0x7f767406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.387+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f768c0ab340 con 0x7f7694144750 2024-04-25T10:02:56.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.387+0000 7f7699a00640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f767406b4d0 0x7f767406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.388+0000 7f7699a00640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f767406b4d0 0x7f767406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7694140820 tx=0x7f7688008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.392+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f768c04b030 con 0x7f7694144750 2024-04-25T10:02:56.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:56.508 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:02:56.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.590+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76940b4700 con 0x7f7694144750 2024-04-25T10:02:56.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.592+0000 7f7691600640 1 -- 172.21.15.165:0/258470962 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f768c079f10 con 0x7f7694144750 2024-04-25T10:02:56.597 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:56.597 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:56.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.594+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f767406b4d0 msgr2=0x7f767406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.594+0000 7f769c400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f767406b4d0 0x7f767406d990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f7694140820 tx=0x7f7688008040 comp rx=0 tx=0).stop 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 msgr2=0x7f7694141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 0x7f7694141d40 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f768c034040 tx=0x7f768c004a40 comp rx=0 tx=0).stop 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 shutdown_connections 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f767406b4d0 0x7f767406d990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7694144750 0x7f7694141d40 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76940cf030 0x7f769413dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 --2- 172.21.15.165:0/258470962 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f769406eb10 0x7f76940b2750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 >> 172.21.15.165:0/258470962 conn(0x7f769406abe0 msgr2=0x7f7694143970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 shutdown_connections 2024-04-25T10:02:56.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.595+0000 7f769c400640 1 -- 172.21.15.165:0/258470962 wait complete. 2024-04-25T10:02:56.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:56.619 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:56.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.860+0000 7f689d600640 1 Processor -- start 2024-04-25T10:02:56.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.861+0000 7f689d600640 1 -- start start 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.862+0000 7f689d600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f689815cfc0 0x7f68980c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.862+0000 7f689d600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6898145520 con 0x7f6898145140 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.862+0000 7f689d600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6898071270 con 0x7f68980ceb20 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.862+0000 7f689d600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6898071550 con 0x7f689815cfc0 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f6897e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6898145140 0x7f68980ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48938/0 (socket says 172.21.15.165:48938) 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f6897e00640 1 -- 172.21.15.165:0/4243805186 learned_addr learned my addr 172.21.15.165:0/4243805186 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:56.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f689cc00640 1 --2- 172.21.15.165:0/4243805186 >> v2:172.21.15.165:3300/0 conn(0x7f689815cfc0 0x7f68980c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f6897400640 1 -- 172.21.15.165:0/4243805186 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823568125 0 0) 0x7f6898145520 con 0x7f6898145140 2024-04-25T10:02:56.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f6897400640 1 -- 172.21.15.165:0/4243805186 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f687c003520 con 0x7f6898145140 2024-04-25T10:02:56.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f689cc00640 1 -- 172.21.15.165:0/4243805186 >> v1:172.21.15.172:6789/0 conn(0x7f68980ceb20 legacy=0x7f68980c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:56.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f689cc00640 1 -- 172.21.15.165:0/4243805186 >> v1:172.21.15.165:6789/0 conn(0x7f6898145140 legacy=0x7f68980ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.863+0000 7f689cc00640 1 -- 172.21.15.165:0/4243805186 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898071830 con 0x7f689815cfc0 2024-04-25T10:02:56.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.864+0000 7f689cc00640 1 --2- 172.21.15.165:0/4243805186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f689815cfc0 0x7f68980c5200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6890009d90 tx=0x7f689002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d871d03b4e781235 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f6897400640 1 -- 172.21.15.165:0/4243805186 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890031600 con 0x7f689815cfc0 2024-04-25T10:02:56.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f6897400640 1 -- 172.21.15.165:0/4243805186 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68900317a0 con 0x7f689815cfc0 2024-04-25T10:02:56.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f6897400640 1 -- 172.21.15.165:0/4243805186 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6890031b70 con 0x7f689815cfc0 2024-04-25T10:02:56.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 -- 172.21.15.165:0/4243805186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f689815cfc0 msgr2=0x7f68980c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 --2- 172.21.15.165:0/4243805186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f689815cfc0 0x7f68980c5200 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6890009d90 tx=0x7f689002a610 comp rx=0 tx=0).stop 2024-04-25T10:02:56.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 -- 172.21.15.165:0/4243805186 shutdown_connections 2024-04-25T10:02:56.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 --2- 172.21.15.165:0/4243805186 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f689815cfc0 0x7f68980c5200 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 -- 172.21.15.165:0/4243805186 >> 172.21.15.165:0/4243805186 conn(0x7f6898061cc0 msgr2=0x7f689806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:56.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 -- 172.21.15.165:0/4243805186 shutdown_connections 2024-04-25T10:02:56.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.865+0000 7f689d600640 1 -- 172.21.15.165:0/4243805186 wait complete. 2024-04-25T10:02:56.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.867+0000 7f689d600640 1 Processor -- start 2024-04-25T10:02:56.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.867+0000 7f689d600640 1 -- start start 2024-04-25T10:02:56.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f689d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68980ceb20 0x7f689813c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f689d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 0x7f6898146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f689d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689815cfc0 0x7f6898146bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f689d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68981665b0 con 0x7f6898145140 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f689d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6898166430 con 0x7f689815cfc0 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f689d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6898166730 con 0x7f68980ceb20 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.868+0000 7f688f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 0x7f6898146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 0x7f6898146170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39158/0 (socket says 172.21.15.165:39158) 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 -- 172.21.15.165:0/3460123704 learned_addr learned my addr 172.21.15.165:0/3460123704 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f6897e00640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68980ceb20 0x7f689813c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f689cc00640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689815cfc0 0x7f6898146bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 -- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68980ceb20 msgr2=0x7f689813c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68980ceb20 0x7f689813c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 -- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689815cfc0 msgr2=0x7f6898146bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689815cfc0 0x7f6898146bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:56.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 -- 172.21.15.165:0/3460123704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68981421f0 con 0x7f6898145140 2024-04-25T10:02:56.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.869+0000 7f688f400640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 0x7f6898146170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f688000c430 tx=0x7f688000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.870+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011790 con 0x7f6898145140 2024-04-25T10:02:56.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.870+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880017070 con 0x7f6898145140 2024-04-25T10:02:56.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.870+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880011bb0 con 0x7f6898145140 2024-04-25T10:02:56.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.870+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68981424d0 con 0x7f6898145140 2024-04-25T10:02:56.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.870+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6898142a90 con 0x7f6898145140 2024-04-25T10:02:56.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.871+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6898073990 con 0x7f6898145140 2024-04-25T10:02:56.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.872+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f688001c460 con 0x7f6898145140 2024-04-25T10:02:56.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.873+0000 7f6894c00640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:56.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.874+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f688008a550 con 0x7f6898145140 2024-04-25T10:02:56.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.874+0000 7f6897e00640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:56.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.874+0000 7f6897e00640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f687406b4d0 0x7f687406d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6888005880 tx=0x7f6888009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:56.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:56.878+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6880058f30 con 0x7f6898145140 2024-04-25T10:02:57.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.075+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68980cb9f0 con 0x7f6898145140 2024-04-25T10:02:57.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.076+0000 7f6894c00640 1 -- 172.21.15.165:0/3460123704 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6880058830 con 0x7f6898145140 2024-04-25T10:02:57.081 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f687406b4d0 msgr2=0x7f687406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f687406b4d0 0x7f687406d990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f6888005880 tx=0x7f6888009040 comp rx=0 tx=0).stop 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 msgr2=0x7f6898146170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 0x7f6898146170 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f688000c430 tx=0x7f688000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 shutdown_connections 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f687406b4d0 0x7f687406d990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f689815cfc0 0x7f6898146bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6898145140 0x7f6898146170 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 --2- 172.21.15.165:0/3460123704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68980ceb20 0x7f689813c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 >> 172.21.15.165:0/3460123704 conn(0x7f6898061cc0 msgr2=0x7f68980ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 shutdown_connections 2024-04-25T10:02:57.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.079+0000 7f689d600640 1 -- 172.21.15.165:0/3460123704 wait complete. 2024-04-25T10:02:57.103 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.302+0000 7f49c2000640 1 Processor -- start 2024-04-25T10:02:57.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.303+0000 7f49c2000640 1 -- start start 2024-04-25T10:02:57.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c2000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49bc145140 0x7f49bc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0ce950 0x7f49bc0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c2000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49bc145520 con 0x7f49bc0c5030 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c2000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49bc0ced90 con 0x7f49bc145140 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc071270 con 0x7f49bc0ce950 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c0c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49bc145140 0x7f49bc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c0c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49bc145140 0x7f49bc0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39164/0 (socket says 172.21.15.165:39164) 2024-04-25T10:02:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49c0c00640 1 -- 172.21.15.165:0/4230283916 learned_addr learned my addr 172.21.15.165:0/4230283916 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.304+0000 7f49bbe00640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0ce950 0x7f49bc0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.305+0000 7f49c0c00640 1 -- 172.21.15.165:0/4230283916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0ce950 msgr2=0x7f49bc0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.305+0000 7f49c0c00640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0ce950 0x7f49bc0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.305+0000 7f49c0c00640 1 -- 172.21.15.165:0/4230283916 >> v1:172.21.15.172:6789/0 conn(0x7f49bc0c5030 legacy=0x7f49bc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.305+0000 7f49c0c00640 1 -- 172.21.15.165:0/4230283916 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc071550 con 0x7f49bc145140 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.305+0000 7f49bbe00640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0ce950 0x7f49bc0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.306+0000 7f49c0c00640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc145140 0x7f49bc0ce410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009870 tx=0x7f49b002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2f83f66243639d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.306+0000 7f49bb400640 1 -- 172.21.15.165:0/4230283916 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0037070 con 0x7f49bc145140 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.306+0000 7f49bb400640 1 -- 172.21.15.165:0/4230283916 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b0032440 con 0x7f49bc145140 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.306+0000 7f49bb400640 1 -- 172.21.15.165:0/4230283916 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b0032810 con 0x7f49bc145140 2024-04-25T10:02:57.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 -- 172.21.15.165:0/4230283916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc145140 msgr2=0x7f49bc0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc145140 0x7f49bc0ce410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009870 tx=0x7f49b002a180 comp rx=0 tx=0).stop 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 -- 172.21.15.165:0/4230283916 shutdown_connections 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0ce950 0x7f49bc0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 --2- 172.21.15.165:0/4230283916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc145140 0x7f49bc0ce410 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 -- 172.21.15.165:0/4230283916 >> 172.21.15.165:0/4230283916 conn(0x7f49bc061cc0 msgr2=0x7f49bc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 -- 172.21.15.165:0/4230283916 shutdown_connections 2024-04-25T10:02:57.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.307+0000 7f49c2000640 1 -- 172.21.15.165:0/4230283916 wait complete. 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.309+0000 7f49c2000640 1 Processor -- start 2024-04-25T10:02:57.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.309+0000 7f49c2000640 1 -- start start 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0c5030 0x7f49bc15b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 0x7f49bc0a88d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49bc145140 0x7f49bc0a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c2000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0c54a0 con 0x7f49bc0ce950 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c2000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc071b10 con 0x7f49bc145140 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc0c5620 con 0x7f49bc0c5030 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 0x7f49bc0a88d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 0x7f49bc0a88d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39178/0 (socket says 172.21.15.165:39178) 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 -- 172.21.15.165:0/1652689802 learned_addr learned my addr 172.21.15.165:0/1652689802 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c0c00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0c5030 0x7f49bc15b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c1600640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49bc145140 0x7f49bc0a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 -- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0c5030 msgr2=0x7f49bc15b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0c5030 0x7f49bc15b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 -- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49bc145140 msgr2=0x7f49bc0a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49bc145140 0x7f49bc0a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49bbe00640 1 -- 172.21.15.165:0/1652689802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc1f4ef0 con 0x7f49bc0ce950 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.310+0000 7f49c1600640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49bc145140 0x7f49bc0a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.311+0000 7f49bbe00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 0x7f49bc0a88d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c7f0 tx=0x7f49a800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.311+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8015070 con 0x7f49bc0ce950 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.311+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49a8007a30 con 0x7f49bc0ce950 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.311+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49a8007e00 con 0x7f49bc0ce950 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.311+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc1f51d0 con 0x7f49bc0ce950 2024-04-25T10:02:57.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.311+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc1f5790 con 0x7f49bc0ce950 2024-04-25T10:02:57.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.312+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc0c4af0 con 0x7f49bc0ce950 2024-04-25T10:02:57.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.313+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49a8018070 con 0x7f49bc0ce950 2024-04-25T10:02:57.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.314+0000 7f49b8c00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f499c06b4d0 0x7f499c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.314+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f49a808a550 con 0x7f49bc0ce950 2024-04-25T10:02:57.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.314+0000 7f49c0c00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f499c06b4d0 0x7f499c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.315+0000 7f49c0c00640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f499c06b4d0 0x7f499c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f49b0034000 tx=0x7f49b002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.319+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49bc0c4af0 con 0x7f49bc0ce950 2024-04-25T10:02:57.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.514+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f49bc067f10 con 0x7f49bc0ce950 2024-04-25T10:02:57.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.515+0000 7f49b8c00640 1 -- 172.21.15.165:0/1652689802 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f49a8058fd0 con 0x7f49bc0ce950 2024-04-25T10:02:57.520 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f499c06b4d0 msgr2=0x7f499c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f499c06b4d0 0x7f499c06d990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f49b0034000 tx=0x7f49b002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 msgr2=0x7f49bc0a88d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 0x7f49bc0a88d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f49a800c7f0 tx=0x7f49a800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 shutdown_connections 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f499c06b4d0 0x7f499c06d990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49bc145140 0x7f49bc0a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49bc0ce950 0x7f49bc0a88d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 --2- 172.21.15.165:0/1652689802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49bc0c5030 0x7f49bc15b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 >> 172.21.15.165:0/1652689802 conn(0x7f49bc061cc0 msgr2=0x7f49bc15c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 shutdown_connections 2024-04-25T10:02:57.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.518+0000 7f49c2000640 1 -- 172.21.15.165:0/1652689802 wait complete. 2024-04-25T10:02:57.542 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.726+0000 7f8bc9c00640 1 Processor -- start 2024-04-25T10:02:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.726+0000 7f8bc9c00640 1 -- start start 2024-04-25T10:02:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc9c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bc40ce230 0x7f8bc40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc9c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8bc415b270 0x7f8bc415b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc9c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc4072280 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc9c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc4071270 con 0x7f8bc415b270 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc9c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40714c0 con 0x7f8bc40ce230 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8bc415b270 0x7f8bc415b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc2000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bc40ce230 0x7f8bc40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8bc415b270 0x7f8bc415b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39190/0 (socket says 172.21.15.165:39190) 2024-04-25T10:02:57.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.728+0000 7f8bc3400640 1 -- 172.21.15.165:0/2665567560 learned_addr learned my addr 172.21.15.165:0/2665567560 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:57.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.729+0000 7f8bc2000640 1 -- 172.21.15.165:0/2665567560 >> v2:172.21.15.165:3300/0 conn(0x7f8bc415b270 msgr2=0x7f8bc415b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:57.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.729+0000 7f8bc2000640 1 --2- 172.21.15.165:0/2665567560 >> v2:172.21.15.165:3300/0 conn(0x7f8bc415b270 0x7f8bc415b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.729+0000 7f8bc2000640 1 -- 172.21.15.165:0/2665567560 >> v1:172.21.15.172:6789/0 conn(0x7f8bc40cc4f0 legacy=0x7f8bc40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.729+0000 7f8bc2000640 1 -- 172.21.15.165:0/2665567560 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40717a0 con 0x7f8bc40ce230 2024-04-25T10:02:57.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.729+0000 7f8bc1600640 1 -- 172.21.15.165:0/2665567560 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3887400532 0 0) 0x7f8bc4072280 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.730+0000 7f8bc2000640 1 --2- 172.21.15.165:0/2665567560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc40cc8d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40b4c00 tx=0x7f8bb002a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e9da04aa2972826 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.730+0000 7f8bc1600640 1 -- 172.21.15.165:0/2665567560 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0037070 con 0x7f8bc40ce230 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.730+0000 7f8bc1600640 1 -- 172.21.15.165:0/2665567560 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb0032440 con 0x7f8bc40ce230 2024-04-25T10:02:57.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.730+0000 7f8bc1600640 1 -- 172.21.15.165:0/2665567560 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0032810 con 0x7f8bc40ce230 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 -- 172.21.15.165:0/2665567560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 msgr2=0x7f8bc40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/2665567560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc40cc8d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40b4c00 tx=0x7f8bb002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 -- 172.21.15.165:0/2665567560 shutdown_connections 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/2665567560 >> v2:172.21.15.165:3300/0 conn(0x7f8bc415b270 0x7f8bc415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/2665567560 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc40cc8d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 -- 172.21.15.165:0/2665567560 >> 172.21.15.165:0/2665567560 conn(0x7f8bc406abe0 msgr2=0x7f8bc406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 -- 172.21.15.165:0/2665567560 shutdown_connections 2024-04-25T10:02:57.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.731+0000 7f8bc9c00640 1 -- 172.21.15.165:0/2665567560 wait complete. 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.733+0000 7f8bc9c00640 1 Processor -- start 2024-04-25T10:02:57.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.734+0000 7f8bc9c00640 1 -- start start 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 0x7f8bc41463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc4151ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc415b270 0x7f8bc41592b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415bc20 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc4071d60 con 0x7f8bc40ce230 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415bda0 con 0x7f8bc415b270 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 0x7f8bc41463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 0x7f8bc41463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39206/0 (socket says 172.21.15.165:39206) 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc2a00640 1 -- 172.21.15.165:0/3735789620 learned_addr learned my addr 172.21.15.165:0/3735789620 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc2000640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc4151ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc3400640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc415b270 0x7f8bc41592b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.735+0000 7f8bc2a00640 1 -- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc415b270 msgr2=0x7f8bc41592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc2a00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc415b270 0x7f8bc41592b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc2a00640 1 -- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 msgr2=0x7f8bc4151ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc2a00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc4151ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc2a00640 1 -- 172.21.15.165:0/3735789620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc4152840 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc3400640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc415b270 0x7f8bc41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc2000640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc2a00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 0x7f8bc41463f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c440 tx=0x7f8bb400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4015070 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb4005480 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc4152b20 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4003830 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.736+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc4153810 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.737+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc4153040 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.738+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bb4018070 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.739+0000 7f8bbaa00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ba406b4d0 0x7f8ba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.739+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f8bb408c4b0 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.739+0000 7f8bc2000640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ba406b4d0 0x7f8ba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.740+0000 7f8bc2000640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ba406b4d0 0x7f8ba406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0034000 tx=0x7f8bb002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:57.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.743+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bb405ae90 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.938+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8bc4073a00 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.939+0000 7f8bbaa00640 1 -- 172.21.15.165:0/3735789620 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8bb405a790 con 0x7f8bc40cc4f0 2024-04-25T10:02:57.943 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ba406b4d0 msgr2=0x7f8ba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ba406b4d0 0x7f8ba406d990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0034000 tx=0x7f8bb002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 msgr2=0x7f8bc41463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 0x7f8bc41463f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c440 tx=0x7f8bb400c900 comp rx=0 tx=0).stop 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 shutdown_connections 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ba406b4d0 0x7f8ba406d990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bc415b270 0x7f8bc41592b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bc40ce230 0x7f8bc4151ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 --2- 172.21.15.165:0/3735789620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bc40cc4f0 0x7f8bc41463f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 >> 172.21.15.165:0/3735789620 conn(0x7f8bc406abe0 msgr2=0x7f8bc406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 shutdown_connections 2024-04-25T10:02:57.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:57.942+0000 7f8bc9c00640 1 -- 172.21.15.165:0/3735789620 wait complete. 2024-04-25T10:02:57.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:02:58.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.165+0000 7fa71b200640 1 Processor -- start 2024-04-25T10:02:58.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.166+0000 7fa71b200640 1 -- start start 2024-04-25T10:02:58.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.167+0000 7fa71b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 0x7fa71c0c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.167+0000 7fa71b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71c145520 con 0x7fa71c145140 2024-04-25T10:02:58.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.167+0000 7fa71b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa71c071270 con 0x7fa71c0ceb20 2024-04-25T10:02:58.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.167+0000 7fa71b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c071550 con 0x7fa71c15cfc0 2024-04-25T10:02:58.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.167+0000 7fa719e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa71c145140 0x7fa71c0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48944/0 (socket says 172.21.15.165:48944) 2024-04-25T10:02:58.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.167+0000 7fa719e00640 1 -- 172.21.15.165:0/355631635 learned_addr learned my addr 172.21.15.165:0/355631635 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa71a800640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 0x7fa71c0c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3508490882 0 0) 0x7fa71c145520 con 0x7fa71c145140 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa700003520 con 0x7fa71c145140 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 815476877 0 0) 0x7fa700003520 con 0x7fa71c145140 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 msgr2=0x7fa71c0c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 0x7fa71c0c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 >> v1:172.21.15.172:6789/0 conn(0x7fa71c0ceb20 legacy=0x7fa71c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c071830 con 0x7fa71c145140 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa70c001690 con 0x7fa71c145140 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 >> v1:172.21.15.165:6789/0 conn(0x7fa71c145140 legacy=0x7fa71c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 0x7fa7000039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa700004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c071550 con 0x7fa700003f60 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70000bb50 con 0x7fa700004d40 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.168+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70000bd50 con 0x7fa71c15cfc0 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa713e00640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa700004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa719e00640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 0x7fa7000039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa713e00640 1 -- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 msgr2=0x7fa7000039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa713e00640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c15cfc0 0x7fa7000039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa713e00640 1 -- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa700004d40 msgr2=0x7fa7000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa71a800640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa713e00640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa700004d40 0x7fa7000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.169+0000 7fa713e00640 1 -- 172.21.15.165:0/355631635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa700003520 con 0x7fa700003f60 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.170+0000 7fa713e00640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa700004800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa704001750 tx=0x7fa700005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61e1618ea03e5d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.170+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704001370 con 0x7fa700003f60 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.170+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7040052d0 con 0x7fa700003f60 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.170+0000 7fa719400640 1 -- 172.21.15.165:0/355631635 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70402abe0 con 0x7fa700003f60 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71b200640 1 -- 172.21.15.165:0/355631635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 msgr2=0x7fa700004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71b200640 1 --2- 172.21.15.165:0/355631635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa700004800 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa704001750 tx=0x7fa700005830 comp rx=0 tx=0).stop 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71a800640 1 -- 172.21.15.165:0/355631635 reap_dead start 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71b200640 1 -- 172.21.15.165:0/355631635 shutdown_connections 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71b200640 1 -- 172.21.15.165:0/355631635 >> 172.21.15.165:0/355631635 conn(0x7fa71c061cc0 msgr2=0x7fa71c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71b200640 1 -- 172.21.15.165:0/355631635 shutdown_connections 2024-04-25T10:02:58.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.171+0000 7fa71b200640 1 -- 172.21.15.165:0/355631635 wait complete. 2024-04-25T10:02:58.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.173+0000 7fa71b200640 1 Processor -- start 2024-04-25T10:02:58.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.173+0000 7fa71b200640 1 -- start start 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa71b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa71c13be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa71b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71c15b4a0 0x7fa71c0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa71b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c141840 0x7fa71c141cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa71b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c5380 con 0x7fa700003f60 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa71b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c5200 con 0x7fa71c15b4a0 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa71b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa71c0c5500 con 0x7fa71c141840 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa719e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa71c13be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa719e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa71c13be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39224/0 (socket says 172.21.15.165:39224) 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.174+0000 7fa719e00640 1 -- 172.21.15.165:0/2939679308 learned_addr learned my addr 172.21.15.165:0/2939679308 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa713e00640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71c15b4a0 0x7fa71c0c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa71a800640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c141840 0x7fa71c141cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa719e00640 1 -- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c141840 msgr2=0x7fa71c141cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa719e00640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c141840 0x7fa71c141cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa719e00640 1 -- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71c15b4a0 msgr2=0x7fa71c0c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa719e00640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71c15b4a0 0x7fa71c0c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa719e00640 1 -- 172.21.15.165:0/2939679308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c0c86a0 con 0x7fa700003f60 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa71a800640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c141840 0x7fa71c141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa719e00640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa71c13be30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e6d0 tx=0x7fa70c00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00c660 con 0x7fa700003f60 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70c018070 con 0x7fa700003f60 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa70c00ca40 con 0x7fa700003f60 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.175+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa71c13c4c0 con 0x7fa700003f60 2024-04-25T10:02:58.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.176+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71c13ca80 con 0x7fa700003f60 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.177+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa71c073990 con 0x7fa700003f60 2024-04-25T10:02:58.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.177+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa70c013420 con 0x7fa700003f60 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.178+0000 7fa712000640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa6f806b500 0x7fa6f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.178+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa70c08a380 con 0x7fa700003f60 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.179+0000 7fa713e00640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa6f806b500 0x7fa6f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.179+0000 7fa713e00640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa6f806b500 0x7fa6f806d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa704001750 tx=0x7fa70402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.183+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa70c058d70 con 0x7fa700003f60 2024-04-25T10:02:58.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.377+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa71c06af60 con 0x7fa700003f60 2024-04-25T10:02:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.378+0000 7fa712000640 1 -- 172.21.15.165:0/2939679308 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa70c058670 con 0x7fa700003f60 2024-04-25T10:02:58.383 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa6f806b500 msgr2=0x7fa6f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa6f806b500 0x7fa6f806d9c0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fa704001750 tx=0x7fa70402e000 comp rx=0 tx=0).stop 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 msgr2=0x7fa71c13be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa71c13be30 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fa70c00e6d0 tx=0x7fa70c00eb90 comp rx=0 tx=0).stop 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 shutdown_connections 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa6f806b500 0x7fa6f806d9c0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa71c141840 0x7fa71c141cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa71c15b4a0 0x7fa71c0c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 --2- 172.21.15.165:0/2939679308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa700003f60 0x7fa71c13be30 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 >> 172.21.15.165:0/2939679308 conn(0x7fa71c061cc0 msgr2=0x7fa71c15dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 shutdown_connections 2024-04-25T10:02:58.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.381+0000 7fa71b200640 1 -- 172.21.15.165:0/2939679308 wait complete. 2024-04-25T10:02:58.405 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:02:58.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.590+0000 7fb2fe200640 1 Processor -- start 2024-04-25T10:02:58.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.590+0000 7fb2fe200640 1 -- start start 2024-04-25T10:02:58.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fe200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2f815e520 0x7fb2f815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fe200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f8070ff0 0x7fb2f8071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fe200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f80ae590 con 0x7fb2f815f040 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fe200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f80aee30 con 0x7fb2f815e520 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fe200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f8071980 con 0x7fb2f8070ff0 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2f815e520 0x7fb2f815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2f815e520 0x7fb2f815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39238/0 (socket says 172.21.15.165:39238) 2024-04-25T10:02:58.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 -- 172.21.15.165:0/3456586558 learned_addr learned my addr 172.21.15.165:0/3456586558 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fd800640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f8070ff0 0x7fb2f8071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 -- 172.21.15.165:0/3456586558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f8070ff0 msgr2=0x7fb2f8071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f8070ff0 0x7fb2f8071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 -- 172.21.15.165:0/3456586558 >> v1:172.21.15.165:6789/0 conn(0x7fb2f815f040 legacy=0x7fb2f80adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.592+0000 7fb2fce00640 1 -- 172.21.15.165:0/3456586558 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f80ae870 con 0x7fb2f815e520 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.593+0000 7fb2fd800640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f8070ff0 0x7fb2f8071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:58.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.594+0000 7fb2fce00640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815e520 0x7fb2f815e900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009870 tx=0x7fb2e802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9bc113eed340b83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.594+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3456586558 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8037070 con 0x7fb2f815e520 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.594+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3456586558 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8032440 con 0x7fb2f815e520 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.594+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3456586558 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8032810 con 0x7fb2f815e520 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.594+0000 7fb2fe200640 1 -- 172.21.15.165:0/3456586558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815e520 msgr2=0x7fb2f815e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.594+0000 7fb2fe200640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815e520 0x7fb2f815e900 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8009870 tx=0x7fb2e802a250 comp rx=0 tx=0).stop 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.595+0000 7fb2fe200640 1 -- 172.21.15.165:0/3456586558 shutdown_connections 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.595+0000 7fb2fe200640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f8070ff0 0x7fb2f8071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.595+0000 7fb2fe200640 1 --2- 172.21.15.165:0/3456586558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815e520 0x7fb2f815e900 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.595+0000 7fb2fe200640 1 -- 172.21.15.165:0/3456586558 >> 172.21.15.165:0/3456586558 conn(0x7fb2f806abe0 msgr2=0x7fb2f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.595+0000 7fb2fe200640 1 -- 172.21.15.165:0/3456586558 shutdown_connections 2024-04-25T10:02:58.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.595+0000 7fb2fe200640 1 -- 172.21.15.165:0/3456586558 wait complete. 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.596+0000 7fb2fe200640 1 Processor -- start 2024-04-25T10:02:58.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.596+0000 7fb2fe200640 1 -- start start 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.597+0000 7fb2fe200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 0x7fb2f815a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.597+0000 7fb2fe200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f815e520 0x7fb2f80ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.597+0000 7fb2fe200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f815f040 0x7fb2f81540c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.597+0000 7fb2fe200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80ae2a0 con 0x7fb2f8070ff0 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.597+0000 7fb2fe200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f806d1f0 con 0x7fb2f815f040 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.597+0000 7fb2fe200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f815f4d0 con 0x7fb2f815e520 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 0x7fb2f815a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 0x7fb2f815a7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39244/0 (socket says 172.21.15.165:39244) 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 -- 172.21.15.165:0/1161686360 learned_addr learned my addr 172.21.15.165:0/1161686360 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 -- 172.21.15.165:0/1161686360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f815e520 msgr2=0x7fb2f80ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f815e520 0x7fb2f80ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 -- 172.21.15.165:0/1161686360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f815f040 msgr2=0x7fb2f81540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f815f040 0x7fb2f81540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 -- 172.21.15.165:0/1161686360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8159730 con 0x7fb2f8070ff0 2024-04-25T10:02:58.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.598+0000 7fb2fce00640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 0x7fb2f815a7d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.599+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8037040 con 0x7fb2f8070ff0 2024-04-25T10:02:58.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.599+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e8042070 con 0x7fb2f8070ff0 2024-04-25T10:02:58.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.599+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e8030bd0 con 0x7fb2f8070ff0 2024-04-25T10:02:58.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.599+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f8159a10 con 0x7fb2f8070ff0 2024-04-25T10:02:58.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.599+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f8159fd0 con 0x7fb2f8070ff0 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.600+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f8073990 con 0x7fb2f8070ff0 2024-04-25T10:02:58.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.601+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2e8047460 con 0x7fb2f8070ff0 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.602+0000 7fb2f5600640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d406b4d0 0x7fb2d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:58.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.602+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb2e80ab340 con 0x7fb2f8070ff0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.602+0000 7fb2efe00640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d406b4d0 0x7fb2d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:58.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.603+0000 7fb2efe00640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d406b4d0 0x7fb2d406d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2f81f6c30 tx=0x7fb2e0007970 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:58.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.606+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2e8045360 con 0x7fb2f8070ff0 2024-04-25T10:02:58.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.786+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2f8162160 con 0x7fb2f8070ff0 2024-04-25T10:02:58.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.789+0000 7fb2f5600640 1 -- 172.21.15.165:0/1161686360 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7fb2e8079d20 con 0x7fb2f8070ff0 2024-04-25T10:02:58.794 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:02:58.794 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:02:58.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d406b4d0 msgr2=0x7fb2d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d406b4d0 0x7fb2d406d990 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fb2f81f6c30 tx=0x7fb2e0007970 comp rx=0 tx=0).stop 2024-04-25T10:02:58.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 msgr2=0x7fb2f815a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:58.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 0x7fb2f815a7d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8000c00 tx=0x7fb2e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 shutdown_connections 2024-04-25T10:02:58.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d406b4d0 0x7fb2d406d990 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f815f040 0x7fb2f81540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f815e520 0x7fb2f80ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 --2- 172.21.15.165:0/1161686360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f8070ff0 0x7fb2f815a7d0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.791+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 >> 172.21.15.165:0/1161686360 conn(0x7fb2f806abe0 msgr2=0x7fb2f80af050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.792+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 shutdown_connections 2024-04-25T10:02:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:58.792+0000 7fb2fe200640 1 -- 172.21.15.165:0/1161686360 wait complete. 2024-04-25T10:02:58.814 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:02:58.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:02:59.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.005+0000 7f6007200640 1 Processor -- start 2024-04-25T10:02:59.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.005+0000 7f6007200640 1 -- start start 2024-04-25T10:02:59.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6007200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f600815dd30 0x7f60080cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6007200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f600806bff0 0x7f60080b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6007200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600815e110 con 0x7f60080b5970 2024-04-25T10:02:59.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6007200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600806d000 con 0x7f600815dd30 2024-04-25T10:02:59.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6007200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080b2fd0 con 0x7f600806bff0 2024-04-25T10:02:59.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6005e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f600815dd30 0x7f60080cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6005e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f600815dd30 0x7f60080cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39248/0 (socket says 172.21.15.165:39248) 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6005e00640 1 -- 172.21.15.165:0/53234401 learned_addr learned my addr 172.21.15.165:0/53234401 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.007+0000 7f6005400640 1 --2- 172.21.15.165:0/53234401 >> v2:172.21.15.172:3300/0 conn(0x7f600806bff0 0x7f60080b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.008+0000 7f6005e00640 1 -- 172.21.15.165:0/53234401 >> v2:172.21.15.172:3300/0 conn(0x7f600806bff0 msgr2=0x7f60080b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.008+0000 7f6005e00640 1 --2- 172.21.15.165:0/53234401 >> v2:172.21.15.172:3300/0 conn(0x7f600806bff0 0x7f60080b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.008+0000 7f6005e00640 1 -- 172.21.15.165:0/53234401 >> v1:172.21.15.172:6789/0 conn(0x7f60080b5970 legacy=0x7f6008161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.008+0000 7f6005e00640 1 -- 172.21.15.165:0/53234401 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600806c480 con 0x7f600815dd30 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.008+0000 7f6005400640 1 --2- 172.21.15.165:0/53234401 >> v2:172.21.15.172:3300/0 conn(0x7f600806bff0 0x7f60080b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:02:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.009+0000 7f6005e00640 1 --2- 172.21.15.165:0/53234401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cebd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f600808c2c0 tx=0x7f5ff802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca0f471139cb9daa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6004a00640 1 -- 172.21.15.165:0/53234401 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037070 con 0x7f600815dd30 2024-04-25T10:02:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6004a00640 1 -- 172.21.15.165:0/53234401 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff8032440 con 0x7f600815dd30 2024-04-25T10:02:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6004a00640 1 -- 172.21.15.165:0/53234401 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8032810 con 0x7f600815dd30 2024-04-25T10:02:59.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6007200640 1 -- 172.21.15.165:0/53234401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 msgr2=0x7f60080cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6007200640 1 --2- 172.21.15.165:0/53234401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cebd0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f600808c2c0 tx=0x7f5ff802a230 comp rx=0 tx=0).stop 2024-04-25T10:02:59.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6007200640 1 -- 172.21.15.165:0/53234401 shutdown_connections 2024-04-25T10:02:59.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6007200640 1 --2- 172.21.15.165:0/53234401 >> v2:172.21.15.172:3300/0 conn(0x7f600806bff0 0x7f60080b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6007200640 1 --2- 172.21.15.165:0/53234401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cebd0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.010+0000 7f6007200640 1 -- 172.21.15.165:0/53234401 >> 172.21.15.165:0/53234401 conn(0x7f60080d0610 msgr2=0x7f60080d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.011+0000 7f6007200640 1 -- 172.21.15.165:0/53234401 shutdown_connections 2024-04-25T10:02:59.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.011+0000 7f6007200640 1 -- 172.21.15.165:0/53234401 wait complete. 2024-04-25T10:02:59.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.012+0000 7f6007200640 1 Processor -- start 2024-04-25T10:02:59.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.012+0000 7f6007200640 1 -- start start 2024-04-25T10:02:59.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.013+0000 7f6007200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60080b5970 0x7f6008156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.013+0000 7f6007200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.013+0000 7f6007200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cc520 0x7f60081552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.013+0000 7f6007200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600806cd20 con 0x7f600815dd30 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.013+0000 7f6007200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600806ca40 con 0x7f60080cc520 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.013+0000 7f6007200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60080cf110 con 0x7f60080b5970 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cb420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39260/0 (socket says 172.21.15.165:39260) 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 -- 172.21.15.165:0/1925423988 learned_addr learned my addr 172.21.15.165:0/1925423988 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005e00640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60080b5970 0x7f6008156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 -- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60080b5970 msgr2=0x7f6008156f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60080b5970 0x7f6008156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 -- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cc520 msgr2=0x7f60081552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cc520 0x7f60081552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 -- 172.21.15.165:0/1925423988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008157490 con 0x7f600815dd30 2024-04-25T10:02:59.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.014+0000 7f6005400640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cb420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e7e0 tx=0x7f5ff000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.015+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff000f040 con 0x7f600815dd30 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.015+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff0004030 con 0x7f600815dd30 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.015+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff0004400 con 0x7f600815dd30 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.015+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6008157860 con 0x7f600815dd30 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.015+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6008157e20 con 0x7f600815dd30 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.016+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008073990 con 0x7f600815dd30 2024-04-25T10:02:59.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.017+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ff000c6d0 con 0x7f600815dd30 2024-04-25T10:02:59.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.018+0000 7f5ffe000640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.018+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5ff008a410 con 0x7f600815dd30 2024-04-25T10:02:59.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.018+0000 7f6005e00640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.018+0000 7f6005e00640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8034000 tx=0x7f5ff802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.022+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff0017020 con 0x7f600815dd30 2024-04-25T10:02:59.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.220+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f60080d16c0 con 0x7f600815dd30 2024-04-25T10:02:59.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.221+0000 7f5ffe000640 1 -- 172.21.15.165:0/1925423988 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ff0058e80 con 0x7f600815dd30 2024-04-25T10:02:59.226 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:02:59.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe406b4d0 msgr2=0x7f5fe406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe406b4d0 0x7f5fe406d990 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8034000 tx=0x7f5ff802e000 comp rx=0 tx=0).stop 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 msgr2=0x7f60080cb420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cb420 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f5ff000e7e0 tx=0x7f5ff000eca0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 shutdown_connections 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe406b4d0 0x7f5fe406d990 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cc520 0x7f60081552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600815dd30 0x7f60080cb420 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 --2- 172.21.15.165:0/1925423988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60080b5970 0x7f6008156f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 >> 172.21.15.165:0/1925423988 conn(0x7f60080d0610 msgr2=0x7f60080b4cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 shutdown_connections 2024-04-25T10:02:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.224+0000 7f6007200640 1 -- 172.21.15.165:0/1925423988 wait complete. 2024-04-25T10:02:59.247 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:02:59.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.444+0000 7fd7c7200640 1 Processor -- start 2024-04-25T10:02:59.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.444+0000 7fd7c7200640 1 -- start start 2024-04-25T10:02:59.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.446+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c813d9f0 0x7fd7c813df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.446+0000 7fd7c7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7c80b0730 0x7fd7c80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.446+0000 7fd7c7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c806d390 con 0x7fd7c806e150 2024-04-25T10:02:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.446+0000 7fd7c7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c80b10d0 con 0x7fd7c80b0730 2024-04-25T10:02:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.446+0000 7fd7c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c806d670 con 0x7fd7c813d9f0 2024-04-25T10:02:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.446+0000 7fd7bfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7c80b0730 0x7fd7c80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7bfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7c80b0730 0x7fd7c80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:38168/0 (socket says 172.21.15.165:38168) 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7bfe00640 1 -- 172.21.15.165:0/294303547 learned_addr learned my addr 172.21.15.165:0/294303547 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7c5e00640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c813d9f0 0x7fd7c813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7bfe00640 1 -- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c813d9f0 msgr2=0x7fd7c813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c813d9f0 0x7fd7c813df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7bfe00640 1 -- 172.21.15.165:0/294303547 >> v1:172.21.15.172:6789/0 conn(0x7fd7c806e150 legacy=0x7fd7c806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7bfe00640 1 -- 172.21.15.165:0/294303547 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c806d950 con 0x7fd7c80b0730 2024-04-25T10:02:59.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.447+0000 7fd7c5e00640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c813d9f0 0x7fd7c813df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:02:59.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.449+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c80b0730 0x7fd7c80b0b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009870 tx=0x7fd7b002a130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=811b3e1d42ab1b5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.449+0000 7fd7c5400640 1 -- 172.21.15.165:0/294303547 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0037070 con 0x7fd7c80b0730 2024-04-25T10:02:59.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.449+0000 7fd7c5400640 1 -- 172.21.15.165:0/294303547 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b0032440 con 0x7fd7c80b0730 2024-04-25T10:02:59.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.449+0000 7fd7c5400640 1 -- 172.21.15.165:0/294303547 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b0032810 con 0x7fd7c80b0730 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 -- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c80b0730 msgr2=0x7fd7c80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c80b0730 0x7fd7c80b0b90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0009870 tx=0x7fd7b002a130 comp rx=0 tx=0).stop 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 -- 172.21.15.165:0/294303547 shutdown_connections 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c80b0730 0x7fd7c80b0b90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 --2- 172.21.15.165:0/294303547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c813d9f0 0x7fd7c813df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 -- 172.21.15.165:0/294303547 >> 172.21.15.165:0/294303547 conn(0x7fd7c8061cc0 msgr2=0x7fd7c806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 -- 172.21.15.165:0/294303547 shutdown_connections 2024-04-25T10:02:59.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.450+0000 7fd7c7200640 1 -- 172.21.15.165:0/294303547 wait complete. 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.452+0000 7fd7c7200640 1 Processor -- start 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.452+0000 7fd7c7200640 1 -- start start 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 0x7fd7c8157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c80b0730 0x7fd7c80cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c813d9f0 0x7fd7c812fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c813e5f0 con 0x7fd7c806e150 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c813e470 con 0x7fd7c813d9f0 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c80ce880 con 0x7fd7c80b0730 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 0x7fd7c8157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 0x7fd7c8157f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39270/0 (socket says 172.21.15.165:39270) 2024-04-25T10:02:59.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c5e00640 1 -- 172.21.15.165:0/1880340115 learned_addr learned my addr 172.21.15.165:0/1880340115 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c6800640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c813d9f0 0x7fd7c812fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c80b0730 0x7fd7c80cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c5e00640 1 -- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c80b0730 msgr2=0x7fd7c80cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.453+0000 7fd7c5e00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c80b0730 0x7fd7c80cc780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c5e00640 1 -- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c813d9f0 msgr2=0x7fd7c812fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c5e00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c813d9f0 0x7fd7c812fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c5e00640 1 -- 172.21.15.165:0/1880340115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c80ccf20 con 0x7fd7c806e150 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c6800640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c813d9f0 0x7fd7c812fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c80b0730 0x7fd7c80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c5e00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 0x7fd7c8157f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800e960 tx=0x7fd7b800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8017070 con 0x7fd7c806e150 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b800ca00 con 0x7fd7c806e150 2024-04-25T10:02:59.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c80cd200 con 0x7fd7c806e150 2024-04-25T10:02:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b800cdd0 con 0x7fd7c806e150 2024-04-25T10:02:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.454+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c80cd7c0 con 0x7fd7c806e150 2024-04-25T10:02:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.455+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c8073990 con 0x7fd7c806e150 2024-04-25T10:02:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.457+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7b801a070 con 0x7fd7c806e150 2024-04-25T10:02:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.457+0000 7fd7be000640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd7a406b4d0 0x7fd7a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.457+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd7b808a870 con 0x7fd7c806e150 2024-04-25T10:02:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.458+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd7a406b4d0 0x7fd7a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.458+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd7a406b4d0 0x7fd7a406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0034000 tx=0x7fd7b002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.462+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7b80592e0 con 0x7fd7c806e150 2024-04-25T10:02:59.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.641+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7c813dfe0 con 0x7fd7c806e150 2024-04-25T10:02:59.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.642+0000 7fd7be000640 1 -- 172.21.15.165:0/1880340115 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7b8058be0 con 0x7fd7c806e150 2024-04-25T10:02:59.647 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:02:59.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.644+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd7a406b4d0 msgr2=0x7fd7a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.644+0000 7fd7c7200640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd7a406b4d0 0x7fd7a406d990 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0034000 tx=0x7fd7b002e000 comp rx=0 tx=0).stop 2024-04-25T10:02:59.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.644+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 msgr2=0x7fd7c8157f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.644+0000 7fd7c7200640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 0x7fd7c8157f90 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800e960 tx=0x7fd7b800ee20 comp rx=0 tx=0).stop 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 shutdown_connections 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd7a406b4d0 0x7fd7a406d990 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c813d9f0 0x7fd7c812fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c80b0730 0x7fd7c80cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 --2- 172.21.15.165:0/1880340115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c806e150 0x7fd7c8157f90 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 >> 172.21.15.165:0/1880340115 conn(0x7fd7c8061cc0 msgr2=0x7fd7c813f460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 shutdown_connections 2024-04-25T10:02:59.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.645+0000 7fd7c7200640 1 -- 172.21.15.165:0/1880340115 wait complete. 2024-04-25T10:02:59.666 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:02:59.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.858+0000 7f9347400640 1 Processor -- start 2024-04-25T10:02:59.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.858+0000 7f9347400640 1 -- start start 2024-04-25T10:02:59.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9347400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f93401446d0 0x7f93400b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9347400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9347400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f93401452a0 0x7f93400cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9347400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93400b2b20 con 0x7f93401452a0 2024-04-25T10:02:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9347400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f934006d400 con 0x7f93401446d0 2024-04-25T10:02:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9347400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93400b49a0 con 0x7f9340144d50 2024-04-25T10:02:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f93401452a0 0x7f93400cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f93401452a0 0x7f93400cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39284/0 (socket says 172.21.15.165:39284) 2024-04-25T10:02:59.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 -- 172.21.15.165:0/39396347 learned_addr learned my addr 172.21.15.165:0/39396347 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f933fe00640 1 --2- 172.21.15.165:0/39396347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9344a00640 1 --2- 172.21.15.165:0/39396347 >> v2:172.21.15.172:3300/0 conn(0x7f93401446d0 0x7f93400b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 -- 172.21.15.165:0/39396347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 msgr2=0x7f93400b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 --2- 172.21.15.165:0/39396347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 -- 172.21.15.165:0/39396347 >> v2:172.21.15.172:3300/0 conn(0x7f93401446d0 msgr2=0x7f93400b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 --2- 172.21.15.165:0/39396347 >> v2:172.21.15.172:3300/0 conn(0x7f93401446d0 0x7f93400b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9345400640 1 -- 172.21.15.165:0/39396347 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400cccc0 con 0x7f93401452a0 2024-04-25T10:02:59.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.860+0000 7f9344a00640 1 --2- 172.21.15.165:0/39396347 >> v2:172.21.15.172:3300/0 conn(0x7f93401446d0 0x7f93400b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:02:59.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9345400640 1 --2- 172.21.15.165:0/39396347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401452a0 0x7f93400cf640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93400cf560 tx=0x7f933802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e71d9b32f514046 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f933f400640 1 -- 172.21.15.165:0/39396347 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338037070 con 0x7f93401452a0 2024-04-25T10:02:59.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f933f400640 1 -- 172.21.15.165:0/39396347 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9338032440 con 0x7f93401452a0 2024-04-25T10:02:59.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f933f400640 1 -- 172.21.15.165:0/39396347 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9338032810 con 0x7f93401452a0 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 -- 172.21.15.165:0/39396347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401452a0 msgr2=0x7f93400cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 --2- 172.21.15.165:0/39396347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401452a0 0x7f93400cf640 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f93400cf560 tx=0x7f933802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 -- 172.21.15.165:0/39396347 shutdown_connections 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 --2- 172.21.15.165:0/39396347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401452a0 0x7f93400cf640 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 --2- 172.21.15.165:0/39396347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 --2- 172.21.15.165:0/39396347 >> v2:172.21.15.172:3300/0 conn(0x7f93401446d0 0x7f93400b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.862+0000 7f9347400640 1 -- 172.21.15.165:0/39396347 >> 172.21.15.165:0/39396347 conn(0x7f934006abe0 msgr2=0x7f934006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:02:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.863+0000 7f9347400640 1 -- 172.21.15.165:0/39396347 shutdown_connections 2024-04-25T10:02:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.863+0000 7f9347400640 1 -- 172.21.15.165:0/39396347 wait complete. 2024-04-25T10:02:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.864+0000 7f9347400640 1 Processor -- start 2024-04-25T10:02:59.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.864+0000 7f9347400640 1 -- start start 2024-04-25T10:02:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9347400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 0x7f93401f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9347400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400a9930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9347400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93401452a0 0x7f93400ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9347400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93400cc700 con 0x7f93401446d0 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9347400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93400cc420 con 0x7f93401452a0 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9347400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93400cc9e0 con 0x7f9340144d50 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.865+0000 7f9344a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 0x7f93401f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 0x7f93401f4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39296/0 (socket says 172.21.15.165:39296) 2024-04-25T10:02:59.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 -- 172.21.15.165:0/4106394849 learned_addr learned my addr 172.21.15.165:0/4106394849 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9345400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93401452a0 0x7f93400ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 -- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 msgr2=0x7f93400a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 -- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93401452a0 msgr2=0x7f93400ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93401452a0 0x7f93400ad650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 -- 172.21.15.165:0/4106394849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93400ac5b0 con 0x7f93401446d0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f9344a00640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 0x7f93401f4d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f932800c7c0 tx=0x7f932800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328015070 con 0x7f93401446d0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9328007a30 con 0x7f93401446d0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.866+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9328007e00 con 0x7f93401446d0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.867+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93400acb70 con 0x7f93401446d0 2024-04-25T10:02:59.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.867+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93400ad130 con 0x7f93401446d0 2024-04-25T10:02:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.868+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93400ad330 con 0x7f93401446d0 2024-04-25T10:02:59.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.868+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9328018070 con 0x7f93401446d0 2024-04-25T10:02:59.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.869+0000 7f933cc00640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f931c06b4d0 0x7f931c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:02:59.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.869+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f932808a570 con 0x7f93401446d0 2024-04-25T10:02:59.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.870+0000 7f933fe00640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f931c06b4d0 0x7f931c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:02:59.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.870+0000 7f933fe00640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f931c06b4d0 0x7f931c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f93401ef120 tx=0x7f9330007390 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:02:59.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:02:59.874+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93400ad330 con 0x7f93401446d0 2024-04-25T10:03:00.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.068+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9340073a00 con 0x7f93401446d0 2024-04-25T10:03:00.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.069+0000 7f933cc00640 1 -- 172.21.15.165:0/4106394849 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9328058ff0 con 0x7f93401446d0 2024-04-25T10:03:00.073 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f931c06b4d0 msgr2=0x7f931c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f931c06b4d0 0x7f931c06d990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f93401ef120 tx=0x7f9330007390 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 msgr2=0x7f93401f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 0x7f93401f4d90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f932800c7c0 tx=0x7f932800cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 shutdown_connections 2024-04-25T10:03:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f931c06b4d0 0x7f931c06d990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93401452a0 0x7f93400ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9340144d50 0x7f93400a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 --2- 172.21.15.165:0/4106394849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93401446d0 0x7f93401f4d90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.071+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 >> 172.21.15.165:0/4106394849 conn(0x7f934006abe0 msgr2=0x7f934006d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.072+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 shutdown_connections 2024-04-25T10:03:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.072+0000 7f9347400640 1 -- 172.21.15.165:0/4106394849 wait complete. 2024-04-25T10:03:00.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:00.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.288+0000 7f859b000640 1 Processor -- start 2024-04-25T10:03:00.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.288+0000 7f859b000640 1 -- start start 2024-04-25T10:03:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f859b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8594144d50 0x7f859406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f859b000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85941452a0 0x7f859406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f859b000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85940cccc0 con 0x7f85941446d0 2024-04-25T10:03:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f859b000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85940b2b20 con 0x7f85941452a0 2024-04-25T10:03:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f859b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f859406d400 con 0x7f8594144d50 2024-04-25T10:03:00.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f8599000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85941452a0 0x7f859406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f8599000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85941452a0 0x7f859406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39304/0 (socket says 172.21.15.165:39304) 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f8599000640 1 -- 172.21.15.165:0/181625681 learned_addr learned my addr 172.21.15.165:0/181625681 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f8599000640 1 -- 172.21.15.165:0/181625681 >> v2:172.21.15.172:3300/0 conn(0x7f8594144d50 msgr2=0x7f859406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.290+0000 7f858b400640 1 --2- 172.21.15.165:0/181625681 >> v2:172.21.15.172:3300/0 conn(0x7f8594144d50 0x7f859406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.291+0000 7f8599000640 1 --2- 172.21.15.165:0/181625681 >> v2:172.21.15.172:3300/0 conn(0x7f8594144d50 0x7f859406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.291+0000 7f8599000640 1 -- 172.21.15.165:0/181625681 >> v1:172.21.15.172:6789/0 conn(0x7f85941446d0 legacy=0x7f85940b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.291+0000 7f8599000640 1 -- 172.21.15.165:0/181625681 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85940cc140 con 0x7f85941452a0 2024-04-25T10:03:00.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.291+0000 7f858b400640 1 --2- 172.21.15.165:0/181625681 >> v2:172.21.15.172:3300/0 conn(0x7f8594144d50 0x7f859406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.292+0000 7f8599000640 1 --2- 172.21.15.165:0/181625681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859406ddd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8594071c70 tx=0x7f858c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e504e3ac33ee90c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.292+0000 7f8593400640 1 -- 172.21.15.165:0/181625681 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c036070 con 0x7f85941452a0 2024-04-25T10:03:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.292+0000 7f8593400640 1 -- 172.21.15.165:0/181625681 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c031600 con 0x7f85941452a0 2024-04-25T10:03:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.292+0000 7f8593400640 1 -- 172.21.15.165:0/181625681 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c0319d0 con 0x7f85941452a0 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 -- 172.21.15.165:0/181625681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 msgr2=0x7f859406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 --2- 172.21.15.165:0/181625681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859406ddd0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f8594071c70 tx=0x7f858c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 -- 172.21.15.165:0/181625681 shutdown_connections 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 --2- 172.21.15.165:0/181625681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859406ddd0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 --2- 172.21.15.165:0/181625681 >> v2:172.21.15.172:3300/0 conn(0x7f8594144d50 0x7f859406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 -- 172.21.15.165:0/181625681 >> 172.21.15.165:0/181625681 conn(0x7f859406abe0 msgr2=0x7f859406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 -- 172.21.15.165:0/181625681 shutdown_connections 2024-04-25T10:03:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.293+0000 7f859b000640 1 -- 172.21.15.165:0/181625681 wait complete. 2024-04-25T10:03:00.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.295+0000 7f859b000640 1 Processor -- start 2024-04-25T10:03:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.295+0000 7f859b000640 1 -- start start 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.296+0000 7f859b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85941446d0 0x7f859415b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.296+0000 7f859b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8594144d50 0x7f85940aad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f859b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859413c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f859b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940cc9e0 con 0x7f85941452a0 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f859b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85940cc700 con 0x7f8594144d50 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f859b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8594144ab0 con 0x7f85941446d0 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859413c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859413c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39312/0 (socket says 172.21.15.165:39312) 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 -- 172.21.15.165:0/881433442 learned_addr learned my addr 172.21.15.165:0/881433442 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8593e00640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85941446d0 0x7f859415b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f858b400640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8594144d50 0x7f85940aad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 -- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85941446d0 msgr2=0x7f859415b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85941446d0 0x7f859415b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 -- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8594144d50 msgr2=0x7f85940aad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8594144d50 0x7f85940aad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f8599000640 1 -- 172.21.15.165:0/881433442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859413d0d0 con 0x7f85941452a0 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.297+0000 7f858b400640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8594144d50 0x7f85940aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:00.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.298+0000 7f8599000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859413c610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f858c009f40 tx=0x7f858c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.298+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c036050 con 0x7f85941452a0 2024-04-25T10:03:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.298+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c042070 con 0x7f85941452a0 2024-04-25T10:03:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.298+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859413d3b0 con 0x7f85941452a0 2024-04-25T10:03:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.298+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c009510 con 0x7f85941452a0 2024-04-25T10:03:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.298+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85940aa870 con 0x7f85941452a0 2024-04-25T10:03:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.300+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85940aa390 con 0x7f85941452a0 2024-04-25T10:03:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.300+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f858c007bd0 con 0x7f85941452a0 2024-04-25T10:03:00.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.301+0000 7f8590c00640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f857006b4d0 0x7f857006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.301+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f858c0ab660 con 0x7f85941452a0 2024-04-25T10:03:00.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.302+0000 7f8593e00640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f857006b4d0 0x7f857006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.302+0000 7f8593e00640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f857006b4d0 0x7f857006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f858400a730 tx=0x7f8584008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.309+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f858c07a040 con 0x7f85941452a0 2024-04-25T10:03:00.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.507+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f8594073a00 con 0x7f85941452a0 2024-04-25T10:03:00.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.508+0000 7f8590c00640 1 -- 172.21.15.165:0/881433442 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f858c079940 con 0x7f85941452a0 2024-04-25T10:03:00.513 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:00.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f857006b4d0 msgr2=0x7f857006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f857006b4d0 0x7f857006d990 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f858400a730 tx=0x7f8584008040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 msgr2=0x7f859413c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859413c610 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f858c009f40 tx=0x7f858c004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 shutdown_connections 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f857006b4d0 0x7f857006d990 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85941452a0 0x7f859413c610 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8594144d50 0x7f85940aad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 --2- 172.21.15.165:0/881433442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85941446d0 0x7f859415b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 >> 172.21.15.165:0/881433442 conn(0x7f859406abe0 msgr2=0x7f859406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 shutdown_connections 2024-04-25T10:03:00.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.511+0000 7f859b000640 1 -- 172.21.15.165:0/881433442 wait complete. 2024-04-25T10:03:00.549 INFO:tasks.cephfs_test_runner:Starting test: test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) test flag scrub_mdsdir 2024-04-25T10:03:00.550 INFO:tasks.cephfs_test_runner:test_flag_scrub_mdsdir (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:03:00.550 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.747+0000 7ff85c000640 1 Processor -- start 2024-04-25T10:03:00.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.747+0000 7ff85c000640 1 -- start start 2024-04-25T10:03:00.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.749+0000 7ff85c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff8540c6df0 0x7ff8540d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.749+0000 7ff85c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 0x7ff8540ccc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.749+0000 7ff85c000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff85415dfe0 con 0x7ff85415dc00 2024-04-25T10:03:00.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.749+0000 7ff85c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8540d2090 con 0x7ff8540c6df0 2024-04-25T10:03:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.749+0000 7ff85c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540d00c0 con 0x7ff8540d1c00 2024-04-25T10:03:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.749+0000 7ff85a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 0x7ff8540ccc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff858c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff8540c6df0 0x7ff8540d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff85a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 0x7ff8540ccc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:46066/0 (socket says 172.21.15.165:46066) 2024-04-25T10:03:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff85a000640 1 -- 172.21.15.165:0/2303792112 learned_addr learned my addr 172.21.15.165:0/2303792112 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff85a000640 1 -- 172.21.15.165:0/2303792112 >> v2:172.21.15.172:3300/0 conn(0x7ff8540c6df0 msgr2=0x7ff8540d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff85a000640 1 --2- 172.21.15.165:0/2303792112 >> v2:172.21.15.172:3300/0 conn(0x7ff8540c6df0 0x7ff8540d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff85a000640 1 -- 172.21.15.165:0/2303792112 >> v1:172.21.15.172:6789/0 conn(0x7ff85415dc00 legacy=0x7ff8540ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.750+0000 7ff85a000640 1 -- 172.21.15.165:0/2303792112 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540d03a0 con 0x7ff8540d1c00 2024-04-25T10:03:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.752+0000 7ff85a000640 1 --2- 172.21.15.165:0/2303792112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 0x7ff8540ccc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009e50 tx=0x7ff84c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e21f624f6c76c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.752+0000 7ff853e00640 1 -- 172.21.15.165:0/2303792112 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c036070 con 0x7ff8540d1c00 2024-04-25T10:03:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.752+0000 7ff853e00640 1 -- 172.21.15.165:0/2303792112 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c031600 con 0x7ff8540d1c00 2024-04-25T10:03:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.752+0000 7ff853e00640 1 -- 172.21.15.165:0/2303792112 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c0319d0 con 0x7ff8540d1c00 2024-04-25T10:03:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 -- 172.21.15.165:0/2303792112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 msgr2=0x7ff8540ccc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 --2- 172.21.15.165:0/2303792112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 0x7ff8540ccc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff84c009e50 tx=0x7ff84c02a710 comp rx=0 tx=0).stop 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 -- 172.21.15.165:0/2303792112 shutdown_connections 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 --2- 172.21.15.165:0/2303792112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8540d1c00 0x7ff8540ccc40 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 --2- 172.21.15.165:0/2303792112 >> v2:172.21.15.172:3300/0 conn(0x7ff8540c6df0 0x7ff8540d16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 -- 172.21.15.165:0/2303792112 >> 172.21.15.165:0/2303792112 conn(0x7ff85406f5d0 msgr2=0x7ff85406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 -- 172.21.15.165:0/2303792112 shutdown_connections 2024-04-25T10:03:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.753+0000 7ff85c000640 1 -- 172.21.15.165:0/2303792112 wait complete. 2024-04-25T10:03:00.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.755+0000 7ff85c000640 1 Processor -- start 2024-04-25T10:03:00.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.755+0000 7ff85c000640 1 -- start start 2024-04-25T10:03:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff85c000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 0x7ff85414a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff85c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8540d1c00 0x7ff8540b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff85c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff85415dc00 0x7ff8540b2430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff85c000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540cc130 con 0x7ff8540c6df0 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff85c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540cc720 con 0x7ff8540d1c00 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff85c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540cfd90 con 0x7ff85415dc00 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff859600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 0x7ff85414a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff859600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 0x7ff85414a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39322/0 (socket says 172.21.15.165:39322) 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.756+0000 7ff859600640 1 -- 172.21.15.165:0/3798681291 learned_addr learned my addr 172.21.15.165:0/3798681291 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff85a000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff85415dc00 0x7ff8540b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff858c00640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8540d1c00 0x7ff8540b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff859600640 1 -- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff85415dc00 msgr2=0x7ff8540b2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff859600640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff85415dc00 0x7ff8540b2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff859600640 1 -- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8540d1c00 msgr2=0x7ff8540b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff859600640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8540d1c00 0x7ff8540b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff859600640 1 -- 172.21.15.165:0/3798681291 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff85414a780 con 0x7ff8540c6df0 2024-04-25T10:03:00.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff85a000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff85415dc00 0x7ff8540b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:00.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff859600640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 0x7ff85414a240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c3c0 tx=0x7ff84800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8480052d0 con 0x7ff8540c6df0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff848005490 con 0x7ff8540c6df0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8540b4f30 con 0x7ff8540c6df0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.757+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff848007260 con 0x7ff8540c6df0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.758+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8540b54f0 con 0x7ff8540c6df0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.759+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff854073990 con 0x7ff8540c6df0 2024-04-25T10:03:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.760+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff848007400 con 0x7ff8540c6df0 2024-04-25T10:03:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.760+0000 7ff851600640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff83406b4d0 0x7ff83406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.761+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff848089c10 con 0x7ff8540c6df0 2024-04-25T10:03:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.761+0000 7ff858c00640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff83406b4d0 0x7ff83406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:00.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.762+0000 7ff858c00640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff83406b4d0 0x7ff83406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8540ca050 tx=0x7ff840008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:00.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.765+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff85415dfe0 con 0x7ff8540c6df0 2024-04-25T10:03:00.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.962+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8540c66c0 con 0x7ff8540c6df0 2024-04-25T10:03:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.966+0000 7ff851600640 1 -- 172.21.15.165:0/3798681291 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7ff848058560 con 0x7ff8540c6df0 2024-04-25T10:03:00.971 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:00.971 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.968+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff83406b4d0 msgr2=0x7ff83406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.968+0000 7ff85c000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff83406b4d0 0x7ff83406d990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff8540ca050 tx=0x7ff840008040 comp rx=0 tx=0).stop 2024-04-25T10:03:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.968+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 msgr2=0x7ff85414a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.968+0000 7ff85c000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 0x7ff85414a240 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff84800c3c0 tx=0x7ff84800c880 comp rx=0 tx=0).stop 2024-04-25T10:03:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 shutdown_connections 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff83406b4d0 0x7ff83406d990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff85415dc00 0x7ff8540b2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8540d1c00 0x7ff8540b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 --2- 172.21.15.165:0/3798681291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8540c6df0 0x7ff85414a240 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 >> 172.21.15.165:0/3798681291 conn(0x7ff85406f5d0 msgr2=0x7ff8540c7250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 shutdown_connections 2024-04-25T10:03:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:00.969+0000 7ff85c000640 1 -- 172.21.15.165:0/3798681291 wait complete. 2024-04-25T10:03:00.993 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:00.994 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:01.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.192+0000 7f33eda00640 1 Processor -- start 2024-04-25T10:03:01.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.193+0000 7f33eda00640 1 -- start start 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.194+0000 7f33eda00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33e815b710 0x7f33e80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.194+0000 7f33eda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 0x7f33e80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.194+0000 7f33eda00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e806f0e0 con 0x7f33e806ec50 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.194+0000 7f33eda00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e815e490 con 0x7f33e815b710 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.194+0000 7f33eda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80cfd80 con 0x7f33e80ac8b0 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.195+0000 7f33dfe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f33e806ec50 0x7f33e815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48956/0 (socket says 172.21.15.165:48956) 2024-04-25T10:03:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.195+0000 7f33dfe00640 1 -- 172.21.15.165:0/1467791843 learned_addr learned my addr 172.21.15.165:0/1467791843 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.195+0000 7f33e6a00640 1 --2- 172.21.15.165:0/1467791843 >> v2:172.21.15.172:3300/0 conn(0x7f33e815b710 0x7f33e80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.195+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1229887393 0 0) 0x7f33e806f0e0 con 0x7f33e806ec50 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.195+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33cc003520 con 0x7f33e806ec50 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.195+0000 7f33e7400640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 0x7f33e80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1331734877 0 0) 0x7f33cc003520 con 0x7f33e806ec50 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 msgr2=0x7f33e80accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 0x7f33e80accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 >> v2:172.21.15.172:3300/0 conn(0x7f33e815b710 msgr2=0x7f33e80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 --2- 172.21.15.165:0/1467791843 >> v2:172.21.15.172:3300/0 conn(0x7f33e815b710 0x7f33e80cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80d0060 con 0x7f33e806ec50 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f33d0001670 con 0x7f33e806ec50 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e7400640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 0x7f33e80accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 >> v1:172.21.15.165:6789/0 conn(0x7f33e806ec50 legacy=0x7f33e815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33cc0039b0 0x7f33cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc004800 0x7f33cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 0x7f33cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e815e490 con 0x7f33cc0039b0 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e80cfd80 con 0x7f33cc004800 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33cc00b9c0 con 0x7f33e80ac8b0 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.196+0000 7f33e6a00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33cc0039b0 0x7f33cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33dfe00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc004800 0x7f33cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33e6a00640 1 -- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 msgr2=0x7f33cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33e6a00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80ac8b0 0x7f33cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33e6a00640 1 -- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc004800 msgr2=0x7f33cc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33e6a00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc004800 0x7f33cc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33e6a00640 1 -- 172.21.15.165:0/1467791843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33cc003520 con 0x7f33cc0039b0 2024-04-25T10:03:01.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.197+0000 7f33dfe00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc004800 0x7f33cc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.198+0000 7f33e6a00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33cc0039b0 0x7f33cc003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009af0 tx=0x7f33cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b47b2c16b32d49f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.198+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8036070 con 0x7f33cc0039b0 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.198+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d8031400 con 0x7f33cc0039b0 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.198+0000 7f33e6000640 1 -- 172.21.15.165:0/1467791843 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8009690 con 0x7f33cc0039b0 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.198+0000 7f33eda00640 1 -- 172.21.15.165:0/1467791843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33cc0039b0 msgr2=0x7f33cc003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.198+0000 7f33eda00640 1 --2- 172.21.15.165:0/1467791843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33cc0039b0 0x7f33cc003dd0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009af0 tx=0x7f33cc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.199+0000 7f33e7400640 1 -- 172.21.15.165:0/1467791843 reap_dead start 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.199+0000 7f33eda00640 1 -- 172.21.15.165:0/1467791843 shutdown_connections 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.199+0000 7f33eda00640 1 -- 172.21.15.165:0/1467791843 >> 172.21.15.165:0/1467791843 conn(0x7f33e806abe0 msgr2=0x7f33e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.199+0000 7f33eda00640 1 -- 172.21.15.165:0/1467791843 shutdown_connections 2024-04-25T10:03:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.199+0000 7f33eda00640 1 -- 172.21.15.165:0/1467791843 wait complete. 2024-04-25T10:03:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.200+0000 7f33eda00640 1 Processor -- start 2024-04-25T10:03:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.201+0000 7f33eda00640 1 -- start start 2024-04-25T10:03:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.201+0000 7f33eda00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc0039b0 0x7f33e814e170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33eda00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 0x7f33e81607f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33eda00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80b3970 0x7f33e80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33eda00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e815bc70 con 0x7f33e8160370 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33eda00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e815baf0 con 0x7f33cc0039b0 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33eda00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e815f030 con 0x7f33e80b3970 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 0x7f33e81607f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 0x7f33e81607f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39342/0 (socket says 172.21.15.165:39342) 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 -- 172.21.15.165:0/3628498552 learned_addr learned my addr 172.21.15.165:0/3628498552 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33e6a00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc0039b0 0x7f33e814e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 -- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80b3970 msgr2=0x7f33e80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80b3970 0x7f33e80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 -- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc0039b0 msgr2=0x7f33e814e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc0039b0 0x7f33e814e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.202+0000 7f33dfe00640 1 -- 172.21.15.165:0/3628498552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e80af600 con 0x7f33e8160370 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.203+0000 7f33dfe00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 0x7f33e81607f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f33d000dae0 tx=0x7f33d0007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.203+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0019070 con 0x7f33e8160370 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.203+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d0004c50 con 0x7f33e8160370 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.203+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d00070d0 con 0x7f33e8160370 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.203+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e80af8e0 con 0x7f33e8160370 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.203+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e80afea0 con 0x7f33e8160370 2024-04-25T10:03:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.204+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8073a00 con 0x7f33e8160370 2024-04-25T10:03:01.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.205+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33d001c070 con 0x7f33e8160370 2024-04-25T10:03:01.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.206+0000 7f33dea00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33c406b4d0 0x7f33c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.206+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f33d008a160 con 0x7f33e8160370 2024-04-25T10:03:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.206+0000 7f33e6a00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33c406b4d0 0x7f33c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.207+0000 7f33e6a00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33c406b4d0 0x7f33c406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f33d8033000 tx=0x7f33d8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.210+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33d0058c70 con 0x7f33e8160370 2024-04-25T10:03:01.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.406+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33e815beb0 con 0x7f33e8160370 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.407+0000 7f33dea00640 1 -- 172.21.15.165:0/3628498552 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33d0058570 con 0x7f33e8160370 2024-04-25T10:03:01.412 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33c406b4d0 msgr2=0x7f33c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33c406b4d0 0x7f33c406d990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f33d8033000 tx=0x7f33d8008030 comp rx=0 tx=0).stop 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 msgr2=0x7f33e81607f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 0x7f33e81607f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f33d000dae0 tx=0x7f33d0007e90 comp rx=0 tx=0).stop 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 shutdown_connections 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33c406b4d0 0x7f33c406d990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33e80b3970 0x7f33e80b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33e8160370 0x7f33e81607f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 --2- 172.21.15.165:0/3628498552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33cc0039b0 0x7f33e814e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 >> 172.21.15.165:0/3628498552 conn(0x7f33e806abe0 msgr2=0x7f33e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 shutdown_connections 2024-04-25T10:03:01.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.410+0000 7f33eda00640 1 -- 172.21.15.165:0/3628498552 wait complete. 2024-04-25T10:03:01.434 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:01.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.636+0000 7f48aaa00640 1 Processor -- start 2024-04-25T10:03:01.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.637+0000 7f48aaa00640 1 -- start start 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48aaa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f48a415b7d0 0x7f48a40cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48aaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f48a406c9e0 0x7f48a40b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48aaa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48a406cf50 con 0x7f48a40b5620 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48aaa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40ce530 con 0x7f48a406c9e0 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48aaa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40ce810 con 0x7f48a415b7d0 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48a8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f48a406c9e0 0x7f48a40b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48a3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f48a40b5620 0x7f48a406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48966/0 (socket says 172.21.15.165:48966) 2024-04-25T10:03:01.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.638+0000 7f48a3400640 1 -- 172.21.15.165:0/1536261477 learned_addr learned my addr 172.21.15.165:0/1536261477 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:01.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.639+0000 7f48a3e00640 1 --2- 172.21.15.165:0/1536261477 >> v2:172.21.15.172:3300/0 conn(0x7f48a415b7d0 0x7f48a40cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.639+0000 7f48a8a00640 1 -- 172.21.15.165:0/1536261477 >> v2:172.21.15.172:3300/0 conn(0x7f48a415b7d0 msgr2=0x7f48a40cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:01.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.639+0000 7f48a8a00640 1 --2- 172.21.15.165:0/1536261477 >> v2:172.21.15.172:3300/0 conn(0x7f48a415b7d0 0x7f48a40cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.639+0000 7f48a8a00640 1 -- 172.21.15.165:0/1536261477 >> v1:172.21.15.165:6789/0 conn(0x7f48a40b5620 legacy=0x7f48a406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.639+0000 7f48a8a00640 1 -- 172.21.15.165:0/1536261477 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40ceaf0 con 0x7f48a406c9e0 2024-04-25T10:03:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.640+0000 7f48a8a00640 1 --2- 172.21.15.165:0/1536261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a406c9e0 0x7f48a40b5ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48a408c0d0 tx=0x7f489c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0d4be339ad6abac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.640+0000 7f48a2a00640 1 -- 172.21.15.165:0/1536261477 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c037070 con 0x7f48a406c9e0 2024-04-25T10:03:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.640+0000 7f48a2a00640 1 -- 172.21.15.165:0/1536261477 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c032440 con 0x7f48a406c9e0 2024-04-25T10:03:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.640+0000 7f48a2a00640 1 -- 172.21.15.165:0/1536261477 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c032810 con 0x7f48a406c9e0 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 -- 172.21.15.165:0/1536261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a406c9e0 msgr2=0x7f48a40b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 --2- 172.21.15.165:0/1536261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a406c9e0 0x7f48a40b5ab0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48a408c0d0 tx=0x7f489c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 -- 172.21.15.165:0/1536261477 shutdown_connections 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 --2- 172.21.15.165:0/1536261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a406c9e0 0x7f48a40b5ab0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 --2- 172.21.15.165:0/1536261477 >> v2:172.21.15.172:3300/0 conn(0x7f48a415b7d0 0x7f48a40cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 -- 172.21.15.165:0/1536261477 >> 172.21.15.165:0/1536261477 conn(0x7f48a40d0800 msgr2=0x7f48a40d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 -- 172.21.15.165:0/1536261477 shutdown_connections 2024-04-25T10:03:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.641+0000 7f48aaa00640 1 -- 172.21.15.165:0/1536261477 wait complete. 2024-04-25T10:03:01.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.643+0000 7f48aaa00640 1 Processor -- start 2024-04-25T10:03:01.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.643+0000 7f48aaa00640 1 -- start start 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.644+0000 7f48aaa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a406c9e0 0x7f48a4159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.644+0000 7f48aaa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40b5620 0x7f48a40b8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.644+0000 7f48aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 0x7f48a40b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.644+0000 7f48aaa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40cf6d0 con 0x7f48a415b7d0 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.644+0000 7f48aaa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40cf550 con 0x7f48a406c9e0 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.644+0000 7f48aaa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a415bbb0 con 0x7f48a40b5620 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 0x7f48a40b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 0x7f48a40b8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39372/0 (socket says 172.21.15.165:39372) 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 -- 172.21.15.165:0/80716749 learned_addr learned my addr 172.21.15.165:0/80716749 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a3e00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a406c9e0 0x7f48a4159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a3400640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40b5620 0x7f48a40b8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 -- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40b5620 msgr2=0x7f48a40b8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40b5620 0x7f48a40b8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 -- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a406c9e0 msgr2=0x7f48a4159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a406c9e0 0x7f48a4159c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 -- 172.21.15.165:0/80716749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40c88c0 con 0x7f48a415b7d0 2024-04-25T10:03:01.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a3400640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40b5620 0x7f48a40b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:01.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.645+0000 7f48a8a00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 0x7f48a40b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f489c035040 tx=0x7f489c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.646+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c037040 con 0x7f48a415b7d0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.646+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489c042070 con 0x7f48a415b7d0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.646+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489c0308a0 con 0x7f48a415b7d0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.646+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a40c8ba0 con 0x7f48a415b7d0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.646+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a40cc3c0 con 0x7f48a415b7d0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.647+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a4073a00 con 0x7f48a415b7d0 2024-04-25T10:03:01.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.648+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f489c030a40 con 0x7f48a415b7d0 2024-04-25T10:03:01.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.649+0000 7f489be00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f488006b4d0 0x7f488006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.649+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f489c0aaea0 con 0x7f48a415b7d0 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.649+0000 7f48a3e00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f488006b4d0 0x7f488006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:01.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.650+0000 7f48a3e00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f488006b4d0 0x7f488006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4894006fd0 tx=0x7f4894008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:01.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.653+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f489c079a30 con 0x7f48a415b7d0 2024-04-25T10:03:01.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.849+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f48a415feb0 con 0x7f48a415b7d0 2024-04-25T10:03:01.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.850+0000 7f489be00640 1 -- 172.21.15.165:0/80716749 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f489c079330 con 0x7f48a415b7d0 2024-04-25T10:03:01.855 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f488006b4d0 msgr2=0x7f488006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f488006b4d0 0x7f488006d990 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4894006fd0 tx=0x7f4894008040 comp rx=0 tx=0).stop 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 msgr2=0x7f48a40b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 0x7f48a40b8aa0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f489c035040 tx=0x7f489c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 shutdown_connections 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f488006b4d0 0x7f488006d990 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48a415b7d0 0x7f48a40b8aa0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48a40b5620 0x7f48a40b8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 --2- 172.21.15.165:0/80716749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48a406c9e0 0x7f48a4159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 >> 172.21.15.165:0/80716749 conn(0x7f48a40d0800 msgr2=0x7f48a4160a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 shutdown_connections 2024-04-25T10:03:01.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:01.853+0000 7f48aaa00640 1 -- 172.21.15.165:0/80716749 wait complete. 2024-04-25T10:03:01.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:02.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.077+0000 7ffb93c00640 1 Processor -- start 2024-04-25T10:03:02.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.078+0000 7ffb93c00640 1 -- start start 2024-04-25T10:03:02.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.079+0000 7ffb93c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffb8c0cf640 0x7ffb8c06ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.079+0000 7ffb93c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0cfad0 con 0x7ffb8c15b710 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb93c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb8c0cfd80 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb93c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8c06f190 con 0x7ffb8c0cf640 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb8be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffb8c0cf640 0x7ffb8c06ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb8be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffb8c0cf640 0x7ffb8c06ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39374/0 (socket says 172.21.15.165:39374) 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb91200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ffb8c15b710 0x7ffb8c15e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48974/0 (socket says 172.21.15.165:48974) 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb8be00640 1 -- 172.21.15.165:0/402782026 learned_addr learned my addr 172.21.15.165:0/402782026 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb8be00640 1 -- 172.21.15.165:0/402782026 >> v1:172.21.15.172:6789/0 conn(0x7ffb8c0ac8b0 legacy=0x7ffb8c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb8be00640 1 -- 172.21.15.165:0/402782026 >> v1:172.21.15.165:6789/0 conn(0x7ffb8c15b710 legacy=0x7ffb8c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.080+0000 7ffb8be00640 1 -- 172.21.15.165:0/402782026 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0d0060 con 0x7ffb8c0cf640 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.081+0000 7ffb8be00640 1 --2- 172.21.15.165:0/402782026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c06ec50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009870 tx=0x7ffb7c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=363765f60531996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb8b400640 1 -- 172.21.15.165:0/402782026 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c037070 con 0x7ffb8c0cf640 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb8b400640 1 -- 172.21.15.165:0/402782026 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7c032440 con 0x7ffb8c0cf640 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb8b400640 1 -- 172.21.15.165:0/402782026 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c032810 con 0x7ffb8c0cf640 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 -- 172.21.15.165:0/402782026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0cf640 msgr2=0x7ffb8c06ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 --2- 172.21.15.165:0/402782026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c06ec50 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c009870 tx=0x7ffb7c02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 -- 172.21.15.165:0/402782026 shutdown_connections 2024-04-25T10:03:02.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 --2- 172.21.15.165:0/402782026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c06ec50 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 -- 172.21.15.165:0/402782026 >> 172.21.15.165:0/402782026 conn(0x7ffb8c06abe0 msgr2=0x7ffb8c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 -- 172.21.15.165:0/402782026 shutdown_connections 2024-04-25T10:03:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.082+0000 7ffb93c00640 1 -- 172.21.15.165:0/402782026 wait complete. 2024-04-25T10:03:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.084+0000 7ffb93c00640 1 Processor -- start 2024-04-25T10:03:02.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.084+0000 7ffb93c00640 1 -- start start 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb93c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 0x7ffb8c146f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb93c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c1f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb93c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb8c15b710 0x7ffb8c0b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb93c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c15baf0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb93c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c15e490 con 0x7ffb8c0cf640 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb93c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb8c15bc70 con 0x7ffb8c15b710 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb91200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 0x7ffb8c146f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.085+0000 7ffb91200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 0x7ffb8c146f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39386/0 (socket says 172.21.15.165:39386) 2024-04-25T10:03:02.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 -- 172.21.15.165:0/1615061554 learned_addr learned my addr 172.21.15.165:0/1615061554 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 -- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb8c15b710 msgr2=0x7ffb8c0b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb8be00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c1f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb8c15b710 0x7ffb8c0b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 -- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb8c0cf640 msgr2=0x7ffb8c1f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c1f2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 -- 172.21.15.165:0/1615061554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb8c0b1490 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb91200640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 0x7ffb8c146f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c3c0 tx=0x7ffb8000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb800074e0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb80007680 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.086+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb8c1f6dc0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.087+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb80004a00 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.087+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb8c1f7380 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.088+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb8c073a00 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.089+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb8001a020 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.089+0000 7ffb88c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb7006b4d0 0x7ffb7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.090+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ffb80089ce0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.090+0000 7ffb8be00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb7006b4d0 0x7ffb7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.090+0000 7ffb8be00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb7006b4d0 0x7ffb7006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c034000 tx=0x7ffb7c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.094+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb800587f0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.291+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb8c0af4b0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.292+0000 7ffb88c00640 1 -- 172.21.15.165:0/1615061554 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb800580f0 con 0x7ffb8c0ac8b0 2024-04-25T10:03:02.297 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:02.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb7006b4d0 msgr2=0x7ffb7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb7006b4d0 0x7ffb7006d990 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c034000 tx=0x7ffb7c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 msgr2=0x7ffb8c146f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 0x7ffb8c146f30 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7ffb8000c3c0 tx=0x7ffb8000c880 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 shutdown_connections 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb7006b4d0 0x7ffb7006d990 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb8c15b710 0x7ffb8c0b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb8c0cf640 0x7ffb8c1f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 --2- 172.21.15.165:0/1615061554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb8c0ac8b0 0x7ffb8c146f30 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 >> 172.21.15.165:0/1615061554 conn(0x7ffb8c06abe0 msgr2=0x7ffb8c06de50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 shutdown_connections 2024-04-25T10:03:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.295+0000 7ffb93c00640 1 -- 172.21.15.165:0/1615061554 wait complete. 2024-04-25T10:03:02.318 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.513+0000 7f30b3000640 1 Processor -- start 2024-04-25T10:03:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.513+0000 7f30b3000640 1 -- start start 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.515+0000 7f30b3000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.515+0000 7f30b3000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30ac144d50 0x7f30ac0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.515+0000 7f30b3000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30ac1452a0 0x7f30ac0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.515+0000 7f30b3000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac0b2b20 con 0x7f30ac144d50 2024-04-25T10:03:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.515+0000 7f30b3000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30ac06d400 con 0x7f30ac1452a0 2024-04-25T10:03:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.515+0000 7f30b3000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0b49a0 con 0x7f30ac1446d0 2024-04-25T10:03:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30ac144d50 0x7f30ac0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30ac144d50 0x7f30ac0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39398/0 (socket says 172.21.15.165:39398) 2024-04-25T10:03:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 -- 172.21.15.165:0/272919066 learned_addr learned my addr 172.21.15.165:0/272919066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30b1000640 1 --2- 172.21.15.165:0/272919066 >> v2:172.21.15.172:3300/0 conn(0x7f30ac1452a0 0x7f30ac0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 -- 172.21.15.165:0/272919066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 msgr2=0x7f30ac0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30abe00640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 -- 172.21.15.165:0/272919066 >> v2:172.21.15.172:3300/0 conn(0x7f30ac1452a0 msgr2=0x7f30ac0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 --2- 172.21.15.165:0/272919066 >> v2:172.21.15.172:3300/0 conn(0x7f30ac1452a0 0x7f30ac0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30ab400640 1 -- 172.21.15.165:0/272919066 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0cccc0 con 0x7f30ac144d50 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30b1000640 1 --2- 172.21.15.165:0/272919066 >> v2:172.21.15.172:3300/0 conn(0x7f30ac1452a0 0x7f30ac0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.516+0000 7f30abe00640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.517+0000 7f30ab400640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac144d50 0x7f30ac0b4460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30ac145880 tx=0x7f309802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307512c06a70a226 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.518+0000 7f30aaa00640 1 -- 172.21.15.165:0/272919066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098037070 con 0x7f30ac144d50 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.518+0000 7f30aaa00640 1 -- 172.21.15.165:0/272919066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3098032440 con 0x7f30ac144d50 2024-04-25T10:03:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.518+0000 7f30aaa00640 1 -- 172.21.15.165:0/272919066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3098032810 con 0x7f30ac144d50 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.518+0000 7f30b3000640 1 -- 172.21.15.165:0/272919066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac144d50 msgr2=0x7f30ac0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.518+0000 7f30b3000640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac144d50 0x7f30ac0b4460 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30ac145880 tx=0x7f309802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 -- 172.21.15.165:0/272919066 shutdown_connections 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 --2- 172.21.15.165:0/272919066 >> v2:172.21.15.172:3300/0 conn(0x7f30ac1452a0 0x7f30ac0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac144d50 0x7f30ac0b4460 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 --2- 172.21.15.165:0/272919066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 -- 172.21.15.165:0/272919066 >> 172.21.15.165:0/272919066 conn(0x7f30ac06abe0 msgr2=0x7f30ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 -- 172.21.15.165:0/272919066 shutdown_connections 2024-04-25T10:03:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.519+0000 7f30b3000640 1 -- 172.21.15.165:0/272919066 wait complete. 2024-04-25T10:03:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.520+0000 7f30b3000640 1 Processor -- start 2024-04-25T10:03:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.521+0000 7f30b3000640 1 -- start start 2024-04-25T10:03:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.521+0000 7f30b3000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac13e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b3000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ac144d50 0x7f30ac0ab9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 0x7f30ac151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0cc700 con 0x7f30ac1452a0 2024-04-25T10:03:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b3000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0cc420 con 0x7f30ac144d50 2024-04-25T10:03:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b3000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30ac0cc9e0 con 0x7f30ac1446d0 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 0x7f30ac151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 0x7f30ac151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39414/0 (socket says 172.21.15.165:39414) 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30b1000640 1 -- 172.21.15.165:0/2619120515 learned_addr learned my addr 172.21.15.165:0/2619120515 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30ab400640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ac144d50 0x7f30ac0ab9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.522+0000 7f30abe00640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac13e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b1000640 1 -- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 msgr2=0x7f30ac13e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b1000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac13e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b1000640 1 -- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ac144d50 msgr2=0x7f30ac0ab9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b1000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ac144d50 0x7f30ac0ab9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b1000640 1 -- 172.21.15.165:0/2619120515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ac0ae3a0 con 0x7f30ac1452a0 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30abe00640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac13e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b1000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 0x7f30ac151640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c7c0 tx=0x7f30a400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4015020 con 0x7f30ac1452a0 2024-04-25T10:03:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a4004230 con 0x7f30ac1452a0 2024-04-25T10:03:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a4007620 con 0x7f30ac1452a0 2024-04-25T10:03:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30ac0ae620 con 0x7f30ac1452a0 2024-04-25T10:03:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.523+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30ac14f740 con 0x7f30ac1452a0 2024-04-25T10:03:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.524+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30ac073a00 con 0x7f30ac1452a0 2024-04-25T10:03:02.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.525+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30a4018020 con 0x7f30ac1452a0 2024-04-25T10:03:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.526+0000 7f30a3e00640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f308c06b4d0 0x7f308c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.526+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f30a408a700 con 0x7f30ac1452a0 2024-04-25T10:03:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.526+0000 7f30abe00640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f308c06b4d0 0x7f308c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.527+0000 7f30abe00640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f308c06b4d0 0x7f308c06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f309c008fc0 tx=0x7f309c008510 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.530+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30a401f020 con 0x7f30ac1452a0 2024-04-25T10:03:02.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.725+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f30ac06e6d0 con 0x7f30ac1452a0 2024-04-25T10:03:02.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.726+0000 7f30a3e00640 1 -- 172.21.15.165:0/2619120515 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f30ac0ae3a0 con 0x7f30ac1452a0 2024-04-25T10:03:02.731 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:02.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f308c06b4d0 msgr2=0x7f308c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f308c06b4d0 0x7f308c06d990 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f309c008fc0 tx=0x7f309c008510 comp rx=0 tx=0).stop 2024-04-25T10:03:02.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 msgr2=0x7f30ac151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 0x7f30ac151640 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f30a400c7c0 tx=0x7f30a400cc80 comp rx=0 tx=0).stop 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 shutdown_connections 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f308c06b4d0 0x7f308c06d990 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30ac1452a0 0x7f30ac151640 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30ac144d50 0x7f30ac0ab9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 --2- 172.21.15.165:0/2619120515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30ac1446d0 0x7f30ac13e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 >> 172.21.15.165:0/2619120515 conn(0x7f30ac06abe0 msgr2=0x7f30ac06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 shutdown_connections 2024-04-25T10:03:02.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.729+0000 7f30b3000640 1 -- 172.21.15.165:0/2619120515 wait complete. 2024-04-25T10:03:02.752 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:03:02.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.947+0000 7f3087e00640 1 Processor -- start 2024-04-25T10:03:02.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.947+0000 7f3087e00640 1 -- start start 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3087e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f308806e870 0x7f30880cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3087e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30880cc840 0x7f308815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3087e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3088144240 con 0x7f3088143e60 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3087e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f308815e220 con 0x7f308806e870 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3087e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f308815e500 con 0x7f30880cc840 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3086a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3088143e60 0x7f308806e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48978/0 (socket says 172.21.15.165:48978) 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f3086a00640 1 -- 172.21.15.165:0/2307075025 learned_addr learned my addr 172.21.15.165:0/2307075025 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:02.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.949+0000 7f307fe00640 1 --2- 172.21.15.165:0/2307075025 >> v2:172.21.15.165:3300/0 conn(0x7f308806e870 0x7f30880cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f3087400640 1 --2- 172.21.15.165:0/2307075025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30880cc840 0x7f308815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f3086000640 1 -- 172.21.15.165:0/2307075025 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 190707120 0 0) 0x7f3088144240 con 0x7f3088143e60 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f3086000640 1 -- 172.21.15.165:0/2307075025 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f306c005730 con 0x7f3088143e60 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f307fe00640 1 -- 172.21.15.165:0/2307075025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30880cc840 msgr2=0x7f308815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f307fe00640 1 --2- 172.21.15.165:0/2307075025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30880cc840 0x7f308815ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f307fe00640 1 -- 172.21.15.165:0/2307075025 >> v1:172.21.15.165:6789/0 conn(0x7f3088143e60 legacy=0x7f308806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.950+0000 7f307fe00640 1 -- 172.21.15.165:0/2307075025 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f308815e7e0 con 0x7f308806e870 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.951+0000 7f307fe00640 1 --2- 172.21.15.165:0/2307075025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f308806e870 0x7f30880cc300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f30700097a0 tx=0x7f307002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef22d705f41e012a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.951+0000 7f3086000640 1 -- 172.21.15.165:0/2307075025 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070037070 con 0x7f308806e870 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.951+0000 7f3086000640 1 -- 172.21.15.165:0/2307075025 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3070032440 con 0x7f308806e870 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.951+0000 7f3086000640 1 -- 172.21.15.165:0/2307075025 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070032810 con 0x7f308806e870 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 -- 172.21.15.165:0/2307075025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f308806e870 msgr2=0x7f30880cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 --2- 172.21.15.165:0/2307075025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f308806e870 0x7f30880cc300 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f30700097a0 tx=0x7f307002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 -- 172.21.15.165:0/2307075025 shutdown_connections 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 --2- 172.21.15.165:0/2307075025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30880cc840 0x7f308815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 --2- 172.21.15.165:0/2307075025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f308806e870 0x7f30880cc300 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 -- 172.21.15.165:0/2307075025 >> 172.21.15.165:0/2307075025 conn(0x7f308806abe0 msgr2=0x7f308806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 -- 172.21.15.165:0/2307075025 shutdown_connections 2024-04-25T10:03:02.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.952+0000 7f3087e00640 1 -- 172.21.15.165:0/2307075025 wait complete. 2024-04-25T10:03:02.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.954+0000 7f3087e00640 1 Processor -- start 2024-04-25T10:03:02.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.954+0000 7f3087e00640 1 -- start start 2024-04-25T10:03:02.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f308806e870 0x7f3088159780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30880cc840 0x7f308815d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 0x7f308815d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30880b2d30 con 0x7f3088143e60 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30880cccf0 con 0x7f30880cc840 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30880b2eb0 con 0x7f308806e870 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 0x7f308815d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 0x7f308815d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39422/0 (socket says 172.21.15.165:39422) 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f3087400640 1 -- 172.21.15.165:0/3234812101 learned_addr learned my addr 172.21.15.165:0/3234812101 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.955+0000 7f307fe00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30880cc840 0x7f308815d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3086a00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f308806e870 0x7f3088159780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087400640 1 -- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f308806e870 msgr2=0x7f3088159780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087400640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f308806e870 0x7f3088159780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087400640 1 -- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30880cc840 msgr2=0x7f308815d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087400640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30880cc840 0x7f308815d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087400640 1 -- 172.21.15.165:0/3234812101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30881f0b70 con 0x7f3088143e60 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3086a00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f308806e870 0x7f3088159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:02.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087400640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 0x7f308815d5e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3080007e70 tx=0x7f308000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080014620 con 0x7f3088143e60 2024-04-25T10:03:02.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3080019070 con 0x7f3088143e60 2024-04-25T10:03:02.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3080014a20 con 0x7f3088143e60 2024-04-25T10:03:02.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.956+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30880aa300 con 0x7f3088143e60 2024-04-25T10:03:02.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.957+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30881f10f0 con 0x7f3088143e60 2024-04-25T10:03:02.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.958+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3088073990 con 0x7f3088143e60 2024-04-25T10:03:02.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.959+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3080014bc0 con 0x7f3088143e60 2024-04-25T10:03:02.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.959+0000 7f307f400640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f306406b4d0 0x7f306406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:02.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.960+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f308008afa0 con 0x7f3088143e60 2024-04-25T10:03:02.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.960+0000 7f3086a00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f306406b4d0 0x7f306406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:02.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.960+0000 7f3086a00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f306406b4d0 0x7f306406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f30880ce750 tx=0x7f3078009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:02.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:02.964+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3080059980 con 0x7f3088143e60 2024-04-25T10:03:03.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.232+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f308815ca80 con 0x7f3088143e60 2024-04-25T10:03:03.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.315+0000 7f307f400640 1 -- 172.21.15.165:0/3234812101 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v81) v1 ==== 151+0+0 (secure 0 0 0) 0x7f3080059280 con 0x7f3088143e60 2024-04-25T10:03:03.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.318+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f306406b4d0 msgr2=0x7f306406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.318+0000 7f3087e00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f306406b4d0 0x7f306406d990 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f30880ce750 tx=0x7f3078009040 comp rx=0 tx=0).stop 2024-04-25T10:03:03.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.318+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 msgr2=0x7f308815d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.318+0000 7f3087e00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 0x7f308815d5e0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3080007e70 tx=0x7f308000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 shutdown_connections 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f306406b4d0 0x7f306406d990 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3088143e60 0x7f308815d5e0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30880cc840 0x7f308815d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 --2- 172.21.15.165:0/3234812101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f308806e870 0x7f3088159780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 >> 172.21.15.165:0/3234812101 conn(0x7f308806abe0 msgr2=0x7f308815bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 shutdown_connections 2024-04-25T10:03:03.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.319+0000 7f3087e00640 1 -- 172.21.15.165:0/3234812101 wait complete. 2024-04-25T10:03:03.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:03:03.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.546+0000 7f971ba00640 1 Processor -- start 2024-04-25T10:03:03.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.547+0000 7f971ba00640 1 -- start start 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.548+0000 7f971ba00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97141616f0 0x7f9714161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.548+0000 7f971ba00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f971406c8d0 con 0x7f97140ce0a0 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.548+0000 7f971ba00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97140c6960 con 0x7f97141610f0 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.548+0000 7f971ba00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97141614d0 con 0x7f97141616f0 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.548+0000 7f9713e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97141616f0 0x7f9714161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.548+0000 7f9719a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f97140ce0a0 0x7f97140d0200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48992/0 (socket says 172.21.15.165:48992) 2024-04-25T10:03:03.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.549+0000 7f9719a00640 1 -- 172.21.15.165:0/954385713 learned_addr learned my addr 172.21.15.165:0/954385713 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:03.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.549+0000 7f9713e00640 1 -- 172.21.15.165:0/954385713 >> v1:172.21.15.172:6789/0 conn(0x7f97141610f0 legacy=0x7f971406c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:03.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.549+0000 7f9713e00640 1 -- 172.21.15.165:0/954385713 >> v1:172.21.15.165:6789/0 conn(0x7f97140ce0a0 legacy=0x7f97140d0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.549+0000 7f9713e00640 1 -- 172.21.15.165:0/954385713 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714162260 con 0x7f97141616f0 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.550+0000 7f9713e00640 1 --2- 172.21.15.165:0/954385713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141616f0 0x7f9714161b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0097a0 tx=0x7f96fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa83dbebe3e0c1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.550+0000 7f9713400640 1 -- 172.21.15.165:0/954385713 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc037070 con 0x7f97141616f0 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.550+0000 7f9713400640 1 -- 172.21.15.165:0/954385713 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc032440 con 0x7f97141616f0 2024-04-25T10:03:03.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.550+0000 7f9713400640 1 -- 172.21.15.165:0/954385713 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc032810 con 0x7f97141616f0 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 -- 172.21.15.165:0/954385713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141616f0 msgr2=0x7f9714161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 --2- 172.21.15.165:0/954385713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141616f0 0x7f9714161b50 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f96fc0097a0 tx=0x7f96fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 -- 172.21.15.165:0/954385713 shutdown_connections 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 --2- 172.21.15.165:0/954385713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141616f0 0x7f9714161b50 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 -- 172.21.15.165:0/954385713 >> 172.21.15.165:0/954385713 conn(0x7f971406f500 msgr2=0x7f971406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 -- 172.21.15.165:0/954385713 shutdown_connections 2024-04-25T10:03:03.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.551+0000 7f971ba00640 1 -- 172.21.15.165:0/954385713 wait complete. 2024-04-25T10:03:03.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.553+0000 7f971ba00640 1 Processor -- start 2024-04-25T10:03:03.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.553+0000 7f971ba00640 1 -- start start 2024-04-25T10:03:03.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f971ba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97140ce0a0 0x7f97141573f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f971ba00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 0x7f97141f3a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f971ba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97141616f0 0x7f9714159c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f971ba00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140cfce0 con 0x7f97141610f0 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f971ba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97140cfa00 con 0x7f97140ce0a0 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f971ba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97140cf690 con 0x7f97141616f0 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f9713e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 0x7f97141f3a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f9713e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 0x7f97141f3a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39454/0 (socket says 172.21.15.165:39454) 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f9713e00640 1 -- 172.21.15.165:0/3990116899 learned_addr learned my addr 172.21.15.165:0/3990116899 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.554+0000 7f9719000640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97140ce0a0 0x7f97141573f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9713e00640 1 -- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97141616f0 msgr2=0x7f9714159c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9713e00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97141616f0 0x7f9714159c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9713e00640 1 -- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97140ce0a0 msgr2=0x7f97141573f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9713e00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97140ce0a0 0x7f97141573f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9713e00640 1 -- 172.21.15.165:0/3990116899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9714157930 con 0x7f97141610f0 2024-04-25T10:03:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9713e00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 0x7f97141f3a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f96fc000c00 tx=0x7f96fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc037040 con 0x7f97141610f0 2024-04-25T10:03:03.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc042070 con 0x7f97141610f0 2024-04-25T10:03:03.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc030a70 con 0x7f97141610f0 2024-04-25T10:03:03.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.555+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9714159440 con 0x7f97141610f0 2024-04-25T10:03:03.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.556+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97140b48a0 con 0x7f97141610f0 2024-04-25T10:03:03.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.557+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9714073a00 con 0x7f97141610f0 2024-04-25T10:03:03.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.557+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f96fc030c10 con 0x7f97141610f0 2024-04-25T10:03:03.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.558+0000 7f9710c00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f96f006b4d0 0x7f96f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:03.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.559+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f96fc0ab120 con 0x7f97141610f0 2024-04-25T10:03:03.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.559+0000 7f9719000640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f96f006b4d0 0x7f96f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:03.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.559+0000 7f9719000640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f96f006b4d0 0x7f96f006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9704004ea0 tx=0x7f9704009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:03.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.563+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f96fc045360 con 0x7f97141610f0 2024-04-25T10:03:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.760+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f971415d9c0 con 0x7f97141610f0 2024-04-25T10:03:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.767+0000 7f9710c00640 1 -- 172.21.15.165:0/3990116899 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f96fc079b90 con 0x7f97141610f0 2024-04-25T10:03:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.770+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f96f006b4d0 msgr2=0x7f96f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.770+0000 7f971ba00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f96f006b4d0 0x7f96f006d990 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9704004ea0 tx=0x7f9704009040 comp rx=0 tx=0).stop 2024-04-25T10:03:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.770+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 msgr2=0x7f97141f3a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.770+0000 7f971ba00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 0x7f97141f3a40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f96fc000c00 tx=0x7f96fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 shutdown_connections 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f96f006b4d0 0x7f96f006d990 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97141616f0 0x7f9714159c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97141610f0 0x7f97141f3a40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 --2- 172.21.15.165:0/3990116899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97140ce0a0 0x7f97141573f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 >> 172.21.15.165:0/3990116899 conn(0x7f971406f500 msgr2=0x7f97140b1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 shutdown_connections 2024-04-25T10:03:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:03.771+0000 7f971ba00640 1 -- 172.21.15.165:0/3990116899 wait complete. 2024-04-25T10:03:03.795 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:03:03.795 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:03.903 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:03:04.025 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:03:04.026 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:04.026 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:04.083 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:04.084 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:04.136 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:04.136 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:04.189 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:04.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.382+0000 7f0aa8a00640 1 Processor -- start 2024-04-25T10:03:04.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.382+0000 7f0aa8a00640 1 -- start start 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415e520 0x7f0aa415e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0aa4070ff0 0x7f0aa4071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa8a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa4071950 con 0x7f0aa415f040 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa8a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40aee30 con 0x7f0aa4070ff0 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40ae590 con 0x7f0aa415e520 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0aa4070ff0 0x7f0aa4071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0aa4070ff0 0x7f0aa4071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39464/0 (socket says 172.21.15.165:39464) 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 -- 172.21.15.165:0/1038902089 learned_addr learned my addr 172.21.15.165:0/1038902089 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa2800640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415e520 0x7f0aa415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 -- 172.21.15.165:0/1038902089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415e520 msgr2=0x7f0aa415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415e520 0x7f0aa415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 -- 172.21.15.165:0/1038902089 >> v1:172.21.15.172:6789/0 conn(0x7f0aa415f040 legacy=0x7f0aa40adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa3200640 1 -- 172.21.15.165:0/1038902089 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40ae870 con 0x7f0aa4070ff0 2024-04-25T10:03:04.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.384+0000 7f0aa2800640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415e520 0x7f0aa415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.385+0000 7f0aa3200640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa4071410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009e50 tx=0x7f0a9c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6b7c01618a0022 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa1e00640 1 -- 172.21.15.165:0/1038902089 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c036070 con 0x7f0aa4070ff0 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa1e00640 1 -- 172.21.15.165:0/1038902089 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a9c031600 con 0x7f0aa4070ff0 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa1e00640 1 -- 172.21.15.165:0/1038902089 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9c0319d0 con 0x7f0aa4070ff0 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa8a00640 1 -- 172.21.15.165:0/1038902089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 msgr2=0x7f0aa4071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa4071410 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c009e50 tx=0x7f0a9c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa8a00640 1 -- 172.21.15.165:0/1038902089 shutdown_connections 2024-04-25T10:03:04.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa4071410 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/1038902089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415e520 0x7f0aa415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.386+0000 7f0aa8a00640 1 -- 172.21.15.165:0/1038902089 >> 172.21.15.165:0/1038902089 conn(0x7f0aa406abe0 msgr2=0x7f0aa406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.387+0000 7f0aa8a00640 1 -- 172.21.15.165:0/1038902089 shutdown_connections 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.387+0000 7f0aa8a00640 1 -- 172.21.15.165:0/1038902089 wait complete. 2024-04-25T10:03:04.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.388+0000 7f0aa8a00640 1 Processor -- start 2024-04-25T10:03:04.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.388+0000 7f0aa8a00640 1 -- start start 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa40b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa415e520 0x7f0aa415a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415f040 0x7f0aa415ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa40ae2a0 con 0x7f0aa4070ff0 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa406d1f0 con 0x7f0aa415e520 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa415f4a0 con 0x7f0aa415f040 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.389+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa40b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa40b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39480/0 (socket says 172.21.15.165:39480) 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 -- 172.21.15.165:0/2907388461 learned_addr learned my addr 172.21.15.165:0/2907388461 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 -- 172.21.15.165:0/2907388461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415f040 msgr2=0x7f0aa415ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415f040 0x7f0aa415ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 -- 172.21.15.165:0/2907388461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa415e520 msgr2=0x7f0aa415a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa415e520 0x7f0aa415a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 -- 172.21.15.165:0/2907388461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40b3f50 con 0x7f0aa4070ff0 2024-04-25T10:03:04.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.390+0000 7f0aa2800640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa40b1980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c7f0 tx=0x7f0a9400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.391+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94015070 con 0x7f0aa4070ff0 2024-04-25T10:03:04.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.391+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a94007a30 con 0x7f0aa4070ff0 2024-04-25T10:03:04.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.391+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a94007e00 con 0x7f0aa4070ff0 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.391+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa40b4230 con 0x7f0aa4070ff0 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.391+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa4130a50 con 0x7f0aa4070ff0 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.392+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa4073990 con 0x7f0aa4070ff0 2024-04-25T10:03:04.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.393+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a94018070 con 0x7f0aa4070ff0 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.394+0000 7f0a9b400640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8006b4d0 0x7f0a8006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.394+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0a9408a650 con 0x7f0aa4070ff0 2024-04-25T10:03:04.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.394+0000 7f0a99e00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8006b4d0 0x7f0a8006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.395+0000 7f0a99e00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8006b4d0 0x7f0a8006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006fd0 tx=0x7f0a8c006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.399+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a94059030 con 0x7f0aa4070ff0 2024-04-25T10:03:04.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.595+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0aa415dc60 con 0x7f0aa4070ff0 2024-04-25T10:03:04.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.597+0000 7f0a9b400640 1 -- 172.21.15.165:0/2907388461 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f0a94058930 con 0x7f0aa4070ff0 2024-04-25T10:03:04.602 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:04.602 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8006b4d0 msgr2=0x7f0a8006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8006b4d0 0x7f0a8006d990 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f0a8c006fd0 tx=0x7f0a8c006bc0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 msgr2=0x7f0aa40b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa40b1980 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400c7f0 tx=0x7f0a9400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 shutdown_connections 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8006b4d0 0x7f0a8006d990 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa415f040 0x7f0aa415ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa415e520 0x7f0aa415a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 --2- 172.21.15.165:0/2907388461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa4070ff0 0x7f0aa40b1980 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 >> 172.21.15.165:0/2907388461 conn(0x7f0aa406abe0 msgr2=0x7f0aa40afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 shutdown_connections 2024-04-25T10:03:04.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.600+0000 7f0aa8a00640 1 -- 172.21.15.165:0/2907388461 wait complete. 2024-04-25T10:03:04.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:04.624 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:04.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.823+0000 7f9dbfe00640 1 Processor -- start 2024-04-25T10:03:04.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.824+0000 7f9dbfe00640 1 -- start start 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.825+0000 7f9dbfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9dc00b1140 0x7f9dc00b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.825+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 0x7f9dc00d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.825+0000 7f9dbfe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc00b5490 con 0x7f9dc00b3bd0 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.825+0000 7f9dbfe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc015ea60 con 0x7f9dc00b1140 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.825+0000 7f9dbfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b5730 con 0x7f9dc00b1fb0 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.826+0000 7f9dbf400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9dc00b3bd0 0x7f9dc0146830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49002/0 (socket says 172.21.15.165:49002) 2024-04-25T10:03:04.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.826+0000 7f9dbf400640 1 -- 172.21.15.165:0/1352034161 learned_addr learned my addr 172.21.15.165:0/1352034161 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:04.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.826+0000 7f9db6000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 0x7f9dc00d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.826+0000 7f9dbea00640 1 --2- 172.21.15.165:0/1352034161 >> v2:172.21.15.172:3300/0 conn(0x7f9dc00b1140 0x7f9dc00b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.826+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4084796228 0 0) 0x7f9dc00b5490 con 0x7f9dc00b3bd0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.826+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9da4003520 con 0x7f9dc00b3bd0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2773609750 0 0) 0x7f9da4003520 con 0x7f9dc00b3bd0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 msgr2=0x7f9dc00d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 0x7f9dc00d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 >> v2:172.21.15.172:3300/0 conn(0x7f9dc00b1140 msgr2=0x7f9dc00b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 --2- 172.21.15.165:0/1352034161 >> v2:172.21.15.172:3300/0 conn(0x7f9dc00b1140 0x7f9dc00b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00b5a10 con 0x7f9dc00b3bd0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9db80019f0 con 0x7f9dc00b3bd0 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 >> v1:172.21.15.165:6789/0 conn(0x7f9dc00b3bd0 legacy=0x7f9dc0146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 0x7f9da40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc015ea60 con 0x7f9da4004800 2024-04-25T10:03:04.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc00b5730 con 0x7f9da40039b0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da400ba40 con 0x7f9dc00b1fb0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9db6000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9db6000640 1 -- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 msgr2=0x7f9da40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9db6000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc00b1fb0 0x7f9da40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9dbea00640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.827+0000 7f9db6000640 1 -- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9da40039b0 msgr2=0x7f9da4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.828+0000 7f9db6000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9da40039b0 0x7f9da4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.828+0000 7f9db6000640 1 -- 172.21.15.165:0/1352034161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da4003520 con 0x7f9da4004800 2024-04-25T10:03:04.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.829+0000 7f9db6000640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009ac0 tx=0x7f9da4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82c26215449fae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.829+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8036070 con 0x7f9da4004800 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.829+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da8031400 con 0x7f9da4004800 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.829+0000 7f9dbe000640 1 -- 172.21.15.165:0/1352034161 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da8009690 con 0x7f9da4004800 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbfe00640 1 -- 172.21.15.165:0/1352034161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 msgr2=0x7f9da4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/1352034161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9da4004c80 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9da8009ac0 tx=0x7f9da4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbf400640 1 -- 172.21.15.165:0/1352034161 reap_dead start 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbfe00640 1 -- 172.21.15.165:0/1352034161 shutdown_connections 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbfe00640 1 -- 172.21.15.165:0/1352034161 >> 172.21.15.165:0/1352034161 conn(0x7f9dc00728b0 msgr2=0x7f9dc0099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbfe00640 1 -- 172.21.15.165:0/1352034161 shutdown_connections 2024-04-25T10:03:04.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.830+0000 7f9dbfe00640 1 -- 172.21.15.165:0/1352034161 wait complete. 2024-04-25T10:03:04.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.832+0000 7f9dbfe00640 1 Processor -- start 2024-04-25T10:03:04.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.832+0000 7f9dbfe00640 1 -- start start 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.833+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9dc009e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc015d410 0x7f9dc00b0430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc01f3700 0x7f9dc015fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc01470c0 con 0x7f9da4004800 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0146f40 con 0x7f9dc015d410 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc0147240 con 0x7f9dc01f3700 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9dc009e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9dc009e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39490/0 (socket says 172.21.15.165:39490) 2024-04-25T10:03:04.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 -- 172.21.15.165:0/2855907978 learned_addr learned my addr 172.21.15.165:0/2855907978 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:04.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbf400640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc01f3700 0x7f9dc015fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9db6000640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc015d410 0x7f9dc00b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 -- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc01f3700 msgr2=0x7f9dc015fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc01f3700 0x7f9dc015fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 -- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc015d410 msgr2=0x7f9dc00b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc015d410 0x7f9dc00b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbea00640 1 -- 172.21.15.165:0/2855907978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc00b0970 con 0x7f9da4004800 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9dbf400640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc01f3700 0x7f9dc015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.834+0000 7f9db6000640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc015d410 0x7f9dc00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.835+0000 7f9dbea00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9dc009e020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9db0010940 tx=0x7f9db0010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.835+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000f040 con 0x7f9da4004800 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.835+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db000ebc0 con 0x7f9da4004800 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.835+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc00b0090 con 0x7f9da4004800 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.835+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db001c400 con 0x7f9da4004800 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.835+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc00afad0 con 0x7f9da4004800 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.836+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc0073990 con 0x7f9da4004800 2024-04-25T10:03:04.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.837+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9db001c5a0 con 0x7f9da4004800 2024-04-25T10:03:04.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.839+0000 7f9db7e00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:04.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.839+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f9db008ace0 con 0x7f9da4004800 2024-04-25T10:03:04.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.839+0000 7f9db6000640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:04.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.840+0000 7f9db6000640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9da8033000 tx=0x7f9da8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:04.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:04.844+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9db0018030 con 0x7f9da4004800 2024-04-25T10:03:05.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.042+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9dc015e750 con 0x7f9da4004800 2024-04-25T10:03:05.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.043+0000 7f9db7e00640 1 -- 172.21.15.165:0/2855907978 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9db0059760 con 0x7f9da4004800 2024-04-25T10:03:05.048 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d9c06b4d0 msgr2=0x7f9d9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f9da8033000 tx=0x7f9da8008030 comp rx=0 tx=0).stop 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 msgr2=0x7f9dc009e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9dc009e020 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9db0010940 tx=0x7f9db0010e00 comp rx=0 tx=0).stop 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 shutdown_connections 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d9c06b4d0 0x7f9d9c06d990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9dc01f3700 0x7f9dc015fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9dc015d410 0x7f9dc00b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 --2- 172.21.15.165:0/2855907978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9da4004800 0x7f9dc009e020 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 >> 172.21.15.165:0/2855907978 conn(0x7f9dc00728b0 msgr2=0x7f9dc006e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 shutdown_connections 2024-04-25T10:03:05.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.046+0000 7f9dbfe00640 1 -- 172.21.15.165:0/2855907978 wait complete. 2024-04-25T10:03:05.071 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:05.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.273+0000 7f1204400640 1 Processor -- start 2024-04-25T10:03:05.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.273+0000 7f1204400640 1 -- start start 2024-04-25T10:03:05.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1204400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 0x7f11fc0b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1204400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11fc06c7e0 0x7f11fc0b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1204400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11fc0d1d50 con 0x7f11fc158230 2024-04-25T10:03:05.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1204400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11fc158610 con 0x7f11fc06c7e0 2024-04-25T10:03:05.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1204400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc06bea0 con 0x7f11fc06c330 2024-04-25T10:03:05.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1201a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f11fc158230 0x7f11fc0d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49006/0 (socket says 172.21.15.165:49006) 2024-04-25T10:03:05.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1201a00640 1 -- 172.21.15.165:0/467669032 learned_addr learned my addr 172.21.15.165:0/467669032 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.275+0000 7f1202400640 1 --2- 172.21.15.165:0/467669032 >> v2:172.21.15.172:3300/0 conn(0x7f11fc06c7e0 0x7f11fc0b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f1201000640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 0x7f11fc0b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128076996 0 0) 0x7f11fc0d1d50 con 0x7f11fc158230 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11e4003520 con 0x7f11fc158230 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3987752918 0 0) 0x7f11e4003520 con 0x7f11fc158230 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 msgr2=0x7f11fc0b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 0x7f11fc0b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 >> v2:172.21.15.172:3300/0 conn(0x7f11fc06c7e0 msgr2=0x7f11fc0b53a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 --2- 172.21.15.165:0/467669032 >> v2:172.21.15.172:3300/0 conn(0x7f11fc06c7e0 0x7f11fc0b53a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc0d2020 con 0x7f11fc158230 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f11f0001670 con 0x7f11fc158230 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 >> v1:172.21.15.165:6789/0 conn(0x7f11fc158230 legacy=0x7f11fc0d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e40039b0 0x7f11e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e4004800 0x7f11e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 0x7f11e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc158610 con 0x7f11e4004800 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc06bea0 con 0x7f11e40039b0 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.276+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e400b9c0 con 0x7f11fc06c330 2024-04-25T10:03:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201000640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e4004800 0x7f11e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1202400640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 0x7f11e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201a00640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e40039b0 0x7f11e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201000640 1 -- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 msgr2=0x7f11e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201000640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11fc06c330 0x7f11e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201000640 1 -- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e40039b0 msgr2=0x7f11e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201000640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11e40039b0 0x7f11e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.277+0000 7f1201000640 1 -- 172.21.15.165:0/467669032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e4003520 con 0x7f11e4004800 2024-04-25T10:03:05.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.278+0000 7f1201000640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e4004800 0x7f11e4004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009ab0 tx=0x7f11e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9cff4e6e0726faf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.278+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8036070 con 0x7f11e4004800 2024-04-25T10:03:05.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.278+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8005990 con 0x7f11e4004800 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.278+0000 7f11fbe00640 1 -- 172.21.15.165:0/467669032 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8009680 con 0x7f11e4004800 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1204400640 1 -- 172.21.15.165:0/467669032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e4004800 msgr2=0x7f11e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1204400640 1 --2- 172.21.15.165:0/467669032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11e4004800 0x7f11e4004c80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009ab0 tx=0x7f11e4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1202400640 1 -- 172.21.15.165:0/467669032 reap_dead start 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1204400640 1 -- 172.21.15.165:0/467669032 shutdown_connections 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1204400640 1 -- 172.21.15.165:0/467669032 >> 172.21.15.165:0/467669032 conn(0x7f11fc061600 msgr2=0x7f11fc0619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1204400640 1 -- 172.21.15.165:0/467669032 shutdown_connections 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.279+0000 7f1204400640 1 -- 172.21.15.165:0/467669032 wait complete. 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.281+0000 7f1204400640 1 Processor -- start 2024-04-25T10:03:05.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.281+0000 7f1204400640 1 -- start start 2024-04-25T10:03:05.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1204400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11e4004800 0x7f11fc0b3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1204400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 0x7f11fc0b1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1204400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc0b1dc0 0x7f11fc0cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1204400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc0d25e0 con 0x7f11fc1e59d0 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1204400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc06bb30 con 0x7f11fc0b1dc0 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1204400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11fc15e320 con 0x7f11e4004800 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1201000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 0x7f11fc0b1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1201000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 0x7f11fc0b1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39498/0 (socket says 172.21.15.165:39498) 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.282+0000 7f1201000640 1 -- 172.21.15.165:0/2729278785 learned_addr learned my addr 172.21.15.165:0/2729278785 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1202400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc0b1dc0 0x7f11fc0cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201a00640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11e4004800 0x7f11fc0b3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201000640 1 -- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11e4004800 msgr2=0x7f11fc0b3960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201000640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11e4004800 0x7f11fc0b3960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201000640 1 -- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc0b1dc0 msgr2=0x7f11fc0cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201000640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc0b1dc0 0x7f11fc0cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201000640 1 -- 172.21.15.165:0/2729278785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11fc1e6930 con 0x7f11fc1e59d0 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201a00640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11e4004800 0x7f11fc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.283+0000 7f1201000640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 0x7f11fc0b1880 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009bd0 tx=0x7f11e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.284+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8036070 con 0x7f11fc1e59d0 2024-04-25T10:03:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.284+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e8037040 con 0x7f11fc1e59d0 2024-04-25T10:03:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.284+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8004030 con 0x7f11fc1e59d0 2024-04-25T10:03:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.284+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11fc1e6090 con 0x7f11fc1e59d0 2024-04-25T10:03:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.284+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11fc1e6650 con 0x7f11fc1e59d0 2024-04-25T10:03:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.285+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11fc068450 con 0x7f11fc1e59d0 2024-04-25T10:03:05.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.286+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11e80041d0 con 0x7f11fc1e59d0 2024-04-25T10:03:05.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.287+0000 7f11f9600640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11dc06b4d0 0x7f11dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.287+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f11e80aa900 con 0x7f11fc1e59d0 2024-04-25T10:03:05.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.287+0000 7f1201a00640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11dc06b4d0 0x7f11dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.288+0000 7f1201a00640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11dc06b4d0 0x7f11dc06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f11f00058b0 tx=0x7f11f0001010 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.291+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11fc068450 con 0x7f11fc1e59d0 2024-04-25T10:03:05.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.487+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f11fc06de50 con 0x7f11fc1e59d0 2024-04-25T10:03:05.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.487+0000 7f11f9600640 1 -- 172.21.15.165:0/2729278785 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7f11e8078e40 con 0x7f11fc1e59d0 2024-04-25T10:03:05.492 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11dc06b4d0 msgr2=0x7f11dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11dc06b4d0 0x7f11dc06d990 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f11f00058b0 tx=0x7f11f0001010 comp rx=0 tx=0).stop 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 msgr2=0x7f11fc0b1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 0x7f11fc0b1880 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009bd0 tx=0x7f11e8004730 comp rx=0 tx=0).stop 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 shutdown_connections 2024-04-25T10:03:05.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11dc06b4d0 0x7f11dc06d990 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11fc0b1dc0 0x7f11fc0cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11fc1e59d0 0x7f11fc0b1880 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 --2- 172.21.15.165:0/2729278785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11e4004800 0x7f11fc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.490+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 >> 172.21.15.165:0/2729278785 conn(0x7f11fc061600 msgr2=0x7f11fc0cf8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.491+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 shutdown_connections 2024-04-25T10:03:05.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.491+0000 7f1204400640 1 -- 172.21.15.165:0/2729278785 wait complete. 2024-04-25T10:03:05.514 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:05.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.714+0000 7f9a08200640 1 Processor -- start 2024-04-25T10:03:05.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.715+0000 7f9a08200640 1 -- start start 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.716+0000 7f9a08200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a0015da90 0x7f9a00161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.716+0000 7f9a08200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000b9750 0x7f9a000ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.716+0000 7f9a08200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a00162040 con 0x7f9a00161ba0 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.716+0000 7f9a08200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a00162600 con 0x7f9a0015da90 2024-04-25T10:03:05.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.716+0000 7f9a08200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a00162320 con 0x7f9a000b9750 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.717+0000 7f9a05800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a0015da90 0x7f9a00161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.717+0000 7f9a04e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000b9750 0x7f9a000ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.717+0000 7f9a05800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a0015da90 0x7f9a00161660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:38388/0 (socket says 172.21.15.165:38388) 2024-04-25T10:03:05.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.717+0000 7f9a05800640 1 -- 172.21.15.165:0/1494619865 learned_addr learned my addr 172.21.15.165:0/1494619865 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:05.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.718+0000 7f9a05800640 1 -- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000b9750 msgr2=0x7f9a000ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:05.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.718+0000 7f9a05800640 1 --2- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000b9750 0x7f9a000ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.718+0000 7f9a05800640 1 -- 172.21.15.165:0/1494619865 >> v1:172.21.15.172:6789/0 conn(0x7f9a00161ba0 legacy=0x7f9a0006c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.718+0000 7f99ffe00640 1 -- 172.21.15.165:0/1494619865 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561121386 0 0) 0x7f9a00162040 con 0x7f9a00161ba0 2024-04-25T10:03:05.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.718+0000 7f9a05800640 1 -- 172.21.15.165:0/1494619865 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a000d1b10 con 0x7f9a0015da90 2024-04-25T10:03:05.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.718+0000 7f9a04e00640 1 --2- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000b9750 0x7f9a000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.719+0000 7f9a05800640 1 --2- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a00161660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009870 tx=0x7f99f002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdd808e0ed0083f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.720+0000 7f99ffe00640 1 -- 172.21.15.165:0/1494619865 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0037070 con 0x7f9a0015da90 2024-04-25T10:03:05.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.720+0000 7f99ffe00640 1 -- 172.21.15.165:0/1494619865 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0032440 con 0x7f9a0015da90 2024-04-25T10:03:05.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.720+0000 7f99ffe00640 1 -- 172.21.15.165:0/1494619865 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0032810 con 0x7f9a0015da90 2024-04-25T10:03:05.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 -- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 msgr2=0x7f9a00161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 --2- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a00161660 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f99f0009870 tx=0x7f99f002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 -- 172.21.15.165:0/1494619865 shutdown_connections 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 --2- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000b9750 0x7f9a000ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 --2- 172.21.15.165:0/1494619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a00161660 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 -- 172.21.15.165:0/1494619865 >> 172.21.15.165:0/1494619865 conn(0x7f9a00061a20 msgr2=0x7f9a00061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 -- 172.21.15.165:0/1494619865 shutdown_connections 2024-04-25T10:03:05.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.721+0000 7f9a08200640 1 -- 172.21.15.165:0/1494619865 wait complete. 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.723+0000 7f9a08200640 1 Processor -- start 2024-04-25T10:03:05.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.723+0000 7f9a08200640 1 -- start start 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a08200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a001517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a08200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 0x7f9a000cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a08200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000cbbd0 0x7f9a000b4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a08200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000d3320 con 0x7f9a00161ba0 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a08200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0015de70 con 0x7f9a0015da90 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a08200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000d34a0 con 0x7f9a000cbbd0 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a04e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 0x7f9a000cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a04e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 0x7f9a000cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39500/0 (socket says 172.21.15.165:39500) 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a04e00640 1 -- 172.21.15.165:0/3977084244 learned_addr learned my addr 172.21.15.165:0/3977084244 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a05800640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a001517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a06200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000cbbd0 0x7f9a000b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.724+0000 7f9a04e00640 1 -- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000cbbd0 msgr2=0x7f9a000b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a04e00640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000cbbd0 0x7f9a000b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a04e00640 1 -- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 msgr2=0x7f9a001517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a04e00640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a001517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a04e00640 1 -- 172.21.15.165:0/3977084244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00151ce0 con 0x7f9a00161ba0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a04e00640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 0x7f9a000cb690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ebb0 tx=0x7f99ec00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec017070 con 0x7f9a00161ba0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99ec00cc30 con 0x7f9a00161ba0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99ec013470 con 0x7f9a00161ba0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a001521d0 con 0x7f9a00161ba0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.725+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000c7690 con 0x7f9a00161ba0 2024-04-25T10:03:05.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.726+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a00073a00 con 0x7f9a00161ba0 2024-04-25T10:03:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.727+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99ec01a070 con 0x7f9a00161ba0 2024-04-25T10:03:05.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.729+0000 7f99fd600640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e006b4d0 0x7f99e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:05.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.729+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f99ec08a5d0 con 0x7f9a00161ba0 2024-04-25T10:03:05.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.729+0000 7f9a05800640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e006b4d0 0x7f99e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:05.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.730+0000 7f9a05800640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e006b4d0 0x7f99e006d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034000 tx=0x7f99f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:05.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.733+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99ec059050 con 0x7f9a00161ba0 2024-04-25T10:03:05.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.926+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9a00152420 con 0x7f9a00161ba0 2024-04-25T10:03:05.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.927+0000 7f99fd600640 1 -- 172.21.15.165:0/3977084244 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f99ec058950 con 0x7f9a00161ba0 2024-04-25T10:03:05.931 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:05.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e006b4d0 msgr2=0x7f99e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e006b4d0 0x7f99e006d990 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034000 tx=0x7f99f002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 msgr2=0x7f9a000cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 0x7f9a000cb690 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f99ec00ebb0 tx=0x7f99ec00c520 comp rx=0 tx=0).stop 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 shutdown_connections 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e006b4d0 0x7f99e006d990 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a000cbbd0 0x7f9a000b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a00161ba0 0x7f9a000cb690 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 --2- 172.21.15.165:0/3977084244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a0015da90 0x7f9a001517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.929+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 >> 172.21.15.165:0/3977084244 conn(0x7f9a00061a20 msgr2=0x7f9a000b6a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.930+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 shutdown_connections 2024-04-25T10:03:05.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:05.930+0000 7f9a08200640 1 -- 172.21.15.165:0/3977084244 wait complete. 2024-04-25T10:03:05.953 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:06.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.151+0000 7f23c1400640 1 Processor -- start 2024-04-25T10:03:06.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.152+0000 7f23c1400640 1 -- start start 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.153+0000 7f23c1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f23bc06bff0 0x7f23bc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.153+0000 7f23c1400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc15e110 con 0x7f23bc15dd30 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.153+0000 7f23c1400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23bc0b2fd0 con 0x7f23bc0b5970 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.153+0000 7f23c1400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23bc06d000 con 0x7f23bc06bff0 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.153+0000 7f23bbe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f23bc15dd30 0x7f23bc0cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49012/0 (socket says 172.21.15.165:49012) 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.153+0000 7f23bbe00640 1 -- 172.21.15.165:0/1675532978 learned_addr learned my addr 172.21.15.165:0/1675532978 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:06.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.154+0000 7f23bb400640 1 --2- 172.21.15.165:0/1675532978 >> v2:172.21.15.165:3300/0 conn(0x7f23bc06bff0 0x7f23bc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.154+0000 7f23bb400640 1 -- 172.21.15.165:0/1675532978 >> v1:172.21.15.172:6789/0 conn(0x7f23bc0b5970 legacy=0x7f23bc0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.154+0000 7f23bb400640 1 -- 172.21.15.165:0/1675532978 >> v1:172.21.15.165:6789/0 conn(0x7f23bc15dd30 legacy=0x7f23bc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.154+0000 7f23bb400640 1 -- 172.21.15.165:0/1675532978 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc06c480 con 0x7f23bc06bff0 2024-04-25T10:03:06.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.154+0000 7f23baa00640 1 -- 172.21.15.165:0/1675532978 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693310326 0 0) 0x7f23bc15e110 con 0x7f23bc15dd30 2024-04-25T10:03:06.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.155+0000 7f23bb400640 1 --2- 172.21.15.165:0/1675532978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc06bff0 0x7f23bc0b5430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009d90 tx=0x7f23a802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ce821bf645f81a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23baa00640 1 -- 172.21.15.165:0/1675532978 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8007740 con 0x7f23bc06bff0 2024-04-25T10:03:06.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23baa00640 1 -- 172.21.15.165:0/1675532978 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23a80078e0 con 0x7f23bc06bff0 2024-04-25T10:03:06.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23baa00640 1 -- 172.21.15.165:0/1675532978 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a8007cb0 con 0x7f23bc06bff0 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 -- 172.21.15.165:0/1675532978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc06bff0 msgr2=0x7f23bc0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 --2- 172.21.15.165:0/1675532978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc06bff0 0x7f23bc0b5430 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f23a8009d90 tx=0x7f23a802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 -- 172.21.15.165:0/1675532978 shutdown_connections 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 --2- 172.21.15.165:0/1675532978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc06bff0 0x7f23bc0b5430 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 -- 172.21.15.165:0/1675532978 >> 172.21.15.165:0/1675532978 conn(0x7f23bc0d0610 msgr2=0x7f23bc0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 -- 172.21.15.165:0/1675532978 shutdown_connections 2024-04-25T10:03:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.156+0000 7f23c1400640 1 -- 172.21.15.165:0/1675532978 wait complete. 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.158+0000 7f23c1400640 1 Processor -- start 2024-04-25T10:03:06.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.158+0000 7f23c1400640 1 -- start start 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23bc0b5970 0x7f23bc0cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23bc15dd30 0x7f23bc14ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 0x7f23bc0ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c1400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc06cd20 con 0x7f23bc0abdf0 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c1400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc06ca40 con 0x7f23bc0b5970 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c1400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23bc0d2870 con 0x7f23bc15dd30 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 0x7f23bc0ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 0x7f23bc0ac280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39512/0 (socket says 172.21.15.165:39512) 2024-04-25T10:03:06.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.159+0000 7f23c0a00640 1 -- 172.21.15.165:0/908981626 learned_addr learned my addr 172.21.15.165:0/908981626 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23bbe00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23bc0b5970 0x7f23bc0cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23bb400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23bc15dd30 0x7f23bc14ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23c0a00640 1 -- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23bc15dd30 msgr2=0x7f23bc14ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23c0a00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23bc15dd30 0x7f23bc14ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23c0a00640 1 -- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23bc0b5970 msgr2=0x7f23bc0cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23c0a00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23bc0b5970 0x7f23bc0cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23c0a00640 1 -- 172.21.15.165:0/908981626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0cc810 con 0x7f23bc0abdf0 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23bbe00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23bc0b5970 0x7f23bc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23c0a00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 0x7f23bc0ac280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c410 tx=0x7f23b400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4011790 con 0x7f23bc0abdf0 2024-04-25T10:03:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23b4017070 con 0x7f23bc0abdf0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.160+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23b4011bb0 con 0x7f23bc0abdf0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.161+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23bc0ccaf0 con 0x7f23bc0abdf0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.161+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23bc0cd0b0 con 0x7f23bc0abdf0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.162+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23bc073990 con 0x7f23bc0abdf0 2024-04-25T10:03:06.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.163+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f23b401c460 con 0x7f23bc0abdf0 2024-04-25T10:03:06.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.164+0000 7f23b3e00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f239c06b500 0x7f239c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.164+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f23b408a450 con 0x7f23bc0abdf0 2024-04-25T10:03:06.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.164+0000 7f23bbe00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f239c06b500 0x7f239c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.165+0000 7f23bbe00640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f239c06b500 0x7f239c06d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000f60 tx=0x7f23ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.168+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f23b4058e30 con 0x7f23bc0abdf0 2024-04-25T10:03:06.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.365+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f23bc0d16c0 con 0x7f23bc0abdf0 2024-04-25T10:03:06.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.366+0000 7f23b3e00640 1 -- 172.21.15.165:0/908981626 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f23b4058730 con 0x7f23bc0abdf0 2024-04-25T10:03:06.371 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:06.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.368+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f239c06b500 msgr2=0x7f239c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.368+0000 7f23c1400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f239c06b500 0x7f239c06d9c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f23ac000f60 tx=0x7f23ac009040 comp rx=0 tx=0).stop 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.368+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 msgr2=0x7f23bc0ac280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.368+0000 7f23c1400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 0x7f23bc0ac280 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f23b400c410 tx=0x7f23b400c8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 shutdown_connections 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f239c06b500 0x7f239c06d9c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23bc0abdf0 0x7f23bc0ac280 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23bc15dd30 0x7f23bc14ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 --2- 172.21.15.165:0/908981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23bc0b5970 0x7f23bc0cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 >> 172.21.15.165:0/908981626 conn(0x7f23bc0d0610 msgr2=0x7f23bc0d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 shutdown_connections 2024-04-25T10:03:06.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.369+0000 7f23c1400640 1 -- 172.21.15.165:0/908981626 wait complete. 2024-04-25T10:03:06.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:06.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.595+0000 7f5fd5400640 1 Processor -- start 2024-04-25T10:03:06.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.596+0000 7f5fd5400640 1 -- start start 2024-04-25T10:03:06.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.597+0000 7f5fd5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fd0161fc0 0x7f5fd01623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.597+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd0071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.597+0000 7f5fd5400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fd01628e0 con 0x7f5fd006ef10 2024-04-25T10:03:06.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.597+0000 7f5fd5400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fd0165be0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.597+0000 7f5fd5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0162ba0 con 0x7f5fd006ea90 2024-04-25T10:03:06.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.598+0000 7f5fce800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fd0161fc0 0x7f5fd01623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.598+0000 7f5fc5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd0071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.598+0000 7f5fce800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5fd0161fc0 0x7f5fd01623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:38412/0 (socket says 172.21.15.165:38412) 2024-04-25T10:03:06.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.598+0000 7f5fce800640 1 -- 172.21.15.165:0/879710175 learned_addr learned my addr 172.21.15.165:0/879710175 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:06.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.599+0000 7f5fce800640 1 -- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 msgr2=0x7f5fd0071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:06.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.599+0000 7f5fce800640 1 --2- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd0071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.599+0000 7f5fce800640 1 -- 172.21.15.165:0/879710175 >> v1:172.21.15.172:6789/0 conn(0x7f5fd006ef10 legacy=0x7f5fd006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.599+0000 7f5fce800640 1 -- 172.21.15.165:0/879710175 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0162e80 con 0x7f5fd0161fc0 2024-04-25T10:03:06.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.601+0000 7f5fce800640 1 --2- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd0161fc0 0x7f5fd01623a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009870 tx=0x7f5fc002a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a1453e250cd3151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.601+0000 7f5fcde00640 1 -- 172.21.15.165:0/879710175 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0037070 con 0x7f5fd0161fc0 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.601+0000 7f5fcde00640 1 -- 172.21.15.165:0/879710175 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc0032440 con 0x7f5fd0161fc0 2024-04-25T10:03:06.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.601+0000 7f5fcde00640 1 -- 172.21.15.165:0/879710175 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc0032810 con 0x7f5fd0161fc0 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 -- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd0161fc0 msgr2=0x7f5fd01623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 --2- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd0161fc0 0x7f5fd01623a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009870 tx=0x7f5fc002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 -- 172.21.15.165:0/879710175 shutdown_connections 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 --2- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd0071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 --2- 172.21.15.165:0/879710175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd0161fc0 0x7f5fd01623a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 -- 172.21.15.165:0/879710175 >> 172.21.15.165:0/879710175 conn(0x7f5fd006abe0 msgr2=0x7f5fd006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.602+0000 7f5fd5400640 1 -- 172.21.15.165:0/879710175 shutdown_connections 2024-04-25T10:03:06.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.603+0000 7f5fd5400640 1 -- 172.21.15.165:0/879710175 wait complete. 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.605+0000 7f5fd5400640 1 Processor -- start 2024-04-25T10:03:06.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.605+0000 7f5fd5400640 1 -- start start 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd00cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd006ef10 0x7f5fd00cff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fd5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 0x7f5fd00ad5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fd5400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00719c0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fd5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0163440 con 0x7f5fd006ef10 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fd5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd0071b40 con 0x7f5fd006ea90 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 0x7f5fd00ad5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fcf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 0x7f5fd00ad5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39528/0 (socket says 172.21.15.165:39528) 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fcf200640 1 -- 172.21.15.165:0/3142882580 learned_addr learned my addr 172.21.15.165:0/3142882580 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.606+0000 7f5fc5e00640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd006ef10 0x7f5fd00cff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fce800640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd00cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fcf200640 1 -- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 msgr2=0x7f5fd00cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fcf200640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd00cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fcf200640 1 -- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd006ef10 msgr2=0x7f5fd00cff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fcf200640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd006ef10 0x7f5fd00cff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fcf200640 1 -- 172.21.15.165:0/3142882580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd00ae300 con 0x7f5fd0161fc0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fcf200640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 0x7f5fd00ad5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c730 tx=0x7f5fc800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.607+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8016020 con 0x7f5fd0161fc0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.608+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc8006d50 con 0x7f5fd0161fc0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.608+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8003bb0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.608+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd0150510 con 0x7f5fd0161fc0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.608+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd014fa30 con 0x7f5fd0161fc0 2024-04-25T10:03:06.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.609+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd014d2b0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.610+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fc800a7f0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.611+0000 7f5fc7400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fac06b4d0 0x7f5fac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:06.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.611+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5fc808a1c0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.611+0000 7f5fce800640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fac06b4d0 0x7f5fac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:06.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.612+0000 7f5fce800640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fac06b4d0 0x7f5fac06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0034000 tx=0x7f5fc002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:06.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.616+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fd014d2b0 con 0x7f5fd0161fc0 2024-04-25T10:03:06.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.829+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5fd0073a00 con 0x7f5fd0161fc0 2024-04-25T10:03:06.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.829+0000 7f5fc7400640 1 -- 172.21.15.165:0/3142882580 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5fc8020020 con 0x7f5fd0161fc0 2024-04-25T10:03:06.834 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fac06b4d0 msgr2=0x7f5fac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fac06b4d0 0x7f5fac06d990 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0034000 tx=0x7f5fc002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 msgr2=0x7f5fd00ad5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 0x7f5fd00ad5d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f5fc800c730 tx=0x7f5fc800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 shutdown_connections 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fac06b4d0 0x7f5fac06d990 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fd0161fc0 0x7f5fd00ad5d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fd006ef10 0x7f5fd00cff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 --2- 172.21.15.165:0/3142882580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fd006ea90 0x7f5fd00cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:06.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 >> 172.21.15.165:0/3142882580 conn(0x7f5fd006abe0 msgr2=0x7f5fd00af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:06.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 shutdown_connections 2024-04-25T10:03:06.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:06.832+0000 7f5fd5400640 1 -- 172.21.15.165:0/3142882580 wait complete. 2024-04-25T10:03:06.855 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:03:06.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:06.855 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-04-25T10:03:06.940 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:03:06.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:06.941 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:03:06.990 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:03:06.990 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:07.049 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:03:07.049 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:07.102 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:03:07.102 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:07.156 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.362+0000 7f54d2c00640 1 Processor -- start 2024-04-25T10:03:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.362+0000 7f54d2c00640 1 -- start start 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d2c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54cc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d2c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f54cc06eb10 0x7f54cc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d2c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54cc144b30 con 0x7f54cc0cee60 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d2c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54cc0712d0 con 0x7f54cc06eb10 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d2c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0715b0 con 0x7f54cc144750 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d0c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f54cc0cee60 0x7f54cc15f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36436/0 (socket says 172.21.15.165:36436) 2024-04-25T10:03:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.364+0000 7f54d0c00640 1 -- 172.21.15.165:0/82219487 learned_addr learned my addr 172.21.15.165:0/82219487 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54c3e00640 1 --2- 172.21.15.165:0/82219487 >> v2:172.21.15.172:3300/0 conn(0x7f54cc06eb10 0x7f54cc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cbe00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54cc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347226312 0 0) 0x7f54cc144b30 con 0x7f54cc0cee60 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54b0003520 con 0x7f54cc0cee60 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 519817220 0 0) 0x7f54b0003520 con 0x7f54cc0cee60 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 msgr2=0x7f54cc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54cc06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 >> v2:172.21.15.172:3300/0 conn(0x7f54cc06eb10 msgr2=0x7f54cc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 --2- 172.21.15.165:0/82219487 >> v2:172.21.15.172:3300/0 conn(0x7f54cc06eb10 0x7f54cc0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc071890 con 0x7f54cc0cee60 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f54c40019f0 con 0x7f54cc0cee60 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 >> v1:172.21.15.165:6789/0 conn(0x7f54cc0cee60 legacy=0x7f54cc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54b00039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.365+0000 7f54cb400640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54b0003f80 0x7f54b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54cb400640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54b0004d40 0x7f54b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0712d0 con 0x7f54b0004d40 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc0715b0 con 0x7f54b0003f80 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54b000b9c0 con 0x7f54cc144750 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54d0c00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54b0004d40 0x7f54b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54c3e00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54b0003f80 0x7f54b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54cbe00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54b00039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54d0c00640 1 -- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 msgr2=0x7f54b00039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54d0c00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54b00039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54d0c00640 1 -- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54b0003f80 msgr2=0x7f54b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54d0c00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54b0003f80 0x7f54b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54d0c00640 1 -- 172.21.15.165:0/82219487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0003520 con 0x7f54b0004d40 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.366+0000 7f54cbe00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54cc144750 0x7f54b00039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.367+0000 7f54d0c00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54b0004d40 0x7f54b00051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f54c400a7d0 tx=0x7f54b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290ac0385d79273b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c400a3f0 con 0x7f54b0004d40 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c402f070 con 0x7f54b0004d40 2024-04-25T10:03:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54cb400640 1 -- 172.21.15.165:0/82219487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c402bc50 con 0x7f54b0004d40 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d2c00640 1 -- 172.21.15.165:0/82219487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54b0004d40 msgr2=0x7f54b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d2c00640 1 --2- 172.21.15.165:0/82219487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54b0004d40 0x7f54b00051c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f54c400a7d0 tx=0x7f54b0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d0c00640 1 -- 172.21.15.165:0/82219487 reap_dead start 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d2c00640 1 -- 172.21.15.165:0/82219487 shutdown_connections 2024-04-25T10:03:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d2c00640 1 -- 172.21.15.165:0/82219487 >> 172.21.15.165:0/82219487 conn(0x7f54cc06abe0 msgr2=0x7f54cc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d2c00640 1 -- 172.21.15.165:0/82219487 shutdown_connections 2024-04-25T10:03:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.368+0000 7f54d2c00640 1 -- 172.21.15.165:0/82219487 wait complete. 2024-04-25T10:03:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.370+0000 7f54d2c00640 1 Processor -- start 2024-04-25T10:03:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.370+0000 7f54d2c00640 1 -- start start 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.371+0000 7f54d2c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54b0004d40 0x7f54cc1434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.371+0000 7f54d2c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 0x7f54cc0b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.371+0000 7f54d2c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54cc1f2f40 0x7f54cc1f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.371+0000 7f54d2c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc145450 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.371+0000 7f54d2c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc06f020 con 0x7f54cc1f2f40 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.371+0000 7f54d2c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54cc1455d0 con 0x7f54b0004d40 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 0x7f54cc0b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 0x7f54cc0b1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51396/0 (socket says 172.21.15.165:51396) 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 -- 172.21.15.165:0/402263921 learned_addr learned my addr 172.21.15.165:0/402263921 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54d0c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54cc1f2f40 0x7f54cc1f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54cbe00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54b0004d40 0x7f54cc1434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 -- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54b0004d40 msgr2=0x7f54cc1434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54b0004d40 0x7f54cc1434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 -- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54cc1f2f40 msgr2=0x7f54cc1f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54cc1f2f40 0x7f54cc1f33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 -- 172.21.15.165:0/402263921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc1306c0 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.372+0000 7f54c3e00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 0x7f54cc0b1700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f54b40115e0 tx=0x7f54b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.373+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4004220 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.373+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b40043c0 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.373+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400f930 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.373+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54cc1f8040 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.373+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54cc130960 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.374+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54cc0ada10 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.375+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54b400fad0 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.376+0000 7f54c8c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54a806b500 0x7f54a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.376+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f54b408a9a0 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.377+0000 7f54cbe00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54a806b500 0x7f54a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.377+0000 7f54cbe00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54a806b500 0x7f54a806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0099c0 tx=0x7f54bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.381+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54cc0ada10 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.577+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54cc0d0510 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.579+0000 7f54c8c00640 1 -- 172.21.15.165:0/402263921 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f54b4019030 con 0x7f54cc1f7bc0 2024-04-25T10:03:07.584 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:07.584 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54a806b500 msgr2=0x7f54a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54a806b500 0x7f54a806d9c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f54bc0099c0 tx=0x7f54bc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 msgr2=0x7f54cc0b1700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 0x7f54cc0b1700 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f54b40115e0 tx=0x7f54b4011aa0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 shutdown_connections 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54a806b500 0x7f54a806d9c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54cc1f2f40 0x7f54cc1f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54cc1f7bc0 0x7f54cc0b1700 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 --2- 172.21.15.165:0/402263921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54b0004d40 0x7f54cc1434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 >> 172.21.15.165:0/402263921 conn(0x7f54cc06abe0 msgr2=0x7f54cc0ad300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 shutdown_connections 2024-04-25T10:03:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.582+0000 7f54d2c00640 1 -- 172.21.15.165:0/402263921 wait complete. 2024-04-25T10:03:07.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:07.607 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:07.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.806+0000 7fd32dc00640 1 Processor -- start 2024-04-25T10:03:07.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.806+0000 7fd32dc00640 1 -- start start 2024-04-25T10:03:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.807+0000 7fd32dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 0x7fd3280cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.807+0000 7fd32dc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd328072280 con 0x7fd32815b270 2024-04-25T10:03:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.807+0000 7fd32dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3280714c0 con 0x7fd3280cc4f0 2024-04-25T10:03:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.807+0000 7fd32dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd328071270 con 0x7fd3280ce230 2024-04-25T10:03:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.808+0000 7fd327400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd32815b270 0x7fd32815b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36440/0 (socket says 172.21.15.165:36440) 2024-04-25T10:03:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.808+0000 7fd327400640 1 -- 172.21.15.165:0/1094201574 learned_addr learned my addr 172.21.15.165:0/1094201574 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.808+0000 7fd31e000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 0x7fd3280cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.808+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702137213 0 0) 0x7fd328072280 con 0x7fd32815b270 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd30c003520 con 0x7fd32815b270 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3891446610 0 0) 0x7fd30c003520 con 0x7fd32815b270 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 msgr2=0x7fd3280cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 0x7fd3280cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 >> v1:172.21.15.172:6789/0 conn(0x7fd3280cc4f0 legacy=0x7fd3280cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3280717a0 con 0x7fd32815b270 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd3200019f0 con 0x7fd32815b270 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 >> v1:172.21.15.165:6789/0 conn(0x7fd32815b270 legacy=0x7fd32815b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 0x7fd30c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd30c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd30c004d40 0x7fd30c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd328071270 con 0x7fd30c003f60 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c00dcb0 con 0x7fd30c004d40 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c00deb0 con 0x7fd3280ce230 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.809+0000 7fd31e000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd30c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd326a00640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 0x7fd30c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd327400640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd30c004d40 0x7fd30c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd31e000640 1 -- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 msgr2=0x7fd30c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd31e000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3280ce230 0x7fd30c0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd31e000640 1 -- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd30c004d40 msgr2=0x7fd30c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd31e000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd30c004d40 0x7fd30c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd31e000640 1 -- 172.21.15.165:0/1094201574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30c003520 con 0x7fd30c003f60 2024-04-25T10:03:07.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.810+0000 7fd327400640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd30c004d40 0x7fd30c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.811+0000 7fd31e000640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd30c004800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd310009ac0 tx=0x7fd30c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769f769ee3cd68e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.811+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310036070 con 0x7fd30c003f60 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.811+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd310031400 con 0x7fd30c003f60 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.811+0000 7fd326000640 1 -- 172.21.15.165:0/1094201574 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd310009690 con 0x7fd30c003f60 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd32dc00640 1 -- 172.21.15.165:0/1094201574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 msgr2=0x7fd30c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd32dc00640 1 --2- 172.21.15.165:0/1094201574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd30c004800 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd310009ac0 tx=0x7fd30c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd327400640 1 -- 172.21.15.165:0/1094201574 reap_dead start 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd32dc00640 1 -- 172.21.15.165:0/1094201574 shutdown_connections 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd32dc00640 1 -- 172.21.15.165:0/1094201574 >> 172.21.15.165:0/1094201574 conn(0x7fd32806abe0 msgr2=0x7fd32806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd32dc00640 1 -- 172.21.15.165:0/1094201574 shutdown_connections 2024-04-25T10:03:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.812+0000 7fd32dc00640 1 -- 172.21.15.165:0/1094201574 wait complete. 2024-04-25T10:03:07.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.814+0000 7fd32dc00640 1 Processor -- start 2024-04-25T10:03:07.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.814+0000 7fd32dc00640 1 -- start start 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd32dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd3281e7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd32dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3281e77d0 0x7fd32813f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd32dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd32813f840 0x7fd3281401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd32dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd32806e8a0 con 0x7fd30c003f60 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd32dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd328071d60 con 0x7fd3281e77d0 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd32dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3280cc8d0 con 0x7fd32813f840 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd326a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd3281e7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd326a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd3281e7290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51422/0 (socket says 172.21.15.165:51422) 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.815+0000 7fd326a00640 1 -- 172.21.15.165:0/98168578 learned_addr learned my addr 172.21.15.165:0/98168578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd31e000640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3281e77d0 0x7fd32813f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd327400640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd32813f840 0x7fd3281401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd326a00640 1 -- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd32813f840 msgr2=0x7fd3281401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd326a00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd32813f840 0x7fd3281401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd326a00640 1 -- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3281e77d0 msgr2=0x7fd32813f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd326a00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3281e77d0 0x7fd32813f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd326a00640 1 -- 172.21.15.165:0/98168578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd32813e260 con 0x7fd30c003f60 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd326a00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd3281e7290 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d570 tx=0x7fd31800da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318005480 con 0x7fd30c003f60 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318017070 con 0x7fd30c003f60 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318003810 con 0x7fd30c003f60 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.816+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd32813e540 con 0x7fd30c003f60 2024-04-25T10:03:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.817+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd32813eb00 con 0x7fd30c003f60 2024-04-25T10:03:07.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.818+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd328073a00 con 0x7fd30c003f60 2024-04-25T10:03:07.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.819+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3180039b0 con 0x7fd30c003f60 2024-04-25T10:03:07.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.820+0000 7fd31f400640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30406b4d0 0x7fd30406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:07.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.820+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fd31808a520 con 0x7fd30c003f60 2024-04-25T10:03:07.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.820+0000 7fd31e000640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30406b4d0 0x7fd30406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:07.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.821+0000 7fd31e000640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30406b4d0 0x7fd30406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd310033000 tx=0x7fd310008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:07.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:07.824+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd318058f00 con 0x7fd30c003f60 2024-04-25T10:03:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.019+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd32806d3a0 con 0x7fd30c003f60 2024-04-25T10:03:08.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.020+0000 7fd31f400640 1 -- 172.21.15.165:0/98168578 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd318058800 con 0x7fd30c003f60 2024-04-25T10:03:08.025 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:08.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.022+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30406b4d0 msgr2=0x7fd30406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.022+0000 7fd32dc00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30406b4d0 0x7fd30406d990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd310033000 tx=0x7fd310008030 comp rx=0 tx=0).stop 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 msgr2=0x7fd3281e7290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd3281e7290 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd31800d570 tx=0x7fd31800da30 comp rx=0 tx=0).stop 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 shutdown_connections 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30406b4d0 0x7fd30406d990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd32813f840 0x7fd3281401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3281e77d0 0x7fd32813f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 --2- 172.21.15.165:0/98168578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd30c003f60 0x7fd3281e7290 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 >> 172.21.15.165:0/98168578 conn(0x7fd32806abe0 msgr2=0x7fd3280cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 shutdown_connections 2024-04-25T10:03:08.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.023+0000 7fd32dc00640 1 -- 172.21.15.165:0/98168578 wait complete. 2024-04-25T10:03:08.047 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.253+0000 7fec49a00640 1 Processor -- start 2024-04-25T10:03:08.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.253+0000 7fec49a00640 1 -- start start 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec49a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fec44145140 0x7fec440ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec49a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440ce950 0x7fec440c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec49a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fec440c5030 0x7fec4406d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec49a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440ced90 con 0x7fec44145140 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec49a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec44145520 con 0x7fec440c5030 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec49a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec44071270 con 0x7fec440ce950 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec43e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fec44145140 0x7fec440ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec43e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fec44145140 0x7fec440ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51428/0 (socket says 172.21.15.165:51428) 2024-04-25T10:03:08.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.255+0000 7fec43e00640 1 -- 172.21.15.165:0/3102782109 learned_addr learned my addr 172.21.15.165:0/3102782109 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec49000640 1 --2- 172.21.15.165:0/3102782109 >> v2:172.21.15.172:3300/0 conn(0x7fec440c5030 0x7fec4406d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec3b400640 1 --2- 172.21.15.165:0/3102782109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440ce950 0x7fec440c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec43e00640 1 -- 172.21.15.165:0/3102782109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440ce950 msgr2=0x7fec440c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec43e00640 1 --2- 172.21.15.165:0/3102782109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440ce950 0x7fec440c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec43e00640 1 -- 172.21.15.165:0/3102782109 >> v2:172.21.15.172:3300/0 conn(0x7fec440c5030 msgr2=0x7fec4406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec43e00640 1 --2- 172.21.15.165:0/3102782109 >> v2:172.21.15.172:3300/0 conn(0x7fec440c5030 0x7fec4406d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.256+0000 7fec43e00640 1 -- 172.21.15.165:0/3102782109 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec44071550 con 0x7fec44145140 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.257+0000 7fec43e00640 1 --2- 172.21.15.165:0/3102782109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440ce410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec440b2860 tx=0x7fec3402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aebda94590743f05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.258+0000 7fec43400640 1 -- 172.21.15.165:0/3102782109 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34037070 con 0x7fec44145140 2024-04-25T10:03:08.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.258+0000 7fec43400640 1 -- 172.21.15.165:0/3102782109 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec34032440 con 0x7fec44145140 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.258+0000 7fec43400640 1 -- 172.21.15.165:0/3102782109 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec34032810 con 0x7fec44145140 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 -- 172.21.15.165:0/3102782109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 msgr2=0x7fec440ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 --2- 172.21.15.165:0/3102782109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440ce410 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fec440b2860 tx=0x7fec3402a140 comp rx=0 tx=0).stop 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 -- 172.21.15.165:0/3102782109 shutdown_connections 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 --2- 172.21.15.165:0/3102782109 >> v2:172.21.15.172:3300/0 conn(0x7fec440c5030 0x7fec4406d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 --2- 172.21.15.165:0/3102782109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440ce950 0x7fec440c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 --2- 172.21.15.165:0/3102782109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440ce410 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 -- 172.21.15.165:0/3102782109 >> 172.21.15.165:0/3102782109 conn(0x7fec44061cc0 msgr2=0x7fec4406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 -- 172.21.15.165:0/3102782109 shutdown_connections 2024-04-25T10:03:08.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.259+0000 7fec49a00640 1 -- 172.21.15.165:0/3102782109 wait complete. 2024-04-25T10:03:08.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.261+0000 7fec49a00640 1 Processor -- start 2024-04-25T10:03:08.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.261+0000 7fec49a00640 1 -- start start 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440c5030 0x7fec440ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec440ce950 0x7fec440b0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440c54a0 con 0x7fec44145140 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44071b10 con 0x7fec440ce950 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440c5620 con 0x7fec440c5030 2024-04-25T10:03:08.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440b11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51432/0 (socket says 172.21.15.165:51432) 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 -- 172.21.15.165:0/1944616032 learned_addr learned my addr 172.21.15.165:0/1944616032 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec43e00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440c5030 0x7fec440ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec3b400640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec440ce950 0x7fec440b0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 -- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440c5030 msgr2=0x7fec440ac2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440c5030 0x7fec440ac2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 -- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec440ce950 msgr2=0x7fec440b0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec440ce950 0x7fec440b0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.263+0000 7fec49000640 1 -- 172.21.15.165:0/1944616032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440aa710 con 0x7fec44145140 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec3b400640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec440ce950 0x7fec440b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec49000640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440b11b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fec3c007e60 tx=0x7fec3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c014620 con 0x7fec44145140 2024-04-25T10:03:08.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c019070 con 0x7fec44145140 2024-04-25T10:03:08.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c014a20 con 0x7fec44145140 2024-04-25T10:03:08.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec440ac810 con 0x7fec44145140 2024-04-25T10:03:08.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.264+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec440aa980 con 0x7fec44145140 2024-04-25T10:03:08.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.266+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec44073990 con 0x7fec44145140 2024-04-25T10:03:08.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.267+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec3c014bc0 con 0x7fec44145140 2024-04-25T10:03:08.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.268+0000 7fec40c00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec2006b4d0 0x7fec2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.268+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fec3c08aff0 con 0x7fec44145140 2024-04-25T10:03:08.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.268+0000 7fec43e00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec2006b4d0 0x7fec2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.269+0000 7fec43e00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec2006b4d0 0x7fec2006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec34034000 tx=0x7fec3402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.272+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec440ac810 con 0x7fec44145140 2024-04-25T10:03:08.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.472+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fec440cc410 con 0x7fec44145140 2024-04-25T10:03:08.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.473+0000 7fec40c00640 1 -- 172.21.15.165:0/1944616032 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fec3c059940 con 0x7fec44145140 2024-04-25T10:03:08.478 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec2006b4d0 msgr2=0x7fec2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec2006b4d0 0x7fec2006d990 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fec34034000 tx=0x7fec3402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 msgr2=0x7fec440b11b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440b11b0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fec3c007e60 tx=0x7fec3c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 shutdown_connections 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec2006b4d0 0x7fec2006d990 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec44145140 0x7fec440b11b0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec440ce950 0x7fec440b0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 --2- 172.21.15.165:0/1944616032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec440c5030 0x7fec440ac2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 >> 172.21.15.165:0/1944616032 conn(0x7fec44061cc0 msgr2=0x7fec4415e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 shutdown_connections 2024-04-25T10:03:08.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.476+0000 7fec49a00640 1 -- 172.21.15.165:0/1944616032 wait complete. 2024-04-25T10:03:08.501 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.702+0000 7f27cb200640 1 Processor -- start 2024-04-25T10:03:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.702+0000 7f27cb200640 1 -- start start 2024-04-25T10:03:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.704+0000 7f27cb200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27cc15dd30 0x7f27cc0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.704+0000 7f27cb200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc15e110 con 0x7f27cc06bff0 2024-04-25T10:03:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.704+0000 7f27cb200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc0b2fd0 con 0x7f27cc0b5b40 2024-04-25T10:03:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.704+0000 7f27cb200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc06d000 con 0x7f27cc15dd30 2024-04-25T10:03:08.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.704+0000 7f27c3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f27cc06bff0 0x7f27cc0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36442/0 (socket says 172.21.15.165:36442) 2024-04-25T10:03:08.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c3e00640 1 -- 172.21.15.165:0/1939849880 learned_addr learned my addr 172.21.15.165:0/1939849880 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:08.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9e00640 1 --2- 172.21.15.165:0/1939849880 >> v2:172.21.15.172:3300/0 conn(0x7f27cc15dd30 0x7f27cc0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134563709 0 0) 0x7f27cc15e110 con 0x7f27cc06bff0 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27b0003520 con 0x7f27cc06bff0 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3675664255 0 0) 0x7f27b0003520 con 0x7f27cc06bff0 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 >> v2:172.21.15.172:3300/0 conn(0x7f27cc15dd30 msgr2=0x7f27cc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 --2- 172.21.15.165:0/1939849880 >> v2:172.21.15.172:3300/0 conn(0x7f27cc15dd30 0x7f27cc0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 >> v1:172.21.15.172:6789/0 conn(0x7f27cc0b5b40 legacy=0x7f27cc0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc06c480 con 0x7f27cc06bff0 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f27b4001670 con 0x7f27cc06bff0 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 >> v1:172.21.15.165:6789/0 conn(0x7f27cc06bff0 legacy=0x7f27cc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.705+0000 7f27c9400640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27b00039b0 0x7f27b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c9400640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c9400640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b00051c0 0x7f27b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc06d000 con 0x7f27b0004800 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b000c060 con 0x7f27b00039b0 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b000c260 con 0x7f27b00051c0 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c3e00640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c9e00640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27b00039b0 0x7f27b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27ca800640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b00051c0 0x7f27b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c3e00640 1 -- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b00051c0 msgr2=0x7f27b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c3e00640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b00051c0 0x7f27b000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c3e00640 1 -- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27b00039b0 msgr2=0x7f27b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.706+0000 7f27c3e00640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27b00039b0 0x7f27b0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.707+0000 7f27c3e00640 1 -- 172.21.15.165:0/1939849880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0003520 con 0x7f27b0004800 2024-04-25T10:03:08.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.707+0000 7f27ca800640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b00051c0 0x7f27b000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:08.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.707+0000 7f27ca800640 1 -- 172.21.15.165:0/1939849880 reap_dead start 2024-04-25T10:03:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.708+0000 7f27c3e00640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27b400c240 tx=0x7f27b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b584a39a5812218b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.708+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4007d10 con 0x7f27b0004800 2024-04-25T10:03:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.708+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27b4005ad0 con 0x7f27b0004800 2024-04-25T10:03:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.708+0000 7f27c9400640 1 -- 172.21.15.165:0/1939849880 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27b4007070 con 0x7f27b0004800 2024-04-25T10:03:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 -- 172.21.15.165:0/1939849880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b0004800 msgr2=0x7f27b0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f27b400c240 tx=0x7f27b0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 -- 172.21.15.165:0/1939849880 shutdown_connections 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 --2- 172.21.15.165:0/1939849880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27b0004800 0x7f27b0004c80 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 -- 172.21.15.165:0/1939849880 >> 172.21.15.165:0/1939849880 conn(0x7f27cc0d0610 msgr2=0x7f27cc0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 -- 172.21.15.165:0/1939849880 shutdown_connections 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.709+0000 7f27cb200640 1 -- 172.21.15.165:0/1939849880 wait complete. 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.711+0000 7f27cb200640 1 Processor -- start 2024-04-25T10:03:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.711+0000 7f27cb200640 1 -- start start 2024-04-25T10:03:08.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27cb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b0004800 0x7f27cc152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27cb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27cc159af0 0x7f27cc159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27cb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 0x7f27cc1566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27cb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc06cd20 con 0x7f27cc156230 2024-04-25T10:03:08.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27cb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc06ca40 con 0x7f27cc159af0 2024-04-25T10:03:08.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27cb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc0b5430 con 0x7f27b0004800 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27ca800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 0x7f27cc1566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27ca800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 0x7f27cc1566e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51450/0 (socket says 172.21.15.165:51450) 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.712+0000 7f27ca800640 1 -- 172.21.15.165:0/2214792925 learned_addr learned my addr 172.21.15.165:0/2214792925 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27c9e00640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b0004800 0x7f27cc152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27c3e00640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27cc159af0 0x7f27cc159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27ca800640 1 -- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b0004800 msgr2=0x7f27cc152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27ca800640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b0004800 0x7f27cc152550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27ca800640 1 -- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27cc159af0 msgr2=0x7f27cc159f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27ca800640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27cc159af0 0x7f27cc159f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27ca800640 1 -- 172.21.15.165:0/2214792925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc153040 con 0x7f27cc156230 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27ca800640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 0x7f27cc1566e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0b6560 tx=0x7f27c400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4018070 con 0x7f27cc156230 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c4003da0 con 0x7f27cc156230 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.713+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c4005330 con 0x7f27cc156230 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.714+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc154480 con 0x7f27cc156230 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.714+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc154a40 con 0x7f27cc156230 2024-04-25T10:03:08.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.715+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc0b57c0 con 0x7f27cc156230 2024-04-25T10:03:08.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.716+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27c401b070 con 0x7f27cc156230 2024-04-25T10:03:08.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.717+0000 7f27c2000640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27a806b4d0 0x7f27a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:08.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.717+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f27c4089cd0 con 0x7f27cc156230 2024-04-25T10:03:08.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.717+0000 7f27c9e00640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27a806b4d0 0x7f27a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.718+0000 7f27c9e00640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27a806b4d0 0x7f27a806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009a90 tx=0x7f27bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.721+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27cc0b57c0 con 0x7f27cc156230 2024-04-25T10:03:08.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.925+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27cc0cc090 con 0x7f27cc156230 2024-04-25T10:03:08.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.926+0000 7f27c2000640 1 -- 172.21.15.165:0/2214792925 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27cc0cc090 con 0x7f27cc156230 2024-04-25T10:03:08.931 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27a806b4d0 msgr2=0x7f27a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27a806b4d0 0x7f27a806d990 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009a90 tx=0x7f27bc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 msgr2=0x7f27cc1566e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 0x7f27cc1566e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0b6560 tx=0x7f27c400d9f0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 shutdown_connections 2024-04-25T10:03:08.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27a806b4d0 0x7f27a806d990 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27cc156230 0x7f27cc1566e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27cc159af0 0x7f27cc159f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 --2- 172.21.15.165:0/2214792925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27b0004800 0x7f27cc152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 >> 172.21.15.165:0/2214792925 conn(0x7f27cc0d0610 msgr2=0x7f27cc0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 shutdown_connections 2024-04-25T10:03:08.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:08.929+0000 7f27cb200640 1 -- 172.21.15.165:0/2214792925 wait complete. 2024-04-25T10:03:08.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:09.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.154+0000 7f289ac00640 1 Processor -- start 2024-04-25T10:03:09.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.154+0000 7f289ac00640 1 -- start start 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.156+0000 7f289ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f28940d0d40 0x7f2894146830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.156+0000 7f289ac00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940b5490 con 0x7f28940b1140 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.156+0000 7f289ac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28940b5730 con 0x7f28940b1fb0 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.156+0000 7f289ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289415ea60 con 0x7f28940d0d40 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.156+0000 7f2899800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f28940b1140 0x7f28940b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36444/0 (socket says 172.21.15.165:36444) 2024-04-25T10:03:09.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.156+0000 7f2899800640 1 -- 172.21.15.165:0/3524981488 learned_addr learned my addr 172.21.15.165:0/3524981488 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f289a200640 1 --2- 172.21.15.165:0/3524981488 >> v2:172.21.15.172:3300/0 conn(0x7f28940d0d40 0x7f2894146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149586532 0 0) 0x7f28940b5490 con 0x7f28940b1140 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f287c003520 con 0x7f28940b1140 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2617801284 0 0) 0x7f287c003520 con 0x7f28940b1140 2024-04-25T10:03:09.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 >> v2:172.21.15.172:3300/0 conn(0x7f28940d0d40 msgr2=0x7f2894146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f2893e00640 1 --2- 172.21.15.165:0/3524981488 >> v2:172.21.15.172:3300/0 conn(0x7f28940d0d40 0x7f2894146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.157+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 >> v1:172.21.15.172:6789/0 conn(0x7f28940b1fb0 legacy=0x7f28940d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28940b5a10 con 0x7f28940b1140 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2888001670 con 0x7f28940b1140 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 >> v1:172.21.15.165:6789/0 conn(0x7f28940b1140 legacy=0x7f28940b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f287c0039b0 0x7f287c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f287c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f287c0052e0 0x7f287c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289415ea60 con 0x7f287c004800 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f287c00c1b0 con 0x7f287c0052e0 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f287c00c3b0 con 0x7f287c0039b0 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2898e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f287c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f289a200640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f287c0052e0 0x7f287c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2898e00640 1 -- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f287c0039b0 msgr2=0x7f287c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2898e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f287c0039b0 0x7f287c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2898e00640 1 -- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f287c0052e0 msgr2=0x7f287c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2898e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f287c0052e0 0x7f287c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f2898e00640 1 -- 172.21.15.165:0/3524981488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f287c003520 con 0x7f287c004800 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f289a200640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f287c0052e0 0x7f287c00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:09.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.158+0000 7f289a200640 1 -- 172.21.15.165:0/3524981488 reap_dead start 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.159+0000 7f2898e00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f287c004c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2880001720 tx=0x7f287c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec1146cf943e6ac3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2880001370 con 0x7f287c004800 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28800052d0 con 0x7f287c004800 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f2893e00640 1 -- 172.21.15.165:0/3524981488 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288002ab90 con 0x7f287c004800 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 -- 172.21.15.165:0/3524981488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 msgr2=0x7f287c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f287c004c80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f2880001720 tx=0x7f287c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 -- 172.21.15.165:0/3524981488 shutdown_connections 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 --2- 172.21.15.165:0/3524981488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f287c004c80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 -- 172.21.15.165:0/3524981488 >> 172.21.15.165:0/3524981488 conn(0x7f28940728b0 msgr2=0x7f2894099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 -- 172.21.15.165:0/3524981488 shutdown_connections 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.160+0000 7f289ac00640 1 -- 172.21.15.165:0/3524981488 wait complete. 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.162+0000 7f289ac00640 1 Processor -- start 2024-04-25T10:03:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.162+0000 7f289ac00640 1 -- start start 2024-04-25T10:03:09.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f289ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f28941f73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f289ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289409d700 0x7f28940ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f289ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894159c90 0x7f289415a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f289ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d07b0 con 0x7f287c004800 2024-04-25T10:03:09.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f289ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d0630 con 0x7f289409d700 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f289ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28940d0930 con 0x7f2894159c90 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.163+0000 7f2899800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f28941f73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f28941f73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51466/0 (socket says 172.21.15.165:51466) 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 -- 172.21.15.165:0/989038650 learned_addr learned my addr 172.21.15.165:0/989038650 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f289a200640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894159c90 0x7f289415a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 -- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894159c90 msgr2=0x7f289415a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894159c90 0x7f289415a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2898e00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289409d700 0x7f28940ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 -- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289409d700 msgr2=0x7f28940ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289409d700 0x7f28940ae8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 -- 172.21.15.165:0/989038650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28941f7910 con 0x7f287c004800 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.164+0000 7f2899800640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f28941f73d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f288800c730 tx=0x7f288800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.165+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2888018070 con 0x7f287c004800 2024-04-25T10:03:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.165+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28880072f0 con 0x7f287c004800 2024-04-25T10:03:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.165+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28880047e0 con 0x7f287c004800 2024-04-25T10:03:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.165+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289409dcc0 con 0x7f287c004800 2024-04-25T10:03:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.165+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289409e280 con 0x7f287c004800 2024-04-25T10:03:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.166+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28940b1610 con 0x7f287c004800 2024-04-25T10:03:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.167+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2888004b90 con 0x7f287c004800 2024-04-25T10:03:09.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.168+0000 7f2891600640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f287406b4d0 0x7f287406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.168+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2888089ad0 con 0x7f287c004800 2024-04-25T10:03:09.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.168+0000 7f2898e00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f287406b4d0 0x7f287406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.169+0000 7f2898e00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f287406b4d0 0x7f287406d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2880032000 tx=0x7f288002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.173+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f28940b1610 con 0x7f287c004800 2024-04-25T10:03:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.367+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f2894073990 con 0x7f287c004800 2024-04-25T10:03:09.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.368+0000 7f2891600640 1 -- 172.21.15.165:0/989038650 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f28941f7910 con 0x7f287c004800 2024-04-25T10:03:09.373 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f287406b4d0 msgr2=0x7f287406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f287406b4d0 0x7f287406d990 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2880032000 tx=0x7f288002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 msgr2=0x7f28941f73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f28941f73d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f288800c730 tx=0x7f288800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 shutdown_connections 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f287406b4d0 0x7f287406d990 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2894159c90 0x7f289415a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f289409d700 0x7f28940ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 --2- 172.21.15.165:0/989038650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f287c004800 0x7f28941f73d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 >> 172.21.15.165:0/989038650 conn(0x7f28940728b0 msgr2=0x7f289406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.371+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 shutdown_connections 2024-04-25T10:03:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.372+0000 7f289ac00640 1 -- 172.21.15.165:0/989038650 wait complete. 2024-04-25T10:03:09.395 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:09.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.593+0000 7f2dd6e00640 1 Processor -- start 2024-04-25T10:03:09.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.594+0000 7f2dd6e00640 1 -- start start 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.595+0000 7f2dd6e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2dd0145140 0x7f2dd00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.595+0000 7f2dd6e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2dd00ce950 0x7f2dd00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.595+0000 7f2dd6e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dd0145520 con 0x7f2dd00c5030 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.595+0000 7f2dd6e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd00ced90 con 0x7f2dd00ce950 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.595+0000 7f2dd6e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dd0071270 con 0x7f2dd0145140 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd5000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2dd00ce950 0x7f2dd00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd6400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2dd00c5030 0x7f2dd006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36452/0 (socket says 172.21.15.165:36452) 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd6400640 1 -- 172.21.15.165:0/56610001 learned_addr learned my addr 172.21.15.165:0/56610001 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd5a00640 1 --2- 172.21.15.165:0/56610001 >> v2:172.21.15.172:3300/0 conn(0x7f2dd0145140 0x7f2dd00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd5000640 1 -- 172.21.15.165:0/56610001 >> v2:172.21.15.172:3300/0 conn(0x7f2dd0145140 msgr2=0x7f2dd00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd5000640 1 --2- 172.21.15.165:0/56610001 >> v2:172.21.15.172:3300/0 conn(0x7f2dd0145140 0x7f2dd00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd5000640 1 -- 172.21.15.165:0/56610001 >> v1:172.21.15.165:6789/0 conn(0x7f2dd00c5030 legacy=0x7f2dd006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dd5000640 1 -- 172.21.15.165:0/56610001 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd0071550 con 0x7f2dd00ce950 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.596+0000 7f2dcfe00640 1 -- 172.21.15.165:0/56610001 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570050307 0 0) 0x7f2dd0145520 con 0x7f2dd00c5030 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.597+0000 7f2dd5000640 1 --2- 172.21.15.165:0/56610001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00ce950 0x7f2dd00c4af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2dd006f690 tx=0x7f2dbc02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e3d9d61655c294 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dcfe00640 1 -- 172.21.15.165:0/56610001 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc037070 con 0x7f2dd00ce950 2024-04-25T10:03:09.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dcfe00640 1 -- 172.21.15.165:0/56610001 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc032440 con 0x7f2dd00ce950 2024-04-25T10:03:09.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dcfe00640 1 -- 172.21.15.165:0/56610001 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc02ae60 con 0x7f2dd00ce950 2024-04-25T10:03:09.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dd6e00640 1 -- 172.21.15.165:0/56610001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00ce950 msgr2=0x7f2dd00c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/56610001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00ce950 0x7f2dd00c4af0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2dd006f690 tx=0x7f2dbc02a110 comp rx=0 tx=0).stop 2024-04-25T10:03:09.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dd6e00640 1 -- 172.21.15.165:0/56610001 shutdown_connections 2024-04-25T10:03:09.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/56610001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00ce950 0x7f2dd00c4af0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/56610001 >> v2:172.21.15.172:3300/0 conn(0x7f2dd0145140 0x7f2dd00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.598+0000 7f2dd6e00640 1 -- 172.21.15.165:0/56610001 >> 172.21.15.165:0/56610001 conn(0x7f2dd0061cc0 msgr2=0x7f2dd006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.599+0000 7f2dd6e00640 1 -- 172.21.15.165:0/56610001 shutdown_connections 2024-04-25T10:03:09.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.599+0000 7f2dd6e00640 1 -- 172.21.15.165:0/56610001 wait complete. 2024-04-25T10:03:09.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.600+0000 7f2dd6e00640 1 Processor -- start 2024-04-25T10:03:09.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.600+0000 7f2dd6e00640 1 -- start start 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.601+0000 7f2dd6e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 0x7f2dd015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.601+0000 7f2dd6e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dd00ce950 0x7f2dd013bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.601+0000 7f2dd6e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dd0145140 0x7f2dd01f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.601+0000 7f2dd6e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00c5480 con 0x7f2dd00c5030 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.601+0000 7f2dd6e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd0071b10 con 0x7f2dd00ce950 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.601+0000 7f2dd6e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dd00c5600 con 0x7f2dd0145140 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 0x7f2dd015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 0x7f2dd015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51488/0 (socket says 172.21.15.165:51488) 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 -- 172.21.15.165:0/3197630935 learned_addr learned my addr 172.21.15.165:0/3197630935 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5000640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dd00ce950 0x7f2dd013bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd6400640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dd0145140 0x7f2dd01f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 -- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dd0145140 msgr2=0x7f2dd01f11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dd0145140 0x7f2dd01f11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 -- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dd00ce950 msgr2=0x7f2dd013bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dd00ce950 0x7f2dd013bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.602+0000 7f2dd5a00640 1 -- 172.21.15.165:0/3197630935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dd013b2e0 con 0x7f2dd00c5030 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dd6400640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dd0145140 0x7f2dd01f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dd5a00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 0x7f2dd015afa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4007e70 tx=0x7f2dc400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4014420 con 0x7f2dd00c5030 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc4019070 con 0x7f2dd00c5030 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dd01f1d00 con 0x7f2dd00c5030 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc4014800 con 0x7f2dd00c5030 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.603+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dd013b880 con 0x7f2dd00c5030 2024-04-25T10:03:09.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.605+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dd01449a0 con 0x7f2dd00c5030 2024-04-25T10:03:09.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.606+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2dc40149a0 con 0x7f2dd00c5030 2024-04-25T10:03:09.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.607+0000 7f2dcd600640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2db006b4d0 0x7f2db006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:09.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.607+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f2dc408ae20 con 0x7f2dd00c5030 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.607+0000 7f2dd5000640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2db006b4d0 0x7f2db006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:09.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.608+0000 7f2dd5000640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2db006b4d0 0x7f2db006d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc034000 tx=0x7f2dbc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:09.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.613+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2dd01449a0 con 0x7f2dd00c5030 2024-04-25T10:03:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.823+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2dd0073990 con 0x7f2dd00c5030 2024-04-25T10:03:09.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.823+0000 7f2dcd600640 1 -- 172.21.15.165:0/3197630935 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2dc4059800 con 0x7f2dd00c5030 2024-04-25T10:03:09.828 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:03:09.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2db006b4d0 msgr2=0x7f2db006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2db006b4d0 0x7f2db006d990 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f2dbc034000 tx=0x7f2dbc02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:09.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 msgr2=0x7f2dd015afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 0x7f2dd015afa0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4007e70 tx=0x7f2dc400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 shutdown_connections 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2db006b4d0 0x7f2db006d990 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2dd0145140 0x7f2dd01f11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2dd00ce950 0x7f2dd013bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 --2- 172.21.15.165:0/3197630935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2dd00c5030 0x7f2dd015afa0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.826+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 >> 172.21.15.165:0/3197630935 conn(0x7f2dd0061cc0 msgr2=0x7f2dd006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.827+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 shutdown_connections 2024-04-25T10:03:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:09.827+0000 7f2dd6e00640 1 -- 172.21.15.165:0/3197630935 wait complete. 2024-04-25T10:03:09.851 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-25T10:03:09.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.851 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-04-25T10:03:09.950 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:03:09.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:09.950 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-25T10:03:10.010 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:10.010 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:10.069 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:10.069 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:10.124 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:10.124 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:10.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:10.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.384+0000 7f5a28c00640 1 Processor -- start 2024-04-25T10:03:10.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.384+0000 7f5a28c00640 1 -- start start 2024-04-25T10:03:10.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.385+0000 7f5a28c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5a240ce920 0x7f5a240cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.385+0000 7f5a28c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.385+0000 7f5a28c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a24144b30 con 0x7f5a24144750 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.385+0000 7f5a28c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a240712d0 con 0x7f5a240ce920 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a28c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a240715b0 con 0x7f5a2406d290 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5a240ce920 0x7f5a240cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5a240ce920 0x7f5a240cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51504/0 (socket says 172.21.15.165:51504) 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 -- 172.21.15.165:0/1183898794 learned_addr learned my addr 172.21.15.165:0/1183898794 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:10.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a22800640 1 --2- 172.21.15.165:0/1183898794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 -- 172.21.15.165:0/1183898794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 msgr2=0x7f5a2406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 --2- 172.21.15.165:0/1183898794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2406d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 -- 172.21.15.165:0/1183898794 >> v1:172.21.15.172:6789/0 conn(0x7f5a24144750 legacy=0x7f5a2406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.386+0000 7f5a1be00640 1 -- 172.21.15.165:0/1183898794 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a24071890 con 0x7f5a240ce920 2024-04-25T10:03:10.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.387+0000 7f5a1be00640 1 --2- 172.21.15.165:0/1183898794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a240cedb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009870 tx=0x7f5a0c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4aac8d99e2821e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.388+0000 7f5a21400640 1 -- 172.21.15.165:0/1183898794 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c037070 con 0x7f5a240ce920 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.388+0000 7f5a21400640 1 -- 172.21.15.165:0/1183898794 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c032440 con 0x7f5a240ce920 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.388+0000 7f5a21400640 1 -- 172.21.15.165:0/1183898794 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c032810 con 0x7f5a240ce920 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.388+0000 7f5a28c00640 1 -- 172.21.15.165:0/1183898794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 msgr2=0x7f5a240cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.388+0000 7f5a28c00640 1 --2- 172.21.15.165:0/1183898794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a240cedb0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c009870 tx=0x7f5a0c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.389+0000 7f5a28c00640 1 -- 172.21.15.165:0/1183898794 shutdown_connections 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.389+0000 7f5a28c00640 1 --2- 172.21.15.165:0/1183898794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.389+0000 7f5a28c00640 1 --2- 172.21.15.165:0/1183898794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a240cedb0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.389+0000 7f5a28c00640 1 -- 172.21.15.165:0/1183898794 >> 172.21.15.165:0/1183898794 conn(0x7f5a2406abe0 msgr2=0x7f5a2406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.389+0000 7f5a28c00640 1 -- 172.21.15.165:0/1183898794 shutdown_connections 2024-04-25T10:03:10.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.389+0000 7f5a28c00640 1 -- 172.21.15.165:0/1183898794 wait complete. 2024-04-25T10:03:10.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.390+0000 7f5a28c00640 1 Processor -- start 2024-04-25T10:03:10.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.391+0000 7f5a28c00640 1 -- start start 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.391+0000 7f5a28c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2413e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.391+0000 7f5a28c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a24141d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.391+0000 7f5a28c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a24144750 0x7f5a24142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a28c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2406f020 con 0x7f5a240ce920 2024-04-25T10:03:10.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a28c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2406ece0 con 0x7f5a24144750 2024-04-25T10:03:10.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a28c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a24145450 con 0x7f5a2406d290 2024-04-25T10:03:10.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a24141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a24141d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51520/0 (socket says 172.21.15.165:51520) 2024-04-25T10:03:10.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 -- 172.21.15.165:0/658061446 learned_addr learned my addr 172.21.15.165:0/658061446 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a21e00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2413e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a22800640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a24144750 0x7f5a24142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 -- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 msgr2=0x7f5a2413e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2413e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 -- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a24144750 msgr2=0x7f5a24142280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a24144750 0x7f5a24142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.392+0000 7f5a1be00640 1 -- 172.21.15.165:0/658061446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a2413e5c0 con 0x7f5a240ce920 2024-04-25T10:03:10.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.393+0000 7f5a1be00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a24141d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c000c00 tx=0x7f5a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.393+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c041020 con 0x7f5a240ce920 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.393+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a0c0306c0 con 0x7f5a240ce920 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.393+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a0c037040 con 0x7f5a240ce920 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.393+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a241fbc20 con 0x7f5a240ce920 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.393+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a241fc1e0 con 0x7f5a240ce920 2024-04-25T10:03:10.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.394+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a24073990 con 0x7f5a240ce920 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.395+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a0c044020 con 0x7f5a240ce920 2024-04-25T10:03:10.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.396+0000 7f5a1a000640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0006b4d0 0x7f5a0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.396+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f5a0c0aae10 con 0x7f5a240ce920 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.396+0000 7f5a21e00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0006b4d0 0x7f5a0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.397+0000 7f5a21e00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0006b4d0 0x7f5a0006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a14003d90 tx=0x7f5a14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.401+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a0c0ae030 con 0x7f5a240ce920 2024-04-25T10:03:10.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.600+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5a240d0510 con 0x7f5a240ce920 2024-04-25T10:03:10.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.601+0000 7f5a1a000640 1 -- 172.21.15.165:0/658061446 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f5a0c04b030 con 0x7f5a240ce920 2024-04-25T10:03:10.606 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:10.607 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0006b4d0 msgr2=0x7f5a0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0006b4d0 0x7f5a0006d990 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f5a14003d90 tx=0x7f5a14009040 comp rx=0 tx=0).stop 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 msgr2=0x7f5a24141d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a24141d40 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f5a0c000c00 tx=0x7f5a0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 shutdown_connections 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0006b4d0 0x7f5a0006d990 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a24144750 0x7f5a24142280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a240ce920 0x7f5a24141d40 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 --2- 172.21.15.165:0/658061446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a2406d290 0x7f5a2413e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 >> 172.21.15.165:0/658061446 conn(0x7f5a2406abe0 msgr2=0x7f5a2406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 shutdown_connections 2024-04-25T10:03:10.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.604+0000 7f5a28c00640 1 -- 172.21.15.165:0/658061446 wait complete. 2024-04-25T10:03:10.628 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:10.628 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:10.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.830+0000 7f2647e00640 1 Processor -- start 2024-04-25T10:03:10.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.830+0000 7f2647e00640 1 -- start start 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f264813d9f0 0x7f264813df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26480b0730 0x7f26480b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f264806d390 con 0x7f264806e150 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264806d670 con 0x7f264813d9f0 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f264806d950 con 0x7f26480b0730 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2646a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f264813d9f0 0x7f264813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f264806e150 0x7f264806e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36460/0 (socket says 172.21.15.165:36460) 2024-04-25T10:03:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.832+0000 7f2647400640 1 -- 172.21.15.165:0/4039359392 learned_addr learned my addr 172.21.15.165:0/4039359392 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.833+0000 7f263fe00640 1 --2- 172.21.15.165:0/4039359392 >> v2:172.21.15.172:3300/0 conn(0x7f26480b0730 0x7f26480b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.833+0000 7f2646a00640 1 -- 172.21.15.165:0/4039359392 >> v2:172.21.15.172:3300/0 conn(0x7f26480b0730 msgr2=0x7f26480b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.833+0000 7f2646a00640 1 --2- 172.21.15.165:0/4039359392 >> v2:172.21.15.172:3300/0 conn(0x7f26480b0730 0x7f26480b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.833+0000 7f2646a00640 1 -- 172.21.15.165:0/4039359392 >> v1:172.21.15.165:6789/0 conn(0x7f264806e150 legacy=0x7f264806e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.833+0000 7f2646a00640 1 -- 172.21.15.165:0/4039359392 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f264806dc30 con 0x7f264813d9f0 2024-04-25T10:03:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.833+0000 7f2646000640 1 -- 172.21.15.165:0/4039359392 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215999909 0 0) 0x7f264806d390 con 0x7f264806e150 2024-04-25T10:03:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.834+0000 7f2646a00640 1 --2- 172.21.15.165:0/4039359392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f264813d9f0 0x7f264813df30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f264806ef50 tx=0x7f263802a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56b2cc850e588cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.834+0000 7f2646000640 1 -- 172.21.15.165:0/4039359392 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638037070 con 0x7f264813d9f0 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.834+0000 7f2646000640 1 -- 172.21.15.165:0/4039359392 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f263802add0 con 0x7f264813d9f0 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.834+0000 7f2646000640 1 -- 172.21.15.165:0/4039359392 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2638032660 con 0x7f264813d9f0 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 -- 172.21.15.165:0/4039359392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f264813d9f0 msgr2=0x7f264813df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 --2- 172.21.15.165:0/4039359392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f264813d9f0 0x7f264813df30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f264806ef50 tx=0x7f263802a080 comp rx=0 tx=0).stop 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 -- 172.21.15.165:0/4039359392 shutdown_connections 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 --2- 172.21.15.165:0/4039359392 >> v2:172.21.15.172:3300/0 conn(0x7f26480b0730 0x7f26480b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 --2- 172.21.15.165:0/4039359392 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f264813d9f0 0x7f264813df30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 -- 172.21.15.165:0/4039359392 >> 172.21.15.165:0/4039359392 conn(0x7f2648061cc0 msgr2=0x7f264806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 -- 172.21.15.165:0/4039359392 shutdown_connections 2024-04-25T10:03:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.835+0000 7f2647e00640 1 -- 172.21.15.165:0/4039359392 wait complete. 2024-04-25T10:03:10.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.837+0000 7f2647e00640 1 Processor -- start 2024-04-25T10:03:10.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.837+0000 7f2647e00640 1 -- start start 2024-04-25T10:03:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2647e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f264806e150 0x7f26481e3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2647e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 0x7f26481f3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2647e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f264813d9f0 0x7f2648139bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2647e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26480ce880 con 0x7f26480b0730 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2647e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f264813e680 con 0x7f264806e150 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2647e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26480cea00 con 0x7f264813d9f0 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 0x7f26481f3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 0x7f26481f3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51546/0 (socket says 172.21.15.165:51546) 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 -- 172.21.15.165:0/4141640445 learned_addr learned my addr 172.21.15.165:0/4141640445 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f2646a00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f264806e150 0x7f26481e3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 -- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f264813d9f0 msgr2=0x7f2648139bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f264813d9f0 0x7f2648139bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 -- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f264806e150 msgr2=0x7f26481e3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f264806e150 0x7f26481e3c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.838+0000 7f263fe00640 1 -- 172.21.15.165:0/4141640445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26480b2500 con 0x7f26480b0730 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.839+0000 7f263fe00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 0x7f26481f3360 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f263000d800 tx=0x7f263000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.839+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000b6c0 con 0x7f26480b0730 2024-04-25T10:03:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.839+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2630019070 con 0x7f26480b0730 2024-04-25T10:03:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.839+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26480b27a0 con 0x7f26480b0730 2024-04-25T10:03:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.839+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f263000ba80 con 0x7f26480b0730 2024-04-25T10:03:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.839+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26480b2d60 con 0x7f26480b0730 2024-04-25T10:03:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.840+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2648073990 con 0x7f26480b0730 2024-04-25T10:03:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.841+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f263000bc20 con 0x7f26480b0730 2024-04-25T10:03:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.842+0000 7f263ea00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f262406b4d0 0x7f262406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.842+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f263008ab50 con 0x7f26480b0730 2024-04-25T10:03:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.842+0000 7f2646a00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f262406b4d0 0x7f262406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.843+0000 7f2646a00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f262406b4d0 0x7f262406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2638034000 tx=0x7f263802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:10.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:10.847+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2630059530 con 0x7f26480b0730 2024-04-25T10:03:11.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.043+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f264813f2e0 con 0x7f26480b0730 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.044+0000 7f263ea00640 1 -- 172.21.15.165:0/4141640445 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2630058e30 con 0x7f26480b0730 2024-04-25T10:03:11.048 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f262406b4d0 msgr2=0x7f262406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f262406b4d0 0x7f262406d990 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f2638034000 tx=0x7f263802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 msgr2=0x7f26481f3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 0x7f26481f3360 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f263000d800 tx=0x7f263000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 shutdown_connections 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f262406b4d0 0x7f262406d990 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f264813d9f0 0x7f2648139bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26480b0730 0x7f26481f3360 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 --2- 172.21.15.165:0/4141640445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f264806e150 0x7f26481e3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 >> 172.21.15.165:0/4141640445 conn(0x7f2648061cc0 msgr2=0x7f264813c750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.046+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 shutdown_connections 2024-04-25T10:03:11.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.047+0000 7f2647e00640 1 -- 172.21.15.165:0/4141640445 wait complete. 2024-04-25T10:03:11.070 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:11.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.266+0000 7fae1e800640 1 Processor -- start 2024-04-25T10:03:11.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.266+0000 7fae1e800640 1 -- start start 2024-04-25T10:03:11.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1e800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fae180ce950 0x7fae180c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1e800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae18145520 con 0x7fae180c5030 2024-04-25T10:03:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1e800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae180ced90 con 0x7fae180ce950 2024-04-25T10:03:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae18071270 con 0x7fae18145140 2024-04-25T10:03:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fae180ce950 0x7fae180c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fae180ce950 0x7fae180c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51554/0 (socket says 172.21.15.165:51554) 2024-04-25T10:03:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1ca00640 1 -- 172.21.15.165:0/1991896321 learned_addr learned my addr 172.21.15.165:0/1991896321 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:11.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1d400640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1ca00640 1 -- 172.21.15.165:0/1991896321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 msgr2=0x7fae180ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1ca00640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.268+0000 7fae1ca00640 1 -- 172.21.15.165:0/1991896321 >> v1:172.21.15.172:6789/0 conn(0x7fae180c5030 legacy=0x7fae1806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.269+0000 7fae1ca00640 1 -- 172.21.15.165:0/1991896321 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae18071550 con 0x7fae180ce950 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.269+0000 7fae1d400640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.270+0000 7fae1ca00640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180c4af0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae04009870 tx=0x7fae0402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14121997108ba6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.270+0000 7fae17e00640 1 -- 172.21.15.165:0/1991896321 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04037070 con 0x7fae180ce950 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.270+0000 7fae17e00640 1 -- 172.21.15.165:0/1991896321 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04032440 con 0x7fae180ce950 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.270+0000 7fae17e00640 1 -- 172.21.15.165:0/1991896321 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04032810 con 0x7fae180ce950 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 -- 172.21.15.165:0/1991896321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 msgr2=0x7fae180c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180c4af0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fae04009870 tx=0x7fae0402a180 comp rx=0 tx=0).stop 2024-04-25T10:03:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 -- 172.21.15.165:0/1991896321 shutdown_connections 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180c4af0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 --2- 172.21.15.165:0/1991896321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 -- 172.21.15.165:0/1991896321 >> 172.21.15.165:0/1991896321 conn(0x7fae18061cc0 msgr2=0x7fae1806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 -- 172.21.15.165:0/1991896321 shutdown_connections 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.271+0000 7fae1e800640 1 -- 172.21.15.165:0/1991896321 wait complete. 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.272+0000 7fae1e800640 1 Processor -- start 2024-04-25T10:03:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.273+0000 7fae1e800640 1 -- start start 2024-04-25T10:03:11.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae180c5030 0x7fae18143440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180cae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c54a0 con 0x7fae180ce950 2024-04-25T10:03:11.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18071b10 con 0x7fae180c5030 2024-04-25T10:03:11.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae180c5620 con 0x7fae18145140 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51558/0 (socket says 172.21.15.165:51558) 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 -- 172.21.15.165:0/4025832621 learned_addr learned my addr 172.21.15.165:0/4025832621 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 -- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 msgr2=0x7fae180cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180cae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1d400640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae180c5030 0x7fae18143440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 -- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae180c5030 msgr2=0x7fae18143440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae180c5030 0x7fae18143440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1ca00640 1 -- 172.21.15.165:0/4025832621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae180a85d0 con 0x7fae180ce950 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.274+0000 7fae1d400640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae180c5030 0x7fae18143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.275+0000 7fae1ca00640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180acd40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae04009870 tx=0x7fae040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.275+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04041020 con 0x7fae180ce950 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.275+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae04030970 con 0x7fae180ce950 2024-04-25T10:03:11.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.275+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae04037040 con 0x7fae180ce950 2024-04-25T10:03:11.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.275+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae180a88b0 con 0x7fae180ce950 2024-04-25T10:03:11.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.275+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae180a8e70 con 0x7fae180ce950 2024-04-25T10:03:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.276+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae18073990 con 0x7fae180ce950 2024-04-25T10:03:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.277+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae04044020 con 0x7fae180ce950 2024-04-25T10:03:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.278+0000 7fae15600640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fadf806b4d0 0x7fadf806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.278+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fae040ab060 con 0x7fae180ce950 2024-04-25T10:03:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.278+0000 7fae1d400640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fadf806b4d0 0x7fadf806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.279+0000 7fae1d400640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fadf806b4d0 0x7fadf806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae0c006fd0 tx=0x7fae0c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.282+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae0404b030 con 0x7fae180ce950 2024-04-25T10:03:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.478+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fae18144630 con 0x7fae180ce950 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.479+0000 7fae15600640 1 -- 172.21.15.165:0/4025832621 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fae04079ad0 con 0x7fae180ce950 2024-04-25T10:03:11.484 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fadf806b4d0 msgr2=0x7fadf806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fadf806b4d0 0x7fadf806d990 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fae0c006fd0 tx=0x7fae0c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 msgr2=0x7fae180acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180acd40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fae04009870 tx=0x7fae040046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 shutdown_connections 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fadf806b4d0 0x7fadf806d990 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae18145140 0x7fae180cae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae180ce950 0x7fae180acd40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 --2- 172.21.15.165:0/4025832621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae180c5030 0x7fae18143440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 >> 172.21.15.165:0/4025832621 conn(0x7fae18061cc0 msgr2=0x7fae1815c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 shutdown_connections 2024-04-25T10:03:11.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.482+0000 7fae1e800640 1 -- 172.21.15.165:0/4025832621 wait complete. 2024-04-25T10:03:11.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:11.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.709+0000 7fa532e00640 1 Processor -- start 2024-04-25T10:03:11.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.709+0000 7fa532e00640 1 -- start start 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa532e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa52c145140 0x7fa52c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa532e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 0x7fa52c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa532e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52c0c5650 con 0x7fa52c0ce950 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa532e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52c0ced90 con 0x7fa52c145140 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa532e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c145520 con 0x7fa52c0c5200 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa531000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa52c0ce950 0x7fa52c0c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36464/0 (socket says 172.21.15.165:36464) 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa531000640 1 -- 172.21.15.165:0/1110534215 learned_addr learned my addr 172.21.15.165:0/1110534215 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa532400640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 0x7fa52c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.711+0000 7fa531a00640 1 --2- 172.21.15.165:0/1110534215 >> v2:172.21.15.172:3300/0 conn(0x7fa52c145140 0x7fa52c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 91750436 0 0) 0x7fa52c0c5650 con 0x7fa52c0ce950 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa514003520 con 0x7fa52c0ce950 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1520083806 0 0) 0x7fa514003520 con 0x7fa52c0ce950 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 msgr2=0x7fa52c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 0x7fa52c15bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 >> v2:172.21.15.172:3300/0 conn(0x7fa52c145140 msgr2=0x7fa52c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 --2- 172.21.15.165:0/1110534215 >> v2:172.21.15.172:3300/0 conn(0x7fa52c145140 0x7fa52c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c071270 con 0x7fa52c0ce950 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa518001670 con 0x7fa52c0ce950 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 >> v1:172.21.15.165:6789/0 conn(0x7fa52c0ce950 legacy=0x7fa52c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 0x7fa5140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa514003ef0 0x7fa514004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514004d40 0x7fa5140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c0ced90 con 0x7fa514003ef0 2024-04-25T10:03:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c145520 con 0x7fa514004d40 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.712+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa51400b9c0 con 0x7fa52c0c5200 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.713+0000 7fa531000640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa514003ef0 0x7fa514004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.713+0000 7fa531000640 1 -- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 msgr2=0x7fa5140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.713+0000 7fa531000640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c5200 0x7fa5140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.713+0000 7fa531000640 1 -- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514004d40 msgr2=0x7fa5140051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.713+0000 7fa531000640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514004d40 0x7fa5140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.713+0000 7fa531000640 1 -- 172.21.15.165:0/1110534215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa514003520 con 0x7fa514003ef0 2024-04-25T10:03:11.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.714+0000 7fa531000640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa514003ef0 0x7fa514004800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa518005010 tx=0x7fa514005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373b53eb53188087 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.714+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518004be0 con 0x7fa514003ef0 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.714+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa51802f030 con 0x7fa514003ef0 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.714+0000 7fa52be00640 1 -- 172.21.15.165:0/1110534215 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518005cf0 con 0x7fa514003ef0 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532e00640 1 -- 172.21.15.165:0/1110534215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa514003ef0 msgr2=0x7fa514004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532e00640 1 --2- 172.21.15.165:0/1110534215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa514003ef0 0x7fa514004800 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fa518005010 tx=0x7fa514005830 comp rx=0 tx=0).stop 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532400640 1 -- 172.21.15.165:0/1110534215 reap_dead start 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532e00640 1 -- 172.21.15.165:0/1110534215 shutdown_connections 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532e00640 1 -- 172.21.15.165:0/1110534215 >> 172.21.15.165:0/1110534215 conn(0x7fa52c061cc0 msgr2=0x7fa52c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532e00640 1 -- 172.21.15.165:0/1110534215 shutdown_connections 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.715+0000 7fa532e00640 1 -- 172.21.15.165:0/1110534215 wait complete. 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.717+0000 7fa532e00640 1 Processor -- start 2024-04-25T10:03:11.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.717+0000 7fa532e00640 1 -- start start 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa532e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514003ef0 0x7fa52c138210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa532e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 0x7fa52c140750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa532e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c7b40 0x7fa52c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa532e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c071b10 con 0x7fa52c1402d0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa532e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c071830 con 0x7fa514003ef0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa532e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52c15bfb0 con 0x7fa52c0c7b40 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa531000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 0x7fa52c140750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa531000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 0x7fa52c140750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51580/0 (socket says 172.21.15.165:51580) 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa531000640 1 -- 172.21.15.165:0/3292597963 learned_addr learned my addr 172.21.15.165:0/3292597963 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.718+0000 7fa531a00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514003ef0 0x7fa52c138210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa531000640 1 -- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c7b40 msgr2=0x7fa52c0c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa531000640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c7b40 0x7fa52c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa531000640 1 -- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514003ef0 msgr2=0x7fa52c138210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa531000640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514003ef0 0x7fa52c138210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa531000640 1 -- 172.21.15.165:0/3292597963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52c0c9950 con 0x7fa52c1402d0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa531000640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 0x7fa52c140750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa518005130 tx=0x7fa51802d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518004be0 con 0x7fa52c1402d0 2024-04-25T10:03:11.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5180058b0 con 0x7fa52c1402d0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa518005c80 con 0x7fa52c1402d0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa52c0c88b0 con 0x7fa52c1402d0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.719+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa52c0c8e70 con 0x7fa52c1402d0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.720+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa52c073990 con 0x7fa52c1402d0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.721+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa51802b780 con 0x7fa52c1402d0 2024-04-25T10:03:11.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.722+0000 7fa529600640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:11.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.722+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa5180a9fe0 con 0x7fa52c1402d0 2024-04-25T10:03:11.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.723+0000 7fa531a00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:11.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.723+0000 7fa531a00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50c06b4d0 0x7fa50c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa5200073f0 tx=0x7fa52000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:11.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.727+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa518005e20 con 0x7fa52c1402d0 2024-04-25T10:03:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.912+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa52c15eca0 con 0x7fa52c1402d0 2024-04-25T10:03:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.913+0000 7fa529600640 1 -- 172.21.15.165:0/3292597963 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa5180789e0 con 0x7fa52c1402d0 2024-04-25T10:03:11.917 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50c06b4d0 msgr2=0x7fa50c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50c06b4d0 0x7fa50c06d990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa5200073f0 tx=0x7fa52000b040 comp rx=0 tx=0).stop 2024-04-25T10:03:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 msgr2=0x7fa52c140750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 0x7fa52c140750 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa518005130 tx=0x7fa51802d070 comp rx=0 tx=0).stop 2024-04-25T10:03:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 shutdown_connections 2024-04-25T10:03:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50c06b4d0 0x7fa50c06d990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52c0c7b40 0x7fa52c0c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52c1402d0 0x7fa52c140750 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.915+0000 7fa532e00640 1 --2- 172.21.15.165:0/3292597963 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa514003ef0 0x7fa52c138210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.916+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 >> 172.21.15.165:0/3292597963 conn(0x7fa52c061cc0 msgr2=0x7fa52c13fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.916+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 shutdown_connections 2024-04-25T10:03:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:11.916+0000 7fa532e00640 1 -- 172.21.15.165:0/3292597963 wait complete. 2024-04-25T10:03:11.938 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:12.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.127+0000 7fecacc00640 1 Processor -- start 2024-04-25T10:03:12.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.127+0000 7fecacc00640 1 -- start start 2024-04-25T10:03:12.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.129+0000 7fecacc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7feca8144d50 0x7feca806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.129+0000 7fecacc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80cccc0 con 0x7feca81452a0 2024-04-25T10:03:12.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.129+0000 7fecacc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feca80cc140 con 0x7feca81446d0 2024-04-25T10:03:12.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.129+0000 7fecacc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feca80b2b20 con 0x7feca8144d50 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca6800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7feca81452a0 0x7feca806ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36468/0 (socket says 172.21.15.165:36468) 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca6800640 1 -- 172.21.15.165:0/2057683945 learned_addr learned my addr 172.21.15.165:0/2057683945 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca5400640 1 --2- 172.21.15.165:0/2057683945 >> v2:172.21.15.165:3300/0 conn(0x7feca8144d50 0x7feca806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca5400640 1 -- 172.21.15.165:0/2057683945 >> v1:172.21.15.172:6789/0 conn(0x7feca81446d0 legacy=0x7feca80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca5400640 1 -- 172.21.15.165:0/2057683945 >> v1:172.21.15.165:6789/0 conn(0x7feca81452a0 legacy=0x7feca806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca5400640 1 -- 172.21.15.165:0/2057683945 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80cc420 con 0x7feca8144d50 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.130+0000 7feca4a00640 1 -- 172.21.15.165:0/2057683945 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2259337612 0 0) 0x7feca80cccc0 con 0x7feca81452a0 2024-04-25T10:03:12.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.131+0000 7feca5400640 1 --2- 172.21.15.165:0/2057683945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca8144d50 0x7feca806d890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec94009d90 tx=0x7fec9402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697c6211a4250201 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7feca4a00640 1 -- 172.21.15.165:0/2057683945 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94007740 con 0x7feca8144d50 2024-04-25T10:03:12.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7feca4a00640 1 -- 172.21.15.165:0/2057683945 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec940078e0 con 0x7feca8144d50 2024-04-25T10:03:12.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7feca4a00640 1 -- 172.21.15.165:0/2057683945 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec94007cb0 con 0x7feca8144d50 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7fecacc00640 1 -- 172.21.15.165:0/2057683945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca8144d50 msgr2=0x7feca806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7fecacc00640 1 --2- 172.21.15.165:0/2057683945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca8144d50 0x7feca806d890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fec94009d90 tx=0x7fec9402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7fecacc00640 1 -- 172.21.15.165:0/2057683945 shutdown_connections 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7fecacc00640 1 --2- 172.21.15.165:0/2057683945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca8144d50 0x7feca806d890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.132+0000 7fecacc00640 1 -- 172.21.15.165:0/2057683945 >> 172.21.15.165:0/2057683945 conn(0x7feca806abe0 msgr2=0x7feca806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.133+0000 7fecacc00640 1 -- 172.21.15.165:0/2057683945 shutdown_connections 2024-04-25T10:03:12.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.133+0000 7fecacc00640 1 -- 172.21.15.165:0/2057683945 wait complete. 2024-04-25T10:03:12.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.134+0000 7fecacc00640 1 Processor -- start 2024-04-25T10:03:12.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.134+0000 7fecacc00640 1 -- start start 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.135+0000 7fecacc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 0x7feca80c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.135+0000 7fecacc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8144d50 0x7feca80c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.135+0000 7fecacc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca81452a0 0x7feca80b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.135+0000 7fecacc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca80cc9e0 con 0x7feca81446d0 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.135+0000 7fecacc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feca806d400 con 0x7feca8144d50 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.135+0000 7fecacc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feca8144ab0 con 0x7feca81452a0 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 0x7feca80c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 0x7feca80c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51600/0 (socket says 172.21.15.165:51600) 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 -- 172.21.15.165:0/3258487713 learned_addr learned my addr 172.21.15.165:0/3258487713 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5400640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8144d50 0x7feca80c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca6800640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca81452a0 0x7feca80b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 -- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca81452a0 msgr2=0x7feca80b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca81452a0 0x7feca80b17e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 -- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8144d50 msgr2=0x7feca80c6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8144d50 0x7feca80c6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 -- 172.21.15.165:0/3258487713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feca80c50b0 con 0x7feca81446d0 2024-04-25T10:03:12.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5400640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8144d50 0x7feca80c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:12.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.136+0000 7feca5e00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 0x7feca80c46d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fec9800b420 tx=0x7fec9800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.137+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98011830 con 0x7feca81446d0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.137+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec98016070 con 0x7feca81446d0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.137+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feca80c5390 con 0x7feca81446d0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.137+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec98011c10 con 0x7feca81446d0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.137+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feca80c5950 con 0x7feca81446d0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.138+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feca8073a00 con 0x7feca81446d0 2024-04-25T10:03:12.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.139+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fec9801b460 con 0x7feca81446d0 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.140+0000 7fec9e000640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8806b4d0 0x7fec8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.140+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fec9808a180 con 0x7feca81446d0 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.140+0000 7feca5400640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8806b4d0 0x7fec8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.141+0000 7feca5400640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8806b4d0 0x7fec8806d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec94034000 tx=0x7fec9402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.145+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fec98020020 con 0x7feca81446d0 2024-04-25T10:03:12.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.343+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7feca8071710 con 0x7feca81446d0 2024-04-25T10:03:12.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.343+0000 7fec9e000640 1 -- 172.21.15.165:0/3258487713 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7fec98058b60 con 0x7feca81446d0 2024-04-25T10:03:12.348 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8806b4d0 msgr2=0x7fec8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8806b4d0 0x7fec8806d990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fec94034000 tx=0x7fec9402f000 comp rx=0 tx=0).stop 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 msgr2=0x7feca80c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 0x7feca80c46d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fec9800b420 tx=0x7fec9800b8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 shutdown_connections 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8806b4d0 0x7fec8806d990 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feca81452a0 0x7feca80b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feca8144d50 0x7feca80c6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 --2- 172.21.15.165:0/3258487713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feca81446d0 0x7feca80c46d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 >> 172.21.15.165:0/3258487713 conn(0x7feca806abe0 msgr2=0x7feca8143860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 shutdown_connections 2024-04-25T10:03:12.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.346+0000 7fecacc00640 1 -- 172.21.15.165:0/3258487713 wait complete. 2024-04-25T10:03:12.368 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:12.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.560+0000 7f1e2a600640 1 Processor -- start 2024-04-25T10:03:12.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.561+0000 7f1e2a600640 1 -- start start 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.562+0000 7f1e2a600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e240b1140 0x7f1e240b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.562+0000 7f1e2a600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e240b5490 con 0x7f1e240b1fb0 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.562+0000 7f1e2a600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e240b5730 con 0x7f1e240d0d40 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.562+0000 7f1e2a600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2415ea60 con 0x7f1e240b1140 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.562+0000 7f1e23e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1e240b1fb0 0x7f1e240d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36470/0 (socket says 172.21.15.165:36470) 2024-04-25T10:03:12.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23e00640 1 -- 172.21.15.165:0/3337565081 learned_addr learned my addr 172.21.15.165:0/3337565081 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e29200640 1 --2- 172.21.15.165:0/3337565081 >> v2:172.21.15.172:3300/0 conn(0x7f1e240b1140 0x7f1e240b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2494600558 0 0) 0x7f1e240b5490 con 0x7f1e240b1fb0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c003520 con 0x7f1e240b1fb0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753395071 0 0) 0x7f1e0c003520 con 0x7f1e240b1fb0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 >> v2:172.21.15.172:3300/0 conn(0x7f1e240b1140 msgr2=0x7f1e240b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 --2- 172.21.15.165:0/3337565081 >> v2:172.21.15.172:3300/0 conn(0x7f1e240b1140 0x7f1e240b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 >> v1:172.21.15.172:6789/0 conn(0x7f1e240d0d40 legacy=0x7f1e24146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240b5a10 con 0x7f1e240b1fb0 2024-04-25T10:03:12.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1e10001670 con 0x7f1e240b1fb0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 >> v1:172.21.15.165:6789/0 conn(0x7f1e240b1fb0 legacy=0x7f1e240d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.563+0000 7f1e23400640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e23400640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e0c004800 0x7f1e0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e23400640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e0c0052e0 0x7f1e0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2415ea60 con 0x7f1e0c0039b0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c00c1b0 con 0x7f1e0c0052e0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c00c3b0 con 0x7f1e0c004800 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29200640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29200640 1 -- 172.21.15.165:0/3337565081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e0c004800 msgr2=0x7f1e0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29200640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e0c004800 0x7f1e0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29200640 1 -- 172.21.15.165:0/3337565081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e0c0052e0 msgr2=0x7f1e0c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:12.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29200640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e0c0052e0 0x7f1e0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29200640 1 -- 172.21.15.165:0/3337565081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c003520 con 0x7f1e0c0039b0 2024-04-25T10:03:12.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.564+0000 7f1e29c00640 1 -- 172.21.15.165:0/3337565081 reap_dead start 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.565+0000 7f1e29200640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009a90 tx=0x7f1e0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a400da8dd81ca6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.565+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18036070 con 0x7f1e0c0039b0 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.565+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e18031400 con 0x7f1e0c0039b0 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.565+0000 7f1e23400640 1 -- 172.21.15.165:0/3337565081 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18009690 con 0x7f1e0c0039b0 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 -- 172.21.15.165:0/3337565081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 msgr2=0x7f1e0c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009a90 tx=0x7f1e0c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 -- 172.21.15.165:0/3337565081 shutdown_connections 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 --2- 172.21.15.165:0/3337565081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e0c003dd0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 -- 172.21.15.165:0/3337565081 >> 172.21.15.165:0/3337565081 conn(0x7f1e240728b0 msgr2=0x7f1e24099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 -- 172.21.15.165:0/3337565081 shutdown_connections 2024-04-25T10:03:12.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.566+0000 7f1e2a600640 1 -- 172.21.15.165:0/3337565081 wait complete. 2024-04-25T10:03:12.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.568+0000 7f1e2a600640 1 Processor -- start 2024-04-25T10:03:12.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.568+0000 7f1e2a600640 1 -- start start 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e2a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e24155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e2a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e240afbe0 0x7f1e24157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e2a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e240adde0 0x7f1e240ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e2a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240d07b0 con 0x7f1e0c0039b0 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e2a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240d0630 con 0x7f1e240adde0 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e2a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240d0930 con 0x7f1e240afbe0 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e29200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e24155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e29200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e24155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51622/0 (socket says 172.21.15.165:51622) 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.569+0000 7f1e29200640 1 -- 172.21.15.165:0/1924540990 learned_addr learned my addr 172.21.15.165:0/1924540990 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e23e00640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e240afbe0 0x7f1e24157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e29200640 1 -- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e240afbe0 msgr2=0x7f1e24157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e29200640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e240afbe0 0x7f1e24157e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e29200640 1 -- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e240adde0 msgr2=0x7f1e240ae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e29200640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e240adde0 0x7f1e240ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e29200640 1 -- 172.21.15.165:0/1924540990 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240ad600 con 0x7f1e0c0039b0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e29200640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e24155c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009bb0 tx=0x7f1e18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1803f020 con 0x7f1e0c0039b0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e18037040 con 0x7f1e0c0039b0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e18036070 con 0x7f1e0c0039b0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e241564c0 con 0x7f1e0c0039b0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.570+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e24156a80 con 0x7f1e0c0039b0 2024-04-25T10:03:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.571+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e24073990 con 0x7f1e0c0039b0 2024-04-25T10:03:12.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.572+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e18042020 con 0x7f1e0c0039b0 2024-04-25T10:03:12.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.573+0000 7f1e20c00640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:12.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.573+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f1e180aa6f0 con 0x7f1e0c0039b0 2024-04-25T10:03:12.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.574+0000 7f1e23e00640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:12.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.574+0000 7f1e23e00640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e0406b4d0 0x7f1e0406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f60 tx=0x7f1e10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.578+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e18049030 con 0x7f1e0c0039b0 2024-04-25T10:03:12.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.786+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e2406fa90 con 0x7f1e0c0039b0 2024-04-25T10:03:12.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.787+0000 7f1e20c00640 1 -- 172.21.15.165:0/1924540990 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1e18079040 con 0x7f1e0c0039b0 2024-04-25T10:03:12.791 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e0406b4d0 msgr2=0x7f1e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e0406b4d0 0x7f1e0406d990 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1e10000f60 tx=0x7f1e10009040 comp rx=0 tx=0).stop 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 msgr2=0x7f1e24155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e24155c50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f1e18009bb0 tx=0x7f1e18004730 comp rx=0 tx=0).stop 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 shutdown_connections 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e0406b4d0 0x7f1e0406d990 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e240adde0 0x7f1e240ae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e240afbe0 0x7f1e24157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 --2- 172.21.15.165:0/1924540990 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e0c0039b0 0x7f1e24155c50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:12.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 >> 172.21.15.165:0/1924540990 conn(0x7f1e240728b0 msgr2=0x7f1e2406e780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:12.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 shutdown_connections 2024-04-25T10:03:12.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:12.790+0000 7f1e2a600640 1 -- 172.21.15.165:0/1924540990 wait complete. 2024-04-25T10:03:12.811 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-04-25T10:03:12.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:12.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-04-25T10:03:12.879 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:03:12.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:12.879 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-04-25T10:03:12.917 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:03:12.917 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:12.974 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:03:12.974 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:13.027 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:03:13.027 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:13.083 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:13.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.275+0000 7eff61200640 1 Processor -- start 2024-04-25T10:03:13.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.275+0000 7eff61200640 1 -- start start 2024-04-25T10:03:13.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff61200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7eff5c154ee0 0x7eff5c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff61200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7eff5c0ce590 0x7eff5c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff61200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c071480 con 0x7eff5c0cfcf0 2024-04-25T10:03:13.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff61200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c072090 con 0x7eff5c154ee0 2024-04-25T10:03:13.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff61200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c071230 con 0x7eff5c0ce590 2024-04-25T10:03:13.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7eff5c154ee0 0x7eff5c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7eff5c154ee0 0x7eff5c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51624/0 (socket says 172.21.15.165:51624) 2024-04-25T10:03:13.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 -- 172.21.15.165:0/2625745475 learned_addr learned my addr 172.21.15.165:0/2625745475 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:13.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 -- 172.21.15.165:0/2625745475 >> v2:172.21.15.172:3300/0 conn(0x7eff5c0ce590 msgr2=0x7eff5c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff5a800640 1 --2- 172.21.15.165:0/2625745475 >> v2:172.21.15.172:3300/0 conn(0x7eff5c0ce590 0x7eff5c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 --2- 172.21.15.165:0/2625745475 >> v2:172.21.15.172:3300/0 conn(0x7eff5c0ce590 0x7eff5c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 -- 172.21.15.165:0/2625745475 >> v1:172.21.15.172:6789/0 conn(0x7eff5c0cfcf0 legacy=0x7eff5c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff59e00640 1 -- 172.21.15.165:0/2625745475 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c071760 con 0x7eff5c154ee0 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.277+0000 7eff5a800640 1 --2- 172.21.15.165:0/2625745475 >> v2:172.21.15.172:3300/0 conn(0x7eff5c0ce590 0x7eff5c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.278+0000 7eff59e00640 1 --2- 172.21.15.165:0/2625745475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0cf7b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0d0700 tx=0x7eff4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e4fe316e4838a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.279+0000 7eff59400640 1 -- 172.21.15.165:0/2625745475 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c036070 con 0x7eff5c154ee0 2024-04-25T10:03:13.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.279+0000 7eff59400640 1 -- 172.21.15.165:0/2625745475 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff4c031600 con 0x7eff5c154ee0 2024-04-25T10:03:13.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.279+0000 7eff59400640 1 -- 172.21.15.165:0/2625745475 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff4c0319d0 con 0x7eff5c154ee0 2024-04-25T10:03:13.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 -- 172.21.15.165:0/2625745475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 msgr2=0x7eff5c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 --2- 172.21.15.165:0/2625745475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0cf7b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7eff5c0d0700 tx=0x7eff4c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 -- 172.21.15.165:0/2625745475 shutdown_connections 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 --2- 172.21.15.165:0/2625745475 >> v2:172.21.15.172:3300/0 conn(0x7eff5c0ce590 0x7eff5c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 --2- 172.21.15.165:0/2625745475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0cf7b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 -- 172.21.15.165:0/2625745475 >> 172.21.15.165:0/2625745475 conn(0x7eff5c06abe0 msgr2=0x7eff5c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 -- 172.21.15.165:0/2625745475 shutdown_connections 2024-04-25T10:03:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.280+0000 7eff61200640 1 -- 172.21.15.165:0/2625745475 wait complete. 2024-04-25T10:03:13.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.282+0000 7eff61200640 1 Processor -- start 2024-04-25T10:03:13.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.282+0000 7eff61200640 1 -- start start 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff61200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0ce590 0x7eff5c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff61200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff5c0cfcf0 0x7eff5c158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff61200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff61200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c1552c0 con 0x7eff5c154ee0 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff61200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c071d20 con 0x7eff5c0ce590 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff61200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c155440 con 0x7eff5c0cfcf0 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0ad120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51630/0 (socket says 172.21.15.165:51630) 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 -- 172.21.15.165:0/2374888489 learned_addr learned my addr 172.21.15.165:0/2374888489 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff59e00640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0ce590 0x7eff5c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff51400640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff5c0cfcf0 0x7eff5c158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 -- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff5c0cfcf0 msgr2=0x7eff5c158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff5c0cfcf0 0x7eff5c158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 -- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0ce590 msgr2=0x7eff5c154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.283+0000 7eff5a800640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0ce590 0x7eff5c154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff5a800640 1 -- 172.21.15.165:0/2374888489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c159d90 con 0x7eff5c154ee0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff5a800640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0ad120 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c7f0 tx=0x7eff5400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54015070 con 0x7eff5c154ee0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff54007820 con 0x7eff5c154ee0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff54007c10 con 0x7eff5c154ee0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff5c158f50 con 0x7eff5c154ee0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.284+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c1594f0 con 0x7eff5c154ee0 2024-04-25T10:03:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.286+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c073990 con 0x7eff5c154ee0 2024-04-25T10:03:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.286+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7eff54018070 con 0x7eff5c154ee0 2024-04-25T10:03:13.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.287+0000 7eff52a00640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7eff3806b4d0 0x7eff3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.287+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7eff5408a7d0 con 0x7eff5c154ee0 2024-04-25T10:03:13.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.288+0000 7eff59e00640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7eff3806b4d0 0x7eff3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.288+0000 7eff59e00640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7eff3806b4d0 0x7eff3806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff4c033000 tx=0x7eff4c009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.292+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7eff540591b0 con 0x7eff5c154ee0 2024-04-25T10:03:13.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.489+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff5c06d360 con 0x7eff5c154ee0 2024-04-25T10:03:13.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.491+0000 7eff52a00640 1 -- 172.21.15.165:0/2374888489 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7eff54058ab0 con 0x7eff5c154ee0 2024-04-25T10:03:13.496 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:13.496 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7eff3806b4d0 msgr2=0x7eff3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7eff3806b4d0 0x7eff3806d990 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7eff4c033000 tx=0x7eff4c009000 comp rx=0 tx=0).stop 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 msgr2=0x7eff5c0ad120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0ad120 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7eff5400c7f0 tx=0x7eff5400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 shutdown_connections 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7eff3806b4d0 0x7eff3806d990 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7eff5c154ee0 0x7eff5c0ad120 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7eff5c0cfcf0 0x7eff5c158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 --2- 172.21.15.165:0/2374888489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7eff5c0ce590 0x7eff5c154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.493+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 >> 172.21.15.165:0/2374888489 conn(0x7eff5c06abe0 msgr2=0x7eff5c06d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.494+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 shutdown_connections 2024-04-25T10:03:13.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.494+0000 7eff61200640 1 -- 172.21.15.165:0/2374888489 wait complete. 2024-04-25T10:03:13.518 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:13.518 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:13.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.702+0000 7f7aaec00640 1 Processor -- start 2024-04-25T10:03:13.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.702+0000 7f7aaec00640 1 -- start start 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aaec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7aa815dc20 0x7f7aa80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aaec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa8161710 0x7f7aa80cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aaec00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aa815e000 con 0x7f7aa80b9330 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aaec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aa80d1de0 con 0x7f7aa815dc20 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aaec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa8161c60 con 0x7f7aa8161710 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aa7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7aa815dc20 0x7f7aa80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aacc00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7aa80b9330 0x7f7aa806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36484/0 (socket says 172.21.15.165:36484) 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.704+0000 7f7aacc00640 1 -- 172.21.15.165:0/978875759 learned_addr learned my addr 172.21.15.165:0/978875759 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:13.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.705+0000 7f7aa7400640 1 --2- 172.21.15.165:0/978875759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa8161710 0x7f7aa80cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.705+0000 7f7aa7e00640 1 -- 172.21.15.165:0/978875759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa8161710 msgr2=0x7f7aa80cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:13.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.705+0000 7f7aa7e00640 1 --2- 172.21.15.165:0/978875759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa8161710 0x7f7aa80cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.705+0000 7f7aa7e00640 1 -- 172.21.15.165:0/978875759 >> v1:172.21.15.165:6789/0 conn(0x7f7aa80b9330 legacy=0x7f7aa806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.705+0000 7f7aa7e00640 1 -- 172.21.15.165:0/978875759 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa8161f40 con 0x7f7aa815dc20 2024-04-25T10:03:13.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.705+0000 7f7aa6a00640 1 -- 172.21.15.165:0/978875759 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897039937 0 0) 0x7f7aa815e000 con 0x7f7aa80b9330 2024-04-25T10:03:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.706+0000 7f7aa7e00640 1 --2- 172.21.15.165:0/978875759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa815dc20 0x7f7aa80ce670 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097a0 tx=0x7f7a9802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e1a47af4647f82c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.706+0000 7f7aa6a00640 1 -- 172.21.15.165:0/978875759 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98037070 con 0x7f7aa815dc20 2024-04-25T10:03:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.706+0000 7f7aa6a00640 1 -- 172.21.15.165:0/978875759 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a98032440 con 0x7f7aa815dc20 2024-04-25T10:03:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.706+0000 7f7aa6a00640 1 -- 172.21.15.165:0/978875759 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98032810 con 0x7f7aa815dc20 2024-04-25T10:03:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 -- 172.21.15.165:0/978875759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa815dc20 msgr2=0x7f7aa80ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 --2- 172.21.15.165:0/978875759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa815dc20 0x7f7aa80ce670 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097a0 tx=0x7f7a9802a180 comp rx=0 tx=0).stop 2024-04-25T10:03:13.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 -- 172.21.15.165:0/978875759 shutdown_connections 2024-04-25T10:03:13.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 --2- 172.21.15.165:0/978875759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa8161710 0x7f7aa80cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 --2- 172.21.15.165:0/978875759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa815dc20 0x7f7aa80ce670 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 -- 172.21.15.165:0/978875759 >> 172.21.15.165:0/978875759 conn(0x7f7aa80d0420 msgr2=0x7f7aa80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 -- 172.21.15.165:0/978875759 shutdown_connections 2024-04-25T10:03:13.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.707+0000 7f7aaec00640 1 -- 172.21.15.165:0/978875759 wait complete. 2024-04-25T10:03:13.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.709+0000 7f7aaec00640 1 Processor -- start 2024-04-25T10:03:13.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.709+0000 7f7aaec00640 1 -- start start 2024-04-25T10:03:13.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aaec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 0x7f7aa815ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aaec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa815dc20 0x7f7aa80d3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aaec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7aa8161710 0x7f7aa80d3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aaec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80b5e50 con 0x7f7aa80b9330 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aaec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa8162500 con 0x7f7aa8161710 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aaec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aa80b5fd0 con 0x7f7aa815dc20 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.710+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 0x7f7aa815ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 0x7f7aa815ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51648/0 (socket says 172.21.15.165:51648) 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 -- 172.21.15.165:0/3151764678 learned_addr learned my addr 172.21.15.165:0/3151764678 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 -- 172.21.15.165:0/3151764678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa815dc20 msgr2=0x7f7aa80d3980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa815dc20 0x7f7aa80d3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 -- 172.21.15.165:0/3151764678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7aa8161710 msgr2=0x7f7aa80d3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7aa8161710 0x7f7aa80d3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 -- 172.21.15.165:0/3151764678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa81f38a0 con 0x7f7aa80b9330 2024-04-25T10:03:13.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.711+0000 7f7aa7e00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 0x7f7aa815ce40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097a0 tx=0x7f7a980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.712+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98037050 con 0x7f7aa80b9330 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.712+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a98038040 con 0x7f7aa80b9330 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.712+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a98004470 con 0x7f7aa80b9330 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.712+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aa81f3b80 con 0x7f7aa80b9330 2024-04-25T10:03:13.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.712+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aa81f4140 con 0x7f7aa80b9330 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.713+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aa8073a00 con 0x7f7aa80b9330 2024-04-25T10:03:13.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.714+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a98030900 con 0x7f7aa80b9330 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.715+0000 7f7a9fe00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a8406b4d0 0x7f7a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.715+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f7a980ab0f0 con 0x7f7aa80b9330 2024-04-25T10:03:13.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.715+0000 7f7aa7400640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a8406b4d0 0x7f7a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:13.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.716+0000 7f7aa7400640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a8406b4d0 0x7f7a8406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a90006fd0 tx=0x7f7a90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:13.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.719+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a98079b60 con 0x7f7aa80b9330 2024-04-25T10:03:13.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.917+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7aa80d16b0 con 0x7f7aa80b9330 2024-04-25T10:03:13.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.918+0000 7f7a9fe00640 1 -- 172.21.15.165:0/3151764678 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7a98079460 con 0x7f7aa80b9330 2024-04-25T10:03:13.923 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:13.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a8406b4d0 msgr2=0x7f7a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a8406b4d0 0x7f7a8406d990 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7a90006fd0 tx=0x7f7a90008040 comp rx=0 tx=0).stop 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 msgr2=0x7f7aa815ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 0x7f7aa815ce40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7a980097a0 tx=0x7f7a980046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 shutdown_connections 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a8406b4d0 0x7f7a8406d990 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7aa8161710 0x7f7aa80d3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7aa815dc20 0x7f7aa80d3980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 --2- 172.21.15.165:0/3151764678 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7aa80b9330 0x7f7aa815ce40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 >> 172.21.15.165:0/3151764678 conn(0x7f7aa80d0420 msgr2=0x7f7aa80b8600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 shutdown_connections 2024-04-25T10:03:13.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:13.921+0000 7f7aaec00640 1 -- 172.21.15.165:0/3151764678 wait complete. 2024-04-25T10:03:13.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:14.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.118+0000 7fb654200640 1 Processor -- start 2024-04-25T10:03:14.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.118+0000 7fb654200640 1 -- start start 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb654200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb64c15b710 0x7fb64c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb654200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb64c06ec50 0x7fb64c15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb654200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb64c06f0e0 con 0x7fb64c15ee60 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb654200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c15e490 con 0x7fb64c06ec50 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb654200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb64c0cfd80 con 0x7fb64c15b710 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb650e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb64c06ec50 0x7fb64c15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb652200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb64c15ee60 0x7fb64c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36498/0 (socket says 172.21.15.165:36498) 2024-04-25T10:03:14.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.120+0000 7fb652200640 1 -- 172.21.15.165:0/3677528808 learned_addr learned my addr 172.21.15.165:0/3677528808 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.121+0000 7fb651800640 1 --2- 172.21.15.165:0/3677528808 >> v2:172.21.15.172:3300/0 conn(0x7fb64c15b710 0x7fb64c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.121+0000 7fb650e00640 1 -- 172.21.15.165:0/3677528808 >> v2:172.21.15.172:3300/0 conn(0x7fb64c15b710 msgr2=0x7fb64c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.121+0000 7fb650e00640 1 --2- 172.21.15.165:0/3677528808 >> v2:172.21.15.172:3300/0 conn(0x7fb64c15b710 0x7fb64c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.121+0000 7fb650e00640 1 -- 172.21.15.165:0/3677528808 >> v1:172.21.15.165:6789/0 conn(0x7fb64c15ee60 legacy=0x7fb64c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.121+0000 7fb64be00640 1 -- 172.21.15.165:0/3677528808 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 510475954 0 0) 0x7fb64c06f0e0 con 0x7fb64c15ee60 2024-04-25T10:03:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.121+0000 7fb650e00640 1 -- 172.21.15.165:0/3677528808 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c0d0060 con 0x7fb64c06ec50 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.122+0000 7fb650e00640 1 --2- 172.21.15.165:0/3677528808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c06ec50 0x7fb64c15e920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb64c06b950 tx=0x7fb63802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e12ab10eeef552ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.122+0000 7fb64be00640 1 -- 172.21.15.165:0/3677528808 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638037070 con 0x7fb64c06ec50 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.122+0000 7fb64be00640 1 -- 172.21.15.165:0/3677528808 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb638032440 con 0x7fb64c06ec50 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.122+0000 7fb64be00640 1 -- 172.21.15.165:0/3677528808 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb638032810 con 0x7fb64c06ec50 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 -- 172.21.15.165:0/3677528808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c06ec50 msgr2=0x7fb64c15e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 --2- 172.21.15.165:0/3677528808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c06ec50 0x7fb64c15e920 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb64c06b950 tx=0x7fb63802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 -- 172.21.15.165:0/3677528808 shutdown_connections 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 --2- 172.21.15.165:0/3677528808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c06ec50 0x7fb64c15e920 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 --2- 172.21.15.165:0/3677528808 >> v2:172.21.15.172:3300/0 conn(0x7fb64c15b710 0x7fb64c0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 -- 172.21.15.165:0/3677528808 >> 172.21.15.165:0/3677528808 conn(0x7fb64c06abe0 msgr2=0x7fb64c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 -- 172.21.15.165:0/3677528808 shutdown_connections 2024-04-25T10:03:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.123+0000 7fb654200640 1 -- 172.21.15.165:0/3677528808 wait complete. 2024-04-25T10:03:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.125+0000 7fb654200640 1 Processor -- start 2024-04-25T10:03:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.125+0000 7fb654200640 1 -- start start 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb654200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb64c06ec50 0x7fb64c1479d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb654200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb64c15b710 0x7fb64c0b0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb654200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 0x7fb64c149170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb654200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c15bc70 con 0x7fb64c15ee60 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb654200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c15baf0 con 0x7fb64c06ec50 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb654200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64c0acfc0 con 0x7fb64c15b710 2024-04-25T10:03:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 0x7fb64c149170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 0x7fb64c149170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51660/0 (socket says 172.21.15.165:51660) 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 -- 172.21.15.165:0/2731345753 learned_addr learned my addr 172.21.15.165:0/2731345753 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb651800640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb64c06ec50 0x7fb64c1479d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 -- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb64c15b710 msgr2=0x7fb64c0b0ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb64c15b710 0x7fb64c0b0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 -- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb64c06ec50 msgr2=0x7fb64c1479d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb64c06ec50 0x7fb64c1479d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.126+0000 7fb652200640 1 -- 172.21.15.165:0/2731345753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb64c1480d0 con 0x7fb64c15ee60 2024-04-25T10:03:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.127+0000 7fb652200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 0x7fb64c149170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c730 tx=0x7fb64400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.127+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb644018070 con 0x7fb64c15ee60 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.127+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb64400ce40 con 0x7fb64c15ee60 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.127+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64c1483b0 con 0x7fb64c15ee60 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.127+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64400a860 con 0x7fb64c15ee60 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.127+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64c148970 con 0x7fb64c15ee60 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.128+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb64c073a00 con 0x7fb64c15ee60 2024-04-25T10:03:14.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.129+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb64401b070 con 0x7fb64c15ee60 2024-04-25T10:03:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.130+0000 7fb649600640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb62c06b4d0 0x7fb62c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.130+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fb644089c00 con 0x7fb64c15ee60 2024-04-25T10:03:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.130+0000 7fb651800640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb62c06b4d0 0x7fb62c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.131+0000 7fb651800640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb62c06b4d0 0x7fb62c06d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb640006fd0 tx=0x7fb640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.135+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb644090080 con 0x7fb64c15ee60 2024-04-25T10:03:14.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.332+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb64c06e180 con 0x7fb64c15ee60 2024-04-25T10:03:14.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.333+0000 7fb649600640 1 -- 172.21.15.165:0/2731345753 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb644058680 con 0x7fb64c15ee60 2024-04-25T10:03:14.338 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb62c06b4d0 msgr2=0x7fb62c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb62c06b4d0 0x7fb62c06d990 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb640006fd0 tx=0x7fb640008040 comp rx=0 tx=0).stop 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 msgr2=0x7fb64c149170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 0x7fb64c149170 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c730 tx=0x7fb64400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 shutdown_connections 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb62c06b4d0 0x7fb62c06d990 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb64c15ee60 0x7fb64c149170 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb64c15b710 0x7fb64c0b0ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 --2- 172.21.15.165:0/2731345753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb64c06ec50 0x7fb64c1479d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 >> 172.21.15.165:0/2731345753 conn(0x7fb64c06abe0 msgr2=0x7fb64c0ae5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 shutdown_connections 2024-04-25T10:03:14.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.336+0000 7fb654200640 1 -- 172.21.15.165:0/2731345753 wait complete. 2024-04-25T10:03:14.359 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:14.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.559+0000 7f4e23600640 1 Processor -- start 2024-04-25T10:03:14.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.560+0000 7f4e23600640 1 -- start start 2024-04-25T10:03:14.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.561+0000 7f4e23600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e1c0ce7b0 0x7f4e1c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.561+0000 7f4e23600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1c06c8d0 con 0x7f4e1c1610f0 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.561+0000 7f4e23600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e1c0c6960 con 0x7f4e1c1616f0 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.561+0000 7f4e23600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c1614d0 con 0x7f4e1c0ce7b0 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.562+0000 7f4e20c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4e1c1610f0 0x7f4e1c06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36506/0 (socket says 172.21.15.165:36506) 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.562+0000 7f4e20c00640 1 -- 172.21.15.165:0/2315654064 learned_addr learned my addr 172.21.15.165:0/2315654064 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.562+0000 7f4e21600640 1 --2- 172.21.15.165:0/2315654064 >> v2:172.21.15.165:3300/0 conn(0x7f4e1c0ce7b0 0x7f4e1c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.562+0000 7f4e1b400640 1 -- 172.21.15.165:0/2315654064 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424887972 0 0) 0x7f4e1c06c8d0 con 0x7f4e1c1610f0 2024-04-25T10:03:14.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.562+0000 7f4e1b400640 1 -- 172.21.15.165:0/2315654064 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e00003520 con 0x7f4e1c1610f0 2024-04-25T10:03:14.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.563+0000 7f4e21600640 1 -- 172.21.15.165:0/2315654064 >> v1:172.21.15.172:6789/0 conn(0x7f4e1c1616f0 legacy=0x7f4e1c0ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.563+0000 7f4e21600640 1 -- 172.21.15.165:0/2315654064 >> v1:172.21.15.165:6789/0 conn(0x7f4e1c1610f0 legacy=0x7f4e1c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.563+0000 7f4e1b400640 1 -- 172.21.15.165:0/2315654064 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2194931243 0 0) 0x7f4e00003520 con 0x7f4e1c1610f0 2024-04-25T10:03:14.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.563+0000 7f4e21600640 1 -- 172.21.15.165:0/2315654064 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c162260 con 0x7f4e1c0ce7b0 2024-04-25T10:03:14.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.564+0000 7f4e21600640 1 --2- 172.21.15.165:0/2315654064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c0ce7b0 0x7f4e1c161b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38657ed93d40b485 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.564+0000 7f4e1b400640 1 -- 172.21.15.165:0/2315654064 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c037070 con 0x7f4e1c0ce7b0 2024-04-25T10:03:14.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.564+0000 7f4e1b400640 1 -- 172.21.15.165:0/2315654064 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c02ac50 con 0x7f4e1c0ce7b0 2024-04-25T10:03:14.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.564+0000 7f4e1b400640 1 -- 172.21.15.165:0/2315654064 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14001670 con 0x7f4e1c0ce7b0 2024-04-25T10:03:14.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 -- 172.21.15.165:0/2315654064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c0ce7b0 msgr2=0x7f4e1c161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 --2- 172.21.15.165:0/2315654064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c0ce7b0 0x7f4e1c161b50 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 -- 172.21.15.165:0/2315654064 shutdown_connections 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 --2- 172.21.15.165:0/2315654064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c0ce7b0 0x7f4e1c161b50 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 -- 172.21.15.165:0/2315654064 >> 172.21.15.165:0/2315654064 conn(0x7f4e1c06f500 msgr2=0x7f4e1c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 -- 172.21.15.165:0/2315654064 shutdown_connections 2024-04-25T10:03:14.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.565+0000 7f4e23600640 1 -- 172.21.15.165:0/2315654064 wait complete. 2024-04-25T10:03:14.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.566+0000 7f4e23600640 1 Processor -- start 2024-04-25T10:03:14.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.567+0000 7f4e23600640 1 -- start start 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.567+0000 7f4e23600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e1c0ce7b0 0x7f4e1c15fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.567+0000 7f4e23600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e1c1610f0 0x7f4e1c1605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.567+0000 7f4e23600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 0x7f4e1c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.567+0000 7f4e23600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0cfce0 con 0x7f4e1c1616f0 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e23600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0cfa00 con 0x7f4e1c1610f0 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e23600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0cf690 con 0x7f4e1c0ce7b0 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 0x7f4e1c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 0x7f4e1c15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51674/0 (socket says 172.21.15.165:51674) 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 -- 172.21.15.165:0/3408756978 learned_addr learned my addr 172.21.15.165:0/3408756978 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e1be00640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e1c1610f0 0x7f4e1c1605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 -- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e1c0ce7b0 msgr2=0x7f4e1c15fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e1c0ce7b0 0x7f4e1c15fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 -- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e1c1610f0 msgr2=0x7f4e1c1605a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e1c1610f0 0x7f4e1c1605a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e21600640 1 -- 172.21.15.165:0/3408756978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c1514d0 con 0x7f4e1c1616f0 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.568+0000 7f4e1be00640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e1c1610f0 0x7f4e1c1605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:14.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.569+0000 7f4e21600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 0x7f4e1c15e6e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.569+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c041070 con 0x7f4e1c1616f0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.569+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c03c460 con 0x7f4e1c1616f0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.569+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c037040 con 0x7f4e1c1616f0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.569+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c15ed30 con 0x7f4e1c1616f0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.569+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c15f2f0 con 0x7f4e1c1616f0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.570+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c073a00 con 0x7f4e1c1616f0 2024-04-25T10:03:14.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.571+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e0c044070 con 0x7f4e1c1616f0 2024-04-25T10:03:14.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.572+0000 7f4e18c00640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4df806b4d0 0x7f4df806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:14.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.572+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f4e0c0aabb0 con 0x7f4e1c1616f0 2024-04-25T10:03:14.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.572+0000 7f4e20c00640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4df806b4d0 0x7f4df806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:14.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.573+0000 7f4e20c00640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4df806b4d0 0x7f4df806d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005880 tx=0x7f4e14001060 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:14.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.577+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e0c079630 con 0x7f4e1c1616f0 2024-04-25T10:03:14.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.771+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e1c070cf0 con 0x7f4e1c1616f0 2024-04-25T10:03:14.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.772+0000 7f4e18c00640 1 -- 172.21.15.165:0/3408756978 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4e0c078f30 con 0x7f4e1c1616f0 2024-04-25T10:03:14.776 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:14.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4df806b4d0 msgr2=0x7f4df806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4df806b4d0 0x7f4df806d990 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f4e14005880 tx=0x7f4e14001060 comp rx=0 tx=0).stop 2024-04-25T10:03:14.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 msgr2=0x7f4e1c15e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:14.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 0x7f4e1c15e6e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0097a0 tx=0x7f4e0c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 shutdown_connections 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4df806b4d0 0x7f4df806d990 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e1c1616f0 0x7f4e1c15e6e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e1c1610f0 0x7f4e1c1605a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 --2- 172.21.15.165:0/3408756978 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e1c0ce7b0 0x7f4e1c15fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.774+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 >> 172.21.15.165:0/3408756978 conn(0x7f4e1c06f500 msgr2=0x7f4e1c06fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.775+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 shutdown_connections 2024-04-25T10:03:14.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:14.775+0000 7f4e23600640 1 -- 172.21.15.165:0/3408756978 wait complete. 2024-04-25T10:03:14.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:15.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.004+0000 7f977f400640 1 Processor -- start 2024-04-25T10:03:15.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.004+0000 7f977f400640 1 -- start start 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97781446d0 0x7f97780b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9778144d50 0x7f97780b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977f400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97780cccc0 con 0x7f97781452a0 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977f400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97780b2b20 con 0x7f97781446d0 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f977806d400 con 0x7f9778144d50 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97781446d0 0x7f97780b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97781446d0 0x7f97780b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51688/0 (socket says 172.21.15.165:51688) 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f977ca00640 1 -- 172.21.15.165:0/3903374085 learned_addr learned my addr 172.21.15.165:0/3903374085 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:15.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.006+0000 7f9777e00640 1 --2- 172.21.15.165:0/3903374085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9778144d50 0x7f97780b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.007+0000 7f977ca00640 1 -- 172.21.15.165:0/3903374085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9778144d50 msgr2=0x7f97780b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.007+0000 7f977ca00640 1 --2- 172.21.15.165:0/3903374085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9778144d50 0x7f97780b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.007+0000 7f977ca00640 1 -- 172.21.15.165:0/3903374085 >> v1:172.21.15.172:6789/0 conn(0x7f97781452a0 legacy=0x7f97780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.007+0000 7f977ca00640 1 -- 172.21.15.165:0/3903374085 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97780cc140 con 0x7f97781446d0 2024-04-25T10:03:15.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.008+0000 7f977ca00640 1 --2- 172.21.15.165:0/3903374085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97781446d0 0x7f97780b3f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9768009870 tx=0x7f976802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=344878f332d849d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.008+0000 7f9777400640 1 -- 172.21.15.165:0/3903374085 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768037070 con 0x7f97781446d0 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.008+0000 7f9777400640 1 -- 172.21.15.165:0/3903374085 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9768032440 con 0x7f97781446d0 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.008+0000 7f9777400640 1 -- 172.21.15.165:0/3903374085 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9768032810 con 0x7f97781446d0 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.008+0000 7f977f400640 1 -- 172.21.15.165:0/3903374085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97781446d0 msgr2=0x7f97780b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.008+0000 7f977f400640 1 --2- 172.21.15.165:0/3903374085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97781446d0 0x7f97780b3f20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9768009870 tx=0x7f976802a210 comp rx=0 tx=0).stop 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.009+0000 7f977f400640 1 -- 172.21.15.165:0/3903374085 shutdown_connections 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.009+0000 7f977f400640 1 --2- 172.21.15.165:0/3903374085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9778144d50 0x7f97780b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.009+0000 7f977f400640 1 --2- 172.21.15.165:0/3903374085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97781446d0 0x7f97780b3f20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.009+0000 7f977f400640 1 -- 172.21.15.165:0/3903374085 >> 172.21.15.165:0/3903374085 conn(0x7f977806abe0 msgr2=0x7f977806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.009+0000 7f977f400640 1 -- 172.21.15.165:0/3903374085 shutdown_connections 2024-04-25T10:03:15.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.009+0000 7f977f400640 1 -- 172.21.15.165:0/3903374085 wait complete. 2024-04-25T10:03:15.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.010+0000 7f977f400640 1 Processor -- start 2024-04-25T10:03:15.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 -- start start 2024-04-25T10:03:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97781446d0 0x7f977813a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 0x7f97780ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97781452a0 0x7f977813c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cc700 con 0x7f9778144d50 2024-04-25T10:03:15.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97780b49a0 con 0x7f97781452a0 2024-04-25T10:03:15.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.011+0000 7f977f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97780cc9e0 con 0x7f97781446d0 2024-04-25T10:03:15.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 0x7f97780ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 0x7f97780ac270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51700/0 (socket says 172.21.15.165:51700) 2024-04-25T10:03:15.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 -- 172.21.15.165:0/165150406 learned_addr learned my addr 172.21.15.165:0/165150406 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:15.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f977ca00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97781446d0 0x7f977813a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f977d400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97781452a0 0x7f977813c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 -- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97781446d0 msgr2=0x7f977813a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97781446d0 0x7f977813a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 -- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97781452a0 msgr2=0x7f977813c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97781452a0 0x7f977813c5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f9777e00640 1 -- 172.21.15.165:0/165150406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f977813a940 con 0x7f9778144d50 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.012+0000 7f977d400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97781452a0 0x7f977813c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.013+0000 7f9777e00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 0x7f97780ac270 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f976400d800 tx=0x7f976400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.013+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976400b6c0 con 0x7f9778144d50 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.013+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9764019070 con 0x7f9778144d50 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.013+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f976400ba80 con 0x7f9778144d50 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.013+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f977813cf10 con 0x7f9778144d50 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.013+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97780a8540 con 0x7f9778144d50 2024-04-25T10:03:15.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.014+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9778073a00 con 0x7f9778144d50 2024-04-25T10:03:15.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.015+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f976400bc20 con 0x7f9778144d50 2024-04-25T10:03:15.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.016+0000 7f9774c00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f975806b4d0 0x7f975806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.016+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f976408aac0 con 0x7f9778144d50 2024-04-25T10:03:15.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.016+0000 7f977ca00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f975806b4d0 0x7f975806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.017+0000 7f977ca00640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f975806b4d0 0x7f975806d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9768034000 tx=0x7f976802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.020+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9764059530 con 0x7f9778144d50 2024-04-25T10:03:15.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.213+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f977806c250 con 0x7f9778144d50 2024-04-25T10:03:15.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.214+0000 7f9774c00640 1 -- 172.21.15.165:0/165150406 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9764058e30 con 0x7f9778144d50 2024-04-25T10:03:15.219 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f975806b4d0 msgr2=0x7f975806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f975806b4d0 0x7f975806d990 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f9768034000 tx=0x7f976802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 msgr2=0x7f97780ac270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 0x7f97780ac270 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f976400d800 tx=0x7f976400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 shutdown_connections 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f975806b4d0 0x7f975806d990 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97781452a0 0x7f977813c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9778144d50 0x7f97780ac270 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 --2- 172.21.15.165:0/165150406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97781446d0 0x7f977813a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 >> 172.21.15.165:0/165150406 conn(0x7f977806abe0 msgr2=0x7f9778143630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 shutdown_connections 2024-04-25T10:03:15.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.217+0000 7f977f400640 1 -- 172.21.15.165:0/165150406 wait complete. 2024-04-25T10:03:15.239 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:03:15.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.429+0000 7ff158a00640 1 Processor -- start 2024-04-25T10:03:15.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.430+0000 7ff158a00640 1 -- start start 2024-04-25T10:03:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff158a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff15406bff0 0x7ff1540b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff158a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff1540d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff158a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff15415e110 con 0x7ff15415dd30 2024-04-25T10:03:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff158a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15406d000 con 0x7ff15406bff0 2024-04-25T10:03:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff158a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1540b2fd0 con 0x7ff1540b5970 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff149e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff15406bff0 0x7ff1540b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff149e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff15406bff0 0x7ff1540b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51706/0 (socket says 172.21.15.165:51706) 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.431+0000 7ff149e00640 1 -- 172.21.15.165:0/1296214772 learned_addr learned my addr 172.21.15.165:0/1296214772 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.432+0000 7ff153200640 1 --2- 172.21.15.165:0/1296214772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff1540d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.432+0000 7ff149e00640 1 -- 172.21.15.165:0/1296214772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 msgr2=0x7ff1540d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.432+0000 7ff149e00640 1 --2- 172.21.15.165:0/1296214772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff1540d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.432+0000 7ff149e00640 1 -- 172.21.15.165:0/1296214772 >> v1:172.21.15.172:6789/0 conn(0x7ff15415dd30 legacy=0x7ff1540cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:15.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.432+0000 7ff149e00640 1 -- 172.21.15.165:0/1296214772 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15406c480 con 0x7ff15406bff0 2024-04-25T10:03:15.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.433+0000 7ff149e00640 1 --2- 172.21.15.165:0/1296214772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15406bff0 0x7ff1540b5430 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009870 tx=0x7ff13c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56df457c5d1a2b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.433+0000 7ff151e00640 1 -- 172.21.15.165:0/1296214772 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c037070 con 0x7ff15406bff0 2024-04-25T10:03:15.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.433+0000 7ff151e00640 1 -- 172.21.15.165:0/1296214772 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c032440 con 0x7ff15406bff0 2024-04-25T10:03:15.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.433+0000 7ff151e00640 1 -- 172.21.15.165:0/1296214772 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c032810 con 0x7ff15406bff0 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 -- 172.21.15.165:0/1296214772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15406bff0 msgr2=0x7ff1540b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 --2- 172.21.15.165:0/1296214772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15406bff0 0x7ff1540b5430 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009870 tx=0x7ff13c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 -- 172.21.15.165:0/1296214772 shutdown_connections 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 --2- 172.21.15.165:0/1296214772 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff1540d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 --2- 172.21.15.165:0/1296214772 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15406bff0 0x7ff1540b5430 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 -- 172.21.15.165:0/1296214772 >> 172.21.15.165:0/1296214772 conn(0x7ff1540d0610 msgr2=0x7ff1540d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 -- 172.21.15.165:0/1296214772 shutdown_connections 2024-04-25T10:03:15.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.434+0000 7ff158a00640 1 -- 172.21.15.165:0/1296214772 wait complete. 2024-04-25T10:03:15.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.436+0000 7ff158a00640 1 Processor -- start 2024-04-25T10:03:15.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.436+0000 7ff158a00640 1 -- start start 2024-04-25T10:03:15.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff158a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff15414ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff158a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 0x7ff1541578c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff158a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1541570b0 0x7ff1540b8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff158a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406cd20 con 0x7ff15415dd30 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff158a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15406ca40 con 0x7ff1541570b0 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff158a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1540d26a0 con 0x7ff1540b5970 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff149e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 0x7ff1541578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff149e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 0x7ff1541578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51712/0 (socket says 172.21.15.165:51712) 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.437+0000 7ff149e00640 1 -- 172.21.15.165:0/4122762506 learned_addr learned my addr 172.21.15.165:0/4122762506 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff149e00640 1 -- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 msgr2=0x7ff15414ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff153200640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1541570b0 0x7ff1540b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff152800640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff15414ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff149e00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff15414ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff149e00640 1 -- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1541570b0 msgr2=0x7ff1540b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff149e00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1541570b0 0x7ff1540b8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff149e00640 1 -- 172.21.15.165:0/4122762506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15414e330 con 0x7ff15415dd30 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff152800640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff15414ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:15.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff149e00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 0x7ff1541578c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009870 tx=0x7ff13c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c041020 con 0x7ff15415dd30 2024-04-25T10:03:15.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c0306c0 con 0x7ff15415dd30 2024-04-25T10:03:15.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c037040 con 0x7ff15415dd30 2024-04-25T10:03:15.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1540cabb0 con 0x7ff15415dd30 2024-04-25T10:03:15.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.438+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1540cb170 con 0x7ff15415dd30 2024-04-25T10:03:15.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.440+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1540cbc30 con 0x7ff15415dd30 2024-04-25T10:03:15.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.440+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff13c044020 con 0x7ff15415dd30 2024-04-25T10:03:15.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.441+0000 7ff14b400640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff13006b500 0x7ff13006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:15.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.441+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff13c0aae40 con 0x7ff15415dd30 2024-04-25T10:03:15.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.442+0000 7ff152800640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff13006b500 0x7ff13006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:15.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.442+0000 7ff152800640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff13006b500 0x7ff13006d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff14400a830 tx=0x7ff144008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:15.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.446+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1540cbc30 con 0x7ff15415dd30 2024-04-25T10:03:15.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.652+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1540d16c0 con 0x7ff15415dd30 2024-04-25T10:03:15.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.652+0000 7ff14b400640 1 -- 172.21.15.165:0/4122762506 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v26) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff13c04b030 con 0x7ff15415dd30 2024-04-25T10:03:15.657 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff13006b500 msgr2=0x7ff13006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff13006b500 0x7ff13006d9c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff14400a830 tx=0x7ff144008040 comp rx=0 tx=0).stop 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 msgr2=0x7ff1541578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 0x7ff1541578c0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff13c009870 tx=0x7ff13c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 shutdown_connections 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff13006b500 0x7ff13006d9c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1541570b0 0x7ff1540b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff15415dd30 0x7ff1541578c0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 --2- 172.21.15.165:0/4122762506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1540b5970 0x7ff15414ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:15.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 >> 172.21.15.165:0/4122762506 conn(0x7ff1540d0610 msgr2=0x7ff1540d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:15.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 shutdown_connections 2024-04-25T10:03:15.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:15.655+0000 7ff158a00640 1 -- 172.21.15.165:0/4122762506 wait complete. 2024-04-25T10:03:15.679 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-04-25T10:03:15.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:15.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-04-25T10:03:15.778 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:03:15.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:15.779 DEBUG:teuthology.orchestra.run.smithi172:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-04-25T10:03:15.831 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:03:15.831 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:16.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.034+0000 7fa92de00640 1 Processor -- start 2024-04-25T10:03:16.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.034+0000 7fa92de00640 1 -- start start 2024-04-25T10:03:16.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.035+0000 7fa92de00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa928164970 0x7fa928166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.035+0000 7fa92de00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa928144b30 con 0x7fa9280ce920 2024-04-25T10:03:16.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.035+0000 7fa92de00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9280712d0 con 0x7fa928144750 2024-04-25T10:03:16.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.035+0000 7fa92de00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9280715b0 con 0x7fa928164970 2024-04-25T10:03:16.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.036+0000 7fa91e000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa9280ce920 0x7fa9280cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36508/0 (socket says 172.21.15.165:36508) 2024-04-25T10:03:16.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.036+0000 7fa91e000640 1 -- 172.21.15.165:0/2966277257 learned_addr learned my addr 172.21.15.165:0/2966277257 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.036+0000 7fa927400640 1 --2- 172.21.15.165:0/2966277257 >> v2:172.21.15.172:3300/0 conn(0x7fa928164970 0x7fa928166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.036+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1553892592 0 0) 0x7fa928144b30 con 0x7fa9280ce920 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.036+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa90c003520 con 0x7fa9280ce920 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270232376 0 0) 0x7fa90c003520 con 0x7fa9280ce920 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 >> v2:172.21.15.172:3300/0 conn(0x7fa928164970 msgr2=0x7fa928166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 --2- 172.21.15.165:0/2966277257 >> v2:172.21.15.172:3300/0 conn(0x7fa928164970 0x7fa928166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 >> v1:172.21.15.172:6789/0 conn(0x7fa928144750 legacy=0x7fa92806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928071890 con 0x7fa9280ce920 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa910001670 con 0x7fa9280ce920 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 >> v1:172.21.15.165:6789/0 conn(0x7fa9280ce920 legacy=0x7fa9280cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa90c0039b0 0x7fa90c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa90c004800 0x7fa90c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa90c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9280715b0 con 0x7fa90c0051c0 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c00c160 con 0x7fa90c0039b0 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90c00c360 con 0x7fa90c004800 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa927400640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa90c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.037+0000 7fa926a00640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa90c0039b0 0x7fa90c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.038+0000 7fa927400640 1 -- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa90c004800 msgr2=0x7fa90c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.038+0000 7fa927400640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa90c004800 0x7fa90c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.038+0000 7fa927400640 1 -- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa90c0039b0 msgr2=0x7fa90c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.038+0000 7fa927400640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa90c0039b0 0x7fa90c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.038+0000 7fa927400640 1 -- 172.21.15.165:0/2966277257 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90c003520 con 0x7fa90c0051c0 2024-04-25T10:03:16.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.038+0000 7fa927400640 1 -- 172.21.15.165:0/2966277257 reap_dead start 2024-04-25T10:03:16.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.039+0000 7fa927400640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa90c00b9c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa920009fc0 tx=0x7fa90c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04c5f75caf06043 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.039+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920035020 con 0x7fa90c0051c0 2024-04-25T10:03:16.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.039+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa920003b90 con 0x7fa90c0051c0 2024-04-25T10:03:16.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.039+0000 7fa926000640 1 -- 172.21.15.165:0/2966277257 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa920009c10 con 0x7fa90c0051c0 2024-04-25T10:03:16.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.039+0000 7fa92de00640 1 -- 172.21.15.165:0/2966277257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 msgr2=0x7fa90c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.039+0000 7fa92de00640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa90c00b9c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fa920009fc0 tx=0x7fa90c005830 comp rx=0 tx=0).stop 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.040+0000 7fa92de00640 1 -- 172.21.15.165:0/2966277257 shutdown_connections 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.040+0000 7fa92de00640 1 --2- 172.21.15.165:0/2966277257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa90c00b9c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.040+0000 7fa92de00640 1 -- 172.21.15.165:0/2966277257 >> 172.21.15.165:0/2966277257 conn(0x7fa92806abe0 msgr2=0x7fa92806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.040+0000 7fa92de00640 1 -- 172.21.15.165:0/2966277257 shutdown_connections 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.040+0000 7fa92de00640 1 -- 172.21.15.165:0/2966277257 wait complete. 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.041+0000 7fa92de00640 1 Processor -- start 2024-04-25T10:03:16.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.042+0000 7fa92de00640 1 -- start start 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.042+0000 7fa92de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa92813b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.042+0000 7fa92de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa92813feb0 0x7fa928158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.042+0000 7fa92de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa92815d960 0x7fa92815dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.042+0000 7fa92de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9281455d0 con 0x7fa90c0051c0 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.042+0000 7fa92de00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa928145450 con 0x7fa92813feb0 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa92de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92815c000 con 0x7fa92815d960 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa92813b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa92813b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51738/0 (socket says 172.21.15.165:51738) 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 -- 172.21.15.165:0/2065132907 learned_addr learned my addr 172.21.15.165:0/2065132907 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 -- 172.21.15.165:0/2065132907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa92815d960 msgr2=0x7fa92815dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa92815d960 0x7fa92815dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 -- 172.21.15.165:0/2065132907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa92813feb0 msgr2=0x7fa928158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa92813feb0 0x7fa928158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 -- 172.21.15.165:0/2065132907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92815f290 con 0x7fa90c0051c0 2024-04-25T10:03:16.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.043+0000 7fa926a00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa92813b990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e410 tx=0x7fa91800e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.044+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918019070 con 0x7fa90c0051c0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.044+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9180052e0 con 0x7fa90c0051c0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.044+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92815ecd0 con 0x7fa90c0051c0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.044+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa918003810 con 0x7fa90c0051c0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.044+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa928140500 con 0x7fa90c0051c0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.045+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa928073990 con 0x7fa90c0051c0 2024-04-25T10:03:16.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.046+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa91801c070 con 0x7fa90c0051c0 2024-04-25T10:03:16.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.047+0000 7fa91fe00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa90406b500 0x7fa90406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.047+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fa918089bc0 con 0x7fa90c0051c0 2024-04-25T10:03:16.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.047+0000 7fa91e000640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa90406b500 0x7fa90406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.048+0000 7fa91e000640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa90406b500 0x7fa90406d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9281f7040 tx=0x7fa910001500 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.051+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa918057fe0 con 0x7fa90c0051c0 2024-04-25T10:03:16.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.291+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa9280d0510 con 0x7fa90c0051c0 2024-04-25T10:03:16.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.291+0000 7fa91fe00640 1 -- 172.21.15.165:0/2065132907 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7fa91805c080 con 0x7fa90c0051c0 2024-04-25T10:03:16.296 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:16.296 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:16.297 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa90406b500 msgr2=0x7fa90406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa90406b500 0x7fa90406d9c0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa9281f7040 tx=0x7fa910001500 comp rx=0 tx=0).stop 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 msgr2=0x7fa92813b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa92813b990 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fa91800e410 tx=0x7fa91800e8d0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 shutdown_connections 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa90406b500 0x7fa90406d9c0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa92815d960 0x7fa92815dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa92813feb0 0x7fa928158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 --2- 172.21.15.165:0/2065132907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa90c0051c0 0x7fa92813b990 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 >> 172.21.15.165:0/2065132907 conn(0x7fa92806abe0 msgr2=0x7fa9280ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 shutdown_connections 2024-04-25T10:03:16.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.294+0000 7fa92de00640 1 -- 172.21.15.165:0/2065132907 wait complete. 2024-04-25T10:03:16.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:16.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.497+0000 7f0d3a800640 1 Processor -- start 2024-04-25T10:03:16.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.497+0000 7f0d3a800640 1 -- start start 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d3a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d34144440 0x7f0d34070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d3a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d34071530 0x7f0d340cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d3a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d3415bf30 con 0x7f0d340cd860 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d3a800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d3415c860 con 0x7f0d34144440 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d3a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d34071970 con 0x7f0d34071530 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d39400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d34144440 0x7f0d34070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d39400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0d34144440 0x7f0d34070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51742/0 (socket says 172.21.15.165:51742) 2024-04-25T10:03:16.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.499+0000 7f0d39400640 1 -- 172.21.15.165:0/2082851890 learned_addr learned my addr 172.21.15.165:0/2082851890 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:16.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.500+0000 7f0d38a00640 1 --2- 172.21.15.165:0/2082851890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d34071530 0x7f0d340cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.500+0000 7f0d39400640 1 -- 172.21.15.165:0/2082851890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d34071530 msgr2=0x7f0d340cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.500+0000 7f0d39400640 1 --2- 172.21.15.165:0/2082851890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d34071530 0x7f0d340cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.500+0000 7f0d39400640 1 -- 172.21.15.165:0/2082851890 >> v1:172.21.15.172:6789/0 conn(0x7f0d340cd860 legacy=0x7f0d3406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.500+0000 7f0d39400640 1 -- 172.21.15.165:0/2082851890 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d3415c210 con 0x7f0d34144440 2024-04-25T10:03:16.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.501+0000 7f0d39400640 1 --2- 172.21.15.165:0/2082851890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34144440 0x7f0d34070ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009870 tx=0x7f0d2802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=769eac69fc19d68f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.501+0000 7f0d33e00640 1 -- 172.21.15.165:0/2082851890 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28037070 con 0x7f0d34144440 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.501+0000 7f0d33e00640 1 -- 172.21.15.165:0/2082851890 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d28032440 con 0x7f0d34144440 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.501+0000 7f0d33e00640 1 -- 172.21.15.165:0/2082851890 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28032810 con 0x7f0d34144440 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 -- 172.21.15.165:0/2082851890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34144440 msgr2=0x7f0d34070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 --2- 172.21.15.165:0/2082851890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34144440 0x7f0d34070ff0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009870 tx=0x7f0d2802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 -- 172.21.15.165:0/2082851890 shutdown_connections 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 --2- 172.21.15.165:0/2082851890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d34071530 0x7f0d340cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 --2- 172.21.15.165:0/2082851890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34144440 0x7f0d34070ff0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 -- 172.21.15.165:0/2082851890 >> 172.21.15.165:0/2082851890 conn(0x7f0d3406abe0 msgr2=0x7f0d3406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 -- 172.21.15.165:0/2082851890 shutdown_connections 2024-04-25T10:03:16.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.502+0000 7f0d3a800640 1 -- 172.21.15.165:0/2082851890 wait complete. 2024-04-25T10:03:16.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.504+0000 7f0d3a800640 1 Processor -- start 2024-04-25T10:03:16.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.504+0000 7f0d3a800640 1 -- start start 2024-04-25T10:03:16.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d3a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 0x7f0d340cbc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d3a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d340cd860 0x7f0d341f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d3a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d34144440 0x7f0d340c97b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d3a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340cdcd0 con 0x7f0d34071530 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d3a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d3415bc30 con 0x7f0d34144440 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d3a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d340cde50 con 0x7f0d340cd860 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 0x7f0d340cbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 0x7f0d340cbc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51758/0 (socket says 172.21.15.165:51758) 2024-04-25T10:03:16.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 -- 172.21.15.165:0/4116239238 learned_addr learned my addr 172.21.15.165:0/4116239238 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d38a00640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d340cd860 0x7f0d341f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 -- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d340cd860 msgr2=0x7f0d341f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39e00640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d34144440 0x7f0d340c97b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d340cd860 0x7f0d341f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 -- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d34144440 msgr2=0x7f0d340c97b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d34144440 0x7f0d340c97b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.505+0000 7f0d39400640 1 -- 172.21.15.165:0/4116239238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d340c9cf0 con 0x7f0d34071530 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.506+0000 7f0d39400640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 0x7f0d340cbc50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009870 tx=0x7f0d280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.506+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28041020 con 0x7f0d34071530 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.506+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d280306c0 con 0x7f0d34071530 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.506+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d340cc3c0 con 0x7f0d34071530 2024-04-25T10:03:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.506+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d28037040 con 0x7f0d34071530 2024-04-25T10:03:16.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.506+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d340b25e0 con 0x7f0d34071530 2024-04-25T10:03:16.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.507+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d34073990 con 0x7f0d34071530 2024-04-25T10:03:16.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.508+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d28044020 con 0x7f0d34071530 2024-04-25T10:03:16.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.510+0000 7f0d31600640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d1406b4d0 0x7f0d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.510+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f0d280aae30 con 0x7f0d34071530 2024-04-25T10:03:16.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.510+0000 7f0d38a00640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d1406b4d0 0x7f0d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.510+0000 7f0d38a00640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d1406b4d0 0x7f0d1406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d341f50d0 tx=0x7f0d20008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.514+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d2804b030 con 0x7f0d34071530 2024-04-25T10:03:16.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.694+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d340cea70 con 0x7f0d34071530 2024-04-25T10:03:16.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.697+0000 7f0d31600640 1 -- 172.21.15.165:0/4116239238 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f0d280798a0 con 0x7f0d34071530 2024-04-25T10:03:16.702 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:16.702 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:16.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d1406b4d0 msgr2=0x7f0d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d1406b4d0 0x7f0d1406d990 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f0d341f50d0 tx=0x7f0d20008040 comp rx=0 tx=0).stop 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 msgr2=0x7f0d340cbc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 0x7f0d340cbc50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f0d28009870 tx=0x7f0d280046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 shutdown_connections 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d1406b4d0 0x7f0d1406d990 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d34144440 0x7f0d340c97b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d340cd860 0x7f0d341f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 --2- 172.21.15.165:0/4116239238 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d34071530 0x7f0d340cbc50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 >> 172.21.15.165:0/4116239238 conn(0x7f0d3406abe0 msgr2=0x7f0d340c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 shutdown_connections 2024-04-25T10:03:16.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.700+0000 7f0d3a800640 1 -- 172.21.15.165:0/4116239238 wait complete. 2024-04-25T10:03:16.723 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:02:01.402047+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:02:02.584890+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-25T10:02:06.186614+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-25T10:03:16.724 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:16.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.915+0000 7f4826800640 1 Processor -- start 2024-04-25T10:03:16.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.915+0000 7f4826800640 1 -- start start 2024-04-25T10:03:16.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4826800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f482015dd30 0x7f48200cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4826800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482015e110 con 0x7f482006bff0 2024-04-25T10:03:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4826800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48200b2fd0 con 0x7f48200b5b40 2024-04-25T10:03:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4826800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482006d000 con 0x7f482015dd30 2024-04-25T10:03:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4824a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f482006bff0 0x7f48200b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40314/0 (socket says 172.21.15.165:40314) 2024-04-25T10:03:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4824a00640 1 -- 172.21.15.165:0/700207666 learned_addr learned my addr 172.21.15.165:0/700207666 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.917+0000 7f4825400640 1 --2- 172.21.15.165:0/700207666 >> v2:172.21.15.165:3300/0 conn(0x7f482015dd30 0x7f48200cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.918+0000 7f4825400640 1 -- 172.21.15.165:0/700207666 >> v1:172.21.15.172:6789/0 conn(0x7f48200b5b40 legacy=0x7f48200d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.918+0000 7f4825400640 1 -- 172.21.15.165:0/700207666 >> v1:172.21.15.165:6789/0 conn(0x7f482006bff0 legacy=0x7f48200b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.918+0000 7f4825400640 1 -- 172.21.15.165:0/700207666 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482006c480 con 0x7f482015dd30 2024-04-25T10:03:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.918+0000 7f481fe00640 1 -- 172.21.15.165:0/700207666 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2583239365 0 0) 0x7f482015e110 con 0x7f482006bff0 2024-04-25T10:03:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.919+0000 7f4825400640 1 --2- 172.21.15.165:0/700207666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cebd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f48140097a0 tx=0x7f481402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328a76b576c5e81b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.919+0000 7f481fe00640 1 -- 172.21.15.165:0/700207666 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814037070 con 0x7f482015dd30 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.919+0000 7f481fe00640 1 -- 172.21.15.165:0/700207666 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4814032440 con 0x7f482015dd30 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.919+0000 7f481fe00640 1 -- 172.21.15.165:0/700207666 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4814032810 con 0x7f482015dd30 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 -- 172.21.15.165:0/700207666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 msgr2=0x7f48200cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 --2- 172.21.15.165:0/700207666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cebd0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f48140097a0 tx=0x7f481402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 -- 172.21.15.165:0/700207666 shutdown_connections 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 --2- 172.21.15.165:0/700207666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cebd0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 -- 172.21.15.165:0/700207666 >> 172.21.15.165:0/700207666 conn(0x7f48200d0610 msgr2=0x7f48200d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 -- 172.21.15.165:0/700207666 shutdown_connections 2024-04-25T10:03:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.920+0000 7f4826800640 1 -- 172.21.15.165:0/700207666 wait complete. 2024-04-25T10:03:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.922+0000 7f4826800640 1 Processor -- start 2024-04-25T10:03:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.922+0000 7f4826800640 1 -- start start 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4826800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48200b5b40 0x7f48200b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4826800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4826800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48200cc010 0x7f4820151a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4826800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482006cd20 con 0x7f482015dd30 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4826800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482006ca40 con 0x7f48200b5b40 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4826800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48200cf110 con 0x7f48200cc010 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43556/0 (socket says 172.21.15.165:43556) 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 -- 172.21.15.165:0/3254101289 learned_addr learned my addr 172.21.15.165:0/3254101289 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 -- 172.21.15.165:0/3254101289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48200cc010 msgr2=0x7f4820151a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48200cc010 0x7f4820151a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 -- 172.21.15.165:0/3254101289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48200b5b40 msgr2=0x7f48200b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48200b5b40 0x7f48200b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.923+0000 7f4824a00640 1 -- 172.21.15.165:0/3254101289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200ccf50 con 0x7f482015dd30 2024-04-25T10:03:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.924+0000 7f4824a00640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cbad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c500 tx=0x7f480c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.924+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c018070 con 0x7f482015dd30 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.924+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f480c007580 con 0x7f482015dd30 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.924+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f480c004a20 con 0x7f482015dd30 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.924+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48200cc6b0 con 0x7f482015dd30 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.924+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48200ccc70 con 0x7f482015dd30 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.925+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4820073990 con 0x7f482015dd30 2024-04-25T10:03:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.926+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f480c004bc0 con 0x7f482015dd30 2024-04-25T10:03:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.927+0000 7f481d600640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f480006b4d0 0x7f480006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.927+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f480c0898c0 con 0x7f482015dd30 2024-04-25T10:03:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.927+0000 7f4825400640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f480006b4d0 0x7f480006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.928+0000 7f4825400640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f480006b4d0 0x7f480006d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4814034000 tx=0x7f481402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:16.931+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f480c058330 con 0x7f482015dd30 2024-04-25T10:03:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.126+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48200d16c0 con 0x7f482015dd30 2024-04-25T10:03:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.127+0000 7f481d600640 1 -- 172.21.15.165:0/3254101289 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7f480c057c30 con 0x7f482015dd30 2024-04-25T10:03:17.131 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:17.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f480006b4d0 msgr2=0x7f480006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f480006b4d0 0x7f480006d990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f4814034000 tx=0x7f481402c000 comp rx=0 tx=0).stop 2024-04-25T10:03:17.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 msgr2=0x7f48200cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cbad0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f480c00c500 tx=0x7f480c00c9c0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 shutdown_connections 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f480006b4d0 0x7f480006d990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48200cc010 0x7f4820151a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f482015dd30 0x7f48200cbad0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 --2- 172.21.15.165:0/3254101289 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48200b5b40 0x7f48200b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.129+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 >> 172.21.15.165:0/3254101289 conn(0x7f48200d0610 msgr2=0x7f48200b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.130+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 shutdown_connections 2024-04-25T10:03:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.130+0000 7f4826800640 1 -- 172.21.15.165:0/3254101289 wait complete. 2024-04-25T10:03:17.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:03:17.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.350+0000 7ff139000640 1 Processor -- start 2024-04-25T10:03:17.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.350+0000 7ff139000640 1 -- start start 2024-04-25T10:03:17.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff139000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 0x7ff1340cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff139000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff13415b270 0x7ff13415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff139000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff134072280 con 0x7ff1340cc4f0 2024-04-25T10:03:17.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff139000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff134071270 con 0x7ff1340ce230 2024-04-25T10:03:17.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff139000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340714c0 con 0x7ff13415b270 2024-04-25T10:03:17.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff131400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 0x7ff1340cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.352+0000 7ff132800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff13415b270 0x7ff13415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff131400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 0x7ff1340cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:47422/0 (socket says 172.21.15.165:47422) 2024-04-25T10:03:17.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff131400640 1 -- 172.21.15.165:0/3276604405 learned_addr learned my addr 172.21.15.165:0/3276604405 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:17.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff131e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7ff1340cc4f0 0x7ff1340cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:60164/0 (socket says 172.21.15.165:60164) 2024-04-25T10:03:17.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff132800640 1 -- 172.21.15.165:0/3276604405 >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 msgr2=0x7ff1340cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff132800640 1 --2- 172.21.15.165:0/3276604405 >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 0x7ff1340cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff132800640 1 -- 172.21.15.165:0/3276604405 >> v1:172.21.15.172:6789/0 conn(0x7ff1340cc4f0 legacy=0x7ff1340cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff132800640 1 -- 172.21.15.165:0/3276604405 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340717a0 con 0x7ff13415b270 2024-04-25T10:03:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.353+0000 7ff131400640 1 --2- 172.21.15.165:0/3276604405 >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 0x7ff1340cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.355+0000 7ff132800640 1 --2- 172.21.15.165:0/3276604405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff13415b270 0x7ff13415b6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009d90 tx=0x7ff12c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ec701d75b788213 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.355+0000 7ff130a00640 1 -- 172.21.15.165:0/3276604405 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c007740 con 0x7ff13415b270 2024-04-25T10:03:17.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.355+0000 7ff130a00640 1 -- 172.21.15.165:0/3276604405 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c0078e0 con 0x7ff13415b270 2024-04-25T10:03:17.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.355+0000 7ff130a00640 1 -- 172.21.15.165:0/3276604405 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c007cb0 con 0x7ff13415b270 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 -- 172.21.15.165:0/3276604405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff13415b270 msgr2=0x7ff13415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 --2- 172.21.15.165:0/3276604405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff13415b270 0x7ff13415b6b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009d90 tx=0x7ff12c02a660 comp rx=0 tx=0).stop 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 -- 172.21.15.165:0/3276604405 shutdown_connections 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 --2- 172.21.15.165:0/3276604405 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff13415b270 0x7ff13415b6b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 --2- 172.21.15.165:0/3276604405 >> v2:172.21.15.172:3300/0 conn(0x7ff1340ce230 0x7ff1340cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 -- 172.21.15.165:0/3276604405 >> 172.21.15.165:0/3276604405 conn(0x7ff13406abe0 msgr2=0x7ff13406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 -- 172.21.15.165:0/3276604405 shutdown_connections 2024-04-25T10:03:17.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.356+0000 7ff139000640 1 -- 172.21.15.165:0/3276604405 wait complete. 2024-04-25T10:03:17.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.358+0000 7ff139000640 1 Processor -- start 2024-04-25T10:03:17.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.358+0000 7ff139000640 1 -- start start 2024-04-25T10:03:17.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff139000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1340cc4f0 0x7ff13414d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff139000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1340ce230 0x7ff13414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff139000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 0x7ff13413dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff139000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415bbf0 con 0x7ff13415b270 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff139000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff134071d60 con 0x7ff1340cc4f0 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff139000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13415bd70 con 0x7ff1340ce230 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff132800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 0x7ff13413dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff132800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 0x7ff13413dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43566/0 (socket says 172.21.15.165:43566) 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.359+0000 7ff132800640 1 -- 172.21.15.165:0/3461868655 learned_addr learned my addr 172.21.15.165:0/3461868655 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff131e00640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1340cc4f0 0x7ff13414d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff131400640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1340ce230 0x7ff13414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff132800640 1 -- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1340ce230 msgr2=0x7ff13414f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff132800640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1340ce230 0x7ff13414f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff132800640 1 -- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1340cc4f0 msgr2=0x7ff13414d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff132800640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1340cc4f0 0x7ff13414d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff132800640 1 -- 172.21.15.165:0/3461868655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13414b770 con 0x7ff13415b270 2024-04-25T10:03:17.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff132800640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 0x7ff13413dd10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02ac50 tx=0x7ff12c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.360+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c040070 con 0x7ff13415b270 2024-04-25T10:03:17.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.361+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c031c60 con 0x7ff13415b270 2024-04-25T10:03:17.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.361+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c009450 con 0x7ff13415b270 2024-04-25T10:03:17.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.361+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1341f6ff0 con 0x7ff13415b270 2024-04-25T10:03:17.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.361+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1341f75b0 con 0x7ff13415b270 2024-04-25T10:03:17.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.362+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff134073a00 con 0x7ff13415b270 2024-04-25T10:03:17.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.363+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff12c039040 con 0x7ff13415b270 2024-04-25T10:03:17.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.364+0000 7ff12a000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff11406b4d0 0x7ff11406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.364+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7ff12c0ab780 con 0x7ff13415b270 2024-04-25T10:03:17.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.364+0000 7ff131e00640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff11406b4d0 0x7ff11406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.365+0000 7ff131e00640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff11406b4d0 0x7ff11406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff124000f60 tx=0x7ff124007ce0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.368+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff12c042310 con 0x7ff13415b270 2024-04-25T10:03:17.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.564+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1341517e0 con 0x7ff13415b270 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.565+0000 7ff12a000640 1 -- 172.21.15.165:0/3461868655 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff12c07a160 con 0x7ff13415b270 2024-04-25T10:03:17.570 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff11406b4d0 msgr2=0x7ff11406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff11406b4d0 0x7ff11406d990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff124000f60 tx=0x7ff124007ce0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 msgr2=0x7ff13413dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 0x7ff13413dd10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7ff12c02ac50 tx=0x7ff12c004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 shutdown_connections 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff11406b4d0 0x7ff11406d990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff13415b270 0x7ff13413dd10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1340ce230 0x7ff13414f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 --2- 172.21.15.165:0/3461868655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1340cc4f0 0x7ff13414d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 >> 172.21.15.165:0/3461868655 conn(0x7ff13406abe0 msgr2=0x7ff13406e120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 shutdown_connections 2024-04-25T10:03:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.568+0000 7ff139000640 1 -- 172.21.15.165:0/3461868655 wait complete. 2024-04-25T10:03:17.591 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:03:17.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.788+0000 7fca7e600640 1 Processor -- start 2024-04-25T10:03:17.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.789+0000 7fca7e600640 1 -- start start 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca7e600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fca780ce230 0x7fca780cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca7e600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca78072280 con 0x7fca780cc4f0 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca7e600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca780714c0 con 0x7fca7815b270 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca7e600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca78071270 con 0x7fca780ce230 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca77400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fca780cc4f0 0x7fca780cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40324/0 (socket says 172.21.15.165:40324) 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca77400640 1 -- 172.21.15.165:0/1830286404 learned_addr learned my addr 172.21.15.165:0/1830286404 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:17.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.790+0000 7fca6fe00640 1 --2- 172.21.15.165:0/1830286404 >> v2:172.21.15.165:3300/0 conn(0x7fca780ce230 0x7fca780cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.791+0000 7fca76a00640 1 -- 172.21.15.165:0/1830286404 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513742845 0 0) 0x7fca78072280 con 0x7fca780cc4f0 2024-04-25T10:03:17.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.791+0000 7fca76a00640 1 -- 172.21.15.165:0/1830286404 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca5c003520 con 0x7fca780cc4f0 2024-04-25T10:03:17.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.791+0000 7fca6fe00640 1 -- 172.21.15.165:0/1830286404 >> v1:172.21.15.172:6789/0 conn(0x7fca7815b270 legacy=0x7fca7815b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:17.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.791+0000 7fca6fe00640 1 -- 172.21.15.165:0/1830286404 >> v1:172.21.15.165:6789/0 conn(0x7fca780cc4f0 legacy=0x7fca780cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.791+0000 7fca6fe00640 1 -- 172.21.15.165:0/1830286404 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca780717a0 con 0x7fca780ce230 2024-04-25T10:03:17.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.792+0000 7fca6fe00640 1 --2- 172.21.15.165:0/1830286404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780ce230 0x7fca780cc8d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fca600097a0 tx=0x7fca6002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ecc425e0d0dc85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.792+0000 7fca76a00640 1 -- 172.21.15.165:0/1830286404 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60037070 con 0x7fca780ce230 2024-04-25T10:03:17.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.792+0000 7fca76a00640 1 -- 172.21.15.165:0/1830286404 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca60032440 con 0x7fca780ce230 2024-04-25T10:03:17.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.792+0000 7fca76a00640 1 -- 172.21.15.165:0/1830286404 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca60032810 con 0x7fca780ce230 2024-04-25T10:03:17.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 -- 172.21.15.165:0/1830286404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780ce230 msgr2=0x7fca780cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 --2- 172.21.15.165:0/1830286404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780ce230 0x7fca780cc8d0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fca600097a0 tx=0x7fca6002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 -- 172.21.15.165:0/1830286404 shutdown_connections 2024-04-25T10:03:17.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 --2- 172.21.15.165:0/1830286404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780ce230 0x7fca780cc8d0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 -- 172.21.15.165:0/1830286404 >> 172.21.15.165:0/1830286404 conn(0x7fca7806abe0 msgr2=0x7fca7806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:17.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 -- 172.21.15.165:0/1830286404 shutdown_connections 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.793+0000 7fca7e600640 1 -- 172.21.15.165:0/1830286404 wait complete. 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.795+0000 7fca7e600640 1 Processor -- start 2024-04-25T10:03:17.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.795+0000 7fca7e600640 1 -- start start 2024-04-25T10:03:17.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca7e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 0x7fca780b1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca7e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca780ce230 0x7fca781f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca7e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca7815b270 0x7fca781f3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca7e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7806e8a0 con 0x7fca780cc4f0 2024-04-25T10:03:17.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca7e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca78071d60 con 0x7fca780ce230 2024-04-25T10:03:17.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca7e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca780ae500 con 0x7fca7815b270 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca77400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 0x7fca780b1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca77400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 0x7fca780b1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43578/0 (socket says 172.21.15.165:43578) 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.796+0000 7fca77400640 1 -- 172.21.15.165:0/3060452488 learned_addr learned my addr 172.21.15.165:0/3060452488 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca77400640 1 -- 172.21.15.165:0/3060452488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca7815b270 msgr2=0x7fca781f3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca77400640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca7815b270 0x7fca781f3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca77400640 1 -- 172.21.15.165:0/3060452488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca780ce230 msgr2=0x7fca781f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca77400640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca780ce230 0x7fca781f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca77400640 1 -- 172.21.15.165:0/3060452488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca781511e0 con 0x7fca780cc4f0 2024-04-25T10:03:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca77400640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 0x7fca780b1230 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca6800c3c0 tx=0x7fca6800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca68017070 con 0x7fca780cc4f0 2024-04-25T10:03:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca68005ce0 con 0x7fca780cc4f0 2024-04-25T10:03:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca680072f0 con 0x7fca780cc4f0 2024-04-25T10:03:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.797+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca780ad780 con 0x7fca780cc4f0 2024-04-25T10:03:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.798+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca780add40 con 0x7fca780cc4f0 2024-04-25T10:03:17.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.799+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca78073a00 con 0x7fca780cc4f0 2024-04-25T10:03:17.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.800+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca6801a070 con 0x7fca780cc4f0 2024-04-25T10:03:17.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.801+0000 7fca6f400640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca5406b4d0 0x7fca5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:17.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.801+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fca680899d0 con 0x7fca780cc4f0 2024-04-25T10:03:17.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.801+0000 7fca6fe00640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca5406b4d0 0x7fca5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.801+0000 7fca6fe00640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca5406b4d0 0x7fca5406d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca60034000 tx=0x7fca6002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:17.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:17.805+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca68058450 con 0x7fca780cc4f0 2024-04-25T10:03:18.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.000+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca7806d3a0 con 0x7fca780cc4f0 2024-04-25T10:03:18.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.001+0000 7fca6f400640 1 -- 172.21.15.165:0/3060452488 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca68057d50 con 0x7fca780cc4f0 2024-04-25T10:03:18.005 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:03:18.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.003+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca5406b4d0 msgr2=0x7fca5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.003+0000 7fca7e600640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca5406b4d0 0x7fca5406d990 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca60034000 tx=0x7fca6002c000 comp rx=0 tx=0).stop 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.003+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 msgr2=0x7fca780b1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.003+0000 7fca7e600640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 0x7fca780b1230 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fca6800c3c0 tx=0x7fca6800c880 comp rx=0 tx=0).stop 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 shutdown_connections 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca5406b4d0 0x7fca5406d990 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca7815b270 0x7fca781f3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca780ce230 0x7fca781f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 --2- 172.21.15.165:0/3060452488 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca780cc4f0 0x7fca780b1230 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 >> 172.21.15.165:0/3060452488 conn(0x7fca7806abe0 msgr2=0x7fca78159370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 shutdown_connections 2024-04-25T10:03:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.004+0000 7fca7e600640 1 -- 172.21.15.165:0/3060452488 wait complete. 2024-04-25T10:03:18.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-25T10:03:18.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.229+0000 7f6849600640 1 Processor -- start 2024-04-25T10:03:18.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.229+0000 7f6849600640 1 -- start start 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f6849600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f684406ec50 0x7f6844071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f6849600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68440cfad0 con 0x7f68440cf640 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f6849600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68440cfd80 con 0x7f684415b710 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f6849600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f684415e490 con 0x7f684406ec50 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f6843400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f684406ec50 0x7f6844071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f683a000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f68440cf640 0x7f68440ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40336/0 (socket says 172.21.15.165:40336) 2024-04-25T10:03:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.231+0000 7f683a000640 1 -- 172.21.15.165:0/229099697 learned_addr learned my addr 172.21.15.165:0/229099697 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:18.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.232+0000 7f6843400640 1 -- 172.21.15.165:0/229099697 >> v1:172.21.15.172:6789/0 conn(0x7f684415b710 legacy=0x7f684415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.232+0000 7f6843400640 1 -- 172.21.15.165:0/229099697 >> v1:172.21.15.165:6789/0 conn(0x7f68440cf640 legacy=0x7f68440ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.232+0000 7f6843400640 1 -- 172.21.15.165:0/229099697 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440d0060 con 0x7f684406ec50 2024-04-25T10:03:18.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.233+0000 7f6843400640 1 --2- 172.21.15.165:0/229099697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f684406ec50 0x7f6844071bc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d90 tx=0x7f683c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701a1f6645a0425d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.233+0000 7f6842000640 1 -- 172.21.15.165:0/229099697 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c007740 con 0x7f684406ec50 2024-04-25T10:03:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.233+0000 7f6842000640 1 -- 172.21.15.165:0/229099697 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f683c0078e0 con 0x7f684406ec50 2024-04-25T10:03:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.233+0000 7f6842000640 1 -- 172.21.15.165:0/229099697 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683c007cb0 con 0x7f684406ec50 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 -- 172.21.15.165:0/229099697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f684406ec50 msgr2=0x7f6844071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 --2- 172.21.15.165:0/229099697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f684406ec50 0x7f6844071bc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f683c009d90 tx=0x7f683c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 -- 172.21.15.165:0/229099697 shutdown_connections 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 --2- 172.21.15.165:0/229099697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f684406ec50 0x7f6844071bc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 -- 172.21.15.165:0/229099697 >> 172.21.15.165:0/229099697 conn(0x7f684406abe0 msgr2=0x7f684406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 -- 172.21.15.165:0/229099697 shutdown_connections 2024-04-25T10:03:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.234+0000 7f6849600640 1 -- 172.21.15.165:0/229099697 wait complete. 2024-04-25T10:03:18.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.236+0000 7f6849600640 1 Processor -- start 2024-04-25T10:03:18.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.236+0000 7f6849600640 1 -- start start 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f6849600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f684406ec50 0x7f68440afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f6849600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 0x7f6844155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f6849600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f684415b710 0x7f6844156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f6849600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684415baf0 con 0x7f68440cf640 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f6849600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684406f0a0 con 0x7f684406ec50 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f6849600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684415bc70 con 0x7f684415b710 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f683a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 0x7f6844155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f683a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 0x7f6844155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43594/0 (socket says 172.21.15.165:43594) 2024-04-25T10:03:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.237+0000 7f683a000640 1 -- 172.21.15.165:0/3411155115 learned_addr learned my addr 172.21.15.165:0/3411155115 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f6842a00640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f684406ec50 0x7f68440afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683a000640 1 -- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f684415b710 msgr2=0x7f6844156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683a000640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f684415b710 0x7f6844156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683a000640 1 -- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f684406ec50 msgr2=0x7f68440afd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683a000640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f684406ec50 0x7f68440afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683a000640 1 -- 172.21.15.165:0/3411155115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68440b1170 con 0x7f68440cf640 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683a000640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 0x7f6844155140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c400 tx=0x7f682c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c017070 con 0x7f68440cf640 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c005ce0 con 0x7f68440cf640 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68440b05f0 con 0x7f68440cf640 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c0072f0 con 0x7f68440cf640 2024-04-25T10:03:18.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.238+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68440b0e90 con 0x7f68440cf640 2024-04-25T10:03:18.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.240+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6844073a00 con 0x7f68440cf640 2024-04-25T10:03:18.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.240+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f682c01a070 con 0x7f68440cf640 2024-04-25T10:03:18.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.241+0000 7f683b400640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682006b4d0 0x7f682006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.241+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f682c089a90 con 0x7f68440cf640 2024-04-25T10:03:18.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.242+0000 7f6842a00640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682006b4d0 0x7f682006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.243+0000 7f6842a00640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682006b4d0 0x7f682006d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f683400a750 tx=0x7f6834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.245+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68440b1170 con 0x7f68440cf640 2024-04-25T10:03:18.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.440+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f684406b440 con 0x7f68440cf640 2024-04-25T10:03:18.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.441+0000 7f683b400640 1 -- 172.21.15.165:0/3411155115 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v26) v1 ==== 103+0+11 (secure 0 0 0) 0x7f682c058470 con 0x7f68440cf640 2024-04-25T10:03:18.446 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:03:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682006b4d0 msgr2=0x7f682006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682006b4d0 0x7f682006d990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f683400a750 tx=0x7f6834008040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 msgr2=0x7f6844155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 0x7f6844155140 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f682c00c400 tx=0x7f682c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 shutdown_connections 2024-04-25T10:03:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682006b4d0 0x7f682006d990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f684415b710 0x7f6844156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68440cf640 0x7f6844155140 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 --2- 172.21.15.165:0/3411155115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f684406ec50 0x7f68440afd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 >> 172.21.15.165:0/3411155115 conn(0x7f684406abe0 msgr2=0x7f68440ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 shutdown_connections 2024-04-25T10:03:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.444+0000 7f6849600640 1 -- 172.21.15.165:0/3411155115 wait complete. 2024-04-25T10:03:18.469 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.662+0000 7f91d1600640 1 Processor -- start 2024-04-25T10:03:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.662+0000 7f91d1600640 1 -- start start 2024-04-25T10:03:18.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91d1600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f91cc0ce590 0x7f91cc165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91d1600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc071480 con 0x7f91cc0cfec0 2024-04-25T10:03:18.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91d1600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91cc071760 con 0x7f91cc154ee0 2024-04-25T10:03:18.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91d1600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91cc072090 con 0x7f91cc0ce590 2024-04-25T10:03:18.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91ca000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f91cc0cfec0 0x7f91cc0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40342/0 (socket says 172.21.15.165:40342) 2024-04-25T10:03:18.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91ca000640 1 -- 172.21.15.165:0/1924640441 learned_addr learned my addr 172.21.15.165:0/1924640441 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91cb400640 1 --2- 172.21.15.165:0/1924640441 >> v2:172.21.15.172:3300/0 conn(0x7f91cc0ce590 0x7f91cc165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.664+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2981672765 0 0) 0x7f91cc071480 con 0x7f91cc0cfec0 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91b0003520 con 0x7f91cc0cfec0 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 901043081 0 0) 0x7f91b0003520 con 0x7f91cc0cfec0 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 >> v2:172.21.15.172:3300/0 conn(0x7f91cc0ce590 msgr2=0x7f91cc165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 --2- 172.21.15.165:0/1924640441 >> v2:172.21.15.172:3300/0 conn(0x7f91cc0ce590 0x7f91cc165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 >> v1:172.21.15.172:6789/0 conn(0x7f91cc154ee0 legacy=0x7f91cc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc071a40 con 0x7f91cc0cfec0 2024-04-25T10:03:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f91b8001670 con 0x7f91cc0cfec0 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 >> v1:172.21.15.165:6789/0 conn(0x7f91cc0cfec0 legacy=0x7f91cc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91b00039b0 0x7f91b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b0004800 0x7f91b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000c030 con 0x7f91b00051c0 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000c1e0 con 0x7f91b0004800 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.665+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b000e5f0 con 0x7f91b00039b0 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91cb400640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91cb400640 1 -- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91b00039b0 msgr2=0x7f91b0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91ca000640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b0004800 0x7f91b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91cb400640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91b00039b0 0x7f91b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91cb400640 1 -- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b0004800 msgr2=0x7f91b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91cb400640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91b0004800 0x7f91b0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.666+0000 7f91cb400640 1 -- 172.21.15.165:0/1924640441 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0003520 con 0x7f91b00051c0 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.667+0000 7f91cb400640 1 -- 172.21.15.165:0/1924640441 reap_dead start 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.667+0000 7f91cb400640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91b000b9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009f90 tx=0x7f91b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8901f9bb9e93b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.667+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4035020 con 0x7f91b00051c0 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.667+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c4003b90 con 0x7f91b00051c0 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.667+0000 7f91c9600640 1 -- 172.21.15.165:0/1924640441 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4009c10 con 0x7f91b00051c0 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 -- 172.21.15.165:0/1924640441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 msgr2=0x7f91b000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91b000b9c0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f91c4009f90 tx=0x7f91b0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 -- 172.21.15.165:0/1924640441 shutdown_connections 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 --2- 172.21.15.165:0/1924640441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91b000b9c0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 -- 172.21.15.165:0/1924640441 >> 172.21.15.165:0/1924640441 conn(0x7f91cc06abe0 msgr2=0x7f91cc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 -- 172.21.15.165:0/1924640441 shutdown_connections 2024-04-25T10:03:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.668+0000 7f91d1600640 1 -- 172.21.15.165:0/1924640441 wait complete. 2024-04-25T10:03:18.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.670+0000 7f91d1600640 1 Processor -- start 2024-04-25T10:03:18.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.670+0000 7f91d1600640 1 -- start start 2024-04-25T10:03:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91d1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91cc158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91d1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91cc151610 0x7f91cc148220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91d1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91cc1576b0 0x7f91cc157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91d1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc1552c0 con 0x7f91b00051c0 2024-04-25T10:03:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91d1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc071230 con 0x7f91cc1576b0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91d1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91cc155440 con 0x7f91cc151610 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91cc158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91cc158c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43618/0 (socket says 172.21.15.165:43618) 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.671+0000 7f91caa00640 1 -- 172.21.15.165:0/4242015977 learned_addr learned my addr 172.21.15.165:0/4242015977 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91ca000640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91cc151610 0x7f91cc148220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91caa00640 1 -- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91cc151610 msgr2=0x7f91cc148220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91caa00640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91cc151610 0x7f91cc148220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91cb400640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91cc1576b0 0x7f91cc157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91caa00640 1 -- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91cc1576b0 msgr2=0x7f91cc157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91caa00640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91cc1576b0 0x7f91cc157b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91caa00640 1 -- 172.21.15.165:0/4242015977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91cc156610 con 0x7f91b00051c0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91caa00640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91cc158c60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c430 tx=0x7f91bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0052d0 con 0x7f91b00051c0 2024-04-25T10:03:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc007020 con 0x7f91b00051c0 2024-04-25T10:03:18.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc0054b0 con 0x7f91b00051c0 2024-04-25T10:03:18.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.672+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91cc1568f0 con 0x7f91b00051c0 2024-04-25T10:03:18.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.673+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91cc156eb0 con 0x7f91b00051c0 2024-04-25T10:03:18.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.674+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91cc0ae200 con 0x7f91b00051c0 2024-04-25T10:03:18.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.675+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91bc007410 con 0x7f91b00051c0 2024-04-25T10:03:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.676+0000 7f91c2a00640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91ac06b500 0x7f91ac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.676+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f91bc089a20 con 0x7f91b00051c0 2024-04-25T10:03:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.676+0000 7f91ca000640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91ac06b500 0x7f91ac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:18.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.677+0000 7f91ca000640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91ac06b500 0x7f91ac06d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ad9e0 tx=0x7f91b8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:18.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.680+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91bc0584a0 con 0x7f91b00051c0 2024-04-25T10:03:18.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.920+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91cc06d360 con 0x7f91b00051c0 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.922+0000 7f91c2a00640 1 -- 172.21.15.165:0/4242015977 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f91bc057da0 con 0x7f91b00051c0 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:18.926 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:18.927 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91ac06b500 msgr2=0x7f91ac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91ac06b500 0x7f91ac06d9c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f91cc0ad9e0 tx=0x7f91b8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 msgr2=0x7f91cc158c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91cc158c60 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f91bc00c430 tx=0x7f91bc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 shutdown_connections 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91ac06b500 0x7f91ac06d9c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91cc1576b0 0x7f91cc157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91cc151610 0x7f91cc148220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 --2- 172.21.15.165:0/4242015977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91b00051c0 0x7f91cc158c60 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:18.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.924+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 >> 172.21.15.165:0/4242015977 conn(0x7f91cc06abe0 msgr2=0x7f91cc15c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:18.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.925+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 shutdown_connections 2024-04-25T10:03:18.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:18.925+0000 7f91d1600640 1 -- 172.21.15.165:0/4242015977 wait complete. 2024-04-25T10:03:18.948 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:19.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.140+0000 7f6659800640 1 Processor -- start 2024-04-25T10:03:19.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.140+0000 7f6659800640 1 -- start start 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6659800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f66540ce920 0x7f66540cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6659800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f665406d290 0x7f665406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6659800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6654144b30 con 0x7f6654144750 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6659800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66540712d0 con 0x7f665406d290 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6659800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66540715b0 con 0x7f66540ce920 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6653400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f665406d290 0x7f665406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6653400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f665406d290 0x7f665406d6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43630/0 (socket says 172.21.15.165:43630) 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6653400640 1 -- 172.21.15.165:0/126079667 learned_addr learned my addr 172.21.15.165:0/126079667 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:19.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.142+0000 7f6652000640 1 --2- 172.21.15.165:0/126079667 >> v2:172.21.15.172:3300/0 conn(0x7f66540ce920 0x7f66540cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.143+0000 7f6653400640 1 -- 172.21.15.165:0/126079667 >> v2:172.21.15.172:3300/0 conn(0x7f66540ce920 msgr2=0x7f66540cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.143+0000 7f6653400640 1 --2- 172.21.15.165:0/126079667 >> v2:172.21.15.172:3300/0 conn(0x7f66540ce920 0x7f66540cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.143+0000 7f6653400640 1 -- 172.21.15.165:0/126079667 >> v1:172.21.15.172:6789/0 conn(0x7f6654144750 legacy=0x7f665406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.143+0000 7f6653400640 1 -- 172.21.15.165:0/126079667 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654071890 con 0x7f665406d290 2024-04-25T10:03:19.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.144+0000 7f6653400640 1 --2- 172.21.15.165:0/126079667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665406d290 0x7f665406d6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66541606b0 tx=0x7f664c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b970aa4341b48d6b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.144+0000 7f6651600640 1 -- 172.21.15.165:0/126079667 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c036070 con 0x7f665406d290 2024-04-25T10:03:19.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.144+0000 7f6651600640 1 -- 172.21.15.165:0/126079667 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c031600 con 0x7f665406d290 2024-04-25T10:03:19.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6651600640 1 -- 172.21.15.165:0/126079667 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c0319d0 con 0x7f665406d290 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 -- 172.21.15.165:0/126079667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665406d290 msgr2=0x7f665406d6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 --2- 172.21.15.165:0/126079667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665406d290 0x7f665406d6c0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66541606b0 tx=0x7f664c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 -- 172.21.15.165:0/126079667 shutdown_connections 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 --2- 172.21.15.165:0/126079667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f665406d290 0x7f665406d6c0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 --2- 172.21.15.165:0/126079667 >> v2:172.21.15.172:3300/0 conn(0x7f66540ce920 0x7f66540cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 -- 172.21.15.165:0/126079667 >> 172.21.15.165:0/126079667 conn(0x7f665406abe0 msgr2=0x7f665406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 -- 172.21.15.165:0/126079667 shutdown_connections 2024-04-25T10:03:19.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.145+0000 7f6659800640 1 -- 172.21.15.165:0/126079667 wait complete. 2024-04-25T10:03:19.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.147+0000 7f6659800640 1 Processor -- start 2024-04-25T10:03:19.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.147+0000 7f6659800640 1 -- start start 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6659800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f665406d290 0x7f665413e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6659800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 0x7f665413f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6659800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6654144750 0x7f665413fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6659800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665406f020 con 0x7f66540ce920 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6659800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665406ece0 con 0x7f665406d290 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6659800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6654145450 con 0x7f6654144750 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.148+0000 7f6652000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 0x7f665413f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 0x7f665413f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43634/0 (socket says 172.21.15.165:43634) 2024-04-25T10:03:19.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 -- 172.21.15.165:0/1348084771 learned_addr learned my addr 172.21.15.165:0/1348084771 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652a00640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f665406d290 0x7f665413e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6653400640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6654144750 0x7f665413fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 -- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6654144750 msgr2=0x7f665413fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6654144750 0x7f665413fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 -- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f665406d290 msgr2=0x7f665413e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f665406d290 0x7f665413e0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 -- 172.21.15.165:0/1348084771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540af860 con 0x7f66540ce920 2024-04-25T10:03:19.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652a00640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f665406d290 0x7f665413e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.149+0000 7f6652000640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 0x7f665413f940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f664000d620 tx=0x7f664000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.150+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000b620 con 0x7f66540ce920 2024-04-25T10:03:19.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.150+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6640019070 con 0x7f66540ce920 2024-04-25T10:03:19.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.150+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6654157330 con 0x7f66540ce920 2024-04-25T10:03:19.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.150+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664000ba00 con 0x7f66540ce920 2024-04-25T10:03:19.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.150+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665415cfd0 con 0x7f66540ce920 2024-04-25T10:03:19.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.151+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6654073990 con 0x7f66540ce920 2024-04-25T10:03:19.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.152+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f664000bba0 con 0x7f66540ce920 2024-04-25T10:03:19.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.153+0000 7f664aa00640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f663406b4d0 0x7f663406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.153+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f664008aae0 con 0x7f66540ce920 2024-04-25T10:03:19.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.153+0000 7f6652a00640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f663406b4d0 0x7f663406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.154+0000 7f6652a00640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f663406b4d0 0x7f663406d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6644003eb0 tx=0x7f6644009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.158+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f66540af860 con 0x7f66540ce920 2024-04-25T10:03:19.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.352+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66540d0510 con 0x7f66540ce920 2024-04-25T10:03:19.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.356+0000 7f664aa00640 1 -- 172.21.15.165:0/1348084771 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7f66400594c0 con 0x7f66540ce920 2024-04-25T10:03:19.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:19.363 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f663406b4d0 msgr2=0x7f663406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f663406b4d0 0x7f663406d990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6644003eb0 tx=0x7f6644009040 comp rx=0 tx=0).stop 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 msgr2=0x7f665413f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 0x7f665413f940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f664000d620 tx=0x7f664000dae0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 shutdown_connections 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f663406b4d0 0x7f663406d990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6654144750 0x7f665413fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66540ce920 0x7f665413f940 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 --2- 172.21.15.165:0/1348084771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f665406d290 0x7f665413e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 >> 172.21.15.165:0/1348084771 conn(0x7f665406abe0 msgr2=0x7f665406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 shutdown_connections 2024-04-25T10:03:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.359+0000 7f6659800640 1 -- 172.21.15.165:0/1348084771 wait complete. 2024-04-25T10:03:19.383 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:03:19.383 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.571+0000 7f524be00640 1 Processor -- start 2024-04-25T10:03:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.571+0000 7f524be00640 1 -- start start 2024-04-25T10:03:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.572+0000 7f524be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f524c143e60 0x7f524c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.572+0000 7f524be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c144240 con 0x7f524c06e6a0 2024-04-25T10:03:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f524be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524c15e220 con 0x7f524c0cca10 2024-04-25T10:03:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f524be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f524c15e500 con 0x7f524c143e60 2024-04-25T10:03:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f5242000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f524c06e6a0 0x7f524c0cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40346/0 (socket says 172.21.15.165:40346) 2024-04-25T10:03:19.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f5242000640 1 -- 172.21.15.165:0/4076678533 learned_addr learned my addr 172.21.15.165:0/4076678533 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:19.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f524aa00640 1 --2- 172.21.15.165:0/4076678533 >> v2:172.21.15.172:3300/0 conn(0x7f524c143e60 0x7f524c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815702468 0 0) 0x7f524c144240 con 0x7f524c06e6a0 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.573+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5230003520 con 0x7f524c06e6a0 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 856122967 0 0) 0x7f5230003520 con 0x7f524c06e6a0 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 >> v2:172.21.15.172:3300/0 conn(0x7f524c143e60 msgr2=0x7f524c06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 --2- 172.21.15.165:0/4076678533 >> v2:172.21.15.172:3300/0 conn(0x7f524c143e60 0x7f524c06e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 >> v1:172.21.15.172:6789/0 conn(0x7f524c0cca10 legacy=0x7f524c15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c15e7e0 con 0x7f524c06e6a0 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5234001670 con 0x7f524c06e6a0 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 >> v1:172.21.15.165:6789/0 conn(0x7f524c06e6a0 legacy=0x7f524c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5230003a00 0x7f5230003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f5230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300051c0 0x7f523000dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c15e500 con 0x7f5230004800 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523000e2d0 con 0x7f52300051c0 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523000e4d0 con 0x7f5230003a00 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.574+0000 7f5242000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f5230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f524aa00640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5230003a00 0x7f5230003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f524b400640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300051c0 0x7f523000dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f5242000640 1 -- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5230003a00 msgr2=0x7f5230003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f5242000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5230003a00 0x7f5230003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f5242000640 1 -- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300051c0 msgr2=0x7f523000dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f5242000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300051c0 0x7f523000dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f5242000640 1 -- 172.21.15.165:0/4076678533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5230003520 con 0x7f5230004800 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f524aa00640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5230003a00 0x7f5230003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f524b400640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52300051c0 0x7f523000dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.575+0000 7f524b400640 1 -- 172.21.15.165:0/4076678533 reap_dead start 2024-04-25T10:03:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.576+0000 7f5242000640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f5230004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f523400c110 tx=0x7f5230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1d9ab9b0be64e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.576+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234007c40 con 0x7f5230004800 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.576+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5234005af0 con 0x7f5230004800 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.576+0000 7f524a000640 1 -- 172.21.15.165:0/4076678533 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5234007070 con 0x7f5230004800 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 -- 172.21.15.165:0/4076678533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 msgr2=0x7f5230004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f5230004c80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f523400c110 tx=0x7f5230005830 comp rx=0 tx=0).stop 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 -- 172.21.15.165:0/4076678533 shutdown_connections 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 --2- 172.21.15.165:0/4076678533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f5230004c80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 -- 172.21.15.165:0/4076678533 >> 172.21.15.165:0/4076678533 conn(0x7f524c06abe0 msgr2=0x7f524c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 -- 172.21.15.165:0/4076678533 shutdown_connections 2024-04-25T10:03:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.577+0000 7f524be00640 1 -- 172.21.15.165:0/4076678533 wait complete. 2024-04-25T10:03:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.579+0000 7f524be00640 1 Processor -- start 2024-04-25T10:03:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.579+0000 7f524be00640 1 -- start start 2024-04-25T10:03:19.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f524c13dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c0b1c00 0x7f524c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524c13d810 0x7f524c13d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b2eb0 con 0x7f5230004800 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0b2d30 con 0x7f524c0b1c00 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524c0cc300 con 0x7f524c13d810 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f524c13dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f524c13dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43652/0 (socket says 172.21.15.165:43652) 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.580+0000 7f524aa00640 1 -- 172.21.15.165:0/169566533 learned_addr learned my addr 172.21.15.165:0/169566533 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f5242000640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c0b1c00 0x7f524c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524aa00640 1 -- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524c13d810 msgr2=0x7f524c13d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524aa00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524c13d810 0x7f524c13d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524aa00640 1 -- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c0b1c00 msgr2=0x7f524c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524aa00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c0b1c00 0x7f524c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524aa00640 1 -- 172.21.15.165:0/169566533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c15aa90 con 0x7f5230004800 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f5242000640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c0b1c00 0x7f524c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524aa00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f524c13dcf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f523c00f3b0 tx=0x7f523c00f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c00fea0 con 0x7f5230004800 2024-04-25T10:03:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523c003990 con 0x7f5230004800 2024-04-25T10:03:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523c003d60 con 0x7f5230004800 2024-04-25T10:03:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c15ad70 con 0x7f5230004800 2024-04-25T10:03:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.581+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c15b330 con 0x7f5230004800 2024-04-25T10:03:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.583+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c073990 con 0x7f5230004800 2024-04-25T10:03:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.584+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f523c005d10 con 0x7f5230004800 2024-04-25T10:03:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.584+0000 7f5243400640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522806b4d0 0x7f522806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.585+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f523c08afb0 con 0x7f5230004800 2024-04-25T10:03:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.585+0000 7f5242000640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522806b4d0 0x7f522806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:19.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.586+0000 7f5242000640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522806b4d0 0x7f522806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5234032000 tx=0x7f5234009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:19.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.589+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f523c024020 con 0x7f5230004800 2024-04-25T10:03:19.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.824+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f524c06e480 con 0x7f5230004800 2024-04-25T10:03:19.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.824+0000 7f5243400640 1 -- 172.21.15.165:0/169566533 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f523c059990 con 0x7f5230004800 2024-04-25T10:03:19.829 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:19.829 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:19.829 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:03:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522806b4d0 msgr2=0x7f522806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522806b4d0 0x7f522806d990 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5234032000 tx=0x7f5234009030 comp rx=0 tx=0).stop 2024-04-25T10:03:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 msgr2=0x7f524c13dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f524c13dcf0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f523c00f3b0 tx=0x7f523c00f870 comp rx=0 tx=0).stop 2024-04-25T10:03:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 shutdown_connections 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522806b4d0 0x7f522806d990 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524c13d810 0x7f524c13d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f524c0b1c00 0x7f524c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 --2- 172.21.15.165:0/169566533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5230004800 0x7f524c13dcf0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 >> 172.21.15.165:0/169566533 conn(0x7f524c06abe0 msgr2=0x7f524c141df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 shutdown_connections 2024-04-25T10:03:19.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:19.827+0000 7f524be00640 1 -- 172.21.15.165:0/169566533 wait complete. 2024-04-25T10:03:19.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:20.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.036+0000 7f986b200640 1 Processor -- start 2024-04-25T10:03:20.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.037+0000 7f986b200640 1 -- start start 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.038+0000 7f986b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c144440 0x7f986c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.038+0000 7f986b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f986c0cda30 0x7f986c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.038+0000 7f986b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f986c15bf30 con 0x7f986c071530 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.038+0000 7f986b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f986c15c860 con 0x7f986c0cda30 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.038+0000 7f986b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c071970 con 0x7f986c144440 2024-04-25T10:03:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.039+0000 7f986a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f986c0cda30 0x7f986c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.039+0000 7f986a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f986c0cda30 0x7f986c15ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:47516/0 (socket says 172.21.15.165:47516) 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.039+0000 7f986a800640 1 -- 172.21.15.165:0/268475649 learned_addr learned my addr 172.21.15.165:0/268475649 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.039+0000 7f9869e00640 1 --2- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c144440 0x7f986c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.039+0000 7f9863e00640 1 --1- 172.21.15.165:0/268475649 >> v1:172.21.15.172:6789/0 conn(0x7f986c071530 0x7f986c0cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:60222/0 (socket says 172.21.15.165:60222) 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.040+0000 7f986a800640 1 -- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c144440 msgr2=0x7f986c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.040+0000 7f986a800640 1 --2- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c144440 0x7f986c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.040+0000 7f986a800640 1 -- 172.21.15.165:0/268475649 >> v1:172.21.15.172:6789/0 conn(0x7f986c071530 legacy=0x7f986c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.040+0000 7f986a800640 1 -- 172.21.15.165:0/268475649 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c15c210 con 0x7f986c0cda30 2024-04-25T10:03:20.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.041+0000 7f986a800640 1 --2- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c0cda30 0x7f986c15ccf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9864009e50 tx=0x7f986402a7a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7aa7337d03af9f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.041+0000 7f9869400640 1 -- 172.21.15.165:0/268475649 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864036070 con 0x7f986c0cda30 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.041+0000 7f9869400640 1 -- 172.21.15.165:0/268475649 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864031600 con 0x7f986c0cda30 2024-04-25T10:03:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.041+0000 7f9869400640 1 -- 172.21.15.165:0/268475649 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98640319d0 con 0x7f986c0cda30 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 -- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c0cda30 msgr2=0x7f986c15ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 --2- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c0cda30 0x7f986c15ccf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f9864009e50 tx=0x7f986402a7a0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 -- 172.21.15.165:0/268475649 shutdown_connections 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 --2- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c0cda30 0x7f986c15ccf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 --2- 172.21.15.165:0/268475649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c144440 0x7f986c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 -- 172.21.15.165:0/268475649 >> 172.21.15.165:0/268475649 conn(0x7f986c06abe0 msgr2=0x7f986c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 -- 172.21.15.165:0/268475649 shutdown_connections 2024-04-25T10:03:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.042+0000 7f986b200640 1 -- 172.21.15.165:0/268475649 wait complete. 2024-04-25T10:03:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.044+0000 7f986b200640 1 Processor -- start 2024-04-25T10:03:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.044+0000 7f986b200640 1 -- start start 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.045+0000 7f986b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c071530 0x7f986c1e34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.045+0000 7f986b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c0cda30 0x7f986c1e7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.045+0000 7f986b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 0x7f986c13dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.045+0000 7f986b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c15bc30 con 0x7f986c144440 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.045+0000 7f986b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986c0cdea0 con 0x7f986c071530 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.045+0000 7f986b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986c15d230 con 0x7f986c0cda30 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 0x7f986c13dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 0x7f986c13dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43668/0 (socket says 172.21.15.165:43668) 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 -- 172.21.15.165:0/1092125002 learned_addr learned my addr 172.21.15.165:0/1092125002 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f9869e00640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c071530 0x7f986c1e34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f9863e00640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c0cda30 0x7f986c1e7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 -- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c0cda30 msgr2=0x7f986c1e7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c0cda30 0x7f986c1e7700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 -- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c071530 msgr2=0x7f986c1e34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c071530 0x7f986c1e34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f986a800640 1 -- 172.21.15.165:0/1092125002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f986c0cc7f0 con 0x7f986c144440 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.046+0000 7f9869e00640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c071530 0x7f986c1e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.047+0000 7f986a800640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 0x7f986c13dc20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9864009f70 tx=0x7f9864004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.047+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864036050 con 0x7f986c144440 2024-04-25T10:03:20.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.047+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9864042070 con 0x7f986c144440 2024-04-25T10:03:20.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.047+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f986c0ccad0 con 0x7f986c144440 2024-04-25T10:03:20.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.047+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9864009510 con 0x7f986c144440 2024-04-25T10:03:20.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.047+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986c0c92a0 con 0x7f986c144440 2024-04-25T10:03:20.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.048+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f986c073990 con 0x7f986c144440 2024-04-25T10:03:20.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.049+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9864007bf0 con 0x7f986c144440 2024-04-25T10:03:20.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.050+0000 7f9862000640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f984806b4d0 0x7f984806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.050+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7f98640ab3f0 con 0x7f986c144440 2024-04-25T10:03:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.050+0000 7f9869e00640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f984806b4d0 0x7f984806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.051+0000 7f9869e00640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f984806b4d0 0x7f984806d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f985c009a20 tx=0x7f985c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.056+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9864079e60 con 0x7f986c144440 2024-04-25T10:03:20.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.302+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f986c0ccca0 con 0x7f986c144440 2024-04-25T10:03:20.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.303+0000 7f9862000640 1 -- 172.21.15.165:0/1092125002 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4946 (secure 0 0 0) 0x7f9864079760 con 0x7f986c144440 2024-04-25T10:03:20.308 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:20.308 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4229,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3258870932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3258870932},{"type":"v1","addr":"172.21.15.172:6834","nonce":3258870932}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4251,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3526824266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3526824266},{"type":"v1","addr":"172.21.15.165:6835","nonce":3526824266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4257,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3765936966","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3765936966},{"type":"v1","addr":"172.21.15.165:6837","nonce":3765936966}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:02:03.570541+0000","modified":"2024-04-25T10:02:24.722564+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4288},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4288":{"gid":4288,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2597521026","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":2597521026},{"type":"v1","addr":"172.21.15.172:6835","nonce":2597521026}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2024-04-25T10:03:20.309 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 9 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f984806b4d0 msgr2=0x7f984806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f984806b4d0 0x7f984806d990 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f985c009a20 tx=0x7f985c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 msgr2=0x7f986c13dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 0x7f986c13dc20 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9864009f70 tx=0x7f9864004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 shutdown_connections 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f984806b4d0 0x7f984806d990 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f986c144440 0x7f986c13dc20 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f986c0cda30 0x7f986c1e7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 --2- 172.21.15.165:0/1092125002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f986c071530 0x7f986c1e34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.306+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 >> 172.21.15.165:0/1092125002 conn(0x7f986c06abe0 msgr2=0x7f986c0b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.307+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 shutdown_connections 2024-04-25T10:03:20.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.307+0000 7f986b200640 1 -- 172.21.15.165:0/1092125002 wait complete. 2024-04-25T10:03:20.329 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:20.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.531+0000 7efbfea00640 1 Processor -- start 2024-04-25T10:03:20.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.531+0000 7efbfea00640 1 -- start start 2024-04-25T10:03:20.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.533+0000 7efbfea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815db70 0x7efbf80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.533+0000 7efbfea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efbf815df50 0x7efbf80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.533+0000 7efbfea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbf8071650 con 0x7efbf80cec70 2024-04-25T10:03:20.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.533+0000 7efbfea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbf80cbd80 con 0x7efbf815df50 2024-04-25T10:03:20.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.533+0000 7efbfea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8161910 con 0x7efbf815db70 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbefe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efbf815df50 0x7efbf80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbf7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815db70 0x7efbf80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbfca00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efbf80cec70 0x7efbf80cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40360/0 (socket says 172.21.15.165:40360) 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbfca00640 1 -- 172.21.15.165:0/4011699949 learned_addr learned my addr 172.21.15.165:0/4011699949 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbf7400640 1 -- 172.21.15.165:0/4011699949 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3265199644 0 0) 0x7efbf0001490 con 0x7efbf80cec70 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbf7400640 1 -- 172.21.15.165:0/4011699949 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efbdc003520 con 0x7efbf80cec70 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbefe00640 1 -- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815db70 msgr2=0x7efbf80cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbefe00640 1 --2- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815db70 0x7efbf80cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbefe00640 1 -- 172.21.15.165:0/4011699949 >> v1:172.21.15.165:6789/0 conn(0x7efbf80cec70 legacy=0x7efbf80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.534+0000 7efbefe00640 1 -- 172.21.15.165:0/4011699949 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf80cf230 con 0x7efbf815df50 2024-04-25T10:03:20.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.536+0000 7efbefe00640 1 --2- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf815df50 0x7efbf80cf510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efbe00097a0 tx=0x7efbe002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aa25cf2c8809acf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.536+0000 7efbf7400640 1 -- 172.21.15.165:0/4011699949 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0037070 con 0x7efbf815df50 2024-04-25T10:03:20.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.536+0000 7efbf7400640 1 -- 172.21.15.165:0/4011699949 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe0032440 con 0x7efbf815df50 2024-04-25T10:03:20.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.536+0000 7efbf7400640 1 -- 172.21.15.165:0/4011699949 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0032810 con 0x7efbf815df50 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 -- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf815df50 msgr2=0x7efbf80cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 --2- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf815df50 0x7efbf80cf510 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7efbe00097a0 tx=0x7efbe002a180 comp rx=0 tx=0).stop 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 -- 172.21.15.165:0/4011699949 shutdown_connections 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 --2- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf815df50 0x7efbf80cf510 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 --2- 172.21.15.165:0/4011699949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815db70 0x7efbf80cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 -- 172.21.15.165:0/4011699949 >> 172.21.15.165:0/4011699949 conn(0x7efbf806abe0 msgr2=0x7efbf806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 -- 172.21.15.165:0/4011699949 shutdown_connections 2024-04-25T10:03:20.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.537+0000 7efbfea00640 1 -- 172.21.15.165:0/4011699949 wait complete. 2024-04-25T10:03:20.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.539+0000 7efbfea00640 1 Processor -- start 2024-04-25T10:03:20.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.539+0000 7efbfea00640 1 -- start start 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf80cec70 0x7efbf814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 0x7efbf8158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815df50 0x7efbf81f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8162120 con 0x7efbf815db70 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8161e40 con 0x7efbf80cec70 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbf8162640 con 0x7efbf815df50 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 0x7efbf8158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 0x7efbf8158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43680/0 (socket says 172.21.15.165:43680) 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 -- 172.21.15.165:0/2581887007 learned_addr learned my addr 172.21.15.165:0/2581887007 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbf7e00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf80cec70 0x7efbf814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbfca00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815df50 0x7efbf81f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 -- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815df50 msgr2=0x7efbf81f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815df50 0x7efbf81f39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 -- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf80cec70 msgr2=0x7efbf814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf80cec70 0x7efbf814fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbefe00640 1 -- 172.21.15.165:0/2581887007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf8157c80 con 0x7efbf815db70 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.540+0000 7efbf7e00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf80cec70 0x7efbf814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.541+0000 7efbefe00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 0x7efbf8158d20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000c00 tx=0x7efbe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.541+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0041020 con 0x7efbf815db70 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.541+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbe003d640 con 0x7efbf815db70 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.541+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe0037040 con 0x7efbf815db70 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.541+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbf8157f60 con 0x7efbf815db70 2024-04-25T10:03:20.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.541+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbf8158520 con 0x7efbf815db70 2024-04-25T10:03:20.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.542+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbf8073990 con 0x7efbf815db70 2024-04-25T10:03:20.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.543+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efbe0044020 con 0x7efbf815db70 2024-04-25T10:03:20.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.544+0000 7efbf4c00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efbd406b4d0 0x7efbd406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.544+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7efbe00aadd0 con 0x7efbf815db70 2024-04-25T10:03:20.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.545+0000 7efbf7e00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efbd406b4d0 0x7efbd406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.545+0000 7efbf7e00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efbd406b4d0 0x7efbd406d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efbf815ad00 tx=0x7efbe8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.549+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efbe004b030 con 0x7efbf815db70 2024-04-25T10:03:20.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.743+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efbf8156fb0 con 0x7efbf815db70 2024-04-25T10:03:20.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.744+0000 7efbf4c00640 1 -- 172.21.15.165:0/2581887007 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v26) v1 ==== 74+0+18939 (secure 0 0 0) 0x7efbe00797c0 con 0x7efbf815db70 2024-04-25T10:03:20.750 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:20.750 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":26,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:02:45.824777+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:02:01.402047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-25T10:02:02.584890+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-25T10:02:06.186614+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.7","mappings":[{"from":3,"to":1}]},{"pgid":"3.25","mappings":[{"from":3,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.36","mappings":[{"from":5,"to":2}]},{"pgid":"4.e","mappings":[{"from":1,"to":2}]},{"pgid":"4.14","mappings":[{"from":1,"to":5}]},{"pgid":"4.18","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efbd406b4d0 msgr2=0x7efbd406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efbd406b4d0 0x7efbd406d990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7efbf815ad00 tx=0x7efbe8008040 comp rx=0 tx=0).stop 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 msgr2=0x7efbf8158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 0x7efbf8158d20 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7efbe0000c00 tx=0x7efbe00046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 shutdown_connections 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efbd406b4d0 0x7efbd406d990 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efbf815df50 0x7efbf81f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efbf815db70 0x7efbf8158d20 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 --2- 172.21.15.165:0/2581887007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efbf80cec70 0x7efbf814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.747+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 >> 172.21.15.165:0/2581887007 conn(0x7efbf806abe0 msgr2=0x7efbf80adfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.748+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 shutdown_connections 2024-04-25T10:03:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.748+0000 7efbfea00640 1 -- 172.21.15.165:0/2581887007 wait complete. 2024-04-25T10:03:20.772 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:03:20.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.960+0000 7fc4f4a00640 1 Processor -- start 2024-04-25T10:03:20.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.961+0000 7fc4f4a00640 1 -- start start 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4f4a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc4f015b5a0 0x7fc4f00b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4f4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f00b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4f4a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4f00cf320 con 0x7fc4f0162450 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4f4a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f00cf040 con 0x7fc4f015b5a0 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4f4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f0161970 con 0x7fc4f015b980 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4ee800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc4f0162450 0x7fc4f00ce310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40364/0 (socket says 172.21.15.165:40364) 2024-04-25T10:03:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.962+0000 7fc4ee800640 1 -- 172.21.15.165:0/3018567604 learned_addr learned my addr 172.21.15.165:0/3018567604 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:20.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ede00640 1 --2- 172.21.15.165:0/3018567604 >> v2:172.21.15.172:3300/0 conn(0x7fc4f015b5a0 0x7fc4f00b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4e5400640 1 --2- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f00b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ed400640 1 -- 172.21.15.165:0/3018567604 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2534868090 0 0) 0x7fc4e8001110 con 0x7fc4f0162450 2024-04-25T10:03:20.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ed400640 1 -- 172.21.15.165:0/3018567604 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4d4003520 con 0x7fc4f0162450 2024-04-25T10:03:20.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ede00640 1 -- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 msgr2=0x7fc4f00b5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:20.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ed400640 1 -- 172.21.15.165:0/3018567604 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554222168 0 0) 0x7fc4d4003520 con 0x7fc4f0162450 2024-04-25T10:03:20.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ede00640 1 --2- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f00b5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ede00640 1 -- 172.21.15.165:0/3018567604 >> v1:172.21.15.165:6789/0 conn(0x7fc4f0162450 legacy=0x7fc4f00ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.963+0000 7fc4ede00640 1 -- 172.21.15.165:0/3018567604 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0161c50 con 0x7fc4f015b5a0 2024-04-25T10:03:20.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.965+0000 7fc4ede00640 1 --2- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f00b5530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009d90 tx=0x7fc4e002a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6601006fffb92ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.965+0000 7fc4ed400640 1 -- 172.21.15.165:0/3018567604 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8001840 con 0x7fc4f015b5a0 2024-04-25T10:03:20.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.965+0000 7fc4ed400640 1 -- 172.21.15.165:0/3018567604 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e0007540 con 0x7fc4f015b5a0 2024-04-25T10:03:20.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.965+0000 7fc4ed400640 1 -- 172.21.15.165:0/3018567604 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e00078f0 con 0x7fc4f015b5a0 2024-04-25T10:03:20.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 -- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 msgr2=0x7fc4f00b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f00b5530 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009d90 tx=0x7fc4e002a5f0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 -- 172.21.15.165:0/3018567604 shutdown_connections 2024-04-25T10:03:20.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f00b5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/3018567604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f00b5530 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 -- 172.21.15.165:0/3018567604 >> 172.21.15.165:0/3018567604 conn(0x7fc4f00d0530 msgr2=0x7fc4f00d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:20.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 -- 172.21.15.165:0/3018567604 shutdown_connections 2024-04-25T10:03:20.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.966+0000 7fc4f4a00640 1 -- 172.21.15.165:0/3018567604 wait complete. 2024-04-25T10:03:20.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.968+0000 7fc4f4a00640 1 Processor -- start 2024-04-25T10:03:20.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.968+0000 7fc4f4a00640 1 -- start start 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4f4a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f01f7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4f4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f014ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4f4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 0x7fc4f01f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4f4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00ceba0 con 0x7fc4f0162450 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4f4a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00cea20 con 0x7fc4f015b5a0 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4f4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f00ced20 con 0x7fc4f015b980 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4ee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 0x7fc4f01f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4ee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 0x7fc4f01f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43682/0 (socket says 172.21.15.165:43682) 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4ee800640 1 -- 172.21.15.165:0/716061766 learned_addr learned my addr 172.21.15.165:0/716061766 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4ede00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f01f7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.970+0000 7fc4e5400640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f014ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ee800640 1 -- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 msgr2=0x7fc4f014ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ee800640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f014ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ee800640 1 -- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 msgr2=0x7fc4f01f7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ee800640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f01f7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ee800640 1 -- 172.21.15.165:0/716061766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f01f4390 con 0x7fc4f0162450 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ede00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f01f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4ee800640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 0x7fc4f01f38d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800c750 tx=0x7fc4e800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8017070 con 0x7fc4f0162450 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4e8005ad0 con 0x7fc4f0162450 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4e8007070 con 0x7fc4f0162450 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f01f4670 con 0x7fc4f0162450 2024-04-25T10:03:20.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.971+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f014b370 con 0x7fc4f0162450 2024-04-25T10:03:20.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.972+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f0073a00 con 0x7fc4f0162450 2024-04-25T10:03:20.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.973+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4e801a070 con 0x7fc4f0162450 2024-04-25T10:03:20.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.975+0000 7fc4e6a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:20.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.975+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6735+0+0 (secure 0 0 0) 0x7fc4e8089b30 con 0x7fc4f0162450 2024-04-25T10:03:20.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.975+0000 7fc4ede00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:20.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.976+0000 7fc4ede00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0034000 tx=0x7fc4e002f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:20.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:20.978+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4f00cf320 con 0x7fc4f0162450 2024-04-25T10:03:21.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.211+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc4f00d17c0 con 0x7fc4f0162450 2024-04-25T10:03:21.217 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:03:21.212+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:03:21.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.216+0000 7fc4e6a00640 1 -- 172.21.15.165:0/716061766 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc4e80585a0 con 0x7fc4f0162450 2024-04-25T10:03:21.221 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:03:21.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4cc06b4d0 msgr2=0x7fc4cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0034000 tx=0x7fc4e002f000 comp rx=0 tx=0).stop 2024-04-25T10:03:21.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 msgr2=0x7fc4f01f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 0x7fc4f01f38d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc4e800c750 tx=0x7fc4e800cc10 comp rx=0 tx=0).stop 2024-04-25T10:03:21.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 shutdown_connections 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4cc06b4d0 0x7fc4cc06d990 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4f0162450 0x7fc4f01f38d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4f015b980 0x7fc4f014ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 --2- 172.21.15.165:0/716061766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4f015b5a0 0x7fc4f01f7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 >> 172.21.15.165:0/716061766 conn(0x7fc4f00d0530 msgr2=0x7fc4f015d4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.219+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 shutdown_connections 2024-04-25T10:03:21.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.220+0000 7fc4f4a00640 1 -- 172.21.15.165:0/716061766 wait complete. 2024-04-25T10:03:21.243 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:03:21.260 INFO:tasks.ceph.mds.b.smithi172.stderr: -5849> 2024-04-25T10:02:09.484+0000 7f8927c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:21.338 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:03:21.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.443+0000 7f8a60a00640 1 Processor -- start 2024-04-25T10:03:21.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.443+0000 7f8a60a00640 1 -- start start 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a60a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8a5c144d50 0x7f8a5c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a60a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8a5c1452a0 0x7f8a5c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a60a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a5c0cccc0 con 0x7f8a5c1446d0 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a60a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c0b2b20 con 0x7f8a5c144d50 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a60a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a5c06d400 con 0x7f8a5c1452a0 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a59e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8a5c1446d0 0x7f8a5c0b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40378/0 (socket says 172.21.15.165:40378) 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a59e00640 1 -- 172.21.15.165:0/2824007706 learned_addr learned my addr 172.21.15.165:0/2824007706 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:21.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.445+0000 7f8a51400640 1 --2- 172.21.15.165:0/2824007706 >> v2:172.21.15.165:3300/0 conn(0x7f8a5c144d50 0x7f8a5c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a5a800640 1 --2- 172.21.15.165:0/2824007706 >> v2:172.21.15.172:3300/0 conn(0x7f8a5c1452a0 0x7f8a5c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a59400640 1 -- 172.21.15.165:0/2824007706 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 995011774 0 0) 0x7f8a5c0cccc0 con 0x7f8a5c1446d0 2024-04-25T10:03:21.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a59400640 1 -- 172.21.15.165:0/2824007706 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a40003520 con 0x7f8a5c1446d0 2024-04-25T10:03:21.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a51400640 1 -- 172.21.15.165:0/2824007706 >> v2:172.21.15.172:3300/0 conn(0x7f8a5c1452a0 msgr2=0x7f8a5c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:21.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a51400640 1 --2- 172.21.15.165:0/2824007706 >> v2:172.21.15.172:3300/0 conn(0x7f8a5c1452a0 0x7f8a5c06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a51400640 1 -- 172.21.15.165:0/2824007706 >> v1:172.21.15.165:6789/0 conn(0x7f8a5c1446d0 legacy=0x7f8a5c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.446+0000 7f8a51400640 1 -- 172.21.15.165:0/2824007706 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c0cc140 con 0x7f8a5c144d50 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.447+0000 7f8a51400640 1 --2- 172.21.15.165:0/2824007706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c144d50 0x7f8a5c06d890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c15bfe0 tx=0x7f8a4402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a72fb317bc95ac7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.447+0000 7f8a59400640 1 -- 172.21.15.165:0/2824007706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44037070 con 0x7f8a5c144d50 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.447+0000 7f8a59400640 1 -- 172.21.15.165:0/2824007706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a44032440 con 0x7f8a5c144d50 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.447+0000 7f8a59400640 1 -- 172.21.15.165:0/2824007706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a44032810 con 0x7f8a5c144d50 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 -- 172.21.15.165:0/2824007706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c144d50 msgr2=0x7f8a5c06d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2824007706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c144d50 0x7f8a5c06d890 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f8a5c15bfe0 tx=0x7f8a4402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 -- 172.21.15.165:0/2824007706 shutdown_connections 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2824007706 >> v2:172.21.15.172:3300/0 conn(0x7f8a5c1452a0 0x7f8a5c06ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2824007706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c144d50 0x7f8a5c06d890 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 -- 172.21.15.165:0/2824007706 >> 172.21.15.165:0/2824007706 conn(0x7f8a5c06abe0 msgr2=0x7f8a5c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:21.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 -- 172.21.15.165:0/2824007706 shutdown_connections 2024-04-25T10:03:21.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.448+0000 7f8a60a00640 1 -- 172.21.15.165:0/2824007706 wait complete. 2024-04-25T10:03:21.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.450+0000 7f8a60a00640 1 Processor -- start 2024-04-25T10:03:21.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.450+0000 7f8a60a00640 1 -- start start 2024-04-25T10:03:21.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a60a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 0x7f8a5c15b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a60a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a5c144d50 0x7f8a5c139280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a60a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a5c1452a0 0x7f8a5c0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a60a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0cc9e0 con 0x7f8a5c1446d0 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a60a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c0cc700 con 0x7f8a5c1452a0 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a60a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5c144ab0 con 0x7f8a5c144d50 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a59e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 0x7f8a5c15b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a59e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 0x7f8a5c15b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43702/0 (socket says 172.21.15.165:43702) 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.451+0000 7f8a59e00640 1 -- 172.21.15.165:0/2281005652 learned_addr learned my addr 172.21.15.165:0/2281005652 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a51400640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a5c144d50 0x7f8a5c139280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a5a800640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a5c1452a0 0x7f8a5c0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a59e00640 1 -- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a5c144d50 msgr2=0x7f8a5c139280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a59e00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a5c144d50 0x7f8a5c139280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a59e00640 1 -- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a5c1452a0 msgr2=0x7f8a5c0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a59e00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a5c1452a0 0x7f8a5c0b1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a59e00640 1 -- 172.21.15.165:0/2281005652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c13d200 con 0x7f8a5c1446d0 2024-04-25T10:03:21.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a5a800640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a5c1452a0 0x7f8a5c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:21.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a59e00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 0x7f8a5c15b1d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00caa0 tx=0x7f8a4c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c017070 con 0x7f8a5c1446d0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4c00ac00 con 0x7f8a5c1446d0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4c012400 con 0x7f8a5c1446d0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5c13d4e0 con 0x7f8a5c1446d0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.452+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5c13daa0 con 0x7f8a5c1446d0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.454+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a5c073a00 con 0x7f8a5c1446d0 2024-04-25T10:03:21.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.454+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a4c01a070 con 0x7f8a5c1446d0 2024-04-25T10:03:21.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.456+0000 7f8a52a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:21.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.456+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7f8a4c08a440 con 0x7f8a5c1446d0 2024-04-25T10:03:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.456+0000 7f8a51400640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.457+0000 7f8a51400640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3806b4d0 0x7f8a3806d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a44034000 tx=0x7f8a4402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:21.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.460+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a4c058e50 con 0x7f8a5c1446d0 2024-04-25T10:03:21.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:21.692+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8a5c06ed90 con 0x7f8a5c1446d0 2024-04-25T10:03:22.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.220+0000 7f8a52a00640 1 -- 172.21.15.165:0/2281005652 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8a4c058750 con 0x7f8a5c1446d0 2024-04-25T10:03:22.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.224+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3806b4d0 msgr2=0x7f8a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3806b4d0 0x7f8a3806d990 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a44034000 tx=0x7f8a4402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 msgr2=0x7f8a5c15b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 0x7f8a5c15b1d0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f8a4c00caa0 tx=0x7f8a4c00cf60 comp rx=0 tx=0).stop 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 shutdown_connections 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3806b4d0 0x7f8a3806d990 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a5c1452a0 0x7f8a5c0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a5c144d50 0x7f8a5c139280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 --2- 172.21.15.165:0/2281005652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a5c1446d0 0x7f8a5c15b1d0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 >> 172.21.15.165:0/2281005652 conn(0x7f8a5c06abe0 msgr2=0x7f8a5c143690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 shutdown_connections 2024-04-25T10:03:22.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.225+0000 7f8a60a00640 1 -- 172.21.15.165:0/2281005652 wait complete. 2024-04-25T10:03:22.251 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:03:22.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.457+0000 7fc3f3400640 1 Processor -- start 2024-04-25T10:03:22.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.457+0000 7fc3f3400640 1 -- start start 2024-04-25T10:03:22.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.458+0000 7fc3f3400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3ec15b980 0x7fc3ec0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.458+0000 7fc3f3400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec072280 con 0x7fc3ec0cc4f0 2024-04-25T10:03:22.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.458+0000 7fc3f3400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3ec0714c0 con 0x7fc3ec0ce230 2024-04-25T10:03:22.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.458+0000 7fc3f3400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3ec071270 con 0x7fc3ec15b980 2024-04-25T10:03:22.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3f0a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc3ec0cc4f0 0x7fc3ec0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40394/0 (socket says 172.21.15.165:40394) 2024-04-25T10:03:22.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3f0a00640 1 -- 172.21.15.165:0/3710144019 learned_addr learned my addr 172.21.15.165:0/3710144019 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:22.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2247236522 0 0) 0x7fc3ec072280 con 0x7fc3ec0cc4f0 2024-04-25T10:03:22.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3d0003520 con 0x7fc3ec0cc4f0 2024-04-25T10:03:22.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3f1400640 1 --2- 172.21.15.165:0/3710144019 >> v2:172.21.15.172:3300/0 conn(0x7fc3ec15b980 0x7fc3ec0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1901376469 0 0) 0x7fc3d0003520 con 0x7fc3ec0cc4f0 2024-04-25T10:03:22.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 >> v2:172.21.15.172:3300/0 conn(0x7fc3ec15b980 msgr2=0x7fc3ec0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 --2- 172.21.15.165:0/3710144019 >> v2:172.21.15.172:3300/0 conn(0x7fc3ec15b980 0x7fc3ec0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 >> v1:172.21.15.172:6789/0 conn(0x7fc3ec0ce230 legacy=0x7fc3ec15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec0717a0 con 0x7fc3ec0cc4f0 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc3dc001670 con 0x7fc3ec0cc4f0 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 >> v1:172.21.15.165:6789/0 conn(0x7fc3ec0cc4f0 legacy=0x7fc3ec0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.459+0000 7fc3ebe00640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d0003a00 0x7fc3d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3ebe00640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0004800 0x7fc3d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3ebe00640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d0005210 0x7fc3d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec071270 con 0x7fc3d0005210 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000c140 con 0x7fc3d0003a00 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3d000e550 con 0x7fc3d0004800 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f1400640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d0005210 0x7fc3d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f0a00640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d0003a00 0x7fc3d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f1400640 1 -- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0004800 msgr2=0x7fc3d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f1400640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0004800 0x7fc3d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f1400640 1 -- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d0003a00 msgr2=0x7fc3d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f1400640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3d0003a00 0x7fc3d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.460+0000 7fc3f1400640 1 -- 172.21.15.165:0/3710144019 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3d0003520 con 0x7fc3d0005210 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.461+0000 7fc3f1400640 1 -- 172.21.15.165:0/3710144019 reap_dead start 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.461+0000 7fc3f1400640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d0005210 0x7fc3d000b9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007330 tx=0x7fc3d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cbc419323fad4dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4009be0 con 0x7fc3d0005210 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3e4003b90 con 0x7fc3d0005210 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3ebe00640 1 -- 172.21.15.165:0/3710144019 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3e4004020 con 0x7fc3d0005210 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 -- 172.21.15.165:0/3710144019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d0005210 msgr2=0x7fc3d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d0005210 0x7fc3d000b9c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4007330 tx=0x7fc3d0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 -- 172.21.15.165:0/3710144019 shutdown_connections 2024-04-25T10:03:22.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3710144019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3d0005210 0x7fc3d000b9c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 -- 172.21.15.165:0/3710144019 >> 172.21.15.165:0/3710144019 conn(0x7fc3ec06abe0 msgr2=0x7fc3ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:22.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 -- 172.21.15.165:0/3710144019 shutdown_connections 2024-04-25T10:03:22.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.462+0000 7fc3f3400640 1 -- 172.21.15.165:0/3710144019 wait complete. 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.464+0000 7fc3f3400640 1 Processor -- start 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.464+0000 7fc3f3400640 1 -- start start 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.465+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0005210 0x7fc3ec156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.465+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 0x7fc3ec158980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.465+0000 7fc3f3400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3ec158ec0 0x7fc3ec153fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.465+0000 7fc3f3400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0cc8d0 con 0x7fc3ec13daa0 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.465+0000 7fc3f3400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec071d60 con 0x7fc3ec158ec0 2024-04-25T10:03:22.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.465+0000 7fc3f3400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3ec0cca50 con 0x7fc3d0005210 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 0x7fc3ec158980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 0x7fc3ec158980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43716/0 (socket says 172.21.15.165:43716) 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 -- 172.21.15.165:0/3877520015 learned_addr learned my addr 172.21.15.165:0/3877520015 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3f1400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3ec158ec0 0x7fc3ec153fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3f0a00640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0005210 0x7fc3ec156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 -- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0005210 msgr2=0x7fc3ec156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0005210 0x7fc3ec156040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 -- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3ec158ec0 msgr2=0x7fc3ec153fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3ec158ec0 0x7fc3ec153fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 -- 172.21.15.165:0/3877520015 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec13e490 con 0x7fc3ec13daa0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.466+0000 7fc3e3e00640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 0x7fc3ec158980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c3c0 tx=0x7fc3d400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.467+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4011790 con 0x7fc3ec13daa0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.467+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3d4017070 con 0x7fc3ec13daa0 2024-04-25T10:03:22.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.467+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3d4011b70 con 0x7fc3ec13daa0 2024-04-25T10:03:22.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.467+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3ec13e770 con 0x7fc3ec13daa0 2024-04-25T10:03:22.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.467+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3ec13ed30 con 0x7fc3ec13daa0 2024-04-25T10:03:22.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.468+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3ec073a00 con 0x7fc3ec13daa0 2024-04-25T10:03:22.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.469+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3d4011d10 con 0x7fc3ec13daa0 2024-04-25T10:03:22.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.470+0000 7fc3e9600640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3c806b500 0x7fc3c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:22.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.470+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6821+0+0 (secure 0 0 0) 0x7fc3d408a3f0 con 0x7fc3ec13daa0 2024-04-25T10:03:22.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.470+0000 7fc3f0a00640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3c806b500 0x7fc3c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:22.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.471+0000 7fc3f0a00640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3c806b500 0x7fc3c806d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc001050 tx=0x7fc3dc001170 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:22.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.474+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3d4058ec0 con 0x7fc3ec13daa0 2024-04-25T10:03:22.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:22.672+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc3ec1592a0 con 0x7fc3ec13daa0 2024-04-25T10:03:23.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.456+0000 7fc3e9600640 1 -- 172.21.15.165:0/3877520015 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v28) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc3d40587c0 con 0x7fc3ec13daa0 2024-04-25T10:03:23.462 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3c806b500 msgr2=0x7fc3c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3c806b500 0x7fc3c806d9c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc001050 tx=0x7fc3dc001170 comp rx=0 tx=0).stop 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 msgr2=0x7fc3ec158980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 0x7fc3ec158980 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc3d400c3c0 tx=0x7fc3d400c880 comp rx=0 tx=0).stop 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 shutdown_connections 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3c806b500 0x7fc3c806d9c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3ec158ec0 0x7fc3ec153fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3ec13daa0 0x7fc3ec158980 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 --2- 172.21.15.165:0/3877520015 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3d0005210 0x7fc3ec156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 >> 172.21.15.165:0/3877520015 conn(0x7fc3ec06abe0 msgr2=0x7fc3ec15c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.461+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 shutdown_connections 2024-04-25T10:03:23.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.462+0000 7fc3f3400640 1 -- 172.21.15.165:0/3877520015 wait complete. 2024-04-25T10:03:23.490 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:03:23.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.665+0000 7ffa05e00640 1 Processor -- start 2024-04-25T10:03:23.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.666+0000 7ffa05e00640 1 -- start start 2024-04-25T10:03:23.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.667+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 0x7ffa000caa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ffa05e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffa0006bbf0 0x7ffa0006c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ffa05e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa000cb0e0 con 0x7ffa0006ff30 2024-04-25T10:03:23.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ffa05e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000cb3c0 con 0x7ffa0006bbf0 2024-04-25T10:03:23.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ffa05e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cb6a0 con 0x7ffa0015dbd0 2024-04-25T10:03:23.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ff9f7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ffa0006ff30 0x7ffa000cc180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40402/0 (socket says 172.21.15.165:40402) 2024-04-25T10:03:23.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ff9f7e00640 1 -- 172.21.15.165:0/816119410 learned_addr learned my addr 172.21.15.165:0/816119410 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ff9fea00640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 0x7ffa000caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3074500150 0 0) 0x7ffa000cb0e0 con 0x7ffa0006ff30 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.668+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9e4003520 con 0x7ffa0006ff30 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9ff400640 1 --2- 172.21.15.165:0/816119410 >> v2:172.21.15.172:3300/0 conn(0x7ffa0006bbf0 0x7ffa0006c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2080928382 0 0) 0x7ff9e4003520 con 0x7ffa0006ff30 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 msgr2=0x7ffa000caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 0x7ffa000caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 >> v2:172.21.15.172:3300/0 conn(0x7ffa0006bbf0 msgr2=0x7ffa0006c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 --2- 172.21.15.165:0/816119410 >> v2:172.21.15.172:3300/0 conn(0x7ffa0006bbf0 0x7ffa0006c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000cb980 con 0x7ffa0006ff30 2024-04-25T10:03:23.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff9e8001670 con 0x7ffa0006ff30 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 >> v1:172.21.15.165:6789/0 conn(0x7ffa0006ff30 legacy=0x7ffa000cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e40039b0 0x7ff9e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 0x7ff9e4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cb3c0 con 0x7ff9e4004d40 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa000cb6a0 con 0x7ff9e40039b0 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e400ba40 con 0x7ffa0015dbd0 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.669+0000 7ff9ff400640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.670+0000 7ff9fea00640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e40039b0 0x7ff9e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.670+0000 7ff9ff400640 1 -- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 msgr2=0x7ff9e4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.670+0000 7ff9ff400640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa0015dbd0 0x7ff9e4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.670+0000 7ff9ff400640 1 -- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e40039b0 msgr2=0x7ff9e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.670+0000 7ff9ff400640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff9e40039b0 0x7ff9e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.670+0000 7ff9ff400640 1 -- 172.21.15.165:0/816119410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e4003520 con 0x7ff9e4004d40 2024-04-25T10:03:23.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.671+0000 7ff9ff400640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8007360 tx=0x7ff9e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219667a2ac82c239 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.671+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8009be0 con 0x7ff9e4004d40 2024-04-25T10:03:23.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.671+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f802e620 con 0x7ff9e4004d40 2024-04-25T10:03:23.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.671+0000 7ff9fe000640 1 -- 172.21.15.165:0/816119410 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f802e9f0 con 0x7ff9e4004d40 2024-04-25T10:03:23.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ffa05e00640 1 -- 172.21.15.165:0/816119410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 msgr2=0x7ff9e40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ffa05e00640 1 --2- 172.21.15.165:0/816119410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ff9e40051c0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ff9f8007360 tx=0x7ff9e4005830 comp rx=0 tx=0).stop 2024-04-25T10:03:23.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ff9ff400640 1 -- 172.21.15.165:0/816119410 reap_dead start 2024-04-25T10:03:23.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ffa05e00640 1 -- 172.21.15.165:0/816119410 shutdown_connections 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ffa05e00640 1 -- 172.21.15.165:0/816119410 >> 172.21.15.165:0/816119410 conn(0x7ffa00071af0 msgr2=0x7ffa00071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ffa05e00640 1 -- 172.21.15.165:0/816119410 shutdown_connections 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.672+0000 7ffa05e00640 1 -- 172.21.15.165:0/816119410 wait complete. 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.674+0000 7ffa05e00640 1 Processor -- start 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.674+0000 7ffa05e00640 1 -- start start 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ffa00153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa000b3dd0 0x7ffa000c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ffa05e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa000c7460 0x7ffa000addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ffa05e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015e130 con 0x7ff9e4004d40 2024-04-25T10:03:23.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ffa05e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0015dfb0 con 0x7ffa000b3dd0 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ffa05e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00070330 con 0x7ffa000c7460 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ffa00153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ffa00153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43742/0 (socket says 172.21.15.165:43742) 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 -- 172.21.15.165:0/973740317 learned_addr learned my addr 172.21.15.165:0/973740317 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9f7e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa000b3dd0 0x7ffa000c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9ff400640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa000c7460 0x7ffa000addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 -- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa000c7460 msgr2=0x7ffa000addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa000c7460 0x7ffa000addd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 -- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa000b3dd0 msgr2=0x7ffa000c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa000b3dd0 0x7ffa000c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.675+0000 7ff9fea00640 1 -- 172.21.15.165:0/973740317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa001f72d0 con 0x7ff9e4004d40 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ff9f7e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa000b3dd0 0x7ffa000c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:23.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ff9fea00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ffa00153010 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00116d0 tx=0x7ff9f0011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000f9c0 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000fb60 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f0015460 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa001f75b0 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.676+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa001f7b70 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.678+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa000cf9d0 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.679+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff9f0015600 con 0x7ff9e4004d40 2024-04-25T10:03:23.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.680+0000 7ff9f7400640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff9dc06b4d0 0x7ff9dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:23.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.680+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6383+0+0 (secure 0 0 0) 0x7ff9f008af00 con 0x7ff9e4004d40 2024-04-25T10:03:23.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.680+0000 7ff9f7e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff9dc06b4d0 0x7ff9dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:23.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.681+0000 7ff9f7e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff9dc06b4d0 0x7ff9dc06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8001860 tx=0x7ff9e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:23.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.686+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffa000cf9d0 con 0x7ff9e4004d40 2024-04-25T10:03:23.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:23.883+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ffa00073a00 con 0x7ff9e4004d40 2024-04-25T10:03:24.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.467+0000 7ff9f7400640 1 -- 172.21.15.165:0/973740317 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v29) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff9f0019030 con 0x7ff9e4004d40 2024-04-25T10:03:24.472 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:03:24.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff9dc06b4d0 msgr2=0x7ff9dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff9dc06b4d0 0x7ff9dc06d990 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ff9e8001860 tx=0x7ff9e8009040 comp rx=0 tx=0).stop 2024-04-25T10:03:24.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 msgr2=0x7ffa00153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ffa00153010 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7ff9f00116d0 tx=0x7ff9f0011b90 comp rx=0 tx=0).stop 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 shutdown_connections 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff9dc06b4d0 0x7ff9dc06d990 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffa000c7460 0x7ffa000addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffa000b3dd0 0x7ffa000c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 --2- 172.21.15.165:0/973740317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9e4004d40 0x7ffa00153010 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 >> 172.21.15.165:0/973740317 conn(0x7ffa00071af0 msgr2=0x7ffa000b0e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 shutdown_connections 2024-04-25T10:03:24.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.471+0000 7ffa05e00640 1 -- 172.21.15.165:0/973740317 wait complete. 2024-04-25T10:03:24.505 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-25T10:03:24.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.702+0000 7f454c200640 1 Processor -- start 2024-04-25T10:03:24.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.702+0000 7f454c200640 1 -- start start 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.704+0000 7f454c200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f454406fbd0 0x7f45440ce1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.704+0000 7f454c200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f45440ce730 0x7f45440b5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.704+0000 7f454c200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45440b4d80 con 0x7f4544155f80 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.704+0000 7f454c200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440b5390 con 0x7f45440ce730 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.704+0000 7f454c200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45440d07f0 con 0x7f454406fbd0 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.704+0000 7f454a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f45440ce730 0x7f45440b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f454a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f45440ce730 0x7f45440b5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43746/0 (socket says 172.21.15.165:43746) 2024-04-25T10:03:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f454a200640 1 -- 172.21.15.165:0/3926416590 learned_addr learned my addr 172.21.15.165:0/3926416590 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f4548e00640 1 --2- 172.21.15.165:0/3926416590 >> v2:172.21.15.172:3300/0 conn(0x7f454406fbd0 0x7f45440ce1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f454a200640 1 -- 172.21.15.165:0/3926416590 >> v2:172.21.15.172:3300/0 conn(0x7f454406fbd0 msgr2=0x7f45440ce1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f454a200640 1 --2- 172.21.15.165:0/3926416590 >> v2:172.21.15.172:3300/0 conn(0x7f454406fbd0 0x7f45440ce1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f454a200640 1 -- 172.21.15.165:0/3926416590 >> v1:172.21.15.172:6789/0 conn(0x7f4544155f80 legacy=0x7f454406f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.705+0000 7f454a200640 1 -- 172.21.15.165:0/3926416590 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440cef70 con 0x7f45440ce730 2024-04-25T10:03:24.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.706+0000 7f454a200640 1 --2- 172.21.15.165:0/3926416590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45440ce730 0x7f45440b5820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45440d0280 tx=0x7f453c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5e5031cc4cd8527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.707+0000 7f4543e00640 1 -- 172.21.15.165:0/3926416590 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c036070 con 0x7f45440ce730 2024-04-25T10:03:24.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.707+0000 7f4543e00640 1 -- 172.21.15.165:0/3926416590 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c031600 con 0x7f45440ce730 2024-04-25T10:03:24.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.707+0000 7f4543e00640 1 -- 172.21.15.165:0/3926416590 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c0319d0 con 0x7f45440ce730 2024-04-25T10:03:24.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.707+0000 7f454c200640 1 -- 172.21.15.165:0/3926416590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45440ce730 msgr2=0x7f45440b5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.707+0000 7f454c200640 1 --2- 172.21.15.165:0/3926416590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45440ce730 0x7f45440b5820 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f45440d0280 tx=0x7f453c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.708+0000 7f454c200640 1 -- 172.21.15.165:0/3926416590 shutdown_connections 2024-04-25T10:03:24.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.708+0000 7f454c200640 1 --2- 172.21.15.165:0/3926416590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45440ce730 0x7f45440b5820 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.708+0000 7f454c200640 1 --2- 172.21.15.165:0/3926416590 >> v2:172.21.15.172:3300/0 conn(0x7f454406fbd0 0x7f45440ce1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.708+0000 7f454c200640 1 -- 172.21.15.165:0/3926416590 >> 172.21.15.165:0/3926416590 conn(0x7f4544061600 msgr2=0x7f45440619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:24.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.708+0000 7f454c200640 1 -- 172.21.15.165:0/3926416590 shutdown_connections 2024-04-25T10:03:24.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.708+0000 7f454c200640 1 -- 172.21.15.165:0/3926416590 wait complete. 2024-04-25T10:03:24.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.709+0000 7f454c200640 1 Processor -- start 2024-04-25T10:03:24.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.709+0000 7f454c200640 1 -- start start 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.710+0000 7f454c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f454406fbd0 0x7f454414b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.710+0000 7f454c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45440ce730 0x7f454414da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.710+0000 7f454c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 0x7f4544149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.710+0000 7f454c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45440cf250 con 0x7f4544155f80 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.710+0000 7f454c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4544156360 con 0x7f45440ce730 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.710+0000 7f454c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45440cf3d0 con 0x7f454406fbd0 2024-04-25T10:03:24.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 0x7f4544149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 0x7f4544149ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43748/0 (socket says 172.21.15.165:43748) 2024-04-25T10:03:24.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 -- 172.21.15.165:0/2674538447 learned_addr learned my addr 172.21.15.165:0/2674538447 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:24.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 -- 172.21.15.165:0/2674538447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f454406fbd0 msgr2=0x7f454414b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f454406fbd0 0x7f454414b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 -- 172.21.15.165:0/2674538447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45440ce730 msgr2=0x7f454414da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:24.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45440ce730 0x7f454414da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:24.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 -- 172.21.15.165:0/2674538447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f454414a400 con 0x7f4544155f80 2024-04-25T10:03:24.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.711+0000 7f454a200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 0x7f4544149ec0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f453c009f40 tx=0x7f453c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.712+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c036050 con 0x7f4544155f80 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.712+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c042070 con 0x7f4544155f80 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.712+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c009510 con 0x7f4544155f80 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.712+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f454414b780 con 0x7f4544155f80 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.712+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f454414bd40 con 0x7f4544155f80 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.713+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4544073a00 con 0x7f4544155f80 2024-04-25T10:03:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.714+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f453c007bd0 con 0x7f4544155f80 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.715+0000 7f4541600640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f452406b4d0 0x7f452406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:24.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.715+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f453c0ab260 con 0x7f4544155f80 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.715+0000 7f4549800640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f452406b4d0 0x7f452406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:24.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.716+0000 7f4549800640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f452406b4d0 0x7f452406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4538006fd0 tx=0x7f4538006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:24.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.720+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f453c045360 con 0x7f4544155f80 2024-04-25T10:03:24.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:24.931+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f45440b1030 con 0x7f4544155f80 2024-04-25T10:03:25.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.470+0000 7f4541600640 1 -- 172.21.15.165:0/2674538447 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v30) v1 ==== 178+0+0 (secure 0 0 0) 0x7f453c07a040 con 0x7f4544155f80 2024-04-25T10:03:25.476 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data_ec' removed 2024-04-25T10:03:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.474+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f452406b4d0 msgr2=0x7f452406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.474+0000 7f454c200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f452406b4d0 0x7f452406d990 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4538006fd0 tx=0x7f4538006bc0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.474+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 msgr2=0x7f4544149ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:25.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.474+0000 7f454c200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 0x7f4544149ec0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f453c009f40 tx=0x7f453c004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.475+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 shutdown_connections 2024-04-25T10:03:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.475+0000 7f454c200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f452406b4d0 0x7f452406d990 secure :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f4538006fd0 tx=0x7f4538006bc0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.475+0000 7f454c200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4544155f80 0x7f4544149ec0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.475+0000 7f454c200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f45440ce730 0x7f454414da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.475+0000 7f454c200640 1 --2- 172.21.15.165:0/2674538447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f454406fbd0 0x7f454414b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:25.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.475+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 >> 172.21.15.165:0/2674538447 conn(0x7f4544061600 msgr2=0x7f45440b33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:25.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.476+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 shutdown_connections 2024-04-25T10:03:25.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:25.476+0000 7f454c200640 1 -- 172.21.15.165:0/2674538447 wait complete. 2024-04-25T10:03:25.505 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:03:25.505 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:03:25.505 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:03:25.505 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:25.506 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:03:25.506 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:03:25.506 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:03:25.506 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:25.506 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:03:25.506 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:03:25.506 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:03:25.507 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:25.507 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:03:25.507 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:03:25.507 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:03:25.507 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:03:25.526 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:03:25.527 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:03:25.529 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:03:25.529 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:03:25.529 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:03:25.529 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:03:25.576 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:03:25.576 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:25.577 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:03:25.577 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:25.582 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:03:25.582 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:25.727 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:03:25.727 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:03:25.746 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:03:26.635 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:26.643 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:29.165 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:03:31.528 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:03:31.529 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:03:31.537 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:03:31.537 DEBUG:teuthology.parallel:result is None 2024-04-25T10:03:31.537 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:03:31.698 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:03:31.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 Processor -- start 2024-04-25T10:03:31.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 -- start start 2024-04-25T10:03:31.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681446d0 0x7fcb680b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb68144d50 0x7fcb680b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb680cccc0 con 0x7fcb681452a0 2024-04-25T10:03:31.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb680b2b20 con 0x7fcb68144d50 2024-04-25T10:03:31.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.719+0000 7fcb6d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6806d400 con 0x7fcb681446d0 2024-04-25T10:03:31.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb68144d50 0x7fcb680b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb68144d50 0x7fcb680b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48564/0 (socket says 172.21.15.165:48564) 2024-04-25T10:03:31.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 -- 172.21.15.165:0/3554836100 learned_addr learned my addr 172.21.15.165:0/3554836100 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:31.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb66800640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681446d0 0x7fcb680b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 -- 172.21.15.165:0/3554836100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681446d0 msgr2=0x7fcb680b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681446d0 0x7fcb680b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 -- 172.21.15.165:0/3554836100 >> v1:172.21.15.172:6789/0 conn(0x7fcb681452a0 legacy=0x7fcb680cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:31.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb5de00640 1 -- 172.21.15.165:0/3554836100 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb680cc140 con 0x7fcb68144d50 2024-04-25T10:03:31.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.720+0000 7fcb66800640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681446d0 0x7fcb680b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:31.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.721+0000 7fcb5de00640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b4460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009870 tx=0x7fcb5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0605c878b5b211 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb65e00640 1 -- 172.21.15.165:0/3554836100 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50037070 con 0x7fcb68144d50 2024-04-25T10:03:31.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb65e00640 1 -- 172.21.15.165:0/3554836100 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50032440 con 0x7fcb68144d50 2024-04-25T10:03:31.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb65e00640 1 -- 172.21.15.165:0/3554836100 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50032810 con 0x7fcb68144d50 2024-04-25T10:03:31.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 -- 172.21.15.165:0/3554836100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 msgr2=0x7fcb680b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b4460 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fcb50009870 tx=0x7fcb5002a210 comp rx=0 tx=0).stop 2024-04-25T10:03:31.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 -- 172.21.15.165:0/3554836100 shutdown_connections 2024-04-25T10:03:31.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b4460 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 --2- 172.21.15.165:0/3554836100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681446d0 0x7fcb680b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 -- 172.21.15.165:0/3554836100 >> 172.21.15.165:0/3554836100 conn(0x7fcb6806abe0 msgr2=0x7fcb6806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:31.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.722+0000 7fcb6d400640 1 -- 172.21.15.165:0/3554836100 shutdown_connections 2024-04-25T10:03:31.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.723+0000 7fcb6d400640 1 -- 172.21.15.165:0/3554836100 wait complete. 2024-04-25T10:03:31.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.724+0000 7fcb6d400640 1 Processor -- start 2024-04-25T10:03:31.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.724+0000 7fcb6d400640 1 -- start start 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.725+0000 7fcb6d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb681446d0 0x7fcb6812f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.725+0000 7fcb6d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.725+0000 7fcb6d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681452a0 0x7fcb681f7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.725+0000 7fcb6d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cc700 con 0x7fcb68144d50 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.725+0000 7fcb6d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680b49a0 con 0x7fcb681446d0 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.725+0000 7fcb6d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb680cc9e0 con 0x7fcb681452a0 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48578/0 (socket says 172.21.15.165:48578) 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 -- 172.21.15.165:0/289871936 learned_addr learned my addr 172.21.15.165:0/289871936 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 -- 172.21.15.165:0/289871936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681452a0 msgr2=0x7fcb681f7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681452a0 0x7fcb681f7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 -- 172.21.15.165:0/289871936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb681446d0 msgr2=0x7fcb6812f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:31.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb681446d0 0x7fcb6812f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:31.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 -- 172.21.15.165:0/289871936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb681f7cc0 con 0x7fcb68144d50 2024-04-25T10:03:31.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.726+0000 7fcb5de00640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b11d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.727+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50037040 con 0x7fcb68144d50 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.727+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb50042070 con 0x7fcb68144d50 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.727+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb681f7fa0 con 0x7fcb68144d50 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.727+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb50030bd0 con 0x7fcb68144d50 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.727+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb681e1150 con 0x7fcb68144d50 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.728+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb68140470 con 0x7fcb68144d50 2024-04-25T10:03:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.729+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb50047460 con 0x7fcb68144d50 2024-04-25T10:03:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.730+0000 7fcb5f400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4406b4d0 0x7fcb4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.730+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcb500aad80 con 0x7fcb68144d50 2024-04-25T10:03:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.730+0000 7fcb66800640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4406b4d0 0x7fcb4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.731+0000 7fcb66800640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4406b4d0 0x7fcb4406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb58008ea0 tx=0x7fcb580083d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:31.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.735+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb50045360 con 0x7fcb68144d50 2024-04-25T10:03:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:31.940+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fcb68073a00 con 0x7fcb68144d50 2024-04-25T10:03:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.776+0000 7fcb5f400640 1 -- 172.21.15.165:0/289871936 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v35) v1 ==== 97+0+0 (secure 0 0 0) 0x7fcb50079cd0 con 0x7fcb68144d50 2024-04-25T10:03:32.782 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4406b4d0 msgr2=0x7fcb4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4406b4d0 0x7fcb4406d990 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fcb58008ea0 tx=0x7fcb580083d0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 msgr2=0x7fcb680b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b11d0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fcb50000c00 tx=0x7fcb500046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 shutdown_connections 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4406b4d0 0x7fcb4406d990 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb681452a0 0x7fcb681f7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb68144d50 0x7fcb680b11d0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 --2- 172.21.15.165:0/289871936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb681446d0 0x7fcb6812f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:32.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 >> 172.21.15.165:0/289871936 conn(0x7fcb6806abe0 msgr2=0x7fcb680ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:32.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 shutdown_connections 2024-04-25T10:03:32.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.780+0000 7fcb6d400640 1 -- 172.21.15.165:0/289871936 wait complete. 2024-04-25T10:03:32.806 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:03:33.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.996+0000 7f26c3200640 1 Processor -- start 2024-04-25T10:03:33.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.996+0000 7f26c3200640 1 -- start start 2024-04-25T10:03:33.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.997+0000 7f26c3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc144ae0 0x7f26bc0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.997+0000 7f26c3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26bc15bcf0 0x7f26bc15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f26bc15fef0 0x7f26bc06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c3200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc15c520 con 0x7f26bc15fef0 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26bc15c800 con 0x7f26bc15bcf0 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc15cae0 con 0x7f26bc144ae0 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f26bc15fef0 0x7f26bc06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f26bc15fef0 0x7f26bc06d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48584/0 (socket says 172.21.15.165:48584) 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 -- 172.21.15.165:0/3998972847 learned_addr learned my addr 172.21.15.165:0/3998972847 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 -- 172.21.15.165:0/3998972847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc144ae0 msgr2=0x7f26bc0ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 --2- 172.21.15.165:0/3998972847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc144ae0 0x7f26bc0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 -- 172.21.15.165:0/3998972847 >> v2:172.21.15.172:3300/0 conn(0x7f26bc15bcf0 msgr2=0x7f26bc15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c1400640 1 --2- 172.21.15.165:0/3998972847 >> v2:172.21.15.172:3300/0 conn(0x7f26bc15bcf0 0x7f26bc15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 --2- 172.21.15.165:0/3998972847 >> v2:172.21.15.172:3300/0 conn(0x7f26bc15bcf0 0x7f26bc15f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.998+0000 7f26c2800640 1 -- 172.21.15.165:0/3998972847 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc15c150 con 0x7f26bc15fef0 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:32.999+0000 7f26c2800640 1 --2- 172.21.15.165:0/3998972847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc06d270 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0b4ac0 tx=0x7f26b802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d784ed6551f4fdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c0a00640 1 -- 172.21.15.165:0/3998972847 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8036070 con 0x7f26bc15fef0 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c0a00640 1 -- 172.21.15.165:0/3998972847 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b8031600 con 0x7f26bc15fef0 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c0a00640 1 -- 172.21.15.165:0/3998972847 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b80319d0 con 0x7f26bc15fef0 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 -- 172.21.15.165:0/3998972847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 msgr2=0x7f26bc06d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 --2- 172.21.15.165:0/3998972847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc06d270 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f26bc0b4ac0 tx=0x7f26b802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 -- 172.21.15.165:0/3998972847 shutdown_connections 2024-04-25T10:03:33.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 --2- 172.21.15.165:0/3998972847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc06d270 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 --2- 172.21.15.165:0/3998972847 >> v2:172.21.15.172:3300/0 conn(0x7f26bc15bcf0 0x7f26bc15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 --2- 172.21.15.165:0/3998972847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc144ae0 0x7f26bc0ccb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.000+0000 7f26c3200640 1 -- 172.21.15.165:0/3998972847 >> 172.21.15.165:0/3998972847 conn(0x7f26bc061c30 msgr2=0x7f26bc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.001+0000 7f26c3200640 1 -- 172.21.15.165:0/3998972847 shutdown_connections 2024-04-25T10:03:33.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.001+0000 7f26c3200640 1 -- 172.21.15.165:0/3998972847 wait complete. 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.002+0000 7f26c3200640 1 Processor -- start 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.002+0000 7f26c3200640 1 -- start start 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.003+0000 7f26c3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26bc144ae0 0x7f26bc1f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.003+0000 7f26c3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc15bcf0 0x7f26bc13de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.003+0000 7f26c3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.003+0000 7f26c3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc06db40 con 0x7f26bc15fef0 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.003+0000 7f26c3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc06d9c0 con 0x7f26bc144ae0 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.003+0000 7f26c3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26bc071140 con 0x7f26bc15bcf0 2024-04-25T10:03:33.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc0adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48594/0 (socket says 172.21.15.165:48594) 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 -- 172.21.15.165:0/2566169452 learned_addr learned my addr 172.21.15.165:0/2566169452 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 -- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc15bcf0 msgr2=0x7f26bc13de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc15bcf0 0x7f26bc13de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 -- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26bc144ae0 msgr2=0x7f26bc1f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c1e00640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26bc144ae0 0x7f26bc1f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26bc144ae0 0x7f26bc1f3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 -- 172.21.15.165:0/2566169452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26bc0b3620 con 0x7f26bc15fef0 2024-04-25T10:03:33.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.004+0000 7f26c2800640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc0adaf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f26b8034040 tx=0x7f26b8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.005+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8036050 con 0x7f26bc15fef0 2024-04-25T10:03:33.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.005+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b8042070 con 0x7f26bc15fef0 2024-04-25T10:03:33.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.005+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b8009510 con 0x7f26bc15fef0 2024-04-25T10:03:33.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.005+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26bc0aca50 con 0x7f26bc15fef0 2024-04-25T10:03:33.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.005+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26bc0ad010 con 0x7f26bc15fef0 2024-04-25T10:03:33.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.006+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26bc073990 con 0x7f26bc15fef0 2024-04-25T10:03:33.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.007+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26b8007bd0 con 0x7f26bc15fef0 2024-04-25T10:03:33.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.008+0000 7f26b6000640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26a006b4d0 0x7f26a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.008+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f26b80ab0c0 con 0x7f26bc15fef0 2024-04-25T10:03:33.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.008+0000 7f26c1e00640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26a006b4d0 0x7f26a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.009+0000 7f26c1e00640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26a006b4d0 0x7f26a006d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26bc15e260 tx=0x7f26b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.012+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26b8079a10 con 0x7f26bc15fef0 2024-04-25T10:03:33.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.278+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f26bc0b1ed0 con 0x7f26bc15fef0 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.279+0000 7f26b6000640 1 -- 172.21.15.165:0/2566169452 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f26b807dab0 con 0x7f26bc15fef0 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.285 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.286 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:03:33.287 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.289 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:03:33.290 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.291 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:03:33.292 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.293 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.294 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.295 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.bootstrap-mds", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQADKipmxXwmABAAR/NiKf0vZ5+xiD5hcEIddA==", 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile bootstrap-mds" 2024-04-25T10:03:33.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.bootstrap-mgr", 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQADKipm/oUmABAAw/7a7xEiueTX0QKcWBevTw==", 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile bootstrap-mgr" 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.bootstrap-osd", 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQADKipmnZImABAAM/3/HB3tzz0GKNzNv2z5RQ==", 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile bootstrap-osd" 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.297 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.bootstrap-rbd", 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQADKipmPJsmABAAn7WJ2IbWXFDI883ste3sWw==", 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile bootstrap-rbd" 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQADKipm0KMmABAAcxWUsBG5XbTb3XWoTijKBw==", 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.298 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.bootstrap-rgw", 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQADKipmn6wmABAAT1YsQKcLQXskpvYdqozepQ==", 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile bootstrap-rgw" 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.299 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:03:33.300 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26a006b4d0 msgr2=0x7f26a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26a006b4d0 0x7f26a006d990 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26bc15e260 tx=0x7f26b0008040 comp rx=0 tx=0).stop 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 msgr2=0x7f26bc0adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc0adaf0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f26b8034040 tx=0x7f26b8004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 shutdown_connections 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26a006b4d0 0x7f26a006d990 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26bc15fef0 0x7f26bc0adaf0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26bc15bcf0 0x7f26bc13de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 --2- 172.21.15.165:0/2566169452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26bc144ae0 0x7f26bc1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 >> 172.21.15.165:0/2566169452 conn(0x7f26bc061c30 msgr2=0x7f26bc0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 shutdown_connections 2024-04-25T10:03:33.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.282+0000 7f26c3200640 1 -- 172.21.15.165:0/2566169452 wait complete. 2024-04-25T10:03:33.306 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-04-25T10:03:33.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.498+0000 7fecb6200640 1 Processor -- start 2024-04-25T10:03:33.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.498+0000 7fecb6200640 1 -- start start 2024-04-25T10:03:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.499+0000 7fecb6200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fecb015dc60 0x7fecb00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.499+0000 7fecb6200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00cf450 con 0x7fecb00b2f90 2024-04-25T10:03:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.499+0000 7fecb6200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecb00cf730 con 0x7fecb015d880 2024-04-25T10:03:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.499+0000 7fecb6200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecb00d1400 con 0x7fecb015dc60 2024-04-25T10:03:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecb5800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fecb00b2f90 0x7fecb00cffd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34254/0 (socket says 172.21.15.165:34254) 2024-04-25T10:03:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecb5800640 1 -- 172.21.15.165:0/2564483986 learned_addr learned my addr 172.21.15.165:0/2564483986 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7feca7e00640 1 --2- 172.21.15.165:0/2564483986 >> v2:172.21.15.172:3300/0 conn(0x7fecb015dc60 0x7fecb00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3469466348 0 0) 0x7fecb00cf450 con 0x7fecb00b2f90 2024-04-25T10:03:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fec94003520 con 0x7fecb00b2f90 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1756989825 0 0) 0x7fec94003520 con 0x7fecb00b2f90 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 >> v2:172.21.15.172:3300/0 conn(0x7fecb015dc60 msgr2=0x7fecb00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 --2- 172.21.15.165:0/2564483986 >> v2:172.21.15.172:3300/0 conn(0x7fecb015dc60 0x7fecb00cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 >> v1:172.21.15.172:6789/0 conn(0x7fecb015d880 legacy=0x7fecb00d4190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.500+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00d16e0 con 0x7fecb00b2f90 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7feca80019f0 con 0x7fecb00b2f90 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 >> v1:172.21.15.165:6789/0 conn(0x7fecb00b2f90 legacy=0x7fecb00cffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec940039b0 0x7fec94003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec94004800 0x7fec94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fec9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00d1400 con 0x7fec940051c0 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400c0f0 con 0x7fec94004800 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec9400c2f0 con 0x7fec940039b0 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecb5800640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fec9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecb4e00640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec940039b0 0x7fec94003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7feca7e00640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec94004800 0x7fec94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecb5800640 1 -- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec940039b0 msgr2=0x7fec94003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecb5800640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fec940039b0 0x7fec94003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecb5800640 1 -- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec94004800 msgr2=0x7fec94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.501+0000 7fecb5800640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fec94004800 0x7fec94004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.502+0000 7fecb5800640 1 -- 172.21.15.165:0/2564483986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec94003520 con 0x7fec940051c0 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.502+0000 7fecb5800640 1 -- 172.21.15.165:0/2564483986 reap_dead start 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.502+0000 7fecb5800640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fec9400b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feca800a850 tx=0x7fec94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f90ebdd4898f5bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca800a4a0 con 0x7fec940051c0 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca802f070 con 0x7fec940051c0 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecafe00640 1 -- 172.21.15.165:0/2564483986 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca802bcb0 con 0x7fec940051c0 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecb6200640 1 -- 172.21.15.165:0/2564483986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 msgr2=0x7fec9400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecb6200640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fec9400b9c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feca800a850 tx=0x7fec94005830 comp rx=0 tx=0).stop 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecb6200640 1 -- 172.21.15.165:0/2564483986 shutdown_connections 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecb6200640 1 --2- 172.21.15.165:0/2564483986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fec9400b9c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.503+0000 7fecb6200640 1 -- 172.21.15.165:0/2564483986 >> 172.21.15.165:0/2564483986 conn(0x7fecb008afd0 msgr2=0x7fecb00d3790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.504+0000 7fecb6200640 1 -- 172.21.15.165:0/2564483986 shutdown_connections 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.504+0000 7fecb6200640 1 -- 172.21.15.165:0/2564483986 wait complete. 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.505+0000 7fecb6200640 1 Processor -- start 2024-04-25T10:03:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.505+0000 7fecb6200640 1 -- start start 2024-04-25T10:03:33.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.506+0000 7fecb6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fecb01528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.506+0000 7fecb6200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecb00cb6c0 0x7fecb00cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.506+0000 7fecb6200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecb00cc0f0 0x7fecb015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.506+0000 7fecb6200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cefc0 con 0x7fec940051c0 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.506+0000 7fecb6200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cee40 con 0x7fecb00cb6c0 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.506+0000 7fecb6200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecb00cf140 con 0x7fecb00cc0f0 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fecb01528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fecb01528f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48602/0 (socket says 172.21.15.165:48602) 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 -- 172.21.15.165:0/3754936239 learned_addr learned my addr 172.21.15.165:0/3754936239 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 -- 172.21.15.165:0/3754936239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecb00cc0f0 msgr2=0x7fecb015beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecb00cc0f0 0x7fecb015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 -- 172.21.15.165:0/3754936239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecb00cb6c0 msgr2=0x7fecb00cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecb00cb6c0 0x7fecb00cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 -- 172.21.15.165:0/3754936239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb00cc630 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecb4e00640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fecb01528f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feca000e490 tx=0x7feca000e950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.507+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca00135f0 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.508+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feca001a070 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.508+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feca00139d0 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.508+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecb00ca900 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.508+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecb00caec0 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.509+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecb006dcf0 con 0x7fec940051c0 2024-04-25T10:03:33.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.510+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feca0013b70 con 0x7fec940051c0 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.511+0000 7fecad600640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8c06b500 0x7fec8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.511+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7feca008a0e0 con 0x7fec940051c0 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.511+0000 7feca7e00640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8c06b500 0x7fec8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:33.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.512+0000 7feca7e00640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8c06b500 0x7fec8c06d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fecb015b870 tx=0x7fec9800d280 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:33.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.515+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecb006dcf0 con 0x7fec940051c0 2024-04-25T10:03:33.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.780+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fecb0073990 con 0x7fec940051c0 2024-04-25T10:03:33.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.788+0000 7fecad600640 1 -- 172.21.15.165:0/3754936239 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7feca0024020 con 0x7fec940051c0 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.790+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8c06b500 msgr2=0x7fec8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.790+0000 7fecb6200640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8c06b500 0x7fec8c06d9c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fecb015b870 tx=0x7fec9800d280 comp rx=0 tx=0).stop 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.790+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 msgr2=0x7fecb01528f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.790+0000 7fecb6200640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fecb01528f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7feca000e490 tx=0x7feca000e950 comp rx=0 tx=0).stop 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 shutdown_connections 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fec8c06b500 0x7fec8c06d9c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecb00cc0f0 0x7fecb015beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecb00cb6c0 0x7fecb00cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 --2- 172.21.15.165:0/3754936239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fec940051c0 0x7fecb01528f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 >> 172.21.15.165:0/3754936239 conn(0x7fecb008afd0 msgr2=0x7fecb00b5180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 shutdown_connections 2024-04-25T10:03:33.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:33.791+0000 7fecb6200640 1 -- 172.21.15.165:0/3754936239 wait complete. 2024-04-25T10:03:33.815 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-04-25T10:03:34.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.009+0000 7f3f5e000640 1 Processor -- start 2024-04-25T10:03:34.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.009+0000 7f3f5e000640 1 -- start start 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f5e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f581450c0 0x7f3f580cca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f5e000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f58072090 con 0x7f3f581454a0 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f5e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f5806e6a0 con 0x7f3f5815bf80 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f5e000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f5815c4d0 con 0x7f3f581450c0 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f57400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f581450c0 0x7f3f580cca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f56a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3f581454a0 0x7f3f58071270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34260/0 (socket says 172.21.15.165:34260) 2024-04-25T10:03:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.011+0000 7f3f56a00640 1 -- 172.21.15.165:0/1014697516 learned_addr learned my addr 172.21.15.165:0/1014697516 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:34.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.012+0000 7f3f57400640 1 -- 172.21.15.165:0/1014697516 >> v1:172.21.15.172:6789/0 conn(0x7f3f5815bf80 legacy=0x7f3f58071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.012+0000 7f3f56000640 1 -- 172.21.15.165:0/1014697516 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268145340 0 0) 0x7f3f58072090 con 0x7f3f581454a0 2024-04-25T10:03:34.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.012+0000 7f3f57400640 1 -- 172.21.15.165:0/1014697516 >> v1:172.21.15.165:6789/0 conn(0x7f3f581454a0 legacy=0x7f3f58071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.012+0000 7f3f57400640 1 -- 172.21.15.165:0/1014697516 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f5815c7b0 con 0x7f3f581450c0 2024-04-25T10:03:34.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.013+0000 7f3f57400640 1 --2- 172.21.15.165:0/1014697516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581450c0 0x7f3f580cca30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3f480097a0 tx=0x7f3f4802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3d902e0a9592b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.013+0000 7f3f56000640 1 -- 172.21.15.165:0/1014697516 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48037070 con 0x7f3f581450c0 2024-04-25T10:03:34.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.013+0000 7f3f56000640 1 -- 172.21.15.165:0/1014697516 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f48032440 con 0x7f3f581450c0 2024-04-25T10:03:34.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.013+0000 7f3f56000640 1 -- 172.21.15.165:0/1014697516 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48032810 con 0x7f3f581450c0 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 -- 172.21.15.165:0/1014697516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581450c0 msgr2=0x7f3f580cca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 --2- 172.21.15.165:0/1014697516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581450c0 0x7f3f580cca30 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f3f480097a0 tx=0x7f3f4802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 -- 172.21.15.165:0/1014697516 shutdown_connections 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 --2- 172.21.15.165:0/1014697516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581450c0 0x7f3f580cca30 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 -- 172.21.15.165:0/1014697516 >> 172.21.15.165:0/1014697516 conn(0x7f3f5806abe0 msgr2=0x7f3f5806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 -- 172.21.15.165:0/1014697516 shutdown_connections 2024-04-25T10:03:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.014+0000 7f3f5e000640 1 -- 172.21.15.165:0/1014697516 wait complete. 2024-04-25T10:03:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.016+0000 7f3f5e000640 1 Processor -- start 2024-04-25T10:03:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.016+0000 7f3f5e000640 1 -- start start 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 0x7f3f5815b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f5e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f5815bf80 0x7f3f5815e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f5e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f5815e670 0x7f3f5813e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f5e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580ce8e0 con 0x7f3f581454a0 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f5e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5815cd70 con 0x7f3f5815bf80 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f5e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580cea60 con 0x7f3f5815e670 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 0x7f3f5815b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 0x7f3f5815b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48614/0 (socket says 172.21.15.165:48614) 2024-04-25T10:03:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 -- 172.21.15.165:0/4191086352 learned_addr learned my addr 172.21.15.165:0/4191086352 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 -- 172.21.15.165:0/4191086352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f5815e670 msgr2=0x7f3f5813e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f5815e670 0x7f3f5813e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 -- 172.21.15.165:0/4191086352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f5815bf80 msgr2=0x7f3f5815e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.017+0000 7f3f57400640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f5815bf80 0x7f3f5815e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f57400640 1 -- 172.21.15.165:0/4191086352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f581f4870 con 0x7f3f581454a0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f57400640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 0x7f3f5815b3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3f480097a0 tx=0x7f3f480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48041070 con 0x7f3f581454a0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4803c870 con 0x7f3f581454a0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f48037040 con 0x7f3f581454a0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f581f4b50 con 0x7f3f581454a0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.018+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f581f5110 con 0x7f3f581454a0 2024-04-25T10:03:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.019+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5815ea90 con 0x7f3f581454a0 2024-04-25T10:03:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.020+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f48044070 con 0x7f3f581454a0 2024-04-25T10:03:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.022+0000 7f3f4f400640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f3806b4d0 0x7f3f3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.022+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f3f480aac90 con 0x7f3f581454a0 2024-04-25T10:03:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.022+0000 7f3f56a00640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f3806b4d0 0x7f3f3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.023+0000 7f3f56a00640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f3806b4d0 0x7f3f3806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3f44007fd0 tx=0x7f3f44007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.027+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f48079de0 con 0x7f3f581454a0 2024-04-25T10:03:34.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.292+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f3f58073990 con 0x7f3f581454a0 2024-04-25T10:03:34.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.297+0000 7f3f4f400640 1 -- 172.21.15.165:0/4191086352 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f3f480796e0 con 0x7f3f581454a0 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f3806b4d0 msgr2=0x7f3f3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f3806b4d0 0x7f3f3806d990 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3f44007fd0 tx=0x7f3f44007920 comp rx=0 tx=0).stop 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 msgr2=0x7f3f5815b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 0x7f3f5815b3e0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f3f480097a0 tx=0x7f3f480046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 shutdown_connections 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f3806b4d0 0x7f3f3806d990 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f5815e670 0x7f3f5813e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f5815bf80 0x7f3f5815e130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 --2- 172.21.15.165:0/4191086352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f581454a0 0x7f3f5815b3e0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 >> 172.21.15.165:0/4191086352 conn(0x7f3f5806abe0 msgr2=0x7f3f58071a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 shutdown_connections 2024-04-25T10:03:34.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.300+0000 7f3f5e000640 1 -- 172.21.15.165:0/4191086352 wait complete. 2024-04-25T10:03:34.324 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.505+0000 7fcc39600640 1 Processor -- start 2024-04-25T10:03:34.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.505+0000 7fcc39600640 1 -- start start 2024-04-25T10:03:34.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc39600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc3406e3b0 0x7fcc3406e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc39600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc340b2630 0x7fcc340b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc39600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc34144370 con 0x7fcc340ce870 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc39600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc340cf3e0 con 0x7fcc3406e3b0 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc39600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3415b7c0 con 0x7fcc340b2630 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc33e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc3406e3b0 0x7fcc3406e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.507+0000 7fcc33e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc3406e3b0 0x7fcc3406e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48618/0 (socket says 172.21.15.165:48618) 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.508+0000 7fcc33e00640 1 -- 172.21.15.165:0/3894921282 learned_addr learned my addr 172.21.15.165:0/3894921282 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.508+0000 7fcc38c00640 1 --2- 172.21.15.165:0/3894921282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc340b2630 0x7fcc340b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.508+0000 7fcc33e00640 1 -- 172.21.15.165:0/3894921282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc340b2630 msgr2=0x7fcc340b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.508+0000 7fcc33e00640 1 --2- 172.21.15.165:0/3894921282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc340b2630 0x7fcc340b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.508+0000 7fcc33e00640 1 -- 172.21.15.165:0/3894921282 >> v1:172.21.15.172:6789/0 conn(0x7fcc340ce870 legacy=0x7fcc340cecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.508+0000 7fcc33e00640 1 -- 172.21.15.165:0/3894921282 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3415abd0 con 0x7fcc3406e3b0 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.509+0000 7fcc33e00640 1 --2- 172.21.15.165:0/3894921282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc3406e3b0 0x7fcc3406e790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c8c4b388b4222a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.509+0000 7fcc33400640 1 -- 172.21.15.165:0/3894921282 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24037070 con 0x7fcc3406e3b0 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.509+0000 7fcc33400640 1 -- 172.21.15.165:0/3894921282 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc24032440 con 0x7fcc3406e3b0 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.509+0000 7fcc33400640 1 -- 172.21.15.165:0/3894921282 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc24032810 con 0x7fcc3406e3b0 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 -- 172.21.15.165:0/3894921282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc3406e3b0 msgr2=0x7fcc3406e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 --2- 172.21.15.165:0/3894921282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc3406e3b0 0x7fcc3406e790 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fcc24009870 tx=0x7fcc2402a210 comp rx=0 tx=0).stop 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 -- 172.21.15.165:0/3894921282 shutdown_connections 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 --2- 172.21.15.165:0/3894921282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc340b2630 0x7fcc340b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 --2- 172.21.15.165:0/3894921282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc3406e3b0 0x7fcc3406e790 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 -- 172.21.15.165:0/3894921282 >> 172.21.15.165:0/3894921282 conn(0x7fcc3406b000 msgr2=0x7fcc3406b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 -- 172.21.15.165:0/3894921282 shutdown_connections 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.510+0000 7fcc39600640 1 -- 172.21.15.165:0/3894921282 wait complete. 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.512+0000 7fcc39600640 1 Processor -- start 2024-04-25T10:03:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.512+0000 7fcc39600640 1 -- start start 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc39600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3406e3b0 0x7fcc3415c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc39600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 0x7fcc340af310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc39600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc340ce870 0x7fcc340af850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc39600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3415b450 con 0x7fcc340b2630 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc39600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3415b170 con 0x7fcc340ce870 2024-04-25T10:03:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc39600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc3406ecd0 con 0x7fcc3406e3b0 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 0x7fcc340af310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 0x7fcc340af310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48620/0 (socket says 172.21.15.165:48620) 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.513+0000 7fcc2b400640 1 -- 172.21.15.165:0/2474762478 learned_addr learned my addr 172.21.15.165:0/2474762478 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc33e00640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3406e3b0 0x7fcc3415c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc38c00640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc340ce870 0x7fcc340af850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc2b400640 1 -- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3406e3b0 msgr2=0x7fcc3415c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc2b400640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3406e3b0 0x7fcc3415c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc2b400640 1 -- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc340ce870 msgr2=0x7fcc340af850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc2b400640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc340ce870 0x7fcc340af850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc2b400640 1 -- 172.21.15.165:0/2474762478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3415a2b0 con 0x7fcc340b2630 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc38c00640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc340ce870 0x7fcc340af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc2b400640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 0x7fcc340af310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b3f0 tx=0x7fcc1c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c015020 con 0x7fcc340b2630 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc1c011830 con 0x7fcc340b2630 2024-04-25T10:03:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.514+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1c011c20 con 0x7fcc340b2630 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.515+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc340b0760 con 0x7fcc340b2630 2024-04-25T10:03:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.515+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc340b0d20 con 0x7fcc340b2630 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.516+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc34073990 con 0x7fcc340b2630 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.517+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc1c018020 con 0x7fcc340b2630 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.518+0000 7fcc30c00640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc1006b500 0x7fcc1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.518+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcc1c089c20 con 0x7fcc340b2630 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.518+0000 7fcc33e00640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc1006b500 0x7fcc1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:34.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.519+0000 7fcc33e00640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc1006b500 0x7fcc1006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc24034000 tx=0x7fcc2402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:34.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.523+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc1c01f030 con 0x7fcc340b2630 2024-04-25T10:03:34.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.792+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fcc340b3e60 con 0x7fcc340b2630 2024-04-25T10:03:34.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.799+0000 7fcc30c00640 1 -- 172.21.15.165:0/2474762478 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcc1c0172c0 con 0x7fcc340b2630 2024-04-25T10:03:34.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc1006b500 msgr2=0x7fcc1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc1006b500 0x7fcc1006d9c0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fcc24034000 tx=0x7fcc2402e000 comp rx=0 tx=0).stop 2024-04-25T10:03:34.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 msgr2=0x7fcc340af310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:34.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 0x7fcc340af310 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fcc1c00b3f0 tx=0x7fcc1c00b8b0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 shutdown_connections 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc1006b500 0x7fcc1006d9c0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc340ce870 0x7fcc340af850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc340b2630 0x7fcc340af310 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 --2- 172.21.15.165:0/2474762478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc3406e3b0 0x7fcc3415c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 >> 172.21.15.165:0/2474762478 conn(0x7fcc3406b000 msgr2=0x7fcc3415d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 shutdown_connections 2024-04-25T10:03:34.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:34.802+0000 7fcc39600640 1 -- 172.21.15.165:0/2474762478 wait complete. 2024-04-25T10:03:34.826 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-04-25T10:03:35.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.016+0000 7fb654c00640 1 Processor -- start 2024-04-25T10:03:35.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.016+0000 7fb654c00640 1 -- start start 2024-04-25T10:03:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.017+0000 7fb654c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6500b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.017+0000 7fb654c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65015fa20 con 0x7fb65006e290 2024-04-25T10:03:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.017+0000 7fb654c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb65015fd00 con 0x7fb6500cdee0 2024-04-25T10:03:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.017+0000 7fb654c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb65015bf80 con 0x7fb6500b4940 2024-04-25T10:03:35.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.018+0000 7fb64ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb65006e290 0x7fb65015c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34270/0 (socket says 172.21.15.165:34270) 2024-04-25T10:03:35.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.018+0000 7fb64ea00640 1 -- 172.21.15.165:0/3075243276 learned_addr learned my addr 172.21.15.165:0/3075243276 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.018+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778745586 0 0) 0x7fb65015fa20 con 0x7fb65006e290 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.018+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb634003520 con 0x7fb65006e290 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.018+0000 7fb64fe00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6500b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2131719151 0 0) 0x7fb634003520 con 0x7fb65006e290 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 msgr2=0x7fb6500b4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6500b4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 >> v1:172.21.15.172:6789/0 conn(0x7fb6500cdee0 legacy=0x7fb65015f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65015ffe0 con 0x7fb65006e290 2024-04-25T10:03:35.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb63c001670 con 0x7fb65006e290 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 >> v1:172.21.15.165:6789/0 conn(0x7fb65006e290 legacy=0x7fb65015c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6340039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb634003f60 0x7fb634004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb634004d40 0x7fb6340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65015bf80 con 0x7fb634003f60 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb63400baa0 con 0x7fb634004d40 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb63400bca0 con 0x7fb6500b4940 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.019+0000 7fb64ea00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb634003f60 0x7fb634004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64ea00640 1 -- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 msgr2=0x7fb6340039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64f400640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6340039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64fe00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb634004d40 0x7fb6340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64ea00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6340039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64ea00640 1 -- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb634004d40 msgr2=0x7fb6340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64ea00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb634004d40 0x7fb6340051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64ea00640 1 -- 172.21.15.165:0/3075243276 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb634003520 con 0x7fb634003f60 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64f400640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6500b4940 0x7fb6340039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.020+0000 7fb64fe00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb634004d40 0x7fb6340051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.021+0000 7fb64ea00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb634003f60 0x7fb634004800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c210 tx=0x7fb634005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e8aeb0f4ab3df server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.021+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c007d30 con 0x7fb634003f60 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.021+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb63c005ad0 con 0x7fb634003f60 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.021+0000 7fb64e000640 1 -- 172.21.15.165:0/3075243276 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63c007070 con 0x7fb634003f60 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb654c00640 1 -- 172.21.15.165:0/3075243276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb634003f60 msgr2=0x7fb634004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb654c00640 1 --2- 172.21.15.165:0/3075243276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb634003f60 0x7fb634004800 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb63c00c210 tx=0x7fb634005830 comp rx=0 tx=0).stop 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb64fe00640 1 -- 172.21.15.165:0/3075243276 reap_dead start 2024-04-25T10:03:35.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb654c00640 1 -- 172.21.15.165:0/3075243276 shutdown_connections 2024-04-25T10:03:35.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb654c00640 1 -- 172.21.15.165:0/3075243276 >> 172.21.15.165:0/3075243276 conn(0x7fb65006b000 msgr2=0x7fb65006b410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb654c00640 1 -- 172.21.15.165:0/3075243276 shutdown_connections 2024-04-25T10:03:35.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.022+0000 7fb654c00640 1 -- 172.21.15.165:0/3075243276 wait complete. 2024-04-25T10:03:35.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.024+0000 7fb654c00640 1 Processor -- start 2024-04-25T10:03:35.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.024+0000 7fb654c00640 1 -- start start 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb654c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb634003f60 0x7fb650155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb654c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6501559b0 0x7fb6500addf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb654c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 0x7fb650145ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb654c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b4dd0 con 0x7fb6500ae330 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb654c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65015bbe0 con 0x7fb6501559b0 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb654c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6500b4f50 con 0x7fb634003f60 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 0x7fb650145ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 0x7fb650145ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48646/0 (socket says 172.21.15.165:48646) 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 -- 172.21.15.165:0/264727895 learned_addr learned my addr 172.21.15.165:0/264727895 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64ea00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6501559b0 0x7fb6500addf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 -- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb634003f60 msgr2=0x7fb650155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64f400640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb634003f60 0x7fb650155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb634003f60 0x7fb650155470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 -- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6501559b0 msgr2=0x7fb6500addf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6501559b0 0x7fb6500addf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64fe00640 1 -- 172.21.15.165:0/264727895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6500aeb70 con 0x7fb6500ae330 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.025+0000 7fb64ea00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6501559b0 0x7fb6500addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.026+0000 7fb64fe00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 0x7fb650145ef0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb6500cf4e0 tx=0x7fb64800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.026+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb64801c070 con 0x7fb6500ae330 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.026+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb648017400 con 0x7fb6500ae330 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.026+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6480177d0 con 0x7fb6500ae330 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.026+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb650143f00 con 0x7fb6500ae330 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.026+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6501444c0 con 0x7fb6500ae330 2024-04-25T10:03:35.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.027+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb650073990 con 0x7fb6500ae330 2024-04-25T10:03:35.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.028+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb648005d10 con 0x7fb6500ae330 2024-04-25T10:03:35.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.029+0000 7fb647400640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb63006b4d0 0x7fb63006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.029+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb64808a2b0 con 0x7fb6500ae330 2024-04-25T10:03:35.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.030+0000 7fb64f400640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb63006b4d0 0x7fb63006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.030+0000 7fb64f400640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb63006b4d0 0x7fb63006d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb640001630 tx=0x7fb640009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.034+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb648012070 con 0x7fb6500ae330 2024-04-25T10:03:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.286+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fb650155d90 con 0x7fb6500ae330 2024-04-25T10:03:35.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.291+0000 7fb647400640 1 -- 172.21.15.165:0/264727895 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fb6480591c0 con 0x7fb6500ae330 2024-04-25T10:03:35.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb63006b4d0 msgr2=0x7fb63006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb63006b4d0 0x7fb63006d990 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fb640001630 tx=0x7fb640009040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 msgr2=0x7fb650145ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 0x7fb650145ef0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb6500cf4e0 tx=0x7fb64800f4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 shutdown_connections 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb63006b4d0 0x7fb63006d990 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6500ae330 0x7fb650145ef0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb6501559b0 0x7fb6500addf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 --2- 172.21.15.165:0/264727895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb634003f60 0x7fb650155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.293+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 >> 172.21.15.165:0/264727895 conn(0x7fb65006b000 msgr2=0x7fb650152d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.294+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 shutdown_connections 2024-04-25T10:03:35.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.294+0000 7fb654c00640 1 -- 172.21.15.165:0/264727895 wait complete. 2024-04-25T10:03:35.316 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-04-25T10:03:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.509+0000 7fbe19200640 1 Processor -- start 2024-04-25T10:03:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.510+0000 7fbe19200640 1 -- start start 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.511+0000 7fbe19200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe1415b3c0 0x7fbe140b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.511+0000 7fbe19200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b7a0 0x7fbe140cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.511+0000 7fbe19200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbe140cdae0 0x7fbe140cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.511+0000 7fbe19200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe140cd650 con 0x7fbe140cdae0 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.511+0000 7fbe19200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe140b0b80 con 0x7fbe1415b3c0 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.511+0000 7fbe19200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe140cc800 con 0x7fbe1415b7a0 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbe140cdae0 0x7fbe140cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbe140cdae0 0x7fbe140cdf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48648/0 (socket says 172.21.15.165:48648) 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 -- 172.21.15.165:0/4045242208 learned_addr learned my addr 172.21.15.165:0/4045242208 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13400640 1 --2- 172.21.15.165:0/4045242208 >> v2:172.21.15.172:3300/0 conn(0x7fbe1415b3c0 0x7fbe140b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe0be00640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b7a0 0x7fbe140cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 -- 172.21.15.165:0/4045242208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b7a0 msgr2=0x7fbe140cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b7a0 0x7fbe140cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 -- 172.21.15.165:0/4045242208 >> v2:172.21.15.172:3300/0 conn(0x7fbe1415b3c0 msgr2=0x7fbe140b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 --2- 172.21.15.165:0/4045242208 >> v2:172.21.15.172:3300/0 conn(0x7fbe1415b3c0 0x7fbe140b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe13e00640 1 -- 172.21.15.165:0/4045242208 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe140ccae0 con 0x7fbe140cdae0 2024-04-25T10:03:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.512+0000 7fbe0be00640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b7a0 0x7fbe140cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.514+0000 7fbe13e00640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe140cdae0 0x7fbe140cdf40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbe1406f690 tx=0x7fbe0c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f63ff785ea6cd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.514+0000 7fbe12a00640 1 -- 172.21.15.165:0/4045242208 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c036070 con 0x7fbe140cdae0 2024-04-25T10:03:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.514+0000 7fbe12a00640 1 -- 172.21.15.165:0/4045242208 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c031600 con 0x7fbe140cdae0 2024-04-25T10:03:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.514+0000 7fbe12a00640 1 -- 172.21.15.165:0/4045242208 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c0319d0 con 0x7fbe140cdae0 2024-04-25T10:03:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 -- 172.21.15.165:0/4045242208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe140cdae0 msgr2=0x7fbe140cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe140cdae0 0x7fbe140cdf40 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fbe1406f690 tx=0x7fbe0c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 -- 172.21.15.165:0/4045242208 shutdown_connections 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe140cdae0 0x7fbe140cdf40 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 --2- 172.21.15.165:0/4045242208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b7a0 0x7fbe140cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 --2- 172.21.15.165:0/4045242208 >> v2:172.21.15.172:3300/0 conn(0x7fbe1415b3c0 0x7fbe140b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 -- 172.21.15.165:0/4045242208 >> 172.21.15.165:0/4045242208 conn(0x7fbe14061cc0 msgr2=0x7fbe1406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 -- 172.21.15.165:0/4045242208 shutdown_connections 2024-04-25T10:03:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.515+0000 7fbe19200640 1 -- 172.21.15.165:0/4045242208 wait complete. 2024-04-25T10:03:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.517+0000 7fbe19200640 1 Processor -- start 2024-04-25T10:03:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.517+0000 7fbe19200640 1 -- start start 2024-04-25T10:03:35.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe19200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe140cdae0 0x7fbe140cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe19200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b3c0 0x7fbe14146070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe19200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 0x7fbe141478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe19200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1415e190 con 0x7fbe1415b7a0 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe19200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1415e010 con 0x7fbe140cdae0 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe19200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1415e310 con 0x7fbe1415b3c0 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe13e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 0x7fbe141478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe13e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 0x7fbe141478f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48662/0 (socket says 172.21.15.165:48662) 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.518+0000 7fbe13e00640 1 -- 172.21.15.165:0/2108446677 learned_addr learned my addr 172.21.15.165:0/2108446677 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13400640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe140cdae0 0x7fbe140cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe0be00640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b3c0 0x7fbe14146070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13e00640 1 -- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b3c0 msgr2=0x7fbe14146070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13e00640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b3c0 0x7fbe14146070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13e00640 1 -- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe140cdae0 msgr2=0x7fbe140cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13e00640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe140cdae0 0x7fbe140cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13e00640 1 -- 172.21.15.165:0/2108446677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe141465b0 con 0x7fbe1415b7a0 2024-04-25T10:03:35.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe13e00640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 0x7fbe141478f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02ac90 tx=0x7fbe0c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.519+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c03f020 con 0x7fbe1415b7a0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.520+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe0c037040 con 0x7fbe1415b7a0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.520+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe14146850 con 0x7fbe1415b7a0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.520+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe0c036070 con 0x7fbe1415b7a0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.520+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe14146e10 con 0x7fbe1415b7a0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.521+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe14073990 con 0x7fbe1415b7a0 2024-04-25T10:03:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.522+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe0c042020 con 0x7fbe1415b7a0 2024-04-25T10:03:35.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.523+0000 7fbe0b400640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdf006b4d0 0x7fbdf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:35.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.523+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbe0c0aa4f0 con 0x7fbe1415b7a0 2024-04-25T10:03:35.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.523+0000 7fbe13400640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdf006b4d0 0x7fbdf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:35.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.524+0000 7fbe13400640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdf006b4d0 0x7fbdf006d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbe04006fd0 tx=0x7fbe04008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:35.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.527+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe0c049030 con 0x7fbe1415b7a0 2024-04-25T10:03:35.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.791+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fbe14072360 con 0x7fbe1415b7a0 2024-04-25T10:03:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.805+0000 7fbe0b400640 1 -- 172.21.15.165:0/2108446677 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbe0c079520 con 0x7fbe1415b7a0 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdf006b4d0 msgr2=0x7fbdf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdf006b4d0 0x7fbdf006d990 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbe04006fd0 tx=0x7fbe04008040 comp rx=0 tx=0).stop 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 msgr2=0x7fbe141478f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 0x7fbe141478f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fbe0c02ac90 tx=0x7fbe0c004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 shutdown_connections 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdf006b4d0 0x7fbdf006d990 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe1415b7a0 0x7fbe141478f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe1415b3c0 0x7fbe14146070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 --2- 172.21.15.165:0/2108446677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe140cdae0 0x7fbe140cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 >> 172.21.15.165:0/2108446677 conn(0x7fbe14061cc0 msgr2=0x7fbe14071a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 shutdown_connections 2024-04-25T10:03:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:35.808+0000 7fbe19200640 1 -- 172.21.15.165:0/2108446677 wait complete. 2024-04-25T10:03:35.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-04-25T10:03:35.878 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:03:36.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.013+0000 7fe082200640 1 Processor -- start 2024-04-25T10:03:36.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.013+0000 7fe082200640 1 -- start start 2024-04-25T10:03:36.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.014+0000 7fe082200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe07c0cd140 0x7fe07c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe082200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c06dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe082200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07c15c4d0 con 0x7fe07c1450c0 2024-04-25T10:03:36.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe082200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07c15c7b0 con 0x7fe07c0cd140 2024-04-25T10:03:36.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe082200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c15ca90 con 0x7fe07c1454a0 2024-04-25T10:03:36.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe07be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c06dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe07aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe07c0cd140 0x7fe07c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe07be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c06dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:49788/0 (socket says 172.21.15.165:49788) 2024-04-25T10:03:36.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe07aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe07c0cd140 0x7fe07c06d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:34020/0 (socket says 172.21.15.165:34020) 2024-04-25T10:03:36.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.015+0000 7fe07be00640 1 -- 172.21.15.165:0/397035900 learned_addr learned my addr 172.21.15.165:0/397035900 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:36.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.016+0000 7fe07be00640 1 -- 172.21.15.165:0/397035900 >> v2:172.21.15.172:3300/0 conn(0x7fe07c0cd140 msgr2=0x7fe07c06d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.016+0000 7fe07be00640 1 --2- 172.21.15.165:0/397035900 >> v2:172.21.15.172:3300/0 conn(0x7fe07c0cd140 0x7fe07c06d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.016+0000 7fe07be00640 1 -- 172.21.15.165:0/397035900 >> v1:172.21.15.172:6789/0 conn(0x7fe07c1450c0 legacy=0x7fe07c0cca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.016+0000 7fe07be00640 1 -- 172.21.15.165:0/397035900 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c15cd70 con 0x7fe07c1454a0 2024-04-25T10:03:36.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.017+0000 7fe07be00640 1 --2- 172.21.15.165:0/397035900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c06dbd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe074009e50 tx=0x7fe07402a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b4fb9c8311fb850 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.018+0000 7fe07a000640 1 -- 172.21.15.165:0/397035900 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe074036070 con 0x7fe07c1454a0 2024-04-25T10:03:36.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.018+0000 7fe07a000640 1 -- 172.21.15.165:0/397035900 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe074031600 con 0x7fe07c1454a0 2024-04-25T10:03:36.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.018+0000 7fe07a000640 1 -- 172.21.15.165:0/397035900 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0740319d0 con 0x7fe07c1454a0 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 -- 172.21.15.165:0/397035900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 msgr2=0x7fe07c06dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 --2- 172.21.15.165:0/397035900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c06dbd0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe074009e50 tx=0x7fe07402a710 comp rx=0 tx=0).stop 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 -- 172.21.15.165:0/397035900 shutdown_connections 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 --2- 172.21.15.165:0/397035900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c06dbd0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 --2- 172.21.15.165:0/397035900 >> v2:172.21.15.172:3300/0 conn(0x7fe07c0cd140 0x7fe07c06d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 -- 172.21.15.165:0/397035900 >> 172.21.15.165:0/397035900 conn(0x7fe07c06abe0 msgr2=0x7fe07c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 -- 172.21.15.165:0/397035900 shutdown_connections 2024-04-25T10:03:36.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.019+0000 7fe082200640 1 -- 172.21.15.165:0/397035900 wait complete. 2024-04-25T10:03:36.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.021+0000 7fe082200640 1 Processor -- start 2024-04-25T10:03:36.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.021+0000 7fe082200640 1 -- start start 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe082200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07c0cd140 0x7fe07c0c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe082200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 0x7fe07c0b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe082200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c0b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe082200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c06ef30 con 0x7fe07c1450c0 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe082200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c0ceaf0 con 0x7fe07c0cd140 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe082200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c06f0b0 con 0x7fe07c1454a0 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe07aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 0x7fe07c0b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe07aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 0x7fe07c0b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48670/0 (socket says 172.21.15.165:48670) 2024-04-25T10:03:36.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.022+0000 7fe07aa00640 1 -- 172.21.15.165:0/3931691346 learned_addr learned my addr 172.21.15.165:0/3931691346 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07be00640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c0b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07b400640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07c0cd140 0x7fe07c0c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07aa00640 1 -- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 msgr2=0x7fe07c0b1900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07aa00640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c0b1900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07aa00640 1 -- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07c0cd140 msgr2=0x7fe07c0c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07aa00640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07c0cd140 0x7fe07c0c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07aa00640 1 -- 172.21.15.165:0/3931691346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0c5430 con 0x7fe07c1450c0 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07b400640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07c0cd140 0x7fe07c0c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe07aa00640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 0x7fe07c0b13c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe06800e9b0 tx=0x7fe06800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe068017070 con 0x7fe07c1450c0 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe06800c9a0 con 0x7fe07c1450c0 2024-04-25T10:03:36.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe06800cd90 con 0x7fe07c1450c0 2024-04-25T10:03:36.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c0c4e70 con 0x7fe07c1450c0 2024-04-25T10:03:36.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.023+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c0b2970 con 0x7fe07c1450c0 2024-04-25T10:03:36.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.024+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c073990 con 0x7fe07c1450c0 2024-04-25T10:03:36.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.025+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe06801a070 con 0x7fe07c1450c0 2024-04-25T10:03:36.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.026+0000 7fe073400640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.026+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe068089fe0 con 0x7fe07c1450c0 2024-04-25T10:03:36.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.027+0000 7fe07b400640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.027+0000 7fe07b400640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05c06b4d0 0x7fe05c06d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe06c001680 tx=0x7fe06c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.030+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe068059070 con 0x7fe07c1450c0 2024-04-25T10:03:36.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.296+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fe07c0b4850 con 0x7fe07c1450c0 2024-04-25T10:03:36.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.301+0000 7fe073400640 1 -- 172.21.15.165:0/3931691346 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe068058970 con 0x7fe07c1450c0 2024-04-25T10:03:36.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05c06b4d0 msgr2=0x7fe05c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05c06b4d0 0x7fe05c06d990 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe06c001680 tx=0x7fe06c009040 comp rx=0 tx=0).stop 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 msgr2=0x7fe07c0b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 0x7fe07c0b13c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe06800e9b0 tx=0x7fe06800ee70 comp rx=0 tx=0).stop 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 shutdown_connections 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05c06b4d0 0x7fe05c06d990 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe07c1454a0 0x7fe07c0b1900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07c1450c0 0x7fe07c0b13c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 --2- 172.21.15.165:0/3931691346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07c0cd140 0x7fe07c0c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 >> 172.21.15.165:0/3931691346 conn(0x7fe07c06abe0 msgr2=0x7fe07c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 shutdown_connections 2024-04-25T10:03:36.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.304+0000 7fe082200640 1 -- 172.21.15.165:0/3931691346 wait complete. 2024-04-25T10:03:36.329 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:36.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.521+0000 7fd665000640 1 Processor -- start 2024-04-25T10:03:36.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.521+0000 7fd665000640 1 -- start start 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd665000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd66006e870 0x7fd6600cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd665000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6600cc840 0x7fd66015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd665000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd660144240 con 0x7fd660143e60 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd665000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66015e220 con 0x7fd66006e870 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd665000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd66015e500 con 0x7fd6600cc840 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd66006e870 0x7fd6600cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd66006e870 0x7fd6600cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48674/0 (socket says 172.21.15.165:48674) 2024-04-25T10:03:36.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 -- 172.21.15.165:0/2168033066 learned_addr learned my addr 172.21.15.165:0/2168033066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:36.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd65fe00640 1 --2- 172.21.15.165:0/2168033066 >> v2:172.21.15.172:3300/0 conn(0x7fd6600cc840 0x7fd66015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 -- 172.21.15.165:0/2168033066 >> v2:172.21.15.172:3300/0 conn(0x7fd6600cc840 msgr2=0x7fd66015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 --2- 172.21.15.165:0/2168033066 >> v2:172.21.15.172:3300/0 conn(0x7fd6600cc840 0x7fd66015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 -- 172.21.15.165:0/2168033066 >> v1:172.21.15.172:6789/0 conn(0x7fd660143e60 legacy=0x7fd66006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.524+0000 7fd656a00640 1 -- 172.21.15.165:0/2168033066 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66015e7e0 con 0x7fd66006e870 2024-04-25T10:03:36.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.525+0000 7fd656a00640 1 --2- 172.21.15.165:0/2168033066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd66006e870 0x7fd6600cc300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd6600b2710 tx=0x7fd64802c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f040a25085e8dff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.525+0000 7fd65ea00640 1 -- 172.21.15.165:0/2168033066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6480325a0 con 0x7fd66006e870 2024-04-25T10:03:36.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.525+0000 7fd65ea00640 1 -- 172.21.15.165:0/2168033066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd648038070 con 0x7fd66006e870 2024-04-25T10:03:36.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.525+0000 7fd65ea00640 1 -- 172.21.15.165:0/2168033066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd648032960 con 0x7fd66006e870 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 -- 172.21.15.165:0/2168033066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd66006e870 msgr2=0x7fd6600cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 --2- 172.21.15.165:0/2168033066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd66006e870 0x7fd6600cc300 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd6600b2710 tx=0x7fd64802c8e0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 -- 172.21.15.165:0/2168033066 shutdown_connections 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 --2- 172.21.15.165:0/2168033066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd66006e870 0x7fd6600cc300 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 --2- 172.21.15.165:0/2168033066 >> v2:172.21.15.172:3300/0 conn(0x7fd6600cc840 0x7fd66015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 -- 172.21.15.165:0/2168033066 >> 172.21.15.165:0/2168033066 conn(0x7fd66006abe0 msgr2=0x7fd66006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 -- 172.21.15.165:0/2168033066 shutdown_connections 2024-04-25T10:03:36.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.526+0000 7fd665000640 1 -- 172.21.15.165:0/2168033066 wait complete. 2024-04-25T10:03:36.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.528+0000 7fd665000640 1 Processor -- start 2024-04-25T10:03:36.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.528+0000 7fd665000640 1 -- start start 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd665000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd66006e870 0x7fd660137ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd665000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6600cc840 0x7fd660139d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd665000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 0x7fd66013e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd665000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600b2eb0 con 0x7fd660143e60 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd665000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600b2d30 con 0x7fd66006e870 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd665000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6600ccca0 con 0x7fd6600cc840 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 0x7fd66013e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 0x7fd66013e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48686/0 (socket says 172.21.15.165:48686) 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 -- 172.21.15.165:0/2212629513 learned_addr learned my addr 172.21.15.165:0/2212629513 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:36.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 -- 172.21.15.165:0/2212629513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6600cc840 msgr2=0x7fd660139d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6600cc840 0x7fd660139d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 -- 172.21.15.165:0/2212629513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd66006e870 msgr2=0x7fd660137ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.529+0000 7fd65fe00640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd66006e870 0x7fd660137ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd65fe00640 1 -- 172.21.15.165:0/2212629513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66013c8f0 con 0x7fd660143e60 2024-04-25T10:03:36.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd65fe00640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 0x7fd66013e7e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd65800d830 tx=0x7fd65800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800b6c0 con 0x7fd660143e60 2024-04-25T10:03:36.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd658019070 con 0x7fd660143e60 2024-04-25T10:03:36.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd65800bac0 con 0x7fd660143e60 2024-04-25T10:03:36.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd66013cbd0 con 0x7fd660143e60 2024-04-25T10:03:36.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.530+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd66013d190 con 0x7fd660143e60 2024-04-25T10:03:36.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.531+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd660073990 con 0x7fd660143e60 2024-04-25T10:03:36.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.532+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd65801e460 con 0x7fd660143e60 2024-04-25T10:03:36.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.533+0000 7fd657e00640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd63c06b4d0 0x7fd63c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.533+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd65808a6d0 con 0x7fd660143e60 2024-04-25T10:03:36.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.534+0000 7fd65f400640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd63c06b4d0 0x7fd63c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.534+0000 7fd65f400640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd63c06b4d0 0x7fd63c06d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd65000abb0 tx=0x7fd650009250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.537+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd658059700 con 0x7fd660143e60 2024-04-25T10:03:36.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.718+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6600ce4d0 con 0x7fd660143e60 2024-04-25T10:03:36.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.719+0000 7fd657e00640 1 -- 172.21.15.165:0/2212629513 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+13200 (secure 0 0 0) 0x7fd658059000 con 0x7fd660143e60 2024-04-25T10:03:36.725 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:36.725 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":35,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:03:32.766036+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:36.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd63c06b4d0 msgr2=0x7fd63c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd63c06b4d0 0x7fd63c06d990 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd65000abb0 tx=0x7fd650009250 comp rx=0 tx=0).stop 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 msgr2=0x7fd66013e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 0x7fd66013e7e0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd65800d830 tx=0x7fd65800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 shutdown_connections 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd63c06b4d0 0x7fd63c06d990 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd660143e60 0x7fd66013e7e0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6600cc840 0x7fd660139d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 --2- 172.21.15.165:0/2212629513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd66006e870 0x7fd660137ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 >> 172.21.15.165:0/2212629513 conn(0x7fd66006abe0 msgr2=0x7fd6600ae710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 shutdown_connections 2024-04-25T10:03:36.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.722+0000 7fd665000640 1 -- 172.21.15.165:0/2212629513 wait complete. 2024-04-25T10:03:36.746 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:03:36.746 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:03:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.931+0000 7f47fee00640 1 Processor -- start 2024-04-25T10:03:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.931+0000 7f47fee00640 1 -- start start 2024-04-25T10:03:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.932+0000 7f47fee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80ced80 0x7f47f80d3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.932+0000 7f47fee00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80d23a0 con 0x7f47f80d1ce0 2024-04-25T10:03:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.932+0000 7f47fee00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f80d2680 con 0x7f47f815dc20 2024-04-25T10:03:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.932+0000 7f47fee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d2960 con 0x7f47f80ced80 2024-04-25T10:03:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.932+0000 7f47fce00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f47f80d1ce0 0x7f47f80b6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35028/0 (socket says 172.21.15.165:35028) 2024-04-25T10:03:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.932+0000 7f47fce00640 1 -- 172.21.15.165:0/3396962235 learned_addr learned my addr 172.21.15.165:0/3396962235 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f7400640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80ced80 0x7f47f80d3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 258600060 0 0) 0x7f47f80d23a0 con 0x7f47f80d1ce0 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47dc003520 con 0x7f47f80d1ce0 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3136127526 0 0) 0x7f47dc003520 con 0x7f47f80d1ce0 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80ced80 msgr2=0x7f47f80d3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80ced80 0x7f47f80d3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 >> v1:172.21.15.172:6789/0 conn(0x7f47f815dc20 legacy=0x7f47f80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f80d2c40 con 0x7f47f80d1ce0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f47f00019f0 con 0x7f47f80d1ce0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 >> v1:172.21.15.165:6789/0 conn(0x7f47f80d1ce0 legacy=0x7f47f80b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47dc004800 0x7f47dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47dc0052e0 0x7f47dc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d2960 con 0x7f47dc0039e0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc00c110 con 0x7f47dc004800 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47dc00c310 con 0x7f47dc0052e0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.933+0000 7f47f7400640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80ced80 0x7f47f80d3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7400640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47dc004800 0x7f47dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47fce00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47dc0052e0 0x7f47dc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7e00640 1 -- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47dc0052e0 msgr2=0x7f47dc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47dc0052e0 0x7f47dc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7e00640 1 -- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47dc004800 msgr2=0x7f47dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47dc004800 0x7f47dc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.934+0000 7f47f7e00640 1 -- 172.21.15.165:0/3396962235 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47dc003520 con 0x7f47dc0039e0 2024-04-25T10:03:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.935+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47dc003e00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f47e8001750 tx=0x7f47dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb55c659ab54696 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.935+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8001370 con 0x7f47dc0039e0 2024-04-25T10:03:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.935+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e80052d0 con 0x7f47dc0039e0 2024-04-25T10:03:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.935+0000 7f47f6a00640 1 -- 172.21.15.165:0/3396962235 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e802abe0 con 0x7f47dc0039e0 2024-04-25T10:03:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fee00640 1 -- 172.21.15.165:0/3396962235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 msgr2=0x7f47dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fee00640 1 --2- 172.21.15.165:0/3396962235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47dc003e00 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f47e8001750 tx=0x7f47dc005830 comp rx=0 tx=0).stop 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fce00640 1 -- 172.21.15.165:0/3396962235 reap_dead start 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fee00640 1 -- 172.21.15.165:0/3396962235 shutdown_connections 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fee00640 1 -- 172.21.15.165:0/3396962235 >> 172.21.15.165:0/3396962235 conn(0x7f47f8061a20 msgr2=0x7f47f8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fee00640 1 -- 172.21.15.165:0/3396962235 shutdown_connections 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.936+0000 7f47fee00640 1 -- 172.21.15.165:0/3396962235 wait complete. 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.938+0000 7f47fee00640 1 Processor -- start 2024-04-25T10:03:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.938+0000 7f47fee00640 1 -- start start 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47fee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47f80cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47fee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80caa70 0x7f47f80cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47fee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f80d3950 0x7f47f80d3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47fee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d35c0 con 0x7f47dc0039e0 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47fee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d3440 con 0x7f47f80d3950 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47fee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f80d3740 con 0x7f47f80caa70 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47f80cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47f80cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39354/0 (socket says 172.21.15.165:39354) 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47f7e00640 1 -- 172.21.15.165:0/3876309588 learned_addr learned my addr 172.21.15.165:0/3876309588 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47f7e00640 1 -- 172.21.15.165:0/3876309588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80caa70 msgr2=0x7f47f80cbbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.939+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80caa70 0x7f47f80cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47f7e00640 1 -- 172.21.15.165:0/3876309588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f80d3950 msgr2=0x7f47f80d3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f80d3950 0x7f47f80d3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47f7e00640 1 -- 172.21.15.165:0/3876309588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f81f7110 con 0x7f47dc0039e0 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47f7e00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47f80cd590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47e8001870 tx=0x7f47e8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8042070 con 0x7f47dc0039e0 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e802be50 con 0x7f47dc0039e0 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f80cc110 con 0x7f47dc0039e0 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8037040 con 0x7f47dc0039e0 2024-04-25T10:03:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.940+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f81f7380 con 0x7f47dc0039e0 2024-04-25T10:03:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.941+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f8073a00 con 0x7f47dc0039e0 2024-04-25T10:03:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.942+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47e8045070 con 0x7f47dc0039e0 2024-04-25T10:03:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.943+0000 7f47efe00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d406b4d0 0x7f47d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.943+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f47e80a9880 con 0x7f47dc0039e0 2024-04-25T10:03:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.943+0000 7f47f7400640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d406b4d0 0x7f47d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.944+0000 7f47f7400640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d406b4d0 0x7f47d406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f47e0006e80 tx=0x7f47e0006e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:36.948+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47e80788c0 con 0x7f47dc0039e0 2024-04-25T10:03:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.141+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47f815bd30 con 0x7f47dc0039e0 2024-04-25T10:03:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.142+0000 7f47efe00640 1 -- 172.21.15.165:0/3876309588 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v35) v1 ==== 93+0+10 (secure 0 0 0) 0x7f47e80781c0 con 0x7f47dc0039e0 2024-04-25T10:03:37.148 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.144+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d406b4d0 msgr2=0x7f47d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.144+0000 7f47fee00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d406b4d0 0x7f47d406d990 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f47e0006e80 tx=0x7f47e0006e10 comp rx=0 tx=0).stop 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 msgr2=0x7f47f80cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47f80cd590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f47e8001870 tx=0x7f47e8001120 comp rx=0 tx=0).stop 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 shutdown_connections 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d406b4d0 0x7f47d406d990 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f80d3950 0x7f47f80d3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47f80caa70 0x7f47f80cbbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 --2- 172.21.15.165:0/3876309588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47dc0039e0 0x7f47f80cd590 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 >> 172.21.15.165:0/3876309588 conn(0x7f47f8061a20 msgr2=0x7f47f80b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 shutdown_connections 2024-04-25T10:03:37.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.145+0000 7f47fee00640 1 -- 172.21.15.165:0/3876309588 wait complete. 2024-04-25T10:03:37.170 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:03:37.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.366+0000 7fb31fa00640 1 Processor -- start 2024-04-25T10:03:37.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.366+0000 7fb31fa00640 1 -- start start 2024-04-25T10:03:37.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31fa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb3180cc930 0x7fb31806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31fa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb3180ce950 0x7fb3180cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31fa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31815c150 con 0x7fb318144d40 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31fa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb31806d850 con 0x7fb3180cc930 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31fa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3180cd4b0 con 0x7fb3180ce950 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31d000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb318144d40 0x7fb31815c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35038/0 (socket says 172.21.15.165:35038) 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31d000640 1 -- 172.21.15.165:0/2530085615 learned_addr learned my addr 172.21.15.165:0/2530085615 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317e00640 1 --2- 172.21.15.165:0/2530085615 >> v2:172.21.15.165:3300/0 conn(0x7fb3180cc930 0x7fb31806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb31da00640 1 --2- 172.21.15.165:0/2530085615 >> v2:172.21.15.172:3300/0 conn(0x7fb3180ce950 0x7fb3180cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317400640 1 -- 172.21.15.165:0/2530085615 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124399640 0 0) 0x7fb31815c150 con 0x7fb318144d40 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317400640 1 -- 172.21.15.165:0/2530085615 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb300003520 con 0x7fb318144d40 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317e00640 1 -- 172.21.15.165:0/2530085615 >> v2:172.21.15.172:3300/0 conn(0x7fb3180ce950 msgr2=0x7fb3180cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317e00640 1 --2- 172.21.15.165:0/2530085615 >> v2:172.21.15.172:3300/0 conn(0x7fb3180ce950 0x7fb3180cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317e00640 1 -- 172.21.15.165:0/2530085615 >> v1:172.21.15.165:6789/0 conn(0x7fb318144d40 legacy=0x7fb31815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.368+0000 7fb317e00640 1 -- 172.21.15.165:0/2530085615 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3180ccef0 con 0x7fb3180cc930 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.369+0000 7fb317e00640 1 --2- 172.21.15.165:0/2530085615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb31806d310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb3180ccdf0 tx=0x7fb30402a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e305ee57a8d4ca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.369+0000 7fb317400640 1 -- 172.21.15.165:0/2530085615 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304037070 con 0x7fb3180cc930 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.369+0000 7fb317400640 1 -- 172.21.15.165:0/2530085615 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30402add0 con 0x7fb3180cc930 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.369+0000 7fb317400640 1 -- 172.21.15.165:0/2530085615 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb304032660 con 0x7fb3180cc930 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 -- 172.21.15.165:0/2530085615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 msgr2=0x7fb31806d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2530085615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb31806d310 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fb3180ccdf0 tx=0x7fb30402a080 comp rx=0 tx=0).stop 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 -- 172.21.15.165:0/2530085615 shutdown_connections 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2530085615 >> v2:172.21.15.172:3300/0 conn(0x7fb3180ce950 0x7fb3180cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2530085615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb31806d310 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 -- 172.21.15.165:0/2530085615 >> 172.21.15.165:0/2530085615 conn(0x7fb31806abe0 msgr2=0x7fb31806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 -- 172.21.15.165:0/2530085615 shutdown_connections 2024-04-25T10:03:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.370+0000 7fb31fa00640 1 -- 172.21.15.165:0/2530085615 wait complete. 2024-04-25T10:03:37.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.372+0000 7fb31fa00640 1 Processor -- start 2024-04-25T10:03:37.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.372+0000 7fb31fa00640 1 -- start start 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31fa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb318150fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31fa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3180ce950 0x7fb3180ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31fa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb318144d40 0x7fb31814db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31fa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3180cf2f0 con 0x7fb3180cc930 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31fa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31815c3f0 con 0x7fb318144d40 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31fa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31806da60 con 0x7fb3180ce950 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb318150fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb318150fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39380/0 (socket says 172.21.15.165:39380) 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 -- 172.21.15.165:0/2129090998 learned_addr learned my addr 172.21.15.165:0/2129090998 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 -- 172.21.15.165:0/2129090998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3180ce950 msgr2=0x7fb3180ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3180ce950 0x7fb3180ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 -- 172.21.15.165:0/2129090998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb318144d40 msgr2=0x7fb31814db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb318144d40 0x7fb31814db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.374+0000 7fb31d000640 1 -- 172.21.15.165:0/2129090998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31813d1e0 con 0x7fb3180cc930 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.375+0000 7fb31d000640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb318150fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c3c0 tx=0x7fb30c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.375+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c017070 con 0x7fb3180cc930 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.375+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c005ce0 con 0x7fb3180cc930 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.375+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0072f0 con 0x7fb3180cc930 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.375+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31813cc20 con 0x7fb3180cc930 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.375+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31813e8e0 con 0x7fb3180cc930 2024-04-25T10:03:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.377+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb318073a00 con 0x7fb3180cc930 2024-04-25T10:03:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.378+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb30c01a070 con 0x7fb3180cc930 2024-04-25T10:03:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.378+0000 7fb314c00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.379+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb30c089340 con 0x7fb3180cc930 2024-04-25T10:03:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.379+0000 7fb317e00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.380+0000 7fb317e00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb304034000 tx=0x7fb30402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.383+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb31813d1e0 con 0x7fb3180cc930 2024-04-25T10:03:37.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.582+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb3180aeb10 con 0x7fb3180cc930 2024-04-25T10:03:37.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.583+0000 7fb314c00640 1 -- 172.21.15.165:0/2129090998 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v35) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb30c058410 con 0x7fb3180cc930 2024-04-25T10:03:37.589 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:37.589 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:03:37.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:03:37.589 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:03:37.590 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:03:37.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:03:37.590 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.585+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 msgr2=0x7fb2f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.585+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb304034000 tx=0x7fb30402c000 comp rx=0 tx=0).stop 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.585+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 msgr2=0x7fb318150fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.585+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb318150fc0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c3c0 tx=0x7fb30c00c880 comp rx=0 tx=0).stop 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 shutdown_connections 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb318144d40 0x7fb31814db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3180ce950 0x7fb3180ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 --2- 172.21.15.165:0/2129090998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3180cc930 0x7fb318150fc0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 >> 172.21.15.165:0/2129090998 conn(0x7fb31806abe0 msgr2=0x7fb31815e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 shutdown_connections 2024-04-25T10:03:37.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.586+0000 7fb31fa00640 1 -- 172.21.15.165:0/2129090998 wait complete. 2024-04-25T10:03:37.611 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:03:37.612 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:03:37.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.821+0000 7fe450000640 1 Processor -- start 2024-04-25T10:03:37.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.821+0000 7fe450000640 1 -- start start 2024-04-25T10:03:37.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.822+0000 7fe450000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe448161240 0x7fe4480b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.822+0000 7fe450000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.822+0000 7fe450000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe448161620 con 0x7fe4480b54a0 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.822+0000 7fe450000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4480b7120 con 0x7fe448161240 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.822+0000 7fe450000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe448161e00 con 0x7fe448161890 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe448161240 0x7fe4480b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe448161240 0x7fe4480b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39388/0 (socket says 172.21.15.165:39388) 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 -- 172.21.15.165:0/2050777252 learned_addr learned my addr 172.21.15.165:0/2050777252 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44cc00640 1 --2- 172.21.15.165:0/2050777252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 -- 172.21.15.165:0/2050777252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 msgr2=0x7fe4480ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 --2- 172.21.15.165:0/2050777252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 -- 172.21.15.165:0/2050777252 >> v1:172.21.15.172:6789/0 conn(0x7fe4480b54a0 legacy=0x7fe4480d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.823+0000 7fe44d600640 1 -- 172.21.15.165:0/2050777252 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4481620e0 con 0x7fe448161240 2024-04-25T10:03:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.825+0000 7fe44d600640 1 --2- 172.21.15.165:0/2050777252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe448161240 0x7fe4480b6be0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009870 tx=0x7fe43c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdda66dc23c867c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.825+0000 7fe447e00640 1 -- 172.21.15.165:0/2050777252 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c037070 con 0x7fe448161240 2024-04-25T10:03:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.825+0000 7fe447e00640 1 -- 172.21.15.165:0/2050777252 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c032440 con 0x7fe448161240 2024-04-25T10:03:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.825+0000 7fe447e00640 1 -- 172.21.15.165:0/2050777252 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c032810 con 0x7fe448161240 2024-04-25T10:03:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 -- 172.21.15.165:0/2050777252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe448161240 msgr2=0x7fe4480b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 --2- 172.21.15.165:0/2050777252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe448161240 0x7fe4480b6be0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009870 tx=0x7fe43c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 -- 172.21.15.165:0/2050777252 shutdown_connections 2024-04-25T10:03:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 --2- 172.21.15.165:0/2050777252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 --2- 172.21.15.165:0/2050777252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe448161240 0x7fe4480b6be0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 -- 172.21.15.165:0/2050777252 >> 172.21.15.165:0/2050777252 conn(0x7fe4480d04a0 msgr2=0x7fe4480d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 -- 172.21.15.165:0/2050777252 shutdown_connections 2024-04-25T10:03:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.826+0000 7fe450000640 1 -- 172.21.15.165:0/2050777252 wait complete. 2024-04-25T10:03:37.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.828+0000 7fe450000640 1 Processor -- start 2024-04-25T10:03:37.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.828+0000 7fe450000640 1 -- start start 2024-04-25T10:03:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe450000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 0x7fe4480cd410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe450000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe448161240 0x7fe448159010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe450000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe450000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480d15f0 con 0x7fe4480b54a0 2024-04-25T10:03:37.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe450000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4481626a0 con 0x7fe448161240 2024-04-25T10:03:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe450000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4480cf000 con 0x7fe448161890 2024-04-25T10:03:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 0x7fe4480cd410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 0x7fe4480cd410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39392/0 (socket says 172.21.15.165:39392) 2024-04-25T10:03:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44d600640 1 -- 172.21.15.165:0/2010144137 learned_addr learned my addr 172.21.15.165:0/2010144137 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44e000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44d600640 1 -- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 msgr2=0x7fe4480c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44d600640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.829+0000 7fe44d600640 1 -- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe448161240 msgr2=0x7fe448159010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe44d600640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe448161240 0x7fe448159010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe44d600640 1 -- 172.21.15.165:0/2010144137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe448160670 con 0x7fe4480b54a0 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe44d600640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 0x7fe4480cd410 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009870 tx=0x7fe43c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c041070 con 0x7fe4480b54a0 2024-04-25T10:03:37.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c03c890 con 0x7fe4480b54a0 2024-04-25T10:03:37.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c037040 con 0x7fe4480b54a0 2024-04-25T10:03:37.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4481f8410 con 0x7fe4480b54a0 2024-04-25T10:03:37.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4481f89d0 con 0x7fe4480b54a0 2024-04-25T10:03:37.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.832+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe448073a00 con 0x7fe4480b54a0 2024-04-25T10:03:37.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.833+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe43c044070 con 0x7fe4480b54a0 2024-04-25T10:03:37.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.833+0000 7fe445600640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe42806b4d0 0x7fe42806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:37.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.834+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe43c0aaac0 con 0x7fe4480b54a0 2024-04-25T10:03:37.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.834+0000 7fe44cc00640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe42806b4d0 0x7fe42806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:37.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.834+0000 7fe44cc00640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe42806b4d0 0x7fe42806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe434006fd0 tx=0x7fe434008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:37.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:37.838+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe448160670 con 0x7fe4480b54a0 2024-04-25T10:03:38.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.035+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe4480c67c0 con 0x7fe4480b54a0 2024-04-25T10:03:38.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.826+0000 7fe445600640 1 -- 172.21.15.165:0/2010144137 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v36) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe43c079c20 con 0x7fe4480b54a0 2024-04-25T10:03:38.831 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe42806b4d0 msgr2=0x7fe42806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe42806b4d0 0x7fe42806d990 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fe434006fd0 tx=0x7fe434008040 comp rx=0 tx=0).stop 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 msgr2=0x7fe4480cd410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 0x7fe4480cd410 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe43c009870 tx=0x7fe43c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 shutdown_connections 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe42806b4d0 0x7fe42806d990 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe448161890 0x7fe4480c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe448161240 0x7fe448159010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 --2- 172.21.15.165:0/2010144137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4480b54a0 0x7fe4480cd410 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 >> 172.21.15.165:0/2010144137 conn(0x7fe4480d04a0 msgr2=0x7fe4480b3ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:38.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 shutdown_connections 2024-04-25T10:03:38.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:38.830+0000 7fe450000640 1 -- 172.21.15.165:0/2010144137 wait complete. 2024-04-25T10:03:38.862 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:03:39.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.053+0000 7fbf71e00640 1 Processor -- start 2024-04-25T10:03:39.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.053+0000 7fbf71e00640 1 -- start start 2024-04-25T10:03:39.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf71e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbf6c1446d0 0x7fbf6c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf71e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbf6c06e8a0 0x7fbf6c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf71e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf6c0cccc0 con 0x7fbf6c144d50 2024-04-25T10:03:39.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf71e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c0b2b20 con 0x7fbf6c06e8a0 2024-04-25T10:03:39.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf71e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf6c06d400 con 0x7fbf6c1446d0 2024-04-25T10:03:39.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf6b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbf6c06e8a0 0x7fbf6c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf6b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbf6c06e8a0 0x7fbf6c0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39404/0 (socket says 172.21.15.165:39404) 2024-04-25T10:03:39.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf6b400640 1 -- 172.21.15.165:0/2811514417 learned_addr learned my addr 172.21.15.165:0/2811514417 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.055+0000 7fbf6aa00640 1 --2- 172.21.15.165:0/2811514417 >> v2:172.21.15.172:3300/0 conn(0x7fbf6c1446d0 0x7fbf6c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.056+0000 7fbf6b400640 1 -- 172.21.15.165:0/2811514417 >> v2:172.21.15.172:3300/0 conn(0x7fbf6c1446d0 msgr2=0x7fbf6c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.056+0000 7fbf6b400640 1 --2- 172.21.15.165:0/2811514417 >> v2:172.21.15.172:3300/0 conn(0x7fbf6c1446d0 0x7fbf6c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.056+0000 7fbf6b400640 1 -- 172.21.15.165:0/2811514417 >> v1:172.21.15.172:6789/0 conn(0x7fbf6c144d50 legacy=0x7fbf6c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.056+0000 7fbf6b400640 1 -- 172.21.15.165:0/2811514417 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c0cc140 con 0x7fbf6c06e8a0 2024-04-25T10:03:39.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.057+0000 7fbf6b400640 1 --2- 172.21.15.165:0/2811514417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c06e8a0 0x7fbf6c0b4460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0cf560 tx=0x7fbf6402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a99b543b83249b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf69600640 1 -- 172.21.15.165:0/2811514417 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf64036070 con 0x7fbf6c06e8a0 2024-04-25T10:03:39.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf69600640 1 -- 172.21.15.165:0/2811514417 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf64031600 con 0x7fbf6c06e8a0 2024-04-25T10:03:39.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf69600640 1 -- 172.21.15.165:0/2811514417 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf640319d0 con 0x7fbf6c06e8a0 2024-04-25T10:03:39.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf71e00640 1 -- 172.21.15.165:0/2811514417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c06e8a0 msgr2=0x7fbf6c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2811514417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c06e8a0 0x7fbf6c0b4460 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbf6c0cf560 tx=0x7fbf6402a730 comp rx=0 tx=0).stop 2024-04-25T10:03:39.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf71e00640 1 -- 172.21.15.165:0/2811514417 shutdown_connections 2024-04-25T10:03:39.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2811514417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c06e8a0 0x7fbf6c0b4460 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2811514417 >> v2:172.21.15.172:3300/0 conn(0x7fbf6c1446d0 0x7fbf6c0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.058+0000 7fbf71e00640 1 -- 172.21.15.165:0/2811514417 >> 172.21.15.165:0/2811514417 conn(0x7fbf6c06abe0 msgr2=0x7fbf6c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.059+0000 7fbf71e00640 1 -- 172.21.15.165:0/2811514417 shutdown_connections 2024-04-25T10:03:39.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.059+0000 7fbf71e00640 1 -- 172.21.15.165:0/2811514417 wait complete. 2024-04-25T10:03:39.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.060+0000 7fbf71e00640 1 Processor -- start 2024-04-25T10:03:39.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.060+0000 7fbf71e00640 1 -- start start 2024-04-25T10:03:39.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.061+0000 7fbf71e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf6c06e8a0 0x7fbf6c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.061+0000 7fbf71e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 0x7fbf6c1526a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.061+0000 7fbf71e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf6c144d50 0x7fbf6c0a9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.061+0000 7fbf71e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0cc700 con 0x7fbf6c1446d0 2024-04-25T10:03:39.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.061+0000 7fbf71e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0b49a0 con 0x7fbf6c144d50 2024-04-25T10:03:39.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.061+0000 7fbf71e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6c0cc9e0 con 0x7fbf6c06e8a0 2024-04-25T10:03:39.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 0x7fbf6c1526a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 0x7fbf6c1526a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39408/0 (socket says 172.21.15.165:39408) 2024-04-25T10:03:39.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 -- 172.21.15.165:0/2866442915 learned_addr learned my addr 172.21.15.165:0/2866442915 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:39.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6b400640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf6c144d50 0x7fbf6c0a9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6aa00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf6c06e8a0 0x7fbf6c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 -- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf6c06e8a0 msgr2=0x7fbf6c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf6c06e8a0 0x7fbf6c15b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 -- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf6c144d50 msgr2=0x7fbf6c0a9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf6c144d50 0x7fbf6c0a9bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6a000640 1 -- 172.21.15.165:0/2866442915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf6c0aa130 con 0x7fbf6c1446d0 2024-04-25T10:03:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.062+0000 7fbf6aa00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf6c06e8a0 0x7fbf6c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.063+0000 7fbf6a000640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 0x7fbf6c1526a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c3c0 tx=0x7fbf5800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.063+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580052d0 con 0x7fbf6c1446d0 2024-04-25T10:03:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.063+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58005490 con 0x7fbf6c1446d0 2024-04-25T10:03:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.063+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58007260 con 0x7fbf6c1446d0 2024-04-25T10:03:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.063+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf6c0aa3b0 con 0x7fbf6c1446d0 2024-04-25T10:03:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.063+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf6c0aa970 con 0x7fbf6c1446d0 2024-04-25T10:03:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.064+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6c073a00 con 0x7fbf6c1446d0 2024-04-25T10:03:39.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.066+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbf58007400 con 0x7fbf6c1446d0 2024-04-25T10:03:39.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.066+0000 7fbf63400640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.067+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fbf580896f0 con 0x7fbf6c1446d0 2024-04-25T10:03:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.067+0000 7fbf6aa00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.068+0000 7fbf6aa00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c006fd0 tx=0x7fbf5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.072+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbf6c0aa130 con 0x7fbf6c1446d0 2024-04-25T10:03:39.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.266+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbf6c0b0600 con 0x7fbf6c1446d0 2024-04-25T10:03:39.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.833+0000 7fbf63400640 1 -- 172.21.15.165:0/2866442915 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v37) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbf58058520 con 0x7fbf6c1446d0 2024-04-25T10:03:39.838 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:03:39.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbf4c06b4d0 msgr2=0x7fbf4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fbf5c006fd0 tx=0x7fbf5c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:39.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 msgr2=0x7fbf6c1526a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 0x7fbf6c1526a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fbf5800c3c0 tx=0x7fbf5800c880 comp rx=0 tx=0).stop 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 shutdown_connections 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbf4c06b4d0 0x7fbf4c06d990 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbf6c144d50 0x7fbf6c0a9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbf6c1446d0 0x7fbf6c1526a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 --2- 172.21.15.165:0/2866442915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbf6c06e8a0 0x7fbf6c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.838+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 >> 172.21.15.165:0/2866442915 conn(0x7fbf6c06abe0 msgr2=0x7fbf6c071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.839+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 shutdown_connections 2024-04-25T10:03:39.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:39.839+0000 7fbf71e00640 1 -- 172.21.15.165:0/2866442915 wait complete. 2024-04-25T10:03:39.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:03:40.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.045+0000 7f9fba600640 1 Processor -- start 2024-04-25T10:03:40.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.045+0000 7f9fba600640 1 -- start start 2024-04-25T10:03:40.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.047+0000 7f9fba600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9fb415e720 0x7f9fb40b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.047+0000 7f9fba600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40b0b80 con 0x7f9fb415b7a0 2024-04-25T10:03:40.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.047+0000 7f9fba600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fb40cc800 con 0x7f9fb415b3c0 2024-04-25T10:03:40.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.047+0000 7f9fba600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fb40cd650 con 0x7f9fb415e720 2024-04-25T10:03:40.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.047+0000 7f9fabe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9fb415b7a0 0x7f9fb415e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35044/0 (socket says 172.21.15.165:35044) 2024-04-25T10:03:40.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.047+0000 7f9fabe00640 1 -- 172.21.15.165:0/593765855 learned_addr learned my addr 172.21.15.165:0/593765855 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb9c00640 1 --2- 172.21.15.165:0/593765855 >> v2:172.21.15.172:3300/0 conn(0x7f9fb415e720 0x7f9fb40b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 655902608 0 0) 0x7f9f9c001110 con 0x7f9fb415b7a0 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f98003520 con 0x7f9fb415b7a0 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3582357968 0 0) 0x7f9f98003520 con 0x7f9fb415b7a0 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 >> v2:172.21.15.172:3300/0 conn(0x7f9fb415e720 msgr2=0x7f9fb40b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 --2- 172.21.15.165:0/593765855 >> v2:172.21.15.172:3300/0 conn(0x7f9fb415e720 0x7f9fb40b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 >> v1:172.21.15.172:6789/0 conn(0x7f9fb415b3c0 legacy=0x7f9fb40cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb40ccae0 con 0x7f9fb415b7a0 2024-04-25T10:03:40.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9f9c001840 con 0x7f9fb415b7a0 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb9c00640 1 --2- 172.21.15.165:0/593765855 >> v2:172.21.15.172:3300/0 conn(0x7f9fb415e720 0x7f9fb40b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 >> v1:172.21.15.165:6789/0 conn(0x7f9fb415b7a0 legacy=0x7f9fb415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f980039b0 0x7f9f98003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f98004800 0x7f9f98004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f980052e0 0x7f9f9800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40cd650 con 0x7f9f98004800 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9800c1b0 con 0x7f9f980039b0 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.048+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9800c3b0 con 0x7f9f980052e0 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fabe00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f98004800 0x7f9f98004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fabe00640 1 -- 172.21.15.165:0/593765855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f980052e0 msgr2=0x7f9f9800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fabe00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f980052e0 0x7f9f9800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fabe00640 1 -- 172.21.15.165:0/593765855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f980039b0 msgr2=0x7f9f98003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fabe00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f980039b0 0x7f9f98003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fabe00640 1 -- 172.21.15.165:0/593765855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f98003520 con 0x7f9f98004800 2024-04-25T10:03:40.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.049+0000 7f9fb9c00640 1 -- 172.21.15.165:0/593765855 reap_dead start 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.050+0000 7f9fabe00640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f98004800 0x7f9f98004c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9fb415bc00 tx=0x7f9f98005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a96b4e80e23c275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.050+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00c0e0 con 0x7f9f98004800 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.050+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c034070 con 0x7f9f98004800 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.050+0000 7f9fb3e00640 1 -- 172.21.15.165:0/593765855 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c005d70 con 0x7f9f98004800 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 -- 172.21.15.165:0/593765855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f98004800 msgr2=0x7f9f98004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f98004800 0x7f9f98004c80 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9fb415bc00 tx=0x7f9f98005830 comp rx=0 tx=0).stop 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 -- 172.21.15.165:0/593765855 shutdown_connections 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 --2- 172.21.15.165:0/593765855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f98004800 0x7f9f98004c80 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 -- 172.21.15.165:0/593765855 >> 172.21.15.165:0/593765855 conn(0x7f9fb4061cc0 msgr2=0x7f9fb406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 -- 172.21.15.165:0/593765855 shutdown_connections 2024-04-25T10:03:40.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.051+0000 7f9fba600640 1 -- 172.21.15.165:0/593765855 wait complete. 2024-04-25T10:03:40.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.053+0000 7f9fba600640 1 Processor -- start 2024-04-25T10:03:40.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.054+0000 7f9fba600640 1 -- start start 2024-04-25T10:03:40.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fba600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f98004800 0x7f9fb4146870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fba600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 0x7f9fb40ca830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fba600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fb41f2ed0 0x7f9fb40b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fba600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40b10e0 con 0x7f9fb4155ba0 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fba600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40b0f60 con 0x7f9f98004800 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fba600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fb40b1260 con 0x7f9fb41f2ed0 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fabe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 0x7f9fb40ca830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fabe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 0x7f9fb40ca830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39426/0 (socket says 172.21.15.165:39426) 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fabe00640 1 -- 172.21.15.165:0/1330647795 learned_addr learned my addr 172.21.15.165:0/1330647795 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fb9200640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f98004800 0x7f9fb4146870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.055+0000 7f9fb9c00640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fb41f2ed0 0x7f9fb40b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fabe00640 1 -- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fb41f2ed0 msgr2=0x7f9fb40b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fabe00640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fb41f2ed0 0x7f9fb40b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fabe00640 1 -- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f98004800 msgr2=0x7f9fb4146870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fabe00640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f98004800 0x7f9fb4146870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fabe00640 1 -- 172.21.15.165:0/1330647795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb4156080 con 0x7f9fb4155ba0 2024-04-25T10:03:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fabe00640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 0x7f9fb40ca830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c031040 tx=0x7f9fb40cccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c007270 con 0x7f9fb4155ba0 2024-04-25T10:03:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9c03f030 con 0x7f9fb4155ba0 2024-04-25T10:03:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c007610 con 0x7f9fb4155ba0 2024-04-25T10:03:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fb4156360 con 0x7f9fb4155ba0 2024-04-25T10:03:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.056+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fb4156920 con 0x7f9fb4155ba0 2024-04-25T10:03:40.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.058+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fb4073990 con 0x7f9fb4155ba0 2024-04-25T10:03:40.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.059+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f9c038450 con 0x7f9fb4155ba0 2024-04-25T10:03:40.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.060+0000 7f9fb1600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f9006b4d0 0x7f9f9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:40.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.060+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f9f9c0ab470 con 0x7f9fb4155ba0 2024-04-25T10:03:40.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.060+0000 7f9fb9200640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f9006b4d0 0x7f9f9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:40.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.061+0000 7f9fb9200640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f9006b4d0 0x7f9f9006d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4007fd0 tx=0x7f9fa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:40.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.063+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f9c049030 con 0x7f9fb4155ba0 2024-04-25T10:03:40.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.293+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9fb4072360 con 0x7f9fb4155ba0 2024-04-25T10:03:40.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.879+0000 7f9fb1600640 1 -- 172.21.15.165:0/1330647795 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v15) v1 ==== 172+0+0 (secure 0 0 0) 0x7f9fb40b0b80 con 0x7f9fb4155ba0 2024-04-25T10:03:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 5 and data pool 6 2024-04-25T10:03:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.888+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f9006b4d0 msgr2=0x7f9f9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.888+0000 7f9fba600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f9006b4d0 0x7f9f9006d990 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4007fd0 tx=0x7f9fa4009040 comp rx=0 tx=0).stop 2024-04-25T10:03:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.888+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 msgr2=0x7f9fb40ca830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.888+0000 7f9fba600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 0x7f9fb40ca830 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c031040 tx=0x7f9fb40cccb0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 shutdown_connections 2024-04-25T10:03:40.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f9006b4d0 0x7f9f9006d990 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9fb41f2ed0 0x7f9fb40b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9fb4155ba0 0x7f9fb40ca830 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 --2- 172.21.15.165:0/1330647795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f98004800 0x7f9fb4146870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:40.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 >> 172.21.15.165:0/1330647795 conn(0x7f9fb4061cc0 msgr2=0x7f9fb415bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.890+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 shutdown_connections 2024-04-25T10:03:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:40.891+0000 7f9fba600640 1 -- 172.21.15.165:0/1330647795 wait complete. 2024-04-25T10:03:40.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:41.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.092+0000 7f3a57200640 1 Processor -- start 2024-04-25T10:03:41.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.092+0000 7f3a57200640 1 -- start start 2024-04-25T10:03:41.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a57200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3a580d06c0 0x7f3a58168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a57200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a580d33f0 con 0x7f3a5815db90 2024-04-25T10:03:41.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a57200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a58161df0 con 0x7f3a580b93c0 2024-04-25T10:03:41.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a57200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a580d1f50 con 0x7f3a580d06c0 2024-04-25T10:03:41.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a55e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3a5815db90 0x7f3a581615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35052/0 (socket says 172.21.15.165:35052) 2024-04-25T10:03:41.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a55e00640 1 -- 172.21.15.165:0/1239612976 learned_addr learned my addr 172.21.15.165:0/1239612976 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:41.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a56800640 1 --2- 172.21.15.165:0/1239612976 >> v2:172.21.15.165:3300/0 conn(0x7f3a580d06c0 0x7f3a58168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a54a00640 1 -- 172.21.15.165:0/1239612976 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1592985589 0 0) 0x7f3a580d33f0 con 0x7f3a5815db90 2024-04-25T10:03:41.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a54a00640 1 -- 172.21.15.165:0/1239612976 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a3c003520 con 0x7f3a5815db90 2024-04-25T10:03:41.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a56800640 1 -- 172.21.15.165:0/1239612976 >> v1:172.21.15.172:6789/0 conn(0x7f3a580b93c0 legacy=0x7f3a580cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a56800640 1 -- 172.21.15.165:0/1239612976 >> v1:172.21.15.165:6789/0 conn(0x7f3a5815db90 legacy=0x7f3a581615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.094+0000 7f3a56800640 1 -- 172.21.15.165:0/1239612976 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a581620d0 con 0x7f3a580d06c0 2024-04-25T10:03:41.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.095+0000 7f3a56800640 1 --2- 172.21.15.165:0/1239612976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a580d06c0 0x7f3a58168930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097a0 tx=0x7f3a5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87fb55bceb88951 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a54a00640 1 -- 172.21.15.165:0/1239612976 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50037070 con 0x7f3a580d06c0 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a54a00640 1 -- 172.21.15.165:0/1239612976 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a50032440 con 0x7f3a580d06c0 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a54a00640 1 -- 172.21.15.165:0/1239612976 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50032810 con 0x7f3a580d06c0 2024-04-25T10:03:41.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a57200640 1 -- 172.21.15.165:0/1239612976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a580d06c0 msgr2=0x7f3a58168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a57200640 1 --2- 172.21.15.165:0/1239612976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a580d06c0 0x7f3a58168930 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097a0 tx=0x7f3a5002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a57200640 1 -- 172.21.15.165:0/1239612976 shutdown_connections 2024-04-25T10:03:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a57200640 1 --2- 172.21.15.165:0/1239612976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a580d06c0 0x7f3a58168930 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a57200640 1 -- 172.21.15.165:0/1239612976 >> 172.21.15.165:0/1239612976 conn(0x7f3a580b52b0 msgr2=0x7f3a580b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.096+0000 7f3a57200640 1 -- 172.21.15.165:0/1239612976 shutdown_connections 2024-04-25T10:03:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.097+0000 7f3a57200640 1 -- 172.21.15.165:0/1239612976 wait complete. 2024-04-25T10:03:41.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.098+0000 7f3a57200640 1 Processor -- start 2024-04-25T10:03:41.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.098+0000 7f3a57200640 1 -- start start 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a57200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a580b93c0 0x7f3a5814e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a57200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a580d06c0 0x7f3a5815e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a57200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 0x7f3a58151b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a57200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580b68d0 con 0x7f3a5815db90 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a57200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580d1be0 con 0x7f3a580b93c0 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a57200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a580b6a50 con 0x7f3a580d06c0 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 0x7f3a58151b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 0x7f3a58151b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39442/0 (socket says 172.21.15.165:39442) 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.099+0000 7f3a56800640 1 -- 172.21.15.165:0/750754667 learned_addr learned my addr 172.21.15.165:0/750754667 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a55e00640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a580b93c0 0x7f3a5814e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a56800640 1 -- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a580d06c0 msgr2=0x7f3a5815e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a55400640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a580d06c0 0x7f3a5815e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a56800640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a580d06c0 0x7f3a5815e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a56800640 1 -- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a580b93c0 msgr2=0x7f3a5814e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a56800640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a580b93c0 0x7f3a5814e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a56800640 1 -- 172.21.15.165:0/750754667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a580cabe0 con 0x7f3a5815db90 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a56800640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 0x7f3a58151b40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097a0 tx=0x7f3a500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50041020 con 0x7f3a5815db90 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a5003d620 con 0x7f3a5815db90 2024-04-25T10:03:41.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a50037040 con 0x7f3a5815db90 2024-04-25T10:03:41.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.100+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a580caec0 con 0x7f3a5815db90 2024-04-25T10:03:41.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.101+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a580cb480 con 0x7f3a5815db90 2024-04-25T10:03:41.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.102+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a58073990 con 0x7f3a5815db90 2024-04-25T10:03:41.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.103+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a50044020 con 0x7f3a5815db90 2024-04-25T10:03:41.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.104+0000 7f3a4e000640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a3406b4d0 0x7f3a3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.104+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3a500aab80 con 0x7f3a5815db90 2024-04-25T10:03:41.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.104+0000 7f3a55e00640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a3406b4d0 0x7f3a3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.105+0000 7f3a55e00640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a3406b4d0 0x7f3a3406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000f60 tx=0x7f3a40009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.108+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a5004b030 con 0x7f3a5815db90 2024-04-25T10:03:41.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.294+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a580d14b0 con 0x7f3a5815db90 2024-04-25T10:03:41.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.297+0000 7f3a4e000640 1 -- 172.21.15.165:0/750754667 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f3a500797e0 con 0x7f3a5815db90 2024-04-25T10:03:41.302 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:41.303 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":38,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:03:40.294950+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a3406b4d0 msgr2=0x7f3a3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a3406b4d0 0x7f3a3406d990 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f3a40000f60 tx=0x7f3a40009040 comp rx=0 tx=0).stop 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 msgr2=0x7f3a58151b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 0x7f3a58151b40 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3a500097a0 tx=0x7f3a500046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 shutdown_connections 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a3406b4d0 0x7f3a3406d990 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a5815db90 0x7f3a58151b40 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a580d06c0 0x7f3a5815e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 --2- 172.21.15.165:0/750754667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a580b93c0 0x7f3a5814e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 >> 172.21.15.165:0/750754667 conn(0x7f3a580b52b0 msgr2=0x7f3a580d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 shutdown_connections 2024-04-25T10:03:41.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.300+0000 7f3a57200640 1 -- 172.21.15.165:0/750754667 wait complete. 2024-04-25T10:03:41.323 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:41.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.511+0000 7f1112600640 1 Processor -- start 2024-04-25T10:03:41.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.511+0000 7f1112600640 1 -- start start 2024-04-25T10:03:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.512+0000 7f1112600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f110c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.512+0000 7f1112600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f110c06e6a0 0x7f110c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.512+0000 7f1112600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f110c15bb50 con 0x7f110c0cc840 2024-04-25T10:03:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.512+0000 7f1112600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f110c144240 con 0x7f110c06e6a0 2024-04-25T10:03:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.512+0000 7f1112600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c15e220 con 0x7f110c143e60 2024-04-25T10:03:41.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.513+0000 7f1111c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f110c0cc840 0x7f110c070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35058/0 (socket says 172.21.15.165:35058) 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.513+0000 7f1111c00640 1 -- 172.21.15.165:0/2219219142 learned_addr learned my addr 172.21.15.165:0/2219219142 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.513+0000 7f1111200640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f110c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.513+0000 7f1103e00640 1 --2- 172.21.15.165:0/2219219142 >> v2:172.21.15.172:3300/0 conn(0x7f110c06e6a0 0x7f110c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984009418 0 0) 0x7f1104001490 con 0x7f110c0cc840 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10f0005730 con 0x7f110c0cc840 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496508221 0 0) 0x7f10f0005730 con 0x7f110c0cc840 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 msgr2=0x7f110c06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f110c06e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 >> v2:172.21.15.172:3300/0 conn(0x7f110c06e6a0 msgr2=0x7f110c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 --2- 172.21.15.165:0/2219219142 >> v2:172.21.15.172:3300/0 conn(0x7f110c06e6a0 0x7f110c0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c15e500 con 0x7f110c0cc840 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1104001bc0 con 0x7f110c0cc840 2024-04-25T10:03:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 >> v1:172.21.15.165:6789/0 conn(0x7f110c0cc840 legacy=0x7f110c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0005bc0 0x7f10f0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0006a10 0x7f10f0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f10f00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c144240 con 0x7f10f0006a10 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c15e220 con 0x7f10f0005bc0 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f000dbd0 con 0x7f110c143e60 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.514+0000 7f1103e00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0006a10 0x7f10f0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1111200640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0005bc0 0x7f10f0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1103e00640 1 -- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 msgr2=0x7f10f00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1111c00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f10f00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1103e00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f10f00073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1103e00640 1 -- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0005bc0 msgr2=0x7f10f0005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1103e00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0005bc0 0x7f10f0005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1103e00640 1 -- 172.21.15.165:0/2219219142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f0005730 con 0x7f10f0006a10 2024-04-25T10:03:41.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1111200640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0005bc0 0x7f10f0005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.515+0000 7f1111c00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c143e60 0x7f10f00073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.516+0000 7f1103e00640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0006a10 0x7f10f0006e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009ac0 tx=0x7f10f0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce60a9691ecc122 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.516+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4036070 con 0x7f10f0006a10 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.516+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10f4031400 con 0x7f10f0006a10 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.516+0000 7f110be00640 1 -- 172.21.15.165:0/2219219142 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10f4009690 con 0x7f10f0006a10 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1112600640 1 -- 172.21.15.165:0/2219219142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0006a10 msgr2=0x7f10f0006e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1112600640 1 --2- 172.21.15.165:0/2219219142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10f0006a10 0x7f10f0006e90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f10f4009ac0 tx=0x7f10f0007a40 comp rx=0 tx=0).stop 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1111c00640 1 -- 172.21.15.165:0/2219219142 reap_dead start 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1112600640 1 -- 172.21.15.165:0/2219219142 shutdown_connections 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1112600640 1 -- 172.21.15.165:0/2219219142 >> 172.21.15.165:0/2219219142 conn(0x7f110c06abe0 msgr2=0x7f110c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1112600640 1 -- 172.21.15.165:0/2219219142 shutdown_connections 2024-04-25T10:03:41.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.517+0000 7f1112600640 1 -- 172.21.15.165:0/2219219142 wait complete. 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.519+0000 7f1112600640 1 Processor -- start 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.519+0000 7f1112600640 1 -- start start 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1112600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0006a10 0x7f110c0ca990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1112600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c13f6a0 0x7f110c13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1112600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 0x7f110c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1112600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0cccd0 con 0x7f110c0b0bd0 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1112600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f110c15eac0 con 0x7f10f0006a10 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1112600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f110c0cce50 con 0x7f110c13f6a0 2024-04-25T10:03:41.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 0x7f110c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 0x7f110c0b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39460/0 (socket says 172.21.15.165:39460) 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 -- 172.21.15.165:0/1026959675 learned_addr learned my addr 172.21.15.165:0/1026959675 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1103e00640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c13f6a0 0x7f110c13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111200640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0006a10 0x7f110c0ca990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 -- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c13f6a0 msgr2=0x7f110c13fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c13f6a0 0x7f110c13fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 -- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0006a10 msgr2=0x7f110c0ca990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.520+0000 7f1111c00640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0006a10 0x7f110c0ca990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1111c00640 1 -- 172.21.15.165:0/1026959675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f110c13ed20 con 0x7f110c0b0bd0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1111c00640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 0x7f110c0b1050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1104001d30 tx=0x7f110400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104004d60 con 0x7f110c0b0bd0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f110400bb50 con 0x7f110c0b0bd0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1104007070 con 0x7f110c0b0bd0 2024-04-25T10:03:41.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f110c0b1970 con 0x7f110c0b0bd0 2024-04-25T10:03:41.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.521+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f110c13cf90 con 0x7f110c0b0bd0 2024-04-25T10:03:41.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.522+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f110c073990 con 0x7f110c0b0bd0 2024-04-25T10:03:41.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.523+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f110401b020 con 0x7f110c0b0bd0 2024-04-25T10:03:41.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.524+0000 7f1109600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f10e806b4d0 0x7f10e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.524+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f110408a410 con 0x7f110c0b0bd0 2024-04-25T10:03:41.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.525+0000 7f1111200640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f10e806b4d0 0x7f10e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.525+0000 7f1111200640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f10e806b4d0 0x7f10e806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098f0 tx=0x7f10fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.529+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f110c15bb50 con 0x7f110c0b0bd0 2024-04-25T10:03:41.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.722+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f110c0caed0 con 0x7f110c0b0bd0 2024-04-25T10:03:41.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.724+0000 7f1109600640 1 -- 172.21.15.165:0/1026959675 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f1104059060 con 0x7f110c0b0bd0 2024-04-25T10:03:41.729 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:41.730 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":38,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:03:40.294950+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.727+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f10e806b4d0 msgr2=0x7f10e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.727+0000 7f1112600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f10e806b4d0 0x7f10e806d990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f10fc0098f0 tx=0x7f10fc008040 comp rx=0 tx=0).stop 2024-04-25T10:03:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.727+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 msgr2=0x7f110c0b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.727+0000 7f1112600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 0x7f110c0b1050 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f1104001d30 tx=0x7f110400b4a0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 shutdown_connections 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f10e806b4d0 0x7f10e806d990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f110c0b0bd0 0x7f110c0b1050 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f110c13f6a0 0x7f110c13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 --2- 172.21.15.165:0/1026959675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10f0006a10 0x7f110c0ca990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 >> 172.21.15.165:0/1026959675 conn(0x7f110c06abe0 msgr2=0x7f110c0ae8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 shutdown_connections 2024-04-25T10:03:41.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.728+0000 7f1112600640 1 -- 172.21.15.165:0/1026959675 wait complete. 2024-04-25T10:03:41.752 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.955+0000 7f92d2400640 1 Processor -- start 2024-04-25T10:03:41.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.956+0000 7f92d2400640 1 -- start start 2024-04-25T10:03:41.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0ceb20 0x7f92cc0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d2400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f92cc0c5030 0x7f92cc15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d2400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92cc145520 con 0x7f92cc145140 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d2400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92cc071270 con 0x7f92cc0c5030 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc071550 con 0x7f92cc0ceb20 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d1a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f92cc0c5030 0x7f92cc15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d1a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f92cc0c5030 0x7f92cc15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39466/0 (socket says 172.21.15.165:39466) 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.957+0000 7f92d1a00640 1 -- 172.21.15.165:0/3654383166 learned_addr learned my addr 172.21.15.165:0/3654383166 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.958+0000 7f92cbe00640 1 --2- 172.21.15.165:0/3654383166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0ceb20 0x7f92cc0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.958+0000 7f92d1a00640 1 -- 172.21.15.165:0/3654383166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0ceb20 msgr2=0x7f92cc0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.958+0000 7f92d1a00640 1 --2- 172.21.15.165:0/3654383166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0ceb20 0x7f92cc0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.958+0000 7f92d1a00640 1 -- 172.21.15.165:0/3654383166 >> v1:172.21.15.172:6789/0 conn(0x7f92cc145140 legacy=0x7f92cc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:41.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.958+0000 7f92d1a00640 1 -- 172.21.15.165:0/3654383166 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc071830 con 0x7f92cc0c5030 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.959+0000 7f92d1a00640 1 --2- 172.21.15.165:0/3654383166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc0c5030 0x7f92cc15bfb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009e50 tx=0x7f92c402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cec3440bb046f266 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.959+0000 7f92cb400640 1 -- 172.21.15.165:0/3654383166 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4036070 con 0x7f92cc0c5030 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.959+0000 7f92cb400640 1 -- 172.21.15.165:0/3654383166 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4031600 con 0x7f92cc0c5030 2024-04-25T10:03:41.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.959+0000 7f92cb400640 1 -- 172.21.15.165:0/3654383166 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c40319d0 con 0x7f92cc0c5030 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 -- 172.21.15.165:0/3654383166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc0c5030 msgr2=0x7f92cc15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 --2- 172.21.15.165:0/3654383166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc0c5030 0x7f92cc15bfb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009e50 tx=0x7f92c402a630 comp rx=0 tx=0).stop 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 -- 172.21.15.165:0/3654383166 shutdown_connections 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 --2- 172.21.15.165:0/3654383166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc0c5030 0x7f92cc15bfb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 --2- 172.21.15.165:0/3654383166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0ceb20 0x7f92cc0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 -- 172.21.15.165:0/3654383166 >> 172.21.15.165:0/3654383166 conn(0x7f92cc061cc0 msgr2=0x7f92cc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 -- 172.21.15.165:0/3654383166 shutdown_connections 2024-04-25T10:03:41.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.960+0000 7f92d2400640 1 -- 172.21.15.165:0/3654383166 wait complete. 2024-04-25T10:03:41.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.962+0000 7f92d2400640 1 Processor -- start 2024-04-25T10:03:41.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.962+0000 7f92d2400640 1 -- start start 2024-04-25T10:03:41.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0c5030 0x7f92cc0cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d2400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92cc0ceb20 0x7f92cc143000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d2400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 0x7f92cc0b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d2400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0c5640 con 0x7f92cc145140 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d2400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc0c54c0 con 0x7f92cc0ceb20 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92cc15c4f0 con 0x7f92cc0c5030 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 0x7f92cc0b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 0x7f92cc0b0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39478/0 (socket says 172.21.15.165:39478) 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.963+0000 7f92d1a00640 1 -- 172.21.15.165:0/2128287481 learned_addr learned my addr 172.21.15.165:0/2128287481 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92d1a00640 1 -- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0c5030 msgr2=0x7f92cc0cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92d1a00640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0c5030 0x7f92cc0cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92d1a00640 1 -- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92cc0ceb20 msgr2=0x7f92cc143000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92cbe00640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92cc0ceb20 0x7f92cc143000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92d1a00640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92cc0ceb20 0x7f92cc143000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:41.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92d1a00640 1 -- 172.21.15.165:0/2128287481 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92cc1f3930 con 0x7f92cc145140 2024-04-25T10:03:41.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92d1a00640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 0x7f92cc0b0ae0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009f70 tx=0x7f92c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.964+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4036050 con 0x7f92cc145140 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.965+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c4042070 con 0x7f92cc145140 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.965+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92cc1f3c10 con 0x7f92cc145140 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.965+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c4009510 con 0x7f92cc145140 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.965+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92cc1f41d0 con 0x7f92cc145140 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.966+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92cc1f4410 con 0x7f92cc145140 2024-04-25T10:03:41.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.967+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92c4007bd0 con 0x7f92cc145140 2024-04-25T10:03:41.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.968+0000 7f92c8c00640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92ac06b4d0 0x7f92ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:41.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.968+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92c40ab350 con 0x7f92cc145140 2024-04-25T10:03:41.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.968+0000 7f92d1000640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92ac06b4d0 0x7f92ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:41.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.969+0000 7f92d1000640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92ac06b4d0 0x7f92ac06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92c0006fd0 tx=0x7f92c0006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:41.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:41.972+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92cc1f4410 con 0x7f92cc145140 2024-04-25T10:03:42.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.205+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f92cc15e9f0 con 0x7f92cc145140 2024-04-25T10:03:42.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.898+0000 7f92c8c00640 1 -- 172.21.15.165:0/2128287481 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f92c407a040 con 0x7f92cc145140 2024-04-25T10:03:42.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92ac06b4d0 msgr2=0x7f92ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92ac06b4d0 0x7f92ac06d990 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f92c0006fd0 tx=0x7f92c0006bc0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 msgr2=0x7f92cc0b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:42.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 0x7f92cc0b0ae0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f92c4009f70 tx=0x7f92c4004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 shutdown_connections 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92ac06b4d0 0x7f92ac06d990 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92cc145140 0x7f92cc0b0ae0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92cc0ceb20 0x7f92cc143000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 --2- 172.21.15.165:0/2128287481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92cc0c5030 0x7f92cc0cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.901+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 >> 172.21.15.165:0/2128287481 conn(0x7f92cc061cc0 msgr2=0x7f92cc0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.902+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 shutdown_connections 2024-04-25T10:03:42.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:42.902+0000 7f92d2400640 1 -- 172.21.15.165:0/2128287481 wait complete. 2024-04-25T10:03:42.927 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:03:42.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:03:43.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.129+0000 7f67df400640 1 Processor -- start 2024-04-25T10:03:43.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.129+0000 7f67df400640 1 -- start start 2024-04-25T10:03:43.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67df400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f67d80cf640 0x7f67d806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67df400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d80cfad0 con 0x7f67d815b710 2024-04-25T10:03:43.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67df400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d80cfd80 con 0x7f67d80ac8b0 2024-04-25T10:03:43.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67df400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d806f190 con 0x7f67d80cf640 2024-04-25T10:03:43.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67dca00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f67d815b710 0x7f67d815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35068/0 (socket says 172.21.15.165:35068) 2024-04-25T10:03:43.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67dca00640 1 -- 172.21.15.165:0/2567962997 learned_addr learned my addr 172.21.15.165:0/2567962997 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.131+0000 7f67d7e00640 1 --2- 172.21.15.165:0/2567962997 >> v2:172.21.15.172:3300/0 conn(0x7f67d80cf640 0x7f67d806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1276407387 0 0) 0x7f67d80cfad0 con 0x7f67d815b710 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67c0003520 con 0x7f67d815b710 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3333626457 0 0) 0x7f67c0003520 con 0x7f67d815b710 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 >> v2:172.21.15.172:3300/0 conn(0x7f67d80cf640 msgr2=0x7f67d806ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 --2- 172.21.15.165:0/2567962997 >> v2:172.21.15.172:3300/0 conn(0x7f67d80cf640 0x7f67d806ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 >> v1:172.21.15.172:6789/0 conn(0x7f67d80ac8b0 legacy=0x7f67d8071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d80d0060 con 0x7f67d815b710 2024-04-25T10:03:43.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f67cc001670 con 0x7f67d815b710 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 >> v1:172.21.15.165:6789/0 conn(0x7f67d815b710 legacy=0x7f67d815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67c00039b0 0x7f67c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c0004800 0x7f67c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67c00051c0 0x7f67c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d806f190 con 0x7f67c00051c0 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67c000c060 con 0x7f67c0004800 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67c000c260 con 0x7f67c00039b0 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.132+0000 7f67d7e00640 1 --2- 172.21.15.165:0/2567962997 >> v2:172.21.15.172:3300/0 conn(0x7f67d80cf640 0x7f67d806ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.133+0000 7f67dd400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67c00051c0 0x7f67c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.133+0000 7f67dd400640 1 -- 172.21.15.165:0/2567962997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67c00039b0 msgr2=0x7f67c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.133+0000 7f67dd400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67c00039b0 0x7f67c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.133+0000 7f67dd400640 1 -- 172.21.15.165:0/2567962997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c0004800 msgr2=0x7f67c0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.133+0000 7f67dd400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c0004800 0x7f67c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.133+0000 7f67dd400640 1 -- 172.21.15.165:0/2567962997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c0003520 con 0x7f67c00051c0 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.134+0000 7f67dd400640 1 -- 172.21.15.165:0/2567962997 reap_dead start 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.134+0000 7f67dd400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67c00051c0 0x7f67c000b9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67d0001e50 tx=0x7f67c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16453a37dccdd4af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.134+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0001a70 con 0x7f67c00051c0 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.134+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67d0003970 con 0x7f67c00051c0 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.134+0000 7f67d7400640 1 -- 172.21.15.165:0/2567962997 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67d0003d60 con 0x7f67c00051c0 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 -- 172.21.15.165:0/2567962997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67c00051c0 msgr2=0x7f67c000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67c00051c0 0x7f67c000b9c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f67d0001e50 tx=0x7f67c0005830 comp rx=0 tx=0).stop 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 -- 172.21.15.165:0/2567962997 shutdown_connections 2024-04-25T10:03:43.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 --2- 172.21.15.165:0/2567962997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67c00051c0 0x7f67c000b9c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 -- 172.21.15.165:0/2567962997 >> 172.21.15.165:0/2567962997 conn(0x7f67d806abe0 msgr2=0x7f67d806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 -- 172.21.15.165:0/2567962997 shutdown_connections 2024-04-25T10:03:43.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.135+0000 7f67df400640 1 -- 172.21.15.165:0/2567962997 wait complete. 2024-04-25T10:03:43.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.137+0000 7f67df400640 1 Processor -- start 2024-04-25T10:03:43.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.137+0000 7f67df400640 1 -- start start 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67df400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c00051c0 0x7f67d80cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 0x7f67d80afd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67df400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67d81f3430 0x7f67d80b1290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67df400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d815baf0 con 0x7f67d815a650 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67df400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d815e490 con 0x7f67c00051c0 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67df400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d815bc70 con 0x7f67d81f3430 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 0x7f67d80afd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 0x7f67d80afd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39502/0 (socket says 172.21.15.165:39502) 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 -- 172.21.15.165:0/3925439023 learned_addr learned my addr 172.21.15.165:0/3925439023 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67dca00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c00051c0 0x7f67d80cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 -- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67d81f3430 msgr2=0x7f67d80b1290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67d81f3430 0x7f67d80b1290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 -- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c00051c0 msgr2=0x7f67d80cc0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c00051c0 0x7f67d80cc0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.138+0000 7f67d7e00640 1 -- 172.21.15.165:0/3925439023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d815faa0 con 0x7f67d815a650 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.139+0000 7f67d7e00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 0x7f67d80afd90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e900 tx=0x7f67c400edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.139+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c400ca00 con 0x7f67d815a650 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.139+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c400cbc0 con 0x7f67d815a650 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.139+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4004030 con 0x7f67d815a650 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.139+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d815fd80 con 0x7f67d815a650 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.139+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d80b02d0 con 0x7f67d815a650 2024-04-25T10:03:43.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.140+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d8073a00 con 0x7f67d815a650 2024-04-25T10:03:43.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.141+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67c40041d0 con 0x7f67d815a650 2024-04-25T10:03:43.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.142+0000 7f67d4c00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67b806b4d0 0x7f67b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:43.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.142+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67c408a880 con 0x7f67d815a650 2024-04-25T10:03:43.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.142+0000 7f67dca00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67b806b4d0 0x7f67b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:43.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.143+0000 7f67dca00640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67b806b4d0 0x7f67b806d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67cc007a10 tx=0x7f67cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:43.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.146+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67c4017020 con 0x7f67d815a650 2024-04-25T10:03:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.378+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f67d806b2a0 con 0x7f67d815a650 2024-04-25T10:03:43.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.901+0000 7f67d4c00640 1 -- 172.21.15.165:0/3925439023 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v19) v1 ==== 122+0+0 (secure 0 0 0) 0x7f67c4059600 con 0x7f67d815a650 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67b806b4d0 msgr2=0x7f67b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67b806b4d0 0x7f67b806d990 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f67cc007a10 tx=0x7f67cc009040 comp rx=0 tx=0).stop 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 msgr2=0x7f67d80afd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 0x7f67d80afd90 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f67c400e900 tx=0x7f67c400edc0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 shutdown_connections 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67b806b4d0 0x7f67b806d990 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67d81f3430 0x7f67d80b1290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67d815a650 0x7f67d80afd90 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 --2- 172.21.15.165:0/3925439023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67c00051c0 0x7f67d80cc0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 >> 172.21.15.165:0/3925439023 conn(0x7f67d806abe0 msgr2=0x7f67d80adf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:43.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 shutdown_connections 2024-04-25T10:03:43.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:43.905+0000 7f67df400640 1 -- 172.21.15.165:0/3925439023 wait complete. 2024-04-25T10:03:43.931 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:03:43.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.114+0000 7f8c6b200640 1 Processor -- start 2024-04-25T10:03:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.115+0000 7f8c6b200640 1 -- start start 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c6b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8c6c15c100 0x7f8c6c071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c0b3800 0x7f8c6c06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c6b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c6c06eac0 0x7f8c6c165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c6b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b3420 con 0x7f8c6c15c100 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c6b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6c071c80 con 0x7f8c6c06eac0 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c6b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b2810 con 0x7f8c6c0b3800 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.116+0000 7f8c69e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8c6c15c100 0x7f8c6c071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8c6c15c100 0x7f8c6c071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39506/0 (socket says 172.21.15.165:39506) 2024-04-25T10:03:44.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 -- 172.21.15.165:0/1322602665 learned_addr learned my addr 172.21.15.165:0/1322602665 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:44.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c6a800640 1 --2- 172.21.15.165:0/1322602665 >> v2:172.21.15.172:3300/0 conn(0x7f8c6c06eac0 0x7f8c6c165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69400640 1 --2- 172.21.15.165:0/1322602665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c0b3800 0x7f8c6c06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 -- 172.21.15.165:0/1322602665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c0b3800 msgr2=0x7f8c6c06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 --2- 172.21.15.165:0/1322602665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c0b3800 0x7f8c6c06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 -- 172.21.15.165:0/1322602665 >> v2:172.21.15.172:3300/0 conn(0x7f8c6c06eac0 msgr2=0x7f8c6c165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 --2- 172.21.15.165:0/1322602665 >> v2:172.21.15.172:3300/0 conn(0x7f8c6c06eac0 0x7f8c6c165850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.117+0000 7f8c69e00640 1 -- 172.21.15.165:0/1322602665 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c0b2af0 con 0x7f8c6c15c100 2024-04-25T10:03:44.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.118+0000 7f8c69e00640 1 --2- 172.21.15.165:0/1322602665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c15c100 0x7f8c6c071740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009870 tx=0x7f8c5c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67c4c97bb226241 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.119+0000 7f8c68a00640 1 -- 172.21.15.165:0/1322602665 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c037070 con 0x7f8c6c15c100 2024-04-25T10:03:44.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.119+0000 7f8c68a00640 1 -- 172.21.15.165:0/1322602665 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c5c032440 con 0x7f8c6c15c100 2024-04-25T10:03:44.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.119+0000 7f8c68a00640 1 -- 172.21.15.165:0/1322602665 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5c032810 con 0x7f8c6c15c100 2024-04-25T10:03:44.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 -- 172.21.15.165:0/1322602665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c15c100 msgr2=0x7f8c6c071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 --2- 172.21.15.165:0/1322602665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c15c100 0x7f8c6c071740 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c009870 tx=0x7f8c5c02a210 comp rx=0 tx=0).stop 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 -- 172.21.15.165:0/1322602665 shutdown_connections 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 --2- 172.21.15.165:0/1322602665 >> v2:172.21.15.172:3300/0 conn(0x7f8c6c06eac0 0x7f8c6c165850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 --2- 172.21.15.165:0/1322602665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c0b3800 0x7f8c6c06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 --2- 172.21.15.165:0/1322602665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c15c100 0x7f8c6c071740 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 -- 172.21.15.165:0/1322602665 >> 172.21.15.165:0/1322602665 conn(0x7f8c6c06abe0 msgr2=0x7f8c6c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 -- 172.21.15.165:0/1322602665 shutdown_connections 2024-04-25T10:03:44.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.120+0000 7f8c6b200640 1 -- 172.21.15.165:0/1322602665 wait complete. 2024-04-25T10:03:44.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.121+0000 7f8c6b200640 1 Processor -- start 2024-04-25T10:03:44.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 -- start start 2024-04-25T10:03:44.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c6c06eac0 0x7f8c6c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 0x7f8c6c15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c15c100 0x7f8c6c06de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b24e0 con 0x7f8c6c0b3800 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0b30b0 con 0x7f8c6c06eac0 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.122+0000 7f8c6b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6c0711b0 con 0x7f8c6c15c100 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 0x7f8c6c15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 0x7f8c6c15d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39516/0 (socket says 172.21.15.165:39516) 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 -- 172.21.15.165:0/2769615381 learned_addr learned my addr 172.21.15.165:0/2769615381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 -- 172.21.15.165:0/2769615381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c15c100 msgr2=0x7f8c6c06de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c15c100 0x7f8c6c06de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 -- 172.21.15.165:0/2769615381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c6c06eac0 msgr2=0x7f8c6c157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c6c06eac0 0x7f8c6c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 -- 172.21.15.165:0/2769615381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6c15f3e0 con 0x7f8c6c0b3800 2024-04-25T10:03:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.123+0000 7f8c69400640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 0x7f8c6c15d440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8c54007ea0 tx=0x7f8c54007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.124+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54014620 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.124+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c54019070 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.124+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c6c15c680 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.124+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c54014a00 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.124+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c6c15cc40 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.125+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c6c073990 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.126+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c54014ba0 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.127+0000 7f8c62000640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c4806b4d0 0x7f8c4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.127+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c5408ac60 con 0x7f8c6c0b3800 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.127+0000 7f8c69e00640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c4806b4d0 0x7f8c4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.127+0000 7f8c69e00640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c4806b4d0 0x7f8c4806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c034000 tx=0x7f8c5c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.131+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c54059940 con 0x7f8c6c0b3800 2024-04-25T10:03:44.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.362+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c6c0b0c10 con 0x7f8c6c0b3800 2024-04-25T10:03:44.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.363+0000 7f8c62000640 1 -- 172.21.15.165:0/2769615381 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f8c54059240 con 0x7f8c6c0b3800 2024-04-25T10:03:44.368 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:44.369 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:40.296050+0000","modified":"2024-04-25T10:03:43.379748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/3282481342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3282481342},{"type":"v1","addr":"172.21.15.165:6837","nonce":3282481342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:44.369 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 19 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.366+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c4806b4d0 msgr2=0x7f8c4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.366+0000 7f8c6b200640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c4806b4d0 0x7f8c4806d990 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f8c5c034000 tx=0x7f8c5c02c000 comp rx=0 tx=0).stop 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.366+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 msgr2=0x7f8c6c15d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.366+0000 7f8c6b200640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 0x7f8c6c15d440 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8c54007ea0 tx=0x7f8c54007f90 comp rx=0 tx=0).stop 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 shutdown_connections 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c4806b4d0 0x7f8c4806d990 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c6c15c100 0x7f8c6c06de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c6c0b3800 0x7f8c6c15d440 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 --2- 172.21.15.165:0/2769615381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c6c06eac0 0x7f8c6c157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 >> 172.21.15.165:0/2769615381 conn(0x7f8c6c06abe0 msgr2=0x7f8c6c0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 shutdown_connections 2024-04-25T10:03:44.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.367+0000 7f8c6b200640 1 -- 172.21.15.165:0/2769615381 wait complete. 2024-04-25T10:03:44.391 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:03:44.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.587+0000 7ffb7dc00640 1 Processor -- start 2024-04-25T10:03:44.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.588+0000 7ffb7dc00640 1 -- start start 2024-04-25T10:03:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.589+0000 7ffb7dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.589+0000 7ffb7dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffb780b2cb0 0x7ffb780d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.589+0000 7ffb7dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb780b97a0 con 0x7ffb780b92a0 2024-04-25T10:03:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.589+0000 7ffb7dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb7815dfc0 con 0x7ffb780b2cb0 2024-04-25T10:03:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.589+0000 7ffb7dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d1a00 con 0x7ffb7815dbe0 2024-04-25T10:03:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.590+0000 7ffb76a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.590+0000 7ffb76000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffb780b2cb0 0x7ffb780d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.590+0000 7ffb76a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:47508/0 (socket says 172.21.15.165:47508) 2024-04-25T10:03:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.590+0000 7ffb76a00640 1 -- 172.21.15.165:0/2732192763 learned_addr learned my addr 172.21.15.165:0/2732192763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.591+0000 7ffb76a00640 1 -- 172.21.15.165:0/2732192763 >> v2:172.21.15.172:3300/0 conn(0x7ffb780b2cb0 msgr2=0x7ffb780d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.591+0000 7ffb76a00640 1 --2- 172.21.15.165:0/2732192763 >> v2:172.21.15.172:3300/0 conn(0x7ffb780b2cb0 0x7ffb780d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.591+0000 7ffb76a00640 1 -- 172.21.15.165:0/2732192763 >> v1:172.21.15.172:6789/0 conn(0x7ffb780b92a0 legacy=0x7ffb780b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.591+0000 7ffb76a00640 1 -- 172.21.15.165:0/2732192763 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780d1c50 con 0x7ffb7815dbe0 2024-04-25T10:03:44.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.592+0000 7ffb76a00640 1 --2- 172.21.15.165:0/2732192763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780d0910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009870 tx=0x7ffb6802a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60f55e2c161f686b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.592+0000 7ffb75600640 1 -- 172.21.15.165:0/2732192763 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68037070 con 0x7ffb7815dbe0 2024-04-25T10:03:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.592+0000 7ffb75600640 1 -- 172.21.15.165:0/2732192763 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb68032440 con 0x7ffb7815dbe0 2024-04-25T10:03:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.592+0000 7ffb75600640 1 -- 172.21.15.165:0/2732192763 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68032810 con 0x7ffb7815dbe0 2024-04-25T10:03:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 -- 172.21.15.165:0/2732192763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 msgr2=0x7ffb780d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/2732192763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780d0910 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ffb68009870 tx=0x7ffb6802a250 comp rx=0 tx=0).stop 2024-04-25T10:03:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 -- 172.21.15.165:0/2732192763 shutdown_connections 2024-04-25T10:03:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/2732192763 >> v2:172.21.15.172:3300/0 conn(0x7ffb780b2cb0 0x7ffb780d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/2732192763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780d0910 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 -- 172.21.15.165:0/2732192763 >> 172.21.15.165:0/2732192763 conn(0x7ffb78061a20 msgr2=0x7ffb78061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 -- 172.21.15.165:0/2732192763 shutdown_connections 2024-04-25T10:03:44.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.593+0000 7ffb7dc00640 1 -- 172.21.15.165:0/2732192763 wait complete. 2024-04-25T10:03:44.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.595+0000 7ffb7dc00640 1 Processor -- start 2024-04-25T10:03:44.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.596+0000 7ffb7dc00640 1 -- start start 2024-04-25T10:03:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb7dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb780b2cb0 0x7ffb781469b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb7dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 0x7ffb780b7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb7dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780b5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb7dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780b6ab0 con 0x7ffb780b92a0 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb7dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780d2210 con 0x7ffb780b2cb0 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb7dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb780b6c30 con 0x7ffb7815dbe0 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb76000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 0x7ffb780b7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb76000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 0x7ffb780b7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39530/0 (socket says 172.21.15.165:39530) 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb76000640 1 -- 172.21.15.165:0/922343889 learned_addr learned my addr 172.21.15.165:0/922343889 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb76a00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb780b2cb0 0x7ffb781469b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.597+0000 7ffb77400640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780b5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76000640 1 -- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 msgr2=0x7ffb780b5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76000640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780b5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76000640 1 -- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb780b2cb0 msgr2=0x7ffb781469b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76000640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb780b2cb0 0x7ffb781469b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76000640 1 -- 172.21.15.165:0/922343889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb780cae10 con 0x7ffb780b92a0 2024-04-25T10:03:44.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb77400640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:44.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76a00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb780b2cb0 0x7ffb781469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:44.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb76000640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 0x7ffb780b7970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400e9b0 tx=0x7ffb6400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb64017070 con 0x7ffb780b92a0 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb6400c9a0 con 0x7ffb780b92a0 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb780cb0f0 con 0x7ffb780b92a0 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6400cd90 con 0x7ffb780b92a0 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.598+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb780cb6b0 con 0x7ffb780b92a0 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.599+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb78073990 con 0x7ffb780b92a0 2024-04-25T10:03:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.600+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb6401a070 con 0x7ffb780b92a0 2024-04-25T10:03:44.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.601+0000 7ffb6ea00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb5806b4d0 0x7ffb5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:44.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.601+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb6408a780 con 0x7ffb780b92a0 2024-04-25T10:03:44.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.602+0000 7ffb76a00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb5806b4d0 0x7ffb5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:44.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.602+0000 7ffb76a00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb5806b4d0 0x7ffb5806d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb68034000 tx=0x7ffb6802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:44.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.607+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb64059460 con 0x7ffb780b92a0 2024-04-25T10:03:44.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.807+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffb7815c300 con 0x7ffb780b92a0 2024-04-25T10:03:44.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.809+0000 7ffb6ea00640 1 -- 172.21.15.165:0/922343889 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+16978 (secure 0 0 0) 0x7ffb64058d60 con 0x7ffb780b92a0 2024-04-25T10:03:44.815 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:44.815 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":39,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:03:41.875587+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:03:44.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb5806b4d0 msgr2=0x7ffb5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb5806b4d0 0x7ffb5806d990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb68034000 tx=0x7ffb6802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:44.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 msgr2=0x7ffb780b7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:44.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 0x7ffb780b7970 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7ffb6400e9b0 tx=0x7ffb6400ee70 comp rx=0 tx=0).stop 2024-04-25T10:03:44.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 shutdown_connections 2024-04-25T10:03:44.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb5806b4d0 0x7ffb5806d990 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7ffb68034000 tx=0x7ffb6802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:44.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb7815dbe0 0x7ffb780b5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb780b92a0 0x7ffb780b7970 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 --2- 172.21.15.165:0/922343889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb780b2cb0 0x7ffb781469b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:44.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.812+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 >> 172.21.15.165:0/922343889 conn(0x7ffb78061a20 msgr2=0x7ffb7815c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:44.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.813+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 shutdown_connections 2024-04-25T10:03:44.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:44.813+0000 7ffb7dc00640 1 -- 172.21.15.165:0/922343889 wait complete. 2024-04-25T10:03:44.835 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:03:44.835 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:45.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.029+0000 7f9131e00640 1 Processor -- start 2024-04-25T10:03:45.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.030+0000 7f9131e00640 1 -- start start 2024-04-25T10:03:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.031+0000 7f9131e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f912c15dbe0 0x7f912c0d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.031+0000 7f9131e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 0x7f912c0d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.031+0000 7f9131e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912c0b97a0 con 0x7f912c0b2cb0 2024-04-25T10:03:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.031+0000 7f9131e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912c15dfc0 con 0x7f912c15dbe0 2024-04-25T10:03:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.031+0000 7f9131e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0d1a00 con 0x7f912c0b92a0 2024-04-25T10:03:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.032+0000 7f912aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f912c15dbe0 0x7f912c0d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.032+0000 7f912b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 0x7f912c0d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.032+0000 7f912aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f912c15dbe0 0x7f912c0d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:44190/0 (socket says 172.21.15.165:44190) 2024-04-25T10:03:45.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.032+0000 7f912b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 0x7f912c0d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:47526/0 (socket says 172.21.15.165:47526) 2024-04-25T10:03:45.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.032+0000 7f912aa00640 1 -- 172.21.15.165:0/2611425045 learned_addr learned my addr 172.21.15.165:0/2611425045 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:45.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.033+0000 7f912aa00640 1 -- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 msgr2=0x7f912c0d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.033+0000 7f912aa00640 1 --2- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 0x7f912c0d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.033+0000 7f912aa00640 1 -- 172.21.15.165:0/2611425045 >> v1:172.21.15.172:6789/0 conn(0x7f912c0b2cb0 legacy=0x7f912c0b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.033+0000 7f912aa00640 1 -- 172.21.15.165:0/2611425045 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c0d1c50 con 0x7f912c15dbe0 2024-04-25T10:03:45.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.033+0000 7f912b400640 1 --2- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 0x7f912c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:03:45.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.034+0000 7f912aa00640 1 --2- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c15dbe0 0x7f912c0d0910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f911c009870 tx=0x7f911c02a250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d1e781b42458c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f912a000640 1 -- 172.21.15.165:0/2611425045 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c037070 con 0x7f912c15dbe0 2024-04-25T10:03:45.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f912a000640 1 -- 172.21.15.165:0/2611425045 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c032440 con 0x7f912c15dbe0 2024-04-25T10:03:45.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f912a000640 1 -- 172.21.15.165:0/2611425045 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c032810 con 0x7f912c15dbe0 2024-04-25T10:03:45.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f9131e00640 1 -- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c15dbe0 msgr2=0x7f912c0d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f9131e00640 1 --2- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c15dbe0 0x7f912c0d0910 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f911c009870 tx=0x7f911c02a250 comp rx=0 tx=0).stop 2024-04-25T10:03:45.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f9131e00640 1 -- 172.21.15.165:0/2611425045 shutdown_connections 2024-04-25T10:03:45.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f9131e00640 1 --2- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b92a0 0x7f912c0d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f9131e00640 1 --2- 172.21.15.165:0/2611425045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c15dbe0 0x7f912c0d0910 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.035+0000 7f9131e00640 1 -- 172.21.15.165:0/2611425045 >> 172.21.15.165:0/2611425045 conn(0x7f912c061a20 msgr2=0x7f912c061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.036+0000 7f9131e00640 1 -- 172.21.15.165:0/2611425045 shutdown_connections 2024-04-25T10:03:45.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.036+0000 7f9131e00640 1 -- 172.21.15.165:0/2611425045 wait complete. 2024-04-25T10:03:45.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.037+0000 7f9131e00640 1 Processor -- start 2024-04-25T10:03:45.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.037+0000 7f9131e00640 1 -- start start 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.038+0000 7f9131e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b2cb0 0x7f912c15f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.038+0000 7f9131e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c0b92a0 0x7f912c15fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.038+0000 7f9131e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 0x7f912c0caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.038+0000 7f9131e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0b6ab0 con 0x7f912c15dbe0 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.038+0000 7f9131e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0d2210 con 0x7f912c0b92a0 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.038+0000 7f9131e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912c0b6c30 con 0x7f912c0b2cb0 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 0x7f912c0caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 0x7f912c0caa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39532/0 (socket says 172.21.15.165:39532) 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 -- 172.21.15.165:0/4207674072 learned_addr learned my addr 172.21.15.165:0/4207674072 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912aa00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b2cb0 0x7f912c15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f9122000640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c0b92a0 0x7f912c15fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 -- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b2cb0 msgr2=0x7f912c15f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b2cb0 0x7f912c15f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 -- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c0b92a0 msgr2=0x7f912c15fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c0b92a0 0x7f912c15fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 -- 172.21.15.165:0/4207674072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912c1f3aa0 con 0x7f912c15dbe0 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912aa00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b2cb0 0x7f912c15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:45.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.039+0000 7f912b400640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 0x7f912c0caa50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f912400efb0 tx=0x7f912400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.040+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124018020 con 0x7f912c15dbe0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.040+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9124014480 con 0x7f912c15dbe0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.040+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9124014850 con 0x7f912c15dbe0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.040+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912c1f3d80 con 0x7f912c15dbe0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.040+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912c1f4340 con 0x7f912c15dbe0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.041+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912c073990 con 0x7f912c15dbe0 2024-04-25T10:03:45.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.042+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9124005d10 con 0x7f912c15dbe0 2024-04-25T10:03:45.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.043+0000 7f9123e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f910806b4d0 0x7f910806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.043+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f912408b1e0 con 0x7f912c15dbe0 2024-04-25T10:03:45.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.043+0000 7f912aa00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f910806b4d0 0x7f910806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.044+0000 7f912aa00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f910806b4d0 0x7f910806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f911c034000 tx=0x7f911c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.047+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9124022030 con 0x7f912c15dbe0 2024-04-25T10:03:45.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.240+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f912c1527a0 con 0x7f910806b4d0 2024-04-25T10:03:45.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.260+0000 7f9123e00640 1 -- 172.21.15.165:0/4207674072 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251939 (secure 0 0 0) 0x7f912c1527a0 con 0x7f910806b4d0 2024-04-25T10:03:45.269 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:45.270 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:03:45.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f910806b4d0 msgr2=0x7f910806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f910806b4d0 0x7f910806d990 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f911c034000 tx=0x7f911c02e000 comp rx=0 tx=0).stop 2024-04-25T10:03:45.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 msgr2=0x7f912c0caa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 0x7f912c0caa50 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f912400efb0 tx=0x7f912400c490 comp rx=0 tx=0).stop 2024-04-25T10:03:45.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 shutdown_connections 2024-04-25T10:03:45.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f910806b4d0 0x7f910806d990 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f912c15dbe0 0x7f912c0caa50 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f912c0b92a0 0x7f912c15fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 --2- 172.21.15.165:0/4207674072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f912c0b2cb0 0x7f912c15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.266+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 >> 172.21.15.165:0/4207674072 conn(0x7f912c061a20 msgr2=0x7f912c0b6ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.267+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 shutdown_connections 2024-04-25T10:03:45.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.267+0000 7f9131e00640 1 -- 172.21.15.165:0/4207674072 wait complete. 2024-04-25T10:03:45.291 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-04-25T10:03:44.907324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220828,"kb_used_data":4660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753892,"statfs":{"total":773094113280,"available":772867985408,"internally_reserved":0,"allocated":4771840,"data_stored":2704656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.070866"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888541+0000","last_change":"2024-04-25T10:03:40.912706+0000","last_active":"2024-04-25T10:03:41.888541+0000","last_peered":"2024-04-25T10:03:41.888541+0000","last_clean":"2024-04-25T10:03:41.888541+0000","last_became_active":"2024-04-25T10:03:40.912441+0000","last_became_peered":"2024-04-25T10:03:40.912441+0000","last_unstale":"2024-04-25T10:03:41.888541+0000","last_undegraded":"2024-04-25T10:03:41.888541+0000","last_fullsized":"2024-04-25T10:03:41.888541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:39.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888689+0000","last_change":"2024-04-25T10:03:40.910560+0000","last_active":"2024-04-25T10:03:41.888689+0000","last_peered":"2024-04-25T10:03:41.888689+0000","last_clean":"2024-04-25T10:03:41.888689+0000","last_became_active":"2024-04-25T10:03:40.907119+0000","last_became_peered":"2024-04-25T10:03:40.907119+0000","last_unstale":"2024-04-25T10:03:41.888689+0000","last_undegraded":"2024-04-25T10:03:41.888689+0000","last_fullsized":"2024-04-25T10:03:41.888689+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:49.177756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.909758+0000","last_change":"2024-04-25T10:03:40.901864+0000","last_active":"2024-04-25T10:03:40.909758+0000","last_peered":"2024-04-25T10:03:40.909758+0000","last_clean":"2024-04-25T10:03:40.909758+0000","last_became_active":"2024-04-25T10:03:40.901551+0000","last_became_peered":"2024-04-25T10:03:40.901551+0000","last_unstale":"2024-04-25T10:03:40.909758+0000","last_undegraded":"2024-04-25T10:03:40.909758+0000","last_fullsized":"2024-04-25T10:03:40.909758+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.898944+0000","last_change":"2024-04-25T10:03:40.901144+0000","last_active":"2024-04-25T10:03:41.898944+0000","last_peered":"2024-04-25T10:03:41.898944+0000","last_clean":"2024-04-25T10:03:41.898944+0000","last_became_active":"2024-04-25T10:03:40.900853+0000","last_became_peered":"2024-04-25T10:03:40.900853+0000","last_unstale":"2024-04-25T10:03:41.898944+0000","last_undegraded":"2024-04-25T10:03:41.898944+0000","last_fullsized":"2024-04-25T10:03:41.898944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:00.308840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899187+0000","last_change":"2024-04-25T10:03:40.889200+0000","last_active":"2024-04-25T10:03:41.899187+0000","last_peered":"2024-04-25T10:03:41.899187+0000","last_clean":"2024-04-25T10:03:41.899187+0000","last_became_active":"2024-04-25T10:03:40.888863+0000","last_became_peered":"2024-04-25T10:03:40.888863+0000","last_unstale":"2024-04-25T10:03:41.899187+0000","last_undegraded":"2024-04-25T10:03:41.899187+0000","last_fullsized":"2024-04-25T10:03:41.899187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.111365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.897111+0000","last_change":"2024-04-25T10:03:40.894308+0000","last_active":"2024-04-25T10:03:40.897111+0000","last_peered":"2024-04-25T10:03:40.897111+0000","last_clean":"2024-04-25T10:03:40.897111+0000","last_became_active":"2024-04-25T10:03:40.893203+0000","last_became_peered":"2024-04-25T10:03:40.893203+0000","last_unstale":"2024-04-25T10:03:40.897111+0000","last_undegraded":"2024-04-25T10:03:40.897111+0000","last_fullsized":"2024-04-25T10:03:40.897111+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.904812+0000","last_change":"2024-04-25T10:03:40.900741+0000","last_active":"2024-04-25T10:03:41.904812+0000","last_peered":"2024-04-25T10:03:41.904812+0000","last_clean":"2024-04-25T10:03:41.904812+0000","last_became_active":"2024-04-25T10:03:40.900034+0000","last_became_peered":"2024-04-25T10:03:40.900034+0000","last_unstale":"2024-04-25T10:03:41.904812+0000","last_undegraded":"2024-04-25T10:03:41.904812+0000","last_fullsized":"2024-04-25T10:03:41.904812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:51.736962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887377+0000","last_change":"2024-04-25T10:03:40.898985+0000","last_active":"2024-04-25T10:03:41.887377+0000","last_peered":"2024-04-25T10:03:41.887377+0000","last_clean":"2024-04-25T10:03:41.887377+0000","last_became_active":"2024-04-25T10:03:40.898366+0000","last_became_peered":"2024-04-25T10:03:40.898366+0000","last_unstale":"2024-04-25T10:03:41.887377+0000","last_undegraded":"2024-04-25T10:03:41.887377+0000","last_fullsized":"2024-04-25T10:03:41.887377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:16.559780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.906155+0000","last_change":"2024-04-25T10:03:40.898020+0000","last_active":"2024-04-25T10:03:40.906155+0000","last_peered":"2024-04-25T10:03:40.906155+0000","last_clean":"2024-04-25T10:03:40.906155+0000","last_became_active":"2024-04-25T10:03:40.897715+0000","last_became_peered":"2024-04-25T10:03:40.897715+0000","last_unstale":"2024-04-25T10:03:40.906155+0000","last_undegraded":"2024-04-25T10:03:40.906155+0000","last_fullsized":"2024-04-25T10:03:40.906155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905777+0000","last_change":"2024-04-25T10:03:40.899893+0000","last_active":"2024-04-25T10:03:41.905777+0000","last_peered":"2024-04-25T10:03:41.905777+0000","last_clean":"2024-04-25T10:03:41.905777+0000","last_became_active":"2024-04-25T10:03:40.898809+0000","last_became_peered":"2024-04-25T10:03:40.898809+0000","last_unstale":"2024-04-25T10:03:41.905777+0000","last_undegraded":"2024-04-25T10:03:41.905777+0000","last_fullsized":"2024-04-25T10:03:41.905777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:40.244006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905749+0000","last_change":"2024-04-25T10:03:40.901034+0000","last_active":"2024-04-25T10:03:41.905749+0000","last_peered":"2024-04-25T10:03:41.905749+0000","last_clean":"2024-04-25T10:03:41.905749+0000","last_became_active":"2024-04-25T10:03:40.898164+0000","last_became_peered":"2024-04-25T10:03:40.898164+0000","last_unstale":"2024-04-25T10:03:41.905749+0000","last_undegraded":"2024-04-25T10:03:41.905749+0000","last_fullsized":"2024-04-25T10:03:41.905749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:47.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.906375+0000","last_change":"2024-04-25T10:03:40.896092+0000","last_active":"2024-04-25T10:03:40.906375+0000","last_peered":"2024-04-25T10:03:40.906375+0000","last_clean":"2024-04-25T10:03:40.906375+0000","last_became_active":"2024-04-25T10:03:40.895716+0000","last_became_peered":"2024-04-25T10:03:40.895716+0000","last_unstale":"2024-04-25T10:03:40.906375+0000","last_undegraded":"2024-04-25T10:03:40.906375+0000","last_fullsized":"2024-04-25T10:03:40.906375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905291+0000","last_change":"2024-04-25T10:03:40.899368+0000","last_active":"2024-04-25T10:03:41.905291+0000","last_peered":"2024-04-25T10:03:41.905291+0000","last_clean":"2024-04-25T10:03:41.905291+0000","last_became_active":"2024-04-25T10:03:40.899018+0000","last_became_peered":"2024-04-25T10:03:40.899018+0000","last_unstale":"2024-04-25T10:03:41.905291+0000","last_undegraded":"2024-04-25T10:03:41.905291+0000","last_fullsized":"2024-04-25T10:03:41.905291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:15.814576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893008+0000","last_change":"2024-04-25T10:03:40.894053+0000","last_active":"2024-04-25T10:03:41.893008+0000","last_peered":"2024-04-25T10:03:41.893008+0000","last_clean":"2024-04-25T10:03:41.893008+0000","last_became_active":"2024-04-25T10:03:40.893440+0000","last_became_peered":"2024-04-25T10:03:40.893440+0000","last_unstale":"2024-04-25T10:03:41.893008+0000","last_undegraded":"2024-04-25T10:03:41.893008+0000","last_fullsized":"2024-04-25T10:03:41.893008+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.904864+0000","last_change":"2024-04-25T10:03:40.903659+0000","last_active":"2024-04-25T10:03:41.904864+0000","last_peered":"2024-04-25T10:03:41.904864+0000","last_clean":"2024-04-25T10:03:41.904864+0000","last_became_active":"2024-04-25T10:03:40.903228+0000","last_became_peered":"2024-04-25T10:03:40.903228+0000","last_unstale":"2024-04-25T10:03:41.904864+0000","last_undegraded":"2024-04-25T10:03:41.904864+0000","last_fullsized":"2024-04-25T10:03:41.904864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:30.413499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.901548+0000","last_change":"2024-04-25T10:03:40.897335+0000","last_active":"2024-04-25T10:03:40.901548+0000","last_peered":"2024-04-25T10:03:40.901548+0000","last_clean":"2024-04-25T10:03:40.901548+0000","last_became_active":"2024-04-25T10:03:40.894037+0000","last_became_peered":"2024-04-25T10:03:40.894037+0000","last_unstale":"2024-04-25T10:03:40.901548+0000","last_undegraded":"2024-04-25T10:03:40.901548+0000","last_fullsized":"2024-04-25T10:03:40.901548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:43.477459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.902444+0000","last_change":"2024-04-25T10:03:40.898377+0000","last_active":"2024-04-25T10:03:40.902444+0000","last_peered":"2024-04-25T10:03:40.902444+0000","last_clean":"2024-04-25T10:03:40.902444+0000","last_became_active":"2024-04-25T10:03:40.897981+0000","last_became_peered":"2024-04-25T10:03:40.897981+0000","last_unstale":"2024-04-25T10:03:40.902444+0000","last_undegraded":"2024-04-25T10:03:40.902444+0000","last_fullsized":"2024-04-25T10:03:40.902444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:20.279938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892698+0000","last_change":"2024-04-25T10:03:40.901119+0000","last_active":"2024-04-25T10:03:41.892698+0000","last_peered":"2024-04-25T10:03:41.892698+0000","last_clean":"2024-04-25T10:03:41.892698+0000","last_became_active":"2024-04-25T10:03:40.900833+0000","last_became_peered":"2024-04-25T10:03:40.900833+0000","last_unstale":"2024-04-25T10:03:41.892698+0000","last_undegraded":"2024-04-25T10:03:41.892698+0000","last_fullsized":"2024-04-25T10:03:41.892698+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899236+0000","last_change":"2024-04-25T10:03:40.889075+0000","last_active":"2024-04-25T10:03:41.899236+0000","last_peered":"2024-04-25T10:03:41.899236+0000","last_clean":"2024-04-25T10:03:41.899236+0000","last_became_active":"2024-04-25T10:03:40.883420+0000","last_became_peered":"2024-04-25T10:03:40.883420+0000","last_unstale":"2024-04-25T10:03:41.899236+0000","last_undegraded":"2024-04-25T10:03:41.899236+0000","last_fullsized":"2024-04-25T10:03:41.899236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:43.218058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892130+0000","last_change":"2024-04-25T10:03:40.899796+0000","last_active":"2024-04-25T10:03:41.892130+0000","last_peered":"2024-04-25T10:03:41.892130+0000","last_clean":"2024-04-25T10:03:41.892130+0000","last_became_active":"2024-04-25T10:03:40.894608+0000","last_became_peered":"2024-04-25T10:03:40.894608+0000","last_unstale":"2024-04-25T10:03:41.892130+0000","last_undegraded":"2024-04-25T10:03:41.892130+0000","last_fullsized":"2024-04-25T10:03:41.892130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:32.540901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905357+0000","last_change":"2024-04-25T10:03:40.900719+0000","last_active":"2024-04-25T10:03:41.905357+0000","last_peered":"2024-04-25T10:03:41.905357+0000","last_clean":"2024-04-25T10:03:41.905357+0000","last_became_active":"2024-04-25T10:03:40.899725+0000","last_became_peered":"2024-04-25T10:03:40.899725+0000","last_unstale":"2024-04-25T10:03:41.905357+0000","last_undegraded":"2024-04-25T10:03:41.905357+0000","last_fullsized":"2024-04-25T10:03:41.905357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:59.873433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893723+0000","last_change":"2024-04-25T10:03:40.901138+0000","last_active":"2024-04-25T10:03:41.893723+0000","last_peered":"2024-04-25T10:03:41.893723+0000","last_clean":"2024-04-25T10:03:41.893723+0000","last_became_active":"2024-04-25T10:03:40.899231+0000","last_became_peered":"2024-04-25T10:03:40.899231+0000","last_unstale":"2024-04-25T10:03:41.893723+0000","last_undegraded":"2024-04-25T10:03:41.893723+0000","last_fullsized":"2024-04-25T10:03:41.893723+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:43.477459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.892707+0000","last_change":"2024-04-25T10:03:40.888030+0000","last_active":"2024-04-25T10:03:40.892707+0000","last_peered":"2024-04-25T10:03:40.892707+0000","last_clean":"2024-04-25T10:03:40.892707+0000","last_became_active":"2024-04-25T10:03:40.887217+0000","last_became_peered":"2024-04-25T10:03:40.887217+0000","last_unstale":"2024-04-25T10:03:40.892707+0000","last_undegraded":"2024-04-25T10:03:40.892707+0000","last_fullsized":"2024-04-25T10:03:40.892707+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899289+0000","last_change":"2024-04-25T10:03:40.878731+0000","last_active":"2024-04-25T10:03:41.899289+0000","last_peered":"2024-04-25T10:03:41.899289+0000","last_clean":"2024-04-25T10:03:41.899289+0000","last_became_active":"2024-04-25T10:03:40.878181+0000","last_became_peered":"2024-04-25T10:03:40.878181+0000","last_unstale":"2024-04-25T10:03:41.899289+0000","last_undegraded":"2024-04-25T10:03:41.899289+0000","last_fullsized":"2024-04-25T10:03:41.899289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:47.963110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892173+0000","last_change":"2024-04-25T10:03:40.892717+0000","last_active":"2024-04-25T10:03:41.892173+0000","last_peered":"2024-04-25T10:03:41.892173+0000","last_clean":"2024-04-25T10:03:41.892173+0000","last_became_active":"2024-04-25T10:03:40.892110+0000","last_became_peered":"2024-04-25T10:03:40.892110+0000","last_unstale":"2024-04-25T10:03:41.892173+0000","last_undegraded":"2024-04-25T10:03:41.892173+0000","last_fullsized":"2024-04-25T10:03:41.892173+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.831483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903600+0000","last_change":"2024-04-25T10:03:40.899915+0000","last_active":"2024-04-25T10:03:40.903600+0000","last_peered":"2024-04-25T10:03:40.903600+0000","last_clean":"2024-04-25T10:03:40.903600+0000","last_became_active":"2024-04-25T10:03:40.899215+0000","last_became_peered":"2024-04-25T10:03:40.899215+0000","last_unstale":"2024-04-25T10:03:40.903600+0000","last_undegraded":"2024-04-25T10:03:40.903600+0000","last_fullsized":"2024-04-25T10:03:40.903600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893046+0000","last_change":"2024-04-25T10:03:40.878836+0000","last_active":"2024-04-25T10:03:41.893046+0000","last_peered":"2024-04-25T10:03:41.893046+0000","last_clean":"2024-04-25T10:03:41.893046+0000","last_became_active":"2024-04-25T10:03:40.878501+0000","last_became_peered":"2024-04-25T10:03:40.878501+0000","last_unstale":"2024-04-25T10:03:41.893046+0000","last_undegraded":"2024-04-25T10:03:41.893046+0000","last_fullsized":"2024-04-25T10:03:41.893046+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905681+0000","last_change":"2024-04-25T10:03:40.899447+0000","last_active":"2024-04-25T10:03:41.905681+0000","last_peered":"2024-04-25T10:03:41.905681+0000","last_clean":"2024-04-25T10:03:41.905681+0000","last_became_active":"2024-04-25T10:03:40.899091+0000","last_became_peered":"2024-04-25T10:03:40.899091+0000","last_unstale":"2024-04-25T10:03:41.905681+0000","last_undegraded":"2024-04-25T10:03:41.905681+0000","last_fullsized":"2024-04-25T10:03:41.905681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:08.179803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892364+0000","last_change":"2024-04-25T10:03:40.885055+0000","last_active":"2024-04-25T10:03:41.892364+0000","last_peered":"2024-04-25T10:03:41.892364+0000","last_clean":"2024-04-25T10:03:41.892364+0000","last_became_active":"2024-04-25T10:03:40.884721+0000","last_became_peered":"2024-04-25T10:03:40.884721+0000","last_unstale":"2024-04-25T10:03:41.892364+0000","last_undegraded":"2024-04-25T10:03:41.892364+0000","last_fullsized":"2024-04-25T10:03:41.892364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892636+0000","last_change":"2024-04-25T10:03:40.883369+0000","last_active":"2024-04-25T10:03:41.892636+0000","last_peered":"2024-04-25T10:03:41.892636+0000","last_clean":"2024-04-25T10:03:41.892636+0000","last_became_active":"2024-04-25T10:03:40.882725+0000","last_became_peered":"2024-04-25T10:03:40.882725+0000","last_unstale":"2024-04-25T10:03:41.892636+0000","last_undegraded":"2024-04-25T10:03:41.892636+0000","last_fullsized":"2024-04-25T10:03:41.892636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.892841+0000","last_change":"2024-04-25T10:03:40.888117+0000","last_active":"2024-04-25T10:03:40.892841+0000","last_peered":"2024-04-25T10:03:40.892841+0000","last_clean":"2024-04-25T10:03:40.892841+0000","last_became_active":"2024-04-25T10:03:40.887646+0000","last_became_peered":"2024-04-25T10:03:40.887646+0000","last_unstale":"2024-04-25T10:03:40.892841+0000","last_undegraded":"2024-04-25T10:03:40.892841+0000","last_fullsized":"2024-04-25T10:03:40.892841+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.895986+0000","last_change":"2024-04-25T10:03:40.888431+0000","last_active":"2024-04-25T10:03:40.895986+0000","last_peered":"2024-04-25T10:03:40.895986+0000","last_clean":"2024-04-25T10:03:40.895986+0000","last_became_active":"2024-04-25T10:03:40.887936+0000","last_became_peered":"2024-04-25T10:03:40.887936+0000","last_unstale":"2024-04-25T10:03:40.895986+0000","last_undegraded":"2024-04-25T10:03:40.895986+0000","last_fullsized":"2024-04-25T10:03:40.895986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.902492+0000","last_change":"2024-04-25T10:03:40.899022+0000","last_active":"2024-04-25T10:03:40.902492+0000","last_peered":"2024-04-25T10:03:40.902492+0000","last_clean":"2024-04-25T10:03:40.902492+0000","last_became_active":"2024-04-25T10:03:40.898705+0000","last_became_peered":"2024-04-25T10:03:40.898705+0000","last_unstale":"2024-04-25T10:03:40.902492+0000","last_undegraded":"2024-04-25T10:03:40.902492+0000","last_fullsized":"2024-04-25T10:03:40.902492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.860339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892875+0000","last_change":"2024-04-25T10:03:39.864865+0000","last_active":"2024-04-25T10:03:41.892875+0000","last_peered":"2024-04-25T10:03:41.892875+0000","last_clean":"2024-04-25T10:03:41.892875+0000","last_became_active":"2024-04-25T10:03:39.864581+0000","last_became_peered":"2024-04-25T10:03:39.864581+0000","last_unstale":"2024-04-25T10:03:41.892875+0000","last_undegraded":"2024-04-25T10:03:41.892875+0000","last_fullsized":"2024-04-25T10:03:41.892875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:24.958966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903931+0000","last_change":"2024-04-25T10:03:40.899415+0000","last_active":"2024-04-25T10:03:40.903931+0000","last_peered":"2024-04-25T10:03:40.903931+0000","last_clean":"2024-04-25T10:03:40.903931+0000","last_became_active":"2024-04-25T10:03:40.899047+0000","last_became_peered":"2024-04-25T10:03:40.899047+0000","last_unstale":"2024-04-25T10:03:40.903931+0000","last_undegraded":"2024-04-25T10:03:40.903931+0000","last_fullsized":"2024-04-25T10:03:40.903931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.918125+0000","last_change":"2024-04-25T10:03:39.869670+0000","last_active":"2024-04-25T10:03:40.918125+0000","last_peered":"2024-04-25T10:03:40.918125+0000","last_clean":"2024-04-25T10:03:40.918125+0000","last_became_active":"2024-04-25T10:03:39.869358+0000","last_became_peered":"2024-04-25T10:03:39.869358+0000","last_unstale":"2024-04-25T10:03:40.918125+0000","last_undegraded":"2024-04-25T10:03:40.918125+0000","last_fullsized":"2024-04-25T10:03:40.918125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:05.756001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893513+0000","last_change":"2024-04-25T10:03:40.902854+0000","last_active":"2024-04-25T10:03:41.893513+0000","last_peered":"2024-04-25T10:03:41.893513+0000","last_clean":"2024-04-25T10:03:41.893513+0000","last_became_active":"2024-04-25T10:03:40.901884+0000","last_became_peered":"2024-04-25T10:03:40.901884+0000","last_unstale":"2024-04-25T10:03:41.893513+0000","last_undegraded":"2024-04-25T10:03:41.893513+0000","last_fullsized":"2024-04-25T10:03:41.893513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:26.483271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.907476+0000","last_change":"2024-04-25T10:03:39.862767+0000","last_active":"2024-04-25T10:03:40.907476+0000","last_peered":"2024-04-25T10:03:40.907476+0000","last_clean":"2024-04-25T10:03:40.907476+0000","last_became_active":"2024-04-25T10:03:39.862317+0000","last_became_peered":"2024-04-25T10:03:39.862317+0000","last_unstale":"2024-04-25T10:03:40.907476+0000","last_undegraded":"2024-04-25T10:03:40.907476+0000","last_fullsized":"2024-04-25T10:03:40.907476+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888344+0000","last_change":"2024-04-25T10:03:40.889570+0000","last_active":"2024-04-25T10:03:41.888344+0000","last_peered":"2024-04-25T10:03:41.888344+0000","last_clean":"2024-04-25T10:03:41.888344+0000","last_became_active":"2024-04-25T10:03:40.889298+0000","last_became_peered":"2024-04-25T10:03:40.889298+0000","last_unstale":"2024-04-25T10:03:41.888344+0000","last_undegraded":"2024-04-25T10:03:41.888344+0000","last_fullsized":"2024-04-25T10:03:41.888344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:40.244006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893528+0000","last_change":"2024-04-25T10:03:39.863483+0000","last_active":"2024-04-25T10:03:41.893528+0000","last_peered":"2024-04-25T10:03:41.893528+0000","last_clean":"2024-04-25T10:03:41.893528+0000","last_became_active":"2024-04-25T10:03:39.863047+0000","last_became_peered":"2024-04-25T10:03:39.863047+0000","last_unstale":"2024-04-25T10:03:41.893528+0000","last_undegraded":"2024-04-25T10:03:41.893528+0000","last_fullsized":"2024-04-25T10:03:41.893528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:10.876888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.894093+0000","last_change":"2024-04-25T10:03:40.889146+0000","last_active":"2024-04-25T10:03:40.894093+0000","last_peered":"2024-04-25T10:03:40.894093+0000","last_clean":"2024-04-25T10:03:40.894093+0000","last_became_active":"2024-04-25T10:03:40.888873+0000","last_became_peered":"2024-04-25T10:03:40.888873+0000","last_unstale":"2024-04-25T10:03:40.894093+0000","last_undegraded":"2024-04-25T10:03:40.894093+0000","last_fullsized":"2024-04-25T10:03:40.894093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:32.540901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905636+0000","last_change":"2024-04-25T10:03:39.869417+0000","last_active":"2024-04-25T10:03:41.905636+0000","last_peered":"2024-04-25T10:03:41.905636+0000","last_clean":"2024-04-25T10:03:41.905636+0000","last_became_active":"2024-04-25T10:03:39.869040+0000","last_became_peered":"2024-04-25T10:03:39.869040+0000","last_unstale":"2024-04-25T10:03:41.905636+0000","last_undegraded":"2024-04-25T10:03:41.905636+0000","last_fullsized":"2024-04-25T10:03:41.905636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:12.821238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905687+0000","last_change":"2024-04-25T10:03:40.903324+0000","last_active":"2024-04-25T10:03:41.905687+0000","last_peered":"2024-04-25T10:03:41.905687+0000","last_clean":"2024-04-25T10:03:41.905687+0000","last_became_active":"2024-04-25T10:03:40.900440+0000","last_became_peered":"2024-04-25T10:03:40.900440+0000","last_unstale":"2024-04-25T10:03:41.905687+0000","last_undegraded":"2024-04-25T10:03:41.905687+0000","last_fullsized":"2024-04-25T10:03:41.905687+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:06.915964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.917813+0000","last_change":"2024-04-25T10:03:39.877608+0000","last_active":"2024-04-25T10:03:40.917813+0000","last_peered":"2024-04-25T10:03:40.917813+0000","last_clean":"2024-04-25T10:03:40.917813+0000","last_became_active":"2024-04-25T10:03:39.876902+0000","last_became_peered":"2024-04-25T10:03:39.876902+0000","last_unstale":"2024-04-25T10:03:40.917813+0000","last_undegraded":"2024-04-25T10:03:40.917813+0000","last_fullsized":"2024-04-25T10:03:40.917813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893771+0000","last_change":"2024-04-25T10:03:40.894117+0000","last_active":"2024-04-25T10:03:41.893771+0000","last_peered":"2024-04-25T10:03:41.893771+0000","last_clean":"2024-04-25T10:03:41.893771+0000","last_became_active":"2024-04-25T10:03:40.893814+0000","last_became_peered":"2024-04-25T10:03:40.893814+0000","last_unstale":"2024-04-25T10:03:41.893771+0000","last_undegraded":"2024-04-25T10:03:41.893771+0000","last_fullsized":"2024-04-25T10:03:41.893771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.860339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.879448+0000","last_change":"2024-04-25T10:03:39.870355+0000","last_active":"2024-04-25T10:03:40.879448+0000","last_peered":"2024-04-25T10:03:40.879448+0000","last_clean":"2024-04-25T10:03:40.879448+0000","last_became_active":"2024-04-25T10:03:39.870077+0000","last_became_peered":"2024-04-25T10:03:39.870077+0000","last_unstale":"2024-04-25T10:03:40.879448+0000","last_undegraded":"2024-04-25T10:03:40.879448+0000","last_fullsized":"2024-04-25T10:03:40.879448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887888+0000","last_change":"2024-04-25T10:03:40.911285+0000","last_active":"2024-04-25T10:03:41.887888+0000","last_peered":"2024-04-25T10:03:41.887888+0000","last_clean":"2024-04-25T10:03:41.887888+0000","last_became_active":"2024-04-25T10:03:40.902637+0000","last_became_peered":"2024-04-25T10:03:40.902637+0000","last_unstale":"2024-04-25T10:03:41.887888+0000","last_undegraded":"2024-04-25T10:03:41.887888+0000","last_fullsized":"2024-04-25T10:03:41.887888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:08.179803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892123+0000","last_change":"2024-04-25T10:03:39.869809+0000","last_active":"2024-04-25T10:03:41.892123+0000","last_peered":"2024-04-25T10:03:41.892123+0000","last_clean":"2024-04-25T10:03:41.892123+0000","last_became_active":"2024-04-25T10:03:39.869567+0000","last_became_peered":"2024-04-25T10:03:39.869567+0000","last_unstale":"2024-04-25T10:03:41.892123+0000","last_undegraded":"2024-04-25T10:03:41.892123+0000","last_fullsized":"2024-04-25T10:03:41.892123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899288+0000","last_change":"2024-04-25T10:03:40.894558+0000","last_active":"2024-04-25T10:03:41.899288+0000","last_peered":"2024-04-25T10:03:41.899288+0000","last_clean":"2024-04-25T10:03:41.899288+0000","last_became_active":"2024-04-25T10:03:40.894004+0000","last_became_peered":"2024-04-25T10:03:40.894004+0000","last_unstale":"2024-04-25T10:03:41.899288+0000","last_undegraded":"2024-04-25T10:03:41.899288+0000","last_fullsized":"2024-04-25T10:03:41.899288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:07.147847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888484+0000","last_change":"2024-04-25T10:03:39.867012+0000","last_active":"2024-04-25T10:03:41.888484+0000","last_peered":"2024-04-25T10:03:41.888484+0000","last_clean":"2024-04-25T10:03:41.888484+0000","last_became_active":"2024-04-25T10:03:39.866623+0000","last_became_peered":"2024-04-25T10:03:39.866623+0000","last_unstale":"2024-04-25T10:03:41.888484+0000","last_undegraded":"2024-04-25T10:03:41.888484+0000","last_fullsized":"2024-04-25T10:03:41.888484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.463505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.891631+0000","last_change":"2024-04-25T10:03:40.884604+0000","last_active":"2024-04-25T10:03:40.891631+0000","last_peered":"2024-04-25T10:03:40.891631+0000","last_clean":"2024-04-25T10:03:40.891631+0000","last_became_active":"2024-04-25T10:03:40.884019+0000","last_became_peered":"2024-04-25T10:03:40.884019+0000","last_unstale":"2024-04-25T10:03:40.891631+0000","last_undegraded":"2024-04-25T10:03:40.891631+0000","last_fullsized":"2024-04-25T10:03:40.891631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906040+0000","last_change":"2024-04-25T10:03:39.869902+0000","last_active":"2024-04-25T10:03:41.906040+0000","last_peered":"2024-04-25T10:03:41.906040+0000","last_clean":"2024-04-25T10:03:41.906040+0000","last_became_active":"2024-04-25T10:03:39.869473+0000","last_became_peered":"2024-04-25T10:03:39.869473+0000","last_unstale":"2024-04-25T10:03:41.906040+0000","last_undegraded":"2024-04-25T10:03:41.906040+0000","last_fullsized":"2024-04-25T10:03:41.906040+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:44.056597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.894986+0000","last_change":"2024-04-25T10:03:40.890150+0000","last_active":"2024-04-25T10:03:40.894986+0000","last_peered":"2024-04-25T10:03:40.894986+0000","last_clean":"2024-04-25T10:03:40.894986+0000","last_became_active":"2024-04-25T10:03:40.889885+0000","last_became_peered":"2024-04-25T10:03:40.889885+0000","last_unstale":"2024-04-25T10:03:40.894986+0000","last_undegraded":"2024-04-25T10:03:40.894986+0000","last_fullsized":"2024-04-25T10:03:40.894986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.831483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906234+0000","last_change":"2024-04-25T10:03:39.871342+0000","last_active":"2024-04-25T10:03:41.906234+0000","last_peered":"2024-04-25T10:03:41.906234+0000","last_clean":"2024-04-25T10:03:41.906234+0000","last_became_active":"2024-04-25T10:03:39.870851+0000","last_became_peered":"2024-04-25T10:03:39.870851+0000","last_unstale":"2024-04-25T10:03:41.906234+0000","last_undegraded":"2024-04-25T10:03:41.906234+0000","last_fullsized":"2024-04-25T10:03:41.906234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893718+0000","last_change":"2024-04-25T10:03:40.897889+0000","last_active":"2024-04-25T10:03:41.893718+0000","last_peered":"2024-04-25T10:03:41.893718+0000","last_clean":"2024-04-25T10:03:41.893718+0000","last_became_active":"2024-04-25T10:03:40.897617+0000","last_became_peered":"2024-04-25T10:03:40.897617+0000","last_unstale":"2024-04-25T10:03:41.893718+0000","last_undegraded":"2024-04-25T10:03:41.893718+0000","last_fullsized":"2024-04-25T10:03:41.893718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:20.279938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888199+0000","last_change":"2024-04-25T10:03:39.869052+0000","last_active":"2024-04-25T10:03:41.888199+0000","last_peered":"2024-04-25T10:03:41.888199+0000","last_clean":"2024-04-25T10:03:41.888199+0000","last_became_active":"2024-04-25T10:03:39.867982+0000","last_became_peered":"2024-04-25T10:03:39.867982+0000","last_unstale":"2024-04-25T10:03:41.888199+0000","last_undegraded":"2024-04-25T10:03:41.888199+0000","last_fullsized":"2024-04-25T10:03:41.888199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:19.265985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903161+0000","last_change":"2024-04-25T10:03:40.897645+0000","last_active":"2024-04-25T10:03:40.903161+0000","last_peered":"2024-04-25T10:03:40.903161+0000","last_clean":"2024-04-25T10:03:40.903161+0000","last_became_active":"2024-04-25T10:03:40.897351+0000","last_became_peered":"2024-04-25T10:03:40.897351+0000","last_unstale":"2024-04-25T10:03:40.903161+0000","last_undegraded":"2024-04-25T10:03:40.903161+0000","last_fullsized":"2024-04-25T10:03:40.903161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888489+0000","last_change":"2024-04-25T10:03:39.865211+0000","last_active":"2024-04-25T10:03:41.888489+0000","last_peered":"2024-04-25T10:03:41.888489+0000","last_clean":"2024-04-25T10:03:41.888489+0000","last_became_active":"2024-04-25T10:03:39.864951+0000","last_became_peered":"2024-04-25T10:03:39.864951+0000","last_unstale":"2024-04-25T10:03:41.888489+0000","last_undegraded":"2024-04-25T10:03:41.888489+0000","last_fullsized":"2024-04-25T10:03:41.888489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:48.846386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887599+0000","last_change":"2024-04-25T10:03:40.912706+0000","last_active":"2024-04-25T10:03:41.887599+0000","last_peered":"2024-04-25T10:03:41.887599+0000","last_clean":"2024-04-25T10:03:41.887599+0000","last_became_active":"2024-04-25T10:03:40.912377+0000","last_became_peered":"2024-04-25T10:03:40.912377+0000","last_unstale":"2024-04-25T10:03:41.887599+0000","last_undegraded":"2024-04-25T10:03:41.887599+0000","last_fullsized":"2024-04-25T10:03:41.887599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:06.001065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.878699+0000","last_change":"2024-04-25T10:03:39.870564+0000","last_active":"2024-04-25T10:03:40.878699+0000","last_peered":"2024-04-25T10:03:40.878699+0000","last_clean":"2024-04-25T10:03:40.878699+0000","last_became_active":"2024-04-25T10:03:39.870211+0000","last_became_peered":"2024-04-25T10:03:39.870211+0000","last_unstale":"2024-04-25T10:03:40.878699+0000","last_undegraded":"2024-04-25T10:03:40.878699+0000","last_fullsized":"2024-04-25T10:03:40.878699+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:30.626295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.892884+0000","last_change":"2024-04-25T10:03:40.888109+0000","last_active":"2024-04-25T10:03:40.892884+0000","last_peered":"2024-04-25T10:03:40.892884+0000","last_clean":"2024-04-25T10:03:40.892884+0000","last_became_active":"2024-04-25T10:03:40.887787+0000","last_became_peered":"2024-04-25T10:03:40.887787+0000","last_unstale":"2024-04-25T10:03:40.892884+0000","last_undegraded":"2024-04-25T10:03:40.892884+0000","last_fullsized":"2024-04-25T10:03:40.892884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.891308+0000","last_change":"2024-04-25T10:03:39.864186+0000","last_active":"2024-04-25T10:03:41.891308+0000","last_peered":"2024-04-25T10:03:41.891308+0000","last_clean":"2024-04-25T10:03:41.891308+0000","last_became_active":"2024-04-25T10:03:39.863831+0000","last_became_peered":"2024-04-25T10:03:39.863831+0000","last_unstale":"2024-04-25T10:03:41.891308+0000","last_undegraded":"2024-04-25T10:03:41.891308+0000","last_fullsized":"2024-04-25T10:03:41.891308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:28.999175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888120+0000","last_change":"2024-04-25T10:03:40.912658+0000","last_active":"2024-04-25T10:03:41.888120+0000","last_peered":"2024-04-25T10:03:41.888120+0000","last_clean":"2024-04-25T10:03:41.888120+0000","last_became_active":"2024-04-25T10:03:40.912395+0000","last_became_peered":"2024-04-25T10:03:40.912395+0000","last_unstale":"2024-04-25T10:03:41.888120+0000","last_undegraded":"2024-04-25T10:03:41.888120+0000","last_fullsized":"2024-04-25T10:03:41.888120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:06.915964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.918606+0000","last_change":"2024-04-25T10:03:39.877418+0000","last_active":"2024-04-25T10:03:40.918606+0000","last_peered":"2024-04-25T10:03:40.918606+0000","last_clean":"2024-04-25T10:03:40.918606+0000","last_became_active":"2024-04-25T10:03:39.876987+0000","last_became_peered":"2024-04-25T10:03:39.876987+0000","last_unstale":"2024-04-25T10:03:40.918606+0000","last_undegraded":"2024-04-25T10:03:40.918606+0000","last_fullsized":"2024-04-25T10:03:40.918606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:30.626295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888305+0000","last_change":"2024-04-25T10:03:40.894469+0000","last_active":"2024-04-25T10:03:41.888305+0000","last_peered":"2024-04-25T10:03:41.888305+0000","last_clean":"2024-04-25T10:03:41.888305+0000","last_became_active":"2024-04-25T10:03:40.894141+0000","last_became_peered":"2024-04-25T10:03:40.894141+0000","last_unstale":"2024-04-25T10:03:41.888305+0000","last_undegraded":"2024-04-25T10:03:41.888305+0000","last_fullsized":"2024-04-25T10:03:41.888305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:47.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905737+0000","last_change":"2024-04-25T10:03:39.875439+0000","last_active":"2024-04-25T10:03:41.905737+0000","last_peered":"2024-04-25T10:03:41.905737+0000","last_clean":"2024-04-25T10:03:41.905737+0000","last_became_active":"2024-04-25T10:03:39.875014+0000","last_became_peered":"2024-04-25T10:03:39.875014+0000","last_unstale":"2024-04-25T10:03:41.905737+0000","last_undegraded":"2024-04-25T10:03:41.905737+0000","last_fullsized":"2024-04-25T10:03:41.905737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:10.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906106+0000","last_change":"2024-04-25T10:03:40.903516+0000","last_active":"2024-04-25T10:03:41.906106+0000","last_peered":"2024-04-25T10:03:41.906106+0000","last_clean":"2024-04-25T10:03:41.906106+0000","last_became_active":"2024-04-25T10:03:40.902419+0000","last_became_peered":"2024-04-25T10:03:40.902419+0000","last_unstale":"2024-04-25T10:03:41.906106+0000","last_undegraded":"2024-04-25T10:03:41.906106+0000","last_fullsized":"2024-04-25T10:03:41.906106+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.859279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893800+0000","last_change":"2024-04-25T10:03:39.873987+0000","last_active":"2024-04-25T10:03:41.893800+0000","last_peered":"2024-04-25T10:03:41.893800+0000","last_clean":"2024-04-25T10:03:41.893800+0000","last_became_active":"2024-04-25T10:03:39.873693+0000","last_became_peered":"2024-04-25T10:03:39.873693+0000","last_unstale":"2024-04-25T10:03:41.893800+0000","last_undegraded":"2024-04-25T10:03:41.893800+0000","last_fullsized":"2024-04-25T10:03:41.893800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893070+0000","last_change":"2024-04-25T10:03:40.895114+0000","last_active":"2024-04-25T10:03:41.893070+0000","last_peered":"2024-04-25T10:03:41.893070+0000","last_clean":"2024-04-25T10:03:41.893070+0000","last_became_active":"2024-04-25T10:03:40.894833+0000","last_became_peered":"2024-04-25T10:03:40.894833+0000","last_unstale":"2024-04-25T10:03:41.893070+0000","last_undegraded":"2024-04-25T10:03:41.893070+0000","last_fullsized":"2024-04-25T10:03:41.893070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899271+0000","last_change":"2024-04-25T10:03:39.869576+0000","last_active":"2024-04-25T10:03:41.899271+0000","last_peered":"2024-04-25T10:03:41.899271+0000","last_clean":"2024-04-25T10:03:41.899271+0000","last_became_active":"2024-04-25T10:03:39.869232+0000","last_became_peered":"2024-04-25T10:03:39.869232+0000","last_unstale":"2024-04-25T10:03:41.899271+0000","last_undegraded":"2024-04-25T10:03:41.899271+0000","last_fullsized":"2024-04-25T10:03:41.899271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:22.416577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899579+0000","last_change":"2024-04-25T10:03:40.885179+0000","last_active":"2024-04-25T10:03:41.899579+0000","last_peered":"2024-04-25T10:03:41.899579+0000","last_clean":"2024-04-25T10:03:41.899579+0000","last_became_active":"2024-04-25T10:03:40.883917+0000","last_became_peered":"2024-04-25T10:03:40.883917+0000","last_unstale":"2024-04-25T10:03:41.899579+0000","last_undegraded":"2024-04-25T10:03:41.899579+0000","last_fullsized":"2024-04-25T10:03:41.899579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:25.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.880068+0000","last_change":"2024-04-25T10:03:39.862572+0000","last_active":"2024-04-25T10:03:40.880068+0000","last_peered":"2024-04-25T10:03:40.880068+0000","last_clean":"2024-04-25T10:03:40.880068+0000","last_became_active":"2024-04-25T10:03:39.862305+0000","last_became_peered":"2024-04-25T10:03:39.862305+0000","last_unstale":"2024-04-25T10:03:40.880068+0000","last_undegraded":"2024-04-25T10:03:40.880068+0000","last_fullsized":"2024-04-25T10:03:40.880068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.526948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.896630+0000","last_change":"2024-04-25T10:03:40.888980+0000","last_active":"2024-04-25T10:03:40.896630+0000","last_peered":"2024-04-25T10:03:40.896630+0000","last_clean":"2024-04-25T10:03:40.896630+0000","last_became_active":"2024-04-25T10:03:40.888568+0000","last_became_peered":"2024-04-25T10:03:40.888568+0000","last_unstale":"2024-04-25T10:03:40.896630+0000","last_undegraded":"2024-04-25T10:03:40.896630+0000","last_fullsized":"2024-04-25T10:03:40.896630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899078+0000","last_change":"2024-04-25T10:03:39.860759+0000","last_active":"2024-04-25T10:03:41.899078+0000","last_peered":"2024-04-25T10:03:41.899078+0000","last_clean":"2024-04-25T10:03:41.899078+0000","last_became_active":"2024-04-25T10:03:39.858548+0000","last_became_peered":"2024-04-25T10:03:39.858548+0000","last_unstale":"2024-04-25T10:03:41.899078+0000","last_undegraded":"2024-04-25T10:03:41.899078+0000","last_fullsized":"2024-04-25T10:03:41.899078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:20.225426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.901962+0000","last_change":"2024-04-25T10:03:40.893560+0000","last_active":"2024-04-25T10:03:40.901962+0000","last_peered":"2024-04-25T10:03:40.901962+0000","last_clean":"2024-04-25T10:03:40.901962+0000","last_became_active":"2024-04-25T10:03:40.892177+0000","last_became_peered":"2024-04-25T10:03:40.892177+0000","last_unstale":"2024-04-25T10:03:40.901962+0000","last_undegraded":"2024-04-25T10:03:40.901962+0000","last_fullsized":"2024-04-25T10:03:40.901962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899250+0000","last_change":"2024-04-25T10:03:39.862781+0000","last_active":"2024-04-25T10:03:41.899250+0000","last_peered":"2024-04-25T10:03:41.899250+0000","last_clean":"2024-04-25T10:03:41.899250+0000","last_became_active":"2024-04-25T10:03:39.862522+0000","last_became_peered":"2024-04-25T10:03:39.862522+0000","last_unstale":"2024-04-25T10:03:41.899250+0000","last_undegraded":"2024-04-25T10:03:41.899250+0000","last_fullsized":"2024-04-25T10:03:41.899250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:27.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905489+0000","last_change":"2024-04-25T10:03:40.895865+0000","last_active":"2024-04-25T10:03:41.905489+0000","last_peered":"2024-04-25T10:03:41.905489+0000","last_clean":"2024-04-25T10:03:41.905489+0000","last_became_active":"2024-04-25T10:03:40.895537+0000","last_became_peered":"2024-04-25T10:03:40.895537+0000","last_unstale":"2024-04-25T10:03:41.905489+0000","last_undegraded":"2024-04-25T10:03:41.905489+0000","last_fullsized":"2024-04-25T10:03:41.905489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:06.001065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.869714+0000","last_change":"2024-04-25T10:03:39.872887+0000","last_active":"2024-04-25T10:03:40.869714+0000","last_peered":"2024-04-25T10:03:40.869714+0000","last_clean":"2024-04-25T10:03:40.869714+0000","last_became_active":"2024-04-25T10:03:39.872476+0000","last_became_peered":"2024-04-25T10:03:39.872476+0000","last_unstale":"2024-04-25T10:03:40.869714+0000","last_undegraded":"2024-04-25T10:03:40.869714+0000","last_fullsized":"2024-04-25T10:03:40.869714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.526948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888343+0000","last_change":"2024-04-25T10:03:40.902919+0000","last_active":"2024-04-25T10:03:41.888343+0000","last_peered":"2024-04-25T10:03:41.888343+0000","last_clean":"2024-04-25T10:03:41.888343+0000","last_became_active":"2024-04-25T10:03:40.902243+0000","last_became_peered":"2024-04-25T10:03:40.902243+0000","last_unstale":"2024-04-25T10:03:41.888343+0000","last_undegraded":"2024-04-25T10:03:41.888343+0000","last_fullsized":"2024-04-25T10:03:41.888343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.859279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.898963+0000","last_change":"2024-04-25T10:03:39.859341+0000","last_active":"2024-04-25T10:03:41.898963+0000","last_peered":"2024-04-25T10:03:41.898963+0000","last_clean":"2024-04-25T10:03:41.898963+0000","last_became_active":"2024-04-25T10:03:39.859083+0000","last_became_peered":"2024-04-25T10:03:39.859083+0000","last_unstale":"2024-04-25T10:03:41.898963+0000","last_undegraded":"2024-04-25T10:03:41.898963+0000","last_fullsized":"2024-04-25T10:03:41.898963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:10.822877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903505+0000","last_change":"2024-04-25T10:03:40.899614+0000","last_active":"2024-04-25T10:03:40.903505+0000","last_peered":"2024-04-25T10:03:40.903505+0000","last_clean":"2024-04-25T10:03:40.903505+0000","last_became_active":"2024-04-25T10:03:40.898918+0000","last_became_peered":"2024-04-25T10:03:40.898918+0000","last_unstale":"2024-04-25T10:03:40.903505+0000","last_undegraded":"2024-04-25T10:03:40.903505+0000","last_fullsized":"2024-04-25T10:03:40.903505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906023+0000","last_change":"2024-04-25T10:03:39.873821+0000","last_active":"2024-04-25T10:03:41.906023+0000","last_peered":"2024-04-25T10:03:41.906023+0000","last_clean":"2024-04-25T10:03:41.906023+0000","last_became_active":"2024-04-25T10:03:39.873488+0000","last_became_peered":"2024-04-25T10:03:39.873488+0000","last_unstale":"2024-04-25T10:03:41.906023+0000","last_undegraded":"2024-04-25T10:03:41.906023+0000","last_fullsized":"2024-04-25T10:03:41.906023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:31.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.896948+0000","last_change":"2024-04-25T10:03:40.891772+0000","last_active":"2024-04-25T10:03:40.896948+0000","last_peered":"2024-04-25T10:03:40.896948+0000","last_clean":"2024-04-25T10:03:40.896948+0000","last_became_active":"2024-04-25T10:03:40.891432+0000","last_became_peered":"2024-04-25T10:03:40.891432+0000","last_unstale":"2024-04-25T10:03:40.896948+0000","last_undegraded":"2024-04-25T10:03:40.896948+0000","last_fullsized":"2024-04-25T10:03:40.896948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:26.483271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.903154+0000","last_change":"2024-04-25T10:03:39.873720+0000","last_active":"2024-04-25T10:03:41.903154+0000","last_peered":"2024-04-25T10:03:41.903154+0000","last_clean":"2024-04-25T10:03:41.903154+0000","last_became_active":"2024-04-25T10:03:39.873395+0000","last_became_peered":"2024-04-25T10:03:39.873395+0000","last_unstale":"2024-04-25T10:03:41.903154+0000","last_undegraded":"2024-04-25T10:03:41.903154+0000","last_fullsized":"2024-04-25T10:03:41.903154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:54.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888714+0000","last_change":"2024-04-25T10:03:40.896741+0000","last_active":"2024-04-25T10:03:41.888714+0000","last_peered":"2024-04-25T10:03:41.888714+0000","last_clean":"2024-04-25T10:03:41.888714+0000","last_became_active":"2024-04-25T10:03:40.896455+0000","last_became_peered":"2024-04-25T10:03:40.896455+0000","last_unstale":"2024-04-25T10:03:41.888714+0000","last_undegraded":"2024-04-25T10:03:41.888714+0000","last_fullsized":"2024-04-25T10:03:41.888714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:26.573521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.882922+0000","last_change":"2024-04-25T10:03:39.866920+0000","last_active":"2024-04-25T10:03:40.882922+0000","last_peered":"2024-04-25T10:03:40.882922+0000","last_clean":"2024-04-25T10:03:40.882922+0000","last_became_active":"2024-04-25T10:03:39.866592+0000","last_became_peered":"2024-04-25T10:03:39.866592+0000","last_unstale":"2024-04-25T10:03:40.882922+0000","last_undegraded":"2024-04-25T10:03:40.882922+0000","last_fullsized":"2024-04-25T10:03:40.882922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:30.626295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.914094+0000","last_change":"2024-04-25T10:03:40.909026+0000","last_active":"2024-04-25T10:03:40.914094+0000","last_peered":"2024-04-25T10:03:40.914094+0000","last_clean":"2024-04-25T10:03:40.914094+0000","last_became_active":"2024-04-25T10:03:40.908715+0000","last_became_peered":"2024-04-25T10:03:40.908715+0000","last_unstale":"2024-04-25T10:03:40.914094+0000","last_undegraded":"2024-04-25T10:03:40.914094+0000","last_fullsized":"2024-04-25T10:03:40.914094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.883823+0000","last_change":"2024-04-25T10:03:39.869498+0000","last_active":"2024-04-25T10:03:40.883823+0000","last_peered":"2024-04-25T10:03:40.883823+0000","last_clean":"2024-04-25T10:03:40.883823+0000","last_became_active":"2024-04-25T10:03:39.869221+0000","last_became_peered":"2024-04-25T10:03:39.869221+0000","last_unstale":"2024-04-25T10:03:40.883823+0000","last_undegraded":"2024-04-25T10:03:40.883823+0000","last_fullsized":"2024-04-25T10:03:40.883823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.895955+0000","last_change":"2024-04-25T10:03:40.890606+0000","last_active":"2024-04-25T10:03:40.895955+0000","last_peered":"2024-04-25T10:03:40.895955+0000","last_clean":"2024-04-25T10:03:40.895955+0000","last_became_active":"2024-04-25T10:03:40.890338+0000","last_became_peered":"2024-04-25T10:03:40.890338+0000","last_unstale":"2024-04-25T10:03:40.895955+0000","last_undegraded":"2024-04-25T10:03:40.895955+0000","last_fullsized":"2024-04-25T10:03:40.895955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892906+0000","last_change":"2024-04-25T10:03:39.865429+0000","last_active":"2024-04-25T10:03:41.892906+0000","last_peered":"2024-04-25T10:03:41.892906+0000","last_clean":"2024-04-25T10:03:41.892906+0000","last_became_active":"2024-04-25T10:03:39.865102+0000","last_became_peered":"2024-04-25T10:03:39.865102+0000","last_unstale":"2024-04-25T10:03:41.892906+0000","last_undegraded":"2024-04-25T10:03:41.892906+0000","last_fullsized":"2024-04-25T10:03:41.892906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:46.949157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893231+0000","last_change":"2024-04-25T10:03:40.893459+0000","last_active":"2024-04-25T10:03:41.893231+0000","last_peered":"2024-04-25T10:03:41.893231+0000","last_clean":"2024-04-25T10:03:41.893231+0000","last_became_active":"2024-04-25T10:03:40.892510+0000","last_became_peered":"2024-04-25T10:03:40.892510+0000","last_unstale":"2024-04-25T10:03:41.893231+0000","last_undegraded":"2024-04-25T10:03:41.893231+0000","last_fullsized":"2024-04-25T10:03:41.893231+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892593+0000","last_change":"2024-04-25T10:03:39.867195+0000","last_active":"2024-04-25T10:03:41.892593+0000","last_peered":"2024-04-25T10:03:41.892593+0000","last_clean":"2024-04-25T10:03:41.892593+0000","last_became_active":"2024-04-25T10:03:39.862857+0000","last_became_peered":"2024-04-25T10:03:39.862857+0000","last_unstale":"2024-04-25T10:03:41.892593+0000","last_undegraded":"2024-04-25T10:03:41.892593+0000","last_fullsized":"2024-04-25T10:03:41.892593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:20.557967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.896752+0000","last_change":"2024-04-25T10:03:40.890274+0000","last_active":"2024-04-25T10:03:40.896752+0000","last_peered":"2024-04-25T10:03:40.896752+0000","last_clean":"2024-04-25T10:03:40.896752+0000","last_became_active":"2024-04-25T10:03:40.889950+0000","last_became_peered":"2024-04-25T10:03:40.889950+0000","last_unstale":"2024-04-25T10:03:40.896752+0000","last_undegraded":"2024-04-25T10:03:40.896752+0000","last_fullsized":"2024-04-25T10:03:40.896752+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887383+0000","last_change":"2024-04-25T10:03:39.860569+0000","last_active":"2024-04-25T10:03:41.887383+0000","last_peered":"2024-04-25T10:03:41.887383+0000","last_clean":"2024-04-25T10:03:41.887383+0000","last_became_active":"2024-04-25T10:03:39.860301+0000","last_became_peered":"2024-04-25T10:03:39.860301+0000","last_unstale":"2024-04-25T10:03:41.887383+0000","last_undegraded":"2024-04-25T10:03:41.887383+0000","last_fullsized":"2024-04-25T10:03:41.887383+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:25.469318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905453+0000","last_change":"2024-04-25T10:03:40.913830+0000","last_active":"2024-04-25T10:03:41.905453+0000","last_peered":"2024-04-25T10:03:41.905453+0000","last_clean":"2024-04-25T10:03:41.905453+0000","last_became_active":"2024-04-25T10:03:40.913506+0000","last_became_peered":"2024-04-25T10:03:40.913506+0000","last_unstale":"2024-04-25T10:03:41.905453+0000","last_undegraded":"2024-04-25T10:03:41.905453+0000","last_fullsized":"2024-04-25T10:03:41.905453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:16.559780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.870172+0000","last_change":"2024-04-25T10:03:39.872650+0000","last_active":"2024-04-25T10:03:40.870172+0000","last_peered":"2024-04-25T10:03:40.870172+0000","last_clean":"2024-04-25T10:03:40.870172+0000","last_became_active":"2024-04-25T10:03:39.872192+0000","last_became_peered":"2024-04-25T10:03:39.872192+0000","last_unstale":"2024-04-25T10:03:40.870172+0000","last_undegraded":"2024-04-25T10:03:40.870172+0000","last_fullsized":"2024-04-25T10:03:40.870172+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.869142+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:03:40.869142+0000","last_peered":"2024-04-25T10:03:40.869142+0000","last_clean":"2024-04-25T10:03:40.869142+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:03:40.869142+0000","last_undegraded":"2024-04-25T10:03:40.869142+0000","last_fullsized":"2024-04-25T10:03:40.869142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":454656,"data_stored":190697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":189141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":779453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:45.296 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:03:45.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.530+0000 7f2337200640 1 Processor -- start 2024-04-25T10:03:45.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.530+0000 7f2337200640 1 -- start start 2024-04-25T10:03:45.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.531+0000 7f2337200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.531+0000 7f2337200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2338144240 con 0x7f23380cca10 2024-04-25T10:03:45.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.531+0000 7f2337200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233815e220 con 0x7f233806e6a0 2024-04-25T10:03:45.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.531+0000 7f2337200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f233815e500 con 0x7f2338143e60 2024-04-25T10:03:45.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.532+0000 7f2336800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f23380cca10 0x7f233815ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35078/0 (socket says 172.21.15.165:35078) 2024-04-25T10:03:45.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.532+0000 7f2336800640 1 -- 172.21.15.165:0/3865729504 learned_addr learned my addr 172.21.15.165:0/3865729504 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.532+0000 7f2335e00640 1 --2- 172.21.15.165:0/3865729504 >> v2:172.21.15.172:3300/0 conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.532+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1701708204 0 0) 0x7f2338144240 con 0x7f23380cca10 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.532+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f231c005730 con 0x7f23380cca10 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1050697075 0 0) 0x7f231c005730 con 0x7f23380cca10 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 >> v2:172.21.15.172:3300/0 conn(0x7f2338143e60 msgr2=0x7f233806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 --2- 172.21.15.165:0/3865729504 >> v2:172.21.15.172:3300/0 conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 >> v1:172.21.15.172:6789/0 conn(0x7f233806e6a0 legacy=0x7f23380cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233815e7e0 con 0x7f23380cca10 2024-04-25T10:03:45.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f23300019f0 con 0x7f23380cca10 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 >> v1:172.21.15.165:6789/0 conn(0x7f23380cca10 legacy=0x7f233815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f231c005bc0 0x7f231c005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c006a10 0x7f231c006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f231c0073d0 0x7f231c00dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233815e500 con 0x7f231c005bc0 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00e270 con 0x7f231c0073d0 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c00e470 con 0x7f231c006a10 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335e00640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f231c005bc0 0x7f231c005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2336800640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f231c0073d0 0x7f231c00dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f2335e00640 1 -- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c006a10 msgr2=0x7f231c006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.533+0000 7f232fe00640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c006a10 0x7f231c006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.534+0000 7f2335e00640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c006a10 0x7f231c006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.534+0000 7f2335e00640 1 -- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f231c0073d0 msgr2=0x7f231c00dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.534+0000 7f2335e00640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f231c0073d0 0x7f231c00dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.534+0000 7f2335e00640 1 -- 172.21.15.165:0/3865729504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c005730 con 0x7f231c005bc0 2024-04-25T10:03:45.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.534+0000 7f2336800640 1 -- 172.21.15.165:0/3865729504 reap_dead start 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2335e00640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f231c005bc0 0x7f231c005fe0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2328009ac0 tx=0x7f231c007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adbca87942c15d60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328036070 con 0x7f231c005bc0 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328031400 con 0x7f231c005bc0 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2335400640 1 -- 172.21.15.165:0/3865729504 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328009690 con 0x7f231c005bc0 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2337200640 1 -- 172.21.15.165:0/3865729504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f231c005bc0 msgr2=0x7f231c005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2337200640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f231c005bc0 0x7f231c005fe0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f2328009ac0 tx=0x7f231c007a40 comp rx=0 tx=0).stop 2024-04-25T10:03:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2337200640 1 -- 172.21.15.165:0/3865729504 shutdown_connections 2024-04-25T10:03:45.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2337200640 1 --2- 172.21.15.165:0/3865729504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f231c005bc0 0x7f231c005fe0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.535+0000 7f2337200640 1 -- 172.21.15.165:0/3865729504 >> 172.21.15.165:0/3865729504 conn(0x7f233806abe0 msgr2=0x7f233806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.536+0000 7f2337200640 1 -- 172.21.15.165:0/3865729504 shutdown_connections 2024-04-25T10:03:45.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.536+0000 7f2337200640 1 -- 172.21.15.165:0/3865729504 wait complete. 2024-04-25T10:03:45.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.537+0000 7f2337200640 1 Processor -- start 2024-04-25T10:03:45.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.537+0000 7f2337200640 1 -- start start 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.538+0000 7f2337200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c005bc0 0x7f233815a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.538+0000 7f2337200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 0x7f23380ca5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.538+0000 7f2337200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f233815bae0 0x7f233815bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.538+0000 7f2337200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23380b2eb0 con 0x7f23380ca180 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.538+0000 7f2337200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23380b2d30 con 0x7f233815bae0 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.538+0000 7f2337200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380cc300 con 0x7f231c005bc0 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 0x7f23380ca5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 0x7f23380ca5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39544/0 (socket says 172.21.15.165:39544) 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 -- 172.21.15.165:0/261885679 learned_addr learned my addr 172.21.15.165:0/261885679 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 -- 172.21.15.165:0/261885679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c005bc0 msgr2=0x7f233815a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c005bc0 0x7f233815a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 -- 172.21.15.165:0/261885679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f233815bae0 msgr2=0x7f233815bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f233815bae0 0x7f233815bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 -- 172.21.15.165:0/261885679 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233815a600 con 0x7f23380ca180 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.539+0000 7f232fe00640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 0x7f23380ca5e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f232000d430 tx=0x7f232000d8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.540+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2320018070 con 0x7f23380ca180 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.540+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2320003610 con 0x7f23380ca180 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.540+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23200039e0 con 0x7f23380ca180 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.540+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23380cac60 con 0x7f23380ca180 2024-04-25T10:03:45.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.540+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23380cb220 con 0x7f23380ca180 2024-04-25T10:03:45.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.541+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338073990 con 0x7f23380ca180 2024-04-25T10:03:45.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.542+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f232001b070 con 0x7f23380ca180 2024-04-25T10:03:45.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.543+0000 7f232e000640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:45.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.543+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f2320089c00 con 0x7f23380ca180 2024-04-25T10:03:45.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.544+0000 7f2335e00640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:45.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.544+0000 7f2335e00640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2328033000 tx=0x7f232802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:45.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.548+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2320058150 con 0x7f23380ca180 2024-04-25T10:03:45.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.739+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2338140750 con 0x7f231406b4d0 2024-04-25T10:03:45.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.759+0000 7f232e000640 1 -- 172.21.15.165:0/261885679 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251939 (secure 0 0 0) 0x7f232807e840 con 0x7f231406b4d0 2024-04-25T10:03:45.769 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:45.770 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:03:45.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 msgr2=0x7f231406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f2328033000 tx=0x7f232802e000 comp rx=0 tx=0).stop 2024-04-25T10:03:45.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 msgr2=0x7f23380ca5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:45.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 0x7f23380ca5e0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f232000d430 tx=0x7f232000d8f0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 shutdown_connections 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f233815bae0 0x7f233815bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23380ca180 0x7f23380ca5e0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 --2- 172.21.15.165:0/261885679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231c005bc0 0x7f233815a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 >> 172.21.15.165:0/261885679 conn(0x7f233806abe0 msgr2=0x7f2338070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 shutdown_connections 2024-04-25T10:03:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:45.764+0000 7f2337200640 1 -- 172.21.15.165:0/261885679 wait complete. 2024-04-25T10:03:45.789 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":89,"stamp":"2024-04-25T10:03:44.907324+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":134,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220828,"kb_used_data":4660,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754753892,"statfs":{"total":773094113280,"available":772867985408,"internally_reserved":0,"allocated":4771840,"data_stored":2704656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":221236786},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.070866"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888541+0000","last_change":"2024-04-25T10:03:40.912706+0000","last_active":"2024-04-25T10:03:41.888541+0000","last_peered":"2024-04-25T10:03:41.888541+0000","last_clean":"2024-04-25T10:03:41.888541+0000","last_became_active":"2024-04-25T10:03:40.912441+0000","last_became_peered":"2024-04-25T10:03:40.912441+0000","last_unstale":"2024-04-25T10:03:41.888541+0000","last_undegraded":"2024-04-25T10:03:41.888541+0000","last_fullsized":"2024-04-25T10:03:41.888541+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:19:39.559917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888689+0000","last_change":"2024-04-25T10:03:40.910560+0000","last_active":"2024-04-25T10:03:41.888689+0000","last_peered":"2024-04-25T10:03:41.888689+0000","last_clean":"2024-04-25T10:03:41.888689+0000","last_became_active":"2024-04-25T10:03:40.907119+0000","last_became_peered":"2024-04-25T10:03:40.907119+0000","last_unstale":"2024-04-25T10:03:41.888689+0000","last_undegraded":"2024-04-25T10:03:41.888689+0000","last_fullsized":"2024-04-25T10:03:41.888689+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:49.177756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.909758+0000","last_change":"2024-04-25T10:03:40.901864+0000","last_active":"2024-04-25T10:03:40.909758+0000","last_peered":"2024-04-25T10:03:40.909758+0000","last_clean":"2024-04-25T10:03:40.909758+0000","last_became_active":"2024-04-25T10:03:40.901551+0000","last_became_peered":"2024-04-25T10:03:40.901551+0000","last_unstale":"2024-04-25T10:03:40.909758+0000","last_undegraded":"2024-04-25T10:03:40.909758+0000","last_fullsized":"2024-04-25T10:03:40.909758+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.898944+0000","last_change":"2024-04-25T10:03:40.901144+0000","last_active":"2024-04-25T10:03:41.898944+0000","last_peered":"2024-04-25T10:03:41.898944+0000","last_clean":"2024-04-25T10:03:41.898944+0000","last_became_active":"2024-04-25T10:03:40.900853+0000","last_became_peered":"2024-04-25T10:03:40.900853+0000","last_unstale":"2024-04-25T10:03:41.898944+0000","last_undegraded":"2024-04-25T10:03:41.898944+0000","last_fullsized":"2024-04-25T10:03:41.898944+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:00.308840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899187+0000","last_change":"2024-04-25T10:03:40.889200+0000","last_active":"2024-04-25T10:03:41.899187+0000","last_peered":"2024-04-25T10:03:41.899187+0000","last_clean":"2024-04-25T10:03:41.899187+0000","last_became_active":"2024-04-25T10:03:40.888863+0000","last_became_peered":"2024-04-25T10:03:40.888863+0000","last_unstale":"2024-04-25T10:03:41.899187+0000","last_undegraded":"2024-04-25T10:03:41.899187+0000","last_fullsized":"2024-04-25T10:03:41.899187+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:49.111365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.897111+0000","last_change":"2024-04-25T10:03:40.894308+0000","last_active":"2024-04-25T10:03:40.897111+0000","last_peered":"2024-04-25T10:03:40.897111+0000","last_clean":"2024-04-25T10:03:40.897111+0000","last_became_active":"2024-04-25T10:03:40.893203+0000","last_became_peered":"2024-04-25T10:03:40.893203+0000","last_unstale":"2024-04-25T10:03:40.897111+0000","last_undegraded":"2024-04-25T10:03:40.897111+0000","last_fullsized":"2024-04-25T10:03:40.897111+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.904812+0000","last_change":"2024-04-25T10:03:40.900741+0000","last_active":"2024-04-25T10:03:41.904812+0000","last_peered":"2024-04-25T10:03:41.904812+0000","last_clean":"2024-04-25T10:03:41.904812+0000","last_became_active":"2024-04-25T10:03:40.900034+0000","last_became_peered":"2024-04-25T10:03:40.900034+0000","last_unstale":"2024-04-25T10:03:41.904812+0000","last_undegraded":"2024-04-25T10:03:41.904812+0000","last_fullsized":"2024-04-25T10:03:41.904812+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:51.736962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887377+0000","last_change":"2024-04-25T10:03:40.898985+0000","last_active":"2024-04-25T10:03:41.887377+0000","last_peered":"2024-04-25T10:03:41.887377+0000","last_clean":"2024-04-25T10:03:41.887377+0000","last_became_active":"2024-04-25T10:03:40.898366+0000","last_became_peered":"2024-04-25T10:03:40.898366+0000","last_unstale":"2024-04-25T10:03:41.887377+0000","last_undegraded":"2024-04-25T10:03:41.887377+0000","last_fullsized":"2024-04-25T10:03:41.887377+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:16.559780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.906155+0000","last_change":"2024-04-25T10:03:40.898020+0000","last_active":"2024-04-25T10:03:40.906155+0000","last_peered":"2024-04-25T10:03:40.906155+0000","last_clean":"2024-04-25T10:03:40.906155+0000","last_became_active":"2024-04-25T10:03:40.897715+0000","last_became_peered":"2024-04-25T10:03:40.897715+0000","last_unstale":"2024-04-25T10:03:40.906155+0000","last_undegraded":"2024-04-25T10:03:40.906155+0000","last_fullsized":"2024-04-25T10:03:40.906155+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905777+0000","last_change":"2024-04-25T10:03:40.899893+0000","last_active":"2024-04-25T10:03:41.905777+0000","last_peered":"2024-04-25T10:03:41.905777+0000","last_clean":"2024-04-25T10:03:41.905777+0000","last_became_active":"2024-04-25T10:03:40.898809+0000","last_became_peered":"2024-04-25T10:03:40.898809+0000","last_unstale":"2024-04-25T10:03:41.905777+0000","last_undegraded":"2024-04-25T10:03:41.905777+0000","last_fullsized":"2024-04-25T10:03:41.905777+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:40.244006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905749+0000","last_change":"2024-04-25T10:03:40.901034+0000","last_active":"2024-04-25T10:03:41.905749+0000","last_peered":"2024-04-25T10:03:41.905749+0000","last_clean":"2024-04-25T10:03:41.905749+0000","last_became_active":"2024-04-25T10:03:40.898164+0000","last_became_peered":"2024-04-25T10:03:40.898164+0000","last_unstale":"2024-04-25T10:03:41.905749+0000","last_undegraded":"2024-04-25T10:03:41.905749+0000","last_fullsized":"2024-04-25T10:03:41.905749+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:47.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.906375+0000","last_change":"2024-04-25T10:03:40.896092+0000","last_active":"2024-04-25T10:03:40.906375+0000","last_peered":"2024-04-25T10:03:40.906375+0000","last_clean":"2024-04-25T10:03:40.906375+0000","last_became_active":"2024-04-25T10:03:40.895716+0000","last_became_peered":"2024-04-25T10:03:40.895716+0000","last_unstale":"2024-04-25T10:03:40.906375+0000","last_undegraded":"2024-04-25T10:03:40.906375+0000","last_fullsized":"2024-04-25T10:03:40.906375+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905291+0000","last_change":"2024-04-25T10:03:40.899368+0000","last_active":"2024-04-25T10:03:41.905291+0000","last_peered":"2024-04-25T10:03:41.905291+0000","last_clean":"2024-04-25T10:03:41.905291+0000","last_became_active":"2024-04-25T10:03:40.899018+0000","last_became_peered":"2024-04-25T10:03:40.899018+0000","last_unstale":"2024-04-25T10:03:41.905291+0000","last_undegraded":"2024-04-25T10:03:41.905291+0000","last_fullsized":"2024-04-25T10:03:41.905291+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:15.814576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893008+0000","last_change":"2024-04-25T10:03:40.894053+0000","last_active":"2024-04-25T10:03:41.893008+0000","last_peered":"2024-04-25T10:03:41.893008+0000","last_clean":"2024-04-25T10:03:41.893008+0000","last_became_active":"2024-04-25T10:03:40.893440+0000","last_became_peered":"2024-04-25T10:03:40.893440+0000","last_unstale":"2024-04-25T10:03:41.893008+0000","last_undegraded":"2024-04-25T10:03:41.893008+0000","last_fullsized":"2024-04-25T10:03:41.893008+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.904864+0000","last_change":"2024-04-25T10:03:40.903659+0000","last_active":"2024-04-25T10:03:41.904864+0000","last_peered":"2024-04-25T10:03:41.904864+0000","last_clean":"2024-04-25T10:03:41.904864+0000","last_became_active":"2024-04-25T10:03:40.903228+0000","last_became_peered":"2024-04-25T10:03:40.903228+0000","last_unstale":"2024-04-25T10:03:41.904864+0000","last_undegraded":"2024-04-25T10:03:41.904864+0000","last_fullsized":"2024-04-25T10:03:41.904864+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:30.413499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.901548+0000","last_change":"2024-04-25T10:03:40.897335+0000","last_active":"2024-04-25T10:03:40.901548+0000","last_peered":"2024-04-25T10:03:40.901548+0000","last_clean":"2024-04-25T10:03:40.901548+0000","last_became_active":"2024-04-25T10:03:40.894037+0000","last_became_peered":"2024-04-25T10:03:40.894037+0000","last_unstale":"2024-04-25T10:03:40.901548+0000","last_undegraded":"2024-04-25T10:03:40.901548+0000","last_fullsized":"2024-04-25T10:03:40.901548+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:43.477459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.902444+0000","last_change":"2024-04-25T10:03:40.898377+0000","last_active":"2024-04-25T10:03:40.902444+0000","last_peered":"2024-04-25T10:03:40.902444+0000","last_clean":"2024-04-25T10:03:40.902444+0000","last_became_active":"2024-04-25T10:03:40.897981+0000","last_became_peered":"2024-04-25T10:03:40.897981+0000","last_unstale":"2024-04-25T10:03:40.902444+0000","last_undegraded":"2024-04-25T10:03:40.902444+0000","last_fullsized":"2024-04-25T10:03:40.902444+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:20.279938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892698+0000","last_change":"2024-04-25T10:03:40.901119+0000","last_active":"2024-04-25T10:03:41.892698+0000","last_peered":"2024-04-25T10:03:41.892698+0000","last_clean":"2024-04-25T10:03:41.892698+0000","last_became_active":"2024-04-25T10:03:40.900833+0000","last_became_peered":"2024-04-25T10:03:40.900833+0000","last_unstale":"2024-04-25T10:03:41.892698+0000","last_undegraded":"2024-04-25T10:03:41.892698+0000","last_fullsized":"2024-04-25T10:03:41.892698+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899236+0000","last_change":"2024-04-25T10:03:40.889075+0000","last_active":"2024-04-25T10:03:41.899236+0000","last_peered":"2024-04-25T10:03:41.899236+0000","last_clean":"2024-04-25T10:03:41.899236+0000","last_became_active":"2024-04-25T10:03:40.883420+0000","last_became_peered":"2024-04-25T10:03:40.883420+0000","last_unstale":"2024-04-25T10:03:41.899236+0000","last_undegraded":"2024-04-25T10:03:41.899236+0000","last_fullsized":"2024-04-25T10:03:41.899236+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:48:43.218058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892130+0000","last_change":"2024-04-25T10:03:40.899796+0000","last_active":"2024-04-25T10:03:41.892130+0000","last_peered":"2024-04-25T10:03:41.892130+0000","last_clean":"2024-04-25T10:03:41.892130+0000","last_became_active":"2024-04-25T10:03:40.894608+0000","last_became_peered":"2024-04-25T10:03:40.894608+0000","last_unstale":"2024-04-25T10:03:41.892130+0000","last_undegraded":"2024-04-25T10:03:41.892130+0000","last_fullsized":"2024-04-25T10:03:41.892130+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:32.540901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905357+0000","last_change":"2024-04-25T10:03:40.900719+0000","last_active":"2024-04-25T10:03:41.905357+0000","last_peered":"2024-04-25T10:03:41.905357+0000","last_clean":"2024-04-25T10:03:41.905357+0000","last_became_active":"2024-04-25T10:03:40.899725+0000","last_became_peered":"2024-04-25T10:03:40.899725+0000","last_unstale":"2024-04-25T10:03:41.905357+0000","last_undegraded":"2024-04-25T10:03:41.905357+0000","last_fullsized":"2024-04-25T10:03:41.905357+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:38:59.873433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893723+0000","last_change":"2024-04-25T10:03:40.901138+0000","last_active":"2024-04-25T10:03:41.893723+0000","last_peered":"2024-04-25T10:03:41.893723+0000","last_clean":"2024-04-25T10:03:41.893723+0000","last_became_active":"2024-04-25T10:03:40.899231+0000","last_became_peered":"2024-04-25T10:03:40.899231+0000","last_unstale":"2024-04-25T10:03:41.893723+0000","last_undegraded":"2024-04-25T10:03:41.893723+0000","last_fullsized":"2024-04-25T10:03:41.893723+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:43.477459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.892707+0000","last_change":"2024-04-25T10:03:40.888030+0000","last_active":"2024-04-25T10:03:40.892707+0000","last_peered":"2024-04-25T10:03:40.892707+0000","last_clean":"2024-04-25T10:03:40.892707+0000","last_became_active":"2024-04-25T10:03:40.887217+0000","last_became_peered":"2024-04-25T10:03:40.887217+0000","last_unstale":"2024-04-25T10:03:40.892707+0000","last_undegraded":"2024-04-25T10:03:40.892707+0000","last_fullsized":"2024-04-25T10:03:40.892707+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899289+0000","last_change":"2024-04-25T10:03:40.878731+0000","last_active":"2024-04-25T10:03:41.899289+0000","last_peered":"2024-04-25T10:03:41.899289+0000","last_clean":"2024-04-25T10:03:41.899289+0000","last_became_active":"2024-04-25T10:03:40.878181+0000","last_became_peered":"2024-04-25T10:03:40.878181+0000","last_unstale":"2024-04-25T10:03:41.899289+0000","last_undegraded":"2024-04-25T10:03:41.899289+0000","last_fullsized":"2024-04-25T10:03:41.899289+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:47.963110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892173+0000","last_change":"2024-04-25T10:03:40.892717+0000","last_active":"2024-04-25T10:03:41.892173+0000","last_peered":"2024-04-25T10:03:41.892173+0000","last_clean":"2024-04-25T10:03:41.892173+0000","last_became_active":"2024-04-25T10:03:40.892110+0000","last_became_peered":"2024-04-25T10:03:40.892110+0000","last_unstale":"2024-04-25T10:03:41.892173+0000","last_undegraded":"2024-04-25T10:03:41.892173+0000","last_fullsized":"2024-04-25T10:03:41.892173+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.831483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903600+0000","last_change":"2024-04-25T10:03:40.899915+0000","last_active":"2024-04-25T10:03:40.903600+0000","last_peered":"2024-04-25T10:03:40.903600+0000","last_clean":"2024-04-25T10:03:40.903600+0000","last_became_active":"2024-04-25T10:03:40.899215+0000","last_became_peered":"2024-04-25T10:03:40.899215+0000","last_unstale":"2024-04-25T10:03:40.903600+0000","last_undegraded":"2024-04-25T10:03:40.903600+0000","last_fullsized":"2024-04-25T10:03:40.903600+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893046+0000","last_change":"2024-04-25T10:03:40.878836+0000","last_active":"2024-04-25T10:03:41.893046+0000","last_peered":"2024-04-25T10:03:41.893046+0000","last_clean":"2024-04-25T10:03:41.893046+0000","last_became_active":"2024-04-25T10:03:40.878501+0000","last_became_peered":"2024-04-25T10:03:40.878501+0000","last_unstale":"2024-04-25T10:03:41.893046+0000","last_undegraded":"2024-04-25T10:03:41.893046+0000","last_fullsized":"2024-04-25T10:03:41.893046+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905681+0000","last_change":"2024-04-25T10:03:40.899447+0000","last_active":"2024-04-25T10:03:41.905681+0000","last_peered":"2024-04-25T10:03:41.905681+0000","last_clean":"2024-04-25T10:03:41.905681+0000","last_became_active":"2024-04-25T10:03:40.899091+0000","last_became_peered":"2024-04-25T10:03:40.899091+0000","last_unstale":"2024-04-25T10:03:41.905681+0000","last_undegraded":"2024-04-25T10:03:41.905681+0000","last_fullsized":"2024-04-25T10:03:41.905681+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:08.179803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892364+0000","last_change":"2024-04-25T10:03:40.885055+0000","last_active":"2024-04-25T10:03:41.892364+0000","last_peered":"2024-04-25T10:03:41.892364+0000","last_clean":"2024-04-25T10:03:41.892364+0000","last_became_active":"2024-04-25T10:03:40.884721+0000","last_became_peered":"2024-04-25T10:03:40.884721+0000","last_unstale":"2024-04-25T10:03:41.892364+0000","last_undegraded":"2024-04-25T10:03:41.892364+0000","last_fullsized":"2024-04-25T10:03:41.892364+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892636+0000","last_change":"2024-04-25T10:03:40.883369+0000","last_active":"2024-04-25T10:03:41.892636+0000","last_peered":"2024-04-25T10:03:41.892636+0000","last_clean":"2024-04-25T10:03:41.892636+0000","last_became_active":"2024-04-25T10:03:40.882725+0000","last_became_peered":"2024-04-25T10:03:40.882725+0000","last_unstale":"2024-04-25T10:03:41.892636+0000","last_undegraded":"2024-04-25T10:03:41.892636+0000","last_fullsized":"2024-04-25T10:03:41.892636+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.892841+0000","last_change":"2024-04-25T10:03:40.888117+0000","last_active":"2024-04-25T10:03:40.892841+0000","last_peered":"2024-04-25T10:03:40.892841+0000","last_clean":"2024-04-25T10:03:40.892841+0000","last_became_active":"2024-04-25T10:03:40.887646+0000","last_became_peered":"2024-04-25T10:03:40.887646+0000","last_unstale":"2024-04-25T10:03:40.892841+0000","last_undegraded":"2024-04-25T10:03:40.892841+0000","last_fullsized":"2024-04-25T10:03:40.892841+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.895986+0000","last_change":"2024-04-25T10:03:40.888431+0000","last_active":"2024-04-25T10:03:40.895986+0000","last_peered":"2024-04-25T10:03:40.895986+0000","last_clean":"2024-04-25T10:03:40.895986+0000","last_became_active":"2024-04-25T10:03:40.887936+0000","last_became_peered":"2024-04-25T10:03:40.887936+0000","last_unstale":"2024-04-25T10:03:40.895986+0000","last_undegraded":"2024-04-25T10:03:40.895986+0000","last_fullsized":"2024-04-25T10:03:40.895986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.902492+0000","last_change":"2024-04-25T10:03:40.899022+0000","last_active":"2024-04-25T10:03:40.902492+0000","last_peered":"2024-04-25T10:03:40.902492+0000","last_clean":"2024-04-25T10:03:40.902492+0000","last_became_active":"2024-04-25T10:03:40.898705+0000","last_became_peered":"2024-04-25T10:03:40.898705+0000","last_unstale":"2024-04-25T10:03:40.902492+0000","last_undegraded":"2024-04-25T10:03:40.902492+0000","last_fullsized":"2024-04-25T10:03:40.902492+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.860339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"38'4","reported_seq":18,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892875+0000","last_change":"2024-04-25T10:03:39.864865+0000","last_active":"2024-04-25T10:03:41.892875+0000","last_peered":"2024-04-25T10:03:41.892875+0000","last_clean":"2024-04-25T10:03:41.892875+0000","last_became_active":"2024-04-25T10:03:39.864581+0000","last_became_peered":"2024-04-25T10:03:39.864581+0000","last_unstale":"2024-04-25T10:03:41.892875+0000","last_undegraded":"2024-04-25T10:03:41.892875+0000","last_fullsized":"2024-04-25T10:03:41.892875+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:24.958966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903931+0000","last_change":"2024-04-25T10:03:40.899415+0000","last_active":"2024-04-25T10:03:40.903931+0000","last_peered":"2024-04-25T10:03:40.903931+0000","last_clean":"2024-04-25T10:03:40.903931+0000","last_became_active":"2024-04-25T10:03:40.899047+0000","last_became_peered":"2024-04-25T10:03:40.899047+0000","last_unstale":"2024-04-25T10:03:40.903931+0000","last_undegraded":"2024-04-25T10:03:40.903931+0000","last_fullsized":"2024-04-25T10:03:40.903931+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:45.070550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.918125+0000","last_change":"2024-04-25T10:03:39.869670+0000","last_active":"2024-04-25T10:03:40.918125+0000","last_peered":"2024-04-25T10:03:40.918125+0000","last_clean":"2024-04-25T10:03:40.918125+0000","last_became_active":"2024-04-25T10:03:39.869358+0000","last_became_peered":"2024-04-25T10:03:39.869358+0000","last_unstale":"2024-04-25T10:03:40.918125+0000","last_undegraded":"2024-04-25T10:03:40.918125+0000","last_fullsized":"2024-04-25T10:03:40.918125+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:05.756001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893513+0000","last_change":"2024-04-25T10:03:40.902854+0000","last_active":"2024-04-25T10:03:41.893513+0000","last_peered":"2024-04-25T10:03:41.893513+0000","last_clean":"2024-04-25T10:03:41.893513+0000","last_became_active":"2024-04-25T10:03:40.901884+0000","last_became_peered":"2024-04-25T10:03:40.901884+0000","last_unstale":"2024-04-25T10:03:41.893513+0000","last_undegraded":"2024-04-25T10:03:41.893513+0000","last_fullsized":"2024-04-25T10:03:41.893513+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:26.483271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"38'2","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.907476+0000","last_change":"2024-04-25T10:03:39.862767+0000","last_active":"2024-04-25T10:03:40.907476+0000","last_peered":"2024-04-25T10:03:40.907476+0000","last_clean":"2024-04-25T10:03:40.907476+0000","last_became_active":"2024-04-25T10:03:39.862317+0000","last_became_peered":"2024-04-25T10:03:39.862317+0000","last_unstale":"2024-04-25T10:03:40.907476+0000","last_undegraded":"2024-04-25T10:03:40.907476+0000","last_fullsized":"2024-04-25T10:03:40.907476+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888344+0000","last_change":"2024-04-25T10:03:40.889570+0000","last_active":"2024-04-25T10:03:41.888344+0000","last_peered":"2024-04-25T10:03:41.888344+0000","last_clean":"2024-04-25T10:03:41.888344+0000","last_became_active":"2024-04-25T10:03:40.889298+0000","last_became_peered":"2024-04-25T10:03:40.889298+0000","last_unstale":"2024-04-25T10:03:41.888344+0000","last_undegraded":"2024-04-25T10:03:41.888344+0000","last_fullsized":"2024-04-25T10:03:41.888344+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:09:40.244006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893528+0000","last_change":"2024-04-25T10:03:39.863483+0000","last_active":"2024-04-25T10:03:41.893528+0000","last_peered":"2024-04-25T10:03:41.893528+0000","last_clean":"2024-04-25T10:03:41.893528+0000","last_became_active":"2024-04-25T10:03:39.863047+0000","last_became_peered":"2024-04-25T10:03:39.863047+0000","last_unstale":"2024-04-25T10:03:41.893528+0000","last_undegraded":"2024-04-25T10:03:41.893528+0000","last_fullsized":"2024-04-25T10:03:41.893528+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:10.876888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.894093+0000","last_change":"2024-04-25T10:03:40.889146+0000","last_active":"2024-04-25T10:03:40.894093+0000","last_peered":"2024-04-25T10:03:40.894093+0000","last_clean":"2024-04-25T10:03:40.894093+0000","last_became_active":"2024-04-25T10:03:40.888873+0000","last_became_peered":"2024-04-25T10:03:40.888873+0000","last_unstale":"2024-04-25T10:03:40.894093+0000","last_undegraded":"2024-04-25T10:03:40.894093+0000","last_fullsized":"2024-04-25T10:03:40.894093+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:32.540901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905636+0000","last_change":"2024-04-25T10:03:39.869417+0000","last_active":"2024-04-25T10:03:41.905636+0000","last_peered":"2024-04-25T10:03:41.905636+0000","last_clean":"2024-04-25T10:03:41.905636+0000","last_became_active":"2024-04-25T10:03:39.869040+0000","last_became_peered":"2024-04-25T10:03:39.869040+0000","last_unstale":"2024-04-25T10:03:41.905636+0000","last_undegraded":"2024-04-25T10:03:41.905636+0000","last_fullsized":"2024-04-25T10:03:41.905636+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:12.821238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905687+0000","last_change":"2024-04-25T10:03:40.903324+0000","last_active":"2024-04-25T10:03:41.905687+0000","last_peered":"2024-04-25T10:03:41.905687+0000","last_clean":"2024-04-25T10:03:41.905687+0000","last_became_active":"2024-04-25T10:03:40.900440+0000","last_became_peered":"2024-04-25T10:03:40.900440+0000","last_unstale":"2024-04-25T10:03:41.905687+0000","last_undegraded":"2024-04-25T10:03:41.905687+0000","last_fullsized":"2024-04-25T10:03:41.905687+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:06.915964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"38'1","reported_seq":13,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.917813+0000","last_change":"2024-04-25T10:03:39.877608+0000","last_active":"2024-04-25T10:03:40.917813+0000","last_peered":"2024-04-25T10:03:40.917813+0000","last_clean":"2024-04-25T10:03:40.917813+0000","last_became_active":"2024-04-25T10:03:39.876902+0000","last_became_peered":"2024-04-25T10:03:39.876902+0000","last_unstale":"2024-04-25T10:03:40.917813+0000","last_undegraded":"2024-04-25T10:03:40.917813+0000","last_fullsized":"2024-04-25T10:03:40.917813+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893771+0000","last_change":"2024-04-25T10:03:40.894117+0000","last_active":"2024-04-25T10:03:41.893771+0000","last_peered":"2024-04-25T10:03:41.893771+0000","last_clean":"2024-04-25T10:03:41.893771+0000","last_became_active":"2024-04-25T10:03:40.893814+0000","last_became_peered":"2024-04-25T10:03:40.893814+0000","last_unstale":"2024-04-25T10:03:41.893771+0000","last_undegraded":"2024-04-25T10:03:41.893771+0000","last_fullsized":"2024-04-25T10:03:41.893771+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:49.860339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.879448+0000","last_change":"2024-04-25T10:03:39.870355+0000","last_active":"2024-04-25T10:03:40.879448+0000","last_peered":"2024-04-25T10:03:40.879448+0000","last_clean":"2024-04-25T10:03:40.879448+0000","last_became_active":"2024-04-25T10:03:39.870077+0000","last_became_peered":"2024-04-25T10:03:39.870077+0000","last_unstale":"2024-04-25T10:03:40.879448+0000","last_undegraded":"2024-04-25T10:03:40.879448+0000","last_fullsized":"2024-04-25T10:03:40.879448+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887888+0000","last_change":"2024-04-25T10:03:40.911285+0000","last_active":"2024-04-25T10:03:41.887888+0000","last_peered":"2024-04-25T10:03:41.887888+0000","last_clean":"2024-04-25T10:03:41.887888+0000","last_became_active":"2024-04-25T10:03:40.902637+0000","last_became_peered":"2024-04-25T10:03:40.902637+0000","last_unstale":"2024-04-25T10:03:41.887888+0000","last_undegraded":"2024-04-25T10:03:41.887888+0000","last_fullsized":"2024-04-25T10:03:41.887888+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:18:08.179803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892123+0000","last_change":"2024-04-25T10:03:39.869809+0000","last_active":"2024-04-25T10:03:41.892123+0000","last_peered":"2024-04-25T10:03:41.892123+0000","last_clean":"2024-04-25T10:03:41.892123+0000","last_became_active":"2024-04-25T10:03:39.869567+0000","last_became_peered":"2024-04-25T10:03:39.869567+0000","last_unstale":"2024-04-25T10:03:41.892123+0000","last_undegraded":"2024-04-25T10:03:41.892123+0000","last_fullsized":"2024-04-25T10:03:41.892123+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:29.168869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899288+0000","last_change":"2024-04-25T10:03:40.894558+0000","last_active":"2024-04-25T10:03:41.899288+0000","last_peered":"2024-04-25T10:03:41.899288+0000","last_clean":"2024-04-25T10:03:41.899288+0000","last_became_active":"2024-04-25T10:03:40.894004+0000","last_became_peered":"2024-04-25T10:03:40.894004+0000","last_unstale":"2024-04-25T10:03:41.899288+0000","last_undegraded":"2024-04-25T10:03:41.899288+0000","last_fullsized":"2024-04-25T10:03:41.899288+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:07.147847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888484+0000","last_change":"2024-04-25T10:03:39.867012+0000","last_active":"2024-04-25T10:03:41.888484+0000","last_peered":"2024-04-25T10:03:41.888484+0000","last_clean":"2024-04-25T10:03:41.888484+0000","last_became_active":"2024-04-25T10:03:39.866623+0000","last_became_peered":"2024-04-25T10:03:39.866623+0000","last_unstale":"2024-04-25T10:03:41.888484+0000","last_undegraded":"2024-04-25T10:03:41.888484+0000","last_fullsized":"2024-04-25T10:03:41.888484+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:55:42.463505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.891631+0000","last_change":"2024-04-25T10:03:40.884604+0000","last_active":"2024-04-25T10:03:40.891631+0000","last_peered":"2024-04-25T10:03:40.891631+0000","last_clean":"2024-04-25T10:03:40.891631+0000","last_became_active":"2024-04-25T10:03:40.884019+0000","last_became_peered":"2024-04-25T10:03:40.884019+0000","last_unstale":"2024-04-25T10:03:40.891631+0000","last_undegraded":"2024-04-25T10:03:40.891631+0000","last_fullsized":"2024-04-25T10:03:40.891631+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906040+0000","last_change":"2024-04-25T10:03:39.869902+0000","last_active":"2024-04-25T10:03:41.906040+0000","last_peered":"2024-04-25T10:03:41.906040+0000","last_clean":"2024-04-25T10:03:41.906040+0000","last_became_active":"2024-04-25T10:03:39.869473+0000","last_became_peered":"2024-04-25T10:03:39.869473+0000","last_unstale":"2024-04-25T10:03:41.906040+0000","last_undegraded":"2024-04-25T10:03:41.906040+0000","last_fullsized":"2024-04-25T10:03:41.906040+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:44.056597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.894986+0000","last_change":"2024-04-25T10:03:40.890150+0000","last_active":"2024-04-25T10:03:40.894986+0000","last_peered":"2024-04-25T10:03:40.894986+0000","last_clean":"2024-04-25T10:03:40.894986+0000","last_became_active":"2024-04-25T10:03:40.889885+0000","last_became_peered":"2024-04-25T10:03:40.889885+0000","last_unstale":"2024-04-25T10:03:40.894986+0000","last_undegraded":"2024-04-25T10:03:40.894986+0000","last_fullsized":"2024-04-25T10:03:40.894986+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:23.831483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"38'3","reported_seq":17,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906234+0000","last_change":"2024-04-25T10:03:39.871342+0000","last_active":"2024-04-25T10:03:41.906234+0000","last_peered":"2024-04-25T10:03:41.906234+0000","last_clean":"2024-04-25T10:03:41.906234+0000","last_became_active":"2024-04-25T10:03:39.870851+0000","last_became_peered":"2024-04-25T10:03:39.870851+0000","last_unstale":"2024-04-25T10:03:41.906234+0000","last_undegraded":"2024-04-25T10:03:41.906234+0000","last_fullsized":"2024-04-25T10:03:41.906234+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893718+0000","last_change":"2024-04-25T10:03:40.897889+0000","last_active":"2024-04-25T10:03:41.893718+0000","last_peered":"2024-04-25T10:03:41.893718+0000","last_clean":"2024-04-25T10:03:41.893718+0000","last_became_active":"2024-04-25T10:03:40.897617+0000","last_became_peered":"2024-04-25T10:03:40.897617+0000","last_unstale":"2024-04-25T10:03:41.893718+0000","last_undegraded":"2024-04-25T10:03:41.893718+0000","last_fullsized":"2024-04-25T10:03:41.893718+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:20.279938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"38'6","reported_seq":20,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888199+0000","last_change":"2024-04-25T10:03:39.869052+0000","last_active":"2024-04-25T10:03:41.888199+0000","last_peered":"2024-04-25T10:03:41.888199+0000","last_clean":"2024-04-25T10:03:41.888199+0000","last_became_active":"2024-04-25T10:03:39.867982+0000","last_became_peered":"2024-04-25T10:03:39.867982+0000","last_unstale":"2024-04-25T10:03:41.888199+0000","last_undegraded":"2024-04-25T10:03:41.888199+0000","last_fullsized":"2024-04-25T10:03:41.888199+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:19.265985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903161+0000","last_change":"2024-04-25T10:03:40.897645+0000","last_active":"2024-04-25T10:03:40.903161+0000","last_peered":"2024-04-25T10:03:40.903161+0000","last_clean":"2024-04-25T10:03:40.903161+0000","last_became_active":"2024-04-25T10:03:40.897351+0000","last_became_peered":"2024-04-25T10:03:40.897351+0000","last_unstale":"2024-04-25T10:03:40.903161+0000","last_undegraded":"2024-04-25T10:03:40.903161+0000","last_fullsized":"2024-04-25T10:03:40.903161+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888489+0000","last_change":"2024-04-25T10:03:39.865211+0000","last_active":"2024-04-25T10:03:41.888489+0000","last_peered":"2024-04-25T10:03:41.888489+0000","last_clean":"2024-04-25T10:03:41.888489+0000","last_became_active":"2024-04-25T10:03:39.864951+0000","last_became_peered":"2024-04-25T10:03:39.864951+0000","last_unstale":"2024-04-25T10:03:41.888489+0000","last_undegraded":"2024-04-25T10:03:41.888489+0000","last_fullsized":"2024-04-25T10:03:41.888489+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:48.846386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887599+0000","last_change":"2024-04-25T10:03:40.912706+0000","last_active":"2024-04-25T10:03:41.887599+0000","last_peered":"2024-04-25T10:03:41.887599+0000","last_clean":"2024-04-25T10:03:41.887599+0000","last_became_active":"2024-04-25T10:03:40.912377+0000","last_became_peered":"2024-04-25T10:03:40.912377+0000","last_unstale":"2024-04-25T10:03:41.887599+0000","last_undegraded":"2024-04-25T10:03:41.887599+0000","last_fullsized":"2024-04-25T10:03:41.887599+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:06.001065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.878699+0000","last_change":"2024-04-25T10:03:39.870564+0000","last_active":"2024-04-25T10:03:40.878699+0000","last_peered":"2024-04-25T10:03:40.878699+0000","last_clean":"2024-04-25T10:03:40.878699+0000","last_became_active":"2024-04-25T10:03:39.870211+0000","last_became_peered":"2024-04-25T10:03:39.870211+0000","last_unstale":"2024-04-25T10:03:40.878699+0000","last_undegraded":"2024-04-25T10:03:40.878699+0000","last_fullsized":"2024-04-25T10:03:40.878699+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:30.626295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.892884+0000","last_change":"2024-04-25T10:03:40.888109+0000","last_active":"2024-04-25T10:03:40.892884+0000","last_peered":"2024-04-25T10:03:40.892884+0000","last_clean":"2024-04-25T10:03:40.892884+0000","last_became_active":"2024-04-25T10:03:40.887787+0000","last_became_peered":"2024-04-25T10:03:40.887787+0000","last_unstale":"2024-04-25T10:03:40.892884+0000","last_undegraded":"2024-04-25T10:03:40.892884+0000","last_fullsized":"2024-04-25T10:03:40.892884+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:32.696527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.891308+0000","last_change":"2024-04-25T10:03:39.864186+0000","last_active":"2024-04-25T10:03:41.891308+0000","last_peered":"2024-04-25T10:03:41.891308+0000","last_clean":"2024-04-25T10:03:41.891308+0000","last_became_active":"2024-04-25T10:03:39.863831+0000","last_became_peered":"2024-04-25T10:03:39.863831+0000","last_unstale":"2024-04-25T10:03:41.891308+0000","last_undegraded":"2024-04-25T10:03:41.891308+0000","last_fullsized":"2024-04-25T10:03:41.891308+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:28.999175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888120+0000","last_change":"2024-04-25T10:03:40.912658+0000","last_active":"2024-04-25T10:03:41.888120+0000","last_peered":"2024-04-25T10:03:41.888120+0000","last_clean":"2024-04-25T10:03:41.888120+0000","last_became_active":"2024-04-25T10:03:40.912395+0000","last_became_peered":"2024-04-25T10:03:40.912395+0000","last_unstale":"2024-04-25T10:03:41.888120+0000","last_undegraded":"2024-04-25T10:03:41.888120+0000","last_fullsized":"2024-04-25T10:03:41.888120+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:06.915964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"38'8","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.918606+0000","last_change":"2024-04-25T10:03:39.877418+0000","last_active":"2024-04-25T10:03:40.918606+0000","last_peered":"2024-04-25T10:03:40.918606+0000","last_clean":"2024-04-25T10:03:40.918606+0000","last_became_active":"2024-04-25T10:03:39.876987+0000","last_became_peered":"2024-04-25T10:03:39.876987+0000","last_unstale":"2024-04-25T10:03:40.918606+0000","last_undegraded":"2024-04-25T10:03:40.918606+0000","last_fullsized":"2024-04-25T10:03:40.918606+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:30.626295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888305+0000","last_change":"2024-04-25T10:03:40.894469+0000","last_active":"2024-04-25T10:03:41.888305+0000","last_peered":"2024-04-25T10:03:41.888305+0000","last_clean":"2024-04-25T10:03:41.888305+0000","last_became_active":"2024-04-25T10:03:40.894141+0000","last_became_peered":"2024-04-25T10:03:40.894141+0000","last_unstale":"2024-04-25T10:03:41.888305+0000","last_undegraded":"2024-04-25T10:03:41.888305+0000","last_fullsized":"2024-04-25T10:03:41.888305+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:53:47.746327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905737+0000","last_change":"2024-04-25T10:03:39.875439+0000","last_active":"2024-04-25T10:03:41.905737+0000","last_peered":"2024-04-25T10:03:41.905737+0000","last_clean":"2024-04-25T10:03:41.905737+0000","last_became_active":"2024-04-25T10:03:39.875014+0000","last_became_peered":"2024-04-25T10:03:39.875014+0000","last_unstale":"2024-04-25T10:03:41.905737+0000","last_undegraded":"2024-04-25T10:03:41.905737+0000","last_fullsized":"2024-04-25T10:03:41.905737+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:10.023000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906106+0000","last_change":"2024-04-25T10:03:40.903516+0000","last_active":"2024-04-25T10:03:41.906106+0000","last_peered":"2024-04-25T10:03:41.906106+0000","last_clean":"2024-04-25T10:03:41.906106+0000","last_became_active":"2024-04-25T10:03:40.902419+0000","last_became_peered":"2024-04-25T10:03:40.902419+0000","last_unstale":"2024-04-25T10:03:41.906106+0000","last_undegraded":"2024-04-25T10:03:41.906106+0000","last_fullsized":"2024-04-25T10:03:41.906106+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.859279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893800+0000","last_change":"2024-04-25T10:03:39.873987+0000","last_active":"2024-04-25T10:03:41.893800+0000","last_peered":"2024-04-25T10:03:41.893800+0000","last_clean":"2024-04-25T10:03:41.893800+0000","last_became_active":"2024-04-25T10:03:39.873693+0000","last_became_peered":"2024-04-25T10:03:39.873693+0000","last_unstale":"2024-04-25T10:03:41.893800+0000","last_undegraded":"2024-04-25T10:03:41.893800+0000","last_fullsized":"2024-04-25T10:03:41.893800+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893070+0000","last_change":"2024-04-25T10:03:40.895114+0000","last_active":"2024-04-25T10:03:41.893070+0000","last_peered":"2024-04-25T10:03:41.893070+0000","last_clean":"2024-04-25T10:03:41.893070+0000","last_became_active":"2024-04-25T10:03:40.894833+0000","last_became_peered":"2024-04-25T10:03:40.894833+0000","last_unstale":"2024-04-25T10:03:41.893070+0000","last_undegraded":"2024-04-25T10:03:41.893070+0000","last_fullsized":"2024-04-25T10:03:41.893070+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"38'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899271+0000","last_change":"2024-04-25T10:03:39.869576+0000","last_active":"2024-04-25T10:03:41.899271+0000","last_peered":"2024-04-25T10:03:41.899271+0000","last_clean":"2024-04-25T10:03:41.899271+0000","last_became_active":"2024-04-25T10:03:39.869232+0000","last_became_peered":"2024-04-25T10:03:39.869232+0000","last_unstale":"2024-04-25T10:03:41.899271+0000","last_undegraded":"2024-04-25T10:03:41.899271+0000","last_fullsized":"2024-04-25T10:03:41.899271+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:59:22.416577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899579+0000","last_change":"2024-04-25T10:03:40.885179+0000","last_active":"2024-04-25T10:03:41.899579+0000","last_peered":"2024-04-25T10:03:41.899579+0000","last_clean":"2024-04-25T10:03:41.899579+0000","last_became_active":"2024-04-25T10:03:40.883917+0000","last_became_peered":"2024-04-25T10:03:40.883917+0000","last_unstale":"2024-04-25T10:03:41.899579+0000","last_undegraded":"2024-04-25T10:03:41.899579+0000","last_fullsized":"2024-04-25T10:03:41.899579+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:34:25.972919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.880068+0000","last_change":"2024-04-25T10:03:39.862572+0000","last_active":"2024-04-25T10:03:40.880068+0000","last_peered":"2024-04-25T10:03:40.880068+0000","last_clean":"2024-04-25T10:03:40.880068+0000","last_became_active":"2024-04-25T10:03:39.862305+0000","last_became_peered":"2024-04-25T10:03:39.862305+0000","last_unstale":"2024-04-25T10:03:40.880068+0000","last_undegraded":"2024-04-25T10:03:40.880068+0000","last_fullsized":"2024-04-25T10:03:40.880068+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.526948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.896630+0000","last_change":"2024-04-25T10:03:40.888980+0000","last_active":"2024-04-25T10:03:40.896630+0000","last_peered":"2024-04-25T10:03:40.896630+0000","last_clean":"2024-04-25T10:03:40.896630+0000","last_became_active":"2024-04-25T10:03:40.888568+0000","last_became_peered":"2024-04-25T10:03:40.888568+0000","last_unstale":"2024-04-25T10:03:40.896630+0000","last_undegraded":"2024-04-25T10:03:40.896630+0000","last_fullsized":"2024-04-25T10:03:40.896630+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899078+0000","last_change":"2024-04-25T10:03:39.860759+0000","last_active":"2024-04-25T10:03:41.899078+0000","last_peered":"2024-04-25T10:03:41.899078+0000","last_clean":"2024-04-25T10:03:41.899078+0000","last_became_active":"2024-04-25T10:03:39.858548+0000","last_became_peered":"2024-04-25T10:03:39.858548+0000","last_unstale":"2024-04-25T10:03:41.899078+0000","last_undegraded":"2024-04-25T10:03:41.899078+0000","last_fullsized":"2024-04-25T10:03:41.899078+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:20.225426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.901962+0000","last_change":"2024-04-25T10:03:40.893560+0000","last_active":"2024-04-25T10:03:40.901962+0000","last_peered":"2024-04-25T10:03:40.901962+0000","last_clean":"2024-04-25T10:03:40.901962+0000","last_became_active":"2024-04-25T10:03:40.892177+0000","last_became_peered":"2024-04-25T10:03:40.892177+0000","last_unstale":"2024-04-25T10:03:40.901962+0000","last_undegraded":"2024-04-25T10:03:40.901962+0000","last_fullsized":"2024-04-25T10:03:40.901962+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:13.835191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.899250+0000","last_change":"2024-04-25T10:03:39.862781+0000","last_active":"2024-04-25T10:03:41.899250+0000","last_peered":"2024-04-25T10:03:41.899250+0000","last_clean":"2024-04-25T10:03:41.899250+0000","last_became_active":"2024-04-25T10:03:39.862522+0000","last_became_peered":"2024-04-25T10:03:39.862522+0000","last_unstale":"2024-04-25T10:03:41.899250+0000","last_undegraded":"2024-04-25T10:03:41.899250+0000","last_fullsized":"2024-04-25T10:03:41.899250+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:27.649320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905489+0000","last_change":"2024-04-25T10:03:40.895865+0000","last_active":"2024-04-25T10:03:41.905489+0000","last_peered":"2024-04-25T10:03:41.905489+0000","last_clean":"2024-04-25T10:03:41.905489+0000","last_became_active":"2024-04-25T10:03:40.895537+0000","last_became_peered":"2024-04-25T10:03:40.895537+0000","last_unstale":"2024-04-25T10:03:41.905489+0000","last_undegraded":"2024-04-25T10:03:41.905489+0000","last_fullsized":"2024-04-25T10:03:41.905489+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:32:06.001065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.869714+0000","last_change":"2024-04-25T10:03:39.872887+0000","last_active":"2024-04-25T10:03:40.869714+0000","last_peered":"2024-04-25T10:03:40.869714+0000","last_clean":"2024-04-25T10:03:40.869714+0000","last_became_active":"2024-04-25T10:03:39.872476+0000","last_became_peered":"2024-04-25T10:03:39.872476+0000","last_unstale":"2024-04-25T10:03:40.869714+0000","last_undegraded":"2024-04-25T10:03:40.869714+0000","last_fullsized":"2024-04-25T10:03:40.869714+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:50:31.526948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888343+0000","last_change":"2024-04-25T10:03:40.902919+0000","last_active":"2024-04-25T10:03:41.888343+0000","last_peered":"2024-04-25T10:03:41.888343+0000","last_clean":"2024-04-25T10:03:41.888343+0000","last_became_active":"2024-04-25T10:03:40.902243+0000","last_became_peered":"2024-04-25T10:03:40.902243+0000","last_unstale":"2024-04-25T10:03:41.888343+0000","last_undegraded":"2024-04-25T10:03:41.888343+0000","last_fullsized":"2024-04-25T10:03:41.888343+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:53:33.859279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.898963+0000","last_change":"2024-04-25T10:03:39.859341+0000","last_active":"2024-04-25T10:03:41.898963+0000","last_peered":"2024-04-25T10:03:41.898963+0000","last_clean":"2024-04-25T10:03:41.898963+0000","last_became_active":"2024-04-25T10:03:39.859083+0000","last_became_peered":"2024-04-25T10:03:39.859083+0000","last_unstale":"2024-04-25T10:03:41.898963+0000","last_undegraded":"2024-04-25T10:03:41.898963+0000","last_fullsized":"2024-04-25T10:03:41.898963+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:10.822877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.903505+0000","last_change":"2024-04-25T10:03:40.899614+0000","last_active":"2024-04-25T10:03:40.903505+0000","last_peered":"2024-04-25T10:03:40.903505+0000","last_clean":"2024-04-25T10:03:40.903505+0000","last_became_active":"2024-04-25T10:03:40.898918+0000","last_became_peered":"2024-04-25T10:03:40.898918+0000","last_unstale":"2024-04-25T10:03:40.903505+0000","last_undegraded":"2024-04-25T10:03:40.903505+0000","last_fullsized":"2024-04-25T10:03:40.903505+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:59:59.873587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"38'2","reported_seq":16,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.906023+0000","last_change":"2024-04-25T10:03:39.873821+0000","last_active":"2024-04-25T10:03:41.906023+0000","last_peered":"2024-04-25T10:03:41.906023+0000","last_clean":"2024-04-25T10:03:41.906023+0000","last_became_active":"2024-04-25T10:03:39.873488+0000","last_became_peered":"2024-04-25T10:03:39.873488+0000","last_unstale":"2024-04-25T10:03:41.906023+0000","last_undegraded":"2024-04-25T10:03:41.906023+0000","last_fullsized":"2024-04-25T10:03:41.906023+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:11:31.682574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.896948+0000","last_change":"2024-04-25T10:03:40.891772+0000","last_active":"2024-04-25T10:03:40.896948+0000","last_peered":"2024-04-25T10:03:40.896948+0000","last_clean":"2024-04-25T10:03:40.896948+0000","last_became_active":"2024-04-25T10:03:40.891432+0000","last_became_peered":"2024-04-25T10:03:40.891432+0000","last_unstale":"2024-04-25T10:03:40.896948+0000","last_undegraded":"2024-04-25T10:03:40.896948+0000","last_fullsized":"2024-04-25T10:03:40.896948+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:26.483271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"39'1","reported_seq":15,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.903154+0000","last_change":"2024-04-25T10:03:39.873720+0000","last_active":"2024-04-25T10:03:41.903154+0000","last_peered":"2024-04-25T10:03:41.903154+0000","last_clean":"2024-04-25T10:03:41.903154+0000","last_became_active":"2024-04-25T10:03:39.873395+0000","last_became_peered":"2024-04-25T10:03:39.873395+0000","last_unstale":"2024-04-25T10:03:41.903154+0000","last_undegraded":"2024-04-25T10:03:41.903154+0000","last_fullsized":"2024-04-25T10:03:41.903154+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:54.385884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.888714+0000","last_change":"2024-04-25T10:03:40.896741+0000","last_active":"2024-04-25T10:03:41.888714+0000","last_peered":"2024-04-25T10:03:41.888714+0000","last_clean":"2024-04-25T10:03:41.888714+0000","last_became_active":"2024-04-25T10:03:40.896455+0000","last_became_peered":"2024-04-25T10:03:40.896455+0000","last_unstale":"2024-04-25T10:03:41.888714+0000","last_undegraded":"2024-04-25T10:03:41.888714+0000","last_fullsized":"2024-04-25T10:03:41.888714+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:26.573521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.882922+0000","last_change":"2024-04-25T10:03:39.866920+0000","last_active":"2024-04-25T10:03:40.882922+0000","last_peered":"2024-04-25T10:03:40.882922+0000","last_clean":"2024-04-25T10:03:40.882922+0000","last_became_active":"2024-04-25T10:03:39.866592+0000","last_became_peered":"2024-04-25T10:03:39.866592+0000","last_unstale":"2024-04-25T10:03:40.882922+0000","last_undegraded":"2024-04-25T10:03:40.882922+0000","last_fullsized":"2024-04-25T10:03:40.882922+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:29:30.626295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.914094+0000","last_change":"2024-04-25T10:03:40.909026+0000","last_active":"2024-04-25T10:03:40.914094+0000","last_peered":"2024-04-25T10:03:40.914094+0000","last_clean":"2024-04-25T10:03:40.914094+0000","last_became_active":"2024-04-25T10:03:40.908715+0000","last_became_peered":"2024-04-25T10:03:40.908715+0000","last_unstale":"2024-04-25T10:03:40.914094+0000","last_undegraded":"2024-04-25T10:03:40.914094+0000","last_fullsized":"2024-04-25T10:03:40.914094+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.883823+0000","last_change":"2024-04-25T10:03:39.869498+0000","last_active":"2024-04-25T10:03:40.883823+0000","last_peered":"2024-04-25T10:03:40.883823+0000","last_clean":"2024-04-25T10:03:40.883823+0000","last_became_active":"2024-04-25T10:03:39.869221+0000","last_became_peered":"2024-04-25T10:03:39.869221+0000","last_unstale":"2024-04-25T10:03:40.883823+0000","last_undegraded":"2024-04-25T10:03:40.883823+0000","last_fullsized":"2024-04-25T10:03:40.883823+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:47.746435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.895955+0000","last_change":"2024-04-25T10:03:40.890606+0000","last_active":"2024-04-25T10:03:40.895955+0000","last_peered":"2024-04-25T10:03:40.895955+0000","last_clean":"2024-04-25T10:03:40.895955+0000","last_became_active":"2024-04-25T10:03:40.890338+0000","last_became_peered":"2024-04-25T10:03:40.890338+0000","last_unstale":"2024-04-25T10:03:40.895955+0000","last_undegraded":"2024-04-25T10:03:40.895955+0000","last_fullsized":"2024-04-25T10:03:40.895955+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:11.036953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892906+0000","last_change":"2024-04-25T10:03:39.865429+0000","last_active":"2024-04-25T10:03:41.892906+0000","last_peered":"2024-04-25T10:03:41.892906+0000","last_clean":"2024-04-25T10:03:41.892906+0000","last_became_active":"2024-04-25T10:03:39.865102+0000","last_became_peered":"2024-04-25T10:03:39.865102+0000","last_unstale":"2024-04-25T10:03:41.892906+0000","last_undegraded":"2024-04-25T10:03:41.892906+0000","last_fullsized":"2024-04-25T10:03:41.892906+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:16:46.949157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.893231+0000","last_change":"2024-04-25T10:03:40.893459+0000","last_active":"2024-04-25T10:03:41.893231+0000","last_peered":"2024-04-25T10:03:41.893231+0000","last_clean":"2024-04-25T10:03:41.893231+0000","last_became_active":"2024-04-25T10:03:40.892510+0000","last_became_peered":"2024-04-25T10:03:40.892510+0000","last_unstale":"2024-04-25T10:03:41.893231+0000","last_undegraded":"2024-04-25T10:03:41.893231+0000","last_fullsized":"2024-04-25T10:03:41.893231+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.892593+0000","last_change":"2024-04-25T10:03:39.867195+0000","last_active":"2024-04-25T10:03:41.892593+0000","last_peered":"2024-04-25T10:03:41.892593+0000","last_clean":"2024-04-25T10:03:41.892593+0000","last_became_active":"2024-04-25T10:03:39.862857+0000","last_became_peered":"2024-04-25T10:03:39.862857+0000","last_unstale":"2024-04-25T10:03:41.892593+0000","last_undegraded":"2024-04-25T10:03:41.892593+0000","last_fullsized":"2024-04-25T10:03:41.892593+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:25:20.557967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.896752+0000","last_change":"2024-04-25T10:03:40.890274+0000","last_active":"2024-04-25T10:03:40.896752+0000","last_peered":"2024-04-25T10:03:40.896752+0000","last_clean":"2024-04-25T10:03:40.896752+0000","last_became_active":"2024-04-25T10:03:40.889950+0000","last_became_peered":"2024-04-25T10:03:40.889950+0000","last_unstale":"2024-04-25T10:03:40.896752+0000","last_undegraded":"2024-04-25T10:03:40.896752+0000","last_fullsized":"2024-04-25T10:03:40.896752+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:04.706739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":14,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.887383+0000","last_change":"2024-04-25T10:03:39.860569+0000","last_active":"2024-04-25T10:03:41.887383+0000","last_peered":"2024-04-25T10:03:41.887383+0000","last_clean":"2024-04-25T10:03:41.887383+0000","last_became_active":"2024-04-25T10:03:39.860301+0000","last_became_peered":"2024-04-25T10:03:39.860301+0000","last_unstale":"2024-04-25T10:03:41.887383+0000","last_undegraded":"2024-04-25T10:03:41.887383+0000","last_fullsized":"2024-04-25T10:03:41.887383+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:25.469318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-25T10:03:41.905453+0000","last_change":"2024-04-25T10:03:40.913830+0000","last_active":"2024-04-25T10:03:41.905453+0000","last_peered":"2024-04-25T10:03:41.905453+0000","last_clean":"2024-04-25T10:03:41.905453+0000","last_became_active":"2024-04-25T10:03:40.913506+0000","last_became_peered":"2024-04-25T10:03:40.913506+0000","last_unstale":"2024-04-25T10:03:41.905453+0000","last_undegraded":"2024-04-25T10:03:41.905453+0000","last_fullsized":"2024-04-25T10:03:41.905453+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:39.826639+0000","last_clean_scrub_stamp":"2024-04-25T10:03:39.826639+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:16.559780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.870172+0000","last_change":"2024-04-25T10:03:39.872650+0000","last_active":"2024-04-25T10:03:40.870172+0000","last_peered":"2024-04-25T10:03:40.870172+0000","last_clean":"2024-04-25T10:03:40.870172+0000","last_became_active":"2024-04-25T10:03:39.872192+0000","last_became_peered":"2024-04-25T10:03:39.872192+0000","last_unstale":"2024-04-25T10:03:40.870172+0000","last_undegraded":"2024-04-25T10:03:40.870172+0000","last_fullsized":"2024-04-25T10:03:40.870172+0000","mapping_epoch":36,"log_start":"0'0","ondisk_log_start":"0'0","created":36,"last_epoch_clean":37,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:03:38.812685+0000","last_clean_scrub_stamp":"2024-04-25T10:03:38.812685+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:30:22.817529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":204,"reported_epoch":38,"state":"active+clean","last_fresh":"2024-04-25T10:03:40.869142+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:03:40.869142+0000","last_peered":"2024-04-25T10:03:40.869142+0000","last_clean":"2024-04-25T10:03:40.869142+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:03:40.869142+0000","last_undegraded":"2024-04-25T10:03:40.869142+0000","last_fullsized":"2024-04-25T10:03:40.869142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28044,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343796,"statfs":{"total":96636764160,"available":96608047104,"internally_reserved":0,"allocated":1048576,"data_stored":781101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[1,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":442368,"data_stored":189633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189163,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574874,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27456,"kb_used_data":436,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344384,"statfs":{"total":96636764160,"available":96608649216,"internally_reserved":0,"allocated":446464,"data_stored":189655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27460,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344380,"statfs":{"total":96636764160,"available":96608645120,"internally_reserved":0,"allocated":454656,"data_stored":190697,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574873,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27468,"kb_used_data":444,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344372,"statfs":{"total":96636764160,"available":96608636928,"internally_reserved":0,"allocated":454656,"data_stored":195813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27452,"kb_used_data":432,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344388,"statfs":{"total":96636764160,"available":96608653312,"internally_reserved":0,"allocated":442368,"data_stored":189141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343808,"statfs":{"total":96636764160,"available":96608059392,"internally_reserved":0,"allocated":1036288,"data_stored":779453,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:03:45.793 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:03:45.793 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:03:46.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.044+0000 7f1036800640 1 Processor -- start 2024-04-25T10:03:46.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.044+0000 7f1036800640 1 -- start start 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1036800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1030154350 0x7f103006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1036800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10300b0e80 0x7f10300711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1036800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f103015ae90 con 0x7f1030154730 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1036800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f103015b450 con 0x7f1030154350 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1036800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103015ac30 con 0x7f10300b0e80 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1035400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1030154350 0x7f103006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1035400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1030154350 0x7f103006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39552/0 (socket says 172.21.15.165:39552) 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1035400640 1 -- 172.21.15.165:0/3418761791 learned_addr learned my addr 172.21.15.165:0/3418761791 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:46.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.046+0000 7f1035e00640 1 --2- 172.21.15.165:0/3418761791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10300b0e80 0x7f10300711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.047+0000 7f1035400640 1 -- 172.21.15.165:0/3418761791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10300b0e80 msgr2=0x7f10300711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.047+0000 7f1035400640 1 --2- 172.21.15.165:0/3418761791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10300b0e80 0x7f10300711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.047+0000 7f1035400640 1 -- 172.21.15.165:0/3418761791 >> v1:172.21.15.172:6789/0 conn(0x7f1030154730 legacy=0x7f103015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.047+0000 7f1035400640 1 -- 172.21.15.165:0/3418761791 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103015b170 con 0x7f1030154350 2024-04-25T10:03:46.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.048+0000 7f1035400640 1 --2- 172.21.15.165:0/3418761791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1030154350 0x7f103006db20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1024009870 tx=0x7f102402a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a1ad857489ea6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.048+0000 7f102fe00640 1 -- 172.21.15.165:0/3418761791 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024037070 con 0x7f1030154350 2024-04-25T10:03:46.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.048+0000 7f102fe00640 1 -- 172.21.15.165:0/3418761791 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102402ae00 con 0x7f1030154350 2024-04-25T10:03:46.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.048+0000 7f102fe00640 1 -- 172.21.15.165:0/3418761791 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024032660 con 0x7f1030154350 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 -- 172.21.15.165:0/3418761791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1030154350 msgr2=0x7f103006db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 --2- 172.21.15.165:0/3418761791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1030154350 0x7f103006db20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f1024009870 tx=0x7f102402a0f0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 -- 172.21.15.165:0/3418761791 shutdown_connections 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 --2- 172.21.15.165:0/3418761791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10300b0e80 0x7f10300711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 --2- 172.21.15.165:0/3418761791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1030154350 0x7f103006db20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 -- 172.21.15.165:0/3418761791 >> 172.21.15.165:0/3418761791 conn(0x7f1030061cc0 msgr2=0x7f103006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 -- 172.21.15.165:0/3418761791 shutdown_connections 2024-04-25T10:03:46.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.049+0000 7f1036800640 1 -- 172.21.15.165:0/3418761791 wait complete. 2024-04-25T10:03:46.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.051+0000 7f1036800640 1 Processor -- start 2024-04-25T10:03:46.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.051+0000 7f1036800640 1 -- start start 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1036800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 0x7f10301439b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1036800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1030154350 0x7f103015d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1036800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1030154730 0x7f103015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1036800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103006e060 con 0x7f10300b0e80 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1036800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103006d600 con 0x7f1030154350 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1036800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1030071710 con 0x7f1030154730 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1035400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 0x7f10301439b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1035400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 0x7f10301439b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39558/0 (socket says 172.21.15.165:39558) 2024-04-25T10:03:46.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1035400640 1 -- 172.21.15.165:0/350143223 learned_addr learned my addr 172.21.15.165:0/350143223 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1035400640 1 -- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1030154730 msgr2=0x7f103015ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1035400640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1030154730 0x7f103015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1035400640 1 -- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1030154350 msgr2=0x7f103015d870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.052+0000 7f1034a00640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1030154350 0x7f103015d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f1035400640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1030154350 0x7f103015d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f1035400640 1 -- 172.21.15.165:0/350143223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10301f04a0 con 0x7f10300b0e80 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f1035400640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 0x7f10301439b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1024009870 tx=0x7f10240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024042070 con 0x7f10300b0e80 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f102403e670 con 0x7f10300b0e80 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10301f2eb0 con 0x7f10300b0e80 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1024037040 con 0x7f10300b0e80 2024-04-25T10:03:46.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.053+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10301f3470 con 0x7f10300b0e80 2024-04-25T10:03:46.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.055+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1030073990 con 0x7f10300b0e80 2024-04-25T10:03:46.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.055+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f102403a040 con 0x7f10300b0e80 2024-04-25T10:03:46.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.056+0000 7f102d600640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f101006b4d0 0x7f101006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.056+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f102402ae00 con 0x7f10300b0e80 2024-04-25T10:03:46.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.057+0000 7f1034a00640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f101006b4d0 0x7f101006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.057+0000 7f1034a00640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f101006b4d0 0x7f101006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1030152ec0 tx=0x7f101c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.061+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1024079320 con 0x7f10300b0e80 2024-04-25T10:03:46.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.330+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f103006e750 con 0x7f10300b0e80 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.334+0000 7f102d600640 1 -- 172.21.15.165:0/350143223 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1024078c20 con 0x7f10300b0e80 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:03:46.339 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:03:46.340 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:03:46.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f101006b4d0 msgr2=0x7f101006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f101006b4d0 0x7f101006d990 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f1030152ec0 tx=0x7f101c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 msgr2=0x7f10301439b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 0x7f10301439b0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f1024009870 tx=0x7f10240046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 shutdown_connections 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f101006b4d0 0x7f101006d990 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1030154730 0x7f103015ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1030154350 0x7f103015d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 --2- 172.21.15.165:0/350143223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10300b0e80 0x7f10301439b0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 >> 172.21.15.165:0/350143223 conn(0x7f1030061cc0 msgr2=0x7f1030151170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 shutdown_connections 2024-04-25T10:03:46.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.337+0000 7f1036800640 1 -- 172.21.15.165:0/350143223 wait complete. 2024-04-25T10:03:46.360 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:03:46.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.548+0000 7fb921000640 1 Processor -- start 2024-04-25T10:03:46.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.548+0000 7fb921000640 1 -- start start 2024-04-25T10:03:46.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.549+0000 7fb921000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb91c144750 0x7fb91c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.549+0000 7fb921000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb91c06eb10 0x7fb91c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.549+0000 7fb921000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91c144b30 con 0x7fb91c0cee60 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.549+0000 7fb921000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c0712d0 con 0x7fb91c06eb10 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.549+0000 7fb921000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c0715b0 con 0x7fb91c144750 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb91c06eb10 0x7fb91c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb91c06eb10 0x7fb91c0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39574/0 (socket says 172.21.15.165:39574) 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 -- 172.21.15.165:0/2499809537 learned_addr learned my addr 172.21.15.165:0/2499809537 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb919e00640 1 --2- 172.21.15.165:0/2499809537 >> v2:172.21.15.172:3300/0 conn(0x7fb91c144750 0x7fb91c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 -- 172.21.15.165:0/2499809537 >> v2:172.21.15.172:3300/0 conn(0x7fb91c144750 msgr2=0x7fb91c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 --2- 172.21.15.165:0/2499809537 >> v2:172.21.15.172:3300/0 conn(0x7fb91c144750 0x7fb91c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 -- 172.21.15.165:0/2499809537 >> v1:172.21.15.172:6789/0 conn(0x7fb91c0cee60 legacy=0x7fb91c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.550+0000 7fb911400640 1 -- 172.21.15.165:0/2499809537 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c071890 con 0x7fb91c06eb10 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.551+0000 7fb911400640 1 --2- 172.21.15.165:0/2499809537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c0ce920 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb91c06f5d0 tx=0x7fb90402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e88a73f5915fe621 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb919400640 1 -- 172.21.15.165:0/2499809537 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904037070 con 0x7fb91c06eb10 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb919400640 1 -- 172.21.15.165:0/2499809537 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb904032440 con 0x7fb91c06eb10 2024-04-25T10:03:46.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb919400640 1 -- 172.21.15.165:0/2499809537 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb904032810 con 0x7fb91c06eb10 2024-04-25T10:03:46.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb921000640 1 -- 172.21.15.165:0/2499809537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 msgr2=0x7fb91c0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb921000640 1 --2- 172.21.15.165:0/2499809537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c0ce920 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fb91c06f5d0 tx=0x7fb90402a230 comp rx=0 tx=0).stop 2024-04-25T10:03:46.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb921000640 1 -- 172.21.15.165:0/2499809537 shutdown_connections 2024-04-25T10:03:46.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb921000640 1 --2- 172.21.15.165:0/2499809537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c0ce920 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb921000640 1 --2- 172.21.15.165:0/2499809537 >> v2:172.21.15.172:3300/0 conn(0x7fb91c144750 0x7fb91c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.552+0000 7fb921000640 1 -- 172.21.15.165:0/2499809537 >> 172.21.15.165:0/2499809537 conn(0x7fb91c06abe0 msgr2=0x7fb91c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:46.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.553+0000 7fb921000640 1 -- 172.21.15.165:0/2499809537 shutdown_connections 2024-04-25T10:03:46.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.553+0000 7fb921000640 1 -- 172.21.15.165:0/2499809537 wait complete. 2024-04-25T10:03:46.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.554+0000 7fb921000640 1 Processor -- start 2024-04-25T10:03:46.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.554+0000 7fb921000640 1 -- start start 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb921000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c159b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb921000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb91c0cee60 0x7fb91c15ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb921000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb91c144750 0x7fb91c15b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb921000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c145450 con 0x7fb91c06eb10 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb921000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c06f020 con 0x7fb91c0cee60 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb921000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c1455d0 con 0x7fb91c144750 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb919e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c159b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb919e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c159b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39590/0 (socket says 172.21.15.165:39590) 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.555+0000 7fb919e00640 1 -- 172.21.15.165:0/1342912789 learned_addr learned my addr 172.21.15.165:0/1342912789 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb919e00640 1 -- 172.21.15.165:0/1342912789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb91c144750 msgr2=0x7fb91c15b180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb919e00640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb91c144750 0x7fb91c15b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb919e00640 1 -- 172.21.15.165:0/1342912789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb91c0cee60 msgr2=0x7fb91c15ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb919e00640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb91c0cee60 0x7fb91c15ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb919e00640 1 -- 172.21.15.165:0/1342912789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c1e7320 con 0x7fb91c06eb10 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb919e00640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c159b80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb90c007ef0 tx=0x7fb90c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c014620 con 0x7fb91c06eb10 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c019070 con 0x7fb91c06eb10 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c1e7600 con 0x7fb91c06eb10 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.556+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c014a00 con 0x7fb91c06eb10 2024-04-25T10:03:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.557+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c0afa20 con 0x7fb91c06eb10 2024-04-25T10:03:46.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.558+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c073990 con 0x7fb91c06eb10 2024-04-25T10:03:46.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.559+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb90c014ba0 con 0x7fb91c06eb10 2024-04-25T10:03:46.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.560+0000 7fb912a00640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:46.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.560+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fb90c08acf0 con 0x7fb91c06eb10 2024-04-25T10:03:46.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.560+0000 7fb911400640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:46.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.561+0000 7fb911400640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8f806b4d0 0x7fb8f806d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb904034000 tx=0x7fb90402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:46.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.564+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb90c0599b0 con 0x7fb91c06eb10 2024-04-25T10:03:46.769 INFO:tasks.ceph.mds.a.smithi165.stderr:2024-04-25T10:03:46.764+0000 7f5aad400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:03:46.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.796+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb91c141b00 con 0x7fb91c06eb10 2024-04-25T10:03:46.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.797+0000 7fb912a00640 1 -- 172.21.15.165:0/1342912789 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb90c0592b0 con 0x7fb91c06eb10 2024-04-25T10:03:46.802 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:03:46.803 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":19,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:03:40.296050+0000","modified":"2024-04-25T10:03:43.379748+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/3282481342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3282481342},{"type":"v1","addr":"172.21.15.165:6837","nonce":3282481342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:03:46.803 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 19 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8f806b4d0 msgr2=0x7fb8f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8f806b4d0 0x7fb8f806d990 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb904034000 tx=0x7fb90402c000 comp rx=0 tx=0).stop 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 msgr2=0x7fb91c159b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c159b80 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fb90c007ef0 tx=0x7fb90c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 shutdown_connections 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8f806b4d0 0x7fb8f806d990 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb91c144750 0x7fb91c15b180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb91c0cee60 0x7fb91c15ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 --2- 172.21.15.165:0/1342912789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb91c06eb10 0x7fb91c159b80 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:46.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 >> 172.21.15.165:0/1342912789 conn(0x7fb91c06abe0 msgr2=0x7fb91c13e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:46.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 shutdown_connections 2024-04-25T10:03:46.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:46.800+0000 7fb921000640 1 -- 172.21.15.165:0/1342912789 wait complete. 2024-04-25T10:03:47.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 19, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:03:40.296050+0000', 'modified': '2024-04-25T10:03:43.379748+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5337}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5337': {'gid': 5337, 'name': 'a', 'rank': 0, 'incarnation': 15, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.165:6837/3282481342', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 3282481342}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 3282481342}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:03:47.226 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:03:47.226 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:03:47.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.434+0000 7fc6f3e00640 1 Processor -- start 2024-04-25T10:03:47.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.434+0000 7fc6f3e00640 1 -- start start 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6ec1446d0 0x7fc6ec0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6ec0cccc0 con 0x7fc6ec1452a0 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f3e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6ec0b2b20 con 0x7fc6ec1446d0 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec06d400 con 0x7fc6ec144d50 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6ec1446d0 0x7fc6ec0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6ec1446d0 0x7fc6ec0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60596/0 (socket says 172.21.15.165:60596) 2024-04-25T10:03:47.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f1400640 1 -- 172.21.15.165:0/4065646223 learned_addr learned my addr 172.21.15.165:0/4065646223 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.436+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/4065646223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.437+0000 7fc6f1400640 1 -- 172.21.15.165:0/4065646223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 msgr2=0x7fc6ec0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.437+0000 7fc6f1400640 1 --2- 172.21.15.165:0/4065646223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.437+0000 7fc6f1400640 1 -- 172.21.15.165:0/4065646223 >> v1:172.21.15.172:6789/0 conn(0x7fc6ec1452a0 legacy=0x7fc6ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.437+0000 7fc6f1400640 1 -- 172.21.15.165:0/4065646223 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0cc140 con 0x7fc6ec1446d0 2024-04-25T10:03:47.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.438+0000 7fc6f1400640 1 --2- 172.21.15.165:0/4065646223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0b3f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009870 tx=0x7fc6e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8e69f23762ca0d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.439+0000 7fc6ebe00640 1 -- 172.21.15.165:0/4065646223 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0037070 con 0x7fc6ec1446d0 2024-04-25T10:03:47.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.439+0000 7fc6ebe00640 1 -- 172.21.15.165:0/4065646223 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e0032440 con 0x7fc6ec1446d0 2024-04-25T10:03:47.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.439+0000 7fc6ebe00640 1 -- 172.21.15.165:0/4065646223 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0032810 con 0x7fc6ec1446d0 2024-04-25T10:03:47.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.439+0000 7fc6f3e00640 1 -- 172.21.15.165:0/4065646223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1446d0 msgr2=0x7fc6ec0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.439+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/4065646223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0b3f20 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009870 tx=0x7fc6e002a210 comp rx=0 tx=0).stop 2024-04-25T10:03:47.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.440+0000 7fc6f3e00640 1 -- 172.21.15.165:0/4065646223 shutdown_connections 2024-04-25T10:03:47.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.440+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/4065646223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.440+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/4065646223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0b3f20 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.440+0000 7fc6f3e00640 1 -- 172.21.15.165:0/4065646223 >> 172.21.15.165:0/4065646223 conn(0x7fc6ec06abe0 msgr2=0x7fc6ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.440+0000 7fc6f3e00640 1 -- 172.21.15.165:0/4065646223 shutdown_connections 2024-04-25T10:03:47.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.440+0000 7fc6f3e00640 1 -- 172.21.15.165:0/4065646223 wait complete. 2024-04-25T10:03:47.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.441+0000 7fc6f3e00640 1 Processor -- start 2024-04-25T10:03:47.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 -- start start 2024-04-25T10:03:47.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0afd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec1f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 0x7fc6ec13d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0cc700 con 0x7fc6ec1452a0 2024-04-25T10:03:47.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0b49a0 con 0x7fc6ec1446d0 2024-04-25T10:03:47.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.442+0000 7fc6f3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec0cc9e0 con 0x7fc6ec144d50 2024-04-25T10:03:47.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 0x7fc6ec13d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 0x7fc6ec13d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60612/0 (socket says 172.21.15.165:60612) 2024-04-25T10:03:47.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 -- 172.21.15.165:0/2659687749 learned_addr learned my addr 172.21.15.165:0/2659687749 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1400640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0afd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 -- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 msgr2=0x7fc6ec1f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec1f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 -- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec1446d0 msgr2=0x7fc6ec0afd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0afd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 -- 172.21.15.165:0/2659687749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6ec0b0e70 con 0x7fc6ec1452a0 2024-04-25T10:03:47.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.443+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 0x7fc6ec13d840 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b970 tx=0x7fc6e400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.444+0000 7fc6e9600640 1 -- 172.21.15.165:0/2659687749 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4019070 con 0x7fc6ec1452a0 2024-04-25T10:03:47.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.444+0000 7fc6e9600640 1 -- 172.21.15.165:0/2659687749 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e400cb40 con 0x7fc6ec1452a0 2024-04-25T10:03:47.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.444+0000 7fc6e9600640 1 -- 172.21.15.165:0/2659687749 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4014440 con 0x7fc6ec1452a0 2024-04-25T10:03:47.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.444+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6ec0b02f0 con 0x7fc6ec1452a0 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.444+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6ec0b08b0 con 0x7fc6ec1452a0 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.446+0000 7fc6e9600640 1 -- 172.21.15.165:0/2659687749 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6e401c070 con 0x7fc6ec1452a0 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.447+0000 7fc6e9600640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6cc06b500 0x7fc6cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.447+0000 7fc6e9600640 1 -- 172.21.15.165:0/2659687749 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc6e4089fe0 con 0x7fc6ec1452a0 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.447+0000 7fc6f1400640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6cc06b500 0x7fc6cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.448+0000 7fc6f6ba6740 1 Processor -- start 2024-04-25T10:03:47.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.448+0000 7fc6f1400640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6cc06b500 0x7fc6cc06d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0034000 tx=0x7fc6e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.448+0000 7fc6f6ba6740 1 -- start start 2024-04-25T10:03:47.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b386950 0x55d58b3ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b386d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3cf5c0 0x55d58b385110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58b2ea620 con 0x55d58b3aecd0 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58b293ca0 con 0x55d58b3cf5c0 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d58b2e66c0 con 0x55d58b386950 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b386d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b386d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60626/0 (socket says 172.21.15.165:60626) 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.449+0000 7fc6f0a00640 1 -- 172.21.15.165:0/2462814781 learned_addr learned my addr 172.21.15.165:0/2462814781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f1400640 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b386950 0x55d58b3ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3cf5c0 0x55d58b385110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f0a00640 1 -- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b386950 msgr2=0x55d58b3ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b386950 0x55d58b3ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f0a00640 1 -- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3cf5c0 msgr2=0x55d58b385110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3cf5c0 0x55d58b385110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f0a00640 1 -- 172.21.15.165:0/2462814781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d58b29af40 con 0x55d58b3aecd0 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3cf5c0 0x55d58b385110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b386d30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80077b0 tx=0x7fc6d8007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9135b1e4738fba0b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6ddc00640 1 -- 172.21.15.165:0/2462814781 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800d950 con 0x55d58b3aecd0 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6ddc00640 1 -- 172.21.15.165:0/2462814781 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8018070 con 0x55d58b3aecd0 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.450+0000 7fc6ddc00640 1 -- 172.21.15.165:0/2462814781 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d800dd50 con 0x55d58b3aecd0 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/2462814781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 msgr2=0x55d58b386d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b386d30 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fc6d80077b0 tx=0x7fc6d8007c70 comp rx=0 tx=0).stop 2024-04-25T10:03:47.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/2462814781 shutdown_connections 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3cf5c0 0x55d58b385110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b386d30 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/2462814781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b386950 0x55d58b3ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/2462814781 >> 172.21.15.165:0/2462814781 conn(0x55d58b383b70 msgr2=0x55d58b3cc550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/2462814781 shutdown_connections 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.451+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/2462814781 wait complete. 2024-04-25T10:03:47.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.452+0000 7fc6f6ba6740 1 -- start start 2024-04-25T10:03:47.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.452+0000 7fc6f6ba6740 1 Processor -- start 2024-04-25T10:03:47.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3aecd0 0x55d58b3df980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b3aced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3a8060 0x55d58b3a84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58b3ae300 con 0x55d58b3cf5c0 2024-04-25T10:03:47.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58b2891f0 con 0x55d58b3aecd0 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d58b3adcf0 con 0x55d58b3a8060 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f6ba6740 10 client.0 _ll_register_callbacks cb 0x55d58b2a6ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b3aced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b3aced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60630/0 (socket says 172.21.15.165:60630) 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.453+0000 7fc6f0a00640 1 -- 172.21.15.165:0/3849443151 learned_addr learned my addr 172.21.15.165:0/3849443151 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f1400640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3aecd0 0x55d58b3df980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3a8060 0x55d58b3a84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f0a00640 1 -- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3a8060 msgr2=0x55d58b3a84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3a8060 0x55d58b3a84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f0a00640 1 -- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3aecd0 msgr2=0x55d58b3df980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3aecd0 0x55d58b3df980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f0a00640 1 -- 172.21.15.165:0/3849443151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d58b2a8c50 con 0x55d58b3cf5c0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f1400640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3aecd0 0x55d58b3df980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b3aced0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8006e80 tx=0x7fc6d8010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ae12dd21faf0fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.454+0000 7fc6d2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6d2000640 1 -- 172.21.15.165:0/3849443151 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8007400 con 0x55d58b3cf5c0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6d8018070 con 0x55d58b3cf5c0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6d2000640 1 -- 172.21.15.165:0/3849443151 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc6d8013680 con 0x55d58b3cf5c0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6d2000640 1 -- 172.21.15.165:0/3849443151 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6d802f050 con 0x55d58b3cf5c0 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 10 client.5493 fetch_fsmap learned FSMap version 19 2024-04-25T10:03:47.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x55d58afff2d0 con 0x55d58b3cf5c0 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6d2000640 1 -- 172.21.15.165:0/3849443151 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6d8018070 con 0x55d58b3cf5c0 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6d2000640 1 -- 172.21.15.165:0/3849443151 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc6d8032020 con 0x55d58b3cf5c0 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 10 client.5493 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 10 client.5493 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 20 client.5493 populate_metadata read hostname 'smithi165' 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3aa960 0x55d58b3aad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 4 client.5493 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.455+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d58b2f7b60 con 0x55d58b3aa960 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.456+0000 7fc6f1400640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3aa960 0x55d58b3aad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.456+0000 7fc6f1400640 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3aa960 0x55d58b3aad40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=316afaed0754857f server_cookie=f8aefe976ee0ef84 in_seq=0 out_seq=0 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.456+0000 7fc6d2000640 10 client.5493 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:03:47.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.457+0000 7fc6d2000640 1 -- 172.21.15.165:0/3849443151 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d58b2f7b60 con 0x55d58b3aa960 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.457+0000 7fc6d2000640 10 client.5493 handle_command_reply: tid=0 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.457+0000 7fc6f6ba6740 1 client.5493 shutdown 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 msgr2=0x55d58b3aced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b3aced0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8006e80 tx=0x7fc6d8010000 comp rx=0 tx=0).stop 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 shutdown_connections 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3a8060 0x55d58b3a84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3aa960 0x55d58b3aad40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b3aced0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/3849443151 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3aecd0 0x55d58b3df980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6d2000640 0 client.5493 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:03:47.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 >> 172.21.15.165:0/3849443151 conn(0x55d58b386950 msgr2=0x55d58b3cfa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 shutdown_connections 2024-04-25T10:03:47.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/3849443151 wait complete. 2024-04-25T10:03:47.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.458+0000 7fc6f6ba6740 20 client.5493 trim_cache size 0 max 16384 2024-04-25T10:03:47.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.475+0000 7fc6f6ba6740 1 Processor -- start 2024-04-25T10:03:47.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.475+0000 7fc6f6ba6740 1 -- start start 2024-04-25T10:03:47.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.476+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b3aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.476+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3aecd0 0x55d58b385110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.476+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b385650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.476+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec200570 con 0x55d58b3cf5c0 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.476+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1ff6f0 con 0x55d58b3a8060 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.476+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1f4c60 con 0x55d58b3aecd0 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b385650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b385650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60636/0 (socket says 172.21.15.165:60636) 2024-04-25T10:03:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 -- 172.21.15.165:0/249114508 learned_addr learned my addr 172.21.15.165:0/249114508 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3aecd0 0x55d58b385110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1400640 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b3aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 -- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3aecd0 msgr2=0x55d58b385110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3aecd0 0x55d58b385110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 -- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 msgr2=0x55d58b3aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b3aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 -- 172.21.15.165:0/249114508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d58b3af420 con 0x55d58b3cf5c0 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.477+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b385650 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4058550 tx=0x7fc6e40585c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe837f8b5c4b1807 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6dd200640 1 -- 172.21.15.165:0/249114508 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4070020 con 0x55d58b3cf5c0 2024-04-25T10:03:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6dd200640 1 -- 172.21.15.165:0/249114508 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e40568c0 con 0x55d58b3cf5c0 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6dd200640 1 -- 172.21.15.165:0/249114508 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4060840 con 0x55d58b3cf5c0 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/249114508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 msgr2=0x55d58b385650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b385650 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fc6e4058550 tx=0x7fc6e40585c0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/249114508 shutdown_connections 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3cf5c0 0x55d58b385650 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3aecd0 0x55d58b385110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/249114508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b3aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.478+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/249114508 >> 172.21.15.165:0/249114508 conn(0x55d58b386950 msgr2=0x55d58b3a2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.479+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/249114508 shutdown_connections 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.479+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/249114508 wait complete. 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.479+0000 7fc6f6ba6740 1 -- start start 2024-04-25T10:03:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.479+0000 7fc6f6ba6740 1 Processor -- start 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.480+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b385110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.480+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b3a97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.480+0000 7fc6f6ba6740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3cf5c0 0x55d58b3a6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.480+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d58b3dc9f0 con 0x55d58b3aecd0 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.480+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6ec1ff0f0 con 0x55d58b3a8060 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f6ba6740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d58b3a52c0 con 0x55d58b3cf5c0 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f6ba6740 10 client.0 _ll_register_callbacks cb 0x7fc6d8007630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b3a97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b3a97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60646/0 (socket says 172.21.15.165:60646) 2024-04-25T10:03:47.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 -- 172.21.15.165:0/1078729736 learned_addr learned my addr 172.21.15.165:0/1078729736 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f1e00640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3cf5c0 0x55d58b3a6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f1400640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b385110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 -- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3cf5c0 msgr2=0x55d58b3a6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3cf5c0 0x55d58b3a6a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 -- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 msgr2=0x55d58b385110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b385110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.481+0000 7fc6f0a00640 1 -- 172.21.15.165:0/1078729736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d58b0c96b0 con 0x55d58b3aecd0 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6f0a00640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b3a97f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8007700 tx=0x7fc6d8006e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145312396257079f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6d2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:03:47.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6d2000640 1 -- 172.21.15.165:0/1078729736 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6d8037070 con 0x55d58b3aecd0 2024-04-25T10:03:47.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc6e40568c0 con 0x55d58b3aecd0 2024-04-25T10:03:47.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6d2000640 1 -- 172.21.15.165:0/1078729736 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7fc6d8004040 con 0x55d58b3aecd0 2024-04-25T10:03:47.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6d2000640 1 -- 172.21.15.165:0/1078729736 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=19) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6d802c020 con 0x55d58b3aecd0 2024-04-25T10:03:47.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6f6ba6740 10 client.5499 fetch_fsmap learned FSMap version 19 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=19,monmap=2+}) v3 -- 0x55d58b3a1d60 con 0x55d58b3aecd0 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6d2000640 1 -- 172.21.15.165:0/1078729736 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e40568c0 con 0x55d58b3aecd0 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.482+0000 7fc6d2000640 1 -- 172.21.15.165:0/1078729736 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 19) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc6d8032050 con 0x55d58b3aecd0 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f6ba6740 10 client.5499 fetch_fsmap finished waiting for FSMap version 19 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f6ba6740 10 client.5499 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f6ba6740 20 client.5499 populate_metadata read hostname 'smithi165' 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3a6f50 0x55d58b3e57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f6ba6740 4 client.5499 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d58b2f1eb0 con 0x55d58b3a6f50 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f1400640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3a6f50 0x55d58b3e57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6f1400640 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3a6f50 0x55d58b3e57c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9bfc19871825f04 server_cookie=3d8a680b1d40b840 in_seq=0 out_seq=0 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.483+0000 7fc6d2000640 10 client.5499 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6d2000640 1 -- 172.21.15.165:0/1078729736 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55d58b2f1eb0 con 0x55d58b3a6f50 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6d2000640 10 client.5499 handle_command_reply: tid=0 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 client.5499 shutdown 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 msgr2=0x55d58b3a97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b3a97f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fc6d8007700 tx=0x7fc6d8006e10 comp rx=0 tx=0).stop 2024-04-25T10:03:47.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 shutdown_connections 2024-04-25T10:03:47.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d58b3cf5c0 0x55d58b3a6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55d58b3a6f50 0x55d58b3e57c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d58b3aecd0 0x55d58b3a97f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 --2- 172.21.15.165:0/1078729736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d58b3a8060 0x55d58b385110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6d2000640 0 client.5499 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:03:47.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 >> 172.21.15.165:0/1078729736 conn(0x55d58b386950 msgr2=0x55d58b3a5a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 shutdown_connections 2024-04-25T10:03:47.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 1 -- 172.21.15.165:0/1078729736 wait complete. 2024-04-25T10:03:47.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.484+0000 7fc6f6ba6740 20 client.5499 trim_cache size 0 max 16384 2024-04-25T10:03:47.498 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 5337, 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.6031923470000002, 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 19, 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 41, 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 39, 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 15.786402776999999 2024-04-25T10:03:47.499 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:03:47.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6cc06b500 msgr2=0x7fc6cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6cc06b500 0x7fc6cc06d9c0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0034000 tx=0x7fc6e002e000 comp rx=0 tx=0).stop 2024-04-25T10:03:47.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 msgr2=0x7fc6ec13d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 0x7fc6ec13d840 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400b970 tx=0x7fc6e400be30 comp rx=0 tx=0).stop 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 shutdown_connections 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6cc06b500 0x7fc6cc06d9c0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6ec1452a0 0x7fc6ec13d840 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6ec144d50 0x7fc6ec1f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 --2- 172.21.15.165:0/2659687749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6ec1446d0 0x7fc6ec0afd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 >> 172.21.15.165:0/2659687749 conn(0x7fc6ec06abe0 msgr2=0x7fc6ec143870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 shutdown_connections 2024-04-25T10:03:47.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.485+0000 7fc6f3e00640 1 -- 172.21.15.165:0/2659687749 wait complete. 2024-04-25T10:03:47.507 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:03:47.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.699+0000 7f9246600640 1 Processor -- start 2024-04-25T10:03:47.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.699+0000 7f9246600640 1 -- start start 2024-04-25T10:03:47.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9246600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f924015e520 0x7f924015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9246600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f924015f040 0x7f92400adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9246600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92400ae590 con 0x7f92400ae0d0 2024-04-25T10:03:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9246600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400aee30 con 0x7f924015e520 2024-04-25T10:03:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9246600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92400ae870 con 0x7f924015f040 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9245200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f924015e520 0x7f924015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f923fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f924015f040 0x7f92400adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f923fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f924015f040 0x7f92400adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:34730/0 (socket says 172.21.15.165:34730) 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f9245200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f924015e520 0x7f924015e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60648/0 (socket says 172.21.15.165:60648) 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.701+0000 7f923fe00640 1 -- 172.21.15.165:0/596100400 learned_addr learned my addr 172.21.15.165:0/596100400 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.702+0000 7f9245200640 1 -- 172.21.15.165:0/596100400 >> v2:172.21.15.172:3300/0 conn(0x7f924015f040 msgr2=0x7f92400adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.702+0000 7f9245200640 1 --2- 172.21.15.165:0/596100400 >> v2:172.21.15.172:3300/0 conn(0x7f924015f040 0x7f92400adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.702+0000 7f9245200640 1 -- 172.21.15.165:0/596100400 >> v1:172.21.15.172:6789/0 conn(0x7f92400ae0d0 legacy=0x7f9240070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.702+0000 7f9245200640 1 -- 172.21.15.165:0/596100400 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92400aeb50 con 0x7f924015e520 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.704+0000 7f9245200640 1 --2- 172.21.15.165:0/596100400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924015e520 0x7f924015e900 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f92400cf050 tx=0x7f923402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44495c62477c26f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f923f400640 1 -- 172.21.15.165:0/596100400 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234037070 con 0x7f924015e520 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f923f400640 1 -- 172.21.15.165:0/596100400 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234032440 con 0x7f924015e520 2024-04-25T10:03:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f923f400640 1 -- 172.21.15.165:0/596100400 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234032810 con 0x7f924015e520 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f9246600640 1 -- 172.21.15.165:0/596100400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924015e520 msgr2=0x7f924015e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f9246600640 1 --2- 172.21.15.165:0/596100400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924015e520 0x7f924015e900 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f92400cf050 tx=0x7f923402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f9246600640 1 -- 172.21.15.165:0/596100400 shutdown_connections 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f9246600640 1 --2- 172.21.15.165:0/596100400 >> v2:172.21.15.172:3300/0 conn(0x7f924015f040 0x7f92400adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f9246600640 1 --2- 172.21.15.165:0/596100400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f924015e520 0x7f924015e900 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.705+0000 7f9246600640 1 -- 172.21.15.165:0/596100400 >> 172.21.15.165:0/596100400 conn(0x7f924006abe0 msgr2=0x7f924006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.706+0000 7f9246600640 1 -- 172.21.15.165:0/596100400 shutdown_connections 2024-04-25T10:03:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.706+0000 7f9246600640 1 -- 172.21.15.165:0/596100400 wait complete. 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.707+0000 7f9246600640 1 Processor -- start 2024-04-25T10:03:47.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.707+0000 7f9246600640 1 -- start start 2024-04-25T10:03:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.708+0000 7f9246600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 0x7f92401593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.708+0000 7f9246600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924015e520 0x7f92400b3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.708+0000 7f9246600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924015f040 0x7f92400b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.708+0000 7f9246600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924015f650 con 0x7f92400ae0d0 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.708+0000 7f9246600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924015f4d0 con 0x7f924015e520 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.708+0000 7f9246600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9240071700 con 0x7f924015f040 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 0x7f92401593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 0x7f92401593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60660/0 (socket says 172.21.15.165:60660) 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 -- 172.21.15.165:0/2676913462 learned_addr learned my addr 172.21.15.165:0/2676913462 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f923fe00640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924015e520 0x7f92400b3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 -- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924015f040 msgr2=0x7f92400b4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924015f040 0x7f92400b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 -- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924015e520 msgr2=0x7f92400b3ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924015e520 0x7f92400b3ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 -- 172.21.15.165:0/2676913462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9240159e30 con 0x7f92400ae0d0 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.709+0000 7f9245200640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 0x7f92401593e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f923402a1e0 tx=0x7f92340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.710+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234037040 con 0x7f92400ae0d0 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.710+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92340304c0 con 0x7f92400ae0d0 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.710+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234030890 con 0x7f92400ae0d0 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.710+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924015a110 con 0x7f92400ae0d0 2024-04-25T10:03:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.710+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924015a6d0 con 0x7f92400ae0d0 2024-04-25T10:03:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.711+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9240073990 con 0x7f92400ae0d0 2024-04-25T10:03:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.712+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9234044020 con 0x7f92400ae0d0 2024-04-25T10:03:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.713+0000 7f923cc00640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922006b4d0 0x7f922006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:47.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.713+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f92340aa770 con 0x7f92400ae0d0 2024-04-25T10:03:47.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.713+0000 7f923fe00640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922006b4d0 0x7f922006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:47.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.714+0000 7f923fe00640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922006b4d0 0x7f922006d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9240160cc0 tx=0x7f922c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:47.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.718+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92340793a0 con 0x7f92400ae0d0 2024-04-25T10:03:47.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.955+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f92400af2d0 con 0x7f92400ae0d0 2024-04-25T10:03:47.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.955+0000 7f923cc00640 1 -- 172.21.15.165:0/2676913462 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v19) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9234078ca0 con 0x7f92400ae0d0 2024-04-25T10:03:47.960 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:03:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922006b4d0 msgr2=0x7f922006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922006b4d0 0x7f922006d990 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9240160cc0 tx=0x7f922c008040 comp rx=0 tx=0).stop 2024-04-25T10:03:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 msgr2=0x7f92401593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 0x7f92401593e0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f923402a1e0 tx=0x7f92340046c0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 shutdown_connections 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922006b4d0 0x7f922006d990 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924015f040 0x7f92400b4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924015e520 0x7f92400b3ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 --2- 172.21.15.165:0/2676913462 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92400ae0d0 0x7f92401593e0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 >> 172.21.15.165:0/2676913462 conn(0x7f924006abe0 msgr2=0x7f92400b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 shutdown_connections 2024-04-25T10:03:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:47.958+0000 7f9246600640 1 -- 172.21.15.165:0/2676913462 wait complete. 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:03:47.982 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:47.983 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:47.983 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:03:48.034 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:03:48.034 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:03:48.034 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:03:48.034 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:48.035 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42537sec preferred_lft 42537sec 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:48.036 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:03:48.037 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:03:48.037 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.037 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:03:48.037 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:03:48.037 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:03:48.037 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:03:48.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.038 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:03:48.116 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:03:48.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.121 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:03:48.174 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:03:48.174 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:03:48.175 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:03:48.175 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:03:48.175 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:03:48.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.176 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:03:48.176 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:03:48.176 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:03:48.176 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:03:48.176 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:03:48.176 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:03:48.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.325 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:03:48.374 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:03:48.374 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:03:48.374 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:48.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.374 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:03:48.447 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:48.447 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:48.497 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:03:48.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:03:48.497 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:48.545 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:03:48.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:48.716 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:48.716 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:03:48.717 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:03:48.740 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:48.740 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:03:48.740 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:03:48.740 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:03:48.740 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:03:48.766 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:03:48.766 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:03:48.766 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:03:48.766 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:03:48.767 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:03:48.767 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:03:48.767 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:03:48.767 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:03:48.769 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:03:48.769 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:03:48.769 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:03:48.769 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:03:48.769 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:03:48.772 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:03:48.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.772 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:03:48.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:48.842 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:03:48.906 INFO:teuthology.orchestra.run.smithi165.stdout:client5508 2024-04-25T10:03:48.907 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:48.907 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:48.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:03:49.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.196+0000 7f0c64a00640 1 Processor -- start 2024-04-25T10:03:49.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.197+0000 7f0c64a00640 1 -- start start 2024-04-25T10:03:49.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.198+0000 7f0c64a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c60145140 0x7f0c600ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.198+0000 7f0c64a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0c600c5200 0x7f0c6015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.198+0000 7f0c64a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c600c5670 con 0x7f0c600ce950 2024-04-25T10:03:49.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.198+0000 7f0c64a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c600ced90 con 0x7f0c600c5200 2024-04-25T10:03:49.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c64a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60145520 con 0x7f0c60145140 2024-04-25T10:03:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0c600c5200 0x7f0c6015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0c600c5200 0x7f0c6015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60664/0 (socket says 172.21.15.165:60664) 2024-04-25T10:03:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 -- 172.21.15.165:0/621846890 learned_addr learned my addr 172.21.15.165:0/621846890 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5f400640 1 --2- 172.21.15.165:0/621846890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c60145140 0x7f0c600ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 -- 172.21.15.165:0/621846890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c60145140 msgr2=0x7f0c600ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/621846890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c60145140 0x7f0c600ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 -- 172.21.15.165:0/621846890 >> v1:172.21.15.172:6789/0 conn(0x7f0c600ce950 legacy=0x7f0c600c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:03:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.199+0000 7f0c5fe00640 1 -- 172.21.15.165:0/621846890 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c60071270 con 0x7f0c600c5200 2024-04-25T10:03:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.201+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/621846890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c600c5200 0x7f0c6015bfb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009e50 tx=0x7f0c5802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdd4ca1cc317840 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.201+0000 7f0c5ea00640 1 -- 172.21.15.165:0/621846890 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58036070 con 0x7f0c600c5200 2024-04-25T10:03:49.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.201+0000 7f0c5ea00640 1 -- 172.21.15.165:0/621846890 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c58031600 con 0x7f0c600c5200 2024-04-25T10:03:49.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.201+0000 7f0c5ea00640 1 -- 172.21.15.165:0/621846890 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580319d0 con 0x7f0c600c5200 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 -- 172.21.15.165:0/621846890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c600c5200 msgr2=0x7f0c6015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 --2- 172.21.15.165:0/621846890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c600c5200 0x7f0c6015bfb0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009e50 tx=0x7f0c5802a630 comp rx=0 tx=0).stop 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 -- 172.21.15.165:0/621846890 shutdown_connections 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 --2- 172.21.15.165:0/621846890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c600c5200 0x7f0c6015bfb0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 --2- 172.21.15.165:0/621846890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c60145140 0x7f0c600ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 -- 172.21.15.165:0/621846890 >> 172.21.15.165:0/621846890 conn(0x7f0c60061cc0 msgr2=0x7f0c6006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 -- 172.21.15.165:0/621846890 shutdown_connections 2024-04-25T10:03:49.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.202+0000 7f0c64a00640 1 -- 172.21.15.165:0/621846890 wait complete. 2024-04-25T10:03:49.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.204+0000 7f0c64a00640 1 Processor -- start 2024-04-25T10:03:49.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.204+0000 7f0c64a00640 1 -- start start 2024-04-25T10:03:49.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c64a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600c5200 0x7f0c6015b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c64a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c600ce950 0x7f0c600ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c64a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 0x7f0c6013e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c64a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60071b10 con 0x7f0c60145140 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c64a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c60071830 con 0x7f0c600ce950 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c64a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015c4f0 con 0x7f0c600c5200 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 0x7f0c6013e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 0x7f0c6013e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60670/0 (socket says 172.21.15.165:60670) 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.205+0000 7f0c5fe00640 1 -- 172.21.15.165:0/1925010311 learned_addr learned my addr 172.21.15.165:0/1925010311 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5f400640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600c5200 0x7f0c6015b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c57e00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c600ce950 0x7f0c600ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5fe00640 1 -- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600c5200 msgr2=0x7f0c6015b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600c5200 0x7f0c6015b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5fe00640 1 -- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c600ce950 msgr2=0x7f0c600ccbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c600ce950 0x7f0c600ccbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:49.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5fe00640 1 -- 172.21.15.165:0/1925010311 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c600cd5f0 con 0x7f0c60145140 2024-04-25T10:03:49.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 0x7f0c6013e670 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009f70 tx=0x7f0c58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.206+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58036050 con 0x7f0c60145140 2024-04-25T10:03:49.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.207+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c58042070 con 0x7f0c60145140 2024-04-25T10:03:49.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.207+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58009510 con 0x7f0c60145140 2024-04-25T10:03:49.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.207+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c600cd8d0 con 0x7f0c60145140 2024-04-25T10:03:49.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.207+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c601f7910 con 0x7f0c60145140 2024-04-25T10:03:49.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.208+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c600cdcc0 con 0x7f0c60145140 2024-04-25T10:03:49.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.209+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c58007bd0 con 0x7f0c60145140 2024-04-25T10:03:49.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.210+0000 7f0c57400640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c3c06b4d0 0x7f0c3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:03:49.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.210+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f0c580ab3b0 con 0x7f0c60145140 2024-04-25T10:03:49.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.210+0000 7f0c5f400640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c3c06b4d0 0x7f0c3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:03:49.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.211+0000 7f0c5f400640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c3c06b4d0 0x7f0c3c06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c50006fd0 tx=0x7f0c50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:03:49.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.215+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c58045360 con 0x7f0c60145140 2024-04-25T10:03:49.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:49.452+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0c60068490 con 0x7f0c60145140 2024-04-25T10:03:50.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.032+0000 7f0c57400640 1 -- 172.21.15.165:0/1925010311 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v20) v1 ==== 167+0+0 (secure 0 0 0) 0x7f0c60068490 con 0x7f0c60145140 2024-04-25T10:03:50.038 INFO:teuthology.orchestra.run.smithi165.stderr:enabled new snapshots 2024-04-25T10:03:50.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c3c06b4d0 msgr2=0x7f0c3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c3c06b4d0 0x7f0c3c06d990 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0c50006fd0 tx=0x7f0c50008040 comp rx=0 tx=0).stop 2024-04-25T10:03:50.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 msgr2=0x7f0c6013e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:03:50.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 0x7f0c6013e670 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f0c58009f70 tx=0x7f0c58004a40 comp rx=0 tx=0).stop 2024-04-25T10:03:50.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 shutdown_connections 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c3c06b4d0 0x7f0c3c06d990 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60145140 0x7f0c6013e670 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c600ce950 0x7f0c600ccbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 --2- 172.21.15.165:0/1925010311 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600c5200 0x7f0c6015b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 >> 172.21.15.165:0/1925010311 conn(0x7f0c60061cc0 msgr2=0x7f0c600c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.036+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 shutdown_connections 2024-04-25T10:03:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:03:50.037+0000 7f0c64a00640 1 -- 172.21.15.165:0/1925010311 wait complete. 2024-04-25T10:03:50.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:03:50.062 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:03:50.123 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:03:50.123 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> import uuid 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> n = 2000 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> create_hard_links = False 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> if 3 > 0: 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> create_hard_links = True 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> dpath = os.path.dirname(path) 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> fnameprefix = os.path.basename(path) 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:03:50.174 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> for i in range(n): 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> f.write(f"{i}") 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> f.flush() 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(f.fileno()) 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> os.unlink(fpath) 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(dirfd) 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> if create_hard_links: 2024-04-25T10:03:50.175 DEBUG:teuthology.orchestra.run.smithi165:> for j in range(3): 2024-04-25T10:03:50.176 DEBUG:teuthology.orchestra.run.smithi165:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:03:50.176 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:03:50.176 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(dirfd) 2024-04-25T10:03:50.176 DEBUG:teuthology.orchestra.run.smithi165:> finally: 2024-04-25T10:03:50.176 DEBUG:teuthology.orchestra.run.smithi165:> os.close(dirfd) 2024-04-25T10:03:50.176 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:03:56.766 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:03:56.771 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:26.897 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:26.904 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:36.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:04:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:04:36.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:36.399 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:04:36.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:04:36.498 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:04:57.026 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:04:57.069 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:27.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:27.220 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:30.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:30.864 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:05:30.947 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:05:30.948 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:05:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.153+0000 7f59e1400640 1 Processor -- start 2024-04-25T10:05:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.154+0000 7f59e1400640 1 -- start start 2024-04-25T10:05:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.155+0000 7f59e1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59dc06bff0 0x7f59dc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.155+0000 7f59e1400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc15e110 con 0x7f59dc15dd30 2024-04-25T10:05:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.155+0000 7f59e1400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59dc0b2fd0 con 0x7f59dc0b5970 2024-04-25T10:05:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.155+0000 7f59e1400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59dc06d000 con 0x7f59dc06bff0 2024-04-25T10:05:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.155+0000 7f59dbe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f59dc15dd30 0x7f59dc0cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47886/0 (socket says 172.21.15.165:47886) 2024-04-25T10:05:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.155+0000 7f59dbe00640 1 -- 172.21.15.165:0/1209953550 learned_addr learned my addr 172.21.15.165:0/1209953550 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:05:31.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2320895016 0 0) 0x7f59dc15e110 con 0x7f59dc15dd30 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59bc003520 con 0x7f59dc15dd30 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59db400640 1 --2- 172.21.15.165:0/1209953550 >> v2:172.21.15.172:3300/0 conn(0x7f59dc06bff0 0x7f59dc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1742972239 0 0) 0x7f59bc003520 con 0x7f59dc15dd30 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 >> v2:172.21.15.172:3300/0 conn(0x7f59dc06bff0 msgr2=0x7f59dc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 --2- 172.21.15.165:0/1209953550 >> v2:172.21.15.172:3300/0 conn(0x7f59dc06bff0 0x7f59dc0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 >> v1:172.21.15.172:6789/0 conn(0x7f59dc0b5970 legacy=0x7f59dc0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc06c480 con 0x7f59dc15dd30 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f59cc001670 con 0x7f59dc15dd30 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 >> v1:172.21.15.165:6789/0 conn(0x7f59dc15dd30 legacy=0x7f59dc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59bc0039b0 0x7f59bc003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59bc004800 0x7f59bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc0051c0 0x7f59bc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc06d000 con 0x7f59bc004800 2024-04-25T10:05:31.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00c0f0 con 0x7f59bc0051c0 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.156+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59bc00c2f0 con 0x7f59bc0039b0 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59db400640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59bc004800 0x7f59bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59dbe00640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59bc0039b0 0x7f59bc003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59e0a00640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc0051c0 0x7f59bc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59db400640 1 -- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59bc0039b0 msgr2=0x7f59bc003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59db400640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59bc0039b0 0x7f59bc003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59db400640 1 -- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc0051c0 msgr2=0x7f59bc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59db400640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc0051c0 0x7f59bc00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59db400640 1 -- 172.21.15.165:0/1209953550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc003520 con 0x7f59bc004800 2024-04-25T10:05:31.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.157+0000 7f59e0a00640 1 -- 172.21.15.165:0/1209953550 reap_dead start 2024-04-25T10:05:31.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.158+0000 7f59db400640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59bc004800 0x7f59bc004c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f59c4007330 tx=0x7f59bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33715653ddf0a859 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.158+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4033070 con 0x7f59bc004800 2024-04-25T10:05:31.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.158+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59c402f650 con 0x7f59bc004800 2024-04-25T10:05:31.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.158+0000 7f59daa00640 1 -- 172.21.15.165:0/1209953550 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4009be0 con 0x7f59bc004800 2024-04-25T10:05:31.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 -- 172.21.15.165:0/1209953550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59bc004800 msgr2=0x7f59bc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59bc004800 0x7f59bc004c80 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f59c4007330 tx=0x7f59bc005830 comp rx=0 tx=0).stop 2024-04-25T10:05:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 -- 172.21.15.165:0/1209953550 shutdown_connections 2024-04-25T10:05:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 --2- 172.21.15.165:0/1209953550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59bc004800 0x7f59bc004c80 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 -- 172.21.15.165:0/1209953550 >> 172.21.15.165:0/1209953550 conn(0x7f59dc0d0610 msgr2=0x7f59dc0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 -- 172.21.15.165:0/1209953550 shutdown_connections 2024-04-25T10:05:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.159+0000 7f59e1400640 1 -- 172.21.15.165:0/1209953550 wait complete. 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.161+0000 7f59e1400640 1 Processor -- start 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.161+0000 7f59e1400640 1 -- start start 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e1400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc004800 0x7f59dc157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e1400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59dc1f34c0 0x7f59dc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 0x7f59dc1567c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e1400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc06cd20 con 0x7f59dc15a570 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e1400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc06ca40 con 0x7f59bc004800 2024-04-25T10:05:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e1400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc0b5430 con 0x7f59dc1f34c0 2024-04-25T10:05:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 0x7f59dc1567c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 0x7f59dc1567c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42276/0 (socket says 172.21.15.165:42276) 2024-04-25T10:05:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.162+0000 7f59e0a00640 1 -- 172.21.15.165:0/1502244250 learned_addr learned my addr 172.21.15.165:0/1502244250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59db400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59dc1f34c0 0x7f59dc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59dbe00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc004800 0x7f59dc157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e0a00640 1 -- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59dc1f34c0 msgr2=0x7f59dc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e0a00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59dc1f34c0 0x7f59dc15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e0a00640 1 -- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc004800 msgr2=0x7f59dc157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e0a00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc004800 0x7f59dc157e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e0a00640 1 -- 172.21.15.165:0/1502244250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59dc15a9b0 con 0x7f59dc15a570 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59db400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59dc1f34c0 0x7f59dc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e0a00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 0x7f59dc1567c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0b93c0 tx=0x7f59d400d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59d3e00640 1 -- 172.21.15.165:0/1502244250 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4005230 con 0x7f59dc15a570 2024-04-25T10:05:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59d3e00640 1 -- 172.21.15.165:0/1502244250 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d40053d0 con 0x7f59dc15a570 2024-04-25T10:05:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59d3e00640 1 -- 172.21.15.165:0/1502244250 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4012840 con 0x7f59dc15a570 2024-04-25T10:05:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.163+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59dc15f660 con 0x7f59dc15a570 2024-04-25T10:05:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.164+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59dc15fc20 con 0x7f59dc15a570 2024-04-25T10:05:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.166+0000 7f59d3e00640 1 -- 172.21.15.165:0/1502244250 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59d40129e0 con 0x7f59dc15a570 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.166+0000 7f59d3e00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59b806b4d0 0x7f59b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.167+0000 7f59d3e00640 1 -- 172.21.15.165:0/1502244250 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f59d4089ec0 con 0x7f59dc15a570 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.167+0000 7f59dbe00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59b806b4d0 0x7f59b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.167+0000 7f59e59c5740 1 Processor -- start 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.167+0000 7f59e59c5740 1 -- start start 2024-04-25T10:05:31.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.167+0000 7f59dbe00640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59b806b4d0 0x7f59b806d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0017e0 tx=0x7f59cc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.168+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc054a870 0x560bc054ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.168+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc051aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.168+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 0x560bc0542cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.168+0000 7f59e59c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc0483870 con 0x560bc051afb0 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.168+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc042cd60 con 0x560bc054b290 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.168+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bc047f910 con 0x560bc054a870 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 0x560bc0542cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 0x560bc0542cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42292/0 (socket says 172.21.15.165:42292) 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 -- 172.21.15.165:0/476825074 learned_addr learned my addr 172.21.15.165:0/476825074 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59db400640 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc051aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 -- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc054a870 msgr2=0x560bc054ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc054a870 0x560bc054ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 -- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 msgr2=0x560bc051aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc051aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 -- 172.21.15.165:0/476825074 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bc02627d0 con 0x560bc051afb0 2024-04-25T10:05:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59e0a00640 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 0x560bc0542cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f59d4088170 tx=0x7f59d4022310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd01d42392188d79 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.169+0000 7f59cbe00640 1 -- 172.21.15.165:0/476825074 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d405c5d0 con 0x560bc051afb0 2024-04-25T10:05:31.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59cbe00640 1 -- 172.21.15.165:0/476825074 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4060b90 con 0x560bc051afb0 2024-04-25T10:05:31.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59cbe00640 1 -- 172.21.15.165:0/476825074 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4054af0 con 0x560bc051afb0 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 -- 172.21.15.165:0/476825074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 msgr2=0x560bc0542cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 0x560bc0542cf0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f59d4088170 tx=0x7f59d4022310 comp rx=0 tx=0).stop 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 -- 172.21.15.165:0/476825074 shutdown_connections 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc051afb0 0x560bc0542cf0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc051aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 --2- 172.21.15.165:0/476825074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc054a870 0x560bc054ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 -- 172.21.15.165:0/476825074 >> 172.21.15.165:0/476825074 conn(0x560bc051cef0 msgr2=0x560bc0565680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 -- 172.21.15.165:0/476825074 shutdown_connections 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.170+0000 7f59e59c5740 1 -- 172.21.15.165:0/476825074 wait complete. 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.171+0000 7f59e59c5740 1 -- start start 2024-04-25T10:05:31.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.171+0000 7f59e59c5740 1 Processor -- start 2024-04-25T10:05:31.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc0549850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0542d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0541cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc0546fc0 con 0x560bc054a870 2024-04-25T10:05:31.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc04401c0 con 0x560bc054b290 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bc0547260 con 0x560bc0541850 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59e59c5740 10 client.0 _ll_register_callbacks cb 0x560bc047e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.172+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc0549850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc0549850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42298/0 (socket says 172.21.15.165:42298) 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 -- 172.21.15.165:0/2029569271 learned_addr learned my addr 172.21.15.165:0/2029569271 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59db400640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0542d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59e0a00640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0541cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 -- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 msgr2=0x560bc0541cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0541cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 -- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 msgr2=0x560bc0542d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0542d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 -- 172.21.15.165:0/2029569271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bc01580f0 con 0x560bc054a870 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59db400640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0542d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59dbe00640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc0549850 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f59cc005ad0 tx=0x7f59cc005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b971f52eeb83d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.173+0000 7f59c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59c8c00640 1 -- 172.21.15.165:0/2029569271 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59cc013680 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59c8c00640 1 -- 172.21.15.165:0/2029569271 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f59cc023c80 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59d4060b90 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59c8c00640 1 -- 172.21.15.165:0/2029569271 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59cc013820 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 10 client.5526 fetch_fsmap learned FSMap version 20 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x560bc02c2f60 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59c8c00640 1 -- 172.21.15.165:0/2029569271 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d4060b90 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59c8c00640 1 -- 172.21.15.165:0/2029569271 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f59cc02a050 con 0x560bc054a870 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 10 client.5526 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 10 client.5526 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 20 client.5526 populate_metadata read hostname 'smithi165' 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.174+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc054d760 0x560bc054db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.175+0000 7f59e59c5740 4 client.5526 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:31.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.175+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560bc0459690 con 0x560bc054d760 2024-04-25T10:05:31.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.175+0000 7f59db400640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc054d760 0x560bc054db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.175+0000 7f59db400640 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc054d760 0x560bc054db70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=beb51c484dd61744 server_cookie=4f2a46e4048081b5 in_seq=0 out_seq=0 2024-04-25T10:05:31.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.175+0000 7f59c8c00640 10 client.5526 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.176+0000 7f59c8c00640 1 -- 172.21.15.165:0/2029569271 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560bc0459690 con 0x560bc054d760 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.176+0000 7f59c8c00640 10 client.5526 handle_command_reply: tid=0 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.176+0000 7f59e59c5740 1 client.5526 shutdown 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.176+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 msgr2=0x560bc0549850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.176+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc0549850 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f59cc005ad0 tx=0x7f59cc005ed0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.176+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 shutdown_connections 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0541cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0542d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc054d760 0x560bc054db70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2029569271 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc0549850 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 >> 172.21.15.165:0/2029569271 conn(0x560bc051afb0 msgr2=0x560bc051a1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59c8c00640 0 client.5526 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 shutdown_connections 2024-04-25T10:05:31.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 1 -- 172.21.15.165:0/2029569271 wait complete. 2024-04-25T10:05:31.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.177+0000 7f59e59c5740 20 client.5526 trim_cache size 0 max 16384 2024-04-25T10:05:31.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.192+0000 7f59e59c5740 1 Processor -- start 2024-04-25T10:05:31.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.192+0000 7f59e59c5740 1 -- start start 2024-04-25T10:05:31.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.193+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0568a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.193+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054a870 0x560bc05c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.193+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 0x560bc054c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.193+0000 7f59e59c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc200a00 con 0x560bc054b290 2024-04-25T10:05:31.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.193+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc1ffbe0 con 0x560bc054a870 2024-04-25T10:05:31.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.193+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc1fefe0 con 0x560bc0541850 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 0x560bc054c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 0x560bc054c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42300/0 (socket says 172.21.15.165:42300) 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 -- 172.21.15.165:0/2904727606 learned_addr learned my addr 172.21.15.165:0/2904727606 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59dbe00640 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0568a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 -- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 msgr2=0x560bc0568a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0568a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 -- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054a870 msgr2=0x560bc05c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054a870 0x560bc05c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 -- 172.21.15.165:0/2904727606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560bc0542040 con 0x560bc054b290 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.194+0000 7f59e0a00640 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 0x560bc054c860 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f59d406f040 tx=0x7f59d401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa6a54c5cc21a63c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59cb400640 1 -- 172.21.15.165:0/2904727606 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4060b90 con 0x560bc054b290 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59cb400640 1 -- 172.21.15.165:0/2904727606 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d405c5d0 con 0x560bc054b290 2024-04-25T10:05:31.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59cb400640 1 -- 172.21.15.165:0/2904727606 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59d4054af0 con 0x560bc054b290 2024-04-25T10:05:31.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59e59c5740 1 -- 172.21.15.165:0/2904727606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 msgr2=0x560bc054c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 0x560bc054c860 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f59d406f040 tx=0x7f59d401c000 comp rx=0 tx=0).stop 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59e59c5740 1 -- 172.21.15.165:0/2904727606 shutdown_connections 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054b290 0x560bc054c860 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.195+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054a870 0x560bc05c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.196+0000 7f59e59c5740 1 --2- 172.21.15.165:0/2904727606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc0568a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.196+0000 7f59e59c5740 1 -- 172.21.15.165:0/2904727606 >> 172.21.15.165:0/2904727606 conn(0x560bc051afb0 msgr2=0x560bc0547660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.196+0000 7f59e59c5740 1 -- 172.21.15.165:0/2904727606 shutdown_connections 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.196+0000 7f59e59c5740 1 -- 172.21.15.165:0/2904727606 wait complete. 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.196+0000 7f59e59c5740 1 -- start start 2024-04-25T10:05:31.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.197+0000 7f59e59c5740 1 Processor -- start 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc05c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc05770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0568a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560bc0521120 con 0x560bc054a870 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59dc1ff5e0 con 0x560bc054b290 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560bc054df00 con 0x560bc0541850 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e59c5740 10 client.0 _ll_register_callbacks cb 0x7f59d4007e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc05770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc05770e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42302/0 (socket says 172.21.15.165:42302) 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59db400640 1 -- 172.21.15.165:0/38546178 learned_addr learned my addr 172.21.15.165:0/38546178 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59e0a00640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0568a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.198+0000 7f59dbe00640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc05c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59db400640 1 -- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 msgr2=0x560bc05c1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59db400640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc05c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59db400640 1 -- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 msgr2=0x560bc0568a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59db400640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0568a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59db400640 1 -- 172.21.15.165:0/38546178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560bc054bd90 con 0x560bc054a870 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59e0a00640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0568a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59db400640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc05770e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f59c402b030 tx=0x7f59c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9241561db62d3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59c8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:05:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59c8c00640 1 -- 172.21.15.165:0/38546178 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59c4048020 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59c8c00640 1 -- 172.21.15.165:0/38546178 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f59c40447c0 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.199+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59d405c5d0 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59c8c00640 1 -- 172.21.15.165:0/38546178 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59c4052050 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 10 client.5532 fetch_fsmap learned FSMap version 20 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x560bc01311b0 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59c8c00640 1 -- 172.21.15.165:0/38546178 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59d405c5d0 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59c8c00640 1 -- 172.21.15.165:0/38546178 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f59c4055020 con 0x560bc054a870 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 10 client.5532 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 10 client.5532 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 20 client.5532 populate_metadata read hostname 'smithi165' 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc0568fb0 0x560bc05407b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 4 client.5532 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x560bc0490db0 con 0x560bc0568fb0 2024-04-25T10:05:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.200+0000 7f59dbe00640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc0568fb0 0x560bc05407b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.201+0000 7f59c8c00640 10 client.5532 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.201+0000 7f59dbe00640 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc0568fb0 0x560bc05407b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83913c4e59d9ab0b server_cookie=2cb3ef79ff8f586b in_seq=0 out_seq=0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.201+0000 7f59c8c00640 1 -- 172.21.15.165:0/38546178 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17925 (crc 0 0 0) 0x560bc0490db0 con 0x560bc0568fb0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.201+0000 7f59c8c00640 10 client.5532 handle_command_reply: tid=0 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.201+0000 7f59e59c5740 1 client.5532 shutdown 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 msgr2=0x560bc05770e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc05770e0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f59c402b030 tx=0x7f59c4004ab0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 shutdown_connections 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560bc054b290 0x560bc0568a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x560bc0568fb0 0x560bc05407b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560bc054a870 0x560bc05770e0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 --2- 172.21.15.165:0/38546178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560bc0541850 0x560bc05c1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59c8c00640 0 client.5532 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 >> 172.21.15.165:0/38546178 conn(0x560bc051afb0 msgr2=0x560bc0548ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 shutdown_connections 2024-04-25T10:05:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 1 -- 172.21.15.165:0/38546178 wait complete. 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.202+0000 7f59e59c5740 20 client.5532 trim_cache size 0 max 16384 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1745, 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1739, 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 540259, 2024-04-25T10:05:31.210 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 17373457, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.389857919, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.195382585, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.084209891, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.084274023, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1738, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.388275354, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000223403 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-04-25T10:05:31.211 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000212, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000106 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 120576, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 26608 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1884, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1908, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 470206, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 24214092, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.427270927, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.231591282, 2024-04-25T10:05:31.212 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.074974837, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.097356932, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1907, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.439573981, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000230505 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 126912 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:05:31.213 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 30966, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 51811, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 10719157, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 77160386, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 8.590575745, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 6.183542479, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 2.464420376, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.179835277, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 51810, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.706883555, 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000071547 2024-04-25T10:05:31.214 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 45928, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.020676604, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000450 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:05:31.215 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5089, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 27.590005285, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005421498 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2105, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.969841202, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000935791 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 25965, 2024-04-25T10:05:31.216 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 25965, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 25965, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 500.736105471, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019285041 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 12, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 19872, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-25T10:05:31.217 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 19872, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 19872, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 162, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 162, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 48774, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 48214, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 171, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-25T10:05:31.218 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:05:31.219 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 871, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 4039, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 3911, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 59, 2024-04-25T10:05:31.220 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 14059 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 2000, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 4121, 2024-04-25T10:05:31.221 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 1840, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 2230, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-25T10:05:31.222 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 25232, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 25232, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 33, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 33, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 4, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-25T10:05:31.223 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 95413947, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4455, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 130.530528569, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.029299781 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 2014, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 2014, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 0, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 13, 2024-04-25T10:05:31.224 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 13, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 19872, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 19873, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 1, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 162, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 7876, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 7714, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 106864, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215396 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 29743, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-25T10:05:31.225 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 25965, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 7, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2000, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.668961226, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001334480 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 29, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006740537, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000232432 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.226 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6000, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 13.460080925, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002243346 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 7693, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 14.266013958, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001854414 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.227 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.228 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000065330, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000065330 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.229 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.005215566, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005215566 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.016444525, 2024-04-25T10:05:31.230 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.016444525 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.153999593, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019249949 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2230, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 439.303651437, 2024-04-25T10:05:31.231 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.196997153 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.232 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.021811205, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.021811205 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:05:31.233 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000310609, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000310609 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-25T10:05:31.234 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8000, 2024-04-25T10:05:31.235 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 30.832810560, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003854101 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-25T10:05:31.236 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 15017, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 15017, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 97, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:05:31.237 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:05:31.238 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:05:31.239 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 16782471, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 6219, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 440, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 5, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 5520, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 78, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-25T10:05:31.240 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 21031247, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 878379, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 6642, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 91614716, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 31, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 6642, 2024-04-25T10:05:31.241 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6642, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 48.335263318, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.007277215 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6642, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6797 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 6642, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 72, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 6570, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-25T10:05:31.242 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 11, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 72, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 4527, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 46, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 1840, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 33, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 22, 2024-04-25T10:05:31.243 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-25T10:05:31.244 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 41, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 3, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 8, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 8, 2024-04-25T10:05:31.245 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 196, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 50 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-25T10:05:31.246 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 32, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 1840, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 1840, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 34591, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 9056897, 2024-04-25T10:05:31.247 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 34591, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 34591, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 9056897, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:05:31.248 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 6642, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 91815276, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6642, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 91815276, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.249 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 6642, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 6642, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6642, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 6642, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.250 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1840, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 200560, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1840, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 72, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 200560, 2024-04-25T10:05:31.251 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55be7e3686e0": { 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 84, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 25232, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 91219643, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 25232, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:05:31.252 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4455, 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 91219559, 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.253 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59b806b4d0 msgr2=0x7f59b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59b806b4d0 0x7f59b806d990 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f59cc0017e0 tx=0x7f59cc009040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 msgr2=0x7f59dc1567c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 0x7f59dc1567c0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f59dc0b93c0 tx=0x7f59d400d890 comp rx=0 tx=0).stop 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 shutdown_connections 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59b806b4d0 0x7f59b806d990 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59dc15a570 0x7f59dc1567c0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59dc1f34c0 0x7f59dc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 --2- 172.21.15.165:0/1502244250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59bc004800 0x7f59dc157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 >> 172.21.15.165:0/1502244250 conn(0x7f59dc0d0610 msgr2=0x7f59dc15cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 shutdown_connections 2024-04-25T10:05:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.203+0000 7f59e1400640 1 -- 172.21.15.165:0/1502244250 wait complete. 2024-04-25T10:05:31.256 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T10:05:31.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.256 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub start / recursive,scrub_mdsdir 2024-04-25T10:05:31.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.462+0000 7f6cadc00640 1 Processor -- start 2024-04-25T10:05:31.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.462+0000 7f6cadc00640 1 -- start start 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6cadc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6ca815dd30 0x7f6ca80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6cadc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 0x7f6ca80d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6cadc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ca815e110 con 0x7f6ca806bff0 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6cadc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ca806d000 con 0x7f6ca815dd30 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6cadc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80b2fd0 con 0x7f6ca80b5b40 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6ca7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6ca806bff0 0x7f6ca80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47896/0 (socket says 172.21.15.165:47896) 2024-04-25T10:05:31.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6ca7400640 1 -- 172.21.15.165:0/1628858674 learned_addr learned my addr 172.21.15.165:0/1628858674 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:05:31.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/1628858674 >> v2:172.21.15.172:3300/0 conn(0x7f6ca815dd30 0x7f6ca80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6cad200640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 0x7f6ca80d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.464+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501508593 0 0) 0x7f6ca815e110 con 0x7f6ca806bff0 2024-04-25T10:05:31.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c8c003520 con 0x7f6ca806bff0 2024-04-25T10:05:31.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 140639425 0 0) 0x7f6c8c003520 con 0x7f6ca806bff0 2024-04-25T10:05:31.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 msgr2=0x7f6ca80d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 0x7f6ca80d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 >> v2:172.21.15.172:3300/0 conn(0x7f6ca815dd30 msgr2=0x7f6ca80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 --2- 172.21.15.165:0/1628858674 >> v2:172.21.15.172:3300/0 conn(0x7f6ca815dd30 0x7f6ca80cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca806c480 con 0x7f6ca806bff0 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6c90001670 con 0x7f6ca806bff0 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 >> v1:172.21.15.165:6789/0 conn(0x7f6ca806bff0 legacy=0x7f6ca80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 0x7f6c8c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c003f30 0x7f6c8c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c004d40 0x7f6c8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806d000 con 0x7f6c8c003f30 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80b2fd0 con 0x7f6c8c004d40 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c00b9c0 con 0x7f6ca80b5b40 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.465+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c003f30 0x7f6c8c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6cad200640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c004d40 0x7f6c8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 0x7f6c8c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6ca7400640 1 -- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 msgr2=0x7f6c8c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6ca80b5b40 0x7f6c8c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6ca7400640 1 -- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c004d40 msgr2=0x7f6c8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c8c004d40 0x7f6c8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.466+0000 7f6ca7400640 1 -- 172.21.15.165:0/1628858674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c003520 con 0x7f6c8c003f30 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.467+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c003f30 0x7f6c8c004800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c210 tx=0x7f6c8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bc00d23ac71118 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.467+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90007d50 con 0x7f6c8c003f30 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.467+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9002e070 con 0x7f6c8c003f30 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.467+0000 7f6ca6a00640 1 -- 172.21.15.165:0/1628858674 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90005d70 con 0x7f6c8c003f30 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cadc00640 1 -- 172.21.15.165:0/1628858674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c003f30 msgr2=0x7f6c8c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cadc00640 1 --2- 172.21.15.165:0/1628858674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c8c003f30 0x7f6c8c004800 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f6c9000c210 tx=0x7f6c8c005830 comp rx=0 tx=0).stop 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cad200640 1 -- 172.21.15.165:0/1628858674 reap_dead start 2024-04-25T10:05:31.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cadc00640 1 -- 172.21.15.165:0/1628858674 shutdown_connections 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cadc00640 1 -- 172.21.15.165:0/1628858674 >> 172.21.15.165:0/1628858674 conn(0x7f6ca80d0610 msgr2=0x7f6ca80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cadc00640 1 -- 172.21.15.165:0/1628858674 shutdown_connections 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.468+0000 7f6cadc00640 1 -- 172.21.15.165:0/1628858674 wait complete. 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.470+0000 7f6cadc00640 1 Processor -- start 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.470+0000 7f6cadc00640 1 -- start start 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6cadc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c8c003f30 0x7f6ca8156880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6cadc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 0x7f6ca80b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6cadc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca81590e0 0x7f6ca8159590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6cadc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806cd20 con 0x7f6ca8158ad0 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6cadc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca806ca40 con 0x7f6ca81590e0 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6cadc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80cf110 con 0x7f6c8c003f30 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 0x7f6ca80b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 0x7f6ca80b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42314/0 (socket says 172.21.15.165:42314) 2024-04-25T10:05:31.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6ca7400640 1 -- 172.21.15.165:0/3579283899 learned_addr learned my addr 172.21.15.165:0/3579283899 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.471+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c8c003f30 0x7f6ca8156880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6cad200640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca81590e0 0x7f6ca8159590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6ca7400640 1 -- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c8c003f30 msgr2=0x7f6ca8156880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6ca7400640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c8c003f30 0x7f6ca8156880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6ca7400640 1 -- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca81590e0 msgr2=0x7f6ca8159590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6ca7400640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca81590e0 0x7f6ca8159590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6ca7400640 1 -- 172.21.15.165:0/3579283899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca8156dc0 con 0x7f6ca8158ad0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6ca7400640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 0x7f6ca80b25e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6c90034040 tx=0x7f6c90001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.472+0000 7f6c9fe00640 1 -- 172.21.15.165:0/3579283899 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90042020 con 0x7f6ca8158ad0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.473+0000 7f6c9fe00640 1 -- 172.21.15.165:0/3579283899 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c90007270 con 0x7f6ca8158ad0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.473+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ca80b2b20 con 0x7f6ca8158ad0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.473+0000 7f6c9fe00640 1 -- 172.21.15.165:0/3579283899 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90001c90 con 0x7f6ca8158ad0 2024-04-25T10:05:31.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.473+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ca8157a30 con 0x7f6ca8158ad0 2024-04-25T10:05:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.475+0000 7f6c9fe00640 1 -- 172.21.15.165:0/3579283899 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c90045020 con 0x7f6ca8158ad0 2024-04-25T10:05:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.476+0000 7f6c9fe00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c8406b500 0x7f6c8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.476+0000 7f6c9fe00640 1 -- 172.21.15.165:0/3579283899 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6c900aa690 con 0x7f6ca8158ad0 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.476+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c8406b500 0x7f6c8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.477+0000 7f6cb207b740 1 Processor -- start 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.477+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c8406b500 0x7f6c8406d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c98007520 tx=0x7f6c9800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.477+0000 7f6cb207b740 1 -- start start 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd4a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd7a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7a820 0x55918cd4f630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6cb207b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918ccb39f0 con 0x55918cd7af60 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918cc5ceb0 con 0x55918cd97e40 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55918ccafa90 con 0x55918cd7a820 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd7a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd7a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42324/0 (socket says 172.21.15.165:42324) 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 -- 172.21.15.165:0/2193675577 learned_addr learned my addr 172.21.15.165:0/2193675577 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd4a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 -- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7a820 msgr2=0x55918cd4f630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7a820 0x55918cd4f630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 -- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 msgr2=0x55918cd4a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd4a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.478+0000 7f6ca7400640 1 -- 172.21.15.165:0/2193675577 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918ca94940 con 0x55918cd7af60 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.479+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd4a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.479+0000 7f6ca7400640 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd7a2e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6c900750b0 tx=0x7f6c900786e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bcc60dd1ea5b6b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.479+0000 7f6c9cc00640 1 -- 172.21.15.165:0/2193675577 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c90078c50 con 0x55918cd7af60 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.479+0000 7f6c9cc00640 1 -- 172.21.15.165:0/2193675577 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9002e040 con 0x55918cd7af60 2024-04-25T10:05:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.479+0000 7f6c9cc00640 1 -- 172.21.15.165:0/2193675577 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c9007cda0 con 0x55918cd7af60 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 -- 172.21.15.165:0/2193675577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 msgr2=0x55918cd7a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd7a2e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f6c900750b0 tx=0x7f6c900786e0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 -- 172.21.15.165:0/2193675577 shutdown_connections 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7a820 0x55918cd4f630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd7a2e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2193675577 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd4a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 -- 172.21.15.165:0/2193675577 >> 172.21.15.165:0/2193675577 conn(0x55918cd4ca40 msgr2=0x55918cd952e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 -- 172.21.15.165:0/2193675577 shutdown_connections 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.480+0000 7f6cb207b740 1 -- 172.21.15.165:0/2193675577 wait complete. 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.481+0000 7f6cb207b740 1 -- start start 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.481+0000 7f6cb207b740 1 Processor -- start 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd952e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd7bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd792f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918cc5ceb0 con 0x55918cd7af60 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918cc52370 con 0x55918cd97e40 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55918ccb39f0 con 0x55918cd6ae40 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6cb207b740 10 client.0 _ll_register_callbacks cb 0x55918c9c7d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6ca7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd952e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42330/0 (socket says 172.21.15.165:42330) 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.482+0000 7f6ca7e00640 1 -- 172.21.15.165:0/1324629094 learned_addr learned my addr 172.21.15.165:0/1324629094 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd7bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6cad200640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd792f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7e00640 1 -- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 msgr2=0x55918cd792f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd792f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7e00640 1 -- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 msgr2=0x55918cd7bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd7bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7e00640 1 -- 172.21.15.165:0/1324629094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55918caf50d0 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd952e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6c980045e0 tx=0x7f6c98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef8af1b157ec668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6c95600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.483+0000 7f6c95600640 1 -- 172.21.15.165:0/1324629094 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c98016460 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6c95600640 1 -- 172.21.15.165:0/1324629094 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6c98016600 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c9002e040 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6c95600640 1 -- 172.21.15.165:0/1324629094 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c98016800 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 10 client.5544 fetch_fsmap learned FSMap version 20 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55918c988230 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6c95600640 1 -- 172.21.15.165:0/1324629094 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c9002e040 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6c95600640 1 -- 172.21.15.165:0/1324629094 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6c98034080 con 0x55918cd7af60 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 10 client.5544 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 10 client.5544 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 20 client.5544 populate_metadata read hostname 'smithi165' 2024-04-25T10:05:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cd7c1f0 0x55918cd79830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 4 client.5544 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.484+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55918ccc0f30 con 0x55918cd7c1f0 2024-04-25T10:05:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.485+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cd7c1f0 0x55918cd79830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.485+0000 7f6ca7400640 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cd7c1f0 0x55918cd79830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7506708d9f76b1af server_cookie=3c20bc7ba1eab0d2 in_seq=0 out_seq=0 2024-04-25T10:05:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.485+0000 7f6c95600640 10 client.5544 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6c95600640 1 -- 172.21.15.165:0/1324629094 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55918ccc0f30 con 0x55918cd7c1f0 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6c95600640 10 client.5544 handle_command_reply: tid=0 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 client.5544 shutdown 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 msgr2=0x55918cd952e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd952e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f6c980045e0 tx=0x7f6c98004a40 comp rx=0 tx=0).stop 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 shutdown_connections 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd792f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd97e40 0x55918cd7bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cd7c1f0 0x55918cd79830 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 --2- 172.21.15.165:0/1324629094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd7af60 0x55918cd952e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 >> 172.21.15.165:0/1324629094 conn(0x55918cd7a820 msgr2=0x55918cd49e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6c95600640 0 client.5544 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 shutdown_connections 2024-04-25T10:05:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 1 -- 172.21.15.165:0/1324629094 wait complete. 2024-04-25T10:05:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.486+0000 7f6cb207b740 20 client.5544 trim_cache size 0 max 16384 2024-04-25T10:05:31.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.503+0000 7f6cb207b740 1 Processor -- start 2024-04-25T10:05:31.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.503+0000 7f6cb207b740 1 -- start start 2024-04-25T10:05:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd988f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd7af60 0x55918cda39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cda3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cb207b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca8200b70 con 0x55918cd97e40 2024-04-25T10:05:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca81ffcd0 con 0x55918cd7af60 2024-04-25T10:05:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca80cb7f0 con 0x55918cd6ae40 2024-04-25T10:05:31.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cda3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cda3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42336/0 (socket says 172.21.15.165:42336) 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 -- 172.21.15.165:0/201796876 learned_addr learned my addr 172.21.15.165:0/201796876 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6ca7400640 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd7af60 0x55918cda39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 -- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 msgr2=0x55918cd988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 -- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd7af60 msgr2=0x55918cda39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd7af60 0x55918cda39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6cad200640 1 -- 172.21.15.165:0/201796876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55918cd4db70 con 0x55918cd97e40 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.504+0000 7f6ca7400640 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd7af60 0x55918cda39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.505+0000 7f6cad200640 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cda3f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6ca808c1f0 tx=0x7f6ca0009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e252d3c5ce6c237 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.505+0000 7f6c97e00640 1 -- 172.21.15.165:0/201796876 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca001b070 con 0x55918cd97e40 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.505+0000 7f6c97e00640 1 -- 172.21.15.165:0/201796876 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0004250 con 0x55918cd97e40 2024-04-25T10:05:31.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.505+0000 7f6c97e00640 1 -- 172.21.15.165:0/201796876 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca00158c0 con 0x55918cd97e40 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 -- 172.21.15.165:0/201796876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 msgr2=0x55918cda3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cda3f20 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6ca808c1f0 tx=0x7f6ca0009ce0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 -- 172.21.15.165:0/201796876 shutdown_connections 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cda3f20 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd7af60 0x55918cda39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 --2- 172.21.15.165:0/201796876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd6ae40 0x55918cd988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 -- 172.21.15.165:0/201796876 >> 172.21.15.165:0/201796876 conn(0x55918cd7a820 msgr2=0x55918cd6a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 -- 172.21.15.165:0/201796876 shutdown_connections 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.506+0000 7f6cb207b740 1 -- 172.21.15.165:0/201796876 wait complete. 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.507+0000 7f6cb207b740 1 -- start start 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.507+0000 7f6cb207b740 1 Processor -- start 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd6ae40 0x55918cdffe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7af60 0x55918cdae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cdaed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918cd4d170 con 0x55918cd97e40 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55918cb55c10 con 0x55918cd6ae40 2024-04-25T10:05:31.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55918cd6ac50 con 0x55918cd7af60 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cb207b740 10 client.0 _ll_register_callbacks cb 0x7f6c90078a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cdaed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cdaed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42342/0 (socket says 172.21.15.165:42342) 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6cad200640 1 -- 172.21.15.165:0/2044142326 learned_addr learned my addr 172.21.15.165:0/2044142326 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.508+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd6ae40 0x55918cdffe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6ca7400640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7af60 0x55918cdae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6cad200640 1 -- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7af60 msgr2=0x55918cdae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6cad200640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7af60 0x55918cdae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6cad200640 1 -- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd6ae40 msgr2=0x55918cdffe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6cad200640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd6ae40 0x55918cdffe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6cad200640 1 -- 172.21.15.165:0/2044142326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55918cd969c0 con 0x55918cd97e40 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6cad200640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cdaed60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009d20 tx=0x7f6ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812b071b5b152ed7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6c95600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.509+0000 7f6c95600640 1 -- 172.21.15.165:0/2044142326 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ca002f060 con 0x55918cd97e40 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6c95600640 1 -- 172.21.15.165:0/2044142326 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7f6ca0015d50 con 0x55918cd97e40 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ca0004250 con 0x55918cd97e40 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6c95600640 1 -- 172.21.15.165:0/2044142326 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ca0005990 con 0x55918cd97e40 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 10 client.5550 fetch_fsmap learned FSMap version 20 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55918ce006d0 con 0x55918cd97e40 2024-04-25T10:05:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6c95600640 1 -- 172.21.15.165:0/2044142326 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ca0004250 con 0x55918cd97e40 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6c95600640 1 -- 172.21.15.165:0/2044142326 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6ca0005b30 con 0x55918cd97e40 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 10 client.5550 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 10 client.5550 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 20 client.5550 populate_metadata read hostname 'smithi165' 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cdb7850 0x55918cd4fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 4 client.5550 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}] 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.510+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "scrub_mdsdir"]}) v1 -- 0x55918ccbb280 con 0x55918cdb7850 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.511+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cdb7850 0x55918cd4fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.511+0000 7f6c95600640 10 client.5550 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.511+0000 7f6ca7e00640 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cdb7850 0x55918cd4fd00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cfdac113060bee30 server_cookie=c12d974c2b9cf69c in_seq=0 out_seq=0 2024-04-25T10:05:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6c95600640 1 -- 172.21.15.165:0/2044142326 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55918ccbb280 con 0x55918cdb7850 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6c95600640 10 client.5550 handle_command_reply: tid=0 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 client.5550 shutdown 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 msgr2=0x55918cdaed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cdaed60 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6ca0009d20 tx=0x7f6ca00046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 shutdown_connections 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55918cdb7850 0x55918cd4fd00 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55918cd97e40 0x55918cdaed60 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55918cd7af60 0x55918cdae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 --2- 172.21.15.165:0/2044142326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55918cd6ae40 0x55918cdffe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 >> 172.21.15.165:0/2044142326 conn(0x55918cd7a820 msgr2=0x55918cd7d6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.517+0000 7f6c95600640 0 client.5550 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.518+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 shutdown_connections 2024-04-25T10:05:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.518+0000 7f6cb207b740 1 -- 172.21.15.165:0/2044142326 wait complete. 2024-04-25T10:05:31.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.518+0000 7f6cb207b740 20 client.5550 trim_cache size 0 max 16384 2024-04-25T10:05:31.524 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:05:31.524 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:05:31.524 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "2f31443a-be00-4561-b5e4-67ff4ca89fb6", 2024-04-25T10:05:31.524 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:05:31.525 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:05:31.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c8406b500 msgr2=0x7f6c8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c8406b500 0x7f6c8406d9c0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6c98007520 tx=0x7f6c9800b040 comp rx=0 tx=0).stop 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 msgr2=0x7f6ca80b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 0x7f6ca80b25e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f6c90034040 tx=0x7f6c90001a40 comp rx=0 tx=0).stop 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 shutdown_connections 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c8406b500 0x7f6c8406d9c0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ca81590e0 0x7f6ca8159590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ca8158ad0 0x7f6ca80b25e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 --2- 172.21.15.165:0/3579283899 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c8c003f30 0x7f6ca8156880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 >> 172.21.15.165:0/3579283899 conn(0x7f6ca80d0610 msgr2=0x7f6ca80b4e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 shutdown_connections 2024-04-25T10:05:31.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.519+0000 7f6cadc00640 1 -- 172.21.15.165:0/3579283899 wait complete. 2024-04-25T10:05:31.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:05:31.542 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:05:31.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.754+0000 7ff016400640 1 Processor -- start 2024-04-25T10:05:31.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.754+0000 7ff016400640 1 -- start start 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff016400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff01015dd30 0x7ff0100cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff016400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff01006bff0 0x7ff0100b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff016400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0100b5970 0x7ff010161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff016400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01006d000 con 0x7ff01006bff0 2024-04-25T10:05:31.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff016400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff01015e110 con 0x7ff01015dd30 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff016400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100b2fd0 con 0x7ff0100b5970 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff00fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff01006bff0 0x7ff0100b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff00fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff01006bff0 0x7ff0100b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42344/0 (socket says 172.21.15.165:42344) 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.756+0000 7ff00fe00640 1 -- 172.21.15.165:0/1828753856 learned_addr learned my addr 172.21.15.165:0/1828753856 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff015000640 1 --2- 172.21.15.165:0/1828753856 >> v2:172.21.15.172:3300/0 conn(0x7ff01015dd30 0x7ff0100cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff015a00640 1 --2- 172.21.15.165:0/1828753856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0100b5970 0x7ff010161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff00fe00640 1 -- 172.21.15.165:0/1828753856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0100b5970 msgr2=0x7ff010161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff00fe00640 1 --2- 172.21.15.165:0/1828753856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0100b5970 0x7ff010161820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff00fe00640 1 -- 172.21.15.165:0/1828753856 >> v2:172.21.15.172:3300/0 conn(0x7ff01015dd30 msgr2=0x7ff0100cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff00fe00640 1 --2- 172.21.15.165:0/1828753856 >> v2:172.21.15.172:3300/0 conn(0x7ff01015dd30 0x7ff0100cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff00fe00640 1 -- 172.21.15.165:0/1828753856 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff01006c480 con 0x7ff01006bff0 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.757+0000 7ff015000640 1 --2- 172.21.15.165:0/1828753856 >> v2:172.21.15.172:3300/0 conn(0x7ff01015dd30 0x7ff0100cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.758+0000 7ff00fe00640 1 --2- 172.21.15.165:0/1828753856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff01006bff0 0x7ff0100b5430 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff01008bfe0 tx=0x7feffc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=334f7b958137f1a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.759+0000 7ff00f400640 1 -- 172.21.15.165:0/1828753856 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc037070 con 0x7ff01006bff0 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.759+0000 7ff00f400640 1 -- 172.21.15.165:0/1828753856 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feffc032440 con 0x7ff01006bff0 2024-04-25T10:05:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.759+0000 7ff00f400640 1 -- 172.21.15.165:0/1828753856 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc032810 con 0x7ff01006bff0 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.759+0000 7ff016400640 1 -- 172.21.15.165:0/1828753856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff01006bff0 msgr2=0x7ff0100b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.759+0000 7ff016400640 1 --2- 172.21.15.165:0/1828753856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff01006bff0 0x7ff0100b5430 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7ff01008bfe0 tx=0x7feffc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 -- 172.21.15.165:0/1828753856 shutdown_connections 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 --2- 172.21.15.165:0/1828753856 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0100b5970 0x7ff010161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 --2- 172.21.15.165:0/1828753856 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff01006bff0 0x7ff0100b5430 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 --2- 172.21.15.165:0/1828753856 >> v2:172.21.15.172:3300/0 conn(0x7ff01015dd30 0x7ff0100cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 -- 172.21.15.165:0/1828753856 >> 172.21.15.165:0/1828753856 conn(0x7ff0100d0610 msgr2=0x7ff0100d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 -- 172.21.15.165:0/1828753856 shutdown_connections 2024-04-25T10:05:31.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.760+0000 7ff016400640 1 -- 172.21.15.165:0/1828753856 wait complete. 2024-04-25T10:05:31.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.762+0000 7ff016400640 1 Processor -- start 2024-04-25T10:05:31.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.762+0000 7ff016400640 1 -- start start 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.763+0000 7ff016400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 0x7ff010156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.763+0000 7ff016400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01015dd30 0x7ff01015e320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.763+0000 7ff016400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff010156e50 0x7ff010160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.763+0000 7ff016400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01006cd20 con 0x7ff0100b5970 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.763+0000 7ff016400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff01006ca40 con 0x7ff01015dd30 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.763+0000 7ff016400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0100cf110 con 0x7ff010156e50 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.764+0000 7ff015000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 0x7ff010156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.764+0000 7ff015000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 0x7ff010156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42358/0 (socket says 172.21.15.165:42358) 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.764+0000 7ff015000640 1 -- 172.21.15.165:0/47751831 learned_addr learned my addr 172.21.15.165:0/47751831 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.764+0000 7ff00fe00640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01015dd30 0x7ff01015e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.764+0000 7ff015a00640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff010156e50 0x7ff010160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.765+0000 7ff015000640 1 -- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff010156e50 msgr2=0x7ff010160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.765+0000 7ff015000640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff010156e50 0x7ff010160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.765+0000 7ff015000640 1 -- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01015dd30 msgr2=0x7ff01015e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.765+0000 7ff015000640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01015dd30 0x7ff01015e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.765+0000 7ff015000640 1 -- 172.21.15.165:0/47751831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff010156b90 con 0x7ff0100b5970 2024-04-25T10:05:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.765+0000 7ff015000640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 0x7ff010156650 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c820 tx=0x7ff00400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.766+0000 7ff00cc00640 1 -- 172.21.15.165:0/47751831 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004015070 con 0x7ff0100b5970 2024-04-25T10:05:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.766+0000 7ff00cc00640 1 -- 172.21.15.165:0/47751831 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff004007a30 con 0x7ff0100b5970 2024-04-25T10:05:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.766+0000 7ff00cc00640 1 -- 172.21.15.165:0/47751831 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff004007de0 con 0x7ff0100b5970 2024-04-25T10:05:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.766+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff01015ec20 con 0x7ff0100b5970 2024-04-25T10:05:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.766+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff01015f1e0 con 0x7ff0100b5970 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.771+0000 7ff01a808740 1 Processor -- start 2024-04-25T10:05:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.771+0000 7ff01a808740 1 -- start start 2024-04-25T10:05:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.772+0000 7ff01a808740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 0x55b5ea5b9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.772+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5b8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.772+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5e2830 0x55b5ea5e2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.772+0000 7ff01a808740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea51bc70 con 0x55b5ea5b7d70 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.772+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea4c4cf0 con 0x55b5ea5f4440 2024-04-25T10:05:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.772+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea517d10 con 0x55b5ea5e2830 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.773+0000 7ff015a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 0x55b5ea5b9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.773+0000 7ff015a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 0x55b5ea5b9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42368/0 (socket says 172.21.15.165:42368) 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.773+0000 7ff015a00640 1 -- 172.21.15.165:0/2098437080 learned_addr learned my addr 172.21.15.165:0/2098437080 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.773+0000 7ff00fe00640 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5e2830 0x55b5ea5e2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.774+0000 7ff015a00640 1 -- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5e2830 msgr2=0x55b5ea5e2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.774+0000 7ff015a00640 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5e2830 0x55b5ea5e2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.774+0000 7ff015a00640 1 -- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 msgr2=0x55b5ea5b8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.774+0000 7ff015a00640 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5b8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.774+0000 7ff015a00640 1 -- 172.21.15.165:0/2098437080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ea2fa7d0 con 0x55b5ea5b7d70 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.775+0000 7ff015a00640 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 0x55b5ea5b9ed0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff008007fd0 tx=0x7ff00800e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6b42ad78207088 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.776+0000 7ff00cc00640 1 -- 172.21.15.165:0/47751831 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff004018070 con 0x7ff0100b5970 2024-04-25T10:05:31.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.777+0000 7ff00cc00640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feff006b500 0x7feff006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.777+0000 7ff00cc00640 1 -- 172.21.15.165:0/47751831 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6130+0+0 (secure 0 0 0) 0x7ff00408c440 con 0x7ff0100b5970 2024-04-25T10:05:31.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.777+0000 7ff015000640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feff006b500 0x7feff006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.778+0000 7ff015000640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feff006b500 0x7feff006d9c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff004062b50 tx=0x7ff0040046c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.781+0000 7ff001600640 1 -- 172.21.15.165:0/2098437080 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800f6c0 con 0x55b5ea5b7d70 2024-04-25T10:05:31.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.781+0000 7ff001600640 1 -- 172.21.15.165:0/2098437080 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00800f860 con 0x55b5ea5b7d70 2024-04-25T10:05:31.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.781+0000 7ff001600640 1 -- 172.21.15.165:0/2098437080 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800fc10 con 0x55b5ea5b7d70 2024-04-25T10:05:31.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.782+0000 7ff01a808740 1 -- 172.21.15.165:0/2098437080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 msgr2=0x55b5ea5b9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.782+0000 7ff01a808740 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 0x55b5ea5b9ed0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff008007fd0 tx=0x7ff00800e820 comp rx=0 tx=0).stop 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 -- 172.21.15.165:0/2098437080 shutdown_connections 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5e2830 0x55b5ea5e2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5b8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 --2- 172.21.15.165:0/2098437080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5b7d70 0x55b5ea5b9ed0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 -- 172.21.15.165:0/2098437080 >> 172.21.15.165:0/2098437080 conn(0x55b5ea57fa30 msgr2=0x55b5ea5fd830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 -- 172.21.15.165:0/2098437080 shutdown_connections 2024-04-25T10:05:31.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.783+0000 7ff01a808740 1 -- 172.21.15.165:0/2098437080 wait complete. 2024-04-25T10:05:31.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.784+0000 7ff01a808740 1 -- start start 2024-04-25T10:05:31.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.786+0000 7ff01a808740 1 Processor -- start 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.787+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5f8260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.787+0000 7ff01a808740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e5380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.787+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5d4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff01a808740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea4ba3a0 con 0x55b5ea5e2830 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea5e0380 con 0x55b5ea5f4440 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea60d8e0 con 0x55b5ea5b7d70 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff01a808740 10 client.0 _ll_register_callbacks cb 0x55b5ea4d7b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff015a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5d4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff015a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5d4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:41394/0 (socket says 172.21.15.165:41394) 2024-04-25T10:05:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff015a00640 1 -- 172.21.15.165:0/3757096969 learned_addr learned my addr 172.21.15.165:0/3757096969 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.788+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.789+0000 7ff00fe00640 1 -- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 msgr2=0x55b5ea5f8260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.789+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5f8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.789+0000 7ff00fe00640 1 -- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 msgr2=0x55b5ea5d4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.789+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5d4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.789+0000 7ff00fe00640 1 -- 172.21.15.165:0/3757096969 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5ea2303f0 con 0x55b5ea5e2830 2024-04-25T10:05:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.790+0000 7ff015a00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5d4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.791+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e5380 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7feffc031000 tx=0x7feffc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c777cccf7672bf93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.793+0000 7feff6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:05:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.794+0000 7feff6000640 1 -- 172.21.15.165:0/3757096969 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc037040 con 0x55b5ea5e2830 2024-04-25T10:05:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.794+0000 7feff6000640 1 -- 172.21.15.165:0/3757096969 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6186+0+0 (secure 0 0 0) 0x7feffc046410 con 0x55b5ea5e2830 2024-04-25T10:05:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.795+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff00800f860 con 0x55b5ea5e2830 2024-04-25T10:05:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.797+0000 7feff6000640 1 -- 172.21.15.165:0/3757096969 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc049940 con 0x55b5ea5e2830 2024-04-25T10:05:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.797+0000 7ff01a808740 10 client.5562 fetch_fsmap learned FSMap version 20 2024-04-25T10:05:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.797+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55b5ea1f00f0 con 0x55b5ea5e2830 2024-04-25T10:05:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.799+0000 7feff6000640 1 -- 172.21.15.165:0/3757096969 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff00800f860 con 0x55b5ea5e2830 2024-04-25T10:05:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.799+0000 7feff6000640 1 -- 172.21.15.165:0/3757096969 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feffc049ae0 con 0x55b5ea5e2830 2024-04-25T10:05:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.800+0000 7ff01a808740 10 client.5562 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:05:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.800+0000 7ff01a808740 10 client.5562 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.800+0000 7ff01a808740 20 client.5562 populate_metadata read hostname 'smithi165' 2024-04-25T10:05:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.800+0000 7ff01a808740 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5e58c0 0x55b5ea610e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.800+0000 7ff01a808740 4 client.5562 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:05:31.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.800+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5ea5291b0 con 0x55b5ea5e58c0 2024-04-25T10:05:31.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.801+0000 7ff015a00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5e58c0 0x55b5ea610e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.804+0000 7ff015a00640 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5e58c0 0x55b5ea610e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c189693ac294b16b server_cookie=7e2e152e0e2ab49e in_seq=0 out_seq=0 2024-04-25T10:05:31.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.804+0000 7feff6000640 10 client.5562 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.806+0000 7feff6000640 1 -- 172.21.15.165:0/3757096969 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55b5ea5291b0 con 0x55b5ea5e58c0 2024-04-25T10:05:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.807+0000 7feff6000640 10 client.5562 handle_command_reply: tid=0 2024-04-25T10:05:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.807+0000 7ff01a808740 1 client.5562 shutdown 2024-04-25T10:05:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.809+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 msgr2=0x55b5ea5e5380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.809+0000 7ff01a808740 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e5380 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7feffc031000 tx=0x7feffc004730 comp rx=0 tx=0).stop 2024-04-25T10:05:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.811+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 shutdown_connections 2024-04-25T10:05:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.812+0000 7ff01a808740 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5d4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.812+0000 7ff01a808740 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5e58c0 0x55b5ea610e90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.812+0000 7ff01a808740 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e5380 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.812+0000 7ff01a808740 1 --2- 172.21.15.165:0/3757096969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5f8260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.812+0000 7feff6000640 0 client.5562 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.812+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 >> 172.21.15.165:0/3757096969 conn(0x55b5ea57fa30 msgr2=0x55b5ea6008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.813+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 shutdown_connections 2024-04-25T10:05:31.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.813+0000 7ff01a808740 1 -- 172.21.15.165:0/3757096969 wait complete. 2024-04-25T10:05:31.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.813+0000 7ff01a808740 20 client.5562 trim_cache size 0 max 16384 2024-04-25T10:05:31.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.853+0000 7ff01a808740 1 Processor -- start 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.853+0000 7ff01a808740 1 -- start start 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff01a808740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5b2110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5f31c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff01a808740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010204970 con 0x55b5ea5e2830 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010203b30 con 0x55b5ea5f4440 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0101f8b60 con 0x55b5ea5b7d70 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.855+0000 7ff015a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.856+0000 7ff015a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:48950/0 (socket says 172.21.15.165:48950) 2024-04-25T10:05:31.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.856+0000 7ff015a00640 1 -- 172.21.15.165:0/1528930840 learned_addr learned my addr 172.21.15.165:0/1528930840 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.856+0000 7ff015a00640 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5f31c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.856+0000 7ff00fe00640 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.857+0000 7ff015a00640 1 -- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 msgr2=0x55b5ea5f31c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.857+0000 7ff015a00640 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5f31c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.857+0000 7ff015a00640 1 -- 172.21.15.165:0/1528930840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 msgr2=0x55b5ea5b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.857+0000 7ff015a00640 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.857+0000 7ff015a00640 1 -- 172.21.15.165:0/1528930840 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5ea5d46d0 con 0x55b5ea5b7d70 2024-04-25T10:05:31.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.858+0000 7ff00fe00640 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:05:31.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.858+0000 7ff015a00640 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5b1bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff008029530 tx=0x7ff008011070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6107b9267ffbe467 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.859+0000 7ff000c00640 1 -- 172.21.15.165:0/1528930840 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00801d040 con 0x55b5ea5b7d70 2024-04-25T10:05:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.859+0000 7ff000c00640 1 -- 172.21.15.165:0/1528930840 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008029850 con 0x55b5ea5b7d70 2024-04-25T10:05:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.859+0000 7ff000c00640 1 -- 172.21.15.165:0/1528930840 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00800f7a0 con 0x55b5ea5b7d70 2024-04-25T10:05:31.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.860+0000 7ff01a808740 1 -- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 msgr2=0x55b5ea5b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.860+0000 7ff01a808740 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5b1bd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff008029530 tx=0x7ff008011070 comp rx=0 tx=0).stop 2024-04-25T10:05:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.861+0000 7ff01a808740 1 -- 172.21.15.165:0/1528930840 shutdown_connections 2024-04-25T10:05:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.861+0000 7ff01a808740 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5f31c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.861+0000 7ff01a808740 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.861+0000 7ff01a808740 1 --2- 172.21.15.165:0/1528930840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5b1bd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.861+0000 7ff01a808740 1 -- 172.21.15.165:0/1528930840 >> 172.21.15.165:0/1528930840 conn(0x55b5ea57fa30 msgr2=0x55b5ea5e3f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.862+0000 7ff01a808740 1 -- 172.21.15.165:0/1528930840 shutdown_connections 2024-04-25T10:05:31.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.862+0000 7ff01a808740 1 -- 172.21.15.165:0/1528930840 wait complete. 2024-04-25T10:05:31.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.863+0000 7ff01a808740 1 -- start start 2024-04-25T10:05:31.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.863+0000 7ff01a808740 1 Processor -- start 2024-04-25T10:05:31.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5e48a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5b6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea611270 con 0x55b5ea5e2830 2024-04-25T10:05:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff010203560 con 0x55b5ea5f4440 2024-04-25T10:05:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5ea4d8fb0 con 0x55b5ea5b7d70 2024-04-25T10:05:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.865+0000 7ff01a808740 10 client.0 _ll_register_callbacks cb 0x7feffc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:05:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.866+0000 7ff00fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.866+0000 7ff00fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42390/0 (socket says 172.21.15.165:42390) 2024-04-25T10:05:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.866+0000 7ff00fe00640 1 -- 172.21.15.165:0/3565819834 learned_addr learned my addr 172.21.15.165:0/3565819834 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:05:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.867+0000 7ff015a00640 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5e48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.867+0000 7ff00fe00640 1 -- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 msgr2=0x55b5ea5e48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.867+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5e48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.867+0000 7ff00fe00640 1 -- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 msgr2=0x55b5ea5b6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:05:31.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.867+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5b6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.867+0000 7ff00fe00640 1 -- 172.21.15.165:0/3565819834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5ea5b8f00 con 0x55b5ea5e2830 2024-04-25T10:05:31.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.868+0000 7ff00fe00640 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e4de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7feffc0090b0 tx=0x7feffc009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6401dcf55c9ceca5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:05:31.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.868+0000 7feff6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:05:31.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.869+0000 7feff6000640 1 -- 172.21.15.165:0/3565819834 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feffc056060 con 0x55b5ea5e2830 2024-04-25T10:05:31.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.869+0000 7feff6000640 1 -- 172.21.15.165:0/3565819834 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6186+0+0 (secure 0 0 0) 0x7feffc0491b0 con 0x55b5ea5e2830 2024-04-25T10:05:31.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.869+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff008029850 con 0x55b5ea5e2830 2024-04-25T10:05:31.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.869+0000 7feff6000640 1 -- 172.21.15.165:0/3565819834 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7feffc037070 con 0x55b5ea5e2830 2024-04-25T10:05:31.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.871+0000 7ff01a808740 10 client.5568 fetch_fsmap learned FSMap version 20 2024-04-25T10:05:31.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.871+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55b5ea5f3930 con 0x55b5ea5e2830 2024-04-25T10:05:31.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.872+0000 7feff6000640 1 -- 172.21.15.165:0/3565819834 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff008029850 con 0x55b5ea5e2830 2024-04-25T10:05:31.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7feff6000640 1 -- 172.21.15.165:0/3565819834 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7feffc05f080 con 0x55b5ea5e2830 2024-04-25T10:05:31.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7ff01a808740 10 client.5568 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:05:31.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7ff01a808740 10 client.5568 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:05:31.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7ff01a808740 20 client.5568 populate_metadata read hostname 'smithi165' 2024-04-25T10:05:31.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7ff01a808740 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5b7770 0x55b5ea5b6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:05:31.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7ff01a808740 4 client.5568 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:05:31.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.874+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5ea523500 con 0x55b5ea5b7770 2024-04-25T10:05:31.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.875+0000 7ff015a00640 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5b7770 0x55b5ea5b6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:05:31.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.879+0000 7feff6000640 10 client.5568 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.879+0000 7ff015a00640 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5b7770 0x55b5ea5b6a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d32039c4886889 server_cookie=50ab82b2157636cc in_seq=0 out_seq=0 2024-04-25T10:05:31.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.880+0000 7feff6000640 1 -- 172.21.15.165:0/3565819834 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+274 (crc 0 0 0) 0x55b5ea523500 con 0x55b5ea5b7770 2024-04-25T10:05:31.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.880+0000 7feff6000640 10 client.5568 handle_command_reply: tid=0 2024-04-25T10:05:31.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.880+0000 7ff01a808740 1 client.5568 shutdown 2024-04-25T10:05:31.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.880+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 msgr2=0x55b5ea5e4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e4de0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7feffc0090b0 tx=0x7feffc009240 comp rx=0 tx=0).stop 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 shutdown_connections 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55b5ea5f4440 0x55b5ea5b6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55b5ea5b7770 0x55b5ea5b6a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7feff6000640 0 client.5568 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55b5ea5e2830 0x55b5ea5e4de0 secure :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7feffc0090b0 tx=0x7feffc009240 comp rx=0 tx=0).stop 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 --2- 172.21.15.165:0/3565819834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55b5ea5b7d70 0x55b5ea5e48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.881+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 >> 172.21.15.165:0/3565819834 conn(0x55b5ea57fa30 msgr2=0x55b5ea5f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.882+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 shutdown_connections 2024-04-25T10:05:31.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.882+0000 7ff01a808740 1 -- 172.21.15.165:0/3565819834 wait complete. 2024-04-25T10:05:31.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.883+0000 7ff01a808740 20 client.5568 trim_cache size 0 max 16384 2024-04-25T10:05:31.890 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:05:31.890 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "scrub active (649 inodes in the stack)", 2024-04-25T10:05:31.890 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout: "2f31443a-be00-4561-b5e4-67ff4ca89fb6": { 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/", 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "2f31443a-be00-4561-b5e4-67ff4ca89fb6", 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive,scrub_mdsdir" 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:05:31.891 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.884+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feff006b500 msgr2=0x7feff006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.884+0000 7ff016400640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feff006b500 0x7feff006d9c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7ff004062b50 tx=0x7ff0040046c0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.884+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 msgr2=0x7ff010156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.884+0000 7ff016400640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 0x7ff010156650 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c820 tx=0x7ff00400cce0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 shutdown_connections 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feff006b500 0x7feff006d9c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff010156e50 0x7ff010160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff01015dd30 0x7ff01015e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 --2- 172.21.15.165:0/47751831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0100b5970 0x7ff010156650 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:05:31.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 >> 172.21.15.165:0/47751831 conn(0x7ff0100d0610 msgr2=0x7ff0100b6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:05:31.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.885+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 shutdown_connections 2024-04-25T10:05:31.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:05:31.886+0000 7ff016400640 1 -- 172.21.15.165:0/47751831 wait complete. 2024-04-25T10:05:31.922 INFO:tasks.cephfs.filesystem:scrub status for tag:2f31443a-be00-4561-b5e4-67ff4ca89fb6 - {'path': '/', 'tag': '2f31443a-be00-4561-b5e4-67ff4ca89fb6', 'options': 'recursive,scrub_mdsdir'} 2024-04-25T10:05:57.340 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:05:57.350 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:01.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:01.923 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.2:0 scrub status 2024-04-25T10:06:02.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.130+0000 7fdf21800640 1 Processor -- start 2024-04-25T10:06:02.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.130+0000 7fdf21800640 1 -- start start 2024-04-25T10:06:02.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.131+0000 7fdf21800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0ce230 0x7fdf1c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.131+0000 7fdf21800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdf1c15b270 0x7fdf1c15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.131+0000 7fdf21800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf1c072280 con 0x7fdf1c0cc4f0 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf21800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c071270 con 0x7fdf1c15b270 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf21800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf1c0714c0 con 0x7fdf1c0ce230 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdf1c15b270 0x7fdf1c15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdf1c15b270 0x7fdf1c15b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49210/0 (socket says 172.21.15.165:49210) 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 -- 172.21.15.165:0/1251520564 learned_addr learned my addr 172.21.15.165:0/1251520564 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1aa00640 1 --1- 172.21.15.165:0/1251520564 >> v1:172.21.15.165:6789/0 conn(0x7fdf1c0cc4f0 0x7fdf1c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42638/0 (socket says 172.21.15.165:42638) 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 -- 172.21.15.165:0/1251520564 >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0ce230 msgr2=0x7fdf1c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1a000640 1 --2- 172.21.15.165:0/1251520564 >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0ce230 0x7fdf1c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf19600640 1 -- 172.21.15.165:0/1251520564 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2302541015 0 0) 0x7fdf1c072280 con 0x7fdf1c0cc4f0 2024-04-25T10:06:02.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 --2- 172.21.15.165:0/1251520564 >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0ce230 0x7fdf1c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 -- 172.21.15.165:0/1251520564 >> v1:172.21.15.165:6789/0 conn(0x7fdf1c0cc4f0 legacy=0x7fdf1c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1b400640 1 -- 172.21.15.165:0/1251520564 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c0717a0 con 0x7fdf1c15b270 2024-04-25T10:06:02.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.132+0000 7fdf1a000640 1 --2- 172.21.15.165:0/1251520564 >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0ce230 0x7fdf1c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.133+0000 7fdf1b400640 1 --2- 172.21.15.165:0/1251520564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c15b6e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0b4c00 tx=0x7fdf1402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee430b252e42baf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf19600640 1 -- 172.21.15.165:0/1251520564 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14037070 con 0x7fdf1c15b270 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf19600640 1 -- 172.21.15.165:0/1251520564 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf14032440 con 0x7fdf1c15b270 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf19600640 1 -- 172.21.15.165:0/1251520564 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14032810 con 0x7fdf1c15b270 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf21800640 1 -- 172.21.15.165:0/1251520564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 msgr2=0x7fdf1c15b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf21800640 1 --2- 172.21.15.165:0/1251520564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c15b6e0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c0b4c00 tx=0x7fdf1402a270 comp rx=0 tx=0).stop 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf21800640 1 -- 172.21.15.165:0/1251520564 shutdown_connections 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf21800640 1 --2- 172.21.15.165:0/1251520564 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c15b6e0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf21800640 1 --2- 172.21.15.165:0/1251520564 >> v2:172.21.15.172:3300/0 conn(0x7fdf1c0ce230 0x7fdf1c0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.134+0000 7fdf21800640 1 -- 172.21.15.165:0/1251520564 >> 172.21.15.165:0/1251520564 conn(0x7fdf1c06abe0 msgr2=0x7fdf1c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.135+0000 7fdf21800640 1 -- 172.21.15.165:0/1251520564 shutdown_connections 2024-04-25T10:06:02.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.135+0000 7fdf21800640 1 -- 172.21.15.165:0/1251520564 wait complete. 2024-04-25T10:06:02.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.136+0000 7fdf21800640 1 Processor -- start 2024-04-25T10:06:02.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.136+0000 7fdf21800640 1 -- start start 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.137+0000 7fdf21800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c0cc4f0 0x7fdf1c1568b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.137+0000 7fdf21800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0ce230 0x7fdf1c14e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.137+0000 7fdf21800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.137+0000 7fdf21800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c15bc20 con 0x7fdf1c15b270 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.137+0000 7fdf21800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c071d60 con 0x7fdf1c0ce230 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.137+0000 7fdf21800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c15bda0 con 0x7fdf1c0cc4f0 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c14ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49212/0 (socket says 172.21.15.165:49212) 2024-04-25T10:06:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 -- 172.21.15.165:0/3041814368 learned_addr learned my addr 172.21.15.165:0/3041814368 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c0cc4f0 0x7fdf1c1568b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 -- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c0cc4f0 msgr2=0x7fdf1c1568b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c0cc4f0 0x7fdf1c1568b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1a000640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0ce230 0x7fdf1c14e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 -- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0ce230 msgr2=0x7fdf1c14e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0ce230 0x7fdf1c14e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 -- 172.21.15.165:0/3041814368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c154bd0 con 0x7fdf1c15b270 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.138+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c14ee60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdf1402a2b0 tx=0x7fdf140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.139+0000 7fdf12a00640 1 -- 172.21.15.165:0/3041814368 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14037040 con 0x7fdf1c15b270 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.139+0000 7fdf12a00640 1 -- 172.21.15.165:0/3041814368 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf14042070 con 0x7fdf1c15b270 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.139+0000 7fdf12a00640 1 -- 172.21.15.165:0/3041814368 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14030c90 con 0x7fdf1c15b270 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.139+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c154eb0 con 0x7fdf1c15b270 2024-04-25T10:06:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.139+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c155470 con 0x7fdf1c15b270 2024-04-25T10:06:02.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.141+0000 7fdf12a00640 1 -- 172.21.15.165:0/3041814368 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf14047460 con 0x7fdf1c15b270 2024-04-25T10:06:02.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.143+0000 7fdf12a00640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdef806b4d0 0x7fdef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.143+0000 7fdf12a00640 1 -- 172.21.15.165:0/3041814368 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdf140ab140 con 0x7fdf1c15b270 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.143+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdef806b4d0 0x7fdef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.143+0000 7fdf245f7740 1 Processor -- start 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 -- start start 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdef806b4d0 0x7fdef806d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0012e0 tx=0x7fdf0c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e27d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e62280 0x55eb43e25f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 0x55eb43e50af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb43d89c70 con 0x55eb43e50670 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb43d32cf0 con 0x55eb43e25bb0 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.144+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb43d85d10 con 0x55eb43e62280 2024-04-25T10:06:02.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 0x55eb43e50af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 0x55eb43e50af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49226/0 (socket says 172.21.15.165:49226) 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 -- 172.21.15.165:0/4276269366 learned_addr learned my addr 172.21.15.165:0/4276269366 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e27d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 -- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e62280 msgr2=0x55eb43e25f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e62280 0x55eb43e25f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 -- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 msgr2=0x55eb43e27d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e27d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 -- 172.21.15.165:0/4276269366 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb43b687d0 con 0x55eb43e50670 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.145+0000 7fdf1b400640 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 0x55eb43e50af0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdf14081840 tx=0x7fdf14079070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adb357c17bc221cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.146+0000 7fdf0b400640 1 -- 172.21.15.165:0/4276269366 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14079710 con 0x55eb43e50670 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.146+0000 7fdf0b400640 1 -- 172.21.15.165:0/4276269366 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1407d7b0 con 0x55eb43e50670 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.146+0000 7fdf0b400640 1 -- 172.21.15.165:0/4276269366 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14081d70 con 0x55eb43e50670 2024-04-25T10:06:02.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.146+0000 7fdf245f7740 1 -- 172.21.15.165:0/4276269366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 msgr2=0x55eb43e50af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.146+0000 7fdf245f7740 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 0x55eb43e50af0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fdf14081840 tx=0x7fdf14079070 comp rx=0 tx=0).stop 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.146+0000 7fdf245f7740 1 -- 172.21.15.165:0/4276269366 shutdown_connections 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e50670 0x55eb43e50af0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e62280 0x55eb43e25f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 --2- 172.21.15.165:0/4276269366 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e27d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 -- 172.21.15.165:0/4276269366 >> 172.21.15.165:0/4276269366 conn(0x55eb43deda30 msgr2=0x55eb43e6b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 -- 172.21.15.165:0/4276269366 shutdown_connections 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 -- 172.21.15.165:0/4276269366 wait complete. 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 -- start start 2024-04-25T10:06:02.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.147+0000 7fdf245f7740 1 Processor -- start 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.148+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.148+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e3dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.148+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e63fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.148+0000 7fdf245f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb43d89c70 con 0x55eb43e62280 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.148+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb43d283a0 con 0x55eb43e25bb0 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.148+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb43e3ccf0 con 0x55eb43e50670 2024-04-25T10:06:02.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf245f7740 10 client.0 _ll_register_callbacks cb 0x55eb43d45b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e63fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e63fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49238/0 (socket says 172.21.15.165:49238) 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 -- 172.21.15.165:0/3769049913 learned_addr learned my addr 172.21.15.165:0/3769049913 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 -- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 msgr2=0x55eb43e3dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1a000640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e3dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e3dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 -- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 msgr2=0x55eb43e3b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3b760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.149+0000 7fdf1b400640 1 -- 172.21.15.165:0/3769049913 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb43a9e3f0 con 0x55eb43e62280 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e63fa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdf14079410 tx=0x7fdf14009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f89dd6d122911e37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdeffe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdeffe00640 1 -- 172.21.15.165:0/3769049913 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140a9120 con 0x55eb43e62280 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdeffe00640 1 -- 172.21.15.165:0/3769049913 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdf14079970 con 0x55eb43e62280 2024-04-25T10:06:02.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdf1407d7b0 con 0x55eb43e62280 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdeffe00640 1 -- 172.21.15.165:0/3769049913 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf14079710 con 0x55eb43e62280 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 10 client.5583 fetch_fsmap learned FSMap version 20 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55eb43a5e0f0 con 0x55eb43e62280 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdeffe00640 1 -- 172.21.15.165:0/3769049913 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1407d7b0 con 0x55eb43e62280 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdeffe00640 1 -- 172.21.15.165:0/3769049913 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdf1403a030 con 0x55eb43e62280 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 10 client.5583 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:06:02.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 10 client.5583 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:06:02.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 20 client.5583 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:02.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e3e4e0 0x55eb43e644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 4 client.5583 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:02.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.150+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb43d971b0 con 0x55eb43e3e4e0 2024-04-25T10:06:02.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.151+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e3e4e0 0x55eb43e644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.151+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e3e4e0 0x55eb43e644e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e45b04fd15e0a481 server_cookie=88d31ebb15c2eaf2 in_seq=0 out_seq=0 2024-04-25T10:06:02.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.151+0000 7fdeffe00640 10 client.5583 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdeffe00640 1 -- 172.21.15.165:0/3769049913 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55eb43d971b0 con 0x55eb43e3e4e0 2024-04-25T10:06:02.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdeffe00640 10 client.5583 handle_command_reply: tid=0 2024-04-25T10:06:02.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 client.5583 shutdown 2024-04-25T10:06:02.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 msgr2=0x55eb43e63fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e63fa0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fdf14079410 tx=0x7fdf14009240 comp rx=0 tx=0).stop 2024-04-25T10:06:02.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 shutdown_connections 2024-04-25T10:06:02.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e3e4e0 0x55eb43e644e0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e63fa0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e3dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3769049913 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdeffe00640 0 client.5583 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.152+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 >> 172.21.15.165:0/3769049913 conn(0x55eb43deda30 msgr2=0x55eb43e4d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.153+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 shutdown_connections 2024-04-25T10:06:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.153+0000 7fdf245f7740 1 -- 172.21.15.165:0/3769049913 wait complete. 2024-04-25T10:06:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.153+0000 7fdf245f7740 20 client.5583 trim_cache size 0 max 16384 2024-04-25T10:06:02.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.169+0000 7fdf245f7740 1 Processor -- start 2024-04-25T10:06:02.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.169+0000 7fdf245f7740 1 -- start start 2024-04-25T10:06:02.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e6de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e6e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf245f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c200360 con 0x55eb43e62280 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1ff520 con 0x55eb43e25bb0 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1598d0 con 0x55eb43e50670 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e6e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e6e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49240/0 (socket says 172.21.15.165:49240) 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 -- 172.21.15.165:0/3467565447 learned_addr learned my addr 172.21.15.165:0/3467565447 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 -- 172.21.15.165:0/3467565447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 msgr2=0x55eb43e6de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e6de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 -- 172.21.15.165:0/3467565447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 msgr2=0x55eb43e3c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.170+0000 7fdf1b400640 1 -- 172.21.15.165:0/3467565447 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb43e3bd00 con 0x55eb43e62280 2024-04-25T10:06:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf1b400640 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e6e3a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdf14075b70 tx=0x7fdf1403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9f24af8bc69e2e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf0aa00640 1 -- 172.21.15.165:0/3467565447 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf140759b0 con 0x55eb43e62280 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf0aa00640 1 -- 172.21.15.165:0/3467565447 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf140a9120 con 0x55eb43e62280 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf0aa00640 1 -- 172.21.15.165:0/3467565447 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf14098240 con 0x55eb43e62280 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 -- 172.21.15.165:0/3467565447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 msgr2=0x55eb43e6e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e6e3a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fdf14075b70 tx=0x7fdf1403e000 comp rx=0 tx=0).stop 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 -- 172.21.15.165:0/3467565447 shutdown_connections 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e62280 0x55eb43e6e3a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e6de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 --2- 172.21.15.165:0/3467565447 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e25bb0 0x55eb43e3c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.171+0000 7fdf245f7740 1 -- 172.21.15.165:0/3467565447 >> 172.21.15.165:0/3467565447 conn(0x55eb43deda30 msgr2=0x55eb43e26a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.172+0000 7fdf245f7740 1 -- 172.21.15.165:0/3467565447 shutdown_connections 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.172+0000 7fdf245f7740 1 -- 172.21.15.165:0/3467565447 wait complete. 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.172+0000 7fdf245f7740 1 -- start start 2024-04-25T10:06:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.172+0000 7fdf245f7740 1 Processor -- start 2024-04-25T10:06:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 0x55eb43e61000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e61540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e62280 0x55eb43e24bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1fef20 con 0x55eb43e25bb0 2024-04-25T10:06:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb43e3bd00 con 0x55eb43e62280 2024-04-25T10:06:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55eb43e64a70 con 0x55eb43e50670 2024-04-25T10:06:02.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf245f7740 10 client.0 _ll_register_callbacks cb 0x7fdf14090070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.173+0000 7fdf1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 0x55eb43e61000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 0x55eb43e61000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49250/0 (socket says 172.21.15.165:49250) 2024-04-25T10:06:02.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 -- 172.21.15.165:0/1124341175 learned_addr learned my addr 172.21.15.165:0/1124341175 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 -- 172.21.15.165:0/1124341175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 msgr2=0x55eb43e61540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e61540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 -- 172.21.15.165:0/1124341175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e62280 msgr2=0x55eb43e24bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e62280 0x55eb43e24bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 -- 172.21.15.165:0/1124341175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb43e3d910 con 0x55eb43e25bb0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdf1aa00640 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 0x55eb43e61000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005e80 tx=0x7fdf0c001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf6f2e9b3affdd01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdeffe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.174+0000 7fdeffe00640 1 -- 172.21.15.165:0/1124341175 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c0226e0 con 0x55eb43e25bb0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdeffe00640 1 -- 172.21.15.165:0/1124341175 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fdf0c022880 con 0x55eb43e25bb0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb43d91500 con 0x55eb43e25bb0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdeffe00640 1 -- 172.21.15.165:0/1124341175 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf0c026020 con 0x55eb43e25bb0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 10 client.5589 fetch_fsmap learned FSMap version 20 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55eb43e23b30 con 0x55eb43e25bb0 2024-04-25T10:06:02.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdeffe00640 1 -- 172.21.15.165:0/1124341175 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55eb43d91500 con 0x55eb43e25bb0 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdeffe00640 1 -- 172.21.15.165:0/1124341175 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fdf0c007520 con 0x55eb43e25bb0 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 10 client.5589 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 10 client.5589 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 20 client.5589 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e25180 0x55eb43ed46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 4 client.5589 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55eb43d0f550 con 0x55eb43e25180 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.175+0000 7fdf1a000640 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e25180 0x55eb43ed46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdf1a000640 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e25180 0x55eb43ed46e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d835c3c202cb373 server_cookie=652d3b20e45668eb in_seq=0 out_seq=0 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdeffe00640 10 client.5589 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdeffe00640 1 -- 172.21.15.165:0/1124341175 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55eb43d0f550 con 0x55eb43e25180 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdeffe00640 10 client.5589 handle_command_reply: tid=0 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdf245f7740 1 client.5589 shutdown 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 msgr2=0x55eb43e61000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdf245f7740 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 0x55eb43e61000 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c005e80 tx=0x7fdf0c001250 comp rx=0 tx=0).stop 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.176+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 shutdown_connections 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55eb43e62280 0x55eb43e24bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55eb43e50670 0x55eb43e61540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x55eb43e25180 0x55eb43ed46e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 --2- 172.21.15.165:0/1124341175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55eb43e25bb0 0x55eb43e61000 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 >> 172.21.15.165:0/1124341175 conn(0x55eb43deda30 msgr2=0x55eb43e6de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdeffe00640 0 client.5589 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 shutdown_connections 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 1 -- 172.21.15.165:0/1124341175 wait complete. 2024-04-25T10:06:02.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf245f7740 20 client.5589 trim_cache size 0 max 16384 2024-04-25T10:06:02.189 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:02.189 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:06:02.189 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:06:02.189 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdef806b4d0 msgr2=0x7fdef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf21800640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdef806b4d0 0x7fdef806d990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c0012e0 tx=0x7fdf0c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 msgr2=0x7fdf1c14ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.177+0000 7fdf21800640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c14ee60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fdf1402a2b0 tx=0x7fdf140046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 shutdown_connections 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdef806b4d0 0x7fdef806d990 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf1c15b270 0x7fdf1c14ee60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf1c0ce230 0x7fdf1c14e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 --2- 172.21.15.165:0/3041814368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf1c0cc4f0 0x7fdf1c1568b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 >> 172.21.15.165:0/3041814368 conn(0x7fdf1c06abe0 msgr2=0x7fdf1c15c280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 shutdown_connections 2024-04-25T10:06:02.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.178+0000 7fdf21800640 1 -- 172.21.15.165:0/3041814368 wait complete. 2024-04-25T10:06:02.200 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:02.200 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:02.201 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.2:0 perf dump 2024-04-25T10:06:02.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.385+0000 7fa7e5800640 1 Processor -- start 2024-04-25T10:06:02.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.386+0000 7fa7e5800640 1 -- start start 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e5800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa7e0145140 0x7fa7e00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e00ce950 0x7fa7e00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e5800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa7e00c5030 0x7fa7e006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e5800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e00ced90 con 0x7fa7e00c5030 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e5800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7e0145520 con 0x7fa7e0145140 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0071270 con 0x7fa7e00ce950 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.387+0000 7fa7e4e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa7e00c5030 0x7fa7e006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa7e00c5030 0x7fa7e006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49256/0 (socket says 172.21.15.165:49256) 2024-04-25T10:06:02.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 -- 172.21.15.165:0/2470982635 learned_addr learned my addr 172.21.15.165:0/2470982635 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/2470982635 >> v2:172.21.15.172:3300/0 conn(0x7fa7e0145140 0x7fa7e00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7d7400640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e00ce950 0x7fa7e00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 -- 172.21.15.165:0/2470982635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e00ce950 msgr2=0x7fa7e00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e00ce950 0x7fa7e00c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 -- 172.21.15.165:0/2470982635 >> v2:172.21.15.172:3300/0 conn(0x7fa7e0145140 msgr2=0x7fa7e00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/2470982635 >> v2:172.21.15.172:3300/0 conn(0x7fa7e0145140 0x7fa7e00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7e4e00640 1 -- 172.21.15.165:0/2470982635 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0071550 con 0x7fa7e00c5030 2024-04-25T10:06:02.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/2470982635 >> v2:172.21.15.172:3300/0 conn(0x7fa7e0145140 0x7fa7e00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.388+0000 7fa7d7400640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e00ce950 0x7fa7e00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.389+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00c5030 0x7fa7e006d1f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa7e006f690 tx=0x7fa7d802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e1a4c14488a349c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.390+0000 7fa7df400640 1 -- 172.21.15.165:0/2470982635 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8036070 con 0x7fa7e00c5030 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.390+0000 7fa7df400640 1 -- 172.21.15.165:0/2470982635 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8031600 con 0x7fa7e00c5030 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.390+0000 7fa7df400640 1 -- 172.21.15.165:0/2470982635 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d80319d0 con 0x7fa7e00c5030 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.390+0000 7fa7e5800640 1 -- 172.21.15.165:0/2470982635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00c5030 msgr2=0x7fa7e006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.390+0000 7fa7e5800640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00c5030 0x7fa7e006d1f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fa7e006f690 tx=0x7fa7d802a610 comp rx=0 tx=0).stop 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 -- 172.21.15.165:0/2470982635 shutdown_connections 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00c5030 0x7fa7e006d1f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 --2- 172.21.15.165:0/2470982635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e00ce950 0x7fa7e00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 --2- 172.21.15.165:0/2470982635 >> v2:172.21.15.172:3300/0 conn(0x7fa7e0145140 0x7fa7e00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 -- 172.21.15.165:0/2470982635 >> 172.21.15.165:0/2470982635 conn(0x7fa7e0061cc0 msgr2=0x7fa7e006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 -- 172.21.15.165:0/2470982635 shutdown_connections 2024-04-25T10:06:02.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.391+0000 7fa7e5800640 1 -- 172.21.15.165:0/2470982635 wait complete. 2024-04-25T10:06:02.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.392+0000 7fa7e5800640 1 Processor -- start 2024-04-25T10:06:02.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.392+0000 7fa7e5800640 1 -- start start 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.393+0000 7fa7e5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e00c5030 0x7fa7e0143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.393+0000 7fa7e5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 0x7fa7e015aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.393+0000 7fa7e5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e0145140 0x7fa7e01f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.393+0000 7fa7e5800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00c54a0 con 0x7fa7e00ce950 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7e5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0071b10 con 0x7fa7e00c5030 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7e5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00c5620 con 0x7fa7e0145140 2024-04-25T10:06:02.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 0x7fa7e015aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 0x7fa7e015aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49270/0 (socket says 172.21.15.165:49270) 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 -- 172.21.15.165:0/1327395471 learned_addr learned my addr 172.21.15.165:0/1327395471 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 -- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e0145140 msgr2=0x7fa7e01f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e0145140 0x7fa7e01f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e00c5030 0x7fa7e0143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 -- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e00c5030 msgr2=0x7fa7e0143500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e00c5030 0x7fa7e0143500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.394+0000 7fa7d7400640 1 -- 172.21.15.165:0/1327395471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e0143a40 con 0x7fa7e00ce950 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.395+0000 7fa7d7400640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 0x7fa7e015aac0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c7c0 tx=0x7fa7c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.395+0000 7fa7dcc00640 1 -- 172.21.15.165:0/1327395471 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8015070 con 0x7fa7e00ce950 2024-04-25T10:06:02.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.395+0000 7fa7dcc00640 1 -- 172.21.15.165:0/1327395471 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c80079d0 con 0x7fa7e00ce950 2024-04-25T10:06:02.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.395+0000 7fa7dcc00640 1 -- 172.21.15.165:0/1327395471 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8007da0 con 0x7fa7e00ce950 2024-04-25T10:06:02.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.395+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e0141660 con 0x7fa7e00ce950 2024-04-25T10:06:02.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.395+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e0141be0 con 0x7fa7e00ce950 2024-04-25T10:06:02.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.397+0000 7fa7dcc00640 1 -- 172.21.15.165:0/1327395471 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7c8018070 con 0x7fa7e00ce950 2024-04-25T10:06:02.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.398+0000 7fa7dcc00640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7bc06b4d0 0x7fa7bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.398+0000 7fa7dcc00640 1 -- 172.21.15.165:0/1327395471 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa7c808a300 con 0x7fa7e00ce950 2024-04-25T10:06:02.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.398+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7bc06b4d0 0x7fa7bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.399+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7bc06b4d0 0x7fa7bc06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b840 tx=0x7fa7d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.399+0000 7fa7e9c6e740 1 Processor -- start 2024-04-25T10:06:02.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 -- start start 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c4fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c1ffb0 0x5654d2c47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654d2b88870 con 0x5654d2c50290 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654d2b31d60 con 0x5654d2c1ffb0 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.400+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654d2b84910 con 0x5654d2c4f870 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49286/0 (socket says 172.21.15.165:49286) 2024-04-25T10:06:02.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 -- 172.21.15.165:0/3340702209 learned_addr learned my addr 172.21.15.165:0/3340702209 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 -- 172.21.15.165:0/3340702209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 msgr2=0x5654d2c4fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c4fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 -- 172.21.15.165:0/3340702209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c1ffb0 msgr2=0x5654d2c47cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c1ffb0 0x5654d2c47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 -- 172.21.15.165:0/3340702209 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654d29677d0 con 0x5654d2c50290 2024-04-25T10:06:02.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.401+0000 7fa7d7400640 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1fa70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8060a00 tx=0x7fa7c8058fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b086173ba4a8ebf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7d4c00640 1 -- 172.21.15.165:0/3340702209 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c801f040 con 0x5654d2c50290 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7d4c00640 1 -- 172.21.15.165:0/3340702209 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c808a0c0 con 0x5654d2c50290 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7d4c00640 1 -- 172.21.15.165:0/3340702209 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8060f30 con 0x5654d2c50290 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3340702209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 msgr2=0x5654d2c1fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1fa70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8060a00 tx=0x7fa7c8058fe0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3340702209 shutdown_connections 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c1ffb0 0x5654d2c47cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1fa70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3340702209 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c4fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.402+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3340702209 >> 172.21.15.165:0/3340702209 conn(0x5654d2c21ef0 msgr2=0x5654d2c6a680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.403+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3340702209 shutdown_connections 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.403+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3340702209 wait complete. 2024-04-25T10:06:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.403+0000 7fa7e9c6e740 1 -- start start 2024-04-25T10:06:02.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.404+0000 7fa7e9c6e740 1 Processor -- start 2024-04-25T10:06:02.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.404+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c452d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c50290 0x5654d2c71f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c72440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654d2c4bfc0 con 0x5654d2c4f870 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654d2b451c0 con 0x5654d2c45e90 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654d2c4c260 con 0x5654d2c50290 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7e9c6e740 10 client.0 _ll_register_callbacks cb 0x5654d2b838f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c452d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c452d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49302/0 (socket says 172.21.15.165:49302) 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 -- 172.21.15.165:0/4110611412 learned_addr learned my addr 172.21.15.165:0/4110611412 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 -- 172.21.15.165:0/4110611412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c50290 msgr2=0x5654d2c71f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c50290 0x5654d2c71f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 -- 172.21.15.165:0/4110611412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 msgr2=0x5654d2c72440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c72440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 -- 172.21.15.165:0/4110611412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654d285d0f0 con 0x5654d2c4f870 2024-04-25T10:06:02.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c452d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0012750 tx=0x7fa7d0012840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de52dc622abd9083 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.405+0000 7fa7cd600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7cd600640 1 -- 172.21.15.165:0/4110611412 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d0023060 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7cd600640 1 -- 172.21.15.165:0/4110611412 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa7d001e600 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7c808a0c0 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7cd600640 1 -- 172.21.15.165:0/4110611412 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7d0029050 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7e9c6e740 10 client.5601 fetch_fsmap learned FSMap version 20 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x5654d29c7f60 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7cd600640 1 -- 172.21.15.165:0/4110611412 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c808a0c0 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7cd600640 1 -- 172.21.15.165:0/4110611412 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa7d002c020 con 0x5654d2c4f870 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7e9c6e740 10 client.5601 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.406+0000 7fa7e9c6e740 10 client.5601 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7e9c6e740 20 client.5601 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2c6f220 0x5654d2c7d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7e9c6e740 4 client.5601 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:02.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654d2b5e690 con 0x5654d2c6f220 2024-04-25T10:06:02.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7d7400640 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2c6f220 0x5654d2c7d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7d7400640 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2c6f220 0x5654d2c7d6b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d0ebe5591aa7d53 server_cookie=d98ac049e284e306 in_seq=0 out_seq=0 2024-04-25T10:06:02.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.407+0000 7fa7cd600640 10 client.5601 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7cd600640 1 -- 172.21.15.165:0/4110611412 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5654d2b5e690 con 0x5654d2c6f220 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7cd600640 10 client.5601 handle_command_reply: tid=0 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 client.5601 shutdown 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 msgr2=0x5654d2c452d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c452d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0012750 tx=0x7fa7d0012840 comp rx=0 tx=0).stop 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 shutdown_connections 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c72440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c50290 0x5654d2c71f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2c6f220 0x5654d2c7d6b0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/4110611412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c452d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 >> 172.21.15.165:0/4110611412 conn(0x5654d2c1ffb0 msgr2=0x5654d2c23430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.408+0000 7fa7cd600640 0 client.5601 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.409+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 shutdown_connections 2024-04-25T10:06:02.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.409+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/4110611412 wait complete. 2024-04-25T10:06:02.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.409+0000 7fa7e9c6e740 20 client.5601 trim_cache size 0 max 16384 2024-04-25T10:06:02.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.424+0000 7fa7e9c6e740 1 Processor -- start 2024-04-25T10:06:02.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.424+0000 7fa7e9c6e740 1 -- start start 2024-04-25T10:06:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c45e90 0x5654d2c43e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c26870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c50290 0x5654d2c1ec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e0200270 con 0x5654d2c4f870 2024-04-25T10:06:02.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01ff3b0 con 0x5654d2c50290 2024-04-25T10:06:02.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e013f950 con 0x5654d2c45e90 2024-04-25T10:06:02.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c26870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c26870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49312/0 (socket says 172.21.15.165:49312) 2024-04-25T10:06:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.425+0000 7fa7d7400640 1 -- 172.21.15.165:0/152862034 learned_addr learned my addr 172.21.15.165:0/152862034 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7d7400640 1 -- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c45e90 msgr2=0x5654d2c43e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c50290 0x5654d2c1ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c45e90 0x5654d2c43e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7d7400640 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c45e90 0x5654d2c43e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7d7400640 1 -- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c50290 msgr2=0x5654d2c1ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7d7400640 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c50290 0x5654d2c1ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7d7400640 1 -- 172.21.15.165:0/152862034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654d28361b0 con 0x5654d2c4f870 2024-04-25T10:06:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c50290 0x5654d2c1ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7d7400640 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c26870 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800cfd0 tx=0x7fa7c805e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b813fc23700363 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.426+0000 7fa7cfe00640 1 -- 172.21.15.165:0/152862034 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c8086070 con 0x5654d2c4f870 2024-04-25T10:06:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7cfe00640 1 -- 172.21.15.165:0/152862034 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8056b20 con 0x5654d2c4f870 2024-04-25T10:06:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7cfe00640 1 -- 172.21.15.165:0/152862034 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c801f040 con 0x5654d2c4f870 2024-04-25T10:06:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/152862034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 msgr2=0x5654d2c26870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c26870 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800cfd0 tx=0x7fa7c805e000 comp rx=0 tx=0).stop 2024-04-25T10:06:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/152862034 shutdown_connections 2024-04-25T10:06:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c50290 0x5654d2c1ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c4f870 0x5654d2c26870 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/152862034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c45e90 0x5654d2c43e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/152862034 >> 172.21.15.165:0/152862034 conn(0x5654d2c1ffb0 msgr2=0x5654d2c1b120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/152862034 shutdown_connections 2024-04-25T10:06:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.427+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/152862034 wait complete. 2024-04-25T10:06:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.428+0000 7fa7e9c6e740 1 -- start start 2024-04-25T10:06:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.428+0000 7fa7e9c6e740 1 Processor -- start 2024-04-25T10:06:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c69580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c1ed40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1f280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e01fee10 con 0x5654d2c50290 2024-04-25T10:06:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654d2c500d0 con 0x5654d2c45e90 2024-04-25T10:06:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5654d2ce0a20 con 0x5654d2c4f870 2024-04-25T10:06:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e9c6e740 10 client.0 _ll_register_callbacks cb 0x7fa7c800ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49328/0 (socket says 172.21.15.165:49328) 2024-04-25T10:06:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.429+0000 7fa7e4e00640 1 -- 172.21.15.165:0/3594740565 learned_addr learned my addr 172.21.15.165:0/3594740565 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:02.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7d7400640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c1ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c69580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e4e00640 1 -- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 msgr2=0x5654d2c1ed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c1ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e4e00640 1 -- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 msgr2=0x5654d2c69580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:02.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c69580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e4e00640 1 -- 172.21.15.165:0/3594740565 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5654d2c69ac0 con 0x5654d2c50290 2024-04-25T10:06:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c69580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e4e00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1f280 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009f40 tx=0x7fa7d8031b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fc96469c0719 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7cd600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7cd600640 1 -- 172.21.15.165:0/3594740565 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8041020 con 0x5654d2c50290 2024-04-25T10:06:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa7c8056b20 con 0x5654d2c50290 2024-04-25T10:06:02.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.430+0000 7fa7cd600640 1 -- 172.21.15.165:0/3594740565 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa7d803c470 con 0x5654d2c50290 2024-04-25T10:06:02.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7cd600640 1 -- 172.21.15.165:0/3594740565 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7d804b050 con 0x5654d2c50290 2024-04-25T10:06:02.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 10 client.5607 fetch_fsmap learned FSMap version 20 2024-04-25T10:06:02.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x5654d2c25940 con 0x5654d2c50290 2024-04-25T10:06:02.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7cd600640 1 -- 172.21.15.165:0/3594740565 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c8056b20 con 0x5654d2c50290 2024-04-25T10:06:02.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7cd600640 1 -- 172.21.15.165:0/3594740565 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 20) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa7d804e020 con 0x5654d2c50290 2024-04-25T10:06:02.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 10 client.5607 fetch_fsmap finished waiting for FSMap version 20 2024-04-25T10:06:02.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 10 client.5607 resolve_mds: resolved 2:0 to role '2:0' aka daemon mds.a 2024-04-25T10:06:02.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 20 client.5607 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:02.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2ce02b0 0x5654d2cdc140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:02.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 4 client.5607 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 --> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5654d2b95db0 con 0x5654d2ce02b0 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.431+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2ce02b0 0x5654d2cdc140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.432+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2ce02b0 0x5654d2cdc140 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a64c09b47af4e372 server_cookie=1cab5154f42460a1 in_seq=0 out_seq=0 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.432+0000 7fa7cd600640 10 client.5607 ms_handle_connect on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.432+0000 7fa7cd600640 1 -- 172.21.15.165:0/3594740565 <== mds.0 v2:172.21.15.165:6836/3282481342 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17947 (crc 0 0 0) 0x5654d2b95db0 con 0x5654d2ce02b0 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.432+0000 7fa7cd600640 10 client.5607 handle_command_reply: tid=0 2024-04-25T10:06:02.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.432+0000 7fa7e9c6e740 1 client.5607 shutdown 2024-04-25T10:06:02.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 msgr2=0x5654d2c1f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1f280 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8009f40 tx=0x7fa7d8031b50 comp rx=0 tx=0).stop 2024-04-25T10:06:02.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 shutdown_connections 2024-04-25T10:06:02.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:6836/3282481342,v1:172.21.15.165:6837/3282481342] conn(0x5654d2ce02b0 0x5654d2cdc140 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5654d2c50290 0x5654d2c1f280 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5654d2c4f870 0x5654d2c1ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 --2- 172.21.15.165:0/3594740565 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5654d2c45e90 0x5654d2c69580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7cd600640 0 client.5607 ms_handle_reset on v2:172.21.15.165:6836/3282481342 2024-04-25T10:06:02.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 >> 172.21.15.165:0/3594740565 conn(0x5654d2c1ffb0 msgr2=0x5654d2c8cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 shutdown_connections 2024-04-25T10:06:02.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 1 -- 172.21.15.165:0/3594740565 wait complete. 2024-04-25T10:06:02.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.433+0000 7fa7e9c6e740 20 client.5607 trim_cache size 0 max 16384 2024-04-25T10:06:02.457 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:02.457 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:06:02.457 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 2747, 2024-04-25T10:06:02.457 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 2741, 2024-04-25T10:06:02.457 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 794695, 2024-04-25T10:06:02.457 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 17948306, 2024-04-25T10:06:02.458 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 16, 2024-04-25T10:06:02.458 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-04-25T10:06:02.458 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.510476786, 2024-04-25T10:06:02.458 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.238429215, 2024-04-25T10:06:02.458 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.113181487, 2024-04-25T10:06:02.458 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.119606048, 2024-04-25T10:06:02.459 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:06:02.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2740, 2024-04-25T10:06:02.459 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.594799592, 2024-04-25T10:06:02.459 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000217080 2024-04-25T10:06:02.459 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.459 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000001184, 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000118 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 124880, 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 33616 2024-04-25T10:06:02.460 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3502, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3533, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 881284, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 25559496, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:06:02.461 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.614633597, 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.316067891, 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.111966115, 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.150621781, 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3532, 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.747670875, 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000211684 2024-04-25T10:06:02.462 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 139504 2024-04-25T10:06:02.463 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 32441, 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 53254, 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 11305375, 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 77836496, 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 8, 2024-04-25T10:06:02.464 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 8.762359148, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 6.248394599, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 2.502054583, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.234483356, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 53253, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.954598749, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000074260 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.465 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 45930, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.020678660, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000450 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:06:02.466 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6688, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 28.205653760, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004217352 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4082, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.337326019, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000572593 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-25T10:06:02.467 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 25965, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 25965, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 25965, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 500.736105471, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019285041 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 12, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-25T10:06:02.468 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 17872, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 17872, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 17872, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 2, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 2, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 48775, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 48215, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-25T10:06:02.469 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 171, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 2013, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-25T10:06:02.470 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 2013, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:06:02.471 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 13, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 2, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 11, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 2000, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 871, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 4039, 2024-04-25T10:06:02.472 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:06:02.473 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:06:02.473 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:06:02.473 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 3911, 2024-04-25T10:06:02.473 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:06:02.473 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 60, 2024-04-25T10:06:02.474 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 14059 2024-04-25T10:06:02.474 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.474 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-25T10:06:02.474 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-25T10:06:02.474 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-25T10:06:02.474 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-25T10:06:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-25T10:06:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-25T10:06:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-25T10:06:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-25T10:06:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:06:02.475 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 4121, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 3840, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 2230, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-25T10:06:02.476 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-25T10:06:02.477 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.478 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-25T10:06:02.478 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-25T10:06:02.478 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 27240, 2024-04-25T10:06:02.478 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-25T10:06:02.478 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-25T10:06:02.478 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 27240, 2024-04-25T10:06:02.479 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-25T10:06:02.479 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-25T10:06:02.479 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 36, 2024-04-25T10:06:02.479 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-25T10:06:02.479 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-25T10:06:02.479 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 36, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 4, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 96619203, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-25T10:06:02.480 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4460, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 130.604728767, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.029283571 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 14, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 2016, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 2002, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 13, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 13, 2024-04-25T10:06:02.481 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 17872, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 19873, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 2001, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 2, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 7876, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 7874, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 97032, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215396 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 29743, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 25965, 2024-04-25T10:06:02.482 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 9, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2000, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.668961226, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001334480 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 29, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006740537, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000232432 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.483 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6000, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 13.460080925, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002243346 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 7693, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 14.266013958, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001854414 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.484 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-25T10:06:02.485 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000065330, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000065330 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.486 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.005215566, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005215566 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.016444525, 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.016444525 2024-04-25T10:06:02.487 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.153999593, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.019249949 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2230, 2024-04-25T10:06:02.488 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 439.303651437, 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.196997153 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-25T10:06:02.489 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.021811205, 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.021811205 2024-04-25T10:06:02.490 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000310609, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000310609 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.491 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.492 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8000, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 30.832810560, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003854101 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-25T10:06:02.493 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 15017, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 15017, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 132, 2024-04-25T10:06:02.494 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:06:02.495 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:06:02.496 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 250803, 2024-04-25T10:06:02.497 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 92, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 440, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 5, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-25T10:06:02.498 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 5624, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 80, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 13457741, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 661297, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-25T10:06:02.499 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 10680, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 93034412, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 31, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 10680, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10680, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 65.482002325, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006131273 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-25T10:06:02.500 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10680, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 10836 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 10680, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 2095, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 8585, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 11, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 80, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 4538, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 48, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-25T10:06:02.501 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 1840, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 2034, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 2015, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 22, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-25T10:06:02.502 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-25T10:06:02.503 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 43, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 5, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 8, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 8, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 197, 2024-04-25T10:06:02.504 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 51 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-25T10:06:02.505 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 32, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 3840, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 3840, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:06:02.506 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 38686, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 9939908, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 38686, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 38686, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 9939908, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.507 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 10680, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 93436972, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 10680, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 93436972, 2024-04-25T10:06:02.508 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:06:02.509 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 10680, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 10680, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 10680, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 10680, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:06:02.510 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 3840, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 402560, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 3840, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 78, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 402560, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:02.511 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55be7e3686e0": { 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 27240, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 92424899, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 27240, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4460, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 92424899, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:06:02.512 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:06:02.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.513 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:02.513 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:02.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7bc06b4d0 msgr2=0x7fa7bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7bc06b4d0 0x7fa7bc06d990 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fa7d000b840 tx=0x7fa7d0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:02.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 msgr2=0x7fa7e015aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:02.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 0x7fa7e015aac0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fa7c800c7c0 tx=0x7fa7c800cc80 comp rx=0 tx=0).stop 2024-04-25T10:06:02.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 shutdown_connections 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7bc06b4d0 0x7fa7bc06d990 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e0145140 0x7fa7e01f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7e00ce950 0x7fa7e015aac0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 --2- 172.21.15.165:0/1327395471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e00c5030 0x7fa7e0143500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 >> 172.21.15.165:0/1327395471 conn(0x7fa7e0061cc0 msgr2=0x7fa7e00cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 shutdown_connections 2024-04-25T10:06:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:02.434+0000 7fa7e5800640 1 -- 172.21.15.165:0/1327395471 wait complete. 2024-04-25T10:06:02.516 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:02.516 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:02.637 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:02.742 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:02.743 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:02.743 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:02.800 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:02.801 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:02.852 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:03.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.099+0000 7f63a0400640 1 Processor -- start 2024-04-25T10:06:03.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.100+0000 7f63a0400640 1 -- start start 2024-04-25T10:06:03.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.101+0000 7f63a0400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63980aecf0 0x7f63980cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.101+0000 7f63a0400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f63980cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.101+0000 7f63a0400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63980cf330 con 0x7f6398165680 2024-04-25T10:06:03.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.101+0000 7f63a0400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6398072340 con 0x7f63980aecf0 2024-04-25T10:06:03.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.101+0000 7f63a0400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398071670 con 0x7f63980b2650 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.102+0000 7f639d000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63980aecf0 0x7f63980cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.102+0000 7f639e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f63980cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.102+0000 7f639da00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f6398165680 0x7f63980cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:36046/0 (socket says 172.21.15.165:36046) 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.102+0000 7f639da00640 1 -- 172.21.15.165:0/3245483787 learned_addr learned my addr 172.21.15.165:0/3245483787 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:03.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.103+0000 7f639d000640 1 -- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 msgr2=0x7f63980cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.103+0000 7f639d000640 1 --2- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f63980cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.103+0000 7f639d000640 1 -- 172.21.15.165:0/3245483787 >> v1:172.21.15.172:6789/0 conn(0x7f6398165680 legacy=0x7f63980cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.103+0000 7f639d000640 1 -- 172.21.15.165:0/3245483787 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6398071d80 con 0x7f63980aecf0 2024-04-25T10:06:03.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.103+0000 7f639e400640 1 --2- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f63980cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.104+0000 7f639d000640 1 --2- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980cec70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63840097a0 tx=0x7f638402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7760a194f4c6f30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.105+0000 7f6397e00640 1 -- 172.21.15.165:0/3245483787 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384037070 con 0x7f63980aecf0 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.105+0000 7f6397e00640 1 -- 172.21.15.165:0/3245483787 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f638402ae50 con 0x7f63980aecf0 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.105+0000 7f6397e00640 1 -- 172.21.15.165:0/3245483787 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6384032660 con 0x7f63980aecf0 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 -- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 msgr2=0x7f63980cec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 --2- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980cec70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f63840097a0 tx=0x7f638402a100 comp rx=0 tx=0).stop 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 -- 172.21.15.165:0/3245483787 shutdown_connections 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 --2- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f63980cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 --2- 172.21.15.165:0/3245483787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980cec70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 -- 172.21.15.165:0/3245483787 >> 172.21.15.165:0/3245483787 conn(0x7f639806abe0 msgr2=0x7f639806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 -- 172.21.15.165:0/3245483787 shutdown_connections 2024-04-25T10:06:03.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.106+0000 7f63a0400640 1 -- 172.21.15.165:0/3245483787 wait complete. 2024-04-25T10:06:03.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.108+0000 7f63a0400640 1 Processor -- start 2024-04-25T10:06:03.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.108+0000 7f63a0400640 1 -- start start 2024-04-25T10:06:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f63a0400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f63a0400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f6398157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f63a0400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 0x7f639815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f63a0400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6398165a60 con 0x7f6398165680 2024-04-25T10:06:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f63a0400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639806ea30 con 0x7f63980aecf0 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f63a0400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63980b2a60 con 0x7f63980b2650 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f639e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 0x7f639815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f639e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 0x7f639815ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49336/0 (socket says 172.21.15.165:49336) 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.109+0000 7f639e400640 1 -- 172.21.15.165:0/430442846 learned_addr learned my addr 172.21.15.165:0/430442846 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639da00640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639d000640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f6398157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639e400640 1 -- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 msgr2=0x7f6398157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639e400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f6398157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639e400640 1 -- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 msgr2=0x7f63980af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639e400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639e400640 1 -- 172.21.15.165:0/430442846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63980af800 con 0x7f6398165680 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639d000640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f6398157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f639e400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 0x7f639815ead0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6390009f00 tx=0x7f639000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.110+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390018030 con 0x7f6398165680 2024-04-25T10:06:03.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.111+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f639000ce50 con 0x7f6398165680 2024-04-25T10:06:03.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.111+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63980afad0 con 0x7f6398165680 2024-04-25T10:06:03.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.111+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390003bb0 con 0x7f6398165680 2024-04-25T10:06:03.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.111+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63980b0090 con 0x7f6398165680 2024-04-25T10:06:03.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.112+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398073990 con 0x7f6398165680 2024-04-25T10:06:03.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.113+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6390005d10 con 0x7f6398165680 2024-04-25T10:06:03.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.114+0000 7f6395600640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.114+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f639008ae90 con 0x7f6398165680 2024-04-25T10:06:03.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.114+0000 7f639da00640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.115+0000 7f639da00640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63980b1810 tx=0x7f638c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.118+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6390022070 con 0x7f6398165680 2024-04-25T10:06:03.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.314+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63980ceef0 con 0x7f6398165680 2024-04-25T10:06:03.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.316+0000 7f6395600640 1 -- 172.21.15.165:0/430442846 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17036 (secure 0 0 0) 0x7f639008c090 con 0x7f6398165680 2024-04-25T10:06:03.322 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:03.322 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":43,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:05:39.026672+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:03.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.319+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 msgr2=0x7f637806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.319+0000 7f63a0400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f63980b1810 tx=0x7f638c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:03.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.319+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 msgr2=0x7f639815ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.319+0000 7f63a0400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 0x7f639815ead0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6390009f00 tx=0x7f639000c490 comp rx=0 tx=0).stop 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.319+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 shutdown_connections 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6398165680 0x7f639815ead0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980b2650 0x7f6398157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 --2- 172.21.15.165:0/430442846 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63980aecf0 0x7f63980af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 >> 172.21.15.165:0/430442846 conn(0x7f639806abe0 msgr2=0x7f639806d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 shutdown_connections 2024-04-25T10:06:03.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.320+0000 7f63a0400640 1 -- 172.21.15.165:0/430442846 wait complete. 2024-04-25T10:06:03.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:38.087466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:39.315047+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:03.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:03.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.545+0000 7fab4b000640 1 Processor -- start 2024-04-25T10:06:03.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.546+0000 7fab4b000640 1 -- start start 2024-04-25T10:06:03.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.547+0000 7fab4b000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fab441446d0 0x7fab440b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.547+0000 7fab4b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab440b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.547+0000 7fab4b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fab441452a0 0x7fab440cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.547+0000 7fab4b000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab440b2b20 con 0x7fab441446d0 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.547+0000 7fab4b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4406d400 con 0x7fab441452a0 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.547+0000 7fab4b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab440b49a0 con 0x7fab44144d50 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fab441446d0 0x7fab440b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fab441446d0 0x7fab440b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49340/0 (socket says 172.21.15.165:49340) 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 -- 172.21.15.165:0/3149237237 learned_addr learned my addr 172.21.15.165:0/3149237237 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 -- 172.21.15.165:0/3149237237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 msgr2=0x7fab440b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab49000640 1 --2- 172.21.15.165:0/3149237237 >> v2:172.21.15.172:3300/0 conn(0x7fab441452a0 0x7fab440cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43400640 1 --2- 172.21.15.165:0/3149237237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab440b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 --2- 172.21.15.165:0/3149237237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab440b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 -- 172.21.15.165:0/3149237237 >> v2:172.21.15.172:3300/0 conn(0x7fab441452a0 msgr2=0x7fab440cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 --2- 172.21.15.165:0/3149237237 >> v2:172.21.15.172:3300/0 conn(0x7fab441452a0 0x7fab440cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab43e00640 1 -- 172.21.15.165:0/3149237237 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab440cccc0 con 0x7fab441446d0 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.548+0000 7fab49000640 1 --2- 172.21.15.165:0/3149237237 >> v2:172.21.15.172:3300/0 conn(0x7fab441452a0 0x7fab440cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.549+0000 7fab43e00640 1 --2- 172.21.15.165:0/3149237237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441446d0 0x7fab440b3f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fab44145880 tx=0x7fab3402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67af4f6d2e536f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab42a00640 1 -- 172.21.15.165:0/3149237237 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34037070 con 0x7fab441446d0 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab42a00640 1 -- 172.21.15.165:0/3149237237 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab34032440 con 0x7fab441446d0 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab42a00640 1 -- 172.21.15.165:0/3149237237 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab34032810 con 0x7fab441446d0 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 -- 172.21.15.165:0/3149237237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441446d0 msgr2=0x7fab440b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 --2- 172.21.15.165:0/3149237237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441446d0 0x7fab440b3f20 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fab44145880 tx=0x7fab3402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 -- 172.21.15.165:0/3149237237 shutdown_connections 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 --2- 172.21.15.165:0/3149237237 >> v2:172.21.15.172:3300/0 conn(0x7fab441452a0 0x7fab440cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 --2- 172.21.15.165:0/3149237237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab440b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 --2- 172.21.15.165:0/3149237237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441446d0 0x7fab440b3f20 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.550+0000 7fab4b000640 1 -- 172.21.15.165:0/3149237237 >> 172.21.15.165:0/3149237237 conn(0x7fab4406abe0 msgr2=0x7fab4406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.551+0000 7fab4b000640 1 -- 172.21.15.165:0/3149237237 shutdown_connections 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.551+0000 7fab4b000640 1 -- 172.21.15.165:0/3149237237 wait complete. 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.552+0000 7fab4b000640 1 Processor -- start 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.552+0000 7fab4b000640 1 -- start start 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.553+0000 7fab4b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab441446d0 0x7fab4413dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.553+0000 7fab4b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab441e4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.553+0000 7fab4b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 0x7fab440b1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.553+0000 7fab4b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440cc700 con 0x7fab441452a0 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.553+0000 7fab4b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab440cc420 con 0x7fab441446d0 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.553+0000 7fab4b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab440cc9e0 con 0x7fab44144d50 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 0x7fab440b1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 0x7fab440b1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49350/0 (socket says 172.21.15.165:49350) 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 -- 172.21.15.165:0/1761629523 learned_addr learned my addr 172.21.15.165:0/1761629523 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 -- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 msgr2=0x7fab441e4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab441e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab43e00640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab441446d0 0x7fab4413dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 -- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab441446d0 msgr2=0x7fab4413dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab441446d0 0x7fab4413dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 -- 172.21.15.165:0/1761629523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab441e3170 con 0x7fab441452a0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.554+0000 7fab49000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 0x7fab440b1010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00cd80 tx=0x7fab3c00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.555+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00f680 con 0x7fab441452a0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.555+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3c00f840 con 0x7fab441452a0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.555+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3c00fc30 con 0x7fab441452a0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.555+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab441e3450 con 0x7fab441452a0 2024-04-25T10:06:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.555+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab441e3a10 con 0x7fab441452a0 2024-04-25T10:06:03.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.556+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab441e5740 con 0x7fab441452a0 2024-04-25T10:06:03.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.557+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab3c00e3e0 con 0x7fab441452a0 2024-04-25T10:06:03.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.559+0000 7fab3be00640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab2006b4d0 0x7fab2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.559+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fab3c08b220 con 0x7fab441452a0 2024-04-25T10:06:03.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.559+0000 7fab43e00640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab2006b4d0 0x7fab2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.560+0000 7fab43e00640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab2006b4d0 0x7fab2006d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab34034000 tx=0x7fab3402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.563+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab3c014030 con 0x7fab441452a0 2024-04-25T10:06:03.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.756+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fab4406eb00 con 0x7fab441452a0 2024-04-25T10:06:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.757+0000 7fab3be00640 1 -- 172.21.15.165:0/1761629523 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7fab3c059e40 con 0x7fab441452a0 2024-04-25T10:06:03.763 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab2006b4d0 msgr2=0x7fab2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab2006b4d0 0x7fab2006d990 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fab34034000 tx=0x7fab3402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 msgr2=0x7fab440b1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 0x7fab440b1010 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fab3c00cd80 tx=0x7fab3c00a650 comp rx=0 tx=0).stop 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 shutdown_connections 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab2006b4d0 0x7fab2006d990 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab441452a0 0x7fab440b1010 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab44144d50 0x7fab441e4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 --2- 172.21.15.165:0/1761629523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab441446d0 0x7fab4413dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.760+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 >> 172.21.15.165:0/1761629523 conn(0x7fab4406abe0 msgr2=0x7fab441437e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.761+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 shutdown_connections 2024-04-25T10:06:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.761+0000 7fab4b000640 1 -- 172.21.15.165:0/1761629523 wait complete. 2024-04-25T10:06:03.784 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:03.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.979+0000 7f3fb2c00640 1 Processor -- start 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.980+0000 7f3fb2c00640 1 -- start start 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.981+0000 7f3fb2c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3fac165a80 0x7f3fac165f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.981+0000 7f3fb2c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac0cdcd0 con 0x7f3fac0ccf30 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.981+0000 7f3fb2c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fac1632d0 con 0x7f3fac15f950 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.981+0000 7f3fb2c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fac06f100 con 0x7f3fac165a80 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.982+0000 7f3fb0e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3fac0ccf30 0x7f3fac162bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42650/0 (socket says 172.21.15.165:42650) 2024-04-25T10:06:03.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.982+0000 7f3fb0e00640 1 -- 172.21.15.165:0/3042363933 learned_addr learned my addr 172.21.15.165:0/3042363933 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:03.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.982+0000 7f3fb2200640 1 --2- 172.21.15.165:0/3042363933 >> v2:172.21.15.172:3300/0 conn(0x7f3fac165a80 0x7f3fac165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.982+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3824895151 0 0) 0x7f3fac0cdcd0 con 0x7f3fac0ccf30 2024-04-25T10:06:03.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.982+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f94003520 con 0x7f3fac0ccf30 2024-04-25T10:06:03.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2375062296 0 0) 0x7f3f94003520 con 0x7f3fac0ccf30 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 >> v2:172.21.15.172:3300/0 conn(0x7f3fac165a80 msgr2=0x7f3fac165f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 --2- 172.21.15.165:0/3042363933 >> v2:172.21.15.172:3300/0 conn(0x7f3fac165a80 0x7f3fac165f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 >> v1:172.21.15.172:6789/0 conn(0x7f3fac15f950 legacy=0x7f3fac06e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac0cd430 con 0x7f3fac0ccf30 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3f9c001670 con 0x7f3fac0ccf30 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 >> v1:172.21.15.165:6789/0 conn(0x7f3fac0ccf30 legacy=0x7f3fac162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f94003a00 0x7f3f94003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3f94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f94005210 0x7f3f9400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac06f100 con 0x7f3f94004800 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9400c160 con 0x7f3f94003a00 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9400c360 con 0x7f3f94005210 2024-04-25T10:06:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb0e00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3f94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb1800640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f94003a00 0x7f3f94003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb0e00640 1 -- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f94005210 msgr2=0x7f3f9400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb0e00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f94005210 0x7f3f9400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb0e00640 1 -- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f94003a00 msgr2=0x7f3f94003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb0e00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f94003a00 0x7f3f94003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.983+0000 7f3fb0e00640 1 -- 172.21.15.165:0/3042363933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94003520 con 0x7f3f94004800 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.984+0000 7f3fb1800640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f94003a00 0x7f3f94003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.984+0000 7f3fb2200640 1 -- 172.21.15.165:0/3042363933 reap_dead start 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.984+0000 7f3fb0e00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3f94004c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c007fd0 tx=0x7f3f94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ac2b6275f9aed1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c007c20 con 0x7f3f94004800 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f9c005af0 con 0x7f3f94004800 2024-04-25T10:06:03.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fabe00640 1 -- 172.21.15.165:0/3042363933 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c007070 con 0x7f3f94004800 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fb2c00640 1 -- 172.21.15.165:0/3042363933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 msgr2=0x7f3f94004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3f94004c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c007fd0 tx=0x7f3f94005830 comp rx=0 tx=0).stop 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fb2c00640 1 -- 172.21.15.165:0/3042363933 shutdown_connections 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/3042363933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3f94004c80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.985+0000 7f3fb2c00640 1 -- 172.21.15.165:0/3042363933 >> 172.21.15.165:0/3042363933 conn(0x7f3fac061cc0 msgr2=0x7f3fac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.986+0000 7f3fb2c00640 1 -- 172.21.15.165:0/3042363933 shutdown_connections 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.986+0000 7f3fb2c00640 1 -- 172.21.15.165:0/3042363933 wait complete. 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.987+0000 7f3fb2c00640 1 Processor -- start 2024-04-25T10:06:03.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.987+0000 7f3fb2c00640 1 -- start start 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.988+0000 7f3fb2c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3fac0cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.988+0000 7f3fb2c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fac15b710 0x7f3fac0c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.988+0000 7f3fb2c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fac1fb120 0x7f3fac1fb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.988+0000 7f3fb2c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac0cd9f0 con 0x7f3f94004800 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.988+0000 7f3fb2c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac162440 con 0x7f3fac15b710 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.988+0000 7f3fb2c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac1626a0 con 0x7f3fac1fb120 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3fac0cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3fac0cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49366/0 (socket says 172.21.15.165:49366) 2024-04-25T10:06:03.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 -- 172.21.15.165:0/2453142345 learned_addr learned my addr 172.21.15.165:0/2453142345 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:03.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 -- 172.21.15.165:0/2453142345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fac1fb120 msgr2=0x7f3fac1fb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fac1fb120 0x7f3fac1fb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 -- 172.21.15.165:0/2453142345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fac15b710 msgr2=0x7f3fac0c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:03.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fac15b710 0x7f3fac0c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:03.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 -- 172.21.15.165:0/2453142345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fac15bc80 con 0x7f3f94004800 2024-04-25T10:06:03.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.989+0000 7f3fb1800640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3fac0cb2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000d480 tx=0x7f3fa000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.990+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0005480 con 0x7f3f94004800 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.990+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0017070 con 0x7f3f94004800 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.990+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0003870 con 0x7f3f94004800 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.990+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac15bf60 con 0x7f3f94004800 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.990+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac15c520 con 0x7f3f94004800 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.991+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac15d070 con 0x7f3f94004800 2024-04-25T10:06:03.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.992+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fa0003a10 con 0x7f3f94004800 2024-04-25T10:06:03.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.993+0000 7f3fa9600640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f9006b500 0x7f3f9006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:03.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.993+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f3fa008a060 con 0x7f3f94004800 2024-04-25T10:06:03.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.994+0000 7f3fb0e00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f9006b500 0x7f3f9006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:03.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.994+0000 7f3fb0e00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f9006b500 0x7f3f9006d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c032000 tx=0x7f3f9c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:03.998+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fa0058e70 con 0x7f3f94004800 2024-04-25T10:06:04.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.192+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3fac073990 con 0x7f3f94004800 2024-04-25T10:06:04.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.193+0000 7f3fa9600640 1 -- 172.21.15.165:0/2453142345 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3fa0058770 con 0x7f3f94004800 2024-04-25T10:06:04.198 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:06:04.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f9006b500 msgr2=0x7f3f9006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f9006b500 0x7f3f9006d9c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c032000 tx=0x7f3f9c007000 comp rx=0 tx=0).stop 2024-04-25T10:06:04.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 msgr2=0x7f3fac0cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3fac0cb2f0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000d480 tx=0x7f3fa000d940 comp rx=0 tx=0).stop 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 shutdown_connections 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f9006b500 0x7f3f9006d9c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fac1fb120 0x7f3fac1fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fac15b710 0x7f3fac0c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 --2- 172.21.15.165:0/2453142345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f94004800 0x7f3fac0cb2f0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 >> 172.21.15.165:0/2453142345 conn(0x7f3fac061cc0 msgr2=0x7f3fac161460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 shutdown_connections 2024-04-25T10:06:04.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.196+0000 7f3fb2c00640 1 -- 172.21.15.165:0/2453142345 wait complete. 2024-04-25T10:06:04.220 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:04.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.421+0000 7fa900800640 1 Processor -- start 2024-04-25T10:06:04.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.422+0000 7fa900800640 1 -- start start 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa900800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f80b9750 0x7fa8f80ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa900800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa8f80cebb0 0x7fa8f80d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa900800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f8161d70 con 0x7fa8f815da90 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa900800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f8162600 con 0x7fa8f80cebb0 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa900800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80cf020 con 0x7fa8f80b9750 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa8fde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa8f815da90 0x7fa8f8161660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42660/0 (socket says 172.21.15.165:42660) 2024-04-25T10:06:04.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.423+0000 7fa8fde00640 1 -- 172.21.15.165:0/2331322164 learned_addr learned my addr 172.21.15.165:0/2331322164 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f80b9750 0x7fa8f80ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263802819 0 0) 0x7fa8f8161d70 con 0x7fa8f815da90 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fe800640 1 --2- 172.21.15.165:0/2331322164 >> v2:172.21.15.172:3300/0 conn(0x7fa8f80cebb0 0x7fa8f80d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8e4003520 con 0x7fa8f815da90 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3500956064 0 0) 0x7fa8e4003520 con 0x7fa8f815da90 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f80b9750 msgr2=0x7fa8f80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f80b9750 0x7fa8f80ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 >> v2:172.21.15.172:3300/0 conn(0x7fa8f80cebb0 msgr2=0x7fa8f80d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 --2- 172.21.15.165:0/2331322164 >> v2:172.21.15.172:3300/0 conn(0x7fa8f80cebb0 0x7fa8f80d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f8162040 con 0x7fa8f815da90 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa8f00019f0 con 0x7fa8f815da90 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 >> v1:172.21.15.165:6789/0 conn(0x7fa8f815da90 legacy=0x7fa8f8161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e40039b0 0x7fa8e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e4004800 0x7fa8e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8e40051c0 0x7fa8e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8162600 con 0x7fa8e4004800 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80cf020 con 0x7fa8e40039b0 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e400bf00 con 0x7fa8e40051c0 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.424+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e4004800 0x7fa8e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fe800640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8e40051c0 0x7fa8e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fde00640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e40039b0 0x7fa8e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fd400640 1 -- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8e40051c0 msgr2=0x7fa8e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8e40051c0 0x7fa8e400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fd400640 1 -- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e40039b0 msgr2=0x7fa8e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e40039b0 0x7fa8e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fd400640 1 -- 172.21.15.165:0/2331322164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e4003520 con 0x7fa8e4004800 2024-04-25T10:06:04.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.425+0000 7fa8fde00640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e40039b0 0x7fa8e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.426+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e4004800 0x7fa8e4004c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009a90 tx=0x7fa8e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5bda88cf61596f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.426+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8036070 con 0x7fa8e4004800 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.426+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e8031400 con 0x7fa8e4004800 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.426+0000 7fa8fca00640 1 -- 172.21.15.165:0/2331322164 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8009690 con 0x7fa8e4004800 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa900800640 1 -- 172.21.15.165:0/2331322164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e4004800 msgr2=0x7fa8e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa900800640 1 --2- 172.21.15.165:0/2331322164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8e4004800 0x7fa8e4004c80 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009a90 tx=0x7fa8e4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa8fe800640 1 -- 172.21.15.165:0/2331322164 reap_dead start 2024-04-25T10:06:04.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa900800640 1 -- 172.21.15.165:0/2331322164 shutdown_connections 2024-04-25T10:06:04.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa900800640 1 -- 172.21.15.165:0/2331322164 >> 172.21.15.165:0/2331322164 conn(0x7fa8f8061a20 msgr2=0x7fa8f8061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa900800640 1 -- 172.21.15.165:0/2331322164 shutdown_connections 2024-04-25T10:06:04.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.427+0000 7fa900800640 1 -- 172.21.15.165:0/2331322164 wait complete. 2024-04-25T10:06:04.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.429+0000 7fa900800640 1 Processor -- start 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.429+0000 7fa900800640 1 -- start start 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa900800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e4004800 0x7fa8f81f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa900800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f815ff30 0x7fa8f81598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa900800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 0x7fa8f8159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa900800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80d1df0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa900800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f80d1b10 con 0x7fa8e4004800 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa900800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f815de70 con 0x7fa8f815ff30 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa8fe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 0x7fa8f8159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa8fe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 0x7fa8f8159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49386/0 (socket says 172.21.15.165:49386) 2024-04-25T10:06:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.430+0000 7fa8fe800640 1 -- 172.21.15.165:0/2086496190 learned_addr learned my addr 172.21.15.165:0/2086496190 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fde00640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e4004800 0x7fa8f81f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f815ff30 0x7fa8f81598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fe800640 1 -- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f815ff30 msgr2=0x7fa8f81598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fe800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f815ff30 0x7fa8f81598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fe800640 1 -- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e4004800 msgr2=0x7fa8f81f3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fe800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e4004800 0x7fa8f81f3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fe800640 1 -- 172.21.15.165:0/2086496190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f81f4440 con 0x7fa8f81f30f0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fd400640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f815ff30 0x7fa8f81598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8fe800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 0x7fa8f8159de0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b6080 tx=0x7fa8f4010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.431+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400f040 con 0x7fa8f81f30f0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.432+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f400e8d0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.432+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f400ecc0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.432+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f81603f0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.432+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f81609b0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.433+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f8073a00 con 0x7fa8f81f30f0 2024-04-25T10:06:04.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.434+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8f40143f0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.435+0000 7fa8ee000640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8dc06b4d0 0x7fa8dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.435+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fa8f4089fe0 con 0x7fa8f81f30f0 2024-04-25T10:06:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.435+0000 7fa8fde00640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8dc06b4d0 0x7fa8dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.436+0000 7fa8fde00640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8dc06b4d0 0x7fa8dc06d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0000f80 tx=0x7fa8f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.440+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8f4023070 con 0x7fa8f81f30f0 2024-04-25T10:06:04.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.634+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa8f80d1590 con 0x7fa8f81f30f0 2024-04-25T10:06:04.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.635+0000 7fa8ee000640 1 -- 172.21.15.165:0/2086496190 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa8f4018030 con 0x7fa8f81f30f0 2024-04-25T10:06:04.640 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:06:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.637+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8dc06b4d0 msgr2=0x7fa8dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.637+0000 7fa900800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8dc06b4d0 0x7fa8dc06d990 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0000f80 tx=0x7fa8f0009040 comp rx=0 tx=0).stop 2024-04-25T10:06:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 msgr2=0x7fa8f8159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 0x7fa8f8159de0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fa8f80b6080 tx=0x7fa8f4010c20 comp rx=0 tx=0).stop 2024-04-25T10:06:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 shutdown_connections 2024-04-25T10:06:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8dc06b4d0 0x7fa8dc06d990 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8f81f30f0 0x7fa8f8159de0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8f815ff30 0x7fa8f81598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 --2- 172.21.15.165:0/2086496190 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8e4004800 0x7fa8f81f3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 >> 172.21.15.165:0/2086496190 conn(0x7fa8f8061a20 msgr2=0x7fa8f815b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 shutdown_connections 2024-04-25T10:06:04.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.638+0000 7fa900800640 1 -- 172.21.15.165:0/2086496190 wait complete. 2024-04-25T10:06:04.661 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:06:04.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.858+0000 7fcb4b400640 1 Processor -- start 2024-04-25T10:06:04.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.858+0000 7fcb4b400640 1 -- start start 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.859+0000 7fcb4b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb440ced80 0x7fcb440d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.859+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.859+0000 7fcb4b400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4415e000 con 0x7fcb4415dc20 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.859+0000 7fcb4b400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb440d1de0 con 0x7fcb440ced80 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.859+0000 7fcb4b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44161c60 con 0x7fcb44161710 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb48a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcb4415dc20 0x7fcb440ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42672/0 (socket says 172.21.15.165:42672) 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb48a00640 1 -- 172.21.15.165:0/2257257943 learned_addr learned my addr 172.21.15.165:0/2257257943 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:04.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb43e00640 1 --2- 172.21.15.165:0/2257257943 >> v2:172.21.15.165:3300/0 conn(0x7fcb440ced80 0x7fcb440d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb49400640 1 --2- 172.21.15.165:0/2257257943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb43e00640 1 -- 172.21.15.165:0/2257257943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 msgr2=0x7fcb440d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:04.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb43e00640 1 --2- 172.21.15.165:0/2257257943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb43e00640 1 -- 172.21.15.165:0/2257257943 >> v1:172.21.15.165:6789/0 conn(0x7fcb4415dc20 legacy=0x7fcb440ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb43400640 1 -- 172.21.15.165:0/2257257943 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 379715133 0 0) 0x7fcb4415e000 con 0x7fcb4415dc20 2024-04-25T10:06:04.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.860+0000 7fcb43e00640 1 -- 172.21.15.165:0/2257257943 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb44161f40 con 0x7fcb440ced80 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.862+0000 7fcb43e00640 1 --2- 172.21.15.165:0/2257257943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb440d22e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009d90 tx=0x7fcb2c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3acf1118f59b7bd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.862+0000 7fcb43400640 1 -- 172.21.15.165:0/2257257943 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c007740 con 0x7fcb440ced80 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.862+0000 7fcb43400640 1 -- 172.21.15.165:0/2257257943 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2c0078e0 con 0x7fcb440ced80 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.862+0000 7fcb43400640 1 -- 172.21.15.165:0/2257257943 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2c007cb0 con 0x7fcb440ced80 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.862+0000 7fcb4b400640 1 -- 172.21.15.165:0/2257257943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 msgr2=0x7fcb440d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 --2- 172.21.15.165:0/2257257943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb440d22e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c009d90 tx=0x7fcb2c02a660 comp rx=0 tx=0).stop 2024-04-25T10:06:04.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 -- 172.21.15.165:0/2257257943 shutdown_connections 2024-04-25T10:06:04.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 --2- 172.21.15.165:0/2257257943 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 --2- 172.21.15.165:0/2257257943 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb440d22e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 -- 172.21.15.165:0/2257257943 >> 172.21.15.165:0/2257257943 conn(0x7fcb440d0420 msgr2=0x7fcb440d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:04.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 -- 172.21.15.165:0/2257257943 shutdown_connections 2024-04-25T10:06:04.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.863+0000 7fcb4b400640 1 -- 172.21.15.165:0/2257257943 wait complete. 2024-04-25T10:06:04.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.864+0000 7fcb4b400640 1 Processor -- start 2024-04-25T10:06:04.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.865+0000 7fcb4b400640 1 -- start start 2024-04-25T10:06:04.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.865+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb4415b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4415dc20 0x7fcb440c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb4b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440cc390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb4b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440b5e50 con 0x7fcb440ced80 2024-04-25T10:06:04.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb4b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb44162500 con 0x7fcb4415dc20 2024-04-25T10:06:04.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb4b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb440b5fd0 con 0x7fcb44161710 2024-04-25T10:06:04.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb4415b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb4415b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49408/0 (socket says 172.21.15.165:49408) 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 -- 172.21.15.165:0/4265234116 learned_addr learned my addr 172.21.15.165:0/4265234116 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 -- 172.21.15.165:0/4265234116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 msgr2=0x7fcb440cc390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440cc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 -- 172.21.15.165:0/4265234116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4415dc20 msgr2=0x7fcb440c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4415dc20 0x7fcb440c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.866+0000 7fcb48a00640 1 -- 172.21.15.165:0/4265234116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb440c7ce0 con 0x7fcb440ced80 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.867+0000 7fcb48a00640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb4415b2b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c430 tx=0x7fcb3400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.867+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340074e0 con 0x7fcb440ced80 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.867+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb340047e0 con 0x7fcb440ced80 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.867+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb34004bd0 con 0x7fcb440ced80 2024-04-25T10:06:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.867+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb440c7fc0 con 0x7fcb440ced80 2024-04-25T10:06:04.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.867+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb440c8580 con 0x7fcb440ced80 2024-04-25T10:06:04.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.868+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb440cf940 con 0x7fcb440ced80 2024-04-25T10:06:04.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.869+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb34011070 con 0x7fcb440ced80 2024-04-25T10:06:04.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.870+0000 7fcb40c00640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb2006b4d0 0x7fcb2006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:04.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.870+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fcb3408a080 con 0x7fcb440ced80 2024-04-25T10:06:04.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.870+0000 7fcb43e00640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb2006b4d0 0x7fcb2006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:04.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.871+0000 7fcb43e00640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb2006b4d0 0x7fcb2006d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c034000 tx=0x7fcb2c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:04.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:04.875+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb440cf940 con 0x7fcb440ced80 2024-04-25T10:06:05.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.086+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcb440d1b00 con 0x7fcb440ced80 2024-04-25T10:06:05.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.087+0000 7fcb40c00640 1 -- 172.21.15.165:0/4265234116 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v43) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcb440d1b00 con 0x7fcb440ced80 2024-04-25T10:06:05.093 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb2006b4d0 msgr2=0x7fcb2006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb2006b4d0 0x7fcb2006d990 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c034000 tx=0x7fcb2c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 msgr2=0x7fcb4415b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb4415b2b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fcb3400c430 tx=0x7fcb3400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 shutdown_connections 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb2006b4d0 0x7fcb2006d990 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb44161710 0x7fcb440cc390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb4415dc20 0x7fcb440c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 --2- 172.21.15.165:0/4265234116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb440ced80 0x7fcb4415b2b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 >> 172.21.15.165:0/4265234116 conn(0x7fcb440d0420 msgr2=0x7fcb440d3080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 shutdown_connections 2024-04-25T10:06:05.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.089+0000 7fcb4b400640 1 -- 172.21.15.165:0/4265234116 wait complete. 2024-04-25T10:06:05.111 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:06:05.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:05.111 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:05.202 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:06:05.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:05.202 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:06:05.252 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:06:05.252 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:05.292 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:05.293 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:05.293 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:05.335 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:05.336 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:06:05.336 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:05.385 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:05.385 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:05.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.574+0000 7f17ed600640 1 Processor -- start 2024-04-25T10:06:05.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.574+0000 7f17ed600640 1 -- start start 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.576+0000 7f17ed600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.576+0000 7f17ed600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f17e8070ff0 0x7f17e8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.576+0000 7f17ed600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17e8071950 con 0x7f17e815f040 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.576+0000 7f17ed600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17e80aee30 con 0x7f17e8070ff0 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.576+0000 7f17ed600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e80ae590 con 0x7f17e815e520 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f17e8070ff0 0x7f17e8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f17e8070ff0 0x7f17e8071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49412/0 (socket says 172.21.15.165:49412) 2024-04-25T10:06:05.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 -- 172.21.15.165:0/1846609697 learned_addr learned my addr 172.21.15.165:0/1846609697 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17e7e00640 1 --2- 172.21.15.165:0/1846609697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 -- 172.21.15.165:0/1846609697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 msgr2=0x7f17e815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 --2- 172.21.15.165:0/1846609697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e815e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 -- 172.21.15.165:0/1846609697 >> v1:172.21.15.172:6789/0 conn(0x7f17e815f040 legacy=0x7f17e80adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.577+0000 7f17ecc00640 1 -- 172.21.15.165:0/1846609697 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e80ae870 con 0x7f17e8070ff0 2024-04-25T10:06:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.578+0000 7f17ecc00640 1 --2- 172.21.15.165:0/1846609697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e8070ff0 0x7f17e8071410 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009e50 tx=0x7f17e002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6513c33cd4a014e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.578+0000 7f17e6a00640 1 -- 172.21.15.165:0/1846609697 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0036070 con 0x7f17e8070ff0 2024-04-25T10:06:05.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.578+0000 7f17e6a00640 1 -- 172.21.15.165:0/1846609697 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e0031600 con 0x7f17e8070ff0 2024-04-25T10:06:05.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.578+0000 7f17e6a00640 1 -- 172.21.15.165:0/1846609697 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e00319d0 con 0x7f17e8070ff0 2024-04-25T10:06:05.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 -- 172.21.15.165:0/1846609697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e8070ff0 msgr2=0x7f17e8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 --2- 172.21.15.165:0/1846609697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e8070ff0 0x7f17e8071410 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009e50 tx=0x7f17e002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 -- 172.21.15.165:0/1846609697 shutdown_connections 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 --2- 172.21.15.165:0/1846609697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e8070ff0 0x7f17e8071410 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 --2- 172.21.15.165:0/1846609697 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 -- 172.21.15.165:0/1846609697 >> 172.21.15.165:0/1846609697 conn(0x7f17e806abe0 msgr2=0x7f17e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 -- 172.21.15.165:0/1846609697 shutdown_connections 2024-04-25T10:06:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.579+0000 7f17ed600640 1 -- 172.21.15.165:0/1846609697 wait complete. 2024-04-25T10:06:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.581+0000 7f17ed600640 1 Processor -- start 2024-04-25T10:06:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.581+0000 7f17ed600640 1 -- start start 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ed600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17e8070ff0 0x7f17e81f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ed600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e8158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ed600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 0x7f17e80b3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ed600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e80ae2a0 con 0x7f17e815f040 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ed600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17e806d1f0 con 0x7f17e8070ff0 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ed600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17e815f4a0 con 0x7f17e815e520 2024-04-25T10:06:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 0x7f17e80b3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 0x7f17e80b3810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49422/0 (socket says 172.21.15.165:49422) 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 -- 172.21.15.165:0/1677013862 learned_addr learned my addr 172.21.15.165:0/1677013862 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17e7400640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e8158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 -- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 msgr2=0x7f17e8158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e8158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 -- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17e8070ff0 msgr2=0x7f17e81f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17e8070ff0 0x7f17e81f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.582+0000 7f17ecc00640 1 -- 172.21.15.165:0/1677013862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17e80b1260 con 0x7f17e815f040 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.583+0000 7f17ecc00640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 0x7f17e80b3810 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009e50 tx=0x7f17e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.583+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0036050 con 0x7f17e815f040 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.583+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17e0042070 con 0x7f17e815f040 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.583+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17e0009510 con 0x7f17e815f040 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.583+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17e8159960 con 0x7f17e815f040 2024-04-25T10:06:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.583+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17e8159f20 con 0x7f17e815f040 2024-04-25T10:06:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.584+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17e8073990 con 0x7f17e815f040 2024-04-25T10:06:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.585+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17e0007bd0 con 0x7f17e815f040 2024-04-25T10:06:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.586+0000 7f17dfe00640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17c806b4d0 0x7f17c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.586+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f17e00ab420 con 0x7f17e815f040 2024-04-25T10:06:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.586+0000 7f17e7e00640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17c806b4d0 0x7f17c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.587+0000 7f17e7e00640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17c806b4d0 0x7f17c806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f17d8006fd0 tx=0x7f17d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.590+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f17e0045360 con 0x7f17e815f040 2024-04-25T10:06:05.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.821+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17e806bff0 con 0x7f17e815f040 2024-04-25T10:06:05.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.822+0000 7f17dfe00640 1 -- 172.21.15.165:0/1677013862 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f17e007a040 con 0x7f17e815f040 2024-04-25T10:06:05.830 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:05.830 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:40.296050+0000","modified":"2024-04-25T10:03:49.454381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/3282481342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3282481342},{"type":"v1","addr":"172.21.15.165:6837","nonce":3282481342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 20 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17c806b4d0 msgr2=0x7f17c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17c806b4d0 0x7f17c806d990 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f17d8006fd0 tx=0x7f17d8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 msgr2=0x7f17e80b3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 0x7f17e80b3810 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f17e0009e50 tx=0x7f17e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 shutdown_connections 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17c806b4d0 0x7f17c806d990 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17e815f040 0x7f17e80b3810 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f17e815e520 0x7f17e8158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 --2- 172.21.15.165:0/1677013862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17e8070ff0 0x7f17e81f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:05.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 >> 172.21.15.165:0/1677013862 conn(0x7f17e806abe0 msgr2=0x7f17e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:05.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 shutdown_connections 2024-04-25T10:06:05.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:05.825+0000 7f17ed600640 1 -- 172.21.15.165:0/1677013862 wait complete. 2024-04-25T10:06:05.850 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.046+0000 7fcbf6c00640 1 Processor -- start 2024-04-25T10:06:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.046+0000 7fcbf6c00640 1 -- start start 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.047+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbf00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.047+0000 7fcbf6c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcbf00cd860 0x7fcbf015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.047+0000 7fcbf6c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbf015bf30 con 0x7fcbf0144440 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.047+0000 7fcbf6c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbf015c860 con 0x7fcbf00cd860 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.047+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015c210 con 0x7fcbf0071700 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.048+0000 7fcbf5800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcbf0144440 0x7fcbf0070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42680/0 (socket says 172.21.15.165:42680) 2024-04-25T10:06:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.048+0000 7fcbf5800640 1 -- 172.21.15.165:0/2516322090 learned_addr learned my addr 172.21.15.165:0/2516322090 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.048+0000 7fcbf4e00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbf00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.048+0000 7fcbf6200640 1 --2- 172.21.15.165:0/2516322090 >> v2:172.21.15.172:3300/0 conn(0x7fcbf00cd860 0x7fcbf015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.048+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970533985 0 0) 0x7fcbf015bf30 con 0x7fcbf0144440 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.048+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbd8003520 con 0x7fcbf0144440 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347598070 0 0) 0x7fcbd8003520 con 0x7fcbf0144440 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 msgr2=0x7fcbf00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbf00cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 >> v2:172.21.15.172:3300/0 conn(0x7fcbf00cd860 msgr2=0x7fcbf015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 --2- 172.21.15.165:0/2516322090 >> v2:172.21.15.172:3300/0 conn(0x7fcbf00cd860 0x7fcbf015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf015c4f0 con 0x7fcbf0144440 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fcbe4001670 con 0x7fcbf0144440 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 >> v1:172.21.15.165:6789/0 conn(0x7fcbf0144440 legacy=0x7fcbf0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbd8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015c860 con 0x7fcbd8004d40 2024-04-25T10:06:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf015c210 con 0x7fcbd80039b0 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd800b9c0 con 0x7fcbf0071700 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.049+0000 7fcbf6200640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf6200640 1 -- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 msgr2=0x7fcbd8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf4e00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbd8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf5800640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf6200640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbd8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf6200640 1 -- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbd80039b0 msgr2=0x7fcbd8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf6200640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbd80039b0 0x7fcbd8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf6200640 1 -- 172.21.15.165:0/2516322090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd8003520 con 0x7fcbd8004d40 2024-04-25T10:06:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.050+0000 7fcbf4e00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf0071700 0x7fcbd8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:06.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.051+0000 7fcbf6200640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009fc0 tx=0x7fcbd8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1f965d57137287f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.051+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe8009c10 con 0x7fcbd8004d40 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.051+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe8036070 con 0x7fcbd8004d40 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.051+0000 7fcbefe00640 1 -- 172.21.15.165:0/2516322090 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe802d070 con 0x7fcbd8004d40 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6c00640 1 -- 172.21.15.165:0/2516322090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 msgr2=0x7fcbd80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/2516322090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbd80051c0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcbe8009fc0 tx=0x7fcbd8005830 comp rx=0 tx=0).stop 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6200640 1 -- 172.21.15.165:0/2516322090 reap_dead start 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6c00640 1 -- 172.21.15.165:0/2516322090 shutdown_connections 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6c00640 1 -- 172.21.15.165:0/2516322090 >> 172.21.15.165:0/2516322090 conn(0x7fcbf006abe0 msgr2=0x7fcbf006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6c00640 1 -- 172.21.15.165:0/2516322090 shutdown_connections 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.052+0000 7fcbf6c00640 1 -- 172.21.15.165:0/2516322090 wait complete. 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.054+0000 7fcbf6c00640 1 Processor -- start 2024-04-25T10:06:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.054+0000 7fcbf6c00640 1 -- start start 2024-04-25T10:06:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.054+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbf015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf00cba40 0x7fcbf00b3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf6c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf01f36d0 0x7fcbf00cc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cde70 con 0x7fcbd8004d40 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cdcf0 con 0x7fcbf01f36d0 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf00cd320 con 0x7fcbf00cba40 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbf015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbf015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49442/0 (socket says 172.21.15.165:49442) 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 -- 172.21.15.165:0/3152669706 learned_addr learned my addr 172.21.15.165:0/3152669706 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf4e00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf00cba40 0x7fcbf00b3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 -- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf00cba40 msgr2=0x7fcbf00b3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf00cba40 0x7fcbf00b3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf6200640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf01f36d0 0x7fcbf00cc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 -- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf01f36d0 msgr2=0x7fcbf00cc4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf01f36d0 0x7fcbf00cc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.055+0000 7fcbf5800640 1 -- 172.21.15.165:0/3152669706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00b38d0 con 0x7fcbd8004d40 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.056+0000 7fcbf5800640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbf015b010 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40018e0 tx=0x7fcbe400dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.056+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4019070 con 0x7fcbd8004d40 2024-04-25T10:06:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.056+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbe40074e0 con 0x7fcbd8004d40 2024-04-25T10:06:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.056+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf00cbee0 con 0x7fcbd8004d40 2024-04-25T10:06:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.056+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbe4004a00 con 0x7fcbd8004d40 2024-04-25T10:06:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.056+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf013efb0 con 0x7fcbd8004d40 2024-04-25T10:06:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.057+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbf013edc0 con 0x7fcbd8004d40 2024-04-25T10:06:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.058+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcbe401c070 con 0x7fcbd8004d40 2024-04-25T10:06:06.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.059+0000 7fcbed600640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbd006b4d0 0x7fcbd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.059+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fcbe4089f60 con 0x7fcbd8004d40 2024-04-25T10:06:06.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.060+0000 7fcbf4e00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbd006b4d0 0x7fcbd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.060+0000 7fcbf4e00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbd006b4d0 0x7fcbd006d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0143050 tx=0x7fcbdc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.064+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcbe4058b80 con 0x7fcbd8004d40 2024-04-25T10:06:06.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.261+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcbf0073990 con 0x7fcbd8004d40 2024-04-25T10:06:06.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.263+0000 7fcbed600640 1 -- 172.21.15.165:0/3152669706 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17036 (secure 0 0 0) 0x7fcbe4058480 con 0x7fcbd8004d40 2024-04-25T10:06:06.269 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:06.269 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":43,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:05:39.026672+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:06.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbd006b4d0 msgr2=0x7fcbd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbd006b4d0 0x7fcbd006d990 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0143050 tx=0x7fcbdc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 msgr2=0x7fcbf015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbf015b010 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fcbe40018e0 tx=0x7fcbe400dd20 comp rx=0 tx=0).stop 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 shutdown_connections 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbd006b4d0 0x7fcbd006d990 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf01f36d0 0x7fcbf00cc4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbf00cba40 0x7fcbf00b3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 --2- 172.21.15.165:0/3152669706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbd8004d40 0x7fcbf015b010 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 >> 172.21.15.165:0/3152669706 conn(0x7fcbf006abe0 msgr2=0x7fcbf01435a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 shutdown_connections 2024-04-25T10:06:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.266+0000 7fcbf6c00640 1 -- 172.21.15.165:0/3152669706 wait complete. 2024-04-25T10:06:06.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:03:38.087466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '38', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:03:39.315047+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 42, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:06.291 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:06.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.498+0000 7f300c600640 1 Processor -- start 2024-04-25T10:06:06.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.498+0000 7f300c600640 1 -- start start 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.499+0000 7f300c600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 0x7f300406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.499+0000 7f300c600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f30041452a0 0x7f300406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.499+0000 7f300c600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30040cccc0 con 0x7f30041446d0 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.499+0000 7f300c600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30040b2b20 con 0x7f30041452a0 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.499+0000 7f300c600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f300406d400 con 0x7f3004144d50 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3009c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f30041446d0 0x7f30040b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42694/0 (socket says 172.21.15.165:42694) 2024-04-25T10:06:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3009c00640 1 -- 172.21.15.165:0/1712532326 learned_addr learned my addr 172.21.15.165:0/1712532326 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f300a600640 1 --2- 172.21.15.165:0/1712532326 >> v2:172.21.15.172:3300/0 conn(0x7f30041452a0 0x7f300406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615291460 0 0) 0x7f30040cccc0 con 0x7f30041446d0 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fec003520 con 0x7f30041446d0 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3009200640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 0x7f300406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 164972706 0 0) 0x7f2fec003520 con 0x7f30041446d0 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 msgr2=0x7f300406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.500+0000 7f3003e00640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 0x7f300406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 >> v2:172.21.15.172:3300/0 conn(0x7f30041452a0 msgr2=0x7f300406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 --2- 172.21.15.165:0/1712532326 >> v2:172.21.15.172:3300/0 conn(0x7f30041452a0 0x7f300406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30040cc140 con 0x7f30041446d0 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2ff8001670 con 0x7f30041446d0 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 >> v1:172.21.15.165:6789/0 conn(0x7f30041446d0 legacy=0x7f30040b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 0x7f2fec0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fec003f30 0x7f2fec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fec004d40 0x7f2fec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040b2b20 con 0x7f2fec004d40 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f300406d400 con 0x7f2fec003f30 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fec00ba10 con 0x7f3004144d50 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f300a600640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fec004d40 0x7f2fec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3009c00640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 0x7f2fec0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f3009200640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fec003f30 0x7f2fec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.501+0000 7f300a600640 1 -- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 msgr2=0x7f2fec0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.502+0000 7f300a600640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3004144d50 0x7f2fec0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.502+0000 7f300a600640 1 -- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fec003f30 msgr2=0x7f2fec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.502+0000 7f300a600640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fec003f30 0x7f2fec004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.502+0000 7f300a600640 1 -- 172.21.15.165:0/1712532326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec003520 con 0x7f2fec004d40 2024-04-25T10:06:06.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.503+0000 7f300a600640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fec004d40 0x7f2fec0051c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fc0 tx=0x7f2fec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fd7fa9eb9e4d07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.503+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc009c10 con 0x7f2fec004d40 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.503+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc036070 con 0x7f2fec004d40 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.503+0000 7f3003e00640 1 -- 172.21.15.165:0/1712532326 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc02d070 con 0x7f2fec004d40 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300c600640 1 -- 172.21.15.165:0/1712532326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fec004d40 msgr2=0x7f2fec0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300c600640 1 --2- 172.21.15.165:0/1712532326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fec004d40 0x7f2fec0051c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc009fc0 tx=0x7f2fec005830 comp rx=0 tx=0).stop 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300a600640 1 -- 172.21.15.165:0/1712532326 reap_dead start 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300c600640 1 -- 172.21.15.165:0/1712532326 shutdown_connections 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300c600640 1 -- 172.21.15.165:0/1712532326 >> 172.21.15.165:0/1712532326 conn(0x7f300406abe0 msgr2=0x7f300406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300c600640 1 -- 172.21.15.165:0/1712532326 shutdown_connections 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.504+0000 7f300c600640 1 -- 172.21.15.165:0/1712532326 wait complete. 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.506+0000 7f300c600640 1 Processor -- start 2024-04-25T10:06:06.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.506+0000 7f300c600640 1 -- start start 2024-04-25T10:06:06.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300c600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30040b0f90 0x7f300413e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300c600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f300413e880 0x7f300413cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300c600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 0x7f300413d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300c600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040cc9e0 con 0x7f30040b1370 2024-04-25T10:06:06.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300c600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30040cc700 con 0x7f30040b0f90 2024-04-25T10:06:06.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300c600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3004144ab0 con 0x7f300413e880 2024-04-25T10:06:06.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 0x7f300413d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 0x7f300413d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49458/0 (socket says 172.21.15.165:49458) 2024-04-25T10:06:06.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.507+0000 7f300a600640 1 -- 172.21.15.165:0/908402228 learned_addr learned my addr 172.21.15.165:0/908402228 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f3009c00640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30040b0f90 0x7f300413e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f3009200640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f300413e880 0x7f300413cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300a600640 1 -- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f300413e880 msgr2=0x7f300413cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300a600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f300413e880 0x7f300413cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300a600640 1 -- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30040b0f90 msgr2=0x7f300413e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300a600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30040b0f90 0x7f300413e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300a600640 1 -- 172.21.15.165:0/908402228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f300414d2a0 con 0x7f30040b1370 2024-04-25T10:06:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300a600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 0x7f300413d240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc033040 tx=0x7f2ffc009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc009c10 con 0x7f30040b1370 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ffc003990 con 0x7f30040b1370 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ffc003d60 con 0x7f30040b1370 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.508+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f300413ed90 con 0x7f30040b1370 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.509+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f300414a2e0 con 0x7f30040b1370 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.510+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3004073a00 con 0x7f30040b1370 2024-04-25T10:06:06.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.511+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ffc005d10 con 0x7f30040b1370 2024-04-25T10:06:06.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.512+0000 7f3001600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2fe406b500 0x7f2fe406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.512+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f2ffc0ab830 con 0x7f30040b1370 2024-04-25T10:06:06.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.512+0000 7f3009c00640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2fe406b500 0x7f2fe406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.513+0000 7f3009c00640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2fe406b500 0x7f2fe406d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8001eb0 tx=0x7f2ff8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.516+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ffc04a060 con 0x7f30040b1370 2024-04-25T10:06:06.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.709+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f30040cccc0 con 0x7f30040b1370 2024-04-25T10:06:06.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.710+0000 7f3001600640 1 -- 172.21.15.165:0/908402228 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f300414d2a0 con 0x7f30040b1370 2024-04-25T10:06:06.716 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:06:06.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.713+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2fe406b500 msgr2=0x7f2fe406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.713+0000 7f300c600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2fe406b500 0x7f2fe406d9c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8001eb0 tx=0x7f2ff8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:06.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.713+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 msgr2=0x7f300413d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.713+0000 7f300c600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 0x7f300413d240 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f2ffc033040 tx=0x7f2ffc009750 comp rx=0 tx=0).stop 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 shutdown_connections 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2fe406b500 0x7f2fe406d9c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30040b1370 0x7f300413d240 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f300413e880 0x7f300413cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 --2- 172.21.15.165:0/908402228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30040b0f90 0x7f300413e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 >> 172.21.15.165:0/908402228 conn(0x7f300406abe0 msgr2=0x7f300406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 shutdown_connections 2024-04-25T10:06:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.714+0000 7f300c600640 1 -- 172.21.15.165:0/908402228 wait complete. 2024-04-25T10:06:06.738 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:06:06.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.938+0000 7f7f2d600640 1 Processor -- start 2024-04-25T10:06:06.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.938+0000 7f7f2d600640 1 -- start start 2024-04-25T10:06:06.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.940+0000 7f7f2d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7f28145140 0x7f7f280ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.940+0000 7f7f2d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280c5200 0x7f7f2815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.940+0000 7f7f2d600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f280c5650 con 0x7f7f280ce950 2024-04-25T10:06:06.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.940+0000 7f7f2d600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f280ced90 con 0x7f7f28145140 2024-04-25T10:06:06.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.940+0000 7f7f2d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28145520 con 0x7f7f280c5200 2024-04-25T10:06:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.941+0000 7f7f27e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7f28145140 0x7f7f280ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.941+0000 7f7f2cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280c5200 0x7f7f2815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.941+0000 7f7f27e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7f28145140 0x7f7f280ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:36080/0 (socket says 172.21.15.165:36080) 2024-04-25T10:06:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.941+0000 7f7f27e00640 1 -- 172.21.15.165:0/3660569637 learned_addr learned my addr 172.21.15.165:0/3660569637 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.941+0000 7f7f27e00640 1 -- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280c5200 msgr2=0x7f7f2815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.942+0000 7f7f27e00640 1 --2- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280c5200 0x7f7f2815bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.942+0000 7f7f27e00640 1 -- 172.21.15.165:0/3660569637 >> v1:172.21.15.172:6789/0 conn(0x7f7f280ce950 legacy=0x7f7f280c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.942+0000 7f7f27e00640 1 -- 172.21.15.165:0/3660569637 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f28071270 con 0x7f7f28145140 2024-04-25T10:06:06.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.943+0000 7f7f27e00640 1 --2- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f280ce410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009870 tx=0x7f7f1802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd53be411494596c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.943+0000 7f7f26a00640 1 -- 172.21.15.165:0/3660569637 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18037070 con 0x7f7f28145140 2024-04-25T10:06:06.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.943+0000 7f7f26a00640 1 -- 172.21.15.165:0/3660569637 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18032440 con 0x7f7f28145140 2024-04-25T10:06:06.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.943+0000 7f7f26a00640 1 -- 172.21.15.165:0/3660569637 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18032810 con 0x7f7f28145140 2024-04-25T10:06:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 -- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 msgr2=0x7f7f280ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 --2- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f280ce410 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f7f18009870 tx=0x7f7f1802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 -- 172.21.15.165:0/3660569637 shutdown_connections 2024-04-25T10:06:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 --2- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280c5200 0x7f7f2815bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 --2- 172.21.15.165:0/3660569637 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f280ce410 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 -- 172.21.15.165:0/3660569637 >> 172.21.15.165:0/3660569637 conn(0x7f7f28061cc0 msgr2=0x7f7f2806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:06.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 -- 172.21.15.165:0/3660569637 shutdown_connections 2024-04-25T10:06:06.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.944+0000 7f7f2d600640 1 -- 172.21.15.165:0/3660569637 wait complete. 2024-04-25T10:06:06.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.946+0000 7f7f2d600640 1 Processor -- start 2024-04-25T10:06:06.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.947+0000 7f7f2d600640 1 -- start start 2024-04-25T10:06:06.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f2d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 0x7f7f281f7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f2d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280ce950 0x7f7f2815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f2d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f2815d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f2d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28071b10 con 0x7f7f280c5200 2024-04-25T10:06:06.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f2d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28071830 con 0x7f7f28145140 2024-04-25T10:06:06.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f2d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815c4f0 con 0x7f7f280ce950 2024-04-25T10:06:06.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 0x7f7f281f7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f27e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 0x7f7f281f7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41388/0 (socket says 172.21.15.165:41388) 2024-04-25T10:06:06.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f27e00640 1 -- 172.21.15.165:0/1965971806 learned_addr learned my addr 172.21.15.165:0/1965971806 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:06.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.948+0000 7f7f27e00640 1 -- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280ce950 msgr2=0x7f7f2815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27400640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280ce950 0x7f7f2815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f2cc00640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f2815d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27e00640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280ce950 0x7f7f2815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27e00640 1 -- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 msgr2=0x7f7f2815d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27e00640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f2815d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27e00640 1 -- 172.21.15.165:0/1965971806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f281f79c0 con 0x7f7f280c5200 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27400640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280ce950 0x7f7f2815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f27e00640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 0x7f7f281f7480 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000c00 tx=0x7f7f180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18037040 con 0x7f7f280c5200 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f18042070 con 0x7f7f280c5200 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18030880 con 0x7f7f280c5200 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.949+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f281e42a0 con 0x7f7f280c5200 2024-04-25T10:06:06.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.950+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f281e4860 con 0x7f7f280c5200 2024-04-25T10:06:06.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.951+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f280b09e0 con 0x7f7f280c5200 2024-04-25T10:06:06.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.952+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f1802ad10 con 0x7f7f280c5200 2024-04-25T10:06:06.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.953+0000 7f7f1fe00640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f0806b4d0 0x7f7f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:06.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.953+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f7f180aa970 con 0x7f7f280c5200 2024-04-25T10:06:06.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.953+0000 7f7f27400640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f0806b4d0 0x7f7f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.954+0000 7f7f27400640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f0806b4d0 0x7f7f0806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f281f69d0 tx=0x7f7f14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:06.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:06.957+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f280b09e0 con 0x7f7f280c5200 2024-04-25T10:06:07.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.152+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7f28073990 con 0x7f7f280c5200 2024-04-25T10:06:07.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.153+0000 7f7f1fe00640 1 -- 172.21.15.165:0/1965971806 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7f281f79c0 con 0x7f7f280c5200 2024-04-25T10:06:07.159 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:06:07.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f0806b4d0 msgr2=0x7f7f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f0806b4d0 0x7f7f0806d990 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7f281f69d0 tx=0x7f7f14009040 comp rx=0 tx=0).stop 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 msgr2=0x7f7f281f7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 0x7f7f281f7480 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f7f18000c00 tx=0x7f7f180046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 shutdown_connections 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f0806b4d0 0x7f7f0806d990 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f28145140 0x7f7f2815d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f280ce950 0x7f7f2815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 --2- 172.21.15.165:0/1965971806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f280c5200 0x7f7f281f7480 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 >> 172.21.15.165:0/1965971806 conn(0x7f7f28061cc0 msgr2=0x7f7f280c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.156+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 shutdown_connections 2024-04-25T10:06:07.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.157+0000 7f7f2d600640 1 -- 172.21.15.165:0/1965971806 wait complete. 2024-04-25T10:06:07.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:06:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.370+0000 7f6b3cc00640 1 Processor -- start 2024-04-25T10:06:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.371+0000 7f6b3cc00640 1 -- start start 2024-04-25T10:06:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.372+0000 7f6b3cc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6b380cadb0 0x7f6b380cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.372+0000 7f6b3cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380ccfe0 0x7f6b380cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.372+0000 7f6b3cc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b380afc80 con 0x7f6b3815de80 2024-04-25T10:06:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.372+0000 7f6b3cc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b380cb240 con 0x7f6b380cadb0 2024-04-25T10:06:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.372+0000 7f6b3cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380ce4d0 con 0x7f6b380ccfe0 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b35e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6b3815de80 0x7f6b380cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51372/0 (socket says 172.21.15.165:51372) 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b35e00640 1 -- 172.21.15.165:0/1222214904 learned_addr learned my addr 172.21.15.165:0/1222214904 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b2fe00640 1 --2- 172.21.15.165:0/1222214904 >> v2:172.21.15.165:3300/0 conn(0x7f6b380cadb0 0x7f6b380cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b36800640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380ccfe0 0x7f6b380cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b35400640 1 -- 172.21.15.165:0/1222214904 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303204366 0 0) 0x7f6b380afc80 con 0x7f6b3815de80 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1222214904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380ccfe0 msgr2=0x7f6b380cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b2fe00640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380ccfe0 0x7f6b380cd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1222214904 >> v1:172.21.15.165:6789/0 conn(0x7f6b3815de80 legacy=0x7f6b380cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1222214904 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380ce7b0 con 0x7f6b380cadb0 2024-04-25T10:06:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.373+0000 7f6b36800640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380ccfe0 0x7f6b380cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.374+0000 7f6b2fe00640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380cadb0 0x7f6b380cf6c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b200097a0 tx=0x7f6b2002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0625e0dd2be72c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.374+0000 7f6b35400640 1 -- 172.21.15.165:0/1222214904 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20037070 con 0x7f6b380cadb0 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b35400640 1 -- 172.21.15.165:0/1222214904 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20032440 con 0x7f6b380cadb0 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b35400640 1 -- 172.21.15.165:0/1222214904 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20032810 con 0x7f6b380cadb0 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1222214904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380cadb0 msgr2=0x7f6b380cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380cadb0 0x7f6b380cf6c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6b200097a0 tx=0x7f6b2002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1222214904 shutdown_connections 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380ccfe0 0x7f6b380cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1222214904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380cadb0 0x7f6b380cf6c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1222214904 >> 172.21.15.165:0/1222214904 conn(0x7f6b380b2390 msgr2=0x7f6b380b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1222214904 shutdown_connections 2024-04-25T10:06:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.375+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1222214904 wait complete. 2024-04-25T10:06:07.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.377+0000 7f6b3cc00640 1 Processor -- start 2024-04-25T10:06:07.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.377+0000 7f6b3cc00640 1 -- start start 2024-04-25T10:06:07.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b3cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380cadb0 0x7f6b3814d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b3cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 0x7f6b380b5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b3cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b3815de80 0x7f6b380b5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b3cc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b3815e260 con 0x7f6b380ccfe0 2024-04-25T10:06:07.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b3cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380cfc00 con 0x7f6b3815de80 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b3cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b380b2eb0 con 0x7f6b380cadb0 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 0x7f6b380b5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.378+0000 7f6b2fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 0x7f6b380b5210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41402/0 (socket says 172.21.15.165:41402) 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1182114580 learned_addr learned my addr 172.21.15.165:0/1182114580 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b35e00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380cadb0 0x7f6b3814d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b36800640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b3815de80 0x7f6b380b5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380cadb0 msgr2=0x7f6b3814d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380cadb0 0x7f6b3814d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b3815de80 msgr2=0x7f6b380b5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b3815de80 0x7f6b380b5750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 -- 172.21.15.165:0/1182114580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b380c7d90 con 0x7f6b380ccfe0 2024-04-25T10:06:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.379+0000 7f6b2fe00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 0x7f6b380b5210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b200097a0 tx=0x7f6b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.380+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20037050 con 0x7f6b380ccfe0 2024-04-25T10:06:07.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.380+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b20038040 con 0x7f6b380ccfe0 2024-04-25T10:06:07.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.380+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b20004490 con 0x7f6b380ccfe0 2024-04-25T10:06:07.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.380+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b380c8070 con 0x7f6b380ccfe0 2024-04-25T10:06:07.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.380+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3815a240 con 0x7f6b380ccfe0 2024-04-25T10:06:07.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.381+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b380c5600 con 0x7f6b380ccfe0 2024-04-25T10:06:07.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.382+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b20030980 con 0x7f6b380ccfe0 2024-04-25T10:06:07.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.383+0000 7f6b2e000640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b1406b4d0 0x7f6b1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.383+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f6b200ab180 con 0x7f6b380ccfe0 2024-04-25T10:06:07.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.383+0000 7f6b35e00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b1406b4d0 0x7f6b1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.384+0000 7f6b35e00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b1406b4d0 0x7f6b1406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000f60 tx=0x7f6b28007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.387+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b380c5600 con 0x7f6b380ccfe0 2024-04-25T10:06:07.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.576+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6b38067be0 con 0x7f6b380ccfe0 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.577+0000 7f6b2e000640 1 -- 172.21.15.165:0/1182114580 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6b38067be0 con 0x7f6b380ccfe0 2024-04-25T10:06:07.583 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b1406b4d0 msgr2=0x7f6b1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b1406b4d0 0x7f6b1406d990 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b28000f60 tx=0x7f6b28007c00 comp rx=0 tx=0).stop 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 msgr2=0x7f6b380b5210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 0x7f6b380b5210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6b200097a0 tx=0x7f6b200046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 shutdown_connections 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b1406b4d0 0x7f6b1406d990 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b3815de80 0x7f6b380b5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b380ccfe0 0x7f6b380b5210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 --2- 172.21.15.165:0/1182114580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b380cadb0 0x7f6b3814d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.580+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 >> 172.21.15.165:0/1182114580 conn(0x7f6b380b2390 msgr2=0x7f6b380cee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.581+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 shutdown_connections 2024-04-25T10:06:07.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.581+0000 7f6b3cc00640 1 -- 172.21.15.165:0/1182114580 wait complete. 2024-04-25T10:06:07.604 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.793+0000 7f90ff000640 1 Processor -- start 2024-04-25T10:06:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.793+0000 7f90ff000640 1 -- start start 2024-04-25T10:06:07.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.794+0000 7f90ff000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f90f8144440 0x7f90f8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.794+0000 7f90ff000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90f80cda30 0x7f90f815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90ff000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f815bf30 con 0x7f90f8071530 2024-04-25T10:06:07.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90ff000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f80719a0 con 0x7f90f80cda30 2024-04-25T10:06:07.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90ff000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f815c860 con 0x7f90f8144440 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fd200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f90f8071530 0x7f90f80cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51374/0 (socket says 172.21.15.165:51374) 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fd200640 1 -- 172.21.15.165:0/2151067356 learned_addr learned my addr 172.21.15.165:0/2151067356 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fe600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90f80cda30 0x7f90f815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fdc00640 1 --2- 172.21.15.165:0/2151067356 >> v2:172.21.15.172:3300/0 conn(0x7f90f8144440 0x7f90f8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fe600640 1 -- 172.21.15.165:0/2151067356 >> v2:172.21.15.172:3300/0 conn(0x7f90f8144440 msgr2=0x7f90f8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fe600640 1 --2- 172.21.15.165:0/2151067356 >> v2:172.21.15.172:3300/0 conn(0x7f90f8144440 0x7f90f8070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fe600640 1 -- 172.21.15.165:0/2151067356 >> v1:172.21.15.165:6789/0 conn(0x7f90f8071530 legacy=0x7f90f80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90fe600640 1 -- 172.21.15.165:0/2151067356 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f815c210 con 0x7f90f80cda30 2024-04-25T10:06:07.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.795+0000 7f90f7e00640 1 -- 172.21.15.165:0/2151067356 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475487335 0 0) 0x7f90f815bf30 con 0x7f90f8071530 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.797+0000 7f90fe600640 1 --2- 172.21.15.165:0/2151067356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f80cda30 0x7f90f815ccf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f90f806f470 tx=0x7f90f002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8e4294d731de0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.797+0000 7f90f7e00640 1 -- 172.21.15.165:0/2151067356 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0007740 con 0x7f90f80cda30 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.797+0000 7f90f7e00640 1 -- 172.21.15.165:0/2151067356 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f00078e0 con 0x7f90f80cda30 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.797+0000 7f90f7e00640 1 -- 172.21.15.165:0/2151067356 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0007cb0 con 0x7f90f80cda30 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.797+0000 7f90ff000640 1 -- 172.21.15.165:0/2151067356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f80cda30 msgr2=0x7f90f815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.797+0000 7f90ff000640 1 --2- 172.21.15.165:0/2151067356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f80cda30 0x7f90f815ccf0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f90f806f470 tx=0x7f90f002a710 comp rx=0 tx=0).stop 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.798+0000 7f90ff000640 1 -- 172.21.15.165:0/2151067356 shutdown_connections 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.798+0000 7f90ff000640 1 --2- 172.21.15.165:0/2151067356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f80cda30 0x7f90f815ccf0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.798+0000 7f90ff000640 1 --2- 172.21.15.165:0/2151067356 >> v2:172.21.15.172:3300/0 conn(0x7f90f8144440 0x7f90f8070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.798+0000 7f90ff000640 1 -- 172.21.15.165:0/2151067356 >> 172.21.15.165:0/2151067356 conn(0x7f90f806abe0 msgr2=0x7f90f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:07.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.798+0000 7f90ff000640 1 -- 172.21.15.165:0/2151067356 shutdown_connections 2024-04-25T10:06:07.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.798+0000 7f90ff000640 1 -- 172.21.15.165:0/2151067356 wait complete. 2024-04-25T10:06:07.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.799+0000 7f90ff000640 1 Processor -- start 2024-04-25T10:06:07.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.800+0000 7f90ff000640 1 -- start start 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.800+0000 7f90ff000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f8071530 0x7f90f815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90ff000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f80cda30 0x7f90f8137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90ff000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 0x7f90f81390e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90ff000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f815bc30 con 0x7f90f8144440 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90ff000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f80cdea0 con 0x7f90f8071530 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90ff000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f815d230 con 0x7f90f80cda30 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 0x7f90f81390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 0x7f90f81390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41430/0 (socket says 172.21.15.165:41430) 2024-04-25T10:06:07.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 -- 172.21.15.165:0/2228110971 learned_addr learned my addr 172.21.15.165:0/2228110971 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fdc00640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f8071530 0x7f90f815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fd200640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f80cda30 0x7f90f8137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 -- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f80cda30 msgr2=0x7f90f8137b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f80cda30 0x7f90f8137b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 -- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f8071530 msgr2=0x7f90f815ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f8071530 0x7f90f815ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fe600640 1 -- 172.21.15.165:0/2228110971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f813bef0 con 0x7f90f8144440 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.801+0000 7f90fdc00640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f8071530 0x7f90f815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.802+0000 7f90fe600640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 0x7f90f81390e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f90f002ad00 tx=0x7f90f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.802+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0040070 con 0x7f90f8144440 2024-04-25T10:06:07.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.802+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f0031bc0 con 0x7f90f8144440 2024-04-25T10:06:07.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.802+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f0045460 con 0x7f90f8144440 2024-04-25T10:06:07.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.802+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f813c1d0 con 0x7f90f8144440 2024-04-25T10:06:07.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.802+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f80c9dc0 con 0x7f90f8144440 2024-04-25T10:06:07.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.803+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f813c3a0 con 0x7f90f8144440 2024-04-25T10:06:07.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.804+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90f0039040 con 0x7f90f8144440 2024-04-25T10:06:07.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.806+0000 7f90f5600640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d806b4d0 0x7f90d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:07.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.806+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f90f00ab890 con 0x7f90f8144440 2024-04-25T10:06:07.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.806+0000 7f90fdc00640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d806b4d0 0x7f90d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.807+0000 7f90fdc00640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d806b4d0 0x7f90d806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90ec006fd0 tx=0x7f90ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:07.810+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90f004c070 con 0x7f90f8144440 2024-04-25T10:06:08.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.041+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90f80c9fc0 con 0x7f90f8144440 2024-04-25T10:06:08.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.043+0000 7f90f5600640 1 -- 172.21.15.165:0/2228110971 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f90f007a5d0 con 0x7f90f8144440 2024-04-25T10:06:08.048 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:08.048 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:40.296050+0000","modified":"2024-04-25T10:03:49.454381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/3282481342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3282481342},{"type":"v1","addr":"172.21.15.165:6837","nonce":3282481342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:08.048 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 20 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d806b4d0 msgr2=0x7f90d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d806b4d0 0x7f90d806d990 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f90ec006fd0 tx=0x7f90ec008040 comp rx=0 tx=0).stop 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 msgr2=0x7f90f81390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 0x7f90f81390e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f90f002ad00 tx=0x7f90f0004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 shutdown_connections 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d806b4d0 0x7f90d806d990 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f8144440 0x7f90f81390e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f80cda30 0x7f90f8137b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 --2- 172.21.15.165:0/2228110971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f8071530 0x7f90f815ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 >> 172.21.15.165:0/2228110971 conn(0x7f90f806abe0 msgr2=0x7f90f8140fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 shutdown_connections 2024-04-25T10:06:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.045+0000 7f90ff000640 1 -- 172.21.15.165:0/2228110971 wait complete. 2024-04-25T10:06:08.070 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:08.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.268+0000 7f60c6e00640 1 Processor -- start 2024-04-25T10:06:08.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.268+0000 7f60c6e00640 1 -- start start 2024-04-25T10:06:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.269+0000 7f60c6e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f60c00b93c0 0x7f60c00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.269+0000 7f60c6e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c00d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.269+0000 7f60c6e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c0161df0 con 0x7f60c015db90 2024-04-25T10:06:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.269+0000 7f60c6e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c00d0980 con 0x7f60c00b93c0 2024-04-25T10:06:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.270+0000 7f60c6e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00d1f50 con 0x7f60c00d04f0 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.270+0000 7f60c5000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f60c00b93c0 0x7f60c00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.270+0000 7f60c5000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f60c00b93c0 0x7f60c00cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41432/0 (socket says 172.21.15.165:41432) 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.270+0000 7f60c5000640 1 -- 172.21.15.165:0/1191044822 learned_addr learned my addr 172.21.15.165:0/1191044822 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.270+0000 7f60c6400640 1 --2- 172.21.15.165:0/1191044822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c00d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.271+0000 7f60c5000640 1 -- 172.21.15.165:0/1191044822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 msgr2=0x7f60c00d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.271+0000 7f60c5000640 1 --2- 172.21.15.165:0/1191044822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c00d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.271+0000 7f60c5000640 1 -- 172.21.15.165:0/1191044822 >> v1:172.21.15.172:6789/0 conn(0x7f60c015db90 legacy=0x7f60c01615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.271+0000 7f60c5000640 1 -- 172.21.15.165:0/1191044822 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c01620d0 con 0x7f60c00b93c0 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.272+0000 7f60c5000640 1 --2- 172.21.15.165:0/1191044822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c00b93c0 0x7f60c00cffb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009870 tx=0x7f60ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=445173206d75f3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.272+0000 7f60bfe00640 1 -- 172.21.15.165:0/1191044822 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac037070 con 0x7f60c00b93c0 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.272+0000 7f60bfe00640 1 -- 172.21.15.165:0/1191044822 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac032440 con 0x7f60c00b93c0 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.272+0000 7f60bfe00640 1 -- 172.21.15.165:0/1191044822 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac032810 con 0x7f60c00b93c0 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.272+0000 7f60c6e00640 1 -- 172.21.15.165:0/1191044822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c00b93c0 msgr2=0x7f60c00cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.272+0000 7f60c6e00640 1 --2- 172.21.15.165:0/1191044822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c00b93c0 0x7f60c00cffb0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f60ac009870 tx=0x7f60ac02a210 comp rx=0 tx=0).stop 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.273+0000 7f60c6e00640 1 -- 172.21.15.165:0/1191044822 shutdown_connections 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.273+0000 7f60c6e00640 1 --2- 172.21.15.165:0/1191044822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c00d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.273+0000 7f60c6e00640 1 --2- 172.21.15.165:0/1191044822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c00b93c0 0x7f60c00cffb0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.273+0000 7f60c6e00640 1 -- 172.21.15.165:0/1191044822 >> 172.21.15.165:0/1191044822 conn(0x7f60c00b52b0 msgr2=0x7f60c00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.273+0000 7f60c6e00640 1 -- 172.21.15.165:0/1191044822 shutdown_connections 2024-04-25T10:06:08.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.273+0000 7f60c6e00640 1 -- 172.21.15.165:0/1191044822 wait complete. 2024-04-25T10:06:08.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.274+0000 7f60c6e00640 1 Processor -- start 2024-04-25T10:06:08.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.274+0000 7f60c6e00640 1 -- start start 2024-04-25T10:06:08.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.275+0000 7f60c6e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c00b93c0 0x7f60c00c92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.275+0000 7f60c6e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.275+0000 7f60c6e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 0x7f60c01500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.275+0000 7f60c6e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00b68d0 con 0x7f60c015db90 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.275+0000 7f60c6e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00d1be0 con 0x7f60c00b93c0 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.275+0000 7f60c6e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c00b6a50 con 0x7f60c00d04f0 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 0x7f60c01500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 0x7f60c01500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41448/0 (socket says 172.21.15.165:41448) 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 -- 172.21.15.165:0/2409131767 learned_addr learned my addr 172.21.15.165:0/2409131767 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c5a00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c00b93c0 0x7f60c00c92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 -- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 msgr2=0x7f60c014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c5000640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c014fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 -- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c00b93c0 msgr2=0x7f60c00c92d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c00b93c0 0x7f60c00c92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 -- 172.21.15.165:0/2409131767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c01f3900 con 0x7f60c015db90 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.276+0000 7f60c6400640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 0x7f60c01500b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f60b8007dc0 tx=0x7f60b800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.277+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8014420 con 0x7f60c015db90 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.277+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b8019070 con 0x7f60c015db90 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.277+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b8014820 con 0x7f60c015db90 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.277+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c01f3bc0 con 0x7f60c015db90 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.277+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c01f4180 con 0x7f60c015db90 2024-04-25T10:06:08.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.278+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c0073990 con 0x7f60c015db90 2024-04-25T10:06:08.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.279+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60b80149c0 con 0x7f60c015db90 2024-04-25T10:06:08.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.280+0000 7f60bd600640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60a006b4d0 0x7f60a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.280+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f60b808ab90 con 0x7f60c015db90 2024-04-25T10:06:08.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.280+0000 7f60c5a00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60a006b4d0 0x7f60a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.281+0000 7f60c5a00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60a006b4d0 0x7f60a006d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f60b400a750 tx=0x7f60b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.285+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60b80598e0 con 0x7f60c015db90 2024-04-25T10:06:08.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.485+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f60c00d14b0 con 0x7f60c015db90 2024-04-25T10:06:08.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.488+0000 7f60bd600640 1 -- 172.21.15.165:0/2409131767 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17036 (secure 0 0 0) 0x7f60b80591e0 con 0x7f60c015db90 2024-04-25T10:06:08.494 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:08.494 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":43,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:05:39.026672+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60a006b4d0 msgr2=0x7f60a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60a006b4d0 0x7f60a006d990 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f60b400a750 tx=0x7f60b4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 msgr2=0x7f60c01500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 0x7f60c01500b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f60b8007dc0 tx=0x7f60b800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 shutdown_connections 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60a006b4d0 0x7f60a006d990 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60c015db90 0x7f60c01500b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60c00d04f0 0x7f60c014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 --2- 172.21.15.165:0/2409131767 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60c00b93c0 0x7f60c00c92d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.491+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 >> 172.21.15.165:0/2409131767 conn(0x7f60c00b52b0 msgr2=0x7f60c00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.492+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 shutdown_connections 2024-04-25T10:06:08.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.492+0000 7f60c6e00640 1 -- 172.21.15.165:0/2409131767 wait complete. 2024-04-25T10:06:08.516 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:06:08.516 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:08.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.697+0000 7fd8ff200640 1 Processor -- start 2024-04-25T10:06:08.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.697+0000 7fd8ff200640 1 -- start start 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.698+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd900071700 0x7fd9000cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.698+0000 7fd8ff200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd9000cd860 0x7fd90015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8ff200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90015bf30 con 0x7fd900144440 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8ff200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90015c860 con 0x7fd9000cd860 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8ff200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90015c210 con 0x7fd900071700 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8fde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd900144440 0x7fd900070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51382/0 (socket says 172.21.15.165:51382) 2024-04-25T10:06:08.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8fde00640 1 -- 172.21.15.165:0/134459018 learned_addr learned my addr 172.21.15.165:0/134459018 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:08.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8fd400640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd900071700 0x7fd9000cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8fe800640 1 --2- 172.21.15.165:0/134459018 >> v2:172.21.15.172:3300/0 conn(0x7fd9000cd860 0x7fd90015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570778725 0 0) 0x7fd90015bf30 con 0x7fd900144440 2024-04-25T10:06:08.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.699+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8e4003520 con 0x7fd900144440 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2141035232 0 0) 0x7fd8e4003520 con 0x7fd900144440 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd900071700 msgr2=0x7fd9000cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd900071700 0x7fd9000cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 >> v2:172.21.15.172:3300/0 conn(0x7fd9000cd860 msgr2=0x7fd90015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 --2- 172.21.15.165:0/134459018 >> v2:172.21.15.172:3300/0 conn(0x7fd9000cd860 0x7fd90015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90015c4f0 con 0x7fd900144440 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd8f0001670 con 0x7fd900144440 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 >> v1:172.21.15.165:6789/0 conn(0x7fd900144440 legacy=0x7fd900070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8e40039b0 0x7fd8e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e4004800 0x7fd8e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e40051c0 0x7fd8e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90015c860 con 0x7fd8e40039b0 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90015c210 con 0x7fd8e40051c0 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8e400bf30 con 0x7fd8e4004800 2024-04-25T10:06:08.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fd400640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd900071700 0x7fd9000cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.700+0000 7fd8fde00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8e40039b0 0x7fd8e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.701+0000 7fd8fde00640 1 -- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e4004800 msgr2=0x7fd8e4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.701+0000 7fd8fde00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e4004800 0x7fd8e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.701+0000 7fd8fde00640 1 -- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e40051c0 msgr2=0x7fd8e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.701+0000 7fd8fde00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8e40051c0 0x7fd8e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.701+0000 7fd8fde00640 1 -- 172.21.15.165:0/134459018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8e4003520 con 0x7fd8e40039b0 2024-04-25T10:06:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8fde00640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8e40039b0 0x7fd8e4003dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c290 tx=0x7fd8e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65637c143741ca06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0007d50 con 0x7fd8e40039b0 2024-04-25T10:06:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8f0005af0 con 0x7fd8e40039b0 2024-04-25T10:06:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8fca00640 1 -- 172.21.15.165:0/134459018 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8f0007070 con 0x7fd8e40039b0 2024-04-25T10:06:08.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8ff200640 1 -- 172.21.15.165:0/134459018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8e40039b0 msgr2=0x7fd8e4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8ff200640 1 --2- 172.21.15.165:0/134459018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8e40039b0 0x7fd8e4003dd0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fd8f000c290 tx=0x7fd8e4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.702+0000 7fd8fe800640 1 -- 172.21.15.165:0/134459018 reap_dead start 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.703+0000 7fd8ff200640 1 -- 172.21.15.165:0/134459018 shutdown_connections 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.703+0000 7fd8ff200640 1 -- 172.21.15.165:0/134459018 >> 172.21.15.165:0/134459018 conn(0x7fd90006abe0 msgr2=0x7fd90006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.703+0000 7fd8ff200640 1 -- 172.21.15.165:0/134459018 shutdown_connections 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.703+0000 7fd8ff200640 1 -- 172.21.15.165:0/134459018 wait complete. 2024-04-25T10:06:08.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.704+0000 7fd8ff200640 1 Processor -- start 2024-04-25T10:06:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.705+0000 7fd8ff200640 1 -- start start 2024-04-25T10:06:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.705+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e40039b0 0x7fd90015af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.705+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 0x7fd9001f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8ff200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9001f51a0 0x7fd9001e3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8ff200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000cde70 con 0x7fd9000c9170 2024-04-25T10:06:08.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8ff200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000cdcf0 con 0x7fd9001f51a0 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8ff200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9000cd320 con 0x7fd8e40039b0 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 0x7fd9001f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 0x7fd9001f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41466/0 (socket says 172.21.15.165:41466) 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 -- 172.21.15.165:0/1653794347 learned_addr learned my addr 172.21.15.165:0/1653794347 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 -- 172.21.15.165:0/1653794347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e40039b0 msgr2=0x7fd90015af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e40039b0 0x7fd90015af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 -- 172.21.15.165:0/1653794347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9001f51a0 msgr2=0x7fd9001e3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9001f51a0 0x7fd9001e3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 -- 172.21.15.165:0/1653794347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9001318d0 con 0x7fd9000c9170 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.706+0000 7fd8fd400640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 0x7fd9001f4c60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e570 tx=0x7fd8e800ea30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.707+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e800f040 con 0x7fd9000c9170 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.707+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8e8004250 con 0x7fd9000c9170 2024-04-25T10:06:08.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.707+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9001f57d0 con 0x7fd9000c9170 2024-04-25T10:06:08.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.707+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8e8003610 con 0x7fd9000c9170 2024-04-25T10:06:08.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.707+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9001e1a00 con 0x7fd9000c9170 2024-04-25T10:06:08.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.708+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd900073990 con 0x7fd9000c9170 2024-04-25T10:06:08.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.709+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8e80037b0 con 0x7fd9000c9170 2024-04-25T10:06:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.710+0000 7fd8f6000640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd8dc06b500 0x7fd8dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.710+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fd8e8089f10 con 0x7fd9000c9170 2024-04-25T10:06:08.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.710+0000 7fd8fde00640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd8dc06b500 0x7fd8dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:08.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.711+0000 7fd8fde00640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd8dc06b500 0x7fd8dc06d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0033000 tx=0x7fd8f0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:08.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.714+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8e8017020 con 0x7fd9000c9170 2024-04-25T10:06:08.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.944+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd900143c60 con 0x7fd9000c9170 2024-04-25T10:06:08.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.945+0000 7fd8f6000640 1 -- 172.21.15.165:0/1653794347 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fd8e8058b30 con 0x7fd9000c9170 2024-04-25T10:06:08.950 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:08.950 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:40.296050+0000","modified":"2024-04-25T10:03:49.454381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/3282481342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3282481342},{"type":"v1","addr":"172.21.15.165:6837","nonce":3282481342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:08.950 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 20 2024-04-25T10:06:08.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.947+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd8dc06b500 msgr2=0x7fd8dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.947+0000 7fd8ff200640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd8dc06b500 0x7fd8dc06d9c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd8f0033000 tx=0x7fd8f0007000 comp rx=0 tx=0).stop 2024-04-25T10:06:08.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.947+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 msgr2=0x7fd9001f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:08.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.947+0000 7fd8ff200640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 0x7fd9001f4c60 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fd8e800e570 tx=0x7fd8e800ea30 comp rx=0 tx=0).stop 2024-04-25T10:06:08.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 shutdown_connections 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd8dc06b500 0x7fd8dc06d9c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9001f51a0 0x7fd9001e3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9000c9170 0x7fd9001f4c60 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 --2- 172.21.15.165:0/1653794347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8e40039b0 0x7fd90015af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 >> 172.21.15.165:0/1653794347 conn(0x7fd90006abe0 msgr2=0x7fd9000c5c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 shutdown_connections 2024-04-25T10:06:08.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:08.948+0000 7fd8ff200640 1 -- 172.21.15.165:0/1653794347 wait complete. 2024-04-25T10:06:08.971 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:09.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.163+0000 7fb5abe00640 1 Processor -- start 2024-04-25T10:06:09.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.163+0000 7fb5abe00640 1 -- start start 2024-04-25T10:06:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5abe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5ac15e900 0x7fb5ac0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5abe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac15ed90 con 0x7fb5ac15e520 2024-04-25T10:06:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5abe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ac0ae590 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5abe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ac0aee30 con 0x7fb5ac15e900 2024-04-25T10:06:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5a3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5ac15e900 0x7fb5ac0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5a3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5ac15e900 0x7fb5ac0adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41480/0 (socket says 172.21.15.165:41480) 2024-04-25T10:06:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.165+0000 7fb5a3e00640 1 -- 172.21.15.165:0/2457741467 learned_addr learned my addr 172.21.15.165:0/2457741467 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.166+0000 7fb5a3e00640 1 -- 172.21.15.165:0/2457741467 >> v1:172.21.15.172:6789/0 conn(0x7fb5ac0ae0d0 legacy=0x7fb5ac070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.166+0000 7fb5a3e00640 1 -- 172.21.15.165:0/2457741467 >> v1:172.21.15.165:6789/0 conn(0x7fb5ac15e520 legacy=0x7fb5ac15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.166+0000 7fb5a3e00640 1 -- 172.21.15.165:0/2457741467 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac0ae870 con 0x7fb5ac15e900 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.167+0000 7fb5a3e00640 1 --2- 172.21.15.165:0/2457741467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac15e900 0x7fb5ac0adb90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb594009870 tx=0x7fb59402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8070040772730e4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.167+0000 7fb5aa000640 1 -- 172.21.15.165:0/2457741467 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594037070 con 0x7fb5ac15e900 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.167+0000 7fb5aa000640 1 -- 172.21.15.165:0/2457741467 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb594032440 con 0x7fb5ac15e900 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.167+0000 7fb5aa000640 1 -- 172.21.15.165:0/2457741467 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb594032810 con 0x7fb5ac15e900 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.167+0000 7fb5abe00640 1 -- 172.21.15.165:0/2457741467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac15e900 msgr2=0x7fb5ac0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.167+0000 7fb5abe00640 1 --2- 172.21.15.165:0/2457741467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac15e900 0x7fb5ac0adb90 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fb594009870 tx=0x7fb59402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.168+0000 7fb5abe00640 1 -- 172.21.15.165:0/2457741467 shutdown_connections 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.168+0000 7fb5abe00640 1 --2- 172.21.15.165:0/2457741467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac15e900 0x7fb5ac0adb90 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.168+0000 7fb5abe00640 1 -- 172.21.15.165:0/2457741467 >> 172.21.15.165:0/2457741467 conn(0x7fb5ac06abe0 msgr2=0x7fb5ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.168+0000 7fb5abe00640 1 -- 172.21.15.165:0/2457741467 shutdown_connections 2024-04-25T10:06:09.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.168+0000 7fb5abe00640 1 -- 172.21.15.165:0/2457741467 wait complete. 2024-04-25T10:06:09.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.169+0000 7fb5abe00640 1 Processor -- start 2024-04-25T10:06:09.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 -- start start 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 0x7fb5ac1f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ac15e520 0x7fb5ac0b3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5ac15e900 0x7fb5ac158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac071700 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac06d1f0 con 0x7fb5ac15e520 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.170+0000 7fb5abe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ac071880 con 0x7fb5ac15e900 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 0x7fb5ac1f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 0x7fb5ac1f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41486/0 (socket says 172.21.15.165:41486) 2024-04-25T10:06:09.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 -- 172.21.15.165:0/1622190618 learned_addr learned my addr 172.21.15.165:0/1622190618 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:09.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5a3e00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ac15e520 0x7fb5ac0b3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5ab400640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5ac15e900 0x7fb5ac158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 -- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5ac15e900 msgr2=0x7fb5ac158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5ac15e900 0x7fb5ac158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 -- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ac15e520 msgr2=0x7fb5ac0b3750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ac15e520 0x7fb5ac0b3750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5aaa00640 1 -- 172.21.15.165:0/1622190618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ac1f7530 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5a3e00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ac15e520 0x7fb5ac0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.171+0000 7fb5ab400640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5ac15e900 0x7fb5ac158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.172+0000 7fb5aaa00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 0x7fb5ac1f6e50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c400 tx=0x7fb59c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.172+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c0052d0 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.172+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb59c005490 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.172+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ac1f7810 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.172+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb59c007260 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.172+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ac0b1ef0 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.173+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ac073990 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.174+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb59c007400 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.175+0000 7fb5a2a00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb58806b4d0 0x7fb58806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.175+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7fb59c0898e0 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.175+0000 7fb5a3e00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb58806b4d0 0x7fb58806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.176+0000 7fb5a3e00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb58806b4d0 0x7fb58806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb594034000 tx=0x7fb59402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.180+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb59c058590 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.412+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5ac15f170 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.417+0000 7fb5a2a00640 1 -- 172.21.15.165:0/1622190618 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb59c057e90 con 0x7fb5ac0ae0d0 2024-04-25T10:06:09.422 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:09.423 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":20,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:03:40.296050+0000","modified":"2024-04-25T10:03:49.454381+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"a","rank":0,"incarnation":15,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/3282481342","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":3282481342},{"type":"v1","addr":"172.21.15.165:6837","nonce":3282481342}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2024-04-25T10:06:09.423 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 20 2024-04-25T10:06:09.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.419+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb58806b4d0 msgr2=0x7fb58806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.419+0000 7fb5abe00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb58806b4d0 0x7fb58806d990 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb594034000 tx=0x7fb59402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:09.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.419+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 msgr2=0x7fb5ac1f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.419+0000 7fb5abe00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 0x7fb5ac1f6e50 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fb59c00c400 tx=0x7fb59c00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 shutdown_connections 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb58806b4d0 0x7fb58806d990 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb5ac15e900 0x7fb5ac158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5ac15e520 0x7fb5ac0b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 --2- 172.21.15.165:0/1622190618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5ac0ae0d0 0x7fb5ac1f6e50 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 >> 172.21.15.165:0/1622190618 conn(0x7fb5ac06abe0 msgr2=0x7fb5ac0b4400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 shutdown_connections 2024-04-25T10:06:09.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.420+0000 7fb5abe00640 1 -- 172.21.15.165:0/1622190618 wait complete. 2024-04-25T10:06:09.444 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:09.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.633+0000 7f3bcd400640 1 Processor -- start 2024-04-25T10:06:09.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.633+0000 7f3bcd400640 1 -- start start 2024-04-25T10:06:09.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bcd400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b93c0 0x7f3bc80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bcd400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3bc80d04f0 0x7f3bc80d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bcd400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc8161df0 con 0x7f3bc815db90 2024-04-25T10:06:09.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bcd400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc80d0980 con 0x7f3bc80d04f0 2024-04-25T10:06:09.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bcd400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80d1f50 con 0x7f3bc80b93c0 2024-04-25T10:06:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bcca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3bc80d04f0 0x7f3bc80d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.635+0000 7f3bbf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b93c0 0x7f3bc80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bcca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3bc80d04f0 0x7f3bc80d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:36182/0 (socket says 172.21.15.165:36182) 2024-04-25T10:06:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bbf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b93c0 0x7f3bc80cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:51518/0 (socket says 172.21.15.165:51518) 2024-04-25T10:06:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bbf400640 1 -- 172.21.15.165:0/3735486519 learned_addr learned my addr 172.21.15.165:0/3735486519 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bcca00640 1 -- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b93c0 msgr2=0x7f3bc80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bcca00640 1 --2- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b93c0 0x7f3bc80cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bcca00640 1 -- 172.21.15.165:0/3735486519 >> v1:172.21.15.172:6789/0 conn(0x7f3bc815db90 legacy=0x7f3bc81615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.636+0000 7f3bcca00640 1 -- 172.21.15.165:0/3735486519 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc81620d0 con 0x7f3bc80d04f0 2024-04-25T10:06:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.637+0000 7f3bcca00640 1 --2- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc80d23e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009d90 tx=0x7f3bc002a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b41294f18f996835 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.638+0000 7f3bc7400640 1 -- 172.21.15.165:0/3735486519 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0007740 con 0x7f3bc80d04f0 2024-04-25T10:06:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.638+0000 7f3bc7400640 1 -- 172.21.15.165:0/3735486519 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc00078e0 con 0x7f3bc80d04f0 2024-04-25T10:06:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.638+0000 7f3bc7400640 1 -- 172.21.15.165:0/3735486519 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0007cb0 con 0x7f3bc80d04f0 2024-04-25T10:06:09.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.638+0000 7f3bcd400640 1 -- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 msgr2=0x7f3bc80d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.638+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc80d23e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3bc0009d90 tx=0x7f3bc002a660 comp rx=0 tx=0).stop 2024-04-25T10:06:09.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.639+0000 7f3bcd400640 1 -- 172.21.15.165:0/3735486519 shutdown_connections 2024-04-25T10:06:09.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.639+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b93c0 0x7f3bc80cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.639+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3735486519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc80d23e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.639+0000 7f3bcd400640 1 -- 172.21.15.165:0/3735486519 >> 172.21.15.165:0/3735486519 conn(0x7f3bc80b52b0 msgr2=0x7f3bc80b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.639+0000 7f3bcd400640 1 -- 172.21.15.165:0/3735486519 shutdown_connections 2024-04-25T10:06:09.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.639+0000 7f3bcd400640 1 -- 172.21.15.165:0/3735486519 wait complete. 2024-04-25T10:06:09.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.640+0000 7f3bcd400640 1 Processor -- start 2024-04-25T10:06:09.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.641+0000 7f3bcd400640 1 -- start start 2024-04-25T10:06:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.641+0000 7f3bcd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 0x7f3bc80cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.641+0000 7f3bcd400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc815b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.641+0000 7f3bcd400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc815db90 0x7f3bc81574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.641+0000 7f3bcd400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80b68d0 con 0x7f3bc80b93c0 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bcd400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80d1be0 con 0x7f3bc80d04f0 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bcd400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80b6a50 con 0x7f3bc815db90 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 0x7f3bc80cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 0x7f3bc80cc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41500/0 (socket says 172.21.15.165:41500) 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3263592886 learned_addr learned my addr 172.21.15.165:0/3263592886 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bcca00640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc815db90 0x7f3bc81574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bbf400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc815b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc815db90 msgr2=0x7f3bc81574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc815db90 0x7f3bc81574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 msgr2=0x7f3bc815b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc815b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3263592886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc80cd0e0 con 0x7f3bc80b93c0 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.642+0000 7f3bbf400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.643+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 0x7f3bc80cc850 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800c3c0 tx=0x7f3bb800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.643+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb80052d0 con 0x7f3bc80b93c0 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.643+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8005490 con 0x7f3bc80b93c0 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.643+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc80ca970 con 0x7f3bc80b93c0 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.643+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8007260 con 0x7f3bc80b93c0 2024-04-25T10:06:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.643+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc80ccd90 con 0x7f3bc80b93c0 2024-04-25T10:06:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.644+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc8073990 con 0x7f3bc80b93c0 2024-04-25T10:06:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.645+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bb8007400 con 0x7f3bc80b93c0 2024-04-25T10:06:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.646+0000 7f3bc4c00640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.646+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f3bb8089940 con 0x7f3bc80b93c0 2024-04-25T10:06:09.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.647+0000 7f3bbf400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:09.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.647+0000 7f3bbf400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3bc81528e0 tx=0x7f3bb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:09.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.651+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bc80ca970 con 0x7f3bc80b93c0 2024-04-25T10:06:09.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.849+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3bc80c9c90 con 0x7f3bc80b93c0 2024-04-25T10:06:09.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.852+0000 7f3bc4c00640 1 -- 172.21.15.165:0/3263592886 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17036 (secure 0 0 0) 0x7f3bb8058560 con 0x7f3bc80b93c0 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:09.858 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":43,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:05:39.026672+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:03:38.087466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"38","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_data","create_time":"2024-04-25T10:03:39.315047+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":42,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.33","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:09.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.854+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 msgr2=0x7f3ba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3bc81528e0 tx=0x7f3bb0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:09.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 msgr2=0x7f3bc80cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:09.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 0x7f3bc80cc850 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800c3c0 tx=0x7f3bb800c880 comp rx=0 tx=0).stop 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 shutdown_connections 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc815db90 0x7f3bc81574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc80d04f0 0x7f3bc815b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 --2- 172.21.15.165:0/3263592886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc80b93c0 0x7f3bc80cc850 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 >> 172.21.15.165:0/3263592886 conn(0x7f3bc80b52b0 msgr2=0x7f3bc8158410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 shutdown_connections 2024-04-25T10:06:09.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:09.855+0000 7f3bcd400640 1 -- 172.21.15.165:0/3263592886 wait complete. 2024-04-25T10:06:09.879 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:06:10.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.069+0000 7f541a600640 1 Processor -- start 2024-04-25T10:06:10.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.070+0000 7f541a600640 1 -- start start 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f541a600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54140d1120 0x7f541406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f541a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f541415de10 0x7f54140cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f541a600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54140d0080 con 0x7f5414099cb0 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f541a600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f541406c4b0 con 0x7f54140d1120 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f541a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54140cfe30 con 0x7f541415de10 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f5419200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5414099cb0 0x7f541415d700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51398/0 (socket says 172.21.15.165:51398) 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.071+0000 7f5419200640 1 -- 172.21.15.165:0/4016101034 learned_addr learned my addr 172.21.15.165:0/4016101034 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:10.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413e00640 1 --2- 172.21.15.165:0/4016101034 >> v2:172.21.15.165:3300/0 conn(0x7f54140d1120 0x7f541406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413400640 1 -- 172.21.15.165:0/4016101034 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501087816 0 0) 0x7f54140d0080 con 0x7f5414099cb0 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5419c00640 1 --2- 172.21.15.165:0/4016101034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f541415de10 0x7f54140cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413400640 1 -- 172.21.15.165:0/4016101034 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53f8003520 con 0x7f5414099cb0 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413e00640 1 -- 172.21.15.165:0/4016101034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f541415de10 msgr2=0x7f54140cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413e00640 1 --2- 172.21.15.165:0/4016101034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f541415de10 0x7f54140cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413e00640 1 -- 172.21.15.165:0/4016101034 >> v1:172.21.15.165:6789/0 conn(0x7f5414099cb0 legacy=0x7f541415d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413400640 1 -- 172.21.15.165:0/4016101034 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017798613 0 0) 0x7f53f8003520 con 0x7f5414099cb0 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.072+0000 7f5413e00640 1 -- 172.21.15.165:0/4016101034 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54140d0360 con 0x7f54140d1120 2024-04-25T10:06:10.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.073+0000 7f5413e00640 1 --2- 172.21.15.165:0/4016101034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f541406bf70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097a0 tx=0x7f53fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d645fac1bfaf7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.073+0000 7f5413400640 1 -- 172.21.15.165:0/4016101034 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc037070 con 0x7f54140d1120 2024-04-25T10:06:10.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f5413400640 1 -- 172.21.15.165:0/4016101034 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc02ac30 con 0x7f54140d1120 2024-04-25T10:06:10.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f5413400640 1 -- 172.21.15.165:0/4016101034 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54040019f0 con 0x7f54140d1120 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 -- 172.21.15.165:0/4016101034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 msgr2=0x7f541406bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 --2- 172.21.15.165:0/4016101034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f541406bf70 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f53fc0097a0 tx=0x7f53fc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 -- 172.21.15.165:0/4016101034 shutdown_connections 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 --2- 172.21.15.165:0/4016101034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f541415de10 0x7f54140cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 --2- 172.21.15.165:0/4016101034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f541406bf70 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 -- 172.21.15.165:0/4016101034 >> 172.21.15.165:0/4016101034 conn(0x7f54140b3250 msgr2=0x7f54140b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 -- 172.21.15.165:0/4016101034 shutdown_connections 2024-04-25T10:06:10.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.074+0000 7f541a600640 1 -- 172.21.15.165:0/4016101034 wait complete. 2024-04-25T10:06:10.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.076+0000 7f541a600640 1 Processor -- start 2024-04-25T10:06:10.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.076+0000 7f541a600640 1 -- start start 2024-04-25T10:06:10.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.077+0000 7f541a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5414099cb0 0x7f5414158db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.077+0000 7f541a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f54140cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.077+0000 7f541a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541415de10 0x7f54140cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.077+0000 7f541a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54140d0c00 con 0x7f54140d1120 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.077+0000 7f541a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54140d0920 con 0x7f541415de10 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.077+0000 7f541a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f541409a090 con 0x7f5414099cb0 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f54140cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f54140cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41524/0 (socket says 172.21.15.165:41524) 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 -- 172.21.15.165:0/3281484925 learned_addr learned my addr 172.21.15.165:0/3281484925 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5419200640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5414099cb0 0x7f5414158db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5419c00640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541415de10 0x7f54140cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 -- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5414099cb0 msgr2=0x7f5414158db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5414099cb0 0x7f5414158db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 -- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541415de10 msgr2=0x7f54140cbf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541415de10 0x7f54140cbf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5413e00640 1 -- 172.21.15.165:0/3281484925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54141592f0 con 0x7f54140d1120 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5419200640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5414099cb0 0x7f5414158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.078+0000 7f5419c00640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541415de10 0x7f54140cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.079+0000 7f5413e00640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f54140cdf80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f53fc035040 tx=0x7f53fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.079+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc041070 con 0x7f54140d1120 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.079+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc03c660 con 0x7f54140d1120 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.079+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54140ce7f0 con 0x7f54140d1120 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.079+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc037040 con 0x7f54140d1120 2024-04-25T10:06:10.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.079+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54140b4d80 con 0x7f54140d1120 2024-04-25T10:06:10.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.080+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54140b4010 con 0x7f54140d1120 2024-04-25T10:06:10.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.081+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53fc044070 con 0x7f54140d1120 2024-04-25T10:06:10.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.082+0000 7f5410c00640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53f006b4d0 0x7f53f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.082+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6158+0+0 (secure 0 0 0) 0x7f53fc0aacd0 con 0x7f54140d1120 2024-04-25T10:06:10.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.083+0000 7f5419200640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53f006b4d0 0x7f53f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.083+0000 7f5419200640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53f006b4d0 0x7f53f006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5404005c00 tx=0x7f5404009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.087+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53fc0798f0 con 0x7f54140d1120 2024-04-25T10:06:10.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.319+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f541415e590 con 0x7f54140d1120 2024-04-25T10:06:10.325 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:06:10.320+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:06:10.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.324+0000 7f5410c00640 1 -- 172.21.15.165:0/3281484925 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7f53fc0791f0 con 0x7f54140d1120 2024-04-25T10:06:10.329 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:06:10.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.327+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53f006b4d0 msgr2=0x7f53f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.327+0000 7f541a600640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53f006b4d0 0x7f53f006d990 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5404005c00 tx=0x7f5404009040 comp rx=0 tx=0).stop 2024-04-25T10:06:10.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.327+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 msgr2=0x7f54140cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.327+0000 7f541a600640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f54140cdf80 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f53fc035040 tx=0x7f53fc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.328+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 shutdown_connections 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.328+0000 7f541a600640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53f006b4d0 0x7f53f006d990 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.328+0000 7f541a600640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f541415de10 0x7f54140cbf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.328+0000 7f541a600640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54140d1120 0x7f54140cdf80 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.328+0000 7f541a600640 1 --2- 172.21.15.165:0/3281484925 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5414099cb0 0x7f5414158db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.328+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 >> 172.21.15.165:0/3281484925 conn(0x7f54140b3250 msgr2=0x7f541406bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.329+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 shutdown_connections 2024-04-25T10:06:10.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.329+0000 7f541a600640 1 -- 172.21.15.165:0/3281484925 wait complete. 2024-04-25T10:06:10.353 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:06:10.447 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.540+0000 7fbe3f200640 1 Processor -- start 2024-04-25T10:06:10.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.541+0000 7fbe3f200640 1 -- start start 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.542+0000 7fbe3f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe40145140 0x7fbe400ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.542+0000 7fbe3f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe400c5650 con 0x7fbe400c5200 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.542+0000 7fbe3f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe40145520 con 0x7fbe400ce950 2024-04-25T10:06:10.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.542+0000 7fbe3f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe400ced90 con 0x7fbe40145140 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.542+0000 7fbe3e800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbe400c5200 0x7fbe4015bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51422/0 (socket says 172.21.15.165:51422) 2024-04-25T10:06:10.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.542+0000 7fbe3e800640 1 -- 172.21.15.165:0/2499410266 learned_addr learned my addr 172.21.15.165:0/2499410266 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3de00640 1 --2- 172.21.15.165:0/2499410266 >> v2:172.21.15.172:3300/0 conn(0x7fbe40145140 0x7fbe400ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636291283 0 0) 0x7fbe400c5650 con 0x7fbe400c5200 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe24003520 con 0x7fbe400c5200 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2912660536 0 0) 0x7fbe24003520 con 0x7fbe400c5200 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 >> v2:172.21.15.172:3300/0 conn(0x7fbe40145140 msgr2=0x7fbe400ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 --2- 172.21.15.165:0/2499410266 >> v2:172.21.15.172:3300/0 conn(0x7fbe40145140 0x7fbe400ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 >> v1:172.21.15.172:6789/0 conn(0x7fbe400ce950 legacy=0x7fbe400c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe40071270 con 0x7fbe400c5200 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fbe380019f0 con 0x7fbe400c5200 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 >> v1:172.21.15.165:6789/0 conn(0x7fbe400c5200 legacy=0x7fbe4015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe240051c0 0x7fbe2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400ced90 con 0x7fbe240039b0 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2400c160 con 0x7fbe24004800 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.543+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2400c360 con 0x7fbe240051c0 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3de00640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe35400640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3de00640 1 -- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe240051c0 msgr2=0x7fbe2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3de00640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe240051c0 0x7fbe2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3de00640 1 -- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe24004800 msgr2=0x7fbe24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3de00640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3de00640 1 -- 172.21.15.165:0/2499410266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe24003520 con 0x7fbe240039b0 2024-04-25T10:06:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe35400640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe24004800 0x7fbe24004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.544+0000 7fbe3e800640 1 -- 172.21.15.165:0/2499410266 reap_dead start 2024-04-25T10:06:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.545+0000 7fbe3de00640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009ac0 tx=0x7fbe24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=346591f99f0f97aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.545+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30036070 con 0x7fbe240039b0 2024-04-25T10:06:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.545+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30031400 con 0x7fbe240039b0 2024-04-25T10:06:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.545+0000 7fbe3d400640 1 -- 172.21.15.165:0/2499410266 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30009690 con 0x7fbe240039b0 2024-04-25T10:06:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 -- 172.21.15.165:0/2499410266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe240039b0 msgr2=0x7fbe24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fbe30009ac0 tx=0x7fbe24005830 comp rx=0 tx=0).stop 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 -- 172.21.15.165:0/2499410266 shutdown_connections 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 --2- 172.21.15.165:0/2499410266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe240039b0 0x7fbe24003dd0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 -- 172.21.15.165:0/2499410266 >> 172.21.15.165:0/2499410266 conn(0x7fbe40061cc0 msgr2=0x7fbe4006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 -- 172.21.15.165:0/2499410266 shutdown_connections 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.546+0000 7fbe3f200640 1 -- 172.21.15.165:0/2499410266 wait complete. 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.548+0000 7fbe3f200640 1 Processor -- start 2024-04-25T10:06:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.548+0000 7fbe3f200640 1 -- start start 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe240039b0 0x7fbe400c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 0x7fbe401e71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe3f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe4012fa30 0x7fbe4012feb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe3f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40071b10 con 0x7fbe401e6d90 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe3f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe40071830 con 0x7fbe240039b0 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe3f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe400ce410 con 0x7fbe4012fa30 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 0x7fbe401e71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 0x7fbe401e71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41556/0 (socket says 172.21.15.165:41556) 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.549+0000 7fbe35400640 1 -- 172.21.15.165:0/3875932612 learned_addr learned my addr 172.21.15.165:0/3875932612 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe3e800640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe4012fa30 0x7fbe4012feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe3de00640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe240039b0 0x7fbe400c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe35400640 1 -- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe4012fa30 msgr2=0x7fbe4012feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe35400640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe4012fa30 0x7fbe4012feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe35400640 1 -- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe240039b0 msgr2=0x7fbe400c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe35400640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe240039b0 0x7fbe400c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe35400640 1 -- 172.21.15.165:0/3875932612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe4012d9a0 con 0x7fbe401e6d90 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe3de00640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe240039b0 0x7fbe400c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe35400640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 0x7fbe401e71d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800d4a0 tx=0x7fbe2800d960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28019070 con 0x7fbe401e6d90 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe28005480 con 0x7fbe401e6d90 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe28007230 con 0x7fbe401e6d90 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.550+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe401e7af0 con 0x7fbe401e6d90 2024-04-25T10:06:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.551+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe4012df40 con 0x7fbe401e6d90 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.552+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe40073990 con 0x7fbe401e6d90 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.552+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe280073d0 con 0x7fbe401e6d90 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.553+0000 7fbe36a00640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.553+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7fbe28089970 con 0x7fbe401e6d90 2024-04-25T10:06:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.554+0000 7fbe3de00640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.554+0000 7fbe3de00640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe30033000 tx=0x7fbe30008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:10.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.558+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe28057f00 con 0x7fbe401e6d90 2024-04-25T10:06:10.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:10.772+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbe4015d2f0 con 0x7fbe401e6d90 2024-04-25T10:06:11.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.427+0000 7fbe36a00640 1 -- 172.21.15.165:0/3875932612 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbe2805bfa0 con 0x7fbe401e6d90 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1c06b4d0 msgr2=0x7fbe1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fbe30033000 tx=0x7fbe30008030 comp rx=0 tx=0).stop 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 msgr2=0x7fbe401e71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 0x7fbe401e71d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fbe2800d4a0 tx=0x7fbe2800d960 comp rx=0 tx=0).stop 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 shutdown_connections 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1c06b4d0 0x7fbe1c06d990 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe4012fa30 0x7fbe4012feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe401e6d90 0x7fbe401e71d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 --2- 172.21.15.165:0/3875932612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe240039b0 0x7fbe400c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 >> 172.21.15.165:0/3875932612 conn(0x7fbe40061cc0 msgr2=0x7fbe40144400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 shutdown_connections 2024-04-25T10:06:11.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.432+0000 7fbe3f200640 1 -- 172.21.15.165:0/3875932612 wait complete. 2024-04-25T10:06:11.460 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:06:11.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.661+0000 7f4509400640 1 Processor -- start 2024-04-25T10:06:11.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.661+0000 7f4509400640 1 -- start start 2024-04-25T10:06:11.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4509400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f45040cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4509400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f450406ec50 0x7f450415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4509400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f450415ee60 0x7f45040ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4509400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f450415e490 con 0x7f450415ee60 2024-04-25T10:06:11.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4509400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45040cfd80 con 0x7f450406ec50 2024-04-25T10:06:11.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4509400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040d0060 con 0x7f450415b710 2024-04-25T10:06:11.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4503200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f450415ee60 0x7f45040ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4503200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f450415ee60 0x7f45040ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41562/0 (socket says 172.21.15.165:41562) 2024-04-25T10:06:11.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4503200640 1 -- 172.21.15.165:0/1782149999 learned_addr learned my addr 172.21.15.165:0/1782149999 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.663+0000 7f4502800640 1 --2- 172.21.15.165:0/1782149999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f45040cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4501e00640 1 --2- 172.21.15.165:0/1782149999 >> v2:172.21.15.172:3300/0 conn(0x7f450406ec50 0x7f450415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4503200640 1 -- 172.21.15.165:0/1782149999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 msgr2=0x7f45040cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4503200640 1 --2- 172.21.15.165:0/1782149999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f45040cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4503200640 1 -- 172.21.15.165:0/1782149999 >> v2:172.21.15.172:3300/0 conn(0x7f450406ec50 msgr2=0x7f450415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4503200640 1 --2- 172.21.15.165:0/1782149999 >> v2:172.21.15.172:3300/0 conn(0x7f450406ec50 0x7f450415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4503200640 1 -- 172.21.15.165:0/1782149999 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45040d0340 con 0x7f450415ee60 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.664+0000 7f4501e00640 1 --2- 172.21.15.165:0/1782149999 >> v2:172.21.15.172:3300/0 conn(0x7f450406ec50 0x7f450415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.665+0000 7f4503200640 1 --2- 172.21.15.165:0/1782149999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f45040ac8b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f45040d07a0 tx=0x7f44fc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d9424cfdbaa7ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.665+0000 7f4501400640 1 -- 172.21.15.165:0/1782149999 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc036070 con 0x7f450415ee60 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.665+0000 7f4501400640 1 -- 172.21.15.165:0/1782149999 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc031600 con 0x7f450415ee60 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.665+0000 7f4501400640 1 -- 172.21.15.165:0/1782149999 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc0319d0 con 0x7f450415ee60 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 -- 172.21.15.165:0/1782149999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 msgr2=0x7f45040ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 --2- 172.21.15.165:0/1782149999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f45040ac8b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f45040d07a0 tx=0x7f44fc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 -- 172.21.15.165:0/1782149999 shutdown_connections 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 --2- 172.21.15.165:0/1782149999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f45040ac8b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 --2- 172.21.15.165:0/1782149999 >> v2:172.21.15.172:3300/0 conn(0x7f450406ec50 0x7f450415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 --2- 172.21.15.165:0/1782149999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f45040cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 -- 172.21.15.165:0/1782149999 >> 172.21.15.165:0/1782149999 conn(0x7f450406abe0 msgr2=0x7f450406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 -- 172.21.15.165:0/1782149999 shutdown_connections 2024-04-25T10:06:11.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.666+0000 7f4509400640 1 -- 172.21.15.165:0/1782149999 wait complete. 2024-04-25T10:06:11.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.668+0000 7f4509400640 1 Processor -- start 2024-04-25T10:06:11.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.668+0000 7f4509400640 1 -- start start 2024-04-25T10:06:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4509400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f450406ec50 0x7f45040b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4509400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f4504148740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4509400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f450406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4509400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450415bc70 con 0x7f450415ee60 2024-04-25T10:06:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4509400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450415baf0 con 0x7f450406ec50 2024-04-25T10:06:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4509400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45040acdf0 con 0x7f450415b710 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4503200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f450406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4503200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f450406d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41566/0 (socket says 172.21.15.165:41566) 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4503200640 1 -- 172.21.15.165:0/1418716210 learned_addr learned my addr 172.21.15.165:0/1418716210 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4501e00640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f4504148740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4502800640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f450406ec50 0x7f45040b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4503200640 1 -- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 msgr2=0x7f4504148740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.669+0000 7f4503200640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f4504148740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f4503200640 1 -- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f450406ec50 msgr2=0x7f45040b3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:11.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f4503200640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f450406ec50 0x7f45040b3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:11.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f4503200640 1 -- 172.21.15.165:0/1418716210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4504146410 con 0x7f450415ee60 2024-04-25T10:06:11.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f4503200640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f450406d2d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009f40 tx=0x7f44fc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc041020 con 0x7f450415ee60 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44fc007780 con 0x7f450415ee60 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44fc036040 con 0x7f450415ee60 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.670+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45041466f0 con 0x7f450415ee60 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.671+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4504146cb0 con 0x7f450415ee60 2024-04-25T10:06:11.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.672+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4504145a20 con 0x7f450415ee60 2024-04-25T10:06:11.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.673+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f44fc044020 con 0x7f450415ee60 2024-04-25T10:06:11.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.674+0000 7f44faa00640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f44e406b4d0 0x7f44e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:11.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.674+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6216+0+0 (secure 0 0 0) 0x7f44fc0ab180 con 0x7f450415ee60 2024-04-25T10:06:11.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.674+0000 7f4502800640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f44e406b4d0 0x7f44e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:11.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.675+0000 7f4502800640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f44e406b4d0 0x7f44e406d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006fd0 tx=0x7f44f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:11.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.679+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4504145a20 con 0x7f450415ee60 2024-04-25T10:06:11.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:11.867+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f45040acac0 con 0x7f450415ee60 2024-04-25T10:06:12.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.451+0000 7f44faa00640 1 -- 172.21.15.165:0/1418716210 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v45) v1 ==== 181+0+0 (secure 0 0 0) 0x7f44fc04b030 con 0x7f450415ee60 2024-04-25T10:06:12.457 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:06:12.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.456+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f44e406b4d0 msgr2=0x7f44e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.456+0000 7f4509400640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f44e406b4d0 0x7f44e406d990 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f44f4006fd0 tx=0x7f44f4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:12.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.456+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 msgr2=0x7f450406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.456+0000 7f4509400640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f450406d2d0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f44fc009f40 tx=0x7f44fc004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:12.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.457+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 shutdown_connections 2024-04-25T10:06:12.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.457+0000 7f4509400640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f44e406b4d0 0x7f44e406d990 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.457+0000 7f4509400640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f450415ee60 0x7f450406d2d0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.457+0000 7f4509400640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f450415b710 0x7f4504148740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.457+0000 7f4509400640 1 --2- 172.21.15.165:0/1418716210 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f450406ec50 0x7f45040b3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.457+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 >> 172.21.15.165:0/1418716210 conn(0x7f450406abe0 msgr2=0x7f450415a480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.458+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 shutdown_connections 2024-04-25T10:06:12.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.458+0000 7f4509400640 1 -- 172.21.15.165:0/1418716210 wait complete. 2024-04-25T10:06:12.486 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:06:12.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.671+0000 7fd931e00640 1 Processor -- start 2024-04-25T10:06:12.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.671+0000 7fd931e00640 1 -- start start 2024-04-25T10:06:12.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd931e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd92c06ec50 0x7fd92c071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd931e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c0cfad0 con 0x7fd92c0cf640 2024-04-25T10:06:12.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd931e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd92c0cfd80 con 0x7fd92c15b710 2024-04-25T10:06:12.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd931e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd92c15e490 con 0x7fd92c06ec50 2024-04-25T10:06:12.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd92a000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd92c0cf640 0x7fd92c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51436/0 (socket says 172.21.15.165:51436) 2024-04-25T10:06:12.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd92a000640 1 -- 172.21.15.165:0/3164110088 learned_addr learned my addr 172.21.15.165:0/3164110088 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:12.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd92b400640 1 --2- 172.21.15.165:0/3164110088 >> v2:172.21.15.172:3300/0 conn(0x7fd92c06ec50 0x7fd92c071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.673+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3321508657 0 0) 0x7fd92c0cfad0 con 0x7fd92c0cf640 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd910003520 con 0x7fd92c0cf640 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1870000865 0 0) 0x7fd910003520 con 0x7fd92c0cf640 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 >> v2:172.21.15.172:3300/0 conn(0x7fd92c06ec50 msgr2=0x7fd92c071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 --2- 172.21.15.165:0/3164110088 >> v2:172.21.15.172:3300/0 conn(0x7fd92c06ec50 0x7fd92c071bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 >> v1:172.21.15.172:6789/0 conn(0x7fd92c15b710 legacy=0x7fd92c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c0d0060 con 0x7fd92c0cf640 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd918001670 con 0x7fd92c0cf640 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 >> v1:172.21.15.165:6789/0 conn(0x7fd92c0cf640 legacy=0x7fd92c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100039b0 0x7fd910003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd910004800 0x7fd910004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100051c0 0x7fd91000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c15e490 con 0x7fd910004800 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91000c0f0 con 0x7fd9100051c0 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91000c2f0 con 0x7fd9100039b0 2024-04-25T10:06:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.674+0000 7fd92a000640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd910004800 0x7fd910004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92aa00640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100039b0 0x7fd910003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92b400640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100051c0 0x7fd91000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92a000640 1 -- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100039b0 msgr2=0x7fd910003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92a000640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100039b0 0x7fd910003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92a000640 1 -- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100051c0 msgr2=0x7fd91000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92a000640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100051c0 0x7fd91000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92a000640 1 -- 172.21.15.165:0/3164110088 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd910003520 con 0x7fd910004800 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92aa00640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9100039b0 0x7fd910003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92b400640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9100051c0 0x7fd91000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.675+0000 7fd92b400640 1 -- 172.21.15.165:0/3164110088 reap_dead start 2024-04-25T10:06:12.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.676+0000 7fd92a000640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd910004800 0x7fd910004c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c210 tx=0x7fd910005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0abbc7973196a08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.676+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918007d30 con 0x7fd910004800 2024-04-25T10:06:12.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.676+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd918005af0 con 0x7fd910004800 2024-04-25T10:06:12.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.676+0000 7fd929600640 1 -- 172.21.15.165:0/3164110088 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd918007070 con 0x7fd910004800 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 -- 172.21.15.165:0/3164110088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd910004800 msgr2=0x7fd910004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd910004800 0x7fd910004c80 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fd91800c210 tx=0x7fd910005830 comp rx=0 tx=0).stop 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 -- 172.21.15.165:0/3164110088 shutdown_connections 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 --2- 172.21.15.165:0/3164110088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd910004800 0x7fd910004c80 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 -- 172.21.15.165:0/3164110088 >> 172.21.15.165:0/3164110088 conn(0x7fd92c06abe0 msgr2=0x7fd92c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 -- 172.21.15.165:0/3164110088 shutdown_connections 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.677+0000 7fd931e00640 1 -- 172.21.15.165:0/3164110088 wait complete. 2024-04-25T10:06:12.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.679+0000 7fd931e00640 1 Processor -- start 2024-04-25T10:06:12.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.680+0000 7fd931e00640 1 -- start start 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd931e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd910004800 0x7fd92c160f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd931e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd92c15f970 0x7fd92c14a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd931e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 0x7fd92c0c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd931e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c15baf0 con 0x7fd92c0cc340 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd931e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c06f0a0 con 0x7fd92c15f970 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd931e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c15bc70 con 0x7fd910004800 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd92b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 0x7fd92c0c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd92b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 0x7fd92c0c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41586/0 (socket says 172.21.15.165:41586) 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd92b400640 1 -- 172.21.15.165:0/460759119 learned_addr learned my addr 172.21.15.165:0/460759119 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd92a000640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd92c15f970 0x7fd92c14a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.681+0000 7fd92aa00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd910004800 0x7fd92c160f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92b400640 1 -- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd910004800 msgr2=0x7fd92c160f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92b400640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd910004800 0x7fd92c160f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92b400640 1 -- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd92c15f970 msgr2=0x7fd92c14a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92b400640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd92c15f970 0x7fd92c14a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92b400640 1 -- 172.21.15.165:0/460759119 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd92c157330 con 0x7fd92c0cc340 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92aa00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd910004800 0x7fd92c160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:12.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd92b400640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 0x7fd92c0c46d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0ccf60 tx=0x7fd924009d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924019070 con 0x7fd92c0cc340 2024-04-25T10:06:12.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd924007c90 con 0x7fd92c0cc340 2024-04-25T10:06:12.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd924014640 con 0x7fd92c0cc340 2024-04-25T10:06:12.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92c15fe70 con 0x7fd92c0cc340 2024-04-25T10:06:12.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.682+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c160430 con 0x7fd92c0cc340 2024-04-25T10:06:12.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.684+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c073a00 con 0x7fd92c0cc340 2024-04-25T10:06:12.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.684+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd92401c070 con 0x7fd92c0cc340 2024-04-25T10:06:12.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.686+0000 7fd923400640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd90c06b500 0x7fd90c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:12.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.686+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5778+0+0 (secure 0 0 0) 0x7fd924089f60 con 0x7fd92c0cc340 2024-04-25T10:06:12.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.686+0000 7fd92aa00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd90c06b500 0x7fd90c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:12.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.687+0000 7fd92aa00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd90c06b500 0x7fd90c06d9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd92c068c30 tx=0x7fd91c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:12.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.690+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd92c157330 con 0x7fd92c0cc340 2024-04-25T10:06:12.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:12.895+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd92c14aec0 con 0x7fd92c0cc340 2024-04-25T10:06:13.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.464+0000 7fd923400640 1 -- 172.21.15.165:0/460759119 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd924058d30 con 0x7fd92c0cc340 2024-04-25T10:06:13.470 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:06:13.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.467+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd90c06b500 msgr2=0x7fd90c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.467+0000 7fd931e00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd90c06b500 0x7fd90c06d9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fd92c068c30 tx=0x7fd91c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.467+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 msgr2=0x7fd92c0c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.467+0000 7fd931e00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 0x7fd92c0c46d0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fd92c0ccf60 tx=0x7fd924009d30 comp rx=0 tx=0).stop 2024-04-25T10:06:13.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.468+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 shutdown_connections 2024-04-25T10:06:13.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.468+0000 7fd931e00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd90c06b500 0x7fd90c06d9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.468+0000 7fd931e00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd92c0cc340 0x7fd92c0c46d0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.468+0000 7fd931e00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd92c15f970 0x7fd92c14a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.468+0000 7fd931e00640 1 --2- 172.21.15.165:0/460759119 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd910004800 0x7fd92c160f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.468+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 >> 172.21.15.165:0/460759119 conn(0x7fd92c06abe0 msgr2=0x7fd92c0b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.471+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 shutdown_connections 2024-04-25T10:06:13.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.471+0000 7fd931e00640 1 -- 172.21.15.165:0/460759119 wait complete. 2024-04-25T10:06:13.500 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.688+0000 7f1eda800640 1 Processor -- start 2024-04-25T10:06:13.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.688+0000 7f1eda800640 1 -- start start 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1eda800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 0x7f1ed40d0630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1eda800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ed40d0b70 0x7f1ed4146830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1eda800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ed40b5490 con 0x7f1ed40b1140 2024-04-25T10:06:13.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1eda800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ed415ea60 con 0x7f1ed40d0b70 2024-04-25T10:06:13.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1eda800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40b5730 con 0x7f1ed40b1fb0 2024-04-25T10:06:13.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1ed9400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1ed40b1140 0x7f1ed40b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51440/0 (socket says 172.21.15.165:51440) 2024-04-25T10:06:13.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.690+0000 7f1ed9400640 1 -- 172.21.15.165:0/2669963761 learned_addr learned my addr 172.21.15.165:0/2669963761 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:13.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992855181 0 0) 0x7f1ed40b5490 con 0x7f1ed40b1140 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ebc003520 con 0x7f1ed40b1140 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed9e00640 1 --2- 172.21.15.165:0/2669963761 >> v2:172.21.15.172:3300/0 conn(0x7f1ed40d0b70 0x7f1ed4146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed8a00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 0x7f1ed40d0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2431011804 0 0) 0x7f1ebc003520 con 0x7f1ed40b1140 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 msgr2=0x7f1ed40d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 0x7f1ed40d0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 >> v2:172.21.15.172:3300/0 conn(0x7f1ed40d0b70 msgr2=0x7f1ed4146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 --2- 172.21.15.165:0/2669963761 >> v2:172.21.15.172:3300/0 conn(0x7f1ed40d0b70 0x7f1ed4146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed40b5a10 con 0x7f1ed40b1140 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1ec8001670 con 0x7f1ed40b1140 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 >> v1:172.21.15.165:6789/0 conn(0x7f1ed40b1140 legacy=0x7f1ed40b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc0039b0 0x7f1ebc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ebc004800 0x7f1ebc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 0x7f1ebc005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed415ea60 con 0x7f1ebc004800 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40b5730 con 0x7f1ebc0039b0 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc00bc00 con 0x7f1ed40b1fb0 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.691+0000 7f1ed8a00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ebc004800 0x7f1ebc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed9e00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 0x7f1ebc005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed8a00640 1 -- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 msgr2=0x7f1ebc005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed8a00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed40b1fb0 0x7f1ebc005290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed8a00640 1 -- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc0039b0 msgr2=0x7f1ebc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed9400640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc0039b0 0x7f1ebc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed8a00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc0039b0 0x7f1ebc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.692+0000 7f1ed8a00640 1 -- 172.21.15.165:0/2669963761 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc003520 con 0x7f1ebc004800 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.693+0000 7f1ed8a00640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ebc004800 0x7f1ebc004c80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000ba30 tx=0x7f1ebc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8606fe63d234c848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.693+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0030070 con 0x7f1ebc004800 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.693+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec002ade0 con 0x7f1ebc004800 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.693+0000 7f1ed3e00640 1 -- 172.21.15.165:0/2669963761 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec000b600 con 0x7f1ebc004800 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1eda800640 1 -- 172.21.15.165:0/2669963761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ebc004800 msgr2=0x7f1ebc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1eda800640 1 --2- 172.21.15.165:0/2669963761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ebc004800 0x7f1ebc004c80 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f1ec000ba30 tx=0x7f1ebc005830 comp rx=0 tx=0).stop 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1ed9e00640 1 -- 172.21.15.165:0/2669963761 reap_dead start 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1eda800640 1 -- 172.21.15.165:0/2669963761 shutdown_connections 2024-04-25T10:06:13.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1eda800640 1 -- 172.21.15.165:0/2669963761 >> 172.21.15.165:0/2669963761 conn(0x7f1ed40728b0 msgr2=0x7f1ed4099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1eda800640 1 -- 172.21.15.165:0/2669963761 shutdown_connections 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.694+0000 7f1eda800640 1 -- 172.21.15.165:0/2669963761 wait complete. 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.696+0000 7f1eda800640 1 Processor -- start 2024-04-25T10:06:13.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.696+0000 7f1eda800640 1 -- start start 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1eda800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc004800 0x7f1ed41511c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1eda800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed41f57f0 0x7f1ed41f5c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1eda800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 0x7f1ed40c9e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1eda800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40d0630 con 0x7f1ed40c9a10 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1eda800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40b3da0 con 0x7f1ebc004800 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1eda800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ed40d07b0 con 0x7f1ed41f57f0 2024-04-25T10:06:13.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1ed9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 0x7f1ed40c9e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1ed9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 0x7f1ed40c9e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41602/0 (socket says 172.21.15.165:41602) 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1ed9e00640 1 -- 172.21.15.165:0/1171211509 learned_addr learned my addr 172.21.15.165:0/1171211509 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.697+0000 7f1ed9400640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc004800 0x7f1ed41511c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9e00640 1 -- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed41f57f0 msgr2=0x7f1ed41f5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed8a00640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed41f57f0 0x7f1ed41f5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9e00640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed41f57f0 0x7f1ed41f5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9e00640 1 -- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc004800 msgr2=0x7f1ed41511c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9e00640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc004800 0x7f1ed41511c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9e00640 1 -- 172.21.15.165:0/1171211509 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ed41f62e0 con 0x7f1ed40c9a10 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9400640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc004800 0x7f1ed41511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed9e00640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 0x7f1ed40c9e90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1ed406bca0 tx=0x7f1ecc009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc019070 con 0x7f1ed40c9a10 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc007cb0 con 0x7f1ed40c9a10 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc007e60 con 0x7f1ed40c9a10 2024-04-25T10:06:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ed41f65c0 con 0x7f1ed40c9a10 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.698+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ed409a7b0 con 0x7f1ed40c9a10 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.699+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ed409a9b0 con 0x7f1ed40c9a10 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.700+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ecc01c070 con 0x7f1ed40c9a10 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.701+0000 7f1ed1600640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eb406b500 0x7f1eb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:13.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.701+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1ecc089c60 con 0x7f1ed40c9a10 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.702+0000 7f1ed9400640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eb406b500 0x7f1eb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:13.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.702+0000 7f1ed9400640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eb406b500 0x7f1eb406d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.706+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ed409a9b0 con 0x7f1ed40c9a10 2024-04-25T10:06:13.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.923+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1ed4073990 con 0x7f1ed40c9a10 2024-04-25T10:06:13.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.927+0000 7f1ed1600640 1 -- 172.21.15.165:0/1171211509 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1ecc058c40 con 0x7f1ed40c9a10 2024-04-25T10:06:13.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eb406b500 msgr2=0x7f1eb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eb406b500 0x7f1eb406d9c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f1ec8000c00 tx=0x7f1ec8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:13.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 msgr2=0x7f1ed40c9e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:13.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 0x7f1ed40c9e90 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f1ed406bca0 tx=0x7f1ecc009d40 comp rx=0 tx=0).stop 2024-04-25T10:06:13.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 shutdown_connections 2024-04-25T10:06:13.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eb406b500 0x7f1eb406d9c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ed40c9a10 0x7f1ed40c9e90 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ed41f57f0 0x7f1ed41f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 --2- 172.21.15.165:0/1171211509 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ebc004800 0x7f1ed41511c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:13.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 >> 172.21.15.165:0/1171211509 conn(0x7f1ed40728b0 msgr2=0x7f1ed406f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:13.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 shutdown_connections 2024-04-25T10:06:13.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:13.930+0000 7f1eda800640 1 -- 172.21.15.165:0/1171211509 wait complete. 2024-04-25T10:06:13.952 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir' 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.120+0000 7f6e35600640 1 Processor -- start 2024-04-25T10:06:14.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.120+0000 7f6e35600640 1 -- start start 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.122+0000 7f6e35600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e300cedf0 0x7f6e301616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.122+0000 7f6e35600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e300b93a0 0x7f6e300d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.122+0000 7f6e35600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3015e000 con 0x7f6e3015dc20 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.122+0000 7f6e35600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e300b6bb0 con 0x7f6e300cedf0 2024-04-25T10:06:14.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.122+0000 7f6e35600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300b6030 con 0x7f6e300b93a0 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e300cedf0 0x7f6e301616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e300cedf0 0x7f6e301616b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41612/0 (socket says 172.21.15.165:41612) 2024-04-25T10:06:14.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 -- 172.21.15.165:0/2244968873 learned_addr learned my addr 172.21.15.165:0/2244968873 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2f400640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e300b93a0 0x7f6e300d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 -- 172.21.15.165:0/2244968873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e300b93a0 msgr2=0x7f6e300d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e300b93a0 0x7f6e300d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 -- 172.21.15.165:0/2244968873 >> v1:172.21.15.172:6789/0 conn(0x7f6e3015dc20 legacy=0x7f6e300ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2e000640 1 -- 172.21.15.165:0/2244968873 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30161de0 con 0x7f6e300cedf0 2024-04-25T10:06:14.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.123+0000 7f6e2f400640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e300b93a0 0x7f6e300d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.124+0000 7f6e2e000640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300cedf0 0x7f6e301616b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009e50 tx=0x7f6e1802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed5ef69735a65244 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.124+0000 7f6e2d600640 1 -- 172.21.15.165:0/2244968873 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e18036070 con 0x7f6e300cedf0 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e2d600640 1 -- 172.21.15.165:0/2244968873 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e18031600 con 0x7f6e300cedf0 2024-04-25T10:06:14.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e2d600640 1 -- 172.21.15.165:0/2244968873 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e180319d0 con 0x7f6e300cedf0 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 -- 172.21.15.165:0/2244968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300cedf0 msgr2=0x7f6e301616b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300cedf0 0x7f6e301616b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6e18009e50 tx=0x7f6e1802a710 comp rx=0 tx=0).stop 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 -- 172.21.15.165:0/2244968873 shutdown_connections 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e300b93a0 0x7f6e300d2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 --2- 172.21.15.165:0/2244968873 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300cedf0 0x7f6e301616b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 -- 172.21.15.165:0/2244968873 >> 172.21.15.165:0/2244968873 conn(0x7f6e300d0490 msgr2=0x7f6e300d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.125+0000 7f6e35600640 1 -- 172.21.15.165:0/2244968873 shutdown_connections 2024-04-25T10:06:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.126+0000 7f6e35600640 1 -- 172.21.15.165:0/2244968873 wait complete. 2024-04-25T10:06:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.127+0000 7f6e35600640 1 Processor -- start 2024-04-25T10:06:14.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.127+0000 7f6e35600640 1 -- start start 2024-04-25T10:06:14.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.128+0000 7f6e35600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 0x7f6e30161260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.128+0000 7f6e35600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e300cedf0 0x7f6e300cc570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.128+0000 7f6e35600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3015dc20 0x7f6e300cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.128+0000 7f6e35600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300d2a50 con 0x7f6e300b93a0 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.128+0000 7f6e35600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e301623a0 con 0x7f6e300cedf0 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.128+0000 7f6e35600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e300d2bd0 con 0x7f6e3015dc20 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 0x7f6e30161260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 0x7f6e30161260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41618/0 (socket says 172.21.15.165:41618) 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 -- 172.21.15.165:0/573175837 learned_addr learned my addr 172.21.15.165:0/573175837 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2f400640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3015dc20 0x7f6e300cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2e000640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e300cedf0 0x7f6e300cc570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 -- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3015dc20 msgr2=0x7f6e300cbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3015dc20 0x7f6e300cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 -- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e300cedf0 msgr2=0x7f6e300cc570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e300cedf0 0x7f6e300cc570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 -- 172.21.15.165:0/573175837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e300cc320 con 0x7f6e300b93a0 2024-04-25T10:06:14.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.129+0000 7f6e2ea00640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 0x7f6e30161260 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c3c0 tx=0x7f6e2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.130+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e200052d0 con 0x7f6e300b93a0 2024-04-25T10:06:14.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.130+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20005490 con 0x7f6e300b93a0 2024-04-25T10:06:14.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.130+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20007260 con 0x7f6e300b93a0 2024-04-25T10:06:14.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.130+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e30158240 con 0x7f6e300b93a0 2024-04-25T10:06:14.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.130+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e30158800 con 0x7f6e300b93a0 2024-04-25T10:06:14.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.131+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e300b9150 con 0x7f6e300b93a0 2024-04-25T10:06:14.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.132+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e20007400 con 0x7f6e300b93a0 2024-04-25T10:06:14.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.133+0000 7f6e26a00640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e0c06b500 0x7f6e0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.134+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6e200894f0 con 0x7f6e300b93a0 2024-04-25T10:06:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.134+0000 7f6e2e000640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e0c06b500 0x7f6e0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.134+0000 7f6e2e000640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e0c06b500 0x7f6e0c06d9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e18033000 tx=0x7f6e18009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.138+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e3015e000 con 0x7f6e300b93a0 2024-04-25T10:06:14.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.412+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]} v 0) v1 -- 0x7f6e30073990 con 0x7f6e300b93a0 2024-04-25T10:06:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.467+0000 7f6e26a00640 1 -- 172.21.15.165:0/573175837 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_flag_scrub_mdsdir"]}]=0 v208) v1 ==== 148+0+0 (secure 0 0 0) 0x7f6e200584d0 con 0x7f6e300b93a0 2024-04-25T10:06:14.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e0c06b500 msgr2=0x7f6e0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e0c06b500 0x7f6e0c06d9c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f6e18033000 tx=0x7f6e18009000 comp rx=0 tx=0).stop 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 msgr2=0x7f6e30161260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 0x7f6e30161260 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000c3c0 tx=0x7f6e2000c880 comp rx=0 tx=0).stop 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 shutdown_connections 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e0c06b500 0x7f6e0c06d9c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3015dc20 0x7f6e300cbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e300cedf0 0x7f6e300cc570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 --2- 172.21.15.165:0/573175837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e300b93a0 0x7f6e30161260 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.471+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 >> 172.21.15.165:0/573175837 conn(0x7f6e300d0490 msgr2=0x7f6e300b5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.472+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 shutdown_connections 2024-04-25T10:06:14.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.472+0000 7f6e35600640 1 -- 172.21.15.165:0/573175837 wait complete. 2024-04-25T10:06:14.497 INFO:tasks.cephfs_test_runner:test flag scrub_mdsdir ... ok 2024-04-25T10:06:14.497 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:06:14.498 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:14.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.694+0000 7f3502400640 1 Processor -- start 2024-04-25T10:06:14.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.694+0000 7f3502400640 1 -- start start 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3502400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc15c100 0x7f34fc071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3502400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f34fc0b3800 0x7f34fc06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3502400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34fc06eac0 0x7f34fc165850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3502400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc0b3420 con 0x7f34fc06eac0 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3502400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34fc071c80 con 0x7f34fc0b3800 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3502400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0b2810 con 0x7f34fc15c100 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.696+0000 7f3501a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34fc06eac0 0x7f34fc165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34fc06eac0 0x7f34fc165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41628/0 (socket says 172.21.15.165:41628) 2024-04-25T10:06:14.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 -- 172.21.15.165:0/4496072 learned_addr learned my addr 172.21.15.165:0/4496072 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f34fbe00640 1 --2- 172.21.15.165:0/4496072 >> v2:172.21.15.172:3300/0 conn(0x7f34fc0b3800 0x7f34fc06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501000640 1 --2- 172.21.15.165:0/4496072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc15c100 0x7f34fc071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 -- 172.21.15.165:0/4496072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc15c100 msgr2=0x7f34fc071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 --2- 172.21.15.165:0/4496072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc15c100 0x7f34fc071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 -- 172.21.15.165:0/4496072 >> v2:172.21.15.172:3300/0 conn(0x7f34fc0b3800 msgr2=0x7f34fc06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 --2- 172.21.15.165:0/4496072 >> v2:172.21.15.172:3300/0 conn(0x7f34fc0b3800 0x7f34fc06e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.697+0000 7f3501a00640 1 -- 172.21.15.165:0/4496072 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc0b2af0 con 0x7f34fc06eac0 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.698+0000 7f3501a00640 1 --2- 172.21.15.165:0/4496072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc165850 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009e50 tx=0x7f34f402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df52b14ddeaf5ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.698+0000 7f34fb400640 1 -- 172.21.15.165:0/4496072 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f4036070 con 0x7f34fc06eac0 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.698+0000 7f34fb400640 1 -- 172.21.15.165:0/4496072 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f4031600 con 0x7f34fc06eac0 2024-04-25T10:06:14.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.698+0000 7f34fb400640 1 -- 172.21.15.165:0/4496072 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f40319d0 con 0x7f34fc06eac0 2024-04-25T10:06:14.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 -- 172.21.15.165:0/4496072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 msgr2=0x7f34fc165850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 --2- 172.21.15.165:0/4496072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc165850 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f34f4009e50 tx=0x7f34f402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 -- 172.21.15.165:0/4496072 shutdown_connections 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 --2- 172.21.15.165:0/4496072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc165850 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 --2- 172.21.15.165:0/4496072 >> v2:172.21.15.172:3300/0 conn(0x7f34fc0b3800 0x7f34fc06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 --2- 172.21.15.165:0/4496072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc15c100 0x7f34fc071740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 -- 172.21.15.165:0/4496072 >> 172.21.15.165:0/4496072 conn(0x7f34fc06abe0 msgr2=0x7f34fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 -- 172.21.15.165:0/4496072 shutdown_connections 2024-04-25T10:06:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.699+0000 7f3502400640 1 -- 172.21.15.165:0/4496072 wait complete. 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.701+0000 7f3502400640 1 Processor -- start 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.701+0000 7f3502400640 1 -- start start 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3502400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc1f7ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3502400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0b3800 0x7f34fc15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3502400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc15c100 0x7f34fc0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3502400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0b24e0 con 0x7f34fc06eac0 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3502400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0b30b0 con 0x7f34fc15c100 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3502400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34fc0711b0 con 0x7f34fc0b3800 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3501000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc1f7ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3501000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc1f7ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41638/0 (socket says 172.21.15.165:41638) 2024-04-25T10:06:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.702+0000 7f3501000640 1 -- 172.21.15.165:0/2841310004 learned_addr learned my addr 172.21.15.165:0/2841310004 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:14.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3501000640 1 -- 172.21.15.165:0/2841310004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0b3800 msgr2=0x7f34fc15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3501000640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0b3800 0x7f34fc15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3501000640 1 -- 172.21.15.165:0/2841310004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc15c100 msgr2=0x7f34fc0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3501000640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc15c100 0x7f34fc0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3501000640 1 -- 172.21.15.165:0/2841310004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34fc15d860 con 0x7f34fc06eac0 2024-04-25T10:06:14.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3501000640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc1f7ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c820 tx=0x7f34f000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f0015070 con 0x7f34fc06eac0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34f0004240 con 0x7f34fc06eac0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34f000ce10 con 0x7f34fc06eac0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34fc15db40 con 0x7f34fc06eac0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.703+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34fc1f8400 con 0x7f34fc06eac0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.704+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34fc073990 con 0x7f34fc06eac0 2024-04-25T10:06:14.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.705+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34f0007830 con 0x7f34fc06eac0 2024-04-25T10:06:14.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.706+0000 7f34f8c00640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34dc06b500 0x7f34dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:14.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.706+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f34f0089f40 con 0x7f34fc06eac0 2024-04-25T10:06:14.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.707+0000 7f34fbe00640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34dc06b500 0x7f34dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:14.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.707+0000 7f34fbe00640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34dc06b500 0x7f34dc06d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f34e8006fd0 tx=0x7f34e8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:14.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.711+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34f0058e90 con 0x7f34fc06eac0 2024-04-25T10:06:14.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.906+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f34fc0ae7f0 con 0x7f34fc06eac0 2024-04-25T10:06:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.908+0000 7f34f8c00640 1 -- 172.21.15.165:0/2841310004 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+13331 (secure 0 0 0) 0x7f34f0058790 con 0x7f34fc06eac0 2024-04-25T10:06:14.914 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:14.914 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":46,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:13.453167+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/3282481342":"2024-04-26T10:06:10.321094+0000","172.21.15.165:6836/3282481342":"2024-04-26T10:06:10.321094+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:14.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34dc06b500 msgr2=0x7f34dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34dc06b500 0x7f34dc06d9c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f34e8006fd0 tx=0x7f34e8008040 comp rx=0 tx=0).stop 2024-04-25T10:06:14.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 msgr2=0x7f34fc1f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:14.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc1f7ec0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f34f000c820 tx=0x7f34f000cce0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 shutdown_connections 2024-04-25T10:06:14.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34dc06b500 0x7f34dc06d9c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34fc15c100 0x7f34fc0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34fc0b3800 0x7f34fc15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 --2- 172.21.15.165:0/2841310004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34fc06eac0 0x7f34fc1f7ec0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 >> 172.21.15.165:0/2841310004 conn(0x7f34fc06abe0 msgr2=0x7f34fc0b43a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 shutdown_connections 2024-04-25T10:06:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:14.911+0000 7f3502400640 1 -- 172.21.15.165:0/2841310004 wait complete. 2024-04-25T10:06:14.934 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:14.935 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:15.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.130+0000 7fe695e00640 1 Processor -- start 2024-04-25T10:06:15.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.131+0000 7fe695e00640 1 -- start start 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.132+0000 7fe695e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe69015dc20 0x7fe6900ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.132+0000 7fe695e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe6900cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.132+0000 7fe695e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe69015e000 con 0x7fe690161710 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.132+0000 7fe695e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900d1de0 con 0x7fe69015dc20 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.132+0000 7fe695e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe690161c60 con 0x7fe6900d32f0 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68e000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe690161710 0x7fe69006bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51442/0 (socket says 172.21.15.165:51442) 2024-04-25T10:06:15.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68e000640 1 -- 172.21.15.165:0/859664240 learned_addr learned my addr 172.21.15.165:0/859664240 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:15.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68ea00640 1 --2- 172.21.15.165:0/859664240 >> v2:172.21.15.172:3300/0 conn(0x7fe69015dc20 0x7fe6900ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386436729 0 0) 0x7fe69015e000 con 0x7fe690161710 2024-04-25T10:06:15.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68f400640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe6900cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe674003520 con 0x7fe690161710 2024-04-25T10:06:15.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987038943 0 0) 0x7fe674003520 con 0x7fe690161710 2024-04-25T10:06:15.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 msgr2=0x7fe6900cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe6900cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 >> v2:172.21.15.172:3300/0 conn(0x7fe69015dc20 msgr2=0x7fe6900ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 --2- 172.21.15.165:0/859664240 >> v2:172.21.15.172:3300/0 conn(0x7fe69015dc20 0x7fe6900ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690161f40 con 0x7fe690161710 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.133+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe678001670 con 0x7fe690161710 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 >> v1:172.21.15.165:6789/0 conn(0x7fe690161710 legacy=0x7fe69006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6740039b0 0x7fe674003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe674004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe6740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900d1de0 con 0x7fe674004d40 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690161c60 con 0x7fe6740039b0 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67400ba70 con 0x7fe6900d32f0 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68f400640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe6740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68ea00640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6740039b0 0x7fe674003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.134+0000 7fe68e000640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe674004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.135+0000 7fe68f400640 1 -- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 msgr2=0x7fe674004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.135+0000 7fe68f400640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe674004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.135+0000 7fe68f400640 1 -- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6740039b0 msgr2=0x7fe674003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.135+0000 7fe68f400640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6740039b0 0x7fe674003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.135+0000 7fe68f400640 1 -- 172.21.15.165:0/859664240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674003520 con 0x7fe674004d40 2024-04-25T10:06:15.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.135+0000 7fe68e000640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900d32f0 0x7fe674004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe68f400640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe6740051c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe688007360 tx=0x7fe674005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1c4b0b189712fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688009be0 con 0x7fe674004d40 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688036600 con 0x7fe674004d40 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe68d600640 1 -- 172.21.15.165:0/859664240 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6880369d0 con 0x7fe674004d40 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe695e00640 1 -- 172.21.15.165:0/859664240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 msgr2=0x7fe6740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe695e00640 1 --2- 172.21.15.165:0/859664240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe6740051c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fe688007360 tx=0x7fe674005830 comp rx=0 tx=0).stop 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.136+0000 7fe68f400640 1 -- 172.21.15.165:0/859664240 reap_dead start 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.137+0000 7fe695e00640 1 -- 172.21.15.165:0/859664240 shutdown_connections 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.137+0000 7fe695e00640 1 -- 172.21.15.165:0/859664240 >> 172.21.15.165:0/859664240 conn(0x7fe6900d0420 msgr2=0x7fe6900d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.137+0000 7fe695e00640 1 -- 172.21.15.165:0/859664240 shutdown_connections 2024-04-25T10:06:15.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.137+0000 7fe695e00640 1 -- 172.21.15.165:0/859664240 wait complete. 2024-04-25T10:06:15.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.138+0000 7fe695e00640 1 Processor -- start 2024-04-25T10:06:15.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.139+0000 7fe695e00640 1 -- start start 2024-04-25T10:06:15.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe695e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe69015cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe695e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900cbac0 0x7fe6900cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe695e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6900aca60 0x7fe6900acee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe695e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900b5e50 con 0x7fe674004d40 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe695e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690162500 con 0x7fe6900aca60 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe695e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900b5fd0 con 0x7fe6900cbac0 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe69015cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe69015cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41660/0 (socket says 172.21.15.165:41660) 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 -- 172.21.15.165:0/3293247194 learned_addr learned my addr 172.21.15.165:0/3293247194 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 -- 172.21.15.165:0/3293247194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900cbac0 msgr2=0x7fe6900cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900cbac0 0x7fe6900cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 -- 172.21.15.165:0/3293247194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6900aca60 msgr2=0x7fe6900acee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6900aca60 0x7fe6900acee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.140+0000 7fe68ea00640 1 -- 172.21.15.165:0/3293247194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6901f3ba0 con 0x7fe674004d40 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.141+0000 7fe68ea00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe69015cda0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe68000f630 tx=0x7fe68000faf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.141+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680019020 con 0x7fe674004d40 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.141+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe680014870 con 0x7fe674004d40 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.141+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe680014c20 con 0x7fe674004d40 2024-04-25T10:06:15.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.141+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6900cc630 con 0x7fe674004d40 2024-04-25T10:06:15.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.141+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6900ccbf0 con 0x7fe674004d40 2024-04-25T10:06:15.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.142+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe690073a00 con 0x7fe674004d40 2024-04-25T10:06:15.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.143+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe68001c020 con 0x7fe674004d40 2024-04-25T10:06:15.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.144+0000 7fe687400640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe66c06b500 0x7fe66c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.144+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe68008a8a0 con 0x7fe674004d40 2024-04-25T10:06:15.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.144+0000 7fe68e000640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe66c06b500 0x7fe66c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.145+0000 7fe68e000640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe66c06b500 0x7fe66c06d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe678000ff0 tx=0x7fe678009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.149+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe680023020 con 0x7fe674004d40 2024-04-25T10:06:15.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.344+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe6900d16b0 con 0x7fe674004d40 2024-04-25T10:06:15.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.345+0000 7fe687400640 1 -- 172.21.15.165:0/3293247194 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe6800597f0 con 0x7fe674004d40 2024-04-25T10:06:15.350 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.347+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe66c06b500 msgr2=0x7fe66c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.347+0000 7fe695e00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe66c06b500 0x7fe66c06d9c0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fe678000ff0 tx=0x7fe678009040 comp rx=0 tx=0).stop 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.347+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 msgr2=0x7fe69015cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.347+0000 7fe695e00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe69015cda0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fe68000f630 tx=0x7fe68000faf0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 shutdown_connections 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe66c06b500 0x7fe66c06d9c0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6900aca60 0x7fe6900acee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe6900cbac0 0x7fe6900cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 --2- 172.21.15.165:0/3293247194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe674004d40 0x7fe69015cda0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 >> 172.21.15.165:0/3293247194 conn(0x7fe6900d0420 msgr2=0x7fe6900c9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 shutdown_connections 2024-04-25T10:06:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.348+0000 7fe695e00640 1 -- 172.21.15.165:0/3293247194 wait complete. 2024-04-25T10:06:15.372 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:06:15.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.568+0000 7f4072800640 1 Processor -- start 2024-04-25T10:06:15.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.569+0000 7f4072800640 1 -- start start 2024-04-25T10:06:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f4072800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15db70 0x7f406c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f4072800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f406c15df50 0x7f406c0cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f4072800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f406c0cec70 0x7f406c0cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f4072800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c0cbd80 con 0x7f406c0cec70 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f4072800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f406c161910 con 0x7f406c15df50 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f4072800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c071650 con 0x7f406c15db70 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.570+0000 7f406be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f406c0cec70 0x7f406c0cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f406c0cec70 0x7f406c0cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41674/0 (socket says 172.21.15.165:41674) 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 -- 172.21.15.165:0/2400481918 learned_addr learned my addr 172.21.15.165:0/2400481918 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f4062a00640 1 --2- 172.21.15.165:0/2400481918 >> v2:172.21.15.172:3300/0 conn(0x7f406c15df50 0x7f406c0cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 -- 172.21.15.165:0/2400481918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15db70 msgr2=0x7f406c0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406b400640 1 --2- 172.21.15.165:0/2400481918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15db70 0x7f406c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 --2- 172.21.15.165:0/2400481918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15db70 0x7f406c0cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 -- 172.21.15.165:0/2400481918 >> v2:172.21.15.172:3300/0 conn(0x7f406c15df50 msgr2=0x7f406c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 --2- 172.21.15.165:0/2400481918 >> v2:172.21.15.172:3300/0 conn(0x7f406c15df50 0x7f406c0cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f406be00640 1 -- 172.21.15.165:0/2400481918 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c0cf230 con 0x7f406c0cec70 2024-04-25T10:06:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.571+0000 7f4062a00640 1 --2- 172.21.15.165:0/2400481918 >> v2:172.21.15.172:3300/0 conn(0x7f406c15df50 0x7f406c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:15.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.572+0000 7f406be00640 1 --2- 172.21.15.165:0/2400481918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c0cfa50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f406c06bae0 tx=0x7f406402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90352303e0498a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.572+0000 7f406aa00640 1 -- 172.21.15.165:0/2400481918 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4064036070 con 0x7f406c0cec70 2024-04-25T10:06:15.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.572+0000 7f406aa00640 1 -- 172.21.15.165:0/2400481918 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4064031600 con 0x7f406c0cec70 2024-04-25T10:06:15.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.572+0000 7f406aa00640 1 -- 172.21.15.165:0/2400481918 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40640319d0 con 0x7f406c0cec70 2024-04-25T10:06:15.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 -- 172.21.15.165:0/2400481918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 msgr2=0x7f406c0cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 --2- 172.21.15.165:0/2400481918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c0cfa50 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f406c06bae0 tx=0x7f406402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 -- 172.21.15.165:0/2400481918 shutdown_connections 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 --2- 172.21.15.165:0/2400481918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c0cfa50 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 --2- 172.21.15.165:0/2400481918 >> v2:172.21.15.172:3300/0 conn(0x7f406c15df50 0x7f406c0cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 --2- 172.21.15.165:0/2400481918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15db70 0x7f406c0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 -- 172.21.15.165:0/2400481918 >> 172.21.15.165:0/2400481918 conn(0x7f406c06abe0 msgr2=0x7f406c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 -- 172.21.15.165:0/2400481918 shutdown_connections 2024-04-25T10:06:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.573+0000 7f4072800640 1 -- 172.21.15.165:0/2400481918 wait complete. 2024-04-25T10:06:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.575+0000 7f4072800640 1 Processor -- start 2024-04-25T10:06:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.575+0000 7f4072800640 1 -- start start 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f4072800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c15cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f4072800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f406c15db70 0x7f406c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f4072800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15df50 0x7f406c15aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f4072800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c162120 con 0x7f406c0cec70 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f4072800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f406c161e40 con 0x7f406c15db70 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f4072800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f406c162640 con 0x7f406c15df50 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f406b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c15cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f406b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c15cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41690/0 (socket says 172.21.15.165:41690) 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.576+0000 7f406b400640 1 -- 172.21.15.165:0/600985887 learned_addr learned my addr 172.21.15.165:0/600985887 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f406b400640 1 -- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15df50 msgr2=0x7f406c15aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f4062a00640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f406c15db70 0x7f406c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f406b400640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15df50 0x7f406c15aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f406b400640 1 -- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f406c15db70 msgr2=0x7f406c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f406b400640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f406c15db70 0x7f406c160830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f406b400640 1 -- 172.21.15.165:0/600985887 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f406c1f7500 con 0x7f406c0cec70 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f406b400640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c15cd70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c820 tx=0x7f405c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c015070 con 0x7f406c0cec70 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.577+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f405c004240 con 0x7f406c0cec70 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.578+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f406c1f77e0 con 0x7f406c0cec70 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.578+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f405c00ce10 con 0x7f406c0cec70 2024-04-25T10:06:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.578+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f406c1f7da0 con 0x7f406c0cec70 2024-04-25T10:06:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.579+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f406c073990 con 0x7f406c0cec70 2024-04-25T10:06:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.580+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f405c007830 con 0x7f406c0cec70 2024-04-25T10:06:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.580+0000 7f4063e00640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f404806b4d0 0x7f404806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.581+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f405c089fe0 con 0x7f406c0cec70 2024-04-25T10:06:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.581+0000 7f4062a00640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f404806b4d0 0x7f404806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.581+0000 7f4062a00640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f404806b4d0 0x7f404806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f406c1f6f70 tx=0x7f4054008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:15.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.585+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f405c058fd0 con 0x7f406c0cec70 2024-04-25T10:06:15.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:15.850+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7f406c071ef0 con 0x7f406c0cec70 2024-04-25T10:06:16.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.482+0000 7f4063e00640 1 -- 172.21.15.165:0/600985887 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v210) v1 ==== 146+0+0 (secure 0 0 0) 0x7f405c0588d0 con 0x7f406c0cec70 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f404806b4d0 msgr2=0x7f404806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f404806b4d0 0x7f404806d990 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f406c1f6f70 tx=0x7f4054008040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 msgr2=0x7f406c15cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c15cd70 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f405c00c820 tx=0x7f405c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 shutdown_connections 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f404806b4d0 0x7f404806d990 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f406c15df50 0x7f406c15aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f406c15db70 0x7f406c160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 --2- 172.21.15.165:0/600985887 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f406c0cec70 0x7f406c15cd70 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 >> 172.21.15.165:0/600985887 conn(0x7f406c06abe0 msgr2=0x7f406c0ae600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 shutdown_connections 2024-04-25T10:06:16.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.486+0000 7f4072800640 1 -- 172.21.15.165:0/600985887 wait complete. 2024-04-25T10:06:16.512 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:06:16.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.713+0000 7f6a62400640 1 Processor -- start 2024-04-25T10:06:16.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.713+0000 7f6a62400640 1 -- start start 2024-04-25T10:06:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a62400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6a5c0b95c0 0x7f6a5c0b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a62400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a5c0b5910 0x7f6a5c0d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a62400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a5c161e70 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a62400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d1ce0 con 0x7f6a5c0b95c0 2024-04-25T10:06:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a62400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5c162150 con 0x7f6a5c0b5910 2024-04-25T10:06:16.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a53e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6a5c0b95c0 0x7f6a5c0b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a61000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6a5c15dbd0 0x7f6a5c161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51456/0 (socket says 172.21.15.165:51456) 2024-04-25T10:06:16.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.715+0000 7f6a61000640 1 -- 172.21.15.165:0/2759111606 learned_addr learned my addr 172.21.15.165:0/2759111606 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:16.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a61a00640 1 --2- 172.21.15.165:0/2759111606 >> v2:172.21.15.172:3300/0 conn(0x7f6a5c0b5910 0x7f6a5c0d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a53e00640 1 -- 172.21.15.165:0/2759111606 >> v2:172.21.15.172:3300/0 conn(0x7f6a5c0b5910 msgr2=0x7f6a5c0d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:16.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a53e00640 1 --2- 172.21.15.165:0/2759111606 >> v2:172.21.15.172:3300/0 conn(0x7f6a5c0b5910 0x7f6a5c0d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a53e00640 1 -- 172.21.15.165:0/2759111606 >> v1:172.21.15.165:6789/0 conn(0x7f6a5c15dbd0 legacy=0x7f6a5c161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a5be00640 1 -- 172.21.15.165:0/2759111606 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 943219013 0 0) 0x7f6a5c161e70 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a53e00640 1 -- 172.21.15.165:0/2759111606 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c162430 con 0x7f6a5c0b95c0 2024-04-25T10:06:16.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.716+0000 7f6a61a00640 1 --2- 172.21.15.165:0/2759111606 >> v2:172.21.15.172:3300/0 conn(0x7f6a5c0b5910 0x7f6a5c0d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.717+0000 7f6a53e00640 1 --2- 172.21.15.165:0/2759111606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c0b95c0 0x7f6a5c0b53d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0b6850 tx=0x7f6a4402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84519d8c0338c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.717+0000 7f6a5be00640 1 -- 172.21.15.165:0/2759111606 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44037070 con 0x7f6a5c0b95c0 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.717+0000 7f6a5be00640 1 -- 172.21.15.165:0/2759111606 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a44032440 con 0x7f6a5c0b95c0 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.717+0000 7f6a5be00640 1 -- 172.21.15.165:0/2759111606 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a44032810 con 0x7f6a5c0b95c0 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 -- 172.21.15.165:0/2759111606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c0b95c0 msgr2=0x7f6a5c0b53d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 --2- 172.21.15.165:0/2759111606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c0b95c0 0x7f6a5c0b53d0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c0b6850 tx=0x7f6a4402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 -- 172.21.15.165:0/2759111606 shutdown_connections 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 --2- 172.21.15.165:0/2759111606 >> v2:172.21.15.172:3300/0 conn(0x7f6a5c0b5910 0x7f6a5c0d2170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 --2- 172.21.15.165:0/2759111606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c0b95c0 0x7f6a5c0b53d0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 -- 172.21.15.165:0/2759111606 >> 172.21.15.165:0/2759111606 conn(0x7f6a5c0d03d0 msgr2=0x7f6a5c0d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 -- 172.21.15.165:0/2759111606 shutdown_connections 2024-04-25T10:06:16.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.718+0000 7f6a62400640 1 -- 172.21.15.165:0/2759111606 wait complete. 2024-04-25T10:06:16.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.720+0000 7f6a62400640 1 Processor -- start 2024-04-25T10:06:16.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.720+0000 7f6a62400640 1 -- start start 2024-04-25T10:06:16.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a62400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c0b5910 0x7f6a5c1551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a62400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a5c0b95c0 0x7f6a5c15fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a62400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 0x7f6a5c0d3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a62400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d2830 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a62400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d26b0 con 0x7f6a5c0b5910 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a62400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c0d29b0 con 0x7f6a5c0b95c0 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 0x7f6a5c0d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 0x7f6a5c0d3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41708/0 (socket says 172.21.15.165:41708) 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61a00640 1 -- 172.21.15.165:0/1529066406 learned_addr learned my addr 172.21.15.165:0/1529066406 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61a00640 1 -- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a5c0b95c0 msgr2=0x7f6a5c15fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61a00640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a5c0b95c0 0x7f6a5c15fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61000640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c0b5910 0x7f6a5c1551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.721+0000 7f6a61a00640 1 -- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c0b5910 msgr2=0x7f6a5c1551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a61a00640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c0b5910 0x7f6a5c1551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a61a00640 1 -- 172.21.15.165:0/1529066406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0c7920 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a61000640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c0b5910 0x7f6a5c1551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:16.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a61a00640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 0x7f6a5c0d3950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ce40 tx=0x7f6a5400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5401a070 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5400d840 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400dc30 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5c0c7c00 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.722+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5c0c81c0 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.724+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a5c0d1520 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.725+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a5401d070 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.726+0000 7f6a59600640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:16.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.726+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6a5408a930 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.726+0000 7f6a61000640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:16.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.727+0000 7f6a61000640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a3806b4d0 0x7f6a3806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c000f60 tx=0x7f6a4c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:16.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.730+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a540598c0 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.929+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6a5c1592f0 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.934+0000 7f6a59600640 1 -- 172.21.15.165:0/1529066406 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a540591c0 con 0x7f6a5c15dbd0 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.937+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a3806b4d0 msgr2=0x7f6a3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.937+0000 7f6a62400640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a3806b4d0 0x7f6a3806d990 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c000f60 tx=0x7f6a4c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.937+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 msgr2=0x7f6a5c0d3950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.937+0000 7f6a62400640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 0x7f6a5c0d3950 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400ce40 tx=0x7f6a5400a710 comp rx=0 tx=0).stop 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 shutdown_connections 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a3806b4d0 0x7f6a3806d990 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c15dbd0 0x7f6a5c0d3950 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a5c0b95c0 0x7f6a5c15fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 --2- 172.21.15.165:0/1529066406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c0b5910 0x7f6a5c1551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 >> 172.21.15.165:0/1529066406 conn(0x7f6a5c0d03d0 msgr2=0x7f6a5c0b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 shutdown_connections 2024-04-25T10:06:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:16.938+0000 7f6a62400640 1 -- 172.21.15.165:0/1529066406 wait complete. 2024-04-25T10:06:16.963 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:06:16.963 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:17.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:06:17.180 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:06:17.180 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:17.180 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:17.242 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:17.242 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:06:17.242 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:17.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:17.300 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:17.300 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:17.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:06:17.358 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:06:17.359 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:17.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:06:17.417 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:06:17.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:17.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.622+0000 7f7bdbe00640 1 Processor -- start 2024-04-25T10:06:17.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.622+0000 7f7bdbe00640 1 -- start start 2024-04-25T10:06:17.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.624+0000 7f7bdbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7bdc15e900 0x7f7bdc0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.624+0000 7f7bdbe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bdc15ed90 con 0x7f7bdc0ae0d0 2024-04-25T10:06:17.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.624+0000 7f7bdbe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bdc0ae590 con 0x7f7bdc15e520 2024-04-25T10:06:17.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.624+0000 7f7bdbe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bdc0aee30 con 0x7f7bdc15e900 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.624+0000 7f7bd2000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7bdc15e900 0x7f7bdc0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.624+0000 7f7bdb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7bdc0ae0d0 0x7f7bdc070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60700/0 (socket says 172.21.15.165:60700) 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.625+0000 7f7bdb400640 1 -- 172.21.15.165:0/1174988391 learned_addr learned my addr 172.21.15.165:0/1174988391 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.625+0000 7f7bd2000640 1 -- 172.21.15.165:0/1174988391 >> v1:172.21.15.172:6789/0 conn(0x7f7bdc15e520 legacy=0x7f7bdc15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.625+0000 7f7bd2000640 1 -- 172.21.15.165:0/1174988391 >> v1:172.21.15.165:6789/0 conn(0x7f7bdc0ae0d0 legacy=0x7f7bdc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.625+0000 7f7bda000640 1 -- 172.21.15.165:0/1174988391 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793602448 0 0) 0x7f7bdc15ed90 con 0x7f7bdc0ae0d0 2024-04-25T10:06:17.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.625+0000 7f7bd2000640 1 -- 172.21.15.165:0/1174988391 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0ae870 con 0x7f7bdc15e900 2024-04-25T10:06:17.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.626+0000 7f7bd2000640 1 --2- 172.21.15.165:0/1174988391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e900 0x7f7bdc0adb90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40097a0 tx=0x7f7bc402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=820e98ed107bcc75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.626+0000 7f7bda000640 1 -- 172.21.15.165:0/1174988391 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4037070 con 0x7f7bdc15e900 2024-04-25T10:06:17.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.626+0000 7f7bda000640 1 -- 172.21.15.165:0/1174988391 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc4032440 con 0x7f7bdc15e900 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.626+0000 7f7bda000640 1 -- 172.21.15.165:0/1174988391 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4032810 con 0x7f7bdc15e900 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 -- 172.21.15.165:0/1174988391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e900 msgr2=0x7f7bdc0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/1174988391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e900 0x7f7bdc0adb90 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40097a0 tx=0x7f7bc402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 -- 172.21.15.165:0/1174988391 shutdown_connections 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/1174988391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e900 0x7f7bdc0adb90 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 -- 172.21.15.165:0/1174988391 >> 172.21.15.165:0/1174988391 conn(0x7f7bdc06abe0 msgr2=0x7f7bdc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 -- 172.21.15.165:0/1174988391 shutdown_connections 2024-04-25T10:06:17.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.627+0000 7f7bdbe00640 1 -- 172.21.15.165:0/1174988391 wait complete. 2024-04-25T10:06:17.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.629+0000 7f7bdbe00640 1 Processor -- start 2024-04-25T10:06:17.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.629+0000 7f7bdbe00640 1 -- start start 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bdc0ae0d0 0x7f7bdc0b42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 0x7f7bdc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bdbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bdc15e900 0x7f7bdc144940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bdbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc071700 con 0x7f7bdc15e520 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bdbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc06d1f0 con 0x7f7bdc15e900 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bdbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bdc071880 con 0x7f7bdc0ae0d0 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 0x7f7bdc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 0x7f7bdc15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41576/0 (socket says 172.21.15.165:41576) 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 -- 172.21.15.165:0/4153468512 learned_addr learned my addr 172.21.15.165:0/4153468512 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 -- 172.21.15.165:0/4153468512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bdc0ae0d0 msgr2=0x7f7bdc0b42b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bdc0ae0d0 0x7f7bdc0b42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 -- 172.21.15.165:0/4153468512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bdc15e900 msgr2=0x7f7bdc144940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bdc15e900 0x7f7bdc144940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.630+0000 7f7bd2000640 1 -- 172.21.15.165:0/4153468512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc0b47f0 con 0x7f7bdc15e520 2024-04-25T10:06:17.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.631+0000 7f7bd2000640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 0x7f7bdc15d320 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40097a0 tx=0x7f7bc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.631+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4037040 con 0x7f7bdc15e520 2024-04-25T10:06:17.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.631+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc4004250 con 0x7f7bdc15e520 2024-04-25T10:06:17.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.631+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4032e20 con 0x7f7bdc15e520 2024-04-25T10:06:17.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.631+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bdc0b11c0 con 0x7f7bdc15e520 2024-04-25T10:06:17.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.631+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bdc0b37d0 con 0x7f7bdc15e520 2024-04-25T10:06:17.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.633+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bdc073990 con 0x7f7bdc15e520 2024-04-25T10:06:17.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.633+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bc4030770 con 0x7f7bdc15e520 2024-04-25T10:06:17.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.634+0000 7f7bd3e00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb806b4d0 0x7f7bb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:17.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.634+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f7bc40ab3e0 con 0x7f7bdc15e520 2024-04-25T10:06:17.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.635+0000 7f7bdaa00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb806b4d0 0x7f7bb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:17.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.635+0000 7f7bdaa00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb806b4d0 0x7f7bb806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0cb5b0 tx=0x7f7bcc0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:17.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.639+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bc407a330 con 0x7f7bdc15e520 2024-04-25T10:06:17.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.877+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bdc06bff0 con 0x7f7bdc15e520 2024-04-25T10:06:17.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.878+0000 7f7bd3e00640 1 -- 172.21.15.165:0/4153468512 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+3708 (secure 0 0 0) 0x7f7bc4079c30 con 0x7f7bdc15e520 2024-04-25T10:06:17.884 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:17.884 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4870,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/849782933","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":849782933},{"type":"v1","addr":"172.21.15.172:6834","nonce":849782933}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":12},{"gid":4873,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3789196358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3789196358},{"type":"v1","addr":"172.21.15.172:6835","nonce":3789196358}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1736114136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1736114136},{"type":"v1","addr":"172.21.15.165:6835","nonce":1736114136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":13},{"gid":5715,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1535981436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":1535981436},{"type":"v1","addr":"172.21.15.165:6837","nonce":1535981436}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":22}],"filesystems":[]} 2024-04-25T10:06:17.884 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 22 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb806b4d0 msgr2=0x7f7bb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb806b4d0 0x7f7bb806d990 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7bdc0cb5b0 tx=0x7f7bcc0079b0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 msgr2=0x7f7bdc15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 0x7f7bdc15d320 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f7bc40097a0 tx=0x7f7bc40046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 shutdown_connections 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb806b4d0 0x7f7bb806d990 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bdc15e900 0x7f7bdc144940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bdc15e520 0x7f7bdc15d320 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 --2- 172.21.15.165:0/4153468512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bdc0ae0d0 0x7f7bdc0b42b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 >> 172.21.15.165:0/4153468512 conn(0x7f7bdc06abe0 msgr2=0x7f7bdc0b0400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.882+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 shutdown_connections 2024-04-25T10:06:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:17.883+0000 7f7bdbe00640 1 -- 172.21.15.165:0/4153468512 wait complete. 2024-04-25T10:06:17.905 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:06:17.906 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:06:17.906 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:06:17.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:17.906 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:06:17.906 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:06:17.907 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:06:17.907 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:17.907 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:06:17.907 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:06:17.907 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:06:17.907 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:17.907 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:06:17.907 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:06:17.908 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:06:17.908 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:06:17.929 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:06:17.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:06:17.930 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:06:17.930 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:06:17.931 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:06:17.932 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:06:17.941 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:06:17.941 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:06:17.975 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:06:17.975 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:17.981 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:06:17.981 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:17.988 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:06:17.989 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:17.995 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:06:17.995 DEBUG:teuthology.parallel:result is None 2024-04-25T10:06:17.996 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:06:18.124 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:06:18.127 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:06:18.151 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:06:18.151 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:06:18.153 INFO:tasks.ceph.mds.b.smithi172.stderr:2024-04-25T10:06:18.148+0000 7fa933200640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.170+0000 7fd469400640 1 Processor -- start 2024-04-25T10:06:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.170+0000 7fd469400640 1 -- start start 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd469400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd46415b710 0x7fd4640cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd469400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd46406ec50 0x7fd46415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd469400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd46406f0e0 con 0x7fd46415ee60 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd469400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd46415e490 con 0x7fd46406ec50 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd469400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4640cfd80 con 0x7fd46415b710 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd461e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd46406ec50 0x7fd46415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd462800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd46415b710 0x7fd4640cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd463200640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fd46415ee60 0x7fd4640ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:43040/0 (socket says 172.21.15.165:43040) 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd462800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd46415b710 0x7fd4640cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:59902/0 (socket says 172.21.15.165:59902) 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.171+0000 7fd463200640 1 -- 172.21.15.165:0/1944977518 learned_addr learned my addr 172.21.15.165:0/1944977518 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.172+0000 7fd462800640 1 -- 172.21.15.165:0/1944977518 >> v2:172.21.15.165:3300/0 conn(0x7fd46406ec50 msgr2=0x7fd46415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.172+0000 7fd462800640 1 --2- 172.21.15.165:0/1944977518 >> v2:172.21.15.165:3300/0 conn(0x7fd46406ec50 0x7fd46415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.172+0000 7fd462800640 1 -- 172.21.15.165:0/1944977518 >> v1:172.21.15.172:6789/0 conn(0x7fd46415ee60 legacy=0x7fd4640ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.172+0000 7fd462800640 1 -- 172.21.15.165:0/1944977518 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4640d0060 con 0x7fd46415b710 2024-04-25T10:06:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.172+0000 7fd461400640 1 -- 172.21.15.165:0/1944977518 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3206220766 0 0) 0x7fd46406f0e0 con 0x7fd46415ee60 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.173+0000 7fd462800640 1 --2- 172.21.15.165:0/1944977518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46415b710 0x7fd4640cf640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd46406b950 tx=0x7fd45402a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9cf9ee90455553d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.173+0000 7fd461400640 1 -- 172.21.15.165:0/1944977518 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454037070 con 0x7fd46415b710 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.173+0000 7fd461400640 1 -- 172.21.15.165:0/1944977518 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd454032440 con 0x7fd46415b710 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.173+0000 7fd461400640 1 -- 172.21.15.165:0/1944977518 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454032810 con 0x7fd46415b710 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 -- 172.21.15.165:0/1944977518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46415b710 msgr2=0x7fd4640cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 --2- 172.21.15.165:0/1944977518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46415b710 0x7fd4640cf640 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fd46406b950 tx=0x7fd45402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 -- 172.21.15.165:0/1944977518 shutdown_connections 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 --2- 172.21.15.165:0/1944977518 >> v2:172.21.15.165:3300/0 conn(0x7fd46406ec50 0x7fd46415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 --2- 172.21.15.165:0/1944977518 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46415b710 0x7fd4640cf640 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 -- 172.21.15.165:0/1944977518 >> 172.21.15.165:0/1944977518 conn(0x7fd46406abe0 msgr2=0x7fd46406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 -- 172.21.15.165:0/1944977518 shutdown_connections 2024-04-25T10:06:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.174+0000 7fd469400640 1 -- 172.21.15.165:0/1944977518 wait complete. 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.175+0000 7fd469400640 1 Processor -- start 2024-04-25T10:06:18.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.176+0000 7fd469400640 1 -- start start 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.176+0000 7fd469400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46406ec50 0x7fd464155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd469400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd46415b710 0x7fd4641f33e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd469400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 0x7fd46415d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd469400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46415bc70 con 0x7fd46415ee60 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd469400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd46415baf0 con 0x7fd46406ec50 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd469400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640acfc0 con 0x7fd46415b710 2024-04-25T10:06:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 0x7fd46415d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 0x7fd46415d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41616/0 (socket says 172.21.15.165:41616) 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 -- 172.21.15.165:0/475644346 learned_addr learned my addr 172.21.15.165:0/475644346 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd462800640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46406ec50 0x7fd464155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd461e00640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd46415b710 0x7fd4641f33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 -- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd46415b710 msgr2=0x7fd4641f33e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd46415b710 0x7fd4641f33e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 -- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46406ec50 msgr2=0x7fd464155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46406ec50 0x7fd464155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.177+0000 7fd463200640 1 -- 172.21.15.165:0/475644346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4641f3d50 con 0x7fd46415ee60 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.178+0000 7fd463200640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 0x7fd46415d360 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c780 tx=0x7fd45c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.178+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c018070 con 0x7fd46415ee60 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.178+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd45c00a640 con 0x7fd46415ee60 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.178+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4641f4030 con 0x7fd46415ee60 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.178+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45c00aa30 con 0x7fd46415ee60 2024-04-25T10:06:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.178+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4641f4640 con 0x7fd46415ee60 2024-04-25T10:06:18.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.179+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd464073a00 con 0x7fd46415ee60 2024-04-25T10:06:18.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.180+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd45c01b070 con 0x7fd46415ee60 2024-04-25T10:06:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.181+0000 7fd45aa00640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd44406b4d0 0x7fd44406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.181+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd45c089630 con 0x7fd46415ee60 2024-04-25T10:06:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.181+0000 7fd462800640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd44406b4d0 0x7fd44406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.182+0000 7fd462800640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd44406b4d0 0x7fd44406d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd454034000 tx=0x7fd45402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.185+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd45c057fc0 con 0x7fd46415ee60 2024-04-25T10:06:18.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.392+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd4640ccd20 con 0x7fd46415ee60 2024-04-25T10:06:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.512+0000 7fd45aa00640 1 -- 172.21.15.165:0/475644346 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v48) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd45c05c060 con 0x7fd46415ee60 2024-04-25T10:06:18.518 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:06:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd44406b4d0 msgr2=0x7fd44406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd44406b4d0 0x7fd44406d990 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fd454034000 tx=0x7fd45402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 msgr2=0x7fd46415d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 0x7fd46415d360 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd45c00c780 tx=0x7fd45c00cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 shutdown_connections 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd44406b4d0 0x7fd44406d990 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd46415ee60 0x7fd46415d360 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd46415b710 0x7fd4641f33e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 --2- 172.21.15.165:0/475644346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd46406ec50 0x7fd464155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 >> 172.21.15.165:0/475644346 conn(0x7fd46406abe0 msgr2=0x7fd46415a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 shutdown_connections 2024-04-25T10:06:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.516+0000 7fd469400640 1 -- 172.21.15.165:0/475644346 wait complete. 2024-04-25T10:06:18.541 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:06:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.728+0000 7fa01be00640 1 Processor -- start 2024-04-25T10:06:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.728+0000 7fa01be00640 1 -- start start 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.729+0000 7fa01be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa01406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.729+0000 7fa01be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0140cc930 0x7fa01415c910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.729+0000 7fa01be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa0140ce950 0x7fa0140ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.729+0000 7fa01be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01406d850 con 0x7fa0140ce950 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa01be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa01415c150 con 0x7fa0140cc930 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa01be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140cd4b0 con 0x7fa014144d40 2024-04-25T10:06:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa0140ce950 0x7fa0140ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa0140ce950 0x7fa0140ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41624/0 (socket says 172.21.15.165:41624) 2024-04-25T10:06:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 -- 172.21.15.165:0/1316037232 learned_addr learned my addr 172.21.15.165:0/1316037232 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019400640 1 --2- 172.21.15.165:0/1316037232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa01406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa018a00640 1 --2- 172.21.15.165:0/1316037232 >> v2:172.21.15.172:3300/0 conn(0x7fa0140cc930 0x7fa01415c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 -- 172.21.15.165:0/1316037232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 msgr2=0x7fa01406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 --2- 172.21.15.165:0/1316037232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa01406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 -- 172.21.15.165:0/1316037232 >> v2:172.21.15.172:3300/0 conn(0x7fa0140cc930 msgr2=0x7fa01415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:18.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 --2- 172.21.15.165:0/1316037232 >> v2:172.21.15.172:3300/0 conn(0x7fa0140cc930 0x7fa01415c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.730+0000 7fa019e00640 1 -- 172.21.15.165:0/1316037232 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140ccef0 con 0x7fa0140ce950 2024-04-25T10:06:18.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.732+0000 7fa019e00640 1 --2- 172.21.15.165:0/1316037232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140ce950 0x7fa0140ceda0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa01406d1e0 tx=0x7fa00c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae229517084c9d75 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.732+0000 7fa013e00640 1 -- 172.21.15.165:0/1316037232 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c036070 con 0x7fa0140ce950 2024-04-25T10:06:18.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.732+0000 7fa013e00640 1 -- 172.21.15.165:0/1316037232 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c031600 con 0x7fa0140ce950 2024-04-25T10:06:18.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.732+0000 7fa013e00640 1 -- 172.21.15.165:0/1316037232 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c0319d0 con 0x7fa0140ce950 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 -- 172.21.15.165:0/1316037232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140ce950 msgr2=0x7fa0140ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 --2- 172.21.15.165:0/1316037232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140ce950 0x7fa0140ceda0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa01406d1e0 tx=0x7fa00c02a610 comp rx=0 tx=0).stop 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 -- 172.21.15.165:0/1316037232 shutdown_connections 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 --2- 172.21.15.165:0/1316037232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140ce950 0x7fa0140ceda0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 --2- 172.21.15.165:0/1316037232 >> v2:172.21.15.172:3300/0 conn(0x7fa0140cc930 0x7fa01415c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 --2- 172.21.15.165:0/1316037232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa01406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 -- 172.21.15.165:0/1316037232 >> 172.21.15.165:0/1316037232 conn(0x7fa01406abe0 msgr2=0x7fa01406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 -- 172.21.15.165:0/1316037232 shutdown_connections 2024-04-25T10:06:18.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.733+0000 7fa01be00640 1 -- 172.21.15.165:0/1316037232 wait complete. 2024-04-25T10:06:18.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.735+0000 7fa01be00640 1 Processor -- start 2024-04-25T10:06:18.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.735+0000 7fa01be00640 1 -- start start 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa01be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 0x7fa01415e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa01be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0140ce950 0x7fa01415af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa01be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa0140a96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa01be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01415ce50 con 0x7fa0140cc930 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa01be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa01415c3f0 con 0x7fa0140ce950 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa01be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0140cf2e0 con 0x7fa014144d40 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa019400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 0x7fa01415e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.736+0000 7fa019400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 0x7fa01415e7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41636/0 (socket says 172.21.15.165:41636) 2024-04-25T10:06:18.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 -- 172.21.15.165:0/273901203 learned_addr learned my addr 172.21.15.165:0/273901203 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:18.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 -- 172.21.15.165:0/273901203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 msgr2=0x7fa0140a96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa0140a96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 -- 172.21.15.165:0/273901203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0140ce950 msgr2=0x7fa01415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0140ce950 0x7fa01415af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 -- 172.21.15.165:0/273901203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0140a8640 con 0x7fa0140cc930 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa019400640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 0x7fa01415e7b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d5f0 tx=0x7fa00800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080125f0 con 0x7fa0140cc930 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa008019070 con 0x7fa0140cc930 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.737+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0080129b0 con 0x7fa0140cc930 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.738+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0140a8920 con 0x7fa0140cc930 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.738+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0140a8ee0 con 0x7fa0140cc930 2024-04-25T10:06:18.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.739+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa014073a00 con 0x7fa0140cc930 2024-04-25T10:06:18.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.740+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa008012b50 con 0x7fa0140cc930 2024-04-25T10:06:18.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.741+0000 7fa011600640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9ff406b500 0x7f9ff406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:18.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.741+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa00808a4f0 con 0x7fa0140cc930 2024-04-25T10:06:18.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.741+0000 7fa018a00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9ff406b500 0x7f9ff406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:18.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.741+0000 7fa018a00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9ff406b500 0x7f9ff406d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa014143520 tx=0x7fa000008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:18.745+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa008059490 con 0x7fa0140cc930 2024-04-25T10:06:19.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.010+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa01406b060 con 0x7fa0140cc930 2024-04-25T10:06:19.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.011+0000 7fa011600640 1 -- 172.21.15.165:0/273901203 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa008058d90 con 0x7fa0140cc930 2024-04-25T10:06:19.016 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.017 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:06:19.018 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:06:19.019 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:06:19.020 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.021 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.022 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.023 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.024 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:06:19.026 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.027 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:06:19.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:06:19.029 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:19.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.013+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9ff406b500 msgr2=0x7f9ff406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9ff406b500 0x7f9ff406d9c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fa014143520 tx=0x7fa000008040 comp rx=0 tx=0).stop 2024-04-25T10:06:19.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 msgr2=0x7fa01415e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 0x7fa01415e7b0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa00800d5f0 tx=0x7fa00800dab0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 shutdown_connections 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9ff406b500 0x7f9ff406d9c0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa014144d40 0x7fa0140a96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0140ce950 0x7fa01415af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 --2- 172.21.15.165:0/273901203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0140cc930 0x7fa01415e7b0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 >> 172.21.15.165:0/273901203 conn(0x7fa01406abe0 msgr2=0x7fa0140b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 shutdown_connections 2024-04-25T10:06:19.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.014+0000 7fa01be00640 1 -- 172.21.15.165:0/273901203 wait complete. 2024-04-25T10:06:19.037 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.233+0000 7ff185400640 1 Processor -- start 2024-04-25T10:06:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.233+0000 7ff185400640 1 -- start start 2024-04-25T10:06:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.234+0000 7ff185400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff18015db70 0x7ff1800cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff185400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 0x7ff1800cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff185400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff180071650 con 0x7ff1800cec70 2024-04-25T10:06:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff185400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1800cbd80 con 0x7ff18015df50 2024-04-25T10:06:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff185400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff180161910 con 0x7ff18015db70 2024-04-25T10:06:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff17de00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 0x7ff1800cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff17e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff18015db70 0x7ff1800cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff17de00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 0x7ff1800cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:59936/0 (socket says 172.21.15.165:59936) 2024-04-25T10:06:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff17de00640 1 -- 172.21.15.165:0/488949192 learned_addr learned my addr 172.21.15.165:0/488949192 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.235+0000 7ff17f200640 1 --1- 172.21.15.165:0/488949192 >> v1:172.21.15.172:6789/0 conn(0x7ff1800cec70 0x7ff1800cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:43048/0 (socket says 172.21.15.165:43048) 2024-04-25T10:06:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.236+0000 7ff17e800640 1 -- 172.21.15.165:0/488949192 >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 msgr2=0x7ff1800cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.236+0000 7ff17e800640 1 --2- 172.21.15.165:0/488949192 >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 0x7ff1800cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.236+0000 7ff17e800640 1 -- 172.21.15.165:0/488949192 >> v1:172.21.15.172:6789/0 conn(0x7ff1800cec70 legacy=0x7ff1800cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.236+0000 7ff17e800640 1 -- 172.21.15.165:0/488949192 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1800cf230 con 0x7ff18015db70 2024-04-25T10:06:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.236+0000 7ff17de00640 1 --2- 172.21.15.165:0/488949192 >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 0x7ff1800cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.238+0000 7ff17e800640 1 --2- 172.21.15.165:0/488949192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff18015db70 0x7ff1800cb840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff1700097a0 tx=0x7ff17002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be3d1082bd1c9f32 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.238+0000 7ff17d400640 1 -- 172.21.15.165:0/488949192 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170037070 con 0x7ff18015db70 2024-04-25T10:06:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff17d400640 1 -- 172.21.15.165:0/488949192 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff170032440 con 0x7ff18015db70 2024-04-25T10:06:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff17d400640 1 -- 172.21.15.165:0/488949192 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff170032810 con 0x7ff18015db70 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 -- 172.21.15.165:0/488949192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff18015db70 msgr2=0x7ff1800cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 --2- 172.21.15.165:0/488949192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff18015db70 0x7ff1800cb840 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff1700097a0 tx=0x7ff17002a180 comp rx=0 tx=0).stop 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 -- 172.21.15.165:0/488949192 shutdown_connections 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 --2- 172.21.15.165:0/488949192 >> v2:172.21.15.172:3300/0 conn(0x7ff18015df50 0x7ff1800cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 --2- 172.21.15.165:0/488949192 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff18015db70 0x7ff1800cb840 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 -- 172.21.15.165:0/488949192 >> 172.21.15.165:0/488949192 conn(0x7ff18006abe0 msgr2=0x7ff18006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 -- 172.21.15.165:0/488949192 shutdown_connections 2024-04-25T10:06:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.239+0000 7ff185400640 1 -- 172.21.15.165:0/488949192 wait complete. 2024-04-25T10:06:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.241+0000 7ff185400640 1 Processor -- start 2024-04-25T10:06:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.241+0000 7ff185400640 1 -- start start 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.242+0000 7ff185400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1800cec70 0x7ff1801410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.242+0000 7ff185400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 0x7ff1801543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.242+0000 7ff185400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff18015df50 0x7ff1801548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.242+0000 7ff185400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff180162120 con 0x7ff18015db70 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.242+0000 7ff185400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff180161e40 con 0x7ff18015df50 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.242+0000 7ff185400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff180162640 con 0x7ff1800cec70 2024-04-25T10:06:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 0x7ff1801543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 0x7ff1801543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41640/0 (socket says 172.21.15.165:41640) 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 -- 172.21.15.165:0/530322211 learned_addr learned my addr 172.21.15.165:0/530322211 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17e800640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1800cec70 0x7ff1801410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17f200640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff18015df50 0x7ff1801548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 -- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1800cec70 msgr2=0x7ff1801410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1800cec70 0x7ff1801410b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 -- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff18015df50 msgr2=0x7ff1801548e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff18015df50 0x7ff1801548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 -- 172.21.15.165:0/530322211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18014e690 con 0x7ff18015db70 2024-04-25T10:06:19.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.243+0000 7ff17de00640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 0x7ff1801543a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e9b0 tx=0x7ff16c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.244+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c017070 con 0x7ff18015db70 2024-04-25T10:06:19.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.244+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c00c9a0 con 0x7ff18015db70 2024-04-25T10:06:19.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.244+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff180151fc0 con 0x7ff18015db70 2024-04-25T10:06:19.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.244+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00cd50 con 0x7ff18015db70 2024-04-25T10:06:19.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.244+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff180150160 con 0x7ff18015db70 2024-04-25T10:06:19.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.245+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18014fc90 con 0x7ff18015db70 2024-04-25T10:06:19.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.246+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff16c01a070 con 0x7ff18015db70 2024-04-25T10:06:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.247+0000 7ff176a00640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff16006b4d0 0x7ff16006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.247+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff16c08a2a0 con 0x7ff18015db70 2024-04-25T10:06:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.247+0000 7ff17e800640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff16006b4d0 0x7ff16006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.248+0000 7ff17e800640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff16006b4d0 0x7ff16006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff170034000 tx=0x7ff17002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.252+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff16c0592d0 con 0x7ff18015db70 2024-04-25T10:06:19.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.447+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff180073990 con 0x7ff18015db70 2024-04-25T10:06:19.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.448+0000 7ff176a00640 1 -- 172.21.15.165:0/530322211 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13200 (secure 0 0 0) 0x7ff16c058bd0 con 0x7ff18015db70 2024-04-25T10:06:19.454 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:19.454 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":50,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:18.530694+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:19.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff16006b4d0 msgr2=0x7ff16006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff16006b4d0 0x7ff16006d990 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7ff170034000 tx=0x7ff17002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 msgr2=0x7ff1801543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 0x7ff1801543a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00e9b0 tx=0x7ff16c00ee70 comp rx=0 tx=0).stop 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 shutdown_connections 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff16006b4d0 0x7ff16006d990 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff18015df50 0x7ff1801548e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff18015db70 0x7ff1801543a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 --2- 172.21.15.165:0/530322211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1800cec70 0x7ff1801410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 >> 172.21.15.165:0/530322211 conn(0x7ff18006abe0 msgr2=0x7ff1800ae390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 shutdown_connections 2024-04-25T10:06:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.451+0000 7ff185400640 1 -- 172.21.15.165:0/530322211 wait complete. 2024-04-25T10:06:19.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:06:19.475 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:06:19.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.671+0000 7fd9c1000640 1 Processor -- start 2024-04-25T10:06:19.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.671+0000 7fd9c1000640 1 -- start start 2024-04-25T10:06:19.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9c1000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd9bc1446d0 0x7fd9bc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9c1000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9c1000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9bc0cccc0 con 0x7fd9bc144d50 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9c1000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9bc0b2b20 con 0x7fd9bc1446d0 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9c1000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc06d400 con 0x7fd9bc06e8a0 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9b9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd9bc1446d0 0x7fd9bc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9b9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd9bc1446d0 0x7fd9bc0b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41650/0 (socket says 172.21.15.165:41650) 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.673+0000 7fd9b9e00640 1 -- 172.21.15.165:0/1634354633 learned_addr learned my addr 172.21.15.165:0/1634354633 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:19.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.674+0000 7fd9ba800640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.674+0000 7fd9b9e00640 1 -- 172.21.15.165:0/1634354633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 msgr2=0x7fd9bc0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.674+0000 7fd9b9e00640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.674+0000 7fd9b9e00640 1 -- 172.21.15.165:0/1634354633 >> v1:172.21.15.172:6789/0 conn(0x7fd9bc144d50 legacy=0x7fd9bc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:19.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.674+0000 7fd9b9e00640 1 -- 172.21.15.165:0/1634354633 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc0cc140 con 0x7fd9bc1446d0 2024-04-25T10:06:19.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.674+0000 7fd9ba800640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:19.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.675+0000 7fd9b9e00640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0b3f20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009870 tx=0x7fd9ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1fe78524b24152e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.675+0000 7fd9b8a00640 1 -- 172.21.15.165:0/1634354633 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac037070 con 0x7fd9bc1446d0 2024-04-25T10:06:19.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.675+0000 7fd9b8a00640 1 -- 172.21.15.165:0/1634354633 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac032440 con 0x7fd9bc1446d0 2024-04-25T10:06:19.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.675+0000 7fd9b8a00640 1 -- 172.21.15.165:0/1634354633 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac032810 con 0x7fd9bc1446d0 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 -- 172.21.15.165:0/1634354633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 msgr2=0x7fd9bc0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0b3f20 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac009870 tx=0x7fd9ac02a210 comp rx=0 tx=0).stop 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 -- 172.21.15.165:0/1634354633 shutdown_connections 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 --2- 172.21.15.165:0/1634354633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0b3f20 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 -- 172.21.15.165:0/1634354633 >> 172.21.15.165:0/1634354633 conn(0x7fd9bc06abe0 msgr2=0x7fd9bc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 -- 172.21.15.165:0/1634354633 shutdown_connections 2024-04-25T10:06:19.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.676+0000 7fd9c1000640 1 -- 172.21.15.165:0/1634354633 wait complete. 2024-04-25T10:06:19.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.678+0000 7fd9c1000640 1 Processor -- start 2024-04-25T10:06:19.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.678+0000 7fd9c1000640 1 -- start start 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9c1000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc1f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9c1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9c1000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc144d50 0x7fd9bc0ad400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9c1000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0cc700 con 0x7fd9bc1446d0 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9c1000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0b49a0 con 0x7fd9bc144d50 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9c1000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9bc0cc9e0 con 0x7fd9bc06e8a0 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9b9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9b9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41652/0 (socket says 172.21.15.165:41652) 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9b9400640 1 -- 172.21.15.165:0/2632903207 learned_addr learned my addr 172.21.15.165:0/2632903207 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9b9e00640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc1f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.679+0000 7fd9b9400640 1 -- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 msgr2=0x7fd9bc1f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b9400640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc1f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b9400640 1 -- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc144d50 msgr2=0x7fd9bc0ad400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9ba800640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc144d50 0x7fd9bc0ad400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b9400640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc144d50 0x7fd9bc0ad400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b9400640 1 -- 172.21.15.165:0/2632903207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9bc1f7190 con 0x7fd9bc1446d0 2024-04-25T10:06:19.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b9400640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0c7210 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b3f0 tx=0x7fd9a800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8015020 con 0x7fd9bc1446d0 2024-04-25T10:06:19.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a8011830 con 0x7fd9bc1446d0 2024-04-25T10:06:19.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a8011c20 con 0x7fd9bc1446d0 2024-04-25T10:06:19.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9bc1f7470 con 0x7fd9bc1446d0 2024-04-25T10:06:19.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.680+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9bc1f7a30 con 0x7fd9bc1446d0 2024-04-25T10:06:19.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.681+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9bc073a00 con 0x7fd9bc1446d0 2024-04-25T10:06:19.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.682+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9a8018020 con 0x7fd9bc1446d0 2024-04-25T10:06:19.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.683+0000 7fd9b2000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:19.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.683+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd9a8089ce0 con 0x7fd9bc1446d0 2024-04-25T10:06:19.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.684+0000 7fd9b9e00640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:19.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.684+0000 7fd9b9e00640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac034000 tx=0x7fd9ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:19.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.688+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9a801f030 con 0x7fd9bc1446d0 2024-04-25T10:06:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.886+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9bc06c250 con 0x7fd9bc1446d0 2024-04-25T10:06:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.887+0000 7fd9b2000640 1 -- 172.21.15.165:0/2632903207 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9a80172c0 con 0x7fd9bc1446d0 2024-04-25T10:06:19.893 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 msgr2=0x7fd99c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac034000 tx=0x7fd9ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 msgr2=0x7fd9bc0c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0c7210 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fd9a800b3f0 tx=0x7fd9a800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 shutdown_connections 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9bc144d50 0x7fd9bc0ad400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9bc1446d0 0x7fd9bc0c7210 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 --2- 172.21.15.165:0/2632903207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9bc06e8a0 0x7fd9bc1f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.890+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 >> 172.21.15.165:0/2632903207 conn(0x7fd9bc06abe0 msgr2=0x7fd9bc140d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.891+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 shutdown_connections 2024-04-25T10:06:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:19.891+0000 7fd9c1000640 1 -- 172.21.15.165:0/2632903207 wait complete. 2024-04-25T10:06:19.915 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:06:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.112+0000 7fcdea600640 1 Processor -- start 2024-04-25T10:06:20.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.112+0000 7fcdea600640 1 -- start start 2024-04-25T10:06:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcdea600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde40cc930 0x7fcde406d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcdea600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcde40ce950 0x7fcde40ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcdea600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde415c150 con 0x7fcde4144d40 2024-04-25T10:06:20.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcdea600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde406d850 con 0x7fcde40ce950 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcdea600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40cd4b0 con 0x7fcde40cc930 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcde40ce950 0x7fcde40ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcde40ce950 0x7fcde40ceda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41656/0 (socket says 172.21.15.165:41656) 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 -- 172.21.15.165:0/3435323934 learned_addr learned my addr 172.21.15.165:0/3435323934 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde2a00640 1 --2- 172.21.15.165:0/3435323934 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde40cc930 0x7fcde406d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 -- 172.21.15.165:0/3435323934 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde40cc930 msgr2=0x7fcde406d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 --2- 172.21.15.165:0/3435323934 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde40cc930 0x7fcde406d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 -- 172.21.15.165:0/3435323934 >> v1:172.21.15.172:6789/0 conn(0x7fcde4144d40 legacy=0x7fcde415c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.114+0000 7fcde3e00640 1 -- 172.21.15.165:0/3435323934 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40ccef0 con 0x7fcde40ce950 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.116+0000 7fcde3e00640 1 --2- 172.21.15.165:0/3435323934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40ce950 0x7fcde40ceda0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc02a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37320d17112b4990 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.116+0000 7fcde2000640 1 -- 172.21.15.165:0/3435323934 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc036070 con 0x7fcde40ce950 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.116+0000 7fcde2000640 1 -- 172.21.15.165:0/3435323934 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcddc031600 con 0x7fcde40ce950 2024-04-25T10:06:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.116+0000 7fcde2000640 1 -- 172.21.15.165:0/3435323934 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc0319d0 con 0x7fcde40ce950 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 -- 172.21.15.165:0/3435323934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40ce950 msgr2=0x7fcde40ceda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 --2- 172.21.15.165:0/3435323934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40ce950 0x7fcde40ceda0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fcddc009e50 tx=0x7fcddc02a5b0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 -- 172.21.15.165:0/3435323934 shutdown_connections 2024-04-25T10:06:20.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 --2- 172.21.15.165:0/3435323934 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40ce950 0x7fcde40ceda0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 --2- 172.21.15.165:0/3435323934 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde40cc930 0x7fcde406d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 -- 172.21.15.165:0/3435323934 >> 172.21.15.165:0/3435323934 conn(0x7fcde406abe0 msgr2=0x7fcde406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 -- 172.21.15.165:0/3435323934 shutdown_connections 2024-04-25T10:06:20.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.117+0000 7fcdea600640 1 -- 172.21.15.165:0/3435323934 wait complete. 2024-04-25T10:06:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.119+0000 7fcdea600640 1 Processor -- start 2024-04-25T10:06:20.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.119+0000 7fcdea600640 1 -- start start 2024-04-25T10:06:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.120+0000 7fcdea600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 0x7fcde413f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.120+0000 7fcdea600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde40ce950 0x7fcde41525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.120+0000 7fcdea600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde4144d40 0x7fcde40b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.120+0000 7fcdea600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde40cf2e0 con 0x7fcde40cc930 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.120+0000 7fcdea600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde415c3f0 con 0x7fcde40ce950 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.120+0000 7fcdea600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde406da60 con 0x7fcde4144d40 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 0x7fcde413f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 0x7fcde413f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41672/0 (socket says 172.21.15.165:41672) 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 -- 172.21.15.165:0/3661764156 learned_addr learned my addr 172.21.15.165:0/3661764156 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 -- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde4144d40 msgr2=0x7fcde40b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde4144d40 0x7fcde40b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 -- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde40ce950 msgr2=0x7fcde41525c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde2a00640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde40ce950 0x7fcde41525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde40ce950 0x7fcde41525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 -- 172.21.15.165:0/3661764156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde40b4870 con 0x7fcde40cc930 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.121+0000 7fcde3400640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 0x7fcde413f120 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b420 tx=0x7fcdd400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.122+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4011830 con 0x7fcde40cc930 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.122+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd4016070 con 0x7fcde40cc930 2024-04-25T10:06:20.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.122+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4011c10 con 0x7fcde40cc930 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.122+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde40b3530 con 0x7fcde40cc930 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.122+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40b3ab0 con 0x7fcde40cc930 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.123+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde4073a00 con 0x7fcde40cc930 2024-04-25T10:06:20.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.124+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdd401b460 con 0x7fcde40cc930 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.125+0000 7fcddb400640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc406b4d0 0x7fcdc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.125+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcdd4089b80 con 0x7fcde40cc930 2024-04-25T10:06:20.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.125+0000 7fcde2a00640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc406b4d0 0x7fcdc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.126+0000 7fcde2a00640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc406b4d0 0x7fcdc406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcde406d610 tx=0x7fcdd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.129+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdd4020020 con 0x7fcde40cc930 2024-04-25T10:06:20.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.332+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcde40ce3c0 con 0x7fcde40cc930 2024-04-25T10:06:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.333+0000 7fcddb400640 1 -- 172.21.15.165:0/3661764156 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcdd4058bb0 con 0x7fcde40cc930 2024-04-25T10:06:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:20.338 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:06:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:06:20.338 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:06:20.339 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:20.339 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc406b4d0 msgr2=0x7fcdc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc406b4d0 0x7fcdc406d990 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fcde406d610 tx=0x7fcdd0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 msgr2=0x7fcde413f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 0x7fcde413f120 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fcdd400b420 tx=0x7fcdd400b8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 shutdown_connections 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc406b4d0 0x7fcdc406d990 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde4144d40 0x7fcde40b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde40ce950 0x7fcde41525c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 --2- 172.21.15.165:0/3661764156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde40cc930 0x7fcde413f120 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 >> 172.21.15.165:0/3661764156 conn(0x7fcde406abe0 msgr2=0x7fcde406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 shutdown_connections 2024-04-25T10:06:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.335+0000 7fcdea600640 1 -- 172.21.15.165:0/3661764156 wait complete. 2024-04-25T10:06:20.359 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:06:20.360 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:06:20.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.565+0000 7f2b05600640 1 Processor -- start 2024-04-25T10:06:20.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.566+0000 7f2b05600640 1 -- start start 2024-04-25T10:06:20.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.567+0000 7f2b05600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 0x7f2b0006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.567+0000 7f2b05600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b00165a80 0x7f2b00165f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.567+0000 7f2b05600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b000cdcd0 con 0x7f2b000ccf30 2024-04-25T10:06:20.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.567+0000 7f2b05600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b00162440 con 0x7f2b0015f950 2024-04-25T10:06:20.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.567+0000 7f2b05600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001632d0 con 0x7f2b00165a80 2024-04-25T10:06:20.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.568+0000 7f2affe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 0x7f2b0006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.568+0000 7f2b04c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b00165a80 0x7f2b00165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.568+0000 7f2affe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 0x7f2b0006e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:59976/0 (socket says 172.21.15.165:59976) 2024-04-25T10:06:20.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.568+0000 7f2affe00640 1 -- 172.21.15.165:0/1379101999 learned_addr learned my addr 172.21.15.165:0/1379101999 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:20.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.569+0000 7f2b04c00640 1 -- 172.21.15.165:0/1379101999 >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 msgr2=0x7f2b0006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:20.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.569+0000 7f2b04c00640 1 --2- 172.21.15.165:0/1379101999 >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 0x7f2b0006e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.569+0000 7f2b04c00640 1 -- 172.21.15.165:0/1379101999 >> v1:172.21.15.172:6789/0 conn(0x7f2b000ccf30 legacy=0x7f2b00162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.569+0000 7f2b04c00640 1 -- 172.21.15.165:0/1379101999 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b000cd430 con 0x7f2b00165a80 2024-04-25T10:06:20.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.569+0000 7f2affe00640 1 --2- 172.21.15.165:0/1379101999 >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 0x7f2b0006e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:06:20.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.570+0000 7f2b04c00640 1 --2- 172.21.15.165:0/1379101999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b00165a80 0x7f2b00165f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009d90 tx=0x7f2af802a5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4051b9d0e9b6b057 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.571+0000 7f2aff400640 1 -- 172.21.15.165:0/1379101999 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8031600 con 0x7f2b00165a80 2024-04-25T10:06:20.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.571+0000 7f2aff400640 1 -- 172.21.15.165:0/1379101999 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af80317a0 con 0x7f2b00165a80 2024-04-25T10:06:20.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.571+0000 7f2aff400640 1 -- 172.21.15.165:0/1379101999 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8031b70 con 0x7f2b00165a80 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.571+0000 7f2b05600640 1 -- 172.21.15.165:0/1379101999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b00165a80 msgr2=0x7f2b00165f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.571+0000 7f2b05600640 1 --2- 172.21.15.165:0/1379101999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b00165a80 0x7f2b00165f30 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009d90 tx=0x7f2af802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.572+0000 7f2b05600640 1 -- 172.21.15.165:0/1379101999 shutdown_connections 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.572+0000 7f2b05600640 1 --2- 172.21.15.165:0/1379101999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b00165a80 0x7f2b00165f30 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.572+0000 7f2b05600640 1 --2- 172.21.15.165:0/1379101999 >> v2:172.21.15.172:3300/0 conn(0x7f2b0015f950 0x7f2b0006e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.572+0000 7f2b05600640 1 -- 172.21.15.165:0/1379101999 >> 172.21.15.165:0/1379101999 conn(0x7f2b00061cc0 msgr2=0x7f2b0006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.572+0000 7f2b05600640 1 -- 172.21.15.165:0/1379101999 shutdown_connections 2024-04-25T10:06:20.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.572+0000 7f2b05600640 1 -- 172.21.15.165:0/1379101999 wait complete. 2024-04-25T10:06:20.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.574+0000 7f2b05600640 1 Processor -- start 2024-04-25T10:06:20.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.574+0000 7f2b05600640 1 -- start start 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.575+0000 7f2b05600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 0x7f2b001563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.575+0000 7f2b05600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b0015f950 0x7f2b00163790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.575+0000 7f2b05600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00165a80 0x7f2b00153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.575+0000 7f2b05600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b001626a0 con 0x7f2b000ccf30 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.575+0000 7f2b05600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b000cd9f0 con 0x7f2b00165a80 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.575+0000 7f2b05600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0015fd30 con 0x7f2b0015f950 2024-04-25T10:06:20.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 0x7f2b001563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 0x7f2b001563b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41686/0 (socket says 172.21.15.165:41686) 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 -- 172.21.15.165:0/2061862537 learned_addr learned my addr 172.21.15.165:0/2061862537 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2b04c00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00165a80 0x7f2b00153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 -- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b0015f950 msgr2=0x7f2b00163790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2af7e00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b0015f950 0x7f2b00163790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b0015f950 0x7f2b00163790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 -- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00165a80 msgr2=0x7f2b00153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00165a80 0x7f2b00153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 -- 172.21.15.165:0/2061862537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b00163cd0 con 0x7f2b000ccf30 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2b04c00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00165a80 0x7f2b00153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.576+0000 7f2affe00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 0x7f2b001563b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e9d0 tx=0x7f2af000ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.577+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af0017070 con 0x7f2b000ccf30 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.577+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af000ca60 con 0x7f2b000ccf30 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.577+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af000ce10 con 0x7f2b000ccf30 2024-04-25T10:06:20.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.577+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b00156a20 con 0x7f2b000ccf30 2024-04-25T10:06:20.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.577+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b00156fe0 con 0x7f2b000ccf30 2024-04-25T10:06:20.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.578+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b00073990 con 0x7f2b000ccf30 2024-04-25T10:06:20.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.579+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2af001a070 con 0x7f2b000ccf30 2024-04-25T10:06:20.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.580+0000 7f2afcc00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2adc06b4d0 0x7f2adc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:20.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.580+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2af008a340 con 0x7f2b000ccf30 2024-04-25T10:06:20.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.580+0000 7f2af7e00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2adc06b4d0 0x7f2adc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:20.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.581+0000 7f2af7e00640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2adc06b4d0 0x7f2adc06d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b0015bb10 tx=0x7f2ae8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:20.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.586+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2af0059370 con 0x7f2b000ccf30 2024-04-25T10:06:20.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:20.806+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b000b1290 con 0x7f2b000ccf30 2024-04-25T10:06:21.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.530+0000 7f2afcc00640 1 -- 172.21.15.165:0/2061862537 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2af0058c70 con 0x7f2b000ccf30 2024-04-25T10:06:21.536 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2adc06b4d0 msgr2=0x7f2adc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2adc06b4d0 0x7f2adc06d990 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2b0015bb10 tx=0x7f2ae8009040 comp rx=0 tx=0).stop 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 msgr2=0x7f2b001563b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 0x7f2b001563b0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f2af000e9d0 tx=0x7f2af000ee90 comp rx=0 tx=0).stop 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 shutdown_connections 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2adc06b4d0 0x7f2adc06d990 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b00165a80 0x7f2b00153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b0015f950 0x7f2b00163790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.534+0000 7f2b05600640 1 --2- 172.21.15.165:0/2061862537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b000ccf30 0x7f2b001563b0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.535+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 >> 172.21.15.165:0/2061862537 conn(0x7f2b00061cc0 msgr2=0x7f2b0006e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.535+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 shutdown_connections 2024-04-25T10:06:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.535+0000 7f2b05600640 1 -- 172.21.15.165:0/2061862537 wait complete. 2024-04-25T10:06:21.568 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:06:21.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.750+0000 7f1700c00640 1 Processor -- start 2024-04-25T10:06:21.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.750+0000 7f1700c00640 1 -- start start 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.751+0000 7f1700c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.751+0000 7f1700c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f16fc06ec50 0x7f16fc15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.751+0000 7f1700c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f16fc15ee60 0x7f16fc0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.751+0000 7f1700c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc15e490 con 0x7f16fc15ee60 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.751+0000 7f1700c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16fc0cfd80 con 0x7f16fc06ec50 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.751+0000 7f1700c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0d0060 con 0x7f16fc15b710 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.752+0000 7f16fa800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f16fc15ee60 0x7f16fc0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.752+0000 7f16fa800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f16fc15ee60 0x7f16fc0ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41702/0 (socket says 172.21.15.165:41702) 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.752+0000 7f16fa800640 1 -- 172.21.15.165:0/823220294 learned_addr learned my addr 172.21.15.165:0/823220294 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.752+0000 7f16f9e00640 1 --2- 172.21.15.165:0/823220294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.752+0000 7f16f3e00640 1 --2- 172.21.15.165:0/823220294 >> v2:172.21.15.172:3300/0 conn(0x7f16fc06ec50 0x7f16fc15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.753+0000 7f16fa800640 1 -- 172.21.15.165:0/823220294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 msgr2=0x7f16fc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.753+0000 7f16fa800640 1 --2- 172.21.15.165:0/823220294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.753+0000 7f16fa800640 1 -- 172.21.15.165:0/823220294 >> v2:172.21.15.172:3300/0 conn(0x7f16fc06ec50 msgr2=0x7f16fc15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:21.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.753+0000 7f16fa800640 1 --2- 172.21.15.165:0/823220294 >> v2:172.21.15.172:3300/0 conn(0x7f16fc06ec50 0x7f16fc15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.753+0000 7f16fa800640 1 -- 172.21.15.165:0/823220294 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc0d0340 con 0x7f16fc15ee60 2024-04-25T10:06:21.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.754+0000 7f16fa800640 1 --2- 172.21.15.165:0/823220294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc0ac8b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0d07a0 tx=0x7f16f402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3243146cdb15f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.754+0000 7f16f9400640 1 -- 172.21.15.165:0/823220294 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4036070 con 0x7f16fc15ee60 2024-04-25T10:06:21.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.754+0000 7f16f9400640 1 -- 172.21.15.165:0/823220294 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4031600 con 0x7f16fc15ee60 2024-04-25T10:06:21.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.754+0000 7f16f9400640 1 -- 172.21.15.165:0/823220294 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f40319d0 con 0x7f16fc15ee60 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 -- 172.21.15.165:0/823220294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 msgr2=0x7f16fc0ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 --2- 172.21.15.165:0/823220294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc0ac8b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f16fc0d07a0 tx=0x7f16f402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 -- 172.21.15.165:0/823220294 shutdown_connections 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 --2- 172.21.15.165:0/823220294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc0ac8b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 --2- 172.21.15.165:0/823220294 >> v2:172.21.15.172:3300/0 conn(0x7f16fc06ec50 0x7f16fc15e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 --2- 172.21.15.165:0/823220294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc0cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 -- 172.21.15.165:0/823220294 >> 172.21.15.165:0/823220294 conn(0x7f16fc06abe0 msgr2=0x7f16fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 -- 172.21.15.165:0/823220294 shutdown_connections 2024-04-25T10:06:21.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.755+0000 7f1700c00640 1 -- 172.21.15.165:0/823220294 wait complete. 2024-04-25T10:06:21.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.757+0000 7f1700c00640 1 Processor -- start 2024-04-25T10:06:21.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.757+0000 7f1700c00640 1 -- start start 2024-04-25T10:06:21.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f1700c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16fc06ec50 0x7f16fc0af2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f1700c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc160650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f1700c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc141d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f1700c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc15bc70 con 0x7f16fc15ee60 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f1700c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc15baf0 con 0x7f16fc06ec50 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f1700c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16fc0acdf0 con 0x7f16fc15b710 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f16fa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc141d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f16fa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc141d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41718/0 (socket says 172.21.15.165:41718) 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.758+0000 7f16fa800640 1 -- 172.21.15.165:0/2043820420 learned_addr learned my addr 172.21.15.165:0/2043820420 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16f9e00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16fc06ec50 0x7f16fc0af2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16f3e00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc160650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16fa800640 1 -- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 msgr2=0x7f16fc160650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16fa800640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc160650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16fa800640 1 -- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16fc06ec50 msgr2=0x7f16fc0af2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16fa800640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16fc06ec50 0x7f16fc0af2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16fa800640 1 -- 172.21.15.165:0/2043820420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16fc145d70 con 0x7f16fc15ee60 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16fa800640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc141d30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009f40 tx=0x7f16f4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.759+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4041020 con 0x7f16fc15ee60 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.760+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16f4007780 con 0x7f16fc15ee60 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.760+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16f4036040 con 0x7f16fc15ee60 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.760+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16fc0af810 con 0x7f16fc15ee60 2024-04-25T10:06:21.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.760+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16fc1465d0 con 0x7f16fc15ee60 2024-04-25T10:06:21.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.761+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16fc146070 con 0x7f16fc15ee60 2024-04-25T10:06:21.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.763+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16f4044020 con 0x7f16fc15ee60 2024-04-25T10:06:21.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.764+0000 7f16f2000640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d806b4d0 0x7f16d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:21.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.764+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f16f40ab2a0 con 0x7f16fc15ee60 2024-04-25T10:06:21.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.764+0000 7f16f9e00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d806b4d0 0x7f16d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:21.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.765+0000 7f16f9e00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d806b4d0 0x7f16d806d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f16ec006fd0 tx=0x7f16ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:21.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.768+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16fc146070 con 0x7f16fc15ee60 2024-04-25T10:06:21.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:21.956+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f16fc000900 con 0x7f16fc15ee60 2024-04-25T10:06:22.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.536+0000 7f16f2000640 1 -- 172.21.15.165:0/2043820420 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f16f404b030 con 0x7f16fc15ee60 2024-04-25T10:06:22.542 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:06:22.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d806b4d0 msgr2=0x7f16d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d806b4d0 0x7f16d806d990 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f16ec006fd0 tx=0x7f16ec008040 comp rx=0 tx=0).stop 2024-04-25T10:06:22.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 msgr2=0x7f16fc141d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc141d30 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f16f4009f40 tx=0x7f16f4004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 shutdown_connections 2024-04-25T10:06:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d806b4d0 0x7f16d806d990 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16fc15ee60 0x7f16fc141d30 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16fc15b710 0x7f16fc160650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.540+0000 7f1700c00640 1 --2- 172.21.15.165:0/2043820420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16fc06ec50 0x7f16fc0af2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.541+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 >> 172.21.15.165:0/2043820420 conn(0x7f16fc06abe0 msgr2=0x7f16fc15c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.541+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 shutdown_connections 2024-04-25T10:06:22.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.541+0000 7f1700c00640 1 -- 172.21.15.165:0/2043820420 wait complete. 2024-04-25T10:06:22.587 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:06:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.756+0000 7f5f34000640 1 Processor -- start 2024-04-25T10:06:22.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.756+0000 7f5f34000640 1 -- start start 2024-04-25T10:06:22.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f34000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 0x7f5f2c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f34000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f2c06e8a0 0x7f5f2c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f34000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f2c0cccc0 con 0x7f5f2c144d50 2024-04-25T10:06:22.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f34000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b2b20 con 0x7f5f2c06e8a0 2024-04-25T10:06:22.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f34000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c06d400 con 0x7f5f2c1446d0 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f30c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5f2c144d50 0x7f5f2c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60726/0 (socket says 172.21.15.165:60726) 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f30c00640 1 -- 172.21.15.165:0/341174008 learned_addr learned my addr 172.21.15.165:0/341174008 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f31600640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 0x7f5f2c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.758+0000 7f5f32000640 1 --2- 172.21.15.165:0/341174008 >> v2:172.21.15.172:3300/0 conn(0x7f5f2c06e8a0 0x7f5f2c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789031484 0 0) 0x7f5f2c0cccc0 con 0x7f5f2c144d50 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f10003520 con 0x7f5f2c144d50 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3667961036 0 0) 0x7f5f10003520 con 0x7f5f2c144d50 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 msgr2=0x7f5f2c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 0x7f5f2c0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 >> v2:172.21.15.172:3300/0 conn(0x7f5f2c06e8a0 msgr2=0x7f5f2c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 --2- 172.21.15.165:0/341174008 >> v2:172.21.15.172:3300/0 conn(0x7f5f2c06e8a0 0x7f5f2c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0cc140 con 0x7f5f2c144d50 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5f180019f0 con 0x7f5f2c144d50 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 >> v1:172.21.15.165:6789/0 conn(0x7f5f2c144d50 legacy=0x7f5f2c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100039b0 0x7f5f10003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 0x7f5f10004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f10004d40 0x7f5f100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b2b20 con 0x7f5f10004d40 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c06d400 con 0x7f5f100039b0 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.759+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f1000b9c0 con 0x7f5f2c1446d0 2024-04-25T10:06:22.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f32000640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f10004d40 0x7f5f100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f30c00640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 0x7f5f10004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f31600640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100039b0 0x7f5f10003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f32000640 1 -- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 msgr2=0x7f5f10004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f32000640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f2c1446d0 0x7f5f10004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f32000640 1 -- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100039b0 msgr2=0x7f5f10003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f32000640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f100039b0 0x7f5f10003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.760+0000 7f5f32000640 1 -- 172.21.15.165:0/341174008 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f10003520 con 0x7f5f10004d40 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.761+0000 7f5f32000640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f10004d40 0x7f5f100051c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ac0 tx=0x7f5f10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476474100a6f0c74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.761+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24036070 con 0x7f5f10004d40 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.761+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24031400 con 0x7f5f10004d40 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.761+0000 7f5f2be00640 1 -- 172.21.15.165:0/341174008 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24009690 con 0x7f5f10004d40 2024-04-25T10:06:22.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f34000640 1 -- 172.21.15.165:0/341174008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f10004d40 msgr2=0x7f5f100051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f34000640 1 --2- 172.21.15.165:0/341174008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f10004d40 0x7f5f100051c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ac0 tx=0x7f5f10005830 comp rx=0 tx=0).stop 2024-04-25T10:06:22.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f32000640 1 -- 172.21.15.165:0/341174008 reap_dead start 2024-04-25T10:06:22.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f34000640 1 -- 172.21.15.165:0/341174008 shutdown_connections 2024-04-25T10:06:22.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f34000640 1 -- 172.21.15.165:0/341174008 >> 172.21.15.165:0/341174008 conn(0x7f5f2c06abe0 msgr2=0x7f5f2c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:22.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f34000640 1 -- 172.21.15.165:0/341174008 shutdown_connections 2024-04-25T10:06:22.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.762+0000 7f5f34000640 1 -- 172.21.15.165:0/341174008 wait complete. 2024-04-25T10:06:22.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.764+0000 7f5f34000640 1 Processor -- start 2024-04-25T10:06:22.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.764+0000 7f5f34000640 1 -- start start 2024-04-25T10:06:22.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f34000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10004d40 0x7f5f2c0abe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f34000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f2c1f3650 0x7f5f2c0ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f34000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 0x7f5f2c0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f34000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cc700 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f34000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0b49a0 con 0x7f5f2c1f3650 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f34000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c0cc9e0 con 0x7f5f10004d40 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 0x7f5f2c0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 0x7f5f2c0ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41732/0 (socket says 172.21.15.165:41732) 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.765+0000 7f5f32000640 1 -- 172.21.15.165:0/3186448103 learned_addr learned my addr 172.21.15.165:0/3186448103 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f30c00640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f2c1f3650 0x7f5f2c0ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f31600640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10004d40 0x7f5f2c0abe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f32000640 1 -- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10004d40 msgr2=0x7f5f2c0abe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f32000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10004d40 0x7f5f2c0abe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f32000640 1 -- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f2c1f3650 msgr2=0x7f5f2c0ab6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f32000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f2c1f3650 0x7f5f2c0ab6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f32000640 1 -- 172.21.15.165:0/3186448103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c0abbf0 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f31600640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10004d40 0x7f5f2c0abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:22.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f32000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 0x7f5f2c0ae030 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ac0 tx=0x7f5f24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.766+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24036040 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.767+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f24041070 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.767+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f24009690 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.767+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f2c0ac530 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.767+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f2c0c9a70 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.768+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f2c0cb9e0 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.769+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f24039070 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.770+0000 7f5f29600640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f0c06b4d0 0x7f5f0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:22.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.770+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f5f240a9cb0 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.770+0000 7f5f31600640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f0c06b4d0 0x7f5f0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:22.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.771+0000 7f5f31600640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f0c06b4d0 0x7f5f0c06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0099e0 tx=0x7f5f1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:22.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.775+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f2c0cb9e0 con 0x7f5f2c0adbb0 2024-04-25T10:06:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:22.991+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f5f2c073a00 con 0x7f5f2c0adbb0 2024-04-25T10:06:23.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.592+0000 7f5f29600640 1 -- 172.21.15.165:0/3186448103 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v25) v1 ==== 172+0+0 (secure 0 0 0) 0x7f5f2404a020 con 0x7f5f2c0adbb0 2024-04-25T10:06:23.602 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 7 and data pool 8 2024-04-25T10:06:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.603+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f0c06b4d0 msgr2=0x7f5f0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.603+0000 7f5f34000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f0c06b4d0 0x7f5f0c06d990 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5f1c0099e0 tx=0x7f5f1c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.603+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 msgr2=0x7f5f2c0ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.603+0000 7f5f34000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 0x7f5f2c0ae030 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009ac0 tx=0x7f5f24004730 comp rx=0 tx=0).stop 2024-04-25T10:06:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.603+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 shutdown_connections 2024-04-25T10:06:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.604+0000 7f5f34000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f0c06b4d0 0x7f5f0c06d990 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.604+0000 7f5f34000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f2c0adbb0 0x7f5f2c0ae030 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.604+0000 7f5f34000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f2c1f3650 0x7f5f2c0ab6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.604+0000 7f5f34000640 1 --2- 172.21.15.165:0/3186448103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f10004d40 0x7f5f2c0abe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.604+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 >> 172.21.15.165:0/3186448103 conn(0x7f5f2c06abe0 msgr2=0x7f5f2c071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.605+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 shutdown_connections 2024-04-25T10:06:23.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.605+0000 7f5f34000640 1 -- 172.21.15.165:0/3186448103 wait complete. 2024-04-25T10:06:23.641 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:23.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.806+0000 7f5360000640 1 Processor -- start 2024-04-25T10:06:23.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.806+0000 7f5360000640 1 -- start start 2024-04-25T10:06:23.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.808+0000 7f5360000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f535813d190 0x7f53580cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.808+0000 7f5360000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 0x7f53580cc4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.808+0000 7f5360000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5358071ed0 con 0x7f53580cf620 2024-04-25T10:06:23.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.808+0000 7f5360000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f535813d570 con 0x7f535813d190 2024-04-25T10:06:23.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.808+0000 7f5360000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f535815a840 con 0x7f535815b320 2024-04-25T10:06:23.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 0x7f53580cc4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f535813d190 0x7f53580cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 0x7f53580cc4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:55456/0 (socket says 172.21.15.165:55456) 2024-04-25T10:06:23.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 -- 172.21.15.165:0/1918038730 learned_addr learned my addr 172.21.15.165:0/1918038730 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:23.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 -- 172.21.15.165:0/1918038730 >> v2:172.21.15.172:3300/0 conn(0x7f535813d190 msgr2=0x7f53580cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 --2- 172.21.15.165:0/1918038730 >> v2:172.21.15.172:3300/0 conn(0x7f535813d190 0x7f53580cbfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 -- 172.21.15.165:0/1918038730 >> v1:172.21.15.172:6789/0 conn(0x7f53580cf620 legacy=0x7f5358071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:23.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.809+0000 7f535cc00640 1 -- 172.21.15.165:0/1918038730 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f535815ab20 con 0x7f535815b320 2024-04-25T10:06:23.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.811+0000 7f535cc00640 1 --2- 172.21.15.165:0/1918038730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 0x7f53580cc4f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5344009870 tx=0x7f534402c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=966ff8bbc8a88033 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.811+0000 7f5357e00640 1 -- 172.21.15.165:0/1918038730 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344039070 con 0x7f535815b320 2024-04-25T10:06:23.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5357e00640 1 -- 172.21.15.165:0/1918038730 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5344034640 con 0x7f535815b320 2024-04-25T10:06:23.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5357e00640 1 -- 172.21.15.165:0/1918038730 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5344034a10 con 0x7f535815b320 2024-04-25T10:06:23.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 -- 172.21.15.165:0/1918038730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 msgr2=0x7f53580cc4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 --2- 172.21.15.165:0/1918038730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 0x7f53580cc4f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5344009870 tx=0x7f534402c390 comp rx=0 tx=0).stop 2024-04-25T10:06:23.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 -- 172.21.15.165:0/1918038730 shutdown_connections 2024-04-25T10:06:23.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 --2- 172.21.15.165:0/1918038730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535815b320 0x7f53580cc4f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 --2- 172.21.15.165:0/1918038730 >> v2:172.21.15.172:3300/0 conn(0x7f535813d190 0x7f53580cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 -- 172.21.15.165:0/1918038730 >> 172.21.15.165:0/1918038730 conn(0x7f535806abe0 msgr2=0x7f535806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:23.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.812+0000 7f5360000640 1 -- 172.21.15.165:0/1918038730 shutdown_connections 2024-04-25T10:06:23.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.813+0000 7f5360000640 1 -- 172.21.15.165:0/1918038730 wait complete. 2024-04-25T10:06:23.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.814+0000 7f5360000640 1 Processor -- start 2024-04-25T10:06:23.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.814+0000 7f5360000640 1 -- start start 2024-04-25T10:06:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.815+0000 7f5360000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53580cf620 0x7f535815cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.815+0000 7f5360000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535813d190 0x7f53580cd500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.815+0000 7f5360000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 0x7f53580cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.815+0000 7f5360000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580cfc30 con 0x7f535815b320 2024-04-25T10:06:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.815+0000 7f5360000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53580cfab0 con 0x7f53580cf620 2024-04-25T10:06:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.815+0000 7f5360000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5358071940 con 0x7f535813d190 2024-04-25T10:06:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 0x7f53580cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 0x7f53580cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41736/0 (socket says 172.21.15.165:41736) 2024-04-25T10:06:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 -- 172.21.15.165:0/3835783184 learned_addr learned my addr 172.21.15.165:0/3835783184 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535cc00640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535813d190 0x7f53580cd500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535d600640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53580cf620 0x7f535815cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 -- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535813d190 msgr2=0x7f53580cd500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535813d190 0x7f53580cd500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 -- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53580cf620 msgr2=0x7f535815cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:23.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53580cf620 0x7f535815cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:23.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 -- 172.21.15.165:0/3835783184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53581e9e60 con 0x7f535815b320 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.816+0000 7f535e000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 0x7f53580cda40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f535000c780 tx=0x7f535000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.817+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5350004b60 con 0x7f535815b320 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.817+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5350004d00 con 0x7f535815b320 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.817+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f535000aa90 con 0x7f535815b320 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.817+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f535813c940 con 0x7f535815b320 2024-04-25T10:06:23.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.817+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f535813c380 con 0x7f535815b320 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.818+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5358073990 con 0x7f535815b320 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.819+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f535000ac30 con 0x7f535815b320 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.820+0000 7f5355600640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f533806b4d0 0x7f533806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.820+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f535008a680 con 0x7f535815b320 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.820+0000 7f535d600640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f533806b4d0 0x7f533806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.821+0000 7f535d600640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f533806b4d0 0x7f533806d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f534c009a10 tx=0x7f534c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:23.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:23.824+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5350020070 con 0x7f535815b320 2024-04-25T10:06:24.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.018+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f53580cfeb0 con 0x7f535815b320 2024-04-25T10:06:24.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.019+0000 7f5355600640 1 -- 172.21.15.165:0/3835783184 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f53500592d0 con 0x7f535815b320 2024-04-25T10:06:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:24.025 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":53,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:22.993228+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f533806b4d0 msgr2=0x7f533806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f533806b4d0 0x7f533806d990 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f534c009a10 tx=0x7f534c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 msgr2=0x7f53580cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 0x7f53580cda40 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f535000c780 tx=0x7f535000cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 shutdown_connections 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f533806b4d0 0x7f533806d990 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f535815b320 0x7f53580cda40 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f535813d190 0x7f53580cd500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 --2- 172.21.15.165:0/3835783184 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53580cf620 0x7f535815cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 >> 172.21.15.165:0/3835783184 conn(0x7f535806abe0 msgr2=0x7f5358157840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 shutdown_connections 2024-04-25T10:06:24.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.022+0000 7f5360000640 1 -- 172.21.15.165:0/3835783184 wait complete. 2024-04-25T10:06:24.050 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:24.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.236+0000 7f4a47000640 1 Processor -- start 2024-04-25T10:06:24.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.236+0000 7f4a47000640 1 -- start start 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a47000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4a40154ee0 0x7f4a400cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a47000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a400cfcf0 0x7f4a400cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a47000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a40071480 con 0x7f4a400ce3c0 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a47000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a400ce830 con 0x7f4a40154ee0 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a47000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40072090 con 0x7f4a400cfcf0 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a3fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4a40154ee0 0x7f4a400cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a3fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4a40154ee0 0x7f4a400cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41748/0 (socket says 172.21.15.165:41748) 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a3fe00640 1 -- 172.21.15.165:0/1680650419 learned_addr learned my addr 172.21.15.165:0/1680650419 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.238+0000 7f4a37400640 1 --2- 172.21.15.165:0/1680650419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a400cfcf0 0x7f4a400cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.239+0000 7f4a3fe00640 1 -- 172.21.15.165:0/1680650419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a400cfcf0 msgr2=0x7f4a400cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.239+0000 7f4a3fe00640 1 --2- 172.21.15.165:0/1680650419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a400cfcf0 0x7f4a400cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.239+0000 7f4a3fe00640 1 -- 172.21.15.165:0/1680650419 >> v1:172.21.15.172:6789/0 conn(0x7f4a400ce3c0 legacy=0x7f4a4015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.239+0000 7f4a3fe00640 1 -- 172.21.15.165:0/1680650419 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40071760 con 0x7f4a40154ee0 2024-04-25T10:06:24.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.240+0000 7f4a3fe00640 1 --2- 172.21.15.165:0/1680650419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a40154ee0 0x7f4a400cf7b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000bb10 tx=0x7f4a3002c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3d11a85fb21add server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.240+0000 7f4a3f400640 1 -- 172.21.15.165:0/1680650419 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30032790 con 0x7f4a40154ee0 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.240+0000 7f4a3f400640 1 -- 172.21.15.165:0/1680650419 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a30038070 con 0x7f4a40154ee0 2024-04-25T10:06:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.240+0000 7f4a3f400640 1 -- 172.21.15.165:0/1680650419 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a30032b50 con 0x7f4a40154ee0 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 -- 172.21.15.165:0/1680650419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a40154ee0 msgr2=0x7f4a400cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 --2- 172.21.15.165:0/1680650419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a40154ee0 0x7f4a400cf7b0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f4a3000bb10 tx=0x7f4a3002c8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 -- 172.21.15.165:0/1680650419 shutdown_connections 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 --2- 172.21.15.165:0/1680650419 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a400cfcf0 0x7f4a400cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 --2- 172.21.15.165:0/1680650419 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a40154ee0 0x7f4a400cf7b0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 -- 172.21.15.165:0/1680650419 >> 172.21.15.165:0/1680650419 conn(0x7f4a4006abe0 msgr2=0x7f4a4006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 -- 172.21.15.165:0/1680650419 shutdown_connections 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.241+0000 7f4a47000640 1 -- 172.21.15.165:0/1680650419 wait complete. 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.243+0000 7f4a47000640 1 Processor -- start 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.243+0000 7f4a47000640 1 -- start start 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a47000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a400ce3c0 0x7f4a40159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a47000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 0x7f4a40146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a47000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a40154ee0 0x7f4a40147120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a47000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40071230 con 0x7f4a400cfcf0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a47000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a40071d20 con 0x7f4a400ce3c0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a47000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a401552c0 con 0x7f4a40154ee0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a37400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 0x7f4a40146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a37400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 0x7f4a40146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41756/0 (socket says 172.21.15.165:41756) 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.244+0000 7f4a37400640 1 -- 172.21.15.165:0/3891413587 learned_addr learned my addr 172.21.15.165:0/3891413587 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a3fe00640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a400ce3c0 0x7f4a40159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a37400640 1 -- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a40154ee0 msgr2=0x7f4a40147120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a37400640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a40154ee0 0x7f4a40147120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a37400640 1 -- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a400ce3c0 msgr2=0x7f4a40159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a37400640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a400ce3c0 0x7f4a40159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a37400640 1 -- 172.21.15.165:0/3891413587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a401e4120 con 0x7f4a400cfcf0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a37400640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 0x7f4a40146be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800d800 tx=0x7f4a2800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800b6c0 con 0x7f4a400cfcf0 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a28019070 con 0x7f4a400cfcf0 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800ba80 con 0x7f4a400cfcf0 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a401e4400 con 0x7f4a400cfcf0 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.245+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a400a9810 con 0x7f4a400cfcf0 2024-04-25T10:06:24.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.246+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a40073990 con 0x7f4a400cfcf0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.247+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4a2800bc20 con 0x7f4a400cfcf0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.249+0000 7f4a3cc00640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a1c06b4d0 0x7f4a1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.249+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4a2808a850 con 0x7f4a400cfcf0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.249+0000 7f4a3fe00640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a1c06b4d0 0x7f4a1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.250+0000 7f4a3fe00640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a1c06b4d0 0x7f4a1c06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4a30037000 tx=0x7f4a30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.254+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4a28059530 con 0x7f4a400cfcf0 2024-04-25T10:06:24.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.448+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a4006d360 con 0x7f4a400cfcf0 2024-04-25T10:06:24.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.450+0000 7f4a3cc00640 1 -- 172.21.15.165:0/3891413587 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+16991 (secure 0 0 0) 0x7f4a28058e30 con 0x7f4a400cfcf0 2024-04-25T10:06:24.459 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:24.459 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":53,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:22.993228+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a1c06b4d0 msgr2=0x7f4a1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a1c06b4d0 0x7f4a1c06d990 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f4a30037000 tx=0x7f4a30008040 comp rx=0 tx=0).stop 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 msgr2=0x7f4a40146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 0x7f4a40146be0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4a2800d800 tx=0x7f4a2800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 shutdown_connections 2024-04-25T10:06:24.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a1c06b4d0 0x7f4a1c06d990 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a40154ee0 0x7f4a40147120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a400cfcf0 0x7f4a40146be0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 --2- 172.21.15.165:0/3891413587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a400ce3c0 0x7f4a40159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 >> 172.21.15.165:0/3891413587 conn(0x7f4a4006abe0 msgr2=0x7f4a4015d000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 shutdown_connections 2024-04-25T10:06:24.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.453+0000 7f4a47000640 1 -- 172.21.15.165:0/3891413587 wait complete. 2024-04-25T10:06:24.477 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:06:24.531 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:06:24.525+0000 7fa94dc00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:06:24.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.662+0000 7f0947200640 1 Processor -- start 2024-04-25T10:06:24.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.663+0000 7f0947200640 1 -- start start 2024-04-25T10:06:24.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.664+0000 7f0947200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f094813d9f0 0x7f094813df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.664+0000 7f0947200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f094806e860 0x7f09480b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.664+0000 7f0947200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f094806d390 con 0x7f09480b0730 2024-04-25T10:06:24.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.664+0000 7f0947200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094806d670 con 0x7f094806e860 2024-04-25T10:06:24.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.664+0000 7f0947200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f094806d950 con 0x7f094813d9f0 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.664+0000 7f0946800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f094806e860 0x7f09480b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0946800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f094806e860 0x7f09480b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41768/0 (socket says 172.21.15.165:41768) 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0946800640 1 -- 172.21.15.165:0/1385881528 learned_addr learned my addr 172.21.15.165:0/1385881528 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0945e00640 1 --2- 172.21.15.165:0/1385881528 >> v2:172.21.15.172:3300/0 conn(0x7f094813d9f0 0x7f094813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0946800640 1 -- 172.21.15.165:0/1385881528 >> v2:172.21.15.172:3300/0 conn(0x7f094813d9f0 msgr2=0x7f094813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0946800640 1 --2- 172.21.15.165:0/1385881528 >> v2:172.21.15.172:3300/0 conn(0x7f094813d9f0 0x7f094813df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0946800640 1 -- 172.21.15.165:0/1385881528 >> v1:172.21.15.172:6789/0 conn(0x7f09480b0730 legacy=0x7f094806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:24.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.665+0000 7f0946800640 1 -- 172.21.15.165:0/1385881528 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094806dc30 con 0x7f094806e860 2024-04-25T10:06:24.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.666+0000 7f0946800640 1 --2- 172.21.15.165:0/1385881528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f094806e860 0x7f09480b0bc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0948072520 tx=0x7f094002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6d3dfb324d04a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.666+0000 7f0945400640 1 -- 172.21.15.165:0/1385881528 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940036070 con 0x7f094806e860 2024-04-25T10:06:24.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.666+0000 7f0945400640 1 -- 172.21.15.165:0/1385881528 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940031600 con 0x7f094806e860 2024-04-25T10:06:24.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.666+0000 7f0945400640 1 -- 172.21.15.165:0/1385881528 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09400319d0 con 0x7f094806e860 2024-04-25T10:06:24.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 -- 172.21.15.165:0/1385881528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f094806e860 msgr2=0x7f09480b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 --2- 172.21.15.165:0/1385881528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f094806e860 0x7f09480b0bc0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0948072520 tx=0x7f094002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 -- 172.21.15.165:0/1385881528 shutdown_connections 2024-04-25T10:06:24.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 --2- 172.21.15.165:0/1385881528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f094806e860 0x7f09480b0bc0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 --2- 172.21.15.165:0/1385881528 >> v2:172.21.15.172:3300/0 conn(0x7f094813d9f0 0x7f094813df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 -- 172.21.15.165:0/1385881528 >> 172.21.15.165:0/1385881528 conn(0x7f0948061cc0 msgr2=0x7f094806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 -- 172.21.15.165:0/1385881528 shutdown_connections 2024-04-25T10:06:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.667+0000 7f0947200640 1 -- 172.21.15.165:0/1385881528 wait complete. 2024-04-25T10:06:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.669+0000 7f0947200640 1 Processor -- start 2024-04-25T10:06:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.669+0000 7f0947200640 1 -- start start 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f0947200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f094806e860 0x7f09480b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f0947200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 0x7f09480b19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f0947200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094813d9f0 0x7f09481f0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f0947200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09480ce880 con 0x7f09480b0730 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f0947200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094813e680 con 0x7f094813d9f0 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f0947200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09480cea00 con 0x7f094806e860 2024-04-25T10:06:24.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f093d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 0x7f09480b19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f093d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 0x7f09480b19f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41778/0 (socket says 172.21.15.165:41778) 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.670+0000 7f093d400640 1 -- 172.21.15.165:0/3673396104 learned_addr learned my addr 172.21.15.165:0/3673396104 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093d400640 1 -- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f094806e860 msgr2=0x7f09480b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f0945e00640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f094806e860 0x7f09480b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f0946800640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094813d9f0 0x7f09481f0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093d400640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f094806e860 0x7f09480b14b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093d400640 1 -- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094813d9f0 msgr2=0x7f09481f0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093d400640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094813d9f0 0x7f09481f0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093d400640 1 -- 172.21.15.165:0/3673396104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09480b29b0 con 0x7f09480b0730 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093d400640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 0x7f09480b19f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f093000c430 tx=0x7f093000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930011790 con 0x7f09480b0730 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0930017070 con 0x7f09480b0730 2024-04-25T10:06:24.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0930011bb0 con 0x7f09480b0730 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09481f1cd0 con 0x7f09480b0730 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.671+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09481f2290 con 0x7f09480b0730 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.672+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0948073990 con 0x7f09480b0730 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.673+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f093001c460 con 0x7f09480b0730 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.674+0000 7f093ea00640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f092406b4d0 0x7f092406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.674+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f093008a280 con 0x7f09480b0730 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.675+0000 7f0945e00640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f092406b4d0 0x7f092406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:24.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.675+0000 7f0945e00640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f092406b4d0 0x7f092406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f093800b7c0 tx=0x7f0938008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:24.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.679+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0930058f60 con 0x7f09480b0730 2024-04-25T10:06:24.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:24.914+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0948071df0 con 0x7f09480b0730 2024-04-25T10:06:25.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.590+0000 7f093ea00640 1 -- 172.21.15.165:0/3673396104 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v27) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0930058860 con 0x7f09480b0730 2024-04-25T10:06:25.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.593+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f092406b4d0 msgr2=0x7f092406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.593+0000 7f0947200640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f092406b4d0 0x7f092406d990 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f093800b7c0 tx=0x7f0938008040 comp rx=0 tx=0).stop 2024-04-25T10:06:25.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.593+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 msgr2=0x7f09480b19f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.593+0000 7f0947200640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 0x7f09480b19f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f093000c430 tx=0x7f093000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 shutdown_connections 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f092406b4d0 0x7f092406d990 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094813d9f0 0x7f09481f0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09480b0730 0x7f09480b19f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 --2- 172.21.15.165:0/3673396104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f094806e860 0x7f09480b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 >> 172.21.15.165:0/3673396104 conn(0x7f0948061cc0 msgr2=0x7f0948143dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 shutdown_connections 2024-04-25T10:06:25.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.594+0000 7f0947200640 1 -- 172.21.15.165:0/3673396104 wait complete. 2024-04-25T10:06:25.619 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:06:25.620 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:06:25.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.819+0000 7f992f400640 1 Processor -- start 2024-04-25T10:06:25.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.819+0000 7f992f400640 1 -- start start 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f992f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f992815dc20 0x7f99280ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f992f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9928161710 0x7f99280cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f992f400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992815e000 con 0x7f99280b9330 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f992f400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99280d1de0 con 0x7f9928161710 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f992f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9928161c60 con 0x7f992815dc20 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f991fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9928161710 0x7f99280cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f991fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9928161710 0x7f99280cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41790/0 (socket says 172.21.15.165:41790) 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f991fe00640 1 -- 172.21.15.165:0/3807271033 learned_addr learned my addr 172.21.15.165:0/3807271033 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:25.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.820+0000 7f992d400640 1 --1- 172.21.15.165:0/3807271033 >> v1:172.21.15.165:6789/0 conn(0x7f99280b9330 0x7f992806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60728/0 (socket says 172.21.15.165:60728) 2024-04-25T10:06:25.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f992ca00640 1 --2- 172.21.15.165:0/3807271033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f992815dc20 0x7f99280ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f9927e00640 1 -- 172.21.15.165:0/3807271033 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786366749 0 0) 0x7f992815e000 con 0x7f99280b9330 2024-04-25T10:06:25.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f9927e00640 1 -- 172.21.15.165:0/3807271033 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f990c003520 con 0x7f99280b9330 2024-04-25T10:06:25.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f991fe00640 1 -- 172.21.15.165:0/3807271033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f992815dc20 msgr2=0x7f99280ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:25.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f991fe00640 1 --2- 172.21.15.165:0/3807271033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f992815dc20 0x7f99280ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f991fe00640 1 -- 172.21.15.165:0/3807271033 >> v1:172.21.15.165:6789/0 conn(0x7f99280b9330 legacy=0x7f992806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.821+0000 7f991fe00640 1 -- 172.21.15.165:0/3807271033 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928161f40 con 0x7f9928161710 2024-04-25T10:06:25.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.822+0000 7f991fe00640 1 --2- 172.21.15.165:0/3807271033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9928161710 0x7f99280cebb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9910009870 tx=0x7f991002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59795e11f839c994 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f9927e00640 1 -- 172.21.15.165:0/3807271033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9910037070 con 0x7f9928161710 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f9927e00640 1 -- 172.21.15.165:0/3807271033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9910032440 con 0x7f9928161710 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f9927e00640 1 -- 172.21.15.165:0/3807271033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99100327f0 con 0x7f9928161710 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 -- 172.21.15.165:0/3807271033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9928161710 msgr2=0x7f99280cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 --2- 172.21.15.165:0/3807271033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9928161710 0x7f99280cebb0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f9910009870 tx=0x7f991002a270 comp rx=0 tx=0).stop 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 -- 172.21.15.165:0/3807271033 shutdown_connections 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 --2- 172.21.15.165:0/3807271033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9928161710 0x7f99280cebb0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 --2- 172.21.15.165:0/3807271033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f992815dc20 0x7f99280ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 -- 172.21.15.165:0/3807271033 >> 172.21.15.165:0/3807271033 conn(0x7f99280d0420 msgr2=0x7f99280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:25.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.823+0000 7f992f400640 1 -- 172.21.15.165:0/3807271033 shutdown_connections 2024-04-25T10:06:25.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.824+0000 7f992f400640 1 -- 172.21.15.165:0/3807271033 wait complete. 2024-04-25T10:06:25.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.825+0000 7f992f400640 1 Processor -- start 2024-04-25T10:06:25.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.825+0000 7f992f400640 1 -- start start 2024-04-25T10:06:25.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 0x7f992815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f992815dc20 0x7f99281600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9928161710 0x7f9928158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992f400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99280b5e50 con 0x7f99280b9330 2024-04-25T10:06:25.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9928162500 con 0x7f992815dc20 2024-04-25T10:06:25.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99280b5fd0 con 0x7f9928161710 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.826+0000 7f992ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 0x7f992815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 0x7f992815ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41800/0 (socket says 172.21.15.165:41800) 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 -- 172.21.15.165:0/3072794193 learned_addr learned my addr 172.21.15.165:0/3072794193 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 -- 172.21.15.165:0/3072794193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9928161710 msgr2=0x7f9928158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9928161710 0x7f9928158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 -- 172.21.15.165:0/3072794193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f992815dc20 msgr2=0x7f99281600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f992815dc20 0x7f99281600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 -- 172.21.15.165:0/3072794193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9928158750 con 0x7f99280b9330 2024-04-25T10:06:25.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f992ca00640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 0x7f992815ebc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f991800d620 tx=0x7f991800dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.827+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800b620 con 0x7f99280b9330 2024-04-25T10:06:25.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.828+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9918019070 con 0x7f99280b9330 2024-04-25T10:06:25.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.828+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99280caba0 con 0x7f99280b9330 2024-04-25T10:06:25.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.828+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991800ba00 con 0x7f99280b9330 2024-04-25T10:06:25.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.828+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99280cb160 con 0x7f99280b9330 2024-04-25T10:06:25.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.829+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99280d1150 con 0x7f99280b9330 2024-04-25T10:06:25.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.830+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991800bba0 con 0x7f99280b9330 2024-04-25T10:06:25.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.831+0000 7f9925600640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f990406b500 0x7f990406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:25.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.831+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f991808a780 con 0x7f99280b9330 2024-04-25T10:06:25.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.831+0000 7f991fe00640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f990406b500 0x7f990406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:25.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.832+0000 7f991fe00640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f990406b500 0x7f990406d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9910034000 tx=0x7f991002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:25.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:25.835+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9918059460 con 0x7f99280b9330 2024-04-25T10:06:26.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.074+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9928073a00 con 0x7f99280b9330 2024-04-25T10:06:26.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.593+0000 7f9925600640 1 -- 172.21.15.165:0/3072794193 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9918058d60 con 0x7f99280b9330 2024-04-25T10:06:26.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f990406b500 msgr2=0x7f990406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f990406b500 0x7f990406d9c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f9910034000 tx=0x7f991002c000 comp rx=0 tx=0).stop 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 msgr2=0x7f992815ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 0x7f992815ebc0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f991800d620 tx=0x7f991800dae0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 shutdown_connections 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f990406b500 0x7f990406d9c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9928161710 0x7f9928158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f992815dc20 0x7f99281600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 --2- 172.21.15.165:0/3072794193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99280b9330 0x7f992815ebc0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 >> 172.21.15.165:0/3072794193 conn(0x7f99280d0420 msgr2=0x7f99280d29d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 shutdown_connections 2024-04-25T10:06:26.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.597+0000 7f992f400640 1 -- 172.21.15.165:0/3072794193 wait complete. 2024-04-25T10:06:26.624 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:06:26.624 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:26.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.818+0000 7fddd9a00640 1 Processor -- start 2024-04-25T10:06:26.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.819+0000 7fddd9a00640 1 -- start start 2024-04-25T10:06:26.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fddd40b93c0 0x7fddd40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fddd40d04f0 0x7fddd40d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd4161df0 con 0x7fddd415db90 2024-04-25T10:06:26.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd40d1f50 con 0x7fddd40d04f0 2024-04-25T10:06:26.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd40d0940 con 0x7fddd40b93c0 2024-04-25T10:06:26.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fddd40d04f0 0x7fddd40d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fddd40d04f0 0x7fddd40d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33244/0 (socket says 172.21.15.165:33244) 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.820+0000 7fddd9000640 1 -- 172.21.15.165:0/2616934461 learned_addr learned my addr 172.21.15.165:0/2616934461 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.821+0000 7fddcb400640 1 --2- 172.21.15.165:0/2616934461 >> v2:172.21.15.172:3300/0 conn(0x7fddd40b93c0 0x7fddd40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.821+0000 7fddd9000640 1 -- 172.21.15.165:0/2616934461 >> v2:172.21.15.172:3300/0 conn(0x7fddd40b93c0 msgr2=0x7fddd40cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.821+0000 7fddd9000640 1 --2- 172.21.15.165:0/2616934461 >> v2:172.21.15.172:3300/0 conn(0x7fddd40b93c0 0x7fddd40cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.821+0000 7fddd9000640 1 -- 172.21.15.165:0/2616934461 >> v1:172.21.15.172:6789/0 conn(0x7fddd415db90 legacy=0x7fddd41615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:26.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.821+0000 7fddd9000640 1 -- 172.21.15.165:0/2616934461 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd41620d0 con 0x7fddd40d04f0 2024-04-25T10:06:26.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.822+0000 7fddd9000640 1 --2- 172.21.15.165:0/2616934461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40d23e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fddd40d1b30 tx=0x7fddcc02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c556c17e726917 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.822+0000 7fddd3400640 1 -- 172.21.15.165:0/2616934461 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc036070 con 0x7fddd40d04f0 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.822+0000 7fddd3400640 1 -- 172.21.15.165:0/2616934461 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddcc031600 con 0x7fddd40d04f0 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.822+0000 7fddd3400640 1 -- 172.21.15.165:0/2616934461 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddcc0319d0 con 0x7fddd40d04f0 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 -- 172.21.15.165:0/2616934461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 msgr2=0x7fddd40d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 --2- 172.21.15.165:0/2616934461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40d23e0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fddd40d1b30 tx=0x7fddcc02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 -- 172.21.15.165:0/2616934461 shutdown_connections 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 --2- 172.21.15.165:0/2616934461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40d23e0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 --2- 172.21.15.165:0/2616934461 >> v2:172.21.15.172:3300/0 conn(0x7fddd40b93c0 0x7fddd40cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 -- 172.21.15.165:0/2616934461 >> 172.21.15.165:0/2616934461 conn(0x7fddd40b52b0 msgr2=0x7fddd40b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 -- 172.21.15.165:0/2616934461 shutdown_connections 2024-04-25T10:06:26.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.823+0000 7fddd9a00640 1 -- 172.21.15.165:0/2616934461 wait complete. 2024-04-25T10:06:26.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.825+0000 7fddd9a00640 1 Processor -- start 2024-04-25T10:06:26.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.825+0000 7fddd9a00640 1 -- start start 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddd40b93c0 0x7fddd415e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddd415db90 0x7fddd40cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40b68d0 con 0x7fddd40d04f0 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40d1be0 con 0x7fddd415db90 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd40b6a50 con 0x7fddd40b93c0 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33258/0 (socket says 172.21.15.165:33258) 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 -- 172.21.15.165:0/4270735324 learned_addr learned my addr 172.21.15.165:0/4270735324 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9000640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddd415db90 0x7fddd40cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 -- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddd40b93c0 msgr2=0x7fddd415e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd3e00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddd40b93c0 0x7fddd415e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddd40b93c0 0x7fddd415e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 -- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddd415db90 msgr2=0x7fddd40cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:26.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddd415db90 0x7fddd40cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:26.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddcb400640 1 -- 172.21.15.165:0/4270735324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd415f440 con 0x7fddd40d04f0 2024-04-25T10:06:26.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.826+0000 7fddd9000640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddd415db90 0x7fddd40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:26.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.827+0000 7fddcb400640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40cbed0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00d830 tx=0x7fddbc00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.827+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00b660 con 0x7fddd40d04f0 2024-04-25T10:06:26.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.827+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddbc019070 con 0x7fddd40d04f0 2024-04-25T10:06:26.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.827+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddbc00ba20 con 0x7fddd40d04f0 2024-04-25T10:06:26.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.827+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd415f720 con 0x7fddd40d04f0 2024-04-25T10:06:26.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.827+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd414fcc0 con 0x7fddd40d04f0 2024-04-25T10:06:26.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.828+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd4073990 con 0x7fddd40d04f0 2024-04-25T10:06:26.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.829+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fddbc01e460 con 0x7fddd40d04f0 2024-04-25T10:06:26.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.830+0000 7fddd0c00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddb006b4d0 0x7fddb006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:26.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.830+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fddbc08aab0 con 0x7fddd40d04f0 2024-04-25T10:06:26.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.831+0000 7fddd3e00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddb006b4d0 0x7fddb006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:26.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.831+0000 7fddd3e00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddb006b4d0 0x7fddb006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fddc400a750 tx=0x7fddc4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:26.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:26.834+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fddbc059700 con 0x7fddd40d04f0 2024-04-25T10:06:27.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.064+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fddd4160480 con 0x7fddd40d04f0 2024-04-25T10:06:27.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.065+0000 7fddd0c00640 1 -- 172.21.15.165:0/4270735324 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fddbc059000 con 0x7fddd40d04f0 2024-04-25T10:06:27.071 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:27.071 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:27.071 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddb006b4d0 msgr2=0x7fddb006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddb006b4d0 0x7fddb006d990 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fddc400a750 tx=0x7fddc4008040 comp rx=0 tx=0).stop 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 msgr2=0x7fddd40cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40cbed0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fddbc00d830 tx=0x7fddbc00dcf0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 shutdown_connections 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fddb006b4d0 0x7fddb006d990 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fddd415db90 0x7fddd40cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fddd40d04f0 0x7fddd40cbed0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 --2- 172.21.15.165:0/4270735324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fddd40b93c0 0x7fddd415e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 >> 172.21.15.165:0/4270735324 conn(0x7fddd40b52b0 msgr2=0x7fddd41578e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.068+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 shutdown_connections 2024-04-25T10:06:27.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.069+0000 7fddd9a00640 1 -- 172.21.15.165:0/4270735324 wait complete. 2024-04-25T10:06:27.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:06:27.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.290+0000 7f608ee00640 1 Processor -- start 2024-04-25T10:06:27.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.290+0000 7f608ee00640 1 -- start start 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.292+0000 7f608ee00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60880b96d0 0x7f60880d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.292+0000 7f608ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880d2370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.292+0000 7f608ee00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608815e810 con 0x7f608815dd20 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.292+0000 7f608ee00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60880d1c00 con 0x7f60880b96d0 2024-04-25T10:06:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.292+0000 7f608ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f608815eac0 con 0x7f60880d0fe0 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.293+0000 7f6087e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f608815dd20 0x7f608815e100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:58100/0 (socket says 172.21.15.165:58100) 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.293+0000 7f608ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880d2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.293+0000 7f6087e00640 1 -- 172.21.15.165:0/1283134810 learned_addr learned my addr 172.21.15.165:0/1283134810 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:27.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.293+0000 7f6087400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60880b96d0 0x7f60880d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.293+0000 7f6086a00640 1 -- 172.21.15.165:0/1283134810 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 279102410 0 0) 0x7f608815e810 con 0x7f608815dd20 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.294+0000 7f6086a00640 1 -- 172.21.15.165:0/1283134810 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f606c003520 con 0x7f608815dd20 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.294+0000 7f608ce00640 1 -- 172.21.15.165:0/1283134810 >> v2:172.21.15.172:3300/0 conn(0x7f60880b96d0 msgr2=0x7f60880d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.294+0000 7f608ce00640 1 --2- 172.21.15.165:0/1283134810 >> v2:172.21.15.172:3300/0 conn(0x7f60880b96d0 0x7f60880d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.294+0000 7f608ce00640 1 -- 172.21.15.165:0/1283134810 >> v1:172.21.15.172:6789/0 conn(0x7f608815dd20 legacy=0x7f608815e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.294+0000 7f608ce00640 1 -- 172.21.15.165:0/1283134810 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f608815eda0 con 0x7f60880d0fe0 2024-04-25T10:06:27.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.295+0000 7f608ce00640 1 --2- 172.21.15.165:0/1283134810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880d2370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60800097a0 tx=0x7f608002a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9108069707eec9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.295+0000 7f6086a00640 1 -- 172.21.15.165:0/1283134810 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080037070 con 0x7f60880d0fe0 2024-04-25T10:06:27.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f6086a00640 1 -- 172.21.15.165:0/1283134810 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f608002ae40 con 0x7f60880d0fe0 2024-04-25T10:06:27.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f6086a00640 1 -- 172.21.15.165:0/1283134810 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6080032660 con 0x7f60880d0fe0 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 -- 172.21.15.165:0/1283134810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 msgr2=0x7f60880d2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 --2- 172.21.15.165:0/1283134810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880d2370 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f60800097a0 tx=0x7f608002a0f0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 -- 172.21.15.165:0/1283134810 shutdown_connections 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 --2- 172.21.15.165:0/1283134810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880d2370 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 --2- 172.21.15.165:0/1283134810 >> v2:172.21.15.172:3300/0 conn(0x7f60880b96d0 0x7f60880d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 -- 172.21.15.165:0/1283134810 >> 172.21.15.165:0/1283134810 conn(0x7f6088061a20 msgr2=0x7f6088061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 -- 172.21.15.165:0/1283134810 shutdown_connections 2024-04-25T10:06:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.296+0000 7f608ee00640 1 -- 172.21.15.165:0/1283134810 wait complete. 2024-04-25T10:06:27.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.298+0000 7f608ee00640 1 Processor -- start 2024-04-25T10:06:27.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.298+0000 7f608ee00640 1 -- start start 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.299+0000 7f608ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 0x7f60880aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.299+0000 7f608ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880ccb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.299+0000 7f608ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f608815dd20 0x7f60880cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.299+0000 7f608ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880d28b0 con 0x7f60880b96d0 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.299+0000 7f608ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60880d1e50 con 0x7f608815dd20 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.299+0000 7f608ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60880d2a30 con 0x7f60880d0fe0 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 0x7f60880aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 0x7f60880aa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33260/0 (socket says 172.21.15.165:33260) 2024-04-25T10:06:27.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 -- 172.21.15.165:0/2139067252 learned_addr learned my addr 172.21.15.165:0/2139067252 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 -- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 msgr2=0x7f60880ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880ccb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 -- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f608815dd20 msgr2=0x7f60880cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f608ce00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f608815dd20 0x7f60880cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f608815dd20 0x7f60880cd060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 -- 172.21.15.165:0/2139067252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60880cbf10 con 0x7f60880b96d0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.300+0000 7f6087e00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 0x7f60880aa710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f607000c400 tx=0x7f607000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.301+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070017070 con 0x7f60880b96d0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.301+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6070005ce0 con 0x7f60880b96d0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.301+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60880c7db0 con 0x7f60880b96d0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.301+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60700072d0 con 0x7f60880b96d0 2024-04-25T10:06:27.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.301+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60880c8370 con 0x7f60880b96d0 2024-04-25T10:06:27.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.302+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6088073990 con 0x7f60880b96d0 2024-04-25T10:06:27.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.303+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f607001a070 con 0x7f60880b96d0 2024-04-25T10:06:27.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.304+0000 7f607fe00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f606406b4d0 0x7f606406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.304+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6070089730 con 0x7f60880b96d0 2024-04-25T10:06:27.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.304+0000 7f6087400640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f606406b4d0 0x7f606406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.305+0000 7f6087400640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f606406b4d0 0x7f606406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6088153ae0 tx=0x7f607800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.308+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6070057d30 con 0x7f60880b96d0 2024-04-25T10:06:27.470 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:27.478 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:27.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.508+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f608815f9c0 con 0x7f60880b96d0 2024-04-25T10:06:27.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.509+0000 7f607fe00640 1 -- 172.21.15.165:0/2139067252 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+16978 (secure 0 0 0) 0x7f607005bdd0 con 0x7f60880b96d0 2024-04-25T10:06:27.515 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:27.515 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":54,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:24.563647+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:06:27.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f606406b4d0 msgr2=0x7f606406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f606406b4d0 0x7f606406d990 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6088153ae0 tx=0x7f607800b040 comp rx=0 tx=0).stop 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 msgr2=0x7f60880aa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 0x7f60880aa710 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f607000c400 tx=0x7f607000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 shutdown_connections 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f606406b4d0 0x7f606406d990 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f608815dd20 0x7f60880cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60880d0fe0 0x7f60880ccb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 --2- 172.21.15.165:0/2139067252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60880b96d0 0x7f60880aa710 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 >> 172.21.15.165:0/2139067252 conn(0x7f6088061a20 msgr2=0x7f6088160c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 shutdown_connections 2024-04-25T10:06:27.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.512+0000 7f608ee00640 1 -- 172.21.15.165:0/2139067252 wait complete. 2024-04-25T10:06:27.536 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:06:27.536 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:27.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.820+0000 7f97eea00640 1 Processor -- start 2024-04-25T10:06:27.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.820+0000 7f97eea00640 1 -- start start 2024-04-25T10:06:27.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97e815db70 0x7f97e80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97e815df50 0x7f97e80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97e8071650 con 0x7f97e80cec70 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eea00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e80cbd80 con 0x7f97e815db70 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97e8161910 con 0x7f97e815df50 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97e7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f97e815db70 0x7f97e80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eca00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f97e80cec70 0x7f97e80cc030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56694/0 (socket says 172.21.15.165:56694) 2024-04-25T10:06:27.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.822+0000 7f97eca00640 1 -- 172.21.15.165:0/2746889303 learned_addr learned my addr 172.21.15.165:0/2746889303 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:27.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.823+0000 7f97e7400640 1 --2- 172.21.15.165:0/2746889303 >> v2:172.21.15.172:3300/0 conn(0x7f97e815df50 0x7f97e80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.823+0000 7f97e7e00640 1 -- 172.21.15.165:0/2746889303 >> v2:172.21.15.172:3300/0 conn(0x7f97e815df50 msgr2=0x7f97e80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:27.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.823+0000 7f97e7e00640 1 --2- 172.21.15.165:0/2746889303 >> v2:172.21.15.172:3300/0 conn(0x7f97e815df50 0x7f97e80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.823+0000 7f97e7e00640 1 -- 172.21.15.165:0/2746889303 >> v1:172.21.15.165:6789/0 conn(0x7f97e80cec70 legacy=0x7f97e80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.823+0000 7f97e7e00640 1 -- 172.21.15.165:0/2746889303 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e80cf230 con 0x7f97e815db70 2024-04-25T10:06:27.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.823+0000 7f97e6a00640 1 -- 172.21.15.165:0/2746889303 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956281261 0 0) 0x7f97e0001490 con 0x7f97e80cec70 2024-04-25T10:06:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.824+0000 7f97e7e00640 1 --2- 172.21.15.165:0/2746889303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815db70 0x7f97e80cb840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f97e806f2a0 tx=0x7f97d802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1328adef5afb8f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97e6a00640 1 -- 172.21.15.165:0/2746889303 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8037070 con 0x7f97e815db70 2024-04-25T10:06:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97e6a00640 1 -- 172.21.15.165:0/2746889303 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97d8032440 con 0x7f97e815db70 2024-04-25T10:06:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97e6a00640 1 -- 172.21.15.165:0/2746889303 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97d8032810 con 0x7f97e815db70 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 -- 172.21.15.165:0/2746889303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815db70 msgr2=0x7f97e80cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 --2- 172.21.15.165:0/2746889303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815db70 0x7f97e80cb840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f97e806f2a0 tx=0x7f97d802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 -- 172.21.15.165:0/2746889303 shutdown_connections 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 --2- 172.21.15.165:0/2746889303 >> v2:172.21.15.172:3300/0 conn(0x7f97e815df50 0x7f97e80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 --2- 172.21.15.165:0/2746889303 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815db70 0x7f97e80cb840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 -- 172.21.15.165:0/2746889303 >> 172.21.15.165:0/2746889303 conn(0x7f97e806abe0 msgr2=0x7f97e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 -- 172.21.15.165:0/2746889303 shutdown_connections 2024-04-25T10:06:27.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.825+0000 7f97eea00640 1 -- 172.21.15.165:0/2746889303 wait complete. 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.827+0000 7f97eea00640 1 Processor -- start 2024-04-25T10:06:27.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.827+0000 7f97eea00640 1 -- start start 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.828+0000 7f97eea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97e80cec70 0x7f97e8151df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.828+0000 7f97eea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97e815db70 0x7f97e814fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.828+0000 7f97eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 0x7f97e8155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.828+0000 7f97eea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8162120 con 0x7f97e815df50 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.828+0000 7f97eea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8161e40 con 0x7f97e80cec70 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.828+0000 7f97eea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8162640 con 0x7f97e815db70 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 0x7f97e8155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 0x7f97e8155020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33282/0 (socket says 172.21.15.165:33282) 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 -- 172.21.15.165:0/3384239722 learned_addr learned my addr 172.21.15.165:0/3384239722 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97e7e00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97e80cec70 0x7f97e8151df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 -- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97e815db70 msgr2=0x7f97e814fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97e815db70 0x7f97e814fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 -- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97e80cec70 msgr2=0x7f97e8151df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97e80cec70 0x7f97e8151df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 -- 172.21.15.165:0/3384239722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e8150c90 con 0x7f97e815df50 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.829+0000 7f97eca00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 0x7f97e8155020 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c700 tx=0x7f97e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.830+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0018070 con 0x7f97e815df50 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.830+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e000ce10 con 0x7f97e815df50 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.830+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e000a860 con 0x7f97e815df50 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.830+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e81503f0 con 0x7f97e815df50 2024-04-25T10:06:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.830+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e81509b0 con 0x7f97e815df50 2024-04-25T10:06:27.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.831+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e8073990 con 0x7f97e815df50 2024-04-25T10:06:27.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.832+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97e001b070 con 0x7f97e815df50 2024-04-25T10:06:27.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.833+0000 7f97dfe00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97c806b520 0x7f97c806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:27.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.833+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97e0089a80 con 0x7f97e815df50 2024-04-25T10:06:27.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.833+0000 7f97e7e00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97c806b520 0x7f97c806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:27.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.834+0000 7f97e7e00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97c806b520 0x7f97c806d9e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f97d8034000 tx=0x7f97d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:27.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:27.838+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97e008be20 con 0x7f97e815df50 2024-04-25T10:06:28.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.026+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f97e8158d10 con 0x7f97c806b520 2024-04-25T10:06:28.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.037+0000 7f97dfe00640 1 -- 172.21.15.165:0/3384239722 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240078 (secure 0 0 0) 0x7f97e8158d10 con 0x7f97c806b520 2024-04-25T10:06:28.045 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:28.047 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97c806b520 msgr2=0x7f97c806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97c806b520 0x7f97c806d9e0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f97d8034000 tx=0x7f97d802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 msgr2=0x7f97e8155020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 0x7f97e8155020 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f97e000c700 tx=0x7f97e000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 shutdown_connections 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97c806b520 0x7f97c806d9e0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97e815df50 0x7f97e8155020 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97e815db70 0x7f97e814fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 --2- 172.21.15.165:0/3384239722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97e80cec70 0x7f97e8151df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.043+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 >> 172.21.15.165:0/3384239722 conn(0x7f97e806abe0 msgr2=0x7f97e80cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.044+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 shutdown_connections 2024-04-25T10:06:28.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.044+0000 7f97eea00640 1 -- 172.21.15.165:0/3384239722 wait complete. 2024-04-25T10:06:28.069 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":184,"stamp":"2024-04-25T10:06:26.215602+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":314996,"kb_used_data":5872,"kb_used_omap":12,"kb_used_meta":308979,"kb_avail":754659724,"statfs":{"total":773094113280,"available":772771557376,"internally_reserved":0,"allocated":6012928,"data_stored":3317854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":316395058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":17,"apply_latency_ms":17,"commit_latency_ns":17000000,"apply_latency_ns":17000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.682660"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.544565+0000","last_change":"2024-04-25T10:06:21.544565+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.544565+0000","last_undegraded":"2024-04-25T10:06:21.544565+0000","last_fullsized":"2024-04-25T10:06:21.544565+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.545778+0000","last_change":"2024-04-25T10:06:21.545778+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.545778+0000","last_undegraded":"2024-04-25T10:06:21.545778+0000","last_fullsized":"2024-04-25T10:06:21.545778+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.545079+0000","last_change":"2024-04-25T10:06:21.545079+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.545079+0000","last_undegraded":"2024-04-25T10:06:21.545079+0000","last_fullsized":"2024-04-25T10:06:21.545079+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.546840+0000","last_change":"2024-04-25T10:06:21.546840+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.546840+0000","last_undegraded":"2024-04-25T10:06:21.546840+0000","last_fullsized":"2024-04-25T10:06:21.546840+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.542237+0000","last_change":"2024-04-25T10:06:21.542237+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.542237+0000","last_undegraded":"2024-04-25T10:06:21.542237+0000","last_fullsized":"2024-04-25T10:06:21.542237+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.537884+0000","last_change":"2024-04-25T10:06:21.537884+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.537884+0000","last_undegraded":"2024-04-25T10:06:21.537884+0000","last_fullsized":"2024-04-25T10:06:21.537884+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.539747+0000","last_change":"2024-04-25T10:06:21.539747+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.539747+0000","last_undegraded":"2024-04-25T10:06:21.539747+0000","last_fullsized":"2024-04-25T10:06:21.539747+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.541303+0000","last_change":"2024-04-25T10:06:21.541303+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.541303+0000","last_undegraded":"2024-04-25T10:06:21.541303+0000","last_fullsized":"2024-04-25T10:06:21.541303+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.547078+0000","last_change":"2024-04-25T10:06:21.547078+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.547078+0000","last_undegraded":"2024-04-25T10:06:21.547078+0000","last_fullsized":"2024-04-25T10:06:21.547078+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.537831+0000","last_change":"2024-04-25T10:06:21.537831+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.537831+0000","last_undegraded":"2024-04-25T10:06:21.537831+0000","last_fullsized":"2024-04-25T10:06:21.537831+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.546010+0000","last_change":"2024-04-25T10:06:21.546010+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.546010+0000","last_undegraded":"2024-04-25T10:06:21.546010+0000","last_fullsized":"2024-04-25T10:06:21.546010+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.546072+0000","last_change":"2024-04-25T10:06:21.546072+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.546072+0000","last_undegraded":"2024-04-25T10:06:21.546072+0000","last_fullsized":"2024-04-25T10:06:21.546072+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.541351+0000","last_change":"2024-04-25T10:06:21.541351+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.541351+0000","last_undegraded":"2024-04-25T10:06:21.541351+0000","last_fullsized":"2024-04-25T10:06:21.541351+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.537787+0000","last_change":"2024-04-25T10:06:21.537787+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.537787+0000","last_undegraded":"2024-04-25T10:06:21.537787+0000","last_fullsized":"2024-04-25T10:06:21.537787+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.543988+0000","last_change":"2024-04-25T10:06:21.543988+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.543988+0000","last_undegraded":"2024-04-25T10:06:21.543988+0000","last_fullsized":"2024-04-25T10:06:21.543988+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.543114+0000","last_change":"2024-04-25T10:06:21.543114+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.543114+0000","last_undegraded":"2024-04-25T10:06:21.543114+0000","last_fullsized":"2024-04-25T10:06:21.543114+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.541901+0000","last_change":"2024-04-25T10:06:21.541901+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.541901+0000","last_undegraded":"2024-04-25T10:06:21.541901+0000","last_fullsized":"2024-04-25T10:06:21.541901+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.538287+0000","last_change":"2024-04-25T10:06:21.538287+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.538287+0000","last_undegraded":"2024-04-25T10:06:21.538287+0000","last_fullsized":"2024-04-25T10:06:21.538287+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.539984+0000","last_change":"2024-04-25T10:06:21.539984+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.539984+0000","last_undegraded":"2024-04-25T10:06:21.539984+0000","last_fullsized":"2024-04-25T10:06:21.539984+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.539987+0000","last_change":"2024-04-25T10:06:21.539987+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.539987+0000","last_undegraded":"2024-04-25T10:06:21.539987+0000","last_fullsized":"2024-04-25T10:06:21.539987+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.545048+0000","last_change":"2024-04-25T10:06:21.545048+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.545048+0000","last_undegraded":"2024-04-25T10:06:21.545048+0000","last_fullsized":"2024-04-25T10:06:21.545048+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.548338+0000","last_change":"2024-04-25T10:06:21.548338+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.548338+0000","last_undegraded":"2024-04-25T10:06:21.548338+0000","last_fullsized":"2024-04-25T10:06:21.548338+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.541298+0000","last_change":"2024-04-25T10:06:21.541298+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.541298+0000","last_undegraded":"2024-04-25T10:06:21.541298+0000","last_fullsized":"2024-04-25T10:06:21.541298+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.538267+0000","last_change":"2024-04-25T10:06:21.538267+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.538267+0000","last_undegraded":"2024-04-25T10:06:21.538267+0000","last_fullsized":"2024-04-25T10:06:21.538267+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.543436+0000","last_change":"2024-04-25T10:06:21.543436+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.543436+0000","last_undegraded":"2024-04-25T10:06:21.543436+0000","last_fullsized":"2024-04-25T10:06:21.543436+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.544003+0000","last_change":"2024-04-25T10:06:21.544003+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.544003+0000","last_undegraded":"2024-04-25T10:06:21.544003+0000","last_fullsized":"2024-04-25T10:06:21.544003+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.542891+0000","last_change":"2024-04-25T10:06:21.542891+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.542891+0000","last_undegraded":"2024-04-25T10:06:21.542891+0000","last_fullsized":"2024-04-25T10:06:21.542891+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.545202+0000","last_change":"2024-04-25T10:06:21.545202+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.545202+0000","last_undegraded":"2024-04-25T10:06:21.545202+0000","last_fullsized":"2024-04-25T10:06:21.545202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.547498+0000","last_change":"2024-04-25T10:06:21.547498+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.547498+0000","last_undegraded":"2024-04-25T10:06:21.547498+0000","last_fullsized":"2024-04-25T10:06:21.547498+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.537788+0000","last_change":"2024-04-25T10:06:21.537788+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.537788+0000","last_undegraded":"2024-04-25T10:06:21.537788+0000","last_fullsized":"2024-04-25T10:06:21.537788+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.546337+0000","last_change":"2024-04-25T10:06:21.546337+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.546337+0000","last_undegraded":"2024-04-25T10:06:21.546337+0000","last_fullsized":"2024-04-25T10:06:21.546337+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:06:22.530503+0000","last_change":"2024-04-25T10:06:22.530503+0000","last_active":"2024-04-25T10:06:22.530503+0000","last_peered":"2024-04-25T10:06:22.530503+0000","last_clean":"2024-04-25T10:06:22.530503+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:22.530503+0000","last_undegraded":"2024-04-25T10:06:22.530503+0000","last_fullsized":"2024-04-25T10:06:22.530503+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":1,"reported_epoch":51,"state":"creating+peering","last_fresh":"2024-04-25T10:06:21.541899+0000","last_change":"2024-04-25T10:06:21.541899+0000","last_active":"2024-04-25T10:06:21.525472+0000","last_peered":"2024-04-25T10:06:21.525472+0000","last_clean":"2024-04-25T10:06:21.525472+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:06:21.541899+0000","last_undegraded":"2024-04-25T10:06:21.541899+0000","last_fullsized":"2024-04-25T10:06:21.541899+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":231,"reported_epoch":51,"state":"active+clean","last_fresh":"2024-04-25T10:06:21.548883+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:06:21.548883+0000","last_peered":"2024-04-25T10:06:21.548883+0000","last_clean":"2024-04-25T10:06:21.548883+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:06:21.548883+0000","last_undegraded":"2024-04-25T10:06:21.548883+0000","last_fullsized":"2024-04-25T10:06:21.548883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36520,"kb_used_data":1172,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335320,"statfs":{"total":96636764160,"available":96599367680,"internally_reserved":0,"allocated":1200128,"data_stored":859060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":606208,"data_stored":268692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574906,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35944,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335896,"statfs":{"total":96636764160,"available":96599957504,"internally_reserved":0,"allocated":606208,"data_stored":268692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40092,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331748,"statfs":{"total":96636764160,"available":96595709952,"internally_reserved":0,"allocated":606208,"data_stored":268692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35932,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335908,"statfs":{"total":96636764160,"available":96599969792,"internally_reserved":0,"allocated":606208,"data_stored":268692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35928,"kb_used_data":580,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335912,"statfs":{"total":96636764160,"available":96599973888,"internally_reserved":0,"allocated":593920,"data_stored":262483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35940,"kb_used_data":592,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335900,"statfs":{"total":96636764160,"available":96599961600,"internally_reserved":0,"allocated":606208,"data_stored":268692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574906,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58708,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313132,"statfs":{"total":96636764160,"available":96576647168,"internally_reserved":0,"allocated":1187840,"data_stored":852851,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:28.074 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:06:28.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.313+0000 7fb284e00640 1 Processor -- start 2024-04-25T10:06:28.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.313+0000 7fb284e00640 1 -- start start 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb284e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2800aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb284e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2800cec70 0x7fb2800b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb284e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2800cf330 con 0x7fb2800cc6c0 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb284e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb280072340 con 0x7fb2800cec70 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb284e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800b2b90 con 0x7fb280165680 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb275400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2800cec70 0x7fb2800b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb275400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2800cec70 0x7fb2800b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33288/0 (socket says 172.21.15.165:33288) 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.315+0000 7fb275400640 1 -- 172.21.15.165:0/3072665164 learned_addr learned my addr 172.21.15.165:0/3072665164 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.316+0000 7fb27de00640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2800aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.316+0000 7fb275400640 1 -- 172.21.15.165:0/3072665164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 msgr2=0x7fb2800aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.316+0000 7fb275400640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2800aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.316+0000 7fb275400640 1 -- 172.21.15.165:0/3072665164 >> v1:172.21.15.165:6789/0 conn(0x7fb2800cc6c0 legacy=0x7fb2800cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.316+0000 7fb275400640 1 -- 172.21.15.165:0/3072665164 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb280071d80 con 0x7fb2800cec70 2024-04-25T10:06:28.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.316+0000 7fb27de00640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2800aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.317+0000 7fb275400640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b2650 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb268009e50 tx=0x7fb26802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13928015a401e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.317+0000 7fb27d400640 1 -- 172.21.15.165:0/3072665164 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268036070 con 0x7fb2800cec70 2024-04-25T10:06:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.317+0000 7fb27d400640 1 -- 172.21.15.165:0/3072665164 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268031600 con 0x7fb2800cec70 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.317+0000 7fb27d400640 1 -- 172.21.15.165:0/3072665164 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2680319d0 con 0x7fb2800cec70 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 -- 172.21.15.165:0/3072665164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 msgr2=0x7fb2800b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b2650 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb268009e50 tx=0x7fb26802a680 comp rx=0 tx=0).stop 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 -- 172.21.15.165:0/3072665164 shutdown_connections 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b2650 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 --2- 172.21.15.165:0/3072665164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2800aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 -- 172.21.15.165:0/3072665164 >> 172.21.15.165:0/3072665164 conn(0x7fb28006abe0 msgr2=0x7fb28006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 -- 172.21.15.165:0/3072665164 shutdown_connections 2024-04-25T10:06:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.318+0000 7fb284e00640 1 -- 172.21.15.165:0/3072665164 wait complete. 2024-04-25T10:06:28.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.320+0000 7fb284e00640 1 Processor -- start 2024-04-25T10:06:28.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.320+0000 7fb284e00640 1 -- start start 2024-04-25T10:06:28.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb284e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2800cc6c0 0x7fb280153700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb284e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb284e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2801f4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb284e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2800cf0d0 con 0x7fb2800cec70 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb284e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb280071670 con 0x7fb2800cc6c0 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb284e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb28006ea30 con 0x7fb280165680 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33290/0 (socket says 172.21.15.165:33290) 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 -- 172.21.15.165:0/1546803373 learned_addr learned my addr 172.21.15.165:0/1546803373 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb27e800640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2801f4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb27de00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2800cc6c0 0x7fb280153700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 -- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 msgr2=0x7fb2801f4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2801f4e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 -- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2800cc6c0 msgr2=0x7fb280153700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2800cc6c0 0x7fb280153700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.321+0000 7fb275400640 1 -- 172.21.15.165:0/1546803373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2801fb900 con 0x7fb2800cec70 2024-04-25T10:06:28.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.322+0000 7fb275400640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b42a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb268009f70 tx=0x7fb268004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.322+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb26803f020 con 0x7fb2800cec70 2024-04-25T10:06:28.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.322+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb268037040 con 0x7fb2800cec70 2024-04-25T10:06:28.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.322+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb268036070 con 0x7fb2800cec70 2024-04-25T10:06:28.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.322+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2801fbbe0 con 0x7fb2800cec70 2024-04-25T10:06:28.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.322+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2801fc1a0 con 0x7fb2800cec70 2024-04-25T10:06:28.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.324+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb280073990 con 0x7fb2800cec70 2024-04-25T10:06:28.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.324+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb268042020 con 0x7fb2800cec70 2024-04-25T10:06:28.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.325+0000 7fb276a00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.325+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2680aa9a0 con 0x7fb2800cec70 2024-04-25T10:06:28.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.326+0000 7fb27de00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.326+0000 7fb27de00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb270008850 tx=0x7fb270008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.330+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb268049030 con 0x7fb2800cec70 2024-04-25T10:06:28.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.523+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb280165af0 con 0x7fb25c06b4d0 2024-04-25T10:06:28.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.541+0000 7fb276a00640 1 -- 172.21.15.165:0/1546803373 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251957 (secure 0 0 0) 0x7fb280165af0 con 0x7fb25c06b4d0 2024-04-25T10:06:28.552 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:06:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb25c06b4d0 msgr2=0x7fb25c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb25c06b4d0 0x7fb25c06d990 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb270008850 tx=0x7fb270008040 comp rx=0 tx=0).stop 2024-04-25T10:06:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 msgr2=0x7fb2800b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b42a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb268009f70 tx=0x7fb268004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 shutdown_connections 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb25c06b4d0 0x7fb25c06d990 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb280165680 0x7fb2801f4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2800cec70 0x7fb2800b42a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 --2- 172.21.15.165:0/1546803373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2800cc6c0 0x7fb280153700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 >> 172.21.15.165:0/1546803373 conn(0x7fb28006abe0 msgr2=0x7fb28015a200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.547+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 shutdown_connections 2024-04-25T10:06:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.548+0000 7fb284e00640 1 -- 172.21.15.165:0/1546803373 wait complete. 2024-04-25T10:06:28.572 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":185,"stamp":"2024-04-25T10:06:28.216202+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":315368,"kb_used_data":6248,"kb_used_omap":12,"kb_used_meta":308979,"kb_avail":754659352,"statfs":{"total":773094113280,"available":772771176448,"internally_reserved":0,"allocated":6397952,"data_stored":3499300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":316395058},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001877"},"pg_stats":[{"pgid":"8.3f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.571219+0000","last_change":"2024-04-25T10:06:23.587863+0000","last_active":"2024-04-25T10:06:24.571219+0000","last_peered":"2024-04-25T10:06:24.571219+0000","last_clean":"2024-04-25T10:06:24.571219+0000","last_became_active":"2024-04-25T10:06:23.586781+0000","last_became_peered":"2024-04-25T10:06:23.586781+0000","last_unstale":"2024-04-25T10:06:24.571219+0000","last_undegraded":"2024-04-25T10:06:24.571219+0000","last_fullsized":"2024-04-25T10:06:24.571219+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:55.374247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612423+0000","last_change":"2024-04-25T10:06:23.608923+0000","last_active":"2024-04-25T10:06:24.612423+0000","last_peered":"2024-04-25T10:06:24.612423+0000","last_clean":"2024-04-25T10:06:24.612423+0000","last_became_active":"2024-04-25T10:06:23.607715+0000","last_became_peered":"2024-04-25T10:06:23.607715+0000","last_unstale":"2024-04-25T10:06:24.612423+0000","last_undegraded":"2024-04-25T10:06:24.612423+0000","last_fullsized":"2024-04-25T10:06:24.612423+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:13.019865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605956+0000","last_change":"2024-04-25T10:06:23.606599+0000","last_active":"2024-04-25T10:06:24.605956+0000","last_peered":"2024-04-25T10:06:24.605956+0000","last_clean":"2024-04-25T10:06:24.605956+0000","last_became_active":"2024-04-25T10:06:23.605285+0000","last_became_peered":"2024-04-25T10:06:23.605285+0000","last_unstale":"2024-04-25T10:06:24.605956+0000","last_undegraded":"2024-04-25T10:06:24.605956+0000","last_fullsized":"2024-04-25T10:06:24.605956+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:59.820651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612171+0000","last_change":"2024-04-25T10:06:23.606876+0000","last_active":"2024-04-25T10:06:24.612171+0000","last_peered":"2024-04-25T10:06:24.612171+0000","last_clean":"2024-04-25T10:06:24.612171+0000","last_became_active":"2024-04-25T10:06:23.604408+0000","last_became_peered":"2024-04-25T10:06:23.604408+0000","last_unstale":"2024-04-25T10:06:24.612171+0000","last_undegraded":"2024-04-25T10:06:24.612171+0000","last_fullsized":"2024-04-25T10:06:24.612171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.934457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.652921+0000","last_change":"2024-04-25T10:06:23.612896+0000","last_active":"2024-04-25T10:06:24.652921+0000","last_peered":"2024-04-25T10:06:24.652921+0000","last_clean":"2024-04-25T10:06:24.652921+0000","last_became_active":"2024-04-25T10:06:23.611064+0000","last_became_peered":"2024-04-25T10:06:23.611064+0000","last_unstale":"2024-04-25T10:06:24.652921+0000","last_undegraded":"2024-04-25T10:06:24.652921+0000","last_fullsized":"2024-04-25T10:06:24.652921+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:31.853591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570530+0000","last_change":"2024-04-25T10:06:23.602483+0000","last_active":"2024-04-25T10:06:24.570530+0000","last_peered":"2024-04-25T10:06:24.570530+0000","last_clean":"2024-04-25T10:06:24.570530+0000","last_became_active":"2024-04-25T10:06:23.602186+0000","last_became_peered":"2024-04-25T10:06:23.602186+0000","last_unstale":"2024-04-25T10:06:24.570530+0000","last_undegraded":"2024-04-25T10:06:24.570530+0000","last_fullsized":"2024-04-25T10:06:24.570530+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:13.187460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604575+0000","last_change":"2024-04-25T10:06:23.613760+0000","last_active":"2024-04-25T10:06:24.604575+0000","last_peered":"2024-04-25T10:06:24.604575+0000","last_clean":"2024-04-25T10:06:24.604575+0000","last_became_active":"2024-04-25T10:06:23.612905+0000","last_became_peered":"2024-04-25T10:06:23.612905+0000","last_unstale":"2024-04-25T10:06:24.604575+0000","last_undegraded":"2024-04-25T10:06:24.604575+0000","last_fullsized":"2024-04-25T10:06:24.604575+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:24.530151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184152+0000","last_change":"2024-04-25T10:06:23.603661+0000","last_active":"2024-04-25T10:06:26.184152+0000","last_peered":"2024-04-25T10:06:26.184152+0000","last_clean":"2024-04-25T10:06:26.184152+0000","last_became_active":"2024-04-25T10:06:23.600801+0000","last_became_peered":"2024-04-25T10:06:23.600801+0000","last_unstale":"2024-04-25T10:06:26.184152+0000","last_undegraded":"2024-04-25T10:06:26.184152+0000","last_fullsized":"2024-04-25T10:06:26.184152+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:08.983034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185470+0000","last_change":"2024-04-25T10:06:23.619952+0000","last_active":"2024-04-25T10:06:26.185470+0000","last_peered":"2024-04-25T10:06:26.185470+0000","last_clean":"2024-04-25T10:06:26.185470+0000","last_became_active":"2024-04-25T10:06:23.619623+0000","last_became_peered":"2024-04-25T10:06:23.619623+0000","last_unstale":"2024-04-25T10:06:26.185470+0000","last_undegraded":"2024-04-25T10:06:26.185470+0000","last_fullsized":"2024-04-25T10:06:26.185470+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:19.529113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570907+0000","last_change":"2024-04-25T10:06:23.597224+0000","last_active":"2024-04-25T10:06:24.570907+0000","last_peered":"2024-04-25T10:06:24.570907+0000","last_clean":"2024-04-25T10:06:24.570907+0000","last_became_active":"2024-04-25T10:06:23.596819+0000","last_became_peered":"2024-04-25T10:06:23.596819+0000","last_unstale":"2024-04-25T10:06:24.570907+0000","last_undegraded":"2024-04-25T10:06:24.570907+0000","last_fullsized":"2024-04-25T10:06:24.570907+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:46.670683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.186482+0000","last_change":"2024-04-25T10:06:23.606620+0000","last_active":"2024-04-25T10:06:26.186482+0000","last_peered":"2024-04-25T10:06:26.186482+0000","last_clean":"2024-04-25T10:06:26.186482+0000","last_became_active":"2024-04-25T10:06:23.606110+0000","last_became_peered":"2024-04-25T10:06:23.606110+0000","last_unstale":"2024-04-25T10:06:26.186482+0000","last_undegraded":"2024-04-25T10:06:26.186482+0000","last_fullsized":"2024-04-25T10:06:26.186482+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:12:23.365239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185385+0000","last_change":"2024-04-25T10:06:23.601349+0000","last_active":"2024-04-25T10:06:26.185385+0000","last_peered":"2024-04-25T10:06:26.185385+0000","last_clean":"2024-04-25T10:06:26.185385+0000","last_became_active":"2024-04-25T10:06:23.599935+0000","last_became_peered":"2024-04-25T10:06:23.599935+0000","last_unstale":"2024-04-25T10:06:26.185385+0000","last_undegraded":"2024-04-25T10:06:26.185385+0000","last_fullsized":"2024-04-25T10:06:26.185385+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:15.874567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570902+0000","last_change":"2024-04-25T10:06:23.603336+0000","last_active":"2024-04-25T10:06:24.570902+0000","last_peered":"2024-04-25T10:06:24.570902+0000","last_clean":"2024-04-25T10:06:24.570902+0000","last_became_active":"2024-04-25T10:06:23.603085+0000","last_became_peered":"2024-04-25T10:06:23.603085+0000","last_unstale":"2024-04-25T10:06:24.570902+0000","last_undegraded":"2024-04-25T10:06:24.570902+0000","last_fullsized":"2024-04-25T10:06:24.570902+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:18:38.864764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605401+0000","last_change":"2024-04-25T10:06:23.602400+0000","last_active":"2024-04-25T10:06:24.605401+0000","last_peered":"2024-04-25T10:06:24.605401+0000","last_clean":"2024-04-25T10:06:24.605401+0000","last_became_active":"2024-04-25T10:06:23.602042+0000","last_became_peered":"2024-04-25T10:06:23.602042+0000","last_unstale":"2024-04-25T10:06:24.605401+0000","last_undegraded":"2024-04-25T10:06:24.605401+0000","last_fullsized":"2024-04-25T10:06:24.605401+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:21:14.024374+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.611681+0000","last_change":"2024-04-25T10:06:23.607433+0000","last_active":"2024-04-25T10:06:24.611681+0000","last_peered":"2024-04-25T10:06:24.611681+0000","last_clean":"2024-04-25T10:06:24.611681+0000","last_became_active":"2024-04-25T10:06:23.604510+0000","last_became_peered":"2024-04-25T10:06:23.604510+0000","last_unstale":"2024-04-25T10:06:24.611681+0000","last_undegraded":"2024-04-25T10:06:24.611681+0000","last_fullsized":"2024-04-25T10:06:24.611681+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.280133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.30","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570246+0000","last_change":"2024-04-25T10:06:23.602836+0000","last_active":"2024-04-25T10:06:24.570246+0000","last_peered":"2024-04-25T10:06:24.570246+0000","last_clean":"2024-04-25T10:06:24.570246+0000","last_became_active":"2024-04-25T10:06:23.602540+0000","last_became_peered":"2024-04-25T10:06:23.602540+0000","last_unstale":"2024-04-25T10:06:24.570246+0000","last_undegraded":"2024-04-25T10:06:24.570246+0000","last_fullsized":"2024-04-25T10:06:24.570246+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:55.648921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606788+0000","last_change":"2024-04-25T10:06:23.611892+0000","last_active":"2024-04-25T10:06:24.606788+0000","last_peered":"2024-04-25T10:06:24.606788+0000","last_clean":"2024-04-25T10:06:24.606788+0000","last_became_active":"2024-04-25T10:06:23.611545+0000","last_became_peered":"2024-04-25T10:06:23.611545+0000","last_unstale":"2024-04-25T10:06:24.606788+0000","last_undegraded":"2024-04-25T10:06:24.606788+0000","last_fullsized":"2024-04-25T10:06:24.606788+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:15.874567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612683+0000","last_change":"2024-04-25T10:06:23.609133+0000","last_active":"2024-04-25T10:06:24.612683+0000","last_peered":"2024-04-25T10:06:24.612683+0000","last_clean":"2024-04-25T10:06:24.612683+0000","last_became_active":"2024-04-25T10:06:23.608389+0000","last_became_peered":"2024-04-25T10:06:23.608389+0000","last_unstale":"2024-04-25T10:06:24.612683+0000","last_undegraded":"2024-04-25T10:06:24.612683+0000","last_fullsized":"2024-04-25T10:06:24.612683+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:03:24.530151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605938+0000","last_change":"2024-04-25T10:06:23.606849+0000","last_active":"2024-04-25T10:06:24.605938+0000","last_peered":"2024-04-25T10:06:24.605938+0000","last_clean":"2024-04-25T10:06:24.605938+0000","last_became_active":"2024-04-25T10:06:23.605847+0000","last_became_peered":"2024-04-25T10:06:23.605847+0000","last_unstale":"2024-04-25T10:06:24.605938+0000","last_undegraded":"2024-04-25T10:06:24.605938+0000","last_fullsized":"2024-04-25T10:06:24.605938+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:08.983034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579048+0000","last_change":"2024-04-25T10:06:23.596905+0000","last_active":"2024-04-25T10:06:24.579048+0000","last_peered":"2024-04-25T10:06:24.579048+0000","last_clean":"2024-04-25T10:06:24.579048+0000","last_became_active":"2024-04-25T10:06:23.596648+0000","last_became_peered":"2024-04-25T10:06:23.596648+0000","last_unstale":"2024-04-25T10:06:24.579048+0000","last_undegraded":"2024-04-25T10:06:24.579048+0000","last_fullsized":"2024-04-25T10:06:24.579048+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:51.020669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.578755+0000","last_change":"2024-04-25T10:06:23.601603+0000","last_active":"2024-04-25T10:06:24.578755+0000","last_peered":"2024-04-25T10:06:24.578755+0000","last_clean":"2024-04-25T10:06:24.578755+0000","last_became_active":"2024-04-25T10:06:23.601333+0000","last_became_peered":"2024-04-25T10:06:23.601333+0000","last_unstale":"2024-04-25T10:06:24.578755+0000","last_undegraded":"2024-04-25T10:06:24.578755+0000","last_fullsized":"2024-04-25T10:06:24.578755+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:31.853591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184016+0000","last_change":"2024-04-25T10:06:23.603922+0000","last_active":"2024-04-25T10:06:26.184016+0000","last_peered":"2024-04-25T10:06:26.184016+0000","last_clean":"2024-04-25T10:06:26.184016+0000","last_became_active":"2024-04-25T10:06:23.603615+0000","last_became_peered":"2024-04-25T10:06:23.603615+0000","last_unstale":"2024-04-25T10:06:26.184016+0000","last_undegraded":"2024-04-25T10:06:26.184016+0000","last_fullsized":"2024-04-25T10:06:26.184016+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:09.661552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185468+0000","last_change":"2024-04-25T10:06:23.605783+0000","last_active":"2024-04-25T10:06:26.185468+0000","last_peered":"2024-04-25T10:06:26.185468+0000","last_clean":"2024-04-25T10:06:26.185468+0000","last_became_active":"2024-04-25T10:06:23.605434+0000","last_became_peered":"2024-04-25T10:06:23.605434+0000","last_unstale":"2024-04-25T10:06:26.185468+0000","last_undegraded":"2024-04-25T10:06:26.185468+0000","last_fullsized":"2024-04-25T10:06:26.185468+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:19.437316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604819+0000","last_change":"2024-04-25T10:06:23.608189+0000","last_active":"2024-04-25T10:06:24.604819+0000","last_peered":"2024-04-25T10:06:24.604819+0000","last_clean":"2024-04-25T10:06:24.604819+0000","last_became_active":"2024-04-25T10:06:23.607737+0000","last_became_peered":"2024-04-25T10:06:23.607737+0000","last_unstale":"2024-04-25T10:06:24.604819+0000","last_undegraded":"2024-04-25T10:06:24.604819+0000","last_fullsized":"2024-04-25T10:06:24.604819+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:49.070441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606684+0000","last_change":"2024-04-25T10:06:23.603185+0000","last_active":"2024-04-25T10:06:24.606684+0000","last_peered":"2024-04-25T10:06:24.606684+0000","last_clean":"2024-04-25T10:06:24.606684+0000","last_became_active":"2024-04-25T10:06:23.602905+0000","last_became_peered":"2024-04-25T10:06:23.602905+0000","last_unstale":"2024-04-25T10:06:24.606684+0000","last_undegraded":"2024-04-25T10:06:24.606684+0000","last_fullsized":"2024-04-25T10:06:24.606684+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:43.052798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184552+0000","last_change":"2024-04-25T10:06:23.605101+0000","last_active":"2024-04-25T10:06:26.184552+0000","last_peered":"2024-04-25T10:06:26.184552+0000","last_clean":"2024-04-25T10:06:26.184552+0000","last_became_active":"2024-04-25T10:06:23.604819+0000","last_became_peered":"2024-04-25T10:06:23.604819+0000","last_unstale":"2024-04-25T10:06:26.184552+0000","last_undegraded":"2024-04-25T10:06:26.184552+0000","last_fullsized":"2024-04-25T10:06:26.184552+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.086235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.571121+0000","last_change":"2024-04-25T10:06:23.602140+0000","last_active":"2024-04-25T10:06:24.571121+0000","last_peered":"2024-04-25T10:06:24.571121+0000","last_clean":"2024-04-25T10:06:24.571121+0000","last_became_active":"2024-04-25T10:06:23.601691+0000","last_became_peered":"2024-04-25T10:06:23.601691+0000","last_unstale":"2024-04-25T10:06:24.571121+0000","last_undegraded":"2024-04-25T10:06:24.571121+0000","last_fullsized":"2024-04-25T10:06:24.571121+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:13.893207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579101+0000","last_change":"2024-04-25T10:06:23.599568+0000","last_active":"2024-04-25T10:06:24.579101+0000","last_peered":"2024-04-25T10:06:24.579101+0000","last_clean":"2024-04-25T10:06:24.579101+0000","last_became_active":"2024-04-25T10:06:23.599297+0000","last_became_peered":"2024-04-25T10:06:23.599297+0000","last_unstale":"2024-04-25T10:06:24.579101+0000","last_undegraded":"2024-04-25T10:06:24.579101+0000","last_fullsized":"2024-04-25T10:06:24.579101+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:55.955186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604979+0000","last_change":"2024-04-25T10:06:23.621138+0000","last_active":"2024-04-25T10:06:24.604979+0000","last_peered":"2024-04-25T10:06:24.604979+0000","last_clean":"2024-04-25T10:06:24.604979+0000","last_became_active":"2024-04-25T10:06:23.618352+0000","last_became_peered":"2024-04-25T10:06:23.618352+0000","last_unstale":"2024-04-25T10:06:24.604979+0000","last_undegraded":"2024-04-25T10:06:24.604979+0000","last_fullsized":"2024-04-25T10:06:24.604979+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:20:00.338484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.611845+0000","last_change":"2024-04-25T10:06:23.605127+0000","last_active":"2024-04-25T10:06:24.611845+0000","last_peered":"2024-04-25T10:06:24.611845+0000","last_clean":"2024-04-25T10:06:24.611845+0000","last_became_active":"2024-04-25T10:06:23.604870+0000","last_became_peered":"2024-04-25T10:06:23.604870+0000","last_unstale":"2024-04-25T10:06:24.611845+0000","last_undegraded":"2024-04-25T10:06:24.611845+0000","last_fullsized":"2024-04-25T10:06:24.611845+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.272912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604784+0000","last_change":"2024-04-25T10:06:23.612680+0000","last_active":"2024-04-25T10:06:24.604784+0000","last_peered":"2024-04-25T10:06:24.604784+0000","last_clean":"2024-04-25T10:06:24.604784+0000","last_became_active":"2024-04-25T10:06:23.612398+0000","last_became_peered":"2024-04-25T10:06:23.612398+0000","last_unstale":"2024-04-25T10:06:24.604784+0000","last_undegraded":"2024-04-25T10:06:24.604784+0000","last_fullsized":"2024-04-25T10:06:24.604784+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:39.700337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.575970+0000","last_change":"2024-04-25T10:06:23.603524+0000","last_active":"2024-04-25T10:06:24.575970+0000","last_peered":"2024-04-25T10:06:24.575970+0000","last_clean":"2024-04-25T10:06:24.575970+0000","last_became_active":"2024-04-25T10:06:23.601028+0000","last_became_peered":"2024-04-25T10:06:23.601028+0000","last_unstale":"2024-04-25T10:06:24.575970+0000","last_undegraded":"2024-04-25T10:06:24.575970+0000","last_fullsized":"2024-04-25T10:06:24.575970+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:32.281279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.611413+0000","last_change":"2024-04-25T10:06:23.606217+0000","last_active":"2024-04-25T10:06:24.611413+0000","last_peered":"2024-04-25T10:06:24.611413+0000","last_clean":"2024-04-25T10:06:24.611413+0000","last_became_active":"2024-04-25T10:06:23.605939+0000","last_became_peered":"2024-04-25T10:06:23.605939+0000","last_unstale":"2024-04-25T10:06:24.611413+0000","last_undegraded":"2024-04-25T10:06:24.611413+0000","last_fullsized":"2024-04-25T10:06:24.611413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.035028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"53'4","reported_seq":18,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579717+0000","last_change":"2024-04-25T10:06:22.567418+0000","last_active":"2024-04-25T10:06:24.579717+0000","last_peered":"2024-04-25T10:06:24.579717+0000","last_clean":"2024-04-25T10:06:24.579717+0000","last_became_active":"2024-04-25T10:06:22.567083+0000","last_became_peered":"2024-04-25T10:06:22.567083+0000","last_unstale":"2024-04-25T10:06:24.579717+0000","last_undegraded":"2024-04-25T10:06:24.579717+0000","last_fullsized":"2024-04-25T10:06:24.579717+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:13:48.186253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.577557+0000","last_change":"2024-04-25T10:06:23.605542+0000","last_active":"2024-04-25T10:06:24.577557+0000","last_peered":"2024-04-25T10:06:24.577557+0000","last_clean":"2024-04-25T10:06:24.577557+0000","last_became_active":"2024-04-25T10:06:23.605203+0000","last_became_peered":"2024-04-25T10:06:23.605203+0000","last_unstale":"2024-04-25T10:06:24.577557+0000","last_undegraded":"2024-04-25T10:06:24.577557+0000","last_fullsized":"2024-04-25T10:06:24.577557+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.524720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570974+0000","last_change":"2024-04-25T10:06:22.559140+0000","last_active":"2024-04-25T10:06:24.570974+0000","last_peered":"2024-04-25T10:06:24.570974+0000","last_clean":"2024-04-25T10:06:24.570974+0000","last_became_active":"2024-04-25T10:06:22.558843+0000","last_became_peered":"2024-04-25T10:06:22.558843+0000","last_unstale":"2024-04-25T10:06:24.570974+0000","last_undegraded":"2024-04-25T10:06:24.570974+0000","last_fullsized":"2024-04-25T10:06:24.570974+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:07.845383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.578832+0000","last_change":"2024-04-25T10:06:23.605617+0000","last_active":"2024-04-25T10:06:24.578832+0000","last_peered":"2024-04-25T10:06:24.578832+0000","last_clean":"2024-04-25T10:06:24.578832+0000","last_became_active":"2024-04-25T10:06:23.604783+0000","last_became_peered":"2024-04-25T10:06:23.604783+0000","last_unstale":"2024-04-25T10:06:24.578832+0000","last_undegraded":"2024-04-25T10:06:24.578832+0000","last_fullsized":"2024-04-25T10:06:24.578832+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:17.489183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.571256+0000","last_change":"2024-04-25T10:06:22.568850+0000","last_active":"2024-04-25T10:06:24.571256+0000","last_peered":"2024-04-25T10:06:24.571256+0000","last_clean":"2024-04-25T10:06:24.571256+0000","last_became_active":"2024-04-25T10:06:22.568402+0000","last_became_peered":"2024-04-25T10:06:22.568402+0000","last_unstale":"2024-04-25T10:06:24.571256+0000","last_undegraded":"2024-04-25T10:06:24.571256+0000","last_fullsized":"2024-04-25T10:06:24.571256+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:05.121186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185061+0000","last_change":"2024-04-25T10:06:23.605171+0000","last_active":"2024-04-25T10:06:26.185061+0000","last_peered":"2024-04-25T10:06:26.185061+0000","last_clean":"2024-04-25T10:06:26.185061+0000","last_became_active":"2024-04-25T10:06:23.604770+0000","last_became_peered":"2024-04-25T10:06:23.604770+0000","last_unstale":"2024-04-25T10:06:26.185061+0000","last_undegraded":"2024-04-25T10:06:26.185061+0000","last_fullsized":"2024-04-25T10:06:26.185061+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:17.345654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.576739+0000","last_change":"2024-04-25T10:06:22.572847+0000","last_active":"2024-04-25T10:06:24.576739+0000","last_peered":"2024-04-25T10:06:24.576739+0000","last_clean":"2024-04-25T10:06:24.576739+0000","last_became_active":"2024-04-25T10:06:22.572533+0000","last_became_peered":"2024-04-25T10:06:22.572533+0000","last_unstale":"2024-04-25T10:06:24.576739+0000","last_undegraded":"2024-04-25T10:06:24.576739+0000","last_fullsized":"2024-04-25T10:06:24.576739+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:23.708951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.653071+0000","last_change":"2024-04-25T10:06:23.616681+0000","last_active":"2024-04-25T10:06:24.653071+0000","last_peered":"2024-04-25T10:06:24.653071+0000","last_clean":"2024-04-25T10:06:24.653071+0000","last_became_active":"2024-04-25T10:06:23.616361+0000","last_became_peered":"2024-04-25T10:06:23.616361+0000","last_unstale":"2024-04-25T10:06:24.653071+0000","last_undegraded":"2024-04-25T10:06:24.653071+0000","last_fullsized":"2024-04-25T10:06:24.653071+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:51.020669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604981+0000","last_change":"2024-04-25T10:06:22.572591+0000","last_active":"2024-04-25T10:06:24.604981+0000","last_peered":"2024-04-25T10:06:24.604981+0000","last_clean":"2024-04-25T10:06:24.604981+0000","last_became_active":"2024-04-25T10:06:22.572249+0000","last_became_peered":"2024-04-25T10:06:22.572249+0000","last_unstale":"2024-04-25T10:06:24.604981+0000","last_undegraded":"2024-04-25T10:06:24.604981+0000","last_fullsized":"2024-04-25T10:06:24.604981+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:46.922451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612206+0000","last_change":"2024-04-25T10:06:23.603703+0000","last_active":"2024-04-25T10:06:24.612206+0000","last_peered":"2024-04-25T10:06:24.612206+0000","last_clean":"2024-04-25T10:06:24.612206+0000","last_became_active":"2024-04-25T10:06:23.603400+0000","last_became_peered":"2024-04-25T10:06:23.603400+0000","last_unstale":"2024-04-25T10:06:24.612206+0000","last_undegraded":"2024-04-25T10:06:24.612206+0000","last_fullsized":"2024-04-25T10:06:24.612206+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:21.524720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.652346+0000","last_change":"2024-04-25T10:06:22.569438+0000","last_active":"2024-04-25T10:06:24.652346+0000","last_peered":"2024-04-25T10:06:24.652346+0000","last_clean":"2024-04-25T10:06:24.652346+0000","last_became_active":"2024-04-25T10:06:22.569090+0000","last_became_peered":"2024-04-25T10:06:22.569090+0000","last_unstale":"2024-04-25T10:06:24.652346+0000","last_undegraded":"2024-04-25T10:06:24.652346+0000","last_fullsized":"2024-04-25T10:06:24.652346+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:26.190398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184995+0000","last_change":"2024-04-25T10:06:23.607427+0000","last_active":"2024-04-25T10:06:26.184995+0000","last_peered":"2024-04-25T10:06:26.184995+0000","last_clean":"2024-04-25T10:06:26.184995+0000","last_became_active":"2024-04-25T10:06:23.604117+0000","last_became_peered":"2024-04-25T10:06:23.604117+0000","last_unstale":"2024-04-25T10:06:26.184995+0000","last_undegraded":"2024-04-25T10:06:26.184995+0000","last_fullsized":"2024-04-25T10:06:26.184995+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:25.217631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.578800+0000","last_change":"2024-04-25T10:06:22.574559+0000","last_active":"2024-04-25T10:06:24.578800+0000","last_peered":"2024-04-25T10:06:24.578800+0000","last_clean":"2024-04-25T10:06:24.578800+0000","last_became_active":"2024-04-25T10:06:22.574157+0000","last_became_peered":"2024-04-25T10:06:22.574157+0000","last_unstale":"2024-04-25T10:06:24.578800+0000","last_undegraded":"2024-04-25T10:06:24.578800+0000","last_fullsized":"2024-04-25T10:06:24.578800+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:26.190398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612143+0000","last_change":"2024-04-25T10:06:23.618511+0000","last_active":"2024-04-25T10:06:24.612143+0000","last_peered":"2024-04-25T10:06:24.612143+0000","last_clean":"2024-04-25T10:06:24.612143+0000","last_became_active":"2024-04-25T10:06:23.618031+0000","last_became_peered":"2024-04-25T10:06:23.618031+0000","last_unstale":"2024-04-25T10:06:24.612143+0000","last_undegraded":"2024-04-25T10:06:24.612143+0000","last_fullsized":"2024-04-25T10:06:24.612143+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:26.699954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604645+0000","last_change":"2024-04-25T10:06:22.568292+0000","last_active":"2024-04-25T10:06:24.604645+0000","last_peered":"2024-04-25T10:06:24.604645+0000","last_clean":"2024-04-25T10:06:24.604645+0000","last_became_active":"2024-04-25T10:06:22.566910+0000","last_became_peered":"2024-04-25T10:06:22.566910+0000","last_unstale":"2024-04-25T10:06:24.604645+0000","last_undegraded":"2024-04-25T10:06:24.604645+0000","last_fullsized":"2024-04-25T10:06:24.604645+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:54.950155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.652462+0000","last_change":"2024-04-25T10:06:23.610190+0000","last_active":"2024-04-25T10:06:24.652462+0000","last_peered":"2024-04-25T10:06:24.652462+0000","last_clean":"2024-04-25T10:06:24.652462+0000","last_became_active":"2024-04-25T10:06:23.609903+0000","last_became_peered":"2024-04-25T10:06:23.609903+0000","last_unstale":"2024-04-25T10:06:24.652462+0000","last_undegraded":"2024-04-25T10:06:24.652462+0000","last_fullsized":"2024-04-25T10:06:24.652462+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:47.484455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.611902+0000","last_change":"2024-04-25T10:06:22.578005+0000","last_active":"2024-04-25T10:06:24.611902+0000","last_peered":"2024-04-25T10:06:24.611902+0000","last_clean":"2024-04-25T10:06:24.611902+0000","last_became_active":"2024-04-25T10:06:22.577713+0000","last_became_peered":"2024-04-25T10:06:22.577713+0000","last_unstale":"2024-04-25T10:06:24.611902+0000","last_undegraded":"2024-04-25T10:06:24.611902+0000","last_fullsized":"2024-04-25T10:06:24.611902+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:23.708951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.575994+0000","last_change":"2024-04-25T10:06:23.596167+0000","last_active":"2024-04-25T10:06:24.575994+0000","last_peered":"2024-04-25T10:06:24.575994+0000","last_clean":"2024-04-25T10:06:24.575994+0000","last_became_active":"2024-04-25T10:06:23.595841+0000","last_became_peered":"2024-04-25T10:06:23.595841+0000","last_unstale":"2024-04-25T10:06:24.575994+0000","last_undegraded":"2024-04-25T10:06:24.575994+0000","last_fullsized":"2024-04-25T10:06:24.575994+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:22.035028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.652963+0000","last_change":"2024-04-25T10:06:22.573664+0000","last_active":"2024-04-25T10:06:24.652963+0000","last_peered":"2024-04-25T10:06:24.652963+0000","last_clean":"2024-04-25T10:06:24.652963+0000","last_became_active":"2024-04-25T10:06:22.572943+0000","last_became_peered":"2024-04-25T10:06:22.572943+0000","last_unstale":"2024-04-25T10:06:24.652963+0000","last_undegraded":"2024-04-25T10:06:24.652963+0000","last_fullsized":"2024-04-25T10:06:24.652963+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:23.179795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605517+0000","last_change":"2024-04-25T10:06:23.617170+0000","last_active":"2024-04-25T10:06:24.605517+0000","last_peered":"2024-04-25T10:06:24.605517+0000","last_clean":"2024-04-25T10:06:24.605517+0000","last_became_active":"2024-04-25T10:06:23.616807+0000","last_became_peered":"2024-04-25T10:06:23.616807+0000","last_unstale":"2024-04-25T10:06:24.605517+0000","last_undegraded":"2024-04-25T10:06:24.605517+0000","last_fullsized":"2024-04-25T10:06:24.605517+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:31.330485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"53'3","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612386+0000","last_change":"2024-04-25T10:06:22.577343+0000","last_active":"2024-04-25T10:06:24.612386+0000","last_peered":"2024-04-25T10:06:24.612386+0000","last_clean":"2024-04-25T10:06:24.612386+0000","last_became_active":"2024-04-25T10:06:22.576799+0000","last_became_peered":"2024-04-25T10:06:22.576799+0000","last_unstale":"2024-04-25T10:06:24.612386+0000","last_undegraded":"2024-04-25T10:06:24.612386+0000","last_fullsized":"2024-04-25T10:06:24.612386+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:54.643890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.653067+0000","last_change":"2024-04-25T10:06:23.607727+0000","last_active":"2024-04-25T10:06:24.653067+0000","last_peered":"2024-04-25T10:06:24.653067+0000","last_clean":"2024-04-25T10:06:24.653067+0000","last_became_active":"2024-04-25T10:06:23.607413+0000","last_became_peered":"2024-04-25T10:06:23.607413+0000","last_unstale":"2024-04-25T10:06:24.653067+0000","last_undegraded":"2024-04-25T10:06:24.653067+0000","last_fullsized":"2024-04-25T10:06:24.653067+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:17.489183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"53'6","reported_seq":20,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184963+0000","last_change":"2024-04-25T10:06:22.559527+0000","last_active":"2024-04-25T10:06:26.184963+0000","last_peered":"2024-04-25T10:06:26.184963+0000","last_clean":"2024-04-25T10:06:26.184963+0000","last_became_active":"2024-04-25T10:06:22.557581+0000","last_became_peered":"2024-04-25T10:06:22.557581+0000","last_unstale":"2024-04-25T10:06:26.184963+0000","last_undegraded":"2024-04-25T10:06:26.184963+0000","last_fullsized":"2024-04-25T10:06:26.184963+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:21.029997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.576198+0000","last_change":"2024-04-25T10:06:23.610772+0000","last_active":"2024-04-25T10:06:24.576198+0000","last_peered":"2024-04-25T10:06:24.576198+0000","last_clean":"2024-04-25T10:06:24.576198+0000","last_became_active":"2024-04-25T10:06:23.610468+0000","last_became_peered":"2024-04-25T10:06:23.610468+0000","last_unstale":"2024-04-25T10:06:24.576198+0000","last_undegraded":"2024-04-25T10:06:24.576198+0000","last_fullsized":"2024-04-25T10:06:24.576198+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:50:39.280133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605037+0000","last_change":"2024-04-25T10:06:22.577864+0000","last_active":"2024-04-25T10:06:24.605037+0000","last_peered":"2024-04-25T10:06:24.605037+0000","last_clean":"2024-04-25T10:06:24.605037+0000","last_became_active":"2024-04-25T10:06:22.577604+0000","last_became_peered":"2024-04-25T10:06:22.577604+0000","last_unstale":"2024-04-25T10:06:24.605037+0000","last_undegraded":"2024-04-25T10:06:24.605037+0000","last_fullsized":"2024-04-25T10:06:24.605037+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:55.784890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606363+0000","last_change":"2024-04-25T10:06:23.607078+0000","last_active":"2024-04-25T10:06:24.606363+0000","last_peered":"2024-04-25T10:06:24.606363+0000","last_clean":"2024-04-25T10:06:24.606363+0000","last_became_active":"2024-04-25T10:06:23.606293+0000","last_became_peered":"2024-04-25T10:06:23.606293+0000","last_unstale":"2024-04-25T10:06:24.606363+0000","last_undegraded":"2024-04-25T10:06:24.606363+0000","last_fullsized":"2024-04-25T10:06:24.606363+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:25.217631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.578504+0000","last_change":"2024-04-25T10:06:22.570888+0000","last_active":"2024-04-25T10:06:24.578504+0000","last_peered":"2024-04-25T10:06:24.578504+0000","last_clean":"2024-04-25T10:06:24.578504+0000","last_became_active":"2024-04-25T10:06:22.570557+0000","last_became_peered":"2024-04-25T10:06:22.570557+0000","last_unstale":"2024-04-25T10:06:24.578504+0000","last_undegraded":"2024-04-25T10:06:24.578504+0000","last_fullsized":"2024-04-25T10:06:24.578504+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:21.691684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184651+0000","last_change":"2024-04-25T10:06:23.609474+0000","last_active":"2024-04-25T10:06:26.184651+0000","last_peered":"2024-04-25T10:06:26.184651+0000","last_clean":"2024-04-25T10:06:26.184651+0000","last_became_active":"2024-04-25T10:06:23.609218+0000","last_became_peered":"2024-04-25T10:06:23.609218+0000","last_unstale":"2024-04-25T10:06:26.184651+0000","last_undegraded":"2024-04-25T10:06:26.184651+0000","last_fullsized":"2024-04-25T10:06:26.184651+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.934380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.611833+0000","last_change":"2024-04-25T10:06:22.576900+0000","last_active":"2024-04-25T10:06:24.611833+0000","last_peered":"2024-04-25T10:06:24.611833+0000","last_clean":"2024-04-25T10:06:24.611833+0000","last_became_active":"2024-04-25T10:06:22.576595+0000","last_became_peered":"2024-04-25T10:06:22.576595+0000","last_unstale":"2024-04-25T10:06:24.611833+0000","last_undegraded":"2024-04-25T10:06:24.611833+0000","last_fullsized":"2024-04-25T10:06:24.611833+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:55.784890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606478+0000","last_change":"2024-04-25T10:06:23.613603+0000","last_active":"2024-04-25T10:06:24.606478+0000","last_peered":"2024-04-25T10:06:24.606478+0000","last_clean":"2024-04-25T10:06:24.606478+0000","last_became_active":"2024-04-25T10:06:23.613287+0000","last_became_peered":"2024-04-25T10:06:23.613287+0000","last_unstale":"2024-04-25T10:06:24.606478+0000","last_undegraded":"2024-04-25T10:06:24.606478+0000","last_fullsized":"2024-04-25T10:06:24.606478+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:17.345654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"53'8","reported_seq":22,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184337+0000","last_change":"2024-04-25T10:06:22.568736+0000","last_active":"2024-04-25T10:06:26.184337+0000","last_peered":"2024-04-25T10:06:26.184337+0000","last_clean":"2024-04-25T10:06:26.184337+0000","last_became_active":"2024-04-25T10:06:22.568418+0000","last_became_peered":"2024-04-25T10:06:22.568418+0000","last_unstale":"2024-04-25T10:06:26.184337+0000","last_undegraded":"2024-04-25T10:06:26.184337+0000","last_fullsized":"2024-04-25T10:06:26.184337+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:20:31.276248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184326+0000","last_change":"2024-04-25T10:06:23.607316+0000","last_active":"2024-04-25T10:06:26.184326+0000","last_peered":"2024-04-25T10:06:26.184326+0000","last_clean":"2024-04-25T10:06:26.184326+0000","last_became_active":"2024-04-25T10:06:23.606499+0000","last_became_peered":"2024-04-25T10:06:23.606499+0000","last_unstale":"2024-04-25T10:06:26.184326+0000","last_undegraded":"2024-04-25T10:06:26.184326+0000","last_fullsized":"2024-04-25T10:06:26.184326+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:59.820651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606756+0000","last_change":"2024-04-25T10:06:22.573736+0000","last_active":"2024-04-25T10:06:24.606756+0000","last_peered":"2024-04-25T10:06:24.606756+0000","last_clean":"2024-04-25T10:06:24.606756+0000","last_became_active":"2024-04-25T10:06:22.573471+0000","last_became_peered":"2024-04-25T10:06:22.573471+0000","last_unstale":"2024-04-25T10:06:24.606756+0000","last_undegraded":"2024-04-25T10:06:24.606756+0000","last_fullsized":"2024-04-25T10:06:24.606756+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:23.708951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570935+0000","last_change":"2024-04-25T10:06:23.609023+0000","last_active":"2024-04-25T10:06:24.570935+0000","last_peered":"2024-04-25T10:06:24.570935+0000","last_clean":"2024-04-25T10:06:24.570935+0000","last_became_active":"2024-04-25T10:06:23.608312+0000","last_became_peered":"2024-04-25T10:06:23.608312+0000","last_unstale":"2024-04-25T10:06:24.570935+0000","last_undegraded":"2024-04-25T10:06:24.570935+0000","last_fullsized":"2024-04-25T10:06:24.570935+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:21:15.555761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.186697+0000","last_change":"2024-04-25T10:06:22.561592+0000","last_active":"2024-04-25T10:06:26.186697+0000","last_peered":"2024-04-25T10:06:26.186697+0000","last_clean":"2024-04-25T10:06:26.186697+0000","last_became_active":"2024-04-25T10:06:22.561324+0000","last_became_peered":"2024-04-25T10:06:22.561324+0000","last_unstale":"2024-04-25T10:06:26.186697+0000","last_undegraded":"2024-04-25T10:06:26.186697+0000","last_fullsized":"2024-04-25T10:06:26.186697+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:51.530200+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.652513+0000","last_change":"2024-04-25T10:06:23.617850+0000","last_active":"2024-04-25T10:06:24.652513+0000","last_peered":"2024-04-25T10:06:24.652513+0000","last_clean":"2024-04-25T10:06:24.652513+0000","last_became_active":"2024-04-25T10:06:23.616352+0000","last_became_peered":"2024-04-25T10:06:23.616352+0000","last_unstale":"2024-04-25T10:06:24.652513+0000","last_undegraded":"2024-04-25T10:06:24.652513+0000","last_fullsized":"2024-04-25T10:06:24.652513+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:14.160671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"53'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579425+0000","last_change":"2024-04-25T10:06:22.575230+0000","last_active":"2024-04-25T10:06:24.579425+0000","last_peered":"2024-04-25T10:06:24.579425+0000","last_clean":"2024-04-25T10:06:24.579425+0000","last_became_active":"2024-04-25T10:06:22.574674+0000","last_became_peered":"2024-04-25T10:06:22.574674+0000","last_unstale":"2024-04-25T10:06:24.579425+0000","last_undegraded":"2024-04-25T10:06:24.579425+0000","last_fullsized":"2024-04-25T10:06:24.579425+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:00:03.777299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570744+0000","last_change":"2024-04-25T10:06:23.585846+0000","last_active":"2024-04-25T10:06:24.570744+0000","last_peered":"2024-04-25T10:06:24.570744+0000","last_clean":"2024-04-25T10:06:24.570744+0000","last_became_active":"2024-04-25T10:06:23.585045+0000","last_became_peered":"2024-04-25T10:06:23.585045+0000","last_unstale":"2024-04-25T10:06:24.570744+0000","last_undegraded":"2024-04-25T10:06:24.570744+0000","last_fullsized":"2024-04-25T10:06:24.570744+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:47:24.713982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579172+0000","last_change":"2024-04-25T10:06:22.574589+0000","last_active":"2024-04-25T10:06:24.579172+0000","last_peered":"2024-04-25T10:06:24.579172+0000","last_clean":"2024-04-25T10:06:24.579172+0000","last_became_active":"2024-04-25T10:06:22.574215+0000","last_became_peered":"2024-04-25T10:06:22.574215+0000","last_unstale":"2024-04-25T10:06:24.579172+0000","last_undegraded":"2024-04-25T10:06:24.579172+0000","last_fullsized":"2024-04-25T10:06:24.579172+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.576615+0000","last_change":"2024-04-25T10:06:23.600106+0000","last_active":"2024-04-25T10:06:24.576615+0000","last_peered":"2024-04-25T10:06:24.576615+0000","last_clean":"2024-04-25T10:06:24.576615+0000","last_became_active":"2024-04-25T10:06:23.599821+0000","last_became_peered":"2024-04-25T10:06:23.599821+0000","last_unstale":"2024-04-25T10:06:24.576615+0000","last_undegraded":"2024-04-25T10:06:24.576615+0000","last_fullsized":"2024-04-25T10:06:24.576615+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:21:26.699954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.571063+0000","last_change":"2024-04-25T10:06:22.565634+0000","last_active":"2024-04-25T10:06:24.571063+0000","last_peered":"2024-04-25T10:06:24.571063+0000","last_clean":"2024-04-25T10:06:24.571063+0000","last_became_active":"2024-04-25T10:06:22.565200+0000","last_became_peered":"2024-04-25T10:06:22.565200+0000","last_unstale":"2024-04-25T10:06:24.571063+0000","last_undegraded":"2024-04-25T10:06:24.571063+0000","last_fullsized":"2024-04-25T10:06:24.571063+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:49.071194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579106+0000","last_change":"2024-04-25T10:06:23.603422+0000","last_active":"2024-04-25T10:06:24.579106+0000","last_peered":"2024-04-25T10:06:24.579106+0000","last_clean":"2024-04-25T10:06:24.579106+0000","last_became_active":"2024-04-25T10:06:23.603126+0000","last_became_peered":"2024-04-25T10:06:23.603126+0000","last_unstale":"2024-04-25T10:06:24.579106+0000","last_undegraded":"2024-04-25T10:06:24.579106+0000","last_fullsized":"2024-04-25T10:06:24.579106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:23.481849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184046+0000","last_change":"2024-04-25T10:06:22.562297+0000","last_active":"2024-04-25T10:06:26.184046+0000","last_peered":"2024-04-25T10:06:26.184046+0000","last_clean":"2024-04-25T10:06:26.184046+0000","last_became_active":"2024-04-25T10:06:22.561983+0000","last_became_peered":"2024-04-25T10:06:22.561983+0000","last_unstale":"2024-04-25T10:06:26.184046+0000","last_undegraded":"2024-04-25T10:06:26.184046+0000","last_fullsized":"2024-04-25T10:06:26.184046+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:12.888176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606073+0000","last_change":"2024-04-25T10:06:23.601437+0000","last_active":"2024-04-25T10:06:24.606073+0000","last_peered":"2024-04-25T10:06:24.606073+0000","last_clean":"2024-04-25T10:06:24.606073+0000","last_became_active":"2024-04-25T10:06:23.601101+0000","last_became_peered":"2024-04-25T10:06:23.601101+0000","last_unstale":"2024-04-25T10:06:24.606073+0000","last_undegraded":"2024-04-25T10:06:24.606073+0000","last_fullsized":"2024-04-25T10:06:24.606073+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:43.086235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.604779+0000","last_change":"2024-04-25T10:06:22.559269+0000","last_active":"2024-04-25T10:06:24.604779+0000","last_peered":"2024-04-25T10:06:24.604779+0000","last_clean":"2024-04-25T10:06:24.604779+0000","last_became_active":"2024-04-25T10:06:22.558985+0000","last_became_peered":"2024-04-25T10:06:22.558985+0000","last_unstale":"2024-04-25T10:06:24.604779+0000","last_undegraded":"2024-04-25T10:06:24.604779+0000","last_fullsized":"2024-04-25T10:06:24.604779+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:22.476818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.571323+0000","last_change":"2024-04-25T10:06:23.604211+0000","last_active":"2024-04-25T10:06:24.571323+0000","last_peered":"2024-04-25T10:06:24.571323+0000","last_clean":"2024-04-25T10:06:24.571323+0000","last_became_active":"2024-04-25T10:06:23.603947+0000","last_became_peered":"2024-04-25T10:06:23.603947+0000","last_unstale":"2024-04-25T10:06:24.571323+0000","last_undegraded":"2024-04-25T10:06:24.571323+0000","last_fullsized":"2024-04-25T10:06:24.571323+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:39.113942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.183778+0000","last_change":"2024-04-25T10:06:22.572755+0000","last_active":"2024-04-25T10:06:26.183778+0000","last_peered":"2024-04-25T10:06:26.183778+0000","last_clean":"2024-04-25T10:06:26.183778+0000","last_became_active":"2024-04-25T10:06:22.572477+0000","last_became_peered":"2024-04-25T10:06:22.572477+0000","last_unstale":"2024-04-25T10:06:26.183778+0000","last_undegraded":"2024-04-25T10:06:26.183778+0000","last_fullsized":"2024-04-25T10:06:26.183778+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:09:55.040206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.576421+0000","last_change":"2024-04-25T10:06:23.602392+0000","last_active":"2024-04-25T10:06:24.576421+0000","last_peered":"2024-04-25T10:06:24.576421+0000","last_clean":"2024-04-25T10:06:24.576421+0000","last_became_active":"2024-04-25T10:06:23.600661+0000","last_became_peered":"2024-04-25T10:06:23.600661+0000","last_unstale":"2024-04-25T10:06:24.576421+0000","last_undegraded":"2024-04-25T10:06:24.576421+0000","last_fullsized":"2024-04-25T10:06:24.576421+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:12:39.272912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"53'2","reported_seq":16,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.579726+0000","last_change":"2024-04-25T10:06:22.572777+0000","last_active":"2024-04-25T10:06:24.579726+0000","last_peered":"2024-04-25T10:06:24.579726+0000","last_clean":"2024-04-25T10:06:24.579726+0000","last_became_active":"2024-04-25T10:06:22.572455+0000","last_became_peered":"2024-04-25T10:06:22.572455+0000","last_unstale":"2024-04-25T10:06:24.579726+0000","last_undegraded":"2024-04-25T10:06:24.579726+0000","last_fullsized":"2024-04-25T10:06:24.579726+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:46:23.179795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.570829+0000","last_change":"2024-04-25T10:06:23.601267+0000","last_active":"2024-04-25T10:06:24.570829+0000","last_peered":"2024-04-25T10:06:24.570829+0000","last_clean":"2024-04-25T10:06:24.570829+0000","last_became_active":"2024-04-25T10:06:23.598492+0000","last_became_peered":"2024-04-25T10:06:23.598492+0000","last_unstale":"2024-04-25T10:06:24.570829+0000","last_undegraded":"2024-04-25T10:06:24.570829+0000","last_fullsized":"2024-04-25T10:06:24.570829+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:35.735682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.612407+0000","last_change":"2024-04-25T10:06:22.577121+0000","last_active":"2024-04-25T10:06:24.612407+0000","last_peered":"2024-04-25T10:06:24.612407+0000","last_clean":"2024-04-25T10:06:24.612407+0000","last_became_active":"2024-04-25T10:06:22.576513+0000","last_became_peered":"2024-04-25T10:06:22.576513+0000","last_unstale":"2024-04-25T10:06:24.612407+0000","last_undegraded":"2024-04-25T10:06:24.612407+0000","last_fullsized":"2024-04-25T10:06:24.612407+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:12.182429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185145+0000","last_change":"2024-04-25T10:06:23.599784+0000","last_active":"2024-04-25T10:06:26.185145+0000","last_peered":"2024-04-25T10:06:26.185145+0000","last_clean":"2024-04-25T10:06:26.185145+0000","last_became_active":"2024-04-25T10:06:23.598587+0000","last_became_peered":"2024-04-25T10:06:23.598587+0000","last_unstale":"2024-04-25T10:06:26.185145+0000","last_undegraded":"2024-04-25T10:06:26.185145+0000","last_fullsized":"2024-04-25T10:06:26.185145+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:43.052798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605346+0000","last_change":"2024-04-25T10:06:22.576013+0000","last_active":"2024-04-25T10:06:24.605346+0000","last_peered":"2024-04-25T10:06:24.605346+0000","last_clean":"2024-04-25T10:06:24.605346+0000","last_became_active":"2024-04-25T10:06:22.575256+0000","last_became_peered":"2024-04-25T10:06:22.575256+0000","last_unstale":"2024-04-25T10:06:24.605346+0000","last_undegraded":"2024-04-25T10:06:24.605346+0000","last_fullsized":"2024-04-25T10:06:24.605346+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:54.643890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.605573+0000","last_change":"2024-04-25T10:06:23.608455+0000","last_active":"2024-04-25T10:06:24.605573+0000","last_peered":"2024-04-25T10:06:24.605573+0000","last_clean":"2024-04-25T10:06:24.605573+0000","last_became_active":"2024-04-25T10:06:23.607460+0000","last_became_peered":"2024-04-25T10:06:23.607460+0000","last_unstale":"2024-04-25T10:06:24.605573+0000","last_undegraded":"2024-04-25T10:06:24.605573+0000","last_fullsized":"2024-04-25T10:06:24.605573+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:40:09.661552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185195+0000","last_change":"2024-04-25T10:06:22.554815+0000","last_active":"2024-04-25T10:06:26.185195+0000","last_peered":"2024-04-25T10:06:26.185195+0000","last_clean":"2024-04-25T10:06:26.185195+0000","last_became_active":"2024-04-25T10:06:22.554515+0000","last_became_peered":"2024-04-25T10:06:22.554515+0000","last_unstale":"2024-04-25T10:06:26.185195+0000","last_undegraded":"2024-04-25T10:06:26.185195+0000","last_fullsized":"2024-04-25T10:06:26.185195+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:10:38.108911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.577321+0000","last_change":"2024-04-25T10:06:23.604677+0000","last_active":"2024-04-25T10:06:24.577321+0000","last_peered":"2024-04-25T10:06:24.577321+0000","last_clean":"2024-04-25T10:06:24.577321+0000","last_became_active":"2024-04-25T10:06:23.604406+0000","last_became_peered":"2024-04-25T10:06:23.604406+0000","last_unstale":"2024-04-25T10:06:24.577321+0000","last_undegraded":"2024-04-25T10:06:24.577321+0000","last_fullsized":"2024-04-25T10:06:24.577321+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:40.934457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.184156+0000","last_change":"2024-04-25T10:06:22.570695+0000","last_active":"2024-04-25T10:06:26.184156+0000","last_peered":"2024-04-25T10:06:26.184156+0000","last_clean":"2024-04-25T10:06:26.184156+0000","last_became_active":"2024-04-25T10:06:22.570401+0000","last_became_peered":"2024-04-25T10:06:22.570401+0000","last_unstale":"2024-04-25T10:06:26.184156+0000","last_undegraded":"2024-04-25T10:06:26.184156+0000","last_fullsized":"2024-04-25T10:06:26.184156+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:54.369216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606165+0000","last_change":"2024-04-25T10:06:23.612090+0000","last_active":"2024-04-25T10:06:24.606165+0000","last_peered":"2024-04-25T10:06:24.606165+0000","last_clean":"2024-04-25T10:06:24.606165+0000","last_became_active":"2024-04-25T10:06:23.606673+0000","last_became_peered":"2024-04-25T10:06:23.606673+0000","last_unstale":"2024-04-25T10:06:24.606165+0000","last_undegraded":"2024-04-25T10:06:24.606165+0000","last_fullsized":"2024-04-25T10:06:24.606165+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:09.934380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.652183+0000","last_change":"2024-04-25T10:06:22.572073+0000","last_active":"2024-04-25T10:06:24.652183+0000","last_peered":"2024-04-25T10:06:24.652183+0000","last_clean":"2024-04-25T10:06:24.652183+0000","last_became_active":"2024-04-25T10:06:22.571552+0000","last_became_peered":"2024-04-25T10:06:22.571552+0000","last_unstale":"2024-04-25T10:06:24.652183+0000","last_undegraded":"2024-04-25T10:06:24.652183+0000","last_fullsized":"2024-04-25T10:06:24.652183+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:58:28.517106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:26.185877+0000","last_change":"2024-04-25T10:06:23.605991+0000","last_active":"2024-04-25T10:06:26.185877+0000","last_peered":"2024-04-25T10:06:26.185877+0000","last_clean":"2024-04-25T10:06:26.185877+0000","last_became_active":"2024-04-25T10:06:23.605630+0000","last_became_peered":"2024-04-25T10:06:23.605630+0000","last_unstale":"2024-04-25T10:06:26.185877+0000","last_undegraded":"2024-04-25T10:06:26.185877+0000","last_fullsized":"2024-04-25T10:06:26.185877+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:56.257977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.576172+0000","last_change":"2024-04-25T10:06:22.572140+0000","last_active":"2024-04-25T10:06:24.576172+0000","last_peered":"2024-04-25T10:06:24.576172+0000","last_clean":"2024-04-25T10:06:24.576172+0000","last_became_active":"2024-04-25T10:06:22.571848+0000","last_became_peered":"2024-04-25T10:06:22.571848+0000","last_unstale":"2024-04-25T10:06:24.576172+0000","last_undegraded":"2024-04-25T10:06:24.576172+0000","last_fullsized":"2024-04-25T10:06:24.576172+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:12.182429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.578666+0000","last_change":"2024-04-25T10:06:23.584713+0000","last_active":"2024-04-25T10:06:24.578666+0000","last_peered":"2024-04-25T10:06:24.578666+0000","last_clean":"2024-04-25T10:06:24.578666+0000","last_became_active":"2024-04-25T10:06:23.582946+0000","last_became_peered":"2024-04-25T10:06:23.582946+0000","last_unstale":"2024-04-25T10:06:24.578666+0000","last_undegraded":"2024-04-25T10:06:24.578666+0000","last_fullsized":"2024-04-25T10:06:24.578666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:22.530503+0000","last_clean_scrub_stamp":"2024-04-25T10:06:22.530503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:14.160671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"54'1","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.606173+0000","last_change":"2024-04-25T10:06:22.578153+0000","last_active":"2024-04-25T10:06:24.606173+0000","last_peered":"2024-04-25T10:06:24.606173+0000","last_clean":"2024-04-25T10:06:24.606173+0000","last_became_active":"2024-04-25T10:06:22.577578+0000","last_became_peered":"2024-04-25T10:06:22.577578+0000","last_unstale":"2024-04-25T10:06:24.606173+0000","last_undegraded":"2024-04-25T10:06:24.606173+0000","last_fullsized":"2024-04-25T10:06:24.606173+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:06:21.525472+0000","last_clean_scrub_stamp":"2024-04-25T10:06:21.525472+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:17:12.182429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":237,"reported_epoch":54,"state":"active+clean","last_fresh":"2024-04-25T10:06:24.611235+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:06:24.611235+0000","last_peered":"2024-04-25T10:06:24.611235+0000","last_clean":"2024-04-25T10:06:24.611235+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:06:24.611235+0000","last_undegraded":"2024-04-25T10:06:24.611235+0000","last_fullsized":"2024-04-25T10:06:24.611235+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574907,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36564,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335276,"statfs":{"total":96636764160,"available":96599322624,"internally_reserved":0,"allocated":1245184,"data_stored":879710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":36174283},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":651264,"data_stored":289870,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574907,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35976,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335864,"statfs":{"total":96636764160,"available":96599924736,"internally_reserved":0,"allocated":643072,"data_stored":289218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40132,"kb_used_data":632,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331708,"statfs":{"total":96636764160,"available":96595668992,"internally_reserved":0,"allocated":647168,"data_stored":289800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35988,"kb_used_data":644,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335852,"statfs":{"total":96636764160,"available":96599912448,"internally_reserved":0,"allocated":659456,"data_stored":290852,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574906,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":655360,"data_stored":290830,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35980,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335860,"statfs":{"total":96636764160,"available":96599920640,"internally_reserved":0,"allocated":651264,"data_stored":289378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574907,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58768,"kb_used_data":1216,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313072,"statfs":{"total":96636764160,"available":96576585728,"internally_reserved":0,"allocated":1245184,"data_stored":879642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:06:28.576 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:06:28.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:06:28.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.817+0000 7f854da00640 1 Processor -- start 2024-04-25T10:06:28.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.818+0000 7f854da00640 1 -- start start 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.819+0000 7f854da00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85481446d0 0x7f85480b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.819+0000 7f854da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8548144d50 0x7f85480b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.819+0000 7f854da00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85480cccc0 con 0x7f85481452a0 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.819+0000 7f854da00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85480b2b20 con 0x7f85481446d0 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.819+0000 7f854da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f854806d400 con 0x7f8548144d50 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85481446d0 0x7f85480b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85481446d0 0x7f85480b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33298/0 (socket says 172.21.15.165:33298) 2024-04-25T10:06:28.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 -- 172.21.15.165:0/377923703 learned_addr learned my addr 172.21.15.165:0/377923703 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 -- 172.21.15.165:0/377923703 >> v2:172.21.15.172:3300/0 conn(0x7f8548144d50 msgr2=0x7f85480b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f853e000640 1 --2- 172.21.15.165:0/377923703 >> v2:172.21.15.172:3300/0 conn(0x7f8548144d50 0x7f85480b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 --2- 172.21.15.165:0/377923703 >> v2:172.21.15.172:3300/0 conn(0x7f8548144d50 0x7f85480b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 -- 172.21.15.165:0/377923703 >> v1:172.21.15.165:6789/0 conn(0x7f85481452a0 legacy=0x7f85480cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.820+0000 7f8546a00640 1 -- 172.21.15.165:0/377923703 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480cc140 con 0x7f85481446d0 2024-04-25T10:06:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.822+0000 7f8546a00640 1 --2- 172.21.15.165:0/377923703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85481446d0 0x7f85480b3f20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8548071c70 tx=0x7f853802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e750579cfa1269ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.822+0000 7f8546000640 1 -- 172.21.15.165:0/377923703 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538037070 con 0x7f85481446d0 2024-04-25T10:06:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.822+0000 7f8546000640 1 -- 172.21.15.165:0/377923703 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8538032440 con 0x7f85481446d0 2024-04-25T10:06:28.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.822+0000 7f8546000640 1 -- 172.21.15.165:0/377923703 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8538032810 con 0x7f85481446d0 2024-04-25T10:06:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 -- 172.21.15.165:0/377923703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85481446d0 msgr2=0x7f85480b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 --2- 172.21.15.165:0/377923703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85481446d0 0x7f85480b3f20 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8548071c70 tx=0x7f853802a270 comp rx=0 tx=0).stop 2024-04-25T10:06:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 -- 172.21.15.165:0/377923703 shutdown_connections 2024-04-25T10:06:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 --2- 172.21.15.165:0/377923703 >> v2:172.21.15.172:3300/0 conn(0x7f8548144d50 0x7f85480b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 --2- 172.21.15.165:0/377923703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85481446d0 0x7f85480b3f20 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 -- 172.21.15.165:0/377923703 >> 172.21.15.165:0/377923703 conn(0x7f854806abe0 msgr2=0x7f854806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 -- 172.21.15.165:0/377923703 shutdown_connections 2024-04-25T10:06:28.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.823+0000 7f854da00640 1 -- 172.21.15.165:0/377923703 wait complete. 2024-04-25T10:06:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.825+0000 7f854da00640 1 Processor -- start 2024-04-25T10:06:28.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.825+0000 7f854da00640 1 -- start start 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f854da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85481446d0 0x7f85480b17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f854da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 0x7f854813a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f854da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85481452a0 0x7f854813a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f854da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85480cc700 con 0x7f8548144d50 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f854da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85480b49a0 con 0x7f85481446d0 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f854da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85480cc9e0 con 0x7f85481452a0 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f853e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 0x7f854813a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f853e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 0x7f854813a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33306/0 (socket says 172.21.15.165:33306) 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f853e000640 1 -- 172.21.15.165:0/2370046877 learned_addr learned my addr 172.21.15.165:0/2370046877 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.826+0000 7f8546a00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85481446d0 0x7f85480b17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853e000640 1 -- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85481452a0 msgr2=0x7f854813a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853e000640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85481452a0 0x7f854813a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853e000640 1 -- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85481446d0 msgr2=0x7f85480b17a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853e000640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85481446d0 0x7f85480b17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853e000640 1 -- 172.21.15.165:0/2370046877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480b0000 con 0x7f8548144d50 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853e000640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 0x7f854813a070 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f853000c7f0 tx=0x7f853000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530015070 con 0x7f8548144d50 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530004220 con 0x7f8548144d50 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85480b02e0 con 0x7f8548144d50 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530007620 con 0x7f8548144d50 2024-04-25T10:06:28.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.827+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85480b08a0 con 0x7f8548144d50 2024-04-25T10:06:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.829+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8548073a00 con 0x7f8548144d50 2024-04-25T10:06:28.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.829+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85300079d0 con 0x7f8548144d50 2024-04-25T10:06:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.830+0000 7f853f400640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f852406b500 0x7f852406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.831+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f853008a390 con 0x7f8548144d50 2024-04-25T10:06:28.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.831+0000 7f8546a00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f852406b500 0x7f852406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:28.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.831+0000 7f8546a00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f852406b500 0x7f852406d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8538034000 tx=0x7f853802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:28.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:28.835+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8530059070 con 0x7f8548144d50 2024-04-25T10:06:29.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.105+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8548071600 con 0x7f8548144d50 2024-04-25T10:06:29.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.108+0000 7f853f400640 1 -- 172.21.15.165:0/2370046877 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+147 (secure 0 0 0) 0x7f8530058970 con 0x7f8548144d50 2024-04-25T10:06:29.114 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:06:29.114 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:06:29.114 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:06:29.114 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:06:29.114 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:06:29.114 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:06:29.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f852406b500 msgr2=0x7f852406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f852406b500 0x7f852406d9c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f8538034000 tx=0x7f853802e000 comp rx=0 tx=0).stop 2024-04-25T10:06:29.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 msgr2=0x7f854813a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 0x7f854813a070 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f853000c7f0 tx=0x7f853000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 shutdown_connections 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f852406b500 0x7f852406d9c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85481452a0 0x7f854813a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8548144d50 0x7f854813a070 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 --2- 172.21.15.165:0/2370046877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85481446d0 0x7f85480b17a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 >> 172.21.15.165:0/2370046877 conn(0x7f854806abe0 msgr2=0x7f85480aca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 shutdown_connections 2024-04-25T10:06:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.111+0000 7f854da00640 1 -- 172.21.15.165:0/2370046877 wait complete. 2024-04-25T10:06:29.134 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:29.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.322+0000 7f2806c00640 1 Processor -- start 2024-04-25T10:06:29.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.323+0000 7f2806c00640 1 -- start start 2024-04-25T10:06:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.324+0000 7f2806c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2800144440 0x7f2800070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.324+0000 7f2806c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f28000cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.324+0000 7f2806c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f280015bf30 con 0x7f28000cd860 2024-04-25T10:06:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.324+0000 7f2806c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f280015c860 con 0x7f2800144440 2024-04-25T10:06:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.324+0000 7f2806c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2800071970 con 0x7f2800071530 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.324+0000 7f2805800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2800144440 0x7f2800070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2805800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2800144440 0x7f2800070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33318/0 (socket says 172.21.15.165:33318) 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2805800640 1 -- 172.21.15.165:0/1167471733 learned_addr learned my addr 172.21.15.165:0/1167471733 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2804e00640 1 --2- 172.21.15.165:0/1167471733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f28000cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2805800640 1 -- 172.21.15.165:0/1167471733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 msgr2=0x7f28000cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2805800640 1 --2- 172.21.15.165:0/1167471733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f28000cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2805800640 1 -- 172.21.15.165:0/1167471733 >> v1:172.21.15.172:6789/0 conn(0x7f28000cd860 legacy=0x7f280006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.325+0000 7f2805800640 1 -- 172.21.15.165:0/1167471733 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f280015c210 con 0x7f2800144440 2024-04-25T10:06:29.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.326+0000 7f2805800640 1 --2- 172.21.15.165:0/1167471733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f2800070ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009e50 tx=0x7f27f402a750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d315753e7f364893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.326+0000 7f27ffe00640 1 -- 172.21.15.165:0/1167471733 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4036070 con 0x7f2800144440 2024-04-25T10:06:29.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.326+0000 7f27ffe00640 1 -- 172.21.15.165:0/1167471733 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4031600 con 0x7f2800144440 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.326+0000 7f27ffe00640 1 -- 172.21.15.165:0/1167471733 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f40319d0 con 0x7f2800144440 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 -- 172.21.15.165:0/1167471733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 msgr2=0x7f2800070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 --2- 172.21.15.165:0/1167471733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f2800070ff0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f27f4009e50 tx=0x7f27f402a750 comp rx=0 tx=0).stop 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 -- 172.21.15.165:0/1167471733 shutdown_connections 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 --2- 172.21.15.165:0/1167471733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f28000cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 --2- 172.21.15.165:0/1167471733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f2800070ff0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 -- 172.21.15.165:0/1167471733 >> 172.21.15.165:0/1167471733 conn(0x7f280006abe0 msgr2=0x7f280006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 -- 172.21.15.165:0/1167471733 shutdown_connections 2024-04-25T10:06:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.327+0000 7f2806c00640 1 -- 172.21.15.165:0/1167471733 wait complete. 2024-04-25T10:06:29.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.329+0000 7f2806c00640 1 Processor -- start 2024-04-25T10:06:29.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.329+0000 7f2806c00640 1 -- start start 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f2800142640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28000cd860 0x7f28001e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f280015afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28000cdcd0 con 0x7f2800144440 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280015bc30 con 0x7f28000cd860 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28000cde50 con 0x7f2800071530 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f280015afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f280015afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33320/0 (socket says 172.21.15.165:33320) 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.330+0000 7f2806200640 1 -- 172.21.15.165:0/930777124 learned_addr learned my addr 172.21.15.165:0/930777124 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2805800640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f2800142640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2804e00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28000cd860 0x7f28001e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806200640 1 -- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 msgr2=0x7f2800142640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806200640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f2800142640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806200640 1 -- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28000cd860 msgr2=0x7f28001e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806200640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28000cd860 0x7f28001e5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806200640 1 -- 172.21.15.165:0/930777124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28001e6100 con 0x7f2800144440 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2804e00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28000cd860 0x7f28001e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806200640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f280015afa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f27f800c410 tx=0x7f27f800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8011790 con 0x7f2800144440 2024-04-25T10:06:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f8017070 con 0x7f2800144440 2024-04-25T10:06:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8011bb0 con 0x7f2800144440 2024-04-25T10:06:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.331+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28001e63c0 con 0x7f2800144440 2024-04-25T10:06:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.332+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28001e6980 con 0x7f2800144440 2024-04-25T10:06:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.333+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2800073990 con 0x7f2800144440 2024-04-25T10:06:29.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.334+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27f801c460 con 0x7f2800144440 2024-04-25T10:06:29.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.334+0000 7f27fd600640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e006b4d0 0x7f27e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:29.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.335+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27f808a060 con 0x7f2800144440 2024-04-25T10:06:29.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.335+0000 7f2805800640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e006b4d0 0x7f27e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:29.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.336+0000 7f2805800640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e006b4d0 0x7f27e006d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f27f4033000 tx=0x7f27f4009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:29.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.339+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27f8021020 con 0x7f2800144440 2024-04-25T10:06:29.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.568+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2800143c40 con 0x7f2800144440 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.569+0000 7f27fd600640 1 -- 172.21.15.165:0/930777124 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f27f8058cb0 con 0x7f2800144440 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:29.575 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e006b4d0 msgr2=0x7f27e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e006b4d0 0x7f27e006d990 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f27f4033000 tx=0x7f27f4009000 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 msgr2=0x7f280015afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f280015afa0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f27f800c410 tx=0x7f27f800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 shutdown_connections 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e006b4d0 0x7f27e006d990 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2800144440 0x7f280015afa0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f28000cd860 0x7f28001e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 --2- 172.21.15.165:0/930777124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2800071530 0x7f2800142640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:29.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.572+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 >> 172.21.15.165:0/930777124 conn(0x7f280006abe0 msgr2=0x7f28000b2480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:29.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.573+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 shutdown_connections 2024-04-25T10:06:29.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:29.573+0000 7f2806c00640 1 -- 172.21.15.165:0/930777124 wait complete. 2024-04-25T10:06:30.000 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:06:22.994073+0000', 'modified': '2024-04-25T10:06:26.075717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5811}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5811': {'gid': 5811, 'name': 'd', 'rank': 0, 'incarnation': 25, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6834/3236509347', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 3236509347}, {'type': 'v1', 'addr': '172.21.15.172:6834', 'nonce': 3236509347}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:06:30.001 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:06:30.001 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:06:30.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.199+0000 7fe763e00640 1 Processor -- start 2024-04-25T10:06:30.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.199+0000 7fe763e00640 1 -- start start 2024-04-25T10:06:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.201+0000 7fe763e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 0x7fe764072ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.201+0000 7fe763e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7640b43d0 con 0x7fe7640af720 2024-04-25T10:06:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.201+0000 7fe763e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7640b46b0 con 0x7fe76406ae50 2024-04-25T10:06:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.201+0000 7fe763e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640b4990 con 0x7fe76406b230 2024-04-25T10:06:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.201+0000 7fe763400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe7640af720 0x7fe7640afb50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56714/0 (socket says 172.21.15.165:56714) 2024-04-25T10:06:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.201+0000 7fe763400640 1 -- 172.21.15.165:0/818215505 learned_addr learned my addr 172.21.15.165:0/818215505 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe75a000640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 0x7fe764072ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2324574322 0 0) 0x7fe7640b43d0 con 0x7fe7640af720 2024-04-25T10:06:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe748003520 con 0x7fe7640af720 2024-04-25T10:06:30.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1568421240 0 0) 0x7fe748003520 con 0x7fe7640af720 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 msgr2=0x7fe764072ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 0x7fe764072ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 >> v1:172.21.15.172:6789/0 conn(0x7fe76406ae50 legacy=0x7fe764161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe76406fc20 con 0x7fe7640af720 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe75c0019f0 con 0x7fe7640af720 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 >> v1:172.21.15.165:6789/0 conn(0x7fe7640af720 legacy=0x7fe7640afb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.202+0000 7fe762000640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7480039e0 0x7fe748003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762000640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 0x7fe748004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762000640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe748004d40 0x7fe7480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7640b4990 con 0x7fe7480039e0 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe74800bbd0 con 0x7fe748004d40 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe74800bdd0 con 0x7fe76406b230 2024-04-25T10:06:30.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762a00640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7480039e0 0x7fe748003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe75a000640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 0x7fe748004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762a00640 1 -- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 msgr2=0x7fe748004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762a00640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe76406b230 0x7fe748004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762a00640 1 -- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe748004d40 msgr2=0x7fe7480051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762a00640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe748004d40 0x7fe7480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.203+0000 7fe762a00640 1 -- 172.21.15.165:0/818215505 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe748003520 con 0x7fe7480039e0 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.204+0000 7fe762a00640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7480039e0 0x7fe748003e20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe754001750 tx=0x7fe748005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5741fb8bfe7ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.204+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe754001370 con 0x7fe7480039e0 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7540052d0 con 0x7fe7480039e0 2024-04-25T10:06:30.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe762000640 1 -- 172.21.15.165:0/818215505 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75402abe0 con 0x7fe7480039e0 2024-04-25T10:06:30.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763e00640 1 -- 172.21.15.165:0/818215505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7480039e0 msgr2=0x7fe748003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763e00640 1 --2- 172.21.15.165:0/818215505 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7480039e0 0x7fe748003e20 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fe754001750 tx=0x7fe748005830 comp rx=0 tx=0).stop 2024-04-25T10:06:30.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763400640 1 -- 172.21.15.165:0/818215505 reap_dead start 2024-04-25T10:06:30.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763e00640 1 -- 172.21.15.165:0/818215505 shutdown_connections 2024-04-25T10:06:30.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763e00640 1 -- 172.21.15.165:0/818215505 >> 172.21.15.165:0/818215505 conn(0x7fe76406e3b0 msgr2=0x7fe76406e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763e00640 1 -- 172.21.15.165:0/818215505 shutdown_connections 2024-04-25T10:06:30.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.205+0000 7fe763e00640 1 -- 172.21.15.165:0/818215505 wait complete. 2024-04-25T10:06:30.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.207+0000 7fe763e00640 1 Processor -- start 2024-04-25T10:06:30.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.207+0000 7fe763e00640 1 -- start start 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7480039e0 0x7fe7641533c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe764158d90 0x7fe76415b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 0x7fe764147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764072c30 con 0x7fe764146b90 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764072ab0 con 0x7fe7480039e0 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe764072db0 con 0x7fe764158d90 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 0x7fe764147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 0x7fe764147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33336/0 (socket says 172.21.15.165:33336) 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.208+0000 7fe763400640 1 -- 172.21.15.165:0/3799513138 learned_addr learned my addr 172.21.15.165:0/3799513138 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe75a000640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe764158d90 0x7fe76415b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe762a00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7480039e0 0x7fe7641533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe763400640 1 -- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe764158d90 msgr2=0x7fe76415b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe763400640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe764158d90 0x7fe76415b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe763400640 1 -- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7480039e0 msgr2=0x7fe7641533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe763400640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7480039e0 0x7fe7641533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe763400640 1 -- 172.21.15.165:0/3799513138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe764151490 con 0x7fe764146b90 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe762a00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7480039e0 0x7fe7641533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe763400640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 0x7fe764147010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe75c001350 tx=0x7fe75c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.209+0000 7fe75b400640 1 -- 172.21.15.165:0/3799513138 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00ce70 con 0x7fe764146b90 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.210+0000 7fe75b400640 1 -- 172.21.15.165:0/3799513138 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe75c00a610 con 0x7fe764146b90 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.210+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe764159420 con 0x7fe764146b90 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.210+0000 7fe75b400640 1 -- 172.21.15.165:0/3799513138 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c00aa00 con 0x7fe764146b90 2024-04-25T10:06:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.210+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7641599e0 con 0x7fe764146b90 2024-04-25T10:06:30.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.212+0000 7fe75b400640 1 -- 172.21.15.165:0/3799513138 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe75c011070 con 0x7fe764146b90 2024-04-25T10:06:30.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.213+0000 7fe75b400640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe74006b500 0x7fe74006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.213+0000 7fe75b400640 1 -- 172.21.15.165:0/3799513138 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe75c089de0 con 0x7fe764146b90 2024-04-25T10:06:30.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.213+0000 7fe762a00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe74006b500 0x7fe74006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.213+0000 7fe76c597740 1 Processor -- start 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.214+0000 7fe76c597740 1 -- start start 2024-04-25T10:06:30.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.214+0000 7fe762a00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe74006b500 0x7fe74006d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe754032000 tx=0x7fe75402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b4fb30 0x55aa74b77970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe76c597740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b4ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b987a0 0x55aa74b4e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe76c597740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa74ab3620 con 0x55aa74b77eb0 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa74a5cca0 con 0x55aa74b987a0 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa74aaf6c0 con 0x55aa74b4fb30 2024-04-25T10:06:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b4ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b4ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33338/0 (socket says 172.21.15.165:33338) 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 -- 172.21.15.165:0/566833791 learned_addr learned my addr 172.21.15.165:0/566833791 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe762a00640 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b4fb30 0x55aa74b77970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe763400640 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b987a0 0x55aa74b4e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 -- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b4fb30 msgr2=0x55aa74b77970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b4fb30 0x55aa74b77970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 -- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b987a0 msgr2=0x55aa74b4e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b987a0 0x55aa74b4e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.215+0000 7fe75a000640 1 -- 172.21.15.165:0/566833791 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa74a63f40 con 0x55aa74b77eb0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.216+0000 7fe75a000640 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b4ff10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0099e0 tx=0x7fe74c009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa56323f58dd73b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.216+0000 7fe753000640 1 -- 172.21.15.165:0/566833791 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c019070 con 0x55aa74b77eb0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.216+0000 7fe753000640 1 -- 172.21.15.165:0/566833791 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c011b00 con 0x55aa74b77eb0 2024-04-25T10:06:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.216+0000 7fe753000640 1 -- 172.21.15.165:0/566833791 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c015460 con 0x55aa74b77eb0 2024-04-25T10:06:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 -- 172.21.15.165:0/566833791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 msgr2=0x55aa74b4ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b4ff10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0099e0 tx=0x7fe74c009190 comp rx=0 tx=0).stop 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 -- 172.21.15.165:0/566833791 shutdown_connections 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b987a0 0x55aa74b4e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b4ff10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 --2- 172.21.15.165:0/566833791 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b4fb30 0x55aa74b77970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 -- 172.21.15.165:0/566833791 >> 172.21.15.165:0/566833791 conn(0x55aa74b4cd50 msgr2=0x55aa74b95730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 -- 172.21.15.165:0/566833791 shutdown_connections 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.217+0000 7fe76c597740 1 -- 172.21.15.165:0/566833791 wait complete. 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.218+0000 7fe76c597740 1 -- start start 2024-04-25T10:06:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.218+0000 7fe76c597740 1 Processor -- start 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b77eb0 0x55aa74b6f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74ba6180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b6b7e0 0x55aa74b6dc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa74b774e0 con 0x55aa74b987a0 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa74a521f0 con 0x55aa74b77eb0 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa74b76ed0 con 0x55aa74b6b7e0 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe76c597740 10 client.0 _ll_register_callbacks cb 0x55aa74a6fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74ba6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74ba6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33354/0 (socket says 172.21.15.165:33354) 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe75a000640 1 -- 172.21.15.165:0/1553777237 learned_addr learned my addr 172.21.15.165:0/1553777237 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe763400640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b6b7e0 0x55aa74b6dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.219+0000 7fe762a00640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b77eb0 0x55aa74b6f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe75a000640 1 -- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b6b7e0 msgr2=0x55aa74b6dc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe75a000640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b6b7e0 0x55aa74b6dc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe75a000640 1 -- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b77eb0 msgr2=0x55aa74b6f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe75a000640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b77eb0 0x55aa74b6f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe75a000640 1 -- 172.21.15.165:0/1553777237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa74a71c50 con 0x55aa74b987a0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe763400640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b6b7e0 0x55aa74b6dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe762a00640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b77eb0 0x55aa74b6f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe75a000640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74ba6180 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe74c018000 tx=0x7fe74c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ff557bda90a53a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe747400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe747400640 1 -- 172.21.15.165:0/1553777237 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c019050 con 0x55aa74b987a0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe747400640 1 -- 172.21.15.165:0/1553777237 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe74c011900 con 0x55aa74b987a0 2024-04-25T10:06:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe74c011b00 con 0x55aa74b987a0 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.220+0000 7fe747400640 1 -- 172.21.15.165:0/1553777237 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe74c004500 con 0x55aa74b987a0 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 10 client.5934 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55aa747c82d0 con 0x55aa74b987a0 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe747400640 1 -- 172.21.15.165:0/1553777237 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c011b00 con 0x55aa74b987a0 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe747400640 1 -- 172.21.15.165:0/1553777237 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe74c033080 con 0x55aa74b987a0 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 10 client.5934 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 10 client.5934 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 20 client.5934 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b73af0 0x55aa74b73ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 4 client.5934 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.221+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa74ac0b60 con 0x55aa74b73af0 2024-04-25T10:06:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.222+0000 7fe762a00640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b73af0 0x55aa74b73ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.223+0000 7fe762a00640 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b73af0 0x55aa74b73ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7ce34b9fe9688499 server_cookie=6d5a0fe2322af5bd in_seq=0 out_seq=0 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.223+0000 7fe747400640 10 client.5934 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.224+0000 7fe747400640 1 -- 172.21.15.165:0/1553777237 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aa74ac0b60 con 0x55aa74b73af0 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.224+0000 7fe747400640 10 client.5934 handle_command_reply: tid=0 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.224+0000 7fe76c597740 1 client.5934 shutdown 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.224+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 msgr2=0x55aa74ba6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.224+0000 7fe76c597740 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74ba6180 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe74c018000 tx=0x7fe74c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.224+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 shutdown_connections 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b73af0 0x55aa74b73ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b6b7e0 0x55aa74b6dc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74ba6180 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 --2- 172.21.15.165:0/1553777237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b77eb0 0x55aa74b6f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 >> 172.21.15.165:0/1553777237 conn(0x55aa74b4fb30 msgr2=0x55aa74b498b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe747400640 0 client.5934 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 shutdown_connections 2024-04-25T10:06:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 1 -- 172.21.15.165:0/1553777237 wait complete. 2024-04-25T10:06:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.225+0000 7fe76c597740 20 client.5934 trim_cache size 0 max 16384 2024-04-25T10:06:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 Processor -- start 2024-04-25T10:06:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 -- start start 2024-04-25T10:06:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 0x55aa74b498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b49df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b987a0 0x55aa74b685b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641ffc80 con 0x55aa74b77eb0 2024-04-25T10:06:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe764156fc0 con 0x55aa74b6b7e0 2024-04-25T10:06:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.243+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7641ff680 con 0x55aa74b987a0 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b49df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b49df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33356/0 (socket says 172.21.15.165:33356) 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 -- 172.21.15.165:0/3159023534 learned_addr learned my addr 172.21.15.165:0/3159023534 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 -- 172.21.15.165:0/3159023534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b987a0 msgr2=0x55aa74b685b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b987a0 0x55aa74b685b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 -- 172.21.15.165:0/3159023534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 msgr2=0x55aa74b498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 0x55aa74b498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 -- 172.21.15.165:0/3159023534 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa748926b0 con 0x55aa74b77eb0 2024-04-25T10:06:30.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.244+0000 7fe75a000640 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b49df0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe74c003760 tx=0x7fe74c023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e574c4d6147e255c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe752600640 1 -- 172.21.15.165:0/3159023534 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c019040 con 0x55aa74b77eb0 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe752600640 1 -- 172.21.15.165:0/3159023534 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c011b00 con 0x55aa74b77eb0 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe752600640 1 -- 172.21.15.165:0/3159023534 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c003a20 con 0x55aa74b77eb0 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 -- 172.21.15.165:0/3159023534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 msgr2=0x55aa74b49df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b49df0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fe74c003760 tx=0x7fe74c023000 comp rx=0 tx=0).stop 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 -- 172.21.15.165:0/3159023534 shutdown_connections 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b987a0 0x55aa74b685b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b77eb0 0x55aa74b49df0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 --2- 172.21.15.165:0/3159023534 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 0x55aa74b498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.245+0000 7fe76c597740 1 -- 172.21.15.165:0/3159023534 >> 172.21.15.165:0/3159023534 conn(0x55aa74b4fb30 msgr2=0x55aa74b95530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.246+0000 7fe76c597740 1 -- 172.21.15.165:0/3159023534 shutdown_connections 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.246+0000 7fe76c597740 1 -- 172.21.15.165:0/3159023534 wait complete. 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.246+0000 7fe76c597740 1 -- start start 2024-04-25T10:06:30.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.246+0000 7fe76c597740 1 Processor -- start 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 0x55aa74b69650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b77eb0 0x55aa74be6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74b498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa74ba69f0 con 0x55aa74b987a0 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa74b6d6e0 con 0x55aa74b6b7e0 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aa74b6e140 con 0x55aa74b77eb0 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.247+0000 7fe76c597740 10 client.0 _ll_register_callbacks cb 0x7fe74c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74b498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74b498b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33364/0 (socket says 172.21.15.165:33364) 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 -- 172.21.15.165:0/2448297471 learned_addr learned my addr 172.21.15.165:0/2448297471 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe75a000640 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b77eb0 0x55aa74be6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 -- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b77eb0 msgr2=0x55aa74be6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b77eb0 0x55aa74be6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 -- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 msgr2=0x55aa74b69650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 0x55aa74b69650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 -- 172.21.15.165:0/2448297471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aa74b6ed00 con 0x55aa74b987a0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe763400640 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74b498b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe75c088080 tx=0x7fe75c0583f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d4c8ff1a60aac2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.248+0000 7fe747400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe747400640 1 -- 172.21.15.165:0/2448297471 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75c087060 con 0x55aa74b987a0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe747400640 1 -- 172.21.15.165:0/2448297471 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe75c0605b0 con 0x55aa74b987a0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe74c011b00 con 0x55aa74b987a0 2024-04-25T10:06:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe747400640 1 -- 172.21.15.165:0/2448297471 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe75c076050 con 0x55aa74b987a0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 10 client.5940 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55aa74b4eed0 con 0x55aa74b987a0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe747400640 1 -- 172.21.15.165:0/2448297471 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c011b00 con 0x55aa74b987a0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe747400640 1 -- 172.21.15.165:0/2448297471 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe75c06d040 con 0x55aa74b987a0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 10 client.5940 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 10 client.5940 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 20 client.5940 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b4e7b0 0x55aa74baece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 4 client.5940 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.249+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55aa74b4d4b0 con 0x55aa74b4e7b0 2024-04-25T10:06:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.250+0000 7fe762a00640 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b4e7b0 0x55aa74baece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe762a00640 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b4e7b0 0x55aa74baece0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3303791305f6701e server_cookie=a0173b35aa062f5e in_seq=0 out_seq=0 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe747400640 10 client.5940 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe747400640 1 -- 172.21.15.165:0/2448297471 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55aa74b4d4b0 con 0x55aa74b4e7b0 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe747400640 10 client.5940 handle_command_reply: tid=0 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 client.5940 shutdown 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 msgr2=0x55aa74b498b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74b498b0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fe75c088080 tx=0x7fe75c0583f0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 shutdown_connections 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55aa74b4e7b0 0x55aa74baece0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aa74b987a0 0x55aa74b498b0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aa74b77eb0 0x55aa74be6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 --2- 172.21.15.165:0/2448297471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aa74b6b7e0 0x55aa74b69650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 >> 172.21.15.165:0/2448297471 conn(0x55aa74b4fb30 msgr2=0x55aa74bafdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.251+0000 7fe747400640 0 client.5940 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 shutdown_connections 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe76c597740 1 -- 172.21.15.165:0/2448297471 wait complete. 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe76c597740 20 client.5940 trim_cache size 0 max 16384 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 5811, 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:06:30.260 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.6592093429999997, 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 28, 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 54, 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 54, 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 12.100311221 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe74006b500 msgr2=0x7fe74006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe74006b500 0x7fe74006d9c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fe754032000 tx=0x7fe75402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 msgr2=0x7fe764147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 0x7fe764147010 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fe75c001350 tx=0x7fe75c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 shutdown_connections 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe74006b500 0x7fe74006d9c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe764146b90 0x7fe764147010 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe764158d90 0x7fe76415b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 --2- 172.21.15.165:0/3799513138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7480039e0 0x7fe7641533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 >> 172.21.15.165:0/3799513138 conn(0x7fe76406e3b0 msgr2=0x7fe7640afb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.252+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 shutdown_connections 2024-04-25T10:06:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.253+0000 7fe763e00640 1 -- 172.21.15.165:0/3799513138 wait complete. 2024-04-25T10:06:30.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:06:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.461+0000 7fb01ea00640 1 Processor -- start 2024-04-25T10:06:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.461+0000 7fb01ea00640 1 -- start start 2024-04-25T10:06:30.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.463+0000 7fb01ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb018071700 0x7fb0180cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.463+0000 7fb01ea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01815bf30 con 0x7fb018144440 2024-04-25T10:06:30.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.463+0000 7fb01ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb01815c210 con 0x7fb0180cd860 2024-04-25T10:06:30.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.463+0000 7fb01ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb01815c860 con 0x7fb018071700 2024-04-25T10:06:30.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.463+0000 7fb01d600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb018144440 0x7fb018070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56730/0 (socket says 172.21.15.165:56730) 2024-04-25T10:06:30.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.463+0000 7fb01d600640 1 -- 172.21.15.165:0/1765409178 learned_addr learned my addr 172.21.15.165:0/1765409178 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb01cc00640 1 --2- 172.21.15.165:0/1765409178 >> v2:172.21.15.172:3300/0 conn(0x7fb018071700 0x7fb0180cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448366826 0 0) 0x7fb01815bf30 con 0x7fb018144440 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb000003520 con 0x7fb018144440 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 755884783 0 0) 0x7fb000003520 con 0x7fb018144440 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 >> v2:172.21.15.172:3300/0 conn(0x7fb018071700 msgr2=0x7fb0180cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 --2- 172.21.15.165:0/1765409178 >> v2:172.21.15.172:3300/0 conn(0x7fb018071700 0x7fb0180cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 >> v1:172.21.15.172:6789/0 conn(0x7fb0180cd860 legacy=0x7fb01815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01815c4f0 con 0x7fb018144440 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb00c001670 con 0x7fb018144440 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 >> v1:172.21.15.165:6789/0 conn(0x7fb018144440 legacy=0x7fb018070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000039b0 0x7fb000003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb000004800 0x7fb000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0000051c0 0x7fb00000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb01815c860 con 0x7fb0000051c0 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00000c070 con 0x7fb000004800 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.464+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00000c270 con 0x7fb0000039b0 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01e000640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0000051c0 0x7fb00000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01e000640 1 -- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000039b0 msgr2=0x7fb000003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01e000640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000039b0 0x7fb000003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01e000640 1 -- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb000004800 msgr2=0x7fb000004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01cc00640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb000004800 0x7fb000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01e000640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb000004800 0x7fb000004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.465+0000 7fb01e000640 1 -- 172.21.15.165:0/1765409178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb000003520 con 0x7fb0000051c0 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.466+0000 7fb01e000640 1 -- 172.21.15.165:0/1765409178 reap_dead start 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.466+0000 7fb01e000640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0000051c0 0x7fb00000b9c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb010001e50 tx=0x7fb000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5582033c3fbf41 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.466+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010001a70 con 0x7fb0000051c0 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.466+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb010003970 con 0x7fb0000051c0 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.466+0000 7fb017e00640 1 -- 172.21.15.165:0/1765409178 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb010003d60 con 0x7fb0000051c0 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 -- 172.21.15.165:0/1765409178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0000051c0 msgr2=0x7fb00000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0000051c0 0x7fb00000b9c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fb010001e50 tx=0x7fb000005830 comp rx=0 tx=0).stop 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 -- 172.21.15.165:0/1765409178 shutdown_connections 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 --2- 172.21.15.165:0/1765409178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0000051c0 0x7fb00000b9c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 -- 172.21.15.165:0/1765409178 >> 172.21.15.165:0/1765409178 conn(0x7fb01806abe0 msgr2=0x7fb01806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 -- 172.21.15.165:0/1765409178 shutdown_connections 2024-04-25T10:06:30.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.467+0000 7fb01ea00640 1 -- 172.21.15.165:0/1765409178 wait complete. 2024-04-25T10:06:30.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.469+0000 7fb01ea00640 1 Processor -- start 2024-04-25T10:06:30.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.469+0000 7fb01ea00640 1 -- start start 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000051c0 0x7fb01813ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 0x7fb0180cc650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0181e4120 0x7fb0181e45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180cde70 con 0x7fb0180cc1d0 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180cdcf0 con 0x7fb0181e4120 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0180cd320 con 0x7fb0000051c0 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 0x7fb0180cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 0x7fb0180cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33386/0 (socket says 172.21.15.165:33386) 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.470+0000 7fb01cc00640 1 -- 172.21.15.165:0/365682654 learned_addr learned my addr 172.21.15.165:0/365682654 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:30.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01d600640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000051c0 0x7fb01813ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01e000640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0181e4120 0x7fb0181e45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01cc00640 1 -- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000051c0 msgr2=0x7fb01813ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01cc00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000051c0 0x7fb01813ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01cc00640 1 -- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0181e4120 msgr2=0x7fb0181e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01cc00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0181e4120 0x7fb0181e45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01cc00640 1 -- 172.21.15.165:0/365682654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb01813d3a0 con 0x7fb0180cc1d0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb01cc00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 0x7fb0180cc650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb004010860 tx=0x7fb004010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400f040 con 0x7fb0180cc1d0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.471+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00400eb60 con 0x7fb0180cc1d0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.472+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00401c400 con 0x7fb0180cc1d0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.472+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb01814c870 con 0x7fb0180cc1d0 2024-04-25T10:06:30.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.472+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb01814ce30 con 0x7fb0180cc1d0 2024-04-25T10:06:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.473+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0180ccb90 con 0x7fb0180cc1d0 2024-04-25T10:06:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.474+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb00401c5a0 con 0x7fb0180cc1d0 2024-04-25T10:06:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.474+0000 7fb015600640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faff806b4d0 0x7faff806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.475+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb00408aa70 con 0x7fb0180cc1d0 2024-04-25T10:06:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.475+0000 7fb01d600640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faff806b4d0 0x7faff806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.475+0000 7fb01d600640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faff806b4d0 0x7faff806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb00c001eb0 tx=0x7fb00c001cc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.479+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb01813d3a0 con 0x7fb0180cc1d0 2024-04-25T10:06:30.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.717+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb0180ce480 con 0x7fb0180cc1d0 2024-04-25T10:06:30.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.718+0000 7fb015600640 1 -- 172.21.15.165:0/365682654 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb0180ce480 con 0x7fb0180cc1d0 2024-04-25T10:06:30.724 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faff806b4d0 msgr2=0x7faff806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faff806b4d0 0x7faff806d990 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb00c001eb0 tx=0x7fb00c001cc0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 msgr2=0x7fb0180cc650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 0x7fb0180cc650 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb004010860 tx=0x7fb004010d20 comp rx=0 tx=0).stop 2024-04-25T10:06:30.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 shutdown_connections 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faff806b4d0 0x7faff806d990 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.720+0000 7fb01ea00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0181e4120 0x7fb0181e45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.721+0000 7fb01ea00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0180cc1d0 0x7fb0180cc650 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.721+0000 7fb01ea00640 1 --2- 172.21.15.165:0/365682654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb0000051c0 0x7fb01813ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.721+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 >> 172.21.15.165:0/365682654 conn(0x7fb01806abe0 msgr2=0x7fb0180b17f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.721+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 shutdown_connections 2024-04-25T10:06:30.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:30.721+0000 7fb01ea00640 1 -- 172.21.15.165:0/365682654 wait complete. 2024-04-25T10:06:30.744 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:06:30.744 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:06:30.744 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:06:30.745 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:06:30.745 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:30.745 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:06:30.745 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:06:30.745 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:06:30.745 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:30.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:30.745 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.795 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42375sec preferred_lft 42375sec 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.796 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:06:30.797 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:06:30.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:30.798 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:06:30.874 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:06:30.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:30.878 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:06:30.930 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:06:30.930 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:06:30.930 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:06:30.930 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:06:30.931 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:06:30.931 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:30.931 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:06:30.931 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:06:30.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:06:30.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:06:30.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:06:30.931 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:06:31.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.078 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:06:31.127 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:06:31.127 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:06:31.128 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:31.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.128 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:06:31.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:31.211 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:31.264 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:06:31.264 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:06:31.264 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:31.314 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:06:31.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.314 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:31.442 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:31.442 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:31.442 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:06:31.464 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:31.464 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:31.464 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:06:31.464 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:06:31.464 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:06:31.491 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:06:31.493 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:06:31.493 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:06:31.493 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:06:31.493 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:06:31.494 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:06:31.497 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:06:31.497 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:31.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.564 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:06:31.637 INFO:teuthology.orchestra.run.smithi165.stdout:client4826 2024-04-25T10:06:31.638 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:31.638 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:31.688 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:31.688 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:06:31.688 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:06:31.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.689 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> import json 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> import sys 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> except OSError as e: 2024-04-25T10:06:31.801 DEBUG:teuthology.orchestra.run.smithi165:> sys.exit(e.errno) 2024-04-25T10:06:31.802 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:06:31.802 DEBUG:teuthology.orchestra.run.smithi165:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:06:31.802 DEBUG:teuthology.orchestra.run.smithi165:> print(json.dumps( 2024-04-25T10:06:31.802 DEBUG:teuthology.orchestra.run.smithi165:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:06:31.802 DEBUG:teuthology.orchestra.run.smithi165:> indent=2)) 2024-04-25T10:06:31.802 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:06:31.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:31 stdin-killer INFO: command exited with status 2: exiting normally with same code! 2024-04-25T10:06:31.962 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-25T10:06:31.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:31.963 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec git clone --branch giant http://github.com/ceph/ceph-qa-suite /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite) 2024-04-25T10:06:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:Cloning into '/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'... 2024-04-25T10:06:32.372 INFO:teuthology.orchestra.run.smithi165.stderr:warning: redirecting to https://github.com/ceph/ceph-qa-suite/ 2024-04-25T10:06:40.368 INFO:teuthology.orchestra.run.smithi165.stderr:Updating files: 17% (249/1398) Updating files: 18% (252/1398) Updating files: 19% (266/1398) Updating files: 20% (280/1398) Updating files: 21% (294/1398) Updating files: 22% (308/1398) Updating files: 23% (322/1398) Updating files: 24% (336/1398) Updating files: 25% (350/1398) Updating files: 26% (364/1398) Updating files: 27% (378/1398) Updating files: 28% (392/1398) Updating files: 29% (406/1398) Updating files: 30% (420/1398) Updating files: 31% (434/1398) Updating files: 32% (448/1398) Updating files: 33% (462/1398) Updating files: 34% (476/1398) Updating files: 35% (490/1398) Updating files: 36% (504/1398) Updating files: 36% (511/1398) Updating files: 37% (518/1398) Updating files: 38% (532/1398) Updating files: 39% (546/1398) Updating files: 40% (560/1398) Updating files: 41% (574/1398) Updating files: 42% (588/1398) Updating files: 43% (602/1398) Updating files: 44% (616/1398) Updating files: 45% (630/1398) Updating files: 46% (644/1398) Updating files: 47% (658/1398) Updating files: 48% (672/1398) Updating files: 49% (686/1398) Updating files: 50% (699/1398) Updating files: 51% (713/1398) Updating files: 52% (727/1398) Updating files: 53% (741/1398) Updating files: 54% (755/1398) Updating files: 54% (759/1398) Updating files: 55% (769/1398) Updating files: 56% (783/1398) Updating files: 57% (797/1398) Updating files: 58% (811/1398) Updating files: 59% (825/1398) Updating files: 60% (839/1398) Updating files: 61% (853/1398) Updating files: 62% (867/1398) Updating files: 63% (881/1398) Updating files: 63% (886/1398) Updating files: 64% (895/1398) Updating files: 65% (909/1398) Updating files: 66% (923/1398) Updating files: 67% (937/1398) Updating files: 68% (951/1398) Updating files: 69% (965/1398) Updating files: 70% (979/1398) Updating files: 71% (993/1398) Updating files: 72% (1007/1398) Updating files: 73% (1021/1398) Updating files: 74% (1035/1398) Updating files: 75% (1049/1398) Updating files: 76% (1063/1398) Updating files: 77% (1077/1398) Updating files: 77% (1087/1398) Updating files: 78% (1091/1398) Updating files: 79% (1105/1398) Updating files: 80% (1119/1398) Updating files: 81% (1133/1398) Updating files: 82% (1147/1398) Updating files: 83% (1161/1398) Updating files: 84% (1175/1398) Updating files: 85% (1189/1398) Updating files: 86% (1203/1398) Updating files: 87% (1217/1398) Updating files: 88% (1231/1398) Updating files: 89% (1245/1398) Updating files: 90% (1259/1398) Updating files: 91% (1273/1398) Updating files: 92% (1287/1398) Updating files: 93% (1301/1398) Updating files: 94% (1315/1398) Updating files: 95% (1329/1398) Updating files: 96% (1343/1398) Updating files: 97% (1357/1398) Updating files: 98% (1371/1398) Updating files: 99% (1385/1398) Updating files: 100% (1398/1398) Updating files: 100% (1398/1398), done. 2024-04-25T10:06:40.384 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 0 2024-04-25T10:06:40.384 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:06:40.384 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:40.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.588+0000 7f7b27400640 1 Processor -- start 2024-04-25T10:06:40.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.589+0000 7f7b27400640 1 -- start start 2024-04-25T10:06:40.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.590+0000 7f7b27400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b200ce920 0x7f7b200ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.590+0000 7f7b27400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7b2006d290 0x7f7b2006d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.590+0000 7f7b27400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b20144b30 con 0x7f7b20144750 2024-04-25T10:06:40.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.590+0000 7f7b27400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b200712d0 con 0x7f7b2006d290 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.590+0000 7f7b27400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b200715b0 con 0x7f7b200ce920 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.590+0000 7f7b25400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7b2006d290 0x7f7b2006d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b25400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7b2006d290 0x7f7b2006d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55450/0 (socket says 172.21.15.165:55450) 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b25400640 1 -- 172.21.15.165:0/2086001459 learned_addr learned my addr 172.21.15.165:0/2086001459 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b17e00640 1 --2- 172.21.15.165:0/2086001459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b200ce920 0x7f7b200ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b25400640 1 -- 172.21.15.165:0/2086001459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b200ce920 msgr2=0x7f7b200ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b25400640 1 --2- 172.21.15.165:0/2086001459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b200ce920 0x7f7b200ced80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b25400640 1 -- 172.21.15.165:0/2086001459 >> v1:172.21.15.172:6789/0 conn(0x7f7b20144750 legacy=0x7f7b2006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.591+0000 7f7b25400640 1 -- 172.21.15.165:0/2086001459 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b20071890 con 0x7f7b2006d290 2024-04-25T10:06:40.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.592+0000 7f7b25400640 1 --2- 172.21.15.165:0/2086001459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b2006d290 0x7f7b2006d720 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009e50 tx=0x7f7b1802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3b3bac38ab53a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.592+0000 7f7b1fe00640 1 -- 172.21.15.165:0/2086001459 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b18036070 con 0x7f7b2006d290 2024-04-25T10:06:40.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.592+0000 7f7b1fe00640 1 -- 172.21.15.165:0/2086001459 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b18031600 con 0x7f7b2006d290 2024-04-25T10:06:40.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.592+0000 7f7b1fe00640 1 -- 172.21.15.165:0/2086001459 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b180319d0 con 0x7f7b2006d290 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 -- 172.21.15.165:0/2086001459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b2006d290 msgr2=0x7f7b2006d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 --2- 172.21.15.165:0/2086001459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b2006d290 0x7f7b2006d720 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7b18009e50 tx=0x7f7b1802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 -- 172.21.15.165:0/2086001459 shutdown_connections 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 --2- 172.21.15.165:0/2086001459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b2006d290 0x7f7b2006d720 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 --2- 172.21.15.165:0/2086001459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b200ce920 0x7f7b200ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 -- 172.21.15.165:0/2086001459 >> 172.21.15.165:0/2086001459 conn(0x7f7b2006abe0 msgr2=0x7f7b2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 -- 172.21.15.165:0/2086001459 shutdown_connections 2024-04-25T10:06:40.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.593+0000 7f7b27400640 1 -- 172.21.15.165:0/2086001459 wait complete. 2024-04-25T10:06:40.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.595+0000 7f7b27400640 1 Processor -- start 2024-04-25T10:06:40.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.595+0000 7f7b27400640 1 -- start start 2024-04-25T10:06:40.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b27400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b2006d290 0x7f7b200addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b27400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 0x7f7b20159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b27400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b20144750 0x7f7b2015f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b27400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2006f020 con 0x7f7b200ce920 2024-04-25T10:06:40.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b27400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b2006ece0 con 0x7f7b2006d290 2024-04-25T10:06:40.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b27400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b20145450 con 0x7f7b20144750 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 0x7f7b20159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b17e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 0x7f7b20159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55466/0 (socket says 172.21.15.165:55466) 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b17e00640 1 -- 172.21.15.165:0/2036952421 learned_addr learned my addr 172.21.15.165:0/2036952421 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.596+0000 7f7b24a00640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b2006d290 0x7f7b200addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b25400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b20144750 0x7f7b2015f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b17e00640 1 -- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b20144750 msgr2=0x7f7b2015f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b17e00640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b20144750 0x7f7b2015f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b17e00640 1 -- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b2006d290 msgr2=0x7f7b200addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b17e00640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b2006d290 0x7f7b200addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b17e00640 1 -- 172.21.15.165:0/2036952421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b200acd20 con 0x7f7b200ce920 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b17e00640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 0x7f7b20159a00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c7f0 tx=0x7f7b0800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08015070 con 0x7f7b200ce920 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b08004240 con 0x7f7b200ce920 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b08007620 con 0x7f7b200ce920 2024-04-25T10:06:40.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b200ad000 con 0x7f7b200ce920 2024-04-25T10:06:40.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.597+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b200ad5c0 con 0x7f7b200ce920 2024-04-25T10:06:40.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.598+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b200685e0 con 0x7f7b200ce920 2024-04-25T10:06:40.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.599+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b080079d0 con 0x7f7b200ce920 2024-04-25T10:06:40.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.600+0000 7f7b1d600640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:40.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.600+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b0808a530 con 0x7f7b200ce920 2024-04-25T10:06:40.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.601+0000 7f7b24a00640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:40.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.601+0000 7f7b24a00640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7afc06b4d0 0x7f7afc06d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b10004ea0 tx=0x7f7b10009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:40.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.605+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b200685e0 con 0x7f7b200ce920 2024-04-25T10:06:40.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.846+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b20073990 con 0x7f7b200ce920 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.847+0000 7f7b1d600640 1 -- 172.21.15.165:0/2036952421 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f7b08059180 con 0x7f7b200ce920 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:40.853 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:40.854 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7afc06b4d0 msgr2=0x7f7afc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7afc06b4d0 0x7f7afc06d990 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f7b10004ea0 tx=0x7f7b10009040 comp rx=0 tx=0).stop 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 msgr2=0x7f7b20159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 0x7f7b20159a00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7b0800c7f0 tx=0x7f7b0800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 shutdown_connections 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7afc06b4d0 0x7f7afc06d990 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b20144750 0x7f7b2015f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b200ce920 0x7f7b20159a00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 --2- 172.21.15.165:0/2036952421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b2006d290 0x7f7b200addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 >> 172.21.15.165:0/2036952421 conn(0x7f7b2006abe0 msgr2=0x7f7b200b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:40.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 shutdown_connections 2024-04-25T10:06:40.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:40.850+0000 7f7b27400640 1 -- 172.21.15.165:0/2036952421 wait complete. 2024-04-25T10:06:40.876 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:06:41.089 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:41.089 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": -2 2024-04-25T10:06:41.089 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:41.104 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:06:41.105 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:06:41.105 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:41.105 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:06:41.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.302+0000 7f0a5a600640 1 Processor -- start 2024-04-25T10:06:41.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.302+0000 7f0a5a600640 1 -- start start 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a5a600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a5415b860 0x7f0a54072200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a5a600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0a540cd2f0 0x7f0a5415bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540d0c60 0x7f0a540d10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a5a600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a5406e800 con 0x7f0a540cd2f0 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a5a600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a540d15e0 con 0x7f0a5415b860 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a5a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54072880 con 0x7f0a540d0c60 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0a540cd2f0 0x7f0a5415bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0a540cd2f0 0x7f0a5415bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55468/0 (socket says 172.21.15.165:55468) 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 -- 172.21.15.165:0/3453968376 learned_addr learned my addr 172.21.15.165:0/3453968376 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a59200640 1 --2- 172.21.15.165:0/3453968376 >> v2:172.21.15.172:3300/0 conn(0x7f0a5415b860 0x7f0a54072200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a59c00640 1 --2- 172.21.15.165:0/3453968376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540d0c60 0x7f0a540d10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 -- 172.21.15.165:0/3453968376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540d0c60 msgr2=0x7f0a540d10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 --2- 172.21.15.165:0/3453968376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540d0c60 0x7f0a540d10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 -- 172.21.15.165:0/3453968376 >> v2:172.21.15.172:3300/0 conn(0x7f0a5415b860 msgr2=0x7f0a54072200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.304+0000 7f0a53e00640 1 --2- 172.21.15.165:0/3453968376 >> v2:172.21.15.172:3300/0 conn(0x7f0a5415b860 0x7f0a54072200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.305+0000 7f0a53e00640 1 -- 172.21.15.165:0/3453968376 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5406ea50 con 0x7f0a540cd2f0 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.306+0000 7f0a53e00640 1 --2- 172.21.15.165:0/3453968376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a540cd2f0 0x7f0a5415bc40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406acf0 tx=0x7f0a4002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e511709e23af24c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.306+0000 7f0a53400640 1 -- 172.21.15.165:0/3453968376 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40037070 con 0x7f0a540cd2f0 2024-04-25T10:06:41.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.306+0000 7f0a53400640 1 -- 172.21.15.165:0/3453968376 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a40032440 con 0x7f0a540cd2f0 2024-04-25T10:06:41.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.306+0000 7f0a53400640 1 -- 172.21.15.165:0/3453968376 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40032810 con 0x7f0a540cd2f0 2024-04-25T10:06:41.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 -- 172.21.15.165:0/3453968376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a540cd2f0 msgr2=0x7f0a5415bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 --2- 172.21.15.165:0/3453968376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a540cd2f0 0x7f0a5415bc40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0a5406acf0 tx=0x7f0a4002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 -- 172.21.15.165:0/3453968376 shutdown_connections 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 --2- 172.21.15.165:0/3453968376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540d0c60 0x7f0a540d10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 --2- 172.21.15.165:0/3453968376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a540cd2f0 0x7f0a5415bc40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 --2- 172.21.15.165:0/3453968376 >> v2:172.21.15.172:3300/0 conn(0x7f0a5415b860 0x7f0a54072200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 -- 172.21.15.165:0/3453968376 >> 172.21.15.165:0/3453968376 conn(0x7f0a540b5d00 msgr2=0x7f0a540b60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 -- 172.21.15.165:0/3453968376 shutdown_connections 2024-04-25T10:06:41.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.307+0000 7f0a5a600640 1 -- 172.21.15.165:0/3453968376 wait complete. 2024-04-25T10:06:41.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.309+0000 7f0a5a600640 1 Processor -- start 2024-04-25T10:06:41.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.309+0000 7f0a5a600640 1 -- start start 2024-04-25T10:06:41.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540cd2f0 0x7f0a5414a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a540d0c60 0x7f0a5414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a5a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 0x7f0a5414da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a5a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a5415c180 con 0x7f0a5415b860 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a5a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cd740 con 0x7f0a540d0c60 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a5a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a540cfad0 con 0x7f0a540cd2f0 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 0x7f0a5414da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 0x7f0a5414da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55474/0 (socket says 172.21.15.165:55474) 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a59c00640 1 -- 172.21.15.165:0/568773591 learned_addr learned my addr 172.21.15.165:0/568773591 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a59200640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540cd2f0 0x7f0a5414a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.310+0000 7f0a59c00640 1 -- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540cd2f0 msgr2=0x7f0a5414a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a59c00640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540cd2f0 0x7f0a5414a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a59c00640 1 -- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a540d0c60 msgr2=0x7f0a5414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a53e00640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a540d0c60 0x7f0a5414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a59c00640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a540d0c60 0x7f0a5414b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a59c00640 1 -- 172.21.15.165:0/568773591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a5414c1f0 con 0x7f0a5415b860 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a59200640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540cd2f0 0x7f0a5414a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a59c00640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 0x7f0a5414da00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00ccf0 tx=0x7f0a4c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a50c00640 1 -- 172.21.15.165:0/568773591 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c003b90 con 0x7f0a5415b860 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a50c00640 1 -- 172.21.15.165:0/568773591 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c003d30 con 0x7f0a5415b860 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a50c00640 1 -- 172.21.15.165:0/568773591 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c004240 con 0x7f0a5415b860 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a541320f0 con 0x7f0a5415b860 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.311+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a5414a720 con 0x7f0a5415b860 2024-04-25T10:06:41.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.313+0000 7f0a50c00640 1 -- 172.21.15.165:0/568773591 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a4c0043e0 con 0x7f0a5415b860 2024-04-25T10:06:41.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.314+0000 7f0a50c00640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a3406b4d0 0x7f0a3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.314+0000 7f0a50c00640 1 -- 172.21.15.165:0/568773591 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a4c08aeb0 con 0x7f0a5415b860 2024-04-25T10:06:41.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.315+0000 7f0a59200640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a3406b4d0 0x7f0a3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.315+0000 7f0a59200640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a3406b4d0 0x7f0a3406d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b820 tx=0x7f0a48008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.315+0000 7f0a5ea3a740 1 Processor -- start 2024-04-25T10:06:41.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.315+0000 7f0a5ea3a740 1 -- start start 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.316+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c1407d20 0x5559c1409e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.316+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c1408100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.316+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 0x5559c1432c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.316+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c136ab90 con 0x5559c14327e0 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.316+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c13140b0 con 0x5559c14443f0 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.316+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559c1366c30 con 0x5559c1407d20 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 0x5559c1432c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 0x5559c1432c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55490/0 (socket says 172.21.15.165:55490) 2024-04-25T10:06:41.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 -- 172.21.15.165:0/1693954250 learned_addr learned my addr 172.21.15.165:0/1693954250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 -- 172.21.15.165:0/1693954250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c1407d20 msgr2=0x5559c1409e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c1407d20 0x5559c1409e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 -- 172.21.15.165:0/1693954250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 msgr2=0x5559c1408100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c1408100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 -- 172.21.15.165:0/1693954250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c114b9f0 con 0x5559c14327e0 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a59c00640 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 0x5559c1432c60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c055860 tx=0x7f0a4c059300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=879e1efc9d928124 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a45600640 1 -- 172.21.15.165:0/1693954250 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c087070 con 0x5559c14327e0 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a45600640 1 -- 172.21.15.165:0/1693954250 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c061b10 con 0x5559c14327e0 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.317+0000 7f0a45600640 1 -- 172.21.15.165:0/1693954250 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0046d0 con 0x5559c14327e0 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/1693954250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 msgr2=0x5559c1432c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 0x5559c1432c60 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c055860 tx=0x7f0a4c059300 comp rx=0 tx=0).stop 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/1693954250 shutdown_connections 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14327e0 0x5559c1432c60 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c1408100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/1693954250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c1407d20 0x5559c1409e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/1693954250 >> 172.21.15.165:0/1693954250 conn(0x5559c13cffb0 msgr2=0x5559c144d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/1693954250 shutdown_connections 2024-04-25T10:06:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.318+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/1693954250 wait complete. 2024-04-25T10:06:41.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.319+0000 7f0a5ea3a740 1 -- start start 2024-04-25T10:06:41.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.319+0000 7f0a5ea3a740 1 Processor -- start 2024-04-25T10:06:41.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c1424070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1434680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c145d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c1309570 con 0x5559c1407d20 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c1430330 con 0x5559c14443f0 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559c145de80 con 0x5559c14327e0 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a5ea3a740 10 client.0 _ll_register_callbacks cb 0x5559c107edd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a59200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c1424070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.320+0000 7f0a59200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c1424070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55500/0 (socket says 172.21.15.165:55500) 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 -- 172.21.15.165:0/2312559117 learned_addr learned my addr 172.21.15.165:0/2312559117 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a53e00640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1434680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 -- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 msgr2=0x5559c1434680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59c00640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c145d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1434680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 -- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 msgr2=0x5559c145d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c145d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 -- 172.21.15.165:0/2312559117 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559c103f2e0 con 0x5559c1407d20 2024-04-25T10:06:41.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a59200640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c1424070 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a480126f0 tx=0x7f0a48012720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dec0c01f060deec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.321+0000 7f0a3a000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a3a000640 1 -- 172.21.15.165:0/2312559117 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48023060 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a4c061b10 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a3a000640 1 -- 172.21.15.165:0/2312559117 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a4801e600 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a3a000640 1 -- 172.21.15.165:0/2312559117 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a48029050 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 10 client.5964 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5559c11ac180 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a3a000640 1 -- 172.21.15.165:0/2312559117 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c061b10 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a3a000640 1 -- 172.21.15.165:0/2312559117 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a4802c020 con 0x5559c1407d20 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 10 client.5964 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 10 client.5964 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 20 client.5964 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c142eec0 0x5559c1435f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 4 client.5964 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.322+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5559c13780d0 con 0x5559c142eec0 2024-04-25T10:06:41.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.323+0000 7f0a53e00640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c142eec0 0x5559c1435f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.324+0000 7f0a53e00640 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c142eec0 0x5559c1435f60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=656c9d5365e8a345 server_cookie=597076366701300 in_seq=0 out_seq=0 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.324+0000 7f0a3a000640 10 client.5964 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a3a000640 1 -- 172.21.15.165:0/2312559117 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5559c13780d0 con 0x5559c142eec0 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a3a000640 10 client.5964 handle_command_reply: tid=0 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 client.5964 shutdown 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 msgr2=0x5559c1424070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c1424070 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a480126f0 tx=0x7f0a48012720 comp rx=0 tx=0).stop 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 shutdown_connections 2024-04-25T10:06:41.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c142eec0 0x5559c1435f60 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14443f0 0x5559c145d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1434680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2312559117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c1424070 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 >> 172.21.15.165:0/2312559117 conn(0x5559c13cffb0 msgr2=0x5559c142fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a3a000640 0 client.5964 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 shutdown_connections 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2312559117 wait complete. 2024-04-25T10:06:41.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.326+0000 7f0a5ea3a740 20 client.5964 trim_cache size 0 max 16384 2024-04-25T10:06:41.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.343+0000 7f0a5ea3a740 1 Processor -- start 2024-04-25T10:06:41.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.343+0000 7f0a5ea3a740 1 -- start start 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c1407d20 0x5559c1449550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1443360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 0x5559c14438a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a542002e0 con 0x5559c14443f0 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a541ff4c0 con 0x5559c1407d20 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54134900 con 0x5559c14327e0 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 0x5559c14438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a59c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 0x5559c14438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55516/0 (socket says 172.21.15.165:55516) 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a59c00640 1 -- 172.21.15.165:0/348353233 learned_addr learned my addr 172.21.15.165:0/348353233 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a53e00640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1443360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.344+0000 7f0a59200640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c1407d20 0x5559c1449550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59c00640 1 -- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 msgr2=0x5559c1443360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59c00640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1443360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59c00640 1 -- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c1407d20 msgr2=0x5559c1449550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59c00640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c1407d20 0x5559c1449550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59c00640 1 -- 172.21.15.165:0/348353233 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5559c145d5e0 con 0x5559c14443f0 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a53e00640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1443360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59200640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c1407d20 0x5559c1449550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:41.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a59c00640 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 0x5559c14438a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0891b0 tx=0x7f0a4c06e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350aaa4cca3b5af7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a44c00640 1 -- 172.21.15.165:0/348353233 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c087070 con 0x5559c14443f0 2024-04-25T10:06:41.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a44c00640 1 -- 172.21.15.165:0/348353233 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c061b10 con 0x5559c14443f0 2024-04-25T10:06:41.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.345+0000 7f0a44c00640 1 -- 172.21.15.165:0/348353233 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c0046d0 con 0x5559c14443f0 2024-04-25T10:06:41.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/348353233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 msgr2=0x5559c14438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 0x5559c14438a0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c0891b0 tx=0x7f0a4c06e000 comp rx=0 tx=0).stop 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/348353233 shutdown_connections 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c14443f0 0x5559c14438a0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14327e0 0x5559c1443360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/348353233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c1407d20 0x5559c1449550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/348353233 >> 172.21.15.165:0/348353233 conn(0x5559c13cffb0 msgr2=0x5559c1423e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/348353233 shutdown_connections 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.346+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/348353233 wait complete. 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.347+0000 7f0a5ea3a740 1 -- start start 2024-04-25T10:06:41.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.347+0000 7f0a5ea3a740 1 Processor -- start 2024-04-25T10:06:41.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c14667f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14327e0 0x5559c146d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14443f0 0x5559c146d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5559c1409e30 con 0x5559c1407d20 2024-04-25T10:06:41.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a541ff0c0 con 0x5559c14327e0 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5559c14300e0 con 0x5559c14443f0 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a5ea3a740 10 client.0 _ll_register_callbacks cb 0x7f0a4c0078f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c14667f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c14667f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55524/0 (socket says 172.21.15.165:55524) 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 -- 172.21.15.165:0/2749148382 learned_addr learned my addr 172.21.15.165:0/2749148382 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 -- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14443f0 msgr2=0x5559c146d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14443f0 0x5559c146d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 -- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14327e0 msgr2=0x5559c146d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14327e0 0x5559c146d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.348+0000 7f0a59200640 1 -- 172.21.15.165:0/2749148382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5559c10183a0 con 0x5559c1407d20 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a59200640 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c14667f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800dbd0 tx=0x7f0a48013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1865e57131ddc1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a3a000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a3a000640 1 -- 172.21.15.165:0/2749148382 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a48023070 con 0x5559c1407d20 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a3a000640 1 -- 172.21.15.165:0/2749148382 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a48013800 con 0x5559c1407d20 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0a4c061b10 con 0x5559c1407d20 2024-04-25T10:06:41.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a3a000640 1 -- 172.21.15.165:0/2749148382 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a4800de10 con 0x5559c1407d20 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a5ea3a740 10 client.5970 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.349+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5559c1407240 con 0x5559c1407d20 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a3a000640 1 -- 172.21.15.165:0/2749148382 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4c061b10 con 0x5559c1407d20 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a3a000640 1 -- 172.21.15.165:0/2749148382 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0a4803d020 con 0x5559c1407d20 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a5ea3a740 10 client.5970 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a5ea3a740 10 client.5970 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a5ea3a740 20 client.5970 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c1432c00 0x5559c1422670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a5ea3a740 4 client.5970 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:06:41.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x5559c1372420 con 0x5559c1432c00 2024-04-25T10:06:41.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.350+0000 7f0a53e00640 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c1432c00 0x5559c1422670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.351+0000 7f0a53e00640 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c1432c00 0x5559c1422670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7884fb86e32be36 server_cookie=9255e85642bbce27 in_seq=0 out_seq=0 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.351+0000 7f0a3a000640 10 client.5970 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a3a000640 1 -- 172.21.15.165:0/2749148382 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x5559c1372420 con 0x5559c1432c00 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a3a000640 10 client.5970 handle_command_reply: tid=0 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 client.5970 shutdown 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 msgr2=0x5559c14667f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c14667f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800dbd0 tx=0x7f0a48013000 comp rx=0 tx=0).stop 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 shutdown_connections 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5559c1432c00 0x5559c1422670 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5559c14443f0 0x5559c146d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5559c14327e0 0x5559c146d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a3a000640 0 client.5970 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 --2- 172.21.15.165:0/2749148382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5559c1407d20 0x5559c14667f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.352+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 >> 172.21.15.165:0/2749148382 conn(0x5559c13cffb0 msgr2=0x5559c1443360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 shutdown_connections 2024-04-25T10:06:41.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5ea3a740 1 -- 172.21.15.165:0/2749148382 wait complete. 2024-04-25T10:06:41.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5ea3a740 20 client.5970 trim_cache size 0 max 16384 2024-04-25T10:06:41.361 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:41.361 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": -2 2024-04-25T10:06:41.361 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a3406b4d0 msgr2=0x7f0a3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a3406b4d0 0x7f0a3406d990 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f0a4800b820 tx=0x7f0a48008040 comp rx=0 tx=0).stop 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 msgr2=0x7f0a5414da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 0x7f0a5414da00 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0a4c00ccf0 tx=0x7f0a4c00a660 comp rx=0 tx=0).stop 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 shutdown_connections 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a3406b4d0 0x7f0a3406d990 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.353+0000 7f0a5a600640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a5415b860 0x7f0a5414da00 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.354+0000 7f0a5a600640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a540d0c60 0x7f0a5414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.354+0000 7f0a5a600640 1 --2- 172.21.15.165:0/568773591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a540cd2f0 0x7f0a5414a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.354+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 >> 172.21.15.165:0/568773591 conn(0x7f0a540b5d00 msgr2=0x7f0a54159e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.354+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 shutdown_connections 2024-04-25T10:06:41.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.354+0000 7f0a5a600640 1 -- 172.21.15.165:0/568773591 wait complete. 2024-04-25T10:06:41.378 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:06:41.378 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:41.378 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:06:41.378 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:41.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.568+0000 7fd5c6600640 1 Processor -- start 2024-04-25T10:06:41.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.568+0000 7fd5c6600640 1 -- start start 2024-04-25T10:06:41.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.569+0000 7fd5c6600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd5c0144750 0x7fd5c006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.569+0000 7fd5c6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c006eb10 0x7fd5c00ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.569+0000 7fd5c6600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c0144b30 con 0x7fd5c00cee60 2024-04-25T10:06:41.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.569+0000 7fd5c6600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c00712d0 con 0x7fd5c0144750 2024-04-25T10:06:41.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.569+0000 7fd5c6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c00715b0 con 0x7fd5c006eb10 2024-04-25T10:06:41.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd5c0144750 0x7fd5c006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd5c0144750 0x7fd5c006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55532/0 (socket says 172.21.15.165:55532) 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 -- 172.21.15.165:0/1711727775 learned_addr learned my addr 172.21.15.165:0/1711727775 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bfe00640 1 --1- 172.21.15.165:0/1711727775 >> v1:172.21.15.165:6789/0 conn(0x7fd5c00cee60 0x7fd5c015f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39974/0 (socket says 172.21.15.165:39974) 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 -- 172.21.15.165:0/1711727775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c006eb10 msgr2=0x7fd5c00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 --2- 172.21.15.165:0/1711727775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c006eb10 0x7fd5c00ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 -- 172.21.15.165:0/1711727775 >> v1:172.21.15.165:6789/0 conn(0x7fd5c00cee60 legacy=0x7fd5c015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.570+0000 7fd5bf400640 1 -- 172.21.15.165:0/1711727775 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c0071890 con 0x7fd5c0144750 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.571+0000 7fd5bf400640 1 --2- 172.21.15.165:0/1711727775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c0144750 0x7fd5c006e5d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb27fadacf2c8aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5bea00640 1 -- 172.21.15.165:0/1711727775 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0037070 con 0x7fd5c0144750 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5bea00640 1 -- 172.21.15.165:0/1711727775 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0032440 con 0x7fd5c0144750 2024-04-25T10:06:41.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5bea00640 1 -- 172.21.15.165:0/1711727775 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0032810 con 0x7fd5c0144750 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 -- 172.21.15.165:0/1711727775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c0144750 msgr2=0x7fd5c006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1711727775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c0144750 0x7fd5c006e5d0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b002a270 comp rx=0 tx=0).stop 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 -- 172.21.15.165:0/1711727775 shutdown_connections 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1711727775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c006eb10 0x7fd5c00ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1711727775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c0144750 0x7fd5c006e5d0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 -- 172.21.15.165:0/1711727775 >> 172.21.15.165:0/1711727775 conn(0x7fd5c006abe0 msgr2=0x7fd5c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.572+0000 7fd5c6600640 1 -- 172.21.15.165:0/1711727775 shutdown_connections 2024-04-25T10:06:41.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.573+0000 7fd5c6600640 1 -- 172.21.15.165:0/1711727775 wait complete. 2024-04-25T10:06:41.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.574+0000 7fd5c6600640 1 Processor -- start 2024-04-25T10:06:41.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 -- start start 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 0x7fd5c00a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c00cee60 0x7fd5c01e0f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c0144750 0x7fd5c00ad540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c0145450 con 0x7fd5c006eb10 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c006f020 con 0x7fd5c0144750 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.575+0000 7fd5c6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c01455d0 con 0x7fd5c00cee60 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 0x7fd5c00a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 0x7fd5c00a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55540/0 (socket says 172.21.15.165:55540) 2024-04-25T10:06:41.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 -- 172.21.15.165:0/1395488762 learned_addr learned my addr 172.21.15.165:0/1395488762 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 -- 172.21.15.165:0/1395488762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c00cee60 msgr2=0x7fd5c01e0f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c00cee60 0x7fd5c01e0f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 -- 172.21.15.165:0/1395488762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c0144750 msgr2=0x7fd5c00ad540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c0144750 0x7fd5c00ad540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 -- 172.21.15.165:0/1395488762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c014f6e0 con 0x7fd5c006eb10 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.576+0000 7fd5bf400640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 0x7fd5c00a9100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.577+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0041020 con 0x7fd5c006eb10 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.577+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b0030970 con 0x7fd5c006eb10 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.577+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b0037040 con 0x7fd5c006eb10 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.577+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5c01e1460 con 0x7fd5c006eb10 2024-04-25T10:06:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.577+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c00a99d0 con 0x7fd5c006eb10 2024-04-25T10:06:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.578+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c0073990 con 0x7fd5c006eb10 2024-04-25T10:06:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.579+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5b0044020 con 0x7fd5c006eb10 2024-04-25T10:06:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.580+0000 7fd5b7e00640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59c06b4d0 0x7fd59c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.580+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5b00aadc0 con 0x7fd5c006eb10 2024-04-25T10:06:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.580+0000 7fd5b6a00640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59c06b4d0 0x7fd59c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.581+0000 7fd5b6a00640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59c06b4d0 0x7fd59c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd5c01f4f90 tx=0x7fd5a8006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:41.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.584+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5b004b030 con 0x7fd5c006eb10 2024-04-25T10:06:41.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.819+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd5c00d0510 con 0x7fd5c006eb10 2024-04-25T10:06:41.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.820+0000 7fd5b7e00640 1 -- 172.21.15.165:0/1395488762 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fd5c014f6e0 con 0x7fd5c006eb10 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:41.827 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59c06b4d0 msgr2=0x7fd59c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59c06b4d0 0x7fd59c06d990 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd5c01f4f90 tx=0x7fd5a8006c90 comp rx=0 tx=0).stop 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 msgr2=0x7fd5c00a9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 0x7fd5c00a9100 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fd5b0009870 tx=0x7fd5b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 shutdown_connections 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59c06b4d0 0x7fd59c06d990 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5c0144750 0x7fd5c00ad540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5c00cee60 0x7fd5c01e0f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 --2- 172.21.15.165:0/1395488762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5c006eb10 0x7fd5c00a9100 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 >> 172.21.15.165:0/1395488762 conn(0x7fd5c006abe0 msgr2=0x7fd5c00b19c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 shutdown_connections 2024-04-25T10:06:41.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:41.823+0000 7fd5c6600640 1 -- 172.21.15.165:0/1395488762 wait complete. 2024-04-25T10:06:41.849 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:42.285 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:42.285 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:42.301 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:42.301 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:06:42.301 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:42.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:42.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.501+0000 7fbe41e00640 1 Processor -- start 2024-04-25T10:06:42.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.502+0000 7fbe41e00640 1 -- start start 2024-04-25T10:06:42.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.503+0000 7fbe41e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 0x7fbe3c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.503+0000 7fbe41e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c072280 con 0x7fbe3c0cc4f0 2024-04-25T10:06:42.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.503+0000 7fbe41e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe3c0714c0 con 0x7fbe3c15b270 2024-04-25T10:06:42.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.503+0000 7fbe41e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c071270 con 0x7fbe3c0ce230 2024-04-25T10:06:42.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbe3c0cc4f0 0x7fbe3c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39986/0 (socket says 172.21.15.165:39986) 2024-04-25T10:06:42.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3aa00640 1 -- 172.21.15.165:0/248271851 learned_addr learned my addr 172.21.15.165:0/248271851 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2301242544 0 0) 0x7fbe3c072280 con 0x7fbe3c0cc4f0 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe32000640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 0x7fbe3c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe20003520 con 0x7fbe3c0cc4f0 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1998042186 0 0) 0x7fbe20003520 con 0x7fbe3c0cc4f0 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 msgr2=0x7fbe3c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 0x7fbe3c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 >> v1:172.21.15.172:6789/0 conn(0x7fbe3c15b270 legacy=0x7fbe3c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0717a0 con 0x7fbe3c0cc4f0 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fbe2c001670 con 0x7fbe3c0cc4f0 2024-04-25T10:06:42.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 >> v1:172.21.15.165:6789/0 conn(0x7fbe3c0cc4f0 legacy=0x7fbe3c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.504+0000 7fbe3a000640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe20003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3a000640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 0x7fbe20004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3a000640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe20004d40 0x7fbe200051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c071270 con 0x7fbe200039e0 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2000baf0 con 0x7fbe20004d40 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe2000df00 con 0x7fbe3c0ce230 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe20003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3b400640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe20004d40 0x7fbe200051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe32000640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 0x7fbe20004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3aa00640 1 -- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 msgr2=0x7fbe20004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0ce230 0x7fbe20004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3aa00640 1 -- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe20004d40 msgr2=0x7fbe200051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe20004d40 0x7fbe200051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.505+0000 7fbe3aa00640 1 -- 172.21.15.165:0/248271851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe20003520 con 0x7fbe200039e0 2024-04-25T10:06:42.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.506+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe20003de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c240 tx=0x7fbe20005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad2bd44d7c5c495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c007d30 con 0x7fbe200039e0 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c005af0 con 0x7fbe200039e0 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe3a000640 1 -- 172.21.15.165:0/248271851 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c007070 con 0x7fbe200039e0 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe41e00640 1 -- 172.21.15.165:0/248271851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 msgr2=0x7fbe20003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe41e00640 1 --2- 172.21.15.165:0/248271851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe20003de0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c240 tx=0x7fbe20005830 comp rx=0 tx=0).stop 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe3b400640 1 -- 172.21.15.165:0/248271851 reap_dead start 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe41e00640 1 -- 172.21.15.165:0/248271851 shutdown_connections 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe41e00640 1 -- 172.21.15.165:0/248271851 >> 172.21.15.165:0/248271851 conn(0x7fbe3c06abe0 msgr2=0x7fbe3c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe41e00640 1 -- 172.21.15.165:0/248271851 shutdown_connections 2024-04-25T10:06:42.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.507+0000 7fbe41e00640 1 -- 172.21.15.165:0/248271851 wait complete. 2024-04-25T10:06:42.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.509+0000 7fbe41e00640 1 Processor -- start 2024-04-25T10:06:42.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.509+0000 7fbe41e00640 1 -- start start 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe41e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe3c14e300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe41e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0c80f0 0x7fbe3c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe41e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3c0ca020 0x7fbe3c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe41e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c06e8a0 con 0x7fbe200039e0 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe41e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c071d60 con 0x7fbe3c0ca020 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe41e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c0cc8d0 con 0x7fbe3c0c80f0 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe3c14e300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe3c14e300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55554/0 (socket says 172.21.15.165:55554) 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.510+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2504748449 learned_addr learned my addr 172.21.15.165:0/2504748449 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe32000640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0c80f0 0x7fbe3c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3b400640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3c0ca020 0x7fbe3c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0c80f0 msgr2=0x7fbe3c0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0c80f0 0x7fbe3c0c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3c0ca020 msgr2=0x7fbe3c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3c0ca020 0x7fbe3c0ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2504748449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe3c0c69c0 con 0x7fbe200039e0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe3c14e300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c02ad80 tx=0x7fbe3c15af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.511+0000 7fbe33e00640 1 -- 172.21.15.165:0/2504748449 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c041020 con 0x7fbe200039e0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.512+0000 7fbe33e00640 1 -- 172.21.15.165:0/2504748449 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c001430 con 0x7fbe200039e0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.512+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe3c0c8c60 con 0x7fbe200039e0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.512+0000 7fbe33e00640 1 -- 172.21.15.165:0/2504748449 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c007070 con 0x7fbe200039e0 2024-04-25T10:06:42.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.512+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe3c0c9220 con 0x7fbe200039e0 2024-04-25T10:06:42.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.514+0000 7fbe33e00640 1 -- 172.21.15.165:0/2504748449 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe2c044020 con 0x7fbe200039e0 2024-04-25T10:06:42.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.515+0000 7fbe33e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1806b4d0 0x7fbe1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.515+0000 7fbe33e00640 1 -- 172.21.15.165:0/2504748449 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe2c0aa310 con 0x7fbe200039e0 2024-04-25T10:06:42.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.516+0000 7fbe32000640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1806b4d0 0x7fbe1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.516+0000 7fbe44c82740 1 Processor -- start 2024-04-25T10:06:42.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.516+0000 7fbe32000640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1806b4d0 0x7fbe1806d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0b46e0 tx=0x7fbe24008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.516+0000 7fbe44c82740 1 -- start start 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce929a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929ae10 0x55dce92970d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe44c82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce91d3c40 con 0x55dce929a4f0 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce917d050 con 0x55dce92abf10 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dce91cfce0 con 0x55dce929ae10 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.517+0000 7fbe3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55556/0 (socket says 172.21.15.165:55556) 2024-04-25T10:06:42.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 -- 172.21.15.165:0/4190436004 learned_addr learned my addr 172.21.15.165:0/4190436004 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 -- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929ae10 msgr2=0x55dce92970d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe32000640 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce929a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929ae10 0x55dce92970d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 -- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 msgr2=0x55dce929a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce929a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 -- 172.21.15.165:0/4190436004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dce8fb4b00 con 0x55dce929a4f0 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92b5a80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c2a0 tx=0x7fbe2c074cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bddb8a8a3964a80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.518+0000 7fbe2b400640 1 -- 172.21.15.165:0/4190436004 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c081090 con 0x55dce929a4f0 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe2b400640 1 -- 172.21.15.165:0/4190436004 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c0aa0c0 con 0x55dce929a4f0 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe2b400640 1 -- 172.21.15.165:0/4190436004 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c076bf0 con 0x55dce929a4f0 2024-04-25T10:06:42.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 -- 172.21.15.165:0/4190436004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 msgr2=0x55dce92b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92b5a80 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c00c2a0 tx=0x7fbe2c074cc0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 -- 172.21.15.165:0/4190436004 shutdown_connections 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929ae10 0x55dce92970d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce929a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 --2- 172.21.15.165:0/4190436004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92b5a80 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 -- 172.21.15.165:0/4190436004 >> 172.21.15.165:0/4190436004 conn(0x55dce92375e0 msgr2=0x55dce92b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 -- 172.21.15.165:0/4190436004 shutdown_connections 2024-04-25T10:06:42.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.519+0000 7fbe44c82740 1 -- 172.21.15.165:0/4190436004 wait complete. 2024-04-25T10:06:42.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.520+0000 7fbe44c82740 1 -- start start 2024-04-25T10:06:42.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.520+0000 7fbe44c82740 1 Processor -- start 2024-04-25T10:06:42.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.521+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce9284ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.521+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce929b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.521+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce9292090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.521+0000 7fbe44c82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce917d050 con 0x55dce929ae10 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.521+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce91725d0 con 0x55dce92abf10 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.521+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dce91d3c40 con 0x55dce929a4f0 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe44c82740 10 client.0 _ll_register_callbacks cb 0x55dce91cba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce929b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce929b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55558/0 (socket says 172.21.15.165:55558) 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 -- 172.21.15.165:0/1874374580 learned_addr learned my addr 172.21.15.165:0/1874374580 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce9284ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 -- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 msgr2=0x55dce9284ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce9284ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 -- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 msgr2=0x55dce9292090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce9292090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.522+0000 7fbe32000640 1 -- 172.21.15.165:0/1874374580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dce8ea83f0 con 0x55dce929ae10 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe32000640 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce929b280 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbe240154a0 tx=0x7fbe240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b52675430b28a04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:42.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe1fe00640 1 -- 172.21.15.165:0/1874374580 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2400f040 con 0x55dce929ae10 2024-04-25T10:06:42.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe1fe00640 1 -- 172.21.15.165:0/1874374580 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe24016830 con 0x55dce929ae10 2024-04-25T10:06:42.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe2c0aa0c0 con 0x55dce929ae10 2024-04-25T10:06:42.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe1fe00640 1 -- 172.21.15.165:0/1874374580 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe2402f050 con 0x55dce929ae10 2024-04-25T10:06:42.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe44c82740 10 client.5991 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:42.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55dce9015290 con 0x55dce929ae10 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe1fe00640 1 -- 172.21.15.165:0/1874374580 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c0aa0c0 con 0x55dce929ae10 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.523+0000 7fbe1fe00640 1 -- 172.21.15.165:0/1874374580 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbe24032020 con 0x55dce929ae10 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe44c82740 10 client.5991 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe44c82740 10 client.5991 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe44c82740 20 client.5991 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe44c82740 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce929ce90 0x55dce929d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe44c82740 4 client.5991 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dce91e1180 con 0x55dce929ce90 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.524+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce929ce90 0x55dce929d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.525+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce929ce90 0x55dce929d2d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195326665eafe8c2 server_cookie=c9d6d6f5c81d22fd in_seq=0 out_seq=0 2024-04-25T10:06:42.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.525+0000 7fbe1fe00640 10 client.5991 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:42.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.527+0000 7fbe1fe00640 1 -- 172.21.15.165:0/1874374580 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dce91e1180 con 0x55dce929ce90 2024-04-25T10:06:42.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.527+0000 7fbe1fe00640 10 client.5991 handle_command_reply: tid=0 2024-04-25T10:06:42.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.527+0000 7fbe44c82740 1 client.5991 shutdown 2024-04-25T10:06:42.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.527+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 msgr2=0x55dce929b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.527+0000 7fbe44c82740 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce929b280 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbe240154a0 tx=0x7fbe240046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 shutdown_connections 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce929ce90 0x55dce929d2d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce9292090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce929b280 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 --2- 172.21.15.165:0/1874374580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce9284ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 >> 172.21.15.165:0/1874374580 conn(0x55dce92375e0 msgr2=0x55dce92695e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe1fe00640 0 client.5991 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 shutdown_connections 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 1 -- 172.21.15.165:0/1874374580 wait complete. 2024-04-25T10:06:42.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.528+0000 7fbe44c82740 20 client.5991 trim_cache size 0 max 16384 2024-04-25T10:06:42.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.545+0000 7fbe44c82740 1 Processor -- start 2024-04-25T10:06:42.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.545+0000 7fbe44c82740 1 -- start start 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92695e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce929ae10 0x55dce9297500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce92abf10 0x55dce92918a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe44c82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1fc820 con 0x55dce929a4f0 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1fba50 con 0x55dce929ae10 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c142e20 con 0x55dce92abf10 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92695e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92695e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55560/0 (socket says 172.21.15.165:55560) 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2508828093 learned_addr learned my addr 172.21.15.165:0/2508828093 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2508828093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce92abf10 msgr2=0x55dce92918a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce92abf10 0x55dce92918a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2508828093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce929ae10 msgr2=0x55dce9297500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce929ae10 0x55dce9297500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.546+0000 7fbe3aa00640 1 -- 172.21.15.165:0/2508828093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dce8e814b0 con 0x55dce929a4f0 2024-04-25T10:06:42.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92695e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c074d50 tx=0x7fbe2c08d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22de7b53eda430c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe2aa00640 1 -- 172.21.15.165:0/2508828093 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c0a5070 con 0x55dce929a4f0 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe2aa00640 1 -- 172.21.15.165:0/2508828093 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c0aa0c0 con 0x55dce929a4f0 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe2aa00640 1 -- 172.21.15.165:0/2508828093 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe2c076bf0 con 0x55dce929a4f0 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 -- 172.21.15.165:0/2508828093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 msgr2=0x55dce92695e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92695e0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c074d50 tx=0x7fbe2c08d000 comp rx=0 tx=0).stop 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 -- 172.21.15.165:0/2508828093 shutdown_connections 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce92abf10 0x55dce92918a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce929ae10 0x55dce9297500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2508828093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929a4f0 0x55dce92695e0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.547+0000 7fbe44c82740 1 -- 172.21.15.165:0/2508828093 >> 172.21.15.165:0/2508828093 conn(0x55dce92375e0 msgr2=0x55dce92d7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.548+0000 7fbe44c82740 1 -- 172.21.15.165:0/2508828093 shutdown_connections 2024-04-25T10:06:42.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.548+0000 7fbe44c82740 1 -- 172.21.15.165:0/2508828093 wait complete. 2024-04-25T10:06:42.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.548+0000 7fbe44c82740 1 -- start start 2024-04-25T10:06:42.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.548+0000 7fbe44c82740 1 Processor -- start 2024-04-25T10:06:42.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.549+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce92aac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.549+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce92b4330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.549+0000 7fbe44c82740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce92b4870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.549+0000 7fbe44c82740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dce9199210 con 0x55dce929ae10 2024-04-25T10:06:42.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.549+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe3c1fb450 con 0x55dce92abf10 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.549+0000 7fbe44c82740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dce926cfe0 con 0x55dce929a4f0 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe44c82740 10 client.0 _ll_register_callbacks cb 0x7fbe2c074c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce92b4330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce92b4330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55572/0 (socket says 172.21.15.165:55572) 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 -- 172.21.15.165:0/2692898390 learned_addr learned my addr 172.21.15.165:0/2692898390 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 -- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 msgr2=0x55dce92aac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce92aac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 -- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 msgr2=0x55dce92b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce92b4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 -- 172.21.15.165:0/2692898390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dce9328ed0 con 0x55dce929ae10 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe32000640 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce92b4330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbe240154a0 tx=0x7fbe24004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d974e6f7b0549d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.550+0000 7fbe1fe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe1fe00640 1 -- 172.21.15.165:0/2692898390 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe24039020 con 0x55dce929ae10 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe1fe00640 1 -- 172.21.15.165:0/2692898390 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbe24022820 con 0x55dce929ae10 2024-04-25T10:06:42.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe2c0aa0c0 con 0x55dce929ae10 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe1fe00640 1 -- 172.21.15.165:0/2692898390 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe24043050 con 0x55dce929ae10 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 10 client.5997 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55dce92d2440 con 0x55dce929ae10 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe1fe00640 1 -- 172.21.15.165:0/2692898390 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe2c0aa0c0 con 0x55dce929ae10 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe1fe00640 1 -- 172.21.15.165:0/2692898390 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbe24046020 con 0x55dce929ae10 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 10 client.5997 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 10 client.5997 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 20 client.5997 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.551+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce92ab1d0 0x55dce92b5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.552+0000 7fbe44c82740 4 client.5997 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.552+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55dce92b8520 con 0x55dce92ab1d0 2024-04-25T10:06:42.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.552+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce92ab1d0 0x55dce92b5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.553+0000 7fbe3aa00640 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce92ab1d0 0x55dce92b5300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0a2843198a23cc5 server_cookie=e520cb42838185e3 in_seq=0 out_seq=0 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.553+0000 7fbe1fe00640 10 client.5997 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.554+0000 7fbe1fe00640 1 -- 172.21.15.165:0/2692898390 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dce92b8520 con 0x55dce92ab1d0 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.554+0000 7fbe1fe00640 10 client.5997 handle_command_reply: tid=0 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.554+0000 7fbe44c82740 1 client.5997 shutdown 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 msgr2=0x55dce92b4330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce92b4330 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbe240154a0 tx=0x7fbe24004730 comp rx=0 tx=0).stop 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 shutdown_connections 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55dce92ab1d0 0x55dce92b5300 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dce92abf10 0x55dce92b4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dce929ae10 0x55dce92b4330 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 --2- 172.21.15.165:0/2692898390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dce929a4f0 0x55dce92aac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe1fe00640 0 client.5997 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:42.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 >> 172.21.15.165:0/2692898390 conn(0x55dce92375e0 msgr2=0x55dce9269e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 shutdown_connections 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 1 -- 172.21.15.165:0/2692898390 wait complete. 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe44c82740 20 client.5997 trim_cache size 0 max 16384 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "489c5330-c907-4bcb-8e36-d6fdc4c8abf5", 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:42.563 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:42.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1806b4d0 msgr2=0x7fbe1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.555+0000 7fbe41e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1806b4d0 0x7fbe1806d990 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c0b46e0 tx=0x7fbe24008040 comp rx=0 tx=0).stop 2024-04-25T10:06:42.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 msgr2=0x7fbe3c14e300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe3c14e300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c02ad80 tx=0x7fbe3c15af60 comp rx=0 tx=0).stop 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 shutdown_connections 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe1806b4d0 0x7fbe1806d990 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe3c0ca020 0x7fbe3c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe3c0c80f0 0x7fbe3c0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 --2- 172.21.15.165:0/2504748449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe200039e0 0x7fbe3c14e300 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 >> 172.21.15.165:0/2504748449 conn(0x7fbe3c06abe0 msgr2=0x7fbe3c15d5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 shutdown_connections 2024-04-25T10:06:42.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.556+0000 7fbe41e00640 1 -- 172.21.15.165:0/2504748449 wait complete. 2024-04-25T10:06:42.579 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': '489c5330-c907-4bcb-8e36-d6fdc4c8abf5', 'mode': 'asynchronous'}' 2024-04-25T10:06:42.579 INFO:tasks.cephfs.test_scrub_checks:First run: flushing /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:42.579 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:06:42.579 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:42.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.768+0000 7fefb9800640 1 Processor -- start 2024-04-25T10:06:42.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.768+0000 7fefb9800640 1 -- start start 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.769+0000 7fefb9800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefb40ce920 0x7fefb40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.769+0000 7fefb9800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fefb406d290 0x7fefb406d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.769+0000 7fefb9800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb4144b30 con 0x7fefb4144750 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.769+0000 7fefb9800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40712d0 con 0x7fefb40ce920 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.769+0000 7fefb9800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb40715b0 con 0x7fefb406d290 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefb40ce920 0x7fefb40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefb40ce920 0x7fefb40cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55588/0 (socket says 172.21.15.165:55588) 2024-04-25T10:06:42.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 -- 172.21.15.165:0/3043269699 learned_addr learned my addr 172.21.15.165:0/3043269699 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 -- 172.21.15.165:0/3043269699 >> v2:172.21.15.172:3300/0 conn(0x7fefb406d290 msgr2=0x7fefb406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 --2- 172.21.15.165:0/3043269699 >> v2:172.21.15.172:3300/0 conn(0x7fefb406d290 0x7fefb406d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 -- 172.21.15.165:0/3043269699 >> v1:172.21.15.172:6789/0 conn(0x7fefb4144750 legacy=0x7fefb406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:42.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.770+0000 7fefaa000640 1 -- 172.21.15.165:0/3043269699 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb4071890 con 0x7fefb40ce920 2024-04-25T10:06:42.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.771+0000 7fefaa000640 1 --2- 172.21.15.165:0/3043269699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb40cedb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fefb406f720 tx=0x7fef9c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fce976b69353072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb2000640 1 -- 172.21.15.165:0/3043269699 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c037070 con 0x7fefb40ce920 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb2000640 1 -- 172.21.15.165:0/3043269699 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c032440 con 0x7fefb40ce920 2024-04-25T10:06:42.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb2000640 1 -- 172.21.15.165:0/3043269699 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c032810 con 0x7fefb40ce920 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb9800640 1 -- 172.21.15.165:0/3043269699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 msgr2=0x7fefb40cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb9800640 1 --2- 172.21.15.165:0/3043269699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb40cedb0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fefb406f720 tx=0x7fef9c02a230 comp rx=0 tx=0).stop 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb9800640 1 -- 172.21.15.165:0/3043269699 shutdown_connections 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb9800640 1 --2- 172.21.15.165:0/3043269699 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb40cedb0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb9800640 1 --2- 172.21.15.165:0/3043269699 >> v2:172.21.15.172:3300/0 conn(0x7fefb406d290 0x7fefb406d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.772+0000 7fefb9800640 1 -- 172.21.15.165:0/3043269699 >> 172.21.15.165:0/3043269699 conn(0x7fefb406abe0 msgr2=0x7fefb406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.773+0000 7fefb9800640 1 -- 172.21.15.165:0/3043269699 shutdown_connections 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.773+0000 7fefb9800640 1 -- 172.21.15.165:0/3043269699 wait complete. 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.774+0000 7fefb9800640 1 Processor -- start 2024-04-25T10:06:42.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.774+0000 7fefb9800640 1 -- start start 2024-04-25T10:06:42.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.775+0000 7fefb9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefb406d290 0x7fefb41f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.775+0000 7fefb9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb413db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.775+0000 7fefb9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb4144750 0x7fefb40b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.775+0000 7fefb9800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb406f020 con 0x7fefb40ce920 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.775+0000 7fefb9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb406ece0 con 0x7fefb4144750 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.775+0000 7fefb9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb4145450 con 0x7fefb406d290 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb413db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb413db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55590/0 (socket says 172.21.15.165:55590) 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 -- 172.21.15.165:0/225908743 learned_addr learned my addr 172.21.15.165:0/225908743 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefb2a00640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefb406d290 0x7fefb41f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefb3400640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb4144750 0x7fefb40b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 -- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefb406d290 msgr2=0x7fefb41f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefb406d290 0x7fefb41f1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 -- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb4144750 msgr2=0x7fefb40b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb4144750 0x7fefb40b1200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefaa000640 1 -- 172.21.15.165:0/225908743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb415c4c0 con 0x7fefb40ce920 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.776+0000 7fefb3400640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb4144750 0x7fefb40b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.777+0000 7fefaa000640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb413db10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000c00 tx=0x7fef9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.777+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c041070 con 0x7fefb40ce920 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.777+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef9c03c870 con 0x7fefb40ce920 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.777+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef9c037040 con 0x7fefb40ce920 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.777+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb415c7a0 con 0x7fefb40ce920 2024-04-25T10:06:42.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.777+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb415cd60 con 0x7fefb40ce920 2024-04-25T10:06:42.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.778+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb4073990 con 0x7fefb40ce920 2024-04-25T10:06:42.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.779+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fef9c044070 con 0x7fefb40ce920 2024-04-25T10:06:42.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.780+0000 7fefab400640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef9006b4d0 0x7fef9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:42.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.780+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fef9c0aaf40 con 0x7fefb40ce920 2024-04-25T10:06:42.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.780+0000 7fefb2a00640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef9006b4d0 0x7fef9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:42.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.781+0000 7fefb2a00640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef9006b4d0 0x7fef9006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefa4007fd0 tx=0x7fefa4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:42.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:42.784+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fef9c079d40 con 0x7fefb40ce920 2024-04-25T10:06:43.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.018+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fefb415b3e0 con 0x7fefb40ce920 2024-04-25T10:06:43.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.019+0000 7fefab400640 1 -- 172.21.15.165:0/225908743 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fef9c079640 con 0x7fefb40ce920 2024-04-25T10:06:43.026 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:43.026 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:43.026 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:43.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef9006b4d0 msgr2=0x7fef9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef9006b4d0 0x7fef9006d990 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fefa4007fd0 tx=0x7fefa4009040 comp rx=0 tx=0).stop 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 msgr2=0x7fefb413db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb413db10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000c00 tx=0x7fef9c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 shutdown_connections 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef9006b4d0 0x7fef9006d990 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefb4144750 0x7fefb40b1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefb40ce920 0x7fefb413db10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 --2- 172.21.15.165:0/225908743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefb406d290 0x7fefb41f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 >> 172.21.15.165:0/225908743 conn(0x7fefb406abe0 msgr2=0x7fefb4157cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 shutdown_connections 2024-04-25T10:06:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.022+0000 7fefb9800640 1 -- 172.21.15.165:0/225908743 wait complete. 2024-04-25T10:06:43.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:43.363 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:43.364 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:43.381 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:43.382 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:06:43.382 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:43.382 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.584+0000 7fd347800640 1 Processor -- start 2024-04-25T10:06:43.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.584+0000 7fd347800640 1 -- start start 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.586+0000 7fd347800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd3401446d0 0x7fd3400b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.586+0000 7fd347800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd340144d50 0x7fd3400b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.586+0000 7fd347800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3400cccc0 con 0x7fd3401452a0 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.586+0000 7fd347800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3400b2b20 con 0x7fd3401446d0 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.586+0000 7fd347800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd34006d400 con 0x7fd340144d50 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd3401446d0 0x7fd3400b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd3401446d0 0x7fd3400b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55600/0 (socket says 172.21.15.165:55600) 2024-04-25T10:06:43.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 -- 172.21.15.165:0/3755442795 learned_addr learned my addr 172.21.15.165:0/3755442795 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3755442795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd340144d50 0x7fd3400b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 -- 172.21.15.165:0/3755442795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd340144d50 msgr2=0x7fd3400b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 --2- 172.21.15.165:0/3755442795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd340144d50 0x7fd3400b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 -- 172.21.15.165:0/3755442795 >> v1:172.21.15.172:6789/0 conn(0x7fd3401452a0 legacy=0x7fd3400cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.587+0000 7fd344e00640 1 -- 172.21.15.165:0/3755442795 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3400cc140 con 0x7fd3401446d0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.588+0000 7fd344e00640 1 --2- 172.21.15.165:0/3755442795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3401446d0 0x7fd3400b3f20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd334009870 tx=0x7fd33402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4479bd4d17106c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.589+0000 7fd33f400640 1 -- 172.21.15.165:0/3755442795 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334037070 con 0x7fd3401446d0 2024-04-25T10:06:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.589+0000 7fd33f400640 1 -- 172.21.15.165:0/3755442795 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd334032440 con 0x7fd3401446d0 2024-04-25T10:06:43.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.589+0000 7fd33f400640 1 -- 172.21.15.165:0/3755442795 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334032810 con 0x7fd3401446d0 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 -- 172.21.15.165:0/3755442795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3401446d0 msgr2=0x7fd3400b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 --2- 172.21.15.165:0/3755442795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3401446d0 0x7fd3400b3f20 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fd334009870 tx=0x7fd33402a210 comp rx=0 tx=0).stop 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 -- 172.21.15.165:0/3755442795 shutdown_connections 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 --2- 172.21.15.165:0/3755442795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd340144d50 0x7fd3400b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 --2- 172.21.15.165:0/3755442795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3401446d0 0x7fd3400b3f20 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 -- 172.21.15.165:0/3755442795 >> 172.21.15.165:0/3755442795 conn(0x7fd34006abe0 msgr2=0x7fd34006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 -- 172.21.15.165:0/3755442795 shutdown_connections 2024-04-25T10:06:43.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.590+0000 7fd347800640 1 -- 172.21.15.165:0/3755442795 wait complete. 2024-04-25T10:06:43.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.592+0000 7fd347800640 1 Processor -- start 2024-04-25T10:06:43.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.592+0000 7fd347800640 1 -- start start 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd347800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3401446d0 0x7fd34015b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd347800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 0x7fd34013b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd347800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3401452a0 0x7fd3400cf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd347800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400cc700 con 0x7fd340144d50 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd347800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400b49a0 con 0x7fd3401446d0 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd347800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3400cc9e0 con 0x7fd3401452a0 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 0x7fd34013b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 0x7fd34013b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55606/0 (socket says 172.21.15.165:55606) 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 -- 172.21.15.165:0/3794413104 learned_addr learned my addr 172.21.15.165:0/3794413104 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd344e00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3401446d0 0x7fd34015b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 -- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3401452a0 msgr2=0x7fd3400cf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd345800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3401452a0 0x7fd3400cf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3401452a0 0x7fd3400cf490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 -- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3401446d0 msgr2=0x7fd34015b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3401446d0 0x7fd34015b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd33fe00640 1 -- 172.21.15.165:0/3794413104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd340139590 con 0x7fd340144d50 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.593+0000 7fd344e00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3401446d0 0x7fd34015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.594+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 0x7fd34013b020 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00c820 tx=0x7fd32c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.594+0000 7fd33cc00640 1 -- 172.21.15.165:0/3794413104 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c015070 con 0x7fd340144d50 2024-04-25T10:06:43.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.594+0000 7fd33cc00640 1 -- 172.21.15.165:0/3794413104 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c004240 con 0x7fd340144d50 2024-04-25T10:06:43.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.594+0000 7fd33cc00640 1 -- 172.21.15.165:0/3794413104 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c00ce10 con 0x7fd340144d50 2024-04-25T10:06:43.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.594+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd340139870 con 0x7fd340144d50 2024-04-25T10:06:43.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.594+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd340139e30 con 0x7fd340144d50 2024-04-25T10:06:43.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.596+0000 7fd33cc00640 1 -- 172.21.15.165:0/3794413104 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd32c007830 con 0x7fd340144d50 2024-04-25T10:06:43.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.597+0000 7fd33cc00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd32006b4d0 0x7fd32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.597+0000 7fd33cc00640 1 -- 172.21.15.165:0/3794413104 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd32c08a200 con 0x7fd340144d50 2024-04-25T10:06:43.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.598+0000 7fd344e00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd32006b4d0 0x7fd32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.598+0000 7fd344e00640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd32006b4d0 0x7fd32006d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd334034000 tx=0x7fd33402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.598+0000 7fd34a640740 1 Processor -- start 2024-04-25T10:06:43.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.598+0000 7fd34a640740 1 -- start start 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.599+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 0x55fca749a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.599+0000 7fd34a640740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca74c9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.599+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca060 0x55fca749ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.599+0000 7fd34a640740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fca7402cf0 con 0x55fca74ca7a0 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.599+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fca73ac1d0 con 0x55fca74ca060 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.599+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fca73fed90 con 0x55fca74e7680 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca74c9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca74c9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55612/0 (socket says 172.21.15.165:55612) 2024-04-25T10:06:43.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 -- 172.21.15.165:0/3245934472 learned_addr learned my addr 172.21.15.165:0/3245934472 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 -- 172.21.15.165:0/3245934472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 msgr2=0x55fca749a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 0x55fca749a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 -- 172.21.15.165:0/3245934472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca060 msgr2=0x55fca749ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca060 0x55fca749ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 -- 172.21.15.165:0/3245934472 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fca71e3f00 con 0x55fca74ca7a0 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd33fe00640 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca74c9b20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd32c006fd0 tx=0x7fd32c0589a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757855338863bc20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd331600640 1 -- 172.21.15.165:0/3245934472 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c01e040 con 0x55fca74ca7a0 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.600+0000 7fd331600640 1 -- 172.21.15.165:0/3245934472 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c056b60 con 0x55fca74ca7a0 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd331600640 1 -- 172.21.15.165:0/3245934472 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c070170 con 0x55fca74ca7a0 2024-04-25T10:06:43.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 -- 172.21.15.165:0/3245934472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 msgr2=0x55fca74c9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca74c9b20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd32c006fd0 tx=0x7fd32c0589a0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 -- 172.21.15.165:0/3245934472 shutdown_connections 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca060 0x55fca749ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca74c9b20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 --2- 172.21.15.165:0/3245934472 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 0x55fca749a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 -- 172.21.15.165:0/3245934472 >> 172.21.15.165:0/3245934472 conn(0x55fca749c280 msgr2=0x55fca74e4b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 -- 172.21.15.165:0/3245934472 shutdown_connections 2024-04-25T10:06:43.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.601+0000 7fd34a640740 1 -- 172.21.15.165:0/3245934472 wait complete. 2024-04-25T10:06:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.602+0000 7fd34a640740 1 -- start start 2024-04-25T10:06:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.602+0000 7fd34a640740 1 Processor -- start 2024-04-25T10:06:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca7a0 0x55fca74c44b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 0x55fca74c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74c9090 0x55fca74f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fca749e430 con 0x55fca74e7680 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fca73a1b50 con 0x55fca74ca7a0 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fca749d820 con 0x55fca74c9090 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.603+0000 7fd34a640740 10 client.0 _ll_register_callbacks cb 0x55fca73c3370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 0x55fca74c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 0x55fca74c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55620/0 (socket says 172.21.15.165:55620) 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 -- 172.21.15.165:0/2733121970 learned_addr learned my addr 172.21.15.165:0/2733121970 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd344e00640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca7a0 0x55fca74c44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd345800640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74c9090 0x55fca74f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 -- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74c9090 msgr2=0x55fca74f4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74c9090 0x55fca74f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 -- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca7a0 msgr2=0x55fca74c44b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca7a0 0x55fca74c44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 -- 172.21.15.165:0/2733121970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fca70d77f0 con 0x55fca74e7680 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd33fe00640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 0x55fca74c8b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd32c058560 tx=0x7fd32c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8f2b3ca28481b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd326000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd326000640 1 -- 172.21.15.165:0/2733121970 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c075060 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.604+0000 7fd326000640 1 -- 172.21.15.165:0/2733121970 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd32c0882c0 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd32c056b60 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd326000640 1 -- 172.21.15.165:0/2733121970 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32c01e050 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 10 client.6015 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55fca7244690 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd326000640 1 -- 172.21.15.165:0/2733121970 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32c056b60 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd326000640 1 -- 172.21.15.165:0/2733121970 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd32c059b90 con 0x55fca74e7680 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 10 client.6015 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 10 client.6015 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 20 client.6015 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e9b40 0x55fca74e9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 4 client.6015 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.605+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fca7410230 con 0x55fca74e9b40 2024-04-25T10:06:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.606+0000 7fd344e00640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e9b40 0x55fca74e9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.607+0000 7fd344e00640 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e9b40 0x55fca74e9f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d991c7527d31cf3 server_cookie=fefa12c2abe70d0c in_seq=0 out_seq=0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.607+0000 7fd326000640 10 client.6015 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.608+0000 7fd326000640 1 -- 172.21.15.165:0/2733121970 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55fca7410230 con 0x55fca74e9b40 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.608+0000 7fd326000640 10 client.6015 handle_command_reply: tid=0 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 client.6015 shutdown 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 msgr2=0x55fca74c8b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 0x55fca74c8b50 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd32c058560 tx=0x7fd32c000bc0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 shutdown_connections 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e9b40 0x55fca74e9f50 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74c9090 0x55fca74f4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74e7680 0x55fca74c8b50 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd326000640 0 client.6015 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 --2- 172.21.15.165:0/2733121970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74ca7a0 0x55fca74c44b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 >> 172.21.15.165:0/2733121970 conn(0x55fca74ca060 msgr2=0x55fca74e7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 shutdown_connections 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 1 -- 172.21.15.165:0/2733121970 wait complete. 2024-04-25T10:06:43.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.609+0000 7fd34a640740 20 client.6015 trim_cache size 0 max 16384 2024-04-25T10:06:43.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.626+0000 7fd34a640740 1 Processor -- start 2024-04-25T10:06:43.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.626+0000 7fd34a640740 1 -- start start 2024-04-25T10:06:43.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd34a640740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 0x55fca74c9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74ca7a0 0x55fca754eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74e7680 0x55fca7499d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd34a640740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401fc180 con 0x55fca74c9090 2024-04-25T10:06:43.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401fb360 con 0x55fca74e7680 2024-04-25T10:06:43.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401f01d0 con 0x55fca74ca7a0 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd344e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 0x55fca74c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd344e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 0x55fca74c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55626/0 (socket says 172.21.15.165:55626) 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd344e00640 1 -- 172.21.15.165:0/443545591 learned_addr learned my addr 172.21.15.165:0/443545591 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd345800640 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74e7680 0x55fca7499d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd33fe00640 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74ca7a0 0x55fca754eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.627+0000 7fd344e00640 1 -- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74ca7a0 msgr2=0x55fca754eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd344e00640 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74ca7a0 0x55fca754eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd344e00640 1 -- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74e7680 msgr2=0x55fca7499d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd344e00640 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74e7680 0x55fca7499d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd344e00640 1 -- 172.21.15.165:0/443545591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fca70b08b0 con 0x55fca74c9090 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd33fe00640 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74ca7a0 0x55fca754eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd344e00640 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 0x55fca74c9470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd33402a250 tx=0x7fd33402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301ef0433de47106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd330c00640 1 -- 172.21.15.165:0/443545591 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334037040 con 0x55fca74c9090 2024-04-25T10:06:43.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd330c00640 1 -- 172.21.15.165:0/443545591 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33404a710 con 0x55fca74c9090 2024-04-25T10:06:43.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.628+0000 7fd330c00640 1 -- 172.21.15.165:0/443545591 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33403c680 con 0x55fca74c9090 2024-04-25T10:06:43.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 -- 172.21.15.165:0/443545591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 msgr2=0x55fca74c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 0x55fca74c9470 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fd33402a250 tx=0x7fd33402c000 comp rx=0 tx=0).stop 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 -- 172.21.15.165:0/443545591 shutdown_connections 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74e7680 0x55fca7499d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74ca7a0 0x55fca754eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 --2- 172.21.15.165:0/443545591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74c9090 0x55fca74c9470 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 -- 172.21.15.165:0/443545591 >> 172.21.15.165:0/443545591 conn(0x55fca74ca060 msgr2=0x55fca74caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 -- 172.21.15.165:0/443545591 shutdown_connections 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.629+0000 7fd34a640740 1 -- 172.21.15.165:0/443545591 wait complete. 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.630+0000 7fd34a640740 1 -- start start 2024-04-25T10:06:43.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.630+0000 7fd34a640740 1 Processor -- start 2024-04-25T10:06:43.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74c9090 0x55fca7506250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca7506790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 0x55fca7501130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fca74c99b0 con 0x55fca74ca7a0 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3401f07d0 con 0x55fca74c9090 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fca74cd6f0 con 0x55fca74e7680 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd34a640740 10 client.0 _ll_register_callbacks cb 0x7fd32c058630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca7506790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd33fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca7506790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55642/0 (socket says 172.21.15.165:55642) 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd33fe00640 1 -- 172.21.15.165:0/800737043 learned_addr learned my addr 172.21.15.165:0/800737043 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.631+0000 7fd33fe00640 1 -- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 msgr2=0x55fca7501130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd33fe00640 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 0x55fca7501130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd344e00640 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74c9090 0x55fca7506250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd33fe00640 1 -- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74c9090 msgr2=0x55fca7506250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd33fe00640 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74c9090 0x55fca7506250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd33fe00640 1 -- 172.21.15.165:0/800737043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fca754d190 con 0x55fca74ca7a0 2024-04-25T10:06:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd33fe00640 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca7506790 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00c7f0 tx=0x7fd32c0586e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3f815a378cbb18e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd326000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd326000640 1 -- 172.21.15.165:0/800737043 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32c075070 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd326000640 1 -- 172.21.15.165:0/800737043 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd32c059b90 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd33404a710 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd326000640 1 -- 172.21.15.165:0/800737043 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd32c068050 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd34a640740 10 client.6021 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.632+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55fca754d190 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd326000640 1 -- 172.21.15.165:0/800737043 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33404a710 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd326000640 1 -- 172.21.15.165:0/800737043 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd32c06b020 con 0x55fca74ca7a0 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd34a640740 10 client.6021 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd34a640740 10 client.6021 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd34a640740 20 client.6021 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd34a640740 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e8160 0x55fca7501670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd34a640740 4 client.6021 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:06:43.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55fca740a580 con 0x55fca74e8160 2024-04-25T10:06:43.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.633+0000 7fd344e00640 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e8160 0x55fca7501670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.634+0000 7fd344e00640 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e8160 0x55fca7501670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f89f9b068a7af48a server_cookie=449591e15ce39afc in_seq=0 out_seq=0 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.634+0000 7fd326000640 10 client.6021 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.635+0000 7fd326000640 1 -- 172.21.15.165:0/800737043 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fca740a580 con 0x55fca74e8160 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.635+0000 7fd326000640 10 client.6021 handle_command_reply: tid=0 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 client.6021 shutdown 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 msgr2=0x55fca7506790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca7506790 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00c7f0 tx=0x7fd32c0586e0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 shutdown_connections 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55fca74e8160 0x55fca7501670 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55fca74e7680 0x55fca7501130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55fca74ca7a0 0x55fca7506790 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 --2- 172.21.15.165:0/800737043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55fca74c9090 0x55fca7506250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 >> 172.21.15.165:0/800737043 conn(0x55fca74ca060 msgr2=0x55fca74c6b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd326000640 0 client.6021 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 shutdown_connections 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 1 -- 172.21.15.165:0/800737043 wait complete. 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.636+0000 7fd34a640740 20 client.6021 trim_cache size 0 max 16384 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:43.644 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "8f1eb115-1ccc-4ca6-ac37-65cd989778e4", 2024-04-25T10:06:43.645 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:43.645 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:43.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd32006b4d0 msgr2=0x7fd32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd32006b4d0 0x7fd32006d990 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fd334034000 tx=0x7fd33402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 msgr2=0x7fd34013b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 0x7fd34013b020 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd32c00c820 tx=0x7fd32c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 shutdown_connections 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd32006b4d0 0x7fd32006d990 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3401452a0 0x7fd3400cf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd340144d50 0x7fd34013b020 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 --2- 172.21.15.165:0/3794413104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3401446d0 0x7fd34015b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 >> 172.21.15.165:0/3794413104 conn(0x7fd34006abe0 msgr2=0x7fd34006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 shutdown_connections 2024-04-25T10:06:43.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.637+0000 7fd347800640 1 -- 172.21.15.165:0/3794413104 wait complete. 2024-04-25T10:06:43.662 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': '8f1eb115-1ccc-4ca6-ac37-65cd989778e4', 'mode': 'asynchronous'}' 2024-04-25T10:06:43.663 INFO:tasks.cephfs.test_scrub_checks:First run: flushing base dir / 2024-04-25T10:06:43.663 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:06:43.663 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:43.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.852+0000 7f4b7b800640 1 Processor -- start 2024-04-25T10:06:43.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.853+0000 7f4b7b800640 1 -- start start 2024-04-25T10:06:43.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.854+0000 7f4b7b800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4b740ce920 0x7f4b740cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.854+0000 7f4b7b800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b74144b30 con 0x7f4b74144750 2024-04-25T10:06:43.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.854+0000 7f4b7b800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b740712d0 con 0x7f4b7406d290 2024-04-25T10:06:43.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.854+0000 7f4b7b800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b740715b0 con 0x7f4b740ce920 2024-04-25T10:06:43.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.855+0000 7f4b78e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4b74144750 0x7f4b7406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40000/0 (socket says 172.21.15.165:40000) 2024-04-25T10:06:43.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.855+0000 7f4b78e00640 1 -- 172.21.15.165:0/2266936724 learned_addr learned my addr 172.21.15.165:0/2266936724 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:43.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.855+0000 7f4b6be00640 1 --2- 172.21.15.165:0/2266936724 >> v2:172.21.15.172:3300/0 conn(0x7f4b740ce920 0x7f4b740cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.855+0000 7f4b73e00640 1 -- 172.21.15.165:0/2266936724 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823592969 0 0) 0x7f4b74144b30 con 0x7f4b74144750 2024-04-25T10:06:43.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.856+0000 7f4b73e00640 1 -- 172.21.15.165:0/2266936724 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b58003520 con 0x7f4b74144750 2024-04-25T10:06:43.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.856+0000 7f4b73e00640 1 -- 172.21.15.165:0/2266936724 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2630153362 0 0) 0x7f4b58003520 con 0x7f4b74144750 2024-04-25T10:06:43.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.856+0000 7f4b6be00640 1 -- 172.21.15.165:0/2266936724 >> v1:172.21.15.172:6789/0 conn(0x7f4b7406d290 legacy=0x7f4b7406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.856+0000 7f4b6be00640 1 -- 172.21.15.165:0/2266936724 >> v1:172.21.15.165:6789/0 conn(0x7f4b74144750 legacy=0x7f4b7406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.856+0000 7f4b6be00640 1 -- 172.21.15.165:0/2266936724 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b74071890 con 0x7f4b740ce920 2024-04-25T10:06:43.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.857+0000 7f4b6be00640 1 --2- 172.21.15.165:0/2266936724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740cedb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0097a0 tx=0x7f4b5c02a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3cae580ec175e2e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.858+0000 7f4b73e00640 1 -- 172.21.15.165:0/2266936724 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c037070 con 0x7f4b740ce920 2024-04-25T10:06:43.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.858+0000 7f4b73e00640 1 -- 172.21.15.165:0/2266936724 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c02ac50 con 0x7f4b740ce920 2024-04-25T10:06:43.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.858+0000 7f4b73e00640 1 -- 172.21.15.165:0/2266936724 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64001670 con 0x7f4b740ce920 2024-04-25T10:06:43.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 -- 172.21.15.165:0/2266936724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 msgr2=0x7f4b740cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 --2- 172.21.15.165:0/2266936724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740cedb0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c0097a0 tx=0x7f4b5c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 -- 172.21.15.165:0/2266936724 shutdown_connections 2024-04-25T10:06:43.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 --2- 172.21.15.165:0/2266936724 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740cedb0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 -- 172.21.15.165:0/2266936724 >> 172.21.15.165:0/2266936724 conn(0x7f4b7406abe0 msgr2=0x7f4b7406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:43.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 -- 172.21.15.165:0/2266936724 shutdown_connections 2024-04-25T10:06:43.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.859+0000 7f4b7b800640 1 -- 172.21.15.165:0/2266936724 wait complete. 2024-04-25T10:06:43.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.861+0000 7f4b7b800640 1 Processor -- start 2024-04-25T10:06:43.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.861+0000 7f4b7b800640 1 -- start start 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b7b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 0x7f4b7413df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b7b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740afc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b7b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b74144750 0x7f4b74140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b7b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7406f020 con 0x7f4b7406d290 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b7b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b7406ece0 con 0x7f4b740ce920 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b7b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b74145450 con 0x7f4b74144750 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b78e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 0x7f4b7413df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.862+0000 7f4b78e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 0x7f4b7413df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55654/0 (socket says 172.21.15.165:55654) 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 -- 172.21.15.165:0/954633490 learned_addr learned my addr 172.21.15.165:0/954633490 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:43.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b6be00640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740afc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b79800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b74144750 0x7f4b74140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 -- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b74144750 msgr2=0x7f4b74140290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b74144750 0x7f4b74140290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 -- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 msgr2=0x7f4b740afc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740afc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 -- 172.21.15.165:0/954633490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b7413f250 con 0x7f4b7406d290 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b79800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b74144750 0x7f4b74140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.863+0000 7f4b78e00640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 0x7f4b7413df60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c750 tx=0x7f4b6400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.864+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64017070 con 0x7f4b7406d290 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.864+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b64005ad0 con 0x7f4b7406d290 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.864+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64007070 con 0x7f4b7406d290 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.864+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b7413f4d0 con 0x7f4b7406d290 2024-04-25T10:06:43.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.864+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7413fa90 con 0x7f4b7406d290 2024-04-25T10:06:43.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.865+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b74073990 con 0x7f4b7406d290 2024-04-25T10:06:43.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.866+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4b6401a070 con 0x7f4b7406d290 2024-04-25T10:06:43.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.867+0000 7f4b71600640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:43.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.867+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4b64089930 con 0x7f4b7406d290 2024-04-25T10:06:43.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.867+0000 7f4b6be00640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:43.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.868+0000 7f4b6be00640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b5006b4d0 0x7f4b5006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c034000 tx=0x7f4b5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:43.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:43.872+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4b7413f250 con 0x7f4b7406d290 2024-04-25T10:06:44.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.108+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b740d0510 con 0x7f4b7406d290 2024-04-25T10:06:44.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.108+0000 7f4b71600640 1 -- 172.21.15.165:0/954633490 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f4b64058610 con 0x7f4b7406d290 2024-04-25T10:06:44.115 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:44.115 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:44.116 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b5006b4d0 msgr2=0x7f4b5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b5006b4d0 0x7f4b5006d990 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4b5c034000 tx=0x7f4b5c02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 msgr2=0x7f4b7413df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 0x7f4b7413df60 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4b6400c750 tx=0x7f4b6400cc10 comp rx=0 tx=0).stop 2024-04-25T10:06:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 shutdown_connections 2024-04-25T10:06:44.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b5006b4d0 0x7f4b5006d990 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4b74144750 0x7f4b74140290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4b740ce920 0x7f4b740afc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 --2- 172.21.15.165:0/954633490 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4b7406d290 0x7f4b7413df60 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 >> 172.21.15.165:0/954633490 conn(0x7f4b7406abe0 msgr2=0x7f4b74143990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 shutdown_connections 2024-04-25T10:06:44.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.112+0000 7f4b7b800640 1 -- 172.21.15.165:0/954633490 wait complete. 2024-04-25T10:06:44.137 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:06:44.350 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:44.351 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:44.351 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:44.366 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:44.366 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:06:44.366 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:44.366 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:06:44.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.572+0000 7f6a6d800640 1 Processor -- start 2024-04-25T10:06:44.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.572+0000 7f6a6d800640 1 -- start start 2024-04-25T10:06:44.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.574+0000 7f6a6d800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a680b5b40 0x7f6a68168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.574+0000 7f6a6d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6815e110 con 0x7f6a6815dd30 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.574+0000 7f6a6d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a680b2fd0 con 0x7f6a6806bff0 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.574+0000 7f6a6d800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6806d000 con 0x7f6a680b5b40 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.574+0000 7f6a67e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6a6815dd30 0x7f6a680cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40006/0 (socket says 172.21.15.165:40006) 2024-04-25T10:06:44.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.574+0000 7f6a67e00640 1 -- 172.21.15.165:0/4020324136 learned_addr learned my addr 172.21.15.165:0/4020324136 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/4020324136 >> v2:172.21.15.172:3300/0 conn(0x7f6a680b5b40 0x7f6a68168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100050442 0 0) 0x7f6a6815e110 con 0x7f6a6815dd30 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a4c003520 con 0x7f6a6815dd30 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3666716409 0 0) 0x7f6a4c003520 con 0x7f6a6815dd30 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 >> v2:172.21.15.172:3300/0 conn(0x7f6a680b5b40 msgr2=0x7f6a68168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 --2- 172.21.15.165:0/4020324136 >> v2:172.21.15.172:3300/0 conn(0x7f6a680b5b40 0x7f6a68168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 >> v1:172.21.15.172:6789/0 conn(0x7f6a6806bff0 legacy=0x7f6a680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6806c480 con 0x7f6a6815dd30 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6a580019f0 con 0x7f6a6815dd30 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 >> v1:172.21.15.165:6789/0 conn(0x7f6a6815dd30 legacy=0x7f6a680cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0039b0 0x7f6a4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a4c004800 0x7f6a4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a4c0051c0 0x7f6a4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6806d000 con 0x7f6a4c0051c0 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00c060 con 0x7f6a4c0039b0 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.575+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a4c00c260 con 0x7f6a4c004800 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a4c0051c0 0x7f6a4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a67e00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0039b0 0x7f6a4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a67400640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a4c004800 0x7f6a4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a6ce00640 1 -- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a4c004800 msgr2=0x7f6a4c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a4c004800 0x7f6a4c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a6ce00640 1 -- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0039b0 msgr2=0x7f6a4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0039b0 0x7f6a4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a6ce00640 1 -- 172.21.15.165:0/4020324136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a4c003520 con 0x7f6a4c0051c0 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a67e00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0039b0 0x7f6a4c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:06:44.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.576+0000 7f6a67400640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a4c004800 0x7f6a4c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.577+0000 7f6a6ce00640 1 -- 172.21.15.165:0/4020324136 reap_dead start 2024-04-25T10:06:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.577+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a4c0051c0 0x7f6a4c00b9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6a6808c1f0 tx=0x7f6a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca15bba9898457 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60036070 con 0x7f6a4c0051c0 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a60031400 con 0x7f6a4c0051c0 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a66a00640 1 -- 172.21.15.165:0/4020324136 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60009690 con 0x7f6a4c0051c0 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a6d800640 1 -- 172.21.15.165:0/4020324136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a4c0051c0 msgr2=0x7f6a4c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a6d800640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a4c0051c0 0x7f6a4c00b9c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f6a6808c1f0 tx=0x7f6a4c005830 comp rx=0 tx=0).stop 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a6d800640 1 -- 172.21.15.165:0/4020324136 shutdown_connections 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a6d800640 1 --2- 172.21.15.165:0/4020324136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a4c0051c0 0x7f6a4c00b9c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.578+0000 7f6a6d800640 1 -- 172.21.15.165:0/4020324136 >> 172.21.15.165:0/4020324136 conn(0x7f6a680d0610 msgr2=0x7f6a680d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.579+0000 7f6a6d800640 1 -- 172.21.15.165:0/4020324136 shutdown_connections 2024-04-25T10:06:44.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.579+0000 7f6a6d800640 1 -- 172.21.15.165:0/4020324136 wait complete. 2024-04-25T10:06:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.580+0000 7f6a6d800640 1 Processor -- start 2024-04-25T10:06:44.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.580+0000 7f6a6d800640 1 -- start start 2024-04-25T10:06:44.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.581+0000 7f6a6d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0051c0 0x7f6a680caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.581+0000 7f6a6d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 0x7f6a680abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.581+0000 7f6a6d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a681f3d40 0x7f6a680ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.581+0000 7f6a6d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6806cd20 con 0x7f6a680cc100 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.581+0000 7f6a6d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6806ca40 con 0x7f6a4c0051c0 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.581+0000 7f6a6d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a680b5430 con 0x7f6a681f3d40 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 0x7f6a680abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 0x7f6a680abd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55668/0 (socket says 172.21.15.165:55668) 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 -- 172.21.15.165:0/2192717028 learned_addr learned my addr 172.21.15.165:0/2192717028 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67e00640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0051c0 0x7f6a680caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a681f3d40 0x7f6a680ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 -- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a681f3d40 msgr2=0x7f6a680ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a681f3d40 0x7f6a680ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 -- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0051c0 msgr2=0x7f6a680caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0051c0 0x7f6a680caa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 -- 172.21.15.165:0/2192717028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a680cb060 con 0x7f6a680cc100 2024-04-25T10:06:44.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.582+0000 7f6a67400640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 0x7f6a680abd50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400d770 tx=0x7f6a5400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.583+0000 7f6a5fe00640 1 -- 172.21.15.165:0/2192717028 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54005270 con 0x7f6a680cc100 2024-04-25T10:06:44.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.583+0000 7f6a5fe00640 1 -- 172.21.15.165:0/2192717028 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54017070 con 0x7f6a680cc100 2024-04-25T10:06:44.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.583+0000 7f6a5fe00640 1 -- 172.21.15.165:0/2192717028 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5400b6b0 con 0x7f6a680cc100 2024-04-25T10:06:44.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.583+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a680cb340 con 0x7f6a680cc100 2024-04-25T10:06:44.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.583+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a680cb900 con 0x7f6a680cc100 2024-04-25T10:06:44.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.585+0000 7f6a5fe00640 1 -- 172.21.15.165:0/2192717028 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a5400b850 con 0x7f6a680cc100 2024-04-25T10:06:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.586+0000 7f6a5fe00640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a4806b500 0x7f6a4806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.586+0000 7f6a5fe00640 1 -- 172.21.15.165:0/2192717028 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a5408a1d0 con 0x7f6a680cc100 2024-04-25T10:06:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.587+0000 7f6a67e00640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a4806b500 0x7f6a4806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.587+0000 7f6a71c34740 1 Processor -- start 2024-04-25T10:06:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.587+0000 7f6a71c34740 1 -- start start 2024-04-25T10:06:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.587+0000 7f6a67e00640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a4806b500 0x7f6a4806d9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a58002230 tx=0x7f6a58001250 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 0x55acdc73f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc76f560 0x55acdc744370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a71c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdc6a8880 con 0x55acdc78cb80 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdc651d00 con 0x55acdc76f560 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdc6a4920 con 0x55acdc76fca0 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 0x55acdc73f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 0x55acdc73f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55684/0 (socket says 172.21.15.165:55684) 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 -- 172.21.15.165:0/108213355 learned_addr learned my addr 172.21.15.165:0/108213355 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 -- 172.21.15.165:0/108213355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 msgr2=0x55acdc76f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 -- 172.21.15.165:0/108213355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc76f560 msgr2=0x55acdc744370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc76f560 0x55acdc744370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.588+0000 7f6a67e00640 1 -- 172.21.15.165:0/108213355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdc65e1f0 con 0x55acdc78cb80 2024-04-25T10:06:44.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.589+0000 7f6a67e00640 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 0x55acdc73f510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6a58007b10 tx=0x7f6a58012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46612a4290b78d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.589+0000 7f6a5cc00640 1 -- 172.21.15.165:0/108213355 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58025070 con 0x55acdc78cb80 2024-04-25T10:06:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.589+0000 7f6a5cc00640 1 -- 172.21.15.165:0/108213355 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5801b050 con 0x55acdc78cb80 2024-04-25T10:06:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.589+0000 7f6a5cc00640 1 -- 172.21.15.165:0/108213355 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58010040 con 0x55acdc78cb80 2024-04-25T10:06:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 -- 172.21.15.165:0/108213355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 msgr2=0x55acdc73f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 0x55acdc73f510 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f6a58007b10 tx=0x7f6a58012000 comp rx=0 tx=0).stop 2024-04-25T10:06:44.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 -- 172.21.15.165:0/108213355 shutdown_connections 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc76f560 0x55acdc744370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 --2- 172.21.15.165:0/108213355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc78cb80 0x55acdc73f510 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 -- 172.21.15.165:0/108213355 >> 172.21.15.165:0/108213355 conn(0x55acdc741780 msgr2=0x55acdc78a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 -- 172.21.15.165:0/108213355 shutdown_connections 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 -- 172.21.15.165:0/108213355 wait complete. 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.590+0000 7f6a71c34740 1 -- start start 2024-04-25T10:06:44.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.591+0000 7f6a71c34740 1 Processor -- start 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc75fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc7702e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc76b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdc743930 con 0x55acdc76fca0 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdc6471c0 con 0x55acdc78cb80 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdc742d20 con 0x55acdc770820 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a71c34740 10 client.0 _ll_register_callbacks cb 0x55acdc664b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc75fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc75fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55700/0 (socket says 172.21.15.165:55700) 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 -- 172.21.15.165:0/1579927403 learned_addr learned my addr 172.21.15.165:0/1579927403 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67400640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc7702e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc76b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 -- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 msgr2=0x55acdc76b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc76b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 -- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 msgr2=0x55acdc7702e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc7702e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.592+0000 7f6a67e00640 1 -- 172.21.15.165:0/1579927403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acdc489690 con 0x55acdc76fca0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a67e00640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc75fa80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001b60 tx=0x7f6a680b62b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eb60985e7a07d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a51600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a51600640 1 -- 172.21.15.165:0/1579927403 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5802f030 con 0x55acdc76fca0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a51600640 1 -- 172.21.15.165:0/1579927403 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a58014e00 con 0x55acdc76fca0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a5801b050 con 0x55acdc76fca0 2024-04-25T10:06:44.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a51600640 1 -- 172.21.15.165:0/1579927403 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a58038050 con 0x55acdc76fca0 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a71c34740 10 client.6039 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.593+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55acdc4e9e20 con 0x55acdc76fca0 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a51600640 1 -- 172.21.15.165:0/1579927403 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a5801b050 con 0x55acdc76fca0 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a51600640 1 -- 172.21.15.165:0/1579927403 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a5803b020 con 0x55acdc76fca0 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a71c34740 10 client.6039 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a71c34740 10 client.6039 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a71c34740 20 client.6039 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a71c34740 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc79dad0 0x55acdc79deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a71c34740 4 client.6039 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.594+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acdc6b5dc0 con 0x55acdc79dad0 2024-04-25T10:06:44.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.595+0000 7f6a67400640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc79dad0 0x55acdc79deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.596+0000 7f6a67400640 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc79dad0 0x55acdc79deb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce7e334ec4bf647d server_cookie=bde4c543f515a1a1 in_seq=0 out_seq=0 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.596+0000 7f6a51600640 10 client.6039 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.597+0000 7f6a51600640 1 -- 172.21.15.165:0/1579927403 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55acdc6b5dc0 con 0x55acdc79dad0 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.597+0000 7f6a51600640 10 client.6039 handle_command_reply: tid=0 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 client.6039 shutdown 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 msgr2=0x55acdc75fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc75fa80 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001b60 tx=0x7f6a680b62b0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 shutdown_connections 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc79dad0 0x55acdc79deb0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc76b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc7702e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 --2- 172.21.15.165:0/1579927403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc75fa80 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a51600640 0 client.6039 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 >> 172.21.15.165:0/1579927403 conn(0x55acdc76f560 msgr2=0x55acdc78a020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 shutdown_connections 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 1 -- 172.21.15.165:0/1579927403 wait complete. 2024-04-25T10:06:44.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.598+0000 7f6a71c34740 20 client.6039 trim_cache size 0 max 16384 2024-04-25T10:06:44.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.615+0000 7f6a71c34740 1 Processor -- start 2024-04-25T10:06:44.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.615+0000 7f6a71c34740 1 -- start start 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.615+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 0x55acdc78a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a71c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68200960 con 0x55acdc770820 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681ffac0 con 0x55acdc78cb80 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681feec0 con 0x55acdc76fca0 2024-04-25T10:06:44.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 0x55acdc78a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 0x55acdc78a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55702/0 (socket says 172.21.15.165:55702) 2024-04-25T10:06:44.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 -- 172.21.15.165:0/213330706 learned_addr learned my addr 172.21.15.165:0/213330706 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 -- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 msgr2=0x55acdc76a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67e00640 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a6ce00640 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 -- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 msgr2=0x55acdc78a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.616+0000 7f6a67400640 1 -- 172.21.15.165:0/213330706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acdc37cf80 con 0x55acdc770820 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.617+0000 7f6a67400640 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 0x55acdc78a270 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6a54058740 tx=0x7f6a5400df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2aa785b80f84c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.617+0000 7f6a53e00640 1 -- 172.21.15.165:0/213330706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5405c900 con 0x55acdc770820 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.617+0000 7f6a53e00640 1 -- 172.21.15.165:0/213330706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54070020 con 0x55acdc770820 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.617+0000 7f6a53e00640 1 -- 172.21.15.165:0/213330706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a54060ec0 con 0x55acdc770820 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.617+0000 7f6a71c34740 1 -- 172.21.15.165:0/213330706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 msgr2=0x55acdc78a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.617+0000 7f6a71c34740 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 0x55acdc78a270 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6a54058740 tx=0x7f6a5400df70 comp rx=0 tx=0).stop 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 -- 172.21.15.165:0/213330706 shutdown_connections 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc770820 0x55acdc78a270 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 --2- 172.21.15.165:0/213330706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc76fca0 0x55acdc76a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 -- 172.21.15.165:0/213330706 >> 172.21.15.165:0/213330706 conn(0x55acdc76f560 msgr2=0x55acdc742320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 -- 172.21.15.165:0/213330706 shutdown_connections 2024-04-25T10:06:44.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 -- 172.21.15.165:0/213330706 wait complete. 2024-04-25T10:06:44.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.618+0000 7f6a71c34740 1 -- start start 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 Processor -- start 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc7e1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc789e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acdc78acf0 con 0x55acdc76fca0 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a681ff4c0 con 0x55acdc78cb80 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.619+0000 7f6a71c34740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55acdc75f9c0 con 0x55acdc770820 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a71c34740 10 client.0 _ll_register_callbacks cb 0x7f6a58023070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:44.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc7e1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc7e1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55718/0 (socket says 172.21.15.165:55718) 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 -- 172.21.15.165:0/2331348162 learned_addr learned my addr 172.21.15.165:0/2331348162 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 -- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 msgr2=0x55acdc789e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc789e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 -- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 msgr2=0x55acdc78a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 -- 172.21.15.165:0/2331348162 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55acdc76bce0 con 0x55acdc76fca0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a67e00640 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc7e1cb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6a58020800 tx=0x7f6a58009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ae77fd3dd56252f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.620+0000 7f6a51600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a51600640 1 -- 172.21.15.165:0/2331348162 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5802f070 con 0x55acdc76fca0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a51600640 1 -- 172.21.15.165:0/2331348162 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a5802b8d0 con 0x55acdc76fca0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a54070020 con 0x55acdc76fca0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a51600640 1 -- 172.21.15.165:0/2331348162 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a58046050 con 0x55acdc76fca0 2024-04-25T10:06:44.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 10 client.6045 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55acdc7fd440 con 0x55acdc76fca0 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a51600640 1 -- 172.21.15.165:0/2331348162 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a54070020 con 0x55acdc76fca0 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a51600640 1 -- 172.21.15.165:0/2331348162 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6a58049020 con 0x55acdc76fca0 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 10 client.6045 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 10 client.6045 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 20 client.6045 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc788e30 0x55acdc7e25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 4 client.6045 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:06:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.621+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55acdc6b0110 con 0x55acdc788e30 2024-04-25T10:06:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.622+0000 7f6a67400640 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc788e30 0x55acdc7e25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.623+0000 7f6a67400640 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc788e30 0x55acdc7e25e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f08c18c2392e36f4 server_cookie=816bab7a01e14f90 in_seq=0 out_seq=0 2024-04-25T10:06:44.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.623+0000 7f6a51600640 10 client.6045 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:44.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.624+0000 7f6a51600640 1 -- 172.21.15.165:0/2331348162 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55acdc6b0110 con 0x55acdc788e30 2024-04-25T10:06:44.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.624+0000 7f6a51600640 10 client.6045 handle_command_reply: tid=0 2024-04-25T10:06:44.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.624+0000 7f6a71c34740 1 client.6045 shutdown 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.624+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 msgr2=0x55acdc7e1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.624+0000 7f6a71c34740 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc7e1cb0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6a58020800 tx=0x7f6a58009000 comp rx=0 tx=0).stop 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 shutdown_connections 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55acdc788e30 0x55acdc7e25e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55acdc78cb80 0x55acdc78a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55acdc770820 0x55acdc789e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 --2- 172.21.15.165:0/2331348162 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55acdc76fca0 0x55acdc7e1cb0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a51600640 0 client.6045 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 >> 172.21.15.165:0/2331348162 conn(0x55acdc76f560 msgr2=0x55acdc75eeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 shutdown_connections 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 1 -- 172.21.15.165:0/2331348162 wait complete. 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a71c34740 20 client.6045 trim_cache size 0 max 16384 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "0b7996f5-fc36-40ee-960d-af90e831e9f9", 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:44.638 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a4806b500 msgr2=0x7f6a4806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a6d800640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a4806b500 0x7f6a4806d9c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f6a58002230 tx=0x7f6a58001250 comp rx=0 tx=0).stop 2024-04-25T10:06:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 msgr2=0x7f6a680abd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.625+0000 7f6a6d800640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 0x7f6a680abd50 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f6a5400d770 tx=0x7f6a5400dc30 comp rx=0 tx=0).stop 2024-04-25T10:06:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 shutdown_connections 2024-04-25T10:06:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a4806b500 0x7f6a4806d9c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a681f3d40 0x7f6a680ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a680cc100 0x7f6a680abd50 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 --2- 172.21.15.165:0/2192717028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a4c0051c0 0x7f6a680caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 >> 172.21.15.165:0/2192717028 conn(0x7f6a680d0610 msgr2=0x7f6a680b83a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 shutdown_connections 2024-04-25T10:06:44.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.626+0000 7f6a6d800640 1 -- 172.21.15.165:0/2192717028 wait complete. 2024-04-25T10:06:44.650 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '0b7996f5-fc36-40ee-960d-af90e831e9f9', 'mode': 'asynchronous'}' 2024-04-25T10:06:44.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:44.651 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_0) 2024-04-25T10:06:44.704 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' 2024-04-25T10:06:44.705 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.897+0000 7fb54e400640 1 Processor -- start 2024-04-25T10:06:44.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.898+0000 7fb54e400640 1 -- start start 2024-04-25T10:06:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.899+0000 7fb54e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb548144750 0x7fb54806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.899+0000 7fb54e400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5480cf030 0x7fb54806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.899+0000 7fb54e400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb548144b30 con 0x7fb54806eb10 2024-04-25T10:06:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.899+0000 7fb54e400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480712d0 con 0x7fb5480cf030 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.899+0000 7fb54e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480715b0 con 0x7fb548144750 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.899+0000 7fb547e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5480cf030 0x7fb54806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb5480cf030 0x7fb54806d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55726/0 (socket says 172.21.15.165:55726) 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547e00640 1 -- 172.21.15.165:0/122795081 learned_addr learned my addr 172.21.15.165:0/122795081 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547400640 1 --2- 172.21.15.165:0/122795081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb548144750 0x7fb54806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547e00640 1 -- 172.21.15.165:0/122795081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb548144750 msgr2=0x7fb54806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547e00640 1 --2- 172.21.15.165:0/122795081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb548144750 0x7fb54806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547e00640 1 -- 172.21.15.165:0/122795081 >> v1:172.21.15.172:6789/0 conn(0x7fb54806eb10 legacy=0x7fb5480ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.900+0000 7fb547e00640 1 -- 172.21.15.165:0/122795081 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548071890 con 0x7fb5480cf030 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.901+0000 7fb547e00640 1 --2- 172.21.15.165:0/122795081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5480cf030 0x7fb54806d290 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb540009e50 tx=0x7fb54002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=692fde29e68935e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.901+0000 7fb546a00640 1 -- 172.21.15.165:0/122795081 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540036070 con 0x7fb5480cf030 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.901+0000 7fb546a00640 1 -- 172.21.15.165:0/122795081 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb540031600 con 0x7fb5480cf030 2024-04-25T10:06:44.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.901+0000 7fb546a00640 1 -- 172.21.15.165:0/122795081 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5400319d0 con 0x7fb5480cf030 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 -- 172.21.15.165:0/122795081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5480cf030 msgr2=0x7fb54806d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 --2- 172.21.15.165:0/122795081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5480cf030 0x7fb54806d290 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb540009e50 tx=0x7fb54002a630 comp rx=0 tx=0).stop 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 -- 172.21.15.165:0/122795081 shutdown_connections 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 --2- 172.21.15.165:0/122795081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb5480cf030 0x7fb54806d290 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 --2- 172.21.15.165:0/122795081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb548144750 0x7fb54806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 -- 172.21.15.165:0/122795081 >> 172.21.15.165:0/122795081 conn(0x7fb54806abe0 msgr2=0x7fb54806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:44.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 -- 172.21.15.165:0/122795081 shutdown_connections 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.902+0000 7fb54e400640 1 -- 172.21.15.165:0/122795081 wait complete. 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.904+0000 7fb54e400640 1 Processor -- start 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.904+0000 7fb54e400640 1 -- start start 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb54e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54806eb10 0x7fb54815eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb54e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5480cf030 0x7fb5481e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb54e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 0x7fb548130d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb54e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5481455d0 con 0x7fb548144750 2024-04-25T10:06:44.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb54e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548145450 con 0x7fb5480cf030 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb54e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54815c000 con 0x7fb54806eb10 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb547e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 0x7fb548130d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb547e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 0x7fb548130d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55736/0 (socket says 172.21.15.165:55736) 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.905+0000 7fb547e00640 1 -- 172.21.15.165:0/2120349945 learned_addr learned my addr 172.21.15.165:0/2120349945 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54806eb10 0x7fb54815eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb53ea00640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5480cf030 0x7fb5481e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547e00640 1 -- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54806eb10 msgr2=0x7fb54815eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547e00640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54806eb10 0x7fb54815eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547e00640 1 -- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5480cf030 msgr2=0x7fb5481e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547e00640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5480cf030 0x7fb5481e7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547e00640 1 -- 172.21.15.165:0/2120349945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54815f3a0 con 0x7fb548144750 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb53ea00640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5480cf030 0x7fb5481e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb547e00640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 0x7fb548130d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb540034040 tx=0x7fb540004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.906+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540036050 con 0x7fb548144750 2024-04-25T10:06:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.907+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb540042070 con 0x7fb548144750 2024-04-25T10:06:44.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.907+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009510 con 0x7fb548144750 2024-04-25T10:06:44.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.907+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54815f680 con 0x7fb548144750 2024-04-25T10:06:44.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.907+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54815fc40 con 0x7fb548144750 2024-04-25T10:06:44.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.908+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548160020 con 0x7fb548144750 2024-04-25T10:06:44.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.909+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb540007bd0 con 0x7fb548144750 2024-04-25T10:06:44.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.910+0000 7fb53fe00640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52406b4d0 0x7fb52406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:44.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.910+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb5400ab290 con 0x7fb548144750 2024-04-25T10:06:44.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.910+0000 7fb547400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52406b4d0 0x7fb52406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:44.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.911+0000 7fb547400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52406b4d0 0x7fb52406d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb538006fd0 tx=0x7fb538008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:44.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:44.915+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb548160020 con 0x7fb548144750 2024-04-25T10:06:45.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.150+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb548073990 con 0x7fb548144750 2024-04-25T10:06:45.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.151+0000 7fb53fe00640 1 -- 172.21.15.165:0/2120349945 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7fb540045360 con 0x7fb548144750 2024-04-25T10:06:45.158 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:45.158 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:45.158 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:45.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.154+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52406b4d0 msgr2=0x7fb52406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.154+0000 7fb54e400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52406b4d0 0x7fb52406d990 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fb538006fd0 tx=0x7fb538008040 comp rx=0 tx=0).stop 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.154+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 msgr2=0x7fb548130d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.154+0000 7fb54e400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 0x7fb548130d70 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fb540034040 tx=0x7fb540004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 shutdown_connections 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52406b4d0 0x7fb52406d990 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb548144750 0x7fb548130d70 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb5480cf030 0x7fb5481e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 --2- 172.21.15.165:0/2120349945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54806eb10 0x7fb54815eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 >> 172.21.15.165:0/2120349945 conn(0x7fb54806abe0 msgr2=0x7fb54813e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 shutdown_connections 2024-04-25T10:06:45.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.155+0000 7fb54e400640 1 -- 172.21.15.165:0/2120349945 wait complete. 2024-04-25T10:06:45.180 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_0 2024-04-25T10:06:45.415 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:45.415 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:45.416 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:45.430 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:45.431 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:45.431 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:45.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:45.488 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:06:45.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:06:45.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:06:45.570 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286608 s, 36.6 MB/s 2024-04-25T10:06:45.572 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:06:45.572 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.774+0000 7f2315a00640 1 Processor -- start 2024-04-25T10:06:45.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.774+0000 7f2315a00640 1 -- start start 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.775+0000 7f2315a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 0x7f2310070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.775+0000 7f2315a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2310071530 0x7f23100cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.775+0000 7f2315a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231015bf30 con 0x7f23100cd860 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.775+0000 7f2315a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231015c860 con 0x7f2310071530 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.775+0000 7f2315a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2310071970 con 0x7f2310144440 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.776+0000 7f2315000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f23100cd860 0x7f231006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40008/0 (socket says 172.21.15.165:40008) 2024-04-25T10:06:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.776+0000 7f2315000640 1 -- 172.21.15.165:0/3607649862 learned_addr learned my addr 172.21.15.165:0/3607649862 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.776+0000 7f230f400640 1 --2- 172.21.15.165:0/3607649862 >> v2:172.21.15.172:3300/0 conn(0x7f2310071530 0x7f23100cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.776+0000 7f230fe00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 0x7f2310070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.776+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492227946 0 0) 0x7f231015bf30 con 0x7f23100cd860 2024-04-25T10:06:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.776+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22f4003520 con 0x7f23100cd860 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2054924072 0 0) 0x7f22f4003520 con 0x7f23100cd860 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 msgr2=0x7f2310070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 0x7f2310070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 >> v2:172.21.15.172:3300/0 conn(0x7f2310071530 msgr2=0x7f23100cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 --2- 172.21.15.165:0/3607649862 >> v2:172.21.15.172:3300/0 conn(0x7f2310071530 0x7f23100cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231015c210 con 0x7f23100cd860 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f23080019f0 con 0x7f23100cd860 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 >> v1:172.21.15.165:6789/0 conn(0x7f23100cd860 legacy=0x7f231006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f22f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22f4004800 0x7f22f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 0x7f22f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231015c860 con 0x7f22f40039b0 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2310071970 con 0x7f22f4004800 2024-04-25T10:06:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22f400b9c0 con 0x7f2310144440 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230fe00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f22f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.777+0000 7f230f400640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22f4004800 0x7f22f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.778+0000 7f2315000640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 0x7f22f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.778+0000 7f230fe00640 1 -- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 msgr2=0x7f22f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.778+0000 7f230fe00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2310144440 0x7f22f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.778+0000 7f230fe00640 1 -- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22f4004800 msgr2=0x7f22f4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.778+0000 7f230fe00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22f4004800 0x7f22f4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.778+0000 7f230fe00640 1 -- 172.21.15.165:0/3607649862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f4003520 con 0x7f22f40039b0 2024-04-25T10:06:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.779+0000 7f230fe00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f22f4003dd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2300009ac0 tx=0x7f22f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9675d155c85a3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.779+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300036070 con 0x7f22f40039b0 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.779+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300031400 con 0x7f22f40039b0 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.779+0000 7f230ea00640 1 -- 172.21.15.165:0/3607649862 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300009690 con 0x7f22f40039b0 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315a00640 1 -- 172.21.15.165:0/3607649862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 msgr2=0x7f22f4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315a00640 1 --2- 172.21.15.165:0/3607649862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f22f4003dd0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f2300009ac0 tx=0x7f22f4005830 comp rx=0 tx=0).stop 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315000640 1 -- 172.21.15.165:0/3607649862 reap_dead start 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315a00640 1 -- 172.21.15.165:0/3607649862 shutdown_connections 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315a00640 1 -- 172.21.15.165:0/3607649862 >> 172.21.15.165:0/3607649862 conn(0x7f231006abe0 msgr2=0x7f231006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315a00640 1 -- 172.21.15.165:0/3607649862 shutdown_connections 2024-04-25T10:06:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.780+0000 7f2315a00640 1 -- 172.21.15.165:0/3607649862 wait complete. 2024-04-25T10:06:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.782+0000 7f2315a00640 1 Processor -- start 2024-04-25T10:06:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.782+0000 7f2315a00640 1 -- start start 2024-04-25T10:06:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f2315a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f23101e3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f2315a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23101e4020 0x7f23101e7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f2315a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23101e8170 0x7f23101e6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f2315a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23100cdcd0 con 0x7f22f40039b0 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f2315a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231015bc30 con 0x7f23101e8170 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f2315a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23100cde50 con 0x7f23101e4020 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f230fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f23101e3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f230fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f23101e3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55750/0 (socket says 172.21.15.165:55750) 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.783+0000 7f230fe00640 1 -- 172.21.15.165:0/2311897559 learned_addr learned my addr 172.21.15.165:0/2311897559 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f230fe00640 1 -- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23101e4020 msgr2=0x7f23101e7c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f230fe00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23101e4020 0x7f23101e7c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f230fe00640 1 -- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23101e8170 msgr2=0x7f23101e6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f2315000640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23101e8170 0x7f23101e6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f230fe00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23101e8170 0x7f23101e6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f230fe00640 1 -- 172.21.15.165:0/2311897559 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23101e52f0 con 0x7f22f40039b0 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f230fe00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f23101e3ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2300034040 tx=0x7f2300004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.784+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300040020 con 0x7f22f40039b0 2024-04-25T10:06:45.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.785+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300003940 con 0x7f22f40039b0 2024-04-25T10:06:45.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.785+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300036040 con 0x7f22f40039b0 2024-04-25T10:06:45.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.785+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23101e55d0 con 0x7f22f40039b0 2024-04-25T10:06:45.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.785+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23101e5b90 con 0x7f22f40039b0 2024-04-25T10:06:45.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.786+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231006e270 con 0x7f22f40039b0 2024-04-25T10:06:45.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.787+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2300043020 con 0x7f22f40039b0 2024-04-25T10:06:45.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.788+0000 7f2307e00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f006b500 0x7f22f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:45.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.788+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f23000aa680 con 0x7f22f40039b0 2024-04-25T10:06:45.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.788+0000 7f230f400640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f006b500 0x7f22f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:45.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.789+0000 7f230f400640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f006b500 0x7f22f006d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f23101e5280 tx=0x7f22fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:45.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:45.792+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f231006e270 con 0x7f22f40039b0 2024-04-25T10:06:46.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.027+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2310073990 con 0x7f22f40039b0 2024-04-25T10:06:46.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.028+0000 7f2307e00640 1 -- 172.21.15.165:0/2311897559 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4947 (secure 0 0 0) 0x7f230004a070 con 0x7f22f40039b0 2024-04-25T10:06:46.035 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:46.035 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":28,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":28,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:26.075717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:46.035 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 28 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f006b500 msgr2=0x7f22f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f006b500 0x7f22f006d9c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f23101e5280 tx=0x7f22fc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 msgr2=0x7f23101e3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f23101e3ae0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f2300034040 tx=0x7f2300004730 comp rx=0 tx=0).stop 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 shutdown_connections 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f006b500 0x7f22f006d9c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23101e8170 0x7f23101e6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23101e4020 0x7f23101e7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.031+0000 7f2315a00640 1 --2- 172.21.15.165:0/2311897559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22f40039b0 0x7f23101e3ae0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.032+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 >> 172.21.15.165:0/2311897559 conn(0x7f231006abe0 msgr2=0x7f2310142d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.032+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 shutdown_connections 2024-04-25T10:06:46.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.032+0000 7f2315a00640 1 -- 172.21.15.165:0/2311897559 wait complete. 2024-04-25T10:06:46.057 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:06:46.280 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:46.281 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:46.281 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:46.295 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_0' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:46.296 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:06:46.296 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:06:46.296 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:06:46.296 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:06:46.296 DEBUG:teuthology.orchestra.run.smithi165:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_0").st_ino) 2024-04-25T10:06:46.297 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:06:46.447 INFO:teuthology.orchestra.run.smithi165.stdout:1099511630188 2024-04-25T10:06:46.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:46.462 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:46.462 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:46.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.710+0000 7f3b38a00640 1 Processor -- start 2024-04-25T10:06:46.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.710+0000 7f3b38a00640 1 -- start start 2024-04-25T10:06:46.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b38a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b340ceb20 0x7f3b340c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b38a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3b340c5030 0x7f3b3415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b38a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b34145520 con 0x7f3b34145140 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b38a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b34071270 con 0x7f3b340c5030 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b38a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34071550 con 0x7f3b340ceb20 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b33200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3b340c5030 0x7f3b3415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b33200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3b340c5030 0x7f3b3415bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55756/0 (socket says 172.21.15.165:55756) 2024-04-25T10:06:46.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.712+0000 7f3b33200640 1 -- 172.21.15.165:0/1627776763 learned_addr learned my addr 172.21.15.165:0/1627776763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.713+0000 7f3b33200640 1 -- 172.21.15.165:0/1627776763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b340ceb20 msgr2=0x7f3b340c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.713+0000 7f3b2be00640 1 --2- 172.21.15.165:0/1627776763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b340ceb20 0x7f3b340c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.713+0000 7f3b33200640 1 --2- 172.21.15.165:0/1627776763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b340ceb20 0x7f3b340c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.713+0000 7f3b33200640 1 -- 172.21.15.165:0/1627776763 >> v1:172.21.15.172:6789/0 conn(0x7f3b34145140 legacy=0x7f3b340ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.713+0000 7f3b33200640 1 -- 172.21.15.165:0/1627776763 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b34071830 con 0x7f3b340c5030 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.714+0000 7f3b33200640 1 --2- 172.21.15.165:0/1627776763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b3415bfb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009e50 tx=0x7f3b2c02a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=426cd1bb62c89bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.714+0000 7f3b31e00640 1 -- 172.21.15.165:0/1627776763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c036070 con 0x7f3b340c5030 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.714+0000 7f3b31e00640 1 -- 172.21.15.165:0/1627776763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c031600 con 0x7f3b340c5030 2024-04-25T10:06:46.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.714+0000 7f3b31e00640 1 -- 172.21.15.165:0/1627776763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c0319d0 con 0x7f3b340c5030 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 -- 172.21.15.165:0/1627776763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 msgr2=0x7f3b3415bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 --2- 172.21.15.165:0/1627776763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b3415bfb0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009e50 tx=0x7f3b2c02a630 comp rx=0 tx=0).stop 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 -- 172.21.15.165:0/1627776763 shutdown_connections 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 --2- 172.21.15.165:0/1627776763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b3415bfb0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 --2- 172.21.15.165:0/1627776763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b340ceb20 0x7f3b340c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 -- 172.21.15.165:0/1627776763 >> 172.21.15.165:0/1627776763 conn(0x7f3b34061cc0 msgr2=0x7f3b3406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 -- 172.21.15.165:0/1627776763 shutdown_connections 2024-04-25T10:06:46.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.715+0000 7f3b38a00640 1 -- 172.21.15.165:0/1627776763 wait complete. 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.717+0000 7f3b38a00640 1 Processor -- start 2024-04-25T10:06:46.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.717+0000 7f3b38a00640 1 -- start start 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b38a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b340cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b38a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340ceb20 0x7f3b340cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b38a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34145140 0x7f3b340cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b38a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c5640 con 0x7f3b340c5030 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b38a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b340c54c0 con 0x7f3b340ceb20 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b38a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3415c4f0 con 0x7f3b34145140 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b340cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b340cdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55762/0 (socket says 172.21.15.165:55762) 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 -- 172.21.15.165:0/4106310904 learned_addr learned my addr 172.21.15.165:0/4106310904 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b2be00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340ceb20 0x7f3b340cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 -- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34145140 msgr2=0x7f3b340cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b33200640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34145140 0x7f3b340cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34145140 0x7f3b340cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 -- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340ceb20 msgr2=0x7f3b340cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.718+0000 7f3b32800640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340ceb20 0x7f3b340cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b32800640 1 -- 172.21.15.165:0/4106310904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b340b1b30 con 0x7f3b340c5030 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b2be00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340ceb20 0x7f3b340cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b32800640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b340cdb00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b3f0 tx=0x7f3b2400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b2aa00640 1 -- 172.21.15.165:0/4106310904 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24015020 con 0x7f3b340c5030 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b2aa00640 1 -- 172.21.15.165:0/4106310904 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b24011830 con 0x7f3b340c5030 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b2aa00640 1 -- 172.21.15.165:0/4106310904 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b24011c20 con 0x7f3b340c5030 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b341f3350 con 0x7f3b340c5030 2024-04-25T10:06:46.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.719+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3415d2a0 con 0x7f3b340c5030 2024-04-25T10:06:46.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.721+0000 7f3b2aa00640 1 -- 172.21.15.165:0/4106310904 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b24018020 con 0x7f3b340c5030 2024-04-25T10:06:46.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.722+0000 7f3b2aa00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b1006b4d0 0x7f3b1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.723+0000 7f3b2aa00640 1 -- 172.21.15.165:0/4106310904 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3b2408a060 con 0x7f3b340c5030 2024-04-25T10:06:46.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.723+0000 7f3b2be00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b1006b4d0 0x7f3b1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.723+0000 7f3b3c254740 1 Processor -- start 2024-04-25T10:06:46.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b2be00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b1006b4d0 0x7f3b1006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b340cd8c0 tx=0x7f3b1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 -- start start 2024-04-25T10:06:46.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60d5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a608dcf0 0x55c9a60d3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a5ff0d00 con 0x55c9a60d58b0 2024-04-25T10:06:46.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a5f99f50 con 0x55c9a608dcf0 2024-04-25T10:06:46.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.724+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9a5fecda0 con 0x55c9a60b8010 2024-04-25T10:06:46.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60d5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60d5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55778/0 (socket says 172.21.15.165:55778) 2024-04-25T10:06:46.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 -- 172.21.15.165:0/3804669778 learned_addr learned my addr 172.21.15.165:0/3804669778 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b33200640 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a608dcf0 0x55c9a60d3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 -- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 msgr2=0x55c9a60b83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b32800640 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 -- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a608dcf0 msgr2=0x55c9a60d3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a608dcf0 0x55c9a60d3f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.725+0000 7f3b2be00640 1 -- 172.21.15.165:0/3804669778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9a5fbd130 con 0x55c9a60d58b0 2024-04-25T10:06:46.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.726+0000 7f3b2be00640 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60d5e00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00db30 tx=0x7f3b1c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1f54d22f764ff71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.726+0000 7f3b23000640 1 -- 172.21.15.165:0/3804669778 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c019050 con 0x55c9a60d58b0 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.726+0000 7f3b23000640 1 -- 172.21.15.165:0/3804669778 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c023070 con 0x55c9a60d58b0 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.726+0000 7f3b23000640 1 -- 172.21.15.165:0/3804669778 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c012c50 con 0x55c9a60d58b0 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 -- 172.21.15.165:0/3804669778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 msgr2=0x55c9a60d5e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60d5e00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c00db30 tx=0x7f3b1c010000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 -- 172.21.15.165:0/3804669778 shutdown_connections 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a608dcf0 0x55c9a60d3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60d5e00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 --2- 172.21.15.165:0/3804669778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 -- 172.21.15.165:0/3804669778 >> 172.21.15.165:0/3804669778 conn(0x55c9a608a620 msgr2=0x55c9a60d2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 -- 172.21.15.165:0/3804669778 shutdown_connections 2024-04-25T10:06:46.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.727+0000 7f3b3c254740 1 -- 172.21.15.165:0/3804669778 wait complete. 2024-04-25T10:06:46.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.728+0000 7f3b3c254740 1 -- start start 2024-04-25T10:06:46.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.728+0000 7f3b3c254740 1 Processor -- start 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60d58b0 0x55c9a60b9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60e6880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a608ea90 con 0x55c9a60cb010 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a5f8f530 con 0x55c9a60d58b0 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9a6088550 con 0x55c9a60b8010 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b3c254740 10 client.0 _ll_register_callbacks cb 0x55c9a5fe8430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b33200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60e6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b33200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60e6880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55792/0 (socket says 172.21.15.165:55792) 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.729+0000 7f3b33200640 1 -- 172.21.15.165:0/627772717 learned_addr learned my addr 172.21.15.165:0/627772717 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b2be00640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60d58b0 0x55c9a60b9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b32800640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b33200640 1 -- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 msgr2=0x55c9a60b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b33200640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b33200640 1 -- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60d58b0 msgr2=0x55c9a60b9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b33200640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60d58b0 0x55c9a60b9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b33200640 1 -- 172.21.15.165:0/627772717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9a5fb7dd0 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b2be00640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60d58b0 0x55c9a60b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b33200640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60e6880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009f70 tx=0x7f3b2c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34982a2b1190eaad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b1b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b1b400640 1 -- 172.21.15.165:0/627772717 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c042070 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b1b400640 1 -- 172.21.15.165:0/627772717 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3b2c031dc0 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.730+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b1c023070 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b1b400640 1 -- 172.21.15.165:0/627772717 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b2c04d050 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 10 client.6069 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c9a5d053f0 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b1b400640 1 -- 172.21.15.165:0/627772717 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c023070 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b1b400640 1 -- 172.21.15.165:0/627772717 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3b2c050020 con 0x55c9a60cb010 2024-04-25T10:06:46.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 10 client.6069 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:46.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 10 client.6069 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:46.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 20 client.6069 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:46.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a60d1ad0 0x55c9a60e6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 4 client.6069 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:46.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.731+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9a5ffe240 con 0x55c9a60d1ad0 2024-04-25T10:06:46.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.732+0000 7f3b32800640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a60d1ad0 0x55c9a60e6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.733+0000 7f3b32800640 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a60d1ad0 0x55c9a60e6dc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39aac6250ec2f425 server_cookie=de3cc41255e9994c in_seq=0 out_seq=0 2024-04-25T10:06:46.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.733+0000 7f3b1b400640 10 client.6069 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:46.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.734+0000 7f3b1b400640 1 -- 172.21.15.165:0/627772717 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9a5ffe240 con 0x55c9a60d1ad0 2024-04-25T10:06:46.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.734+0000 7f3b1b400640 10 client.6069 handle_command_reply: tid=0 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.734+0000 7f3b3c254740 1 client.6069 shutdown 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.734+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 msgr2=0x55c9a60e6880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.734+0000 7f3b3c254740 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60e6880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c009f70 tx=0x7f3b2c004ab0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 shutdown_connections 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a60d1ad0 0x55c9a60e6dc0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60e6880 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60d58b0 0x55c9a60b9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 --2- 172.21.15.165:0/627772717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60b8010 0x55c9a60b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b1b400640 0 client.6069 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 >> 172.21.15.165:0/627772717 conn(0x55c9a608dcf0 msgr2=0x55c9a60d63a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 shutdown_connections 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 1 -- 172.21.15.165:0/627772717 wait complete. 2024-04-25T10:06:46.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.735+0000 7f3b3c254740 20 client.6069 trim_cache size 0 max 16384 2024-04-25T10:06:46.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.748+0000 7f3b3c254740 1 Processor -- start 2024-04-25T10:06:46.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.748+0000 7f3b3c254740 1 -- start start 2024-04-25T10:06:46.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a60d36e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60cb010 0x55c9a60e68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b3c254740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b34200360 con 0x55c9a60d58b0 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341ff4a0 con 0x55c9a60b8010 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341fe8a0 con 0x55c9a60cb010 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b33200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b33200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60b5d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55808/0 (socket says 172.21.15.165:55808) 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.749+0000 7f3b33200640 1 -- 172.21.15.165:0/200332410 learned_addr learned my addr 172.21.15.165:0/200332410 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b2be00640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60cb010 0x55c9a60e68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b32800640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a60d36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b33200640 1 -- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60cb010 msgr2=0x55c9a60e68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b33200640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60cb010 0x55c9a60e68a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b33200640 1 -- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 msgr2=0x55c9a60d36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b33200640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a60d36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b33200640 1 -- 172.21.15.165:0/200332410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9a5dcf7d0 con 0x55c9a60d58b0 2024-04-25T10:06:46.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b2be00640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60cb010 0x55c9a60e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b32800640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a60d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b33200640 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60b5d80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02ac20 tx=0x7f3b2c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8356849cbce0bd00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b22600640 1 -- 172.21.15.165:0/200332410 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c042040 con 0x55c9a60d58b0 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b22600640 1 -- 172.21.15.165:0/200332410 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c03c540 con 0x55c9a60d58b0 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.750+0000 7f3b22600640 1 -- 172.21.15.165:0/200332410 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b2c036040 con 0x55c9a60d58b0 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 -- 172.21.15.165:0/200332410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 msgr2=0x55c9a60b5d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60b5d80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3b2c02ac20 tx=0x7f3b2c03c000 comp rx=0 tx=0).stop 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 -- 172.21.15.165:0/200332410 shutdown_connections 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60d58b0 0x55c9a60b5d80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60cb010 0x55c9a60e68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 --2- 172.21.15.165:0/200332410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a60d36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 -- 172.21.15.165:0/200332410 >> 172.21.15.165:0/200332410 conn(0x55c9a608dcf0 msgr2=0x55c9a60e0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 -- 172.21.15.165:0/200332410 shutdown_connections 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.751+0000 7f3b3c254740 1 -- 172.21.15.165:0/200332410 wait complete. 2024-04-25T10:06:46.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.752+0000 7f3b3c254740 1 -- start start 2024-04-25T10:06:46.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.752+0000 7f3b3c254740 1 Processor -- start 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a6137350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60d58b0 0x55c9a60cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a608f600 con 0x55c9a60cb010 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b341feea0 con 0x55c9a60b8010 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9a60d18e0 con 0x55c9a60d58b0 2024-04-25T10:06:46.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b3c254740 10 client.0 _ll_register_callbacks cb 0x55c9a6088a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60d3c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55816/0 (socket says 172.21.15.165:55816) 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.753+0000 7f3b2be00640 1 -- 172.21.15.165:0/915109110 learned_addr learned my addr 172.21.15.165:0/915109110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b32800640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a6137350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b2be00640 1 -- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60d58b0 msgr2=0x55c9a60cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b33200640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60d58b0 0x55c9a60cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b2be00640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60d58b0 0x55c9a60cc2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b2be00640 1 -- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 msgr2=0x55c9a6137350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b2be00640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a6137350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b2be00640 1 -- 172.21.15.165:0/915109110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9a60c8a60 con 0x55c9a60cb010 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b32800640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a6137350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b2be00640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60d3c40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c020040 tx=0x7f3b1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d466293faca9a76 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b1b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b1b400640 1 -- 172.21.15.165:0/915109110 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c01e640 con 0x55c9a60cb010 2024-04-25T10:06:46.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b1b400640 1 -- 172.21.15.165:0/915109110 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3b1c033ca0 con 0x55c9a60cb010 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.754+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b2c03c540 con 0x55c9a60cb010 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b1b400640 1 -- 172.21.15.165:0/915109110 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b1c019020 con 0x55c9a60cb010 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 10 client.6075 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55c9a613d770 con 0x55c9a60cb010 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b1b400640 1 -- 172.21.15.165:0/915109110 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b2c03c540 con 0x55c9a60cb010 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b1b400640 1 -- 172.21.15.165:0/915109110 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3b1c01e7e0 con 0x55c9a60cb010 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 10 client.6075 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 10 client.6075 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 20 client.6075 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a613d260 0x55c9a60cc820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 4 client.6075 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.755+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c9a5ff8590 con 0x55c9a613d260 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.756+0000 7f3b32800640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a613d260 0x55c9a60cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b32800640 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a613d260 0x55c9a60cc820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fa052d7cebd2e97 server_cookie=c7ee7a75d39ce408 in_seq=0 out_seq=0 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b1b400640 10 client.6075 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b1b400640 1 -- 172.21.15.165:0/915109110 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c9a5ff8590 con 0x55c9a613d260 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b1b400640 10 client.6075 handle_command_reply: tid=0 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 client.6075 shutdown 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 msgr2=0x55c9a60d3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60d3c40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3b1c020040 tx=0x7f3b1c004730 comp rx=0 tx=0).stop 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 shutdown_connections 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55c9a613d260 0x55c9a60cc820 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9a60d58b0 0x55c9a60cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9a60cb010 0x55c9a60d3c40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 --2- 172.21.15.165:0/915109110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9a60b8010 0x55c9a6137350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 >> 172.21.15.165:0/915109110 conn(0x55c9a608dcf0 msgr2=0x55c9a60e0350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.757+0000 7f3b1b400640 0 client.6075 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:46.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 shutdown_connections 2024-04-25T10:06:46.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b3c254740 1 -- 172.21.15.165:0/915109110 wait complete. 2024-04-25T10:06:46.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b3c254740 20 client.6075 trim_cache size 0 max 16384 2024-04-25T10:06:46.768 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b1006b4d0 msgr2=0x7f3b1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b1006b4d0 0x7f3b1006d990 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f3b340cd8c0 tx=0x7f3b1c008040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 msgr2=0x7f3b340cdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b340cdb00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f3b2400b3f0 tx=0x7f3b2400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 shutdown_connections 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b1006b4d0 0x7f3b1006d990 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b34145140 0x7f3b340cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b340ceb20 0x7f3b340cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 --2- 172.21.15.165:0/4106310904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b340c5030 0x7f3b340cdb00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.758+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 >> 172.21.15.165:0/4106310904 conn(0x7f3b34061cc0 msgr2=0x7f3b340ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.759+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 shutdown_connections 2024-04-25T10:06:46.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.759+0000 7f3b38a00640 1 -- 172.21.15.165:0/4106310904 wait complete. 2024-04-25T10:06:46.781 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096c.00000000 parent 2024-04-25T10:06:46.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.874+0000 7ff4845e0f40 1 Processor -- start 2024-04-25T10:06:46.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.874+0000 7ff4845e0f40 1 -- start start 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.875+0000 7ff4845e0f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e356f0 0x558e69e1d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.875+0000 7ff4845e0f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69e37b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.875+0000 7ff4845e0f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e3abb0 0x558e69e1f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.875+0000 7ff4845e0f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e69deb840 con 0x558e69e3a7d0 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.875+0000 7ff4845e0f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e69d94dd0 con 0x558e69e3abb0 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.875+0000 7ff4845e0f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e69debae0 con 0x558e69e356f0 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69e37b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69e37b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44356/0 (socket says 172.21.15.165:44356) 2024-04-25T10:06:46.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 -- 172.21.15.165:0/3427033774 learned_addr learned my addr 172.21.15.165:0/3427033774 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff482200640 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e3abb0 0x558e69e1f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 -- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e356f0 msgr2=0x558e69e1d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e356f0 0x558e69e1d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 -- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e3abb0 msgr2=0x558e69e1f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e3abb0 0x558e69e1f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 -- 172.21.15.165:0/3427033774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e69e1e100 con 0x558e69e3a7d0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff480e00640 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69e37b30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00a7a0 tx=0x7ff46c00a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a974eed7d2080bcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.876+0000 7ff47b400640 1 -- 172.21.15.165:0/3427033774 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c018070 con 0x558e69e3a7d0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff47b400640 1 -- 172.21.15.165:0/3427033774 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c007600 con 0x558e69e3a7d0 2024-04-25T10:06:46.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff47b400640 1 -- 172.21.15.165:0/3427033774 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0079b0 con 0x558e69e3a7d0 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3427033774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 msgr2=0x558e69e37b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69e37b30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7ff46c00a7a0 tx=0x7ff46c00a890 comp rx=0 tx=0).stop 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3427033774 shutdown_connections 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e3abb0 0x558e69e1f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69e37b30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3427033774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e356f0 0x558e69e1d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3427033774 >> 172.21.15.165:0/3427033774 conn(0x558e69e1ce60 msgr2=0x558e69e1a230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3427033774 shutdown_connections 2024-04-25T10:06:46.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.877+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3427033774 wait complete. 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.878+0000 7ff4845e0f40 1 Processor -- start 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.878+0000 7ff4845e0f40 1 -- start start 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.879+0000 7ff4845e0f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e356f0 0x558e69ec6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.879+0000 7ff4845e0f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69eb2530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.879+0000 7ff4845e0f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e3abb0 0x558e69eb2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.879+0000 7ff4845e0f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e69ebb080 con 0x558e69e3a7d0 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.879+0000 7ff4845e0f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e69e1d780 con 0x558e69e356f0 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.879+0000 7ff4845e0f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e69ec7ce0 con 0x558e69e3abb0 2024-04-25T10:06:46.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69eb2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69eb2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44360/0 (socket says 172.21.15.165:44360) 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 -- 172.21.15.165:0/3002302785 learned_addr learned my addr 172.21.15.165:0/3002302785 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff481800640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e356f0 0x558e69ec6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 -- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e3abb0 msgr2=0x558e69eb2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff482200640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e3abb0 0x558e69eb2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e3abb0 0x558e69eb2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 -- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e356f0 msgr2=0x558e69ec6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e356f0 0x558e69ec6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 -- 172.21.15.165:0/3002302785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e69eb6710 con 0x558e69e3a7d0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff481800640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e69e356f0 0x558e69ec6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff482200640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e69e3abb0 0x558e69eb2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.880+0000 7ff480e00640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69eb2530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016040 tx=0x7ff46c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.881+0000 7ff47a000640 1 -- 172.21.15.165:0/3002302785 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c021020 con 0x558e69e3a7d0 2024-04-25T10:06:46.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.881+0000 7ff47a000640 1 -- 172.21.15.165:0/3002302785 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff46c019040 con 0x558e69e3a7d0 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.881+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x558e69ec78d0 con 0x558e69e3a7d0 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.881+0000 7ff47a000640 1 -- 172.21.15.165:0/3002302785 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c018070 con 0x558e69e3a7d0 2024-04-25T10:06:46.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.881+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e69ebd870 con 0x558e69e3a7d0 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.883+0000 7ff47a000640 1 -- 172.21.15.165:0/3002302785 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff46c024020 con 0x558e69e3a7d0 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.884+0000 7ff47a000640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff460069eb0 0x7ff46006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.884+0000 7ff47a000640 1 -- 172.21.15.165:0/3002302785 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff46c089f80 con 0x558e69e3a7d0 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.884+0000 7ff481800640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff460069eb0 0x7ff46006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.884+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x558e69ec73a0 0x558e69f54c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.884+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- osd_op(unknown.0.0:1 8.2b 8:d716f75e:::1000000096c.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) v9 -- 0x558e69dd7900 con 0x558e69ec73a0 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.885+0000 7ff481800640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff460069eb0 0x7ff46006c370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4740026c0 tx=0x7ff474008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.885+0000 7ff482200640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x558e69ec73a0 0x558e69f54c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:46.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.886+0000 7ff482200640 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x558e69ec73a0 0x558e69f54c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff482200640 1 -- 172.21.15.165:0/3002302785 <== osd.6 v2:172.21.15.172:6808/4001543510 1 ==== osd_op_reply(1 1000000096c.00000000 [rmxattr] v56'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7ff47c054630 con 0x558e69ec73a0 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x558e69ec73a0 msgr2=0x558e69f54c00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x558e69ec73a0 0x558e69f54c00 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff460069eb0 msgr2=0x7ff46006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff460069eb0 0x7ff46006c370 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff4740026c0 tx=0x7ff474008040 comp rx=0 tx=0).stop 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 msgr2=0x558e69eb2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 --2- 172.21.15.165:0/3002302785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e69e3a7d0 0x558e69eb2530 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7ff46c016040 tx=0x7ff46c003020 comp rx=0 tx=0).stop 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff482200640 1 -- 172.21.15.165:0/3002302785 reap_dead start 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 shutdown_connections 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 >> 172.21.15.165:0/3002302785 conn(0x558e69e1ce60 msgr2=0x558e69e37cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 shutdown_connections 2024-04-25T10:06:46.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:46.890+0000 7ff4845e0f40 1 -- 172.21.15.165:0/3002302785 wait complete. 2024-04-25T10:06:46.901 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' 2024-04-25T10:06:46.901 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:46.901 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_0 2024-04-25T10:06:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.084+0000 7f105a800640 1 Processor -- start 2024-04-25T10:06:47.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.084+0000 7f105a800640 1 -- start start 2024-04-25T10:06:47.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f105a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10540cfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f105a800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10540c75e0 con 0x7f105415bb50 2024-04-25T10:06:47.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f105a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10540c78c0 con 0x7f10540cbf20 2024-04-25T10:06:47.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f105a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10540c8680 con 0x7f10540cf600 2024-04-25T10:06:47.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f1053400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f105415bb50 0x7f10540c6b90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54108/0 (socket says 172.21.15.165:54108) 2024-04-25T10:06:47.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f1053400640 1 -- 172.21.15.165:0/1224598933 learned_addr learned my addr 172.21.15.165:0/1224598933 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f1052a00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10540cfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.086+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038238907 0 0) 0x7f10540c75e0 con 0x7f105415bb50 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1038003520 con 0x7f105415bb50 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2314410288 0 0) 0x7f1038003520 con 0x7f105415bb50 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 msgr2=0x7f10540cfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10540cfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 >> v1:172.21.15.172:6789/0 conn(0x7f10540cbf20 legacy=0x7f10540cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10540c7ba0 con 0x7f105415bb50 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1044001670 con 0x7f105415bb50 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 >> v1:172.21.15.165:6789/0 conn(0x7f105415bb50 legacy=0x7f10540c6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10380039e0 0x7f1038003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1038004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10540c8680 con 0x7f1038004800 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103800bb00 con 0x7f10380039e0 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.087+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103800bd00 con 0x7f10540cf600 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1052a00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1038004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1053400640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10380039e0 0x7f1038003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1053e00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1052a00640 1 -- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 msgr2=0x7f10380051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1052a00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10380051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1052a00640 1 -- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10380039e0 msgr2=0x7f1038003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1052a00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10380039e0 0x7f1038003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1052a00640 1 -- 172.21.15.165:0/1224598933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038003520 con 0x7f1038004800 2024-04-25T10:06:47.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.088+0000 7f1053e00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10540cf600 0x7f10380051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.089+0000 7f1052a00640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1038004c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1040009a90 tx=0x7f1038005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d698015f50a7b0b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.089+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040036070 con 0x7f1038004800 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.089+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1040031400 con 0x7f1038004800 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.089+0000 7f1052000640 1 -- 172.21.15.165:0/1224598933 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1040009690 con 0x7f1038004800 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f105a800640 1 -- 172.21.15.165:0/1224598933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 msgr2=0x7f1038004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f105a800640 1 --2- 172.21.15.165:0/1224598933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1038004c80 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1040009a90 tx=0x7f1038005830 comp rx=0 tx=0).stop 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f1053e00640 1 -- 172.21.15.165:0/1224598933 reap_dead start 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f105a800640 1 -- 172.21.15.165:0/1224598933 shutdown_connections 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f105a800640 1 -- 172.21.15.165:0/1224598933 >> 172.21.15.165:0/1224598933 conn(0x7f105406f500 msgr2=0x7f105406f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f105a800640 1 -- 172.21.15.165:0/1224598933 shutdown_connections 2024-04-25T10:06:47.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.090+0000 7f105a800640 1 -- 172.21.15.165:0/1224598933 wait complete. 2024-04-25T10:06:47.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.092+0000 7f105a800640 1 Processor -- start 2024-04-25T10:06:47.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.092+0000 7f105a800640 1 -- start start 2024-04-25T10:06:47.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f105a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1054155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f105a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f105415bf30 0x7f105414d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f105a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f105415dba0 0x7f105415e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f105a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10540c72a0 con 0x7f1038004800 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f105a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10540c8160 con 0x7f105415dba0 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f105a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105415f6d0 con 0x7f105415bf30 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f1053400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1054155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f1053400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1054155870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44380/0 (socket says 172.21.15.165:44380) 2024-04-25T10:06:47.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.093+0000 7f1053400640 1 -- 172.21.15.165:0/4284399695 learned_addr learned my addr 172.21.15.165:0/4284399695 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f1053400640 1 -- 172.21.15.165:0/4284399695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f105415bf30 msgr2=0x7f105414d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f1053400640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f105415bf30 0x7f105414d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f1053400640 1 -- 172.21.15.165:0/4284399695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f105415dba0 msgr2=0x7f105415e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f1053400640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f105415dba0 0x7f105415e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f1053400640 1 -- 172.21.15.165:0/4284399695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105415e560 con 0x7f1038004800 2024-04-25T10:06:47.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f1053400640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1054155870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f104400e870 tx=0x7f104400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f104b400640 1 -- 172.21.15.165:0/4284399695 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400c8d0 con 0x7f1038004800 2024-04-25T10:06:47.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f104b400640 1 -- 172.21.15.165:0/4284399695 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044018070 con 0x7f1038004800 2024-04-25T10:06:47.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f104b400640 1 -- 172.21.15.165:0/4284399695 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400ccd0 con 0x7f1038004800 2024-04-25T10:06:47.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105415c460 con 0x7f1038004800 2024-04-25T10:06:47.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.094+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105406d1f0 con 0x7f1038004800 2024-04-25T10:06:47.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.097+0000 7f104b400640 1 -- 172.21.15.165:0/4284399695 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1044013420 con 0x7f1038004800 2024-04-25T10:06:47.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.097+0000 7f104b400640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f103406b500 0x7f103406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.098+0000 7f104b400640 1 -- 172.21.15.165:0/4284399695 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f104408a080 con 0x7f1038004800 2024-04-25T10:06:47.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.098+0000 7f1052a00640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f103406b500 0x7f103406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.098+0000 7f105d635740 1 Processor -- start 2024-04-25T10:06:47.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.098+0000 7f105d635740 1 -- start start 2024-04-25T10:06:47.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f1052a00640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f103406b500 0x7f103406d9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1040033000 tx=0x7f104002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d090380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08aeb0 0x55726d08eba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f105d635740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d08f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f105d635740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726cff3d80 con 0x55726d08f0e0 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726cf9ccc0 con 0x55726d08aeb0 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.099+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55726cfefe20 con 0x55726d08ffa0 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d08f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d08f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44388/0 (socket says 172.21.15.165:44388) 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 -- 172.21.15.165:0/4179258309 learned_addr learned my addr 172.21.15.165:0/4179258309 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053400640 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d090380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 -- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 msgr2=0x55726d090380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d090380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 -- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08aeb0 msgr2=0x55726d08eba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1052a00640 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08aeb0 0x55726d08eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08aeb0 0x55726d08eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053e00640 1 -- 172.21.15.165:0/4179258309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55726cdd4c00 con 0x55726d08f0e0 2024-04-25T10:06:47.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.100+0000 7f1053400640 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d090380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f1053e00640 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d08f520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f105406f270 tx=0x7f104c002020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b7738caf299e53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f103fe00640 1 -- 172.21.15.165:0/4179258309 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c018070 con 0x55726d08f0e0 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f103fe00640 1 -- 172.21.15.165:0/4179258309 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c003da0 con 0x55726d08f0e0 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f103fe00640 1 -- 172.21.15.165:0/4179258309 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00da00 con 0x55726d08f0e0 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f105d635740 1 -- 172.21.15.165:0/4179258309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 msgr2=0x55726d08f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f105d635740 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d08f520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f105406f270 tx=0x7f104c002020 comp rx=0 tx=0).stop 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.101+0000 7f105d635740 1 -- 172.21.15.165:0/4179258309 shutdown_connections 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d08f520 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08aeb0 0x55726d08eba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 --2- 172.21.15.165:0/4179258309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d090380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 -- 172.21.15.165:0/4179258309 >> 172.21.15.165:0/4179258309 conn(0x55726d08d330 msgr2=0x55726d0d5b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 -- 172.21.15.165:0/4179258309 shutdown_connections 2024-04-25T10:06:47.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 -- 172.21.15.165:0/4179258309 wait complete. 2024-04-25T10:06:47.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.102+0000 7f105d635740 1 -- start start 2024-04-25T10:06:47.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.103+0000 7f105d635740 1 Processor -- start 2024-04-25T10:06:47.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d0b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08ffa0 0x55726d0b6e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d0b14e0 0x55726d0d9470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726d0bbb30 con 0x55726d08f0e0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726cf92240 con 0x55726d08ffa0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55726d0bafb0 con 0x55726d0b14e0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f105d635740 10 client.0 _ll_register_callbacks cb 0x55726cfeb280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d0b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d0b4bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44392/0 (socket says 172.21.15.165:44392) 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 -- 172.21.15.165:0/2331723681 learned_addr learned my addr 172.21.15.165:0/2331723681 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1052a00640 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08ffa0 0x55726d0b6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 -- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d0b14e0 msgr2=0x55726d0d9470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d0b14e0 0x55726d0d9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 -- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08ffa0 msgr2=0x55726d0b6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08ffa0 0x55726d0b6e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.104+0000 7f1053400640 1 -- 172.21.15.165:0/2331723681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55726ccc84f0 con 0x55726d08f0e0 2024-04-25T10:06:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f1053400640 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d0b4bb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1044001bf0 tx=0x7f1044058f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab0cd9670550fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f103cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f103cc00640 1 -- 172.21.15.165:0/2331723681 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044085070 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f103cc00640 1 -- 172.21.15.165:0/2331723681 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1044060970 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f104c003da0 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f103cc00640 1 -- 172.21.15.165:0/2331723681 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1044060e00 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 10 client.6096 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55726ce35390 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f103cc00640 1 -- 172.21.15.165:0/2331723681 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c003da0 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f103cc00640 1 -- 172.21.15.165:0/2331723681 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1044057b40 con 0x55726d08f0e0 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 10 client.6096 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 10 client.6096 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 20 client.6096 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d0da230 0x55726d0da680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 4 client.6096 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:47.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.105+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55726d0012c0 con 0x55726d0da230 2024-04-25T10:06:47.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.106+0000 7f1052a00640 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d0da230 0x55726d0da680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.107+0000 7f1052a00640 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d0da230 0x55726d0da680 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18c624568ddf1f91 server_cookie=8813609dc00cdd26 in_seq=0 out_seq=0 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.107+0000 7f103cc00640 10 client.6096 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f103cc00640 1 -- 172.21.15.165:0/2331723681 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55726d0012c0 con 0x55726d0da230 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f103cc00640 10 client.6096 handle_command_reply: tid=0 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 client.6096 shutdown 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 msgr2=0x55726d0b4bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d0b4bb0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1044001bf0 tx=0x7f1044058f30 comp rx=0 tx=0).stop 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 shutdown_connections 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d0da230 0x55726d0da680 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d0b14e0 0x55726d0d9470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08ffa0 0x55726d0b6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 --2- 172.21.15.165:0/2331723681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d08f0e0 0x55726d0b4bb0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f103cc00640 0 client.6096 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 >> 172.21.15.165:0/2331723681 conn(0x55726d08aeb0 msgr2=0x55726d090a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 shutdown_connections 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 1 -- 172.21.15.165:0/2331723681 wait complete. 2024-04-25T10:06:47.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.109+0000 7f105d635740 20 client.6096 trim_cache size 0 max 16384 2024-04-25T10:06:47.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.126+0000 7f105d635740 1 Processor -- start 2024-04-25T10:06:47.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.126+0000 7f105d635740 1 -- start start 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.126+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0b6900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.126+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0e8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f105d635740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d0d8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f105d635740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1054200a80 con 0x55726d0b14e0 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541ffc60 con 0x55726d08f0e0 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10541ff150 con 0x55726d08ffa0 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d0d8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d0d8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44404/0 (socket says 172.21.15.165:44404) 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 -- 172.21.15.165:0/1970775907 learned_addr learned my addr 172.21.15.165:0/1970775907 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1052a00640 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0e8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 -- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 msgr2=0x55726d0e8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0e8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 -- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 msgr2=0x55726d0b6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0b6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.127+0000 7f1053e00640 1 -- 172.21.15.165:0/1970775907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55726cca1670 con 0x55726d0b14e0 2024-04-25T10:06:47.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f1053e00640 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d0d8cd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f104c012000 tx=0x7f104c010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb34b42cf572df68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f103f400640 1 -- 172.21.15.165:0/1970775907 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c018070 con 0x55726d0b14e0 2024-04-25T10:06:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f103f400640 1 -- 172.21.15.165:0/1970775907 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c019040 con 0x55726d0b14e0 2024-04-25T10:06:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f103f400640 1 -- 172.21.15.165:0/1970775907 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c006b50 con 0x55726d0b14e0 2024-04-25T10:06:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 -- 172.21.15.165:0/1970775907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 msgr2=0x55726d0d8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d0d8cd0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f104c012000 tx=0x7f104c010000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 -- 172.21.15.165:0/1970775907 shutdown_connections 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d0d8cd0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0e8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 --2- 172.21.15.165:0/1970775907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0b6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.128+0000 7f105d635740 1 -- 172.21.15.165:0/1970775907 >> 172.21.15.165:0/1970775907 conn(0x55726d08aeb0 msgr2=0x55726d0e56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.129+0000 7f105d635740 1 -- 172.21.15.165:0/1970775907 shutdown_connections 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.129+0000 7f105d635740 1 -- 172.21.15.165:0/1970775907 wait complete. 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.129+0000 7f105d635740 1 -- start start 2024-04-25T10:06:47.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.129+0000 7f105d635740 1 Processor -- start 2024-04-25T10:06:47.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d13ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55726d08cbd0 con 0x55726d0b14e0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10541ff690 con 0x55726d08f0e0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55726d0bbb30 con 0x55726d08ffa0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.130+0000 7f105d635740 10 client.0 _ll_register_callbacks cb 0x7f104c007ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d13ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d13ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44416/0 (socket says 172.21.15.165:44416) 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 -- 172.21.15.165:0/1394386709 learned_addr learned my addr 172.21.15.165:0/1394386709 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053400640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1052a00640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 -- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 msgr2=0x55726d0ef5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 -- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 msgr2=0x55726d0ef0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0ef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 -- 172.21.15.165:0/1394386709 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55726d0dbda0 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f1053e00640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d13ffa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f104c016040 tx=0x7f104c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=154ee85c89b6d7bf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.131+0000 7f103cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f103cc00640 1 -- 172.21.15.165:0/1394386709 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c018070 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f104c019040 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f103cc00640 1 -- 172.21.15.165:0/1394386709 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f104c006de0 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f103cc00640 1 -- 172.21.15.165:0/1394386709 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f104c033050 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 10 client.6102 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x55726d0ba980 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f103cc00640 1 -- 172.21.15.165:0/1394386709 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c019040 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f103cc00640 1 -- 172.21.15.165:0/1394386709 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f104c036020 con 0x55726d0b14e0 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 10 client.6102 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 10 client.6102 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:47.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 20 client.6102 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:47.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d1404e0 0x55726d0b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 4 client.6102 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}] 2024-04-25T10:06:47.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.132+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_0"}) v1 -- 0x55726cffb610 con 0x55726d1404e0 2024-04-25T10:06:47.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.133+0000 7f1053400640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d1404e0 0x55726d0b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.142 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:06:47.136+0000 7fa948c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096c [2,head] /scrub_test_path/ceph-qa-suite/new_file_0 auth v654 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={4826=0-4194304@1} caps={4826=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=4826 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55eaae7b1b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096c:[<0x10000000001/new_file_0 v654>,<0x10000000000/ceph-qa-suite v63>,<0x1/scrub_test_path v26>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:06:47.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.134+0000 7f1053400640 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d1404e0 0x55726d0b40e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=894764b7372f1a41 server_cookie=f741e05faedfcb83 in_seq=0 out_seq=0 2024-04-25T10:06:47.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.134+0000 7f103cc00640 10 client.6102 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f103cc00640 1 -- 172.21.15.165:0/1394386709 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55726cffb610 con 0x55726d1404e0 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f103cc00640 10 client.6102 handle_command_reply: tid=0 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 client.6102 shutdown 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 msgr2=0x55726d13ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d13ffa0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f104c016040 tx=0x7f104c011000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 shutdown_connections 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55726d1404e0 0x55726d0b40e0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55726d0b14e0 0x55726d13ffa0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55726d08ffa0 0x55726d0ef5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 --2- 172.21.15.165:0/1394386709 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55726d08f0e0 0x55726d0ef0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f103cc00640 0 client.6102 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 >> 172.21.15.165:0/1394386709 conn(0x55726d08aeb0 msgr2=0x55726d090570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 shutdown_connections 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.135+0000 7f105d635740 1 -- 172.21.15.165:0/1394386709 wait complete. 2024-04-25T10:06:47.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105d635740 20 client.6102 trim_cache size 0 max 16384 2024-04-25T10:06:47.145 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:47.145 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:47.145 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "7174402d-362e-485d-95f2-5f0b3212fa60", 2024-04-25T10:06:47.145 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:47.145 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f103406b500 msgr2=0x7f103406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f103406b500 0x7f103406d9c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1040033000 tx=0x7f104002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 msgr2=0x7f1054155870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1054155870 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f104400e870 tx=0x7f104400ed30 comp rx=0 tx=0).stop 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 shutdown_connections 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f103406b500 0x7f103406d9c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f105415dba0 0x7f105415e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f105415bf30 0x7f105414d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 --2- 172.21.15.165:0/4284399695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1038004800 0x7f1054155870 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 >> 172.21.15.165:0/4284399695 conn(0x7f105406f500 msgr2=0x7f10540cdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 shutdown_connections 2024-04-25T10:06:47.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.136+0000 7f105a800640 1 -- 172.21.15.165:0/4284399695 wait complete. 2024-04-25T10:06:47.159 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_0' returned '{'return_code': 0, 'scrub_tag': '7174402d-362e-485d-95f2-5f0b3212fa60', 'mode': 'asynchronous'}' 2024-04-25T10:06:47.160 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:47.160 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:47.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.352+0000 7fbe58e00640 1 Processor -- start 2024-04-25T10:06:47.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.352+0000 7fbe58e00640 1 -- start start 2024-04-25T10:06:47.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe58e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbe540ceb20 0x7fbe540c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe58e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe540c5030 0x7fbe5415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe58e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe54145520 con 0x7fbe54145140 2024-04-25T10:06:47.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe58e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe54071270 con 0x7fbe540ceb20 2024-04-25T10:06:47.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe58e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54071550 con 0x7fbe540c5030 2024-04-25T10:06:47.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe52a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbe540ceb20 0x7fbe540c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe52a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbe540ceb20 0x7fbe540c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44432/0 (socket says 172.21.15.165:44432) 2024-04-25T10:06:47.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe52a00640 1 -- 172.21.15.165:0/2106799863 learned_addr learned my addr 172.21.15.165:0/2106799863 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.354+0000 7fbe53e00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe540c5030 0x7fbe5415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.355+0000 7fbe52a00640 1 -- 172.21.15.165:0/2106799863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe540c5030 msgr2=0x7fbe5415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.355+0000 7fbe52a00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe540c5030 0x7fbe5415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.355+0000 7fbe52a00640 1 -- 172.21.15.165:0/2106799863 >> v1:172.21.15.165:6789/0 conn(0x7fbe54145140 legacy=0x7fbe540ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.355+0000 7fbe52a00640 1 -- 172.21.15.165:0/2106799863 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe54071830 con 0x7fbe540ceb20 2024-04-25T10:06:47.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.355+0000 7fbe53e00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe540c5030 0x7fbe5415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.356+0000 7fbe52a00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540ceb20 0x7fbe540c4af0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009870 tx=0x7fbe3c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd7b714990dcc08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.356+0000 7fbe52000640 1 -- 172.21.15.165:0/2106799863 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c037070 con 0x7fbe540ceb20 2024-04-25T10:06:47.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.356+0000 7fbe52000640 1 -- 172.21.15.165:0/2106799863 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c032440 con 0x7fbe540ceb20 2024-04-25T10:06:47.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.356+0000 7fbe52000640 1 -- 172.21.15.165:0/2106799863 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c032810 con 0x7fbe540ceb20 2024-04-25T10:06:47.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 -- 172.21.15.165:0/2106799863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540ceb20 msgr2=0x7fbe540c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540ceb20 0x7fbe540c4af0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c009870 tx=0x7fbe3c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 -- 172.21.15.165:0/2106799863 shutdown_connections 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe540c5030 0x7fbe5415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 --2- 172.21.15.165:0/2106799863 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540ceb20 0x7fbe540c4af0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 -- 172.21.15.165:0/2106799863 >> 172.21.15.165:0/2106799863 conn(0x7fbe54061cc0 msgr2=0x7fbe5406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 -- 172.21.15.165:0/2106799863 shutdown_connections 2024-04-25T10:06:47.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.357+0000 7fbe58e00640 1 -- 172.21.15.165:0/2106799863 wait complete. 2024-04-25T10:06:47.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.359+0000 7fbe58e00640 1 Processor -- start 2024-04-25T10:06:47.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.359+0000 7fbe58e00640 1 -- start start 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe58e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 0x7fbe5415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe58e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe540ceb20 0x7fbe540cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe58e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe54145140 0x7fbe540cd610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe58e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540c5620 con 0x7fbe540c5030 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe58e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe540c54a0 con 0x7fbe540ceb20 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe58e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe5415c4f0 con 0x7fbe54145140 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 0x7fbe5415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 0x7fbe5415b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44446/0 (socket says 172.21.15.165:44446) 2024-04-25T10:06:47.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 -- 172.21.15.165:0/3707659821 learned_addr learned my addr 172.21.15.165:0/3707659821 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 -- 172.21.15.165:0/3707659821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe54145140 msgr2=0x7fbe540cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe54145140 0x7fbe540cd610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 -- 172.21.15.165:0/3707659821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe540ceb20 msgr2=0x7fbe540cd0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.360+0000 7fbe53400640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe540ceb20 0x7fbe540cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe53400640 1 -- 172.21.15.165:0/3707659821 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe540b1170 con 0x7fbe540c5030 2024-04-25T10:06:47.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe53400640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 0x7fbe5415b130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c3f0 tx=0x7fbe4400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe4b400640 1 -- 172.21.15.165:0/3707659821 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe440052d0 con 0x7fbe540c5030 2024-04-25T10:06:47.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe4b400640 1 -- 172.21.15.165:0/3707659821 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe44005490 con 0x7fbe540c5030 2024-04-25T10:06:47.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe540b1450 con 0x7fbe540c5030 2024-04-25T10:06:47.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe4b400640 1 -- 172.21.15.165:0/3707659821 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44007260 con 0x7fbe540c5030 2024-04-25T10:06:47.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.361+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe540b1a10 con 0x7fbe540c5030 2024-04-25T10:06:47.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.363+0000 7fbe4b400640 1 -- 172.21.15.165:0/3707659821 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe44007400 con 0x7fbe540c5030 2024-04-25T10:06:47.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.364+0000 7fbe4b400640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe3006b4d0 0x7fbe3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.365+0000 7fbe4b400640 1 -- 172.21.15.165:0/3707659821 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbe44089940 con 0x7fbe540c5030 2024-04-25T10:06:47.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.365+0000 7fbe52a00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe3006b4d0 0x7fbe3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.365+0000 7fbe5d399740 1 Processor -- start 2024-04-25T10:06:47.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe52a00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe3006b4d0 0x7fbe3006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c034000 tx=0x7fbe3c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 -- start start 2024-04-25T10:06:47.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c7ff5d70 0x5622c7ff7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8032440 0x5622c7ff6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c8020cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c7f59c70 con 0x5622c8020830 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c7f02cf0 con 0x5622c8032440 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.366+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622c7f55d10 con 0x5622c7ff5d70 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c8020cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c8020cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44460/0 (socket says 172.21.15.165:44460) 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 -- 172.21.15.165:0/2764448412 learned_addr learned my addr 172.21.15.165:0/2764448412 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53400640 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c7ff5d70 0x5622c7ff7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 -- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c7ff5d70 msgr2=0x5622c7ff7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c7ff5d70 0x5622c7ff7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 -- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8032440 msgr2=0x5622c7ff6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8032440 0x5622c7ff6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53e00640 1 -- 172.21.15.165:0/2764448412 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c7d387d0 con 0x5622c8020830 2024-04-25T10:06:47.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.367+0000 7fbe53400640 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c7ff5d70 0x5622c7ff7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.368+0000 7fbe53e00640 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c8020cb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c008ed0 tx=0x7fbe4c00d7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7537782944b0c0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.368+0000 7fbe43e00640 1 -- 172.21.15.165:0/2764448412 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00e850 con 0x5622c8020830 2024-04-25T10:06:47.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.368+0000 7fbe43e00640 1 -- 172.21.15.165:0/2764448412 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00e9f0 con 0x5622c8020830 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.368+0000 7fbe43e00640 1 -- 172.21.15.165:0/2764448412 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4c00ede0 con 0x5622c8020830 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.368+0000 7fbe5d399740 1 -- 172.21.15.165:0/2764448412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 msgr2=0x5622c8020cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.368+0000 7fbe5d399740 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c8020cb0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fbe4c008ed0 tx=0x7fbe4c00d7a0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 -- 172.21.15.165:0/2764448412 shutdown_connections 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c8020cb0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8032440 0x5622c7ff6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 --2- 172.21.15.165:0/2764448412 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c7ff5d70 0x5622c7ff7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 -- 172.21.15.165:0/2764448412 >> 172.21.15.165:0/2764448412 conn(0x5622c7fbda30 msgr2=0x5622c803b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 -- 172.21.15.165:0/2764448412 shutdown_connections 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 -- 172.21.15.165:0/2764448412 wait complete. 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.369+0000 7fbe5d399740 1 -- start start 2024-04-25T10:06:47.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.370+0000 7fbe5d399740 1 Processor -- start 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8023a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 0x5622c8020f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c80214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c7ef83a0 con 0x5622c7ff5d70 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c801e380 con 0x5622c8020830 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622c804c810 con 0x5622c8032440 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe5d399740 10 client.0 _ll_register_callbacks cb 0x5622c7f15b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8023a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8023a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44466/0 (socket says 172.21.15.165:44466) 2024-04-25T10:06:47.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 -- 172.21.15.165:0/1613985802 learned_addr learned my addr 172.21.15.165:0/1613985802 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 -- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 msgr2=0x5622c80214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c80214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 -- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 msgr2=0x5622c8020f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 0x5622c8020f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.371+0000 7fbe53400640 1 -- 172.21.15.165:0/1613985802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c7c6e3f0 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe53400640 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8023a70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbe440542c0 tx=0x7fbe44057fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7721870a97cd2afa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe40c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe40c00640 1 -- 172.21.15.165:0/1613985802 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4406f070 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe40c00640 1 -- 172.21.15.165:0/1613985802 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbe4405e070 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe4c00e9f0 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe40c00640 1 -- 172.21.15.165:0/1613985802 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe4405e290 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe5d399740 10 client.6117 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5622c7c2e0f0 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe40c00640 1 -- 172.21.15.165:0/1613985802 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4c00e9f0 con 0x5622c7ff5d70 2024-04-25T10:06:47.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe40c00640 1 -- 172.21.15.165:0/1613985802 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbe440572b0 con 0x5622c7ff5d70 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe5d399740 10 client.6117 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.372+0000 7fbe5d399740 10 client.6117 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.373+0000 7fbe5d399740 20 client.6117 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.373+0000 7fbe5d399740 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8011500 0x5622c8033040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.373+0000 7fbe5d399740 4 client.6117 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.373+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5622c7f671b0 con 0x5622c8011500 2024-04-25T10:06:47.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.373+0000 7fbe52a00640 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8011500 0x5622c8033040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.375+0000 7fbe52a00640 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8011500 0x5622c8033040 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7871def696045a8b server_cookie=78aaf5a65d20f9a6 in_seq=0 out_seq=0 2024-04-25T10:06:47.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.375+0000 7fbe40c00640 10 client.6117 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.376+0000 7fbe40c00640 1 -- 172.21.15.165:0/1613985802 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5622c7f671b0 con 0x5622c8011500 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.376+0000 7fbe40c00640 10 client.6117 handle_command_reply: tid=0 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.376+0000 7fbe5d399740 1 client.6117 shutdown 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 msgr2=0x5622c8023a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8023a70 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbe440542c0 tx=0x7fbe44057fa0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 shutdown_connections 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8011500 0x5622c8033040 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c80214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 0x5622c8020f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 --2- 172.21.15.165:0/1613985802 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8023a70 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 >> 172.21.15.165:0/1613985802 conn(0x5622c7fbda30 msgr2=0x5622c803edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe40c00640 0 client.6117 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 shutdown_connections 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 1 -- 172.21.15.165:0/1613985802 wait complete. 2024-04-25T10:06:47.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.377+0000 7fbe5d399740 20 client.6117 trim_cache size 0 max 16384 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.393+0000 7fbe5d399740 1 Processor -- start 2024-04-25T10:06:47.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.393+0000 7fbe5d399740 1 -- start start 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.394+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8020f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.394+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 0x5622c8021440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.394+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8032880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.394+0000 7fbe5d399740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe54200460 con 0x5622c7ff5d70 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.394+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541ff5e0 con 0x5622c8020830 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.394+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541fea80 con 0x5622c8032440 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8020f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8020f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44472/0 (socket says 172.21.15.165:44472) 2024-04-25T10:06:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 -- 172.21.15.165:0/869273842 learned_addr learned my addr 172.21.15.165:0/869273842 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 -- 172.21.15.165:0/869273842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 msgr2=0x5622c8032880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8032880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 -- 172.21.15.165:0/869273842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 msgr2=0x5622c8021440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 0x5622c8021440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 -- 172.21.15.165:0/869273842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5622c7f8cca0 con 0x5622c7ff5d70 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe53400640 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8020f00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbe44057b60 tx=0x7fbe4405e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f964ae3e9c4a38c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe43400640 1 -- 172.21.15.165:0/869273842 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe44054100 con 0x5622c7ff5d70 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe43400640 1 -- 172.21.15.165:0/869273842 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe440544d0 con 0x5622c7ff5d70 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.395+0000 7fbe43400640 1 -- 172.21.15.165:0/869273842 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4405e290 con 0x5622c7ff5d70 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 -- 172.21.15.165:0/869273842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 msgr2=0x5622c8020f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8020f00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fbe44057b60 tx=0x7fbe4405e000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 -- 172.21.15.165:0/869273842 shutdown_connections 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8032880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c8020830 0x5622c8021440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 --2- 172.21.15.165:0/869273842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c7ff5d70 0x5622c8020f00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 -- 172.21.15.165:0/869273842 >> 172.21.15.165:0/869273842 conn(0x5622c7fbda30 msgr2=0x5622c8035f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 -- 172.21.15.165:0/869273842 shutdown_connections 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.396+0000 7fbe5d399740 1 -- 172.21.15.165:0/869273842 wait complete. 2024-04-25T10:06:47.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.397+0000 7fbe5d399740 1 -- start start 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.397+0000 7fbe5d399740 1 Processor -- start 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c7ff5d70 0x5622c8031b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c80a52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8020f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5622c801b5c0 con 0x5622c8020830 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe541fefe0 con 0x5622c7ff5d70 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5622c805e450 con 0x5622c8032440 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe5d399740 10 client.0 _ll_register_callbacks cb 0x7fbe44057c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c80a52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe52a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c80a52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44476/0 (socket says 172.21.15.165:44476) 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.398+0000 7fbe52a00640 1 -- 172.21.15.165:0/3515978822 learned_addr learned my addr 172.21.15.165:0/3515978822 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe53e00640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8020f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe53400640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c7ff5d70 0x5622c8031b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe52a00640 1 -- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 msgr2=0x5622c8020f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe52a00640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8020f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe52a00640 1 -- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c7ff5d70 msgr2=0x5622c8031b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe52a00640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c7ff5d70 0x5622c8031b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe52a00640 1 -- 172.21.15.165:0/3515978822 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5622c801dcb0 con 0x5622c8020830 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe52a00640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c80a52a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c02a200 tx=0x7fbe3c006a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e5302fc5e86c54e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe40c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe40c00640 1 -- 172.21.15.165:0/3515978822 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c037040 con 0x5622c8020830 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe40c00640 1 -- 172.21.15.165:0/3515978822 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbe3c004440 con 0x5622c8020830 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.399+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe440544d0 con 0x5622c8020830 2024-04-25T10:06:47.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe40c00640 1 -- 172.21.15.165:0/3515978822 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe3c059050 con 0x5622c8020830 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 10 client.6123 fetch_fsmap learned FSMap version 28 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=2+}) v3 -- 0x5622c803a290 con 0x5622c8020830 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe40c00640 1 -- 172.21.15.165:0/3515978822 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe440544d0 con 0x5622c8020830 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe40c00640 1 -- 172.21.15.165:0/3515978822 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 28) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fbe3c05c020 con 0x5622c8020830 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 10 client.6123 fetch_fsmap finished waiting for FSMap version 28 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 10 client.6123 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 20 client.6123 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8021440 0x5622c80a5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 4 client.6123 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.400+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5622c7f61500 con 0x5622c8021440 2024-04-25T10:06:47.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.401+0000 7fbe53400640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8021440 0x5622c80a5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.402+0000 7fbe53400640 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8021440 0x5622c80a5be0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5d597c1dc68ebc9 server_cookie=7cf427da0efa1d14 in_seq=0 out_seq=0 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.402+0000 7fbe40c00640 10 client.6123 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.402+0000 7fbe40c00640 1 -- 172.21.15.165:0/3515978822 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+696 (crc 0 0 0) 0x5622c7f61500 con 0x5622c8021440 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.402+0000 7fbe40c00640 10 client.6123 handle_command_reply: tid=0 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.402+0000 7fbe5d399740 1 client.6123 shutdown 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 msgr2=0x5622c80a52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c80a52a0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c02a200 tx=0x7fbe3c006a70 comp rx=0 tx=0).stop 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 shutdown_connections 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5622c8021440 0x5622c80a5be0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5622c8032440 0x5622c8020f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5622c8020830 0x5622c80a52a0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 --2- 172.21.15.165:0/3515978822 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5622c7ff5d70 0x5622c8031b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe40c00640 0 client.6123 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 >> 172.21.15.165:0/3515978822 conn(0x5622c7fbda30 msgr2=0x5622c7fec380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 shutdown_connections 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 1 -- 172.21.15.165:0/3515978822 wait complete. 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe5d399740 20 client.6123 trim_cache size 0 max 16384 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:06:47.412 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "0b7996f5-fc36-40ee-960d-af90e831e9f9": { 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/", 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "0b7996f5-fc36-40ee-960d-af90e831e9f9", 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "" 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "7174402d-362e-485d-95f2-5f0b3212fa60": { 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0", 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "7174402d-362e-485d-95f2-5f0b3212fa60", 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "" 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "8f1eb115-1ccc-4ca6-ac37-65cd989778e4": { 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml", 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "8f1eb115-1ccc-4ca6-ac37-65cd989778e4", 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "" 2024-04-25T10:06:47.413 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:47.414 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:47.414 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:47.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe3006b4d0 msgr2=0x7fbe3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe58e00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe3006b4d0 0x7fbe3006d990 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c034000 tx=0x7fbe3c02c000 comp rx=0 tx=0).stop 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 msgr2=0x7fbe5415b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.403+0000 7fbe58e00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 0x7fbe5415b130 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400c3f0 tx=0x7fbe4400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 shutdown_connections 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe3006b4d0 0x7fbe3006d990 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe54145140 0x7fbe540cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe540ceb20 0x7fbe540cd0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 --2- 172.21.15.165:0/3707659821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe540c5030 0x7fbe5415b130 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 >> 172.21.15.165:0/3707659821 conn(0x7fbe54061cc0 msgr2=0x7fbe540b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 shutdown_connections 2024-04-25T10:06:47.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:47.404+0000 7fbe58e00640 1 -- 172.21.15.165:0/3707659821 wait complete. 2024-04-25T10:06:51.685 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:06:51.679+0000 7f5484c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:06:52.428 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:52.428 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:52.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.629+0000 7ff07d800640 1 Processor -- start 2024-04-25T10:06:52.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.629+0000 7ff07d800640 1 -- start start 2024-04-25T10:06:52.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff07d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0780cd8a0 0x7ff07815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff07d800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff078150b90 0x7ff078151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff07d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff07815e6e0 con 0x7ff0780cd4c0 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff07d800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff078151560 con 0x7ff078150b90 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff07d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815e9c0 con 0x7ff0780cd8a0 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff077400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff078150b90 0x7ff078151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff077400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff078150b90 0x7ff078151020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44478/0 (socket says 172.21.15.165:44478) 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff077400640 1 -- 172.21.15.165:0/676502140 learned_addr learned my addr 172.21.15.165:0/676502140 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff06e000640 1 --2- 172.21.15.165:0/676502140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0780cd8a0 0x7ff07815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.631+0000 7ff077400640 1 -- 172.21.15.165:0/676502140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0780cd8a0 msgr2=0x7ff07815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.632+0000 7ff077400640 1 --2- 172.21.15.165:0/676502140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0780cd8a0 0x7ff07815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.632+0000 7ff077400640 1 -- 172.21.15.165:0/676502140 >> v1:172.21.15.172:6789/0 conn(0x7ff0780cd4c0 legacy=0x7ff07815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.632+0000 7ff077400640 1 -- 172.21.15.165:0/676502140 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff07815a9c0 con 0x7ff078150b90 2024-04-25T10:06:52.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.633+0000 7ff077400640 1 --2- 172.21.15.165:0/676502140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff078150b90 0x7ff078151020 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff07000d150 tx=0x7ff07002c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=391d03f8db6c4edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.633+0000 7ff076000640 1 -- 172.21.15.165:0/676502140 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070035400 con 0x7ff078150b90 2024-04-25T10:06:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.633+0000 7ff076000640 1 -- 172.21.15.165:0/676502140 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0700355a0 con 0x7ff078150b90 2024-04-25T10:06:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.633+0000 7ff076000640 1 -- 172.21.15.165:0/676502140 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff070035970 con 0x7ff078150b90 2024-04-25T10:06:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 -- 172.21.15.165:0/676502140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff078150b90 msgr2=0x7ff078151020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 --2- 172.21.15.165:0/676502140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff078150b90 0x7ff078151020 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7ff07000d150 tx=0x7ff07002c8e0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 -- 172.21.15.165:0/676502140 shutdown_connections 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 --2- 172.21.15.165:0/676502140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff078150b90 0x7ff078151020 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 --2- 172.21.15.165:0/676502140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0780cd8a0 0x7ff07815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 -- 172.21.15.165:0/676502140 >> 172.21.15.165:0/676502140 conn(0x7ff07806abe0 msgr2=0x7ff07806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 -- 172.21.15.165:0/676502140 shutdown_connections 2024-04-25T10:06:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.634+0000 7ff07d800640 1 -- 172.21.15.165:0/676502140 wait complete. 2024-04-25T10:06:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.636+0000 7ff07d800640 1 Processor -- start 2024-04-25T10:06:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.636+0000 7ff07d800640 1 -- start start 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff07d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 0x7ff078141f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff07d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0780cd8a0 0x7ff0780b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff07d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff078150b90 0x7ff07806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff07d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815e370 con 0x7ff0780cd4c0 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff07d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff07815a670 con 0x7ff0780cd8a0 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff07d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff07806eca0 con 0x7ff078150b90 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff076a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 0x7ff078141f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff076a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 0x7ff078141f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44490/0 (socket says 172.21.15.165:44490) 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.637+0000 7ff076a00640 1 -- 172.21.15.165:0/3660867253 learned_addr learned my addr 172.21.15.165:0/3660867253 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff077400640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff078150b90 0x7ff07806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff06e000640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0780cd8a0 0x7ff0780b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff076a00640 1 -- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff078150b90 msgr2=0x7ff07806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff076a00640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff078150b90 0x7ff07806d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff076a00640 1 -- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0780cd8a0 msgr2=0x7ff0780b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff076a00640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0780cd8a0 0x7ff0780b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff076a00640 1 -- 172.21.15.165:0/3660867253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0781424c0 con 0x7ff0780cd4c0 2024-04-25T10:06:52.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff076a00640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 0x7ff078141f80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c460 tx=0x7ff06800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff06f400640 1 -- 172.21.15.165:0/3660867253 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068016020 con 0x7ff0780cd4c0 2024-04-25T10:06:52.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff06f400640 1 -- 172.21.15.165:0/3660867253 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff068012870 con 0x7ff0780cd4c0 2024-04-25T10:06:52.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff06f400640 1 -- 172.21.15.165:0/3660867253 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068012c60 con 0x7ff0780cd4c0 2024-04-25T10:06:52.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0780b1ab0 con 0x7ff0780cd4c0 2024-04-25T10:06:52.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.638+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0780b2070 con 0x7ff0780cd4c0 2024-04-25T10:06:52.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.641+0000 7ff06f400640 1 -- 172.21.15.165:0/3660867253 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff068019020 con 0x7ff0780cd4c0 2024-04-25T10:06:52.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.641+0000 7ff06f400640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff05406b500 0x7ff05406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.642+0000 7ff06f400640 1 -- 172.21.15.165:0/3660867253 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff06808a100 con 0x7ff0780cd4c0 2024-04-25T10:06:52.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.642+0000 7ff06e000640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff05406b500 0x7ff05406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.642+0000 7ff080795740 1 Processor -- start 2024-04-25T10:06:52.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.642+0000 7ff080795740 1 -- start start 2024-04-25T10:06:52.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.642+0000 7ff06e000640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff05406b500 0x7ff05406d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff078148f00 tx=0x7ff060008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.643+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62d9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.643+0000 7ff080795740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc62d7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.643+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6302af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.643+0000 7ff080795740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc623bc70 con 0x55cfc6314280 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.643+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc61e4cf0 con 0x55cfc6302670 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.643+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfc6237d10 con 0x55cfc62d7bb0 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc62d7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc62d7f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44502/0 (socket says 172.21.15.165:44502) 2024-04-25T10:06:52.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 -- 172.21.15.165:0/1376087256 learned_addr learned my addr 172.21.15.165:0/1376087256 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff076a00640 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62d9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 -- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 msgr2=0x55cfc62d9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62d9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff077400640 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6302af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 -- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 msgr2=0x55cfc6302af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6302af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 -- 172.21.15.165:0/1376087256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfc601a7d0 con 0x55cfc6314280 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff076a00640 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62d9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.644+0000 7ff06e000640 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc62d7f90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e840 tx=0x7ff060011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2557298fe5771b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff067400640 1 -- 172.21.15.165:0/1376087256 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060025070 con 0x55cfc6314280 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff067400640 1 -- 172.21.15.165:0/1376087256 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060015640 con 0x55cfc6314280 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff067400640 1 -- 172.21.15.165:0/1376087256 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06000f040 con 0x55cfc6314280 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff080795740 1 -- 172.21.15.165:0/1376087256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 msgr2=0x55cfc62d7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff080795740 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc62d7f90 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7ff06000e840 tx=0x7ff060011000 comp rx=0 tx=0).stop 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff080795740 1 -- 172.21.15.165:0/1376087256 shutdown_connections 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.645+0000 7ff080795740 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6302af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.646+0000 7ff080795740 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc62d7f90 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.646+0000 7ff080795740 1 --2- 172.21.15.165:0/1376087256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62d9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.646+0000 7ff080795740 1 -- 172.21.15.165:0/1376087256 >> 172.21.15.165:0/1376087256 conn(0x55cfc629fa30 msgr2=0x55cfc631d670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.646+0000 7ff080795740 1 -- 172.21.15.165:0/1376087256 shutdown_connections 2024-04-25T10:06:52.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.646+0000 7ff080795740 1 -- 172.21.15.165:0/1376087256 wait complete. 2024-04-25T10:06:52.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.646+0000 7ff080795740 1 -- start start 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.647+0000 7ff080795740 1 Processor -- start 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.647+0000 7ff080795740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 0x55cfc631fbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.647+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc6302670 0x55cfc6315b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 0x55cfc62eea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff080795740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc61da3a0 con 0x55cfc62d7bb0 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc63001c0 con 0x55cfc6314280 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfc62f0750 con 0x55cfc6302670 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff080795740 10 client.0 _ll_register_callbacks cb 0x55cfc61f7b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 0x55cfc631fbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 0x55cfc631fbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44504/0 (socket says 172.21.15.165:44504) 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 -- 172.21.15.165:0/1418053375 learned_addr learned my addr 172.21.15.165:0/1418053375 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff06e000640 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc6302670 0x55cfc6315b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 -- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc6302670 msgr2=0x55cfc6315b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc6302670 0x55cfc6315b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 -- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 msgr2=0x55cfc62eea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 0x55cfc62eea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 -- 172.21.15.165:0/1418053375 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfc5f503f0 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff076a00640 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 0x55cfc631fbd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff068058640 tx=0x7ff068058770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70f266b730a311ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.648+0000 7ff05be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff05be00640 1 -- 172.21.15.165:0/1418053375 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff068086060 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff05be00640 1 -- 172.21.15.165:0/1418053375 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff0680701d0 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff060015640 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff05be00640 1 -- 172.21.15.165:0/1418053375 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff068070030 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 10 client.6135 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55cfc5f100f0 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff05be00640 1 -- 172.21.15.165:0/1418053375 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff060015640 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff05be00640 1 -- 172.21.15.165:0/1418053375 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff068054d20 con 0x55cfc62d7bb0 2024-04-25T10:06:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 10 client.6135 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:52.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 10 client.6135 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:52.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 20 client.6135 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:52.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc63160a0 0x55cfc62ef8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 4 client.6135 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:52.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.649+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cfc62491b0 con 0x55cfc63160a0 2024-04-25T10:06:52.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.650+0000 7ff06e000640 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc63160a0 0x55cfc62ef8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.651+0000 7ff06e000640 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc63160a0 0x55cfc62ef8c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f951c5ab69cdb7f3 server_cookie=b323b2030c6bc2cf in_seq=0 out_seq=0 2024-04-25T10:06:52.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.651+0000 7ff05be00640 10 client.6135 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff05be00640 1 -- 172.21.15.165:0/1418053375 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cfc62491b0 con 0x55cfc63160a0 2024-04-25T10:06:52.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff05be00640 10 client.6135 handle_command_reply: tid=0 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 client.6135 shutdown 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 msgr2=0x55cfc631fbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 0x55cfc631fbd0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7ff068058640 tx=0x7ff068058770 comp rx=0 tx=0).stop 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 shutdown_connections 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc63160a0 0x55cfc62ef8c0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 0x55cfc62eea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc6302670 0x55cfc6315b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 --2- 172.21.15.165:0/1418053375 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc62d7bb0 0x55cfc631fbd0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff05be00640 0 client.6135 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 >> 172.21.15.165:0/1418053375 conn(0x55cfc629fa30 msgr2=0x55cfc63209d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 shutdown_connections 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.653+0000 7ff080795740 1 -- 172.21.15.165:0/1418053375 wait complete. 2024-04-25T10:06:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.654+0000 7ff080795740 20 client.6135 trim_cache size 0 max 16384 2024-04-25T10:06:52.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.673+0000 7ff080795740 1 Processor -- start 2024-04-25T10:06:52.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.674+0000 7ff080795740 1 -- start start 2024-04-25T10:06:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62ee7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6303b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff080795740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc63040a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff080795740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff078200570 con 0x55cfc6314280 2024-04-25T10:06:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0781ff760 con 0x55cfc6302670 2024-04-25T10:06:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0781f7800 con 0x55cfc62d7bb0 2024-04-25T10:06:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc63040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc63040a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44510/0 (socket says 172.21.15.165:44510) 2024-04-25T10:06:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 -- 172.21.15.165:0/3855632154 learned_addr learned my addr 172.21.15.165:0/3855632154 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff06e000640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6303b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff076a00640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62ee7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 -- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 msgr2=0x55cfc62ee7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62ee7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 -- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 msgr2=0x55cfc6303b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6303b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff077400640 1 -- 172.21.15.165:0/3855632154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cfc62d77c0 con 0x55cfc6314280 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff06e000640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6303b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.675+0000 7ff076a00640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62ee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.676+0000 7ff077400640 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc63040a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff07002ced0 tx=0x7ff07002f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=106b672934d61f58 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.676+0000 7ff066a00640 1 -- 172.21.15.165:0/3855632154 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07000b0c0 con 0x55cfc6314280 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.676+0000 7ff066a00640 1 -- 172.21.15.165:0/3855632154 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07000b260 con 0x55cfc6314280 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.676+0000 7ff066a00640 1 -- 172.21.15.165:0/3855632154 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07003e420 con 0x55cfc6314280 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 -- 172.21.15.165:0/3855632154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 msgr2=0x55cfc63040a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc63040a0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7ff07002ced0 tx=0x7ff07002f000 comp rx=0 tx=0).stop 2024-04-25T10:06:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 -- 172.21.15.165:0/3855632154 shutdown_connections 2024-04-25T10:06:52.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6314280 0x55cfc63040a0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6302670 0x55cfc6303b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 --2- 172.21.15.165:0/3855632154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc62ee7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 -- 172.21.15.165:0/3855632154 >> 172.21.15.165:0/3855632154 conn(0x55cfc629fa30 msgr2=0x55cfc6340490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 -- 172.21.15.165:0/3855632154 shutdown_connections 2024-04-25T10:06:52.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.677+0000 7ff080795740 1 -- 172.21.15.165:0/3855632154 wait complete. 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.678+0000 7ff080795740 1 -- start start 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.678+0000 7ff080795740 1 Processor -- start 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc632fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 0x55cfc6304700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 0x55cfc6304c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cfc62efa20 con 0x55cfc6302670 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0781ff160 con 0x55cfc6314280 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cfc6388e80 con 0x55cfc62d7bb0 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff080795740 10 client.0 _ll_register_callbacks cb 0x7ff068058db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff06e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 0x55cfc6304700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff06e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 0x55cfc6304700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44526/0 (socket says 172.21.15.165:44526) 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff06e000640 1 -- 172.21.15.165:0/3697717754 learned_addr learned my addr 172.21.15.165:0/3697717754 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.679+0000 7ff076a00640 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc632fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff06e000640 1 -- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 msgr2=0x55cfc632fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff06e000640 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc632fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff06e000640 1 -- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 msgr2=0x55cfc6304c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff06e000640 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 0x55cfc6304c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff06e000640 1 -- 172.21.15.165:0/3697717754 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cfc62d96d0 con 0x55cfc6302670 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff06e000640 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 0x55cfc6304700 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff060020000 tx=0x7ff060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def8697a11cba21d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff05be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff05be00640 1 -- 172.21.15.165:0/3697717754 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff060034070 con 0x55cfc6302670 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff05be00640 1 -- 172.21.15.165:0/3697717754 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff0600158d0 con 0x55cfc6302670 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff07000b260 con 0x55cfc6302670 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.680+0000 7ff05be00640 1 -- 172.21.15.165:0/3697717754 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff06003c050 con 0x55cfc6302670 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 10 client.6141 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:52.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55cfc6330ac0 con 0x55cfc6302670 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff05be00640 1 -- 172.21.15.165:0/3697717754 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07000b260 con 0x55cfc6302670 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff05be00640 1 -- 172.21.15.165:0/3697717754 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff06003f020 con 0x55cfc6302670 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 10 client.6141 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 10 client.6141 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 20 client.6141 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc6388670 0x55cfc6302a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 4 client.6141 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.681+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cfc6243500 con 0x55cfc6388670 2024-04-25T10:06:52.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.682+0000 7ff076a00640 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc6388670 0x55cfc6302a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.682+0000 7ff076a00640 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc6388670 0x55cfc6302a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e70155bc754f387 server_cookie=7f3b7061b1b26d15 in_seq=0 out_seq=0 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.682+0000 7ff05be00640 10 client.6141 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff05be00640 1 -- 172.21.15.165:0/3697717754 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55cfc6243500 con 0x55cfc6388670 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff05be00640 10 client.6141 handle_command_reply: tid=0 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 client.6141 shutdown 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 msgr2=0x55cfc6304700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 0x55cfc6304700 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7ff060020000 tx=0x7ff060004730 comp rx=0 tx=0).stop 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 shutdown_connections 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55cfc6388670 0x55cfc6302a50 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cfc6314280 0x55cfc6304c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cfc6302670 0x55cfc6304700 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 --2- 172.21.15.165:0/3697717754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cfc62d7bb0 0x55cfc632fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff05be00640 0 client.6141 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.683+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 >> 172.21.15.165:0/3697717754 conn(0x55cfc629fa30 msgr2=0x55cfc6316b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 shutdown_connections 2024-04-25T10:06:52.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff080795740 1 -- 172.21.15.165:0/3697717754 wait complete. 2024-04-25T10:06:52.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff080795740 20 client.6141 trim_cache size 0 max 16384 2024-04-25T10:06:52.693 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:52.693 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:06:52.693 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:06:52.693 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff05406b500 msgr2=0x7ff05406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff07d800640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff05406b500 0x7ff05406d9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff078148f00 tx=0x7ff060008040 comp rx=0 tx=0).stop 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 msgr2=0x7ff078141f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.684+0000 7ff07d800640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 0x7ff078141f80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff06800c460 tx=0x7ff06800c920 comp rx=0 tx=0).stop 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 shutdown_connections 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff05406b500 0x7ff05406d9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff078150b90 0x7ff07806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0780cd8a0 0x7ff0780b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 --2- 172.21.15.165:0/3660867253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0780cd4c0 0x7ff078141f80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 >> 172.21.15.165:0/3660867253 conn(0x7ff07806abe0 msgr2=0x7ff0781599c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 shutdown_connections 2024-04-25T10:06:52.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.685+0000 7ff07d800640 1 -- 172.21.15.165:0/3660867253 wait complete. 2024-04-25T10:06:52.708 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:06:52.708 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:52.708 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:52.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.910+0000 7f4256800640 1 Processor -- start 2024-04-25T10:06:52.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.910+0000 7f4256800640 1 -- start start 2024-04-25T10:06:52.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.911+0000 7f4256800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 0x7f425006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.911+0000 7f4256800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500cfd80 con 0x7f425015b710 2024-04-25T10:06:52.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.911+0000 7f4256800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42500d0060 con 0x7f42500ac8b0 2024-04-25T10:06:52.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.911+0000 7f4256800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425006f190 con 0x7f42500cf640 2024-04-25T10:06:52.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.912+0000 7f424f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f425015b710 0x7f425015e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54118/0 (socket says 172.21.15.165:54118) 2024-04-25T10:06:52.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.912+0000 7f424f400640 1 -- 172.21.15.165:0/3722942352 learned_addr learned my addr 172.21.15.165:0/3722942352 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:52.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.912+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3726524850 0 0) 0x7f42500cfd80 con 0x7f425015b710 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.912+0000 7f4247e00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 0x7f425006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.912+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4234003520 con 0x7f425015b710 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.912+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842613384 0 0) 0x7f4234003520 con 0x7f425015b710 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 msgr2=0x7f425006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 0x7f425006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 >> v1:172.21.15.172:6789/0 conn(0x7f42500ac8b0 legacy=0x7f4250071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42500d0340 con 0x7f425015b710 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4240001670 con 0x7f425015b710 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 >> v1:172.21.15.165:6789/0 conn(0x7f425015b710 legacy=0x7f425015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42340039e0 0x7f4234003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 0x7f4234004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4234004d40 0x7f42340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425006f190 con 0x7f4234004d40 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f423400bad0 con 0x7f42340039e0 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f423400bcd0 con 0x7f42500cf640 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424fe00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4234004d40 0x7f42340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f4247e00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 0x7f4234004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.913+0000 7f424f400640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42340039e0 0x7f4234003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.914+0000 7f424fe00640 1 -- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 msgr2=0x7f4234004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.914+0000 7f424fe00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500cf640 0x7f4234004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.914+0000 7f424fe00640 1 -- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42340039e0 msgr2=0x7f4234003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.914+0000 7f424fe00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42340039e0 0x7f4234003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.914+0000 7f424fe00640 1 -- 172.21.15.165:0/3722942352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4234003520 con 0x7f4234004d40 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.915+0000 7f424fe00640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4234004d40 0x7f42340051c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4248001e20 tx=0x7f4234005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5377ca0c6d062954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.915+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248001a70 con 0x7f4234004d40 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.915+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424802ae40 con 0x7f4234004d40 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.915+0000 7f424ea00640 1 -- 172.21.15.165:0/3722942352 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248003bb0 con 0x7f4234004d40 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f4256800640 1 -- 172.21.15.165:0/3722942352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4234004d40 msgr2=0x7f42340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f4256800640 1 --2- 172.21.15.165:0/3722942352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4234004d40 0x7f42340051c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f4248001e20 tx=0x7f4234005830 comp rx=0 tx=0).stop 2024-04-25T10:06:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f424fe00640 1 -- 172.21.15.165:0/3722942352 reap_dead start 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f4256800640 1 -- 172.21.15.165:0/3722942352 shutdown_connections 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f4256800640 1 -- 172.21.15.165:0/3722942352 >> 172.21.15.165:0/3722942352 conn(0x7f425006abe0 msgr2=0x7f425006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f4256800640 1 -- 172.21.15.165:0/3722942352 shutdown_connections 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.916+0000 7f4256800640 1 -- 172.21.15.165:0/3722942352 wait complete. 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.918+0000 7f4256800640 1 Processor -- start 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.918+0000 7f4256800640 1 -- start start 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4256800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4234004d40 0x7f4250156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4256800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 0x7f4250146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4256800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500af2a0 0x7f4250146da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4256800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425015baf0 con 0x7f42500aee20 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4256800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425015e490 con 0x7f4234004d40 2024-04-25T10:06:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4256800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425015bc70 con 0x7f42500af2a0 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 0x7f4250146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 0x7f4250146860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44544/0 (socket says 172.21.15.165:44544) 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 -- 172.21.15.165:0/2425966840 learned_addr learned my addr 172.21.15.165:0/2425966840 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f424f400640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4234004d40 0x7f4250156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f424fe00640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500af2a0 0x7f4250146da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 -- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500af2a0 msgr2=0x7f4250146da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500af2a0 0x7f4250146da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 -- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4234004d40 msgr2=0x7f4250156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4234004d40 0x7f4250156840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.919+0000 7f4247e00640 1 -- 172.21.15.165:0/2425966840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425015f460 con 0x7f42500aee20 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.920+0000 7f4247e00640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 0x7f4250146860 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4238011600 tx=0x7f4238011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.920+0000 7f4247400640 1 -- 172.21.15.165:0/2425966840 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800f8c0 con 0x7f42500aee20 2024-04-25T10:06:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.920+0000 7f4247400640 1 -- 172.21.15.165:0/2425966840 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f423800fa60 con 0x7f42500aee20 2024-04-25T10:06:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.920+0000 7f4247400640 1 -- 172.21.15.165:0/2425966840 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f423800fe10 con 0x7f42500aee20 2024-04-25T10:06:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.920+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425015f740 con 0x7f42500aee20 2024-04-25T10:06:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.920+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425015fd00 con 0x7f42500aee20 2024-04-25T10:06:52.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.923+0000 7f4247400640 1 -- 172.21.15.165:0/2425966840 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4238015460 con 0x7f42500aee20 2024-04-25T10:06:52.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.924+0000 7f4247400640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.924+0000 7f4247400640 1 -- 172.21.15.165:0/2425966840 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f423808ace0 con 0x7f42500aee20 2024-04-25T10:06:52.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.924+0000 7f424f400640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.925+0000 7f425962e740 1 Processor -- start 2024-04-25T10:06:52.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.925+0000 7f425962e740 1 -- start start 2024-04-25T10:06:52.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.925+0000 7f424f400640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f422c06b4d0 0x7f422c06d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4240000ff0 tx=0x7f4240001430 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f425962e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9be200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9dbc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d993b00 0x56139d9d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f425962e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139d8f6d00 con 0x56139d9bde20 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139d89ff50 con 0x56139d993b00 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56139d8f2da0 con 0x56139d9db6c0 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f424f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9be200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f424f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9be200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44558/0 (socket says 172.21.15.165:44558) 2024-04-25T10:06:52.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.926+0000 7f424f400640 1 -- 172.21.15.165:0/4098478919 learned_addr learned my addr 172.21.15.165:0/4098478919 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f4247e00640 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9dbc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f424f400640 1 -- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 msgr2=0x56139d9dbc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f424f400640 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9dbc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f424f400640 1 -- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d993b00 msgr2=0x56139d9d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f424f400640 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d993b00 0x56139d9d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f424f400640 1 -- 172.21.15.165:0/4098478919 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56139d8c3130 con 0x56139d9bde20 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f424f400640 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9be200 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f424000fa80 tx=0x7f4240012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d646d89531528800 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f423fe00640 1 -- 172.21.15.165:0/4098478919 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240010040 con 0x56139d9bde20 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f423fe00640 1 -- 172.21.15.165:0/4098478919 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240016110 con 0x56139d9bde20 2024-04-25T10:06:52.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.927+0000 7f423fe00640 1 -- 172.21.15.165:0/4098478919 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4240016b70 con 0x56139d9bde20 2024-04-25T10:06:52.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 -- 172.21.15.165:0/4098478919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 msgr2=0x56139d9be200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9be200 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f424000fa80 tx=0x7f4240012000 comp rx=0 tx=0).stop 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 -- 172.21.15.165:0/4098478919 shutdown_connections 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d993b00 0x56139d9d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9dbc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 --2- 172.21.15.165:0/4098478919 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9be200 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 -- 172.21.15.165:0/4098478919 >> 172.21.15.165:0/4098478919 conn(0x56139d990430 msgr2=0x56139d9d8c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 -- 172.21.15.165:0/4098478919 shutdown_connections 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.928+0000 7f425962e740 1 -- 172.21.15.165:0/4098478919 wait complete. 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.929+0000 7f425962e740 1 -- start start 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.929+0000 7f425962e740 1 Processor -- start 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9bde20 0x56139d9d0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 0x56139d9c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139d9948a0 con 0x56139d9d1490 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139d895530 con 0x56139d9bde20 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56139d98e360 con 0x56139d9db6c0 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f425962e740 10 client.0 _ll_register_callbacks cb 0x56139d8ee430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.930+0000 7f424fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 0x56139d9c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 0x56139d9c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44560/0 (socket says 172.21.15.165:44560) 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 -- 172.21.15.165:0/1980552642 learned_addr learned my addr 172.21.15.165:0/1980552642 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424f400640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9bde20 0x56139d9d0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 -- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 msgr2=0x56139d9c0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f4247e00640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9c0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 -- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9bde20 msgr2=0x56139d9d0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9bde20 0x56139d9d0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 -- 172.21.15.165:0/1980552642 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56139d8bddd0 con 0x56139d9d1490 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424f400640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9bde20 0x56139d9d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f424fe00640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 0x56139d9c10b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4248001e20 tx=0x7f42480014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd975371fd74e8bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.931+0000 7f423cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f423cc00640 1 -- 172.21.15.165:0/1980552642 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248037040 con 0x56139d9d1490 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f423cc00640 1 -- 172.21.15.165:0/1980552642 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4248006c10 con 0x56139d9d1490 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4240016110 con 0x56139d9d1490 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f423cc00640 1 -- 172.21.15.165:0/1980552642 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4248047050 con 0x56139d9d1490 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 10 client.6153 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56139d60b3f0 con 0x56139d9d1490 2024-04-25T10:06:52.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f423cc00640 1 -- 172.21.15.165:0/1980552642 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4240016110 con 0x56139d9d1490 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f423cc00640 1 -- 172.21.15.165:0/1980552642 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f424804a020 con 0x56139d9d1490 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 10 client.6153 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 10 client.6153 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 20 client.6153 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9d01b0 0x56139d9d05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 4 client.6153 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.932+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56139d904240 con 0x56139d9d01b0 2024-04-25T10:06:52.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.933+0000 7f424f400640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9d01b0 0x56139d9d05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.934+0000 7f424f400640 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9d01b0 0x56139d9d05c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3ef860a74d6d3e2 server_cookie=30e072457b5b4e06 in_seq=0 out_seq=0 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.934+0000 7f423cc00640 10 client.6153 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f423cc00640 1 -- 172.21.15.165:0/1980552642 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56139d904240 con 0x56139d9d01b0 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f423cc00640 10 client.6153 handle_command_reply: tid=0 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 client.6153 shutdown 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 msgr2=0x56139d9c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 0x56139d9c10b0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f4248001e20 tx=0x7f42480014a0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 shutdown_connections 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9d01b0 0x56139d9d05c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9d1490 0x56139d9c10b0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9db6c0 0x56139d9c0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 --2- 172.21.15.165:0/1980552642 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9bde20 0x56139d9d0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f423cc00640 0 client.6153 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 >> 172.21.15.165:0/1980552642 conn(0x56139d993b00 msgr2=0x56139d9dca40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 shutdown_connections 2024-04-25T10:06:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 1 -- 172.21.15.165:0/1980552642 wait complete. 2024-04-25T10:06:52.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.936+0000 7f425962e740 20 client.6153 trim_cache size 0 max 16384 2024-04-25T10:06:52.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.949+0000 7f425962e740 1 Processor -- start 2024-04-25T10:06:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.949+0000 7f425962e740 1 -- start start 2024-04-25T10:06:52.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.950+0000 7f425962e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9d0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.950+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9d1490 0x56139d9da870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.950+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9db6c0 0x56139d9ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.950+0000 7f425962e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4250200d20 con 0x56139d9bde20 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.950+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501fff30 con 0x56139d9db6c0 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.950+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42500b1970 con 0x56139d9d1490 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9d0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9d0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44576/0 (socket says 172.21.15.165:44576) 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 -- 172.21.15.165:0/4017889840 learned_addr learned my addr 172.21.15.165:0/4017889840 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 -- 172.21.15.165:0/4017889840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9d1490 msgr2=0x56139d9da870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9d1490 0x56139d9da870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 -- 172.21.15.165:0/4017889840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9db6c0 msgr2=0x56139d9ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9db6c0 0x56139d9ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 -- 172.21.15.165:0/4017889840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56139d6d57d0 con 0x56139d9bde20 2024-04-25T10:06:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f424f400640 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9d0b00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4240016820 tx=0x7f4240007030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69854112b0d2fb54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f423f400640 1 -- 172.21.15.165:0/4017889840 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424002d3e0 con 0x56139d9bde20 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f423f400640 1 -- 172.21.15.165:0/4017889840 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424002d580 con 0x56139d9bde20 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.951+0000 7f423f400640 1 -- 172.21.15.165:0/4017889840 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42400168d0 con 0x56139d9bde20 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 -- 172.21.15.165:0/4017889840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 msgr2=0x56139d9d0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9d0b00 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4240016820 tx=0x7f4240007030 comp rx=0 tx=0).stop 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 -- 172.21.15.165:0/4017889840 shutdown_connections 2024-04-25T10:06:52.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9db6c0 0x56139d9ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9d1490 0x56139d9da870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 --2- 172.21.15.165:0/4017889840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9bde20 0x56139d9d0b00 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 -- 172.21.15.165:0/4017889840 >> 172.21.15.165:0/4017889840 conn(0x56139d993b00 msgr2=0x56139d9d7570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 -- 172.21.15.165:0/4017889840 shutdown_connections 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.952+0000 7f425962e740 1 -- 172.21.15.165:0/4017889840 wait complete. 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.953+0000 7f425962e740 1 -- start start 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.953+0000 7f425962e740 1 Processor -- start 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9bde20 0x56139da42db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9d1490 0x56139d9e91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 0x56139d992a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42501ff930 con 0x56139d9db6c0 2024-04-25T10:06:52.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56139d904240 con 0x56139d9d1490 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56139d9e9a80 con 0x56139d9bde20 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f425962e740 10 client.0 _ll_register_callbacks cb 0x56139d98e840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f424fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 0x56139d992a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f424fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 0x56139d992a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44586/0 (socket says 172.21.15.165:44586) 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.954+0000 7f424fe00640 1 -- 172.21.15.165:0/1210576418 learned_addr learned my addr 172.21.15.165:0/1210576418 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f424fe00640 1 -- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9bde20 msgr2=0x56139da42db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f424fe00640 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9bde20 0x56139da42db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f4247e00640 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9d1490 0x56139d9e91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f424fe00640 1 -- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9d1490 msgr2=0x56139d9e91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f424fe00640 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9d1490 0x56139d9e91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f424fe00640 1 -- 172.21.15.165:0/1210576418 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56139d9f2dc0 con 0x56139d9db6c0 2024-04-25T10:06:52.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f424fe00640 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 0x56139d992a80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x56139d9cf730 tx=0x7f4248001610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7f52d2982fe03f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:52.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f423cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:52.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f423cc00640 1 -- 172.21.15.165:0/1210576418 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4248050060 con 0x56139d9db6c0 2024-04-25T10:06:52.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f423cc00640 1 -- 172.21.15.165:0/1210576418 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f424802a980 con 0x56139d9db6c0 2024-04-25T10:06:52.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.955+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f424002d580 con 0x56139d9db6c0 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f423cc00640 1 -- 172.21.15.165:0/1210576418 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4248056050 con 0x56139d9db6c0 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 10 client.6159 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56139d9c1ce0 con 0x56139d9db6c0 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f423cc00640 1 -- 172.21.15.165:0/1210576418 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424002d580 con 0x56139d9db6c0 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f423cc00640 1 -- 172.21.15.165:0/1210576418 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4248059020 con 0x56139d9db6c0 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 10 client.6159 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 10 client.6159 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 20 client.6159 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9ced90 0x56139d9cf1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 4 client.6159 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:52.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.956+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56139d8fe590 con 0x56139d9ced90 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.957+0000 7f424f400640 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9ced90 0x56139d9cf1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f424f400640 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9ced90 0x56139d9cf1a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f6ca29ccc9e0c54 server_cookie=405c34594971f557 in_seq=0 out_seq=0 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f423cc00640 10 client.6159 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f423cc00640 1 -- 172.21.15.165:0/1210576418 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x56139d8fe590 con 0x56139d9ced90 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f423cc00640 10 client.6159 handle_command_reply: tid=0 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 client.6159 shutdown 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 msgr2=0x56139d992a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 0x56139d992a80 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x56139d9cf730 tx=0x7f4248001610 comp rx=0 tx=0).stop 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 shutdown_connections 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56139d9ced90 0x56139d9cf1a0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56139d9db6c0 0x56139d992a80 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56139d9d1490 0x56139d9e91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 --2- 172.21.15.165:0/1210576418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56139d9bde20 0x56139da42db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 >> 172.21.15.165:0/1210576418 conn(0x56139d993b00 msgr2=0x56139d9da870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.958+0000 7f423cc00640 0 client.6159 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 shutdown_connections 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f425962e740 1 -- 172.21.15.165:0/1210576418 wait complete. 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f425962e740 20 client.6159 trim_cache size 0 max 16384 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:06:52.969 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:06:52.970 INFO:teuthology.orchestra.run.smithi165.stdout: "damage_type": "backtrace", 2024-04-25T10:06:52.970 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 3783141116, 2024-04-25T10:06:52.970 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 1099511630188, 2024-04-25T10:06:52.970 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_0" 2024-04-25T10:06:52.970 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:52.970 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f422c06b4d0 msgr2=0x7f422c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f422c06b4d0 0x7f422c06d990 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f4240000ff0 tx=0x7f4240001430 comp rx=0 tx=0).stop 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 msgr2=0x7f4250146860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 0x7f4250146860 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f4238011600 tx=0x7f4238011ac0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 shutdown_connections 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f422c06b4d0 0x7f422c06d990 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42500af2a0 0x7f4250146da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42500aee20 0x7f4250146860 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 --2- 172.21.15.165:0/2425966840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4234004d40 0x7f4250156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.959+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 >> 172.21.15.165:0/2425966840 conn(0x7f425006abe0 msgr2=0x7f425006dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.960+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 shutdown_connections 2024-04-25T10:06:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:52.960+0000 7f4256800640 1 -- 172.21.15.165:0/2425966840 wait complete. 2024-04-25T10:06:52.984 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 Processor -- start 2024-04-25T10:06:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 -- start start 2024-04-25T10:06:53.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd718143e60 0x7fd71806e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd7180cca10 0x7fd71815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd718144240 con 0x7fd71806e6a0 2024-04-25T10:06:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd71815e220 con 0x7fd7180cca10 2024-04-25T10:06:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.178+0000 7fd71cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815e500 con 0x7fd718143e60 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd7180cca10 0x7fd71815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd7180cca10 0x7fd71815ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44598/0 (socket says 172.21.15.165:44598) 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 -- 172.21.15.165:0/1226789298 learned_addr learned my addr 172.21.15.165:0/1226789298 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717400640 1 --2- 172.21.15.165:0/1226789298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd718143e60 0x7fd71806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 -- 172.21.15.165:0/1226789298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd718143e60 msgr2=0x7fd71806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 --2- 172.21.15.165:0/1226789298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd718143e60 0x7fd71806e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 -- 172.21.15.165:0/1226789298 >> v1:172.21.15.165:6789/0 conn(0x7fd71806e6a0 legacy=0x7fd7180cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd717e00640 1 -- 172.21.15.165:0/1226789298 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71815e7e0 con 0x7fd7180cca10 2024-04-25T10:06:53.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.179+0000 7fd716a00640 1 -- 172.21.15.165:0/1226789298 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3989062099 0 0) 0x7fd718144240 con 0x7fd71806e6a0 2024-04-25T10:06:53.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.180+0000 7fd717e00640 1 --2- 172.21.15.165:0/1226789298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd71815ab40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd710009e50 tx=0x7fd71002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e7cd5787178927 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd716a00640 1 -- 172.21.15.165:0/1226789298 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd710036070 con 0x7fd7180cca10 2024-04-25T10:06:53.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd716a00640 1 -- 172.21.15.165:0/1226789298 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd710031600 con 0x7fd7180cca10 2024-04-25T10:06:53.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd716a00640 1 -- 172.21.15.165:0/1226789298 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7100319d0 con 0x7fd7180cca10 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 -- 172.21.15.165:0/1226789298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 msgr2=0x7fd71815ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 --2- 172.21.15.165:0/1226789298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd71815ab40 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fd710009e50 tx=0x7fd71002a710 comp rx=0 tx=0).stop 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 -- 172.21.15.165:0/1226789298 shutdown_connections 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 --2- 172.21.15.165:0/1226789298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd71815ab40 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 --2- 172.21.15.165:0/1226789298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd718143e60 0x7fd71806e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 -- 172.21.15.165:0/1226789298 >> 172.21.15.165:0/1226789298 conn(0x7fd71806abe0 msgr2=0x7fd71806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 -- 172.21.15.165:0/1226789298 shutdown_connections 2024-04-25T10:06:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.181+0000 7fd71cc00640 1 -- 172.21.15.165:0/1226789298 wait complete. 2024-04-25T10:06:53.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.183+0000 7fd71cc00640 1 Processor -- start 2024-04-25T10:06:53.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.183+0000 7fd71cc00640 1 -- start start 2024-04-25T10:06:53.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.184+0000 7fd71cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd71806e6a0 0x7fd7181f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.184+0000 7fd71cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd7180c7360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.184+0000 7fd71cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd718143e60 0x7fd7180ae840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.184+0000 7fd71cc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180b2eb0 con 0x7fd7180cca10 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.184+0000 7fd71cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7180b2d30 con 0x7fd718143e60 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.184+0000 7fd71cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd71815b080 con 0x7fd71806e6a0 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd7180c7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd7180c7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44608/0 (socket says 172.21.15.165:44608) 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 -- 172.21.15.165:0/959128045 learned_addr learned my addr 172.21.15.165:0/959128045 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 -- 172.21.15.165:0/959128045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd71806e6a0 msgr2=0x7fd7181f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd71806e6a0 0x7fd7181f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 -- 172.21.15.165:0/959128045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd718143e60 msgr2=0x7fd7180ae840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd718143e60 0x7fd7180ae840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 -- 172.21.15.165:0/959128045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7180aed80 con 0x7fd7180cca10 2024-04-25T10:06:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70ea00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd7180c7360 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c4b0 tx=0x7fd70000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.185+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700017070 con 0x7fd7180cca10 2024-04-25T10:06:53.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.186+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700005480 con 0x7fd7180cca10 2024-04-25T10:06:53.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.186+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700003830 con 0x7fd7180cca10 2024-04-25T10:06:53.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.186+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7181f3700 con 0x7fd7180cca10 2024-04-25T10:06:53.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.186+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7181f3cc0 con 0x7fd7180cca10 2024-04-25T10:06:53.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.187+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd718073990 con 0x7fd7180cca10 2024-04-25T10:06:53.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.188+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd70001a070 con 0x7fd7180cca10 2024-04-25T10:06:53.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.189+0000 7fd70fe00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6f406b4d0 0x7fd6f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.189+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd7000896e0 con 0x7fd7180cca10 2024-04-25T10:06:53.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.189+0000 7fd717400640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6f406b4d0 0x7fd6f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.190+0000 7fd717400640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6f406b4d0 0x7fd6f406d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd708006fd0 tx=0x7fd708008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.194+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd700058430 con 0x7fd7180cca10 2024-04-25T10:06:53.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.430+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd71806d160 con 0x7fd7180cca10 2024-04-25T10:06:53.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.431+0000 7fd70fe00640 1 -- 172.21.15.165:0/959128045 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fd700057d30 con 0x7fd7180cca10 2024-04-25T10:06:53.437 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:53.437 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:51.630520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:53.438 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 29 2024-04-25T10:06:53.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6f406b4d0 msgr2=0x7fd6f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6f406b4d0 0x7fd6f406d990 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd708006fd0 tx=0x7fd708008040 comp rx=0 tx=0).stop 2024-04-25T10:06:53.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 msgr2=0x7fd7180c7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd7180c7360 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fd70000c4b0 tx=0x7fd70000c970 comp rx=0 tx=0).stop 2024-04-25T10:06:53.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 shutdown_connections 2024-04-25T10:06:53.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6f406b4d0 0x7fd6f406d990 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd718143e60 0x7fd7180ae840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7180cca10 0x7fd7180c7360 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 --2- 172.21.15.165:0/959128045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd71806e6a0 0x7fd7181f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.433+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 >> 172.21.15.165:0/959128045 conn(0x7fd71806abe0 msgr2=0x7fd7180cbea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.434+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 shutdown_connections 2024-04-25T10:06:53.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.434+0000 7fd71cc00640 1 -- 172.21.15.165:0/959128045 wait complete. 2024-04-25T10:06:53.463 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 3783141116 2024-04-25T10:06:53.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.640+0000 7f3699a00640 1 Processor -- start 2024-04-25T10:06:53.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.640+0000 7f3699a00640 1 -- start start 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.641+0000 7f3699a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f369415cfc0 0x7f36940c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.641+0000 7f3699a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3694145520 con 0x7f36940ceb20 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.641+0000 7f3699a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3694071270 con 0x7f3694145140 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.641+0000 7f3699a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3694071550 con 0x7f369415cfc0 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3699000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f369415cfc0 0x7f36940c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3699000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f369415cfc0 0x7f36940c5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44618/0 (socket says 172.21.15.165:44618) 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3699000640 1 -- 172.21.15.165:0/3996113112 learned_addr learned my addr 172.21.15.165:0/3996113112 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3693400640 1 --1- 172.21.15.165:0/3996113112 >> v1:172.21.15.165:6789/0 conn(0x7f36940ceb20 0x7f36940c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54142/0 (socket says 172.21.15.165:54142) 2024-04-25T10:06:53.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3699000640 1 -- 172.21.15.165:0/3996113112 >> v1:172.21.15.172:6789/0 conn(0x7f3694145140 legacy=0x7f36940ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3699000640 1 -- 172.21.15.165:0/3996113112 >> v1:172.21.15.165:6789/0 conn(0x7f36940ceb20 legacy=0x7f36940c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3699000640 1 -- 172.21.15.165:0/3996113112 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694071830 con 0x7f369415cfc0 2024-04-25T10:06:53.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.642+0000 7f3692a00640 1 -- 172.21.15.165:0/3996113112 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 799512854 0 0) 0x7f3694145520 con 0x7f36940ceb20 2024-04-25T10:06:53.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.643+0000 7f3699000640 1 --2- 172.21.15.165:0/3996113112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f369415cfc0 0x7f36940c5200 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e50 tx=0x7f368c02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86541f3f60509a8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3692a00640 1 -- 172.21.15.165:0/3996113112 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c036070 con 0x7f369415cfc0 2024-04-25T10:06:53.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3692a00640 1 -- 172.21.15.165:0/3996113112 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f368c031600 con 0x7f369415cfc0 2024-04-25T10:06:53.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3692a00640 1 -- 172.21.15.165:0/3996113112 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368c0319d0 con 0x7f369415cfc0 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 -- 172.21.15.165:0/3996113112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f369415cfc0 msgr2=0x7f36940c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 --2- 172.21.15.165:0/3996113112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f369415cfc0 0x7f36940c5200 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f368c009e50 tx=0x7f368c02a730 comp rx=0 tx=0).stop 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 -- 172.21.15.165:0/3996113112 shutdown_connections 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 --2- 172.21.15.165:0/3996113112 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f369415cfc0 0x7f36940c5200 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 -- 172.21.15.165:0/3996113112 >> 172.21.15.165:0/3996113112 conn(0x7f3694061cc0 msgr2=0x7f369406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 -- 172.21.15.165:0/3996113112 shutdown_connections 2024-04-25T10:06:53.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.644+0000 7f3699a00640 1 -- 172.21.15.165:0/3996113112 wait complete. 2024-04-25T10:06:53.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.646+0000 7f3699a00640 1 Processor -- start 2024-04-25T10:06:53.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.646+0000 7f3699a00640 1 -- start start 2024-04-25T10:06:53.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3699a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 0x7f36940cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3699a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3694145140 0x7f3694143650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3699a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f369415cfc0 0x7f369415aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3699a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36941665b0 con 0x7f36940ceb20 2024-04-25T10:06:53.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3699a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3694166430 con 0x7f369415cfc0 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3699a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3694166730 con 0x7f3694145140 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 0x7f36940cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 0x7f36940cb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44632/0 (socket says 172.21.15.165:44632) 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.647+0000 7f3693e00640 1 -- 172.21.15.165:0/2644238804 learned_addr learned my addr 172.21.15.165:0/2644238804 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3693e00640 1 -- 172.21.15.165:0/2644238804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3694145140 msgr2=0x7f3694143650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3693e00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3694145140 0x7f3694143650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3693e00640 1 -- 172.21.15.165:0/2644238804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f369415cfc0 msgr2=0x7f369415aae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3693e00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f369415cfc0 0x7f369415aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3693e00640 1 -- 172.21.15.165:0/2644238804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3694141f80 con 0x7f36940ceb20 2024-04-25T10:06:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3693e00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 0x7f36940cb100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f368400c450 tx=0x7f368400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f368be00640 1 -- 172.21.15.165:0/2644238804 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840052d0 con 0x7f36940ceb20 2024-04-25T10:06:53.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f368be00640 1 -- 172.21.15.165:0/2644238804 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684005470 con 0x7f36940ceb20 2024-04-25T10:06:53.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3694142260 con 0x7f36940ceb20 2024-04-25T10:06:53.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.648+0000 7f368be00640 1 -- 172.21.15.165:0/2644238804 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684012870 con 0x7f36940ceb20 2024-04-25T10:06:53.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.649+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3694142820 con 0x7f36940ceb20 2024-04-25T10:06:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.651+0000 7f368be00640 1 -- 172.21.15.165:0/2644238804 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3684012a10 con 0x7f36940ceb20 2024-04-25T10:06:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.653+0000 7f368be00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f367406b4d0 0x7f367406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.653+0000 7f368be00640 1 -- 172.21.15.165:0/2644238804 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f368408a0a0 con 0x7f36940ceb20 2024-04-25T10:06:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.653+0000 7f3693400640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f367406b4d0 0x7f367406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.653+0000 7f369ddff740 1 Processor -- start 2024-04-25T10:06:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.653+0000 7f369ddff740 1 -- start start 2024-04-25T10:06:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.653+0000 7f3693400640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f367406b4d0 0x7f367406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3680007fd0 tx=0x7f3680007900 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.654+0000 7f369ddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f04e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.654+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009eff700 0x561009f03680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.654+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f04000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.654+0000 7f369ddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009e68820 con 0x561009f04a80 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.654+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009e11d00 con 0x561009eff700 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.654+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561009e648c0 con 0x561009f03bc0 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f04e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f04e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44644/0 (socket says 172.21.15.165:44644) 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 -- 172.21.15.165:0/231637738 learned_addr learned my addr 172.21.15.165:0/231637738 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3699000640 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f04000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693400640 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009eff700 0x561009f03680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 -- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 msgr2=0x561009f04000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f04000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 -- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009eff700 msgr2=0x561009f03680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009eff700 0x561009f03680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 -- 172.21.15.165:0/231637738 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561009c49710 con 0x561009f04a80 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.655+0000 7f3693e00640 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f04e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3684000f30 tx=0x7f36840585c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=596ec088868ae34a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.656+0000 7f3688c00640 1 -- 172.21.15.165:0/231637738 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36840568c0 con 0x561009f04a80 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.656+0000 7f3688c00640 1 -- 172.21.15.165:0/231637738 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684012d00 con 0x561009f04a80 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.656+0000 7f3688c00640 1 -- 172.21.15.165:0/231637738 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684060840 con 0x561009f04a80 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.656+0000 7f369ddff740 1 -- 172.21.15.165:0/231637738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 msgr2=0x561009f04e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.656+0000 7f369ddff740 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f04e60 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3684000f30 tx=0x7f36840585c0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 -- 172.21.15.165:0/231637738 shutdown_connections 2024-04-25T10:06:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f04000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009eff700 0x561009f03680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 --2- 172.21.15.165:0/231637738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f04e60 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 -- 172.21.15.165:0/231637738 >> 172.21.15.165:0/231637738 conn(0x561009f01b80 msgr2=0x561009f4a640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 -- 172.21.15.165:0/231637738 shutdown_connections 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 -- 172.21.15.165:0/231637738 wait complete. 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.657+0000 7f369ddff740 1 -- start start 2024-04-25T10:06:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.658+0000 7f369ddff740 1 Processor -- start 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.658+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f1dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f369ddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f4d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 0x561009f4d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f369ddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009f30610 con 0x561009f04a80 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009e07360 con 0x561009f409f0 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561009f2fa90 con 0x561009f03bc0 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f369ddff740 10 client.0 _ll_register_callbacks cb 0x561009e13c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f4d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f4d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44646/0 (socket says 172.21.15.165:44646) 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 -- 172.21.15.165:0/476519353 learned_addr learned my addr 172.21.15.165:0/476519353 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 -- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 msgr2=0x561009f1dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f1dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 -- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 msgr2=0x561009f4d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 0x561009f4d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 -- 172.21.15.165:0/476519353 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561009ca9ea0 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.659+0000 7f3693400640 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f4d2b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3680005af0 tx=0x7f3680000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20e5ceaa9b2fa1d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f367d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f367d600640 1 -- 172.21.15.165:0/476519353 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680012440 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f367d600640 1 -- 172.21.15.165:0/476519353 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3680022c90 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3684012d00 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f367d600640 1 -- 172.21.15.165:0/476519353 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36800125e0 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f369ddff740 10 client.6183 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x561009b3d000 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f367d600640 1 -- 172.21.15.165:0/476519353 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684012d00 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f367d600640 1 -- 172.21.15.165:0/476519353 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3680029050 con 0x561009f04a80 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.660+0000 7f369ddff740 10 client.6183 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.661+0000 7f369ddff740 10 client.6183 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.661+0000 7f369ddff740 20 client.6183 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:53.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.661+0000 7f369ddff740 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f1e930 0x561009f210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.661+0000 7f369ddff740 4 client.6183 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:53.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.661+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561009e75d60 con 0x561009f1e930 2024-04-25T10:06:53.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.661+0000 7f3693e00640 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f1e930 0x561009f210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.662+0000 7f367d600640 10 client.6183 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:53.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.662+0000 7f3693e00640 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f1e930 0x561009f210f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ced328e3d6f9e28c server_cookie=abf49e9261dd8b6c in_seq=0 out_seq=0 2024-04-25T10:06:53.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f367d600640 1 -- 172.21.15.165:0/476519353 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561009e75d60 con 0x561009f1e930 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f367d600640 10 client.6183 handle_command_reply: tid=0 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 client.6183 shutdown 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 msgr2=0x561009f4d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f4d2b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3680005af0 tx=0x7f3680000ed0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 shutdown_connections 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f1e930 0x561009f210f0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 0x561009f4d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f4d2b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 --2- 172.21.15.165:0/476519353 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f03bc0 0x561009f1dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f367d600640 0 client.6183 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 >> 172.21.15.165:0/476519353 conn(0x561009eff700 msgr2=0x561009efee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 shutdown_connections 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.664+0000 7f369ddff740 1 -- 172.21.15.165:0/476519353 wait complete. 2024-04-25T10:06:53.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.665+0000 7f369ddff740 20 client.6183 trim_cache size 0 max 16384 2024-04-25T10:06:53.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.678+0000 7f369ddff740 1 Processor -- start 2024-04-25T10:06:53.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.678+0000 7f369ddff740 1 -- start start 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f369ddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 0x561009f64350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f04a80 0x561009f64890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 0x561009f3ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f369ddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36941ff4e0 con 0x561009f03bc0 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369413bb00 con 0x561009f409f0 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36940b3010 con 0x561009f04a80 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 0x561009f64350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 0x561009f64350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44650/0 (socket says 172.21.15.165:44650) 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 -- 172.21.15.165:0/3746334902 learned_addr learned my addr 172.21.15.165:0/3746334902 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 -- 172.21.15.165:0/3746334902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f04a80 msgr2=0x561009f64890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f04a80 0x561009f64890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 -- 172.21.15.165:0/3746334902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 msgr2=0x561009f3ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 0x561009f3ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.679+0000 7f3693e00640 1 -- 172.21.15.165:0/3746334902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561009f4b9b0 con 0x561009f03bc0 2024-04-25T10:06:53.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f3693e00640 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 0x561009f64350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3684058270 tx=0x7f368405b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=848da45fcc10b59d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f367fe00640 1 -- 172.21.15.165:0/3746334902 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684054860 con 0x561009f03bc0 2024-04-25T10:06:53.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f367fe00640 1 -- 172.21.15.165:0/3746334902 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684012d00 con 0x561009f03bc0 2024-04-25T10:06:53.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f367fe00640 1 -- 172.21.15.165:0/3746334902 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3684054c30 con 0x561009f03bc0 2024-04-25T10:06:53.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f369ddff740 1 -- 172.21.15.165:0/3746334902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 msgr2=0x561009f64350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f369ddff740 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 0x561009f64350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3684058270 tx=0x7f368405b000 comp rx=0 tx=0).stop 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f369ddff740 1 -- 172.21.15.165:0/3746334902 shutdown_connections 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.680+0000 7f369ddff740 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f409f0 0x561009f3ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.681+0000 7f369ddff740 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f04a80 0x561009f64890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.681+0000 7f369ddff740 1 --2- 172.21.15.165:0/3746334902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f03bc0 0x561009f64350 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.681+0000 7f369ddff740 1 -- 172.21.15.165:0/3746334902 >> 172.21.15.165:0/3746334902 conn(0x561009eff700 msgr2=0x561009efa870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.681+0000 7f369ddff740 1 -- 172.21.15.165:0/3746334902 shutdown_connections 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.681+0000 7f369ddff740 1 -- 172.21.15.165:0/3746334902 wait complete. 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.681+0000 7f369ddff740 1 -- start start 2024-04-25T10:06:53.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.682+0000 7f369ddff740 1 Processor -- start 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.682+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f03bc0 0x561009fb6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.682+0000 7f369ddff740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f219d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f369ddff740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f409f0 0x561009f21f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f369ddff740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009f1e7b0 con 0x561009f04a80 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561009f4dfd0 con 0x561009f03bc0 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f369ddff740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561009f1ed60 con 0x561009f409f0 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f369ddff740 10 client.0 _ll_register_callbacks cb 0x7f3684058390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f219d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f219d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44660/0 (socket says 172.21.15.165:44660) 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 -- 172.21.15.165:0/2930348138 learned_addr learned my addr 172.21.15.165:0/2930348138 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 -- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f409f0 msgr2=0x561009f21f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f409f0 0x561009f21f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693e00640 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f03bc0 0x561009fb6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 -- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f03bc0 msgr2=0x561009fb6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f03bc0 0x561009fb6260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 -- 172.21.15.165:0/2930348138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561009f4d970 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.683+0000 7f3693400640 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f219d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f3680007000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c0892b79dc456f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f367d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f367d600640 1 -- 172.21.15.165:0/2930348138 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3680012440 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f367d600640 1 -- 172.21.15.165:0/2930348138 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f368002b680 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3684012d00 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f367d600640 1 -- 172.21.15.165:0/2930348138 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3680014800 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f369ddff740 10 client.6189 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x561009fb6cc0 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f367d600640 1 -- 172.21.15.165:0/2930348138 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3684012d00 con 0x561009f04a80 2024-04-25T10:06:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f367d600640 1 -- 172.21.15.165:0/2930348138 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3680029050 con 0x561009f04a80 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f369ddff740 10 client.6189 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f369ddff740 10 client.6189 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.684+0000 7f369ddff740 20 client.6189 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.685+0000 7f369ddff740 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f3fa10 0x561009fb9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.685+0000 7f369ddff740 4 client.6189 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 3783141116}] 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.685+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "damage rm", "damage_id": 3783141116}) v1 -- 0x561009e700b0 con 0x561009f3fa10 2024-04-25T10:06:53.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.685+0000 7f3693e00640 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f3fa10 0x561009fb9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.686+0000 7f3693e00640 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f3fa10 0x561009fb9ea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d704cb559b75c67 server_cookie=ac6b426e3a023bd8 in_seq=0 out_seq=0 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.686+0000 7f367d600640 10 client.6189 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f367d600640 1 -- 172.21.15.165:0/2930348138 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x561009e700b0 con 0x561009f3fa10 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f367d600640 10 client.6189 handle_command_reply: tid=0 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 client.6189 shutdown 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 msgr2=0x561009f219d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f219d0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f3680007000 comp rx=0 tx=0).stop 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 shutdown_connections 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561009f3fa10 0x561009fb9ea0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561009f409f0 0x561009f21f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561009f04a80 0x561009f219d0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 --2- 172.21.15.165:0/2930348138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561009f03bc0 0x561009fb6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f367d600640 0 client.6189 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 >> 172.21.15.165:0/2930348138 conn(0x561009eff700 msgr2=0x561009f64350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 shutdown_connections 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 1 -- 172.21.15.165:0/2930348138 wait complete. 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.687+0000 7f369ddff740 20 client.6189 trim_cache size 0 max 16384 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f367406b4d0 msgr2=0x7f367406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f367406b4d0 0x7f367406d990 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f3680007fd0 tx=0x7f3680007900 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 msgr2=0x7f36940cb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 0x7f36940cb100 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f368400c450 tx=0x7f368400c910 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 shutdown_connections 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f367406b4d0 0x7f367406d990 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f369415cfc0 0x7f369415aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3694145140 0x7f3694143650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 --2- 172.21.15.165:0/2644238804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f36940ceb20 0x7f36940cb100 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 >> 172.21.15.165:0/2644238804 conn(0x7f3694061cc0 msgr2=0x7f369406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 shutdown_connections 2024-04-25T10:06:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.688+0000 7f3699a00640 1 -- 172.21.15.165:0/2644238804 wait complete. 2024-04-25T10:06:53.711 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:06:53.712 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.904+0000 7fe271200640 1 Processor -- start 2024-04-25T10:06:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.904+0000 7fe271200640 1 -- start start 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe271200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c15db90 0x7fe26c0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe271200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe26c0d06c0 0x7fe26c0d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe271200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe26c161df0 con 0x7fe26c1615a0 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe271200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe26c0d1f50 con 0x7fe26c0d06c0 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe271200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c1620d0 con 0x7fe26c15db90 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe26be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe26c0d06c0 0x7fe26c0d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe26be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe26c0d06c0 0x7fe26c0d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44668/0 (socket says 172.21.15.165:44668) 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.905+0000 7fe26be00640 1 -- 172.21.15.165:0/1194294103 learned_addr learned my addr 172.21.15.165:0/1194294103 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.906+0000 7fe26b400640 1 --2- 172.21.15.165:0/1194294103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c15db90 0x7fe26c0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.906+0000 7fe26be00640 1 -- 172.21.15.165:0/1194294103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c15db90 msgr2=0x7fe26c0b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.906+0000 7fe26be00640 1 --2- 172.21.15.165:0/1194294103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c15db90 0x7fe26c0b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.906+0000 7fe26be00640 1 -- 172.21.15.165:0/1194294103 >> v1:172.21.15.172:6789/0 conn(0x7fe26c1615a0 legacy=0x7fe26c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.906+0000 7fe26be00640 1 -- 172.21.15.165:0/1194294103 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c1623b0 con 0x7fe26c0d06c0 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.907+0000 7fe26be00640 1 --2- 172.21.15.165:0/1194294103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c0d06c0 0x7fe26c0d23e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe264009870 tx=0x7fe26402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a758559a96b3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.907+0000 7fe26a000640 1 -- 172.21.15.165:0/1194294103 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264037070 con 0x7fe26c0d06c0 2024-04-25T10:06:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.907+0000 7fe26a000640 1 -- 172.21.15.165:0/1194294103 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe264032440 con 0x7fe26c0d06c0 2024-04-25T10:06:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.907+0000 7fe26a000640 1 -- 172.21.15.165:0/1194294103 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264032810 con 0x7fe26c0d06c0 2024-04-25T10:06:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 -- 172.21.15.165:0/1194294103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c0d06c0 msgr2=0x7fe26c0d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 --2- 172.21.15.165:0/1194294103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c0d06c0 0x7fe26c0d23e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe264009870 tx=0x7fe26402a210 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 -- 172.21.15.165:0/1194294103 shutdown_connections 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 --2- 172.21.15.165:0/1194294103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c0d06c0 0x7fe26c0d23e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 --2- 172.21.15.165:0/1194294103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c15db90 0x7fe26c0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 -- 172.21.15.165:0/1194294103 >> 172.21.15.165:0/1194294103 conn(0x7fe26c0b52b0 msgr2=0x7fe26c0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 -- 172.21.15.165:0/1194294103 shutdown_connections 2024-04-25T10:06:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.908+0000 7fe271200640 1 -- 172.21.15.165:0/1194294103 wait complete. 2024-04-25T10:06:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.910+0000 7fe271200640 1 Processor -- start 2024-04-25T10:06:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.910+0000 7fe271200640 1 -- start start 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe271200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c0d06c0 0x7fe26c0cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe271200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe26c15db90 0x7fe26c15e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe271200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 0x7fe26c151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe271200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c161ba0 con 0x7fe26c1615a0 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe271200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c161a20 con 0x7fe26c15db90 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe271200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe26c0b68d0 con 0x7fe26c0d06c0 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe26be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 0x7fe26c151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe26be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 0x7fe26c151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44670/0 (socket says 172.21.15.165:44670) 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe26be00640 1 -- 172.21.15.165:0/437455193 learned_addr learned my addr 172.21.15.165:0/437455193 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe26aa00640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe26c15db90 0x7fe26c15e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.911+0000 7fe26b400640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c0d06c0 0x7fe26c0cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26be00640 1 -- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c0d06c0 msgr2=0x7fe26c0cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26be00640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c0d06c0 0x7fe26c0cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26be00640 1 -- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe26c15db90 msgr2=0x7fe26c15e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26be00640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe26c15db90 0x7fe26c15e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26be00640 1 -- 172.21.15.165:0/437455193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe26c15eec0 con 0x7fe26c1615a0 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26aa00640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe26c15db90 0x7fe26c15e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26b400640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c0d06c0 0x7fe26c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe26be00640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 0x7fe26c151960 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe26402a250 tx=0x7fe2640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264041020 con 0x7fe26c1615a0 2024-04-25T10:06:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26403d640 con 0x7fe26c1615a0 2024-04-25T10:06:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26c151ea0 con 0x7fe26c1615a0 2024-04-25T10:06:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe264037040 con 0x7fe26c1615a0 2024-04-25T10:06:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.912+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26c15f440 con 0x7fe26c1615a0 2024-04-25T10:06:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.914+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe26c073990 con 0x7fe26c1615a0 2024-04-25T10:06:53.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.914+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe264044020 con 0x7fe26c1615a0 2024-04-25T10:06:53.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.915+0000 7fe263400640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe24c06b4d0 0x7fe24c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:53.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.915+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe2640aab60 con 0x7fe26c1615a0 2024-04-25T10:06:53.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.916+0000 7fe26b400640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe24c06b4d0 0x7fe24c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:53.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.916+0000 7fe26b400640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe24c06b4d0 0x7fe24c06d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe25c007330 tx=0x7fe25c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:53.920+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe26404b030 con 0x7fe26c1615a0 2024-04-25T10:06:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.150+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe26c0b7ab0 con 0x7fe26c1615a0 2024-04-25T10:06:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.152+0000 7fe263400640 1 -- 172.21.15.165:0/437455193 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fe264079790 con 0x7fe26c1615a0 2024-04-25T10:06:54.158 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:54.158 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:51.630520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 29 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe24c06b4d0 msgr2=0x7fe24c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe24c06b4d0 0x7fe24c06d990 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe25c007330 tx=0x7fe25c00b040 comp rx=0 tx=0).stop 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 msgr2=0x7fe26c151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 0x7fe26c151960 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fe26402a250 tx=0x7fe2640046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 shutdown_connections 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe24c06b4d0 0x7fe24c06d990 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe26c1615a0 0x7fe26c151960 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe26c15db90 0x7fe26c15e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 --2- 172.21.15.165:0/437455193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe26c0d06c0 0x7fe26c0cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 >> 172.21.15.165:0/437455193 conn(0x7fe26c0b52b0 msgr2=0x7fe26c0b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 shutdown_connections 2024-04-25T10:06:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.154+0000 7fe271200640 1 -- 172.21.15.165:0/437455193 wait complete. 2024-04-25T10:06:54.178 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:06:54.394 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:54.394 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:54.395 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:54.410 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:54.410 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:06:54.410 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_test_path 2024-04-25T10:06:54.410 INFO:tasks.cephfs.test_scrub_checks:Cloning repo into place 2024-04-25T10:06:54.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:54.411 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p /home/ubuntu/cephtest/mnt.0/scrub_test_path) 2024-04-25T10:06:54.472 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:06:54.472 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:06:54.472 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:06:54.472 DEBUG:teuthology.orchestra.run.smithi165:> import json 2024-04-25T10:06:54.472 DEBUG:teuthology.orchestra.run.smithi165:> import sys 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite'"'"') 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> except OSError as e: 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> sys.exit(e.errno) 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> print(json.dumps( 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> indent=2)) 2024-04-25T10:06:54.473 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:06:54.617 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_mode": 16877, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_ino": 1099511627777, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_dev": 40, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_nlink": 16, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_uid": 1000, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_gid": 1316, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_size": 18, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_atime": 1714039592.0581374, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_mtime": 1714039605.5339327, 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout: "st_ctime": 1714039605.5339327 2024-04-25T10:06:54.618 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:54.631 INFO:tasks.cephfs.test_scrub_checks:Initiating mds_scrub_checks on mds.0 test_path /scrub_test_path, run_seq 1 2024-04-25T10:06:54.631 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/i/dont/exist' 2024-04-25T10:06:54.631 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:54.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.872+0000 7ff0fce00640 1 Processor -- start 2024-04-25T10:06:54.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.873+0000 7ff0fce00640 1 -- start start 2024-04-25T10:06:54.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0fce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0f8154ee0 0x7ff0f80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0fce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f80cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0fce00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff0f80ce3c0 0x7ff0f815e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0fce00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f80ce830 con 0x7ff0f8154ee0 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0fce00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0f8072090 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0fce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f80d0150 con 0x7ff0f80cfcf0 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0f5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0f8154ee0 0x7ff0f80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.874+0000 7ff0f5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0f8154ee0 0x7ff0f80cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44682/0 (socket says 172.21.15.165:44682) 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3474083256 learned_addr learned my addr 172.21.15.165:0/3474083256 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0ed400640 1 --2- 172.21.15.165:0/3474083256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f80cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f6800640 1 --2- 172.21.15.165:0/3474083256 >> v2:172.21.15.172:3300/0 conn(0x7ff0f80ce3c0 0x7ff0f815e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3474083256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 msgr2=0x7ff0f80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f5e00640 1 --2- 172.21.15.165:0/3474083256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f80cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3474083256 >> v2:172.21.15.172:3300/0 conn(0x7ff0f80ce3c0 msgr2=0x7ff0f815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:54.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f5e00640 1 --2- 172.21.15.165:0/3474083256 >> v2:172.21.15.172:3300/0 conn(0x7ff0f80ce3c0 0x7ff0f815e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.875+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3474083256 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f8071480 con 0x7ff0f8154ee0 2024-04-25T10:06:54.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.876+0000 7ff0f5e00640 1 --2- 172.21.15.165:0/3474083256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80cf7b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff0f806f6b0 tx=0x7ff0e802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6893ac19cc0219 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.876+0000 7ff0f5400640 1 -- 172.21.15.165:0/3474083256 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8037070 con 0x7ff0f8154ee0 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.876+0000 7ff0f5400640 1 -- 172.21.15.165:0/3474083256 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8032440 con 0x7ff0f8154ee0 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.876+0000 7ff0f5400640 1 -- 172.21.15.165:0/3474083256 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8032810 con 0x7ff0f8154ee0 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 -- 172.21.15.165:0/3474083256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f8154ee0 msgr2=0x7ff0f80cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3474083256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80cf7b0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7ff0f806f6b0 tx=0x7ff0e802a210 comp rx=0 tx=0).stop 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 -- 172.21.15.165:0/3474083256 shutdown_connections 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3474083256 >> v2:172.21.15.172:3300/0 conn(0x7ff0f80ce3c0 0x7ff0f815e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3474083256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f80cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3474083256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80cf7b0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 -- 172.21.15.165:0/3474083256 >> 172.21.15.165:0/3474083256 conn(0x7ff0f806abe0 msgr2=0x7ff0f806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:54.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 -- 172.21.15.165:0/3474083256 shutdown_connections 2024-04-25T10:06:54.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.877+0000 7ff0fce00640 1 -- 172.21.15.165:0/3474083256 wait complete. 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.879+0000 7ff0fce00640 1 Processor -- start 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.879+0000 7ff0fce00640 1 -- start start 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0fce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 0x7ff0f81f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0fce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f81f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0fce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0fce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8071a40 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0fce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8071230 con 0x7ff0f8154ee0 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0fce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0f8071d20 con 0x7ff0f80cfcf0 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 0x7ff0f81f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 0x7ff0f81f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44696/0 (socket says 172.21.15.165:44696) 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.880+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3110883830 learned_addr learned my addr 172.21.15.165:0/3110883830 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f6800640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0ed400640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f81f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 msgr2=0x7ff0f81f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f5e00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f81f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8154ee0 msgr2=0x7ff0f80b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f5e00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f5e00640 1 -- 172.21.15.165:0/3110883830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0f815cac0 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f6800640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0ed400640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f81f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0f5e00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 0x7ff0f81f3fa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8000c00 tx=0x7ff0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8037040 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0e8042070 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0e8030bd0 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.881+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0f815cda0 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.882+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0f81f8280 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.883+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0f80ccfd0 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.884+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0e8047460 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.884+0000 7ff0eea00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0d406b4d0 0x7ff0d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.885+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff0e80ab0b0 con 0x7ff0f80ce3c0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.885+0000 7ff0ed400640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0d406b4d0 0x7ff0d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.885+0000 7ff0ed400640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0d406b4d0 0x7ff0d406d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0f806e3e0 tx=0x7ff0e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:54.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:54.889+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff0f80ccfd0 con 0x7ff0f80ce3c0 2024-04-25T10:06:55.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.125+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0f81f84c0 con 0x7ff0f80ce3c0 2024-04-25T10:06:55.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.127+0000 7ff0eea00640 1 -- 172.21.15.165:0/3110883830 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4948 (secure 0 0 0) 0x7ff0e8045360 con 0x7ff0f80ce3c0 2024-04-25T10:06:55.132 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:55.132 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:51.630520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:55.132 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 29 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0d406b4d0 msgr2=0x7ff0d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0d406b4d0 0x7ff0d406d990 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff0f806e3e0 tx=0x7ff0e0008040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 msgr2=0x7ff0f81f3fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 0x7ff0f81f3fa0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7ff0e8000c00 tx=0x7ff0e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 shutdown_connections 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0d406b4d0 0x7ff0d406d990 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0f8154ee0 0x7ff0f80b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0f80cfcf0 0x7ff0f81f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 --2- 172.21.15.165:0/3110883830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0f80ce3c0 0x7ff0f81f3fa0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 >> 172.21.15.165:0/3110883830 conn(0x7ff0f806abe0 msgr2=0x7ff0f8152dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 shutdown_connections 2024-04-25T10:06:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.129+0000 7ff0fce00640 1 -- 172.21.15.165:0/3110883830 wait complete. 2024-04-25T10:06:55.154 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/i/dont/exist 2024-04-25T10:06:55.365 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:55.365 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": -2 2024-04-25T10:06:55.365 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:55.380 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/i/dont/exist' got response code '0' and stdout '{ "return_code": -2 } ' 2024-04-25T10:06:55.380 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/i/dont/exist' 2024-04-25T10:06:55.380 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:55.380 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/i/dont/exist 2024-04-25T10:06:55.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.587+0000 7f29b8200640 1 Processor -- start 2024-04-25T10:06:55.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.587+0000 7f29b8200640 1 -- start start 2024-04-25T10:06:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.588+0000 7f29b8200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29b01446d0 0x7f29b00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.588+0000 7f29b8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.588+0000 7f29b8200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29b00cccc0 con 0x7f29b0144d50 2024-04-25T10:06:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.588+0000 7f29b8200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29b00b2b20 con 0x7f29b01446d0 2024-04-25T10:06:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.588+0000 7f29b8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b006d400 con 0x7f29b006e8a0 2024-04-25T10:06:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29b01446d0 0x7f29b00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29b01446d0 0x7f29b00b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44710/0 (socket says 172.21.15.165:44710) 2024-04-25T10:06:55.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 -- 172.21.15.165:0/2789566345 learned_addr learned my addr 172.21.15.165:0/2789566345 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 -- 172.21.15.165:0/2789566345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 msgr2=0x7f29b00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 --2- 172.21.15.165:0/2789566345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 -- 172.21.15.165:0/2789566345 >> v1:172.21.15.172:6789/0 conn(0x7f29b0144d50 legacy=0x7f29b00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.589+0000 7f29b5800640 1 -- 172.21.15.165:0/2789566345 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b00cc140 con 0x7f29b01446d0 2024-04-25T10:06:55.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.590+0000 7f29b5800640 1 --2- 172.21.15.165:0/2789566345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b01446d0 0x7f29b00b3f20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009870 tx=0x7f29a402a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459b48a81512468f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.591+0000 7f29afe00640 1 -- 172.21.15.165:0/2789566345 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4037070 con 0x7f29b01446d0 2024-04-25T10:06:55.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.591+0000 7f29afe00640 1 -- 172.21.15.165:0/2789566345 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a4032440 con 0x7f29b01446d0 2024-04-25T10:06:55.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.591+0000 7f29afe00640 1 -- 172.21.15.165:0/2789566345 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a4032810 con 0x7f29b01446d0 2024-04-25T10:06:55.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.591+0000 7f29b8200640 1 -- 172.21.15.165:0/2789566345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b01446d0 msgr2=0x7f29b00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.591+0000 7f29b8200640 1 --2- 172.21.15.165:0/2789566345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b01446d0 0x7f29b00b3f20 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f29a4009870 tx=0x7f29a402a200 comp rx=0 tx=0).stop 2024-04-25T10:06:55.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.592+0000 7f29b8200640 1 -- 172.21.15.165:0/2789566345 shutdown_connections 2024-04-25T10:06:55.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.592+0000 7f29b8200640 1 --2- 172.21.15.165:0/2789566345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.592+0000 7f29b8200640 1 --2- 172.21.15.165:0/2789566345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b01446d0 0x7f29b00b3f20 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.592+0000 7f29b8200640 1 -- 172.21.15.165:0/2789566345 >> 172.21.15.165:0/2789566345 conn(0x7f29b006abe0 msgr2=0x7f29b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.592+0000 7f29b8200640 1 -- 172.21.15.165:0/2789566345 shutdown_connections 2024-04-25T10:06:55.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.592+0000 7f29b8200640 1 -- 172.21.15.165:0/2789566345 wait complete. 2024-04-25T10:06:55.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.594+0000 7f29b8200640 1 Processor -- start 2024-04-25T10:06:55.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.594+0000 7f29b8200640 1 -- start start 2024-04-25T10:06:55.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b013c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b8200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b01446d0 0x7f29b0142260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b8200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 0x7f29b01427a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b8200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00cc700 con 0x7f29b0144d50 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b8200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00b49a0 con 0x7f29b01446d0 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00cc9e0 con 0x7f29b006e8a0 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.595+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 0x7f29b01427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 0x7f29b01427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44720/0 (socket says 172.21.15.165:44720) 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 -- 172.21.15.165:0/1259472696 learned_addr learned my addr 172.21.15.165:0/1259472696 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b5800640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b013c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b4e00640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b01446d0 0x7f29b0142260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 -- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 msgr2=0x7f29b013c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b013c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 -- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b01446d0 msgr2=0x7f29b0142260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b01446d0 0x7f29b0142260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 -- 172.21.15.165:0/1259472696 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29b01e4330 con 0x7f29b0144d50 2024-04-25T10:06:55.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.596+0000 7f29b6200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 0x7f29b01427a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f29a800b9e0 tx=0x7f29a800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.597+0000 7f29ad600640 1 -- 172.21.15.165:0/1259472696 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8019070 con 0x7f29b0144d50 2024-04-25T10:06:55.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.597+0000 7f29ad600640 1 -- 172.21.15.165:0/1259472696 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a800cb20 con 0x7f29b0144d50 2024-04-25T10:06:55.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.597+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29b01e4610 con 0x7f29b0144d50 2024-04-25T10:06:55.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.597+0000 7f29ad600640 1 -- 172.21.15.165:0/1259472696 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8014440 con 0x7f29b0144d50 2024-04-25T10:06:55.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.597+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29b013e130 con 0x7f29b0144d50 2024-04-25T10:06:55.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.599+0000 7f29ad600640 1 -- 172.21.15.165:0/1259472696 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29a801c070 con 0x7f29b0144d50 2024-04-25T10:06:55.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.600+0000 7f29ad600640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f299006b4d0 0x7f299006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.600+0000 7f29ad600640 1 -- 172.21.15.165:0/1259472696 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f29a808a140 con 0x7f29b0144d50 2024-04-25T10:06:55.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.601+0000 7f29b5800640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f299006b4d0 0x7f299006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.601+0000 7f29bb017740 1 Processor -- start 2024-04-25T10:06:55.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.601+0000 7f29b5800640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f299006b4d0 0x7f299006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.602+0000 7f29bb017740 1 -- start start 2024-04-25T10:06:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a2e940 0x5596f4a2ed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29bb017740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 0x5596f4a05bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29bb017740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f4969b90 con 0x5596f4a4dca0 2024-04-25T10:06:55.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f49130b0 con 0x5596f4a08cf0 2024-04-25T10:06:55.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f4965c30 con 0x5596f4a2e940 2024-04-25T10:06:55.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 0x5596f4a05bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 0x5596f4a05bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44726/0 (socket says 172.21.15.165:44726) 2024-04-25T10:06:55.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.603+0000 7f29b6200640 1 -- 172.21.15.165:0/513163516 learned_addr learned my addr 172.21.15.165:0/513163516 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b6200640 1 -- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a2e940 msgr2=0x5596f4a2ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b5800640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a2e940 0x5596f4a2ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b4e00640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b6200640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a2e940 0x5596f4a2ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b6200640 1 -- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 msgr2=0x5596f4a4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b6200640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b6200640 1 -- 172.21.15.165:0/513163516 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f474a9f0 con 0x5596f4a4dca0 2024-04-25T10:06:55.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b4e00640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b5800640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a2e940 0x5596f4a2ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29b6200640 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 0x5596f4a05bb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29a8058610 tx=0x7f29a8058680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b079c29b51ca95a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.604+0000 7f29a2000640 1 -- 172.21.15.165:0/513163516 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8056980 con 0x5596f4a4dca0 2024-04-25T10:06:55.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.605+0000 7f29a2000640 1 -- 172.21.15.165:0/513163516 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8075e50 con 0x5596f4a4dca0 2024-04-25T10:06:55.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.605+0000 7f29a2000640 1 -- 172.21.15.165:0/513163516 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80770c0 con 0x5596f4a4dca0 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.605+0000 7f29bb017740 1 -- 172.21.15.165:0/513163516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 msgr2=0x5596f4a05bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.605+0000 7f29bb017740 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 0x5596f4a05bb0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f29a8058610 tx=0x7f29a8058680 comp rx=0 tx=0).stop 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 -- 172.21.15.165:0/513163516 shutdown_connections 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a4dca0 0x5596f4a05bb0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 --2- 172.21.15.165:0/513163516 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a2e940 0x5596f4a2ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 -- 172.21.15.165:0/513163516 >> 172.21.15.165:0/513163516 conn(0x5596f49cefb0 msgr2=0x5596f4a4c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 -- 172.21.15.165:0/513163516 shutdown_connections 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.606+0000 7f29bb017740 1 -- 172.21.15.165:0/513163516 wait complete. 2024-04-25T10:06:55.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.607+0000 7f29bb017740 1 -- start start 2024-04-25T10:06:55.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.607+0000 7f29bb017740 1 Processor -- start 2024-04-25T10:06:55.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a08cf0 0x5596f4a33f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a2e940 0x5596f4a4fb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 0x5596f4a500d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f4a033e0 con 0x5596f4a33ab0 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f4908570 con 0x5596f4a2e940 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f4a07c80 con 0x5596f4a08cf0 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.608+0000 7f29bb017740 10 client.0 _ll_register_callbacks cb 0x5596f467ddd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 0x5596f4a500d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 0x5596f4a500d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44742/0 (socket says 172.21.15.165:44742) 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 -- 172.21.15.165:0/2866258118 learned_addr learned my addr 172.21.15.165:0/2866258118 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b5800640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a08cf0 0x5596f4a33f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b4e00640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a2e940 0x5596f4a4fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 -- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a08cf0 msgr2=0x5596f4a33f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a08cf0 0x5596f4a33f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 -- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a2e940 msgr2=0x5596f4a4fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a2e940 0x5596f4a4fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 -- 172.21.15.165:0/2866258118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596f463e2e0 con 0x5596f4a33ab0 2024-04-25T10:06:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b4e00640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a2e940 0x5596f4a4fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f29b6200640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 0x5596f4a500d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f29a805c990 tx=0x7f29a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f712703fdc511b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.609+0000 7f2996a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:55.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f2996a00640 1 -- 172.21.15.165:0/2866258118 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a80770c0 con 0x5596f4a33ab0 2024-04-25T10:06:55.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f2996a00640 1 -- 172.21.15.165:0/2866258118 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f29a8077260 con 0x5596f4a33ab0 2024-04-25T10:06:55.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f29a8075e50 con 0x5596f4a33ab0 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f2996a00640 1 -- 172.21.15.165:0/2866258118 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a8060900 con 0x5596f4a33ab0 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 10 client.6213 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5596f47ab180 con 0x5596f4a33ab0 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f2996a00640 1 -- 172.21.15.165:0/2866258118 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29a8075e50 con 0x5596f4a33ab0 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f2996a00640 1 -- 172.21.15.165:0/2866258118 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29a8057ad0 con 0x5596f4a33ab0 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 10 client.6213 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 10 client.6213 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 20 client.6213 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:55.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a31440 0x5596f4a2f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 4 client.6213 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:55.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.610+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596f49770d0 con 0x5596f4a31440 2024-04-25T10:06:55.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.611+0000 7f29b5800640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a31440 0x5596f4a2f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.612+0000 7f29b5800640 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a31440 0x5596f4a2f2d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb912894ffbcc7a8 server_cookie=b955c6a077725731 in_seq=0 out_seq=0 2024-04-25T10:06:55.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.612+0000 7f2996a00640 10 client.6213 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f2996a00640 1 -- 172.21.15.165:0/2866258118 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5596f49770d0 con 0x5596f4a31440 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f2996a00640 10 client.6213 handle_command_reply: tid=0 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f29bb017740 1 client.6213 shutdown 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 msgr2=0x5596f4a500d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f29bb017740 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 0x5596f4a500d0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f29a805c990 tx=0x7f29a8004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 shutdown_connections 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.614+0000 7f29bb017740 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a31440 0x5596f4a2f2d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a33ab0 0x5596f4a500d0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a2e940 0x5596f4a4fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 1 --2- 172.21.15.165:0/2866258118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a08cf0 0x5596f4a33f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f2996a00640 0 client.6213 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 >> 172.21.15.165:0/2866258118 conn(0x5596f49cefb0 msgr2=0x5596f4a085d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 shutdown_connections 2024-04-25T10:06:55.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 1 -- 172.21.15.165:0/2866258118 wait complete. 2024-04-25T10:06:55.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.615+0000 7f29bb017740 20 client.6213 trim_cache size 0 max 16384 2024-04-25T10:06:55.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.631+0000 7f29bb017740 1 Processor -- start 2024-04-25T10:06:55.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.631+0000 7f29bb017740 1 -- start start 2024-04-25T10:06:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.632+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a085d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.632+0000 7f29bb017740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a01770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.632+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a27730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.632+0000 7f29bb017740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b01fc2c0 con 0x5596f4a2e940 2024-04-25T10:06:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.632+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00ceb40 con 0x5596f4a08cf0 2024-04-25T10:06:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.632+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29b013f5a0 con 0x5596f4a04a80 2024-04-25T10:06:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a01770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a01770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44750/0 (socket says 172.21.15.165:44750) 2024-04-25T10:06:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 -- 172.21.15.165:0/1028008159 learned_addr learned my addr 172.21.15.165:0/1028008159 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b6200640 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a27730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b5800640 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a085d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 -- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 msgr2=0x5596f4a27730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a27730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 -- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 msgr2=0x5596f4a085d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a085d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b4e00640 1 -- 172.21.15.165:0/1028008159 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f46173a0 con 0x5596f4a2e940 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.633+0000 7f29b5800640 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.634+0000 7f29b4e00640 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a01770 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e800 tx=0x7f299c00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a88894049b46d7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.634+0000 7f29a1600640 1 -- 172.21.15.165:0/1028008159 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c016070 con 0x5596f4a2e940 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.634+0000 7f29a1600640 1 -- 172.21.15.165:0/1028008159 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c00cc20 con 0x5596f4a2e940 2024-04-25T10:06:55.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.634+0000 7f29a1600640 1 -- 172.21.15.165:0/1028008159 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c012450 con 0x5596f4a2e940 2024-04-25T10:06:55.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 -- 172.21.15.165:0/1028008159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 msgr2=0x5596f4a01770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a01770 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f299c00e800 tx=0x7f299c00ecc0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 -- 172.21.15.165:0/1028008159 shutdown_connections 2024-04-25T10:06:55.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a27730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a01770 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 --2- 172.21.15.165:0/1028008159 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a085d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 -- 172.21.15.165:0/1028008159 >> 172.21.15.165:0/1028008159 conn(0x5596f49cefb0 msgr2=0x5596f4a05310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 -- 172.21.15.165:0/1028008159 shutdown_connections 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.635+0000 7f29bb017740 1 -- 172.21.15.165:0/1028008159 wait complete. 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.636+0000 7f29bb017740 1 -- start start 2024-04-25T10:06:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.636+0000 7f29bb017740 1 Processor -- start 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a6a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a16130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a31fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f4a08050 con 0x5596f4a2e940 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29b00ce540 con 0x5596f4a08cf0 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5596f49770d0 con 0x5596f4a04a80 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29bb017740 10 client.0 _ll_register_callbacks cb 0x7f29a8058450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a31fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a31fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44752/0 (socket says 172.21.15.165:44752) 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 -- 172.21.15.165:0/3579180974 learned_addr learned my addr 172.21.15.165:0/3579180974 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 -- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 msgr2=0x5596f4a6a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a6a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 -- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 msgr2=0x5596f4a16130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.637+0000 7f29b6200640 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a16130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f29b6200640 1 -- 172.21.15.165:0/3579180974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596f4a2f640 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f29b6200640 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a31fa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f29a8058610 tx=0x7f29a8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38de3582259dfcdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f2996a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f2996a00640 1 -- 172.21.15.165:0/3579180974 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29a8060b90 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f2996a00640 1 -- 172.21.15.165:0/3579180974 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f29a8015e50 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f299c00cc20 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f2996a00640 1 -- 172.21.15.165:0/3579180974 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29a80772d0 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f29bb017740 10 client.6219 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5596f4a2f640 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.638+0000 7f2996a00640 1 -- 172.21.15.165:0/3579180974 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c00cc20 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f2996a00640 1 -- 172.21.15.165:0/3579180974 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f29a80545d0 con 0x5596f4a2e940 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29bb017740 10 client.6219 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29bb017740 10 client.6219 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:55.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29bb017740 20 client.6219 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29bb017740 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a16670 0x5596f4a4e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29bb017740 4 client.6219 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}] 2024-04-25T10:06:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/i/dont/exist"}) v1 -- 0x5596f4a30dc0 con 0x5596f4a16670 2024-04-25T10:06:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.639+0000 7f29b5800640 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a16670 0x5596f4a4e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.640+0000 7f29b5800640 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a16670 0x5596f4a4e090 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786b3af22e2ae55d server_cookie=76cdaf51182ce285 in_seq=0 out_seq=0 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.640+0000 7f2996a00640 10 client.6219 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f2996a00640 1 -- 172.21.15.165:0/3579180974 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+26 (crc 0 0 0) 0x5596f4a30dc0 con 0x5596f4a16670 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f2996a00640 10 client.6219 handle_command_reply: tid=0 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 client.6219 shutdown 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 msgr2=0x5596f4a31fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a31fa0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f29a8058610 tx=0x7f29a8004b50 comp rx=0 tx=0).stop 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 shutdown_connections 2024-04-25T10:06:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5596f4a16670 0x5596f4a4e090 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5596f4a2e940 0x5596f4a31fa0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5596f4a08cf0 0x5596f4a16130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 --2- 172.21.15.165:0/3579180974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5596f4a04a80 0x5596f4a6a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f2996a00640 0 client.6219 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.641+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 >> 172.21.15.165:0/3579180974 conn(0x5596f49cefb0 msgr2=0x5596f4a4fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 shutdown_connections 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29bb017740 1 -- 172.21.15.165:0/3579180974 wait complete. 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29bb017740 20 client.6219 trim_cache size 0 max 16384 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": -2 2024-04-25T10:06:55.649 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:55.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f299006b4d0 msgr2=0x7f299006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f299006b4d0 0x7f299006d990 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f29a4034000 tx=0x7f29a402e000 comp rx=0 tx=0).stop 2024-04-25T10:06:55.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 msgr2=0x7f29b01427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 0x7f29b01427a0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f29a800b9e0 tx=0x7f29a800bea0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 shutdown_connections 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f299006b4d0 0x7f299006d990 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29b0144d50 0x7f29b01427a0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29b01446d0 0x7f29b0142260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 --2- 172.21.15.165:0/1259472696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29b006e8a0 0x7f29b013c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.642+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 >> 172.21.15.165:0/1259472696 conn(0x7f29b006abe0 msgr2=0x7f29b00af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.643+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 shutdown_connections 2024-04-25T10:06:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.643+0000 7f29b8200640 1 -- 172.21.15.165:0/1259472696 wait complete. 2024-04-25T10:06:55.665 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/i/dont/exist' returned '{'return_code': -2}' 2024-04-25T10:06:55.665 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites' 2024-04-25T10:06:55.665 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:55.665 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites 2024-04-25T10:06:55.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.864+0000 7f2d9f200640 1 Processor -- start 2024-04-25T10:06:55.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.864+0000 7f2d9f200640 1 -- start start 2024-04-25T10:06:55.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d9f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 0x7f2da006e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d9f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2da0165a80 0x7f2da0165f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d9f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2da00cdcd0 con 0x7f2da00ccf30 2024-04-25T10:06:55.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d9f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2da0162440 con 0x7f2da0165a80 2024-04-25T10:06:55.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d9f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01632d0 con 0x7f2da015f950 2024-04-25T10:06:55.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d95400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2da00ccf30 0x7f2da0162bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54152/0 (socket says 172.21.15.165:54152) 2024-04-25T10:06:55.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d95400640 1 -- 172.21.15.165:0/1474966096 learned_addr learned my addr 172.21.15.165:0/1474966096 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.866+0000 7f2d9de00640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 0x7f2da006e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1474966096 >> v2:172.21.15.172:3300/0 conn(0x7f2da0165a80 0x7f2da0165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3184146406 0 0) 0x7f2da00cdcd0 con 0x7f2da00ccf30 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d84003520 con 0x7f2da00ccf30 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627011636 0 0) 0x7f2d84003520 con 0x7f2da00ccf30 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 msgr2=0x7f2da006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 0x7f2da006e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 >> v2:172.21.15.172:3300/0 conn(0x7f2da0165a80 msgr2=0x7f2da0165f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 --2- 172.21.15.165:0/1474966096 >> v2:172.21.15.172:3300/0 conn(0x7f2da0165a80 0x7f2da0165f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da00cd430 con 0x7f2da00ccf30 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2d88001670 con 0x7f2da00ccf30 2024-04-25T10:06:55.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 >> v1:172.21.15.165:6789/0 conn(0x7f2da00ccf30 legacy=0x7f2da0162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d840039b0 0x7f2d84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 0x7f2d84004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d84004d40 0x7f2d840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0162440 con 0x7f2d84004d40 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01632d0 con 0x7f2d840039b0 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.867+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8400b9c0 con 0x7f2da015f950 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d84004d40 0x7f2d840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9de00640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d840039b0 0x7f2d84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d95400640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 0x7f2d84004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9e800640 1 -- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 msgr2=0x7f2d84004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da015f950 0x7f2d84004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9e800640 1 -- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d840039b0 msgr2=0x7f2d84003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d840039b0 0x7f2d84003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.868+0000 7f2d9e800640 1 -- 172.21.15.165:0/1474966096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d84003520 con 0x7f2d84004d40 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.869+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d84004d40 0x7f2d840051c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009fc0 tx=0x7f2d84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6fca12ecb88150 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98009c10 con 0x7f2d84004d40 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98036070 con 0x7f2d84004d40 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9d400640 1 -- 172.21.15.165:0/1474966096 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9802d070 con 0x7f2d84004d40 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9f200640 1 -- 172.21.15.165:0/1474966096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d84004d40 msgr2=0x7f2d840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9f200640 1 --2- 172.21.15.165:0/1474966096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d84004d40 0x7f2d840051c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f2d98009fc0 tx=0x7f2d84005830 comp rx=0 tx=0).stop 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9e800640 1 -- 172.21.15.165:0/1474966096 reap_dead start 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9f200640 1 -- 172.21.15.165:0/1474966096 shutdown_connections 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.870+0000 7f2d9f200640 1 -- 172.21.15.165:0/1474966096 >> 172.21.15.165:0/1474966096 conn(0x7f2da0061cc0 msgr2=0x7f2da006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.871+0000 7f2d9f200640 1 -- 172.21.15.165:0/1474966096 shutdown_connections 2024-04-25T10:06:55.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.871+0000 7f2d9f200640 1 -- 172.21.15.165:0/1474966096 wait complete. 2024-04-25T10:06:55.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.872+0000 7f2d9f200640 1 Processor -- start 2024-04-25T10:06:55.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.873+0000 7f2d9f200640 1 -- start start 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d84004d40 0x7f2da015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da0151070 0x7f2da0157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 0x7f2da0159600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da01626a0 con 0x7f2da0158170 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da00cd9f0 con 0x7f2d84004d40 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da015fd30 con 0x7f2da0151070 2024-04-25T10:06:55.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 0x7f2da0159600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 0x7f2da0159600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44760/0 (socket says 172.21.15.165:44760) 2024-04-25T10:06:55.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 -- 172.21.15.165:0/2527583753 learned_addr learned my addr 172.21.15.165:0/2527583753 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d95400640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da0151070 0x7f2da0157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9de00640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d84004d40 0x7f2da015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 -- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da0151070 msgr2=0x7f2da0157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da0151070 0x7f2da0157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 -- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d84004d40 msgr2=0x7f2da015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d84004d40 0x7f2da015bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.874+0000 7f2d9e800640 1 -- 172.21.15.165:0/2527583753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da0155800 con 0x7f2da0158170 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.875+0000 7f2d9e800640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 0x7f2da0159600 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2d98033040 tx=0x7f2d98009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.875+0000 7f2d96a00640 1 -- 172.21.15.165:0/2527583753 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98009c10 con 0x7f2da0158170 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.875+0000 7f2d96a00640 1 -- 172.21.15.165:0/2527583753 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98003990 con 0x7f2da0158170 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.875+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2da0152d80 con 0x7f2da0158170 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.875+0000 7f2d96a00640 1 -- 172.21.15.165:0/2527583753 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98003d80 con 0x7f2da0158170 2024-04-25T10:06:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.875+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2da0155240 con 0x7f2da0158170 2024-04-25T10:06:55.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.878+0000 7f2d96a00640 1 -- 172.21.15.165:0/2527583753 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d98005d10 con 0x7f2da0158170 2024-04-25T10:06:55.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.878+0000 7f2d96a00640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.879+0000 7f2d96a00640 1 -- 172.21.15.165:0/2527583753 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2d980abc80 con 0x7f2da0158170 2024-04-25T10:06:55.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.879+0000 7f2d9de00640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.879+0000 7f2da8014740 1 Processor -- start 2024-04-25T10:06:55.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.879+0000 7f2da8014740 1 -- start start 2024-04-25T10:06:55.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2d9de00640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2da0152be0 tx=0x7f2d90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3279c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 0x55bfb325ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2da8014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb325b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2da8014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb3197c40 con 0x55bfb325ef90 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb3141050 con 0x55bfb3270090 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.880+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb3193ce0 con 0x55bfb325e670 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb325b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb325b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44762/0 (socket says 172.21.15.165:44762) 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 -- 172.21.15.165:0/1976153693 learned_addr learned my addr 172.21.15.165:0/1976153693 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 -- 172.21.15.165:0/1976153693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 msgr2=0x55bfb3279c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3279c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 -- 172.21.15.165:0/1976153693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 msgr2=0x55bfb325ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 0x55bfb325ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 -- 172.21.15.165:0/1976153693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb2f78b00 con 0x55bfb325ef90 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.881+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb325b250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2d980824b0 tx=0x7f2d98079f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=282f1a1837f87f74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2d8ea00640 1 -- 172.21.15.165:0/1976153693 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d980a8070 con 0x55bfb325ef90 2024-04-25T10:06:55.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2d8ea00640 1 -- 172.21.15.165:0/1976153693 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98036030 con 0x55bfb325ef90 2024-04-25T10:06:55.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2d8ea00640 1 -- 172.21.15.165:0/1976153693 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9807e420 con 0x55bfb325ef90 2024-04-25T10:06:55.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 -- 172.21.15.165:0/1976153693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 msgr2=0x55bfb325b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb325b250 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2d980824b0 tx=0x7f2d98079f40 comp rx=0 tx=0).stop 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 -- 172.21.15.165:0/1976153693 shutdown_connections 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb325b250 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 0x55bfb325ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 --2- 172.21.15.165:0/1976153693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3279c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 -- 172.21.15.165:0/1976153693 >> 172.21.15.165:0/1976153693 conn(0x55bfb31fb5e0 msgr2=0x55bfb3279480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.882+0000 7f2da8014740 1 -- 172.21.15.165:0/1976153693 shutdown_connections 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.883+0000 7f2da8014740 1 -- 172.21.15.165:0/1976153693 wait complete. 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.883+0000 7f2da8014740 1 -- start start 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.883+0000 7f2da8014740 1 Processor -- start 2024-04-25T10:06:55.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3255140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325ef90 0x55bfb3277f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 0x55bfb32784d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb3141050 con 0x55bfb3270090 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb31365d0 con 0x55bfb325ef90 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb3197c40 con 0x55bfb325e670 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.884+0000 7f2da8014740 10 client.0 _ll_register_callbacks cb 0x55bfb318fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 0x55bfb32784d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 0x55bfb32784d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44770/0 (socket says 172.21.15.165:44770) 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 -- 172.21.15.165:0/2899353229 learned_addr learned my addr 172.21.15.165:0/2899353229 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9de00640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3255140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d95400640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325ef90 0x55bfb3277f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 -- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 msgr2=0x55bfb3255140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3255140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 -- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325ef90 msgr2=0x55bfb3277f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325ef90 0x55bfb3277f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 -- 172.21.15.165:0/2899353229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfb2e6c3f0 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d9e800640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 0x55bfb32784d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2d980aa0d0 tx=0x7f2d98000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22682220bb91257c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.885+0000 7f2d83400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2d83400640 1 -- 172.21.15.165:0/2899353229 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98096030 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2d83400640 1 -- 172.21.15.165:0/2899353229 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2d980785d0 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d98036030 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2d83400640 1 -- 172.21.15.165:0/2899353229 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d980a3050 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 10 client.6231 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55bfb2fd9290 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2d83400640 1 -- 172.21.15.165:0/2899353229 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d98036030 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2d83400640 1 -- 172.21.15.165:0/2899353229 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d9809d060 con 0x55bfb3270090 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 10 client.6231 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 10 client.6231 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 20 client.6231 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:55.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3278a10 0x55bfb3261400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 4 client.6231 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:55.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.886+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfb31a5180 con 0x55bfb3278a10 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.887+0000 7f2d9de00640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3278a10 0x55bfb3261400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.888+0000 7f2d9de00640 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3278a10 0x55bfb3261400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e316a9f0bd86ec17 server_cookie=249af5c4608eff80 in_seq=0 out_seq=0 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.888+0000 7f2d83400640 10 client.6231 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.889+0000 7f2d83400640 1 -- 172.21.15.165:0/2899353229 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55bfb31a5180 con 0x55bfb3278a10 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.889+0000 7f2d83400640 10 client.6231 handle_command_reply: tid=0 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.889+0000 7f2da8014740 1 client.6231 shutdown 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 msgr2=0x55bfb32784d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 0x55bfb32784d0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2d980aa0d0 tx=0x7f2d98000f40 comp rx=0 tx=0).stop 2024-04-25T10:06:55.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 shutdown_connections 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3278a10 0x55bfb3261400 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb3270090 0x55bfb32784d0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325ef90 0x55bfb3277f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 --2- 172.21.15.165:0/2899353229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3255140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2d83400640 0 client.6231 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 >> 172.21.15.165:0/2899353229 conn(0x55bfb31fb5e0 msgr2=0x55bfb32566f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 shutdown_connections 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 1 -- 172.21.15.165:0/2899353229 wait complete. 2024-04-25T10:06:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.890+0000 7f2da8014740 20 client.6231 trim_cache size 0 max 16384 2024-04-25T10:06:55.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 Processor -- start 2024-04-25T10:06:55.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 -- start start 2024-04-25T10:06:55.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3272dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3276980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 0x55bfb322e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0204d70 con 0x55bfb325ef90 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0203f20 con 0x55bfb3270090 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.907+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0152740 con 0x55bfb325e670 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3276980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3276980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44782/0 (socket says 172.21.15.165:44782) 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 -- 172.21.15.165:0/3197472140 learned_addr learned my addr 172.21.15.165:0/3197472140 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 -- 172.21.15.165:0/3197472140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 msgr2=0x55bfb3272dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3272dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 -- 172.21.15.165:0/3197472140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 msgr2=0x55bfb322e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 0x55bfb322e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 -- 172.21.15.165:0/3197472140 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfb2e454b0 con 0x55bfb325ef90 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.908+0000 7f2d95400640 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3276980 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005850 tx=0x7f2d88005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cbbe64cf52a5862 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2d8e000640 1 -- 172.21.15.165:0/3197472140 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d880109c0 con 0x55bfb325ef90 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2d8e000640 1 -- 172.21.15.165:0/3197472140 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d88019070 con 0x55bfb325ef90 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2d8e000640 1 -- 172.21.15.165:0/3197472140 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88010dc0 con 0x55bfb325ef90 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 -- 172.21.15.165:0/3197472140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 msgr2=0x55bfb3276980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3276980 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005850 tx=0x7f2d88005d10 comp rx=0 tx=0).stop 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 -- 172.21.15.165:0/3197472140 shutdown_connections 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb3270090 0x55bfb322e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3276980 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 --2- 172.21.15.165:0/3197472140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb325e670 0x55bfb3272dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 -- 172.21.15.165:0/3197472140 >> 172.21.15.165:0/3197472140 conn(0x55bfb31fb5e0 msgr2=0x55bfb325fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.909+0000 7f2da8014740 1 -- 172.21.15.165:0/3197472140 shutdown_connections 2024-04-25T10:06:55.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.910+0000 7f2da8014740 1 -- 172.21.15.165:0/3197472140 wait complete. 2024-04-25T10:06:55.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.910+0000 7f2da8014740 1 -- start start 2024-04-25T10:06:55.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.910+0000 7f2da8014740 1 Processor -- start 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325e670 0x55bfb3261400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3261940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb3270090 0x55bfb327a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfb325fbe0 con 0x55bfb325ef90 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da0203920 con 0x55bfb325e670 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55bfb31a58b0 con 0x55bfb3270090 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.911+0000 7f2da8014740 10 client.0 _ll_register_callbacks cb 0x7f2d98076570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3261940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3261940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44798/0 (socket says 172.21.15.165:44798) 2024-04-25T10:06:55.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 -- 172.21.15.165:0/1867746359 learned_addr learned my addr 172.21.15.165:0/1867746359 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d9e800640 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb3270090 0x55bfb327a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 -- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb3270090 msgr2=0x55bfb327a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb3270090 0x55bfb327a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 -- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325e670 msgr2=0x55bfb3261400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325e670 0x55bfb3261400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 -- 172.21.15.165:0/1867746359 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bfb3287df0 con 0x55bfb325ef90 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d95400640 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3261940 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005820 tx=0x7f2da0163480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bd4680c5ca5295f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.912+0000 7f2d83400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2d83400640 1 -- 172.21.15.165:0/1867746359 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88001430 con 0x55bfb325ef90 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2d83400640 1 -- 172.21.15.165:0/1867746359 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2d8801e840 con 0x55bfb325ef90 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d88019070 con 0x55bfb325ef90 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2d83400640 1 -- 172.21.15.165:0/1867746359 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d8802f050 con 0x55bfb325ef90 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 10 client.6237 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55bfb3288790 con 0x55bfb325ef90 2024-04-25T10:06:55.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2d83400640 1 -- 172.21.15.165:0/1867746359 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d88019070 con 0x55bfb325ef90 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2d83400640 1 -- 172.21.15.165:0/1867746359 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2d88032020 con 0x55bfb325ef90 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 10 client.6237 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 10 client.6237 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 20 client.6237 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3293230 0x55bfb327add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 4 client.6237 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}] 2024-04-25T10:06:55.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.913+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites"}) v1 -- 0x55bfb319f4d0 con 0x55bfb3293230 2024-04-25T10:06:55.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.914+0000 7f2d9de00640 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3293230 0x55bfb327add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:55.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.915+0000 7f2d9de00640 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3293230 0x55bfb327add0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e1367709f8d275b server_cookie=4ff66f48382f5b6e in_seq=0 out_seq=0 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.915+0000 7f2d83400640 10 client.6237 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.916+0000 7f2d83400640 1 -- 172.21.15.165:0/1867746359 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bfb319f4d0 con 0x55bfb3293230 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.916+0000 7f2d83400640 10 client.6237 handle_command_reply: tid=0 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.916+0000 7f2da8014740 1 client.6237 shutdown 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 msgr2=0x55bfb3261940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3261940 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2d88005820 tx=0x7f2da0163480 comp rx=0 tx=0).stop 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 shutdown_connections 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55bfb3293230 0x55bfb327add0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55bfb3270090 0x55bfb327a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55bfb325ef90 0x55bfb3261940 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 --2- 172.21.15.165:0/1867746359 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55bfb325e670 0x55bfb3261400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 >> 172.21.15.165:0/1867746359 conn(0x55bfb31fb5e0 msgr2=0x55bfb329a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2d83400640 0 client.6237 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 shutdown_connections 2024-04-25T10:06:55.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 1 -- 172.21.15.165:0/1867746359 wait complete. 2024-04-25T10:06:55.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.917+0000 7f2da8014740 20 client.6237 trim_cache size 0 max 16384 2024-04-25T10:06:55.924 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:55.924 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:55.924 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "b73890c6-55d6-4db3-941d-d0893b26b6a1", 2024-04-25T10:06:55.924 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:55.924 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d7c06b500 msgr2=0x7f2d7c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2da0152be0 tx=0x7f2d90008040 comp rx=0 tx=0).stop 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 msgr2=0x7f2da0159600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 0x7f2da0159600 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f2d98033040 tx=0x7f2d98009750 comp rx=0 tx=0).stop 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 shutdown_connections 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d7c06b500 0x7f2d7c06d9c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2da0158170 0x7f2da0159600 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2da0151070 0x7f2da0157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 --2- 172.21.15.165:0/2527583753 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d84004d40 0x7f2da015bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:55.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 >> 172.21.15.165:0/2527583753 conn(0x7f2da0061cc0 msgr2=0x7f2da0161b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:55.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 shutdown_connections 2024-04-25T10:06:55.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:55.918+0000 7f2d9f200640 1 -- 172.21.15.165:0/2527583753 wait complete. 2024-04-25T10:06:55.941 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites' returned '{'return_code': 0, 'scrub_tag': 'b73890c6-55d6-4db3-941d-d0893b26b6a1', 'mode': 'asynchronous'}' 2024-04-25T10:06:55.941 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' 2024-04-25T10:06:55.941 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:55.942 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml 2024-04-25T10:06:56.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.142+0000 7f1a52e00640 1 Processor -- start 2024-04-25T10:06:56.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.142+0000 7f1a52e00640 1 -- start start 2024-04-25T10:06:56.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a52e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c145140 0x7f1a4c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a52e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1a4c0ce950 0x7f1a4c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a52e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c145520 con 0x7f1a4c0c5030 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a52e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4c0ced90 con 0x7f1a4c0ce950 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a52e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c071270 con 0x7f1a4c145140 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a51000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1a4c0ce950 0x7f1a4c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a51000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1a4c0ce950 0x7f1a4c0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44802/0 (socket says 172.21.15.165:44802) 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a51000640 1 -- 172.21.15.165:0/706157527 learned_addr learned my addr 172.21.15.165:0/706157527 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a52400640 1 --1- 172.21.15.165:0/706157527 >> v1:172.21.15.165:6789/0 conn(0x7f1a4c0c5030 0x7f1a4c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54168/0 (socket says 172.21.15.165:54168) 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.144+0000 7f1a51a00640 1 --2- 172.21.15.165:0/706157527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c145140 0x7f1a4c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.145+0000 7f1a51000640 1 -- 172.21.15.165:0/706157527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c145140 msgr2=0x7f1a4c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.145+0000 7f1a51000640 1 --2- 172.21.15.165:0/706157527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c145140 0x7f1a4c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.145+0000 7f1a51000640 1 -- 172.21.15.165:0/706157527 >> v1:172.21.15.165:6789/0 conn(0x7f1a4c0c5030 legacy=0x7f1a4c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.145+0000 7f1a4be00640 1 -- 172.21.15.165:0/706157527 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621542393 0 0) 0x7f1a4c145520 con 0x7f1a4c0c5030 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.145+0000 7f1a51000640 1 -- 172.21.15.165:0/706157527 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c071550 con 0x7f1a4c0ce950 2024-04-25T10:06:56.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.146+0000 7f1a51000640 1 --2- 172.21.15.165:0/706157527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0c4af0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a3802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333cd67e715973ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.146+0000 7f1a4be00640 1 -- 172.21.15.165:0/706157527 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38037070 con 0x7f1a4c0ce950 2024-04-25T10:06:56.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.146+0000 7f1a4be00640 1 -- 172.21.15.165:0/706157527 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a38032440 con 0x7f1a4c0ce950 2024-04-25T10:06:56.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.146+0000 7f1a4be00640 1 -- 172.21.15.165:0/706157527 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38032810 con 0x7f1a4c0ce950 2024-04-25T10:06:56.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 -- 172.21.15.165:0/706157527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c0ce950 msgr2=0x7f1a4c0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 --2- 172.21.15.165:0/706157527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0c4af0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a3802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 -- 172.21.15.165:0/706157527 shutdown_connections 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 --2- 172.21.15.165:0/706157527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0c4af0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 --2- 172.21.15.165:0/706157527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c145140 0x7f1a4c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 -- 172.21.15.165:0/706157527 >> 172.21.15.165:0/706157527 conn(0x7f1a4c061cc0 msgr2=0x7f1a4c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 -- 172.21.15.165:0/706157527 shutdown_connections 2024-04-25T10:06:56.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.147+0000 7f1a52e00640 1 -- 172.21.15.165:0/706157527 wait complete. 2024-04-25T10:06:56.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.149+0000 7f1a52e00640 1 Processor -- start 2024-04-25T10:06:56.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.149+0000 7f1a52e00640 1 -- start start 2024-04-25T10:06:56.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c0c5030 0x7f1a4c1356b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0b1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 0x7f1a4c136100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c54a0 con 0x7f1a4c145140 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c071b10 con 0x7f1a4c0ce950 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c5620 con 0x7f1a4c0c5030 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 0x7f1a4c136100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 0x7f1a4c136100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44810/0 (socket says 172.21.15.165:44810) 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 -- 172.21.15.165:0/1999857614 learned_addr learned my addr 172.21.15.165:0/1999857614 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 -- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c0c5030 msgr2=0x7f1a4c1356b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c0c5030 0x7f1a4c1356b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a51000640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c0c5030 0x7f1a4c1356b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 -- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c0ce950 msgr2=0x7f1a4c0b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0b1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.150+0000 7f1a52400640 1 -- 172.21.15.165:0/1999857614 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c135bf0 con 0x7f1a4c145140 2024-04-25T10:06:56.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.151+0000 7f1a52400640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 0x7f1a4c136100 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c7a0 tx=0x7f1a4400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.151+0000 7f1a49600640 1 -- 172.21.15.165:0/1999857614 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44018070 con 0x7f1a4c145140 2024-04-25T10:06:56.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.151+0000 7f1a49600640 1 -- 172.21.15.165:0/1999857614 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4400a640 con 0x7f1a4c145140 2024-04-25T10:06:56.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.151+0000 7f1a49600640 1 -- 172.21.15.165:0/1999857614 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4400aab0 con 0x7f1a4c145140 2024-04-25T10:06:56.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.151+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c132fb0 con 0x7f1a4c145140 2024-04-25T10:06:56.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.151+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c133570 con 0x7f1a4c145140 2024-04-25T10:06:56.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.154+0000 7f1a49600640 1 -- 172.21.15.165:0/1999857614 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a4400ac50 con 0x7f1a4c145140 2024-04-25T10:06:56.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.155+0000 7f1a49600640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.155+0000 7f1a49600640 1 -- 172.21.15.165:0/1999857614 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a44089ae0 con 0x7f1a4c145140 2024-04-25T10:06:56.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.155+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.155+0000 7f1a57214740 1 Processor -- start 2024-04-25T10:06:56.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.156+0000 7f1a57214740 1 -- start start 2024-04-25T10:06:56.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.156+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1a40006fd0 tx=0x7f1a40008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb86810 0x55771eb391a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a57214740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb68cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb691f0 0x55771eb3e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a57214740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771eaa1cf0 con 0x55771eb69930 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771ea4b1d0 con 0x55771eb86810 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55771ea9dd90 con 0x55771eb691f0 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb68cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb68cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44826/0 (socket says 172.21.15.165:44826) 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 -- 172.21.15.165:0/886570877 learned_addr learned my addr 172.21.15.165:0/886570877 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51a00640 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb86810 0x55771eb391a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 -- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb691f0 msgr2=0x55771eb3e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb691f0 0x55771eb3e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 -- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb86810 msgr2=0x55771eb391a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb86810 0x55771eb391a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.157+0000 7f1a51000640 1 -- 172.21.15.165:0/886570877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55771e882f00 con 0x55771eb69930 2024-04-25T10:06:56.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.158+0000 7f1a51000640 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb68cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a3802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a6533f4ead7c08c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.158+0000 7f1a3e000640 1 -- 172.21.15.165:0/886570877 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a38037040 con 0x55771eb69930 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.158+0000 7f1a3e000640 1 -- 172.21.15.165:0/886570877 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a38042070 con 0x55771eb69930 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.158+0000 7f1a3e000640 1 -- 172.21.15.165:0/886570877 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3803c8a0 con 0x55771eb69930 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 -- 172.21.15.165:0/886570877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 msgr2=0x55771eb68cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb68cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f1a38009870 tx=0x7f1a3802c000 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 -- 172.21.15.165:0/886570877 shutdown_connections 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb691f0 0x55771eb3e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb68cb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 --2- 172.21.15.165:0/886570877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb86810 0x55771eb391a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 -- 172.21.15.165:0/886570877 >> 172.21.15.165:0/886570877 conn(0x55771eb3b410 msgr2=0x55771eb83cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 -- 172.21.15.165:0/886570877 shutdown_connections 2024-04-25T10:06:56.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.159+0000 7f1a57214740 1 -- 172.21.15.165:0/886570877 wait complete. 2024-04-25T10:06:56.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.160+0000 7f1a57214740 1 -- start start 2024-04-25T10:06:56.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.160+0000 7f1a57214740 1 Processor -- start 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb86810 0x55771eb66c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb671b0 0x55771eb6a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771eb3d5c0 con 0x55771eb69930 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771ea40b50 con 0x55771eb671b0 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55771eb3c9b0 con 0x55771eb86810 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a57214740 10 client.0 _ll_register_callbacks cb 0x55771ea62370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb56ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44832/0 (socket says 172.21.15.165:44832) 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 -- 172.21.15.165:0/1306239045 learned_addr learned my addr 172.21.15.165:0/1306239045 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51000640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb86810 0x55771eb66c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a52400640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb671b0 0x55771eb6a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 -- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb86810 msgr2=0x55771eb66c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb86810 0x55771eb66c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 -- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb671b0 msgr2=0x55771eb6a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb671b0 0x55771eb6a870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.161+0000 7f1a51a00640 1 -- 172.21.15.165:0/1306239045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55771e7767f0 con 0x55771eb69930 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb56ce0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000ed70 tx=0x7f1a40004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d950ef9d3c0a243 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a32a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a32a00640 1 -- 172.21.15.165:0/1306239045 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4000f040 con 0x55771eb69930 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a32a00640 1 -- 172.21.15.165:0/1306239045 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a40015dd0 con 0x55771eb69930 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a38042070 con 0x55771eb69930 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a32a00640 1 -- 172.21.15.165:0/1306239045 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4002d050 con 0x55771eb69930 2024-04-25T10:06:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a57214740 10 client.6252 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.162+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55771e8e3690 con 0x55771eb69930 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a32a00640 1 -- 172.21.15.165:0/1306239045 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a38042070 con 0x55771eb69930 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a32a00640 1 -- 172.21.15.165:0/1306239045 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a40030020 con 0x55771eb69930 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a57214740 10 client.6252 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a57214740 10 client.6252 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a57214740 20 client.6252 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a57214740 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771eb65a80 0x55771eb65e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a57214740 4 client.6252 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.163+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55771eaaf230 con 0x55771eb65a80 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.164+0000 7f1a51000640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771eb65a80 0x55771eb65e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.165+0000 7f1a51000640 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771eb65a80 0x55771eb65e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67224463d8e8614b server_cookie=64fb9f7c735a7f51 in_seq=0 out_seq=0 2024-04-25T10:06:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.165+0000 7f1a32a00640 10 client.6252 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a32a00640 1 -- 172.21.15.165:0/1306239045 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55771eaaf230 con 0x55771eb65a80 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a32a00640 10 client.6252 handle_command_reply: tid=0 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 client.6252 shutdown 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 msgr2=0x55771eb56ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb56ce0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000ed70 tx=0x7f1a40004730 comp rx=0 tx=0).stop 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 shutdown_connections 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771eb65a80 0x55771eb65e90 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb671b0 0x55771eb6a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb86810 0x55771eb66c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a32a00640 0 client.6252 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 --2- 172.21.15.165:0/1306239045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb69930 0x55771eb56ce0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.166+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 >> 172.21.15.165:0/1306239045 conn(0x55771eb691f0 msgr2=0x55771eb38830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.167+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 shutdown_connections 2024-04-25T10:06:56.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.167+0000 7f1a57214740 1 -- 172.21.15.165:0/1306239045 wait complete. 2024-04-25T10:06:56.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.167+0000 7f1a57214740 20 client.6252 trim_cache size 0 max 16384 2024-04-25T10:06:56.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.183+0000 7f1a57214740 1 Processor -- start 2024-04-25T10:06:56.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.183+0000 7f1a57214740 1 -- start start 2024-04-25T10:06:56.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a57214740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb67f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a57214740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c200240 con 0x55771eb86810 2024-04-25T10:06:56.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1ff400 con 0x55771eb69930 2024-04-25T10:06:56.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1f7490 con 0x55771eb671b0 2024-04-25T10:06:56.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a52400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb67f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a52400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb67f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44834/0 (socket says 172.21.15.165:44834) 2024-04-25T10:06:56.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.184+0000 7f1a52400640 1 -- 172.21.15.165:0/1418433589 learned_addr learned my addr 172.21.15.165:0/1418433589 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a52400640 1 -- 172.21.15.165:0/1418433589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 msgr2=0x55771eb963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a52400640 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a52400640 1 -- 172.21.15.165:0/1418433589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 msgr2=0x55771eb96910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a52400640 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a52400640 1 -- 172.21.15.165:0/1418433589 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55771eb66fe0 con 0x55771eb86810 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a52400640 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb67f90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1a440601e0 tx=0x7f1a44058700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e458bbaf69dd51 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a3d600640 1 -- 172.21.15.165:0/1418433589 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4405c150 con 0x55771eb86810 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a3d600640 1 -- 172.21.15.165:0/1418433589 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44060710 con 0x55771eb86810 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.185+0000 7f1a3d600640 1 -- 172.21.15.165:0/1418433589 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44054670 con 0x55771eb86810 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 -- 172.21.15.165:0/1418433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 msgr2=0x55771eb67f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb67f90 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f1a440601e0 tx=0x7f1a44058700 comp rx=0 tx=0).stop 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 -- 172.21.15.165:0/1418433589 shutdown_connections 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb67f90 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 --2- 172.21.15.165:0/1418433589 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 -- 172.21.15.165:0/1418433589 >> 172.21.15.165:0/1418433589 conn(0x55771eb691f0 msgr2=0x55771eb3eca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 -- 172.21.15.165:0/1418433589 shutdown_connections 2024-04-25T10:06:56.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.186+0000 7f1a57214740 1 -- 172.21.15.165:0/1418433589 wait complete. 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.187+0000 7f1a57214740 1 -- start start 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.187+0000 7f1a57214740 1 Processor -- start 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb37fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1fee00 con 0x55771eb86810 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55771eb9d200 con 0x55771eb69930 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55771ebf1e40 con 0x55771eb671b0 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a57214740 10 client.0 _ll_register_callbacks cb 0x7f1a38035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb37fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb37fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44836/0 (socket says 172.21.15.165:44836) 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 -- 172.21.15.165:0/1014791511 learned_addr learned my addr 172.21.15.165:0/1014791511 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 -- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 msgr2=0x55771eb963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a51000640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 -- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 msgr2=0x55771eb96910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.188+0000 7f1a52400640 1 -- 172.21.15.165:0/1014791511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55771e74f8b0 con 0x55771eb86810 2024-04-25T10:06:56.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a52400640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb37fa0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1a440601e0 tx=0x7f1a4401e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aff85ea0f65de2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a32a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a32a00640 1 -- 172.21.15.165:0/1014791511 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44073030 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a32a00640 1 -- 172.21.15.165:0/1014791511 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a44086040 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a44060710 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a32a00640 1 -- 172.21.15.165:0/1014791511 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4407f050 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a57214740 10 client.6258 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55771eb9dd90 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.189+0000 7f1a32a00640 1 -- 172.21.15.165:0/1014791511 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44060710 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a32a00640 1 -- 172.21.15.165:0/1014791511 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1a440812f0 con 0x55771eb86810 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a57214740 10 client.6258 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a57214740 10 client.6258 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a57214740 20 client.6258 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a57214740 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771ebebcd0 0x55771eb9d200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a57214740 4 client.6258 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}] 2024-04-25T10:06:56.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml"}) v1 -- 0x55771eaa9580 con 0x55771ebebcd0 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.190+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771ebebcd0 0x55771eb9d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.191+0000 7f1a51a00640 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771ebebcd0 0x55771eb9d200 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74d5be9d26553078 server_cookie=988a265d4993578a in_seq=0 out_seq=0 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.191+0000 7f1a32a00640 10 client.6258 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a32a00640 1 -- 172.21.15.165:0/1014791511 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55771eaa9580 con 0x55771ebebcd0 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a32a00640 10 client.6258 handle_command_reply: tid=0 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 client.6258 shutdown 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 msgr2=0x55771eb37fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb37fa0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1a440601e0 tx=0x7f1a4401e070 comp rx=0 tx=0).stop 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 shutdown_connections 2024-04-25T10:06:56.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55771ebebcd0 0x55771eb9d200 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55771eb86810 0x55771eb37fa0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55771eb69930 0x55771eb96910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a32a00640 0 client.6258 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 --2- 172.21.15.165:0/1014791511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55771eb671b0 0x55771eb963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 >> 172.21.15.165:0/1014791511 conn(0x55771eb691f0 msgr2=0x55771eb346c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 shutdown_connections 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 1 -- 172.21.15.165:0/1014791511 wait complete. 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.193+0000 7f1a57214740 20 client.6258 trim_cache size 0 max 16384 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:56.200 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "bd826d76-e235-4960-823b-76bc9b2b3f63", 2024-04-25T10:06:56.201 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:56.201 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:56.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1a2c06b4d0 msgr2=0x7f1a2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1a40006fd0 tx=0x7f1a40008040 comp rx=0 tx=0).stop 2024-04-25T10:06:56.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 msgr2=0x7f1a4c136100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 0x7f1a4c136100 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c7a0 tx=0x7f1a4400cc60 comp rx=0 tx=0).stop 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 shutdown_connections 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1a2c06b4d0 0x7f1a2c06d990 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a4c145140 0x7f1a4c136100 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a4c0ce950 0x7f1a4c0b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 --2- 172.21.15.165:0/1999857614 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a4c0c5030 0x7f1a4c1356b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 >> 172.21.15.165:0/1999857614 conn(0x7f1a4c061cc0 msgr2=0x7f1a4c144470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 shutdown_connections 2024-04-25T10:06:56.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.194+0000 7f1a52e00640 1 -- 172.21.15.165:0/1999857614 wait complete. 2024-04-25T10:06:56.217 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/suites/fs/verify/validater/valgrind.yaml' returned '{'return_code': 0, 'scrub_tag': 'bd826d76-e235-4960-823b-76bc9b2b3f63', 'mode': 'asynchronous'}' 2024-04-25T10:06:56.218 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /' 2024-04-25T10:06:56.218 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:56.218 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / 2024-04-25T10:06:56.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.414+0000 7f1146400640 1 Processor -- start 2024-04-25T10:06:56.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.415+0000 7f1146400640 1 -- start start 2024-04-25T10:06:56.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f1146400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114015b7d0 0x7f11400cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f1146400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11400b5620 0x7f11400b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f1146400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f114006cf50 con 0x7f114006c2d0 2024-04-25T10:06:56.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f1146400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11400b5fc0 con 0x7f11400b5620 2024-04-25T10:06:56.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f1146400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11400ce530 con 0x7f114015b7d0 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f113ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11400b5620 0x7f11400b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f113ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11400b5620 0x7f11400b5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44844/0 (socket says 172.21.15.165:44844) 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.416+0000 7f113fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f114006c2d0 0x7f114006c760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54172/0 (socket says 172.21.15.165:54172) 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113ea00640 1 -- 172.21.15.165:0/1272975178 learned_addr learned my addr 172.21.15.165:0/1272975178 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113f400640 1 --2- 172.21.15.165:0/1272975178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114015b7d0 0x7f11400cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113ea00640 1 -- 172.21.15.165:0/1272975178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114015b7d0 msgr2=0x7f11400cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113ea00640 1 --2- 172.21.15.165:0/1272975178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114015b7d0 0x7f11400cf010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113ea00640 1 -- 172.21.15.165:0/1272975178 >> v1:172.21.15.165:6789/0 conn(0x7f114006c2d0 legacy=0x7f114006c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113ea00640 1 -- 172.21.15.165:0/1272975178 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11400ce810 con 0x7f11400b5620 2024-04-25T10:06:56.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.417+0000 7f113e000640 1 -- 172.21.15.165:0/1272975178 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 813022072 0 0) 0x7f114006cf50 con 0x7f114006c2d0 2024-04-25T10:06:56.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.418+0000 7f113ea00640 1 --2- 172.21.15.165:0/1272975178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400b5a80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1128009870 tx=0x7f112802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e1dbf91addc0f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.418+0000 7f113e000640 1 -- 172.21.15.165:0/1272975178 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128037070 con 0x7f11400b5620 2024-04-25T10:06:56.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.418+0000 7f113e000640 1 -- 172.21.15.165:0/1272975178 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1128032440 con 0x7f11400b5620 2024-04-25T10:06:56.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.418+0000 7f113e000640 1 -- 172.21.15.165:0/1272975178 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128032810 con 0x7f11400b5620 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 -- 172.21.15.165:0/1272975178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 msgr2=0x7f11400b5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 --2- 172.21.15.165:0/1272975178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400b5a80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1128009870 tx=0x7f112802a270 comp rx=0 tx=0).stop 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 -- 172.21.15.165:0/1272975178 shutdown_connections 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 --2- 172.21.15.165:0/1272975178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400b5a80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 --2- 172.21.15.165:0/1272975178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114015b7d0 0x7f11400cf010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 -- 172.21.15.165:0/1272975178 >> 172.21.15.165:0/1272975178 conn(0x7f11400d0800 msgr2=0x7f11400d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 -- 172.21.15.165:0/1272975178 shutdown_connections 2024-04-25T10:06:56.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.419+0000 7f1146400640 1 -- 172.21.15.165:0/1272975178 wait complete. 2024-04-25T10:06:56.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.421+0000 7f1146400640 1 Processor -- start 2024-04-25T10:06:56.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.421+0000 7f1146400640 1 -- start start 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f1146400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114006c2d0 0x7f11401f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f1146400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400cd850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f1146400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f114015b7d0 0x7f11400b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f1146400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11400cf550 con 0x7f11400b5620 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f1146400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f114006d230 con 0x7f114015b7d0 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f1146400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11400cf6d0 con 0x7f114006c2d0 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f113ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f113ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44850/0 (socket says 172.21.15.165:44850) 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.422+0000 7f113ea00640 1 -- 172.21.15.165:0/1543013987 learned_addr learned my addr 172.21.15.165:0/1543013987 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113f400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114006c2d0 0x7f11401f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113ea00640 1 -- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114006c2d0 msgr2=0x7f11401f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113ea00640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114006c2d0 0x7f11401f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113ea00640 1 -- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f114015b7d0 msgr2=0x7f11400b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113ea00640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f114015b7d0 0x7f11400b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113ea00640 1 -- 172.21.15.165:0/1543013987 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11401f33a0 con 0x7f11400b5620 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f113ea00640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400cd850 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1128000c00 tx=0x7f11280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.423+0000 7f1137400640 1 -- 172.21.15.165:0/1543013987 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128041020 con 0x7f11400b5620 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.424+0000 7f1137400640 1 -- 172.21.15.165:0/1543013987 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11280306c0 con 0x7f11400b5620 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.424+0000 7f1137400640 1 -- 172.21.15.165:0/1543013987 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128037040 con 0x7f11400b5620 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.424+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11401f6fd0 con 0x7f11400b5620 2024-04-25T10:06:56.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.424+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11401f7590 con 0x7f11400b5620 2024-04-25T10:06:56.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.426+0000 7f1137400640 1 -- 172.21.15.165:0/1543013987 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1128044020 con 0x7f11400b5620 2024-04-25T10:06:56.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.427+0000 7f1137400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.427+0000 7f1137400640 1 -- 172.21.15.165:0/1543013987 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f11280aa920 con 0x7f11400b5620 2024-04-25T10:06:56.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.427+0000 7f113f400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.428+0000 7f1149250740 1 Processor -- start 2024-04-25T10:06:56.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.428+0000 7f1149250740 1 -- start start 2024-04-25T10:06:56.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.428+0000 7f113f400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f111c06b4d0 0x7f111c06d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f11300099c0 tx=0x7f1130003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f1149250740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41054360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41084af0 0x55ea41083e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea410843b0 0x55ea410591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f1149250740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40fbd880 con 0x55ea410a19d0 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40f66d00 con 0x55ea41084af0 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea40fb9920 con 0x55ea410843b0 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41054360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41054360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44856/0 (socket says 172.21.15.165:44856) 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f113f400640 1 -- 172.21.15.165:0/1958827718 learned_addr learned my addr 172.21.15.165:0/1958827718 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.429+0000 7f113ea00640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41084af0 0x55ea41083e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113fe00640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea410843b0 0x55ea410591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113f400640 1 -- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea410843b0 msgr2=0x55ea410591c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113f400640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea410843b0 0x55ea410591c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113f400640 1 -- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41084af0 msgr2=0x55ea41083e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113f400640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41084af0 0x55ea41083e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113f400640 1 -- 172.21.15.165:0/1958827718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea40f731f0 con 0x55ea410a19d0 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113ea00640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41084af0 0x55ea41083e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113fe00640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea410843b0 0x55ea410591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f113f400640 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41054360 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f11300049c0 tx=0x7f1130012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd11184986620495 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f112fe00640 1 -- 172.21.15.165:0/1958827718 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130026070 con 0x55ea410a19d0 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f112fe00640 1 -- 172.21.15.165:0/1958827718 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11300147e0 con 0x55ea410a19d0 2024-04-25T10:06:56.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.430+0000 7f112fe00640 1 -- 172.21.15.165:0/1958827718 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130014b90 con 0x55ea410a19d0 2024-04-25T10:06:56.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 -- 172.21.15.165:0/1958827718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 msgr2=0x55ea41054360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41054360 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f11300049c0 tx=0x7f1130012000 comp rx=0 tx=0).stop 2024-04-25T10:06:56.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 -- 172.21.15.165:0/1958827718 shutdown_connections 2024-04-25T10:06:56.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea410843b0 0x55ea410591c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41084af0 0x55ea41083e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 --2- 172.21.15.165:0/1958827718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41054360 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 -- 172.21.15.165:0/1958827718 >> 172.21.15.165:0/1958827718 conn(0x55ea410565d0 msgr2=0x55ea4109ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 -- 172.21.15.165:0/1958827718 shutdown_connections 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.431+0000 7f1149250740 1 -- 172.21.15.165:0/1958827718 wait complete. 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.432+0000 7f1149250740 1 -- start start 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.432+0000 7f1149250740 1 Processor -- start 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41074d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41087170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41096540 0x55ea4109ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41058780 con 0x55ea410a19d0 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea40f5c1c0 con 0x55ea41096540 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea41057b70 con 0x55ea41084af0 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f1149250740 10 client.0 _ll_register_callbacks cb 0x55ea40f79b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f113ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41087170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f113ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41087170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44870/0 (socket says 172.21.15.165:44870) 2024-04-25T10:06:56.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.433+0000 7f113ea00640 1 -- 172.21.15.165:0/918904966 learned_addr learned my addr 172.21.15.165:0/918904966 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113fe00640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41096540 0x55ea4109ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113f400640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41074d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113ea00640 1 -- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 msgr2=0x55ea41074d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113ea00640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41074d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113ea00640 1 -- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41096540 msgr2=0x55ea4109ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113ea00640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41096540 0x55ea4109ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113ea00640 1 -- 172.21.15.165:0/918904966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea40d9e690 con 0x55ea410a19d0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f113ea00640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41087170 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1128078b40 tx=0x7f1128079600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d698194adcec1d4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f112cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f112cc00640 1 -- 172.21.15.165:0/918904966 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112807b070 con 0x55ea410a19d0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f112cc00640 1 -- 172.21.15.165:0/918904966 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f112807f0b0 con 0x55ea410a19d0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11300147e0 con 0x55ea410a19d0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f112cc00640 1 -- 172.21.15.165:0/918904966 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f112807cf90 con 0x55ea410a19d0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.434+0000 7f1149250740 10 client.6273 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55ea40dfee20 con 0x55ea410a19d0 2024-04-25T10:06:56.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f112cc00640 1 -- 172.21.15.165:0/918904966 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11300147e0 con 0x55ea410a19d0 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f112cc00640 1 -- 172.21.15.165:0/918904966 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f112803a030 con 0x55ea410a19d0 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 10 client.6273 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 10 client.6273 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 20 client.6273 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea410725a0 0x55ea41058b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 4 client.6273 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.435+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea40fcadc0 con 0x55ea410725a0 2024-04-25T10:06:56.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.436+0000 7f113f400640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea410725a0 0x55ea41058b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.437+0000 7f113f400640 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea410725a0 0x55ea41058b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ae2ee2025743699 server_cookie=a17d347cc00f9453 in_seq=0 out_seq=0 2024-04-25T10:06:56.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.437+0000 7f112cc00640 10 client.6273 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.438+0000 7f112cc00640 1 -- 172.21.15.165:0/918904966 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ea40fcadc0 con 0x55ea410725a0 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.438+0000 7f112cc00640 10 client.6273 handle_command_reply: tid=0 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.438+0000 7f1149250740 1 client.6273 shutdown 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 msgr2=0x55ea41087170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41087170 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f1128078b40 tx=0x7f1128079600 comp rx=0 tx=0).stop 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 shutdown_connections 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea410725a0 0x55ea41058b80 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea41096540 0x55ea4109ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea410a19d0 0x55ea41087170 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 --2- 172.21.15.165:0/918904966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41074d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f112cc00640 0 client.6273 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 >> 172.21.15.165:0/918904966 conn(0x55ea410843b0 msgr2=0x55ea410813a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 shutdown_connections 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 1 -- 172.21.15.165:0/918904966 wait complete. 2024-04-25T10:06:56.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.439+0000 7f1149250740 20 client.6273 trim_cache size 0 max 16384 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.455+0000 7f1149250740 1 Processor -- start 2024-04-25T10:06:56.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.455+0000 7f1149250740 1 -- start start 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41095810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f1149250740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 0x55ea41095d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f1149250740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1140200480 con 0x55ea41096540 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401ff640 con 0x55ea410a19d0 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11400c9c60 con 0x55ea41084af0 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f113ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 0x55ea41095d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.456+0000 7f113ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 0x55ea41095d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44874/0 (socket says 172.21.15.165:44874) 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 -- 172.21.15.165:0/216032894 learned_addr learned my addr 172.21.15.165:0/216032894 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 -- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 msgr2=0x55ea41095810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113f400640 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41095810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41095810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 -- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 msgr2=0x55ea410a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 -- 172.21.15.165:0/216032894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea40c91f80 con 0x55ea41096540 2024-04-25T10:06:56.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f113ea00640 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 0x55ea41095d50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1128006da0 tx=0x7f1128008000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a0b1b3db627fce2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f112f400640 1 -- 172.21.15.165:0/216032894 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112807b070 con 0x55ea41096540 2024-04-25T10:06:56.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f112f400640 1 -- 172.21.15.165:0/216032894 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112807f0b0 con 0x55ea41096540 2024-04-25T10:06:56.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.457+0000 7f112f400640 1 -- 172.21.15.165:0/216032894 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1128090df0 con 0x55ea41096540 2024-04-25T10:06:56.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 -- 172.21.15.165:0/216032894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 msgr2=0x55ea41095d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 0x55ea41095d50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f1128006da0 tx=0x7f1128008000 comp rx=0 tx=0).stop 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 -- 172.21.15.165:0/216032894 shutdown_connections 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41096540 0x55ea41095d50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 --2- 172.21.15.165:0/216032894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41084af0 0x55ea41095810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 -- 172.21.15.165:0/216032894 >> 172.21.15.165:0/216032894 conn(0x55ea410843b0 msgr2=0x55ea410811f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 -- 172.21.15.165:0/216032894 shutdown_connections 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.458+0000 7f1149250740 1 -- 172.21.15.165:0/216032894 wait complete. 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.459+0000 7f1149250740 1 -- start start 2024-04-25T10:06:56.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.459+0000 7f1149250740 1 Processor -- start 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 0x55ea411099b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41096540 0x55ea410a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11401ff040 con 0x55ea41084af0 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea41094930 con 0x55ea410a19d0 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ea410b1ed0 con 0x55ea41096540 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f1149250740 10 client.0 _ll_register_callbacks cb 0x7f11280750e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 0x55ea411099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f113f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 0x55ea411099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44886/0 (socket says 172.21.15.165:44886) 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.460+0000 7f113f400640 1 -- 172.21.15.165:0/3645501347 learned_addr learned my addr 172.21.15.165:0/3645501347 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113f400640 1 -- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41096540 msgr2=0x55ea410a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113ea00640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41096540 0x55ea410a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113fe00640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113f400640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41096540 0x55ea410a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113f400640 1 -- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 msgr2=0x55ea410a27d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113f400640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113f400640 1 -- 172.21.15.165:0/3645501347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea410ae6e0 con 0x55ea41084af0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f113f400640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 0x55ea411099b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1130004010 tx=0x7f1130004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1478c951b9ef7e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f112cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f112cc00640 1 -- 172.21.15.165:0/3645501347 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130035070 con 0x55ea41084af0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f112cc00640 1 -- 172.21.15.165:0/3645501347 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1130030180 con 0x55ea41084af0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f112807f0b0 con 0x55ea41084af0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f112cc00640 1 -- 172.21.15.165:0/3645501347 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f113003c050 con 0x55ea41084af0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f1149250740 10 client.6279 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.461+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x55ea41087060 con 0x55ea41084af0 2024-04-25T10:06:56.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f112cc00640 1 -- 172.21.15.165:0/3645501347 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112807f0b0 con 0x55ea41084af0 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f112cc00640 1 -- 172.21.15.165:0/3645501347 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f113003f020 con 0x55ea41084af0 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f1149250740 10 client.6279 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f1149250740 10 client.6279 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f1149250740 20 client.6279 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f1149250740 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea41084fe0 0x55ea410853f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f1149250740 4 client.6279 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/"}] 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/"}) v1 -- 0x55ea40fc5110 con 0x55ea41084fe0 2024-04-25T10:06:56.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.462+0000 7f113ea00640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea41084fe0 0x55ea410853f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.463+0000 7f113ea00640 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea41084fe0 0x55ea410853f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8882a324cf412d9e server_cookie=b448f95b74b4ad11 in_seq=0 out_seq=0 2024-04-25T10:06:56.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.463+0000 7f112cc00640 10 client.6279 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f112cc00640 1 -- 172.21.15.165:0/3645501347 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55ea40fc5110 con 0x55ea41084fe0 2024-04-25T10:06:56.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f112cc00640 10 client.6279 handle_command_reply: tid=0 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 client.6279 shutdown 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 msgr2=0x55ea411099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 0x55ea411099b0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1130004010 tx=0x7f1130004ab0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 shutdown_connections 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55ea41084fe0 0x55ea410853f0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ea410a19d0 0x55ea410a27d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ea41096540 0x55ea410a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 --2- 172.21.15.165:0/3645501347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ea41084af0 0x55ea411099b0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 >> 172.21.15.165:0/3645501347 conn(0x55ea410843b0 msgr2=0x55ea410811d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f112cc00640 0 client.6279 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 shutdown_connections 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 1 -- 172.21.15.165:0/3645501347 wait complete. 2024-04-25T10:06:56.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.465+0000 7f1149250740 20 client.6279 trim_cache size 0 max 16384 2024-04-25T10:06:56.473 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:56.473 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:56.473 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "7b664422-e976-42e6-9572-69902e690162", 2024-04-25T10:06:56.473 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:56.473 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f111c06b4d0 msgr2=0x7f111c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f111c06b4d0 0x7f111c06d990 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f11300099c0 tx=0x7f1130003990 comp rx=0 tx=0).stop 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 msgr2=0x7f11400cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400cd850 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f1128000c00 tx=0x7f11280046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 shutdown_connections 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f111c06b4d0 0x7f111c06d990 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f114015b7d0 0x7f11400b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11400b5620 0x7f11400cd850 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 --2- 172.21.15.165:0/1543013987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f114006c2d0 0x7f11401f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 >> 172.21.15.165:0/1543013987 conn(0x7f11400d0800 msgr2=0x7f11400d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 shutdown_connections 2024-04-25T10:06:56.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.466+0000 7f1146400640 1 -- 172.21.15.165:0/1543013987 wait complete. 2024-04-25T10:06:56.489 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /' returned '{'return_code': 0, 'scrub_tag': '7b664422-e976-42e6-9572-69902e690162', 'mode': 'asynchronous'}' 2024-04-25T10:06:56.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:56.489 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_dir_1) 2024-04-25T10:06:56.542 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' 2024-04-25T10:06:56.542 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:56.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.735+0000 7f1a16200640 1 Processor -- start 2024-04-25T10:06:56.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.735+0000 7f1a16200640 1 -- start start 2024-04-25T10:06:56.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.736+0000 7f1a16200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1a10144750 0x7f1a1006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a16200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a1006eb10 0x7f1a100ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a16200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a10144b30 con 0x7f1a100cee60 2024-04-25T10:06:56.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a16200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a100712d0 con 0x7f1a10144750 2024-04-25T10:06:56.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a16200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a100715b0 con 0x7f1a1006eb10 2024-04-25T10:06:56.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1a10144750 0x7f1a1006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1a10144750 0x7f1a1006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44900/0 (socket says 172.21.15.165:44900) 2024-04-25T10:06:56.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0f400640 1 -- 172.21.15.165:0/130205918 learned_addr learned my addr 172.21.15.165:0/130205918 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0ea00640 1 --2- 172.21.15.165:0/130205918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a1006eb10 0x7f1a100ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0f400640 1 -- 172.21.15.165:0/130205918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a1006eb10 msgr2=0x7f1a100ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0f400640 1 --2- 172.21.15.165:0/130205918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a1006eb10 0x7f1a100ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.737+0000 7f1a0f400640 1 -- 172.21.15.165:0/130205918 >> v1:172.21.15.172:6789/0 conn(0x7f1a100cee60 legacy=0x7f1a1015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:56.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.738+0000 7f1a0f400640 1 -- 172.21.15.165:0/130205918 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a10071890 con 0x7f1a10144750 2024-04-25T10:06:56.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.739+0000 7f1a0f400640 1 --2- 172.21.15.165:0/130205918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a10144750 0x7f1a1006e5d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009870 tx=0x7f1a0002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53129229842ad84e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.739+0000 7f1a0e000640 1 -- 172.21.15.165:0/130205918 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00037070 con 0x7f1a10144750 2024-04-25T10:06:56.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.739+0000 7f1a0e000640 1 -- 172.21.15.165:0/130205918 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a00032440 con 0x7f1a10144750 2024-04-25T10:06:56.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.739+0000 7f1a0e000640 1 -- 172.21.15.165:0/130205918 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00032810 con 0x7f1a10144750 2024-04-25T10:06:56.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 -- 172.21.15.165:0/130205918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a10144750 msgr2=0x7f1a1006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 --2- 172.21.15.165:0/130205918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a10144750 0x7f1a1006e5d0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f1a00009870 tx=0x7f1a0002a210 comp rx=0 tx=0).stop 2024-04-25T10:06:56.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 -- 172.21.15.165:0/130205918 shutdown_connections 2024-04-25T10:06:56.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 --2- 172.21.15.165:0/130205918 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a1006eb10 0x7f1a100ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 --2- 172.21.15.165:0/130205918 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a10144750 0x7f1a1006e5d0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 -- 172.21.15.165:0/130205918 >> 172.21.15.165:0/130205918 conn(0x7f1a1006abe0 msgr2=0x7f1a1006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 -- 172.21.15.165:0/130205918 shutdown_connections 2024-04-25T10:06:56.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.740+0000 7f1a16200640 1 -- 172.21.15.165:0/130205918 wait complete. 2024-04-25T10:06:56.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.741+0000 7f1a16200640 1 Processor -- start 2024-04-25T10:06:56.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.742+0000 7f1a16200640 1 -- start start 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a16200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 0x7f1a100b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a16200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a100cee60 0x7f1a1015e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a16200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a10144750 0x7f1a1015ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a16200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a10145450 con 0x7f1a1006eb10 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a16200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a1006f020 con 0x7f1a100cee60 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a16200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a101455d0 con 0x7f1a10144750 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 0x7f1a100b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 0x7f1a100b0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44908/0 (socket says 172.21.15.165:44908) 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 -- 172.21.15.165:0/2241147372 learned_addr learned my addr 172.21.15.165:0/2241147372 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 -- 172.21.15.165:0/2241147372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a10144750 msgr2=0x7f1a1015ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a10144750 0x7f1a1015ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 -- 172.21.15.165:0/2241147372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a100cee60 msgr2=0x7f1a1015e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a100cee60 0x7f1a1015e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.743+0000 7f1a0f400640 1 -- 172.21.15.165:0/2241147372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a1015ef60 con 0x7f1a1006eb10 2024-04-25T10:06:56.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.744+0000 7f1a0f400640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 0x7f1a100b0a20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1a0002a250 tx=0x7f1a000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.744+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00041020 con 0x7f1a1006eb10 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.744+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a000306c0 con 0x7f1a1006eb10 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.744+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a1015f210 con 0x7f1a1006eb10 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.744+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a00037040 con 0x7f1a1006eb10 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.744+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a1015f7d0 con 0x7f1a1006eb10 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.745+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a10073990 con 0x7f1a1006eb10 2024-04-25T10:06:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.746+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1a00044020 con 0x7f1a1006eb10 2024-04-25T10:06:56.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.747+0000 7f1a07400640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19f006b4d0 0x7f19f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:56.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.747+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1a000aac60 con 0x7f1a1006eb10 2024-04-25T10:06:56.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.747+0000 7f1a0ea00640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19f006b4d0 0x7f19f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:56.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.748+0000 7f1a0ea00640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19f006b4d0 0x7f19f006d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1a100b14b0 tx=0x7f19fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:56.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.751+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1a0004b030 con 0x7f1a1006eb10 2024-04-25T10:06:56.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.984+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a100d0510 con 0x7f1a1006eb10 2024-04-25T10:06:56.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.985+0000 7f1a07400640 1 -- 172.21.15.165:0/2241147372 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f1a00079880 con 0x7f1a1006eb10 2024-04-25T10:06:56.991 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:56.991 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:51.630520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:56.991 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 29 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19f006b4d0 msgr2=0x7f19f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19f006b4d0 0x7f19f006d990 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1a100b14b0 tx=0x7f19fc008040 comp rx=0 tx=0).stop 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 msgr2=0x7f1a100b0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 0x7f1a100b0a20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1a0002a250 tx=0x7f1a000046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 shutdown_connections 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19f006b4d0 0x7f19f006d990 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1a10144750 0x7f1a1015ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1a100cee60 0x7f1a1015e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 --2- 172.21.15.165:0/2241147372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1a1006eb10 0x7f1a100b0a20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 >> 172.21.15.165:0/2241147372 conn(0x7f1a1006abe0 msgr2=0x7f1a10140e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:56.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 shutdown_connections 2024-04-25T10:06:56.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:56.988+0000 7f1a16200640 1 -- 172.21.15.165:0/2241147372 wait complete. 2024-04-25T10:06:57.013 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_dir_1 2024-04-25T10:06:57.239 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:57.239 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:57.239 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:57.256 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_dir_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:57.256 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:06:57.256 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:06:57.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:06:57.314 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1 bs=1M conv=fdatasync count=1 seek=0) 2024-04-25T10:06:57.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:06:57.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:06:57.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244692 s, 42.9 MB/s 2024-04-25T10:06:57.392 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:06:57.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:06:57.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.584+0000 7f2ee1600640 1 Processor -- start 2024-04-25T10:06:57.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.584+0000 7f2ee1600640 1 -- start start 2024-04-25T10:06:57.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2ee1600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2edc144750 0x7f2edc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2ee1600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc144b30 con 0x7f2edc06eb10 2024-04-25T10:06:57.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2ee1600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2edc0712d0 con 0x7f2edc0cf030 2024-04-25T10:06:57.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2ee1600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2edc0715b0 con 0x7f2edc144750 2024-04-25T10:06:57.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2edaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2edc144750 0x7f2edc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2edaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2edc144750 0x7f2edc06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48276/0 (socket says 172.21.15.165:48276) 2024-04-25T10:06:57.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2edaa00640 1 -- 172.21.15.165:0/3413533324 learned_addr learned my addr 172.21.15.165:0/3413533324 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:57.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.586+0000 7f2eda000640 1 --1- 172.21.15.165:0/3413533324 >> v1:172.21.15.165:6789/0 conn(0x7f2edc06eb10 0x7f2edc0ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33782/0 (socket says 172.21.15.165:33782) 2024-04-25T10:06:57.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.587+0000 7f2edaa00640 1 -- 172.21.15.165:0/3413533324 >> v1:172.21.15.172:6789/0 conn(0x7f2edc0cf030 legacy=0x7f2edc06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:57.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.587+0000 7f2edaa00640 1 -- 172.21.15.165:0/3413533324 >> v1:172.21.15.165:6789/0 conn(0x7f2edc06eb10 legacy=0x7f2edc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.587+0000 7f2edaa00640 1 -- 172.21.15.165:0/3413533324 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc071890 con 0x7f2edc144750 2024-04-25T10:06:57.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.588+0000 7f2edaa00640 1 --2- 172.21.15.165:0/3413533324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc144750 0x7f2edc06e5d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009870 tx=0x7f2ecc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cdd04d8b15f3375 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.588+0000 7f2ed9600640 1 -- 172.21.15.165:0/3413533324 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc037070 con 0x7f2edc144750 2024-04-25T10:06:57.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.588+0000 7f2ed9600640 1 -- 172.21.15.165:0/3413533324 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ecc02ae30 con 0x7f2edc144750 2024-04-25T10:06:57.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.588+0000 7f2ed9600640 1 -- 172.21.15.165:0/3413533324 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ecc032640 con 0x7f2edc144750 2024-04-25T10:06:57.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 -- 172.21.15.165:0/3413533324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc144750 msgr2=0x7f2edc06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3413533324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc144750 0x7f2edc06e5d0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc009870 tx=0x7f2ecc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 -- 172.21.15.165:0/3413533324 shutdown_connections 2024-04-25T10:06:57.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3413533324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc144750 0x7f2edc06e5d0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 -- 172.21.15.165:0/3413533324 >> 172.21.15.165:0/3413533324 conn(0x7f2edc06abe0 msgr2=0x7f2edc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 -- 172.21.15.165:0/3413533324 shutdown_connections 2024-04-25T10:06:57.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.589+0000 7f2ee1600640 1 -- 172.21.15.165:0/3413533324 wait complete. 2024-04-25T10:06:57.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.590+0000 7f2ee1600640 1 Processor -- start 2024-04-25T10:06:57.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.591+0000 7f2ee1600640 1 -- start start 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2ee1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2edc06eb10 0x7f2edc15b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2ee1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 0x7f2edc13c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2ee1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2edc144750 0x7f2edc15e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2ee1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc1455d0 con 0x7f2edc0cf030 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2ee1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc145450 con 0x7f2edc06eb10 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2ee1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2edc15c000 con 0x7f2edc144750 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 0x7f2edc13c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 0x7f2edc13c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48286/0 (socket says 172.21.15.165:48286) 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 -- 172.21.15.165:0/3898252964 learned_addr learned my addr 172.21.15.165:0/3898252964 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2edaa00640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2edc06eb10 0x7f2edc15b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 -- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2edc144750 msgr2=0x7f2edc15e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2edc144750 0x7f2edc15e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 -- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2edc06eb10 msgr2=0x7f2edc15b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2edc06eb10 0x7f2edc15b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.592+0000 7f2eda000640 1 -- 172.21.15.165:0/3898252964 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc15b8c0 con 0x7f2edc0cf030 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.593+0000 7f2eda000640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 0x7f2edc13c270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c570 tx=0x7f2ec800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.593+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8018070 con 0x7f2edc0cf030 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.593+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ec80074e0 con 0x7f2edc0cf030 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.593+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ec8004a20 con 0x7f2edc0cf030 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.593+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2edc15f060 con 0x7f2edc0cf030 2024-04-25T10:06:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.593+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2edc15ed80 con 0x7f2edc0cf030 2024-04-25T10:06:57.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.594+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2edc073990 con 0x7f2edc0cf030 2024-04-25T10:06:57.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.595+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2ec8004bc0 con 0x7f2edc0cf030 2024-04-25T10:06:57.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.596+0000 7f2ed2a00640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:57.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.596+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f2ec80896e0 con 0x7f2edc0cf030 2024-04-25T10:06:57.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.596+0000 7f2edaa00640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:57.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.597+0000 7f2edaa00640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc034000 tx=0x7f2ecc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:57.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.600+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2ec8058390 con 0x7f2edc0cf030 2024-04-25T10:06:57.613 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:57.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:06:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.843+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2edc06c190 con 0x7f2edc0cf030 2024-04-25T10:06:57.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.844+0000 7f2ed2a00640 1 -- 172.21.15.165:0/3898252964 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f2ec8057c90 con 0x7f2edc0cf030 2024-04-25T10:06:57.850 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:06:57.850 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":29,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":29,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:06:51.630520+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":10,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:06:57.850 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 29 2024-04-25T10:06:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.847+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ebc06b500 msgr2=0x7f2ebc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.847+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f2ecc034000 tx=0x7f2ecc02e000 comp rx=0 tx=0).stop 2024-04-25T10:06:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.847+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 msgr2=0x7f2edc13c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.847+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 0x7f2edc13c270 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2ec800c570 tx=0x7f2ec800ca30 comp rx=0 tx=0).stop 2024-04-25T10:06:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 shutdown_connections 2024-04-25T10:06:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ebc06b500 0x7f2ebc06d9c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2edc144750 0x7f2edc15e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2edc0cf030 0x7f2edc13c270 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 --2- 172.21.15.165:0/3898252964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2edc06eb10 0x7f2edc15b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 >> 172.21.15.165:0/3898252964 conn(0x7f2edc06abe0 msgr2=0x7f2edc0af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 shutdown_connections 2024-04-25T10:06:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:57.848+0000 7f2ee1600640 1 -- 172.21.15.165:0/3898252964 wait complete. 2024-04-25T10:06:57.871 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:06:58.094 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:06:58.095 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:06:58.095 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:06:58.110 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /scrub_test_path/ceph-qa-suite/new_file_1' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:06:58.111 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:06:58.111 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:06:58.111 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:06:58.111 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:06:58.111 DEBUG:teuthology.orchestra.run.smithi165:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_test_path/ceph-qa-suite/new_file_1").st_ino) 2024-04-25T10:06:58.111 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:06:58.260 INFO:teuthology.orchestra.run.smithi165.stdout:1099511630190 2024-04-25T10:06:58.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:06:58.274 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:58.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:06:58.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.517+0000 7fe10f200640 1 Processor -- start 2024-04-25T10:06:58.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.517+0000 7fe10f200640 1 -- start start 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.518+0000 7fe10f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe11015dd30 0x7fe1100cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe11006bff0 0x7fe1100b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe11015e110 con 0x7fe1100b5970 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe11006d000 con 0x7fe11015dd30 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100b2fd0 con 0x7fe11006bff0 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe11015dd30 0x7fe1100cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe11015dd30 0x7fe1100cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48290/0 (socket says 172.21.15.165:48290) 2024-04-25T10:06:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10de00640 1 -- 172.21.15.165:0/2556711747 learned_addr learned my addr 172.21.15.165:0/2556711747 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10de00640 1 -- 172.21.15.165:0/2556711747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe11006bff0 msgr2=0x7fe1100b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10d400640 1 --2- 172.21.15.165:0/2556711747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe11006bff0 0x7fe1100b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.519+0000 7fe10de00640 1 --2- 172.21.15.165:0/2556711747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe11006bff0 0x7fe1100b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.520+0000 7fe10de00640 1 -- 172.21.15.165:0/2556711747 >> v1:172.21.15.172:6789/0 conn(0x7fe1100b5970 legacy=0x7fe110161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.520+0000 7fe10de00640 1 -- 172.21.15.165:0/2556711747 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe11006c480 con 0x7fe11015dd30 2024-04-25T10:06:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.521+0000 7fe10de00640 1 --2- 172.21.15.165:0/2556711747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe11015dd30 0x7fe1100cebd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe100009870 tx=0x7fe10002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1a0953364c6ecb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.521+0000 7fe10ca00640 1 -- 172.21.15.165:0/2556711747 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100037070 con 0x7fe11015dd30 2024-04-25T10:06:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.521+0000 7fe10ca00640 1 -- 172.21.15.165:0/2556711747 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100032440 con 0x7fe11015dd30 2024-04-25T10:06:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.521+0000 7fe10ca00640 1 -- 172.21.15.165:0/2556711747 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100032810 con 0x7fe11015dd30 2024-04-25T10:06:58.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 -- 172.21.15.165:0/2556711747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe11015dd30 msgr2=0x7fe1100cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 --2- 172.21.15.165:0/2556711747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe11015dd30 0x7fe1100cebd0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fe100009870 tx=0x7fe10002a210 comp rx=0 tx=0).stop 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 -- 172.21.15.165:0/2556711747 shutdown_connections 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 --2- 172.21.15.165:0/2556711747 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe11006bff0 0x7fe1100b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 --2- 172.21.15.165:0/2556711747 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe11015dd30 0x7fe1100cebd0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 -- 172.21.15.165:0/2556711747 >> 172.21.15.165:0/2556711747 conn(0x7fe1100d0610 msgr2=0x7fe1100d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 -- 172.21.15.165:0/2556711747 shutdown_connections 2024-04-25T10:06:58.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.522+0000 7fe10f200640 1 -- 172.21.15.165:0/2556711747 wait complete. 2024-04-25T10:06:58.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.524+0000 7fe10f200640 1 Processor -- start 2024-04-25T10:06:58.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.524+0000 7fe10f200640 1 -- start start 2024-04-25T10:06:58.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.525+0000 7fe10f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1100b5970 0x7fe1101f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.525+0000 7fe10f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe11015dd30 0x7fe110159020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.525+0000 7fe10f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 0x7fe1100b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.525+0000 7fe10f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11006cd20 con 0x7fe1100cc040 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.525+0000 7fe10f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe11006ca40 con 0x7fe11015dd30 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100cf110 con 0x7fe1100b5970 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 0x7fe1100b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 0x7fe1100b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48302/0 (socket says 172.21.15.165:48302) 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 -- 172.21.15.165:0/2329902315 learned_addr learned my addr 172.21.15.165:0/2329902315 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10d400640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe11015dd30 0x7fe110159020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10de00640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1100b5970 0x7fe1101f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 -- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1100b5970 msgr2=0x7fe1101f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1100b5970 0x7fe1101f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 -- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe11015dd30 msgr2=0x7fe110159020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe11015dd30 0x7fe110159020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10e800640 1 -- 172.21.15.165:0/2329902315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1100cc4f0 con 0x7fe1100cc040 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10de00640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1100b5970 0x7fe1101f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.526+0000 7fe10d400640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe11015dd30 0x7fe110159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.527+0000 7fe10e800640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 0x7fe1100b3bf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c780 tx=0x7fe10800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.527+0000 7fe106000640 1 -- 172.21.15.165:0/2329902315 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108004b60 con 0x7fe1100cc040 2024-04-25T10:06:58.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.527+0000 7fe106000640 1 -- 172.21.15.165:0/2329902315 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108004d00 con 0x7fe1100cc040 2024-04-25T10:06:58.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.527+0000 7fe106000640 1 -- 172.21.15.165:0/2329902315 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10800aa30 con 0x7fe1100cc040 2024-04-25T10:06:58.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.527+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1100cd930 con 0x7fe1100cc040 2024-04-25T10:06:58.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.527+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1100cce50 con 0x7fe1100cc040 2024-04-25T10:06:58.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.530+0000 7fe106000640 1 -- 172.21.15.165:0/2329902315 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe10800abd0 con 0x7fe1100cc040 2024-04-25T10:06:58.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.531+0000 7fe106000640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe0f006b4d0 0x7fe0f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.531+0000 7fe106000640 1 -- 172.21.15.165:0/2329902315 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe10808a8f0 con 0x7fe1100cc040 2024-04-25T10:06:58.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.531+0000 7fe10de00640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe0f006b4d0 0x7fe0f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.532+0000 7fe117dc8740 1 Processor -- start 2024-04-25T10:06:58.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.532+0000 7fe117dc8740 1 -- start start 2024-04-25T10:06:58.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.532+0000 7fe10de00640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe0f006b4d0 0x7fe0f006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe100034000 tx=0x7fe10002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159cce53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cd02e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159ccbacf0 0x56159cd00f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe117dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159cc1dd00 con 0x56159cce5010 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159cbc6f50 con 0x56159cd028b0 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56159cc19da0 con 0x56159ccbacf0 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159cce53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159cce53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48304/0 (socket says 172.21.15.165:48304) 2024-04-25T10:06:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 -- 172.21.15.165:0/3033450420 learned_addr learned my addr 172.21.15.165:0/3033450420 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 -- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159ccbacf0 msgr2=0x56159cd00f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10e800640 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159ccbacf0 0x56159cd00f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10d400640 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cd02e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159ccbacf0 0x56159cd00f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 -- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 msgr2=0x56159cd02e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cd02e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.533+0000 7fe10de00640 1 -- 172.21.15.165:0/3033450420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159cbea130 con 0x56159cce5010 2024-04-25T10:06:58.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.534+0000 7fe10de00640 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159cce53f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe10002a250 tx=0x7fe10002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3331602f13228c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.534+0000 7fe0faa00640 1 -- 172.21.15.165:0/3033450420 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10004d070 con 0x56159cce5010 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.534+0000 7fe0faa00640 1 -- 172.21.15.165:0/3033450420 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe100032e40 con 0x56159cce5010 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.534+0000 7fe0faa00640 1 -- 172.21.15.165:0/3033450420 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe100037040 con 0x56159cce5010 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 -- 172.21.15.165:0/3033450420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 msgr2=0x56159cce53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159cce53f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7fe10002a250 tx=0x7fe10002c000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 -- 172.21.15.165:0/3033450420 shutdown_connections 2024-04-25T10:06:58.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159ccbacf0 0x56159cd00f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cd02e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 --2- 172.21.15.165:0/3033450420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159cce53f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 -- 172.21.15.165:0/3033450420 >> 172.21.15.165:0/3033450420 conn(0x56159ccb7620 msgr2=0x56159ccffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 -- 172.21.15.165:0/3033450420 shutdown_connections 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.535+0000 7fe117dc8740 1 -- 172.21.15.165:0/3033450420 wait complete. 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.536+0000 7fe117dc8740 1 -- start start 2024-04-25T10:06:58.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.536+0000 7fe117dc8740 1 Processor -- start 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159ccdb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cce2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce28d0 0x56159cce8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159ccbba90 con 0x56159cce5010 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159cbbc530 con 0x56159cd028b0 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56159ccb5550 con 0x56159cce28d0 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe117dc8740 10 client.0 _ll_register_callbacks cb 0x56159cc15430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe10de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159ccdb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe10de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159ccdb360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48314/0 (socket says 172.21.15.165:48314) 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.537+0000 7fe10de00640 1 -- 172.21.15.165:0/1358667881 learned_addr learned my addr 172.21.15.165:0/1358667881 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10d400640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cce2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10e800640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce28d0 0x56159cce8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10de00640 1 -- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce28d0 msgr2=0x56159cce8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10de00640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce28d0 0x56159cce8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10de00640 1 -- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 msgr2=0x56159cce2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10de00640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cce2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10de00640 1 -- 172.21.15.165:0/1358667881 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159cbe4dd0 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10d400640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cce2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe10de00640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159ccdb360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe100009990 tx=0x7fe100004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5e21bec87a983be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe0ef400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe0ef400640 1 -- 172.21.15.165:0/1358667881 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10004d030 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe0ef400640 1 -- 172.21.15.165:0/1358667881 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe100043040 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.538+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56159cc2b240 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe0ef400640 1 -- 172.21.15.165:0/1358667881 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe100037070 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 10 client.6306 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56159c9323f0 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe0ef400640 1 -- 172.21.15.165:0/1358667881 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56159cc2b240 con 0x56159cce5010 2024-04-25T10:06:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe0ef400640 1 -- 172.21.15.165:0/1358667881 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe10005e020 con 0x56159cce5010 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 10 client.6306 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 10 client.6306 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 20 client.6306 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cce6ed0 0x56159cce72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 4 client.6306 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.539+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56159cc25590 con 0x56159cce6ed0 2024-04-25T10:06:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.540+0000 7fe10d400640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cce6ed0 0x56159cce72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.541+0000 7fe10d400640 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cce6ed0 0x56159cce72f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4e1da5a3c07e255 server_cookie=34e57ac2b827feed in_seq=0 out_seq=0 2024-04-25T10:06:58.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.541+0000 7fe0ef400640 10 client.6306 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.542+0000 7fe0ef400640 1 -- 172.21.15.165:0/1358667881 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56159cc25590 con 0x56159cce6ed0 2024-04-25T10:06:58.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.542+0000 7fe0ef400640 10 client.6306 handle_command_reply: tid=0 2024-04-25T10:06:58.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.542+0000 7fe117dc8740 1 client.6306 shutdown 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.542+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 msgr2=0x56159ccdb360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.542+0000 7fe117dc8740 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159ccdb360 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fe100009990 tx=0x7fe100004730 comp rx=0 tx=0).stop 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 shutdown_connections 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cce6ed0 0x56159cce72f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce28d0 0x56159cce8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cd028b0 0x56159cce2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 --2- 172.21.15.165:0/1358667881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cce5010 0x56159ccdb360 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe0ef400640 0 client.6306 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 >> 172.21.15.165:0/1358667881 conn(0x56159ccbacf0 msgr2=0x56159cd03c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 shutdown_connections 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 1 -- 172.21.15.165:0/1358667881 wait complete. 2024-04-25T10:06:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.543+0000 7fe117dc8740 20 client.6306 trim_cache size 0 max 16384 2024-04-25T10:06:58.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.556+0000 7fe117dc8740 1 Processor -- start 2024-04-25T10:06:58.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.556+0000 7fe117dc8740 1 -- start start 2024-04-25T10:06:58.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cce1f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd0faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159cd10030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe117dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe110200430 con 0x56159cd028b0 2024-04-25T10:06:58.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1101ff590 con 0x56159cce28d0 2024-04-25T10:06:58.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1100cb980 con 0x56159cce5010 2024-04-25T10:06:58.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe10e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159cd10030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe10e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159cd10030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48330/0 (socket says 172.21.15.165:48330) 2024-04-25T10:06:58.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.557+0000 7fe10e800640 1 -- 172.21.15.165:0/956523865 learned_addr learned my addr 172.21.15.165:0/956523865 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10d400640 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd0faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10de00640 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cce1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10e800640 1 -- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 msgr2=0x56159cd0faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10e800640 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd0faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10e800640 1 -- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 msgr2=0x56159cce1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10e800640 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cce1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10e800640 1 -- 172.21.15.165:0/956523865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56159c9fc7d0 con 0x56159cd028b0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe10e800640 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159cd10030 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe108088b30 tx=0x7fe108058770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75848561c891751e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe0fa000640 1 -- 172.21.15.165:0/956523865 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10805cf60 con 0x56159cd028b0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe0fa000640 1 -- 172.21.15.165:0/956523865 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108087070 con 0x56159cd028b0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.558+0000 7fe0fa000640 1 -- 172.21.15.165:0/956523865 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108061520 con 0x56159cd028b0 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 -- 172.21.15.165:0/956523865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 msgr2=0x56159cd10030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159cd10030 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fe108088b30 tx=0x7fe108058770 comp rx=0 tx=0).stop 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 -- 172.21.15.165:0/956523865 shutdown_connections 2024-04-25T10:06:58.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159cd10030 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd0faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 --2- 172.21.15.165:0/956523865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cce1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 -- 172.21.15.165:0/956523865 >> 172.21.15.165:0/956523865 conn(0x56159ccbacf0 msgr2=0x56159ccdb170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 -- 172.21.15.165:0/956523865 shutdown_connections 2024-04-25T10:06:58.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.559+0000 7fe117dc8740 1 -- 172.21.15.165:0/956523865 wait complete. 2024-04-25T10:06:58.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.560+0000 7fe117dc8740 1 -- start start 2024-04-25T10:06:58.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.560+0000 7fe117dc8740 1 Processor -- start 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cd00410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd00950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159ccdded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56159ccbbb80 con 0x56159cd028b0 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1101fefc0 con 0x56159cce28d0 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56159cc25590 con 0x56159cce5010 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe117dc8740 10 client.0 _ll_register_callbacks cb 0x56159ccb5a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159ccdded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159ccdded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48332/0 (socket says 172.21.15.165:48332) 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 -- 172.21.15.165:0/2748683654 learned_addr learned my addr 172.21.15.165:0/2748683654 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 -- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 msgr2=0x56159cd00950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd00950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 -- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 msgr2=0x56159cd00410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cd00410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.561+0000 7fe10e800640 1 -- 172.21.15.165:0/2748683654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56159ccf7760 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe10e800640 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159ccdded0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe108084040 tx=0x7fe10806d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a13613c0c5bfe9b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe0ef400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe0ef400640 1 -- 172.21.15.165:0/2748683654 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe10807d060 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe0ef400640 1 -- 172.21.15.165:0/2748683654 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe108022da0 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe108087070 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe0ef400640 1 -- 172.21.15.165:0/2748683654 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe10805cf60 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe117dc8740 10 client.6312 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x56159ccba4a0 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe0ef400640 1 -- 172.21.15.165:0/2748683654 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108087070 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe0ef400640 1 -- 172.21.15.165:0/2748683654 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe108077ce0 con 0x56159cd028b0 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe117dc8740 10 client.6312 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe117dc8740 10 client.6312 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.562+0000 7fe117dc8740 20 client.6312 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.563+0000 7fe117dc8740 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cd6a960 0x56159ccde410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.563+0000 7fe117dc8740 4 client.6312 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:06:58.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.563+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56159ccb6750 con 0x56159cd6a960 2024-04-25T10:06:58.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.563+0000 7fe10de00640 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cd6a960 0x56159ccde410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.564+0000 7fe10de00640 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cd6a960 0x56159ccde410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2aef0bd0bc501b2 server_cookie=ef14c3b3bc2933c2 in_seq=0 out_seq=0 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.564+0000 7fe0ef400640 10 client.6312 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe0ef400640 1 -- 172.21.15.165:0/2748683654 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56159ccb6750 con 0x56159cd6a960 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe0ef400640 10 client.6312 handle_command_reply: tid=0 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 client.6312 shutdown 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 msgr2=0x56159ccdded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159ccdded0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7fe108084040 tx=0x7fe10806d000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 shutdown_connections 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x56159cd6a960 0x56159ccde410 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56159cd028b0 0x56159ccdded0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56159cce5010 0x56159cd00950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 --2- 172.21.15.165:0/2748683654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56159cce28d0 0x56159cd00410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe0ef400640 0 client.6312 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 >> 172.21.15.165:0/2748683654 conn(0x56159ccbacf0 msgr2=0x56159cd0faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 shutdown_connections 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 1 -- 172.21.15.165:0/2748683654 wait complete. 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.565+0000 7fe117dc8740 20 client.6312 trim_cache size 0 max 16384 2024-04-25T10:06:58.573 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-25T10:06:58.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe0f006b4d0 msgr2=0x7fe0f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe0f006b4d0 0x7fe0f006d990 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fe100034000 tx=0x7fe10002e000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 msgr2=0x7fe1100b3bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 0x7fe1100b3bf0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c780 tx=0x7fe10800cc40 comp rx=0 tx=0).stop 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 shutdown_connections 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe0f006b4d0 0x7fe0f006d990 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1100cc040 0x7fe1100b3bf0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe11015dd30 0x7fe110159020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 --2- 172.21.15.165:0/2329902315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1100b5970 0x7fe1101f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 >> 172.21.15.165:0/2329902315 conn(0x7fe1100d0610 msgr2=0x7fe1100b6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 shutdown_connections 2024-04-25T10:06:58.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.566+0000 7fe10f200640 1 -- 172.21.15.165:0/2329902315 wait complete. 2024-04-25T10:06:58.588 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data rmxattr 1000000096e.00000000 parent 2024-04-25T10:06:58.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.671+0000 7f9c1c120f40 1 Processor -- start 2024-04-25T10:06:58.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.671+0000 7f9c1c120f40 1 -- start start 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c1c120f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c91e90 0x560b36ca4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c1c120f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36c94590 0x560b36d38c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c1c120f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 0x560b36c63080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c1c120f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36c93130 con 0x560b36d385b0 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c1c120f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36c0add0 con 0x560b36c94590 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c1c120f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b36c933d0 con 0x560b36c91e90 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c19400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 0x560b36c63080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c19400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 0x560b36c63080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48358/0 (socket says 172.21.15.165:48358) 2024-04-25T10:06:58.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.672+0000 7f9c19400640 1 -- 172.21.15.165:0/1890112935 learned_addr learned my addr 172.21.15.165:0/1890112935 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c19400640 1 -- 172.21.15.165:0/1890112935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c91e90 msgr2=0x560b36ca4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c19400640 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c91e90 0x560b36ca4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c19400640 1 -- 172.21.15.165:0/1890112935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36c94590 msgr2=0x560b36d38c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c19400640 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36c94590 0x560b36d38c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c19400640 1 -- 172.21.15.165:0/1890112935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b36c94360 con 0x560b36d385b0 2024-04-25T10:06:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c19400640 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 0x560b36c63080 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9c14025ae0 tx=0x7f9c14051000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ed222847074fb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c12a00640 1 -- 172.21.15.165:0/1890112935 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14067070 con 0x560b36d385b0 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c12a00640 1 -- 172.21.15.165:0/1890112935 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c140585f0 con 0x560b36d385b0 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.673+0000 7f9c12a00640 1 -- 172.21.15.165:0/1890112935 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14058db0 con 0x560b36d385b0 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 -- 172.21.15.165:0/1890112935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 msgr2=0x560b36c63080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 0x560b36c63080 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f9c14025ae0 tx=0x7f9c14051000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 -- 172.21.15.165:0/1890112935 shutdown_connections 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36d385b0 0x560b36c63080 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36c94590 0x560b36d38c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/1890112935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c91e90 0x560b36ca4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 -- 172.21.15.165:0/1890112935 >> 172.21.15.165:0/1890112935 conn(0x560b36c90fd0 msgr2=0x560b36c913b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 -- 172.21.15.165:0/1890112935 shutdown_connections 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.674+0000 7f9c1c120f40 1 -- 172.21.15.165:0/1890112935 wait complete. 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.675+0000 7f9c1c120f40 1 Processor -- start 2024-04-25T10:06:58.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.675+0000 7f9c1c120f40 1 -- start start 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c1c120f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36c91e90 0x560b36ca9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c1c120f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c94590 0x560b36caa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c1c120f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36d385b0 0x560b36ca8120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c1c120f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36ca5290 con 0x560b36c91e90 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c1c120f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b36ca5530 con 0x560b36d385b0 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c1c120f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560b36ca5810 con 0x560b36c94590 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c18a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36c91e90 0x560b36ca9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c18a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36c91e90 0x560b36ca9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48366/0 (socket says 172.21.15.165:48366) 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c18a00640 1 -- 172.21.15.165:0/683362902 learned_addr learned my addr 172.21.15.165:0/683362902 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c19400640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36d385b0 0x560b36ca8120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c13e00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c94590 0x560b36caa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c18a00640 1 -- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c94590 msgr2=0x560b36caa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.676+0000 7f9c18a00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c94590 0x560b36caa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c18a00640 1 -- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36d385b0 msgr2=0x560b36ca8120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c18a00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36d385b0 0x560b36ca8120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c18a00640 1 -- 172.21.15.165:0/683362902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b36d37720 con 0x560b36c91e90 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c13e00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560b36c94590 0x560b36caa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c19400640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560b36d385b0 0x560b36ca8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c18a00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36c91e90 0x560b36ca9b40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009da0 tx=0x7f9c0c006750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c11600640 1 -- 172.21.15.165:0/683362902 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c017070 con 0x560b36c91e90 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c11600640 1 -- 172.21.15.165:0/683362902 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c0c0090d0 con 0x560b36c91e90 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x560b36d37440 con 0x560b36c91e90 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c11600640 1 -- 172.21.15.165:0/683362902 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c0c012630 con 0x560b36c91e90 2024-04-25T10:06:58.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.677+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b36ca94e0 con 0x560b36c91e90 2024-04-25T10:06:58.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.679+0000 7f9c11600640 1 -- 172.21.15.165:0/683362902 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c0c01a070 con 0x560b36c91e90 2024-04-25T10:06:58.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.680+0000 7f9c11600640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bf8069eb0 0x7f9bf806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.680+0000 7f9c11600640 1 -- 172.21.15.165:0/683362902 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f9c0c088180 con 0x560b36c91e90 2024-04-25T10:06:58.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.680+0000 7f9c13e00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bf8069eb0 0x7f9bf806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.681+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x560b36d32cd0 0x560b36ca8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.681+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 --> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] -- osd_op(unknown.0.0:1 8.27 8:e5ebf85a:::1000000096e.00000000:head [rmxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e56) v9 -- 0x560b36d322a0 con 0x560b36d32cd0 2024-04-25T10:06:58.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.681+0000 7f9c19400640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x560b36d32cd0 0x560b36ca8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.681+0000 7f9c13e00640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bf8069eb0 0x7f9bf806c370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x560b36d38ee0 tx=0x7f9c04001420 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.682+0000 7f9c19400640 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x560b36d32cd0 0x560b36ca8b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c19400640 1 -- 172.21.15.165:0/683362902 <== osd.0 v2:172.21.15.165:6800/4162285384 1 ==== osd_op_reply(1 1000000096e.00000000 [rmxattr] v56'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9c1404e040 con 0x560b36d32cd0 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x560b36d32cd0 msgr2=0x560b36ca8b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x560b36d32cd0 0x560b36ca8b70 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bf8069eb0 msgr2=0x7f9bf806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bf8069eb0 0x7f9bf806c370 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x560b36d38ee0 tx=0x7f9c04001420 comp rx=0 tx=0).stop 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36c91e90 msgr2=0x560b36ca9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c1c120f40 1 --2- 172.21.15.165:0/683362902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560b36c91e90 0x560b36ca9b40 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f9c0c009da0 tx=0x7f9c0c006750 comp rx=0 tx=0).stop 2024-04-25T10:06:58.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.683+0000 7f9c19400640 1 -- 172.21.15.165:0/683362902 reap_dead start 2024-04-25T10:06:58.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.684+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 shutdown_connections 2024-04-25T10:06:58.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.684+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 >> 172.21.15.165:0/683362902 conn(0x560b36c90fd0 msgr2=0x560b36ca43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.684+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 shutdown_connections 2024-04-25T10:06:58.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.684+0000 7f9c1c120f40 1 -- 172.21.15.165:0/683362902 wait complete. 2024-04-25T10:06:58.694 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' 2024-04-25T10:06:58.694 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:06:58.694 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start /scrub_test_path/ceph-qa-suite/new_file_1 2024-04-25T10:06:58.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.877+0000 7f67a5600640 1 Processor -- start 2024-04-25T10:06:58.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.877+0000 7f67a5600640 1 -- start start 2024-04-25T10:06:58.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f67a5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67a0145140 0x7f67a00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f67a5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f67a00c5200 0x7f67a015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f67a5600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a00c5650 con 0x7f67a00ce950 2024-04-25T10:06:58.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f67a5600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a00ced90 con 0x7f67a0145140 2024-04-25T10:06:58.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f67a5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a0145520 con 0x7f67a00c5200 2024-04-25T10:06:58.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f679fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67a0145140 0x7f67a00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f679fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67a0145140 0x7f67a00ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48382/0 (socket says 172.21.15.165:48382) 2024-04-25T10:06:58.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f679fe00640 1 -- 172.21.15.165:0/1232607146 learned_addr learned my addr 172.21.15.165:0/1232607146 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.879+0000 7f67a4c00640 1 --2- 172.21.15.165:0/1232607146 >> v2:172.21.15.172:3300/0 conn(0x7f67a00c5200 0x7f67a015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.880+0000 7f679fe00640 1 -- 172.21.15.165:0/1232607146 >> v2:172.21.15.172:3300/0 conn(0x7f67a00c5200 msgr2=0x7f67a015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.880+0000 7f679fe00640 1 --2- 172.21.15.165:0/1232607146 >> v2:172.21.15.172:3300/0 conn(0x7f67a00c5200 0x7f67a015bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.880+0000 7f679fe00640 1 -- 172.21.15.165:0/1232607146 >> v1:172.21.15.172:6789/0 conn(0x7f67a00ce950 legacy=0x7f67a00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.880+0000 7f679fe00640 1 -- 172.21.15.165:0/1232607146 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a0071270 con 0x7f67a0145140 2024-04-25T10:06:58.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.881+0000 7f679fe00640 1 --2- 172.21.15.165:0/1232607146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a0145140 0x7f67a00ce410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f67a006bb90 tx=0x7f679002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=690a478b1f96832b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.881+0000 7f679ea00640 1 -- 172.21.15.165:0/1232607146 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790037070 con 0x7f67a0145140 2024-04-25T10:06:58.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.881+0000 7f679ea00640 1 -- 172.21.15.165:0/1232607146 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790032440 con 0x7f67a0145140 2024-04-25T10:06:58.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.881+0000 7f679ea00640 1 -- 172.21.15.165:0/1232607146 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790032810 con 0x7f67a0145140 2024-04-25T10:06:58.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 -- 172.21.15.165:0/1232607146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a0145140 msgr2=0x7f67a00ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 --2- 172.21.15.165:0/1232607146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a0145140 0x7f67a00ce410 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f67a006bb90 tx=0x7f679002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 -- 172.21.15.165:0/1232607146 shutdown_connections 2024-04-25T10:06:58.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 --2- 172.21.15.165:0/1232607146 >> v2:172.21.15.172:3300/0 conn(0x7f67a00c5200 0x7f67a015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 --2- 172.21.15.165:0/1232607146 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a0145140 0x7f67a00ce410 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 -- 172.21.15.165:0/1232607146 >> 172.21.15.165:0/1232607146 conn(0x7f67a0061cc0 msgr2=0x7f67a006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 -- 172.21.15.165:0/1232607146 shutdown_connections 2024-04-25T10:06:58.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.882+0000 7f67a5600640 1 -- 172.21.15.165:0/1232607146 wait complete. 2024-04-25T10:06:58.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.884+0000 7f67a5600640 1 Processor -- start 2024-04-25T10:06:58.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.884+0000 7f67a5600640 1 -- start start 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f67a5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 0x7f67a00cae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f67a5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67a00ce950 0x7f67a00cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f67a5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67a0145140 0x7f67a00b0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f67a5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0071b10 con 0x7f67a00c5200 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f67a5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a0071830 con 0x7f67a00ce950 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f67a5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015c4f0 con 0x7f67a0145140 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 0x7f67a00cae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 0x7f67a00cae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48388/0 (socket says 172.21.15.165:48388) 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 -- 172.21.15.165:0/1701483307 learned_addr learned my addr 172.21.15.165:0/1701483307 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 -- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67a0145140 msgr2=0x7f67a00b0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67a0145140 0x7f67a00b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679f400640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67a00ce950 0x7f67a00cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 -- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67a00ce950 msgr2=0x7f67a00cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67a00ce950 0x7f67a00cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.885+0000 7f679fe00640 1 -- 172.21.15.165:0/1701483307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00cccc0 con 0x7f67a00c5200 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.886+0000 7f679fe00640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 0x7f67a00cae10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6790000c00 tx=0x7f67900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.886+0000 7f6797e00640 1 -- 172.21.15.165:0/1701483307 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790041020 con 0x7f67a00c5200 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.886+0000 7f6797e00640 1 -- 172.21.15.165:0/1701483307 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900306c0 con 0x7f67a00c5200 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.886+0000 7f6797e00640 1 -- 172.21.15.165:0/1701483307 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790037040 con 0x7f67a00c5200 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.886+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a00ccfa0 con 0x7f67a00c5200 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.886+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a00cd560 con 0x7f67a00c5200 2024-04-25T10:06:58.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.888+0000 7f6797e00640 1 -- 172.21.15.165:0/1701483307 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6790044020 con 0x7f67a00c5200 2024-04-25T10:06:58.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.889+0000 7f6797e00640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f678006b4d0 0x7f678006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.889+0000 7f6797e00640 1 -- 172.21.15.165:0/1701483307 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f67900aabd0 con 0x7f67a00c5200 2024-04-25T10:06:58.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.889+0000 7f679f400640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f678006b4d0 0x7f678006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.890+0000 7f67a9a7c740 1 Processor -- start 2024-04-25T10:06:58.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.890+0000 7f679f400640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f678006b4d0 0x7f678006d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f67a00b4610 tx=0x7f678c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.890+0000 7f67a9a7c740 1 -- start start 2024-04-25T10:06:58.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb871520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb86c050 0x5557eb86fd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb7d4d80 con 0x5557eb870280 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb77dcc0 con 0x5557eb86c050 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557eb7d0e20 con 0x5557eb871140 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48402/0 (socket says 172.21.15.165:48402) 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.891+0000 7f67a4c00640 1 -- 172.21.15.165:0/2141703857 learned_addr learned my addr 172.21.15.165:0/2141703857 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f679fe00640 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb871520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f679f400640 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb86c050 0x5557eb86fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f67a4c00640 1 -- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 msgr2=0x5557eb871520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f67a4c00640 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb871520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f67a4c00640 1 -- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb86c050 msgr2=0x5557eb86fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f67a4c00640 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb86c050 0x5557eb86fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f67a4c00640 1 -- 172.21.15.165:0/2141703857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557eb5b5c00 con 0x5557eb870280 2024-04-25T10:06:58.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f679f400640 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb86c050 0x5557eb86fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:58.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f67a4c00640 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8706c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6798009fb0 tx=0x7f67980077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2a10bef4be078f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.892+0000 7f678be00640 1 -- 172.21.15.165:0/2141703857 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798019070 con 0x5557eb870280 2024-04-25T10:06:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f678be00640 1 -- 172.21.15.165:0/2141703857 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798012710 con 0x5557eb870280 2024-04-25T10:06:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f678be00640 1 -- 172.21.15.165:0/2141703857 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798012ac0 con 0x5557eb870280 2024-04-25T10:06:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2141703857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 msgr2=0x5557eb8706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8706c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6798009fb0 tx=0x7f67980077c0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2141703857 shutdown_connections 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8706c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb86c050 0x5557eb86fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2141703857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb871520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2141703857 >> 172.21.15.165:0/2141703857 conn(0x5557eb86e4d0 msgr2=0x5557eb8b6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2141703857 shutdown_connections 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.893+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2141703857 wait complete. 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.894+0000 7f67a9a7c740 1 -- start start 2024-04-25T10:06:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.894+0000 7f67a9a7c740 1 Processor -- start 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.895+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8be8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.895+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb8bee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb8b9950 0x5557eb8b9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb89ccd0 con 0x5557eb870280 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb773240 con 0x5557eb8b9950 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557eb89c150 con 0x5557eb871140 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f67a9a7c740 10 client.0 _ll_register_callbacks cb 0x5557eb7cc280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8be8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8be8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48416/0 (socket says 172.21.15.165:48416) 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 -- 172.21.15.165:0/3844288219 learned_addr learned my addr 172.21.15.165:0/3844288219 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 -- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 msgr2=0x5557eb8bee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb8bee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 -- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb8b9950 msgr2=0x5557eb8b9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb8b9950 0x5557eb8b9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 -- 172.21.15.165:0/3844288219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557eb4a94f0 con 0x5557eb870280 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f679fe00640 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8be8f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f67900a8ed0 tx=0x7f6790031e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7eace18f1911fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.896+0000 7f6788c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f6788c00640 1 -- 172.21.15.165:0/3844288219 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790081600 con 0x5557eb870280 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f6788c00640 1 -- 172.21.15.165:0/3844288219 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f67900785d0 con 0x5557eb870280 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6798012710 con 0x5557eb870280 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f6788c00640 1 -- 172.21.15.165:0/3844288219 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f679007d2d0 con 0x5557eb870280 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 10 client.6336 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5557eb616390 con 0x5557eb870280 2024-04-25T10:06:58.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f6788c00640 1 -- 172.21.15.165:0/3844288219 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798012710 con 0x5557eb870280 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f6788c00640 1 -- 172.21.15.165:0/3844288219 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f679003a030 con 0x5557eb870280 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 10 client.6336 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 10 client.6336 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 20 client.6336 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb88e510 0x5557eb8d44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 4 client.6336 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.897+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5557eb7e22c0 con 0x5557eb88e510 2024-04-25T10:06:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.898+0000 7f679f400640 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb88e510 0x5557eb8d44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.899+0000 7f679f400640 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb88e510 0x5557eb8d44a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aae00c8a0b30fa5b server_cookie=59ffd923701351ad in_seq=0 out_seq=0 2024-04-25T10:06:58.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.899+0000 7f6788c00640 10 client.6336 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f6788c00640 1 -- 172.21.15.165:0/3844288219 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5557eb7e22c0 con 0x5557eb88e510 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f6788c00640 10 client.6336 handle_command_reply: tid=0 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 client.6336 shutdown 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 msgr2=0x5557eb8be8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8be8f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f67900a8ed0 tx=0x7f6790031e80 comp rx=0 tx=0).stop 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 shutdown_connections 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb88e510 0x5557eb8d44a0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb8b9950 0x5557eb8b9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb8bee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/3844288219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb870280 0x5557eb8be8f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f6788c00640 0 client.6336 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 >> 172.21.15.165:0/3844288219 conn(0x5557eb86c050 msgr2=0x5557eb86b6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 shutdown_connections 2024-04-25T10:06:58.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 1 -- 172.21.15.165:0/3844288219 wait complete. 2024-04-25T10:06:58.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.901+0000 7f67a9a7c740 20 client.6336 trim_cache size 0 max 16384 2024-04-25T10:06:58.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.918+0000 7f67a9a7c740 1 Processor -- start 2024-04-25T10:06:58.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.918+0000 7f67a9a7c740 1 -- start start 2024-04-25T10:06:58.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb8919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb891f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8926b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a02004f0 con 0x5557eb8b9950 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01ff670 con 0x5557eb870280 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01fea70 con 0x5557eb871140 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8926b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8926b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48432/0 (socket says 172.21.15.165:48432) 2024-04-25T10:06:58.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a4c00640 1 -- 172.21.15.165:0/2134567876 learned_addr learned my addr 172.21.15.165:0/2134567876 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a4c00640 1 -- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 msgr2=0x5557eb891f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a4c00640 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb891f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f67a4c00640 1 -- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 msgr2=0x5557eb8919d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.919+0000 7f679fe00640 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb8919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.920+0000 7f67a4c00640 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb8919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.920+0000 7f67a4c00640 1 -- 172.21.15.165:0/2134567876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557eb8ba140 con 0x5557eb8b9950 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.920+0000 7f67a4c00640 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8926b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f67980094c0 tx=0x7f6798011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0290cc44d68ae21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.920+0000 7f678b400640 1 -- 172.21.15.165:0/2134567876 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798019040 con 0x5557eb8b9950 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.920+0000 7f678b400640 1 -- 172.21.15.165:0/2134567876 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798003b90 con 0x5557eb8b9950 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.920+0000 7f678b400640 1 -- 172.21.15.165:0/2134567876 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798004040 con 0x5557eb8b9950 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2134567876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 msgr2=0x5557eb8926b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8926b0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f67980094c0 tx=0x7f6798011000 comp rx=0 tx=0).stop 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2134567876 shutdown_connections 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8926b0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb891f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/2134567876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb8919d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2134567876 >> 172.21.15.165:0/2134567876 conn(0x5557eb86c050 msgr2=0x5557eb8717c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2134567876 shutdown_connections 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 -- 172.21.15.165:0/2134567876 wait complete. 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.921+0000 7f67a9a7c740 1 -- start start 2024-04-25T10:06:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.922+0000 7f67a9a7c740 1 Processor -- start 2024-04-25T10:06:58.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb898480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb9122d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557eb808d10 con 0x5557eb8b9950 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01ff070 con 0x5557eb870280 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557eb8b9f80 con 0x5557eb871140 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a9a7c740 10 client.0 _ll_register_callbacks cb 0x7f6798009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8c9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48434/0 (socket says 172.21.15.165:48434) 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 -- 172.21.15.165:0/340410095 learned_addr learned my addr 172.21.15.165:0/340410095 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f679fe00640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb898480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f679f400640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb9122d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 -- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 msgr2=0x5557eb9122d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb9122d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 -- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 msgr2=0x5557eb898480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:58.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb898480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.923+0000 7f67a4c00640 1 -- 172.21.15.165:0/340410095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557eb482670 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f67a4c00640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8c9460 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6798009950 tx=0x7f6798009450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b5dab5ab7dc883 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f6788c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f6788c00640 1 -- 172.21.15.165:0/340410095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798019040 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f6788c00640 1 -- 172.21.15.165:0/340410095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6798003990 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6798003b90 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f6788c00640 1 -- 172.21.15.165:0/340410095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6798036050 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f67a9a7c740 10 client.6342 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.924+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x5557eb8c8fc0 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f6788c00640 1 -- 172.21.15.165:0/340410095 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6798003b90 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f6788c00640 1 -- 172.21.15.165:0/340410095 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f6798039020 con 0x5557eb8b9950 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f67a9a7c740 10 client.6342 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f67a9a7c740 10 client.6342 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f67a9a7c740 20 client.6342 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb8c99a0 0x5557eb871520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:58.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f67a9a7c740 4 client.6342 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}] 2024-04-25T10:06:58.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_test_path/ceph-qa-suite/new_file_1"}) v1 -- 0x5557eb8960e0 con 0x5557eb8c99a0 2024-04-25T10:06:58.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.925+0000 7f679fe00640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb8c99a0 0x5557eb871520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:58.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.926+0000 7f679fe00640 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb8c99a0 0x5557eb871520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3482c53a18e3f50 server_cookie=bd93caf16d950df5 in_seq=0 out_seq=0 2024-04-25T10:06:58.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.926+0000 7f6788c00640 10 client.6342 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.934 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:06:58.929+0000 7fa948c00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000096e [2,head] /scrub_test_path/ceph-qa-suite/new_file_1 auth v669 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={4826=0-4194304@1} caps={4826=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@3},l=4826 | request=0 lock=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x55eaae7b2c00]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(8)0x1000000096e:[<0x10000000001/new_file_1 v669>,<0x10000000000/ceph-qa-suite v82>,<0x1/scrub_test_path v38>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":-61} 2024-04-25T10:06:58.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f6788c00640 1 -- 172.21.15.165:0/340410095 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5557eb8960e0 con 0x5557eb8c99a0 2024-04-25T10:06:58.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f6788c00640 10 client.6342 handle_command_reply: tid=0 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 client.6342 shutdown 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 msgr2=0x5557eb8c9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8c9460 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f6798009950 tx=0x7f6798009450 comp rx=0 tx=0).stop 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 shutdown_connections 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x5557eb8c99a0 0x5557eb871520 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5557eb8b9950 0x5557eb8c9460 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5557eb871140 0x5557eb9122d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f6788c00640 0 client.6342 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 --2- 172.21.15.165:0/340410095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5557eb870280 0x5557eb898480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 >> 172.21.15.165:0/340410095 conn(0x5557eb86c050 msgr2=0x5557eb8d84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 shutdown_connections 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 1 -- 172.21.15.165:0/340410095 wait complete. 2024-04-25T10:06:58.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.928+0000 7f67a9a7c740 20 client.6342 trim_cache size 0 max 16384 2024-04-25T10:06:58.937 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:58.937 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:06:58.937 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "95eea238-eca8-427c-b0b4-5ec50c897923", 2024-04-25T10:06:58.937 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:06:58.937 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f678006b4d0 msgr2=0x7f678006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f678006b4d0 0x7f678006d990 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f67a00b4610 tx=0x7f678c009040 comp rx=0 tx=0).stop 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 msgr2=0x7f67a00cae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 0x7f67a00cae10 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6790000c00 tx=0x7f67900046c0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 shutdown_connections 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f678006b4d0 0x7f678006d990 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67a0145140 0x7f67a00b0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67a00ce950 0x7f67a00cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 --2- 172.21.15.165:0/1701483307 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67a00c5200 0x7f67a00cae10 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:58.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 >> 172.21.15.165:0/1701483307 conn(0x7f67a0061cc0 msgr2=0x7f67a015c900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:58.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 shutdown_connections 2024-04-25T10:06:58.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:58.929+0000 7f67a5600640 1 -- 172.21.15.165:0/1701483307 wait complete. 2024-04-25T10:06:58.951 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_test_path/ceph-qa-suite/new_file_1' returned '{'return_code': 0, 'scrub_tag': '95eea238-eca8-427c-b0b4-5ec50c897923', 'mode': 'asynchronous'}' 2024-04-25T10:06:58.951 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:06:58.951 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:06:59.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.145+0000 7efd0f200640 1 Processor -- start 2024-04-25T10:06:59.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.145+0000 7efd0f200640 1 -- start start 2024-04-25T10:06:59.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd1013d9f0 0x7efd1013df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd1006e860 0x7efd100b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1006d390 con 0x7efd100b0730 2024-04-25T10:06:59.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1006d670 con 0x7efd1006e860 2024-04-25T10:06:59.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1006d950 con 0x7efd1013d9f0 2024-04-25T10:06:59.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0e800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd1006e860 0x7efd100b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0e800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd1006e860 0x7efd100b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48440/0 (socket says 172.21.15.165:48440) 2024-04-25T10:06:59.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0e800640 1 -- 172.21.15.165:0/2951915826 learned_addr learned my addr 172.21.15.165:0/2951915826 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:59.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.147+0000 7efd0de00640 1 --2- 172.21.15.165:0/2951915826 >> v2:172.21.15.172:3300/0 conn(0x7efd1013d9f0 0x7efd1013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.148+0000 7efd0e800640 1 -- 172.21.15.165:0/2951915826 >> v2:172.21.15.172:3300/0 conn(0x7efd1013d9f0 msgr2=0x7efd1013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.148+0000 7efd0e800640 1 --2- 172.21.15.165:0/2951915826 >> v2:172.21.15.172:3300/0 conn(0x7efd1013d9f0 0x7efd1013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.148+0000 7efd0e800640 1 -- 172.21.15.165:0/2951915826 >> v1:172.21.15.172:6789/0 conn(0x7efd100b0730 legacy=0x7efd1006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.148+0000 7efd0e800640 1 -- 172.21.15.165:0/2951915826 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1006dc30 con 0x7efd1006e860 2024-04-25T10:06:59.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.149+0000 7efd0e800640 1 --2- 172.21.15.165:0/2951915826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1006e860 0x7efd100b0bc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efd10072520 tx=0x7efd0802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3bf5a2180afd57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.149+0000 7efd0ca00640 1 -- 172.21.15.165:0/2951915826 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08038070 con 0x7efd1006e860 2024-04-25T10:06:59.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.149+0000 7efd0ca00640 1 -- 172.21.15.165:0/2951915826 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08033600 con 0x7efd1006e860 2024-04-25T10:06:59.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.149+0000 7efd0ca00640 1 -- 172.21.15.165:0/2951915826 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd080339d0 con 0x7efd1006e860 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 -- 172.21.15.165:0/2951915826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1006e860 msgr2=0x7efd100b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 --2- 172.21.15.165:0/2951915826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1006e860 0x7efd100b0bc0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7efd10072520 tx=0x7efd0802c7e0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 -- 172.21.15.165:0/2951915826 shutdown_connections 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 --2- 172.21.15.165:0/2951915826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1006e860 0x7efd100b0bc0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 --2- 172.21.15.165:0/2951915826 >> v2:172.21.15.172:3300/0 conn(0x7efd1013d9f0 0x7efd1013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 -- 172.21.15.165:0/2951915826 >> 172.21.15.165:0/2951915826 conn(0x7efd10061cc0 msgr2=0x7efd1006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 -- 172.21.15.165:0/2951915826 shutdown_connections 2024-04-25T10:06:59.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.150+0000 7efd0f200640 1 -- 172.21.15.165:0/2951915826 wait complete. 2024-04-25T10:06:59.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.152+0000 7efd0f200640 1 Processor -- start 2024-04-25T10:06:59.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.152+0000 7efd0f200640 1 -- start start 2024-04-25T10:06:59.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1006e860 0x7efd10135450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd100b0730 0x7efd1013a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 0x7efd10133b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd100ce880 con 0x7efd1013d9f0 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1013e680 con 0x7efd1006e860 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd100cea00 con 0x7efd100b0730 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 0x7efd10133b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 0x7efd10133b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48442/0 (socket says 172.21.15.165:48442) 2024-04-25T10:06:59.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.153+0000 7efd0e800640 1 -- 172.21.15.165:0/518175835 learned_addr learned my addr 172.21.15.165:0/518175835 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0de00640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1006e860 0x7efd10135450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0e800640 1 -- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd100b0730 msgr2=0x7efd1013a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0d400640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd100b0730 0x7efd1013a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0e800640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd100b0730 0x7efd1013a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0e800640 1 -- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1006e860 msgr2=0x7efd10135450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0e800640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1006e860 0x7efd10135450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0e800640 1 -- 172.21.15.165:0/518175835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd10139520 con 0x7efd1013d9f0 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0de00640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1006e860 0x7efd10135450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0d400640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd100b0730 0x7efd1013a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0e800640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 0x7efd10133b90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efd08036040 tx=0x7efd08004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd06000640 1 -- 172.21.15.165:0/518175835 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08038050 con 0x7efd1013d9f0 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd06000640 1 -- 172.21.15.165:0/518175835 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08044070 con 0x7efd1013d9f0 2024-04-25T10:06:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1013ab50 con 0x7efd1013d9f0 2024-04-25T10:06:59.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.154+0000 7efd06000640 1 -- 172.21.15.165:0/518175835 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08007bb0 con 0x7efd1013d9f0 2024-04-25T10:06:59.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.155+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd100ccb50 con 0x7efd1013d9f0 2024-04-25T10:06:59.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.157+0000 7efd06000640 1 -- 172.21.15.165:0/518175835 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd08049460 con 0x7efd1013d9f0 2024-04-25T10:06:59.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.158+0000 7efd06000640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.158+0000 7efd06000640 1 -- 172.21.15.165:0/518175835 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7efd080ad3b0 con 0x7efd1013d9f0 2024-04-25T10:06:59.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.158+0000 7efd0de00640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.158+0000 7efd17d76740 1 Processor -- start 2024-04-25T10:06:59.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.159+0000 7efd17d76740 1 -- start start 2024-04-25T10:06:59.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.159+0000 7efd0de00640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcf006b4d0 0x7efcf006d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd00006fd0 tx=0x7efd00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e43cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd17d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 0x561e9e45b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e45bec0 0x561e9e413dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd17d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e378c70 con 0x561e9e416f10 2024-04-25T10:06:59.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e321cf0 con 0x561e9e43cb60 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e9e374d10 con 0x561e9e45bec0 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 0x561e9e45b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 0x561e9e45b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48452/0 (socket says 172.21.15.165:48452) 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 -- 172.21.15.165:0/371158688 learned_addr learned my addr 172.21.15.165:0/371158688 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0de00640 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e43cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0e800640 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e45bec0 0x561e9e413dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 -- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e45bec0 msgr2=0x561e9e413dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e45bec0 0x561e9e413dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 -- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 msgr2=0x561e9e43cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e43cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.160+0000 7efd0d400640 1 -- 172.21.15.165:0/371158688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9e1577d0 con 0x561e9e416f10 2024-04-25T10:06:59.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.161+0000 7efd0d400640 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 0x561e9e45b980 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efcfc003ec0 tx=0x7efcfc004380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da16557b78365864 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.161+0000 7efcfaa00640 1 -- 172.21.15.165:0/371158688 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc005270 con 0x561e9e416f10 2024-04-25T10:06:59.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.161+0000 7efcfaa00640 1 -- 172.21.15.165:0/371158688 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc005410 con 0x561e9e416f10 2024-04-25T10:06:59.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.161+0000 7efcfaa00640 1 -- 172.21.15.165:0/371158688 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc00db20 con 0x561e9e416f10 2024-04-25T10:06:59.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 -- 172.21.15.165:0/371158688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 msgr2=0x561e9e45b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 0x561e9e45b980 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7efcfc003ec0 tx=0x7efcfc004380 comp rx=0 tx=0).stop 2024-04-25T10:06:59.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 -- 172.21.15.165:0/371158688 shutdown_connections 2024-04-25T10:06:59.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e45bec0 0x561e9e413dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e416f10 0x561e9e45b980 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 --2- 172.21.15.165:0/371158688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e43cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 -- 172.21.15.165:0/371158688 >> 172.21.15.165:0/371158688 conn(0x561e9e3dca30 msgr2=0x561e9e45a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 -- 172.21.15.165:0/371158688 shutdown_connections 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.162+0000 7efd17d76740 1 -- 172.21.15.165:0/371158688 wait complete. 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.163+0000 7efd17d76740 1 -- start start 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.163+0000 7efd17d76740 1 Processor -- start 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e416f10 0x561e9e43f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e43cb60 0x561e9e442590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e43b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e411370 con 0x561e9e436c10 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e3173a0 con 0x561e9e416f10 2024-04-25T10:06:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e9e415ea0 con 0x561e9e43cb60 2024-04-25T10:06:59.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd17d76740 10 client.0 _ll_register_callbacks cb 0x561e9e334b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:59.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e43b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e43b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48458/0 (socket says 172.21.15.165:48458) 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 -- 172.21.15.165:0/3544812223 learned_addr learned my addr 172.21.15.165:0/3544812223 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0de00640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e416f10 0x561e9e43f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 -- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e43cb60 msgr2=0x561e9e442590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0d400640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e43cb60 0x561e9e442590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e43cb60 0x561e9e442590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 -- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e416f10 msgr2=0x561e9e43f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.164+0000 7efd0e800640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e416f10 0x561e9e43f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efd0e800640 1 -- 172.21.15.165:0/3544812223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9e08d3f0 con 0x561e9e436c10 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efd0de00640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e416f10 0x561e9e43f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efd0e800640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e43b1e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7efd08083ab0 tx=0x7efd0807b230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a116c5b13afdb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efcef400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efcef400640 1 -- 172.21.15.165:0/3544812223 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0807b8d0 con 0x561e9e436c10 2024-04-25T10:06:59.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efcef400640 1 -- 172.21.15.165:0/3544812223 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7efd08007d50 con 0x561e9e436c10 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcfc005410 con 0x561e9e436c10 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efcef400640 1 -- 172.21.15.165:0/3544812223 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efd080ab050 con 0x561e9e436c10 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efd17d76740 10 client.6354 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x561e9e04d0f0 con 0x561e9e436c10 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.165+0000 7efcef400640 1 -- 172.21.15.165:0/3544812223 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcfc005410 con 0x561e9e436c10 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efcef400640 1 -- 172.21.15.165:0/3544812223 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efd080ab1f0 con 0x561e9e436c10 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd17d76740 10 client.6354 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd17d76740 10 client.6354 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd17d76740 20 client.6354 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd17d76740 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e43d4f0 0x561e9e43d900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd17d76740 4 client.6354 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:06:59.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e9e3861b0 con 0x561e9e43d4f0 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.166+0000 7efd0de00640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e43d4f0 0x561e9e43d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.168+0000 7efd0de00640 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e43d4f0 0x561e9e43d900 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=713ffc22fa868787 server_cookie=73615d9bad0d600b in_seq=0 out_seq=0 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.168+0000 7efcef400640 10 client.6354 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.169+0000 7efcef400640 1 -- 172.21.15.165:0/3544812223 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561e9e3861b0 con 0x561e9e43d4f0 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.169+0000 7efcef400640 10 client.6354 handle_command_reply: tid=0 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.169+0000 7efd17d76740 1 client.6354 shutdown 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 msgr2=0x561e9e43b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e43b1e0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7efd08083ab0 tx=0x7efd0807b230 comp rx=0 tx=0).stop 2024-04-25T10:06:59.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 shutdown_connections 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e43d4f0 0x561e9e43d900 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e43b1e0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e43cb60 0x561e9e442590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 --2- 172.21.15.165:0/3544812223 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e416f10 0x561e9e43f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 >> 172.21.15.165:0/3544812223 conn(0x561e9e3dca30 msgr2=0x561e9e45dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efcef400640 0 client.6354 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 shutdown_connections 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 1 -- 172.21.15.165:0/3544812223 wait complete. 2024-04-25T10:06:59.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.170+0000 7efd17d76740 20 client.6354 trim_cache size 0 max 16384 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.186+0000 7efd17d76740 1 Processor -- start 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 -- start start 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 0x561e9e45dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e436c10 0x561e9e43be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 0x561e9e43c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101fc020 con 0x561e9e43cb60 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101fb200 con 0x561e9e436c10 2024-04-25T10:06:59.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.187+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1013bc20 con 0x561e9e416f10 2024-04-25T10:06:59.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 0x561e9e43c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 0x561e9e43c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48460/0 (socket says 172.21.15.165:48460) 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 -- 172.21.15.165:0/3369720598 learned_addr learned my addr 172.21.15.165:0/3369720598 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0d400640 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e436c10 0x561e9e43be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 -- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 msgr2=0x561e9e45dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 0x561e9e45dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 -- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e436c10 msgr2=0x561e9e43be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e436c10 0x561e9e43be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 -- 172.21.15.165:0/3369720598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e9e40bcb0 con 0x561e9e43cb60 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0d400640 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e436c10 0x561e9e43be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.188+0000 7efd0e800640 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 0x561e9e43c380 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7efd08009680 tx=0x7efd0800e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=149ee86746dc7b39 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efcfa000640 1 -- 172.21.15.165:0/3369720598 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0807b8d0 con 0x561e9e43cb60 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efcfa000640 1 -- 172.21.15.165:0/3369720598 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08039040 con 0x561e9e43cb60 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efcfa000640 1 -- 172.21.15.165:0/3369720598 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0807b6a0 con 0x561e9e43cb60 2024-04-25T10:06:59.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 -- 172.21.15.165:0/3369720598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 msgr2=0x561e9e43c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 0x561e9e43c380 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7efd08009680 tx=0x7efd0800e000 comp rx=0 tx=0).stop 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 -- 172.21.15.165:0/3369720598 shutdown_connections 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e43cb60 0x561e9e43c380 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e436c10 0x561e9e43be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 --2- 172.21.15.165:0/3369720598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 0x561e9e45dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.189+0000 7efd17d76740 1 -- 172.21.15.165:0/3369720598 >> 172.21.15.165:0/3369720598 conn(0x561e9e3dca30 msgr2=0x561e9e443180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.190+0000 7efd17d76740 1 -- 172.21.15.165:0/3369720598 shutdown_connections 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.190+0000 7efd17d76740 1 -- 172.21.15.165:0/3369720598 wait complete. 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.190+0000 7efd17d76740 1 -- start start 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.190+0000 7efd17d76740 1 Processor -- start 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 0x561e9e4cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e4430f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e443630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e9e412d60 con 0x561e9e436c10 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101fac00 con 0x561e9e43cb60 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e9e4280e0 con 0x561e9e416f10 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.191+0000 7efd17d76740 10 client.0 _ll_register_callbacks cb 0x7efd080095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e4430f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e4430f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48466/0 (socket says 172.21.15.165:48466) 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 -- 172.21.15.165:0/1343032215 learned_addr learned my addr 172.21.15.165:0/1343032215 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0e800640 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e443630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 -- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 msgr2=0x561e9e4cd370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 0x561e9e4cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 -- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 msgr2=0x561e9e443630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e443630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 -- 172.21.15.165:0/1343032215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e9e40fc80 con 0x561e9e436c10 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efd0d400640 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e4430f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efcfc007000 tx=0x7efcfc0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a62a186892ab3f57 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:06:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.192+0000 7efcef400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:06:59.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efcef400640 1 -- 172.21.15.165:0/1343032215 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcfc005460 con 0x561e9e436c10 2024-04-25T10:06:59.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efcef400640 1 -- 172.21.15.165:0/1343032215 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7efcfc007030 con 0x561e9e436c10 2024-04-25T10:06:59.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efd08039040 con 0x561e9e436c10 2024-04-25T10:06:59.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efcef400640 1 -- 172.21.15.165:0/1343032215 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcfc02e050 con 0x561e9e436c10 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 10 client.6360 fetch_fsmap learned FSMap version 29 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=2+}) v3 -- 0x561e9e4ce0c0 con 0x561e9e436c10 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efcef400640 1 -- 172.21.15.165:0/1343032215 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08039040 con 0x561e9e436c10 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efcef400640 1 -- 172.21.15.165:0/1343032215 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 29) v1 ==== 3294+0+0 (secure 0 0 0) 0x7efcfc031020 con 0x561e9e436c10 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 10 client.6360 fetch_fsmap finished waiting for FSMap version 29 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 10 client.6360 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 20 client.6360 populate_metadata read hostname 'smithi165' 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e4ce5e0 0x561e9e4d0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 4 client.6360 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.193+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561e9e380500 con 0x561e9e4ce5e0 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.194+0000 7efd0de00640 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e4ce5e0 0x561e9e4d0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.195+0000 7efd0de00640 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e4ce5e0 0x561e9e4d0ec0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2106ed4d976fe614 server_cookie=bb8f1a21f1bfc168 in_seq=0 out_seq=0 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.195+0000 7efcef400640 10 client.6360 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.195+0000 7efcef400640 1 -- 172.21.15.165:0/1343032215 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+284 (crc 0 0 0) 0x561e9e380500 con 0x561e9e4ce5e0 2024-04-25T10:06:59.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.195+0000 7efcef400640 10 client.6360 handle_command_reply: tid=0 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.195+0000 7efd17d76740 1 client.6360 shutdown 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 msgr2=0x561e9e4430f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e4430f0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7efcfc007000 tx=0x7efcfc0036f0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 shutdown_connections 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x561e9e4ce5e0 0x561e9e4d0ec0 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561e9e43cb60 0x561e9e443630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561e9e436c10 0x561e9e4430f0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 --2- 172.21.15.165:0/1343032215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561e9e416f10 0x561e9e4cd370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 >> 172.21.15.165:0/1343032215 conn(0x561e9e3dca30 msgr2=0x561e9e43bde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efcef400640 0 client.6360 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:06:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 shutdown_connections 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 1 -- 172.21.15.165:0/1343032215 wait complete. 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.196+0000 7efd17d76740 20 client.6360 trim_cache size 0 max 16384 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout: "95eea238-eca8-427c-b0b4-5ec50c897923": { 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1", 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "95eea238-eca8-427c-b0b4-5ec50c897923", 2024-04-25T10:06:59.206 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "" 2024-04-25T10:06:59.207 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:59.207 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:06:59.207 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcf006b4d0 msgr2=0x7efcf006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcf006b4d0 0x7efcf006d990 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efd00006fd0 tx=0x7efd00008040 comp rx=0 tx=0).stop 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 msgr2=0x7efd10133b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 0x7efd10133b90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7efd08036040 tx=0x7efd08004a40 comp rx=0 tx=0).stop 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 shutdown_connections 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcf006b4d0 0x7efcf006d990 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1013d9f0 0x7efd10133b90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd100b0730 0x7efd1013a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 --2- 172.21.15.165:0/518175835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1006e860 0x7efd10135450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 >> 172.21.15.165:0/518175835 conn(0x7efd10061cc0 msgr2=0x7efd1013cb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:06:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 shutdown_connections 2024-04-25T10:06:59.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:06:59.197+0000 7efd0f200640 1 -- 172.21.15.165:0/518175835 wait complete. 2024-04-25T10:07:04.220 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:07:04.220 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.3:0 scrub status 2024-04-25T10:07:04.304 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:07:04.298+0000 7f5484c00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2024-04-25T10:07:04.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.431+0000 7f467f200640 1 Processor -- start 2024-04-25T10:07:04.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.431+0000 7f467f200640 1 -- start start 2024-04-25T10:07:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.432+0000 7f467f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46800ceb20 0x7f46800c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.432+0000 7f467f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4680145520 con 0x7f4680145140 2024-04-25T10:07:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.432+0000 7f467f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4680071270 con 0x7f46800c5030 2024-04-25T10:07:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.432+0000 7f467f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4680071550 con 0x7f46800ceb20 2024-04-25T10:07:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.433+0000 7f467de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4680145140 0x7f46800ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33800/0 (socket says 172.21.15.165:33800) 2024-04-25T10:07:04.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.433+0000 7f467de00640 1 -- 172.21.15.165:0/1270343415 learned_addr learned my addr 172.21.15.165:0/1270343415 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.433+0000 7f4675400640 1 --2- 172.21.15.165:0/1270343415 >> v2:172.21.15.172:3300/0 conn(0x7f46800ceb20 0x7f46800c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.433+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882472393 0 0) 0x7f4680145520 con 0x7f4680145140 2024-04-25T10:07:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.433+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4664003520 con 0x7f4680145140 2024-04-25T10:07:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2305968346 0 0) 0x7f4664003520 con 0x7f4680145140 2024-04-25T10:07:04.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 >> v2:172.21.15.172:3300/0 conn(0x7f46800ceb20 msgr2=0x7f46800c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 --2- 172.21.15.165:0/1270343415 >> v2:172.21.15.172:3300/0 conn(0x7f46800ceb20 0x7f46800c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 >> v1:172.21.15.172:6789/0 conn(0x7f46800c5030 legacy=0x7f468015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4680071830 con 0x7f4680145140 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4670001670 con 0x7f4680145140 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 >> v1:172.21.15.165:6789/0 conn(0x7f4680145140 legacy=0x7f46800ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664003a00 0x7f4664003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4664004800 0x7f4664004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4664005210 0x7f466400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4680071550 con 0x7f4664005210 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466400c160 con 0x7f4664004800 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466400c360 con 0x7f4664003a00 2024-04-25T10:07:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467e800640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4664005210 0x7f466400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.434+0000 7f467de00640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664003a00 0x7f4664003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f467e800640 1 -- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664003a00 msgr2=0x7f4664003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f4675400640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4664004800 0x7f4664004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f467e800640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664003a00 0x7f4664003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f467e800640 1 -- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4664004800 msgr2=0x7f4664004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f467e800640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4664004800 0x7f4664004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f467e800640 1 -- 172.21.15.165:0/1270343415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4664003520 con 0x7f4664005210 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f4675400640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4664004800 0x7f4664004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.435+0000 7f467e800640 1 -- 172.21.15.165:0/1270343415 reap_dead start 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.436+0000 7f467e800640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4664005210 0x7f466400b9c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4678001e20 tx=0x7f4664005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef35e499b9a97cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.436+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678001a70 con 0x7f4664005210 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.436+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467802ae40 con 0x7f4664005210 2024-04-25T10:07:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.436+0000 7f467d400640 1 -- 172.21.15.165:0/1270343415 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678003bb0 con 0x7f4664005210 2024-04-25T10:07:04.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 -- 172.21.15.165:0/1270343415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4664005210 msgr2=0x7f466400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4664005210 0x7f466400b9c0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f4678001e20 tx=0x7f4664005830 comp rx=0 tx=0).stop 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 -- 172.21.15.165:0/1270343415 shutdown_connections 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 --2- 172.21.15.165:0/1270343415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4664005210 0x7f466400b9c0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 -- 172.21.15.165:0/1270343415 >> 172.21.15.165:0/1270343415 conn(0x7f4680061cc0 msgr2=0x7f468006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 -- 172.21.15.165:0/1270343415 shutdown_connections 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.437+0000 7f467f200640 1 -- 172.21.15.165:0/1270343415 wait complete. 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.439+0000 7f467f200640 1 Processor -- start 2024-04-25T10:07:04.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.439+0000 7f467f200640 1 -- start start 2024-04-25T10:07:04.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.440+0000 7f467f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664005210 0x7f4680143270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.440+0000 7f467f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 0x7f468013f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.440+0000 7f467f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f468013bf60 0x7f468013c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.440+0000 7f467f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c5620 con 0x7f468013ecd0 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.440+0000 7f467f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c54a0 con 0x7f468013bf60 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.440+0000 7f467f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800c4af0 con 0x7f4664005210 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 0x7f468013f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 0x7f468013f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48472/0 (socket says 172.21.15.165:48472) 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 -- 172.21.15.165:0/1269647773 learned_addr learned my addr 172.21.15.165:0/1269647773 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 -- 172.21.15.165:0/1269647773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664005210 msgr2=0x7f4680143270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664005210 0x7f4680143270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 -- 172.21.15.165:0/1269647773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f468013bf60 msgr2=0x7f468013c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f468013bf60 0x7f468013c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.441+0000 7f4675400640 1 -- 172.21.15.165:0/1269647773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46801f4b00 con 0x7f468013ecd0 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.442+0000 7f4675400640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 0x7f468013f150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f46680108b0 tx=0x7f4668010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.442+0000 7f4676a00640 1 -- 172.21.15.165:0/1269647773 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466800f040 con 0x7f468013ecd0 2024-04-25T10:07:04.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.442+0000 7f4676a00640 1 -- 172.21.15.165:0/1269647773 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f466800eb60 con 0x7f468013ecd0 2024-04-25T10:07:04.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.442+0000 7f4676a00640 1 -- 172.21.15.165:0/1269647773 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f466801c400 con 0x7f468013ecd0 2024-04-25T10:07:04.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.442+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46801f4da0 con 0x7f468013ecd0 2024-04-25T10:07:04.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.442+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46801f5360 con 0x7f468013ecd0 2024-04-25T10:07:04.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.444+0000 7f4676a00640 1 -- 172.21.15.165:0/1269647773 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f466801c5a0 con 0x7f468013ecd0 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.445+0000 7f4676a00640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f465c06b500 0x7f465c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.445+0000 7f4676a00640 1 -- 172.21.15.165:0/1269647773 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f466808aae0 con 0x7f468013ecd0 2024-04-25T10:07:04.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.445+0000 7f467de00640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f465c06b500 0x7f465c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.446+0000 7f46879bf740 1 Processor -- start 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.446+0000 7f467de00640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f465c06b500 0x7f465c06d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4670000ff0 tx=0x7f4670009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.446+0000 7f46879bf740 1 -- start start 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f46879bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 0x55d9a70722e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a7070560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f46879bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a6fcfc70 con 0x55d9a7070180 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a6f78cf0 con 0x55d9a70ac850 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a6fcbd10 con 0x55d9a709ac40 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f467de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 0x55d9a70722e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f467de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 0x55d9a70722e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48484/0 (socket says 172.21.15.165:48484) 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.447+0000 7f467de00640 1 -- 172.21.15.165:0/2472874959 learned_addr learned my addr 172.21.15.165:0/2472874959 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467de00640 1 -- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 msgr2=0x55d9a709b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467e800640 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f4675400640 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a7070560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467de00640 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467de00640 1 -- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 msgr2=0x55d9a7070560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467de00640 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a7070560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467de00640 1 -- 172.21.15.165:0/2472874959 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9a6dae7d0 con 0x55d9a7070180 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f467de00640 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 0x55d9a70722e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f46700073c0 tx=0x7f4670001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f00d25841e032c86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f466e600640 1 -- 172.21.15.165:0/2472874959 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467000dbe0 con 0x55d9a7070180 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f466e600640 1 -- 172.21.15.165:0/2472874959 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467000dda0 con 0x55d9a7070180 2024-04-25T10:07:04.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.448+0000 7f466e600640 1 -- 172.21.15.165:0/2472874959 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4670020690 con 0x55d9a7070180 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 -- 172.21.15.165:0/2472874959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 msgr2=0x55d9a70722e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 0x55d9a70722e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f46700073c0 tx=0x7f4670001e70 comp rx=0 tx=0).stop 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 -- 172.21.15.165:0/2472874959 shutdown_connections 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a7070560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 --2- 172.21.15.165:0/2472874959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a7070180 0x55d9a70722e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 -- 172.21.15.165:0/2472874959 >> 172.21.15.165:0/2472874959 conn(0x55d9a7033a30 msgr2=0x55d9a70b5c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 -- 172.21.15.165:0/2472874959 shutdown_connections 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.449+0000 7f46879bf740 1 -- 172.21.15.165:0/2472874959 wait complete. 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.450+0000 7f46879bf740 1 -- start start 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.450+0000 7f46879bf740 1 Processor -- start 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 0x55d9a70977b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a70ac850 0x55d9a70b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a6f6e3a0 con 0x55d9a709ac40 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a7098790 con 0x55d9a7070180 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a70c5970 con 0x55d9a70ac850 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.451+0000 7f46879bf740 10 client.0 _ll_register_callbacks cb 0x55d9a6f8bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48494/0 (socket says 172.21.15.165:48494) 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 -- 172.21.15.165:0/1833923828 learned_addr learned my addr 172.21.15.165:0/1833923828 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 -- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a70ac850 msgr2=0x55d9a70b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a70ac850 0x55d9a70b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 -- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 msgr2=0x55d9a70977b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 0x55d9a70977b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 -- 172.21.15.165:0/1833923828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9a6ce43f0 con 0x55d9a709ac40 2024-04-25T10:07:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4675400640 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4668022030 tx=0x7f4668059170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b103b3e31021eb61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4662a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.452+0000 7f4662a00640 1 -- 172.21.15.165:0/1833923828 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668061480 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f4662a00640 1 -- 172.21.15.165:0/1833923828 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4668088a20 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f467000dda0 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f4662a00640 1 -- 172.21.15.165:0/1833923828 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4668058500 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 10 client.6372 fetch_fsmap learned FSMap version 31 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55d9a6ca40f0 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f4662a00640 1 -- 172.21.15.165:0/1833923828 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467000dda0 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f4662a00640 1 -- 172.21.15.165:0/1833923828 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f46680586a0 con 0x55d9a709ac40 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 10 client.6372 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 10 client.6372 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 20 client.6372 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a70b0150 0x55d9a709b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 4 client.6372 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.453+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9a6fdd1b0 con 0x55d9a70b0150 2024-04-25T10:07:04.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.454+0000 7f467de00640 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a70b0150 0x55d9a709b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.455+0000 7f4662a00640 10 client.6372 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.455+0000 7f467de00640 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a70b0150 0x55d9a709b0c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b0a06a7124f1750 server_cookie=60c129ae502bc1c9 in_seq=0 out_seq=0 2024-04-25T10:07:04.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f4662a00640 1 -- 172.21.15.165:0/1833923828 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9a6fdd1b0 con 0x55d9a70b0150 2024-04-25T10:07:04.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f4662a00640 10 client.6372 handle_command_reply: tid=0 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 client.6372 shutdown 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 msgr2=0x55d9a70b0b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b60 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f4668022030 tx=0x7f4668059170 comp rx=0 tx=0).stop 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 shutdown_connections 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a70b0150 0x55d9a709b0c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a70ac850 0x55d9a70b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b60 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 --2- 172.21.15.165:0/1833923828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 0x55d9a70977b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 >> 172.21.15.165:0/1833923828 conn(0x55d9a7033a30 msgr2=0x55d9a70b8fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f4662a00640 0 client.6372 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 shutdown_connections 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 1 -- 172.21.15.165:0/1833923828 wait complete. 2024-04-25T10:07:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.457+0000 7f46879bf740 20 client.6372 trim_cache size 0 max 16384 2024-04-25T10:07:04.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.474+0000 7f46879bf740 1 Processor -- start 2024-04-25T10:07:04.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.474+0000 7f46879bf740 1 -- start start 2024-04-25T10:07:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.474+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 0x55d9a70b1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.474+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.474+0000 7f46879bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 0x55d9a70b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.474+0000 7f46879bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46801fc850 con 0x55d9a70ac850 2024-04-25T10:07:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46801fbba0 con 0x55d9a7070180 2024-04-25T10:07:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800cd6a0 con 0x55d9a709ac40 2024-04-25T10:07:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 0x55d9a70b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 0x55d9a70b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48502/0 (socket says 172.21.15.165:48502) 2024-04-25T10:07:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 -- 172.21.15.165:0/2965751113 learned_addr learned my addr 172.21.15.165:0/2965751113 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 -- 172.21.15.165:0/2965751113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 msgr2=0x55d9a709b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 -- 172.21.15.165:0/2965751113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 msgr2=0x55d9a70b1b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 0x55d9a70b1b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.475+0000 7f467e800640 1 -- 172.21.15.165:0/2965751113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9a706d5d0 con 0x55d9a70ac850 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f467e800640 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 0x55d9a70b0b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f46780012e0 tx=0x7f4678001510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=111646d2d6aec52c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f466dc00640 1 -- 172.21.15.165:0/2965751113 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678001a70 con 0x55d9a70ac850 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f466dc00640 1 -- 172.21.15.165:0/2965751113 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678006b50 con 0x55d9a70ac850 2024-04-25T10:07:04.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f466dc00640 1 -- 172.21.15.165:0/2965751113 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678003970 con 0x55d9a70ac850 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f46879bf740 1 -- 172.21.15.165:0/2965751113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 msgr2=0x55d9a70b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f46879bf740 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 0x55d9a70b0b70 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f46780012e0 tx=0x7f4678001510 comp rx=0 tx=0).stop 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f46879bf740 1 -- 172.21.15.165:0/2965751113 shutdown_connections 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f46879bf740 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a70ac850 0x55d9a70b0b70 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f46879bf740 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a709ac40 0x55d9a709b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.476+0000 7f46879bf740 1 --2- 172.21.15.165:0/2965751113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a7070180 0x55d9a70b1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.477+0000 7f46879bf740 1 -- 172.21.15.165:0/2965751113 >> 172.21.15.165:0/2965751113 conn(0x55d9a7033a30 msgr2=0x55d9a70975c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.477+0000 7f46879bf740 1 -- 172.21.15.165:0/2965751113 shutdown_connections 2024-04-25T10:07:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.477+0000 7f46879bf740 1 -- 172.21.15.165:0/2965751113 wait complete. 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.477+0000 7f46879bf740 1 -- start start 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.477+0000 7f46879bf740 1 Processor -- start 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a7070180 0x55d9a7124a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a70b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9a70af2d0 con 0x55d9a709ac40 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800cdca0 con 0x55d9a70ac850 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9a706ae80 con 0x55d9a7070180 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.478+0000 7f46879bf740 10 client.0 _ll_register_callbacks cb 0x7f46680046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48504/0 (socket says 172.21.15.165:48504) 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 -- 172.21.15.165:0/4245542652 learned_addr learned my addr 172.21.15.165:0/4245542652 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 -- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a7070180 msgr2=0x55d9a7124a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a7070180 0x55d9a7124a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 -- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 msgr2=0x55d9a70b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a70b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 -- 172.21.15.165:0/4245542652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9a70d2400 con 0x55d9a709ac40 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4675400640 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f466800e540 tx=0x7f4668088c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4438c876c735fe3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.479+0000 7f4662a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f4662a00640 1 -- 172.21.15.165:0/4245542652 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4668021070 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f4662a00640 1 -- 172.21.15.165:0/4245542652 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4668054e00 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4678006b50 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f4662a00640 1 -- 172.21.15.165:0/4245542652 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f466805f340 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 10 client.6378 fetch_fsmap learned FSMap version 31 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55d9a70d26e0 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f4662a00640 1 -- 172.21.15.165:0/4245542652 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4678006b50 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f4662a00640 1 -- 172.21.15.165:0/4245542652 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4668057f70 con 0x55d9a709ac40 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 10 client.6378 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 10 client.6378 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 20 client.6378 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a711acb0 0x55d9a71275e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 4 client.6378 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:07:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.480+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d9a7071690 con 0x55d9a711acb0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.481+0000 7f467de00640 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a711acb0 0x55d9a71275e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.482+0000 7f4662a00640 10 client.6378 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.482+0000 7f467de00640 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a711acb0 0x55d9a71275e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c05258cbbb46027 server_cookie=120622b1d9877198 in_seq=0 out_seq=0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.482+0000 7f4662a00640 1 -- 172.21.15.165:0/4245542652 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d9a7071690 con 0x55d9a711acb0 2024-04-25T10:07:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.482+0000 7f4662a00640 10 client.6378 handle_command_reply: tid=0 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.482+0000 7f46879bf740 1 client.6378 shutdown 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 msgr2=0x55d9a70b0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f466800e540 tx=0x7f4668088c80 comp rx=0 tx=0).stop 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 shutdown_connections 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55d9a711acb0 0x55d9a71275e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9a70ac850 0x55d9a70b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9a709ac40 0x55d9a70b0b70 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 --2- 172.21.15.165:0/4245542652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9a7070180 0x55d9a7124a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f4662a00640 0 client.6378 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 >> 172.21.15.165:0/4245542652 conn(0x55d9a7033a30 msgr2=0x55d9a7062150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 shutdown_connections 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 1 -- 172.21.15.165:0/4245542652 wait complete. 2024-04-25T10:07:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f46879bf740 20 client.6378 trim_cache size 0 max 16384 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:07:04.492 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f465c06b500 msgr2=0x7f465c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.483+0000 7f467f200640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f465c06b500 0x7f465c06d9c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4670000ff0 tx=0x7f4670009040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 msgr2=0x7f468013f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 0x7f468013f150 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f46680108b0 tx=0x7f4668010d70 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 shutdown_connections 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f465c06b500 0x7f465c06d9c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f468013bf60 0x7f468013c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f468013ecd0 0x7f468013f150 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 --2- 172.21.15.165:0/1269647773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4664005210 0x7f4680143270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 >> 172.21.15.165:0/1269647773 conn(0x7f4680061cc0 msgr2=0x7f468006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 shutdown_connections 2024-04-25T10:07:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.484+0000 7f467f200640 1 -- 172.21.15.165:0/1269647773 wait complete. 2024-04-25T10:07:04.508 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:07:04.508 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:07:04.508 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 damage ls 2024-04-25T10:07:04.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.692+0000 7f61bb200640 1 Processor -- start 2024-04-25T10:07:04.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.693+0000 7f61bb200640 1 -- start start 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61bb200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61bc06bff0 0x7f61bc0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61bb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61bc0d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61bb200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61bc15e110 con 0x7f61bc15dd30 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61bb200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61bc06d000 con 0x7f61bc06bff0 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61bb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0b2fd0 con 0x7f61bc0b5970 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61b9e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f61bc15dd30 0x7f61bc0cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33804/0 (socket says 172.21.15.165:33804) 2024-04-25T10:07:04.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.694+0000 7f61b9e00640 1 -- 172.21.15.165:0/2407404840 learned_addr learned my addr 172.21.15.165:0/2407404840 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b1400640 1 --2- 172.21.15.165:0/2407404840 >> v2:172.21.15.172:3300/0 conn(0x7f61bc06bff0 0x7f61bc0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61ba800640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61bc0d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 814717333 0 0) 0x7f61bc15e110 con 0x7f61bc15dd30 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61a0003520 con 0x7f61bc15dd30 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1692964691 0 0) 0x7f61a0003520 con 0x7f61bc15dd30 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 msgr2=0x7f61bc0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61bc0d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 >> v2:172.21.15.172:3300/0 conn(0x7f61bc06bff0 msgr2=0x7f61bc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 --2- 172.21.15.165:0/2407404840 >> v2:172.21.15.172:3300/0 conn(0x7f61bc06bff0 0x7f61bc0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc06c480 con 0x7f61bc15dd30 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f61ac001670 con 0x7f61bc15dd30 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.695+0000 7f61ba800640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61bc0d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 >> v1:172.21.15.165:6789/0 conn(0x7f61bc15dd30 legacy=0x7f61bc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a00039b0 0x7f61a0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61a0004d40 0x7f61a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc06d000 con 0x7f61a0004d40 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0b2fd0 con 0x7f61a00039b0 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61a000b9c0 con 0x7f61bc0b5970 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61ba800640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61a0004d40 0x7f61a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a00039b0 0x7f61a0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61b1400640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61ba800640 1 -- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 msgr2=0x7f61a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61ba800640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61bc0b5970 0x7f61a0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61ba800640 1 -- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a00039b0 msgr2=0x7f61a0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61ba800640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61a00039b0 0x7f61a0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.696+0000 7f61ba800640 1 -- 172.21.15.165:0/2407404840 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61a0003520 con 0x7f61a0004d40 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.697+0000 7f61ba800640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61a0004d40 0x7f61a00051c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f61b40073e0 tx=0x7f61a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee9ac9a22deff5b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4009c80 con 0x7f61a0004d40 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b402e660 con 0x7f61a0004d40 2024-04-25T10:07:04.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61b9400640 1 -- 172.21.15.165:0/2407404840 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b402ea10 con 0x7f61a0004d40 2024-04-25T10:07:04.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61bb200640 1 -- 172.21.15.165:0/2407404840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61a0004d40 msgr2=0x7f61a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61bb200640 1 --2- 172.21.15.165:0/2407404840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61a0004d40 0x7f61a00051c0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f61b40073e0 tx=0x7f61a0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:04.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61ba800640 1 -- 172.21.15.165:0/2407404840 reap_dead start 2024-04-25T10:07:04.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61bb200640 1 -- 172.21.15.165:0/2407404840 shutdown_connections 2024-04-25T10:07:04.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61bb200640 1 -- 172.21.15.165:0/2407404840 >> 172.21.15.165:0/2407404840 conn(0x7f61bc0d0610 msgr2=0x7f61bc0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61bb200640 1 -- 172.21.15.165:0/2407404840 shutdown_connections 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.698+0000 7f61bb200640 1 -- 172.21.15.165:0/2407404840 wait complete. 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.700+0000 7f61bb200640 1 Processor -- start 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.700+0000 7f61bb200640 1 -- start start 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.701+0000 7f61bb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a0004d40 0x7f61bc0c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.701+0000 7f61bb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc0cd6c0 0x7f61bc0cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.701+0000 7f61bb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 0x7f61bc158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.701+0000 7f61bb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc06cd20 con 0x7f61bc157dc0 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.701+0000 7f61bb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc06ca40 con 0x7f61bc0cd6c0 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.701+0000 7f61bb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc0d2160 con 0x7f61a0004d40 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 0x7f61bc158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 0x7f61bc158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48516/0 (socket says 172.21.15.165:48516) 2024-04-25T10:07:04.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 -- 172.21.15.165:0/2681027001 learned_addr learned my addr 172.21.15.165:0/2681027001 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a0004d40 0x7f61bc0c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61b1400640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc0cd6c0 0x7f61bc0cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 -- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a0004d40 msgr2=0x7f61bc0c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a0004d40 0x7f61bc0c9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 -- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc0cd6c0 msgr2=0x7f61bc0cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc0cd6c0 0x7f61bc0cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 -- 172.21.15.165:0/2681027001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc158780 con 0x7f61bc157dc0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61b1400640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc0cd6c0 0x7f61bc0cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.702+0000 7f61ba800640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 0x7f61bc158240 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f61b40073e0 tx=0x7f61b4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.703+0000 7f61b2a00640 1 -- 172.21.15.165:0/2681027001 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4009c80 con 0x7f61bc157dc0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.703+0000 7f61b2a00640 1 -- 172.21.15.165:0/2681027001 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61b402e460 con 0x7f61bc157dc0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.703+0000 7f61b2a00640 1 -- 172.21.15.165:0/2681027001 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4038280 con 0x7f61bc157dc0 2024-04-25T10:07:04.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.703+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc0c9ac0 con 0x7f61bc157dc0 2024-04-25T10:07:04.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.703+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc158cf0 con 0x7f61bc157dc0 2024-04-25T10:07:04.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.705+0000 7f61b2a00640 1 -- 172.21.15.165:0/2681027001 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61b4038650 con 0x7f61bc157dc0 2024-04-25T10:07:04.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.706+0000 7f61b2a00640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f619806b500 0x7f619806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.706+0000 7f61b2a00640 1 -- 172.21.15.165:0/2681027001 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f61b40aa940 con 0x7f61bc157dc0 2024-04-25T10:07:04.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.706+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f619806b500 0x7f619806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.707+0000 7f61c416a740 1 Processor -- start 2024-04-25T10:07:04.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.707+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f619806b500 0x7f619806d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f61ac000c00 tx=0x7f61ac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.707+0000 7f61c416a740 1 -- start start 2024-04-25T10:07:04.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61c416a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 0x55f689c683f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 0x55f689c85e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c3dcf0 0x55f689c83f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61c416a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f689ba0d00 con 0x55f689c68010 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f689b49f50 con 0x55f689c858b0 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f689b9cda0 con 0x55f689c3dcf0 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61b9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 0x55f689c683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61b9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 0x55f689c683f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48524/0 (socket says 172.21.15.165:48524) 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.708+0000 7f61b9e00640 1 -- 172.21.15.165:0/2753648247 learned_addr learned my addr 172.21.15.165:0/2753648247 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61b9e00640 1 -- 172.21.15.165:0/2753648247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c3dcf0 msgr2=0x55f689c83f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c3dcf0 0x55f689c83f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61b9e00640 1 -- 172.21.15.165:0/2753648247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 msgr2=0x55f689c85e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 0x55f689c85e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61b9e00640 1 -- 172.21.15.165:0/2753648247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f689b6d130 con 0x55f689c68010 2024-04-25T10:07:04.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61b9e00640 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 0x55f689c683f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f61ac005b60 tx=0x7f61ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc6b3c3f16d75b71 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61aaa00640 1 -- 172.21.15.165:0/2753648247 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac004bf0 con 0x55f689c68010 2024-04-25T10:07:04.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61aaa00640 1 -- 172.21.15.165:0/2753648247 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac004d90 con 0x55f689c68010 2024-04-25T10:07:04.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.709+0000 7f61aaa00640 1 -- 172.21.15.165:0/2753648247 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ac00ea10 con 0x55f689c68010 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 -- 172.21.15.165:0/2753648247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 msgr2=0x55f689c683f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 0x55f689c683f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f61ac005b60 tx=0x7f61ac011000 comp rx=0 tx=0).stop 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 -- 172.21.15.165:0/2753648247 shutdown_connections 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c3dcf0 0x55f689c83f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 0x55f689c85e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 --2- 172.21.15.165:0/2753648247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c68010 0x55f689c683f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 -- 172.21.15.165:0/2753648247 >> 172.21.15.165:0/2753648247 conn(0x55f689c3a620 msgr2=0x55f689c82e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 -- 172.21.15.165:0/2753648247 shutdown_connections 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.710+0000 7f61c416a740 1 -- 172.21.15.165:0/2753648247 wait complete. 2024-04-25T10:07:04.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.711+0000 7f61c416a740 1 -- start start 2024-04-25T10:07:04.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.711+0000 7f61c416a740 1 Processor -- start 2024-04-25T10:07:04.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c68010 0x55f689c663e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 0x55f689c6b300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c7bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f689c3ea90 con 0x55f689c7a8e0 2024-04-25T10:07:04.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f689b3f530 con 0x55f689c858b0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f689c38550 con 0x55f689c68010 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61c416a740 10 client.0 _ll_register_callbacks cb 0x55f689b98430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.712+0000 7f61ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c7bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c7bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48536/0 (socket says 172.21.15.165:48536) 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 -- 172.21.15.165:0/1233868729 learned_addr learned my addr 172.21.15.165:0/1233868729 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 -- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c68010 msgr2=0x55f689c663e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c68010 0x55f689c663e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 -- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 msgr2=0x55f689c6b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 0x55f689c6b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 -- 172.21.15.165:0/1233868729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f689b67dd0 con 0x55f689c7a8e0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61ba800640 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c7bc40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f61b40752c0 tx=0x7f61b4079560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93098fafe54cfb3e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f619f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f619f400640 1 -- 172.21.15.165:0/1233868729 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4047040 con 0x55f689c7a8e0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f619f400640 1 -- 172.21.15.165:0/1233868729 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f61b40810e0 con 0x55f689c7a8e0 2024-04-25T10:07:04.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.713+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61ac004d90 con 0x55f689c7a8e0 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f619f400640 1 -- 172.21.15.165:0/1233868729 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b407cfb0 con 0x55f689c7a8e0 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 10 client.6390 fetch_fsmap learned FSMap version 31 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55f6898b53f0 con 0x55f689c7a8e0 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f619f400640 1 -- 172.21.15.165:0/1233868729 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ac004d90 con 0x55f689c7a8e0 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f619f400640 1 -- 172.21.15.165:0/1233868729 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61b40782b0 con 0x55f689c7a8e0 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 10 client.6390 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 10 client.6390 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 20 client.6390 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689c69b30 0x55f689c69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 4 client.6390 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.714+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f689bae240 con 0x55f689c69b30 2024-04-25T10:07:04.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.715+0000 7f61b9e00640 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689c69b30 0x55f689c69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.715+0000 7f61b9e00640 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689c69b30 0x55f689c69f10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ee85438b242995e server_cookie=31bb4e6f3948710b in_seq=0 out_seq=0 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.715+0000 7f619f400640 10 client.6390 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f619f400640 1 -- 172.21.15.165:0/1233868729 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f689bae240 con 0x55f689c69b30 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f619f400640 10 client.6390 handle_command_reply: tid=0 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 client.6390 shutdown 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 msgr2=0x55f689c7bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c7bc40 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f61b40752c0 tx=0x7f61b4079560 comp rx=0 tx=0).stop 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 shutdown_connections 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689c69b30 0x55f689c69f10 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c7bc40 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c858b0 0x55f689c6b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f619f400640 0 client.6390 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 --2- 172.21.15.165:0/1233868729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c68010 0x55f689c663e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.717+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 >> 172.21.15.165:0/1233868729 conn(0x55f689c3dcf0 msgr2=0x55f689c863a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.718+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 shutdown_connections 2024-04-25T10:07:04.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.718+0000 7f61c416a740 1 -- 172.21.15.165:0/1233868729 wait complete. 2024-04-25T10:07:04.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.718+0000 7f61c416a740 20 client.6390 trim_cache size 0 max 16384 2024-04-25T10:07:04.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.730+0000 7f61c416a740 1 Processor -- start 2024-04-25T10:07:04.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 -- start start 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 0x55f689ce6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc200b00 con 0x55f689c7a8e0 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1ffcd0 con 0x55f689c68010 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.731+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1ff100 con 0x55f689c858b0 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48538/0 (socket says 172.21.15.165:48538) 2024-04-25T10:07:04.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 -- 172.21.15.165:0/2551943523 learned_addr learned my addr 172.21.15.165:0/2551943523 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 -- 172.21.15.165:0/2551943523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c858b0 msgr2=0x55f689c6a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 -- 172.21.15.165:0/2551943523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 msgr2=0x55f689ce6fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 0x55f689ce6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 -- 172.21.15.165:0/2551943523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f68997f7d0 con 0x55f689c7a8e0 2024-04-25T10:07:04.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.732+0000 7f61b1400640 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c69f10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009a60 tx=0x7f61a4009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c202e0a38dcbf142 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61aa000640 1 -- 172.21.15.165:0/2551943523 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4004230 con 0x55f689c7a8e0 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61aa000640 1 -- 172.21.15.165:0/2551943523 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a40043d0 con 0x55f689c7a8e0 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61aa000640 1 -- 172.21.15.165:0/2551943523 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61a4011a60 con 0x55f689c7a8e0 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 -- 172.21.15.165:0/2551943523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 msgr2=0x55f689c69f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c69f10 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f61a4009a60 tx=0x7f61a4009f20 comp rx=0 tx=0).stop 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 -- 172.21.15.165:0/2551943523 shutdown_connections 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 --2- 172.21.15.165:0/2551943523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 0x55f689ce6fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 -- 172.21.15.165:0/2551943523 >> 172.21.15.165:0/2551943523 conn(0x55f689c3dcf0 msgr2=0x55f689c661f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 -- 172.21.15.165:0/2551943523 shutdown_connections 2024-04-25T10:07:04.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.733+0000 7f61c416a740 1 -- 172.21.15.165:0/2551943523 wait complete. 2024-04-25T10:07:04.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.734+0000 7f61c416a740 1 -- start start 2024-04-25T10:07:04.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.734+0000 7f61c416a740 1 Processor -- start 2024-04-25T10:07:04.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 0x55f689c9cc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f689c3b560 con 0x55f689c858b0 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc1ff6d0 con 0x55f689c68010 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f689c94410 con 0x55f689c7a8e0 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.735+0000 7f61c416a740 10 client.0 _ll_register_callbacks cb 0x55f689c38a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48550/0 (socket says 172.21.15.165:48550) 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 -- 172.21.15.165:0/3550063687 learned_addr learned my addr 172.21.15.165:0/3550063687 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 -- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c7a8e0 msgr2=0x55f689c69f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 -- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 msgr2=0x55f689c9cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 0x55f689c9cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 -- 172.21.15.165:0/3550063687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f689ce3b90 con 0x55f689c858b0 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61ba800640 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 0x55f689c6a450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f61b40757e0 tx=0x7f61b403a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5eceab4529a0412 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f619f400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f619f400640 1 -- 172.21.15.165:0/3550063687 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61b4047040 con 0x55f689c858b0 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f619f400640 1 -- 172.21.15.165:0/3550063687 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f61b40812f0 con 0x55f689c858b0 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.736+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f61a40043d0 con 0x55f689c858b0 2024-04-25T10:07:04.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f619f400640 1 -- 172.21.15.165:0/3550063687 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f61b40789a0 con 0x55f689c858b0 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 10 client.6396 fetch_fsmap learned FSMap version 31 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55f689c7f660 con 0x55f689c858b0 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f619f400640 1 -- 172.21.15.165:0/3550063687 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61a40043d0 con 0x55f689c858b0 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f619f400640 1 -- 172.21.15.165:0/3550063687 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f61b4074bb0 con 0x55f689c858b0 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 10 client.6396 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 10 client.6396 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 20 client.6396 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689ce3420 0x55f689c7fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 4 client.6396 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.737+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55f689ba8590 con 0x55f689ce3420 2024-04-25T10:07:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.738+0000 7f61b9e00640 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689ce3420 0x55f689c7fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.738+0000 7f61b9e00640 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689ce3420 0x55f689c7fb80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf7ad781d5d2db3 server_cookie=a8786b1e51a20aab in_seq=0 out_seq=0 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.738+0000 7f619f400640 10 client.6396 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f619f400640 1 -- 172.21.15.165:0/3550063687 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+168 (crc 0 0 0) 0x55f689ba8590 con 0x55f689ce3420 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f619f400640 10 client.6396 handle_command_reply: tid=0 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 client.6396 shutdown 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 msgr2=0x55f689c6a450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 0x55f689c6a450 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f61b40757e0 tx=0x7f61b403a000 comp rx=0 tx=0).stop 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 shutdown_connections 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x55f689ce3420 0x55f689c7fb80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f689c858b0 0x55f689c6a450 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f689c7a8e0 0x55f689c69f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f619f400640 0 client.6396 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.739+0000 7f61c416a740 1 --2- 172.21.15.165:0/3550063687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f689c68010 0x55f689c9cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 >> 172.21.15.165:0/3550063687 conn(0x55f689c3dcf0 msgr2=0x55f689c825d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 shutdown_connections 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61c416a740 1 -- 172.21.15.165:0/3550063687 wait complete. 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61c416a740 20 client.6396 trim_cache size 0 max 16384 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout: "damage_type": "backtrace", 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 1922395323, 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 1099511630190, 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_test_path/ceph-qa-suite/new_file_1" 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:04.749 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:07:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f619806b500 msgr2=0x7f619806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f619806b500 0x7f619806d9c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f61ac000c00 tx=0x7f61ac009040 comp rx=0 tx=0).stop 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 msgr2=0x7f61bc158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 0x7f61bc158240 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f61b40073e0 tx=0x7f61b4004ab0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 shutdown_connections 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f619806b500 0x7f619806d9c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61bc157dc0 0x7f61bc158240 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61bc0cd6c0 0x7f61bc0cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 --2- 172.21.15.165:0/2681027001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61a0004d40 0x7f61bc0c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.740+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 >> 172.21.15.165:0/2681027001 conn(0x7f61bc0d0610 msgr2=0x7f61bc0b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.741+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 shutdown_connections 2024-04-25T10:07:04.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.741+0000 7f61bb200640 1 -- 172.21.15.165:0/2681027001 wait complete. 2024-04-25T10:07:04.764 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:04.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.951+0000 7fda61600640 1 Processor -- start 2024-04-25T10:07:04.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.952+0000 7fda61600640 1 -- start start 2024-04-25T10:07:04.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.953+0000 7fda61600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fda5c15b8f0 0x7fda5c0c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.953+0000 7fda61600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda5c0cf340 0x7fda5c0b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.953+0000 7fda61600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c0b3920 0x7fda5c06f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.953+0000 7fda61600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0d03c0 con 0x7fda5c0cf340 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.953+0000 7fda61600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda5c0c7150 con 0x7fda5c15b8f0 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.953+0000 7fda61600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c7f90 con 0x7fda5c0b3920 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda5c0cf340 0x7fda5c0b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda5c0cf340 0x7fda5c0b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48554/0 (socket says 172.21.15.165:48554) 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 -- 172.21.15.165:0/427960869 learned_addr learned my addr 172.21.15.165:0/427960869 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5aa00640 1 --2- 172.21.15.165:0/427960869 >> v2:172.21.15.172:3300/0 conn(0x7fda5c15b8f0 0x7fda5c0c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 -- 172.21.15.165:0/427960869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c0b3920 msgr2=0x7fda5c06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 --2- 172.21.15.165:0/427960869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c0b3920 0x7fda5c06f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 -- 172.21.15.165:0/427960869 >> v2:172.21.15.172:3300/0 conn(0x7fda5c15b8f0 msgr2=0x7fda5c0c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 --2- 172.21.15.165:0/427960869 >> v2:172.21.15.172:3300/0 conn(0x7fda5c15b8f0 0x7fda5c0c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.954+0000 7fda5a000640 1 -- 172.21.15.165:0/427960869 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0c7380 con 0x7fda5c0cf340 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.955+0000 7fda5a000640 1 --2- 172.21.15.165:0/427960869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0b33e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0d27d0 tx=0x7fda4402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49b73951914c9845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda59600640 1 -- 172.21.15.165:0/427960869 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44037070 con 0x7fda5c0cf340 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda59600640 1 -- 172.21.15.165:0/427960869 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda44032440 con 0x7fda5c0cf340 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda59600640 1 -- 172.21.15.165:0/427960869 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44032810 con 0x7fda5c0cf340 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 -- 172.21.15.165:0/427960869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 msgr2=0x7fda5c0b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 --2- 172.21.15.165:0/427960869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0b33e0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fda5c0d27d0 tx=0x7fda4402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 -- 172.21.15.165:0/427960869 shutdown_connections 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 --2- 172.21.15.165:0/427960869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c0b3920 0x7fda5c06f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 --2- 172.21.15.165:0/427960869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0b33e0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 --2- 172.21.15.165:0/427960869 >> v2:172.21.15.172:3300/0 conn(0x7fda5c15b8f0 0x7fda5c0c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 -- 172.21.15.165:0/427960869 >> 172.21.15.165:0/427960869 conn(0x7fda5c0d1800 msgr2=0x7fda5c0d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:04.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 -- 172.21.15.165:0/427960869 shutdown_connections 2024-04-25T10:07:04.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.956+0000 7fda61600640 1 -- 172.21.15.165:0/427960869 wait complete. 2024-04-25T10:07:04.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.958+0000 7fda61600640 1 Processor -- start 2024-04-25T10:07:04.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.958+0000 7fda61600640 1 -- start start 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda61600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda5c0b3920 0x7fda5c0b6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda61600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0ce930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda61600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c15b8f0 0x7fda5c14dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda61600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c7940 con 0x7fda5c0cf340 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda61600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0b3d60 con 0x7fda5c0b3920 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda61600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5c0c7c20 con 0x7fda5c15b8f0 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.959+0000 7fda5a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0ce930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0ce930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48564/0 (socket says 172.21.15.165:48564) 2024-04-25T10:07:04.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 -- 172.21.15.165:0/3236608993 learned_addr learned my addr 172.21.15.165:0/3236608993 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5aa00640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda5c0b3920 0x7fda5c0b6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 -- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c15b8f0 msgr2=0x7fda5c14dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5b400640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c15b8f0 0x7fda5c14dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c15b8f0 0x7fda5c14dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 -- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda5c0b3920 msgr2=0x7fda5c0b6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda5c0b3920 0x7fda5c0b6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 -- 172.21.15.165:0/3236608993 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c0b6d20 con 0x7fda5c0cf340 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.960+0000 7fda5a000640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0ce930 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fda44009870 tx=0x7fda440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.961+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44041020 con 0x7fda5c0cf340 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.961+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda440306c0 con 0x7fda5c0cf340 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.961+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda5c0b7000 con 0x7fda5c0cf340 2024-04-25T10:07:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.961+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda44037040 con 0x7fda5c0cf340 2024-04-25T10:07:04.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.961+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda5c0b4830 con 0x7fda5c0cf340 2024-04-25T10:07:04.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.962+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5c073990 con 0x7fda5c0cf340 2024-04-25T10:07:04.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.963+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda44044020 con 0x7fda5c0cf340 2024-04-25T10:07:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.964+0000 7fda52a00640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda3806b4d0 0x7fda3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.964+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fda440aac40 con 0x7fda5c0cf340 2024-04-25T10:07:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.964+0000 7fda5aa00640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda3806b4d0 0x7fda3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.965+0000 7fda5aa00640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda3806b4d0 0x7fda3806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009e90 tx=0x7fda4c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:04.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:04.969+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda4404b030 con 0x7fda5c0cf340 2024-04-25T10:07:05.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.202+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda5c1586a0 con 0x7fda5c0cf340 2024-04-25T10:07:05.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.203+0000 7fda52a00640 1 -- 172.21.15.165:0/3236608993 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fda44079860 con 0x7fda5c0cf340 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:07:03.631703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:05.210 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 31 2024-04-25T10:07:05.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda3806b4d0 msgr2=0x7fda3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda3806b4d0 0x7fda3806d990 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fda4c009e90 tx=0x7fda4c00b040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 msgr2=0x7fda5c0ce930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0ce930 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fda44009870 tx=0x7fda440046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 shutdown_connections 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda3806b4d0 0x7fda3806d990 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda5c15b8f0 0x7fda5c14dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda5c0cf340 0x7fda5c0ce930 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 --2- 172.21.15.165:0/3236608993 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda5c0b3920 0x7fda5c0b6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 >> 172.21.15.165:0/3236608993 conn(0x7fda5c0d1800 msgr2=0x7fda5c155b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 shutdown_connections 2024-04-25T10:07:05.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.206+0000 7fda61600640 1 -- 172.21.15.165:0/3236608993 wait complete. 2024-04-25T10:07:05.231 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage rm 1922395323 2024-04-25T10:07:05.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.433+0000 7fc3b7400640 1 Processor -- start 2024-04-25T10:07:05.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.433+0000 7fc3b7400640 1 -- start start 2024-04-25T10:07:05.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b7400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc3b015dc20 0x7fc3b00ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3b00d32f0 0x7fc3b00cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b7400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3b015e000 con 0x7fc3b0161710 2024-04-25T10:07:05.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b7400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b00d1de0 con 0x7fc3b00d32f0 2024-04-25T10:07:05.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b7400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3b0161c60 con 0x7fc3b015dc20 2024-04-25T10:07:05.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3b00d32f0 0x7fc3b00cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc3b00d32f0 0x7fc3b00cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48574/0 (socket says 172.21.15.165:48574) 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.435+0000 7fc3b5400640 1 -- 172.21.15.165:0/1621065697 learned_addr learned my addr 172.21.15.165:0/1621065697 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.436+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/1621065697 >> v2:172.21.15.172:3300/0 conn(0x7fc3b015dc20 0x7fc3b00ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.436+0000 7fc3b5400640 1 -- 172.21.15.165:0/1621065697 >> v2:172.21.15.172:3300/0 conn(0x7fc3b015dc20 msgr2=0x7fc3b00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.436+0000 7fc3b5400640 1 --2- 172.21.15.165:0/1621065697 >> v2:172.21.15.172:3300/0 conn(0x7fc3b015dc20 0x7fc3b00ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.436+0000 7fc3b5400640 1 -- 172.21.15.165:0/1621065697 >> v1:172.21.15.172:6789/0 conn(0x7fc3b0161710 legacy=0x7fc3b006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.436+0000 7fc3b5400640 1 -- 172.21.15.165:0/1621065697 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b0161f40 con 0x7fc3b00d32f0 2024-04-25T10:07:05.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.437+0000 7fc3b5400640 1 --2- 172.21.15.165:0/1621065697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b00d32f0 0x7fc3b00cebb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00d1520 tx=0x7fc3a802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee33f11bef68cc91 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.437+0000 7fc3af400640 1 -- 172.21.15.165:0/1621065697 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8037070 con 0x7fc3b00d32f0 2024-04-25T10:07:05.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.437+0000 7fc3af400640 1 -- 172.21.15.165:0/1621065697 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8032440 con 0x7fc3b00d32f0 2024-04-25T10:07:05.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.437+0000 7fc3af400640 1 -- 172.21.15.165:0/1621065697 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8032810 con 0x7fc3b00d32f0 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 -- 172.21.15.165:0/1621065697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b00d32f0 msgr2=0x7fc3b00cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 --2- 172.21.15.165:0/1621065697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b00d32f0 0x7fc3b00cebb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fc3b00d1520 tx=0x7fc3a802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 -- 172.21.15.165:0/1621065697 shutdown_connections 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 --2- 172.21.15.165:0/1621065697 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b00d32f0 0x7fc3b00cebb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 --2- 172.21.15.165:0/1621065697 >> v2:172.21.15.172:3300/0 conn(0x7fc3b015dc20 0x7fc3b00ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 -- 172.21.15.165:0/1621065697 >> 172.21.15.165:0/1621065697 conn(0x7fc3b00d0420 msgr2=0x7fc3b00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 -- 172.21.15.165:0/1621065697 shutdown_connections 2024-04-25T10:07:05.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.438+0000 7fc3b7400640 1 -- 172.21.15.165:0/1621065697 wait complete. 2024-04-25T10:07:05.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.440+0000 7fc3b7400640 1 Processor -- start 2024-04-25T10:07:05.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.440+0000 7fc3b7400640 1 -- start start 2024-04-25T10:07:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b7400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3b00d32f0 0x7fc3b0158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b7400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3b015dc20 0x7fc3b015ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 0x7fc3b00cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b7400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00b5e50 con 0x7fc3b0161710 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b7400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b0162500 con 0x7fc3b015dc20 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b7400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b00b5fd0 con 0x7fc3b00d32f0 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 0x7fc3b00cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 0x7fc3b00cc7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48580/0 (socket says 172.21.15.165:48580) 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b5400640 1 -- 172.21.15.165:0/4270889930 learned_addr learned my addr 172.21.15.165:0/4270889930 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3b00d32f0 0x7fc3b0158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.441+0000 7fc3afe00640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3b015dc20 0x7fc3b015ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b5400640 1 -- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3b00d32f0 msgr2=0x7fc3b0158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b5400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3b00d32f0 0x7fc3b0158e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b5400640 1 -- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3b015dc20 msgr2=0x7fc3b015ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b5400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3b015dc20 0x7fc3b015ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b5400640 1 -- 172.21.15.165:0/4270889930 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b015f6a0 con 0x7fc3b0161710 2024-04-25T10:07:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b5400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 0x7fc3b00cc7c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80090b0 tx=0x7fc3a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3acc00640 1 -- 172.21.15.165:0/4270889930 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8041020 con 0x7fc3b0161710 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3acc00640 1 -- 172.21.15.165:0/4270889930 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8030760 con 0x7fc3b0161710 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3acc00640 1 -- 172.21.15.165:0/4270889930 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8037040 con 0x7fc3b0161710 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.442+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b015f980 con 0x7fc3b0161710 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.443+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b015a300 con 0x7fc3b0161710 2024-04-25T10:07:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.445+0000 7fc3acc00640 1 -- 172.21.15.165:0/4270889930 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3a8044020 con 0x7fc3b0161710 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.446+0000 7fc3acc00640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc38c06b500 0x7fc38c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.446+0000 7fc3acc00640 1 -- 172.21.15.165:0/4270889930 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc3a80aacd0 con 0x7fc3b0161710 2024-04-25T10:07:05.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.446+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc38c06b500 0x7fc38c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.447+0000 7fc3ba171740 1 Processor -- start 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.447+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc38c06b500 0x7fc38c06d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b7c0 tx=0x7fc398008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.447+0000 7fc3ba171740 1 -- start start 2024-04-25T10:07:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 0x555688ae4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688adf4e0 0x555688ae3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 0x555688ae3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3ba171740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688a48820 con 0x555688ae39a0 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556889f1d00 con 0x555688adf4e0 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x555688a448c0 con 0x555688ae4860 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 0x555688ae3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 0x555688ae3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48584/0 (socket says 172.21.15.165:48584) 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 -- 172.21.15.165:0/420705569 learned_addr learned my addr 172.21.15.165:0/420705569 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 -- 172.21.15.165:0/420705569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 msgr2=0x555688ae4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 0x555688ae4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 -- 172.21.15.165:0/420705569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688adf4e0 msgr2=0x555688ae3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688adf4e0 0x555688ae3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.448+0000 7fc3b5400640 1 -- 172.21.15.165:0/420705569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555688829710 con 0x555688ae39a0 2024-04-25T10:07:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.449+0000 7fc3b5400640 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 0x555688ae3de0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a820 tx=0x7fc3a8078b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=267936d6a61a4844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.449+0000 7fc3a5200640 1 -- 172.21.15.165:0/420705569 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a80791f0 con 0x555688ae39a0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.449+0000 7fc3a5200640 1 -- 172.21.15.165:0/420705569 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a807d340 con 0x555688ae39a0 2024-04-25T10:07:05.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.449+0000 7fc3a5200640 1 -- 172.21.15.165:0/420705569 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8081900 con 0x555688ae39a0 2024-04-25T10:07:05.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.449+0000 7fc3ba171740 1 -- 172.21.15.165:0/420705569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 msgr2=0x555688ae3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.449+0000 7fc3ba171740 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 0x555688ae3de0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a820 tx=0x7fc3a8078b50 comp rx=0 tx=0).stop 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 -- 172.21.15.165:0/420705569 shutdown_connections 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688ae39a0 0x555688ae3de0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688adf4e0 0x555688ae3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 --2- 172.21.15.165:0/420705569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 0x555688ae4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 -- 172.21.15.165:0/420705569 >> 172.21.15.165:0/420705569 conn(0x555688ae1960 msgr2=0x555688b2a420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 -- 172.21.15.165:0/420705569 shutdown_connections 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 -- 172.21.15.165:0/420705569 wait complete. 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.450+0000 7fc3ba171740 1 -- start start 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.451+0000 7fc3ba171740 1 Processor -- start 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b2c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b2ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688a48820 con 0x555688b0cfa0 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556889e7360 con 0x555688ae4860 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x555688b103f0 con 0x555688ae39a0 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3ba171740 10 client.0 _ll_register_callbacks cb 0x5556889f3c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b2ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b2ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48588/0 (socket says 172.21.15.165:48588) 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 -- 172.21.15.165:0/760261477 learned_addr learned my addr 172.21.15.165:0/760261477 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 -- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 msgr2=0x555688b3a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 -- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 msgr2=0x555688b2c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b2c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.452+0000 7fc3b5400640 1 -- 172.21.15.165:0/760261477 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555688889ea0 con 0x555688b0cfa0 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc3b5400640 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b2ca80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a820 tx=0x7fc3a8009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2adfeecafbed96f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc39d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc39d600640 1 -- 172.21.15.165:0/760261477 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8081670 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc39d600640 1 -- 172.21.15.165:0/760261477 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc3a8047c70 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3a807d340 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc39d600640 1 -- 172.21.15.165:0/760261477 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a80791f0 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc3ba171740 10 client.6414 fetch_fsmap learned FSMap version 31 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55568871d000 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc39d600640 1 -- 172.21.15.165:0/760261477 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a807d340 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc39d600640 1 -- 172.21.15.165:0/760261477 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3a803a030 con 0x555688b0cfa0 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.453+0000 7fc3ba171740 10 client.6414 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.454+0000 7fc3ba171740 10 client.6414 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.454+0000 7fc3ba171740 20 client.6414 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.454+0000 7fc3ba171740 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688b46010 0x555688b48400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.454+0000 7fc3ba171740 4 client.6414 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.454+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555688a55d60 con 0x555688b46010 2024-04-25T10:07:05.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.454+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688b46010 0x555688b48400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.455+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688b46010 0x555688b48400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a26dcd972408671 server_cookie=8aa7b7d0678d2d8a in_seq=0 out_seq=0 2024-04-25T10:07:05.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.455+0000 7fc39d600640 10 client.6414 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:05.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc39d600640 1 -- 172.21.15.165:0/760261477 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x555688a55d60 con 0x555688b46010 2024-04-25T10:07:05.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc39d600640 10 client.6414 handle_command_reply: tid=0 2024-04-25T10:07:05.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 client.6414 shutdown 2024-04-25T10:07:05.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 msgr2=0x555688b2ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b2ca80 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a820 tx=0x7fc3a8009240 comp rx=0 tx=0).stop 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 shutdown_connections 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688b46010 0x555688b48400 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b2ca80 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b2c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 --2- 172.21.15.165:0/760261477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc39d600640 0 client.6414 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.457+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 >> 172.21.15.165:0/760261477 conn(0x555688adf4e0 msgr2=0x555688b0fc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.458+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 shutdown_connections 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.458+0000 7fc3ba171740 1 -- 172.21.15.165:0/760261477 wait complete. 2024-04-25T10:07:05.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.458+0000 7fc3ba171740 20 client.6414 trim_cache size 0 max 16384 2024-04-25T10:07:05.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.470+0000 7fc3ba171740 1 Processor -- start 2024-04-25T10:07:05.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 -- start start 2024-04-25T10:07:05.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae39a0 0x555688b2d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 0x555688b8fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b90250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01ff940 con 0x555688b0cfa0 2024-04-25T10:07:05.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b015be30 con 0x555688ae39a0 2024-04-25T10:07:05.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.471+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b01ff340 con 0x555688ae4860 2024-04-25T10:07:05.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b90250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b90250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48596/0 (socket says 172.21.15.165:48596) 2024-04-25T10:07:05.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 -- 172.21.15.165:0/2033617651 learned_addr learned my addr 172.21.15.165:0/2033617651 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 -- 172.21.15.165:0/2033617651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 msgr2=0x555688b8fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 0x555688b8fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 -- 172.21.15.165:0/2033617651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae39a0 msgr2=0x555688b2d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae39a0 0x555688b2d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 -- 172.21.15.165:0/2033617651 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55568871d000 con 0x555688b0cfa0 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.472+0000 7fc3b5400640 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b90250 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8075600 tx=0x7fc3a80a5000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b2b46d01fc09e4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc39fe00640 1 -- 172.21.15.165:0/2033617651 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8081470 con 0x555688b0cfa0 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc39fe00640 1 -- 172.21.15.165:0/2033617651 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8081650 con 0x555688b0cfa0 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc39fe00640 1 -- 172.21.15.165:0/2033617651 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8078880 con 0x555688b0cfa0 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 -- 172.21.15.165:0/2033617651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 msgr2=0x555688b90250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b90250 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8075600 tx=0x7fc3a80a5000 comp rx=0 tx=0).stop 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 -- 172.21.15.165:0/2033617651 shutdown_connections 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b90250 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae4860 0x555688b8fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 --2- 172.21.15.165:0/2033617651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae39a0 0x555688b2d450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 -- 172.21.15.165:0/2033617651 >> 172.21.15.165:0/2033617651 conn(0x555688adf4e0 msgr2=0x555688b0f3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 -- 172.21.15.165:0/2033617651 shutdown_connections 2024-04-25T10:07:05.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.473+0000 7fc3ba171740 1 -- 172.21.15.165:0/2033617651 wait complete. 2024-04-25T10:07:05.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.474+0000 7fc3ba171740 1 -- start start 2024-04-25T10:07:05.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.474+0000 7fc3ba171740 1 Processor -- start 2024-04-25T10:07:05.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b8ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b39680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x555688a48820 con 0x555688b0cfa0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556889e7360 con 0x555688ae4860 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x555688b2ed50 con 0x555688ae39a0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.475+0000 7fc3ba171740 10 client.0 _ll_register_callbacks cb 0x7fc3a802a770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b39680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b39680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48606/0 (socket says 172.21.15.165:48606) 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 -- 172.21.15.165:0/249375212 learned_addr learned my addr 172.21.15.165:0/249375212 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 -- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 msgr2=0x555688b3ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3afe00640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b8ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3ffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 -- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 msgr2=0x555688b8ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b8ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 -- 172.21.15.165:0/249375212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555688adaea0 con 0x555688b0cfa0 2024-04-25T10:07:05.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3b5400640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b39680 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a820 tx=0x7fc3a8009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a84ca239dd43ddb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc39d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc39d600640 1 -- 172.21.15.165:0/249375212 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3a8081470 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc39d600640 1 -- 172.21.15.165:0/249375212 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc3a8074f40 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.476+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc3a8081650 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc39d600640 1 -- 172.21.15.165:0/249375212 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3a80798f0 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 10 client.6420 fetch_fsmap learned FSMap version 31 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x555688adaea0 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc39d600640 1 -- 172.21.15.165:0/249375212 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3a8081650 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc39d600640 1 -- 172.21.15.165:0/249375212 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 31) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc3a8075140 con 0x555688b0cfa0 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 10 client.6420 fetch_fsmap finished waiting for FSMap version 31 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 10 client.6420 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 20 client.6420 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688ade620 0x555688adea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 4 client.6420 mds_command: new command op to 5811 tid=0 multi_id=0 [{"prefix": "damage rm", "damage_id": 1922395323}] 2024-04-25T10:07:05.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.477+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 --> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] -- command(tid 0: {"prefix": "damage rm", "damage_id": 1922395323}) v1 -- 0x555688ae0cf0 con 0x555688ade620 2024-04-25T10:07:05.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.478+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688ade620 0x555688adea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc3b4a00640 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688ade620 0x555688adea00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c5bb2fddf1b7f3a server_cookie=8fe039895445e75 in_seq=0 out_seq=0 2024-04-25T10:07:05.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc39d600640 10 client.6420 ms_handle_connect on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc39d600640 1 -- 172.21.15.165:0/249375212 <== mds.0 v2:172.21.15.172:6832/3236509347 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+0 (crc 0 0 0) 0x555688ae0cf0 con 0x555688ade620 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc39d600640 10 client.6420 handle_command_reply: tid=0 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc3ba171740 1 client.6420 shutdown 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 msgr2=0x555688b39680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.479+0000 7fc3ba171740 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b39680 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fc3a802a820 tx=0x7fc3a8009550 comp rx=0 tx=0).stop 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 shutdown_connections 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:6832/3236509347,v1:172.21.15.172:6834/3236509347] conn(0x555688ade620 0x555688adea00 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x555688b0cfa0 0x555688b39680 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x555688ae4860 0x555688b8ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 --2- 172.21.15.165:0/249375212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x555688ae39a0 0x555688b3ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc39d600640 0 client.6420 ms_handle_reset on v2:172.21.15.172:6832/3236509347 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 >> 172.21.15.165:0/249375212 conn(0x555688adf4e0 msgr2=0x555688b2d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 shutdown_connections 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 1 -- 172.21.15.165:0/249375212 wait complete. 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3ba171740 20 client.6420 trim_cache size 0 max 16384 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc38c06b500 msgr2=0x7fc38c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3b7400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc38c06b500 0x7fc38c06d9c0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc39800b7c0 tx=0x7fc398008040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 msgr2=0x7fc3b00cc7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.480+0000 7fc3b7400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 0x7fc3b00cc7c0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80090b0 tx=0x7fc3a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 shutdown_connections 2024-04-25T10:07:05.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc38c06b500 0x7fc38c06d9c0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3b0161710 0x7fc3b00cc7c0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc3b015dc20 0x7fc3b015ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 --2- 172.21.15.165:0/4270889930 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3b00d32f0 0x7fc3b0158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 >> 172.21.15.165:0/4270889930 conn(0x7fc3b00d0420 msgr2=0x7fc3b00b82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 shutdown_connections 2024-04-25T10:07:05.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.481+0000 7fc3b7400640 1 -- 172.21.15.165:0/4270889930 wait complete. 2024-04-25T10:07:05.503 INFO:tasks.cephfs.test_scrub_checks:Running command 'flush_path /' 2024-04-25T10:07:05.503 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:05.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.697+0000 7fbd28200640 1 Processor -- start 2024-04-25T10:07:05.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.697+0000 7fbd28200640 1 -- start start 2024-04-25T10:07:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd28200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd20144750 0x7fbd2006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd28200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbd200cf030 0x7fbd2006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd28200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd20144b30 con 0x7fbd2006eb10 2024-04-25T10:07:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd28200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd200712d0 con 0x7fbd200cf030 2024-04-25T10:07:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd28200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd200715b0 con 0x7fbd20144750 2024-04-25T10:07:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd26200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbd200cf030 0x7fbd2006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd24e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbd2006eb10 0x7fbd200ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33806/0 (socket says 172.21.15.165:33806) 2024-04-25T10:07:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd24e00640 1 -- 172.21.15.165:0/4116124618 learned_addr learned my addr 172.21.15.165:0/4116124618 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd25800640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd20144750 0x7fbd2006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.699+0000 7fbd26200640 1 -- 172.21.15.165:0/4116124618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd20144750 msgr2=0x7fbd2006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.700+0000 7fbd26200640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd20144750 0x7fbd2006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.700+0000 7fbd26200640 1 -- 172.21.15.165:0/4116124618 >> v1:172.21.15.165:6789/0 conn(0x7fbd2006eb10 legacy=0x7fbd200ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.700+0000 7fbd26200640 1 -- 172.21.15.165:0/4116124618 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20071890 con 0x7fbd200cf030 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.700+0000 7fbd25800640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd20144750 0x7fbd2006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.701+0000 7fbd26200640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd200cf030 0x7fbd2006d290 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009d90 tx=0x7fbd1802a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=542d8b66a3811c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.701+0000 7fbd1fe00640 1 -- 172.21.15.165:0/4116124618 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18031600 con 0x7fbd200cf030 2024-04-25T10:07:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.701+0000 7fbd1fe00640 1 -- 172.21.15.165:0/4116124618 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd180317a0 con 0x7fbd200cf030 2024-04-25T10:07:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.701+0000 7fbd1fe00640 1 -- 172.21.15.165:0/4116124618 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18031b70 con 0x7fbd200cf030 2024-04-25T10:07:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 -- 172.21.15.165:0/4116124618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd200cf030 msgr2=0x7fbd2006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd200cf030 0x7fbd2006d290 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009d90 tx=0x7fbd1802a5d0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 -- 172.21.15.165:0/4116124618 shutdown_connections 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd200cf030 0x7fbd2006d290 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 --2- 172.21.15.165:0/4116124618 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd20144750 0x7fbd2006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 -- 172.21.15.165:0/4116124618 >> 172.21.15.165:0/4116124618 conn(0x7fbd2006abe0 msgr2=0x7fbd2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 -- 172.21.15.165:0/4116124618 shutdown_connections 2024-04-25T10:07:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.702+0000 7fbd28200640 1 -- 172.21.15.165:0/4116124618 wait complete. 2024-04-25T10:07:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.704+0000 7fbd28200640 1 Processor -- start 2024-04-25T10:07:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.704+0000 7fbd28200640 1 -- start start 2024-04-25T10:07:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd28200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd2006eb10 0x7fbd2015ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd28200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd200cf030 0x7fbd2015cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd28200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 0x7fbd200ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd28200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd201455d0 con 0x7fbd20144750 2024-04-25T10:07:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd28200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd20145450 con 0x7fbd200cf030 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd28200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd2015c000 con 0x7fbd2006eb10 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 0x7fbd200ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 0x7fbd200ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48624/0 (socket says 172.21.15.165:48624) 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 -- 172.21.15.165:0/4009651601 learned_addr learned my addr 172.21.15.165:0/4009651601 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd25800640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd2006eb10 0x7fbd2015ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd24e00640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd200cf030 0x7fbd2015cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 -- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd2006eb10 msgr2=0x7fbd2015ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd2006eb10 0x7fbd2015ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 -- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd200cf030 msgr2=0x7fbd2015cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.705+0000 7fbd26200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd200cf030 0x7fbd2015cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd26200640 1 -- 172.21.15.165:0/4009651601 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd2015d730 con 0x7fbd20144750 2024-04-25T10:07:05.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd25800640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd2006eb10 0x7fbd2015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:05.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd26200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 0x7fbd200ad990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802abc0 tx=0x7fbd18004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1803f070 con 0x7fbd20144750 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd18030b60 con 0x7fbd20144750 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd2015da10 con 0x7fbd20144750 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd18044460 con 0x7fbd20144750 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.706+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd2015dfd0 con 0x7fbd20144750 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.707+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd20073990 con 0x7fbd20144750 2024-04-25T10:07:05.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.708+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd1803b410 con 0x7fbd20144750 2024-04-25T10:07:05.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.710+0000 7fbd1d600640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd0006b500 0x7fbd0006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:05.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.710+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fbd180abab0 con 0x7fbd20144750 2024-04-25T10:07:05.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.710+0000 7fbd25800640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd0006b500 0x7fbd0006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:05.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.711+0000 7fbd25800640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd0006b500 0x7fbd0006d9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd14006fd0 tx=0x7fbd14008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:05.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.715+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd1807a140 con 0x7fbd20144750 2024-04-25T10:07:05.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.951+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd200d0510 con 0x7fbd20144750 2024-04-25T10:07:05.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.951+0000 7fbd1d600640 1 -- 172.21.15.165:0/4009651601 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fbd1807e1e0 con 0x7fbd20144750 2024-04-25T10:07:05.957 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:05.957 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:07:03.631703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:05.958 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 31 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd0006b500 msgr2=0x7fbd0006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd0006b500 0x7fbd0006d9c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fbd14006fd0 tx=0x7fbd14008040 comp rx=0 tx=0).stop 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 msgr2=0x7fbd200ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 0x7fbd200ad990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fbd1802abc0 tx=0x7fbd18004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 shutdown_connections 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd0006b500 0x7fbd0006d9c0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbd20144750 0x7fbd200ad990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbd200cf030 0x7fbd2015cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 --2- 172.21.15.165:0/4009651601 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbd2006eb10 0x7fbd2015ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 >> 172.21.15.165:0/4009651601 conn(0x7fbd2006abe0 msgr2=0x7fbd20141d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 shutdown_connections 2024-04-25T10:07:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:05.954+0000 7fbd28200640 1 -- 172.21.15.165:0/4009651601 wait complete. 2024-04-25T10:07:05.978 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok flush_path / 2024-04-25T10:07:06.195 INFO:teuthology.orchestra.run.smithi172.stdout:{ 2024-04-25T10:07:06.195 INFO:teuthology.orchestra.run.smithi172.stdout: "return_code": 0 2024-04-25T10:07:06.195 INFO:teuthology.orchestra.run.smithi172.stdout:} 2024-04-25T10:07:06.210 INFO:tasks.cephfs.test_scrub_checks:command 'flush_path /' got response code '0' and stdout '{ "return_code": 0 } ' 2024-04-25T10:07:06.210 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:06.210 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:06.315 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:06.420 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:06.420 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:06.420 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:06.477 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:06.477 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:06.532 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:06.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.732+0000 7fe3db200640 1 Processor -- start 2024-04-25T10:07:06.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.732+0000 7fe3db200640 1 -- start start 2024-04-25T10:07:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3db200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3dc071700 0x7fe3dc0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3db200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3dc15bf30 con 0x7fe3dc144440 2024-04-25T10:07:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3db200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3dc15c860 con 0x7fe3dc071700 2024-04-25T10:07:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc15c210 con 0x7fe3dc0cd860 2024-04-25T10:07:06.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3d3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3dc071700 0x7fe3dc0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3d3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe3dc071700 0x7fe3dc0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:44716/0 (socket says 172.21.15.165:44716) 2024-04-25T10:07:06.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3d3e00640 1 -- 172.21.15.165:0/480270189 learned_addr learned my addr 172.21.15.165:0/480270189 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:06.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.734+0000 7fe3da800640 1 --2- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.735+0000 7fe3d3e00640 1 -- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 msgr2=0x7fe3dc15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.735+0000 7fe3d3e00640 1 --2- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc15ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.735+0000 7fe3d3e00640 1 -- 172.21.15.165:0/480270189 >> v1:172.21.15.172:6789/0 conn(0x7fe3dc144440 legacy=0x7fe3dc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:06.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.735+0000 7fe3d3e00640 1 -- 172.21.15.165:0/480270189 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc15c4f0 con 0x7fe3dc071700 2024-04-25T10:07:06.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.736+0000 7fe3d3e00640 1 --2- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc0cd320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009870 tx=0x7fe3c402a2a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fe26168e74914785 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.737+0000 7fe3d9400640 1 -- 172.21.15.165:0/480270189 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4037070 con 0x7fe3dc071700 2024-04-25T10:07:06.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.737+0000 7fe3d9400640 1 -- 172.21.15.165:0/480270189 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c4032440 con 0x7fe3dc071700 2024-04-25T10:07:06.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.737+0000 7fe3d9400640 1 -- 172.21.15.165:0/480270189 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4032810 con 0x7fe3dc071700 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 -- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 msgr2=0x7fe3dc0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 --2- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc0cd320 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4009870 tx=0x7fe3c402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 -- 172.21.15.165:0/480270189 shutdown_connections 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 --2- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 --2- 172.21.15.165:0/480270189 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc0cd320 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 -- 172.21.15.165:0/480270189 >> 172.21.15.165:0/480270189 conn(0x7fe3dc06abe0 msgr2=0x7fe3dc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 -- 172.21.15.165:0/480270189 shutdown_connections 2024-04-25T10:07:06.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.738+0000 7fe3db200640 1 -- 172.21.15.165:0/480270189 wait complete. 2024-04-25T10:07:06.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.740+0000 7fe3db200640 1 Processor -- start 2024-04-25T10:07:06.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.741+0000 7fe3db200640 1 -- start start 2024-04-25T10:07:06.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3db200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc13c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc0cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 0x7fe3dc1f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3db200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0cde40 con 0x7fe3dc144440 2024-04-25T10:07:06.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3db200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc0cdcc0 con 0x7fe3dc071700 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3dc15d230 con 0x7fe3dc0cd860 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3da800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 0x7fe3dc1f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3da800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 0x7fe3dc1f4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48626/0 (socket says 172.21.15.165:48626) 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3da800640 1 -- 172.21.15.165:0/2731992507 learned_addr learned my addr 172.21.15.165:0/2731992507 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3d3e00640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc0cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.742+0000 7fe3d9e00640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc13c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3da800640 1 -- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 msgr2=0x7fe3dc0cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3da800640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc0cc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3da800640 1 -- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 msgr2=0x7fe3dc13c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3da800640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc13c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3da800640 1 -- 172.21.15.165:0/2731992507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3dc13b190 con 0x7fe3dc144440 2024-04-25T10:07:06.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3da800640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 0x7fe3dc1f4770 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400efc0 tx=0x7fe3d400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4018020 con 0x7fe3dc144440 2024-04-25T10:07:06.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d4014480 con 0x7fe3dc144440 2024-04-25T10:07:06.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4014870 con 0x7fe3dc144440 2024-04-25T10:07:06.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3dc13b470 con 0x7fe3dc144440 2024-04-25T10:07:06.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.743+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3dc13ba30 con 0x7fe3dc144440 2024-04-25T10:07:06.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.744+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3dc073990 con 0x7fe3dc144440 2024-04-25T10:07:06.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.746+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3d4005d10 con 0x7fe3dc144440 2024-04-25T10:07:06.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.746+0000 7fe3d2000640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3b806b4d0 0x7fe3b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:06.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.747+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe3d408afc0 con 0x7fe3dc144440 2024-04-25T10:07:06.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.747+0000 7fe3d9e00640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3b806b4d0 0x7fe3b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:06.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.747+0000 7fe3d9e00640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3b806b4d0 0x7fe3b806d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006fd0 tx=0x7fe3cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:06.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.751+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3d4022070 con 0x7fe3dc144440 2024-04-25T10:07:06.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.947+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe3dc0008d0 con 0x7fe3dc144440 2024-04-25T10:07:06.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.950+0000 7fe3d2000640 1 -- 172.21.15.165:0/2731992507 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17018 (secure 0 0 0) 0x7fe3d4010040 con 0x7fe3dc144440 2024-04-25T10:07:06.956 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:06.956 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":56,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:46.336008+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:06.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.952+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3b806b4d0 msgr2=0x7fe3b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.952+0000 7fe3db200640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3b806b4d0 0x7fe3b806d990 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc006fd0 tx=0x7fe3cc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:06.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.952+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 msgr2=0x7fe3dc1f4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:06.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.952+0000 7fe3db200640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 0x7fe3dc1f4770 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400efc0 tx=0x7fe3d400c490 comp rx=0 tx=0).stop 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 shutdown_connections 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3b806b4d0 0x7fe3b806d990 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3dc144440 0x7fe3dc1f4770 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3dc0cd860 0x7fe3dc0cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 --2- 172.21.15.165:0/2731992507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3dc071700 0x7fe3dc13c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 >> 172.21.15.165:0/2731992507 conn(0x7fe3dc06abe0 msgr2=0x7fe3dc0c50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 shutdown_connections 2024-04-25T10:07:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:06.953+0000 7fe3db200640 1 -- 172.21.15.165:0/2731992507 wait complete. 2024-04-25T10:07:06.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:20.857210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:22.010030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:06.978 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:07.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.172+0000 7fdfcb800640 1 Processor -- start 2024-04-25T10:07:07.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.172+0000 7fdfcb800640 1 -- start start 2024-04-25T10:07:07.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfcb800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfcb800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdfc4144d50 0x7fdfc40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfcb800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc40cccc0 con 0x7fdfc41452a0 2024-04-25T10:07:07.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfcb800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc40b2b20 con 0x7fdfc4144d50 2024-04-25T10:07:07.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfcb800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc406d400 con 0x7fdfc41446d0 2024-04-25T10:07:07.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfc9800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fdfc41452a0 0x7fdfc40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44796/0 (socket says 172.21.15.165:44796) 2024-04-25T10:07:07.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfc3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdfc4144d50 0x7fdfc40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfc9800640 1 -- 172.21.15.165:0/3284667903 learned_addr learned my addr 172.21.15.165:0/3284667903 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:07.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.174+0000 7fdfc8e00640 1 --2- 172.21.15.165:0/3284667903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.175+0000 7fdfc3e00640 1 -- 172.21.15.165:0/3284667903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 msgr2=0x7fdfc40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.175+0000 7fdfc3e00640 1 --2- 172.21.15.165:0/3284667903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc40b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.175+0000 7fdfc3e00640 1 -- 172.21.15.165:0/3284667903 >> v1:172.21.15.165:6789/0 conn(0x7fdfc41452a0 legacy=0x7fdfc40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.175+0000 7fdfc3e00640 1 -- 172.21.15.165:0/3284667903 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40cc140 con 0x7fdfc4144d50 2024-04-25T10:07:07.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.175+0000 7fdfc3400640 1 -- 172.21.15.165:0/3284667903 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163066347 0 0) 0x7fdfc40cccc0 con 0x7fdfc41452a0 2024-04-25T10:07:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.176+0000 7fdfc3e00640 1 --2- 172.21.15.165:0/3284667903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b4460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00097a0 tx=0x7fdfb002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda96eaba69f2385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.176+0000 7fdfc3400640 1 -- 172.21.15.165:0/3284667903 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037070 con 0x7fdfc4144d50 2024-04-25T10:07:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.176+0000 7fdfc3400640 1 -- 172.21.15.165:0/3284667903 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb0032440 con 0x7fdfc4144d50 2024-04-25T10:07:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.176+0000 7fdfc3400640 1 -- 172.21.15.165:0/3284667903 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0032810 con 0x7fdfc4144d50 2024-04-25T10:07:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 -- 172.21.15.165:0/3284667903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 msgr2=0x7fdfc40b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 --2- 172.21.15.165:0/3284667903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b4460 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fdfb00097a0 tx=0x7fdfb002a180 comp rx=0 tx=0).stop 2024-04-25T10:07:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 -- 172.21.15.165:0/3284667903 shutdown_connections 2024-04-25T10:07:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 --2- 172.21.15.165:0/3284667903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b4460 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 --2- 172.21.15.165:0/3284667903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc40b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 -- 172.21.15.165:0/3284667903 >> 172.21.15.165:0/3284667903 conn(0x7fdfc406abe0 msgr2=0x7fdfc406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 -- 172.21.15.165:0/3284667903 shutdown_connections 2024-04-25T10:07:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.177+0000 7fdfcb800640 1 -- 172.21.15.165:0/3284667903 wait complete. 2024-04-25T10:07:07.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.178+0000 7fdfcb800640 1 Processor -- start 2024-04-25T10:07:07.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.179+0000 7fdfcb800640 1 -- start start 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfcb800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfcb800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfcb800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc41452a0 0x7fdfc413ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfcb800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40cc700 con 0x7fdfc4144d50 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfcb800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b49a0 con 0x7fdfc41452a0 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfcb800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40cc9e0 con 0x7fdfc41446d0 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b09a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47336/0 (socket says 172.21.15.165:47336) 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 -- 172.21.15.165:0/2325156325 learned_addr learned my addr 172.21.15.165:0/2325156325 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc8e00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc9800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc41452a0 0x7fdfc413ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 -- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 msgr2=0x7fdfc415b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 -- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc41452a0 msgr2=0x7fdfc413ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc41452a0 0x7fdfc413ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.180+0000 7fdfc3e00640 1 -- 172.21.15.165:0/2325156325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40b1310 con 0x7fdfc4144d50 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.181+0000 7fdfc3e00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b09a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0035040 tx=0x7fdfb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.181+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0037040 con 0x7fdfc4144d50 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.181+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfb0030770 con 0x7fdfc4144d50 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.181+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfb0030b80 con 0x7fdfc4144d50 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.181+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40b15f0 con 0x7fdfc4144d50 2024-04-25T10:07:07.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.181+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40b1bb0 con 0x7fdfc4144d50 2024-04-25T10:07:07.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.182+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc4073a00 con 0x7fdfc4144d50 2024-04-25T10:07:07.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.183+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfb0044020 con 0x7fdfc4144d50 2024-04-25T10:07:07.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.184+0000 7fdfc0c00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa406b4d0 0x7fdfa406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.184+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdfb00aa8f0 con 0x7fdfc4144d50 2024-04-25T10:07:07.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.185+0000 7fdfc8e00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa406b4d0 0x7fdfa406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.185+0000 7fdfc8e00640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa406b4d0 0x7fdfa406d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.189+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfb0032e20 con 0x7fdfc4144d50 2024-04-25T10:07:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.380+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdfc415b9c0 con 0x7fdfc4144d50 2024-04-25T10:07:07.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.381+0000 7fdfc0c00640 1 -- 172.21.15.165:0/2325156325 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdfb00795a0 con 0x7fdfc4144d50 2024-04-25T10:07:07.386 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:07:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa406b4d0 msgr2=0x7fdfa406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa406b4d0 0x7fdfa406d990 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdfb8006fd0 tx=0x7fdfb8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 msgr2=0x7fdfc40b09a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b09a0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fdfb0035040 tx=0x7fdfb00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 shutdown_connections 2024-04-25T10:07:07.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa406b4d0 0x7fdfa406d990 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc41452a0 0x7fdfc413ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc4144d50 0x7fdfc40b09a0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 --2- 172.21.15.165:0/2325156325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc41446d0 0x7fdfc415b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.383+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 >> 172.21.15.165:0/2325156325 conn(0x7fdfc406abe0 msgr2=0x7fdfc40cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.384+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 shutdown_connections 2024-04-25T10:07:07.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.384+0000 7fdfcb800640 1 -- 172.21.15.165:0/2325156325 wait complete. 2024-04-25T10:07:07.406 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:07.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.601+0000 7f5ef7800640 1 Processor -- start 2024-04-25T10:07:07.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.601+0000 7f5ef7800640 1 -- start start 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5ef7800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ef00ced80 0x7f5ef00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5ef7800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ef0161710 0x7f5ef00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5ef7800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef015e000 con 0x7f5ef015dc20 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5ef7800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef00d1de0 con 0x7f5ef00ced80 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5ef7800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef0161c60 con 0x7f5ef0161710 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5eefe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ef00ced80 0x7f5ef00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5eefe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ef00ced80 0x7f5ef00d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47350/0 (socket says 172.21.15.165:47350) 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.603+0000 7f5eefe00640 1 -- 172.21.15.165:0/2275662436 learned_addr learned my addr 172.21.15.165:0/2275662436 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:07.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.604+0000 7f5ef5800640 1 --2- 172.21.15.165:0/2275662436 >> v2:172.21.15.172:3300/0 conn(0x7f5ef0161710 0x7f5ef00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.604+0000 7f5eefe00640 1 -- 172.21.15.165:0/2275662436 >> v2:172.21.15.172:3300/0 conn(0x7f5ef0161710 msgr2=0x7f5ef00d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.604+0000 7f5eefe00640 1 --2- 172.21.15.165:0/2275662436 >> v2:172.21.15.172:3300/0 conn(0x7f5ef0161710 0x7f5ef00d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.604+0000 7f5eefe00640 1 -- 172.21.15.165:0/2275662436 >> v1:172.21.15.172:6789/0 conn(0x7f5ef015dc20 legacy=0x7f5ef00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.604+0000 7f5eefe00640 1 -- 172.21.15.165:0/2275662436 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0161f40 con 0x7f5ef00ced80 2024-04-25T10:07:07.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.605+0000 7f5eefe00640 1 --2- 172.21.15.165:0/2275662436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef00ced80 0x7f5ef00d22e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00b6570 tx=0x7f5edc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9889b67f103425b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.605+0000 7f5eef400640 1 -- 172.21.15.165:0/2275662436 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc037070 con 0x7f5ef00ced80 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.605+0000 7f5eef400640 1 -- 172.21.15.165:0/2275662436 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5edc032440 con 0x7f5ef00ced80 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.605+0000 7f5eef400640 1 -- 172.21.15.165:0/2275662436 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5edc032810 con 0x7f5ef00ced80 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 -- 172.21.15.165:0/2275662436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef00ced80 msgr2=0x7f5ef00d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 --2- 172.21.15.165:0/2275662436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef00ced80 0x7f5ef00d22e0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00b6570 tx=0x7f5edc02a230 comp rx=0 tx=0).stop 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 -- 172.21.15.165:0/2275662436 shutdown_connections 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 --2- 172.21.15.165:0/2275662436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef00ced80 0x7f5ef00d22e0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 --2- 172.21.15.165:0/2275662436 >> v2:172.21.15.172:3300/0 conn(0x7f5ef0161710 0x7f5ef00d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 -- 172.21.15.165:0/2275662436 >> 172.21.15.165:0/2275662436 conn(0x7f5ef00d0420 msgr2=0x7f5ef00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 -- 172.21.15.165:0/2275662436 shutdown_connections 2024-04-25T10:07:07.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.606+0000 7f5ef7800640 1 -- 172.21.15.165:0/2275662436 wait complete. 2024-04-25T10:07:07.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.608+0000 7f5ef7800640 1 Processor -- start 2024-04-25T10:07:07.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.608+0000 7f5ef7800640 1 -- start start 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef7800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ef00ced80 0x7f5ef01534b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef7800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ef015dc20 0x7f5ef0156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef7800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 0x7f5ef00c9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef7800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00b5e50 con 0x7f5ef0161710 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef7800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0162500 con 0x7f5ef015dc20 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef7800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00b5fd0 con 0x7f5ef00ced80 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 0x7f5ef00c9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 0x7f5ef00c9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47358/0 (socket says 172.21.15.165:47358) 2024-04-25T10:07:07.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.609+0000 7f5ef5800640 1 -- 172.21.15.165:0/1823762606 learned_addr learned my addr 172.21.15.165:0/1823762606 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef5800640 1 -- 172.21.15.165:0/1823762606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ef00ced80 msgr2=0x7f5ef01534b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef5800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ef00ced80 0x7f5ef01534b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef5800640 1 -- 172.21.15.165:0/1823762606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ef015dc20 msgr2=0x7f5ef0156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef5800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ef015dc20 0x7f5ef0156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef5800640 1 -- 172.21.15.165:0/1823762606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef0156650 con 0x7f5ef0161710 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef5800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 0x7f5ef00c9480 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800d490 tx=0x7f5ee800d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80125f0 con 0x7f5ef0161710 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee8019070 con 0x7f5ef0161710 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef00c9cf0 con 0x7f5ef0161710 2024-04-25T10:07:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.610+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee80129f0 con 0x7f5ef0161710 2024-04-25T10:07:07.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.611+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef015e9f0 con 0x7f5ef0161710 2024-04-25T10:07:07.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.612+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef0073a00 con 0x7f5ef0161710 2024-04-25T10:07:07.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.613+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ee8012b90 con 0x7f5ef0161710 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.613+0000 7f5eecc00640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ed006b4d0 0x7f5ed006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.614+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5ee808a740 con 0x7f5ef0161710 2024-04-25T10:07:07.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.614+0000 7f5ef4e00640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ed006b4d0 0x7f5ed006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:07.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.615+0000 7f5ef4e00640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ed006b4d0 0x7f5ed006d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000ab90 tx=0x7f5ee0009230 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:07.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.618+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ee8059360 con 0x7f5ef0161710 2024-04-25T10:07:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.811+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5ef00c6fc0 con 0x7f5ef0161710 2024-04-25T10:07:07.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.812+0000 7f5eecc00640 1 -- 172.21.15.165:0/1823762606 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ee8058c60 con 0x7f5ef0161710 2024-04-25T10:07:07.817 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ed006b4d0 msgr2=0x7f5ed006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ed006b4d0 0x7f5ed006d990 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000ab90 tx=0x7f5ee0009230 comp rx=0 tx=0).stop 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 msgr2=0x7f5ef00c9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 0x7f5ef00c9480 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800d490 tx=0x7f5ee800d950 comp rx=0 tx=0).stop 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 shutdown_connections 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ed006b4d0 0x7f5ed006d990 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ef0161710 0x7f5ef00c9480 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ef015dc20 0x7f5ef0156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 --2- 172.21.15.165:0/1823762606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ef00ced80 0x7f5ef01534b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 >> 172.21.15.165:0/1823762606 conn(0x7f5ef00d0420 msgr2=0x7f5ef00b76b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.814+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 shutdown_connections 2024-04-25T10:07:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:07.815+0000 7f5ef7800640 1 -- 172.21.15.165:0/1823762606 wait complete. 2024-04-25T10:07:07.839 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:08.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.029+0000 7fae4ee00640 1 Processor -- start 2024-04-25T10:07:08.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.029+0000 7fae4ee00640 1 -- start start 2024-04-25T10:07:08.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae4ee00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fae4815b710 0x7fae480cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae4ee00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4806f0e0 con 0x7fae480ac8b0 2024-04-25T10:07:08.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae4ee00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae480cfd80 con 0x7fae4806ec50 2024-04-25T10:07:08.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae4ee00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae4815e490 con 0x7fae4815b710 2024-04-25T10:07:08.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae4ce00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fae480ac8b0 0x7fae48071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44812/0 (socket says 172.21.15.165:44812) 2024-04-25T10:07:08.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae4ce00640 1 -- 172.21.15.165:0/4006941944 learned_addr learned my addr 172.21.15.165:0/4006941944 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:08.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.031+0000 7fae47e00640 1 --2- 172.21.15.165:0/4006941944 >> v2:172.21.15.165:3300/0 conn(0x7fae4815b710 0x7fae480cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.032+0000 7fae46a00640 1 -- 172.21.15.165:0/4006941944 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203505985 0 0) 0x7fae4806f0e0 con 0x7fae480ac8b0 2024-04-25T10:07:08.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.032+0000 7fae46a00640 1 -- 172.21.15.165:0/4006941944 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae2c003520 con 0x7fae480ac8b0 2024-04-25T10:07:08.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.032+0000 7fae47e00640 1 -- 172.21.15.165:0/4006941944 >> v1:172.21.15.172:6789/0 conn(0x7fae4806ec50 legacy=0x7fae4815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.032+0000 7fae47e00640 1 -- 172.21.15.165:0/4006941944 >> v1:172.21.15.165:6789/0 conn(0x7fae480ac8b0 legacy=0x7fae48071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.032+0000 7fae47e00640 1 -- 172.21.15.165:0/4006941944 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae480d0060 con 0x7fae4815b710 2024-04-25T10:07:08.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.033+0000 7fae47e00640 1 --2- 172.21.15.165:0/4006941944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4815b710 0x7fae480cf640 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fae380097a0 tx=0x7fae3802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88024304f8f54580 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.034+0000 7fae46a00640 1 -- 172.21.15.165:0/4006941944 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38037070 con 0x7fae4815b710 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.034+0000 7fae46a00640 1 -- 172.21.15.165:0/4006941944 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38032440 con 0x7fae4815b710 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.034+0000 7fae46a00640 1 -- 172.21.15.165:0/4006941944 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae380327f0 con 0x7fae4815b710 2024-04-25T10:07:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.034+0000 7fae4ee00640 1 -- 172.21.15.165:0/4006941944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4815b710 msgr2=0x7fae480cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.034+0000 7fae4ee00640 1 --2- 172.21.15.165:0/4006941944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4815b710 0x7fae480cf640 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fae380097a0 tx=0x7fae3802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.035+0000 7fae4ee00640 1 -- 172.21.15.165:0/4006941944 shutdown_connections 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.035+0000 7fae4ee00640 1 --2- 172.21.15.165:0/4006941944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4815b710 0x7fae480cf640 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.035+0000 7fae4ee00640 1 -- 172.21.15.165:0/4006941944 >> 172.21.15.165:0/4006941944 conn(0x7fae4806abe0 msgr2=0x7fae4806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.035+0000 7fae4ee00640 1 -- 172.21.15.165:0/4006941944 shutdown_connections 2024-04-25T10:07:08.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.035+0000 7fae4ee00640 1 -- 172.21.15.165:0/4006941944 wait complete. 2024-04-25T10:07:08.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.036+0000 7fae4ee00640 1 Processor -- start 2024-04-25T10:07:08.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.036+0000 7fae4ee00640 1 -- start start 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.037+0000 7fae4ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 0x7fae48141ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.037+0000 7fae4ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae480ac8b0 0x7fae4813eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.037+0000 7fae4ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae4815b710 0x7fae48154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.037+0000 7fae4ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4815bc70 con 0x7fae4806ec50 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.037+0000 7fae4ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4815baf0 con 0x7fae480ac8b0 2024-04-25T10:07:08.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.037+0000 7fae4ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae4815f030 con 0x7fae4815b710 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 0x7fae48141ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 0x7fae48141ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47376/0 (socket says 172.21.15.165:47376) 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 -- 172.21.15.165:0/3271106437 learned_addr learned my addr 172.21.15.165:0/3271106437 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 -- 172.21.15.165:0/3271106437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae4815b710 msgr2=0x7fae48154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae4815b710 0x7fae48154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 -- 172.21.15.165:0/3271106437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae480ac8b0 msgr2=0x7fae4813eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae480ac8b0 0x7fae4813eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 -- 172.21.15.165:0/3271106437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae4813eff0 con 0x7fae4806ec50 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.038+0000 7fae47e00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 0x7fae48141ec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fae38035040 tx=0x7fae380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.039+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38037050 con 0x7fae4806ec50 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.039+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae38043070 con 0x7fae4806ec50 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.039+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae38032810 con 0x7fae4806ec50 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.039+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae481549d0 con 0x7fae4806ec50 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.039+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae48154f90 con 0x7fae4806ec50 2024-04-25T10:07:08.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.040+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae480724a0 con 0x7fae4806ec50 2024-04-25T10:07:08.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.041+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae380329b0 con 0x7fae4806ec50 2024-04-25T10:07:08.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.042+0000 7fae3fe00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae2806b4d0 0x7fae2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.042+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fae380aaec0 con 0x7fae4806ec50 2024-04-25T10:07:08.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.042+0000 7fae47400640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae2806b4d0 0x7fae2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.043+0000 7fae47400640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae2806b4d0 0x7fae2806d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae34000f60 tx=0x7fae340079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.046+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fae480724a0 con 0x7fae4806ec50 2024-04-25T10:07:08.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.243+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fae48073a00 con 0x7fae4806ec50 2024-04-25T10:07:08.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.244+0000 7fae3fe00640 1 -- 172.21.15.165:0/3271106437 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7fae38079c00 con 0x7fae4806ec50 2024-04-25T10:07:08.249 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:07:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae2806b4d0 msgr2=0x7fae2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae2806b4d0 0x7fae2806d990 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fae34000f60 tx=0x7fae340079d0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 msgr2=0x7fae48141ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 0x7fae48141ec0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fae38035040 tx=0x7fae380046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 shutdown_connections 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae2806b4d0 0x7fae2806d990 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae4815b710 0x7fae48154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae480ac8b0 0x7fae4813eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 --2- 172.21.15.165:0/3271106437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae4806ec50 0x7fae48141ec0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.246+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 >> 172.21.15.165:0/3271106437 conn(0x7fae4806abe0 msgr2=0x7fae480713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.247+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 shutdown_connections 2024-04-25T10:07:08.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.247+0000 7fae4ee00640 1 -- 172.21.15.165:0/3271106437 wait complete. 2024-04-25T10:07:08.270 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:08.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.519+0000 7f0b35400640 1 Processor -- start 2024-04-25T10:07:08.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.520+0000 7f0b35400640 1 -- start start 2024-04-25T10:07:08.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b35400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b3006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b35400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300cccc0 con 0x7f0b301446d0 2024-04-25T10:07:08.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b35400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b300cc140 con 0x7f0b30144d50 2024-04-25T10:07:08.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b35400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300b2b20 con 0x7f0b301452a0 2024-04-25T10:07:08.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b2e800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0b301446d0 0x7f0b300b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44820/0 (socket says 172.21.15.165:44820) 2024-04-25T10:07:08.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b2e800640 1 -- 172.21.15.165:0/1653751741 learned_addr learned my addr 172.21.15.165:0/1653751741 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:08.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.521+0000 7f0b2f200640 1 --2- 172.21.15.165:0/1653751741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b3006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2256299685 0 0) 0x7f0b300cccc0 con 0x7f0b301446d0 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b14003520 con 0x7f0b301446d0 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886453901 0 0) 0x7f0b300cc140 con 0x7f0b30144d50 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b300cccc0 con 0x7f0b30144d50 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2f200640 1 -- 172.21.15.165:0/1653751741 >> v1:172.21.15.172:6789/0 conn(0x7f0b30144d50 legacy=0x7f0b3006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2f200640 1 -- 172.21.15.165:0/1653751741 >> v1:172.21.15.165:6789/0 conn(0x7f0b301446d0 legacy=0x7f0b300b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.522+0000 7f0b2f200640 1 -- 172.21.15.165:0/1653751741 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b300cc420 con 0x7f0b301452a0 2024-04-25T10:07:08.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.523+0000 7f0b2f200640 1 --2- 172.21.15.165:0/1653751741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b3006dfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009dc0 tx=0x7f0b2802a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1549b575df2a2d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.523+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28007740 con 0x7f0b301452a0 2024-04-25T10:07:08.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.523+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b280078e0 con 0x7f0b301452a0 2024-04-25T10:07:08.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.523+0000 7f0b2de00640 1 -- 172.21.15.165:0/1653751741 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b28007cb0 con 0x7f0b301452a0 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 -- 172.21.15.165:0/1653751741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 msgr2=0x7f0b3006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 --2- 172.21.15.165:0/1653751741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b3006dfa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b28009dc0 tx=0x7f0b2802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 -- 172.21.15.165:0/1653751741 shutdown_connections 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 --2- 172.21.15.165:0/1653751741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b3006dfa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 -- 172.21.15.165:0/1653751741 >> 172.21.15.165:0/1653751741 conn(0x7f0b3006abe0 msgr2=0x7f0b3006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 -- 172.21.15.165:0/1653751741 shutdown_connections 2024-04-25T10:07:08.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.524+0000 7f0b35400640 1 -- 172.21.15.165:0/1653751741 wait complete. 2024-04-25T10:07:08.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.526+0000 7f0b35400640 1 Processor -- start 2024-04-25T10:07:08.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.526+0000 7f0b35400640 1 -- start start 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b35400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301446d0 0x7f0b300cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 0x7f0b300a9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b35400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b301f16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b35400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b300cc9e0 con 0x7f0b30144d50 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b35400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b3006d400 con 0x7f0b301446d0 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b35400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b30144ab0 con 0x7f0b301452a0 2024-04-25T10:07:08.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b25e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 0x7f0b300a9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b25e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 0x7f0b300a9e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47392/0 (socket says 172.21.15.165:47392) 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b25e00640 1 -- 172.21.15.165:0/3388066703 learned_addr learned my addr 172.21.15.165:0/3388066703 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b2e800640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301446d0 0x7f0b300cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.527+0000 7f0b2f200640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b301f16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b25e00640 1 -- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 msgr2=0x7f0b301f16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b25e00640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b301f16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b25e00640 1 -- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301446d0 msgr2=0x7f0b300cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b25e00640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301446d0 0x7f0b300cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b25e00640 1 -- 172.21.15.165:0/3388066703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b301f20e0 con 0x7f0b30144d50 2024-04-25T10:07:08.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b25e00640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 0x7f0b300a9e20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800c500 tx=0x7f0b1800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b180052d0 con 0x7f0b30144d50 2024-04-25T10:07:08.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b18005490 con 0x7f0b30144d50 2024-04-25T10:07:08.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b18012870 con 0x7f0b30144d50 2024-04-25T10:07:08.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b301f23c0 con 0x7f0b30144d50 2024-04-25T10:07:08.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.528+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b300a9900 con 0x7f0b30144d50 2024-04-25T10:07:08.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.529+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b30073a00 con 0x7f0b30144d50 2024-04-25T10:07:08.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.530+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b18012a10 con 0x7f0b30144d50 2024-04-25T10:07:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.531+0000 7f0b27400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.531+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0b18089fe0 con 0x7f0b30144d50 2024-04-25T10:07:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.532+0000 7f0b2e800640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:08.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.532+0000 7f0b2e800640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b20000f60 tx=0x7f0b20007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:08.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.536+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b180585e0 con 0x7f0b30144d50 2024-04-25T10:07:08.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.756+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0b30067df0 con 0x7f0b30144d50 2024-04-25T10:07:08.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.757+0000 7f0b27400640 1 -- 172.21.15.165:0/3388066703 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v56) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0b1805c680 con 0x7f0b30144d50 2024-04-25T10:07:08.762 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:07:08.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b0c06b4d0 msgr2=0x7f0b0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f0b20000f60 tx=0x7f0b20007d30 comp rx=0 tx=0).stop 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 msgr2=0x7f0b300a9e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 0x7f0b300a9e20 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f0b1800c500 tx=0x7f0b1800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 shutdown_connections 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b0c06b4d0 0x7f0b0c06d990 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b301452a0 0x7f0b301f16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b30144d50 0x7f0b300a9e20 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 --2- 172.21.15.165:0/3388066703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b301446d0 0x7f0b300cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 >> 172.21.15.165:0/3388066703 conn(0x7f0b3006abe0 msgr2=0x7f0b300b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 shutdown_connections 2024-04-25T10:07:08.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:08.760+0000 7f0b35400640 1 -- 172.21.15.165:0/3388066703 wait complete. 2024-04-25T10:07:08.782 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:08.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:08.782 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:09.324 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:07:09.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:09.325 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:09.367 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:07:09.367 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:09.406 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:09.406 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:09.406 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:09.460 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:09.461 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:07:09.461 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:09.511 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:09.512 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:09.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.716+0000 7f4582600640 1 Processor -- start 2024-04-25T10:07:09.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.716+0000 7f4582600640 1 -- start start 2024-04-25T10:07:09.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.717+0000 7f4582600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f457c144440 0x7f457c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.717+0000 7f4582600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f457c0cda30 0x7f457c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.717+0000 7f4582600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f457c15bf30 con 0x7f457c071530 2024-04-25T10:07:09.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.717+0000 7f4582600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457c0719a0 con 0x7f457c144440 2024-04-25T10:07:09.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.717+0000 7f4582600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f457c15c860 con 0x7f457c0cda30 2024-04-25T10:07:09.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f457c144440 0x7f457c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f457c144440 0x7f457c070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47394/0 (socket says 172.21.15.165:47394) 2024-04-25T10:07:09.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 -- 172.21.15.165:0/2798501163 learned_addr learned my addr 172.21.15.165:0/2798501163 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:09.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 -- 172.21.15.165:0/2798501163 >> v2:172.21.15.172:3300/0 conn(0x7f457c0cda30 msgr2=0x7f457c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:09.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 --2- 172.21.15.165:0/2798501163 >> v2:172.21.15.172:3300/0 conn(0x7f457c0cda30 0x7f457c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 -- 172.21.15.165:0/2798501163 >> v1:172.21.15.165:6789/0 conn(0x7f457c071530 legacy=0x7f457c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.718+0000 7f4581200640 1 -- 172.21.15.165:0/2798501163 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c15c210 con 0x7f457c144440 2024-04-25T10:07:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.719+0000 7f4581200640 1 --2- 172.21.15.165:0/2798501163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c070ff0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f457c0b2280 tx=0x7f456402a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30715d7603c5b9f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.719+0000 7f457be00640 1 -- 172.21.15.165:0/2798501163 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564037070 con 0x7f457c144440 2024-04-25T10:07:09.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f457be00640 1 -- 172.21.15.165:0/2798501163 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456402ae50 con 0x7f457c144440 2024-04-25T10:07:09.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f457be00640 1 -- 172.21.15.165:0/2798501163 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4564032860 con 0x7f457c144440 2024-04-25T10:07:09.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f4582600640 1 -- 172.21.15.165:0/2798501163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 msgr2=0x7f457c070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f4582600640 1 --2- 172.21.15.165:0/2798501163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c070ff0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f457c0b2280 tx=0x7f456402a300 comp rx=0 tx=0).stop 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f4582600640 1 -- 172.21.15.165:0/2798501163 shutdown_connections 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f4582600640 1 --2- 172.21.15.165:0/2798501163 >> v2:172.21.15.172:3300/0 conn(0x7f457c0cda30 0x7f457c15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f4582600640 1 --2- 172.21.15.165:0/2798501163 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c070ff0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.720+0000 7f4582600640 1 -- 172.21.15.165:0/2798501163 >> 172.21.15.165:0/2798501163 conn(0x7f457c06abe0 msgr2=0x7f457c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.721+0000 7f4582600640 1 -- 172.21.15.165:0/2798501163 shutdown_connections 2024-04-25T10:07:09.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.721+0000 7f4582600640 1 -- 172.21.15.165:0/2798501163 wait complete. 2024-04-25T10:07:09.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.722+0000 7f4582600640 1 Processor -- start 2024-04-25T10:07:09.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.722+0000 7f4582600640 1 -- start start 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.723+0000 7f4582600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f457c071530 0x7f457c13f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.723+0000 7f4582600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f457c0cda30 0x7f457c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.723+0000 7f4582600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c1f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.723+0000 7f4582600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15bc30 con 0x7f457c144440 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.723+0000 7f4582600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f457c0cdea0 con 0x7f457c0cda30 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.723+0000 7f4582600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f457c15d230 con 0x7f457c071530 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c1f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c1f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47396/0 (socket says 172.21.15.165:47396) 2024-04-25T10:07:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 -- 172.21.15.165:0/3575504944 learned_addr learned my addr 172.21.15.165:0/3575504944 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 -- 172.21.15.165:0/3575504944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f457c071530 msgr2=0x7f457c13f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f457c071530 0x7f457c13f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 -- 172.21.15.165:0/3575504944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f457c0cda30 msgr2=0x7f457c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f457c0cda30 0x7f457c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 -- 172.21.15.165:0/3575504944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f457c13f550 con 0x7f457c144440 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.724+0000 7f4581c00640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c1f48f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f457400b970 tx=0x7f457400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.725+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574019070 con 0x7f457c144440 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.725+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457400cb60 con 0x7f457c144440 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.725+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4574014440 con 0x7f457c144440 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.725+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f457c0cb610 con 0x7f457c144440 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.725+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f457c0cbbb0 con 0x7f457c144440 2024-04-25T10:07:09.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.726+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f457c073990 con 0x7f457c144440 2024-04-25T10:07:09.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.727+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f457401c070 con 0x7f457c144440 2024-04-25T10:07:09.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.728+0000 7f4579600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f455806b4d0 0x7f455806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:09.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.728+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f457408a100 con 0x7f457c144440 2024-04-25T10:07:09.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.728+0000 7f4581200640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f455806b4d0 0x7f455806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:09.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.729+0000 7f4581200640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f455806b4d0 0x7f455806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4564034000 tx=0x7f456402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:09.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.732+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4574058db0 con 0x7f457c144440 2024-04-25T10:07:09.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.965+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f457c15cf80 con 0x7f457c144440 2024-04-25T10:07:09.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.966+0000 7f4579600640 1 -- 172.21.15.165:0/3575504944 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4948 (secure 0 0 0) 0x7f45740586b0 con 0x7f457c144440 2024-04-25T10:07:09.972 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:09.972 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:07:03.631703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:09.972 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 31 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.968+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f455806b4d0 msgr2=0x7f455806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.968+0000 7f4582600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f455806b4d0 0x7f455806d990 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f4564034000 tx=0x7f456402c000 comp rx=0 tx=0).stop 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 msgr2=0x7f457c1f48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c1f48f0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f457400b970 tx=0x7f457400be30 comp rx=0 tx=0).stop 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 shutdown_connections 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f455806b4d0 0x7f455806d990 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f457c144440 0x7f457c1f48f0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f457c0cda30 0x7f457c0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 --2- 172.21.15.165:0/3575504944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f457c071530 0x7f457c13f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 >> 172.21.15.165:0/3575504944 conn(0x7f457c06abe0 msgr2=0x7f457c13fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 shutdown_connections 2024-04-25T10:07:09.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:09.969+0000 7f4582600640 1 -- 172.21.15.165:0/3575504944 wait complete. 2024-04-25T10:07:09.993 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.186+0000 7fcd05800640 1 Processor -- start 2024-04-25T10:07:10.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.187+0000 7fcd05800640 1 -- start start 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.188+0000 7fcd05800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd00144750 0x7fcd0006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.188+0000 7fcd05800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd00144b30 con 0x7fcd000cf030 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.188+0000 7fcd05800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd000712d0 con 0x7fcd0006eb10 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.188+0000 7fcd05800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000715b0 con 0x7fcd00144750 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccff400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcd000cf030 0x7fcd0006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44836/0 (socket says 172.21.15.165:44836) 2024-04-25T10:07:10.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccff400640 1 -- 172.21.15.165:0/3109213324 learned_addr learned my addr 172.21.15.165:0/3109213324 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:10.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfea00640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd00144750 0x7fcd0006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082553102 0 0) 0x7fcd00144b30 con 0x7fcd000cf030 2024-04-25T10:07:10.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce4003520 con 0x7fcd000cf030 2024-04-25T10:07:10.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2808089652 0 0) 0x7fcce4003520 con 0x7fcd000cf030 2024-04-25T10:07:10.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd00144750 msgr2=0x7fcd0006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd00144750 0x7fcd0006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 >> v1:172.21.15.172:6789/0 conn(0x7fcd0006eb10 legacy=0x7fcd000ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd00071890 con 0x7fcd000cf030 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fccf8001670 con 0x7fcd000cf030 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 >> v1:172.21.15.165:6789/0 conn(0x7fcd000cf030 legacy=0x7fcd0006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.189+0000 7fccfd600640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce40039e0 0x7fcce4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfd600640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce4004800 0x7fcce4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfd600640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce4005210 0x7fcce400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd000715b0 con 0x7fcce4005210 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce400c110 con 0x7fcce4004800 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce400c310 con 0x7fcce40039e0 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccff400640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce4005210 0x7fcce400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfe000640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce4004800 0x7fcce4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccfea00640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce40039e0 0x7fcce4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccff400640 1 -- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce40039e0 msgr2=0x7fcce4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccff400640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce40039e0 0x7fcce4003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccff400640 1 -- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce4004800 msgr2=0x7fcce4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccff400640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcce4004800 0x7fcce4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.190+0000 7fccff400640 1 -- 172.21.15.165:0/3109213324 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce4003520 con 0x7fcce4005210 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.191+0000 7fccff400640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce4005210 0x7fcce400b9c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c240 tx=0x7fcce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22535e829576bfbb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8007d10 con 0x7fcce4005210 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf8005ad0 con 0x7fcce4005210 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fccfd600640 1 -- 172.21.15.165:0/3109213324 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf8007070 con 0x7fcce4005210 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fcd05800640 1 -- 172.21.15.165:0/3109213324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce4005210 msgr2=0x7fcce400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fcd05800640 1 --2- 172.21.15.165:0/3109213324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcce4005210 0x7fcce400b9c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fccf800c240 tx=0x7fcce4005830 comp rx=0 tx=0).stop 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fccff400640 1 -- 172.21.15.165:0/3109213324 reap_dead start 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fcd05800640 1 -- 172.21.15.165:0/3109213324 shutdown_connections 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fcd05800640 1 -- 172.21.15.165:0/3109213324 >> 172.21.15.165:0/3109213324 conn(0x7fcd0006abe0 msgr2=0x7fcd0006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fcd05800640 1 -- 172.21.15.165:0/3109213324 shutdown_connections 2024-04-25T10:07:10.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.192+0000 7fcd05800640 1 -- 172.21.15.165:0/3109213324 wait complete. 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.194+0000 7fcd05800640 1 Processor -- start 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.194+0000 7fcd05800640 1 -- start start 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fcd05800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce4005210 0x7fcd0015ee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fcd05800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 0x7fcd00136340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fcd05800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd00136880 0x7fcd000b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fcd05800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd001455d0 con 0x7fcd0013a1a0 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fcd05800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd00145450 con 0x7fcd00136880 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fcd05800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0015c000 con 0x7fcce4005210 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fccfe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 0x7fcd00136340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fccfe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 0x7fcd00136340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47402/0 (socket says 172.21.15.165:47402) 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.195+0000 7fccfe000640 1 -- 172.21.15.165:0/354062041 learned_addr learned my addr 172.21.15.165:0/354062041 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:10.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfea00640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce4005210 0x7fcd0015ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfe000640 1 -- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce4005210 msgr2=0x7fcd0015ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfe000640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce4005210 0x7fcd0015ee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfe000640 1 -- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd00136880 msgr2=0x7fcd000b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfe000640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd00136880 0x7fcd000b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfe000640 1 -- 172.21.15.165:0/354062041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0015f380 con 0x7fcd0013a1a0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccfe000640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 0x7fcd00136340 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fccec00e4b0 tx=0x7fccec00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.196+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec019070 con 0x7fcd0013a1a0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.197+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccec0052e0 con 0x7fcd0013a1a0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.197+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd00134090 con 0x7fcd0013a1a0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.197+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccec003810 con 0x7fcd0013a1a0 2024-04-25T10:07:10.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.197+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd00134650 con 0x7fcd0013a1a0 2024-04-25T10:07:10.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.198+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd0015d010 con 0x7fcd0013a1a0 2024-04-25T10:07:10.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.199+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccec01c070 con 0x7fcd0013a1a0 2024-04-25T10:07:10.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.200+0000 7fccf6a00640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b500 0x7fcce006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.200+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fccec089900 con 0x7fcd0013a1a0 2024-04-25T10:07:10.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.200+0000 7fccfea00640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b500 0x7fcce006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.201+0000 7fccfea00640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b500 0x7fcce006d9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd0006f5d0 tx=0x7fccf000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.204+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd0015f380 con 0x7fcd0013a1a0 2024-04-25T10:07:10.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.403+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcd000d0510 con 0x7fcd0013a1a0 2024-04-25T10:07:10.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.405+0000 7fccf6a00640 1 -- 172.21.15.165:0/354062041 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17018 (secure 0 0 0) 0x7fccec057f60 con 0x7fcd0013a1a0 2024-04-25T10:07:10.411 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:10.411 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":56,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:46.336008+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:10.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b500 msgr2=0x7fcce006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b500 0x7fcce006d9c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fcd0006f5d0 tx=0x7fccf000b040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 msgr2=0x7fcd00136340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 0x7fcd00136340 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fccec00e4b0 tx=0x7fccec00e970 comp rx=0 tx=0).stop 2024-04-25T10:07:10.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 shutdown_connections 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b500 0x7fcce006d9c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.407+0000 7fcd05800640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd00136880 0x7fcd000b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.408+0000 7fcd05800640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0013a1a0 0x7fcd00136340 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.408+0000 7fcd05800640 1 --2- 172.21.15.165:0/354062041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcce4005210 0x7fcd0015ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.408+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 >> 172.21.15.165:0/354062041 conn(0x7fcd0006abe0 msgr2=0x7fcd000af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.408+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 shutdown_connections 2024-04-25T10:07:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.408+0000 7fcd05800640 1 -- 172.21.15.165:0/354062041 wait complete. 2024-04-25T10:07:10.431 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:06:20.857210+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:06:22.010030+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:10.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:10.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.629+0000 7f3209200640 1 Processor -- start 2024-04-25T10:07:10.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.629+0000 7f3209200640 1 -- start start 2024-04-25T10:07:10.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.630+0000 7f3209200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3204144d50 0x7f320406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.630+0000 7f3209200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32040cccc0 con 0x7f32041446d0 2024-04-25T10:07:10.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3209200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32040cc140 con 0x7f32041452a0 2024-04-25T10:07:10.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3209200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32040b2b20 con 0x7f3204144d50 2024-04-25T10:07:10.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3204144d50 0x7f320406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3204144d50 0x7f320406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47416/0 (socket says 172.21.15.165:47416) 2024-04-25T10:07:10.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f32041446d0 0x7f32040b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44846/0 (socket says 172.21.15.165:44846) 2024-04-25T10:07:10.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201400640 1 -- 172.21.15.165:0/4128452148 learned_addr learned my addr 172.21.15.165:0/4128452148 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:10.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201400640 1 -- 172.21.15.165:0/4128452148 >> v1:172.21.15.172:6789/0 conn(0x7f32041452a0 legacy=0x7f320406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:10.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201400640 1 -- 172.21.15.165:0/4128452148 >> v1:172.21.15.165:6789/0 conn(0x7f32041446d0 legacy=0x7f32040b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.631+0000 7f3201400640 1 -- 172.21.15.165:0/4128452148 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32040cc420 con 0x7f3204144d50 2024-04-25T10:07:10.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.632+0000 7f3201400640 1 --2- 172.21.15.165:0/4128452148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3204144d50 0x7f320406d890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009870 tx=0x7f31f002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=474410eb6e56b646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3200a00640 1 -- 172.21.15.165:0/4128452148 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0037070 con 0x7f3204144d50 2024-04-25T10:07:10.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3200a00640 1 -- 172.21.15.165:0/4128452148 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31f0032440 con 0x7f3204144d50 2024-04-25T10:07:10.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3200a00640 1 -- 172.21.15.165:0/4128452148 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f0032810 con 0x7f3204144d50 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 -- 172.21.15.165:0/4128452148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3204144d50 msgr2=0x7f320406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 --2- 172.21.15.165:0/4128452148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3204144d50 0x7f320406d890 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f31f0009870 tx=0x7f31f002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 -- 172.21.15.165:0/4128452148 shutdown_connections 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 --2- 172.21.15.165:0/4128452148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3204144d50 0x7f320406d890 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 -- 172.21.15.165:0/4128452148 >> 172.21.15.165:0/4128452148 conn(0x7f320406abe0 msgr2=0x7f320406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 -- 172.21.15.165:0/4128452148 shutdown_connections 2024-04-25T10:07:10.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.633+0000 7f3209200640 1 -- 172.21.15.165:0/4128452148 wait complete. 2024-04-25T10:07:10.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.635+0000 7f3209200640 1 Processor -- start 2024-04-25T10:07:10.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.635+0000 7f3209200640 1 -- start start 2024-04-25T10:07:10.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.636+0000 7f3209200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32041446d0 0x7f32041e5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.636+0000 7f3209200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3204144d50 0x7f320413a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.636+0000 7f3209200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 0x7f3204140530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.636+0000 7f3209200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32040cc9e0 con 0x7f32041452a0 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.636+0000 7f3209200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320406d400 con 0x7f32041446d0 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.636+0000 7f3209200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3204144ab0 con 0x7f3204144d50 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 0x7f3204140530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 0x7f3204140530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47424/0 (socket says 172.21.15.165:47424) 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 -- 172.21.15.165:0/2409985975 learned_addr learned my addr 172.21.15.165:0/2409985975 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3201e00640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32041446d0 0x7f32041e5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3201400640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3204144d50 0x7f320413a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 -- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3204144d50 msgr2=0x7f320413a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3204144d50 0x7f320413a060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 -- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32041446d0 msgr2=0x7f32041e5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32041446d0 0x7f32041e5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 -- 172.21.15.165:0/2409985975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320413a5a0 con 0x7f32041452a0 2024-04-25T10:07:10.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3201e00640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32041446d0 0x7f32041e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:10.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.637+0000 7f3202800640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 0x7f3204140530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c780 tx=0x7f31fc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.638+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc017070 con 0x7f32041452a0 2024-04-25T10:07:10.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.638+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31fc003da0 con 0x7f32041452a0 2024-04-25T10:07:10.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.638+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320413ab20 con 0x7f32041452a0 2024-04-25T10:07:10.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.638+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31fc0052b0 con 0x7f32041452a0 2024-04-25T10:07:10.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.638+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320413b0e0 con 0x7f32041452a0 2024-04-25T10:07:10.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.639+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320413a8a0 con 0x7f32041452a0 2024-04-25T10:07:10.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.640+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31fc01a070 con 0x7f32041452a0 2024-04-25T10:07:10.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.641+0000 7f31fa000640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:10.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.641+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f31fc0899b0 con 0x7f32041452a0 2024-04-25T10:07:10.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.641+0000 7f3201e00640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31e406b4d0 0x7f31e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:10.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.642+0000 7f3201e00640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3204140fc0 tx=0x7f31f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:10.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.645+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f320413a8a0 con 0x7f32041452a0 2024-04-25T10:07:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.834+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3204071a80 con 0x7f32041452a0 2024-04-25T10:07:10.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.835+0000 7f31fa000640 1 -- 172.21.15.165:0/2409985975 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v56) v1 ==== 93+0+10 (secure 0 0 0) 0x7f31fc058700 con 0x7f32041452a0 2024-04-25T10:07:10.841 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:07:10.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31e406b4d0 msgr2=0x7f31e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3204140fc0 tx=0x7f31f4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 msgr2=0x7f3204140530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 0x7f3204140530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f31fc00c780 tx=0x7f31fc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 shutdown_connections 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31e406b4d0 0x7f31e406d990 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3204140fc0 tx=0x7f31f4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32041452a0 0x7f3204140530 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3204144d50 0x7f320413a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 --2- 172.21.15.165:0/2409985975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32041446d0 0x7f32041e5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 >> 172.21.15.165:0/2409985975 conn(0x7f320406abe0 msgr2=0x7f320406e020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 shutdown_connections 2024-04-25T10:07:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:10.838+0000 7f3209200640 1 -- 172.21.15.165:0/2409985975 wait complete. 2024-04-25T10:07:10.862 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:11.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.055+0000 7f6009a00640 1 Processor -- start 2024-04-25T10:07:11.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.055+0000 7f6009a00640 1 -- start start 2024-04-25T10:07:11.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.056+0000 7f6009a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f60040ceb20 0x7f60040c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.056+0000 7f6009a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040c5030 0x7f600415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.056+0000 7f6009a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6004145520 con 0x7f6004145140 2024-04-25T10:07:11.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.056+0000 7f6009a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6004071270 con 0x7f60040ceb20 2024-04-25T10:07:11.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.056+0000 7f6009a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6004071550 con 0x7f60040c5030 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f6003e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6004145140 0x7f60040ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44850/0 (socket says 172.21.15.165:44850) 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f6003e00640 1 -- 172.21.15.165:0/1636298240 learned_addr learned my addr 172.21.15.165:0/1636298240 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f5ffb400640 1 --2- 172.21.15.165:0/1636298240 >> v2:172.21.15.165:3300/0 conn(0x7f60040ceb20 0x7f60040c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f6009000640 1 --2- 172.21.15.165:0/1636298240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040c5030 0x7f600415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f6003400640 1 -- 172.21.15.165:0/1636298240 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828102783 0 0) 0x7f6004145520 con 0x7f6004145140 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f6003400640 1 -- 172.21.15.165:0/1636298240 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fe8003520 con 0x7f6004145140 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f5ffb400640 1 -- 172.21.15.165:0/1636298240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040c5030 msgr2=0x7f600415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f5ffb400640 1 --2- 172.21.15.165:0/1636298240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040c5030 0x7f600415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f5ffb400640 1 -- 172.21.15.165:0/1636298240 >> v1:172.21.15.165:6789/0 conn(0x7f6004145140 legacy=0x7f60040ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.057+0000 7f5ffb400640 1 -- 172.21.15.165:0/1636298240 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6004071830 con 0x7f60040ceb20 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.058+0000 7f5ffb400640 1 --2- 172.21.15.165:0/1636298240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040ceb20 0x7f60040c4af0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0097a0 tx=0x7f5fec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=522a5d4dedff3d6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6003400640 1 -- 172.21.15.165:0/1636298240 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec037070 con 0x7f60040ceb20 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6003400640 1 -- 172.21.15.165:0/1636298240 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fec02ae50 con 0x7f60040ceb20 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6003400640 1 -- 172.21.15.165:0/1636298240 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fec032640 con 0x7f60040ceb20 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 -- 172.21.15.165:0/1636298240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040ceb20 msgr2=0x7f60040c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 --2- 172.21.15.165:0/1636298240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040ceb20 0x7f60040c4af0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f5fec0097a0 tx=0x7f5fec02a100 comp rx=0 tx=0).stop 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 -- 172.21.15.165:0/1636298240 shutdown_connections 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 --2- 172.21.15.165:0/1636298240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040c5030 0x7f600415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 --2- 172.21.15.165:0/1636298240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040ceb20 0x7f60040c4af0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 -- 172.21.15.165:0/1636298240 >> 172.21.15.165:0/1636298240 conn(0x7f6004061cc0 msgr2=0x7f600406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 -- 172.21.15.165:0/1636298240 shutdown_connections 2024-04-25T10:07:11.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.059+0000 7f6009a00640 1 -- 172.21.15.165:0/1636298240 wait complete. 2024-04-25T10:07:11.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.061+0000 7f6009a00640 1 Processor -- start 2024-04-25T10:07:11.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.061+0000 7f6009a00640 1 -- start start 2024-04-25T10:07:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6009a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 0x7f6004143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6009a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040ceb20 0x7f600415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6009a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6004145140 0x7f600413dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6009a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040c55c0 con 0x7f60040c5030 2024-04-25T10:07:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6009a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60040c5440 con 0x7f6004145140 2024-04-25T10:07:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6009a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600415c4f0 con 0x7f60040ceb20 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6003e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 0x7f6004143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6003e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 0x7f6004143d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47436/0 (socket says 172.21.15.165:47436) 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.062+0000 7f6003e00640 1 -- 172.21.15.165:0/663000946 learned_addr learned my addr 172.21.15.165:0/663000946 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f5ffb400640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040ceb20 0x7f600415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6009000640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6004145140 0x7f600413dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6003e00640 1 -- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040ceb20 msgr2=0x7f600415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6003e00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040ceb20 0x7f600415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6003e00640 1 -- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6004145140 msgr2=0x7f600413dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6003e00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6004145140 0x7f600413dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6003e00640 1 -- 172.21.15.165:0/663000946 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60040c89e0 con 0x7f60040c5030 2024-04-25T10:07:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6003e00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 0x7f6004143d40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400caa0 tx=0x7f5ff400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4017070 con 0x7f60040c5030 2024-04-25T10:07:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff400ac00 con 0x7f60040c5030 2024-04-25T10:07:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff4012400 con 0x7f60040c5030 2024-04-25T10:07:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.063+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040c8cc0 con 0x7f60040c5030 2024-04-25T10:07:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.064+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60040c9280 con 0x7f60040c5030 2024-04-25T10:07:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.065+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6004073990 con 0x7f60040c5030 2024-04-25T10:07:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.066+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ff401a070 con 0x7f60040c5030 2024-04-25T10:07:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.067+0000 7f6000c00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe006b500 0x7f5fe006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.067+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f5ff408a170 con 0x7f60040c5030 2024-04-25T10:07:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.067+0000 7f5ffb400640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe006b500 0x7f5fe006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.068+0000 7f5ffb400640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe006b500 0x7f5fe006d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.071+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff4058e30 con 0x7f60040c5030 2024-04-25T10:07:11.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.261+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f600415bbe0 con 0x7f60040c5030 2024-04-25T10:07:11.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.262+0000 7f6000c00640 1 -- 172.21.15.165:0/663000946 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v56) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5ff4058730 con 0x7f60040c5030 2024-04-25T10:07:11.267 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe006b500 msgr2=0x7f5fe006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe006b500 0x7f5fe006d9c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f5fec034000 tx=0x7f5fec02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 msgr2=0x7f6004143d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 0x7f6004143d40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f5ff400caa0 tx=0x7f5ff400cf60 comp rx=0 tx=0).stop 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 shutdown_connections 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe006b500 0x7f5fe006d9c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6004145140 0x7f600413dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60040ceb20 0x7f600415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 --2- 172.21.15.165:0/663000946 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60040c5030 0x7f6004143d40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 >> 172.21.15.165:0/663000946 conn(0x7f6004061cc0 msgr2=0x7f600415d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.264+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 shutdown_connections 2024-04-25T10:07:11.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.265+0000 7f6009a00640 1 -- 172.21.15.165:0/663000946 wait complete. 2024-04-25T10:07:11.288 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.475+0000 7f90ba200640 1 Processor -- start 2024-04-25T10:07:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.475+0000 7f90ba200640 1 -- start start 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90ba200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f90b4145140 0x7f90b40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90ba200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90b40ce950 0x7f90b40c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90ba200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b4145520 con 0x7f90b40c5030 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90ba200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b40ced90 con 0x7f90b40ce950 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90ba200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b4071270 con 0x7f90b4145140 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90abe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90b40ce950 0x7f90b40c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90abe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90b40ce950 0x7f90b40c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47442/0 (socket says 172.21.15.165:47442) 2024-04-25T10:07:11.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.477+0000 7f90abe00640 1 -- 172.21.15.165:0/3706534120 learned_addr learned my addr 172.21.15.165:0/3706534120 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.478+0000 7f90b8e00640 1 --2- 172.21.15.165:0/3706534120 >> v2:172.21.15.172:3300/0 conn(0x7f90b4145140 0x7f90b40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.478+0000 7f90abe00640 1 -- 172.21.15.165:0/3706534120 >> v2:172.21.15.172:3300/0 conn(0x7f90b4145140 msgr2=0x7f90b40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.478+0000 7f90abe00640 1 --2- 172.21.15.165:0/3706534120 >> v2:172.21.15.172:3300/0 conn(0x7f90b4145140 0x7f90b40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.478+0000 7f90abe00640 1 -- 172.21.15.165:0/3706534120 >> v1:172.21.15.172:6789/0 conn(0x7f90b40c5030 legacy=0x7f90b406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.478+0000 7f90abe00640 1 -- 172.21.15.165:0/3706534120 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4071550 con 0x7f90b40ce950 2024-04-25T10:07:11.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.478+0000 7f90b8e00640 1 --2- 172.21.15.165:0/3706534120 >> v2:172.21.15.172:3300/0 conn(0x7f90b4145140 0x7f90b40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.479+0000 7f90abe00640 1 --2- 172.21.15.165:0/3706534120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b40ce950 0x7f90b40c4af0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f90b406f690 tx=0x7f909c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71ca658cf67a3eb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.479+0000 7f90b3e00640 1 -- 172.21.15.165:0/3706534120 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c037070 con 0x7f90b40ce950 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.479+0000 7f90b3e00640 1 -- 172.21.15.165:0/3706534120 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909c032440 con 0x7f90b40ce950 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.479+0000 7f90b3e00640 1 -- 172.21.15.165:0/3706534120 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909c032810 con 0x7f90b40ce950 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 -- 172.21.15.165:0/3706534120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b40ce950 msgr2=0x7f90b40c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 --2- 172.21.15.165:0/3706534120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b40ce950 0x7f90b40c4af0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f90b406f690 tx=0x7f909c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 -- 172.21.15.165:0/3706534120 shutdown_connections 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 --2- 172.21.15.165:0/3706534120 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b40ce950 0x7f90b40c4af0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 --2- 172.21.15.165:0/3706534120 >> v2:172.21.15.172:3300/0 conn(0x7f90b4145140 0x7f90b40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 -- 172.21.15.165:0/3706534120 >> 172.21.15.165:0/3706534120 conn(0x7f90b4061cc0 msgr2=0x7f90b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 -- 172.21.15.165:0/3706534120 shutdown_connections 2024-04-25T10:07:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.480+0000 7f90ba200640 1 -- 172.21.15.165:0/3706534120 wait complete. 2024-04-25T10:07:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.482+0000 7f90ba200640 1 Processor -- start 2024-04-25T10:07:11.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.482+0000 7f90ba200640 1 -- start start 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90ba200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b40c5030 0x7f90b40a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90ba200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90b40ce950 0x7f90b4153e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90ba200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 0x7f90b41f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90ba200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c5480 con 0x7f90b4145140 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90ba200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b4071b10 con 0x7f90b40c5030 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90ba200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c5600 con 0x7f90b40ce950 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 0x7f90b41f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 0x7f90b41f6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47456/0 (socket says 172.21.15.165:47456) 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 -- 172.21.15.165:0/3512003607 learned_addr learned my addr 172.21.15.165:0/3512003607 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:11.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 -- 172.21.15.165:0/3512003607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90b40ce950 msgr2=0x7f90b4153e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90b40ce950 0x7f90b4153e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 -- 172.21.15.165:0/3512003607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b40c5030 msgr2=0x7f90b40a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b40c5030 0x7f90b40a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.483+0000 7f90b9800640 1 -- 172.21.15.165:0/3512003607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b413ed20 con 0x7f90b4145140 2024-04-25T10:07:11.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.484+0000 7f90b9800640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 0x7f90b41f6b50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00c780 tx=0x7f90ac00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.484+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac004b60 con 0x7f90b4145140 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.484+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac004d00 con 0x7f90b4145140 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.484+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac00aa90 con 0x7f90b4145140 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.484+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b413f000 con 0x7f90b4145140 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.484+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b40a89e0 con 0x7f90b4145140 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.485+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b4073990 con 0x7f90b4145140 2024-04-25T10:07:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.486+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90ac00ac30 con 0x7f90b4145140 2024-04-25T10:07:11.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.487+0000 7f90b1600640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f909006b500 0x7f909006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.487+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f90ac08a680 con 0x7f90b4145140 2024-04-25T10:07:11.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.487+0000 7f90b8e00640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f909006b500 0x7f909006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.488+0000 7f90b8e00640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f909006b500 0x7f909006d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.492+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90ac020070 con 0x7f90b4145140 2024-04-25T10:07:11.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.693+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f90b415eb70 con 0x7f90b4145140 2024-04-25T10:07:11.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.693+0000 7f90b1600640 1 -- 172.21.15.165:0/3512003607 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v56) v1 ==== 100+0+11 (secure 0 0 0) 0x7f90ac059330 con 0x7f90b4145140 2024-04-25T10:07:11.699 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:07:11.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.696+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f909006b500 msgr2=0x7f909006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.696+0000 7f90ba200640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f909006b500 0x7f909006d9c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f90a4006fd0 tx=0x7f90a4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.696+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 msgr2=0x7f90b41f6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.696+0000 7f90ba200640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 0x7f90b41f6b50 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f90ac00c780 tx=0x7f90ac00cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 shutdown_connections 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f909006b500 0x7f909006d9c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90b4145140 0x7f90b41f6b50 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90b40ce950 0x7f90b4153e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 --2- 172.21.15.165:0/3512003607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90b40c5030 0x7f90b40a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 >> 172.21.15.165:0/3512003607 conn(0x7f90b4061cc0 msgr2=0x7f90b40c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 shutdown_connections 2024-04-25T10:07:11.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.697+0000 7f90ba200640 1 -- 172.21.15.165:0/3512003607 wait complete. 2024-04-25T10:07:11.721 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:11.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.910+0000 7fae9f200640 1 Processor -- start 2024-04-25T10:07:11.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.910+0000 7fae9f200640 1 -- start start 2024-04-25T10:07:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.911+0000 7fae9f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faea015db90 0x7faea00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.911+0000 7fae9f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea0161df0 con 0x7faea01615a0 2024-04-25T10:07:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.911+0000 7fae9f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea01620d0 con 0x7faea00d06c0 2024-04-25T10:07:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.911+0000 7fae9f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00d1f50 con 0x7faea015db90 2024-04-25T10:07:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9d400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7faea01615a0 0x7faea00cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44860/0 (socket says 172.21.15.165:44860) 2024-04-25T10:07:11.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9d400640 1 -- 172.21.15.165:0/3020057121 learned_addr learned my addr 172.21.15.165:0/3020057121 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:11.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3338446686 0 0) 0x7faea0161df0 con 0x7faea01615a0 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae84003520 con 0x7faea01615a0 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9de00640 1 --2- 172.21.15.165:0/3020057121 >> v2:172.21.15.172:3300/0 conn(0x7faea015db90 0x7faea00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2840290055 0 0) 0x7fae84003520 con 0x7faea01615a0 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 >> v2:172.21.15.172:3300/0 conn(0x7faea015db90 msgr2=0x7faea00b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 --2- 172.21.15.165:0/3020057121 >> v2:172.21.15.172:3300/0 conn(0x7faea015db90 0x7faea00b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 >> v1:172.21.15.172:6789/0 conn(0x7faea00d06c0 legacy=0x7faea00d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea01623b0 con 0x7faea01615a0 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.912+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fae8c001670 con 0x7faea01615a0 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 >> v1:172.21.15.165:6789/0 conn(0x7faea01615a0 legacy=0x7faea00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae840039b0 0x7fae84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae84004800 0x7fae84004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae840051c0 0x7fae8400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00d1f50 con 0x7fae840039b0 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae8400c110 con 0x7fae84004800 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae8400c310 con 0x7fae840051c0 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9de00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae840039b0 0x7fae84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9de00640 1 -- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae840051c0 msgr2=0x7fae8400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9de00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae840051c0 0x7fae8400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9de00640 1 -- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae84004800 msgr2=0x7fae84004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9d400640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae84004800 0x7fae84004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9de00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae84004800 0x7fae84004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9de00640 1 -- 172.21.15.165:0/3020057121 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae84003520 con 0x7fae840039b0 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9e800640 1 -- 172.21.15.165:0/3020057121 reap_dead start 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.913+0000 7fae9d400640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae84004800 0x7fae84004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.914+0000 7fae9de00640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae840039b0 0x7fae84003dd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fae9000ba10 tx=0x7fae84005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea10d1a24426410e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae90030070 con 0x7fae840039b0 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9002ae00 con 0x7fae840039b0 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9ca00640 1 -- 172.21.15.165:0/3020057121 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9000b5e0 con 0x7fae840039b0 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 -- 172.21.15.165:0/3020057121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae840039b0 msgr2=0x7fae84003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae840039b0 0x7fae84003dd0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fae9000ba10 tx=0x7fae84005830 comp rx=0 tx=0).stop 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 -- 172.21.15.165:0/3020057121 shutdown_connections 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 --2- 172.21.15.165:0/3020057121 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fae840039b0 0x7fae84003dd0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 -- 172.21.15.165:0/3020057121 >> 172.21.15.165:0/3020057121 conn(0x7faea00b52b0 msgr2=0x7faea00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 -- 172.21.15.165:0/3020057121 shutdown_connections 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.915+0000 7fae9f200640 1 -- 172.21.15.165:0/3020057121 wait complete. 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.917+0000 7fae9f200640 1 Processor -- start 2024-04-25T10:07:11.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.917+0000 7fae9f200640 1 -- start start 2024-04-25T10:07:11.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae840039b0 0x7faea01f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 0x7faea01f48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faea01f4e30 0x7faea015f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0161ba0 con 0x7faea0160890 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea0161a20 con 0x7fae840039b0 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea00b68d0 con 0x7faea01f4e30 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 0x7faea01f48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 0x7faea01f48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47474/0 (socket says 172.21.15.165:47474) 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.918+0000 7fae9d400640 1 -- 172.21.15.165:0/1441760807 learned_addr learned my addr 172.21.15.165:0/1441760807 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9de00640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae840039b0 0x7faea01f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9d400640 1 -- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faea01f4e30 msgr2=0x7faea015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9d400640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faea01f4e30 0x7faea015f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9d400640 1 -- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae840039b0 msgr2=0x7faea01f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9d400640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae840039b0 0x7faea01f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9d400640 1 -- 172.21.15.165:0/1441760807 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea015e010 con 0x7faea0160890 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9d400640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 0x7faea01f48f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00d810 tx=0x7fae8c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c019070 con 0x7faea0160890 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae8c0074e0 con 0x7faea0160890 2024-04-25T10:07:11.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae8c004a00 con 0x7faea0160890 2024-04-25T10:07:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.919+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea01f46f0 con 0x7faea0160890 2024-04-25T10:07:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.920+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea015e560 con 0x7faea0160890 2024-04-25T10:07:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.921+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea00cab30 con 0x7faea0160890 2024-04-25T10:07:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.922+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fae8c01c070 con 0x7faea0160890 2024-04-25T10:07:11.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.922+0000 7fae96000640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae8006b520 0x7fae8006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:11.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.923+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fae8c089cf0 con 0x7faea0160890 2024-04-25T10:07:11.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.923+0000 7fae9de00640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae8006b520 0x7fae8006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:11.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.924+0000 7fae9de00640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae8006b520 0x7fae8006d9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b9e0 tx=0x7fae90030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:11.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:11.927+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faea01f46f0 con 0x7faea0160890 2024-04-25T10:07:12.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.160+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faea015d080 con 0x7faea0160890 2024-04-25T10:07:12.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.161+0000 7fae96000640 1 -- 172.21.15.165:0/1441760807 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fae8c0583d0 con 0x7faea0160890 2024-04-25T10:07:12.167 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:12.167 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":31,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:07:03.631703+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":13,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:12.167 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 31 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae8006b520 msgr2=0x7fae8006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae8006b520 0x7fae8006d9e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fae9000b9e0 tx=0x7fae90030000 comp rx=0 tx=0).stop 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 msgr2=0x7faea01f48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 0x7faea01f48f0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fae8c00d810 tx=0x7fae8c00dcd0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 shutdown_connections 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fae8006b520 0x7fae8006d9e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faea01f4e30 0x7faea015f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faea0160890 0x7faea01f48f0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 --2- 172.21.15.165:0/1441760807 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae840039b0 0x7faea01f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 >> 172.21.15.165:0/1441760807 conn(0x7faea00b52b0 msgr2=0x7faea00b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 shutdown_connections 2024-04-25T10:07:12.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.164+0000 7fae9f200640 1 -- 172.21.15.165:0/1441760807 wait complete. 2024-04-25T10:07:12.188 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.375+0000 7f533e600640 1 Processor -- start 2024-04-25T10:07:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.375+0000 7f533e600640 1 -- start start 2024-04-25T10:07:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.376+0000 7f533e600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f533815db60 0x7f53380ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.376+0000 7f533e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.376+0000 7f533e600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53380cec00 con 0x7f53380b5f00 2024-04-25T10:07:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.376+0000 7f533e600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53380ce930 con 0x7f533815db60 2024-04-25T10:07:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.376+0000 7f533e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53380b6450 con 0x7f53380d0b10 2024-04-25T10:07:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.377+0000 7f5337e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.377+0000 7f5337400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f533815db60 0x7f53380ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.377+0000 7f5337e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:34238/0 (socket says 172.21.15.165:34238) 2024-04-25T10:07:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.377+0000 7f5337e00640 1 -- 172.21.15.165:0/3999565102 learned_addr learned my addr 172.21.15.165:0/3999565102 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.378+0000 7f5337e00640 1 -- 172.21.15.165:0/3999565102 >> v2:172.21.15.172:3300/0 conn(0x7f533815db60 msgr2=0x7f53380ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.378+0000 7f5337e00640 1 --2- 172.21.15.165:0/3999565102 >> v2:172.21.15.172:3300/0 conn(0x7f533815db60 0x7f53380ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.378+0000 7f5337e00640 1 -- 172.21.15.165:0/3999565102 >> v1:172.21.15.172:6789/0 conn(0x7f53380b5f00 legacy=0x7f53380d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.378+0000 7f5337e00640 1 -- 172.21.15.165:0/3999565102 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53380ceee0 con 0x7f53380d0b10 2024-04-25T10:07:12.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.379+0000 7f5337e00640 1 --2- 172.21.15.165:0/3999565102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380d2270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5330009e50 tx=0x7f533002a630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1daa1ea1a0da20e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.379+0000 7f5336a00640 1 -- 172.21.15.165:0/3999565102 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330036070 con 0x7f53380d0b10 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f5336a00640 1 -- 172.21.15.165:0/3999565102 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5330031600 con 0x7f53380d0b10 2024-04-25T10:07:12.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f5336a00640 1 -- 172.21.15.165:0/3999565102 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300319d0 con 0x7f53380d0b10 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 -- 172.21.15.165:0/3999565102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 msgr2=0x7f53380d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 --2- 172.21.15.165:0/3999565102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380d2270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5330009e50 tx=0x7f533002a630 comp rx=0 tx=0).stop 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 -- 172.21.15.165:0/3999565102 shutdown_connections 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 --2- 172.21.15.165:0/3999565102 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380d2270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 --2- 172.21.15.165:0/3999565102 >> v2:172.21.15.172:3300/0 conn(0x7f533815db60 0x7f53380ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 -- 172.21.15.165:0/3999565102 >> 172.21.15.165:0/3999565102 conn(0x7f5338061600 msgr2=0x7f5338061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.380+0000 7f533e600640 1 -- 172.21.15.165:0/3999565102 shutdown_connections 2024-04-25T10:07:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.381+0000 7f533e600640 1 -- 172.21.15.165:0/3999565102 wait complete. 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.382+0000 7f533e600640 1 Processor -- start 2024-04-25T10:07:12.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.382+0000 7f533e600640 1 -- start start 2024-04-25T10:07:12.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f533e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53380b5f00 0x7f53381527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f533e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380cc800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f533e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 0x7f53380ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f533e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533815df40 con 0x7f533815db60 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f533e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380d1d50 con 0x7f53380b5f00 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f533e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f533815e0c0 con 0x7f53380d0b10 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.383+0000 7f5337e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 0x7f53380ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 0x7f53380ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47486/0 (socket says 172.21.15.165:47486) 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 -- 172.21.15.165:0/2678950808 learned_addr learned my addr 172.21.15.165:0/2678950808 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 -- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 msgr2=0x7f53380cc800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f532ea00640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380cc800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380cc800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 -- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53380b5f00 msgr2=0x7f53381527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53380b5f00 0x7f53381527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 -- 172.21.15.165:0/2678950808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53381f8030 con 0x7f533815db60 2024-04-25T10:07:12.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.384+0000 7f5337e00640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 0x7f53380ccd40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5330009f70 tx=0x7f5330004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.385+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330036050 con 0x7f533815db60 2024-04-25T10:07:12.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.385+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5330042070 con 0x7f533815db60 2024-04-25T10:07:12.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.385+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5330009510 con 0x7f533815db60 2024-04-25T10:07:12.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.385+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53381f7270 con 0x7f533815db60 2024-04-25T10:07:12.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.385+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53381f7830 con 0x7f533815db60 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.386+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5338073270 con 0x7f533815db60 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.387+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5330007bf0 con 0x7f533815db60 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.388+0000 7f532fe00640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f531406b4d0 0x7f531406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.388+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f53300ab1a0 con 0x7f533815db60 2024-04-25T10:07:12.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.388+0000 7f5337400640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f531406b4d0 0x7f531406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.389+0000 7f5337400640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f531406b4d0 0x7f531406d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5328006a90 tx=0x7f5328006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.392+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5330045360 con 0x7f533815db60 2024-04-25T10:07:12.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.588+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5338073990 con 0x7f533815db60 2024-04-25T10:07:12.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.591+0000 7f532fe00640 1 -- 172.21.15.165:0/2678950808 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17018 (secure 0 0 0) 0x7f5330079f70 con 0x7f533815db60 2024-04-25T10:07:12.597 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:12.597 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":56,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:46.336008+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:12.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f531406b4d0 msgr2=0x7f531406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f531406b4d0 0x7f531406d990 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f5328006a90 tx=0x7f5328006a20 comp rx=0 tx=0).stop 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 msgr2=0x7f53380ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 0x7f53380ccd40 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f5330009f70 tx=0x7f5330004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 shutdown_connections 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f531406b4d0 0x7f531406d990 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f533815db60 0x7f53380ccd40 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53380d0b10 0x7f53380cc800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 --2- 172.21.15.165:0/2678950808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53380b5f00 0x7f53381527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 >> 172.21.15.165:0/2678950808 conn(0x7f5338061600 msgr2=0x7f53380b5310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 shutdown_connections 2024-04-25T10:07:12.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.594+0000 7f533e600640 1 -- 172.21.15.165:0/2678950808 wait complete. 2024-04-25T10:07:12.618 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:07:12.618 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:12.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.809+0000 7fb699a00640 1 Processor -- start 2024-04-25T10:07:12.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.809+0000 7fb699a00640 1 -- start start 2024-04-25T10:07:12.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.810+0000 7fb699a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb69415e900 0x7fb6940adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.810+0000 7fb699a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69415ed90 con 0x7fb69415e520 2024-04-25T10:07:12.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.810+0000 7fb699a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6940ae590 con 0x7fb6940ae0d0 2024-04-25T10:07:12.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.810+0000 7fb699a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6940aee30 con 0x7fb69415e900 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb693e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb69415e520 0x7fb69415f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44866/0 (socket says 172.21.15.165:44866) 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb693e00640 1 -- 172.21.15.165:0/2567399703 learned_addr learned my addr 172.21.15.165:0/2567399703 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb68b400640 1 --2- 172.21.15.165:0/2567399703 >> v2:172.21.15.165:3300/0 conn(0x7fb69415e900 0x7fb6940adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb693400640 1 -- 172.21.15.165:0/2567399703 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132133544 0 0) 0x7fb69415ed90 con 0x7fb69415e520 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb693400640 1 -- 172.21.15.165:0/2567399703 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb678003520 con 0x7fb69415e520 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb68b400640 1 -- 172.21.15.165:0/2567399703 >> v1:172.21.15.172:6789/0 conn(0x7fb6940ae0d0 legacy=0x7fb694070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb68b400640 1 -- 172.21.15.165:0/2567399703 >> v1:172.21.15.165:6789/0 conn(0x7fb69415e520 legacy=0x7fb69415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.811+0000 7fb68b400640 1 -- 172.21.15.165:0/2567399703 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940ae870 con 0x7fb69415e900 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.812+0000 7fb68b400640 1 --2- 172.21.15.165:0/2567399703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb69415e900 0x7fb6940adb90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0097a0 tx=0x7fb67c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d2c0a97f8283e1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb693400640 1 -- 172.21.15.165:0/2567399703 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c037070 con 0x7fb69415e900 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb693400640 1 -- 172.21.15.165:0/2567399703 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67c032440 con 0x7fb69415e900 2024-04-25T10:07:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb693400640 1 -- 172.21.15.165:0/2567399703 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67c0327f0 con 0x7fb69415e900 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 -- 172.21.15.165:0/2567399703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb69415e900 msgr2=0x7fb6940adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 --2- 172.21.15.165:0/2567399703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb69415e900 0x7fb6940adb90 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fb67c0097a0 tx=0x7fb67c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 -- 172.21.15.165:0/2567399703 shutdown_connections 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 --2- 172.21.15.165:0/2567399703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb69415e900 0x7fb6940adb90 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 -- 172.21.15.165:0/2567399703 >> 172.21.15.165:0/2567399703 conn(0x7fb69406abe0 msgr2=0x7fb69406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 -- 172.21.15.165:0/2567399703 shutdown_connections 2024-04-25T10:07:12.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.813+0000 7fb699a00640 1 -- 172.21.15.165:0/2567399703 wait complete. 2024-04-25T10:07:12.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.815+0000 7fb699a00640 1 Processor -- start 2024-04-25T10:07:12.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.815+0000 7fb699a00640 1 -- start start 2024-04-25T10:07:12.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.816+0000 7fb699a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 0x7fb694144900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.816+0000 7fb699a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb69415e520 0x7fb6940cc970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.816+0000 7fb699a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e900 0x7fb69415baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.816+0000 7fb699a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb694071700 con 0x7fb6940ae0d0 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.816+0000 7fb699a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69406d1f0 con 0x7fb69415e900 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.816+0000 7fb699a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb694071880 con 0x7fb69415e520 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 0x7fb694144900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 0x7fb694144900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47492/0 (socket says 172.21.15.165:47492) 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 -- 172.21.15.165:0/2599041704 learned_addr learned my addr 172.21.15.165:0/2599041704 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 -- 172.21.15.165:0/2599041704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb69415e520 msgr2=0x7fb6940cc970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb69415e520 0x7fb6940cc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 -- 172.21.15.165:0/2599041704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e900 msgr2=0x7fb69415baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e900 0x7fb69415baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 -- 172.21.15.165:0/2599041704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6940cceb0 con 0x7fb6940ae0d0 2024-04-25T10:07:12.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.817+0000 7fb693e00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 0x7fb694144900 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c430 tx=0x7fb68400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.818+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684017070 con 0x7fb6940ae0d0 2024-04-25T10:07:12.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.818+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb684005ce0 con 0x7fb6940ae0d0 2024-04-25T10:07:12.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.818+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6840072f0 con 0x7fb6940ae0d0 2024-04-25T10:07:12.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.818+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6940b1ae0 con 0x7fb6940ae0d0 2024-04-25T10:07:12.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.818+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6940b20a0 con 0x7fb6940ae0d0 2024-04-25T10:07:12.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.819+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb694073990 con 0x7fb6940ae0d0 2024-04-25T10:07:12.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.820+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb68401a070 con 0x7fb6940ae0d0 2024-04-25T10:07:12.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.821+0000 7fb690c00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb67006b4d0 0x7fb67006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:12.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.821+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb6840898b0 con 0x7fb6940ae0d0 2024-04-25T10:07:12.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.821+0000 7fb68b400640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb67006b4d0 0x7fb67006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:12.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.822+0000 7fb68b400640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb67006b4d0 0x7fb67006d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb67c034000 tx=0x7fb67c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:12.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:12.825+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb684058620 con 0x7fb6940ae0d0 2024-04-25T10:07:13.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.059+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6940cdd60 con 0x7fb6940ae0d0 2024-04-25T10:07:13.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.060+0000 7fb690c00640 1 -- 172.21.15.165:0/2599041704 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fb684057f20 con 0x7fb6940ae0d0 2024-04-25T10:07:13.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:13.066 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:07:11.631523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:13.066 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 32 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb67006b4d0 msgr2=0x7fb67006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb67006b4d0 0x7fb67006d990 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb67c034000 tx=0x7fb67c02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 msgr2=0x7fb694144900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 0x7fb694144900 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fb68400c430 tx=0x7fb68400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 shutdown_connections 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb67006b4d0 0x7fb67006d990 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb69415e900 0x7fb69415baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb69415e520 0x7fb6940cc970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 --2- 172.21.15.165:0/2599041704 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb6940ae0d0 0x7fb694144900 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 >> 172.21.15.165:0/2599041704 conn(0x7fb69406abe0 msgr2=0x7fb6940aff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 shutdown_connections 2024-04-25T10:07:13.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.063+0000 7fb699a00640 1 -- 172.21.15.165:0/2599041704 wait complete. 2024-04-25T10:07:13.089 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:13.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.280+0000 7fe913000640 1 Processor -- start 2024-04-25T10:07:13.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.280+0000 7fe913000640 1 -- start start 2024-04-25T10:07:13.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe913000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe90c164970 0x7fe90c166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe913000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c144b30 con 0x7fe90c0ce920 2024-04-25T10:07:13.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe913000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90c0712d0 con 0x7fe90c144750 2024-04-25T10:07:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe913000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90c0715b0 con 0x7fe90c164970 2024-04-25T10:07:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe90b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe90c0ce920 0x7fe90c0cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44876/0 (socket says 172.21.15.165:44876) 2024-04-25T10:07:13.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe90b400640 1 -- 172.21.15.165:0/3054174373 learned_addr learned my addr 172.21.15.165:0/3054174373 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.282+0000 7fe911000640 1 --2- 172.21.15.165:0/3054174373 >> v2:172.21.15.172:3300/0 conn(0x7fe90c164970 0x7fe90c166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3152470949 0 0) 0x7fe90c144b30 con 0x7fe90c0ce920 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8f0003520 con 0x7fe90c0ce920 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2458713158 0 0) 0x7fe8f0003520 con 0x7fe90c0ce920 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 >> v2:172.21.15.172:3300/0 conn(0x7fe90c164970 msgr2=0x7fe90c166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 --2- 172.21.15.165:0/3054174373 >> v2:172.21.15.172:3300/0 conn(0x7fe90c164970 0x7fe90c166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 >> v1:172.21.15.172:6789/0 conn(0x7fe90c144750 legacy=0x7fe90c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c071890 con 0x7fe90c0ce920 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe8f4001670 con 0x7fe90c0ce920 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 >> v1:172.21.15.165:6789/0 conn(0x7fe90c0ce920 legacy=0x7fe90c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8f00039b0 0x7fe8f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c0715b0 con 0x7fe8f00051c0 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f000c160 con 0x7fe8f00039b0 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.283+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f000c360 con 0x7fe8f0004800 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe911000640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe90b400640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe90be00640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8f00039b0 0x7fe8f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe911000640 1 -- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f0004800 msgr2=0x7fe8f0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe911000640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f0004800 0x7fe8f0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe911000640 1 -- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8f00039b0 msgr2=0x7fe8f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe911000640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8f00039b0 0x7fe8f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe911000640 1 -- 172.21.15.165:0/3054174373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f0003520 con 0x7fe8f00051c0 2024-04-25T10:07:13.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.284+0000 7fe90be00640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8f00039b0 0x7fe8f0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.285+0000 7fe911000640 1 -- 172.21.15.165:0/3054174373 reap_dead start 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.285+0000 7fe911000640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe904009ac0 tx=0x7fe8f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c615c12066bfeb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.285+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe904009690 con 0x7fe8f00051c0 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.285+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe904004020 con 0x7fe8f00051c0 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.285+0000 7fe90aa00640 1 -- 172.21.15.165:0/3054174373 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9040043d0 con 0x7fe8f00051c0 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 -- 172.21.15.165:0/3054174373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8f00051c0 msgr2=0x7fe8f000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7fe904009ac0 tx=0x7fe8f0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 -- 172.21.15.165:0/3054174373 shutdown_connections 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 --2- 172.21.15.165:0/3054174373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8f00051c0 0x7fe8f000b9c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 -- 172.21.15.165:0/3054174373 >> 172.21.15.165:0/3054174373 conn(0x7fe90c06abe0 msgr2=0x7fe90c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 -- 172.21.15.165:0/3054174373 shutdown_connections 2024-04-25T10:07:13.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.286+0000 7fe913000640 1 -- 172.21.15.165:0/3054174373 wait complete. 2024-04-25T10:07:13.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.288+0000 7fe913000640 1 Processor -- start 2024-04-25T10:07:13.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.288+0000 7fe913000640 1 -- start start 2024-04-25T10:07:13.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe913000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f00051c0 0x7fe90c138be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe913000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 0x7fe90c143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe913000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90c143660 0x7fe90c13b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe913000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c1455d0 con 0x7fe90c0b4220 2024-04-25T10:07:13.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe913000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c145450 con 0x7fe90c143660 2024-04-25T10:07:13.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe913000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90c15c000 con 0x7fe8f00051c0 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe90b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 0x7fe90c143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe90b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 0x7fe90c143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47504/0 (socket says 172.21.15.165:47504) 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.289+0000 7fe90b400640 1 -- 172.21.15.165:0/1849501694 learned_addr learned my addr 172.21.15.165:0/1849501694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90be00640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f00051c0 0x7fe90c138be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe911000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90c143660 0x7fe90c13b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90b400640 1 -- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f00051c0 msgr2=0x7fe90c138be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90b400640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f00051c0 0x7fe90c138be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90b400640 1 -- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90c143660 msgr2=0x7fe90c13b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90b400640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90c143660 0x7fe90c13b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90b400640 1 -- 172.21.15.165:0/1849501694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90c0b08b0 con 0x7fe90c0b4220 2024-04-25T10:07:13.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe90b400640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 0x7fe90c143120 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400d7a0 tx=0x7fe8f400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f4019070 con 0x7fe90c0b4220 2024-04-25T10:07:13.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.290+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f400de50 con 0x7fe90c0b4220 2024-04-25T10:07:13.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.291+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90c0b0b50 con 0x7fe90c0b4220 2024-04-25T10:07:13.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.291+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f40047e0 con 0x7fe90c0b4220 2024-04-25T10:07:13.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.291+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90c0b1110 con 0x7fe90c0b4220 2024-04-25T10:07:13.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.292+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90c0b12e0 con 0x7fe90c0b4220 2024-04-25T10:07:13.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.293+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8f401c070 con 0x7fe90c0b4220 2024-04-25T10:07:13.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.293+0000 7fe903e00640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e806b500 0x7fe8e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.294+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe8f4089db0 con 0x7fe90c0b4220 2024-04-25T10:07:13.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.294+0000 7fe90be00640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e806b500 0x7fe8e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.294+0000 7fe90be00640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e806b500 0x7fe8e806d9c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007fd0 tx=0x7fe8fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.297+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8f408e070 con 0x7fe90c0b4220 2024-04-25T10:07:13.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.530+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe90c0d0510 con 0x7fe90c0b4220 2024-04-25T10:07:13.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.531+0000 7fe903e00640 1 -- 172.21.15.165:0/1849501694 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+4948 (secure 0 0 0) 0x7fe8f40589d0 con 0x7fe90c0b4220 2024-04-25T10:07:13.538 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:13.538 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":32,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:06:22.994073+0000","modified":"2024-04-25T10:07:11.631523+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":5811},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5811":{"gid":5811,"name":"d","rank":0,"incarnation":25,"state":"up:active","state_seq":15,"addr":"172.21.15.172:6834/3236509347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3236509347},{"type":"v1","addr":"172.21.15.172:6834","nonce":3236509347}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2024-04-25T10:07:13.538 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 32 2024-04-25T10:07:13.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e806b500 msgr2=0x7fe8e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e806b500 0x7fe8e806d9c0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc007fd0 tx=0x7fe8fc009040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 msgr2=0x7fe90c143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 0x7fe90c143120 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fe8f400d7a0 tx=0x7fe8f400dc60 comp rx=0 tx=0).stop 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 shutdown_connections 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e806b500 0x7fe8e806d9c0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90c143660 0x7fe90c13b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90c0b4220 0x7fe90c143120 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 --2- 172.21.15.165:0/1849501694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8f00051c0 0x7fe90c138be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.534+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 >> 172.21.15.165:0/1849501694 conn(0x7fe90c06abe0 msgr2=0x7fe90c15b630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.535+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 shutdown_connections 2024-04-25T10:07:13.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.535+0000 7fe913000640 1 -- 172.21.15.165:0/1849501694 wait complete. 2024-04-25T10:07:13.560 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.763+0000 7f31ef400640 1 Processor -- start 2024-04-25T10:07:13.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.763+0000 7f31ef400640 1 -- start start 2024-04-25T10:07:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.764+0000 7f31ef400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f31e815db70 0x7f31e80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31ef400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e8071650 con 0x7f31e80cfc20 2024-04-25T10:07:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31ef400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31e80cf230 con 0x7f31e815df50 2024-04-25T10:07:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31ef400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31e80cbd80 con 0x7f31e815db70 2024-04-25T10:07:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31ed400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f31e80cfc20 0x7f31e806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44880/0 (socket says 172.21.15.165:44880) 2024-04-25T10:07:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31ed400640 1 -- 172.21.15.165:0/2648902558 learned_addr learned my addr 172.21.15.165:0/2648902558 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31eca00640 1 --2- 172.21.15.165:0/2648902558 >> v2:172.21.15.172:3300/0 conn(0x7f31e815db70 0x7f31e80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089033748 0 0) 0x7f31e0001490 con 0x7f31e80cfc20 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.765+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31d0003520 con 0x7f31e80cfc20 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1131617063 0 0) 0x7f31d0003520 con 0x7f31e80cfc20 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 >> v2:172.21.15.172:3300/0 conn(0x7f31e815db70 msgr2=0x7f31e80cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 --2- 172.21.15.165:0/2648902558 >> v2:172.21.15.172:3300/0 conn(0x7f31e815db70 0x7f31e80cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 >> v1:172.21.15.172:6789/0 conn(0x7f31e815df50 legacy=0x7f31e80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8161b60 con 0x7f31e80cfc20 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f31e0001bc0 con 0x7f31e80cfc20 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 >> v1:172.21.15.165:6789/0 conn(0x7f31e80cfc20 legacy=0x7f31e806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31d0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31d0005210 0x7f31d000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e80cbd80 con 0x7f31d0003a00 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31d000c160 con 0x7f31d0004800 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31d000c360 con 0x7f31d0005210 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31eca00640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31d0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.766+0000 7f31e7e00640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31ed400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31d0005210 0x7f31d000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31eca00640 1 -- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31d0005210 msgr2=0x7f31d000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31eca00640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31d0005210 0x7f31d000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31eca00640 1 -- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31d0004800 msgr2=0x7f31d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31eca00640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31d0004800 0x7f31d0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31eca00640 1 -- 172.21.15.165:0/2648902558 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31d0003520 con 0x7f31d0003a00 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31ed400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31d0005210 0x7f31d000b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.767+0000 7f31ed400640 1 -- 172.21.15.165:0/2648902558 reap_dead start 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.768+0000 7f31eca00640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31d0003e20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00ba30 tx=0x7f31d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cdc3846c8914ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.768+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc030070 con 0x7f31d0003a00 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.768+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc02ade0 con 0x7f31d0003a00 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.768+0000 7f31e7400640 1 -- 172.21.15.165:0/2648902558 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00b600 con 0x7f31d0003a00 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.768+0000 7f31ef400640 1 -- 172.21.15.165:0/2648902558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 msgr2=0x7f31d0003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.768+0000 7f31ef400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31d0003e20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00ba30 tx=0x7f31d0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.769+0000 7f31ef400640 1 -- 172.21.15.165:0/2648902558 shutdown_connections 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.769+0000 7f31ef400640 1 --2- 172.21.15.165:0/2648902558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31d0003e20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.769+0000 7f31ef400640 1 -- 172.21.15.165:0/2648902558 >> 172.21.15.165:0/2648902558 conn(0x7f31e806abe0 msgr2=0x7f31e806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.769+0000 7f31ef400640 1 -- 172.21.15.165:0/2648902558 shutdown_connections 2024-04-25T10:07:13.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.769+0000 7f31ef400640 1 -- 172.21.15.165:0/2648902558 wait complete. 2024-04-25T10:07:13.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.770+0000 7f31ef400640 1 Processor -- start 2024-04-25T10:07:13.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 -- start start 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31e8152110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e8147f80 0x7f31e815aed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31e815b410 0x7f31e81f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8162120 con 0x7f31d0003a00 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8161910 con 0x7f31e8147f80 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.771+0000 7f31ef400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31e8162640 con 0x7f31e815b410 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31e8152110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31e8152110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47522/0 (socket says 172.21.15.165:47522) 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 -- 172.21.15.165:0/4153563819 learned_addr learned my addr 172.21.15.165:0/4153563819 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 -- 172.21.15.165:0/4153563819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31e815b410 msgr2=0x7f31e81f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31e815b410 0x7f31e81f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 -- 172.21.15.165:0/4153563819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e8147f80 msgr2=0x7f31e815aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e8147f80 0x7f31e815aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 -- 172.21.15.165:0/4153563819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31e8151df0 con 0x7f31d0003a00 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.772+0000 7f31eca00640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31e8152110 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00bb50 tx=0x7f31dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.773+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc043030 con 0x7f31d0003a00 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.773+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31dc02abe0 con 0x7f31d0003a00 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.773+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31dc00b600 con 0x7f31d0003a00 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.773+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31e8159e30 con 0x7f31d0003a00 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.773+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31e815a3f0 con 0x7f31d0003a00 2024-04-25T10:07:13.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.774+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31e8073990 con 0x7f31d0003a00 2024-04-25T10:07:13.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.775+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f31dc046070 con 0x7f31d0003a00 2024-04-25T10:07:13.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.776+0000 7f31e4c00640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31c806b500 0x7f31c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:13.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.776+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f31dc0aae60 con 0x7f31d0003a00 2024-04-25T10:07:13.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.777+0000 7f31e7e00640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31c806b500 0x7f31c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:13.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.777+0000 7f31e7e00640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31c806b500 0x7f31c806d9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007fd0 tx=0x7f31d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:13.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.781+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f31dc0799f0 con 0x7f31d0003a00 2024-04-25T10:07:13.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.966+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f31e80b45e0 con 0x7f31d0003a00 2024-04-25T10:07:13.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.967+0000 7f31e4c00640 1 -- 172.21.15.165:0/4153563819 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v56) v1 ==== 74+0+17018 (secure 0 0 0) 0x7f31dc0792f0 con 0x7f31d0003a00 2024-04-25T10:07:13.973 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:13.974 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":56,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:06:46.336008+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:06:20.857210+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_data","create_time":"2024-04-25T10:06:22.010030+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"8.1d","mappings":[{"from":0,"to":6}]},{"pgid":"8.37","mappings":[{"from":0,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:13.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31c806b500 msgr2=0x7f31c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31c806b500 0x7f31c806d9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f31d4007fd0 tx=0x7f31d4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:13.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 msgr2=0x7f31e8152110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:13.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31e8152110 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f31dc00bb50 tx=0x7f31dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 shutdown_connections 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f31c806b500 0x7f31c806d9c0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f31e815b410 0x7f31e81f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.970+0000 7f31ef400640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f31e8147f80 0x7f31e815aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.971+0000 7f31ef400640 1 --2- 172.21.15.165:0/4153563819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f31d0003a00 0x7f31e8152110 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.971+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 >> 172.21.15.165:0/4153563819 conn(0x7f31e806abe0 msgr2=0x7f31e80aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.971+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 shutdown_connections 2024-04-25T10:07:13.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:13.971+0000 7f31ef400640 1 -- 172.21.15.165:0/4153563819 wait complete. 2024-04-25T10:07:13.995 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:07:14.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.167+0000 7f7f65200640 1 Processor -- start 2024-04-25T10:07:14.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.167+0000 7f7f65200640 1 -- start start 2024-04-25T10:07:14.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.169+0000 7f7f65200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f6006c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.169+0000 7f7f65200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7f600ce7b0 0x7f7f60161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.169+0000 7f7f65200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f600c6960 con 0x7f7f601616f0 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.169+0000 7f7f65200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f601614d0 con 0x7f7f600ce7b0 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.169+0000 7f7f65200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f60162260 con 0x7f7f601610f0 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5d400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7f601616f0 0x7f7f600ce0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44894/0 (socket says 172.21.15.165:44894) 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5d400640 1 -- 172.21.15.165:0/125386542 learned_addr learned my addr 172.21.15.165:0/125386542 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5e800640 1 --2- 172.21.15.165:0/125386542 >> v2:172.21.15.165:3300/0 conn(0x7f7f600ce7b0 0x7f7f60161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5de00640 1 --2- 172.21.15.165:0/125386542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f6006c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5e800640 1 -- 172.21.15.165:0/125386542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 msgr2=0x7f7f6006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5e800640 1 --2- 172.21.15.165:0/125386542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f6006c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5e800640 1 -- 172.21.15.165:0/125386542 >> v1:172.21.15.165:6789/0 conn(0x7f7f601616f0 legacy=0x7f7f600ce0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5e800640 1 -- 172.21.15.165:0/125386542 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f60162540 con 0x7f7f600ce7b0 2024-04-25T10:07:14.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.170+0000 7f7f5ca00640 1 -- 172.21.15.165:0/125386542 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3842928460 0 0) 0x7f7f600c6960 con 0x7f7f601616f0 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.171+0000 7f7f5e800640 1 --2- 172.21.15.165:0/125386542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f60161b50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009d90 tx=0x7f7f5802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eb4a3ead01a0aac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.171+0000 7f7f5ca00640 1 -- 172.21.15.165:0/125386542 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58007740 con 0x7f7f600ce7b0 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f5ca00640 1 -- 172.21.15.165:0/125386542 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f580078e0 con 0x7f7f600ce7b0 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f5ca00640 1 -- 172.21.15.165:0/125386542 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f58007cb0 con 0x7f7f600ce7b0 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 -- 172.21.15.165:0/125386542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 msgr2=0x7f7f60161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 --2- 172.21.15.165:0/125386542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f60161b50 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f7f58009d90 tx=0x7f7f5802a660 comp rx=0 tx=0).stop 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 -- 172.21.15.165:0/125386542 shutdown_connections 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 --2- 172.21.15.165:0/125386542 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f60161b50 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 --2- 172.21.15.165:0/125386542 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f6006c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 -- 172.21.15.165:0/125386542 >> 172.21.15.165:0/125386542 conn(0x7f7f6006f500 msgr2=0x7f7f6006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 -- 172.21.15.165:0/125386542 shutdown_connections 2024-04-25T10:07:14.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.172+0000 7f7f65200640 1 -- 172.21.15.165:0/125386542 wait complete. 2024-04-25T10:07:14.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.174+0000 7f7f65200640 1 Processor -- start 2024-04-25T10:07:14.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.174+0000 7f7f65200640 1 -- start start 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f65200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f601493c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f65200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f60158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f65200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f601616f0 0x7f7f601594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f65200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600cf690 con 0x7f7f600ce7b0 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f65200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f600cfce0 con 0x7f7f601616f0 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f65200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6006c700 con 0x7f7f601610f0 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f5de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f601493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f5de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f601493c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47552/0 (socket says 172.21.15.165:47552) 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.175+0000 7f7f5de00640 1 -- 172.21.15.165:0/3549280054 learned_addr learned my addr 172.21.15.165:0/3549280054 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f5de00640 1 -- 172.21.15.165:0/3549280054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 msgr2=0x7f7f60158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f5de00640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f60158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f5de00640 1 -- 172.21.15.165:0/3549280054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f601616f0 msgr2=0x7f7f601594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f5de00640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f601616f0 0x7f7f601594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f5de00640 1 -- 172.21.15.165:0/3549280054 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f601f4430 con 0x7f7f600ce7b0 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f5de00640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f601493c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000d800 tx=0x7f7f5000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000b6c0 con 0x7f7f600ce7b0 2024-04-25T10:07:14.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f50019070 con 0x7f7f600ce7b0 2024-04-25T10:07:14.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f5000ba80 con 0x7f7f600ce7b0 2024-04-25T10:07:14.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f601f4710 con 0x7f7f600ce7b0 2024-04-25T10:07:14.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.176+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f601f4cd0 con 0x7f7f600ce7b0 2024-04-25T10:07:14.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.178+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f60073a00 con 0x7f7f600ce7b0 2024-04-25T10:07:14.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.179+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f5000bc20 con 0x7f7f600ce7b0 2024-04-25T10:07:14.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.179+0000 7f7f56000640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f4006b4d0 0x7f7f4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.180+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7f5008a920 con 0x7f7f600ce7b0 2024-04-25T10:07:14.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.180+0000 7f7f5d400640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f4006b4d0 0x7f7f4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.180+0000 7f7f5d400640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f4006b4d0 0x7f7f4006d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c007fd0 tx=0x7f7f4c007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.184+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f50059540 con 0x7f7f600ce7b0 2024-04-25T10:07:14.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.416+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7f600cf150 con 0x7f7f600ce7b0 2024-04-25T10:07:14.422 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:07:14.417+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:14.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.421+0000 7f7f56000640 1 -- 172.21.15.165:0/3549280054 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v33) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7f50058e40 con 0x7f7f600ce7b0 2024-04-25T10:07:14.426 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:07:14.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f4006b4d0 msgr2=0x7f7f4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f4006b4d0 0x7f7f4006d990 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f7f4c007fd0 tx=0x7f7f4c007920 comp rx=0 tx=0).stop 2024-04-25T10:07:14.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 msgr2=0x7f7f601493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f601493c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f7f5000d800 tx=0x7f7f5000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 shutdown_connections 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f4006b4d0 0x7f7f4006d990 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f601616f0 0x7f7f601594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f601610f0 0x7f7f60158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 --2- 172.21.15.165:0/3549280054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f600ce7b0 0x7f7f601493c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 >> 172.21.15.165:0/3549280054 conn(0x7f7f6006f500 msgr2=0x7f7f60071830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 shutdown_connections 2024-04-25T10:07:14.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.424+0000 7f7f65200640 1 -- 172.21.15.165:0/3549280054 wait complete. 2024-04-25T10:07:14.447 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:07:14.529 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:07:14.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.637+0000 7f649dc00640 1 Processor -- start 2024-04-25T10:07:14.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.638+0000 7f649dc00640 1 -- start start 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.639+0000 7f649dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 0x7f64980b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.639+0000 7f649dc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f649806cf50 con 0x7f64980b5620 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.639+0000 7f649dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64980ce530 con 0x7f649815b7d0 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.639+0000 7f649dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980ce810 con 0x7f649806c9e0 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.639+0000 7f6496000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f64980b5620 0x7f649806c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44910/0 (socket says 172.21.15.165:44910) 2024-04-25T10:07:14.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.639+0000 7f6496000640 1 -- 172.21.15.165:0/3263026567 learned_addr learned my addr 172.21.15.165:0/3263026567 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6497400640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 0x7f64980b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2847823885 0 0) 0x7f649806cf50 con 0x7f64980b5620 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f647c003520 con 0x7f64980b5620 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2879415235 0 0) 0x7f647c003520 con 0x7f64980b5620 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 msgr2=0x7f64980b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 0x7f64980b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 >> v1:172.21.15.172:6789/0 conn(0x7f649815b7d0 legacy=0x7f64980cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64980ceaf0 con 0x7f64980b5620 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6488001a10 con 0x7f64980b5620 2024-04-25T10:07:14.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 >> v1:172.21.15.165:6789/0 conn(0x7f64980b5620 legacy=0x7f649806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f647c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 0x7f647c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f647c004d40 0x7f647c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64980ce810 con 0x7f647c0039e0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c00baa0 con 0x7f647c004d40 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.640+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c00bca0 con 0x7f649806c9e0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496a00640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f647c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496000640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 0x7f647c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6497400640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f647c004d40 0x7f647c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496a00640 1 -- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 msgr2=0x7f647c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496a00640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f649806c9e0 0x7f647c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496a00640 1 -- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f647c004d40 msgr2=0x7f647c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496a00640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f647c004d40 0x7f647c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.641+0000 7f6496a00640 1 -- 172.21.15.165:0/3263026567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c003520 con 0x7f647c0039e0 2024-04-25T10:07:14.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.642+0000 7f6496a00640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f647c003e00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6480001720 tx=0x7f647c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1009f360ba6b0d7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.642+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480001370 con 0x7f647c0039e0 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.642+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64800052d0 con 0x7f647c0039e0 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.642+0000 7f6495600640 1 -- 172.21.15.165:0/3263026567 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648002abc0 con 0x7f647c0039e0 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f649dc00640 1 -- 172.21.15.165:0/3263026567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 msgr2=0x7f647c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f649dc00640 1 --2- 172.21.15.165:0/3263026567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f647c003e00 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6480001720 tx=0x7f647c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f6497400640 1 -- 172.21.15.165:0/3263026567 reap_dead start 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f649dc00640 1 -- 172.21.15.165:0/3263026567 shutdown_connections 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f649dc00640 1 -- 172.21.15.165:0/3263026567 >> 172.21.15.165:0/3263026567 conn(0x7f64980d0800 msgr2=0x7f64980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f649dc00640 1 -- 172.21.15.165:0/3263026567 shutdown_connections 2024-04-25T10:07:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.643+0000 7f649dc00640 1 -- 172.21.15.165:0/3263026567 wait complete. 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.645+0000 7f649dc00640 1 Processor -- start 2024-04-25T10:07:14.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.645+0000 7f649dc00640 1 -- start start 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f649dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f64981f2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f649dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64981f6b30 0x7f649814c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f649dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64981605e0 0x7f6498160a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f649dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649815bd30 con 0x7f647c0039e0 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f649dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f649815bbb0 con 0x7f64981f6b30 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f649dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64980b5ab0 con 0x7f64981605e0 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f6496a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f64981f2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f6496a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f64981f2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47570/0 (socket says 172.21.15.165:47570) 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.646+0000 7f6496a00640 1 -- 172.21.15.165:0/2372545926 learned_addr learned my addr 172.21.15.165:0/2372545926 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:14.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6497400640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64981605e0 0x7f6498160a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496000640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64981f6b30 0x7f649814c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496a00640 1 -- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64981605e0 msgr2=0x7f6498160a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496a00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64981605e0 0x7f6498160a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496a00640 1 -- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64981f6b30 msgr2=0x7f649814c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496a00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64981f6b30 0x7f649814c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496a00640 1 -- 172.21.15.165:0/2372545926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f649814c790 con 0x7f647c0039e0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496000640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64981f6b30 0x7f649814c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f6496a00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f64981f2f40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6480001840 tx=0x7f6480001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480037040 con 0x7f647c0039e0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6480005480 con 0x7f647c0039e0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64980c7570 con 0x7f647c0039e0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480007a30 con 0x7f647c0039e0 2024-04-25T10:07:14.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.647+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64981f70a0 con 0x7f647c0039e0 2024-04-25T10:07:14.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.649+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6498073a00 con 0x7f647c0039e0 2024-04-25T10:07:14.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.650+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6480044020 con 0x7f647c0039e0 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.651+0000 7f648ea00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f647406b500 0x7f647406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.651+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f64800aa0d0 con 0x7f647c0039e0 2024-04-25T10:07:14.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.651+0000 7f6496000640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f647406b500 0x7f647406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:14.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.652+0000 7f6496000640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f647406b500 0x7f647406d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6488002230 tx=0x7f6488009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:14.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.655+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f649814c790 con 0x7f647c0039e0 2024-04-25T10:07:14.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:14.896+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f64980cda80 con 0x7f647c0039e0 2024-04-25T10:07:15.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.424+0000 7f648ea00640 1 -- 172.21.15.165:0/2372545926 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v34) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6480078c90 con 0x7f647c0039e0 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f647406b500 msgr2=0x7f647406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f647406b500 0x7f647406d9c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f6488002230 tx=0x7f6488009040 comp rx=0 tx=0).stop 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 msgr2=0x7f64981f2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f64981f2f40 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f6480001840 tx=0x7f6480001120 comp rx=0 tx=0).stop 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 shutdown_connections 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f647406b500 0x7f647406d9c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64981605e0 0x7f6498160a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64981f6b30 0x7f649814c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 --2- 172.21.15.165:0/2372545926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f647c0039e0 0x7f64981f2f40 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 >> 172.21.15.165:0/2372545926 conn(0x7f64980d0800 msgr2=0x7f649815f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 shutdown_connections 2024-04-25T10:07:15.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.429+0000 7f649dc00640 1 -- 172.21.15.165:0/2372545926 wait complete. 2024-04-25T10:07:15.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:07:15.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.662+0000 7fa5d3200640 1 Processor -- start 2024-04-25T10:07:15.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.663+0000 7fa5d3200640 1 -- start start 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa5d406bff0 0x7fa5d40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d3200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d415e110 con 0x7fa5d40b5970 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d3200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5d40b2fd0 con 0x7fa5d415dd30 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5d406d000 con 0x7fa5d406bff0 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d2800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa5d40b5970 0x7fa5d40d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44920/0 (socket says 172.21.15.165:44920) 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d2800640 1 -- 172.21.15.165:0/2739420542 learned_addr learned my addr 172.21.15.165:0/2739420542 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.664+0000 7fa5d1400640 1 --2- 172.21.15.165:0/2739420542 >> v2:172.21.15.172:3300/0 conn(0x7fa5d406bff0 0x7fa5d40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.665+0000 7fa5d0a00640 1 -- 172.21.15.165:0/2739420542 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3363322951 0 0) 0x7fa5d415e110 con 0x7fa5d40b5970 2024-04-25T10:07:15.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.665+0000 7fa5d0a00640 1 -- 172.21.15.165:0/2739420542 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5b4003520 con 0x7fa5d40b5970 2024-04-25T10:07:15.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.665+0000 7fa5d0a00640 1 -- 172.21.15.165:0/2739420542 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 116378050 0 0) 0x7fa5b4003520 con 0x7fa5d40b5970 2024-04-25T10:07:15.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.665+0000 7fa5d1400640 1 -- 172.21.15.165:0/2739420542 >> v1:172.21.15.172:6789/0 conn(0x7fa5d415dd30 legacy=0x7fa5d40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.665+0000 7fa5d1400640 1 -- 172.21.15.165:0/2739420542 >> v1:172.21.15.165:6789/0 conn(0x7fa5d40b5970 legacy=0x7fa5d40d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.665+0000 7fa5d1400640 1 -- 172.21.15.165:0/2739420542 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d406c480 con 0x7fa5d406bff0 2024-04-25T10:07:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.666+0000 7fa5d1400640 1 --2- 172.21.15.165:0/2739420542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d406bff0 0x7fa5d40b5430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009d90 tx=0x7fa5bc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ce6f00ae7996041 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.667+0000 7fa5d0a00640 1 -- 172.21.15.165:0/2739420542 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5cc001670 con 0x7fa5d406bff0 2024-04-25T10:07:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.667+0000 7fa5d0a00640 1 -- 172.21.15.165:0/2739420542 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5bc007540 con 0x7fa5d406bff0 2024-04-25T10:07:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.667+0000 7fa5d0a00640 1 -- 172.21.15.165:0/2739420542 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5bc007910 con 0x7fa5d406bff0 2024-04-25T10:07:15.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 -- 172.21.15.165:0/2739420542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d406bff0 msgr2=0x7fa5d40b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 --2- 172.21.15.165:0/2739420542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d406bff0 0x7fa5d40b5430 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc009d90 tx=0x7fa5bc02a680 comp rx=0 tx=0).stop 2024-04-25T10:07:15.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 -- 172.21.15.165:0/2739420542 shutdown_connections 2024-04-25T10:07:15.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 --2- 172.21.15.165:0/2739420542 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d406bff0 0x7fa5d40b5430 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 -- 172.21.15.165:0/2739420542 >> 172.21.15.165:0/2739420542 conn(0x7fa5d40d0610 msgr2=0x7fa5d40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:15.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 -- 172.21.15.165:0/2739420542 shutdown_connections 2024-04-25T10:07:15.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.668+0000 7fa5d3200640 1 -- 172.21.15.165:0/2739420542 wait complete. 2024-04-25T10:07:15.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.669+0000 7fa5d3200640 1 Processor -- start 2024-04-25T10:07:15.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.670+0000 7fa5d3200640 1 -- start start 2024-04-25T10:07:15.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 0x7fa5d4156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5d415dd30 0x7fa5d41f4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d415a5b0 0x7fa5d4158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406cd20 con 0x7fa5d40b5970 2024-04-25T10:07:15.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d406ca40 con 0x7fa5d415a5b0 2024-04-25T10:07:15.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5d40d2870 con 0x7fa5d415dd30 2024-04-25T10:07:15.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 0x7fa5d4156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 0x7fa5d4156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47572/0 (socket says 172.21.15.165:47572) 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 -- 172.21.15.165:0/4196062814 learned_addr learned my addr 172.21.15.165:0/4196062814 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1400640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5d415dd30 0x7fa5d41f4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 -- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5d415dd30 msgr2=0x7fa5d41f4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5d415dd30 0x7fa5d41f4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 -- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d415a5b0 msgr2=0x7fa5d4158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d2800640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d415a5b0 0x7fa5d4158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d415a5b0 0x7fa5d4158410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.671+0000 7fa5d1e00640 1 -- 172.21.15.165:0/4196062814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5d41f4990 con 0x7fa5d40b5970 2024-04-25T10:07:15.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.672+0000 7fa5d1e00640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 0x7fa5d4156e80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c430 tx=0x7fa5c400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.672+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4017070 con 0x7fa5d40b5970 2024-04-25T10:07:15.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.672+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5c40052d0 con 0x7fa5d40b5970 2024-04-25T10:07:15.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.672+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5c4003830 con 0x7fa5d40b5970 2024-04-25T10:07:15.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.672+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5d41f4c70 con 0x7fa5d40b5970 2024-04-25T10:07:15.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.672+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5d41f5230 con 0x7fa5d40b5970 2024-04-25T10:07:15.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.673+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5d4073990 con 0x7fa5d40b5970 2024-04-25T10:07:15.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.674+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa5c401a070 con 0x7fa5d40b5970 2024-04-25T10:07:15.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.675+0000 7fa5ca000640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5b006b4d0 0x7fa5b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:15.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.675+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6245+0+0 (secure 0 0 0) 0x7fa5c40897f0 con 0x7fa5d40b5970 2024-04-25T10:07:15.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.676+0000 7fa5d1400640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5b006b4d0 0x7fa5b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:15.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.676+0000 7fa5d1400640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5b006b4d0 0x7fa5b006d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc034000 tx=0x7fa5bc02f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:15.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.680+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa5c40583b0 con 0x7fa5d40b5970 2024-04-25T10:07:15.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:15.879+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa5d40d16c0 con 0x7fa5d40b5970 2024-04-25T10:07:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.437+0000 7fa5ca000640 1 -- 172.21.15.165:0/4196062814 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v58) v1 ==== 181+0+0 (secure 0 0 0) 0x7fa5c4057cb0 con 0x7fa5d40b5970 2024-04-25T10:07:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.441+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5b006b4d0 msgr2=0x7fa5b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.441+0000 7fa5d3200640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5b006b4d0 0x7fa5b006d990 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa5bc034000 tx=0x7fa5bc02f000 comp rx=0 tx=0).stop 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.441+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 msgr2=0x7fa5d4156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.441+0000 7fa5d3200640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 0x7fa5d4156e80 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fa5c400c430 tx=0x7fa5c400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 shutdown_connections 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5b006b4d0 0x7fa5b006d990 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa5d415a5b0 0x7fa5d4158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa5d415dd30 0x7fa5d41f4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 --2- 172.21.15.165:0/4196062814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa5d40b5970 0x7fa5d4156e80 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 >> 172.21.15.165:0/4196062814 conn(0x7fa5d40d0610 msgr2=0x7fa5d40b45e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 shutdown_connections 2024-04-25T10:07:16.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.443+0000 7fa5d3200640 1 -- 172.21.15.165:0/4196062814 wait complete. 2024-04-25T10:07:16.471 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:07:16.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.652+0000 7f3f20600640 1 Processor -- start 2024-04-25T10:07:16.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.652+0000 7f3f20600640 1 -- start start 2024-04-25T10:07:16.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.653+0000 7f3f20600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f1815b710 0x7f3f180cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f20600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f1806ec50 0x7f3f1815e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f20600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815ee60 0x7f3f180ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f20600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1815e490 con 0x7f3f1806ec50 2024-04-25T10:07:16.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f20600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1806f0e0 con 0x7f3f1815b710 2024-04-25T10:07:16.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f20600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180cfd80 con 0x7f3f1815ee60 2024-04-25T10:07:16.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f1806ec50 0x7f3f1815e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f1806ec50 0x7f3f1815e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47588/0 (socket says 172.21.15.165:47588) 2024-04-25T10:07:16.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 -- 172.21.15.165:0/227252648 learned_addr learned my addr 172.21.15.165:0/227252648 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1e600640 1 --2- 172.21.15.165:0/227252648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815ee60 0x7f3f180ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 -- 172.21.15.165:0/227252648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815ee60 msgr2=0x7f3f180ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 --2- 172.21.15.165:0/227252648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815ee60 0x7f3f180ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1dc00640 1 --2- 172.21.15.165:0/227252648 >> v2:172.21.15.172:3300/0 conn(0x7f3f1815b710 0x7f3f180cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 -- 172.21.15.165:0/227252648 >> v2:172.21.15.172:3300/0 conn(0x7f3f1815b710 msgr2=0x7f3f180cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 --2- 172.21.15.165:0/227252648 >> v2:172.21.15.172:3300/0 conn(0x7f3f1815b710 0x7f3f180cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.654+0000 7f3f1d200640 1 -- 172.21.15.165:0/227252648 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f180d0060 con 0x7f3f1806ec50 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.655+0000 7f3f1dc00640 1 --2- 172.21.15.165:0/227252648 >> v2:172.21.15.172:3300/0 conn(0x7f3f1815b710 0x7f3f180cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:16.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.656+0000 7f3f1d200640 1 --2- 172.21.15.165:0/227252648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f1815e920 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3f1806b950 tx=0x7f3f0402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674af9ac3970e1ea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.656+0000 7f3f17e00640 1 -- 172.21.15.165:0/227252648 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04037070 con 0x7f3f1806ec50 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.656+0000 7f3f17e00640 1 -- 172.21.15.165:0/227252648 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f04032440 con 0x7f3f1806ec50 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.656+0000 7f3f17e00640 1 -- 172.21.15.165:0/227252648 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04032810 con 0x7f3f1806ec50 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.656+0000 7f3f20600640 1 -- 172.21.15.165:0/227252648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 msgr2=0x7f3f1815e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 --2- 172.21.15.165:0/227252648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f1815e920 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f3f1806b950 tx=0x7f3f0402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 -- 172.21.15.165:0/227252648 shutdown_connections 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 --2- 172.21.15.165:0/227252648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815ee60 0x7f3f180ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 --2- 172.21.15.165:0/227252648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f1815e920 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 --2- 172.21.15.165:0/227252648 >> v2:172.21.15.172:3300/0 conn(0x7f3f1815b710 0x7f3f180cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 -- 172.21.15.165:0/227252648 >> 172.21.15.165:0/227252648 conn(0x7f3f1806abe0 msgr2=0x7f3f1806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 -- 172.21.15.165:0/227252648 shutdown_connections 2024-04-25T10:07:16.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.657+0000 7f3f20600640 1 -- 172.21.15.165:0/227252648 wait complete. 2024-04-25T10:07:16.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.658+0000 7f3f20600640 1 Processor -- start 2024-04-25T10:07:16.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.659+0000 7f3f20600640 1 -- start start 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f20600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f181f0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f20600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815b710 0x7f3f180af3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f20600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1815ee60 0x7f3f180af8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f20600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815bc70 con 0x7f3f1806ec50 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f20600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1815baf0 con 0x7f3f1815ee60 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f20600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f180acdf0 con 0x7f3f1815b710 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f181f0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f181f0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47596/0 (socket says 172.21.15.165:47596) 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 -- 172.21.15.165:0/2147603769 learned_addr learned my addr 172.21.15.165:0/2147603769 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:16.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 -- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815b710 msgr2=0x7f3f180af3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1d200640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815b710 0x7f3f180af3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1e600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1815ee60 0x7f3f180af8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815b710 0x7f3f180af3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 -- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1815ee60 msgr2=0x7f3f180af8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1815ee60 0x7f3f180af8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.660+0000 7f3f1dc00640 1 -- 172.21.15.165:0/2147603769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f18160120 con 0x7f3f1806ec50 2024-04-25T10:07:16.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.661+0000 7f3f1dc00640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f181f0c00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00c7c0 tx=0x7f3f0c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.661+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c015070 con 0x7f3f1806ec50 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.661+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c007a30 con 0x7f3f1806ec50 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.661+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f18160400 con 0x7f3f1806ec50 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.661+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c007e00 con 0x7f3f1806ec50 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.661+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f181609c0 con 0x7f3f1806ec50 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.662+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f18073a00 con 0x7f3f1806ec50 2024-04-25T10:07:16.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.663+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f0c018070 con 0x7f3f1806ec50 2024-04-25T10:07:16.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.665+0000 7f3f15600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ef806b4d0 0x7f3ef806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:16.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.665+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f3f0c08a270 con 0x7f3f1806ec50 2024-04-25T10:07:16.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.665+0000 7f3f1d200640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ef806b4d0 0x7f3ef806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:16.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.666+0000 7f3f1d200640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ef806b4d0 0x7f3ef806d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f04034000 tx=0x7f3f0402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:16.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.669+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f0c058ff0 con 0x7f3f1806ec50 2024-04-25T10:07:16.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:16.865+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f180cd610 con 0x7f3f1806ec50 2024-04-25T10:07:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.441+0000 7f3f15600640 1 -- 172.21.15.165:0/2147603769 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v59) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3f0c0588f0 con 0x7f3f1806ec50 2024-04-25T10:07:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:07:17.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.446+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ef806b4d0 msgr2=0x7f3ef806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.446+0000 7f3f20600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ef806b4d0 0x7f3ef806d990 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3f04034000 tx=0x7f3f0402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.446+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 msgr2=0x7f3f181f0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.446+0000 7f3f20600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f181f0c00 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c00c7c0 tx=0x7f3f0c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.447+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 shutdown_connections 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.447+0000 7f3f20600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ef806b4d0 0x7f3ef806d990 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.447+0000 7f3f20600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f1815ee60 0x7f3f180af8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.447+0000 7f3f20600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f1815b710 0x7f3f180af3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.447+0000 7f3f20600640 1 --2- 172.21.15.165:0/2147603769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f1806ec50 0x7f3f181f0c00 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.447+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 >> 172.21.15.165:0/2147603769 conn(0x7f3f1806abe0 msgr2=0x7f3f1815a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.448+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 shutdown_connections 2024-04-25T10:07:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.449+0000 7f3f20600640 1 -- 172.21.15.165:0/2147603769 wait complete. 2024-04-25T10:07:17.484 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:07:17.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.671+0000 7fd6ef200640 1 Processor -- start 2024-04-25T10:07:17.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.671+0000 7fd6ef200640 1 -- start start 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.672+0000 7fd6ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 0x7fd6f00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.672+0000 7fd6ef200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6f00ce950 0x7fd6f00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.672+0000 7fd6ef200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f0145520 con 0x7fd6f00c5030 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.672+0000 7fd6ef200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f00ced90 con 0x7fd6f00ce950 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.672+0000 7fd6ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0071270 con 0x7fd6f0145140 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6ee800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd6f00c5030 0x7fd6f006d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42176/0 (socket says 172.21.15.165:42176) 2024-04-25T10:07:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6ee800640 1 -- 172.21.15.165:0/1592675174 learned_addr learned my addr 172.21.15.165:0/1592675174 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:17.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6ede00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 0x7fd6f00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6ed400640 1 --2- 172.21.15.165:0/1592675174 >> v2:172.21.15.172:3300/0 conn(0x7fd6f00ce950 0x7fd6f00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1316749015 0 0) 0x7fd6f0145520 con 0x7fd6f00c5030 2024-04-25T10:07:17.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d4003520 con 0x7fd6f00c5030 2024-04-25T10:07:17.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.673+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 26474904 0 0) 0x7fd6d4003520 con 0x7fd6f00c5030 2024-04-25T10:07:17.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 msgr2=0x7fd6f00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 0x7fd6f00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 >> v2:172.21.15.172:3300/0 conn(0x7fd6f00ce950 msgr2=0x7fd6f00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 --2- 172.21.15.165:0/1592675174 >> v2:172.21.15.172:3300/0 conn(0x7fd6f00ce950 0x7fd6f00c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0071550 con 0x7fd6f00c5030 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd6e80019f0 con 0x7fd6f00c5030 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 >> v1:172.21.15.165:6789/0 conn(0x7fd6f00c5030 legacy=0x7fd6f006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40039b0 0x7fd6d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4004800 0x7fd6d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 0x7fd6d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00ced90 con 0x7fd6d40039b0 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0071270 con 0x7fd6d4004800 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d400bb20 con 0x7fd6f0145140 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ed400640 1 --2- 172.21.15.165:0/1592675174 >> v2:172.21.15.172:3300/0 conn(0x7fd6f00ce950 0x7fd6f00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ede00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40039b0 0x7fd6d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ede00640 1 -- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 msgr2=0x7fd6d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ede00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f0145140 0x7fd6d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ede00640 1 -- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4004800 msgr2=0x7fd6d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ede00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4004800 0x7fd6d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.674+0000 7fd6ede00640 1 -- 172.21.15.165:0/1592675174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4003520 con 0x7fd6d40039b0 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.675+0000 7fd6ede00640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40039b0 0x7fd6d4003dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000ba00 tx=0x7fd6d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f4aeb0ec1f1b59f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0030070 con 0x7fd6d40039b0 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e002adb0 con 0x7fd6d40039b0 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6eca00640 1 -- 172.21.15.165:0/1592675174 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e000b600 con 0x7fd6d40039b0 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ef200640 1 -- 172.21.15.165:0/1592675174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40039b0 msgr2=0x7fd6d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ef200640 1 --2- 172.21.15.165:0/1592675174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40039b0 0x7fd6d4003dd0 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd6e000ba00 tx=0x7fd6d4005830 comp rx=0 tx=0).stop 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ee800640 1 -- 172.21.15.165:0/1592675174 reap_dead start 2024-04-25T10:07:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ef200640 1 -- 172.21.15.165:0/1592675174 shutdown_connections 2024-04-25T10:07:17.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ef200640 1 -- 172.21.15.165:0/1592675174 >> 172.21.15.165:0/1592675174 conn(0x7fd6f0061cc0 msgr2=0x7fd6f006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ef200640 1 -- 172.21.15.165:0/1592675174 shutdown_connections 2024-04-25T10:07:17.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.676+0000 7fd6ef200640 1 -- 172.21.15.165:0/1592675174 wait complete. 2024-04-25T10:07:17.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.678+0000 7fd6ef200640 1 Processor -- start 2024-04-25T10:07:17.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.678+0000 7fd6ef200640 1 -- start start 2024-04-25T10:07:17.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.679+0000 7fd6ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d40039b0 0x7fd6f00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.679+0000 7fd6ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f01412c0 0x7fd6f01f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.679+0000 7fd6ef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 0x7fd6f01f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.679+0000 7fd6ef200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00c54a0 con 0x7fd6f00cd150 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.679+0000 7fd6ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f0071b10 con 0x7fd6d40039b0 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.679+0000 7fd6ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00c5620 con 0x7fd6f01412c0 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 0x7fd6f01f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 0x7fd6f01f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41418/0 (socket says 172.21.15.165:41418) 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 -- 172.21.15.165:0/3057643270 learned_addr learned my addr 172.21.15.165:0/3057643270 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ede00640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d40039b0 0x7fd6f00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ed400640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f01412c0 0x7fd6f01f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 -- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f01412c0 msgr2=0x7fd6f01f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f01412c0 0x7fd6f01f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 -- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d40039b0 msgr2=0x7fd6f00ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d40039b0 0x7fd6f00ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 -- 172.21.15.165:0/3057643270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f0133c50 con 0x7fd6f00cd150 2024-04-25T10:07:17.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.680+0000 7fd6ee800640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 0x7fd6f01f6f90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4004780 tx=0x7fd6e800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.681+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e8004d60 con 0x7fd6f00cd150 2024-04-25T10:07:17.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.681+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e800a640 con 0x7fd6f00cd150 2024-04-25T10:07:17.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.681+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f00cd5f0 con 0x7fd6f00cd150 2024-04-25T10:07:17.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.681+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e800aa10 con 0x7fd6f00cd150 2024-04-25T10:07:17.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.681+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f0133ef0 con 0x7fd6f00cd150 2024-04-25T10:07:17.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.682+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f0073990 con 0x7fd6f00cd150 2024-04-25T10:07:17.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.683+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6e800abb0 con 0x7fd6f00cd150 2024-04-25T10:07:17.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.684+0000 7fd6e6000640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6d006b4d0 0x7fd6d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:17.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.684+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd6e8089a00 con 0x7fd6f00cd150 2024-04-25T10:07:17.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.684+0000 7fd6ede00640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6d006b4d0 0x7fd6d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:17.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.685+0000 7fd6ede00640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6d006b4d0 0x7fd6d006d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40057e0 tx=0x7fd6e0030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.688+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6e80589e0 con 0x7fd6f00cd150 2024-04-25T10:07:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.882+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd6f0141800 con 0x7fd6f00cd150 2024-04-25T10:07:17.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.890+0000 7fd6e6000640 1 -- 172.21.15.165:0/3057643270 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd6e80582e0 con 0x7fd6f00cd150 2024-04-25T10:07:17.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6d006b4d0 msgr2=0x7fd6d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6d006b4d0 0x7fd6d006d990 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40057e0 tx=0x7fd6e0030000 comp rx=0 tx=0).stop 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 msgr2=0x7fd6f01f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 0x7fd6f01f6f90 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4004780 tx=0x7fd6e800cbe0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 shutdown_connections 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6d006b4d0 0x7fd6d006d990 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6f00cd150 0x7fd6f01f6f90 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6f01412c0 0x7fd6f01f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 --2- 172.21.15.165:0/3057643270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d40039b0 0x7fd6f00ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.892+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 >> 172.21.15.165:0/3057643270 conn(0x7fd6f0061cc0 msgr2=0x7fd6f015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.893+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 shutdown_connections 2024-04-25T10:07:17.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:17.893+0000 7fd6ef200640 1 -- 172.21.15.165:0/3057643270 wait complete. 2024-04-25T10:07:17.918 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks' 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.112+0000 7fa615800640 1 Processor -- start 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.112+0000 7fa615800640 1 -- start start 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa615800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015e520 0x7fa61015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa615800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa610070ff0 0x7fa610071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa615800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa610071950 con 0x7fa61015f040 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa615800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6100aee30 con 0x7fa610070ff0 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa615800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100ae590 con 0x7fa61015e520 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa614e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa610070ff0 0x7fa610071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa614e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa610070ff0 0x7fa610071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41428/0 (socket says 172.21.15.165:41428) 2024-04-25T10:07:18.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.114+0000 7fa614e00640 1 -- 172.21.15.165:0/885582495 learned_addr learned my addr 172.21.15.165:0/885582495 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.115+0000 7fa60fe00640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015e520 0x7fa61015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.115+0000 7fa614e00640 1 -- 172.21.15.165:0/885582495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015e520 msgr2=0x7fa61015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.115+0000 7fa614e00640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015e520 0x7fa61015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.115+0000 7fa614e00640 1 -- 172.21.15.165:0/885582495 >> v1:172.21.15.172:6789/0 conn(0x7fa61015f040 legacy=0x7fa6100adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.115+0000 7fa614e00640 1 -- 172.21.15.165:0/885582495 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100ae870 con 0x7fa610070ff0 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.115+0000 7fa60fe00640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015e520 0x7fa61015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.116+0000 7fa614e00640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa610071410 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa608009e50 tx=0x7fa60802a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6712dde9f304959d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.116+0000 7fa60f400640 1 -- 172.21.15.165:0/885582495 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa608036070 con 0x7fa610070ff0 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.116+0000 7fa60f400640 1 -- 172.21.15.165:0/885582495 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa608031600 con 0x7fa610070ff0 2024-04-25T10:07:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.116+0000 7fa60f400640 1 -- 172.21.15.165:0/885582495 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6080319d0 con 0x7fa610070ff0 2024-04-25T10:07:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 -- 172.21.15.165:0/885582495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 msgr2=0x7fa610071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa610071410 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fa608009e50 tx=0x7fa60802a6d0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 -- 172.21.15.165:0/885582495 shutdown_connections 2024-04-25T10:07:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa610071410 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 --2- 172.21.15.165:0/885582495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015e520 0x7fa61015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 -- 172.21.15.165:0/885582495 >> 172.21.15.165:0/885582495 conn(0x7fa61006abe0 msgr2=0x7fa61006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 -- 172.21.15.165:0/885582495 shutdown_connections 2024-04-25T10:07:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.117+0000 7fa615800640 1 -- 172.21.15.165:0/885582495 wait complete. 2024-04-25T10:07:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.119+0000 7fa615800640 1 Processor -- start 2024-04-25T10:07:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.119+0000 7fa615800640 1 -- start start 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa615800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa6100b1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa615800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa61015e520 0x7fa61015a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa615800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015f040 0x7fa61015ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa615800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6100ae2a0 con 0x7fa610070ff0 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa615800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61006d1f0 con 0x7fa61015e520 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa615800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61015f4a0 con 0x7fa61015f040 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa6100b1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa60fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa6100b1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41436/0 (socket says 172.21.15.165:41436) 2024-04-25T10:07:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.120+0000 7fa60fe00640 1 -- 172.21.15.165:0/3670373998 learned_addr learned my addr 172.21.15.165:0/3670373998 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa614e00640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015f040 0x7fa61015ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa607400640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa61015e520 0x7fa61015a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa60fe00640 1 -- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015f040 msgr2=0x7fa61015ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa60fe00640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015f040 0x7fa61015ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa60fe00640 1 -- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa61015e520 msgr2=0x7fa61015a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa60fe00640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa61015e520 0x7fa61015a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa60fe00640 1 -- 172.21.15.165:0/3670373998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100b3f50 con 0x7fa610070ff0 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa607400640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa61015e520 0x7fa61015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.121+0000 7fa60fe00640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa6100b1980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c7f0 tx=0x7fa60000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.122+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600015070 con 0x7fa610070ff0 2024-04-25T10:07:18.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.122+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa600007a30 con 0x7fa610070ff0 2024-04-25T10:07:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.122+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa600007e00 con 0x7fa610070ff0 2024-04-25T10:07:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.122+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6100b4230 con 0x7fa610070ff0 2024-04-25T10:07:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.122+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa610130a50 con 0x7fa610070ff0 2024-04-25T10:07:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.123+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa610073990 con 0x7fa610070ff0 2024-04-25T10:07:18.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.124+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa600018070 con 0x7fa610070ff0 2024-04-25T10:07:18.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.125+0000 7fa60cc00640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5ec06b4d0 0x7fa5ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.125+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fa60008a0e0 con 0x7fa610070ff0 2024-04-25T10:07:18.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.126+0000 7fa607400640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5ec06b4d0 0x7fa5ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.126+0000 7fa607400640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5ec06b4d0 0x7fa5ec06d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa61015ea10 tx=0x7fa5f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.131+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa600059030 con 0x7fa610070ff0 2024-04-25T10:07:18.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.379+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]} v 0) v1 -- 0x7fa61015dc60 con 0x7fa610070ff0 2024-04-25T10:07:18.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.454+0000 7fa60cc00640 1 -- 172.21.15.165:0/3670373998 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_checks"]}]=0 v267) v1 ==== 143+0+0 (secure 0 0 0) 0x7fa600058930 con 0x7fa610070ff0 2024-04-25T10:07:18.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.457+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5ec06b4d0 msgr2=0x7fa5ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.457+0000 7fa615800640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5ec06b4d0 0x7fa5ec06d990 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa61015ea10 tx=0x7fa5f8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:18.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 msgr2=0x7fa6100b1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa6100b1980 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fa60000c7f0 tx=0x7fa60000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 shutdown_connections 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa5ec06b4d0 0x7fa5ec06d990 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa61015f040 0x7fa61015ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa61015e520 0x7fa61015a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 --2- 172.21.15.165:0/3670373998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa610070ff0 0x7fa6100b1980 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 >> 172.21.15.165:0/3670373998 conn(0x7fa61006abe0 msgr2=0x7fa6100afa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 shutdown_connections 2024-04-25T10:07:18.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.458+0000 7fa615800640 1 -- 172.21.15.165:0/3670373998 wait complete. 2024-04-25T10:07:18.482 INFO:tasks.cephfs_test_runner:test_scrub_checks (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:07:18.482 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:07:18.482 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.671+0000 7f74da000640 1 Processor -- start 2024-04-25T10:07:18.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.671+0000 7f74da000640 1 -- start start 2024-04-25T10:07:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74da000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f74d4154ee0 0x7f74d40cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74da000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f74d40ce590 0x7f74d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74da000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d4071480 con 0x7f74d40cfcf0 2024-04-25T10:07:18.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74da000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d4072090 con 0x7f74d40ce590 2024-04-25T10:07:18.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74da000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d4071230 con 0x7f74d4154ee0 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74d3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f74d40ce590 0x7f74d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74d3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f74d40ce590 0x7f74d406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41450/0 (socket says 172.21.15.165:41450) 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.673+0000 7f74d3e00640 1 -- 172.21.15.165:0/92195763 learned_addr learned my addr 172.21.15.165:0/92195763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.674+0000 7f74d3400640 1 --2- 172.21.15.165:0/92195763 >> v2:172.21.15.172:3300/0 conn(0x7f74d4154ee0 0x7f74d40cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.674+0000 7f74d3e00640 1 -- 172.21.15.165:0/92195763 >> v2:172.21.15.172:3300/0 conn(0x7f74d4154ee0 msgr2=0x7f74d40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.674+0000 7f74d3e00640 1 --2- 172.21.15.165:0/92195763 >> v2:172.21.15.172:3300/0 conn(0x7f74d4154ee0 0x7f74d40cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.674+0000 7f74d3e00640 1 -- 172.21.15.165:0/92195763 >> v1:172.21.15.172:6789/0 conn(0x7f74d40cfcf0 legacy=0x7f74d40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:18.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.674+0000 7f74d3e00640 1 -- 172.21.15.165:0/92195763 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4071760 con 0x7f74d40ce590 2024-04-25T10:07:18.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.675+0000 7f74d3e00640 1 --2- 172.21.15.165:0/92195763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d40ce590 0x7f74d406d6f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f74d40d02e0 tx=0x7f74cc02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9b1a0e6b2b1d581 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.675+0000 7f74d2a00640 1 -- 172.21.15.165:0/92195763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0325a0 con 0x7f74d40ce590 2024-04-25T10:07:18.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.675+0000 7f74d2a00640 1 -- 172.21.15.165:0/92195763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc038070 con 0x7f74d40ce590 2024-04-25T10:07:18.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.675+0000 7f74d2a00640 1 -- 172.21.15.165:0/92195763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc032960 con 0x7f74d40ce590 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 -- 172.21.15.165:0/92195763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d40ce590 msgr2=0x7f74d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 --2- 172.21.15.165:0/92195763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d40ce590 0x7f74d406d6f0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f74d40d02e0 tx=0x7f74cc02c8e0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 -- 172.21.15.165:0/92195763 shutdown_connections 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 --2- 172.21.15.165:0/92195763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d40ce590 0x7f74d406d6f0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 --2- 172.21.15.165:0/92195763 >> v2:172.21.15.172:3300/0 conn(0x7f74d4154ee0 0x7f74d40cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 -- 172.21.15.165:0/92195763 >> 172.21.15.165:0/92195763 conn(0x7f74d406abe0 msgr2=0x7f74d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 -- 172.21.15.165:0/92195763 shutdown_connections 2024-04-25T10:07:18.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.676+0000 7f74da000640 1 -- 172.21.15.165:0/92195763 wait complete. 2024-04-25T10:07:18.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.678+0000 7f74da000640 1 Processor -- start 2024-04-25T10:07:18.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.678+0000 7f74da000640 1 -- start start 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74da000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74d40ce590 0x7f74d41f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74da000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74d40cfcf0 0x7f74d4159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74da000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 0x7f74d4159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74da000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d41552c0 con 0x7f74d4154ee0 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74da000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4071d20 con 0x7f74d40ce590 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74da000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4155440 con 0x7f74d40cfcf0 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 0x7f74d4159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 0x7f74d4159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41456/0 (socket says 172.21.15.165:41456) 2024-04-25T10:07:18.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.679+0000 7f74d3e00640 1 -- 172.21.15.165:0/1445256487 learned_addr learned my addr 172.21.15.165:0/1445256487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3400640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74d40ce590 0x7f74d41f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3e00640 1 -- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74d40cfcf0 msgr2=0x7f74d4159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74caa00640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74d40cfcf0 0x7f74d4159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3e00640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74d40cfcf0 0x7f74d4159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3e00640 1 -- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74d40ce590 msgr2=0x7f74d41f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3e00640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74d40ce590 0x7f74d41f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3e00640 1 -- 172.21.15.165:0/1445256487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d4150eb0 con 0x7f74d4154ee0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.680+0000 7f74d3e00640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 0x7f74d4159f40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00b2c0 tx=0x7f74cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.681+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc043070 con 0x7f74d4154ee0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.681+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc033c90 con 0x7f74d4154ee0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.681+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d4151190 con 0x7f74d4154ee0 2024-04-25T10:07:18.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.681+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc03c660 con 0x7f74d4154ee0 2024-04-25T10:07:18.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.681+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d4151750 con 0x7f74d4154ee0 2024-04-25T10:07:18.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.682+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d4073990 con 0x7f74d4154ee0 2024-04-25T10:07:18.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.683+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74cc046070 con 0x7f74d4154ee0 2024-04-25T10:07:18.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.684+0000 7f74cbe00640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f74b006b4d0 0x7f74b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:18.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.685+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f74cc030020 con 0x7f74d4154ee0 2024-04-25T10:07:18.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.685+0000 7f74d3400640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f74b006b4d0 0x7f74b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:18.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.686+0000 7f74d3400640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f74b006b4d0 0x7f74b006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f74c4008e50 tx=0x7f74c40083a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:18.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.689+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74cc04d070 con 0x7f74d4154ee0 2024-04-25T10:07:18.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.889+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74d406b160 con 0x7f74d4154ee0 2024-04-25T10:07:18.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.895+0000 7f74cbe00640 1 -- 172.21.15.165:0/1445256487 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13331 (secure 0 0 0) 0x7f74cc0aae20 con 0x7f74d4154ee0 2024-04-25T10:07:18.901 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:18.901 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":59,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:17.437851+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6834/3236509347":"2024-04-26T10:07:14.417822+0000","172.21.15.172:6832/3236509347":"2024-04-26T10:07:14.417822+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:18.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f74b006b4d0 msgr2=0x7f74b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f74b006b4d0 0x7f74b006d990 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f74c4008e50 tx=0x7f74c40083a0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 msgr2=0x7f74d4159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 0x7f74d4159f40 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f74cc00b2c0 tx=0x7f74cc004730 comp rx=0 tx=0).stop 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 shutdown_connections 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f74b006b4d0 0x7f74b006d990 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74d4154ee0 0x7f74d4159f40 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74d40cfcf0 0x7f74d4159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 --2- 172.21.15.165:0/1445256487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74d40ce590 0x7f74d41f6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 >> 172.21.15.165:0/1445256487 conn(0x7f74d406abe0 msgr2=0x7f74d415d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 shutdown_connections 2024-04-25T10:07:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:18.898+0000 7f74da000640 1 -- 172.21.15.165:0/1445256487 wait complete. 2024-04-25T10:07:18.921 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:18.922 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.120+0000 7f4f25a00640 1 Processor -- start 2024-04-25T10:07:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.121+0000 7f4f25a00640 1 -- start start 2024-04-25T10:07:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.122+0000 7f4f25a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 0x7f4f200b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.122+0000 7f4f25a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f2006e8a0 0x7f4f200b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.122+0000 7f4f25a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f200cccc0 con 0x7f4f20144d50 2024-04-25T10:07:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.122+0000 7f4f25a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f200b2b20 con 0x7f4f2006e8a0 2024-04-25T10:07:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.122+0000 7f4f25a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2006d400 con 0x7f4f201446d0 2024-04-25T10:07:19.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.123+0000 7f4f1f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f2006e8a0 0x7f4f200b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.123+0000 7f4f1ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 0x7f4f200b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.123+0000 7f4f1ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 0x7f4f200b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:41316/0 (socket says 172.21.15.165:41316) 2024-04-25T10:07:19.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.123+0000 7f4f1f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4f2006e8a0 0x7f4f200b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:41426/0 (socket says 172.21.15.165:41426) 2024-04-25T10:07:19.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.123+0000 7f4f1ea00640 1 -- 172.21.15.165:0/3658910111 learned_addr learned my addr 172.21.15.165:0/3658910111 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:19.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.123+0000 7f4f1e000640 1 --1- 172.21.15.165:0/3658910111 >> v1:172.21.15.172:6789/0 conn(0x7f4f20144d50 0x7f4f200cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:51892/0 (socket says 172.21.15.165:51892) 2024-04-25T10:07:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.124+0000 7f4f1ea00640 1 -- 172.21.15.165:0/3658910111 >> v2:172.21.15.172:3300/0 conn(0x7f4f2006e8a0 msgr2=0x7f4f200b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.124+0000 7f4f1ea00640 1 --2- 172.21.15.165:0/3658910111 >> v2:172.21.15.172:3300/0 conn(0x7f4f2006e8a0 0x7f4f200b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.124+0000 7f4f1ea00640 1 -- 172.21.15.165:0/3658910111 >> v1:172.21.15.172:6789/0 conn(0x7f4f20144d50 legacy=0x7f4f200cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.124+0000 7f4f1ea00640 1 -- 172.21.15.165:0/3658910111 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f200cc140 con 0x7f4f201446d0 2024-04-25T10:07:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.125+0000 7f4f1ea00640 1 --2- 172.21.15.165:0/3658910111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 0x7f4f200b3f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009870 tx=0x7f4f1002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df49475f8ab83747 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.126+0000 7f4f1d600640 1 -- 172.21.15.165:0/3658910111 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10037070 con 0x7f4f201446d0 2024-04-25T10:07:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.126+0000 7f4f1d600640 1 -- 172.21.15.165:0/3658910111 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f10032440 con 0x7f4f201446d0 2024-04-25T10:07:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.126+0000 7f4f1d600640 1 -- 172.21.15.165:0/3658910111 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f10032810 con 0x7f4f201446d0 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.126+0000 7f4f25a00640 1 -- 172.21.15.165:0/3658910111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 msgr2=0x7f4f200b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.126+0000 7f4f25a00640 1 --2- 172.21.15.165:0/3658910111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 0x7f4f200b3f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4f10009870 tx=0x7f4f1002a250 comp rx=0 tx=0).stop 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.127+0000 7f4f25a00640 1 -- 172.21.15.165:0/3658910111 shutdown_connections 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.127+0000 7f4f25a00640 1 --2- 172.21.15.165:0/3658910111 >> v2:172.21.15.172:3300/0 conn(0x7f4f2006e8a0 0x7f4f200b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.127+0000 7f4f25a00640 1 --2- 172.21.15.165:0/3658910111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f201446d0 0x7f4f200b3f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.127+0000 7f4f25a00640 1 -- 172.21.15.165:0/3658910111 >> 172.21.15.165:0/3658910111 conn(0x7f4f2006abe0 msgr2=0x7f4f2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.127+0000 7f4f25a00640 1 -- 172.21.15.165:0/3658910111 shutdown_connections 2024-04-25T10:07:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.127+0000 7f4f25a00640 1 -- 172.21.15.165:0/3658910111 wait complete. 2024-04-25T10:07:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.129+0000 7f4f25a00640 1 Processor -- start 2024-04-25T10:07:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.129+0000 7f4f25a00640 1 -- start start 2024-04-25T10:07:19.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f25a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2006e8a0 0x7f4f2015af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f25a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 0x7f4f200cdcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f25a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20144d50 0x7f4f201e3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f25a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f200cc700 con 0x7f4f201446d0 2024-04-25T10:07:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f25a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f200b49a0 con 0x7f4f20144d50 2024-04-25T10:07:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f25a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f200cc9e0 con 0x7f4f2006e8a0 2024-04-25T10:07:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f1e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 0x7f4f200cdcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f1e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 0x7f4f200cdcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41462/0 (socket says 172.21.15.165:41462) 2024-04-25T10:07:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f1e000640 1 -- 172.21.15.165:0/33463242 learned_addr learned my addr 172.21.15.165:0/33463242 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f1ea00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2006e8a0 0x7f4f2015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.130+0000 7f4f1f400640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20144d50 0x7f4f201e3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f1e000640 1 -- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2006e8a0 msgr2=0x7f4f2015af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f1e000640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2006e8a0 0x7f4f2015af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f1e000640 1 -- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20144d50 msgr2=0x7f4f201e3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f1e000640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20144d50 0x7f4f201e3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f1e000640 1 -- 172.21.15.165:0/33463242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f200ce7b0 con 0x7f4f201446d0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f1e000640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 0x7f4f200cdcf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c730 tx=0x7f4f0800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f08017070 con 0x7f4f201446d0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0800ce40 con 0x7f4f201446d0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0800a860 con 0x7f4f201446d0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f200cea90 con 0x7f4f201446d0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.131+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f2013f680 con 0x7f4f201446d0 2024-04-25T10:07:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.132+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f201e16e0 con 0x7f4f201446d0 2024-04-25T10:07:19.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.134+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f0801a070 con 0x7f4f201446d0 2024-04-25T10:07:19.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.135+0000 7f4f16a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4efc06b500 0x7f4efc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.135+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4f08089600 con 0x7f4f201446d0 2024-04-25T10:07:19.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.135+0000 7f4f1ea00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4efc06b500 0x7f4efc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.136+0000 7f4f1ea00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4efc06b500 0x7f4efc06d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f10034000 tx=0x7f4f1002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.140+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f08058680 con 0x7f4f201446d0 2024-04-25T10:07:19.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.333+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f20073a00 con 0x7f4f201446d0 2024-04-25T10:07:19.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.334+0000 7f4f16a00640 1 -- 172.21.15.165:0/33463242 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f08057f80 con 0x7f4f201446d0 2024-04-25T10:07:19.340 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:07:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4efc06b500 msgr2=0x7f4efc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4efc06b500 0x7f4efc06d9c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4f10034000 tx=0x7f4f1002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 msgr2=0x7f4f200cdcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 0x7f4f200cdcf0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f4f0800c730 tx=0x7f4f0800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 shutdown_connections 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4efc06b500 0x7f4efc06d9c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4f20144d50 0x7f4f201e3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4f201446d0 0x7f4f200cdcf0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 --2- 172.21.15.165:0/33463242 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4f2006e8a0 0x7f4f2015af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 >> 172.21.15.165:0/33463242 conn(0x7f4f2006abe0 msgr2=0x7f4f200af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 shutdown_connections 2024-04-25T10:07:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.337+0000 7f4f25a00640 1 -- 172.21.15.165:0/33463242 wait complete. 2024-04-25T10:07:19.361 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:07:19.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.560+0000 7fcda1c00640 1 Processor -- start 2024-04-25T10:07:19.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.560+0000 7fcda1c00640 1 -- start start 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.561+0000 7fcda1c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcd9c0d0ec0 0x7fcd9c0cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.561+0000 7fcda1c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcd9c0c4cf0 0x7fcd9c0cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.561+0000 7fcda1c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c0cea40 con 0x7fcd9c0cd6f0 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.561+0000 7fcda1c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c0cdad0 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.561+0000 7fcda1c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c0cf1e0 con 0x7fcd9c0d0ec0 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcd9be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcd9c0cd6f0 0x7fcd9c0c45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42182/0 (socket says 172.21.15.165:42182) 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcd9be00640 1 -- 172.21.15.165:0/2479368481 learned_addr learned my addr 172.21.15.165:0/2479368481 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:19.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcda1200640 1 --2- 172.21.15.165:0/2479368481 >> v2:172.21.15.165:3300/0 conn(0x7fcd9c0c4cf0 0x7fcd9c0cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcd9b400640 1 --2- 172.21.15.165:0/2479368481 >> v2:172.21.15.172:3300/0 conn(0x7fcd9c0d0ec0 0x7fcd9c0cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcda1200640 1 -- 172.21.15.165:0/2479368481 >> v2:172.21.15.172:3300/0 conn(0x7fcd9c0d0ec0 msgr2=0x7fcd9c0cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:19.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcda1200640 1 --2- 172.21.15.165:0/2479368481 >> v2:172.21.15.172:3300/0 conn(0x7fcd9c0d0ec0 0x7fcd9c0cbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcda1200640 1 -- 172.21.15.165:0/2479368481 >> v1:172.21.15.165:6789/0 conn(0x7fcd9c0cd6f0 legacy=0x7fcd9c0c45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.562+0000 7fcda1200640 1 -- 172.21.15.165:0/2479368481 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c0ced20 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.563+0000 7fcda1200640 1 --2- 172.21.15.165:0/2479368481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c0cc1a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06eb80 tx=0x7fcd9402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7417a2cda725f73a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.563+0000 7fcd9aa00640 1 -- 172.21.15.165:0/2479368481 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd94037070 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.563+0000 7fcd9aa00640 1 -- 172.21.15.165:0/2479368481 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd94032440 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcd9aa00640 1 -- 172.21.15.165:0/2479368481 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9402ae60 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 -- 172.21.15.165:0/2479368481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 msgr2=0x7fcd9c0cc1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 --2- 172.21.15.165:0/2479368481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c0cc1a0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c06eb80 tx=0x7fcd9402a110 comp rx=0 tx=0).stop 2024-04-25T10:07:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 -- 172.21.15.165:0/2479368481 shutdown_connections 2024-04-25T10:07:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 --2- 172.21.15.165:0/2479368481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c0cc1a0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 --2- 172.21.15.165:0/2479368481 >> v2:172.21.15.172:3300/0 conn(0x7fcd9c0d0ec0 0x7fcd9c0cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 -- 172.21.15.165:0/2479368481 >> 172.21.15.165:0/2479368481 conn(0x7fcd9c0d1610 msgr2=0x7fcd9c0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 -- 172.21.15.165:0/2479368481 shutdown_connections 2024-04-25T10:07:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.564+0000 7fcda1c00640 1 -- 172.21.15.165:0/2479368481 wait complete. 2024-04-25T10:07:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.566+0000 7fcda1c00640 1 Processor -- start 2024-04-25T10:07:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.566+0000 7fcda1c00640 1 -- start start 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcda1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c1584a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcda1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd9c0cd6f0 0x7fcd9c0aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcda1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd9c0d0ec0 0x7fcd9c0b66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcda1c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0b2540 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcda1c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0cf950 con 0x7fcd9c0d0ec0 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcda1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c0b26c0 con 0x7fcd9c0cd6f0 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c1584a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcd9be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c1584a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41474/0 (socket says 172.21.15.165:41474) 2024-04-25T10:07:19.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.567+0000 7fcd9be00640 1 -- 172.21.15.165:0/1319750204 learned_addr learned my addr 172.21.15.165:0/1319750204 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:19.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9b400640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd9c0cd6f0 0x7fcd9c0aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcda1200640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd9c0d0ec0 0x7fcd9c0b66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9be00640 1 -- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd9c0cd6f0 msgr2=0x7fcd9c0aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9be00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd9c0cd6f0 0x7fcd9c0aeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9be00640 1 -- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd9c0d0ec0 msgr2=0x7fcd9c0b66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9be00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd9c0d0ec0 0x7fcd9c0b66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9be00640 1 -- 172.21.15.165:0/1319750204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c0ae800 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd9be00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c1584a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00cb10 tx=0x7fcd8c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c017070 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c00ac70 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd9c0af060 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c012620 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.568+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c1f8130 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.570+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c073990 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.571+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcd8c01a070 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.572+0000 7fcd93e00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcd7806b4d0 0x7fcd7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.572+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcd8c089e70 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.572+0000 7fcd9b400640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcd7806b4d0 0x7fcd7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:19.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.573+0000 7fcd9b400640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcd7806b4d0 0x7fcd7806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006fd0 tx=0x7fcd84008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.576+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd9c0af060 con 0x7fcd9c0c4cf0 2024-04-25T10:07:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:19.842+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7fcd9c09a700 con 0x7fcd9c0c4cf0 2024-04-25T10:07:20.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.468+0000 7fcd93e00640 1 -- 172.21.15.165:0/1319750204 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v269) v1 ==== 146+0+0 (secure 0 0 0) 0x7fcd9c0ae800 con 0x7fcd9c0c4cf0 2024-04-25T10:07:20.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.471+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcd7806b4d0 msgr2=0x7fcd7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.471+0000 7fcda1c00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcd7806b4d0 0x7fcd7806d990 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fcd84006fd0 tx=0x7fcd84008040 comp rx=0 tx=0).stop 2024-04-25T10:07:20.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.471+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 msgr2=0x7fcd9c1584a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.471+0000 7fcda1c00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c1584a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00cb10 tx=0x7fcd8c00a060 comp rx=0 tx=0).stop 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 shutdown_connections 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcd7806b4d0 0x7fcd7806d990 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd9c0d0ec0 0x7fcd9c0b66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd9c0cd6f0 0x7fcd9c0aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 --2- 172.21.15.165:0/1319750204 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd9c0c4cf0 0x7fcd9c1584a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 >> 172.21.15.165:0/1319750204 conn(0x7fcd9c0d1610 msgr2=0x7fcd9c06d030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 shutdown_connections 2024-04-25T10:07:20.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.472+0000 7fcda1c00640 1 -- 172.21.15.165:0/1319750204 wait complete. 2024-04-25T10:07:20.772 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:07:20.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.982+0000 7f518da00640 1 Processor -- start 2024-04-25T10:07:20.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.983+0000 7f518da00640 1 -- start start 2024-04-25T10:07:20.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.984+0000 7f518da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51880b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.984+0000 7f518da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518815b7a0 0x7f51880cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.984+0000 7f518da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880b0b80 con 0x7f51880cdae0 2024-04-25T10:07:20.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.984+0000 7f518da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51880cd650 con 0x7f518815b7a0 2024-04-25T10:07:20.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.984+0000 7f518da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880cc800 con 0x7f518815b3c0 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.985+0000 7f517f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518815b7a0 0x7f51880cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.985+0000 7f5187e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51880b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.985+0000 7f517f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f518815b7a0 0x7f51880cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:41474/0 (socket says 172.21.15.165:41474) 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.985+0000 7f517f400640 1 -- 172.21.15.165:0/1406407041 learned_addr learned my addr 172.21.15.165:0/1406407041 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.986+0000 7f5187e00640 1 -- 172.21.15.165:0/1406407041 >> v2:172.21.15.172:3300/0 conn(0x7f518815b7a0 msgr2=0x7f51880cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.986+0000 7f5187e00640 1 --2- 172.21.15.165:0/1406407041 >> v2:172.21.15.172:3300/0 conn(0x7f518815b7a0 0x7f51880cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.986+0000 7f5187e00640 1 -- 172.21.15.165:0/1406407041 >> v1:172.21.15.172:6789/0 conn(0x7f51880cdae0 legacy=0x7f518815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.986+0000 7f5187e00640 1 -- 172.21.15.165:0/1406407041 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880ccae0 con 0x7f518815b3c0 2024-04-25T10:07:20.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.987+0000 7f5187e00640 1 --2- 172.21.15.165:0/1406407041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51880b0f60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51780097a0 tx=0x7f517802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc34ff60a6039559 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:20.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.988+0000 7f5187400640 1 -- 172.21.15.165:0/1406407041 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178037070 con 0x7f518815b3c0 2024-04-25T10:07:20.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.988+0000 7f5187400640 1 -- 172.21.15.165:0/1406407041 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178032440 con 0x7f518815b3c0 2024-04-25T10:07:20.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.988+0000 7f5187400640 1 -- 172.21.15.165:0/1406407041 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178032810 con 0x7f518815b3c0 2024-04-25T10:07:20.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.988+0000 7f518da00640 1 -- 172.21.15.165:0/1406407041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 msgr2=0x7f51880b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.988+0000 7f518da00640 1 --2- 172.21.15.165:0/1406407041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51880b0f60 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f51780097a0 tx=0x7f517802a180 comp rx=0 tx=0).stop 2024-04-25T10:07:20.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.989+0000 7f518da00640 1 -- 172.21.15.165:0/1406407041 shutdown_connections 2024-04-25T10:07:20.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.989+0000 7f518da00640 1 --2- 172.21.15.165:0/1406407041 >> v2:172.21.15.172:3300/0 conn(0x7f518815b7a0 0x7f51880cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.989+0000 7f518da00640 1 --2- 172.21.15.165:0/1406407041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51880b0f60 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.989+0000 7f518da00640 1 -- 172.21.15.165:0/1406407041 >> 172.21.15.165:0/1406407041 conn(0x7f5188061cc0 msgr2=0x7f518806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:20.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.989+0000 7f518da00640 1 -- 172.21.15.165:0/1406407041 shutdown_connections 2024-04-25T10:07:20.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.989+0000 7f518da00640 1 -- 172.21.15.165:0/1406407041 wait complete. 2024-04-25T10:07:20.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.990+0000 7f518da00640 1 Processor -- start 2024-04-25T10:07:20.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.990+0000 7f518da00640 1 -- start start 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.991+0000 7f518da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 0x7f51880ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.991+0000 7f518da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51881432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.991+0000 7f518da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518815b7a0 0x7f5188144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.991+0000 7f518da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518815e8a0 con 0x7f51880cdae0 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.991+0000 7f518da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f518815e720 con 0x7f518815b7a0 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.991+0000 7f518da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880cdf40 con 0x7f518815b3c0 2024-04-25T10:07:20.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 0x7f51880ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 0x7f51880ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41490/0 (socket says 172.21.15.165:41490) 2024-04-25T10:07:20.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 -- 172.21.15.165:0/135143407 learned_addr learned my addr 172.21.15.165:0/135143407 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f518d000640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518815b7a0 0x7f5188144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f517f400640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51881432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 -- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 msgr2=0x7f51881432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51881432b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 -- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518815b7a0 msgr2=0x7f5188144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518815b7a0 0x7f5188144430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f5187e00640 1 -- 172.21.15.165:0/135143407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5188144970 con 0x7f51880cdae0 2024-04-25T10:07:20.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.992+0000 7f518d000640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518815b7a0 0x7f5188144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.993+0000 7f5187e00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 0x7f51880ca220 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5178035040 tx=0x7f51780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.993+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178037040 con 0x7f51880cdae0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.993+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178030500 con 0x7f51880cdae0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.993+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178030910 con 0x7f51880cdae0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.993+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5188144bf0 con 0x7f51880cdae0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.993+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51881451b0 con 0x7f51880cdae0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.994+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5188073990 con 0x7f51880cdae0 2024-04-25T10:07:21.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.995+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5178044020 con 0x7f51880cdae0 2024-04-25T10:07:21.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.996+0000 7f5184c00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516406b4d0 0x7f516406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.996+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f51780aa3b0 con 0x7f51880cdae0 2024-04-25T10:07:21.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.996+0000 7f517f400640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516406b4d0 0x7f516406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:20.997+0000 7f517f400640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516406b4d0 0x7f516406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f51880cb240 tx=0x7f5170008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.000+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5188144970 con 0x7f51880cdae0 2024-04-25T10:07:21.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.199+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f51880cb820 con 0x7f51880cdae0 2024-04-25T10:07:21.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.204+0000 7f5184c00640 1 -- 172.21.15.165:0/135143407 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5178079300 con 0x7f51880cdae0 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516406b4d0 msgr2=0x7f516406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516406b4d0 0x7f516406d990 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f51880cb240 tx=0x7f5170008040 comp rx=0 tx=0).stop 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 msgr2=0x7f51880ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 0x7f51880ca220 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f5178035040 tx=0x7f51780046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 shutdown_connections 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f516406b4d0 0x7f516406d990 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f518815b7a0 0x7f5188144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f518815b3c0 0x7f51881432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 --2- 172.21.15.165:0/135143407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51880cdae0 0x7f51880ca220 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 >> 172.21.15.165:0/135143407 conn(0x7f5188061cc0 msgr2=0x7f518806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 shutdown_connections 2024-04-25T10:07:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.207+0000 7f518da00640 1 -- 172.21.15.165:0/135143407 wait complete. 2024-04-25T10:07:21.233 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:21.233 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:21.323 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:21.451 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:21.451 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:21.451 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:21.509 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:21.509 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:07:21.509 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:21.562 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:21.562 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:21.562 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:21.621 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:07:21.621 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:07:21.621 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:21.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:07:21.680 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:07:21.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:21.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.881+0000 7f5bc9800640 1 Processor -- start 2024-04-25T10:07:21.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.882+0000 7f5bc9800640 1 -- start start 2024-04-25T10:07:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc9800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5bc415dbe0 0x7f5bc40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc9800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc40b6220 con 0x7f5bc40b92a0 2024-04-25T10:07:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc9800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc40b97a0 con 0x7f5bc40b2cb0 2024-04-25T10:07:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc9800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc415dfc0 con 0x7f5bc415dbe0 2024-04-25T10:07:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5bc415dbe0 0x7f5bc40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5bc40b92a0 0x7f5bc40b5b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42194/0 (socket says 172.21.15.165:42194) 2024-04-25T10:07:21.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.883+0000 7f5bc3400640 1 -- 172.21.15.165:0/3662895608 learned_addr learned my addr 172.21.15.165:0/3662895608 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.884+0000 7f5bc2a00640 1 -- 172.21.15.165:0/3662895608 >> v1:172.21.15.172:6789/0 conn(0x7f5bc40b2cb0 legacy=0x7f5bc40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.884+0000 7f5bc2a00640 1 -- 172.21.15.165:0/3662895608 >> v1:172.21.15.165:6789/0 conn(0x7f5bc40b92a0 legacy=0x7f5bc40b5b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.884+0000 7f5bc2a00640 1 -- 172.21.15.165:0/3662895608 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40d1c50 con 0x7f5bc415dbe0 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.884+0000 7f5bc2000640 1 -- 172.21.15.165:0/3662895608 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359332099 0 0) 0x7f5bc40b6220 con 0x7f5bc40b92a0 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.885+0000 7f5bc2a00640 1 --2- 172.21.15.165:0/3662895608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc415dbe0 0x7f5bc40d0910 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40097a0 tx=0x7f5bb402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dea47fea8905afcc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.885+0000 7f5bc2000640 1 -- 172.21.15.165:0/3662895608 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4037070 con 0x7f5bc415dbe0 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.885+0000 7f5bc2000640 1 -- 172.21.15.165:0/3662895608 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bb402ae50 con 0x7f5bc415dbe0 2024-04-25T10:07:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.885+0000 7f5bc2000640 1 -- 172.21.15.165:0/3662895608 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb4032660 con 0x7f5bc415dbe0 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 -- 172.21.15.165:0/3662895608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc415dbe0 msgr2=0x7f5bc40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 --2- 172.21.15.165:0/3662895608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc415dbe0 0x7f5bc40d0910 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40097a0 tx=0x7f5bb402a140 comp rx=0 tx=0).stop 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 -- 172.21.15.165:0/3662895608 shutdown_connections 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 --2- 172.21.15.165:0/3662895608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc415dbe0 0x7f5bc40d0910 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 -- 172.21.15.165:0/3662895608 >> 172.21.15.165:0/3662895608 conn(0x7f5bc4061a20 msgr2=0x7f5bc4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 -- 172.21.15.165:0/3662895608 shutdown_connections 2024-04-25T10:07:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.886+0000 7f5bc9800640 1 -- 172.21.15.165:0/3662895608 wait complete. 2024-04-25T10:07:21.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.888+0000 7f5bc9800640 1 Processor -- start 2024-04-25T10:07:21.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.888+0000 7f5bc9800640 1 -- start start 2024-04-25T10:07:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc40b2cb0 0x7f5bc415fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 0x7f5bc40c9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc415dbe0 0x7f5bc40b42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc9800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40d2210 con 0x7f5bc40b92a0 2024-04-25T10:07:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40d1a00 con 0x7f5bc40b2cb0 2024-04-25T10:07:21.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40b6ab0 con 0x7f5bc415dbe0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 0x7f5bc40c9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 0x7f5bc40c9bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41514/0 (socket says 172.21.15.165:41514) 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bba000640 1 -- 172.21.15.165:0/306850315 learned_addr learned my addr 172.21.15.165:0/306850315 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.889+0000 7f5bc2a00640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc40b2cb0 0x7f5bc415fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bc3400640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc415dbe0 0x7f5bc40b42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bba000640 1 -- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc415dbe0 msgr2=0x7f5bc40b42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bba000640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc415dbe0 0x7f5bc40b42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bba000640 1 -- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc40b2cb0 msgr2=0x7f5bc415fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bba000640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc40b2cb0 0x7f5bc415fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bba000640 1 -- 172.21.15.165:0/306850315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc40cabf0 con 0x7f5bc40b92a0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bba000640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 0x7f5bc40c9bc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00c400 tx=0x7f5bac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011790 con 0x7f5bc40b92a0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bac017070 con 0x7f5bc40b92a0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bac011bb0 con 0x7f5bc40b92a0 2024-04-25T10:07:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.890+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc40caed0 con 0x7f5bc40b92a0 2024-04-25T10:07:21.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.891+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc40cb490 con 0x7f5bc40b92a0 2024-04-25T10:07:21.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.892+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc40cb690 con 0x7f5bc40b92a0 2024-04-25T10:07:21.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.893+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5bac01c460 con 0x7f5bc40b92a0 2024-04-25T10:07:21.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.893+0000 7f5bbb400640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ba006b500 0x7f5ba006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:21.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.894+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5bac089ec0 con 0x7f5bc40b92a0 2024-04-25T10:07:21.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.894+0000 7f5bc2a00640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ba006b500 0x7f5ba006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:21.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.894+0000 7f5bc2a00640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ba006b500 0x7f5ba006d9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4034000 tx=0x7f5bb402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:21.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:21.898+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5bc40cb690 con 0x7f5bc40b92a0 2024-04-25T10:07:22.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.131+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5bc4073990 con 0x7f5bc40b92a0 2024-04-25T10:07:22.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.132+0000 7f5bbb400640 1 -- 172.21.15.165:0/306850315 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+3705 (secure 0 0 0) 0x7f5bac058ea0 con 0x7f5bc40b92a0 2024-04-25T10:07:22.138 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:22.138 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":34,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4772,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/850480483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":850480483},{"type":"v1","addr":"172.21.15.172:6835","nonce":850480483}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24},{"gid":4940,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/433185229","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":433185229},{"type":"v1","addr":"172.21.15.172:6834","nonce":433185229}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":34},{"gid":5793,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2275608575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2275608575},{"type":"v1","addr":"172.21.15.165:6835","nonce":2275608575}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":23},{"gid":5796,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2042122454","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2042122454},{"type":"v1","addr":"172.21.15.165:6837","nonce":2042122454}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":24}],"filesystems":[]} 2024-04-25T10:07:22.138 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 34 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ba006b500 msgr2=0x7f5ba006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ba006b500 0x7f5ba006d9c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4034000 tx=0x7f5bb402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 msgr2=0x7f5bc40c9bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 0x7f5bc40c9bc0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f5bac00c400 tx=0x7f5bac00c8c0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 shutdown_connections 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5ba006b500 0x7f5ba006d9c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc415dbe0 0x7f5bc40b42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc40b92a0 0x7f5bc40c9bc0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 --2- 172.21.15.165:0/306850315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc40b2cb0 0x7f5bc415fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 >> 172.21.15.165:0/306850315 conn(0x7f5bc4061a20 msgr2=0x7f5bc41527d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 shutdown_connections 2024-04-25T10:07:22.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.135+0000 7f5bc9800640 1 -- 172.21.15.165:0/306850315 wait complete. 2024-04-25T10:07:22.161 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:22.161 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:07:22.161 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:22.161 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:22.162 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:22.162 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:07:22.162 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:22.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:22.162 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:22.162 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:07:22.162 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:22.162 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:22.163 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:22.163 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:07:22.163 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:22.163 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:22.183 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:22.183 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:22.184 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:22.185 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:22.186 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:22.186 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:22.188 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:22.188 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:22.232 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:22.233 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:22.235 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:22.236 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:22.238 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:22.239 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:22.241 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:22.242 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:22.242 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:07:22.381 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:07:22.381 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:07:22.393 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:07:22.394 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:07:22.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.412+0000 7ffadec00640 1 Processor -- start 2024-04-25T10:07:22.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.413+0000 7ffadec00640 1 -- start start 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffadec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffad81446d0 0x7ffad80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffadec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad8144d50 0x7ffad80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffadec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffad80cccc0 con 0x7ffad81452a0 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffadec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffad80b2b20 con 0x7ffad81446d0 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffadec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad806d400 con 0x7ffad8144d50 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffad7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffad81446d0 0x7ffad80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffad7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffad81446d0 0x7ffad80b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41554/0 (socket says 172.21.15.165:41554) 2024-04-25T10:07:22.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffad7e00640 1 -- 172.21.15.165:0/745839139 learned_addr learned my addr 172.21.15.165:0/745839139 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:22.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.414+0000 7ffad7400640 1 --2- 172.21.15.165:0/745839139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad8144d50 0x7ffad80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.415+0000 7ffad7e00640 1 -- 172.21.15.165:0/745839139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad8144d50 msgr2=0x7ffad80b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:22.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.415+0000 7ffad7e00640 1 --2- 172.21.15.165:0/745839139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad8144d50 0x7ffad80b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.415+0000 7ffad7e00640 1 -- 172.21.15.165:0/745839139 >> v1:172.21.15.172:6789/0 conn(0x7ffad81452a0 legacy=0x7ffad80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.415+0000 7ffad7e00640 1 -- 172.21.15.165:0/745839139 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad80cc140 con 0x7ffad81446d0 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.416+0000 7ffad7e00640 1 --2- 172.21.15.165:0/745839139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad81446d0 0x7ffad80b3f20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009870 tx=0x7ffac802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedcda0b8f6eb0f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.416+0000 7ffad6a00640 1 -- 172.21.15.165:0/745839139 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8037070 con 0x7ffad81446d0 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.416+0000 7ffad6a00640 1 -- 172.21.15.165:0/745839139 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac8032440 con 0x7ffad81446d0 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.416+0000 7ffad6a00640 1 -- 172.21.15.165:0/745839139 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac8032810 con 0x7ffad81446d0 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.416+0000 7ffadec00640 1 -- 172.21.15.165:0/745839139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad81446d0 msgr2=0x7ffad80b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.416+0000 7ffadec00640 1 --2- 172.21.15.165:0/745839139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad81446d0 0x7ffad80b3f20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7ffac8009870 tx=0x7ffac802a270 comp rx=0 tx=0).stop 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.417+0000 7ffadec00640 1 -- 172.21.15.165:0/745839139 shutdown_connections 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.417+0000 7ffadec00640 1 --2- 172.21.15.165:0/745839139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad8144d50 0x7ffad80b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.417+0000 7ffadec00640 1 --2- 172.21.15.165:0/745839139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad81446d0 0x7ffad80b3f20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.417+0000 7ffadec00640 1 -- 172.21.15.165:0/745839139 >> 172.21.15.165:0/745839139 conn(0x7ffad806abe0 msgr2=0x7ffad806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.417+0000 7ffadec00640 1 -- 172.21.15.165:0/745839139 shutdown_connections 2024-04-25T10:07:22.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.417+0000 7ffadec00640 1 -- 172.21.15.165:0/745839139 wait complete. 2024-04-25T10:07:22.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.418+0000 7ffadec00640 1 Processor -- start 2024-04-25T10:07:22.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.418+0000 7ffadec00640 1 -- start start 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.419+0000 7ffadec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad81446d0 0x7ffad80cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.419+0000 7ffadec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 0x7ffad813c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.419+0000 7ffadec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad81452a0 0x7ffad813c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.419+0000 7ffadec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80cc700 con 0x7ffad8144d50 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.419+0000 7ffadec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80b49a0 con 0x7ffad81446d0 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.419+0000 7ffadec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffad80cc9e0 con 0x7ffad81452a0 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 0x7ffad813c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 0x7ffad813c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41562/0 (socket says 172.21.15.165:41562) 2024-04-25T10:07:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 -- 172.21.15.165:0/3097310203 learned_addr learned my addr 172.21.15.165:0/3097310203 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7e00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad81446d0 0x7ffad80cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffadcc00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad81452a0 0x7ffad813c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 -- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad81452a0 msgr2=0x7ffad813c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad81452a0 0x7ffad813c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 -- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad81446d0 msgr2=0x7ffad80cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad81446d0 0x7ffad80cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7400640 1 -- 172.21.15.165:0/3097310203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad81f43e0 con 0x7ffad8144d50 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.420+0000 7ffad7e00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad81446d0 0x7ffad80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.421+0000 7ffad7400640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 0x7ffad813c230 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ffac400d800 tx=0x7ffac400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.421+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400b6c0 con 0x7ffad8144d50 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.421+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffac4019070 con 0x7ffad8144d50 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.421+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffac400bac0 con 0x7ffad8144d50 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.421+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad80ac9e0 con 0x7ffad8144d50 2024-04-25T10:07:22.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.421+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad80ad4c0 con 0x7ffad8144d50 2024-04-25T10:07:22.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.422+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8073a00 con 0x7ffad8144d50 2024-04-25T10:07:22.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.423+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffac400bc60 con 0x7ffad8144d50 2024-04-25T10:07:22.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.424+0000 7ffacfe00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffab806b4d0 0x7ffab806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:22.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.424+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ffac408a370 con 0x7ffad8144d50 2024-04-25T10:07:22.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.424+0000 7ffad7e00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffab806b4d0 0x7ffab806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:22.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.425+0000 7ffad7e00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffab806b4d0 0x7ffab806d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffac8034000 tx=0x7ffac802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:22.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.428+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffac4058da0 con 0x7ffad8144d50 2024-04-25T10:07:22.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:22.642+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ffad813e890 con 0x7ffad8144d50 2024-04-25T10:07:23.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.508+0000 7ffacfe00640 1 -- 172.21.15.165:0/3097310203 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v64) v1 ==== 97+0+0 (secure 0 0 0) 0x7ffac405ce40 con 0x7ffad8144d50 2024-04-25T10:07:23.514 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffab806b4d0 msgr2=0x7ffab806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffab806b4d0 0x7ffab806d990 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ffac8034000 tx=0x7ffac802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 msgr2=0x7ffad813c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 0x7ffad813c230 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ffac400d800 tx=0x7ffac400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 shutdown_connections 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffab806b4d0 0x7ffab806d990 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffad81452a0 0x7ffad813c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffad8144d50 0x7ffad813c230 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 --2- 172.21.15.165:0/3097310203 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffad81446d0 0x7ffad80cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.512+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 >> 172.21.15.165:0/3097310203 conn(0x7ffad806abe0 msgr2=0x7ffad8071d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.513+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 shutdown_connections 2024-04-25T10:07:23.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.513+0000 7ffadec00640 1 -- 172.21.15.165:0/3097310203 wait complete. 2024-04-25T10:07:23.538 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:07:23.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.731+0000 7f08de800640 1 Processor -- start 2024-04-25T10:07:23.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.732+0000 7f08de800640 1 -- start start 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.733+0000 7f08de800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08d8144ae0 0x7f08d80ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.733+0000 7f08de800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08d815bcf0 0x7f08d815f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.733+0000 7f08de800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08d815c520 con 0x7f08d815fef0 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.733+0000 7f08de800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08d815c800 con 0x7f08d8144ae0 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.733+0000 7f08de800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d815cae0 con 0x7f08d815bcf0 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08dde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f08d815fef0 0x7f08d806d270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42226/0 (socket says 172.21.15.165:42226) 2024-04-25T10:07:23.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08dde00640 1 -- 172.21.15.165:0/3796209233 learned_addr learned my addr 172.21.15.165:0/3796209233 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08dd400640 1 --2- 172.21.15.165:0/3796209233 >> v2:172.21.15.172:3300/0 conn(0x7f08d8144ae0 0x7f08d80ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618368494 0 0) 0x7f08d815c520 con 0x7f08d815fef0 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08c0003520 con 0x7f08d815fef0 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08dca00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08d815bcf0 0x7f08d815f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1477821315 0 0) 0x7f08c0003520 con 0x7f08d815fef0 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08d815bcf0 msgr2=0x7f08d815f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.734+0000 7f08d7e00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08d815bcf0 0x7f08d815f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 >> v2:172.21.15.172:3300/0 conn(0x7f08d8144ae0 msgr2=0x7f08d80ccb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 --2- 172.21.15.165:0/3796209233 >> v2:172.21.15.172:3300/0 conn(0x7f08d8144ae0 0x7f08d80ccb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d815c150 con 0x7f08d815fef0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f08d00019f0 con 0x7f08d815fef0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 >> v1:172.21.15.165:6789/0 conn(0x7f08d815fef0 legacy=0x7f08d806d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c00039b0 0x7f08c0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08c0004800 0x7f08c0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08c00051c0 0x7f08c000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d815c800 con 0x7f08c0004800 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d815cae0 con 0x7f08c00051c0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c000bf80 con 0x7f08c00039b0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08dca00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08c0004800 0x7f08c0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08dd400640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c00039b0 0x7f08c0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08dde00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08c00051c0 0x7f08c000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08dca00640 1 -- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c00039b0 msgr2=0x7f08c0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08dca00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c00039b0 0x7f08c0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.735+0000 7f08dca00640 1 -- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08c00051c0 msgr2=0x7f08c000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:23.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.736+0000 7f08dca00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08c00051c0 0x7f08c000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.736+0000 7f08dca00640 1 -- 172.21.15.165:0/3796209233 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c0003520 con 0x7f08c0004800 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.737+0000 7f08dca00640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08c0004800 0x7f08c0004c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f08d80af2c0 tx=0x7f08c0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb021e7e243a5915 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.737+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c4030070 con 0x7f08c0004800 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.737+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08c402acc0 con 0x7f08c0004800 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.737+0000 7f08d7e00640 1 -- 172.21.15.165:0/3796209233 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c400b550 con 0x7f08c0004800 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08de800640 1 -- 172.21.15.165:0/3796209233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08c0004800 msgr2=0x7f08c0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08de800640 1 --2- 172.21.15.165:0/3796209233 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08c0004800 0x7f08c0004c80 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f08d80af2c0 tx=0x7f08c0005830 comp rx=0 tx=0).stop 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08dde00640 1 -- 172.21.15.165:0/3796209233 reap_dead start 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08de800640 1 -- 172.21.15.165:0/3796209233 shutdown_connections 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08de800640 1 -- 172.21.15.165:0/3796209233 >> 172.21.15.165:0/3796209233 conn(0x7f08d8061c30 msgr2=0x7f08d806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08de800640 1 -- 172.21.15.165:0/3796209233 shutdown_connections 2024-04-25T10:07:23.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.738+0000 7f08de800640 1 -- 172.21.15.165:0/3796209233 wait complete. 2024-04-25T10:07:23.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.740+0000 7f08de800640 1 Processor -- start 2024-04-25T10:07:23.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.740+0000 7f08de800640 1 -- start start 2024-04-25T10:07:23.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08de800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c0004800 0x7f08d80b31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08de800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08d80c7270 0x7f08d80c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08de800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 0x7f08d8136110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08de800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d815fb30 con 0x7f08d8135c90 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08de800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d815f9b0 con 0x7f08d80c7270 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08de800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08d815fcb0 con 0x7f08c0004800 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08dde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 0x7f08d8136110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08dde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 0x7f08d8136110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41584/0 (socket says 172.21.15.165:41584) 2024-04-25T10:07:23.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.741+0000 7f08dde00640 1 -- 172.21.15.165:0/663715084 learned_addr learned my addr 172.21.15.165:0/663715084 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:23.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dd400640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c0004800 0x7f08d80b31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dde00640 1 -- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c0004800 msgr2=0x7f08d80b31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dde00640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c0004800 0x7f08d80b31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dde00640 1 -- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08d80c7270 msgr2=0x7f08d80c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dca00640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08d80c7270 0x7f08d80c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dde00640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08d80c7270 0x7f08d80c76f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dde00640 1 -- 172.21.15.165:0/663715084 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d80c7c30 con 0x7f08d8135c90 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08dde00640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 0x7f08d8136110 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f08d80b4bb0 tx=0x7f08d0007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d000eb30 con 0x7f08d8135c90 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08d000ecd0 con 0x7f08d8135c90 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08d0014670 con 0x7f08d8135c90 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d80b40e0 con 0x7f08d8135c90 2024-04-25T10:07:23.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.742+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d80b3e00 con 0x7f08d8135c90 2024-04-25T10:07:23.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.744+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d8073990 con 0x7f08d8135c90 2024-04-25T10:07:23.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.745+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08d0014810 con 0x7f08d8135c90 2024-04-25T10:07:23.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.746+0000 7f08d5600640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f08b806b500 0x7f08b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:23.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.746+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f08d0089e70 con 0x7f08d8135c90 2024-04-25T10:07:23.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.746+0000 7f08dd400640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f08b806b500 0x7f08b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:23.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.747+0000 7f08dd400640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f08b806b500 0x7f08b806d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0099d0 tx=0x7f08cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:23.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:23.750+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08d0058e10 con 0x7f08d8135c90 2024-04-25T10:07:24.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.017+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f08d806f1c0 con 0x7f08d8135c90 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.018+0000 7f08d5600640 1 -- 172.21.15.165:0/663715084 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f08d0058710 con 0x7f08d8135c90 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:07:24.024 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.025 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.026 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.027 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:07:24.028 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.029 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.030 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:07:24.031 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.032 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.033 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.034 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:07:24.035 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:07:24.036 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:07:24.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f08b806b500 msgr2=0x7f08b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f08b806b500 0x7f08b806d9c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f08cc0099d0 tx=0x7f08cc008040 comp rx=0 tx=0).stop 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 msgr2=0x7f08d8136110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 0x7f08d8136110 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f08d80b4bb0 tx=0x7f08d0007bc0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 shutdown_connections 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f08b806b500 0x7f08b806d9c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08d8135c90 0x7f08d8136110 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08d80c7270 0x7f08d80c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 --2- 172.21.15.165:0/663715084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08c0004800 0x7f08d80b31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 >> 172.21.15.165:0/663715084 conn(0x7f08d8061c30 msgr2=0x7f08d80b0e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 shutdown_connections 2024-04-25T10:07:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.021+0000 7f08de800640 1 -- 172.21.15.165:0/663715084 wait complete. 2024-04-25T10:07:24.045 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:24.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.241+0000 7f51dd000640 1 Processor -- start 2024-04-25T10:07:24.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.242+0000 7f51dd000640 1 -- start start 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51dd000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f51d815db70 0x7f51d80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51dd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815df50 0x7f51d80cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51dd000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d8071650 con 0x7f51d80cec70 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51dd000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d80cbd80 con 0x7f51d815db70 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51dd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8161910 con 0x7f51d815df50 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51d5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f51d815db70 0x7f51d80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.243+0000 7f51d5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f51d815db70 0x7f51d80cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41592/0 (socket says 172.21.15.165:41592) 2024-04-25T10:07:24.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5e00640 1 -- 172.21.15.165:0/4239147397 learned_addr learned my addr 172.21.15.165:0/4239147397 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:24.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5400640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815df50 0x7f51d80cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5e00640 1 -- 172.21.15.165:0/4239147397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815df50 msgr2=0x7f51d80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5e00640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815df50 0x7f51d80cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5e00640 1 -- 172.21.15.165:0/4239147397 >> v1:172.21.15.172:6789/0 conn(0x7f51d80cec70 legacy=0x7f51d80cc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5e00640 1 -- 172.21.15.165:0/4239147397 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80cf230 con 0x7f51d815db70 2024-04-25T10:07:24.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.244+0000 7f51d5400640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815df50 0x7f51d80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.245+0000 7f51d5e00640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815db70 0x7f51d80cb840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009870 tx=0x7f51c802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0c486b4c9c0a3e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.245+0000 7f51d4a00640 1 -- 172.21.15.165:0/4239147397 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8037070 con 0x7f51d815db70 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.245+0000 7f51d4a00640 1 -- 172.21.15.165:0/4239147397 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c8032440 con 0x7f51d815db70 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.245+0000 7f51d4a00640 1 -- 172.21.15.165:0/4239147397 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c8032810 con 0x7f51d815db70 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 -- 172.21.15.165:0/4239147397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815db70 msgr2=0x7f51d80cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815db70 0x7f51d80cb840 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f51c8009870 tx=0x7f51c802a200 comp rx=0 tx=0).stop 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 -- 172.21.15.165:0/4239147397 shutdown_connections 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815df50 0x7f51d80cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 --2- 172.21.15.165:0/4239147397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815db70 0x7f51d80cb840 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 -- 172.21.15.165:0/4239147397 >> 172.21.15.165:0/4239147397 conn(0x7f51d806abe0 msgr2=0x7f51d806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 -- 172.21.15.165:0/4239147397 shutdown_connections 2024-04-25T10:07:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.246+0000 7f51dd000640 1 -- 172.21.15.165:0/4239147397 wait complete. 2024-04-25T10:07:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.248+0000 7f51dd000640 1 Processor -- start 2024-04-25T10:07:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.248+0000 7f51dd000640 1 -- start start 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51dd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d80cec70 0x7f51d8153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51dd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815db70 0x7f51d8160880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51dd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 0x7f51d81547d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51dd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8162120 con 0x7f51d815df50 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51dd000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8161e40 con 0x7f51d80cec70 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51dd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8162640 con 0x7f51d815db70 2024-04-25T10:07:24.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 0x7f51d81547d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 0x7f51d81547d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41598/0 (socket says 172.21.15.165:41598) 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 -- 172.21.15.165:0/3597373493 learned_addr learned my addr 172.21.15.165:0/3597373493 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d5e00640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d80cec70 0x7f51d8153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 -- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815db70 msgr2=0x7f51d8160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815db70 0x7f51d8160880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 -- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d80cec70 msgr2=0x7f51d8153c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d80cec70 0x7f51d8153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.249+0000 7f51d6800640 1 -- 172.21.15.165:0/3597373493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8154180 con 0x7f51d815df50 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.250+0000 7f51d6800640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 0x7f51d81547d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c410 tx=0x7f51d000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.250+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011790 con 0x7f51d815df50 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.250+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d0017070 con 0x7f51d815df50 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.250+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d0011bb0 con 0x7f51d815df50 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.250+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d8152ba0 con 0x7f51d815df50 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.250+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d8153160 con 0x7f51d815df50 2024-04-25T10:07:24.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.251+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d8073990 con 0x7f51d815df50 2024-04-25T10:07:24.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.252+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f51d001c460 con 0x7f51d815df50 2024-04-25T10:07:24.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.253+0000 7f51ce000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.253+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f51d0089d20 con 0x7f51d815df50 2024-04-25T10:07:24.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.253+0000 7f51d5e00640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.254+0000 7f51d5e00640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f51c8034000 tx=0x7f51c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.257+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f51d0021020 con 0x7f51d815df50 2024-04-25T10:07:24.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.458+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51d80adad0 con 0x7f51d815df50 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.459+0000 7f51ce000640 1 -- 172.21.15.165:0/3597373493 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+13200 (secure 0 0 0) 0x7f51d0058c30 con 0x7f51d815df50 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:24.465 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":64,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:23.505593+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51b806b500 msgr2=0x7f51b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51b806b500 0x7f51b806d9c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f51c8034000 tx=0x7f51c802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 msgr2=0x7f51d81547d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 0x7f51d81547d0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f51d000c410 tx=0x7f51d000c8d0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 shutdown_connections 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51b806b500 0x7f51b806d9c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51d815df50 0x7f51d81547d0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51d815db70 0x7f51d8160880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 --2- 172.21.15.165:0/3597373493 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51d80cec70 0x7f51d8153c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 >> 172.21.15.165:0/3597373493 conn(0x7f51d806abe0 msgr2=0x7f51d80cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 shutdown_connections 2024-04-25T10:07:24.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.462+0000 7f51dd000640 1 -- 172.21.15.165:0/3597373493 wait complete. 2024-04-25T10:07:24.485 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:24.486 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:24.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.688+0000 7f841a000640 1 Processor -- start 2024-04-25T10:07:24.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.689+0000 7f841a000640 1 -- start start 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.690+0000 7f841a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f841415dd30 0x7f84140cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.690+0000 7f841a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f841406bff0 0x7f84140b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.690+0000 7f841a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84140b5970 0x7f8414161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.690+0000 7f841a000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841406d000 con 0x7f841406bff0 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.690+0000 7f841a000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f841415e110 con 0x7f841415dd30 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.690+0000 7f841a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140b2fd0 con 0x7f84140b5970 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f841406bff0 0x7f84140b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f841406bff0 0x7f84140b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41604/0 (socket says 172.21.15.165:41604) 2024-04-25T10:07:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 -- 172.21.15.165:0/3556339179 learned_addr learned my addr 172.21.15.165:0/3556339179 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8418c00640 1 --2- 172.21.15.165:0/3556339179 >> v2:172.21.15.172:3300/0 conn(0x7f841415dd30 0x7f84140cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8419600640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84140b5970 0x7f8414161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 -- 172.21.15.165:0/3556339179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84140b5970 msgr2=0x7f8414161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84140b5970 0x7f8414161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 -- 172.21.15.165:0/3556339179 >> v2:172.21.15.172:3300/0 conn(0x7f841415dd30 msgr2=0x7f84140cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 --2- 172.21.15.165:0/3556339179 >> v2:172.21.15.172:3300/0 conn(0x7f841415dd30 0x7f84140cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8413e00640 1 -- 172.21.15.165:0/3556339179 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f841406c480 con 0x7f841406bff0 2024-04-25T10:07:24.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.691+0000 7f8419600640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84140b5970 0x7f8414161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.692+0000 7f8413e00640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f841406bff0 0x7f84140b5430 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f841408bfe0 tx=0x7f83fc02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02487a81c2848 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.692+0000 7f8413400640 1 -- 172.21.15.165:0/3556339179 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc037070 con 0x7f841406bff0 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f8413400640 1 -- 172.21.15.165:0/3556339179 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83fc032440 con 0x7f841406bff0 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f8413400640 1 -- 172.21.15.165:0/3556339179 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83fc032810 con 0x7f841406bff0 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 -- 172.21.15.165:0/3556339179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f841406bff0 msgr2=0x7f84140b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f841406bff0 0x7f84140b5430 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f841408bfe0 tx=0x7f83fc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 -- 172.21.15.165:0/3556339179 shutdown_connections 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84140b5970 0x7f8414161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 --2- 172.21.15.165:0/3556339179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f841406bff0 0x7f84140b5430 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 --2- 172.21.15.165:0/3556339179 >> v2:172.21.15.172:3300/0 conn(0x7f841415dd30 0x7f84140cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 -- 172.21.15.165:0/3556339179 >> 172.21.15.165:0/3556339179 conn(0x7f84140d0610 msgr2=0x7f84140d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 -- 172.21.15.165:0/3556339179 shutdown_connections 2024-04-25T10:07:24.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.693+0000 7f841a000640 1 -- 172.21.15.165:0/3556339179 wait complete. 2024-04-25T10:07:24.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.695+0000 7f841a000640 1 Processor -- start 2024-04-25T10:07:24.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.695+0000 7f841a000640 1 -- start start 2024-04-25T10:07:24.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f841a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 0x7f84141f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f841a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841415dd30 0x7f84140acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f841a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84140ccb10 0x7f84140ccfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f841a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841406cd20 con 0x7f84140b5970 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f841a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f841406ca40 con 0x7f84140ccb10 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f841a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140cf110 con 0x7f841415dd30 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f8418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 0x7f84141f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f8418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 0x7f84141f3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41606/0 (socket says 172.21.15.165:41606) 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.696+0000 7f8418c00640 1 -- 172.21.15.165:0/345087009 learned_addr learned my addr 172.21.15.165:0/345087009 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8418c00640 1 -- 172.21.15.165:0/345087009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841415dd30 msgr2=0x7f84140acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8418c00640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841415dd30 0x7f84140acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8418c00640 1 -- 172.21.15.165:0/345087009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84140ccb10 msgr2=0x7f84140ccfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8418c00640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84140ccb10 0x7f84140ccfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8418c00640 1 -- 172.21.15.165:0/345087009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140aa4c0 con 0x7f84140b5970 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8418c00640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 0x7f84141f3770 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f840400d5f0 tx=0x7f840400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84040125f0 con 0x7f84140b5970 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8404019070 con 0x7f84140b5970 2024-04-25T10:07:24.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.697+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84140aa7a0 con 0x7f84140b5970 2024-04-25T10:07:24.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.698+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84040129d0 con 0x7f84140b5970 2024-04-25T10:07:24.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.698+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84140aad60 con 0x7f84140b5970 2024-04-25T10:07:24.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.699+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8414073990 con 0x7f84140b5970 2024-04-25T10:07:24.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.700+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8404012b70 con 0x7f84140b5970 2024-04-25T10:07:24.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.700+0000 7f8410c00640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f006b520 0x7f83f006d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:24.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.701+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f840408a350 con 0x7f84140b5970 2024-04-25T10:07:24.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.701+0000 7f8413e00640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f006b520 0x7f83f006d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:24.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.702+0000 7f8413e00640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f006b520 0x7f83f006d9e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f83fc034000 tx=0x7f83fc02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.705+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8404059470 con 0x7f84140b5970 2024-04-25T10:07:24.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.900+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84140d16c0 con 0x7f84140b5970 2024-04-25T10:07:24.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.901+0000 7f8410c00640 1 -- 172.21.15.165:0/345087009 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v64) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8404058d70 con 0x7f84140b5970 2024-04-25T10:07:24.907 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:07:24.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f006b520 msgr2=0x7f83f006d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f006b520 0x7f83f006d9e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f83fc034000 tx=0x7f83fc02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:24.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 msgr2=0x7f84141f3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:24.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 0x7f84141f3770 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f840400d5f0 tx=0x7f840400dab0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 shutdown_connections 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f006b520 0x7f83f006d9e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f84140ccb10 0x7f84140ccfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f841415dd30 0x7f84140acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 --2- 172.21.15.165:0/345087009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84140b5970 0x7f84141f3770 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 >> 172.21.15.165:0/345087009 conn(0x7f84140d0610 msgr2=0x7f84140b6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 shutdown_connections 2024-04-25T10:07:24.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:24.904+0000 7f841a000640 1 -- 172.21.15.165:0/345087009 wait complete. 2024-04-25T10:07:24.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:07:25.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.129+0000 7f2c7fe00640 1 Processor -- start 2024-04-25T10:07:25.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.129+0000 7f2c7fe00640 1 -- start start 2024-04-25T10:07:25.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2c80144ae0 0x7f2c800ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c8006d270 0x7f2c8006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8015c520 con 0x7f2c8015bcf0 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7fe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8015c800 con 0x7f2c80144ae0 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c8015cae0 con 0x7f2c8006d270 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2c80144ae0 0x7f2c800ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2c80144ae0 0x7f2c800ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41616/0 (socket says 172.21.15.165:41616) 2024-04-25T10:07:25.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.131+0000 7f2c7ea00640 1 -- 172.21.15.165:0/2937141840 learned_addr learned my addr 172.21.15.165:0/2937141840 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:25.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.132+0000 7f2c7f400640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c8006d270 0x7f2c8006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.132+0000 7f2c7ea00640 1 -- 172.21.15.165:0/2937141840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c8006d270 msgr2=0x7f2c8006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:25.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.132+0000 7f2c7ea00640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c8006d270 0x7f2c8006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.132+0000 7f2c7ea00640 1 -- 172.21.15.165:0/2937141840 >> v1:172.21.15.172:6789/0 conn(0x7f2c8015bcf0 legacy=0x7f2c8015f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:25.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.132+0000 7f2c7ea00640 1 -- 172.21.15.165:0/2937141840 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8015c180 con 0x7f2c80144ae0 2024-04-25T10:07:25.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.132+0000 7f2c7f400640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c8006d270 0x7f2c8006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:25.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.133+0000 7f2c7ea00640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c80144ae0 0x7f2c800ccb00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c7002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f596d0b7aa6a4b3d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.133+0000 7f2c7e000640 1 -- 172.21.15.165:0/2937141840 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70037070 con 0x7f2c80144ae0 2024-04-25T10:07:25.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.133+0000 7f2c7e000640 1 -- 172.21.15.165:0/2937141840 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c70032440 con 0x7f2c80144ae0 2024-04-25T10:07:25.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.133+0000 7f2c7e000640 1 -- 172.21.15.165:0/2937141840 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70032810 con 0x7f2c80144ae0 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 -- 172.21.15.165:0/2937141840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c80144ae0 msgr2=0x7f2c800ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c80144ae0 0x7f2c800ccb00 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c7002a200 comp rx=0 tx=0).stop 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 -- 172.21.15.165:0/2937141840 shutdown_connections 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c8006d270 0x7f2c8006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/2937141840 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c80144ae0 0x7f2c800ccb00 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 -- 172.21.15.165:0/2937141840 >> 172.21.15.165:0/2937141840 conn(0x7f2c80061c30 msgr2=0x7f2c8006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 -- 172.21.15.165:0/2937141840 shutdown_connections 2024-04-25T10:07:25.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.134+0000 7f2c7fe00640 1 -- 172.21.15.165:0/2937141840 wait complete. 2024-04-25T10:07:25.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.136+0000 7f2c7fe00640 1 Processor -- start 2024-04-25T10:07:25.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.136+0000 7f2c7fe00640 1 -- start start 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 0x7f2c8013f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c80144ae0 0x7f2c800b34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c8015bcf0 0x7f2c800b3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c800712c0 con 0x7f2c8006d270 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80071140 con 0x7f2c8015bcf0 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c80071440 con 0x7f2c80144ae0 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 0x7f2c8013f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.137+0000 7f2c7ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 0x7f2c8013f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41622/0 (socket says 172.21.15.165:41622) 2024-04-25T10:07:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 -- 172.21.15.165:0/597964395 learned_addr learned my addr 172.21.15.165:0/597964395 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7f400640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c8015bcf0 0x7f2c800b3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 -- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c80144ae0 msgr2=0x7f2c800b34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c80144ae0 0x7f2c800b34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 -- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c8015bcf0 msgr2=0x7f2c800b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c8015bcf0 0x7f2c800b3a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 -- 172.21.15.165:0/597964395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8013a660 con 0x7f2c8006d270 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c7ea00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 0x7f2c8013f1a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70037040 con 0x7f2c8006d270 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c70042070 con 0x7f2c8006d270 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.138+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70030bb0 con 0x7f2c8006d270 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.139+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c800ac9f0 con 0x7f2c8006d270 2024-04-25T10:07:25.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.139+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c800acfb0 con 0x7f2c8006d270 2024-04-25T10:07:25.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.140+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8015bb10 con 0x7f2c8006d270 2024-04-25T10:07:25.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.141+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c7002ad10 con 0x7f2c8006d270 2024-04-25T10:07:25.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.142+0000 7f2c77e00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.142+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2c700aa980 con 0x7f2c8006d270 2024-04-25T10:07:25.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.142+0000 7f2c76000640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.143+0000 7f2c76000640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2c8015b110 tx=0x7f2c68009230 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.146+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c8015bb10 con 0x7f2c8006d270 2024-04-25T10:07:25.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.345+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f2c80073990 con 0x7f2c8006d270 2024-04-25T10:07:25.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.345+0000 7f2c77e00640 1 -- 172.21.15.165:0/597964395 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v64) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2c8013a660 con 0x7f2c8006d270 2024-04-25T10:07:25.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:25.352 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:07:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:07:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:07:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:07:25.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:07:25.352 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:07:25.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c5c06b4d0 msgr2=0x7f2c5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f2c8015b110 tx=0x7f2c68009230 comp rx=0 tx=0).stop 2024-04-25T10:07:25.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 msgr2=0x7f2c8013f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 0x7f2c8013f1a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f2c70009870 tx=0x7f2c700046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 shutdown_connections 2024-04-25T10:07:25.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c5c06b4d0 0x7f2c5c06d990 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c8015bcf0 0x7f2c800b3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c80144ae0 0x7f2c800b34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 --2- 172.21.15.165:0/597964395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c8006d270 0x7f2c8013f1a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 >> 172.21.15.165:0/597964395 conn(0x7f2c80061c30 msgr2=0x7f2c8015cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 shutdown_connections 2024-04-25T10:07:25.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.348+0000 7f2c7fe00640 1 -- 172.21.15.165:0/597964395 wait complete. 2024-04-25T10:07:25.373 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:07:25.373 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:07:25.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.576+0000 7f0e34600640 1 Processor -- start 2024-04-25T10:07:25.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.576+0000 7f0e34600640 1 -- start start 2024-04-25T10:07:25.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.577+0000 7f0e34600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0e2c06ec50 0x7f0e2c071bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.577+0000 7f0e34600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c0cfad0 con 0x7f0e2c0cf640 2024-04-25T10:07:25.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.577+0000 7f0e34600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c0cfd80 con 0x7f0e2c15b710 2024-04-25T10:07:25.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.577+0000 7f0e34600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2c15e490 con 0x7f0e2c06ec50 2024-04-25T10:07:25.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.578+0000 7f0e32600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0e2c06ec50 0x7f0e2c071bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.578+0000 7f0e31200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0e2c0cf640 0x7f0e2c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42228/0 (socket says 172.21.15.165:42228) 2024-04-25T10:07:25.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.578+0000 7f0e31200640 1 -- 172.21.15.165:0/2355157947 learned_addr learned my addr 172.21.15.165:0/2355157947 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:25.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.578+0000 7f0e32600640 1 -- 172.21.15.165:0/2355157947 >> v1:172.21.15.172:6789/0 conn(0x7f0e2c15b710 legacy=0x7f0e2c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:25.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.578+0000 7f0e32600640 1 -- 172.21.15.165:0/2355157947 >> v1:172.21.15.165:6789/0 conn(0x7f0e2c0cf640 legacy=0x7f0e2c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.578+0000 7f0e32600640 1 -- 172.21.15.165:0/2355157947 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0d0060 con 0x7f0e2c06ec50 2024-04-25T10:07:25.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.579+0000 7f0e32600640 1 --2- 172.21.15.165:0/2355157947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c071bc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009d90 tx=0x7f0e2402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787cfe174a293d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.579+0000 7f0e2be00640 1 -- 172.21.15.165:0/2355157947 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007740 con 0x7f0e2c06ec50 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e2be00640 1 -- 172.21.15.165:0/2355157947 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e240078e0 con 0x7f0e2c06ec50 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e2be00640 1 -- 172.21.15.165:0/2355157947 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e24007cb0 con 0x7f0e2c06ec50 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 -- 172.21.15.165:0/2355157947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 msgr2=0x7f0e2c071bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 --2- 172.21.15.165:0/2355157947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c071bc0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009d90 tx=0x7f0e2402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 -- 172.21.15.165:0/2355157947 shutdown_connections 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 --2- 172.21.15.165:0/2355157947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c071bc0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 -- 172.21.15.165:0/2355157947 >> 172.21.15.165:0/2355157947 conn(0x7f0e2c06abe0 msgr2=0x7f0e2c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 -- 172.21.15.165:0/2355157947 shutdown_connections 2024-04-25T10:07:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.580+0000 7f0e34600640 1 -- 172.21.15.165:0/2355157947 wait complete. 2024-04-25T10:07:25.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.582+0000 7f0e34600640 1 Processor -- start 2024-04-25T10:07:25.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.582+0000 7f0e34600640 1 -- start start 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e34600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e34600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c0cf640 0x7f0e2c14f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e34600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c15b710 0x7f0e2c15f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e34600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15baf0 con 0x7f0e2c06ec50 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e34600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c06f0a0 con 0x7f0e2c0cf640 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e34600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c15bc70 con 0x7f0e2c15b710 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e31c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e31c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41632/0 (socket says 172.21.15.165:41632) 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.583+0000 7f0e31c00640 1 -- 172.21.15.165:0/1781329218 learned_addr learned my addr 172.21.15.165:0/1781329218 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e31c00640 1 -- 172.21.15.165:0/1781329218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c15b710 msgr2=0x7f0e2c15f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e31c00640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c15b710 0x7f0e2c15f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e31c00640 1 -- 172.21.15.165:0/1781329218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c0cf640 msgr2=0x7f0e2c14f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e31c00640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c0cf640 0x7f0e2c14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e31c00640 1 -- 172.21.15.165:0/1781329218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c154b70 con 0x7f0e2c06ec50 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e31c00640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c154630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b3f0 tx=0x7f0e2000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20015020 con 0x7f0e2c06ec50 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e20011830 con 0x7f0e2c06ec50 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20011c20 con 0x7f0e2c06ec50 2024-04-25T10:07:25.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c1603f0 con 0x7f0e2c06ec50 2024-04-25T10:07:25.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.584+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c153b50 con 0x7f0e2c06ec50 2024-04-25T10:07:25.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.585+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c073a00 con 0x7f0e2c06ec50 2024-04-25T10:07:25.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.586+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e20018020 con 0x7f0e2c06ec50 2024-04-25T10:07:25.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.587+0000 7f0e29600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e0c06b4d0 0x7f0e0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:25.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.587+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0e20089d50 con 0x7f0e2c06ec50 2024-04-25T10:07:25.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.588+0000 7f0e31200640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e0c06b4d0 0x7f0e0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:25.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.588+0000 7f0e31200640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e0c06b4d0 0x7f0e0c06d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0e18005880 tx=0x7f0e18007aa0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:25.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.592+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e2001f030 con 0x7f0e2c06ec50 2024-04-25T10:07:25.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:25.787+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0e2c06e360 con 0x7f0e2c06ec50 2024-04-25T10:07:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.528+0000 7f0e29600640 1 -- 172.21.15.165:0/1781329218 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e200172c0 con 0x7f0e2c06ec50 2024-04-25T10:07:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.531+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e0c06b4d0 msgr2=0x7f0e0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.531+0000 7f0e34600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e0c06b4d0 0x7f0e0c06d990 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f0e18005880 tx=0x7f0e18007aa0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.531+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 msgr2=0x7f0e2c154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.531+0000 7f0e34600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c154630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000b3f0 tx=0x7f0e2000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 shutdown_connections 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e0c06b4d0 0x7f0e0c06d990 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c15b710 0x7f0e2c15f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c0cf640 0x7f0e2c14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 --2- 172.21.15.165:0/1781329218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c06ec50 0x7f0e2c154630 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 >> 172.21.15.165:0/1781329218 conn(0x7f0e2c06abe0 msgr2=0x7f0e2c0ad610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 shutdown_connections 2024-04-25T10:07:26.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.532+0000 7f0e34600640 1 -- 172.21.15.165:0/1781329218 wait complete. 2024-04-25T10:07:26.563 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:07:26.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.753+0000 7fadad600640 1 Processor -- start 2024-04-25T10:07:26.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.754+0000 7fadad600640 1 -- start start 2024-04-25T10:07:26.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fadad600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fada80cf640 0x7fada806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fadad600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada80ac8b0 0x7fada80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fadad600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fada80cfad0 con 0x7fada815b710 2024-04-25T10:07:26.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fadad600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fada806f190 con 0x7fada80cf640 2024-04-25T10:07:26.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fadad600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada815e490 con 0x7fada80ac8b0 2024-04-25T10:07:26.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fada6000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fada80cf640 0x7fada806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fada6a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fada815b710 0x7fada815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42230/0 (socket says 172.21.15.165:42230) 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fada6000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fada80cf640 0x7fada806ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41640/0 (socket says 172.21.15.165:41640) 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.755+0000 7fada6a00640 1 -- 172.21.15.165:0/1494090928 learned_addr learned my addr 172.21.15.165:0/1494090928 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.756+0000 7fada6000640 1 -- 172.21.15.165:0/1494090928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada80ac8b0 msgr2=0x7fada80accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.756+0000 7fada6000640 1 --2- 172.21.15.165:0/1494090928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada80ac8b0 0x7fada80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.756+0000 7fada6000640 1 -- 172.21.15.165:0/1494090928 >> v1:172.21.15.165:6789/0 conn(0x7fada815b710 legacy=0x7fada815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.756+0000 7fada6000640 1 -- 172.21.15.165:0/1494090928 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada80cfd80 con 0x7fada80cf640 2024-04-25T10:07:26.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.757+0000 7fada6000640 1 --2- 172.21.15.165:0/1494090928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada806ec50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fad900097a0 tx=0x7fad9002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2318861231b29e80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.757+0000 7fada5600640 1 -- 172.21.15.165:0/1494090928 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90037070 con 0x7fada80cf640 2024-04-25T10:07:26.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.757+0000 7fada5600640 1 -- 172.21.15.165:0/1494090928 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90032440 con 0x7fada80cf640 2024-04-25T10:07:26.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.757+0000 7fada5600640 1 -- 172.21.15.165:0/1494090928 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90032810 con 0x7fada80cf640 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 -- 172.21.15.165:0/1494090928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 msgr2=0x7fada806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 --2- 172.21.15.165:0/1494090928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada806ec50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fad900097a0 tx=0x7fad9002a180 comp rx=0 tx=0).stop 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 -- 172.21.15.165:0/1494090928 shutdown_connections 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 --2- 172.21.15.165:0/1494090928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada80ac8b0 0x7fada80accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 --2- 172.21.15.165:0/1494090928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada806ec50 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 -- 172.21.15.165:0/1494090928 >> 172.21.15.165:0/1494090928 conn(0x7fada806abe0 msgr2=0x7fada806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 -- 172.21.15.165:0/1494090928 shutdown_connections 2024-04-25T10:07:26.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.758+0000 7fadad600640 1 -- 172.21.15.165:0/1494090928 wait complete. 2024-04-25T10:07:26.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.760+0000 7fadad600640 1 Processor -- start 2024-04-25T10:07:26.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.760+0000 7fadad600640 1 -- start start 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fadad600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80ac8b0 0x7fada81e9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fadad600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada8147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fadad600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada815b710 0x7fada81f7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fadad600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada815baf0 con 0x7fada80cf640 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fadad600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada80d0340 con 0x7fada80ac8b0 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fadad600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fada815bc70 con 0x7fada815b710 2024-04-25T10:07:26.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fada6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada8147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fada6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada8147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41646/0 (socket says 172.21.15.165:41646) 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.761+0000 7fada6000640 1 -- 172.21.15.165:0/3235150319 learned_addr learned my addr 172.21.15.165:0/3235150319 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada7400640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada815b710 0x7fada81f7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6a00640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80ac8b0 0x7fada81e9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6000640 1 -- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada815b710 msgr2=0x7fada81f7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6000640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada815b710 0x7fada81f7410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6000640 1 -- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80ac8b0 msgr2=0x7fada81e9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6000640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80ac8b0 0x7fada81e9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6000640 1 -- 172.21.15.165:0/3235150319 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada81e6e40 con 0x7fada80cf640 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fada6000640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada8147220 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fad900098c0 tx=0x7fad900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90037040 con 0x7fada80cf640 2024-04-25T10:07:26.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad90042070 con 0x7fada80cf640 2024-04-25T10:07:26.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad90030a70 con 0x7fada80cf640 2024-04-25T10:07:26.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.762+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada81e7ea0 con 0x7fada80cf640 2024-04-25T10:07:26.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.763+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada81e73c0 con 0x7fada80cf640 2024-04-25T10:07:26.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.764+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada8068610 con 0x7fada80cf640 2024-04-25T10:07:26.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.765+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad90030c10 con 0x7fada80cf640 2024-04-25T10:07:26.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.766+0000 7fad9ea00640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad8406b4d0 0x7fad8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:26.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.766+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fad900aacb0 con 0x7fada80cf640 2024-04-25T10:07:26.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.766+0000 7fada6a00640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad8406b4d0 0x7fad8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:26.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.767+0000 7fada6a00640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad8406b4d0 0x7fad8406d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fada81e8260 tx=0x7fad98009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:26.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.771+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fada8068610 con 0x7fada80cf640 2024-04-25T10:07:26.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:26.966+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fada8073a00 con 0x7fada80cf640 2024-04-25T10:07:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.545+0000 7fad9ea00640 1 -- 172.21.15.165:0/3235150319 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v66) v1 ==== 170+0+0 (secure 0 0 0) 0x7fad90045360 con 0x7fada80cf640 2024-04-25T10:07:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad8406b4d0 msgr2=0x7fad8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad8406b4d0 0x7fad8406d990 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fada81e8260 tx=0x7fad98009040 comp rx=0 tx=0).stop 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 msgr2=0x7fada8147220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada8147220 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fad900098c0 tx=0x7fad900046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 shutdown_connections 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad8406b4d0 0x7fad8406d990 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fada815b710 0x7fada81f7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fada80cf640 0x7fada8147220 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 --2- 172.21.15.165:0/3235150319 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fada80ac8b0 0x7fada81e9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 >> 172.21.15.165:0/3235150319 conn(0x7fada806abe0 msgr2=0x7fada806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 shutdown_connections 2024-04-25T10:07:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.549+0000 7fadad600640 1 -- 172.21.15.165:0/3235150319 wait complete. 2024-04-25T10:07:27.590 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:07:27.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:27.742 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:27.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.757+0000 7ff381200640 1 Processor -- start 2024-04-25T10:07:27.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.758+0000 7ff381200640 1 -- start start 2024-04-25T10:07:27.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff381200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff37c0ce230 0x7ff37c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff381200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37c072280 con 0x7ff37c0cc4f0 2024-04-25T10:07:27.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff381200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff37c0714c0 con 0x7ff37c15b270 2024-04-25T10:07:27.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff381200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff37c071270 con 0x7ff37c0ce230 2024-04-25T10:07:27.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff371400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff37c0ce230 0x7ff37c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff379e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff37c0cc4f0 0x7ff37c0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37778/0 (socket says 172.21.15.165:37778) 2024-04-25T10:07:27.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.759+0000 7ff379e00640 1 -- 172.21.15.165:0/1475389828 learned_addr learned my addr 172.21.15.165:0/1475389828 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:27.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.760+0000 7ff379400640 1 -- 172.21.15.165:0/1475389828 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1581386827 0 0) 0x7ff37c072280 con 0x7ff37c0cc4f0 2024-04-25T10:07:27.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.760+0000 7ff379400640 1 -- 172.21.15.165:0/1475389828 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff360005730 con 0x7ff37c0cc4f0 2024-04-25T10:07:27.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.760+0000 7ff371400640 1 -- 172.21.15.165:0/1475389828 >> v1:172.21.15.172:6789/0 conn(0x7ff37c15b270 legacy=0x7ff37c15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.760+0000 7ff371400640 1 -- 172.21.15.165:0/1475389828 >> v1:172.21.15.165:6789/0 conn(0x7ff37c0cc4f0 legacy=0x7ff37c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.760+0000 7ff371400640 1 -- 172.21.15.165:0/1475389828 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c0717a0 con 0x7ff37c0ce230 2024-04-25T10:07:27.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.761+0000 7ff371400640 1 --2- 172.21.15.165:0/1475389828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c0ce230 0x7ff37c0cc8d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff3640097a0 tx=0x7ff36402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd8bc5eb50b28ec9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.761+0000 7ff379400640 1 -- 172.21.15.165:0/1475389828 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364037070 con 0x7ff37c0ce230 2024-04-25T10:07:27.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff379400640 1 -- 172.21.15.165:0/1475389828 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff364032440 con 0x7ff37c0ce230 2024-04-25T10:07:27.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff379400640 1 -- 172.21.15.165:0/1475389828 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff364032810 con 0x7ff37c0ce230 2024-04-25T10:07:27.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 -- 172.21.15.165:0/1475389828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c0ce230 msgr2=0x7ff37c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 --2- 172.21.15.165:0/1475389828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c0ce230 0x7ff37c0cc8d0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7ff3640097a0 tx=0x7ff36402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 -- 172.21.15.165:0/1475389828 shutdown_connections 2024-04-25T10:07:27.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 --2- 172.21.15.165:0/1475389828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c0ce230 0x7ff37c0cc8d0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 -- 172.21.15.165:0/1475389828 >> 172.21.15.165:0/1475389828 conn(0x7ff37c06abe0 msgr2=0x7ff37c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:27.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 -- 172.21.15.165:0/1475389828 shutdown_connections 2024-04-25T10:07:27.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.762+0000 7ff381200640 1 -- 172.21.15.165:0/1475389828 wait complete. 2024-04-25T10:07:27.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.764+0000 7ff381200640 1 Processor -- start 2024-04-25T10:07:27.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.764+0000 7ff381200640 1 -- start start 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.765+0000 7ff381200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff37c0cc4f0 0x7ff37c1f6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.765+0000 7ff381200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff37c0ce230 0x7ff37c0ae8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.765+0000 7ff381200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 0x7ff37c0aee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.765+0000 7ff381200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c06e8a0 con 0x7ff37c15b270 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.765+0000 7ff381200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c071d60 con 0x7ff37c0cc4f0 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.765+0000 7ff381200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c0ae500 con 0x7ff37c0ce230 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 0x7ff37c0aee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff379e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff37c0cc4f0 0x7ff37c1f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 0x7ff37c0aee00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36752/0 (socket says 172.21.15.165:36752) 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 -- 172.21.15.165:0/2276284144 learned_addr learned my addr 172.21.15.165:0/2276284144 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff371400640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff37c0ce230 0x7ff37c0ae8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 -- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff37c0ce230 msgr2=0x7ff37c0ae8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff37c0ce230 0x7ff37c0ae8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 -- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff37c0cc4f0 msgr2=0x7ff37c1f6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:27.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff37c0cc4f0 0x7ff37c1f6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:27.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.766+0000 7ff37a800640 1 -- 172.21.15.165:0/2276284144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff37c1496b0 con 0x7ff37c15b270 2024-04-25T10:07:27.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.767+0000 7ff37a800640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 0x7ff37c0aee00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c7a0 tx=0x7ff37400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.767+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff374003b70 con 0x7ff37c15b270 2024-04-25T10:07:27.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.767+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff374003d30 con 0x7ff37c15b270 2024-04-25T10:07:27.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.767+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37400aa90 con 0x7ff37c15b270 2024-04-25T10:07:27.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.767+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff37c0af410 con 0x7ff37c15b270 2024-04-25T10:07:27.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.767+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c0af9d0 con 0x7ff37c15b270 2024-04-25T10:07:27.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.769+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff37c0afd10 con 0x7ff37c15b270 2024-04-25T10:07:27.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.769+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff37400ac30 con 0x7ff37c15b270 2024-04-25T10:07:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.770+0000 7ff372a00640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff35806b4d0 0x7ff35806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.770+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6001+0+0 (secure 0 0 0) 0x7ff37408a150 con 0x7ff37c15b270 2024-04-25T10:07:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.771+0000 7ff379e00640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff35806b4d0 0x7ff35806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:27.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.771+0000 7ff379e00640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff35806b4d0 0x7ff35806d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff36c007fd0 tx=0x7ff36c007220 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:27.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:27.776+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff37c0afd10 con 0x7ff37c15b270 2024-04-25T10:07:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.011+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff37c15a9a0 con 0x7ff37c15b270 2024-04-25T10:07:28.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.572+0000 7ff372a00640 1 -- 172.21.15.165:0/2276284144 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v37) v1 ==== 173+0+0 (secure 0 0 0) 0x7ff374058f30 con 0x7ff37c15b270 2024-04-25T10:07:28.579 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 9 and data pool 10 2024-04-25T10:07:28.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.581+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff35806b4d0 msgr2=0x7ff35806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.581+0000 7ff381200640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff35806b4d0 0x7ff35806d990 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff36c007fd0 tx=0x7ff36c007220 comp rx=0 tx=0).stop 2024-04-25T10:07:28.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.581+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 msgr2=0x7ff37c0aee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.581+0000 7ff381200640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 0x7ff37c0aee00 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7ff37400c7a0 tx=0x7ff37400cc60 comp rx=0 tx=0).stop 2024-04-25T10:07:28.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.582+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 shutdown_connections 2024-04-25T10:07:28.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.582+0000 7ff381200640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff35806b4d0 0x7ff35806d990 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.582+0000 7ff381200640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff37c15b270 0x7ff37c0aee00 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.582+0000 7ff381200640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff37c0ce230 0x7ff37c0ae8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.582+0000 7ff381200640 1 --2- 172.21.15.165:0/2276284144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff37c0cc4f0 0x7ff37c1f6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.582+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 >> 172.21.15.165:0/2276284144 conn(0x7ff37c06abe0 msgr2=0x7ff37c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.584+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 shutdown_connections 2024-04-25T10:07:28.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.584+0000 7ff381200640 1 -- 172.21.15.165:0/2276284144 wait complete. 2024-04-25T10:07:28.621 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:28.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.789+0000 7f095ca00640 1 Processor -- start 2024-04-25T10:07:28.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.789+0000 7f095ca00640 1 -- start start 2024-04-25T10:07:28.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f095ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f095815db90 0x7f09580b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f095ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f09580d06c0 0x7f09580d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f095ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0958161df0 con 0x7f09581615a0 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f095ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580d1f50 con 0x7f09580d06c0 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f095ca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09581620d0 con 0x7f095815db90 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f0957e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f09580d06c0 0x7f09580d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f0957e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f09580d06c0 0x7f09580d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36754/0 (socket says 172.21.15.165:36754) 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.791+0000 7f0957e00640 1 -- 172.21.15.165:0/2794091423 learned_addr learned my addr 172.21.15.165:0/2794091423 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.792+0000 7f0957400640 1 --2- 172.21.15.165:0/2794091423 >> v2:172.21.15.172:3300/0 conn(0x7f095815db90 0x7f09580b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.792+0000 7f0957e00640 1 -- 172.21.15.165:0/2794091423 >> v2:172.21.15.172:3300/0 conn(0x7f095815db90 msgr2=0x7f09580b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.792+0000 7f0957e00640 1 --2- 172.21.15.165:0/2794091423 >> v2:172.21.15.172:3300/0 conn(0x7f095815db90 0x7f09580b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.792+0000 7f0957e00640 1 -- 172.21.15.165:0/2794091423 >> v1:172.21.15.172:6789/0 conn(0x7f09581615a0 legacy=0x7f09580cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.792+0000 7f0957e00640 1 -- 172.21.15.165:0/2794091423 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09581623b0 con 0x7f09580d06c0 2024-04-25T10:07:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.792+0000 7f0957400640 1 --2- 172.21.15.165:0/2794091423 >> v2:172.21.15.172:3300/0 conn(0x7f095815db90 0x7f09580b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.793+0000 7f0957e00640 1 --2- 172.21.15.165:0/2794091423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09580d06c0 0x7f09580d23e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f095806ceb0 tx=0x7f095002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e616f481c1c4b015 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.793+0000 7f0956000640 1 -- 172.21.15.165:0/2794091423 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950037070 con 0x7f09580d06c0 2024-04-25T10:07:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.793+0000 7f0956000640 1 -- 172.21.15.165:0/2794091423 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950032440 con 0x7f09580d06c0 2024-04-25T10:07:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.793+0000 7f0956000640 1 -- 172.21.15.165:0/2794091423 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950032810 con 0x7f09580d06c0 2024-04-25T10:07:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 -- 172.21.15.165:0/2794091423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09580d06c0 msgr2=0x7f09580d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 --2- 172.21.15.165:0/2794091423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09580d06c0 0x7f09580d23e0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f095806ceb0 tx=0x7f095002a230 comp rx=0 tx=0).stop 2024-04-25T10:07:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 -- 172.21.15.165:0/2794091423 shutdown_connections 2024-04-25T10:07:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 --2- 172.21.15.165:0/2794091423 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09580d06c0 0x7f09580d23e0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 --2- 172.21.15.165:0/2794091423 >> v2:172.21.15.172:3300/0 conn(0x7f095815db90 0x7f09580b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 -- 172.21.15.165:0/2794091423 >> 172.21.15.165:0/2794091423 conn(0x7f09580b52b0 msgr2=0x7f09580b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 -- 172.21.15.165:0/2794091423 shutdown_connections 2024-04-25T10:07:28.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.794+0000 7f095ca00640 1 -- 172.21.15.165:0/2794091423 wait complete. 2024-04-25T10:07:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.796+0000 7f095ca00640 1 Processor -- start 2024-04-25T10:07:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.796+0000 7f095ca00640 1 -- start start 2024-04-25T10:07:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.797+0000 7f095ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09580d06c0 0x7f095814e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.797+0000 7f095ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f095815db90 0x7f095815df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.797+0000 7f095ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 0x7f09580ccaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.797+0000 7f095ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958161ba0 con 0x7f09581615a0 2024-04-25T10:07:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.797+0000 7f095ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0958161a20 con 0x7f095815db90 2024-04-25T10:07:28.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.797+0000 7f095ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580b68d0 con 0x7f09580d06c0 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 0x7f09580ccaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 0x7f09580ccaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36760/0 (socket says 172.21.15.165:36760) 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 -- 172.21.15.165:0/2973538885 learned_addr learned my addr 172.21.15.165:0/2973538885 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957400640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09580d06c0 0x7f095814e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 -- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09580d06c0 msgr2=0x7f095814e7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09580d06c0 0x7f095814e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 -- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f095815db90 msgr2=0x7f095815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f095815db90 0x7f095815df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.798+0000 7f0957e00640 1 -- 172.21.15.165:0/2973538885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095815e6c0 con 0x7f09581615a0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.799+0000 7f0957e00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 0x7f09580ccaa0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f09500090b0 tx=0x7f09500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.799+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950041020 con 0x7f09581615a0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.799+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0950030820 con 0x7f09581615a0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.799+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0950037040 con 0x7f09581615a0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.799+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095815e9a0 con 0x7f09581615a0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.799+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095815ef60 con 0x7f09581615a0 2024-04-25T10:07:28.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.800+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0958073990 con 0x7f09581615a0 2024-04-25T10:07:28.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.801+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0950044020 con 0x7f09581615a0 2024-04-25T10:07:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.803+0000 7f094f400640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f093806b500 0x7f093806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.803+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f09500aac90 con 0x7f09581615a0 2024-04-25T10:07:28.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.803+0000 7f0957400640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f093806b500 0x7f093806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:28.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.804+0000 7f0957400640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f093806b500 0x7f093806d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0948008f80 tx=0x7f09480084d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:28.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:28.807+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f095004b030 con 0x7f09581615a0 2024-04-25T10:07:29.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.005+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09580cdb30 con 0x7f09581615a0 2024-04-25T10:07:29.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.007+0000 7f094f400640 1 -- 172.21.15.165:0/2973538885 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f0950079a90 con 0x7f09581615a0 2024-04-25T10:07:29.013 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:29.013 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":67,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:28.013469+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:29.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f093806b500 msgr2=0x7f093806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f093806b500 0x7f093806d9c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f0948008f80 tx=0x7f09480084d0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 msgr2=0x7f09580ccaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 0x7f09580ccaa0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f09500090b0 tx=0x7f09500046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 shutdown_connections 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f093806b500 0x7f093806d9c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09581615a0 0x7f09580ccaa0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f095815db90 0x7f095815df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 --2- 172.21.15.165:0/2973538885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09580d06c0 0x7f095814e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 >> 172.21.15.165:0/2973538885 conn(0x7f09580b52b0 msgr2=0x7f09580c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 shutdown_connections 2024-04-25T10:07:29.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.010+0000 7f095ca00640 1 -- 172.21.15.165:0/2973538885 wait complete. 2024-04-25T10:07:29.035 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.213+0000 7f71b8800640 1 Processor -- start 2024-04-25T10:07:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.213+0000 7f71b8800640 1 -- start start 2024-04-25T10:07:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b8800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 0x7f71b00ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b8800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b0144b30 con 0x7f71b006d290 2024-04-25T10:07:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b8800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00712d0 con 0x7f71b0144750 2024-04-25T10:07:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b8800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00715b0 con 0x7f71b00ce920 2024-04-25T10:07:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b6800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f71b006d290 0x7f71b006d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37784/0 (socket says 172.21.15.165:37784) 2024-04-25T10:07:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b6800640 1 -- 172.21.15.165:0/2968253402 learned_addr learned my addr 172.21.15.165:0/2968253402 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b5400640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 0x7f71b00ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974841418 0 0) 0x7f71b0144b30 con 0x7f71b006d290 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.215+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7198003520 con 0x7f71b006d290 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2486616476 0 0) 0x7f7198003520 con 0x7f71b006d290 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 msgr2=0x7f71b00ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 0x7f71b00ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 >> v1:172.21.15.172:6789/0 conn(0x7f71b0144750 legacy=0x7f71b006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0071890 con 0x7f71b006d290 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f71ac0019f0 con 0x7f71b006d290 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 >> v1:172.21.15.165:6789/0 conn(0x7f71b006d290 legacy=0x7f71b006d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71980039e0 0x7f7198003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7198004800 0x7f7198004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 0x7f7198005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00715b0 con 0x7f7198004800 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719800bb20 con 0x7f71980039e0 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719800bd20 con 0x7f71b00ce920 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b5400640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7198004800 0x7f7198004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.216+0000 7f71b5e00640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71980039e0 0x7f7198003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.217+0000 7f71b6800640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 0x7f7198005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.217+0000 7f71b5400640 1 -- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 msgr2=0x7f7198005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.217+0000 7f71b5400640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71b00ce920 0x7f7198005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.217+0000 7f71b5400640 1 -- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71980039e0 msgr2=0x7f7198003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.217+0000 7f71b5400640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71980039e0 0x7f7198003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.217+0000 7f71b5400640 1 -- 172.21.15.165:0/2968253402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7198003520 con 0x7f7198004800 2024-04-25T10:07:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.218+0000 7f71b5400640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7198004800 0x7f7198004c80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f71a000ba30 tx=0x7f7198005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6364c53c4c10658a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.218+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0030070 con 0x7f7198004800 2024-04-25T10:07:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.218+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a002ade0 con 0x7f7198004800 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.218+0000 7f71b4a00640 1 -- 172.21.15.165:0/2968253402 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000b600 con 0x7f7198004800 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b8800640 1 -- 172.21.15.165:0/2968253402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7198004800 msgr2=0x7f7198004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b8800640 1 --2- 172.21.15.165:0/2968253402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7198004800 0x7f7198004c80 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f71a000ba30 tx=0x7f7198005830 comp rx=0 tx=0).stop 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b6800640 1 -- 172.21.15.165:0/2968253402 reap_dead start 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b8800640 1 -- 172.21.15.165:0/2968253402 shutdown_connections 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b8800640 1 -- 172.21.15.165:0/2968253402 >> 172.21.15.165:0/2968253402 conn(0x7f71b006abe0 msgr2=0x7f71b006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b8800640 1 -- 172.21.15.165:0/2968253402 shutdown_connections 2024-04-25T10:07:29.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.219+0000 7f71b8800640 1 -- 172.21.15.165:0/2968253402 wait complete. 2024-04-25T10:07:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.220+0000 7f71b8800640 1 Processor -- start 2024-04-25T10:07:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.221+0000 7f71b8800640 1 -- start start 2024-04-25T10:07:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.221+0000 7f71b8800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7198004800 0x7f71b00b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.221+0000 7f71b8800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 0x7f71b01f47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b8800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71b015e940 0x7f71b015edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b8800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b0145450 con 0x7f71b01f4370 2024-04-25T10:07:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b8800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b006f020 con 0x7f71b015e940 2024-04-25T10:07:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b8800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b01455d0 con 0x7f7198004800 2024-04-25T10:07:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 0x7f71b01f47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 0x7f71b01f47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36786/0 (socket says 172.21.15.165:36786) 2024-04-25T10:07:29.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 -- 172.21.15.165:0/855852049 learned_addr learned my addr 172.21.15.165:0/855852049 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 -- 172.21.15.165:0/855852049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7198004800 msgr2=0x7f71b00b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7198004800 0x7f71b00b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 -- 172.21.15.165:0/855852049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71b015e940 msgr2=0x7f71b015edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71b015e940 0x7f71b015edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.222+0000 7f71b5400640 1 -- 172.21.15.165:0/855852049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b015f600 con 0x7f71b01f4370 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.223+0000 7f71b5400640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 0x7f71b01f47f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f71a000bb20 tx=0x7f71a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.223+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a0043030 con 0x7f71b01f4370 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.223+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a002abe0 con 0x7f71b01f4370 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.223+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b015f8e0 con 0x7f71b01f4370 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.223+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a000b600 con 0x7f71b01f4370 2024-04-25T10:07:29.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.223+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b015fea0 con 0x7f71b01f4370 2024-04-25T10:07:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.224+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b006eb00 con 0x7f71b01f4370 2024-04-25T10:07:29.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.225+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71a0046070 con 0x7f71b01f4370 2024-04-25T10:07:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.226+0000 7f71aa000640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.227+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71a00aaf20 con 0x7f71b01f4370 2024-04-25T10:07:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.227+0000 7f71b5e00640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.227+0000 7f71b5e00640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f719406b4d0 0x7f719406d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fd0 tx=0x7f71a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.231+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71b006eb00 con 0x7f71b01f4370 2024-04-25T10:07:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.424+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71b0073990 con 0x7f71b01f4370 2024-04-25T10:07:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.426+0000 7f71aa000640 1 -- 172.21.15.165:0/855852049 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f71a00333b0 con 0x7f71b01f4370 2024-04-25T10:07:29.432 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:29.433 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":67,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:28.013469+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:29.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f719406b4d0 msgr2=0x7f719406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f719406b4d0 0x7f719406d990 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f71a4007fd0 tx=0x7f71a4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:29.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 msgr2=0x7f71b01f47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 0x7f71b01f47f0 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f71a000bb20 tx=0x7f71a00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 shutdown_connections 2024-04-25T10:07:29.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f719406b4d0 0x7f719406d990 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71b015e940 0x7f71b015edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71b01f4370 0x7f71b01f47f0 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 --2- 172.21.15.165:0/855852049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7198004800 0x7f71b00b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.429+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 >> 172.21.15.165:0/855852049 conn(0x7f71b006abe0 msgr2=0x7f71b0140bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.430+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 shutdown_connections 2024-04-25T10:07:29.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.430+0000 7f71b8800640 1 -- 172.21.15.165:0/855852049 wait complete. 2024-04-25T10:07:29.454 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:07:29.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.647+0000 7f83a1800640 1 Processor -- start 2024-04-25T10:07:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.647+0000 7f83a1800640 1 -- start start 2024-04-25T10:07:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.648+0000 7f83a1800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f839c15de80 0x7f839c0cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f83a1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c0cefb0 0x7f839c0cf410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f83a1800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f839c0cf950 0x7f839c06abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f83a1800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0afc80 con 0x7f839c15de80 2024-04-25T10:07:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f83a1800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0ce4d0 con 0x7f839c0cf950 2024-04-25T10:07:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f83a1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0ce7b0 con 0x7f839c0cefb0 2024-04-25T10:07:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f839c15de80 0x7f839c0cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f839c15de80 0x7f839c0cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36792/0 (socket says 172.21.15.165:36792) 2024-04-25T10:07:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 -- 172.21.15.165:0/1424158013 learned_addr learned my addr 172.21.15.165:0/1424158013 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f8392000640 1 --2- 172.21.15.165:0/1424158013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c0cefb0 0x7f839c0cf410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 -- 172.21.15.165:0/1424158013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c0cefb0 msgr2=0x7f839c0cf410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839b400640 1 --2- 172.21.15.165:0/1424158013 >> v2:172.21.15.172:3300/0 conn(0x7f839c0cf950 0x7f839c06abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 --2- 172.21.15.165:0/1424158013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c0cefb0 0x7f839c0cf410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 -- 172.21.15.165:0/1424158013 >> v2:172.21.15.172:3300/0 conn(0x7f839c0cf950 msgr2=0x7f839c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 --2- 172.21.15.165:0/1424158013 >> v2:172.21.15.172:3300/0 conn(0x7f839c0cf950 0x7f839c06abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.649+0000 7f839aa00640 1 -- 172.21.15.165:0/1424158013 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0cea90 con 0x7f839c15de80 2024-04-25T10:07:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.650+0000 7f839b400640 1 --2- 172.21.15.165:0/1424158013 >> v2:172.21.15.172:3300/0 conn(0x7f839c0cf950 0x7f839c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:29.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.651+0000 7f839aa00640 1 --2- 172.21.15.165:0/1424158013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c15de80 0x7f839c0cadb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f839c0d0360 tx=0x7f838c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76e1d2ebec6fa77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.651+0000 7f839a000640 1 -- 172.21.15.165:0/1424158013 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c037070 con 0x7f839c15de80 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.651+0000 7f839a000640 1 -- 172.21.15.165:0/1424158013 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c032440 con 0x7f839c15de80 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.651+0000 7f839a000640 1 -- 172.21.15.165:0/1424158013 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c032810 con 0x7f839c15de80 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 -- 172.21.15.165:0/1424158013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c15de80 msgr2=0x7f839c0cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 --2- 172.21.15.165:0/1424158013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c15de80 0x7f839c0cadb0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f839c0d0360 tx=0x7f838c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 -- 172.21.15.165:0/1424158013 shutdown_connections 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 --2- 172.21.15.165:0/1424158013 >> v2:172.21.15.172:3300/0 conn(0x7f839c0cf950 0x7f839c06abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 --2- 172.21.15.165:0/1424158013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c0cefb0 0x7f839c0cf410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 --2- 172.21.15.165:0/1424158013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c15de80 0x7f839c0cadb0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 -- 172.21.15.165:0/1424158013 >> 172.21.15.165:0/1424158013 conn(0x7f839c0b2390 msgr2=0x7f839c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 -- 172.21.15.165:0/1424158013 shutdown_connections 2024-04-25T10:07:29.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.652+0000 7f83a1800640 1 -- 172.21.15.165:0/1424158013 wait complete. 2024-04-25T10:07:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.654+0000 7f83a1800640 1 Processor -- start 2024-04-25T10:07:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.654+0000 7f83a1800640 1 -- start start 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f83a1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f839c0cefb0 0x7f839c0b5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f83a1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 0x7f839c0b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f83a1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c15de80 0x7f839c0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f83a1800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0b3030 con 0x7f839c0cf950 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f83a1800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0b2eb0 con 0x7f839c0cefb0 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f83a1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0c5600 con 0x7f839c15de80 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f8392000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 0x7f839c0b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f8392000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 0x7f839c0b3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36798/0 (socket says 172.21.15.165:36798) 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f8392000640 1 -- 172.21.15.165:0/4252588997 learned_addr learned my addr 172.21.15.165:0/4252588997 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.655+0000 7f839aa00640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f839c0cefb0 0x7f839c0b5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8392000640 1 -- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c15de80 msgr2=0x7f839c0ae030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8392000640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c15de80 0x7f839c0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8392000640 1 -- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f839c0cefb0 msgr2=0x7f839c0b5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8392000640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f839c0cefb0 0x7f839c0b5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8392000640 1 -- 172.21.15.165:0/4252588997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0b6290 con 0x7f839c0cf950 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8392000640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 0x7f839c0b3720 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f838400c820 tx=0x7f838400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384015070 con 0x7f839c0cf950 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384004220 con 0x7f839c0cf950 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400ce10 con 0x7f839c0cf950 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0c7c50 con 0x7f839c0cf950 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.656+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c0ae570 con 0x7f839c0cf950 2024-04-25T10:07:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.657+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c0c7a40 con 0x7f839c0cf950 2024-04-25T10:07:29.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.659+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8384007830 con 0x7f839c0cf950 2024-04-25T10:07:29.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.660+0000 7f8393400640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:29.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.660+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f838408a250 con 0x7f839c0cf950 2024-04-25T10:07:29.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.660+0000 7f839aa00640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:29.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.661+0000 7f839aa00640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f838c034000 tx=0x7f838c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:29.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.664+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8384058ea0 con 0x7f839c0cf950 2024-04-25T10:07:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:29.894+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f839c073a00 con 0x7f839c0cf950 2024-04-25T10:07:30.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.598+0000 7f8393400640 1 -- 172.21.15.165:0/4252588997 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f83840587a0 con 0x7f839c0cf950 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f837806b4d0 msgr2=0x7f837806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f837806b4d0 0x7f837806d990 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f838c034000 tx=0x7f838c02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 msgr2=0x7f839c0b3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 0x7f839c0b3720 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f838400c820 tx=0x7f838400cce0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 shutdown_connections 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f837806b4d0 0x7f837806d990 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f839c15de80 0x7f839c0ae030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f839c0cf950 0x7f839c0b3720 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 --2- 172.21.15.165:0/4252588997 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f839c0cefb0 0x7f839c0b5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 >> 172.21.15.165:0/4252588997 conn(0x7f839c0b2390 msgr2=0x7f839c0d06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 shutdown_connections 2024-04-25T10:07:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.603+0000 7f83a1800640 1 -- 172.21.15.165:0/4252588997 wait complete. 2024-04-25T10:07:30.629 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:07:30.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:07:30.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.832+0000 7f2b58400640 1 Processor -- start 2024-04-25T10:07:30.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.832+0000 7f2b58400640 1 -- start start 2024-04-25T10:07:30.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.834+0000 7f2b58400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b500b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.834+0000 7f2b58400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b500cccc0 con 0x7f2b5006e8a0 2024-04-25T10:07:30.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.834+0000 7f2b58400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b500d0460 con 0x7f2b50144d50 2024-04-25T10:07:30.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.834+0000 7f2b58400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b2b20 con 0x7f2b501446d0 2024-04-25T10:07:30.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.834+0000 7f2b56400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2b5006e8a0 0x7f2b500cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37794/0 (socket says 172.21.15.165:37794) 2024-04-25T10:07:30.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.834+0000 7f2b56400640 1 -- 172.21.15.165:0/206641643 learned_addr learned my addr 172.21.15.165:0/206641643 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:30.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b55a00640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b500b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1153619074 0 0) 0x7f2b500cccc0 con 0x7f2b5006e8a0 2024-04-25T10:07:30.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b38003520 con 0x7f2b5006e8a0 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1356325231 0 0) 0x7f2b38003520 con 0x7f2b5006e8a0 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 msgr2=0x7f2b500b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b500b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 >> v1:172.21.15.172:6789/0 conn(0x7f2b50144d50 legacy=0x7f2b500cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b500cc140 con 0x7f2b5006e8a0 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2b480019f0 con 0x7f2b5006e8a0 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 >> v1:172.21.15.165:6789/0 conn(0x7f2b5006e8a0 legacy=0x7f2b500cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b380039e0 0x7f2b38003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b38004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500b2b20 con 0x7f2b38004800 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3800baf0 con 0x7f2b380039e0 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.835+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3800bcf0 con 0x7f2b501446d0 2024-04-25T10:07:30.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55000640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b38004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b56400640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55a00640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b380039e0 0x7f2b38003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55000640 1 -- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 msgr2=0x7f2b38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55000640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b38005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55000640 1 -- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b380039e0 msgr2=0x7f2b38003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55000640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b380039e0 0x7f2b38003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b55000640 1 -- 172.21.15.165:0/206641643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b38003520 con 0x7f2b38004800 2024-04-25T10:07:30.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.836+0000 7f2b56400640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b501446d0 0x7f2b38005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.837+0000 7f2b55000640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b38004c80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001750 tx=0x7f2b38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=536c15006bfeafd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.837+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c001370 con 0x7f2b38004800 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.837+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b3c0052d0 con 0x7f2b38004800 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.837+0000 7f2b4fe00640 1 -- 172.21.15.165:0/206641643 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b3c02abe0 con 0x7f2b38004800 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b58400640 1 -- 172.21.15.165:0/206641643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 msgr2=0x7f2b38004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b58400640 1 --2- 172.21.15.165:0/206641643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b38004c80 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c001750 tx=0x7f2b38005830 comp rx=0 tx=0).stop 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b56400640 1 -- 172.21.15.165:0/206641643 reap_dead start 2024-04-25T10:07:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b58400640 1 -- 172.21.15.165:0/206641643 shutdown_connections 2024-04-25T10:07:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b58400640 1 -- 172.21.15.165:0/206641643 >> 172.21.15.165:0/206641643 conn(0x7f2b5006abe0 msgr2=0x7f2b5006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b58400640 1 -- 172.21.15.165:0/206641643 shutdown_connections 2024-04-25T10:07:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.838+0000 7f2b58400640 1 -- 172.21.15.165:0/206641643 wait complete. 2024-04-25T10:07:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.840+0000 7f2b58400640 1 Processor -- start 2024-04-25T10:07:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.840+0000 7f2b58400640 1 -- start start 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b58400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b501f7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b58400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b50156990 0x7f2b501590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b58400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b50140220 0x7f2b500ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b58400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500cc700 con 0x7f2b38004800 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b58400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b5006d400 con 0x7f2b50156990 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b58400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b500cc9e0 con 0x7f2b50140220 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b501f7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b501f7df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36806/0 (socket says 172.21.15.165:36806) 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 -- 172.21.15.165:0/4088478587 learned_addr learned my addr 172.21.15.165:0/4088478587 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 -- 172.21.15.165:0/4088478587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b50140220 msgr2=0x7f2b500ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b50140220 0x7f2b500ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 -- 172.21.15.165:0/4088478587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b50156990 msgr2=0x7f2b501590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b50156990 0x7f2b501590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.841+0000 7f2b55a00640 1 -- 172.21.15.165:0/4088478587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50159980 con 0x7f2b38004800 2024-04-25T10:07:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.842+0000 7f2b55a00640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b501f7df0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400e7e0 tx=0x7f2b4400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.842+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b44017070 con 0x7f2b38004800 2024-04-25T10:07:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.842+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b4400c6d0 con 0x7f2b38004800 2024-04-25T10:07:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.842+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b4400f040 con 0x7f2b38004800 2024-04-25T10:07:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.842+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b50159c60 con 0x7f2b38004800 2024-04-25T10:07:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.842+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b501f7610 con 0x7f2b38004800 2024-04-25T10:07:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.843+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b50073a00 con 0x7f2b38004800 2024-04-25T10:07:30.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.844+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b4401a070 con 0x7f2b38004800 2024-04-25T10:07:30.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.845+0000 7f2b4d600640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2b3006b500 0x7f2b3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:30.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.845+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2b44089fe0 con 0x7f2b38004800 2024-04-25T10:07:30.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.845+0000 7f2b55000640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2b3006b500 0x7f2b3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:30.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.846+0000 7f2b55000640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2b3006b500 0x7f2b3006d9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c032000 tx=0x7f2b3c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:30.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:30.849+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b44058cc0 con 0x7f2b38004800 2024-04-25T10:07:31.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.080+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2b50142540 con 0x7f2b38004800 2024-04-25T10:07:31.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.601+0000 7f2b4d600640 1 -- 172.21.15.165:0/4088478587 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2b440585c0 con 0x7f2b38004800 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2b3006b500 msgr2=0x7f2b3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2b3006b500 0x7f2b3006d9c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f2b3c032000 tx=0x7f2b3c02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 msgr2=0x7f2b501f7df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b501f7df0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f2b4400e7e0 tx=0x7f2b4400eca0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 shutdown_connections 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2b3006b500 0x7f2b3006d9c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b50140220 0x7f2b500ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b50156990 0x7f2b501590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 --2- 172.21.15.165:0/4088478587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b38004800 0x7f2b501f7df0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 >> 172.21.15.165:0/4088478587 conn(0x7f2b5006abe0 msgr2=0x7f2b500af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 shutdown_connections 2024-04-25T10:07:31.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.605+0000 7f2b58400640 1 -- 172.21.15.165:0/4088478587 wait complete. 2024-04-25T10:07:31.631 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:07:31.631 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:31.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.821+0000 7f72e1c00640 1 Processor -- start 2024-04-25T10:07:31.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.821+0000 7f72e1c00640 1 -- start start 2024-04-25T10:07:31.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.822+0000 7f72e1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc0ce920 0x7f72dc0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72e1c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f72dc06d290 0x7f72dc06d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72e1c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72dc144b30 con 0x7f72dc144750 2024-04-25T10:07:31.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72e1c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72dc0712d0 con 0x7f72dc06d290 2024-04-25T10:07:31.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72e1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc0715b0 con 0x7f72dc0ce920 2024-04-25T10:07:31.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72db400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f72dc06d290 0x7f72dc06d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72db400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f72dc06d290 0x7f72dc06d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36818/0 (socket says 172.21.15.165:36818) 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72db400640 1 -- 172.21.15.165:0/41853720 learned_addr learned my addr 172.21.15.165:0/41853720 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.823+0000 7f72da000640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc0ce920 0x7f72dc0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.824+0000 7f72db400640 1 -- 172.21.15.165:0/41853720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc0ce920 msgr2=0x7f72dc0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.824+0000 7f72db400640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc0ce920 0x7f72dc0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.824+0000 7f72db400640 1 -- 172.21.15.165:0/41853720 >> v1:172.21.15.172:6789/0 conn(0x7f72dc144750 legacy=0x7f72dc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.824+0000 7f72db400640 1 -- 172.21.15.165:0/41853720 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc071890 con 0x7f72dc06d290 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.824+0000 7f72da000640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc0ce920 0x7f72dc0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:31.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.825+0000 7f72db400640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc06d720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009e50 tx=0x7f72d402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a3034b79ceb7cd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.825+0000 7f72d9600640 1 -- 172.21.15.165:0/41853720 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d4036070 con 0x7f72dc06d290 2024-04-25T10:07:31.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72d9600640 1 -- 172.21.15.165:0/41853720 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d4031600 con 0x7f72dc06d290 2024-04-25T10:07:31.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72d9600640 1 -- 172.21.15.165:0/41853720 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d40319d0 con 0x7f72dc06d290 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 -- 172.21.15.165:0/41853720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 msgr2=0x7f72dc06d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc06d720 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f72d4009e50 tx=0x7f72d402a710 comp rx=0 tx=0).stop 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 -- 172.21.15.165:0/41853720 shutdown_connections 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc06d720 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 --2- 172.21.15.165:0/41853720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc0ce920 0x7f72dc0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 -- 172.21.15.165:0/41853720 >> 172.21.15.165:0/41853720 conn(0x7f72dc06abe0 msgr2=0x7f72dc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:31.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 -- 172.21.15.165:0/41853720 shutdown_connections 2024-04-25T10:07:31.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.826+0000 7f72e1c00640 1 -- 172.21.15.165:0/41853720 wait complete. 2024-04-25T10:07:31.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.828+0000 7f72e1c00640 1 Processor -- start 2024-04-25T10:07:31.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.828+0000 7f72e1c00640 1 -- start start 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72e1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc13e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72e1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72dc0ce920 0x7f72dc15ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72e1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc144750 0x7f72dc0b0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72e1c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc06f020 con 0x7f72dc06d290 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72e1c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc06ece0 con 0x7f72dc0ce920 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72e1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72dc145450 con 0x7f72dc144750 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.829+0000 7f72daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc13e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc13e080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36822/0 (socket says 172.21.15.165:36822) 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 -- 172.21.15.165:0/2211065250 learned_addr learned my addr 172.21.15.165:0/2211065250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 -- 172.21.15.165:0/2211065250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc144750 msgr2=0x7f72dc0b0ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc144750 0x7f72dc0b0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 -- 172.21.15.165:0/2211065250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72dc0ce920 msgr2=0x7f72dc15ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72dc0ce920 0x7f72dc15ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 -- 172.21.15.165:0/2211065250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc15d2b0 con 0x7f72dc06d290 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72daa00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc13e080 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c430 tx=0x7f72cc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc0052d0 con 0x7f72dc06d290 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72cc005490 con 0x7f72dc06d290 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72cc007260 con 0x7f72dc06d290 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.830+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72dc15d590 con 0x7f72dc06d290 2024-04-25T10:07:31.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.831+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72dc15db50 con 0x7f72dc06d290 2024-04-25T10:07:31.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.832+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72dc073990 con 0x7f72dc06d290 2024-04-25T10:07:31.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.833+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72cc007400 con 0x7f72dc06d290 2024-04-25T10:07:31.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.833+0000 7f72d2a00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72bc06b4d0 0x7f72bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:31.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.834+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72cc089850 con 0x7f72dc06d290 2024-04-25T10:07:31.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.834+0000 7f72da000640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72bc06b4d0 0x7f72bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:31.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.834+0000 7f72da000640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72bc06b4d0 0x7f72bc06d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72c8006fd0 tx=0x7f72c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:31.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:31.838+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72cc058530 con 0x7f72dc06d290 2024-04-25T10:07:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.072+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72dc0d0510 con 0x7f72dc06d290 2024-04-25T10:07:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.074+0000 7f72d2a00640 1 -- 172.21.15.165:0/2211065250 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f72cc057e30 con 0x7f72dc06d290 2024-04-25T10:07:32.079 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:32.080 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/57109722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":57109722},{"type":"v1","addr":"172.21.15.172:6834","nonce":57109722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":4982,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3177295444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3177295444},{"type":"v1","addr":"172.21.15.172:6835","nonce":3177295444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/2353868161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2353868161},{"type":"v1","addr":"172.21.15.165:6836","nonce":2353868161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:31.082094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1883921643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1883921643},{"type":"v1","addr":"172.21.15.165:6837","nonce":1883921643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 40 2024-04-25T10:07:32.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.076+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72bc06b4d0 msgr2=0x7f72bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.076+0000 7f72e1c00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72bc06b4d0 0x7f72bc06d990 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f72c8006fd0 tx=0x7f72c8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:32.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.076+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 msgr2=0x7f72dc13e080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.076+0000 7f72e1c00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc13e080 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f72cc00c430 tx=0x7f72cc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 shutdown_connections 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72bc06b4d0 0x7f72bc06d990 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72dc144750 0x7f72dc0b0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72dc0ce920 0x7f72dc15ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 --2- 172.21.15.165:0/2211065250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72dc06d290 0x7f72dc13e080 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 >> 172.21.15.165:0/2211065250 conn(0x7f72dc06abe0 msgr2=0x7f72dc0cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 shutdown_connections 2024-04-25T10:07:32.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.077+0000 7f72e1c00640 1 -- 172.21.15.165:0/2211065250 wait complete. 2024-04-25T10:07:32.100 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:32.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.293+0000 7faa26000640 1 Processor -- start 2024-04-25T10:07:32.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.294+0000 7faa26000640 1 -- start start 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa26000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faa20143e60 0x7faa2006e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa26000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa200cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa26000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa2015bb50 con 0x7faa200cc840 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa26000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa20144240 con 0x7faa20143e60 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa26000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa2015e220 con 0x7faa2006e6a0 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa25600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7faa200cc840 0x7faa20070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37802/0 (socket says 172.21.15.165:37802) 2024-04-25T10:07:32.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.295+0000 7faa25600640 1 -- 172.21.15.165:0/1559856782 learned_addr learned my addr 172.21.15.165:0/1559856782 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:32.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa17e00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa200cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa24c00640 1 --2- 172.21.15.165:0/1559856782 >> v2:172.21.15.172:3300/0 conn(0x7faa20143e60 0x7faa2006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 216184538 0 0) 0x7faa18001490 con 0x7faa200cc840 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa04003520 con 0x7faa200cc840 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3337120513 0 0) 0x7faa04003520 con 0x7faa200cc840 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 msgr2=0x7faa200cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa200cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 >> v2:172.21.15.172:3300/0 conn(0x7faa20143e60 msgr2=0x7faa2006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 --2- 172.21.15.165:0/1559856782 >> v2:172.21.15.172:3300/0 conn(0x7faa20143e60 0x7faa2006e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa2015e500 con 0x7faa200cc840 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7faa18001bc0 con 0x7faa200cc840 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 >> v1:172.21.15.165:6789/0 conn(0x7faa200cc840 legacy=0x7faa20070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa040039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004030 0x7faa04004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa04004d40 0x7faa040051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa20144240 con 0x7faa04004d40 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2015e220 con 0x7faa04004030 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0400bb20 con 0x7faa2006e6a0 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.296+0000 7faa25600640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa04004d40 0x7faa040051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa24c00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa040039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa17e00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004030 0x7faa04004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa25600640 1 -- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 msgr2=0x7faa040039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa25600640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa040039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa25600640 1 -- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004030 msgr2=0x7faa04004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa25600640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004030 0x7faa04004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa25600640 1 -- 172.21.15.165:0/1559856782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa04003520 con 0x7faa04004d40 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.297+0000 7faa24c00640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2006e6a0 0x7faa040039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.298+0000 7faa25600640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa04004d40 0x7faa040051c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7faa1800a9a0 tx=0x7faa04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d81554f962fd61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.298+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1800a5c0 con 0x7faa04004d40 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.298+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa1802f070 con 0x7faa04004d40 2024-04-25T10:07:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.298+0000 7faa1fe00640 1 -- 172.21.15.165:0/1559856782 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa1802be50 con 0x7faa04004d40 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa26000640 1 -- 172.21.15.165:0/1559856782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa04004d40 msgr2=0x7faa040051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa26000640 1 --2- 172.21.15.165:0/1559856782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa04004d40 0x7faa040051c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7faa1800a9a0 tx=0x7faa04005830 comp rx=0 tx=0).stop 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa25600640 1 -- 172.21.15.165:0/1559856782 reap_dead start 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa26000640 1 -- 172.21.15.165:0/1559856782 shutdown_connections 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa26000640 1 -- 172.21.15.165:0/1559856782 >> 172.21.15.165:0/1559856782 conn(0x7faa2006abe0 msgr2=0x7faa2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa26000640 1 -- 172.21.15.165:0/1559856782 shutdown_connections 2024-04-25T10:07:32.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.299+0000 7faa26000640 1 -- 172.21.15.165:0/1559856782 wait complete. 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.300+0000 7faa26000640 1 Processor -- start 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 -- start start 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004d40 0x7faa20155430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 0x7faa200b0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2013bb30 0x7faa2013bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa200cccd0 con 0x7faa200b0910 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa2015eac0 con 0x7faa04004d40 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.301+0000 7faa26000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa200cce50 con 0x7faa2013bb30 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 0x7faa200b0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 0x7faa200b0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36842/0 (socket says 172.21.15.165:36842) 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 -- 172.21.15.165:0/1686127893 learned_addr learned my addr 172.21.15.165:0/1686127893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 -- 172.21.15.165:0/1686127893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2013bb30 msgr2=0x7faa2013bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2013bb30 0x7faa2013bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 -- 172.21.15.165:0/1686127893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004d40 msgr2=0x7faa20155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004d40 0x7faa20155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 -- 172.21.15.165:0/1686127893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa200b13d0 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa17e00640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 0x7faa200b0d90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7faa0800e810 tx=0x7faa0800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.302+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa08017070 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.303+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0800c5f0 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.303+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0800f040 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.303+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa200b16b0 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.303+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa200b1c70 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.304+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa20073990 con 0x7faa200b0910 2024-04-25T10:07:32.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.305+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa0801a070 con 0x7faa200b0910 2024-04-25T10:07:32.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.306+0000 7faa1d600640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9fc06b500 0x7fa9fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.306+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa08089f10 con 0x7faa200b0910 2024-04-25T10:07:32.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.306+0000 7faa24c00640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9fc06b500 0x7fa9fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.307+0000 7faa24c00640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9fc06b500 0x7fa9fc06d9c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa10006fd0 tx=0x7faa100093d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.311+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa08058b60 con 0x7faa200b0910 2024-04-25T10:07:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.507+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa20067df0 con 0x7faa200b0910 2024-04-25T10:07:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.508+0000 7faa1d600640 1 -- 172.21.15.165:0/1686127893 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16920 (secure 0 0 0) 0x7faa08058460 con 0x7faa200b0910 2024-04-25T10:07:32.514 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:32.515 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":68,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:29.572452+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9fc06b500 msgr2=0x7fa9fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9fc06b500 0x7fa9fc06d9c0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7faa10006fd0 tx=0x7faa100093d0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 msgr2=0x7faa200b0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 0x7faa200b0d90 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7faa0800e810 tx=0x7faa0800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 shutdown_connections 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9fc06b500 0x7fa9fc06d9c0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa2013bb30 0x7faa2013bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa200b0910 0x7faa200b0d90 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 --2- 172.21.15.165:0/1686127893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa04004d40 0x7faa20155430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 >> 172.21.15.165:0/1686127893 conn(0x7faa2006abe0 msgr2=0x7faa200b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 shutdown_connections 2024-04-25T10:07:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.511+0000 7faa26000640 1 -- 172.21.15.165:0/1686127893 wait complete. 2024-04-25T10:07:32.537 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:07:32.537 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.732+0000 7f880a800640 1 Processor -- start 2024-04-25T10:07:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.733+0000 7f880a800640 1 -- start start 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.734+0000 7f880a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804144440 0x7f8804070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.734+0000 7f880a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8804071530 0x7f88040cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.734+0000 7f880a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f88040cd860 0x7f880406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.734+0000 7f880a800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880415c860 con 0x7f88040cd860 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.734+0000 7f880a800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8804071970 con 0x7f8804071530 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.734+0000 7f880a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880415bf30 con 0x7f8804144440 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f88040cd860 0x7f880406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f88040cd860 0x7f880406d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36850/0 (socket says 172.21.15.165:36850) 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 -- 172.21.15.165:0/2451857510 learned_addr learned my addr 172.21.15.165:0/2451857510 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:32.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809400640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804144440 0x7f8804070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8808a00640 1 --2- 172.21.15.165:0/2451857510 >> v2:172.21.15.172:3300/0 conn(0x7f8804071530 0x7f88040cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 -- 172.21.15.165:0/2451857510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804144440 msgr2=0x7f8804070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804144440 0x7f8804070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 -- 172.21.15.165:0/2451857510 >> v2:172.21.15.172:3300/0 conn(0x7f8804071530 msgr2=0x7f88040cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 --2- 172.21.15.165:0/2451857510 >> v2:172.21.15.172:3300/0 conn(0x7f8804071530 0x7f88040cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809e00640 1 -- 172.21.15.165:0/2451857510 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880415c210 con 0x7f88040cd860 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8809400640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804144440 0x7f8804070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:32.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.735+0000 7f8808a00640 1 --2- 172.21.15.165:0/2451857510 >> v2:172.21.15.172:3300/0 conn(0x7f8804071530 0x7f88040cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:32.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.736+0000 7f8809e00640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f880406d050 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8804072a00 tx=0x7f87fc02a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8be1d3eefebceb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f8803e00640 1 -- 172.21.15.165:0/2451857510 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc036070 con 0x7f88040cd860 2024-04-25T10:07:32.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f8803e00640 1 -- 172.21.15.165:0/2451857510 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc031600 con 0x7f88040cd860 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f8803e00640 1 -- 172.21.15.165:0/2451857510 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0319d0 con 0x7f88040cd860 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 -- 172.21.15.165:0/2451857510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 msgr2=0x7f880406d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f880406d050 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f8804072a00 tx=0x7f87fc02a730 comp rx=0 tx=0).stop 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 -- 172.21.15.165:0/2451857510 shutdown_connections 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f880406d050 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 --2- 172.21.15.165:0/2451857510 >> v2:172.21.15.172:3300/0 conn(0x7f8804071530 0x7f88040cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 --2- 172.21.15.165:0/2451857510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804144440 0x7f8804070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 -- 172.21.15.165:0/2451857510 >> 172.21.15.165:0/2451857510 conn(0x7f880406abe0 msgr2=0x7f880406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.737+0000 7f880a800640 1 -- 172.21.15.165:0/2451857510 shutdown_connections 2024-04-25T10:07:32.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.738+0000 7f880a800640 1 -- 172.21.15.165:0/2451857510 wait complete. 2024-04-25T10:07:32.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.740+0000 7f880a800640 1 Processor -- start 2024-04-25T10:07:32.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.740+0000 7f880a800640 1 -- start start 2024-04-25T10:07:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f880a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804071530 0x7f880413e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f880a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f88041f18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f880a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8804144440 0x7f88041e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f880a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040cdcd0 con 0x7f88040cd860 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f880a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880415bc30 con 0x7f8804144440 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f880a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88040cde50 con 0x7f8804071530 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f8808a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f88041f18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f8808a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f88041f18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36858/0 (socket says 172.21.15.165:36858) 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.741+0000 7f8808a00640 1 -- 172.21.15.165:0/3291401710 learned_addr learned my addr 172.21.15.165:0/3291401710 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8809400640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804071530 0x7f880413e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8808a00640 1 -- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804071530 msgr2=0x7f880413e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8808a00640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804071530 0x7f880413e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8809e00640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8804144440 0x7f88041e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8808a00640 1 -- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8804144440 msgr2=0x7f88041e39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8808a00640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8804144440 0x7f88041e39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8808a00640 1 -- 172.21.15.165:0/3291401710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88041f1e10 con 0x7f88040cd860 2024-04-25T10:07:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8809400640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804071530 0x7f880413e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8808a00640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f88041f18d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c7c0 tx=0x7f87f000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.742+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0015070 con 0x7f88040cd860 2024-04-25T10:07:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.743+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f0007a30 con 0x7f88040cd860 2024-04-25T10:07:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.743+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88040cbb60 con 0x7f88040cd860 2024-04-25T10:07:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.743+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f0007e00 con 0x7f88040cd860 2024-04-25T10:07:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.743+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88041e20a0 con 0x7f88040cd860 2024-04-25T10:07:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.744+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8804073990 con 0x7f88040cd860 2024-04-25T10:07:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.745+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87f0018070 con 0x7f88040cd860 2024-04-25T10:07:32.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.746+0000 7f8801600640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87e406b4d0 0x7f87e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:32.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.746+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87f008a320 con 0x7f88040cd860 2024-04-25T10:07:32.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.747+0000 7f8809400640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87e406b4d0 0x7f87e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:32.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.747+0000 7f8809400640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87e406b4d0 0x7f87e406d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006fd0 tx=0x7f87f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:32.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.751+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88041f1e10 con 0x7f88040cd860 2024-04-25T10:07:32.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.947+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f880406bff0 con 0x7f87e406b4d0 2024-04-25T10:07:32.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.955+0000 7f8801600640 1 -- 172.21.15.165:0/3291401710 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252028 (secure 0 0 0) 0x7f880406bff0 con 0x7f87e406b4d0 2024-04-25T10:07:32.963 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:32.965 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:07:32.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87e406b4d0 msgr2=0x7f87e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87e406b4d0 0x7f87e406d990 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f87f8006fd0 tx=0x7f87f8008040 comp rx=0 tx=0).stop 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 msgr2=0x7f88041f18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f88041f18d0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f87f000c7c0 tx=0x7f87f000cc80 comp rx=0 tx=0).stop 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 shutdown_connections 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87e406b4d0 0x7f87e406d990 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8804144440 0x7f88041e39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88040cd860 0x7f88041f18d0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 --2- 172.21.15.165:0/3291401710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8804071530 0x7f880413e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 >> 172.21.15.165:0/3291401710 conn(0x7f880406abe0 msgr2=0x7f88040ccdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 shutdown_connections 2024-04-25T10:07:32.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:32.961+0000 7f880a800640 1 -- 172.21.15.165:0/3291401710 wait complete. 2024-04-25T10:07:32.984 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-04-25T10:07:32.239388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320936,"kb_used_data":7656,"kb_used_omap":12,"kb_used_meta":313139,"kb_avail":754653784,"statfs":{"total":773094113280,"available":772765474816,"internally_reserved":0,"allocated":7839744,"data_stored":4188524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":320654898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.708353"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.981016+0000","last_change":"2024-04-25T10:07:28.583707+0000","last_active":"2024-04-25T10:07:29.981016+0000","last_peered":"2024-04-25T10:07:29.981016+0000","last_clean":"2024-04-25T10:07:29.981016+0000","last_became_active":"2024-04-25T10:07:28.583434+0000","last_became_peered":"2024-04-25T10:07:28.583434+0000","last_unstale":"2024-04-25T10:07:29.981016+0000","last_undegraded":"2024-04-25T10:07:29.981016+0000","last_fullsized":"2024-04-25T10:07:29.981016+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:25.826659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980751+0000","last_change":"2024-04-25T10:07:28.580443+0000","last_active":"2024-04-25T10:07:29.980751+0000","last_peered":"2024-04-25T10:07:29.980751+0000","last_clean":"2024-04-25T10:07:29.980751+0000","last_became_active":"2024-04-25T10:07:28.580106+0000","last_became_peered":"2024-04-25T10:07:28.580106+0000","last_unstale":"2024-04-25T10:07:29.980751+0000","last_undegraded":"2024-04-25T10:07:29.980751+0000","last_fullsized":"2024-04-25T10:07:29.980751+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:32.844960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585658+0000","last_change":"2024-04-25T10:07:28.592962+0000","last_active":"2024-04-25T10:07:29.585658+0000","last_peered":"2024-04-25T10:07:29.585658+0000","last_clean":"2024-04-25T10:07:29.585658+0000","last_became_active":"2024-04-25T10:07:28.591998+0000","last_became_peered":"2024-04-25T10:07:28.591998+0000","last_unstale":"2024-04-25T10:07:29.585658+0000","last_undegraded":"2024-04-25T10:07:29.585658+0000","last_fullsized":"2024-04-25T10:07:29.585658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:33.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976838+0000","last_change":"2024-04-25T10:07:28.582105+0000","last_active":"2024-04-25T10:07:29.976838+0000","last_peered":"2024-04-25T10:07:29.976838+0000","last_clean":"2024-04-25T10:07:29.976838+0000","last_became_active":"2024-04-25T10:07:28.581716+0000","last_became_peered":"2024-04-25T10:07:28.581716+0000","last_unstale":"2024-04-25T10:07:29.976838+0000","last_undegraded":"2024-04-25T10:07:29.976838+0000","last_fullsized":"2024-04-25T10:07:29.976838+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:24.527924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583807+0000","last_change":"2024-04-25T10:07:28.579534+0000","last_active":"2024-04-25T10:07:29.583807+0000","last_peered":"2024-04-25T10:07:29.583807+0000","last_clean":"2024-04-25T10:07:29.583807+0000","last_became_active":"2024-04-25T10:07:28.576106+0000","last_became_peered":"2024-04-25T10:07:28.576106+0000","last_unstale":"2024-04-25T10:07:29.583807+0000","last_undegraded":"2024-04-25T10:07:29.583807+0000","last_fullsized":"2024-04-25T10:07:29.583807+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.668084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978347+0000","last_change":"2024-04-25T10:07:28.587766+0000","last_active":"2024-04-25T10:07:29.978347+0000","last_peered":"2024-04-25T10:07:29.978347+0000","last_clean":"2024-04-25T10:07:29.978347+0000","last_became_active":"2024-04-25T10:07:28.587470+0000","last_became_peered":"2024-04-25T10:07:28.587470+0000","last_unstale":"2024-04-25T10:07:29.978347+0000","last_undegraded":"2024-04-25T10:07:29.978347+0000","last_fullsized":"2024-04-25T10:07:29.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:30.539972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.581007+0000","last_change":"2024-04-25T10:07:28.594904+0000","last_active":"2024-04-25T10:07:29.581007+0000","last_peered":"2024-04-25T10:07:29.581007+0000","last_clean":"2024-04-25T10:07:29.581007+0000","last_became_active":"2024-04-25T10:07:28.594586+0000","last_became_peered":"2024-04-25T10:07:28.594586+0000","last_unstale":"2024-04-25T10:07:29.581007+0000","last_undegraded":"2024-04-25T10:07:29.581007+0000","last_fullsized":"2024-04-25T10:07:29.581007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:22.487994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598125+0000","last_change":"2024-04-25T10:07:28.600499+0000","last_active":"2024-04-25T10:07:29.598125+0000","last_peered":"2024-04-25T10:07:29.598125+0000","last_clean":"2024-04-25T10:07:29.598125+0000","last_became_active":"2024-04-25T10:07:28.597407+0000","last_became_peered":"2024-04-25T10:07:28.597407+0000","last_unstale":"2024-04-25T10:07:29.598125+0000","last_undegraded":"2024-04-25T10:07:29.598125+0000","last_fullsized":"2024-04-25T10:07:29.598125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.470219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586609+0000","last_change":"2024-04-25T10:07:28.592105+0000","last_active":"2024-04-25T10:07:29.586609+0000","last_peered":"2024-04-25T10:07:29.586609+0000","last_clean":"2024-04-25T10:07:29.586609+0000","last_became_active":"2024-04-25T10:07:28.591588+0000","last_became_peered":"2024-04-25T10:07:28.591588+0000","last_unstale":"2024-04-25T10:07:29.586609+0000","last_undegraded":"2024-04-25T10:07:29.586609+0000","last_fullsized":"2024-04-25T10:07:29.586609+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:04.011580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583413+0000","last_change":"2024-04-25T10:07:28.595500+0000","last_active":"2024-04-25T10:07:29.583413+0000","last_peered":"2024-04-25T10:07:29.583413+0000","last_clean":"2024-04-25T10:07:29.583413+0000","last_became_active":"2024-04-25T10:07:28.594889+0000","last_became_peered":"2024-04-25T10:07:28.594889+0000","last_unstale":"2024-04-25T10:07:29.583413+0000","last_undegraded":"2024-04-25T10:07:29.583413+0000","last_fullsized":"2024-04-25T10:07:29.583413+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:25.685599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976224+0000","last_change":"2024-04-25T10:07:28.577622+0000","last_active":"2024-04-25T10:07:29.976224+0000","last_peered":"2024-04-25T10:07:29.976224+0000","last_clean":"2024-04-25T10:07:29.976224+0000","last_became_active":"2024-04-25T10:07:28.577275+0000","last_became_peered":"2024-04-25T10:07:28.577275+0000","last_unstale":"2024-04-25T10:07:29.976224+0000","last_undegraded":"2024-04-25T10:07:29.976224+0000","last_fullsized":"2024-04-25T10:07:29.976224+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:30.216442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598629+0000","last_change":"2024-04-25T10:07:28.596382+0000","last_active":"2024-04-25T10:07:29.598629+0000","last_peered":"2024-04-25T10:07:29.598629+0000","last_clean":"2024-04-25T10:07:29.598629+0000","last_became_active":"2024-04-25T10:07:28.596093+0000","last_became_peered":"2024-04-25T10:07:28.596093+0000","last_unstale":"2024-04-25T10:07:29.598629+0000","last_undegraded":"2024-04-25T10:07:29.598629+0000","last_fullsized":"2024-04-25T10:07:29.598629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:25.327162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598129+0000","last_change":"2024-04-25T10:07:28.591893+0000","last_active":"2024-04-25T10:07:29.598129+0000","last_peered":"2024-04-25T10:07:29.598129+0000","last_clean":"2024-04-25T10:07:29.598129+0000","last_became_active":"2024-04-25T10:07:28.591565+0000","last_became_peered":"2024-04-25T10:07:28.591565+0000","last_unstale":"2024-04-25T10:07:29.598129+0000","last_undegraded":"2024-04-25T10:07:29.598129+0000","last_fullsized":"2024-04-25T10:07:29.598129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:14.966243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584664+0000","last_change":"2024-04-25T10:07:28.590676+0000","last_active":"2024-04-25T10:07:29.584664+0000","last_peered":"2024-04-25T10:07:29.584664+0000","last_clean":"2024-04-25T10:07:29.584664+0000","last_became_active":"2024-04-25T10:07:28.590326+0000","last_became_peered":"2024-04-25T10:07:28.590326+0000","last_unstale":"2024-04-25T10:07:29.584664+0000","last_undegraded":"2024-04-25T10:07:29.584664+0000","last_fullsized":"2024-04-25T10:07:29.584664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:50.286556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586885+0000","last_change":"2024-04-25T10:07:28.595307+0000","last_active":"2024-04-25T10:07:29.586885+0000","last_peered":"2024-04-25T10:07:29.586885+0000","last_clean":"2024-04-25T10:07:29.586885+0000","last_became_active":"2024-04-25T10:07:28.594903+0000","last_became_peered":"2024-04-25T10:07:28.594903+0000","last_unstale":"2024-04-25T10:07:29.586885+0000","last_undegraded":"2024-04-25T10:07:29.586885+0000","last_fullsized":"2024-04-25T10:07:29.586885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:31.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.580673+0000","last_change":"2024-04-25T10:07:28.596023+0000","last_active":"2024-04-25T10:07:29.580673+0000","last_peered":"2024-04-25T10:07:29.580673+0000","last_clean":"2024-04-25T10:07:29.580673+0000","last_became_active":"2024-04-25T10:07:28.595666+0000","last_became_peered":"2024-04-25T10:07:28.595666+0000","last_unstale":"2024-04-25T10:07:29.580673+0000","last_undegraded":"2024-04-25T10:07:29.580673+0000","last_fullsized":"2024-04-25T10:07:29.580673+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:19.159482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.981059+0000","last_change":"2024-04-25T10:07:28.575621+0000","last_active":"2024-04-25T10:07:29.981059+0000","last_peered":"2024-04-25T10:07:29.981059+0000","last_clean":"2024-04-25T10:07:29.981059+0000","last_became_active":"2024-04-25T10:07:28.575297+0000","last_became_peered":"2024-04-25T10:07:28.575297+0000","last_unstale":"2024-04-25T10:07:29.981059+0000","last_undegraded":"2024-04-25T10:07:29.981059+0000","last_fullsized":"2024-04-25T10:07:29.981059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:12.091515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976925+0000","last_change":"2024-04-25T10:07:28.581544+0000","last_active":"2024-04-25T10:07:29.976925+0000","last_peered":"2024-04-25T10:07:29.976925+0000","last_clean":"2024-04-25T10:07:29.976925+0000","last_became_active":"2024-04-25T10:07:28.581184+0000","last_became_peered":"2024-04-25T10:07:28.581184+0000","last_unstale":"2024-04-25T10:07:29.976925+0000","last_undegraded":"2024-04-25T10:07:29.976925+0000","last_fullsized":"2024-04-25T10:07:29.976925+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:37.207640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585204+0000","last_change":"2024-04-25T10:07:28.600844+0000","last_active":"2024-04-25T10:07:29.585204+0000","last_peered":"2024-04-25T10:07:29.585204+0000","last_clean":"2024-04-25T10:07:29.585204+0000","last_became_active":"2024-04-25T10:07:28.599836+0000","last_became_peered":"2024-04-25T10:07:28.599836+0000","last_unstale":"2024-04-25T10:07:29.585204+0000","last_undegraded":"2024-04-25T10:07:29.585204+0000","last_fullsized":"2024-04-25T10:07:29.585204+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:10.865465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977011+0000","last_change":"2024-04-25T10:07:28.590691+0000","last_active":"2024-04-25T10:07:29.977011+0000","last_peered":"2024-04-25T10:07:29.977011+0000","last_clean":"2024-04-25T10:07:29.977011+0000","last_became_active":"2024-04-25T10:07:28.590387+0000","last_became_peered":"2024-04-25T10:07:28.590387+0000","last_unstale":"2024-04-25T10:07:29.977011+0000","last_undegraded":"2024-04-25T10:07:29.977011+0000","last_fullsized":"2024-04-25T10:07:29.977011+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:25.053354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598659+0000","last_change":"2024-04-25T10:07:28.585670+0000","last_active":"2024-04-25T10:07:29.598659+0000","last_peered":"2024-04-25T10:07:29.598659+0000","last_clean":"2024-04-25T10:07:29.598659+0000","last_became_active":"2024-04-25T10:07:28.585343+0000","last_became_peered":"2024-04-25T10:07:28.585343+0000","last_unstale":"2024-04-25T10:07:29.598659+0000","last_undegraded":"2024-04-25T10:07:29.598659+0000","last_fullsized":"2024-04-25T10:07:29.598659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:40.197673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975914+0000","last_change":"2024-04-25T10:07:28.598238+0000","last_active":"2024-04-25T10:07:29.975914+0000","last_peered":"2024-04-25T10:07:29.975914+0000","last_clean":"2024-04-25T10:07:29.975914+0000","last_became_active":"2024-04-25T10:07:28.597962+0000","last_became_peered":"2024-04-25T10:07:28.597962+0000","last_unstale":"2024-04-25T10:07:29.975914+0000","last_undegraded":"2024-04-25T10:07:29.975914+0000","last_fullsized":"2024-04-25T10:07:29.975914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:30.539972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975952+0000","last_change":"2024-04-25T10:07:28.596707+0000","last_active":"2024-04-25T10:07:29.975952+0000","last_peered":"2024-04-25T10:07:29.975952+0000","last_clean":"2024-04-25T10:07:29.975952+0000","last_became_active":"2024-04-25T10:07:28.596414+0000","last_became_peered":"2024-04-25T10:07:28.596414+0000","last_unstale":"2024-04-25T10:07:29.975952+0000","last_undegraded":"2024-04-25T10:07:29.975952+0000","last_fullsized":"2024-04-25T10:07:29.975952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.737727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584527+0000","last_change":"2024-04-25T10:07:28.591555+0000","last_active":"2024-04-25T10:07:29.584527+0000","last_peered":"2024-04-25T10:07:29.584527+0000","last_clean":"2024-04-25T10:07:29.584527+0000","last_became_active":"2024-04-25T10:07:28.590204+0000","last_became_peered":"2024-04-25T10:07:28.590204+0000","last_unstale":"2024-04-25T10:07:29.584527+0000","last_undegraded":"2024-04-25T10:07:29.584527+0000","last_fullsized":"2024-04-25T10:07:29.584527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:40.448447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598929+0000","last_change":"2024-04-25T10:07:28.602552+0000","last_active":"2024-04-25T10:07:29.598929+0000","last_peered":"2024-04-25T10:07:29.598929+0000","last_clean":"2024-04-25T10:07:29.598929+0000","last_became_active":"2024-04-25T10:07:28.600662+0000","last_became_peered":"2024-04-25T10:07:28.600662+0000","last_unstale":"2024-04-25T10:07:29.598929+0000","last_undegraded":"2024-04-25T10:07:29.598929+0000","last_fullsized":"2024-04-25T10:07:29.598929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:06.950405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598861+0000","last_change":"2024-04-25T10:07:28.599332+0000","last_active":"2024-04-25T10:07:29.598861+0000","last_peered":"2024-04-25T10:07:29.598861+0000","last_clean":"2024-04-25T10:07:29.598861+0000","last_became_active":"2024-04-25T10:07:28.599019+0000","last_became_peered":"2024-04-25T10:07:28.599019+0000","last_unstale":"2024-04-25T10:07:29.598861+0000","last_undegraded":"2024-04-25T10:07:29.598861+0000","last_fullsized":"2024-04-25T10:07:29.598861+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:24.644589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976302+0000","last_change":"2024-04-25T10:07:28.601954+0000","last_active":"2024-04-25T10:07:29.976302+0000","last_peered":"2024-04-25T10:07:29.976302+0000","last_clean":"2024-04-25T10:07:29.976302+0000","last_became_active":"2024-04-25T10:07:28.601624+0000","last_became_peered":"2024-04-25T10:07:28.601624+0000","last_unstale":"2024-04-25T10:07:29.976302+0000","last_undegraded":"2024-04-25T10:07:29.976302+0000","last_fullsized":"2024-04-25T10:07:29.976302+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:22.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980406+0000","last_change":"2024-04-25T10:07:28.582929+0000","last_active":"2024-04-25T10:07:29.980406+0000","last_peered":"2024-04-25T10:07:29.980406+0000","last_clean":"2024-04-25T10:07:29.980406+0000","last_became_active":"2024-04-25T10:07:28.579625+0000","last_became_peered":"2024-04-25T10:07:28.579625+0000","last_unstale":"2024-04-25T10:07:29.980406+0000","last_undegraded":"2024-04-25T10:07:29.980406+0000","last_fullsized":"2024-04-25T10:07:29.980406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:48.018677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583363+0000","last_change":"2024-04-25T10:07:28.593823+0000","last_active":"2024-04-25T10:07:29.583363+0000","last_peered":"2024-04-25T10:07:29.583363+0000","last_clean":"2024-04-25T10:07:29.583363+0000","last_became_active":"2024-04-25T10:07:28.593541+0000","last_became_peered":"2024-04-25T10:07:28.593541+0000","last_unstale":"2024-04-25T10:07:29.583363+0000","last_undegraded":"2024-04-25T10:07:29.583363+0000","last_fullsized":"2024-04-25T10:07:29.583363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:06.950405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.597981+0000","last_change":"2024-04-25T10:07:28.585452+0000","last_active":"2024-04-25T10:07:29.597981+0000","last_peered":"2024-04-25T10:07:29.597981+0000","last_clean":"2024-04-25T10:07:29.597981+0000","last_became_active":"2024-04-25T10:07:28.584761+0000","last_became_peered":"2024-04-25T10:07:28.584761+0000","last_unstale":"2024-04-25T10:07:29.597981+0000","last_undegraded":"2024-04-25T10:07:29.597981+0000","last_fullsized":"2024-04-25T10:07:29.597981+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:45.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976343+0000","last_change":"2024-04-25T10:07:28.601450+0000","last_active":"2024-04-25T10:07:29.976343+0000","last_peered":"2024-04-25T10:07:29.976343+0000","last_clean":"2024-04-25T10:07:29.976343+0000","last_became_active":"2024-04-25T10:07:28.598645+0000","last_became_peered":"2024-04-25T10:07:28.598645+0000","last_unstale":"2024-04-25T10:07:29.976343+0000","last_undegraded":"2024-04-25T10:07:29.976343+0000","last_fullsized":"2024-04-25T10:07:29.976343+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:48.051609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976078+0000","last_change":"2024-04-25T10:07:28.599283+0000","last_active":"2024-04-25T10:07:29.976078+0000","last_peered":"2024-04-25T10:07:29.976078+0000","last_clean":"2024-04-25T10:07:29.976078+0000","last_became_active":"2024-04-25T10:07:28.598962+0000","last_became_peered":"2024-04-25T10:07:28.598962+0000","last_unstale":"2024-04-25T10:07:29.976078+0000","last_undegraded":"2024-04-25T10:07:29.976078+0000","last_fullsized":"2024-04-25T10:07:29.976078+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:31.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584833+0000","last_change":"2024-04-25T10:07:28.582305+0000","last_active":"2024-04-25T10:07:29.584833+0000","last_peered":"2024-04-25T10:07:29.584833+0000","last_clean":"2024-04-25T10:07:29.584833+0000","last_became_active":"2024-04-25T10:07:28.581892+0000","last_became_peered":"2024-04-25T10:07:28.581892+0000","last_unstale":"2024-04-25T10:07:29.584833+0000","last_undegraded":"2024-04-25T10:07:29.584833+0000","last_fullsized":"2024-04-25T10:07:29.584833+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:28.570344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.981096+0000","last_change":"2024-04-25T10:07:27.581044+0000","last_active":"2024-04-25T10:07:29.981096+0000","last_peered":"2024-04-25T10:07:29.981096+0000","last_clean":"2024-04-25T10:07:29.981096+0000","last_became_active":"2024-04-25T10:07:27.580782+0000","last_became_peered":"2024-04-25T10:07:27.580782+0000","last_unstale":"2024-04-25T10:07:29.981096+0000","last_undegraded":"2024-04-25T10:07:29.981096+0000","last_fullsized":"2024-04-25T10:07:29.981096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:47.654284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976694+0000","last_change":"2024-04-25T10:07:28.587534+0000","last_active":"2024-04-25T10:07:29.976694+0000","last_peered":"2024-04-25T10:07:29.976694+0000","last_clean":"2024-04-25T10:07:29.976694+0000","last_became_active":"2024-04-25T10:07:28.584511+0000","last_became_peered":"2024-04-25T10:07:28.584511+0000","last_unstale":"2024-04-25T10:07:29.976694+0000","last_undegraded":"2024-04-25T10:07:29.976694+0000","last_fullsized":"2024-04-25T10:07:29.976694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:20.873378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.979563+0000","last_change":"2024-04-25T10:07:27.580646+0000","last_active":"2024-04-25T10:07:29.979563+0000","last_peered":"2024-04-25T10:07:29.979563+0000","last_clean":"2024-04-25T10:07:29.979563+0000","last_became_active":"2024-04-25T10:07:27.580255+0000","last_became_peered":"2024-04-25T10:07:27.580255+0000","last_unstale":"2024-04-25T10:07:29.979563+0000","last_undegraded":"2024-04-25T10:07:29.979563+0000","last_fullsized":"2024-04-25T10:07:29.979563+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:23.630789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.979371+0000","last_change":"2024-04-25T10:07:28.599327+0000","last_active":"2024-04-25T10:07:29.979371+0000","last_peered":"2024-04-25T10:07:29.979371+0000","last_clean":"2024-04-25T10:07:29.979371+0000","last_became_active":"2024-04-25T10:07:28.599041+0000","last_became_peered":"2024-04-25T10:07:28.599041+0000","last_unstale":"2024-04-25T10:07:29.979371+0000","last_undegraded":"2024-04-25T10:07:29.979371+0000","last_fullsized":"2024-04-25T10:07:29.979371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.737727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978380+0000","last_change":"2024-04-25T10:07:27.585009+0000","last_active":"2024-04-25T10:07:29.978380+0000","last_peered":"2024-04-25T10:07:29.978380+0000","last_clean":"2024-04-25T10:07:29.978380+0000","last_became_active":"2024-04-25T10:07:27.584679+0000","last_became_peered":"2024-04-25T10:07:27.584679+0000","last_unstale":"2024-04-25T10:07:29.978380+0000","last_undegraded":"2024-04-25T10:07:29.978380+0000","last_fullsized":"2024-04-25T10:07:29.978380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:24.313362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586571+0000","last_change":"2024-04-25T10:07:28.604855+0000","last_active":"2024-04-25T10:07:29.586571+0000","last_peered":"2024-04-25T10:07:29.586571+0000","last_clean":"2024-04-25T10:07:29.586571+0000","last_became_active":"2024-04-25T10:07:28.604524+0000","last_became_peered":"2024-04-25T10:07:28.604524+0000","last_unstale":"2024-04-25T10:07:29.586571+0000","last_undegraded":"2024-04-25T10:07:29.586571+0000","last_fullsized":"2024-04-25T10:07:29.586571+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:50.372337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978242+0000","last_change":"2024-04-25T10:07:27.572839+0000","last_active":"2024-04-25T10:07:29.978242+0000","last_peered":"2024-04-25T10:07:29.978242+0000","last_clean":"2024-04-25T10:07:29.978242+0000","last_became_active":"2024-04-25T10:07:27.568223+0000","last_became_peered":"2024-04-25T10:07:27.568223+0000","last_unstale":"2024-04-25T10:07:29.978242+0000","last_undegraded":"2024-04-25T10:07:29.978242+0000","last_fullsized":"2024-04-25T10:07:29.978242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:45.452132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.580986+0000","last_change":"2024-04-25T10:07:28.590600+0000","last_active":"2024-04-25T10:07:29.580986+0000","last_peered":"2024-04-25T10:07:29.580986+0000","last_clean":"2024-04-25T10:07:29.580986+0000","last_became_active":"2024-04-25T10:07:28.578992+0000","last_became_peered":"2024-04-25T10:07:28.578992+0000","last_unstale":"2024-04-25T10:07:29.580986+0000","last_undegraded":"2024-04-25T10:07:29.580986+0000","last_fullsized":"2024-04-25T10:07:29.580986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:36.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976635+0000","last_change":"2024-04-25T10:07:27.570859+0000","last_active":"2024-04-25T10:07:29.976635+0000","last_peered":"2024-04-25T10:07:29.976635+0000","last_clean":"2024-04-25T10:07:29.976635+0000","last_became_active":"2024-04-25T10:07:27.570549+0000","last_became_peered":"2024-04-25T10:07:27.570549+0000","last_unstale":"2024-04-25T10:07:29.976635+0000","last_undegraded":"2024-04-25T10:07:29.976635+0000","last_fullsized":"2024-04-25T10:07:29.976635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:49.272756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978998+0000","last_change":"2024-04-25T10:07:28.568365+0000","last_active":"2024-04-25T10:07:29.978998+0000","last_peered":"2024-04-25T10:07:29.978998+0000","last_clean":"2024-04-25T10:07:29.978998+0000","last_became_active":"2024-04-25T10:07:28.567439+0000","last_became_peered":"2024-04-25T10:07:28.567439+0000","last_unstale":"2024-04-25T10:07:29.978998+0000","last_undegraded":"2024-04-25T10:07:29.978998+0000","last_fullsized":"2024-04-25T10:07:29.978998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:46.019029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976061+0000","last_change":"2024-04-25T10:07:27.576587+0000","last_active":"2024-04-25T10:07:29.976061+0000","last_peered":"2024-04-25T10:07:29.976061+0000","last_clean":"2024-04-25T10:07:29.976061+0000","last_became_active":"2024-04-25T10:07:27.576051+0000","last_became_peered":"2024-04-25T10:07:27.576051+0000","last_unstale":"2024-04-25T10:07:29.976061+0000","last_undegraded":"2024-04-25T10:07:29.976061+0000","last_fullsized":"2024-04-25T10:07:29.976061+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:24.671799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.582811+0000","last_change":"2024-04-25T10:07:28.579904+0000","last_active":"2024-04-25T10:07:29.582811+0000","last_peered":"2024-04-25T10:07:29.582811+0000","last_clean":"2024-04-25T10:07:29.582811+0000","last_became_active":"2024-04-25T10:07:28.579217+0000","last_became_peered":"2024-04-25T10:07:28.579217+0000","last_unstale":"2024-04-25T10:07:29.582811+0000","last_undegraded":"2024-04-25T10:07:29.582811+0000","last_fullsized":"2024-04-25T10:07:29.582811+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:20.873378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975970+0000","last_change":"2024-04-25T10:07:27.575359+0000","last_active":"2024-04-25T10:07:29.975970+0000","last_peered":"2024-04-25T10:07:29.975970+0000","last_clean":"2024-04-25T10:07:29.975970+0000","last_became_active":"2024-04-25T10:07:27.575069+0000","last_became_peered":"2024-04-25T10:07:27.575069+0000","last_unstale":"2024-04-25T10:07:29.975970+0000","last_undegraded":"2024-04-25T10:07:29.975970+0000","last_fullsized":"2024-04-25T10:07:29.975970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:23.630789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976492+0000","last_change":"2024-04-25T10:07:28.593148+0000","last_active":"2024-04-25T10:07:29.976492+0000","last_peered":"2024-04-25T10:07:29.976492+0000","last_clean":"2024-04-25T10:07:29.976492+0000","last_became_active":"2024-04-25T10:07:28.591511+0000","last_became_peered":"2024-04-25T10:07:28.591511+0000","last_unstale":"2024-04-25T10:07:29.976492+0000","last_undegraded":"2024-04-25T10:07:29.976492+0000","last_fullsized":"2024-04-25T10:07:29.976492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:23.608303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.582482+0000","last_change":"2024-04-25T10:07:27.580357+0000","last_active":"2024-04-25T10:07:29.582482+0000","last_peered":"2024-04-25T10:07:29.582482+0000","last_clean":"2024-04-25T10:07:29.582482+0000","last_became_active":"2024-04-25T10:07:27.579854+0000","last_became_peered":"2024-04-25T10:07:27.579854+0000","last_unstale":"2024-04-25T10:07:29.582482+0000","last_undegraded":"2024-04-25T10:07:29.582482+0000","last_fullsized":"2024-04-25T10:07:29.582482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:28.515162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980530+0000","last_change":"2024-04-25T10:07:28.601898+0000","last_active":"2024-04-25T10:07:29.980530+0000","last_peered":"2024-04-25T10:07:29.980530+0000","last_clean":"2024-04-25T10:07:29.980530+0000","last_became_active":"2024-04-25T10:07:28.601567+0000","last_became_peered":"2024-04-25T10:07:28.601567+0000","last_unstale":"2024-04-25T10:07:29.980530+0000","last_undegraded":"2024-04-25T10:07:29.980530+0000","last_fullsized":"2024-04-25T10:07:29.980530+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:20.597644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586722+0000","last_change":"2024-04-25T10:07:27.586327+0000","last_active":"2024-04-25T10:07:29.586722+0000","last_peered":"2024-04-25T10:07:29.586722+0000","last_clean":"2024-04-25T10:07:29.586722+0000","last_became_active":"2024-04-25T10:07:27.585969+0000","last_became_peered":"2024-04-25T10:07:27.585969+0000","last_unstale":"2024-04-25T10:07:29.586722+0000","last_undegraded":"2024-04-25T10:07:29.586722+0000","last_fullsized":"2024-04-25T10:07:29.586722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.499620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976419+0000","last_change":"2024-04-25T10:07:28.583290+0000","last_active":"2024-04-25T10:07:29.976419+0000","last_peered":"2024-04-25T10:07:29.976419+0000","last_clean":"2024-04-25T10:07:29.976419+0000","last_became_active":"2024-04-25T10:07:28.582971+0000","last_became_peered":"2024-04-25T10:07:28.582971+0000","last_unstale":"2024-04-25T10:07:29.976419+0000","last_undegraded":"2024-04-25T10:07:29.976419+0000","last_fullsized":"2024-04-25T10:07:29.976419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:29.351396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977110+0000","last_change":"2024-04-25T10:07:27.572448+0000","last_active":"2024-04-25T10:07:29.977110+0000","last_peered":"2024-04-25T10:07:29.977110+0000","last_clean":"2024-04-25T10:07:29.977110+0000","last_became_active":"2024-04-25T10:07:27.572132+0000","last_became_peered":"2024-04-25T10:07:27.572132+0000","last_unstale":"2024-04-25T10:07:29.977110+0000","last_undegraded":"2024-04-25T10:07:29.977110+0000","last_fullsized":"2024-04-25T10:07:29.977110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:44.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585715+0000","last_change":"2024-04-25T10:07:28.579192+0000","last_active":"2024-04-25T10:07:29.585715+0000","last_peered":"2024-04-25T10:07:29.585715+0000","last_clean":"2024-04-25T10:07:29.585715+0000","last_became_active":"2024-04-25T10:07:28.578820+0000","last_became_peered":"2024-04-25T10:07:28.578820+0000","last_unstale":"2024-04-25T10:07:29.585715+0000","last_undegraded":"2024-04-25T10:07:29.585715+0000","last_fullsized":"2024-04-25T10:07:29.585715+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:27.615095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585788+0000","last_change":"2024-04-25T10:07:27.587729+0000","last_active":"2024-04-25T10:07:29.585788+0000","last_peered":"2024-04-25T10:07:29.585788+0000","last_clean":"2024-04-25T10:07:29.585788+0000","last_became_active":"2024-04-25T10:07:27.587355+0000","last_became_peered":"2024-04-25T10:07:27.587355+0000","last_unstale":"2024-04-25T10:07:29.585788+0000","last_undegraded":"2024-04-25T10:07:29.585788+0000","last_fullsized":"2024-04-25T10:07:29.585788+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:24.812858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977004+0000","last_change":"2024-04-25T10:07:28.587939+0000","last_active":"2024-04-25T10:07:29.977004+0000","last_peered":"2024-04-25T10:07:29.977004+0000","last_clean":"2024-04-25T10:07:29.977004+0000","last_became_active":"2024-04-25T10:07:28.587588+0000","last_became_peered":"2024-04-25T10:07:28.587588+0000","last_unstale":"2024-04-25T10:07:29.977004+0000","last_undegraded":"2024-04-25T10:07:29.977004+0000","last_fullsized":"2024-04-25T10:07:29.977004+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.668084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980683+0000","last_change":"2024-04-25T10:07:27.577743+0000","last_active":"2024-04-25T10:07:29.980683+0000","last_peered":"2024-04-25T10:07:29.980683+0000","last_clean":"2024-04-25T10:07:29.980683+0000","last_became_active":"2024-04-25T10:07:27.577434+0000","last_became_peered":"2024-04-25T10:07:27.577434+0000","last_unstale":"2024-04-25T10:07:29.980683+0000","last_undegraded":"2024-04-25T10:07:29.980683+0000","last_fullsized":"2024-04-25T10:07:29.980683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:53.268246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976051+0000","last_change":"2024-04-25T10:07:28.586113+0000","last_active":"2024-04-25T10:07:29.976051+0000","last_peered":"2024-04-25T10:07:29.976051+0000","last_clean":"2024-04-25T10:07:29.976051+0000","last_became_active":"2024-04-25T10:07:28.584757+0000","last_became_peered":"2024-04-25T10:07:28.584757+0000","last_unstale":"2024-04-25T10:07:29.976051+0000","last_undegraded":"2024-04-25T10:07:29.976051+0000","last_fullsized":"2024-04-25T10:07:29.976051+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.085046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583475+0000","last_change":"2024-04-25T10:07:27.582455+0000","last_active":"2024-04-25T10:07:29.583475+0000","last_peered":"2024-04-25T10:07:29.583475+0000","last_clean":"2024-04-25T10:07:29.583475+0000","last_became_active":"2024-04-25T10:07:27.582140+0000","last_became_peered":"2024-04-25T10:07:27.582140+0000","last_unstale":"2024-04-25T10:07:29.583475+0000","last_undegraded":"2024-04-25T10:07:29.583475+0000","last_fullsized":"2024-04-25T10:07:29.583475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:11.943908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598034+0000","last_change":"2024-04-25T10:07:28.583474+0000","last_active":"2024-04-25T10:07:29.598034+0000","last_peered":"2024-04-25T10:07:29.598034+0000","last_clean":"2024-04-25T10:07:29.598034+0000","last_became_active":"2024-04-25T10:07:28.583149+0000","last_became_peered":"2024-04-25T10:07:28.583149+0000","last_unstale":"2024-04-25T10:07:29.598034+0000","last_undegraded":"2024-04-25T10:07:29.598034+0000","last_fullsized":"2024-04-25T10:07:29.598034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:21.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598034+0000","last_change":"2024-04-25T10:07:27.583333+0000","last_active":"2024-04-25T10:07:29.598034+0000","last_peered":"2024-04-25T10:07:29.598034+0000","last_clean":"2024-04-25T10:07:29.598034+0000","last_became_active":"2024-04-25T10:07:27.578437+0000","last_became_peered":"2024-04-25T10:07:27.578437+0000","last_unstale":"2024-04-25T10:07:29.598034+0000","last_undegraded":"2024-04-25T10:07:29.598034+0000","last_fullsized":"2024-04-25T10:07:29.598034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:54.049709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583757+0000","last_change":"2024-04-25T10:07:28.597640+0000","last_active":"2024-04-25T10:07:29.583757+0000","last_peered":"2024-04-25T10:07:29.583757+0000","last_clean":"2024-04-25T10:07:29.583757+0000","last_became_active":"2024-04-25T10:07:28.597332+0000","last_became_peered":"2024-04-25T10:07:28.597332+0000","last_unstale":"2024-04-25T10:07:29.583757+0000","last_undegraded":"2024-04-25T10:07:29.583757+0000","last_fullsized":"2024-04-25T10:07:29.583757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:37.207640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.587067+0000","last_change":"2024-04-25T10:07:27.588103+0000","last_active":"2024-04-25T10:07:29.587067+0000","last_peered":"2024-04-25T10:07:29.587067+0000","last_clean":"2024-04-25T10:07:29.587067+0000","last_became_active":"2024-04-25T10:07:27.587749+0000","last_became_peered":"2024-04-25T10:07:27.587749+0000","last_unstale":"2024-04-25T10:07:29.587067+0000","last_undegraded":"2024-04-25T10:07:29.587067+0000","last_fullsized":"2024-04-25T10:07:29.587067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:57.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980165+0000","last_change":"2024-04-25T10:07:28.589567+0000","last_active":"2024-04-25T10:07:29.980165+0000","last_peered":"2024-04-25T10:07:29.980165+0000","last_clean":"2024-04-25T10:07:29.980165+0000","last_became_active":"2024-04-25T10:07:28.588738+0000","last_became_peered":"2024-04-25T10:07:28.588738+0000","last_unstale":"2024-04-25T10:07:29.980165+0000","last_undegraded":"2024-04-25T10:07:29.980165+0000","last_fullsized":"2024-04-25T10:07:29.980165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:15.675146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975412+0000","last_change":"2024-04-25T10:07:27.582838+0000","last_active":"2024-04-25T10:07:29.975412+0000","last_peered":"2024-04-25T10:07:29.975412+0000","last_clean":"2024-04-25T10:07:29.975412+0000","last_became_active":"2024-04-25T10:07:27.581144+0000","last_became_peered":"2024-04-25T10:07:27.581144+0000","last_unstale":"2024-04-25T10:07:29.975412+0000","last_undegraded":"2024-04-25T10:07:29.975412+0000","last_fullsized":"2024-04-25T10:07:29.975412+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:30.684965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585186+0000","last_change":"2024-04-25T10:07:28.592638+0000","last_active":"2024-04-25T10:07:29.585186+0000","last_peered":"2024-04-25T10:07:29.585186+0000","last_clean":"2024-04-25T10:07:29.585186+0000","last_became_active":"2024-04-25T10:07:28.592318+0000","last_became_peered":"2024-04-25T10:07:28.592318+0000","last_unstale":"2024-04-25T10:07:29.585186+0000","last_undegraded":"2024-04-25T10:07:29.585186+0000","last_fullsized":"2024-04-25T10:07:29.585186+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:36.477385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598801+0000","last_change":"2024-04-25T10:07:27.580589+0000","last_active":"2024-04-25T10:07:29.598801+0000","last_peered":"2024-04-25T10:07:29.598801+0000","last_clean":"2024-04-25T10:07:29.598801+0000","last_became_active":"2024-04-25T10:07:27.578437+0000","last_became_peered":"2024-04-25T10:07:27.578437+0000","last_unstale":"2024-04-25T10:07:29.598801+0000","last_undegraded":"2024-04-25T10:07:29.598801+0000","last_fullsized":"2024-04-25T10:07:29.598801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:53.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585647+0000","last_change":"2024-04-25T10:07:28.577018+0000","last_active":"2024-04-25T10:07:29.585647+0000","last_peered":"2024-04-25T10:07:29.585647+0000","last_clean":"2024-04-25T10:07:29.585647+0000","last_became_active":"2024-04-25T10:07:28.575703+0000","last_became_peered":"2024-04-25T10:07:28.575703+0000","last_unstale":"2024-04-25T10:07:29.585647+0000","last_undegraded":"2024-04-25T10:07:29.585647+0000","last_fullsized":"2024-04-25T10:07:29.585647+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:21.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584192+0000","last_change":"2024-04-25T10:07:27.584098+0000","last_active":"2024-04-25T10:07:29.584192+0000","last_peered":"2024-04-25T10:07:29.584192+0000","last_clean":"2024-04-25T10:07:29.584192+0000","last_became_active":"2024-04-25T10:07:27.583801+0000","last_became_peered":"2024-04-25T10:07:27.583801+0000","last_unstale":"2024-04-25T10:07:29.584192+0000","last_undegraded":"2024-04-25T10:07:29.584192+0000","last_fullsized":"2024-04-25T10:07:29.584192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:40.631123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976928+0000","last_change":"2024-04-25T10:07:28.579045+0000","last_active":"2024-04-25T10:07:29.976928+0000","last_peered":"2024-04-25T10:07:29.976928+0000","last_clean":"2024-04-25T10:07:29.976928+0000","last_became_active":"2024-04-25T10:07:28.576958+0000","last_became_peered":"2024-04-25T10:07:28.576958+0000","last_unstale":"2024-04-25T10:07:29.976928+0000","last_undegraded":"2024-04-25T10:07:29.976928+0000","last_fullsized":"2024-04-25T10:07:29.976928+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.256788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586068+0000","last_change":"2024-04-25T10:07:27.585602+0000","last_active":"2024-04-25T10:07:29.586068+0000","last_peered":"2024-04-25T10:07:29.586068+0000","last_clean":"2024-04-25T10:07:29.586068+0000","last_became_active":"2024-04-25T10:07:27.585337+0000","last_became_peered":"2024-04-25T10:07:27.585337+0000","last_unstale":"2024-04-25T10:07:29.586068+0000","last_undegraded":"2024-04-25T10:07:29.586068+0000","last_fullsized":"2024-04-25T10:07:29.586068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:11.077715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584956+0000","last_change":"2024-04-25T10:07:28.592281+0000","last_active":"2024-04-25T10:07:29.584956+0000","last_peered":"2024-04-25T10:07:29.584956+0000","last_clean":"2024-04-25T10:07:29.584956+0000","last_became_active":"2024-04-25T10:07:28.591984+0000","last_became_peered":"2024-04-25T10:07:28.591984+0000","last_unstale":"2024-04-25T10:07:29.584956+0000","last_undegraded":"2024-04-25T10:07:29.584956+0000","last_fullsized":"2024-04-25T10:07:29.584956+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:21.511828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977147+0000","last_change":"2024-04-25T10:07:27.576141+0000","last_active":"2024-04-25T10:07:29.977147+0000","last_peered":"2024-04-25T10:07:29.977147+0000","last_clean":"2024-04-25T10:07:29.977147+0000","last_became_active":"2024-04-25T10:07:27.575875+0000","last_became_peered":"2024-04-25T10:07:27.575875+0000","last_unstale":"2024-04-25T10:07:29.977147+0000","last_undegraded":"2024-04-25T10:07:29.977147+0000","last_fullsized":"2024-04-25T10:07:29.977147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:35.463585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976045+0000","last_change":"2024-04-25T10:07:28.594751+0000","last_active":"2024-04-25T10:07:29.976045+0000","last_peered":"2024-04-25T10:07:29.976045+0000","last_clean":"2024-04-25T10:07:29.976045+0000","last_became_active":"2024-04-25T10:07:28.594372+0000","last_became_peered":"2024-04-25T10:07:28.594372+0000","last_unstale":"2024-04-25T10:07:29.976045+0000","last_undegraded":"2024-04-25T10:07:29.976045+0000","last_fullsized":"2024-04-25T10:07:29.976045+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:04.011580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977591+0000","last_change":"2024-04-25T10:07:27.570293+0000","last_active":"2024-04-25T10:07:29.977591+0000","last_peered":"2024-04-25T10:07:29.977591+0000","last_clean":"2024-04-25T10:07:29.977591+0000","last_became_active":"2024-04-25T10:07:27.569996+0000","last_became_peered":"2024-04-25T10:07:27.569996+0000","last_unstale":"2024-04-25T10:07:29.977591+0000","last_undegraded":"2024-04-25T10:07:29.977591+0000","last_fullsized":"2024-04-25T10:07:29.977591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:13.952443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598067+0000","last_change":"2024-04-25T10:07:28.599733+0000","last_active":"2024-04-25T10:07:29.598067+0000","last_peered":"2024-04-25T10:07:29.598067+0000","last_clean":"2024-04-25T10:07:29.598067+0000","last_became_active":"2024-04-25T10:07:28.599451+0000","last_became_peered":"2024-04-25T10:07:28.599451+0000","last_unstale":"2024-04-25T10:07:29.598067+0000","last_undegraded":"2024-04-25T10:07:29.598067+0000","last_fullsized":"2024-04-25T10:07:29.598067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:28.328551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598070+0000","last_change":"2024-04-25T10:07:27.579007+0000","last_active":"2024-04-25T10:07:29.598070+0000","last_peered":"2024-04-25T10:07:29.598070+0000","last_clean":"2024-04-25T10:07:29.598070+0000","last_became_active":"2024-04-25T10:07:27.578695+0000","last_became_peered":"2024-04-25T10:07:27.578695+0000","last_unstale":"2024-04-25T10:07:29.598070+0000","last_undegraded":"2024-04-25T10:07:29.598070+0000","last_fullsized":"2024-04-25T10:07:29.598070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:24.171984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583208+0000","last_change":"2024-04-25T10:07:28.580131+0000","last_active":"2024-04-25T10:07:29.583208+0000","last_peered":"2024-04-25T10:07:29.583208+0000","last_clean":"2024-04-25T10:07:29.583208+0000","last_became_active":"2024-04-25T10:07:28.579622+0000","last_became_peered":"2024-04-25T10:07:28.579622+0000","last_unstale":"2024-04-25T10:07:29.583208+0000","last_undegraded":"2024-04-25T10:07:29.583208+0000","last_fullsized":"2024-04-25T10:07:29.583208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.256788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977355+0000","last_change":"2024-04-25T10:07:27.580265+0000","last_active":"2024-04-25T10:07:29.977355+0000","last_peered":"2024-04-25T10:07:29.977355+0000","last_clean":"2024-04-25T10:07:29.977355+0000","last_became_active":"2024-04-25T10:07:27.579936+0000","last_became_peered":"2024-04-25T10:07:27.579936+0000","last_unstale":"2024-04-25T10:07:29.977355+0000","last_undegraded":"2024-04-25T10:07:29.977355+0000","last_fullsized":"2024-04-25T10:07:29.977355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:05.936605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978071+0000","last_change":"2024-04-25T10:07:28.576851+0000","last_active":"2024-04-25T10:07:29.978071+0000","last_peered":"2024-04-25T10:07:29.978071+0000","last_clean":"2024-04-25T10:07:29.978071+0000","last_became_active":"2024-04-25T10:07:28.576507+0000","last_became_peered":"2024-04-25T10:07:28.576507+0000","last_unstale":"2024-04-25T10:07:29.978071+0000","last_undegraded":"2024-04-25T10:07:29.978071+0000","last_fullsized":"2024-04-25T10:07:29.978071+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:08.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.599020+0000","last_change":"2024-04-25T10:07:27.583926+0000","last_active":"2024-04-25T10:07:29.599020+0000","last_peered":"2024-04-25T10:07:29.599020+0000","last_clean":"2024-04-25T10:07:29.599020+0000","last_became_active":"2024-04-25T10:07:27.583491+0000","last_became_peered":"2024-04-25T10:07:27.583491+0000","last_unstale":"2024-04-25T10:07:29.599020+0000","last_undegraded":"2024-04-25T10:07:29.599020+0000","last_fullsized":"2024-04-25T10:07:29.599020+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:14.661346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.581592+0000","last_change":"2024-04-25T10:07:28.597964+0000","last_active":"2024-04-25T10:07:29.581592+0000","last_peered":"2024-04-25T10:07:29.581592+0000","last_clean":"2024-04-25T10:07:29.581592+0000","last_became_active":"2024-04-25T10:07:28.597617+0000","last_became_peered":"2024-04-25T10:07:28.597617+0000","last_unstale":"2024-04-25T10:07:29.581592+0000","last_undegraded":"2024-04-25T10:07:29.581592+0000","last_fullsized":"2024-04-25T10:07:29.581592+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:00.953997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.599140+0000","last_change":"2024-04-25T10:07:27.584858+0000","last_active":"2024-04-25T10:07:29.599140+0000","last_peered":"2024-04-25T10:07:29.599140+0000","last_clean":"2024-04-25T10:07:29.599140+0000","last_became_active":"2024-04-25T10:07:27.584497+0000","last_became_peered":"2024-04-25T10:07:27.584497+0000","last_unstale":"2024-04-25T10:07:29.599140+0000","last_undegraded":"2024-04-25T10:07:29.599140+0000","last_fullsized":"2024-04-25T10:07:29.599140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:45.005229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598864+0000","last_change":"2024-04-25T10:07:28.584205+0000","last_active":"2024-04-25T10:07:29.598864+0000","last_peered":"2024-04-25T10:07:29.598864+0000","last_clean":"2024-04-25T10:07:29.598864+0000","last_became_active":"2024-04-25T10:07:28.583887+0000","last_became_peered":"2024-04-25T10:07:28.583887+0000","last_unstale":"2024-04-25T10:07:29.598864+0000","last_undegraded":"2024-04-25T10:07:29.598864+0000","last_fullsized":"2024-04-25T10:07:29.598864+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:46.465932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978787+0000","last_change":"2024-04-25T10:07:27.583573+0000","last_active":"2024-04-25T10:07:29.978787+0000","last_peered":"2024-04-25T10:07:29.978787+0000","last_clean":"2024-04-25T10:07:29.978787+0000","last_became_active":"2024-04-25T10:07:27.583265+0000","last_became_peered":"2024-04-25T10:07:27.583265+0000","last_unstale":"2024-04-25T10:07:29.978787+0000","last_undegraded":"2024-04-25T10:07:29.978787+0000","last_fullsized":"2024-04-25T10:07:29.978787+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:39.183873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585496+0000","last_change":"2024-04-25T10:07:28.586853+0000","last_active":"2024-04-25T10:07:29.585496+0000","last_peered":"2024-04-25T10:07:29.585496+0000","last_clean":"2024-04-25T10:07:29.585496+0000","last_became_active":"2024-04-25T10:07:28.586574+0000","last_became_peered":"2024-04-25T10:07:28.586574+0000","last_unstale":"2024-04-25T10:07:29.585496+0000","last_undegraded":"2024-04-25T10:07:29.585496+0000","last_fullsized":"2024-04-25T10:07:29.585496+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:38.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980737+0000","last_change":"2024-04-25T10:07:27.577452+0000","last_active":"2024-04-25T10:07:29.980737+0000","last_peered":"2024-04-25T10:07:29.980737+0000","last_clean":"2024-04-25T10:07:29.980737+0000","last_became_active":"2024-04-25T10:07:27.576605+0000","last_became_peered":"2024-04-25T10:07:27.576605+0000","last_unstale":"2024-04-25T10:07:29.980737+0000","last_undegraded":"2024-04-25T10:07:29.980737+0000","last_fullsized":"2024-04-25T10:07:29.980737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:36.193840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.582911+0000","last_change":"2024-04-25T10:07:28.573782+0000","last_active":"2024-04-25T10:07:29.582911+0000","last_peered":"2024-04-25T10:07:29.582911+0000","last_clean":"2024-04-25T10:07:29.582911+0000","last_became_active":"2024-04-25T10:07:28.573436+0000","last_became_peered":"2024-04-25T10:07:28.573436+0000","last_unstale":"2024-04-25T10:07:29.582911+0000","last_undegraded":"2024-04-25T10:07:29.582911+0000","last_fullsized":"2024-04-25T10:07:29.582911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:24.527924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583676+0000","last_change":"2024-04-25T10:07:27.588101+0000","last_active":"2024-04-25T10:07:29.583676+0000","last_peered":"2024-04-25T10:07:29.583676+0000","last_clean":"2024-04-25T10:07:29.583676+0000","last_became_active":"2024-04-25T10:07:27.587728+0000","last_became_peered":"2024-04-25T10:07:27.587728+0000","last_unstale":"2024-04-25T10:07:29.583676+0000","last_undegraded":"2024-04-25T10:07:29.583676+0000","last_fullsized":"2024-04-25T10:07:29.583676+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:17.921271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975987+0000","last_change":"2024-04-25T10:07:28.589137+0000","last_active":"2024-04-25T10:07:29.975987+0000","last_peered":"2024-04-25T10:07:29.975987+0000","last_clean":"2024-04-25T10:07:29.975987+0000","last_became_active":"2024-04-25T10:07:28.588798+0000","last_became_peered":"2024-04-25T10:07:28.588798+0000","last_unstale":"2024-04-25T10:07:29.975987+0000","last_undegraded":"2024-04-25T10:07:29.975987+0000","last_fullsized":"2024-04-25T10:07:29.975987+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:14.933191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976330+0000","last_change":"2024-04-25T10:07:27.576354+0000","last_active":"2024-04-25T10:07:29.976330+0000","last_peered":"2024-04-25T10:07:29.976330+0000","last_clean":"2024-04-25T10:07:29.976330+0000","last_became_active":"2024-04-25T10:07:27.575707+0000","last_became_peered":"2024-04-25T10:07:27.575707+0000","last_unstale":"2024-04-25T10:07:29.976330+0000","last_undegraded":"2024-04-25T10:07:29.976330+0000","last_fullsized":"2024-04-25T10:07:29.976330+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:39.434647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.979970+0000","last_change":"2024-04-25T10:07:28.571635+0000","last_active":"2024-04-25T10:07:29.979970+0000","last_peered":"2024-04-25T10:07:29.979970+0000","last_clean":"2024-04-25T10:07:29.979970+0000","last_became_active":"2024-04-25T10:07:28.567394+0000","last_became_peered":"2024-04-25T10:07:28.567394+0000","last_unstale":"2024-04-25T10:07:29.979970+0000","last_undegraded":"2024-04-25T10:07:29.979970+0000","last_fullsized":"2024-04-25T10:07:29.979970+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:54.057726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976083+0000","last_change":"2024-04-25T10:07:27.572815+0000","last_active":"2024-04-25T10:07:29.976083+0000","last_peered":"2024-04-25T10:07:29.976083+0000","last_clean":"2024-04-25T10:07:29.976083+0000","last_became_active":"2024-04-25T10:07:27.572521+0000","last_became_peered":"2024-04-25T10:07:27.572521+0000","last_unstale":"2024-04-25T10:07:29.976083+0000","last_undegraded":"2024-04-25T10:07:29.976083+0000","last_fullsized":"2024-04-25T10:07:29.976083+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:45.452132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.581119+0000","last_change":"2024-04-25T10:07:28.597349+0000","last_active":"2024-04-25T10:07:29.581119+0000","last_peered":"2024-04-25T10:07:29.581119+0000","last_clean":"2024-04-25T10:07:29.581119+0000","last_became_active":"2024-04-25T10:07:28.597048+0000","last_became_peered":"2024-04-25T10:07:28.597048+0000","last_unstale":"2024-04-25T10:07:29.581119+0000","last_undegraded":"2024-04-25T10:07:29.581119+0000","last_fullsized":"2024-04-25T10:07:29.581119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:56.019480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978475+0000","last_change":"2024-04-25T10:07:27.575306+0000","last_active":"2024-04-25T10:07:29.978475+0000","last_peered":"2024-04-25T10:07:29.978475+0000","last_clean":"2024-04-25T10:07:29.978475+0000","last_became_active":"2024-04-25T10:07:27.575032+0000","last_became_peered":"2024-04-25T10:07:27.575032+0000","last_unstale":"2024-04-25T10:07:29.978475+0000","last_undegraded":"2024-04-25T10:07:29.978475+0000","last_fullsized":"2024-04-25T10:07:29.978475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:00.242988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584196+0000","last_change":"2024-04-25T10:07:28.571851+0000","last_active":"2024-04-25T10:07:29.584196+0000","last_peered":"2024-04-25T10:07:29.584196+0000","last_clean":"2024-04-25T10:07:29.584196+0000","last_became_active":"2024-04-25T10:07:28.571478+0000","last_became_peered":"2024-04-25T10:07:28.571478+0000","last_unstale":"2024-04-25T10:07:29.584196+0000","last_undegraded":"2024-04-25T10:07:29.584196+0000","last_fullsized":"2024-04-25T10:07:29.584196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:54.282046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975449+0000","last_change":"2024-04-25T10:07:27.568474+0000","last_active":"2024-04-25T10:07:29.975449+0000","last_peered":"2024-04-25T10:07:29.975449+0000","last_clean":"2024-04-25T10:07:29.975449+0000","last_became_active":"2024-04-25T10:07:27.567961+0000","last_became_peered":"2024-04-25T10:07:27.567961+0000","last_unstale":"2024-04-25T10:07:29.975449+0000","last_undegraded":"2024-04-25T10:07:29.975449+0000","last_fullsized":"2024-04-25T10:07:29.975449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:05.936605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":262,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584880+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:07:29.584880+0000","last_peered":"2024-04-25T10:07:29.584880+0000","last_clean":"2024-04-25T10:07:29.584880+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:07:29.584880+0000","last_undegraded":"2024-04-25T10:07:29.584880+0000","last_fullsized":"2024-04-25T10:07:29.584880+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40904,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330936,"statfs":{"total":96636764160,"available":96594878464,"internally_reserved":0,"allocated":1433600,"data_stored":966515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40316,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331524,"statfs":{"total":96636764160,"available":96595480576,"internally_reserved":0,"allocated":827392,"data_stored":375405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36152,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335688,"statfs":{"total":96636764160,"available":96599744512,"internally_reserved":0,"allocated":827392,"data_stored":375393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40324,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331516,"statfs":{"total":96636764160,"available":96595472384,"internally_reserved":0,"allocated":835584,"data_stored":376983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40304,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331536,"statfs":{"total":96636764160,"available":96595492864,"internally_reserved":0,"allocated":823296,"data_stored":375371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":823296,"data_stored":375371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36172,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335668,"statfs":{"total":96636764160,"available":96599724032,"internally_reserved":0,"allocated":851968,"data_stored":377747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46456,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325384,"statfs":{"total":96636764160,"available":96589193216,"internally_reserved":0,"allocated":1417216,"data_stored":965739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:32.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:07:33.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.224+0000 7fd32f200640 1 Processor -- start 2024-04-25T10:07:33.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.225+0000 7fd32f200640 1 -- start start 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.226+0000 7fd32f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd33015e520 0x7fd33015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.226+0000 7fd32f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd33015f040 0x7fd3300adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.226+0000 7fd32f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3300ae590 con 0x7fd3300ae0d0 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.226+0000 7fd32f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300aee30 con 0x7fd33015f040 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.226+0000 7fd32f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3300ae870 con 0x7fd33015e520 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd327e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd33015f040 0x7fd3300adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd32e800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd3300ae0d0 0x7fd330070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37810/0 (socket says 172.21.15.165:37810) 2024-04-25T10:07:33.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd32e800640 1 -- 172.21.15.165:0/1268059268 learned_addr learned my addr 172.21.15.165:0/1268059268 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd32de00640 1 --2- 172.21.15.165:0/1268059268 >> v2:172.21.15.172:3300/0 conn(0x7fd33015e520 0x7fd33015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd327e00640 1 -- 172.21.15.165:0/1268059268 >> v2:172.21.15.172:3300/0 conn(0x7fd33015e520 msgr2=0x7fd33015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd327e00640 1 --2- 172.21.15.165:0/1268059268 >> v2:172.21.15.172:3300/0 conn(0x7fd33015e520 0x7fd33015e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd327e00640 1 -- 172.21.15.165:0/1268059268 >> v1:172.21.15.165:6789/0 conn(0x7fd3300ae0d0 legacy=0x7fd330070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd327e00640 1 -- 172.21.15.165:0/1268059268 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3300aeb50 con 0x7fd33015f040 2024-04-25T10:07:33.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.227+0000 7fd32d400640 1 -- 172.21.15.165:0/1268059268 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696554017 0 0) 0x7fd3300ae590 con 0x7fd3300ae0d0 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.228+0000 7fd327e00640 1 --2- 172.21.15.165:0/1268059268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd3300adb90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd33006ba10 tx=0x7fd31802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17de310f19553578 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.228+0000 7fd32d400640 1 -- 172.21.15.165:0/1268059268 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318037070 con 0x7fd33015f040 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32d400640 1 -- 172.21.15.165:0/1268059268 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd318032440 con 0x7fd33015f040 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32d400640 1 -- 172.21.15.165:0/1268059268 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd318032810 con 0x7fd33015f040 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 -- 172.21.15.165:0/1268059268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 msgr2=0x7fd3300adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 --2- 172.21.15.165:0/1268059268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd3300adb90 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fd33006ba10 tx=0x7fd31802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 -- 172.21.15.165:0/1268059268 shutdown_connections 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 --2- 172.21.15.165:0/1268059268 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd3300adb90 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 --2- 172.21.15.165:0/1268059268 >> v2:172.21.15.172:3300/0 conn(0x7fd33015e520 0x7fd33015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 -- 172.21.15.165:0/1268059268 >> 172.21.15.165:0/1268059268 conn(0x7fd33006abe0 msgr2=0x7fd33006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 -- 172.21.15.165:0/1268059268 shutdown_connections 2024-04-25T10:07:33.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.229+0000 7fd32f200640 1 -- 172.21.15.165:0/1268059268 wait complete. 2024-04-25T10:07:33.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.231+0000 7fd32f200640 1 Processor -- start 2024-04-25T10:07:33.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.231+0000 7fd32f200640 1 -- start start 2024-04-25T10:07:33.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3300ae0d0 0x7fd3300ce1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd33015e520 0x7fd330141280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd33015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33015f650 con 0x7fd33015f040 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33015f4d0 con 0x7fd33015e520 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd330071700 con 0x7fd3300ae0d0 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd33015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd33015a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36888/0 (socket says 172.21.15.165:36888) 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.232+0000 7fd32e800640 1 -- 172.21.15.165:0/46649890 learned_addr learned my addr 172.21.15.165:0/46649890 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd327e00640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd33015e520 0x7fd330141280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32de00640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3300ae0d0 0x7fd3300ce1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32e800640 1 -- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3300ae0d0 msgr2=0x7fd3300ce1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32e800640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3300ae0d0 0x7fd3300ce1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32e800640 1 -- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd33015e520 msgr2=0x7fd330141280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32e800640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd33015e520 0x7fd330141280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32e800640 1 -- 172.21.15.165:0/46649890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33015db80 con 0x7fd33015f040 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32de00640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3300ae0d0 0x7fd3300ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:33.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32e800640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd33015a010 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd32800c810 tx=0x7fd32800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328018070 con 0x7fd33015f040 2024-04-25T10:07:33.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800a640 con 0x7fd33015f040 2024-04-25T10:07:33.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800aa30 con 0x7fd33015f040 2024-04-25T10:07:33.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.233+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3300cd6c0 con 0x7fd33015f040 2024-04-25T10:07:33.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.234+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33015d810 con 0x7fd33015f040 2024-04-25T10:07:33.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.235+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd330073990 con 0x7fd33015f040 2024-04-25T10:07:33.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.236+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd32801b070 con 0x7fd33015f040 2024-04-25T10:07:33.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.236+0000 7fd326000640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30c06b4d0 0x7fd30c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.237+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd328089ab0 con 0x7fd33015f040 2024-04-25T10:07:33.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.237+0000 7fd32de00640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30c06b4d0 0x7fd30c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.238+0000 7fd32de00640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30c06b4d0 0x7fd30c06d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd320008890 tx=0x7fd320008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.241+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd328058700 con 0x7fd33015f040 2024-04-25T10:07:33.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.432+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd33015de60 con 0x7fd30c06b4d0 2024-04-25T10:07:33.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.440+0000 7fd326000640 1 -- 172.21.15.165:0/46649890 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252028 (secure 0 0 0) 0x7fd33015de60 con 0x7fd30c06b4d0 2024-04-25T10:07:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:33.450 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30c06b4d0 msgr2=0x7fd30c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30c06b4d0 0x7fd30c06d990 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fd320008890 tx=0x7fd320008040 comp rx=0 tx=0).stop 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 msgr2=0x7fd33015a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd33015a010 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fd32800c810 tx=0x7fd32800ccd0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 shutdown_connections 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd30c06b4d0 0x7fd30c06d990 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd33015f040 0x7fd33015a010 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd33015e520 0x7fd330141280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 --2- 172.21.15.165:0/46649890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3300ae0d0 0x7fd3300ce1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 >> 172.21.15.165:0/46649890 conn(0x7fd33006abe0 msgr2=0x7fd33006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.445+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 shutdown_connections 2024-04-25T10:07:33.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.446+0000 7fd32f200640 1 -- 172.21.15.165:0/46649890 wait complete. 2024-04-25T10:07:33.470 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-04-25T10:07:32.239388+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":173,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320936,"kb_used_data":7656,"kb_used_omap":12,"kb_used_meta":313139,"kb_avail":754653784,"statfs":{"total":773094113280,"available":772765474816,"internally_reserved":0,"allocated":7839744,"data_stored":4188524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":320654898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":32,"apply_latency_ms":32,"commit_latency_ns":32000000,"apply_latency_ns":32000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.708353"},"pg_stats":[{"pgid":"10.3f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.981016+0000","last_change":"2024-04-25T10:07:28.583707+0000","last_active":"2024-04-25T10:07:29.981016+0000","last_peered":"2024-04-25T10:07:29.981016+0000","last_clean":"2024-04-25T10:07:29.981016+0000","last_became_active":"2024-04-25T10:07:28.583434+0000","last_became_peered":"2024-04-25T10:07:28.583434+0000","last_unstale":"2024-04-25T10:07:29.981016+0000","last_undegraded":"2024-04-25T10:07:29.981016+0000","last_fullsized":"2024-04-25T10:07:29.981016+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:25.826659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980751+0000","last_change":"2024-04-25T10:07:28.580443+0000","last_active":"2024-04-25T10:07:29.980751+0000","last_peered":"2024-04-25T10:07:29.980751+0000","last_clean":"2024-04-25T10:07:29.980751+0000","last_became_active":"2024-04-25T10:07:28.580106+0000","last_became_peered":"2024-04-25T10:07:28.580106+0000","last_unstale":"2024-04-25T10:07:29.980751+0000","last_undegraded":"2024-04-25T10:07:29.980751+0000","last_fullsized":"2024-04-25T10:07:29.980751+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:44:32.844960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585658+0000","last_change":"2024-04-25T10:07:28.592962+0000","last_active":"2024-04-25T10:07:29.585658+0000","last_peered":"2024-04-25T10:07:29.585658+0000","last_clean":"2024-04-25T10:07:29.585658+0000","last_became_active":"2024-04-25T10:07:28.591998+0000","last_became_peered":"2024-04-25T10:07:28.591998+0000","last_unstale":"2024-04-25T10:07:29.585658+0000","last_undegraded":"2024-04-25T10:07:29.585658+0000","last_fullsized":"2024-04-25T10:07:29.585658+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:33.935446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976838+0000","last_change":"2024-04-25T10:07:28.582105+0000","last_active":"2024-04-25T10:07:29.976838+0000","last_peered":"2024-04-25T10:07:29.976838+0000","last_clean":"2024-04-25T10:07:29.976838+0000","last_became_active":"2024-04-25T10:07:28.581716+0000","last_became_peered":"2024-04-25T10:07:28.581716+0000","last_unstale":"2024-04-25T10:07:29.976838+0000","last_undegraded":"2024-04-25T10:07:29.976838+0000","last_fullsized":"2024-04-25T10:07:29.976838+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:24.527924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583807+0000","last_change":"2024-04-25T10:07:28.579534+0000","last_active":"2024-04-25T10:07:29.583807+0000","last_peered":"2024-04-25T10:07:29.583807+0000","last_clean":"2024-04-25T10:07:29.583807+0000","last_became_active":"2024-04-25T10:07:28.576106+0000","last_became_peered":"2024-04-25T10:07:28.576106+0000","last_unstale":"2024-04-25T10:07:29.583807+0000","last_undegraded":"2024-04-25T10:07:29.583807+0000","last_fullsized":"2024-04-25T10:07:29.583807+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.668084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978347+0000","last_change":"2024-04-25T10:07:28.587766+0000","last_active":"2024-04-25T10:07:29.978347+0000","last_peered":"2024-04-25T10:07:29.978347+0000","last_clean":"2024-04-25T10:07:29.978347+0000","last_became_active":"2024-04-25T10:07:28.587470+0000","last_became_peered":"2024-04-25T10:07:28.587470+0000","last_unstale":"2024-04-25T10:07:29.978347+0000","last_undegraded":"2024-04-25T10:07:29.978347+0000","last_fullsized":"2024-04-25T10:07:29.978347+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:30.539972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.581007+0000","last_change":"2024-04-25T10:07:28.594904+0000","last_active":"2024-04-25T10:07:29.581007+0000","last_peered":"2024-04-25T10:07:29.581007+0000","last_clean":"2024-04-25T10:07:29.581007+0000","last_became_active":"2024-04-25T10:07:28.594586+0000","last_became_peered":"2024-04-25T10:07:28.594586+0000","last_unstale":"2024-04-25T10:07:29.581007+0000","last_undegraded":"2024-04-25T10:07:29.581007+0000","last_fullsized":"2024-04-25T10:07:29.581007+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:22.487994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598125+0000","last_change":"2024-04-25T10:07:28.600499+0000","last_active":"2024-04-25T10:07:29.598125+0000","last_peered":"2024-04-25T10:07:29.598125+0000","last_clean":"2024-04-25T10:07:29.598125+0000","last_became_active":"2024-04-25T10:07:28.597407+0000","last_became_peered":"2024-04-25T10:07:28.597407+0000","last_unstale":"2024-04-25T10:07:29.598125+0000","last_undegraded":"2024-04-25T10:07:29.598125+0000","last_fullsized":"2024-04-25T10:07:29.598125+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:38:06.470219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586609+0000","last_change":"2024-04-25T10:07:28.592105+0000","last_active":"2024-04-25T10:07:29.586609+0000","last_peered":"2024-04-25T10:07:29.586609+0000","last_clean":"2024-04-25T10:07:29.586609+0000","last_became_active":"2024-04-25T10:07:28.591588+0000","last_became_peered":"2024-04-25T10:07:28.591588+0000","last_unstale":"2024-04-25T10:07:29.586609+0000","last_undegraded":"2024-04-25T10:07:29.586609+0000","last_fullsized":"2024-04-25T10:07:29.586609+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:04.011580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583413+0000","last_change":"2024-04-25T10:07:28.595500+0000","last_active":"2024-04-25T10:07:29.583413+0000","last_peered":"2024-04-25T10:07:29.583413+0000","last_clean":"2024-04-25T10:07:29.583413+0000","last_became_active":"2024-04-25T10:07:28.594889+0000","last_became_peered":"2024-04-25T10:07:28.594889+0000","last_unstale":"2024-04-25T10:07:29.583413+0000","last_undegraded":"2024-04-25T10:07:29.583413+0000","last_fullsized":"2024-04-25T10:07:29.583413+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:25.685599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976224+0000","last_change":"2024-04-25T10:07:28.577622+0000","last_active":"2024-04-25T10:07:29.976224+0000","last_peered":"2024-04-25T10:07:29.976224+0000","last_clean":"2024-04-25T10:07:29.976224+0000","last_became_active":"2024-04-25T10:07:28.577275+0000","last_became_peered":"2024-04-25T10:07:28.577275+0000","last_unstale":"2024-04-25T10:07:29.976224+0000","last_undegraded":"2024-04-25T10:07:29.976224+0000","last_fullsized":"2024-04-25T10:07:29.976224+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:30.216442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598629+0000","last_change":"2024-04-25T10:07:28.596382+0000","last_active":"2024-04-25T10:07:29.598629+0000","last_peered":"2024-04-25T10:07:29.598629+0000","last_clean":"2024-04-25T10:07:29.598629+0000","last_became_active":"2024-04-25T10:07:28.596093+0000","last_became_peered":"2024-04-25T10:07:28.596093+0000","last_unstale":"2024-04-25T10:07:29.598629+0000","last_undegraded":"2024-04-25T10:07:29.598629+0000","last_fullsized":"2024-04-25T10:07:29.598629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:25.327162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598129+0000","last_change":"2024-04-25T10:07:28.591893+0000","last_active":"2024-04-25T10:07:29.598129+0000","last_peered":"2024-04-25T10:07:29.598129+0000","last_clean":"2024-04-25T10:07:29.598129+0000","last_became_active":"2024-04-25T10:07:28.591565+0000","last_became_peered":"2024-04-25T10:07:28.591565+0000","last_unstale":"2024-04-25T10:07:29.598129+0000","last_undegraded":"2024-04-25T10:07:29.598129+0000","last_fullsized":"2024-04-25T10:07:29.598129+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:14.966243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584664+0000","last_change":"2024-04-25T10:07:28.590676+0000","last_active":"2024-04-25T10:07:29.584664+0000","last_peered":"2024-04-25T10:07:29.584664+0000","last_clean":"2024-04-25T10:07:29.584664+0000","last_became_active":"2024-04-25T10:07:28.590326+0000","last_became_peered":"2024-04-25T10:07:28.590326+0000","last_unstale":"2024-04-25T10:07:29.584664+0000","last_undegraded":"2024-04-25T10:07:29.584664+0000","last_fullsized":"2024-04-25T10:07:29.584664+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:50.286556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586885+0000","last_change":"2024-04-25T10:07:28.595307+0000","last_active":"2024-04-25T10:07:29.586885+0000","last_peered":"2024-04-25T10:07:29.586885+0000","last_clean":"2024-04-25T10:07:29.586885+0000","last_became_active":"2024-04-25T10:07:28.594903+0000","last_became_peered":"2024-04-25T10:07:28.594903+0000","last_unstale":"2024-04-25T10:07:29.586885+0000","last_undegraded":"2024-04-25T10:07:29.586885+0000","last_fullsized":"2024-04-25T10:07:29.586885+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:31.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.580673+0000","last_change":"2024-04-25T10:07:28.596023+0000","last_active":"2024-04-25T10:07:29.580673+0000","last_peered":"2024-04-25T10:07:29.580673+0000","last_clean":"2024-04-25T10:07:29.580673+0000","last_became_active":"2024-04-25T10:07:28.595666+0000","last_became_peered":"2024-04-25T10:07:28.595666+0000","last_unstale":"2024-04-25T10:07:29.580673+0000","last_undegraded":"2024-04-25T10:07:29.580673+0000","last_fullsized":"2024-04-25T10:07:29.580673+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:19.159482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.981059+0000","last_change":"2024-04-25T10:07:28.575621+0000","last_active":"2024-04-25T10:07:29.981059+0000","last_peered":"2024-04-25T10:07:29.981059+0000","last_clean":"2024-04-25T10:07:29.981059+0000","last_became_active":"2024-04-25T10:07:28.575297+0000","last_became_peered":"2024-04-25T10:07:28.575297+0000","last_unstale":"2024-04-25T10:07:29.981059+0000","last_undegraded":"2024-04-25T10:07:29.981059+0000","last_fullsized":"2024-04-25T10:07:29.981059+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:12.091515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976925+0000","last_change":"2024-04-25T10:07:28.581544+0000","last_active":"2024-04-25T10:07:29.976925+0000","last_peered":"2024-04-25T10:07:29.976925+0000","last_clean":"2024-04-25T10:07:29.976925+0000","last_became_active":"2024-04-25T10:07:28.581184+0000","last_became_peered":"2024-04-25T10:07:28.581184+0000","last_unstale":"2024-04-25T10:07:29.976925+0000","last_undegraded":"2024-04-25T10:07:29.976925+0000","last_fullsized":"2024-04-25T10:07:29.976925+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:37.207640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585204+0000","last_change":"2024-04-25T10:07:28.600844+0000","last_active":"2024-04-25T10:07:29.585204+0000","last_peered":"2024-04-25T10:07:29.585204+0000","last_clean":"2024-04-25T10:07:29.585204+0000","last_became_active":"2024-04-25T10:07:28.599836+0000","last_became_peered":"2024-04-25T10:07:28.599836+0000","last_unstale":"2024-04-25T10:07:29.585204+0000","last_undegraded":"2024-04-25T10:07:29.585204+0000","last_fullsized":"2024-04-25T10:07:29.585204+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:10.865465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977011+0000","last_change":"2024-04-25T10:07:28.590691+0000","last_active":"2024-04-25T10:07:29.977011+0000","last_peered":"2024-04-25T10:07:29.977011+0000","last_clean":"2024-04-25T10:07:29.977011+0000","last_became_active":"2024-04-25T10:07:28.590387+0000","last_became_peered":"2024-04-25T10:07:28.590387+0000","last_unstale":"2024-04-25T10:07:29.977011+0000","last_undegraded":"2024-04-25T10:07:29.977011+0000","last_fullsized":"2024-04-25T10:07:29.977011+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:25.053354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598659+0000","last_change":"2024-04-25T10:07:28.585670+0000","last_active":"2024-04-25T10:07:29.598659+0000","last_peered":"2024-04-25T10:07:29.598659+0000","last_clean":"2024-04-25T10:07:29.598659+0000","last_became_active":"2024-04-25T10:07:28.585343+0000","last_became_peered":"2024-04-25T10:07:28.585343+0000","last_unstale":"2024-04-25T10:07:29.598659+0000","last_undegraded":"2024-04-25T10:07:29.598659+0000","last_fullsized":"2024-04-25T10:07:29.598659+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:40.197673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975914+0000","last_change":"2024-04-25T10:07:28.598238+0000","last_active":"2024-04-25T10:07:29.975914+0000","last_peered":"2024-04-25T10:07:29.975914+0000","last_clean":"2024-04-25T10:07:29.975914+0000","last_became_active":"2024-04-25T10:07:28.597962+0000","last_became_peered":"2024-04-25T10:07:28.597962+0000","last_unstale":"2024-04-25T10:07:29.975914+0000","last_undegraded":"2024-04-25T10:07:29.975914+0000","last_fullsized":"2024-04-25T10:07:29.975914+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:30.539972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975952+0000","last_change":"2024-04-25T10:07:28.596707+0000","last_active":"2024-04-25T10:07:29.975952+0000","last_peered":"2024-04-25T10:07:29.975952+0000","last_clean":"2024-04-25T10:07:29.975952+0000","last_became_active":"2024-04-25T10:07:28.596414+0000","last_became_peered":"2024-04-25T10:07:28.596414+0000","last_unstale":"2024-04-25T10:07:29.975952+0000","last_undegraded":"2024-04-25T10:07:29.975952+0000","last_fullsized":"2024-04-25T10:07:29.975952+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.737727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584527+0000","last_change":"2024-04-25T10:07:28.591555+0000","last_active":"2024-04-25T10:07:29.584527+0000","last_peered":"2024-04-25T10:07:29.584527+0000","last_clean":"2024-04-25T10:07:29.584527+0000","last_became_active":"2024-04-25T10:07:28.590204+0000","last_became_peered":"2024-04-25T10:07:28.590204+0000","last_unstale":"2024-04-25T10:07:29.584527+0000","last_undegraded":"2024-04-25T10:07:29.584527+0000","last_fullsized":"2024-04-25T10:07:29.584527+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:40.448447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598929+0000","last_change":"2024-04-25T10:07:28.602552+0000","last_active":"2024-04-25T10:07:29.598929+0000","last_peered":"2024-04-25T10:07:29.598929+0000","last_clean":"2024-04-25T10:07:29.598929+0000","last_became_active":"2024-04-25T10:07:28.600662+0000","last_became_peered":"2024-04-25T10:07:28.600662+0000","last_unstale":"2024-04-25T10:07:29.598929+0000","last_undegraded":"2024-04-25T10:07:29.598929+0000","last_fullsized":"2024-04-25T10:07:29.598929+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:06.950405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598861+0000","last_change":"2024-04-25T10:07:28.599332+0000","last_active":"2024-04-25T10:07:29.598861+0000","last_peered":"2024-04-25T10:07:29.598861+0000","last_clean":"2024-04-25T10:07:29.598861+0000","last_became_active":"2024-04-25T10:07:28.599019+0000","last_became_peered":"2024-04-25T10:07:28.599019+0000","last_unstale":"2024-04-25T10:07:29.598861+0000","last_undegraded":"2024-04-25T10:07:29.598861+0000","last_fullsized":"2024-04-25T10:07:29.598861+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:24.644589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976302+0000","last_change":"2024-04-25T10:07:28.601954+0000","last_active":"2024-04-25T10:07:29.976302+0000","last_peered":"2024-04-25T10:07:29.976302+0000","last_clean":"2024-04-25T10:07:29.976302+0000","last_became_active":"2024-04-25T10:07:28.601624+0000","last_became_peered":"2024-04-25T10:07:28.601624+0000","last_unstale":"2024-04-25T10:07:29.976302+0000","last_undegraded":"2024-04-25T10:07:29.976302+0000","last_fullsized":"2024-04-25T10:07:29.976302+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:06:22.344465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980406+0000","last_change":"2024-04-25T10:07:28.582929+0000","last_active":"2024-04-25T10:07:29.980406+0000","last_peered":"2024-04-25T10:07:29.980406+0000","last_clean":"2024-04-25T10:07:29.980406+0000","last_became_active":"2024-04-25T10:07:28.579625+0000","last_became_peered":"2024-04-25T10:07:28.579625+0000","last_unstale":"2024-04-25T10:07:29.980406+0000","last_undegraded":"2024-04-25T10:07:29.980406+0000","last_fullsized":"2024-04-25T10:07:29.980406+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:48.018677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583363+0000","last_change":"2024-04-25T10:07:28.593823+0000","last_active":"2024-04-25T10:07:29.583363+0000","last_peered":"2024-04-25T10:07:29.583363+0000","last_clean":"2024-04-25T10:07:29.583363+0000","last_became_active":"2024-04-25T10:07:28.593541+0000","last_became_peered":"2024-04-25T10:07:28.593541+0000","last_unstale":"2024-04-25T10:07:29.583363+0000","last_undegraded":"2024-04-25T10:07:29.583363+0000","last_fullsized":"2024-04-25T10:07:29.583363+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:06.950405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.597981+0000","last_change":"2024-04-25T10:07:28.585452+0000","last_active":"2024-04-25T10:07:29.597981+0000","last_peered":"2024-04-25T10:07:29.597981+0000","last_clean":"2024-04-25T10:07:29.597981+0000","last_became_active":"2024-04-25T10:07:28.584761+0000","last_became_peered":"2024-04-25T10:07:28.584761+0000","last_unstale":"2024-04-25T10:07:29.597981+0000","last_undegraded":"2024-04-25T10:07:29.597981+0000","last_fullsized":"2024-04-25T10:07:29.597981+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:45.947923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976343+0000","last_change":"2024-04-25T10:07:28.601450+0000","last_active":"2024-04-25T10:07:29.976343+0000","last_peered":"2024-04-25T10:07:29.976343+0000","last_clean":"2024-04-25T10:07:29.976343+0000","last_became_active":"2024-04-25T10:07:28.598645+0000","last_became_peered":"2024-04-25T10:07:28.598645+0000","last_unstale":"2024-04-25T10:07:29.976343+0000","last_undegraded":"2024-04-25T10:07:29.976343+0000","last_fullsized":"2024-04-25T10:07:29.976343+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:48.051609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976078+0000","last_change":"2024-04-25T10:07:28.599283+0000","last_active":"2024-04-25T10:07:29.976078+0000","last_peered":"2024-04-25T10:07:29.976078+0000","last_clean":"2024-04-25T10:07:29.976078+0000","last_became_active":"2024-04-25T10:07:28.598962+0000","last_became_peered":"2024-04-25T10:07:28.598962+0000","last_unstale":"2024-04-25T10:07:29.976078+0000","last_undegraded":"2024-04-25T10:07:29.976078+0000","last_fullsized":"2024-04-25T10:07:29.976078+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:22:31.591916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584833+0000","last_change":"2024-04-25T10:07:28.582305+0000","last_active":"2024-04-25T10:07:29.584833+0000","last_peered":"2024-04-25T10:07:29.584833+0000","last_clean":"2024-04-25T10:07:29.584833+0000","last_became_active":"2024-04-25T10:07:28.581892+0000","last_became_peered":"2024-04-25T10:07:28.581892+0000","last_unstale":"2024-04-25T10:07:29.584833+0000","last_undegraded":"2024-04-25T10:07:29.584833+0000","last_fullsized":"2024-04-25T10:07:29.584833+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:28.570344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"67'4","reported_seq":18,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.981096+0000","last_change":"2024-04-25T10:07:27.581044+0000","last_active":"2024-04-25T10:07:29.981096+0000","last_peered":"2024-04-25T10:07:29.981096+0000","last_clean":"2024-04-25T10:07:29.981096+0000","last_became_active":"2024-04-25T10:07:27.580782+0000","last_became_peered":"2024-04-25T10:07:27.580782+0000","last_unstale":"2024-04-25T10:07:29.981096+0000","last_undegraded":"2024-04-25T10:07:29.981096+0000","last_fullsized":"2024-04-25T10:07:29.981096+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:47.654284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976694+0000","last_change":"2024-04-25T10:07:28.587534+0000","last_active":"2024-04-25T10:07:29.976694+0000","last_peered":"2024-04-25T10:07:29.976694+0000","last_clean":"2024-04-25T10:07:29.976694+0000","last_became_active":"2024-04-25T10:07:28.584511+0000","last_became_peered":"2024-04-25T10:07:28.584511+0000","last_unstale":"2024-04-25T10:07:29.976694+0000","last_undegraded":"2024-04-25T10:07:29.976694+0000","last_fullsized":"2024-04-25T10:07:29.976694+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:20.873378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.979563+0000","last_change":"2024-04-25T10:07:27.580646+0000","last_active":"2024-04-25T10:07:29.979563+0000","last_peered":"2024-04-25T10:07:29.979563+0000","last_clean":"2024-04-25T10:07:29.979563+0000","last_became_active":"2024-04-25T10:07:27.580255+0000","last_became_peered":"2024-04-25T10:07:27.580255+0000","last_unstale":"2024-04-25T10:07:29.979563+0000","last_undegraded":"2024-04-25T10:07:29.979563+0000","last_fullsized":"2024-04-25T10:07:29.979563+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:23.630789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.979371+0000","last_change":"2024-04-25T10:07:28.599327+0000","last_active":"2024-04-25T10:07:29.979371+0000","last_peered":"2024-04-25T10:07:29.979371+0000","last_clean":"2024-04-25T10:07:29.979371+0000","last_became_active":"2024-04-25T10:07:28.599041+0000","last_became_peered":"2024-04-25T10:07:28.599041+0000","last_unstale":"2024-04-25T10:07:29.979371+0000","last_undegraded":"2024-04-25T10:07:29.979371+0000","last_fullsized":"2024-04-25T10:07:29.979371+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:19.737727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978380+0000","last_change":"2024-04-25T10:07:27.585009+0000","last_active":"2024-04-25T10:07:29.978380+0000","last_peered":"2024-04-25T10:07:29.978380+0000","last_clean":"2024-04-25T10:07:29.978380+0000","last_became_active":"2024-04-25T10:07:27.584679+0000","last_became_peered":"2024-04-25T10:07:27.584679+0000","last_unstale":"2024-04-25T10:07:29.978380+0000","last_undegraded":"2024-04-25T10:07:29.978380+0000","last_fullsized":"2024-04-25T10:07:29.978380+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:24:24.313362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586571+0000","last_change":"2024-04-25T10:07:28.604855+0000","last_active":"2024-04-25T10:07:29.586571+0000","last_peered":"2024-04-25T10:07:29.586571+0000","last_clean":"2024-04-25T10:07:29.586571+0000","last_became_active":"2024-04-25T10:07:28.604524+0000","last_became_peered":"2024-04-25T10:07:28.604524+0000","last_unstale":"2024-04-25T10:07:29.586571+0000","last_undegraded":"2024-04-25T10:07:29.586571+0000","last_fullsized":"2024-04-25T10:07:29.586571+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:50.372337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978242+0000","last_change":"2024-04-25T10:07:27.572839+0000","last_active":"2024-04-25T10:07:29.978242+0000","last_peered":"2024-04-25T10:07:29.978242+0000","last_clean":"2024-04-25T10:07:29.978242+0000","last_became_active":"2024-04-25T10:07:27.568223+0000","last_became_peered":"2024-04-25T10:07:27.568223+0000","last_unstale":"2024-04-25T10:07:29.978242+0000","last_undegraded":"2024-04-25T10:07:29.978242+0000","last_fullsized":"2024-04-25T10:07:29.978242+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:45.452132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.580986+0000","last_change":"2024-04-25T10:07:28.590600+0000","last_active":"2024-04-25T10:07:29.580986+0000","last_peered":"2024-04-25T10:07:29.580986+0000","last_clean":"2024-04-25T10:07:29.580986+0000","last_became_active":"2024-04-25T10:07:28.578992+0000","last_became_peered":"2024-04-25T10:07:28.578992+0000","last_unstale":"2024-04-25T10:07:29.580986+0000","last_undegraded":"2024-04-25T10:07:29.580986+0000","last_fullsized":"2024-04-25T10:07:29.580986+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:36.852402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976635+0000","last_change":"2024-04-25T10:07:27.570859+0000","last_active":"2024-04-25T10:07:29.976635+0000","last_peered":"2024-04-25T10:07:29.976635+0000","last_clean":"2024-04-25T10:07:29.976635+0000","last_became_active":"2024-04-25T10:07:27.570549+0000","last_became_peered":"2024-04-25T10:07:27.570549+0000","last_unstale":"2024-04-25T10:07:29.976635+0000","last_undegraded":"2024-04-25T10:07:29.976635+0000","last_fullsized":"2024-04-25T10:07:29.976635+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:49.272756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978998+0000","last_change":"2024-04-25T10:07:28.568365+0000","last_active":"2024-04-25T10:07:29.978998+0000","last_peered":"2024-04-25T10:07:29.978998+0000","last_clean":"2024-04-25T10:07:29.978998+0000","last_became_active":"2024-04-25T10:07:28.567439+0000","last_became_peered":"2024-04-25T10:07:28.567439+0000","last_unstale":"2024-04-25T10:07:29.978998+0000","last_undegraded":"2024-04-25T10:07:29.978998+0000","last_fullsized":"2024-04-25T10:07:29.978998+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:46.019029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976061+0000","last_change":"2024-04-25T10:07:27.576587+0000","last_active":"2024-04-25T10:07:29.976061+0000","last_peered":"2024-04-25T10:07:29.976061+0000","last_clean":"2024-04-25T10:07:29.976061+0000","last_became_active":"2024-04-25T10:07:27.576051+0000","last_became_peered":"2024-04-25T10:07:27.576051+0000","last_unstale":"2024-04-25T10:07:29.976061+0000","last_undegraded":"2024-04-25T10:07:29.976061+0000","last_fullsized":"2024-04-25T10:07:29.976061+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:24.671799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.582811+0000","last_change":"2024-04-25T10:07:28.579904+0000","last_active":"2024-04-25T10:07:29.582811+0000","last_peered":"2024-04-25T10:07:29.582811+0000","last_clean":"2024-04-25T10:07:29.582811+0000","last_became_active":"2024-04-25T10:07:28.579217+0000","last_became_peered":"2024-04-25T10:07:28.579217+0000","last_unstale":"2024-04-25T10:07:29.582811+0000","last_undegraded":"2024-04-25T10:07:29.582811+0000","last_fullsized":"2024-04-25T10:07:29.582811+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:20.873378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975970+0000","last_change":"2024-04-25T10:07:27.575359+0000","last_active":"2024-04-25T10:07:29.975970+0000","last_peered":"2024-04-25T10:07:29.975970+0000","last_clean":"2024-04-25T10:07:29.975970+0000","last_became_active":"2024-04-25T10:07:27.575069+0000","last_became_peered":"2024-04-25T10:07:27.575069+0000","last_unstale":"2024-04-25T10:07:29.975970+0000","last_undegraded":"2024-04-25T10:07:29.975970+0000","last_fullsized":"2024-04-25T10:07:29.975970+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:43:23.630789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976492+0000","last_change":"2024-04-25T10:07:28.593148+0000","last_active":"2024-04-25T10:07:29.976492+0000","last_peered":"2024-04-25T10:07:29.976492+0000","last_clean":"2024-04-25T10:07:29.976492+0000","last_became_active":"2024-04-25T10:07:28.591511+0000","last_became_peered":"2024-04-25T10:07:28.591511+0000","last_unstale":"2024-04-25T10:07:29.976492+0000","last_undegraded":"2024-04-25T10:07:29.976492+0000","last_fullsized":"2024-04-25T10:07:29.976492+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:23.608303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.582482+0000","last_change":"2024-04-25T10:07:27.580357+0000","last_active":"2024-04-25T10:07:29.582482+0000","last_peered":"2024-04-25T10:07:29.582482+0000","last_clean":"2024-04-25T10:07:29.582482+0000","last_became_active":"2024-04-25T10:07:27.579854+0000","last_became_peered":"2024-04-25T10:07:27.579854+0000","last_unstale":"2024-04-25T10:07:29.582482+0000","last_undegraded":"2024-04-25T10:07:29.582482+0000","last_fullsized":"2024-04-25T10:07:29.582482+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:04:28.515162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980530+0000","last_change":"2024-04-25T10:07:28.601898+0000","last_active":"2024-04-25T10:07:29.980530+0000","last_peered":"2024-04-25T10:07:29.980530+0000","last_clean":"2024-04-25T10:07:29.980530+0000","last_became_active":"2024-04-25T10:07:28.601567+0000","last_became_peered":"2024-04-25T10:07:28.601567+0000","last_unstale":"2024-04-25T10:07:29.980530+0000","last_undegraded":"2024-04-25T10:07:29.980530+0000","last_fullsized":"2024-04-25T10:07:29.980530+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:20.597644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586722+0000","last_change":"2024-04-25T10:07:27.586327+0000","last_active":"2024-04-25T10:07:29.586722+0000","last_peered":"2024-04-25T10:07:29.586722+0000","last_clean":"2024-04-25T10:07:29.586722+0000","last_became_active":"2024-04-25T10:07:27.585969+0000","last_became_peered":"2024-04-25T10:07:27.585969+0000","last_unstale":"2024-04-25T10:07:29.586722+0000","last_undegraded":"2024-04-25T10:07:29.586722+0000","last_fullsized":"2024-04-25T10:07:29.586722+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:32.499620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976419+0000","last_change":"2024-04-25T10:07:28.583290+0000","last_active":"2024-04-25T10:07:29.976419+0000","last_peered":"2024-04-25T10:07:29.976419+0000","last_clean":"2024-04-25T10:07:29.976419+0000","last_became_active":"2024-04-25T10:07:28.582971+0000","last_became_peered":"2024-04-25T10:07:28.582971+0000","last_unstale":"2024-04-25T10:07:29.976419+0000","last_undegraded":"2024-04-25T10:07:29.976419+0000","last_fullsized":"2024-04-25T10:07:29.976419+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:29.351396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977110+0000","last_change":"2024-04-25T10:07:27.572448+0000","last_active":"2024-04-25T10:07:29.977110+0000","last_peered":"2024-04-25T10:07:29.977110+0000","last_clean":"2024-04-25T10:07:29.977110+0000","last_became_active":"2024-04-25T10:07:27.572132+0000","last_became_peered":"2024-04-25T10:07:27.572132+0000","last_unstale":"2024-04-25T10:07:29.977110+0000","last_undegraded":"2024-04-25T10:07:29.977110+0000","last_fullsized":"2024-04-25T10:07:29.977110+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:44.919468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585715+0000","last_change":"2024-04-25T10:07:28.579192+0000","last_active":"2024-04-25T10:07:29.585715+0000","last_peered":"2024-04-25T10:07:29.585715+0000","last_clean":"2024-04-25T10:07:29.585715+0000","last_became_active":"2024-04-25T10:07:28.578820+0000","last_became_peered":"2024-04-25T10:07:28.578820+0000","last_unstale":"2024-04-25T10:07:29.585715+0000","last_undegraded":"2024-04-25T10:07:29.585715+0000","last_fullsized":"2024-04-25T10:07:29.585715+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:27.615095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"67'3","reported_seq":17,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585788+0000","last_change":"2024-04-25T10:07:27.587729+0000","last_active":"2024-04-25T10:07:29.585788+0000","last_peered":"2024-04-25T10:07:29.585788+0000","last_clean":"2024-04-25T10:07:29.585788+0000","last_became_active":"2024-04-25T10:07:27.587355+0000","last_became_peered":"2024-04-25T10:07:27.587355+0000","last_unstale":"2024-04-25T10:07:29.585788+0000","last_undegraded":"2024-04-25T10:07:29.585788+0000","last_fullsized":"2024-04-25T10:07:29.585788+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:24.812858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977004+0000","last_change":"2024-04-25T10:07:28.587939+0000","last_active":"2024-04-25T10:07:29.977004+0000","last_peered":"2024-04-25T10:07:29.977004+0000","last_clean":"2024-04-25T10:07:29.977004+0000","last_became_active":"2024-04-25T10:07:28.587588+0000","last_became_peered":"2024-04-25T10:07:28.587588+0000","last_unstale":"2024-04-25T10:07:29.977004+0000","last_undegraded":"2024-04-25T10:07:29.977004+0000","last_fullsized":"2024-04-25T10:07:29.977004+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:43:48.668084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"67'6","reported_seq":20,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980683+0000","last_change":"2024-04-25T10:07:27.577743+0000","last_active":"2024-04-25T10:07:29.980683+0000","last_peered":"2024-04-25T10:07:29.980683+0000","last_clean":"2024-04-25T10:07:29.980683+0000","last_became_active":"2024-04-25T10:07:27.577434+0000","last_became_peered":"2024-04-25T10:07:27.577434+0000","last_unstale":"2024-04-25T10:07:29.980683+0000","last_undegraded":"2024-04-25T10:07:29.980683+0000","last_fullsized":"2024-04-25T10:07:29.980683+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:53.268246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976051+0000","last_change":"2024-04-25T10:07:28.586113+0000","last_active":"2024-04-25T10:07:29.976051+0000","last_peered":"2024-04-25T10:07:29.976051+0000","last_clean":"2024-04-25T10:07:29.976051+0000","last_became_active":"2024-04-25T10:07:28.584757+0000","last_became_peered":"2024-04-25T10:07:28.584757+0000","last_unstale":"2024-04-25T10:07:29.976051+0000","last_undegraded":"2024-04-25T10:07:29.976051+0000","last_fullsized":"2024-04-25T10:07:29.976051+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:50:48.085046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583475+0000","last_change":"2024-04-25T10:07:27.582455+0000","last_active":"2024-04-25T10:07:29.583475+0000","last_peered":"2024-04-25T10:07:29.583475+0000","last_clean":"2024-04-25T10:07:29.583475+0000","last_became_active":"2024-04-25T10:07:27.582140+0000","last_became_peered":"2024-04-25T10:07:27.582140+0000","last_unstale":"2024-04-25T10:07:29.583475+0000","last_undegraded":"2024-04-25T10:07:29.583475+0000","last_fullsized":"2024-04-25T10:07:29.583475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:02:11.943908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598034+0000","last_change":"2024-04-25T10:07:28.583474+0000","last_active":"2024-04-25T10:07:29.598034+0000","last_peered":"2024-04-25T10:07:29.598034+0000","last_clean":"2024-04-25T10:07:29.598034+0000","last_became_active":"2024-04-25T10:07:28.583149+0000","last_became_peered":"2024-04-25T10:07:28.583149+0000","last_unstale":"2024-04-25T10:07:29.598034+0000","last_undegraded":"2024-04-25T10:07:29.598034+0000","last_fullsized":"2024-04-25T10:07:29.598034+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:21.622179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598034+0000","last_change":"2024-04-25T10:07:27.583333+0000","last_active":"2024-04-25T10:07:29.598034+0000","last_peered":"2024-04-25T10:07:29.598034+0000","last_clean":"2024-04-25T10:07:29.598034+0000","last_became_active":"2024-04-25T10:07:27.578437+0000","last_became_peered":"2024-04-25T10:07:27.578437+0000","last_unstale":"2024-04-25T10:07:29.598034+0000","last_undegraded":"2024-04-25T10:07:29.598034+0000","last_fullsized":"2024-04-25T10:07:29.598034+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:54.049709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583757+0000","last_change":"2024-04-25T10:07:28.597640+0000","last_active":"2024-04-25T10:07:29.583757+0000","last_peered":"2024-04-25T10:07:29.583757+0000","last_clean":"2024-04-25T10:07:29.583757+0000","last_became_active":"2024-04-25T10:07:28.597332+0000","last_became_peered":"2024-04-25T10:07:28.597332+0000","last_unstale":"2024-04-25T10:07:29.583757+0000","last_undegraded":"2024-04-25T10:07:29.583757+0000","last_fullsized":"2024-04-25T10:07:29.583757+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:37.207640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.587067+0000","last_change":"2024-04-25T10:07:27.588103+0000","last_active":"2024-04-25T10:07:29.587067+0000","last_peered":"2024-04-25T10:07:29.587067+0000","last_clean":"2024-04-25T10:07:29.587067+0000","last_became_active":"2024-04-25T10:07:27.587749+0000","last_became_peered":"2024-04-25T10:07:27.587749+0000","last_unstale":"2024-04-25T10:07:29.587067+0000","last_undegraded":"2024-04-25T10:07:29.587067+0000","last_fullsized":"2024-04-25T10:07:29.587067+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:46:57.530246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980165+0000","last_change":"2024-04-25T10:07:28.589567+0000","last_active":"2024-04-25T10:07:29.980165+0000","last_peered":"2024-04-25T10:07:29.980165+0000","last_clean":"2024-04-25T10:07:29.980165+0000","last_became_active":"2024-04-25T10:07:28.588738+0000","last_became_peered":"2024-04-25T10:07:28.588738+0000","last_unstale":"2024-04-25T10:07:29.980165+0000","last_undegraded":"2024-04-25T10:07:29.980165+0000","last_fullsized":"2024-04-25T10:07:29.980165+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:15.675146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"67'8","reported_seq":22,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975412+0000","last_change":"2024-04-25T10:07:27.582838+0000","last_active":"2024-04-25T10:07:29.975412+0000","last_peered":"2024-04-25T10:07:29.975412+0000","last_clean":"2024-04-25T10:07:29.975412+0000","last_became_active":"2024-04-25T10:07:27.581144+0000","last_became_peered":"2024-04-25T10:07:27.581144+0000","last_unstale":"2024-04-25T10:07:29.975412+0000","last_undegraded":"2024-04-25T10:07:29.975412+0000","last_fullsized":"2024-04-25T10:07:29.975412+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:22:30.684965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585186+0000","last_change":"2024-04-25T10:07:28.592638+0000","last_active":"2024-04-25T10:07:29.585186+0000","last_peered":"2024-04-25T10:07:29.585186+0000","last_clean":"2024-04-25T10:07:29.585186+0000","last_became_active":"2024-04-25T10:07:28.592318+0000","last_became_peered":"2024-04-25T10:07:28.592318+0000","last_unstale":"2024-04-25T10:07:29.585186+0000","last_undegraded":"2024-04-25T10:07:29.585186+0000","last_fullsized":"2024-04-25T10:07:29.585186+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:36.477385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598801+0000","last_change":"2024-04-25T10:07:27.580589+0000","last_active":"2024-04-25T10:07:29.598801+0000","last_peered":"2024-04-25T10:07:29.598801+0000","last_clean":"2024-04-25T10:07:29.598801+0000","last_became_active":"2024-04-25T10:07:27.578437+0000","last_became_peered":"2024-04-25T10:07:27.578437+0000","last_unstale":"2024-04-25T10:07:29.598801+0000","last_undegraded":"2024-04-25T10:07:29.598801+0000","last_fullsized":"2024-04-25T10:07:29.598801+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:53.043926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585647+0000","last_change":"2024-04-25T10:07:28.577018+0000","last_active":"2024-04-25T10:07:29.585647+0000","last_peered":"2024-04-25T10:07:29.585647+0000","last_clean":"2024-04-25T10:07:29.585647+0000","last_became_active":"2024-04-25T10:07:28.575703+0000","last_became_peered":"2024-04-25T10:07:28.575703+0000","last_unstale":"2024-04-25T10:07:29.585647+0000","last_undegraded":"2024-04-25T10:07:29.585647+0000","last_fullsized":"2024-04-25T10:07:29.585647+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:21.016333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584192+0000","last_change":"2024-04-25T10:07:27.584098+0000","last_active":"2024-04-25T10:07:29.584192+0000","last_peered":"2024-04-25T10:07:29.584192+0000","last_clean":"2024-04-25T10:07:29.584192+0000","last_became_active":"2024-04-25T10:07:27.583801+0000","last_became_peered":"2024-04-25T10:07:27.583801+0000","last_unstale":"2024-04-25T10:07:29.584192+0000","last_undegraded":"2024-04-25T10:07:29.584192+0000","last_fullsized":"2024-04-25T10:07:29.584192+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:40.631123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976928+0000","last_change":"2024-04-25T10:07:28.579045+0000","last_active":"2024-04-25T10:07:29.976928+0000","last_peered":"2024-04-25T10:07:29.976928+0000","last_clean":"2024-04-25T10:07:29.976928+0000","last_became_active":"2024-04-25T10:07:28.576958+0000","last_became_peered":"2024-04-25T10:07:28.576958+0000","last_unstale":"2024-04-25T10:07:29.976928+0000","last_undegraded":"2024-04-25T10:07:29.976928+0000","last_fullsized":"2024-04-25T10:07:29.976928+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.256788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"67'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.586068+0000","last_change":"2024-04-25T10:07:27.585602+0000","last_active":"2024-04-25T10:07:29.586068+0000","last_peered":"2024-04-25T10:07:29.586068+0000","last_clean":"2024-04-25T10:07:29.586068+0000","last_became_active":"2024-04-25T10:07:27.585337+0000","last_became_peered":"2024-04-25T10:07:27.585337+0000","last_unstale":"2024-04-25T10:07:29.586068+0000","last_undegraded":"2024-04-25T10:07:29.586068+0000","last_fullsized":"2024-04-25T10:07:29.586068+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:47:11.077715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584956+0000","last_change":"2024-04-25T10:07:28.592281+0000","last_active":"2024-04-25T10:07:29.584956+0000","last_peered":"2024-04-25T10:07:29.584956+0000","last_clean":"2024-04-25T10:07:29.584956+0000","last_became_active":"2024-04-25T10:07:28.591984+0000","last_became_peered":"2024-04-25T10:07:28.591984+0000","last_unstale":"2024-04-25T10:07:29.584956+0000","last_undegraded":"2024-04-25T10:07:29.584956+0000","last_fullsized":"2024-04-25T10:07:29.584956+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:21.511828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977147+0000","last_change":"2024-04-25T10:07:27.576141+0000","last_active":"2024-04-25T10:07:29.977147+0000","last_peered":"2024-04-25T10:07:29.977147+0000","last_clean":"2024-04-25T10:07:29.977147+0000","last_became_active":"2024-04-25T10:07:27.575875+0000","last_became_peered":"2024-04-25T10:07:27.575875+0000","last_unstale":"2024-04-25T10:07:29.977147+0000","last_undegraded":"2024-04-25T10:07:29.977147+0000","last_fullsized":"2024-04-25T10:07:29.977147+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:05:35.463585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976045+0000","last_change":"2024-04-25T10:07:28.594751+0000","last_active":"2024-04-25T10:07:29.976045+0000","last_peered":"2024-04-25T10:07:29.976045+0000","last_clean":"2024-04-25T10:07:29.976045+0000","last_became_active":"2024-04-25T10:07:28.594372+0000","last_became_peered":"2024-04-25T10:07:28.594372+0000","last_unstale":"2024-04-25T10:07:29.976045+0000","last_undegraded":"2024-04-25T10:07:29.976045+0000","last_fullsized":"2024-04-25T10:07:29.976045+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:04.011580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977591+0000","last_change":"2024-04-25T10:07:27.570293+0000","last_active":"2024-04-25T10:07:29.977591+0000","last_peered":"2024-04-25T10:07:29.977591+0000","last_clean":"2024-04-25T10:07:29.977591+0000","last_became_active":"2024-04-25T10:07:27.569996+0000","last_became_peered":"2024-04-25T10:07:27.569996+0000","last_unstale":"2024-04-25T10:07:29.977591+0000","last_undegraded":"2024-04-25T10:07:29.977591+0000","last_fullsized":"2024-04-25T10:07:29.977591+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:38:13.952443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598067+0000","last_change":"2024-04-25T10:07:28.599733+0000","last_active":"2024-04-25T10:07:29.598067+0000","last_peered":"2024-04-25T10:07:29.598067+0000","last_clean":"2024-04-25T10:07:29.598067+0000","last_became_active":"2024-04-25T10:07:28.599451+0000","last_became_peered":"2024-04-25T10:07:28.599451+0000","last_unstale":"2024-04-25T10:07:29.598067+0000","last_undegraded":"2024-04-25T10:07:29.598067+0000","last_fullsized":"2024-04-25T10:07:29.598067+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:28.328551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598070+0000","last_change":"2024-04-25T10:07:27.579007+0000","last_active":"2024-04-25T10:07:29.598070+0000","last_peered":"2024-04-25T10:07:29.598070+0000","last_clean":"2024-04-25T10:07:29.598070+0000","last_became_active":"2024-04-25T10:07:27.578695+0000","last_became_peered":"2024-04-25T10:07:27.578695+0000","last_unstale":"2024-04-25T10:07:29.598070+0000","last_undegraded":"2024-04-25T10:07:29.598070+0000","last_fullsized":"2024-04-25T10:07:29.598070+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:24.171984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583208+0000","last_change":"2024-04-25T10:07:28.580131+0000","last_active":"2024-04-25T10:07:29.583208+0000","last_peered":"2024-04-25T10:07:29.583208+0000","last_clean":"2024-04-25T10:07:29.583208+0000","last_became_active":"2024-04-25T10:07:28.579622+0000","last_became_peered":"2024-04-25T10:07:28.579622+0000","last_unstale":"2024-04-25T10:07:29.583208+0000","last_undegraded":"2024-04-25T10:07:29.583208+0000","last_fullsized":"2024-04-25T10:07:29.583208+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:01.256788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.977355+0000","last_change":"2024-04-25T10:07:27.580265+0000","last_active":"2024-04-25T10:07:29.977355+0000","last_peered":"2024-04-25T10:07:29.977355+0000","last_clean":"2024-04-25T10:07:29.977355+0000","last_became_active":"2024-04-25T10:07:27.579936+0000","last_became_peered":"2024-04-25T10:07:27.579936+0000","last_unstale":"2024-04-25T10:07:29.977355+0000","last_undegraded":"2024-04-25T10:07:29.977355+0000","last_fullsized":"2024-04-25T10:07:29.977355+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:05.936605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978071+0000","last_change":"2024-04-25T10:07:28.576851+0000","last_active":"2024-04-25T10:07:29.978071+0000","last_peered":"2024-04-25T10:07:29.978071+0000","last_clean":"2024-04-25T10:07:29.978071+0000","last_became_active":"2024-04-25T10:07:28.576507+0000","last_became_peered":"2024-04-25T10:07:28.576507+0000","last_unstale":"2024-04-25T10:07:29.978071+0000","last_undegraded":"2024-04-25T10:07:29.978071+0000","last_fullsized":"2024-04-25T10:07:29.978071+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:08.862033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"67'2","reported_seq":16,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.599020+0000","last_change":"2024-04-25T10:07:27.583926+0000","last_active":"2024-04-25T10:07:29.599020+0000","last_peered":"2024-04-25T10:07:29.599020+0000","last_clean":"2024-04-25T10:07:29.599020+0000","last_became_active":"2024-04-25T10:07:27.583491+0000","last_became_peered":"2024-04-25T10:07:27.583491+0000","last_unstale":"2024-04-25T10:07:29.599020+0000","last_undegraded":"2024-04-25T10:07:29.599020+0000","last_fullsized":"2024-04-25T10:07:29.599020+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:14.661346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.581592+0000","last_change":"2024-04-25T10:07:28.597964+0000","last_active":"2024-04-25T10:07:29.581592+0000","last_peered":"2024-04-25T10:07:29.581592+0000","last_clean":"2024-04-25T10:07:29.581592+0000","last_became_active":"2024-04-25T10:07:28.597617+0000","last_became_peered":"2024-04-25T10:07:28.597617+0000","last_unstale":"2024-04-25T10:07:29.581592+0000","last_undegraded":"2024-04-25T10:07:29.581592+0000","last_fullsized":"2024-04-25T10:07:29.581592+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:00.953997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"68'1","reported_seq":15,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.599140+0000","last_change":"2024-04-25T10:07:27.584858+0000","last_active":"2024-04-25T10:07:29.599140+0000","last_peered":"2024-04-25T10:07:29.599140+0000","last_clean":"2024-04-25T10:07:29.599140+0000","last_became_active":"2024-04-25T10:07:27.584497+0000","last_became_peered":"2024-04-25T10:07:27.584497+0000","last_unstale":"2024-04-25T10:07:29.599140+0000","last_undegraded":"2024-04-25T10:07:29.599140+0000","last_fullsized":"2024-04-25T10:07:29.599140+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:17:45.005229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.598864+0000","last_change":"2024-04-25T10:07:28.584205+0000","last_active":"2024-04-25T10:07:29.598864+0000","last_peered":"2024-04-25T10:07:29.598864+0000","last_clean":"2024-04-25T10:07:29.598864+0000","last_became_active":"2024-04-25T10:07:28.583887+0000","last_became_peered":"2024-04-25T10:07:28.583887+0000","last_unstale":"2024-04-25T10:07:29.598864+0000","last_undegraded":"2024-04-25T10:07:29.598864+0000","last_fullsized":"2024-04-25T10:07:29.598864+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:46.465932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978787+0000","last_change":"2024-04-25T10:07:27.583573+0000","last_active":"2024-04-25T10:07:29.978787+0000","last_peered":"2024-04-25T10:07:29.978787+0000","last_clean":"2024-04-25T10:07:29.978787+0000","last_became_active":"2024-04-25T10:07:27.583265+0000","last_became_peered":"2024-04-25T10:07:27.583265+0000","last_unstale":"2024-04-25T10:07:29.978787+0000","last_undegraded":"2024-04-25T10:07:29.978787+0000","last_fullsized":"2024-04-25T10:07:29.978787+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:20:39.183873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.585496+0000","last_change":"2024-04-25T10:07:28.586853+0000","last_active":"2024-04-25T10:07:29.585496+0000","last_peered":"2024-04-25T10:07:29.585496+0000","last_clean":"2024-04-25T10:07:29.585496+0000","last_became_active":"2024-04-25T10:07:28.586574+0000","last_became_peered":"2024-04-25T10:07:28.586574+0000","last_unstale":"2024-04-25T10:07:29.585496+0000","last_undegraded":"2024-04-25T10:07:29.585496+0000","last_fullsized":"2024-04-25T10:07:29.585496+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:38.321120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.980737+0000","last_change":"2024-04-25T10:07:27.577452+0000","last_active":"2024-04-25T10:07:29.980737+0000","last_peered":"2024-04-25T10:07:29.980737+0000","last_clean":"2024-04-25T10:07:29.980737+0000","last_became_active":"2024-04-25T10:07:27.576605+0000","last_became_peered":"2024-04-25T10:07:27.576605+0000","last_unstale":"2024-04-25T10:07:29.980737+0000","last_undegraded":"2024-04-25T10:07:29.980737+0000","last_fullsized":"2024-04-25T10:07:29.980737+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:12:36.193840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.582911+0000","last_change":"2024-04-25T10:07:28.573782+0000","last_active":"2024-04-25T10:07:29.582911+0000","last_peered":"2024-04-25T10:07:29.582911+0000","last_clean":"2024-04-25T10:07:29.582911+0000","last_became_active":"2024-04-25T10:07:28.573436+0000","last_became_peered":"2024-04-25T10:07:28.573436+0000","last_unstale":"2024-04-25T10:07:29.582911+0000","last_undegraded":"2024-04-25T10:07:29.582911+0000","last_fullsized":"2024-04-25T10:07:29.582911+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:24.527924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.583676+0000","last_change":"2024-04-25T10:07:27.588101+0000","last_active":"2024-04-25T10:07:29.583676+0000","last_peered":"2024-04-25T10:07:29.583676+0000","last_clean":"2024-04-25T10:07:29.583676+0000","last_became_active":"2024-04-25T10:07:27.587728+0000","last_became_peered":"2024-04-25T10:07:27.587728+0000","last_unstale":"2024-04-25T10:07:29.583676+0000","last_undegraded":"2024-04-25T10:07:29.583676+0000","last_fullsized":"2024-04-25T10:07:29.583676+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:03:17.921271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975987+0000","last_change":"2024-04-25T10:07:28.589137+0000","last_active":"2024-04-25T10:07:29.975987+0000","last_peered":"2024-04-25T10:07:29.975987+0000","last_clean":"2024-04-25T10:07:29.975987+0000","last_became_active":"2024-04-25T10:07:28.588798+0000","last_became_peered":"2024-04-25T10:07:28.588798+0000","last_unstale":"2024-04-25T10:07:29.975987+0000","last_undegraded":"2024-04-25T10:07:29.975987+0000","last_fullsized":"2024-04-25T10:07:29.975987+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:24:14.933191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976330+0000","last_change":"2024-04-25T10:07:27.576354+0000","last_active":"2024-04-25T10:07:29.976330+0000","last_peered":"2024-04-25T10:07:29.976330+0000","last_clean":"2024-04-25T10:07:29.976330+0000","last_became_active":"2024-04-25T10:07:27.575707+0000","last_became_peered":"2024-04-25T10:07:27.575707+0000","last_unstale":"2024-04-25T10:07:29.976330+0000","last_undegraded":"2024-04-25T10:07:29.976330+0000","last_fullsized":"2024-04-25T10:07:29.976330+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:39.434647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.979970+0000","last_change":"2024-04-25T10:07:28.571635+0000","last_active":"2024-04-25T10:07:29.979970+0000","last_peered":"2024-04-25T10:07:29.979970+0000","last_clean":"2024-04-25T10:07:29.979970+0000","last_became_active":"2024-04-25T10:07:28.567394+0000","last_became_peered":"2024-04-25T10:07:28.567394+0000","last_unstale":"2024-04-25T10:07:29.979970+0000","last_undegraded":"2024-04-25T10:07:29.979970+0000","last_fullsized":"2024-04-25T10:07:29.979970+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:31:54.057726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.976083+0000","last_change":"2024-04-25T10:07:27.572815+0000","last_active":"2024-04-25T10:07:29.976083+0000","last_peered":"2024-04-25T10:07:29.976083+0000","last_clean":"2024-04-25T10:07:29.976083+0000","last_became_active":"2024-04-25T10:07:27.572521+0000","last_became_peered":"2024-04-25T10:07:27.572521+0000","last_unstale":"2024-04-25T10:07:29.976083+0000","last_undegraded":"2024-04-25T10:07:29.976083+0000","last_fullsized":"2024-04-25T10:07:29.976083+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:00:45.452132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.581119+0000","last_change":"2024-04-25T10:07:28.597349+0000","last_active":"2024-04-25T10:07:29.581119+0000","last_peered":"2024-04-25T10:07:29.581119+0000","last_clean":"2024-04-25T10:07:29.581119+0000","last_became_active":"2024-04-25T10:07:28.597048+0000","last_became_peered":"2024-04-25T10:07:28.597048+0000","last_unstale":"2024-04-25T10:07:29.581119+0000","last_undegraded":"2024-04-25T10:07:29.581119+0000","last_fullsized":"2024-04-25T10:07:29.581119+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:56.019480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.978475+0000","last_change":"2024-04-25T10:07:27.575306+0000","last_active":"2024-04-25T10:07:29.978475+0000","last_peered":"2024-04-25T10:07:29.978475+0000","last_clean":"2024-04-25T10:07:29.978475+0000","last_became_active":"2024-04-25T10:07:27.575032+0000","last_became_peered":"2024-04-25T10:07:27.575032+0000","last_unstale":"2024-04-25T10:07:29.978475+0000","last_undegraded":"2024-04-25T10:07:29.978475+0000","last_fullsized":"2024-04-25T10:07:29.978475+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:00.242988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":12,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584196+0000","last_change":"2024-04-25T10:07:28.571851+0000","last_active":"2024-04-25T10:07:29.584196+0000","last_peered":"2024-04-25T10:07:29.584196+0000","last_clean":"2024-04-25T10:07:29.584196+0000","last_became_active":"2024-04-25T10:07:28.571478+0000","last_became_peered":"2024-04-25T10:07:28.571478+0000","last_unstale":"2024-04-25T10:07:29.584196+0000","last_undegraded":"2024-04-25T10:07:29.584196+0000","last_fullsized":"2024-04-25T10:07:29.584196+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:27.529314+0000","last_clean_scrub_stamp":"2024-04-25T10:07:27.529314+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:54.282046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":14,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.975449+0000","last_change":"2024-04-25T10:07:27.568474+0000","last_active":"2024-04-25T10:07:29.975449+0000","last_peered":"2024-04-25T10:07:29.975449+0000","last_clean":"2024-04-25T10:07:29.975449+0000","last_became_active":"2024-04-25T10:07:27.567961+0000","last_became_peered":"2024-04-25T10:07:27.567961+0000","last_unstale":"2024-04-25T10:07:29.975449+0000","last_undegraded":"2024-04-25T10:07:29.975449+0000","last_fullsized":"2024-04-25T10:07:29.975449+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":65,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:07:26.515514+0000","last_clean_scrub_stamp":"2024-04-25T10:07:26.515514+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:05.936605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":262,"reported_epoch":68,"state":"active+clean","last_fresh":"2024-04-25T10:07:29.584880+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:07:29.584880+0000","last_peered":"2024-04-25T10:07:29.584880+0000","last_clean":"2024-04-25T10:07:29.584880+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:07:29.584880+0000","last_undegraded":"2024-04-25T10:07:29.584880+0000","last_fullsized":"2024-04-25T10:07:29.584880+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":9,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574920,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40904,"kb_used_data":1400,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330936,"statfs":{"total":96636764160,"available":96594878464,"internally_reserved":0,"allocated":1433600,"data_stored":966515,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40316,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331524,"statfs":{"total":96636764160,"available":96595480576,"internally_reserved":0,"allocated":827392,"data_stored":375405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,2],"upper_bound":8},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574920,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36152,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335688,"statfs":{"total":96636764160,"available":96599744512,"internally_reserved":0,"allocated":827392,"data_stored":375393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574920,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40324,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331516,"statfs":{"total":96636764160,"available":96595472384,"internally_reserved":0,"allocated":835584,"data_stored":376983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574920,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40304,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331536,"statfs":{"total":96636764160,"available":96595492864,"internally_reserved":0,"allocated":823296,"data_stored":375371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574919,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40308,"kb_used_data":804,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331532,"statfs":{"total":96636764160,"available":96595488768,"internally_reserved":0,"allocated":823296,"data_stored":375371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36172,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335668,"statfs":{"total":96636764160,"available":96599724032,"internally_reserved":0,"allocated":851968,"data_stored":377747,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574920,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46456,"kb_used_data":1384,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325384,"statfs":{"total":96636764160,"available":96589193216,"internally_reserved":0,"allocated":1417216,"data_stored":965739,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:07:33.473 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:07:33.474 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:07:33.518 INFO:tasks.ceph.mds.c.smithi165.stderr:2024-04-25T10:07:33.509+0000 7f2a28800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:07:33.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.721+0000 7fe9ce000640 1 Processor -- start 2024-04-25T10:07:33.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.721+0000 7fe9ce000640 1 -- start start 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9ce000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 0x7fe9c806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9ce000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9c81452a0 0x7fe9c806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9ce000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c80cccc0 con 0x7fe9c81446d0 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9ce000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c80b2b20 con 0x7fe9c81452a0 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9ce000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806d400 con 0x7fe9c8144d50 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9c7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe9c81446d0 0x7fe9c80b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37822/0 (socket says 172.21.15.165:37822) 2024-04-25T10:07:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.722+0000 7fe9c7400640 1 -- 172.21.15.165:0/2731684644 learned_addr learned my addr 172.21.15.165:0/2731684644 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c7e00640 1 --2- 172.21.15.165:0/2731684644 >> v2:172.21.15.172:3300/0 conn(0x7fe9c81452a0 0x7fe9c806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1683731286 0 0) 0x7fe9c80cccc0 con 0x7fe9c81446d0 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 0x7fe9c806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9ac003520 con 0x7fe9c81446d0 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1903607392 0 0) 0x7fe9ac003520 con 0x7fe9c81446d0 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 msgr2=0x7fe9c806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 0x7fe9c806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 >> v2:172.21.15.172:3300/0 conn(0x7fe9c81452a0 msgr2=0x7fe9c806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 --2- 172.21.15.165:0/2731684644 >> v2:172.21.15.172:3300/0 conn(0x7fe9c81452a0 0x7fe9c806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c80cc140 con 0x7fe9c81446d0 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe9b8001670 con 0x7fe9c81446d0 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 >> v1:172.21.15.165:6789/0 conn(0x7fe9c81446d0 legacy=0x7fe9c80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 0x7fe9ac0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9ac004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ac004d40 0x7fe9ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80b2b20 con 0x7fe9ac003fe0 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c806d400 con 0x7fe9ac004d40 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.723+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ac00bac0 con 0x7fe9c8144d50 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9ac004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9c7400640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 0x7fe9ac0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9c7e00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ac004d40 0x7fe9ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9bfe00640 1 -- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 msgr2=0x7fe9ac0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c8144d50 0x7fe9ac0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9bfe00640 1 -- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ac004d40 msgr2=0x7fe9ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ac004d40 0x7fe9ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.724+0000 7fe9bfe00640 1 -- 172.21.15.165:0/2731684644 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ac003520 con 0x7fe9ac003fe0 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.725+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9ac004800 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000ba30 tx=0x7fe9ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2c36f3a2e826c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b0030070 con 0x7fe9ac003fe0 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b002ade0 con 0x7fe9ac003fe0 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9c6a00640 1 -- 172.21.15.165:0/2731684644 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b000b600 con 0x7fe9ac003fe0 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9ce000640 1 -- 172.21.15.165:0/2731684644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 msgr2=0x7fe9ac004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9ce000640 1 --2- 172.21.15.165:0/2731684644 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9ac004800 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000ba30 tx=0x7fe9ac005830 comp rx=0 tx=0).stop 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9c7e00640 1 -- 172.21.15.165:0/2731684644 reap_dead start 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9ce000640 1 -- 172.21.15.165:0/2731684644 shutdown_connections 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.726+0000 7fe9ce000640 1 -- 172.21.15.165:0/2731684644 >> 172.21.15.165:0/2731684644 conn(0x7fe9c806abe0 msgr2=0x7fe9c806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.727+0000 7fe9ce000640 1 -- 172.21.15.165:0/2731684644 shutdown_connections 2024-04-25T10:07:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.727+0000 7fe9ce000640 1 -- 172.21.15.165:0/2731684644 wait complete. 2024-04-25T10:07:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.728+0000 7fe9ce000640 1 Processor -- start 2024-04-25T10:07:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.728+0000 7fe9ce000640 1 -- start start 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.729+0000 7fe9ce000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9c815b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.729+0000 7fe9ce000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c812edb0 0x7fe9c812f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.729+0000 7fe9ce000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9c80a8eb0 0x7fe9c80a9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.729+0000 7fe9ce000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80cc9e0 con 0x7fe9ac003fe0 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.729+0000 7fe9ce000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c80cc700 con 0x7fe9c80a8eb0 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.729+0000 7fe9ce000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c8144ab0 con 0x7fe9c812edb0 2024-04-25T10:07:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9c815b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9c815b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36898/0 (socket says 172.21.15.165:36898) 2024-04-25T10:07:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 -- 172.21.15.165:0/591862360 learned_addr learned my addr 172.21.15.165:0/591862360 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 -- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c812edb0 msgr2=0x7fe9c812f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c812edb0 0x7fe9c812f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7e00640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9c80a8eb0 0x7fe9c80a9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 -- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9c80a8eb0 msgr2=0x7fe9c80a9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9c80a8eb0 0x7fe9c80a9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 -- 172.21.15.165:0/591862360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c8131c10 con 0x7fe9ac003fe0 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.730+0000 7fe9c7400640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9c815b0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800e7b0 tx=0x7fe9b800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.731+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8019070 con 0x7fe9ac003fe0 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.731+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b800c870 con 0x7fe9ac003fe0 2024-04-25T10:07:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.731+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b800cc60 con 0x7fe9ac003fe0 2024-04-25T10:07:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.731+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c81e1060 con 0x7fe9ac003fe0 2024-04-25T10:07:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.731+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c81e1620 con 0x7fe9ac003fe0 2024-04-25T10:07:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.732+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c8073a00 con 0x7fe9ac003fe0 2024-04-25T10:07:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.733+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9b801c070 con 0x7fe9ac003fe0 2024-04-25T10:07:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.734+0000 7fe9bf400640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9a406b4d0 0x7fe9a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.734+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9b8089c00 con 0x7fe9ac003fe0 2024-04-25T10:07:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.734+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9a406b4d0 0x7fe9a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.735+0000 7fe9bfe00640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9a406b4d0 0x7fe9a406d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000ba00 tx=0x7fe9b002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:33.738+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9b8058850 con 0x7fe9ac003fe0 2024-04-25T10:07:34.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.007+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9c80afd90 con 0x7fe9ac003fe0 2024-04-25T10:07:34.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.012+0000 7fe9bf400640 1 -- 172.21.15.165:0/591862360 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe9b8058150 con 0x7fe9ac003fe0 2024-04-25T10:07:34.017 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:07:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:07:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:07:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:07:34.018 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:07:34.018 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:07:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9a406b4d0 msgr2=0x7fe9a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9a406b4d0 0x7fe9a406d990 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fe9b000ba00 tx=0x7fe9b002c000 comp rx=0 tx=0).stop 2024-04-25T10:07:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 msgr2=0x7fe9c815b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9c815b0a0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fe9b800e7b0 tx=0x7fe9b800ec70 comp rx=0 tx=0).stop 2024-04-25T10:07:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 shutdown_connections 2024-04-25T10:07:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9a406b4d0 0x7fe9a406d990 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9c80a8eb0 0x7fe9c80a9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9c812edb0 0x7fe9c812f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 --2- 172.21.15.165:0/591862360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ac003fe0 0x7fe9c815b0a0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 >> 172.21.15.165:0/591862360 conn(0x7fe9c806abe0 msgr2=0x7fe9c8141730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.014+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 shutdown_connections 2024-04-25T10:07:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.015+0000 7fe9ce000640 1 -- 172.21.15.165:0/591862360 wait complete. 2024-04-25T10:07:34.039 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:34.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.235+0000 7ff915800640 1 Processor -- start 2024-04-25T10:07:34.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.235+0000 7ff915800640 1 -- start start 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.236+0000 7ff915800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff91006e870 0x7ff9100cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.236+0000 7ff915800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff910144240 con 0x7ff910143e60 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff915800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff91015e220 con 0x7ff9100cc840 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff915800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff91015e500 con 0x7ff91006e870 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff91006e870 0x7ff9100cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff910143e60 0x7ff91006e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:37830/0 (socket says 172.21.15.165:37830) 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90fe00640 1 -- 172.21.15.165:0/3616787458 learned_addr learned my addr 172.21.15.165:0/3616787458 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90f400640 1 -- 172.21.15.165:0/3616787458 >> v1:172.21.15.172:6789/0 conn(0x7ff9100cc840 legacy=0x7ff91015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90f400640 1 -- 172.21.15.165:0/3616787458 >> v1:172.21.15.165:6789/0 conn(0x7ff910143e60 legacy=0x7ff91006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90f400640 1 -- 172.21.15.165:0/3616787458 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91015e7e0 con 0x7ff91006e870 2024-04-25T10:07:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.237+0000 7ff90ea00640 1 -- 172.21.15.165:0/3616787458 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3977847238 0 0) 0x7ff910144240 con 0x7ff910143e60 2024-04-25T10:07:34.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.238+0000 7ff90f400640 1 --2- 172.21.15.165:0/3616787458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff91006e870 0x7ff9100cc300 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009d90 tx=0x7ff8f802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=991b2255246c15cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff90ea00640 1 -- 172.21.15.165:0/3616787458 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8007740 con 0x7ff91006e870 2024-04-25T10:07:34.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff90ea00640 1 -- 172.21.15.165:0/3616787458 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f80078e0 con 0x7ff91006e870 2024-04-25T10:07:34.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff90ea00640 1 -- 172.21.15.165:0/3616787458 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8007cb0 con 0x7ff91006e870 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff915800640 1 -- 172.21.15.165:0/3616787458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff91006e870 msgr2=0x7ff9100cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff915800640 1 --2- 172.21.15.165:0/3616787458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff91006e870 0x7ff9100cc300 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009d90 tx=0x7ff8f802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff915800640 1 -- 172.21.15.165:0/3616787458 shutdown_connections 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff915800640 1 --2- 172.21.15.165:0/3616787458 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff91006e870 0x7ff9100cc300 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.239+0000 7ff915800640 1 -- 172.21.15.165:0/3616787458 >> 172.21.15.165:0/3616787458 conn(0x7ff91006abe0 msgr2=0x7ff91006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.240+0000 7ff915800640 1 -- 172.21.15.165:0/3616787458 shutdown_connections 2024-04-25T10:07:34.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.240+0000 7ff915800640 1 -- 172.21.15.165:0/3616787458 wait complete. 2024-04-25T10:07:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.242+0000 7ff915800640 1 Processor -- start 2024-04-25T10:07:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.242+0000 7ff915800640 1 -- start start 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.242+0000 7ff915800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e870 0x7ff9100c5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff915800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 0x7ff9100c58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff915800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910143e60 0x7ff9101f32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff915800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100b2eb0 con 0x7ff9100cc840 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff915800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100b2d30 con 0x7ff91006e870 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff915800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9100ccca0 con 0x7ff910143e60 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 0x7ff9100c58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 0x7ff9100c58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36912/0 (socket says 172.21.15.165:36912) 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 -- 172.21.15.165:0/480623916 learned_addr learned my addr 172.21.15.165:0/480623916 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90fe00640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e870 0x7ff9100c5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 -- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910143e60 msgr2=0x7ff9101f32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910143e60 0x7ff9101f32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 -- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e870 msgr2=0x7ff9100c5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e870 0x7ff9100c5380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.243+0000 7ff90f400640 1 -- 172.21.15.165:0/480623916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9100c5e00 con 0x7ff9100cc840 2024-04-25T10:07:34.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.244+0000 7ff90f400640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 0x7ff9100c58c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009d90 tx=0x7ff8f8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.244+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f8040070 con 0x7ff9100cc840 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.244+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f8031c40 con 0x7ff9100cc840 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.244+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f803c420 con 0x7ff9100cc840 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.244+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9100c60b0 con 0x7ff9100cc840 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.244+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9100c6670 con 0x7ff9100cc840 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.245+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff910073990 con 0x7ff9100cc840 2024-04-25T10:07:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.246+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8f8039040 con 0x7ff9100cc840 2024-04-25T10:07:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.247+0000 7ff907e00640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.247+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8f80aba50 con 0x7ff9100cc840 2024-04-25T10:07:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.247+0000 7ff90fe00640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.248+0000 7ff90fe00640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff900000f60 tx=0x7ff900009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.252+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8f807a730 con 0x7ff9100cc840 2024-04-25T10:07:34.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.488+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9100b48c0 con 0x7ff9100cc840 2024-04-25T10:07:34.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.489+0000 7ff907e00640 1 -- 172.21.15.165:0/480623916 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4945 (secure 0 0 0) 0x7ff8f807a030 con 0x7ff9100cc840 2024-04-25T10:07:34.494 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:34.495 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/57109722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":57109722},{"type":"v1","addr":"172.21.15.172:6834","nonce":57109722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":4982,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3177295444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3177295444},{"type":"v1","addr":"172.21.15.172:6835","nonce":3177295444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/2353868161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2353868161},{"type":"v1","addr":"172.21.15.165:6836","nonce":2353868161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:31.082094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1883921643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1883921643},{"type":"v1","addr":"172.21.15.165:6837","nonce":1883921643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:34.495 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 40 2024-04-25T10:07:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8ec06b4d0 msgr2=0x7ff8ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff900000f60 tx=0x7ff900009040 comp rx=0 tx=0).stop 2024-04-25T10:07:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 msgr2=0x7ff9100c58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 0x7ff9100c58c0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7ff8f8009d90 tx=0x7ff8f8004a40 comp rx=0 tx=0).stop 2024-04-25T10:07:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 shutdown_connections 2024-04-25T10:07:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8ec06b4d0 0x7ff8ec06d990 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff910143e60 0x7ff9101f32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9100cc840 0x7ff9100c58c0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 --2- 172.21.15.165:0/480623916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff91006e870 0x7ff9100c5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.491+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 >> 172.21.15.165:0/480623916 conn(0x7ff91006abe0 msgr2=0x7ff9100ade50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.492+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 shutdown_connections 2024-04-25T10:07:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:34.492+0000 7ff915800640 1 -- 172.21.15.165:0/480623916 wait complete. 2024-04-25T10:07:34.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:28.014241+0000', 'modified': '2024-04-25T10:07:31.082094+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6624}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6624': {'gid': 6624, 'name': 'c', 'rank': 0, 'incarnation': 37, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6837/1883921643', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 1883921643}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1883921643}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:34.918 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:34.918 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:07:35.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.126+0000 7fc4e5c00640 1 Processor -- start 2024-04-25T10:07:35.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.126+0000 7fc4e5c00640 1 -- start start 2024-04-25T10:07:35.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4e5c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc4e013d9f0 0x7fc4e013df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4e5c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4e00b0730 0x7fc4e00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4e5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e006e150 0x7fc4e006e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4e5c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e006d390 con 0x7fc4e00b0730 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4e5c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e006d670 con 0x7fc4e013d9f0 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4e5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e006d950 con 0x7fc4e006e150 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4d7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4e00b0730 0x7fc4e00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4d7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4e00b0730 0x7fc4e00b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36914/0 (socket says 172.21.15.165:36914) 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.127+0000 7fc4d7400640 1 -- 172.21.15.165:0/577307843 learned_addr learned my addr 172.21.15.165:0/577307843 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/577307843 >> v2:172.21.15.172:3300/0 conn(0x7fc4e013d9f0 0x7fc4e013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4e5200640 1 --2- 172.21.15.165:0/577307843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e006e150 0x7fc4e006e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4d7400640 1 -- 172.21.15.165:0/577307843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e006e150 msgr2=0x7fc4e006e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4d7400640 1 --2- 172.21.15.165:0/577307843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e006e150 0x7fc4e006e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4d7400640 1 -- 172.21.15.165:0/577307843 >> v2:172.21.15.172:3300/0 conn(0x7fc4e013d9f0 msgr2=0x7fc4e013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4d7400640 1 --2- 172.21.15.165:0/577307843 >> v2:172.21.15.172:3300/0 conn(0x7fc4e013d9f0 0x7fc4e013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.128+0000 7fc4d7400640 1 -- 172.21.15.165:0/577307843 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e006dc30 con 0x7fc4e00b0730 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.129+0000 7fc4d7400640 1 --2- 172.21.15.165:0/577307843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e00b0730 0x7fc4e00b0bc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0072520 tx=0x7fc4c802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd3730a86fe54e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.129+0000 7fc4df400640 1 -- 172.21.15.165:0/577307843 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8037070 con 0x7fc4e00b0730 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.129+0000 7fc4df400640 1 -- 172.21.15.165:0/577307843 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4c802ae00 con 0x7fc4e00b0730 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.129+0000 7fc4df400640 1 -- 172.21.15.165:0/577307843 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c8032660 con 0x7fc4e00b0730 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 -- 172.21.15.165:0/577307843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e00b0730 msgr2=0x7fc4e00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/577307843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e00b0730 0x7fc4e00b0bc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0072520 tx=0x7fc4c802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 -- 172.21.15.165:0/577307843 shutdown_connections 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/577307843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e006e150 0x7fc4e006e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/577307843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e00b0730 0x7fc4e00b0bc0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/577307843 >> v2:172.21.15.172:3300/0 conn(0x7fc4e013d9f0 0x7fc4e013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 -- 172.21.15.165:0/577307843 >> 172.21.15.165:0/577307843 conn(0x7fc4e0061cc0 msgr2=0x7fc4e006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 -- 172.21.15.165:0/577307843 shutdown_connections 2024-04-25T10:07:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.130+0000 7fc4e5c00640 1 -- 172.21.15.165:0/577307843 wait complete. 2024-04-25T10:07:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.132+0000 7fc4e5c00640 1 Processor -- start 2024-04-25T10:07:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.132+0000 7fc4e5c00640 1 -- start start 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4e5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 0x7fc4e01ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4e5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4e00b0730 0x7fc4e01554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4e5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e013d9f0 0x7fc4e0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4e5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e013e5f0 con 0x7fc4e006e150 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4e5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e013e470 con 0x7fc4e00b0730 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4e5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e00ce880 con 0x7fc4e013d9f0 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 0x7fc4e01ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 0x7fc4e01ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36924/0 (socket says 172.21.15.165:36924) 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2013712698 learned_addr learned my addr 172.21.15.165:0/2013712698 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e013d9f0 msgr2=0x7fc4e0153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e013d9f0 0x7fc4e0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4e00b0730 msgr2=0x7fc4e01554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4d7400640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4e00b0730 0x7fc4e01554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4e00b0730 0x7fc4e01554e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.133+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2013712698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e015ddf0 con 0x7fc4e006e150 2024-04-25T10:07:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.134+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 0x7fc4e01ed6a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000d620 tx=0x7fc4d000dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.134+0000 7fc4dcc00640 1 -- 172.21.15.165:0/2013712698 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000b620 con 0x7fc4e006e150 2024-04-25T10:07:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.134+0000 7fc4dcc00640 1 -- 172.21.15.165:0/2013712698 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d0019070 con 0x7fc4e006e150 2024-04-25T10:07:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.134+0000 7fc4dcc00640 1 -- 172.21.15.165:0/2013712698 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000ba00 con 0x7fc4e006e150 2024-04-25T10:07:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.134+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e015cb30 con 0x7fc4e006e150 2024-04-25T10:07:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.134+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e0153850 con 0x7fc4e006e150 2024-04-25T10:07:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.137+0000 7fc4dcc00640 1 -- 172.21.15.165:0/2013712698 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4d000bba0 con 0x7fc4e006e150 2024-04-25T10:07:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.138+0000 7fc4dcc00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4bc06b4d0 0x7fc4bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.138+0000 7fc4dcc00640 1 -- 172.21.15.165:0/2013712698 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4d008a850 con 0x7fc4e006e150 2024-04-25T10:07:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.139+0000 7fc4d7400640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4bc06b4d0 0x7fc4bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.139+0000 7fc4e9faf740 1 Processor -- start 2024-04-25T10:07:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.139+0000 7fc4e9faf740 1 -- start start 2024-04-25T10:07:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.139+0000 7fc4d7400640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4bc06b4d0 0x7fc4bc06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8034000 tx=0x7fc4c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 0x558e008e0970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008b8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e009017a0 0x558e008b72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0081c620 con 0x558e008b8b30 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e007c5ca0 con 0x558e009017a0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e008186c0 con 0x558e008e0eb0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 0x558e008e0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 0x558e008e0970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36928/0 (socket says 172.21.15.165:36928) 2024-04-25T10:07:35.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.140+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2314536723 learned_addr learned my addr 172.21.15.165:0/2314536723 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 msgr2=0x558e008b8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4d7400640 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008b8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4e5200640 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e009017a0 0x558e008b72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008b8f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e009017a0 msgr2=0x558e008b72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e009017a0 0x558e008b72f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4dfe00640 1 -- 172.21.15.165:0/2314536723 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e007ccf40 con 0x558e008b8b30 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 0x558e008e0970 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0058d30 tx=0x7fc4d00594a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fc78a24ab60fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4d4c00640 1 -- 172.21.15.165:0/2314536723 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d005c070 con 0x558e008b8b30 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4d4c00640 1 -- 172.21.15.165:0/2314536723 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00614b0 con 0x558e008b8b30 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.141+0000 7fc4d4c00640 1 -- 172.21.15.165:0/2314536723 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d005f070 con 0x558e008b8b30 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 -- 172.21.15.165:0/2314536723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 msgr2=0x558e008e0970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 0x558e008e0970 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0058d30 tx=0x7fc4d00594a0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 -- 172.21.15.165:0/2314536723 shutdown_connections 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e009017a0 0x558e008b72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008b8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/2314536723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e008b8b30 0x558e008e0970 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 -- 172.21.15.165:0/2314536723 >> 172.21.15.165:0/2314536723 conn(0x558e008b5d50 msgr2=0x558e008fe730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 -- 172.21.15.165:0/2314536723 shutdown_connections 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.142+0000 7fc4e9faf740 1 -- 172.21.15.165:0/2314536723 wait complete. 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.143+0000 7fc4e9faf740 1 -- start start 2024-04-25T10:07:35.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.143+0000 7fc4e9faf740 1 Processor -- start 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008dec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e008df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e16d0 0x558e008e1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e008e04e0 con 0x558e009017a0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e007bb1f0 con 0x558e008e16d0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e008dfed0 con 0x558e008e0eb0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e9faf740 10 client.0 _ll_register_callbacks cb 0x558e007d8ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e008df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e008df1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36932/0 (socket says 172.21.15.165:36932) 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 -- 172.21.15.165:0/1725375827 learned_addr learned my addr 172.21.15.165:0/1725375827 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008dec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4e5200640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e16d0 0x558e008e1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 -- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 msgr2=0x558e008dec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008dec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 -- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e16d0 msgr2=0x558e008e1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.144+0000 7fc4d7400640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e16d0 0x558e008e1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4d7400640 1 -- 172.21.15.165:0/1725375827 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e007dac50 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4d7400640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e008df1b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8035040 tx=0x7fc4c802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe77cc9a5c7ec44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4cd600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4cd600640 1 -- 172.21.15.165:0/1725375827 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c804d060 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d00614b0 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4cd600640 1 -- 172.21.15.165:0/1725375827 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4c803cab0 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4cd600640 1 -- 172.21.15.165:0/1725375827 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4c8053050 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4e9faf740 10 client.6768 fetch_fsmap learned FSMap version 40 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.145+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x558e005312d0 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4cd600640 1 -- 172.21.15.165:0/1725375827 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d00614b0 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4cd600640 1 -- 172.21.15.165:0/1725375827 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4c8056020 con 0x558e009017a0 2024-04-25T10:07:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4e9faf740 10 client.6768 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4e9faf740 10 client.6768 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4e9faf740 20 client.6768 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e0090ed30 0x558e008cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4e9faf740 4 client.6768 mds_command: new command op to 6624 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 --> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e00829b60 con 0x558e0090ed30 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.146+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e0090ed30 0x558e008cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.147+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e0090ed30 0x558e008cf9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2032f4816214d31 server_cookie=b960ca5d9d8b6484 in_seq=0 out_seq=0 2024-04-25T10:07:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.147+0000 7fc4cd600640 10 client.6768 ms_handle_connect on v2:172.21.15.165:6835/1883921643 2024-04-25T10:07:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.147+0000 7fc4cd600640 1 -- 172.21.15.165:0/1725375827 <== mds.0 v2:172.21.15.165:6835/1883921643 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558e00829b60 con 0x558e0090ed30 2024-04-25T10:07:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.147+0000 7fc4cd600640 10 client.6768 handle_command_reply: tid=0 2024-04-25T10:07:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.147+0000 7fc4e9faf740 1 client.6768 shutdown 2024-04-25T10:07:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 msgr2=0x558e008df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e008df1b0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8035040 tx=0x7fc4c802c070 comp rx=0 tx=0).stop 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 shutdown_connections 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e0090ed30 0x558e008cf9d0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e16d0 0x558e008e1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e008df1b0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4cd600640 0 client.6768 ms_handle_reset on v2:172.21.15.165:6835/1883921643 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1725375827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e0eb0 0x558e008dec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 >> 172.21.15.165:0/1725375827 conn(0x558e008b8b30 msgr2=0x558e008fe530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 shutdown_connections 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1725375827 wait complete. 2024-04-25T10:07:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.148+0000 7fc4e9faf740 20 client.6768 trim_cache size 0 max 16384 2024-04-25T10:07:35.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.165+0000 7fc4e9faf740 1 Processor -- start 2024-04-25T10:07:35.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.165+0000 7fc4e9faf740 1 -- start start 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.166+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e008df1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.166+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008deae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.166+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00901b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.166+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e01ff6a0 con 0x558e009017a0 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.166+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e01f7730 con 0x558e008e0eb0 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.166+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e01ff140 con 0x558e008e16d0 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00901b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00901b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36940/0 (socket says 172.21.15.165:36940) 2024-04-25T10:07:35.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 -- 172.21.15.165:0/336395295 learned_addr learned my addr 172.21.15.165:0/336395295 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e008df1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 -- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 msgr2=0x558e008deae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008deae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 -- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 msgr2=0x558e008df1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e008df1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 -- 172.21.15.165:0/336395295 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e005fb6b0 con 0x558e009017a0 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.167+0000 7fc4e5200640 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00901b80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009db0 tx=0x7fc4d800e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a0bba8327d5a14b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4cfe00640 1 -- 172.21.15.165:0/336395295 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d801b070 con 0x558e009017a0 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4cfe00640 1 -- 172.21.15.165:0/336395295 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800c660 con 0x558e009017a0 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4cfe00640 1 -- 172.21.15.165:0/336395295 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800ca30 con 0x558e009017a0 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 -- 172.21.15.165:0/336395295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 msgr2=0x558e00901b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00901b80 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009db0 tx=0x7fc4d800e7c0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 -- 172.21.15.165:0/336395295 shutdown_connections 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00901b80 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008deae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/336395295 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e008df1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.168+0000 7fc4e9faf740 1 -- 172.21.15.165:0/336395295 >> 172.21.15.165:0/336395295 conn(0x558e008b8b30 msgr2=0x558e008e71b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.169+0000 7fc4e9faf740 1 -- 172.21.15.165:0/336395295 shutdown_connections 2024-04-25T10:07:35.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.169+0000 7fc4e9faf740 1 -- 172.21.15.165:0/336395295 wait complete. 2024-04-25T10:07:35.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.169+0000 7fc4e9faf740 1 -- start start 2024-04-25T10:07:35.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.169+0000 7fc4e9faf740 1 Processor -- start 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e009638e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00900540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0090ebb0 con 0x558e009017a0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e0090fb40 con 0x558e008e0eb0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e0090f180 con 0x558e008e16d0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.170+0000 7fc4e9faf740 10 client.0 _ll_register_callbacks cb 0x7fc4c8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00900540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00900540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36954/0 (socket says 172.21.15.165:36954) 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 -- 172.21.15.165:0/1615246161 learned_addr learned my addr 172.21.15.165:0/1615246161 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e009638e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4d7400640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 -- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 msgr2=0x558e008cdbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 -- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 msgr2=0x558e009638e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e009638e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 -- 172.21.15.165:0/1615246161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558e008d0c70 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e009638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4e5200640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00900540 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800ecc0 tx=0x7fc4d8007060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e4af0543aa62da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.171+0000 7fc4cd600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4cd600640 1 -- 172.21.15.165:0/1615246161 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d8026020 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4cd600640 1 -- 172.21.15.165:0/1615246161 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4d800c810 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d800c660 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4cd600640 1 -- 172.21.15.165:0/1615246161 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d8030050 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 10 client.6774 fetch_fsmap learned FSMap version 40 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x558e0090f3a0 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4cd600640 1 -- 172.21.15.165:0/1615246161 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d800c660 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4cd600640 1 -- 172.21.15.165:0/1615246161 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 40) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc4d8033020 con 0x558e009017a0 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 10 client.6774 fetch_fsmap finished waiting for FSMap version 40 2024-04-25T10:07:35.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 10 client.6774 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 20 client.6774 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e00917c20 0x558e00918000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 4 client.6774 mds_command: new command op to 6624 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.172+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 --> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e00823eb0 con 0x558e00917c20 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.173+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e00917c20 0x558e00918000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.173+0000 7fc4cd600640 10 client.6774 ms_handle_connect on v2:172.21.15.165:6835/1883921643 2024-04-25T10:07:35.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.173+0000 7fc4dfe00640 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e00917c20 0x558e00918000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbf55861613f531 server_cookie=6325ff8f1ac3ba48 in_seq=0 out_seq=0 2024-04-25T10:07:35.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.173+0000 7fc4cd600640 1 -- 172.21.15.165:0/1615246161 <== mds.0 v2:172.21.15.165:6835/1883921643 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x558e00823eb0 con 0x558e00917c20 2024-04-25T10:07:35.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.173+0000 7fc4cd600640 10 client.6774 handle_command_reply: tid=0 2024-04-25T10:07:35.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.173+0000 7fc4e9faf740 1 client.6774 shutdown 2024-04-25T10:07:35.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 msgr2=0x558e00900540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00900540 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc4d800ecc0 tx=0x7fc4d8007060 comp rx=0 tx=0).stop 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 shutdown_connections 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:6835/1883921643,v1:172.21.15.165:6837/1883921643] conn(0x558e00917c20 0x558e00918000 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558e009017a0 0x558e00900540 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558e008e16d0 0x558e008cdbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 --2- 172.21.15.165:0/1615246161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558e008e0eb0 0x558e009638e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4cd600640 0 client.6774 ms_handle_reset on v2:172.21.15.165:6835/1883921643 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 >> 172.21.15.165:0/1615246161 conn(0x558e008b8b30 msgr2=0x558e008b9840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 shutdown_connections 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 1 -- 172.21.15.165:0/1615246161 wait complete. 2024-04-25T10:07:35.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.174+0000 7fc4e9faf740 20 client.6774 trim_cache size 0 max 16384 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 6624, 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.597683859, 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 40, 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 68, 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 68, 2024-04-25T10:07:35.184 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 12.793997936 2024-04-25T10:07:35.185 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:07:35.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4bc06b4d0 msgr2=0x7fc4bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4bc06b4d0 0x7fc4bc06d990 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fc4c8034000 tx=0x7fc4c802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:35.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 msgr2=0x7fc4e01ed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 0x7fc4e01ed6a0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000d620 tx=0x7fc4d000dae0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 shutdown_connections 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc4bc06b4d0 0x7fc4bc06d990 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4e013d9f0 0x7fc4e0153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4e00b0730 0x7fc4e01554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 --2- 172.21.15.165:0/2013712698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4e006e150 0x7fc4e01ed6a0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 >> 172.21.15.165:0/2013712698 conn(0x7fc4e0061cc0 msgr2=0x7fc4e013f4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 shutdown_connections 2024-04-25T10:07:35.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.175+0000 7fc4e5c00640 1 -- 172.21.15.165:0/2013712698 wait complete. 2024-04-25T10:07:35.196 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:35.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.375+0000 7f59be000640 1 Processor -- start 2024-04-25T10:07:35.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.375+0000 7f59be000640 1 -- start start 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.376+0000 7f59be000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 0x7f59b8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.376+0000 7f59be000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59b80cda30 0x7f59b815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.376+0000 7f59be000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59b815bf30 con 0x7f59b8071530 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.376+0000 7f59be000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59b815c860 con 0x7f59b80cda30 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.376+0000 7f59be000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b8071970 con 0x7f59b8144440 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.377+0000 7f59bd600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59b80cda30 0x7f59b815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.377+0000 7f59bcc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 0x7f59b8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.377+0000 7f59bcc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 0x7f59b8070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:36080/0 (socket says 172.21.15.165:36080) 2024-04-25T10:07:35.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.377+0000 7f59bcc00640 1 -- 172.21.15.165:0/1099539524 learned_addr learned my addr 172.21.15.165:0/1099539524 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.378+0000 7f59bcc00640 1 -- 172.21.15.165:0/1099539524 >> v2:172.21.15.172:3300/0 conn(0x7f59b80cda30 msgr2=0x7f59b815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:35.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.378+0000 7f59bcc00640 1 --2- 172.21.15.165:0/1099539524 >> v2:172.21.15.172:3300/0 conn(0x7f59b80cda30 0x7f59b815ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.378+0000 7f59bcc00640 1 -- 172.21.15.165:0/1099539524 >> v1:172.21.15.172:6789/0 conn(0x7f59b8071530 legacy=0x7f59b80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.378+0000 7f59bcc00640 1 -- 172.21.15.165:0/1099539524 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b815c210 con 0x7f59b8144440 2024-04-25T10:07:35.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.378+0000 7f59b7400640 1 -- 172.21.15.165:0/1099539524 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761381968 0 0) 0x7f59b815bf30 con 0x7f59b8071530 2024-04-25T10:07:35.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.379+0000 7f59bcc00640 1 --2- 172.21.15.165:0/1099539524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 0x7f59b8070ff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009870 tx=0x7f59ac02a300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a7f3ebce5e48f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59b7400640 1 -- 172.21.15.165:0/1099539524 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac037070 con 0x7f59b8144440 2024-04-25T10:07:35.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59b7400640 1 -- 172.21.15.165:0/1099539524 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59ac02ae50 con 0x7f59b8144440 2024-04-25T10:07:35.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59b7400640 1 -- 172.21.15.165:0/1099539524 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac032860 con 0x7f59b8144440 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 -- 172.21.15.165:0/1099539524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 msgr2=0x7f59b8070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 --2- 172.21.15.165:0/1099539524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 0x7f59b8070ff0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59ac009870 tx=0x7f59ac02a300 comp rx=0 tx=0).stop 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 -- 172.21.15.165:0/1099539524 shutdown_connections 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 --2- 172.21.15.165:0/1099539524 >> v2:172.21.15.172:3300/0 conn(0x7f59b80cda30 0x7f59b815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 --2- 172.21.15.165:0/1099539524 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8144440 0x7f59b8070ff0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 -- 172.21.15.165:0/1099539524 >> 172.21.15.165:0/1099539524 conn(0x7f59b806abe0 msgr2=0x7f59b806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 -- 172.21.15.165:0/1099539524 shutdown_connections 2024-04-25T10:07:35.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.380+0000 7f59be000640 1 -- 172.21.15.165:0/1099539524 wait complete. 2024-04-25T10:07:35.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.382+0000 7f59be000640 1 Processor -- start 2024-04-25T10:07:35.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.382+0000 7f59be000640 1 -- start start 2024-04-25T10:07:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59be000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8071530 0x7f59b81e3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59be000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 0x7f59b8159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59be000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8144440 0x7f59b81e2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59be000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b815bc30 con 0x7f59b80cda30 2024-04-25T10:07:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59be000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b80cdea0 con 0x7f59b8144440 2024-04-25T10:07:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59be000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59b815d230 con 0x7f59b8071530 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.383+0000 7f59b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 0x7f59b8159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 0x7f59b8159180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36962/0 (socket says 172.21.15.165:36962) 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 -- 172.21.15.165:0/642740071 learned_addr learned my addr 172.21.15.165:0/642740071 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59bcc00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8071530 0x7f59b81e3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59bd600640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8144440 0x7f59b81e2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 -- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8071530 msgr2=0x7f59b81e3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8071530 0x7f59b81e3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 -- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8144440 msgr2=0x7f59b81e2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8144440 0x7f59b81e2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 -- 172.21.15.165:0/642740071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59b81e4250 con 0x7f59b80cda30 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.384+0000 7f59b7e00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 0x7f59b8159180 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f59a400c4b0 tx=0x7f59a400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.385+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4017070 con 0x7f59b80cda30 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.385+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a4005480 con 0x7f59b80cda30 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.385+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a4003830 con 0x7f59b80cda30 2024-04-25T10:07:35.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.385+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b81596c0 con 0x7f59b80cda30 2024-04-25T10:07:35.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.385+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b81e2f50 con 0x7f59b80cda30 2024-04-25T10:07:35.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.386+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b8073990 con 0x7f59b80cda30 2024-04-25T10:07:35.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.387+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59a401a070 con 0x7f59b80cda30 2024-04-25T10:07:35.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.388+0000 7f59b4c00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f599806b4d0 0x7f599806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:35.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.388+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f59a40897e0 con 0x7f59b80cda30 2024-04-25T10:07:35.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.388+0000 7f59bcc00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f599806b4d0 0x7f599806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.389+0000 7f59bcc00640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f599806b4d0 0x7f599806d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f59ac034000 tx=0x7f59ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:35.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.392+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59a4058430 con 0x7f59b80cda30 2024-04-25T10:07:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.626+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f59b813f5b0 con 0x7f59b80cda30 2024-04-25T10:07:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.626+0000 7f59b4c00640 1 -- 172.21.15.165:0/642740071 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f59a4057d30 con 0x7f59b80cda30 2024-04-25T10:07:35.632 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f599806b4d0 msgr2=0x7f599806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f599806b4d0 0x7f599806d990 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f59ac034000 tx=0x7f59ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 msgr2=0x7f59b8159180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 0x7f59b8159180 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f59a400c4b0 tx=0x7f59a400c970 comp rx=0 tx=0).stop 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 shutdown_connections 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f599806b4d0 0x7f599806d990 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59b8144440 0x7f59b81e2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59b80cda30 0x7f59b8159180 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 --2- 172.21.15.165:0/642740071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59b8071530 0x7f59b81e3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 >> 172.21.15.165:0/642740071 conn(0x7f59b806abe0 msgr2=0x7f59b80cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 shutdown_connections 2024-04-25T10:07:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:35.629+0000 7f59be000640 1 -- 172.21.15.165:0/642740071 wait complete. 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:07:35.651 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:35.652 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:35.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.652 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:35.697 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.698 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42310sec preferred_lft 42310sec 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:35.699 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:35.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.701 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:35.817 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:07:35.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.822 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:07:35.877 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:07:35.877 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:35.877 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:35.877 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:35.877 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:35.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:35.878 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:07:35.878 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:07:35.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:35.879 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:35.879 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:35.879 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:07:36.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.027 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:07:36.078 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:36.078 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:36.079 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:36.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.079 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:07:36.159 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:36.159 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:36.211 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:36.211 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:36.211 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:36.264 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:36.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.264 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:36.395 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:36.395 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:36.395 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:07:36.416 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:36.417 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:36.417 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:07:36.417 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:07:36.417 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:36.442 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:07:36.443 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:07:36.445 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:07:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:36.445 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:36.450 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:36.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.450 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:36.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.527 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:36.591 INFO:teuthology.orchestra.run.smithi165.stdout:client6780 2024-04-25T10:07:36.592 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:36.592 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:36.641 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.641 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir scrub_repair_path) 2024-04-25T10:07:36.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:36.696 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch scrub_repair_path/file) 2024-04-25T10:07:36.749 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:07:36.749 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:07:36.750 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:07:36.750 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:07:36.750 DEBUG:teuthology.orchestra.run.smithi165:> print(os.stat("/home/ubuntu/cephtest/mnt.0/scrub_repair_path").st_ino) 2024-04-25T10:07:36.750 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:07:36.879 INFO:teuthology.orchestra.run.smithi165.stdout:1099511627776 2024-04-25T10:07:36.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:36 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:07:36.891 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:36.891 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:36.988 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:36.988 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:37.041 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:37.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.236+0000 7fc0c7000640 1 Processor -- start 2024-04-25T10:07:37.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.237+0000 7fc0c7000640 1 -- start start 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.238+0000 7fc0c7000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc0c015e900 0x7fc0c00adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.238+0000 7fc0c7000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc0c00ae0d0 0x7fc0c0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.238+0000 7fc0c7000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c015ed90 con 0x7fc0c015e520 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.238+0000 7fc0c7000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c00aee30 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.238+0000 7fc0c7000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0c00ae590 con 0x7fc0c015e900 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.238+0000 7fc0c5c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc0c015e520 0x7fc0c015f040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58132/0 (socket says 172.21.15.165:58132) 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c5c00640 1 -- 172.21.15.165:0/2522798330 learned_addr learned my addr 172.21.15.165:0/2522798330 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c6600640 1 --2- 172.21.15.165:0/2522798330 >> v2:172.21.15.165:3300/0 conn(0x7fc0c00ae0d0 0x7fc0c0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c5200640 1 --2- 172.21.15.165:0/2522798330 >> v2:172.21.15.172:3300/0 conn(0x7fc0c015e900 0x7fc0c00adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c6600640 1 -- 172.21.15.165:0/2522798330 >> v2:172.21.15.172:3300/0 conn(0x7fc0c015e900 msgr2=0x7fc0c00adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0bfe00640 1 -- 172.21.15.165:0/2522798330 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144885145 0 0) 0x7fc0c015ed90 con 0x7fc0c015e520 2024-04-25T10:07:37.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c6600640 1 --2- 172.21.15.165:0/2522798330 >> v2:172.21.15.172:3300/0 conn(0x7fc0c015e900 0x7fc0c00adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c6600640 1 -- 172.21.15.165:0/2522798330 >> v1:172.21.15.165:6789/0 conn(0x7fc0c015e520 legacy=0x7fc0c015f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.239+0000 7fc0c6600640 1 -- 172.21.15.165:0/2522798330 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c00ae870 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.240+0000 7fc0c6600640 1 --2- 172.21.15.165:0/2522798330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0070ff0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00cf050 tx=0x7fc0b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ac31adb9326803d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0bfe00640 1 -- 172.21.15.165:0/2522798330 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8007740 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0bfe00640 1 -- 172.21.15.165:0/2522798330 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b80078e0 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0bfe00640 1 -- 172.21.15.165:0/2522798330 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b8007cb0 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0c7000640 1 -- 172.21.15.165:0/2522798330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 msgr2=0x7fc0c0070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0c7000640 1 --2- 172.21.15.165:0/2522798330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0070ff0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fc0c00cf050 tx=0x7fc0b802a680 comp rx=0 tx=0).stop 2024-04-25T10:07:37.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0c7000640 1 -- 172.21.15.165:0/2522798330 shutdown_connections 2024-04-25T10:07:37.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0c7000640 1 --2- 172.21.15.165:0/2522798330 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0070ff0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0c7000640 1 --2- 172.21.15.165:0/2522798330 >> v2:172.21.15.172:3300/0 conn(0x7fc0c015e900 0x7fc0c00adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.241+0000 7fc0c7000640 1 -- 172.21.15.165:0/2522798330 >> 172.21.15.165:0/2522798330 conn(0x7fc0c006abe0 msgr2=0x7fc0c006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.242+0000 7fc0c7000640 1 -- 172.21.15.165:0/2522798330 shutdown_connections 2024-04-25T10:07:37.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.242+0000 7fc0c7000640 1 -- 172.21.15.165:0/2522798330 wait complete. 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.243+0000 7fc0c7000640 1 Processor -- start 2024-04-25T10:07:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 -- start start 2024-04-25T10:07:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0c015e520 0x7fc0c0158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0c015e900 0x7fc0c00cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c0071530 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c006d1f0 con 0x7fc0c015e900 2024-04-25T10:07:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.244+0000 7fc0c7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c00716b0 con 0x7fc0c015e520 2024-04-25T10:07:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0157820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55910/0 (socket says 172.21.15.165:55910) 2024-04-25T10:07:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 -- 172.21.15.165:0/3615298404 learned_addr learned my addr 172.21.15.165:0/3615298404 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5200640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0c015e520 0x7fc0c0158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 -- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0c015e520 msgr2=0x7fc0c0158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0c015e520 0x7fc0c0158d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c6600640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0c015e900 0x7fc0c00cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 -- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0c015e900 msgr2=0x7fc0c00cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0c015e900 0x7fc0c00cdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.245+0000 7fc0c5c00640 1 -- 172.21.15.165:0/3615298404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c015a160 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.246+0000 7fc0c5c00640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0157820 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c4d0 tx=0x7fc0b400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.246+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4018070 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.246+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b4007500 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.246+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4004a20 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.246+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c015a440 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.246+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c015aa00 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.247+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c0073990 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.248+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc0b401b070 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.250+0000 7fc0bd600640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc0a406b4d0 0x7fc0a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.250+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0b40896e0 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.250+0000 7fc0c5200640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc0a406b4d0 0x7fc0a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.250+0000 7fc0c5200640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc0a406b4d0 0x7fc0a406d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000b7c0 tx=0x7fc0b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.253+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0b40583c0 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.449+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0c00b0960 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.452+0000 7fc0bd600640 1 -- 172.21.15.165:0/3615298404 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+16920 (secure 0 0 0) 0x7fc0b4057cc0 con 0x7fc0c00ae0d0 2024-04-25T10:07:37.458 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:37.458 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":68,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:29.572452+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc0a406b4d0 msgr2=0x7fc0a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc0a406b4d0 0x7fc0a406d990 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc0b000b7c0 tx=0x7fc0b0008040 comp rx=0 tx=0).stop 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 msgr2=0x7fc0c0157820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0157820 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c4d0 tx=0x7fc0b400c990 comp rx=0 tx=0).stop 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 shutdown_connections 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc0a406b4d0 0x7fc0a406d990 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0c015e900 0x7fc0c00cdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0c015e520 0x7fc0c0158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 --2- 172.21.15.165:0/3615298404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0c00ae0d0 0x7fc0c0157820 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 >> 172.21.15.165:0/3615298404 conn(0x7fc0c006abe0 msgr2=0x7fc0c00b0250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 shutdown_connections 2024-04-25T10:07:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.455+0000 7fc0c7000640 1 -- 172.21.15.165:0/3615298404 wait complete. 2024-04-25T10:07:37.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:25.839490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:27.018634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:37.480 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:37.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.675+0000 7f2934600640 1 Processor -- start 2024-04-25T10:07:37.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.675+0000 7f2934600640 1 -- start start 2024-04-25T10:07:37.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2934600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f292c15dac0 0x7f292c0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2934600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f292c0d1f30 0x7f292c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2934600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292c0d2420 con 0x7f292c0d34c0 2024-04-25T10:07:37.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2934600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292c0d3030 con 0x7f292c0d1f30 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2934600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f292c0d2700 con 0x7f292c15dac0 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2931200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f292c0d1f30 0x7f292c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2931200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f292c0d1f30 0x7f292c0d03a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55916/0 (socket says 172.21.15.165:55916) 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.677+0000 7f2931200640 1 -- 172.21.15.165:0/3531737085 learned_addr learned my addr 172.21.15.165:0/3531737085 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.678+0000 7f2931c00640 1 --2- 172.21.15.165:0/3531737085 >> v2:172.21.15.172:3300/0 conn(0x7f292c15dac0 0x7f292c0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.678+0000 7f2931200640 1 -- 172.21.15.165:0/3531737085 >> v2:172.21.15.172:3300/0 conn(0x7f292c15dac0 msgr2=0x7f292c0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.678+0000 7f2931200640 1 --2- 172.21.15.165:0/3531737085 >> v2:172.21.15.172:3300/0 conn(0x7f292c15dac0 0x7f292c0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.678+0000 7f2931200640 1 -- 172.21.15.165:0/3531737085 >> v1:172.21.15.172:6789/0 conn(0x7f292c0d34c0 legacy=0x7f292c0b5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.678+0000 7f2931200640 1 -- 172.21.15.165:0/3531737085 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c0d29e0 con 0x7f292c0d1f30 2024-04-25T10:07:37.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.679+0000 7f2931200640 1 --2- 172.21.15.165:0/3531737085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d1f30 0x7f292c0d03a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b6520 tx=0x7f291c02a290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e78331c98e6a405b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.679+0000 7f292be00640 1 -- 172.21.15.165:0/3531737085 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c004030 con 0x7f292c0d1f30 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.679+0000 7f292be00640 1 -- 172.21.15.165:0/3531737085 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c0041d0 con 0x7f292c0d1f30 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f292be00640 1 -- 172.21.15.165:0/3531737085 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c003610 con 0x7f292c0d1f30 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 -- 172.21.15.165:0/3531737085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d1f30 msgr2=0x7f292c0d03a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 --2- 172.21.15.165:0/3531737085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d1f30 0x7f292c0d03a0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f292c0b6520 tx=0x7f291c02a290 comp rx=0 tx=0).stop 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 -- 172.21.15.165:0/3531737085 shutdown_connections 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 --2- 172.21.15.165:0/3531737085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d1f30 0x7f292c0d03a0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 --2- 172.21.15.165:0/3531737085 >> v2:172.21.15.172:3300/0 conn(0x7f292c15dac0 0x7f292c0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 -- 172.21.15.165:0/3531737085 >> 172.21.15.165:0/3531737085 conn(0x7f292c061c30 msgr2=0x7f292c06c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 -- 172.21.15.165:0/3531737085 shutdown_connections 2024-04-25T10:07:37.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.680+0000 7f2934600640 1 -- 172.21.15.165:0/3531737085 wait complete. 2024-04-25T10:07:37.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.683+0000 7f2934600640 1 Processor -- start 2024-04-25T10:07:37.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.683+0000 7f2934600640 1 -- start start 2024-04-25T10:07:37.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2934600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c0d1f30 0x7f292c1e62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2934600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 0x7f292c14c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2934600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f292c15dac0 0x7f292c14cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2934600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c0b97d0 con 0x7f292c0d34c0 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2934600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f292c15dea0 con 0x7f292c0d1f30 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2934600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292c0b9950 con 0x7f292c15dac0 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2931200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 0x7f292c14c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2931200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 0x7f292c14c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55926/0 (socket says 172.21.15.165:55926) 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.684+0000 7f2931200640 1 -- 172.21.15.165:0/947250620 learned_addr learned my addr 172.21.15.165:0/947250620 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931c00640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c0d1f30 0x7f292c1e62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2932600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f292c15dac0 0x7f292c14cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931200640 1 -- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f292c15dac0 msgr2=0x7f292c14cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931200640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f292c15dac0 0x7f292c14cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931200640 1 -- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c0d1f30 msgr2=0x7f292c1e62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931200640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c0d1f30 0x7f292c1e62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931200640 1 -- 172.21.15.165:0/947250620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292c0c8cb0 con 0x7f292c0d34c0 2024-04-25T10:07:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.685+0000 7f2931200640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 0x7f292c14c730 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f291c034040 tx=0x7f291c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.686+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c041030 con 0x7f292c0d34c0 2024-04-25T10:07:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.686+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f291c003c50 con 0x7f292c0d34c0 2024-04-25T10:07:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.686+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f291c004030 con 0x7f292c0d34c0 2024-04-25T10:07:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.686+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f292c0c8f90 con 0x7f292c0d34c0 2024-04-25T10:07:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.686+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f292c0c9550 con 0x7f292c0d34c0 2024-04-25T10:07:37.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.687+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f292c073a00 con 0x7f292c0d34c0 2024-04-25T10:07:37.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.688+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f291c044070 con 0x7f292c0d34c0 2024-04-25T10:07:37.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.689+0000 7f2929600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f291006b4d0 0x7f291006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:37.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.689+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f291c0aae70 con 0x7f292c0d34c0 2024-04-25T10:07:37.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.690+0000 7f2931c00640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f291006b4d0 0x7f291006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:37.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.690+0000 7f2931c00640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f291006b4d0 0x7f291006d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:37.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.694+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f291c079b60 con 0x7f292c0d34c0 2024-04-25T10:07:37.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.887+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f292c06c5c0 con 0x7f292c0d34c0 2024-04-25T10:07:37.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.888+0000 7f2929600640 1 -- 172.21.15.165:0/947250620 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7f291c079460 con 0x7f292c0d34c0 2024-04-25T10:07:37.894 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:07:37.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f291006b4d0 msgr2=0x7f291006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f291006b4d0 0x7f291006d990 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f2920006fd0 tx=0x7f2920008040 comp rx=0 tx=0).stop 2024-04-25T10:07:37.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 msgr2=0x7f292c14c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:37.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 0x7f292c14c730 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f291c034040 tx=0x7f291c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 shutdown_connections 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f291006b4d0 0x7f291006d990 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f292c15dac0 0x7f292c14cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f292c0d34c0 0x7f292c14c730 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 --2- 172.21.15.165:0/947250620 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f292c0d1f30 0x7f292c1e62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 >> 172.21.15.165:0/947250620 conn(0x7f292c061c30 msgr2=0x7f292c0b7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.890+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 shutdown_connections 2024-04-25T10:07:37.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:37.891+0000 7f2934600640 1 -- 172.21.15.165:0/947250620 wait complete. 2024-04-25T10:07:37.915 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:07:38.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.113+0000 7fd7a5e00640 1 Processor -- start 2024-04-25T10:07:38.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.114+0000 7fd7a5e00640 1 -- start start 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.115+0000 7fd7a5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd7a015dd30 0x7fd7a00cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.115+0000 7fd7a5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a006bff0 0x7fd7a00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.115+0000 7fd7a5e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a015e110 con 0x7fd7a00b5970 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.115+0000 7fd7a5e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a006d000 con 0x7fd7a015dd30 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.115+0000 7fd7a5e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00b2fd0 con 0x7fd7a006bff0 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.115+0000 7fd7a4a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd7a015dd30 0x7fd7a00cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a5400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd7a00b5970 0x7fd7a0161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58146/0 (socket says 172.21.15.165:58146) 2024-04-25T10:07:38.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a5400640 1 -- 172.21.15.165:0/3886060065 learned_addr learned my addr 172.21.15.165:0/3886060065 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:38.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a4a00640 1 --2- 172.21.15.165:0/3886060065 >> v2:172.21.15.165:3300/0 conn(0x7fd7a015dd30 0x7fd7a00cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55934/0 (socket says 172.21.15.165:55934) 2024-04-25T10:07:38.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd79fe00640 1 --2- 172.21.15.165:0/3886060065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a006bff0 0x7fd7a00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a4a00640 1 -- 172.21.15.165:0/3886060065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a006bff0 msgr2=0x7fd7a00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a4a00640 1 --2- 172.21.15.165:0/3886060065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a006bff0 0x7fd7a00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a4a00640 1 -- 172.21.15.165:0/3886060065 >> v1:172.21.15.165:6789/0 conn(0x7fd7a00b5970 legacy=0x7fd7a0161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd79f400640 1 -- 172.21.15.165:0/3886060065 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3651507714 0 0) 0x7fd7a015e110 con 0x7fd7a00b5970 2024-04-25T10:07:38.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.116+0000 7fd7a4a00640 1 -- 172.21.15.165:0/3886060065 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a006c480 con 0x7fd7a015dd30 2024-04-25T10:07:38.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.117+0000 7fd7a4a00640 1 --2- 172.21.15.165:0/3886060065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cebd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd7880097a0 tx=0x7fd78802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74717b0bbac67528 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd79f400640 1 -- 172.21.15.165:0/3886060065 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788037070 con 0x7fd7a015dd30 2024-04-25T10:07:38.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd79f400640 1 -- 172.21.15.165:0/3886060065 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd788032440 con 0x7fd7a015dd30 2024-04-25T10:07:38.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd79f400640 1 -- 172.21.15.165:0/3886060065 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd788032810 con 0x7fd7a015dd30 2024-04-25T10:07:38.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd7a5e00640 1 -- 172.21.15.165:0/3886060065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a015dd30 msgr2=0x7fd7a00cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/3886060065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cebd0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fd7880097a0 tx=0x7fd78802a180 comp rx=0 tx=0).stop 2024-04-25T10:07:38.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd7a5e00640 1 -- 172.21.15.165:0/3886060065 shutdown_connections 2024-04-25T10:07:38.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.118+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/3886060065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a006bff0 0x7fd7a00b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.119+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/3886060065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cebd0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.119+0000 7fd7a5e00640 1 -- 172.21.15.165:0/3886060065 >> 172.21.15.165:0/3886060065 conn(0x7fd7a00d0610 msgr2=0x7fd7a00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.119+0000 7fd7a5e00640 1 -- 172.21.15.165:0/3886060065 shutdown_connections 2024-04-25T10:07:38.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.119+0000 7fd7a5e00640 1 -- 172.21.15.165:0/3886060065 wait complete. 2024-04-25T10:07:38.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.120+0000 7fd7a5e00640 1 Processor -- start 2024-04-25T10:07:38.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.120+0000 7fd7a5e00640 1 -- start start 2024-04-25T10:07:38.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.121+0000 7fd7a5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a00b5970 0x7fd7a015b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.121+0000 7fd7a5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.121+0000 7fd7a5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 0x7fd7a00cd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.121+0000 7fd7a5e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a006cd20 con 0x7fd7a00caa80 2024-04-25T10:07:38.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.121+0000 7fd7a5e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a006ca40 con 0x7fd7a015dd30 2024-04-25T10:07:38.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.121+0000 7fd7a5e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00cf110 con 0x7fd7a00b5970 2024-04-25T10:07:38.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 0x7fd7a00cd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 0x7fd7a00cd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55936/0 (socket says 172.21.15.165:55936) 2024-04-25T10:07:38.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 -- 172.21.15.165:0/845875794 learned_addr learned my addr 172.21.15.165:0/845875794 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 -- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a00b5970 msgr2=0x7fd7a015b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a4a00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a00b5970 0x7fd7a015b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd79fe00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a00b5970 0x7fd7a015b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 -- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a015dd30 msgr2=0x7fd7a00cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 -- 172.21.15.165:0/845875794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a00caf10 con 0x7fd7a00caa80 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.122+0000 7fd7a5400640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 0x7fd7a00cd4e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c480 tx=0x7fd79800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.123+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798018070 con 0x7fd7a00caa80 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.123+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd798007500 con 0x7fd7a00caa80 2024-04-25T10:07:38.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.123+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a00cb1f0 con 0x7fd7a00caa80 2024-04-25T10:07:38.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.123+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd798004a20 con 0x7fd7a00caa80 2024-04-25T10:07:38.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.123+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a00cb7b0 con 0x7fd7a00caa80 2024-04-25T10:07:38.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.124+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0073990 con 0x7fd7a00caa80 2024-04-25T10:07:38.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.125+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd79801b070 con 0x7fd7a00caa80 2024-04-25T10:07:38.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.126+0000 7fd79cc00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd77c06b4d0 0x7fd77c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.126+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7980896d0 con 0x7fd7a00caa80 2024-04-25T10:07:38.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.126+0000 7fd7a4a00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd77c06b4d0 0x7fd77c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.127+0000 7fd7a4a00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd77c06b4d0 0x7fd77c06d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd788034000 tx=0x7fd78802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.130+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7980583b0 con 0x7fd7a00caa80 2024-04-25T10:07:38.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.324+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7a00d16c0 con 0x7fd7a00caa80 2024-04-25T10:07:38.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.325+0000 7fd79cc00640 1 -- 172.21.15.165:0/845875794 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd798057cb0 con 0x7fd7a00caa80 2024-04-25T10:07:38.331 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:07:38.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd77c06b4d0 msgr2=0x7fd77c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd77c06b4d0 0x7fd77c06d990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd788034000 tx=0x7fd78802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 msgr2=0x7fd7a00cd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 0x7fd7a00cd4e0 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fd79800c480 tx=0x7fd79800c940 comp rx=0 tx=0).stop 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 shutdown_connections 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd77c06b4d0 0x7fd77c06d990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7a00caa80 0x7fd7a00cd4e0 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7a015dd30 0x7fd7a00cbfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 --2- 172.21.15.165:0/845875794 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7a00b5970 0x7fd7a015b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 >> 172.21.15.165:0/845875794 conn(0x7fd7a00d0610 msgr2=0x7fd7a00b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 shutdown_connections 2024-04-25T10:07:38.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.328+0000 7fd7a5e00640 1 -- 172.21.15.165:0/845875794 wait complete. 2024-04-25T10:07:38.353 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:07:38.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.554+0000 7ff62ca00640 1 Processor -- start 2024-04-25T10:07:38.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.554+0000 7ff62ca00640 1 -- start start 2024-04-25T10:07:38.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff62ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff6280ced80 0x7ff6280d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff62ca00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff62815e000 con 0x7ff62815dc20 2024-04-25T10:07:38.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff62ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6280d1de0 con 0x7ff628161710 2024-04-25T10:07:38.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff62ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff628161c60 con 0x7ff6280ced80 2024-04-25T10:07:38.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff625e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff62815dc20 0x7ff6280ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58160/0 (socket says 172.21.15.165:58160) 2024-04-25T10:07:38.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff625e00640 1 -- 172.21.15.165:0/3700233199 learned_addr learned my addr 172.21.15.165:0/3700233199 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:38.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.556+0000 7ff61fe00640 1 --2- 172.21.15.165:0/3700233199 >> v2:172.21.15.165:3300/0 conn(0x7ff6280ced80 0x7ff6280d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.557+0000 7ff625400640 1 -- 172.21.15.165:0/3700233199 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675796226 0 0) 0x7ff62815e000 con 0x7ff62815dc20 2024-04-25T10:07:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.557+0000 7ff625400640 1 -- 172.21.15.165:0/3700233199 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff60c003520 con 0x7ff62815dc20 2024-04-25T10:07:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.557+0000 7ff61fe00640 1 -- 172.21.15.165:0/3700233199 >> v1:172.21.15.172:6789/0 conn(0x7ff628161710 legacy=0x7ff6280d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.557+0000 7ff61fe00640 1 -- 172.21.15.165:0/3700233199 >> v1:172.21.15.165:6789/0 conn(0x7ff62815dc20 legacy=0x7ff6280ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.557+0000 7ff61fe00640 1 -- 172.21.15.165:0/3700233199 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628161f40 con 0x7ff6280ced80 2024-04-25T10:07:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.558+0000 7ff61fe00640 1 --2- 172.21.15.165:0/3700233199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff6280d22e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff6100097a0 tx=0x7ff61002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcca36f68ccc2c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff625400640 1 -- 172.21.15.165:0/3700233199 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610037070 con 0x7ff6280ced80 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff625400640 1 -- 172.21.15.165:0/3700233199 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff610032440 con 0x7ff6280ced80 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff625400640 1 -- 172.21.15.165:0/3700233199 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff610032810 con 0x7ff6280ced80 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 -- 172.21.15.165:0/3700233199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 msgr2=0x7ff6280d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 --2- 172.21.15.165:0/3700233199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff6280d22e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7ff6100097a0 tx=0x7ff61002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 -- 172.21.15.165:0/3700233199 shutdown_connections 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 --2- 172.21.15.165:0/3700233199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff6280d22e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 -- 172.21.15.165:0/3700233199 >> 172.21.15.165:0/3700233199 conn(0x7ff6280d0420 msgr2=0x7ff6280d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 -- 172.21.15.165:0/3700233199 shutdown_connections 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.559+0000 7ff62ca00640 1 -- 172.21.15.165:0/3700233199 wait complete. 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.561+0000 7ff62ca00640 1 Processor -- start 2024-04-25T10:07:38.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.561+0000 7ff62ca00640 1 -- start start 2024-04-25T10:07:38.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff62ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff62814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff62ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff62815dc20 0x7ff62815e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff62ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff628161710 0x7ff62815e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff62ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b5e50 con 0x7ff6280ced80 2024-04-25T10:07:38.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff62ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff628162500 con 0x7ff62815dc20 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff62ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6280b5fd0 con 0x7ff628161710 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff625e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff62814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff625e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff62814f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55958/0 (socket says 172.21.15.165:55958) 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.562+0000 7ff625e00640 1 -- 172.21.15.165:0/2562106201 learned_addr learned my addr 172.21.15.165:0/2562106201 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff61fe00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff62815dc20 0x7ff62815e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff625e00640 1 -- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff628161710 msgr2=0x7ff62815e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff625e00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff628161710 0x7ff62815e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff625e00640 1 -- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff62815dc20 msgr2=0x7ff62815e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff625e00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff62815dc20 0x7ff62815e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff625e00640 1 -- 172.21.15.165:0/2562106201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff62814fa50 con 0x7ff6280ced80 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff625e00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff62814f510 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff61800c430 tx=0x7ff61800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff618017070 con 0x7ff6280ced80 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff618005ce0 con 0x7ff6280ced80 2024-04-25T10:07:38.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6281e82b0 con 0x7ff6280ced80 2024-04-25T10:07:38.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.563+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6180072f0 con 0x7ff6280ced80 2024-04-25T10:07:38.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.564+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6281e8870 con 0x7ff6280ced80 2024-04-25T10:07:38.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.565+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6280b3650 con 0x7ff6280ced80 2024-04-25T10:07:38.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.566+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff61801a070 con 0x7ff6280ced80 2024-04-25T10:07:38.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.566+0000 7ff61e000640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff60406b4d0 0x7ff60406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.567+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff618089800 con 0x7ff6280ced80 2024-04-25T10:07:38.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.567+0000 7ff61fe00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff60406b4d0 0x7ff60406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.567+0000 7ff61fe00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff60406b4d0 0x7ff60406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff610034000 tx=0x7ff61002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.571+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff6280b3650 con 0x7ff6280ced80 2024-04-25T10:07:38.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.770+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff6280b5b10 con 0x7ff6280ced80 2024-04-25T10:07:38.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.771+0000 7ff61e000640 1 -- 172.21.15.165:0/2562106201 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff6280b5b10 con 0x7ff6280ced80 2024-04-25T10:07:38.777 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:07:38.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff60406b4d0 msgr2=0x7ff60406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff60406b4d0 0x7ff60406d990 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff610034000 tx=0x7ff61002e000 comp rx=0 tx=0).stop 2024-04-25T10:07:38.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 msgr2=0x7ff62814f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff62814f510 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7ff61800c430 tx=0x7ff61800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 shutdown_connections 2024-04-25T10:07:38.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff60406b4d0 0x7ff60406d990 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff628161710 0x7ff62815e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff62815dc20 0x7ff62815e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 --2- 172.21.15.165:0/2562106201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6280ced80 0x7ff62814f510 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.773+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 >> 172.21.15.165:0/2562106201 conn(0x7ff6280d0420 msgr2=0x7ff6280b45b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.774+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 shutdown_connections 2024-04-25T10:07:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.774+0000 7ff62ca00640 1 -- 172.21.15.165:0/2562106201 wait complete. 2024-04-25T10:07:38.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:07:38.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.984+0000 7f79c0200640 1 Processor -- start 2024-04-25T10:07:38.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.985+0000 7f79c0200640 1 -- start start 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79c0200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79b81446d0 0x7f79b80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79c0200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f79b8144d50 0x7f79b80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79c0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b81452a0 0x7f79b80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79c0200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b80b2b20 con 0x7f79b8144d50 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79c0200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b806d400 con 0x7f79b81446d0 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79c0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80b49a0 con 0x7f79b81452a0 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.986+0000 7f79bce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f79b8144d50 0x7f79b80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f79b8144d50 0x7f79b80b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55974/0 (socket says 172.21.15.165:55974) 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 -- 172.21.15.165:0/3856148483 learned_addr learned my addr 172.21.15.165:0/3856148483 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bd800640 1 --2- 172.21.15.165:0/3856148483 >> v2:172.21.15.172:3300/0 conn(0x7f79b81446d0 0x7f79b80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79be200640 1 --2- 172.21.15.165:0/3856148483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b81452a0 0x7f79b80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 -- 172.21.15.165:0/3856148483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b81452a0 msgr2=0x7f79b80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 --2- 172.21.15.165:0/3856148483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b81452a0 0x7f79b80cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 -- 172.21.15.165:0/3856148483 >> v2:172.21.15.172:3300/0 conn(0x7f79b81446d0 msgr2=0x7f79b80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 --2- 172.21.15.165:0/3856148483 >> v2:172.21.15.172:3300/0 conn(0x7f79b81446d0 0x7f79b80b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.987+0000 7f79bce00640 1 -- 172.21.15.165:0/3856148483 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b80cccc0 con 0x7f79b8144d50 2024-04-25T10:07:38.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.988+0000 7f79bce00640 1 --2- 172.21.15.165:0/3856148483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b8144d50 0x7f79b80b4460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f79b80cf560 tx=0x7f79a802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5840c16325af3263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:38.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.988+0000 7f79b7e00640 1 -- 172.21.15.165:0/3856148483 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8037070 con 0x7f79b8144d50 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79b7e00640 1 -- 172.21.15.165:0/3856148483 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a8032440 con 0x7f79b8144d50 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79b7e00640 1 -- 172.21.15.165:0/3856148483 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8032810 con 0x7f79b8144d50 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 -- 172.21.15.165:0/3856148483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b8144d50 msgr2=0x7f79b80b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 --2- 172.21.15.165:0/3856148483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b8144d50 0x7f79b80b4460 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f79b80cf560 tx=0x7f79a802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 -- 172.21.15.165:0/3856148483 shutdown_connections 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 --2- 172.21.15.165:0/3856148483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b81452a0 0x7f79b80cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 --2- 172.21.15.165:0/3856148483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b8144d50 0x7f79b80b4460 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 --2- 172.21.15.165:0/3856148483 >> v2:172.21.15.172:3300/0 conn(0x7f79b81446d0 0x7f79b80b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.989+0000 7f79c0200640 1 -- 172.21.15.165:0/3856148483 >> 172.21.15.165:0/3856148483 conn(0x7f79b806abe0 msgr2=0x7f79b806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.990+0000 7f79c0200640 1 -- 172.21.15.165:0/3856148483 shutdown_connections 2024-04-25T10:07:38.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.990+0000 7f79c0200640 1 -- 172.21.15.165:0/3856148483 wait complete. 2024-04-25T10:07:38.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.992+0000 7f79c0200640 1 Processor -- start 2024-04-25T10:07:38.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.992+0000 7f79c0200640 1 -- start start 2024-04-25T10:07:38.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79c0200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79b81446d0 0x7f79b80c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:38.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79c0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b8144d50 0x7f79b80c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79c0200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 0x7f79b80b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79c0200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80cc700 con 0x7f79b81452a0 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79c0200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80cc420 con 0x7f79b81446d0 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79c0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b80cc9e0 con 0x7f79b8144d50 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79be200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 0x7f79b80b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79be200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 0x7f79b80b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55986/0 (socket says 172.21.15.165:55986) 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.993+0000 7f79be200640 1 -- 172.21.15.165:0/4094569110 learned_addr learned my addr 172.21.15.165:0/4094569110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79bd800640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79b81446d0 0x7f79b80c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79bce00640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b8144d50 0x7f79b80c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79be200640 1 -- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b8144d50 msgr2=0x7f79b80c7310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79be200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b8144d50 0x7f79b80c7310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79be200640 1 -- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79b81446d0 msgr2=0x7f79b80c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79be200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79b81446d0 0x7f79b80c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79be200640 1 -- 172.21.15.165:0/4094569110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b8140720 con 0x7f79b81452a0 2024-04-25T10:07:39.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79be200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 0x7f79b80b1050 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f79b000cda0 tx=0x7f79b000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.994+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0003b90 con 0x7f79b81452a0 2024-04-25T10:07:39.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.995+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79b0003d30 con 0x7f79b81452a0 2024-04-25T10:07:39.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.995+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79b0004240 con 0x7f79b81452a0 2024-04-25T10:07:39.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.995+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b80c7cf0 con 0x7f79b81452a0 2024-04-25T10:07:39.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.995+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b80b0850 con 0x7f79b81452a0 2024-04-25T10:07:39.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.996+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b80b0390 con 0x7f79b81452a0 2024-04-25T10:07:39.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.997+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79b00043e0 con 0x7f79b81452a0 2024-04-25T10:07:39.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.998+0000 7f79b5600640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.998+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79b008ae20 con 0x7f79b81452a0 2024-04-25T10:07:39.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.999+0000 7f79bd800640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:38.999+0000 7f79bd800640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f799c06b500 0x7f799c06d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f79b815bb90 tx=0x7f79ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.003+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79b80b0390 con 0x7f79b81452a0 2024-04-25T10:07:39.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.214+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79b8073a00 con 0x7f79b81452a0 2024-04-25T10:07:39.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.215+0000 7f79b5600640 1 -- 172.21.15.165:0/4094569110 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79b0059b10 con 0x7f79b81452a0 2024-04-25T10:07:39.221 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:07:39.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f799c06b500 msgr2=0x7f799c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f799c06b500 0x7f799c06d9c0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f79b815bb90 tx=0x7f79ac008040 comp rx=0 tx=0).stop 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 msgr2=0x7f79b80b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 0x7f79b80b1050 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f79b000cda0 tx=0x7f79b000a660 comp rx=0 tx=0).stop 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 shutdown_connections 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f799c06b500 0x7f799c06d9c0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79b81452a0 0x7f79b80b1050 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79b8144d50 0x7f79b80c7310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 --2- 172.21.15.165:0/4094569110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79b81446d0 0x7f79b80c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 >> 172.21.15.165:0/4094569110 conn(0x7f79b806abe0 msgr2=0x7f79b80cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 shutdown_connections 2024-04-25T10:07:39.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.218+0000 7f79c0200640 1 -- 172.21.15.165:0/4094569110 wait complete. 2024-04-25T10:07:39.242 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:07:39.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:39.242 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:39.359 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:07:39.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:39.360 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:07:39.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:39.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.581+0000 7f5ceae00640 1 Processor -- start 2024-04-25T10:07:39.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.581+0000 7f5ceae00640 1 -- start start 2024-04-25T10:07:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.582+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce406fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.582+0000 7f5ceae00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ce40700a0 0x7f5ce415d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.582+0000 7f5ceae00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ce41626e0 con 0x7f5ce406abe0 2024-04-25T10:07:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.582+0000 7f5ceae00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ce40cd270 con 0x7f5ce40700a0 2024-04-25T10:07:39.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.582+0000 7f5ceae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce40cd550 con 0x7f5ce40ce330 2024-04-25T10:07:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.583+0000 7f5ce8e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ce40700a0 0x7f5ce415d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.583+0000 7f5ce8e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ce40700a0 0x7f5ce415d370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:56928/0 (socket says 172.21.15.165:56928) 2024-04-25T10:07:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.583+0000 7f5ce8e00640 1 -- 172.21.15.165:0/1922536138 learned_addr learned my addr 172.21.15.165:0/1922536138 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.583+0000 7f5ce3400640 1 --2- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce406fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.584+0000 7f5ce8e00640 1 -- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 msgr2=0x7f5ce406fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.584+0000 7f5ce8e00640 1 --2- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce406fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.584+0000 7f5ce8e00640 1 -- 172.21.15.165:0/1922536138 >> v1:172.21.15.172:6789/0 conn(0x7f5ce406abe0 legacy=0x7f5ce406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.584+0000 7f5ce8e00640 1 -- 172.21.15.165:0/1922536138 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce40cd830 con 0x7f5ce40700a0 2024-04-25T10:07:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.585+0000 7f5ce8e00640 1 --2- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce40700a0 0x7f5ce415d370 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009e50 tx=0x7f5cdc02a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e8b4b2943ba11b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.586+0000 7f5ce2a00640 1 -- 172.21.15.165:0/1922536138 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc036070 con 0x7f5ce40700a0 2024-04-25T10:07:39.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.586+0000 7f5ce2a00640 1 -- 172.21.15.165:0/1922536138 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cdc031600 con 0x7f5ce40700a0 2024-04-25T10:07:39.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.586+0000 7f5ce2a00640 1 -- 172.21.15.165:0/1922536138 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cdc0319d0 con 0x7f5ce40700a0 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 -- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce40700a0 msgr2=0x7f5ce415d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 --2- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce40700a0 0x7f5ce415d370 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc009e50 tx=0x7f5cdc02a680 comp rx=0 tx=0).stop 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 -- 172.21.15.165:0/1922536138 shutdown_connections 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 --2- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce406fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 --2- 172.21.15.165:0/1922536138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce40700a0 0x7f5ce415d370 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 -- 172.21.15.165:0/1922536138 >> 172.21.15.165:0/1922536138 conn(0x7f5ce40b2220 msgr2=0x7f5ce40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 -- 172.21.15.165:0/1922536138 shutdown_connections 2024-04-25T10:07:39.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.587+0000 7f5ceae00640 1 -- 172.21.15.165:0/1922536138 wait complete. 2024-04-25T10:07:39.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.589+0000 7f5ceae00640 1 Processor -- start 2024-04-25T10:07:39.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.589+0000 7f5ceae00640 1 -- start start 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce406abe0 0x7f5ce414ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 0x7f5ce415f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce41609d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ceae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce415d8b0 con 0x7f5ce40700a0 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ceae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce4161780 con 0x7f5ce406abe0 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ceae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ce415da30 con 0x7f5ce40ce330 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.590+0000 7f5ce3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 0x7f5ce415f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 0x7f5ce415f0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56002/0 (socket says 172.21.15.165:56002) 2024-04-25T10:07:39.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 -- 172.21.15.165:0/2873226331 learned_addr learned my addr 172.21.15.165:0/2873226331 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce8e00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce41609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3e00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce406abe0 0x7f5ce414ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 -- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 msgr2=0x7f5ce41609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce41609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 -- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce406abe0 msgr2=0x7f5ce414ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce406abe0 0x7f5ce414ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.591+0000 7f5ce3400640 1 -- 172.21.15.165:0/2873226331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ce415e020 con 0x7f5ce40700a0 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.592+0000 7f5ce3400640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 0x7f5ce415f0c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00e870 tx=0x7f5ccc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.592+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc017070 con 0x7f5ce40700a0 2024-04-25T10:07:39.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.592+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ccc00c940 con 0x7f5ce40700a0 2024-04-25T10:07:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.592+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ccc00cd30 con 0x7f5ce40700a0 2024-04-25T10:07:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.592+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce415e300 con 0x7f5ce40700a0 2024-04-25T10:07:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.592+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ce415e8c0 con 0x7f5ce40700a0 2024-04-25T10:07:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.593+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ce4073990 con 0x7f5ce40700a0 2024-04-25T10:07:39.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.594+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ccc01a070 con 0x7f5ce40700a0 2024-04-25T10:07:39.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.595+0000 7f5cdbe00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5cc006b4d0 0x7f5cc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:39.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.595+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ccc08a580 con 0x7f5ce40700a0 2024-04-25T10:07:39.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.595+0000 7f5ce3e00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5cc006b4d0 0x7f5cc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:39.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.596+0000 7f5ce3e00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5cc006b4d0 0x7f5cc006d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4001640 tx=0x7f5cd4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:39.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.600+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ccc0592a0 con 0x7f5ce40700a0 2024-04-25T10:07:39.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.835+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ce40afe50 con 0x7f5ce40700a0 2024-04-25T10:07:39.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.836+0000 7f5cdbe00640 1 -- 172.21.15.165:0/2873226331 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4945 (secure 0 0 0) 0x7f5ccc058ba0 con 0x7f5ce40700a0 2024-04-25T10:07:39.842 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:39.842 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":40,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4979,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/57109722","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":57109722},{"type":"v1","addr":"172.21.15.172:6834","nonce":57109722}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":4982,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3177295444","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3177295444},{"type":"v1","addr":"172.21.15.172:6835","nonce":3177295444}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":36},{"gid":6621,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/2353868161","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2353868161},{"type":"v1","addr":"172.21.15.165:6836","nonce":2353868161}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":35}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:31.082094+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6624},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6624":{"gid":6624,"name":"c","rank":0,"incarnation":37,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1883921643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1883921643},{"type":"v1","addr":"172.21.15.165:6837","nonce":1883921643}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:39.843 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 40 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5cc006b4d0 msgr2=0x7f5cc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5cc006b4d0 0x7f5cc006d990 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5cd4001640 tx=0x7f5cd4009040 comp rx=0 tx=0).stop 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 msgr2=0x7f5ce415f0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 0x7f5ce415f0c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f5ccc00e870 tx=0x7f5ccc00ed30 comp rx=0 tx=0).stop 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 shutdown_connections 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5cc006b4d0 0x7f5cc006d990 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ce40ce330 0x7f5ce41609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ce40700a0 0x7f5ce415f0c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 --2- 172.21.15.165:0/2873226331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ce406abe0 0x7f5ce414ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:39.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 >> 172.21.15.165:0/2873226331 conn(0x7f5ce40b2220 msgr2=0x7f5ce40ce840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:39.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 shutdown_connections 2024-04-25T10:07:39.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:39.839+0000 7f5ceae00640 1 -- 172.21.15.165:0/2873226331 wait complete. 2024-04-25T10:07:39.864 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:07:40.099 INFO:teuthology.orchestra.run.smithi165.stdout:{"message":"","return_code":0} 2024-04-25T10:07:40.099 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:07:40.100 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:40.100 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:07:40.100 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:40.101 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:40.101 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:07:40.101 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:07:40.119 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:07:40.119 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:40.120 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:07:40.120 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:40.123 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:07:40.123 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:40.128 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:07:40.128 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:40.128 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata rmomapkey 10000000000.00000000 file_head 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.273+0000 7f824ac32f40 1 Processor -- start 2024-04-25T10:07:40.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.273+0000 7f824ac32f40 1 -- start start 2024-04-25T10:07:40.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f824ac32f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5276f0 0x55c45f4de200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f824ac32f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 0x55c45f50ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f824ac32f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f52cbb0 0x55c45f649070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f824ac32f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45f4dd840 con 0x55c45f52c7d0 2024-04-25T10:07:40.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f824ac32f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45f486dd0 con 0x55c45f52cbb0 2024-04-25T10:07:40.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f824ac32f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45f4ddae0 con 0x55c45f5276f0 2024-04-25T10:07:40.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.274+0000 7f8247400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 0x55c45f50ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 0x55c45f50ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56006/0 (socket says 172.21.15.165:56006) 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 -- 172.21.15.165:0/711048497 learned_addr learned my addr 172.21.15.165:0/711048497 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8248800640 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f52cbb0 0x55c45f649070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247e00640 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5276f0 0x55c45f4de200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 -- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5276f0 msgr2=0x55c45f4de200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5276f0 0x55c45f4de200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 -- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f52cbb0 msgr2=0x55c45f649070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f52cbb0 0x55c45f649070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 -- 172.21.15.165:0/711048497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45f510f10 con 0x55c45f52c7d0 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.275+0000 7f8247400640 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 0x55c45f50ddf0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8234001ce0 tx=0x7f823400b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae90538c5086d34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f8246000640 1 -- 172.21.15.165:0/711048497 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400bd80 con 0x55c45f52c7d0 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f8246000640 1 -- 172.21.15.165:0/711048497 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823400f400 con 0x55c45f52c7d0 2024-04-25T10:07:40.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f8246000640 1 -- 172.21.15.165:0/711048497 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823400f830 con 0x55c45f52c7d0 2024-04-25T10:07:40.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 -- 172.21.15.165:0/711048497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 msgr2=0x55c45f50ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 0x55c45f50ddf0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f8234001ce0 tx=0x7f823400b5b0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 -- 172.21.15.165:0/711048497 shutdown_connections 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f52cbb0 0x55c45f649070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52c7d0 0x55c45f50ddf0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 --2- 172.21.15.165:0/711048497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5276f0 0x55c45f4de200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 -- 172.21.15.165:0/711048497 >> 172.21.15.165:0/711048497 conn(0x55c45f50ee60 msgr2=0x55c45f5b9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 -- 172.21.15.165:0/711048497 shutdown_connections 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.276+0000 7f824ac32f40 1 -- 172.21.15.165:0/711048497 wait complete. 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.277+0000 7f824ac32f40 1 Processor -- start 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.277+0000 7f824ac32f40 1 -- start start 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f824ac32f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f5276f0 0x55c45f5abf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f824ac32f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52cbb0 0x55c45f5a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f824ac32f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5ac470 0x55c45f5a4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f824ac32f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45f52a200 con 0x55c45f52cbb0 2024-04-25T10:07:40.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f824ac32f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45f52a450 con 0x55c45f5276f0 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f824ac32f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c45f5b0a00 con 0x55c45f5ac470 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f8247400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52cbb0 0x55c45f5a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f8247400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52cbb0 0x55c45f5a4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56014/0 (socket says 172.21.15.165:56014) 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.278+0000 7f8247400640 1 -- 172.21.15.165:0/2396432613 learned_addr learned my addr 172.21.15.165:0/2396432613 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8248800640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5ac470 0x55c45f5a4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247e00640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f5276f0 0x55c45f5abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247400640 1 -- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5ac470 msgr2=0x55c45f5a4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247400640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c45f5ac470 0x55c45f5a4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247400640 1 -- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f5276f0 msgr2=0x55c45f5abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247400640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c45f5276f0 0x55c45f5abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247400640 1 -- 172.21.15.165:0/2396432613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45f645b00 con 0x55c45f52cbb0 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8247400640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52cbb0 0x55c45f5a4880 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x55c45f50eae0 tx=0x7f8234000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8244c00640 1 -- 172.21.15.165:0/2396432613 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8234020070 con 0x55c45f52cbb0 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8244c00640 1 -- 172.21.15.165:0/2396432613 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823401c440 con 0x55c45f52cbb0 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f8244c00640 1 -- 172.21.15.165:0/2396432613 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823401c830 con 0x55c45f52cbb0 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.279+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c45f4dd3f0 con 0x55c45f52cbb0 2024-04-25T10:07:40.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.280+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45f5295c0 con 0x55c45f52cbb0 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.282+0000 7f8244c00640 1 -- 172.21.15.165:0/2396432613 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8234023070 con 0x55c45f52cbb0 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.282+0000 7f8244c00640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8228069eb0 0x7f822806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.283+0000 7f8244c00640 1 -- 172.21.15.165:0/2396432613 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f823408aa60 con 0x55c45f52cbb0 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.283+0000 7f8247e00640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8228069eb0 0x7f822806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.283+0000 7f824ac32f40 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x55c45f5acf60 0x55c45f5a7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.283+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 --> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] -- osd_op(unknown.0.0:1 9.10 9:0cf6ad0f:::10000000000.00000000:head [omap-rm-keys in=17b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e68) v9 -- 0x55c45f645fe0 con 0x55c45f5acf60 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.284+0000 7f8247e00640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8228069eb0 0x7f822806c370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8238056000 tx=0x7f8238007030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.284+0000 7f8248800640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x55c45f5acf60 0x55c45f5a7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.285+0000 7f8248800640 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x55c45f5acf60 0x55c45f5a7250 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f8248800640 1 -- 172.21.15.165:0/2396432613 <== osd.4 v2:172.21.15.172:6817/928908638 1 ==== osd_op_reply(1 10000000000.00000000 [omap-rm-keys] v68'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f8240009720 con 0x55c45f5acf60 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x55c45f5acf60 msgr2=0x55c45f5a7250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x55c45f5acf60 0x55c45f5a7250 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8228069eb0 msgr2=0x7f822806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8228069eb0 0x7f822806c370 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8238056000 tx=0x7f8238007030 comp rx=0 tx=0).stop 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52cbb0 msgr2=0x55c45f5a4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 --2- 172.21.15.165:0/2396432613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c45f52cbb0 0x55c45f5a4880 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x55c45f50eae0 tx=0x7f8234000bc0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f8248800640 1 -- 172.21.15.165:0/2396432613 reap_dead start 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 shutdown_connections 2024-04-25T10:07:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 >> 172.21.15.165:0/2396432613 conn(0x55c45f50ee60 msgr2=0x55c45f5a95d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 shutdown_connections 2024-04-25T10:07:40.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.289+0000 7f824ac32f40 1 -- 172.21.15.165:0/2396432613 wait complete. 2024-04-25T10:07:40.301 ERROR:tasks.ceph.mds.a:tried to stop a non-running daemon 2024-04-25T10:07:40.301 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail a 2024-04-25T10:07:40.302 ERROR:tasks.ceph.mds.c:tried to stop a non-running daemon 2024-04-25T10:07:40.302 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail c 2024-04-25T10:07:40.302 ERROR:tasks.ceph.mds.b:tried to stop a non-running daemon 2024-04-25T10:07:40.303 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail b 2024-04-25T10:07:40.303 ERROR:tasks.ceph.mds.d:tried to stop a non-running daemon 2024-04-25T10:07:40.303 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mds fail d 2024-04-25T10:07:40.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.480+0000 7fe07ce00640 1 Processor -- start 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.480+0000 7f6233e00640 1 Processor -- start 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.480+0000 7f6233e00640 1 -- start start 2024-04-25T10:07:40.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.480+0000 7fe07ce00640 1 -- start start 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.480+0000 7ff564400640 1 Processor -- start 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.480+0000 7ff564400640 1 -- start start 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7f6233e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f623415dbd0 0x7f6234161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7f6233e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62340b95c0 0x7f62340b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7fe07ce00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe07815dd30 0x7fe0780cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7f6233e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62340b5910 0x7f623406bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7f6233e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62340d1ce0 con 0x7f623415dbd0 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7f6233e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6234161e70 con 0x7f62340b95c0 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7f6233e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6234162150 con 0x7f62340b5910 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7f6232a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f623415dbd0 0x7f6234161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.481+0000 7fe07ce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe07806bff0 0x7fe0780b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7fe07ce00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe07815e110 con 0x7fe0780b5970 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7fe07ce00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe07806d000 con 0x7fe07806bff0 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7fe07ce00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0780b2fd0 con 0x7fe07815dd30 2024-04-25T10:07:40.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7ff564400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff55c0ced80 0x7ff55c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7f6232a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f623415dbd0 0x7f6234161620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56028/0 (socket says 172.21.15.165:56028) 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7f6232a00640 1 -- 172.21.15.165:0/2982262674 learned_addr learned my addr 172.21.15.165:0/2982262674 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7ff564400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55c15e000 con 0x7ff55c161710 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7ff564400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55c0d1de0 con 0x7ff55c15dc20 2024-04-25T10:07:40.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7ff564400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55c161c60 con 0x7ff55c0ced80 2024-04-25T10:07:40.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7fe076a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe07806bff0 0x7fe0780b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7f6232000640 1 --2- 172.21.15.165:0/2982262674 >> v2:172.21.15.172:3300/0 conn(0x7f62340b95c0 0x7f62340b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7f6233400640 1 --2- 172.21.15.165:0/2982262674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62340b5910 0x7f623406bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7fe076a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe07806bff0 0x7fe0780b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56040/0 (socket says 172.21.15.165:56040) 2024-04-25T10:07:40.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7f6232a00640 1 -- 172.21.15.165:0/2982262674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62340b5910 msgr2=0x7f623406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7f6232a00640 1 --2- 172.21.15.165:0/2982262674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62340b5910 0x7f623406bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.482+0000 7fe076a00640 1 -- 172.21.15.165:0/4259797942 learned_addr learned my addr 172.21.15.165:0/4259797942 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7f6232a00640 1 -- 172.21.15.165:0/2982262674 >> v2:172.21.15.172:3300/0 conn(0x7f62340b95c0 msgr2=0x7f62340b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7f6232a00640 1 --2- 172.21.15.165:0/2982262674 >> v2:172.21.15.172:3300/0 conn(0x7f62340b95c0 0x7f62340b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7f6232a00640 1 -- 172.21.15.165:0/2982262674 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6234162430 con 0x7f623415dbd0 2024-04-25T10:07:40.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7ff561000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff55c0ced80 0x7ff55c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7ff562400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff55c161710 0x7ff55c0d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58174/0 (socket says 172.21.15.165:58174) 2024-04-25T10:07:40.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7ff561000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff55c0ced80 0x7ff55c0d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:57004/0 (socket says 172.21.15.165:57004) 2024-04-25T10:07:40.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7ff562400640 1 -- 172.21.15.165:0/3013574517 learned_addr learned my addr 172.21.15.165:0/3013574517 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:40.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7fe077400640 1 --2- 172.21.15.165:0/4259797942 >> v2:172.21.15.172:3300/0 conn(0x7fe07815dd30 0x7fe0780cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7fe076a00640 1 -- 172.21.15.165:0/4259797942 >> v2:172.21.15.172:3300/0 conn(0x7fe07815dd30 msgr2=0x7fe0780cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7fe076a00640 1 --2- 172.21.15.165:0/4259797942 >> v2:172.21.15.172:3300/0 conn(0x7fe07815dd30 0x7fe0780cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7fe076a00640 1 -- 172.21.15.165:0/4259797942 >> v1:172.21.15.172:6789/0 conn(0x7fe0780b5970 legacy=0x7fe078161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7fe076a00640 1 -- 172.21.15.165:0/4259797942 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07806c480 con 0x7fe07806bff0 2024-04-25T10:07:40.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7ff55be00640 1 -- 172.21.15.165:0/3013574517 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3559576740 0 0) 0x7ff55c15e000 con 0x7ff55c161710 2024-04-25T10:07:40.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.483+0000 7ff55be00640 1 -- 172.21.15.165:0/3013574517 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff544003520 con 0x7ff55c161710 2024-04-25T10:07:40.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.484+0000 7ff561000640 1 -- 172.21.15.165:0/3013574517 >> v1:172.21.15.172:6789/0 conn(0x7ff55c15dc20 legacy=0x7ff55c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.484+0000 7ff561000640 1 -- 172.21.15.165:0/3013574517 >> v1:172.21.15.165:6789/0 conn(0x7ff55c161710 legacy=0x7ff55c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.484+0000 7ff561000640 1 -- 172.21.15.165:0/3013574517 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c161f40 con 0x7ff55c0ced80 2024-04-25T10:07:40.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.484+0000 7f6232a00640 1 --2- 172.21.15.165:0/2982262674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623415dbd0 0x7f6234161620 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6234161580 tx=0x7f622402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72e2e4d5d0b165f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7fe076a00640 1 --2- 172.21.15.165:0/4259797942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07806bff0 0x7fe0780b5430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe07808bfe0 tx=0x7fe06402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16a75ba630478de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7f6231600640 1 -- 172.21.15.165:0/2982262674 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224037070 con 0x7f623415dbd0 2024-04-25T10:07:40.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7f6231600640 1 -- 172.21.15.165:0/2982262674 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6224032440 con 0x7f623415dbd0 2024-04-25T10:07:40.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7f6231600640 1 -- 172.21.15.165:0/2982262674 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6224032810 con 0x7f623415dbd0 2024-04-25T10:07:40.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7fe076000640 1 -- 172.21.15.165:0/4259797942 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064037070 con 0x7fe07806bff0 2024-04-25T10:07:40.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7fe076000640 1 -- 172.21.15.165:0/4259797942 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe064032440 con 0x7fe07806bff0 2024-04-25T10:07:40.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7fe076000640 1 -- 172.21.15.165:0/4259797942 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe064032810 con 0x7fe07806bff0 2024-04-25T10:07:40.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.485+0000 7ff561000640 1 --2- 172.21.15.165:0/3013574517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c0ced80 0x7ff55c0d22e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5480097a0 tx=0x7ff54802a230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3117248b8864a87a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7ff55be00640 1 -- 172.21.15.165:0/3013574517 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548037070 con 0x7ff55c0ced80 2024-04-25T10:07:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7ff55be00640 1 -- 172.21.15.165:0/3013574517 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548032440 con 0x7ff55c0ced80 2024-04-25T10:07:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7ff55be00640 1 -- 172.21.15.165:0/3013574517 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548032810 con 0x7ff55c0ced80 2024-04-25T10:07:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 -- 172.21.15.165:0/2982262674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623415dbd0 msgr2=0x7f6234161620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 --2- 172.21.15.165:0/2982262674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623415dbd0 0x7f6234161620 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6234161580 tx=0x7f622402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 -- 172.21.15.165:0/4259797942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07806bff0 msgr2=0x7fe0780b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 --2- 172.21.15.165:0/4259797942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07806bff0 0x7fe0780b5430 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fe07808bfe0 tx=0x7fe06402a230 comp rx=0 tx=0).stop 2024-04-25T10:07:40.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 -- 172.21.15.165:0/2982262674 shutdown_connections 2024-04-25T10:07:40.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 --2- 172.21.15.165:0/2982262674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62340b5910 0x7f623406bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 --2- 172.21.15.165:0/2982262674 >> v2:172.21.15.172:3300/0 conn(0x7f62340b95c0 0x7f62340b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 --2- 172.21.15.165:0/2982262674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f623415dbd0 0x7f6234161620 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 -- 172.21.15.165:0/2982262674 >> 172.21.15.165:0/2982262674 conn(0x7f62340d03d0 msgr2=0x7f62340d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 -- 172.21.15.165:0/4259797942 shutdown_connections 2024-04-25T10:07:40.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 --2- 172.21.15.165:0/4259797942 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe07806bff0 0x7fe0780b5430 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 --2- 172.21.15.165:0/4259797942 >> v2:172.21.15.172:3300/0 conn(0x7fe07815dd30 0x7fe0780cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 -- 172.21.15.165:0/4259797942 >> 172.21.15.165:0/4259797942 conn(0x7fe0780d0610 msgr2=0x7fe0780d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7f6233e00640 1 -- 172.21.15.165:0/2982262674 shutdown_connections 2024-04-25T10:07:40.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.486+0000 7fe07ce00640 1 -- 172.21.15.165:0/4259797942 shutdown_connections 2024-04-25T10:07:40.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7f6233e00640 1 -- 172.21.15.165:0/2982262674 wait complete. 2024-04-25T10:07:40.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 -- 172.21.15.165:0/3013574517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c0ced80 msgr2=0x7ff55c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 --2- 172.21.15.165:0/3013574517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c0ced80 0x7ff55c0d22e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff5480097a0 tx=0x7ff54802a230 comp rx=0 tx=0).stop 2024-04-25T10:07:40.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7fe07ce00640 1 -- 172.21.15.165:0/4259797942 wait complete. 2024-04-25T10:07:40.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 -- 172.21.15.165:0/3013574517 shutdown_connections 2024-04-25T10:07:40.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 --2- 172.21.15.165:0/3013574517 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c0ced80 0x7ff55c0d22e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 -- 172.21.15.165:0/3013574517 >> 172.21.15.165:0/3013574517 conn(0x7ff55c0d0420 msgr2=0x7ff55c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 -- 172.21.15.165:0/3013574517 shutdown_connections 2024-04-25T10:07:40.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.487+0000 7ff564400640 1 -- 172.21.15.165:0/3013574517 wait complete. 2024-04-25T10:07:40.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.488+0000 7f9e0fe00640 1 Processor -- start 2024-04-25T10:07:40.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.488+0000 7f9e0fe00640 1 -- start start 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.488+0000 7f6233e00640 1 Processor -- start 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.489+0000 7f6233e00640 1 -- start start 2024-04-25T10:07:40.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.489+0000 7fe07ce00640 1 Processor -- start 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6233e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62340b5910 0x7f62341490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6233e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 0x7f62341495e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.489+0000 7f9e0fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e080b38d0 0x7f9e080b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9e0806bd20 0x7f9e0806c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0815e020 con 0x7f9e0806e360 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e0806b890 con 0x7f9e0806bd20 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0806ac80 con 0x7f9e080b38d0 2024-04-25T10:07:40.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6233e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f623415dbd0 0x7f6234159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6233e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623406c4b0 con 0x7f62340b95c0 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6233e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6234161b60 con 0x7f62340b5910 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6233e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623406c630 con 0x7f623415dbd0 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7ff564400640 1 Processor -- start 2024-04-25T10:07:40.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7fe07ce00640 1 -- start start 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6232000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 0x7f62341495e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6232000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 0x7f62341495e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56052/0 (socket says 172.21.15.165:56052) 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f6232000640 1 -- 172.21.15.165:0/840042736 learned_addr learned my addr 172.21.15.165:0/840042736 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0d400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9e0806e360 0x7f9e0806e740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58182/0 (socket says 172.21.15.165:58182) 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0d400640 1 -- 172.21.15.165:0/2856391556 learned_addr learned my addr 172.21.15.165:0/2856391556 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7f9e0de00640 1 --2- 172.21.15.165:0/2856391556 >> v2:172.21.15.172:3300/0 conn(0x7f9e0806bd20 0x7f9e0806c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.490+0000 7ff564400640 1 -- start start 2024-04-25T10:07:40.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e0ca00640 1 --2- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e080b38d0 0x7f9e080b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6233400640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f623415dbd0 0x7f6234159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6232000640 1 -- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f623415dbd0 msgr2=0x7f6234159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6232000640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f623415dbd0 0x7f6234159cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6232000640 1 -- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62340b5910 msgr2=0x7f62341490a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6232000640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62340b5910 0x7f62341490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6232000640 1 -- 172.21.15.165:0/840042736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623415aa50 con 0x7f62340b95c0 2024-04-25T10:07:40.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e07e00640 1 -- 172.21.15.165:0/2856391556 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467566765 0 0) 0x7f9e0815e020 con 0x7f9e0806e360 2024-04-25T10:07:40.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e07e00640 1 -- 172.21.15.165:0/2856391556 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df0003520 con 0x7f9e0806e360 2024-04-25T10:07:40.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f6232000640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 0x7f62341495e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f622000d800 tx=0x7f622000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0780b5970 0x7fe07814d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07815dd30 0x7fe0780b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e0de00640 1 -- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e080b38d0 msgr2=0x7f9e080b3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e0de00640 1 --2- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e080b38d0 0x7f9e080b3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e0de00640 1 -- 172.21.15.165:0/2856391556 >> v1:172.21.15.165:6789/0 conn(0x7f9e0806e360 legacy=0x7f9e0806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f9e0de00640 1 -- 172.21.15.165:0/2856391556 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0806af60 con 0x7f9e0806bd20 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7fe07ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 0x7fe0780aa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7fe07ce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07806cd20 con 0x7fe0780aa2b0 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7fe07ce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07806ca40 con 0x7fe07815dd30 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7fe07ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0780cf110 con 0x7fe0780b5970 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.491+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000b6c0 con 0x7f62340b95c0 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6220019070 con 0x7f62340b95c0 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622000ba80 con 0x7f62340b95c0 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623415ad30 con 0x7f62340b95c0 2024-04-25T10:07:40.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623415b2f0 con 0x7f62340b95c0 2024-04-25T10:07:40.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7ff564400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 0x7ff55c0c9b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7fe077e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 0x7fe0780aa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7fe077e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 0x7fe0780aa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56054/0 (socket says 172.21.15.165:56054) 2024-04-25T10:07:40.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.492+0000 7fe077e00640 1 -- 172.21.15.165:0/2922648607 learned_addr learned my addr 172.21.15.165:0/2922648607 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7fe076a00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07815dd30 0x7fe0780b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff564400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff55c15dc20 0x7ff55c1604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff564400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c161710 0x7ff55c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff564400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c0b5e50 con 0x7ff55c0ced80 2024-04-25T10:07:40.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff564400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c162500 con 0x7ff55c161710 2024-04-25T10:07:40.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff564400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55c0b5fd0 con 0x7ff55c15dc20 2024-04-25T10:07:40.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7f9e0de00640 1 --2- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e0806bd20 0x7f9e0806c1b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009d90 tx=0x7f9e0002a6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b902e277f4bba9a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff561a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 0x7ff55c0c9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff561a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 0x7ff55c0c9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56056/0 (socket says 172.21.15.165:56056) 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff561a00640 1 -- 172.21.15.165:0/3312379300 learned_addr learned my addr 172.21.15.165:0/3312379300 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7ff561000640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff55c15dc20 0x7ff55c1604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7fe077e00640 1 -- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0780b5970 msgr2=0x7fe07814d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7fe077e00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0780b5970 0x7fe07814d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7fe077e00640 1 -- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07815dd30 msgr2=0x7fe0780b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7fe077e00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07815dd30 0x7fe0780b1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.493+0000 7fe077e00640 1 -- 172.21.15.165:0/2922648607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0780cc080 con 0x7fe0780aa2b0 2024-04-25T10:07:40.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff562400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c161710 0x7ff55c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6234157c30 con 0x7f62340b95c0 2024-04-25T10:07:40.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff561a00640 1 -- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff55c15dc20 msgr2=0x7ff55c1604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff561a00640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff55c15dc20 0x7ff55c1604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff561a00640 1 -- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c161710 msgr2=0x7ff55c156f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff561a00640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c161710 0x7ff55c156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff561a00640 1 -- 172.21.15.165:0/3312379300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55c15f400 con 0x7ff55c0ced80 2024-04-25T10:07:40.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7f9e07e00640 1 -- 172.21.15.165:0/2856391556 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00007740 con 0x7f9e0806bd20 2024-04-25T10:07:40.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7fe077e00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 0x7fe0780aa760 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe07000c780 tx=0x7fe07000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7ff561a00640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 0x7ff55c0c9b40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c400 tx=0x7ff55000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7f9e07e00640 1 -- 172.21.15.165:0/2856391556 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e000078e0 con 0x7f9e0806bd20 2024-04-25T10:07:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.494+0000 7f9e07e00640 1 -- 172.21.15.165:0/2856391556 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e00007c90 con 0x7f9e0806bd20 2024-04-25T10:07:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe070017070 con 0x7fe0780aa2b0 2024-04-25T10:07:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe070003da0 con 0x7fe0780aa2b0 2024-04-25T10:07:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0700052b0 con 0x7fe0780aa2b0 2024-04-25T10:07:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5500052d0 con 0x7ff55c0ced80 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff550005490 con 0x7ff55c0ced80 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff550007260 con 0x7ff55c0ced80 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f9e0fe00640 1 -- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e0806bd20 msgr2=0x7f9e0806c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e0806bd20 0x7f9e0806c1b0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f9e00009d90 tx=0x7f9e0002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0780cc360 con 0x7fe0780aa2b0 2024-04-25T10:07:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0780cc920 con 0x7fe0780aa2b0 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff55c15f6e0 con 0x7ff55c0ced80 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff55c15fca0 con 0x7ff55c0ced80 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f9e0fe00640 1 -- 172.21.15.165:0/2856391556 shutdown_connections 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e0806bd20 0x7f9e0806c1b0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/2856391556 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e080b38d0 0x7f9e080b3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f9e0fe00640 1 -- 172.21.15.165:0/2856391556 >> 172.21.15.165:0/2856391556 conn(0x7f9e080cfec0 msgr2=0x7f9e080d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.496+0000 7f9e0fe00640 1 -- 172.21.15.165:0/2856391556 shutdown_connections 2024-04-25T10:07:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.496+0000 7f9e0fe00640 1 -- 172.21.15.165:0/2856391556 wait complete. 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.495+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f622000bc20 con 0x7f62340b95c0 2024-04-25T10:07:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.496+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe078073990 con 0x7fe0780aa2b0 2024-04-25T10:07:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.497+0000 7f622aa00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f621406b4d0 0x7f621406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.497+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f622008a8e0 con 0x7f62340b95c0 2024-04-25T10:07:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.497+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff55c0d16b0 con 0x7ff55c0ced80 2024-04-25T10:07:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.497+0000 7f6232a00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f621406b4d0 0x7f621406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.498+0000 7f9e0fe00640 1 Processor -- start 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.498+0000 7f9e0fe00640 1 -- start start 2024-04-25T10:07:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.498+0000 7f6232a00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f621406b4d0 0x7f621406d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 0x7f9e081f8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e0806e360 0x7f9e08155f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e080b38d0 0x7f9e08153000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0806ee50 con 0x7f9e0806bd20 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0806b520 con 0x7f9e080b38d0 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080b2fd0 con 0x7f9e0806e360 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 0x7f9e081f8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 0x7f9e081f8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56058/0 (socket says 172.21.15.165:56058) 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.499+0000 7f9e0d400640 1 -- 172.21.15.165:0/1134293986 learned_addr learned my addr 172.21.15.165:0/1134293986 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:40.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0d400640 1 -- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e0806e360 msgr2=0x7f9e08155f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0ca00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e0806e360 0x7f9e08155f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0d400640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e0806e360 0x7f9e08155f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0d400640 1 -- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e080b38d0 msgr2=0x7f9e08153000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0d400640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e080b38d0 0x7f9e08153000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0d400640 1 -- 172.21.15.165:0/1134293986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08154230 con 0x7f9e0806bd20 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.500+0000 7f9e0d400640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 0x7f9e081f8300 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00c550 tx=0x7f9dfc00ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.501+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6220059530 con 0x7f62340b95c0 2024-04-25T10:07:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.501+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc018070 con 0x7f9e0806bd20 2024-04-25T10:07:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.502+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e08154510 con 0x7f9e0806bd20 2024-04-25T10:07:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.502+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e08154ad0 con 0x7f9e0806bd20 2024-04-25T10:07:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.502+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc0074e0 con 0x7f9e0806bd20 2024-04-25T10:07:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.502+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc004a20 con 0x7f9e0806bd20 2024-04-25T10:07:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.503+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9de4005290 con 0x7f9e0806bd20 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.504+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff550007400 con 0x7ff55c0ced80 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.505+0000 7ff559600640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff53c06b550 0x7ff53c06da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.506+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff550054a20 con 0x7ff55c0ced80 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.504+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe07001a070 con 0x7fe0780aa2b0 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.506+0000 7fe06f400640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05806b5a0 0x7fe05806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.506+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe070089de0 con 0x7fe0780aa2b0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.506+0000 7fe077400640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05806b5a0 0x7fe05806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.506+0000 7ff561000640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff53c06b550 0x7ff53c06da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.507+0000 7fe077400640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05806b5a0 0x7fe05806da60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe068006fd0 tx=0x7fe068008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.507+0000 7ff561000640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff53c06b550 0x7ff53c06da10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff548034000 tx=0x7ff54802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.510+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff550058ab0 con 0x7ff55c0ced80 2024-04-25T10:07:40.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.521+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe070021020 con 0x7fe0780aa2b0 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.521+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9dfc01b070 con 0x7f9e0806bd20 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.522+0000 7f9e05600640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9de806b4d0 0x7f9de806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.522+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9dfc089aa0 con 0x7f9e0806bd20 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.522+0000 7f9e0ca00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9de806b4d0 0x7f9de806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.523+0000 7f9e0ca00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9de806b4d0 0x7f9de806d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9e0815bd80 tx=0x7f9df4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.529+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9dfc058800 con 0x7f9e0806bd20 2024-04-25T10:07:40.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.779+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "a"} v 0) v1 -- 0x7f623406c0a0 con 0x7f62340b95c0 2024-04-25T10:07:40.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.782+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "b"} v 0) v1 -- 0x7ff55c0d3390 con 0x7ff55c0ced80 2024-04-25T10:07:40.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.800+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "c"} v 0) v1 -- 0x7fe0780d16c0 con 0x7fe0780aa2b0 2024-04-25T10:07:40.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:40.814+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "mds fail", "role_or_gid": "d"} v 0) v1 -- 0x7f9de4005850 con 0x7f9e0806bd20 2024-04-25T10:07:41.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.050+0000 7f622aa00640 1 -- 172.21.15.165:0/840042736 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "a"}]=0 failed mds gid 6621 v41) v1 ==== 95+0+0 (secure 0 0 0) 0x7f6220058e30 con 0x7f62340b95c0 2024-04-25T10:07:41.056 INFO:teuthology.orchestra.run.smithi165.stderr:failed mds gid 6621 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.053+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f621406b4d0 msgr2=0x7f621406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.053+0000 7f6233e00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f621406b4d0 0x7f621406d990 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402c000 comp rx=0 tx=0).stop 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.053+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 msgr2=0x7f62341495e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.053+0000 7f6233e00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 0x7f62341495e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f622000d800 tx=0x7f622000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 shutdown_connections 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f621406b4d0 0x7f621406d990 secure :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6224034000 tx=0x7f622402c000 comp rx=0 tx=0).stop 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f623415dbd0 0x7f6234159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62340b95c0 0x7f62341495e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 --2- 172.21.15.165:0/840042736 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62340b5910 0x7f62341490a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 >> 172.21.15.165:0/840042736 conn(0x7f62340d03d0 msgr2=0x7f62340d07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 shutdown_connections 2024-04-25T10:07:41.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.054+0000 7f6233e00640 1 -- 172.21.15.165:0/840042736 wait complete. 2024-04-25T10:07:41.079 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:07:41.073+0000 7f547e800640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:07:41.083 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:07:41.083 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:07:41.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.079+0000 7ff559600640 1 -- 172.21.15.165:0/3312379300 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "b"}]=0 failed mds gid 4979 v42) v1 ==== 95+0+0 (secure 0 0 0) 0x7ff5500583b0 con 0x7ff55c0ced80 2024-04-25T10:07:41.089 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:07:41.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.079+0000 7fe06f400640 1 -- 172.21.15.165:0/2922648607 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "c"}]=0 failed mds gid 6624 v42) v1 ==== 95+0+0 (secure 0 0 0) 0x7fe0700584f0 con 0x7fe0780aa2b0 2024-04-25T10:07:41.089 INFO:teuthology.orchestra.run.smithi165.stderr:failed mds gid 6624 2024-04-25T10:07:41.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05806b5a0 msgr2=0x7fe05806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05806b5a0 0x7fe05806da60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe068006fd0 tx=0x7fe068008040 comp rx=0 tx=0).stop 2024-04-25T10:07:41.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 msgr2=0x7fe0780aa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 0x7fe0780aa760 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fe07000c780 tx=0x7fe07000cc40 comp rx=0 tx=0).stop 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 shutdown_connections 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe05806b5a0 0x7fe05806da60 secure :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe068006fd0 tx=0x7fe068008040 comp rx=0 tx=0).stop 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0780aa2b0 0x7fe0780aa760 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe07815dd30 0x7fe0780b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 --2- 172.21.15.165:0/2922648607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0780b5970 0x7fe07814d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 >> 172.21.15.165:0/2922648607 conn(0x7fe0780d0610 msgr2=0x7fe0780d1fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.083+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 shutdown_connections 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.083+0000 7fe07ce00640 1 -- 172.21.15.165:0/2922648607 wait complete. 2024-04-25T10:07:41.090 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:41.090 INFO:teuthology.orchestra.run.smithi165.stderr:failed mds gid 4979 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff53c06b550 msgr2=0x7ff53c06da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff53c06b550 0x7ff53c06da10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff548034000 tx=0x7ff54802e000 comp rx=0 tx=0).stop 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 msgr2=0x7ff55c0c9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 0x7ff55c0c9b40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7ff55000c400 tx=0x7ff55000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 shutdown_connections 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff53c06b550 0x7ff53c06da10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff55c161710 0x7ff55c156f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff55c15dc20 0x7ff55c1604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 --2- 172.21.15.165:0/3312379300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff55c0ced80 0x7ff55c0c9b40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.082+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 >> 172.21.15.165:0/3312379300 conn(0x7ff55c0d0420 msgr2=0x7ff55c0b46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.083+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 shutdown_connections 2024-04-25T10:07:41.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:41.083+0000 7ff564400640 1 -- 172.21.15.165:0/3312379300 wait complete. 2024-04-25T10:07:41.116 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:07:41.116 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:07:41.117 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:07:41.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:07:41.124 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:07:41.125 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:41.165 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:07:41.165 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:41.228 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:07:41.282 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:07:41.304 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:07:42.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.083+0000 7f9e05600640 1 -- 172.21.15.165:0/1134293986 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "mds fail", "role_or_gid": "d"}]=0 failed mds gid 4982 v43) v1 ==== 95+0+0 (secure 0 0 0) 0x7f9dfc058100 con 0x7f9e0806bd20 2024-04-25T10:07:42.089 INFO:teuthology.orchestra.run.smithi165.stderr:failed mds gid 4982 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9de806b4d0 msgr2=0x7f9de806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9de806b4d0 0x7f9de806d990 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f9e0815bd80 tx=0x7f9df4008040 comp rx=0 tx=0).stop 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 msgr2=0x7f9e081f8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 0x7f9e081f8300 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc00c550 tx=0x7f9dfc00ca10 comp rx=0 tx=0).stop 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 shutdown_connections 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9de806b4d0 0x7f9de806d990 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e080b38d0 0x7f9e08153000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e0806e360 0x7f9e08155f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 --2- 172.21.15.165:0/1134293986 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e0806bd20 0x7f9e081f8300 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 >> 172.21.15.165:0/1134293986 conn(0x7f9e080cfec0 msgr2=0x7f9e080acf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 shutdown_connections 2024-04-25T10:07:42.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.087+0000 7f9e0fe00640 1 -- 172.21.15.165:0/1134293986 wait complete. 2024-04-25T10:07:42.117 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:07:42.117 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:07:42.125 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:07:42.125 DEBUG:teuthology.parallel:result is None 2024-04-25T10:07:42.125 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:42.291 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:07:42.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.338+0000 7f233ce00640 1 Processor -- start 2024-04-25T10:07:42.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.339+0000 7f233ce00640 1 -- start start 2024-04-25T10:07:42.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.340+0000 7f233ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.340+0000 7f233ce00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 0x7f23380cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.340+0000 7f233ce00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f233815bb50 con 0x7f23380cc840 2024-04-25T10:07:42.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.340+0000 7f233ce00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2338144240 con 0x7f233806e6a0 2024-04-25T10:07:42.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.340+0000 7f233ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f233815e220 con 0x7f2338143e60 2024-04-25T10:07:42.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.341+0000 7f2337400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.341+0000 7f232ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 0x7f23380cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.341+0000 7f2337400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:35512/0 (socket says 172.21.15.165:35512) 2024-04-25T10:07:42.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.341+0000 7f232ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 0x7f23380cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:57110/0 (socket says 172.21.15.165:57110) 2024-04-25T10:07:42.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.341+0000 7f2337400640 1 -- 172.21.15.165:0/3290889696 learned_addr learned my addr 172.21.15.165:0/3290889696 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:42.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.342+0000 7f2337400640 1 -- 172.21.15.165:0/3290889696 >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 msgr2=0x7f23380cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:42.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.342+0000 7f2337400640 1 --2- 172.21.15.165:0/3290889696 >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 0x7f23380cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.342+0000 7f2337400640 1 -- 172.21.15.165:0/3290889696 >> v1:172.21.15.172:6789/0 conn(0x7f23380cc840 legacy=0x7f2338070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.342+0000 7f2337400640 1 -- 172.21.15.165:0/3290889696 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f233815e500 con 0x7f2338143e60 2024-04-25T10:07:42.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.342+0000 7f232ea00640 1 --2- 172.21.15.165:0/3290889696 >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 0x7f23380cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:42.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.342+0000 7f2336a00640 1 -- 172.21.15.165:0/3290889696 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 390235871 0 0) 0x7f2330001490 con 0x7f23380cc840 2024-04-25T10:07:42.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.343+0000 7f2337400640 1 --2- 172.21.15.165:0/3290889696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233806e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f232800bb10 tx=0x7f232802c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47058758dbdcb887 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.344+0000 7f2336a00640 1 -- 172.21.15.165:0/3290889696 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328036020 con 0x7f2338143e60 2024-04-25T10:07:42.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.344+0000 7f2336a00640 1 -- 172.21.15.165:0/3290889696 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2328032820 con 0x7f2338143e60 2024-04-25T10:07:42.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.344+0000 7f2336a00640 1 -- 172.21.15.165:0/3290889696 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328032bf0 con 0x7f2338143e60 2024-04-25T10:07:42.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 -- 172.21.15.165:0/3290889696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 msgr2=0x7f233806e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 --2- 172.21.15.165:0/3290889696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233806e160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f232800bb10 tx=0x7f232802c920 comp rx=0 tx=0).stop 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 -- 172.21.15.165:0/3290889696 shutdown_connections 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 --2- 172.21.15.165:0/3290889696 >> v2:172.21.15.172:3300/0 conn(0x7f233806e6a0 0x7f23380cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 --2- 172.21.15.165:0/3290889696 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233806e160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 -- 172.21.15.165:0/3290889696 >> 172.21.15.165:0/3290889696 conn(0x7f233806abe0 msgr2=0x7f233806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 -- 172.21.15.165:0/3290889696 shutdown_connections 2024-04-25T10:07:42.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.345+0000 7f233ce00640 1 -- 172.21.15.165:0/3290889696 wait complete. 2024-04-25T10:07:42.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.346+0000 7f233ce00640 1 Processor -- start 2024-04-25T10:07:42.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.347+0000 7f233ce00640 1 -- start start 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.347+0000 7f233ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 0x7f23381635c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.347+0000 7f233ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23380cc840 0x7f233813f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f233ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233813f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f233ce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233815eac0 con 0x7f233806e6a0 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f233ce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233815e7e0 con 0x7f23380cc840 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f233ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380b2d30 con 0x7f2338143e60 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 0x7f23381635c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 0x7f23381635c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56088/0 (socket says 172.21.15.165:56088) 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 -- 172.21.15.165:0/3742497935 learned_addr learned my addr 172.21.15.165:0/3742497935 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f232ea00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23380cc840 0x7f233813f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 -- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 msgr2=0x7f233813f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337e00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233813f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233813f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 -- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23380cc840 msgr2=0x7f233813f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23380cc840 0x7f233813f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337400640 1 -- 172.21.15.165:0/3742497935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2338164260 con 0x7f233806e6a0 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f232ea00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23380cc840 0x7f233813f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.348+0000 7f2337e00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233813f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.349+0000 7f2337400640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 0x7f23381635c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f232800bb10 tx=0x7f2328004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.349+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328041070 con 0x7f233806e6a0 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.349+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232803acf0 con 0x7f233806e6a0 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.349+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2328004270 con 0x7f233806e6a0 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.349+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2338163f80 con 0x7f233806e6a0 2024-04-25T10:07:42.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.349+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233815f630 con 0x7f233806e6a0 2024-04-25T10:07:42.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.350+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2338073990 con 0x7f233806e6a0 2024-04-25T10:07:42.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.351+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2328039020 con 0x7f233806e6a0 2024-04-25T10:07:42.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.352+0000 7f232fe00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:42.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.353+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f23280ab130 con 0x7f233806e6a0 2024-04-25T10:07:42.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.353+0000 7f232ea00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.353+0000 7f232ea00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2338164de0 tx=0x7f2320008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:42.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.357+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f232802f070 con 0x7f233806e6a0 2024-04-25T10:07:42.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.587+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23380cbf40 con 0x7f233806e6a0 2024-04-25T10:07:42.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.588+0000 7f232fe00640 1 -- 172.21.15.165:0/3742497935 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+4155 (secure 0 0 0) 0x7f2328043350 con 0x7f233806e6a0 2024-04-25T10:07:42.594 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:42.594 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":44,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":44,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:42.087711+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:replay","state_seq":1,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:42.594 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 44 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 msgr2=0x7f231406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2338164de0 tx=0x7f2320008040 comp rx=0 tx=0).stop 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 msgr2=0x7f23381635c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 0x7f23381635c0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f232800bb10 tx=0x7f2328004730 comp rx=0 tx=0).stop 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 shutdown_connections 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f231406b4d0 0x7f231406d990 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2338143e60 0x7f233813f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23380cc840 0x7f233813f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 --2- 172.21.15.165:0/3742497935 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f233806e6a0 0x7f23381635c0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 >> 172.21.15.165:0/3742497935 conn(0x7f233806abe0 msgr2=0x7f233815bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 shutdown_connections 2024-04-25T10:07:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:42.591+0000 7f233ce00640 1 -- 172.21.15.165:0/3742497935 wait complete. 2024-04-25T10:07:43.018 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 44, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:28.014241+0000', 'modified': '2024-04-25T10:07:42.087711+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/148834032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 148834032}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 148834032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:43.018 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6870:up:replay 2024-04-25T10:07:44.019 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:44.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.227+0000 7fdfae600640 1 Processor -- start 2024-04-25T10:07:44.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.227+0000 7fdfae600640 1 -- start start 2024-04-25T10:07:44.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.228+0000 7fdfae600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdfa80b93c0 0x7fdfa80cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.228+0000 7fdfae600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa80d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.228+0000 7fdfae600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa8161df0 con 0x7fdfa815db90 2024-04-25T10:07:44.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.228+0000 7fdfae600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa80d0980 con 0x7fdfa80b93c0 2024-04-25T10:07:44.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.228+0000 7fdfae600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80d1f50 con 0x7fdfa80d04f0 2024-04-25T10:07:44.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.229+0000 7fdfa7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdfa80b93c0 0x7fdfa80cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.229+0000 7fdfa7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdfa80b93c0 0x7fdfa80cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56102/0 (socket says 172.21.15.165:56102) 2024-04-25T10:07:44.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.229+0000 7fdfa7e00640 1 -- 172.21.15.165:0/1521269982 learned_addr learned my addr 172.21.15.165:0/1521269982 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:44.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.229+0000 7fdfadc00640 1 --2- 172.21.15.165:0/1521269982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa80d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.229+0000 7fdfa7e00640 1 -- 172.21.15.165:0/1521269982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 msgr2=0x7fdfa80d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:44.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.229+0000 7fdfa7e00640 1 --2- 172.21.15.165:0/1521269982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa80d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.230+0000 7fdfa7e00640 1 -- 172.21.15.165:0/1521269982 >> v1:172.21.15.172:6789/0 conn(0x7fdfa815db90 legacy=0x7fdfa81615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.230+0000 7fdfa7e00640 1 -- 172.21.15.165:0/1521269982 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa81620d0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.231+0000 7fdfa7e00640 1 --2- 172.21.15.165:0/1521269982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80cffb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009870 tx=0x7fdf9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd36500c1cacaa74 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.231+0000 7fdfa7400640 1 -- 172.21.15.165:0/1521269982 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94037070 con 0x7fdfa80b93c0 2024-04-25T10:07:44.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.231+0000 7fdfa7400640 1 -- 172.21.15.165:0/1521269982 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf94032440 con 0x7fdfa80b93c0 2024-04-25T10:07:44.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.231+0000 7fdfa7400640 1 -- 172.21.15.165:0/1521269982 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94032810 con 0x7fdfa80b93c0 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 -- 172.21.15.165:0/1521269982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 msgr2=0x7fdfa80cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 --2- 172.21.15.165:0/1521269982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80cffb0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fdf94009870 tx=0x7fdf9402a270 comp rx=0 tx=0).stop 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 -- 172.21.15.165:0/1521269982 shutdown_connections 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 --2- 172.21.15.165:0/1521269982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa80d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 --2- 172.21.15.165:0/1521269982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80cffb0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 -- 172.21.15.165:0/1521269982 >> 172.21.15.165:0/1521269982 conn(0x7fdfa80b52b0 msgr2=0x7fdfa80b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 -- 172.21.15.165:0/1521269982 shutdown_connections 2024-04-25T10:07:44.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.232+0000 7fdfae600640 1 -- 172.21.15.165:0/1521269982 wait complete. 2024-04-25T10:07:44.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.234+0000 7fdfae600640 1 Processor -- start 2024-04-25T10:07:44.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.234+0000 7fdfae600640 1 -- start start 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.235+0000 7fdfae600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.235+0000 7fdfae600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa8160640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.235+0000 7fdfae600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa815db90 0x7fdfa80cc610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfae600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80b68d0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfae600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80d1be0 con 0x7fdfa815db90 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfae600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa80b6a50 con 0x7fdfa80d04f0 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56112/0 (socket says 172.21.15.165:56112) 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 -- 172.21.15.165:0/2826663774 learned_addr learned my addr 172.21.15.165:0/2826663774 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:44.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfa7e00640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa8160640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 -- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 msgr2=0x7fdfa8160640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa8160640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 -- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa815db90 msgr2=0x7fdfa80cc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa815db90 0x7fdfa80cc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.236+0000 7fdfad200640 1 -- 172.21.15.165:0/2826663774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa815e1c0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.237+0000 7fdfad200640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80ca870 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00c730 tx=0x7fdf9c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.237+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c016020 con 0x7fdfa80b93c0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.237+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf9c006d50 con 0x7fdfa80b93c0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.237+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf9c003bb0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.237+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa815e4a0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.237+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa815ea60 con 0x7fdfa80b93c0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.238+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa8073990 con 0x7fdfa80b93c0 2024-04-25T10:07:44.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.239+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf9c00a850 con 0x7fdfa80b93c0 2024-04-25T10:07:44.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.240+0000 7fdfa4c00640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:44.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.241+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fdf9c08a0a0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.241+0000 7fdfa7e00640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:44.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.241+0000 7fdfa7e00640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf94034000 tx=0x7fdf9402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:44.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.246+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf9c020020 con 0x7fdfa80b93c0 2024-04-25T10:07:44.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.480+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdfa80d14b0 con 0x7fdfa80b93c0 2024-04-25T10:07:44.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.482+0000 7fdfa4c00640 1 -- 172.21.15.165:0/2826663774 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fdf9c019310 con 0x7fdfa80b93c0 2024-04-25T10:07:44.487 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:44.487 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":46,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:43.112736+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:44.488 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 46 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf8806b4d0 msgr2=0x7fdf8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf8806b4d0 0x7fdf8806d990 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fdf94034000 tx=0x7fdf9402e000 comp rx=0 tx=0).stop 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 msgr2=0x7fdfa80ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80ca870 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fdf9c00c730 tx=0x7fdf9c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 shutdown_connections 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf8806b4d0 0x7fdf8806d990 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfa815db90 0x7fdfa80cc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfa80d04f0 0x7fdfa8160640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 --2- 172.21.15.165:0/2826663774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfa80b93c0 0x7fdfa80ca870 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:44.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 >> 172.21.15.165:0/2826663774 conn(0x7fdfa80b52b0 msgr2=0x7fdfa80b7d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:44.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.484+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 shutdown_connections 2024-04-25T10:07:44.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:44.485+0000 7fdfae600640 1 -- 172.21.15.165:0/2826663774 wait complete. 2024-04-25T10:07:44.912 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:28.014241+0000', 'modified': '2024-04-25T10:07:43.112736+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.165:6837/148834032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 148834032}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 148834032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:44.912 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_6870:up:rejoin 2024-04-25T10:07:45.913 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:07:46.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.120+0000 7f3857e00640 1 Processor -- start 2024-04-25T10:07:46.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.120+0000 7f3857e00640 1 -- start start 2024-04-25T10:07:46.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.121+0000 7f3857e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 0x7f38500cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.121+0000 7f3857e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3850071650 con 0x7f385015df50 2024-04-25T10:07:46.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.121+0000 7f3857e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38500cf230 con 0x7f385015db70 2024-04-25T10:07:46.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.121+0000 7f3857e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38500cbd80 con 0x7f38500cfc20 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.122+0000 7f3855e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f385015df50 0x7f385006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58212/0 (socket says 172.21.15.165:58212) 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.122+0000 7f3855e00640 1 -- 172.21.15.165:0/3306564649 learned_addr learned my addr 172.21.15.165:0/3306564649 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.122+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745578985 0 0) 0x7f3848001490 con 0x7f385015df50 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.122+0000 7f3854a00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 0x7f38500cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.122+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3838003520 con 0x7f385015df50 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3734752033 0 0) 0x7f3838003520 con 0x7f385015df50 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 msgr2=0x7f38500cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 0x7f38500cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 >> v1:172.21.15.172:6789/0 conn(0x7f385015db70 legacy=0x7f38500cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3850161b60 con 0x7f385015df50 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3848001bc0 con 0x7f385015df50 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 >> v1:172.21.15.165:6789/0 conn(0x7f385015df50 legacy=0x7f385006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 0x7f38380039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3838004010 0x7f3838004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3838004d40 0x7f38380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38500cbd80 con 0x7f3838004d40 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383800bb80 con 0x7f3838004010 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383800bd80 con 0x7f38500cfc20 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f3855e00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3838004d40 0x7f38380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.123+0000 7f3854a00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3838004010 0x7f3838004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.124+0000 7f3855400640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 0x7f38380039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.124+0000 7f3855e00640 1 -- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 msgr2=0x7f38380039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.124+0000 7f3855e00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38500cfc20 0x7f38380039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.124+0000 7f3855e00640 1 -- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3838004010 msgr2=0x7f3838004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.124+0000 7f3855e00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3838004010 0x7f3838004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.124+0000 7f3855e00640 1 -- 172.21.15.165:0/3306564649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3838003520 con 0x7f3838004d40 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.125+0000 7f3855e00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3838004d40 0x7f38380051c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f384800aa20 tx=0x7f3838005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62aa43cda622d37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.125+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800a640 con 0x7f3838004d40 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.125+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f384802f070 con 0x7f3838004d40 2024-04-25T10:07:46.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.125+0000 7f384fe00640 1 -- 172.21.15.165:0/3306564649 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848034470 con 0x7f3838004d40 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3857e00640 1 -- 172.21.15.165:0/3306564649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3838004d40 msgr2=0x7f38380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3857e00640 1 --2- 172.21.15.165:0/3306564649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3838004d40 0x7f38380051c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f384800aa20 tx=0x7f3838005830 comp rx=0 tx=0).stop 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3855e00640 1 -- 172.21.15.165:0/3306564649 reap_dead start 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3857e00640 1 -- 172.21.15.165:0/3306564649 shutdown_connections 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3857e00640 1 -- 172.21.15.165:0/3306564649 >> 172.21.15.165:0/3306564649 conn(0x7f385006abe0 msgr2=0x7f385006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3857e00640 1 -- 172.21.15.165:0/3306564649 shutdown_connections 2024-04-25T10:07:46.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.126+0000 7f3857e00640 1 -- 172.21.15.165:0/3306564649 wait complete. 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.128+0000 7f3857e00640 1 Processor -- start 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.128+0000 7f3857e00640 1 -- start start 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3857e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3838004d40 0x7f3850145a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3857e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f385014c310 0x7f38501f6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3857e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 0x7f38500af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3857e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850162120 con 0x7f38500af510 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3857e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3850161910 con 0x7f385014c310 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3857e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3850162640 con 0x7f3838004d40 2024-04-25T10:07:46.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3855e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 0x7f38500af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3855e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 0x7f38500af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56138/0 (socket says 172.21.15.165:56138) 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3855e00640 1 -- 172.21.15.165:0/3968964665 learned_addr learned my addr 172.21.15.165:0/3968964665 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.129+0000 7f3854a00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f385014c310 0x7f38501f6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855400640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3838004d40 0x7f3850145a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855e00640 1 -- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3838004d40 msgr2=0x7f3850145a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3838004d40 0x7f3850145a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855e00640 1 -- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f385014c310 msgr2=0x7f38501f6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f385014c310 0x7f38501f6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855e00640 1 -- 172.21.15.165:0/3968964665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f385015aa50 con 0x7f38500af510 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3854a00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f385014c310 0x7f38501f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855400640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3838004d40 0x7f3850145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3855e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 0x7f38500af9c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f384800aa20 tx=0x7f38500cf400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f384800a640 con 0x7f38500af510 2024-04-25T10:07:46.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3848001bb0 con 0x7f38500af510 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3848001490 con 0x7f38500af510 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.130+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38501f76f0 con 0x7f38500af510 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.131+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38501f7cb0 con 0x7f38500af510 2024-04-25T10:07:46.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.132+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3850073990 con 0x7f38500af510 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.133+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3848034470 con 0x7f38500af510 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.134+0000 7f384d600640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f383406b500 0x7f383406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.134+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f38480aa930 con 0x7f38500af510 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.134+0000 7f3855400640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f383406b500 0x7f383406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:46.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.135+0000 7f3855400640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f383406b500 0x7f383406d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3844000c00 tx=0x7f3844009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:46.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.140+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3850071650 con 0x7f38500af510 2024-04-25T10:07:46.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.375+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f38500b3610 con 0x7f38500af510 2024-04-25T10:07:46.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.376+0000 7f384d600640 1 -- 172.21.15.165:0/3968964665 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f38480794a0 con 0x7f38500af510 2024-04-25T10:07:46.381 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:46.382 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:44.122864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:07:46.382 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 47 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f383406b500 msgr2=0x7f383406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f383406b500 0x7f383406d9c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f3844000c00 tx=0x7f3844009040 comp rx=0 tx=0).stop 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 msgr2=0x7f38500af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 0x7f38500af9c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f384800aa20 tx=0x7f38500cf400 comp rx=0 tx=0).stop 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 shutdown_connections 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f383406b500 0x7f383406d9c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f38500af510 0x7f38500af9c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f385014c310 0x7f38501f6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 --2- 172.21.15.165:0/3968964665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3838004d40 0x7f3850145a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:46.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 >> 172.21.15.165:0/3968964665 conn(0x7f385006abe0 msgr2=0x7f38500cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:46.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.378+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 shutdown_connections 2024-04-25T10:07:46.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:46.379+0000 7f3857e00640 1 -- 172.21.15.165:0/3968964665 wait complete. 2024-04-25T10:07:46.805 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 47, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:07:28.014241+0000', 'modified': '2024-04-25T10:07:44.122864+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 71, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6870}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6870': {'gid': 6870, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.165:6837/148834032', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 148834032}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 148834032}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:07:46.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:07:46.806 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:07:47.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.004+0000 7fd3f3200640 1 Processor -- start 2024-04-25T10:07:47.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.004+0000 7fd3f3200640 1 -- start start 2024-04-25T10:07:47.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.005+0000 7fd3f3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.005+0000 7fd3f3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3f40ce950 0x7fd3f40c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.005+0000 7fd3f3200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f4145520 con 0x7fd3f40c5030 2024-04-25T10:07:47.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.005+0000 7fd3f3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f40ced90 con 0x7fd3f40ce950 2024-04-25T10:07:47.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.005+0000 7fd3f3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4071270 con 0x7fd3f4145140 2024-04-25T10:07:47.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.006+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.006+0000 7fd3e9400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3f40ce950 0x7fd3f40c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.006+0000 7fd3e9400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd3f40ce950 0x7fd3f40c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:41744/0 (socket says 172.21.15.165:41744) 2024-04-25T10:07:47.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.006+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f40ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:57452/0 (socket says 172.21.15.165:57452) 2024-04-25T10:07:47.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.006+0000 7fd3e9400640 1 -- 172.21.15.165:0/824901653 learned_addr learned my addr 172.21.15.165:0/824901653 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.007+0000 7fd3e9400640 1 -- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 msgr2=0x7fd3f40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.007+0000 7fd3e9400640 1 --2- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.007+0000 7fd3e9400640 1 -- 172.21.15.165:0/824901653 >> v1:172.21.15.172:6789/0 conn(0x7fd3f40c5030 legacy=0x7fd3f406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.007+0000 7fd3e9400640 1 -- 172.21.15.165:0/824901653 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f4071550 con 0x7fd3f40ce950 2024-04-25T10:07:47.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.007+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:07:47.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.008+0000 7fd3e9400640 1 --2- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f40c4af0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009870 tx=0x7fd3dc02a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e795113ab8ebaa17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.008+0000 7fd3f1400640 1 -- 172.21.15.165:0/824901653 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc037070 con 0x7fd3f40ce950 2024-04-25T10:07:47.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.008+0000 7fd3f1400640 1 -- 172.21.15.165:0/824901653 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc032440 con 0x7fd3f40ce950 2024-04-25T10:07:47.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.008+0000 7fd3f1400640 1 -- 172.21.15.165:0/824901653 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc032810 con 0x7fd3f40ce950 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 -- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 msgr2=0x7fd3f40c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 --2- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f40c4af0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc009870 tx=0x7fd3dc02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 -- 172.21.15.165:0/824901653 shutdown_connections 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 --2- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f40c4af0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 --2- 172.21.15.165:0/824901653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f40ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 -- 172.21.15.165:0/824901653 >> 172.21.15.165:0/824901653 conn(0x7fd3f4061cc0 msgr2=0x7fd3f406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 -- 172.21.15.165:0/824901653 shutdown_connections 2024-04-25T10:07:47.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.009+0000 7fd3f3200640 1 -- 172.21.15.165:0/824901653 wait complete. 2024-04-25T10:07:47.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.011+0000 7fd3f3200640 1 Processor -- start 2024-04-25T10:07:47.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.011+0000 7fd3f3200640 1 -- start start 2024-04-25T10:07:47.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 0x7fd3f41f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f413d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f415ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f40c54a0 con 0x7fd3f40c5030 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f4071b10 con 0x7fd3f40ce950 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f40c5620 con 0x7fd3f4145140 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 0x7fd3f41f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 0x7fd3f41f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54988/0 (socket says 172.21.15.165:54988) 2024-04-25T10:07:47.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.012+0000 7fd3f1e00640 1 -- 172.21.15.165:0/1589153706 learned_addr learned my addr 172.21.15.165:0/1589153706 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3e9400640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f413d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f2800640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f415ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f1e00640 1 -- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 msgr2=0x7fd3f415ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f415ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f1e00640 1 -- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 msgr2=0x7fd3f413d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f413d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f1e00640 1 -- 172.21.15.165:0/1589153706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f41f6f90 con 0x7fd3f40c5030 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f2800640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f415ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3e9400640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f413d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 0x7fd3f41f6740 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400e960 tx=0x7fd3e400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3eaa00640 1 -- 172.21.15.165:0/1589153706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4017070 con 0x7fd3f40c5030 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3eaa00640 1 -- 172.21.15.165:0/1589153706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e400ca00 con 0x7fd3f40c5030 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3eaa00640 1 -- 172.21.15.165:0/1589153706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e400cdd0 con 0x7fd3f40c5030 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3f41f7270 con 0x7fd3f40c5030 2024-04-25T10:07:47.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.013+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f41f6c80 con 0x7fd3f40c5030 2024-04-25T10:07:47.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.016+0000 7fd3eaa00640 1 -- 172.21.15.165:0/1589153706 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd3e401a070 con 0x7fd3f40c5030 2024-04-25T10:07:47.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.016+0000 7fd3eaa00640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd3d006b4d0 0x7fd3d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.017+0000 7fd3eaa00640 1 -- 172.21.15.165:0/1589153706 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd3e408a760 con 0x7fd3f40c5030 2024-04-25T10:07:47.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.017+0000 7fd3e9400640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd3d006b4d0 0x7fd3d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.017+0000 7fd3fb9a0740 1 Processor -- start 2024-04-25T10:07:47.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.017+0000 7fd3fb9a0740 1 -- start start 2024-04-25T10:07:47.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.017+0000 7fd3e9400640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd3d006b4d0 0x7fd3d006d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034000 tx=0x7fd3dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a12fb30 0x563d6a157970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 0x563d6a12ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a12e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6a093620 con 0x563d6a157eb0 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6a03cca0 con 0x563d6a1787a0 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6a08f6c0 con 0x563d6a12fb30 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3e9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 0x563d6a12ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3e9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 0x563d6a12ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55002/0 (socket says 172.21.15.165:55002) 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.018+0000 7fd3e9400640 1 -- 172.21.15.165:0/2428802248 learned_addr learned my addr 172.21.15.165:0/2428802248 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3f2800640 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a12e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a12fb30 0x563d6a157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e9400640 1 -- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a12fb30 msgr2=0x563d6a157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e9400640 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a12fb30 0x563d6a157970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e9400640 1 -- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 msgr2=0x563d6a12e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e9400640 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a12e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e9400640 1 -- 172.21.15.165:0/2428802248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d6a043f40 con 0x563d6a157eb0 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3f2800640 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a12e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e9400640 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 0x563d6a12ff10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc035040 tx=0x7fd3dc02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f75afc8f8f8e3fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e2600640 1 -- 172.21.15.165:0/2428802248 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc04d070 con 0x563d6a157eb0 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e2600640 1 -- 172.21.15.165:0/2428802248 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc03c460 con 0x563d6a157eb0 2024-04-25T10:07:47.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.019+0000 7fd3e2600640 1 -- 172.21.15.165:0/2428802248 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc037040 con 0x563d6a157eb0 2024-04-25T10:07:47.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2428802248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 msgr2=0x563d6a12ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 0x563d6a12ff10 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc035040 tx=0x7fd3dc02c000 comp rx=0 tx=0).stop 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2428802248 shutdown_connections 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a12e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a157eb0 0x563d6a12ff10 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2428802248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a12fb30 0x563d6a157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2428802248 >> 172.21.15.165:0/2428802248 conn(0x563d6a12cd50 msgr2=0x563d6a175730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2428802248 shutdown_connections 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.020+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2428802248 wait complete. 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.021+0000 7fd3fb9a0740 1 -- start start 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.021+0000 7fd3fb9a0740 1 Processor -- start 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a154930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a176270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a14c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6a1574e0 con 0x563d6a153a50 2024-04-25T10:07:47.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6a0321f0 con 0x563d6a1787a0 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6a156ed0 con 0x563d6a157eb0 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3fb9a0740 10 client.0 _ll_register_callbacks cb 0x563d6a04fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3f2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a14c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.022+0000 7fd3f2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a14c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55016/0 (socket says 172.21.15.165:55016) 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 -- 172.21.15.165:0/4039942953 learned_addr learned my addr 172.21.15.165:0/4039942953 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3e9400640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a176270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 -- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 msgr2=0x563d6a154930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a154930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 -- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 msgr2=0x563d6a176270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a176270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 -- 172.21.15.165:0/4039942953 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d6a051c50 con 0x563d6a153a50 2024-04-25T10:07:47.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3e9400640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a176270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3f2800640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a14c1b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e4a0 tx=0x7fd3ec00e960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcfbf82010f92b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.023+0000 7fd3d6a00640 1 -- 172.21.15.165:0/4039942953 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec018060 con 0x563d6a153a50 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3d6a00640 1 -- 172.21.15.165:0/4039942953 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd3ec003d80 con 0x563d6a153a50 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3dc03c460 con 0x563d6a153a50 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3d6a00640 1 -- 172.21.15.165:0/4039942953 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3ec01e050 con 0x563d6a153a50 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 10 client.6897 fetch_fsmap learned FSMap version 47 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x563d69da82d0 con 0x563d6a153a50 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3d6a00640 1 -- 172.21.15.165:0/4039942953 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc03c460 con 0x563d6a153a50 2024-04-25T10:07:47.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3d6a00640 1 -- 172.21.15.165:0/4039942953 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3ec021020 con 0x563d6a153a50 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 10 client.6897 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 10 client.6897 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 20 client.6897 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a191450 0x563d6a193840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 4 client.6897 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.024+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 --> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d6a0a0b60 con 0x563d6a191450 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.025+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a191450 0x563d6a193840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.025+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a191450 0x563d6a193840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=389e7aff76c109bb server_cookie=6a79a658074ad2e6 in_seq=0 out_seq=0 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.025+0000 7fd3d6a00640 10 client.6897 ms_handle_connect on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3d6a00640 1 -- 172.21.15.165:0/4039942953 <== mds.0 v2:172.21.15.165:6836/148834032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563d6a0a0b60 con 0x563d6a191450 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3d6a00640 10 client.6897 handle_command_reply: tid=0 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 client.6897 shutdown 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 msgr2=0x563d6a14c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a14c1b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec00e4a0 tx=0x7fd3ec00e960 comp rx=0 tx=0).stop 2024-04-25T10:07:47.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 shutdown_connections 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a14c1b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a191450 0x563d6a193840 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a176270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.026+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/4039942953 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a154930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.027+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 >> 172.21.15.165:0/4039942953 conn(0x563d6a12fb30 msgr2=0x563d6a178c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.027+0000 7fd3d6a00640 0 client.6897 ms_handle_reset on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.027+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 shutdown_connections 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.027+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/4039942953 wait complete. 2024-04-25T10:07:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.027+0000 7fd3fb9a0740 20 client.6897 trim_cache size 0 max 16384 2024-04-25T10:07:47.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.044+0000 7fd3fb9a0740 1 Processor -- start 2024-04-25T10:07:47.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.044+0000 7fd3fb9a0740 1 -- start start 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a197910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a198060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41ff650 con 0x563d6a153a50 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41fead0 con 0x563d6a1787a0 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f41ff050 con 0x563d6a157eb0 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55028/0 (socket says 172.21.15.165:55028) 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 -- 172.21.15.165:0/2828320041 learned_addr learned my addr 172.21.15.165:0/2828320041 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 -- 172.21.15.165:0/2828320041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 msgr2=0x563d6a198060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a198060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 -- 172.21.15.165:0/2828320041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 msgr2=0x563d6a156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.045+0000 7fd3f1e00640 1 -- 172.21.15.165:0/2828320041 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d69e726b0 con 0x563d6a153a50 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.046+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a197910 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4058b80 tx=0x7fd3e4088870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf7387d01600326 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.046+0000 7fd3e1c00640 1 -- 172.21.15.165:0/2828320041 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4021040 con 0x563d6a153a50 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.046+0000 7fd3e1c00640 1 -- 172.21.15.165:0/2828320041 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e4056c90 con 0x563d6a153a50 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.046+0000 7fd3e1c00640 1 -- 172.21.15.165:0/2828320041 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e4056ef0 con 0x563d6a153a50 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.046+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2828320041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 msgr2=0x563d6a197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.046+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a197910 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fd3e4058b80 tx=0x7fd3e4088870 comp rx=0 tx=0).stop 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2828320041 shutdown_connections 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a1787a0 0x563d6a156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a157eb0 0x563d6a198060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/2828320041 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a153a50 0x563d6a197910 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2828320041 >> 172.21.15.165:0/2828320041 conn(0x563d6a12fb30 msgr2=0x563d6a12f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2828320041 shutdown_connections 2024-04-25T10:07:47.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/2828320041 wait complete. 2024-04-25T10:07:47.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 -- start start 2024-04-25T10:07:47.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.047+0000 7fd3fb9a0740 1 Processor -- start 2024-04-25T10:07:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a153a50 0x563d6a1527e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a157eb0 0x563d6a156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 0x563d6a157410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6a1564c0 con 0x563d6a1787a0 2024-04-25T10:07:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6a1560c0 con 0x563d6a157eb0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563d6a14f390 con 0x563d6a153a50 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.048+0000 7fd3fb9a0740 10 client.0 _ll_register_callbacks cb 0x7fd3ec001560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 0x563d6a157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 0x563d6a157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55040/0 (socket says 172.21.15.165:55040) 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 -- 172.21.15.165:0/332678344 learned_addr learned my addr 172.21.15.165:0/332678344 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 -- 172.21.15.165:0/332678344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a153a50 msgr2=0x563d6a1527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a153a50 0x563d6a1527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 -- 172.21.15.165:0/332678344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a157eb0 msgr2=0x563d6a156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a157eb0 0x563d6a156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 -- 172.21.15.165:0/332678344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d6a148cb0 con 0x563d6a1787a0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3f2800640 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 0x563d6a157410 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005750 tx=0x7fd3ec001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c83ad197fbe4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.049+0000 7fd3d6a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3d6a00640 1 -- 172.21.15.165:0/332678344 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ec028030 con 0x563d6a1787a0 2024-04-25T10:07:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3d6a00640 1 -- 172.21.15.165:0/332678344 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(73..73 src has 1..73) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd3ec002060 con 0x563d6a1787a0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd3e4056c90 con 0x563d6a1787a0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3d6a00640 1 -- 172.21.15.165:0/332678344 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd3ec013650 con 0x563d6a1787a0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 10 client.6903 fetch_fsmap learned FSMap version 47 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x563d6a154180 con 0x563d6a1787a0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3d6a00640 1 -- 172.21.15.165:0/332678344 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e4056c90 con 0x563d6a1787a0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3d6a00640 1 -- 172.21.15.165:0/332678344 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd3ec021040 con 0x563d6a1787a0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 10 client.6903 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 10 client.6903 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 20 client.6903 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a1298b0 0x563d6a129c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 4 client.6903 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.050+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 --> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563d6a09aeb0 con 0x563d6a1298b0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a1298b0 0x563d6a129c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3f1e00640 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a1298b0 0x563d6a129c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfed50808ae74c5 server_cookie=6808572bd7062807 in_seq=0 out_seq=0 2024-04-25T10:07:47.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3d6a00640 10 client.6903 ms_handle_connect on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3d6a00640 1 -- 172.21.15.165:0/332678344 <== mds.0 v2:172.21.15.165:6836/148834032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x563d6a09aeb0 con 0x563d6a1298b0 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3d6a00640 10 client.6903 handle_command_reply: tid=0 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3fb9a0740 1 client.6903 shutdown 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 msgr2=0x563d6a157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 0x563d6a157410 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd3ec005750 tx=0x7fd3ec001250 comp rx=0 tx=0).stop 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 shutdown_connections 2024-04-25T10:07:47.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563d6a1787a0 0x563d6a157410 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.051+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x563d6a1298b0 0x563d6a129c90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563d6a157eb0 0x563d6a156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3fb9a0740 1 --2- 172.21.15.165:0/332678344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563d6a153a50 0x563d6a1527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3d6a00640 0 client.6903 ms_handle_reset on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 >> 172.21.15.165:0/332678344 conn(0x563d6a12fb30 msgr2=0x563d6a198040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 shutdown_connections 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3fb9a0740 1 -- 172.21.15.165:0/332678344 wait complete. 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3fb9a0740 20 client.6903 trim_cache size 0 max 16384 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:07:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 6870, 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 4.9551923520000001, 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 47, 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 73, 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 72, 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 5.7494628790000002 2024-04-25T10:07:47.063 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd3d006b4d0 msgr2=0x7fd3d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3f3200640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd3d006b4d0 0x7fd3d006d990 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fd3dc034000 tx=0x7fd3dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 msgr2=0x7fd3f41f6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.052+0000 7fd3f3200640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 0x7fd3f41f6740 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7fd3e400e960 tx=0x7fd3e400ee20 comp rx=0 tx=0).stop 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 shutdown_connections 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd3d006b4d0 0x7fd3d006d990 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd3f4145140 0x7fd3f415ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd3f40ce950 0x7fd3f413d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 --2- 172.21.15.165:0/1589153706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd3f40c5030 0x7fd3f41f6740 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 >> 172.21.15.165:0/1589153706 conn(0x7fd3f4061cc0 msgr2=0x7fd3f406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 shutdown_connections 2024-04-25T10:07:47.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.053+0000 7fd3f3200640 1 -- 172.21.15.165:0/1589153706 wait complete. 2024-04-25T10:07:47.076 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:07:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.258+0000 7f6a83a00640 1 Processor -- start 2024-04-25T10:07:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.259+0000 7f6a83a00640 1 -- start start 2024-04-25T10:07:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.260+0000 7f6a83a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a7c0cec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.260+0000 7f6a83a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c0cf330 con 0x7f6a7c165680 2024-04-25T10:07:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.260+0000 7f6a83a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a7c071d80 con 0x7f6a7c0aecf0 2024-04-25T10:07:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.260+0000 7f6a83a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c072340 con 0x7f6a7c0cfc90 2024-04-25T10:07:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a81000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6a7c165680 0x7f6a7c0cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:59590/0 (socket says 172.21.15.165:59590) 2024-04-25T10:07:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a81000640 1 -- 172.21.15.165:0/2894606921 learned_addr learned my addr 172.21.15.165:0/2894606921 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:47.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a81a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a7c0cec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505283161 0 0) 0x7f6a7c0cf330 con 0x7f6a7c165680 2024-04-25T10:07:47.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6a5c003520 con 0x7f6a7c165680 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3549590265 0 0) 0x7f6a5c003520 con 0x7f6a7c165680 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 msgr2=0x7f6a7c0cec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a7be00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a7c0cec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.261+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 >> v1:172.21.15.172:6789/0 conn(0x7f6a7c0aecf0 legacy=0x7f6a7c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c072060 con 0x7f6a7c165680 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6a6c001670 con 0x7f6a7c165680 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a81a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a7c0cec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 >> v1:172.21.15.165:6789/0 conn(0x7f6a7c165680 legacy=0x7f6a7c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a5c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c003fd0 0x7f6a5c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a5c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c072340 con 0x7f6a5c004d40 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c00bbb0 con 0x7f6a5c003fd0 2024-04-25T10:07:47.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5c00bdb0 con 0x7f6a7c0cfc90 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a81a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a5c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a73e00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c003fd0 0x7f6a5c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.262+0000 7f6a81000640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a5c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.263+0000 7f6a81a00640 1 -- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 msgr2=0x7f6a5c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.263+0000 7f6a81a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c0cfc90 0x7f6a5c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.263+0000 7f6a81a00640 1 -- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c003fd0 msgr2=0x7f6a5c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.263+0000 7f6a81a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a5c003fd0 0x7f6a5c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.263+0000 7f6a81a00640 1 -- 172.21.15.165:0/2894606921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c003520 con 0x7f6a5c004d40 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a81a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a5c0051c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6a740073e0 tx=0x7f6a5c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0180402910121a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a74009c80 con 0x7f6a5c004d40 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a7402e660 con 0x7f6a5c004d40 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a7be00640 1 -- 172.21.15.165:0/2894606921 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a7402ea10 con 0x7f6a5c004d40 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a83a00640 1 -- 172.21.15.165:0/2894606921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 msgr2=0x7f6a5c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a83a00640 1 --2- 172.21.15.165:0/2894606921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a5c0051c0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6a740073e0 tx=0x7f6a5c005830 comp rx=0 tx=0).stop 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.264+0000 7f6a81a00640 1 -- 172.21.15.165:0/2894606921 reap_dead start 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.265+0000 7f6a83a00640 1 -- 172.21.15.165:0/2894606921 shutdown_connections 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.265+0000 7f6a83a00640 1 -- 172.21.15.165:0/2894606921 >> 172.21.15.165:0/2894606921 conn(0x7f6a7c06abe0 msgr2=0x7f6a7c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.265+0000 7f6a83a00640 1 -- 172.21.15.165:0/2894606921 shutdown_connections 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.265+0000 7f6a83a00640 1 -- 172.21.15.165:0/2894606921 wait complete. 2024-04-25T10:07:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.266+0000 7f6a83a00640 1 Processor -- start 2024-04-25T10:07:47.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 -- start start 2024-04-25T10:07:47.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a7c15b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c160390 0x7f6a7c1608c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a7c0a9080 0x7f6a7c0a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c165a60 con 0x7f6a5c004d40 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c071670 con 0x7f6a7c0a9080 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.267+0000 7f6a83a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a7c0ccdd0 con 0x7f6a7c160390 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a7c15b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a7c15b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55048/0 (socket says 172.21.15.165:55048) 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 -- 172.21.15.165:0/1366972467 learned_addr learned my addr 172.21.15.165:0/1366972467 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 -- 172.21.15.165:0/1366972467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c160390 msgr2=0x7f6a7c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c160390 0x7f6a7c1608c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 -- 172.21.15.165:0/1366972467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a7c0a9080 msgr2=0x7f6a7c0a9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a7c0a9080 0x7f6a7c0a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 -- 172.21.15.165:0/1366972467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a7c161150 con 0x7f6a5c004d40 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.268+0000 7f6a81000640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a7c15b500 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e920 tx=0x7f6a6c00ede0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.269+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00c7f0 con 0x7f6a5c004d40 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.269+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a6c018070 con 0x7f6a5c004d40 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.269+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6c00cbf0 con 0x7f6a5c004d40 2024-04-25T10:07:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.269+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a7c161430 con 0x7f6a5c004d40 2024-04-25T10:07:47.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.269+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a7c1619f0 con 0x7f6a5c004d40 2024-04-25T10:07:47.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.270+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a7c073990 con 0x7f6a5c004d40 2024-04-25T10:07:47.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.271+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a6c01d460 con 0x7f6a5c004d40 2024-04-25T10:07:47.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.272+0000 7f6a79600640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a5806b4d0 0x7f6a5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:47.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.272+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6a6c08a2f0 con 0x7f6a5c004d40 2024-04-25T10:07:47.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.272+0000 7f6a73e00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a5806b4d0 0x7f6a5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:47.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.273+0000 7f6a73e00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a5806b4d0 0x7f6a5806d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a64007fd0 tx=0x7f6a64009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:47.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.276+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a6c058f10 con 0x7f6a5c004d40 2024-04-25T10:07:47.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.512+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6a7c0cf330 con 0x7f6a5c004d40 2024-04-25T10:07:47.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.513+0000 7f6a79600640 1 -- 172.21.15.165:0/1366972467 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v47) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6a6c058810 con 0x7f6a5c004d40 2024-04-25T10:07:47.520 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:07:47.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a5806b4d0 msgr2=0x7f6a5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a5806b4d0 0x7f6a5806d990 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f6a64007fd0 tx=0x7f6a64009040 comp rx=0 tx=0).stop 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 msgr2=0x7f6a7c15b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a7c15b500 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f6a6c00e920 tx=0x7f6a6c00ede0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 shutdown_connections 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a5806b4d0 0x7f6a5806d990 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a7c0a9080 0x7f6a7c0a9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a7c160390 0x7f6a7c1608c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 --2- 172.21.15.165:0/1366972467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a5c004d40 0x7f6a7c15b500 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 >> 172.21.15.165:0/1366972467 conn(0x7f6a7c06abe0 msgr2=0x7f6a7c163c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 shutdown_connections 2024-04-25T10:07:47.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:47.516+0000 7f6a83a00640 1 -- 172.21.15.165:0/1366972467 wait complete. 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:07:47.541 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:47.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:47.542 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:07:47.592 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:07:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:07:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:07:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:07:47.593 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.593 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:07:47.594 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42298sec preferred_lft 42298sec 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:07:47.595 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:07:47.596 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.597 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:07:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:07:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:07:47.597 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:07:47.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:47.598 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:07:47.679 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:07:47.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:47.683 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:07:47.734 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:07:47.734 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:07:47.734 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:07:47.734 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:07:47.734 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:07:47.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:47.735 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:07:47.735 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:07:47.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:07:47.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:07:47.735 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:07:47.735 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:07:47.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:47.870 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:07:47.916 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:07:47.917 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:07:47.917 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:47.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:47.917 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:07:48.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:48.041 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:48.094 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:07:48.094 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:07:48.094 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:48.148 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:07:48.148 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:48.148 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:48.275 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:48.275 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:48.275 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:07:48.298 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:48.299 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:48.299 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:07:48.299 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:07:48.299 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:48.324 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:07:48.324 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:07:48.324 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:07:48.325 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:07:48.325 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:07:48.325 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:07:48.325 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:07:48.325 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:07:48.326 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:07:48.326 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:07:48.326 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:07:48.326 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:07:48.326 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:07:48.330 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:07:48.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:48.330 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:07:48.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:48.399 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:07:48.447 INFO:teuthology.orchestra.run.smithi165.stdout:client5434 2024-04-25T10:07:48.448 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:48.448 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:48.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:48.499 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:07:48.600 INFO:teuthology.orchestra.run.smithi165.stderr:rmdir: failed to remove 'scrub_repair_path': Directory not empty 2024-04-25T10:07:48.603 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:07:48.604 INFO:tasks.cephfs.test_scrub_checks:Running command 'scrub start /scrub_repair_path repair' 2024-04-25T10:07:48.604 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:07:48.604 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start /scrub_repair_path repair 2024-04-25T10:07:48.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.810+0000 7fd9c6a00640 1 Processor -- start 2024-04-25T10:07:48.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.810+0000 7fd9c6a00640 1 -- start start 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.811+0000 7fd9c6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd9c00cc4f0 0x7fd9c00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.811+0000 7fd9c6a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c0072280 con 0x7fd9c00cdb20 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.811+0000 7fd9c6a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9c00714c0 con 0x7fd9c015b980 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.811+0000 7fd9c6a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9c0071270 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.811+0000 7fd9bfe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd9c00cc4f0 0x7fd9c00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.812+0000 7fd9bfe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd9c00cc4f0 0x7fd9c00cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55052/0 (socket says 172.21.15.165:55052) 2024-04-25T10:07:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.812+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1346090286 learned_addr learned my addr 172.21.15.165:0/1346090286 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:48.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.812+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1346090286 >> v1:172.21.15.172:6789/0 conn(0x7fd9c015b980 legacy=0x7fd9c00ae500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.812+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1346090286 >> v1:172.21.15.165:6789/0 conn(0x7fd9c00cdb20 legacy=0x7fd9c015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.812+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1346090286 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00717a0 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.813+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/1346090286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c00cc8d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b002c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e7f54a07921d83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.813+0000 7fd9bf400640 1 -- 172.21.15.165:0/1346090286 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0038070 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9bf400640 1 -- 172.21.15.165:0/1346090286 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0031710 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9bf400640 1 -- 172.21.15.165:0/1346090286 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0031ae0 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1346090286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 msgr2=0x7fd9c00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1346090286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c00cc8d0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b002c4b0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1346090286 shutdown_connections 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1346090286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c00cc8d0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1346090286 >> 172.21.15.165:0/1346090286 conn(0x7fd9c006abe0 msgr2=0x7fd9c006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1346090286 shutdown_connections 2024-04-25T10:07:48.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.814+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1346090286 wait complete. 2024-04-25T10:07:48.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.816+0000 7fd9c6a00640 1 Processor -- start 2024-04-25T10:07:48.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.816+0000 7fd9c6a00640 1 -- start start 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9c6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c01f4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9c6a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9c00cdb20 0x7fd9c0154640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9c6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9c015b980 0x7fd9c0144cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9c6a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0071d60 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9c6a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c015bd80 con 0x7fd9c015b980 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9c6a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c006e8a0 con 0x7fd9c00cdb20 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c01f4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c01f4650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55054/0 (socket says 172.21.15.165:55054) 2024-04-25T10:07:48.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.817+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1552636029 learned_addr learned my addr 172.21.15.165:0/1552636029 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9b7400640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9c00cdb20 0x7fd9c0154640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9c015b980 0x7fd9c0144cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9c00cdb20 msgr2=0x7fd9c0154640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9c00cdb20 0x7fd9c0154640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9c015b980 msgr2=0x7fd9c0144cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9c015b980 0x7fd9c0144cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bfe00640 1 -- 172.21.15.165:0/1552636029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9c00ca760 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c01f4650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bcc00640 1 -- 172.21.15.165:0/1552636029 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0042020 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bcc00640 1 -- 172.21.15.165:0/1552636029 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0033830 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9bcc00640 1 -- 172.21.15.165:0/1552636029 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0038040 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.818+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9c01e36b0 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.819+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9c01e3c70 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.821+0000 7fd9bcc00640 1 -- 172.21.15.165:0/1552636029 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd9b0045020 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.822+0000 7fd9bcc00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.822+0000 7fd9bcc00640 1 -- 172.21.15.165:0/1552636029 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd9b00ab190 con 0x7fd9c00cc4f0 2024-04-25T10:07:48.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.823+0000 7fd9b7400640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.823+0000 7fd9c97ee740 1 Processor -- start 2024-04-25T10:07:48.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.823+0000 7fd9c97ee740 1 -- start start 2024-04-25T10:07:48.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.823+0000 7fd9b7400640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9c01f7120 tx=0x7fd9a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee4b10 0x55a8eaee8880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaee9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8eaeb1a00 con 0x55a8eaee8dc0 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8eae4ddd0 con 0x55a8eaee9c80 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8eadf6a90 con 0x55a8eaee4b10 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaee9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaee9200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55060/0 (socket says 172.21.15.165:55060) 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.824+0000 7fd9c4a00640 1 -- 172.21.15.165:0/2997355669 learned_addr learned my addr 172.21.15.165:0/2997355669 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9b7400640 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee4b10 0x55a8eaee8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9c4a00640 1 -- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee4b10 msgr2=0x55a8eaee8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee4b10 0x55a8eaee8880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9c4a00640 1 -- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 msgr2=0x55a8eaeea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9c4a00640 1 -- 172.21.15.165:0/2997355669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8eac2eaf0 con 0x55a8eaee8dc0 2024-04-25T10:07:48.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaee9200 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8001d20 tx=0x7fd9b80021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e897d0deebc83e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9afe00640 1 -- 172.21.15.165:0/2997355669 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8018070 con 0x55a8eaee8dc0 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9afe00640 1 -- 172.21.15.165:0/2997355669 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8003d80 con 0x55a8eaee8dc0 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.825+0000 7fd9afe00640 1 -- 172.21.15.165:0/2997355669 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b800db30 con 0x55a8eaee8dc0 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 -- 172.21.15.165:0/2997355669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 msgr2=0x55a8eaee9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaee9200 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8001d20 tx=0x7fd9b80021e0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 -- 172.21.15.165:0/2997355669 shutdown_connections 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaee9200 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee4b10 0x55a8eaee8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/2997355669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 -- 172.21.15.165:0/2997355669 >> 172.21.15.165:0/2997355669 conn(0x55a8eaee6d80 msgr2=0x55a8eaf2f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 -- 172.21.15.165:0/2997355669 shutdown_connections 2024-04-25T10:07:48.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.826+0000 7fd9c97ee740 1 -- 172.21.15.165:0/2997355669 wait complete. 2024-04-25T10:07:48.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.827+0000 7fd9c97ee740 1 -- start start 2024-04-25T10:07:48.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.827+0000 7fd9c97ee740 1 Processor -- start 2024-04-25T10:07:48.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf13210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf06f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf03800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8eadec080 con 0x55a8eaf42f90 2024-04-25T10:07:48.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8eae49e70 con 0x55a8eaee8dc0 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8eaf15810 con 0x55a8eaee9c80 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c97ee740 10 client.0 _ll_register_callbacks cb 0x55a8eae44900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf03800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf03800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55068/0 (socket says 172.21.15.165:55068) 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.828+0000 7fd9c4a00640 1 -- 172.21.15.165:0/3594403448 learned_addr learned my addr 172.21.15.165:0/3594403448 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9b7400640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf06f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf13210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c4a00640 1 -- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 msgr2=0x55a8eaf06f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf06f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c4a00640 1 -- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee8dc0 msgr2=0x55a8eaf13210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf13210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c4a00640 1 -- 172.21.15.165:0/3594403448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8eab223e0 con 0x55a8eaf42f90 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9b7400640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf13210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:48.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf03800 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800df70 tx=0x7fd9b8001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c671df8da3a6a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9acc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9acc00640 1 -- 172.21.15.165:0/3594403448 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8022030 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9acc00640 1 -- 172.21.15.165:0/3594403448 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd9b8001830 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.829+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9b8003d80 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9acc00640 1 -- 172.21.15.165:0/3594403448 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b802e050 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 10 client.6921 fetch_fsmap learned FSMap version 47 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55a8eac8f280 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9acc00640 1 -- 172.21.15.165:0/3594403448 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b8003d80 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9acc00640 1 -- 172.21.15.165:0/3594403448 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9b8031020 con 0x55a8eaf42f90 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 10 client.6921 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 10 client.6921 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 20 client.6921 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaf05070 0x55a8eaf308f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 4 client.6921 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:07:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.830+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 --> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a8eae5b310 con 0x55a8eaf05070 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.831+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaf05070 0x55a8eaf308f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.831+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaf05070 0x55a8eaf308f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e54889e4e58b86df server_cookie=799958421e7f4fb3 in_seq=0 out_seq=0 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.831+0000 7fd9acc00640 10 client.6921 ms_handle_connect on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9acc00640 1 -- 172.21.15.165:0/3594403448 <== mds.0 v2:172.21.15.165:6836/148834032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a8eae5b310 con 0x55a8eaf05070 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9acc00640 10 client.6921 handle_command_reply: tid=0 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 client.6921 shutdown 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 msgr2=0x55a8eaf03800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf03800 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd9b800df70 tx=0x7fd9b8001330 comp rx=0 tx=0).stop 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 shutdown_connections 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf03800 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf06f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaf05070 0x55a8eaf308f0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3594403448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf13210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 >> 172.21.15.165:0/3594403448 conn(0x55a8eaee4b10 msgr2=0x55a8eaeea770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9acc00640 0 client.6921 ms_handle_reset on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 shutdown_connections 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3594403448 wait complete. 2024-04-25T10:07:48.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.832+0000 7fd9c97ee740 20 client.6921 trim_cache size 0 max 16384 2024-04-25T10:07:48.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.849+0000 7fd9c97ee740 1 Processor -- start 2024-04-25T10:07:48.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.849+0000 7fd9c97ee740 1 -- start start 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf07d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf311a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaf42f90 0x55a8eaf316e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c0200860 con 0x55a8eaee8dc0 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01ff9c0 con 0x55a8eaf42f90 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c00c9cb0 con 0x55a8eaee9c80 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf07d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf07d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55076/0 (socket says 172.21.15.165:55076) 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.850+0000 7fd9bfe00640 1 -- 172.21.15.165:0/3020145808 learned_addr learned my addr 172.21.15.165:0/3020145808 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:48.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9b7400640 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf311a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaf42f90 0x55a8eaf316e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9bfe00640 1 -- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 msgr2=0x55a8eaf311a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf311a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9bfe00640 1 -- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaf42f90 msgr2=0x55a8eaf316e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaf42f90 0x55a8eaf316e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9bfe00640 1 -- 172.21.15.165:0/3020145808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8eaafb560 con 0x55a8eaee8dc0 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf07d90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00793b0 tx=0x7fd9b00a9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bcd2844a4ea681f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9af400640 1 -- 172.21.15.165:0/3020145808 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b00a7070 con 0x55a8eaee8dc0 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9af400640 1 -- 172.21.15.165:0/3020145808 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0077750 con 0x55a8eaee8dc0 2024-04-25T10:07:48.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.851+0000 7fd9af400640 1 -- 172.21.15.165:0/3020145808 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0091030 con 0x55a8eaee8dc0 2024-04-25T10:07:48.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3020145808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 msgr2=0x55a8eaf07d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf07d90 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd9b00793b0 tx=0x7fd9b00a9330 comp rx=0 tx=0).stop 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3020145808 shutdown_connections 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaf42f90 0x55a8eaf316e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee9c80 0x55a8eaf311a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/3020145808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaee8dc0 0x55a8eaf07d90 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3020145808 >> 172.21.15.165:0/3020145808 conn(0x55a8eaee4b10 msgr2=0x55a8eaee5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3020145808 shutdown_connections 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.852+0000 7fd9c97ee740 1 -- 172.21.15.165:0/3020145808 wait complete. 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.853+0000 7fd9c97ee740 1 -- start start 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.853+0000 7fd9c97ee740 1 Processor -- start 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee8dc0 0x55a8eaeea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf0b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9c01ff510 con 0x55a8eaf42f90 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8eaf15420 con 0x55a8eaee9c80 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8eaf2fef0 con 0x55a8eaee8dc0 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c97ee740 10 client.0 _ll_register_callbacks cb 0x7fd9b8001520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf0b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf0b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55086/0 (socket says 172.21.15.165:55086) 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.854+0000 7fd9c4a00640 1 -- 172.21.15.165:0/611982129 learned_addr learned my addr 172.21.15.165:0/611982129 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9b7400640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee8dc0 0x55a8eaeea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9c4a00640 1 -- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee8dc0 msgr2=0x55a8eaeea060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee8dc0 0x55a8eaeea060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9c4a00640 1 -- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 msgr2=0x55a8eaeea5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9c4a00640 1 -- 172.21.15.165:0/611982129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8eaf055e0 con 0x55a8eaf42f90 2024-04-25T10:07:48.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee8dc0 0x55a8eaeea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:48.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9c4a00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf0b390 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007cc0 tx=0x7fd9b8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdbf248428f84ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:48.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9acc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:07:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.855+0000 7fd9acc00640 1 -- 172.21.15.165:0/611982129 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b8036020 con 0x55a8eaf42f90 2024-04-25T10:07:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9acc00640 1 -- 172.21.15.165:0/611982129 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd9b80051f0 con 0x55a8eaf42f90 2024-04-25T10:07:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd9b0077750 con 0x55a8eaf42f90 2024-04-25T10:07:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9acc00640 1 -- 172.21.15.165:0/611982129 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd9b8040050 con 0x55a8eaf42f90 2024-04-25T10:07:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 10 client.6927 fetch_fsmap learned FSMap version 47 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=2+}) v3 -- 0x55a8eaf33bd0 con 0x55a8eaf42f90 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9acc00640 1 -- 172.21.15.165:0/611982129 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b0077750 con 0x55a8eaf42f90 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9acc00640 1 -- 172.21.15.165:0/611982129 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 47) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd9b8043020 con 0x55a8eaf42f90 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 10 client.6927 fetch_fsmap finished waiting for FSMap version 47 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 10 client.6927 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.c 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 20 client.6927 populate_metadata read hostname 'smithi165' 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaee3df0 0x55a8eaf0b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 4 client.6927 mds_command: new command op to 6870 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}] 2024-04-25T10:07:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 --> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_repair_path", "scrubops": ["repair"]}) v1 -- 0x55a8eae55660 con 0x55a8eaee3df0 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.856+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaee3df0 0x55a8eaf0b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.857+0000 7fd9bfe00640 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaee3df0 0x55a8eaf0b8d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42c67414daf2370 server_cookie=af7eae1c8db9aaf1 in_seq=0 out_seq=0 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.857+0000 7fd9acc00640 10 client.6927 ms_handle_connect on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.857+0000 7fd9acc00640 1 -- 172.21.15.165:0/611982129 <== mds.0 v2:172.21.15.165:6836/148834032 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55a8eae55660 con 0x55a8eaee3df0 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.857+0000 7fd9acc00640 10 client.6927 handle_command_reply: tid=0 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.857+0000 7fd9c97ee740 1 client.6927 shutdown 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 msgr2=0x55a8eaf0b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf0b390 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fd9b8007cc0 tx=0x7fd9b8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 shutdown_connections 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a8eaf42f90 0x55a8eaf0b390 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.165:6836/148834032,v1:172.21.15.165:6837/148834032] conn(0x55a8eaee3df0 0x55a8eaf0b8d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a8eaee9c80 0x55a8eaeea5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 --2- 172.21.15.165:0/611982129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a8eaee8dc0 0x55a8eaeea060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 >> 172.21.15.165:0/611982129 conn(0x55a8eaee4b10 msgr2=0x55a8eaf130c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9acc00640 0 client.6927 ms_handle_reset on v2:172.21.15.165:6836/148834032 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 shutdown_connections 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 1 -- 172.21.15.165:0/611982129 wait complete. 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.858+0000 7fd9c97ee740 20 client.6927 trim_cache size 0 max 16384 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "cf915098-5bf7-41f0-ab47-3d73038d0a00", 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:07:48.869 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:07:48.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 msgr2=0x7fd99c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fd9c01f7120 tx=0x7fd9a8009040 comp rx=0 tx=0).stop 2024-04-25T10:07:48.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 msgr2=0x7fd9c01f4650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c01f4650 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009870 tx=0x7fd9b00046c0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 shutdown_connections 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd99c06b4d0 0x7fd99c06d990 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9c015b980 0x7fd9c0144cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9c00cdb20 0x7fd9c0154640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 --2- 172.21.15.165:0/1552636029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd9c00cc4f0 0x7fd9c01f4650 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 >> 172.21.15.165:0/1552636029 conn(0x7fd9c006abe0 msgr2=0x7fd9c015b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 shutdown_connections 2024-04-25T10:07:48.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:48.859+0000 7fd9c6a00640 1 -- 172.21.15.165:0/1552636029 wait complete. 2024-04-25T10:07:48.884 INFO:tasks.cephfs.test_scrub_checks:command 'scrub start /scrub_repair_path repair' returned '{'return_code': 0, 'scrub_tag': 'cf915098-5bf7-41f0-ab47-3d73038d0a00', 'mode': 'asynchronous'}' 2024-04-25T10:07:57.864 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:57.872 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:07:58.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:07:58.886 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir scrub_repair_path) 2024-04-25T10:07:58.948 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:07:58.948 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:59.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:07:59.155 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:07:59.155 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:59.155 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:59.214 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:07:59.214 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:07:59.269 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:07:59.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.461+0000 7f9025000640 1 Processor -- start 2024-04-25T10:07:59.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.461+0000 7f9025000640 1 -- start start 2024-04-25T10:07:59.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9025000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9020144440 0x7f9020070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9025000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9020071530 0x7f90200cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9025000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902015bf30 con 0x7f90200cd860 2024-04-25T10:07:59.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9025000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902015c860 con 0x7f9020144440 2024-04-25T10:07:59.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9025000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9020071970 con 0x7f9020071530 2024-04-25T10:07:59.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f90200cd860 0x7f902006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47462/0 (socket says 172.21.15.165:47462) 2024-04-25T10:07:59.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901fe00640 1 -- 172.21.15.165:0/3848130218 learned_addr learned my addr 172.21.15.165:0/3848130218 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9016a00640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9020071530 0x7f90200cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901f400640 1 --2- 172.21.15.165:0/3848130218 >> v2:172.21.15.172:3300/0 conn(0x7f9020144440 0x7f9020070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1711281218 0 0) 0x7f902015bf30 con 0x7f90200cd860 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9004003520 con 0x7f90200cd860 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3998056043 0 0) 0x7f9004003520 con 0x7f90200cd860 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9020071530 msgr2=0x7f90200cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9020071530 0x7f90200cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 >> v2:172.21.15.172:3300/0 conn(0x7f9020144440 msgr2=0x7f9020070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 --2- 172.21.15.165:0/3848130218 >> v2:172.21.15.172:3300/0 conn(0x7f9020144440 0x7f9020070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902015c210 con 0x7f90200cd860 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f90180019f0 con 0x7f90200cd860 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 >> v1:172.21.15.165:6789/0 conn(0x7f90200cd860 legacy=0x7f902006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90040039b0 0x7f9004003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9004004800 0x7f9004004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040051c0 0x7f900400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902015c860 con 0x7f90040039b0 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020071970 con 0x7f9004004800 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900400bf80 con 0x7f90040051c0 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f9016a00640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9020071530 0x7f90200cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.463+0000 7f901f400640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90040039b0 0x7f9004003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.464+0000 7f901f400640 1 -- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040051c0 msgr2=0x7f900400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.464+0000 7f901f400640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040051c0 0x7f900400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.464+0000 7f901f400640 1 -- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9004004800 msgr2=0x7f9004004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.464+0000 7f901f400640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9004004800 0x7f9004004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.464+0000 7f901f400640 1 -- 172.21.15.165:0/3848130218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004003520 con 0x7f90040039b0 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.465+0000 7f901f400640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90040039b0 0x7f9004003dd0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9010009ac0 tx=0x7f9004005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30d41d02a82bae8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.465+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010036070 con 0x7f90040039b0 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.465+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9010031400 con 0x7f90040039b0 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.465+0000 7f901ea00640 1 -- 172.21.15.165:0/3848130218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9010009690 con 0x7f90040039b0 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f9025000640 1 -- 172.21.15.165:0/3848130218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90040039b0 msgr2=0x7f9004003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f9025000640 1 --2- 172.21.15.165:0/3848130218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90040039b0 0x7f9004003dd0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f9010009ac0 tx=0x7f9004005830 comp rx=0 tx=0).stop 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f901fe00640 1 -- 172.21.15.165:0/3848130218 reap_dead start 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f9025000640 1 -- 172.21.15.165:0/3848130218 shutdown_connections 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f9025000640 1 -- 172.21.15.165:0/3848130218 >> 172.21.15.165:0/3848130218 conn(0x7f902006abe0 msgr2=0x7f902006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f9025000640 1 -- 172.21.15.165:0/3848130218 shutdown_connections 2024-04-25T10:07:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.466+0000 7f9025000640 1 -- 172.21.15.165:0/3848130218 wait complete. 2024-04-25T10:07:59.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.468+0000 7f9025000640 1 Processor -- start 2024-04-25T10:07:59.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.468+0000 7f9025000640 1 -- start start 2024-04-25T10:07:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9025000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040039b0 0x7f90201f37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9025000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 0x7f902013cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9025000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90201562f0 0x7f90201567a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9025000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200cdcd0 con 0x7f90201385d0 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9025000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902015bc30 con 0x7f90201562f0 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9025000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90200cde50 con 0x7f90040039b0 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9016a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 0x7f902013cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9016a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 0x7f902013cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45322/0 (socket says 172.21.15.165:45322) 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.469+0000 7f9016a00640 1 -- 172.21.15.165:0/3968220763 learned_addr learned my addr 172.21.15.165:0/3968220763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f901f400640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040039b0 0x7f90201f37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9016a00640 1 -- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040039b0 msgr2=0x7f90201f37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9016a00640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040039b0 0x7f90201f37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9016a00640 1 -- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90201562f0 msgr2=0x7f90201567a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9016a00640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90201562f0 0x7f90201567a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9016a00640 1 -- 172.21.15.165:0/3968220763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90201f3d00 con 0x7f90201385d0 2024-04-25T10:07:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9016a00640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 0x7f902013cf30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f900800e5a0 tx=0x7f900800ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f900800f040 con 0x7f90201385d0 2024-04-25T10:07:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9008004250 con 0x7f90201385d0 2024-04-25T10:07:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9008003610 con 0x7f90201385d0 2024-04-25T10:07:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9020138ab0 con 0x7f90201385d0 2024-04-25T10:07:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.470+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9020139070 con 0x7f90201385d0 2024-04-25T10:07:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.472+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902015bf30 con 0x7f90201385d0 2024-04-25T10:07:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.473+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90080037b0 con 0x7f90201385d0 2024-04-25T10:07:59.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.473+0000 7f9017e00640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ffc06b500 0x7f8ffc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.474+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f900808a0a0 con 0x7f90201385d0 2024-04-25T10:07:59.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.474+0000 7f901f400640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ffc06b500 0x7f8ffc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.474+0000 7f901f400640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ffc06b500 0x7f8ffc06d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9010009ac0 tx=0x7f9010008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.479+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f902015bf30 con 0x7f90201385d0 2024-04-25T10:07:59.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.677+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9020143c20 con 0x7f90201385d0 2024-04-25T10:07:59.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.679+0000 7f9017e00640 1 -- 172.21.15.165:0/3968220763 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+17243 (secure 0 0 0) 0x7f9020143c20 con 0x7f90201385d0 2024-04-25T10:07:59.686 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:07:59.686 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:47.141502+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/1883921643":"2024-04-26T10:07:41.073778+0000","172.21.15.165:6835/1883921643":"2024-04-26T10:07:41.073778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:07:59.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ffc06b500 msgr2=0x7f8ffc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ffc06b500 0x7f8ffc06d9c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f9010009ac0 tx=0x7f9010008030 comp rx=0 tx=0).stop 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 msgr2=0x7f902013cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 0x7f902013cf30 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f900800e5a0 tx=0x7f900800ea60 comp rx=0 tx=0).stop 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 shutdown_connections 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ffc06b500 0x7f8ffc06d9c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90201562f0 0x7f90201567a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90201385d0 0x7f902013cf30 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 --2- 172.21.15.165:0/3968220763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90040039b0 0x7f90201f37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 >> 172.21.15.165:0/3968220763 conn(0x7f902006abe0 msgr2=0x7f902006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 shutdown_connections 2024-04-25T10:07:59.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.682+0000 7f9025000640 1 -- 172.21.15.165:0/3968220763 wait complete. 2024-04-25T10:07:59.707 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:25.839490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:27.018634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:07:59.707 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:07:59.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 Processor -- start 2024-04-25T10:07:59.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 -- start start 2024-04-25T10:07:59.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110144d50 0x7f911006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91101452a0 0x7f911006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91100cccc0 con 0x7f91101446d0 2024-04-25T10:07:59.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91100b2b20 con 0x7f91101452a0 2024-04-25T10:07:59.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.902+0000 7f9116800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911006d400 con 0x7f9110144d50 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91101452a0 0x7f911006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91101452a0 0x7f911006ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45324/0 (socket says 172.21.15.165:45324) 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 -- 172.21.15.165:0/1307509201 learned_addr learned my addr 172.21.15.165:0/1307509201 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 -- 172.21.15.165:0/1307509201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110144d50 msgr2=0x7f911006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910ea00640 1 --2- 172.21.15.165:0/1307509201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110144d50 0x7f911006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 --2- 172.21.15.165:0/1307509201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110144d50 0x7f911006d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 -- 172.21.15.165:0/1307509201 >> v1:172.21.15.172:6789/0 conn(0x7f91101446d0 legacy=0x7f91100b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:07:59.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.903+0000 7f910fe00640 1 -- 172.21.15.165:0/1307509201 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100cc140 con 0x7f91101452a0 2024-04-25T10:07:59.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.904+0000 7f910fe00640 1 --2- 172.21.15.165:0/1307509201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101452a0 0x7f911006ddd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9108009e50 tx=0x7f910802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5766ee656fc03a17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f910e000640 1 -- 172.21.15.165:0/1307509201 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108036070 con 0x7f91101452a0 2024-04-25T10:07:59.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f910e000640 1 -- 172.21.15.165:0/1307509201 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9108031600 con 0x7f91101452a0 2024-04-25T10:07:59.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f910e000640 1 -- 172.21.15.165:0/1307509201 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080319d0 con 0x7f91101452a0 2024-04-25T10:07:59.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f9116800640 1 -- 172.21.15.165:0/1307509201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101452a0 msgr2=0x7f911006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f9116800640 1 --2- 172.21.15.165:0/1307509201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101452a0 0x7f911006ddd0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f9108009e50 tx=0x7f910802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f9116800640 1 -- 172.21.15.165:0/1307509201 shutdown_connections 2024-04-25T10:07:59.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.905+0000 7f9116800640 1 --2- 172.21.15.165:0/1307509201 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101452a0 0x7f911006ddd0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.906+0000 7f9116800640 1 --2- 172.21.15.165:0/1307509201 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110144d50 0x7f911006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.906+0000 7f9116800640 1 -- 172.21.15.165:0/1307509201 >> 172.21.15.165:0/1307509201 conn(0x7f911006abe0 msgr2=0x7f911006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:07:59.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.906+0000 7f9116800640 1 -- 172.21.15.165:0/1307509201 shutdown_connections 2024-04-25T10:07:59.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.906+0000 7f9116800640 1 -- 172.21.15.165:0/1307509201 wait complete. 2024-04-25T10:07:59.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.907+0000 7f9116800640 1 Processor -- start 2024-04-25T10:07:59.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 -- start start 2024-04-25T10:07:59.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 0x7f91100cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110144d50 0x7f91101f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91101452a0 0x7f91101f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100cc9e0 con 0x7f91101446d0 2024-04-25T10:07:59.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100cc700 con 0x7f9110144d50 2024-04-25T10:07:59.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.908+0000 7f9116800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9110144ab0 con 0x7f91101452a0 2024-04-25T10:07:59.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 0x7f91100cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 0x7f91100cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45328/0 (socket says 172.21.15.165:45328) 2024-04-25T10:07:59.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 -- 172.21.15.165:0/3828784927 learned_addr learned my addr 172.21.15.165:0/3828784927 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910ea00640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110144d50 0x7f91101f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 -- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91101452a0 msgr2=0x7f91101f40c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910fe00640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91101452a0 0x7f91101f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91101452a0 0x7f91101f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 -- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110144d50 msgr2=0x7f91101f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110144d50 0x7f91101f3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910f400640 1 -- 172.21.15.165:0/3828784927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91100cdbf0 con 0x7f91101446d0 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.909+0000 7f910ea00640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110144d50 0x7f91101f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.910+0000 7f910f400640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 0x7f91100cd2e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f910000c430 tx=0x7f910000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.910+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100011790 con 0x7f91101446d0 2024-04-25T10:07:59.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.910+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100017070 con 0x7f91101446d0 2024-04-25T10:07:59.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.910+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9100011b70 con 0x7f91101446d0 2024-04-25T10:07:59.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.910+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91100cdeb0 con 0x7f91101446d0 2024-04-25T10:07:59.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.910+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91100ce470 con 0x7f91101446d0 2024-04-25T10:07:59.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.911+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110073a00 con 0x7f91101446d0 2024-04-25T10:07:59.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.912+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9100011d10 con 0x7f91101446d0 2024-04-25T10:07:59.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.913+0000 7f9107400640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:07:59.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.913+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f910008a2d0 con 0x7f91101446d0 2024-04-25T10:07:59.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.913+0000 7f910ea00640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:07:59.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.914+0000 7f910ea00640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f911013ced0 tx=0x7f90fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:07:59.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:07:59.917+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9100058ef0 con 0x7f91101446d0 2024-04-25T10:08:00.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.113+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91100affc0 con 0x7f91101446d0 2024-04-25T10:08:00.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.114+0000 7f9107400640 1 -- 172.21.15.165:0/3828784927 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91000587f0 con 0x7f91101446d0 2024-04-25T10:08:00.121 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.116+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 msgr2=0x7f90f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.116+0000 7f9116800640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f911013ced0 tx=0x7f90fc008040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.116+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 msgr2=0x7f91100cd2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.116+0000 7f9116800640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 0x7f91100cd2e0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f910000c430 tx=0x7f910000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 shutdown_connections 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91101452a0 0x7f91101f40c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9110144d50 0x7f91101f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 --2- 172.21.15.165:0/3828784927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91101446d0 0x7f91100cd2e0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 >> 172.21.15.165:0/3828784927 conn(0x7f911006abe0 msgr2=0x7f9110071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 shutdown_connections 2024-04-25T10:08:00.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.117+0000 7f9116800640 1 -- 172.21.15.165:0/3828784927 wait complete. 2024-04-25T10:08:00.145 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:00.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.346+0000 7fdd97800640 1 Processor -- start 2024-04-25T10:08:00.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.347+0000 7fdd97800640 1 -- start start 2024-04-25T10:08:00.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.348+0000 7fdd97800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdd900cc4f0 0x7fdd900cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.348+0000 7fdd97800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd900cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.348+0000 7fdd97800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdd9015b270 0x7fdd9015b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.348+0000 7fdd97800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd900ce4c0 con 0x7fdd900cc4f0 2024-04-25T10:08:00.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.348+0000 7fdd97800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd90071270 con 0x7fdd9015b270 2024-04-25T10:08:00.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.348+0000 7fdd97800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90072280 con 0x7fdd900cdb20 2024-04-25T10:08:00.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdd900cc4f0 0x7fdd900cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdd900cc4f0 0x7fdd900cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45336/0 (socket says 172.21.15.165:45336) 2024-04-25T10:08:00.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 -- 172.21.15.165:0/4106167761 learned_addr learned my addr 172.21.15.165:0/4106167761 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd8fe00640 1 --2- 172.21.15.165:0/4106167761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd900cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd95800640 1 --2- 172.21.15.165:0/4106167761 >> v2:172.21.15.172:3300/0 conn(0x7fdd9015b270 0x7fdd9015b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 -- 172.21.15.165:0/4106167761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 msgr2=0x7fdd900cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 --2- 172.21.15.165:0/4106167761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd900cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 -- 172.21.15.165:0/4106167761 >> v2:172.21.15.172:3300/0 conn(0x7fdd9015b270 msgr2=0x7fdd9015b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 --2- 172.21.15.165:0/4106167761 >> v2:172.21.15.172:3300/0 conn(0x7fdd9015b270 0x7fdd9015b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.349+0000 7fdd94e00640 1 -- 172.21.15.165:0/4106167761 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd900714c0 con 0x7fdd900cc4f0 2024-04-25T10:08:00.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd94e00640 1 --2- 172.21.15.165:0/4106167761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd900cc4f0 0x7fdd900cc8d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fdd90158aa0 tx=0x7fdd8402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38ad36fe05369cdb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd8f400640 1 -- 172.21.15.165:0/4106167761 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84037070 con 0x7fdd900cc4f0 2024-04-25T10:08:00.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd8f400640 1 -- 172.21.15.165:0/4106167761 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd84032440 con 0x7fdd900cc4f0 2024-04-25T10:08:00.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd8f400640 1 -- 172.21.15.165:0/4106167761 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84032810 con 0x7fdd900cc4f0 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 -- 172.21.15.165:0/4106167761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd900cc4f0 msgr2=0x7fdd900cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 --2- 172.21.15.165:0/4106167761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd900cc4f0 0x7fdd900cc8d0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fdd90158aa0 tx=0x7fdd8402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 -- 172.21.15.165:0/4106167761 shutdown_connections 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 --2- 172.21.15.165:0/4106167761 >> v2:172.21.15.172:3300/0 conn(0x7fdd9015b270 0x7fdd9015b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 --2- 172.21.15.165:0/4106167761 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd900cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 --2- 172.21.15.165:0/4106167761 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd900cc4f0 0x7fdd900cc8d0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 -- 172.21.15.165:0/4106167761 >> 172.21.15.165:0/4106167761 conn(0x7fdd9006abe0 msgr2=0x7fdd9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 -- 172.21.15.165:0/4106167761 shutdown_connections 2024-04-25T10:08:00.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.352+0000 7fdd97800640 1 -- 172.21.15.165:0/4106167761 wait complete. 2024-04-25T10:08:00.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.353+0000 7fdd97800640 1 Processor -- start 2024-04-25T10:08:00.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.354+0000 7fdd97800640 1 -- start start 2024-04-25T10:08:00.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.354+0000 7fdd97800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd90149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd97800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9015b270 0x7fdd90168160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd97800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 0x7fdd9015d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd97800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90071d60 con 0x7fdd90153420 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd97800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd90071a80 con 0x7fdd9015b270 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd97800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9015bbc0 con 0x7fdd900cdb20 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 0x7fdd9015d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 0x7fdd9015d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45340/0 (socket says 172.21.15.165:45340) 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 -- 172.21.15.165:0/1606373475 learned_addr learned my addr 172.21.15.165:0/1606373475 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 -- 172.21.15.165:0/1606373475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 msgr2=0x7fdd90149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd90149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 -- 172.21.15.165:0/1606373475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9015b270 msgr2=0x7fdd90168160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9015b270 0x7fdd90168160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.355+0000 7fdd95800640 1 -- 172.21.15.165:0/1606373475 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd90152380 con 0x7fdd90153420 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.356+0000 7fdd95800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 0x7fdd9015d0e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fdd88007e70 tx=0x7fdd8800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.356+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88014420 con 0x7fdd90153420 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.356+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd88019070 con 0x7fdd90153420 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.356+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd90152660 con 0x7fdd90153420 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.356+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88014820 con 0x7fdd90153420 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.356+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd90152c20 con 0x7fdd90153420 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.357+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd90073a00 con 0x7fdd90153420 2024-04-25T10:08:00.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.358+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd880149c0 con 0x7fdd90153420 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.359+0000 7fdd8cc00640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd7006b500 0x7fdd7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.359+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fdd8808ad00 con 0x7fdd90153420 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.359+0000 7fdd94e00640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd7006b500 0x7fdd7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.360+0000 7fdd94e00640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd7006b500 0x7fdd7006d9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdd84034000 tx=0x7fdd8402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.364+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd88059920 con 0x7fdd90153420 2024-04-25T10:08:00.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.557+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd9015b980 con 0x7fdd90153420 2024-04-25T10:08:00.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.558+0000 7fdd8cc00640 1 -- 172.21.15.165:0/1606373475 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdd88059220 con 0x7fdd90153420 2024-04-25T10:08:00.564 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.560+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd7006b500 msgr2=0x7fdd7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd7006b500 0x7fdd7006d9c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdd84034000 tx=0x7fdd8402c000 comp rx=0 tx=0).stop 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 msgr2=0x7fdd9015d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 0x7fdd9015d0e0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fdd88007e70 tx=0x7fdd8800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 shutdown_connections 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd7006b500 0x7fdd7006d9c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd90153420 0x7fdd9015d0e0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd9015b270 0x7fdd90168160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 --2- 172.21.15.165:0/1606373475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd900cdb20 0x7fdd90149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 >> 172.21.15.165:0/1606373475 conn(0x7fdd9006abe0 msgr2=0x7fdd90159e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 shutdown_connections 2024-04-25T10:08:00.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.561+0000 7fdd97800640 1 -- 172.21.15.165:0/1606373475 wait complete. 2024-04-25T10:08:00.585 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.772+0000 7f10ab600640 1 Processor -- start 2024-04-25T10:08:00.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.772+0000 7f10ab600640 1 -- start start 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10ab600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f10a4144d50 0x7f10a406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10ab600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40cccc0 con 0x7f10a41452a0 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10ab600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a40cc140 con 0x7f10a41446d0 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10ab600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a40b2b20 con 0x7f10a4144d50 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10a3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f10a4144d50 0x7f10a406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10a3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f10a4144d50 0x7f10a406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45348/0 (socket says 172.21.15.165:45348) 2024-04-25T10:08:00.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.774+0000 7f10a3e00640 1 -- 172.21.15.165:0/4016114004 learned_addr learned my addr 172.21.15.165:0/4016114004 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:00.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.775+0000 7f10a3e00640 1 -- 172.21.15.165:0/4016114004 >> v1:172.21.15.172:6789/0 conn(0x7f10a41446d0 legacy=0x7f10a40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:00.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.775+0000 7f10a3e00640 1 -- 172.21.15.165:0/4016114004 >> v1:172.21.15.165:6789/0 conn(0x7f10a41452a0 legacy=0x7f10a406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.775+0000 7f10a3e00640 1 -- 172.21.15.165:0/4016114004 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40cc420 con 0x7f10a4144d50 2024-04-25T10:08:00.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.776+0000 7f10a3e00640 1 --2- 172.21.15.165:0/4016114004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a4144d50 0x7f10a406d890 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1098009870 tx=0x7f109802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1bcd93263bf1823 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.776+0000 7f10a3400640 1 -- 172.21.15.165:0/4016114004 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098037070 con 0x7f10a4144d50 2024-04-25T10:08:00.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.776+0000 7f10a3400640 1 -- 172.21.15.165:0/4016114004 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098032440 con 0x7f10a4144d50 2024-04-25T10:08:00.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.776+0000 7f10a3400640 1 -- 172.21.15.165:0/4016114004 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098032810 con 0x7f10a4144d50 2024-04-25T10:08:00.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 -- 172.21.15.165:0/4016114004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a4144d50 msgr2=0x7f10a406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 --2- 172.21.15.165:0/4016114004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a4144d50 0x7f10a406d890 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f1098009870 tx=0x7f109802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 -- 172.21.15.165:0/4016114004 shutdown_connections 2024-04-25T10:08:00.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 --2- 172.21.15.165:0/4016114004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a4144d50 0x7f10a406d890 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 -- 172.21.15.165:0/4016114004 >> 172.21.15.165:0/4016114004 conn(0x7f10a406abe0 msgr2=0x7f10a406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 -- 172.21.15.165:0/4016114004 shutdown_connections 2024-04-25T10:08:00.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.777+0000 7f10ab600640 1 -- 172.21.15.165:0/4016114004 wait complete. 2024-04-25T10:08:00.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.779+0000 7f10ab600640 1 Processor -- start 2024-04-25T10:08:00.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.779+0000 7f10ab600640 1 -- start start 2024-04-25T10:08:00.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10ab600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10a41446d0 0x7f10a413f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10ab600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10a4144d50 0x7f10a40c9a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10ab600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 0x7f10a40c7680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10ab600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a40cc9e0 con 0x7f10a41452a0 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10ab600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a406d400 con 0x7f10a41446d0 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10ab600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a4144ab0 con 0x7f10a4144d50 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10a9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 0x7f10a40c7680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10a9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 0x7f10a40c7680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45350/0 (socket says 172.21.15.165:45350) 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10a9600640 1 -- 172.21.15.165:0/1396291030 learned_addr learned my addr 172.21.15.165:0/1396291030 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.780+0000 7f10a8c00640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10a41446d0 0x7f10a413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a3e00640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10a4144d50 0x7f10a40c9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a9600640 1 -- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10a4144d50 msgr2=0x7f10a40c9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a9600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10a4144d50 0x7f10a40c9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a9600640 1 -- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10a41446d0 msgr2=0x7f10a413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a9600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10a41446d0 0x7f10a413f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a9600640 1 -- 172.21.15.165:0/1396291030 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a40c5090 con 0x7f10a41452a0 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a3e00640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10a4144d50 0x7f10a40c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:00.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a9600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 0x7f10a40c7680 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c7d0 tx=0x7f109c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c003b70 con 0x7f10a41452a0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c003d30 con 0x7f10a41452a0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c00aa90 con 0x7f10a41452a0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.781+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a40c5370 con 0x7f10a41452a0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.782+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a40c5c10 con 0x7f10a41452a0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.783+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a40c5540 con 0x7f10a41452a0 2024-04-25T10:08:00.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.784+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f109c00ac30 con 0x7f10a41452a0 2024-04-25T10:08:00.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.784+0000 7f10a0c00640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b4d0 0x7f108806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:00.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.785+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f109c08a420 con 0x7f10a41452a0 2024-04-25T10:08:00.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.785+0000 7f10a8c00640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b4d0 0x7f108806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:00.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.786+0000 7f10a8c00640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b4d0 0x7f108806d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1094006fd0 tx=0x7f1094008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:00.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.789+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f109c058fb0 con 0x7f10a41452a0 2024-04-25T10:08:00.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.981+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10a4073a00 con 0x7f10a41452a0 2024-04-25T10:08:00.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.982+0000 7f10a0c00640 1 -- 172.21.15.165:0/1396291030 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f109c0588b0 con 0x7f10a41452a0 2024-04-25T10:08:00.988 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.984+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b4d0 msgr2=0x7f108806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.984+0000 7f10ab600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b4d0 0x7f108806d990 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1094006fd0 tx=0x7f1094008040 comp rx=0 tx=0).stop 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.984+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 msgr2=0x7f10a40c7680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.984+0000 7f10ab600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 0x7f10a40c7680 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f109c00c7d0 tx=0x7f109c00cc90 comp rx=0 tx=0).stop 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 shutdown_connections 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b4d0 0x7f108806d990 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10a41452a0 0x7f10a40c7680 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10a4144d50 0x7f10a40c9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 --2- 172.21.15.165:0/1396291030 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10a41446d0 0x7f10a413f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:00.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 >> 172.21.15.165:0/1396291030 conn(0x7f10a406abe0 msgr2=0x7f10a41401d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:00.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 shutdown_connections 2024-04-25T10:08:00.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:00.985+0000 7f10ab600640 1 -- 172.21.15.165:0/1396291030 wait complete. 2024-04-25T10:08:01.008 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:08:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.193+0000 7f241cc00640 1 Processor -- start 2024-04-25T10:08:01.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.193+0000 7f241cc00640 1 -- start start 2024-04-25T10:08:01.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.194+0000 7f241cc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2418145140 0x7f24180ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.194+0000 7f241cc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24180c5650 con 0x7f24180c5200 2024-04-25T10:08:01.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.194+0000 7f241cc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2418145520 con 0x7f24180ce950 2024-04-25T10:08:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.194+0000 7f241cc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24180ced90 con 0x7f2418145140 2024-04-25T10:08:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2418145140 0x7f24180ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2418145140 0x7f24180ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45362/0 (socket says 172.21.15.165:45362) 2024-04-25T10:08:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f24180c5200 0x7f241815bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47474/0 (socket says 172.21.15.165:47474) 2024-04-25T10:08:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417400640 1 -- 172.21.15.165:0/242052846 learned_addr learned my addr 172.21.15.165:0/242052846 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:01.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417400640 1 -- 172.21.15.165:0/242052846 >> v1:172.21.15.172:6789/0 conn(0x7f24180ce950 legacy=0x7f24180c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417400640 1 -- 172.21.15.165:0/242052846 >> v1:172.21.15.165:6789/0 conn(0x7f24180c5200 legacy=0x7f241815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.195+0000 7f2417400640 1 -- 172.21.15.165:0/242052846 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418071270 con 0x7f2418145140 2024-04-25T10:08:01.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.197+0000 7f2417400640 1 --2- 172.21.15.165:0/242052846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2418145140 0x7f24180ce410 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2408009870 tx=0x7f240802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27f9995847a29ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.197+0000 7f2416000640 1 -- 172.21.15.165:0/242052846 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408037070 con 0x7f2418145140 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.197+0000 7f2416000640 1 -- 172.21.15.165:0/242052846 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2408032440 con 0x7f2418145140 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.197+0000 7f2416000640 1 -- 172.21.15.165:0/242052846 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408032810 con 0x7f2418145140 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 -- 172.21.15.165:0/242052846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2418145140 msgr2=0x7f24180ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 --2- 172.21.15.165:0/242052846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2418145140 0x7f24180ce410 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2408009870 tx=0x7f240802a210 comp rx=0 tx=0).stop 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 -- 172.21.15.165:0/242052846 shutdown_connections 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 --2- 172.21.15.165:0/242052846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2418145140 0x7f24180ce410 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 -- 172.21.15.165:0/242052846 >> 172.21.15.165:0/242052846 conn(0x7f2418061cc0 msgr2=0x7f241806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 -- 172.21.15.165:0/242052846 shutdown_connections 2024-04-25T10:08:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.198+0000 7f241cc00640 1 -- 172.21.15.165:0/242052846 wait complete. 2024-04-25T10:08:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.200+0000 7f241cc00640 1 Processor -- start 2024-04-25T10:08:01.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.200+0000 7f241cc00640 1 -- start start 2024-04-25T10:08:01.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f241cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24180c5200 0x7f24180ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f241cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 0x7f24180b0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f241cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418145140 0x7f24180cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f241cc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2418071b10 con 0x7f24180ce950 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f241cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2418071830 con 0x7f2418145140 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f241cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241815c6c0 con 0x7f24180c5200 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 0x7f24180b0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 0x7f24180b0480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45368/0 (socket says 172.21.15.165:45368) 2024-04-25T10:08:01.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 -- 172.21.15.165:0/4009750483 learned_addr learned my addr 172.21.15.165:0/4009750483 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 -- 172.21.15.165:0/4009750483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24180c5200 msgr2=0x7f24180ad0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24180c5200 0x7f24180ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 -- 172.21.15.165:0/4009750483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418145140 msgr2=0x7f24180cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418145140 0x7f24180cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.201+0000 7f2416a00640 1 -- 172.21.15.165:0/4009750483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24180b0f20 con 0x7f24180ce950 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.202+0000 7f2416a00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 0x7f24180b0480 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f240400c430 tx=0x7f240400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.202+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404011790 con 0x7f24180ce950 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.202+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2404017070 con 0x7f24180ce950 2024-04-25T10:08:01.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.202+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24180b1200 con 0x7f24180ce950 2024-04-25T10:08:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.202+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2404011bb0 con 0x7f24180ce950 2024-04-25T10:08:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.202+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24180b17c0 con 0x7f24180ce950 2024-04-25T10:08:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.203+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2418073990 con 0x7f24180ce950 2024-04-25T10:08:01.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.204+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f240401c460 con 0x7f24180ce950 2024-04-25T10:08:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.206+0000 7f240f400640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f23f806b4d0 0x7f23f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.206+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f240408a350 con 0x7f24180ce950 2024-04-25T10:08:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.206+0000 7f2417400640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f23f806b4d0 0x7f23f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.206+0000 7f2417400640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f23f806b4d0 0x7f23f806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2408034000 tx=0x7f240802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.210+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2404058f70 con 0x7f24180ce950 2024-04-25T10:08:01.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.422+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f24180cef40 con 0x7f24180ce950 2024-04-25T10:08:01.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.423+0000 7f240f400640 1 -- 172.21.15.165:0/4009750483 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v74) v1 ==== 80+0+124 (secure 0 0 0) 0x7f2404058870 con 0x7f24180ce950 2024-04-25T10:08:01.441 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165:6837/1883921643 2024-04-26T10:07:41.073778+0000 2024-04-25T10:08:01.441 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.165:6835/1883921643 2024-04-26T10:07:41.073778+0000 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:listed 2 entries 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.425+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f23f806b4d0 msgr2=0x7f23f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.425+0000 7f241cc00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f23f806b4d0 0x7f23f806d990 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f2408034000 tx=0x7f240802c000 comp rx=0 tx=0).stop 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.425+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 msgr2=0x7f24180b0480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 0x7f24180b0480 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f240400c430 tx=0x7f240400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 shutdown_connections 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f23f806b4d0 0x7f23f806d990 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2418145140 0x7f24180cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24180ce950 0x7f24180b0480 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 --2- 172.21.15.165:0/4009750483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24180c5200 0x7f24180ad0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 >> 172.21.15.165:0/4009750483 conn(0x7f2418061cc0 msgr2=0x7f24180c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 shutdown_connections 2024-04-25T10:08:01.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.426+0000 7f241cc00640 1 -- 172.21.15.165:0/4009750483 wait complete. 2024-04-25T10:08:01.449 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:08:01.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.449 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:01.534 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:08:01.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:01.535 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:08:01.581 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:08:01.581 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:01.639 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:01.640 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:08:01.640 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:01.695 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:01.695 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:08:01.695 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:01.753 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:01.754 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:01.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.952+0000 7fb8d4a00640 1 Processor -- start 2024-04-25T10:08:01.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.952+0000 7fb8d4a00640 1 -- start start 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.953+0000 7fb8d4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d0154ee0 0x7fb8d00cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8d4a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8d00cfcf0 0x7fb8d00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8d4a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8d00ce3c0 0x7fb8d015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8d4a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d00ce830 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8d4a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d0072090 con 0x7fb8d00cfcf0 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8d4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d00d0150 con 0x7fb8d0154ee0 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8d00ce3c0 0x7fb8d015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8d00ce3c0 0x7fb8d015e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45384/0 (socket says 172.21.15.165:45384) 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 -- 172.21.15.165:0/3681962454 learned_addr learned my addr 172.21.15.165:0/3681962454 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8c5400640 1 --2- 172.21.15.165:0/3681962454 >> v2:172.21.15.172:3300/0 conn(0x7fb8d00cfcf0 0x7fb8d00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8cde00640 1 --2- 172.21.15.165:0/3681962454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d0154ee0 0x7fb8d00cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 -- 172.21.15.165:0/3681962454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d0154ee0 msgr2=0x7fb8d00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 --2- 172.21.15.165:0/3681962454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d0154ee0 0x7fb8d00cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 -- 172.21.15.165:0/3681962454 >> v2:172.21.15.172:3300/0 conn(0x7fb8d00cfcf0 msgr2=0x7fb8d00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:01.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 --2- 172.21.15.165:0/3681962454 >> v2:172.21.15.172:3300/0 conn(0x7fb8d00cfcf0 0x7fb8d00cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.954+0000 7fb8ce800640 1 -- 172.21.15.165:0/3681962454 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0071480 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.955+0000 7fb8ce800640 1 --2- 172.21.15.165:0/3681962454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d015e570 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00cd350 tx=0x7fb8c802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9405a52fad2b719b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8cd400640 1 -- 172.21.15.165:0/3681962454 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8036070 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8cd400640 1 -- 172.21.15.165:0/3681962454 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c8031600 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8cd400640 1 -- 172.21.15.165:0/3681962454 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c80319d0 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 -- 172.21.15.165:0/3681962454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 msgr2=0x7fb8d015e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/3681962454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d015e570 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7fb8d00cd350 tx=0x7fb8c802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 -- 172.21.15.165:0/3681962454 shutdown_connections 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/3681962454 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d015e570 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/3681962454 >> v2:172.21.15.172:3300/0 conn(0x7fb8d00cfcf0 0x7fb8d00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/3681962454 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d0154ee0 0x7fb8d00cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.956+0000 7fb8d4a00640 1 -- 172.21.15.165:0/3681962454 >> 172.21.15.165:0/3681962454 conn(0x7fb8d006abe0 msgr2=0x7fb8d006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.957+0000 7fb8d4a00640 1 -- 172.21.15.165:0/3681962454 shutdown_connections 2024-04-25T10:08:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.957+0000 7fb8d4a00640 1 -- 172.21.15.165:0/3681962454 wait complete. 2024-04-25T10:08:01.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.958+0000 7fb8d4a00640 1 Processor -- start 2024-04-25T10:08:01.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.958+0000 7fb8d4a00640 1 -- start start 2024-04-25T10:08:01.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8d4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d0155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8d4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d00cfcf0 0x7fb8d0151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8d4a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0154ee0 0x7fb8d0158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8d4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0071a40 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8d4a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0071230 con 0x7fb8d0154ee0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8d4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d0071d20 con 0x7fb8d00cfcf0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.959+0000 7fb8cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d0155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d0155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45396/0 (socket says 172.21.15.165:45396) 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 -- 172.21.15.165:0/2738041773 learned_addr learned my addr 172.21.15.165:0/2738041773 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8ce800640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0154ee0 0x7fb8d0158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8c5400640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d00cfcf0 0x7fb8d0151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 -- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d00cfcf0 msgr2=0x7fb8d0151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d00cfcf0 0x7fb8d0151e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 -- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0154ee0 msgr2=0x7fb8d0158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0154ee0 0x7fb8d0158600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 -- 172.21.15.165:0/2738041773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d0156260 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8c5400640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d00cfcf0 0x7fb8d0151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:01.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.960+0000 7fb8cde00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d0155d20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c7f0 tx=0x7fb8c000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.961+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0015070 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.961+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0004220 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.961+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0007620 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.961+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d0157d60 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.961+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d0158320 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.962+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d0073990 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.963+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8c00079d0 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.964+0000 7fb8c6a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8ac06b4d0 0x7fb8ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:01.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.964+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb8c008a4a0 con 0x7fb8d00ce3c0 2024-04-25T10:08:01.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.964+0000 7fb8c5400640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8ac06b4d0 0x7fb8ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:01.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.965+0000 7fb8c5400640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8ac06b4d0 0x7fb8ac06d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8006fd0 tx=0x7fb8b8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:01.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:01.968+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8c0059030 con 0x7fb8d00ce3c0 2024-04-25T10:08:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.205+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8d015bc10 con 0x7fb8d00ce3c0 2024-04-25T10:08:02.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.207+0000 7fb8c6a00640 1 -- 172.21.15.165:0/2738041773 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fb8c0058930 con 0x7fb8d00ce3c0 2024-04-25T10:08:02.213 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:02.213 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:44.122864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 47 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8ac06b4d0 msgr2=0x7fb8ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8ac06b4d0 0x7fb8ac06d990 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fb8b8006fd0 tx=0x7fb8b8008040 comp rx=0 tx=0).stop 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 msgr2=0x7fb8d0155d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d0155d20 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c7f0 tx=0x7fb8c000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 shutdown_connections 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb8ac06b4d0 0x7fb8ac06d990 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8d0154ee0 0x7fb8d0158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8d00cfcf0 0x7fb8d0151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 --2- 172.21.15.165:0/2738041773 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8d00ce3c0 0x7fb8d0155d20 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.209+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 >> 172.21.15.165:0/2738041773 conn(0x7fb8d006abe0 msgr2=0x7fb8d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.210+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 shutdown_connections 2024-04-25T10:08:02.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.210+0000 7fb8d4a00640 1 -- 172.21.15.165:0/2738041773 wait complete. 2024-04-25T10:08:02.235 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:02.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.428+0000 7fb1d7600640 1 Processor -- start 2024-04-25T10:08:02.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.428+0000 7fb1d7600640 1 -- start start 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d015dd30 0x7fb1d00b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d7600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb1d00d0ad0 0x7fb1d00b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d7600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1d00d1df0 con 0x7fb1d00b9770 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d7600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1d00d20d0 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00d23b0 con 0x7fb1d015dd30 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb1d00d0ad0 0x7fb1d00b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb1d00d0ad0 0x7fb1d00b6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45400/0 (socket says 172.21.15.165:45400) 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.430+0000 7fb1d5600640 1 -- 172.21.15.165:0/2573712468 learned_addr learned my addr 172.21.15.165:0/2573712468 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.431+0000 7fb1d4c00640 1 --2- 172.21.15.165:0/2573712468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d015dd30 0x7fb1d00b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.431+0000 7fb1d5600640 1 -- 172.21.15.165:0/2573712468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d015dd30 msgr2=0x7fb1d00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.431+0000 7fb1d5600640 1 --2- 172.21.15.165:0/2573712468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d015dd30 0x7fb1d00b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.431+0000 7fb1d5600640 1 -- 172.21.15.165:0/2573712468 >> v1:172.21.15.172:6789/0 conn(0x7fb1d00b9770 legacy=0x7fb1d00d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.431+0000 7fb1d5600640 1 -- 172.21.15.165:0/2573712468 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00d2690 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.432+0000 7fb1d5600640 1 --2- 172.21.15.165:0/2573712468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00b6a40 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009e50 tx=0x7fb1c802a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded3da5b51fd6601 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.432+0000 7fb1cfe00640 1 -- 172.21.15.165:0/2573712468 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c8036070 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.432+0000 7fb1cfe00640 1 -- 172.21.15.165:0/2573712468 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c8031600 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.432+0000 7fb1cfe00640 1 -- 172.21.15.165:0/2573712468 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c80319d0 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 -- 172.21.15.165:0/2573712468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 msgr2=0x7fb1d00b6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2573712468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00b6a40 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fb1c8009e50 tx=0x7fb1c802a630 comp rx=0 tx=0).stop 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 -- 172.21.15.165:0/2573712468 shutdown_connections 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2573712468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00b6a40 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2573712468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d015dd30 0x7fb1d00b9230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 -- 172.21.15.165:0/2573712468 >> 172.21.15.165:0/2573712468 conn(0x7fb1d0061c30 msgr2=0x7fb1d006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 -- 172.21.15.165:0/2573712468 shutdown_connections 2024-04-25T10:08:02.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.433+0000 7fb1d7600640 1 -- 172.21.15.165:0/2573712468 wait complete. 2024-04-25T10:08:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.435+0000 7fb1d7600640 1 Processor -- start 2024-04-25T10:08:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.435+0000 7fb1d7600640 1 -- start start 2024-04-25T10:08:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d00b9770 0x7fb1d00c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d7600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d7600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1d015dd30 0x7fb1d015ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d7600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00d1b90 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d7600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00d1a10 con 0x7fb1d015dd30 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1d00d2e90 con 0x7fb1d00b9770 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45414/0 (socket says 172.21.15.165:45414) 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 -- 172.21.15.165:0/2285912426 learned_addr learned my addr 172.21.15.165:0/2285912426 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 -- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d00b9770 msgr2=0x7fb1d00c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d4c00640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d00b9770 0x7fb1d00c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1d5600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1d015dd30 0x7fb1d015ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d00b9770 0x7fb1d00c9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 -- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1d015dd30 msgr2=0x7fb1d015ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1d015dd30 0x7fb1d015ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.436+0000 7fb1c7e00640 1 -- 172.21.15.165:0/2285912426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d00cd460 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.437+0000 7fb1c7e00640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00cc4e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004640 tx=0x7fb1b800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.437+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8011790 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.437+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b8017070 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.437+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b8011bb0 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.437+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1d00ccbc0 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.437+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1d00cd180 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.438+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1d0073990 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.439+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb1b8011d50 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.440+0000 7fb1cd600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1ac06b4d0 0x7fb1ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.440+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb1b808a080 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.440+0000 7fb1d4c00640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1ac06b4d0 0x7fb1ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.441+0000 7fb1d4c00640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1ac06b4d0 0x7fb1ac06d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000b7c0 tx=0x7fb1c0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.444+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb1b8058c10 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.640+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1d00d14d0 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.642+0000 7fb1cd600640 1 -- 172.21.15.165:0/2285912426 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+17243 (secure 0 0 0) 0x7fb1b8058510 con 0x7fb1d00d0ad0 2024-04-25T10:08:02.649 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:02.649 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:47.141502+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/1883921643":"2024-04-26T10:07:41.073778+0000","172.21.15.165:6835/1883921643":"2024-04-26T10:07:41.073778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:02.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1ac06b4d0 msgr2=0x7fb1ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1ac06b4d0 0x7fb1ac06d990 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fb1c000b7c0 tx=0x7fb1c0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:02.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 msgr2=0x7fb1d00cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00cc4e0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fb1b8004640 tx=0x7fb1b800c8e0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 shutdown_connections 2024-04-25T10:08:02.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1ac06b4d0 0x7fb1ac06d990 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb1d015dd30 0x7fb1d015ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb1d00d0ad0 0x7fb1d00cc4e0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 --2- 172.21.15.165:0/2285912426 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb1d00b9770 0x7fb1d00c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.645+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 >> 172.21.15.165:0/2285912426 conn(0x7fb1d0061c30 msgr2=0x7fb1d00ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.646+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 shutdown_connections 2024-04-25T10:08:02.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.646+0000 7fb1d7600640 1 -- 172.21.15.165:0/2285912426 wait complete. 2024-04-25T10:08:02.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:07:25.839490+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '67', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:07:27.018634+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:02.671 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:02.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.875+0000 7fa1e5a00640 1 Processor -- start 2024-04-25T10:08:02.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.875+0000 7fa1e5a00640 1 -- start start 2024-04-25T10:08:02.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1e5a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa1e0144d50 0x7fa1e00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1e5a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e00cccc0 con 0x7fa1e01452a0 2024-04-25T10:08:02.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1e5a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e00b2b20 con 0x7fa1e0144d50 2024-04-25T10:08:02.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e006d400 con 0x7fa1e01446d0 2024-04-25T10:08:02.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.877+0000 7fa1dea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e00b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:41664/0 (socket says 172.21.15.165:41664) 2024-04-25T10:08:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.878+0000 7fa1dea00640 1 -- 172.21.15.165:0/2036435541 learned_addr learned my addr 172.21.15.165:0/2036435541 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.878+0000 7fa1d7e00640 1 --2- 172.21.15.165:0/2036435541 >> v2:172.21.15.172:3300/0 conn(0x7fa1e0144d50 0x7fa1e00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.879+0000 7fa1dea00640 1 -- 172.21.15.165:0/2036435541 >> v2:172.21.15.172:3300/0 conn(0x7fa1e0144d50 msgr2=0x7fa1e00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.879+0000 7fa1dea00640 1 --2- 172.21.15.165:0/2036435541 >> v2:172.21.15.172:3300/0 conn(0x7fa1e0144d50 0x7fa1e00b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.879+0000 7fa1dea00640 1 -- 172.21.15.165:0/2036435541 >> v1:172.21.15.172:6789/0 conn(0x7fa1e01452a0 legacy=0x7fa1e00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.879+0000 7fa1dea00640 1 -- 172.21.15.165:0/2036435541 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00cc140 con 0x7fa1e01446d0 2024-04-25T10:08:02.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.880+0000 7fa1dea00640 1 --2- 172.21.15.165:0/2036435541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e00b3f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009870 tx=0x7fa1d002a250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3153618c6d05ce02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.881+0000 7fa1de000640 1 -- 172.21.15.165:0/2036435541 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0037070 con 0x7fa1e01446d0 2024-04-25T10:08:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.881+0000 7fa1de000640 1 -- 172.21.15.165:0/2036435541 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1d0032440 con 0x7fa1e01446d0 2024-04-25T10:08:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.881+0000 7fa1de000640 1 -- 172.21.15.165:0/2036435541 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1d0032810 con 0x7fa1e01446d0 2024-04-25T10:08:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.881+0000 7fa1e5a00640 1 -- 172.21.15.165:0/2036435541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 msgr2=0x7fa1e00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.881+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/2036435541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e00b3f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0009870 tx=0x7fa1d002a250 comp rx=0 tx=0).stop 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.882+0000 7fa1e5a00640 1 -- 172.21.15.165:0/2036435541 shutdown_connections 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.882+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/2036435541 >> v2:172.21.15.172:3300/0 conn(0x7fa1e0144d50 0x7fa1e00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.882+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/2036435541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e00b3f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.882+0000 7fa1e5a00640 1 -- 172.21.15.165:0/2036435541 >> 172.21.15.165:0/2036435541 conn(0x7fa1e006abe0 msgr2=0x7fa1e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.882+0000 7fa1e5a00640 1 -- 172.21.15.165:0/2036435541 shutdown_connections 2024-04-25T10:08:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.882+0000 7fa1e5a00640 1 -- 172.21.15.165:0/2036435541 wait complete. 2024-04-25T10:08:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.883+0000 7fa1e5a00640 1 Processor -- start 2024-04-25T10:08:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.884+0000 7fa1e5a00640 1 -- start start 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.884+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e013a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 0x7fa1e00b1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1e5a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01452a0 0x7fa1e0071170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00cc700 con 0x7fa1e0144d50 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00b49a0 con 0x7fa1e01452a0 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1e5a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e00cc9e0 con 0x7fa1e01446d0 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 0x7fa1e00b1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 0x7fa1e00b1c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45422/0 (socket says 172.21.15.165:45422) 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 -- 172.21.15.165:0/1616286325 learned_addr learned my addr 172.21.15.165:0/1616286325 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1df400640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01452a0 0x7fa1e0071170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1dea00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e013a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 -- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 msgr2=0x7fa1e013a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e013a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 -- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01452a0 msgr2=0x7fa1e0071170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01452a0 0x7fa1e0071170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1d7e00640 1 -- 172.21.15.165:0/1616286325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e00b0ea0 con 0x7fa1e0144d50 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.885+0000 7fa1dea00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e013a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.886+0000 7fa1d7e00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 0x7fa1e00b1c60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800e9b0 tx=0x7fa1c800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.886+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8017070 con 0x7fa1e0144d50 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.886+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1c800c9a0 con 0x7fa1e0144d50 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.886+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1c800cd90 con 0x7fa1e0144d50 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.886+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e00b1180 con 0x7fa1e0144d50 2024-04-25T10:08:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.886+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e00b1740 con 0x7fa1e0144d50 2024-04-25T10:08:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.887+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e0073a00 con 0x7fa1e0144d50 2024-04-25T10:08:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.888+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1c801a070 con 0x7fa1e0144d50 2024-04-25T10:08:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.889+0000 7fa1d6a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:02.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.889+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa1c808a8d0 con 0x7fa1e0144d50 2024-04-25T10:08:02.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.890+0000 7fa1dea00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:02.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.890+0000 7fa1dea00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0034000 tx=0x7fa1d002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:02.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:02.893+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1c8059460 con 0x7fa1e0144d50 2024-04-25T10:08:03.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.093+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1e00cf2d0 con 0x7fa1e0144d50 2024-04-25T10:08:03.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.094+0000 7fa1d6a00640 1 -- 172.21.15.165:0/1616286325 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v74) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1c8058d60 con 0x7fa1e0144d50 2024-04-25T10:08:03.099 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:08:03.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1bc06b4d0 msgr2=0x7fa1bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fa1d0034000 tx=0x7fa1d002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 msgr2=0x7fa1e00b1c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 0x7fa1e00b1c60 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800e9b0 tx=0x7fa1c800ee70 comp rx=0 tx=0).stop 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 shutdown_connections 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1bc06b4d0 0x7fa1bc06d990 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e01452a0 0x7fa1e0071170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e0144d50 0x7fa1e00b1c60 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 --2- 172.21.15.165:0/1616286325 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e01446d0 0x7fa1e013a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 >> 172.21.15.165:0/1616286325 conn(0x7fa1e006abe0 msgr2=0x7fa1e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 shutdown_connections 2024-04-25T10:08:03.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.097+0000 7fa1e5a00640 1 -- 172.21.15.165:0/1616286325 wait complete. 2024-04-25T10:08:03.123 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:08:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.323+0000 7f9921c00640 1 Processor -- start 2024-04-25T10:08:03.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.323+0000 7f9921c00640 1 -- start start 2024-04-25T10:08:03.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.324+0000 7f9921c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f991c15b3c0 0x7f991c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.324+0000 7f9921c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b7a0 0x7f991c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f9921c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c0b0b80 con 0x7f991c0cdae0 2024-04-25T10:08:03.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f9921c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c0cd650 con 0x7f991c15b3c0 2024-04-25T10:08:03.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f9921c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cc800 con 0x7f991c15b7a0 2024-04-25T10:08:03.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f991c15b3c0 0x7f991c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f991c15b3c0 0x7f991c0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45434/0 (socket says 172.21.15.165:45434) 2024-04-25T10:08:03.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 -- 172.21.15.165:0/645966880 learned_addr learned my addr 172.21.15.165:0/645966880 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:03.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 -- 172.21.15.165:0/645966880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b7a0 msgr2=0x7f991c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 --2- 172.21.15.165:0/645966880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b7a0 0x7f991c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 -- 172.21.15.165:0/645966880 >> v1:172.21.15.172:6789/0 conn(0x7f991c0cdae0 legacy=0x7f991c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.325+0000 7f991be00640 1 -- 172.21.15.165:0/645966880 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c0ccae0 con 0x7f991c15b3c0 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.326+0000 7f991be00640 1 --2- 172.21.15.165:0/645966880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b3c0 0x7f991c0b0f60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f990c009870 tx=0x7f990c02a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b35bcf01d386f0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f991b400640 1 -- 172.21.15.165:0/645966880 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c037070 con 0x7f991c15b3c0 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f991b400640 1 -- 172.21.15.165:0/645966880 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c032440 con 0x7f991c15b3c0 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f991b400640 1 -- 172.21.15.165:0/645966880 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c032810 con 0x7f991c15b3c0 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 -- 172.21.15.165:0/645966880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b3c0 msgr2=0x7f991c0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 --2- 172.21.15.165:0/645966880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b3c0 0x7f991c0b0f60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f990c009870 tx=0x7f990c02a200 comp rx=0 tx=0).stop 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 -- 172.21.15.165:0/645966880 shutdown_connections 2024-04-25T10:08:03.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 --2- 172.21.15.165:0/645966880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b7a0 0x7f991c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 --2- 172.21.15.165:0/645966880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b3c0 0x7f991c0b0f60 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 -- 172.21.15.165:0/645966880 >> 172.21.15.165:0/645966880 conn(0x7f991c061cc0 msgr2=0x7f991c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.327+0000 7f9921c00640 1 -- 172.21.15.165:0/645966880 shutdown_connections 2024-04-25T10:08:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.328+0000 7f9921c00640 1 -- 172.21.15.165:0/645966880 wait complete. 2024-04-25T10:08:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.330+0000 7f9921c00640 1 Processor -- start 2024-04-25T10:08:03.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.330+0000 7f9921c00640 1 -- start start 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cdae0 0x7f991c1f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b3c0 0x7f991c0ca3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 0x7f991c0ca930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c15e8a0 con 0x7f991c15b7a0 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c15e720 con 0x7f991c0cdae0 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cdf40 con 0x7f991c15b3c0 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 0x7f991c0ca930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 0x7f991c0ca930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45446/0 (socket says 172.21.15.165:45446) 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9921200640 1 -- 172.21.15.165:0/1528350706 learned_addr learned my addr 172.21.15.165:0/1528350706 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.331+0000 7f9913e00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b3c0 0x7f991c0ca3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f991be00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cdae0 0x7f991c1f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921200640 1 -- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b3c0 msgr2=0x7f991c0ca3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921200640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b3c0 0x7f991c0ca3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921200640 1 -- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cdae0 msgr2=0x7f991c1f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921200640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cdae0 0x7f991c1f4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921200640 1 -- 172.21.15.165:0/1528350706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c1f4b40 con 0x7f991c15b7a0 2024-04-25T10:08:03.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f991be00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cdae0 0x7f991c1f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921200640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 0x7f991c0ca930 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f991400c780 tx=0x7f991400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9914017070 con 0x7f991c15b7a0 2024-04-25T10:08:03.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9914003da0 con 0x7f991c15b7a0 2024-04-25T10:08:03.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c1e82c0 con 0x7f991c15b7a0 2024-04-25T10:08:03.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99140052b0 con 0x7f991c15b7a0 2024-04-25T10:08:03.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.332+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c0a8900 con 0x7f991c15b7a0 2024-04-25T10:08:03.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.334+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c073990 con 0x7f991c15b7a0 2024-04-25T10:08:03.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.335+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f991401a070 con 0x7f991c15b7a0 2024-04-25T10:08:03.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.336+0000 7f9918c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b500 0x7f98f806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.336+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9914089b10 con 0x7f991c15b7a0 2024-04-25T10:08:03.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.336+0000 7f991be00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b500 0x7f98f806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.337+0000 7f991be00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b500 0x7f98f806d9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f990c034000 tx=0x7f990c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.341+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99140586a0 con 0x7f991c15b7a0 2024-04-25T10:08:03.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.534+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f991c0a8b00 con 0x7f991c15b7a0 2024-04-25T10:08:03.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.535+0000 7f9918c00640 1 -- 172.21.15.165:0/1528350706 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v74) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9914057fa0 con 0x7f991c15b7a0 2024-04-25T10:08:03.540 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:08:03.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b500 msgr2=0x7f98f806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b500 0x7f98f806d9c0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f990c034000 tx=0x7f990c02e000 comp rx=0 tx=0).stop 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 msgr2=0x7f991c0ca930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 0x7f991c0ca930 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f991400c780 tx=0x7f991400cc40 comp rx=0 tx=0).stop 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 shutdown_connections 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b500 0x7f98f806d9c0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f991c15b7a0 0x7f991c0ca930 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c15b3c0 0x7f991c0ca3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 --2- 172.21.15.165:0/1528350706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c0cdae0 0x7f991c1f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.537+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 >> 172.21.15.165:0/1528350706 conn(0x7f991c061cc0 msgr2=0x7f991c15f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.538+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 shutdown_connections 2024-04-25T10:08:03.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.538+0000 7f9921c00640 1 -- 172.21.15.165:0/1528350706 wait complete. 2024-04-25T10:08:03.562 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:08:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.761+0000 7f6795e00640 1 Processor -- start 2024-04-25T10:08:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.761+0000 7f6795e00640 1 -- start start 2024-04-25T10:08:03.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.763+0000 7f6795e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f679006c9e0 0x7f67900b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.763+0000 7f6795e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f679006cf50 con 0x7f679015b7d0 2024-04-25T10:08:03.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.763+0000 7f6795e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67900ce530 con 0x7f67900b5620 2024-04-25T10:08:03.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.763+0000 7f6795e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67900ce810 con 0x7f679006c9e0 2024-04-25T10:08:03.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.763+0000 7f678ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f679015b7d0 0x7f67900cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47488/0 (socket says 172.21.15.165:47488) 2024-04-25T10:08:03.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.763+0000 7f678ea00640 1 -- 172.21.15.165:0/3167744042 learned_addr learned my addr 172.21.15.165:0/3167744042 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:03.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678f400640 1 --2- 172.21.15.165:0/3167744042 >> v2:172.21.15.172:3300/0 conn(0x7f679006c9e0 0x7f67900b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763377827 0 0) 0x7f679006cf50 con 0x7f679015b7d0 2024-04-25T10:08:03.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6774003520 con 0x7f679015b7d0 2024-04-25T10:08:03.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2388020499 0 0) 0x7f6774003520 con 0x7f679015b7d0 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 >> v2:172.21.15.172:3300/0 conn(0x7f679006c9e0 msgr2=0x7f67900b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 --2- 172.21.15.165:0/3167744042 >> v2:172.21.15.172:3300/0 conn(0x7f679006c9e0 0x7f67900b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 >> v1:172.21.15.172:6789/0 conn(0x7f67900b5620 legacy=0x7f679006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67900ceaf0 con 0x7f679015b7d0 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6780001670 con 0x7f679015b7d0 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 >> v1:172.21.15.165:6789/0 conn(0x7f679015b7d0 legacy=0x7f67900cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774004800 0x7f6774004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67740051c0 0x7f677400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67900ce810 con 0x7f67740039b0 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677400c060 con 0x7f6774004800 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.764+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677400c260 con 0x7f67740051c0 2024-04-25T10:08:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678ea00640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678e000640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774004800 0x7f6774004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678f400640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67740051c0 0x7f677400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678ea00640 1 -- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67740051c0 msgr2=0x7f677400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678ea00640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67740051c0 0x7f677400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678ea00640 1 -- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774004800 msgr2=0x7f6774004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678ea00640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774004800 0x7f6774004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678ea00640 1 -- 172.21.15.165:0/3167744042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774003520 con 0x7f67740039b0 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678e000640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6774004800 0x7f6774004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678f400640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67740051c0 0x7f677400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.765+0000 7f678f400640 1 -- 172.21.15.165:0/3167744042 reap_dead start 2024-04-25T10:08:03.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.766+0000 7f678ea00640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f678000c240 tx=0x7f6774005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf5655c77cdcb39f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780007cf0 con 0x7f67740039b0 2024-04-25T10:08:03.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780005af0 con 0x7f67740039b0 2024-04-25T10:08:03.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f678d600640 1 -- 172.21.15.165:0/3167744042 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780007070 con 0x7f67740039b0 2024-04-25T10:08:03.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f6795e00640 1 -- 172.21.15.165:0/3167744042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 msgr2=0x7f6774003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f6795e00640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f678000c240 tx=0x7f6774005830 comp rx=0 tx=0).stop 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f6795e00640 1 -- 172.21.15.165:0/3167744042 shutdown_connections 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f6795e00640 1 --2- 172.21.15.165:0/3167744042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f6774003dd0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.767+0000 7f6795e00640 1 -- 172.21.15.165:0/3167744042 >> 172.21.15.165:0/3167744042 conn(0x7f67900d0800 msgr2=0x7f67900d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.768+0000 7f6795e00640 1 -- 172.21.15.165:0/3167744042 shutdown_connections 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.768+0000 7f6795e00640 1 -- 172.21.15.165:0/3167744042 wait complete. 2024-04-25T10:08:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.769+0000 7f6795e00640 1 Processor -- start 2024-04-25T10:08:03.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.769+0000 7f6795e00640 1 -- start start 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.770+0000 7f6795e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f67900cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.770+0000 7f6795e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f679015cf00 0x7f679015d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.770+0000 7f6795e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67901f2ff0 0x7f67901f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.770+0000 7f6795e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679015bd30 con 0x7f67740039b0 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.770+0000 7f6795e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679015bbb0 con 0x7f679015cf00 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.770+0000 7f6795e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67900b5ab0 con 0x7f67901f2ff0 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f67900cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f67900cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45472/0 (socket says 172.21.15.165:45472) 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 -- 172.21.15.165:0/2048977718 learned_addr learned my addr 172.21.15.165:0/2048977718 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678f400640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67901f2ff0 0x7f67901f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 -- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67901f2ff0 msgr2=0x7f67901f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67901f2ff0 0x7f67901f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 -- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f679015cf00 msgr2=0x7f679015d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f679015cf00 0x7f679015d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.771+0000 7f678ea00640 1 -- 172.21.15.165:0/2048977718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67901f3da0 con 0x7f67740039b0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.772+0000 7f678ea00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f67900cb6a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f678002ad00 tx=0x7f67900cecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.772+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780001670 con 0x7f67740039b0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.772+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6780042070 con 0x7f67740039b0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.772+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780007270 con 0x7f67740039b0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.772+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67901f4080 con 0x7f67740039b0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.772+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67901f4640 con 0x7f67740039b0 2024-04-25T10:08:03.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.773+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6790073a00 con 0x7f67740039b0 2024-04-25T10:08:03.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.774+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6780007410 con 0x7f67740039b0 2024-04-25T10:08:03.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.775+0000 7f6787400640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f677006b4d0 0x7f677006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:03.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.775+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f67800aa470 con 0x7f67740039b0 2024-04-25T10:08:03.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.775+0000 7f678e000640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f677006b4d0 0x7f677006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:03.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.776+0000 7f678e000640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f677006b4d0 0x7f677006d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6790159c40 tx=0x7f677c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:03.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.780+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6780079090 con 0x7f67740039b0 2024-04-25T10:08:03.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.979+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67900cf470 con 0x7f67740039b0 2024-04-25T10:08:03.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.980+0000 7f6787400640 1 -- 172.21.15.165:0/2048977718 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v74) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6780078990 con 0x7f67740039b0 2024-04-25T10:08:03.986 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:08:03.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.982+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f677006b4d0 msgr2=0x7f677006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.982+0000 7f6795e00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f677006b4d0 0x7f677006d990 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f6790159c40 tx=0x7f677c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.982+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 msgr2=0x7f67900cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f67900cb6a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f678002ad00 tx=0x7f67900cecc0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 shutdown_connections 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f677006b4d0 0x7f677006d990 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67901f2ff0 0x7f67901f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f679015cf00 0x7f679015d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 --2- 172.21.15.165:0/2048977718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67740039b0 0x7f67900cb6a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 >> 172.21.15.165:0/2048977718 conn(0x7f67900d0800 msgr2=0x7f679006c2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 shutdown_connections 2024-04-25T10:08:03.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:03.983+0000 7f6795e00640 1 -- 172.21.15.165:0/2048977718 wait complete. 2024-04-25T10:08:04.008 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:04.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.200+0000 7f61f6600640 1 Processor -- start 2024-04-25T10:08:04.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.200+0000 7f61f6600640 1 -- start start 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.201+0000 7f61f6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 0x7f61f00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.201+0000 7f61f6600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61f00ce3c0 0x7f61f006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.201+0000 7f61f6600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f0071480 con 0x7f61f0154ee0 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.201+0000 7f61f6600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f0072090 con 0x7f61f00ce3c0 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.201+0000 7f61f6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0071230 con 0x7f61f00cfec0 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ef400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f61f0154ee0 0x7f61f00cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47498/0 (socket says 172.21.15.165:47498) 2024-04-25T10:08:04.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ef400640 1 -- 172.21.15.165:0/3218553567 learned_addr learned my addr 172.21.15.165:0/3218553567 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61eea00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 0x7f61f00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 696882746 0 0) 0x7f61f0071480 con 0x7f61f0154ee0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d4003520 con 0x7f61f0154ee0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61efe00640 1 --2- 172.21.15.165:0/3218553567 >> v2:172.21.15.172:3300/0 conn(0x7f61f00ce3c0 0x7f61f006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2277855051 0 0) 0x7f61d4003520 con 0x7f61f0154ee0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 msgr2=0x7f61f00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 0x7f61f00cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 >> v2:172.21.15.172:3300/0 conn(0x7f61f00ce3c0 msgr2=0x7f61f006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 --2- 172.21.15.165:0/3218553567 >> v2:172.21.15.172:3300/0 conn(0x7f61f00ce3c0 0x7f61f006d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f0071760 con 0x7f61f0154ee0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f61e0001670 con 0x7f61f0154ee0 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.202+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 >> v1:172.21.15.165:6789/0 conn(0x7f61f0154ee0 legacy=0x7f61f00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ee000640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 0x7f61d40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ee000640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61d4003fe0 0x7f61d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ee000640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d4004d40 0x7f61d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0071230 con 0x7f61d4003fe0 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400ba70 con 0x7f61d4004d40 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d400bc40 con 0x7f61f00cfec0 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61eea00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61d4003fe0 0x7f61d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61ef400640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 0x7f61d40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61efe00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d4004d40 0x7f61d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61eea00640 1 -- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 msgr2=0x7f61d40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61eea00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61f00cfec0 0x7f61d40039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61eea00640 1 -- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d4004d40 msgr2=0x7f61d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61eea00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d4004d40 0x7f61d40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61eea00640 1 -- 172.21.15.165:0/3218553567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d4003520 con 0x7f61d4003fe0 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.203+0000 7f61efe00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61d4004d40 0x7f61d40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.204+0000 7f61eea00640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61d4003fe0 0x7f61d4004800 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00ba30 tx=0x7f61d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07f69b997f12df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc030070 con 0x7f61d4003fe0 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61dc02ade0 con 0x7f61d4003fe0 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61ee000640 1 -- 172.21.15.165:0/3218553567 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61dc00b600 con 0x7f61d4003fe0 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61f6600640 1 -- 172.21.15.165:0/3218553567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61d4003fe0 msgr2=0x7f61d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61f6600640 1 --2- 172.21.15.165:0/3218553567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61d4003fe0 0x7f61d4004800 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f61dc00ba30 tx=0x7f61d4005830 comp rx=0 tx=0).stop 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61efe00640 1 -- 172.21.15.165:0/3218553567 reap_dead start 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61f6600640 1 -- 172.21.15.165:0/3218553567 shutdown_connections 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61f6600640 1 -- 172.21.15.165:0/3218553567 >> 172.21.15.165:0/3218553567 conn(0x7f61f006abe0 msgr2=0x7f61f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61f6600640 1 -- 172.21.15.165:0/3218553567 shutdown_connections 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.205+0000 7f61f6600640 1 -- 172.21.15.165:0/3218553567 wait complete. 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.207+0000 7f61f6600640 1 Processor -- start 2024-04-25T10:08:04.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.207+0000 7f61f6600640 1 -- start start 2024-04-25T10:08:04.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61f6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61d4003fe0 0x7f61f00c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61f6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f0146a20 0x7f61f00a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61f6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 0x7f61f00a9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61f6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f0071d20 con 0x7f61f00a9a30 2024-04-25T10:08:04.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61f6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00ce800 con 0x7f61f0146a20 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61f6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f01552c0 con 0x7f61d4003fe0 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 0x7f61f00a9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 0x7f61f00a9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45490/0 (socket says 172.21.15.165:45490) 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.208+0000 7f61efe00640 1 -- 172.21.15.165:0/3663115778 learned_addr learned my addr 172.21.15.165:0/3663115778 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61ef400640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61d4003fe0 0x7f61f00c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61eea00640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f0146a20 0x7f61f00a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61efe00640 1 -- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61d4003fe0 msgr2=0x7f61f00c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61efe00640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61d4003fe0 0x7f61f00c45c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61efe00640 1 -- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f0146a20 msgr2=0x7f61f00a94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61efe00640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f0146a20 0x7f61f00a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61efe00640 1 -- 172.21.15.165:0/3663115778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00ae1c0 con 0x7f61f00a9a30 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61efe00640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 0x7f61f00a9eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f740 tx=0x7f61e8009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8019070 con 0x7f61f00a9a30 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e8007ce0 con 0x7f61f00a9a30 2024-04-25T10:08:04.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e8014640 con 0x7f61f00a9a30 2024-04-25T10:08:04.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.209+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f00ae4a0 con 0x7f61f00a9a30 2024-04-25T10:08:04.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.210+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f01475e0 con 0x7f61f00a9a30 2024-04-25T10:08:04.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.211+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f0073990 con 0x7f61f00a9a30 2024-04-25T10:08:04.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.212+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f61e801c070 con 0x7f61f00a9a30 2024-04-25T10:08:04.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.213+0000 7f61e7400640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f61d006b4a0 0x7f61d006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.213+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f61e808a0c0 con 0x7f61f00a9a30 2024-04-25T10:08:04.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.213+0000 7f61ef400640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f61d006b4a0 0x7f61d006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.214+0000 7f61ef400640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f61d006b4a0 0x7f61d006d960 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f61e0001eb0 tx=0x7f61e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.218+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61e8058bc0 con 0x7f61f00a9a30 2024-04-25T10:08:04.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.451+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61f006d360 con 0x7f61f00a9a30 2024-04-25T10:08:04.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.452+0000 7f61e7400640 1 -- 172.21.15.165:0/3663115778 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f61e80584c0 con 0x7f61f00a9a30 2024-04-25T10:08:04.458 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:04.458 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:44.122864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:04.459 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 47 2024-04-25T10:08:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f61d006b4a0 msgr2=0x7f61d006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f61d006b4a0 0x7f61d006d960 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f61e0001eb0 tx=0x7f61e0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 msgr2=0x7f61f00a9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 0x7f61f00a9eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f61f006f740 tx=0x7f61e8009d40 comp rx=0 tx=0).stop 2024-04-25T10:08:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 shutdown_connections 2024-04-25T10:08:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f61d006b4a0 0x7f61d006d960 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61f00a9a30 0x7f61f00a9eb0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61f0146a20 0x7f61f00a94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 --2- 172.21.15.165:0/3663115778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f61d4003fe0 0x7f61f00c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 >> 172.21.15.165:0/3663115778 conn(0x7f61f006abe0 msgr2=0x7f61f015c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.455+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 shutdown_connections 2024-04-25T10:08:04.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.456+0000 7f61f6600640 1 -- 172.21.15.165:0/3663115778 wait complete. 2024-04-25T10:08:04.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:04.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.667+0000 7f272e400640 1 Processor -- start 2024-04-25T10:08:04.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.667+0000 7f272e400640 1 -- start start 2024-04-25T10:08:04.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f272e400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2728144750 0x7f272806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f272e400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2728144b30 con 0x7f27280cf030 2024-04-25T10:08:04.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f272e400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27280712d0 con 0x7f272806eb10 2024-04-25T10:08:04.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f272e400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280715b0 con 0x7f2728144750 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f2727e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f27280cf030 0x7f272806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47508/0 (socket says 172.21.15.165:47508) 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f2727e00640 1 -- 172.21.15.165:0/3858604609 learned_addr learned my addr 172.21.15.165:0/3858604609 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.669+0000 7f2727400640 1 --2- 172.21.15.165:0/3858604609 >> v2:172.21.15.172:3300/0 conn(0x7f2728144750 0x7f272806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.670+0000 7f2726000640 1 -- 172.21.15.165:0/3858604609 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820232347 0 0) 0x7f2728144b30 con 0x7f27280cf030 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.670+0000 7f2726000640 1 -- 172.21.15.165:0/3858604609 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f270c003520 con 0x7f27280cf030 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.670+0000 7f2726000640 1 -- 172.21.15.165:0/3858604609 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921893642 0 0) 0x7f270c003520 con 0x7f27280cf030 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.670+0000 7f2727400640 1 -- 172.21.15.165:0/3858604609 >> v1:172.21.15.172:6789/0 conn(0x7f272806eb10 legacy=0x7f27280ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.670+0000 7f2727400640 1 -- 172.21.15.165:0/3858604609 >> v1:172.21.15.165:6789/0 conn(0x7f27280cf030 legacy=0x7f272806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.670+0000 7f2727400640 1 -- 172.21.15.165:0/3858604609 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728071890 con 0x7f2728144750 2024-04-25T10:08:04.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.672+0000 7f2727400640 1 --2- 172.21.15.165:0/3858604609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272806e5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2718009d90 tx=0x7f271802a680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f5d47e5ccc2a15 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.672+0000 7f2726000640 1 -- 172.21.15.165:0/3858604609 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720001670 con 0x7f2728144750 2024-04-25T10:08:04.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.672+0000 7f2726000640 1 -- 172.21.15.165:0/3858604609 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2718007540 con 0x7f2728144750 2024-04-25T10:08:04.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.672+0000 7f2726000640 1 -- 172.21.15.165:0/3858604609 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27180078f0 con 0x7f2728144750 2024-04-25T10:08:04.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 -- 172.21.15.165:0/3858604609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 msgr2=0x7f272806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 --2- 172.21.15.165:0/3858604609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272806e5d0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2718009d90 tx=0x7f271802a680 comp rx=0 tx=0).stop 2024-04-25T10:08:04.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 -- 172.21.15.165:0/3858604609 shutdown_connections 2024-04-25T10:08:04.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 --2- 172.21.15.165:0/3858604609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272806e5d0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 -- 172.21.15.165:0/3858604609 >> 172.21.15.165:0/3858604609 conn(0x7f272806abe0 msgr2=0x7f272806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 -- 172.21.15.165:0/3858604609 shutdown_connections 2024-04-25T10:08:04.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.673+0000 7f272e400640 1 -- 172.21.15.165:0/3858604609 wait complete. 2024-04-25T10:08:04.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.675+0000 7f272e400640 1 Processor -- start 2024-04-25T10:08:04.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.675+0000 7f272e400640 1 -- start start 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f272e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272806eb10 0x7f272813cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f272e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 0x7f272813d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f272e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272814f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f272e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27281455d0 con 0x7f27280cf030 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f272e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2728145450 con 0x7f2728144750 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f272e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f272815c000 con 0x7f272806eb10 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2726a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 0x7f272813d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2726a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 0x7f272813d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45506/0 (socket says 172.21.15.165:45506) 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2726a00640 1 -- 172.21.15.165:0/4171172982 learned_addr learned my addr 172.21.15.165:0/4171172982 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2727400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272806eb10 0x7f272813cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2727e00640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272814f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2726a00640 1 -- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272806eb10 msgr2=0x7f272813cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2726a00640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272806eb10 0x7f272813cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.676+0000 7f2726a00640 1 -- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 msgr2=0x7f272814f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f2726a00640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272814f1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f2726a00640 1 -- 172.21.15.165:0/4171172982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272813f0c0 con 0x7f27280cf030 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f2727e00640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272814f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f2726a00640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 0x7f272813d0f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f271400c400 tx=0x7f271400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714017070 con 0x7f27280cf030 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27140052d0 con 0x7f27280cf030 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2714003830 con 0x7f27280cf030 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272813f3a0 con 0x7f27280cf030 2024-04-25T10:08:04.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.677+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272813f960 con 0x7f27280cf030 2024-04-25T10:08:04.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.678+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27280cded0 con 0x7f27280cf030 2024-04-25T10:08:04.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.679+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f271401a070 con 0x7f27280cf030 2024-04-25T10:08:04.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.680+0000 7f271f400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f270806b500 0x7f270806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:04.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.680+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f2714089830 con 0x7f27280cf030 2024-04-25T10:08:04.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.681+0000 7f2727400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f270806b500 0x7f270806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:04.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.681+0000 7f2727400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f270806b500 0x7f270806d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2718034000 tx=0x7f271802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:04.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.685+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27280cded0 con 0x7f27280cf030 2024-04-25T10:08:04.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.889+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2728073990 con 0x7f27280cf030 2024-04-25T10:08:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.891+0000 7f271f400640 1 -- 172.21.15.165:0/4171172982 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+17243 (secure 0 0 0) 0x7f27140583c0 con 0x7f27280cf030 2024-04-25T10:08:04.897 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:04.897 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:47.141502+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/1883921643":"2024-04-26T10:07:41.073778+0000","172.21.15.165:6835/1883921643":"2024-04-26T10:07:41.073778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f270806b500 msgr2=0x7f270806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f270806b500 0x7f270806d9c0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2718034000 tx=0x7f271802f000 comp rx=0 tx=0).stop 2024-04-25T10:08:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 msgr2=0x7f272813d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 0x7f272813d0f0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f271400c400 tx=0x7f271400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 shutdown_connections 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f270806b500 0x7f270806d9c0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2728144750 0x7f272814f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27280cf030 0x7f272813d0f0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 --2- 172.21.15.165:0/4171172982 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f272806eb10 0x7f272813cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 >> 172.21.15.165:0/4171172982 conn(0x7f272806abe0 msgr2=0x7f27280ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 shutdown_connections 2024-04-25T10:08:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:04.894+0000 7f272e400640 1 -- 172.21.15.165:0/4171172982 wait complete. 2024-04-25T10:08:04.919 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:08:04.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:05.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.110+0000 7fc4be600640 1 Processor -- start 2024-04-25T10:08:05.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.110+0000 7fc4be600640 1 -- start start 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4be600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4b815c100 0x7fc4b8071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4be600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc4b806ec90 0x7fc4b806d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4be600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b80b3420 con 0x7fc4b80b3800 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4be600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b80b2810 con 0x7fc4b815c100 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4be600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b80b2af0 con 0x7fc4b806ec90 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4bd200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4b815c100 0x7fc4b8071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4bd200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4b815c100 0x7fc4b8071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45510/0 (socket says 172.21.15.165:45510) 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4bd200640 1 -- 172.21.15.165:0/610806334 learned_addr learned my addr 172.21.15.165:0/610806334 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.112+0000 7fc4bdc00640 1 --2- 172.21.15.165:0/610806334 >> v2:172.21.15.172:3300/0 conn(0x7fc4b806ec90 0x7fc4b806d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.113+0000 7fc4bd200640 1 -- 172.21.15.165:0/610806334 >> v2:172.21.15.172:3300/0 conn(0x7fc4b806ec90 msgr2=0x7fc4b806d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.113+0000 7fc4bd200640 1 --2- 172.21.15.165:0/610806334 >> v2:172.21.15.172:3300/0 conn(0x7fc4b806ec90 0x7fc4b806d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.113+0000 7fc4bd200640 1 -- 172.21.15.165:0/610806334 >> v1:172.21.15.172:6789/0 conn(0x7fc4b80b3800 legacy=0x7fc4b806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.113+0000 7fc4bd200640 1 -- 172.21.15.165:0/610806334 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b8071c80 con 0x7fc4b815c100 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.114+0000 7fc4bd200640 1 --2- 172.21.15.165:0/610806334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b815c100 0x7fc4b8071740 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009870 tx=0x7fc4a802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f6c168ac4cb4c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.114+0000 7fc4b7400640 1 -- 172.21.15.165:0/610806334 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8037070 con 0x7fc4b815c100 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.114+0000 7fc4b7400640 1 -- 172.21.15.165:0/610806334 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a8032440 con 0x7fc4b815c100 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.114+0000 7fc4b7400640 1 -- 172.21.15.165:0/610806334 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a8032810 con 0x7fc4b815c100 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 -- 172.21.15.165:0/610806334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b815c100 msgr2=0x7fc4b8071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 --2- 172.21.15.165:0/610806334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b815c100 0x7fc4b8071740 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8009870 tx=0x7fc4a802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 -- 172.21.15.165:0/610806334 shutdown_connections 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 --2- 172.21.15.165:0/610806334 >> v2:172.21.15.172:3300/0 conn(0x7fc4b806ec90 0x7fc4b806d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 --2- 172.21.15.165:0/610806334 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b815c100 0x7fc4b8071740 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 -- 172.21.15.165:0/610806334 >> 172.21.15.165:0/610806334 conn(0x7fc4b806abe0 msgr2=0x7fc4b806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 -- 172.21.15.165:0/610806334 shutdown_connections 2024-04-25T10:08:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.115+0000 7fc4be600640 1 -- 172.21.15.165:0/610806334 wait complete. 2024-04-25T10:08:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.117+0000 7fc4be600640 1 Processor -- start 2024-04-25T10:08:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.117+0000 7fc4be600640 1 -- start start 2024-04-25T10:08:05.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4be600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4b806ec90 0x7fc4b81f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4be600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 0x7fc4b8160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4be600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4b815c100 0x7fc4b8157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4be600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80b24e0 con 0x7fc4b80b3800 2024-04-25T10:08:05.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4be600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80b30b0 con 0x7fc4b806ec90 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4be600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b80711b0 con 0x7fc4b815c100 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 0x7fc4b8160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 0x7fc4b8160ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45526/0 (socket says 172.21.15.165:45526) 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.118+0000 7fc4b7e00640 1 -- 172.21.15.165:0/3294285266 learned_addr learned my addr 172.21.15.165:0/3294285266 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4bd200640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4b806ec90 0x7fc4b81f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4bdc00640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4b815c100 0x7fc4b8157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b7e00640 1 -- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4b815c100 msgr2=0x7fc4b8157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b7e00640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4b815c100 0x7fc4b8157590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b7e00640 1 -- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4b806ec90 msgr2=0x7fc4b81f8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b7e00640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4b806ec90 0x7fc4b81f8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b7e00640 1 -- 172.21.15.165:0/3294285266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b81613e0 con 0x7fc4b80b3800 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b7e00640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 0x7fc4b8160ea0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c780 tx=0x7fc4a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0016020 con 0x7fc4b80b3800 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a000aa50 con 0x7fc4b80b3800 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.119+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b815cef0 con 0x7fc4b80b3800 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.120+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0012420 con 0x7fc4b80b3800 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.120+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b815d730 con 0x7fc4b80b3800 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.121+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b8073990 con 0x7fc4b80b3800 2024-04-25T10:08:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.121+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc4a0019020 con 0x7fc4b80b3800 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.122+0000 7fc4b4c00640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc49406b4d0 0x7fc49406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.122+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc4a008a560 con 0x7fc4b80b3800 2024-04-25T10:08:05.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.123+0000 7fc4bd200640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc49406b4d0 0x7fc49406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.123+0000 7fc4bd200640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc49406b4d0 0x7fc49406d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8034000 tx=0x7fc4a802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.126+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc4a0020030 con 0x7fc4b80b3800 2024-04-25T10:08:05.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.357+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4b806b060 con 0x7fc4b80b3800 2024-04-25T10:08:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.358+0000 7fc4b4c00640 1 -- 172.21.15.165:0/3294285266 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fc4b815cef0 con 0x7fc4b80b3800 2024-04-25T10:08:05.364 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:05.364 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:44.122864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:05.365 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 47 2024-04-25T10:08:05.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc49406b4d0 msgr2=0x7fc49406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc49406b4d0 0x7fc49406d990 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fc4a8034000 tx=0x7fc4a802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 msgr2=0x7fc4b8160ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 0x7fc4b8160ea0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fc4a000c780 tx=0x7fc4a000cc40 comp rx=0 tx=0).stop 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 shutdown_connections 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc49406b4d0 0x7fc49406d990 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4b815c100 0x7fc4b8157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4b80b3800 0x7fc4b8160ea0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 --2- 172.21.15.165:0/3294285266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4b806ec90 0x7fc4b81f8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 >> 172.21.15.165:0/3294285266 conn(0x7fc4b806abe0 msgr2=0x7fc4b80affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.361+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 shutdown_connections 2024-04-25T10:08:05.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.362+0000 7fc4be600640 1 -- 172.21.15.165:0/3294285266 wait complete. 2024-04-25T10:08:05.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.580+0000 7f8735600640 1 Processor -- start 2024-04-25T10:08:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.580+0000 7f8735600640 1 -- start start 2024-04-25T10:08:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.582+0000 7f8735600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f873015e520 0x7f873015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.582+0000 7f8735600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300ae590 con 0x7f873015f040 2024-04-25T10:08:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.582+0000 7f8735600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87300ae870 con 0x7f8730070ff0 2024-04-25T10:08:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.582+0000 7f8735600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87300aee30 con 0x7f873015e520 2024-04-25T10:08:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.582+0000 7f8727400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f873015f040 0x7f87300adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47522/0 (socket says 172.21.15.165:47522) 2024-04-25T10:08:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.582+0000 7f8727400640 1 -- 172.21.15.165:0/2930660165 learned_addr learned my addr 172.21.15.165:0/2930660165 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872fe00640 1 --2- 172.21.15.165:0/2930660165 >> v2:172.21.15.172:3300/0 conn(0x7f873015e520 0x7f873015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1533111348 0 0) 0x7f87300ae590 con 0x7f873015f040 2024-04-25T10:08:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8714003520 con 0x7f873015f040 2024-04-25T10:08:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3034672708 0 0) 0x7f8714003520 con 0x7f873015f040 2024-04-25T10:08:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 >> v2:172.21.15.172:3300/0 conn(0x7f873015e520 msgr2=0x7f873015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 --2- 172.21.15.165:0/2930660165 >> v2:172.21.15.172:3300/0 conn(0x7f873015e520 0x7f873015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 >> v1:172.21.15.172:6789/0 conn(0x7f8730070ff0 legacy=0x7f873006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300aeb50 con 0x7f873015f040 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8718001670 con 0x7f873015f040 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 >> v1:172.21.15.165:6789/0 conn(0x7f873015f040 legacy=0x7f87300adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.583+0000 7f872f400640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8714003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872f400640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8714004800 0x7f8714004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872f400640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140051c0 0x7f871400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87300aee30 con 0x7f8714003a00 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871400c0c0 con 0x7f8714004800 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f871400c2c0 con 0x7f87140051c0 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872fe00640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8714003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f8727400640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8714004800 0x7f8714004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872fe00640 1 -- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140051c0 msgr2=0x7f871400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872fe00640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87140051c0 0x7f871400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872fe00640 1 -- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8714004800 msgr2=0x7f8714004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872fe00640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8714004800 0x7f8714004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f872fe00640 1 -- 172.21.15.165:0/2930660165 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714003520 con 0x7f8714003a00 2024-04-25T10:08:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f8727400640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8714004800 0x7f8714004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:05.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.584+0000 7f8734c00640 1 -- 172.21.15.165:0/2930660165 reap_dead start 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.585+0000 7f872fe00640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8714003e20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8720009ac0 tx=0x7f8714005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e111634c74da50f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720036070 con 0x7f8714003a00 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8720031400 con 0x7f8714003a00 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f872f400640 1 -- 172.21.15.165:0/2930660165 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720009690 con 0x7f8714003a00 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 -- 172.21.15.165:0/2930660165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 msgr2=0x7f8714003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8714003e20 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8720009ac0 tx=0x7f8714005830 comp rx=0 tx=0).stop 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 -- 172.21.15.165:0/2930660165 shutdown_connections 2024-04-25T10:08:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 --2- 172.21.15.165:0/2930660165 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8714003e20 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 -- 172.21.15.165:0/2930660165 >> 172.21.15.165:0/2930660165 conn(0x7f873006abe0 msgr2=0x7f873006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 -- 172.21.15.165:0/2930660165 shutdown_connections 2024-04-25T10:08:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.586+0000 7f8735600640 1 -- 172.21.15.165:0/2930660165 wait complete. 2024-04-25T10:08:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.588+0000 7f8735600640 1 Processor -- start 2024-04-25T10:08:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.588+0000 7f8735600640 1 -- start start 2024-04-25T10:08:05.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f8735600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8730139da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f8735600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87300cbc50 0x7f87300b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f8735600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87300ce690 0x7f873006d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f8735600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873015f650 con 0x7f8714003a00 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f8735600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873015f4d0 con 0x7f87300cbc50 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f8735600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f873015e900 con 0x7f87300ce690 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.589+0000 7f872fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8730139da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8730139da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45546/0 (socket says 172.21.15.165:45546) 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 -- 172.21.15.165:0/373944907 learned_addr learned my addr 172.21.15.165:0/373944907 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f8727400640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87300cbc50 0x7f87300b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f8734c00640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87300ce690 0x7f873006d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 -- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87300ce690 msgr2=0x7f873006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87300ce690 0x7f873006d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 -- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87300cbc50 msgr2=0x7f87300b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87300cbc50 0x7f87300b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 -- 172.21.15.165:0/373944907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300cd5f0 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.590+0000 7f872fe00640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8730139da0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8720034040 tx=0x7f8720004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.591+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872003f020 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.591+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8720037040 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.591+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8720036070 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.591+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87300cd8d0 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.591+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87300cde90 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.592+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8730073990 con 0x7f8714003a00 2024-04-25T10:08:05.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.593+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8720042020 con 0x7f8714003a00 2024-04-25T10:08:05.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.594+0000 7f872cc00640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:05.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.594+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f87200aa4f0 con 0x7f8714003a00 2024-04-25T10:08:05.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.594+0000 7f8727400640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:05.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.595+0000 7f8727400640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f870c06b4d0 0x7f870c06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f873015b620 tx=0x7f8718009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:05.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.598+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8720049030 con 0x7f8714003a00 2024-04-25T10:08:05.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.831+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f873006b1d0 con 0x7f8714003a00 2024-04-25T10:08:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.832+0000 7f872cc00640 1 -- 172.21.15.165:0/373944907 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f8720079080 con 0x7f8714003a00 2024-04-25T10:08:05.839 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:05.839 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":47,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":47,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:07:28.014241+0000","modified":"2024-04-25T10:07:44.122864+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":71,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6870},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6870":{"gid":6870,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/148834032","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":148834032},{"type":"v1","addr":"172.21.15.165:6837","nonce":148834032}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2024-04-25T10:08:05.839 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 47 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f870c06b4d0 msgr2=0x7f870c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f870c06b4d0 0x7f870c06d990 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f873015b620 tx=0x7f8718009040 comp rx=0 tx=0).stop 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 msgr2=0x7f8730139da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8730139da0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f8720034040 tx=0x7f8720004730 comp rx=0 tx=0).stop 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 shutdown_connections 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f870c06b4d0 0x7f870c06d990 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87300ce690 0x7f873006d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87300cbc50 0x7f87300b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 --2- 172.21.15.165:0/373944907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8714003a00 0x7f8730139da0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:05.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 >> 172.21.15.165:0/373944907 conn(0x7f873006abe0 msgr2=0x7f87300b01d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:05.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 shutdown_connections 2024-04-25T10:08:05.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:05.835+0000 7f8735600640 1 -- 172.21.15.165:0/373944907 wait complete. 2024-04-25T10:08:05.860 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:06.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.040+0000 7fa10a400640 1 Processor -- start 2024-04-25T10:08:06.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.040+0000 7fa10a400640 1 -- start start 2024-04-25T10:08:06.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa10a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa10415c100 0x7fa104071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa10a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa10406ec90 0x7fa10406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa10a400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040b3420 con 0x7fa1040b3800 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa10a400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040b2810 con 0x7fa10415c100 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa10a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040b2af0 con 0x7fa10406ec90 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa109000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa10415c100 0x7fa104071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa109000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa10415c100 0x7fa104071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45552/0 (socket says 172.21.15.165:45552) 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.042+0000 7fa109000640 1 -- 172.21.15.165:0/3634712900 learned_addr learned my addr 172.21.15.165:0/3634712900 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.043+0000 7fa109a00640 1 --2- 172.21.15.165:0/3634712900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa10406ec90 0x7fa10406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.043+0000 7fa109000640 1 -- 172.21.15.165:0/3634712900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa10406ec90 msgr2=0x7fa10406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.043+0000 7fa109000640 1 --2- 172.21.15.165:0/3634712900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa10406ec90 0x7fa10406d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.043+0000 7fa109000640 1 -- 172.21.15.165:0/3634712900 >> v1:172.21.15.165:6789/0 conn(0x7fa1040b3800 legacy=0x7fa10406e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.043+0000 7fa109000640 1 -- 172.21.15.165:0/3634712900 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104071c80 con 0x7fa10415c100 2024-04-25T10:08:06.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.044+0000 7fa109000640 1 --2- 172.21.15.165:0/3634712900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10415c100 0x7fa104071740 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009870 tx=0x7fa0ec02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1c44fa4d7983f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.045+0000 7fa103400640 1 -- 172.21.15.165:0/3634712900 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec037070 con 0x7fa10415c100 2024-04-25T10:08:06.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.045+0000 7fa103400640 1 -- 172.21.15.165:0/3634712900 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec032440 con 0x7fa10415c100 2024-04-25T10:08:06.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.045+0000 7fa103400640 1 -- 172.21.15.165:0/3634712900 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec032810 con 0x7fa10415c100 2024-04-25T10:08:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.045+0000 7fa10a400640 1 -- 172.21.15.165:0/3634712900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10415c100 msgr2=0x7fa104071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.045+0000 7fa10a400640 1 --2- 172.21.15.165:0/3634712900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10415c100 0x7fa104071740 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009870 tx=0x7fa0ec02a270 comp rx=0 tx=0).stop 2024-04-25T10:08:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.046+0000 7fa10a400640 1 -- 172.21.15.165:0/3634712900 shutdown_connections 2024-04-25T10:08:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.046+0000 7fa10a400640 1 --2- 172.21.15.165:0/3634712900 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa10406ec90 0x7fa10406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.046+0000 7fa10a400640 1 --2- 172.21.15.165:0/3634712900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10415c100 0x7fa104071740 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.046+0000 7fa10a400640 1 -- 172.21.15.165:0/3634712900 >> 172.21.15.165:0/3634712900 conn(0x7fa10406abe0 msgr2=0x7fa10406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.046+0000 7fa10a400640 1 -- 172.21.15.165:0/3634712900 shutdown_connections 2024-04-25T10:08:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.046+0000 7fa10a400640 1 -- 172.21.15.165:0/3634712900 wait complete. 2024-04-25T10:08:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.047+0000 7fa10a400640 1 Processor -- start 2024-04-25T10:08:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.048+0000 7fa10a400640 1 -- start start 2024-04-25T10:08:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.048+0000 7fa10a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 0x7fa104159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa10a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040b3800 0x7fa10415e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa10a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10415c100 0x7fa10415f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa10a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040b24e0 con 0x7fa10406ec90 2024-04-25T10:08:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa10a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040b30b0 con 0x7fa10415c100 2024-04-25T10:08:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa10a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040711b0 con 0x7fa1040b3800 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 0x7fa104159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 0x7fa104159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45568/0 (socket says 172.21.15.165:45568) 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 -- 172.21.15.165:0/1977987860 learned_addr learned my addr 172.21.15.165:0/1977987860 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 -- 172.21.15.165:0/1977987860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040b3800 msgr2=0x7fa10415e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040b3800 0x7fa10415e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 -- 172.21.15.165:0/1977987860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10415c100 msgr2=0x7fa10415f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10415c100 0x7fa10415f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 -- 172.21.15.165:0/1977987860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10415a050 con 0x7fa10406ec90 2024-04-25T10:08:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.049+0000 7fa109000640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 0x7fa104159980 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec000c00 tx=0x7fa0ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.050+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec041020 con 0x7fa10406ec90 2024-04-25T10:08:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.050+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ec0306c0 con 0x7fa10406ec90 2024-04-25T10:08:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.050+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa10415a330 con 0x7fa10406ec90 2024-04-25T10:08:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.050+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ec037040 con 0x7fa10406ec90 2024-04-25T10:08:06.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.050+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10415a8f0 con 0x7fa10406ec90 2024-04-25T10:08:06.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.051+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa104073990 con 0x7fa10406ec90 2024-04-25T10:08:06.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.052+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0ec044020 con 0x7fa10406ec90 2024-04-25T10:08:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.053+0000 7fa100c00640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.053+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fa0ec0aac00 con 0x7fa10406ec90 2024-04-25T10:08:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.053+0000 7fa103e00640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.054+0000 7fa103e00640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4000f60 tx=0x7fa0f40079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.057+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0ec0b1080 con 0x7fa10406ec90 2024-04-25T10:08:06.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.239+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1040b2000 con 0x7fa10406ec90 2024-04-25T10:08:06.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.242+0000 7fa100c00640 1 -- 172.21.15.165:0/1977987860 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+17243 (secure 0 0 0) 0x7fa0ec04b030 con 0x7fa10406ec90 2024-04-25T10:08:06.248 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:06.249 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:07:47.141502+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:07:25.839490+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"67","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_data","create_time":"2024-04-25T10:07:27.018634+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.18","mappings":[{"from":7,"to":2}]},{"pgid":"10.e","mappings":[{"from":4,"to":3}]},{"pgid":"10.10","mappings":[{"from":4,"to":3}]},{"pgid":"10.1f","mappings":[{"from":7,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/1883921643":"2024-04-26T10:07:41.073778+0000","172.21.15.165:6835/1883921643":"2024-04-26T10:07:41.073778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 msgr2=0x7fa0e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4000f60 tx=0x7fa0f40079d0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 msgr2=0x7fa104159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 0x7fa104159980 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec000c00 tx=0x7fa0ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 shutdown_connections 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10415c100 0x7fa10415f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040b3800 0x7fa10415e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 --2- 172.21.15.165:0/1977987860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa10406ec90 0x7fa104159980 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 >> 172.21.15.165:0/1977987860 conn(0x7fa10406abe0 msgr2=0x7fa1040cdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 shutdown_connections 2024-04-25T10:08:06.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.245+0000 7fa10a400640 1 -- 172.21.15.165:0/1977987860 wait complete. 2024-04-25T10:08:06.270 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:08:06.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.466+0000 7fd069400640 1 Processor -- start 2024-04-25T10:08:06.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.466+0000 7fd069400640 1 -- start start 2024-04-25T10:08:06.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.467+0000 7fd069400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd06415dd30 0x7fd0640cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.467+0000 7fd069400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd06406bff0 0x7fd0640b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd069400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0640b5970 0x7fd064161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd069400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06406d000 con 0x7fd06415dd30 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd069400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06415e110 con 0x7fd06406bff0 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd069400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640b2fd0 con 0x7fd0640b5970 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd06415dd30 0x7fd0640cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd06415dd30 0x7fd0640cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45580/0 (socket says 172.21.15.165:45580) 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 -- 172.21.15.165:0/1955562977 learned_addr learned my addr 172.21.15.165:0/1955562977 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd068a00640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0640b5970 0x7fd064161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063400640 1 --2- 172.21.15.165:0/1955562977 >> v2:172.21.15.172:3300/0 conn(0x7fd06406bff0 0x7fd0640b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 -- 172.21.15.165:0/1955562977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0640b5970 msgr2=0x7fd064161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0640b5970 0x7fd064161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 -- 172.21.15.165:0/1955562977 >> v2:172.21.15.172:3300/0 conn(0x7fd06406bff0 msgr2=0x7fd0640b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 --2- 172.21.15.165:0/1955562977 >> v2:172.21.15.172:3300/0 conn(0x7fd06406bff0 0x7fd0640b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.468+0000 7fd063e00640 1 -- 172.21.15.165:0/1955562977 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06406c480 con 0x7fd06415dd30 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.469+0000 7fd068a00640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0640b5970 0x7fd064161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:06.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.469+0000 7fd063400640 1 --2- 172.21.15.165:0/1955562977 >> v2:172.21.15.172:3300/0 conn(0x7fd06406bff0 0x7fd0640b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.470+0000 7fd063e00640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0640cebd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd06408c2c0 tx=0x7fd05402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab1c187cd6f113 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.470+0000 7fd062a00640 1 -- 172.21.15.165:0/1955562977 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054037070 con 0x7fd06415dd30 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.470+0000 7fd062a00640 1 -- 172.21.15.165:0/1955562977 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd054032440 con 0x7fd06415dd30 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.470+0000 7fd062a00640 1 -- 172.21.15.165:0/1955562977 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd054032810 con 0x7fd06415dd30 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.470+0000 7fd069400640 1 -- 172.21.15.165:0/1955562977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 msgr2=0x7fd0640cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.470+0000 7fd069400640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0640cebd0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7fd06408c2c0 tx=0x7fd05402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 -- 172.21.15.165:0/1955562977 shutdown_connections 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0640b5970 0x7fd064161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 --2- 172.21.15.165:0/1955562977 >> v2:172.21.15.172:3300/0 conn(0x7fd06406bff0 0x7fd0640b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 --2- 172.21.15.165:0/1955562977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0640cebd0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 -- 172.21.15.165:0/1955562977 >> 172.21.15.165:0/1955562977 conn(0x7fd0640d0610 msgr2=0x7fd0640d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 -- 172.21.15.165:0/1955562977 shutdown_connections 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.471+0000 7fd069400640 1 -- 172.21.15.165:0/1955562977 wait complete. 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.472+0000 7fd069400640 1 Processor -- start 2024-04-25T10:08:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 -- start start 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0640b5970 0x7fd0640aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0641f6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd064159e50 0x7fd0640ab050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06406cd20 con 0x7fd06415dd30 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06406ca40 con 0x7fd0640b5970 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.473+0000 7fd069400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0640cf110 con 0x7fd064159e50 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0641f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0641f6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45586/0 (socket says 172.21.15.165:45586) 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 -- 172.21.15.165:0/3812065701 learned_addr learned my addr 172.21.15.165:0/3812065701 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063e00640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0640b5970 0x7fd0640aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd068a00640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd064159e50 0x7fd0640ab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 -- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd064159e50 msgr2=0x7fd0640ab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd064159e50 0x7fd0640ab050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 -- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0640b5970 msgr2=0x7fd0640aa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0640b5970 0x7fd0640aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.474+0000 7fd063400640 1 -- 172.21.15.165:0/3812065701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0641f6c40 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.475+0000 7fd063400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0641f6700 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c7c0 tx=0x7fd05000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.475+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050015070 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.475+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd050007a30 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.475+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050007e00 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.475+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0641f5660 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.475+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0641f5c20 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.476+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0640d0e90 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.477+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd050018070 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.478+0000 7fd05be00640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04406b4d0 0x7fd04406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.478+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fd05008a460 con 0x7fd06415dd30 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.478+0000 7fd063e00640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04406b4d0 0x7fd04406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.479+0000 7fd063e00640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04406b4d0 0x7fd04406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd054034000 tx=0x7fd05402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.482+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0640d0e90 con 0x7fd06415dd30 2024-04-25T10:08:06.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.714+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd0640d16c0 con 0x7fd06415dd30 2024-04-25T10:08:06.721 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:08:06.715+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:08:06.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.719+0000 7fd05be00640 1 -- 172.21.15.165:0/3812065701 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd050058ff0 con 0x7fd06415dd30 2024-04-25T10:08:06.725 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:08:06.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04406b4d0 msgr2=0x7fd04406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04406b4d0 0x7fd04406d990 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fd054034000 tx=0x7fd05402e000 comp rx=0 tx=0).stop 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 msgr2=0x7fd0641f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0641f6700 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fd05000c7c0 tx=0x7fd05000cc80 comp rx=0 tx=0).stop 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 shutdown_connections 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04406b4d0 0x7fd04406d990 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd064159e50 0x7fd0640ab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06415dd30 0x7fd0641f6700 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 --2- 172.21.15.165:0/3812065701 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0640b5970 0x7fd0640aa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.722+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 >> 172.21.15.165:0/3812065701 conn(0x7fd0640d0610 msgr2=0x7fd0640b6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.723+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 shutdown_connections 2024-04-25T10:08:06.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.723+0000 7fd069400640 1 -- 172.21.15.165:0/3812065701 wait complete. 2024-04-25T10:08:06.748 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:08:06.804 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:08:06.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.911+0000 7f2a75400640 1 Processor -- start 2024-04-25T10:08:06.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.911+0000 7f2a75400640 1 -- start start 2024-04-25T10:08:06.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a75400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a700b5ae0 0x7f2a70168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a75400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a70161e70 con 0x7f2a7015dbd0 2024-04-25T10:08:06.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a75400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a70162150 con 0x7f2a700b95c0 2024-04-25T10:08:06.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a75400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a700d1ce0 con 0x7f2a700b5ae0 2024-04-25T10:08:06.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2a7015dbd0 0x7f2a70161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41046/0 (socket says 172.21.15.165:41046) 2024-04-25T10:08:06.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6fe00640 1 -- 172.21.15.165:0/1580004010 learned_addr learned my addr 172.21.15.165:0/1580004010 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:06.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579272283 0 0) 0x7f2a70161e70 con 0x7f2a7015dbd0 2024-04-25T10:08:06.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a54003520 con 0x7f2a7015dbd0 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a74a00640 1 --2- 172.21.15.165:0/1580004010 >> v2:172.21.15.172:3300/0 conn(0x7f2a700b5ae0 0x7f2a70168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3376637657 0 0) 0x7f2a54003520 con 0x7f2a7015dbd0 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 >> v2:172.21.15.172:3300/0 conn(0x7f2a700b5ae0 msgr2=0x7f2a70168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6f400640 1 --2- 172.21.15.165:0/1580004010 >> v2:172.21.15.172:3300/0 conn(0x7f2a700b5ae0 0x7f2a70168950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.913+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 >> v1:172.21.15.172:6789/0 conn(0x7f2a700b95c0 legacy=0x7f2a700b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70162430 con 0x7f2a7015dbd0 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2a60001670 con 0x7f2a7015dbd0 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 >> v1:172.21.15.165:6789/0 conn(0x7f2a7015dbd0 legacy=0x7f2a70161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a540039b0 0x7f2a54003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a54004800 0x7f2a54004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a540051c0 0x7f2a5400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700d1ce0 con 0x7f2a540039b0 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5400c0f0 con 0x7f2a54004800 2024-04-25T10:08:06.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5400c2f0 con 0x7f2a540051c0 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a540039b0 0x7f2a54003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6fe00640 1 -- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a540051c0 msgr2=0x7f2a5400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a540051c0 0x7f2a5400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a67400640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a54004800 0x7f2a54004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6fe00640 1 -- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a54004800 msgr2=0x7f2a54004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a54004800 0x7f2a54004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a74a00640 1 -- 172.21.15.165:0/1580004010 reap_dead start 2024-04-25T10:08:06.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.914+0000 7f2a6fe00640 1 -- 172.21.15.165:0/1580004010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54003520 con 0x7f2a540039b0 2024-04-25T10:08:06.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.915+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a540039b0 0x7f2a54003db0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000c210 tx=0x7f2a54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eb9fd3e2e5aa126 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60007d30 con 0x7f2a540039b0 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a600058d0 con 0x7f2a540039b0 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a6f400640 1 -- 172.21.15.165:0/1580004010 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a60005d20 con 0x7f2a540039b0 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 -- 172.21.15.165:0/1580004010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a540039b0 msgr2=0x7f2a54003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a540039b0 0x7f2a54003db0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f2a6000c210 tx=0x7f2a54005830 comp rx=0 tx=0).stop 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 -- 172.21.15.165:0/1580004010 shutdown_connections 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 --2- 172.21.15.165:0/1580004010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a540039b0 0x7f2a54003db0 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 -- 172.21.15.165:0/1580004010 >> 172.21.15.165:0/1580004010 conn(0x7f2a700d03d0 msgr2=0x7f2a700d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 -- 172.21.15.165:0/1580004010 shutdown_connections 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.916+0000 7f2a75400640 1 -- 172.21.15.165:0/1580004010 wait complete. 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.918+0000 7f2a75400640 1 Processor -- start 2024-04-25T10:08:06.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.918+0000 7f2a75400640 1 -- start start 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a75400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a540039b0 0x7f2a70158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a75400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700cbb00 0x7f2a7015ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a75400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 0x7f2a7015bf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a75400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700b5550 con 0x7f2a700cbf80 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a75400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700b53d0 con 0x7f2a540039b0 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a75400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700b56d0 con 0x7f2a700cbb00 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 0x7f2a7015bf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 0x7f2a7015bf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51460/0 (socket says 172.21.15.165:51460) 2024-04-25T10:08:06.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 -- 172.21.15.165:0/2770015924 learned_addr learned my addr 172.21.15.165:0/2770015924 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a67400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700cbb00 0x7f2a7015ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 -- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700cbb00 msgr2=0x7f2a7015ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700cbb00 0x7f2a7015ba10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 -- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a540039b0 msgr2=0x7f2a70158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a540039b0 0x7f2a70158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.919+0000 7f2a74a00640 1 -- 172.21.15.165:0/2770015924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a7015b0d0 con 0x7f2a700cbf80 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.920+0000 7f2a74a00640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 0x7f2a7015bf50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2a7006b850 tx=0x7f2a6800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.920+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a6800ed90 con 0x7f2a700cbf80 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.920+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a68019070 con 0x7f2a700cbf80 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.920+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68014660 con 0x7f2a700cbf80 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.920+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a7015e0e0 con 0x7f2a700cbf80 2024-04-25T10:08:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.920+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a7015e6a0 con 0x7f2a700cbf80 2024-04-25T10:08:06.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.921+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a700d1520 con 0x7f2a700cbf80 2024-04-25T10:08:06.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.922+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a68014800 con 0x7f2a700cbf80 2024-04-25T10:08:06.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.923+0000 7f2a6cc00640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:06.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.923+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f2a6808a860 con 0x7f2a700cbf80 2024-04-25T10:08:06.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.923+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:06.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.924+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2a60033000 tx=0x7f2a60009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:06.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:06.927+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2a7015b0d0 con 0x7f2a700cbf80 2024-04-25T10:08:07.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.142+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f2a700b35c0 con 0x7f2a700cbf80 2024-04-25T10:08:07.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.788+0000 7f2a6cc00640 1 -- 172.21.15.165:0/2770015924 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2a68059420 con 0x7f2a700cbf80 2024-04-25T10:08:07.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 msgr2=0x7f2a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2a60033000 tx=0x7f2a60009030 comp rx=0 tx=0).stop 2024-04-25T10:08:07.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 msgr2=0x7f2a7015bf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:07.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 0x7f2a7015bf50 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f2a7006b850 tx=0x7f2a6800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 shutdown_connections 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700cbf80 0x7f2a7015bf50 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700cbb00 0x7f2a7015ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 --2- 172.21.15.165:0/2770015924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a540039b0 0x7f2a70158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.791+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 >> 172.21.15.165:0/2770015924 conn(0x7f2a700d03d0 msgr2=0x7f2a70161620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.792+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 shutdown_connections 2024-04-25T10:08:07.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:07.792+0000 7f2a75400640 1 -- 172.21.15.165:0/2770015924 wait complete. 2024-04-25T10:08:07.817 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.006+0000 7f6b84e00640 1 Processor -- start 2024-04-25T10:08:08.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.006+0000 7f6b84e00640 1 -- start start 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b84e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6b80145140 0x7f6b800ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b84e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6b800ce950 0x7f6b800c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b84e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b80145520 con 0x7f6b800c5030 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b84e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b800ced90 con 0x7f6b80145140 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b84e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b80071270 con 0x7f6b800ce950 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b7f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6b80145140 0x7f6b800ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b7f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6b80145140 0x7f6b800ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51464/0 (socket says 172.21.15.165:51464) 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b7f400640 1 -- 172.21.15.165:0/446157042 learned_addr learned my addr 172.21.15.165:0/446157042 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:08.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.008+0000 7f6b7ea00640 1 --2- 172.21.15.165:0/446157042 >> v2:172.21.15.172:3300/0 conn(0x7f6b800ce950 0x7f6b800c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.009+0000 7f6b7f400640 1 -- 172.21.15.165:0/446157042 >> v2:172.21.15.172:3300/0 conn(0x7f6b800ce950 msgr2=0x7f6b800c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.009+0000 7f6b7f400640 1 --2- 172.21.15.165:0/446157042 >> v2:172.21.15.172:3300/0 conn(0x7f6b800ce950 0x7f6b800c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.009+0000 7f6b7f400640 1 -- 172.21.15.165:0/446157042 >> v1:172.21.15.172:6789/0 conn(0x7f6b800c5030 legacy=0x7f6b8006d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:08.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.009+0000 7f6b7f400640 1 -- 172.21.15.165:0/446157042 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b80071550 con 0x7f6b80145140 2024-04-25T10:08:08.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.010+0000 7f6b7f400640 1 --2- 172.21.15.165:0/446157042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b80145140 0x7f6b800ce410 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6b800b0c00 tx=0x7f6b7002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc8d2d3f3ef96174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b7e000640 1 -- 172.21.15.165:0/446157042 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70037070 con 0x7f6b80145140 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b7e000640 1 -- 172.21.15.165:0/446157042 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70032440 con 0x7f6b80145140 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b7e000640 1 -- 172.21.15.165:0/446157042 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70032810 con 0x7f6b80145140 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b84e00640 1 -- 172.21.15.165:0/446157042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b80145140 msgr2=0x7f6b800ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b84e00640 1 --2- 172.21.15.165:0/446157042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b80145140 0x7f6b800ce410 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f6b800b0c00 tx=0x7f6b7002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b84e00640 1 -- 172.21.15.165:0/446157042 shutdown_connections 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b84e00640 1 --2- 172.21.15.165:0/446157042 >> v2:172.21.15.172:3300/0 conn(0x7f6b800ce950 0x7f6b800c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b84e00640 1 --2- 172.21.15.165:0/446157042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b80145140 0x7f6b800ce410 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.011+0000 7f6b84e00640 1 -- 172.21.15.165:0/446157042 >> 172.21.15.165:0/446157042 conn(0x7f6b80061cc0 msgr2=0x7f6b8006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.012+0000 7f6b84e00640 1 -- 172.21.15.165:0/446157042 shutdown_connections 2024-04-25T10:08:08.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.012+0000 7f6b84e00640 1 -- 172.21.15.165:0/446157042 wait complete. 2024-04-25T10:08:08.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.013+0000 7f6b84e00640 1 Processor -- start 2024-04-25T10:08:08.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.013+0000 7f6b84e00640 1 -- start start 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.014+0000 7f6b84e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b800c5030 0x7f6b800aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.014+0000 7f6b84e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 0x7f6b801f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.014+0000 7f6b84e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b80145140 0x7f6b801f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.014+0000 7f6b84e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800c5480 con 0x7f6b800ce950 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.014+0000 7f6b84e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b80071b10 con 0x7f6b800c5030 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.014+0000 7f6b84e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b800c5600 con 0x7f6b80145140 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 0x7f6b801f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 0x7f6b801f6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51474/0 (socket says 172.21.15.165:51474) 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 -- 172.21.15.165:0/3537171401 learned_addr learned my addr 172.21.15.165:0/3537171401 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7f400640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b800c5030 0x7f6b800aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 -- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b80145140 msgr2=0x7f6b801f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b80145140 0x7f6b801f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 -- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b800c5030 msgr2=0x7f6b800aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b800c5030 0x7f6b800aca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 -- 172.21.15.165:0/3537171401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b800cd160 con 0x7f6b800ce950 2024-04-25T10:08:08.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.015+0000 7f6b7ea00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 0x7f6b801f6b00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d800 tx=0x7f6b6c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.016+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00b6c0 con 0x7f6b800ce950 2024-04-25T10:08:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.016+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b6c019070 con 0x7f6b800ce950 2024-04-25T10:08:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.016+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00ba80 con 0x7f6b800ce950 2024-04-25T10:08:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.016+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b800cd440 con 0x7f6b800ce950 2024-04-25T10:08:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.016+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b800cda00 con 0x7f6b800ce950 2024-04-25T10:08:08.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.017+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b80073990 con 0x7f6b800ce950 2024-04-25T10:08:08.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.018+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b6c00bc20 con 0x7f6b800ce950 2024-04-25T10:08:08.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.019+0000 7f6b77400640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b6006b4d0 0x7f6b6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:08.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.019+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f6b6c08aa00 con 0x7f6b800ce950 2024-04-25T10:08:08.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.020+0000 7f6b7f400640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b6006b4d0 0x7f6b6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:08.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.020+0000 7f6b7f400640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b6006b4d0 0x7f6b6006d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6b70034000 tx=0x7f6b7002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:08.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.024+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b6c059530 con 0x7f6b800ce950 2024-04-25T10:08:08.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.219+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b800cf160 con 0x7f6b800ce950 2024-04-25T10:08:08.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.807+0000 7f6b77400640 1 -- 172.21.15.165:0/3537171401 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6b6c058e30 con 0x7f6b800ce950 2024-04-25T10:08:08.814 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:08:08.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.811+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b6006b4d0 msgr2=0x7f6b6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b6006b4d0 0x7f6b6006d990 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6b70034000 tx=0x7f6b7002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:08.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 msgr2=0x7f6b801f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:08.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 0x7f6b801f6b00 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00d800 tx=0x7f6b6c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 shutdown_connections 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b6006b4d0 0x7f6b6006d990 secure :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f6b70034000 tx=0x7f6b7002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b80145140 0x7f6b801f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b800ce950 0x7f6b801f6b00 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 --2- 172.21.15.165:0/3537171401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b800c5030 0x7f6b800aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 >> 172.21.15.165:0/3537171401 conn(0x7f6b80061cc0 msgr2=0x7f6b8015dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:08.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.812+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 shutdown_connections 2024-04-25T10:08:08.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:08.814+0000 7f6b84e00640 1 -- 172.21.15.165:0/3537171401 wait complete. 2024-04-25T10:08:08.841 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:08:09.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.033+0000 7f06d4c00640 1 Processor -- start 2024-04-25T10:08:09.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.034+0000 7f06d4c00640 1 -- start start 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06d4c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f06d00ce230 0x7f06d00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06d4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d015b270 0x7f06d015b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06d4c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d0072280 con 0x7f06d00cc4f0 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06d4c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06d0071270 con 0x7f06d00ce230 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06d4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d00714c0 con 0x7f06d015b270 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06cd400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f06d00ce230 0x7f06d00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.035+0000 7f06cde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f06d00cc4f0 0x7f06d00cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41048/0 (socket says 172.21.15.165:41048) 2024-04-25T10:08:09.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06cde00640 1 -- 172.21.15.165:0/2647636067 learned_addr learned my addr 172.21.15.165:0/2647636067 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:09.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06ce800640 1 --2- 172.21.15.165:0/2647636067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d015b270 0x7f06d015b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06cd400640 1 -- 172.21.15.165:0/2647636067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d015b270 msgr2=0x7f06d015b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:09.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06cd400640 1 --2- 172.21.15.165:0/2647636067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d015b270 0x7f06d015b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06cd400640 1 -- 172.21.15.165:0/2647636067 >> v1:172.21.15.165:6789/0 conn(0x7f06d00cc4f0 legacy=0x7f06d00cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06cd400640 1 -- 172.21.15.165:0/2647636067 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d00717a0 con 0x7f06d00ce230 2024-04-25T10:08:09.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.036+0000 7f06cca00640 1 -- 172.21.15.165:0/2647636067 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1409096450 0 0) 0x7f06d0072280 con 0x7f06d00cc4f0 2024-04-25T10:08:09.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.037+0000 7f06cd400640 1 --2- 172.21.15.165:0/2647636067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d00ce230 0x7f06d00cc8d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097a0 tx=0x7f06bc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99533489e5f236b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.037+0000 7f06cca00640 1 -- 172.21.15.165:0/2647636067 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc037070 con 0x7f06d00ce230 2024-04-25T10:08:09.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06cca00640 1 -- 172.21.15.165:0/2647636067 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06bc032440 con 0x7f06d00ce230 2024-04-25T10:08:09.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06cca00640 1 -- 172.21.15.165:0/2647636067 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06bc032810 con 0x7f06d00ce230 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 -- 172.21.15.165:0/2647636067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d00ce230 msgr2=0x7f06d00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2647636067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d00ce230 0x7f06d00cc8d0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f06bc0097a0 tx=0x7f06bc02a180 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 -- 172.21.15.165:0/2647636067 shutdown_connections 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2647636067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d015b270 0x7f06d015b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2647636067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d00ce230 0x7f06d00cc8d0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 -- 172.21.15.165:0/2647636067 >> 172.21.15.165:0/2647636067 conn(0x7f06d006abe0 msgr2=0x7f06d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 -- 172.21.15.165:0/2647636067 shutdown_connections 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.038+0000 7f06d4c00640 1 -- 172.21.15.165:0/2647636067 wait complete. 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.040+0000 7f06d4c00640 1 Processor -- start 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.040+0000 7f06d4c00640 1 -- start start 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.041+0000 7f06d4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d00cc4f0 0x7f06d014f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.041+0000 7f06d4c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06d00ce230 0x7f06d01f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.041+0000 7f06d4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 0x7f06d01e3a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.041+0000 7f06d4c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d015bbf0 con 0x7f06d015b270 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.041+0000 7f06d4c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d0071d60 con 0x7f06d00ce230 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.041+0000 7f06d4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d015bd70 con 0x7f06d00cc4f0 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 0x7f06d01e3a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 0x7f06d01e3a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51488/0 (socket says 172.21.15.165:51488) 2024-04-25T10:08:09.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 -- 172.21.15.165:0/2563901378 learned_addr learned my addr 172.21.15.165:0/2563901378 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06cd400640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06d00ce230 0x7f06d01f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06cde00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d00cc4f0 0x7f06d014f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 -- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d00cc4f0 msgr2=0x7f06d014f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d00cc4f0 0x7f06d014f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 -- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06d00ce230 msgr2=0x7f06d01f3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06d00ce230 0x7f06d01f3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 -- 172.21.15.165:0/2563901378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d014e700 con 0x7f06d015b270 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06cde00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d00cc4f0 0x7f06d014f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06cd400640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06d00ce230 0x7f06d01f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.042+0000 7f06ce800640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 0x7f06d01e3a70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f06c800cda0 tx=0x7f06c800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.043+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8003b90 con 0x7f06d015b270 2024-04-25T10:08:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.043+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8003d30 con 0x7f06d015b270 2024-04-25T10:08:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.043+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8004240 con 0x7f06d015b270 2024-04-25T10:08:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.043+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d014e9e0 con 0x7f06d015b270 2024-04-25T10:08:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.043+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d014efa0 con 0x7f06d015b270 2024-04-25T10:08:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.044+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d0073a00 con 0x7f06d015b270 2024-04-25T10:08:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.045+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06c80043e0 con 0x7f06d015b270 2024-04-25T10:08:09.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.046+0000 7f06c6000640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06b006b4d0 0x7f06b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:09.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.046+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5922+0+0 (secure 0 0 0) 0x7f06c808add0 con 0x7f06d015b270 2024-04-25T10:08:09.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.046+0000 7f06cde00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06b006b4d0 0x7f06b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:09.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.047+0000 7f06cde00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06b006b4d0 0x7f06b006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000f60 tx=0x7f06c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:09.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.050+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f06c801d340 con 0x7f06d015b270 2024-04-25T10:08:09.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.246+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f06d01f35d0 con 0x7f06d015b270 2024-04-25T10:08:09.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.811+0000 7f06c6000640 1 -- 172.21.15.165:0/2563901378 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) v1 ==== 169+0+0 (secure 0 0 0) 0x7f06c808c0d0 con 0x7f06d015b270 2024-04-25T10:08:09.818 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:08:09.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.817+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06b006b4d0 msgr2=0x7f06b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.817+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06b006b4d0 0x7f06b006d990 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f06c0000f60 tx=0x7f06c0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:09.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.817+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 msgr2=0x7f06d01e3a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:09.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.817+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 0x7f06d01e3a70 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f06c800cda0 tx=0x7f06c800a660 comp rx=0 tx=0).stop 2024-04-25T10:08:09.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 shutdown_connections 2024-04-25T10:08:09.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06b006b4d0 0x7f06b006d990 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06d015b270 0x7f06d01e3a70 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06d00ce230 0x7f06d01f3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 --2- 172.21.15.165:0/2563901378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06d00cc4f0 0x7f06d014f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:09.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 >> 172.21.15.165:0/2563901378 conn(0x7f06d006abe0 msgr2=0x7f06d01599a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:09.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 shutdown_connections 2024-04-25T10:08:09.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:09.818+0000 7f06d4c00640 1 -- 172.21.15.165:0/2563901378 wait complete. 2024-04-25T10:08:09.848 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.037+0000 7f0e4da00640 1 Processor -- start 2024-04-25T10:08:10.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.037+0000 7f0e4da00640 1 -- start start 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.039+0000 7f0e4da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0e480ceb20 0x7f0e480c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.039+0000 7f0e4da00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e48145520 con 0x7f0e48145140 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.039+0000 7f0e4da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e48071270 con 0x7f0e480c5030 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.039+0000 7f0e4da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e48071550 con 0x7f0e480ceb20 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.039+0000 7f0e47e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0e48145140 0x7f0e480ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41058/0 (socket says 172.21.15.165:41058) 2024-04-25T10:08:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.039+0000 7f0e47e00640 1 -- 172.21.15.165:0/3487826854 learned_addr learned my addr 172.21.15.165:0/3487826854 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2954266370 0 0) 0x7f0e48145520 con 0x7f0e48145140 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e2c003520 con 0x7f0e48145140 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e47400640 1 --2- 172.21.15.165:0/3487826854 >> v2:172.21.15.172:3300/0 conn(0x7f0e480ceb20 0x7f0e480c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2979807879 0 0) 0x7f0e2c003520 con 0x7f0e48145140 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 >> v2:172.21.15.172:3300/0 conn(0x7f0e480ceb20 msgr2=0x7f0e480c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 --2- 172.21.15.165:0/3487826854 >> v2:172.21.15.172:3300/0 conn(0x7f0e480ceb20 0x7f0e480c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 >> v1:172.21.15.172:6789/0 conn(0x7f0e480c5030 legacy=0x7f0e4815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48071830 con 0x7f0e48145140 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f0e38001670 con 0x7f0e48145140 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 >> v1:172.21.15.165:6789/0 conn(0x7f0e48145140 legacy=0x7f0e480ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c004800 0x7f0e2c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c0052e0 0x7f0e2c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e48071550 con 0x7f0e2c0052e0 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c00c1b0 con 0x7f0e2c004800 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c00c3b0 con 0x7f0e2c0039b0 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.040+0000 7f0e4d000640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c0052e0 0x7f0e2c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e47400640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c004800 0x7f0e2c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e47e00640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e4d000640 1 -- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0039b0 msgr2=0x7f0e2c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e4d000640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0039b0 0x7f0e2c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e4d000640 1 -- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c004800 msgr2=0x7f0e2c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e4d000640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e2c004800 0x7f0e2c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.041+0000 7f0e4d000640 1 -- 172.21.15.165:0/3487826854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c003520 con 0x7f0e2c0052e0 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.042+0000 7f0e4d000640 1 -- 172.21.15.165:0/3487826854 reap_dead start 2024-04-25T10:08:10.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.042+0000 7f0e4d000640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c0052e0 0x7f0e2c00b9c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0e40001e20 tx=0x7f0e2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca1b95bd099c5f6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.042+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40001a70 con 0x7f0e2c0052e0 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.042+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e4002ae40 con 0x7f0e2c0052e0 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.042+0000 7f0e46a00640 1 -- 172.21.15.165:0/3487826854 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40003bb0 con 0x7f0e2c0052e0 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 -- 172.21.15.165:0/3487826854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c0052e0 msgr2=0x7f0e2c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c0052e0 0x7f0e2c00b9c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f0e40001e20 tx=0x7f0e2c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 -- 172.21.15.165:0/3487826854 shutdown_connections 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 --2- 172.21.15.165:0/3487826854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e2c0052e0 0x7f0e2c00b9c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 -- 172.21.15.165:0/3487826854 >> 172.21.15.165:0/3487826854 conn(0x7f0e48061cc0 msgr2=0x7f0e4806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 -- 172.21.15.165:0/3487826854 shutdown_connections 2024-04-25T10:08:10.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.043+0000 7f0e4da00640 1 -- 172.21.15.165:0/3487826854 wait complete. 2024-04-25T10:08:10.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.045+0000 7f0e4da00640 1 Processor -- start 2024-04-25T10:08:10.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.045+0000 7f0e4da00640 1 -- start start 2024-04-25T10:08:10.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0052e0 0x7f0e481e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e48142e50 0x7f0e481e4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 0x7f0e4814d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c5620 con 0x7f0e481417f0 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c54a0 con 0x7f0e48142e50 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e480c4af0 con 0x7f0e2c0052e0 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 0x7f0e4814d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 0x7f0e4814d490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51514/0 (socket says 172.21.15.165:51514) 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.046+0000 7f0e4d000640 1 -- 172.21.15.165:0/1047804221 learned_addr learned my addr 172.21.15.165:0/1047804221 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e47e00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0052e0 0x7f0e481e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e47400640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e48142e50 0x7f0e481e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e4d000640 1 -- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0052e0 msgr2=0x7f0e481e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e4d000640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0052e0 0x7f0e481e3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e4d000640 1 -- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e48142e50 msgr2=0x7f0e481e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e4d000640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e48142e50 0x7f0e481e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e4d000640 1 -- 172.21.15.165:0/1047804221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e48141db0 con 0x7f0e481417f0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.047+0000 7f0e4d000640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 0x7f0e4814d490 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e4002a8f0 tx=0x7f0e400014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.048+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40041070 con 0x7f0e481417f0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.048+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e40005710 con 0x7f0e481417f0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.048+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e40007db0 con 0x7f0e481417f0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.048+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e48142090 con 0x7f0e481417f0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.048+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e48142650 con 0x7f0e481417f0 2024-04-25T10:08:10.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.049+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e48073990 con 0x7f0e481417f0 2024-04-25T10:08:10.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.050+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0e40044070 con 0x7f0e481417f0 2024-04-25T10:08:10.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.051+0000 7f0e3fe00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e2806b500 0x7f0e2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.051+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0e400a9560 con 0x7f0e481417f0 2024-04-25T10:08:10.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.051+0000 7f0e47e00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e2806b500 0x7f0e2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.052+0000 7f0e47e00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e2806b500 0x7f0e2806d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e38001050 tx=0x7f0e38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.056+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0e40039070 con 0x7f0e481417f0 2024-04-25T10:08:10.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.263+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f0e4806b8a0 con 0x7f0e481417f0 2024-04-25T10:08:10.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.271+0000 7f0e3fe00640 1 -- 172.21.15.165:0/1047804221 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0e40077e10 con 0x7f0e481417f0 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.273+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e2806b500 msgr2=0x7f0e2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e2806b500 0x7f0e2806d9c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0e38001050 tx=0x7f0e38009040 comp rx=0 tx=0).stop 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 msgr2=0x7f0e4814d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 0x7f0e4814d490 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f0e4002a8f0 tx=0x7f0e400014a0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 shutdown_connections 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0e2806b500 0x7f0e2806d9c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0e481417f0 0x7f0e4814d490 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0e48142e50 0x7f0e481e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 --2- 172.21.15.165:0/1047804221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0e2c0052e0 0x7f0e481e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 >> 172.21.15.165:0/1047804221 conn(0x7f0e48061cc0 msgr2=0x7f0e480cdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 shutdown_connections 2024-04-25T10:08:10.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.274+0000 7f0e4da00640 1 -- 172.21.15.165:0/1047804221 wait complete. 2024-04-25T10:08:10.300 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair' 2024-04-25T10:08:10.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.487+0000 7f8eb5400640 1 Processor -- start 2024-04-25T10:08:10.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.488+0000 7f8eb5400640 1 -- start start 2024-04-25T10:08:10.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.489+0000 7f8eb5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8eb006e870 0x7f8eb00cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.489+0000 7f8eb5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb00cc840 0x7f8eb015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.489+0000 7f8eb5400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb0144240 con 0x7f8eb0143e60 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.489+0000 7f8eb5400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eb015e220 con 0x7f8eb006e870 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.489+0000 7f8eb5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb015e500 con 0x7f8eb00cc840 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8eb006e870 0x7f8eb00cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8eb006e870 0x7f8eb00cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51530/0 (socket says 172.21.15.165:51530) 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 -- 172.21.15.165:0/2685277477 learned_addr learned my addr 172.21.15.165:0/2685277477 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eb4a00640 1 --2- 172.21.15.165:0/2685277477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb00cc840 0x7f8eb015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 -- 172.21.15.165:0/2685277477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb00cc840 msgr2=0x7f8eb015ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 --2- 172.21.15.165:0/2685277477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb00cc840 0x7f8eb015ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 -- 172.21.15.165:0/2685277477 >> v1:172.21.15.172:6789/0 conn(0x7f8eb0143e60 legacy=0x7f8eb006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:10.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.490+0000 7f8eaf400640 1 -- 172.21.15.165:0/2685277477 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb015e7e0 con 0x7f8eb006e870 2024-04-25T10:08:10.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.491+0000 7f8eaf400640 1 --2- 172.21.15.165:0/2685277477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb006e870 0x7f8eb00cc300 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8eb006b970 tx=0x7f8e9802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0389dfe6939a904 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.491+0000 7f8eaea00640 1 -- 172.21.15.165:0/2685277477 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98037070 con 0x7f8eb006e870 2024-04-25T10:08:10.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eaea00640 1 -- 172.21.15.165:0/2685277477 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e98032440 con 0x7f8eb006e870 2024-04-25T10:08:10.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eaea00640 1 -- 172.21.15.165:0/2685277477 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98032810 con 0x7f8eb006e870 2024-04-25T10:08:10.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 -- 172.21.15.165:0/2685277477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb006e870 msgr2=0x7f8eb00cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 --2- 172.21.15.165:0/2685277477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb006e870 0x7f8eb00cc300 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8eb006b970 tx=0x7f8e9802a200 comp rx=0 tx=0).stop 2024-04-25T10:08:10.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 -- 172.21.15.165:0/2685277477 shutdown_connections 2024-04-25T10:08:10.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 --2- 172.21.15.165:0/2685277477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb00cc840 0x7f8eb015ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 --2- 172.21.15.165:0/2685277477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb006e870 0x7f8eb00cc300 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 -- 172.21.15.165:0/2685277477 >> 172.21.15.165:0/2685277477 conn(0x7f8eb006abe0 msgr2=0x7f8eb006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 -- 172.21.15.165:0/2685277477 shutdown_connections 2024-04-25T10:08:10.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.492+0000 7f8eb5400640 1 -- 172.21.15.165:0/2685277477 wait complete. 2024-04-25T10:08:10.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.494+0000 7f8eb5400640 1 Processor -- start 2024-04-25T10:08:10.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.494+0000 7f8eb5400640 1 -- start start 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eb5400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb006e870 0x7f8eb0140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eb5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 0x7f8eb00a94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eb5400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb0143e60 0x7f8eb0142780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eb5400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b2eb0 con 0x7f8eb00cc840 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eb5400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b2d30 con 0x7f8eb0143e60 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eb5400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00ccc50 con 0x7f8eb006e870 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.495+0000 7f8eaf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 0x7f8eb00a94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 0x7f8eb00a94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51532/0 (socket says 172.21.15.165:51532) 2024-04-25T10:08:10.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 -- 172.21.15.165:0/3560568689 learned_addr learned my addr 172.21.15.165:0/3560568689 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eafe00640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb006e870 0x7f8eb0140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 -- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb006e870 msgr2=0x7f8eb0140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb006e870 0x7f8eb0140fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eb4a00640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb0143e60 0x7f8eb0142780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 -- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb0143e60 msgr2=0x7f8eb0142780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb0143e60 0x7f8eb0142780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 -- 172.21.15.165:0/3560568689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb00b14b0 con 0x7f8eb00cc840 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eafe00640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb006e870 0x7f8eb0140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:10.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.496+0000 7f8eaf400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 0x7f8eb00a94f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.497+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98041070 con 0x7f8eb00cc840 2024-04-25T10:08:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.497+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e9803c890 con 0x7f8eb00cc840 2024-04-25T10:08:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.497+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb00b1790 con 0x7f8eb00cc840 2024-04-25T10:08:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.497+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e98037040 con 0x7f8eb00cc840 2024-04-25T10:08:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.497+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb00b0fa0 con 0x7f8eb00cc840 2024-04-25T10:08:10.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.498+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb00b1230 con 0x7f8eb00cc840 2024-04-25T10:08:10.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.499+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8e98044070 con 0x7f8eb00cc840 2024-04-25T10:08:10.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.500+0000 7f8ea7e00640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:10.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.500+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8e980aaee0 con 0x7f8eb00cc840 2024-04-25T10:08:10.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.500+0000 7f8eafe00640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:10.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.501+0000 7f8eafe00640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0001dc0 tx=0x7f8ea0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:10.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.505+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8eb00b1230 con 0x7f8eb00cc840 2024-04-25T10:08:10.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.781+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]} v 0) v1 -- 0x7f8eb0073990 con 0x7f8eb00cc840 2024-04-25T10:08:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.838+0000 7f8ea7e00640 1 -- 172.21.15.165:0/3560568689 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_scrub_repair"]}]=0 v316) v1 ==== 143+0+0 (secure 0 0 0) 0x7f8e98079f00 con 0x7f8eb00cc840 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8e8c06b4d0 msgr2=0x7f8e8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0001dc0 tx=0x7f8ea0009040 comp rx=0 tx=0).stop 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 msgr2=0x7f8eb00a94f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 0x7f8eb00a94f0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8e98000c00 tx=0x7f8e980046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 shutdown_connections 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8e8c06b4d0 0x7f8e8c06d990 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8eb0143e60 0x7f8eb0142780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8eb00cc840 0x7f8eb00a94f0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 --2- 172.21.15.165:0/3560568689 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8eb006e870 0x7f8eb0140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 >> 172.21.15.165:0/3560568689 conn(0x7f8eb006abe0 msgr2=0x7f8eb01656d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 shutdown_connections 2024-04-25T10:08:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:10.842+0000 7f8eb5400640 1 -- 172.21.15.165:0/3560568689 wait complete. 2024-04-25T10:08:10.868 INFO:tasks.cephfs_test_runner:test_scrub_repair (tasks.cephfs.test_scrub_checks.TestScrubChecks) ... ok 2024-04-25T10:08:10.868 INFO:tasks.cephfs_test_runner:Starting test: test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) test that scrub can iterate over ~mdsdir and evaluate strays 2024-04-25T10:08:10.868 INFO:tasks.cephfs_test_runner:test_stray_evaluation_with_scrub (tasks.cephfs.test_scrub_checks.TestScrubChecks) 2024-04-25T10:08:10.869 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.057+0000 7ff0ba400640 1 Processor -- start 2024-04-25T10:08:11.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.057+0000 7ff0ba400640 1 -- start start 2024-04-25T10:08:11.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0ba400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff0b415e900 0x7ff0b40adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0ba400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0b40ae0d0 0x7ff0b4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0ba400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0b415ed90 con 0x7ff0b415e520 2024-04-25T10:08:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0ba400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b40aee30 con 0x7ff0b40ae0d0 2024-04-25T10:08:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0ba400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0b40ae590 con 0x7ff0b415e900 2024-04-25T10:08:11.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0b9a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0b40ae0d0 0x7ff0b4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0b9a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0b40ae0d0 0x7ff0b4070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51536/0 (socket says 172.21.15.165:51536) 2024-04-25T10:08:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.059+0000 7ff0b9a00640 1 -- 172.21.15.165:0/2059067302 learned_addr learned my addr 172.21.15.165:0/2059067302 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.060+0000 7ff0abe00640 1 --2- 172.21.15.165:0/2059067302 >> v2:172.21.15.172:3300/0 conn(0x7ff0b415e900 0x7ff0b40adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.060+0000 7ff0b9a00640 1 -- 172.21.15.165:0/2059067302 >> v2:172.21.15.172:3300/0 conn(0x7ff0b415e900 msgr2=0x7ff0b40adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.060+0000 7ff0b9a00640 1 --2- 172.21.15.165:0/2059067302 >> v2:172.21.15.172:3300/0 conn(0x7ff0b415e900 0x7ff0b40adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.060+0000 7ff0b9a00640 1 -- 172.21.15.165:0/2059067302 >> v1:172.21.15.172:6789/0 conn(0x7ff0b415e520 legacy=0x7ff0b415f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.060+0000 7ff0b9a00640 1 -- 172.21.15.165:0/2059067302 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b40ae870 con 0x7ff0b40ae0d0 2024-04-25T10:08:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.061+0000 7ff0b9a00640 1 --2- 172.21.15.165:0/2059067302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4070ff0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40cf050 tx=0x7ff0ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405de312a6592f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.061+0000 7ff0b3e00640 1 -- 172.21.15.165:0/2059067302 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac036070 con 0x7ff0b40ae0d0 2024-04-25T10:08:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.061+0000 7ff0b3e00640 1 -- 172.21.15.165:0/2059067302 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0ac031600 con 0x7ff0b40ae0d0 2024-04-25T10:08:11.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.061+0000 7ff0b3e00640 1 -- 172.21.15.165:0/2059067302 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac0319d0 con 0x7ff0b40ae0d0 2024-04-25T10:08:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 -- 172.21.15.165:0/2059067302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b40ae0d0 msgr2=0x7ff0b4070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 --2- 172.21.15.165:0/2059067302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4070ff0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40cf050 tx=0x7ff0ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 -- 172.21.15.165:0/2059067302 shutdown_connections 2024-04-25T10:08:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 --2- 172.21.15.165:0/2059067302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4070ff0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 --2- 172.21.15.165:0/2059067302 >> v2:172.21.15.172:3300/0 conn(0x7ff0b415e900 0x7ff0b40adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 -- 172.21.15.165:0/2059067302 >> 172.21.15.165:0/2059067302 conn(0x7ff0b406abe0 msgr2=0x7ff0b406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 -- 172.21.15.165:0/2059067302 shutdown_connections 2024-04-25T10:08:11.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.062+0000 7ff0ba400640 1 -- 172.21.15.165:0/2059067302 wait complete. 2024-04-25T10:08:11.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.064+0000 7ff0ba400640 1 Processor -- start 2024-04-25T10:08:11.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.064+0000 7ff0ba400640 1 -- start start 2024-04-25T10:08:11.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0ba400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4157290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0ba400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 0x7ff0b40ccb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0ba400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0b415e900 0x7ff0b4156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0ba400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b4071530 con 0x7ff0b415e520 2024-04-25T10:08:11.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0ba400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b406d1f0 con 0x7ff0b40ae0d0 2024-04-25T10:08:11.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0ba400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b40716b0 con 0x7ff0b415e900 2024-04-25T10:08:11.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 0x7ff0b40ccb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 0x7ff0b40ccb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51542/0 (socket says 172.21.15.165:51542) 2024-04-25T10:08:11.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0abe00640 1 -- 172.21.15.165:0/3517056692 learned_addr learned my addr 172.21.15.165:0/3517056692 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:11.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0abe00640 1 -- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0b415e900 msgr2=0x7ff0b4156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0b9a00640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0b415e900 0x7ff0b4156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0abe00640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0b415e900 0x7ff0b4156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0abe00640 1 -- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b40ae0d0 msgr2=0x7ff0b4157290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.065+0000 7ff0b9000640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4157290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0abe00640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4157290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0abe00640 1 -- 172.21.15.165:0/3517056692 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4156c80 con 0x7ff0b415e520 2024-04-25T10:08:11.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0b9000640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0abe00640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 0x7ff0b40ccb80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00d5f0 tx=0x7ff09c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0125f0 con 0x7ff0b415e520 2024-04-25T10:08:11.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c019070 con 0x7ff0b415e520 2024-04-25T10:08:11.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0b40cd2f0 con 0x7ff0b415e520 2024-04-25T10:08:11.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c0129d0 con 0x7ff0b415e520 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.066+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b40cd8b0 con 0x7ff0b415e520 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.068+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0b4073990 con 0x7ff0b415e520 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.069+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff09c012b70 con 0x7ff0b415e520 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.069+0000 7ff0b1600640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff09006b4d0 0x7ff09006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.070+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff09c08a4e0 con 0x7ff0b415e520 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.070+0000 7ff0b9000640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff09006b4d0 0x7ff09006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.071+0000 7ff0b9000640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff09006b4d0 0x7ff09006d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff0b415cf00 tx=0x7ff0a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.074+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff09c059470 con 0x7ff0b415e520 2024-04-25T10:08:11.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.266+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0b406bff0 con 0x7ff0b415e520 2024-04-25T10:08:11.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.268+0000 7ff0b1600640 1 -- 172.21.15.165:0/3517056692 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) v1 ==== 74+0+13462 (secure 0 0 0) 0x7ff09c058d70 con 0x7ff0b415e520 2024-04-25T10:08:11.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:11.274 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":77,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:08:09.808397+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/148834032":"2024-04-26T10:08:06.716135+0000","172.21.15.165:6836/148834032":"2024-04-26T10:08:06.716135+0000","172.21.15.165:6837/1883921643":"2024-04-26T10:07:41.073778+0000","172.21.15.165:6835/1883921643":"2024-04-26T10:07:41.073778+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:11.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.270+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff09006b4d0 msgr2=0x7ff09006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.270+0000 7ff0ba400640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff09006b4d0 0x7ff09006d990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff0b415cf00 tx=0x7ff0a4009040 comp rx=0 tx=0).stop 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.270+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 msgr2=0x7ff0b40ccb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.270+0000 7ff0ba400640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 0x7ff0b40ccb80 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7ff09c00d5f0 tx=0x7ff09c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 shutdown_connections 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff09006b4d0 0x7ff09006d990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0b415e900 0x7ff0b4156740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0b415e520 0x7ff0b40ccb80 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 --2- 172.21.15.165:0/3517056692 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0b40ae0d0 0x7ff0b4157290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 >> 172.21.15.165:0/3517056692 conn(0x7ff0b406abe0 msgr2=0x7ff0b415d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 shutdown_connections 2024-04-25T10:08:11.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.271+0000 7ff0ba400640 1 -- 172.21.15.165:0/3517056692 wait complete. 2024-04-25T10:08:11.295 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:11.295 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.485+0000 7f853a800640 1 Processor -- start 2024-04-25T10:08:11.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.485+0000 7f853a800640 1 -- start start 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f853a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f853415d9c0 0x7f853406c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f853a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f85340ce5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f853a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f853406c880 con 0x7f85340ceae0 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f853a800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85340d1a30 con 0x7f853415d9c0 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f853a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853406cb60 con 0x7f85340b9260 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f8539400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f853415d9c0 0x7f853406c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f8539400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f853415d9c0 0x7f853406c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51552/0 (socket says 172.21.15.165:51552) 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f8539400640 1 -- 172.21.15.165:0/2177106160 learned_addr learned my addr 172.21.15.165:0/2177106160 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:11.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.487+0000 7f8538a00640 1 --2- 172.21.15.165:0/2177106160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f85340ce5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.488+0000 7f8539400640 1 -- 172.21.15.165:0/2177106160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 msgr2=0x7f85340ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.488+0000 7f8539400640 1 --2- 172.21.15.165:0/2177106160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f85340ce5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.488+0000 7f8539400640 1 -- 172.21.15.165:0/2177106160 >> v1:172.21.15.172:6789/0 conn(0x7f85340ceae0 legacy=0x7f8534161640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.488+0000 7f8539400640 1 -- 172.21.15.165:0/2177106160 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853406ce40 con 0x7f853415d9c0 2024-04-25T10:08:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.489+0000 7f8539400640 1 --2- 172.21.15.165:0/2177106160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f853415d9c0 0x7f853406c030 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8528009870 tx=0x7f852802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a05c4c7d95d50675 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.489+0000 7f8533e00640 1 -- 172.21.15.165:0/2177106160 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528037070 con 0x7f853415d9c0 2024-04-25T10:08:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.489+0000 7f8533e00640 1 -- 172.21.15.165:0/2177106160 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8528032440 con 0x7f853415d9c0 2024-04-25T10:08:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f8533e00640 1 -- 172.21.15.165:0/2177106160 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8528032810 con 0x7f853415d9c0 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 -- 172.21.15.165:0/2177106160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f853415d9c0 msgr2=0x7f853406c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 --2- 172.21.15.165:0/2177106160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f853415d9c0 0x7f853406c030 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f8528009870 tx=0x7f852802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 -- 172.21.15.165:0/2177106160 shutdown_connections 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 --2- 172.21.15.165:0/2177106160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f85340ce5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 --2- 172.21.15.165:0/2177106160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f853415d9c0 0x7f853406c030 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 -- 172.21.15.165:0/2177106160 >> 172.21.15.165:0/2177106160 conn(0x7f85340d0770 msgr2=0x7f85340d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 -- 172.21.15.165:0/2177106160 shutdown_connections 2024-04-25T10:08:11.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.490+0000 7f853a800640 1 -- 172.21.15.165:0/2177106160 wait complete. 2024-04-25T10:08:11.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.492+0000 7f853a800640 1 Processor -- start 2024-04-25T10:08:11.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.492+0000 7f853a800640 1 -- start start 2024-04-25T10:08:11.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.493+0000 7f853a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f853414cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.493+0000 7f853a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 0x7f85340cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.493+0000 7f853a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f853415d9c0 0x7f85340cb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.493+0000 7f853a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853415dda0 con 0x7f85340ceae0 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.493+0000 7f853a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853406c570 con 0x7f853415d9c0 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.493+0000 7f853a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f853415df20 con 0x7f85340b9260 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 0x7f85340cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 0x7f85340cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51566/0 (socket says 172.21.15.165:51566) 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 -- 172.21.15.165:0/456932118 learned_addr learned my addr 172.21.15.165:0/456932118 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 -- 172.21.15.165:0/456932118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 msgr2=0x7f853414cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f853414cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 -- 172.21.15.165:0/456932118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f853415d9c0 msgr2=0x7f85340cb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f853415d9c0 0x7f85340cb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 -- 172.21.15.165:0/456932118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85340c6fa0 con 0x7f85340ceae0 2024-04-25T10:08:11.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8538a00640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 0x7f85340cad20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f852000d5f0 tx=0x7f852000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.494+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200125f0 con 0x7f85340ceae0 2024-04-25T10:08:11.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.495+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8520019070 con 0x7f85340ceae0 2024-04-25T10:08:11.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.495+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85200129d0 con 0x7f85340ceae0 2024-04-25T10:08:11.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.495+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85340c7280 con 0x7f85340ceae0 2024-04-25T10:08:11.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.495+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85340c7840 con 0x7f85340ceae0 2024-04-25T10:08:11.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.496+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8534073990 con 0x7f85340ceae0 2024-04-25T10:08:11.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.497+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8520012b70 con 0x7f85340ceae0 2024-04-25T10:08:11.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.498+0000 7f8531600640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f851406b4d0 0x7f851406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.498+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f852008a4e0 con 0x7f85340ceae0 2024-04-25T10:08:11.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.498+0000 7f8539400640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f851406b4d0 0x7f851406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.499+0000 7f8539400640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f851406b4d0 0x7f851406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8528034000 tx=0x7f852802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.502+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8520059470 con 0x7f85340ceae0 2024-04-25T10:08:11.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.680+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f85340d15e0 con 0x7f85340ceae0 2024-04-25T10:08:11.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.681+0000 7f8531600640 1 -- 172.21.15.165:0/456932118 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8520058d70 con 0x7f85340ceae0 2024-04-25T10:08:11.686 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.683+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f851406b4d0 msgr2=0x7f851406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.683+0000 7f853a800640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f851406b4d0 0x7f851406d990 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f8528034000 tx=0x7f852802c000 comp rx=0 tx=0).stop 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.683+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 msgr2=0x7f85340cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.683+0000 7f853a800640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 0x7f85340cad20 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f852000d5f0 tx=0x7f852000dab0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 shutdown_connections 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f851406b4d0 0x7f851406d990 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f853415d9c0 0x7f85340cb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85340ceae0 0x7f85340cad20 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 --2- 172.21.15.165:0/456932118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85340b9260 0x7f853414cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 >> 172.21.15.165:0/456932118 conn(0x7f85340d0770 msgr2=0x7f85340b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 shutdown_connections 2024-04-25T10:08:11.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.684+0000 7f853a800640 1 -- 172.21.15.165:0/456932118 wait complete. 2024-04-25T10:08:11.708 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:08:11.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.898+0000 7f4e8b200640 1 Processor -- start 2024-04-25T10:08:11.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.898+0000 7f4e8b200640 1 -- start start 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.899+0000 7f4e8b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e8c071700 0x7f4e8c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.899+0000 7f4e8b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e8c0cd860 0x7f4e8c15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.899+0000 7f4e8b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e8c15bf30 con 0x7f4e8c144440 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e8b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c15c860 con 0x7f4e8c071700 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e8b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e8c15c210 con 0x7f4e8c0cd860 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e8c071700 0x7f4e8c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e8c071700 0x7f4e8c0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51574/0 (socket says 172.21.15.165:51574) 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 -- 172.21.15.165:0/927537320 learned_addr learned my addr 172.21.15.165:0/927537320 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 -- 172.21.15.165:0/927537320 >> v2:172.21.15.172:3300/0 conn(0x7f4e8c0cd860 msgr2=0x7f4e8c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e8a800640 1 --2- 172.21.15.165:0/927537320 >> v2:172.21.15.172:3300/0 conn(0x7f4e8c0cd860 0x7f4e8c15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 --2- 172.21.15.165:0/927537320 >> v2:172.21.15.172:3300/0 conn(0x7f4e8c0cd860 0x7f4e8c15ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 -- 172.21.15.165:0/927537320 >> v1:172.21.15.172:6789/0 conn(0x7f4e8c144440 legacy=0x7f4e8c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e83e00640 1 -- 172.21.15.165:0/927537320 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c15c4f0 con 0x7f4e8c071700 2024-04-25T10:08:11.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.900+0000 7f4e8a800640 1 --2- 172.21.15.165:0/927537320 >> v2:172.21.15.172:3300/0 conn(0x7f4e8c0cd860 0x7f4e8c15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:11.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.901+0000 7f4e83e00640 1 --2- 172.21.15.165:0/927537320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c0cd320 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c145460 tx=0x7f4e7402a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97055621e2a0b248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e89400640 1 -- 172.21.15.165:0/927537320 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74037070 con 0x7f4e8c071700 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e89400640 1 -- 172.21.15.165:0/927537320 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e74032440 con 0x7f4e8c071700 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e89400640 1 -- 172.21.15.165:0/927537320 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e74032810 con 0x7f4e8c071700 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e8b200640 1 -- 172.21.15.165:0/927537320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 msgr2=0x7f4e8c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e8b200640 1 --2- 172.21.15.165:0/927537320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c0cd320 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c145460 tx=0x7f4e7402a2c0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e8b200640 1 -- 172.21.15.165:0/927537320 shutdown_connections 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e8b200640 1 --2- 172.21.15.165:0/927537320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c0cd320 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.902+0000 7f4e8b200640 1 --2- 172.21.15.165:0/927537320 >> v2:172.21.15.172:3300/0 conn(0x7f4e8c0cd860 0x7f4e8c15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.903+0000 7f4e8b200640 1 -- 172.21.15.165:0/927537320 >> 172.21.15.165:0/927537320 conn(0x7f4e8c06abe0 msgr2=0x7f4e8c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.903+0000 7f4e8b200640 1 -- 172.21.15.165:0/927537320 shutdown_connections 2024-04-25T10:08:11.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.903+0000 7f4e8b200640 1 -- 172.21.15.165:0/927537320 wait complete. 2024-04-25T10:08:11.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.904+0000 7f4e8b200640 1 Processor -- start 2024-04-25T10:08:11.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.904+0000 7f4e8b200640 1 -- start start 2024-04-25T10:08:11.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0cd860 0x7f4e8c1f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e8b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e8c144440 0x7f4e8c1f7290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e8b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0cde70 con 0x7f4e8c071700 2024-04-25T10:08:11.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e8b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c0cdcf0 con 0x7f4e8c0cd860 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e8b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e8c15d230 con 0x7f4e8c144440 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.905+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51580/0 (socket says 172.21.15.165:51580) 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 -- 172.21.15.165:0/527024273 learned_addr learned my addr 172.21.15.165:0/527024273 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 -- 172.21.15.165:0/527024273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e8c144440 msgr2=0x7f4e8c1f7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e8c144440 0x7f4e8c1f7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 -- 172.21.15.165:0/527024273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0cd860 msgr2=0x7f4e8c1f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0cd860 0x7f4e8c1f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 -- 172.21.15.165:0/527024273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e8c0cc9f0 con 0x7f4e8c071700 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e89e00640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c15b010 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00b410 tx=0x7f4e7c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.906+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c015020 con 0x7f4e8c071700 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.907+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7c011830 con 0x7f4e8c071700 2024-04-25T10:08:11.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.907+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e8c0cccd0 con 0x7f4e8c071700 2024-04-25T10:08:11.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.907+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7c011c20 con 0x7f4e8c071700 2024-04-25T10:08:11.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.907+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e8c0c9de0 con 0x7f4e8c071700 2024-04-25T10:08:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.908+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e8c073990 con 0x7f4e8c071700 2024-04-25T10:08:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.909+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e7c018020 con 0x7f4e8c071700 2024-04-25T10:08:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.910+0000 7f4e82000640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e6806b4d0 0x7f4e6806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.910+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4e7c089fb0 con 0x7f4e8c071700 2024-04-25T10:08:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.910+0000 7f4e83e00640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e6806b4d0 0x7f4e6806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:11.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.910+0000 7f4e83e00640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e6806b4d0 0x7f4e6806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e74034000 tx=0x7f4e7402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:11.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:11.914+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e7c01f020 con 0x7f4e8c071700 2024-04-25T10:08:12.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.179+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7f4e8c0b42c0 con 0x7f4e8c071700 2024-04-25T10:08:12.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.851+0000 7f4e82000640 1 -- 172.21.15.165:0/527024273 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v318) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4e7c0172c0 con 0x7f4e8c071700 2024-04-25T10:08:12.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.854+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e6806b4d0 msgr2=0x7f4e6806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.854+0000 7f4e8b200640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e6806b4d0 0x7f4e6806d990 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4e74034000 tx=0x7f4e7402c000 comp rx=0 tx=0).stop 2024-04-25T10:08:12.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.854+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 msgr2=0x7f4e8c15b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:12.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.854+0000 7f4e8b200640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c15b010 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f4e7c00b410 tx=0x7f4e7c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 shutdown_connections 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e6806b4d0 0x7f4e6806d990 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e8c144440 0x7f4e8c1f7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e8c0cd860 0x7f4e8c1f6d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 --2- 172.21.15.165:0/527024273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e8c071700 0x7f4e8c15b010 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 >> 172.21.15.165:0/527024273 conn(0x7f4e8c06abe0 msgr2=0x7f4e8c140ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 shutdown_connections 2024-04-25T10:08:12.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:12.855+0000 7f4e8b200640 1 -- 172.21.15.165:0/527024273 wait complete. 2024-04-25T10:08:12.883 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:08:13.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.083+0000 7fb2fe000640 1 Processor -- start 2024-04-25T10:08:13.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.083+0000 7fb2fe000640 1 -- start start 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.084+0000 7fb2fe000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2f806bff0 0x7fb2f80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.084+0000 7fb2fe000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f80b5970 0x7fb2f80d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.084+0000 7fb2fe000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2f815e110 con 0x7fb2f815dd30 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.084+0000 7fb2fe000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2f806d000 con 0x7fb2f806bff0 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.084+0000 7fb2fe000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80b2fd0 con 0x7fb2f80b5970 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2f806bff0 0x7fb2f80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2f806bff0 0x7fb2f80b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51586/0 (socket says 172.21.15.165:51586) 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 -- 172.21.15.165:0/4154276916 learned_addr learned my addr 172.21.15.165:0/4154276916 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 -- 172.21.15.165:0/4154276916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f80b5970 msgr2=0x7fb2f80d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 --2- 172.21.15.165:0/4154276916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f80b5970 0x7fb2f80d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 -- 172.21.15.165:0/4154276916 >> v1:172.21.15.172:6789/0 conn(0x7fb2f815dd30 legacy=0x7fb2f80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.085+0000 7fb2f7e00640 1 -- 172.21.15.165:0/4154276916 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f806c480 con 0x7fb2f806bff0 2024-04-25T10:08:13.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.086+0000 7fb2f7e00640 1 --2- 172.21.15.165:0/4154276916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f806bff0 0x7fb2f80b5430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009870 tx=0x7fb2e402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3847ebf01286c067 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2f7400640 1 -- 172.21.15.165:0/4154276916 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4037070 con 0x7fb2f806bff0 2024-04-25T10:08:13.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2f7400640 1 -- 172.21.15.165:0/4154276916 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e4032440 con 0x7fb2f806bff0 2024-04-25T10:08:13.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2f7400640 1 -- 172.21.15.165:0/4154276916 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4032810 con 0x7fb2f806bff0 2024-04-25T10:08:13.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 -- 172.21.15.165:0/4154276916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f806bff0 msgr2=0x7fb2f80b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 --2- 172.21.15.165:0/4154276916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f806bff0 0x7fb2f80b5430 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4009870 tx=0x7fb2e402a210 comp rx=0 tx=0).stop 2024-04-25T10:08:13.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 -- 172.21.15.165:0/4154276916 shutdown_connections 2024-04-25T10:08:13.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 --2- 172.21.15.165:0/4154276916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f80b5970 0x7fb2f80d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 --2- 172.21.15.165:0/4154276916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f806bff0 0x7fb2f80b5430 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 -- 172.21.15.165:0/4154276916 >> 172.21.15.165:0/4154276916 conn(0x7fb2f80d0610 msgr2=0x7fb2f80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 -- 172.21.15.165:0/4154276916 shutdown_connections 2024-04-25T10:08:13.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.087+0000 7fb2fe000640 1 -- 172.21.15.165:0/4154276916 wait complete. 2024-04-25T10:08:13.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.089+0000 7fb2fe000640 1 Processor -- start 2024-04-25T10:08:13.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.089+0000 7fb2fe000640 1 -- start start 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.090+0000 7fb2fe000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f80b5970 0x7fb2f81600d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.090+0000 7fb2fe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 0x7fb2f8160610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.090+0000 7fb2fe000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f81f39a0 0x7fb2f81f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.090+0000 7fb2fe000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f806cd20 con 0x7fb2f815dd30 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.090+0000 7fb2fe000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f806ca40 con 0x7fb2f80b5970 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.090+0000 7fb2fe000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2f80d26a0 con 0x7fb2f81f39a0 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 0x7fb2f8160610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 0x7fb2f8160610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51590/0 (socket says 172.21.15.165:51590) 2024-04-25T10:08:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3935492941 learned_addr learned my addr 172.21.15.165:0/3935492941 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2fcc00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f80b5970 0x7fb2f81600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2fd600640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f81f39a0 0x7fb2f81f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f81f39a0 msgr2=0x7fb2f81f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f81f39a0 0x7fb2f81f3e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f80b5970 msgr2=0x7fb2f81600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f80b5970 0x7fb2f81600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 -- 172.21.15.165:0/3935492941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8160e80 con 0x7fb2f815dd30 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2fcc00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f80b5970 0x7fb2f81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.091+0000 7fb2f7e00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 0x7fb2f8160610 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000c00 tx=0x7fb2e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.092+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4041070 con 0x7fb2f815dd30 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.092+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2e403c870 con 0x7fb2f815dd30 2024-04-25T10:08:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.092+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4037040 con 0x7fb2f815dd30 2024-04-25T10:08:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.092+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2f81f4e40 con 0x7fb2f815dd30 2024-04-25T10:08:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.092+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2f81f5400 con 0x7fb2f815dd30 2024-04-25T10:08:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.093+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2f8073990 con 0x7fb2f815dd30 2024-04-25T10:08:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.094+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2e4044070 con 0x7fb2f815dd30 2024-04-25T10:08:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.095+0000 7fb2f4c00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d806b4d0 0x7fb2d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.095+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb2e40aac90 con 0x7fb2f815dd30 2024-04-25T10:08:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.095+0000 7fb2fcc00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d806b4d0 0x7fb2d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.096+0000 7fb2fcc00640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d806b4d0 0x7fb2d806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb2f81e66c0 tx=0x7fb2ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.099+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb2e4079b90 con 0x7fb2f815dd30 2024-04-25T10:08:13.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.290+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fb2f80d16c0 con 0x7fb2f815dd30 2024-04-25T10:08:13.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.298+0000 7fb2f4c00640 1 -- 172.21.15.165:0/3935492941 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fb2e4079490 con 0x7fb2f815dd30 2024-04-25T10:08:13.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.300+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d806b4d0 msgr2=0x7fb2d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.300+0000 7fb2fe000640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d806b4d0 0x7fb2d806d990 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb2f81e66c0 tx=0x7fb2ec008040 comp rx=0 tx=0).stop 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.300+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 msgr2=0x7fb2f8160610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.300+0000 7fb2fe000640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 0x7fb2f8160610 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000c00 tx=0x7fb2e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 shutdown_connections 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2d806b4d0 0x7fb2d806d990 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2f81f39a0 0x7fb2f81f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2f815dd30 0x7fb2f8160610 secure :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fb2e4000c00 tx=0x7fb2e40046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 --2- 172.21.15.165:0/3935492941 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2f80b5970 0x7fb2f81600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 >> 172.21.15.165:0/3935492941 conn(0x7fb2f80d0610 msgr2=0x7fb2f80d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 shutdown_connections 2024-04-25T10:08:13.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.301+0000 7fb2fe000640 1 -- 172.21.15.165:0/3935492941 wait complete. 2024-04-25T10:08:13.324 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:08:13.324 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:13.430 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:08:13.544 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:08:13.544 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:08:13.544 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:13.599 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:13.600 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:08:13.600 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:13.656 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:13.656 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:08:13.656 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:13.712 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:08:13.712 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:08:13.712 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:13.769 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:08:13.769 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:08:13.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:13.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.973+0000 7f6608000640 1 Processor -- start 2024-04-25T10:08:13.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.973+0000 7f6608000640 1 -- start start 2024-04-25T10:08:13.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6608000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 0x7f6600162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6608000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f66000b6cb0 0x7f66000b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6608000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6600162690 con 0x7f66000b5a60 2024-04-25T10:08:13.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6608000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66000d3800 con 0x7f66000b6cb0 2024-04-25T10:08:13.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6608000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000ceee0 con 0x7f660015db70 2024-04-25T10:08:13.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6606000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f66000b5a60 0x7f660006c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41062/0 (socket says 172.21.15.165:41062) 2024-04-25T10:08:13.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6606000640 1 -- 172.21.15.165:0/1886073763 learned_addr learned my addr 172.21.15.165:0/1886073763 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:13.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6604c00640 1 --2- 172.21.15.165:0/1886073763 >> v2:172.21.15.172:3300/0 conn(0x7f66000b6cb0 0x7f66000b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.975+0000 7f6605600640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 0x7f6600162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 435262861 0 0) 0x7f6600162690 con 0x7f66000b5a60 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65e8003520 con 0x7f66000b5a60 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4112116475 0 0) 0x7f65e8003520 con 0x7f66000b5a60 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 msgr2=0x7f6600162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 0x7f6600162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 >> v2:172.21.15.172:3300/0 conn(0x7f66000b6cb0 msgr2=0x7f66000b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 --2- 172.21.15.165:0/1886073763 >> v2:172.21.15.172:3300/0 conn(0x7f66000b6cb0 0x7f66000b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000d1d10 con 0x7f66000b5a60 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f65f8001670 con 0x7f66000b5a60 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 >> v1:172.21.15.165:6789/0 conn(0x7f66000b5a60 legacy=0x7f660006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 0x7f65e80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e8003f80 0x7f65e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65e8004d40 0x7f65e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000d3800 con 0x7f65e8004d40 2024-04-25T10:08:13.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000ceee0 con 0x7f65e8003f80 2024-04-25T10:08:13.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.976+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e800b9c0 con 0x7f660015db70 2024-04-25T10:08:13.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6604c00640 1 --2- 172.21.15.165:0/1886073763 >> v2:172.21.15.172:3300/0 conn(0x7f66000b6cb0 0x7f66000b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:08:13.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6606000640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65e8004d40 0x7f65e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6605600640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 0x7f65e80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6604c00640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e8003f80 0x7f65e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6606000640 1 -- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 msgr2=0x7f65e80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6606000640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f660015db70 0x7f65e80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6606000640 1 -- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e8003f80 msgr2=0x7f65e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6606000640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65e8003f80 0x7f65e8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.977+0000 7f6606000640 1 -- 172.21.15.165:0/1886073763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e8003520 con 0x7f65e8004d40 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.979+0000 7f6606000640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65e8004d40 0x7f65e80051c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f65f800c190 tx=0x7f65e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7fcf836e24b39c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.979+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8007c80 con 0x7f65e8004d40 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.979+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f8005af0 con 0x7f65e8004d40 2024-04-25T10:08:13.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.979+0000 7f65ffe00640 1 -- 172.21.15.165:0/1886073763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f8007070 con 0x7f65e8004d40 2024-04-25T10:08:13.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6608000640 1 -- 172.21.15.165:0/1886073763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65e8004d40 msgr2=0x7f65e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6608000640 1 --2- 172.21.15.165:0/1886073763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65e8004d40 0x7f65e80051c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f65f800c190 tx=0x7f65e8005830 comp rx=0 tx=0).stop 2024-04-25T10:08:13.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6606000640 1 -- 172.21.15.165:0/1886073763 reap_dead start 2024-04-25T10:08:13.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6608000640 1 -- 172.21.15.165:0/1886073763 shutdown_connections 2024-04-25T10:08:13.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6608000640 1 -- 172.21.15.165:0/1886073763 >> 172.21.15.165:0/1886073763 conn(0x7f66000d0520 msgr2=0x7f66000d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:13.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6608000640 1 -- 172.21.15.165:0/1886073763 shutdown_connections 2024-04-25T10:08:13.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.980+0000 7f6608000640 1 -- 172.21.15.165:0/1886073763 wait complete. 2024-04-25T10:08:13.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.982+0000 7f6608000640 1 Processor -- start 2024-04-25T10:08:13.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.982+0000 7f6608000640 1 -- start start 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6608000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66000c9650 0x7f66001545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6608000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 0x7f66000cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6608000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66000cdea0 0x7f66000cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6608000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f660015df50 con 0x7f66001532d0 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6608000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66000d22d0 con 0x7f66000cdea0 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6608000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66000ce530 con 0x7f66000c9650 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6604c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 0x7f66000cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6604c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 0x7f66000cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51614/0 (socket says 172.21.15.165:51614) 2024-04-25T10:08:13.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.983+0000 7f6604c00640 1 -- 172.21.15.165:0/88334404 learned_addr learned my addr 172.21.15.165:0/88334404 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:13.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6605600640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66000c9650 0x7f66001545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6604c00640 1 -- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66000c9650 msgr2=0x7f66001545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6604c00640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66000c9650 0x7f66001545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6604c00640 1 -- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66000cdea0 msgr2=0x7f66000cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6604c00640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66000cdea0 0x7f66000cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6604c00640 1 -- 172.21.15.165:0/88334404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6600154b00 con 0x7f66001532d0 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6605600640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66000c9650 0x7f66001545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6604c00640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 0x7f66000cab70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f65f00114b0 tx=0x7f65f0011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f001a030 con 0x7f66001532d0 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f0003990 con 0x7f66001532d0 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f0003b40 con 0x7f66001532d0 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66000cce00 con 0x7f66001532d0 2024-04-25T10:08:13.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.984+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66000cd3c0 con 0x7f66001532d0 2024-04-25T10:08:13.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.985+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6600073990 con 0x7f66001532d0 2024-04-25T10:08:13.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.986+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65f001d070 con 0x7f66001532d0 2024-04-25T10:08:13.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.988+0000 7f65fd600640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f65e406b4d0 0x7f65e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:13.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.988+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f65f008ad80 con 0x7f66001532d0 2024-04-25T10:08:13.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.988+0000 7f6605600640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f65e406b4d0 0x7f65e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:13.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.989+0000 7f6605600640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f65e406b4d0 0x7f65e406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009ab0 tx=0x7f65f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:13.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:13.992+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6600154b00 con 0x7f66001532d0 2024-04-25T10:08:14.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.220+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66000d1820 con 0x7f66001532d0 2024-04-25T10:08:14.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.221+0000 7f65fd600640 1 -- 172.21.15.165:0/88334404 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f65f0059d10 con 0x7f66001532d0 2024-04-25T10:08:14.227 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:14.227 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":49,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5404,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2344122276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2344122276},{"type":"v1","addr":"172.21.15.172:6833","nonce":2344122276}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":5413,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1493503789","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1493503789},{"type":"v1","addr":"172.21.15.172:6835","nonce":1493503789}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":45},{"gid":6858,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2284100388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2284100388},{"type":"v1","addr":"172.21.15.165:6835","nonce":2284100388}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":43},{"gid":7029,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2314931786","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2314931786},{"type":"v1","addr":"172.21.15.165:6837","nonce":2314931786}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":49}],"filesystems":[]} 2024-04-25T10:08:14.228 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 49 2024-04-25T10:08:14.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f65e406b4d0 msgr2=0x7f65e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f65e406b4d0 0x7f65e406d990 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f65f4009ab0 tx=0x7f65f4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 msgr2=0x7f66000cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 0x7f66000cab70 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f65f00114b0 tx=0x7f65f0011970 comp rx=0 tx=0).stop 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 shutdown_connections 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f65e406b4d0 0x7f65e406d990 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f66000cdea0 0x7f66000cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f66001532d0 0x7f66000cab70 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 --2- 172.21.15.165:0/88334404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f66000c9650 0x7f66001545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 >> 172.21.15.165:0/88334404 conn(0x7f66000d0520 msgr2=0x7f66000d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 shutdown_connections 2024-04-25T10:08:14.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.224+0000 7f6608000640 1 -- 172.21.15.165:0/88334404 wait complete. 2024-04-25T10:08:14.247 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:08:14.247 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:08:14.247 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:08:14.247 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:14.247 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:08:14.247 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:08:14.248 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:08:14.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:14.248 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:08:14.248 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:08:14.248 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:08:14.248 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:14.248 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:08:14.248 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:08:14.249 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:08:14.249 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:08:14.264 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:08:14.265 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:08:14.266 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:08:14.266 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:08:14.267 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:08:14.267 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:08:14.269 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:08:14.269 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:08:14.310 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:08:14.310 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:14.314 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:08:14.314 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:14.315 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:08:14.316 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:14.322 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:08:14.322 DEBUG:teuthology.parallel:result is None 2024-04-25T10:08:14.323 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:08:14.450 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:08:14.454 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:08:14.468 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:08:14.472 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:08:14.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.485+0000 7f8d8a800640 1 Processor -- start 2024-04-25T10:08:14.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.485+0000 7f8d8a800640 1 -- start start 2024-04-25T10:08:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.486+0000 7f8d8a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d841446d0 0x7f8d840b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d8a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8d84144d50 0x7f8d840b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d8a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8d841452a0 0x7f8d840cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d8a800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840b2b20 con 0x7f8d841452a0 2024-04-25T10:08:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d8a800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8406d400 con 0x7f8d84144d50 2024-04-25T10:08:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d8a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840b49a0 con 0x7f8d841446d0 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8d841452a0 0x7f8d840cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8d841452a0 0x7f8d840cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51640/0 (socket says 172.21.15.165:51640) 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 -- 172.21.15.165:0/4138240348 learned_addr learned my addr 172.21.15.165:0/4138240348 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d7aa00640 1 --2- 172.21.15.165:0/4138240348 >> v2:172.21.15.172:3300/0 conn(0x7f8d84144d50 0x7f8d840b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83400640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d841446d0 0x7f8d840b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 -- 172.21.15.165:0/4138240348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d841446d0 msgr2=0x7f8d840b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d841446d0 0x7f8d840b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 -- 172.21.15.165:0/4138240348 >> v2:172.21.15.172:3300/0 conn(0x7f8d84144d50 msgr2=0x7f8d840b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 --2- 172.21.15.165:0/4138240348 >> v2:172.21.15.172:3300/0 conn(0x7f8d84144d50 0x7f8d840b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83e00640 1 -- 172.21.15.165:0/4138240348 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840cccc0 con 0x7f8d841452a0 2024-04-25T10:08:14.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.487+0000 7f8d83400640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d841446d0 0x7f8d840b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:14.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.488+0000 7f8d83e00640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840cf640 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8d840d07a0 tx=0x7f8d7c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40843905cf6794dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d82a00640 1 -- 172.21.15.165:0/4138240348 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c036070 con 0x7f8d841452a0 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d82a00640 1 -- 172.21.15.165:0/4138240348 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c031600 con 0x7f8d841452a0 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d82a00640 1 -- 172.21.15.165:0/4138240348 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c0319d0 con 0x7f8d841452a0 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 -- 172.21.15.165:0/4138240348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 msgr2=0x7f8d840cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840cf640 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8d840d07a0 tx=0x7f8d7c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 -- 172.21.15.165:0/4138240348 shutdown_connections 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840cf640 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 --2- 172.21.15.165:0/4138240348 >> v2:172.21.15.172:3300/0 conn(0x7f8d84144d50 0x7f8d840b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 --2- 172.21.15.165:0/4138240348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d841446d0 0x7f8d840b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.489+0000 7f8d8a800640 1 -- 172.21.15.165:0/4138240348 >> 172.21.15.165:0/4138240348 conn(0x7f8d8406abe0 msgr2=0x7f8d8406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.490+0000 7f8d8a800640 1 -- 172.21.15.165:0/4138240348 shutdown_connections 2024-04-25T10:08:14.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.490+0000 7f8d8a800640 1 -- 172.21.15.165:0/4138240348 wait complete. 2024-04-25T10:08:14.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.491+0000 7f8d8a800640 1 Processor -- start 2024-04-25T10:08:14.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.491+0000 7f8d8a800640 1 -- start start 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d8a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d841446d0 0x7f8d840c5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d8a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d84144d50 0x7f8d84140230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d8a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d8a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cc700 con 0x7f8d841452a0 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d8a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cc420 con 0x7f8d841446d0 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d8a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840cc9e0 con 0x7f8d84144d50 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840c78f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51648/0 (socket says 172.21.15.165:51648) 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.492+0000 7f8d83e00640 1 -- 172.21.15.165:0/2489723569 learned_addr learned my addr 172.21.15.165:0/2489723569 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83400640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d841446d0 0x7f8d840c5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d7aa00640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d84144d50 0x7f8d84140230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83e00640 1 -- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d84144d50 msgr2=0x7f8d84140230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83e00640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d84144d50 0x7f8d84140230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83e00640 1 -- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d841446d0 msgr2=0x7f8d840c5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83e00640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d841446d0 0x7f8d840c5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83e00640 1 -- 172.21.15.165:0/2489723569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840b19f0 con 0x7f8d841452a0 2024-04-25T10:08:14.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83400640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d841446d0 0x7f8d840c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:14.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d7aa00640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d84144d50 0x7f8d84140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:08:14.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d83e00640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840c78f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009f40 tx=0x7f8d7c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.493+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c041020 con 0x7f8d841452a0 2024-04-25T10:08:14.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.494+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d7c0077a0 con 0x7f8d841452a0 2024-04-25T10:08:14.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.494+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7c036040 con 0x7f8d841452a0 2024-04-25T10:08:14.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.494+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d840c9d10 con 0x7f8d841452a0 2024-04-25T10:08:14.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.494+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d840c7e30 con 0x7f8d841452a0 2024-04-25T10:08:14.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.495+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d84073a00 con 0x7f8d841452a0 2024-04-25T10:08:14.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.496+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d7c044020 con 0x7f8d841452a0 2024-04-25T10:08:14.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.496+0000 7f8d7be00640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8d6006b4d0 0x7f8d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:14.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.497+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8d7c0aadf0 con 0x7f8d841452a0 2024-04-25T10:08:14.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.497+0000 7f8d83400640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8d6006b4d0 0x7f8d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:14.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.497+0000 7f8d83400640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8d6006b4d0 0x7f8d6006d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c940 tx=0x7f8d74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:14.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.501+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d7c04b030 con 0x7f8d841452a0 2024-04-25T10:08:14.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.703+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8d840c9ae0 con 0x7f8d841452a0 2024-04-25T10:08:14.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.875+0000 7f8d7be00640 1 -- 172.21.15.165:0/2489723569 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v79) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8d7c079db0 con 0x7f8d841452a0 2024-04-25T10:08:14.881 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8d6006b4d0 msgr2=0x7f8d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8d6006b4d0 0x7f8d6006d990 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f8d7400c940 tx=0x7f8d74008040 comp rx=0 tx=0).stop 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 msgr2=0x7f8d840c78f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840c78f0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f8d7c009f40 tx=0x7f8d7c004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 shutdown_connections 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8d6006b4d0 0x7f8d6006d990 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d841452a0 0x7f8d840c78f0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d84144d50 0x7f8d84140230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 --2- 172.21.15.165:0/2489723569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d841446d0 0x7f8d840c5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:14.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 >> 172.21.15.165:0/2489723569 conn(0x7f8d8406abe0 msgr2=0x7f8d840acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:14.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.878+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 shutdown_connections 2024-04-25T10:08:14.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:14.879+0000 7f8d8a800640 1 -- 172.21.15.165:0/2489723569 wait complete. 2024-04-25T10:08:14.904 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:08:15.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.084+0000 7f5b48200640 1 Processor -- start 2024-04-25T10:08:15.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.084+0000 7f5b48200640 1 -- start start 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.085+0000 7f5b48200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5b4015af20 0x7f5b400b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.085+0000 7f5b48200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5b4015b300 0x7f5b4015b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.085+0000 7f5b48200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b400b4c10 con 0x7f5b4006dfb0 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.085+0000 7f5b48200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b400b2800 con 0x7f5b4015b300 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.085+0000 7f5b48200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b400b2ae0 con 0x7f5b4015af20 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b44e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5b4006dfb0 0x7f5b400b43c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41070/0 (socket says 172.21.15.165:41070) 2024-04-25T10:08:15.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b44e00640 1 -- 172.21.15.165:0/3449140476 learned_addr learned my addr 172.21.15.165:0/3449140476 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:15.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2153159881 0 0) 0x7f5b400b4c10 con 0x7f5b4006dfb0 2024-04-25T10:08:15.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b28005730 con 0x7f5b4006dfb0 2024-04-25T10:08:15.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b45800640 1 --2- 172.21.15.165:0/3449140476 >> v2:172.21.15.172:3300/0 conn(0x7f5b4015af20 0x7f5b400b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b46200640 1 --2- 172.21.15.165:0/3449140476 >> v2:172.21.15.165:3300/0 conn(0x7f5b4015b300 0x7f5b4015b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2232201285 0 0) 0x7f5b28005730 con 0x7f5b4006dfb0 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 >> v2:172.21.15.172:3300/0 conn(0x7f5b4015af20 msgr2=0x7f5b400b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 --2- 172.21.15.165:0/3449140476 >> v2:172.21.15.172:3300/0 conn(0x7f5b4015af20 0x7f5b400b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 >> v2:172.21.15.165:3300/0 conn(0x7f5b4015b300 msgr2=0x7f5b4015b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 --2- 172.21.15.165:0/3449140476 >> v2:172.21.15.165:3300/0 conn(0x7f5b4015b300 0x7f5b4015b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b400b2dc0 con 0x7f5b4006dfb0 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.086+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5b2c001670 con 0x7f5b4006dfb0 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 >> v1:172.21.15.165:6789/0 conn(0x7f5b4006dfb0 legacy=0x7f5b400b43c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b28005bc0 0x7f5b28005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b28006a10 0x7f5b28006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b2800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b2800 con 0x7f5b280073d0 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b2ae0 con 0x7f5b28005bc0 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2800e2d0 con 0x7f5b28006a10 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b46200640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b2800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b44e00640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b28006a10 0x7f5b28006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b45800640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b28005bc0 0x7f5b28005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b46200640 1 -- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b28006a10 msgr2=0x7f5b28006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b46200640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b28006a10 0x7f5b28006e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b46200640 1 -- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b28005bc0 msgr2=0x7f5b28005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b46200640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b28005bc0 0x7f5b28005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b46200640 1 -- 172.21.15.165:0/3449140476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28005730 con 0x7f5b280073d0 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.087+0000 7f5b45800640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b28005bc0 0x7f5b28005fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.088+0000 7f5b46200640 1 -- 172.21.15.165:0/3449140476 reap_dead start 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.088+0000 7f5b46200640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b2800dbd0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5b38007360 tx=0x7f5b28007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e79b673d78cdcea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38009be0 con 0x7f5b280073d0 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b38003b90 con 0x7f5b280073d0 2024-04-25T10:08:15.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b3fe00640 1 -- 172.21.15.165:0/3449140476 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b38004020 con 0x7f5b280073d0 2024-04-25T10:08:15.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 -- 172.21.15.165:0/3449140476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 msgr2=0x7f5b2800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b2800dbd0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f5b38007360 tx=0x7f5b28007a40 comp rx=0 tx=0).stop 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 -- 172.21.15.165:0/3449140476 shutdown_connections 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 --2- 172.21.15.165:0/3449140476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b2800dbd0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 -- 172.21.15.165:0/3449140476 >> 172.21.15.165:0/3449140476 conn(0x7f5b4006abe0 msgr2=0x7f5b4006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 -- 172.21.15.165:0/3449140476 shutdown_connections 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.089+0000 7f5b48200640 1 -- 172.21.15.165:0/3449140476 wait complete. 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.091+0000 7f5b48200640 1 Processor -- start 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.091+0000 7f5b48200640 1 -- start start 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b48200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b40159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b48200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b401595b0 0x7f5b401ee790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b48200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b401eecd0 0x7f5b401ef150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b48200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b3740 con 0x7f5b280073d0 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b48200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b35c0 con 0x7f5b401eecd0 2024-04-25T10:08:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b48200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b400b38c0 con 0x7f5b401595b0 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.092+0000 7f5b45800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b40159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b40159070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51668/0 (socket says 172.21.15.165:51668) 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 -- 172.21.15.165:0/1233139045 learned_addr learned my addr 172.21.15.165:0/1233139045 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 -- 172.21.15.165:0/1233139045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b401595b0 msgr2=0x7f5b401ee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b401595b0 0x7f5b401ee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 -- 172.21.15.165:0/1233139045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b401eecd0 msgr2=0x7f5b401ef150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b401eecd0 0x7f5b401ef150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 -- 172.21.15.165:0/1233139045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40144b20 con 0x7f5b280073d0 2024-04-25T10:08:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b45800640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b40159070 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e7e0 tx=0x7f5b3400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.093+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b34017070 con 0x7f5b280073d0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.094+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400c6d0 con 0x7f5b280073d0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.094+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400f040 con 0x7f5b280073d0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.094+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b40146ea0 con 0x7f5b280073d0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.094+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b40147460 con 0x7f5b280073d0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.095+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b400739e0 con 0x7f5b280073d0 2024-04-25T10:08:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.096+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b3401a070 con 0x7f5b280073d0 2024-04-25T10:08:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.096+0000 7f5b3d600640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b2006b4a0 0x7f5b2006d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.097+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5b34089c60 con 0x7f5b280073d0 2024-04-25T10:08:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.097+0000 7f5b44e00640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b2006b4a0 0x7f5b2006d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.097+0000 7f5b44e00640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b2006b4a0 0x7f5b2006d960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0017e0 tx=0x7f5b2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.100+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b34058c00 con 0x7f5b280073d0 2024-04-25T10:08:15.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.345+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5b400afb70 con 0x7f5b280073d0 2024-04-25T10:08:15.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.346+0000 7f5b3d600640 1 -- 172.21.15.165:0/1233139045 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5b34058500 con 0x7f5b280073d0 2024-04-25T10:08:15.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.352 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:08:15.353 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:08:15.354 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.355 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.356 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:08:15.357 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.358 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:08:15.360 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:08:15.361 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.362 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:08:15.363 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:08:15.364 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:08:15.365 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b2006b4a0 msgr2=0x7f5b2006d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b2006b4a0 0x7f5b2006d960 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c0017e0 tx=0x7f5b2c009040 comp rx=0 tx=0).stop 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 msgr2=0x7f5b40159070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b40159070 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f5b3400e7e0 tx=0x7f5b3400eca0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 shutdown_connections 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b2006b4a0 0x7f5b2006d960 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b401eecd0 0x7f5b401ef150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b401595b0 0x7f5b401ee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 --2- 172.21.15.165:0/1233139045 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b280073d0 0x7f5b40159070 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 >> 172.21.15.165:0/1233139045 conn(0x7f5b4006abe0 msgr2=0x7f5b4014fa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 shutdown_connections 2024-04-25T10:08:15.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.348+0000 7f5b48200640 1 -- 172.21.15.165:0/1233139045 wait complete. 2024-04-25T10:08:15.371 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:15.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.561+0000 7f6e3c200640 1 Processor -- start 2024-04-25T10:08:15.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.561+0000 7f6e3c200640 1 -- start start 2024-04-25T10:08:15.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e3c200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e340c6df0 0x7f6e340d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e3c200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e3415dfe0 con 0x7f6e340d1c00 2024-04-25T10:08:15.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e3c200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e340d00c0 con 0x7f6e3415dc00 2024-04-25T10:08:15.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e3c200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e340d2070 con 0x7f6e340c6df0 2024-04-25T10:08:15.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e38e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e340c6df0 0x7f6e340d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e3a200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6e340d1c00 0x7f6e340ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41072/0 (socket says 172.21.15.165:41072) 2024-04-25T10:08:15.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.563+0000 7f6e3a200640 1 -- 172.21.15.165:0/1176703028 learned_addr learned my addr 172.21.15.165:0/1176703028 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:15.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.564+0000 7f6e33e00640 1 -- 172.21.15.165:0/1176703028 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015046234 0 0) 0x7f6e2c001490 con 0x7f6e340d1c00 2024-04-25T10:08:15.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.564+0000 7f6e33e00640 1 -- 172.21.15.165:0/1176703028 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e18003520 con 0x7f6e340d1c00 2024-04-25T10:08:15.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.564+0000 7f6e38e00640 1 -- 172.21.15.165:0/1176703028 >> v1:172.21.15.172:6789/0 conn(0x7f6e3415dc00 legacy=0x7f6e340ceb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.564+0000 7f6e38e00640 1 -- 172.21.15.165:0/1176703028 >> v1:172.21.15.165:6789/0 conn(0x7f6e340d1c00 legacy=0x7f6e340ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.564+0000 7f6e38e00640 1 -- 172.21.15.165:0/1176703028 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340d03a0 con 0x7f6e340c6df0 2024-04-25T10:08:15.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.565+0000 7f6e38e00640 1 --2- 172.21.15.165:0/1176703028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340c6df0 0x7f6e340d16c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6e200097a0 tx=0x7f6e2002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa9d75c864cc61a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e33e00640 1 -- 172.21.15.165:0/1176703028 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20037070 con 0x7f6e340c6df0 2024-04-25T10:08:15.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e33e00640 1 -- 172.21.15.165:0/1176703028 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20032440 con 0x7f6e340c6df0 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e33e00640 1 -- 172.21.15.165:0/1176703028 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20032810 con 0x7f6e340c6df0 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e3c200640 1 -- 172.21.15.165:0/1176703028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340c6df0 msgr2=0x7f6e340d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e3c200640 1 --2- 172.21.15.165:0/1176703028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340c6df0 0x7f6e340d16c0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6e200097a0 tx=0x7f6e2002a230 comp rx=0 tx=0).stop 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e3c200640 1 -- 172.21.15.165:0/1176703028 shutdown_connections 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e3c200640 1 --2- 172.21.15.165:0/1176703028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340c6df0 0x7f6e340d16c0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.566+0000 7f6e3c200640 1 -- 172.21.15.165:0/1176703028 >> 172.21.15.165:0/1176703028 conn(0x7f6e3406f5d0 msgr2=0x7f6e3406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.567+0000 7f6e3c200640 1 -- 172.21.15.165:0/1176703028 shutdown_connections 2024-04-25T10:08:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.567+0000 7f6e3c200640 1 -- 172.21.15.165:0/1176703028 wait complete. 2024-04-25T10:08:15.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.568+0000 7f6e3c200640 1 Processor -- start 2024-04-25T10:08:15.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.569+0000 7f6e3c200640 1 -- start start 2024-04-25T10:08:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.569+0000 7f6e3c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e340c6df0 0x7f6e340b6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.569+0000 7f6e3c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 0x7f6e340b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e3c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3415dc00 0x7f6e34148d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e3c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340cc130 con 0x7f6e340d1c00 2024-04-25T10:08:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e3c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340cc720 con 0x7f6e340c6df0 2024-04-25T10:08:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e3c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e340cfd90 con 0x7f6e3415dc00 2024-04-25T10:08:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 0x7f6e340b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 0x7f6e340b1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51688/0 (socket says 172.21.15.165:51688) 2024-04-25T10:08:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 -- 172.21.15.165:0/618803141 learned_addr learned my addr 172.21.15.165:0/618803141 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e39800640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e340c6df0 0x7f6e340b6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 -- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3415dc00 msgr2=0x7f6e34148d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3415dc00 0x7f6e34148d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 -- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e340c6df0 msgr2=0x7f6e340b6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e340c6df0 0x7f6e340b6c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.570+0000 7f6e38e00640 1 -- 172.21.15.165:0/618803141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e340b44a0 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.571+0000 7f6e38e00640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 0x7f6e340b1ca0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6e20035040 tx=0x7f6e200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.571+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20037040 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.571+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e20030500 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.571+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e20030910 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.571+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e340b4780 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.571+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e340c8da0 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.572+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e340b4ca0 con 0x7f6e340d1c00 2024-04-25T10:08:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.573+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e20044020 con 0x7f6e340d1c00 2024-04-25T10:08:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.574+0000 7f6e31600640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1406b4d0 0x7f6e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.574+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f6e200aa3d0 con 0x7f6e340d1c00 2024-04-25T10:08:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.575+0000 7f6e39800640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1406b4d0 0x7f6e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.575+0000 7f6e39800640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1406b4d0 0x7f6e1406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000f60 tx=0x7f6e24009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.579+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e20079380 con 0x7f6e340d1c00 2024-04-25T10:08:15.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.773+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e34073990 con 0x7f6e340d1c00 2024-04-25T10:08:15.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.774+0000 7f6e31600640 1 -- 172.21.15.165:0/618803141 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+13201 (secure 0 0 0) 0x7f6e20078c80 con 0x7f6e340d1c00 2024-04-25T10:08:15.780 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:15.780 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":81,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:08:14.888337+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1406b4d0 msgr2=0x7f6e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1406b4d0 0x7f6e1406d990 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f6e24000f60 tx=0x7f6e24009040 comp rx=0 tx=0).stop 2024-04-25T10:08:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 msgr2=0x7f6e340b1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 0x7f6e340b1ca0 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6e20035040 tx=0x7f6e200046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 shutdown_connections 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e1406b4d0 0x7f6e1406d990 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e3415dc00 0x7f6e34148d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e340d1c00 0x7f6e340b1ca0 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 --2- 172.21.15.165:0/618803141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e340c6df0 0x7f6e340b6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 >> 172.21.15.165:0/618803141 conn(0x7f6e3406f5d0 msgr2=0x7f6e340cd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.777+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 shutdown_connections 2024-04-25T10:08:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:15.778+0000 7f6e3c200640 1 -- 172.21.15.165:0/618803141 wait complete. 2024-04-25T10:08:15.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:08:15.802 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:08:16.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.000+0000 7f2851800640 1 Processor -- start 2024-04-25T10:08:16.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.000+0000 7f2851800640 1 -- start start 2024-04-25T10:08:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f2851800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f284c15b710 0x7f284c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f2851800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 0x7f284c15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f2851800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f284c0cfd80 con 0x7f284c15ee60 2024-04-25T10:08:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f2851800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f284c15e490 con 0x7f284c15b710 2024-04-25T10:08:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f2851800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0d0060 con 0x7f284c06ec50 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f284b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f284c15ee60 0x7f284c0ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41076/0 (socket says 172.21.15.165:41076) 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.002+0000 7f284b400640 1 -- 172.21.15.165:0/226415553 learned_addr learned my addr 172.21.15.165:0/226415553 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284aa00640 1 --2- 172.21.15.165:0/226415553 >> v2:172.21.15.172:3300/0 conn(0x7f284c15b710 0x7f284c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f2842000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 0x7f284c15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3552059580 0 0) 0x7f284c0cfd80 con 0x7f284c15ee60 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2830003520 con 0x7f284c15ee60 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3248472828 0 0) 0x7f2830003520 con 0x7f284c15ee60 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 msgr2=0x7f284c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 0x7f284c15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 >> v2:172.21.15.172:3300/0 conn(0x7f284c15b710 msgr2=0x7f284c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 --2- 172.21.15.165:0/226415553 >> v2:172.21.15.172:3300/0 conn(0x7f284c15b710 0x7f284c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0d0340 con 0x7f284c15ee60 2024-04-25T10:08:16.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f28440019f0 con 0x7f284c15ee60 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 >> v1:172.21.15.165:6789/0 conn(0x7f284c15ee60 legacy=0x7f284c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 0x7f28300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2830003f30 0x7f2830004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f28300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15e490 con 0x7f2830004d40 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0d0060 con 0x7f2830003f30 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.003+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f283000b9c0 con 0x7f284c06ec50 2024-04-25T10:08:16.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284b400640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f28300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284aa00640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 0x7f28300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f2842000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2830003f30 0x7f2830004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284b400640 1 -- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 msgr2=0x7f28300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284b400640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c06ec50 0x7f28300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284b400640 1 -- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2830003f30 msgr2=0x7f2830004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284b400640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2830003f30 0x7f2830004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f284b400640 1 -- 172.21.15.165:0/226415553 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2830003520 con 0x7f2830004d40 2024-04-25T10:08:16.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.004+0000 7f2842000640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2830003f30 0x7f2830004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:16.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.005+0000 7f284b400640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f28300051c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f284400a880 tx=0x7f2830005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1bf94244a31f182 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284400a4a0 con 0x7f2830004d40 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f284402f070 con 0x7f2830004d40 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f284a000640 1 -- 172.21.15.165:0/226415553 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284402bd20 con 0x7f2830004d40 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f2851800640 1 -- 172.21.15.165:0/226415553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 msgr2=0x7f28300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f2851800640 1 --2- 172.21.15.165:0/226415553 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f28300051c0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f284400a880 tx=0x7f2830005830 comp rx=0 tx=0).stop 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f284b400640 1 -- 172.21.15.165:0/226415553 reap_dead start 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f2851800640 1 -- 172.21.15.165:0/226415553 shutdown_connections 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f2851800640 1 -- 172.21.15.165:0/226415553 >> 172.21.15.165:0/226415553 conn(0x7f284c06abe0 msgr2=0x7f284c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f2851800640 1 -- 172.21.15.165:0/226415553 shutdown_connections 2024-04-25T10:08:16.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.006+0000 7f2851800640 1 -- 172.21.15.165:0/226415553 wait complete. 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.008+0000 7f2851800640 1 Processor -- start 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.008+0000 7f2851800640 1 -- start start 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f2851800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f284c160820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f2851800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f284c1f2ca0 0x7f284c0b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f2851800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c0c46d0 0x7f284c0c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f2851800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15bc70 con 0x7f2830004d40 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f2851800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f284c15baf0 con 0x7f284c1f2ca0 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f2851800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f284c0acfc0 con 0x7f284c0c46d0 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.009+0000 7f284aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f284c160820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f284c160820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51694/0 (socket says 172.21.15.165:51694) 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 -- 172.21.15.165:0/1608102391 learned_addr learned my addr 172.21.15.165:0/1608102391 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f2842000640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f284c1f2ca0 0x7f284c0b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 -- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c0c46d0 msgr2=0x7f284c0c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c0c46d0 0x7f284c0c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 -- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f284c1f2ca0 msgr2=0x7f284c0b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f284c1f2ca0 0x7f284c0b03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 -- 172.21.15.165:0/1608102391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f284c0b08f0 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.010+0000 7f284aa00640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f284c160820 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f283c0116d0 tx=0x7f283c011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.011+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00f940 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.011+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c00fae0 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.011+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00fe90 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.011+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f284c1f3120 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.011+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f284c0b0e50 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.012+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f284c073a00 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.013+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f283c015460 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.014+0000 7f2843400640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f282806b670 0x7f282806db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.014+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f283c08a640 con 0x7f2830004d40 2024-04-25T10:08:16.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.014+0000 7f2842000640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f282806b670 0x7f282806db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.015+0000 7f2842000640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f282806b670 0x7f282806db30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f28340099c0 tx=0x7f2834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.018+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f284c0b08f0 con 0x7f2830004d40 2024-04-25T10:08:16.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.214+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f284c06c250 con 0x7f2830004d40 2024-04-25T10:08:16.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.214+0000 7f2843400640 1 -- 172.21.15.165:0/1608102391 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f283c019030 con 0x7f2830004d40 2024-04-25T10:08:16.221 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:08:16.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.217+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f282806b670 msgr2=0x7f282806db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.217+0000 7f2851800640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f282806b670 0x7f282806db30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f28340099c0 tx=0x7f2834008040 comp rx=0 tx=0).stop 2024-04-25T10:08:16.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.217+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 msgr2=0x7f284c160820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.217+0000 7f2851800640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f284c160820 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f283c0116d0 tx=0x7f283c011b90 comp rx=0 tx=0).stop 2024-04-25T10:08:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.217+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 shutdown_connections 2024-04-25T10:08:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.217+0000 7f2851800640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f282806b670 0x7f282806db30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.218+0000 7f2851800640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f284c0c46d0 0x7f284c0c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.218+0000 7f2851800640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f284c1f2ca0 0x7f284c0b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.218+0000 7f2851800640 1 --2- 172.21.15.165:0/1608102391 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2830004d40 0x7f284c160820 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.218+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 >> 172.21.15.165:0/1608102391 conn(0x7f284c06abe0 msgr2=0x7f284c0ad340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:16.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.218+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 shutdown_connections 2024-04-25T10:08:16.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.218+0000 7f2851800640 1 -- 172.21.15.165:0/1608102391 wait complete. 2024-04-25T10:08:16.244 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:08:16.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.428+0000 7f55f5800640 1 Processor -- start 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.428+0000 7f55f5800640 1 -- start start 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55f5800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f55f006b700 0x7f55f0161ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55f5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f015fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55f5800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55f006bb20 con 0x7f55f01477c0 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55f5800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55f006e130 con 0x7f55f006b700 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55f5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f006ef90 con 0x7f55f015f690 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55eea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f55f01477c0 0x7f55f0161790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41086/0 (socket says 172.21.15.165:41086) 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55eea00640 1 -- 172.21.15.165:0/3191377659 learned_addr learned my addr 172.21.15.165:0/3191377659 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55ee000640 1 --2- 172.21.15.165:0/3191377659 >> v2:172.21.15.165:3300/0 conn(0x7f55f006b700 0x7f55f0161ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55ef400640 1 --2- 172.21.15.165:0/3191377659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f015fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.430+0000 7f55ed600640 1 -- 172.21.15.165:0/3191377659 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 46387870 0 0) 0x7f55e0001110 con 0x7f55f01477c0 2024-04-25T10:08:16.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.431+0000 7f55ed600640 1 -- 172.21.15.165:0/3191377659 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55d4003520 con 0x7f55f01477c0 2024-04-25T10:08:16.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.431+0000 7f55ee000640 1 -- 172.21.15.165:0/3191377659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 msgr2=0x7f55f015fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.431+0000 7f55ee000640 1 --2- 172.21.15.165:0/3191377659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f015fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.431+0000 7f55ee000640 1 -- 172.21.15.165:0/3191377659 >> v1:172.21.15.165:6789/0 conn(0x7f55f01477c0 legacy=0x7f55f0161790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.431+0000 7f55ee000640 1 -- 172.21.15.165:0/3191377659 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f006e380 con 0x7f55f006b700 2024-04-25T10:08:16.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.432+0000 7f55ee000640 1 --2- 172.21.15.165:0/3191377659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f006b700 0x7f55f0161ea0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0097a0 tx=0x7f55dc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3faea502c6ed007d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.432+0000 7f55ed600640 1 -- 172.21.15.165:0/3191377659 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc037070 con 0x7f55f006b700 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.432+0000 7f55ed600640 1 -- 172.21.15.165:0/3191377659 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc032440 con 0x7f55f006b700 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.432+0000 7f55ed600640 1 -- 172.21.15.165:0/3191377659 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc032810 con 0x7f55f006b700 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 -- 172.21.15.165:0/3191377659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f006b700 msgr2=0x7f55f0161ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 --2- 172.21.15.165:0/3191377659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f006b700 0x7f55f0161ea0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f55dc0097a0 tx=0x7f55dc02a180 comp rx=0 tx=0).stop 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 -- 172.21.15.165:0/3191377659 shutdown_connections 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 --2- 172.21.15.165:0/3191377659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f015fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 --2- 172.21.15.165:0/3191377659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f006b700 0x7f55f0161ea0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 -- 172.21.15.165:0/3191377659 >> 172.21.15.165:0/3191377659 conn(0x7f55f006abe0 msgr2=0x7f55f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 -- 172.21.15.165:0/3191377659 shutdown_connections 2024-04-25T10:08:16.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.433+0000 7f55f5800640 1 -- 172.21.15.165:0/3191377659 wait complete. 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.434+0000 7f55f5800640 1 Processor -- start 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.435+0000 7f55f5800640 1 -- start start 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.435+0000 7f55f5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f006b700 0x7f55f015d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55f5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 0x7f55f0141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55f5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f0143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55f5800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f006ec20 con 0x7f55f01477c0 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55f5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f006e940 con 0x7f55f006b700 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55f5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55f01623e0 con 0x7f55f015f690 2024-04-25T10:08:16.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 0x7f55f0141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 0x7f55f0141710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51708/0 (socket says 172.21.15.165:51708) 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 -- 172.21.15.165:0/1639636501 learned_addr learned my addr 172.21.15.165:0/1639636501 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 -- 172.21.15.165:0/1639636501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 msgr2=0x7f55f0143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f0143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 -- 172.21.15.165:0/1639636501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f006b700 msgr2=0x7f55f015d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f006b700 0x7f55f015d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.436+0000 7f55ee000640 1 -- 172.21.15.165:0/1639636501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f015dbb0 con 0x7f55f01477c0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.437+0000 7f55ee000640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 0x7f55f0141710 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f55dc035040 tx=0x7f55dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.437+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc037040 con 0x7f55f01477c0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.437+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55dc030500 con 0x7f55f01477c0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.437+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55dc030910 con 0x7f55f01477c0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.437+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f0145d50 con 0x7f55f01477c0 2024-04-25T10:08:16.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.437+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f0140f10 con 0x7f55f01477c0 2024-04-25T10:08:16.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.438+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f0073a00 con 0x7f55f01477c0 2024-04-25T10:08:16.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.439+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f55dc044020 con 0x7f55f01477c0 2024-04-25T10:08:16.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.440+0000 7f55e6a00640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f55d006b4d0 0x7f55d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.440+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f55dc0aa310 con 0x7f55f01477c0 2024-04-25T10:08:16.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.440+0000 7f55eea00640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f55d006b4d0 0x7f55d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.441+0000 7f55eea00640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f55d006b4d0 0x7f55d006d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f55e0000f60 tx=0x7f55e0001110 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.444+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55f015dbb0 con 0x7f55f01477c0 2024-04-25T10:08:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.641+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f55f00685f0 con 0x7f55f01477c0 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.642+0000 7f55e6a00640 1 -- 172.21.15.165:0/1639636501 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f55dc079340 con 0x7f55f01477c0 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:08:16.648 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:08:16.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f55d006b4d0 msgr2=0x7f55d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f55d006b4d0 0x7f55d006d990 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f55e0000f60 tx=0x7f55e0001110 comp rx=0 tx=0).stop 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 msgr2=0x7f55f0141710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 0x7f55f0141710 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f55dc035040 tx=0x7f55dc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 shutdown_connections 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f55d006b4d0 0x7f55d006d990 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55f015f690 0x7f55f0143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f55f01477c0 0x7f55f0141710 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 --2- 172.21.15.165:0/1639636501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f55f006b700 0x7f55f015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 >> 172.21.15.165:0/1639636501 conn(0x7f55f006abe0 msgr2=0x7f55f0160c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 shutdown_connections 2024-04-25T10:08:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.645+0000 7f55f5800640 1 -- 172.21.15.165:0/1639636501 wait complete. 2024-04-25T10:08:16.669 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:08:16.669 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:08:16.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.860+0000 7fc9a9600640 1 Processor -- start 2024-04-25T10:08:16.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.860+0000 7fc9a9600640 1 -- start start 2024-04-25T10:08:16.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.862+0000 7fc9a9600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9a4145140 0x7fc9a40ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.862+0000 7fc9a9600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a40c5650 con 0x7fc9a40c5200 2024-04-25T10:08:16.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.862+0000 7fc9a9600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9a4145520 con 0x7fc9a40ce950 2024-04-25T10:08:16.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.862+0000 7fc9a9600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9a40ced90 con 0x7fc9a4145140 2024-04-25T10:08:16.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.862+0000 7fc9a8c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc9a40c5200 0x7fc9a415bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35842/0 (socket says 172.21.15.165:35842) 2024-04-25T10:08:16.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.862+0000 7fc9a8c00640 1 -- 172.21.15.165:0/3126194710 learned_addr learned my addr 172.21.15.165:0/3126194710 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:16.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a3e00640 1 --2- 172.21.15.165:0/3126194710 >> v2:172.21.15.172:3300/0 conn(0x7fc9a4145140 0x7fc9a40ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4268844044 0 0) 0x7fc9a40c5650 con 0x7fc9a40c5200 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc988003520 con 0x7fc9a40c5200 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1851153835 0 0) 0x7fc988003520 con 0x7fc9a40c5200 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 >> v2:172.21.15.172:3300/0 conn(0x7fc9a4145140 msgr2=0x7fc9a40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 --2- 172.21.15.165:0/3126194710 >> v2:172.21.15.172:3300/0 conn(0x7fc9a4145140 0x7fc9a40ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 >> v1:172.21.15.172:6789/0 conn(0x7fc9a40ce950 legacy=0x7fc9a40c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a4071270 con 0x7fc9a40c5200 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc99c0019f0 con 0x7fc9a40c5200 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 >> v1:172.21.15.165:6789/0 conn(0x7fc9a40c5200 legacy=0x7fc9a415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9880039b0 0x7fc988003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988004800 0x7fc988004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc98800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40ced90 con 0x7fc9880051c0 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800c160 con 0x7fc988004800 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.863+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98800c360 con 0x7fc9880039b0 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a8c00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc98800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a8c00640 1 -- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9880039b0 msgr2=0x7fc988003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a8c00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9880039b0 0x7fc988003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a8c00640 1 -- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988004800 msgr2=0x7fc988004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a3400640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988004800 0x7fc988004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a8c00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988004800 0x7fc988004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a8c00640 1 -- 172.21.15.165:0/3126194710 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988003520 con 0x7fc9880051c0 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.864+0000 7fc9a3400640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc988004800 0x7fc988004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.865+0000 7fc9a8c00640 1 -- 172.21.15.165:0/3126194710 reap_dead start 2024-04-25T10:08:16.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.865+0000 7fc9a8c00640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc98800b9c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00a840 tx=0x7fc988005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47299acfdc403e12 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.865+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c00a460 con 0x7fc9880051c0 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.865+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99c02f070 con 0x7fc9880051c0 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.865+0000 7fc9a2a00640 1 -- 172.21.15.165:0/3126194710 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99c02bcb0 con 0x7fc9880051c0 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 -- 172.21.15.165:0/3126194710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 msgr2=0x7fc98800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc98800b9c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fc99c00a840 tx=0x7fc988005830 comp rx=0 tx=0).stop 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 -- 172.21.15.165:0/3126194710 shutdown_connections 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 --2- 172.21.15.165:0/3126194710 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc98800b9c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 -- 172.21.15.165:0/3126194710 >> 172.21.15.165:0/3126194710 conn(0x7fc9a4061cc0 msgr2=0x7fc9a406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 -- 172.21.15.165:0/3126194710 shutdown_connections 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.866+0000 7fc9a9600640 1 -- 172.21.15.165:0/3126194710 wait complete. 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.868+0000 7fc9a9600640 1 Processor -- start 2024-04-25T10:08:16.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.868+0000 7fc9a9600640 1 -- start start 2024-04-25T10:08:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc9a40cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a9600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a415b3c0 0x7fc9a40cc200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a9600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a415b840 0x7fc9a40b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a9600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4071b10 con 0x7fc9880051c0 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a9600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a4071830 con 0x7fc9a415b840 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a9600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a40ce410 con 0x7fc9a415b3c0 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc9a40cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc9a40cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41120/0 (socket says 172.21.15.165:41120) 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.869+0000 7fc9a3e00640 1 -- 172.21.15.165:0/1679976290 learned_addr learned my addr 172.21.15.165:0/1679976290 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a8c00640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a415b840 0x7fc9a40b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3400640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a415b3c0 0x7fc9a40cc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3e00640 1 -- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a415b3c0 msgr2=0x7fc9a40cc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3e00640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a415b3c0 0x7fc9a40cc200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3e00640 1 -- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a415b840 msgr2=0x7fc9a40b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3e00640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a415b840 0x7fc9a40b38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3e00640 1 -- 172.21.15.165:0/1679976290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40cccc0 con 0x7fc9880051c0 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a8c00640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a415b840 0x7fc9a40b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a3e00640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc9a40cdd60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e7a0 tx=0x7fc99400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc994017070 con 0x7fc9880051c0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc99400c940 con 0x7fc9880051c0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc99400cd10 con 0x7fc9880051c0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a40ccfa0 con 0x7fc9880051c0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.870+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a40cd560 con 0x7fc9880051c0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.872+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a4073990 con 0x7fc9880051c0 2024-04-25T10:08:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.873+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc99401a070 con 0x7fc9880051c0 2024-04-25T10:08:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.873+0000 7fc99be00640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc98406b500 0x7fc98406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.874+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc99408a2a0 con 0x7fc9880051c0 2024-04-25T10:08:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.874+0000 7fc9a3400640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc98406b500 0x7fc98406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:16.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.874+0000 7fc9a3400640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc98406b500 0x7fc98406d9c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc990000f60 tx=0x7fc990009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:16.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:16.878+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9940592d0 con 0x7fc9880051c0 2024-04-25T10:08:17.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.073+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc9a40c4d40 con 0x7fc9880051c0 2024-04-25T10:08:17.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.902+0000 7fc99be00640 1 -- 172.21.15.165:0/1679976290 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc994058bd0 con 0x7fc9880051c0 2024-04-25T10:08:17.909 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:08:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc98406b500 msgr2=0x7fc98406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc98406b500 0x7fc98406d9c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fc990000f60 tx=0x7fc990009040 comp rx=0 tx=0).stop 2024-04-25T10:08:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 msgr2=0x7fc9a40cdd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc9a40cdd60 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fc99400e7a0 tx=0x7fc99400ec60 comp rx=0 tx=0).stop 2024-04-25T10:08:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 shutdown_connections 2024-04-25T10:08:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc98406b500 0x7fc98406d9c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9a415b840 0x7fc9a40b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9a415b3c0 0x7fc9a40cc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 --2- 172.21.15.165:0/1679976290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9880051c0 0x7fc9a40cdd60 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.906+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 >> 172.21.15.165:0/1679976290 conn(0x7fc9a4061cc0 msgr2=0x7fc9a415e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.907+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 shutdown_connections 2024-04-25T10:08:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:17.907+0000 7fc9a9600640 1 -- 172.21.15.165:0/1679976290 wait complete. 2024-04-25T10:08:17.939 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:08:18.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.131+0000 7f1811a00640 1 Processor -- start 2024-04-25T10:08:18.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.131+0000 7f1811a00640 1 -- start start 2024-04-25T10:08:18.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.132+0000 7f1811a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 0x7f180c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.132+0000 7f1811a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c072280 con 0x7f180c15b270 2024-04-25T10:08:18.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f1811a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f180c0714c0 con 0x7f180c0cc4f0 2024-04-25T10:08:18.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f1811a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c071270 con 0x7f180c0ce230 2024-04-25T10:08:18.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f180b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f180c15b270 0x7f180c15b6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35844/0 (socket says 172.21.15.165:35844) 2024-04-25T10:08:18.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f180b400640 1 -- 172.21.15.165:0/2683285528 learned_addr learned my addr 172.21.15.165:0/2683285528 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f1802000640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 0x7f180c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3482622635 0 0) 0x7f180c072280 con 0x7f180c15b270 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.133+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17f0003520 con 0x7f180c15b270 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 472718304 0 0) 0x7f17f0003520 con 0x7f180c15b270 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 msgr2=0x7f180c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 0x7f180c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 >> v1:172.21.15.172:6789/0 conn(0x7f180c0cc4f0 legacy=0x7f180c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0717a0 con 0x7f180c15b270 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f18040019f0 con 0x7f180c15b270 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 >> v1:172.21.15.165:6789/0 conn(0x7f180c15b270 legacy=0x7f180c15b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f17f0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 0x7f17f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17f0004d40 0x7f17f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c071270 con 0x7f17f00039e0 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17f000ba70 con 0x7f17f0004d40 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17f000bc70 con 0x7f180c0ce230 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.134+0000 7f180aa00640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f17f0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.135+0000 7f1802000640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 0x7f17f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.135+0000 7f180aa00640 1 -- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 msgr2=0x7f17f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.135+0000 7f180aa00640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c0ce230 0x7f17f0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.135+0000 7f180aa00640 1 -- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17f0004d40 msgr2=0x7f17f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.135+0000 7f180aa00640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17f0004d40 0x7f17f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.135+0000 7f180aa00640 1 -- 172.21.15.165:0/2683285528 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17f0003520 con 0x7f17f00039e0 2024-04-25T10:08:18.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.136+0000 7f180aa00640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f17f0003e00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f17fc001330 tx=0x7f17f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50cd6d777a672486 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.136+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc001370 con 0x7f17f00039e0 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.136+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc02cd90 con 0x7f17f00039e0 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.136+0000 7f180a000640 1 -- 172.21.15.165:0/2683285528 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc005310 con 0x7f17f00039e0 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f1811a00640 1 -- 172.21.15.165:0/2683285528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 msgr2=0x7f17f0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f1811a00640 1 --2- 172.21.15.165:0/2683285528 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f17f0003e00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f17fc001330 tx=0x7f17f0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f180b400640 1 -- 172.21.15.165:0/2683285528 reap_dead start 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f1811a00640 1 -- 172.21.15.165:0/2683285528 shutdown_connections 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f1811a00640 1 -- 172.21.15.165:0/2683285528 >> 172.21.15.165:0/2683285528 conn(0x7f180c06abe0 msgr2=0x7f180c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f1811a00640 1 -- 172.21.15.165:0/2683285528 shutdown_connections 2024-04-25T10:08:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.137+0000 7f1811a00640 1 -- 172.21.15.165:0/2683285528 wait complete. 2024-04-25T10:08:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.138+0000 7f1811a00640 1 Processor -- start 2024-04-25T10:08:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.139+0000 7f1811a00640 1 -- start start 2024-04-25T10:08:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1811a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f180c0c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1811a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f180c140f20 0x7f180c0c93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1811a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c15c5e0 0x7f180c15ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1811a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c06e8a0 con 0x7f17f00039e0 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1811a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f180c071d60 con 0x7f180c140f20 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1811a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f180c0cc8d0 con 0x7f180c15c5e0 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f180c0c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f180c0c7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41138/0 (socket says 172.21.15.165:41138) 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 -- 172.21.15.165:0/793831574 learned_addr learned my addr 172.21.15.165:0/793831574 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180b400640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c15c5e0 0x7f180c15ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f1802000640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f180c140f20 0x7f180c0c93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 -- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c15c5e0 msgr2=0x7f180c15ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c15c5e0 0x7f180c15ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 -- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f180c140f20 msgr2=0x7f180c0c93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f180c140f20 0x7f180c0c93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.140+0000 7f180aa00640 1 -- 172.21.15.165:0/793831574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c0c9ce0 con 0x7f17f00039e0 2024-04-25T10:08:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.141+0000 7f180aa00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f180c0c7ba0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0017e0 tx=0x7f17fc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.141+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc001370 con 0x7f17f00039e0 2024-04-25T10:08:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.141+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17fc041070 con 0x7f17f00039e0 2024-04-25T10:08:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.141+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f180c0c9fc0 con 0x7f17f00039e0 2024-04-25T10:08:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.141+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17fc007a30 con 0x7f17f00039e0 2024-04-25T10:08:18.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.141+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f180c0ca580 con 0x7f17f00039e0 2024-04-25T10:08:18.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.142+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f180c0d04b0 con 0x7f17f00039e0 2024-04-25T10:08:18.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.144+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f17fc0069e0 con 0x7f17f00039e0 2024-04-25T10:08:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.144+0000 7f1803400640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17e806b500 0x7f17e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.145+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f17fc0aa2f0 con 0x7f17f00039e0 2024-04-25T10:08:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.145+0000 7f1802000640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17e806b500 0x7f17e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.145+0000 7f1802000640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17e806b500 0x7f17e806d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f180c141840 tx=0x7f17f4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:18.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.149+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f180c0d04b0 con 0x7f17f00039e0 2024-04-25T10:08:18.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.343+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f180c073a00 con 0x7f17f00039e0 2024-04-25T10:08:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.909+0000 7f1803400640 1 -- 172.21.15.165:0/793831574 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f17fc079090 con 0x7f17f00039e0 2024-04-25T10:08:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:08:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.913+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17e806b500 msgr2=0x7f17e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.913+0000 7f1811a00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17e806b500 0x7f17e806d9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f180c141840 tx=0x7f17f4008040 comp rx=0 tx=0).stop 2024-04-25T10:08:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.913+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 msgr2=0x7f180c0c7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.913+0000 7f1811a00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f180c0c7ba0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f17fc0017e0 tx=0x7f17fc001120 comp rx=0 tx=0).stop 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 shutdown_connections 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f17e806b500 0x7f17e806d9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f180c15c5e0 0x7f180c15ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f180c140f20 0x7f180c0c93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 --2- 172.21.15.165:0/793831574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f17f00039e0 0x7f180c0c7ba0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 >> 172.21.15.165:0/793831574 conn(0x7f180c06abe0 msgr2=0x7f180c15d700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 shutdown_connections 2024-04-25T10:08:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:18.914+0000 7f1811a00640 1 -- 172.21.15.165:0/793831574 wait complete. 2024-04-25T10:08:18.950 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:08:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.122+0000 7f5504600640 1 Processor -- start 2024-04-25T10:08:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.122+0000 7f5504600640 1 -- start start 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5504600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f54fc1446d0 0x7f54fc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5504600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54fc144d50 0x7f54fc0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5504600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0cccc0 con 0x7f54fc1452a0 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5504600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc0b2b20 con 0x7f54fc144d50 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5504600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc06d400 con 0x7f54fc1446d0 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54fc144d50 0x7f54fc0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54fc144d50 0x7f54fc0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41140/0 (socket says 172.21.15.165:41140) 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501200640 1 -- 172.21.15.165:0/1446709022 learned_addr learned my addr 172.21.15.165:0/1446709022 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5502600640 1 --1- 172.21.15.165:0/1446709022 >> v1:172.21.15.165:6789/0 conn(0x7f54fc1452a0 0x7f54fc0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35854/0 (socket says 172.21.15.165:35854) 2024-04-25T10:08:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501c00640 1 --2- 172.21.15.165:0/1446709022 >> v2:172.21.15.172:3300/0 conn(0x7f54fc1446d0 0x7f54fc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501200640 1 -- 172.21.15.165:0/1446709022 >> v2:172.21.15.172:3300/0 conn(0x7f54fc1446d0 msgr2=0x7f54fc0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501200640 1 --2- 172.21.15.165:0/1446709022 >> v2:172.21.15.172:3300/0 conn(0x7f54fc1446d0 0x7f54fc0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.124+0000 7f5501200640 1 -- 172.21.15.165:0/1446709022 >> v1:172.21.15.165:6789/0 conn(0x7f54fc1452a0 legacy=0x7f54fc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.125+0000 7f5501200640 1 -- 172.21.15.165:0/1446709022 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0cc140 con 0x7f54fc144d50 2024-04-25T10:08:19.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.126+0000 7f5501200640 1 --2- 172.21.15.165:0/1446709022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0b4460 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0cf560 tx=0x7f54e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502f02a63bf30ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.126+0000 7f54fbe00640 1 -- 172.21.15.165:0/1446709022 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8037070 con 0x7f54fc144d50 2024-04-25T10:08:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.126+0000 7f54fbe00640 1 -- 172.21.15.165:0/1446709022 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e8032440 con 0x7f54fc144d50 2024-04-25T10:08:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.126+0000 7f54fbe00640 1 -- 172.21.15.165:0/1446709022 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8032810 con 0x7f54fc144d50 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.126+0000 7f5504600640 1 -- 172.21.15.165:0/1446709022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 msgr2=0x7f54fc0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.126+0000 7f5504600640 1 --2- 172.21.15.165:0/1446709022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0b4460 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f54fc0cf560 tx=0x7f54e802a270 comp rx=0 tx=0).stop 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.127+0000 7f5504600640 1 -- 172.21.15.165:0/1446709022 shutdown_connections 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.127+0000 7f5504600640 1 --2- 172.21.15.165:0/1446709022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0b4460 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.127+0000 7f5504600640 1 --2- 172.21.15.165:0/1446709022 >> v2:172.21.15.172:3300/0 conn(0x7f54fc1446d0 0x7f54fc0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.127+0000 7f5504600640 1 -- 172.21.15.165:0/1446709022 >> 172.21.15.165:0/1446709022 conn(0x7f54fc06abe0 msgr2=0x7f54fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.127+0000 7f5504600640 1 -- 172.21.15.165:0/1446709022 shutdown_connections 2024-04-25T10:08:19.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.127+0000 7f5504600640 1 -- 172.21.15.165:0/1446709022 wait complete. 2024-04-25T10:08:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.128+0000 7f5504600640 1 Processor -- start 2024-04-25T10:08:19.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.129+0000 7f5504600640 1 -- start start 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.129+0000 7f5504600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54fc1446d0 0x7f54fc142cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5504600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0cb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5504600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54fc1452a0 0x7f54fc0b1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5504600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cc700 con 0x7f54fc144d50 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5504600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0b49a0 con 0x7f54fc1446d0 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5504600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0cc9e0 con 0x7f54fc1452a0 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0cb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0cb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41156/0 (socket says 172.21.15.165:41156) 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 -- 172.21.15.165:0/4059438837 learned_addr learned my addr 172.21.15.165:0/4059438837 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 -- 172.21.15.165:0/4059438837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54fc1452a0 msgr2=0x7f54fc0b1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54fc1452a0 0x7f54fc0b1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 -- 172.21.15.165:0/4059438837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54fc1446d0 msgr2=0x7f54fc142cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54fc1446d0 0x7f54fc142cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.130+0000 7f5501200640 1 -- 172.21.15.165:0/4059438837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0c49b0 con 0x7f54fc144d50 2024-04-25T10:08:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.131+0000 7f5501200640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0cb070 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.131+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8041020 con 0x7f54fc144d50 2024-04-25T10:08:19.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.131+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54e80306c0 con 0x7f54fc144d50 2024-04-25T10:08:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.131+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54e8037040 con 0x7f54fc144d50 2024-04-25T10:08:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.131+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc0b0aa0 con 0x7f54fc144d50 2024-04-25T10:08:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.131+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc0b1060 con 0x7f54fc144d50 2024-04-25T10:08:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.132+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc073a00 con 0x7f54fc144d50 2024-04-25T10:08:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.133+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f54e8044020 con 0x7f54fc144d50 2024-04-25T10:08:19.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.134+0000 7f54f9600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54dc06b4d0 0x7f54dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:19.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.134+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f54e80aab50 con 0x7f54fc144d50 2024-04-25T10:08:19.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.134+0000 7f5501c00640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54dc06b4d0 0x7f54dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:19.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.135+0000 7f5501c00640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54dc06b4d0 0x7f54dc06d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f54f0006fd0 tx=0x7f54f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:19.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.139+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f54e804b030 con 0x7f54fc144d50 2024-04-25T10:08:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.371+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f54fc06c250 con 0x7f54fc144d50 2024-04-25T10:08:19.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.954+0000 7f54f9600640 1 -- 172.21.15.165:0/4059438837 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v52) v1 ==== 174+0+0 (secure 0 0 0) 0x7f54e8079890 con 0x7f54fc144d50 2024-04-25T10:08:19.962 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 11 and data pool 12 2024-04-25T10:08:19.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54dc06b4d0 msgr2=0x7f54dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54dc06b4d0 0x7f54dc06d990 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f54f0006fd0 tx=0x7f54f0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:19.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 msgr2=0x7f54fc0cb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:19.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0cb070 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f54e8000c00 tx=0x7f54e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 shutdown_connections 2024-04-25T10:08:19.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f54dc06b4d0 0x7f54dc06d990 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54fc1452a0 0x7f54fc0b1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54fc144d50 0x7f54fc0cb070 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 --2- 172.21.15.165:0/4059438837 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f54fc1446d0 0x7f54fc142cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:19.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.964+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 >> 172.21.15.165:0/4059438837 conn(0x7f54fc06abe0 msgr2=0x7f54fc0ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:19.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.965+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 shutdown_connections 2024-04-25T10:08:19.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:19.966+0000 7f5504600640 1 -- 172.21.15.165:0/4059438837 wait complete. 2024-04-25T10:08:20.006 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:20.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.175+0000 7f2dffe00640 1 Processor -- start 2024-04-25T10:08:20.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.175+0000 7f2dffe00640 1 -- start start 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.176+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000b93c0 0x7f2e000cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.176+0000 7f2dffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2e000d04f0 0x7f2e000d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dffe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e00161df0 con 0x7f2e0015db90 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dffe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e000d0980 con 0x7f2e000d04f0 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dffe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000d1f50 con 0x7f2e000b93c0 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2e000d04f0 0x7f2e000d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2e000d04f0 0x7f2e000d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41172/0 (socket says 172.21.15.165:41172) 2024-04-25T10:08:20.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 -- 172.21.15.165:0/2229073033 learned_addr learned my addr 172.21.15.165:0/2229073033 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2df7e00640 1 --2- 172.21.15.165:0/2229073033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000b93c0 0x7f2e000cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 -- 172.21.15.165:0/2229073033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000b93c0 msgr2=0x7f2e000cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 --2- 172.21.15.165:0/2229073033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000b93c0 0x7f2e000cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 -- 172.21.15.165:0/2229073033 >> v1:172.21.15.172:6789/0 conn(0x7f2e0015db90 legacy=0x7f2e001615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.177+0000 7f2dff400640 1 -- 172.21.15.165:0/2229073033 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e001620d0 con 0x7f2e000d04f0 2024-04-25T10:08:20.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.178+0000 7f2dff400640 1 --2- 172.21.15.165:0/2229073033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e000d04f0 0x7f2e000d23e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df802a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a8e819ef9bc3d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.179+0000 7f2dfe000640 1 -- 172.21.15.165:0/2229073033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8036070 con 0x7f2e000d04f0 2024-04-25T10:08:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.179+0000 7f2dfe000640 1 -- 172.21.15.165:0/2229073033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8031600 con 0x7f2e000d04f0 2024-04-25T10:08:20.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.179+0000 7f2dfe000640 1 -- 172.21.15.165:0/2229073033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df80319d0 con 0x7f2e000d04f0 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.179+0000 7f2dffe00640 1 -- 172.21.15.165:0/2229073033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e000d04f0 msgr2=0x7f2e000d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.179+0000 7f2dffe00640 1 --2- 172.21.15.165:0/2229073033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e000d04f0 0x7f2e000d23e0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df802a6c0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.180+0000 7f2dffe00640 1 -- 172.21.15.165:0/2229073033 shutdown_connections 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.180+0000 7f2dffe00640 1 --2- 172.21.15.165:0/2229073033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e000d04f0 0x7f2e000d23e0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.180+0000 7f2dffe00640 1 --2- 172.21.15.165:0/2229073033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000b93c0 0x7f2e000cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.180+0000 7f2dffe00640 1 -- 172.21.15.165:0/2229073033 >> 172.21.15.165:0/2229073033 conn(0x7f2e000b52b0 msgr2=0x7f2e000b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.180+0000 7f2dffe00640 1 -- 172.21.15.165:0/2229073033 shutdown_connections 2024-04-25T10:08:20.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.180+0000 7f2dffe00640 1 -- 172.21.15.165:0/2229073033 wait complete. 2024-04-25T10:08:20.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.181+0000 7f2dffe00640 1 Processor -- start 2024-04-25T10:08:20.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 -- start start 2024-04-25T10:08:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e000b93c0 0x7f2e000c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000d04f0 0x7f2e0015f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 0x7f2e00157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000b68d0 con 0x7f2e0015db90 2024-04-25T10:08:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000d1be0 con 0x7f2e000b93c0 2024-04-25T10:08:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.182+0000 7f2dffe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e000b6a50 con 0x7f2e000d04f0 2024-04-25T10:08:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 0x7f2e00157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 0x7f2e00157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41182/0 (socket says 172.21.15.165:41182) 2024-04-25T10:08:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 -- 172.21.15.165:0/1696225515 learned_addr learned my addr 172.21.15.165:0/1696225515 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2df7e00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000d04f0 0x7f2e0015f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dfea00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e000b93c0 0x7f2e000c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 -- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000d04f0 msgr2=0x7f2e0015f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000d04f0 0x7f2e0015f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 -- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e000b93c0 msgr2=0x7f2e000c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e000b93c0 0x7f2e000c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dff400640 1 -- 172.21.15.165:0/1696225515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0015f9a0 con 0x7f2e0015db90 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2df7e00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000d04f0 0x7f2e0015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.183+0000 7f2dfea00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e000b93c0 0x7f2e000c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.184+0000 7f2dff400640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 0x7f2e00157380 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.184+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8036050 con 0x7f2e0015db90 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.184+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df8042070 con 0x7f2e0015db90 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.184+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df8009510 con 0x7f2e0015db90 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.184+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0015fc40 con 0x7f2e0015db90 2024-04-25T10:08:20.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.184+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e00160200 con 0x7f2e0015db90 2024-04-25T10:08:20.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.185+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e00073990 con 0x7f2e0015db90 2024-04-25T10:08:20.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.186+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2df8007bf0 con 0x7f2e0015db90 2024-04-25T10:08:20.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.187+0000 7f2df6a00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ddc06b4d0 0x7f2ddc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.187+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2df80ab2c0 con 0x7f2e0015db90 2024-04-25T10:08:20.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.187+0000 7f2dfea00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ddc06b4d0 0x7f2ddc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.188+0000 7f2dfea00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ddc06b4d0 0x7f2ddc06d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a830 tx=0x7f2df0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.192+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2df807a0c0 con 0x7f2e0015db90 2024-04-25T10:08:20.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.391+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e000d3170 con 0x7f2e0015db90 2024-04-25T10:08:20.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.393+0000 7f2df6a00640 1 -- 172.21.15.165:0/1696225515 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16994 (secure 0 0 0) 0x7f2df80799c0 con 0x7f2e0015db90 2024-04-25T10:08:20.399 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:20.399 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":84,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:08:19.373529+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ddc06b4d0 msgr2=0x7f2ddc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ddc06b4d0 0x7f2ddc06d990 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f2df000a830 tx=0x7f2df0008040 comp rx=0 tx=0).stop 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 msgr2=0x7f2e00157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 0x7f2e00157380 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009e50 tx=0x7f2df8004a40 comp rx=0 tx=0).stop 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 shutdown_connections 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2ddc06b4d0 0x7f2ddc06d990 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e0015db90 0x7f2e00157380 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e000d04f0 0x7f2e0015f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 --2- 172.21.15.165:0/1696225515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e000b93c0 0x7f2e000c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 >> 172.21.15.165:0/1696225515 conn(0x7f2e000b52b0 msgr2=0x7f2e000d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 shutdown_connections 2024-04-25T10:08:20.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.396+0000 7f2dffe00640 1 -- 172.21.15.165:0/1696225515 wait complete. 2024-04-25T10:08:20.420 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:20.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.611+0000 7f5712600640 1 Processor -- start 2024-04-25T10:08:20.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.611+0000 7f5712600640 1 -- start start 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5712600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f570c150d00 0x7f570c0ae8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5712600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f570c071880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5712600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f570c0aede0 con 0x7f570c071dc0 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5712600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f570c072210 con 0x7f570c150d00 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5712600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c15ae30 con 0x7f570c15e440 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5711c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f570c071dc0 0x7f570c06d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35866/0 (socket says 172.21.15.165:35866) 2024-04-25T10:08:20.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.613+0000 7f5711c00640 1 -- 172.21.15.165:0/3859890185 learned_addr learned my addr 172.21.15.165:0/3859890185 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f5711200640 1 --2- 172.21.15.165:0/3859890185 >> v2:172.21.15.172:3300/0 conn(0x7f570c150d00 0x7f570c0ae8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570be00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f570c071880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146405779 0 0) 0x7f570c0aede0 con 0x7f570c071dc0 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f56f4003520 con 0x7f570c071dc0 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 267180530 0 0) 0x7f56f4003520 con 0x7f570c071dc0 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 msgr2=0x7f570c071880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f570c071880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 >> v2:172.21.15.172:3300/0 conn(0x7f570c150d00 msgr2=0x7f570c0ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 --2- 172.21.15.165:0/3859890185 >> v2:172.21.15.172:3300/0 conn(0x7f570c150d00 0x7f570c0ae8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c15a230 con 0x7f570c071dc0 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5704001a10 con 0x7f570c071dc0 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 >> v1:172.21.15.165:6789/0 conn(0x7f570c071dc0 legacy=0x7f570c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f40039b0 0x7f56f4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f56f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f4004d40 0x7f56f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c072210 con 0x7f56f4004d40 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c15ae30 con 0x7f56f40039b0 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.614+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f400b9c0 con 0x7f570c15e440 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711c00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f4004d40 0x7f56f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f570be00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f56f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711200640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f40039b0 0x7f56f4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711c00640 1 -- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 msgr2=0x7f56f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711c00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f56f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711c00640 1 -- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f40039b0 msgr2=0x7f56f4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:20.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711c00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f40039b0 0x7f56f4003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f5711c00640 1 -- 172.21.15.165:0/3859890185 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f4003520 con 0x7f56f4004d40 2024-04-25T10:08:20.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.615+0000 7f570be00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f570c15e440 0x7f56f4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.616+0000 7f5711c00640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f4004d40 0x7f56f40051c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f570400a8c0 tx=0x7f56f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e4100e3426d568 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.616+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570400a4e0 con 0x7f56f4004d40 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.616+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f570402f070 con 0x7f56f4004d40 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.616+0000 7f570b400640 1 -- 172.21.15.165:0/3859890185 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f570402bcd0 con 0x7f56f4004d40 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5712600640 1 -- 172.21.15.165:0/3859890185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f4004d40 msgr2=0x7f56f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5712600640 1 --2- 172.21.15.165:0/3859890185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f4004d40 0x7f56f40051c0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7f570400a8c0 tx=0x7f56f4005830 comp rx=0 tx=0).stop 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5711c00640 1 -- 172.21.15.165:0/3859890185 reap_dead start 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5712600640 1 -- 172.21.15.165:0/3859890185 shutdown_connections 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5712600640 1 -- 172.21.15.165:0/3859890185 >> 172.21.15.165:0/3859890185 conn(0x7f570c06abe0 msgr2=0x7f570c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5712600640 1 -- 172.21.15.165:0/3859890185 shutdown_connections 2024-04-25T10:08:20.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.617+0000 7f5712600640 1 -- 172.21.15.165:0/3859890185 wait complete. 2024-04-25T10:08:20.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.619+0000 7f5712600640 1 Processor -- start 2024-04-25T10:08:20.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.619+0000 7f5712600640 1 -- start start 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.620+0000 7f5712600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f4004d40 0x7f570c1f4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.620+0000 7f5712600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 0x7f570c152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.620+0000 7f5712600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f570c154920 0x7f570c154da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.620+0000 7f5712600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c15aac0 con 0x7f570c1523d0 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.620+0000 7f5712600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f570c15a7e0 con 0x7f570c154920 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.620+0000 7f5712600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c1510e0 con 0x7f56f4004d40 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 0x7f570c152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 0x7f570c152850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41194/0 (socket says 172.21.15.165:41194) 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 -- 172.21.15.165:0/1829548954 learned_addr learned my addr 172.21.15.165:0/1829548954 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f5711200640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f4004d40 0x7f570c1f4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 -- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f4004d40 msgr2=0x7f570c1f4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f4004d40 0x7f570c1f4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f5711c00640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f570c154920 0x7f570c154da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 -- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f570c154920 msgr2=0x7f570c154da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f570c154920 0x7f570c154da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 -- 172.21.15.165:0/1829548954 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c152f40 con 0x7f570c1523d0 2024-04-25T10:08:20.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.621+0000 7f570be00640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 0x7f570c152850 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f56f80108e0 tx=0x7f56f8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.622+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f800f040 con 0x7f570c1523d0 2024-04-25T10:08:20.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.622+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f800ec20 con 0x7f570c1523d0 2024-04-25T10:08:20.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.622+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f801c400 con 0x7f570c1523d0 2024-04-25T10:08:20.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.622+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f570c153220 con 0x7f570c1523d0 2024-04-25T10:08:20.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.622+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f570c1537e0 con 0x7f570c1523d0 2024-04-25T10:08:20.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.623+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570c073990 con 0x7f570c1523d0 2024-04-25T10:08:20.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.624+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56f801c5a0 con 0x7f570c1523d0 2024-04-25T10:08:20.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.625+0000 7f5708c00640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:20.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.625+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56f808abb0 con 0x7f570c1523d0 2024-04-25T10:08:20.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.625+0000 7f5711200640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:20.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.626+0000 7f5711200640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56ec06b4d0 0x7f56ec06d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f570000aa10 tx=0x7f5700008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:20.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.629+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56f8018030 con 0x7f570c1523d0 2024-04-25T10:08:20.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.827+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f570c15e220 con 0x7f570c1523d0 2024-04-25T10:08:20.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.830+0000 7f5708c00640 1 -- 172.21.15.165:0/1829548954 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16994 (secure 0 0 0) 0x7f56f8059800 con 0x7f570c1523d0 2024-04-25T10:08:20.836 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:20.836 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":84,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:08:19.373529+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.832+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56ec06b4d0 msgr2=0x7f56ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.832+0000 7f5712600640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56ec06b4d0 0x7f56ec06d990 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f570000aa10 tx=0x7f5700008040 comp rx=0 tx=0).stop 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 msgr2=0x7f570c152850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 0x7f570c152850 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f56f80108e0 tx=0x7f56f8010da0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 shutdown_connections 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56ec06b4d0 0x7f56ec06d990 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f570c154920 0x7f570c154da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f570c1523d0 0x7f570c152850 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 --2- 172.21.15.165:0/1829548954 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f4004d40 0x7f570c1f4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 >> 172.21.15.165:0/1829548954 conn(0x7f570c06abe0 msgr2=0x7f570c070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 shutdown_connections 2024-04-25T10:08:20.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:20.833+0000 7f5712600640 1 -- 172.21.15.165:0/1829548954 wait complete. 2024-04-25T10:08:20.857 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:08:21.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.056+0000 7f5b87e00640 1 Processor -- start 2024-04-25T10:08:21.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.056+0000 7f5b87e00640 1 -- start start 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b87e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5b88145140 0x7f5b880ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b880ce950 0x7f5b880c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b87e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b88145520 con 0x7f5b880c5030 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b87e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b880ced90 con 0x7f5b88145140 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b87e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88071270 con 0x7f5b880ce950 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b86a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5b88145140 0x7f5b880ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b86a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5b88145140 0x7f5b880ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41206/0 (socket says 172.21.15.165:41206) 2024-04-25T10:08:21.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.058+0000 7f5b86a00640 1 -- 172.21.15.165:0/268833205 learned_addr learned my addr 172.21.15.165:0/268833205 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:21.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.059+0000 7f5b86000640 1 --2- 172.21.15.165:0/268833205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b880ce950 0x7f5b880c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.059+0000 7f5b86a00640 1 -- 172.21.15.165:0/268833205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b880ce950 msgr2=0x7f5b880c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.059+0000 7f5b86a00640 1 --2- 172.21.15.165:0/268833205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b880ce950 0x7f5b880c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.059+0000 7f5b86a00640 1 -- 172.21.15.165:0/268833205 >> v1:172.21.15.172:6789/0 conn(0x7f5b880c5030 legacy=0x7f5b8806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:21.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.059+0000 7f5b86a00640 1 -- 172.21.15.165:0/268833205 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88071550 con 0x7f5b88145140 2024-04-25T10:08:21.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.060+0000 7f5b86a00640 1 --2- 172.21.15.165:0/268833205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b88145140 0x7f5b880ce410 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009870 tx=0x7f5b7802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459f69d3333cde01 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.061+0000 7f5b85600640 1 -- 172.21.15.165:0/268833205 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78037070 con 0x7f5b88145140 2024-04-25T10:08:21.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.061+0000 7f5b85600640 1 -- 172.21.15.165:0/268833205 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b78032440 con 0x7f5b88145140 2024-04-25T10:08:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.061+0000 7f5b85600640 1 -- 172.21.15.165:0/268833205 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78032810 con 0x7f5b88145140 2024-04-25T10:08:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.061+0000 7f5b87e00640 1 -- 172.21.15.165:0/268833205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b88145140 msgr2=0x7f5b880ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.061+0000 7f5b87e00640 1 --2- 172.21.15.165:0/268833205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b88145140 0x7f5b880ce410 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009870 tx=0x7f5b7802a180 comp rx=0 tx=0).stop 2024-04-25T10:08:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.062+0000 7f5b87e00640 1 -- 172.21.15.165:0/268833205 shutdown_connections 2024-04-25T10:08:21.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.062+0000 7f5b87e00640 1 --2- 172.21.15.165:0/268833205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b880ce950 0x7f5b880c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.062+0000 7f5b87e00640 1 --2- 172.21.15.165:0/268833205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b88145140 0x7f5b880ce410 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.062+0000 7f5b87e00640 1 -- 172.21.15.165:0/268833205 >> 172.21.15.165:0/268833205 conn(0x7f5b88061cc0 msgr2=0x7f5b8806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.062+0000 7f5b87e00640 1 -- 172.21.15.165:0/268833205 shutdown_connections 2024-04-25T10:08:21.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.062+0000 7f5b87e00640 1 -- 172.21.15.165:0/268833205 wait complete. 2024-04-25T10:08:21.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.063+0000 7f5b87e00640 1 Processor -- start 2024-04-25T10:08:21.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.063+0000 7f5b87e00640 1 -- start start 2024-04-25T10:08:21.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.064+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 0x7f5b8814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.064+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b880ce950 0x7f5b881e4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.064+0000 7f5b87e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b88145140 0x7f5b880cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.064+0000 7f5b87e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880c54a0 con 0x7f5b880c5030 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.064+0000 7f5b87e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b88071b10 con 0x7f5b880ce950 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.064+0000 7f5b87e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b880c5620 con 0x7f5b88145140 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 0x7f5b8814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 0x7f5b8814c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41220/0 (socket says 172.21.15.165:41220) 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 -- 172.21.15.165:0/4229550172 learned_addr learned my addr 172.21.15.165:0/4229550172 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 -- 172.21.15.165:0/4229550172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b88145140 msgr2=0x7f5b880cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b88145140 0x7f5b880cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 -- 172.21.15.165:0/4229550172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b880ce950 msgr2=0x7f5b881e4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b880ce950 0x7f5b881e4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 -- 172.21.15.165:0/4229550172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b8814cd10 con 0x7f5b880c5030 2024-04-25T10:08:21.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.065+0000 7f5b86a00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 0x7f5b8814c7d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009870 tx=0x7f5b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.066+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78041020 con 0x7f5b880c5030 2024-04-25T10:08:21.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.066+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b780306c0 con 0x7f5b880c5030 2024-04-25T10:08:21.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.066+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b78037040 con 0x7f5b880c5030 2024-04-25T10:08:21.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.066+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b8814cf90 con 0x7f5b880c5030 2024-04-25T10:08:21.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.066+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b8814d550 con 0x7f5b880c5030 2024-04-25T10:08:21.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.067+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b88073990 con 0x7f5b880c5030 2024-04-25T10:08:21.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.068+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b78044020 con 0x7f5b880c5030 2024-04-25T10:08:21.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.069+0000 7f5b7ea00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b6806b500 0x7f5b6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:21.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.069+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5b780aab90 con 0x7f5b880c5030 2024-04-25T10:08:21.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.069+0000 7f5b86000640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b6806b500 0x7f5b6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:21.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.070+0000 7f5b86000640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b6806b500 0x7f5b6806d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5b880ce920 tx=0x7f5b74008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:21.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.073+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b7804b030 con 0x7f5b880c5030 2024-04-25T10:08:21.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.312+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5b8814dc50 con 0x7f5b880c5030 2024-04-25T10:08:21.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.979+0000 7f5b7ea00640 1 -- 172.21.15.165:0/4229550172 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v55) v1 ==== 118+0+0 (secure 0 0 0) 0x7f5b8814cd10 con 0x7f5b880c5030 2024-04-25T10:08:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.983+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b6806b500 msgr2=0x7f5b6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.983+0000 7f5b87e00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b6806b500 0x7f5b6806d9c0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f5b880ce920 tx=0x7f5b74008040 comp rx=0 tx=0).stop 2024-04-25T10:08:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.983+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 msgr2=0x7f5b8814c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.983+0000 7f5b87e00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 0x7f5b8814c7d0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f5b78009870 tx=0x7f5b780046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 shutdown_connections 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b6806b500 0x7f5b6806d9c0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b88145140 0x7f5b880cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b880ce950 0x7f5b881e4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 --2- 172.21.15.165:0/4229550172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b880c5030 0x7f5b8814c7d0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 >> 172.21.15.165:0/4229550172 conn(0x7f5b88061cc0 msgr2=0x7f5b8815c840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 shutdown_connections 2024-04-25T10:08:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:21.984+0000 7f5b87e00640 1 -- 172.21.15.165:0/4229550172 wait complete. 2024-04-25T10:08:22.011 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:08:22.011 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:08:22.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.214+0000 7f1e3ea00640 1 Processor -- start 2024-04-25T10:08:22.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.214+0000 7f1e3ea00640 1 -- start start 2024-04-25T10:08:22.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.216+0000 7f1e3ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 0x7f1e380b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.216+0000 7f1e3ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e3815b7a0 0x7f1e380cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.216+0000 7f1e3ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380b0b80 con 0x7f1e380cdae0 2024-04-25T10:08:22.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.216+0000 7f1e3ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380cd650 con 0x7f1e3815b7a0 2024-04-25T10:08:22.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.216+0000 7f1e3ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380cc800 con 0x7f1e3815b3c0 2024-04-25T10:08:22.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.217+0000 7f1e3d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 0x7f1e380b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.217+0000 7f1e3d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 0x7f1e380b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:44416/0 (socket says 172.21.15.165:44416) 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.217+0000 7f1e3d600640 1 -- 172.21.15.165:0/2011004247 learned_addr learned my addr 172.21.15.165:0/2011004247 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.217+0000 7f1e3cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e3815b7a0 0x7f1e380cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.218+0000 7f1e3d600640 1 -- 172.21.15.165:0/2011004247 >> v2:172.21.15.172:3300/0 conn(0x7f1e3815b7a0 msgr2=0x7f1e380cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.218+0000 7f1e3d600640 1 --2- 172.21.15.165:0/2011004247 >> v2:172.21.15.172:3300/0 conn(0x7f1e3815b7a0 0x7f1e380cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.218+0000 7f1e3d600640 1 -- 172.21.15.165:0/2011004247 >> v1:172.21.15.172:6789/0 conn(0x7f1e380cdae0 legacy=0x7f1e3815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.218+0000 7f1e3d600640 1 -- 172.21.15.165:0/2011004247 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380ccae0 con 0x7f1e3815b3c0 2024-04-25T10:08:22.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.219+0000 7f1e3d600640 1 --2- 172.21.15.165:0/2011004247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 0x7f1e380b0f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009870 tx=0x7f1e2c02a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef1299109fb777a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.219+0000 7f1e37e00640 1 -- 172.21.15.165:0/2011004247 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c037070 con 0x7f1e3815b3c0 2024-04-25T10:08:22.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e37e00640 1 -- 172.21.15.165:0/2011004247 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c032440 con 0x7f1e3815b3c0 2024-04-25T10:08:22.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e37e00640 1 -- 172.21.15.165:0/2011004247 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c032810 con 0x7f1e3815b3c0 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e3ea00640 1 -- 172.21.15.165:0/2011004247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 msgr2=0x7f1e380b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/2011004247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 0x7f1e380b0f60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c009870 tx=0x7f1e2c02a270 comp rx=0 tx=0).stop 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e3ea00640 1 -- 172.21.15.165:0/2011004247 shutdown_connections 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/2011004247 >> v2:172.21.15.172:3300/0 conn(0x7f1e3815b7a0 0x7f1e380cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/2011004247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b3c0 0x7f1e380b0f60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.220+0000 7f1e3ea00640 1 -- 172.21.15.165:0/2011004247 >> 172.21.15.165:0/2011004247 conn(0x7f1e38061cc0 msgr2=0x7f1e3806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.221+0000 7f1e3ea00640 1 -- 172.21.15.165:0/2011004247 shutdown_connections 2024-04-25T10:08:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.221+0000 7f1e3ea00640 1 -- 172.21.15.165:0/2011004247 wait complete. 2024-04-25T10:08:22.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.223+0000 7f1e3ea00640 1 Processor -- start 2024-04-25T10:08:22.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.223+0000 7f1e3ea00640 1 -- start start 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.224+0000 7f1e3ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 0x7f1e380ca220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.224+0000 7f1e3ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e3815b3c0 0x7f1e381432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.224+0000 7f1e3ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b7a0 0x7f1e38144430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.224+0000 7f1e3ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3815e8a0 con 0x7f1e380cdae0 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.224+0000 7f1e3ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3815e720 con 0x7f1e3815b3c0 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.224+0000 7f1e3ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e380cdf40 con 0x7f1e3815b7a0 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 0x7f1e380ca220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 0x7f1e380ca220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41232/0 (socket says 172.21.15.165:41232) 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 -- 172.21.15.165:0/931926218 learned_addr learned my addr 172.21.15.165:0/931926218 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 -- 172.21.15.165:0/931926218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b7a0 msgr2=0x7f1e38144430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b7a0 0x7f1e38144430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 -- 172.21.15.165:0/931926218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e3815b3c0 msgr2=0x7f1e381432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e3815b3c0 0x7f1e381432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 -- 172.21.15.165:0/931926218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e38144970 con 0x7f1e380cdae0 2024-04-25T10:08:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.225+0000 7f1e3d600640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 0x7f1e380ca220 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c000c00 tx=0x7f1e2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.226+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c037040 con 0x7f1e380cdae0 2024-04-25T10:08:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.226+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c042070 con 0x7f1e380cdae0 2024-04-25T10:08:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.226+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c030880 con 0x7f1e380cdae0 2024-04-25T10:08:22.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.226+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e38144bf0 con 0x7f1e380cdae0 2024-04-25T10:08:22.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.226+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e381451b0 con 0x7f1e380cdae0 2024-04-25T10:08:22.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.227+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e38073990 con 0x7f1e380cdae0 2024-04-25T10:08:22.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.228+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e2c030a20 con 0x7f1e380cdae0 2024-04-25T10:08:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.229+0000 7f1e35600640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e1806b4d0 0x7f1e1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.229+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e2c0aacf0 con 0x7f1e380cdae0 2024-04-25T10:08:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.229+0000 7f1e3cc00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e1806b4d0 0x7f1e1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.230+0000 7f1e3cc00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e1806b4d0 0x7f1e1806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006a90 tx=0x7f1e24006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:22.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.234+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e38144970 con 0x7f1e380cdae0 2024-04-25T10:08:22.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.469+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f1e380cb820 con 0x7f1e380cdae0 2024-04-25T10:08:22.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.982+0000 7f1e35600640 1 -- 172.21.15.165:0/931926218 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v56) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1e2c079940 con 0x7f1e380cdae0 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e1806b4d0 msgr2=0x7f1e1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e1806b4d0 0x7f1e1806d990 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1e24006a90 tx=0x7f1e24006a20 comp rx=0 tx=0).stop 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 msgr2=0x7f1e380ca220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 0x7f1e380ca220 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c000c00 tx=0x7f1e2c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 shutdown_connections 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e1806b4d0 0x7f1e1806d990 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e3815b7a0 0x7f1e38144430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e3815b3c0 0x7f1e381432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 --2- 172.21.15.165:0/931926218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e380cdae0 0x7f1e380ca220 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:22.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 >> 172.21.15.165:0/931926218 conn(0x7f1e38061cc0 msgr2=0x7f1e3806e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:22.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 shutdown_connections 2024-04-25T10:08:22.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:22.986+0000 7f1e3ea00640 1 -- 172.21.15.165:0/931926218 wait complete. 2024-04-25T10:08:23.014 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:08:23.014 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:23.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.204+0000 7f231b800640 1 Processor -- start 2024-04-25T10:08:23.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.204+0000 7f231b800640 1 -- start start 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.205+0000 7f231b800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2314144750 0x7f231406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.205+0000 7f231b800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2314144b30 con 0x7f23140cf030 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.205+0000 7f231b800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23140712d0 con 0x7f231406eb10 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.205+0000 7f231b800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23140715b0 con 0x7f2314144750 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.206+0000 7f2318e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2314144750 0x7f231406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.206+0000 7f2318e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2314144750 0x7f231406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41240/0 (socket says 172.21.15.165:41240) 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.206+0000 7f2318e00640 1 -- 172.21.15.165:0/846216764 learned_addr learned my addr 172.21.15.165:0/846216764 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.206+0000 7f2318e00640 1 -- 172.21.15.165:0/846216764 >> v1:172.21.15.172:6789/0 conn(0x7f231406eb10 legacy=0x7f23140ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.206+0000 7f2318e00640 1 -- 172.21.15.165:0/846216764 >> v1:172.21.15.165:6789/0 conn(0x7f23140cf030 legacy=0x7f231406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.206+0000 7f2318e00640 1 -- 172.21.15.165:0/846216764 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314071890 con 0x7f2314144750 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.207+0000 7f2318e00640 1 --2- 172.21.15.165:0/846216764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2314144750 0x7f231406e5d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2308009870 tx=0x7f230802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc73f909d23da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f2313400640 1 -- 172.21.15.165:0/846216764 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308037070 con 0x7f2314144750 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f2313400640 1 -- 172.21.15.165:0/846216764 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2308032440 con 0x7f2314144750 2024-04-25T10:08:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f2313400640 1 -- 172.21.15.165:0/846216764 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2308032810 con 0x7f2314144750 2024-04-25T10:08:23.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 -- 172.21.15.165:0/846216764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2314144750 msgr2=0x7f231406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 --2- 172.21.15.165:0/846216764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2314144750 0x7f231406e5d0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f2308009870 tx=0x7f230802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 -- 172.21.15.165:0/846216764 shutdown_connections 2024-04-25T10:08:23.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 --2- 172.21.15.165:0/846216764 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2314144750 0x7f231406e5d0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 -- 172.21.15.165:0/846216764 >> 172.21.15.165:0/846216764 conn(0x7f231406abe0 msgr2=0x7f231406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 -- 172.21.15.165:0/846216764 shutdown_connections 2024-04-25T10:08:23.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.208+0000 7f231b800640 1 -- 172.21.15.165:0/846216764 wait complete. 2024-04-25T10:08:23.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.210+0000 7f231b800640 1 Processor -- start 2024-04-25T10:08:23.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.210+0000 7f231b800640 1 -- start start 2024-04-25T10:08:23.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f231b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231406eb10 0x7f231413b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f231b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 0x7f231413bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f231b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314144750 0x7f23140b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f231b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23141455d0 con 0x7f23140cf030 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f231b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2314145450 con 0x7f2314144750 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f231b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231415c000 con 0x7f231406eb10 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f2313e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 0x7f231413bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f2313e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 0x7f231413bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41250/0 (socket says 172.21.15.165:41250) 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.211+0000 7f2313e00640 1 -- 172.21.15.165:0/3980260174 learned_addr learned my addr 172.21.15.165:0/3980260174 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2318e00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231406eb10 0x7f231413b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2319800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314144750 0x7f23140b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2313e00640 1 -- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231406eb10 msgr2=0x7f231413b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2313e00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231406eb10 0x7f231413b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2313e00640 1 -- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314144750 msgr2=0x7f23140b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2313e00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314144750 0x7f23140b04c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2313e00640 1 -- 172.21.15.165:0/3980260174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231413c6d0 con 0x7f23140cf030 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2319800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314144750 0x7f23140b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:23.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2313e00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 0x7f231413bc30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f230000b3f0 tx=0x7f230000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300015020 con 0x7f23140cf030 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2300011830 con 0x7f23140cf030 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2300011c20 con 0x7f23140cf030 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231413c9b0 con 0x7f23140cf030 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.212+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231413da80 con 0x7f23140cf030 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.213+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231413dc80 con 0x7f23140cf030 2024-04-25T10:08:23.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.215+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2300018020 con 0x7f23140cf030 2024-04-25T10:08:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.215+0000 7f2310c00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f406b4d0 0x7f22f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.215+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f230008a070 con 0x7f23140cf030 2024-04-25T10:08:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.216+0000 7f2318e00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f406b4d0 0x7f22f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.216+0000 7f2318e00640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f406b4d0 0x7f22f406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2308034000 tx=0x7f230802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.220+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f231413dc80 con 0x7f23140cf030 2024-04-25T10:08:23.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.446+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23140d0510 con 0x7f23140cf030 2024-04-25T10:08:23.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.447+0000 7f2310c00640 1 -- 172.21.15.165:0/3980260174 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f230001f030 con 0x7f23140cf030 2024-04-25T10:08:23.453 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:23.453 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:19.374715+0000","modified":"2024-04-25T10:08:22.470688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6833/1170361110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1170361110},{"type":"v1","addr":"172.21.15.172:6833","nonce":1170361110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:23.453 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 56 2024-04-25T10:08:23.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f406b4d0 msgr2=0x7f22f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f406b4d0 0x7f22f406d990 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f2308034000 tx=0x7f230802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:23.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 msgr2=0x7f231413bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 0x7f231413bc30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f230000b3f0 tx=0x7f230000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 shutdown_connections 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f22f406b4d0 0x7f22f406d990 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2314144750 0x7f23140b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23140cf030 0x7f231413bc30 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 --2- 172.21.15.165:0/3980260174 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f231406eb10 0x7f231413b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.449+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 >> 172.21.15.165:0/3980260174 conn(0x7f231406abe0 msgr2=0x7f23140b1350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.450+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 shutdown_connections 2024-04-25T10:08:23.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.450+0000 7f231b800640 1 -- 172.21.15.165:0/3980260174 wait complete. 2024-04-25T10:08:23.472 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:08:23.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.643+0000 7f69ef200640 1 Processor -- start 2024-04-25T10:08:23.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.644+0000 7f69ef200640 1 -- start start 2024-04-25T10:08:23.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.645+0000 7f69ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 0x7f69f0168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.645+0000 7f69ef200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f00d33f0 con 0x7f69f015db90 2024-04-25T10:08:23.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.645+0000 7f69ef200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69f0161df0 con 0x7f69f00b93c0 2024-04-25T10:08:23.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.645+0000 7f69ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00d1f50 con 0x7f69f00d06c0 2024-04-25T10:08:23.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69ede00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f69f015db90 0x7f69f01615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35886/0 (socket says 172.21.15.165:35886) 2024-04-25T10:08:23.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69ede00640 1 -- 172.21.15.165:0/4016984862 learned_addr learned my addr 172.21.15.165:0/4016984862 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 626473223 0 0) 0x7f69f00d33f0 con 0x7f69f015db90 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f69d0003520 con 0x7f69f015db90 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69ee800640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 0x7f69f0168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 160095821 0 0) 0x7f69d0003520 con 0x7f69f015db90 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 msgr2=0x7f69f0168930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 0x7f69f0168930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 >> v1:172.21.15.172:6789/0 conn(0x7f69f00b93c0 legacy=0x7f69f00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f01620d0 con 0x7f69f015db90 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.646+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f69e0001670 con 0x7f69f015db90 2024-04-25T10:08:23.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 >> v1:172.21.15.165:6789/0 conn(0x7f69f015db90 legacy=0x7f69f01615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d00039e0 0x7f69d0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 0x7f69d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00d1f50 con 0x7f69d0004800 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69d000baf0 con 0x7f69d00039e0 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69d000bcf0 con 0x7f69f00d06c0 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ed400640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ede00640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d00039e0 0x7f69d0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ed400640 1 -- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 msgr2=0x7f69d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ed400640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00d06c0 0x7f69d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ed400640 1 -- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d00039e0 msgr2=0x7f69d0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ed400640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69d00039e0 0x7f69d0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.647+0000 7f69ed400640 1 -- 172.21.15.165:0/4016984862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0003520 con 0x7f69d0004800 2024-04-25T10:08:23.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.648+0000 7f69ed400640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f69d8001e50 tx=0x7f69d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9285a51fbe237d17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8001a70 con 0x7f69d0004800 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69d8003970 con 0x7f69d0004800 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69eca00640 1 -- 172.21.15.165:0/4016984862 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69d8003d60 con 0x7f69d0004800 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ef200640 1 -- 172.21.15.165:0/4016984862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 msgr2=0x7f69d0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ef200640 1 --2- 172.21.15.165:0/4016984862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69d0004c80 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f69d8001e50 tx=0x7f69d0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ee800640 1 -- 172.21.15.165:0/4016984862 reap_dead start 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ef200640 1 -- 172.21.15.165:0/4016984862 shutdown_connections 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ef200640 1 -- 172.21.15.165:0/4016984862 >> 172.21.15.165:0/4016984862 conn(0x7f69f00b52b0 msgr2=0x7f69f00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ef200640 1 -- 172.21.15.165:0/4016984862 shutdown_connections 2024-04-25T10:08:23.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.649+0000 7f69ef200640 1 -- 172.21.15.165:0/4016984862 wait complete. 2024-04-25T10:08:23.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.651+0000 7f69ef200640 1 Processor -- start 2024-04-25T10:08:23.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.651+0000 7f69ef200640 1 -- start start 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69f01f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f015f990 0x7f69f015fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00c89a0 0x7f69f00c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ef200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00b68d0 con 0x7f69d0004800 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00d1be0 con 0x7f69f015f990 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69f00b6a50 con 0x7f69f00c89a0 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69f01f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69f01f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41264/0 (socket says 172.21.15.165:41264) 2024-04-25T10:08:23.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.652+0000 7f69ede00640 1 -- 172.21.15.165:0/3944534955 learned_addr learned my addr 172.21.15.165:0/3944534955 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:23.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ede00640 1 -- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00c89a0 msgr2=0x7f69f00c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ede00640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00c89a0 0x7f69f00c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ede00640 1 -- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f015f990 msgr2=0x7f69f015fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ed400640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f015f990 0x7f69f015fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ede00640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f015f990 0x7f69f015fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ede00640 1 -- 172.21.15.165:0/3944534955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f01f4a60 con 0x7f69d0004800 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ede00640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69f01f4520 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e850 tx=0x7f69e000ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000c870 con 0x7f69d0004800 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0018070 con 0x7f69d0004800 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e000cc50 con 0x7f69d0004800 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.653+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f01603e0 con 0x7f69d0004800 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.654+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f01609a0 con 0x7f69d0004800 2024-04-25T10:08:23.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.655+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f0073990 con 0x7f69d0004800 2024-04-25T10:08:23.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.656+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69e001d460 con 0x7f69d0004800 2024-04-25T10:08:23.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.657+0000 7f69e6000640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69cc06b4d0 0x7f69cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:23.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.657+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69e008a3c0 con 0x7f69d0004800 2024-04-25T10:08:23.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.657+0000 7f69ed400640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69cc06b4d0 0x7f69cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:23.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.658+0000 7f69ed400640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69cc06b4d0 0x7f69cc06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f69d8032000 tx=0x7f69d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:23.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.661+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69e0059020 con 0x7f69d0004800 2024-04-25T10:08:23.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.860+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69f014f630 con 0x7f69d0004800 2024-04-25T10:08:23.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.861+0000 7f69e6000640 1 -- 172.21.15.165:0/3944534955 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16981 (secure 0 0 0) 0x7f69e0058920 con 0x7f69d0004800 2024-04-25T10:08:23.868 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:23.868 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":85,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:08:20.953526+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:08:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69cc06b4d0 msgr2=0x7f69cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69cc06b4d0 0x7f69cc06d990 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f69d8032000 tx=0x7f69d802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 msgr2=0x7f69f01f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69f01f4520 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f69e000e850 tx=0x7f69e000ed10 comp rx=0 tx=0).stop 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 shutdown_connections 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69cc06b4d0 0x7f69cc06d990 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69f00c89a0 0x7f69f00c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69f015f990 0x7f69f015fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 --2- 172.21.15.165:0/3944534955 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69d0004800 0x7f69f01f4520 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.864+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 >> 172.21.15.165:0/3944534955 conn(0x7f69f00b52b0 msgr2=0x7f69f00d3730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.865+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 shutdown_connections 2024-04-25T10:08:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:23.865+0000 7f69ef200640 1 -- 172.21.15.165:0/3944534955 wait complete. 2024-04-25T10:08:23.889 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:08:23.889 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:24.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.064+0000 7f6574e00640 1 Processor -- start 2024-04-25T10:08:24.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.064+0000 7f6574e00640 1 -- start start 2024-04-25T10:08:24.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.065+0000 7f6574e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.065+0000 7f6574e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f657015f040 0x7f65700adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.065+0000 7f6574e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65700ae590 con 0x7f65700ae0d0 2024-04-25T10:08:24.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.065+0000 7f6574e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65700aee30 con 0x7f657015f040 2024-04-25T10:08:24.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.065+0000 7f6574e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65700ae870 con 0x7f657015e520 2024-04-25T10:08:24.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.066+0000 7f656f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.066+0000 7f656f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:44484/0 (socket says 172.21.15.165:44484) 2024-04-25T10:08:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.066+0000 7f656f400640 1 -- 172.21.15.165:0/2722339110 learned_addr learned my addr 172.21.15.165:0/2722339110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.066+0000 7f6567e00640 1 --2- 172.21.15.165:0/2722339110 >> v2:172.21.15.172:3300/0 conn(0x7f657015f040 0x7f65700adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.067+0000 7f656f400640 1 -- 172.21.15.165:0/2722339110 >> v2:172.21.15.172:3300/0 conn(0x7f657015f040 msgr2=0x7f65700adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.067+0000 7f656f400640 1 --2- 172.21.15.165:0/2722339110 >> v2:172.21.15.172:3300/0 conn(0x7f657015f040 0x7f65700adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.067+0000 7f656f400640 1 -- 172.21.15.165:0/2722339110 >> v1:172.21.15.172:6789/0 conn(0x7f65700ae0d0 legacy=0x7f6570070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.067+0000 7f656f400640 1 -- 172.21.15.165:0/2722339110 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65700aeb50 con 0x7f657015e520 2024-04-25T10:08:24.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.068+0000 7f656f400640 1 --2- 172.21.15.165:0/2722339110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015e900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6560009870 tx=0x7f656002a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e70c80a42f74da8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.069+0000 7f656ea00640 1 -- 172.21.15.165:0/2722339110 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560037070 con 0x7f657015e520 2024-04-25T10:08:24.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.069+0000 7f656ea00640 1 -- 172.21.15.165:0/2722339110 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6560032440 con 0x7f657015e520 2024-04-25T10:08:24.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.069+0000 7f656ea00640 1 -- 172.21.15.165:0/2722339110 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6560032810 con 0x7f657015e520 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.069+0000 7f6574e00640 1 -- 172.21.15.165:0/2722339110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 msgr2=0x7f657015e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.069+0000 7f6574e00640 1 --2- 172.21.15.165:0/2722339110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015e900 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6560009870 tx=0x7f656002a270 comp rx=0 tx=0).stop 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.070+0000 7f6574e00640 1 -- 172.21.15.165:0/2722339110 shutdown_connections 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.070+0000 7f6574e00640 1 --2- 172.21.15.165:0/2722339110 >> v2:172.21.15.172:3300/0 conn(0x7f657015f040 0x7f65700adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.070+0000 7f6574e00640 1 --2- 172.21.15.165:0/2722339110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015e900 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.070+0000 7f6574e00640 1 -- 172.21.15.165:0/2722339110 >> 172.21.15.165:0/2722339110 conn(0x7f657006abe0 msgr2=0x7f657006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.070+0000 7f6574e00640 1 -- 172.21.15.165:0/2722339110 shutdown_connections 2024-04-25T10:08:24.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.070+0000 7f6574e00640 1 -- 172.21.15.165:0/2722339110 wait complete. 2024-04-25T10:08:24.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.072+0000 7f6574e00640 1 Processor -- start 2024-04-25T10:08:24.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.072+0000 7f6574e00640 1 -- start start 2024-04-25T10:08:24.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6574e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700ae0d0 0x7f6570158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6574e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6574e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 0x7f65700b3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6574e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657015f620 con 0x7f657015f040 2024-04-25T10:08:24.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6574e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f657015f4a0 con 0x7f65700ae0d0 2024-04-25T10:08:24.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6574e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6570071700 con 0x7f657015e520 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f656fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 0x7f65700b3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f656fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 0x7f65700b3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41278/0 (socket says 172.21.15.165:41278) 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f656fe00640 1 -- 172.21.15.165:0/3918039588 learned_addr learned my addr 172.21.15.165:0/3918039588 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.073+0000 7f6567e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656f400640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700ae0d0 0x7f6570158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656fe00640 1 -- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 msgr2=0x7f657015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656fe00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656fe00640 1 -- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700ae0d0 msgr2=0x7f6570158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656fe00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700ae0d0 0x7f6570158d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656fe00640 1 -- 172.21.15.165:0/3918039588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6570159340 con 0x7f657015f040 2024-04-25T10:08:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f656fe00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 0x7f65700b3360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f656800c7a0 tx=0x7f656800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6568003b70 con 0x7f657015f040 2024-04-25T10:08:24.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6568003d30 con 0x7f657015f040 2024-04-25T10:08:24.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6570159620 con 0x7f657015f040 2024-04-25T10:08:24.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f656800aa90 con 0x7f657015f040 2024-04-25T10:08:24.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.074+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65700b11c0 con 0x7f657015f040 2024-04-25T10:08:24.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.075+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65701597f0 con 0x7f657015f040 2024-04-25T10:08:24.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.076+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f656800ac30 con 0x7f657015f040 2024-04-25T10:08:24.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.077+0000 7f6567400640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f654c06b4d0 0x7f654c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.077+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f656808a380 con 0x7f657015f040 2024-04-25T10:08:24.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.077+0000 7f656f400640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f654c06b4d0 0x7f654c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.078+0000 7f656f400640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f654c06b4d0 0x7f654c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6560034000 tx=0x7f656002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.082+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6568058fd0 con 0x7f657015f040 2024-04-25T10:08:24.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.260+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f65700b13c0 con 0x7f654c06b4d0 2024-04-25T10:08:24.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.280+0000 7f6567400640 1 -- 172.21.15.165:0/3918039588 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252082 (secure 0 0 0) 0x7f65700b13c0 con 0x7f654c06b4d0 2024-04-25T10:08:24.289 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:24.291 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:08:24.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f654c06b4d0 msgr2=0x7f654c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f654c06b4d0 0x7f654c06d990 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f6560034000 tx=0x7f656002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 msgr2=0x7f65700b3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 0x7f65700b3360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f656800c7a0 tx=0x7f656800cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 shutdown_connections 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f654c06b4d0 0x7f654c06d990 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f657015f040 0x7f65700b3360 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f657015e520 0x7f657015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 --2- 172.21.15.165:0/3918039588 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65700ae0d0 0x7f6570158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 >> 172.21.15.165:0/3918039588 conn(0x7f657006abe0 msgr2=0x7f65700af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 shutdown_connections 2024-04-25T10:08:24.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.284+0000 7f6574e00640 1 -- 172.21.15.165:0/3918039588 wait complete. 2024-04-25T10:08:24.308 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-04-25T10:08:24.255381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322628,"kb_used_data":9352,"kb_used_omap":12,"kb_used_meta":313139,"kb_avail":754652092,"statfs":{"total":773094113280,"available":772763742208,"internally_reserved":0,"allocated":9576448,"data_stored":5042900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":320654898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001814"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318487+0000","last_change":"2024-04-25T10:08:19.967461+0000","last_active":"2024-04-25T10:08:21.318487+0000","last_peered":"2024-04-25T10:08:21.318487+0000","last_clean":"2024-04-25T10:08:21.318487+0000","last_became_active":"2024-04-25T10:08:19.966573+0000","last_became_peered":"2024-04-25T10:08:19.966573+0000","last_unstale":"2024-04-25T10:08:21.318487+0000","last_undegraded":"2024-04-25T10:08:21.318487+0000","last_fullsized":"2024-04-25T10:08:21.318487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:52.728570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980483+0000","last_change":"2024-04-25T10:08:19.986679+0000","last_active":"2024-04-25T10:08:20.980483+0000","last_peered":"2024-04-25T10:08:20.980483+0000","last_clean":"2024-04-25T10:08:20.980483+0000","last_became_active":"2024-04-25T10:08:19.983909+0000","last_became_peered":"2024-04-25T10:08:19.983909+0000","last_unstale":"2024-04-25T10:08:20.980483+0000","last_undegraded":"2024-04-25T10:08:20.980483+0000","last_fullsized":"2024-04-25T10:08:20.980483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:37.487486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962369+0000","last_change":"2024-04-25T10:08:19.991185+0000","last_active":"2024-04-25T10:08:20.962369+0000","last_peered":"2024-04-25T10:08:20.962369+0000","last_clean":"2024-04-25T10:08:20.962369+0000","last_became_active":"2024-04-25T10:08:19.990443+0000","last_became_peered":"2024-04-25T10:08:19.990443+0000","last_unstale":"2024-04-25T10:08:20.962369+0000","last_undegraded":"2024-04-25T10:08:20.962369+0000","last_fullsized":"2024-04-25T10:08:20.962369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.544140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316691+0000","last_change":"2024-04-25T10:08:19.977547+0000","last_active":"2024-04-25T10:08:21.316691+0000","last_peered":"2024-04-25T10:08:21.316691+0000","last_clean":"2024-04-25T10:08:21.316691+0000","last_became_active":"2024-04-25T10:08:19.977019+0000","last_became_peered":"2024-04-25T10:08:19.977019+0000","last_unstale":"2024-04-25T10:08:21.316691+0000","last_undegraded":"2024-04-25T10:08:21.316691+0000","last_fullsized":"2024-04-25T10:08:21.316691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:20.007797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985397+0000","last_change":"2024-04-25T10:08:19.960239+0000","last_active":"2024-04-25T10:08:20.985397+0000","last_peered":"2024-04-25T10:08:20.985397+0000","last_clean":"2024-04-25T10:08:20.985397+0000","last_became_active":"2024-04-25T10:08:19.959593+0000","last_became_peered":"2024-04-25T10:08:19.959593+0000","last_unstale":"2024-04-25T10:08:20.985397+0000","last_undegraded":"2024-04-25T10:08:20.985397+0000","last_fullsized":"2024-04-25T10:08:20.985397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:15.901581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315952+0000","last_change":"2024-04-25T10:08:19.988084+0000","last_active":"2024-04-25T10:08:21.315952+0000","last_peered":"2024-04-25T10:08:21.315952+0000","last_clean":"2024-04-25T10:08:21.315952+0000","last_became_active":"2024-04-25T10:08:19.981592+0000","last_became_peered":"2024-04-25T10:08:19.981592+0000","last_unstale":"2024-04-25T10:08:21.315952+0000","last_undegraded":"2024-04-25T10:08:21.315952+0000","last_fullsized":"2024-04-25T10:08:21.315952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:55.773976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964566+0000","last_change":"2024-04-25T10:08:19.969696+0000","last_active":"2024-04-25T10:08:20.964566+0000","last_peered":"2024-04-25T10:08:20.964566+0000","last_clean":"2024-04-25T10:08:20.964566+0000","last_became_active":"2024-04-25T10:08:19.969406+0000","last_became_peered":"2024-04-25T10:08:19.969406+0000","last_unstale":"2024-04-25T10:08:20.964566+0000","last_undegraded":"2024-04-25T10:08:20.964566+0000","last_fullsized":"2024-04-25T10:08:20.964566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:21.913629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985282+0000","last_change":"2024-04-25T10:08:19.961945+0000","last_active":"2024-04-25T10:08:20.985282+0000","last_peered":"2024-04-25T10:08:20.985282+0000","last_clean":"2024-04-25T10:08:20.985282+0000","last_became_active":"2024-04-25T10:08:19.961594+0000","last_became_peered":"2024-04-25T10:08:19.961594+0000","last_unstale":"2024-04-25T10:08:20.985282+0000","last_undegraded":"2024-04-25T10:08:20.985282+0000","last_fullsized":"2024-04-25T10:08:20.985282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:12.247035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967408+0000","last_change":"2024-04-25T10:08:19.977705+0000","last_active":"2024-04-25T10:08:20.967408+0000","last_peered":"2024-04-25T10:08:20.967408+0000","last_clean":"2024-04-25T10:08:20.967408+0000","last_became_active":"2024-04-25T10:08:19.977402+0000","last_became_peered":"2024-04-25T10:08:19.977402+0000","last_unstale":"2024-04-25T10:08:20.967408+0000","last_undegraded":"2024-04-25T10:08:20.967408+0000","last_fullsized":"2024-04-25T10:08:20.967408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.745994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316909+0000","last_change":"2024-04-25T10:08:19.966171+0000","last_active":"2024-04-25T10:08:21.316909+0000","last_peered":"2024-04-25T10:08:21.316909+0000","last_clean":"2024-04-25T10:08:21.316909+0000","last_became_active":"2024-04-25T10:08:19.965818+0000","last_became_peered":"2024-04-25T10:08:19.965818+0000","last_unstale":"2024-04-25T10:08:21.316909+0000","last_undegraded":"2024-04-25T10:08:21.316909+0000","last_fullsized":"2024-04-25T10:08:21.316909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:58.749731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980057+0000","last_change":"2024-04-25T10:08:19.983504+0000","last_active":"2024-04-25T10:08:20.980057+0000","last_peered":"2024-04-25T10:08:20.980057+0000","last_clean":"2024-04-25T10:08:20.980057+0000","last_became_active":"2024-04-25T10:08:19.983043+0000","last_became_peered":"2024-04-25T10:08:19.983043+0000","last_unstale":"2024-04-25T10:08:20.980057+0000","last_undegraded":"2024-04-25T10:08:20.980057+0000","last_fullsized":"2024-04-25T10:08:20.980057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:18.293222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317782+0000","last_change":"2024-04-25T10:08:19.982135+0000","last_active":"2024-04-25T10:08:21.317782+0000","last_peered":"2024-04-25T10:08:21.317782+0000","last_clean":"2024-04-25T10:08:21.317782+0000","last_became_active":"2024-04-25T10:08:19.981838+0000","last_became_peered":"2024-04-25T10:08:19.981838+0000","last_unstale":"2024-04-25T10:08:21.317782+0000","last_undegraded":"2024-04-25T10:08:21.317782+0000","last_fullsized":"2024-04-25T10:08:21.317782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:07.697822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985662+0000","last_change":"2024-04-25T10:08:19.976777+0000","last_active":"2024-04-25T10:08:20.985662+0000","last_peered":"2024-04-25T10:08:20.985662+0000","last_clean":"2024-04-25T10:08:20.985662+0000","last_became_active":"2024-04-25T10:08:19.976340+0000","last_became_peered":"2024-04-25T10:08:19.976340+0000","last_unstale":"2024-04-25T10:08:20.985662+0000","last_undegraded":"2024-04-25T10:08:20.985662+0000","last_fullsized":"2024-04-25T10:08:20.985662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:28.581297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316149+0000","last_change":"2024-04-25T10:08:19.976990+0000","last_active":"2024-04-25T10:08:21.316149+0000","last_peered":"2024-04-25T10:08:21.316149+0000","last_clean":"2024-04-25T10:08:21.316149+0000","last_became_active":"2024-04-25T10:08:19.975772+0000","last_became_peered":"2024-04-25T10:08:19.975772+0000","last_unstale":"2024-04-25T10:08:21.316149+0000","last_undegraded":"2024-04-25T10:08:21.316149+0000","last_fullsized":"2024-04-25T10:08:21.316149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:25.962057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978689+0000","last_change":"2024-04-25T10:08:19.969262+0000","last_active":"2024-04-25T10:08:20.978689+0000","last_peered":"2024-04-25T10:08:20.978689+0000","last_clean":"2024-04-25T10:08:20.978689+0000","last_became_active":"2024-04-25T10:08:19.969006+0000","last_became_peered":"2024-04-25T10:08:19.969006+0000","last_unstale":"2024-04-25T10:08:20.978689+0000","last_undegraded":"2024-04-25T10:08:20.978689+0000","last_fullsized":"2024-04-25T10:08:20.978689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.430130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317072+0000","last_change":"2024-04-25T10:08:19.962724+0000","last_active":"2024-04-25T10:08:21.317072+0000","last_peered":"2024-04-25T10:08:21.317072+0000","last_clean":"2024-04-25T10:08:21.317072+0000","last_became_active":"2024-04-25T10:08:19.960695+0000","last_became_peered":"2024-04-25T10:08:19.960695+0000","last_unstale":"2024-04-25T10:08:21.317072+0000","last_undegraded":"2024-04-25T10:08:21.317072+0000","last_fullsized":"2024-04-25T10:08:21.317072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:48.882289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986277+0000","last_change":"2024-04-25T10:08:19.957209+0000","last_active":"2024-04-25T10:08:20.986277+0000","last_peered":"2024-04-25T10:08:20.986277+0000","last_clean":"2024-04-25T10:08:20.986277+0000","last_became_active":"2024-04-25T10:08:19.956891+0000","last_became_peered":"2024-04-25T10:08:19.956891+0000","last_unstale":"2024-04-25T10:08:20.986277+0000","last_undegraded":"2024-04-25T10:08:20.986277+0000","last_fullsized":"2024-04-25T10:08:20.986277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:12.885485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961788+0000","last_change":"2024-04-25T10:08:19.979617+0000","last_active":"2024-04-25T10:08:20.961788+0000","last_peered":"2024-04-25T10:08:20.961788+0000","last_clean":"2024-04-25T10:08:20.961788+0000","last_became_active":"2024-04-25T10:08:19.979324+0000","last_became_peered":"2024-04-25T10:08:19.979324+0000","last_unstale":"2024-04-25T10:08:20.961788+0000","last_undegraded":"2024-04-25T10:08:20.961788+0000","last_fullsized":"2024-04-25T10:08:20.961788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:14.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316220+0000","last_change":"2024-04-25T10:08:19.979309+0000","last_active":"2024-04-25T10:08:21.316220+0000","last_peered":"2024-04-25T10:08:21.316220+0000","last_clean":"2024-04-25T10:08:21.316220+0000","last_became_active":"2024-04-25T10:08:19.978859+0000","last_became_peered":"2024-04-25T10:08:19.978859+0000","last_unstale":"2024-04-25T10:08:21.316220+0000","last_undegraded":"2024-04-25T10:08:21.316220+0000","last_fullsized":"2024-04-25T10:08:21.316220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.912266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985686+0000","last_change":"2024-04-25T10:08:19.967430+0000","last_active":"2024-04-25T10:08:20.985686+0000","last_peered":"2024-04-25T10:08:20.985686+0000","last_clean":"2024-04-25T10:08:20.985686+0000","last_became_active":"2024-04-25T10:08:19.967185+0000","last_became_peered":"2024-04-25T10:08:19.967185+0000","last_unstale":"2024-04-25T10:08:20.985686+0000","last_undegraded":"2024-04-25T10:08:20.985686+0000","last_fullsized":"2024-04-25T10:08:20.985686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:40.041741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966697+0000","last_change":"2024-04-25T10:08:19.973682+0000","last_active":"2024-04-25T10:08:20.966697+0000","last_peered":"2024-04-25T10:08:20.966697+0000","last_clean":"2024-04-25T10:08:20.966697+0000","last_became_active":"2024-04-25T10:08:19.972892+0000","last_became_peered":"2024-04-25T10:08:19.972892+0000","last_unstale":"2024-04-25T10:08:20.966697+0000","last_undegraded":"2024-04-25T10:08:20.966697+0000","last_fullsized":"2024-04-25T10:08:20.966697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:31.822103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985412+0000","last_change":"2024-04-25T10:08:19.973046+0000","last_active":"2024-04-25T10:08:20.985412+0000","last_peered":"2024-04-25T10:08:20.985412+0000","last_clean":"2024-04-25T10:08:20.985412+0000","last_became_active":"2024-04-25T10:08:19.972730+0000","last_became_peered":"2024-04-25T10:08:19.972730+0000","last_unstale":"2024-04-25T10:08:20.985412+0000","last_undegraded":"2024-04-25T10:08:20.985412+0000","last_fullsized":"2024-04-25T10:08:20.985412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.630445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966159+0000","last_change":"2024-04-25T10:08:19.975999+0000","last_active":"2024-04-25T10:08:20.966159+0000","last_peered":"2024-04-25T10:08:20.966159+0000","last_clean":"2024-04-25T10:08:20.966159+0000","last_became_active":"2024-04-25T10:08:19.975638+0000","last_became_peered":"2024-04-25T10:08:19.975638+0000","last_unstale":"2024-04-25T10:08:20.966159+0000","last_undegraded":"2024-04-25T10:08:20.966159+0000","last_fullsized":"2024-04-25T10:08:20.966159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.059256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978614+0000","last_change":"2024-04-25T10:08:19.974606+0000","last_active":"2024-04-25T10:08:20.978614+0000","last_peered":"2024-04-25T10:08:20.978614+0000","last_clean":"2024-04-25T10:08:20.978614+0000","last_became_active":"2024-04-25T10:08:19.974336+0000","last_became_peered":"2024-04-25T10:08:19.974336+0000","last_unstale":"2024-04-25T10:08:20.978614+0000","last_undegraded":"2024-04-25T10:08:20.978614+0000","last_fullsized":"2024-04-25T10:08:20.978614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.910865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980532+0000","last_change":"2024-04-25T10:08:19.982998+0000","last_active":"2024-04-25T10:08:20.980532+0000","last_peered":"2024-04-25T10:08:20.980532+0000","last_clean":"2024-04-25T10:08:20.980532+0000","last_became_active":"2024-04-25T10:08:19.982698+0000","last_became_peered":"2024-04-25T10:08:19.982698+0000","last_unstale":"2024-04-25T10:08:20.980532+0000","last_undegraded":"2024-04-25T10:08:20.980532+0000","last_fullsized":"2024-04-25T10:08:20.980532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:36.340349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317360+0000","last_change":"2024-04-25T10:08:19.979566+0000","last_active":"2024-04-25T10:08:21.317360+0000","last_peered":"2024-04-25T10:08:21.317360+0000","last_clean":"2024-04-25T10:08:21.317360+0000","last_became_active":"2024-04-25T10:08:19.979276+0000","last_became_peered":"2024-04-25T10:08:19.979276+0000","last_unstale":"2024-04-25T10:08:21.317360+0000","last_undegraded":"2024-04-25T10:08:21.317360+0000","last_fullsized":"2024-04-25T10:08:21.317360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:40.887734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318210+0000","last_change":"2024-04-25T10:08:19.964116+0000","last_active":"2024-04-25T10:08:21.318210+0000","last_peered":"2024-04-25T10:08:21.318210+0000","last_clean":"2024-04-25T10:08:21.318210+0000","last_became_active":"2024-04-25T10:08:19.960959+0000","last_became_peered":"2024-04-25T10:08:19.960959+0000","last_unstale":"2024-04-25T10:08:21.318210+0000","last_undegraded":"2024-04-25T10:08:21.318210+0000","last_fullsized":"2024-04-25T10:08:21.318210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.910865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985944+0000","last_change":"2024-04-25T10:08:19.974107+0000","last_active":"2024-04-25T10:08:20.985944+0000","last_peered":"2024-04-25T10:08:20.985944+0000","last_clean":"2024-04-25T10:08:20.985944+0000","last_became_active":"2024-04-25T10:08:19.973726+0000","last_became_peered":"2024-04-25T10:08:19.973726+0000","last_unstale":"2024-04-25T10:08:20.985944+0000","last_undegraded":"2024-04-25T10:08:20.985944+0000","last_fullsized":"2024-04-25T10:08:20.985944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:45.655703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979726+0000","last_change":"2024-04-25T10:08:19.991517+0000","last_active":"2024-04-25T10:08:20.979726+0000","last_peered":"2024-04-25T10:08:20.979726+0000","last_clean":"2024-04-25T10:08:20.979726+0000","last_became_active":"2024-04-25T10:08:19.991209+0000","last_became_peered":"2024-04-25T10:08:19.991209+0000","last_unstale":"2024-04-25T10:08:20.979726+0000","last_undegraded":"2024-04-25T10:08:20.979726+0000","last_fullsized":"2024-04-25T10:08:20.979726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:05.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316911+0000","last_change":"2024-04-25T10:08:19.963461+0000","last_active":"2024-04-25T10:08:21.316911+0000","last_peered":"2024-04-25T10:08:21.316911+0000","last_clean":"2024-04-25T10:08:21.316911+0000","last_became_active":"2024-04-25T10:08:19.963127+0000","last_became_peered":"2024-04-25T10:08:19.963127+0000","last_unstale":"2024-04-25T10:08:21.316911+0000","last_undegraded":"2024-04-25T10:08:21.316911+0000","last_fullsized":"2024-04-25T10:08:21.316911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:43.269695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961619+0000","last_change":"2024-04-25T10:08:19.962954+0000","last_active":"2024-04-25T10:08:20.961619+0000","last_peered":"2024-04-25T10:08:20.961619+0000","last_clean":"2024-04-25T10:08:20.961619+0000","last_became_active":"2024-04-25T10:08:19.962644+0000","last_became_peered":"2024-04-25T10:08:19.962644+0000","last_unstale":"2024-04-25T10:08:20.961619+0000","last_undegraded":"2024-04-25T10:08:20.961619+0000","last_fullsized":"2024-04-25T10:08:20.961619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317031+0000","last_change":"2024-04-25T10:08:19.967639+0000","last_active":"2024-04-25T10:08:21.317031+0000","last_peered":"2024-04-25T10:08:21.317031+0000","last_clean":"2024-04-25T10:08:21.317031+0000","last_became_active":"2024-04-25T10:08:19.967335+0000","last_became_peered":"2024-04-25T10:08:19.967335+0000","last_unstale":"2024-04-25T10:08:21.317031+0000","last_undegraded":"2024-04-25T10:08:21.317031+0000","last_fullsized":"2024-04-25T10:08:21.317031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:41.088941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961618+0000","last_change":"2024-04-25T10:08:19.981351+0000","last_active":"2024-04-25T10:08:20.961618+0000","last_peered":"2024-04-25T10:08:20.961618+0000","last_clean":"2024-04-25T10:08:20.961618+0000","last_became_active":"2024-04-25T10:08:19.979781+0000","last_became_peered":"2024-04-25T10:08:19.979781+0000","last_unstale":"2024-04-25T10:08:20.961618+0000","last_undegraded":"2024-04-25T10:08:20.961618+0000","last_fullsized":"2024-04-25T10:08:20.961618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:20.725052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986262+0000","last_change":"2024-04-25T10:08:18.942173+0000","last_active":"2024-04-25T10:08:20.986262+0000","last_peered":"2024-04-25T10:08:20.986262+0000","last_clean":"2024-04-25T10:08:20.986262+0000","last_became_active":"2024-04-25T10:08:18.941884+0000","last_became_peered":"2024-04-25T10:08:18.941884+0000","last_unstale":"2024-04-25T10:08:20.986262+0000","last_undegraded":"2024-04-25T10:08:20.986262+0000","last_fullsized":"2024-04-25T10:08:20.986262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:55.696101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978759+0000","last_change":"2024-04-25T10:08:19.976689+0000","last_active":"2024-04-25T10:08:20.978759+0000","last_peered":"2024-04-25T10:08:20.978759+0000","last_clean":"2024-04-25T10:08:20.978759+0000","last_became_active":"2024-04-25T10:08:19.975469+0000","last_became_peered":"2024-04-25T10:08:19.975469+0000","last_unstale":"2024-04-25T10:08:20.978759+0000","last_undegraded":"2024-04-25T10:08:20.978759+0000","last_fullsized":"2024-04-25T10:08:20.978759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:35.566767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318454+0000","last_change":"2024-04-25T10:08:18.942821+0000","last_active":"2024-04-25T10:08:21.318454+0000","last_peered":"2024-04-25T10:08:21.318454+0000","last_clean":"2024-04-25T10:08:21.318454+0000","last_became_active":"2024-04-25T10:08:18.942482+0000","last_became_peered":"2024-04-25T10:08:18.942482+0000","last_unstale":"2024-04-25T10:08:21.318454+0000","last_undegraded":"2024-04-25T10:08:21.318454+0000","last_fullsized":"2024-04-25T10:08:21.318454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:04.593211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966914+0000","last_change":"2024-04-25T10:08:19.978470+0000","last_active":"2024-04-25T10:08:20.966914+0000","last_peered":"2024-04-25T10:08:20.966914+0000","last_clean":"2024-04-25T10:08:20.966914+0000","last_became_active":"2024-04-25T10:08:19.978112+0000","last_became_peered":"2024-04-25T10:08:19.978112+0000","last_unstale":"2024-04-25T10:08:20.966914+0000","last_undegraded":"2024-04-25T10:08:20.966914+0000","last_fullsized":"2024-04-25T10:08:20.966914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:27.851042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318313+0000","last_change":"2024-04-25T10:08:18.940512+0000","last_active":"2024-04-25T10:08:21.318313+0000","last_peered":"2024-04-25T10:08:21.318313+0000","last_clean":"2024-04-25T10:08:21.318313+0000","last_became_active":"2024-04-25T10:08:18.940191+0000","last_became_peered":"2024-04-25T10:08:18.940191+0000","last_unstale":"2024-04-25T10:08:21.318313+0000","last_undegraded":"2024-04-25T10:08:21.318313+0000","last_fullsized":"2024-04-25T10:08:21.318313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:06.682971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966725+0000","last_change":"2024-04-25T10:08:19.968676+0000","last_active":"2024-04-25T10:08:20.966725+0000","last_peered":"2024-04-25T10:08:20.966725+0000","last_clean":"2024-04-25T10:08:20.966725+0000","last_became_active":"2024-04-25T10:08:19.968356+0000","last_became_peered":"2024-04-25T10:08:19.968356+0000","last_unstale":"2024-04-25T10:08:20.966725+0000","last_undegraded":"2024-04-25T10:08:20.966725+0000","last_fullsized":"2024-04-25T10:08:20.966725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:41.660213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965231+0000","last_change":"2024-04-25T10:08:18.936196+0000","last_active":"2024-04-25T10:08:20.965231+0000","last_peered":"2024-04-25T10:08:20.965231+0000","last_clean":"2024-04-25T10:08:20.965231+0000","last_became_active":"2024-04-25T10:08:18.935774+0000","last_became_peered":"2024-04-25T10:08:18.935774+0000","last_unstale":"2024-04-25T10:08:20.965231+0000","last_undegraded":"2024-04-25T10:08:20.965231+0000","last_fullsized":"2024-04-25T10:08:20.965231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:30.556478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979466+0000","last_change":"2024-04-25T10:08:19.980061+0000","last_active":"2024-04-25T10:08:20.979466+0000","last_peered":"2024-04-25T10:08:20.979466+0000","last_clean":"2024-04-25T10:08:20.979466+0000","last_became_active":"2024-04-25T10:08:19.979701+0000","last_became_peered":"2024-04-25T10:08:19.979701+0000","last_unstale":"2024-04-25T10:08:20.979466+0000","last_undegraded":"2024-04-25T10:08:20.979466+0000","last_fullsized":"2024-04-25T10:08:20.979466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:55.773976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966991+0000","last_change":"2024-04-25T10:08:18.951337+0000","last_active":"2024-04-25T10:08:20.966991+0000","last_peered":"2024-04-25T10:08:20.966991+0000","last_clean":"2024-04-25T10:08:20.966991+0000","last_became_active":"2024-04-25T10:08:18.950508+0000","last_became_peered":"2024-04-25T10:08:18.950508+0000","last_unstale":"2024-04-25T10:08:20.966991+0000","last_undegraded":"2024-04-25T10:08:20.966991+0000","last_fullsized":"2024-04-25T10:08:20.966991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:36.306729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980178+0000","last_change":"2024-04-25T10:08:19.982175+0000","last_active":"2024-04-25T10:08:20.980178+0000","last_peered":"2024-04-25T10:08:20.980178+0000","last_clean":"2024-04-25T10:08:20.980178+0000","last_became_active":"2024-04-25T10:08:19.981840+0000","last_became_peered":"2024-04-25T10:08:19.981840+0000","last_unstale":"2024-04-25T10:08:20.980178+0000","last_undegraded":"2024-04-25T10:08:20.980178+0000","last_fullsized":"2024-04-25T10:08:20.980178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.656634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316325+0000","last_change":"2024-04-25T10:08:18.948996+0000","last_active":"2024-04-25T10:08:21.316325+0000","last_peered":"2024-04-25T10:08:21.316325+0000","last_clean":"2024-04-25T10:08:21.316325+0000","last_became_active":"2024-04-25T10:08:18.948665+0000","last_became_peered":"2024-04-25T10:08:18.948665+0000","last_unstale":"2024-04-25T10:08:21.316325+0000","last_undegraded":"2024-04-25T10:08:21.316325+0000","last_fullsized":"2024-04-25T10:08:21.316325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:20.933525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967749+0000","last_change":"2024-04-25T10:08:19.973442+0000","last_active":"2024-04-25T10:08:20.967749+0000","last_peered":"2024-04-25T10:08:20.967749+0000","last_clean":"2024-04-25T10:08:20.967749+0000","last_became_active":"2024-04-25T10:08:19.973105+0000","last_became_peered":"2024-04-25T10:08:19.973105+0000","last_unstale":"2024-04-25T10:08:20.967749+0000","last_undegraded":"2024-04-25T10:08:20.967749+0000","last_fullsized":"2024-04-25T10:08:20.967749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:55.385237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978968+0000","last_change":"2024-04-25T10:08:18.937233+0000","last_active":"2024-04-25T10:08:20.978968+0000","last_peered":"2024-04-25T10:08:20.978968+0000","last_clean":"2024-04-25T10:08:20.978968+0000","last_became_active":"2024-04-25T10:08:18.936966+0000","last_became_peered":"2024-04-25T10:08:18.936966+0000","last_unstale":"2024-04-25T10:08:20.978968+0000","last_undegraded":"2024-04-25T10:08:20.978968+0000","last_fullsized":"2024-04-25T10:08:20.978968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:06.682971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966135+0000","last_change":"2024-04-25T10:08:19.970670+0000","last_active":"2024-04-25T10:08:20.966135+0000","last_peered":"2024-04-25T10:08:20.966135+0000","last_clean":"2024-04-25T10:08:20.966135+0000","last_became_active":"2024-04-25T10:08:19.970387+0000","last_became_peered":"2024-04-25T10:08:19.970387+0000","last_unstale":"2024-04-25T10:08:20.966135+0000","last_undegraded":"2024-04-25T10:08:20.966135+0000","last_fullsized":"2024-04-25T10:08:20.966135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.022397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985302+0000","last_change":"2024-04-25T10:08:18.944811+0000","last_active":"2024-04-25T10:08:20.985302+0000","last_peered":"2024-04-25T10:08:20.985302+0000","last_clean":"2024-04-25T10:08:20.985302+0000","last_became_active":"2024-04-25T10:08:18.944531+0000","last_became_peered":"2024-04-25T10:08:18.944531+0000","last_unstale":"2024-04-25T10:08:20.985302+0000","last_undegraded":"2024-04-25T10:08:20.985302+0000","last_fullsized":"2024-04-25T10:08:20.985302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:16.882336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978621+0000","last_change":"2024-04-25T10:08:19.980510+0000","last_active":"2024-04-25T10:08:20.978621+0000","last_peered":"2024-04-25T10:08:20.978621+0000","last_clean":"2024-04-25T10:08:20.978621+0000","last_became_active":"2024-04-25T10:08:19.979752+0000","last_became_peered":"2024-04-25T10:08:19.979752+0000","last_unstale":"2024-04-25T10:08:20.978621+0000","last_undegraded":"2024-04-25T10:08:20.978621+0000","last_fullsized":"2024-04-25T10:08:20.978621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:52.728570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318544+0000","last_change":"2024-04-25T10:08:18.941665+0000","last_active":"2024-04-25T10:08:21.318544+0000","last_peered":"2024-04-25T10:08:21.318544+0000","last_clean":"2024-04-25T10:08:21.318544+0000","last_became_active":"2024-04-25T10:08:18.941387+0000","last_became_peered":"2024-04-25T10:08:18.941387+0000","last_unstale":"2024-04-25T10:08:21.318544+0000","last_undegraded":"2024-04-25T10:08:21.318544+0000","last_fullsized":"2024-04-25T10:08:21.318544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:57.734880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316464+0000","last_change":"2024-04-25T10:08:19.975526+0000","last_active":"2024-04-25T10:08:21.316464+0000","last_peered":"2024-04-25T10:08:21.316464+0000","last_clean":"2024-04-25T10:08:21.316464+0000","last_became_active":"2024-04-25T10:08:19.975198+0000","last_became_peered":"2024-04-25T10:08:19.975198+0000","last_unstale":"2024-04-25T10:08:21.316464+0000","last_undegraded":"2024-04-25T10:08:21.316464+0000","last_fullsized":"2024-04-25T10:08:21.316464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:27.968601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317914+0000","last_change":"2024-04-25T10:08:18.949137+0000","last_active":"2024-04-25T10:08:21.317914+0000","last_peered":"2024-04-25T10:08:21.317914+0000","last_clean":"2024-04-25T10:08:21.317914+0000","last_became_active":"2024-04-25T10:08:18.948667+0000","last_became_peered":"2024-04-25T10:08:18.948667+0000","last_unstale":"2024-04-25T10:08:21.317914+0000","last_undegraded":"2024-04-25T10:08:21.317914+0000","last_fullsized":"2024-04-25T10:08:21.317914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:15.026390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316152+0000","last_change":"2024-04-25T10:08:19.976587+0000","last_active":"2024-04-25T10:08:21.316152+0000","last_peered":"2024-04-25T10:08:21.316152+0000","last_clean":"2024-04-25T10:08:21.316152+0000","last_became_active":"2024-04-25T10:08:19.975088+0000","last_became_peered":"2024-04-25T10:08:19.975088+0000","last_unstale":"2024-04-25T10:08:21.316152+0000","last_undegraded":"2024-04-25T10:08:21.316152+0000","last_fullsized":"2024-04-25T10:08:21.316152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:05.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986009+0000","last_change":"2024-04-25T10:08:18.947481+0000","last_active":"2024-04-25T10:08:20.986009+0000","last_peered":"2024-04-25T10:08:20.986009+0000","last_clean":"2024-04-25T10:08:20.986009+0000","last_became_active":"2024-04-25T10:08:18.947151+0000","last_became_peered":"2024-04-25T10:08:18.947151+0000","last_unstale":"2024-04-25T10:08:20.986009+0000","last_undegraded":"2024-04-25T10:08:20.986009+0000","last_fullsized":"2024-04-25T10:08:20.986009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:19.887767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962182+0000","last_change":"2024-04-25T10:08:19.988375+0000","last_active":"2024-04-25T10:08:20.962182+0000","last_peered":"2024-04-25T10:08:20.962182+0000","last_clean":"2024-04-25T10:08:20.962182+0000","last_became_active":"2024-04-25T10:08:19.986299+0000","last_became_peered":"2024-04-25T10:08:19.986299+0000","last_unstale":"2024-04-25T10:08:20.962182+0000","last_undegraded":"2024-04-25T10:08:20.962182+0000","last_fullsized":"2024-04-25T10:08:20.962182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.022397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965347+0000","last_change":"2024-04-25T10:08:18.948358+0000","last_active":"2024-04-25T10:08:20.965347+0000","last_peered":"2024-04-25T10:08:20.965347+0000","last_clean":"2024-04-25T10:08:20.965347+0000","last_became_active":"2024-04-25T10:08:18.945639+0000","last_became_peered":"2024-04-25T10:08:18.945639+0000","last_unstale":"2024-04-25T10:08:20.965347+0000","last_undegraded":"2024-04-25T10:08:20.965347+0000","last_fullsized":"2024-04-25T10:08:20.965347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.003395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317648+0000","last_change":"2024-04-25T10:08:19.975766+0000","last_active":"2024-04-25T10:08:21.317648+0000","last_peered":"2024-04-25T10:08:21.317648+0000","last_clean":"2024-04-25T10:08:21.317648+0000","last_became_active":"2024-04-25T10:08:19.975302+0000","last_became_peered":"2024-04-25T10:08:19.975302+0000","last_unstale":"2024-04-25T10:08:21.317648+0000","last_undegraded":"2024-04-25T10:08:21.317648+0000","last_fullsized":"2024-04-25T10:08:21.317648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:42.765039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316092+0000","last_change":"2024-04-25T10:08:18.947045+0000","last_active":"2024-04-25T10:08:21.316092+0000","last_peered":"2024-04-25T10:08:21.316092+0000","last_clean":"2024-04-25T10:08:21.316092+0000","last_became_active":"2024-04-25T10:08:18.946758+0000","last_became_peered":"2024-04-25T10:08:18.946758+0000","last_unstale":"2024-04-25T10:08:21.316092+0000","last_undegraded":"2024-04-25T10:08:21.316092+0000","last_fullsized":"2024-04-25T10:08:21.316092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.420066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964017+0000","last_change":"2024-04-25T10:08:19.970045+0000","last_active":"2024-04-25T10:08:20.964017+0000","last_peered":"2024-04-25T10:08:20.964017+0000","last_clean":"2024-04-25T10:08:20.964017+0000","last_became_active":"2024-04-25T10:08:19.968064+0000","last_became_peered":"2024-04-25T10:08:19.968064+0000","last_unstale":"2024-04-25T10:08:20.964017+0000","last_undegraded":"2024-04-25T10:08:20.964017+0000","last_fullsized":"2024-04-25T10:08:20.964017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:22.965572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317343+0000","last_change":"2024-04-25T10:08:18.948243+0000","last_active":"2024-04-25T10:08:21.317343+0000","last_peered":"2024-04-25T10:08:21.317343+0000","last_clean":"2024-04-25T10:08:21.317343+0000","last_became_active":"2024-04-25T10:08:18.946281+0000","last_became_peered":"2024-04-25T10:08:18.946281+0000","last_unstale":"2024-04-25T10:08:21.317343+0000","last_undegraded":"2024-04-25T10:08:21.317343+0000","last_fullsized":"2024-04-25T10:08:21.317343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:18.992945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965558+0000","last_change":"2024-04-25T10:08:19.966278+0000","last_active":"2024-04-25T10:08:20.965558+0000","last_peered":"2024-04-25T10:08:20.965558+0000","last_clean":"2024-04-25T10:08:20.965558+0000","last_became_active":"2024-04-25T10:08:19.965964+0000","last_became_peered":"2024-04-25T10:08:19.965964+0000","last_unstale":"2024-04-25T10:08:20.965558+0000","last_undegraded":"2024-04-25T10:08:20.965558+0000","last_fullsized":"2024-04-25T10:08:20.965558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:14.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985104+0000","last_change":"2024-04-25T10:08:18.945515+0000","last_active":"2024-04-25T10:08:20.985104+0000","last_peered":"2024-04-25T10:08:20.985104+0000","last_clean":"2024-04-25T10:08:20.985104+0000","last_became_active":"2024-04-25T10:08:18.944832+0000","last_became_peered":"2024-04-25T10:08:18.944832+0000","last_unstale":"2024-04-25T10:08:20.985104+0000","last_undegraded":"2024-04-25T10:08:20.985104+0000","last_fullsized":"2024-04-25T10:08:20.985104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:08.377481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966114+0000","last_change":"2024-04-25T10:08:19.974531+0000","last_active":"2024-04-25T10:08:20.966114+0000","last_peered":"2024-04-25T10:08:20.966114+0000","last_clean":"2024-04-25T10:08:20.966114+0000","last_became_active":"2024-04-25T10:08:19.972210+0000","last_became_peered":"2024-04-25T10:08:19.972210+0000","last_unstale":"2024-04-25T10:08:20.966114+0000","last_undegraded":"2024-04-25T10:08:20.966114+0000","last_fullsized":"2024-04-25T10:08:20.966114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:16.427011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"84'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978887+0000","last_change":"2024-04-25T10:08:18.946298+0000","last_active":"2024-04-25T10:08:20.978887+0000","last_peered":"2024-04-25T10:08:20.978887+0000","last_clean":"2024-04-25T10:08:20.978887+0000","last_became_active":"2024-04-25T10:08:18.946020+0000","last_became_peered":"2024-04-25T10:08:18.946020+0000","last_unstale":"2024-04-25T10:08:20.978887+0000","last_undegraded":"2024-04-25T10:08:20.978887+0000","last_fullsized":"2024-04-25T10:08:20.978887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:46.279014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315631+0000","last_change":"2024-04-25T10:08:19.973249+0000","last_active":"2024-04-25T10:08:21.315631+0000","last_peered":"2024-04-25T10:08:21.315631+0000","last_clean":"2024-04-25T10:08:21.315631+0000","last_became_active":"2024-04-25T10:08:19.972893+0000","last_became_peered":"2024-04-25T10:08:19.972893+0000","last_unstale":"2024-04-25T10:08:21.315631+0000","last_undegraded":"2024-04-25T10:08:21.315631+0000","last_fullsized":"2024-04-25T10:08:21.315631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.430130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318757+0000","last_change":"2024-04-25T10:08:18.944678+0000","last_active":"2024-04-25T10:08:21.318757+0000","last_peered":"2024-04-25T10:08:21.318757+0000","last_clean":"2024-04-25T10:08:21.318757+0000","last_became_active":"2024-04-25T10:08:18.944351+0000","last_became_peered":"2024-04-25T10:08:18.944351+0000","last_unstale":"2024-04-25T10:08:21.318757+0000","last_undegraded":"2024-04-25T10:08:21.318757+0000","last_fullsized":"2024-04-25T10:08:21.318757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986013+0000","last_change":"2024-04-25T10:08:19.957278+0000","last_active":"2024-04-25T10:08:20.986013+0000","last_peered":"2024-04-25T10:08:20.986013+0000","last_clean":"2024-04-25T10:08:20.986013+0000","last_became_active":"2024-04-25T10:08:19.955777+0000","last_became_peered":"2024-04-25T10:08:19.955777+0000","last_unstale":"2024-04-25T10:08:20.986013+0000","last_undegraded":"2024-04-25T10:08:20.986013+0000","last_fullsized":"2024-04-25T10:08:20.986013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:19.944001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967795+0000","last_change":"2024-04-25T10:08:18.946830+0000","last_active":"2024-04-25T10:08:20.967795+0000","last_peered":"2024-04-25T10:08:20.967795+0000","last_clean":"2024-04-25T10:08:20.967795+0000","last_became_active":"2024-04-25T10:08:18.945619+0000","last_became_peered":"2024-04-25T10:08:18.945619+0000","last_unstale":"2024-04-25T10:08:20.967795+0000","last_undegraded":"2024-04-25T10:08:20.967795+0000","last_fullsized":"2024-04-25T10:08:20.967795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.980984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985144+0000","last_change":"2024-04-25T10:08:19.963094+0000","last_active":"2024-04-25T10:08:20.985144+0000","last_peered":"2024-04-25T10:08:20.985144+0000","last_clean":"2024-04-25T10:08:20.985144+0000","last_became_active":"2024-04-25T10:08:19.962771+0000","last_became_peered":"2024-04-25T10:08:19.962771+0000","last_unstale":"2024-04-25T10:08:20.985144+0000","last_undegraded":"2024-04-25T10:08:20.985144+0000","last_fullsized":"2024-04-25T10:08:20.985144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:39.425266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986176+0000","last_change":"2024-04-25T10:08:18.944720+0000","last_active":"2024-04-25T10:08:20.986176+0000","last_peered":"2024-04-25T10:08:20.986176+0000","last_clean":"2024-04-25T10:08:20.986176+0000","last_became_active":"2024-04-25T10:08:18.944425+0000","last_became_peered":"2024-04-25T10:08:18.944425+0000","last_unstale":"2024-04-25T10:08:20.986176+0000","last_undegraded":"2024-04-25T10:08:20.986176+0000","last_fullsized":"2024-04-25T10:08:20.986176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:35.057954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315850+0000","last_change":"2024-04-25T10:08:19.979662+0000","last_active":"2024-04-25T10:08:21.315850+0000","last_peered":"2024-04-25T10:08:21.315850+0000","last_clean":"2024-04-25T10:08:21.315850+0000","last_became_active":"2024-04-25T10:08:19.979357+0000","last_became_peered":"2024-04-25T10:08:19.979357+0000","last_unstale":"2024-04-25T10:08:21.315850+0000","last_undegraded":"2024-04-25T10:08:21.315850+0000","last_fullsized":"2024-04-25T10:08:21.315850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:35.566767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962055+0000","last_change":"2024-04-25T10:08:18.941845+0000","last_active":"2024-04-25T10:08:20.962055+0000","last_peered":"2024-04-25T10:08:20.962055+0000","last_clean":"2024-04-25T10:08:20.962055+0000","last_became_active":"2024-04-25T10:08:18.941177+0000","last_became_peered":"2024-04-25T10:08:18.941177+0000","last_unstale":"2024-04-25T10:08:20.962055+0000","last_undegraded":"2024-04-25T10:08:20.962055+0000","last_fullsized":"2024-04-25T10:08:20.962055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:40.645361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979531+0000","last_change":"2024-04-25T10:08:19.976404+0000","last_active":"2024-04-25T10:08:20.979531+0000","last_peered":"2024-04-25T10:08:20.979531+0000","last_clean":"2024-04-25T10:08:20.979531+0000","last_became_active":"2024-04-25T10:08:19.976082+0000","last_became_peered":"2024-04-25T10:08:19.976082+0000","last_unstale":"2024-04-25T10:08:20.979531+0000","last_undegraded":"2024-04-25T10:08:20.979531+0000","last_fullsized":"2024-04-25T10:08:20.979531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:25.962057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315872+0000","last_change":"2024-04-25T10:08:18.930499+0000","last_active":"2024-04-25T10:08:21.315872+0000","last_peered":"2024-04-25T10:08:21.315872+0000","last_clean":"2024-04-25T10:08:21.315872+0000","last_became_active":"2024-04-25T10:08:18.928003+0000","last_became_peered":"2024-04-25T10:08:18.928003+0000","last_unstale":"2024-04-25T10:08:21.315872+0000","last_undegraded":"2024-04-25T10:08:21.315872+0000","last_fullsized":"2024-04-25T10:08:21.315872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:45.925929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980532+0000","last_change":"2024-04-25T10:08:19.979443+0000","last_active":"2024-04-25T10:08:20.980532+0000","last_peered":"2024-04-25T10:08:20.980532+0000","last_clean":"2024-04-25T10:08:20.980532+0000","last_became_active":"2024-04-25T10:08:19.979179+0000","last_became_peered":"2024-04-25T10:08:19.979179+0000","last_unstale":"2024-04-25T10:08:20.980532+0000","last_undegraded":"2024-04-25T10:08:20.980532+0000","last_fullsized":"2024-04-25T10:08:20.980532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:08.422234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315633+0000","last_change":"2024-04-25T10:08:18.933138+0000","last_active":"2024-04-25T10:08:21.315633+0000","last_peered":"2024-04-25T10:08:21.315633+0000","last_clean":"2024-04-25T10:08:21.315633+0000","last_became_active":"2024-04-25T10:08:18.932647+0000","last_became_peered":"2024-04-25T10:08:18.932647+0000","last_unstale":"2024-04-25T10:08:21.315633+0000","last_undegraded":"2024-04-25T10:08:21.315633+0000","last_fullsized":"2024-04-25T10:08:21.315633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.341616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966535+0000","last_change":"2024-04-25T10:08:19.965027+0000","last_active":"2024-04-25T10:08:20.966535+0000","last_peered":"2024-04-25T10:08:20.966535+0000","last_clean":"2024-04-25T10:08:20.966535+0000","last_became_active":"2024-04-25T10:08:19.963117+0000","last_became_peered":"2024-04-25T10:08:19.963117+0000","last_unstale":"2024-04-25T10:08:20.966535+0000","last_undegraded":"2024-04-25T10:08:20.966535+0000","last_fullsized":"2024-04-25T10:08:20.966535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.544140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315938+0000","last_change":"2024-04-25T10:08:18.934845+0000","last_active":"2024-04-25T10:08:21.315938+0000","last_peered":"2024-04-25T10:08:21.315938+0000","last_clean":"2024-04-25T10:08:21.315938+0000","last_became_active":"2024-04-25T10:08:18.931926+0000","last_became_peered":"2024-04-25T10:08:18.931926+0000","last_unstale":"2024-04-25T10:08:21.315938+0000","last_undegraded":"2024-04-25T10:08:21.315938+0000","last_fullsized":"2024-04-25T10:08:21.315938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:43.165217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965046+0000","last_change":"2024-04-25T10:08:19.966296+0000","last_active":"2024-04-25T10:08:20.965046+0000","last_peered":"2024-04-25T10:08:20.965046+0000","last_clean":"2024-04-25T10:08:20.965046+0000","last_became_active":"2024-04-25T10:08:19.966008+0000","last_became_peered":"2024-04-25T10:08:19.966008+0000","last_unstale":"2024-04-25T10:08:20.965046+0000","last_undegraded":"2024-04-25T10:08:20.965046+0000","last_fullsized":"2024-04-25T10:08:20.965046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.324062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317397+0000","last_change":"2024-04-25T10:08:18.947796+0000","last_active":"2024-04-25T10:08:21.317397+0000","last_peered":"2024-04-25T10:08:21.317397+0000","last_clean":"2024-04-25T10:08:21.317397+0000","last_became_active":"2024-04-25T10:08:18.947508+0000","last_became_peered":"2024-04-25T10:08:18.947508+0000","last_unstale":"2024-04-25T10:08:21.317397+0000","last_undegraded":"2024-04-25T10:08:21.317397+0000","last_fullsized":"2024-04-25T10:08:21.317397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:47.962610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964945+0000","last_change":"2024-04-25T10:08:19.971283+0000","last_active":"2024-04-25T10:08:20.964945+0000","last_peered":"2024-04-25T10:08:20.964945+0000","last_clean":"2024-04-25T10:08:20.964945+0000","last_became_active":"2024-04-25T10:08:19.971000+0000","last_became_peered":"2024-04-25T10:08:19.971000+0000","last_unstale":"2024-04-25T10:08:20.964945+0000","last_undegraded":"2024-04-25T10:08:20.964945+0000","last_fullsized":"2024-04-25T10:08:20.964945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:20.725052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.983901+0000","last_change":"2024-04-25T10:08:18.944659+0000","last_active":"2024-04-25T10:08:20.983901+0000","last_peered":"2024-04-25T10:08:20.983901+0000","last_clean":"2024-04-25T10:08:20.983901+0000","last_became_active":"2024-04-25T10:08:18.944306+0000","last_became_peered":"2024-04-25T10:08:18.944306+0000","last_unstale":"2024-04-25T10:08:20.983901+0000","last_undegraded":"2024-04-25T10:08:20.983901+0000","last_fullsized":"2024-04-25T10:08:20.983901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:42.254843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967151+0000","last_change":"2024-04-25T10:08:19.978770+0000","last_active":"2024-04-25T10:08:20.967151+0000","last_peered":"2024-04-25T10:08:20.967151+0000","last_clean":"2024-04-25T10:08:20.967151+0000","last_became_active":"2024-04-25T10:08:19.978468+0000","last_became_peered":"2024-04-25T10:08:19.978468+0000","last_unstale":"2024-04-25T10:08:20.967151+0000","last_undegraded":"2024-04-25T10:08:20.967151+0000","last_fullsized":"2024-04-25T10:08:20.967151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:25.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318583+0000","last_change":"2024-04-25T10:08:18.945901+0000","last_active":"2024-04-25T10:08:21.318583+0000","last_peered":"2024-04-25T10:08:21.318583+0000","last_clean":"2024-04-25T10:08:21.318583+0000","last_became_active":"2024-04-25T10:08:18.945568+0000","last_became_peered":"2024-04-25T10:08:18.945568+0000","last_unstale":"2024-04-25T10:08:21.318583+0000","last_undegraded":"2024-04-25T10:08:21.318583+0000","last_fullsized":"2024-04-25T10:08:21.318583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:46.279014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979915+0000","last_change":"2024-04-25T10:08:19.982195+0000","last_active":"2024-04-25T10:08:20.979915+0000","last_peered":"2024-04-25T10:08:20.979915+0000","last_clean":"2024-04-25T10:08:20.979915+0000","last_became_active":"2024-04-25T10:08:19.981826+0000","last_became_peered":"2024-04-25T10:08:19.981826+0000","last_unstale":"2024-04-25T10:08:20.979915+0000","last_undegraded":"2024-04-25T10:08:20.979915+0000","last_fullsized":"2024-04-25T10:08:20.979915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.912266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317912+0000","last_change":"2024-04-25T10:08:18.941173+0000","last_active":"2024-04-25T10:08:21.317912+0000","last_peered":"2024-04-25T10:08:21.317912+0000","last_clean":"2024-04-25T10:08:21.317912+0000","last_became_active":"2024-04-25T10:08:18.940897+0000","last_became_peered":"2024-04-25T10:08:18.940897+0000","last_unstale":"2024-04-25T10:08:21.317912+0000","last_undegraded":"2024-04-25T10:08:21.317912+0000","last_fullsized":"2024-04-25T10:08:21.317912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:41.750187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964904+0000","last_change":"2024-04-25T10:08:19.971882+0000","last_active":"2024-04-25T10:08:20.964904+0000","last_peered":"2024-04-25T10:08:20.964904+0000","last_clean":"2024-04-25T10:08:20.964904+0000","last_became_active":"2024-04-25T10:08:19.971617+0000","last_became_peered":"2024-04-25T10:08:19.971617+0000","last_unstale":"2024-04-25T10:08:20.964904+0000","last_undegraded":"2024-04-25T10:08:20.964904+0000","last_fullsized":"2024-04-25T10:08:20.964904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964888+0000","last_change":"2024-04-25T10:08:18.935168+0000","last_active":"2024-04-25T10:08:20.964888+0000","last_peered":"2024-04-25T10:08:20.964888+0000","last_clean":"2024-04-25T10:08:20.964888+0000","last_became_active":"2024-04-25T10:08:18.931137+0000","last_became_peered":"2024-04-25T10:08:18.931137+0000","last_unstale":"2024-04-25T10:08:20.964888+0000","last_undegraded":"2024-04-25T10:08:20.964888+0000","last_fullsized":"2024-04-25T10:08:20.964888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:15.685967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964034+0000","last_change":"2024-04-25T10:08:19.969101+0000","last_active":"2024-04-25T10:08:20.964034+0000","last_peered":"2024-04-25T10:08:20.964034+0000","last_clean":"2024-04-25T10:08:20.964034+0000","last_became_active":"2024-04-25T10:08:19.968484+0000","last_became_peered":"2024-04-25T10:08:19.968484+0000","last_unstale":"2024-04-25T10:08:20.964034+0000","last_undegraded":"2024-04-25T10:08:20.964034+0000","last_fullsized":"2024-04-25T10:08:20.964034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:00.235689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985538+0000","last_change":"2024-04-25T10:08:18.947971+0000","last_active":"2024-04-25T10:08:20.985538+0000","last_peered":"2024-04-25T10:08:20.985538+0000","last_clean":"2024-04-25T10:08:20.985538+0000","last_became_active":"2024-04-25T10:08:18.947544+0000","last_became_peered":"2024-04-25T10:08:18.947544+0000","last_unstale":"2024-04-25T10:08:20.985538+0000","last_undegraded":"2024-04-25T10:08:20.985538+0000","last_fullsized":"2024-04-25T10:08:20.985538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:44.428058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962039+0000","last_change":"2024-04-25T10:08:19.987566+0000","last_active":"2024-04-25T10:08:20.962039+0000","last_peered":"2024-04-25T10:08:20.962039+0000","last_clean":"2024-04-25T10:08:20.962039+0000","last_became_active":"2024-04-25T10:08:19.987263+0000","last_became_peered":"2024-04-25T10:08:19.987263+0000","last_unstale":"2024-04-25T10:08:20.962039+0000","last_undegraded":"2024-04-25T10:08:20.962039+0000","last_fullsized":"2024-04-25T10:08:20.962039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:16.427011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965246+0000","last_change":"2024-04-25T10:08:18.933657+0000","last_active":"2024-04-25T10:08:20.965246+0000","last_peered":"2024-04-25T10:08:20.965246+0000","last_clean":"2024-04-25T10:08:20.965246+0000","last_became_active":"2024-04-25T10:08:18.932270+0000","last_became_peered":"2024-04-25T10:08:18.932270+0000","last_unstale":"2024-04-25T10:08:20.965246+0000","last_undegraded":"2024-04-25T10:08:20.965246+0000","last_fullsized":"2024-04-25T10:08:20.965246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:05.325048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962407+0000","last_change":"2024-04-25T10:08:19.964445+0000","last_active":"2024-04-25T10:08:20.962407+0000","last_peered":"2024-04-25T10:08:20.962407+0000","last_clean":"2024-04-25T10:08:20.962407+0000","last_became_active":"2024-04-25T10:08:19.964176+0000","last_became_peered":"2024-04-25T10:08:19.964176+0000","last_unstale":"2024-04-25T10:08:20.962407+0000","last_undegraded":"2024-04-25T10:08:20.962407+0000","last_fullsized":"2024-04-25T10:08:20.962407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:30.542619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961382+0000","last_change":"2024-04-25T10:08:18.941880+0000","last_active":"2024-04-25T10:08:20.961382+0000","last_peered":"2024-04-25T10:08:20.961382+0000","last_clean":"2024-04-25T10:08:20.961382+0000","last_became_active":"2024-04-25T10:08:18.941576+0000","last_became_peered":"2024-04-25T10:08:18.941576+0000","last_unstale":"2024-04-25T10:08:20.961382+0000","last_undegraded":"2024-04-25T10:08:20.961382+0000","last_fullsized":"2024-04-25T10:08:20.961382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:30.807252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318781+0000","last_change":"2024-04-25T10:08:19.967200+0000","last_active":"2024-04-25T10:08:21.318781+0000","last_peered":"2024-04-25T10:08:21.318781+0000","last_clean":"2024-04-25T10:08:21.318781+0000","last_became_active":"2024-04-25T10:08:19.966933+0000","last_became_peered":"2024-04-25T10:08:19.966933+0000","last_unstale":"2024-04-25T10:08:21.318781+0000","last_undegraded":"2024-04-25T10:08:21.318781+0000","last_fullsized":"2024-04-25T10:08:21.318781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.430130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316398+0000","last_change":"2024-04-25T10:08:18.935935+0000","last_active":"2024-04-25T10:08:21.316398+0000","last_peered":"2024-04-25T10:08:21.316398+0000","last_clean":"2024-04-25T10:08:21.316398+0000","last_became_active":"2024-04-25T10:08:18.932340+0000","last_became_peered":"2024-04-25T10:08:18.932340+0000","last_unstale":"2024-04-25T10:08:21.316398+0000","last_undegraded":"2024-04-25T10:08:21.316398+0000","last_fullsized":"2024-04-25T10:08:21.316398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:46.175875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":292,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317217+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:08:21.317217+0000","last_peered":"2024-04-25T10:08:21.317217+0000","last_clean":"2024-04-25T10:08:21.317217+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:08:21.317217+0000","last_undegraded":"2024-04-25T10:08:21.317217+0000","last_fullsized":"2024-04-25T10:08:21.317217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41104,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330736,"statfs":{"total":96636764160,"available":96594673664,"internally_reserved":0,"allocated":1638400,"data_stored":1072558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40528,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331312,"statfs":{"total":96636764160,"available":96595263488,"internally_reserved":0,"allocated":1052672,"data_stored":483780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1044480,"data_stored":482202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40544,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331296,"statfs":{"total":96636764160,"available":96595247104,"internally_reserved":0,"allocated":1060864,"data_stored":483884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40520,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331320,"statfs":{"total":96636764160,"available":96595271680,"internally_reserved":0,"allocated":1044480,"data_stored":482750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40536,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331304,"statfs":{"total":96636764160,"available":96595255296,"internally_reserved":0,"allocated":1052672,"data_stored":482910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":482280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":1634304,"data_stored":1072536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:24.313 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:08:24.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.540+0000 7f185ac00640 1 Processor -- start 2024-04-25T10:08:24.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.541+0000 7f185ac00640 1 -- start start 2024-04-25T10:08:24.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.542+0000 7f185ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f185415e520 0x7f185415e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.542+0000 7f185ac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1854070ff0 0x7f1854071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.542+0000 7f185ac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1854071950 con 0x7f185415f040 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.542+0000 7f185ac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18540aee30 con 0x7f1854070ff0 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.542+0000 7f185ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18540ae590 con 0x7f185415e520 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1854070ff0 0x7f1854071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1854070ff0 0x7f1854071410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41280/0 (socket says 172.21.15.165:41280) 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 -- 172.21.15.165:0/1205223343 learned_addr learned my addr 172.21.15.165:0/1205223343 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f1859800640 1 --2- 172.21.15.165:0/1205223343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f185415e520 0x7f185415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 -- 172.21.15.165:0/1205223343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f185415e520 msgr2=0x7f185415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 --2- 172.21.15.165:0/1205223343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f185415e520 0x7f185415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 -- 172.21.15.165:0/1205223343 >> v1:172.21.15.172:6789/0 conn(0x7f185415f040 legacy=0x7f18540adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:24.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.543+0000 7f185a200640 1 -- 172.21.15.165:0/1205223343 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18540ae870 con 0x7f1854070ff0 2024-04-25T10:08:24.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.544+0000 7f185a200640 1 --2- 172.21.15.165:0/1205223343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854070ff0 0x7f1854071410 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f184c009e50 tx=0x7f184c02a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a2a192c3b0d7944 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f1853e00640 1 -- 172.21.15.165:0/1205223343 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c036070 con 0x7f1854070ff0 2024-04-25T10:08:24.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f1853e00640 1 -- 172.21.15.165:0/1205223343 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f184c031600 con 0x7f1854070ff0 2024-04-25T10:08:24.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f1853e00640 1 -- 172.21.15.165:0/1205223343 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f184c0319d0 con 0x7f1854070ff0 2024-04-25T10:08:24.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f185ac00640 1 -- 172.21.15.165:0/1205223343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854070ff0 msgr2=0x7f1854071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f185ac00640 1 --2- 172.21.15.165:0/1205223343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854070ff0 0x7f1854071410 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f184c009e50 tx=0x7f184c02a6c0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f185ac00640 1 -- 172.21.15.165:0/1205223343 shutdown_connections 2024-04-25T10:08:24.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f185ac00640 1 --2- 172.21.15.165:0/1205223343 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1854070ff0 0x7f1854071410 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f185ac00640 1 --2- 172.21.15.165:0/1205223343 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f185415e520 0x7f185415e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.545+0000 7f185ac00640 1 -- 172.21.15.165:0/1205223343 >> 172.21.15.165:0/1205223343 conn(0x7f185406abe0 msgr2=0x7f185406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.546+0000 7f185ac00640 1 -- 172.21.15.165:0/1205223343 shutdown_connections 2024-04-25T10:08:24.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.546+0000 7f185ac00640 1 -- 172.21.15.165:0/1205223343 wait complete. 2024-04-25T10:08:24.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.547+0000 7f185ac00640 1 Processor -- start 2024-04-25T10:08:24.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.547+0000 7f185ac00640 1 -- start start 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f185ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1854070ff0 0x7f18540cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f185ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 0x7f18540b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f185ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f185415f040 0x7f1854156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f185ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18540ae2a0 con 0x7f185415e520 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f185ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185406d1f0 con 0x7f185415f040 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f185ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185415f4a0 con 0x7f1854070ff0 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.548+0000 7f1858e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 0x7f18540b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 0x7f18540b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41292/0 (socket says 172.21.15.165:41292) 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 -- 172.21.15.165:0/199354142 learned_addr learned my addr 172.21.15.165:0/199354142 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 -- 172.21.15.165:0/199354142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1854070ff0 msgr2=0x7f18540cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1854070ff0 0x7f18540cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 -- 172.21.15.165:0/199354142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f185415f040 msgr2=0x7f1854156c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f185415f040 0x7f1854156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 -- 172.21.15.165:0/199354142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1854159950 con 0x7f185415e520 2024-04-25T10:08:24.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1858e00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 0x7f18540b1d20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f184000b3f0 tx=0x7f184000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.549+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840015020 con 0x7f185415e520 2024-04-25T10:08:24.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.550+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1840011830 con 0x7f185415e520 2024-04-25T10:08:24.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.550+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1840011c20 con 0x7f185415e520 2024-04-25T10:08:24.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.550+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1854159c30 con 0x7f185415e520 2024-04-25T10:08:24.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.550+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185415a4d0 con 0x7f185415e520 2024-04-25T10:08:24.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.551+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1854159e00 con 0x7f185415e520 2024-04-25T10:08:24.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.552+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1840018020 con 0x7f185415e520 2024-04-25T10:08:24.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.553+0000 7f1851600640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f183406b500 0x7f183406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:24.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.553+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f184008a0f0 con 0x7f185415e520 2024-04-25T10:08:24.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.553+0000 7f1859800640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f183406b500 0x7f183406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:24.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.554+0000 7f1859800640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f183406b500 0x7f183406d9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f18480098f0 tx=0x7f1848008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:24.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.557+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f184001f030 con 0x7f185415e520 2024-04-25T10:08:24.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.747+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1854073990 con 0x7f183406b500 2024-04-25T10:08:24.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.756+0000 7f1851600640 1 -- 172.21.15.165:0/199354142 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252082 (secure 0 0 0) 0x7f1854073990 con 0x7f183406b500 2024-04-25T10:08:24.765 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:24.766 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:08:24.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.760+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f183406b500 msgr2=0x7f183406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.760+0000 7f185ac00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f183406b500 0x7f183406d9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f18480098f0 tx=0x7f1848008040 comp rx=0 tx=0).stop 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.760+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 msgr2=0x7f18540b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.760+0000 7f185ac00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 0x7f18540b1d20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f184000b3f0 tx=0x7f184000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 shutdown_connections 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f183406b500 0x7f183406d9c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f185415f040 0x7f1854156c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f185415e520 0x7f18540b1d20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 --2- 172.21.15.165:0/199354142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1854070ff0 0x7f18540cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 >> 172.21.15.165:0/199354142 conn(0x7f185406abe0 msgr2=0x7f18541e92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 shutdown_connections 2024-04-25T10:08:24.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:24.761+0000 7f185ac00640 1 -- 172.21.15.165:0/199354142 wait complete. 2024-04-25T10:08:24.786 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":274,"stamp":"2024-04-25T10:08:24.255381+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":120,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322628,"kb_used_data":9352,"kb_used_omap":12,"kb_used_meta":313139,"kb_avail":754652092,"statfs":{"total":773094113280,"available":772763742208,"internally_reserved":0,"allocated":9576448,"data_stored":5042900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":320654898},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001814"},"pg_stats":[{"pgid":"12.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318487+0000","last_change":"2024-04-25T10:08:19.967461+0000","last_active":"2024-04-25T10:08:21.318487+0000","last_peered":"2024-04-25T10:08:21.318487+0000","last_clean":"2024-04-25T10:08:21.318487+0000","last_became_active":"2024-04-25T10:08:19.966573+0000","last_became_peered":"2024-04-25T10:08:19.966573+0000","last_unstale":"2024-04-25T10:08:21.318487+0000","last_undegraded":"2024-04-25T10:08:21.318487+0000","last_fullsized":"2024-04-25T10:08:21.318487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:52.728570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980483+0000","last_change":"2024-04-25T10:08:19.986679+0000","last_active":"2024-04-25T10:08:20.980483+0000","last_peered":"2024-04-25T10:08:20.980483+0000","last_clean":"2024-04-25T10:08:20.980483+0000","last_became_active":"2024-04-25T10:08:19.983909+0000","last_became_peered":"2024-04-25T10:08:19.983909+0000","last_unstale":"2024-04-25T10:08:20.980483+0000","last_undegraded":"2024-04-25T10:08:20.980483+0000","last_fullsized":"2024-04-25T10:08:20.980483+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:37.487486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962369+0000","last_change":"2024-04-25T10:08:19.991185+0000","last_active":"2024-04-25T10:08:20.962369+0000","last_peered":"2024-04-25T10:08:20.962369+0000","last_clean":"2024-04-25T10:08:20.962369+0000","last_became_active":"2024-04-25T10:08:19.990443+0000","last_became_peered":"2024-04-25T10:08:19.990443+0000","last_unstale":"2024-04-25T10:08:20.962369+0000","last_undegraded":"2024-04-25T10:08:20.962369+0000","last_fullsized":"2024-04-25T10:08:20.962369+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.544140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316691+0000","last_change":"2024-04-25T10:08:19.977547+0000","last_active":"2024-04-25T10:08:21.316691+0000","last_peered":"2024-04-25T10:08:21.316691+0000","last_clean":"2024-04-25T10:08:21.316691+0000","last_became_active":"2024-04-25T10:08:19.977019+0000","last_became_peered":"2024-04-25T10:08:19.977019+0000","last_unstale":"2024-04-25T10:08:21.316691+0000","last_undegraded":"2024-04-25T10:08:21.316691+0000","last_fullsized":"2024-04-25T10:08:21.316691+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:20.007797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985397+0000","last_change":"2024-04-25T10:08:19.960239+0000","last_active":"2024-04-25T10:08:20.985397+0000","last_peered":"2024-04-25T10:08:20.985397+0000","last_clean":"2024-04-25T10:08:20.985397+0000","last_became_active":"2024-04-25T10:08:19.959593+0000","last_became_peered":"2024-04-25T10:08:19.959593+0000","last_unstale":"2024-04-25T10:08:20.985397+0000","last_undegraded":"2024-04-25T10:08:20.985397+0000","last_fullsized":"2024-04-25T10:08:20.985397+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:15.901581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315952+0000","last_change":"2024-04-25T10:08:19.988084+0000","last_active":"2024-04-25T10:08:21.315952+0000","last_peered":"2024-04-25T10:08:21.315952+0000","last_clean":"2024-04-25T10:08:21.315952+0000","last_became_active":"2024-04-25T10:08:19.981592+0000","last_became_peered":"2024-04-25T10:08:19.981592+0000","last_unstale":"2024-04-25T10:08:21.315952+0000","last_undegraded":"2024-04-25T10:08:21.315952+0000","last_fullsized":"2024-04-25T10:08:21.315952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:55.773976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964566+0000","last_change":"2024-04-25T10:08:19.969696+0000","last_active":"2024-04-25T10:08:20.964566+0000","last_peered":"2024-04-25T10:08:20.964566+0000","last_clean":"2024-04-25T10:08:20.964566+0000","last_became_active":"2024-04-25T10:08:19.969406+0000","last_became_peered":"2024-04-25T10:08:19.969406+0000","last_unstale":"2024-04-25T10:08:20.964566+0000","last_undegraded":"2024-04-25T10:08:20.964566+0000","last_fullsized":"2024-04-25T10:08:20.964566+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:21.913629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985282+0000","last_change":"2024-04-25T10:08:19.961945+0000","last_active":"2024-04-25T10:08:20.985282+0000","last_peered":"2024-04-25T10:08:20.985282+0000","last_clean":"2024-04-25T10:08:20.985282+0000","last_became_active":"2024-04-25T10:08:19.961594+0000","last_became_peered":"2024-04-25T10:08:19.961594+0000","last_unstale":"2024-04-25T10:08:20.985282+0000","last_undegraded":"2024-04-25T10:08:20.985282+0000","last_fullsized":"2024-04-25T10:08:20.985282+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:12.247035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967408+0000","last_change":"2024-04-25T10:08:19.977705+0000","last_active":"2024-04-25T10:08:20.967408+0000","last_peered":"2024-04-25T10:08:20.967408+0000","last_clean":"2024-04-25T10:08:20.967408+0000","last_became_active":"2024-04-25T10:08:19.977402+0000","last_became_peered":"2024-04-25T10:08:19.977402+0000","last_unstale":"2024-04-25T10:08:20.967408+0000","last_undegraded":"2024-04-25T10:08:20.967408+0000","last_fullsized":"2024-04-25T10:08:20.967408+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:41.745994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316909+0000","last_change":"2024-04-25T10:08:19.966171+0000","last_active":"2024-04-25T10:08:21.316909+0000","last_peered":"2024-04-25T10:08:21.316909+0000","last_clean":"2024-04-25T10:08:21.316909+0000","last_became_active":"2024-04-25T10:08:19.965818+0000","last_became_peered":"2024-04-25T10:08:19.965818+0000","last_unstale":"2024-04-25T10:08:21.316909+0000","last_undegraded":"2024-04-25T10:08:21.316909+0000","last_fullsized":"2024-04-25T10:08:21.316909+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:58.749731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980057+0000","last_change":"2024-04-25T10:08:19.983504+0000","last_active":"2024-04-25T10:08:20.980057+0000","last_peered":"2024-04-25T10:08:20.980057+0000","last_clean":"2024-04-25T10:08:20.980057+0000","last_became_active":"2024-04-25T10:08:19.983043+0000","last_became_peered":"2024-04-25T10:08:19.983043+0000","last_unstale":"2024-04-25T10:08:20.980057+0000","last_undegraded":"2024-04-25T10:08:20.980057+0000","last_fullsized":"2024-04-25T10:08:20.980057+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:18.293222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317782+0000","last_change":"2024-04-25T10:08:19.982135+0000","last_active":"2024-04-25T10:08:21.317782+0000","last_peered":"2024-04-25T10:08:21.317782+0000","last_clean":"2024-04-25T10:08:21.317782+0000","last_became_active":"2024-04-25T10:08:19.981838+0000","last_became_peered":"2024-04-25T10:08:19.981838+0000","last_unstale":"2024-04-25T10:08:21.317782+0000","last_undegraded":"2024-04-25T10:08:21.317782+0000","last_fullsized":"2024-04-25T10:08:21.317782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:07.697822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985662+0000","last_change":"2024-04-25T10:08:19.976777+0000","last_active":"2024-04-25T10:08:20.985662+0000","last_peered":"2024-04-25T10:08:20.985662+0000","last_clean":"2024-04-25T10:08:20.985662+0000","last_became_active":"2024-04-25T10:08:19.976340+0000","last_became_peered":"2024-04-25T10:08:19.976340+0000","last_unstale":"2024-04-25T10:08:20.985662+0000","last_undegraded":"2024-04-25T10:08:20.985662+0000","last_fullsized":"2024-04-25T10:08:20.985662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:28.581297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316149+0000","last_change":"2024-04-25T10:08:19.976990+0000","last_active":"2024-04-25T10:08:21.316149+0000","last_peered":"2024-04-25T10:08:21.316149+0000","last_clean":"2024-04-25T10:08:21.316149+0000","last_became_active":"2024-04-25T10:08:19.975772+0000","last_became_peered":"2024-04-25T10:08:19.975772+0000","last_unstale":"2024-04-25T10:08:21.316149+0000","last_undegraded":"2024-04-25T10:08:21.316149+0000","last_fullsized":"2024-04-25T10:08:21.316149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:25.962057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978689+0000","last_change":"2024-04-25T10:08:19.969262+0000","last_active":"2024-04-25T10:08:20.978689+0000","last_peered":"2024-04-25T10:08:20.978689+0000","last_clean":"2024-04-25T10:08:20.978689+0000","last_became_active":"2024-04-25T10:08:19.969006+0000","last_became_peered":"2024-04-25T10:08:19.969006+0000","last_unstale":"2024-04-25T10:08:20.978689+0000","last_undegraded":"2024-04-25T10:08:20.978689+0000","last_fullsized":"2024-04-25T10:08:20.978689+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.430130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317072+0000","last_change":"2024-04-25T10:08:19.962724+0000","last_active":"2024-04-25T10:08:21.317072+0000","last_peered":"2024-04-25T10:08:21.317072+0000","last_clean":"2024-04-25T10:08:21.317072+0000","last_became_active":"2024-04-25T10:08:19.960695+0000","last_became_peered":"2024-04-25T10:08:19.960695+0000","last_unstale":"2024-04-25T10:08:21.317072+0000","last_undegraded":"2024-04-25T10:08:21.317072+0000","last_fullsized":"2024-04-25T10:08:21.317072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:48.882289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986277+0000","last_change":"2024-04-25T10:08:19.957209+0000","last_active":"2024-04-25T10:08:20.986277+0000","last_peered":"2024-04-25T10:08:20.986277+0000","last_clean":"2024-04-25T10:08:20.986277+0000","last_became_active":"2024-04-25T10:08:19.956891+0000","last_became_peered":"2024-04-25T10:08:19.956891+0000","last_unstale":"2024-04-25T10:08:20.986277+0000","last_undegraded":"2024-04-25T10:08:20.986277+0000","last_fullsized":"2024-04-25T10:08:20.986277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:12.885485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961788+0000","last_change":"2024-04-25T10:08:19.979617+0000","last_active":"2024-04-25T10:08:20.961788+0000","last_peered":"2024-04-25T10:08:20.961788+0000","last_clean":"2024-04-25T10:08:20.961788+0000","last_became_active":"2024-04-25T10:08:19.979324+0000","last_became_peered":"2024-04-25T10:08:19.979324+0000","last_unstale":"2024-04-25T10:08:20.961788+0000","last_undegraded":"2024-04-25T10:08:20.961788+0000","last_fullsized":"2024-04-25T10:08:20.961788+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:14.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316220+0000","last_change":"2024-04-25T10:08:19.979309+0000","last_active":"2024-04-25T10:08:21.316220+0000","last_peered":"2024-04-25T10:08:21.316220+0000","last_clean":"2024-04-25T10:08:21.316220+0000","last_became_active":"2024-04-25T10:08:19.978859+0000","last_became_peered":"2024-04-25T10:08:19.978859+0000","last_unstale":"2024-04-25T10:08:21.316220+0000","last_undegraded":"2024-04-25T10:08:21.316220+0000","last_fullsized":"2024-04-25T10:08:21.316220+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.912266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985686+0000","last_change":"2024-04-25T10:08:19.967430+0000","last_active":"2024-04-25T10:08:20.985686+0000","last_peered":"2024-04-25T10:08:20.985686+0000","last_clean":"2024-04-25T10:08:20.985686+0000","last_became_active":"2024-04-25T10:08:19.967185+0000","last_became_peered":"2024-04-25T10:08:19.967185+0000","last_unstale":"2024-04-25T10:08:20.985686+0000","last_undegraded":"2024-04-25T10:08:20.985686+0000","last_fullsized":"2024-04-25T10:08:20.985686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:40.041741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966697+0000","last_change":"2024-04-25T10:08:19.973682+0000","last_active":"2024-04-25T10:08:20.966697+0000","last_peered":"2024-04-25T10:08:20.966697+0000","last_clean":"2024-04-25T10:08:20.966697+0000","last_became_active":"2024-04-25T10:08:19.972892+0000","last_became_peered":"2024-04-25T10:08:19.972892+0000","last_unstale":"2024-04-25T10:08:20.966697+0000","last_undegraded":"2024-04-25T10:08:20.966697+0000","last_fullsized":"2024-04-25T10:08:20.966697+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:31.822103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985412+0000","last_change":"2024-04-25T10:08:19.973046+0000","last_active":"2024-04-25T10:08:20.985412+0000","last_peered":"2024-04-25T10:08:20.985412+0000","last_clean":"2024-04-25T10:08:20.985412+0000","last_became_active":"2024-04-25T10:08:19.972730+0000","last_became_peered":"2024-04-25T10:08:19.972730+0000","last_unstale":"2024-04-25T10:08:20.985412+0000","last_undegraded":"2024-04-25T10:08:20.985412+0000","last_fullsized":"2024-04-25T10:08:20.985412+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:50:52.630445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966159+0000","last_change":"2024-04-25T10:08:19.975999+0000","last_active":"2024-04-25T10:08:20.966159+0000","last_peered":"2024-04-25T10:08:20.966159+0000","last_clean":"2024-04-25T10:08:20.966159+0000","last_became_active":"2024-04-25T10:08:19.975638+0000","last_became_peered":"2024-04-25T10:08:19.975638+0000","last_unstale":"2024-04-25T10:08:20.966159+0000","last_undegraded":"2024-04-25T10:08:20.966159+0000","last_fullsized":"2024-04-25T10:08:20.966159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:48:17.059256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978614+0000","last_change":"2024-04-25T10:08:19.974606+0000","last_active":"2024-04-25T10:08:20.978614+0000","last_peered":"2024-04-25T10:08:20.978614+0000","last_clean":"2024-04-25T10:08:20.978614+0000","last_became_active":"2024-04-25T10:08:19.974336+0000","last_became_peered":"2024-04-25T10:08:19.974336+0000","last_unstale":"2024-04-25T10:08:20.978614+0000","last_undegraded":"2024-04-25T10:08:20.978614+0000","last_fullsized":"2024-04-25T10:08:20.978614+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.910865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980532+0000","last_change":"2024-04-25T10:08:19.982998+0000","last_active":"2024-04-25T10:08:20.980532+0000","last_peered":"2024-04-25T10:08:20.980532+0000","last_clean":"2024-04-25T10:08:20.980532+0000","last_became_active":"2024-04-25T10:08:19.982698+0000","last_became_peered":"2024-04-25T10:08:19.982698+0000","last_unstale":"2024-04-25T10:08:20.980532+0000","last_undegraded":"2024-04-25T10:08:20.980532+0000","last_fullsized":"2024-04-25T10:08:20.980532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:23:36.340349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317360+0000","last_change":"2024-04-25T10:08:19.979566+0000","last_active":"2024-04-25T10:08:21.317360+0000","last_peered":"2024-04-25T10:08:21.317360+0000","last_clean":"2024-04-25T10:08:21.317360+0000","last_became_active":"2024-04-25T10:08:19.979276+0000","last_became_peered":"2024-04-25T10:08:19.979276+0000","last_unstale":"2024-04-25T10:08:21.317360+0000","last_undegraded":"2024-04-25T10:08:21.317360+0000","last_fullsized":"2024-04-25T10:08:21.317360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:40.887734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318210+0000","last_change":"2024-04-25T10:08:19.964116+0000","last_active":"2024-04-25T10:08:21.318210+0000","last_peered":"2024-04-25T10:08:21.318210+0000","last_clean":"2024-04-25T10:08:21.318210+0000","last_became_active":"2024-04-25T10:08:19.960959+0000","last_became_peered":"2024-04-25T10:08:19.960959+0000","last_unstale":"2024-04-25T10:08:21.318210+0000","last_undegraded":"2024-04-25T10:08:21.318210+0000","last_fullsized":"2024-04-25T10:08:21.318210+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:23.910865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985944+0000","last_change":"2024-04-25T10:08:19.974107+0000","last_active":"2024-04-25T10:08:20.985944+0000","last_peered":"2024-04-25T10:08:20.985944+0000","last_clean":"2024-04-25T10:08:20.985944+0000","last_became_active":"2024-04-25T10:08:19.973726+0000","last_became_peered":"2024-04-25T10:08:19.973726+0000","last_unstale":"2024-04-25T10:08:20.985944+0000","last_undegraded":"2024-04-25T10:08:20.985944+0000","last_fullsized":"2024-04-25T10:08:20.985944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:05:45.655703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979726+0000","last_change":"2024-04-25T10:08:19.991517+0000","last_active":"2024-04-25T10:08:20.979726+0000","last_peered":"2024-04-25T10:08:20.979726+0000","last_clean":"2024-04-25T10:08:20.979726+0000","last_became_active":"2024-04-25T10:08:19.991209+0000","last_became_peered":"2024-04-25T10:08:19.991209+0000","last_unstale":"2024-04-25T10:08:20.979726+0000","last_undegraded":"2024-04-25T10:08:20.979726+0000","last_fullsized":"2024-04-25T10:08:20.979726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:05.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316911+0000","last_change":"2024-04-25T10:08:19.963461+0000","last_active":"2024-04-25T10:08:21.316911+0000","last_peered":"2024-04-25T10:08:21.316911+0000","last_clean":"2024-04-25T10:08:21.316911+0000","last_became_active":"2024-04-25T10:08:19.963127+0000","last_became_peered":"2024-04-25T10:08:19.963127+0000","last_unstale":"2024-04-25T10:08:21.316911+0000","last_undegraded":"2024-04-25T10:08:21.316911+0000","last_fullsized":"2024-04-25T10:08:21.316911+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:43.269695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961619+0000","last_change":"2024-04-25T10:08:19.962954+0000","last_active":"2024-04-25T10:08:20.961619+0000","last_peered":"2024-04-25T10:08:20.961619+0000","last_clean":"2024-04-25T10:08:20.961619+0000","last_became_active":"2024-04-25T10:08:19.962644+0000","last_became_peered":"2024-04-25T10:08:19.962644+0000","last_unstale":"2024-04-25T10:08:20.961619+0000","last_undegraded":"2024-04-25T10:08:20.961619+0000","last_fullsized":"2024-04-25T10:08:20.961619+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317031+0000","last_change":"2024-04-25T10:08:19.967639+0000","last_active":"2024-04-25T10:08:21.317031+0000","last_peered":"2024-04-25T10:08:21.317031+0000","last_clean":"2024-04-25T10:08:21.317031+0000","last_became_active":"2024-04-25T10:08:19.967335+0000","last_became_peered":"2024-04-25T10:08:19.967335+0000","last_unstale":"2024-04-25T10:08:21.317031+0000","last_undegraded":"2024-04-25T10:08:21.317031+0000","last_fullsized":"2024-04-25T10:08:21.317031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:41.088941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961618+0000","last_change":"2024-04-25T10:08:19.981351+0000","last_active":"2024-04-25T10:08:20.961618+0000","last_peered":"2024-04-25T10:08:20.961618+0000","last_clean":"2024-04-25T10:08:20.961618+0000","last_became_active":"2024-04-25T10:08:19.979781+0000","last_became_peered":"2024-04-25T10:08:19.979781+0000","last_unstale":"2024-04-25T10:08:20.961618+0000","last_undegraded":"2024-04-25T10:08:20.961618+0000","last_fullsized":"2024-04-25T10:08:20.961618+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:20.725052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986262+0000","last_change":"2024-04-25T10:08:18.942173+0000","last_active":"2024-04-25T10:08:20.986262+0000","last_peered":"2024-04-25T10:08:20.986262+0000","last_clean":"2024-04-25T10:08:20.986262+0000","last_became_active":"2024-04-25T10:08:18.941884+0000","last_became_peered":"2024-04-25T10:08:18.941884+0000","last_unstale":"2024-04-25T10:08:20.986262+0000","last_undegraded":"2024-04-25T10:08:20.986262+0000","last_fullsized":"2024-04-25T10:08:20.986262+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:21:55.696101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978759+0000","last_change":"2024-04-25T10:08:19.976689+0000","last_active":"2024-04-25T10:08:20.978759+0000","last_peered":"2024-04-25T10:08:20.978759+0000","last_clean":"2024-04-25T10:08:20.978759+0000","last_became_active":"2024-04-25T10:08:19.975469+0000","last_became_peered":"2024-04-25T10:08:19.975469+0000","last_unstale":"2024-04-25T10:08:20.978759+0000","last_undegraded":"2024-04-25T10:08:20.978759+0000","last_fullsized":"2024-04-25T10:08:20.978759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:35.566767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318454+0000","last_change":"2024-04-25T10:08:18.942821+0000","last_active":"2024-04-25T10:08:21.318454+0000","last_peered":"2024-04-25T10:08:21.318454+0000","last_clean":"2024-04-25T10:08:21.318454+0000","last_became_active":"2024-04-25T10:08:18.942482+0000","last_became_peered":"2024-04-25T10:08:18.942482+0000","last_unstale":"2024-04-25T10:08:21.318454+0000","last_undegraded":"2024-04-25T10:08:21.318454+0000","last_fullsized":"2024-04-25T10:08:21.318454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:04.593211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966914+0000","last_change":"2024-04-25T10:08:19.978470+0000","last_active":"2024-04-25T10:08:20.966914+0000","last_peered":"2024-04-25T10:08:20.966914+0000","last_clean":"2024-04-25T10:08:20.966914+0000","last_became_active":"2024-04-25T10:08:19.978112+0000","last_became_peered":"2024-04-25T10:08:19.978112+0000","last_unstale":"2024-04-25T10:08:20.966914+0000","last_undegraded":"2024-04-25T10:08:20.966914+0000","last_fullsized":"2024-04-25T10:08:20.966914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:27.851042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318313+0000","last_change":"2024-04-25T10:08:18.940512+0000","last_active":"2024-04-25T10:08:21.318313+0000","last_peered":"2024-04-25T10:08:21.318313+0000","last_clean":"2024-04-25T10:08:21.318313+0000","last_became_active":"2024-04-25T10:08:18.940191+0000","last_became_peered":"2024-04-25T10:08:18.940191+0000","last_unstale":"2024-04-25T10:08:21.318313+0000","last_undegraded":"2024-04-25T10:08:21.318313+0000","last_fullsized":"2024-04-25T10:08:21.318313+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:06.682971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966725+0000","last_change":"2024-04-25T10:08:19.968676+0000","last_active":"2024-04-25T10:08:20.966725+0000","last_peered":"2024-04-25T10:08:20.966725+0000","last_clean":"2024-04-25T10:08:20.966725+0000","last_became_active":"2024-04-25T10:08:19.968356+0000","last_became_peered":"2024-04-25T10:08:19.968356+0000","last_unstale":"2024-04-25T10:08:20.966725+0000","last_undegraded":"2024-04-25T10:08:20.966725+0000","last_fullsized":"2024-04-25T10:08:20.966725+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:41.660213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965231+0000","last_change":"2024-04-25T10:08:18.936196+0000","last_active":"2024-04-25T10:08:20.965231+0000","last_peered":"2024-04-25T10:08:20.965231+0000","last_clean":"2024-04-25T10:08:20.965231+0000","last_became_active":"2024-04-25T10:08:18.935774+0000","last_became_peered":"2024-04-25T10:08:18.935774+0000","last_unstale":"2024-04-25T10:08:20.965231+0000","last_undegraded":"2024-04-25T10:08:20.965231+0000","last_fullsized":"2024-04-25T10:08:20.965231+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:21:30.556478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979466+0000","last_change":"2024-04-25T10:08:19.980061+0000","last_active":"2024-04-25T10:08:20.979466+0000","last_peered":"2024-04-25T10:08:20.979466+0000","last_clean":"2024-04-25T10:08:20.979466+0000","last_became_active":"2024-04-25T10:08:19.979701+0000","last_became_peered":"2024-04-25T10:08:19.979701+0000","last_unstale":"2024-04-25T10:08:20.979466+0000","last_undegraded":"2024-04-25T10:08:20.979466+0000","last_fullsized":"2024-04-25T10:08:20.979466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:55.773976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966991+0000","last_change":"2024-04-25T10:08:18.951337+0000","last_active":"2024-04-25T10:08:20.966991+0000","last_peered":"2024-04-25T10:08:20.966991+0000","last_clean":"2024-04-25T10:08:20.966991+0000","last_became_active":"2024-04-25T10:08:18.950508+0000","last_became_peered":"2024-04-25T10:08:18.950508+0000","last_unstale":"2024-04-25T10:08:20.966991+0000","last_undegraded":"2024-04-25T10:08:20.966991+0000","last_fullsized":"2024-04-25T10:08:20.966991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:36.306729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980178+0000","last_change":"2024-04-25T10:08:19.982175+0000","last_active":"2024-04-25T10:08:20.980178+0000","last_peered":"2024-04-25T10:08:20.980178+0000","last_clean":"2024-04-25T10:08:20.980178+0000","last_became_active":"2024-04-25T10:08:19.981840+0000","last_became_peered":"2024-04-25T10:08:19.981840+0000","last_unstale":"2024-04-25T10:08:20.980178+0000","last_undegraded":"2024-04-25T10:08:20.980178+0000","last_fullsized":"2024-04-25T10:08:20.980178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:12:14.656634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316325+0000","last_change":"2024-04-25T10:08:18.948996+0000","last_active":"2024-04-25T10:08:21.316325+0000","last_peered":"2024-04-25T10:08:21.316325+0000","last_clean":"2024-04-25T10:08:21.316325+0000","last_became_active":"2024-04-25T10:08:18.948665+0000","last_became_peered":"2024-04-25T10:08:18.948665+0000","last_unstale":"2024-04-25T10:08:21.316325+0000","last_undegraded":"2024-04-25T10:08:21.316325+0000","last_fullsized":"2024-04-25T10:08:21.316325+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:39:20.933525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967749+0000","last_change":"2024-04-25T10:08:19.973442+0000","last_active":"2024-04-25T10:08:20.967749+0000","last_peered":"2024-04-25T10:08:20.967749+0000","last_clean":"2024-04-25T10:08:20.967749+0000","last_became_active":"2024-04-25T10:08:19.973105+0000","last_became_peered":"2024-04-25T10:08:19.973105+0000","last_unstale":"2024-04-25T10:08:20.967749+0000","last_undegraded":"2024-04-25T10:08:20.967749+0000","last_fullsized":"2024-04-25T10:08:20.967749+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:30:55.385237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978968+0000","last_change":"2024-04-25T10:08:18.937233+0000","last_active":"2024-04-25T10:08:20.978968+0000","last_peered":"2024-04-25T10:08:20.978968+0000","last_clean":"2024-04-25T10:08:20.978968+0000","last_became_active":"2024-04-25T10:08:18.936966+0000","last_became_peered":"2024-04-25T10:08:18.936966+0000","last_unstale":"2024-04-25T10:08:20.978968+0000","last_undegraded":"2024-04-25T10:08:20.978968+0000","last_fullsized":"2024-04-25T10:08:20.978968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:06.682971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966135+0000","last_change":"2024-04-25T10:08:19.970670+0000","last_active":"2024-04-25T10:08:20.966135+0000","last_peered":"2024-04-25T10:08:20.966135+0000","last_clean":"2024-04-25T10:08:20.966135+0000","last_became_active":"2024-04-25T10:08:19.970387+0000","last_became_peered":"2024-04-25T10:08:19.970387+0000","last_unstale":"2024-04-25T10:08:20.966135+0000","last_undegraded":"2024-04-25T10:08:20.966135+0000","last_fullsized":"2024-04-25T10:08:20.966135+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.022397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985302+0000","last_change":"2024-04-25T10:08:18.944811+0000","last_active":"2024-04-25T10:08:20.985302+0000","last_peered":"2024-04-25T10:08:20.985302+0000","last_clean":"2024-04-25T10:08:20.985302+0000","last_became_active":"2024-04-25T10:08:18.944531+0000","last_became_peered":"2024-04-25T10:08:18.944531+0000","last_unstale":"2024-04-25T10:08:20.985302+0000","last_undegraded":"2024-04-25T10:08:20.985302+0000","last_fullsized":"2024-04-25T10:08:20.985302+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:16.882336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978621+0000","last_change":"2024-04-25T10:08:19.980510+0000","last_active":"2024-04-25T10:08:20.978621+0000","last_peered":"2024-04-25T10:08:20.978621+0000","last_clean":"2024-04-25T10:08:20.978621+0000","last_became_active":"2024-04-25T10:08:19.979752+0000","last_became_peered":"2024-04-25T10:08:19.979752+0000","last_unstale":"2024-04-25T10:08:20.978621+0000","last_undegraded":"2024-04-25T10:08:20.978621+0000","last_fullsized":"2024-04-25T10:08:20.978621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:27:52.728570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318544+0000","last_change":"2024-04-25T10:08:18.941665+0000","last_active":"2024-04-25T10:08:21.318544+0000","last_peered":"2024-04-25T10:08:21.318544+0000","last_clean":"2024-04-25T10:08:21.318544+0000","last_became_active":"2024-04-25T10:08:18.941387+0000","last_became_peered":"2024-04-25T10:08:18.941387+0000","last_unstale":"2024-04-25T10:08:21.318544+0000","last_undegraded":"2024-04-25T10:08:21.318544+0000","last_fullsized":"2024-04-25T10:08:21.318544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:57.734880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316464+0000","last_change":"2024-04-25T10:08:19.975526+0000","last_active":"2024-04-25T10:08:21.316464+0000","last_peered":"2024-04-25T10:08:21.316464+0000","last_clean":"2024-04-25T10:08:21.316464+0000","last_became_active":"2024-04-25T10:08:19.975198+0000","last_became_peered":"2024-04-25T10:08:19.975198+0000","last_unstale":"2024-04-25T10:08:21.316464+0000","last_undegraded":"2024-04-25T10:08:21.316464+0000","last_fullsized":"2024-04-25T10:08:21.316464+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:27.968601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317914+0000","last_change":"2024-04-25T10:08:18.949137+0000","last_active":"2024-04-25T10:08:21.317914+0000","last_peered":"2024-04-25T10:08:21.317914+0000","last_clean":"2024-04-25T10:08:21.317914+0000","last_became_active":"2024-04-25T10:08:18.948667+0000","last_became_peered":"2024-04-25T10:08:18.948667+0000","last_unstale":"2024-04-25T10:08:21.317914+0000","last_undegraded":"2024-04-25T10:08:21.317914+0000","last_fullsized":"2024-04-25T10:08:21.317914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:17:15.026390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316152+0000","last_change":"2024-04-25T10:08:19.976587+0000","last_active":"2024-04-25T10:08:21.316152+0000","last_peered":"2024-04-25T10:08:21.316152+0000","last_clean":"2024-04-25T10:08:21.316152+0000","last_became_active":"2024-04-25T10:08:19.975088+0000","last_became_peered":"2024-04-25T10:08:19.975088+0000","last_unstale":"2024-04-25T10:08:21.316152+0000","last_undegraded":"2024-04-25T10:08:21.316152+0000","last_fullsized":"2024-04-25T10:08:21.316152+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:05.641258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986009+0000","last_change":"2024-04-25T10:08:18.947481+0000","last_active":"2024-04-25T10:08:20.986009+0000","last_peered":"2024-04-25T10:08:20.986009+0000","last_clean":"2024-04-25T10:08:20.986009+0000","last_became_active":"2024-04-25T10:08:18.947151+0000","last_became_peered":"2024-04-25T10:08:18.947151+0000","last_unstale":"2024-04-25T10:08:20.986009+0000","last_undegraded":"2024-04-25T10:08:20.986009+0000","last_fullsized":"2024-04-25T10:08:20.986009+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:19.887767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962182+0000","last_change":"2024-04-25T10:08:19.988375+0000","last_active":"2024-04-25T10:08:20.962182+0000","last_peered":"2024-04-25T10:08:20.962182+0000","last_clean":"2024-04-25T10:08:20.962182+0000","last_became_active":"2024-04-25T10:08:19.986299+0000","last_became_peered":"2024-04-25T10:08:19.986299+0000","last_unstale":"2024-04-25T10:08:20.962182+0000","last_undegraded":"2024-04-25T10:08:20.962182+0000","last_fullsized":"2024-04-25T10:08:20.962182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:19.022397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"84'6","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965347+0000","last_change":"2024-04-25T10:08:18.948358+0000","last_active":"2024-04-25T10:08:20.965347+0000","last_peered":"2024-04-25T10:08:20.965347+0000","last_clean":"2024-04-25T10:08:20.965347+0000","last_became_active":"2024-04-25T10:08:18.945639+0000","last_became_peered":"2024-04-25T10:08:18.945639+0000","last_unstale":"2024-04-25T10:08:20.965347+0000","last_undegraded":"2024-04-25T10:08:20.965347+0000","last_fullsized":"2024-04-25T10:08:20.965347+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:44:15.003395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317648+0000","last_change":"2024-04-25T10:08:19.975766+0000","last_active":"2024-04-25T10:08:21.317648+0000","last_peered":"2024-04-25T10:08:21.317648+0000","last_clean":"2024-04-25T10:08:21.317648+0000","last_became_active":"2024-04-25T10:08:19.975302+0000","last_became_peered":"2024-04-25T10:08:19.975302+0000","last_unstale":"2024-04-25T10:08:21.317648+0000","last_undegraded":"2024-04-25T10:08:21.317648+0000","last_fullsized":"2024-04-25T10:08:21.317648+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:42.765039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316092+0000","last_change":"2024-04-25T10:08:18.947045+0000","last_active":"2024-04-25T10:08:21.316092+0000","last_peered":"2024-04-25T10:08:21.316092+0000","last_clean":"2024-04-25T10:08:21.316092+0000","last_became_active":"2024-04-25T10:08:18.946758+0000","last_became_peered":"2024-04-25T10:08:18.946758+0000","last_unstale":"2024-04-25T10:08:21.316092+0000","last_undegraded":"2024-04-25T10:08:21.316092+0000","last_fullsized":"2024-04-25T10:08:21.316092+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:09.420066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964017+0000","last_change":"2024-04-25T10:08:19.970045+0000","last_active":"2024-04-25T10:08:20.964017+0000","last_peered":"2024-04-25T10:08:20.964017+0000","last_clean":"2024-04-25T10:08:20.964017+0000","last_became_active":"2024-04-25T10:08:19.968064+0000","last_became_peered":"2024-04-25T10:08:19.968064+0000","last_unstale":"2024-04-25T10:08:20.964017+0000","last_undegraded":"2024-04-25T10:08:20.964017+0000","last_fullsized":"2024-04-25T10:08:20.964017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:22.965572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317343+0000","last_change":"2024-04-25T10:08:18.948243+0000","last_active":"2024-04-25T10:08:21.317343+0000","last_peered":"2024-04-25T10:08:21.317343+0000","last_clean":"2024-04-25T10:08:21.317343+0000","last_became_active":"2024-04-25T10:08:18.946281+0000","last_became_peered":"2024-04-25T10:08:18.946281+0000","last_unstale":"2024-04-25T10:08:21.317343+0000","last_undegraded":"2024-04-25T10:08:21.317343+0000","last_fullsized":"2024-04-25T10:08:21.317343+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:30:18.992945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965558+0000","last_change":"2024-04-25T10:08:19.966278+0000","last_active":"2024-04-25T10:08:20.965558+0000","last_peered":"2024-04-25T10:08:20.965558+0000","last_clean":"2024-04-25T10:08:20.965558+0000","last_became_active":"2024-04-25T10:08:19.965964+0000","last_became_peered":"2024-04-25T10:08:19.965964+0000","last_unstale":"2024-04-25T10:08:20.965558+0000","last_undegraded":"2024-04-25T10:08:20.965558+0000","last_fullsized":"2024-04-25T10:08:20.965558+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:20:14.981959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985104+0000","last_change":"2024-04-25T10:08:18.945515+0000","last_active":"2024-04-25T10:08:20.985104+0000","last_peered":"2024-04-25T10:08:20.985104+0000","last_clean":"2024-04-25T10:08:20.985104+0000","last_became_active":"2024-04-25T10:08:18.944832+0000","last_became_peered":"2024-04-25T10:08:18.944832+0000","last_unstale":"2024-04-25T10:08:20.985104+0000","last_undegraded":"2024-04-25T10:08:20.985104+0000","last_fullsized":"2024-04-25T10:08:20.985104+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:08.377481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966114+0000","last_change":"2024-04-25T10:08:19.974531+0000","last_active":"2024-04-25T10:08:20.966114+0000","last_peered":"2024-04-25T10:08:20.966114+0000","last_clean":"2024-04-25T10:08:20.966114+0000","last_became_active":"2024-04-25T10:08:19.972210+0000","last_became_peered":"2024-04-25T10:08:19.972210+0000","last_unstale":"2024-04-25T10:08:20.966114+0000","last_undegraded":"2024-04-25T10:08:20.966114+0000","last_fullsized":"2024-04-25T10:08:20.966114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:16.427011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"84'8","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.978887+0000","last_change":"2024-04-25T10:08:18.946298+0000","last_active":"2024-04-25T10:08:20.978887+0000","last_peered":"2024-04-25T10:08:20.978887+0000","last_clean":"2024-04-25T10:08:20.978887+0000","last_became_active":"2024-04-25T10:08:18.946020+0000","last_became_peered":"2024-04-25T10:08:18.946020+0000","last_unstale":"2024-04-25T10:08:20.978887+0000","last_undegraded":"2024-04-25T10:08:20.978887+0000","last_fullsized":"2024-04-25T10:08:20.978887+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:46.279014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315631+0000","last_change":"2024-04-25T10:08:19.973249+0000","last_active":"2024-04-25T10:08:21.315631+0000","last_peered":"2024-04-25T10:08:21.315631+0000","last_clean":"2024-04-25T10:08:21.315631+0000","last_became_active":"2024-04-25T10:08:19.972893+0000","last_became_peered":"2024-04-25T10:08:19.972893+0000","last_unstale":"2024-04-25T10:08:21.315631+0000","last_undegraded":"2024-04-25T10:08:21.315631+0000","last_fullsized":"2024-04-25T10:08:21.315631+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.430130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318757+0000","last_change":"2024-04-25T10:08:18.944678+0000","last_active":"2024-04-25T10:08:21.318757+0000","last_peered":"2024-04-25T10:08:21.318757+0000","last_clean":"2024-04-25T10:08:21.318757+0000","last_became_active":"2024-04-25T10:08:18.944351+0000","last_became_peered":"2024-04-25T10:08:18.944351+0000","last_unstale":"2024-04-25T10:08:21.318757+0000","last_undegraded":"2024-04-25T10:08:21.318757+0000","last_fullsized":"2024-04-25T10:08:21.318757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:23:39.872883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986013+0000","last_change":"2024-04-25T10:08:19.957278+0000","last_active":"2024-04-25T10:08:20.986013+0000","last_peered":"2024-04-25T10:08:20.986013+0000","last_clean":"2024-04-25T10:08:20.986013+0000","last_became_active":"2024-04-25T10:08:19.955777+0000","last_became_peered":"2024-04-25T10:08:19.955777+0000","last_unstale":"2024-04-25T10:08:20.986013+0000","last_undegraded":"2024-04-25T10:08:20.986013+0000","last_fullsized":"2024-04-25T10:08:20.986013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:19.944001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967795+0000","last_change":"2024-04-25T10:08:18.946830+0000","last_active":"2024-04-25T10:08:20.967795+0000","last_peered":"2024-04-25T10:08:20.967795+0000","last_clean":"2024-04-25T10:08:20.967795+0000","last_became_active":"2024-04-25T10:08:18.945619+0000","last_became_peered":"2024-04-25T10:08:18.945619+0000","last_unstale":"2024-04-25T10:08:20.967795+0000","last_undegraded":"2024-04-25T10:08:20.967795+0000","last_fullsized":"2024-04-25T10:08:20.967795+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:11.980984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985144+0000","last_change":"2024-04-25T10:08:19.963094+0000","last_active":"2024-04-25T10:08:20.985144+0000","last_peered":"2024-04-25T10:08:20.985144+0000","last_clean":"2024-04-25T10:08:20.985144+0000","last_became_active":"2024-04-25T10:08:19.962771+0000","last_became_peered":"2024-04-25T10:08:19.962771+0000","last_unstale":"2024-04-25T10:08:20.985144+0000","last_undegraded":"2024-04-25T10:08:20.985144+0000","last_fullsized":"2024-04-25T10:08:20.985144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:39.425266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.986176+0000","last_change":"2024-04-25T10:08:18.944720+0000","last_active":"2024-04-25T10:08:20.986176+0000","last_peered":"2024-04-25T10:08:20.986176+0000","last_clean":"2024-04-25T10:08:20.986176+0000","last_became_active":"2024-04-25T10:08:18.944425+0000","last_became_peered":"2024-04-25T10:08:18.944425+0000","last_unstale":"2024-04-25T10:08:20.986176+0000","last_undegraded":"2024-04-25T10:08:20.986176+0000","last_fullsized":"2024-04-25T10:08:20.986176+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:35.057954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315850+0000","last_change":"2024-04-25T10:08:19.979662+0000","last_active":"2024-04-25T10:08:21.315850+0000","last_peered":"2024-04-25T10:08:21.315850+0000","last_clean":"2024-04-25T10:08:21.315850+0000","last_became_active":"2024-04-25T10:08:19.979357+0000","last_became_peered":"2024-04-25T10:08:19.979357+0000","last_unstale":"2024-04-25T10:08:21.315850+0000","last_undegraded":"2024-04-25T10:08:21.315850+0000","last_fullsized":"2024-04-25T10:08:21.315850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:35.566767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962055+0000","last_change":"2024-04-25T10:08:18.941845+0000","last_active":"2024-04-25T10:08:20.962055+0000","last_peered":"2024-04-25T10:08:20.962055+0000","last_clean":"2024-04-25T10:08:20.962055+0000","last_became_active":"2024-04-25T10:08:18.941177+0000","last_became_peered":"2024-04-25T10:08:18.941177+0000","last_unstale":"2024-04-25T10:08:20.962055+0000","last_undegraded":"2024-04-25T10:08:20.962055+0000","last_fullsized":"2024-04-25T10:08:20.962055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:40.645361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979531+0000","last_change":"2024-04-25T10:08:19.976404+0000","last_active":"2024-04-25T10:08:20.979531+0000","last_peered":"2024-04-25T10:08:20.979531+0000","last_clean":"2024-04-25T10:08:20.979531+0000","last_became_active":"2024-04-25T10:08:19.976082+0000","last_became_peered":"2024-04-25T10:08:19.976082+0000","last_unstale":"2024-04-25T10:08:20.979531+0000","last_undegraded":"2024-04-25T10:08:20.979531+0000","last_fullsized":"2024-04-25T10:08:20.979531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:24:25.962057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315872+0000","last_change":"2024-04-25T10:08:18.930499+0000","last_active":"2024-04-25T10:08:21.315872+0000","last_peered":"2024-04-25T10:08:21.315872+0000","last_clean":"2024-04-25T10:08:21.315872+0000","last_became_active":"2024-04-25T10:08:18.928003+0000","last_became_peered":"2024-04-25T10:08:18.928003+0000","last_unstale":"2024-04-25T10:08:21.315872+0000","last_undegraded":"2024-04-25T10:08:21.315872+0000","last_fullsized":"2024-04-25T10:08:21.315872+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:45.925929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.980532+0000","last_change":"2024-04-25T10:08:19.979443+0000","last_active":"2024-04-25T10:08:20.980532+0000","last_peered":"2024-04-25T10:08:20.980532+0000","last_clean":"2024-04-25T10:08:20.980532+0000","last_became_active":"2024-04-25T10:08:19.979179+0000","last_became_peered":"2024-04-25T10:08:19.979179+0000","last_unstale":"2024-04-25T10:08:20.980532+0000","last_undegraded":"2024-04-25T10:08:20.980532+0000","last_fullsized":"2024-04-25T10:08:20.980532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:15:08.422234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315633+0000","last_change":"2024-04-25T10:08:18.933138+0000","last_active":"2024-04-25T10:08:21.315633+0000","last_peered":"2024-04-25T10:08:21.315633+0000","last_clean":"2024-04-25T10:08:21.315633+0000","last_became_active":"2024-04-25T10:08:18.932647+0000","last_became_peered":"2024-04-25T10:08:18.932647+0000","last_unstale":"2024-04-25T10:08:21.315633+0000","last_undegraded":"2024-04-25T10:08:21.315633+0000","last_fullsized":"2024-04-25T10:08:21.315633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:50.341616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.966535+0000","last_change":"2024-04-25T10:08:19.965027+0000","last_active":"2024-04-25T10:08:20.966535+0000","last_peered":"2024-04-25T10:08:20.966535+0000","last_clean":"2024-04-25T10:08:20.966535+0000","last_became_active":"2024-04-25T10:08:19.963117+0000","last_became_peered":"2024-04-25T10:08:19.963117+0000","last_unstale":"2024-04-25T10:08:20.966535+0000","last_undegraded":"2024-04-25T10:08:20.966535+0000","last_fullsized":"2024-04-25T10:08:20.966535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:59:59.544140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.315938+0000","last_change":"2024-04-25T10:08:18.934845+0000","last_active":"2024-04-25T10:08:21.315938+0000","last_peered":"2024-04-25T10:08:21.315938+0000","last_clean":"2024-04-25T10:08:21.315938+0000","last_became_active":"2024-04-25T10:08:18.931926+0000","last_became_peered":"2024-04-25T10:08:18.931926+0000","last_unstale":"2024-04-25T10:08:21.315938+0000","last_undegraded":"2024-04-25T10:08:21.315938+0000","last_fullsized":"2024-04-25T10:08:21.315938+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:24:43.165217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965046+0000","last_change":"2024-04-25T10:08:19.966296+0000","last_active":"2024-04-25T10:08:20.965046+0000","last_peered":"2024-04-25T10:08:20.965046+0000","last_clean":"2024-04-25T10:08:20.965046+0000","last_became_active":"2024-04-25T10:08:19.966008+0000","last_became_peered":"2024-04-25T10:08:19.966008+0000","last_unstale":"2024-04-25T10:08:20.965046+0000","last_undegraded":"2024-04-25T10:08:20.965046+0000","last_fullsized":"2024-04-25T10:08:20.965046+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:18:58.324062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317397+0000","last_change":"2024-04-25T10:08:18.947796+0000","last_active":"2024-04-25T10:08:21.317397+0000","last_peered":"2024-04-25T10:08:21.317397+0000","last_clean":"2024-04-25T10:08:21.317397+0000","last_became_active":"2024-04-25T10:08:18.947508+0000","last_became_peered":"2024-04-25T10:08:18.947508+0000","last_unstale":"2024-04-25T10:08:21.317397+0000","last_undegraded":"2024-04-25T10:08:21.317397+0000","last_fullsized":"2024-04-25T10:08:21.317397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:35:47.962610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964945+0000","last_change":"2024-04-25T10:08:19.971283+0000","last_active":"2024-04-25T10:08:20.964945+0000","last_peered":"2024-04-25T10:08:20.964945+0000","last_clean":"2024-04-25T10:08:20.964945+0000","last_became_active":"2024-04-25T10:08:19.971000+0000","last_became_peered":"2024-04-25T10:08:19.971000+0000","last_unstale":"2024-04-25T10:08:20.964945+0000","last_undegraded":"2024-04-25T10:08:20.964945+0000","last_fullsized":"2024-04-25T10:08:20.964945+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:20.725052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.983901+0000","last_change":"2024-04-25T10:08:18.944659+0000","last_active":"2024-04-25T10:08:20.983901+0000","last_peered":"2024-04-25T10:08:20.983901+0000","last_clean":"2024-04-25T10:08:20.983901+0000","last_became_active":"2024-04-25T10:08:18.944306+0000","last_became_peered":"2024-04-25T10:08:18.944306+0000","last_unstale":"2024-04-25T10:08:20.983901+0000","last_undegraded":"2024-04-25T10:08:20.983901+0000","last_fullsized":"2024-04-25T10:08:20.983901+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:23:42.254843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.967151+0000","last_change":"2024-04-25T10:08:19.978770+0000","last_active":"2024-04-25T10:08:20.967151+0000","last_peered":"2024-04-25T10:08:20.967151+0000","last_clean":"2024-04-25T10:08:20.967151+0000","last_became_active":"2024-04-25T10:08:19.978468+0000","last_became_peered":"2024-04-25T10:08:19.978468+0000","last_unstale":"2024-04-25T10:08:20.967151+0000","last_undegraded":"2024-04-25T10:08:20.967151+0000","last_fullsized":"2024-04-25T10:08:20.967151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:25.309102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318583+0000","last_change":"2024-04-25T10:08:18.945901+0000","last_active":"2024-04-25T10:08:21.318583+0000","last_peered":"2024-04-25T10:08:21.318583+0000","last_clean":"2024-04-25T10:08:21.318583+0000","last_became_active":"2024-04-25T10:08:18.945568+0000","last_became_peered":"2024-04-25T10:08:18.945568+0000","last_unstale":"2024-04-25T10:08:21.318583+0000","last_undegraded":"2024-04-25T10:08:21.318583+0000","last_fullsized":"2024-04-25T10:08:21.318583+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:46.279014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.979915+0000","last_change":"2024-04-25T10:08:19.982195+0000","last_active":"2024-04-25T10:08:20.979915+0000","last_peered":"2024-04-25T10:08:20.979915+0000","last_clean":"2024-04-25T10:08:20.979915+0000","last_became_active":"2024-04-25T10:08:19.981826+0000","last_became_peered":"2024-04-25T10:08:19.981826+0000","last_unstale":"2024-04-25T10:08:20.979915+0000","last_undegraded":"2024-04-25T10:08:20.979915+0000","last_fullsized":"2024-04-25T10:08:20.979915+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.912266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317912+0000","last_change":"2024-04-25T10:08:18.941173+0000","last_active":"2024-04-25T10:08:21.317912+0000","last_peered":"2024-04-25T10:08:21.317912+0000","last_clean":"2024-04-25T10:08:21.317912+0000","last_became_active":"2024-04-25T10:08:18.940897+0000","last_became_peered":"2024-04-25T10:08:18.940897+0000","last_unstale":"2024-04-25T10:08:21.317912+0000","last_undegraded":"2024-04-25T10:08:21.317912+0000","last_fullsized":"2024-04-25T10:08:21.317912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:41.750187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964904+0000","last_change":"2024-04-25T10:08:19.971882+0000","last_active":"2024-04-25T10:08:20.964904+0000","last_peered":"2024-04-25T10:08:20.964904+0000","last_clean":"2024-04-25T10:08:20.964904+0000","last_became_active":"2024-04-25T10:08:19.971617+0000","last_became_peered":"2024-04-25T10:08:19.971617+0000","last_unstale":"2024-04-25T10:08:20.964904+0000","last_undegraded":"2024-04-25T10:08:20.964904+0000","last_fullsized":"2024-04-25T10:08:20.964904+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:11.111384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964888+0000","last_change":"2024-04-25T10:08:18.935168+0000","last_active":"2024-04-25T10:08:20.964888+0000","last_peered":"2024-04-25T10:08:20.964888+0000","last_clean":"2024-04-25T10:08:20.964888+0000","last_became_active":"2024-04-25T10:08:18.931137+0000","last_became_peered":"2024-04-25T10:08:18.931137+0000","last_unstale":"2024-04-25T10:08:20.964888+0000","last_undegraded":"2024-04-25T10:08:20.964888+0000","last_fullsized":"2024-04-25T10:08:20.964888+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:15.685967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.964034+0000","last_change":"2024-04-25T10:08:19.969101+0000","last_active":"2024-04-25T10:08:20.964034+0000","last_peered":"2024-04-25T10:08:20.964034+0000","last_clean":"2024-04-25T10:08:20.964034+0000","last_became_active":"2024-04-25T10:08:19.968484+0000","last_became_peered":"2024-04-25T10:08:19.968484+0000","last_unstale":"2024-04-25T10:08:20.964034+0000","last_undegraded":"2024-04-25T10:08:20.964034+0000","last_fullsized":"2024-04-25T10:08:20.964034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:00.235689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.985538+0000","last_change":"2024-04-25T10:08:18.947971+0000","last_active":"2024-04-25T10:08:20.985538+0000","last_peered":"2024-04-25T10:08:20.985538+0000","last_clean":"2024-04-25T10:08:20.985538+0000","last_became_active":"2024-04-25T10:08:18.947544+0000","last_became_peered":"2024-04-25T10:08:18.947544+0000","last_unstale":"2024-04-25T10:08:20.985538+0000","last_undegraded":"2024-04-25T10:08:20.985538+0000","last_fullsized":"2024-04-25T10:08:20.985538+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:44.428058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962039+0000","last_change":"2024-04-25T10:08:19.987566+0000","last_active":"2024-04-25T10:08:20.962039+0000","last_peered":"2024-04-25T10:08:20.962039+0000","last_clean":"2024-04-25T10:08:20.962039+0000","last_became_active":"2024-04-25T10:08:19.987263+0000","last_became_peered":"2024-04-25T10:08:19.987263+0000","last_unstale":"2024-04-25T10:08:20.962039+0000","last_undegraded":"2024-04-25T10:08:20.962039+0000","last_fullsized":"2024-04-25T10:08:20.962039+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:16.427011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.965246+0000","last_change":"2024-04-25T10:08:18.933657+0000","last_active":"2024-04-25T10:08:20.965246+0000","last_peered":"2024-04-25T10:08:20.965246+0000","last_clean":"2024-04-25T10:08:20.965246+0000","last_became_active":"2024-04-25T10:08:18.932270+0000","last_became_peered":"2024-04-25T10:08:18.932270+0000","last_unstale":"2024-04-25T10:08:20.965246+0000","last_undegraded":"2024-04-25T10:08:20.965246+0000","last_fullsized":"2024-04-25T10:08:20.965246+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:05.325048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.962407+0000","last_change":"2024-04-25T10:08:19.964445+0000","last_active":"2024-04-25T10:08:20.962407+0000","last_peered":"2024-04-25T10:08:20.962407+0000","last_clean":"2024-04-25T10:08:20.962407+0000","last_became_active":"2024-04-25T10:08:19.964176+0000","last_became_peered":"2024-04-25T10:08:19.964176+0000","last_unstale":"2024-04-25T10:08:20.962407+0000","last_undegraded":"2024-04-25T10:08:20.962407+0000","last_fullsized":"2024-04-25T10:08:20.962407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:43:30.542619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:20.961382+0000","last_change":"2024-04-25T10:08:18.941880+0000","last_active":"2024-04-25T10:08:20.961382+0000","last_peered":"2024-04-25T10:08:20.961382+0000","last_clean":"2024-04-25T10:08:20.961382+0000","last_became_active":"2024-04-25T10:08:18.941576+0000","last_became_peered":"2024-04-25T10:08:18.941576+0000","last_unstale":"2024-04-25T10:08:20.961382+0000","last_undegraded":"2024-04-25T10:08:20.961382+0000","last_fullsized":"2024-04-25T10:08:20.961382+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:30.807252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.318781+0000","last_change":"2024-04-25T10:08:19.967200+0000","last_active":"2024-04-25T10:08:21.318781+0000","last_peered":"2024-04-25T10:08:21.318781+0000","last_clean":"2024-04-25T10:08:21.318781+0000","last_became_active":"2024-04-25T10:08:19.966933+0000","last_became_peered":"2024-04-25T10:08:19.966933+0000","last_unstale":"2024-04-25T10:08:21.318781+0000","last_undegraded":"2024-04-25T10:08:21.318781+0000","last_fullsized":"2024-04-25T10:08:21.318781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:18.902971+0000","last_clean_scrub_stamp":"2024-04-25T10:08:18.902971+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:34.430130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.316398+0000","last_change":"2024-04-25T10:08:18.935935+0000","last_active":"2024-04-25T10:08:21.316398+0000","last_peered":"2024-04-25T10:08:21.316398+0000","last_clean":"2024-04-25T10:08:21.316398+0000","last_became_active":"2024-04-25T10:08:18.932340+0000","last_became_peered":"2024-04-25T10:08:18.932340+0000","last_unstale":"2024-04-25T10:08:21.316398+0000","last_undegraded":"2024-04-25T10:08:21.316398+0000","last_fullsized":"2024-04-25T10:08:21.316398+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:08:17.888119+0000","last_clean_scrub_stamp":"2024-04-25T10:08:17.888119+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:46.175875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":292,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-25T10:08:21.317217+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:08:21.317217+0000","last_peered":"2024-04-25T10:08:21.317217+0000","last_clean":"2024-04-25T10:08:21.317217+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:08:21.317217+0000","last_undegraded":"2024-04-25T10:08:21.317217+0000","last_fullsized":"2024-04-25T10:08:21.317217+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574930,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41104,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94330736,"statfs":{"total":96636764160,"available":96594673664,"internally_reserved":0,"allocated":1638400,"data_stored":1072558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":40434123},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40528,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331312,"statfs":{"total":96636764160,"available":96595263488,"internally_reserved":0,"allocated":1052672,"data_stored":483780,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":40434118},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574930,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1044480,"data_stored":482202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574930,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40544,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331296,"statfs":{"total":96636764160,"available":96595247104,"internally_reserved":0,"allocated":1060864,"data_stored":483884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574930,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40520,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331320,"statfs":{"total":96636764160,"available":96595271680,"internally_reserved":0,"allocated":1044480,"data_stored":482750,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574929,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40536,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331304,"statfs":{"total":96636764160,"available":96595255296,"internally_reserved":0,"allocated":1052672,"data_stored":482910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574930,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36364,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335476,"statfs":{"total":96636764160,"available":96599527424,"internally_reserved":0,"allocated":1048576,"data_stored":482280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574930,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46668,"kb_used_data":1596,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325172,"statfs":{"total":96636764160,"available":96588976128,"internally_reserved":0,"allocated":1634304,"data_stored":1072536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:08:24.790 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:08:24.790 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.031+0000 7f9260a00640 1 Processor -- start 2024-04-25T10:08:25.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.031+0000 7f9260a00640 1 -- start start 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9260a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f925c1446d0 0x7f925c0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9260a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f925c06e8a0 0x7f925c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9260a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f925c0cccc0 con 0x7f925c144d50 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9260a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c0b2b20 con 0x7f925c1446d0 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9260a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f925c06d400 con 0x7f925c06e8a0 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9259e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f925c1446d0 0x7f925c0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9259400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f925c144d50 0x7f925c0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35890/0 (socket says 172.21.15.165:35890) 2024-04-25T10:08:25.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.033+0000 7f9259400640 1 -- 172.21.15.165:0/2703172689 learned_addr learned my addr 172.21.15.165:0/2703172689 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:25.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f925a800640 1 --2- 172.21.15.165:0/2703172689 >> v2:172.21.15.172:3300/0 conn(0x7f925c06e8a0 0x7f925c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f9259e00640 1 -- 172.21.15.165:0/2703172689 >> v2:172.21.15.172:3300/0 conn(0x7f925c06e8a0 msgr2=0x7f925c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f9259e00640 1 --2- 172.21.15.165:0/2703172689 >> v2:172.21.15.172:3300/0 conn(0x7f925c06e8a0 0x7f925c0b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f9258a00640 1 -- 172.21.15.165:0/2703172689 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760133723 0 0) 0x7f925c0cccc0 con 0x7f925c144d50 2024-04-25T10:08:25.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f9259e00640 1 -- 172.21.15.165:0/2703172689 >> v1:172.21.15.165:6789/0 conn(0x7f925c144d50 legacy=0x7f925c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f9259e00640 1 -- 172.21.15.165:0/2703172689 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c0cc140 con 0x7f925c1446d0 2024-04-25T10:08:25.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.034+0000 7f925a800640 1 --2- 172.21.15.165:0/2703172689 >> v2:172.21.15.172:3300/0 conn(0x7f925c06e8a0 0x7f925c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.036+0000 7f9259e00640 1 --2- 172.21.15.165:0/2703172689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1446d0 0x7f925c0b3f20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f925c071200 tx=0x7f925402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b86cf2580f83dfcd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.037+0000 7f9258a00640 1 -- 172.21.15.165:0/2703172689 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254007740 con 0x7f925c1446d0 2024-04-25T10:08:25.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.037+0000 7f9258a00640 1 -- 172.21.15.165:0/2703172689 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92540078e0 con 0x7f925c1446d0 2024-04-25T10:08:25.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.037+0000 7f9258a00640 1 -- 172.21.15.165:0/2703172689 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9254007cb0 con 0x7f925c1446d0 2024-04-25T10:08:25.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.037+0000 7f9260a00640 1 -- 172.21.15.165:0/2703172689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1446d0 msgr2=0x7f925c0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.037+0000 7f9260a00640 1 --2- 172.21.15.165:0/2703172689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1446d0 0x7f925c0b3f20 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f925c071200 tx=0x7f925402a680 comp rx=0 tx=0).stop 2024-04-25T10:08:25.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.038+0000 7f9260a00640 1 -- 172.21.15.165:0/2703172689 shutdown_connections 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.038+0000 7f9260a00640 1 --2- 172.21.15.165:0/2703172689 >> v2:172.21.15.172:3300/0 conn(0x7f925c06e8a0 0x7f925c0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.038+0000 7f9260a00640 1 --2- 172.21.15.165:0/2703172689 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c1446d0 0x7f925c0b3f20 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.038+0000 7f9260a00640 1 -- 172.21.15.165:0/2703172689 >> 172.21.15.165:0/2703172689 conn(0x7f925c06abe0 msgr2=0x7f925c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.038+0000 7f9260a00640 1 -- 172.21.15.165:0/2703172689 shutdown_connections 2024-04-25T10:08:25.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.038+0000 7f9260a00640 1 -- 172.21.15.165:0/2703172689 wait complete. 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.040+0000 7f9260a00640 1 Processor -- start 2024-04-25T10:08:25.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.040+0000 7f9260a00640 1 -- start start 2024-04-25T10:08:25.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9260a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f925c06e8a0 0x7f925c1394d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9260a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925c1446d0 0x7f925c15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9260a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 0x7f925c0c4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9260a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cc700 con 0x7f925c144d50 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9260a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0b49a0 con 0x7f925c1446d0 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9260a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f925c0cc9e0 con 0x7f925c06e8a0 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 0x7f925c0c4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 0x7f925c0c4ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41318/0 (socket says 172.21.15.165:41318) 2024-04-25T10:08:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 -- 172.21.15.165:0/759872884 learned_addr learned my addr 172.21.15.165:0/759872884 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9259400640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925c1446d0 0x7f925c15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f9259e00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f925c06e8a0 0x7f925c1394d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 -- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f925c06e8a0 msgr2=0x7f925c1394d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f925c06e8a0 0x7f925c1394d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 -- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925c1446d0 msgr2=0x7f925c15ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.041+0000 7f925a800640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925c1446d0 0x7f925c15ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f925a800640 1 -- 172.21.15.165:0/759872884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f925c138430 con 0x7f925c144d50 2024-04-25T10:08:25.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9259400640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925c1446d0 0x7f925c15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9259e00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f925c06e8a0 0x7f925c1394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f925a800640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 0x7f925c0c4ee0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f924400c7f0 tx=0x7f924400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244015070 con 0x7f925c144d50 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9244007820 con 0x7f925c144d50 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244007c50 con 0x7f925c144d50 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925c138710 con 0x7f925c144d50 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.042+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925c138cd0 con 0x7f925c144d50 2024-04-25T10:08:25.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.043+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f925c073a00 con 0x7f925c144d50 2024-04-25T10:08:25.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.044+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9244018070 con 0x7f925c144d50 2024-04-25T10:08:25.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.045+0000 7f9252000640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f923806b500 0x7f923806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.045+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f924408a500 con 0x7f925c144d50 2024-04-25T10:08:25.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.046+0000 7f9259e00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f923806b500 0x7f923806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.046+0000 7f9259e00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f923806b500 0x7f923806d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9254034000 tx=0x7f925402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.049+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92440591e0 con 0x7f925c144d50 2024-04-25T10:08:25.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.314+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f925c0b4260 con 0x7f925c144d50 2024-04-25T10:08:25.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.319+0000 7f9252000640 1 -- 172.21.15.165:0/759872884 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+147 (secure 0 0 0) 0x7f9244058ae0 con 0x7f925c144d50 2024-04-25T10:08:25.325 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:08:25.325 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:08:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:08:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:08:25.326 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:08:25.326 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:08:25.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f923806b500 msgr2=0x7f923806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f923806b500 0x7f923806d9c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f9254034000 tx=0x7f925402f000 comp rx=0 tx=0).stop 2024-04-25T10:08:25.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 msgr2=0x7f925c0c4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 0x7f925c0c4ee0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f924400c7f0 tx=0x7f924400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 shutdown_connections 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f923806b500 0x7f923806d9c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f925c144d50 0x7f925c0c4ee0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f925c1446d0 0x7f925c15ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 --2- 172.21.15.165:0/759872884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f925c06e8a0 0x7f925c1394d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 >> 172.21.15.165:0/759872884 conn(0x7f925c06abe0 msgr2=0x7f925c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 shutdown_connections 2024-04-25T10:08:25.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.322+0000 7f9260a00640 1 -- 172.21.15.165:0/759872884 wait complete. 2024-04-25T10:08:25.347 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:08:25.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.533+0000 7f74bbe00640 1 Processor -- start 2024-04-25T10:08:25.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.533+0000 7f74bbe00640 1 -- start start 2024-04-25T10:08:25.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74bbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74bc0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74bbe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc15bf30 con 0x7f74bc144440 2024-04-25T10:08:25.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74bbe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74bc15c210 con 0x7f74bc071700 2024-04-25T10:08:25.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74bbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc15c860 con 0x7f74bc15dd00 2024-04-25T10:08:25.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74baa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f74bc144440 0x7f74bc070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35898/0 (socket says 172.21.15.165:35898) 2024-04-25T10:08:25.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74baa00640 1 -- 172.21.15.165:0/225277095 learned_addr learned my addr 172.21.15.165:0/225277095 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74bb400640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74bc0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129295692 0 0) 0x7f74bc15bf30 con 0x7f74bc144440 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.535+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74a0003520 con 0x7f74bc144440 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2324640908 0 0) 0x7f74a0003520 con 0x7f74bc144440 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 msgr2=0x7f74bc0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74bc0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 >> v1:172.21.15.172:6789/0 conn(0x7f74bc071700 legacy=0x7f74bc0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc15c4f0 con 0x7f74bc144440 2024-04-25T10:08:25.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f74ac001670 con 0x7f74bc144440 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 >> v1:172.21.15.165:6789/0 conn(0x7f74bc144440 legacy=0x7f74bc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74a00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a0003f60 0x7f74a0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74a00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc15c860 con 0x7f74a0004d40 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a000bb30 con 0x7f74a0003f60 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a000bd30 con 0x7f74bc15dd00 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.536+0000 7f74bb400640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74a00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74baa00640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74a00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74b3e00640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a0003f60 0x7f74a0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74bb400640 1 -- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 msgr2=0x7f74a00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74bb400640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74a00039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74bb400640 1 -- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a0003f60 msgr2=0x7f74a0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74bb400640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a0003f60 0x7f74a0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74bb400640 1 -- 172.21.15.165:0/225277095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a0003520 con 0x7f74a0004d40 2024-04-25T10:08:25.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74baa00640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15dd00 0x7f74a00039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.537+0000 7f74b3e00640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a0003f60 0x7f74a0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.538+0000 7f74bb400640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74a00051c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009fc0 tx=0x7f74a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0905a7ee81fe8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.538+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4009c10 con 0x7f74a0004d40 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.538+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b4036070 con 0x7f74a0004d40 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.538+0000 7f74ba000640 1 -- 172.21.15.165:0/225277095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b4031880 con 0x7f74a0004d40 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bbe00640 1 -- 172.21.15.165:0/225277095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 msgr2=0x7f74a00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bbe00640 1 --2- 172.21.15.165:0/225277095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74a00051c0 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009fc0 tx=0x7f74a0005830 comp rx=0 tx=0).stop 2024-04-25T10:08:25.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bb400640 1 -- 172.21.15.165:0/225277095 reap_dead start 2024-04-25T10:08:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bbe00640 1 -- 172.21.15.165:0/225277095 shutdown_connections 2024-04-25T10:08:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bbe00640 1 -- 172.21.15.165:0/225277095 >> 172.21.15.165:0/225277095 conn(0x7f74bc06abe0 msgr2=0x7f74bc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bbe00640 1 -- 172.21.15.165:0/225277095 shutdown_connections 2024-04-25T10:08:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.539+0000 7f74bbe00640 1 -- 172.21.15.165:0/225277095 wait complete. 2024-04-25T10:08:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.541+0000 7f74bbe00640 1 Processor -- start 2024-04-25T10:08:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.541+0000 7f74bbe00640 1 -- start start 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74bbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74bc135b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74bbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15aef0 0x7f74bc0a8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74bbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74bc0a8c80 0x7f74bc0ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74bbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0cdbb0 con 0x7f74a0004d40 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74bbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0cda30 con 0x7f74bc0a8c80 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74bbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74bc0cdd30 con 0x7f74bc15aef0 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74bc135b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74bc135b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41338/0 (socket says 172.21.15.165:41338) 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 -- 172.21.15.165:0/370676080 learned_addr learned my addr 172.21.15.165:0/370676080 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:25.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 -- 172.21.15.165:0/370676080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15aef0 msgr2=0x7f74bc0a8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15aef0 0x7f74bc0a8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 -- 172.21.15.165:0/370676080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74bc0a8c80 msgr2=0x7f74bc0ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74bc0a8c80 0x7f74bc0ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.542+0000 7f74baa00640 1 -- 172.21.15.165:0/370676080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74bc0a92b0 con 0x7f74a0004d40 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.543+0000 7f74baa00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74bc135b10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0017e0 tx=0x7f74ac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.543+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac019070 con 0x7f74a0004d40 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.543+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74ac0074e0 con 0x7f74a0004d40 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.543+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0049e0 con 0x7f74a0004d40 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.543+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74bc0a9590 con 0x7f74a0004d40 2024-04-25T10:08:25.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.543+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74bc0a9b50 con 0x7f74a0004d40 2024-04-25T10:08:25.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.544+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74bc073990 con 0x7f74a0004d40 2024-04-25T10:08:25.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.545+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f74ac01c070 con 0x7f74a0004d40 2024-04-25T10:08:25.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.546+0000 7f74b2a00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f749806b4d0 0x7f749806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:25.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.546+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f74ac089e60 con 0x7f74a0004d40 2024-04-25T10:08:25.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.546+0000 7f74b3e00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f749806b4d0 0x7f749806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:25.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.547+0000 7f74b3e00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f749806b4d0 0x7f749806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f74bc138d30 tx=0x7f74a4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:25.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.550+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f74ac058490 con 0x7f74a0004d40 2024-04-25T10:08:25.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.783+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74bc15d480 con 0x7f74a0004d40 2024-04-25T10:08:25.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.784+0000 7f74b2a00640 1 -- 172.21.15.165:0/370676080 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f74ac05c530 con 0x7f74a0004d40 2024-04-25T10:08:25.790 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:08:25.791 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":56,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":56,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:08:19.374715+0000","modified":"2024-04-25T10:08:22.470688+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6833/1170361110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1170361110},{"type":"v1","addr":"172.21.15.172:6833","nonce":1170361110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:08:25.791 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 56 2024-04-25T10:08:25.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f749806b4d0 msgr2=0x7f749806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f749806b4d0 0x7f749806d990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f74bc138d30 tx=0x7f74a4009040 comp rx=0 tx=0).stop 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 msgr2=0x7f74bc135b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74bc135b10 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0017e0 tx=0x7f74ac00dd40 comp rx=0 tx=0).stop 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 shutdown_connections 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f749806b4d0 0x7f749806d990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74bc0a8c80 0x7f74bc0ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74bc15aef0 0x7f74bc0a8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 --2- 172.21.15.165:0/370676080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74a0004d40 0x7f74bc135b10 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.787+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 >> 172.21.15.165:0/370676080 conn(0x7f74bc06abe0 msgr2=0x7f74bc1433b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.788+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 shutdown_connections 2024-04-25T10:08:25.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:25.788+0000 7f74bbe00640 1 -- 172.21.15.165:0/370676080 wait complete. 2024-04-25T10:08:25.898 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:08:25.892+0000 7fbe29c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:08:26.214 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 56, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:08:19.374715+0000', 'modified': '2024-04-25T10:08:22.470688+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7116}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7116': {'gid': 7116, 'name': 'd', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6833/1170361110', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1170361110}, {'type': 'v1', 'addr': '172.21.15.172:6833', 'nonce': 1170361110}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:08:26.215 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:08:26.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:08:26.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.427+0000 7f9e6ee00640 1 Processor -- start 2024-04-25T10:08:26.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.428+0000 7f9e6ee00640 1 -- start start 2024-04-25T10:08:26.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.429+0000 7f9e6ee00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9e6806ea90 0x7f9e68071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.429+0000 7f9e6ee00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e68162ba0 con 0x7f9e68161fc0 2024-04-25T10:08:26.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.429+0000 7f9e6ee00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e68162e80 con 0x7f9e6806d2d0 2024-04-25T10:08:26.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.429+0000 7f9e6ee00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e68165be0 con 0x7f9e6806ea90 2024-04-25T10:08:26.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9e68161fc0 0x7f9e681623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35904/0 (socket says 172.21.15.165:35904) 2024-04-25T10:08:26.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67e00640 1 -- 172.21.15.165:0/468098196 learned_addr learned my addr 172.21.15.165:0/468098196 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:26.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e5f400640 1 --2- 172.21.15.165:0/468098196 >> v2:172.21.15.172:3300/0 conn(0x7f9e6806ea90 0x7f9e68071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3836544295 0 0) 0x7f9e68162ba0 con 0x7f9e68161fc0 2024-04-25T10:08:26.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e4c003520 con 0x7f9e68161fc0 2024-04-25T10:08:26.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3640081293 0 0) 0x7f9e4c003520 con 0x7f9e68161fc0 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 >> v2:172.21.15.172:3300/0 conn(0x7f9e6806ea90 msgr2=0x7f9e68071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 --2- 172.21.15.165:0/468098196 >> v2:172.21.15.172:3300/0 conn(0x7f9e6806ea90 0x7f9e68071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 >> v1:172.21.15.172:6789/0 conn(0x7f9e6806d2d0 legacy=0x7f9e68166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.430+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68163160 con 0x7f9e68161fc0 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9e58001670 con 0x7f9e68161fc0 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 >> v1:172.21.15.165:6789/0 conn(0x7f9e68161fc0 legacy=0x7f9e681623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c0039b0 0x7f9e4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e4c0051c0 0x7f9e4c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00c130 con 0x7f9e4c004800 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00c2e0 con 0x7f9e4c0051c0 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e4c00c4e0 con 0x7f9e4c0039b0 2024-04-25T10:08:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e5f400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e67e00640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c0039b0 0x7f9e4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.431+0000 7f9e5f400640 1 -- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c0039b0 msgr2=0x7f9e4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.432+0000 7f9e5f400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c0039b0 0x7f9e4c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.432+0000 7f9e5f400640 1 -- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e4c0051c0 msgr2=0x7f9e4c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.432+0000 7f9e5f400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e4c0051c0 0x7f9e4c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.432+0000 7f9e5f400640 1 -- 172.21.15.165:0/468098196 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e4c003520 con 0x7f9e4c004800 2024-04-25T10:08:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.432+0000 7f9e6ce00640 1 -- 172.21.15.165:0/468098196 reap_dead start 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.433+0000 7f9e5f400640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009ac0 tx=0x7f9e4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8983903612d5aa35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.433+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50036070 con 0x7f9e4c004800 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.433+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50031400 con 0x7f9e4c004800 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.433+0000 7f9e67400640 1 -- 172.21.15.165:0/468098196 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50009690 con 0x7f9e4c004800 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.433+0000 7f9e6ee00640 1 -- 172.21.15.165:0/468098196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c004800 msgr2=0x7f9e4c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.433+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009ac0 tx=0x7f9e4c005830 comp rx=0 tx=0).stop 2024-04-25T10:08:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.434+0000 7f9e6ee00640 1 -- 172.21.15.165:0/468098196 shutdown_connections 2024-04-25T10:08:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.434+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/468098196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e4c004800 0x7f9e4c004c80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.434+0000 7f9e6ee00640 1 -- 172.21.15.165:0/468098196 >> 172.21.15.165:0/468098196 conn(0x7f9e6806abe0 msgr2=0x7f9e6806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.434+0000 7f9e6ee00640 1 -- 172.21.15.165:0/468098196 shutdown_connections 2024-04-25T10:08:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.434+0000 7f9e6ee00640 1 -- 172.21.15.165:0/468098196 wait complete. 2024-04-25T10:08:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.435+0000 7f9e6ee00640 1 Processor -- start 2024-04-25T10:08:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 -- start start 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c004800 0x7f9e6815ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e680b0040 0x7f9e680b04c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 0x7f9e68160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6806f0c0 con 0x7f9e6815fef0 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6806ef40 con 0x7f9e680b0040 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.436+0000 7f9e6ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6806d700 con 0x7f9e4c004800 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 0x7f9e68160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 0x7f9e68160370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41350/0 (socket says 172.21.15.165:41350) 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 -- 172.21.15.165:0/970423148 learned_addr learned my addr 172.21.15.165:0/970423148 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e67e00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c004800 0x7f9e6815ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 -- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c004800 msgr2=0x7f9e6815ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c004800 0x7f9e6815ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e5f400640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e680b0040 0x7f9e680b04c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 -- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e680b0040 msgr2=0x7f9e680b04c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e680b0040 0x7f9e680b04c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 -- 172.21.15.165:0/970423148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68160c60 con 0x7f9e6815fef0 2024-04-25T10:08:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.437+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 0x7f9e68160370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9e680726e0 tx=0x7f9e6000c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.438+0000 7f9e64c00640 1 -- 172.21.15.165:0/970423148 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60017070 con 0x7f9e6815fef0 2024-04-25T10:08:26.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.438+0000 7f9e64c00640 1 -- 172.21.15.165:0/970423148 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e600070d0 con 0x7f9e6815fef0 2024-04-25T10:08:26.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.438+0000 7f9e64c00640 1 -- 172.21.15.165:0/970423148 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e600074c0 con 0x7f9e6815fef0 2024-04-25T10:08:26.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.438+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e68160f40 con 0x7f9e6815fef0 2024-04-25T10:08:26.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.438+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e68161500 con 0x7f9e6815fef0 2024-04-25T10:08:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.440+0000 7f9e64c00640 1 -- 172.21.15.165:0/970423148 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e6001a070 con 0x7f9e6815fef0 2024-04-25T10:08:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.441+0000 7f9e64c00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e4406b4d0 0x7f9e4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.441+0000 7f9e64c00640 1 -- 172.21.15.165:0/970423148 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e60089650 con 0x7f9e6815fef0 2024-04-25T10:08:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.441+0000 7f9e67e00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e4406b4d0 0x7f9e4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.442+0000 7f9e71bed740 1 Processor -- start 2024-04-25T10:08:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.442+0000 7f9e71bed740 1 -- start start 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.442+0000 7f9e67e00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e4406b4d0 0x7f9e4406d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e68068c30 tx=0x7f9e58009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89ab5950 0x55be89add790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89ab5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89afe5c0 0x55be89ab4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e71bed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be89a19620 con 0x55be89addcd0 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be899c2ca0 con 0x55be89afe5c0 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be89a156c0 con 0x55be89ab5950 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89ab5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89ab5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41352/0 (socket says 172.21.15.165:41352) 2024-04-25T10:08:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 -- 172.21.15.165:0/3924811921 learned_addr learned my addr 172.21.15.165:0/3924811921 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 -- 172.21.15.165:0/3924811921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89ab5950 msgr2=0x55be89add790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89ab5950 0x55be89add790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 -- 172.21.15.165:0/3924811921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89afe5c0 msgr2=0x55be89ab4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89afe5c0 0x55be89ab4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.443+0000 7f9e5f400640 1 -- 172.21.15.165:0/3924811921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be899c9f40 con 0x55be89addcd0 2024-04-25T10:08:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.444+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89ab5d30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9e50034040 tx=0x7f9e5002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4c3abb4e17007ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.444+0000 7f9e5cc00640 1 -- 172.21.15.165:0/3924811921 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50036040 con 0x55be89addcd0 2024-04-25T10:08:26.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.444+0000 7f9e5cc00640 1 -- 172.21.15.165:0/3924811921 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50042070 con 0x55be89addcd0 2024-04-25T10:08:26.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.444+0000 7f9e5cc00640 1 -- 172.21.15.165:0/3924811921 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e50003610 con 0x55be89addcd0 2024-04-25T10:08:26.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.444+0000 7f9e71bed740 1 -- 172.21.15.165:0/3924811921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 msgr2=0x55be89ab5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.444+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89ab5d30 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9e50034040 tx=0x7f9e5002e000 comp rx=0 tx=0).stop 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 -- 172.21.15.165:0/3924811921 shutdown_connections 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89afe5c0 0x55be89ab4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89ab5d30 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3924811921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89ab5950 0x55be89add790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 -- 172.21.15.165:0/3924811921 >> 172.21.15.165:0/3924811921 conn(0x55be89ab2b70 msgr2=0x55be89afb550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 -- 172.21.15.165:0/3924811921 shutdown_connections 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 -- 172.21.15.165:0/3924811921 wait complete. 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.445+0000 7f9e71bed740 1 -- start start 2024-04-25T10:08:26.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.446+0000 7f9e71bed740 1 Processor -- start 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.446+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.446+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89aff330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89afa520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e71bed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be89add300 con 0x55be89afe5c0 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be899b81f0 con 0x55be89ad9770 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be89adccf0 con 0x55be89addcd0 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e71bed740 10 client.0 _ll_register_callbacks cb 0x55be899d5ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89aff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89aff330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41356/0 (socket says 172.21.15.165:41356) 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 -- 172.21.15.165:0/3839770480 learned_addr learned my addr 172.21.15.165:0/3839770480 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e67e00640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 -- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 msgr2=0x55be89ad2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 -- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 msgr2=0x55be89afa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89afa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89afa520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.447+0000 7f9e5f400640 1 -- 172.21.15.165:0/3839770480 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be899d7c50 con 0x55be89afe5c0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89aff330 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009bb0 tx=0x7f9e500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9803150e84ace929 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e55600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e55600640 1 -- 172.21.15.165:0/3839770480 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5004a020 con 0x55be89afe5c0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e50042070 con 0x55be89afe5c0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e55600640 1 -- 172.21.15.165:0/3839770480 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e50005b90 con 0x55be89afe5c0 2024-04-25T10:08:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e55600640 1 -- 172.21.15.165:0/3839770480 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e50054050 con 0x55be89afe5c0 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e71bed740 10 client.7251 fetch_fsmap learned FSMap version 56 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55be8972e2d0 con 0x55be89afe5c0 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.448+0000 7f9e55600640 1 -- 172.21.15.165:0/3839770480 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50042070 con 0x55be89afe5c0 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e55600640 1 -- 172.21.15.165:0/3839770480 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e50057020 con 0x55be89afe5c0 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e71bed740 10 client.7251 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e71bed740 10 client.7251 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e71bed740 20 client.7251 populate_metadata read hostname 'smithi165' 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89b0f5d0 0x55be89adb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e71bed740 4 client.7251 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55be89a26b60 con 0x55be89b0f5d0 2024-04-25T10:08:26.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.449+0000 7f9e67e00640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89b0f5d0 0x55be89adb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.450+0000 7f9e67e00640 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89b0f5d0 0x55be89adb1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97884d228ce15a73 server_cookie=cc7c6ec0c8a7e762 in_seq=0 out_seq=0 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.450+0000 7f9e55600640 10 client.7251 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e55600640 1 -- 172.21.15.165:0/3839770480 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55be89a26b60 con 0x55be89b0f5d0 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e55600640 10 client.7251 handle_command_reply: tid=0 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 client.7251 shutdown 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 msgr2=0x55be89aff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89aff330 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009bb0 tx=0x7f9e500046c0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 shutdown_connections 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89b0f5d0 0x55be89adb1a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89afa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89aff330 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3839770480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e55600640 0 client.7251 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:08:26.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 >> 172.21.15.165:0/3839770480 conn(0x55be89ab5950 msgr2=0x55be89aaf8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 shutdown_connections 2024-04-25T10:08:26.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 1 -- 172.21.15.165:0/3839770480 wait complete. 2024-04-25T10:08:26.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.452+0000 7f9e71bed740 20 client.7251 trim_cache size 0 max 16384 2024-04-25T10:08:26.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.470+0000 7f9e71bed740 1 Processor -- start 2024-04-25T10:08:26.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.470+0000 7f9e71bed740 1 -- start start 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ab0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89afbe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89afe5c0 0x55be89ab1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e71bed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68204170 con 0x55be89addcd0 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e682035a0 con 0x55be89ad9770 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e68203ba0 con 0x55be89afe5c0 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89afbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e5f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89afbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41358/0 (socket says 172.21.15.165:41358) 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e5f400640 1 -- 172.21.15.165:0/3067812296 learned_addr learned my addr 172.21.15.165:0/3067812296 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:26.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e67e00640 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ab0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.471+0000 7f9e5f400640 1 -- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89afe5c0 msgr2=0x55be89ab1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89afe5c0 0x55be89ab1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e5f400640 1 -- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 msgr2=0x55be89ab0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ab0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e5f400640 1 -- 172.21.15.165:0/3067812296 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55be897f86b0 con 0x55be89addcd0 2024-04-25T10:08:26.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e5f400640 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89afbe40 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009ac0 tx=0x7f9e50047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a0e9aa2415a9953 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e57e00640 1 -- 172.21.15.165:0/3067812296 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5005d070 con 0x55be89addcd0 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e57e00640 1 -- 172.21.15.165:0/3067812296 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50042070 con 0x55be89addcd0 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.472+0000 7f9e57e00640 1 -- 172.21.15.165:0/3067812296 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5004a040 con 0x55be89addcd0 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 -- 172.21.15.165:0/3067812296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 msgr2=0x55be89afbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89afbe40 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009ac0 tx=0x7f9e50047000 comp rx=0 tx=0).stop 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 -- 172.21.15.165:0/3067812296 shutdown_connections 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89afe5c0 0x55be89ab1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89addcd0 0x55be89afbe40 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 --2- 172.21.15.165:0/3067812296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ab0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 -- 172.21.15.165:0/3067812296 >> 172.21.15.165:0/3067812296 conn(0x55be89ab5950 msgr2=0x55be89ad4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 -- 172.21.15.165:0/3067812296 shutdown_connections 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.473+0000 7f9e71bed740 1 -- 172.21.15.165:0/3067812296 wait complete. 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.474+0000 7f9e71bed740 1 -- start start 2024-04-25T10:08:26.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.474+0000 7f9e71bed740 1 Processor -- start 2024-04-25T10:08:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ae42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89ad1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be89add300 con 0x55be89afe5c0 2024-04-25T10:08:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55be899b81f0 con 0x55be89ad9770 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55be89b0ea20 con 0x55be89addcd0 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e71bed740 10 client.0 _ll_register_callbacks cb 0x7f9e4c0036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89ad1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89ad1df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41366/0 (socket says 172.21.15.165:41366) 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 -- 172.21.15.165:0/2857320104 learned_addr learned my addr 172.21.15.165:0/2857320104 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e5f400640 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 -- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 msgr2=0x55be89ad18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 -- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 msgr2=0x55be89ae42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ae42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.475+0000 7f9e6ce00640 1 -- 172.21.15.165:0/2857320104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55be89ad7df0 con 0x55be89afe5c0 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e6ce00640 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89ad1df0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9e600070a0 tx=0x7f9e60058470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69cbaca84e3b9553 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e55600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e55600640 1 -- 172.21.15.165:0/2857320104 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e60057280 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e55600640 1 -- 172.21.15.165:0/2857320104 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9e60007660 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e50042070 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e55600640 1 -- 172.21.15.165:0/2857320104 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e60007860 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 10 client.7257 fetch_fsmap learned FSMap version 56 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=2+}) v3 -- 0x55be89ad83b0 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e55600640 1 -- 172.21.15.165:0/2857320104 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e50042070 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e55600640 1 -- 172.21.15.165:0/2857320104 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 56) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9e60053980 con 0x55be89afe5c0 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 10 client.7257 fetch_fsmap finished waiting for FSMap version 56 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 10 client.7257 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 20 client.7257 populate_metadata read hostname 'smithi165' 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.476+0000 7f9e71bed740 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89ac6e10 0x55be89adb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.477+0000 7f9e71bed740 4 client.7257 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:08:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.477+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55be89a20eb0 con 0x55be89ac6e10 2024-04-25T10:08:26.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.477+0000 7f9e67e00640 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89ac6e10 0x55be89adb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.478+0000 7f9e67e00640 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89ac6e10 0x55be89adb1a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf5e9bae5b06b124 server_cookie=611e766da969f004 in_seq=0 out_seq=0 2024-04-25T10:08:26.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.478+0000 7f9e55600640 10 client.7257 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:08:26.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e55600640 1 -- 172.21.15.165:0/2857320104 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55be89a20eb0 con 0x55be89ac6e10 2024-04-25T10:08:26.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e55600640 10 client.7257 handle_command_reply: tid=0 2024-04-25T10:08:26.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 client.7257 shutdown 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 msgr2=0x55be89ad1df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89ad1df0 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f9e600070a0 tx=0x7f9e60058470 comp rx=0 tx=0).stop 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 shutdown_connections 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55be89afe5c0 0x55be89ad1df0 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55be89addcd0 0x55be89ad18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55be89ac6e10 0x55be89adb1a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 --2- 172.21.15.165:0/2857320104 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55be89ad9770 0x55be89ae42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e55600640 0 client.7257 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 >> 172.21.15.165:0/2857320104 conn(0x55be89ab5950 msgr2=0x55be89b1df10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 shutdown_connections 2024-04-25T10:08:26.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 1 -- 172.21.15.165:0/2857320104 wait complete. 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.479+0000 7f9e71bed740 20 client.7257 trim_cache size 0 max 16384 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 7116, 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:08:26.492 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.5183085780000001, 2024-04-25T10:08:26.493 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 56, 2024-04-25T10:08:26.493 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 85, 2024-04-25T10:08:26.493 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 85, 2024-04-25T10:08:26.493 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 12.012168234000001 2024-04-25T10:08:26.493 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e4406b4d0 msgr2=0x7f9e4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e4406b4d0 0x7f9e4406d990 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f9e68068c30 tx=0x7f9e58009040 comp rx=0 tx=0).stop 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 msgr2=0x7f9e68160370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 0x7f9e68160370 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9e680726e0 tx=0x7f9e6000c980 comp rx=0 tx=0).stop 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 shutdown_connections 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e4406b4d0 0x7f9e4406d990 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e6815fef0 0x7f9e68160370 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e680b0040 0x7f9e680b04c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 --2- 172.21.15.165:0/970423148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e4c004800 0x7f9e6815ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 >> 172.21.15.165:0/970423148 conn(0x7f9e6806abe0 msgr2=0x7f9e6815d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 shutdown_connections 2024-04-25T10:08:26.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.480+0000 7f9e6ee00640 1 -- 172.21.15.165:0/970423148 wait complete. 2024-04-25T10:08:26.502 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:08:26.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.687+0000 7fe5ef200640 1 Processor -- start 2024-04-25T10:08:26.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.688+0000 7fe5ef200640 1 -- start start 2024-04-25T10:08:26.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.689+0000 7fe5ef200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe5f00cca10 0x7fe5f0165300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.689+0000 7fe5ef200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f015bb50 con 0x7fe5f0143e60 2024-04-25T10:08:26.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.689+0000 7fe5ef200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f0144240 con 0x7fe5f006e870 2024-04-25T10:08:26.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.689+0000 7fe5ef200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f015e220 con 0x7fe5f00cca10 2024-04-25T10:08:26.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.689+0000 7fe5ede00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe5f0143e60 0x7fe5f006e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35920/0 (socket says 172.21.15.165:35920) 2024-04-25T10:08:26.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.689+0000 7fe5ede00640 1 -- 172.21.15.165:0/4221026241 learned_addr learned my addr 172.21.15.165:0/4221026241 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:26.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3375622192 0 0) 0x7fe5e0001110 con 0x7fe5f0143e60 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d4003520 con 0x7fe5f0143e60 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ee800640 1 --2- 172.21.15.165:0/4221026241 >> v2:172.21.15.172:3300/0 conn(0x7fe5f00cca10 0x7fe5f0165300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 619060175 0 0) 0x7fe5d4003520 con 0x7fe5f0143e60 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 >> v2:172.21.15.172:3300/0 conn(0x7fe5f00cca10 msgr2=0x7fe5f0165300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 --2- 172.21.15.165:0/4221026241 >> v2:172.21.15.172:3300/0 conn(0x7fe5f00cca10 0x7fe5f0165300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 >> v1:172.21.15.172:6789/0 conn(0x7fe5f006e870 legacy=0x7fe5f00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f015e500 con 0x7fe5f0143e60 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe5e0001840 con 0x7fe5f0143e60 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 >> v1:172.21.15.165:6789/0 conn(0x7fe5f0143e60 legacy=0x7fe5f006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d4004800 0x7fe5d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d40052e0 0x7fe5d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f015e220 con 0x7fe5d40039b0 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d400c1e0 con 0x7fe5d4004800 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d400c3e0 con 0x7fe5d40052e0 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.690+0000 7fe5ee800640 1 --2- 172.21.15.165:0/4221026241 >> v2:172.21.15.172:3300/0 conn(0x7fe5f00cca10 0x7fe5f0165300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ede00640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5e5400640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d4004800 0x7fe5d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ede00640 1 -- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d40052e0 msgr2=0x7fe5d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ede00640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5d40052e0 0x7fe5d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ede00640 1 -- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d4004800 msgr2=0x7fe5d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ede00640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d4004800 0x7fe5d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ede00640 1 -- 172.21.15.165:0/4221026241 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4003520 con 0x7fe5d40039b0 2024-04-25T10:08:26.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5ee800640 1 -- 172.21.15.165:0/4221026241 reap_dead start 2024-04-25T10:08:26.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.691+0000 7fe5e5400640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5d4004800 0x7fe5d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:08:26.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.692+0000 7fe5ede00640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5d4003dd0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c4c0 tx=0x7fe5d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752d0f4beddc1068 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.692+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e000c0e0 con 0x7fe5d40039b0 2024-04-25T10:08:26.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.692+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e0034070 con 0x7fe5d40039b0 2024-04-25T10:08:26.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.692+0000 7fe5ed400640 1 -- 172.21.15.165:0/4221026241 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0005d70 con 0x7fe5d40039b0 2024-04-25T10:08:26.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 -- 172.21.15.165:0/4221026241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 msgr2=0x7fe5d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5d4003dd0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c4c0 tx=0x7fe5d4005830 comp rx=0 tx=0).stop 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 -- 172.21.15.165:0/4221026241 shutdown_connections 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 --2- 172.21.15.165:0/4221026241 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5d4003dd0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 -- 172.21.15.165:0/4221026241 >> 172.21.15.165:0/4221026241 conn(0x7fe5f006abe0 msgr2=0x7fe5f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 -- 172.21.15.165:0/4221026241 shutdown_connections 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.693+0000 7fe5ef200640 1 -- 172.21.15.165:0/4221026241 wait complete. 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.695+0000 7fe5ef200640 1 Processor -- start 2024-04-25T10:08:26.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.695+0000 7fe5ef200640 1 -- start start 2024-04-25T10:08:26.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5f0159c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f0159620 0x7fe5f0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f0071530 0x7fe5f00c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ef200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00b2d30 con 0x7fe5d40039b0 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f015eac0 con 0x7fe5f0071530 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f00b2eb0 con 0x7fe5f0159620 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5f0159c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5f0159c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41386/0 (socket says 172.21.15.165:41386) 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.696+0000 7fe5ede00640 1 -- 172.21.15.165:0/3296940851 learned_addr learned my addr 172.21.15.165:0/3296940851 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ee800640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f0071530 0x7fe5f00c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5e5400640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f0159620 0x7fe5f0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ede00640 1 -- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f0159620 msgr2=0x7fe5f0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ede00640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f0159620 0x7fe5f0070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ede00640 1 -- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f0071530 msgr2=0x7fe5f00c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ede00640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f0071530 0x7fe5f00c5c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ede00640 1 -- 172.21.15.165:0/3296940851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f015a200 con 0x7fe5d40039b0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.697+0000 7fe5ede00640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5f0159c20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c5e0 tx=0x7fe5f0144410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.698+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0007270 con 0x7fe5d40039b0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.698+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e0007410 con 0x7fe5d40039b0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.698+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f00c4e60 con 0x7fe5d40039b0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.698+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e0001110 con 0x7fe5d40039b0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.698+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f00c5420 con 0x7fe5d40039b0 2024-04-25T10:08:26.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.699+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f0159a00 con 0x7fe5d40039b0 2024-04-25T10:08:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.700+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5e002f460 con 0x7fe5d40039b0 2024-04-25T10:08:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.701+0000 7fe5e6a00640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5cc06b500 0x7fe5cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.701+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5e00aaa90 con 0x7fe5d40039b0 2024-04-25T10:08:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.701+0000 7fe5e5400640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5cc06b500 0x7fe5cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.701+0000 7fe5e5400640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5cc06b500 0x7fe5cc06d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0071f70 tx=0x7fe5d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:26.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.705+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5e0040070 con 0x7fe5d40039b0 2024-04-25T10:08:26.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.924+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5f0073990 con 0x7fe5d40039b0 2024-04-25T10:08:26.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.925+0000 7fe5e6a00640 1 -- 172.21.15.165:0/3296940851 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v56) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5e00aa020 con 0x7fe5d40039b0 2024-04-25T10:08:26.931 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:08:26.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5cc06b500 msgr2=0x7fe5cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5cc06b500 0x7fe5cc06d9c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fe5f0071f70 tx=0x7fe5d8009040 comp rx=0 tx=0).stop 2024-04-25T10:08:26.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 msgr2=0x7fe5f0159c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:26.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5f0159c20 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fe5e000c5e0 tx=0x7fe5f0144410 comp rx=0 tx=0).stop 2024-04-25T10:08:26.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 shutdown_connections 2024-04-25T10:08:26.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5cc06b500 0x7fe5cc06d9c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f0071530 0x7fe5f00c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f0159620 0x7fe5f0070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 --2- 172.21.15.165:0/3296940851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5d40039b0 0x7fe5f0159c20 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:26.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 >> 172.21.15.165:0/3296940851 conn(0x7fe5f006abe0 msgr2=0x7fe5f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:26.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 shutdown_connections 2024-04-25T10:08:26.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:26.928+0000 7fe5ef200640 1 -- 172.21.15.165:0/3296940851 wait complete. 2024-04-25T10:08:26.953 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:08:26.953 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:08:26.954 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:26.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:26.954 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:08:27.002 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:08:27.002 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:27.003 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42258sec preferred_lft 42258sec 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:08:27.004 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:08:27.005 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:08:27.006 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.006 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:08:27.127 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:08:27.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.131 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:08:27.181 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:08:27.181 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:08:27.181 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:08:27.182 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:08:27.182 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:08:27.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.182 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:08:27.182 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:08:27.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:08:27.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:08:27.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:08:27.182 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:08:27.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.371 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:08:27.418 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:08:27.418 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:08:27.419 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:27.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.419 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:08:27.497 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:27.497 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:27.549 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:08:27.549 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:08:27.549 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:27.596 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:08:27.596 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.596 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:27.763 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:27.764 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:27.764 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:08:27.785 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:27.785 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:27.785 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:08:27.786 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:08:27.786 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:08:27.814 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:08:27.817 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:08:27.817 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:08:27.817 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:08:27.817 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:08:27.817 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:08:27.822 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:08:27.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.822 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:08:27.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:27.892 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:08:27.965 INFO:teuthology.orchestra.run.smithi165.stdout:client7266 2024-04-25T10:08:27.965 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:08:27.965 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:28.017 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_new_snaps true --yes-i-really-mean-it 2024-04-25T10:08:28.053 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:28.072 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:28.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.212+0000 7fb3dd400640 1 Processor -- start 2024-04-25T10:08:28.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.213+0000 7fb3dd400640 1 -- start start 2024-04-25T10:08:28.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.214+0000 7fb3dd400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb3d806abe0 0x7fb3d806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.214+0000 7fb3dd400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d806b500 con 0x7fb3d815dd80 2024-04-25T10:08:28.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.214+0000 7fb3dd400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3d80b3930 con 0x7fb3d80ad570 2024-04-25T10:08:28.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.214+0000 7fb3dd400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d806e410 con 0x7fb3d806abe0 2024-04-25T10:08:28.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.215+0000 7fb3cfe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb3d815dd80 0x7fb3d80b5290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41464/0 (socket says 172.21.15.165:41464) 2024-04-25T10:08:28.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.215+0000 7fb3cfe00640 1 -- 172.21.15.165:0/628125026 learned_addr learned my addr 172.21.15.165:0/628125026 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:08:28.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.215+0000 7fb3d6800640 1 --2- 172.21.15.165:0/628125026 >> v2:172.21.15.172:3300/0 conn(0x7fb3d806abe0 0x7fb3d806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.215+0000 7fb3d5e00640 1 -- 172.21.15.165:0/628125026 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268734309 0 0) 0x7fb3d806b500 con 0x7fb3d815dd80 2024-04-25T10:08:28.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.215+0000 7fb3d5e00640 1 -- 172.21.15.165:0/628125026 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3bc003520 con 0x7fb3d815dd80 2024-04-25T10:08:28.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.216+0000 7fb3d5e00640 1 -- 172.21.15.165:0/628125026 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2442129837 0 0) 0x7fb3bc003520 con 0x7fb3d815dd80 2024-04-25T10:08:28.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.216+0000 7fb3d6800640 1 -- 172.21.15.165:0/628125026 >> v1:172.21.15.172:6789/0 conn(0x7fb3d80ad570 legacy=0x7fb3d806ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.216+0000 7fb3d6800640 1 -- 172.21.15.165:0/628125026 >> v1:172.21.15.165:6789/0 conn(0x7fb3d815dd80 legacy=0x7fb3d80b5290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.216+0000 7fb3d6800640 1 -- 172.21.15.165:0/628125026 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80b3c10 con 0x7fb3d806abe0 2024-04-25T10:08:28.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.217+0000 7fb3d6800640 1 --2- 172.21.15.165:0/628125026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d806afc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80097a0 tx=0x7fb3c802a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ed10cc861fab9e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:28.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.217+0000 7fb3d5e00640 1 -- 172.21.15.165:0/628125026 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c8037070 con 0x7fb3d806abe0 2024-04-25T10:08:28.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.217+0000 7fb3d5e00640 1 -- 172.21.15.165:0/628125026 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c802ac50 con 0x7fb3d806abe0 2024-04-25T10:08:28.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.217+0000 7fb3d5e00640 1 -- 172.21.15.165:0/628125026 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0001670 con 0x7fb3d806abe0 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 -- 172.21.15.165:0/628125026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 msgr2=0x7fb3d806afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 --2- 172.21.15.165:0/628125026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d806afc0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80097a0 tx=0x7fb3c802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 -- 172.21.15.165:0/628125026 shutdown_connections 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 --2- 172.21.15.165:0/628125026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d806afc0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 -- 172.21.15.165:0/628125026 >> 172.21.15.165:0/628125026 conn(0x7fb3d80b5af0 msgr2=0x7fb3d80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 -- 172.21.15.165:0/628125026 shutdown_connections 2024-04-25T10:08:28.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.218+0000 7fb3dd400640 1 -- 172.21.15.165:0/628125026 wait complete. 2024-04-25T10:08:28.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.220+0000 7fb3dd400640 1 Processor -- start 2024-04-25T10:08:28.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.220+0000 7fb3dd400640 1 -- start start 2024-04-25T10:08:28.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3dd400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d8154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3dd400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3d80ad570 0x7fb3d8152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3dd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 0x7fb3d8156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3dd400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8070280 con 0x7fb3d815dd80 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3dd400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d806e940 con 0x7fb3d806abe0 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3dd400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d8070400 con 0x7fb3d80ad570 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3d7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 0x7fb3d8156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3d7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 0x7fb3d8156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33376/0 (socket says 172.21.15.165:33376) 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.221+0000 7fb3d7200640 1 -- 172.21.15.165:0/2824829609 learned_addr learned my addr 172.21.15.165:0/2824829609 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d6800640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d8154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3cfe00640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3d80ad570 0x7fb3d8152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d7200640 1 -- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3d80ad570 msgr2=0x7fb3d8152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d7200640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3d80ad570 0x7fb3d8152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d7200640 1 -- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 msgr2=0x7fb3d8154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d7200640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d8154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d7200640 1 -- 172.21.15.165:0/2824829609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d8155370 con 0x7fb3d815dd80 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3d7200640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 0x7fb3d8156410 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c7a0 tx=0x7fb3d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0003b70 con 0x7fb3d815dd80 2024-04-25T10:08:28.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0003d30 con 0x7fb3d815dd80 2024-04-25T10:08:28.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.222+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d000aa90 con 0x7fb3d815dd80 2024-04-25T10:08:28.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.223+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d8155650 con 0x7fb3d815dd80 2024-04-25T10:08:28.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.223+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d8155c10 con 0x7fb3d815dd80 2024-04-25T10:08:28.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.224+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3d8156100 con 0x7fb3d815dd80 2024-04-25T10:08:28.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.225+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3d000ac30 con 0x7fb3d815dd80 2024-04-25T10:08:28.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.226+0000 7fb3cea00640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3b406b4d0 0x7fb3b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:08:28.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.226+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb3d008a490 con 0x7fb3d815dd80 2024-04-25T10:08:28.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.226+0000 7fb3d6800640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3b406b4d0 0x7fb3b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:08:28.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.227+0000 7fb3d6800640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3b406b4d0 0x7fb3b406d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8034000 tx=0x7fb3c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:08:28.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.230+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3d0058fd0 con 0x7fb3d815dd80 2024-04-25T10:08:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:28.472+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb3d8073a00 con 0x7fb3d815dd80 2024-04-25T10:08:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.332+0000 7fb3cea00640 1 -- 172.21.15.165:0/2824829609 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_new_snaps", "val": "true", "yes_i_really_mean_it": true}]=0 enabled new snapshots v57) v1 ==== 167+0+0 (secure 0 0 0) 0x7fb3d00588d0 con 0x7fb3d815dd80 2024-04-25T10:08:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:enabled new snapshots 2024-04-25T10:08:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3b406b4d0 msgr2=0x7fb3b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3b406b4d0 0x7fb3b406d990 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fb3c8034000 tx=0x7fb3c802e000 comp rx=0 tx=0).stop 2024-04-25T10:08:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 msgr2=0x7fb3d8156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:08:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 0x7fb3d8156410 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c7a0 tx=0x7fb3d000cc60 comp rx=0 tx=0).stop 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 shutdown_connections 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3b406b4d0 0x7fb3b406d990 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3d815dd80 0x7fb3d8156410 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3d80ad570 0x7fb3d8152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 --2- 172.21.15.165:0/2824829609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3d806abe0 0x7fb3d8154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.336+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 >> 172.21.15.165:0/2824829609 conn(0x7fb3d80b5af0 msgr2=0x7fb3d80b1420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.337+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 shutdown_connections 2024-04-25T10:08:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:08:29.337+0000 7fb3dd400640 1 -- 172.21.15.165:0/2824829609 wait complete. 2024-04-25T10:08:29.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:08:29.362 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir stray_eval_dir) 2024-04-25T10:08:29.424 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:08:29.424 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:08:29.473 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:08:29.473 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:08:29.473 DEBUG:teuthology.orchestra.run.smithi165:> import uuid 2024-04-25T10:08:29.473 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:08:29.473 DEBUG:teuthology.orchestra.run.smithi165:> n = 2000 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> create_hard_links = False 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> if 3 > 0: 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> create_hard_links = True 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> path = "/home/ubuntu/cephtest/mnt.0/stray_eval_dir/file" 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> dpath = os.path.dirname(path) 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> fnameprefix = os.path.basename(path) 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> for i in range(n): 2024-04-25T10:08:29.474 DEBUG:teuthology.orchestra.run.smithi165:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> f.write(f"{i}") 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> f.flush() 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(f.fileno()) 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> os.unlink(fpath) 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(dirfd) 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> if create_hard_links: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> for j in range(3): 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(dirfd) 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> finally: 2024-04-25T10:08:29.475 DEBUG:teuthology.orchestra.run.smithi165:> os.close(dirfd) 2024-04-25T10:08:29.476 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:08:58.199 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:08:58.208 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:26.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:09:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:09:26.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:26.898 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:09:27.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:09:27.000 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec find /home/ubuntu/cephtest/mnt.0/stray_eval_dir/ -type f -delete) 2024-04-25T10:09:28.321 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:28.330 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:58.455 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:09:58.464 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:28.717 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:28.727 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:30.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:30.887 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir /home/ubuntu/cephtest/mnt.0/stray_eval_dir/.snap/snap1-stray_eval_dir) 2024-04-25T10:10:31.060 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:10:31.060 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:10:31.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.264+0000 7fcaaa200640 1 Processor -- start 2024-04-25T10:10:31.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.264+0000 7fcaaa200640 1 -- start start 2024-04-25T10:10:31.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaaa200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcaa415dbb0 0x7fcaa406fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaaa200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcaa40cbf00 0x7fcaa406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaaa200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcaa40cae60 con 0x7fcaa40cc390 2024-04-25T10:10:31.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaaa200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40cb140 con 0x7fcaa415dbb0 2024-04-25T10:10:31.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaaa200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcaa40cb420 con 0x7fcaa40cbf00 2024-04-25T10:10:31.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaa3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcaa415dbb0 0x7fcaa406fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaa3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcaa415dbb0 0x7fcaa406fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38382/0 (socket says 172.21.15.165:38382) 2024-04-25T10:10:31.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaa3400640 1 -- 172.21.15.165:0/2936563354 learned_addr learned my addr 172.21.15.165:0/2936563354 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaa3e00640 1 --1- 172.21.15.165:0/2936563354 >> v1:172.21.15.165:6789/0 conn(0x7fcaa40cc390 0x7fcaa406db50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35864/0 (socket says 172.21.15.165:35864) 2024-04-25T10:10:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.266+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/2936563354 >> v2:172.21.15.172:3300/0 conn(0x7fcaa40cbf00 0x7fcaa406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.267+0000 7fcaa3400640 1 -- 172.21.15.165:0/2936563354 >> v2:172.21.15.172:3300/0 conn(0x7fcaa40cbf00 msgr2=0x7fcaa406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.267+0000 7fcaa3400640 1 --2- 172.21.15.165:0/2936563354 >> v2:172.21.15.172:3300/0 conn(0x7fcaa40cbf00 0x7fcaa406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.267+0000 7fcaa3400640 1 -- 172.21.15.165:0/2936563354 >> v1:172.21.15.165:6789/0 conn(0x7fcaa40cc390 legacy=0x7fcaa406db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.267+0000 7fcaa3400640 1 -- 172.21.15.165:0/2936563354 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40cb700 con 0x7fcaa415dbb0 2024-04-25T10:10:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.267+0000 7fcaa2000640 1 -- 172.21.15.165:0/2936563354 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910048309 0 0) 0x7fcaa40cae60 con 0x7fcaa40cc390 2024-04-25T10:10:31.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.267+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/2936563354 >> v2:172.21.15.172:3300/0 conn(0x7fcaa40cbf00 0x7fcaa406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:31.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.268+0000 7fcaa3400640 1 --2- 172.21.15.165:0/2936563354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa415dbb0 0x7fcaa406fea0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406d420 tx=0x7fca9402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae972b5bd2ea7de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.268+0000 7fcaa2000640 1 -- 172.21.15.165:0/2936563354 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94037070 con 0x7fcaa415dbb0 2024-04-25T10:10:31.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.268+0000 7fcaa2000640 1 -- 172.21.15.165:0/2936563354 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca94032440 con 0x7fcaa415dbb0 2024-04-25T10:10:31.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.268+0000 7fcaa2000640 1 -- 172.21.15.165:0/2936563354 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca94032810 con 0x7fcaa415dbb0 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 -- 172.21.15.165:0/2936563354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa415dbb0 msgr2=0x7fcaa406fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 --2- 172.21.15.165:0/2936563354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa415dbb0 0x7fcaa406fea0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fcaa406d420 tx=0x7fca9402a270 comp rx=0 tx=0).stop 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 -- 172.21.15.165:0/2936563354 shutdown_connections 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 --2- 172.21.15.165:0/2936563354 >> v2:172.21.15.172:3300/0 conn(0x7fcaa40cbf00 0x7fcaa406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 --2- 172.21.15.165:0/2936563354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa415dbb0 0x7fcaa406fea0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 -- 172.21.15.165:0/2936563354 >> 172.21.15.165:0/2936563354 conn(0x7fcaa4071af0 msgr2=0x7fcaa4071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 -- 172.21.15.165:0/2936563354 shutdown_connections 2024-04-25T10:10:31.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.269+0000 7fcaaa200640 1 -- 172.21.15.165:0/2936563354 wait complete. 2024-04-25T10:10:31.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.271+0000 7fcaaa200640 1 Processor -- start 2024-04-25T10:10:31.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.271+0000 7fcaaa200640 1 -- start start 2024-04-25T10:10:31.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaaa200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cbf00 0x7fcaa40c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaaa200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 0x7fcaa40b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaaa200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa415dbb0 0x7fcaa40c9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaaa200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40cab10 con 0x7fcaa40cc390 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaaa200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa415df90 con 0x7fcaa415dbb0 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaaa200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa40703e0 con 0x7fcaa40cbf00 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 0x7fcaa40b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 0x7fcaa40b52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38398/0 (socket says 172.21.15.165:38398) 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1580449868 learned_addr learned my addr 172.21.15.165:0/1580449868 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa415dbb0 0x7fcaa40c9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cbf00 msgr2=0x7fcaa40c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cbf00 0x7fcaa40c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cbf00 0x7fcaa40c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa415dbb0 msgr2=0x7fcaa40c9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa415dbb0 0x7fcaa40c9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.272+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1580449868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40c8c60 con 0x7fcaa40cc390 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.273+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 0x7fcaa40b52a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c7f0 tx=0x7fca9000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.273+0000 7fca9b400640 1 -- 172.21.15.165:0/1580449868 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90015070 con 0x7fcaa40cc390 2024-04-25T10:10:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.273+0000 7fca9b400640 1 -- 172.21.15.165:0/1580449868 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca90007820 con 0x7fcaa40cc390 2024-04-25T10:10:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.273+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcaa40c8f40 con 0x7fcaa40cc390 2024-04-25T10:10:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.273+0000 7fca9b400640 1 -- 172.21.15.165:0/1580449868 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90007bd0 con 0x7fcaa40cc390 2024-04-25T10:10:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.273+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcaa40c97e0 con 0x7fcaa40cc390 2024-04-25T10:10:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.276+0000 7fca9b400640 1 -- 172.21.15.165:0/1580449868 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca90018070 con 0x7fcaa40cc390 2024-04-25T10:10:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.277+0000 7fca9b400640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca8406b500 0x7fca8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.277+0000 7fca9b400640 1 -- 172.21.15.165:0/1580449868 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca9008a410 con 0x7fcaa40cc390 2024-04-25T10:10:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.277+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca8406b500 0x7fca8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.278+0000 7fcaacff3740 1 Processor -- start 2024-04-25T10:10:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.278+0000 7fcaacff3740 1 -- start start 2024-04-25T10:10:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.278+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca8406b500 0x7fca8406d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fca94034000 tx=0x7fca9402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1044a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10450b0 0x5640e1014890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 0x5640e103cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaacff3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e0f7d870 con 0x5640e1014dd0 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e0f26d60 con 0x5640e10450b0 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e0f79910 con 0x5640e1044690 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 0x5640e103cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 0x5640e103cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38400/0 (socket says 172.21.15.165:38400) 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 -- 172.21.15.165:0/1187562349 learned_addr learned my addr 172.21.15.165:0/1187562349 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10450b0 0x5640e1014890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 -- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 msgr2=0x5640e1044a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1044a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1044a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 -- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10450b0 msgr2=0x5640e1014890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10450b0 0x5640e1014890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa3e00640 1 -- 172.21.15.165:0/1187562349 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e0d5c7d0 con 0x5640e1014dd0 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.279+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10450b0 0x5640e1014890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:31.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.280+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 0x5640e103cb10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fca9c001dd0 tx=0x7fca9c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d8cf41d0bc089b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.280+0000 7fca8f400640 1 -- 172.21.15.165:0/1187562349 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c00de30 con 0x5640e1014dd0 2024-04-25T10:10:31.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.280+0000 7fca8f400640 1 -- 172.21.15.165:0/1187562349 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c004b60 con 0x5640e1014dd0 2024-04-25T10:10:31.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.280+0000 7fca8f400640 1 -- 172.21.15.165:0/1187562349 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c0070d0 con 0x5640e1014dd0 2024-04-25T10:10:31.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 -- 172.21.15.165:0/1187562349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 msgr2=0x5640e103cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 0x5640e103cb10 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7fca9c001dd0 tx=0x7fca9c00d520 comp rx=0 tx=0).stop 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 -- 172.21.15.165:0/1187562349 shutdown_connections 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1014dd0 0x5640e103cb10 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10450b0 0x5640e1014890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1187562349 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1044a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 -- 172.21.15.165:0/1187562349 >> 172.21.15.165:0/1187562349 conn(0x5640e1016d10 msgr2=0x5640e105f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 -- 172.21.15.165:0/1187562349 shutdown_connections 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.281+0000 7fcaacff3740 1 -- 172.21.15.165:0/1187562349 wait complete. 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.282+0000 7fcaacff3740 1 -- start start 2024-04-25T10:10:31.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.282+0000 7fcaacff3740 1 Processor -- start 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1044690 0x5640e103a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e10450b0 0x5640e1067630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1047040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e1040de0 con 0x5640e1047a90 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e0f3a1c0 con 0x5640e1044690 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e1041080 con 0x5640e10450b0 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaacff3740 10 client.0 _ll_register_callbacks cb 0x5640e0f788f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1047040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1047040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38402/0 (socket says 172.21.15.165:38402) 2024-04-25T10:10:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa3e00640 1 -- 172.21.15.165:0/2214435647 learned_addr learned my addr 172.21.15.165:0/2214435647 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa3400640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1044690 0x5640e103a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e10450b0 0x5640e1067630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa3e00640 1 -- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e10450b0 msgr2=0x5640e1067630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.283+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e10450b0 0x5640e1067630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fcaa3e00640 1 -- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1044690 msgr2=0x5640e103a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1044690 0x5640e103a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fcaa3e00640 1 -- 172.21.15.165:0/2214435647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640e0c520f0 con 0x5640e1047a90 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fcaa3e00640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1047040 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000f80 tx=0x7fca9c001480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72d96c9275a305e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fca83e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fca83e00640 1 -- 172.21.15.165:0/2214435647 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9c023020 con 0x5640e1047a90 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fca83e00640 1 -- 172.21.15.165:0/2214435647 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca9c0072f0 con 0x5640e1047a90 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.284+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca9c004b60 con 0x5640e1047a90 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fca83e00640 1 -- 172.21.15.165:0/2214435647 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca9c0074f0 con 0x5640e1047a90 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 10 client.7287 fetch_fsmap learned FSMap version 57 2024-04-25T10:10:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5640e0dbcf60 con 0x5640e1047a90 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fca83e00640 1 -- 172.21.15.165:0/2214435647 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca9c004b60 con 0x5640e1047a90 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fca83e00640 1 -- 172.21.15.165:0/2214435647 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca9c007690 con 0x5640e1047a90 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 10 client.7287 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 10 client.7287 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 20 client.7287 populate_metadata read hostname 'smithi165' 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e10435d0 0x5640e10454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 4 client.7287 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.285+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640e0f53690 con 0x5640e10435d0 2024-04-25T10:10:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.286+0000 7fcaa3400640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e10435d0 0x5640e10454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.287+0000 7fcaa3400640 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e10435d0 0x5640e10454b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc00ed3811b35dd server_cookie=140990c6afda04bb in_seq=0 out_seq=0 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.287+0000 7fca83e00640 10 client.7287 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fca83e00640 1 -- 172.21.15.165:0/2214435647 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5640e0f53690 con 0x5640e10435d0 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fca83e00640 10 client.7287 handle_command_reply: tid=0 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 client.7287 shutdown 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 msgr2=0x5640e1047040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1047040 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fca9c000f80 tx=0x7fca9c001480 comp rx=0 tx=0).stop 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 shutdown_connections 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1047040 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e10450b0 0x5640e1067630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.288+0000 7fcaacff3740 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e10435d0 0x5640e10454b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.289+0000 7fcaacff3740 1 --2- 172.21.15.165:0/2214435647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1044690 0x5640e103a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.289+0000 7fca83e00640 0 client.7287 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.289+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 >> 172.21.15.165:0/2214435647 conn(0x5640e1014dd0 msgr2=0x5640e1063240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.289+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 shutdown_connections 2024-04-25T10:10:31.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.289+0000 7fcaacff3740 1 -- 172.21.15.165:0/2214435647 wait complete. 2024-04-25T10:10:31.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.289+0000 7fcaacff3740 20 client.7287 trim_cache size 0 max 16384 2024-04-25T10:10:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.304+0000 7fcaacff3740 1 Processor -- start 2024-04-25T10:10:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.304+0000 7fcaacff3740 1 -- start start 2024-04-25T10:10:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e101a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 0x5640e1063f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1047a90 0x5640e1081a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaacff3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640e103c6a0 con 0x5640e10450b0 2024-04-25T10:10:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa42001c0 con 0x5640e1047a90 2024-04-25T10:10:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa41ff370 con 0x5640e1044690 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.305+0000 7fcaa2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 0x5640e1063f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 0x5640e1063f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38416/0 (socket says 172.21.15.165:38416) 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 -- 172.21.15.165:0/751294776 learned_addr learned my addr 172.21.15.165:0/751294776 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 -- 172.21.15.165:0/751294776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 msgr2=0x5640e101a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e101a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 -- 172.21.15.165:0/751294776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1047a90 msgr2=0x5640e1081a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1047a90 0x5640e1081a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 -- 172.21.15.165:0/751294776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640e0c2b1b0 con 0x5640e10450b0 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 0x5640e1063f00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fca900887b0 tx=0x7fca90058990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e38a4e918cdba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fca8ea00640 1 -- 172.21.15.165:0/751294776 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca90070020 con 0x5640e10450b0 2024-04-25T10:10:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fca8ea00640 1 -- 172.21.15.165:0/751294776 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca900611a0 con 0x5640e10450b0 2024-04-25T10:10:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.306+0000 7fca8ea00640 1 -- 172.21.15.165:0/751294776 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9001f040 con 0x5640e10450b0 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 -- 172.21.15.165:0/751294776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 msgr2=0x5640e1063f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 0x5640e1063f00 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fca900887b0 tx=0x7fca90058990 comp rx=0 tx=0).stop 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 -- 172.21.15.165:0/751294776 shutdown_connections 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e1047a90 0x5640e1081a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e10450b0 0x5640e1063f00 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 --2- 172.21.15.165:0/751294776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e101a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 -- 172.21.15.165:0/751294776 >> 172.21.15.165:0/751294776 conn(0x5640e1014dd0 msgr2=0x5640e10663e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 -- 172.21.15.165:0/751294776 shutdown_connections 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.307+0000 7fcaacff3740 1 -- 172.21.15.165:0/751294776 wait complete. 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.308+0000 7fcaacff3740 1 -- start start 2024-04-25T10:10:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.308+0000 7fcaacff3740 1 Processor -- start 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1081c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1046c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10c8d90 0x5640e10471c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa414add0 con 0x5640e1047a90 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcaa415a1a0 con 0x5640e10c8d90 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5640e106e0c0 con 0x5640e1044690 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.309+0000 7fcaacff3740 10 client.0 _ll_register_callbacks cb 0x7fca9c001630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1046c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1046c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38418/0 (socket says 172.21.15.165:38418) 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1657244650 learned_addr learned my addr 172.21.15.165:0/1657244650 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1081c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 msgr2=0x5640e1081c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1081c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10c8d90 msgr2=0x5640e10471c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10c8d90 0x5640e10471c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 -- 172.21.15.165:0/1657244650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640e10821c0 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fcaa2a00640 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1046c80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fca90060c70 tx=0x7fca9001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f01d259052a995b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fca83e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.310+0000 7fca83e00640 1 -- 172.21.15.165:0/1657244650 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca9001f040 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fca83e00640 1 -- 172.21.15.165:0/1657244650 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca90060f10 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fca900611a0 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fca83e00640 1 -- 172.21.15.165:0/1657244650 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca90057ee0 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 10 client.7293 fetch_fsmap learned FSMap version 57 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5640e106de40 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fca83e00640 1 -- 172.21.15.165:0/1657244650 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca900611a0 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fca83e00640 1 -- 172.21.15.165:0/1657244650 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fca9007d050 con 0x5640e1047a90 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 10 client.7293 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:10:31.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 10 client.7293 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:31.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 20 client.7293 populate_metadata read hostname 'smithi165' 2024-04-25T10:10:31.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e103ad20 0x5640e103b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 4 client.7293 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:10:31.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.311+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x5640e0f8adb0 con 0x5640e103ad20 2024-04-25T10:10:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.312+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e103ad20 0x5640e103b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.313+0000 7fca83e00640 10 client.7293 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.313+0000 7fcaa3400640 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e103ad20 0x5640e103b130 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e3c899b77eb1c96 server_cookie=722306c8ea9cdd5 in_seq=0 out_seq=0 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.314+0000 7fca83e00640 1 -- 172.21.15.165:0/1657244650 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17925 (crc 0 0 0) 0x5640e0f8adb0 con 0x5640e103ad20 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.314+0000 7fca83e00640 10 client.7293 handle_command_reply: tid=0 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.314+0000 7fcaacff3740 1 client.7293 shutdown 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 msgr2=0x5640e1046c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1046c80 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fca90060c70 tx=0x7fca9001d000 comp rx=0 tx=0).stop 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 shutdown_connections 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5640e103ad20 0x5640e103b130 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5640e10c8d90 0x5640e10471c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5640e1047a90 0x5640e1046c80 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 --2- 172.21.15.165:0/1657244650 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5640e1044690 0x5640e1081c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fca83e00640 0 client.7293 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 >> 172.21.15.165:0/1657244650 conn(0x5640e1014dd0 msgr2=0x5640e1066380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 shutdown_connections 2024-04-25T10:10:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 1 -- 172.21.15.165:0/1657244650 wait complete. 2024-04-25T10:10:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaacff3740 20 client.7293 trim_cache size 0 max 16384 2024-04-25T10:10:31.322 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:10:31.322 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:10:31.322 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 1903, 2024-04-25T10:10:31.322 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 1904, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 679414, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 17711622, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 9, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 3, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.446504821, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.214367101, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.090896309, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.105060762, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1902, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.596401181, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000313565 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:10:31.323 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000470, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000235 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 29152, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 49922, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 10168913, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 54218028, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 7.496962213, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 4.556537157, 2024-04-25T10:10:31.324 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 3.100034805, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.122334514, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 49922, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.378149649, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000087699 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 45354, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.028173054, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000621 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 121168, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 28592 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3256, 2024-04-25T10:10:31.325 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3284, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 806876, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 46320388, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.858376666, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.447146535, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.159706743, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.192031522, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3283, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.622260734, 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000189540 2024-04-25T10:10:31.326 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 135696 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 5166, 2024-04-25T10:10:31.327 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 31.225552405, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006044435 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2139, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 1.927815667, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000901269 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 25982, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 25982, 2024-04-25T10:10:31.328 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 25982, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 543.141546994, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.020904531 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 12, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 19853, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 19853, 2024-04-25T10:10:31.329 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 19853, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 15, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 15, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 48396, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 47884, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 129, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-25T10:10:31.330 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:10:31.331 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 409, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 4010, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 3458, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 67, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 14026 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.332 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 2000, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 167, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 4137, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 1820, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 2221, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:10:31.333 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 24154, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-25T10:10:31.334 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 24154, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 32, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 32, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 3, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 94001264, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-25T10:10:31.335 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4532, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 147.441951409, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.032533528 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 2014, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 2014, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 0, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 13, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 13, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 19853, 2024-04-25T10:10:31.336 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 19854, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 1, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 15, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 9243, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 9228, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 105536, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215396 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 29524, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 25982, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 8, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-25T10:10:31.337 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2000, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.883408637, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001941704 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 29, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013006806, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000448510 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.338 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6000, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 19.812293539, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003302048 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 7719, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 16.090234043, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002084497 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.339 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.340 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000200615, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000200615 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006478252, 2024-04-25T10:10:31.341 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006478252 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.015964897, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.015964897 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.342 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.208880503, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.026110062 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2221, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 468.774135953, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.211064446 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.343 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.120012793, 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.120012793 2024-04-25T10:10:31.344 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000519228, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000519228 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.345 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.346 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8000, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 34.216411728, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004277051 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:10:31.347 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 13999, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 13999, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 121, 2024-04-25T10:10:31.348 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:10:31.349 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-25T10:10:31.350 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 17121070, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 6318, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 440, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 5, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-25T10:10:31.351 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 5424, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 76, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 21480495, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 885444, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-25T10:10:31.352 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 6763, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 90199443, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 29, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 6763, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6763, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 46.997847433, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006949260 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-25T10:10:31.353 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6763, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 6928 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 6763, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 93, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 6670, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 11, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 93, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 4625, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 56, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-25T10:10:31.354 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 1820, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 33, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 22, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:10:31.355 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-25T10:10:31.356 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 85, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 1, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 8, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 8, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 228, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 40 2024-04-25T10:10:31.357 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 32, 2024-04-25T10:10:31.358 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 1820, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 1820, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 34307, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 9001392, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 34307, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 34307, 2024-04-25T10:10:31.359 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 9001392, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:10:31.360 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 6763, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 90397823, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6763, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 90397823, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:10:31.361 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 6763, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 6763, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 6763, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 6763, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:10:31.362 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 1820, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 198380, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 1820, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 93, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 198380, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.363 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55b0cda906e0": { 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 84, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 24154, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 89806960, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 24154, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4532, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 89806876, 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:10:31.364 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:10:31.365 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:10:31.365 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:10:31.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.365 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:31.365 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:10:31.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca8406b500 msgr2=0x7fca8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaaa200640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca8406b500 0x7fca8406d9c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fca94034000 tx=0x7fca9402e000 comp rx=0 tx=0).stop 2024-04-25T10:10:31.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 msgr2=0x7fcaa40b52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.315+0000 7fcaaa200640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 0x7fcaa40b52a0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fca9000c7f0 tx=0x7fca9000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 shutdown_connections 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca8406b500 0x7fca8406d9c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcaa415dbb0 0x7fcaa40c9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcaa40cc390 0x7fcaa40b52a0 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 --2- 172.21.15.165:0/1580449868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcaa40cbf00 0x7fcaa40c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 >> 172.21.15.165:0/1580449868 conn(0x7fcaa4071af0 msgr2=0x7fcaa415c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 shutdown_connections 2024-04-25T10:10:31.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.316+0000 7fcaaa200640 1 -- 172.21.15.165:0/1580449868 wait complete. 2024-04-25T10:10:31.368 INFO:tasks.cephfs.test_scrub_checks:num of strays: 2000 2024-04-25T10:10:31.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:31.368 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub start '~mdsdir' recursive 2024-04-25T10:10:31.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.573+0000 7f1419600640 1 Processor -- start 2024-04-25T10:10:31.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.574+0000 7f1419600640 1 -- start start 2024-04-25T10:10:31.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.575+0000 7f1419600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f141406bff0 0x7f14140b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.575+0000 7f1419600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141415e110 con 0x7f14140b5970 2024-04-25T10:10:31.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.575+0000 7f1419600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14140b2fd0 con 0x7f141415dd30 2024-04-25T10:10:31.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.575+0000 7f1419600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f141406d000 con 0x7f141406bff0 2024-04-25T10:10:31.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.575+0000 7f1418c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f14140b5970 0x7f14140d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35868/0 (socket says 172.21.15.165:35868) 2024-04-25T10:10:31.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.575+0000 7f1418c00640 1 -- 172.21.15.165:0/1059378711 learned_addr learned my addr 172.21.15.165:0/1059378711 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1413400640 1 --2- 172.21.15.165:0/1059378711 >> v2:172.21.15.172:3300/0 conn(0x7f141406bff0 0x7f14140b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 838049859 0 0) 0x7f141415e110 con 0x7f14140b5970 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13f8003520 con 0x7f14140b5970 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3409568261 0 0) 0x7f13f8003520 con 0x7f14140b5970 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 >> v2:172.21.15.172:3300/0 conn(0x7f141406bff0 msgr2=0x7f14140b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 --2- 172.21.15.165:0/1059378711 >> v2:172.21.15.172:3300/0 conn(0x7f141406bff0 0x7f14140b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 >> v1:172.21.15.172:6789/0 conn(0x7f141415dd30 legacy=0x7f14140cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141406c480 con 0x7f14140b5970 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f140c001670 con 0x7f14140b5970 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 >> v1:172.21.15.165:6789/0 conn(0x7f14140b5970 legacy=0x7f14140d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80039b0 0x7f13f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f8004800 0x7f13f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f80051c0 0x7f13f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141406d000 con 0x7f13f80051c0 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.576+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f800c110 con 0x7f13f8004800 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f800c310 con 0x7f13f80039b0 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1418c00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f80051c0 0x7f13f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1418c00640 1 -- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80039b0 msgr2=0x7f13f8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1413400640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f8004800 0x7f13f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1413e00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80039b0 0x7f13f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1418c00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80039b0 0x7f13f8003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1418c00640 1 -- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f8004800 msgr2=0x7f13f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1418c00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f8004800 0x7f13f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.577+0000 7f1418c00640 1 -- 172.21.15.165:0/1059378711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8003520 con 0x7f13f80051c0 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.578+0000 7f1418c00640 1 -- 172.21.15.165:0/1059378711 reap_dead start 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.578+0000 7f1418c00640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f80051c0 0x7f13f800b9c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c210 tx=0x7f13f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac5f1f0700624b06 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.578+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c007d30 con 0x7f13f80051c0 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.578+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c005ad0 con 0x7f13f80051c0 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.578+0000 7f1412a00640 1 -- 172.21.15.165:0/1059378711 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c007070 con 0x7f13f80051c0 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 -- 172.21.15.165:0/1059378711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f80051c0 msgr2=0x7f13f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f80051c0 0x7f13f800b9c0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c210 tx=0x7f13f8005830 comp rx=0 tx=0).stop 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 -- 172.21.15.165:0/1059378711 shutdown_connections 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 --2- 172.21.15.165:0/1059378711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f80051c0 0x7f13f800b9c0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 -- 172.21.15.165:0/1059378711 >> 172.21.15.165:0/1059378711 conn(0x7f14140d0610 msgr2=0x7f14140d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 -- 172.21.15.165:0/1059378711 shutdown_connections 2024-04-25T10:10:31.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.579+0000 7f1419600640 1 -- 172.21.15.165:0/1059378711 wait complete. 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.581+0000 7f1419600640 1 Processor -- start 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.581+0000 7f1419600640 1 -- start start 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1419600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80051c0 0x7f1414152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1419600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14141602e0 0x7f1414160760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1419600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 0x7f1414155ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1419600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141406cd20 con 0x7f1414155a70 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1419600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141406ca40 con 0x7f14141602e0 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1419600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14140b5430 con 0x7f13f80051c0 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.582+0000 7f1418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 0x7f1414155ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 0x7f1414155ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38428/0 (socket says 172.21.15.165:38428) 2024-04-25T10:10:31.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 -- 172.21.15.165:0/1542069487 learned_addr learned my addr 172.21.15.165:0/1542069487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1413400640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14141602e0 0x7f1414160760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1413e00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80051c0 0x7f1414152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 -- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80051c0 msgr2=0x7f1414152b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80051c0 0x7f1414152b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 -- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14141602e0 msgr2=0x7f1414160760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14141602e0 0x7f1414160760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 -- 172.21.15.165:0/1542069487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14141567c0 con 0x7f1414155a70 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.583+0000 7f1418c00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 0x7f1414155ef0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c330 tx=0x7f140c001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.584+0000 7f140be00640 1 -- 172.21.15.165:0/1542069487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c042070 con 0x7f1414155a70 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.584+0000 7f140be00640 1 -- 172.21.15.165:0/1542069487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c000ed0 con 0x7f1414155a70 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.584+0000 7f140be00640 1 -- 172.21.15.165:0/1542069487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c007270 con 0x7f1414155a70 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.584+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1414156aa0 con 0x7f1414155a70 2024-04-25T10:10:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.584+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1414143fe0 con 0x7f1414155a70 2024-04-25T10:10:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.586+0000 7f140be00640 1 -- 172.21.15.165:0/1542069487 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f140c045070 con 0x7f1414155a70 2024-04-25T10:10:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.587+0000 7f140be00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13f006b500 0x7f13f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.587+0000 7f140be00640 1 -- 172.21.15.165:0/1542069487 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f140c0a9c00 con 0x7f1414155a70 2024-04-25T10:10:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.587+0000 7f1413e00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13f006b500 0x7f13f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.588+0000 7f1413e00640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13f006b500 0x7f13f006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1404000f80 tx=0x7f1404009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.588+0000 7f141db69740 1 Processor -- start 2024-04-25T10:10:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.588+0000 7f141db69740 1 -- start start 2024-04-25T10:10:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07540 0x55e9efb0aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb07920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f141db69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb26f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f141db69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9efa43ee0 con 0x55e9efb07e60 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9ef9eccf0 con 0x55e9efb07540 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9efa3ff80 con 0x55e9efb0b010 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f1418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb26f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f1418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb26f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38432/0 (socket says 172.21.15.165:38432) 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f1418c00640 1 -- 172.21.15.165:0/679258593 learned_addr learned my addr 172.21.15.165:0/679258593 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.589+0000 7f1413e00640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07540 0x55e9efb0aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1413400640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb07920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1418c00640 1 -- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 msgr2=0x55e9efb07920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1418c00640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb07920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1418c00640 1 -- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07540 msgr2=0x55e9efb0aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1418c00640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07540 0x55e9efb0aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1418c00640 1 -- 172.21.15.165:0/679258593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9ef824a20 con 0x55e9efb07e60 2024-04-25T10:10:31.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1413e00640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07540 0x55e9efb0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:31.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1413400640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb07920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:31.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1418c00640 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb26f50 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f140c080330 tx=0x7f140c078850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322089f6240179d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1403e00640 1 -- 172.21.15.165:0/679258593 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c07c2a0 con 0x55e9efb07e60 2024-04-25T10:10:31.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1403e00640 1 -- 172.21.15.165:0/679258593 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c080860 con 0x55e9efb07e60 2024-04-25T10:10:31.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.590+0000 7f1403e00640 1 -- 172.21.15.165:0/679258593 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c076450 con 0x55e9efb07e60 2024-04-25T10:10:31.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 -- 172.21.15.165:0/679258593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 msgr2=0x55e9efb26f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb26f50 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f140c080330 tx=0x7f140c078850 comp rx=0 tx=0).stop 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 -- 172.21.15.165:0/679258593 shutdown_connections 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb26f50 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb07920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 --2- 172.21.15.165:0/679258593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07540 0x55e9efb0aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 -- 172.21.15.165:0/679258593 >> 172.21.15.165:0/679258593 conn(0x55e9efadd330 msgr2=0x55e9efade330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 -- 172.21.15.165:0/679258593 shutdown_connections 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.591+0000 7f141db69740 1 -- 172.21.15.165:0/679258593 wait complete. 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.592+0000 7f141db69740 1 -- start start 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.592+0000 7f141db69740 1 Processor -- start 2024-04-25T10:10:31.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07e60 0x55e9efb0d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 0x55e9efb0d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0de20 0x55e9efb0c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9ef9eccf0 con 0x55e9efb0b010 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9ef9e2240 con 0x55e9efb07e60 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e9efa43ee0 con 0x55e9efb0de20 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f141db69740 10 client.0 _ll_register_callbacks cb 0x55e9ef757e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f1413400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 0x55e9efb0d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f1413400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 0x55e9efb0d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38446/0 (socket says 172.21.15.165:38446) 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.593+0000 7f1413400640 1 -- 172.21.15.165:0/2590932196 learned_addr learned my addr 172.21.15.165:0/2590932196 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413e00640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07e60 0x55e9efb0d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1418c00640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0de20 0x55e9efb0c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413400640 1 -- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0de20 msgr2=0x55e9efb0c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413400640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0de20 0x55e9efb0c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413400640 1 -- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07e60 msgr2=0x55e9efb0d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413400640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07e60 0x55e9efb0d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413400640 1 -- 172.21.15.165:0/2590932196 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9ef718310 con 0x55e9efb0b010 2024-04-25T10:10:31.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1413400640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 0x55e9efb0d8e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f13fc011590 tx=0x7f13fc011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61370c01450fa740 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1400c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:10:31.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1400c00640 1 -- 172.21.15.165:0/2590932196 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13fc015810 con 0x55e9efb0b010 2024-04-25T10:10:31.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f1400c00640 1 -- 172.21.15.165:0/2590932196 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13fc0159b0 con 0x55e9efb0b010 2024-04-25T10:10:31.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.594+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f140c080860 con 0x55e9efb0b010 2024-04-25T10:10:31.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f1400c00640 1 -- 172.21.15.165:0/2590932196 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13fc025050 con 0x55e9efb0b010 2024-04-25T10:10:31.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 10 client.7305 fetch_fsmap learned FSMap version 57 2024-04-25T10:10:31.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55e9ef8851b0 con 0x55e9efb0b010 2024-04-25T10:10:31.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f1400c00640 1 -- 172.21.15.165:0/2590932196 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c080860 con 0x55e9efb0b010 2024-04-25T10:10:31.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f1400c00640 1 -- 172.21.15.165:0/2590932196 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f13fc028020 con 0x55e9efb0b010 2024-04-25T10:10:31.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 10 client.7305 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 10 client.7305 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 20 client.7305 populate_metadata read hostname 'smithi165' 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb02790 0x55e9efb02b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 4 client.7305 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.595+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e9efaa20c0 con 0x55e9efb02790 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.596+0000 7f1413e00640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb02790 0x55e9efb02b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.597+0000 7f1413e00640 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb02790 0x55e9efb02b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a198dabfee243bb server_cookie=7d32939d122a0cc in_seq=0 out_seq=0 2024-04-25T10:10:31.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.597+0000 7f1400c00640 10 client.7305 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f1400c00640 1 -- 172.21.15.165:0/2590932196 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e9efaa20c0 con 0x55e9efb02790 2024-04-25T10:10:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f1400c00640 10 client.7305 handle_command_reply: tid=0 2024-04-25T10:10:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 client.7305 shutdown 2024-04-25T10:10:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 msgr2=0x55e9efb0d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 0x55e9efb0d8e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7f13fc011590 tx=0x7f13fc011a50 comp rx=0 tx=0).stop 2024-04-25T10:10:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 shutdown_connections 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0de20 0x55e9efb0c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0b010 0x55e9efb0d8e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb02790 0x55e9efb02b90 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 --2- 172.21.15.165:0/2590932196 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb07e60 0x55e9efb0d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 >> 172.21.15.165:0/2590932196 conn(0x55e9efb07540 msgr2=0x55e9efadb640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f1400c00640 0 client.7305 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 shutdown_connections 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 1 -- 172.21.15.165:0/2590932196 wait complete. 2024-04-25T10:10:31.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.599+0000 7f141db69740 20 client.7305 trim_cache size 0 max 16384 2024-04-25T10:10:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.616+0000 7f141db69740 1 Processor -- start 2024-04-25T10:10:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.616+0000 7f141db69740 1 -- start start 2024-04-25T10:10:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.616+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb07e60 0x55e9efb0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0b010 0x55e9efb0cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f141db69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 0x55e9efb0d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f141db69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9efb02140 con 0x55e9efb0de20 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e9efb3fcf0 con 0x55e9efb0b010 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14142007e0 con 0x55e9efb07e60 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 0x55e9efb0d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 0x55e9efb0d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38458/0 (socket says 172.21.15.165:38458) 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 -- 172.21.15.165:0/2647534539 learned_addr learned my addr 172.21.15.165:0/2647534539 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 -- 172.21.15.165:0/2647534539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb07e60 msgr2=0x55e9efb0a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb07e60 0x55e9efb0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 -- 172.21.15.165:0/2647534539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0b010 msgr2=0x55e9efb0cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0b010 0x55e9efb0cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 -- 172.21.15.165:0/2647534539 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e9efb04050 con 0x55e9efb0de20 2024-04-25T10:10:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.617+0000 7f1418c00640 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 0x55e9efb0d510 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f140c0780e0 tx=0x7f140c0a9000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69abec5705a48ac9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f1403400640 1 -- 172.21.15.165:0/2647534539 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c080860 con 0x55e9efb0de20 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f1403400640 1 -- 172.21.15.165:0/2647534539 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c0761f0 con 0x55e9efb0de20 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f1403400640 1 -- 172.21.15.165:0/2647534539 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c007d10 con 0x55e9efb0de20 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 -- 172.21.15.165:0/2647534539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 msgr2=0x55e9efb0d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 0x55e9efb0d510 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f140c0780e0 tx=0x7f140c0a9000 comp rx=0 tx=0).stop 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 -- 172.21.15.165:0/2647534539 shutdown_connections 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb0de20 0x55e9efb0d510 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0b010 0x55e9efb0cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 --2- 172.21.15.165:0/2647534539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb07e60 0x55e9efb0a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 -- 172.21.15.165:0/2647534539 >> 172.21.15.165:0/2647534539 conn(0x55e9efb07540 msgr2=0x55e9efad6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 -- 172.21.15.165:0/2647534539 shutdown_connections 2024-04-25T10:10:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.618+0000 7f141db69740 1 -- 172.21.15.165:0/2647534539 wait complete. 2024-04-25T10:10:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.619+0000 7f141db69740 1 -- start start 2024-04-25T10:10:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.619+0000 7f141db69740 1 Processor -- start 2024-04-25T10:10:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb210f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb0a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0de20 0x55e9efb0a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1414145750 con 0x55e9efb07e60 2024-04-25T10:10:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14141ff9a0 con 0x55e9efb0de20 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14141ff430 con 0x55e9efb0b010 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f141db69740 10 client.0 _ll_register_callbacks cb 0x7f140c02ad50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f1413e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb210f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f1413e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb210f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38468/0 (socket says 172.21.15.165:38468) 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.620+0000 7f1413e00640 1 -- 172.21.15.165:0/2210821387 learned_addr learned my addr 172.21.15.165:0/2210821387 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1418c00640 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0de20 0x55e9efb0a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1413e00640 1 -- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 msgr2=0x55e9efb0a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1413e00640 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb0a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1413e00640 1 -- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0de20 msgr2=0x55e9efb0a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1413e00640 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0de20 0x55e9efb0a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1413e00640 1 -- 172.21.15.165:0/2210821387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e9ef6f1490 con 0x55e9efb07e60 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1413e00640 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb210f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f14141f70a0 tx=0x7f140400ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98dcb532a087e216 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1400c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1400c00640 1 -- 172.21.15.165:0/2210821387 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1404019030 con 0x55e9efb07e60 2024-04-25T10:10:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f1400c00640 1 -- 172.21.15.165:0/2210821387 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14040260b0 con 0x55e9efb07e60 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.621+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f140c0761f0 con 0x55e9efb07e60 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f1400c00640 1 -- 172.21.15.165:0/2210821387 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1404033050 con 0x55e9efb07e60 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 10 client.7311 fetch_fsmap learned FSMap version 57 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55e9efb99fd0 con 0x55e9efb07e60 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f1400c00640 1 -- 172.21.15.165:0/2210821387 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c0761f0 con 0x55e9efb07e60 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f1400c00640 1 -- 172.21.15.165:0/2210821387 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1404036020 con 0x55e9efb07e60 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 10 client.7311 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 10 client.7311 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 20 client.7311 populate_metadata read hostname 'smithi165' 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb23a90 0x55e9efb23e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 4 client.7311 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:10:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.622+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55e9efa51420 con 0x55e9efb23a90 2024-04-25T10:10:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.623+0000 7f1413400640 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb23a90 0x55e9efb23e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.624+0000 7f1413400640 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb23a90 0x55e9efb23e70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53b0d1f22c8ce265 server_cookie=10838b781668e5 in_seq=0 out_seq=0 2024-04-25T10:10:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.624+0000 7f1400c00640 10 client.7311 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.667+0000 7f1400c00640 1 -- 172.21.15.165:0/2210821387 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e9efa51420 con 0x55e9efb23a90 2024-04-25T10:10:31.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.667+0000 7f1400c00640 10 client.7311 handle_command_reply: tid=0 2024-04-25T10:10:31.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.667+0000 7f141db69740 1 client.7311 shutdown 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.667+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 msgr2=0x55e9efb210f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.667+0000 7f141db69740 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb210f0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f14141f70a0 tx=0x7f140400ff40 comp rx=0 tx=0).stop 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 shutdown_connections 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55e9efb23a90 0x55e9efb23e70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e9efb0de20 0x55e9efb0a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e9efb0b010 0x55e9efb0a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 --2- 172.21.15.165:0/2210821387 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e9efb07e60 0x55e9efb210f0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 >> 172.21.15.165:0/2210821387 conn(0x55e9efb07540 msgr2=0x55e9efad6710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f1400c00640 0 client.7311 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 shutdown_connections 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 1 -- 172.21.15.165:0/2210821387 wait complete. 2024-04-25T10:10:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.668+0000 7f141db69740 20 client.7311 trim_cache size 0 max 16384 2024-04-25T10:10:31.675 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:10:31.675 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:10:31.675 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "f460c099-af77-4a42-8069-db1ba34d714d", 2024-04-25T10:10:31.675 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:10:31.675 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13f006b500 msgr2=0x7f13f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13f006b500 0x7f13f006d9c0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f1404000f80 tx=0x7f1404009040 comp rx=0 tx=0).stop 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 msgr2=0x7f1414155ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 0x7f1414155ef0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c330 tx=0x7f140c001a40 comp rx=0 tx=0).stop 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 shutdown_connections 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13f006b500 0x7f13f006d9c0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1414155a70 0x7f1414155ef0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14141602e0 0x7f1414160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 --2- 172.21.15.165:0/1542069487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f80051c0 0x7f1414152b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.669+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 >> 172.21.15.165:0/1542069487 conn(0x7f14140d0610 msgr2=0x7f14140ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.670+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 shutdown_connections 2024-04-25T10:10:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.670+0000 7f1419600640 1 -- 172.21.15.165:0/1542069487 wait complete. 2024-04-25T10:10:31.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:10:31.699 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:10:31.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.938+0000 7fcc9dc00640 1 Processor -- start 2024-04-25T10:10:31.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.938+0000 7fcc9dc00640 1 -- start start 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.939+0000 7fcc9dc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc9815da90 0x7fcc98161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.939+0000 7fcc9dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc98161ba0 0x7fcc980d2310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.939+0000 7fcc9dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc98162040 con 0x7fcc980b9750 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.939+0000 7fcc9dc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc98162600 con 0x7fcc9815da90 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.939+0000 7fcc9dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98162320 con 0x7fcc98161ba0 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc9815da90 0x7fcc98161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fcc980b9750 0x7fcc980ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:37728/0 (socket says 172.21.15.165:37728) 2024-04-25T10:10:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96000640 1 -- 172.21.15.165:0/2948042176 learned_addr learned my addr 172.21.15.165:0/2948042176 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:10:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc97400640 1 --2- 172.21.15.165:0/2948042176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc98161ba0 0x7fcc980d2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96a00640 1 -- 172.21.15.165:0/2948042176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc98161ba0 msgr2=0x7fcc980d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96a00640 1 --2- 172.21.15.165:0/2948042176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc98161ba0 0x7fcc980d2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96a00640 1 -- 172.21.15.165:0/2948042176 >> v1:172.21.15.172:6789/0 conn(0x7fcc980b9750 legacy=0x7fcc980ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.940+0000 7fcc96a00640 1 -- 172.21.15.165:0/2948042176 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980d1b10 con 0x7fcc9815da90 2024-04-25T10:10:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.942+0000 7fcc96a00640 1 --2- 172.21.15.165:0/2948042176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815da90 0x7fcc98161660 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcc880097a0 tx=0x7fcc8802a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceaf8c7a93a4af68 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.942+0000 7fcc95600640 1 -- 172.21.15.165:0/2948042176 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88037070 con 0x7fcc9815da90 2024-04-25T10:10:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.942+0000 7fcc95600640 1 -- 172.21.15.165:0/2948042176 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc8802ae40 con 0x7fcc9815da90 2024-04-25T10:10:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.942+0000 7fcc95600640 1 -- 172.21.15.165:0/2948042176 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc88032660 con 0x7fcc9815da90 2024-04-25T10:10:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 -- 172.21.15.165:0/2948042176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815da90 msgr2=0x7fcc98161660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/2948042176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815da90 0x7fcc98161660 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7fcc880097a0 tx=0x7fcc8802a0f0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 -- 172.21.15.165:0/2948042176 shutdown_connections 2024-04-25T10:10:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/2948042176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc98161ba0 0x7fcc980d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/2948042176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815da90 0x7fcc98161660 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 -- 172.21.15.165:0/2948042176 >> 172.21.15.165:0/2948042176 conn(0x7fcc98061a20 msgr2=0x7fcc98061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 -- 172.21.15.165:0/2948042176 shutdown_connections 2024-04-25T10:10:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.943+0000 7fcc9dc00640 1 -- 172.21.15.165:0/2948042176 wait complete. 2024-04-25T10:10:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.945+0000 7fcc9dc00640 1 Processor -- start 2024-04-25T10:10:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.945+0000 7fcc9dc00640 1 -- start start 2024-04-25T10:10:31.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc9dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815da90 0x7fcc9815ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc9dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc98161ba0 0x7fcc9815dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc9dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e530 0x7fcc980b1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc9dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980ced80 con 0x7fcc98161ba0 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc9dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815de70 con 0x7fcc9815da90 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc9dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980cef00 con 0x7fcc9815e530 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc96000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc98161ba0 0x7fcc9815dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815da90 0x7fcc9815ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc97400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e530 0x7fcc980b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc97400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e530 0x7fcc980b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:56020/0 (socket says 172.21.15.165:56020) 2024-04-25T10:10:31.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.947+0000 7fcc97400640 1 -- 172.21.15.165:0/1470969681 learned_addr learned my addr 172.21.15.165:0/1470969681 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc96000640 1 -- 172.21.15.165:0/1470969681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e530 msgr2=0x7fcc980b1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc96000640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e530 0x7fcc980b1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc96000640 1 -- 172.21.15.165:0/1470969681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815da90 msgr2=0x7fcc9815ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc96000640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815da90 0x7fcc9815ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc96000640 1 -- 172.21.15.165:0/1470969681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9815f070 con 0x7fcc98161ba0 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc96000640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc98161ba0 0x7fcc9815dff0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcc980b6520 tx=0x7fcc8400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.948+0000 7fcc8ea00640 1 -- 172.21.15.165:0/1470969681 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84017070 con 0x7fcc98161ba0 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.949+0000 7fcc8ea00640 1 -- 172.21.15.165:0/1470969681 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84005ce0 con 0x7fcc98161ba0 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.949+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc9815eab0 con 0x7fcc98161ba0 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.949+0000 7fcc8ea00640 1 -- 172.21.15.165:0/1470969681 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84007310 con 0x7fcc98161ba0 2024-04-25T10:10:31.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.949+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc980cab10 con 0x7fcc98161ba0 2024-04-25T10:10:31.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.952+0000 7fcc8ea00640 1 -- 172.21.15.165:0/1470969681 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc8401a070 con 0x7fcc98161ba0 2024-04-25T10:10:31.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.953+0000 7fcc8ea00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.953+0000 7fcc8ea00640 1 -- 172.21.15.165:0/1470969681 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc84089620 con 0x7fcc98161ba0 2024-04-25T10:10:31.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.953+0000 7fcc96a00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.953+0000 7fcca0a15740 1 Processor -- start 2024-04-25T10:10:31.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.954+0000 7fcca0a15740 1 -- start start 2024-04-25T10:10:31.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.954+0000 7fcc96a00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcc88034000 tx=0x7fcc8802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae25d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae23f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae4e670 0x55c00ae4eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcca0a15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00ad87c70 con 0x55c00ae23bb0 2024-04-25T10:10:31.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00ad30cf0 con 0x55c00ae4e670 2024-04-25T10:10:31.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c00ad83d10 con 0x55c00ae60280 2024-04-25T10:10:31.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae25d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae25d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38480/0 (socket says 172.21.15.165:38480) 2024-04-25T10:10:31.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcc96a00640 1 -- 172.21.15.165:0/605991733 learned_addr learned my addr 172.21.15.165:0/605991733 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcc96000640 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae23f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.955+0000 7fcc97400640 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae4e670 0x55c00ae4eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc96000640 1 -- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae4e670 msgr2=0x55c00ae4eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc96000640 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae4e670 0x55c00ae4eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc96000640 1 -- 172.21.15.165:0/605991733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 msgr2=0x55c00ae25d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc96000640 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae25d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc96000640 1 -- 172.21.15.165:0/605991733 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c00ab667d0 con 0x55c00ae60280 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc97400640 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae4e670 0x55c00ae4eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.956+0000 7fcc96000640 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae23f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc84087980 tx=0x7fcc840576d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62e16861ed93be5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.957+0000 7fcc83400640 1 -- 172.21.15.165:0/605991733 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84060310 con 0x55c00ae60280 2024-04-25T10:10:31.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.957+0000 7fcc83400640 1 -- 172.21.15.165:0/605991733 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84055fb0 con 0x55c00ae60280 2024-04-25T10:10:31.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.957+0000 7fcc83400640 1 -- 172.21.15.165:0/605991733 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8406bbc0 con 0x55c00ae60280 2024-04-25T10:10:31.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 -- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 msgr2=0x55c00ae23f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae23f90 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fcc84087980 tx=0x7fcc840576d0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 -- 172.21.15.165:0/605991733 shutdown_connections 2024-04-25T10:10:31.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae4e670 0x55c00ae4eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae23f90 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 --2- 172.21.15.165:0/605991733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae25d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 -- 172.21.15.165:0/605991733 >> 172.21.15.165:0/605991733 conn(0x55c00adeba30 msgr2=0x55c00ae69670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 -- 172.21.15.165:0/605991733 shutdown_connections 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.958+0000 7fcca0a15740 1 -- 172.21.15.165:0/605991733 wait complete. 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.959+0000 7fcca0a15740 1 -- start start 2024-04-25T10:10:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.959+0000 7fcca0a15740 1 Processor -- start 2024-04-25T10:10:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.960+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae3bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.960+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae4f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae4fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcca0a15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00ad263a0 con 0x55c00ae23bb0 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00ae4c1c0 con 0x55c00ae60280 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c00ae3c420 con 0x55c00ae4e670 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcca0a15740 10 client.0 _ll_register_callbacks cb 0x55c00ad43b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae3bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae3bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38494/0 (socket says 172.21.15.165:38494) 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 -- 172.21.15.165:0/889159413 learned_addr learned my addr 172.21.15.165:0/889159413 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc97400640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae4fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96000640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae4f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 -- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 msgr2=0x55c00ae4f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae4f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 -- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 msgr2=0x55c00ae4fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae4fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.961+0000 7fcc96a00640 1 -- 172.21.15.165:0/889159413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c00aa9c3f0 con 0x55c00ae23bb0 2024-04-25T10:10:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.962+0000 7fcc96000640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae4f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:10:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.962+0000 7fcc96a00640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae3bee0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009890 tx=0x7fcc88004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a5e999c0aa5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.962+0000 7fcc77e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.962+0000 7fcc77e00640 1 -- 172.21.15.165:0/889159413 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8804e020 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.963+0000 7fcc77e00640 1 -- 172.21.15.165:0/889159413 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc88049850 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.963+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc84055fb0 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.963+0000 7fcc77e00640 1 -- 172.21.15.165:0/889159413 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc88037040 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.963+0000 7fcca0a15740 10 client.7323 fetch_fsmap learned FSMap version 57 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.963+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55c00aa5c0f0 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.963+0000 7fcc77e00640 1 -- 172.21.15.165:0/889159413 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84055fb0 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcc77e00640 1 -- 172.21.15.165:0/889159413 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc8805b080 con 0x55c00ae23bb0 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcca0a15740 10 client.7323 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcca0a15740 10 client.7323 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcca0a15740 20 client.7323 populate_metadata read hostname 'smithi165' 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcca0a15740 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae7cbb0 0x55c00ae7d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcca0a15740 4 client.7323 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:10:31.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c00ad951b0 con 0x55c00ae7cbb0 2024-04-25T10:10:31.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.964+0000 7fcc96000640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae7cbb0 0x55c00ae7d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.965+0000 7fcc96000640 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae7cbb0 0x55c00ae7d000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7e8a643765807f8 server_cookie=9227d19e9fc914c7 in_seq=0 out_seq=0 2024-04-25T10:10:31.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.965+0000 7fcc77e00640 10 client.7323 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.966+0000 7fcc77e00640 1 -- 172.21.15.165:0/889159413 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c00ad951b0 con 0x55c00ae7cbb0 2024-04-25T10:10:31.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.966+0000 7fcc77e00640 10 client.7323 handle_command_reply: tid=0 2024-04-25T10:10:31.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.966+0000 7fcca0a15740 1 client.7323 shutdown 2024-04-25T10:10:31.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 msgr2=0x55c00ae3bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae3bee0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009890 tx=0x7fcc88004730 comp rx=0 tx=0).stop 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 shutdown_connections 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae7cbb0 0x55c00ae7d000 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae4fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae4f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 --2- 172.21.15.165:0/889159413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae3bee0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcc77e00640 0 client.7323 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 >> 172.21.15.165:0/889159413 conn(0x55c00adeba30 msgr2=0x55c00ae4c3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 shutdown_connections 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 1 -- 172.21.15.165:0/889159413 wait complete. 2024-04-25T10:10:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.967+0000 7fcca0a15740 20 client.7323 trim_cache size 0 max 16384 2024-04-25T10:10:31.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.989+0000 7fcca0a15740 1 Processor -- start 2024-04-25T10:10:31.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.989+0000 7fcca0a15740 1 -- start start 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae4faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae61e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae49660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcca0a15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98200600 con 0x55c00ae23bb0 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc981ff7d0 con 0x55c00ae60280 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980b7b00 con 0x55c00ae4e670 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae4faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae4faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38506/0 (socket says 172.21.15.165:38506) 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.990+0000 7fcc96a00640 1 -- 172.21.15.165:0/2071430527 learned_addr learned my addr 172.21.15.165:0/2071430527 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc97400640 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae49660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96000640 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae61e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:31.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96a00640 1 -- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 msgr2=0x55c00ae61e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96a00640 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae61e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96a00640 1 -- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 msgr2=0x55c00ae49660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96a00640 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae49660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96a00640 1 -- 172.21.15.165:0/2071430527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c00ae69bb0 con 0x55c00ae23bb0 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.991+0000 7fcc96a00640 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae4faf0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcc880093f0 tx=0x7fcc88047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59342f42fd465477 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.992+0000 7fcc82a00640 1 -- 172.21.15.165:0/2071430527 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8805e030 con 0x55c00ae23bb0 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.992+0000 7fcc82a00640 1 -- 172.21.15.165:0/2071430527 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88049b90 con 0x55c00ae23bb0 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.992+0000 7fcc82a00640 1 -- 172.21.15.165:0/2071430527 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc8804e040 con 0x55c00ae23bb0 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.992+0000 7fcca0a15740 1 -- 172.21.15.165:0/2071430527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 msgr2=0x55c00ae4faf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.992+0000 7fcca0a15740 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae4faf0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7fcc880093f0 tx=0x7fcc88047000 comp rx=0 tx=0).stop 2024-04-25T10:10:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 -- 172.21.15.165:0/2071430527 shutdown_connections 2024-04-25T10:10:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae60280 0x55c00ae49660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae4e670 0x55c00ae61e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 --2- 172.21.15.165:0/2071430527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae23bb0 0x55c00ae4faf0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 -- 172.21.15.165:0/2071430527 >> 172.21.15.165:0/2071430527 conn(0x55c00adeba30 msgr2=0x55c00ae8c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 -- 172.21.15.165:0/2071430527 shutdown_connections 2024-04-25T10:10:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.993+0000 7fcca0a15740 1 -- 172.21.15.165:0/2071430527 wait complete. 2024-04-25T10:10:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.994+0000 7fcca0a15740 1 -- start start 2024-04-25T10:10:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.994+0000 7fcca0a15740 1 Processor -- start 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae23bb0 0x55c00ae4d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 0x55c00ae833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae7baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c00ae49ba0 con 0x55c00ae4e670 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980b8100 con 0x55c00ae23bb0 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c00adbaca0 con 0x55c00ae60280 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcca0a15740 10 client.0 _ll_register_callbacks cb 0x7fcc88009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.995+0000 7fcc96000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 0x55c00ae833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 0x55c00ae833c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38516/0 (socket says 172.21.15.165:38516) 2024-04-25T10:10:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae23bb0 0x55c00ae4d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 -- 172.21.15.165:0/3203286499 learned_addr learned my addr 172.21.15.165:0/3203286499 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:10:32.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc97400640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae7baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 -- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 msgr2=0x55c00ae7baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae7baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 -- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae23bb0 msgr2=0x55c00ae4d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae23bb0 0x55c00ae4d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 -- 172.21.15.165:0/3203286499 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c00ae83900 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96a00640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae23bb0 0x55c00ae4d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc96000640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 0x55c00ae833c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcc84054110 tx=0x7fcc8401e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=637bb8f311697d88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.996+0000 7fcc77e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcc77e00640 1 -- 172.21.15.165:0/3203286499 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84073070 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcc77e00640 1 -- 172.21.15.165:0/3203286499 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(85..85 src has 1..85) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc84054320 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc88049b90 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcc77e00640 1 -- 172.21.15.165:0/3203286499 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc8407b050 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 10 client.7329 fetch_fsmap learned FSMap version 57 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x55c00ae5f000 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcc77e00640 1 -- 172.21.15.165:0/3203286499 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc88049b90 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcc77e00640 1 -- 172.21.15.165:0/3203286499 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcc8407e020 con 0x55c00ae4e670 2024-04-25T10:10:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 10 client.7329 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:10:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 10 client.7329 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:10:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 20 client.7329 populate_metadata read hostname 'smithi165' 2024-04-25T10:10:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae60e70 0x55c00ae61280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:10:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 4 client.7329 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:10:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.997+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c00ad8f500 con 0x55c00ae60e70 2024-04-25T10:10:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.998+0000 7fcc96a00640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae60e70 0x55c00ae61280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:10:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.999+0000 7fcc96a00640 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae60e70 0x55c00ae61280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc2be12298e76f73 server_cookie=6dd721d3d680fc32 in_seq=0 out_seq=0 2024-04-25T10:10:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:31.999+0000 7fcc77e00640 10 client.7329 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcc77e00640 1 -- 172.21.15.165:0/3203286499 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c00ad8f500 con 0x55c00ae60e70 2024-04-25T10:10:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcc77e00640 10 client.7329 handle_command_reply: tid=0 2024-04-25T10:10:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 client.7329 shutdown 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 msgr2=0x55c00ae833c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 0x55c00ae833c0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcc84054110 tx=0x7fcc8401e040 comp rx=0 tx=0).stop 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 shutdown_connections 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c00ae60280 0x55c00ae7baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c00ae4e670 0x55c00ae833c0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x55c00ae60e70 0x55c00ae61280 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 --2- 172.21.15.165:0/3203286499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c00ae23bb0 0x55c00ae4d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 >> 172.21.15.165:0/3203286499 conn(0x55c00adeba30 msgr2=0x55c00ae8c140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcc77e00640 0 client.7329 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 shutdown_connections 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 1 -- 172.21.15.165:0/3203286499 wait complete. 2024-04-25T10:10:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.000+0000 7fcca0a15740 20 client.7329 trim_cache size 0 max 16384 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "scrub active (253 inodes in the stack)", 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: "f460c099-af77-4a42-8069-db1ba34d714d": { 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "~mds0", 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "f460c099-af77-4a42-8069-db1ba34d714d", 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive" 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:10:32.008 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 msgr2=0x7fcc7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fcc88034000 tx=0x7fcc8802e000 comp rx=0 tx=0).stop 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc98161ba0 msgr2=0x7fcc9815dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc98161ba0 0x7fcc9815dff0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7fcc980b6520 tx=0x7fcc8400c800 comp rx=0 tx=0).stop 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 shutdown_connections 2024-04-25T10:10:32.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e530 0x7fcc980b1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc98161ba0 0x7fcc9815dff0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 --2- 172.21.15.165:0/1470969681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815da90 0x7fcc9815ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.001+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 >> 172.21.15.165:0/1470969681 conn(0x7fcc98061a20 msgr2=0x7fcc980d2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.002+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 shutdown_connections 2024-04-25T10:10:32.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:10:32.002+0000 7fcc9dc00640 1 -- 172.21.15.165:0/1470969681 wait complete. 2024-04-25T10:10:32.028 INFO:tasks.cephfs.filesystem:scrub status for tag:f460c099-af77-4a42-8069-db1ba34d714d - {'path': '~mds0', 'tag': 'f460c099-af77-4a42-8069-db1ba34d714d', 'options': 'recursive'} 2024-04-25T10:10:58.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:10:58.879 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:02.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:02.029 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.5:0 scrub status 2024-04-25T10:11:02.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.239+0000 7fce7a400640 1 Processor -- start 2024-04-25T10:11:02.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.239+0000 7fce7a400640 1 -- start start 2024-04-25T10:11:02.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.240+0000 7fce7a400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fce7415dd30 0x7fce740cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.240+0000 7fce7a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce7406bff0 0x7fce740b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.240+0000 7fce7a400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7415e110 con 0x7fce740b5970 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.240+0000 7fce7a400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7406d000 con 0x7fce7406bff0 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.240+0000 7fce7a400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce740b2fd0 con 0x7fce7415dd30 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce7406bff0 0x7fce740b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce7406bff0 0x7fce740b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59760/0 (socket says 172.21.15.165:59760) 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 -- 172.21.15.165:0/2248459092 learned_addr learned my addr 172.21.15.165:0/2248459092 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 -- 172.21.15.165:0/2248459092 >> v2:172.21.15.172:3300/0 conn(0x7fce7415dd30 msgr2=0x7fce740cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 --2- 172.21.15.165:0/2248459092 >> v2:172.21.15.172:3300/0 conn(0x7fce7415dd30 0x7fce740cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 -- 172.21.15.165:0/2248459092 >> v1:172.21.15.172:6789/0 conn(0x7fce740b5970 legacy=0x7fce74161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.241+0000 7fce73e00640 1 -- 172.21.15.165:0/2248459092 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7406c480 con 0x7fce7406bff0 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.242+0000 7fce73e00640 1 --2- 172.21.15.165:0/2248459092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce7406bff0 0x7fce740b5430 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fce7408bfe0 tx=0x7fce5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9034886e9137b1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce73400640 1 -- 172.21.15.165:0/2248459092 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c037070 con 0x7fce7406bff0 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce73400640 1 -- 172.21.15.165:0/2248459092 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c032440 con 0x7fce7406bff0 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce73400640 1 -- 172.21.15.165:0/2248459092 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c032810 con 0x7fce7406bff0 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce7a400640 1 -- 172.21.15.165:0/2248459092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce7406bff0 msgr2=0x7fce740b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce7a400640 1 --2- 172.21.15.165:0/2248459092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce7406bff0 0x7fce740b5430 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fce7408bfe0 tx=0x7fce5c02a230 comp rx=0 tx=0).stop 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce7a400640 1 -- 172.21.15.165:0/2248459092 shutdown_connections 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce7a400640 1 --2- 172.21.15.165:0/2248459092 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce7406bff0 0x7fce740b5430 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce7a400640 1 --2- 172.21.15.165:0/2248459092 >> v2:172.21.15.172:3300/0 conn(0x7fce7415dd30 0x7fce740cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.243+0000 7fce7a400640 1 -- 172.21.15.165:0/2248459092 >> 172.21.15.165:0/2248459092 conn(0x7fce740d0610 msgr2=0x7fce740d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.244+0000 7fce7a400640 1 -- 172.21.15.165:0/2248459092 shutdown_connections 2024-04-25T10:11:02.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.244+0000 7fce7a400640 1 -- 172.21.15.165:0/2248459092 wait complete. 2024-04-25T10:11:02.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.245+0000 7fce7a400640 1 Processor -- start 2024-04-25T10:11:02.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.245+0000 7fce7a400640 1 -- start start 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.246+0000 7fce7a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 0x7fce74160250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.246+0000 7fce7a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce7415dd30 0x7fce740acbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.246+0000 7fce7a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce741f3600 0x7fce74152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.246+0000 7fce7a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7406cd20 con 0x7fce740b5970 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.246+0000 7fce7a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7406ca40 con 0x7fce741f3600 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.246+0000 7fce7a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce740cf110 con 0x7fce7415dd30 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 0x7fce74160250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 0x7fce74160250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59772/0 (socket says 172.21.15.165:59772) 2024-04-25T10:11:02.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 -- 172.21.15.165:0/304515795 learned_addr learned my addr 172.21.15.165:0/304515795 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 -- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce7415dd30 msgr2=0x7fce740acbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce7415dd30 0x7fce740acbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 -- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce741f3600 msgr2=0x7fce74152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79a00640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce741f3600 0x7fce74152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce741f3600 0x7fce74152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79000640 1 -- 172.21.15.165:0/304515795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce74160790 con 0x7fce740b5970 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.247+0000 7fce79a00640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce741f3600 0x7fce74152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.248+0000 7fce79000640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 0x7fce74160250 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fce6400ba50 tx=0x7fce6400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.248+0000 7fce70c00640 1 -- 172.21.15.165:0/304515795 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64019070 con 0x7fce740b5970 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.248+0000 7fce70c00640 1 -- 172.21.15.165:0/304515795 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6400cc00 con 0x7fce740b5970 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.248+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce74160a30 con 0x7fce740b5970 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.248+0000 7fce70c00640 1 -- 172.21.15.165:0/304515795 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64014440 con 0x7fce740b5970 2024-04-25T10:11:02.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.248+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce740aa710 con 0x7fce740b5970 2024-04-25T10:11:02.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.251+0000 7fce70c00640 1 -- 172.21.15.165:0/304515795 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce6401c070 con 0x7fce740b5970 2024-04-25T10:11:02.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.252+0000 7fce70c00640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce5006b500 0x7fce5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.252+0000 7fce70c00640 1 -- 172.21.15.165:0/304515795 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fce6408a1f0 con 0x7fce740b5970 2024-04-25T10:11:02.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.252+0000 7fce73e00640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce5006b500 0x7fce5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.253+0000 7fce7e81f740 1 Processor -- start 2024-04-25T10:11:02.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.253+0000 7fce7e81f740 1 -- start start 2024-04-25T10:11:02.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.253+0000 7fce73e00640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce5006b500 0x7fce5006d9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fce5c034000 tx=0x7fce5c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 0x55630114fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630114e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x556301178cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce7e81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563010b1c70 con 0x55630114dd70 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630105acf0 con 0x55630118a440 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563010add10 con 0x556301178830 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce79000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 0x55630114fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce79000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 0x55630114fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59784/0 (socket says 172.21.15.165:59784) 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce79000640 1 -- 172.21.15.165:0/3539183256 learned_addr learned my addr 172.21.15.165:0/3539183256 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.254+0000 7fce73e00640 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630114e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79a00640 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x556301178cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79000640 1 -- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 msgr2=0x556301178cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79000640 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x556301178cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79000640 1 -- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 msgr2=0x55630114e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79000640 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630114e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79000640 1 -- 172.21.15.165:0/3539183256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556300e907d0 con 0x55630114dd70 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79a00640 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x556301178cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:02.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce79000640 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 0x55630114fed0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fce6400cfd0 tx=0x7fce64058620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c62ccb58350ce34 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce69600640 1 -- 172.21.15.165:0/3539183256 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64060e30 con 0x55630114dd70 2024-04-25T10:11:02.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce69600640 1 -- 172.21.15.165:0/3539183256 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64056a20 con 0x55630114dd70 2024-04-25T10:11:02.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.255+0000 7fce69600640 1 -- 172.21.15.165:0/3539183256 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce64054d90 con 0x55630114dd70 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 -- 172.21.15.165:0/3539183256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 msgr2=0x55630114fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 0x55630114fed0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fce6400cfd0 tx=0x7fce64058620 comp rx=0 tx=0).stop 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 -- 172.21.15.165:0/3539183256 shutdown_connections 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x556301178cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630114e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 --2- 172.21.15.165:0/3539183256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630114dd70 0x55630114fed0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 -- 172.21.15.165:0/3539183256 >> 172.21.15.165:0/3539183256 conn(0x556301115a30 msgr2=0x556301193830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 -- 172.21.15.165:0/3539183256 shutdown_connections 2024-04-25T10:11:02.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.256+0000 7fce7e81f740 1 -- 172.21.15.165:0/3539183256 wait complete. 2024-04-25T10:11:02.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.257+0000 7fce7e81f740 1 -- start start 2024-04-25T10:11:02.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.257+0000 7fce7e81f740 1 Processor -- start 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630114dd70 0x556301194390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x556301175330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630118cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5563010503a0 con 0x556301178830 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x556301176380 con 0x55630118a440 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55630117c9a0 con 0x55630114dd70 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.258+0000 7fce7e81f740 10 client.0 _ll_register_callbacks cb 0x55630106db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x556301175330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x556301175330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59792/0 (socket says 172.21.15.165:59792) 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 -- 172.21.15.165:0/6480066 learned_addr learned my addr 172.21.15.165:0/6480066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce79a00640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630118cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce79000640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630114dd70 0x556301194390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 -- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630114dd70 msgr2=0x556301194390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630114dd70 0x556301194390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 -- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 msgr2=0x55630118cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630118cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 -- 172.21.15.165:0/6480066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556300dc63f0 con 0x556301178830 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce73e00640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x556301175330 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009960 tx=0x7fce5c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49243c50af57f264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.259+0000 7fce62000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce62000640 1 -- 172.21.15.165:0/6480066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c04c070 con 0x556301178830 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce62000640 1 -- 172.21.15.165:0/6480066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fce5c03cab0 con 0x556301178830 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce64056a20 con 0x556301178830 2024-04-25T10:11:02.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce62000640 1 -- 172.21.15.165:0/6480066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce5c057050 con 0x556301178830 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 10 client.7341 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x556300d860f0 con 0x556301178830 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce62000640 1 -- 172.21.15.165:0/6480066 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce64056a20 con 0x556301178830 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce62000640 1 -- 172.21.15.165:0/6480066 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce5c05a020 con 0x556301178830 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 10 client.7341 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 10 client.7341 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 20 client.7341 populate_metadata read hostname 'smithi165' 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x556301178cb0 0x556301179090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 4 client.7341 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.260+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5563010bf1b0 con 0x556301178cb0 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.261+0000 7fce79000640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x556301178cb0 0x556301179090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.262+0000 7fce79000640 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x556301178cb0 0x556301179090 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560650761901030 server_cookie=996c4414c024d5fd in_seq=0 out_seq=0 2024-04-25T10:11:02.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.262+0000 7fce62000640 10 client.7341 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce62000640 1 -- 172.21.15.165:0/6480066 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5563010bf1b0 con 0x556301178cb0 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce62000640 10 client.7341 handle_command_reply: tid=0 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 client.7341 shutdown 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 msgr2=0x556301175330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x556301175330 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fce5c009960 tx=0x7fce5c004730 comp rx=0 tx=0).stop 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 shutdown_connections 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x556301178cb0 0x556301179090 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630118a440 0x55630118cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x556301175330 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 --2- 172.21.15.165:0/6480066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630114dd70 0x556301194390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce62000640 0 client.7341 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 >> 172.21.15.165:0/6480066 conn(0x556301115a30 msgr2=0x556301196db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 shutdown_connections 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.264+0000 7fce7e81f740 1 -- 172.21.15.165:0/6480066 wait complete. 2024-04-25T10:11:02.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.265+0000 7fce7e81f740 20 client.7341 trim_cache size 0 max 16384 2024-04-25T10:11:02.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.281+0000 7fce7e81f740 1 Processor -- start 2024-04-25T10:11:02.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.281+0000 7fce7e81f740 1 -- start start 2024-04-25T10:11:02.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.282+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a6170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.282+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x5563011f00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.282+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.282+0000 7fce7e81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce74200550 con 0x556301178830 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.282+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce741ff6f0 con 0x55630114dd70 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.282+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce740abb50 con 0x55630118a440 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x5563011f00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x5563011f00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59794/0 (socket says 172.21.15.165:59794) 2024-04-25T10:11:02.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 -- 172.21.15.165:0/1817983285 learned_addr learned my addr 172.21.15.165:0/1817983285 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce79000640 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 -- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630118a440 msgr2=0x556301193c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 -- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 msgr2=0x5563011a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 -- 172.21.15.165:0/1817983285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5563011a66b0 con 0x556301178830 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.283+0000 7fce73e00640 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x5563011f00f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fce5c032fd0 tx=0x7fce5c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffa0395561ab2b8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce68c00640 1 -- 172.21.15.165:0/1817983285 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c04c040 con 0x556301178830 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce68c00640 1 -- 172.21.15.165:0/1817983285 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c041040 con 0x556301178830 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce68c00640 1 -- 172.21.15.165:0/1817983285 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce5c037040 con 0x556301178830 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 -- 172.21.15.165:0/1817983285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 msgr2=0x5563011f00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x5563011f00f0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fce5c032fd0 tx=0x7fce5c046000 comp rx=0 tx=0).stop 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 -- 172.21.15.165:0/1817983285 shutdown_connections 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556301178830 0x5563011f00f0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1817983285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 -- 172.21.15.165:0/1817983285 >> 172.21.15.165:0/1817983285 conn(0x556301115a30 msgr2=0x55630118c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.284+0000 7fce7e81f740 1 -- 172.21.15.165:0/1817983285 shutdown_connections 2024-04-25T10:11:02.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.285+0000 7fce7e81f740 1 -- 172.21.15.165:0/1817983285 wait complete. 2024-04-25T10:11:02.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.285+0000 7fce7e81f740 1 -- start start 2024-04-25T10:11:02.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.285+0000 7fce7e81f740 1 Processor -- start 2024-04-25T10:11:02.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x5563011e2db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630114e1e0 con 0x55630118a440 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce741ff0f0 con 0x55630114dd70 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5563011f0630 con 0x556301178830 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.286+0000 7fce7e81f740 10 client.0 _ll_register_callbacks cb 0x7fce5c035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59808/0 (socket says 172.21.15.165:59808) 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 -- 172.21.15.165:0/1161346175 learned_addr learned my addr 172.21.15.165:0/1161346175 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79000640 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 -- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 msgr2=0x5563011e2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x5563011e2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 -- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 msgr2=0x5563011a2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 -- 172.21.15.165:0/1161346175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556301189260 con 0x55630118a440 2024-04-25T10:11:02.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce79a00640 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 0x556301193c00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fce6c007fd0 tx=0x7fce6c00f600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b255e20cff82c2e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.287+0000 7fce62000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce62000640 1 -- 172.21.15.165:0/1161346175 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c0052d0 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce62000640 1 -- 172.21.15.165:0/1161346175 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fce6c0035f0 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce5c041040 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce62000640 1 -- 172.21.15.165:0/1161346175 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce6c020050 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 10 client.7347 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x5563011e3a90 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce62000640 1 -- 172.21.15.165:0/1161346175 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce5c041040 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce62000640 1 -- 172.21.15.165:0/1161346175 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fce6c023020 con 0x55630118a440 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 10 client.7347 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 10 client.7347 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 20 client.7347 populate_metadata read hostname 'smithi165' 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5563011e3570 0x556301194140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 4 client.7347 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:11:02.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.288+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5563010b9500 con 0x5563011e3570 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.289+0000 7fce79000640 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5563011e3570 0x556301194140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce79000640 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5563011e3570 0x556301194140 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f0330311896aade server_cookie=4e500bd7993409b8 in_seq=0 out_seq=0 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce62000640 10 client.7347 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce62000640 1 -- 172.21.15.165:0/1161346175 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5563010b9500 con 0x5563011e3570 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce62000640 10 client.7347 handle_command_reply: tid=0 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 client.7347 shutdown 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 msgr2=0x556301193c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 0x556301193c00 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fce6c007fd0 tx=0x7fce6c00f600 comp rx=0 tx=0).stop 2024-04-25T10:11:02.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 shutdown_connections 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55630118a440 0x556301193c00 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556301178830 0x5563011e2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.290+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x5563011e3570 0x556301194140 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7e81f740 1 --2- 172.21.15.165:0/1161346175 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55630114dd70 0x5563011a2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce62000640 0 client.7347 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 >> 172.21.15.165:0/1161346175 conn(0x556301115a30 msgr2=0x5563011efbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 shutdown_connections 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7e81f740 1 -- 172.21.15.165:0/1161346175 wait complete. 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7e81f740 20 client.7347 trim_cache size 0 max 16384 2024-04-25T10:11:02.298 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:11:02.299 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:11:02.299 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:11:02.299 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce5006b500 msgr2=0x7fce5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7a400640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce5006b500 0x7fce5006d9c0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fce5c034000 tx=0x7fce5c02c000 comp rx=0 tx=0).stop 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 msgr2=0x7fce74160250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7a400640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 0x7fce74160250 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fce6400ba50 tx=0x7fce6400bf10 comp rx=0 tx=0).stop 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 shutdown_connections 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.291+0000 7fce7a400640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce5006b500 0x7fce5006d9c0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.292+0000 7fce7a400640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce741f3600 0x7fce74152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.292+0000 7fce7a400640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce7415dd30 0x7fce740acbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.292+0000 7fce7a400640 1 --2- 172.21.15.165:0/304515795 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce740b5970 0x7fce74160250 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.292+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 >> 172.21.15.165:0/304515795 conn(0x7fce740d0610 msgr2=0x7fce740b78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.292+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 shutdown_connections 2024-04-25T10:11:02.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.292+0000 7fce7a400640 1 -- 172.21.15.165:0/304515795 wait complete. 2024-04-25T10:11:02.313 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:11:02.314 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:11:02.314 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.5:0 perf dump 2024-04-25T10:11:02.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.509+0000 7fe8ee000640 1 Processor -- start 2024-04-25T10:11:02.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.509+0000 7fe8ee000640 1 -- start start 2024-04-25T10:11:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8ee000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe8e8144d50 0x7fe8e806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8ee000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e80cccc0 con 0x7fe8e81452a0 2024-04-25T10:11:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8ee000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e80cc140 con 0x7fe8e81446d0 2024-04-25T10:11:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8ee000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e80b2b20 con 0x7fe8e8144d50 2024-04-25T10:11:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8e7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe8e81452a0 0x7fe8e806ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36646/0 (socket says 172.21.15.165:36646) 2024-04-25T10:11:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8e7e00640 1 -- 172.21.15.165:0/3914204483 learned_addr learned my addr 172.21.15.165:0/3914204483 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443514398 0 0) 0x7fe8e80cccc0 con 0x7fe8e81452a0 2024-04-25T10:11:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8cc003520 con 0x7fe8e81452a0 2024-04-25T10:11:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.511+0000 7fe8dea00640 1 --2- 172.21.15.165:0/3914204483 >> v2:172.21.15.172:3300/0 conn(0x7fe8e8144d50 0x7fe8e806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771143025 0 0) 0x7fe8cc003520 con 0x7fe8e81452a0 2024-04-25T10:11:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 >> v2:172.21.15.172:3300/0 conn(0x7fe8e8144d50 msgr2=0x7fe8e806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 --2- 172.21.15.165:0/3914204483 >> v2:172.21.15.172:3300/0 conn(0x7fe8e8144d50 0x7fe8e806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 >> v1:172.21.15.172:6789/0 conn(0x7fe8e81446d0 legacy=0x7fe8e80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e80cc420 con 0x7fe8e81452a0 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe8e0001670 con 0x7fe8e81452a0 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 >> v1:172.21.15.165:6789/0 conn(0x7fe8e81452a0 legacy=0x7fe8e806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc0039b0 0x7fe8cc003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8cc004800 0x7fe8cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8cc0051c0 0x7fe8cc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80b2b20 con 0x7fe8cc004800 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc00c060 con 0x7fe8cc0051c0 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc00c260 con 0x7fe8cc0039b0 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.512+0000 7fe8dea00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8cc004800 0x7fe8cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8e7400640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc0039b0 0x7fe8cc003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8dea00640 1 -- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc0039b0 msgr2=0x7fe8cc003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8dea00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc0039b0 0x7fe8cc003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8dea00640 1 -- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8cc0051c0 msgr2=0x7fe8cc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8dea00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8cc0051c0 0x7fe8cc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8dea00640 1 -- 172.21.15.165:0/3914204483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8cc003520 con 0x7fe8cc004800 2024-04-25T10:11:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.513+0000 7fe8e7e00640 1 -- 172.21.15.165:0/3914204483 reap_dead start 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.514+0000 7fe8dea00640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8cc004800 0x7fe8cc004c80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009fc0 tx=0x7fe8cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f1514ab85acc1ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.514+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0034020 con 0x7fe8cc004800 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.514+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d00305f0 con 0x7fe8cc004800 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.514+0000 7fe8e6a00640 1 -- 172.21.15.165:0/3914204483 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0009c10 con 0x7fe8cc004800 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 -- 172.21.15.165:0/3914204483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8cc004800 msgr2=0x7fe8cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8cc004800 0x7fe8cc004c80 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0009fc0 tx=0x7fe8cc005830 comp rx=0 tx=0).stop 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 -- 172.21.15.165:0/3914204483 shutdown_connections 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 --2- 172.21.15.165:0/3914204483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8cc004800 0x7fe8cc004c80 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 -- 172.21.15.165:0/3914204483 >> 172.21.15.165:0/3914204483 conn(0x7fe8e806abe0 msgr2=0x7fe8e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 -- 172.21.15.165:0/3914204483 shutdown_connections 2024-04-25T10:11:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.515+0000 7fe8ee000640 1 -- 172.21.15.165:0/3914204483 wait complete. 2024-04-25T10:11:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.517+0000 7fe8ee000640 1 Processor -- start 2024-04-25T10:11:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.517+0000 7fe8ee000640 1 -- start start 2024-04-25T10:11:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8ee000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc004800 0x7fe8e8155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8ee000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e80b0f80 0x7fe8e80b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8ee000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 0x7fe8e81f3300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8ee000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e80cc9e0 con 0x7fe8e81f2e80 2024-04-25T10:11:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8ee000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e806d400 con 0x7fe8e80b0f80 2024-04-25T10:11:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8ee000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8144ab0 con 0x7fe8cc004800 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 0x7fe8e81f3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 0x7fe8e81f3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59836/0 (socket says 172.21.15.165:59836) 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.518+0000 7fe8e7e00640 1 -- 172.21.15.165:0/531808214 learned_addr learned my addr 172.21.15.165:0/531808214 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8dea00640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e80b0f80 0x7fe8e80b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7400640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc004800 0x7fe8e8155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7e00640 1 -- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc004800 msgr2=0x7fe8e8155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc004800 0x7fe8e8155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7e00640 1 -- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e80b0f80 msgr2=0x7fe8e80b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e80b0f80 0x7fe8e80b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7e00640 1 -- 172.21.15.165:0/531808214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e80b1940 con 0x7fe8e81f2e80 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8dea00640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e80b0f80 0x7fe8e80b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 0x7fe8e81f3300 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80cc090 tx=0x7fe8e000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.519+0000 7fe8dfe00640 1 -- 172.21.15.165:0/531808214 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0004bf0 con 0x7fe8e81f2e80 2024-04-25T10:11:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.520+0000 7fe8dfe00640 1 -- 172.21.15.165:0/531808214 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0004d90 con 0x7fe8e81f2e80 2024-04-25T10:11:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.520+0000 7fe8dfe00640 1 -- 172.21.15.165:0/531808214 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0007270 con 0x7fe8e81f2e80 2024-04-25T10:11:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.520+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e813caa0 con 0x7fe8e81f2e80 2024-04-25T10:11:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.520+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e813d060 con 0x7fe8e81f2e80 2024-04-25T10:11:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.523+0000 7fe8dfe00640 1 -- 172.21.15.165:0/531808214 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8e0007410 con 0x7fe8e81f2e80 2024-04-25T10:11:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.524+0000 7fe8dfe00640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8c406b4d0 0x7fe8c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.524+0000 7fe8dfe00640 1 -- 172.21.15.165:0/531808214 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe8e0089900 con 0x7fe8e81f2e80 2024-04-25T10:11:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.524+0000 7fe8e7400640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8c406b4d0 0x7fe8c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.524+0000 7fe8f0d68740 1 Processor -- start 2024-04-25T10:11:02.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.524+0000 7fe8f0d68740 1 -- start start 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8e7400640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8c406b4d0 0x7fe8c406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8000c00 tx=0x7fe8d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279db6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279d86890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 0x557279daeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557279cef870 con 0x557279d86dd0 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557279c98d60 con 0x557279db70b0 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.525+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557279ceb910 con 0x557279db6690 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 0x557279daeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 0x557279daeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59840/0 (socket says 172.21.15.165:59840) 2024-04-25T10:11:02.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 -- 172.21.15.165:0/904245009 learned_addr learned my addr 172.21.15.165:0/904245009 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 -- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 msgr2=0x557279db6a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279db6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 -- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 msgr2=0x557279d86890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8dea00640 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279d86890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279d86890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 -- 172.21.15.165:0/904245009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557279ace7d0 con 0x557279d86dd0 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.526+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 0x557279daeb10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0087b20 tx=0x7fe8e0057ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd8c8d6a3717c10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8d7400640 1 -- 172.21.15.165:0/904245009 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0060600 con 0x557279d86dd0 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8d7400640 1 -- 172.21.15.165:0/904245009 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0087070 con 0x557279d86dd0 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8d7400640 1 -- 172.21.15.165:0/904245009 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e00544d0 con 0x557279d86dd0 2024-04-25T10:11:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 -- 172.21.15.165:0/904245009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 msgr2=0x557279daeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 0x557279daeb10 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0087b20 tx=0x7fe8e0057ee0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 -- 172.21.15.165:0/904245009 shutdown_connections 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279d86dd0 0x557279daeb10 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279d86890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/904245009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279db6a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 -- 172.21.15.165:0/904245009 >> 172.21.15.165:0/904245009 conn(0x557279d88d10 msgr2=0x557279dd14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 -- 172.21.15.165:0/904245009 shutdown_connections 2024-04-25T10:11:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.527+0000 7fe8f0d68740 1 -- 172.21.15.165:0/904245009 wait complete. 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.528+0000 7fe8f0d68740 1 -- start start 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.528+0000 7fe8f0d68740 1 Processor -- start 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279db2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279daebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 0x557279db53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557279db2de0 con 0x557279daf130 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557279cac1c0 con 0x557279db70b0 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557279db3080 con 0x557279db6690 2024-04-25T10:11:02.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.529+0000 7fe8f0d68740 10 client.0 _ll_register_callbacks cb 0x557279cea8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 0x557279db53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 0x557279db53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59842/0 (socket says 172.21.15.165:59842) 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 -- 172.21.15.165:0/279176868 learned_addr learned my addr 172.21.15.165:0/279176868 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 -- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 msgr2=0x557279db2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279db2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 -- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 msgr2=0x557279daebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279daebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 -- 172.21.15.165:0/279176868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5572799c40f0 con 0x557279daf130 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 0x557279db53a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0084040 tx=0x7fe8e0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c278416dc0414a1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8cbe00640 1 -- 172.21.15.165:0/279176868 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0074070 con 0x557279daf130 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.530+0000 7fe8cbe00640 1 -- 172.21.15.165:0/279176868 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe8e006fdc0 con 0x557279daf130 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8e0087070 con 0x557279daf130 2024-04-25T10:11:02.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8cbe00640 1 -- 172.21.15.165:0/279176868 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8e00572b0 con 0x557279daf130 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 10 client.7359 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x557279b2ef60 con 0x557279daf130 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8cbe00640 1 -- 172.21.15.165:0/279176868 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0087070 con 0x557279daf130 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8cbe00640 1 -- 172.21.15.165:0/279176868 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe8e0057450 con 0x557279daf130 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 10 client.7359 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 10 client.7359 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 20 client.7359 populate_metadata read hostname 'smithi165' 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db7980 0x557279db7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 4 client.7359 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.531+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557279cc5690 con 0x557279db7980 2024-04-25T10:11:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.532+0000 7fe8e7400640 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db7980 0x557279db7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.533+0000 7fe8e7400640 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db7980 0x557279db7d90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd6e57ed79123a1 server_cookie=ad77166d14e901a1 in_seq=0 out_seq=0 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.533+0000 7fe8cbe00640 10 client.7359 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.534+0000 7fe8cbe00640 1 -- 172.21.15.165:0/279176868 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557279cc5690 con 0x557279db7980 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.534+0000 7fe8cbe00640 10 client.7359 handle_command_reply: tid=0 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 client.7359 shutdown 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 msgr2=0x557279db53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 0x557279db53a0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0084040 tx=0x7fe8e0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 shutdown_connections 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279daf130 0x557279db53a0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db7980 0x557279db7d90 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279db70b0 0x557279daebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/279176868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279db2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8cbe00640 0 client.7359 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 >> 172.21.15.165:0/279176868 conn(0x557279d86dd0 msgr2=0x557279d856b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 shutdown_connections 2024-04-25T10:11:02.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 1 -- 172.21.15.165:0/279176868 wait complete. 2024-04-25T10:11:02.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.535+0000 7fe8f0d68740 20 client.7359 trim_cache size 0 max 16384 2024-04-25T10:11:02.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.551+0000 7fe8f0d68740 1 Processor -- start 2024-04-25T10:11:02.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.551+0000 7fe8f0d68740 1 -- start start 2024-04-25T10:11:02.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279d895a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279dd02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 0x557279db8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557279dd8c20 con 0x557279db70b0 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557279deb680 con 0x557279daf130 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e8200c50 con 0x557279db6690 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 0x557279db8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 0x557279db8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59858/0 (socket says 172.21.15.165:59858) 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 -- 172.21.15.165:0/3208538917 learned_addr learned my addr 172.21.15.165:0/3208538917 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 -- 172.21.15.165:0/3208538917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 msgr2=0x557279dd02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279dd02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 -- 172.21.15.165:0/3208538917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 msgr2=0x557279d895a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279d895a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.552+0000 7fe8e7e00640 1 -- 172.21.15.165:0/3208538917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55727999d1b0 con 0x557279db70b0 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 0x557279db8500 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0057df0 tx=0x7fe8e0074000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef25aca5685359d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8d6a00640 1 -- 172.21.15.165:0/3208538917 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e0057450 con 0x557279db70b0 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8d6a00640 1 -- 172.21.15.165:0/3208538917 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0087040 con 0x557279db70b0 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8d6a00640 1 -- 172.21.15.165:0/3208538917 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8e006fe00 con 0x557279db70b0 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 -- 172.21.15.165:0/3208538917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 msgr2=0x557279db8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 0x557279db8500 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7fe8e0057df0 tx=0x7fe8e0074000 comp rx=0 tx=0).stop 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 -- 172.21.15.165:0/3208538917 shutdown_connections 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db70b0 0x557279db8500 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db6690 0x557279dd02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/3208538917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279d895a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.553+0000 7fe8f0d68740 1 -- 172.21.15.165:0/3208538917 >> 172.21.15.165:0/3208538917 conn(0x557279d86dd0 msgr2=0x557279db2160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.554+0000 7fe8f0d68740 1 -- 172.21.15.165:0/3208538917 shutdown_connections 2024-04-25T10:11:02.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.554+0000 7fe8f0d68740 1 -- 172.21.15.165:0/3208538917 wait complete. 2024-04-25T10:11:02.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.554+0000 7fe8f0d68740 1 -- start start 2024-04-25T10:11:02.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.554+0000 7fe8f0d68740 1 Processor -- start 2024-04-25T10:11:02.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279dd0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 0x557279e45da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db70b0 0x557279db0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e81f7f40 con 0x557279db6690 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e813f740 con 0x557279daf130 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e813f140 con 0x557279db70b0 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.555+0000 7fe8f0d68740 10 client.0 _ll_register_callbacks cb 0x7fe8e0054180 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 0x557279e45da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 0x557279e45da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59860/0 (socket says 172.21.15.165:59860) 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 -- 172.21.15.165:0/4055033264 learned_addr learned my addr 172.21.15.165:0/4055033264 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8e7e00640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db70b0 0x557279db0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8e7400640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279dd0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 -- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db70b0 msgr2=0x557279db0870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db70b0 0x557279db0870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 -- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 msgr2=0x557279dd0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279dd0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8dea00640 1 -- 172.21.15.165:0/4055033264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557279dd8f40 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.556+0000 7fe8e7400640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279dd0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8dea00640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 0x557279e45da0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e230 tx=0x7fe8d0007020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f17677ff6e6a67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8cbe00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8cbe00640 1 -- 172.21.15.165:0/4055033264 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0048060 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8e0087040 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8cbe00640 1 -- 172.21.15.165:0/4055033264 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe8d0004040 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8cbe00640 1 -- 172.21.15.165:0/4055033264 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8d0034040 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8f0d68740 10 client.7365 fetch_fsmap learned FSMap version 57 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=2+}) v3 -- 0x557279dd9ac0 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.557+0000 7fe8cbe00640 1 -- 172.21.15.165:0/4055033264 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8e0087040 con 0x557279db6690 2024-04-25T10:11:02.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8cbe00640 1 -- 172.21.15.165:0/4055033264 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 57) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe8d0051080 con 0x557279db6690 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8f0d68740 10 client.7365 fetch_fsmap finished waiting for FSMap version 57 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8f0d68740 10 client.7365 resolve_mds: resolved 5:0 to role '5:0' aka daemon mds.d 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8f0d68740 20 client.7365 populate_metadata read hostname 'smithi165' 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db0db0 0x557279de0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8f0d68740 4 client.7365 mds_command: new command op to 7116 tid=0 multi_id=0 [{"prefix": "perf dump"}] 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 --> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] -- command(tid 0: {"prefix": "perf dump"}) v1 -- 0x557279cfcdb0 con 0x557279db0db0 2024-04-25T10:11:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.558+0000 7fe8e7400640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db0db0 0x557279de0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:02.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.559+0000 7fe8e7400640 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db0db0 0x557279de0090 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7325c44f9cc7c0 server_cookie=fadf2d173899e2c3 in_seq=0 out_seq=0 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.559+0000 7fe8cbe00640 10 client.7365 ms_handle_connect on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8cbe00640 1 -- 172.21.15.165:0/4055033264 <== mds.0 v2:172.21.15.172:6832/1170361110 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17952 (crc 0 0 0) 0x557279cfcdb0 con 0x557279db0db0 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8cbe00640 10 client.7365 handle_command_reply: tid=0 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 client.7365 shutdown 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 msgr2=0x557279e45da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 0x557279e45da0 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7fe8d000e230 tx=0x7fe8d0007020 comp rx=0 tx=0).stop 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 shutdown_connections 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557279db70b0 0x557279db0870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557279db6690 0x557279e45da0 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:6832/1170361110,v1:172.21.15.172:6833/1170361110] conn(0x557279db0db0 0x557279de0090 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 --2- 172.21.15.165:0/4055033264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557279daf130 0x557279dd0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 >> 172.21.15.165:0/4055033264 conn(0x557279d86dd0 msgr2=0x557279dad8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8cbe00640 0 client.7365 ms_handle_reset on v2:172.21.15.172:6832/1170361110 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 shutdown_connections 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 1 -- 172.21.15.165:0/4055033264 wait complete. 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.561+0000 7fe8f0d68740 20 client.7365 trim_cache size 0 max 16384 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-0": { 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 3566, 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 3567, 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1303296, 2024-04-25T10:11:02.569 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 18553543, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 17, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 6, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.617925950, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.278832927, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.132549223, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.157477788, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 3565, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.811952021, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000227756 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:11:02.570 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000001710, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000171 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 0 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-1": { 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 30078, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 50816, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 10409335, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 55729052, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 7, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 7.621902040, 2024-04-25T10:11:02.571 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 4.616280195, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 3.125349881, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.155336536, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 50816, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 4.514524338, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000088840 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 45356, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.028175170, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000621 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 125648, 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 35808 2024-04-25T10:11:02.572 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "AsyncMessenger::Worker-2": { 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_messages": 4599, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages": 4633, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_bytes": 1148638, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_bytes": 46773163, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_created_connections": 6, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_active_connections": 4, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_total_time": 0.986775152, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_send_time": 0.494516966, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_recv_time": 0.189922901, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_running_fast_dispatch_time": 0.231789886, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_messages_queue_lat": { 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4632, 2024-04-25T10:11:02.573 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.821521489, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000177357 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_handle_ack_lat": { 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_recv_encrypted_bytes": 1728, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "msgr_send_encrypted_bytes": 146496 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "cct": { 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "total_workers": 1, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "unhealthy_workers": 0 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.574 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-MDSRank": { 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6577, 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 31.698290633, 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004819566 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "finisher-PurgeQueue": { 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "queue_len": 0, 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "complete_latency": { 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4168, 2024-04-25T10:11:02.575 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 2.362734036, 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000566874 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": { 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "request": 25982, 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "reply": 25982, 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "reply_latency": { 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 25982, 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 543.141546994, 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.020904531 2024-04-25T10:11:02.576 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_reply": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "forward": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_complete": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_fetch_keys": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_commit": 12, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_split": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_merge": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes": 17853, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_top": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_bottom": 17853, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pin_tail": 0, 2024-04-25T10:11:02.577 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_pinned": 17853, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_expired": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "inodes_with_caps": 2, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": 2, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "subtrees": 2, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse": 48397, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_hit": 47885, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_forward": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_discover": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_dir_fetch": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_remote_ino": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "traverse_lock": 129, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "load_cent": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "q": 0, 2024-04-25T10:11:02.578 INFO:teuthology.orchestra.run.smithi165.stdout: "exported": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "exported_inodes": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "imported": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "imported_inodes": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_dir_fetch": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_backtrace_fetch": 1844, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "openino_peer_discover": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rfiles": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rbytes": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "root_rsnaps": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_fetch": 1844, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_set_tag": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_backtrace_repaired": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_inotable_repaired": 0, 2024-04-25T10:11:02.579 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_inodes": 11, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dir_base_inodes": 1, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_dirfrag_rstats": 10, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_file_inodes": 1833, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_inode_file_caps": 0, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_revoke": 409, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_grant": 4010, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_trunc": 0, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps": 3458, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_caps_dirty": 2000, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_cap_release": 68, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "process_request_cap_release": 14026 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.580 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_cache": { 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_update_receipt": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_try_discover": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_send_discover": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "dir_handle_discover": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_delayed": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "num_strays_enqueuing": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_created": 4137, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_enqueued": 3820, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_reintegrated": 2221, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "strays_migrated": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_processing": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_enqueued": 0, 2024-04-25T10:11:02.581 INFO:teuthology.orchestra.run.smithi165.stdout: "num_recovering_prioritized": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_started": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "recovery_completed": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_enqueue_scrub": 1, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_exportdir": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_flush": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragmentdir": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_fragstats": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "ireq_inodestats": 0 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_log": { 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "evlrg": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "evadd": 26163, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "evex": 0, 2024-04-25T10:11:02.582 INFO:teuthology.orchestra.run.smithi165.stdout: "evtrm": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "ev": 26163, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "evexg": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "evexd": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "segadd": 35, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "segex": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "segtrm": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "seg": 35, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "segmjr": 4, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "segexg": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "segexd": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "expos": 4194304, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "wrpos": 95216758, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "rdpos": 0, 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "jlat": { 2024-04-25T10:11:02.583 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 4536, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 147.499339120, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.032517490 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "replayed": 0 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_mem": { 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 14, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "ino+": 2015, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "ino-": 2001, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "dir": 13, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "dir+": 13, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "dir-": 0, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "dn": 17853, 2024-04-25T10:11:02.584 INFO:teuthology.orchestra.run.smithi165.stdout: "dn+": 19854, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "dn-": 2001, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "cap": 2, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "cap+": 9243, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "cap-": 9241, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "rss": 95860, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "heap": 215396 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_server": { 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_client_request": 29524, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "dispatch_server_request": 0, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_request": 25982, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_client_session": 10, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "handle_peer_request": 0, 2024-04-25T10:11:02.585 INFO:teuthology.orchestra.run.smithi165.stdout: "req_create_latency": { 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2000, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 3.883408637, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.001941704 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getattr_latency": { 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 29, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.013006806, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000448510 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getfilelock_latency": { 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "req_link_latency": { 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 6000, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 19.812293539, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.003302048 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookup_latency": { 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 7719, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 16.090234043, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.002084497 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookuphash_latency": { 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.587 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupino_latency": { 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupname_latency": { 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupparent_latency": { 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.588 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lookupsnap_latency": { 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000200615, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000200615 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "req_lssnap_latency": { 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.589 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mkdir_latency": { 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.006478252, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.006478252 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mknod_latency": { 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "req_mksnap_latency": { 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.015964897, 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.015964897 2024-04-25T10:11:02.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "req_open_latency": { 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "req_readdir_latency": { 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.208880503, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.026110062 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rename_latency": { 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 2221, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 468.774135953, 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.211064446 2024-04-25T10:11:02.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "req_renamesnap_latency": { 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "req_snapdiff_latency": { 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmdir_latency": { 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.592 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmsnap_latency": { 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.120012793, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.120012793 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "req_rmxattr_latency": { 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setattr_latency": { 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 1, 2024-04-25T10:11:02.593 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000519228, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000519228 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setdirlayout_latency": { 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setfilelock_latency": { 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setlayout_latency": { 2024-04-25T10:11:02.594 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "req_setxattr_latency": { 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "req_symlink_latency": { 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.595 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "req_unlink_latency": { 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 8000, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 34.216411728, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.004277051 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_revoke_eviction": 0, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "cap_acquisition_throttle": 0, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "req_getvxattr_latency": { 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_sessions": { 2024-04-25T10:11:02.596 INFO:teuthology.orchestra.run.smithi165.stdout: "session_count": 1, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "session_add": 1, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "session_remove": 0, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_open": 1, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "sessions_stale": 0, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "total_load": 13999, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "average_load": 13999, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "avg_session_uptime": 151, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "mdthresh_evicted": 0 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "mempool": { 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_bytes": 0, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "bloom_filter_items": 0, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_bytes": 0, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_alloc_items": 0, 2024-04-25T10:11:02.597 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_bytes": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_data_items": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_onode_items": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_meta_items": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_bytes": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_other_items": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_cache_buffer_items": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_bytes": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_extent_items": 0, 2024-04-25T10:11:02.598 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_blob_items": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_shared_blob_items": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_inline_bl_items": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_fsck_items": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_txc_items": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_deferred_items": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_bytes": 0, 2024-04-25T10:11:02.599 INFO:teuthology.orchestra.run.smithi165.stdout: "bluestore_writing_items": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_bytes": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_items": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_bytes": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_reader_items": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_bytes": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "bluefs_file_writer_items": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_bytes": 250812, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_anon_items": 92, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_bytes": 440, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "buffer_meta_items": 5, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_bytes": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_items": 0, 2024-04-25T10:11:02.600 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_bytes": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_mapbl_items": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_bytes": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_pglog_items": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_bytes": 5528, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_items": 78, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_bytes": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_mapping_items": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_bytes": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "pgmap_items": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_bytes": 13443485, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "mds_co_items": 661102, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_bytes": 0, 2024-04-25T10:11:02.601 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_1_items": 0, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_bytes": 0, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "unittest_2_items": 0 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "objecter": { 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_active": 0, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_laggy": 0, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send": 10637, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_send_bytes": 91628774, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_resend": 29, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_reply": 10637, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "op_latency": { 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10637, 2024-04-25T10:11:02.602 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 61.135500566, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.005747438 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "op_inflight": 0, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "oplen_avg": { 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 10637, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 10805 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "op": 10637, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "op_r": 1947, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "op_w": 8690, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "op_rmw": 0, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "op_pg": 0, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_stat": 0, 2024-04-25T10:11:02.603 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_create": 11, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_read": 102, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_write": 4637, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writefull": 61, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_writesame": 0, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_append": 0, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_zero": 1820, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_truncate": 0, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_delete": 2033, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_mapext": 0, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_sparse_read": 0, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_clonerange": 0, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_getxattr": 1845, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_setxattr": 22, 2024-04-25T10:11:02.604 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_cmpxattr": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_rmxattr": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_resetxattrs": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_call": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_watch": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_notify": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_src_cmpxattr": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_pgls_filter": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "osdop_other": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_active": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_send": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_resend": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "linger_ping": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_active": 0, 2024-04-25T10:11:02.605 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_send": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "poolop_resend": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_active": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_send": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "poolstat_resend": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_active": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_send": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "statfs_resend": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "command_active": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "command_send": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "command_resend": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "map_epoch": 87, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "map_full": 0, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "map_inc": 3, 2024-04-25T10:11:02.606 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_sessions": 8, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_open": 8, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_session_close": 0, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "osd_laggy": 0, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_wr": 232, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_rd": 0, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_del": 42 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "oft": { 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_objs": 1, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_kv_pairs": 0, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_updates": 0, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "omap_total_removes": 0 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "osd-slow-ops": { 2024-04-25T10:11:02.607 INFO:teuthology.orchestra.run.smithi165.stdout: "slow_ops_count": 0 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "purge_queue": { 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops": 0, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_ops_high_water": 32, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing": 0, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executing_high_water": 32, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed_ops": 3820, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_executed": 3820, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "pq_item_in_journal": 0 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:11:02.608 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 38239, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 9853367, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 38239, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 38239, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 9853367, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.609 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_bytes": { 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 104857600, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 10637, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 92029154, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 10637, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 92029154, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-objecter_ops": { 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 1024, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 0, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 0, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 0, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 10637, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 10637, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 10637, 2024-04-25T10:11:02.611 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 10637, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle": { 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 3820, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 400380, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 3820, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:11:02.612 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 101, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 400380, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "throttle-write_buf_throttle-0x55b0cda906e0": { 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "val": 0, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "max": 3758096384, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "get_started": 0, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "get": 26163, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "get_sum": 91022454, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_fail": 0, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "get_or_fail_success": 26163, 2024-04-25T10:11:02.613 INFO:teuthology.orchestra.run.smithi165.stdout: "take": 0, 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "take_sum": 0, 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "put": 4536, 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "put_sum": 91022454, 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "wait": { 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "avgcount": 0, 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "sum": 0.000000000, 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: "avgtime": 0.000000000 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:02.614 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8c406b4d0 msgr2=0x7fe8c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8c406b4d0 0x7fe8c406d990 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fe8d8000c00 tx=0x7fe8d8009040 comp rx=0 tx=0).stop 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 msgr2=0x7fe8e81f3300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 0x7fe8e81f3300 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fe8e80cc090 tx=0x7fe8e000c890 comp rx=0 tx=0).stop 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 shutdown_connections 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8c406b4d0 0x7fe8c406d990 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe8e81f2e80 0x7fe8e81f3300 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe8e80b0f80 0x7fe8e80b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 --2- 172.21.15.165:0/531808214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe8cc004800 0x7fe8e8155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 >> 172.21.15.165:0/531808214 conn(0x7fe8e806abe0 msgr2=0x7fe8e80ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 shutdown_connections 2024-04-25T10:11:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:02.562+0000 7fe8ee000640 1 -- 172.21.15.165:0/531808214 wait complete. 2024-04-25T10:11:02.617 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:02.618 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:02.726 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:02.842 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:02.842 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:11:02.842 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:02.899 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:11:02.899 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:02.954 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:03.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.157+0000 7f8aba000640 1 Processor -- start 2024-04-25T10:11:03.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.157+0000 7f8aba000640 1 -- start start 2024-04-25T10:11:03.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.158+0000 7f8aba000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8ab406e870 0x7f8ab40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.158+0000 7f8aba000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab4144240 con 0x7f8ab40cc840 2024-04-25T10:11:03.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.158+0000 7f8aba000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ab415e220 con 0x7f8ab4143e60 2024-04-25T10:11:03.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.158+0000 7f8aba000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ab415e500 con 0x7f8ab406e870 2024-04-25T10:11:03.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab9600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8ab40cc840 0x7f8ab415ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36658/0 (socket says 172.21.15.165:36658) 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab9600640 1 -- 172.21.15.165:0/325021463 learned_addr learned my addr 172.21.15.165:0/325021463 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424124752 0 0) 0x7f8ab4144240 con 0x7f8ab40cc840 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a98005730 con 0x7f8ab40cc840 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3e00640 1 --2- 172.21.15.165:0/325021463 >> v2:172.21.15.172:3300/0 conn(0x7f8ab406e870 0x7f8ab40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 574083632 0 0) 0x7f8a98005730 con 0x7f8ab40cc840 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 >> v2:172.21.15.172:3300/0 conn(0x7f8ab406e870 msgr2=0x7f8ab40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 --2- 172.21.15.165:0/325021463 >> v2:172.21.15.172:3300/0 conn(0x7f8ab406e870 0x7f8ab40cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 >> v1:172.21.15.172:6789/0 conn(0x7f8ab4143e60 legacy=0x7f8ab406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab415e7e0 con 0x7f8ab40cc840 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8aac001670 con 0x7f8ab40cc840 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 >> v1:172.21.15.165:6789/0 conn(0x7f8ab40cc840 legacy=0x7f8ab415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.159+0000 7f8ab3400640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a98005bc0 0x7f8a98005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3400640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a98006a10 0x7f8a98006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3400640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8a9800dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab415e500 con 0x7f8a980073d0 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9800e320 con 0x7f8a98005bc0 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9800e520 con 0x7f8a98006a10 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab9600640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8a9800dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab8c00640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a98005bc0 0x7f8a98005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3e00640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a98006a10 0x7f8a98006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab9600640 1 -- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a98006a10 msgr2=0x7f8a98006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab9600640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a98006a10 0x7f8a98006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab9600640 1 -- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a98005bc0 msgr2=0x7f8a98005fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab9600640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a98005bc0 0x7f8a98005fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab9600640 1 -- 172.21.15.165:0/325021463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98005730 con 0x7f8a980073d0 2024-04-25T10:11:03.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.160+0000 7f8ab3e00640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a98006a10 0x7f8a98006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:03.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.161+0000 7f8ab9600640 1 -- 172.21.15.165:0/325021463 reap_dead start 2024-04-25T10:11:03.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.161+0000 7f8ab9600640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8a9800dbd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c240 tx=0x7f8a98007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b916cc6dab362cd8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007d10 con 0x7f8a980073d0 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac005ad0 con 0x7f8a980073d0 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8ab3400640 1 -- 172.21.15.165:0/325021463 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac007070 con 0x7f8a980073d0 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 -- 172.21.15.165:0/325021463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 msgr2=0x7f8a9800dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8a9800dbd0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f8aac00c240 tx=0x7f8a98007a40 comp rx=0 tx=0).stop 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 -- 172.21.15.165:0/325021463 shutdown_connections 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 --2- 172.21.15.165:0/325021463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8a9800dbd0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 -- 172.21.15.165:0/325021463 >> 172.21.15.165:0/325021463 conn(0x7f8ab406abe0 msgr2=0x7f8ab406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 -- 172.21.15.165:0/325021463 shutdown_connections 2024-04-25T10:11:03.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.162+0000 7f8aba000640 1 -- 172.21.15.165:0/325021463 wait complete. 2024-04-25T10:11:03.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.164+0000 7f8aba000640 1 Processor -- start 2024-04-25T10:11:03.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.164+0000 7f8aba000640 1 -- start start 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8aba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8ab4159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8aba000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ab414acf0 0x7f8ab414b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8aba000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ab4148f70 0x7f8ab41493f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8aba000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40b2eb0 con 0x7f8a980073d0 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8aba000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40b2d30 con 0x7f8ab4148f70 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8aba000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ab40ccca0 con 0x7f8ab414acf0 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.165+0000 7f8ab8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8ab4159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8ab4159d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59888/0 (socket says 172.21.15.165:59888) 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 -- 172.21.15.165:0/1105547666 learned_addr learned my addr 172.21.15.165:0/1105547666 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab9600640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ab4148f70 0x7f8ab41493f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab3e00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ab414acf0 0x7f8ab414b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 -- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ab414acf0 msgr2=0x7f8ab414b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ab414acf0 0x7f8ab414b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 -- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ab4148f70 msgr2=0x7f8ab41493f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ab4148f70 0x7f8ab41493f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 -- 172.21.15.165:0/1105547666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab4149ca0 con 0x7f8a980073d0 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab8c00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8ab4159d40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e820 tx=0x7f8aa400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.166+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa4003b80 con 0x7f8a980073d0 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.167+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa4018070 con 0x7f8a980073d0 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.167+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa40050d0 con 0x7f8a980073d0 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.167+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ab4149f80 con 0x7f8a980073d0 2024-04-25T10:11:03.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.167+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ab414a540 con 0x7f8a980073d0 2024-04-25T10:11:03.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.168+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ab4073990 con 0x7f8a980073d0 2024-04-25T10:11:03.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.169+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8aa4005270 con 0x7f8a980073d0 2024-04-25T10:11:03.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.170+0000 7f8ab0c00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9406b470 0x7f8a9406d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.170+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f8aa408a440 con 0x7f8a980073d0 2024-04-25T10:11:03.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.170+0000 7f8ab3e00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9406b470 0x7f8a9406d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.171+0000 7f8ab3e00640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9406b470 0x7f8a9406d930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4140640 tx=0x7f8aa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.175+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8aa401a310 con 0x7f8a980073d0 2024-04-25T10:11:03.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.374+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ab40ceb60 con 0x7f8a980073d0 2024-04-25T10:11:03.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.376+0000 7f8ab0c00640 1 -- 172.21.15.165:0/1105547666 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+17047 (secure 0 0 0) 0x7f8aa4059080 con 0x7f8a980073d0 2024-04-25T10:11:03.383 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:03.383 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":87,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:10:39.107124+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.379+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9406b470 msgr2=0x7f8a9406d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.379+0000 7f8aba000640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9406b470 0x7f8a9406d930 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f8ab4140640 tx=0x7f8aa0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.379+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 msgr2=0x7f8ab4159d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.379+0000 7f8aba000640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8ab4159d40 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f8aa400e820 tx=0x7f8aa400ece0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 shutdown_connections 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9406b470 0x7f8a9406d930 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ab4148f70 0x7f8ab41493f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ab414acf0 0x7f8ab414b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 --2- 172.21.15.165:0/1105547666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a980073d0 0x7f8ab4159d40 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 >> 172.21.15.165:0/1105547666 conn(0x7f8ab406abe0 msgr2=0x7f8ab415bd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 shutdown_connections 2024-04-25T10:11:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.380+0000 7f8aba000640 1 -- 172.21.15.165:0/1105547666 wait complete. 2024-04-25T10:11:03.405 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:17.127559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:18.397615+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:03.406 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:03.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.600+0000 7f5bc4c00640 1 Processor -- start 2024-04-25T10:11:03.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.601+0000 7f5bc4c00640 1 -- start start 2024-04-25T10:11:03.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bc4c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5bc006bff0 0x7f5bc00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bc4c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc015e110 con 0x7f5bc00b5970 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bc4c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc00b2fd0 con 0x7f5bc015dd30 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bc4c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc006d000 con 0x7f5bc006bff0 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bb7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5bc006bff0 0x7f5bc00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bb7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5bc006bff0 0x7f5bc00b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59896/0 (socket says 172.21.15.165:59896) 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bb7e00640 1 -- 172.21.15.165:0/4082732079 learned_addr learned my addr 172.21.15.165:0/4082732079 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:03.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.602+0000 7f5bbfe00640 1 --1- 172.21.15.165:0/4082732079 >> v1:172.21.15.165:6789/0 conn(0x7f5bc00b5970 0x7f5bc00d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36664/0 (socket says 172.21.15.165:36664) 2024-04-25T10:11:03.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.603+0000 7f5bb7e00640 1 -- 172.21.15.165:0/4082732079 >> v1:172.21.15.172:6789/0 conn(0x7f5bc015dd30 legacy=0x7f5bc00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:03.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.603+0000 7f5bb7e00640 1 -- 172.21.15.165:0/4082732079 >> v1:172.21.15.165:6789/0 conn(0x7f5bc00b5970 legacy=0x7f5bc00d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.603+0000 7f5bb7e00640 1 -- 172.21.15.165:0/4082732079 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc006c480 con 0x7f5bc006bff0 2024-04-25T10:11:03.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.603+0000 7f5bbea00640 1 -- 172.21.15.165:0/4082732079 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142901579 0 0) 0x7f5bc015e110 con 0x7f5bc00b5970 2024-04-25T10:11:03.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.604+0000 7f5bb7e00640 1 --2- 172.21.15.165:0/4082732079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc006bff0 0x7f5bc00b5430 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009870 tx=0x7f5ba802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ac7cfe2fd24a455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.604+0000 7f5bbea00640 1 -- 172.21.15.165:0/4082732079 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8037070 con 0x7f5bc006bff0 2024-04-25T10:11:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.604+0000 7f5bbea00640 1 -- 172.21.15.165:0/4082732079 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba8032440 con 0x7f5bc006bff0 2024-04-25T10:11:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.604+0000 7f5bbea00640 1 -- 172.21.15.165:0/4082732079 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8032810 con 0x7f5bc006bff0 2024-04-25T10:11:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 -- 172.21.15.165:0/4082732079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc006bff0 msgr2=0x7f5bc00b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/4082732079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc006bff0 0x7f5bc00b5430 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8009870 tx=0x7f5ba802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 -- 172.21.15.165:0/4082732079 shutdown_connections 2024-04-25T10:11:03.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/4082732079 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc006bff0 0x7f5bc00b5430 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 -- 172.21.15.165:0/4082732079 >> 172.21.15.165:0/4082732079 conn(0x7f5bc00d0610 msgr2=0x7f5bc00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 -- 172.21.15.165:0/4082732079 shutdown_connections 2024-04-25T10:11:03.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.605+0000 7f5bc4c00640 1 -- 172.21.15.165:0/4082732079 wait complete. 2024-04-25T10:11:03.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.607+0000 7f5bc4c00640 1 Processor -- start 2024-04-25T10:11:03.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.607+0000 7f5bc4c00640 1 -- start start 2024-04-25T10:11:03.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bc4c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc00b5970 0x7f5bc00cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bc4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 0x7f5bc01f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bc4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc00b84d0 0x7f5bc0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bc4c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc006cd20 con 0x7f5bc015dd30 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bc4c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc006ca40 con 0x7f5bc00b5970 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bc4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc00d2870 con 0x7f5bc00b84d0 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 0x7f5bc01f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bb7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 0x7f5bc01f36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59904/0 (socket says 172.21.15.165:59904) 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.608+0000 7f5bb7e00640 1 -- 172.21.15.165:0/2075267133 learned_addr learned my addr 172.21.15.165:0/2075267133 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bb7e00640 1 -- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc00b84d0 msgr2=0x7f5bc0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bbfe00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc00b84d0 0x7f5bc0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bb7e00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc00b84d0 0x7f5bc0159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bb7e00640 1 -- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc00b5970 msgr2=0x7f5bc00cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bbf400640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc00b5970 0x7f5bc00cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bb7e00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc00b5970 0x7f5bc00cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bb7e00640 1 -- 172.21.15.165:0/2075267133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc01f3c30 con 0x7f5bc015dd30 2024-04-25T10:11:03.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bbfe00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc00b84d0 0x7f5bc0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.609+0000 7f5bb7e00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 0x7f5bc01f36f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8000c00 tx=0x7f5ba80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.610+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8041070 con 0x7f5bc015dd30 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.610+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba803c870 con 0x7f5bc015dd30 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.610+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba8037040 con 0x7f5bc015dd30 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.610+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc01f3f00 con 0x7f5bc015dd30 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.610+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc01f44c0 con 0x7f5bc015dd30 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.611+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc0073990 con 0x7f5bc015dd30 2024-04-25T10:11:03.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.612+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ba8044070 con 0x7f5bc015dd30 2024-04-25T10:11:03.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.613+0000 7f5bb7400640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:03.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.613+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5ba80aaf90 con 0x7f5bc015dd30 2024-04-25T10:11:03.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.614+0000 7f5bbf400640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:03.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.614+0000 7f5bbf400640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000a750 tx=0x7f5bb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:03.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.618+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ba8079c50 con 0x7f5bc015dd30 2024-04-25T10:11:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.813+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5bc00d16c0 con 0x7f5bc015dd30 2024-04-25T10:11:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.813+0000 7f5bb7400640 1 -- 172.21.15.165:0/2075267133 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ba8079550 con 0x7f5bc015dd30 2024-04-25T10:11:03.820 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b9c06b4d0 msgr2=0x7f5b9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5bb000a750 tx=0x7f5bb0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 msgr2=0x7f5bc01f36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 0x7f5bc01f36f0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f5ba8000c00 tx=0x7f5ba80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 shutdown_connections 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b9c06b4d0 0x7f5b9c06d990 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5bc00b84d0 0x7f5bc0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5bc015dd30 0x7f5bc01f36f0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 --2- 172.21.15.165:0/2075267133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5bc00b5970 0x7f5bc00cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 >> 172.21.15.165:0/2075267133 conn(0x7f5bc00d0610 msgr2=0x7f5bc00b6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.816+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 shutdown_connections 2024-04-25T10:11:03.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:03.817+0000 7f5bc4c00640 1 -- 172.21.15.165:0/2075267133 wait complete. 2024-04-25T10:11:03.842 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:04.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.037+0000 7f2095a00640 1 Processor -- start 2024-04-25T10:11:04.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.037+0000 7f2095a00640 1 -- start start 2024-04-25T10:11:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f20900b0e80 0x7f20900711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f209015ae90 con 0x7f2090154730 2024-04-25T10:11:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f209015b450 con 0x7f20900b0e80 2024-04-25T10:11:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f209015ac30 con 0x7f2090154350 2024-04-25T10:11:04.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f20900b0e80 0x7f20900711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f20900b0e80 0x7f20900711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59914/0 (socket says 172.21.15.165:59914) 2024-04-25T10:11:04.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.039+0000 7f2095000640 1 -- 172.21.15.165:0/1987640345 learned_addr learned my addr 172.21.15.165:0/1987640345 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:04.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.040+0000 7f2095000640 1 -- 172.21.15.165:0/1987640345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 msgr2=0x7f209006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.040+0000 7f208fe00640 1 --2- 172.21.15.165:0/1987640345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.040+0000 7f2095000640 1 --2- 172.21.15.165:0/1987640345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209006db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.040+0000 7f2095000640 1 -- 172.21.15.165:0/1987640345 >> v1:172.21.15.172:6789/0 conn(0x7f2090154730 legacy=0x7f209015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.040+0000 7f2095000640 1 -- 172.21.15.165:0/1987640345 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f209015b170 con 0x7f20900b0e80 2024-04-25T10:11:04.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.041+0000 7f2095000640 1 --2- 172.21.15.165:0/1987640345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f20900711d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f2088009e50 tx=0x7f208802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731a1d7422a09dd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.041+0000 7f208ea00640 1 -- 172.21.15.165:0/1987640345 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2088036070 con 0x7f20900b0e80 2024-04-25T10:11:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.041+0000 7f208ea00640 1 -- 172.21.15.165:0/1987640345 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2088031600 con 0x7f20900b0e80 2024-04-25T10:11:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.041+0000 7f208ea00640 1 -- 172.21.15.165:0/1987640345 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20880319d0 con 0x7f20900b0e80 2024-04-25T10:11:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 -- 172.21.15.165:0/1987640345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 msgr2=0x7f20900711d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 --2- 172.21.15.165:0/1987640345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f20900711d0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f2088009e50 tx=0x7f208802a5b0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 -- 172.21.15.165:0/1987640345 shutdown_connections 2024-04-25T10:11:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 --2- 172.21.15.165:0/1987640345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f20900711d0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 --2- 172.21.15.165:0/1987640345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209006db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 -- 172.21.15.165:0/1987640345 >> 172.21.15.165:0/1987640345 conn(0x7f2090061cc0 msgr2=0x7f209006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 -- 172.21.15.165:0/1987640345 shutdown_connections 2024-04-25T10:11:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.042+0000 7f2095a00640 1 -- 172.21.15.165:0/1987640345 wait complete. 2024-04-25T10:11:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.043+0000 7f2095a00640 1 Processor -- start 2024-04-25T10:11:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.044+0000 7f2095a00640 1 -- start start 2024-04-25T10:11:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f2095a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f209014ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f2095a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209014ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f2095a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2090154730 0x7f209015df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f2095a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209006e060 con 0x7f20900b0e80 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f2095a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209006d600 con 0x7f2090154730 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f2095a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2090071710 con 0x7f2090154350 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f209014ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f209014ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59928/0 (socket says 172.21.15.165:59928) 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 -- 172.21.15.165:0/4251831787 learned_addr learned my addr 172.21.15.165:0/4251831787 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 -- 172.21.15.165:0/4251831787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 msgr2=0x7f209014ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209014ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 -- 172.21.15.165:0/4251831787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2090154730 msgr2=0x7f209015df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2090154730 0x7f209015df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.045+0000 7f208fe00640 1 -- 172.21.15.165:0/4251831787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2090138fd0 con 0x7f20900b0e80 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.046+0000 7f208fe00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f209014ea60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f208000c7f0 tx=0x7f208000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.046+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080015070 con 0x7f20900b0e80 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.046+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2080004240 con 0x7f20900b0e80 2024-04-25T10:11:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.046+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2080003610 con 0x7f20900b0e80 2024-04-25T10:11:04.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.046+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2090139270 con 0x7f20900b0e80 2024-04-25T10:11:04.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.046+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2090139830 con 0x7f20900b0e80 2024-04-25T10:11:04.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.047+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2090068490 con 0x7f20900b0e80 2024-04-25T10:11:04.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.048+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20800039c0 con 0x7f20900b0e80 2024-04-25T10:11:04.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.049+0000 7f2087e00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f207006b4d0 0x7f207006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.049+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f208008a770 con 0x7f20900b0e80 2024-04-25T10:11:04.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.050+0000 7f208f400640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f207006b4d0 0x7f207006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.050+0000 7f208f400640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f207006b4d0 0x7f207006d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f207c006fd0 tx=0x7f207c006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.053+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2090068490 con 0x7f20900b0e80 2024-04-25T10:11:04.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.251+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2090073990 con 0x7f20900b0e80 2024-04-25T10:11:04.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.252+0000 7f2087e00640 1 -- 172.21.15.165:0/4251831787 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2080059440 con 0x7f20900b0e80 2024-04-25T10:11:04.259 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:11:04.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f207006b4d0 msgr2=0x7f207006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f207006b4d0 0x7f207006d990 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f207c006fd0 tx=0x7f207c006bc0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 msgr2=0x7f209014ea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f209014ea60 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f208000c7f0 tx=0x7f208000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 shutdown_connections 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f207006b4d0 0x7f207006d990 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2090154730 0x7f209015df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2090154350 0x7f209014ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 --2- 172.21.15.165:0/4251831787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20900b0e80 0x7f209014ea60 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 >> 172.21.15.165:0/4251831787 conn(0x7f2090061cc0 msgr2=0x7f20901534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 shutdown_connections 2024-04-25T10:11:04.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.255+0000 7f2095a00640 1 -- 172.21.15.165:0/4251831787 wait complete. 2024-04-25T10:11:04.280 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:04.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.471+0000 7f3f05a00640 1 Processor -- start 2024-04-25T10:11:04.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.471+0000 7f3f05a00640 1 -- start start 2024-04-25T10:11:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3f05a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f001446d0 0x7f3f000b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3f05a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f00144d50 0x7f3f000b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3f05a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f001452a0 0x7f3f000cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3f05a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f000b2b20 con 0x7f3f00144d50 2024-04-25T10:11:04.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3f05a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f0006d400 con 0x7f3f001446d0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3f05a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000b49a0 con 0x7f3f001452a0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3ef6000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f00144d50 0x7f3f000b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3ef6000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f00144d50 0x7f3f000b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59942/0 (socket says 172.21.15.165:59942) 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.473+0000 7f3ef6000640 1 -- 172.21.15.165:0/707148251 learned_addr learned my addr 172.21.15.165:0/707148251 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3eff400640 1 --2- 172.21.15.165:0/707148251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f001452a0 0x7f3f000cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3efea00640 1 --2- 172.21.15.165:0/707148251 >> v2:172.21.15.172:3300/0 conn(0x7f3f001446d0 0x7f3f000b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3ef6000640 1 -- 172.21.15.165:0/707148251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f001452a0 msgr2=0x7f3f000cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3ef6000640 1 --2- 172.21.15.165:0/707148251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f001452a0 0x7f3f000cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3ef6000640 1 -- 172.21.15.165:0/707148251 >> v2:172.21.15.172:3300/0 conn(0x7f3f001446d0 msgr2=0x7f3f000b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3ef6000640 1 --2- 172.21.15.165:0/707148251 >> v2:172.21.15.172:3300/0 conn(0x7f3f001446d0 0x7f3f000b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.474+0000 7f3ef6000640 1 -- 172.21.15.165:0/707148251 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000cccc0 con 0x7f3f00144d50 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.475+0000 7f3ef6000640 1 --2- 172.21.15.165:0/707148251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00144d50 0x7f3f000b4460 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3f000cf560 tx=0x7f3ee802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a216633a4f4837d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.475+0000 7f3efe000640 1 -- 172.21.15.165:0/707148251 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8037070 con 0x7f3f00144d50 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.475+0000 7f3efe000640 1 -- 172.21.15.165:0/707148251 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee8032440 con 0x7f3f00144d50 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.475+0000 7f3efe000640 1 -- 172.21.15.165:0/707148251 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee8032810 con 0x7f3f00144d50 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 -- 172.21.15.165:0/707148251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00144d50 msgr2=0x7f3f000b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 --2- 172.21.15.165:0/707148251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00144d50 0x7f3f000b4460 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7f3f000cf560 tx=0x7f3ee802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 -- 172.21.15.165:0/707148251 shutdown_connections 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 --2- 172.21.15.165:0/707148251 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f001452a0 0x7f3f000cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 --2- 172.21.15.165:0/707148251 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f00144d50 0x7f3f000b4460 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 --2- 172.21.15.165:0/707148251 >> v2:172.21.15.172:3300/0 conn(0x7f3f001446d0 0x7f3f000b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 -- 172.21.15.165:0/707148251 >> 172.21.15.165:0/707148251 conn(0x7f3f0006abe0 msgr2=0x7f3f0006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 -- 172.21.15.165:0/707148251 shutdown_connections 2024-04-25T10:11:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.476+0000 7f3f05a00640 1 -- 172.21.15.165:0/707148251 wait complete. 2024-04-25T10:11:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.478+0000 7f3f05a00640 1 Processor -- start 2024-04-25T10:11:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.478+0000 7f3f05a00640 1 -- start start 2024-04-25T10:11:04.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3f05a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 0x7f3f000adbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3f05a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f00144d50 0x7f3f000ce9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3f05a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f001452a0 0x7f3f000cef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3f05a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000cc700 con 0x7f3f001446d0 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3f05a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000cc420 con 0x7f3f001452a0 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3f05a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000cc9e0 con 0x7f3f00144d50 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.479+0000 7f3efea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 0x7f3f000adbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 0x7f3f000adbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59952/0 (socket says 172.21.15.165:59952) 2024-04-25T10:11:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 -- 172.21.15.165:0/983622066 learned_addr learned my addr 172.21.15.165:0/983622066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3ef6000640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f00144d50 0x7f3f000ce9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 -- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f00144d50 msgr2=0x7f3f000ce9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f00144d50 0x7f3f000ce9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 -- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f001452a0 msgr2=0x7f3f000cef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f001452a0 0x7f3f000cef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 -- 172.21.15.165:0/983622066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f001e3700 con 0x7f3f001446d0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.480+0000 7f3efea00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 0x7f3f000adbc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0007e60 tx=0x7f3ef000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.481+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef0014620 con 0x7f3f001446d0 2024-04-25T10:11:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.481+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef0019070 con 0x7f3f001446d0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.481+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef00149e0 con 0x7f3f001446d0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.481+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f001e39e0 con 0x7f3f001446d0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.481+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f001e3fa0 con 0x7f3f001446d0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.482+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f00073a00 con 0x7f3f001446d0 2024-04-25T10:11:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.483+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ef0014b80 con 0x7f3f001446d0 2024-04-25T10:11:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.484+0000 7f3ef7400640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3edc06b4d0 0x7f3edc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.485+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f3ef008ac20 con 0x7f3f001446d0 2024-04-25T10:11:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.485+0000 7f3ef6000640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3edc06b4d0 0x7f3edc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.485+0000 7f3ef6000640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3edc06b4d0 0x7f3edc06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8034000 tx=0x7f3ee802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.489+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3ef00598e0 con 0x7f3f001446d0 2024-04-25T10:11:04.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.671+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3f000b1780 con 0x7f3f001446d0 2024-04-25T10:11:04.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.672+0000 7f3ef7400640 1 -- 172.21.15.165:0/983622066 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3ef00591e0 con 0x7f3f001446d0 2024-04-25T10:11:04.679 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:11:04.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3edc06b4d0 msgr2=0x7f3edc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3edc06b4d0 0x7f3edc06d990 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8034000 tx=0x7f3ee802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:04.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 msgr2=0x7f3f000adbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 0x7f3f000adbc0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0007e60 tx=0x7f3ef000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 shutdown_connections 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3edc06b4d0 0x7f3edc06d990 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f001452a0 0x7f3f000cef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f00144d50 0x7f3f000ce9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 --2- 172.21.15.165:0/983622066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f001446d0 0x7f3f000adbc0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 >> 172.21.15.165:0/983622066 conn(0x7f3f0006abe0 msgr2=0x7f3f00143900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 shutdown_connections 2024-04-25T10:11:04.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.675+0000 7f3f05a00640 1 -- 172.21.15.165:0/983622066 wait complete. 2024-04-25T10:11:04.699 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:11:04.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.887+0000 7f311b200640 1 Processor -- start 2024-04-25T10:11:04.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.888+0000 7f311b200640 1 -- start start 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.889+0000 7f311b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.889+0000 7f311b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f311c0ce950 0x7f311c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.889+0000 7f311b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f311c145520 con 0x7f311c0c5030 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.889+0000 7f311b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f311c0ced90 con 0x7f311c0ce950 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.889+0000 7f311b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c071270 con 0x7f311c145140 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f311c0ce950 0x7f311c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f311c0ce950 0x7f311c0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59960/0 (socket says 172.21.15.165:59960) 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 -- 172.21.15.165:0/3580303972 learned_addr learned my addr 172.21.15.165:0/3580303972 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:04.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f311a800640 1 --1- 172.21.15.165:0/3580303972 >> v1:172.21.15.165:6789/0 conn(0x7f311c0c5030 0x7f311c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36666/0 (socket says 172.21.15.165:36666) 2024-04-25T10:11:04.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3119e00640 1 --2- 172.21.15.165:0/3580303972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 -- 172.21.15.165:0/3580303972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 msgr2=0x7f311c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:04.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 --2- 172.21.15.165:0/3580303972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 -- 172.21.15.165:0/3580303972 >> v1:172.21.15.165:6789/0 conn(0x7f311c0c5030 legacy=0x7f311c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3111400640 1 -- 172.21.15.165:0/3580303972 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c071550 con 0x7f311c0ce950 2024-04-25T10:11:04.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.890+0000 7f3119400640 1 -- 172.21.15.165:0/3580303972 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 681012529 0 0) 0x7f311c145520 con 0x7f311c0c5030 2024-04-25T10:11:04.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.891+0000 7f3111400640 1 --2- 172.21.15.165:0/3580303972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0ce950 0x7f311c0c4af0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3104009870 tx=0x7f310402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891a4ceb16f8d51b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f3119400640 1 -- 172.21.15.165:0/3580303972 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104037070 con 0x7f311c0ce950 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f3119400640 1 -- 172.21.15.165:0/3580303972 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3104032440 con 0x7f311c0ce950 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f3119400640 1 -- 172.21.15.165:0/3580303972 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3104032810 con 0x7f311c0ce950 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 -- 172.21.15.165:0/3580303972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0ce950 msgr2=0x7f311c0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 --2- 172.21.15.165:0/3580303972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0ce950 0x7f311c0c4af0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f3104009870 tx=0x7f310402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 -- 172.21.15.165:0/3580303972 shutdown_connections 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 --2- 172.21.15.165:0/3580303972 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0ce950 0x7f311c0c4af0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 --2- 172.21.15.165:0/3580303972 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 -- 172.21.15.165:0/3580303972 >> 172.21.15.165:0/3580303972 conn(0x7f311c061cc0 msgr2=0x7f311c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.892+0000 7f311b200640 1 -- 172.21.15.165:0/3580303972 shutdown_connections 2024-04-25T10:11:04.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.893+0000 7f311b200640 1 -- 172.21.15.165:0/3580303972 wait complete. 2024-04-25T10:11:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.894+0000 7f311b200640 1 Processor -- start 2024-04-25T10:11:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.894+0000 7f311b200640 1 -- start start 2024-04-25T10:11:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f311b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 0x7f311c1f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f311b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f311c0ce950 0x7f311c143220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f311b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f311b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0c54a0 con 0x7f311c0c5030 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f311b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f311c071b10 con 0x7f311c0ce950 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f311b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f311c0c5620 con 0x7f311c145140 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f3119e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 0x7f311c1f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f3119e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 0x7f311c1f6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59962/0 (socket says 172.21.15.165:59962) 2024-04-25T10:11:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.895+0000 7f3119e00640 1 -- 172.21.15.165:0/2784848071 learned_addr learned my addr 172.21.15.165:0/2784848071 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:04.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3111400640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f311c0ce950 0x7f311c143220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f311a800640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3119e00640 1 -- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 msgr2=0x7f311c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3119e00640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c15b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3119e00640 1 -- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f311c0ce950 msgr2=0x7f311c143220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3119e00640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f311c0ce950 0x7f311c143220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3119e00640 1 -- 172.21.15.165:0/2784848071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f311c1f53a0 con 0x7f311c0c5030 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3111400640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f311c0ce950 0x7f311c143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3119e00640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 0x7f311c1f6440 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f310c00d5f0 tx=0x7f310c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.896+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c0125f0 con 0x7f311c0c5030 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.897+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f310c019070 con 0x7f311c0c5030 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.897+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f310c0129d0 con 0x7f311c0c5030 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.897+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f311c1f5680 con 0x7f311c0c5030 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.897+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f311c1f5c40 con 0x7f311c0c5030 2024-04-25T10:11:04.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.898+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f311c073990 con 0x7f311c0c5030 2024-04-25T10:11:04.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.899+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f310c012b70 con 0x7f311c0c5030 2024-04-25T10:11:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.900+0000 7f3112a00640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30f806b4d0 0x7f30f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.900+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f310c08a840 con 0x7f311c0c5030 2024-04-25T10:11:04.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.901+0000 7f3111400640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30f806b4d0 0x7f30f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:04.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.901+0000 7f3111400640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30f806b4d0 0x7f30f806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3104034000 tx=0x7f310402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:04.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:04.904+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f310c059470 con 0x7f311c0c5030 2024-04-25T10:11:05.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.111+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f311c15bbc0 con 0x7f311c0c5030 2024-04-25T10:11:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.112+0000 7f3112a00640 1 -- 172.21.15.165:0/2784848071 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v87) v1 ==== 80+0+0 (secure 0 0 0) 0x7f310c058d70 con 0x7f311c0c5030 2024-04-25T10:11:05.118 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:11:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30f806b4d0 msgr2=0x7f30f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30f806b4d0 0x7f30f806d990 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f3104034000 tx=0x7f310402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 msgr2=0x7f311c1f6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 0x7f311c1f6440 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f310c00d5f0 tx=0x7f310c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 shutdown_connections 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30f806b4d0 0x7f30f806d990 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f311c145140 0x7f311c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f311c0ce950 0x7f311c143220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 --2- 172.21.15.165:0/2784848071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f311c0c5030 0x7f311c1f6440 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 >> 172.21.15.165:0/2784848071 conn(0x7f311c061cc0 msgr2=0x7f311c0caed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 shutdown_connections 2024-04-25T10:11:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.115+0000 7f311b200640 1 -- 172.21.15.165:0/2784848071 wait complete. 2024-04-25T10:11:05.137 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:11:05.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:05.137 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:05.239 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:11:05.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:05.240 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:11:05.286 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:11:05.286 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:05.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:05.327 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:11:05.327 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:05.367 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:05.367 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:11:05.367 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:05.414 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:05.414 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.607+0000 7fce85a00640 1 Processor -- start 2024-04-25T10:11:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.607+0000 7fce85a00640 1 -- start start 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce85a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fce80144750 0x7fce8006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce85a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce8006eb10 0x7fce800ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce85a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce800cee60 0x7fce8015f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce85a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce80144b30 con 0x7fce8006eb10 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce85a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce800712d0 con 0x7fce80144750 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce85a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce800715b0 con 0x7fce800cee60 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce7e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce8006eb10 0x7fce800ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce7e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce8006eb10 0x7fce800ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59964/0 (socket says 172.21.15.165:59964) 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.609+0000 7fce7e000640 1 -- 172.21.15.165:0/305211440 learned_addr learned my addr 172.21.15.165:0/305211440 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7ea00640 1 --2- 172.21.15.165:0/305211440 >> v2:172.21.15.172:3300/0 conn(0x7fce80144750 0x7fce8006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7e000640 1 -- 172.21.15.165:0/305211440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce800cee60 msgr2=0x7fce8015f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7f400640 1 --2- 172.21.15.165:0/305211440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce800cee60 0x7fce8015f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7e000640 1 --2- 172.21.15.165:0/305211440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce800cee60 0x7fce8015f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7e000640 1 -- 172.21.15.165:0/305211440 >> v2:172.21.15.172:3300/0 conn(0x7fce80144750 msgr2=0x7fce8006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7e000640 1 --2- 172.21.15.165:0/305211440 >> v2:172.21.15.172:3300/0 conn(0x7fce80144750 0x7fce8006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7e000640 1 -- 172.21.15.165:0/305211440 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80071890 con 0x7fce8006eb10 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.610+0000 7fce7ea00640 1 --2- 172.21.15.165:0/305211440 >> v2:172.21.15.172:3300/0 conn(0x7fce80144750 0x7fce8006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.611+0000 7fce7e000640 1 --2- 172.21.15.165:0/305211440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8006eb10 0x7fce800ce920 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fce8006f5d0 tx=0x7fce6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b804f847bf0dd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.611+0000 7fce7d600640 1 -- 172.21.15.165:0/305211440 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c037070 con 0x7fce8006eb10 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.611+0000 7fce7d600640 1 -- 172.21.15.165:0/305211440 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c032440 con 0x7fce8006eb10 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.611+0000 7fce7d600640 1 -- 172.21.15.165:0/305211440 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c032810 con 0x7fce8006eb10 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 -- 172.21.15.165:0/305211440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8006eb10 msgr2=0x7fce800ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 --2- 172.21.15.165:0/305211440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8006eb10 0x7fce800ce920 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fce8006f5d0 tx=0x7fce6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 -- 172.21.15.165:0/305211440 shutdown_connections 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 --2- 172.21.15.165:0/305211440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce800cee60 0x7fce8015f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 --2- 172.21.15.165:0/305211440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8006eb10 0x7fce800ce920 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 --2- 172.21.15.165:0/305211440 >> v2:172.21.15.172:3300/0 conn(0x7fce80144750 0x7fce8006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 -- 172.21.15.165:0/305211440 >> 172.21.15.165:0/305211440 conn(0x7fce8006abe0 msgr2=0x7fce8006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 -- 172.21.15.165:0/305211440 shutdown_connections 2024-04-25T10:11:05.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.612+0000 7fce85a00640 1 -- 172.21.15.165:0/305211440 wait complete. 2024-04-25T10:11:05.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.615+0000 7fce85a00640 1 Processor -- start 2024-04-25T10:11:05.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.615+0000 7fce85a00640 1 -- start start 2024-04-25T10:11:05.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce85a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8006eb10 0x7fce800af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce85a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 0x7fce80142890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce85a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce80144750 0x7fce80142dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce85a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce80145450 con 0x7fce800cee60 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce85a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8006f020 con 0x7fce8006eb10 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce85a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce801455d0 con 0x7fce80144750 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce7e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 0x7fce80142890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce7e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 0x7fce80142890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59972/0 (socket says 172.21.15.165:59972) 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.616+0000 7fce7e000640 1 -- 172.21.15.165:0/1395554217 learned_addr learned my addr 172.21.15.165:0/1395554217 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7ea00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8006eb10 0x7fce800af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7f400640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce80144750 0x7fce80142dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7e000640 1 -- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce80144750 msgr2=0x7fce80142dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7e000640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce80144750 0x7fce80142dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7e000640 1 -- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8006eb10 msgr2=0x7fce800af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7e000640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8006eb10 0x7fce800af990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7e000640 1 -- 172.21.15.165:0/1395554217 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce800afed0 con 0x7fce800cee60 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7f400640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce80144750 0x7fce80142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.617+0000 7fce7e000640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 0x7fce80142890 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000c00 tx=0x7fce6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.618+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c037040 con 0x7fce800cee60 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.618+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6c042070 con 0x7fce800cee60 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.618+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6c030bb0 con 0x7fce800cee60 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.618+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce80138bb0 con 0x7fce800cee60 2024-04-25T10:11:05.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.618+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce80139170 con 0x7fce800cee60 2024-04-25T10:11:05.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.619+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce80073990 con 0x7fce800cee60 2024-04-25T10:11:05.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.620+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce6c047460 con 0x7fce800cee60 2024-04-25T10:11:05.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.621+0000 7fce76a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6006b500 0x7fce6006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:05.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.621+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fce6c0aaf70 con 0x7fce800cee60 2024-04-25T10:11:05.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.621+0000 7fce7ea00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6006b500 0x7fce6006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:05.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.622+0000 7fce7ea00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6006b500 0x7fce6006d9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fce7000a990 tx=0x7fce70008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:05.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.625+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce6c045360 con 0x7fce800cee60 2024-04-25T10:11:05.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.849+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce8006e8f0 con 0x7fce800cee60 2024-04-25T10:11:05.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.850+0000 7fce76a00640 1 -- 172.21.15.165:0/1395554217 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fce6c079c30 con 0x7fce800cee60 2024-04-25T10:11:05.856 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:05.856 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:08:19.374715+0000","modified":"2024-04-25T10:08:28.473904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6833/1170361110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1170361110},{"type":"v1","addr":"172.21.15.172:6833","nonce":1170361110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:05.857 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 57 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6006b500 msgr2=0x7fce6006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6006b500 0x7fce6006d9c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fce7000a990 tx=0x7fce70008040 comp rx=0 tx=0).stop 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 msgr2=0x7fce80142890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 0x7fce80142890 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fce6c000c00 tx=0x7fce6c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 shutdown_connections 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6006b500 0x7fce6006d9c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce80144750 0x7fce80142dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce800cee60 0x7fce80142890 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 --2- 172.21.15.165:0/1395554217 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8006eb10 0x7fce800af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.852+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 >> 172.21.15.165:0/1395554217 conn(0x7fce8006abe0 msgr2=0x7fce8013f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:05.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.853+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 shutdown_connections 2024-04-25T10:11:05.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:05.853+0000 7fce85a00640 1 -- 172.21.15.165:0/1395554217 wait complete. 2024-04-25T10:11:05.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:06.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.066+0000 7efd22e00640 1 Processor -- start 2024-04-25T10:11:06.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.066+0000 7efd22e00640 1 -- start start 2024-04-25T10:11:06.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd22e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd1c15ee10 0x7efd1c15f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd22e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd1c0b2540 0x7efd1c0cc670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd22e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c09a6a0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd22e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c09a450 con 0x7efd1c15ee10 2024-04-25T10:11:06.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd22e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c09a980 con 0x7efd1c0b2540 2024-04-25T10:11:06.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd21a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd1c15ee10 0x7efd1c15f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd21000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efd1c0c50d0 0x7efd1c15fb40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36676/0 (socket says 172.21.15.165:36676) 2024-04-25T10:11:06.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.068+0000 7efd21000640 1 -- 172.21.15.165:0/1494680221 learned_addr learned my addr 172.21.15.165:0/1494680221 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:06.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.069+0000 7efd22400640 1 --2- 172.21.15.165:0/1494680221 >> v2:172.21.15.172:3300/0 conn(0x7efd1c0b2540 0x7efd1c0cc670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.069+0000 7efd21a00640 1 -- 172.21.15.165:0/1494680221 >> v2:172.21.15.172:3300/0 conn(0x7efd1c0b2540 msgr2=0x7efd1c0cc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.069+0000 7efd21a00640 1 --2- 172.21.15.165:0/1494680221 >> v2:172.21.15.172:3300/0 conn(0x7efd1c0b2540 0x7efd1c0cc670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.069+0000 7efd21a00640 1 -- 172.21.15.165:0/1494680221 >> v1:172.21.15.165:6789/0 conn(0x7efd1c0c50d0 legacy=0x7efd1c15fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.069+0000 7efd21a00640 1 -- 172.21.15.165:0/1494680221 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c09ac60 con 0x7efd1c15ee10 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.069+0000 7efd1be00640 1 -- 172.21.15.165:0/1494680221 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674649702 0 0) 0x7efd1c09a6a0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.070+0000 7efd21a00640 1 --2- 172.21.15.165:0/1494680221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c15ee10 0x7efd1c15f600 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efd1c071c50 tx=0x7efd1002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d51b3b888c175a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.070+0000 7efd1be00640 1 -- 172.21.15.165:0/1494680221 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10037070 con 0x7efd1c15ee10 2024-04-25T10:11:06.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.070+0000 7efd1be00640 1 -- 172.21.15.165:0/1494680221 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd10032440 con 0x7efd1c15ee10 2024-04-25T10:11:06.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.070+0000 7efd1be00640 1 -- 172.21.15.165:0/1494680221 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd10032810 con 0x7efd1c15ee10 2024-04-25T10:11:06.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 -- 172.21.15.165:0/1494680221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c15ee10 msgr2=0x7efd1c15f600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 --2- 172.21.15.165:0/1494680221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c15ee10 0x7efd1c15f600 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7efd1c071c50 tx=0x7efd1002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 -- 172.21.15.165:0/1494680221 shutdown_connections 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 --2- 172.21.15.165:0/1494680221 >> v2:172.21.15.172:3300/0 conn(0x7efd1c0b2540 0x7efd1c0cc670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 --2- 172.21.15.165:0/1494680221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c15ee10 0x7efd1c15f600 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 -- 172.21.15.165:0/1494680221 >> 172.21.15.165:0/1494680221 conn(0x7efd1c0b3070 msgr2=0x7efd1c0b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 -- 172.21.15.165:0/1494680221 shutdown_connections 2024-04-25T10:11:06.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.071+0000 7efd22e00640 1 -- 172.21.15.165:0/1494680221 wait complete. 2024-04-25T10:11:06.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.073+0000 7efd22e00640 1 Processor -- start 2024-04-25T10:11:06.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.073+0000 7efd22e00640 1 -- start start 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd22e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0b2540 0x7efd1c0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd22e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 0x7efd1c150670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd22e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c15ee10 0x7efd1c150bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd22e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0cc1d0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd22e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0cc050 con 0x7efd1c15ee10 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd22e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c160250 con 0x7efd1c0b2540 2024-04-25T10:11:06.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd21000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 0x7efd1c150670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd21000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 0x7efd1c150670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59988/0 (socket says 172.21.15.165:59988) 2024-04-25T10:11:06.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd21000640 1 -- 172.21.15.165:0/979333785 learned_addr learned my addr 172.21.15.165:0/979333785 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.074+0000 7efd21a00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0b2540 0x7efd1c0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd22400640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c15ee10 0x7efd1c150bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd21000640 1 -- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0b2540 msgr2=0x7efd1c0ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd21000640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0b2540 0x7efd1c0ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd21000640 1 -- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c15ee10 msgr2=0x7efd1c150bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd21000640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c15ee10 0x7efd1c150bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd21000640 1 -- 172.21.15.165:0/979333785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c0c9580 con 0x7efd1c0c50d0 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd21000640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 0x7efd1c150670 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c410 tx=0x7efd0800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd080074e0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd080047e0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08004bd0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c0c9860 con 0x7efd1c0c50d0 2024-04-25T10:11:06.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.075+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c0c9e20 con 0x7efd1c0c50d0 2024-04-25T10:11:06.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.077+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c073990 con 0x7efd1c0c50d0 2024-04-25T10:11:06.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.078+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd08011070 con 0x7efd1c0c50d0 2024-04-25T10:11:06.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.078+0000 7efd19600640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcfc06b4d0 0x7efcfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.079+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7efd08089f90 con 0x7efd1c0c50d0 2024-04-25T10:11:06.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.079+0000 7efd21a00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcfc06b4d0 0x7efcfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.080+0000 7efd21a00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcfc06b4d0 0x7efcfc06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efd10034000 tx=0x7efd1002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.083+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd0800e280 con 0x7efd1c0c50d0 2024-04-25T10:11:06.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.265+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd1c159cf0 con 0x7efd1c0c50d0 2024-04-25T10:11:06.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.267+0000 7efd19600640 1 -- 172.21.15.165:0/979333785 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+17047 (secure 0 0 0) 0x7efd08058c60 con 0x7efd1c0c50d0 2024-04-25T10:11:06.273 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:06.273 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":87,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:10:39.107124+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:06.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcfc06b4d0 msgr2=0x7efcfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcfc06b4d0 0x7efcfc06d990 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7efd10034000 tx=0x7efd1002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 msgr2=0x7efd1c150670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 0x7efd1c150670 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7efd0800c410 tx=0x7efd0800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 shutdown_connections 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efcfc06b4d0 0x7efcfc06d990 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c15ee10 0x7efd1c150bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0c50d0 0x7efd1c150670 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 --2- 172.21.15.165:0/979333785 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0b2540 0x7efd1c0ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 >> 172.21.15.165:0/979333785 conn(0x7efd1c0b3070 msgr2=0x7efd1c15f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 shutdown_connections 2024-04-25T10:11:06.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.269+0000 7efd22e00640 1 -- 172.21.15.165:0/979333785 wait complete. 2024-04-25T10:11:06.292 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:08:17.127559+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:08:18.397615+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '86', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 3, 'snap_epoch': 86, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:06.293 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:06.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.491+0000 7f90fa200640 1 Processor -- start 2024-04-25T10:11:06.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.491+0000 7f90fa200640 1 -- start start 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90fa200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90f41446d0 0x7f90f40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90fa200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f4144d50 0x7f90f40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90fa200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90f40cccc0 con 0x7f90f41452a0 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90fa200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90f40b2b20 con 0x7f90f41446d0 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90fa200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f406d400 con 0x7f90f4144d50 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90f41446d0 0x7f90f40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f90f41452a0 0x7f90f40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36682/0 (socket says 172.21.15.165:36682) 2024-04-25T10:11:06.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3e00640 1 -- 172.21.15.165:0/401156607 learned_addr learned my addr 172.21.15.165:0/401156607 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f2a00640 1 --2- 172.21.15.165:0/401156607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f4144d50 0x7f90f40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3400640 1 -- 172.21.15.165:0/401156607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f4144d50 msgr2=0x7f90f40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3400640 1 --2- 172.21.15.165:0/401156607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f4144d50 0x7f90f40b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3400640 1 -- 172.21.15.165:0/401156607 >> v1:172.21.15.165:6789/0 conn(0x7f90f41452a0 legacy=0x7f90f40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.493+0000 7f90f3400640 1 -- 172.21.15.165:0/401156607 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40cc140 con 0x7f90f41446d0 2024-04-25T10:11:06.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.495+0000 7f90f3400640 1 --2- 172.21.15.165:0/401156607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f41446d0 0x7f90f40b3f20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f90e40097a0 tx=0x7f90e402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bccdb84dd8c9ff67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.495+0000 7f90f2000640 1 -- 172.21.15.165:0/401156607 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4037070 con 0x7f90f41446d0 2024-04-25T10:11:06.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.495+0000 7f90f2000640 1 -- 172.21.15.165:0/401156607 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e4032440 con 0x7f90f41446d0 2024-04-25T10:11:06.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.495+0000 7f90f2000640 1 -- 172.21.15.165:0/401156607 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4032810 con 0x7f90f41446d0 2024-04-25T10:11:06.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.495+0000 7f90fa200640 1 -- 172.21.15.165:0/401156607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f41446d0 msgr2=0x7f90f40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.495+0000 7f90fa200640 1 --2- 172.21.15.165:0/401156607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f41446d0 0x7f90f40b3f20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f90e40097a0 tx=0x7f90e402a180 comp rx=0 tx=0).stop 2024-04-25T10:11:06.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.496+0000 7f90fa200640 1 -- 172.21.15.165:0/401156607 shutdown_connections 2024-04-25T10:11:06.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.496+0000 7f90fa200640 1 --2- 172.21.15.165:0/401156607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f4144d50 0x7f90f40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.496+0000 7f90fa200640 1 --2- 172.21.15.165:0/401156607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f41446d0 0x7f90f40b3f20 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.496+0000 7f90fa200640 1 -- 172.21.15.165:0/401156607 >> 172.21.15.165:0/401156607 conn(0x7f90f406abe0 msgr2=0x7f90f406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.496+0000 7f90fa200640 1 -- 172.21.15.165:0/401156607 shutdown_connections 2024-04-25T10:11:06.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.496+0000 7f90fa200640 1 -- 172.21.15.165:0/401156607 wait complete. 2024-04-25T10:11:06.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.497+0000 7f90fa200640 1 Processor -- start 2024-04-25T10:11:06.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.497+0000 7f90fa200640 1 -- start start 2024-04-25T10:11:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.498+0000 7f90fa200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f41446d0 0x7f90f413ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.498+0000 7f90fa200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 0x7f90f41f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.498+0000 7f90fa200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f41452a0 0x7f90f41f69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.498+0000 7f90fa200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40cc700 con 0x7f90f4144d50 2024-04-25T10:11:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.498+0000 7f90fa200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40b49a0 con 0x7f90f41446d0 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.498+0000 7f90fa200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f40cc9e0 con 0x7f90f41452a0 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 0x7f90f41f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 0x7f90f41f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:60010/0 (socket says 172.21.15.165:60010) 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 -- 172.21.15.165:0/1512745063 learned_addr learned my addr 172.21.15.165:0/1512745063 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f3e00640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f41452a0 0x7f90f41f69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f3400640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f41446d0 0x7f90f413ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 -- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f41452a0 msgr2=0x7f90f41f69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f41452a0 0x7f90f41f69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 -- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f41446d0 msgr2=0x7f90f413ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f41446d0 0x7f90f413ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f2a00640 1 -- 172.21.15.165:0/1512745063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40acae0 con 0x7f90f4144d50 2024-04-25T10:11:06.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.499+0000 7f90f3e00640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f41452a0 0x7f90f41f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:06.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.500+0000 7f90f2a00640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 0x7f90f41f6480 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c7f0 tx=0x7f90e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.500+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0015070 con 0x7f90f4144d50 2024-04-25T10:11:06.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.500+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90e0004240 con 0x7f90f4144d50 2024-04-25T10:11:06.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.500+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e0007620 con 0x7f90f4144d50 2024-04-25T10:11:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.500+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f40acd50 con 0x7f90f4144d50 2024-04-25T10:11:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.500+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f40ad310 con 0x7f90f4144d50 2024-04-25T10:11:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.501+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f4073a00 con 0x7f90f4144d50 2024-04-25T10:11:06.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.502+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f90e00079d0 con 0x7f90f4144d50 2024-04-25T10:11:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.503+0000 7f90eb400640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.503+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f90e008a7d0 con 0x7f90f4144d50 2024-04-25T10:11:06.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.503+0000 7f90f3400640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.504+0000 7f90f3400640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034000 tx=0x7f90e402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.508+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f90e0059490 con 0x7f90f4144d50 2024-04-25T10:11:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.703+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f90f406ee10 con 0x7f90f4144d50 2024-04-25T10:11:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.704+0000 7f90eb400640 1 -- 172.21.15.165:0/1512745063 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f90e0058d90 con 0x7f90f4144d50 2024-04-25T10:11:06.710 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d406b4d0 msgr2=0x7f90d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d406b4d0 0x7f90d406d990 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f90e4034000 tx=0x7f90e402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 msgr2=0x7f90f41f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 0x7f90f41f6480 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f90e000c7f0 tx=0x7f90e000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 shutdown_connections 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90d406b4d0 0x7f90d406d990 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90f41452a0 0x7f90f41f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4144d50 0x7f90f41f6480 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 --2- 172.21.15.165:0/1512745063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f41446d0 0x7f90f413ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.706+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 >> 172.21.15.165:0/1512745063 conn(0x7f90f406abe0 msgr2=0x7f90f40af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.707+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 shutdown_connections 2024-04-25T10:11:06.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.707+0000 7f90fa200640 1 -- 172.21.15.165:0/1512745063 wait complete. 2024-04-25T10:11:06.732 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:11:06.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.930+0000 7f33dbe00640 1 Processor -- start 2024-04-25T10:11:06.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.930+0000 7f33dbe00640 1 -- start start 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33dbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33d41446d0 0x7f33d40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33dbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33d406e8a0 0x7f33d40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33dbe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d40cccc0 con 0x7f33d4144d50 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33dbe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d40b2b20 con 0x7f33d406e8a0 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33dbe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d406d400 con 0x7f33d41446d0 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33d9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33d406e8a0 0x7f33d40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33d8a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f33d4144d50 0x7f33d40cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40632/0 (socket says 172.21.15.165:40632) 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33d8a00640 1 -- 172.21.15.165:0/430545538 learned_addr learned my addr 172.21.15.165:0/430545538 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.932+0000 7f33d9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33d406e8a0 0x7f33d40b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44888/0 (socket says 172.21.15.165:44888) 2024-04-25T10:11:06.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.933+0000 7f33d9400640 1 --2- 172.21.15.165:0/430545538 >> v2:172.21.15.172:3300/0 conn(0x7f33d41446d0 0x7f33d40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.933+0000 7f33d9e00640 1 -- 172.21.15.165:0/430545538 >> v2:172.21.15.172:3300/0 conn(0x7f33d41446d0 msgr2=0x7f33d40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:06.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.933+0000 7f33d9e00640 1 --2- 172.21.15.165:0/430545538 >> v2:172.21.15.172:3300/0 conn(0x7f33d41446d0 0x7f33d40b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.933+0000 7f33d9e00640 1 -- 172.21.15.165:0/430545538 >> v1:172.21.15.165:6789/0 conn(0x7f33d4144d50 legacy=0x7f33d40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.933+0000 7f33d9e00640 1 -- 172.21.15.165:0/430545538 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d40cc140 con 0x7f33d406e8a0 2024-04-25T10:11:06.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.933+0000 7f33d3e00640 1 -- 172.21.15.165:0/430545538 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262895680 0 0) 0x7f33d40cccc0 con 0x7f33d4144d50 2024-04-25T10:11:06.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.934+0000 7f33d9e00640 1 --2- 172.21.15.165:0/430545538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d406e8a0 0x7f33d40b4460 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f33d40cf560 tx=0x7f33cc02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2b75ae85c87b328 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.934+0000 7f33d3e00640 1 -- 172.21.15.165:0/430545538 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc007740 con 0x7f33d406e8a0 2024-04-25T10:11:06.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.934+0000 7f33d3e00640 1 -- 172.21.15.165:0/430545538 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc0078e0 con 0x7f33d406e8a0 2024-04-25T10:11:06.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.934+0000 7f33d3e00640 1 -- 172.21.15.165:0/430545538 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc007cb0 con 0x7f33d406e8a0 2024-04-25T10:11:06.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 -- 172.21.15.165:0/430545538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d406e8a0 msgr2=0x7f33d40b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 --2- 172.21.15.165:0/430545538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d406e8a0 0x7f33d40b4460 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f33d40cf560 tx=0x7f33cc02a680 comp rx=0 tx=0).stop 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 -- 172.21.15.165:0/430545538 shutdown_connections 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 --2- 172.21.15.165:0/430545538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d406e8a0 0x7f33d40b4460 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 --2- 172.21.15.165:0/430545538 >> v2:172.21.15.172:3300/0 conn(0x7f33d41446d0 0x7f33d40b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 -- 172.21.15.165:0/430545538 >> 172.21.15.165:0/430545538 conn(0x7f33d406abe0 msgr2=0x7f33d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 -- 172.21.15.165:0/430545538 shutdown_connections 2024-04-25T10:11:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.935+0000 7f33dbe00640 1 -- 172.21.15.165:0/430545538 wait complete. 2024-04-25T10:11:06.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.937+0000 7f33dbe00640 1 Processor -- start 2024-04-25T10:11:06.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.937+0000 7f33dbe00640 1 -- start start 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d406e8a0 0x7f33d4150680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d41446d0 0x7f33d40b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 0x7f33d40b2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33dbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40cc700 con 0x7f33d4144d50 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33dbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40b49a0 con 0x7f33d406e8a0 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33dbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40cc9e0 con 0x7f33d41446d0 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 0x7f33d40b2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 0x7f33d40b2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44894/0 (socket says 172.21.15.165:44894) 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.938+0000 7f33d9e00640 1 -- 172.21.15.165:0/819585875 learned_addr learned my addr 172.21.15.165:0/819585875 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:06.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d8a00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d41446d0 0x7f33d40b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9e00640 1 -- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d41446d0 msgr2=0x7f33d40b1d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9400640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d406e8a0 0x7f33d4150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9e00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d41446d0 0x7f33d40b1d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9e00640 1 -- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d406e8a0 msgr2=0x7f33d4150680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9e00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d406e8a0 0x7f33d4150680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9e00640 1 -- 172.21.15.165:0/819585875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d40ad670 con 0x7f33d4144d50 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d8a00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d41446d0 0x7f33d40b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.939+0000 7f33d9e00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 0x7f33d40b2260 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f33cc02ac70 tx=0x7f33cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.940+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc040070 con 0x7f33d4144d50 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.940+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33cc031bc0 con 0x7f33d4144d50 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.940+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33cc045460 con 0x7f33d4144d50 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.940+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d40ad950 con 0x7f33d4144d50 2024-04-25T10:11:06.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.940+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d40adf10 con 0x7f33d4144d50 2024-04-25T10:11:06.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.941+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d4073a00 con 0x7f33d4144d50 2024-04-25T10:11:06.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.942+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33cc039040 con 0x7f33d4144d50 2024-04-25T10:11:06.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.943+0000 7f33d1600640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:06.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.943+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f33cc0abdc0 con 0x7f33d4144d50 2024-04-25T10:11:06.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.944+0000 7f33d9400640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:06.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.944+0000 7f33d9400640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006fd0 tx=0x7f33c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:06.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:06.948+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33cc07a9f0 con 0x7f33d4144d50 2024-04-25T10:11:07.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.150+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f33d40cbbc0 con 0x7f33d4144d50 2024-04-25T10:11:07.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.151+0000 7f33d1600640 1 -- 172.21.15.165:0/819585875 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f33cc043e40 con 0x7f33d4144d50 2024-04-25T10:11:07.157 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.153+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 msgr2=0x7f33b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.153+0000 7f33dbe00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f33c8006fd0 tx=0x7f33c8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.153+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 msgr2=0x7f33d40b2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.153+0000 7f33dbe00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 0x7f33d40b2260 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f33cc02ac70 tx=0x7f33cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 shutdown_connections 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d4144d50 0x7f33d40b2260 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d41446d0 0x7f33d40b1d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 --2- 172.21.15.165:0/819585875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d406e8a0 0x7f33d4150680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 >> 172.21.15.165:0/819585875 conn(0x7f33d406abe0 msgr2=0x7f33d40af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 shutdown_connections 2024-04-25T10:11:07.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.154+0000 7f33dbe00640 1 -- 172.21.15.165:0/819585875 wait complete. 2024-04-25T10:11:07.177 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:11:07.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.361+0000 7f4e0a200640 1 Processor -- start 2024-04-25T10:11:07.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.361+0000 7f4e0a200640 1 -- start start 2024-04-25T10:11:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e0a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e0415b3c0 0x7f4e040b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e0a200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e0415b7a0 0x7f4e040cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e0a200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e040b0b80 con 0x7f4e040cdae0 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e0a200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e040cd650 con 0x7f4e0415b3c0 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e0a200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e040cc800 con 0x7f4e0415b7a0 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e08e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e0415b3c0 0x7f4e040b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e08e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4e0415b3c0 0x7f4e040b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44898/0 (socket says 172.21.15.165:44898) 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e08e00640 1 -- 172.21.15.165:0/4286536028 learned_addr learned my addr 172.21.15.165:0/4286536028 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e09800640 1 --1- 172.21.15.165:0/4286536028 >> v1:172.21.15.165:6789/0 conn(0x7f4e040cdae0 0x7f4e0415e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40636/0 (socket says 172.21.15.165:40636) 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e08e00640 1 -- 172.21.15.165:0/4286536028 >> v2:172.21.15.172:3300/0 conn(0x7f4e0415b7a0 msgr2=0x7f4e040cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.363+0000 7f4e03e00640 1 --2- 172.21.15.165:0/4286536028 >> v2:172.21.15.172:3300/0 conn(0x7f4e0415b7a0 0x7f4e040cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.364+0000 7f4e08e00640 1 --2- 172.21.15.165:0/4286536028 >> v2:172.21.15.172:3300/0 conn(0x7f4e0415b7a0 0x7f4e040cc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.364+0000 7f4e08e00640 1 -- 172.21.15.165:0/4286536028 >> v1:172.21.15.165:6789/0 conn(0x7f4e040cdae0 legacy=0x7f4e0415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.364+0000 7f4e08e00640 1 -- 172.21.15.165:0/4286536028 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040ccae0 con 0x7f4e0415b3c0 2024-04-25T10:11:07.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.364+0000 7f4e03400640 1 -- 172.21.15.165:0/4286536028 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349489131 0 0) 0x7f4dfc001110 con 0x7f4e040cdae0 2024-04-25T10:11:07.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.364+0000 7f4e03e00640 1 --2- 172.21.15.165:0/4286536028 >> v2:172.21.15.172:3300/0 conn(0x7f4e0415b7a0 0x7f4e040cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e08e00640 1 --2- 172.21.15.165:0/4286536028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e040b0f60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4e0415f180 tx=0x7f4df402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a930c1fedecdd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e03400640 1 -- 172.21.15.165:0/4286536028 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4036070 con 0x7f4e0415b3c0 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e03400640 1 -- 172.21.15.165:0/4286536028 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df4031600 con 0x7f4e0415b3c0 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e03400640 1 -- 172.21.15.165:0/4286536028 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df40319d0 con 0x7f4e0415b3c0 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e0a200640 1 -- 172.21.15.165:0/4286536028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 msgr2=0x7f4e040b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e0a200640 1 --2- 172.21.15.165:0/4286536028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e040b0f60 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4e0415f180 tx=0x7f4df402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e0a200640 1 -- 172.21.15.165:0/4286536028 shutdown_connections 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e0a200640 1 --2- 172.21.15.165:0/4286536028 >> v2:172.21.15.172:3300/0 conn(0x7f4e0415b7a0 0x7f4e040cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.365+0000 7f4e0a200640 1 --2- 172.21.15.165:0/4286536028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e040b0f60 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.366+0000 7f4e0a200640 1 -- 172.21.15.165:0/4286536028 >> 172.21.15.165:0/4286536028 conn(0x7f4e04061cc0 msgr2=0x7f4e0406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.366+0000 7f4e0a200640 1 -- 172.21.15.165:0/4286536028 shutdown_connections 2024-04-25T10:11:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.366+0000 7f4e0a200640 1 -- 172.21.15.165:0/4286536028 wait complete. 2024-04-25T10:11:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.367+0000 7f4e0a200640 1 Processor -- start 2024-04-25T10:11:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.367+0000 7f4e0a200640 1 -- start start 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.368+0000 7f4e0a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e040cdae0 0x7f4e04145fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.368+0000 7f4e0a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e04156b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.368+0000 7f4e0a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0415b7a0 0x7f4e041570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.368+0000 7f4e0a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0415e8a0 con 0x7f4e0415b3c0 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.368+0000 7f4e0a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0415e720 con 0x7f4e0415b7a0 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.368+0000 7f4e0a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e040cdf70 con 0x7f4e040cdae0 2024-04-25T10:11:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e04156b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e04156b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44912/0 (socket says 172.21.15.165:44912) 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 -- 172.21.15.165:0/485115335 learned_addr learned my addr 172.21.15.165:0/485115335 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 -- 172.21.15.165:0/485115335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e040cdae0 msgr2=0x7f4e04145fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e040cdae0 0x7f4e04145fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 -- 172.21.15.165:0/485115335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0415b7a0 msgr2=0x7f4e041570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0415b7a0 0x7f4e041570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.369+0000 7f4e03e00640 1 -- 172.21.15.165:0/485115335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e041e6340 con 0x7f4e0415b3c0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.370+0000 7f4e03e00640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e04156b70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00c7c0 tx=0x7f4dec00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.370+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec015070 con 0x7f4e0415b3c0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.370+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dec007820 con 0x7f4e0415b3c0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.370+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dec007bf0 con 0x7f4e0415b3c0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.370+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e041e6620 con 0x7f4e0415b3c0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.370+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e04156680 con 0x7f4e0415b3c0 2024-04-25T10:11:07.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.371+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e04073990 con 0x7f4e0415b3c0 2024-04-25T10:11:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.372+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4dec018070 con 0x7f4e0415b3c0 2024-04-25T10:11:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.373+0000 7f4e00c00640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4de006b4d0 0x7f4de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.374+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f4dec08a340 con 0x7f4e0415b3c0 2024-04-25T10:11:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.374+0000 7f4e08e00640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4de006b4d0 0x7f4de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.375+0000 7f4e08e00640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4de006b4d0 0x7f4de006d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4df4033000 tx=0x7f4df402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.378+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4dec058f70 con 0x7f4e0415b3c0 2024-04-25T10:11:07.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.578+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4e04072360 con 0x7f4e0415b3c0 2024-04-25T10:11:07.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.579+0000 7f4e00c00640 1 -- 172.21.15.165:0/485115335 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4dec058870 con 0x7f4e0415b3c0 2024-04-25T10:11:07.585 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:11:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.581+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4de006b4d0 msgr2=0x7f4de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.581+0000 7f4e0a200640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4de006b4d0 0x7f4de006d990 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f4df4033000 tx=0x7f4df402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.581+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 msgr2=0x7f4e04156b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.581+0000 7f4e0a200640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e04156b70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f4dec00c7c0 tx=0x7f4dec00cc80 comp rx=0 tx=0).stop 2024-04-25T10:11:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 shutdown_connections 2024-04-25T10:11:07.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4de006b4d0 0x7f4de006d990 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e0415b7a0 0x7f4e041570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e0415b3c0 0x7f4e04156b70 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 --2- 172.21.15.165:0/485115335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e040cdae0 0x7f4e04145fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 >> 172.21.15.165:0/485115335 conn(0x7f4e04061cc0 msgr2=0x7f4e040b2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 shutdown_connections 2024-04-25T10:11:07.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.582+0000 7f4e0a200640 1 -- 172.21.15.165:0/485115335 wait complete. 2024-04-25T10:11:07.607 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:07.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.803+0000 7f5e62800640 1 Processor -- start 2024-04-25T10:11:07.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.804+0000 7f5e62800640 1 -- start start 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.805+0000 7f5e62800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e5c154ee0 0x7f5e5c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.805+0000 7f5e62800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e5c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.805+0000 7f5e62800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c071480 con 0x7f5e5c0ce3c0 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.805+0000 7f5e62800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ce830 con 0x7f5e5c154ee0 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.805+0000 7f5e62800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c072090 con 0x7f5e5c0cfcf0 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.805+0000 7f5e5be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5e5c0ce3c0 0x7f5e5c15e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40646/0 (socket says 172.21.15.165:40646) 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5be00640 1 -- 172.21.15.165:0/4076508949 learned_addr learned my addr 172.21.15.165:0/4076508949 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:07.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e53e00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e5c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5b400640 1 --2- 172.21.15.165:0/4076508949 >> v2:172.21.15.172:3300/0 conn(0x7f5e5c154ee0 0x7f5e5c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3966894446 0 0) 0x7f5e5c071480 con 0x7f5e5c0ce3c0 2024-04-25T10:11:07.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e3c003520 con 0x7f5e5c0ce3c0 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1995874506 0 0) 0x7f5e3c003520 con 0x7f5e5c0ce3c0 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 msgr2=0x7f5e5c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e5c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 >> v2:172.21.15.172:3300/0 conn(0x7f5e5c154ee0 msgr2=0x7f5e5c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 --2- 172.21.15.165:0/4076508949 >> v2:172.21.15.172:3300/0 conn(0x7f5e5c154ee0 0x7f5e5c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c071760 con 0x7f5e5c0ce3c0 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5e540019f0 con 0x7f5e5c0ce3c0 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 >> v1:172.21.15.165:6789/0 conn(0x7f5e5c0ce3c0 legacy=0x7f5e5c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3c0039b0 0x7f5e3c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3c004800 0x7f5e3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.806+0000 7f5e5aa00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0ce830 con 0x7f5e3c004800 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c00b9c0 con 0x7f5e3c0039b0 2024-04-25T10:11:07.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c00bb40 con 0x7f5e5c0cfcf0 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e53e00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3c004800 0x7f5e3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e53e00640 1 -- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 msgr2=0x7f5e3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e5be00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e53e00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e53e00640 1 -- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3c0039b0 msgr2=0x7f5e3c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e53e00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3c0039b0 0x7f5e3c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e53e00640 1 -- 172.21.15.165:0/4076508949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c003520 con 0x7f5e3c004800 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.807+0000 7f5e5be00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e5c0cfcf0 0x7f5e3c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.808+0000 7f5e53e00640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3c004800 0x7f5e3c004c80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009650 tx=0x7f5e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c214ac50ae2ba0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.808+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44038070 con 0x7f5e3c004800 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44033400 con 0x7f5e3c004800 2024-04-25T10:11:07.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e5aa00640 1 -- 172.21.15.165:0/4076508949 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44009690 con 0x7f5e3c004800 2024-04-25T10:11:07.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e62800640 1 -- 172.21.15.165:0/4076508949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3c004800 msgr2=0x7f5e3c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e62800640 1 --2- 172.21.15.165:0/4076508949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3c004800 0x7f5e3c004c80 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f5e44009650 tx=0x7f5e3c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:07.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e5be00640 1 -- 172.21.15.165:0/4076508949 reap_dead start 2024-04-25T10:11:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e62800640 1 -- 172.21.15.165:0/4076508949 shutdown_connections 2024-04-25T10:11:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e62800640 1 -- 172.21.15.165:0/4076508949 >> 172.21.15.165:0/4076508949 conn(0x7f5e5c06abe0 msgr2=0x7f5e5c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e62800640 1 -- 172.21.15.165:0/4076508949 shutdown_connections 2024-04-25T10:11:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.809+0000 7f5e62800640 1 -- 172.21.15.165:0/4076508949 wait complete. 2024-04-25T10:11:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.811+0000 7f5e62800640 1 Processor -- start 2024-04-25T10:11:07.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.811+0000 7f5e62800640 1 -- start start 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.812+0000 7f5e62800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3c004800 0x7f5e5c14bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.812+0000 7f5e62800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e5c14c450 0x7f5e5c0acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.812+0000 7f5e62800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 0x7f5e5c0a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.812+0000 7f5e62800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c071230 con 0x7f5e5c0ad500 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.812+0000 7f5e62800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c071d20 con 0x7f5e5c14c450 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.812+0000 7f5e62800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c1552c0 con 0x7f5e3c004800 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 0x7f5e5c0a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 0x7f5e5c0a8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44936/0 (socket says 172.21.15.165:44936) 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 -- 172.21.15.165:0/3828056107 learned_addr learned my addr 172.21.15.165:0/3828056107 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:07.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5b400640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3c004800 0x7f5e5c14bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 -- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3c004800 msgr2=0x7f5e5c14bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3c004800 0x7f5e5c14bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 -- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e5c14c450 msgr2=0x7f5e5c0acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e5c14c450 0x7f5e5c0acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 -- 172.21.15.165:0/3828056107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0ad990 con 0x7f5e5c0ad500 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.813+0000 7f5e5be00640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 0x7f5e5c0a8f90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0cd2d0 tx=0x7f5e5400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.814+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54004d60 con 0x7f5e5c0ad500 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.814+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5400bb80 con 0x7f5e5c0ad500 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.814+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c0a9cc0 con 0x7f5e5c0ad500 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.814+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54007070 con 0x7f5e5c0ad500 2024-04-25T10:11:07.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.814+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c146be0 con 0x7f5e5c0ad500 2024-04-25T10:11:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.815+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c073990 con 0x7f5e5c0ad500 2024-04-25T10:11:07.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.816+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e5401b020 con 0x7f5e5c0ad500 2024-04-25T10:11:07.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.817+0000 7f5e53400640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e3806b500 0x7f5e3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:07.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.817+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f5e5408a840 con 0x7f5e5c0ad500 2024-04-25T10:11:07.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.817+0000 7f5e5b400640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e3806b500 0x7f5e3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:07.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.818+0000 7f5e5b400640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e3806b500 0x7f5e3806d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c004770 tx=0x7f5e4c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:07.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:07.821+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e5c0ad990 con 0x7f5e5c0ad500 2024-04-25T10:11:08.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.061+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5e5c06d360 con 0x7f5e5c0ad500 2024-04-25T10:11:08.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.062+0000 7f5e53400640 1 -- 172.21.15.165:0/3828056107 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f5e54059530 con 0x7f5e5c0ad500 2024-04-25T10:11:08.069 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:08.069 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:08:19.374715+0000","modified":"2024-04-25T10:08:28.473904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6833/1170361110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1170361110},{"type":"v1","addr":"172.21.15.172:6833","nonce":1170361110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:08.069 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 57 2024-04-25T10:11:08.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e3806b500 msgr2=0x7f5e3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e3806b500 0x7f5e3806d9c0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f5e4c004770 tx=0x7f5e4c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:08.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 msgr2=0x7f5e5c0a8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 0x7f5e5c0a8f90 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0cd2d0 tx=0x7f5e5400b4a0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 shutdown_connections 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e3806b500 0x7f5e3806d9c0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e5c0ad500 0x7f5e5c0a8f90 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e5c14c450 0x7f5e5c0acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 --2- 172.21.15.165:0/3828056107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3c004800 0x7f5e5c14bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 >> 172.21.15.165:0/3828056107 conn(0x7f5e5c06abe0 msgr2=0x7f5e5c0b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 shutdown_connections 2024-04-25T10:11:08.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.065+0000 7f5e62800640 1 -- 172.21.15.165:0/3828056107 wait complete. 2024-04-25T10:11:08.090 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:08.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.282+0000 7fde47200640 1 Processor -- start 2024-04-25T10:11:08.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.282+0000 7fde47200640 1 -- start start 2024-04-25T10:11:08.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde47200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fde4015dbe0 0x7fde400d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde47200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fde400b92a0 0x7fde400d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde47200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde400b97a0 con 0x7fde400b2cb0 2024-04-25T10:11:08.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde47200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4015dfc0 con 0x7fde4015dbe0 2024-04-25T10:11:08.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde47200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde400d1a00 con 0x7fde400b92a0 2024-04-25T10:11:08.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde3fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fde4015dbe0 0x7fde400d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde3fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fde4015dbe0 0x7fde400d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44942/0 (socket says 172.21.15.165:44942) 2024-04-25T10:11:08.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.284+0000 7fde3fe00640 1 -- 172.21.15.165:0/1965107929 learned_addr learned my addr 172.21.15.165:0/1965107929 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:08.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.285+0000 7fde45200640 1 --2- 172.21.15.165:0/1965107929 >> v2:172.21.15.172:3300/0 conn(0x7fde400b92a0 0x7fde400d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.285+0000 7fde3fe00640 1 -- 172.21.15.165:0/1965107929 >> v2:172.21.15.172:3300/0 conn(0x7fde400b92a0 msgr2=0x7fde400d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.285+0000 7fde3fe00640 1 --2- 172.21.15.165:0/1965107929 >> v2:172.21.15.172:3300/0 conn(0x7fde400b92a0 0x7fde400d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.285+0000 7fde3fe00640 1 -- 172.21.15.165:0/1965107929 >> v1:172.21.15.172:6789/0 conn(0x7fde400b2cb0 legacy=0x7fde400b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.285+0000 7fde3fe00640 1 -- 172.21.15.165:0/1965107929 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400d1c50 con 0x7fde4015dbe0 2024-04-25T10:11:08.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.287+0000 7fde3fe00640 1 --2- 172.21.15.165:0/1965107929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde4015dbe0 0x7fde400d0910 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fde400cfb90 tx=0x7fde3002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a8d1e37e346a07 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.287+0000 7fde3f400640 1 -- 172.21.15.165:0/1965107929 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30036070 con 0x7fde4015dbe0 2024-04-25T10:11:08.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.287+0000 7fde3f400640 1 -- 172.21.15.165:0/1965107929 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde30031600 con 0x7fde4015dbe0 2024-04-25T10:11:08.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.287+0000 7fde3f400640 1 -- 172.21.15.165:0/1965107929 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde300319d0 con 0x7fde4015dbe0 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 -- 172.21.15.165:0/1965107929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde4015dbe0 msgr2=0x7fde400d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 --2- 172.21.15.165:0/1965107929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde4015dbe0 0x7fde400d0910 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fde400cfb90 tx=0x7fde3002a660 comp rx=0 tx=0).stop 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 -- 172.21.15.165:0/1965107929 shutdown_connections 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 --2- 172.21.15.165:0/1965107929 >> v2:172.21.15.172:3300/0 conn(0x7fde400b92a0 0x7fde400d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 --2- 172.21.15.165:0/1965107929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde4015dbe0 0x7fde400d0910 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 -- 172.21.15.165:0/1965107929 >> 172.21.15.165:0/1965107929 conn(0x7fde40061a20 msgr2=0x7fde40061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 -- 172.21.15.165:0/1965107929 shutdown_connections 2024-04-25T10:11:08.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.288+0000 7fde47200640 1 -- 172.21.15.165:0/1965107929 wait complete. 2024-04-25T10:11:08.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.290+0000 7fde47200640 1 Processor -- start 2024-04-25T10:11:08.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.290+0000 7fde47200640 1 -- start start 2024-04-25T10:11:08.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde47200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde400b2cb0 0x7fde400b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde47200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 0x7fde400b5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde47200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4015dbe0 0x7fde400c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde47200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400b6ab0 con 0x7fde400b92a0 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde47200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400d2210 con 0x7fde400b2cb0 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde47200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde400b6c30 con 0x7fde4015dbe0 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 0x7fde400b5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 0x7fde400b5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44944/0 (socket says 172.21.15.165:44944) 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 -- 172.21.15.165:0/3610401109 learned_addr learned my addr 172.21.15.165:0/3610401109 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 -- 172.21.15.165:0/3610401109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4015dbe0 msgr2=0x7fde400c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4015dbe0 0x7fde400c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 -- 172.21.15.165:0/3610401109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde400b2cb0 msgr2=0x7fde400b32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde400b2cb0 0x7fde400b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.291+0000 7fde37400640 1 -- 172.21.15.165:0/3610401109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400caaf0 con 0x7fde400b92a0 2024-04-25T10:11:08.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.292+0000 7fde37400640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 0x7fde400b5a10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c400 tx=0x7fde2800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.292+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28011790 con 0x7fde400b92a0 2024-04-25T10:11:08.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.292+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde28017070 con 0x7fde400b92a0 2024-04-25T10:11:08.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.292+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28011bb0 con 0x7fde400b92a0 2024-04-25T10:11:08.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.292+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde400cadd0 con 0x7fde400b92a0 2024-04-25T10:11:08.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.292+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde400cb390 con 0x7fde400b92a0 2024-04-25T10:11:08.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.293+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde40073990 con 0x7fde400b92a0 2024-04-25T10:11:08.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.294+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde2801c460 con 0x7fde400b92a0 2024-04-25T10:11:08.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.295+0000 7fde3cc00640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde1c06b4d0 0x7fde1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.295+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fde2808a270 con 0x7fde400b92a0 2024-04-25T10:11:08.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.296+0000 7fde3fe00640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde1c06b4d0 0x7fde1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.296+0000 7fde3fe00640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde1c06b4d0 0x7fde1c06d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde30033000 tx=0x7fde3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.300+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde28058f30 con 0x7fde400b92a0 2024-04-25T10:11:08.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.495+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde400cb7d0 con 0x7fde400b92a0 2024-04-25T10:11:08.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.498+0000 7fde3cc00640 1 -- 172.21.15.165:0/3610401109 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+17047 (secure 0 0 0) 0x7fde28058830 con 0x7fde400b92a0 2024-04-25T10:11:08.505 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:08.505 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":87,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:10:39.107124+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.501+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde1c06b4d0 msgr2=0x7fde1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.501+0000 7fde47200640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde1c06b4d0 0x7fde1c06d990 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fde30033000 tx=0x7fde3002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.501+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 msgr2=0x7fde400b5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.501+0000 7fde47200640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 0x7fde400b5a10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c400 tx=0x7fde2800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.501+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 shutdown_connections 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.501+0000 7fde47200640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde1c06b4d0 0x7fde1c06d990 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.502+0000 7fde47200640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4015dbe0 0x7fde400c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.502+0000 7fde47200640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde400b92a0 0x7fde400b5a10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.502+0000 7fde47200640 1 --2- 172.21.15.165:0/3610401109 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde400b2cb0 0x7fde400b32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.502+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 >> 172.21.15.165:0/3610401109 conn(0x7fde40061a20 msgr2=0x7fde40152840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.502+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 shutdown_connections 2024-04-25T10:11:08.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.502+0000 7fde47200640 1 -- 172.21.15.165:0/3610401109 wait complete. 2024-04-25T10:11:08.526 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:11:08.527 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:08.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.717+0000 7fe5fb000640 1 Processor -- start 2024-04-25T10:11:08.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.717+0000 7fe5fb000640 1 -- start start 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5fb000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5f4144440 0x7fe5f4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f4071530 0x7fe5f40cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5fb000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5f415bf30 con 0x7fe5f40cd860 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5fb000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5f415c860 con 0x7fe5f4144440 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5fb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f4071970 con 0x7fe5f4071530 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5f9c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5f4144440 0x7fe5f4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5f9c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5f4144440 0x7fe5f4070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44948/0 (socket says 172.21.15.165:44948) 2024-04-25T10:11:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5f9c00640 1 -- 172.21.15.165:0/2372910823 learned_addr learned my addr 172.21.15.165:0/2372910823 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5f9200640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f4071530 0x7fe5f40cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.719+0000 7fe5f9c00640 1 -- 172.21.15.165:0/2372910823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f4071530 msgr2=0x7fe5f40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.720+0000 7fe5f9c00640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f4071530 0x7fe5f40cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.720+0000 7fe5f9c00640 1 -- 172.21.15.165:0/2372910823 >> v1:172.21.15.165:6789/0 conn(0x7fe5f40cd860 legacy=0x7fe5f406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.720+0000 7fe5f9c00640 1 -- 172.21.15.165:0/2372910823 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f415c210 con 0x7fe5f4144440 2024-04-25T10:11:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.720+0000 7fe5f3e00640 1 -- 172.21.15.165:0/2372910823 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716091393 0 0) 0x7fe5f415bf30 con 0x7fe5f40cd860 2024-04-25T10:11:08.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.720+0000 7fe5f9200640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f4071530 0x7fe5f40cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:08.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.721+0000 7fe5f9c00640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4144440 0x7fe5f4070ff0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009870 tx=0x7fe5e802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c337d26bfeee9eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.721+0000 7fe5f3e00640 1 -- 172.21.15.165:0/2372910823 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8037070 con 0x7fe5f4144440 2024-04-25T10:11:08.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.721+0000 7fe5f3e00640 1 -- 172.21.15.165:0/2372910823 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e802ae50 con 0x7fe5f4144440 2024-04-25T10:11:08.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.721+0000 7fe5f3e00640 1 -- 172.21.15.165:0/2372910823 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8032860 con 0x7fe5f4144440 2024-04-25T10:11:08.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 -- 172.21.15.165:0/2372910823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4144440 msgr2=0x7fe5f4070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4144440 0x7fe5f4070ff0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8009870 tx=0x7fe5e802a300 comp rx=0 tx=0).stop 2024-04-25T10:11:08.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 -- 172.21.15.165:0/2372910823 shutdown_connections 2024-04-25T10:11:08.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f4071530 0x7fe5f40cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 --2- 172.21.15.165:0/2372910823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4144440 0x7fe5f4070ff0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 -- 172.21.15.165:0/2372910823 >> 172.21.15.165:0/2372910823 conn(0x7fe5f406abe0 msgr2=0x7fe5f406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 -- 172.21.15.165:0/2372910823 shutdown_connections 2024-04-25T10:11:08.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.722+0000 7fe5fb000640 1 -- 172.21.15.165:0/2372910823 wait complete. 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.724+0000 7fe5fb000640 1 Processor -- start 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.724+0000 7fe5fb000640 1 -- start start 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 0x7fe5f413fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f40cd860 0x7fe5f41f6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5fb000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f4144440 0x7fe5f41e2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5fb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40cdcd0 con 0x7fe5f4071530 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5fb000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f415bc30 con 0x7fe5f4144440 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5fb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5f40cde50 con 0x7fe5f40cd860 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5f9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 0x7fe5f413fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5f9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 0x7fe5f413fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44950/0 (socket says 172.21.15.165:44950) 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.725+0000 7fe5f9c00640 1 -- 172.21.15.165:0/804279774 learned_addr learned my addr 172.21.15.165:0/804279774 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f9c00640 1 -- 172.21.15.165:0/804279774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f40cd860 msgr2=0x7fe5f41f6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f9c00640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f40cd860 0x7fe5f41f6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f9c00640 1 -- 172.21.15.165:0/804279774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f4144440 msgr2=0x7fe5f41e2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f9c00640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f4144440 0x7fe5f41e2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f9c00640 1 -- 172.21.15.165:0/804279774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5f41e2ba0 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f9c00640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 0x7fe5f413fe60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8000c00 tx=0x7fe5e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8041020 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5e803c890 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5e8037040 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5f41e2e20 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.726+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5f41e33e0 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.728+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5f4073990 con 0x7fe5f4071530 2024-04-25T10:11:08.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.728+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5e8044020 con 0x7fe5f4071530 2024-04-25T10:11:08.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.729+0000 7fe5f1600640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5d406b4d0 0x7fe5d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:08.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.729+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fe5e80aadc0 con 0x7fe5f4071530 2024-04-25T10:11:08.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.730+0000 7fe5f9200640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5d406b4d0 0x7fe5d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:08.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.730+0000 7fe5f9200640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5d406b4d0 0x7fe5d406d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0006fd0 tx=0x7fe5e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:08.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.734+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5e804b030 con 0x7fe5f4071530 2024-04-25T10:11:08.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.967+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5f406eaa0 con 0x7fe5f4071530 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.968+0000 7fe5f1600640 1 -- 172.21.15.165:0/804279774 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fe5f41e2ba0 con 0x7fe5f4071530 2024-04-25T10:11:08.974 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:08.975 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:08:19.374715+0000","modified":"2024-04-25T10:08:28.473904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6833/1170361110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1170361110},{"type":"v1","addr":"172.21.15.172:6833","nonce":1170361110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:08.975 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 57 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5d406b4d0 msgr2=0x7fe5d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5d406b4d0 0x7fe5d406d990 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fe5e0006fd0 tx=0x7fe5e0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:08.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 msgr2=0x7fe5f413fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 0x7fe5f413fe60 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fe5e8000c00 tx=0x7fe5e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 shutdown_connections 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5d406b4d0 0x7fe5d406d990 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5f4144440 0x7fe5f41e2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5f40cd860 0x7fe5f41f6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 --2- 172.21.15.165:0/804279774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5f4071530 0x7fe5f413fe60 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 >> 172.21.15.165:0/804279774 conn(0x7fe5f406abe0 msgr2=0x7fe5f40c57a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 shutdown_connections 2024-04-25T10:11:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:08.971+0000 7fe5fb000640 1 -- 172.21.15.165:0/804279774 wait complete. 2024-04-25T10:11:08.996 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:09.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.193+0000 7f7579e00640 1 Processor -- start 2024-04-25T10:11:09.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.193+0000 7f7579e00640 1 -- start start 2024-04-25T10:11:09.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.194+0000 7f7579e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7574144440 0x7f7574070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.194+0000 7f7579e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 0x7f75740cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.194+0000 7f7579e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757415bf30 con 0x7f75740cd860 2024-04-25T10:11:09.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.194+0000 7f7579e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757415c860 con 0x7f7574144440 2024-04-25T10:11:09.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.194+0000 7f7579e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7574071970 con 0x7f7574071530 2024-04-25T10:11:09.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7579400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f75740cd860 0x7f757406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40654/0 (socket says 172.21.15.165:40654) 2024-04-25T10:11:09.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7579400640 1 -- 172.21.15.165:0/3931380609 learned_addr learned my addr 172.21.15.165:0/3931380609 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:09.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573e00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 0x7f75740cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919672093 0 0) 0x7f757415bf30 con 0x7f75740cd860 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f755c003520 con 0x7f75740cd860 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7578a00640 1 --2- 172.21.15.165:0/3931380609 >> v2:172.21.15.172:3300/0 conn(0x7f7574144440 0x7f7574070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2975001353 0 0) 0x7f755c003520 con 0x7f75740cd860 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 msgr2=0x7f75740cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 0x7f75740cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 >> v2:172.21.15.172:3300/0 conn(0x7f7574144440 msgr2=0x7f7574070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 --2- 172.21.15.165:0/3931380609 >> v2:172.21.15.172:3300/0 conn(0x7f7574144440 0x7f7574070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757415c210 con 0x7f75740cd860 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f756c0019f0 con 0x7f75740cd860 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 >> v1:172.21.15.165:6789/0 conn(0x7f75740cd860 legacy=0x7f757406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.195+0000 7f7573400640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 0x7f755c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573400640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f755c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573400640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755c004d40 0x7f755c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415c860 con 0x7f755c003f30 2024-04-25T10:11:09.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7574071970 con 0x7f755c004d40 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f755c00baa0 con 0x7f7574071530 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573e00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f755c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7578a00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 0x7f755c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573e00640 1 -- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 msgr2=0x7f755c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573e00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7574071530 0x7f755c0039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7579400640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755c004d40 0x7f755c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573e00640 1 -- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755c004d40 msgr2=0x7f755c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573e00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755c004d40 0x7f755c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7573e00640 1 -- 172.21.15.165:0/3931380609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f755c003520 con 0x7f755c003f30 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.196+0000 7f7579400640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f755c004d40 0x7f755c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.197+0000 7f7573e00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f755c004800 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7560009ac0 tx=0x7f755c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3873dedb1c893982 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560036070 con 0x7f755c003f30 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7560031400 con 0x7f755c003f30 2024-04-25T10:11:09.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7573400640 1 -- 172.21.15.165:0/3931380609 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7560009690 con 0x7f755c003f30 2024-04-25T10:11:09.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579e00640 1 -- 172.21.15.165:0/3931380609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 msgr2=0x7f755c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579e00640 1 --2- 172.21.15.165:0/3931380609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f755c004800 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f7560009ac0 tx=0x7f755c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579400640 1 -- 172.21.15.165:0/3931380609 reap_dead start 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579e00640 1 -- 172.21.15.165:0/3931380609 shutdown_connections 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579e00640 1 -- 172.21.15.165:0/3931380609 >> 172.21.15.165:0/3931380609 conn(0x7f757406abe0 msgr2=0x7f757406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579e00640 1 -- 172.21.15.165:0/3931380609 shutdown_connections 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.198+0000 7f7579e00640 1 -- 172.21.15.165:0/3931380609 wait complete. 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.200+0000 7f7579e00640 1 Processor -- start 2024-04-25T10:11:09.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.201+0000 7f7579e00640 1 -- start start 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7579e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f75741f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7579e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574156c80 0x7f75740c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7579e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75740c9ee0 0x7f75741f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7579e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cdcd0 con 0x7f755c003f30 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7579e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757415bc30 con 0x7f7574156c80 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7579e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75740cde50 con 0x7f75740c9ee0 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7578a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f75741f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7578a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f75741f2cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44978/0 (socket says 172.21.15.165:44978) 2024-04-25T10:11:09.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.202+0000 7f7578a00640 1 -- 172.21.15.165:0/908584022 learned_addr learned my addr 172.21.15.165:0/908584022 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:09.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7578a00640 1 -- 172.21.15.165:0/908584022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75740c9ee0 msgr2=0x7f75741f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7578a00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75740c9ee0 0x7f75741f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7578a00640 1 -- 172.21.15.165:0/908584022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574156c80 msgr2=0x7f75740c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7578a00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574156c80 0x7f75740c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7578a00640 1 -- 172.21.15.165:0/908584022 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574155c20 con 0x7f755c003f30 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7578a00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f75741f2cf0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f756800e810 tx=0x7f756800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7568017070 con 0x7f755c003f30 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756800c6d0 con 0x7f755c003f30 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756800f040 con 0x7f755c003f30 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7574155ec0 con 0x7f755c003f30 2024-04-25T10:11:09.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.203+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7574156480 con 0x7f755c003f30 2024-04-25T10:11:09.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.204+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7574073990 con 0x7f755c003f30 2024-04-25T10:11:09.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.205+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f756801a070 con 0x7f755c003f30 2024-04-25T10:11:09.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.206+0000 7f7570c00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f755406b500 0x7f755406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.206+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f7568089fc0 con 0x7f755c003f30 2024-04-25T10:11:09.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.206+0000 7f7573e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f755406b500 0x7f755406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.207+0000 7f7573e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f755406b500 0x7f755406d9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7560033000 tx=0x7f756002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.210+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7568058bf0 con 0x7f755c003f30 2024-04-25T10:11:09.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.447+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75740c9480 con 0x7f755c003f30 2024-04-25T10:11:09.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.448+0000 7f7570c00640 1 -- 172.21.15.165:0/908584022 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 57 v57) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f75680584f0 con 0x7f755c003f30 2024-04-25T10:11:09.455 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:09.455 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":57,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":57,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":2,"explicitly_allowed_features":2,"created":"2024-04-25T10:08:19.374715+0000","modified":"2024-04-25T10:08:28.473904+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7116},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7116":{"gid":7116,"name":"d","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6833/1170361110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1170361110},{"type":"v1","addr":"172.21.15.172:6833","nonce":1170361110}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2024-04-25T10:11:09.455 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 57 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f755406b500 msgr2=0x7f755406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f755406b500 0x7f755406d9c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f7560033000 tx=0x7f756002e000 comp rx=0 tx=0).stop 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 msgr2=0x7f75741f2cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f75741f2cf0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f756800e810 tx=0x7f756800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 shutdown_connections 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f755406b500 0x7f755406d9c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75740c9ee0 0x7f75741f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7574156c80 0x7f75740c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 --2- 172.21.15.165:0/908584022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f755c003f30 0x7f75741f2cf0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 >> 172.21.15.165:0/908584022 conn(0x7f757406abe0 msgr2=0x7f75740ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 shutdown_connections 2024-04-25T10:11:09.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.452+0000 7f7579e00640 1 -- 172.21.15.165:0/908584022 wait complete. 2024-04-25T10:11:09.476 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.670+0000 7fede8a00640 1 Processor -- start 2024-04-25T10:11:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.670+0000 7fede8a00640 1 -- start start 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.671+0000 7fede8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fede415db70 0x7fede40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.671+0000 7fede8a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede4071650 con 0x7fede40cfc20 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.671+0000 7fede8a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fede40cf230 con 0x7fede415df50 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.671+0000 7fede8a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fede40cbd80 con 0x7fede415db70 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fede415db70 0x7fede40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fede415db70 0x7fede40cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:44992/0 (socket says 172.21.15.165:44992) 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede1e00640 1 -- 172.21.15.165:0/883945228 learned_addr learned my addr 172.21.15.165:0/883945228 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:09.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede2800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fede40cfc20 0x7fede406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40660/0 (socket says 172.21.15.165:40660) 2024-04-25T10:11:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede1e00640 1 -- 172.21.15.165:0/883945228 >> v1:172.21.15.172:6789/0 conn(0x7fede415df50 legacy=0x7fede40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede1e00640 1 -- 172.21.15.165:0/883945228 >> v1:172.21.15.165:6789/0 conn(0x7fede40cfc20 legacy=0x7fede406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.672+0000 7fede1e00640 1 -- 172.21.15.165:0/883945228 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede4161b60 con 0x7fede415db70 2024-04-25T10:11:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.673+0000 7fede1e00640 1 --2- 172.21.15.165:0/883945228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede40cb840 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009870 tx=0x7fedd402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f286670ce1a2356 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.673+0000 7fede1400640 1 -- 172.21.15.165:0/883945228 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4037070 con 0x7fede415db70 2024-04-25T10:11:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede1400640 1 -- 172.21.15.165:0/883945228 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd4032440 con 0x7fede415db70 2024-04-25T10:11:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede1400640 1 -- 172.21.15.165:0/883945228 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd4032810 con 0x7fede415db70 2024-04-25T10:11:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 -- 172.21.15.165:0/883945228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 msgr2=0x7fede40cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 --2- 172.21.15.165:0/883945228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede40cb840 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7fedd4009870 tx=0x7fedd402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 -- 172.21.15.165:0/883945228 shutdown_connections 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 --2- 172.21.15.165:0/883945228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede40cb840 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 -- 172.21.15.165:0/883945228 >> 172.21.15.165:0/883945228 conn(0x7fede406abe0 msgr2=0x7fede406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 -- 172.21.15.165:0/883945228 shutdown_connections 2024-04-25T10:11:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.674+0000 7fede8a00640 1 -- 172.21.15.165:0/883945228 wait complete. 2024-04-25T10:11:09.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.676+0000 7fede8a00640 1 Processor -- start 2024-04-25T10:11:09.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.676+0000 7fede8a00640 1 -- start start 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fede8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fede40cfc20 0x7fede4145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fede8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede41599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fede8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede415df50 0x7fede4142730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fede8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4162120 con 0x7fede415db70 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fede8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fede4161910 con 0x7fede415df50 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fede8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fede4162640 con 0x7fede40cfc20 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fedd9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede41599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fedd9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede41599b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45006/0 (socket says 172.21.15.165:45006) 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.677+0000 7fedd9400640 1 -- 172.21.15.165:0/3802470612 learned_addr learned my addr 172.21.15.165:0/3802470612 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fede2800640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede415df50 0x7fede4142730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fede1e00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fede40cfc20 0x7fede4145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fedd9400640 1 -- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fede40cfc20 msgr2=0x7fede4145b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fedd9400640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fede40cfc20 0x7fede4145b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fedd9400640 1 -- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede415df50 msgr2=0x7fede4142730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fedd9400640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede415df50 0x7fede4142730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fedd9400640 1 -- 172.21.15.165:0/3802470612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fede4142c70 con 0x7fede415db70 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fede1e00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fede40cfc20 0x7fede4145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:09.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.678+0000 7fedd9400640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede41599b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c460 tx=0x7fedcc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.679+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc016020 con 0x7fede415db70 2024-04-25T10:11:09.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.679+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedcc012870 con 0x7fede415db70 2024-04-25T10:11:09.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.679+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedcc012c60 con 0x7fede415db70 2024-04-25T10:11:09.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.679+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fede4154a00 con 0x7fede415db70 2024-04-25T10:11:09.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.679+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fede4154fc0 con 0x7fede415db70 2024-04-25T10:11:09.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.680+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fede4073990 con 0x7fede415db70 2024-04-25T10:11:09.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.681+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fedcc019020 con 0x7fede415db70 2024-04-25T10:11:09.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.681+0000 7feddaa00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedc006b4d0 0x7fedc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:09.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.682+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7fedcc08a100 con 0x7fede415db70 2024-04-25T10:11:09.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.682+0000 7fede1e00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedc006b4d0 0x7fedc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:09.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.682+0000 7fede1e00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedc006b4d0 0x7fedc006d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fedd4034000 tx=0x7fedd402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:09.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.686+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fedcc020030 con 0x7fede415db70 2024-04-25T10:11:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.879+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fede415a100 con 0x7fede415db70 2024-04-25T10:11:09.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.881+0000 7feddaa00640 1 -- 172.21.15.165:0/3802470612 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+17047 (secure 0 0 0) 0x7fedcc0182c0 con 0x7fede415db70 2024-04-25T10:11:09.887 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:09.887 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":87,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:10:39.107124+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:08:17.127559+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_data","create_time":"2024-04-25T10:08:18.397615+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":86,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":12,"snaps":[{"begin":2,"length":2}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedc006b4d0 msgr2=0x7fedc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedc006b4d0 0x7fedc006d990 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fedd4034000 tx=0x7fedd402e000 comp rx=0 tx=0).stop 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 msgr2=0x7fede41599b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede41599b0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7fedcc00c460 tx=0x7fedcc00c920 comp rx=0 tx=0).stop 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 shutdown_connections 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedc006b4d0 0x7fedc006d990 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fede415df50 0x7fede4142730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fede415db70 0x7fede41599b0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 --2- 172.21.15.165:0/3802470612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fede40cfc20 0x7fede4145b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.883+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 >> 172.21.15.165:0/3802470612 conn(0x7fede406abe0 msgr2=0x7fede415e710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.884+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 shutdown_connections 2024-04-25T10:11:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:09.884+0000 7fede8a00640 1 -- 172.21.15.165:0/3802470612 wait complete. 2024-04-25T10:11:09.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:11:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.099+0000 7f8849a00640 1 Processor -- start 2024-04-25T10:11:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.100+0000 7f8849a00640 1 -- start start 2024-04-25T10:11:10.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f884415dd30 0x7f88440cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884406bff0 0x7f88440b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f884415e110 con 0x7f88440b5970 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f884406d000 con 0x7f884415dd30 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88440b2fd0 con 0x7f884406bff0 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f88440b5970 0x7f8844161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40668/0 (socket says 172.21.15.165:40668) 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8849000640 1 -- 172.21.15.165:0/2855288027 learned_addr learned my addr 172.21.15.165:0/2855288027 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8843e00640 1 --2- 172.21.15.165:0/2855288027 >> v2:172.21.15.172:3300/0 conn(0x7f884415dd30 0x7f88440cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.101+0000 7f8843400640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884406bff0 0x7f88440b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649172502 0 0) 0x7f884415e110 con 0x7f88440b5970 2024-04-25T10:11:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8828003520 con 0x7f88440b5970 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 82157275 0 0) 0x7f8828003520 con 0x7f88440b5970 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884406bff0 msgr2=0x7f88440b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884406bff0 0x7f88440b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 >> v2:172.21.15.172:3300/0 conn(0x7f884415dd30 msgr2=0x7f88440cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 --2- 172.21.15.165:0/2855288027 >> v2:172.21.15.172:3300/0 conn(0x7f884415dd30 0x7f88440cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f884406c480 con 0x7f88440b5970 2024-04-25T10:11:10.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f883c0019f0 con 0x7f88440b5970 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 >> v1:172.21.15.165:6789/0 conn(0x7f88440b5970 legacy=0x7f8844161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88280039b0 0x7f8828003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004310 0x7f8828004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8828004e50 0x7f88280052d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884406d000 con 0x7f8828004e50 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88440b2fd0 con 0x7f8828004310 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f882800bd40 con 0x7f88280039b0 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.102+0000 7f8849000640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8828004e50 0x7f88280052d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8843e00640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88280039b0 0x7f8828003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8843400640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004310 0x7f8828004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8849000640 1 -- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88280039b0 msgr2=0x7f8828003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8849000640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f88280039b0 0x7f8828003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8849000640 1 -- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004310 msgr2=0x7f8828004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8849000640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004310 0x7f8828004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.103+0000 7f8849000640 1 -- 172.21.15.165:0/2855288027 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8828003520 con 0x7f8828004e50 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.104+0000 7f8849000640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8828004e50 0x7f88280052d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f883c00a8a0 tx=0x7f8828005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=902b8a0ef43df4c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.104+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00a4c0 con 0x7f8828004e50 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.104+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c02f070 con 0x7f8828004e50 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.104+0000 7f8842a00640 1 -- 172.21.15.165:0/2855288027 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c02bd70 con 0x7f8828004e50 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849a00640 1 -- 172.21.15.165:0/2855288027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8828004e50 msgr2=0x7f88280052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849a00640 1 --2- 172.21.15.165:0/2855288027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8828004e50 0x7f88280052d0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f883c00a8a0 tx=0x7f8828005830 comp rx=0 tx=0).stop 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849000640 1 -- 172.21.15.165:0/2855288027 reap_dead start 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849a00640 1 -- 172.21.15.165:0/2855288027 shutdown_connections 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849a00640 1 -- 172.21.15.165:0/2855288027 >> 172.21.15.165:0/2855288027 conn(0x7f88440d0610 msgr2=0x7f88440d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849a00640 1 -- 172.21.15.165:0/2855288027 shutdown_connections 2024-04-25T10:11:10.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.105+0000 7f8849a00640 1 -- 172.21.15.165:0/2855288027 wait complete. 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.107+0000 7f8849a00640 1 Processor -- start 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.107+0000 7f8849a00640 1 -- start start 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.107+0000 7f8849a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004e50 0x7f8844149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884414c5e0 0x7f88440c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 0x7f88441e6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884406cd20 con 0x7f884414b250 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f884406ca40 con 0x7f8828004e50 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88440cf110 con 0x7f884414c5e0 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 0x7f88441e6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 0x7f88441e6720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45028/0 (socket says 172.21.15.165:45028) 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 -- 172.21.15.165:0/3355510917 learned_addr learned my addr 172.21.15.165:0/3355510917 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8843400640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884414c5e0 0x7f88440c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 -- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884414c5e0 msgr2=0x7f88440c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884414c5e0 0x7f88440c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 -- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004e50 msgr2=0x7f8844149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004e50 0x7f8844149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.108+0000 7f8849000640 1 -- 172.21.15.165:0/3355510917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88440c9b70 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.109+0000 7f8849000640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 0x7f88441e6720 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f883c00a9c0 tx=0x7f883c001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.109+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c044070 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.109+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f883c034470 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.109+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f884414ca60 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.109+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f883c00a4c0 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.109+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f884414a490 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.110+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88440d4750 con 0x7f884414b250 2024-04-25T10:11:10.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.111+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f883c047070 con 0x7f884414b250 2024-04-25T10:11:10.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.112+0000 7f883be00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f882006b500 0x7f882006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.112+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 6129+0+0 (secure 0 0 0) 0x7f883c0aa5b0 con 0x7f884414b250 2024-04-25T10:11:10.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.112+0000 7f8843e00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f882006b500 0x7f882006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.113+0000 7f8843e00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f882006b500 0x7f882006d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8834009910 tx=0x7f8834008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.116+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f88440d4750 con 0x7f884414b250 2024-04-25T10:11:10.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.335+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f8844073990 con 0x7f884414b250 2024-04-25T10:11:10.342 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:11:10.336+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:11:10.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.339+0000 7f883be00640 1 -- 172.21.15.165:0/3355510917 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v58) v1 ==== 161+0+0 (secure 0 0 0) 0x7f88440c9b70 con 0x7f884414b250 2024-04-25T10:11:10.346 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.342+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f882006b500 msgr2=0x7f882006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.342+0000 7f8849a00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f882006b500 0x7f882006d9c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8834009910 tx=0x7f8834008040 comp rx=0 tx=0).stop 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.342+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 msgr2=0x7f88441e6720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.342+0000 7f8849a00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 0x7f88441e6720 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f883c00a9c0 tx=0x7f883c001dc0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 shutdown_connections 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f882006b500 0x7f882006d9c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f884414b250 0x7f88441e6720 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f884414c5e0 0x7f88440c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 --2- 172.21.15.165:0/3355510917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8828004e50 0x7f8844149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 >> 172.21.15.165:0/3355510917 conn(0x7f88440d0610 msgr2=0x7f884415cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.343+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 shutdown_connections 2024-04-25T10:11:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.344+0000 7f8849a00640 1 -- 172.21.15.165:0/3355510917 wait complete. 2024-04-25T10:11:10.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:11:10.483 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:11:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.553+0000 7ff68ca00640 1 Processor -- start 2024-04-25T10:11:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.554+0000 7ff68ca00640 1 -- start start 2024-04-25T10:11:10.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.555+0000 7ff68ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff68806f2c0 0x7ff6880b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.555+0000 7ff68ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff6880cd1e0 0x7ff688071660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.555+0000 7ff68ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68806d5d0 con 0x7ff6880711d0 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.555+0000 7ff68ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6880b0e00 con 0x7ff68806f2c0 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.555+0000 7ff68ca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68806d8b0 con 0x7ff6880cd1e0 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff68806f2c0 0x7ff6880b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff68806f2c0 0x7ff6880b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45036/0 (socket says 172.21.15.165:45036) 2024-04-25T10:11:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 -- 172.21.15.165:0/3625863763 learned_addr learned my addr 172.21.15.165:0/3625863763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687e00640 1 --2- 172.21.15.165:0/3625863763 >> v2:172.21.15.172:3300/0 conn(0x7ff6880cd1e0 0x7ff688071660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 -- 172.21.15.165:0/3625863763 >> v2:172.21.15.172:3300/0 conn(0x7ff6880cd1e0 msgr2=0x7ff688071660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 --2- 172.21.15.165:0/3625863763 >> v2:172.21.15.172:3300/0 conn(0x7ff6880cd1e0 0x7ff688071660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 -- 172.21.15.165:0/3625863763 >> v1:172.21.15.172:6789/0 conn(0x7ff6880711d0 legacy=0x7ff68815d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.556+0000 7ff687400640 1 -- 172.21.15.165:0/3625863763 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68806db90 con 0x7ff68806f2c0 2024-04-25T10:11:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.558+0000 7ff687400640 1 --2- 172.21.15.165:0/3625863763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff68806f2c0 0x7ff6880b08c0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff68806f250 tx=0x7ff67802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5331f06a9a7925d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.558+0000 7ff686a00640 1 -- 172.21.15.165:0/3625863763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678036070 con 0x7ff68806f2c0 2024-04-25T10:11:10.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.558+0000 7ff686a00640 1 -- 172.21.15.165:0/3625863763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678031600 con 0x7ff68806f2c0 2024-04-25T10:11:10.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.558+0000 7ff686a00640 1 -- 172.21.15.165:0/3625863763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780319d0 con 0x7ff68806f2c0 2024-04-25T10:11:10.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 -- 172.21.15.165:0/3625863763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff68806f2c0 msgr2=0x7ff6880b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 --2- 172.21.15.165:0/3625863763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff68806f2c0 0x7ff6880b08c0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7ff68806f250 tx=0x7ff67802a660 comp rx=0 tx=0).stop 2024-04-25T10:11:10.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 -- 172.21.15.165:0/3625863763 shutdown_connections 2024-04-25T10:11:10.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 --2- 172.21.15.165:0/3625863763 >> v2:172.21.15.172:3300/0 conn(0x7ff6880cd1e0 0x7ff688071660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 --2- 172.21.15.165:0/3625863763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff68806f2c0 0x7ff6880b08c0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 -- 172.21.15.165:0/3625863763 >> 172.21.15.165:0/3625863763 conn(0x7ff688061cc0 msgr2=0x7ff68806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:10.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 -- 172.21.15.165:0/3625863763 shutdown_connections 2024-04-25T10:11:10.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.559+0000 7ff68ca00640 1 -- 172.21.15.165:0/3625863763 wait complete. 2024-04-25T10:11:10.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.561+0000 7ff68ca00640 1 Processor -- start 2024-04-25T10:11:10.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.561+0000 7ff68ca00640 1 -- start start 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff68ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff68806f2c0 0x7ff688155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff68ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6880711d0 0x7ff68815ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff68ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 0x7ff6880b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff68ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68806d370 con 0x7ff6880cd1e0 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff68ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68806d1f0 con 0x7ff6880711d0 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff68ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6880b1150 con 0x7ff68806f2c0 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff687e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 0x7ff6880b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff687e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 0x7ff6880b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45052/0 (socket says 172.21.15.165:45052) 2024-04-25T10:11:10.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff687e00640 1 -- 172.21.15.165:0/4232000164 learned_addr learned my addr 172.21.15.165:0/4232000164 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff687400640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff68806f2c0 0x7ff688155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff67ea00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6880711d0 0x7ff68815ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff687e00640 1 -- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff68806f2c0 msgr2=0x7ff688155190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.562+0000 7ff687e00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff68806f2c0 0x7ff688155190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff687e00640 1 -- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6880711d0 msgr2=0x7ff68815ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff687e00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6880711d0 0x7ff68815ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff687e00640 1 -- 172.21.15.165:0/4232000164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68815ca30 con 0x7ff6880cd1e0 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff67ea00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6880711d0 0x7ff68815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff687e00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 0x7ff6880b3d90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d5d0 tx=0x7ff68000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800125f0 con 0x7ff6880cd1e0 2024-04-25T10:11:10.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff680019070 con 0x7ff6880cd1e0 2024-04-25T10:11:10.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6800129f0 con 0x7ff6880cd1e0 2024-04-25T10:11:10.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68815d200 con 0x7ff6880cd1e0 2024-04-25T10:11:10.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.563+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6880a8c40 con 0x7ff6880cd1e0 2024-04-25T10:11:10.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.565+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff688073990 con 0x7ff6880cd1e0 2024-04-25T10:11:10.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.566+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff680012b90 con 0x7ff6880cd1e0 2024-04-25T10:11:10.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.567+0000 7ff67fe00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff66406b4d0 0x7ff66406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:10.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.567+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff68008a790 con 0x7ff6880cd1e0 2024-04-25T10:11:10.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.567+0000 7ff687400640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff66406b4d0 0x7ff66406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:10.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.568+0000 7ff687400640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff66406b4d0 0x7ff66406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff678033000 tx=0x7ff678009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:10.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.571+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff68815d200 con 0x7ff6880cd1e0 2024-04-25T10:11:10.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:10.813+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff688074400 con 0x7ff6880cd1e0 2024-04-25T10:11:11.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.343+0000 7ff67fe00640 1 -- 172.21.15.165:0/4232000164 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v59) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff68815ca30 con 0x7ff6880cd1e0 2024-04-25T10:11:11.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff66406b4d0 msgr2=0x7ff66406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff66406b4d0 0x7ff66406d990 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7ff678033000 tx=0x7ff678009000 comp rx=0 tx=0).stop 2024-04-25T10:11:11.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 msgr2=0x7ff6880b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 0x7ff6880b3d90 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7ff68000d5d0 tx=0x7ff68000da90 comp rx=0 tx=0).stop 2024-04-25T10:11:11.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 shutdown_connections 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff66406b4d0 0x7ff66406d990 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff6880cd1e0 0x7ff6880b3d90 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff6880711d0 0x7ff68815ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 --2- 172.21.15.165:0/4232000164 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff68806f2c0 0x7ff688155190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 >> 172.21.15.165:0/4232000164 conn(0x7ff688061cc0 msgr2=0x7ff6880cb2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 shutdown_connections 2024-04-25T10:11:11.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.347+0000 7ff68ca00640 1 -- 172.21.15.165:0/4232000164 wait complete. 2024-04-25T10:11:11.375 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:11:11.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.579+0000 7fd5b0c00640 1 Processor -- start 2024-04-25T10:11:11.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.579+0000 7fd5b0c00640 1 -- start start 2024-04-25T10:11:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.581+0000 7fd5b0c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 0x7fd5ac0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.581+0000 7fd5b0c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac145520 con 0x7fd5ac0c5030 2024-04-25T10:11:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.581+0000 7fd5b0c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5ac071270 con 0x7fd5ac145140 2024-04-25T10:11:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.581+0000 7fd5b0c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac071550 con 0x7fd5ac0ceb20 2024-04-25T10:11:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.581+0000 7fd5abe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd5ac0c5030 0x7fd5ac15bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40674/0 (socket says 172.21.15.165:40674) 2024-04-25T10:11:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.581+0000 7fd5abe00640 1 -- 172.21.15.165:0/569751665 learned_addr learned my addr 172.21.15.165:0/569751665 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 0x7fd5ac0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278259197 0 0) 0x7fd5ac145520 con 0x7fd5ac0c5030 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd590003520 con 0x7fd5ac0c5030 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 579910830 0 0) 0x7fd590003520 con 0x7fd5ac0c5030 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 msgr2=0x7fd5ac0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 0x7fd5ac0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 >> v1:172.21.15.172:6789/0 conn(0x7fd5ac145140 legacy=0x7fd5ac0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac071830 con 0x7fd5ac0c5030 2024-04-25T10:11:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd5a40019f0 con 0x7fd5ac0c5030 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 >> v1:172.21.15.165:6789/0 conn(0x7fd5ac0c5030 legacy=0x7fd5ac15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5900039e0 0x7fd590003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 0x7fd590005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac071550 con 0x7fd590004800 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd59000baf0 con 0x7fd5900039e0 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.582+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59000bcf0 con 0x7fd5ac0ceb20 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5aaa00640 1 -- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 msgr2=0x7fd590005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5ab400640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5900039e0 0x7fd590003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5abe00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 0x7fd590005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac0ceb20 0x7fd590005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5aaa00640 1 -- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5900039e0 msgr2=0x7fd590003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5900039e0 0x7fd590003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.583+0000 7fd5aaa00640 1 -- 172.21.15.165:0/569751665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd590003520 con 0x7fd590004800 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.584+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd590004c80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ac0 tx=0x7fd590005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d5b642340dd499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.584+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598036070 con 0x7fd590004800 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.584+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd598031400 con 0x7fd590004800 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.584+0000 7fd5aa000640 1 -- 172.21.15.165:0/569751665 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd598009690 con 0x7fd590004800 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5b0c00640 1 -- 172.21.15.165:0/569751665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 msgr2=0x7fd590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/569751665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd590004c80 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fd598009ac0 tx=0x7fd590005830 comp rx=0 tx=0).stop 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5abe00640 1 -- 172.21.15.165:0/569751665 reap_dead start 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5b0c00640 1 -- 172.21.15.165:0/569751665 shutdown_connections 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5b0c00640 1 -- 172.21.15.165:0/569751665 >> 172.21.15.165:0/569751665 conn(0x7fd5ac061cc0 msgr2=0x7fd5ac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5b0c00640 1 -- 172.21.15.165:0/569751665 shutdown_connections 2024-04-25T10:11:11.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.585+0000 7fd5b0c00640 1 -- 172.21.15.165:0/569751665 wait complete. 2024-04-25T10:11:11.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.587+0000 7fd5b0c00640 1 Processor -- start 2024-04-25T10:11:11.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.587+0000 7fd5b0c00640 1 -- start start 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5b0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd5ac13e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5b0c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac0a84a0 0x7fd5ac0a93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5b0c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac1f22a0 0x7fd5ac0aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5b0c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c5640 con 0x7fd590004800 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5b0c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c54c0 con 0x7fd5ac0a84a0 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5b0c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5ac0c4af0 con 0x7fd5ac1f22a0 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd5ac13e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd5ac13e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45076/0 (socket says 172.21.15.165:45076) 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.588+0000 7fd5ab400640 1 -- 172.21.15.165:0/3394339398 learned_addr learned my addr 172.21.15.165:0/3394339398 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5ab400640 1 -- 172.21.15.165:0/3394339398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac1f22a0 msgr2=0x7fd5ac0aaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5ab400640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac1f22a0 0x7fd5ac0aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5ab400640 1 -- 172.21.15.165:0/3394339398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac0a84a0 msgr2=0x7fd5ac0a93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:11.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5ab400640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac0a84a0 0x7fd5ac0a93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:11.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5ab400640 1 -- 172.21.15.165:0/3394339398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5ac0a9900 con 0x7fd590004800 2024-04-25T10:11:11.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5ab400640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd5ac13e7d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00d570 tx=0x7fd59c00da30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c005480 con 0x7fd590004800 2024-04-25T10:11:11.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c017070 con 0x7fd590004800 2024-04-25T10:11:11.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5ac0c8b30 con 0x7fd590004800 2024-04-25T10:11:11.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.589+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c003810 con 0x7fd590004800 2024-04-25T10:11:11.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.590+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5ac0c90f0 con 0x7fd590004800 2024-04-25T10:11:11.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.591+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5ac073990 con 0x7fd590004800 2024-04-25T10:11:11.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.592+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59c0039b0 con 0x7fd590004800 2024-04-25T10:11:11.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.593+0000 7fd5a3400640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd58c06b4d0 0x7fd58c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:11.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.593+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd59c08a220 con 0x7fd590004800 2024-04-25T10:11:11.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.593+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd58c06b4d0 0x7fd58c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:11.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.594+0000 7fd5aaa00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd58c06b4d0 0x7fd58c06d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd598033000 tx=0x7fd59802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:11.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.598+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd59c058eb0 con 0x7fd590004800 2024-04-25T10:11:11.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:11.795+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd5ac0b4570 con 0x7fd590004800 2024-04-25T10:11:12.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.346+0000 7fd5a3400640 1 -- 172.21.15.165:0/3394339398 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v89) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd59c0587b0 con 0x7fd590004800 2024-04-25T10:11:12.353 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.350+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd58c06b4d0 msgr2=0x7fd58c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.350+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd58c06b4d0 0x7fd58c06d990 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fd598033000 tx=0x7fd59802e000 comp rx=0 tx=0).stop 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.350+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 msgr2=0x7fd5ac13e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.350+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd5ac13e7d0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fd59c00d570 tx=0x7fd59c00da30 comp rx=0 tx=0).stop 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.351+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 shutdown_connections 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.351+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd58c06b4d0 0x7fd58c06d990 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.351+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5ac1f22a0 0x7fd5ac0aaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.351+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5ac0a84a0 0x7fd5ac0a93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.351+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3394339398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd590004800 0x7fd5ac13e7d0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.351+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 >> 172.21.15.165:0/3394339398 conn(0x7fd5ac061cc0 msgr2=0x7fd5ac15ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.352+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 shutdown_connections 2024-04-25T10:11:12.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.352+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3394339398 wait complete. 2024-04-25T10:11:12.380 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:11:12.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.574+0000 7f3067e00640 1 Processor -- start 2024-04-25T10:11:12.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.574+0000 7f3067e00640 1 -- start start 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3067e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30680b5b40 0x7f3068168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3067e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f306815e110 con 0x7f306815dd30 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3067e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30680b2fd0 con 0x7f306806bff0 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3067e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f306806d000 con 0x7f30680b5b40 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3066a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f306815dd30 0x7f30680cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40680/0 (socket says 172.21.15.165:40680) 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3066a00640 1 -- 172.21.15.165:0/2508710360 learned_addr learned my addr 172.21.15.165:0/2508710360 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:12.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.576+0000 7f3067400640 1 --2- 172.21.15.165:0/2508710360 >> v2:172.21.15.165:3300/0 conn(0x7f30680b5b40 0x7f3068168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.577+0000 7f3067400640 1 -- 172.21.15.165:0/2508710360 >> v1:172.21.15.172:6789/0 conn(0x7f306806bff0 legacy=0x7f30680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:12.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.577+0000 7f3067400640 1 -- 172.21.15.165:0/2508710360 >> v1:172.21.15.165:6789/0 conn(0x7f306815dd30 legacy=0x7f30680cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.577+0000 7f3067400640 1 -- 172.21.15.165:0/2508710360 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f306806c480 con 0x7f30680b5b40 2024-04-25T10:11:12.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.578+0000 7f3067400640 1 --2- 172.21.15.165:0/2508710360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30680b5b40 0x7f3068168b50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3060009d90 tx=0x7f306002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc9e7ae6beb6dc81 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.578+0000 7f3065600640 1 -- 172.21.15.165:0/2508710360 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060007740 con 0x7f30680b5b40 2024-04-25T10:11:12.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.578+0000 7f3065600640 1 -- 172.21.15.165:0/2508710360 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30600078e0 con 0x7f30680b5b40 2024-04-25T10:11:12.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.578+0000 7f3065600640 1 -- 172.21.15.165:0/2508710360 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3060007cb0 con 0x7f30680b5b40 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.578+0000 7f3067e00640 1 -- 172.21.15.165:0/2508710360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30680b5b40 msgr2=0x7f3068168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.578+0000 7f3067e00640 1 --2- 172.21.15.165:0/2508710360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30680b5b40 0x7f3068168b50 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f3060009d90 tx=0x7f306002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.579+0000 7f3067e00640 1 -- 172.21.15.165:0/2508710360 shutdown_connections 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.579+0000 7f3067e00640 1 --2- 172.21.15.165:0/2508710360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30680b5b40 0x7f3068168b50 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.579+0000 7f3067e00640 1 -- 172.21.15.165:0/2508710360 >> 172.21.15.165:0/2508710360 conn(0x7f30680d0610 msgr2=0x7f30680d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.579+0000 7f3067e00640 1 -- 172.21.15.165:0/2508710360 shutdown_connections 2024-04-25T10:11:12.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.579+0000 7f3067e00640 1 -- 172.21.15.165:0/2508710360 wait complete. 2024-04-25T10:11:12.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.580+0000 7f3067e00640 1 Processor -- start 2024-04-25T10:11:12.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.581+0000 7f3067e00640 1 -- start start 2024-04-25T10:11:12.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.581+0000 7f3067e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30680b5b40 0x7f306814df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.581+0000 7f3067e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 0x7f30680cc240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.581+0000 7f3067e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30680cb700 0x7f306815b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3067e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306806cd20 con 0x7f306815dd30 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3067e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306806ca40 con 0x7f30680b5b40 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3067e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30680d3170 con 0x7f30680cb700 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 0x7f30680cc240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 0x7f30680cc240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45088/0 (socket says 172.21.15.165:45088) 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 -- 172.21.15.165:0/752923138 learned_addr learned my addr 172.21.15.165:0/752923138 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066a00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30680b5b40 0x7f306814df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 -- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30680cb700 msgr2=0x7f306815b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30680cb700 0x7f306815b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 -- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30680b5b40 msgr2=0x7f306814df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30680b5b40 0x7f306814df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.582+0000 7f3066000640 1 -- 172.21.15.165:0/752923138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30681f3750 con 0x7f306815dd30 2024-04-25T10:11:12.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.583+0000 7f3066000640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 0x7f30680cc240 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f305000c430 tx=0x7f305000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.583+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050011790 con 0x7f306815dd30 2024-04-25T10:11:12.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.583+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050017070 con 0x7f306815dd30 2024-04-25T10:11:12.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.583+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050011bb0 con 0x7f306815dd30 2024-04-25T10:11:12.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.583+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30681f39f0 con 0x7f306815dd30 2024-04-25T10:11:12.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.583+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30681f3fb0 con 0x7f306815dd30 2024-04-25T10:11:12.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.584+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3068073990 con 0x7f306815dd30 2024-04-25T10:11:12.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.585+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f305001c460 con 0x7f306815dd30 2024-04-25T10:11:12.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.586+0000 7f305ea00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f304406b500 0x7f304406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:12.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.586+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f305008a0f0 con 0x7f306815dd30 2024-04-25T10:11:12.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.586+0000 7f3066a00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f304406b500 0x7f304406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:12.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.587+0000 7f3066a00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f304406b500 0x7f304406d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3068157da0 tx=0x7f3058009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:12.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.590+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3050058ea0 con 0x7f306815dd30 2024-04-25T10:11:12.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:12.794+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30680b9600 con 0x7f306815dd30 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.358+0000 7f305ea00640 1 -- 172.21.15.165:0/752923138 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v90) v1 ==== 169+0+0 (secure 0 0 0) 0x7f30500587a0 con 0x7f306815dd30 2024-04-25T10:11:13.364 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:11:13.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.365+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f304406b500 msgr2=0x7f304406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.365+0000 7f3067e00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f304406b500 0x7f304406d9c0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f3068157da0 tx=0x7f3058009040 comp rx=0 tx=0).stop 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.365+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 msgr2=0x7f30680cc240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.365+0000 7f3067e00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 0x7f30680cc240 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f305000c430 tx=0x7f305000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.366+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 shutdown_connections 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.366+0000 7f3067e00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f304406b500 0x7f304406d9c0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.366+0000 7f3067e00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30680cb700 0x7f306815b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.366+0000 7f3067e00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f306815dd30 0x7f30680cc240 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.366+0000 7f3067e00640 1 --2- 172.21.15.165:0/752923138 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30680b5b40 0x7f306814df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.366+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 >> 172.21.15.165:0/752923138 conn(0x7f30680d0610 msgr2=0x7f30680b4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.367+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 shutdown_connections 2024-04-25T10:11:13.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.367+0000 7f3067e00640 1 -- 172.21.15.165:0/752923138 wait complete. 2024-04-25T10:11:13.398 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:11:13.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.578+0000 7fc9cd000640 1 Processor -- start 2024-04-25T10:11:13.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.578+0000 7fc9cd000640 1 -- start start 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9cd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c806c330 0x7fc9c80b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9cd000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9c806c7e0 0x7fc9c80b53a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9cd000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9c80d1d50 con 0x7fc9c8158230 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9cd000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9c8158610 con 0x7fc9c806c7e0 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9cd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c806bea0 con 0x7fc9c806c330 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9c5e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc9c8158230 0x7fc9c80d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40684/0 (socket says 172.21.15.165:40684) 2024-04-25T10:11:13.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9c5e00640 1 -- 172.21.15.165:0/1993487789 learned_addr learned my addr 172.21.15.165:0/1993487789 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:13.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9c6800640 1 --2- 172.21.15.165:0/1993487789 >> v2:172.21.15.165:3300/0 conn(0x7fc9c806c7e0 0x7fc9c80b53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9c5400640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c806c330 0x7fc9c80b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.580+0000 7fc9c4a00640 1 -- 172.21.15.165:0/1993487789 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967416090 0 0) 0x7fc9c80d1d50 con 0x7fc9c8158230 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.581+0000 7fc9c4a00640 1 -- 172.21.15.165:0/1993487789 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9ac003520 con 0x7fc9c8158230 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.581+0000 7fc9c6800640 1 -- 172.21.15.165:0/1993487789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c806c330 msgr2=0x7fc9c80b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.581+0000 7fc9c6800640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c806c330 0x7fc9c80b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.581+0000 7fc9c6800640 1 -- 172.21.15.165:0/1993487789 >> v1:172.21.15.165:6789/0 conn(0x7fc9c8158230 legacy=0x7fc9c80d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.581+0000 7fc9c6800640 1 -- 172.21.15.165:0/1993487789 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c80d2020 con 0x7fc9c806c7e0 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.581+0000 7fc9c5400640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c806c330 0x7fc9c80b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.582+0000 7fc9c6800640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c80b53a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009d90 tx=0x7fc9c002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=355f1cc01935498b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.582+0000 7fc9c4a00640 1 -- 172.21.15.165:0/1993487789 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0007740 con 0x7fc9c806c7e0 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.582+0000 7fc9c4a00640 1 -- 172.21.15.165:0/1993487789 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c00078e0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.582+0000 7fc9c4a00640 1 -- 172.21.15.165:0/1993487789 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0007cb0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 -- 172.21.15.165:0/1993487789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 msgr2=0x7fc9c80b53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c80b53a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fc9c0009d90 tx=0x7fc9c002a660 comp rx=0 tx=0).stop 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 -- 172.21.15.165:0/1993487789 shutdown_connections 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c80b53a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1993487789 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c806c330 0x7fc9c80b4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 -- 172.21.15.165:0/1993487789 >> 172.21.15.165:0/1993487789 conn(0x7fc9c8061600 msgr2=0x7fc9c80619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 -- 172.21.15.165:0/1993487789 shutdown_connections 2024-04-25T10:11:13.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.583+0000 7fc9cd000640 1 -- 172.21.15.165:0/1993487789 wait complete. 2024-04-25T10:11:13.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.585+0000 7fc9cd000640 1 Processor -- start 2024-04-25T10:11:13.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.585+0000 7fc9cd000640 1 -- start start 2024-04-25T10:11:13.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9cd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9c806c330 0x7fc9c814f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9cd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c8158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9cd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c8158230 0x7fc9c8158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9cd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c80d25e0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9cd000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c806bb30 con 0x7fc9c806c330 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9cd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c815e320 con 0x7fc9c8158230 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c8158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c8158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45110/0 (socket says 172.21.15.165:45110) 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 -- 172.21.15.165:0/1788319012 learned_addr learned my addr 172.21.15.165:0/1788319012 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 -- 172.21.15.165:0/1788319012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c8158230 msgr2=0x7fc9c8158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c8158230 0x7fc9c8158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 -- 172.21.15.165:0/1788319012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9c806c330 msgr2=0x7fc9c814f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9c806c330 0x7fc9c814f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.586+0000 7fc9c5400640 1 -- 172.21.15.165:0/1788319012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c814fbc0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.587+0000 7fc9c5400640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c8158790 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c7f0 tx=0x7fc9b400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.587+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4004220 con 0x7fc9c806c7e0 2024-04-25T10:11:13.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.587+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9b40043c0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.587+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9b4005bb0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.587+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c81598d0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.587+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c8159e90 con 0x7fc9c806c7e0 2024-04-25T10:11:13.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.588+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c80739e0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.589+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc9b4006030 con 0x7fc9c806c7e0 2024-04-25T10:11:13.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.590+0000 7fc9be000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc9a806b500 0x7fc9a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:13.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.590+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc9b408a6f0 con 0x7fc9c806c7e0 2024-04-25T10:11:13.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.590+0000 7fc9c5e00640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc9a806b500 0x7fc9a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:13.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.591+0000 7fc9c5e00640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc9a806b500 0x7fc9a806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8000f60 tx=0x7fc9b80078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:13.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.595+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9b401e020 con 0x7fc9c806c7e0 2024-04-25T10:11:13.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.811+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fc9c80d0b80 con 0x7fc9c806c7e0 2024-04-25T10:11:13.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.816+0000 7fc9be000640 1 -- 172.21.15.165:0/1788319012 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc9b4015030 con 0x7fc9c806c7e0 2024-04-25T10:11:13.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc9a806b500 msgr2=0x7fc9a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc9a806b500 0x7fc9a806d9c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fc9b8000f60 tx=0x7fc9b80078f0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 msgr2=0x7fc9c8158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:13.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c8158790 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fc9b400c7f0 tx=0x7fc9b400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 shutdown_connections 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc9a806b500 0x7fc9a806d9c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9c8158230 0x7fc9c8158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9c806c7e0 0x7fc9c8158790 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 --2- 172.21.15.165:0/1788319012 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc9c806c330 0x7fc9c814f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.819+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 >> 172.21.15.165:0/1788319012 conn(0x7fc9c8061600 msgr2=0x7fc9c80cee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.820+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 shutdown_connections 2024-04-25T10:11:13.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:13.820+0000 7fc9cd000640 1 -- 172.21.15.165:0/1788319012 wait complete. 2024-04-25T10:11:13.843 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub' 2024-04-25T10:11:14.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.025+0000 7efd69400640 1 Processor -- start 2024-04-25T10:11:14.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.026+0000 7efd69400640 1 -- start start 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd69400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 0x7efd640b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd69400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd640d06c0 0x7efd640d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd69400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd64161df0 con 0x7efd641615a0 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd69400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd640d1f50 con 0x7efd640d06c0 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd69400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd641620d0 con 0x7efd6415db90 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd63400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efd641615a0 0x7efd640cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40698/0 (socket says 172.21.15.165:40698) 2024-04-25T10:11:14.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.027+0000 7efd63400640 1 -- 172.21.15.165:0/3188061615 learned_addr learned my addr 172.21.15.165:0/3188061615 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd68a00640 1 --2- 172.21.15.165:0/3188061615 >> v2:172.21.15.172:3300/0 conn(0x7efd640d06c0 0x7efd640d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd63e00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 0x7efd640b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213502364 0 0) 0x7efd64161df0 con 0x7efd641615a0 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd48003520 con 0x7efd641615a0 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1202445368 0 0) 0x7efd48003520 con 0x7efd641615a0 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 msgr2=0x7efd640b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 0x7efd640b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 >> v2:172.21.15.172:3300/0 conn(0x7efd640d06c0 msgr2=0x7efd640d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 --2- 172.21.15.165:0/3188061615 >> v2:172.21.15.172:3300/0 conn(0x7efd640d06c0 0x7efd640d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd641623b0 con 0x7efd641615a0 2024-04-25T10:11:14.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7efd50001670 con 0x7efd641615a0 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 >> v1:172.21.15.165:6789/0 conn(0x7efd641615a0 legacy=0x7efd640cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 0x7efd480039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48003fe0 0x7efd48004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd640d1f50 con 0x7efd48004d40 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd641620d0 con 0x7efd48003fe0 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd4800bac0 con 0x7efd6415db90 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.028+0000 7efd68a00640 1 --2- 172.21.15.165:0/3188061615 >> v2:172.21.15.172:3300/0 conn(0x7efd640d06c0 0x7efd640d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd68a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd63400640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48003fe0 0x7efd48004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd63e00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 0x7efd480039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd68a00640 1 -- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 msgr2=0x7efd480039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd68a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd6415db90 0x7efd480039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd68a00640 1 -- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48003fe0 msgr2=0x7efd48004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd68a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd48003fe0 0x7efd48004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.029+0000 7efd68a00640 1 -- 172.21.15.165:0/3188061615 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd48003520 con 0x7efd48004d40 2024-04-25T10:11:14.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.030+0000 7efd68a00640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd480051c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7efd640d0ad0 tx=0x7efd48005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bf22bbec090803 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.030+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c037070 con 0x7efd48004d40 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.030+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c02ad00 con 0x7efd48004d40 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.030+0000 7efd62a00640 1 -- 172.21.15.165:0/3188061615 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c00f040 con 0x7efd48004d40 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd69400640 1 -- 172.21.15.165:0/3188061615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 msgr2=0x7efd480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd69400640 1 --2- 172.21.15.165:0/3188061615 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd480051c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7efd640d0ad0 tx=0x7efd48005830 comp rx=0 tx=0).stop 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd68a00640 1 -- 172.21.15.165:0/3188061615 reap_dead start 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd69400640 1 -- 172.21.15.165:0/3188061615 shutdown_connections 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd69400640 1 -- 172.21.15.165:0/3188061615 >> 172.21.15.165:0/3188061615 conn(0x7efd640b52b0 msgr2=0x7efd640b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd69400640 1 -- 172.21.15.165:0/3188061615 shutdown_connections 2024-04-25T10:11:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.031+0000 7efd69400640 1 -- 172.21.15.165:0/3188061615 wait complete. 2024-04-25T10:11:14.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.034+0000 7efd69400640 1 Processor -- start 2024-04-25T10:11:14.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.034+0000 7efd69400640 1 -- start start 2024-04-25T10:11:14.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd69400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd640cd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd69400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd641604c0 0x7efd641f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd69400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd641f3a20 0x7efd641f3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd69400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64161ba0 con 0x7efd48004d40 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd69400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd64161a20 con 0x7efd641604c0 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd69400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd640b68d0 con 0x7efd641f3a20 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd63e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd640cd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.035+0000 7efd63e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd640cd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45126/0 (socket says 172.21.15.165:45126) 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 -- 172.21.15.165:0/2579597020 learned_addr learned my addr 172.21.15.165:0/2579597020 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd641604c0 0x7efd641f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 -- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd641f3a20 msgr2=0x7efd641f3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd68a00640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd641f3a20 0x7efd641f3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd641f3a20 0x7efd641f3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 -- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd641604c0 msgr2=0x7efd641f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd641604c0 0x7efd641f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 -- 172.21.15.165:0/2579597020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd64160a70 con 0x7efd48004d40 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd641604c0 0x7efd641f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.036+0000 7efd63e00640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd640cd4b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7efd54009cb0 tx=0x7efd54007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.037+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54014610 con 0x7efd48004d40 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.037+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd54019070 con 0x7efd48004d40 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.037+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540149d0 con 0x7efd48004d40 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.037+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd64160d50 con 0x7efd48004d40 2024-04-25T10:11:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.037+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd6415b6d0 con 0x7efd48004d40 2024-04-25T10:11:14.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.038+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd64073990 con 0x7efd48004d40 2024-04-25T10:11:14.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.039+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd54014b70 con 0x7efd48004d40 2024-04-25T10:11:14.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.040+0000 7efd5be00640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd4406b4d0 0x7efd4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.040+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efd5408a9b0 con 0x7efd48004d40 2024-04-25T10:11:14.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.041+0000 7efd63400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd4406b4d0 0x7efd4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.041+0000 7efd63400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd4406b4d0 0x7efd4406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd50001eb0 tx=0x7efd50000f30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.045+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd54059a30 con 0x7efd48004d40 2024-04-25T10:11:14.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.313+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]} v 0) v1 -- 0x7efd64158fb0 con 0x7efd48004d40 2024-04-25T10:11:14.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.361+0000 7efd5be00640 1 -- 172.21.15.165:0/2579597020 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubChecks.test_stray_evaluation_with_scrub"]}]=0 v423) v1 ==== 158+0+0 (secure 0 0 0) 0x7efd54059330 con 0x7efd48004d40 2024-04-25T10:11:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.364+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd4406b4d0 msgr2=0x7efd4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd4406b4d0 0x7efd4406d990 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efd50001eb0 tx=0x7efd50000f30 comp rx=0 tx=0).stop 2024-04-25T10:11:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 msgr2=0x7efd640cd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd640cd4b0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7efd54009cb0 tx=0x7efd54007700 comp rx=0 tx=0).stop 2024-04-25T10:11:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 shutdown_connections 2024-04-25T10:11:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd4406b4d0 0x7efd4406d990 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd641f3a20 0x7efd641f3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd641604c0 0x7efd641f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 --2- 172.21.15.165:0/2579597020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd48004d40 0x7efd640cd4b0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 >> 172.21.15.165:0/2579597020 conn(0x7efd640b52b0 msgr2=0x7efd640b71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 shutdown_connections 2024-04-25T10:11:14.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.365+0000 7efd69400640 1 -- 172.21.15.165:0/2579597020 wait complete. 2024-04-25T10:11:14.390 INFO:tasks.cephfs_test_runner:test that scrub can iterate over ~mdsdir and evaluate strays ... ok 2024-04-25T10:11:14.390 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:11:14.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:14.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.583+0000 7fe907200640 1 Processor -- start 2024-04-25T10:11:14.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.584+0000 7fe907200640 1 -- start start 2024-04-25T10:11:14.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe907200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe90815db90 0x7fe9080b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe907200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe908161df0 con 0x7fe9080d06c0 2024-04-25T10:11:14.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe907200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9081620d0 con 0x7fe9081615a0 2024-04-25T10:11:14.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe907200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9080d1f50 con 0x7fe90815db90 2024-04-25T10:11:14.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe905e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe90815db90 0x7fe9080b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe906800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe9080d06c0 0x7fe9080d23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40714/0 (socket says 172.21.15.165:40714) 2024-04-25T10:11:14.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.585+0000 7fe906800640 1 -- 172.21.15.165:0/1022567287 learned_addr learned my addr 172.21.15.165:0/1022567287 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:14.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.586+0000 7fe905e00640 1 -- 172.21.15.165:0/1022567287 >> v1:172.21.15.172:6789/0 conn(0x7fe9081615a0 legacy=0x7fe9080cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:14.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.586+0000 7fe905e00640 1 -- 172.21.15.165:0/1022567287 >> v1:172.21.15.165:6789/0 conn(0x7fe9080d06c0 legacy=0x7fe9080d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.586+0000 7fe905e00640 1 -- 172.21.15.165:0/1022567287 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9081623b0 con 0x7fe90815db90 2024-04-25T10:11:14.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.587+0000 7fe905e00640 1 --2- 172.21.15.165:0/1022567287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90815db90 0x7fe9080b93c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097a0 tx=0x7fe8f802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f19ab55a019f102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.587+0000 7fe905400640 1 -- 172.21.15.165:0/1022567287 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8037070 con 0x7fe90815db90 2024-04-25T10:11:14.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.587+0000 7fe905400640 1 -- 172.21.15.165:0/1022567287 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f8032440 con 0x7fe90815db90 2024-04-25T10:11:14.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.587+0000 7fe905400640 1 -- 172.21.15.165:0/1022567287 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8032810 con 0x7fe90815db90 2024-04-25T10:11:14.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 -- 172.21.15.165:0/1022567287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90815db90 msgr2=0x7fe9080b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 --2- 172.21.15.165:0/1022567287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90815db90 0x7fe9080b93c0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097a0 tx=0x7fe8f802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 -- 172.21.15.165:0/1022567287 shutdown_connections 2024-04-25T10:11:14.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 --2- 172.21.15.165:0/1022567287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe90815db90 0x7fe9080b93c0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 -- 172.21.15.165:0/1022567287 >> 172.21.15.165:0/1022567287 conn(0x7fe9080b52b0 msgr2=0x7fe9080b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 -- 172.21.15.165:0/1022567287 shutdown_connections 2024-04-25T10:11:14.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.588+0000 7fe907200640 1 -- 172.21.15.165:0/1022567287 wait complete. 2024-04-25T10:11:14.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.590+0000 7fe907200640 1 Processor -- start 2024-04-25T10:11:14.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.590+0000 7fe907200640 1 -- start start 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe907200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 0x7fe90814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe907200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90815db90 0x7fe90815ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe907200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9081615a0 0x7fe9080c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe907200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe908161ba0 con 0x7fe9080d06c0 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe907200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe908161a20 con 0x7fe90815db90 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe907200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9080b68d0 con 0x7fe9081615a0 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe905e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 0x7fe90814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe905e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 0x7fe90814e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45134/0 (socket says 172.21.15.165:45134) 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.591+0000 7fe905e00640 1 -- 172.21.15.165:0/1152912731 learned_addr learned my addr 172.21.15.165:0/1152912731 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe8fd400640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90815db90 0x7fe90815ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe906800640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9081615a0 0x7fe9080c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe905e00640 1 -- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9081615a0 msgr2=0x7fe9080c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe905e00640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9081615a0 0x7fe9080c8cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe905e00640 1 -- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90815db90 msgr2=0x7fe90815ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe905e00640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90815db90 0x7fe90815ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe905e00640 1 -- 172.21.15.165:0/1152912731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90814ee00 con 0x7fe9080d06c0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe8fd400640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90815db90 0x7fe90815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.592+0000 7fe905e00640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 0x7fe90814e8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097a0 tx=0x7fe8f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.593+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8041070 con 0x7fe9080d06c0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.593+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f803c870 con 0x7fe9080d06c0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.593+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8037040 con 0x7fe9080d06c0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.593+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9080cad70 con 0x7fe9080d06c0 2024-04-25T10:11:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.593+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9080cb330 con 0x7fe9080d06c0 2024-04-25T10:11:14.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.594+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90814fca0 con 0x7fe9080d06c0 2024-04-25T10:11:14.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.595+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8f8044070 con 0x7fe9080d06c0 2024-04-25T10:11:14.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.596+0000 7fe8fea00640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:14.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.596+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe8f80aac90 con 0x7fe9080d06c0 2024-04-25T10:11:14.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.596+0000 7fe8fd400640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:14.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.597+0000 7fe8fd400640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe908068130 tx=0x7fe8f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:14.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.601+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe90814fca0 con 0x7fe9080d06c0 2024-04-25T10:11:14.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.798+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe908073990 con 0x7fe9080d06c0 2024-04-25T10:11:14.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.801+0000 7fe8fea00640 1 -- 172.21.15.165:0/1152912731 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) v1 ==== 74+0+13332 (secure 0 0 0) 0x7fe8f8079d90 con 0x7fe9080d06c0 2024-04-25T10:11:14.807 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:14.807 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:13.348657+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6833/1170361110":"2024-04-26T10:11:10.338063+0000","172.21.15.172:6832/1170361110":"2024-04-26T10:11:10.338063+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:14.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 msgr2=0x7fe8e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe908068130 tx=0x7fe8f0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 msgr2=0x7fe90814e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 0x7fe90814e8c0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097a0 tx=0x7fe8f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 shutdown_connections 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9081615a0 0x7fe9080c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe90815db90 0x7fe90815ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 --2- 172.21.15.165:0/1152912731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9080d06c0 0x7fe90814e8c0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 >> 172.21.15.165:0/1152912731 conn(0x7fe9080b52b0 msgr2=0x7fe9080d27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.803+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 shutdown_connections 2024-04-25T10:11:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:14.804+0000 7fe907200640 1 -- 172.21.15.165:0/1152912731 wait complete. 2024-04-25T10:11:14.827 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:14.828 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:15.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.022+0000 7fb9a1600640 1 Processor -- start 2024-04-25T10:11:15.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.023+0000 7fb9a1600640 1 -- start start 2024-04-25T10:11:15.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb9a1600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb99c0cc4f0 0x7fb99c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb9a1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c0cdb20 0x7fb99c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb9a1600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb99c15b270 0x7fb99c15b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb9a1600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c0ce4c0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb9a1600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb99c071270 con 0x7fb99c15b270 2024-04-25T10:11:15.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb9a1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c072280 con 0x7fb99c0cdb20 2024-04-25T10:11:15.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb99aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb99c0cc4f0 0x7fb99c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb99aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb99c0cc4f0 0x7fb99c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45138/0 (socket says 172.21.15.165:45138) 2024-04-25T10:11:15.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.024+0000 7fb99aa00640 1 -- 172.21.15.165:0/3192597168 learned_addr learned my addr 172.21.15.165:0/3192597168 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99a000640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c0cdb20 0x7fb99c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99aa00640 1 -- 172.21.15.165:0/3192597168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c0cdb20 msgr2=0x7fb99c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99b400640 1 --2- 172.21.15.165:0/3192597168 >> v2:172.21.15.172:3300/0 conn(0x7fb99c15b270 0x7fb99c15b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99aa00640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c0cdb20 0x7fb99c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99aa00640 1 -- 172.21.15.165:0/3192597168 >> v2:172.21.15.172:3300/0 conn(0x7fb99c15b270 msgr2=0x7fb99c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99aa00640 1 --2- 172.21.15.165:0/3192597168 >> v2:172.21.15.172:3300/0 conn(0x7fb99c15b270 0x7fb99c15b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99aa00640 1 -- 172.21.15.165:0/3192597168 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c0714c0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.025+0000 7fb99a000640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c0cdb20 0x7fb99c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.026+0000 7fb99aa00640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c0cc8d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb99c158aa0 tx=0x7fb98c02a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3603feb7aa07d8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.026+0000 7fb999600640 1 -- 172.21.15.165:0/3192597168 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c037070 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.026+0000 7fb999600640 1 -- 172.21.15.165:0/3192597168 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c032440 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.026+0000 7fb999600640 1 -- 172.21.15.165:0/3192597168 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c032810 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 -- 172.21.15.165:0/3192597168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 msgr2=0x7fb99c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c0cc8d0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb99c158aa0 tx=0x7fb98c02a1d0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 -- 172.21.15.165:0/3192597168 shutdown_connections 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3192597168 >> v2:172.21.15.172:3300/0 conn(0x7fb99c15b270 0x7fb99c15b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c0cdb20 0x7fb99c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3192597168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c0cc8d0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 -- 172.21.15.165:0/3192597168 >> 172.21.15.165:0/3192597168 conn(0x7fb99c06abe0 msgr2=0x7fb99c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 -- 172.21.15.165:0/3192597168 shutdown_connections 2024-04-25T10:11:15.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.027+0000 7fb9a1600640 1 -- 172.21.15.165:0/3192597168 wait complete. 2024-04-25T10:11:15.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.029+0000 7fb9a1600640 1 Processor -- start 2024-04-25T10:11:15.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.029+0000 7fb9a1600640 1 -- start start 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb9a1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c143500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb9a1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99c0cdb20 0x7fb99c1466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb9a1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c15b270 0x7fb99c145290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb9a1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c071d60 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb9a1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c071a80 con 0x7fb99c0cdb20 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb9a1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb99c15bbc0 con 0x7fb99c15b270 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb99aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c143500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb99aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c143500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45140/0 (socket says 172.21.15.165:45140) 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb99aa00640 1 -- 172.21.15.165:0/3074146026 learned_addr learned my addr 172.21.15.165:0/3074146026 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb99a000640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99c0cdb20 0x7fb99c1466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb99aa00640 1 -- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c15b270 msgr2=0x7fb99c145290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.030+0000 7fb99aa00640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c15b270 0x7fb99c145290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb99aa00640 1 -- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99c0cdb20 msgr2=0x7fb99c1466a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb99aa00640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99c0cdb20 0x7fb99c1466a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb99aa00640 1 -- 172.21.15.165:0/3074146026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb99c146be0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb99a000640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99c0cdb20 0x7fb99c1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb99aa00640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c143500 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb98c000c00 tx=0x7fb98c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c037040 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb98c042070 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99c1441f0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98c030bb0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.031+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99c1447b0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.033+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99c073a00 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.033+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb98c047460 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.034+0000 7fb992a00640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb97806b4d0 0x7fb97806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.035+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb98c0aada0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.035+0000 7fb99a000640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb97806b4d0 0x7fb97806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.035+0000 7fb99a000640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb97806b4d0 0x7fb97806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb99c153410 tx=0x7fb984008450 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.038+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb98c045360 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.234+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb99c0ce650 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.235+0000 7fb992a00640 1 -- 172.21.15.165:0/3074146026 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v90) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb98c079cf0 con 0x7fb99c0cc4f0 2024-04-25T10:11:15.242 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb97806b4d0 msgr2=0x7fb97806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb97806b4d0 0x7fb97806d990 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fb99c153410 tx=0x7fb984008450 comp rx=0 tx=0).stop 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 msgr2=0x7fb99c143500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c143500 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb98c000c00 tx=0x7fb98c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 shutdown_connections 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb97806b4d0 0x7fb97806d990 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb99c15b270 0x7fb99c145290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb99c0cdb20 0x7fb99c1466a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.238+0000 7fb9a1600640 1 --2- 172.21.15.165:0/3074146026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb99c0cc4f0 0x7fb99c143500 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.239+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 >> 172.21.15.165:0/3074146026 conn(0x7fb99c06abe0 msgr2=0x7fb99c159720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.239+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 shutdown_connections 2024-04-25T10:11:15.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.239+0000 7fb9a1600640 1 -- 172.21.15.165:0/3074146026 wait complete. 2024-04-25T10:11:15.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:11:15.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.461+0000 7f5a0dc00640 1 Processor -- start 2024-04-25T10:11:15.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.461+0000 7f5a0dc00640 1 -- start start 2024-04-25T10:11:15.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f5a0dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a08144440 0x7f5a08070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f5a0dc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5a08071530 0x7f5a080cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f5a0dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a080cd860 0x7f5a0806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f5a0dc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080719a0 con 0x7f5a08071530 2024-04-25T10:11:15.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f5a0dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a0815c860 con 0x7f5a08144440 2024-04-25T10:11:15.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f5a0dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0815bf30 con 0x7f5a080cd860 2024-04-25T10:11:15.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f59ff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5a08071530 0x7f5a080cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f59ff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5a08071530 0x7f5a080cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45152/0 (socket says 172.21.15.165:45152) 2024-04-25T10:11:15.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.462+0000 7f59ff400640 1 -- 172.21.15.165:0/796421552 learned_addr learned my addr 172.21.15.165:0/796421552 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f5a0d200640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a080cd860 0x7f5a0806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f5a07e00640 1 --2- 172.21.15.165:0/796421552 >> v2:172.21.15.172:3300/0 conn(0x7f5a08144440 0x7f5a08070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f59ff400640 1 -- 172.21.15.165:0/796421552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a080cd860 msgr2=0x7f5a0806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f59ff400640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a080cd860 0x7f5a0806d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f59ff400640 1 -- 172.21.15.165:0/796421552 >> v2:172.21.15.172:3300/0 conn(0x7f5a08144440 msgr2=0x7f5a08070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f59ff400640 1 --2- 172.21.15.165:0/796421552 >> v2:172.21.15.172:3300/0 conn(0x7f5a08144440 0x7f5a08070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f59ff400640 1 -- 172.21.15.165:0/796421552 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a0815c210 con 0x7f5a08071530 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f5a0d200640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a080cd860 0x7f5a0806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.463+0000 7f5a07e00640 1 --2- 172.21.15.165:0/796421552 >> v2:172.21.15.172:3300/0 conn(0x7f5a08144440 0x7f5a08070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:15.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.464+0000 7f59ff400640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a08071530 0x7f5a080cd320 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5a0806f470 tx=0x7f59f002a260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfc905364dc9372b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.464+0000 7f5a07400640 1 -- 172.21.15.165:0/796421552 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0037070 con 0x7f5a08071530 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.464+0000 7f5a07400640 1 -- 172.21.15.165:0/796421552 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0032440 con 0x7f5a08071530 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.464+0000 7f5a07400640 1 -- 172.21.15.165:0/796421552 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0032810 con 0x7f5a08071530 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 -- 172.21.15.165:0/796421552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a08071530 msgr2=0x7f5a080cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a08071530 0x7f5a080cd320 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f5a0806f470 tx=0x7f59f002a260 comp rx=0 tx=0).stop 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 -- 172.21.15.165:0/796421552 shutdown_connections 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a080cd860 0x7f5a0806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/796421552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a08071530 0x7f5a080cd320 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/796421552 >> v2:172.21.15.172:3300/0 conn(0x7f5a08144440 0x7f5a08070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 -- 172.21.15.165:0/796421552 >> 172.21.15.165:0/796421552 conn(0x7f5a0806abe0 msgr2=0x7f5a0806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 -- 172.21.15.165:0/796421552 shutdown_connections 2024-04-25T10:11:15.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.465+0000 7f5a0dc00640 1 -- 172.21.15.165:0/796421552 wait complete. 2024-04-25T10:11:15.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.467+0000 7f5a0dc00640 1 Processor -- start 2024-04-25T10:11:15.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.467+0000 7f5a0dc00640 1 -- start start 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f5a0dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a08071530 0x7f5a081f2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f5a0dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 0x7f5a081e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f5a0dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a08144440 0x7f5a081e4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f5a0dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080cdcd0 con 0x7f5a080cd860 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f5a0dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a0815bc30 con 0x7f5a08144440 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f5a0dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080cde50 con 0x7f5a08071530 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.468+0000 7f59ff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 0x7f5a081e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 0x7f5a081e2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45158/0 (socket says 172.21.15.165:45158) 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 -- 172.21.15.165:0/3916052193 learned_addr learned my addr 172.21.15.165:0/3916052193 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f5a07e00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a08071530 0x7f5a081f2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f5a0d200640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a08144440 0x7f5a081e4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 -- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a08071530 msgr2=0x7f5a081f2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a08071530 0x7f5a081f2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 -- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a08144440 msgr2=0x7f5a081e4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a08144440 0x7f5a081e4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 -- 172.21.15.165:0/3916052193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a081e34c0 con 0x7f5a080cd860 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.469+0000 7f59ff400640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 0x7f5a081e2a30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f59f002a2a0 tx=0x7f59f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.470+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0037040 con 0x7f5a080cd860 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.470+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f0042070 con 0x7f5a080cd860 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.470+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f0030bb0 con 0x7f5a080cd860 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.470+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a081e37a0 con 0x7f5a080cd860 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.470+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a081e3d60 con 0x7f5a080cd860 2024-04-25T10:11:15.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.471+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a080ce4b0 con 0x7f5a080cd860 2024-04-25T10:11:15.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.472+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59f0047460 con 0x7f5a080cd860 2024-04-25T10:11:15.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.473+0000 7f5a04c00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59e406b4d0 0x7f59e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:15.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.474+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f59f00aad50 con 0x7f5a080cd860 2024-04-25T10:11:15.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.474+0000 7f5a07e00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59e406b4d0 0x7f59e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:15.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.474+0000 7f5a07e00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59e406b4d0 0x7f59e406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f59f800a950 tx=0x7f59f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:15.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.478+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a080ce4b0 con 0x7f5a080cd860 2024-04-25T10:11:15.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:15.745+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7f5a08073990 con 0x7f5a080cd860 2024-04-25T10:11:16.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.373+0000 7f5a04c00640 1 -- 172.21.15.165:0/3916052193 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v425) v1 ==== 147+0+0 (secure 0 0 0) 0x7f59f0079d30 con 0x7f5a080cd860 2024-04-25T10:11:16.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59e406b4d0 msgr2=0x7f59e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59e406b4d0 0x7f59e406d990 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f59f800a950 tx=0x7f59f8008040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 msgr2=0x7f5a081e2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 0x7f5a081e2a30 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f59f002a2a0 tx=0x7f59f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 shutdown_connections 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59e406b4d0 0x7f59e406d990 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a08144440 0x7f5a081e4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a080cd860 0x7f5a081e2a30 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 --2- 172.21.15.165:0/3916052193 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a08071530 0x7f5a081f2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 >> 172.21.15.165:0/3916052193 conn(0x7f5a0806abe0 msgr2=0x7f5a080cce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 shutdown_connections 2024-04-25T10:11:16.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.379+0000 7f5a0dc00640 1 -- 172.21.15.165:0/3916052193 wait complete. 2024-04-25T10:11:16.407 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:11:16.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.611+0000 7f2547200640 1 Processor -- start 2024-04-25T10:11:16.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.611+0000 7f2547200640 1 -- start start 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.613+0000 7f2547200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f254806bff0 0x7f25480b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.613+0000 7f2547200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254815e110 con 0x7f25480b5970 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.613+0000 7f2547200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25480b2fd0 con 0x7f254815dd30 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.613+0000 7f2547200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254806d000 con 0x7f254806bff0 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.613+0000 7f2546800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f25480b5970 0x7f25480d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40718/0 (socket says 172.21.15.165:40718) 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.613+0000 7f2546800640 1 -- 172.21.15.165:0/2995523723 learned_addr learned my addr 172.21.15.165:0/2995523723 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:16.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f253fe00640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f254806bff0 0x7f25480b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643006879 0 0) 0x7f254815e110 con 0x7f25480b5970 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f252c003520 con 0x7f25480b5970 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1314473376 0 0) 0x7f252c003520 con 0x7f25480b5970 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f254806bff0 msgr2=0x7f25480b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f254806bff0 0x7f25480b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 >> v1:172.21.15.172:6789/0 conn(0x7f254815dd30 legacy=0x7f25480cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254806c480 con 0x7f25480b5970 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f25400019f0 con 0x7f25480b5970 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 >> v1:172.21.15.165:6789/0 conn(0x7f25480b5970 legacy=0x7f25480d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c0039e0 0x7f252c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f252c004340 0x7f252c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.614+0000 7f2545400640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252c005020 0x7f252c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254806d000 con 0x7f252c005020 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252c00bf00 con 0x7f252c004340 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f252c00c0b0 con 0x7f252c0039e0 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2546800640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252c005020 0x7f252c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f253fe00640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f252c004340 0x7f252c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2545e00640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c0039e0 0x7f252c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2546800640 1 -- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c0039e0 msgr2=0x7f252c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2546800640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c0039e0 0x7f252c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2546800640 1 -- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f252c004340 msgr2=0x7f252c004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:16.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2546800640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f252c004340 0x7f252c004820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.615+0000 7f2546800640 1 -- 172.21.15.165:0/2995523723 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c003520 con 0x7f252c005020 2024-04-25T10:11:16.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.616+0000 7f2546800640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252c005020 0x7f252c00b9c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f254000a850 tx=0x7f252c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee748ffc93639b38 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000a4a0 con 0x7f252c005020 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f254002f070 con 0x7f252c005020 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2545400640 1 -- 172.21.15.165:0/2995523723 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254002bd20 con 0x7f252c005020 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2547200640 1 -- 172.21.15.165:0/2995523723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252c005020 msgr2=0x7f252c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2547200640 1 --2- 172.21.15.165:0/2995523723 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252c005020 0x7f252c00b9c0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f254000a850 tx=0x7f252c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2546800640 1 -- 172.21.15.165:0/2995523723 reap_dead start 2024-04-25T10:11:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2547200640 1 -- 172.21.15.165:0/2995523723 shutdown_connections 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2547200640 1 -- 172.21.15.165:0/2995523723 >> 172.21.15.165:0/2995523723 conn(0x7f25480d0610 msgr2=0x7f25480d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2547200640 1 -- 172.21.15.165:0/2995523723 shutdown_connections 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.617+0000 7f2547200640 1 -- 172.21.15.165:0/2995523723 wait complete. 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.619+0000 7f2547200640 1 Processor -- start 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.619+0000 7f2547200640 1 -- start start 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f2547200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c005020 0x7f254815f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f2547200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 0x7f25480b1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f2547200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25480b21b0 0x7f25480ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f2547200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254806cd20 con 0x7f25481f3540 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f2547200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254806ca40 con 0x7f25480b21b0 2024-04-25T10:11:16.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f2547200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25480b5430 con 0x7f252c005020 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.620+0000 7f253fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 0x7f25480b1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 0x7f25480b1c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45166/0 (socket says 172.21.15.165:45166) 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 -- 172.21.15.165:0/1103414613 learned_addr learned my addr 172.21.15.165:0/1103414613 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f2545e00640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c005020 0x7f254815f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f2546800640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25480b21b0 0x7f25480ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 -- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c005020 msgr2=0x7f254815f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c005020 0x7f254815f290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 -- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25480b21b0 msgr2=0x7f25480ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25480b21b0 0x7f25480ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 -- 172.21.15.165:0/1103414613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254815fc00 con 0x7f25481f3540 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.621+0000 7f253fe00640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 0x7f25480b1c70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2530011560 tx=0x7f2530011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.622+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530004220 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.622+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25300043c0 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.622+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530015870 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.622+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254815fee0 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.622+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25481604a0 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.623+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2548073990 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.624+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2530015a10 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.625+0000 7f253e000640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252406b4d0 0x7f252406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.625+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f253008a6a0 con 0x7f25481f3540 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.626+0000 7f2545e00640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252406b4d0 0x7f252406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:16.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.626+0000 7f2545e00640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252406b4d0 0x7f252406d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f25480b7660 tx=0x7f2538009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:16.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.630+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2530019030 con 0x7f25481f3540 2024-04-25T10:11:16.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.829+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f25480b7c90 con 0x7f25481f3540 2024-04-25T10:11:16.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.837+0000 7f253e000640 1 -- 172.21.15.165:0/1103414613 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2530059680 con 0x7f25481f3540 2024-04-25T10:11:16.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.839+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252406b4d0 msgr2=0x7f252406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.839+0000 7f2547200640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252406b4d0 0x7f252406d990 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f25480b7660 tx=0x7f2538009040 comp rx=0 tx=0).stop 2024-04-25T10:11:16.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 msgr2=0x7f25480b1c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:16.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 0x7f25480b1c70 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f2530011560 tx=0x7f2530011a20 comp rx=0 tx=0).stop 2024-04-25T10:11:16.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 shutdown_connections 2024-04-25T10:11:16.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f252406b4d0 0x7f252406d990 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25480b21b0 0x7f25480ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25481f3540 0x7f25480b1c70 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 --2- 172.21.15.165:0/1103414613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252c005020 0x7f254815f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 >> 172.21.15.165:0/1103414613 conn(0x7f25480d0610 msgr2=0x7f25480b4a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 shutdown_connections 2024-04-25T10:11:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:16.840+0000 7f2547200640 1 -- 172.21.15.165:0/1103414613 wait complete. 2024-04-25T10:11:16.865 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:11:16.865 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:16.960 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:11:17.071 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:11:17.071 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:11:17.071 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:17.130 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:17.131 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:11:17.131 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:17.187 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:17.188 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:11:17.188 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:17.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:11:17.241 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:11:17.242 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:17.300 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:11:17.300 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:11:17.300 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:17.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.501+0000 7f722c000640 1 Processor -- start 2024-04-25T10:11:17.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.501+0000 7f722c000640 1 -- start start 2024-04-25T10:11:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.503+0000 7f722c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f72240ce590 0x7f7224165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.503+0000 7f722c000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7224071480 con 0x7f7224154ee0 2024-04-25T10:11:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.503+0000 7f722c000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7224071760 con 0x7f72240cfec0 2024-04-25T10:11:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.503+0000 7f722c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7224072090 con 0x7f72240ce590 2024-04-25T10:11:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.503+0000 7f7229600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7224154ee0 0x7f72240cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35302/0 (socket says 172.21.15.165:35302) 2024-04-25T10:11:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.503+0000 7f7229600640 1 -- 172.21.15.165:0/1254226434 learned_addr learned my addr 172.21.15.165:0/1254226434 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f722a000640 1 --2- 172.21.15.165:0/1254226434 >> v2:172.21.15.172:3300/0 conn(0x7f72240ce590 0x7f7224165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 233513673 0 0) 0x7f7224071480 con 0x7f7224154ee0 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f720c003520 con 0x7f7224154ee0 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2270914234 0 0) 0x7f720c003520 con 0x7f7224154ee0 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 >> v2:172.21.15.172:3300/0 conn(0x7f72240ce590 msgr2=0x7f7224165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 --2- 172.21.15.165:0/1254226434 >> v2:172.21.15.172:3300/0 conn(0x7f72240ce590 0x7f7224165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 >> v1:172.21.15.172:6789/0 conn(0x7f72240cfec0 legacy=0x7f72240cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7224071a40 con 0x7f7224154ee0 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7218001670 con 0x7f7224154ee0 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 >> v1:172.21.15.165:6789/0 conn(0x7f7224154ee0 legacy=0x7f72240cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f720c003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f720c004800 0x7f720c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f720c0051c0 0x7f720c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c00c090 con 0x7f720c003a00 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f720c00c240 con 0x7f720c004800 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f720c00c440 con 0x7f720c0051c0 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.504+0000 7f7229600640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f720c003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f7228c00640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f720c004800 0x7f720c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f7229600640 1 -- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f720c0051c0 msgr2=0x7f720c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f7229600640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f720c0051c0 0x7f720c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f7229600640 1 -- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f720c004800 msgr2=0x7f720c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f7229600640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f720c004800 0x7f720c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f7229600640 1 -- 172.21.15.165:0/1254226434 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c003520 con 0x7f720c003a00 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.505+0000 7f722a000640 1 -- 172.21.15.165:0/1254226434 reap_dead start 2024-04-25T10:11:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.506+0000 7f7229600640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f720c003e20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f721800c110 tx=0x7f720c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd5b31232684d24f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.506+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218007c40 con 0x7f720c003a00 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.506+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7218005af0 con 0x7f720c003a00 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.506+0000 7f7223e00640 1 -- 172.21.15.165:0/1254226434 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218007070 con 0x7f720c003a00 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 -- 172.21.15.165:0/1254226434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 msgr2=0x7f720c003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f720c003e20 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f721800c110 tx=0x7f720c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 -- 172.21.15.165:0/1254226434 shutdown_connections 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 --2- 172.21.15.165:0/1254226434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f720c003e20 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 -- 172.21.15.165:0/1254226434 >> 172.21.15.165:0/1254226434 conn(0x7f722406abe0 msgr2=0x7f722406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 -- 172.21.15.165:0/1254226434 shutdown_connections 2024-04-25T10:11:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.507+0000 7f722c000640 1 -- 172.21.15.165:0/1254226434 wait complete. 2024-04-25T10:11:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.509+0000 7f722c000640 1 Processor -- start 2024-04-25T10:11:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.509+0000 7f722c000640 1 -- start start 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722c000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f7224144f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72241e31f0 0x7f7224159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7224143780 0x7f7224143c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722c000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72241552c0 con 0x7f720c003a00 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224071230 con 0x7f7224143780 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7224155440 con 0x7f72241e31f0 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f7229600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f7224144f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f7229600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f7224144f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55414/0 (socket says 172.21.15.165:55414) 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f7229600640 1 -- 172.21.15.165:0/551660489 learned_addr learned my addr 172.21.15.165:0/551660489 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.510+0000 7f722a000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7224143780 0x7f7224143c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7228c00640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72241e31f0 0x7f7224159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7229600640 1 -- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72241e31f0 msgr2=0x7f7224159c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7229600640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72241e31f0 0x7f7224159c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7229600640 1 -- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7224143780 msgr2=0x7f7224143c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7229600640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7224143780 0x7f7224143c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7229600640 1 -- 172.21.15.165:0/551660489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7224145640 con 0x7f720c003a00 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7228c00640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72241e31f0 0x7f7224159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7229600640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f7224144f10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f721802ad20 tx=0x7f7224071650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218041070 con 0x7f720c003a00 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72180047e0 con 0x7f720c003a00 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7218037040 con 0x7f720c003a00 2024-04-25T10:11:17.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7224145920 con 0x7f720c003a00 2024-04-25T10:11:17.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.511+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7224145ee0 con 0x7f720c003a00 2024-04-25T10:11:17.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.513+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7224073990 con 0x7f720c003a00 2024-04-25T10:11:17.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.514+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7218044070 con 0x7f720c003a00 2024-04-25T10:11:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.514+0000 7f7221600640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f720406b4d0 0x7f720406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.515+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f72180a93b0 con 0x7f720c003a00 2024-04-25T10:11:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.515+0000 7f7228c00640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f720406b4d0 0x7f720406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.515+0000 7f7228c00640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f720406b4d0 0x7f720406d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7224153570 tx=0x7f7210009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:17.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.519+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7218077d40 con 0x7f720c003a00 2024-04-25T10:11:17.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.753+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f722406d360 con 0x7f720c003a00 2024-04-25T10:11:17.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.754+0000 7f7221600640 1 -- 172.21.15.165:0/551660489 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 59 v59) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f721807bde0 con 0x7f720c003a00 2024-04-25T10:11:17.759 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:17.759 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":59,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5530,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1453460689","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1453460689},{"type":"v1","addr":"172.21.15.172:6835","nonce":1453460689}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51},{"gid":5698,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/4239673649","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":4239673649},{"type":"v1","addr":"172.21.15.172:6833","nonce":4239673649}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":59},{"gid":7101,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3086635383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3086635383},{"type":"v1","addr":"172.21.15.165:6835","nonce":3086635383}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":50},{"gid":7104,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2592112033","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2592112033},{"type":"v1","addr":"172.21.15.165:6837","nonce":2592112033}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":51}],"filesystems":[]} 2024-04-25T10:11:17.760 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 59 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.756+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f720406b4d0 msgr2=0x7f720406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.756+0000 7f722c000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f720406b4d0 0x7f720406d990 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f7224153570 tx=0x7f7210009040 comp rx=0 tx=0).stop 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 msgr2=0x7f7224144f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f7224144f10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f721802ad20 tx=0x7f7224071650 comp rx=0 tx=0).stop 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 shutdown_connections 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f720406b4d0 0x7f720406d990 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7224143780 0x7f7224143c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72241e31f0 0x7f7224159c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 --2- 172.21.15.165:0/551660489 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f720c003a00 0x7f7224144f10 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:17.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 >> 172.21.15.165:0/551660489 conn(0x7f722406abe0 msgr2=0x7f72240b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:17.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 shutdown_connections 2024-04-25T10:11:17.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:17.757+0000 7f722c000640 1 -- 172.21.15.165:0/551660489 wait complete. 2024-04-25T10:11:17.781 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:11:17.781 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:11:17.781 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:11:17.781 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:17.782 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:11:17.782 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:11:17.782 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:11:17.782 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:17.782 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:11:17.782 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:11:17.782 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:11:17.782 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:17.783 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:11:17.783 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:11:17.783 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:11:17.783 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:11:17.799 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:11:17.799 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:11:17.800 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:11:17.800 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:11:17.802 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:11:17.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:11:17.813 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:11:17.813 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:11:17.848 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:11:17.849 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:17.855 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:11:17.855 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:17.861 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:11:17.861 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:17.867 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:11:17.867 DEBUG:teuthology.parallel:result is None 2024-04-25T10:11:17.868 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:11:17.992 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:11:17.998 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:11:18.013 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:11:18.022 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:11:18.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.040+0000 7f4bc2e00640 1 Processor -- start 2024-04-25T10:11:18.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.040+0000 7f4bc2e00640 1 -- start start 2024-04-25T10:11:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bc2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc1446d0 0x7f4bbc0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bc2e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bbc0cccc0 con 0x7f4bbc144d50 2024-04-25T10:11:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bc2e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bbc0d0460 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bc2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc0b2b20 con 0x7f4bbc1446d0 2024-04-25T10:11:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bb3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4bbc144d50 0x7f4bbc0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35316/0 (socket says 172.21.15.165:35316) 2024-04-25T10:11:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bb3400640 1 -- 172.21.15.165:0/1510515872 learned_addr learned my addr 172.21.15.165:0/1510515872 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:18.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.042+0000 7f4bbbe00640 1 --2- 172.21.15.165:0/1510515872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc1446d0 0x7f4bbc0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.043+0000 7f4bbb400640 1 -- 172.21.15.165:0/1510515872 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 429628632 0 0) 0x7f4bbc0cccc0 con 0x7f4bbc144d50 2024-04-25T10:11:18.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.043+0000 7f4bbb400640 1 -- 172.21.15.165:0/1510515872 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ba0003520 con 0x7f4bbc144d50 2024-04-25T10:11:18.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.043+0000 7f4bbbe00640 1 -- 172.21.15.165:0/1510515872 >> v1:172.21.15.172:6789/0 conn(0x7f4bbc06e8a0 legacy=0x7f4bbc0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.043+0000 7f4bbbe00640 1 -- 172.21.15.165:0/1510515872 >> v1:172.21.15.165:6789/0 conn(0x7f4bbc144d50 legacy=0x7f4bbc0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.043+0000 7f4bbbe00640 1 -- 172.21.15.165:0/1510515872 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc0cc140 con 0x7f4bbc1446d0 2024-04-25T10:11:18.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.044+0000 7f4bbbe00640 1 --2- 172.21.15.165:0/1510515872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc1446d0 0x7f4bbc0b3f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009dc0 tx=0x7f4bac02a6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2ce1a3b8703c27b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.044+0000 7f4bbb400640 1 -- 172.21.15.165:0/1510515872 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007740 con 0x7f4bbc1446d0 2024-04-25T10:11:18.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.044+0000 7f4bbb400640 1 -- 172.21.15.165:0/1510515872 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac0078e0 con 0x7f4bbc1446d0 2024-04-25T10:11:18.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.044+0000 7f4bbb400640 1 -- 172.21.15.165:0/1510515872 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac007cb0 con 0x7f4bbc1446d0 2024-04-25T10:11:18.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 -- 172.21.15.165:0/1510515872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc1446d0 msgr2=0x7f4bbc0b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/1510515872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc1446d0 0x7f4bbc0b3f20 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bac009dc0 tx=0x7f4bac02a6d0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 -- 172.21.15.165:0/1510515872 shutdown_connections 2024-04-25T10:11:18.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/1510515872 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc1446d0 0x7f4bbc0b3f20 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 -- 172.21.15.165:0/1510515872 >> 172.21.15.165:0/1510515872 conn(0x7f4bbc06abe0 msgr2=0x7f4bbc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 -- 172.21.15.165:0/1510515872 shutdown_connections 2024-04-25T10:11:18.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.045+0000 7f4bc2e00640 1 -- 172.21.15.165:0/1510515872 wait complete. 2024-04-25T10:11:18.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.047+0000 7f4bc2e00640 1 Processor -- start 2024-04-25T10:11:18.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.047+0000 7f4bc2e00640 1 -- start start 2024-04-25T10:11:18.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc2e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 0x7f4bbc13f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc2e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bbc1446d0 0x7f4bbc1e0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc2e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc144d50 0x7f4bbc1eee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc2e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc0cc700 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc2e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc06d400 con 0x7f4bbc1446d0 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc2e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bbc0cc9e0 con 0x7f4bbc144d50 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 0x7f4bbc13f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 0x7f4bbc13f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55458/0 (socket says 172.21.15.165:55458) 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bbbe00640 1 -- 172.21.15.165:0/3408525448 learned_addr learned my addr 172.21.15.165:0/3408525448 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bc0e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc144d50 0x7f4bbc1eee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.048+0000 7f4bb3400640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bbc1446d0 0x7f4bbc1e0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bbbe00640 1 -- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc144d50 msgr2=0x7f4bbc1eee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bbbe00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc144d50 0x7f4bbc1eee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bbbe00640 1 -- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bbc1446d0 msgr2=0x7f4bbc1e0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bbbe00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bbc1446d0 0x7f4bbc1e0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bbbe00640 1 -- 172.21.15.165:0/3408525448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bbc1e1700 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bb3400640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bbc1446d0 0x7f4bbc1e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:18.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bbbe00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 0x7f4bbc13f2f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4bac000f80 tx=0x7f4bac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac040070 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bac031c40 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bbc1e19e0 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bac009450 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.049+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bbc1e1fa0 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.050+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bbc13c8b0 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.051+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bac039040 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.052+0000 7f4bb8c00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b9806b4d0 0x7f4b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.052+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4bac0ab8d0 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.053+0000 7f4bb3400640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b9806b4d0 0x7f4b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.053+0000 7f4bb3400640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b9806b4d0 0x7f4b9806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc1411b0 tx=0x7f4ba4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.056+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bbc13c8b0 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.264+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4bbc143e50 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.404+0000 7f4bb8c00640 1 -- 172.21.15.165:0/3408525448 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4bac07a8b0 con 0x7f4bbc06e8a0 2024-04-25T10:11:18.410 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.407+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b9806b4d0 msgr2=0x7f4b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.407+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b9806b4d0 0x7f4b9806d990 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f4bbc1411b0 tx=0x7f4ba4009040 comp rx=0 tx=0).stop 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.407+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 msgr2=0x7f4bbc13f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.407+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 0x7f4bbc13f2f0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f4bac000f80 tx=0x7f4bac004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 shutdown_connections 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b9806b4d0 0x7f4b9806d990 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bbc144d50 0x7f4bbc1eee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bbc1446d0 0x7f4bbc1e0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 --2- 172.21.15.165:0/3408525448 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bbc06e8a0 0x7f4bbc13f2f0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 >> 172.21.15.165:0/3408525448 conn(0x7f4bbc06abe0 msgr2=0x7f4bbc0af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 shutdown_connections 2024-04-25T10:11:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.408+0000 7f4bc2e00640 1 -- 172.21.15.165:0/3408525448 wait complete. 2024-04-25T10:11:18.432 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:11:18.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.619+0000 7fa525200640 1 Processor -- start 2024-04-25T10:11:18.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.619+0000 7fa525200640 1 -- start start 2024-04-25T10:11:18.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa525200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa520144ae0 0x7fa5200ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa525200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52006d270 0x7fa52006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa525200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa52015c520 con 0x7fa52015bcf0 2024-04-25T10:11:18.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa525200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa52015c800 con 0x7fa520144ae0 2024-04-25T10:11:18.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa525200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52015cae0 con 0x7fa52006d270 2024-04-25T10:11:18.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa51f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa520144ae0 0x7fa5200ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.621+0000 7fa51ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa52015bcf0 0x7fa52015f9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35324/0 (socket says 172.21.15.165:35324) 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51ea00640 1 -- 172.21.15.165:0/4067643140 learned_addr learned my addr 172.21.15.165:0/4067643140 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51fe00640 1 --2- 172.21.15.165:0/4067643140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52006d270 0x7fa52006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51f400640 1 -- 172.21.15.165:0/4067643140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52006d270 msgr2=0x7fa52006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51f400640 1 --2- 172.21.15.165:0/4067643140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52006d270 0x7fa52006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51f400640 1 -- 172.21.15.165:0/4067643140 >> v1:172.21.15.165:6789/0 conn(0x7fa52015bcf0 legacy=0x7fa52015f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51f400640 1 -- 172.21.15.165:0/4067643140 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa52015c180 con 0x7fa520144ae0 2024-04-25T10:11:18.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.622+0000 7fa51e000640 1 -- 172.21.15.165:0/4067643140 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2064452339 0 0) 0x7fa52015c520 con 0x7fa52015bcf0 2024-04-25T10:11:18.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.623+0000 7fa51f400640 1 --2- 172.21.15.165:0/4067643140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa520144ae0 0x7fa5200ccb00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097a0 tx=0x7fa51002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0b38f5a7645821a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.623+0000 7fa51e000640 1 -- 172.21.15.165:0/4067643140 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510037070 con 0x7fa520144ae0 2024-04-25T10:11:18.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.623+0000 7fa51e000640 1 -- 172.21.15.165:0/4067643140 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510032440 con 0x7fa520144ae0 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.623+0000 7fa51e000640 1 -- 172.21.15.165:0/4067643140 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510032810 con 0x7fa520144ae0 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 -- 172.21.15.165:0/4067643140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa520144ae0 msgr2=0x7fa5200ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 --2- 172.21.15.165:0/4067643140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa520144ae0 0x7fa5200ccb00 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fa5100097a0 tx=0x7fa51002a180 comp rx=0 tx=0).stop 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 -- 172.21.15.165:0/4067643140 shutdown_connections 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 --2- 172.21.15.165:0/4067643140 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52006d270 0x7fa52006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 --2- 172.21.15.165:0/4067643140 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa520144ae0 0x7fa5200ccb00 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 -- 172.21.15.165:0/4067643140 >> 172.21.15.165:0/4067643140 conn(0x7fa520061c30 msgr2=0x7fa52006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 -- 172.21.15.165:0/4067643140 shutdown_connections 2024-04-25T10:11:18.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.624+0000 7fa525200640 1 -- 172.21.15.165:0/4067643140 wait complete. 2024-04-25T10:11:18.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.626+0000 7fa525200640 1 Processor -- start 2024-04-25T10:11:18.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.626+0000 7fa525200640 1 -- start start 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa525200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 0x7fa5200adf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa525200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa520144ae0 0x7fa52013f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa525200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52015bcf0 0x7fa520150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa525200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5200712c0 con 0x7fa52006d270 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa525200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa520071140 con 0x7fa520144ae0 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa525200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa520071440 con 0x7fa52015bcf0 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa51f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 0x7fa5200adf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa51f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 0x7fa5200adf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55464/0 (socket says 172.21.15.165:55464) 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.627+0000 7fa51f400640 1 -- 172.21.15.165:0/2044588440 learned_addr learned my addr 172.21.15.165:0/2044588440 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa51f400640 1 -- 172.21.15.165:0/2044588440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52015bcf0 msgr2=0x7fa520150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa51f400640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52015bcf0 0x7fa520150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa51f400640 1 -- 172.21.15.165:0/2044588440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa520144ae0 msgr2=0x7fa52013f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa51f400640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa520144ae0 0x7fa52013f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa51f400640 1 -- 172.21.15.165:0/2044588440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5200ae680 con 0x7fa52006d270 2024-04-25T10:11:18.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa51f400640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 0x7fa5200adf80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa510035040 tx=0x7fa5100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.628+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510037040 con 0x7fa52006d270 2024-04-25T10:11:18.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.629+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa510030770 con 0x7fa52006d270 2024-04-25T10:11:18.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.629+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa510030b80 con 0x7fa52006d270 2024-04-25T10:11:18.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.629+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5200ae960 con 0x7fa52006d270 2024-04-25T10:11:18.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.629+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5200a8c00 con 0x7fa52006d270 2024-04-25T10:11:18.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.630+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa520073990 con 0x7fa52006d270 2024-04-25T10:11:18.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.631+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa510044020 con 0x7fa52006d270 2024-04-25T10:11:18.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.632+0000 7fa517400640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50006b500 0x7fa50006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:18.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.632+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa5100aa570 con 0x7fa52006d270 2024-04-25T10:11:18.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.632+0000 7fa51ea00640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50006b500 0x7fa50006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:18.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.633+0000 7fa51ea00640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50006b500 0x7fa50006d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007fd0 tx=0x7fa50c007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:18.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.636+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa510032e20 con 0x7fa52006d270 2024-04-25T10:11:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.901+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa520143420 con 0x7fa52006d270 2024-04-25T10:11:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.902+0000 7fa517400640 1 -- 172.21.15.165:0/2044588440 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa5100795a0 con 0x7fa52006d270 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.908 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:11:18.909 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:11:18.910 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:11:18.911 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.912 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.913 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.914 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.915 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.916 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.917 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.918 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:18.919 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:11:18.920 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:11:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50006b500 msgr2=0x7fa50006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50006b500 0x7fa50006d9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fa50c007fd0 tx=0x7fa50c007960 comp rx=0 tx=0).stop 2024-04-25T10:11:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 msgr2=0x7fa5200adf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 0x7fa5200adf80 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fa510035040 tx=0x7fa5100046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 shutdown_connections 2024-04-25T10:11:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa50006b500 0x7fa50006d9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa52015bcf0 0x7fa520150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa520144ae0 0x7fa52013f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 --2- 172.21.15.165:0/2044588440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa52006d270 0x7fa5200adf80 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 >> 172.21.15.165:0/2044588440 conn(0x7fa520061c30 msgr2=0x7fa5200afae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 shutdown_connections 2024-04-25T10:11:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:18.905+0000 7fa525200640 1 -- 172.21.15.165:0/2044588440 wait complete. 2024-04-25T10:11:18.929 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.110+0000 7efc65400640 1 Processor -- start 2024-04-25T10:11:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.110+0000 7efc65400640 1 -- start start 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc65400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efc60144440 0x7efc60070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc65400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efc600cda30 0x7efc6015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc65400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc6015bf30 con 0x7efc60071530 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc65400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc600719a0 con 0x7efc600cda30 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc65400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc6015c860 con 0x7efc60144440 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc5f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efc60071530 0x7efc600cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35336/0 (socket says 172.21.15.165:35336) 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc5f400640 1 -- 172.21.15.165:0/1008173788 learned_addr learned my addr 172.21.15.165:0/1008173788 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:19.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc64a00640 1 --2- 172.21.15.165:0/1008173788 >> v2:172.21.15.165:3300/0 conn(0x7efc600cda30 0x7efc6015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc5fe00640 1 --2- 172.21.15.165:0/1008173788 >> v2:172.21.15.172:3300/0 conn(0x7efc60144440 0x7efc60070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.112+0000 7efc5ea00640 1 -- 172.21.15.165:0/1008173788 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4104827385 0 0) 0x7efc6015bf30 con 0x7efc60071530 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc5ea00640 1 -- 172.21.15.165:0/1008173788 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efc44003520 con 0x7efc60071530 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc64a00640 1 -- 172.21.15.165:0/1008173788 >> v2:172.21.15.172:3300/0 conn(0x7efc60144440 msgr2=0x7efc60070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc64a00640 1 --2- 172.21.15.165:0/1008173788 >> v2:172.21.15.172:3300/0 conn(0x7efc60144440 0x7efc60070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc64a00640 1 -- 172.21.15.165:0/1008173788 >> v1:172.21.15.165:6789/0 conn(0x7efc60071530 legacy=0x7efc600cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc5ea00640 1 -- 172.21.15.165:0/1008173788 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042444545 0 0) 0x7efc44003520 con 0x7efc60071530 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc64a00640 1 -- 172.21.15.165:0/1008173788 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6015c210 con 0x7efc600cda30 2024-04-25T10:11:19.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.113+0000 7efc5fe00640 1 --2- 172.21.15.165:0/1008173788 >> v2:172.21.15.172:3300/0 conn(0x7efc60144440 0x7efc60070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.114+0000 7efc64a00640 1 --2- 172.21.15.165:0/1008173788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc600cda30 0x7efc6015ccf0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efc6006f470 tx=0x7efc5802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc74045597c4b57b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.114+0000 7efc5ea00640 1 -- 172.21.15.165:0/1008173788 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58037070 con 0x7efc600cda30 2024-04-25T10:11:19.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.114+0000 7efc5ea00640 1 -- 172.21.15.165:0/1008173788 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc58032440 con 0x7efc600cda30 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.114+0000 7efc5ea00640 1 -- 172.21.15.165:0/1008173788 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc58032810 con 0x7efc600cda30 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 -- 172.21.15.165:0/1008173788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc600cda30 msgr2=0x7efc6015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 --2- 172.21.15.165:0/1008173788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc600cda30 0x7efc6015ccf0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7efc6006f470 tx=0x7efc5802a250 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 -- 172.21.15.165:0/1008173788 shutdown_connections 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 --2- 172.21.15.165:0/1008173788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc600cda30 0x7efc6015ccf0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 --2- 172.21.15.165:0/1008173788 >> v2:172.21.15.172:3300/0 conn(0x7efc60144440 0x7efc60070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 -- 172.21.15.165:0/1008173788 >> 172.21.15.165:0/1008173788 conn(0x7efc6006abe0 msgr2=0x7efc6006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 -- 172.21.15.165:0/1008173788 shutdown_connections 2024-04-25T10:11:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.115+0000 7efc65400640 1 -- 172.21.15.165:0/1008173788 wait complete. 2024-04-25T10:11:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.117+0000 7efc65400640 1 Processor -- start 2024-04-25T10:11:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.117+0000 7efc65400640 1 -- start start 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc65400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 0x7efc600ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc65400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc600cda30 0x7efc600ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc65400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc60144440 0x7efc600cbe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc65400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc6015bc30 con 0x7efc60071530 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc65400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc600cdea0 con 0x7efc60144440 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc65400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc6015d230 con 0x7efc600cda30 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 0x7efc600ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 0x7efc600ac9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55494/0 (socket says 172.21.15.165:55494) 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.118+0000 7efc5fe00640 1 -- 172.21.15.165:0/2718328286 learned_addr learned my addr 172.21.15.165:0/2718328286 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc5fe00640 1 -- 172.21.15.165:0/2718328286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc600cda30 msgr2=0x7efc600ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc5fe00640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc600cda30 0x7efc600ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc5fe00640 1 -- 172.21.15.165:0/2718328286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc60144440 msgr2=0x7efc600cbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc5fe00640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc60144440 0x7efc600cbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc5fe00640 1 -- 172.21.15.165:0/2718328286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc600cae10 con 0x7efc60071530 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc5fe00640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 0x7efc600ac9d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7efc5000d7d0 tx=0x7efc5000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50004030 con 0x7efc60071530 2024-04-25T10:11:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc500041d0 con 0x7efc60071530 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc600cb0b0 con 0x7efc60071530 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc50009640 con 0x7efc60071530 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.119+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc600cb670 con 0x7efc60071530 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.121+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc6015ccf0 con 0x7efc60071530 2024-04-25T10:11:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.121+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efc500097e0 con 0x7efc60071530 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.122+0000 7efc57e00640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efc3c06b4d0 0x7efc3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.122+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5249+0+0 (secure 0 0 0) 0x7efc5008a280 con 0x7efc60071530 2024-04-25T10:11:19.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.123+0000 7efc5f400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efc3c06b4d0 0x7efc3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.123+0000 7efc5f400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efc3c06b4d0 0x7efc3c06d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efc48001560 tx=0x7efc48001310 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.127+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efc6015ccf0 con 0x7efc60071530 2024-04-25T10:11:19.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.309+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efc60073990 con 0x7efc60071530 2024-04-25T10:11:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.311+0000 7efc57e00640 1 -- 172.21.15.165:0/2718328286 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+13201 (secure 0 0 0) 0x7efc500592c0 con 0x7efc60071530 2024-04-25T10:11:19.316 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:19.316 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:18.415901+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efc3c06b4d0 msgr2=0x7efc3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efc3c06b4d0 0x7efc3c06d990 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7efc48001560 tx=0x7efc48001310 comp rx=0 tx=0).stop 2024-04-25T10:11:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 msgr2=0x7efc600ac9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 0x7efc600ac9d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7efc5000d7d0 tx=0x7efc5000dc90 comp rx=0 tx=0).stop 2024-04-25T10:11:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 shutdown_connections 2024-04-25T10:11:19.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efc3c06b4d0 0x7efc3c06d990 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efc60144440 0x7efc600cbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efc600cda30 0x7efc600ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 --2- 172.21.15.165:0/2718328286 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efc60071530 0x7efc600ac9d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 >> 172.21.15.165:0/2718328286 conn(0x7efc6006abe0 msgr2=0x7efc600c58a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 shutdown_connections 2024-04-25T10:11:19.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.314+0000 7efc65400640 1 -- 172.21.15.165:0/2718328286 wait complete. 2024-04-25T10:11:19.338 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:11:19.338 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:11:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.525+0000 7f8d0be00640 1 Processor -- start 2024-04-25T10:11:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.525+0000 7f8d0be00640 1 -- start start 2024-04-25T10:11:19.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 0x7f8d0c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c0b0b80 con 0x7f8d0c15b7a0 2024-04-25T10:11:19.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d0c0cc800 con 0x7f8d0c15b3c0 2024-04-25T10:11:19.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0cd650 con 0x7f8d0c15e720 2024-04-25T10:11:19.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0a000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8d0c15b7a0 0x7f8d0c15e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35350/0 (socket says 172.21.15.165:35350) 2024-04-25T10:11:19.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0a000640 1 -- 172.21.15.165:0/4199752344 learned_addr learned my addr 172.21.15.165:0/4199752344 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:19.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927004982 0 0) 0x7f8cf8001110 con 0x7f8d0c15b7a0 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cf0003520 con 0x7f8d0c15b7a0 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.527+0000 7f8d0b400640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 0x7f8d0c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 822524114 0 0) 0x7f8cf0003520 con 0x7f8d0c15b7a0 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 msgr2=0x7f8d0c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 0x7f8d0c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 >> v1:172.21.15.172:6789/0 conn(0x7f8d0c15b3c0 legacy=0x7f8d0c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0ccae0 con 0x7f8d0c15b7a0 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8cf8001840 con 0x7f8d0c15b7a0 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 >> v1:172.21.15.165:6789/0 conn(0x7f8d0c15b7a0 legacy=0x7f8d0c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf00039e0 0x7f8cf0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0004800 0x7f8cf0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 0x7f8cf00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0cd650 con 0x7f8cf0004800 2024-04-25T10:11:19.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf000bad0 con 0x7f8cf00039e0 2024-04-25T10:11:19.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf000bcd0 con 0x7f8d0c15e720 2024-04-25T10:11:19.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.528+0000 7f8d0a000640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0004800 0x7f8cf0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.529+0000 7f8d0aa00640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf00039e0 0x7f8cf0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.529+0000 7f8d0a000640 1 -- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 msgr2=0x7f8cf00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.529+0000 7f8d0a000640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c15e720 0x7f8cf00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.529+0000 7f8d0a000640 1 -- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf00039e0 msgr2=0x7f8cf0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.529+0000 7f8d0a000640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf00039e0 0x7f8cf0003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.529+0000 7f8d0a000640 1 -- 172.21.15.165:0/4199752344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf0003520 con 0x7f8cf0004800 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d0a000640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0004800 0x7f8cf0004c80 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c15bc00 tx=0x7f8cf0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15fcc48fb3a6d9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8036070 con 0x7f8cf0004800 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf8005ad0 con 0x7f8cf0004800 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d09600640 1 -- 172.21.15.165:0/4199752344 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8007e10 con 0x7f8cf0004800 2024-04-25T10:11:19.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d0be00640 1 -- 172.21.15.165:0/4199752344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0004800 msgr2=0x7f8cf0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d0be00640 1 --2- 172.21.15.165:0/4199752344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8cf0004800 0x7f8cf0004c80 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c15bc00 tx=0x7f8cf0005830 comp rx=0 tx=0).stop 2024-04-25T10:11:19.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.530+0000 7f8d0b400640 1 -- 172.21.15.165:0/4199752344 reap_dead start 2024-04-25T10:11:19.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.531+0000 7f8d0be00640 1 -- 172.21.15.165:0/4199752344 shutdown_connections 2024-04-25T10:11:19.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.531+0000 7f8d0be00640 1 -- 172.21.15.165:0/4199752344 >> 172.21.15.165:0/4199752344 conn(0x7f8d0c061cc0 msgr2=0x7f8d0c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.531+0000 7f8d0be00640 1 -- 172.21.15.165:0/4199752344 shutdown_connections 2024-04-25T10:11:19.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.531+0000 7f8d0be00640 1 -- 172.21.15.165:0/4199752344 wait complete. 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.532+0000 7f8d0be00640 1 Processor -- start 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.532+0000 7f8d0be00640 1 -- start start 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf0004800 0x7f8d0c148c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 0x7f8d0c0a85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c0aafe0 0x7f8d0c0ab460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0ce370 con 0x7f8d0c0c9720 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0ce1f0 con 0x7f8cf0004800 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d0c0b0f60 con 0x7f8d0c0aafe0 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.533+0000 7f8d0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 0x7f8d0c0a85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 0x7f8d0c0a85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55514/0 (socket says 172.21.15.165:55514) 2024-04-25T10:11:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 -- 172.21.15.165:0/2158317133 learned_addr learned my addr 172.21.15.165:0/2158317133 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:19.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 -- 172.21.15.165:0/2158317133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c0aafe0 msgr2=0x7f8d0c0ab460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c0aafe0 0x7f8d0c0ab460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 -- 172.21.15.165:0/2158317133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf0004800 msgr2=0x7f8d0c148c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf0004800 0x7f8d0c148c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 -- 172.21.15.165:0/2158317133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d0c0a8b00 con 0x7f8d0c0c9720 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0a000640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 0x7f8d0c0a85c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c3e0 tx=0x7f8d0c0cccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf8036040 con 0x7f8d0c0c9720 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf8005c80 con 0x7f8d0c0c9720 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf80017a0 con 0x7f8d0c0c9720 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.534+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d0c0c9c70 con 0x7f8d0c0c9720 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.535+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d0c0ca230 con 0x7f8d0c0c9720 2024-04-25T10:11:19.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.536+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d0c073990 con 0x7f8d0c0c9720 2024-04-25T10:11:19.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.537+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8cf8007070 con 0x7f8d0c0c9720 2024-04-25T10:11:19.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.538+0000 7f8d02a00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cec06b500 0x7f8cec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.538+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8cf80aa2f0 con 0x7f8d0c0c9720 2024-04-25T10:11:19.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.538+0000 7f8d0aa00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cec06b500 0x7f8cec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.539+0000 7f8d0aa00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cec06b500 0x7f8cec06d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc001640 tx=0x7f8cfc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.542+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8cf8041020 con 0x7f8d0c0c9720 2024-04-25T10:11:19.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.726+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d0c0a8d30 con 0x7f8d0c0c9720 2024-04-25T10:11:19.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.727+0000 7f8d02a00640 1 -- 172.21.15.165:0/2158317133 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d0c0b0b80 con 0x7f8d0c0c9720 2024-04-25T10:11:19.732 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:11:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cec06b500 msgr2=0x7f8cec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cec06b500 0x7f8cec06d9c0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc001640 tx=0x7f8cfc009040 comp rx=0 tx=0).stop 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 msgr2=0x7f8d0c0a85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 0x7f8d0c0a85c0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f8cf800c3e0 tx=0x7f8d0c0cccb0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 shutdown_connections 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cec06b500 0x7f8cec06d9c0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d0c0aafe0 0x7f8d0c0ab460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d0c0c9720 0x7f8d0c0a85c0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 --2- 172.21.15.165:0/2158317133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8cf0004800 0x7f8d0c148c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 >> 172.21.15.165:0/2158317133 conn(0x7f8d0c061cc0 msgr2=0x7f8d0c0b1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.729+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 shutdown_connections 2024-04-25T10:11:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.730+0000 7f8d0be00640 1 -- 172.21.15.165:0/2158317133 wait complete. 2024-04-25T10:11:19.753 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:11:19.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.929+0000 7f867be00640 1 Processor -- start 2024-04-25T10:11:19.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.930+0000 7f867be00640 1 -- start start 2024-04-25T10:11:19.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f867c144ae0 0x7f867c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f867c15fef0 0x7f867c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c15c520 con 0x7f867c144ae0 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f867c15c800 con 0x7f867c15fef0 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c15cae0 con 0x7f867c15bcf0 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.931+0000 7f867aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f867c144ae0 0x7f867c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f867c144ae0 0x7f867c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55530/0 (socket says 172.21.15.165:55530) 2024-04-25T10:11:19.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 -- 172.21.15.165:0/3836208776 learned_addr learned my addr 172.21.15.165:0/3836208776 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867b400640 1 --2- 172.21.15.165:0/3836208776 >> v2:172.21.15.172:3300/0 conn(0x7f867c15fef0 0x7f867c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f8673e00640 1 --2- 172.21.15.165:0/3836208776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 -- 172.21.15.165:0/3836208776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 msgr2=0x7f867c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 --2- 172.21.15.165:0/3836208776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c15f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 -- 172.21.15.165:0/3836208776 >> v2:172.21.15.172:3300/0 conn(0x7f867c15fef0 msgr2=0x7f867c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 --2- 172.21.15.165:0/3836208776 >> v2:172.21.15.172:3300/0 conn(0x7f867c15fef0 0x7f867c06d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.932+0000 7f867aa00640 1 -- 172.21.15.165:0/3836208776 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c15c150 con 0x7f867c144ae0 2024-04-25T10:11:19.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.933+0000 7f867aa00640 1 --2- 172.21.15.165:0/3836208776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c144ae0 0x7f867c0ccb00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f867c06bb90 tx=0x7f866c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c4d5c19061980 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.933+0000 7f867a000640 1 -- 172.21.15.165:0/3836208776 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c037070 con 0x7f867c144ae0 2024-04-25T10:11:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.933+0000 7f867a000640 1 -- 172.21.15.165:0/3836208776 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866c032440 con 0x7f867c144ae0 2024-04-25T10:11:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.933+0000 7f867a000640 1 -- 172.21.15.165:0/3836208776 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866c032810 con 0x7f867c144ae0 2024-04-25T10:11:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 -- 172.21.15.165:0/3836208776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c144ae0 msgr2=0x7f867c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 --2- 172.21.15.165:0/3836208776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c144ae0 0x7f867c0ccb00 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f867c06bb90 tx=0x7f866c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 -- 172.21.15.165:0/3836208776 shutdown_connections 2024-04-25T10:11:19.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 --2- 172.21.15.165:0/3836208776 >> v2:172.21.15.172:3300/0 conn(0x7f867c15fef0 0x7f867c06d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 --2- 172.21.15.165:0/3836208776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 --2- 172.21.15.165:0/3836208776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c144ae0 0x7f867c0ccb00 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 -- 172.21.15.165:0/3836208776 >> 172.21.15.165:0/3836208776 conn(0x7f867c061c30 msgr2=0x7f867c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:19.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 -- 172.21.15.165:0/3836208776 shutdown_connections 2024-04-25T10:11:19.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.934+0000 7f867be00640 1 -- 172.21.15.165:0/3836208776 wait complete. 2024-04-25T10:11:19.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.936+0000 7f867be00640 1 Processor -- start 2024-04-25T10:11:19.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.936+0000 7f867be00640 1 -- start start 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c144ae0 0x7f867c15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c13e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 0x7f867c0ab080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c06db40 con 0x7f867c15fef0 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f867c06d9c0 con 0x7f867c144ae0 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f867c071140 con 0x7f867c15bcf0 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 0x7f867c0ab080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 0x7f867c0ab080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55534/0 (socket says 172.21.15.165:55534) 2024-04-25T10:11:19.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867b400640 1 -- 172.21.15.165:0/326528056 learned_addr learned my addr 172.21.15.165:0/326528056 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f8673e00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c13e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.937+0000 7f867aa00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c144ae0 0x7f867c15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867b400640 1 -- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 msgr2=0x7f867c13e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867b400640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c13e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867b400640 1 -- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c144ae0 msgr2=0x7f867c15ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867b400640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c144ae0 0x7f867c15ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867b400640 1 -- 172.21.15.165:0/326528056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f867c15b1d0 con 0x7f867c15fef0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867b400640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 0x7f867c0ab080 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8674007e60 tx=0x7f867400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674014420 con 0x7f867c15fef0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674019070 con 0x7f867c15fef0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674014820 con 0x7f867c15fef0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f867c13ec80 con 0x7f867c15fef0 2024-04-25T10:11:19.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.938+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f867c13f240 con 0x7f867c15fef0 2024-04-25T10:11:19.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.939+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f867c073990 con 0x7f867c15fef0 2024-04-25T10:11:19.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.941+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86740149c0 con 0x7f867c15fef0 2024-04-25T10:11:19.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.942+0000 7f8672a00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f865806b4d0 0x7f865806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:19.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.942+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f867408a8b0 con 0x7f867c15fef0 2024-04-25T10:11:19.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.942+0000 7f867aa00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f865806b4d0 0x7f865806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:19.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.943+0000 7f867aa00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f865806b4d0 0x7f865806d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f866c034000 tx=0x7f866c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:19.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:19.946+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f867c15b1d0 con 0x7f867c15fef0 2024-04-25T10:11:20.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.145+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f867c1404e0 con 0x7f867c15fef0 2024-04-25T10:11:20.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.145+0000 7f8672a00640 1 -- 172.21.15.165:0/326528056 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v94) v1 ==== 84+0+65 (secure 0 0 0) 0x7f86740598e0 con 0x7f867c15fef0 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:11:20.151 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f865806b4d0 msgr2=0x7f865806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f865806b4d0 0x7f865806d990 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f866c034000 tx=0x7f866c02e000 comp rx=0 tx=0).stop 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 msgr2=0x7f867c0ab080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 0x7f867c0ab080 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8674007e60 tx=0x7f867400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 shutdown_connections 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f865806b4d0 0x7f865806d990 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f867c15fef0 0x7f867c0ab080 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f867c15bcf0 0x7f867c13e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 --2- 172.21.15.165:0/326528056 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f867c144ae0 0x7f867c15ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 >> 172.21.15.165:0/326528056 conn(0x7f867c061c30 msgr2=0x7f867c0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 shutdown_connections 2024-04-25T10:11:20.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.148+0000 7f867be00640 1 -- 172.21.15.165:0/326528056 wait complete. 2024-04-25T10:11:20.172 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:11:20.172 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:11:20.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.372+0000 7ff0a0800640 1 Processor -- start 2024-04-25T10:11:20.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.372+0000 7ff0a0800640 1 -- start start 2024-04-25T10:11:20.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.374+0000 7ff0a0800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0980cf640 0x7ff09806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.374+0000 7ff0a0800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0980cfad0 con 0x7ff09815b710 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.374+0000 7ff0a0800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0980cfd80 con 0x7ff0980ac8b0 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.374+0000 7ff0a0800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff09806f190 con 0x7ff0980cf640 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.374+0000 7ff09de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff09815b710 0x7ff09815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35354/0 (socket says 172.21.15.165:35354) 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09de00640 1 -- 172.21.15.165:0/20140433 learned_addr learned my addr 172.21.15.165:0/20140433 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09d400640 1 --2- 172.21.15.165:0/20140433 >> v2:172.21.15.165:3300/0 conn(0x7ff0980cf640 0x7ff09806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09ca00640 1 -- 172.21.15.165:0/20140433 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969166438 0 0) 0x7ff0980cfad0 con 0x7ff09815b710 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09ca00640 1 -- 172.21.15.165:0/20140433 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff080003520 con 0x7ff09815b710 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09d400640 1 -- 172.21.15.165:0/20140433 >> v1:172.21.15.172:6789/0 conn(0x7ff0980ac8b0 legacy=0x7ff098071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09d400640 1 -- 172.21.15.165:0/20140433 >> v1:172.21.15.165:6789/0 conn(0x7ff09815b710 legacy=0x7ff09815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.375+0000 7ff09d400640 1 -- 172.21.15.165:0/20140433 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0980d0060 con 0x7ff0980cf640 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.376+0000 7ff09d400640 1 --2- 172.21.15.165:0/20140433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980cf640 0x7ff09806ec50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff0880097a0 tx=0x7ff08802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a6ac993e805af97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff09ca00640 1 -- 172.21.15.165:0/20140433 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff088037070 con 0x7ff0980cf640 2024-04-25T10:11:20.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff09ca00640 1 -- 172.21.15.165:0/20140433 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff088032440 con 0x7ff0980cf640 2024-04-25T10:11:20.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff09ca00640 1 -- 172.21.15.165:0/20140433 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0880327f0 con 0x7ff0980cf640 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff0a0800640 1 -- 172.21.15.165:0/20140433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980cf640 msgr2=0x7ff09806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff0a0800640 1 --2- 172.21.15.165:0/20140433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980cf640 0x7ff09806ec50 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7ff0880097a0 tx=0x7ff08802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff0a0800640 1 -- 172.21.15.165:0/20140433 shutdown_connections 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.377+0000 7ff0a0800640 1 --2- 172.21.15.165:0/20140433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980cf640 0x7ff09806ec50 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.378+0000 7ff0a0800640 1 -- 172.21.15.165:0/20140433 >> 172.21.15.165:0/20140433 conn(0x7ff09806abe0 msgr2=0x7ff09806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.378+0000 7ff0a0800640 1 -- 172.21.15.165:0/20140433 shutdown_connections 2024-04-25T10:11:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.378+0000 7ff0a0800640 1 -- 172.21.15.165:0/20140433 wait complete. 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.379+0000 7ff0a0800640 1 Processor -- start 2024-04-25T10:11:20.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.380+0000 7ff0a0800640 1 -- start start 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.380+0000 7ff0a0800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 0x7ff098159dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff0a0800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980cf640 0x7ff098148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff0a0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09815b710 0x7ff0981f2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff0a0800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09815baf0 con 0x7ff0980ac8b0 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff0a0800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09815e490 con 0x7ff0980cf640 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff0a0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09815bc70 con 0x7ff09815b710 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 0x7ff098159dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 0x7ff098159dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55546/0 (socket says 172.21.15.165:55546) 2024-04-25T10:11:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 -- 172.21.15.165:0/3410388449 learned_addr learned my addr 172.21.15.165:0/3410388449 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09e800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09815b710 0x7ff0981f2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 -- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09815b710 msgr2=0x7ff0981f2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09815b710 0x7ff0981f2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 -- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980cf640 msgr2=0x7ff098148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980cf640 0x7ff098148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.381+0000 7ff09de00640 1 -- 172.21.15.165:0/3410388449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff09815a4d0 con 0x7ff0980ac8b0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.382+0000 7ff09de00640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 0x7ff098159dd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c390 tx=0x7ff08c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.382+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c017070 con 0x7ff0980ac8b0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.382+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff08c005ce0 con 0x7ff0980ac8b0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.382+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff08c0072f0 con 0x7ff0980ac8b0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.382+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09815a7b0 con 0x7ff0980ac8b0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.382+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0981f3270 con 0x7ff0980ac8b0 2024-04-25T10:11:20.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.383+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09806f450 con 0x7ff0980ac8b0 2024-04-25T10:11:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.384+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff08c01a070 con 0x7ff0980ac8b0 2024-04-25T10:11:20.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.386+0000 7ff092000640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07c06b4d0 0x7ff07c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:20.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.386+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff08c089460 con 0x7ff0980ac8b0 2024-04-25T10:11:20.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.386+0000 7ff09d400640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07c06b4d0 0x7ff07c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:20.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.387+0000 7ff09d400640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07c06b4d0 0x7ff07c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff088034000 tx=0x7ff08802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.390+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff09806f450 con 0x7ff0980ac8b0 2024-04-25T10:11:20.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:20.587+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff098073a00 con 0x7ff0980ac8b0 2024-04-25T10:11:21.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.432+0000 7ff092000640 1 -- 172.21.15.165:0/3410388449 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v95) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff08c058490 con 0x7ff0980ac8b0 2024-04-25T10:11:21.437 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07c06b4d0 msgr2=0x7ff07c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07c06b4d0 0x7ff07c06d990 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7ff088034000 tx=0x7ff08802c000 comp rx=0 tx=0).stop 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 msgr2=0x7ff098159dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 0x7ff098159dd0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7ff08c00c390 tx=0x7ff08c00c850 comp rx=0 tx=0).stop 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 shutdown_connections 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07c06b4d0 0x7ff07c06d990 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09815b710 0x7ff0981f2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0980cf640 0x7ff098148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 --2- 172.21.15.165:0/3410388449 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0980ac8b0 0x7ff098159dd0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 >> 172.21.15.165:0/3410388449 conn(0x7ff09806abe0 msgr2=0x7ff098160830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 shutdown_connections 2024-04-25T10:11:21.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.436+0000 7ff0a0800640 1 -- 172.21.15.165:0/3410388449 wait complete. 2024-04-25T10:11:21.467 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:11:21.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.663+0000 7f9455a00640 1 Processor -- start 2024-04-25T10:11:21.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.663+0000 7f9455a00640 1 -- start start 2024-04-25T10:11:21.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.664+0000 7f9455a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f94501446d0 0x7f94500b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.664+0000 7f9455a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9450144d50 0x7f94500b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f9455a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f94501452a0 0x7f94500cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f9455a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94500b2b20 con 0x7f94501452a0 2024-04-25T10:11:21.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f9455a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945006d400 con 0x7f94501446d0 2024-04-25T10:11:21.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f9455a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500b49a0 con 0x7f9450144d50 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f944f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f94501452a0 0x7f94500cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f944f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f94501452a0 0x7f94500cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55562/0 (socket says 172.21.15.165:55562) 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f944f400640 1 -- 172.21.15.165:0/3892583250 learned_addr learned my addr 172.21.15.165:0/3892583250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f944e000640 1 --2- 172.21.15.165:0/3892583250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9450144d50 0x7f94500b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.665+0000 7f944ea00640 1 --2- 172.21.15.165:0/3892583250 >> v2:172.21.15.172:3300/0 conn(0x7f94501446d0 0x7f94500b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.666+0000 7f944f400640 1 -- 172.21.15.165:0/3892583250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9450144d50 msgr2=0x7f94500b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.666+0000 7f944f400640 1 --2- 172.21.15.165:0/3892583250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9450144d50 0x7f94500b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.666+0000 7f944f400640 1 -- 172.21.15.165:0/3892583250 >> v2:172.21.15.172:3300/0 conn(0x7f94501446d0 msgr2=0x7f94500b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.666+0000 7f944f400640 1 --2- 172.21.15.165:0/3892583250 >> v2:172.21.15.172:3300/0 conn(0x7f94501446d0 0x7f94500b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.666+0000 7f944f400640 1 -- 172.21.15.165:0/3892583250 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500cccc0 con 0x7f94501452a0 2024-04-25T10:11:21.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.666+0000 7f944ea00640 1 --2- 172.21.15.165:0/3892583250 >> v2:172.21.15.172:3300/0 conn(0x7f94501446d0 0x7f94500b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:21.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.667+0000 7f944f400640 1 --2- 172.21.15.165:0/3892583250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94501452a0 0x7f94500cf640 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f94500cf560 tx=0x7f944802a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=477463b8b6ad8cb9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.667+0000 7f944d600640 1 -- 172.21.15.165:0/3892583250 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448036070 con 0x7f94501452a0 2024-04-25T10:11:21.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.667+0000 7f944d600640 1 -- 172.21.15.165:0/3892583250 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448031600 con 0x7f94501452a0 2024-04-25T10:11:21.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.667+0000 7f944d600640 1 -- 172.21.15.165:0/3892583250 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480319d0 con 0x7f94501452a0 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 -- 172.21.15.165:0/3892583250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94501452a0 msgr2=0x7f94500cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 --2- 172.21.15.165:0/3892583250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94501452a0 0x7f94500cf640 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f94500cf560 tx=0x7f944802a690 comp rx=0 tx=0).stop 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 -- 172.21.15.165:0/3892583250 shutdown_connections 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 --2- 172.21.15.165:0/3892583250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94501452a0 0x7f94500cf640 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 --2- 172.21.15.165:0/3892583250 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9450144d50 0x7f94500b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 --2- 172.21.15.165:0/3892583250 >> v2:172.21.15.172:3300/0 conn(0x7f94501446d0 0x7f94500b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 -- 172.21.15.165:0/3892583250 >> 172.21.15.165:0/3892583250 conn(0x7f945006abe0 msgr2=0x7f945006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:21.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 -- 172.21.15.165:0/3892583250 shutdown_connections 2024-04-25T10:11:21.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.668+0000 7f9455a00640 1 -- 172.21.15.165:0/3892583250 wait complete. 2024-04-25T10:11:21.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.670+0000 7f9455a00640 1 Processor -- start 2024-04-25T10:11:21.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.670+0000 7f9455a00640 1 -- start start 2024-04-25T10:11:21.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f9455a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94501446d0 0x7f94500c5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f9455a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 0x7f94500c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f9455a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94501452a0 0x7f94501f6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f9455a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500cc700 con 0x7f9450144d50 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f9455a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94500cc420 con 0x7f94501446d0 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f9455a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94500cc9e0 con 0x7f94501452a0 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f944e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 0x7f94500c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f944e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 0x7f94500c7f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55570/0 (socket says 172.21.15.165:55570) 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.671+0000 7f944e000640 1 -- 172.21.15.165:0/2375495743 learned_addr learned my addr 172.21.15.165:0/2375495743 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944f400640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94501452a0 0x7f94501f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944ea00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94501446d0 0x7f94500c5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944e000640 1 -- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94501452a0 msgr2=0x7f94501f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944e000640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94501452a0 0x7f94501f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944e000640 1 -- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94501446d0 msgr2=0x7f94500c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944e000640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94501446d0 0x7f94500c5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944e000640 1 -- 172.21.15.165:0/2375495743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94500b0990 con 0x7f9450144d50 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f944e000640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 0x7f94500c7f50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f943c00d800 tx=0x7f943c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00b6c0 con 0x7f9450144d50 2024-04-25T10:11:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943c019070 con 0x7f9450144d50 2024-04-25T10:11:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f943c00ba80 con 0x7f9450144d50 2024-04-25T10:11:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94500b0c70 con 0x7f9450144d50 2024-04-25T10:11:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.672+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94500b1230 con 0x7f9450144d50 2024-04-25T10:11:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.674+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9450073a00 con 0x7f9450144d50 2024-04-25T10:11:21.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.675+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f943c00bc20 con 0x7f9450144d50 2024-04-25T10:11:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.676+0000 7f9446a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.676+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f943c08a730 con 0x7f9450144d50 2024-04-25T10:11:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.676+0000 7f944ea00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:21.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.677+0000 7f944ea00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943006b4d0 0x7f943006d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9440006fd0 tx=0x7f9440008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.681+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f943c01c360 con 0x7f9450144d50 2024-04-25T10:11:21.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:21.877+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f945006ec90 con 0x7f9450144d50 2024-04-25T10:11:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.438+0000 7f9446a00640 1 -- 172.21.15.165:0/2375495743 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v96) v1 ==== 170+0+0 (secure 0 0 0) 0x7f943c059560 con 0x7f9450144d50 2024-04-25T10:11:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:11:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.441+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943006b4d0 msgr2=0x7f943006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.441+0000 7f9455a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943006b4d0 0x7f943006d990 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f9440006fd0 tx=0x7f9440008040 comp rx=0 tx=0).stop 2024-04-25T10:11:22.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.441+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 msgr2=0x7f94500c7f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 0x7f94500c7f50 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f943c00d800 tx=0x7f943c00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 shutdown_connections 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943006b4d0 0x7f943006d990 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94501452a0 0x7f94501f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9450144d50 0x7f94500c7f50 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 --2- 172.21.15.165:0/2375495743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94501446d0 0x7f94500c5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.442+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 >> 172.21.15.165:0/2375495743 conn(0x7f945006abe0 msgr2=0x7f94501439a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.443+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 shutdown_connections 2024-04-25T10:11:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.443+0000 7f9455a00640 1 -- 172.21.15.165:0/2375495743 wait complete. 2024-04-25T10:11:22.485 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:11:22.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.656+0000 7f8514000640 1 Processor -- start 2024-04-25T10:11:22.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.656+0000 7f8514000640 1 -- start start 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.657+0000 7f8514000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f850c06ea90 0x7f850c071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.657+0000 7f8514000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f850c06d2d0 0x7f850c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8514000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c162ba0 con 0x7f850c161fc0 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8514000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c165be0 con 0x7f850c06ea90 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8514000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c06dc40 con 0x7f850c06d2d0 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f850c06ea90 0x7f850c071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f850c06ea90 0x7f850c071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55578/0 (socket says 172.21.15.165:55578) 2024-04-25T10:11:22.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 -- 172.21.15.165:0/2269323184 learned_addr learned my addr 172.21.15.165:0/2269323184 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 -- 172.21.15.165:0/2269323184 >> v2:172.21.15.172:3300/0 conn(0x7f850c06d2d0 msgr2=0x7f850c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 --2- 172.21.15.165:0/2269323184 >> v2:172.21.15.172:3300/0 conn(0x7f850c06d2d0 0x7f850c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 -- 172.21.15.165:0/2269323184 >> v1:172.21.15.172:6789/0 conn(0x7f850c161fc0 legacy=0x7f850c1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.658+0000 7f8510c00640 1 -- 172.21.15.165:0/2269323184 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c162e80 con 0x7f850c06ea90 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.659+0000 7f8510c00640 1 --2- 172.21.15.165:0/2269323184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c071480 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f850c162ab0 tx=0x7f84f802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b8a7382cf404ef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f850be00640 1 -- 172.21.15.165:0/2269323184 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8037070 con 0x7f850c06ea90 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f850be00640 1 -- 172.21.15.165:0/2269323184 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f8032440 con 0x7f850c06ea90 2024-04-25T10:11:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f850be00640 1 -- 172.21.15.165:0/2269323184 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8032810 con 0x7f850c06ea90 2024-04-25T10:11:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f8514000640 1 -- 172.21.15.165:0/2269323184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 msgr2=0x7f850c071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f8514000640 1 --2- 172.21.15.165:0/2269323184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c071480 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f850c162ab0 tx=0x7f84f802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f8514000640 1 -- 172.21.15.165:0/2269323184 shutdown_connections 2024-04-25T10:11:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f8514000640 1 --2- 172.21.15.165:0/2269323184 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c071480 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f8514000640 1 --2- 172.21.15.165:0/2269323184 >> v2:172.21.15.172:3300/0 conn(0x7f850c06d2d0 0x7f850c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.660+0000 7f8514000640 1 -- 172.21.15.165:0/2269323184 >> 172.21.15.165:0/2269323184 conn(0x7f850c06abe0 msgr2=0x7f850c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.661+0000 7f8514000640 1 -- 172.21.15.165:0/2269323184 shutdown_connections 2024-04-25T10:11:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.661+0000 7f8514000640 1 -- 172.21.15.165:0/2269323184 wait complete. 2024-04-25T10:11:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.662+0000 7f8514000640 1 Processor -- start 2024-04-25T10:11:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.662+0000 7f8514000640 1 -- start start 2024-04-25T10:11:22.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.663+0000 7f8514000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850c06d2d0 0x7f850c157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.663+0000 7f8514000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.663+0000 7f8514000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f850c161fc0 0x7f850c159740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.663+0000 7f8514000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c0719c0 con 0x7f850c06ea90 2024-04-25T10:11:22.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.663+0000 7f8514000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c163440 con 0x7f850c06d2d0 2024-04-25T10:11:22.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.663+0000 7f8514000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c071b40 con 0x7f850c161fc0 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55584/0 (socket says 172.21.15.165:55584) 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 -- 172.21.15.165:0/163909809 learned_addr learned my addr 172.21.15.165:0/163909809 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8511600640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850c06d2d0 0x7f850c157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8512000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f850c161fc0 0x7f850c159740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 -- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f850c161fc0 msgr2=0x7f850c159740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f850c161fc0 0x7f850c159740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 -- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850c06d2d0 msgr2=0x7f850c157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850c06d2d0 0x7f850c157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 -- 172.21.15.165:0/163909809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c14cc40 con 0x7f850c06ea90 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8511600640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850c06d2d0 0x7f850c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:22.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.664+0000 7f8510c00640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c157c50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f84f8000c00 tx=0x7f84f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.665+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8041070 con 0x7f850c06ea90 2024-04-25T10:11:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.665+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f803c870 con 0x7f850c06ea90 2024-04-25T10:11:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.665+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8037040 con 0x7f850c06ea90 2024-04-25T10:11:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.665+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c0b21c0 con 0x7f850c06ea90 2024-04-25T10:11:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.665+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c15a970 con 0x7f850c06ea90 2024-04-25T10:11:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.666+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c073a00 con 0x7f850c06ea90 2024-04-25T10:11:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.667+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f84f8044070 con 0x7f850c06ea90 2024-04-25T10:11:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.668+0000 7f8509600640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84ec06b4d0 0x7f84ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.668+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f84f80aaee0 con 0x7f850c06ea90 2024-04-25T10:11:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.668+0000 7f8511600640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84ec06b4d0 0x7f84ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.669+0000 7f8511600640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84ec06b4d0 0x7f84ec06d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f850000a810 tx=0x7f8500008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:22.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.673+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f84f8079d40 con 0x7f850c06ea90 2024-04-25T10:11:22.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:22.908+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f850c165690 con 0x7f850c06ea90 2024-04-25T10:11:23.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.488+0000 7f8509600640 1 -- 172.21.15.165:0/163909809 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7f84f8079640 con 0x7f850c06ea90 2024-04-25T10:11:23.495 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 13 and data pool 14 2024-04-25T10:11:23.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.497+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84ec06b4d0 msgr2=0x7f84ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.497+0000 7f8514000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84ec06b4d0 0x7f84ec06d990 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f850000a810 tx=0x7f8500008040 comp rx=0 tx=0).stop 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.497+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 msgr2=0x7f850c157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.497+0000 7f8514000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c157c50 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f84f8000c00 tx=0x7f84f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.498+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 shutdown_connections 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.498+0000 7f8514000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84ec06b4d0 0x7f84ec06d990 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.498+0000 7f8514000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f850c161fc0 0x7f850c159740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.498+0000 7f8514000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f850c06ea90 0x7f850c157c50 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.498+0000 7f8514000640 1 --2- 172.21.15.165:0/163909809 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f850c06d2d0 0x7f850c157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.498+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 >> 172.21.15.165:0/163909809 conn(0x7f850c06abe0 msgr2=0x7f850c164f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.504+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 shutdown_connections 2024-04-25T10:11:23.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.504+0000 7f8514000640 1 -- 172.21.15.165:0/163909809 wait complete. 2024-04-25T10:11:23.540 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:23.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.707+0000 7f79e6000640 1 Processor -- start 2024-04-25T10:11:23.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.707+0000 7f79e6000640 1 -- start start 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79e6000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79e00ce920 0x7f79e00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79e6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 0x7f79e006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79e6000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79e0144b30 con 0x7f79e0144750 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79e6000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79e00712d0 con 0x7f79e00ce920 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79e6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00715b0 con 0x7f79e006d290 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79df400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f79e0144750 0x7f79e006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35370/0 (socket says 172.21.15.165:35370) 2024-04-25T10:11:23.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.709+0000 7f79df400640 1 -- 172.21.15.165:0/2539344940 learned_addr learned my addr 172.21.15.165:0/2539344940 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2363751796 0 0) 0x7f79e0144b30 con 0x7f79e0144750 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c4003520 con 0x7f79e0144750 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79dea00640 1 --2- 172.21.15.165:0/2539344940 >> v2:172.21.15.172:3300/0 conn(0x7f79e00ce920 0x7f79e00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79dfe00640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 0x7f79e006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793014060 0 0) 0x7f79c4003520 con 0x7f79e0144750 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 msgr2=0x7f79e006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 0x7f79e006d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 >> v2:172.21.15.172:3300/0 conn(0x7f79e00ce920 msgr2=0x7f79e00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 --2- 172.21.15.165:0/2539344940 >> v2:172.21.15.172:3300/0 conn(0x7f79e00ce920 0x7f79e00cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0071890 con 0x7f79e0144750 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f79d0001670 con 0x7f79e0144750 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 >> v1:172.21.15.165:6789/0 conn(0x7f79e0144750 legacy=0x7f79e006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79c40039b0 0x7f79c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 0x7f79c4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c4004d40 0x7f79c40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00712d0 con 0x7f79c40039b0 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e00715b0 con 0x7f79c4004d40 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79c400bba0 con 0x7f79e006d290 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.710+0000 7f79df400640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79c40039b0 0x7f79c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79dfe00640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c4004d40 0x7f79c40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79dea00640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 0x7f79c4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79df400640 1 -- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 msgr2=0x7f79c4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79df400640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e006d290 0x7f79c4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79df400640 1 -- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c4004d40 msgr2=0x7f79c40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79df400640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c4004d40 0x7f79c40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79df400640 1 -- 172.21.15.165:0/2539344940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c4003520 con 0x7f79c40039b0 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.711+0000 7f79dfe00640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c4004d40 0x7f79c40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.712+0000 7f79df400640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79c40039b0 0x7f79c4003dd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c1c0 tx=0x7f79c4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b8eec0d1a4927c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.712+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0007c80 con 0x7f79c40039b0 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.712+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d002e070 con 0x7f79c40039b0 2024-04-25T10:11:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.712+0000 7f79de000640 1 -- 172.21.15.165:0/2539344940 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0005d30 con 0x7f79c40039b0 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79e6000640 1 -- 172.21.15.165:0/2539344940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79c40039b0 msgr2=0x7f79c4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79e6000640 1 --2- 172.21.15.165:0/2539344940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79c40039b0 0x7f79c4003dd0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f79d000c1c0 tx=0x7f79c4005830 comp rx=0 tx=0).stop 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79dfe00640 1 -- 172.21.15.165:0/2539344940 reap_dead start 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79e6000640 1 -- 172.21.15.165:0/2539344940 shutdown_connections 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79e6000640 1 -- 172.21.15.165:0/2539344940 >> 172.21.15.165:0/2539344940 conn(0x7f79e006abe0 msgr2=0x7f79e006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79e6000640 1 -- 172.21.15.165:0/2539344940 shutdown_connections 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.713+0000 7f79e6000640 1 -- 172.21.15.165:0/2539344940 wait complete. 2024-04-25T10:11:23.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.715+0000 7f79e6000640 1 Processor -- start 2024-04-25T10:11:23.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.715+0000 7f79e6000640 1 -- start start 2024-04-25T10:11:23.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79e6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c40039b0 0x7f79e0158530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79e6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e0139890 0x7f79e013c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79e6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 0x7f79e00b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79e6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e006f020 con 0x7f79e013ae30 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79e6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e006ece0 con 0x7f79c40039b0 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79e6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e0145450 con 0x7f79e0139890 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 0x7f79e00b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 0x7f79e00b1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55604/0 (socket says 172.21.15.165:55604) 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79dfe00640 1 -- 172.21.15.165:0/2092836229 learned_addr learned my addr 172.21.15.165:0/2092836229 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79df400640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c40039b0 0x7f79e0158530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79dfe00640 1 -- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e0139890 msgr2=0x7f79e013c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.716+0000 7f79dea00640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e0139890 0x7f79e013c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79dfe00640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e0139890 0x7f79e013c1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79dfe00640 1 -- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c40039b0 msgr2=0x7f79e0158530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79dfe00640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c40039b0 0x7f79e0158530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79dfe00640 1 -- 172.21.15.165:0/2092836229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e0139d90 con 0x7f79e013ae30 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79df400640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c40039b0 0x7f79e0158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79dfe00640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 0x7f79e00b1460 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f79d800d6f0 tx=0x7f79d800dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d8003b90 con 0x7f79e013ae30 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d8012070 con 0x7f79e013ae30 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d800b880 con 0x7f79e013ae30 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e013a070 con 0x7f79e013ae30 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.717+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e013a630 con 0x7f79e013ae30 2024-04-25T10:11:23.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.719+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e00ccf70 con 0x7f79e013ae30 2024-04-25T10:11:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.719+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79d800ba20 con 0x7f79e013ae30 2024-04-25T10:11:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.720+0000 7f79d7400640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79bc06b500 0x7f79bc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.720+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79d808a250 con 0x7f79e013ae30 2024-04-25T10:11:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.721+0000 7f79df400640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79bc06b500 0x7f79bc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.721+0000 7f79df400640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79bc06b500 0x7f79bc06d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f79d0032000 tx=0x7f79d0007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:23.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.725+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79e00ccf70 con 0x7f79e013ae30 2024-04-25T10:11:23.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.926+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79e0073990 con 0x7f79e013ae30 2024-04-25T10:11:23.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.929+0000 7f79d7400640 1 -- 172.21.15.165:0/2092836229 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16934 (secure 0 0 0) 0x7f79d801d020 con 0x7f79e013ae30 2024-04-25T10:11:23.935 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:23.935 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":97,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:22.910361+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:23.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.932+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79bc06b500 msgr2=0x7f79bc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.932+0000 7f79e6000640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79bc06b500 0x7f79bc06d9c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f79d0032000 tx=0x7f79d0007000 comp rx=0 tx=0).stop 2024-04-25T10:11:23.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.932+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 msgr2=0x7f79e00b1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:23.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.932+0000 7f79e6000640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 0x7f79e00b1460 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f79d800d6f0 tx=0x7f79d800dbb0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 shutdown_connections 2024-04-25T10:11:23.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79bc06b500 0x7f79bc06d9c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79e013ae30 0x7f79e00b1460 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79e0139890 0x7f79e013c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 --2- 172.21.15.165:0/2092836229 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79c40039b0 0x7f79e0158530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:23.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 >> 172.21.15.165:0/2092836229 conn(0x7f79e006abe0 msgr2=0x7f79e015b600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:23.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 shutdown_connections 2024-04-25T10:11:23.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:23.933+0000 7f79e6000640 1 -- 172.21.15.165:0/2092836229 wait complete. 2024-04-25T10:11:23.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:24.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.152+0000 7f015b200640 1 Processor -- start 2024-04-25T10:11:24.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.152+0000 7f015b200640 1 -- start start 2024-04-25T10:11:24.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.153+0000 7f015b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01540cfe90 0x7f015406f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.153+0000 7f015b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f01540ce830 0x7f015415f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.153+0000 7f015b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540d0270 0x7f015415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.153+0000 7f015b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0154160430 con 0x7f01540ce830 2024-04-25T10:11:24.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f015b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f015415fe70 con 0x7f01540cfe90 2024-04-25T10:11:24.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f015b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0154160150 con 0x7f01540d0270 2024-04-25T10:11:24.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f0159400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f01540ce830 0x7f015415f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f0159400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f01540ce830 0x7f015415f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55616/0 (socket says 172.21.15.165:55616) 2024-04-25T10:11:24.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f0159400640 1 -- 172.21.15.165:0/3106521586 learned_addr learned my addr 172.21.15.165:0/3106521586 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f0159e00640 1 --2- 172.21.15.165:0/3106521586 >> v2:172.21.15.172:3300/0 conn(0x7f01540cfe90 0x7f015406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.154+0000 7f015a800640 1 --2- 172.21.15.165:0/3106521586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540d0270 0x7f015415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.155+0000 7f0159400640 1 -- 172.21.15.165:0/3106521586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540d0270 msgr2=0x7f015415e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.155+0000 7f0159400640 1 --2- 172.21.15.165:0/3106521586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540d0270 0x7f015415e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.155+0000 7f0159400640 1 -- 172.21.15.165:0/3106521586 >> v2:172.21.15.172:3300/0 conn(0x7f01540cfe90 msgr2=0x7f015406f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.155+0000 7f0159400640 1 --2- 172.21.15.165:0/3106521586 >> v2:172.21.15.172:3300/0 conn(0x7f01540cfe90 0x7f015406f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.155+0000 7f0159400640 1 -- 172.21.15.165:0/3106521586 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540cc410 con 0x7f01540ce830 2024-04-25T10:11:24.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.156+0000 7f0159400640 1 --2- 172.21.15.165:0/3106521586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540ce830 0x7f015415f510 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f01540d22e0 tx=0x7f014402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71787ab373d6732 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.156+0000 7f0158a00640 1 -- 172.21.15.165:0/3106521586 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144037070 con 0x7f01540ce830 2024-04-25T10:11:24.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.156+0000 7f0158a00640 1 -- 172.21.15.165:0/3106521586 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0144032440 con 0x7f01540ce830 2024-04-25T10:11:24.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.156+0000 7f0158a00640 1 -- 172.21.15.165:0/3106521586 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0144032810 con 0x7f01540ce830 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.156+0000 7f015b200640 1 -- 172.21.15.165:0/3106521586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540ce830 msgr2=0x7f015415f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.156+0000 7f015b200640 1 --2- 172.21.15.165:0/3106521586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540ce830 0x7f015415f510 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f01540d22e0 tx=0x7f014402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 -- 172.21.15.165:0/3106521586 shutdown_connections 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 --2- 172.21.15.165:0/3106521586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540d0270 0x7f015415e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 --2- 172.21.15.165:0/3106521586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540ce830 0x7f015415f510 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 --2- 172.21.15.165:0/3106521586 >> v2:172.21.15.172:3300/0 conn(0x7f01540cfe90 0x7f015406f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 -- 172.21.15.165:0/3106521586 >> 172.21.15.165:0/3106521586 conn(0x7f01540d1630 msgr2=0x7f01540d1a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 -- 172.21.15.165:0/3106521586 shutdown_connections 2024-04-25T10:11:24.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.157+0000 7f015b200640 1 -- 172.21.15.165:0/3106521586 wait complete. 2024-04-25T10:11:24.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.158+0000 7f015b200640 1 Processor -- start 2024-04-25T10:11:24.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.159+0000 7f015b200640 1 -- start start 2024-04-25T10:11:24.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.159+0000 7f015b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01540ce830 0x7f01540c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.159+0000 7f015b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540cfe90 0x7f0154159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 0x7f01540ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015415fa50 con 0x7f01540d0270 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f015406fc40 con 0x7f01540ce830 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f015415fbd0 con 0x7f01540cfe90 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 0x7f01540ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 0x7f01540ccaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55620/0 (socket says 172.21.15.165:55620) 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 -- 172.21.15.165:0/824945064 learned_addr learned my addr 172.21.15.165:0/824945064 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f0159400640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540cfe90 0x7f0154159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f0159e00640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01540ce830 0x7f01540c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 -- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540cfe90 msgr2=0x7f0154159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540cfe90 0x7f0154159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 -- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01540ce830 msgr2=0x7f01540c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01540ce830 0x7f01540c5b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.160+0000 7f015a800640 1 -- 172.21.15.165:0/824945064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01540c6480 con 0x7f01540d0270 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.161+0000 7f015a800640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 0x7f01540ccaf0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0150007e90 tx=0x7f015000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.161+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150014420 con 0x7f01540d0270 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.161+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0150019070 con 0x7f01540d0270 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.161+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0150014800 con 0x7f01540d0270 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.161+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01540c6760 con 0x7f01540d0270 2024-04-25T10:11:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.161+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01540c6d20 con 0x7f01540d0270 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.162+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0154073990 con 0x7f01540d0270 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.163+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01500149a0 con 0x7f01540d0270 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.164+0000 7f014e000640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f013806b500 0x7f013806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.164+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f015008aa50 con 0x7f01540d0270 2024-04-25T10:11:24.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.164+0000 7f0159e00640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f013806b500 0x7f013806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.165+0000 7f0159e00640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f013806b500 0x7f013806d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0148006fd0 tx=0x7f0148008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.168+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0150059770 con 0x7f01540d0270 2024-04-25T10:11:24.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.363+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01540c4e50 con 0x7f01540d0270 2024-04-25T10:11:24.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.365+0000 7f014e000640 1 -- 172.21.15.165:0/824945064 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+16934 (secure 0 0 0) 0x7f015008a020 con 0x7f01540d0270 2024-04-25T10:11:24.370 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:24.370 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":97,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:22.910361+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:24.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f013806b500 msgr2=0x7f013806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f013806b500 0x7f013806d9c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0148006fd0 tx=0x7f0148008040 comp rx=0 tx=0).stop 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 msgr2=0x7f01540ccaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 0x7f01540ccaf0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f0150007e90 tx=0x7f015000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 shutdown_connections 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f013806b500 0x7f013806d9c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01540d0270 0x7f01540ccaf0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01540cfe90 0x7f0154159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 --2- 172.21.15.165:0/824945064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01540ce830 0x7f01540c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.367+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 >> 172.21.15.165:0/824945064 conn(0x7f01540d1630 msgr2=0x7f01540b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.368+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 shutdown_connections 2024-04-25T10:11:24.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.368+0000 7f015b200640 1 -- 172.21.15.165:0/824945064 wait complete. 2024-04-25T10:11:24.391 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:11:24.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.591+0000 7f33c8a00640 1 Processor -- start 2024-04-25T10:11:24.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.591+0000 7f33c8a00640 1 -- start start 2024-04-25T10:11:24.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.592+0000 7f33c8a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33c40b0730 0x7f33c40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.592+0000 7f33c8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33c406e150 0x7f33c406e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.592+0000 7f33c8a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c413e640 con 0x7f33c413d9f0 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.592+0000 7f33c8a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c406d390 con 0x7f33c406e150 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.592+0000 7f33c8a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c406d670 con 0x7f33c40b0730 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.592+0000 7f33c3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33c406e150 0x7f33c406e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.593+0000 7f33c3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33c406e150 0x7f33c406e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55624/0 (socket says 172.21.15.165:55624) 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.593+0000 7f33c3200640 1 -- 172.21.15.165:0/749851443 learned_addr learned my addr 172.21.15.165:0/749851443 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.593+0000 7f33c3200640 1 -- 172.21.15.165:0/749851443 >> v2:172.21.15.172:3300/0 conn(0x7f33c40b0730 msgr2=0x7f33c40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.593+0000 7f33c3200640 1 --2- 172.21.15.165:0/749851443 >> v2:172.21.15.172:3300/0 conn(0x7f33c40b0730 0x7f33c40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.593+0000 7f33c3200640 1 -- 172.21.15.165:0/749851443 >> v1:172.21.15.172:6789/0 conn(0x7f33c413d9f0 legacy=0x7f33c413df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.593+0000 7f33c3200640 1 -- 172.21.15.165:0/749851443 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c406d950 con 0x7f33c406e150 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.594+0000 7f33c3200640 1 --2- 172.21.15.165:0/749851443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c406e580 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f33c4140fa0 tx=0x7f33bc02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d276016338db9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.594+0000 7f33c1e00640 1 -- 172.21.15.165:0/749851443 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc038070 con 0x7f33c406e150 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.594+0000 7f33c1e00640 1 -- 172.21.15.165:0/749851443 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33bc033600 con 0x7f33c406e150 2024-04-25T10:11:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.594+0000 7f33c1e00640 1 -- 172.21.15.165:0/749851443 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33bc0339d0 con 0x7f33c406e150 2024-04-25T10:11:24.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 -- 172.21.15.165:0/749851443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 msgr2=0x7f33c406e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 --2- 172.21.15.165:0/749851443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c406e580 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f33c4140fa0 tx=0x7f33bc02c7e0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 -- 172.21.15.165:0/749851443 shutdown_connections 2024-04-25T10:11:24.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 --2- 172.21.15.165:0/749851443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c406e580 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 --2- 172.21.15.165:0/749851443 >> v2:172.21.15.172:3300/0 conn(0x7f33c40b0730 0x7f33c40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 -- 172.21.15.165:0/749851443 >> 172.21.15.165:0/749851443 conn(0x7f33c4061cc0 msgr2=0x7f33c406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:24.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 -- 172.21.15.165:0/749851443 shutdown_connections 2024-04-25T10:11:24.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.595+0000 7f33c8a00640 1 -- 172.21.15.165:0/749851443 wait complete. 2024-04-25T10:11:24.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.597+0000 7f33c8a00640 1 Processor -- start 2024-04-25T10:11:24.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.597+0000 7f33c8a00640 1 -- start start 2024-04-25T10:11:24.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c4138a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33c40b0730 0x7f33c41f3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c413d9f0 0x7f33c415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40ce880 con 0x7f33c406e150 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c406eac0 con 0x7f33c413d9f0 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c40cea00 con 0x7f33c40b0730 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c4138a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c4138a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55628/0 (socket says 172.21.15.165:55628) 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.598+0000 7f33c2800640 1 -- 172.21.15.165:0/3501198020 learned_addr learned my addr 172.21.15.165:0/3501198020 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33b9e00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33c40b0730 0x7f33c41f3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c2800640 1 -- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33c40b0730 msgr2=0x7f33c41f3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c3200640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c413d9f0 0x7f33c415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c2800640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33c40b0730 0x7f33c41f3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c2800640 1 -- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c413d9f0 msgr2=0x7f33c415ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c2800640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c413d9f0 0x7f33c415ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:24.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c2800640 1 -- 172.21.15.165:0/3501198020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c4138fb0 con 0x7f33c406e150 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c2800640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c4138a70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f33b400b3e0 tx=0x7f33b400b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4015020 con 0x7f33c406e150 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33b4011850 con 0x7f33c406e150 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33b4011c40 con 0x7f33c406e150 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c41f72d0 con 0x7f33c406e150 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.599+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c41f7890 con 0x7f33c406e150 2024-04-25T10:11:24.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.600+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c4073990 con 0x7f33c406e150 2024-04-25T10:11:24.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.601+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33b4018020 con 0x7f33c406e150 2024-04-25T10:11:24.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.602+0000 7f33bb400640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33a006b4d0 0x7f33a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:24.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.602+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33b408a2b0 con 0x7f33c406e150 2024-04-25T10:11:24.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.603+0000 7f33b9e00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33a006b4d0 0x7f33a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:24.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.603+0000 7f33b9e00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33a006b4d0 0x7f33a006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33c415e0b0 tx=0x7f33ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:24.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.607+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33b401f020 con 0x7f33c406e150 2024-04-25T10:11:24.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:24.842+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f33c4074100 con 0x7f33c406e150 2024-04-25T10:11:25.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.485+0000 7f33bb400640 1 -- 172.21.15.165:0/3501198020 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v64) v1 ==== 118+0+0 (secure 0 0 0) 0x7f33b40172c0 con 0x7f33c406e150 2024-04-25T10:11:25.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.489+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33a006b4d0 msgr2=0x7f33a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.489+0000 7f33c8a00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33a006b4d0 0x7f33a006d990 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f33c415e0b0 tx=0x7f33ac008040 comp rx=0 tx=0).stop 2024-04-25T10:11:25.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.489+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 msgr2=0x7f33c4138a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.489+0000 7f33c8a00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c4138a70 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f33b400b3e0 tx=0x7f33b400b8a0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 shutdown_connections 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33a006b4d0 0x7f33a006d990 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33c413d9f0 0x7f33c415ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33c40b0730 0x7f33c41f3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 --2- 172.21.15.165:0/3501198020 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33c406e150 0x7f33c4138a70 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 >> 172.21.15.165:0/3501198020 conn(0x7f33c4061cc0 msgr2=0x7f33c40b2880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 shutdown_connections 2024-04-25T10:11:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.490+0000 7f33c8a00640 1 -- 172.21.15.165:0/3501198020 wait complete. 2024-04-25T10:11:25.514 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:11:25.514 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:11:25.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.716+0000 7f5ff0e00640 1 Processor -- start 2024-04-25T10:11:25.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.716+0000 7f5ff0e00640 1 -- start start 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5ff0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5fec0b0730 0x7f5fec0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5ff0e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec13e640 con 0x7f5fec13d9f0 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5ff0e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fec06d390 con 0x7f5fec06e150 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5ff0e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fec06d670 con 0x7f5fec0b0730 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5fe2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5fec0b0730 0x7f5fec0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5fe2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5fec0b0730 0x7f5fec0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55630/0 (socket says 172.21.15.165:55630) 2024-04-25T10:11:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.718+0000 7f5fe2a00640 1 -- 172.21.15.165:0/1854915552 learned_addr learned my addr 172.21.15.165:0/1854915552 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:25.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.719+0000 7f5fe2a00640 1 -- 172.21.15.165:0/1854915552 >> v1:172.21.15.172:6789/0 conn(0x7f5fec06e150 legacy=0x7f5fec06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:25.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.719+0000 7f5fe2a00640 1 -- 172.21.15.165:0/1854915552 >> v1:172.21.15.165:6789/0 conn(0x7f5fec13d9f0 legacy=0x7f5fec13df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.719+0000 7f5fe2a00640 1 -- 172.21.15.165:0/1854915552 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec06d950 con 0x7f5fec0b0730 2024-04-25T10:11:25.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.720+0000 7f5fe2a00640 1 --2- 172.21.15.165:0/1854915552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec0b0730 0x7f5fec0b0bc0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400bb10 tx=0x7f5fd402c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4c1c71cc9054a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.720+0000 7f5feaa00640 1 -- 172.21.15.165:0/1854915552 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4032710 con 0x7f5fec0b0730 2024-04-25T10:11:25.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.720+0000 7f5feaa00640 1 -- 172.21.15.165:0/1854915552 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd40328b0 con 0x7f5fec0b0730 2024-04-25T10:11:25.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.720+0000 7f5feaa00640 1 -- 172.21.15.165:0/1854915552 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd4032ca0 con 0x7f5fec0b0730 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1854915552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec0b0730 msgr2=0x7f5fec0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1854915552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec0b0730 0x7f5fec0b0bc0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5fd400bb10 tx=0x7f5fd402c850 comp rx=0 tx=0).stop 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1854915552 shutdown_connections 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1854915552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec0b0730 0x7f5fec0b0bc0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1854915552 >> 172.21.15.165:0/1854915552 conn(0x7f5fec061cc0 msgr2=0x7f5fec06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1854915552 shutdown_connections 2024-04-25T10:11:25.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.721+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1854915552 wait complete. 2024-04-25T10:11:25.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.723+0000 7f5ff0e00640 1 Processor -- start 2024-04-25T10:11:25.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.723+0000 7f5ff0e00640 1 -- start start 2024-04-25T10:11:25.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5ff0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 0x7f5fec151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5ff0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fec0b0730 0x7f5fec15d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5ff0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fec13d9f0 0x7f5fec0cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5ff0e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0cea00 con 0x7f5fec06e150 2024-04-25T10:11:25.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5ff0e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0ce880 con 0x7f5fec13d9f0 2024-04-25T10:11:25.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5ff0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fec0ceb80 con 0x7f5fec0b0730 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 0x7f5fec151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 0x7f5fec151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55640/0 (socket says 172.21.15.165:55640) 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 -- 172.21.15.165:0/1722984694 learned_addr learned my addr 172.21.15.165:0/1722984694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 -- 172.21.15.165:0/1722984694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fec0b0730 msgr2=0x7f5fec15d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fec0b0730 0x7f5fec15d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 -- 172.21.15.165:0/1722984694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fec13d9f0 msgr2=0x7f5fec0cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fec13d9f0 0x7f5fec0cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:25.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.724+0000 7f5feb400640 1 -- 172.21.15.165:0/1722984694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fec0cd0f0 con 0x7f5fec06e150 2024-04-25T10:11:25.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.725+0000 7f5feb400640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 0x7f5fec151f40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c730 tx=0x7f5fdc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.725+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc016020 con 0x7f5fec06e150 2024-04-25T10:11:25.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.725+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc006d50 con 0x7f5fec06e150 2024-04-25T10:11:25.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.725+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fec0cd3d0 con 0x7f5fec06e150 2024-04-25T10:11:25.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.725+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc003bb0 con 0x7f5fec06e150 2024-04-25T10:11:25.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.725+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fec153950 con 0x7f5fec06e150 2024-04-25T10:11:25.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.726+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fec073990 con 0x7f5fec06e150 2024-04-25T10:11:25.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.727+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5fdc00a850 con 0x7f5fec06e150 2024-04-25T10:11:25.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.728+0000 7f5fe3e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fc806b4d0 0x7f5fc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:25.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.728+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5fdc089ff0 con 0x7f5fec06e150 2024-04-25T10:11:25.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.728+0000 7f5fe2a00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fc806b4d0 0x7f5fc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:25.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.729+0000 7f5fe2a00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fc806b4d0 0x7f5fc806d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4037000 tx=0x7f5fd4031000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:25.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.732+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5fdc020020 con 0x7f5fec06e150 2024-04-25T10:11:25.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:25.970+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5fec13fa90 con 0x7f5fec06e150 2024-04-25T10:11:26.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.489+0000 7f5fe3e00640 1 -- 172.21.15.165:0/1722984694 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v65) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5fdc019310 con 0x7f5fec06e150 2024-04-25T10:11:26.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fc806b4d0 msgr2=0x7f5fc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fc806b4d0 0x7f5fc806d990 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f5fd4037000 tx=0x7f5fd4031000 comp rx=0 tx=0).stop 2024-04-25T10:11:26.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 msgr2=0x7f5fec151f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 0x7f5fec151f40 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c730 tx=0x7f5fdc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 shutdown_connections 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fc806b4d0 0x7f5fc806d990 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5fec13d9f0 0x7f5fec0cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5fec0b0730 0x7f5fec15d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 --2- 172.21.15.165:0/1722984694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5fec06e150 0x7f5fec151f40 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.493+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 >> 172.21.15.165:0/1722984694 conn(0x7f5fec061cc0 msgr2=0x7f5fec15ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.494+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 shutdown_connections 2024-04-25T10:11:26.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.494+0000 7f5ff0e00640 1 -- 172.21.15.165:0/1722984694 wait complete. 2024-04-25T10:11:26.519 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:11:26.520 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:26.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.716+0000 7efd3da00640 1 Processor -- start 2024-04-25T10:11:26.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.716+0000 7efd3da00640 1 -- start start 2024-04-25T10:11:26.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.717+0000 7efd3da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7efd38144750 0x7efd3806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.717+0000 7efd3da00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd38144b30 con 0x7efd380cf030 2024-04-25T10:11:26.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.717+0000 7efd3da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd380712d0 con 0x7efd3806eb10 2024-04-25T10:11:26.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.717+0000 7efd3da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd380715b0 con 0x7efd38144750 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.718+0000 7efd37400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efd380cf030 0x7efd3806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35390/0 (socket says 172.21.15.165:35390) 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.718+0000 7efd37400640 1 -- 172.21.15.165:0/3390695737 learned_addr learned my addr 172.21.15.165:0/3390695737 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.718+0000 7efd36a00640 1 --2- 172.21.15.165:0/3390695737 >> v2:172.21.15.172:3300/0 conn(0x7efd38144750 0x7efd3806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.718+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199875968 0 0) 0x7efd38144b30 con 0x7efd380cf030 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.718+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd1c003520 con 0x7efd380cf030 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 137921971 0 0) 0x7efd1c003520 con 0x7efd380cf030 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 >> v2:172.21.15.172:3300/0 conn(0x7efd38144750 msgr2=0x7efd3806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 --2- 172.21.15.165:0/3390695737 >> v2:172.21.15.172:3300/0 conn(0x7efd38144750 0x7efd3806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 >> v1:172.21.15.172:6789/0 conn(0x7efd3806eb10 legacy=0x7efd380ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd38071890 con 0x7efd380cf030 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7efd300019f0 con 0x7efd380cf030 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 >> v1:172.21.15.165:6789/0 conn(0x7efd380cf030 legacy=0x7efd3806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0039b0 0x7efd1c003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0043d0 0x7efd1c0048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0050a0 0x7efd1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd380715b0 con 0x7efd1c0043d0 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00bfe0 con 0x7efd1c0050a0 2024-04-25T10:11:26.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c00c1e0 con 0x7efd1c0039b0 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd2fe00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0043d0 0x7efd1c0048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd36a00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0039b0 0x7efd1c003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.719+0000 7efd2fe00640 1 -- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0039b0 msgr2=0x7efd1c003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.720+0000 7efd2fe00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0039b0 0x7efd1c003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.720+0000 7efd2fe00640 1 -- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0050a0 msgr2=0x7efd1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.720+0000 7efd2fe00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0050a0 0x7efd1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.720+0000 7efd37400640 1 -- 172.21.15.165:0/3390695737 reap_dead start 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.720+0000 7efd2fe00640 1 -- 172.21.15.165:0/3390695737 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c003520 con 0x7efd1c0043d0 2024-04-25T10:11:26.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.720+0000 7efd36a00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd1c0039b0 0x7efd1c003db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.721+0000 7efd2fe00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0043d0 0x7efd1c0048a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efd20001780 tx=0x7efd1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e39bf62a9a10a37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.721+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd20001350 con 0x7efd1c0043d0 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.721+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd200052d0 con 0x7efd1c0043d0 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.721+0000 7efd36000640 1 -- 172.21.15.165:0/3390695737 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2002ab40 con 0x7efd1c0043d0 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.721+0000 7efd3da00640 1 -- 172.21.15.165:0/3390695737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0043d0 msgr2=0x7efd1c0048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.721+0000 7efd3da00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0043d0 0x7efd1c0048a0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7efd20001780 tx=0x7efd1c005830 comp rx=0 tx=0).stop 2024-04-25T10:11:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.722+0000 7efd3da00640 1 -- 172.21.15.165:0/3390695737 shutdown_connections 2024-04-25T10:11:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.722+0000 7efd3da00640 1 --2- 172.21.15.165:0/3390695737 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd1c0043d0 0x7efd1c0048a0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.722+0000 7efd3da00640 1 -- 172.21.15.165:0/3390695737 >> 172.21.15.165:0/3390695737 conn(0x7efd3806abe0 msgr2=0x7efd3806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.722+0000 7efd3da00640 1 -- 172.21.15.165:0/3390695737 shutdown_connections 2024-04-25T10:11:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.722+0000 7efd3da00640 1 -- 172.21.15.165:0/3390695737 wait complete. 2024-04-25T10:11:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.724+0000 7efd3da00640 1 Processor -- start 2024-04-25T10:11:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.724+0000 7efd3da00640 1 -- start start 2024-04-25T10:11:26.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd3da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0043d0 0x7efd380b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd3da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd380b1220 0x7efd3815d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd3da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 0x7efd3815dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd3da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd381455d0 con 0x7efd3815d890 2024-04-25T10:11:26.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd3da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd38145450 con 0x7efd1c0043d0 2024-04-25T10:11:26.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd3da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd3815c000 con 0x7efd380b1220 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 0x7efd3815dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 0x7efd3815dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55660/0 (socket says 172.21.15.165:55660) 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 -- 172.21.15.165:0/1190553522 learned_addr learned my addr 172.21.15.165:0/1190553522 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd2fe00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd380b1220 0x7efd3815d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd36a00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0043d0 0x7efd380b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 -- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd380b1220 msgr2=0x7efd3815d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd380b1220 0x7efd3815d350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 -- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0043d0 msgr2=0x7efd380b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0043d0 0x7efd380b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd37400640 1 -- 172.21.15.165:0/1190553522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd381428c0 con 0x7efd3815d890 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd2fe00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd380b1220 0x7efd3815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.725+0000 7efd36a00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0043d0 0x7efd380b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.726+0000 7efd37400640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 0x7efd3815dd10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efd3806ba60 tx=0x7efd3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.726+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000a610 con 0x7efd3815d890 2024-04-25T10:11:26.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.726+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3000a7b0 con 0x7efd3815d890 2024-04-25T10:11:26.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.726+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd38142ba0 con 0x7efd3815d890 2024-04-25T10:11:26.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.726+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3000aba0 con 0x7efd3815d890 2024-04-25T10:11:26.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.726+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd38143160 con 0x7efd3815d890 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.727+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd38073990 con 0x7efd3815d890 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.728+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd30011070 con 0x7efd3815d890 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.730+0000 7efd2ea00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.730+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd30089f70 con 0x7efd3815d890 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.730+0000 7efd36a00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:26.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.731+0000 7efd36a00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efd380692f0 tx=0x7efd28008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:26.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.734+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd3001a2b0 con 0x7efd3815d890 2024-04-25T10:11:26.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.966+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efd380b1600 con 0x7efd3815d890 2024-04-25T10:11:26.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.967+0000 7efd2ea00640 1 -- 172.21.15.165:0/1190553522 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7efd30058c50 con 0x7efd3815d890 2024-04-25T10:11:26.972 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:26.973 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:26.973 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:11:26.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd1406b4d0 msgr2=0x7efd1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd1406b4d0 0x7efd1406d990 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7efd380692f0 tx=0x7efd28008040 comp rx=0 tx=0).stop 2024-04-25T10:11:26.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 msgr2=0x7efd3815dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:26.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 0x7efd3815dd10 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7efd3806ba60 tx=0x7efd3000cc20 comp rx=0 tx=0).stop 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 shutdown_connections 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd1406b4d0 0x7efd1406d990 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd3815d890 0x7efd3815dd10 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd380b1220 0x7efd3815d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 --2- 172.21.15.165:0/1190553522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd1c0043d0 0x7efd380b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 >> 172.21.15.165:0/1190553522 conn(0x7efd3806abe0 msgr2=0x7efd380af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 shutdown_connections 2024-04-25T10:11:26.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:26.970+0000 7efd3da00640 1 -- 172.21.15.165:0/1190553522 wait complete. 2024-04-25T10:11:26.995 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:11:27.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.186+0000 7ff094c00640 1 Processor -- start 2024-04-25T10:11:27.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.186+0000 7ff094c00640 1 -- start start 2024-04-25T10:11:27.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.187+0000 7ff094c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff0900b93c0 0x7ff0900cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.187+0000 7ff094c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff0900d04f0 0x7ff0900d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.187+0000 7ff094c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff090161df0 con 0x7ff09015db90 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff094c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0900d1f50 con 0x7ff0900b93c0 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff094c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0900d0940 con 0x7ff0900d04f0 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff09015db90 0x7ff0901615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50114/0 (socket says 172.21.15.165:50114) 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08f400640 1 -- 172.21.15.165:0/3704119269 learned_addr learned my addr 172.21.15.165:0/3704119269 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:27.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08ea00640 1 --2- 172.21.15.165:0/3704119269 >> v2:172.21.15.165:3300/0 conn(0x7ff0900b93c0 0x7ff0900cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08fe00640 1 --2- 172.21.15.165:0/3704119269 >> v2:172.21.15.172:3300/0 conn(0x7ff0900d04f0 0x7ff0900d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08e000640 1 -- 172.21.15.165:0/3704119269 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060428863 0 0) 0x7ff090161df0 con 0x7ff09015db90 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08e000640 1 -- 172.21.15.165:0/3704119269 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff074003520 con 0x7ff09015db90 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08ea00640 1 -- 172.21.15.165:0/3704119269 >> v2:172.21.15.172:3300/0 conn(0x7ff0900d04f0 msgr2=0x7ff0900d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08ea00640 1 --2- 172.21.15.165:0/3704119269 >> v2:172.21.15.172:3300/0 conn(0x7ff0900d04f0 0x7ff0900d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08ea00640 1 -- 172.21.15.165:0/3704119269 >> v1:172.21.15.165:6789/0 conn(0x7ff09015db90 legacy=0x7ff0901615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.188+0000 7ff08ea00640 1 -- 172.21.15.165:0/3704119269 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0901620d0 con 0x7ff0900b93c0 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.190+0000 7ff08ea00640 1 --2- 172.21.15.165:0/3704119269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900b93c0 0x7ff0900cffb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff09008cc30 tx=0x7ff07c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ae28e5fb458a4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.190+0000 7ff08e000640 1 -- 172.21.15.165:0/3704119269 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c037070 con 0x7ff0900b93c0 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.190+0000 7ff08e000640 1 -- 172.21.15.165:0/3704119269 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c032440 con 0x7ff0900b93c0 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.190+0000 7ff08e000640 1 -- 172.21.15.165:0/3704119269 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c0327f0 con 0x7ff0900b93c0 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 -- 172.21.15.165:0/3704119269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900b93c0 msgr2=0x7ff0900cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 --2- 172.21.15.165:0/3704119269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900b93c0 0x7ff0900cffb0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7ff09008cc30 tx=0x7ff07c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 -- 172.21.15.165:0/3704119269 shutdown_connections 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 --2- 172.21.15.165:0/3704119269 >> v2:172.21.15.172:3300/0 conn(0x7ff0900d04f0 0x7ff0900d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 --2- 172.21.15.165:0/3704119269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900b93c0 0x7ff0900cffb0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 -- 172.21.15.165:0/3704119269 >> 172.21.15.165:0/3704119269 conn(0x7ff0900b52b0 msgr2=0x7ff0900b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 -- 172.21.15.165:0/3704119269 shutdown_connections 2024-04-25T10:11:27.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.191+0000 7ff094c00640 1 -- 172.21.15.165:0/3704119269 wait complete. 2024-04-25T10:11:27.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.192+0000 7ff094c00640 1 Processor -- start 2024-04-25T10:11:27.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 -- start start 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0900b93c0 0x7ff09014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 0x7ff0900ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09015db90 0x7ff0900cd150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900b68d0 con 0x7ff0900d04f0 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900d1be0 con 0x7ff0900b93c0 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.193+0000 7ff094c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0900b6a50 con 0x7ff09015db90 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 0x7ff0900ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 0x7ff0900ccc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35764/0 (socket says 172.21.15.165:35764) 2024-04-25T10:11:27.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 -- 172.21.15.165:0/3032974264 learned_addr learned my addr 172.21.15.165:0/3032974264 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:27.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 -- 172.21.15.165:0/3032974264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09015db90 msgr2=0x7ff0900cd150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09015db90 0x7ff0900cd150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 -- 172.21.15.165:0/3032974264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0900b93c0 msgr2=0x7ff09014eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0900b93c0 0x7ff09014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 -- 172.21.15.165:0/3032974264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff090159b50 con 0x7ff0900d04f0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.194+0000 7ff08ea00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 0x7ff0900ccc10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff07c02a1e0 tx=0x7ff07c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.195+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c037040 con 0x7ff0900d04f0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.195+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff07c0304c0 con 0x7ff0900d04f0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.195+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff07c030890 con 0x7ff0900d04f0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.195+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff090159e30 con 0x7ff0900d04f0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.195+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09015a3f0 con 0x7ff0900d04f0 2024-04-25T10:11:27.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.196+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09015a5f0 con 0x7ff0900d04f0 2024-04-25T10:11:27.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.197+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff07c044020 con 0x7ff0900d04f0 2024-04-25T10:11:27.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.198+0000 7ff087400640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07006b4d0 0x7ff07006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.198+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff07c0aa730 con 0x7ff0900d04f0 2024-04-25T10:11:27.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.198+0000 7ff08f400640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07006b4d0 0x7ff07006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.199+0000 7ff08f400640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07006b4d0 0x7ff07006d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff080000f60 tx=0x7ff0800078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.202+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff09015a5f0 con 0x7ff0900d04f0 2024-04-25T10:11:27.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.398+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff090073990 con 0x7ff0900d04f0 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.401+0000 7ff087400640 1 -- 172.21.15.165:0/3032974264 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v98) v1 ==== 74+0+16921 (secure 0 0 0) 0x7ff07c079380 con 0x7ff0900d04f0 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:27.406 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":98,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:24.461270+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07006b4d0 msgr2=0x7ff07006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07006b4d0 0x7ff07006d990 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7ff080000f60 tx=0x7ff0800078f0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 msgr2=0x7ff0900ccc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 0x7ff0900ccc10 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7ff07c02a1e0 tx=0x7ff07c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 shutdown_connections 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff07006b4d0 0x7ff07006d990 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff09015db90 0x7ff0900cd150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0900d04f0 0x7ff0900ccc10 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 --2- 172.21.15.165:0/3032974264 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0900b93c0 0x7ff09014eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.403+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 >> 172.21.15.165:0/3032974264 conn(0x7ff0900b52b0 msgr2=0x7ff0900c6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.404+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 shutdown_connections 2024-04-25T10:11:27.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.404+0000 7ff094c00640 1 -- 172.21.15.165:0/3032974264 wait complete. 2024-04-25T10:11:27.428 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:11:27.428 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.622+0000 7f043d600640 1 Processor -- start 2024-04-25T10:11:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.622+0000 7f043d600640 1 -- start start 2024-04-25T10:11:27.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.623+0000 7f043d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f043815dd20 0x7f04380b96d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f043d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 0x7f04380d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f043d600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f043815eac0 con 0x7f043815e560 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f043d600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04380d1c00 con 0x7f043815dd20 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f043d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f043815eda0 con 0x7f043815e100 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f0436000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 0x7f04380d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f0436a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f043815dd20 0x7f04380b96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f0436000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 0x7f04380d0aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37788/0 (socket says 172.21.15.165:37788) 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.624+0000 7f0436000640 1 -- 172.21.15.165:0/3774734627 learned_addr learned my addr 172.21.15.165:0/3774734627 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.625+0000 7f0436000640 1 -- 172.21.15.165:0/3774734627 >> v2:172.21.15.172:3300/0 conn(0x7f043815dd20 msgr2=0x7f04380b96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.625+0000 7f0436000640 1 --2- 172.21.15.165:0/3774734627 >> v2:172.21.15.172:3300/0 conn(0x7f043815dd20 0x7f04380b96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.625+0000 7f0436000640 1 -- 172.21.15.165:0/3774734627 >> v1:172.21.15.172:6789/0 conn(0x7f043815e560 legacy=0x7f04380d3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.625+0000 7f0436000640 1 -- 172.21.15.165:0/3774734627 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f043815f080 con 0x7f043815e100 2024-04-25T10:11:27.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.626+0000 7f0436000640 1 --2- 172.21.15.165:0/3774734627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 0x7f04380d0aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0420009e50 tx=0x7f042002a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d6df5fba6e7f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.627+0000 7f0435600640 1 -- 172.21.15.165:0/3774734627 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420036070 con 0x7f043815e100 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.627+0000 7f0435600640 1 -- 172.21.15.165:0/3774734627 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0420031600 con 0x7f043815e100 2024-04-25T10:11:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.627+0000 7f0435600640 1 -- 172.21.15.165:0/3774734627 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200319d0 con 0x7f043815e100 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 -- 172.21.15.165:0/3774734627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 msgr2=0x7f04380d0aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 --2- 172.21.15.165:0/3774734627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 0x7f04380d0aa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0420009e50 tx=0x7f042002a680 comp rx=0 tx=0).stop 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 -- 172.21.15.165:0/3774734627 shutdown_connections 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 --2- 172.21.15.165:0/3774734627 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e100 0x7f04380d0aa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 --2- 172.21.15.165:0/3774734627 >> v2:172.21.15.172:3300/0 conn(0x7f043815dd20 0x7f04380b96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 -- 172.21.15.165:0/3774734627 >> 172.21.15.165:0/3774734627 conn(0x7f0438061a20 msgr2=0x7f0438061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 -- 172.21.15.165:0/3774734627 shutdown_connections 2024-04-25T10:11:27.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.628+0000 7f043d600640 1 -- 172.21.15.165:0/3774734627 wait complete. 2024-04-25T10:11:27.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.630+0000 7f043d600640 1 Processor -- start 2024-04-25T10:11:27.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.630+0000 7f043d600640 1 -- start start 2024-04-25T10:11:27.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f043d600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043815dd20 0x7f0438151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f043d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 0x7f04380aa4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f043d600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e560 0x7f04380abf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f043d600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380d3500 con 0x7f043815e100 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f043d600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380d3380 con 0x7f043815dd20 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f043d600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380d3680 con 0x7f043815e560 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 0x7f04380aa4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 0x7f04380aa4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35778/0 (socket says 172.21.15.165:35778) 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 -- 172.21.15.165:0/1649842125 learned_addr learned my addr 172.21.15.165:0/1649842125 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436a00640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043815dd20 0x7f0438151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 -- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e560 msgr2=0x7f04380abf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0437400640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e560 0x7f04380abf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e560 0x7f04380abf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 -- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043815dd20 msgr2=0x7f0438151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043815dd20 0x7f0438151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.631+0000 7f0436000640 1 -- 172.21.15.165:0/1649842125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04381524d0 con 0x7f043815e100 2024-04-25T10:11:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f0437400640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e560 0x7f04380abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f0436a00640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043815dd20 0x7f0438151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f0436000640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 0x7f04380aa4a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0420034040 tx=0x7f0420004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420036050 con 0x7f043815e100 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0420042070 con 0x7f043815e100 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04380aae80 con 0x7f043815e100 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0420009510 con 0x7f043815e100 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.632+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04380ab440 con 0x7f043815e100 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.633+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0438073990 con 0x7f043815e100 2024-04-25T10:11:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.634+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0420007c70 con 0x7f043815e100 2024-04-25T10:11:27.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.635+0000 7f042ea00640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f041406b4d0 0x7f041406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:27.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.635+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04200ab250 con 0x7f043815e100 2024-04-25T10:11:27.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.636+0000 7f0436a00640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f041406b4d0 0x7f041406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.636+0000 7f0436a00640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f041406b4d0 0x7f041406d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0428009ac0 tx=0x7f0428008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.639+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0420045360 con 0x7f043815e100 2024-04-25T10:11:27.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.831+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f04380b52a0 con 0x7f041406b4d0 2024-04-25T10:11:27.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.839+0000 7f042ea00640 1 -- 172.21.15.165:0/1649842125 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240189 (secure 0 0 0) 0x7f04380b52a0 con 0x7f041406b4d0 2024-04-25T10:11:27.848 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:27.849 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f041406b4d0 msgr2=0x7f041406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f041406b4d0 0x7f041406d990 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f0428009ac0 tx=0x7f0428008040 comp rx=0 tx=0).stop 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 msgr2=0x7f04380aa4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 0x7f04380aa4a0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0420034040 tx=0x7f0420004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 shutdown_connections 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f041406b4d0 0x7f041406d990 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f043815e560 0x7f04380abf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f043815e100 0x7f04380aa4a0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 --2- 172.21.15.165:0/1649842125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f043815dd20 0x7f0438151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:27.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 >> 172.21.15.165:0/1649842125 conn(0x7f0438061a20 msgr2=0x7f043815fde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:27.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 shutdown_connections 2024-04-25T10:11:27.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:27.844+0000 7f043d600640 1 -- 172.21.15.165:0/1649842125 wait complete. 2024-04-25T10:11:27.868 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-04-25T10:11:26.321301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":416620,"kb_used_data":10300,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754558100,"statfs":{"total":773094113280,"available":772667494400,"internally_reserved":0,"allocated":10547200,"data_stored":5512823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":415944242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.886663"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442770+0000","last_change":"2024-04-25T10:11:21.442770+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442770+0000","last_undegraded":"2024-04-25T10:11:21.442770+0000","last_fullsized":"2024-04-25T10:11:21.442770+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.440840+0000","last_change":"2024-04-25T10:11:21.440840+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.440840+0000","last_undegraded":"2024-04-25T10:11:21.440840+0000","last_fullsized":"2024-04-25T10:11:21.440840+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449103+0000","last_change":"2024-04-25T10:11:21.449103+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449103+0000","last_undegraded":"2024-04-25T10:11:21.449103+0000","last_fullsized":"2024-04-25T10:11:21.449103+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.440368+0000","last_change":"2024-04-25T10:11:21.440368+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.440368+0000","last_undegraded":"2024-04-25T10:11:21.440368+0000","last_fullsized":"2024-04-25T10:11:21.440368+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439191+0000","last_change":"2024-04-25T10:11:21.439191+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439191+0000","last_undegraded":"2024-04-25T10:11:21.439191+0000","last_fullsized":"2024-04-25T10:11:21.439191+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442059+0000","last_change":"2024-04-25T10:11:21.442059+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442059+0000","last_undegraded":"2024-04-25T10:11:21.442059+0000","last_fullsized":"2024-04-25T10:11:21.442059+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.452665+0000","last_change":"2024-04-25T10:11:21.452665+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.452665+0000","last_undegraded":"2024-04-25T10:11:21.452665+0000","last_fullsized":"2024-04-25T10:11:21.452665+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.450719+0000","last_change":"2024-04-25T10:11:21.450719+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.450719+0000","last_undegraded":"2024-04-25T10:11:21.450719+0000","last_fullsized":"2024-04-25T10:11:21.450719+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.450486+0000","last_change":"2024-04-25T10:11:21.450486+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.450486+0000","last_undegraded":"2024-04-25T10:11:21.450486+0000","last_fullsized":"2024-04-25T10:11:21.450486+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.453086+0000","last_change":"2024-04-25T10:11:21.453086+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.453086+0000","last_undegraded":"2024-04-25T10:11:21.453086+0000","last_fullsized":"2024-04-25T10:11:21.453086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442790+0000","last_change":"2024-04-25T10:11:21.442790+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442790+0000","last_undegraded":"2024-04-25T10:11:21.442790+0000","last_fullsized":"2024-04-25T10:11:21.442790+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.443905+0000","last_change":"2024-04-25T10:11:21.443905+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.443905+0000","last_undegraded":"2024-04-25T10:11:21.443905+0000","last_fullsized":"2024-04-25T10:11:21.443905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449593+0000","last_change":"2024-04-25T10:11:21.449593+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449593+0000","last_undegraded":"2024-04-25T10:11:21.449593+0000","last_fullsized":"2024-04-25T10:11:21.449593+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.444770+0000","last_change":"2024-04-25T10:11:21.444770+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.444770+0000","last_undegraded":"2024-04-25T10:11:21.444770+0000","last_fullsized":"2024-04-25T10:11:21.444770+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442086+0000","last_change":"2024-04-25T10:11:21.442086+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442086+0000","last_undegraded":"2024-04-25T10:11:21.442086+0000","last_fullsized":"2024-04-25T10:11:21.442086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442114+0000","last_change":"2024-04-25T10:11:21.442114+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442114+0000","last_undegraded":"2024-04-25T10:11:21.442114+0000","last_fullsized":"2024-04-25T10:11:21.442114+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.441622+0000","last_change":"2024-04-25T10:11:21.441622+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.441622+0000","last_undegraded":"2024-04-25T10:11:21.441622+0000","last_fullsized":"2024-04-25T10:11:21.441622+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449760+0000","last_change":"2024-04-25T10:11:21.449760+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449760+0000","last_undegraded":"2024-04-25T10:11:21.449760+0000","last_fullsized":"2024-04-25T10:11:21.449760+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.441668+0000","last_change":"2024-04-25T10:11:21.441668+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.441668+0000","last_undegraded":"2024-04-25T10:11:21.441668+0000","last_fullsized":"2024-04-25T10:11:21.441668+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.448428+0000","last_change":"2024-04-25T10:11:21.448428+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.448428+0000","last_undegraded":"2024-04-25T10:11:21.448428+0000","last_fullsized":"2024-04-25T10:11:21.448428+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439326+0000","last_change":"2024-04-25T10:11:21.439326+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439326+0000","last_undegraded":"2024-04-25T10:11:21.439326+0000","last_fullsized":"2024-04-25T10:11:21.439326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.447145+0000","last_change":"2024-04-25T10:11:21.447145+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.447145+0000","last_undegraded":"2024-04-25T10:11:21.447145+0000","last_fullsized":"2024-04-25T10:11:21.447145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439322+0000","last_change":"2024-04-25T10:11:21.439322+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439322+0000","last_undegraded":"2024-04-25T10:11:21.439322+0000","last_fullsized":"2024-04-25T10:11:21.439322+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449678+0000","last_change":"2024-04-25T10:11:21.449678+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449678+0000","last_undegraded":"2024-04-25T10:11:21.449678+0000","last_fullsized":"2024-04-25T10:11:21.449678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439143+0000","last_change":"2024-04-25T10:11:21.439143+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439143+0000","last_undegraded":"2024-04-25T10:11:21.439143+0000","last_fullsized":"2024-04-25T10:11:21.439143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439217+0000","last_change":"2024-04-25T10:11:21.439217+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439217+0000","last_undegraded":"2024-04-25T10:11:21.439217+0000","last_fullsized":"2024-04-25T10:11:21.439217+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439018+0000","last_change":"2024-04-25T10:11:21.439018+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439018+0000","last_undegraded":"2024-04-25T10:11:21.439018+0000","last_fullsized":"2024-04-25T10:11:21.439018+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.444165+0000","last_change":"2024-04-25T10:11:21.444165+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.444165+0000","last_undegraded":"2024-04-25T10:11:21.444165+0000","last_fullsized":"2024-04-25T10:11:21.444165+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.447909+0000","last_change":"2024-04-25T10:11:21.447909+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.447909+0000","last_undegraded":"2024-04-25T10:11:21.447909+0000","last_fullsized":"2024-04-25T10:11:21.447909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.440554+0000","last_change":"2024-04-25T10:11:21.440554+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.440554+0000","last_undegraded":"2024-04-25T10:11:21.440554+0000","last_fullsized":"2024-04-25T10:11:21.440554+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449512+0000","last_change":"2024-04-25T10:11:21.449512+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449512+0000","last_undegraded":"2024-04-25T10:11:21.449512+0000","last_fullsized":"2024-04-25T10:11:21.449512+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442096+0000","last_change":"2024-04-25T10:11:21.442096+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442096+0000","last_undegraded":"2024-04-25T10:11:21.442096+0000","last_fullsized":"2024-04-25T10:11:21.442096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:11:21.452816+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:11:21.452816+0000","last_peered":"2024-04-25T10:11:21.452816+0000","last_clean":"2024-04-25T10:11:21.452816+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:11:21.452816+0000","last_undegraded":"2024-04-25T10:11:21.452816+0000","last_fullsized":"2024-04-25T10:11:21.452816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50956,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320884,"statfs":{"total":96636764160,"available":96584585216,"internally_reserved":0,"allocated":1765376,"data_stored":1132655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50372,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321468,"statfs":{"total":96636764160,"available":96585183232,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46212,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325628,"statfs":{"total":96636764160,"available":96589443072,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54532,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317308,"statfs":{"total":96636764160,"available":96580923392,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54536,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317304,"statfs":{"total":96636764160,"available":96580919296,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":1159168,"data_stored":536078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50380,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321460,"statfs":{"total":96636764160,"available":96585175040,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1765376,"data_stored":1132655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:27.872 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:28.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.087+0000 7ff5f7200640 1 Processor -- start 2024-04-25T10:11:28.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.087+0000 7ff5f7200640 1 -- start start 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5f7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5f0154ee0 0x7ff5f00cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5f7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5f00cfcf0 0x7ff5f00cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5f7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5f0071480 con 0x7ff5f00ce3c0 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5f7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f00ce830 con 0x7ff5f0154ee0 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5f7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5f0072090 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5efe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5f0154ee0 0x7ff5f00cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5efe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5f0154ee0 0x7ff5f00cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35784/0 (socket says 172.21.15.165:35784) 2024-04-25T10:11:28.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5efe00640 1 -- 172.21.15.165:0/1476112067 learned_addr learned my addr 172.21.15.165:0/1476112067 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:28.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5ef400640 1 --2- 172.21.15.165:0/1476112067 >> v2:172.21.15.172:3300/0 conn(0x7ff5f00cfcf0 0x7ff5f00cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.089+0000 7ff5efe00640 1 -- 172.21.15.165:0/1476112067 >> v2:172.21.15.172:3300/0 conn(0x7ff5f00cfcf0 msgr2=0x7ff5f00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.090+0000 7ff5efe00640 1 --2- 172.21.15.165:0/1476112067 >> v2:172.21.15.172:3300/0 conn(0x7ff5f00cfcf0 0x7ff5f00cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.090+0000 7ff5efe00640 1 -- 172.21.15.165:0/1476112067 >> v1:172.21.15.172:6789/0 conn(0x7ff5f00ce3c0 legacy=0x7ff5f015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.090+0000 7ff5efe00640 1 -- 172.21.15.165:0/1476112067 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0071760 con 0x7ff5f0154ee0 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.090+0000 7ff5efe00640 1 --2- 172.21.15.165:0/1476112067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f0154ee0 0x7ff5f00cf7b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00cf560 tx=0x7ff5e002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e04e1ad0ce5130a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.091+0000 7ff5eea00640 1 -- 172.21.15.165:0/1476112067 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0036070 con 0x7ff5f0154ee0 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.091+0000 7ff5eea00640 1 -- 172.21.15.165:0/1476112067 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0031600 con 0x7ff5f0154ee0 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.091+0000 7ff5eea00640 1 -- 172.21.15.165:0/1476112067 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e00319d0 con 0x7ff5f0154ee0 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.091+0000 7ff5f7200640 1 -- 172.21.15.165:0/1476112067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f0154ee0 msgr2=0x7ff5f00cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.091+0000 7ff5f7200640 1 --2- 172.21.15.165:0/1476112067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f0154ee0 0x7ff5f00cf7b0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7ff5f00cf560 tx=0x7ff5e002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.092+0000 7ff5f7200640 1 -- 172.21.15.165:0/1476112067 shutdown_connections 2024-04-25T10:11:28.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.092+0000 7ff5f7200640 1 --2- 172.21.15.165:0/1476112067 >> v2:172.21.15.172:3300/0 conn(0x7ff5f00cfcf0 0x7ff5f00cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.092+0000 7ff5f7200640 1 --2- 172.21.15.165:0/1476112067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f0154ee0 0x7ff5f00cf7b0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.092+0000 7ff5f7200640 1 -- 172.21.15.165:0/1476112067 >> 172.21.15.165:0/1476112067 conn(0x7ff5f006abe0 msgr2=0x7ff5f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.092+0000 7ff5f7200640 1 -- 172.21.15.165:0/1476112067 shutdown_connections 2024-04-25T10:11:28.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.092+0000 7ff5f7200640 1 -- 172.21.15.165:0/1476112067 wait complete. 2024-04-25T10:11:28.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.093+0000 7ff5f7200640 1 Processor -- start 2024-04-25T10:11:28.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.093+0000 7ff5f7200640 1 -- start start 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.094+0000 7ff5f7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5f00ce3c0 0x7ff5f0151ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.094+0000 7ff5f7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 0x7ff5f0156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.094+0000 7ff5f7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5f0154ee0 0x7ff5f0159d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.094+0000 7ff5f7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0071d20 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.094+0000 7ff5f7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0071230 con 0x7ff5f00ce3c0 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.094+0000 7ff5f7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f01552c0 con 0x7ff5f0154ee0 2024-04-25T10:11:28.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 0x7ff5f0156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 0x7ff5f0156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35798/0 (socket says 172.21.15.165:35798) 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 -- 172.21.15.165:0/961814005 learned_addr learned my addr 172.21.15.165:0/961814005 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5efe00640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5f00ce3c0 0x7ff5f0151ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 -- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5f0154ee0 msgr2=0x7ff5f0159d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5f0154ee0 0x7ff5f0159d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 -- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5f00ce3c0 msgr2=0x7ff5f0151ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5f00ce3c0 0x7ff5f0151ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 -- 172.21.15.165:0/961814005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0158fc0 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.095+0000 7ff5ef400640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 0x7ff5f0156640 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c820 tx=0x7ff5d800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.096+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8015070 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.096+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d8004240 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.096+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d800ce10 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.096+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5f0159260 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.096+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5f0159820 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.097+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5f0073990 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.098+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5d8007830 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.099+0000 7ff5e7e00640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.099+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5d808a280 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.099+0000 7ff5efe00640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.100+0000 7ff5efe00640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0033000 tx=0x7ff5e0009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.103+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5d8058f60 con 0x7ff5f00cfcf0 2024-04-25T10:11:28.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.281+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5f006d360 con 0x7ff5cc06b4d0 2024-04-25T10:11:28.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.287+0000 7ff5e7e00640 1 -- 172.21.15.165:0/961814005 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240189 (secure 0 0 0) 0x7ff5f006d360 con 0x7ff5cc06b4d0 2024-04-25T10:11:28.296 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:28.297 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 msgr2=0x7ff5cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0033000 tx=0x7ff5e0009000 comp rx=0 tx=0).stop 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 msgr2=0x7ff5f0156640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 0x7ff5f0156640 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c820 tx=0x7ff5d800cce0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 shutdown_connections 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5f0154ee0 0x7ff5f0159d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5f00cfcf0 0x7ff5f0156640 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 --2- 172.21.15.165:0/961814005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5f00ce3c0 0x7ff5f0151ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 >> 172.21.15.165:0/961814005 conn(0x7ff5f006abe0 msgr2=0x7ff5f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 shutdown_connections 2024-04-25T10:11:28.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.293+0000 7ff5f7200640 1 -- 172.21.15.165:0/961814005 wait complete. 2024-04-25T10:11:28.316 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":378,"stamp":"2024-04-25T10:11:26.321301+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":416620,"kb_used_data":10300,"kb_used_omap":12,"kb_used_meta":406195,"kb_avail":754558100,"statfs":{"total":773094113280,"available":772667494400,"internally_reserved":0,"allocated":10547200,"data_stored":5512823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":415944242},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.886663"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442770+0000","last_change":"2024-04-25T10:11:21.442770+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442770+0000","last_undegraded":"2024-04-25T10:11:21.442770+0000","last_fullsized":"2024-04-25T10:11:21.442770+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.440840+0000","last_change":"2024-04-25T10:11:21.440840+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.440840+0000","last_undegraded":"2024-04-25T10:11:21.440840+0000","last_fullsized":"2024-04-25T10:11:21.440840+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449103+0000","last_change":"2024-04-25T10:11:21.449103+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449103+0000","last_undegraded":"2024-04-25T10:11:21.449103+0000","last_fullsized":"2024-04-25T10:11:21.449103+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.440368+0000","last_change":"2024-04-25T10:11:21.440368+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.440368+0000","last_undegraded":"2024-04-25T10:11:21.440368+0000","last_fullsized":"2024-04-25T10:11:21.440368+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439191+0000","last_change":"2024-04-25T10:11:21.439191+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439191+0000","last_undegraded":"2024-04-25T10:11:21.439191+0000","last_fullsized":"2024-04-25T10:11:21.439191+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442059+0000","last_change":"2024-04-25T10:11:21.442059+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442059+0000","last_undegraded":"2024-04-25T10:11:21.442059+0000","last_fullsized":"2024-04-25T10:11:21.442059+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.452665+0000","last_change":"2024-04-25T10:11:21.452665+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.452665+0000","last_undegraded":"2024-04-25T10:11:21.452665+0000","last_fullsized":"2024-04-25T10:11:21.452665+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.450719+0000","last_change":"2024-04-25T10:11:21.450719+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.450719+0000","last_undegraded":"2024-04-25T10:11:21.450719+0000","last_fullsized":"2024-04-25T10:11:21.450719+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.450486+0000","last_change":"2024-04-25T10:11:21.450486+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.450486+0000","last_undegraded":"2024-04-25T10:11:21.450486+0000","last_fullsized":"2024-04-25T10:11:21.450486+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.453086+0000","last_change":"2024-04-25T10:11:21.453086+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.453086+0000","last_undegraded":"2024-04-25T10:11:21.453086+0000","last_fullsized":"2024-04-25T10:11:21.453086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442790+0000","last_change":"2024-04-25T10:11:21.442790+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442790+0000","last_undegraded":"2024-04-25T10:11:21.442790+0000","last_fullsized":"2024-04-25T10:11:21.442790+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.443905+0000","last_change":"2024-04-25T10:11:21.443905+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.443905+0000","last_undegraded":"2024-04-25T10:11:21.443905+0000","last_fullsized":"2024-04-25T10:11:21.443905+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449593+0000","last_change":"2024-04-25T10:11:21.449593+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449593+0000","last_undegraded":"2024-04-25T10:11:21.449593+0000","last_fullsized":"2024-04-25T10:11:21.449593+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.444770+0000","last_change":"2024-04-25T10:11:21.444770+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.444770+0000","last_undegraded":"2024-04-25T10:11:21.444770+0000","last_fullsized":"2024-04-25T10:11:21.444770+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442086+0000","last_change":"2024-04-25T10:11:21.442086+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442086+0000","last_undegraded":"2024-04-25T10:11:21.442086+0000","last_fullsized":"2024-04-25T10:11:21.442086+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442114+0000","last_change":"2024-04-25T10:11:21.442114+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442114+0000","last_undegraded":"2024-04-25T10:11:21.442114+0000","last_fullsized":"2024-04-25T10:11:21.442114+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.441622+0000","last_change":"2024-04-25T10:11:21.441622+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.441622+0000","last_undegraded":"2024-04-25T10:11:21.441622+0000","last_fullsized":"2024-04-25T10:11:21.441622+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449760+0000","last_change":"2024-04-25T10:11:21.449760+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449760+0000","last_undegraded":"2024-04-25T10:11:21.449760+0000","last_fullsized":"2024-04-25T10:11:21.449760+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.441668+0000","last_change":"2024-04-25T10:11:21.441668+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.441668+0000","last_undegraded":"2024-04-25T10:11:21.441668+0000","last_fullsized":"2024-04-25T10:11:21.441668+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.448428+0000","last_change":"2024-04-25T10:11:21.448428+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.448428+0000","last_undegraded":"2024-04-25T10:11:21.448428+0000","last_fullsized":"2024-04-25T10:11:21.448428+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439326+0000","last_change":"2024-04-25T10:11:21.439326+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439326+0000","last_undegraded":"2024-04-25T10:11:21.439326+0000","last_fullsized":"2024-04-25T10:11:21.439326+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.447145+0000","last_change":"2024-04-25T10:11:21.447145+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.447145+0000","last_undegraded":"2024-04-25T10:11:21.447145+0000","last_fullsized":"2024-04-25T10:11:21.447145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439322+0000","last_change":"2024-04-25T10:11:21.439322+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439322+0000","last_undegraded":"2024-04-25T10:11:21.439322+0000","last_fullsized":"2024-04-25T10:11:21.439322+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449678+0000","last_change":"2024-04-25T10:11:21.449678+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449678+0000","last_undegraded":"2024-04-25T10:11:21.449678+0000","last_fullsized":"2024-04-25T10:11:21.449678+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439143+0000","last_change":"2024-04-25T10:11:21.439143+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439143+0000","last_undegraded":"2024-04-25T10:11:21.439143+0000","last_fullsized":"2024-04-25T10:11:21.439143+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439217+0000","last_change":"2024-04-25T10:11:21.439217+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439217+0000","last_undegraded":"2024-04-25T10:11:21.439217+0000","last_fullsized":"2024-04-25T10:11:21.439217+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.439018+0000","last_change":"2024-04-25T10:11:21.439018+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.439018+0000","last_undegraded":"2024-04-25T10:11:21.439018+0000","last_fullsized":"2024-04-25T10:11:21.439018+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.444165+0000","last_change":"2024-04-25T10:11:21.444165+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.444165+0000","last_undegraded":"2024-04-25T10:11:21.444165+0000","last_fullsized":"2024-04-25T10:11:21.444165+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.447909+0000","last_change":"2024-04-25T10:11:21.447909+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.447909+0000","last_undegraded":"2024-04-25T10:11:21.447909+0000","last_fullsized":"2024-04-25T10:11:21.447909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.440554+0000","last_change":"2024-04-25T10:11:21.440554+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.440554+0000","last_undegraded":"2024-04-25T10:11:21.440554+0000","last_fullsized":"2024-04-25T10:11:21.440554+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.449512+0000","last_change":"2024-04-25T10:11:21.449512+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.449512+0000","last_undegraded":"2024-04-25T10:11:21.449512+0000","last_fullsized":"2024-04-25T10:11:21.449512+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:11:22.432352+0000","last_change":"2024-04-25T10:11:22.432352+0000","last_active":"2024-04-25T10:11:22.432352+0000","last_peered":"2024-04-25T10:11:22.432352+0000","last_clean":"2024-04-25T10:11:22.432352+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:22.432352+0000","last_undegraded":"2024-04-25T10:11:22.432352+0000","last_fullsized":"2024-04-25T10:11:22.432352+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":1,"reported_epoch":95,"state":"creating+peering","last_fresh":"2024-04-25T10:11:21.442096+0000","last_change":"2024-04-25T10:11:21.442096+0000","last_active":"2024-04-25T10:11:21.416487+0000","last_peered":"2024-04-25T10:11:21.416487+0000","last_clean":"2024-04-25T10:11:21.416487+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:11:21.442096+0000","last_undegraded":"2024-04-25T10:11:21.442096+0000","last_fullsized":"2024-04-25T10:11:21.442096+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-25T10:11:21.452816+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:11:21.452816+0000","last_peered":"2024-04-25T10:11:21.452816+0000","last_clean":"2024-04-25T10:11:21.452816+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:11:21.452816+0000","last_undegraded":"2024-04-25T10:11:21.452816+0000","last_fullsized":"2024-04-25T10:11:21.452816+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50956,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320884,"statfs":{"total":96636764160,"available":96584585216,"internally_reserved":0,"allocated":1765376,"data_stored":1132655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574966,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50372,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321468,"statfs":{"total":96636764160,"available":96585183232,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574966,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46212,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325628,"statfs":{"total":96636764160,"available":96589443072,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54532,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317308,"statfs":{"total":96636764160,"available":96580923392,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54536,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317304,"statfs":{"total":96636764160,"available":96580919296,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574965,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54520,"kb_used_data":1132,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317320,"statfs":{"total":96636764160,"available":96580935680,"internally_reserved":0,"allocated":1159168,"data_stored":536078,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50380,"kb_used_data":1144,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321460,"statfs":{"total":96636764160,"available":96585175040,"internally_reserved":0,"allocated":1171456,"data_stored":542287,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574966,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55112,"kb_used_data":1724,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316728,"statfs":{"total":96636764160,"available":96580329472,"internally_reserved":0,"allocated":1765376,"data_stored":1132655,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:28.321 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:28.400 INFO:tasks.ceph.mds.a.smithi165.stderr:2024-04-25T10:11:28.395+0000 7fab75400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:11:28.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.559+0000 7f1b53c00640 1 Processor -- start 2024-04-25T10:11:28.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.559+0000 7f1b53c00640 1 -- start start 2024-04-25T10:11:28.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b53c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c15dc90 0x7f1b4c1625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b53c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b4c0cc8e0 0x7f1b4c15ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b53c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b4c15f390 con 0x7f1b4c0d1810 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b53c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cce50 con 0x7f1b4c0cc8e0 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b53c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c161af0 con 0x7f1b4c15dc90 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b4be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b4c0cc8e0 0x7f1b4c15ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b4be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b4c0cc8e0 0x7f1b4c15ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35800/0 (socket says 172.21.15.165:35800) 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.561+0000 7f1b4be00640 1 -- 172.21.15.165:0/457151083 learned_addr learned my addr 172.21.15.165:0/457151083 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.562+0000 7f1b51200640 1 --2- 172.21.15.165:0/457151083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c15dc90 0x7f1b4c1625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.562+0000 7f1b4be00640 1 -- 172.21.15.165:0/457151083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c15dc90 msgr2=0x7f1b4c1625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.562+0000 7f1b4be00640 1 --2- 172.21.15.165:0/457151083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c15dc90 0x7f1b4c1625d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.562+0000 7f1b4be00640 1 -- 172.21.15.165:0/457151083 >> v1:172.21.15.172:6789/0 conn(0x7f1b4c0d1810 legacy=0x7f1b4c0cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:28.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.562+0000 7f1b4be00640 1 -- 172.21.15.165:0/457151083 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c161dd0 con 0x7f1b4c0cc8e0 2024-04-25T10:11:28.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.563+0000 7f1b4be00640 1 --2- 172.21.15.165:0/457151083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0cc8e0 0x7f1b4c15ee50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009e50 tx=0x7f1b3402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bfec79137fb3c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.563+0000 7f1b4b400640 1 -- 172.21.15.165:0/457151083 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34036070 con 0x7f1b4c0cc8e0 2024-04-25T10:11:28.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.563+0000 7f1b4b400640 1 -- 172.21.15.165:0/457151083 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b34031600 con 0x7f1b4c0cc8e0 2024-04-25T10:11:28.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.563+0000 7f1b4b400640 1 -- 172.21.15.165:0/457151083 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b340319d0 con 0x7f1b4c0cc8e0 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 -- 172.21.15.165:0/457151083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0cc8e0 msgr2=0x7f1b4c15ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 --2- 172.21.15.165:0/457151083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0cc8e0 0x7f1b4c15ee50 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009e50 tx=0x7f1b3402a630 comp rx=0 tx=0).stop 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 -- 172.21.15.165:0/457151083 shutdown_connections 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 --2- 172.21.15.165:0/457151083 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0cc8e0 0x7f1b4c15ee50 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 --2- 172.21.15.165:0/457151083 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c15dc90 0x7f1b4c1625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 -- 172.21.15.165:0/457151083 >> 172.21.15.165:0/457151083 conn(0x7f1b4c06f590 msgr2=0x7f1b4c06f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 -- 172.21.15.165:0/457151083 shutdown_connections 2024-04-25T10:11:28.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.564+0000 7f1b53c00640 1 -- 172.21.15.165:0/457151083 wait complete. 2024-04-25T10:11:28.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.566+0000 7f1b53c00640 1 Processor -- start 2024-04-25T10:11:28.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.566+0000 7f1b53c00640 1 -- start start 2024-04-25T10:11:28.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b53c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c0cc8e0 0x7f1b4c153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b53c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 0x7f1b4c0b6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b53c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4c15dc90 0x7f1b4c0ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b53c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15e070 con 0x7f1b4c0d1810 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b53c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c0cd130 con 0x7f1b4c15dc90 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b53c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b4c15e1f0 con 0x7f1b4c0cc8e0 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.567+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 0x7f1b4c0b6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 0x7f1b4c0b6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35810/0 (socket says 172.21.15.165:35810) 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 -- 172.21.15.165:0/3974590080 learned_addr learned my addr 172.21.15.165:0/3974590080 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 -- 172.21.15.165:0/3974590080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c0cc8e0 msgr2=0x7f1b4c153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c0cc8e0 0x7f1b4c153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 -- 172.21.15.165:0/3974590080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4c15dc90 msgr2=0x7f1b4c0ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4c15dc90 0x7f1b4c0ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 -- 172.21.15.165:0/3974590080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b4c1f7630 con 0x7f1b4c0d1810 2024-04-25T10:11:28.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b4be00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 0x7f1b4c0b6960 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009e50 tx=0x7f1b34004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.568+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34036050 con 0x7f1b4c0d1810 2024-04-25T10:11:28.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.569+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b34042070 con 0x7f1b4c0d1810 2024-04-25T10:11:28.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.569+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b34009510 con 0x7f1b4c0d1810 2024-04-25T10:11:28.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.569+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b4c1f78d0 con 0x7f1b4c0d1810 2024-04-25T10:11:28.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.569+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b4c1f7e90 con 0x7f1b4c0d1810 2024-04-25T10:11:28.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.570+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b4c073990 con 0x7f1b4c0d1810 2024-04-25T10:11:28.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.571+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b34007c50 con 0x7f1b4c0d1810 2024-04-25T10:11:28.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.572+0000 7f1b48c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b2806b4d0 0x7f1b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:28.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.572+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b340ab420 con 0x7f1b4c0d1810 2024-04-25T10:11:28.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.572+0000 7f1b51200640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b2806b4d0 0x7f1b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:28.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.573+0000 7f1b51200640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b2806b4d0 0x7f1b2806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006fd0 tx=0x7f1b3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:28.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.576+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b4c15f390 con 0x7f1b4c0d1810 2024-04-25T10:11:28.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.757+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b4c0cf6b0 con 0x7f1b2806b4d0 2024-04-25T10:11:28.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.767+0000 7f1b48c00640 1 -- 172.21.15.165:0/3974590080 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7f1b4c0cf6b0 con 0x7f1b2806b4d0 2024-04-25T10:11:28.776 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:28.778 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:11:28.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b2806b4d0 msgr2=0x7f1b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b2806b4d0 0x7f1b2806d990 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c006fd0 tx=0x7f1b3c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:28.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 msgr2=0x7f1b4c0b6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 0x7f1b4c0b6960 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f1b34009e50 tx=0x7f1b34004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 shutdown_connections 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b2806b4d0 0x7f1b2806d990 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b4c15dc90 0x7f1b4c0ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b4c0d1810 0x7f1b4c0b6960 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 --2- 172.21.15.165:0/3974590080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b4c0cc8e0 0x7f1b4c153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 >> 172.21.15.165:0/3974590080 conn(0x7f1b4c06f590 msgr2=0x7f1b4c15cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 shutdown_connections 2024-04-25T10:11:28.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:28.773+0000 7f1b53c00640 1 -- 172.21.15.165:0/3974590080 wait complete. 2024-04-25T10:11:28.796 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-25T10:11:28.321878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421140,"kb_used_data":10664,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754553580,"statfs":{"total":773094113280,"available":772662865920,"internally_reserved":0,"allocated":10919936,"data_stored":5688060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":420204082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001959"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475221+0000","last_change":"2024-04-25T10:11:23.514302+0000","last_active":"2024-04-25T10:11:24.475221+0000","last_peered":"2024-04-25T10:11:24.475221+0000","last_clean":"2024-04-25T10:11:24.475221+0000","last_became_active":"2024-04-25T10:11:23.509239+0000","last_became_peered":"2024-04-25T10:11:23.509239+0000","last_unstale":"2024-04-25T10:11:24.475221+0000","last_undegraded":"2024-04-25T10:11:24.475221+0000","last_fullsized":"2024-04-25T10:11:24.475221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:47.357088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521483+0000","last_change":"2024-04-25T10:11:23.511726+0000","last_active":"2024-04-25T10:11:24.521483+0000","last_peered":"2024-04-25T10:11:24.521483+0000","last_clean":"2024-04-25T10:11:24.521483+0000","last_became_active":"2024-04-25T10:11:23.511477+0000","last_became_peered":"2024-04-25T10:11:23.511477+0000","last_unstale":"2024-04-25T10:11:24.521483+0000","last_undegraded":"2024-04-25T10:11:24.521483+0000","last_fullsized":"2024-04-25T10:11:24.521483+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:21.609016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474979+0000","last_change":"2024-04-25T10:11:23.504635+0000","last_active":"2024-04-25T10:11:24.474979+0000","last_peered":"2024-04-25T10:11:24.474979+0000","last_clean":"2024-04-25T10:11:24.474979+0000","last_became_active":"2024-04-25T10:11:23.504397+0000","last_became_peered":"2024-04-25T10:11:23.504397+0000","last_unstale":"2024-04-25T10:11:24.474979+0000","last_undegraded":"2024-04-25T10:11:24.474979+0000","last_fullsized":"2024-04-25T10:11:24.474979+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:15.008119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968039+0000","last_change":"2024-04-25T10:11:23.501516+0000","last_active":"2024-04-25T10:11:25.968039+0000","last_peered":"2024-04-25T10:11:25.968039+0000","last_clean":"2024-04-25T10:11:25.968039+0000","last_became_active":"2024-04-25T10:11:23.501180+0000","last_became_peered":"2024-04-25T10:11:23.501180+0000","last_unstale":"2024-04-25T10:11:25.968039+0000","last_undegraded":"2024-04-25T10:11:25.968039+0000","last_fullsized":"2024-04-25T10:11:25.968039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.781341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478461+0000","last_change":"2024-04-25T10:11:23.493320+0000","last_active":"2024-04-25T10:11:24.478461+0000","last_peered":"2024-04-25T10:11:24.478461+0000","last_clean":"2024-04-25T10:11:24.478461+0000","last_became_active":"2024-04-25T10:11:23.492988+0000","last_became_peered":"2024-04-25T10:11:23.492988+0000","last_unstale":"2024-04-25T10:11:24.478461+0000","last_undegraded":"2024-04-25T10:11:24.478461+0000","last_fullsized":"2024-04-25T10:11:24.478461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:43.297903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863718+0000","last_change":"2024-04-25T10:11:23.503887+0000","last_active":"2024-04-25T10:11:24.863718+0000","last_peered":"2024-04-25T10:11:24.863718+0000","last_clean":"2024-04-25T10:11:24.863718+0000","last_became_active":"2024-04-25T10:11:23.503580+0000","last_became_peered":"2024-04-25T10:11:23.503580+0000","last_unstale":"2024-04-25T10:11:24.863718+0000","last_undegraded":"2024-04-25T10:11:24.863718+0000","last_fullsized":"2024-04-25T10:11:24.863718+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:00.002046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950146+0000","last_change":"2024-04-25T10:11:23.506780+0000","last_active":"2024-04-25T10:11:25.950146+0000","last_peered":"2024-04-25T10:11:25.950146+0000","last_clean":"2024-04-25T10:11:25.950146+0000","last_became_active":"2024-04-25T10:11:23.506380+0000","last_became_peered":"2024-04-25T10:11:23.506380+0000","last_unstale":"2024-04-25T10:11:25.950146+0000","last_undegraded":"2024-04-25T10:11:25.950146+0000","last_fullsized":"2024-04-25T10:11:25.950146+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:35.118925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967401+0000","last_change":"2024-04-25T10:11:23.496772+0000","last_active":"2024-04-25T10:11:25.967401+0000","last_peered":"2024-04-25T10:11:25.967401+0000","last_clean":"2024-04-25T10:11:25.967401+0000","last_became_active":"2024-04-25T10:11:23.496466+0000","last_became_peered":"2024-04-25T10:11:23.496466+0000","last_unstale":"2024-04-25T10:11:25.967401+0000","last_undegraded":"2024-04-25T10:11:25.967401+0000","last_fullsized":"2024-04-25T10:11:25.967401+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480543+0000","last_change":"2024-04-25T10:11:23.496579+0000","last_active":"2024-04-25T10:11:24.480543+0000","last_peered":"2024-04-25T10:11:24.480543+0000","last_clean":"2024-04-25T10:11:24.480543+0000","last_became_active":"2024-04-25T10:11:23.495543+0000","last_became_peered":"2024-04-25T10:11:23.495543+0000","last_unstale":"2024-04-25T10:11:24.480543+0000","last_undegraded":"2024-04-25T10:11:24.480543+0000","last_fullsized":"2024-04-25T10:11:24.480543+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:46.821062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966387+0000","last_change":"2024-04-25T10:11:23.507660+0000","last_active":"2024-04-25T10:11:25.966387+0000","last_peered":"2024-04-25T10:11:25.966387+0000","last_clean":"2024-04-25T10:11:25.966387+0000","last_became_active":"2024-04-25T10:11:23.507339+0000","last_became_peered":"2024-04-25T10:11:23.507339+0000","last_unstale":"2024-04-25T10:11:25.966387+0000","last_undegraded":"2024-04-25T10:11:25.966387+0000","last_fullsized":"2024-04-25T10:11:25.966387+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.303357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479723+0000","last_change":"2024-04-25T10:11:23.495970+0000","last_active":"2024-04-25T10:11:24.479723+0000","last_peered":"2024-04-25T10:11:24.479723+0000","last_clean":"2024-04-25T10:11:24.479723+0000","last_became_active":"2024-04-25T10:11:23.495635+0000","last_became_peered":"2024-04-25T10:11:23.495635+0000","last_unstale":"2024-04-25T10:11:24.479723+0000","last_undegraded":"2024-04-25T10:11:24.479723+0000","last_fullsized":"2024-04-25T10:11:24.479723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:46.519451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965939+0000","last_change":"2024-04-25T10:11:23.497428+0000","last_active":"2024-04-25T10:11:25.965939+0000","last_peered":"2024-04-25T10:11:25.965939+0000","last_clean":"2024-04-25T10:11:25.965939+0000","last_became_active":"2024-04-25T10:11:23.495767+0000","last_became_peered":"2024-04-25T10:11:23.495767+0000","last_unstale":"2024-04-25T10:11:25.965939+0000","last_undegraded":"2024-04-25T10:11:25.965939+0000","last_fullsized":"2024-04-25T10:11:25.965939+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:56.257951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968172+0000","last_change":"2024-04-25T10:11:23.505393+0000","last_active":"2024-04-25T10:11:25.968172+0000","last_peered":"2024-04-25T10:11:25.968172+0000","last_clean":"2024-04-25T10:11:25.968172+0000","last_became_active":"2024-04-25T10:11:23.502745+0000","last_became_peered":"2024-04-25T10:11:23.502745+0000","last_unstale":"2024-04-25T10:11:25.968172+0000","last_undegraded":"2024-04-25T10:11:25.968172+0000","last_fullsized":"2024-04-25T10:11:25.968172+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:22.077916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478359+0000","last_change":"2024-04-25T10:11:23.504530+0000","last_active":"2024-04-25T10:11:24.478359+0000","last_peered":"2024-04-25T10:11:24.478359+0000","last_clean":"2024-04-25T10:11:24.478359+0000","last_became_active":"2024-04-25T10:11:23.504230+0000","last_became_peered":"2024-04-25T10:11:23.504230+0000","last_unstale":"2024-04-25T10:11:24.478359+0000","last_undegraded":"2024-04-25T10:11:24.478359+0000","last_fullsized":"2024-04-25T10:11:24.478359+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:31.827948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968022+0000","last_change":"2024-04-25T10:11:23.514234+0000","last_active":"2024-04-25T10:11:25.968022+0000","last_peered":"2024-04-25T10:11:25.968022+0000","last_clean":"2024-04-25T10:11:25.968022+0000","last_became_active":"2024-04-25T10:11:23.513691+0000","last_became_peered":"2024-04-25T10:11:23.513691+0000","last_unstale":"2024-04-25T10:11:25.968022+0000","last_undegraded":"2024-04-25T10:11:25.968022+0000","last_fullsized":"2024-04-25T10:11:25.968022+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.091532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864271+0000","last_change":"2024-04-25T10:11:23.496506+0000","last_active":"2024-04-25T10:11:24.864271+0000","last_peered":"2024-04-25T10:11:24.864271+0000","last_clean":"2024-04-25T10:11:24.864271+0000","last_became_active":"2024-04-25T10:11:23.496229+0000","last_became_peered":"2024-04-25T10:11:23.496229+0000","last_unstale":"2024-04-25T10:11:24.864271+0000","last_undegraded":"2024-04-25T10:11:24.864271+0000","last_fullsized":"2024-04-25T10:11:24.864271+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:40.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863850+0000","last_change":"2024-04-25T10:11:23.518147+0000","last_active":"2024-04-25T10:11:24.863850+0000","last_peered":"2024-04-25T10:11:24.863850+0000","last_clean":"2024-04-25T10:11:24.863850+0000","last_became_active":"2024-04-25T10:11:23.517364+0000","last_became_peered":"2024-04-25T10:11:23.517364+0000","last_unstale":"2024-04-25T10:11:24.863850+0000","last_undegraded":"2024-04-25T10:11:24.863850+0000","last_fullsized":"2024-04-25T10:11:24.863850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:55.872365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475072+0000","last_change":"2024-04-25T10:11:23.506788+0000","last_active":"2024-04-25T10:11:24.475072+0000","last_peered":"2024-04-25T10:11:24.475072+0000","last_clean":"2024-04-25T10:11:24.475072+0000","last_became_active":"2024-04-25T10:11:23.506508+0000","last_became_peered":"2024-04-25T10:11:23.506508+0000","last_unstale":"2024-04-25T10:11:24.475072+0000","last_undegraded":"2024-04-25T10:11:24.475072+0000","last_fullsized":"2024-04-25T10:11:24.475072+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:35.118925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479440+0000","last_change":"2024-04-25T10:11:23.505949+0000","last_active":"2024-04-25T10:11:24.479440+0000","last_peered":"2024-04-25T10:11:24.479440+0000","last_clean":"2024-04-25T10:11:24.479440+0000","last_became_active":"2024-04-25T10:11:23.501690+0000","last_became_peered":"2024-04-25T10:11:23.501690+0000","last_unstale":"2024-04-25T10:11:24.479440+0000","last_undegraded":"2024-04-25T10:11:24.479440+0000","last_fullsized":"2024-04-25T10:11:24.479440+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.972961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968249+0000","last_change":"2024-04-25T10:11:23.514427+0000","last_active":"2024-04-25T10:11:25.968249+0000","last_peered":"2024-04-25T10:11:25.968249+0000","last_clean":"2024-04-25T10:11:25.968249+0000","last_became_active":"2024-04-25T10:11:23.514184+0000","last_became_peered":"2024-04-25T10:11:23.514184+0000","last_unstale":"2024-04-25T10:11:25.968249+0000","last_undegraded":"2024-04-25T10:11:25.968249+0000","last_fullsized":"2024-04-25T10:11:25.968249+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:36.868831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478219+0000","last_change":"2024-04-25T10:11:23.501619+0000","last_active":"2024-04-25T10:11:24.478219+0000","last_peered":"2024-04-25T10:11:24.478219+0000","last_clean":"2024-04-25T10:11:24.478219+0000","last_became_active":"2024-04-25T10:11:23.500466+0000","last_became_peered":"2024-04-25T10:11:23.500466+0000","last_unstale":"2024-04-25T10:11:24.478219+0000","last_undegraded":"2024-04-25T10:11:24.478219+0000","last_fullsized":"2024-04-25T10:11:24.478219+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478814+0000","last_change":"2024-04-25T10:11:23.502202+0000","last_active":"2024-04-25T10:11:24.478814+0000","last_peered":"2024-04-25T10:11:24.478814+0000","last_clean":"2024-04-25T10:11:24.478814+0000","last_became_active":"2024-04-25T10:11:23.501391+0000","last_became_peered":"2024-04-25T10:11:23.501391+0000","last_unstale":"2024-04-25T10:11:24.478814+0000","last_undegraded":"2024-04-25T10:11:24.478814+0000","last_fullsized":"2024-04-25T10:11:24.478814+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.937312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478106+0000","last_change":"2024-04-25T10:11:23.501608+0000","last_active":"2024-04-25T10:11:24.478106+0000","last_peered":"2024-04-25T10:11:24.478106+0000","last_clean":"2024-04-25T10:11:24.478106+0000","last_became_active":"2024-04-25T10:11:23.500874+0000","last_became_peered":"2024-04-25T10:11:23.500874+0000","last_unstale":"2024-04-25T10:11:24.478106+0000","last_undegraded":"2024-04-25T10:11:24.478106+0000","last_fullsized":"2024-04-25T10:11:24.478106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:39.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474639+0000","last_change":"2024-04-25T10:11:23.504056+0000","last_active":"2024-04-25T10:11:24.474639+0000","last_peered":"2024-04-25T10:11:24.474639+0000","last_clean":"2024-04-25T10:11:24.474639+0000","last_became_active":"2024-04-25T10:11:23.503801+0000","last_became_peered":"2024-04-25T10:11:23.503801+0000","last_unstale":"2024-04-25T10:11:24.474639+0000","last_undegraded":"2024-04-25T10:11:24.474639+0000","last_fullsized":"2024-04-25T10:11:24.474639+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:32.459765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521718+0000","last_change":"2024-04-25T10:11:23.490475+0000","last_active":"2024-04-25T10:11:24.521718+0000","last_peered":"2024-04-25T10:11:24.521718+0000","last_clean":"2024-04-25T10:11:24.521718+0000","last_became_active":"2024-04-25T10:11:23.489746+0000","last_became_peered":"2024-04-25T10:11:23.489746+0000","last_unstale":"2024-04-25T10:11:24.521718+0000","last_undegraded":"2024-04-25T10:11:24.521718+0000","last_fullsized":"2024-04-25T10:11:24.521718+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:41.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479424+0000","last_change":"2024-04-25T10:11:23.503039+0000","last_active":"2024-04-25T10:11:24.479424+0000","last_peered":"2024-04-25T10:11:24.479424+0000","last_clean":"2024-04-25T10:11:24.479424+0000","last_became_active":"2024-04-25T10:11:23.502130+0000","last_became_peered":"2024-04-25T10:11:23.502130+0000","last_unstale":"2024-04-25T10:11:24.479424+0000","last_undegraded":"2024-04-25T10:11:24.479424+0000","last_fullsized":"2024-04-25T10:11:24.479424+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966420+0000","last_change":"2024-04-25T10:11:23.501635+0000","last_active":"2024-04-25T10:11:25.966420+0000","last_peered":"2024-04-25T10:11:25.966420+0000","last_clean":"2024-04-25T10:11:25.966420+0000","last_became_active":"2024-04-25T10:11:23.501356+0000","last_became_peered":"2024-04-25T10:11:23.501356+0000","last_unstale":"2024-04-25T10:11:25.966420+0000","last_undegraded":"2024-04-25T10:11:25.966420+0000","last_fullsized":"2024-04-25T10:11:25.966420+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:29.491438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968463+0000","last_change":"2024-04-25T10:11:23.499753+0000","last_active":"2024-04-25T10:11:25.968463+0000","last_peered":"2024-04-25T10:11:25.968463+0000","last_clean":"2024-04-25T10:11:25.968463+0000","last_became_active":"2024-04-25T10:11:23.499460+0000","last_became_peered":"2024-04-25T10:11:23.499460+0000","last_unstale":"2024-04-25T10:11:25.968463+0000","last_undegraded":"2024-04-25T10:11:25.968463+0000","last_fullsized":"2024-04-25T10:11:25.968463+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:34.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950652+0000","last_change":"2024-04-25T10:11:23.500199+0000","last_active":"2024-04-25T10:11:25.950652+0000","last_peered":"2024-04-25T10:11:25.950652+0000","last_clean":"2024-04-25T10:11:25.950652+0000","last_became_active":"2024-04-25T10:11:23.499904+0000","last_became_peered":"2024-04-25T10:11:23.499904+0000","last_unstale":"2024-04-25T10:11:25.950652+0000","last_undegraded":"2024-04-25T10:11:25.950652+0000","last_fullsized":"2024-04-25T10:11:25.950652+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:41.977671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479200+0000","last_change":"2024-04-25T10:11:23.494112+0000","last_active":"2024-04-25T10:11:24.479200+0000","last_peered":"2024-04-25T10:11:24.479200+0000","last_clean":"2024-04-25T10:11:24.479200+0000","last_became_active":"2024-04-25T10:11:23.493781+0000","last_became_peered":"2024-04-25T10:11:23.493781+0000","last_unstale":"2024-04-25T10:11:24.479200+0000","last_undegraded":"2024-04-25T10:11:24.479200+0000","last_fullsized":"2024-04-25T10:11:24.479200+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:37.491564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520698+0000","last_change":"2024-04-25T10:11:23.504171+0000","last_active":"2024-04-25T10:11:24.520698+0000","last_peered":"2024-04-25T10:11:24.520698+0000","last_clean":"2024-04-25T10:11:24.520698+0000","last_became_active":"2024-04-25T10:11:23.501873+0000","last_became_peered":"2024-04-25T10:11:23.501873+0000","last_unstale":"2024-04-25T10:11:24.520698+0000","last_undegraded":"2024-04-25T10:11:24.520698+0000","last_fullsized":"2024-04-25T10:11:24.520698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:13.126891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864462+0000","last_change":"2024-04-25T10:11:23.496202+0000","last_active":"2024-04-25T10:11:24.864462+0000","last_peered":"2024-04-25T10:11:24.864462+0000","last_clean":"2024-04-25T10:11:24.864462+0000","last_became_active":"2024-04-25T10:11:23.495923+0000","last_became_peered":"2024-04-25T10:11:23.495923+0000","last_unstale":"2024-04-25T10:11:24.864462+0000","last_undegraded":"2024-04-25T10:11:24.864462+0000","last_fullsized":"2024-04-25T10:11:24.864462+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:26.229309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864558+0000","last_change":"2024-04-25T10:11:23.500336+0000","last_active":"2024-04-25T10:11:24.864558+0000","last_peered":"2024-04-25T10:11:24.864558+0000","last_clean":"2024-04-25T10:11:24.864558+0000","last_became_active":"2024-04-25T10:11:23.498911+0000","last_became_peered":"2024-04-25T10:11:23.498911+0000","last_unstale":"2024-04-25T10:11:24.864558+0000","last_undegraded":"2024-04-25T10:11:24.864558+0000","last_fullsized":"2024-04-25T10:11:24.864558+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.192265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"97'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480329+0000","last_change":"2024-04-25T10:11:22.465629+0000","last_active":"2024-04-25T10:11:24.480329+0000","last_peered":"2024-04-25T10:11:24.480329+0000","last_clean":"2024-04-25T10:11:24.480329+0000","last_became_active":"2024-04-25T10:11:22.465239+0000","last_became_peered":"2024-04-25T10:11:22.465239+0000","last_unstale":"2024-04-25T10:11:24.480329+0000","last_undegraded":"2024-04-25T10:11:24.480329+0000","last_fullsized":"2024-04-25T10:11:24.480329+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:29.898591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864201+0000","last_change":"2024-04-25T10:11:23.492228+0000","last_active":"2024-04-25T10:11:24.864201+0000","last_peered":"2024-04-25T10:11:24.864201+0000","last_clean":"2024-04-25T10:11:24.864201+0000","last_became_active":"2024-04-25T10:11:23.491931+0000","last_became_peered":"2024-04-25T10:11:23.491931+0000","last_unstale":"2024-04-25T10:11:24.864201+0000","last_undegraded":"2024-04-25T10:11:24.864201+0000","last_fullsized":"2024-04-25T10:11:24.864201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:07.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479937+0000","last_change":"2024-04-25T10:11:22.467541+0000","last_active":"2024-04-25T10:11:24.479937+0000","last_peered":"2024-04-25T10:11:24.479937+0000","last_clean":"2024-04-25T10:11:24.479937+0000","last_became_active":"2024-04-25T10:11:22.467211+0000","last_became_peered":"2024-04-25T10:11:22.467211+0000","last_unstale":"2024-04-25T10:11:24.479937+0000","last_undegraded":"2024-04-25T10:11:24.479937+0000","last_fullsized":"2024-04-25T10:11:24.479937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:29.678377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950850+0000","last_change":"2024-04-25T10:11:23.505136+0000","last_active":"2024-04-25T10:11:25.950850+0000","last_peered":"2024-04-25T10:11:25.950850+0000","last_clean":"2024-04-25T10:11:25.950850+0000","last_became_active":"2024-04-25T10:11:23.504848+0000","last_became_peered":"2024-04-25T10:11:23.504848+0000","last_unstale":"2024-04-25T10:11:25.950850+0000","last_undegraded":"2024-04-25T10:11:25.950850+0000","last_fullsized":"2024-04-25T10:11:25.950850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:34.764490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950348+0000","last_change":"2024-04-25T10:11:22.474715+0000","last_active":"2024-04-25T10:11:25.950348+0000","last_peered":"2024-04-25T10:11:25.950348+0000","last_clean":"2024-04-25T10:11:25.950348+0000","last_became_active":"2024-04-25T10:11:22.472397+0000","last_became_peered":"2024-04-25T10:11:22.472397+0000","last_unstale":"2024-04-25T10:11:25.950348+0000","last_undegraded":"2024-04-25T10:11:25.950348+0000","last_fullsized":"2024-04-25T10:11:25.950348+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:03.107761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863968+0000","last_change":"2024-04-25T10:11:23.486363+0000","last_active":"2024-04-25T10:11:24.863968+0000","last_peered":"2024-04-25T10:11:24.863968+0000","last_clean":"2024-04-25T10:11:24.863968+0000","last_became_active":"2024-04-25T10:11:23.485997+0000","last_became_peered":"2024-04-25T10:11:23.485997+0000","last_unstale":"2024-04-25T10:11:24.863968+0000","last_undegraded":"2024-04-25T10:11:24.863968+0000","last_fullsized":"2024-04-25T10:11:24.863968+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:39.127870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480923+0000","last_change":"2024-04-25T10:11:22.470977+0000","last_active":"2024-04-25T10:11:24.480923+0000","last_peered":"2024-04-25T10:11:24.480923+0000","last_clean":"2024-04-25T10:11:24.480923+0000","last_became_active":"2024-04-25T10:11:22.469808+0000","last_became_peered":"2024-04-25T10:11:22.469808+0000","last_unstale":"2024-04-25T10:11:24.480923+0000","last_undegraded":"2024-04-25T10:11:24.480923+0000","last_fullsized":"2024-04-25T10:11:24.480923+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:33.243743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968174+0000","last_change":"2024-04-25T10:11:23.514752+0000","last_active":"2024-04-25T10:11:25.968174+0000","last_peered":"2024-04-25T10:11:25.968174+0000","last_clean":"2024-04-25T10:11:25.968174+0000","last_became_active":"2024-04-25T10:11:23.514468+0000","last_became_peered":"2024-04-25T10:11:23.514468+0000","last_unstale":"2024-04-25T10:11:25.968174+0000","last_undegraded":"2024-04-25T10:11:25.968174+0000","last_fullsized":"2024-04-25T10:11:25.968174+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.677097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478648+0000","last_change":"2024-04-25T10:11:22.462923+0000","last_active":"2024-04-25T10:11:24.478648+0000","last_peered":"2024-04-25T10:11:24.478648+0000","last_clean":"2024-04-25T10:11:24.478648+0000","last_became_active":"2024-04-25T10:11:22.462615+0000","last_became_peered":"2024-04-25T10:11:22.462615+0000","last_unstale":"2024-04-25T10:11:24.478648+0000","last_undegraded":"2024-04-25T10:11:24.478648+0000","last_fullsized":"2024-04-25T10:11:24.478648+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:13.272502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479515+0000","last_change":"2024-04-25T10:11:23.489900+0000","last_active":"2024-04-25T10:11:24.479515+0000","last_peered":"2024-04-25T10:11:24.479515+0000","last_clean":"2024-04-25T10:11:24.479515+0000","last_became_active":"2024-04-25T10:11:23.489383+0000","last_became_peered":"2024-04-25T10:11:23.489383+0000","last_unstale":"2024-04-25T10:11:24.479515+0000","last_undegraded":"2024-04-25T10:11:24.479515+0000","last_fullsized":"2024-04-25T10:11:24.479515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.937312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967741+0000","last_change":"2024-04-25T10:11:22.466216+0000","last_active":"2024-04-25T10:11:25.967741+0000","last_peered":"2024-04-25T10:11:25.967741+0000","last_clean":"2024-04-25T10:11:25.967741+0000","last_became_active":"2024-04-25T10:11:22.465096+0000","last_became_peered":"2024-04-25T10:11:22.465096+0000","last_unstale":"2024-04-25T10:11:25.967741+0000","last_undegraded":"2024-04-25T10:11:25.967741+0000","last_fullsized":"2024-04-25T10:11:25.967741+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:33.243743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863752+0000","last_change":"2024-04-25T10:11:23.502474+0000","last_active":"2024-04-25T10:11:24.863752+0000","last_peered":"2024-04-25T10:11:24.863752+0000","last_clean":"2024-04-25T10:11:24.863752+0000","last_became_active":"2024-04-25T10:11:23.502027+0000","last_became_peered":"2024-04-25T10:11:23.502027+0000","last_unstale":"2024-04-25T10:11:24.863752+0000","last_undegraded":"2024-04-25T10:11:24.863752+0000","last_fullsized":"2024-04-25T10:11:24.863752+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:48.650402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474679+0000","last_change":"2024-04-25T10:11:22.470683+0000","last_active":"2024-04-25T10:11:24.474679+0000","last_peered":"2024-04-25T10:11:24.474679+0000","last_clean":"2024-04-25T10:11:24.474679+0000","last_became_active":"2024-04-25T10:11:22.470361+0000","last_became_peered":"2024-04-25T10:11:22.470361+0000","last_unstale":"2024-04-25T10:11:24.474679+0000","last_undegraded":"2024-04-25T10:11:24.474679+0000","last_fullsized":"2024-04-25T10:11:24.474679+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:17.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967639+0000","last_change":"2024-04-25T10:11:23.508721+0000","last_active":"2024-04-25T10:11:25.967639+0000","last_peered":"2024-04-25T10:11:25.967639+0000","last_clean":"2024-04-25T10:11:25.967639+0000","last_became_active":"2024-04-25T10:11:23.508450+0000","last_became_peered":"2024-04-25T10:11:23.508450+0000","last_unstale":"2024-04-25T10:11:25.967639+0000","last_undegraded":"2024-04-25T10:11:25.967639+0000","last_fullsized":"2024-04-25T10:11:25.967639+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.972961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.949937+0000","last_change":"2024-04-25T10:11:22.470874+0000","last_active":"2024-04-25T10:11:25.949937+0000","last_peered":"2024-04-25T10:11:25.949937+0000","last_clean":"2024-04-25T10:11:25.949937+0000","last_became_active":"2024-04-25T10:11:22.470265+0000","last_became_peered":"2024-04-25T10:11:22.470265+0000","last_unstale":"2024-04-25T10:11:25.949937+0000","last_undegraded":"2024-04-25T10:11:25.949937+0000","last_fullsized":"2024-04-25T10:11:25.949937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:23.973293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479393+0000","last_change":"2024-04-25T10:11:23.504435+0000","last_active":"2024-04-25T10:11:24.479393+0000","last_peered":"2024-04-25T10:11:24.479393+0000","last_clean":"2024-04-25T10:11:24.479393+0000","last_became_active":"2024-04-25T10:11:23.504141+0000","last_became_peered":"2024-04-25T10:11:23.504141+0000","last_unstale":"2024-04-25T10:11:24.479393+0000","last_undegraded":"2024-04-25T10:11:24.479393+0000","last_fullsized":"2024-04-25T10:11:24.479393+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:00.443531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521846+0000","last_change":"2024-04-25T10:11:22.473533+0000","last_active":"2024-04-25T10:11:24.521846+0000","last_peered":"2024-04-25T10:11:24.521846+0000","last_clean":"2024-04-25T10:11:24.521846+0000","last_became_active":"2024-04-25T10:11:22.473208+0000","last_became_peered":"2024-04-25T10:11:22.473208+0000","last_unstale":"2024-04-25T10:11:24.521846+0000","last_undegraded":"2024-04-25T10:11:24.521846+0000","last_fullsized":"2024-04-25T10:11:24.521846+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.661232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950916+0000","last_change":"2024-04-25T10:11:23.500410+0000","last_active":"2024-04-25T10:11:25.950916+0000","last_peered":"2024-04-25T10:11:25.950916+0000","last_clean":"2024-04-25T10:11:25.950916+0000","last_became_active":"2024-04-25T10:11:23.499418+0000","last_became_peered":"2024-04-25T10:11:23.499418+0000","last_unstale":"2024-04-25T10:11:25.950916+0000","last_undegraded":"2024-04-25T10:11:25.950916+0000","last_fullsized":"2024-04-25T10:11:25.950916+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.222396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475145+0000","last_change":"2024-04-25T10:11:22.465237+0000","last_active":"2024-04-25T10:11:24.475145+0000","last_peered":"2024-04-25T10:11:24.475145+0000","last_clean":"2024-04-25T10:11:24.475145+0000","last_became_active":"2024-04-25T10:11:22.464506+0000","last_became_peered":"2024-04-25T10:11:22.464506+0000","last_unstale":"2024-04-25T10:11:24.475145+0000","last_undegraded":"2024-04-25T10:11:24.475145+0000","last_fullsized":"2024-04-25T10:11:24.475145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:11.893976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475166+0000","last_change":"2024-04-25T10:11:23.512957+0000","last_active":"2024-04-25T10:11:24.475166+0000","last_peered":"2024-04-25T10:11:24.475166+0000","last_clean":"2024-04-25T10:11:24.475166+0000","last_became_active":"2024-04-25T10:11:23.512233+0000","last_became_peered":"2024-04-25T10:11:23.512233+0000","last_unstale":"2024-04-25T10:11:24.475166+0000","last_undegraded":"2024-04-25T10:11:24.475166+0000","last_fullsized":"2024-04-25T10:11:24.475166+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:41.977671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"97'3","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479786+0000","last_change":"2024-04-25T10:11:22.461965+0000","last_active":"2024-04-25T10:11:24.479786+0000","last_peered":"2024-04-25T10:11:24.479786+0000","last_clean":"2024-04-25T10:11:24.479786+0000","last_became_active":"2024-04-25T10:11:22.461218+0000","last_became_peered":"2024-04-25T10:11:22.461218+0000","last_unstale":"2024-04-25T10:11:24.479786+0000","last_undegraded":"2024-04-25T10:11:24.479786+0000","last_fullsized":"2024-04-25T10:11:24.479786+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:33.930030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968031+0000","last_change":"2024-04-25T10:11:23.499871+0000","last_active":"2024-04-25T10:11:25.968031+0000","last_peered":"2024-04-25T10:11:25.968031+0000","last_clean":"2024-04-25T10:11:25.968031+0000","last_became_active":"2024-04-25T10:11:23.499555+0000","last_became_peered":"2024-04-25T10:11:23.499555+0000","last_unstale":"2024-04-25T10:11:25.968031+0000","last_undegraded":"2024-04-25T10:11:25.968031+0000","last_fullsized":"2024-04-25T10:11:25.968031+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:46.821062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"97'6","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864178+0000","last_change":"2024-04-25T10:11:22.461868+0000","last_active":"2024-04-25T10:11:24.864178+0000","last_peered":"2024-04-25T10:11:24.864178+0000","last_clean":"2024-04-25T10:11:24.864178+0000","last_became_active":"2024-04-25T10:11:22.461381+0000","last_became_peered":"2024-04-25T10:11:22.461381+0000","last_unstale":"2024-04-25T10:11:24.864178+0000","last_undegraded":"2024-04-25T10:11:24.864178+0000","last_fullsized":"2024-04-25T10:11:24.864178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:33.748625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967812+0000","last_change":"2024-04-25T10:11:23.496310+0000","last_active":"2024-04-25T10:11:25.967812+0000","last_peered":"2024-04-25T10:11:25.967812+0000","last_clean":"2024-04-25T10:11:25.967812+0000","last_became_active":"2024-04-25T10:11:23.495975+0000","last_became_peered":"2024-04-25T10:11:23.495975+0000","last_unstale":"2024-04-25T10:11:25.967812+0000","last_undegraded":"2024-04-25T10:11:25.967812+0000","last_fullsized":"2024-04-25T10:11:25.967812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:46.519451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950177+0000","last_change":"2024-04-25T10:11:22.474187+0000","last_active":"2024-04-25T10:11:25.950177+0000","last_peered":"2024-04-25T10:11:25.950177+0000","last_clean":"2024-04-25T10:11:25.950177+0000","last_became_active":"2024-04-25T10:11:22.473866+0000","last_became_peered":"2024-04-25T10:11:22.473866+0000","last_unstale":"2024-04-25T10:11:25.950177+0000","last_undegraded":"2024-04-25T10:11:25.950177+0000","last_fullsized":"2024-04-25T10:11:25.950177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:17.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478658+0000","last_change":"2024-04-25T10:11:23.498466+0000","last_active":"2024-04-25T10:11:24.478658+0000","last_peered":"2024-04-25T10:11:24.478658+0000","last_clean":"2024-04-25T10:11:24.478658+0000","last_became_active":"2024-04-25T10:11:23.497025+0000","last_became_peered":"2024-04-25T10:11:23.497025+0000","last_unstale":"2024-04-25T10:11:24.478658+0000","last_undegraded":"2024-04-25T10:11:24.478658+0000","last_fullsized":"2024-04-25T10:11:24.478658+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:39.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863851+0000","last_change":"2024-04-25T10:11:22.469146+0000","last_active":"2024-04-25T10:11:24.863851+0000","last_peered":"2024-04-25T10:11:24.863851+0000","last_clean":"2024-04-25T10:11:24.863851+0000","last_became_active":"2024-04-25T10:11:22.468838+0000","last_became_peered":"2024-04-25T10:11:22.468838+0000","last_unstale":"2024-04-25T10:11:24.863851+0000","last_undegraded":"2024-04-25T10:11:24.863851+0000","last_fullsized":"2024-04-25T10:11:24.863851+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:46.341223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965886+0000","last_change":"2024-04-25T10:11:23.504259+0000","last_active":"2024-04-25T10:11:25.965886+0000","last_peered":"2024-04-25T10:11:25.965886+0000","last_clean":"2024-04-25T10:11:25.965886+0000","last_became_active":"2024-04-25T10:11:23.503980+0000","last_became_peered":"2024-04-25T10:11:23.503980+0000","last_unstale":"2024-04-25T10:11:25.965886+0000","last_undegraded":"2024-04-25T10:11:25.965886+0000","last_fullsized":"2024-04-25T10:11:25.965886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:27.440246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967382+0000","last_change":"2024-04-25T10:11:22.470404+0000","last_active":"2024-04-25T10:11:25.967382+0000","last_peered":"2024-04-25T10:11:25.967382+0000","last_clean":"2024-04-25T10:11:25.967382+0000","last_became_active":"2024-04-25T10:11:22.470111+0000","last_became_peered":"2024-04-25T10:11:22.470111+0000","last_unstale":"2024-04-25T10:11:25.967382+0000","last_undegraded":"2024-04-25T10:11:25.967382+0000","last_fullsized":"2024-04-25T10:11:25.967382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:32.968772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966320+0000","last_change":"2024-04-25T10:11:23.500138+0000","last_active":"2024-04-25T10:11:25.966320+0000","last_peered":"2024-04-25T10:11:25.966320+0000","last_clean":"2024-04-25T10:11:25.966320+0000","last_became_active":"2024-04-25T10:11:23.499378+0000","last_became_peered":"2024-04-25T10:11:23.499378+0000","last_unstale":"2024-04-25T10:11:25.966320+0000","last_undegraded":"2024-04-25T10:11:25.966320+0000","last_fullsized":"2024-04-25T10:11:25.966320+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.096148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"97'8","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967909+0000","last_change":"2024-04-25T10:11:22.465509+0000","last_active":"2024-04-25T10:11:25.967909+0000","last_peered":"2024-04-25T10:11:25.967909+0000","last_clean":"2024-04-25T10:11:25.967909+0000","last_became_active":"2024-04-25T10:11:22.465243+0000","last_became_peered":"2024-04-25T10:11:22.465243+0000","last_unstale":"2024-04-25T10:11:25.967909+0000","last_undegraded":"2024-04-25T10:11:25.967909+0000","last_fullsized":"2024-04-25T10:11:25.967909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:29.898591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864035+0000","last_change":"2024-04-25T10:11:23.500428+0000","last_active":"2024-04-25T10:11:24.864035+0000","last_peered":"2024-04-25T10:11:24.864035+0000","last_clean":"2024-04-25T10:11:24.864035+0000","last_became_active":"2024-04-25T10:11:23.500144+0000","last_became_peered":"2024-04-25T10:11:23.500144+0000","last_unstale":"2024-04-25T10:11:24.864035+0000","last_undegraded":"2024-04-25T10:11:24.864035+0000","last_fullsized":"2024-04-25T10:11:24.864035+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:30.247684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480875+0000","last_change":"2024-04-25T10:11:22.471844+0000","last_active":"2024-04-25T10:11:24.480875+0000","last_peered":"2024-04-25T10:11:24.480875+0000","last_clean":"2024-04-25T10:11:24.480875+0000","last_became_active":"2024-04-25T10:11:22.471304+0000","last_became_peered":"2024-04-25T10:11:22.471304+0000","last_unstale":"2024-04-25T10:11:24.480875+0000","last_undegraded":"2024-04-25T10:11:24.480875+0000","last_fullsized":"2024-04-25T10:11:24.480875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:29.865607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480738+0000","last_change":"2024-04-25T10:11:23.487951+0000","last_active":"2024-04-25T10:11:24.480738+0000","last_peered":"2024-04-25T10:11:24.480738+0000","last_clean":"2024-04-25T10:11:24.480738+0000","last_became_active":"2024-04-25T10:11:23.487535+0000","last_became_peered":"2024-04-25T10:11:23.487535+0000","last_unstale":"2024-04-25T10:11:24.480738+0000","last_undegraded":"2024-04-25T10:11:24.480738+0000","last_fullsized":"2024-04-25T10:11:24.480738+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950349+0000","last_change":"2024-04-25T10:11:22.476577+0000","last_active":"2024-04-25T10:11:25.950349+0000","last_peered":"2024-04-25T10:11:25.950349+0000","last_clean":"2024-04-25T10:11:25.950349+0000","last_became_active":"2024-04-25T10:11:22.476222+0000","last_became_peered":"2024-04-25T10:11:22.476222+0000","last_unstale":"2024-04-25T10:11:25.950349+0000","last_undegraded":"2024-04-25T10:11:25.950349+0000","last_fullsized":"2024-04-25T10:11:25.950349+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:55.296806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479787+0000","last_change":"2024-04-25T10:11:23.495714+0000","last_active":"2024-04-25T10:11:24.479787+0000","last_peered":"2024-04-25T10:11:24.479787+0000","last_clean":"2024-04-25T10:11:24.479787+0000","last_became_active":"2024-04-25T10:11:23.495066+0000","last_became_peered":"2024-04-25T10:11:23.495066+0000","last_unstale":"2024-04-25T10:11:24.479787+0000","last_undegraded":"2024-04-25T10:11:24.479787+0000","last_fullsized":"2024-04-25T10:11:24.479787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.091532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479884+0000","last_change":"2024-04-25T10:11:22.470350+0000","last_active":"2024-04-25T10:11:24.479884+0000","last_peered":"2024-04-25T10:11:24.479884+0000","last_clean":"2024-04-25T10:11:24.479884+0000","last_became_active":"2024-04-25T10:11:22.470032+0000","last_became_peered":"2024-04-25T10:11:22.470032+0000","last_unstale":"2024-04-25T10:11:24.479884+0000","last_undegraded":"2024-04-25T10:11:24.479884+0000","last_fullsized":"2024-04-25T10:11:24.479884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.503976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864599+0000","last_change":"2024-04-25T10:11:23.501944+0000","last_active":"2024-04-25T10:11:24.864599+0000","last_peered":"2024-04-25T10:11:24.864599+0000","last_clean":"2024-04-25T10:11:24.864599+0000","last_became_active":"2024-04-25T10:11:23.501477+0000","last_became_peered":"2024-04-25T10:11:23.501477+0000","last_unstale":"2024-04-25T10:11:24.864599+0000","last_undegraded":"2024-04-25T10:11:24.864599+0000","last_fullsized":"2024-04-25T10:11:24.864599+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.961651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520557+0000","last_change":"2024-04-25T10:11:22.473753+0000","last_active":"2024-04-25T10:11:24.520557+0000","last_peered":"2024-04-25T10:11:24.520557+0000","last_clean":"2024-04-25T10:11:24.520557+0000","last_became_active":"2024-04-25T10:11:22.473434+0000","last_became_peered":"2024-04-25T10:11:22.473434+0000","last_unstale":"2024-04-25T10:11:24.520557+0000","last_undegraded":"2024-04-25T10:11:24.520557+0000","last_fullsized":"2024-04-25T10:11:24.520557+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:53.765476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520767+0000","last_change":"2024-04-25T10:11:23.496038+0000","last_active":"2024-04-25T10:11:24.520767+0000","last_peered":"2024-04-25T10:11:24.520767+0000","last_clean":"2024-04-25T10:11:24.520767+0000","last_became_active":"2024-04-25T10:11:23.495699+0000","last_became_peered":"2024-04-25T10:11:23.495699+0000","last_unstale":"2024-04-25T10:11:24.520767+0000","last_undegraded":"2024-04-25T10:11:24.520767+0000","last_fullsized":"2024-04-25T10:11:24.520767+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:33.466669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966036+0000","last_change":"2024-04-25T10:11:22.462653+0000","last_active":"2024-04-25T10:11:25.966036+0000","last_peered":"2024-04-25T10:11:25.966036+0000","last_clean":"2024-04-25T10:11:25.966036+0000","last_became_active":"2024-04-25T10:11:22.462355+0000","last_became_peered":"2024-04-25T10:11:22.462355+0000","last_unstale":"2024-04-25T10:11:25.966036+0000","last_undegraded":"2024-04-25T10:11:25.966036+0000","last_fullsized":"2024-04-25T10:11:25.966036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:49.454297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521025+0000","last_change":"2024-04-25T10:11:23.504455+0000","last_active":"2024-04-25T10:11:24.521025+0000","last_peered":"2024-04-25T10:11:24.521025+0000","last_clean":"2024-04-25T10:11:24.521025+0000","last_became_active":"2024-04-25T10:11:23.504135+0000","last_became_peered":"2024-04-25T10:11:23.504135+0000","last_unstale":"2024-04-25T10:11:24.521025+0000","last_undegraded":"2024-04-25T10:11:24.521025+0000","last_fullsized":"2024-04-25T10:11:24.521025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.347401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520952+0000","last_change":"2024-04-25T10:11:22.476111+0000","last_active":"2024-04-25T10:11:24.520952+0000","last_peered":"2024-04-25T10:11:24.520952+0000","last_clean":"2024-04-25T10:11:24.520952+0000","last_became_active":"2024-04-25T10:11:22.474623+0000","last_became_peered":"2024-04-25T10:11:22.474623+0000","last_unstale":"2024-04-25T10:11:24.520952+0000","last_undegraded":"2024-04-25T10:11:24.520952+0000","last_fullsized":"2024-04-25T10:11:24.520952+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:21.062051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863947+0000","last_change":"2024-04-25T10:11:23.490610+0000","last_active":"2024-04-25T10:11:24.863947+0000","last_peered":"2024-04-25T10:11:24.863947+0000","last_clean":"2024-04-25T10:11:24.863947+0000","last_became_active":"2024-04-25T10:11:23.489790+0000","last_became_peered":"2024-04-25T10:11:23.489790+0000","last_unstale":"2024-04-25T10:11:24.863947+0000","last_undegraded":"2024-04-25T10:11:24.863947+0000","last_fullsized":"2024-04-25T10:11:24.863947+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965445+0000","last_change":"2024-04-25T10:11:22.471275+0000","last_active":"2024-04-25T10:11:25.965445+0000","last_peered":"2024-04-25T10:11:25.965445+0000","last_clean":"2024-04-25T10:11:25.965445+0000","last_became_active":"2024-04-25T10:11:22.471004+0000","last_became_peered":"2024-04-25T10:11:22.471004+0000","last_unstale":"2024-04-25T10:11:25.965445+0000","last_undegraded":"2024-04-25T10:11:25.965445+0000","last_fullsized":"2024-04-25T10:11:25.965445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.869983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480482+0000","last_change":"2024-04-25T10:11:23.498595+0000","last_active":"2024-04-25T10:11:24.480482+0000","last_peered":"2024-04-25T10:11:24.480482+0000","last_clean":"2024-04-25T10:11:24.480482+0000","last_became_active":"2024-04-25T10:11:23.497684+0000","last_became_peered":"2024-04-25T10:11:23.497684+0000","last_unstale":"2024-04-25T10:11:24.480482+0000","last_undegraded":"2024-04-25T10:11:24.480482+0000","last_fullsized":"2024-04-25T10:11:24.480482+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:00.443531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950639+0000","last_change":"2024-04-25T10:11:22.477558+0000","last_active":"2024-04-25T10:11:25.950639+0000","last_peered":"2024-04-25T10:11:25.950639+0000","last_clean":"2024-04-25T10:11:25.950639+0000","last_became_active":"2024-04-25T10:11:22.476031+0000","last_became_peered":"2024-04-25T10:11:22.476031+0000","last_unstale":"2024-04-25T10:11:25.950639+0000","last_undegraded":"2024-04-25T10:11:25.950639+0000","last_fullsized":"2024-04-25T10:11:25.950639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.891229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864672+0000","last_change":"2024-04-25T10:11:23.498208+0000","last_active":"2024-04-25T10:11:24.864672+0000","last_peered":"2024-04-25T10:11:24.864672+0000","last_clean":"2024-04-25T10:11:24.864672+0000","last_became_active":"2024-04-25T10:11:23.497917+0000","last_became_peered":"2024-04-25T10:11:23.497917+0000","last_unstale":"2024-04-25T10:11:24.864672+0000","last_undegraded":"2024-04-25T10:11:24.864672+0000","last_fullsized":"2024-04-25T10:11:24.864672+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:45.774608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"98'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480212+0000","last_change":"2024-04-25T10:11:22.476964+0000","last_active":"2024-04-25T10:11:24.480212+0000","last_peered":"2024-04-25T10:11:24.480212+0000","last_clean":"2024-04-25T10:11:24.480212+0000","last_became_active":"2024-04-25T10:11:22.476572+0000","last_became_peered":"2024-04-25T10:11:22.476572+0000","last_unstale":"2024-04-25T10:11:24.480212+0000","last_undegraded":"2024-04-25T10:11:24.480212+0000","last_fullsized":"2024-04-25T10:11:24.480212+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.402891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863704+0000","last_change":"2024-04-25T10:11:23.494385+0000","last_active":"2024-04-25T10:11:24.863704+0000","last_peered":"2024-04-25T10:11:24.863704+0000","last_clean":"2024-04-25T10:11:24.863704+0000","last_became_active":"2024-04-25T10:11:23.494070+0000","last_became_peered":"2024-04-25T10:11:23.494070+0000","last_unstale":"2024-04-25T10:11:24.863704+0000","last_undegraded":"2024-04-25T10:11:24.863704+0000","last_fullsized":"2024-04-25T10:11:24.863704+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:10.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479943+0000","last_change":"2024-04-25T10:11:22.463847+0000","last_active":"2024-04-25T10:11:24.479943+0000","last_peered":"2024-04-25T10:11:24.479943+0000","last_clean":"2024-04-25T10:11:24.479943+0000","last_became_active":"2024-04-25T10:11:22.463559+0000","last_became_peered":"2024-04-25T10:11:22.463559+0000","last_unstale":"2024-04-25T10:11:24.479943+0000","last_undegraded":"2024-04-25T10:11:24.479943+0000","last_fullsized":"2024-04-25T10:11:24.479943+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.936278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521840+0000","last_change":"2024-04-25T10:11:23.499509+0000","last_active":"2024-04-25T10:11:24.521840+0000","last_peered":"2024-04-25T10:11:24.521840+0000","last_clean":"2024-04-25T10:11:24.521840+0000","last_became_active":"2024-04-25T10:11:23.498796+0000","last_became_peered":"2024-04-25T10:11:23.498796+0000","last_unstale":"2024-04-25T10:11:24.521840+0000","last_undegraded":"2024-04-25T10:11:24.521840+0000","last_fullsized":"2024-04-25T10:11:24.521840+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:12.174658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480989+0000","last_change":"2024-04-25T10:11:22.458990+0000","last_active":"2024-04-25T10:11:24.480989+0000","last_peered":"2024-04-25T10:11:24.480989+0000","last_clean":"2024-04-25T10:11:24.480989+0000","last_became_active":"2024-04-25T10:11:22.458519+0000","last_became_peered":"2024-04-25T10:11:22.458519+0000","last_unstale":"2024-04-25T10:11:24.480989+0000","last_undegraded":"2024-04-25T10:11:24.480989+0000","last_fullsized":"2024-04-25T10:11:24.480989+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:28.483485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966106+0000","last_change":"2024-04-25T10:11:23.495054+0000","last_active":"2024-04-25T10:11:25.966106+0000","last_peered":"2024-04-25T10:11:25.966106+0000","last_clean":"2024-04-25T10:11:25.966106+0000","last_became_active":"2024-04-25T10:11:23.494672+0000","last_became_peered":"2024-04-25T10:11:23.494672+0000","last_unstale":"2024-04-25T10:11:25.966106+0000","last_undegraded":"2024-04-25T10:11:25.966106+0000","last_fullsized":"2024-04-25T10:11:25.966106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.959511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864095+0000","last_change":"2024-04-25T10:11:22.469251+0000","last_active":"2024-04-25T10:11:24.864095+0000","last_peered":"2024-04-25T10:11:24.864095+0000","last_clean":"2024-04-25T10:11:24.864095+0000","last_became_active":"2024-04-25T10:11:22.468896+0000","last_became_peered":"2024-04-25T10:11:22.468896+0000","last_unstale":"2024-04-25T10:11:24.864095+0000","last_undegraded":"2024-04-25T10:11:24.864095+0000","last_fullsized":"2024-04-25T10:11:24.864095+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:40.961806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474640+0000","last_change":"2024-04-25T10:11:23.511617+0000","last_active":"2024-04-25T10:11:24.474640+0000","last_peered":"2024-04-25T10:11:24.474640+0000","last_clean":"2024-04-25T10:11:24.474640+0000","last_became_active":"2024-04-25T10:11:23.511352+0000","last_became_peered":"2024-04-25T10:11:23.511352+0000","last_unstale":"2024-04-25T10:11:24.474640+0000","last_undegraded":"2024-04-25T10:11:24.474640+0000","last_fullsized":"2024-04-25T10:11:24.474640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.588960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521129+0000","last_change":"2024-04-25T10:11:22.472035+0000","last_active":"2024-04-25T10:11:24.521129+0000","last_peered":"2024-04-25T10:11:24.521129+0000","last_clean":"2024-04-25T10:11:24.521129+0000","last_became_active":"2024-04-25T10:11:22.471686+0000","last_became_peered":"2024-04-25T10:11:22.471686+0000","last_unstale":"2024-04-25T10:11:24.521129+0000","last_undegraded":"2024-04-25T10:11:24.521129+0000","last_fullsized":"2024-04-25T10:11:24.521129+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:45.805197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479618+0000","last_change":"2024-04-25T10:11:23.503580+0000","last_active":"2024-04-25T10:11:24.479618+0000","last_peered":"2024-04-25T10:11:24.479618+0000","last_clean":"2024-04-25T10:11:24.479618+0000","last_became_active":"2024-04-25T10:11:23.503299+0000","last_became_peered":"2024-04-25T10:11:23.503299+0000","last_unstale":"2024-04-25T10:11:24.479618+0000","last_undegraded":"2024-04-25T10:11:24.479618+0000","last_fullsized":"2024-04-25T10:11:24.479618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:37.491564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479662+0000","last_change":"2024-04-25T10:11:22.458531+0000","last_active":"2024-04-25T10:11:24.479662+0000","last_peered":"2024-04-25T10:11:24.479662+0000","last_clean":"2024-04-25T10:11:24.479662+0000","last_became_active":"2024-04-25T10:11:22.457900+0000","last_became_peered":"2024-04-25T10:11:22.457900+0000","last_unstale":"2024-04-25T10:11:24.479662+0000","last_undegraded":"2024-04-25T10:11:24.479662+0000","last_fullsized":"2024-04-25T10:11:24.479662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:32.968772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864134+0000","last_change":"2024-04-25T10:11:23.498761+0000","last_active":"2024-04-25T10:11:24.864134+0000","last_peered":"2024-04-25T10:11:24.864134+0000","last_clean":"2024-04-25T10:11:24.864134+0000","last_became_active":"2024-04-25T10:11:23.498483+0000","last_became_peered":"2024-04-25T10:11:23.498483+0000","last_unstale":"2024-04-25T10:11:24.864134+0000","last_undegraded":"2024-04-25T10:11:24.864134+0000","last_fullsized":"2024-04-25T10:11:24.864134+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:31.072700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.949962+0000","last_change":"2024-04-25T10:11:22.470587+0000","last_active":"2024-04-25T10:11:25.949962+0000","last_peered":"2024-04-25T10:11:25.949962+0000","last_clean":"2024-04-25T10:11:25.949962+0000","last_became_active":"2024-04-25T10:11:22.469775+0000","last_became_peered":"2024-04-25T10:11:22.469775+0000","last_unstale":"2024-04-25T10:11:25.949962+0000","last_undegraded":"2024-04-25T10:11:25.949962+0000","last_fullsized":"2024-04-25T10:11:25.949962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:11.893976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950687+0000","last_change":"2024-04-25T10:11:23.517824+0000","last_active":"2024-04-25T10:11:25.950687+0000","last_peered":"2024-04-25T10:11:25.950687+0000","last_clean":"2024-04-25T10:11:25.950687+0000","last_became_active":"2024-04-25T10:11:23.517479+0000","last_became_peered":"2024-04-25T10:11:23.517479+0000","last_unstale":"2024-04-25T10:11:25.950687+0000","last_undegraded":"2024-04-25T10:11:25.950687+0000","last_fullsized":"2024-04-25T10:11:25.950687+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:47.357088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968304+0000","last_change":"2024-04-25T10:11:22.474875+0000","last_active":"2024-04-25T10:11:25.968304+0000","last_peered":"2024-04-25T10:11:25.968304+0000","last_clean":"2024-04-25T10:11:25.968304+0000","last_became_active":"2024-04-25T10:11:22.474581+0000","last_became_peered":"2024-04-25T10:11:22.474581+0000","last_unstale":"2024-04-25T10:11:25.968304+0000","last_undegraded":"2024-04-25T10:11:25.968304+0000","last_fullsized":"2024-04-25T10:11:25.968304+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:29.865607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":318,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950229+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:11:25.950229+0000","last_peered":"2024-04-25T10:11:25.950229+0000","last_clean":"2024-04-25T10:11:25.950229+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:11:25.950229+0000","last_undegraded":"2024-04-25T10:11:25.950229+0000","last_fullsized":"2024-04-25T10:11:25.950229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50996,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320844,"statfs":{"total":96636764160,"available":96584544256,"internally_reserved":0,"allocated":1810432,"data_stored":1153237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50428,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321412,"statfs":{"total":96636764160,"available":96585125888,"internally_reserved":0,"allocated":1228800,"data_stored":565029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58736,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313104,"statfs":{"total":96636764160,"available":96576618496,"internally_reserved":0,"allocated":1216512,"data_stored":562973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54572,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317268,"statfs":{"total":96636764160,"available":96580882432,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50412,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321428,"statfs":{"total":96636764160,"available":96585142272,"internally_reserved":0,"allocated":1212416,"data_stored":562883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55172,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316668,"statfs":{"total":96636764160,"available":96580268032,"internally_reserved":0,"allocated":1826816,"data_stored":1155499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:28.801 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:11:29.006 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:29.014 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:29.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.033+0000 7f05e6c00640 1 Processor -- start 2024-04-25T10:11:29.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.034+0000 7f05e6c00640 1 -- start start 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e015ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e6c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f05e00ce8e0 0x7f05e00d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e6c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05e015efa0 con 0x7f05e0162230 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e6c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05e00cf130 con 0x7f05e00ce8e0 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0161da0 con 0x7f05e00d0050 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e4e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f05e0162230 0x7f05e00d0430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50116/0 (socket says 172.21.15.165:50116) 2024-04-25T10:11:29.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.035+0000 7f05e4e00640 1 -- 172.21.15.165:0/983763007 learned_addr learned my addr 172.21.15.165:0/983763007 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:29.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05e5800640 1 --2- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e015ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05e6200640 1 --2- 172.21.15.165:0/983763007 >> v2:172.21.15.172:3300/0 conn(0x7f05e00ce8e0 0x7f05e00d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05dfe00640 1 -- 172.21.15.165:0/983763007 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862340004 0 0) 0x7f05cc001110 con 0x7f05e0162230 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05dfe00640 1 -- 172.21.15.165:0/983763007 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05c8003520 con 0x7f05e0162230 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05e6200640 1 -- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 msgr2=0x7f05e015ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05e6200640 1 --2- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e015ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05dfe00640 1 -- 172.21.15.165:0/983763007 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2727925285 0 0) 0x7f05c8003520 con 0x7f05e0162230 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05e6200640 1 -- 172.21.15.165:0/983763007 >> v1:172.21.15.165:6789/0 conn(0x7f05e0162230 legacy=0x7f05e00d0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.036+0000 7f05e6200640 1 -- 172.21.15.165:0/983763007 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00cee50 con 0x7f05e00ce8e0 2024-04-25T10:11:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.038+0000 7f05e6200640 1 --2- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00d1d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009d90 tx=0x7f05d802a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e0779a3216a51e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.038+0000 7f05dfe00640 1 -- 172.21.15.165:0/983763007 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05cc001840 con 0x7f05e00ce8e0 2024-04-25T10:11:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.038+0000 7f05dfe00640 1 -- 172.21.15.165:0/983763007 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8007540 con 0x7f05e00ce8e0 2024-04-25T10:11:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.038+0000 7f05dfe00640 1 -- 172.21.15.165:0/983763007 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d80078f0 con 0x7f05e00ce8e0 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 -- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 msgr2=0x7f05e00d1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 --2- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00d1d30 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009d90 tx=0x7f05d802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 -- 172.21.15.165:0/983763007 shutdown_connections 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 --2- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00d1d30 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 --2- 172.21.15.165:0/983763007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e015ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 -- 172.21.15.165:0/983763007 >> 172.21.15.165:0/983763007 conn(0x7f05e0061600 msgr2=0x7f05e0061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 -- 172.21.15.165:0/983763007 shutdown_connections 2024-04-25T10:11:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.039+0000 7f05e6c00640 1 -- 172.21.15.165:0/983763007 wait complete. 2024-04-25T10:11:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.041+0000 7f05e6c00640 1 Processor -- start 2024-04-25T10:11:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.041+0000 7f05e6c00640 1 -- start start 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e00b5940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 0x7f05e01e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00d3540 con 0x7f05e0162230 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05e0161a30 con 0x7f05e00ce8e0 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05e00d2270 con 0x7f05e00d0050 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 0x7f05e01e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 0x7f05e01e73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35816/0 (socket says 172.21.15.165:35816) 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e6200640 1 -- 172.21.15.165:0/144615476 learned_addr learned my addr 172.21.15.165:0/144615476 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.042+0000 7f05e4e00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e00b5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e5800640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e6200640 1 -- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 msgr2=0x7f05e00b5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e6200640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e00b5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e6200640 1 -- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 msgr2=0x7f05e00c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e6200640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e6200640 1 -- 172.21.15.165:0/144615476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05e00b48a0 con 0x7f05e0162230 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e4e00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e00b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e5800640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:29.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05e6200640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 0x7f05e01e73d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009eb0 tx=0x7f05d8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.043+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8040070 con 0x7f05e0162230 2024-04-25T10:11:29.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.044+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05d8031c40 con 0x7f05e0162230 2024-04-25T10:11:29.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.044+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05e00b4b80 con 0x7f05e0162230 2024-04-25T10:11:29.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.044+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05d8009450 con 0x7f05e0162230 2024-04-25T10:11:29.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.044+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05e00b5140 con 0x7f05e0162230 2024-04-25T10:11:29.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.045+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05e0073990 con 0x7f05e0162230 2024-04-25T10:11:29.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.046+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f05d8039040 con 0x7f05e0162230 2024-04-25T10:11:29.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.047+0000 7f05dd600640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f05c006b4d0 0x7f05c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.047+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f05d80ab8c0 con 0x7f05e0162230 2024-04-25T10:11:29.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.047+0000 7f05e5800640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f05c006b4d0 0x7f05c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.048+0000 7f05e5800640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f05c006b4d0 0x7f05c006d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009b90 tx=0x7f05d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.051+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f05d807a510 con 0x7f05e0162230 2024-04-25T10:11:29.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.312+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f05e00b91a0 con 0x7f05e0162230 2024-04-25T10:11:29.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.313+0000 7f05dd600640 1 -- 172.21.15.165:0/144615476 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1175 (secure 0 0 0) 0x7f05d8079e10 con 0x7f05e0162230 2024-04-25T10:11:29.318 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:29.318 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":590,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":98,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":431247360,"bytes_avail":772662865920,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7569}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":21,"modified":"2024-04-25T10:10:42.306525+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f05c006b4d0 msgr2=0x7f05c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f05c006b4d0 0x7f05c006d990 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009b90 tx=0x7f05d4008040 comp rx=0 tx=0).stop 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 msgr2=0x7f05e01e73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 0x7f05e01e73d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f05d8009eb0 tx=0x7f05d8004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 shutdown_connections 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f05c006b4d0 0x7f05c006d990 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f05e0162230 0x7f05e01e73d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05e00d0050 0x7f05e00b5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 --2- 172.21.15.165:0/144615476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05e00ce8e0 0x7f05e00c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.315+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 >> 172.21.15.165:0/144615476 conn(0x7f05e0061600 msgr2=0x7f05e015f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.316+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 shutdown_connections 2024-04-25T10:11:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.316+0000 7f05e6c00640 1 -- 172.21.15.165:0/144615476 wait complete. 2024-04-25T10:11:29.338 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:11:29.338 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:29.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.520+0000 7fd182600640 1 Processor -- start 2024-04-25T10:11:29.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.520+0000 7fd182600640 1 -- start start 2024-04-25T10:11:29.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd182600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd17c15a9b0 0x7fd17c0b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd182600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c0b4180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd182600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd17c0b46c0 0x7fd17c0adda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd182600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c15ad90 con 0x7fd17c0b46c0 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd182600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17c06f370 con 0x7fd17c15a9b0 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd182600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c06f650 con 0x7fd17c06ee40 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd17be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd17c0b46c0 0x7fd17c0adda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd17be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd17c0b46c0 0x7fd17c0adda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35826/0 (socket says 172.21.15.165:35826) 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd17be00640 1 -- 172.21.15.165:0/1878735259 learned_addr learned my addr 172.21.15.165:0/1878735259 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.522+0000 7fd17b400640 1 --2- 172.21.15.165:0/1878735259 >> v2:172.21.15.172:3300/0 conn(0x7fd17c15a9b0 0x7fd17c0b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.523+0000 7fd17be00640 1 -- 172.21.15.165:0/1878735259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 msgr2=0x7fd17c0b4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.523+0000 7fd17be00640 1 --2- 172.21.15.165:0/1878735259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c0b4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.523+0000 7fd17be00640 1 -- 172.21.15.165:0/1878735259 >> v2:172.21.15.172:3300/0 conn(0x7fd17c15a9b0 msgr2=0x7fd17c0b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.523+0000 7fd17be00640 1 --2- 172.21.15.165:0/1878735259 >> v2:172.21.15.172:3300/0 conn(0x7fd17c15a9b0 0x7fd17c0b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.523+0000 7fd17be00640 1 -- 172.21.15.165:0/1878735259 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c06f930 con 0x7fd17c0b46c0 2024-04-25T10:11:29.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.524+0000 7fd17be00640 1 --2- 172.21.15.165:0/1878735259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0adda0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0726e0 tx=0x7fd17402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a85d58ac71f91e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.524+0000 7fd17a000640 1 -- 172.21.15.165:0/1878735259 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174036070 con 0x7fd17c0b46c0 2024-04-25T10:11:29.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.524+0000 7fd17a000640 1 -- 172.21.15.165:0/1878735259 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174031600 con 0x7fd17c0b46c0 2024-04-25T10:11:29.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.524+0000 7fd17a000640 1 -- 172.21.15.165:0/1878735259 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740319d0 con 0x7fd17c0b46c0 2024-04-25T10:11:29.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 -- 172.21.15.165:0/1878735259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 msgr2=0x7fd17c0adda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 --2- 172.21.15.165:0/1878735259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0adda0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7fd17c0726e0 tx=0x7fd17402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 -- 172.21.15.165:0/1878735259 shutdown_connections 2024-04-25T10:11:29.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 --2- 172.21.15.165:0/1878735259 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0adda0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 --2- 172.21.15.165:0/1878735259 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 --2- 172.21.15.165:0/1878735259 >> v2:172.21.15.172:3300/0 conn(0x7fd17c15a9b0 0x7fd17c0b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 -- 172.21.15.165:0/1878735259 >> 172.21.15.165:0/1878735259 conn(0x7fd17c0b5af0 msgr2=0x7fd17c0b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 -- 172.21.15.165:0/1878735259 shutdown_connections 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.525+0000 7fd182600640 1 -- 172.21.15.165:0/1878735259 wait complete. 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.526+0000 7fd182600640 1 Processor -- start 2024-04-25T10:11:29.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.527+0000 7fd182600640 1 -- start start 2024-04-25T10:11:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.527+0000 7fd182600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c1f75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd182600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd182600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17c15a9b0 0x7fd17c0cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd182600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0b3cf0 con 0x7fd17c0b46c0 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd182600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c06fef0 con 0x7fd17c15a9b0 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd182600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd17c0b3e70 con 0x7fd17c06ee40 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35830/0 (socket says 172.21.15.165:35830) 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 -- 172.21.15.165:0/2544024142 learned_addr learned my addr 172.21.15.165:0/2544024142 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17b400640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c1f75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 -- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 msgr2=0x7fd17c1f75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17be00640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17c15a9b0 0x7fd17c0cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c1f75e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 -- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17c15a9b0 msgr2=0x7fd17c0cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17c15a9b0 0x7fd17c0cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.528+0000 7fd17aa00640 1 -- 172.21.15.165:0/2544024142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd17c0cbb80 con 0x7fd17c0b46c0 2024-04-25T10:11:29.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.529+0000 7fd17aa00640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0afa00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b410 tx=0x7fd16800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.529+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168015020 con 0x7fd17c0b46c0 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.529+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd168011850 con 0x7fd17c0b46c0 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.529+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd168011c40 con 0x7fd17c0b46c0 2024-04-25T10:11:29.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.529+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd17c0cbe60 con 0x7fd17c0b46c0 2024-04-25T10:11:29.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.529+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd17c1f6de0 con 0x7fd17c0b46c0 2024-04-25T10:11:29.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.530+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd17c073990 con 0x7fd17c0b46c0 2024-04-25T10:11:29.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.531+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd168018020 con 0x7fd17c0b46c0 2024-04-25T10:11:29.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.532+0000 7fd173400640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd15c06b4d0 0x7fd15c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:29.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.532+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd16808a290 con 0x7fd17c0b46c0 2024-04-25T10:11:29.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.532+0000 7fd17b400640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd15c06b4d0 0x7fd15c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:29.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.533+0000 7fd17b400640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd15c06b4d0 0x7fd15c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd16c006fd0 tx=0x7fd16c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:29.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.537+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd16801f020 con 0x7fd17c0b46c0 2024-04-25T10:11:29.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.726+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd17c155330 con 0x7fd15c06b4d0 2024-04-25T10:11:29.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.734+0000 7fd173400640 1 -- 172.21.15.165:0/2544024142 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252075 (secure 0 0 0) 0x7fd17c155330 con 0x7fd15c06b4d0 2024-04-25T10:11:29.742 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:11:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd15c06b4d0 msgr2=0x7fd15c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd15c06b4d0 0x7fd15c06d990 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd16c006fd0 tx=0x7fd16c008040 comp rx=0 tx=0).stop 2024-04-25T10:11:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 msgr2=0x7fd17c0afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0afa00 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7fd16800b410 tx=0x7fd16800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 shutdown_connections 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd15c06b4d0 0x7fd15c06d990 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd17c15a9b0 0x7fd17c0cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd17c0b46c0 0x7fd17c0afa00 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 --2- 172.21.15.165:0/2544024142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd17c06ee40 0x7fd17c1f75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 >> 172.21.15.165:0/2544024142 conn(0x7fd17c0b5af0 msgr2=0x7fd17c155b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 shutdown_connections 2024-04-25T10:11:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:29.741+0000 7fd182600640 1 -- 172.21.15.165:0/2544024142 wait complete. 2024-04-25T10:11:29.764 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":379,"stamp":"2024-04-25T10:11:28.321878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":154,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421140,"kb_used_data":10664,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754553580,"statfs":{"total":773094113280,"available":772662865920,"internally_reserved":0,"allocated":10919936,"data_stored":5688060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":420204082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001959"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475221+0000","last_change":"2024-04-25T10:11:23.514302+0000","last_active":"2024-04-25T10:11:24.475221+0000","last_peered":"2024-04-25T10:11:24.475221+0000","last_clean":"2024-04-25T10:11:24.475221+0000","last_became_active":"2024-04-25T10:11:23.509239+0000","last_became_peered":"2024-04-25T10:11:23.509239+0000","last_unstale":"2024-04-25T10:11:24.475221+0000","last_undegraded":"2024-04-25T10:11:24.475221+0000","last_fullsized":"2024-04-25T10:11:24.475221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:47.357088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521483+0000","last_change":"2024-04-25T10:11:23.511726+0000","last_active":"2024-04-25T10:11:24.521483+0000","last_peered":"2024-04-25T10:11:24.521483+0000","last_clean":"2024-04-25T10:11:24.521483+0000","last_became_active":"2024-04-25T10:11:23.511477+0000","last_became_peered":"2024-04-25T10:11:23.511477+0000","last_unstale":"2024-04-25T10:11:24.521483+0000","last_undegraded":"2024-04-25T10:11:24.521483+0000","last_fullsized":"2024-04-25T10:11:24.521483+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:21.609016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474979+0000","last_change":"2024-04-25T10:11:23.504635+0000","last_active":"2024-04-25T10:11:24.474979+0000","last_peered":"2024-04-25T10:11:24.474979+0000","last_clean":"2024-04-25T10:11:24.474979+0000","last_became_active":"2024-04-25T10:11:23.504397+0000","last_became_peered":"2024-04-25T10:11:23.504397+0000","last_unstale":"2024-04-25T10:11:24.474979+0000","last_undegraded":"2024-04-25T10:11:24.474979+0000","last_fullsized":"2024-04-25T10:11:24.474979+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:15.008119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968039+0000","last_change":"2024-04-25T10:11:23.501516+0000","last_active":"2024-04-25T10:11:25.968039+0000","last_peered":"2024-04-25T10:11:25.968039+0000","last_clean":"2024-04-25T10:11:25.968039+0000","last_became_active":"2024-04-25T10:11:23.501180+0000","last_became_peered":"2024-04-25T10:11:23.501180+0000","last_unstale":"2024-04-25T10:11:25.968039+0000","last_undegraded":"2024-04-25T10:11:25.968039+0000","last_fullsized":"2024-04-25T10:11:25.968039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.781341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478461+0000","last_change":"2024-04-25T10:11:23.493320+0000","last_active":"2024-04-25T10:11:24.478461+0000","last_peered":"2024-04-25T10:11:24.478461+0000","last_clean":"2024-04-25T10:11:24.478461+0000","last_became_active":"2024-04-25T10:11:23.492988+0000","last_became_peered":"2024-04-25T10:11:23.492988+0000","last_unstale":"2024-04-25T10:11:24.478461+0000","last_undegraded":"2024-04-25T10:11:24.478461+0000","last_fullsized":"2024-04-25T10:11:24.478461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:43.297903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863718+0000","last_change":"2024-04-25T10:11:23.503887+0000","last_active":"2024-04-25T10:11:24.863718+0000","last_peered":"2024-04-25T10:11:24.863718+0000","last_clean":"2024-04-25T10:11:24.863718+0000","last_became_active":"2024-04-25T10:11:23.503580+0000","last_became_peered":"2024-04-25T10:11:23.503580+0000","last_unstale":"2024-04-25T10:11:24.863718+0000","last_undegraded":"2024-04-25T10:11:24.863718+0000","last_fullsized":"2024-04-25T10:11:24.863718+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:00.002046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950146+0000","last_change":"2024-04-25T10:11:23.506780+0000","last_active":"2024-04-25T10:11:25.950146+0000","last_peered":"2024-04-25T10:11:25.950146+0000","last_clean":"2024-04-25T10:11:25.950146+0000","last_became_active":"2024-04-25T10:11:23.506380+0000","last_became_peered":"2024-04-25T10:11:23.506380+0000","last_unstale":"2024-04-25T10:11:25.950146+0000","last_undegraded":"2024-04-25T10:11:25.950146+0000","last_fullsized":"2024-04-25T10:11:25.950146+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:35.118925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967401+0000","last_change":"2024-04-25T10:11:23.496772+0000","last_active":"2024-04-25T10:11:25.967401+0000","last_peered":"2024-04-25T10:11:25.967401+0000","last_clean":"2024-04-25T10:11:25.967401+0000","last_became_active":"2024-04-25T10:11:23.496466+0000","last_became_peered":"2024-04-25T10:11:23.496466+0000","last_unstale":"2024-04-25T10:11:25.967401+0000","last_undegraded":"2024-04-25T10:11:25.967401+0000","last_fullsized":"2024-04-25T10:11:25.967401+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480543+0000","last_change":"2024-04-25T10:11:23.496579+0000","last_active":"2024-04-25T10:11:24.480543+0000","last_peered":"2024-04-25T10:11:24.480543+0000","last_clean":"2024-04-25T10:11:24.480543+0000","last_became_active":"2024-04-25T10:11:23.495543+0000","last_became_peered":"2024-04-25T10:11:23.495543+0000","last_unstale":"2024-04-25T10:11:24.480543+0000","last_undegraded":"2024-04-25T10:11:24.480543+0000","last_fullsized":"2024-04-25T10:11:24.480543+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:46.821062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966387+0000","last_change":"2024-04-25T10:11:23.507660+0000","last_active":"2024-04-25T10:11:25.966387+0000","last_peered":"2024-04-25T10:11:25.966387+0000","last_clean":"2024-04-25T10:11:25.966387+0000","last_became_active":"2024-04-25T10:11:23.507339+0000","last_became_peered":"2024-04-25T10:11:23.507339+0000","last_unstale":"2024-04-25T10:11:25.966387+0000","last_undegraded":"2024-04-25T10:11:25.966387+0000","last_fullsized":"2024-04-25T10:11:25.966387+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.303357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479723+0000","last_change":"2024-04-25T10:11:23.495970+0000","last_active":"2024-04-25T10:11:24.479723+0000","last_peered":"2024-04-25T10:11:24.479723+0000","last_clean":"2024-04-25T10:11:24.479723+0000","last_became_active":"2024-04-25T10:11:23.495635+0000","last_became_peered":"2024-04-25T10:11:23.495635+0000","last_unstale":"2024-04-25T10:11:24.479723+0000","last_undegraded":"2024-04-25T10:11:24.479723+0000","last_fullsized":"2024-04-25T10:11:24.479723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:46.519451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965939+0000","last_change":"2024-04-25T10:11:23.497428+0000","last_active":"2024-04-25T10:11:25.965939+0000","last_peered":"2024-04-25T10:11:25.965939+0000","last_clean":"2024-04-25T10:11:25.965939+0000","last_became_active":"2024-04-25T10:11:23.495767+0000","last_became_peered":"2024-04-25T10:11:23.495767+0000","last_unstale":"2024-04-25T10:11:25.965939+0000","last_undegraded":"2024-04-25T10:11:25.965939+0000","last_fullsized":"2024-04-25T10:11:25.965939+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:56.257951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968172+0000","last_change":"2024-04-25T10:11:23.505393+0000","last_active":"2024-04-25T10:11:25.968172+0000","last_peered":"2024-04-25T10:11:25.968172+0000","last_clean":"2024-04-25T10:11:25.968172+0000","last_became_active":"2024-04-25T10:11:23.502745+0000","last_became_peered":"2024-04-25T10:11:23.502745+0000","last_unstale":"2024-04-25T10:11:25.968172+0000","last_undegraded":"2024-04-25T10:11:25.968172+0000","last_fullsized":"2024-04-25T10:11:25.968172+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:22.077916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478359+0000","last_change":"2024-04-25T10:11:23.504530+0000","last_active":"2024-04-25T10:11:24.478359+0000","last_peered":"2024-04-25T10:11:24.478359+0000","last_clean":"2024-04-25T10:11:24.478359+0000","last_became_active":"2024-04-25T10:11:23.504230+0000","last_became_peered":"2024-04-25T10:11:23.504230+0000","last_unstale":"2024-04-25T10:11:24.478359+0000","last_undegraded":"2024-04-25T10:11:24.478359+0000","last_fullsized":"2024-04-25T10:11:24.478359+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:31.827948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968022+0000","last_change":"2024-04-25T10:11:23.514234+0000","last_active":"2024-04-25T10:11:25.968022+0000","last_peered":"2024-04-25T10:11:25.968022+0000","last_clean":"2024-04-25T10:11:25.968022+0000","last_became_active":"2024-04-25T10:11:23.513691+0000","last_became_peered":"2024-04-25T10:11:23.513691+0000","last_unstale":"2024-04-25T10:11:25.968022+0000","last_undegraded":"2024-04-25T10:11:25.968022+0000","last_fullsized":"2024-04-25T10:11:25.968022+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.091532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864271+0000","last_change":"2024-04-25T10:11:23.496506+0000","last_active":"2024-04-25T10:11:24.864271+0000","last_peered":"2024-04-25T10:11:24.864271+0000","last_clean":"2024-04-25T10:11:24.864271+0000","last_became_active":"2024-04-25T10:11:23.496229+0000","last_became_peered":"2024-04-25T10:11:23.496229+0000","last_unstale":"2024-04-25T10:11:24.864271+0000","last_undegraded":"2024-04-25T10:11:24.864271+0000","last_fullsized":"2024-04-25T10:11:24.864271+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:40.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863850+0000","last_change":"2024-04-25T10:11:23.518147+0000","last_active":"2024-04-25T10:11:24.863850+0000","last_peered":"2024-04-25T10:11:24.863850+0000","last_clean":"2024-04-25T10:11:24.863850+0000","last_became_active":"2024-04-25T10:11:23.517364+0000","last_became_peered":"2024-04-25T10:11:23.517364+0000","last_unstale":"2024-04-25T10:11:24.863850+0000","last_undegraded":"2024-04-25T10:11:24.863850+0000","last_fullsized":"2024-04-25T10:11:24.863850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:55.872365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475072+0000","last_change":"2024-04-25T10:11:23.506788+0000","last_active":"2024-04-25T10:11:24.475072+0000","last_peered":"2024-04-25T10:11:24.475072+0000","last_clean":"2024-04-25T10:11:24.475072+0000","last_became_active":"2024-04-25T10:11:23.506508+0000","last_became_peered":"2024-04-25T10:11:23.506508+0000","last_unstale":"2024-04-25T10:11:24.475072+0000","last_undegraded":"2024-04-25T10:11:24.475072+0000","last_fullsized":"2024-04-25T10:11:24.475072+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:35.118925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479440+0000","last_change":"2024-04-25T10:11:23.505949+0000","last_active":"2024-04-25T10:11:24.479440+0000","last_peered":"2024-04-25T10:11:24.479440+0000","last_clean":"2024-04-25T10:11:24.479440+0000","last_became_active":"2024-04-25T10:11:23.501690+0000","last_became_peered":"2024-04-25T10:11:23.501690+0000","last_unstale":"2024-04-25T10:11:24.479440+0000","last_undegraded":"2024-04-25T10:11:24.479440+0000","last_fullsized":"2024-04-25T10:11:24.479440+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.972961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968249+0000","last_change":"2024-04-25T10:11:23.514427+0000","last_active":"2024-04-25T10:11:25.968249+0000","last_peered":"2024-04-25T10:11:25.968249+0000","last_clean":"2024-04-25T10:11:25.968249+0000","last_became_active":"2024-04-25T10:11:23.514184+0000","last_became_peered":"2024-04-25T10:11:23.514184+0000","last_unstale":"2024-04-25T10:11:25.968249+0000","last_undegraded":"2024-04-25T10:11:25.968249+0000","last_fullsized":"2024-04-25T10:11:25.968249+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:36.868831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478219+0000","last_change":"2024-04-25T10:11:23.501619+0000","last_active":"2024-04-25T10:11:24.478219+0000","last_peered":"2024-04-25T10:11:24.478219+0000","last_clean":"2024-04-25T10:11:24.478219+0000","last_became_active":"2024-04-25T10:11:23.500466+0000","last_became_peered":"2024-04-25T10:11:23.500466+0000","last_unstale":"2024-04-25T10:11:24.478219+0000","last_undegraded":"2024-04-25T10:11:24.478219+0000","last_fullsized":"2024-04-25T10:11:24.478219+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478814+0000","last_change":"2024-04-25T10:11:23.502202+0000","last_active":"2024-04-25T10:11:24.478814+0000","last_peered":"2024-04-25T10:11:24.478814+0000","last_clean":"2024-04-25T10:11:24.478814+0000","last_became_active":"2024-04-25T10:11:23.501391+0000","last_became_peered":"2024-04-25T10:11:23.501391+0000","last_unstale":"2024-04-25T10:11:24.478814+0000","last_undegraded":"2024-04-25T10:11:24.478814+0000","last_fullsized":"2024-04-25T10:11:24.478814+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.937312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478106+0000","last_change":"2024-04-25T10:11:23.501608+0000","last_active":"2024-04-25T10:11:24.478106+0000","last_peered":"2024-04-25T10:11:24.478106+0000","last_clean":"2024-04-25T10:11:24.478106+0000","last_became_active":"2024-04-25T10:11:23.500874+0000","last_became_peered":"2024-04-25T10:11:23.500874+0000","last_unstale":"2024-04-25T10:11:24.478106+0000","last_undegraded":"2024-04-25T10:11:24.478106+0000","last_fullsized":"2024-04-25T10:11:24.478106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:39.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474639+0000","last_change":"2024-04-25T10:11:23.504056+0000","last_active":"2024-04-25T10:11:24.474639+0000","last_peered":"2024-04-25T10:11:24.474639+0000","last_clean":"2024-04-25T10:11:24.474639+0000","last_became_active":"2024-04-25T10:11:23.503801+0000","last_became_peered":"2024-04-25T10:11:23.503801+0000","last_unstale":"2024-04-25T10:11:24.474639+0000","last_undegraded":"2024-04-25T10:11:24.474639+0000","last_fullsized":"2024-04-25T10:11:24.474639+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:32.459765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521718+0000","last_change":"2024-04-25T10:11:23.490475+0000","last_active":"2024-04-25T10:11:24.521718+0000","last_peered":"2024-04-25T10:11:24.521718+0000","last_clean":"2024-04-25T10:11:24.521718+0000","last_became_active":"2024-04-25T10:11:23.489746+0000","last_became_peered":"2024-04-25T10:11:23.489746+0000","last_unstale":"2024-04-25T10:11:24.521718+0000","last_undegraded":"2024-04-25T10:11:24.521718+0000","last_fullsized":"2024-04-25T10:11:24.521718+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:41.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479424+0000","last_change":"2024-04-25T10:11:23.503039+0000","last_active":"2024-04-25T10:11:24.479424+0000","last_peered":"2024-04-25T10:11:24.479424+0000","last_clean":"2024-04-25T10:11:24.479424+0000","last_became_active":"2024-04-25T10:11:23.502130+0000","last_became_peered":"2024-04-25T10:11:23.502130+0000","last_unstale":"2024-04-25T10:11:24.479424+0000","last_undegraded":"2024-04-25T10:11:24.479424+0000","last_fullsized":"2024-04-25T10:11:24.479424+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966420+0000","last_change":"2024-04-25T10:11:23.501635+0000","last_active":"2024-04-25T10:11:25.966420+0000","last_peered":"2024-04-25T10:11:25.966420+0000","last_clean":"2024-04-25T10:11:25.966420+0000","last_became_active":"2024-04-25T10:11:23.501356+0000","last_became_peered":"2024-04-25T10:11:23.501356+0000","last_unstale":"2024-04-25T10:11:25.966420+0000","last_undegraded":"2024-04-25T10:11:25.966420+0000","last_fullsized":"2024-04-25T10:11:25.966420+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:29.491438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968463+0000","last_change":"2024-04-25T10:11:23.499753+0000","last_active":"2024-04-25T10:11:25.968463+0000","last_peered":"2024-04-25T10:11:25.968463+0000","last_clean":"2024-04-25T10:11:25.968463+0000","last_became_active":"2024-04-25T10:11:23.499460+0000","last_became_peered":"2024-04-25T10:11:23.499460+0000","last_unstale":"2024-04-25T10:11:25.968463+0000","last_undegraded":"2024-04-25T10:11:25.968463+0000","last_fullsized":"2024-04-25T10:11:25.968463+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:34.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950652+0000","last_change":"2024-04-25T10:11:23.500199+0000","last_active":"2024-04-25T10:11:25.950652+0000","last_peered":"2024-04-25T10:11:25.950652+0000","last_clean":"2024-04-25T10:11:25.950652+0000","last_became_active":"2024-04-25T10:11:23.499904+0000","last_became_peered":"2024-04-25T10:11:23.499904+0000","last_unstale":"2024-04-25T10:11:25.950652+0000","last_undegraded":"2024-04-25T10:11:25.950652+0000","last_fullsized":"2024-04-25T10:11:25.950652+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:41.977671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479200+0000","last_change":"2024-04-25T10:11:23.494112+0000","last_active":"2024-04-25T10:11:24.479200+0000","last_peered":"2024-04-25T10:11:24.479200+0000","last_clean":"2024-04-25T10:11:24.479200+0000","last_became_active":"2024-04-25T10:11:23.493781+0000","last_became_peered":"2024-04-25T10:11:23.493781+0000","last_unstale":"2024-04-25T10:11:24.479200+0000","last_undegraded":"2024-04-25T10:11:24.479200+0000","last_fullsized":"2024-04-25T10:11:24.479200+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:37.491564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520698+0000","last_change":"2024-04-25T10:11:23.504171+0000","last_active":"2024-04-25T10:11:24.520698+0000","last_peered":"2024-04-25T10:11:24.520698+0000","last_clean":"2024-04-25T10:11:24.520698+0000","last_became_active":"2024-04-25T10:11:23.501873+0000","last_became_peered":"2024-04-25T10:11:23.501873+0000","last_unstale":"2024-04-25T10:11:24.520698+0000","last_undegraded":"2024-04-25T10:11:24.520698+0000","last_fullsized":"2024-04-25T10:11:24.520698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:13.126891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864462+0000","last_change":"2024-04-25T10:11:23.496202+0000","last_active":"2024-04-25T10:11:24.864462+0000","last_peered":"2024-04-25T10:11:24.864462+0000","last_clean":"2024-04-25T10:11:24.864462+0000","last_became_active":"2024-04-25T10:11:23.495923+0000","last_became_peered":"2024-04-25T10:11:23.495923+0000","last_unstale":"2024-04-25T10:11:24.864462+0000","last_undegraded":"2024-04-25T10:11:24.864462+0000","last_fullsized":"2024-04-25T10:11:24.864462+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:26.229309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864558+0000","last_change":"2024-04-25T10:11:23.500336+0000","last_active":"2024-04-25T10:11:24.864558+0000","last_peered":"2024-04-25T10:11:24.864558+0000","last_clean":"2024-04-25T10:11:24.864558+0000","last_became_active":"2024-04-25T10:11:23.498911+0000","last_became_peered":"2024-04-25T10:11:23.498911+0000","last_unstale":"2024-04-25T10:11:24.864558+0000","last_undegraded":"2024-04-25T10:11:24.864558+0000","last_fullsized":"2024-04-25T10:11:24.864558+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.192265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"97'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480329+0000","last_change":"2024-04-25T10:11:22.465629+0000","last_active":"2024-04-25T10:11:24.480329+0000","last_peered":"2024-04-25T10:11:24.480329+0000","last_clean":"2024-04-25T10:11:24.480329+0000","last_became_active":"2024-04-25T10:11:22.465239+0000","last_became_peered":"2024-04-25T10:11:22.465239+0000","last_unstale":"2024-04-25T10:11:24.480329+0000","last_undegraded":"2024-04-25T10:11:24.480329+0000","last_fullsized":"2024-04-25T10:11:24.480329+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:29.898591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864201+0000","last_change":"2024-04-25T10:11:23.492228+0000","last_active":"2024-04-25T10:11:24.864201+0000","last_peered":"2024-04-25T10:11:24.864201+0000","last_clean":"2024-04-25T10:11:24.864201+0000","last_became_active":"2024-04-25T10:11:23.491931+0000","last_became_peered":"2024-04-25T10:11:23.491931+0000","last_unstale":"2024-04-25T10:11:24.864201+0000","last_undegraded":"2024-04-25T10:11:24.864201+0000","last_fullsized":"2024-04-25T10:11:24.864201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:07.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479937+0000","last_change":"2024-04-25T10:11:22.467541+0000","last_active":"2024-04-25T10:11:24.479937+0000","last_peered":"2024-04-25T10:11:24.479937+0000","last_clean":"2024-04-25T10:11:24.479937+0000","last_became_active":"2024-04-25T10:11:22.467211+0000","last_became_peered":"2024-04-25T10:11:22.467211+0000","last_unstale":"2024-04-25T10:11:24.479937+0000","last_undegraded":"2024-04-25T10:11:24.479937+0000","last_fullsized":"2024-04-25T10:11:24.479937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:29.678377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950850+0000","last_change":"2024-04-25T10:11:23.505136+0000","last_active":"2024-04-25T10:11:25.950850+0000","last_peered":"2024-04-25T10:11:25.950850+0000","last_clean":"2024-04-25T10:11:25.950850+0000","last_became_active":"2024-04-25T10:11:23.504848+0000","last_became_peered":"2024-04-25T10:11:23.504848+0000","last_unstale":"2024-04-25T10:11:25.950850+0000","last_undegraded":"2024-04-25T10:11:25.950850+0000","last_fullsized":"2024-04-25T10:11:25.950850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:34.764490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950348+0000","last_change":"2024-04-25T10:11:22.474715+0000","last_active":"2024-04-25T10:11:25.950348+0000","last_peered":"2024-04-25T10:11:25.950348+0000","last_clean":"2024-04-25T10:11:25.950348+0000","last_became_active":"2024-04-25T10:11:22.472397+0000","last_became_peered":"2024-04-25T10:11:22.472397+0000","last_unstale":"2024-04-25T10:11:25.950348+0000","last_undegraded":"2024-04-25T10:11:25.950348+0000","last_fullsized":"2024-04-25T10:11:25.950348+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:03.107761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863968+0000","last_change":"2024-04-25T10:11:23.486363+0000","last_active":"2024-04-25T10:11:24.863968+0000","last_peered":"2024-04-25T10:11:24.863968+0000","last_clean":"2024-04-25T10:11:24.863968+0000","last_became_active":"2024-04-25T10:11:23.485997+0000","last_became_peered":"2024-04-25T10:11:23.485997+0000","last_unstale":"2024-04-25T10:11:24.863968+0000","last_undegraded":"2024-04-25T10:11:24.863968+0000","last_fullsized":"2024-04-25T10:11:24.863968+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:39.127870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480923+0000","last_change":"2024-04-25T10:11:22.470977+0000","last_active":"2024-04-25T10:11:24.480923+0000","last_peered":"2024-04-25T10:11:24.480923+0000","last_clean":"2024-04-25T10:11:24.480923+0000","last_became_active":"2024-04-25T10:11:22.469808+0000","last_became_peered":"2024-04-25T10:11:22.469808+0000","last_unstale":"2024-04-25T10:11:24.480923+0000","last_undegraded":"2024-04-25T10:11:24.480923+0000","last_fullsized":"2024-04-25T10:11:24.480923+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:33.243743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968174+0000","last_change":"2024-04-25T10:11:23.514752+0000","last_active":"2024-04-25T10:11:25.968174+0000","last_peered":"2024-04-25T10:11:25.968174+0000","last_clean":"2024-04-25T10:11:25.968174+0000","last_became_active":"2024-04-25T10:11:23.514468+0000","last_became_peered":"2024-04-25T10:11:23.514468+0000","last_unstale":"2024-04-25T10:11:25.968174+0000","last_undegraded":"2024-04-25T10:11:25.968174+0000","last_fullsized":"2024-04-25T10:11:25.968174+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.677097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478648+0000","last_change":"2024-04-25T10:11:22.462923+0000","last_active":"2024-04-25T10:11:24.478648+0000","last_peered":"2024-04-25T10:11:24.478648+0000","last_clean":"2024-04-25T10:11:24.478648+0000","last_became_active":"2024-04-25T10:11:22.462615+0000","last_became_peered":"2024-04-25T10:11:22.462615+0000","last_unstale":"2024-04-25T10:11:24.478648+0000","last_undegraded":"2024-04-25T10:11:24.478648+0000","last_fullsized":"2024-04-25T10:11:24.478648+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:13.272502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479515+0000","last_change":"2024-04-25T10:11:23.489900+0000","last_active":"2024-04-25T10:11:24.479515+0000","last_peered":"2024-04-25T10:11:24.479515+0000","last_clean":"2024-04-25T10:11:24.479515+0000","last_became_active":"2024-04-25T10:11:23.489383+0000","last_became_peered":"2024-04-25T10:11:23.489383+0000","last_unstale":"2024-04-25T10:11:24.479515+0000","last_undegraded":"2024-04-25T10:11:24.479515+0000","last_fullsized":"2024-04-25T10:11:24.479515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.937312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967741+0000","last_change":"2024-04-25T10:11:22.466216+0000","last_active":"2024-04-25T10:11:25.967741+0000","last_peered":"2024-04-25T10:11:25.967741+0000","last_clean":"2024-04-25T10:11:25.967741+0000","last_became_active":"2024-04-25T10:11:22.465096+0000","last_became_peered":"2024-04-25T10:11:22.465096+0000","last_unstale":"2024-04-25T10:11:25.967741+0000","last_undegraded":"2024-04-25T10:11:25.967741+0000","last_fullsized":"2024-04-25T10:11:25.967741+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:33.243743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863752+0000","last_change":"2024-04-25T10:11:23.502474+0000","last_active":"2024-04-25T10:11:24.863752+0000","last_peered":"2024-04-25T10:11:24.863752+0000","last_clean":"2024-04-25T10:11:24.863752+0000","last_became_active":"2024-04-25T10:11:23.502027+0000","last_became_peered":"2024-04-25T10:11:23.502027+0000","last_unstale":"2024-04-25T10:11:24.863752+0000","last_undegraded":"2024-04-25T10:11:24.863752+0000","last_fullsized":"2024-04-25T10:11:24.863752+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:48.650402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474679+0000","last_change":"2024-04-25T10:11:22.470683+0000","last_active":"2024-04-25T10:11:24.474679+0000","last_peered":"2024-04-25T10:11:24.474679+0000","last_clean":"2024-04-25T10:11:24.474679+0000","last_became_active":"2024-04-25T10:11:22.470361+0000","last_became_peered":"2024-04-25T10:11:22.470361+0000","last_unstale":"2024-04-25T10:11:24.474679+0000","last_undegraded":"2024-04-25T10:11:24.474679+0000","last_fullsized":"2024-04-25T10:11:24.474679+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:17.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967639+0000","last_change":"2024-04-25T10:11:23.508721+0000","last_active":"2024-04-25T10:11:25.967639+0000","last_peered":"2024-04-25T10:11:25.967639+0000","last_clean":"2024-04-25T10:11:25.967639+0000","last_became_active":"2024-04-25T10:11:23.508450+0000","last_became_peered":"2024-04-25T10:11:23.508450+0000","last_unstale":"2024-04-25T10:11:25.967639+0000","last_undegraded":"2024-04-25T10:11:25.967639+0000","last_fullsized":"2024-04-25T10:11:25.967639+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.972961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.949937+0000","last_change":"2024-04-25T10:11:22.470874+0000","last_active":"2024-04-25T10:11:25.949937+0000","last_peered":"2024-04-25T10:11:25.949937+0000","last_clean":"2024-04-25T10:11:25.949937+0000","last_became_active":"2024-04-25T10:11:22.470265+0000","last_became_peered":"2024-04-25T10:11:22.470265+0000","last_unstale":"2024-04-25T10:11:25.949937+0000","last_undegraded":"2024-04-25T10:11:25.949937+0000","last_fullsized":"2024-04-25T10:11:25.949937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:23.973293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479393+0000","last_change":"2024-04-25T10:11:23.504435+0000","last_active":"2024-04-25T10:11:24.479393+0000","last_peered":"2024-04-25T10:11:24.479393+0000","last_clean":"2024-04-25T10:11:24.479393+0000","last_became_active":"2024-04-25T10:11:23.504141+0000","last_became_peered":"2024-04-25T10:11:23.504141+0000","last_unstale":"2024-04-25T10:11:24.479393+0000","last_undegraded":"2024-04-25T10:11:24.479393+0000","last_fullsized":"2024-04-25T10:11:24.479393+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:00.443531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521846+0000","last_change":"2024-04-25T10:11:22.473533+0000","last_active":"2024-04-25T10:11:24.521846+0000","last_peered":"2024-04-25T10:11:24.521846+0000","last_clean":"2024-04-25T10:11:24.521846+0000","last_became_active":"2024-04-25T10:11:22.473208+0000","last_became_peered":"2024-04-25T10:11:22.473208+0000","last_unstale":"2024-04-25T10:11:24.521846+0000","last_undegraded":"2024-04-25T10:11:24.521846+0000","last_fullsized":"2024-04-25T10:11:24.521846+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.661232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950916+0000","last_change":"2024-04-25T10:11:23.500410+0000","last_active":"2024-04-25T10:11:25.950916+0000","last_peered":"2024-04-25T10:11:25.950916+0000","last_clean":"2024-04-25T10:11:25.950916+0000","last_became_active":"2024-04-25T10:11:23.499418+0000","last_became_peered":"2024-04-25T10:11:23.499418+0000","last_unstale":"2024-04-25T10:11:25.950916+0000","last_undegraded":"2024-04-25T10:11:25.950916+0000","last_fullsized":"2024-04-25T10:11:25.950916+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.222396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475145+0000","last_change":"2024-04-25T10:11:22.465237+0000","last_active":"2024-04-25T10:11:24.475145+0000","last_peered":"2024-04-25T10:11:24.475145+0000","last_clean":"2024-04-25T10:11:24.475145+0000","last_became_active":"2024-04-25T10:11:22.464506+0000","last_became_peered":"2024-04-25T10:11:22.464506+0000","last_unstale":"2024-04-25T10:11:24.475145+0000","last_undegraded":"2024-04-25T10:11:24.475145+0000","last_fullsized":"2024-04-25T10:11:24.475145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:11.893976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475166+0000","last_change":"2024-04-25T10:11:23.512957+0000","last_active":"2024-04-25T10:11:24.475166+0000","last_peered":"2024-04-25T10:11:24.475166+0000","last_clean":"2024-04-25T10:11:24.475166+0000","last_became_active":"2024-04-25T10:11:23.512233+0000","last_became_peered":"2024-04-25T10:11:23.512233+0000","last_unstale":"2024-04-25T10:11:24.475166+0000","last_undegraded":"2024-04-25T10:11:24.475166+0000","last_fullsized":"2024-04-25T10:11:24.475166+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:41.977671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"97'3","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479786+0000","last_change":"2024-04-25T10:11:22.461965+0000","last_active":"2024-04-25T10:11:24.479786+0000","last_peered":"2024-04-25T10:11:24.479786+0000","last_clean":"2024-04-25T10:11:24.479786+0000","last_became_active":"2024-04-25T10:11:22.461218+0000","last_became_peered":"2024-04-25T10:11:22.461218+0000","last_unstale":"2024-04-25T10:11:24.479786+0000","last_undegraded":"2024-04-25T10:11:24.479786+0000","last_fullsized":"2024-04-25T10:11:24.479786+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:33.930030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968031+0000","last_change":"2024-04-25T10:11:23.499871+0000","last_active":"2024-04-25T10:11:25.968031+0000","last_peered":"2024-04-25T10:11:25.968031+0000","last_clean":"2024-04-25T10:11:25.968031+0000","last_became_active":"2024-04-25T10:11:23.499555+0000","last_became_peered":"2024-04-25T10:11:23.499555+0000","last_unstale":"2024-04-25T10:11:25.968031+0000","last_undegraded":"2024-04-25T10:11:25.968031+0000","last_fullsized":"2024-04-25T10:11:25.968031+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:46.821062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"97'6","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864178+0000","last_change":"2024-04-25T10:11:22.461868+0000","last_active":"2024-04-25T10:11:24.864178+0000","last_peered":"2024-04-25T10:11:24.864178+0000","last_clean":"2024-04-25T10:11:24.864178+0000","last_became_active":"2024-04-25T10:11:22.461381+0000","last_became_peered":"2024-04-25T10:11:22.461381+0000","last_unstale":"2024-04-25T10:11:24.864178+0000","last_undegraded":"2024-04-25T10:11:24.864178+0000","last_fullsized":"2024-04-25T10:11:24.864178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:33.748625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967812+0000","last_change":"2024-04-25T10:11:23.496310+0000","last_active":"2024-04-25T10:11:25.967812+0000","last_peered":"2024-04-25T10:11:25.967812+0000","last_clean":"2024-04-25T10:11:25.967812+0000","last_became_active":"2024-04-25T10:11:23.495975+0000","last_became_peered":"2024-04-25T10:11:23.495975+0000","last_unstale":"2024-04-25T10:11:25.967812+0000","last_undegraded":"2024-04-25T10:11:25.967812+0000","last_fullsized":"2024-04-25T10:11:25.967812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:46.519451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950177+0000","last_change":"2024-04-25T10:11:22.474187+0000","last_active":"2024-04-25T10:11:25.950177+0000","last_peered":"2024-04-25T10:11:25.950177+0000","last_clean":"2024-04-25T10:11:25.950177+0000","last_became_active":"2024-04-25T10:11:22.473866+0000","last_became_peered":"2024-04-25T10:11:22.473866+0000","last_unstale":"2024-04-25T10:11:25.950177+0000","last_undegraded":"2024-04-25T10:11:25.950177+0000","last_fullsized":"2024-04-25T10:11:25.950177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:17.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478658+0000","last_change":"2024-04-25T10:11:23.498466+0000","last_active":"2024-04-25T10:11:24.478658+0000","last_peered":"2024-04-25T10:11:24.478658+0000","last_clean":"2024-04-25T10:11:24.478658+0000","last_became_active":"2024-04-25T10:11:23.497025+0000","last_became_peered":"2024-04-25T10:11:23.497025+0000","last_unstale":"2024-04-25T10:11:24.478658+0000","last_undegraded":"2024-04-25T10:11:24.478658+0000","last_fullsized":"2024-04-25T10:11:24.478658+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:39.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863851+0000","last_change":"2024-04-25T10:11:22.469146+0000","last_active":"2024-04-25T10:11:24.863851+0000","last_peered":"2024-04-25T10:11:24.863851+0000","last_clean":"2024-04-25T10:11:24.863851+0000","last_became_active":"2024-04-25T10:11:22.468838+0000","last_became_peered":"2024-04-25T10:11:22.468838+0000","last_unstale":"2024-04-25T10:11:24.863851+0000","last_undegraded":"2024-04-25T10:11:24.863851+0000","last_fullsized":"2024-04-25T10:11:24.863851+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:46.341223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965886+0000","last_change":"2024-04-25T10:11:23.504259+0000","last_active":"2024-04-25T10:11:25.965886+0000","last_peered":"2024-04-25T10:11:25.965886+0000","last_clean":"2024-04-25T10:11:25.965886+0000","last_became_active":"2024-04-25T10:11:23.503980+0000","last_became_peered":"2024-04-25T10:11:23.503980+0000","last_unstale":"2024-04-25T10:11:25.965886+0000","last_undegraded":"2024-04-25T10:11:25.965886+0000","last_fullsized":"2024-04-25T10:11:25.965886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:27.440246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967382+0000","last_change":"2024-04-25T10:11:22.470404+0000","last_active":"2024-04-25T10:11:25.967382+0000","last_peered":"2024-04-25T10:11:25.967382+0000","last_clean":"2024-04-25T10:11:25.967382+0000","last_became_active":"2024-04-25T10:11:22.470111+0000","last_became_peered":"2024-04-25T10:11:22.470111+0000","last_unstale":"2024-04-25T10:11:25.967382+0000","last_undegraded":"2024-04-25T10:11:25.967382+0000","last_fullsized":"2024-04-25T10:11:25.967382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:32.968772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966320+0000","last_change":"2024-04-25T10:11:23.500138+0000","last_active":"2024-04-25T10:11:25.966320+0000","last_peered":"2024-04-25T10:11:25.966320+0000","last_clean":"2024-04-25T10:11:25.966320+0000","last_became_active":"2024-04-25T10:11:23.499378+0000","last_became_peered":"2024-04-25T10:11:23.499378+0000","last_unstale":"2024-04-25T10:11:25.966320+0000","last_undegraded":"2024-04-25T10:11:25.966320+0000","last_fullsized":"2024-04-25T10:11:25.966320+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.096148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"97'8","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967909+0000","last_change":"2024-04-25T10:11:22.465509+0000","last_active":"2024-04-25T10:11:25.967909+0000","last_peered":"2024-04-25T10:11:25.967909+0000","last_clean":"2024-04-25T10:11:25.967909+0000","last_became_active":"2024-04-25T10:11:22.465243+0000","last_became_peered":"2024-04-25T10:11:22.465243+0000","last_unstale":"2024-04-25T10:11:25.967909+0000","last_undegraded":"2024-04-25T10:11:25.967909+0000","last_fullsized":"2024-04-25T10:11:25.967909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:29.898591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864035+0000","last_change":"2024-04-25T10:11:23.500428+0000","last_active":"2024-04-25T10:11:24.864035+0000","last_peered":"2024-04-25T10:11:24.864035+0000","last_clean":"2024-04-25T10:11:24.864035+0000","last_became_active":"2024-04-25T10:11:23.500144+0000","last_became_peered":"2024-04-25T10:11:23.500144+0000","last_unstale":"2024-04-25T10:11:24.864035+0000","last_undegraded":"2024-04-25T10:11:24.864035+0000","last_fullsized":"2024-04-25T10:11:24.864035+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:30.247684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480875+0000","last_change":"2024-04-25T10:11:22.471844+0000","last_active":"2024-04-25T10:11:24.480875+0000","last_peered":"2024-04-25T10:11:24.480875+0000","last_clean":"2024-04-25T10:11:24.480875+0000","last_became_active":"2024-04-25T10:11:22.471304+0000","last_became_peered":"2024-04-25T10:11:22.471304+0000","last_unstale":"2024-04-25T10:11:24.480875+0000","last_undegraded":"2024-04-25T10:11:24.480875+0000","last_fullsized":"2024-04-25T10:11:24.480875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:29.865607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480738+0000","last_change":"2024-04-25T10:11:23.487951+0000","last_active":"2024-04-25T10:11:24.480738+0000","last_peered":"2024-04-25T10:11:24.480738+0000","last_clean":"2024-04-25T10:11:24.480738+0000","last_became_active":"2024-04-25T10:11:23.487535+0000","last_became_peered":"2024-04-25T10:11:23.487535+0000","last_unstale":"2024-04-25T10:11:24.480738+0000","last_undegraded":"2024-04-25T10:11:24.480738+0000","last_fullsized":"2024-04-25T10:11:24.480738+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950349+0000","last_change":"2024-04-25T10:11:22.476577+0000","last_active":"2024-04-25T10:11:25.950349+0000","last_peered":"2024-04-25T10:11:25.950349+0000","last_clean":"2024-04-25T10:11:25.950349+0000","last_became_active":"2024-04-25T10:11:22.476222+0000","last_became_peered":"2024-04-25T10:11:22.476222+0000","last_unstale":"2024-04-25T10:11:25.950349+0000","last_undegraded":"2024-04-25T10:11:25.950349+0000","last_fullsized":"2024-04-25T10:11:25.950349+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:55.296806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479787+0000","last_change":"2024-04-25T10:11:23.495714+0000","last_active":"2024-04-25T10:11:24.479787+0000","last_peered":"2024-04-25T10:11:24.479787+0000","last_clean":"2024-04-25T10:11:24.479787+0000","last_became_active":"2024-04-25T10:11:23.495066+0000","last_became_peered":"2024-04-25T10:11:23.495066+0000","last_unstale":"2024-04-25T10:11:24.479787+0000","last_undegraded":"2024-04-25T10:11:24.479787+0000","last_fullsized":"2024-04-25T10:11:24.479787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.091532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479884+0000","last_change":"2024-04-25T10:11:22.470350+0000","last_active":"2024-04-25T10:11:24.479884+0000","last_peered":"2024-04-25T10:11:24.479884+0000","last_clean":"2024-04-25T10:11:24.479884+0000","last_became_active":"2024-04-25T10:11:22.470032+0000","last_became_peered":"2024-04-25T10:11:22.470032+0000","last_unstale":"2024-04-25T10:11:24.479884+0000","last_undegraded":"2024-04-25T10:11:24.479884+0000","last_fullsized":"2024-04-25T10:11:24.479884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.503976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864599+0000","last_change":"2024-04-25T10:11:23.501944+0000","last_active":"2024-04-25T10:11:24.864599+0000","last_peered":"2024-04-25T10:11:24.864599+0000","last_clean":"2024-04-25T10:11:24.864599+0000","last_became_active":"2024-04-25T10:11:23.501477+0000","last_became_peered":"2024-04-25T10:11:23.501477+0000","last_unstale":"2024-04-25T10:11:24.864599+0000","last_undegraded":"2024-04-25T10:11:24.864599+0000","last_fullsized":"2024-04-25T10:11:24.864599+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.961651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520557+0000","last_change":"2024-04-25T10:11:22.473753+0000","last_active":"2024-04-25T10:11:24.520557+0000","last_peered":"2024-04-25T10:11:24.520557+0000","last_clean":"2024-04-25T10:11:24.520557+0000","last_became_active":"2024-04-25T10:11:22.473434+0000","last_became_peered":"2024-04-25T10:11:22.473434+0000","last_unstale":"2024-04-25T10:11:24.520557+0000","last_undegraded":"2024-04-25T10:11:24.520557+0000","last_fullsized":"2024-04-25T10:11:24.520557+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:53.765476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520767+0000","last_change":"2024-04-25T10:11:23.496038+0000","last_active":"2024-04-25T10:11:24.520767+0000","last_peered":"2024-04-25T10:11:24.520767+0000","last_clean":"2024-04-25T10:11:24.520767+0000","last_became_active":"2024-04-25T10:11:23.495699+0000","last_became_peered":"2024-04-25T10:11:23.495699+0000","last_unstale":"2024-04-25T10:11:24.520767+0000","last_undegraded":"2024-04-25T10:11:24.520767+0000","last_fullsized":"2024-04-25T10:11:24.520767+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:33.466669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966036+0000","last_change":"2024-04-25T10:11:22.462653+0000","last_active":"2024-04-25T10:11:25.966036+0000","last_peered":"2024-04-25T10:11:25.966036+0000","last_clean":"2024-04-25T10:11:25.966036+0000","last_became_active":"2024-04-25T10:11:22.462355+0000","last_became_peered":"2024-04-25T10:11:22.462355+0000","last_unstale":"2024-04-25T10:11:25.966036+0000","last_undegraded":"2024-04-25T10:11:25.966036+0000","last_fullsized":"2024-04-25T10:11:25.966036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:49.454297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521025+0000","last_change":"2024-04-25T10:11:23.504455+0000","last_active":"2024-04-25T10:11:24.521025+0000","last_peered":"2024-04-25T10:11:24.521025+0000","last_clean":"2024-04-25T10:11:24.521025+0000","last_became_active":"2024-04-25T10:11:23.504135+0000","last_became_peered":"2024-04-25T10:11:23.504135+0000","last_unstale":"2024-04-25T10:11:24.521025+0000","last_undegraded":"2024-04-25T10:11:24.521025+0000","last_fullsized":"2024-04-25T10:11:24.521025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.347401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520952+0000","last_change":"2024-04-25T10:11:22.476111+0000","last_active":"2024-04-25T10:11:24.520952+0000","last_peered":"2024-04-25T10:11:24.520952+0000","last_clean":"2024-04-25T10:11:24.520952+0000","last_became_active":"2024-04-25T10:11:22.474623+0000","last_became_peered":"2024-04-25T10:11:22.474623+0000","last_unstale":"2024-04-25T10:11:24.520952+0000","last_undegraded":"2024-04-25T10:11:24.520952+0000","last_fullsized":"2024-04-25T10:11:24.520952+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:21.062051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863947+0000","last_change":"2024-04-25T10:11:23.490610+0000","last_active":"2024-04-25T10:11:24.863947+0000","last_peered":"2024-04-25T10:11:24.863947+0000","last_clean":"2024-04-25T10:11:24.863947+0000","last_became_active":"2024-04-25T10:11:23.489790+0000","last_became_peered":"2024-04-25T10:11:23.489790+0000","last_unstale":"2024-04-25T10:11:24.863947+0000","last_undegraded":"2024-04-25T10:11:24.863947+0000","last_fullsized":"2024-04-25T10:11:24.863947+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965445+0000","last_change":"2024-04-25T10:11:22.471275+0000","last_active":"2024-04-25T10:11:25.965445+0000","last_peered":"2024-04-25T10:11:25.965445+0000","last_clean":"2024-04-25T10:11:25.965445+0000","last_became_active":"2024-04-25T10:11:22.471004+0000","last_became_peered":"2024-04-25T10:11:22.471004+0000","last_unstale":"2024-04-25T10:11:25.965445+0000","last_undegraded":"2024-04-25T10:11:25.965445+0000","last_fullsized":"2024-04-25T10:11:25.965445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.869983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480482+0000","last_change":"2024-04-25T10:11:23.498595+0000","last_active":"2024-04-25T10:11:24.480482+0000","last_peered":"2024-04-25T10:11:24.480482+0000","last_clean":"2024-04-25T10:11:24.480482+0000","last_became_active":"2024-04-25T10:11:23.497684+0000","last_became_peered":"2024-04-25T10:11:23.497684+0000","last_unstale":"2024-04-25T10:11:24.480482+0000","last_undegraded":"2024-04-25T10:11:24.480482+0000","last_fullsized":"2024-04-25T10:11:24.480482+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:00.443531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950639+0000","last_change":"2024-04-25T10:11:22.477558+0000","last_active":"2024-04-25T10:11:25.950639+0000","last_peered":"2024-04-25T10:11:25.950639+0000","last_clean":"2024-04-25T10:11:25.950639+0000","last_became_active":"2024-04-25T10:11:22.476031+0000","last_became_peered":"2024-04-25T10:11:22.476031+0000","last_unstale":"2024-04-25T10:11:25.950639+0000","last_undegraded":"2024-04-25T10:11:25.950639+0000","last_fullsized":"2024-04-25T10:11:25.950639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.891229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864672+0000","last_change":"2024-04-25T10:11:23.498208+0000","last_active":"2024-04-25T10:11:24.864672+0000","last_peered":"2024-04-25T10:11:24.864672+0000","last_clean":"2024-04-25T10:11:24.864672+0000","last_became_active":"2024-04-25T10:11:23.497917+0000","last_became_peered":"2024-04-25T10:11:23.497917+0000","last_unstale":"2024-04-25T10:11:24.864672+0000","last_undegraded":"2024-04-25T10:11:24.864672+0000","last_fullsized":"2024-04-25T10:11:24.864672+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:45.774608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"98'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480212+0000","last_change":"2024-04-25T10:11:22.476964+0000","last_active":"2024-04-25T10:11:24.480212+0000","last_peered":"2024-04-25T10:11:24.480212+0000","last_clean":"2024-04-25T10:11:24.480212+0000","last_became_active":"2024-04-25T10:11:22.476572+0000","last_became_peered":"2024-04-25T10:11:22.476572+0000","last_unstale":"2024-04-25T10:11:24.480212+0000","last_undegraded":"2024-04-25T10:11:24.480212+0000","last_fullsized":"2024-04-25T10:11:24.480212+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.402891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863704+0000","last_change":"2024-04-25T10:11:23.494385+0000","last_active":"2024-04-25T10:11:24.863704+0000","last_peered":"2024-04-25T10:11:24.863704+0000","last_clean":"2024-04-25T10:11:24.863704+0000","last_became_active":"2024-04-25T10:11:23.494070+0000","last_became_peered":"2024-04-25T10:11:23.494070+0000","last_unstale":"2024-04-25T10:11:24.863704+0000","last_undegraded":"2024-04-25T10:11:24.863704+0000","last_fullsized":"2024-04-25T10:11:24.863704+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:10.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479943+0000","last_change":"2024-04-25T10:11:22.463847+0000","last_active":"2024-04-25T10:11:24.479943+0000","last_peered":"2024-04-25T10:11:24.479943+0000","last_clean":"2024-04-25T10:11:24.479943+0000","last_became_active":"2024-04-25T10:11:22.463559+0000","last_became_peered":"2024-04-25T10:11:22.463559+0000","last_unstale":"2024-04-25T10:11:24.479943+0000","last_undegraded":"2024-04-25T10:11:24.479943+0000","last_fullsized":"2024-04-25T10:11:24.479943+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.936278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521840+0000","last_change":"2024-04-25T10:11:23.499509+0000","last_active":"2024-04-25T10:11:24.521840+0000","last_peered":"2024-04-25T10:11:24.521840+0000","last_clean":"2024-04-25T10:11:24.521840+0000","last_became_active":"2024-04-25T10:11:23.498796+0000","last_became_peered":"2024-04-25T10:11:23.498796+0000","last_unstale":"2024-04-25T10:11:24.521840+0000","last_undegraded":"2024-04-25T10:11:24.521840+0000","last_fullsized":"2024-04-25T10:11:24.521840+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:12.174658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480989+0000","last_change":"2024-04-25T10:11:22.458990+0000","last_active":"2024-04-25T10:11:24.480989+0000","last_peered":"2024-04-25T10:11:24.480989+0000","last_clean":"2024-04-25T10:11:24.480989+0000","last_became_active":"2024-04-25T10:11:22.458519+0000","last_became_peered":"2024-04-25T10:11:22.458519+0000","last_unstale":"2024-04-25T10:11:24.480989+0000","last_undegraded":"2024-04-25T10:11:24.480989+0000","last_fullsized":"2024-04-25T10:11:24.480989+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:28.483485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966106+0000","last_change":"2024-04-25T10:11:23.495054+0000","last_active":"2024-04-25T10:11:25.966106+0000","last_peered":"2024-04-25T10:11:25.966106+0000","last_clean":"2024-04-25T10:11:25.966106+0000","last_became_active":"2024-04-25T10:11:23.494672+0000","last_became_peered":"2024-04-25T10:11:23.494672+0000","last_unstale":"2024-04-25T10:11:25.966106+0000","last_undegraded":"2024-04-25T10:11:25.966106+0000","last_fullsized":"2024-04-25T10:11:25.966106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.959511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864095+0000","last_change":"2024-04-25T10:11:22.469251+0000","last_active":"2024-04-25T10:11:24.864095+0000","last_peered":"2024-04-25T10:11:24.864095+0000","last_clean":"2024-04-25T10:11:24.864095+0000","last_became_active":"2024-04-25T10:11:22.468896+0000","last_became_peered":"2024-04-25T10:11:22.468896+0000","last_unstale":"2024-04-25T10:11:24.864095+0000","last_undegraded":"2024-04-25T10:11:24.864095+0000","last_fullsized":"2024-04-25T10:11:24.864095+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:40.961806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474640+0000","last_change":"2024-04-25T10:11:23.511617+0000","last_active":"2024-04-25T10:11:24.474640+0000","last_peered":"2024-04-25T10:11:24.474640+0000","last_clean":"2024-04-25T10:11:24.474640+0000","last_became_active":"2024-04-25T10:11:23.511352+0000","last_became_peered":"2024-04-25T10:11:23.511352+0000","last_unstale":"2024-04-25T10:11:24.474640+0000","last_undegraded":"2024-04-25T10:11:24.474640+0000","last_fullsized":"2024-04-25T10:11:24.474640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.588960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521129+0000","last_change":"2024-04-25T10:11:22.472035+0000","last_active":"2024-04-25T10:11:24.521129+0000","last_peered":"2024-04-25T10:11:24.521129+0000","last_clean":"2024-04-25T10:11:24.521129+0000","last_became_active":"2024-04-25T10:11:22.471686+0000","last_became_peered":"2024-04-25T10:11:22.471686+0000","last_unstale":"2024-04-25T10:11:24.521129+0000","last_undegraded":"2024-04-25T10:11:24.521129+0000","last_fullsized":"2024-04-25T10:11:24.521129+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:45.805197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479618+0000","last_change":"2024-04-25T10:11:23.503580+0000","last_active":"2024-04-25T10:11:24.479618+0000","last_peered":"2024-04-25T10:11:24.479618+0000","last_clean":"2024-04-25T10:11:24.479618+0000","last_became_active":"2024-04-25T10:11:23.503299+0000","last_became_peered":"2024-04-25T10:11:23.503299+0000","last_unstale":"2024-04-25T10:11:24.479618+0000","last_undegraded":"2024-04-25T10:11:24.479618+0000","last_fullsized":"2024-04-25T10:11:24.479618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:37.491564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479662+0000","last_change":"2024-04-25T10:11:22.458531+0000","last_active":"2024-04-25T10:11:24.479662+0000","last_peered":"2024-04-25T10:11:24.479662+0000","last_clean":"2024-04-25T10:11:24.479662+0000","last_became_active":"2024-04-25T10:11:22.457900+0000","last_became_peered":"2024-04-25T10:11:22.457900+0000","last_unstale":"2024-04-25T10:11:24.479662+0000","last_undegraded":"2024-04-25T10:11:24.479662+0000","last_fullsized":"2024-04-25T10:11:24.479662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:32.968772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864134+0000","last_change":"2024-04-25T10:11:23.498761+0000","last_active":"2024-04-25T10:11:24.864134+0000","last_peered":"2024-04-25T10:11:24.864134+0000","last_clean":"2024-04-25T10:11:24.864134+0000","last_became_active":"2024-04-25T10:11:23.498483+0000","last_became_peered":"2024-04-25T10:11:23.498483+0000","last_unstale":"2024-04-25T10:11:24.864134+0000","last_undegraded":"2024-04-25T10:11:24.864134+0000","last_fullsized":"2024-04-25T10:11:24.864134+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:31.072700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.949962+0000","last_change":"2024-04-25T10:11:22.470587+0000","last_active":"2024-04-25T10:11:25.949962+0000","last_peered":"2024-04-25T10:11:25.949962+0000","last_clean":"2024-04-25T10:11:25.949962+0000","last_became_active":"2024-04-25T10:11:22.469775+0000","last_became_peered":"2024-04-25T10:11:22.469775+0000","last_unstale":"2024-04-25T10:11:25.949962+0000","last_undegraded":"2024-04-25T10:11:25.949962+0000","last_fullsized":"2024-04-25T10:11:25.949962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:11.893976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950687+0000","last_change":"2024-04-25T10:11:23.517824+0000","last_active":"2024-04-25T10:11:25.950687+0000","last_peered":"2024-04-25T10:11:25.950687+0000","last_clean":"2024-04-25T10:11:25.950687+0000","last_became_active":"2024-04-25T10:11:23.517479+0000","last_became_peered":"2024-04-25T10:11:23.517479+0000","last_unstale":"2024-04-25T10:11:25.950687+0000","last_undegraded":"2024-04-25T10:11:25.950687+0000","last_fullsized":"2024-04-25T10:11:25.950687+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:47.357088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968304+0000","last_change":"2024-04-25T10:11:22.474875+0000","last_active":"2024-04-25T10:11:25.968304+0000","last_peered":"2024-04-25T10:11:25.968304+0000","last_clean":"2024-04-25T10:11:25.968304+0000","last_became_active":"2024-04-25T10:11:22.474581+0000","last_became_peered":"2024-04-25T10:11:22.474581+0000","last_unstale":"2024-04-25T10:11:25.968304+0000","last_undegraded":"2024-04-25T10:11:25.968304+0000","last_fullsized":"2024-04-25T10:11:25.968304+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:29.865607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":318,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950229+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:11:25.950229+0000","last_peered":"2024-04-25T10:11:25.950229+0000","last_clean":"2024-04-25T10:11:25.950229+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:11:25.950229+0000","last_undegraded":"2024-04-25T10:11:25.950229+0000","last_fullsized":"2024-04-25T10:11:25.950229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50996,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320844,"statfs":{"total":96636764160,"available":96584544256,"internally_reserved":0,"allocated":1810432,"data_stored":1153237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574967,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50428,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321412,"statfs":{"total":96636764160,"available":96585125888,"internally_reserved":0,"allocated":1228800,"data_stored":565029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58736,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313104,"statfs":{"total":96636764160,"available":96576618496,"internally_reserved":0,"allocated":1216512,"data_stored":562973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574967,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574966,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54572,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317268,"statfs":{"total":96636764160,"available":96580882432,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574967,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50412,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321428,"statfs":{"total":96636764160,"available":96585142272,"internally_reserved":0,"allocated":1212416,"data_stored":562883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574967,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55172,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316668,"statfs":{"total":96636764160,"available":96580268032,"internally_reserved":0,"allocated":1826816,"data_stored":1155499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:32.768 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:11:32.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.973+0000 7fd099000640 1 Processor -- start 2024-04-25T10:11:32.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.973+0000 7fd099000640 1 -- start start 2024-04-25T10:11:32.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd099000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd094144440 0x7fd094070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd099000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094071530 0x7fd0940cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd099000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd09415bf30 con 0x7fd0940cd860 2024-04-25T10:11:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd099000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd09415c860 con 0x7fd094144440 2024-04-25T10:11:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd099000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd094071970 con 0x7fd094071530 2024-04-25T10:11:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd093400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd094144440 0x7fd094070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd093400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd094144440 0x7fd094070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:33308/0 (socket says 172.21.15.165:33308) 2024-04-25T10:11:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.975+0000 7fd093400640 1 -- 172.21.15.165:0/3157890957 learned_addr learned my addr 172.21.15.165:0/3157890957 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.976+0000 7fd092a00640 1 --2- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094071530 0x7fd0940cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.976+0000 7fd093400640 1 -- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094071530 msgr2=0x7fd0940cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.976+0000 7fd093400640 1 --2- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094071530 0x7fd0940cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.976+0000 7fd093400640 1 -- 172.21.15.165:0/3157890957 >> v1:172.21.15.172:6789/0 conn(0x7fd0940cd860 legacy=0x7fd09406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.976+0000 7fd093400640 1 -- 172.21.15.165:0/3157890957 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd09415c210 con 0x7fd094144440 2024-04-25T10:11:32.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.976+0000 7fd092000640 1 -- 172.21.15.165:0/3157890957 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938808889 0 0) 0x7fd09415bf30 con 0x7fd0940cd860 2024-04-25T10:11:32.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.978+0000 7fd093400640 1 --2- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd094144440 0x7fd094070ff0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd084009870 tx=0x7fd08402a300 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a47972cd30dccc28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.978+0000 7fd092000640 1 -- 172.21.15.165:0/3157890957 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084037070 con 0x7fd094144440 2024-04-25T10:11:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.978+0000 7fd092000640 1 -- 172.21.15.165:0/3157890957 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08402ae50 con 0x7fd094144440 2024-04-25T10:11:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.978+0000 7fd092000640 1 -- 172.21.15.165:0/3157890957 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084032860 con 0x7fd094144440 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 -- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd094144440 msgr2=0x7fd094070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 --2- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd094144440 0x7fd094070ff0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd084009870 tx=0x7fd08402a300 comp rx=0 tx=0).stop 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 -- 172.21.15.165:0/3157890957 shutdown_connections 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 --2- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094071530 0x7fd0940cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 --2- 172.21.15.165:0/3157890957 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd094144440 0x7fd094070ff0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 -- 172.21.15.165:0/3157890957 >> 172.21.15.165:0/3157890957 conn(0x7fd09406abe0 msgr2=0x7fd09406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 -- 172.21.15.165:0/3157890957 shutdown_connections 2024-04-25T10:11:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.979+0000 7fd099000640 1 -- 172.21.15.165:0/3157890957 wait complete. 2024-04-25T10:11:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.981+0000 7fd099000640 1 Processor -- start 2024-04-25T10:11:32.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.981+0000 7fd099000640 1 -- start start 2024-04-25T10:11:32.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd099000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 0x7fd0940ccd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd099000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0940cd860 0x7fd0941e51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd099000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094144440 0x7fd0940c7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd099000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940cdcd0 con 0x7fd094071530 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd099000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09415bc30 con 0x7fd0940cd860 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd099000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0940cde50 con 0x7fd094144440 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd093400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 0x7fd0940ccd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd093400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 0x7fd0940ccd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35846/0 (socket says 172.21.15.165:35846) 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.982+0000 7fd093400640 1 -- 172.21.15.165:0/168990459 learned_addr learned my addr 172.21.15.165:0/168990459 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd092a00640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0940cd860 0x7fd0941e51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093e00640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094144440 0x7fd0940c7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093400640 1 -- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094144440 msgr2=0x7fd0940c7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093400640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094144440 0x7fd0940c7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093400640 1 -- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0940cd860 msgr2=0x7fd0941e51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093400640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0940cd860 0x7fd0941e51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:32.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093400640 1 -- 172.21.15.165:0/168990459 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0941e3e40 con 0x7fd094071530 2024-04-25T10:11:32.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd093400640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 0x7fd0940ccd40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd084000c00 tx=0x7fd0840046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084041070 con 0x7fd094071530 2024-04-25T10:11:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0941e4120 con 0x7fd094071530 2024-04-25T10:11:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08403c660 con 0x7fd094071530 2024-04-25T10:11:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.983+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd084037040 con 0x7fd094071530 2024-04-25T10:11:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.984+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0941e46e0 con 0x7fd094071530 2024-04-25T10:11:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.985+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd094073990 con 0x7fd094071530 2024-04-25T10:11:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.986+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd084044070 con 0x7fd094071530 2024-04-25T10:11:32.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.987+0000 7fd08b400640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd07406b4d0 0x7fd07406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:32.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.987+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0840aaaa0 con 0x7fd094071530 2024-04-25T10:11:32.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.987+0000 7fd092a00640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd07406b4d0 0x7fd07406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:32.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.988+0000 7fd092a00640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd07406b4d0 0x7fd07406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd09406eac0 tx=0x7fd080008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:32.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:32.991+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd0840798c0 con 0x7fd094071530 2024-04-25T10:11:33.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.182+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd0940c53c0 con 0x7fd07406b4d0 2024-04-25T10:11:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.201+0000 7fd08b400640 1 -- 172.21.15.165:0/168990459 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+251964 (secure 0 0 0) 0x7fd0940c53c0 con 0x7fd07406b4d0 2024-04-25T10:11:33.209 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:33.211 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:11:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd07406b4d0 msgr2=0x7fd07406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd07406b4d0 0x7fd07406d990 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd09406eac0 tx=0x7fd080008040 comp rx=0 tx=0).stop 2024-04-25T10:11:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 msgr2=0x7fd0940ccd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 0x7fd0940ccd40 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7fd084000c00 tx=0x7fd0840046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 shutdown_connections 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd07406b4d0 0x7fd07406d990 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd094144440 0x7fd0940c7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0940cd860 0x7fd0941e51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 --2- 172.21.15.165:0/168990459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd094071530 0x7fd0940ccd40 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.206+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 >> 172.21.15.165:0/168990459 conn(0x7fd09406abe0 msgr2=0x7fd0940c56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.207+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 shutdown_connections 2024-04-25T10:11:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.207+0000 7fd099000640 1 -- 172.21.15.165:0/168990459 wait complete. 2024-04-25T10:11:33.230 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":381,"stamp":"2024-04-25T10:11:32.323173+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":421140,"kb_used_data":10664,"kb_used_omap":12,"kb_used_meta":410355,"kb_avail":754553580,"statfs":{"total":773094113280,"available":772662865920,"internally_reserved":0,"allocated":10919936,"data_stored":5688060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":420204082},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.857970"},"pg_stats":[{"pgid":"14.3f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475221+0000","last_change":"2024-04-25T10:11:23.514302+0000","last_active":"2024-04-25T10:11:24.475221+0000","last_peered":"2024-04-25T10:11:24.475221+0000","last_clean":"2024-04-25T10:11:24.475221+0000","last_became_active":"2024-04-25T10:11:23.509239+0000","last_became_peered":"2024-04-25T10:11:23.509239+0000","last_unstale":"2024-04-25T10:11:24.475221+0000","last_undegraded":"2024-04-25T10:11:24.475221+0000","last_fullsized":"2024-04-25T10:11:24.475221+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:47.357088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521483+0000","last_change":"2024-04-25T10:11:23.511726+0000","last_active":"2024-04-25T10:11:24.521483+0000","last_peered":"2024-04-25T10:11:24.521483+0000","last_clean":"2024-04-25T10:11:24.521483+0000","last_became_active":"2024-04-25T10:11:23.511477+0000","last_became_peered":"2024-04-25T10:11:23.511477+0000","last_unstale":"2024-04-25T10:11:24.521483+0000","last_undegraded":"2024-04-25T10:11:24.521483+0000","last_fullsized":"2024-04-25T10:11:24.521483+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:20:21.609016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474979+0000","last_change":"2024-04-25T10:11:23.504635+0000","last_active":"2024-04-25T10:11:24.474979+0000","last_peered":"2024-04-25T10:11:24.474979+0000","last_clean":"2024-04-25T10:11:24.474979+0000","last_became_active":"2024-04-25T10:11:23.504397+0000","last_became_peered":"2024-04-25T10:11:23.504397+0000","last_unstale":"2024-04-25T10:11:24.474979+0000","last_undegraded":"2024-04-25T10:11:24.474979+0000","last_fullsized":"2024-04-25T10:11:24.474979+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:58:15.008119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.3c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968039+0000","last_change":"2024-04-25T10:11:23.501516+0000","last_active":"2024-04-25T10:11:25.968039+0000","last_peered":"2024-04-25T10:11:25.968039+0000","last_clean":"2024-04-25T10:11:25.968039+0000","last_became_active":"2024-04-25T10:11:23.501180+0000","last_became_peered":"2024-04-25T10:11:23.501180+0000","last_unstale":"2024-04-25T10:11:25.968039+0000","last_undegraded":"2024-04-25T10:11:25.968039+0000","last_fullsized":"2024-04-25T10:11:25.968039+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:54.781341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478461+0000","last_change":"2024-04-25T10:11:23.493320+0000","last_active":"2024-04-25T10:11:24.478461+0000","last_peered":"2024-04-25T10:11:24.478461+0000","last_clean":"2024-04-25T10:11:24.478461+0000","last_became_active":"2024-04-25T10:11:23.492988+0000","last_became_peered":"2024-04-25T10:11:23.492988+0000","last_unstale":"2024-04-25T10:11:24.478461+0000","last_undegraded":"2024-04-25T10:11:24.478461+0000","last_fullsized":"2024-04-25T10:11:24.478461+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:43.297903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863718+0000","last_change":"2024-04-25T10:11:23.503887+0000","last_active":"2024-04-25T10:11:24.863718+0000","last_peered":"2024-04-25T10:11:24.863718+0000","last_clean":"2024-04-25T10:11:24.863718+0000","last_became_active":"2024-04-25T10:11:23.503580+0000","last_became_peered":"2024-04-25T10:11:23.503580+0000","last_unstale":"2024-04-25T10:11:24.863718+0000","last_undegraded":"2024-04-25T10:11:24.863718+0000","last_fullsized":"2024-04-25T10:11:24.863718+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:00.002046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950146+0000","last_change":"2024-04-25T10:11:23.506780+0000","last_active":"2024-04-25T10:11:25.950146+0000","last_peered":"2024-04-25T10:11:25.950146+0000","last_clean":"2024-04-25T10:11:25.950146+0000","last_became_active":"2024-04-25T10:11:23.506380+0000","last_became_peered":"2024-04-25T10:11:23.506380+0000","last_unstale":"2024-04-25T10:11:25.950146+0000","last_undegraded":"2024-04-25T10:11:25.950146+0000","last_fullsized":"2024-04-25T10:11:25.950146+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:35.118925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967401+0000","last_change":"2024-04-25T10:11:23.496772+0000","last_active":"2024-04-25T10:11:25.967401+0000","last_peered":"2024-04-25T10:11:25.967401+0000","last_clean":"2024-04-25T10:11:25.967401+0000","last_became_active":"2024-04-25T10:11:23.496466+0000","last_became_peered":"2024-04-25T10:11:23.496466+0000","last_unstale":"2024-04-25T10:11:25.967401+0000","last_undegraded":"2024-04-25T10:11:25.967401+0000","last_fullsized":"2024-04-25T10:11:25.967401+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480543+0000","last_change":"2024-04-25T10:11:23.496579+0000","last_active":"2024-04-25T10:11:24.480543+0000","last_peered":"2024-04-25T10:11:24.480543+0000","last_clean":"2024-04-25T10:11:24.480543+0000","last_became_active":"2024-04-25T10:11:23.495543+0000","last_became_peered":"2024-04-25T10:11:23.495543+0000","last_unstale":"2024-04-25T10:11:24.480543+0000","last_undegraded":"2024-04-25T10:11:24.480543+0000","last_fullsized":"2024-04-25T10:11:24.480543+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:46.821062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966387+0000","last_change":"2024-04-25T10:11:23.507660+0000","last_active":"2024-04-25T10:11:25.966387+0000","last_peered":"2024-04-25T10:11:25.966387+0000","last_clean":"2024-04-25T10:11:25.966387+0000","last_became_active":"2024-04-25T10:11:23.507339+0000","last_became_peered":"2024-04-25T10:11:23.507339+0000","last_unstale":"2024-04-25T10:11:25.966387+0000","last_undegraded":"2024-04-25T10:11:25.966387+0000","last_fullsized":"2024-04-25T10:11:25.966387+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:59.303357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479723+0000","last_change":"2024-04-25T10:11:23.495970+0000","last_active":"2024-04-25T10:11:24.479723+0000","last_peered":"2024-04-25T10:11:24.479723+0000","last_clean":"2024-04-25T10:11:24.479723+0000","last_became_active":"2024-04-25T10:11:23.495635+0000","last_became_peered":"2024-04-25T10:11:23.495635+0000","last_unstale":"2024-04-25T10:11:24.479723+0000","last_undegraded":"2024-04-25T10:11:24.479723+0000","last_fullsized":"2024-04-25T10:11:24.479723+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:46.519451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965939+0000","last_change":"2024-04-25T10:11:23.497428+0000","last_active":"2024-04-25T10:11:25.965939+0000","last_peered":"2024-04-25T10:11:25.965939+0000","last_clean":"2024-04-25T10:11:25.965939+0000","last_became_active":"2024-04-25T10:11:23.495767+0000","last_became_peered":"2024-04-25T10:11:23.495767+0000","last_unstale":"2024-04-25T10:11:25.965939+0000","last_undegraded":"2024-04-25T10:11:25.965939+0000","last_fullsized":"2024-04-25T10:11:25.965939+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:56.257951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968172+0000","last_change":"2024-04-25T10:11:23.505393+0000","last_active":"2024-04-25T10:11:25.968172+0000","last_peered":"2024-04-25T10:11:25.968172+0000","last_clean":"2024-04-25T10:11:25.968172+0000","last_became_active":"2024-04-25T10:11:23.502745+0000","last_became_peered":"2024-04-25T10:11:23.502745+0000","last_unstale":"2024-04-25T10:11:25.968172+0000","last_undegraded":"2024-04-25T10:11:25.968172+0000","last_fullsized":"2024-04-25T10:11:25.968172+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:22.077916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478359+0000","last_change":"2024-04-25T10:11:23.504530+0000","last_active":"2024-04-25T10:11:24.478359+0000","last_peered":"2024-04-25T10:11:24.478359+0000","last_clean":"2024-04-25T10:11:24.478359+0000","last_became_active":"2024-04-25T10:11:23.504230+0000","last_became_peered":"2024-04-25T10:11:23.504230+0000","last_unstale":"2024-04-25T10:11:24.478359+0000","last_undegraded":"2024-04-25T10:11:24.478359+0000","last_fullsized":"2024-04-25T10:11:24.478359+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:30:31.827948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968022+0000","last_change":"2024-04-25T10:11:23.514234+0000","last_active":"2024-04-25T10:11:25.968022+0000","last_peered":"2024-04-25T10:11:25.968022+0000","last_clean":"2024-04-25T10:11:25.968022+0000","last_became_active":"2024-04-25T10:11:23.513691+0000","last_became_peered":"2024-04-25T10:11:23.513691+0000","last_unstale":"2024-04-25T10:11:25.968022+0000","last_undegraded":"2024-04-25T10:11:25.968022+0000","last_fullsized":"2024-04-25T10:11:25.968022+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.091532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864271+0000","last_change":"2024-04-25T10:11:23.496506+0000","last_active":"2024-04-25T10:11:24.864271+0000","last_peered":"2024-04-25T10:11:24.864271+0000","last_clean":"2024-04-25T10:11:24.864271+0000","last_became_active":"2024-04-25T10:11:23.496229+0000","last_became_peered":"2024-04-25T10:11:23.496229+0000","last_unstale":"2024-04-25T10:11:24.864271+0000","last_undegraded":"2024-04-25T10:11:24.864271+0000","last_fullsized":"2024-04-25T10:11:24.864271+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:23:40.275078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863850+0000","last_change":"2024-04-25T10:11:23.518147+0000","last_active":"2024-04-25T10:11:24.863850+0000","last_peered":"2024-04-25T10:11:24.863850+0000","last_clean":"2024-04-25T10:11:24.863850+0000","last_became_active":"2024-04-25T10:11:23.517364+0000","last_became_peered":"2024-04-25T10:11:23.517364+0000","last_unstale":"2024-04-25T10:11:24.863850+0000","last_undegraded":"2024-04-25T10:11:24.863850+0000","last_fullsized":"2024-04-25T10:11:24.863850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:55.872365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475072+0000","last_change":"2024-04-25T10:11:23.506788+0000","last_active":"2024-04-25T10:11:24.475072+0000","last_peered":"2024-04-25T10:11:24.475072+0000","last_clean":"2024-04-25T10:11:24.475072+0000","last_became_active":"2024-04-25T10:11:23.506508+0000","last_became_peered":"2024-04-25T10:11:23.506508+0000","last_unstale":"2024-04-25T10:11:24.475072+0000","last_undegraded":"2024-04-25T10:11:24.475072+0000","last_fullsized":"2024-04-25T10:11:24.475072+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:35.118925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479440+0000","last_change":"2024-04-25T10:11:23.505949+0000","last_active":"2024-04-25T10:11:24.479440+0000","last_peered":"2024-04-25T10:11:24.479440+0000","last_clean":"2024-04-25T10:11:24.479440+0000","last_became_active":"2024-04-25T10:11:23.501690+0000","last_became_peered":"2024-04-25T10:11:23.501690+0000","last_unstale":"2024-04-25T10:11:24.479440+0000","last_undegraded":"2024-04-25T10:11:24.479440+0000","last_fullsized":"2024-04-25T10:11:24.479440+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.972961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968249+0000","last_change":"2024-04-25T10:11:23.514427+0000","last_active":"2024-04-25T10:11:25.968249+0000","last_peered":"2024-04-25T10:11:25.968249+0000","last_clean":"2024-04-25T10:11:25.968249+0000","last_became_active":"2024-04-25T10:11:23.514184+0000","last_became_peered":"2024-04-25T10:11:23.514184+0000","last_unstale":"2024-04-25T10:11:25.968249+0000","last_undegraded":"2024-04-25T10:11:25.968249+0000","last_fullsized":"2024-04-25T10:11:25.968249+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:35:36.868831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478219+0000","last_change":"2024-04-25T10:11:23.501619+0000","last_active":"2024-04-25T10:11:24.478219+0000","last_peered":"2024-04-25T10:11:24.478219+0000","last_clean":"2024-04-25T10:11:24.478219+0000","last_became_active":"2024-04-25T10:11:23.500466+0000","last_became_peered":"2024-04-25T10:11:23.500466+0000","last_unstale":"2024-04-25T10:11:24.478219+0000","last_undegraded":"2024-04-25T10:11:24.478219+0000","last_fullsized":"2024-04-25T10:11:24.478219+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:50.068988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478814+0000","last_change":"2024-04-25T10:11:23.502202+0000","last_active":"2024-04-25T10:11:24.478814+0000","last_peered":"2024-04-25T10:11:24.478814+0000","last_clean":"2024-04-25T10:11:24.478814+0000","last_became_active":"2024-04-25T10:11:23.501391+0000","last_became_peered":"2024-04-25T10:11:23.501391+0000","last_unstale":"2024-04-25T10:11:24.478814+0000","last_undegraded":"2024-04-25T10:11:24.478814+0000","last_fullsized":"2024-04-25T10:11:24.478814+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.937312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478106+0000","last_change":"2024-04-25T10:11:23.501608+0000","last_active":"2024-04-25T10:11:24.478106+0000","last_peered":"2024-04-25T10:11:24.478106+0000","last_clean":"2024-04-25T10:11:24.478106+0000","last_became_active":"2024-04-25T10:11:23.500874+0000","last_became_peered":"2024-04-25T10:11:23.500874+0000","last_unstale":"2024-04-25T10:11:24.478106+0000","last_undegraded":"2024-04-25T10:11:24.478106+0000","last_fullsized":"2024-04-25T10:11:24.478106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:39.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474639+0000","last_change":"2024-04-25T10:11:23.504056+0000","last_active":"2024-04-25T10:11:24.474639+0000","last_peered":"2024-04-25T10:11:24.474639+0000","last_clean":"2024-04-25T10:11:24.474639+0000","last_became_active":"2024-04-25T10:11:23.503801+0000","last_became_peered":"2024-04-25T10:11:23.503801+0000","last_unstale":"2024-04-25T10:11:24.474639+0000","last_undegraded":"2024-04-25T10:11:24.474639+0000","last_fullsized":"2024-04-25T10:11:24.474639+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:09:32.459765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521718+0000","last_change":"2024-04-25T10:11:23.490475+0000","last_active":"2024-04-25T10:11:24.521718+0000","last_peered":"2024-04-25T10:11:24.521718+0000","last_clean":"2024-04-25T10:11:24.521718+0000","last_became_active":"2024-04-25T10:11:23.489746+0000","last_became_peered":"2024-04-25T10:11:23.489746+0000","last_unstale":"2024-04-25T10:11:24.521718+0000","last_undegraded":"2024-04-25T10:11:24.521718+0000","last_fullsized":"2024-04-25T10:11:24.521718+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:09:41.915789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479424+0000","last_change":"2024-04-25T10:11:23.503039+0000","last_active":"2024-04-25T10:11:24.479424+0000","last_peered":"2024-04-25T10:11:24.479424+0000","last_clean":"2024-04-25T10:11:24.479424+0000","last_became_active":"2024-04-25T10:11:23.502130+0000","last_became_peered":"2024-04-25T10:11:23.502130+0000","last_unstale":"2024-04-25T10:11:24.479424+0000","last_undegraded":"2024-04-25T10:11:24.479424+0000","last_fullsized":"2024-04-25T10:11:24.479424+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966420+0000","last_change":"2024-04-25T10:11:23.501635+0000","last_active":"2024-04-25T10:11:25.966420+0000","last_peered":"2024-04-25T10:11:25.966420+0000","last_clean":"2024-04-25T10:11:25.966420+0000","last_became_active":"2024-04-25T10:11:23.501356+0000","last_became_peered":"2024-04-25T10:11:23.501356+0000","last_unstale":"2024-04-25T10:11:25.966420+0000","last_undegraded":"2024-04-25T10:11:25.966420+0000","last_fullsized":"2024-04-25T10:11:25.966420+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:29.491438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968463+0000","last_change":"2024-04-25T10:11:23.499753+0000","last_active":"2024-04-25T10:11:25.968463+0000","last_peered":"2024-04-25T10:11:25.968463+0000","last_clean":"2024-04-25T10:11:25.968463+0000","last_became_active":"2024-04-25T10:11:23.499460+0000","last_became_peered":"2024-04-25T10:11:23.499460+0000","last_unstale":"2024-04-25T10:11:25.968463+0000","last_undegraded":"2024-04-25T10:11:25.968463+0000","last_fullsized":"2024-04-25T10:11:25.968463+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:06:34.591440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950652+0000","last_change":"2024-04-25T10:11:23.500199+0000","last_active":"2024-04-25T10:11:25.950652+0000","last_peered":"2024-04-25T10:11:25.950652+0000","last_clean":"2024-04-25T10:11:25.950652+0000","last_became_active":"2024-04-25T10:11:23.499904+0000","last_became_peered":"2024-04-25T10:11:23.499904+0000","last_unstale":"2024-04-25T10:11:25.950652+0000","last_undegraded":"2024-04-25T10:11:25.950652+0000","last_fullsized":"2024-04-25T10:11:25.950652+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:41.977671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479200+0000","last_change":"2024-04-25T10:11:23.494112+0000","last_active":"2024-04-25T10:11:24.479200+0000","last_peered":"2024-04-25T10:11:24.479200+0000","last_clean":"2024-04-25T10:11:24.479200+0000","last_became_active":"2024-04-25T10:11:23.493781+0000","last_became_peered":"2024-04-25T10:11:23.493781+0000","last_unstale":"2024-04-25T10:11:24.479200+0000","last_undegraded":"2024-04-25T10:11:24.479200+0000","last_fullsized":"2024-04-25T10:11:24.479200+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:37.491564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520698+0000","last_change":"2024-04-25T10:11:23.504171+0000","last_active":"2024-04-25T10:11:24.520698+0000","last_peered":"2024-04-25T10:11:24.520698+0000","last_clean":"2024-04-25T10:11:24.520698+0000","last_became_active":"2024-04-25T10:11:23.501873+0000","last_became_peered":"2024-04-25T10:11:23.501873+0000","last_unstale":"2024-04-25T10:11:24.520698+0000","last_undegraded":"2024-04-25T10:11:24.520698+0000","last_fullsized":"2024-04-25T10:11:24.520698+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:13.126891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864462+0000","last_change":"2024-04-25T10:11:23.496202+0000","last_active":"2024-04-25T10:11:24.864462+0000","last_peered":"2024-04-25T10:11:24.864462+0000","last_clean":"2024-04-25T10:11:24.864462+0000","last_became_active":"2024-04-25T10:11:23.495923+0000","last_became_peered":"2024-04-25T10:11:23.495923+0000","last_unstale":"2024-04-25T10:11:24.864462+0000","last_undegraded":"2024-04-25T10:11:24.864462+0000","last_fullsized":"2024-04-25T10:11:24.864462+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:01:26.229309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864558+0000","last_change":"2024-04-25T10:11:23.500336+0000","last_active":"2024-04-25T10:11:24.864558+0000","last_peered":"2024-04-25T10:11:24.864558+0000","last_clean":"2024-04-25T10:11:24.864558+0000","last_became_active":"2024-04-25T10:11:23.498911+0000","last_became_peered":"2024-04-25T10:11:23.498911+0000","last_unstale":"2024-04-25T10:11:24.864558+0000","last_undegraded":"2024-04-25T10:11:24.864558+0000","last_fullsized":"2024-04-25T10:11:24.864558+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:00.192265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"97'4","reported_seq":18,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480329+0000","last_change":"2024-04-25T10:11:22.465629+0000","last_active":"2024-04-25T10:11:24.480329+0000","last_peered":"2024-04-25T10:11:24.480329+0000","last_clean":"2024-04-25T10:11:24.480329+0000","last_became_active":"2024-04-25T10:11:22.465239+0000","last_became_peered":"2024-04-25T10:11:22.465239+0000","last_unstale":"2024-04-25T10:11:24.480329+0000","last_undegraded":"2024-04-25T10:11:24.480329+0000","last_fullsized":"2024-04-25T10:11:24.480329+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:29.898591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864201+0000","last_change":"2024-04-25T10:11:23.492228+0000","last_active":"2024-04-25T10:11:24.864201+0000","last_peered":"2024-04-25T10:11:24.864201+0000","last_clean":"2024-04-25T10:11:24.864201+0000","last_became_active":"2024-04-25T10:11:23.491931+0000","last_became_peered":"2024-04-25T10:11:23.491931+0000","last_unstale":"2024-04-25T10:11:24.864201+0000","last_undegraded":"2024-04-25T10:11:24.864201+0000","last_fullsized":"2024-04-25T10:11:24.864201+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:07.616518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479937+0000","last_change":"2024-04-25T10:11:22.467541+0000","last_active":"2024-04-25T10:11:24.479937+0000","last_peered":"2024-04-25T10:11:24.479937+0000","last_clean":"2024-04-25T10:11:24.479937+0000","last_became_active":"2024-04-25T10:11:22.467211+0000","last_became_peered":"2024-04-25T10:11:22.467211+0000","last_unstale":"2024-04-25T10:11:24.479937+0000","last_undegraded":"2024-04-25T10:11:24.479937+0000","last_fullsized":"2024-04-25T10:11:24.479937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:57:29.678377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950850+0000","last_change":"2024-04-25T10:11:23.505136+0000","last_active":"2024-04-25T10:11:25.950850+0000","last_peered":"2024-04-25T10:11:25.950850+0000","last_clean":"2024-04-25T10:11:25.950850+0000","last_became_active":"2024-04-25T10:11:23.504848+0000","last_became_peered":"2024-04-25T10:11:23.504848+0000","last_unstale":"2024-04-25T10:11:25.950850+0000","last_undegraded":"2024-04-25T10:11:25.950850+0000","last_fullsized":"2024-04-25T10:11:25.950850+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:34.764490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950348+0000","last_change":"2024-04-25T10:11:22.474715+0000","last_active":"2024-04-25T10:11:25.950348+0000","last_peered":"2024-04-25T10:11:25.950348+0000","last_clean":"2024-04-25T10:11:25.950348+0000","last_became_active":"2024-04-25T10:11:22.472397+0000","last_became_peered":"2024-04-25T10:11:22.472397+0000","last_unstale":"2024-04-25T10:11:25.950348+0000","last_undegraded":"2024-04-25T10:11:25.950348+0000","last_fullsized":"2024-04-25T10:11:25.950348+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:03.107761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863968+0000","last_change":"2024-04-25T10:11:23.486363+0000","last_active":"2024-04-25T10:11:24.863968+0000","last_peered":"2024-04-25T10:11:24.863968+0000","last_clean":"2024-04-25T10:11:24.863968+0000","last_became_active":"2024-04-25T10:11:23.485997+0000","last_became_peered":"2024-04-25T10:11:23.485997+0000","last_unstale":"2024-04-25T10:11:24.863968+0000","last_undegraded":"2024-04-25T10:11:24.863968+0000","last_fullsized":"2024-04-25T10:11:24.863968+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:39.127870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480923+0000","last_change":"2024-04-25T10:11:22.470977+0000","last_active":"2024-04-25T10:11:24.480923+0000","last_peered":"2024-04-25T10:11:24.480923+0000","last_clean":"2024-04-25T10:11:24.480923+0000","last_became_active":"2024-04-25T10:11:22.469808+0000","last_became_peered":"2024-04-25T10:11:22.469808+0000","last_unstale":"2024-04-25T10:11:24.480923+0000","last_undegraded":"2024-04-25T10:11:24.480923+0000","last_fullsized":"2024-04-25T10:11:24.480923+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:33.243743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968174+0000","last_change":"2024-04-25T10:11:23.514752+0000","last_active":"2024-04-25T10:11:25.968174+0000","last_peered":"2024-04-25T10:11:25.968174+0000","last_clean":"2024-04-25T10:11:25.968174+0000","last_became_active":"2024-04-25T10:11:23.514468+0000","last_became_peered":"2024-04-25T10:11:23.514468+0000","last_unstale":"2024-04-25T10:11:25.968174+0000","last_undegraded":"2024-04-25T10:11:25.968174+0000","last_fullsized":"2024-04-25T10:11:25.968174+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:38.677097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478648+0000","last_change":"2024-04-25T10:11:22.462923+0000","last_active":"2024-04-25T10:11:24.478648+0000","last_peered":"2024-04-25T10:11:24.478648+0000","last_clean":"2024-04-25T10:11:24.478648+0000","last_became_active":"2024-04-25T10:11:22.462615+0000","last_became_peered":"2024-04-25T10:11:22.462615+0000","last_unstale":"2024-04-25T10:11:24.478648+0000","last_undegraded":"2024-04-25T10:11:24.478648+0000","last_fullsized":"2024-04-25T10:11:24.478648+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:13.272502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479515+0000","last_change":"2024-04-25T10:11:23.489900+0000","last_active":"2024-04-25T10:11:24.479515+0000","last_peered":"2024-04-25T10:11:24.479515+0000","last_clean":"2024-04-25T10:11:24.479515+0000","last_became_active":"2024-04-25T10:11:23.489383+0000","last_became_peered":"2024-04-25T10:11:23.489383+0000","last_unstale":"2024-04-25T10:11:24.479515+0000","last_undegraded":"2024-04-25T10:11:24.479515+0000","last_fullsized":"2024-04-25T10:11:24.479515+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:00.937312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967741+0000","last_change":"2024-04-25T10:11:22.466216+0000","last_active":"2024-04-25T10:11:25.967741+0000","last_peered":"2024-04-25T10:11:25.967741+0000","last_clean":"2024-04-25T10:11:25.967741+0000","last_became_active":"2024-04-25T10:11:22.465096+0000","last_became_peered":"2024-04-25T10:11:22.465096+0000","last_unstale":"2024-04-25T10:11:25.967741+0000","last_undegraded":"2024-04-25T10:11:25.967741+0000","last_fullsized":"2024-04-25T10:11:25.967741+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:33.243743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863752+0000","last_change":"2024-04-25T10:11:23.502474+0000","last_active":"2024-04-25T10:11:24.863752+0000","last_peered":"2024-04-25T10:11:24.863752+0000","last_clean":"2024-04-25T10:11:24.863752+0000","last_became_active":"2024-04-25T10:11:23.502027+0000","last_became_peered":"2024-04-25T10:11:23.502027+0000","last_unstale":"2024-04-25T10:11:24.863752+0000","last_undegraded":"2024-04-25T10:11:24.863752+0000","last_fullsized":"2024-04-25T10:11:24.863752+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:11:48.650402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474679+0000","last_change":"2024-04-25T10:11:22.470683+0000","last_active":"2024-04-25T10:11:24.474679+0000","last_peered":"2024-04-25T10:11:24.474679+0000","last_clean":"2024-04-25T10:11:24.474679+0000","last_became_active":"2024-04-25T10:11:22.470361+0000","last_became_peered":"2024-04-25T10:11:22.470361+0000","last_unstale":"2024-04-25T10:11:24.474679+0000","last_undegraded":"2024-04-25T10:11:24.474679+0000","last_fullsized":"2024-04-25T10:11:24.474679+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:17.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967639+0000","last_change":"2024-04-25T10:11:23.508721+0000","last_active":"2024-04-25T10:11:25.967639+0000","last_peered":"2024-04-25T10:11:25.967639+0000","last_clean":"2024-04-25T10:11:25.967639+0000","last_became_active":"2024-04-25T10:11:23.508450+0000","last_became_peered":"2024-04-25T10:11:23.508450+0000","last_unstale":"2024-04-25T10:11:25.967639+0000","last_undegraded":"2024-04-25T10:11:25.967639+0000","last_fullsized":"2024-04-25T10:11:25.967639+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:59.972961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.949937+0000","last_change":"2024-04-25T10:11:22.470874+0000","last_active":"2024-04-25T10:11:25.949937+0000","last_peered":"2024-04-25T10:11:25.949937+0000","last_clean":"2024-04-25T10:11:25.949937+0000","last_became_active":"2024-04-25T10:11:22.470265+0000","last_became_peered":"2024-04-25T10:11:22.470265+0000","last_unstale":"2024-04-25T10:11:25.949937+0000","last_undegraded":"2024-04-25T10:11:25.949937+0000","last_fullsized":"2024-04-25T10:11:25.949937+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:23.973293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479393+0000","last_change":"2024-04-25T10:11:23.504435+0000","last_active":"2024-04-25T10:11:24.479393+0000","last_peered":"2024-04-25T10:11:24.479393+0000","last_clean":"2024-04-25T10:11:24.479393+0000","last_became_active":"2024-04-25T10:11:23.504141+0000","last_became_peered":"2024-04-25T10:11:23.504141+0000","last_unstale":"2024-04-25T10:11:24.479393+0000","last_undegraded":"2024-04-25T10:11:24.479393+0000","last_fullsized":"2024-04-25T10:11:24.479393+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:00.443531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521846+0000","last_change":"2024-04-25T10:11:22.473533+0000","last_active":"2024-04-25T10:11:24.521846+0000","last_peered":"2024-04-25T10:11:24.521846+0000","last_clean":"2024-04-25T10:11:24.521846+0000","last_became_active":"2024-04-25T10:11:22.473208+0000","last_became_peered":"2024-04-25T10:11:22.473208+0000","last_unstale":"2024-04-25T10:11:24.521846+0000","last_undegraded":"2024-04-25T10:11:24.521846+0000","last_fullsized":"2024-04-25T10:11:24.521846+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:37.661232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950916+0000","last_change":"2024-04-25T10:11:23.500410+0000","last_active":"2024-04-25T10:11:25.950916+0000","last_peered":"2024-04-25T10:11:25.950916+0000","last_clean":"2024-04-25T10:11:25.950916+0000","last_became_active":"2024-04-25T10:11:23.499418+0000","last_became_peered":"2024-04-25T10:11:23.499418+0000","last_unstale":"2024-04-25T10:11:25.950916+0000","last_undegraded":"2024-04-25T10:11:25.950916+0000","last_fullsized":"2024-04-25T10:11:25.950916+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:43:58.222396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475145+0000","last_change":"2024-04-25T10:11:22.465237+0000","last_active":"2024-04-25T10:11:24.475145+0000","last_peered":"2024-04-25T10:11:24.475145+0000","last_clean":"2024-04-25T10:11:24.475145+0000","last_became_active":"2024-04-25T10:11:22.464506+0000","last_became_peered":"2024-04-25T10:11:22.464506+0000","last_unstale":"2024-04-25T10:11:24.475145+0000","last_undegraded":"2024-04-25T10:11:24.475145+0000","last_fullsized":"2024-04-25T10:11:24.475145+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:11.893976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.475166+0000","last_change":"2024-04-25T10:11:23.512957+0000","last_active":"2024-04-25T10:11:24.475166+0000","last_peered":"2024-04-25T10:11:24.475166+0000","last_clean":"2024-04-25T10:11:24.475166+0000","last_became_active":"2024-04-25T10:11:23.512233+0000","last_became_peered":"2024-04-25T10:11:23.512233+0000","last_unstale":"2024-04-25T10:11:24.475166+0000","last_undegraded":"2024-04-25T10:11:24.475166+0000","last_fullsized":"2024-04-25T10:11:24.475166+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:41.977671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"97'3","reported_seq":17,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479786+0000","last_change":"2024-04-25T10:11:22.461965+0000","last_active":"2024-04-25T10:11:24.479786+0000","last_peered":"2024-04-25T10:11:24.479786+0000","last_clean":"2024-04-25T10:11:24.479786+0000","last_became_active":"2024-04-25T10:11:22.461218+0000","last_became_peered":"2024-04-25T10:11:22.461218+0000","last_unstale":"2024-04-25T10:11:24.479786+0000","last_undegraded":"2024-04-25T10:11:24.479786+0000","last_fullsized":"2024-04-25T10:11:24.479786+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:15:33.930030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968031+0000","last_change":"2024-04-25T10:11:23.499871+0000","last_active":"2024-04-25T10:11:25.968031+0000","last_peered":"2024-04-25T10:11:25.968031+0000","last_clean":"2024-04-25T10:11:25.968031+0000","last_became_active":"2024-04-25T10:11:23.499555+0000","last_became_peered":"2024-04-25T10:11:23.499555+0000","last_unstale":"2024-04-25T10:11:25.968031+0000","last_undegraded":"2024-04-25T10:11:25.968031+0000","last_fullsized":"2024-04-25T10:11:25.968031+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:46.821062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"97'6","reported_seq":20,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864178+0000","last_change":"2024-04-25T10:11:22.461868+0000","last_active":"2024-04-25T10:11:24.864178+0000","last_peered":"2024-04-25T10:11:24.864178+0000","last_clean":"2024-04-25T10:11:24.864178+0000","last_became_active":"2024-04-25T10:11:22.461381+0000","last_became_peered":"2024-04-25T10:11:22.461381+0000","last_unstale":"2024-04-25T10:11:24.864178+0000","last_undegraded":"2024-04-25T10:11:24.864178+0000","last_fullsized":"2024-04-25T10:11:24.864178+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:33.748625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967812+0000","last_change":"2024-04-25T10:11:23.496310+0000","last_active":"2024-04-25T10:11:25.967812+0000","last_peered":"2024-04-25T10:11:25.967812+0000","last_clean":"2024-04-25T10:11:25.967812+0000","last_became_active":"2024-04-25T10:11:23.495975+0000","last_became_peered":"2024-04-25T10:11:23.495975+0000","last_unstale":"2024-04-25T10:11:25.967812+0000","last_undegraded":"2024-04-25T10:11:25.967812+0000","last_fullsized":"2024-04-25T10:11:25.967812+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:46.519451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950177+0000","last_change":"2024-04-25T10:11:22.474187+0000","last_active":"2024-04-25T10:11:25.950177+0000","last_peered":"2024-04-25T10:11:25.950177+0000","last_clean":"2024-04-25T10:11:25.950177+0000","last_became_active":"2024-04-25T10:11:22.473866+0000","last_became_peered":"2024-04-25T10:11:22.473866+0000","last_unstale":"2024-04-25T10:11:25.950177+0000","last_undegraded":"2024-04-25T10:11:25.950177+0000","last_fullsized":"2024-04-25T10:11:25.950177+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:36:17.286825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.478658+0000","last_change":"2024-04-25T10:11:23.498466+0000","last_active":"2024-04-25T10:11:24.478658+0000","last_peered":"2024-04-25T10:11:24.478658+0000","last_clean":"2024-04-25T10:11:24.478658+0000","last_became_active":"2024-04-25T10:11:23.497025+0000","last_became_peered":"2024-04-25T10:11:23.497025+0000","last_unstale":"2024-04-25T10:11:24.478658+0000","last_undegraded":"2024-04-25T10:11:24.478658+0000","last_fullsized":"2024-04-25T10:11:24.478658+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:39.029521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863851+0000","last_change":"2024-04-25T10:11:22.469146+0000","last_active":"2024-04-25T10:11:24.863851+0000","last_peered":"2024-04-25T10:11:24.863851+0000","last_clean":"2024-04-25T10:11:24.863851+0000","last_became_active":"2024-04-25T10:11:22.468838+0000","last_became_peered":"2024-04-25T10:11:22.468838+0000","last_unstale":"2024-04-25T10:11:24.863851+0000","last_undegraded":"2024-04-25T10:11:24.863851+0000","last_fullsized":"2024-04-25T10:11:24.863851+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:46.341223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965886+0000","last_change":"2024-04-25T10:11:23.504259+0000","last_active":"2024-04-25T10:11:25.965886+0000","last_peered":"2024-04-25T10:11:25.965886+0000","last_clean":"2024-04-25T10:11:25.965886+0000","last_became_active":"2024-04-25T10:11:23.503980+0000","last_became_peered":"2024-04-25T10:11:23.503980+0000","last_unstale":"2024-04-25T10:11:25.965886+0000","last_undegraded":"2024-04-25T10:11:25.965886+0000","last_fullsized":"2024-04-25T10:11:25.965886+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:18:27.440246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967382+0000","last_change":"2024-04-25T10:11:22.470404+0000","last_active":"2024-04-25T10:11:25.967382+0000","last_peered":"2024-04-25T10:11:25.967382+0000","last_clean":"2024-04-25T10:11:25.967382+0000","last_became_active":"2024-04-25T10:11:22.470111+0000","last_became_peered":"2024-04-25T10:11:22.470111+0000","last_unstale":"2024-04-25T10:11:25.967382+0000","last_undegraded":"2024-04-25T10:11:25.967382+0000","last_fullsized":"2024-04-25T10:11:25.967382+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:32.968772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966320+0000","last_change":"2024-04-25T10:11:23.500138+0000","last_active":"2024-04-25T10:11:25.966320+0000","last_peered":"2024-04-25T10:11:25.966320+0000","last_clean":"2024-04-25T10:11:25.966320+0000","last_became_active":"2024-04-25T10:11:23.499378+0000","last_became_peered":"2024-04-25T10:11:23.499378+0000","last_unstale":"2024-04-25T10:11:25.966320+0000","last_undegraded":"2024-04-25T10:11:25.966320+0000","last_fullsized":"2024-04-25T10:11:25.966320+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.096148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"97'8","reported_seq":22,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.967909+0000","last_change":"2024-04-25T10:11:22.465509+0000","last_active":"2024-04-25T10:11:25.967909+0000","last_peered":"2024-04-25T10:11:25.967909+0000","last_clean":"2024-04-25T10:11:25.967909+0000","last_became_active":"2024-04-25T10:11:22.465243+0000","last_became_peered":"2024-04-25T10:11:22.465243+0000","last_unstale":"2024-04-25T10:11:25.967909+0000","last_undegraded":"2024-04-25T10:11:25.967909+0000","last_fullsized":"2024-04-25T10:11:25.967909+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:29.898591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864035+0000","last_change":"2024-04-25T10:11:23.500428+0000","last_active":"2024-04-25T10:11:24.864035+0000","last_peered":"2024-04-25T10:11:24.864035+0000","last_clean":"2024-04-25T10:11:24.864035+0000","last_became_active":"2024-04-25T10:11:23.500144+0000","last_became_peered":"2024-04-25T10:11:23.500144+0000","last_unstale":"2024-04-25T10:11:24.864035+0000","last_undegraded":"2024-04-25T10:11:24.864035+0000","last_fullsized":"2024-04-25T10:11:24.864035+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:25:30.247684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480875+0000","last_change":"2024-04-25T10:11:22.471844+0000","last_active":"2024-04-25T10:11:24.480875+0000","last_peered":"2024-04-25T10:11:24.480875+0000","last_clean":"2024-04-25T10:11:24.480875+0000","last_became_active":"2024-04-25T10:11:22.471304+0000","last_became_peered":"2024-04-25T10:11:22.471304+0000","last_unstale":"2024-04-25T10:11:24.480875+0000","last_undegraded":"2024-04-25T10:11:24.480875+0000","last_fullsized":"2024-04-25T10:11:24.480875+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:29.865607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480738+0000","last_change":"2024-04-25T10:11:23.487951+0000","last_active":"2024-04-25T10:11:24.480738+0000","last_peered":"2024-04-25T10:11:24.480738+0000","last_clean":"2024-04-25T10:11:24.480738+0000","last_became_active":"2024-04-25T10:11:23.487535+0000","last_became_peered":"2024-04-25T10:11:23.487535+0000","last_unstale":"2024-04-25T10:11:24.480738+0000","last_undegraded":"2024-04-25T10:11:24.480738+0000","last_fullsized":"2024-04-25T10:11:24.480738+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:40:58.084466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950349+0000","last_change":"2024-04-25T10:11:22.476577+0000","last_active":"2024-04-25T10:11:25.950349+0000","last_peered":"2024-04-25T10:11:25.950349+0000","last_clean":"2024-04-25T10:11:25.950349+0000","last_became_active":"2024-04-25T10:11:22.476222+0000","last_became_peered":"2024-04-25T10:11:22.476222+0000","last_unstale":"2024-04-25T10:11:25.950349+0000","last_undegraded":"2024-04-25T10:11:25.950349+0000","last_fullsized":"2024-04-25T10:11:25.950349+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:47:55.296806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479787+0000","last_change":"2024-04-25T10:11:23.495714+0000","last_active":"2024-04-25T10:11:24.479787+0000","last_peered":"2024-04-25T10:11:24.479787+0000","last_clean":"2024-04-25T10:11:24.479787+0000","last_became_active":"2024-04-25T10:11:23.495066+0000","last_became_peered":"2024-04-25T10:11:23.495066+0000","last_unstale":"2024-04-25T10:11:24.479787+0000","last_undegraded":"2024-04-25T10:11:24.479787+0000","last_fullsized":"2024-04-25T10:11:24.479787+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:04:25.091532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"97'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479884+0000","last_change":"2024-04-25T10:11:22.470350+0000","last_active":"2024-04-25T10:11:24.479884+0000","last_peered":"2024-04-25T10:11:24.479884+0000","last_clean":"2024-04-25T10:11:24.479884+0000","last_became_active":"2024-04-25T10:11:22.470032+0000","last_became_peered":"2024-04-25T10:11:22.470032+0000","last_unstale":"2024-04-25T10:11:24.479884+0000","last_undegraded":"2024-04-25T10:11:24.479884+0000","last_fullsized":"2024-04-25T10:11:24.479884+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:03.503976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864599+0000","last_change":"2024-04-25T10:11:23.501944+0000","last_active":"2024-04-25T10:11:24.864599+0000","last_peered":"2024-04-25T10:11:24.864599+0000","last_clean":"2024-04-25T10:11:24.864599+0000","last_became_active":"2024-04-25T10:11:23.501477+0000","last_became_peered":"2024-04-25T10:11:23.501477+0000","last_unstale":"2024-04-25T10:11:24.864599+0000","last_undegraded":"2024-04-25T10:11:24.864599+0000","last_fullsized":"2024-04-25T10:11:24.864599+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:14:52.961651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520557+0000","last_change":"2024-04-25T10:11:22.473753+0000","last_active":"2024-04-25T10:11:24.520557+0000","last_peered":"2024-04-25T10:11:24.520557+0000","last_clean":"2024-04-25T10:11:24.520557+0000","last_became_active":"2024-04-25T10:11:22.473434+0000","last_became_peered":"2024-04-25T10:11:22.473434+0000","last_unstale":"2024-04-25T10:11:24.520557+0000","last_undegraded":"2024-04-25T10:11:24.520557+0000","last_fullsized":"2024-04-25T10:11:24.520557+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:29:53.765476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520767+0000","last_change":"2024-04-25T10:11:23.496038+0000","last_active":"2024-04-25T10:11:24.520767+0000","last_peered":"2024-04-25T10:11:24.520767+0000","last_clean":"2024-04-25T10:11:24.520767+0000","last_became_active":"2024-04-25T10:11:23.495699+0000","last_became_peered":"2024-04-25T10:11:23.495699+0000","last_unstale":"2024-04-25T10:11:24.520767+0000","last_undegraded":"2024-04-25T10:11:24.520767+0000","last_fullsized":"2024-04-25T10:11:24.520767+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:39:33.466669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966036+0000","last_change":"2024-04-25T10:11:22.462653+0000","last_active":"2024-04-25T10:11:25.966036+0000","last_peered":"2024-04-25T10:11:25.966036+0000","last_clean":"2024-04-25T10:11:25.966036+0000","last_became_active":"2024-04-25T10:11:22.462355+0000","last_became_peered":"2024-04-25T10:11:22.462355+0000","last_unstale":"2024-04-25T10:11:25.966036+0000","last_undegraded":"2024-04-25T10:11:25.966036+0000","last_fullsized":"2024-04-25T10:11:25.966036+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:49.454297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521025+0000","last_change":"2024-04-25T10:11:23.504455+0000","last_active":"2024-04-25T10:11:24.521025+0000","last_peered":"2024-04-25T10:11:24.521025+0000","last_clean":"2024-04-25T10:11:24.521025+0000","last_became_active":"2024-04-25T10:11:23.504135+0000","last_became_peered":"2024-04-25T10:11:23.504135+0000","last_unstale":"2024-04-25T10:11:24.521025+0000","last_undegraded":"2024-04-25T10:11:24.521025+0000","last_fullsized":"2024-04-25T10:11:24.521025+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:00.347401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.520952+0000","last_change":"2024-04-25T10:11:22.476111+0000","last_active":"2024-04-25T10:11:24.520952+0000","last_peered":"2024-04-25T10:11:24.520952+0000","last_clean":"2024-04-25T10:11:24.520952+0000","last_became_active":"2024-04-25T10:11:22.474623+0000","last_became_peered":"2024-04-25T10:11:22.474623+0000","last_unstale":"2024-04-25T10:11:24.520952+0000","last_undegraded":"2024-04-25T10:11:24.520952+0000","last_fullsized":"2024-04-25T10:11:24.520952+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:21.062051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863947+0000","last_change":"2024-04-25T10:11:23.490610+0000","last_active":"2024-04-25T10:11:24.863947+0000","last_peered":"2024-04-25T10:11:24.863947+0000","last_clean":"2024-04-25T10:11:24.863947+0000","last_became_active":"2024-04-25T10:11:23.489790+0000","last_became_peered":"2024-04-25T10:11:23.489790+0000","last_unstale":"2024-04-25T10:11:24.863947+0000","last_undegraded":"2024-04-25T10:11:24.863947+0000","last_fullsized":"2024-04-25T10:11:24.863947+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:13.456681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.965445+0000","last_change":"2024-04-25T10:11:22.471275+0000","last_active":"2024-04-25T10:11:25.965445+0000","last_peered":"2024-04-25T10:11:25.965445+0000","last_clean":"2024-04-25T10:11:25.965445+0000","last_became_active":"2024-04-25T10:11:22.471004+0000","last_became_peered":"2024-04-25T10:11:22.471004+0000","last_unstale":"2024-04-25T10:11:25.965445+0000","last_undegraded":"2024-04-25T10:11:25.965445+0000","last_fullsized":"2024-04-25T10:11:25.965445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:20:53.869983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480482+0000","last_change":"2024-04-25T10:11:23.498595+0000","last_active":"2024-04-25T10:11:24.480482+0000","last_peered":"2024-04-25T10:11:24.480482+0000","last_clean":"2024-04-25T10:11:24.480482+0000","last_became_active":"2024-04-25T10:11:23.497684+0000","last_became_peered":"2024-04-25T10:11:23.497684+0000","last_unstale":"2024-04-25T10:11:24.480482+0000","last_undegraded":"2024-04-25T10:11:24.480482+0000","last_fullsized":"2024-04-25T10:11:24.480482+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:00.443531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"97'2","reported_seq":16,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950639+0000","last_change":"2024-04-25T10:11:22.477558+0000","last_active":"2024-04-25T10:11:25.950639+0000","last_peered":"2024-04-25T10:11:25.950639+0000","last_clean":"2024-04-25T10:11:25.950639+0000","last_became_active":"2024-04-25T10:11:22.476031+0000","last_became_peered":"2024-04-25T10:11:22.476031+0000","last_unstale":"2024-04-25T10:11:25.950639+0000","last_undegraded":"2024-04-25T10:11:25.950639+0000","last_fullsized":"2024-04-25T10:11:25.950639+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:07.891229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864672+0000","last_change":"2024-04-25T10:11:23.498208+0000","last_active":"2024-04-25T10:11:24.864672+0000","last_peered":"2024-04-25T10:11:24.864672+0000","last_clean":"2024-04-25T10:11:24.864672+0000","last_became_active":"2024-04-25T10:11:23.497917+0000","last_became_peered":"2024-04-25T10:11:23.497917+0000","last_unstale":"2024-04-25T10:11:24.864672+0000","last_undegraded":"2024-04-25T10:11:24.864672+0000","last_fullsized":"2024-04-25T10:11:24.864672+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:36:45.774608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"98'1","reported_seq":15,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480212+0000","last_change":"2024-04-25T10:11:22.476964+0000","last_active":"2024-04-25T10:11:24.480212+0000","last_peered":"2024-04-25T10:11:24.480212+0000","last_clean":"2024-04-25T10:11:24.480212+0000","last_became_active":"2024-04-25T10:11:22.476572+0000","last_became_peered":"2024-04-25T10:11:22.476572+0000","last_unstale":"2024-04-25T10:11:24.480212+0000","last_undegraded":"2024-04-25T10:11:24.480212+0000","last_fullsized":"2024-04-25T10:11:24.480212+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:09:18.402891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.863704+0000","last_change":"2024-04-25T10:11:23.494385+0000","last_active":"2024-04-25T10:11:24.863704+0000","last_peered":"2024-04-25T10:11:24.863704+0000","last_clean":"2024-04-25T10:11:24.863704+0000","last_became_active":"2024-04-25T10:11:23.494070+0000","last_became_peered":"2024-04-25T10:11:23.494070+0000","last_unstale":"2024-04-25T10:11:24.863704+0000","last_undegraded":"2024-04-25T10:11:24.863704+0000","last_fullsized":"2024-04-25T10:11:24.863704+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:14:10.899875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479943+0000","last_change":"2024-04-25T10:11:22.463847+0000","last_active":"2024-04-25T10:11:24.479943+0000","last_peered":"2024-04-25T10:11:24.479943+0000","last_clean":"2024-04-25T10:11:24.479943+0000","last_became_active":"2024-04-25T10:11:22.463559+0000","last_became_peered":"2024-04-25T10:11:22.463559+0000","last_unstale":"2024-04-25T10:11:24.479943+0000","last_undegraded":"2024-04-25T10:11:24.479943+0000","last_fullsized":"2024-04-25T10:11:24.479943+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:16:29.936278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521840+0000","last_change":"2024-04-25T10:11:23.499509+0000","last_active":"2024-04-25T10:11:24.521840+0000","last_peered":"2024-04-25T10:11:24.521840+0000","last_clean":"2024-04-25T10:11:24.521840+0000","last_became_active":"2024-04-25T10:11:23.498796+0000","last_became_peered":"2024-04-25T10:11:23.498796+0000","last_unstale":"2024-04-25T10:11:24.521840+0000","last_undegraded":"2024-04-25T10:11:24.521840+0000","last_fullsized":"2024-04-25T10:11:24.521840+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:12.174658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.480989+0000","last_change":"2024-04-25T10:11:22.458990+0000","last_active":"2024-04-25T10:11:24.480989+0000","last_peered":"2024-04-25T10:11:24.480989+0000","last_clean":"2024-04-25T10:11:24.480989+0000","last_became_active":"2024-04-25T10:11:22.458519+0000","last_became_peered":"2024-04-25T10:11:22.458519+0000","last_unstale":"2024-04-25T10:11:24.480989+0000","last_undegraded":"2024-04-25T10:11:24.480989+0000","last_fullsized":"2024-04-25T10:11:24.480989+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:28.483485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.966106+0000","last_change":"2024-04-25T10:11:23.495054+0000","last_active":"2024-04-25T10:11:25.966106+0000","last_peered":"2024-04-25T10:11:25.966106+0000","last_clean":"2024-04-25T10:11:25.966106+0000","last_became_active":"2024-04-25T10:11:23.494672+0000","last_became_peered":"2024-04-25T10:11:23.494672+0000","last_unstale":"2024-04-25T10:11:25.966106+0000","last_undegraded":"2024-04-25T10:11:25.966106+0000","last_fullsized":"2024-04-25T10:11:25.966106+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:37.959511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864095+0000","last_change":"2024-04-25T10:11:22.469251+0000","last_active":"2024-04-25T10:11:24.864095+0000","last_peered":"2024-04-25T10:11:24.864095+0000","last_clean":"2024-04-25T10:11:24.864095+0000","last_became_active":"2024-04-25T10:11:22.468896+0000","last_became_peered":"2024-04-25T10:11:22.468896+0000","last_unstale":"2024-04-25T10:11:24.864095+0000","last_undegraded":"2024-04-25T10:11:24.864095+0000","last_fullsized":"2024-04-25T10:11:24.864095+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:40.961806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.474640+0000","last_change":"2024-04-25T10:11:23.511617+0000","last_active":"2024-04-25T10:11:24.474640+0000","last_peered":"2024-04-25T10:11:24.474640+0000","last_clean":"2024-04-25T10:11:24.474640+0000","last_became_active":"2024-04-25T10:11:23.511352+0000","last_became_peered":"2024-04-25T10:11:23.511352+0000","last_unstale":"2024-04-25T10:11:24.474640+0000","last_undegraded":"2024-04-25T10:11:24.474640+0000","last_fullsized":"2024-04-25T10:11:24.474640+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:17.588960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.521129+0000","last_change":"2024-04-25T10:11:22.472035+0000","last_active":"2024-04-25T10:11:24.521129+0000","last_peered":"2024-04-25T10:11:24.521129+0000","last_clean":"2024-04-25T10:11:24.521129+0000","last_became_active":"2024-04-25T10:11:22.471686+0000","last_became_peered":"2024-04-25T10:11:22.471686+0000","last_unstale":"2024-04-25T10:11:24.521129+0000","last_undegraded":"2024-04-25T10:11:24.521129+0000","last_fullsized":"2024-04-25T10:11:24.521129+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:45.805197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479618+0000","last_change":"2024-04-25T10:11:23.503580+0000","last_active":"2024-04-25T10:11:24.479618+0000","last_peered":"2024-04-25T10:11:24.479618+0000","last_clean":"2024-04-25T10:11:24.479618+0000","last_became_active":"2024-04-25T10:11:23.503299+0000","last_became_peered":"2024-04-25T10:11:23.503299+0000","last_unstale":"2024-04-25T10:11:24.479618+0000","last_undegraded":"2024-04-25T10:11:24.479618+0000","last_fullsized":"2024-04-25T10:11:24.479618+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:37.491564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.479662+0000","last_change":"2024-04-25T10:11:22.458531+0000","last_active":"2024-04-25T10:11:24.479662+0000","last_peered":"2024-04-25T10:11:24.479662+0000","last_clean":"2024-04-25T10:11:24.479662+0000","last_became_active":"2024-04-25T10:11:22.457900+0000","last_became_peered":"2024-04-25T10:11:22.457900+0000","last_unstale":"2024-04-25T10:11:24.479662+0000","last_undegraded":"2024-04-25T10:11:24.479662+0000","last_fullsized":"2024-04-25T10:11:24.479662+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:32.968772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:24.864134+0000","last_change":"2024-04-25T10:11:23.498761+0000","last_active":"2024-04-25T10:11:24.864134+0000","last_peered":"2024-04-25T10:11:24.864134+0000","last_clean":"2024-04-25T10:11:24.864134+0000","last_became_active":"2024-04-25T10:11:23.498483+0000","last_became_peered":"2024-04-25T10:11:23.498483+0000","last_unstale":"2024-04-25T10:11:24.864134+0000","last_undegraded":"2024-04-25T10:11:24.864134+0000","last_fullsized":"2024-04-25T10:11:24.864134+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:56:31.072700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.949962+0000","last_change":"2024-04-25T10:11:22.470587+0000","last_active":"2024-04-25T10:11:25.949962+0000","last_peered":"2024-04-25T10:11:25.949962+0000","last_clean":"2024-04-25T10:11:25.949962+0000","last_became_active":"2024-04-25T10:11:22.469775+0000","last_became_peered":"2024-04-25T10:11:22.469775+0000","last_unstale":"2024-04-25T10:11:25.949962+0000","last_undegraded":"2024-04-25T10:11:25.949962+0000","last_fullsized":"2024-04-25T10:11:25.949962+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:55:11.893976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":12,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950687+0000","last_change":"2024-04-25T10:11:23.517824+0000","last_active":"2024-04-25T10:11:25.950687+0000","last_peered":"2024-04-25T10:11:25.950687+0000","last_clean":"2024-04-25T10:11:25.950687+0000","last_became_active":"2024-04-25T10:11:23.517479+0000","last_became_peered":"2024-04-25T10:11:23.517479+0000","last_unstale":"2024-04-25T10:11:25.950687+0000","last_undegraded":"2024-04-25T10:11:25.950687+0000","last_fullsized":"2024-04-25T10:11:25.950687+0000","mapping_epoch":96,"log_start":"0'0","ondisk_log_start":"0'0","created":96,"last_epoch_clean":97,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:22.432352+0000","last_clean_scrub_stamp":"2024-04-25T10:11:22.432352+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:16:47.357088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":14,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.968304+0000","last_change":"2024-04-25T10:11:22.474875+0000","last_active":"2024-04-25T10:11:25.968304+0000","last_peered":"2024-04-25T10:11:25.968304+0000","last_clean":"2024-04-25T10:11:25.968304+0000","last_became_active":"2024-04-25T10:11:22.474581+0000","last_became_peered":"2024-04-25T10:11:22.474581+0000","last_unstale":"2024-04-25T10:11:25.968304+0000","last_undegraded":"2024-04-25T10:11:25.968304+0000","last_fullsized":"2024-04-25T10:11:25.968304+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":95,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:11:21.416487+0000","last_clean_scrub_stamp":"2024-04-25T10:11:21.416487+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:29.865607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":318,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-25T10:11:25.950229+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:11:25.950229+0000","last_peered":"2024-04-25T10:11:25.950229+0000","last_clean":"2024-04-25T10:11:25.950229+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:11:25.950229+0000","last_undegraded":"2024-04-25T10:11:25.950229+0000","last_fullsized":"2024-04-25T10:11:25.950229+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574968,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50996,"kb_used_data":1768,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320844,"statfs":{"total":96636764160,"available":96584544256,"internally_reserved":0,"allocated":1810432,"data_stored":1153237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574968,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50428,"kb_used_data":1200,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321412,"statfs":{"total":96636764160,"available":96585125888,"internally_reserved":0,"allocated":1228800,"data_stored":565029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574968,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46248,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325592,"statfs":{"total":96636764160,"available":96589406208,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58736,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94313104,"statfs":{"total":96636764160,"available":96576618496,"internally_reserved":0,"allocated":1216512,"data_stored":562973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574968,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54576,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317264,"statfs":{"total":96636764160,"available":96580878336,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":54655431},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574967,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54572,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317268,"statfs":{"total":96636764160,"available":96580882432,"internally_reserved":0,"allocated":1208320,"data_stored":562813,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50412,"kb_used_data":1184,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321428,"statfs":{"total":96636764160,"available":96585142272,"internally_reserved":0,"allocated":1212416,"data_stored":562883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574968,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55172,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316668,"statfs":{"total":96636764160,"available":96580268032,"internally_reserved":0,"allocated":1826816,"data_stored":1155499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:11:33.236 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:11:33.236 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:11:33.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.481+0000 7f27a5800640 1 Processor -- start 2024-04-25T10:11:33.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.481+0000 7f27a5800640 1 -- start start 2024-04-25T10:11:33.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.482+0000 7f27a5800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f27a01446d0 0x7f27a00b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.482+0000 7f27a5800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27a0144d50 0x7f27a00b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.482+0000 7f27a5800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27a00cccc0 con 0x7f27a01452a0 2024-04-25T10:11:33.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.482+0000 7f27a5800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a00b2b20 con 0x7f27a01446d0 2024-04-25T10:11:33.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f27a5800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27a006d400 con 0x7f27a0144d50 2024-04-25T10:11:33.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f27a01446d0 0x7f27a00b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f27a01446d0 0x7f27a00b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35848/0 (socket says 172.21.15.165:35848) 2024-04-25T10:11:33.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 -- 172.21.15.165:0/1298070407 learned_addr learned my addr 172.21.15.165:0/1298070407 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:33.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f2797e00640 1 --2- 172.21.15.165:0/1298070407 >> v2:172.21.15.172:3300/0 conn(0x7f27a0144d50 0x7f27a00b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 -- 172.21.15.165:0/1298070407 >> v2:172.21.15.172:3300/0 conn(0x7f27a0144d50 msgr2=0x7f27a00b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 --2- 172.21.15.165:0/1298070407 >> v2:172.21.15.172:3300/0 conn(0x7f27a0144d50 0x7f27a00b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 -- 172.21.15.165:0/1298070407 >> v1:172.21.15.172:6789/0 conn(0x7f27a01452a0 legacy=0x7f27a00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.483+0000 7f279ea00640 1 -- 172.21.15.165:0/1298070407 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a00cc140 con 0x7f27a01446d0 2024-04-25T10:11:33.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.484+0000 7f279ea00640 1 --2- 172.21.15.165:0/1298070407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00b3f20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f27a0071c70 tx=0x7f279002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab7637520e7db0c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f279e000640 1 -- 172.21.15.165:0/1298070407 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790037070 con 0x7f27a01446d0 2024-04-25T10:11:33.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f279e000640 1 -- 172.21.15.165:0/1298070407 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790032440 con 0x7f27a01446d0 2024-04-25T10:11:33.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f279e000640 1 -- 172.21.15.165:0/1298070407 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790032810 con 0x7f27a01446d0 2024-04-25T10:11:33.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 -- 172.21.15.165:0/1298070407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 msgr2=0x7f27a00b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 --2- 172.21.15.165:0/1298070407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00b3f20 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f27a0071c70 tx=0x7f279002a230 comp rx=0 tx=0).stop 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 -- 172.21.15.165:0/1298070407 shutdown_connections 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 --2- 172.21.15.165:0/1298070407 >> v2:172.21.15.172:3300/0 conn(0x7f27a0144d50 0x7f27a00b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 --2- 172.21.15.165:0/1298070407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00b3f20 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 -- 172.21.15.165:0/1298070407 >> 172.21.15.165:0/1298070407 conn(0x7f27a006abe0 msgr2=0x7f27a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.485+0000 7f27a5800640 1 -- 172.21.15.165:0/1298070407 shutdown_connections 2024-04-25T10:11:33.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.486+0000 7f27a5800640 1 -- 172.21.15.165:0/1298070407 wait complete. 2024-04-25T10:11:33.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.487+0000 7f27a5800640 1 Processor -- start 2024-04-25T10:11:33.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.487+0000 7f27a5800640 1 -- start start 2024-04-25T10:11:33.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.488+0000 7f27a5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00c57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.488+0000 7f27a5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a0144d50 0x7f27a00c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.488+0000 7f27a5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27a01452a0 0x7f27a00ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.488+0000 7f27a5800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00cc700 con 0x7f27a01446d0 2024-04-25T10:11:33.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.488+0000 7f27a5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00b49a0 con 0x7f27a0144d50 2024-04-25T10:11:33.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.488+0000 7f27a5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27a00cc9e0 con 0x7f27a01452a0 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00c57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00c57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35850/0 (socket says 172.21.15.165:35850) 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 -- 172.21.15.165:0/526431572 learned_addr learned my addr 172.21.15.165:0/526431572 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 -- 172.21.15.165:0/526431572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27a01452a0 msgr2=0x7f27a00ce9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27a01452a0 0x7f27a00ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 -- 172.21.15.165:0/526431572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a0144d50 msgr2=0x7f27a00c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a0144d50 0x7f27a00c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.489+0000 7f279ea00640 1 -- 172.21.15.165:0/526431572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27a00c4770 con 0x7f27a01446d0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.490+0000 7f279ea00640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00c57d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2790000c00 tx=0x7f27900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.490+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790041020 con 0x7f27a01446d0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.490+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2790030990 con 0x7f27a01446d0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.490+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27a00c5d10 con 0x7f27a01446d0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.490+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2790037040 con 0x7f27a01446d0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.490+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27a00c4fd0 con 0x7f27a01446d0 2024-04-25T10:11:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.491+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27a00c4b20 con 0x7f27a01446d0 2024-04-25T10:11:33.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.492+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2790044020 con 0x7f27a01446d0 2024-04-25T10:11:33.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.493+0000 7f2796a00640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f277c06b4d0 0x7f277c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.493+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27900aaa70 con 0x7f27a01446d0 2024-04-25T10:11:33.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.493+0000 7f2797e00640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f277c06b4d0 0x7f277c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.494+0000 7f2797e00640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f277c06b4d0 0x7f277c06d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f27a00aed60 tx=0x7f2788008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.498+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f279004b070 con 0x7f27a01446d0 2024-04-25T10:11:33.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.760+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f27a01431e0 con 0x7f27a01446d0 2024-04-25T10:11:33.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.764+0000 7f2796a00640 1 -- 172.21.15.165:0/526431572 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+147 (secure 0 0 0) 0x7f2790079750 con 0x7f27a01446d0 2024-04-25T10:11:33.769 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:11:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:11:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:11:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:11:33.770 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:11:33.770 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f277c06b4d0 msgr2=0x7f277c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f277c06b4d0 0x7f277c06d990 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f27a00aed60 tx=0x7f2788008040 comp rx=0 tx=0).stop 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 msgr2=0x7f27a00c57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00c57d0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f2790000c00 tx=0x7f27900046c0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 shutdown_connections 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f277c06b4d0 0x7f277c06d990 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27a01452a0 0x7f27a00ce9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27a0144d50 0x7f27a00c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 --2- 172.21.15.165:0/526431572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27a01446d0 0x7f27a00c57d0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 >> 172.21.15.165:0/526431572 conn(0x7f27a006abe0 msgr2=0x7f27a0143550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 shutdown_connections 2024-04-25T10:11:33.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.768+0000 7f27a5800640 1 -- 172.21.15.165:0/526431572 wait complete. 2024-04-25T10:11:33.789 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:11:33.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.978+0000 7ffb61a00640 1 Processor -- start 2024-04-25T10:11:33.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.978+0000 7ffb61a00640 1 -- start start 2024-04-25T10:11:33.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.980+0000 7ffb61a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c0ce920 0x7ffb5c0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.980+0000 7ffb61a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c144b30 con 0x7ffb5c144750 2024-04-25T10:11:33.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.980+0000 7ffb61a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb5c0712d0 con 0x7ffb5c06d290 2024-04-25T10:11:33.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.980+0000 7ffb61a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0715b0 con 0x7ffb5c0ce920 2024-04-25T10:11:33.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.980+0000 7ffb5aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ffb5c144750 0x7ffb5c06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50128/0 (socket says 172.21.15.165:50128) 2024-04-25T10:11:33.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.980+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2915558598 learned_addr learned my addr 172.21.15.165:0/2915558598 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c0ce920 0x7ffb5c0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764831282 0 0) 0x7ffb5c144b30 con 0x7ffb5c144750 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb40003520 con 0x7ffb5c144750 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 298677219 0 0) 0x7ffb40003520 con 0x7ffb5c144750 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c0ce920 msgr2=0x7ffb5c0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c0ce920 0x7ffb5c0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 >> v1:172.21.15.172:6789/0 conn(0x7ffb5c06d290 legacy=0x7ffb5c06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c071890 con 0x7ffb5c144750 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ffb4c001670 con 0x7ffb5c144750 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 >> v1:172.21.15.165:6789/0 conn(0x7ffb5c144750 legacy=0x7ffb5c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb40003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb40004800 0x7ffb40004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb400051c0 0x7ffb4000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0715b0 con 0x7ffb400039e0 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4000c060 con 0x7ffb40004800 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4000c260 con 0x7ffb400051c0 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.981+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c0ce920 0x7ffb5c0ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb40003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb40004800 0x7ffb40004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb400051c0 msgr2=0x7ffb4000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5b400640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb400051c0 0x7ffb4000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb400051c0 0x7ffb4000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb40004800 msgr2=0x7ffb40004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:33.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb40004800 0x7ffb40004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2915558598 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb40003520 con 0x7ffb400039e0 2024-04-25T10:11:33.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.982+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb40004800 0x7ffb40004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.983+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb40003e00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c240 tx=0x7ffb40005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4da6bcfcc4e92cbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.983+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007d10 con 0x7ffb400039e0 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.983+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c005ad0 con 0x7ffb400039e0 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.983+0000 7ffb5a000640 1 -- 172.21.15.165:0/2915558598 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007070 con 0x7ffb400039e0 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb61a00640 1 -- 172.21.15.165:0/2915558598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 msgr2=0x7ffb40003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2915558598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb40003e00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00c240 tx=0x7ffb40005830 comp rx=0 tx=0).stop 2024-04-25T10:11:33.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb5b400640 1 -- 172.21.15.165:0/2915558598 reap_dead start 2024-04-25T10:11:33.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb61a00640 1 -- 172.21.15.165:0/2915558598 shutdown_connections 2024-04-25T10:11:33.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb61a00640 1 -- 172.21.15.165:0/2915558598 >> 172.21.15.165:0/2915558598 conn(0x7ffb5c06abe0 msgr2=0x7ffb5c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:33.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb61a00640 1 -- 172.21.15.165:0/2915558598 shutdown_connections 2024-04-25T10:11:33.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.984+0000 7ffb61a00640 1 -- 172.21.15.165:0/2915558598 wait complete. 2024-04-25T10:11:33.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.986+0000 7ffb61a00640 1 Processor -- start 2024-04-25T10:11:33.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.986+0000 7ffb61a00640 1 -- start start 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb61a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb5c15e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb61a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c0cc850 0x7ffb5c1e75b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb61a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c1e7af0 0x7ffb5c0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb61a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c145450 con 0x7ffb400039e0 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb61a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c06f020 con 0x7ffb5c0cc850 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb61a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c1455d0 con 0x7ffb5c1e7af0 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb5aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb5c15e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb5aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb5c15e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35868/0 (socket says 172.21.15.165:35868) 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.987+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2666777081 learned_addr learned my addr 172.21.15.165:0/2666777081 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5b400640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c1e7af0 0x7ffb5c0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c0cc850 0x7ffb5c1e75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c1e7af0 msgr2=0x7ffb5c0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c1e7af0 0x7ffb5c0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c0cc850 msgr2=0x7ffb5c1e75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c0cc850 0x7ffb5c1e75b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5aa00640 1 -- 172.21.15.165:0/2666777081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb5c0b00b0 con 0x7ffb400039e0 2024-04-25T10:11:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb5aa00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb5c15e6e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c034040 tx=0x7ffb5c071b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c004bf0 con 0x7ffb400039e0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb4c004d90 con 0x7ffb400039e0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c007270 con 0x7ffb400039e0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.988+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c0b1a90 con 0x7ffb400039e0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.989+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c0b0390 con 0x7ffb400039e0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.990+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb5c0b2150 con 0x7ffb400039e0 2024-04-25T10:11:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.991+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb4c007410 con 0x7ffb400039e0 2024-04-25T10:11:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.992+0000 7ffb52a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3806b500 0x7ffb3806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.992+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb4c0aab00 con 0x7ffb400039e0 2024-04-25T10:11:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.992+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3806b500 0x7ffb3806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.993+0000 7ffb53e00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3806b500 0x7ffb3806d9c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c06d880 tx=0x7ffb44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:33.996+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb5c0b2150 con 0x7ffb400039e0 2024-04-25T10:11:34.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.229+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb5c0d0510 con 0x7ffb400039e0 2024-04-25T10:11:34.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.230+0000 7ffb52a00640 1 -- 172.21.15.165:0/2666777081 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7ffb4c0aae20 con 0x7ffb400039e0 2024-04-25T10:11:34.235 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:11:34.235 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:11:34.235 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:11:34.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3806b500 msgr2=0x7ffb3806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3806b500 0x7ffb3806d9c0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c06d880 tx=0x7ffb44008040 comp rx=0 tx=0).stop 2024-04-25T10:11:34.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 msgr2=0x7ffb5c15e6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb5c15e6e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c034040 tx=0x7ffb5c071b70 comp rx=0 tx=0).stop 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 shutdown_connections 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3806b500 0x7ffb3806d9c0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb5c1e7af0 0x7ffb5c0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb5c0cc850 0x7ffb5c1e75b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 --2- 172.21.15.165:0/2666777081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb400039e0 0x7ffb5c15e6e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 >> 172.21.15.165:0/2666777081 conn(0x7ffb5c06abe0 msgr2=0x7ffb5c0cf580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 shutdown_connections 2024-04-25T10:11:34.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.233+0000 7ffb61a00640 1 -- 172.21.15.165:0/2666777081 wait complete. 2024-04-25T10:11:34.659 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 65, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:11:22.911237+0000', 'modified': '2024-04-25T10:11:25.971934+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7569}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7569': {'gid': 7569, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6835/2500333822', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 2500333822}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 2500333822}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:11:34.660 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:11:34.660 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:11:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.865+0000 7f3302400640 1 Processor -- start 2024-04-25T10:11:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.865+0000 7f3302400640 1 -- start start 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3302400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3302400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f32fc0c5200 0x7f32fc15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3302400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32fc0c5670 con 0x7f32fc0ce950 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3302400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32fc0ced90 con 0x7f32fc0c5200 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3302400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc145520 con 0x7f32fc145140 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3301a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f32fc0c5200 0x7f32fc15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3301a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f32fc0c5200 0x7f32fc15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35874/0 (socket says 172.21.15.165:35874) 2024-04-25T10:11:34.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.867+0000 7f3301a00640 1 -- 172.21.15.165:0/823111150 learned_addr learned my addr 172.21.15.165:0/823111150 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:34.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.868+0000 7f3301a00640 1 -- 172.21.15.165:0/823111150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 msgr2=0x7f32fc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.868+0000 7f3301a00640 1 --2- 172.21.15.165:0/823111150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.868+0000 7f3301a00640 1 -- 172.21.15.165:0/823111150 >> v1:172.21.15.172:6789/0 conn(0x7f32fc0ce950 legacy=0x7f32fc0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.868+0000 7f3301a00640 1 -- 172.21.15.165:0/823111150 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc071270 con 0x7f32fc0c5200 2024-04-25T10:11:34.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.869+0000 7f3301a00640 1 --2- 172.21.15.165:0/823111150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0c5200 0x7f32fc15bfb0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009e50 tx=0x7f32f402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e5468c4560533d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.869+0000 7f32fb400640 1 -- 172.21.15.165:0/823111150 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4036070 con 0x7f32fc0c5200 2024-04-25T10:11:34.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.869+0000 7f32fb400640 1 -- 172.21.15.165:0/823111150 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4031600 con 0x7f32fc0c5200 2024-04-25T10:11:34.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.869+0000 7f32fb400640 1 -- 172.21.15.165:0/823111150 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f40319d0 con 0x7f32fc0c5200 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 -- 172.21.15.165:0/823111150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0c5200 msgr2=0x7f32fc15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 --2- 172.21.15.165:0/823111150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0c5200 0x7f32fc15bfb0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f32f4009e50 tx=0x7f32f402a630 comp rx=0 tx=0).stop 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 -- 172.21.15.165:0/823111150 shutdown_connections 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 --2- 172.21.15.165:0/823111150 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0c5200 0x7f32fc15bfb0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 --2- 172.21.15.165:0/823111150 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 -- 172.21.15.165:0/823111150 >> 172.21.15.165:0/823111150 conn(0x7f32fc061cc0 msgr2=0x7f32fc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 -- 172.21.15.165:0/823111150 shutdown_connections 2024-04-25T10:11:34.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.870+0000 7f3302400640 1 -- 172.21.15.165:0/823111150 wait complete. 2024-04-25T10:11:34.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.872+0000 7f3302400640 1 Processor -- start 2024-04-25T10:11:34.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.872+0000 7f3302400640 1 -- start start 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3302400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0c5200 0x7f32fc1e4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3302400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 0x7f32fc15ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3302400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc1e8980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3302400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc071b10 con 0x7f32fc0ce950 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3302400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc071830 con 0x7f32fc0c5200 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3302400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc15c4f0 con 0x7f32fc145140 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 0x7f32fc15ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 0x7f32fc15ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35880/0 (socket says 172.21.15.165:35880) 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 -- 172.21.15.165:0/829821581 learned_addr learned my addr 172.21.15.165:0/829821581 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3301000640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0c5200 0x7f32fc1e4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f3301a00640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc1e8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 -- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 msgr2=0x7f32fc1e8980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc1e8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 -- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0c5200 msgr2=0x7f32fc1e4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0c5200 0x7f32fc1e4cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.873+0000 7f32fbe00640 1 -- 172.21.15.165:0/829821581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc1e5c40 con 0x7f32fc0ce950 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.874+0000 7f32fbe00640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 0x7f32fc15ac90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c410 tx=0x7f32e800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.874+0000 7f32f8c00640 1 -- 172.21.15.165:0/829821581 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8017070 con 0x7f32fc0ce950 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.874+0000 7f32f8c00640 1 -- 172.21.15.165:0/829821581 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e80035f0 con 0x7f32fc0ce950 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.874+0000 7f32f8c00640 1 -- 172.21.15.165:0/829821581 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8003a20 con 0x7f32fc0ce950 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.874+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32fc15b1d0 con 0x7f32fc0ce950 2024-04-25T10:11:34.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.874+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc1e61c0 con 0x7f32fc0ce950 2024-04-25T10:11:34.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.877+0000 7f32f8c00640 1 -- 172.21.15.165:0/829821581 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f32e8003bc0 con 0x7f32fc0ce950 2024-04-25T10:11:34.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.878+0000 7f32f8c00640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f32dc06b4d0 0x7f32dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.878+0000 7f32f8c00640 1 -- 172.21.15.165:0/829821581 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32e80896e0 con 0x7f32fc0ce950 2024-04-25T10:11:34.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.878+0000 7f3301000640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f32dc06b4d0 0x7f32dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.878+0000 7f330677a740 1 Processor -- start 2024-04-25T10:11:34.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 -- start start 2024-04-25T10:11:34.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f3301000640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f32dc06b4d0 0x7f32dc06d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f32fc06ec20 tx=0x7f32f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 0x55c6e954e970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9526f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e956f7a0 0x55c6e95252f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e948a620 con 0x55c6e9526b30 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e9433ca0 con 0x55c6e954eeb0 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.879+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6e94866c0 con 0x55c6e956f7a0 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 0x55c6e954e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 0x55c6e954e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35892/0 (socket says 172.21.15.165:35892) 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 -- 172.21.15.165:0/2351557969 learned_addr learned my addr 172.21.15.165:0/2351557969 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 -- 172.21.15.165:0/2351557969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e956f7a0 msgr2=0x55c6e95252f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e956f7a0 0x55c6e95252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 -- 172.21.15.165:0/2351557969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 msgr2=0x55c6e9526f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9526f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.880+0000 7f3301000640 1 -- 172.21.15.165:0/2351557969 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e943af40 con 0x55c6e9526b30 2024-04-25T10:11:34.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.881+0000 7f3301000640 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 0x55c6e954e970 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c970 tx=0x7f32f000f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ae01e27d4666b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.881+0000 7f32ed600640 1 -- 172.21.15.165:0/2351557969 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0019040 con 0x55c6e9526b30 2024-04-25T10:11:34.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.881+0000 7f32ed600640 1 -- 172.21.15.165:0/2351557969 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0022070 con 0x55c6e9526b30 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.881+0000 7f32ed600640 1 -- 172.21.15.165:0/2351557969 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f0013d80 con 0x55c6e9526b30 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 -- 172.21.15.165:0/2351557969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 msgr2=0x55c6e954e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 0x55c6e954e970 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f32f000c970 tx=0x7f32f000f000 comp rx=0 tx=0).stop 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 -- 172.21.15.165:0/2351557969 shutdown_connections 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e956f7a0 0x55c6e95252f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9526f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 --2- 172.21.15.165:0/2351557969 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e9526b30 0x55c6e954e970 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 -- 172.21.15.165:0/2351557969 >> 172.21.15.165:0/2351557969 conn(0x55c6e9523d50 msgr2=0x55c6e956c730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 -- 172.21.15.165:0/2351557969 shutdown_connections 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 -- 172.21.15.165:0/2351557969 wait complete. 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.882+0000 7f330677a740 1 -- start start 2024-04-25T10:11:34.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.883+0000 7f330677a740 1 Processor -- start 2024-04-25T10:11:34.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e957c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e953f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e956e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e954e4e0 con 0x55c6e956f7a0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e94291f0 con 0x55c6e953f5d0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6e954ded0 con 0x55c6e954eeb0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f330677a740 10 client.0 _ll_register_callbacks cb 0x55c6e9446ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e953f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f32fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e953f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35904/0 (socket says 172.21.15.165:35904) 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f32fbe00640 1 -- 172.21.15.165:0/1630539439 learned_addr learned my addr 172.21.15.165:0/1630539439 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f32fbe00640 1 -- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 msgr2=0x55c6e957c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f3301a00640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e956e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.884+0000 7f3301000640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e957c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32fbe00640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e957c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32fbe00640 1 -- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 msgr2=0x55c6e956e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32fbe00640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e956e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32fbe00640 1 -- 172.21.15.165:0/1630539439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6e9448c50 con 0x55c6e956f7a0 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f3301000640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e957c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f3301a00640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e956e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32fbe00640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e953f090 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f32e801e030 tx=0x7f32e8057cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b26e137be7f2ce1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32e2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32e2000640 1 -- 172.21.15.165:0/1630539439 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e8057310 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32e2000640 1 -- 172.21.15.165:0/1630539439 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32e8053a10 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32f0022070 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f32e2000640 1 -- 172.21.15.165:0/1630539439 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32e805e0c0 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f330677a740 10 client.7737 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.885+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55c6e919f2d0 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f32e2000640 1 -- 172.21.15.165:0/1630539439 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f0022070 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f32e2000640 1 -- 172.21.15.165:0/1630539439 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32e8053c10 con 0x55c6e956f7a0 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f330677a740 10 client.7737 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f330677a740 10 client.7737 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f330677a740 20 client.7737 populate_metadata read hostname 'smithi165' 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f330677a740 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e954f7d0 0x55c6e954fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f330677a740 4 client.7737 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:11:34.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6e9497b60 con 0x55c6e954f7d0 2024-04-25T10:11:34.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.886+0000 7f3301000640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e954f7d0 0x55c6e954fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.887+0000 7f3301000640 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e954f7d0 0x55c6e954fbb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1250e3aab00cf4e server_cookie=fba7ebba2c2e67af in_seq=0 out_seq=0 2024-04-25T10:11:34.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.887+0000 7f32e2000640 10 client.7737 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f32e2000640 1 -- 172.21.15.165:0/1630539439 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c6e9497b60 con 0x55c6e954f7d0 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f32e2000640 10 client.7737 handle_command_reply: tid=0 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 client.7737 shutdown 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 msgr2=0x55c6e953f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e953f090 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f32e801e030 tx=0x7f32e8057cc0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 shutdown_connections 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e954f7d0 0x55c6e954fbb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e956e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e953f090 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 --2- 172.21.15.165:0/1630539439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e957c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f32e2000640 0 client.7737 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 >> 172.21.15.165:0/1630539439 conn(0x55c6e9526b30 msgr2=0x55c6e9521240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 shutdown_connections 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 1 -- 172.21.15.165:0/1630539439 wait complete. 2024-04-25T10:11:34.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.888+0000 7f330677a740 20 client.7737 trim_cache size 0 max 16384 2024-04-25T10:11:34.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.905+0000 7f330677a740 1 Processor -- start 2024-04-25T10:11:34.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 -- start start 2024-04-25T10:11:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e95553a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e9569690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e9569bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1ff640 con 0x55c6e956f7a0 2024-04-25T10:11:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1e77a0 con 0x55c6e953f5d0 2024-04-25T10:11:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.906+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32fc1e7da0 con 0x55c6e954eeb0 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e9569bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e9569bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35908/0 (socket says 172.21.15.165:35908) 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 -- 172.21.15.165:0/1864158567 learned_addr learned my addr 172.21.15.165:0/1864158567 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301000640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e95553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f32fbe00640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e9569690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 -- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 msgr2=0x55c6e9569690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e9569690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 -- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 msgr2=0x55c6e95553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e95553a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301a00640 1 -- 172.21.15.165:0/1864158567 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6e92696b0 con 0x55c6e956f7a0 2024-04-25T10:11:34.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f3301000640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e95553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.907+0000 7f32fbe00640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e9569690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.908+0000 7f3301a00640 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e9569bd0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f32f4034040 tx=0x7f32f402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47064264bbc9ee00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.908+0000 7f32ecc00640 1 -- 172.21.15.165:0/1864158567 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4042070 con 0x55c6e956f7a0 2024-04-25T10:11:34.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.908+0000 7f32ecc00640 1 -- 172.21.15.165:0/1864158567 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4031680 con 0x55c6e956f7a0 2024-04-25T10:11:34.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.908+0000 7f32ecc00640 1 -- 172.21.15.165:0/1864158567 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4036040 con 0x55c6e956f7a0 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 -- 172.21.15.165:0/1864158567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 msgr2=0x55c6e9569bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e9569bd0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7f32f4034040 tx=0x7f32f402c000 comp rx=0 tx=0).stop 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 -- 172.21.15.165:0/1864158567 shutdown_connections 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e9569bd0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e954eeb0 0x55c6e9569690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 --2- 172.21.15.165:0/1864158567 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e953f5d0 0x55c6e95553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 -- 172.21.15.165:0/1864158567 >> 172.21.15.165:0/1864158567 conn(0x55c6e9526b30 msgr2=0x55c6e951cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 -- 172.21.15.165:0/1864158567 shutdown_connections 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 -- 172.21.15.165:0/1864158567 wait complete. 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.909+0000 7f330677a740 1 -- start start 2024-04-25T10:11:34.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.910+0000 7f330677a740 1 Processor -- start 2024-04-25T10:11:34.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e953f5d0 0x55c6e95255b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9525af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e95df3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e953db90 con 0x55c6e956f7a0 2024-04-25T10:11:34.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6e9580af0 con 0x55c6e954eeb0 2024-04-25T10:11:34.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c6e953d0b0 con 0x55c6e953f5d0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f330677a740 10 client.0 _ll_register_callbacks cb 0x7f32e8053f60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e95df3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e95df3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35924/0 (socket says 172.21.15.165:35924) 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 -- 172.21.15.165:0/658389720 learned_addr learned my addr 172.21.15.165:0/658389720 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301000640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e953f5d0 0x55c6e95255b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f32fbe00640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9525af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 -- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e953f5d0 msgr2=0x55c6e95255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e953f5d0 0x55c6e95255b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 -- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 msgr2=0x55c6e9525af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9525af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.911+0000 7f3301a00640 1 -- 172.21.15.165:0/658389720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6e957c9e0 con 0x55c6e956f7a0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f32fbe00640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9525af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f3301a00640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e95df3e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f32f402ac20 tx=0x7f32f4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77866f3057ed437 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f32e2000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:11:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f32e2000640 1 -- 172.21.15.165:0/658389720 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f404b070 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f32e2000640 1 -- 172.21.15.165:0/658389720 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f32f4031c40 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32f4031680 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f32e2000640 1 -- 172.21.15.165:0/658389720 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32f4042070 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f330677a740 10 client.7743 fetch_fsmap learned FSMap version 65 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.912+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55c6e956e5d0 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f32e2000640 1 -- 172.21.15.165:0/658389720 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32f4031680 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f32e2000640 1 -- 172.21.15.165:0/658389720 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f32f4059080 con 0x55c6e956f7a0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f330677a740 10 client.7743 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f330677a740 10 client.7743 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f330677a740 20 client.7743 populate_metadata read hostname 'smithi165' 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f330677a740 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e956de60 0x55c6e957eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f330677a740 4 client.7743 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c6e9491eb0 con 0x55c6e956de60 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.913+0000 7f3301000640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e956de60 0x55c6e957eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f3301000640 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e956de60 0x55c6e957eea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72828dd76854047f server_cookie=e822e47174ffdd8f in_seq=0 out_seq=0 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f32e2000640 10 client.7743 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f32e2000640 1 -- 172.21.15.165:0/658389720 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55c6e9491eb0 con 0x55c6e956de60 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f32e2000640 10 client.7743 handle_command_reply: tid=0 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 client.7743 shutdown 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 msgr2=0x55c6e95df3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e95df3e0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f32f402ac20 tx=0x7f32f4004ab0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 shutdown_connections 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55c6e956de60 0x55c6e957eea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c6e956f7a0 0x55c6e95df3e0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c6e954eeb0 0x55c6e9525af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 --2- 172.21.15.165:0/658389720 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c6e953f5d0 0x55c6e95255b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f32e2000640 0 client.7743 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 >> 172.21.15.165:0/658389720 conn(0x55c6e9526b30 msgr2=0x55c6e95553a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 shutdown_connections 2024-04-25T10:11:34.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 1 -- 172.21.15.165:0/658389720 wait complete. 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.914+0000 7f330677a740 20 client.7743 trim_cache size 0 max 16384 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 7569, 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 11.421233957, 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 65, 2024-04-25T10:11:34.923 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 98, 2024-04-25T10:11:34.924 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 98, 2024-04-25T10:11:34.924 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 16.922792219000002 2024-04-25T10:11:34.924 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:11:34.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f32dc06b4d0 msgr2=0x7f32dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f32dc06b4d0 0x7f32dc06d990 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f32fc06ec20 tx=0x7f32f0008040 comp rx=0 tx=0).stop 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 msgr2=0x7f32fc15ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 0x7f32fc15ac90 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f32e800c410 tx=0x7f32e800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 shutdown_connections 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f32dc06b4d0 0x7f32dc06d990 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f32fc145140 0x7f32fc1e8980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f32fc0ce950 0x7f32fc15ac90 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 --2- 172.21.15.165:0/829821581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f32fc0c5200 0x7f32fc1e4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 >> 172.21.15.165:0/829821581 conn(0x7f32fc061cc0 msgr2=0x7f32fc0cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 shutdown_connections 2024-04-25T10:11:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:34.915+0000 7f3302400640 1 -- 172.21.15.165:0/829821581 wait complete. 2024-04-25T10:11:34.937 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:11:35.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.100+0000 7fb211200640 1 Processor -- start 2024-04-25T10:11:35.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.100+0000 7fb211200640 1 -- start start 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb211200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c15db70 0x7fb20c162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb211200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb20c0b5c30 0x7fb20c0d27f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb211200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb20c162690 con 0x7fb20c0b6cb0 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb211200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb20c0ceee0 con 0x7fb20c0b5c30 2024-04-25T10:11:35.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb211200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0d1d10 con 0x7fb20c15db70 2024-04-25T10:11:35.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb20a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb20c0b5c30 0x7fb20c0d27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb20a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb20c0b5c30 0x7fb20c0d27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35940/0 (socket says 172.21.15.165:35940) 2024-04-25T10:11:35.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb20a800640 1 -- 172.21.15.165:0/2800207017 learned_addr learned my addr 172.21.15.165:0/2800207017 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:35.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.102+0000 7fb209e00640 1 --2- 172.21.15.165:0/2800207017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c15db70 0x7fb20c162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.103+0000 7fb20a800640 1 -- 172.21.15.165:0/2800207017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c15db70 msgr2=0x7fb20c162150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.103+0000 7fb20a800640 1 --2- 172.21.15.165:0/2800207017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c15db70 0x7fb20c162150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.103+0000 7fb20a800640 1 -- 172.21.15.165:0/2800207017 >> v1:172.21.15.172:6789/0 conn(0x7fb20c0b6cb0 legacy=0x7fb20c0b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.103+0000 7fb20a800640 1 -- 172.21.15.165:0/2800207017 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0d1ff0 con 0x7fb20c0b5c30 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.104+0000 7fb20a800640 1 --2- 172.21.15.165:0/2800207017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c0b5c30 0x7fb20c0d27f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e50 tx=0x7fb20402a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b260234dc1dde2b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.104+0000 7fb209400640 1 -- 172.21.15.165:0/2800207017 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204036070 con 0x7fb20c0b5c30 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.104+0000 7fb209400640 1 -- 172.21.15.165:0/2800207017 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204031600 con 0x7fb20c0b5c30 2024-04-25T10:11:35.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.104+0000 7fb209400640 1 -- 172.21.15.165:0/2800207017 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2040319d0 con 0x7fb20c0b5c30 2024-04-25T10:11:35.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.104+0000 7fb211200640 1 -- 172.21.15.165:0/2800207017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c0b5c30 msgr2=0x7fb20c0d27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.104+0000 7fb211200640 1 --2- 172.21.15.165:0/2800207017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c0b5c30 0x7fb20c0d27f0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e50 tx=0x7fb20402a640 comp rx=0 tx=0).stop 2024-04-25T10:11:35.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.105+0000 7fb211200640 1 -- 172.21.15.165:0/2800207017 shutdown_connections 2024-04-25T10:11:35.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.105+0000 7fb211200640 1 --2- 172.21.15.165:0/2800207017 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c0b5c30 0x7fb20c0d27f0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.105+0000 7fb211200640 1 --2- 172.21.15.165:0/2800207017 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c15db70 0x7fb20c162150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.105+0000 7fb211200640 1 -- 172.21.15.165:0/2800207017 >> 172.21.15.165:0/2800207017 conn(0x7fb20c0d0520 msgr2=0x7fb20c0d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.105+0000 7fb211200640 1 -- 172.21.15.165:0/2800207017 shutdown_connections 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.105+0000 7fb211200640 1 -- 172.21.15.165:0/2800207017 wait complete. 2024-04-25T10:11:35.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.107+0000 7fb211200640 1 Processor -- start 2024-04-25T10:11:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.107+0000 7fb211200640 1 -- start start 2024-04-25T10:11:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb211200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb20c0b5c30 0x7fb20c156640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb211200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c0b6cb0 0x7fb20c0b4950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb211200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 0x7fb20c156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb211200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0ce530 con 0x7fb20c15db70 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb211200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c15df50 con 0x7fb20c0b5c30 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb211200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb20c0cf1c0 con 0x7fb20c0b6cb0 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.108+0000 7fb20a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 0x7fb20c156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 0x7fb20c156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35944/0 (socket says 172.21.15.165:35944) 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 -- 172.21.15.165:0/4255065641 learned_addr learned my addr 172.21.15.165:0/4255065641 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 -- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c0b6cb0 msgr2=0x7fb20c0b4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb201400640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c0b6cb0 0x7fb20c0b4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c0b6cb0 0x7fb20c0b4950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 -- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb20c0b5c30 msgr2=0x7fb20c156640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb209e00640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb20c0b5c30 0x7fb20c156640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb20c0b5c30 0x7fb20c156640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 -- 172.21.15.165:0/4255065641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb20c0b4fe0 con 0x7fb20c15db70 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.109+0000 7fb20a800640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 0x7fb20c156f90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e50 tx=0x7fb204004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.110+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204036050 con 0x7fb20c15db70 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.110+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb204042070 con 0x7fb20c15db70 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.110+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb20c0b52c0 con 0x7fb20c15db70 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.110+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb204009510 con 0x7fb20c15db70 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.110+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb20c0c78b0 con 0x7fb20c15db70 2024-04-25T10:11:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.111+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb20c073990 con 0x7fb20c15db70 2024-04-25T10:11:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.112+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb204007bd0 con 0x7fb20c15db70 2024-04-25T10:11:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.113+0000 7fb202a00640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1e806b4d0 0x7fb1e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:11:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.113+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2040ab3e0 con 0x7fb20c15db70 2024-04-25T10:11:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.113+0000 7fb209e00640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1e806b4d0 0x7fb1e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:11:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.114+0000 7fb209e00640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1e806b4d0 0x7fb1e806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006fd0 tx=0x7fb1fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:11:35.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.117+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb20402ae20 con 0x7fb20c15db70 2024-04-25T10:11:35.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.333+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fb20c06d2e0 con 0x7fb20c15db70 2024-04-25T10:11:35.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.334+0000 7fb202a00640 1 -- 172.21.15.165:0/4255065641 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v65) v1 ==== 53+0+73 (secure 0 0 0) 0x7fb204079fa0 con 0x7fb20c15db70 2024-04-25T10:11:35.339 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.336+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1e806b4d0 msgr2=0x7fb1e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.336+0000 7fb211200640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1e806b4d0 0x7fb1e806d990 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006fd0 tx=0x7fb1fc008040 comp rx=0 tx=0).stop 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.336+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 msgr2=0x7fb20c156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.336+0000 7fb211200640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 0x7fb20c156f90 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fb204009e50 tx=0x7fb204004a40 comp rx=0 tx=0).stop 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 shutdown_connections 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb1e806b4d0 0x7fb1e806d990 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb20c15db70 0x7fb20c156f90 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb20c0b6cb0 0x7fb20c0b4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 --2- 172.21.15.165:0/4255065641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb20c0b5c30 0x7fb20c156640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:11:35.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 >> 172.21.15.165:0/4255065641 conn(0x7fb20c0d0520 msgr2=0x7fb20c157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:11:35.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 shutdown_connections 2024-04-25T10:11:35.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:11:35.337+0000 7fb211200640 1 -- 172.21.15.165:0/4255065641 wait complete. 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:11:35.359 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:11:35.360 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:35.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.360 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:35.411 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 42070sec preferred_lft 42070sec 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.412 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:11:35.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.413 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:11:35.490 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:11:35.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.494 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:11:35.545 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:11:35.545 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:11:35.545 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:11:35.545 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:11:35.545 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:11:35.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.546 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:11:35.546 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:35.546 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:11:35.547 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:11:35.547 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:11:35.547 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:35.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.684 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:11:35.733 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:11:35.733 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:11:35.734 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:35.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:11:35.815 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:35.815 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:35.869 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:11:35.870 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:11:35.870 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:35.919 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:11:35.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:35.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:36.035 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:36.036 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:36.036 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:11:36.056 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:36.056 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:36.056 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:11:36.056 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:11:36.056 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:11:36.084 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:11:36.085 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:11:36.085 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:11:36.087 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:11:36.087 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:11:36.087 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:11:36.087 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:11:36.087 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:11:36.093 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:11:36.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.094 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:11:36.165 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.165 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:11:36.223 INFO:teuthology.orchestra.run.smithi165.stdout:client5321 2024-04-25T10:11:36.223 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:11:36.223 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:11:36.270 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:36.270 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:36.271 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:36.271 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:11:36.271 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:36.271 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:36.271 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:36.271 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.048592 s, 21.6 MB/s 2024-04-25T10:11:36.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.477 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03474 s, 30.2 MB/s 2024-04-25T10:11:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0417927 s, 25.1 MB/s 2024-04-25T10:11:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0371498 s, 28.2 MB/s 2024-04-25T10:11:36.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.603 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0344513 s, 30.4 MB/s 2024-04-25T10:11:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225635 s, 46.5 MB/s 2024-04-25T10:11:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230656 s, 45.5 MB/s 2024-04-25T10:11:36.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178406 s, 58.8 MB/s 2024-04-25T10:11:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0336092 s, 31.2 MB/s 2024-04-25T10:11:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.034963 s, 30.0 MB/s 2024-04-25T10:11:36.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.784 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236701 s, 44.3 MB/s 2024-04-25T10:11:36.829 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.829 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.830 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0416255 s, 25.2 MB/s 2024-04-25T10:11:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334111 s, 31.4 MB/s 2024-04-25T10:11:36.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.890 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195014 s, 53.8 MB/s 2024-04-25T10:11:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215674 s, 48.6 MB/s 2024-04-25T10:11:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033401 s, 31.4 MB/s 2024-04-25T10:11:36.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:36.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:36.987 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291504 s, 36.0 MB/s 2024-04-25T10:11:37.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225087 s, 46.6 MB/s 2024-04-25T10:11:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0364223 s, 28.8 MB/s 2024-04-25T10:11:37.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213667 s, 49.1 MB/s 2024-04-25T10:11:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022526 s, 46.5 MB/s 2024-04-25T10:11:37.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0327276 s, 32.0 MB/s 2024-04-25T10:11:37.174 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.174 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.174 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281993 s, 37.2 MB/s 2024-04-25T10:11:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0480099 s, 21.8 MB/s 2024-04-25T10:11:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267399 s, 39.2 MB/s 2024-04-25T10:11:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245422 s, 42.7 MB/s 2024-04-25T10:11:37.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0404509 s, 25.9 MB/s 2024-04-25T10:11:37.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227202 s, 46.2 MB/s 2024-04-25T10:11:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192193 s, 54.6 MB/s 2024-04-25T10:11:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276921 s, 37.9 MB/s 2024-04-25T10:11:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194804 s, 53.8 MB/s 2024-04-25T10:11:37.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187384 s, 56.0 MB/s 2024-04-25T10:11:37.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:37.462 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:37.462 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:37.462 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:11:37.462 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:37.462 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:37.462 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:37.463 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:37.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.549 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230117 s, 45.6 MB/s 2024-04-25T10:11:37.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.575 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225418 s, 46.5 MB/s 2024-04-25T10:11:37.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.601 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214449 s, 48.9 MB/s 2024-04-25T10:11:37.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245853 s, 42.7 MB/s 2024-04-25T10:11:37.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239032 s, 43.9 MB/s 2024-04-25T10:11:37.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.685 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224851 s, 46.6 MB/s 2024-04-25T10:11:37.712 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.712 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.712 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230954 s, 45.4 MB/s 2024-04-25T10:11:37.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.737 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212136 s, 49.4 MB/s 2024-04-25T10:11:37.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.768 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270081 s, 38.8 MB/s 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.805 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0326429 s, 32.1 MB/s 2024-04-25T10:11:37.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216071 s, 48.5 MB/s 2024-04-25T10:11:37.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224434 s, 46.7 MB/s 2024-04-25T10:11:37.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230721 s, 45.4 MB/s 2024-04-25T10:11:37.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.912 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228299 s, 45.9 MB/s 2024-04-25T10:11:37.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292908 s, 35.8 MB/s 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188015 s, 55.8 MB/s 2024-04-25T10:11:37.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:37.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:37.992 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203004 s, 51.7 MB/s 2024-04-25T10:11:38.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.024 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265002 s, 39.6 MB/s 2024-04-25T10:11:38.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021796 s, 48.1 MB/s 2024-04-25T10:11:38.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249465 s, 42.0 MB/s 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.108 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235634 s, 44.5 MB/s 2024-04-25T10:11:38.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226196 s, 46.4 MB/s 2024-04-25T10:11:38.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.156 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187132 s, 56.0 MB/s 2024-04-25T10:11:38.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.184 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228325 s, 45.9 MB/s 2024-04-25T10:11:38.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.214 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265968 s, 39.4 MB/s 2024-04-25T10:11:38.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.241 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225722 s, 46.5 MB/s 2024-04-25T10:11:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218956 s, 47.9 MB/s 2024-04-25T10:11:38.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.290 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186947 s, 56.1 MB/s 2024-04-25T10:11:38.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192885 s, 54.4 MB/s 2024-04-25T10:11:38.335 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.335 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.335 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182969 s, 57.3 MB/s 2024-04-25T10:11:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213937 s, 49.0 MB/s 2024-04-25T10:11:38.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217819 s, 48.1 MB/s 2024-04-25T10:11:38.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:38.387 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:38.387 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:38.387 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:11:38.388 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:38.388 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:38.388 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:38.388 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:38.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232019 s, 45.2 MB/s 2024-04-25T10:11:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211163 s, 49.7 MB/s 2024-04-25T10:11:38.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215046 s, 48.8 MB/s 2024-04-25T10:11:38.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.545 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018076 s, 58.0 MB/s 2024-04-25T10:11:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185414 s, 56.6 MB/s 2024-04-25T10:11:38.592 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.592 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.592 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218464 s, 48.0 MB/s 2024-04-25T10:11:38.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.614 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018127 s, 57.8 MB/s 2024-04-25T10:11:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189939 s, 55.2 MB/s 2024-04-25T10:11:38.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178707 s, 58.7 MB/s 2024-04-25T10:11:38.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0400878 s, 26.2 MB/s 2024-04-25T10:11:38.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.729 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230202 s, 45.6 MB/s 2024-04-25T10:11:38.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.753 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208507 s, 50.3 MB/s 2024-04-25T10:11:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.780 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022794 s, 46.0 MB/s 2024-04-25T10:11:38.804 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.804 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.804 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193242 s, 54.3 MB/s 2024-04-25T10:11:38.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238947 s, 43.9 MB/s 2024-04-25T10:11:38.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.860 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233861 s, 44.8 MB/s 2024-04-25T10:11:38.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.882 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191701 s, 54.7 MB/s 2024-04-25T10:11:38.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.911 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024093 s, 43.5 MB/s 2024-04-25T10:11:38.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.933 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187414 s, 55.9 MB/s 2024-04-25T10:11:38.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185797 s, 56.4 MB/s 2024-04-25T10:11:38.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:38.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:38.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199589 s, 52.5 MB/s 2024-04-25T10:11:39.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231319 s, 45.3 MB/s 2024-04-25T10:11:39.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.037 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262988 s, 39.9 MB/s 2024-04-25T10:11:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.063 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220848 s, 47.5 MB/s 2024-04-25T10:11:39.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210747 s, 49.8 MB/s 2024-04-25T10:11:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263964 s, 39.7 MB/s 2024-04-25T10:11:39.141 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.141 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.141 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184124 s, 56.9 MB/s 2024-04-25T10:11:39.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252116 s, 41.6 MB/s 2024-04-25T10:11:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190951 s, 54.9 MB/s 2024-04-25T10:11:39.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.224 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267345 s, 39.2 MB/s 2024-04-25T10:11:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183932 s, 57.0 MB/s 2024-04-25T10:11:39.269 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.269 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.270 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178429 s, 58.8 MB/s 2024-04-25T10:11:39.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:39.272 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:39.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.357 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235302 s, 44.6 MB/s 2024-04-25T10:11:39.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.380 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193135 s, 54.3 MB/s 2024-04-25T10:11:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217671 s, 48.2 MB/s 2024-04-25T10:11:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022392 s, 46.8 MB/s 2024-04-25T10:11:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.458 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221034 s, 47.4 MB/s 2024-04-25T10:11:39.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.483 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215024 s, 48.8 MB/s 2024-04-25T10:11:39.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241728 s, 43.4 MB/s 2024-04-25T10:11:39.537 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.538 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224085 s, 46.8 MB/s 2024-04-25T10:11:39.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.561 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198998 s, 52.7 MB/s 2024-04-25T10:11:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.589 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244461 s, 42.9 MB/s 2024-04-25T10:11:39.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.612 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198675 s, 52.8 MB/s 2024-04-25T10:11:39.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.636 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195418 s, 53.7 MB/s 2024-04-25T10:11:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198253 s, 52.9 MB/s 2024-04-25T10:11:39.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.687 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228111 s, 46.0 MB/s 2024-04-25T10:11:39.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.713 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227881 s, 46.0 MB/s 2024-04-25T10:11:39.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.739 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211586 s, 49.6 MB/s 2024-04-25T10:11:39.764 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.765 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207578 s, 50.5 MB/s 2024-04-25T10:11:39.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234014 s, 44.8 MB/s 2024-04-25T10:11:39.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025698 s, 40.8 MB/s 2024-04-25T10:11:39.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.848 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207396 s, 50.6 MB/s 2024-04-25T10:11:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.875 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224154 s, 46.8 MB/s 2024-04-25T10:11:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.905 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255596 s, 41.0 MB/s 2024-04-25T10:11:39.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.930 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213458 s, 49.1 MB/s 2024-04-25T10:11:39.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.957 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234375 s, 44.7 MB/s 2024-04-25T10:11:39.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:39.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:39.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185816 s, 56.4 MB/s 2024-04-25T10:11:40.003 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.004 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201118 s, 52.1 MB/s 2024-04-25T10:11:40.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.032 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242649 s, 43.2 MB/s 2024-04-25T10:11:40.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236214 s, 44.4 MB/s 2024-04-25T10:11:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223273 s, 47.0 MB/s 2024-04-25T10:11:40.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207852 s, 50.4 MB/s 2024-04-25T10:11:40.135 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.136 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215053 s, 48.8 MB/s 2024-04-25T10:11:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238528 s, 44.0 MB/s 2024-04-25T10:11:40.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:40.167 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:40.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242918 s, 43.2 MB/s 2024-04-25T10:11:40.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208934 s, 50.2 MB/s 2024-04-25T10:11:40.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.303 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198698 s, 52.8 MB/s 2024-04-25T10:11:40.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.325 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179403 s, 58.4 MB/s 2024-04-25T10:11:40.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.347 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191305 s, 54.8 MB/s 2024-04-25T10:11:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188822 s, 55.5 MB/s 2024-04-25T10:11:40.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.395 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212401 s, 49.4 MB/s 2024-04-25T10:11:40.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.421 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.421 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212265 s, 49.4 MB/s 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225185 s, 46.6 MB/s 2024-04-25T10:11:40.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208873 s, 50.2 MB/s 2024-04-25T10:11:40.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.492 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167439 s, 62.6 MB/s 2024-04-25T10:11:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207607 s, 50.5 MB/s 2024-04-25T10:11:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193097 s, 54.3 MB/s 2024-04-25T10:11:40.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.562 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187143 s, 56.0 MB/s 2024-04-25T10:11:40.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.588 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219039 s, 47.9 MB/s 2024-04-25T10:11:40.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.612 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200378 s, 52.3 MB/s 2024-04-25T10:11:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161878 s, 64.8 MB/s 2024-04-25T10:11:40.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020945 s, 50.1 MB/s 2024-04-25T10:11:40.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.682 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217859 s, 48.1 MB/s 2024-04-25T10:11:40.712 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.713 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261688 s, 40.1 MB/s 2024-04-25T10:11:40.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.742 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252672 s, 41.5 MB/s 2024-04-25T10:11:40.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.766 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199342 s, 52.6 MB/s 2024-04-25T10:11:40.793 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.793 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.793 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226182 s, 46.4 MB/s 2024-04-25T10:11:40.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198329 s, 52.9 MB/s 2024-04-25T10:11:40.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.842 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218118 s, 48.1 MB/s 2024-04-25T10:11:40.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253432 s, 41.4 MB/s 2024-04-25T10:11:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022261 s, 47.1 MB/s 2024-04-25T10:11:40.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.922 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213206 s, 49.2 MB/s 2024-04-25T10:11:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188975 s, 55.5 MB/s 2024-04-25T10:11:40.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.972 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233429 s, 44.9 MB/s 2024-04-25T10:11:40.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:40.997 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:40.997 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194763 s, 53.8 MB/s 2024-04-25T10:11:41.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.019 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190054 s, 55.2 MB/s 2024-04-25T10:11:41.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:41.022 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:41.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.102 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021906 s, 47.9 MB/s 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.127 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225624 s, 46.5 MB/s 2024-04-25T10:11:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214888 s, 48.8 MB/s 2024-04-25T10:11:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216133 s, 48.5 MB/s 2024-04-25T10:11:41.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244581 s, 42.9 MB/s 2024-04-25T10:11:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022146 s, 47.3 MB/s 2024-04-25T10:11:41.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02224 s, 47.1 MB/s 2024-04-25T10:11:41.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184222 s, 56.9 MB/s 2024-04-25T10:11:41.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.302 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200136 s, 52.4 MB/s 2024-04-25T10:11:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200044 s, 52.4 MB/s 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.351 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213322 s, 49.2 MB/s 2024-04-25T10:11:41.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221937 s, 47.2 MB/s 2024-04-25T10:11:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166141 s, 63.1 MB/s 2024-04-25T10:11:41.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.423 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210823 s, 49.7 MB/s 2024-04-25T10:11:41.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.448 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216846 s, 48.4 MB/s 2024-04-25T10:11:41.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206127 s, 50.9 MB/s 2024-04-25T10:11:41.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.497 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205571 s, 51.0 MB/s 2024-04-25T10:11:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.520 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192381 s, 54.5 MB/s 2024-04-25T10:11:41.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.548 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242537 s, 43.2 MB/s 2024-04-25T10:11:41.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.575 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219231 s, 47.8 MB/s 2024-04-25T10:11:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024048 s, 43.6 MB/s 2024-04-25T10:11:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224847 s, 46.6 MB/s 2024-04-25T10:11:41.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201977 s, 51.9 MB/s 2024-04-25T10:11:41.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210696 s, 49.8 MB/s 2024-04-25T10:11:41.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.705 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231655 s, 45.3 MB/s 2024-04-25T10:11:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230417 s, 45.5 MB/s 2024-04-25T10:11:41.759 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.759 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.759 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223978 s, 46.8 MB/s 2024-04-25T10:11:41.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240603 s, 43.6 MB/s 2024-04-25T10:11:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238868 s, 43.9 MB/s 2024-04-25T10:11:41.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.842 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021489 s, 48.8 MB/s 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.866 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207066 s, 50.6 MB/s 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.890 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208334 s, 50.3 MB/s 2024-04-25T10:11:41.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:41.893 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:41.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.976 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021488 s, 48.8 MB/s 2024-04-25T10:11:41.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:41.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:41.998 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182527 s, 57.4 MB/s 2024-04-25T10:11:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212868 s, 49.3 MB/s 2024-04-25T10:11:42.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194747 s, 53.8 MB/s 2024-04-25T10:11:42.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.070 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201922 s, 51.9 MB/s 2024-04-25T10:11:42.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.092 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184105 s, 57.0 MB/s 2024-04-25T10:11:42.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250346 s, 41.9 MB/s 2024-04-25T10:11:42.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261853 s, 40.0 MB/s 2024-04-25T10:11:42.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231667 s, 45.3 MB/s 2024-04-25T10:11:42.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.202 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01954 s, 53.7 MB/s 2024-04-25T10:11:42.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.230 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244768 s, 42.8 MB/s 2024-04-25T10:11:42.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195216 s, 53.7 MB/s 2024-04-25T10:11:42.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.282 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245377 s, 42.7 MB/s 2024-04-25T10:11:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212341 s, 49.4 MB/s 2024-04-25T10:11:42.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.327 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173836 s, 60.3 MB/s 2024-04-25T10:11:42.353 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.354 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228356 s, 45.9 MB/s 2024-04-25T10:11:42.384 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.384 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.384 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263767 s, 39.8 MB/s 2024-04-25T10:11:42.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214594 s, 48.9 MB/s 2024-04-25T10:11:42.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226155 s, 46.4 MB/s 2024-04-25T10:11:42.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018423 s, 56.9 MB/s 2024-04-25T10:11:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211916 s, 49.5 MB/s 2024-04-25T10:11:42.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.514 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258472 s, 40.6 MB/s 2024-04-25T10:11:42.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220168 s, 47.6 MB/s 2024-04-25T10:11:42.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218274 s, 48.0 MB/s 2024-04-25T10:11:42.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201238 s, 52.1 MB/s 2024-04-25T10:11:42.615 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.615 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.615 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206337 s, 50.8 MB/s 2024-04-25T10:11:42.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.645 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025536 s, 41.1 MB/s 2024-04-25T10:11:42.668 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202108 s, 51.9 MB/s 2024-04-25T10:11:42.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.693 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205127 s, 51.1 MB/s 2024-04-25T10:11:42.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.719 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216526 s, 48.4 MB/s 2024-04-25T10:11:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.741 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182496 s, 57.5 MB/s 2024-04-25T10:11:42.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.767 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217931 s, 48.1 MB/s 2024-04-25T10:11:42.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:42.769 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:42.769 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:42.769 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:11:42.770 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:42.770 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:42.770 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:42.770 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:42.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212279 s, 49.4 MB/s 2024-04-25T10:11:42.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222809 s, 47.1 MB/s 2024-04-25T10:11:42.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.903 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212695 s, 49.3 MB/s 2024-04-25T10:11:42.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221418 s, 47.4 MB/s 2024-04-25T10:11:42.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170139 s, 61.6 MB/s 2024-04-25T10:11:42.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:42.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:42.974 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203294 s, 51.6 MB/s 2024-04-25T10:11:43.003 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.003 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.003 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254938 s, 41.1 MB/s 2024-04-25T10:11:43.031 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.031 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.031 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217777 s, 48.1 MB/s 2024-04-25T10:11:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255089 s, 41.1 MB/s 2024-04-25T10:11:43.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.084 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202314 s, 51.8 MB/s 2024-04-25T10:11:43.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216814 s, 48.4 MB/s 2024-04-25T10:11:43.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.133 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203176 s, 51.6 MB/s 2024-04-25T10:11:43.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168478 s, 62.2 MB/s 2024-04-25T10:11:43.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.178 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204679 s, 51.2 MB/s 2024-04-25T10:11:43.203 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.203 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.203 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214652 s, 48.9 MB/s 2024-04-25T10:11:43.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248715 s, 42.2 MB/s 2024-04-25T10:11:43.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020669 s, 50.7 MB/s 2024-04-25T10:11:43.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218346 s, 48.0 MB/s 2024-04-25T10:11:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193543 s, 54.2 MB/s 2024-04-25T10:11:43.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.327 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174086 s, 60.2 MB/s 2024-04-25T10:11:43.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.350 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191301 s, 54.8 MB/s 2024-04-25T10:11:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247055 s, 42.4 MB/s 2024-04-25T10:11:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208763 s, 50.2 MB/s 2024-04-25T10:11:43.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217807 s, 48.1 MB/s 2024-04-25T10:11:43.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246701 s, 42.5 MB/s 2024-04-25T10:11:43.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.485 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025524 s, 41.1 MB/s 2024-04-25T10:11:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221858 s, 47.3 MB/s 2024-04-25T10:11:43.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237595 s, 44.1 MB/s 2024-04-25T10:11:43.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.562 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192478 s, 54.5 MB/s 2024-04-25T10:11:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.586 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198984 s, 52.7 MB/s 2024-04-25T10:11:43.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.610 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213334 s, 49.2 MB/s 2024-04-25T10:11:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.640 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257157 s, 40.8 MB/s 2024-04-25T10:11:43.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:43.642 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:43.642 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:43.642 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:11:43.642 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:43.642 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:43.643 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:43.643 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:43.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.727 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225717 s, 46.5 MB/s 2024-04-25T10:11:43.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230256 s, 45.5 MB/s 2024-04-25T10:11:43.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203514 s, 51.5 MB/s 2024-04-25T10:11:43.806 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.806 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.806 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242063 s, 43.3 MB/s 2024-04-25T10:11:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019539 s, 53.7 MB/s 2024-04-25T10:11:43.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.851 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183638 s, 57.1 MB/s 2024-04-25T10:11:43.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233808 s, 44.8 MB/s 2024-04-25T10:11:43.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.905 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229489 s, 45.7 MB/s 2024-04-25T10:11:43.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206421 s, 50.8 MB/s 2024-04-25T10:11:43.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.955 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224447 s, 46.7 MB/s 2024-04-25T10:11:43.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:43.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:43.976 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170879 s, 61.4 MB/s 2024-04-25T10:11:44.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.002 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206776 s, 50.7 MB/s 2024-04-25T10:11:44.029 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.030 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241403 s, 43.4 MB/s 2024-04-25T10:11:44.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.053 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199461 s, 52.6 MB/s 2024-04-25T10:11:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214097 s, 49.0 MB/s 2024-04-25T10:11:44.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221655 s, 47.3 MB/s 2024-04-25T10:11:44.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231295 s, 45.3 MB/s 2024-04-25T10:11:44.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177067 s, 59.2 MB/s 2024-04-25T10:11:44.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222488 s, 47.1 MB/s 2024-04-25T10:11:44.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.206 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023148 s, 45.3 MB/s 2024-04-25T10:11:44.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221711 s, 47.3 MB/s 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210474 s, 49.8 MB/s 2024-04-25T10:11:44.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.285 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238663 s, 43.9 MB/s 2024-04-25T10:11:44.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.312 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.312 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222829 s, 47.1 MB/s 2024-04-25T10:11:44.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.334 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188107 s, 55.7 MB/s 2024-04-25T10:11:44.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019016 s, 55.1 MB/s 2024-04-25T10:11:44.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.381 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189199 s, 55.4 MB/s 2024-04-25T10:11:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214042 s, 49.0 MB/s 2024-04-25T10:11:44.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252441 s, 41.5 MB/s 2024-04-25T10:11:44.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.462 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221371 s, 47.4 MB/s 2024-04-25T10:11:44.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192459 s, 54.5 MB/s 2024-04-25T10:11:44.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.509 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.509 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216928 s, 48.3 MB/s 2024-04-25T10:11:44.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:44.511 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:44.511 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:44.512 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:11:44.512 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:44.512 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:44.512 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:44.512 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:44.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.594 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215807 s, 48.6 MB/s 2024-04-25T10:11:44.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258647 s, 40.5 MB/s 2024-04-25T10:11:44.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190041 s, 55.2 MB/s 2024-04-25T10:11:44.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.673 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234178 s, 44.8 MB/s 2024-04-25T10:11:44.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190079 s, 55.2 MB/s 2024-04-25T10:11:44.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226268 s, 46.3 MB/s 2024-04-25T10:11:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230086 s, 45.6 MB/s 2024-04-25T10:11:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215786 s, 48.6 MB/s 2024-04-25T10:11:44.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.803 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225065 s, 46.6 MB/s 2024-04-25T10:11:44.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204349 s, 51.3 MB/s 2024-04-25T10:11:44.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.851 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203024 s, 51.6 MB/s 2024-04-25T10:11:44.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223027 s, 47.0 MB/s 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.905 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229355 s, 45.7 MB/s 2024-04-25T10:11:44.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199304 s, 52.6 MB/s 2024-04-25T10:11:44.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274217 s, 38.2 MB/s 2024-04-25T10:11:44.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:44.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:44.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212481 s, 49.3 MB/s 2024-04-25T10:11:45.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216693 s, 48.4 MB/s 2024-04-25T10:11:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199167 s, 52.6 MB/s 2024-04-25T10:11:45.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.062 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225736 s, 46.5 MB/s 2024-04-25T10:11:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219943 s, 47.7 MB/s 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214836 s, 48.8 MB/s 2024-04-25T10:11:45.139 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.139 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210126 s, 49.9 MB/s 2024-04-25T10:11:45.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219318 s, 47.8 MB/s 2024-04-25T10:11:45.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235874 s, 44.5 MB/s 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231932 s, 45.2 MB/s 2024-04-25T10:11:45.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223255 s, 47.0 MB/s 2024-04-25T10:11:45.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.271 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211254 s, 49.6 MB/s 2024-04-25T10:11:45.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241385 s, 43.4 MB/s 2024-04-25T10:11:45.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.322 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019012 s, 55.2 MB/s 2024-04-25T10:11:45.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.345 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183385 s, 57.2 MB/s 2024-04-25T10:11:45.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.372 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218268 s, 48.0 MB/s 2024-04-25T10:11:45.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257208 s, 40.8 MB/s 2024-04-25T10:11:45.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:45.404 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:45.405 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:45.405 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:11:45.405 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:45.405 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:45.405 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:45.405 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:45.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025191 s, 41.6 MB/s 2024-04-25T10:11:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210091 s, 49.9 MB/s 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.542 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193572 s, 54.2 MB/s 2024-04-25T10:11:45.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216235 s, 48.5 MB/s 2024-04-25T10:11:45.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208673 s, 50.2 MB/s 2024-04-25T10:11:45.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.616 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204924 s, 51.2 MB/s 2024-04-25T10:11:45.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.640 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021013 s, 49.9 MB/s 2024-04-25T10:11:45.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.663 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197305 s, 53.1 MB/s 2024-04-25T10:11:45.691 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249437 s, 42.0 MB/s 2024-04-25T10:11:45.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.719 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229295 s, 45.7 MB/s 2024-04-25T10:11:45.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247494 s, 42.4 MB/s 2024-04-25T10:11:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282622 s, 37.1 MB/s 2024-04-25T10:11:45.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.812 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275158 s, 38.1 MB/s 2024-04-25T10:11:45.835 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.835 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.835 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191811 s, 54.7 MB/s 2024-04-25T10:11:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.861 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214369 s, 48.9 MB/s 2024-04-25T10:11:45.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.884 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193458 s, 54.2 MB/s 2024-04-25T10:11:45.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.910 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213929 s, 49.0 MB/s 2024-04-25T10:11:45.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.936 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.936 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218558 s, 48.0 MB/s 2024-04-25T10:11:45.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.968 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280962 s, 37.3 MB/s 2024-04-25T10:11:45.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:45.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:45.992 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207237 s, 50.6 MB/s 2024-04-25T10:11:46.017 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.018 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214166 s, 49.0 MB/s 2024-04-25T10:11:46.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.044 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221852 s, 47.3 MB/s 2024-04-25T10:11:46.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234363 s, 44.7 MB/s 2024-04-25T10:11:46.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.099 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248749 s, 42.2 MB/s 2024-04-25T10:11:46.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.123 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019397 s, 54.1 MB/s 2024-04-25T10:11:46.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.146 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019182 s, 54.7 MB/s 2024-04-25T10:11:46.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205249 s, 51.1 MB/s 2024-04-25T10:11:46.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212145 s, 49.4 MB/s 2024-04-25T10:11:46.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.215 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170504 s, 61.5 MB/s 2024-04-25T10:11:46.237 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.237 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181719 s, 57.7 MB/s 2024-04-25T10:11:46.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221417 s, 47.4 MB/s 2024-04-25T10:11:46.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.288 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193193 s, 54.3 MB/s 2024-04-25T10:11:46.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:46.291 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:46.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194989 s, 53.8 MB/s 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.395 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195103 s, 53.7 MB/s 2024-04-25T10:11:46.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.420 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219956 s, 47.7 MB/s 2024-04-25T10:11:46.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253147 s, 41.4 MB/s 2024-04-25T10:11:46.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201447 s, 52.1 MB/s 2024-04-25T10:11:46.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275513 s, 38.1 MB/s 2024-04-25T10:11:46.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194129 s, 54.0 MB/s 2024-04-25T10:11:46.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189733 s, 55.3 MB/s 2024-04-25T10:11:46.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.578 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.578 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224678 s, 46.7 MB/s 2024-04-25T10:11:46.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.606 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227325 s, 46.1 MB/s 2024-04-25T10:11:46.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020994 s, 49.9 MB/s 2024-04-25T10:11:46.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188497 s, 55.6 MB/s 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.678 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201458 s, 52.0 MB/s 2024-04-25T10:11:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021416 s, 49.0 MB/s 2024-04-25T10:11:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198189 s, 52.9 MB/s 2024-04-25T10:11:46.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019003 s, 55.2 MB/s 2024-04-25T10:11:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205536 s, 51.0 MB/s 2024-04-25T10:11:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020647 s, 50.8 MB/s 2024-04-25T10:11:46.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217399 s, 48.2 MB/s 2024-04-25T10:11:46.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223108 s, 47.0 MB/s 2024-04-25T10:11:46.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237714 s, 44.1 MB/s 2024-04-25T10:11:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237494 s, 44.2 MB/s 2024-04-25T10:11:46.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214464 s, 48.9 MB/s 2024-04-25T10:11:46.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200853 s, 52.2 MB/s 2024-04-25T10:11:46.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:46.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:46.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223044 s, 47.0 MB/s 2024-04-25T10:11:47.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.006 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023028 s, 45.5 MB/s 2024-04-25T10:11:47.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.033 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225425 s, 46.5 MB/s 2024-04-25T10:11:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177889 s, 58.9 MB/s 2024-04-25T10:11:47.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197264 s, 53.2 MB/s 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.104 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220512 s, 47.6 MB/s 2024-04-25T10:11:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221416 s, 47.4 MB/s 2024-04-25T10:11:47.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.156 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226987 s, 46.2 MB/s 2024-04-25T10:11:47.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:47.163 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:47.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216544 s, 48.4 MB/s 2024-04-25T10:11:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203165 s, 51.6 MB/s 2024-04-25T10:11:47.296 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.296 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.296 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206034 s, 50.9 MB/s 2024-04-25T10:11:47.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.320 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204197 s, 51.4 MB/s 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252229 s, 41.6 MB/s 2024-04-25T10:11:47.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197144 s, 53.2 MB/s 2024-04-25T10:11:47.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.400 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232787 s, 45.0 MB/s 2024-04-25T10:11:47.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217652 s, 48.2 MB/s 2024-04-25T10:11:47.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198636 s, 52.8 MB/s 2024-04-25T10:11:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209344 s, 50.1 MB/s 2024-04-25T10:11:47.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.500 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203278 s, 51.6 MB/s 2024-04-25T10:11:47.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256087 s, 40.9 MB/s 2024-04-25T10:11:47.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.558 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246953 s, 42.5 MB/s 2024-04-25T10:11:47.579 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.580 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171471 s, 61.2 MB/s 2024-04-25T10:11:47.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.605 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211996 s, 49.5 MB/s 2024-04-25T10:11:47.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203998 s, 51.4 MB/s 2024-04-25T10:11:47.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.652 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198469 s, 52.8 MB/s 2024-04-25T10:11:47.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.675 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198632 s, 52.8 MB/s 2024-04-25T10:11:47.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.698 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187644 s, 55.9 MB/s 2024-04-25T10:11:47.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019726 s, 53.2 MB/s 2024-04-25T10:11:47.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021933 s, 47.8 MB/s 2024-04-25T10:11:47.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248912 s, 42.1 MB/s 2024-04-25T10:11:47.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187497 s, 55.9 MB/s 2024-04-25T10:11:47.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222149 s, 47.2 MB/s 2024-04-25T10:11:47.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257495 s, 40.7 MB/s 2024-04-25T10:11:47.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271264 s, 38.7 MB/s 2024-04-25T10:11:47.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.911 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206045 s, 50.9 MB/s 2024-04-25T10:11:47.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.937 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222257 s, 47.2 MB/s 2024-04-25T10:11:47.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183356 s, 57.2 MB/s 2024-04-25T10:11:47.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:47.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:47.983 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186314 s, 56.3 MB/s 2024-04-25T10:11:48.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200654 s, 52.3 MB/s 2024-04-25T10:11:48.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.033 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02183 s, 48.0 MB/s 2024-04-25T10:11:48.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:48.036 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:48.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.111 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.111 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229527 s, 45.7 MB/s 2024-04-25T10:11:48.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208429 s, 50.3 MB/s 2024-04-25T10:11:48.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.168 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313966 s, 33.4 MB/s 2024-04-25T10:11:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211916 s, 49.5 MB/s 2024-04-25T10:11:48.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.224 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02586 s, 40.5 MB/s 2024-04-25T10:11:48.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018404 s, 57.0 MB/s 2024-04-25T10:11:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283583 s, 37.0 MB/s 2024-04-25T10:11:48.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.303 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02144 s, 48.9 MB/s 2024-04-25T10:11:48.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200182 s, 52.4 MB/s 2024-04-25T10:11:48.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178244 s, 58.8 MB/s 2024-04-25T10:11:48.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232009 s, 45.2 MB/s 2024-04-25T10:11:48.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.399 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209732 s, 50.0 MB/s 2024-04-25T10:11:48.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.423 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199887 s, 52.5 MB/s 2024-04-25T10:11:48.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.449 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207948 s, 50.4 MB/s 2024-04-25T10:11:48.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.477 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234083 s, 44.8 MB/s 2024-04-25T10:11:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223114 s, 47.0 MB/s 2024-04-25T10:11:48.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022467 s, 46.7 MB/s 2024-04-25T10:11:48.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.559 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256246 s, 40.9 MB/s 2024-04-25T10:11:48.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.581 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183189 s, 57.2 MB/s 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019793 s, 53.0 MB/s 2024-04-25T10:11:48.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.630 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219345 s, 47.8 MB/s 2024-04-25T10:11:48.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234924 s, 44.6 MB/s 2024-04-25T10:11:48.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.682 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207142 s, 50.6 MB/s 2024-04-25T10:11:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226126 s, 46.4 MB/s 2024-04-25T10:11:48.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.734 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211196 s, 49.6 MB/s 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.763 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250913 s, 41.8 MB/s 2024-04-25T10:11:48.791 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.791 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.791 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240395 s, 43.6 MB/s 2024-04-25T10:11:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235863 s, 44.5 MB/s 2024-04-25T10:11:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244364 s, 42.9 MB/s 2024-04-25T10:11:48.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216081 s, 48.5 MB/s 2024-04-25T10:11:48.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.898 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213595 s, 49.1 MB/s 2024-04-25T10:11:48.926 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:48.926 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:48.926 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239682 s, 43.7 MB/s 2024-04-25T10:11:48.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:48.928 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:49.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243864 s, 43.0 MB/s 2024-04-25T10:11:49.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255354 s, 41.1 MB/s 2024-04-25T10:11:49.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020351 s, 51.5 MB/s 2024-04-25T10:11:49.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022123 s, 47.4 MB/s 2024-04-25T10:11:49.118 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.118 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.118 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218942 s, 47.9 MB/s 2024-04-25T10:11:49.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200974 s, 52.2 MB/s 2024-04-25T10:11:49.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.166 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183695 s, 57.1 MB/s 2024-04-25T10:11:49.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253143 s, 41.4 MB/s 2024-04-25T10:11:49.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218621 s, 48.0 MB/s 2024-04-25T10:11:49.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.244 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192247 s, 54.5 MB/s 2024-04-25T10:11:49.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194712 s, 53.9 MB/s 2024-04-25T10:11:49.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.291 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202527 s, 51.8 MB/s 2024-04-25T10:11:49.316 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.316 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.316 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209065 s, 50.2 MB/s 2024-04-25T10:11:49.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.339 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184633 s, 56.8 MB/s 2024-04-25T10:11:49.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.365 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223867 s, 46.8 MB/s 2024-04-25T10:11:49.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186294 s, 56.3 MB/s 2024-04-25T10:11:49.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227904 s, 46.0 MB/s 2024-04-25T10:11:49.445 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.445 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.445 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277319 s, 37.8 MB/s 2024-04-25T10:11:49.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.468 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018993 s, 55.2 MB/s 2024-04-25T10:11:49.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219698 s, 47.7 MB/s 2024-04-25T10:11:49.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.519 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213121 s, 49.2 MB/s 2024-04-25T10:11:49.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.544 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200211 s, 52.4 MB/s 2024-04-25T10:11:49.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.574 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257044 s, 40.8 MB/s 2024-04-25T10:11:49.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.599 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207819 s, 50.5 MB/s 2024-04-25T10:11:49.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.624 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217342 s, 48.2 MB/s 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.647 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194862 s, 53.8 MB/s 2024-04-25T10:11:49.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.678 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264276 s, 39.7 MB/s 2024-04-25T10:11:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257709 s, 40.7 MB/s 2024-04-25T10:11:49.731 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.731 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.731 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195338 s, 53.7 MB/s 2024-04-25T10:11:49.761 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.761 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.761 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257966 s, 40.6 MB/s 2024-04-25T10:11:49.783 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.783 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.783 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187387 s, 56.0 MB/s 2024-04-25T10:11:49.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199752 s, 52.5 MB/s 2024-04-25T10:11:49.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:49.811 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:49.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.895 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239744 s, 43.7 MB/s 2024-04-25T10:11:49.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218709 s, 47.9 MB/s 2024-04-25T10:11:49.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254181 s, 41.3 MB/s 2024-04-25T10:11:49.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.972 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184162 s, 56.9 MB/s 2024-04-25T10:11:49.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:49.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:49.995 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194268 s, 54.0 MB/s 2024-04-25T10:11:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185853 s, 56.4 MB/s 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.044 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216419 s, 48.5 MB/s 2024-04-25T10:11:50.068 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.068 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.068 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203819 s, 51.4 MB/s 2024-04-25T10:11:50.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.095 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023892 s, 43.9 MB/s 2024-04-25T10:11:50.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.119 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202008 s, 51.9 MB/s 2024-04-25T10:11:50.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.146 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221924 s, 47.2 MB/s 2024-04-25T10:11:50.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.171 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215849 s, 48.6 MB/s 2024-04-25T10:11:50.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198854 s, 52.7 MB/s 2024-04-25T10:11:50.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.221 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216415 s, 48.5 MB/s 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018513 s, 56.6 MB/s 2024-04-25T10:11:50.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020236 s, 51.8 MB/s 2024-04-25T10:11:50.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.291 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202881 s, 51.7 MB/s 2024-04-25T10:11:50.317 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.318 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239178 s, 43.8 MB/s 2024-04-25T10:11:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185108 s, 56.6 MB/s 2024-04-25T10:11:50.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.367 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022819 s, 46.0 MB/s 2024-04-25T10:11:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194393 s, 53.9 MB/s 2024-04-25T10:11:50.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212073 s, 49.4 MB/s 2024-04-25T10:11:50.443 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.444 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.444 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237874 s, 44.1 MB/s 2024-04-25T10:11:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.469 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.469 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209204 s, 50.1 MB/s 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251117 s, 41.8 MB/s 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227093 s, 46.2 MB/s 2024-04-25T10:11:50.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227704 s, 46.0 MB/s 2024-04-25T10:11:50.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206466 s, 50.8 MB/s 2024-04-25T10:11:50.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.602 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234433 s, 44.7 MB/s 2024-04-25T10:11:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218505 s, 48.0 MB/s 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.652 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196256 s, 53.4 MB/s 2024-04-25T10:11:50.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.675 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184678 s, 56.8 MB/s 2024-04-25T10:11:50.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:50.678 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:50.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.762 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223971 s, 46.8 MB/s 2024-04-25T10:11:50.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219977 s, 47.7 MB/s 2024-04-25T10:11:50.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.815 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232592 s, 45.1 MB/s 2024-04-25T10:11:50.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.840 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215784 s, 48.6 MB/s 2024-04-25T10:11:50.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231752 s, 45.2 MB/s 2024-04-25T10:11:50.892 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.892 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.892 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213583 s, 49.1 MB/s 2024-04-25T10:11:50.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240773 s, 43.6 MB/s 2024-04-25T10:11:50.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.940 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173589 s, 60.4 MB/s 2024-04-25T10:11:50.962 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.962 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.962 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174799 s, 60.0 MB/s 2024-04-25T10:11:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221658 s, 47.3 MB/s 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229804 s, 45.6 MB/s 2024-04-25T10:11:51.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236414 s, 44.4 MB/s 2024-04-25T10:11:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226747 s, 46.2 MB/s 2024-04-25T10:11:51.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196505 s, 53.4 MB/s 2024-04-25T10:11:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022976 s, 45.6 MB/s 2024-04-25T10:11:51.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.147 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021318 s, 49.2 MB/s 2024-04-25T10:11:51.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.171 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020681 s, 50.7 MB/s 2024-04-25T10:11:51.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190231 s, 55.1 MB/s 2024-04-25T10:11:51.216 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183903 s, 57.0 MB/s 2024-04-25T10:11:51.242 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.242 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.242 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214194 s, 49.0 MB/s 2024-04-25T10:11:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0159204 s, 65.9 MB/s 2024-04-25T10:11:51.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018199 s, 57.6 MB/s 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227206 s, 46.2 MB/s 2024-04-25T10:11:51.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.334 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021102 s, 49.7 MB/s 2024-04-25T10:11:51.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205723 s, 51.0 MB/s 2024-04-25T10:11:51.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217722 s, 48.2 MB/s 2024-04-25T10:11:51.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.411 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217298 s, 48.3 MB/s 2024-04-25T10:11:51.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197944 s, 53.0 MB/s 2024-04-25T10:11:51.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.460 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205187 s, 51.1 MB/s 2024-04-25T10:11:51.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212181 s, 49.4 MB/s 2024-04-25T10:11:51.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.508 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192296 s, 54.5 MB/s 2024-04-25T10:11:51.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.538 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255976 s, 41.0 MB/s 2024-04-25T10:11:51.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:51.542 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:51.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.627 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210209 s, 49.9 MB/s 2024-04-25T10:11:51.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027705 s, 37.8 MB/s 2024-04-25T10:11:51.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187967 s, 55.8 MB/s 2024-04-25T10:11:51.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.704 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019403 s, 54.0 MB/s 2024-04-25T10:11:51.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193947 s, 54.1 MB/s 2024-04-25T10:11:51.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.753 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214418 s, 48.9 MB/s 2024-04-25T10:11:51.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.781 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234326 s, 44.7 MB/s 2024-04-25T10:11:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202579 s, 51.8 MB/s 2024-04-25T10:11:51.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.830 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223356 s, 46.9 MB/s 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277001 s, 37.9 MB/s 2024-04-25T10:11:51.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.890 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237806 s, 44.1 MB/s 2024-04-25T10:11:51.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.921 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274274 s, 38.2 MB/s 2024-04-25T10:11:51.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.944 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184988 s, 56.7 MB/s 2024-04-25T10:11:51.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197237 s, 53.2 MB/s 2024-04-25T10:11:51.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:51.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:51.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233199 s, 45.0 MB/s 2024-04-25T10:11:52.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.024 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249468 s, 42.0 MB/s 2024-04-25T10:11:52.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234442 s, 44.7 MB/s 2024-04-25T10:11:52.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.076 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209127 s, 50.1 MB/s 2024-04-25T10:11:52.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.096 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167301 s, 62.7 MB/s 2024-04-25T10:11:52.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020105 s, 52.2 MB/s 2024-04-25T10:11:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021975 s, 47.7 MB/s 2024-04-25T10:11:52.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195952 s, 53.5 MB/s 2024-04-25T10:11:52.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297511 s, 35.2 MB/s 2024-04-25T10:11:52.229 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.229 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.229 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209064 s, 50.2 MB/s 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.252 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186258 s, 56.3 MB/s 2024-04-25T10:11:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191568 s, 54.7 MB/s 2024-04-25T10:11:52.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.303 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222853 s, 47.1 MB/s 2024-04-25T10:11:52.328 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.328 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.328 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195985 s, 53.5 MB/s 2024-04-25T10:11:52.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.353 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.353 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214621 s, 48.9 MB/s 2024-04-25T10:11:52.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.379 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219597 s, 47.8 MB/s 2024-04-25T10:11:52.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.405 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216462 s, 48.4 MB/s 2024-04-25T10:11:52.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.431 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.431 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218528 s, 48.0 MB/s 2024-04-25T10:11:52.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:52.434 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:52.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202271 s, 51.8 MB/s 2024-04-25T10:11:52.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024688 s, 42.5 MB/s 2024-04-25T10:11:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218459 s, 48.0 MB/s 2024-04-25T10:11:52.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202764 s, 51.7 MB/s 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.617 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185168 s, 56.6 MB/s 2024-04-25T10:11:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194622 s, 53.9 MB/s 2024-04-25T10:11:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.661 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172755 s, 60.7 MB/s 2024-04-25T10:11:52.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.688 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226212 s, 46.4 MB/s 2024-04-25T10:11:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223617 s, 46.9 MB/s 2024-04-25T10:11:52.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.742 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241967 s, 43.3 MB/s 2024-04-25T10:11:52.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.772 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261528 s, 40.1 MB/s 2024-04-25T10:11:52.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214048 s, 49.0 MB/s 2024-04-25T10:11:52.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201774 s, 52.0 MB/s 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.846 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208217 s, 50.4 MB/s 2024-04-25T10:11:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217718 s, 48.2 MB/s 2024-04-25T10:11:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214865 s, 48.8 MB/s 2024-04-25T10:11:52.926 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247724 s, 42.3 MB/s 2024-04-25T10:11:52.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229537 s, 45.7 MB/s 2024-04-25T10:11:52.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:52.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:52.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226213 s, 46.4 MB/s 2024-04-25T10:11:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246342 s, 42.6 MB/s 2024-04-25T10:11:53.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.044 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305153 s, 34.4 MB/s 2024-04-25T10:11:53.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244601 s, 42.9 MB/s 2024-04-25T10:11:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227371 s, 46.1 MB/s 2024-04-25T10:11:53.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191034 s, 54.9 MB/s 2024-04-25T10:11:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226576 s, 46.3 MB/s 2024-04-25T10:11:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170435 s, 61.5 MB/s 2024-04-25T10:11:53.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.192 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181028 s, 57.9 MB/s 2024-04-25T10:11:53.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.214 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191194 s, 54.8 MB/s 2024-04-25T10:11:53.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214092 s, 49.0 MB/s 2024-04-25T10:11:53.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225517 s, 46.5 MB/s 2024-04-25T10:11:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231138 s, 45.4 MB/s 2024-04-25T10:11:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205821 s, 50.9 MB/s 2024-04-25T10:11:53.321 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:53.322 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187973 s, 55.8 MB/s 2024-04-25T10:11:53.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.428 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211885 s, 49.5 MB/s 2024-04-25T10:11:53.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.451 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185167 s, 56.6 MB/s 2024-04-25T10:11:53.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.474 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189735 s, 55.3 MB/s 2024-04-25T10:11:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209953 s, 49.9 MB/s 2024-04-25T10:11:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226111 s, 46.4 MB/s 2024-04-25T10:11:53.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214629 s, 48.9 MB/s 2024-04-25T10:11:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220793 s, 47.5 MB/s 2024-04-25T10:11:53.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223368 s, 46.9 MB/s 2024-04-25T10:11:53.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183078 s, 57.3 MB/s 2024-04-25T10:11:53.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230718 s, 45.4 MB/s 2024-04-25T10:11:53.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.678 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214998 s, 48.8 MB/s 2024-04-25T10:11:53.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185207 s, 56.6 MB/s 2024-04-25T10:11:53.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.721 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174209 s, 60.2 MB/s 2024-04-25T10:11:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.744 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181909 s, 57.6 MB/s 2024-04-25T10:11:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227326 s, 46.1 MB/s 2024-04-25T10:11:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239015 s, 43.9 MB/s 2024-04-25T10:11:53.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.827 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241569 s, 43.4 MB/s 2024-04-25T10:11:53.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217727 s, 48.2 MB/s 2024-04-25T10:11:53.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275712 s, 38.0 MB/s 2024-04-25T10:11:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170613 s, 61.5 MB/s 2024-04-25T10:11:53.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.938 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253655 s, 41.3 MB/s 2024-04-25T10:11:53.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285364 s, 36.7 MB/s 2024-04-25T10:11:53.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:53.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:53.996 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223028 s, 47.0 MB/s 2024-04-25T10:11:54.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.021 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203563 s, 51.5 MB/s 2024-04-25T10:11:54.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.048 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235421 s, 44.5 MB/s 2024-04-25T10:11:54.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.075 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222884 s, 47.0 MB/s 2024-04-25T10:11:54.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.099 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209919 s, 50.0 MB/s 2024-04-25T10:11:54.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210161 s, 49.9 MB/s 2024-04-25T10:11:54.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.151 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223517 s, 46.9 MB/s 2024-04-25T10:11:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222474 s, 47.1 MB/s 2024-04-25T10:11:54.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.208 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268201 s, 39.1 MB/s 2024-04-25T10:11:54.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:54.211 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196601 s, 53.3 MB/s 2024-04-25T10:11:54.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218131 s, 48.1 MB/s 2024-04-25T10:11:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.337 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202739 s, 51.7 MB/s 2024-04-25T10:11:54.362 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.363 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214179 s, 49.0 MB/s 2024-04-25T10:11:54.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194608 s, 53.9 MB/s 2024-04-25T10:11:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183777 s, 57.1 MB/s 2024-04-25T10:11:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231094 s, 45.4 MB/s 2024-04-25T10:11:54.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226198 s, 46.4 MB/s 2024-04-25T10:11:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209018 s, 50.2 MB/s 2024-04-25T10:11:54.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.507 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175276 s, 59.8 MB/s 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.531 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205516 s, 51.0 MB/s 2024-04-25T10:11:54.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168986 s, 62.1 MB/s 2024-04-25T10:11:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.579 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.579 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223793 s, 46.9 MB/s 2024-04-25T10:11:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184389 s, 56.9 MB/s 2024-04-25T10:11:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219926 s, 47.7 MB/s 2024-04-25T10:11:54.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.654 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219733 s, 47.7 MB/s 2024-04-25T10:11:54.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222244 s, 47.2 MB/s 2024-04-25T10:11:54.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232036 s, 45.2 MB/s 2024-04-25T10:11:54.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198048 s, 52.9 MB/s 2024-04-25T10:11:54.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231665 s, 45.3 MB/s 2024-04-25T10:11:54.783 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.783 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.783 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021587 s, 48.6 MB/s 2024-04-25T10:11:54.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227881 s, 46.0 MB/s 2024-04-25T10:11:54.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237584 s, 44.1 MB/s 2024-04-25T10:11:54.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.860 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188262 s, 55.7 MB/s 2024-04-25T10:11:54.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178816 s, 58.6 MB/s 2024-04-25T10:11:54.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.906 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189646 s, 55.3 MB/s 2024-04-25T10:11:54.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243284 s, 43.1 MB/s 2024-04-25T10:11:54.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216727 s, 48.4 MB/s 2024-04-25T10:11:54.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:54.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:54.983 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193339 s, 54.2 MB/s 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189127 s, 55.4 MB/s 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203939 s, 51.4 MB/s 2024-04-25T10:11:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231161 s, 45.4 MB/s 2024-04-25T10:11:55.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:55.062 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:55.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186317 s, 56.3 MB/s 2024-04-25T10:11:55.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023419 s, 44.8 MB/s 2024-04-25T10:11:55.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.186 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197072 s, 53.2 MB/s 2024-04-25T10:11:55.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.212 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223553 s, 46.9 MB/s 2024-04-25T10:11:55.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.234 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186193 s, 56.3 MB/s 2024-04-25T10:11:55.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.260 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214877 s, 48.8 MB/s 2024-04-25T10:11:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212807 s, 49.3 MB/s 2024-04-25T10:11:55.312 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.312 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.312 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02211 s, 47.4 MB/s 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.337 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205747 s, 51.0 MB/s 2024-04-25T10:11:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188803 s, 55.5 MB/s 2024-04-25T10:11:55.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221853 s, 47.3 MB/s 2024-04-25T10:11:55.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209971 s, 49.9 MB/s 2024-04-25T10:11:55.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209177 s, 50.1 MB/s 2024-04-25T10:11:55.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.460 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187461 s, 55.9 MB/s 2024-04-25T10:11:55.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.487 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226697 s, 46.3 MB/s 2024-04-25T10:11:55.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284113 s, 36.9 MB/s 2024-04-25T10:11:55.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.553 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274212 s, 38.2 MB/s 2024-04-25T10:11:55.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.583 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260052 s, 40.3 MB/s 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200705 s, 52.2 MB/s 2024-04-25T10:11:55.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.632 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193976 s, 54.1 MB/s 2024-04-25T10:11:55.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201694 s, 52.0 MB/s 2024-04-25T10:11:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202208 s, 51.9 MB/s 2024-04-25T10:11:55.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198723 s, 52.8 MB/s 2024-04-25T10:11:55.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020739 s, 50.6 MB/s 2024-04-25T10:11:55.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.756 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242611 s, 43.2 MB/s 2024-04-25T10:11:55.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214785 s, 48.8 MB/s 2024-04-25T10:11:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202575 s, 51.8 MB/s 2024-04-25T10:11:55.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216581 s, 48.4 MB/s 2024-04-25T10:11:55.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255729 s, 41.0 MB/s 2024-04-25T10:11:55.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235498 s, 44.5 MB/s 2024-04-25T10:11:55.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.911 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172017 s, 61.0 MB/s 2024-04-25T10:11:55.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:55.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:55.943 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279888 s, 37.5 MB/s 2024-04-25T10:11:55.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:55.946 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:56.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.024 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185207 s, 56.6 MB/s 2024-04-25T10:11:56.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.047 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188417 s, 55.7 MB/s 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.069 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187342 s, 56.0 MB/s 2024-04-25T10:11:56.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.095 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219465 s, 47.8 MB/s 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227804 s, 46.0 MB/s 2024-04-25T10:11:56.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217739 s, 48.2 MB/s 2024-04-25T10:11:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02018 s, 52.0 MB/s 2024-04-25T10:11:56.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018759 s, 55.9 MB/s 2024-04-25T10:11:56.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.219 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201923 s, 51.9 MB/s 2024-04-25T10:11:56.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217324 s, 48.2 MB/s 2024-04-25T10:11:56.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229521 s, 45.7 MB/s 2024-04-25T10:11:56.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.300 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218945 s, 47.9 MB/s 2024-04-25T10:11:56.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.325 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218557 s, 48.0 MB/s 2024-04-25T10:11:56.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.350 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202643 s, 51.7 MB/s 2024-04-25T10:11:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024856 s, 42.2 MB/s 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.403 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212641 s, 49.3 MB/s 2024-04-25T10:11:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.428 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202783 s, 51.7 MB/s 2024-04-25T10:11:56.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235821 s, 44.5 MB/s 2024-04-25T10:11:56.481 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.481 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.481 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216208 s, 48.5 MB/s 2024-04-25T10:11:56.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.507 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219398 s, 47.8 MB/s 2024-04-25T10:11:56.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.533 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217287 s, 48.3 MB/s 2024-04-25T10:11:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240619 s, 43.6 MB/s 2024-04-25T10:11:56.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.585 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196309 s, 53.4 MB/s 2024-04-25T10:11:56.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189059 s, 55.5 MB/s 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181991 s, 57.6 MB/s 2024-04-25T10:11:56.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.655 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021447 s, 48.9 MB/s 2024-04-25T10:11:56.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215551 s, 48.6 MB/s 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215401 s, 48.7 MB/s 2024-04-25T10:11:56.736 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.736 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.736 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243754 s, 43.0 MB/s 2024-04-25T10:11:56.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.766 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258961 s, 40.5 MB/s 2024-04-25T10:11:56.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289463 s, 36.2 MB/s 2024-04-25T10:11:56.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019446 s, 53.9 MB/s 2024-04-25T10:11:56.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:56.828 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:56.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.914 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285976 s, 36.7 MB/s 2024-04-25T10:11:56.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.937 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197865 s, 53.0 MB/s 2024-04-25T10:11:56.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.966 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249424 s, 42.0 MB/s 2024-04-25T10:11:56.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:56.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:56.990 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020226 s, 51.8 MB/s 2024-04-25T10:11:57.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018214 s, 57.6 MB/s 2024-04-25T10:11:57.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181699 s, 57.7 MB/s 2024-04-25T10:11:57.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.063 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235431 s, 44.5 MB/s 2024-04-25T10:11:57.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.087 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203882 s, 51.4 MB/s 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.112 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213733 s, 49.1 MB/s 2024-04-25T10:11:57.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.136 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020432 s, 51.3 MB/s 2024-04-25T10:11:57.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238122 s, 44.0 MB/s 2024-04-25T10:11:57.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230229 s, 45.5 MB/s 2024-04-25T10:11:57.216 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226033 s, 46.4 MB/s 2024-04-25T10:11:57.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.239 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179729 s, 58.3 MB/s 2024-04-25T10:11:57.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232063 s, 45.2 MB/s 2024-04-25T10:11:57.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.291 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204615 s, 51.2 MB/s 2024-04-25T10:11:57.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198827 s, 52.7 MB/s 2024-04-25T10:11:57.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.340 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211995 s, 49.5 MB/s 2024-04-25T10:11:57.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300232 s, 34.9 MB/s 2024-04-25T10:11:57.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.395 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161836 s, 64.8 MB/s 2024-04-25T10:11:57.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.422 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220587 s, 47.5 MB/s 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.446 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021099 s, 49.7 MB/s 2024-04-25T10:11:57.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209499 s, 50.1 MB/s 2024-04-25T10:11:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204127 s, 51.4 MB/s 2024-04-25T10:11:57.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239014 s, 43.9 MB/s 2024-04-25T10:11:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231808 s, 45.2 MB/s 2024-04-25T10:11:57.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.581 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243014 s, 43.1 MB/s 2024-04-25T10:11:57.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.606 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198333 s, 52.9 MB/s 2024-04-25T10:11:57.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.632 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222383 s, 47.2 MB/s 2024-04-25T10:11:57.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022357 s, 46.9 MB/s 2024-04-25T10:11:57.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01899 s, 55.2 MB/s 2024-04-25T10:11:57.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212397 s, 49.4 MB/s 2024-04-25T10:11:57.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:57.710 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:57.710 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:57.711 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:11:57.711 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:57.711 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:57.711 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:57.711 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:57.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.795 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231601 s, 45.3 MB/s 2024-04-25T10:11:57.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256547 s, 40.9 MB/s 2024-04-25T10:11:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235412 s, 44.5 MB/s 2024-04-25T10:11:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228118 s, 46.0 MB/s 2024-04-25T10:11:57.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.906 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229813 s, 45.6 MB/s 2024-04-25T10:11:57.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237872 s, 44.1 MB/s 2024-04-25T10:11:57.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223639 s, 46.9 MB/s 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:57.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:57.988 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234118 s, 44.8 MB/s 2024-04-25T10:11:58.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02235 s, 46.9 MB/s 2024-04-25T10:11:58.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.037 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194504 s, 53.9 MB/s 2024-04-25T10:11:58.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254105 s, 41.3 MB/s 2024-04-25T10:11:58.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.094 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225897 s, 46.4 MB/s 2024-04-25T10:11:58.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.119 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211886 s, 49.5 MB/s 2024-04-25T10:11:58.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193683 s, 54.1 MB/s 2024-04-25T10:11:58.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231125 s, 45.4 MB/s 2024-04-25T10:11:58.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203729 s, 51.5 MB/s 2024-04-25T10:11:58.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.213 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161889 s, 64.8 MB/s 2024-04-25T10:11:58.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.234 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170321 s, 61.6 MB/s 2024-04-25T10:11:58.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.260 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218881 s, 47.9 MB/s 2024-04-25T10:11:58.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185228 s, 56.6 MB/s 2024-04-25T10:11:58.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178725 s, 58.7 MB/s 2024-04-25T10:11:58.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198112 s, 52.9 MB/s 2024-04-25T10:11:58.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.349 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0154297 s, 68.0 MB/s 2024-04-25T10:11:58.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.373 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193812 s, 54.1 MB/s 2024-04-25T10:11:58.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.396 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187986 s, 55.8 MB/s 2024-04-25T10:11:58.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.420 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196992 s, 53.2 MB/s 2024-04-25T10:11:58.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.446 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227287 s, 46.1 MB/s 2024-04-25T10:11:58.470 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.470 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.470 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204901 s, 51.2 MB/s 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199852 s, 52.5 MB/s 2024-04-25T10:11:58.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.526 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269407 s, 38.9 MB/s 2024-04-25T10:11:58.555 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.555 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.555 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250941 s, 41.8 MB/s 2024-04-25T10:11:58.579 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.579 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.579 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204898 s, 51.2 MB/s 2024-04-25T10:11:58.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:58.582 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:58.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.667 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215115 s, 48.7 MB/s 2024-04-25T10:11:58.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237321 s, 44.2 MB/s 2024-04-25T10:11:58.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.720 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204372 s, 51.3 MB/s 2024-04-25T10:11:58.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.745 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215544 s, 48.6 MB/s 2024-04-25T10:11:58.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207173 s, 50.6 MB/s 2024-04-25T10:11:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.797 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225009 s, 46.6 MB/s 2024-04-25T10:11:58.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.824 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237116 s, 44.2 MB/s 2024-04-25T10:11:58.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.854 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249231 s, 42.1 MB/s 2024-04-25T10:11:58.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217691 s, 48.2 MB/s 2024-04-25T10:11:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023564 s, 44.5 MB/s 2024-04-25T10:11:58.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242379 s, 43.3 MB/s 2024-04-25T10:11:58.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225985 s, 46.4 MB/s 2024-04-25T10:11:58.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:58.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:58.987 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219811 s, 47.7 MB/s 2024-04-25T10:11:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226515 s, 46.3 MB/s 2024-04-25T10:11:59.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.037 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184375 s, 56.9 MB/s 2024-04-25T10:11:59.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188827 s, 55.5 MB/s 2024-04-25T10:11:59.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.084 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201734 s, 52.0 MB/s 2024-04-25T10:11:59.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022226 s, 47.2 MB/s 2024-04-25T10:11:59.127 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:59.136 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:11:59.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225811 s, 46.4 MB/s 2024-04-25T10:11:59.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.158 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01757 s, 59.7 MB/s 2024-04-25T10:11:59.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.181 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200574 s, 52.3 MB/s 2024-04-25T10:11:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210557 s, 49.8 MB/s 2024-04-25T10:11:59.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.229 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207055 s, 50.6 MB/s 2024-04-25T10:11:59.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.256 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240492 s, 43.6 MB/s 2024-04-25T10:11:59.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196725 s, 53.3 MB/s 2024-04-25T10:11:59.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214483 s, 48.9 MB/s 2024-04-25T10:11:59.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225428 s, 46.5 MB/s 2024-04-25T10:11:59.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.358 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206894 s, 50.7 MB/s 2024-04-25T10:11:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.388 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264196 s, 39.7 MB/s 2024-04-25T10:11:59.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214596 s, 48.9 MB/s 2024-04-25T10:11:59.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.439 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216372 s, 48.5 MB/s 2024-04-25T10:11:59.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.465 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216101 s, 48.5 MB/s 2024-04-25T10:11:59.468 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:11:59.468 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:11:59.469 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:11:59.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.602 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266288 s, 39.4 MB/s 2024-04-25T10:11:59.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226449 s, 46.3 MB/s 2024-04-25T10:11:59.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020222 s, 51.9 MB/s 2024-04-25T10:11:59.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235481 s, 44.5 MB/s 2024-04-25T10:11:59.702 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.702 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.702 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185618 s, 56.5 MB/s 2024-04-25T10:11:59.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021664 s, 48.4 MB/s 2024-04-25T10:11:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022403 s, 46.8 MB/s 2024-04-25T10:11:59.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0156998 s, 66.8 MB/s 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239936 s, 43.7 MB/s 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021797 s, 48.1 MB/s 2024-04-25T10:11:59.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.854 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230423 s, 45.5 MB/s 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.875 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176499 s, 59.4 MB/s 2024-04-25T10:11:59.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.903 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237289 s, 44.2 MB/s 2024-04-25T10:11:59.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.930 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232136 s, 45.2 MB/s 2024-04-25T10:11:59.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.957 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235553 s, 44.5 MB/s 2024-04-25T10:11:59.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:11:59.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:11:59.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190835 s, 54.9 MB/s 2024-04-25T10:12:00.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.008 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238287 s, 44.0 MB/s 2024-04-25T10:12:00.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.030 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192452 s, 54.5 MB/s 2024-04-25T10:12:00.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.055 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203651 s, 51.5 MB/s 2024-04-25T10:12:00.081 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.081 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.081 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226799 s, 46.2 MB/s 2024-04-25T10:12:00.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206398 s, 50.8 MB/s 2024-04-25T10:12:00.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233245 s, 45.0 MB/s 2024-04-25T10:12:00.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.158 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197104 s, 53.2 MB/s 2024-04-25T10:12:00.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.181 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191801 s, 54.7 MB/s 2024-04-25T10:12:00.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187135 s, 56.0 MB/s 2024-04-25T10:12:00.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221182 s, 47.4 MB/s 2024-04-25T10:12:00.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167265 s, 62.7 MB/s 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213665 s, 49.1 MB/s 2024-04-25T10:12:00.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185761 s, 56.4 MB/s 2024-04-25T10:12:00.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.325 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212692 s, 49.3 MB/s 2024-04-25T10:12:00.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237024 s, 44.2 MB/s 2024-04-25T10:12:00.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202841 s, 51.7 MB/s 2024-04-25T10:12:00.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:00.379 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:00.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192194 s, 54.6 MB/s 2024-04-25T10:12:00.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.485 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201181 s, 52.1 MB/s 2024-04-25T10:12:00.509 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.509 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.509 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196542 s, 53.4 MB/s 2024-04-25T10:12:00.541 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.541 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.541 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275343 s, 38.1 MB/s 2024-04-25T10:12:00.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209536 s, 50.0 MB/s 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.590 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194509 s, 53.9 MB/s 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.614 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208812 s, 50.2 MB/s 2024-04-25T10:12:00.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258458 s, 40.6 MB/s 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021697 s, 48.3 MB/s 2024-04-25T10:12:00.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.694 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173429 s, 60.5 MB/s 2024-04-25T10:12:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226532 s, 46.3 MB/s 2024-04-25T10:12:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213606 s, 49.1 MB/s 2024-04-25T10:12:00.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193569 s, 54.2 MB/s 2024-04-25T10:12:00.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.797 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235254 s, 44.6 MB/s 2024-04-25T10:12:00.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.819 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184873 s, 56.7 MB/s 2024-04-25T10:12:00.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215694 s, 48.6 MB/s 2024-04-25T10:12:00.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197858 s, 53.0 MB/s 2024-04-25T10:12:00.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238174 s, 44.0 MB/s 2024-04-25T10:12:00.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023946 s, 43.8 MB/s 2024-04-25T10:12:00.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252488 s, 41.5 MB/s 2024-04-25T10:12:00.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:00.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:00.976 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183301 s, 57.2 MB/s 2024-04-25T10:12:01.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.002 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211605 s, 49.6 MB/s 2024-04-25T10:12:01.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.031 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250026 s, 41.9 MB/s 2024-04-25T10:12:01.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.055 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204959 s, 51.2 MB/s 2024-04-25T10:12:01.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209938 s, 49.9 MB/s 2024-04-25T10:12:01.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261408 s, 40.1 MB/s 2024-04-25T10:12:01.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246688 s, 42.5 MB/s 2024-04-25T10:12:01.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.162 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195018 s, 53.8 MB/s 2024-04-25T10:12:01.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.185 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189753 s, 55.3 MB/s 2024-04-25T10:12:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.211 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218067 s, 48.1 MB/s 2024-04-25T10:12:01.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.235 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019227 s, 54.5 MB/s 2024-04-25T10:12:01.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.258 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187813 s, 55.8 MB/s 2024-04-25T10:12:01.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:01.260 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:01.260 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:01.261 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:12:01.261 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:01.261 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:01.261 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:01.261 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209808 s, 50.0 MB/s 2024-04-25T10:12:01.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.365 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224198 s, 46.8 MB/s 2024-04-25T10:12:01.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.389 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204485 s, 51.3 MB/s 2024-04-25T10:12:01.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160867 s, 65.2 MB/s 2024-04-25T10:12:01.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.434 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229561 s, 45.7 MB/s 2024-04-25T10:12:01.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.457 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019375 s, 54.1 MB/s 2024-04-25T10:12:01.480 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.480 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.480 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192897 s, 54.4 MB/s 2024-04-25T10:12:01.506 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.507 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234267 s, 44.8 MB/s 2024-04-25T10:12:01.531 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.531 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.531 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210719 s, 49.8 MB/s 2024-04-25T10:12:01.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.561 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252772 s, 41.5 MB/s 2024-04-25T10:12:01.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.586 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218797 s, 47.9 MB/s 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.617 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268726 s, 39.0 MB/s 2024-04-25T10:12:01.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.642 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201356 s, 52.1 MB/s 2024-04-25T10:12:01.668 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.668 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.668 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217542 s, 48.2 MB/s 2024-04-25T10:12:01.689 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.690 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.690 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178724 s, 58.7 MB/s 2024-04-25T10:12:01.715 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.715 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.715 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021635 s, 48.5 MB/s 2024-04-25T10:12:01.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.743 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223547 s, 46.9 MB/s 2024-04-25T10:12:01.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.772 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251861 s, 41.6 MB/s 2024-04-25T10:12:01.796 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.796 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.796 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202605 s, 51.8 MB/s 2024-04-25T10:12:01.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231794 s, 45.2 MB/s 2024-04-25T10:12:01.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260436 s, 40.3 MB/s 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02558 s, 41.0 MB/s 2024-04-25T10:12:01.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217374 s, 48.2 MB/s 2024-04-25T10:12:01.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211795 s, 49.5 MB/s 2024-04-25T10:12:01.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.958 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200586 s, 52.3 MB/s 2024-04-25T10:12:01.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:01.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:01.982 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188033 s, 55.8 MB/s 2024-04-25T10:12:02.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232097 s, 45.2 MB/s 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.035 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225239 s, 46.6 MB/s 2024-04-25T10:12:02.058 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203512 s, 51.5 MB/s 2024-04-25T10:12:02.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.086 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234216 s, 44.8 MB/s 2024-04-25T10:12:02.111 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.112 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224512 s, 46.7 MB/s 2024-04-25T10:12:02.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264953 s, 39.6 MB/s 2024-04-25T10:12:02.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:02.145 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:02.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243317 s, 43.1 MB/s 2024-04-25T10:12:02.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.256 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201661 s, 52.0 MB/s 2024-04-25T10:12:02.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209222 s, 50.1 MB/s 2024-04-25T10:12:02.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208751 s, 50.2 MB/s 2024-04-25T10:12:02.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.332 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219798 s, 47.7 MB/s 2024-04-25T10:12:02.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.356 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202374 s, 51.8 MB/s 2024-04-25T10:12:02.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186272 s, 56.3 MB/s 2024-04-25T10:12:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197488 s, 53.1 MB/s 2024-04-25T10:12:02.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.428 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221859 s, 47.3 MB/s 2024-04-25T10:12:02.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.457 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024931 s, 42.1 MB/s 2024-04-25T10:12:02.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234119 s, 44.8 MB/s 2024-04-25T10:12:02.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273951 s, 38.3 MB/s 2024-04-25T10:12:02.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.542 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216049 s, 48.5 MB/s 2024-04-25T10:12:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.568 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222302 s, 47.2 MB/s 2024-04-25T10:12:02.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185162 s, 56.6 MB/s 2024-04-25T10:12:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.616 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215689 s, 48.6 MB/s 2024-04-25T10:12:02.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.638 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192565 s, 54.5 MB/s 2024-04-25T10:12:02.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.660 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.660 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173451 s, 60.5 MB/s 2024-04-25T10:12:02.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.686 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228797 s, 45.8 MB/s 2024-04-25T10:12:02.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.714 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233577 s, 44.9 MB/s 2024-04-25T10:12:02.735 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.735 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.736 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187601 s, 55.9 MB/s 2024-04-25T10:12:02.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.760 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201438 s, 52.1 MB/s 2024-04-25T10:12:02.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.784 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206451 s, 50.8 MB/s 2024-04-25T10:12:02.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.812 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239809 s, 43.7 MB/s 2024-04-25T10:12:02.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.842 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260457 s, 40.3 MB/s 2024-04-25T10:12:02.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021511 s, 48.7 MB/s 2024-04-25T10:12:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255281 s, 41.1 MB/s 2024-04-25T10:12:02.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.919 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018323 s, 57.2 MB/s 2024-04-25T10:12:02.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.944 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218302 s, 48.0 MB/s 2024-04-25T10:12:02.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.974 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251368 s, 41.7 MB/s 2024-04-25T10:12:02.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:02.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:02.997 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191382 s, 54.8 MB/s 2024-04-25T10:12:03.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.021 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211028 s, 49.7 MB/s 2024-04-25T10:12:03.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.023 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:03.023 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:03.024 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:12:03.024 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:03.024 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:03.024 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:03.024 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:03.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.159 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222787 s, 47.1 MB/s 2024-04-25T10:12:03.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.185 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222345 s, 47.2 MB/s 2024-04-25T10:12:03.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.210 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216412 s, 48.5 MB/s 2024-04-25T10:12:03.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.240 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252122 s, 41.6 MB/s 2024-04-25T10:12:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209516 s, 50.0 MB/s 2024-04-25T10:12:03.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.290 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216469 s, 48.4 MB/s 2024-04-25T10:12:03.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188895 s, 55.5 MB/s 2024-04-25T10:12:03.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.339 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208476 s, 50.3 MB/s 2024-04-25T10:12:03.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.363 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207308 s, 50.6 MB/s 2024-04-25T10:12:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200522 s, 52.3 MB/s 2024-04-25T10:12:03.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220061 s, 47.6 MB/s 2024-04-25T10:12:03.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019267 s, 54.4 MB/s 2024-04-25T10:12:03.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.457 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170399 s, 61.5 MB/s 2024-04-25T10:12:03.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0148738 s, 70.5 MB/s 2024-04-25T10:12:03.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240128 s, 43.7 MB/s 2024-04-25T10:12:03.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.534 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246178 s, 42.6 MB/s 2024-04-25T10:12:03.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219026 s, 47.9 MB/s 2024-04-25T10:12:03.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.589 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250912 s, 41.8 MB/s 2024-04-25T10:12:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179552 s, 58.4 MB/s 2024-04-25T10:12:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217519 s, 48.2 MB/s 2024-04-25T10:12:03.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.661 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210765 s, 49.8 MB/s 2024-04-25T10:12:03.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.686 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213451 s, 49.1 MB/s 2024-04-25T10:12:03.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203321 s, 51.6 MB/s 2024-04-25T10:12:03.735 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.735 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.735 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020144 s, 52.1 MB/s 2024-04-25T10:12:03.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166012 s, 63.2 MB/s 2024-04-25T10:12:03.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222216 s, 47.2 MB/s 2024-04-25T10:12:03.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228756 s, 45.8 MB/s 2024-04-25T10:12:03.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201186 s, 52.1 MB/s 2024-04-25T10:12:03.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225278 s, 46.5 MB/s 2024-04-25T10:12:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231853 s, 45.2 MB/s 2024-04-25T10:12:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222755 s, 47.1 MB/s 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:03.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:03.944 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027889 s, 37.6 MB/s 2024-04-25T10:12:03.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:03.946 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:03.946 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:03.946 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:12:03.946 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:03.946 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:03.947 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:03.947 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:04.027 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.027 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.027 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023854 s, 44.0 MB/s 2024-04-25T10:12:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225796 s, 46.4 MB/s 2024-04-25T10:12:04.082 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.082 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.082 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259875 s, 40.3 MB/s 2024-04-25T10:12:04.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.107 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209016 s, 50.2 MB/s 2024-04-25T10:12:04.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.131 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198489 s, 52.8 MB/s 2024-04-25T10:12:04.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178161 s, 58.9 MB/s 2024-04-25T10:12:04.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023573 s, 44.5 MB/s 2024-04-25T10:12:04.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217683 s, 48.2 MB/s 2024-04-25T10:12:04.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223812 s, 46.9 MB/s 2024-04-25T10:12:04.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189662 s, 55.3 MB/s 2024-04-25T10:12:04.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198117 s, 52.9 MB/s 2024-04-25T10:12:04.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.301 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190616 s, 55.0 MB/s 2024-04-25T10:12:04.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256589 s, 40.9 MB/s 2024-04-25T10:12:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.353 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196855 s, 53.3 MB/s 2024-04-25T10:12:04.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.377 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197035 s, 53.2 MB/s 2024-04-25T10:12:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.401 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207436 s, 50.5 MB/s 2024-04-25T10:12:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186709 s, 56.2 MB/s 2024-04-25T10:12:04.452 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.452 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.452 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238418 s, 44.0 MB/s 2024-04-25T10:12:04.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174408 s, 60.1 MB/s 2024-04-25T10:12:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210223 s, 49.9 MB/s 2024-04-25T10:12:04.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.524 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219195 s, 47.8 MB/s 2024-04-25T10:12:04.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.549 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210202 s, 49.9 MB/s 2024-04-25T10:12:04.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.581 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028436 s, 36.9 MB/s 2024-04-25T10:12:04.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198943 s, 52.7 MB/s 2024-04-25T10:12:04.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019429 s, 54.0 MB/s 2024-04-25T10:12:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169111 s, 62.0 MB/s 2024-04-25T10:12:04.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254714 s, 41.2 MB/s 2024-04-25T10:12:04.702 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215404 s, 48.7 MB/s 2024-04-25T10:12:04.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.727 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204546 s, 51.3 MB/s 2024-04-25T10:12:04.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227908 s, 46.0 MB/s 2024-04-25T10:12:04.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.781 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235187 s, 44.6 MB/s 2024-04-25T10:12:04.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:04.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:04.805 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212084 s, 49.4 MB/s 2024-04-25T10:12:04.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:04.808 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:12:05.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.002+0000 7f42c1c00640 1 Processor -- start 2024-04-25T10:12:05.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.003+0000 7f42c1c00640 1 -- start start 2024-04-25T10:12:05.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.004+0000 7f42c1c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f42bc161240 0x7f42bc0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.004+0000 7f42c1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc161890 0x7f42bc0ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.004+0000 7f42c1c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42bc161620 con 0x7f42bc0b54a0 2024-04-25T10:12:05.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.004+0000 7f42c1c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42bc0b7120 con 0x7f42bc161240 2024-04-25T10:12:05.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.004+0000 7f42c1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc161e00 con 0x7f42bc161890 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f42bc161240 0x7f42bc0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f42bc161240 0x7f42bc0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36604/0 (socket says 172.21.15.165:36604) 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 -- 172.21.15.165:0/2708279363 learned_addr learned my addr 172.21.15.165:0/2708279363 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 -- 172.21.15.165:0/2708279363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc161890 msgr2=0x7f42bc0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 --2- 172.21.15.165:0/2708279363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc161890 0x7f42bc0ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 -- 172.21.15.165:0/2708279363 >> v1:172.21.15.172:6789/0 conn(0x7f42bc0b54a0 legacy=0x7f42bc0d1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.005+0000 7f42baa00640 1 -- 172.21.15.165:0/2708279363 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc1620e0 con 0x7f42bc161240 2024-04-25T10:12:05.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.006+0000 7f42baa00640 1 --2- 172.21.15.165:0/2708279363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0b6be0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009870 tx=0x7f42ac02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792239630744041d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42b9600640 1 -- 172.21.15.165:0/2708279363 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac037070 con 0x7f42bc161240 2024-04-25T10:12:05.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42b9600640 1 -- 172.21.15.165:0/2708279363 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac032440 con 0x7f42bc161240 2024-04-25T10:12:05.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42b9600640 1 -- 172.21.15.165:0/2708279363 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac032810 con 0x7f42bc161240 2024-04-25T10:12:05.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 -- 172.21.15.165:0/2708279363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 msgr2=0x7f42bc0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 --2- 172.21.15.165:0/2708279363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0b6be0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f42ac009870 tx=0x7f42ac02a210 comp rx=0 tx=0).stop 2024-04-25T10:12:05.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 -- 172.21.15.165:0/2708279363 shutdown_connections 2024-04-25T10:12:05.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 --2- 172.21.15.165:0/2708279363 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc161890 0x7f42bc0ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 --2- 172.21.15.165:0/2708279363 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0b6be0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 -- 172.21.15.165:0/2708279363 >> 172.21.15.165:0/2708279363 conn(0x7f42bc0d04a0 msgr2=0x7f42bc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 -- 172.21.15.165:0/2708279363 shutdown_connections 2024-04-25T10:12:05.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.007+0000 7f42c1c00640 1 -- 172.21.15.165:0/2708279363 wait complete. 2024-04-25T10:12:05.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.009+0000 7f42c1c00640 1 Processor -- start 2024-04-25T10:12:05.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.009+0000 7f42c1c00640 1 -- start start 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.010+0000 7f42c1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc0b54a0 0x7f42bc0c9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.010+0000 7f42c1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.010+0000 7f42c1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42bc161890 0x7f42bc0ccc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.010+0000 7f42c1c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0d15f0 con 0x7f42bc161240 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.010+0000 7f42c1c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1626a0 con 0x7f42bc161890 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.010+0000 7f42c1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc0cf000 con 0x7f42bc0b54a0 2024-04-25T10:12:05.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36606/0 (socket says 172.21.15.165:36606) 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 -- 172.21.15.165:0/873936479 learned_addr learned my addr 172.21.15.165:0/873936479 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 -- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc0b54a0 msgr2=0x7f42bc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc0b54a0 0x7f42bc0c9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42bb400640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42bc161890 0x7f42bc0ccc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 -- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42bc161890 msgr2=0x7f42bc0ccc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42bc161890 0x7f42bc0ccc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 -- 172.21.15.165:0/873936479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42bc0c9770 con 0x7f42bc161240 2024-04-25T10:12:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.011+0000 7f42ba000640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0cc740 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f42a800b440 tx=0x7f42a800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.012+0000 7f42b2a00640 1 -- 172.21.15.165:0/873936479 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8015020 con 0x7f42bc161240 2024-04-25T10:12:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.012+0000 7f42b2a00640 1 -- 172.21.15.165:0/873936479 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a8011890 con 0x7f42bc161240 2024-04-25T10:12:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.012+0000 7f42b2a00640 1 -- 172.21.15.165:0/873936479 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8011c80 con 0x7f42bc161240 2024-04-25T10:12:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.012+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42bc0c9a50 con 0x7f42bc161240 2024-04-25T10:12:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.012+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42bc0ca010 con 0x7f42bc161240 2024-04-25T10:12:05.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.014+0000 7f42b2a00640 1 -- 172.21.15.165:0/873936479 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42a8018020 con 0x7f42bc161240 2024-04-25T10:12:05.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.015+0000 7f42b2a00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f429c06b4d0 0x7f429c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.015+0000 7f42b2a00640 1 -- 172.21.15.165:0/873936479 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f42a808a500 con 0x7f42bc161240 2024-04-25T10:12:05.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.015+0000 7f42baa00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f429c06b4d0 0x7f429c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.016+0000 7f42c49c1740 1 Processor -- start 2024-04-25T10:12:05.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.016+0000 7f42c49c1740 1 -- start start 2024-04-25T10:12:05.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.016+0000 7f42baa00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f429c06b4d0 0x7f429c06d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f42ac034000 tx=0x7f42ac02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c56a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c53fab0 0x55d17c585d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42c49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17c49df00 con 0x55d17c569dd0 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17c4a2f50 con 0x55d17c587670 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d17c44c0b0 con 0x55d17c53fab0 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c56a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c56a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36618/0 (socket says 172.21.15.165:36618) 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.017+0000 7f42baa00640 1 -- 172.21.15.165:0/2274583617 learned_addr learned my addr 172.21.15.165:0/2274583617 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42ba000640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42bb400640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c53fab0 0x55d17c585d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42baa00640 1 -- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c53fab0 msgr2=0x55d17c585d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42baa00640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c53fab0 0x55d17c585d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42baa00640 1 -- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 msgr2=0x55d17c587bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42baa00640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42baa00640 1 -- 172.21.15.165:0/2274583617 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17c283ba0 con 0x55d17c569dd0 2024-04-25T10:12:05.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42ba000640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42bb400640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c53fab0 0x55d17c585d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42baa00640 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c56a1b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f42ac02a250 tx=0x7f42ac008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4843f7a1ab931be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42a7400640 1 -- 172.21.15.165:0/2274583617 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac037040 con 0x55d17c569dd0 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42a7400640 1 -- 172.21.15.165:0/2274583617 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac04d070 con 0x55d17c569dd0 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.018+0000 7f42a7400640 1 -- 172.21.15.165:0/2274583617 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac047680 con 0x55d17c569dd0 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 -- 172.21.15.165:0/2274583617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 msgr2=0x55d17c56a1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c56a1b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f42ac02a250 tx=0x7f42ac008030 comp rx=0 tx=0).stop 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 -- 172.21.15.165:0/2274583617 shutdown_connections 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c53fab0 0x55d17c585d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2274583617 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c56a1b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 -- 172.21.15.165:0/2274583617 >> 172.21.15.165:0/2274583617 conn(0x55d17c53c670 msgr2=0x55d17c584be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 -- 172.21.15.165:0/2274583617 shutdown_connections 2024-04-25T10:12:05.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.019+0000 7f42c49c1740 1 -- 172.21.15.165:0/2274583617 wait complete. 2024-04-25T10:12:05.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.020+0000 7f42c49c1740 1 -- start start 2024-04-25T10:12:05.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.020+0000 7f42c49c1740 1 Processor -- start 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c5990f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c56d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c594590 0x55d17c594a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17c441620 con 0x55d17c569dd0 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17c49eff0 con 0x55d17c587670 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d17c540850 con 0x55d17c594590 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42c49c1740 10 client.0 _ll_register_callbacks cb 0x55d17c1b6f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c5990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42baa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c5990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36622/0 (socket says 172.21.15.165:36622) 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.021+0000 7f42baa00640 1 -- 172.21.15.165:0/2682630398 learned_addr learned my addr 172.21.15.165:0/2682630398 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42ba000640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c56d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42bb400640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c594590 0x55d17c594a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42baa00640 1 -- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c594590 msgr2=0x55d17c594a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42baa00640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c594590 0x55d17c594a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42baa00640 1 -- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 msgr2=0x55d17c56d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42baa00640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c56d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42baa00640 1 -- 172.21.15.165:0/2682630398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d17c177490 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f42baa00640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c5990f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f42ac02a250 tx=0x7f42ac004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e6108b04f8a2bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f429be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.022+0000 7f429be00640 1 -- 172.21.15.165:0/2682630398 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac055070 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42ac04d070 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f429be00640 1 -- 172.21.15.165:0/2682630398 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f42ac003810 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f429be00640 1 -- 172.21.15.165:0/2682630398 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42ac062050 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 10 client.7761 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d17c2e4330 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f429be00640 1 -- 172.21.15.165:0/2682630398 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac04d070 con 0x55d17c569dd0 2024-04-25T10:12:05.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f429be00640 1 -- 172.21.15.165:0/2682630398 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f42ac065020 con 0x55d17c569dd0 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 10 client.7761 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 10 client.7761 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 20 client.7761 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c583750 0x55d17c583ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 4 client.7761 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.023+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d17c4b0490 con 0x55d17c583750 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.024+0000 7f42ba000640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c583750 0x55d17c583ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.024+0000 7f429be00640 10 client.7761 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.024+0000 7f42ba000640 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c583750 0x55d17c583ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54ac931649286575 server_cookie=b6be5829ba25c2c8 in_seq=0 out_seq=0 2024-04-25T10:12:05.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f429be00640 1 -- 172.21.15.165:0/2682630398 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x7f42a8087a90 con 0x55d17c583750 2024-04-25T10:12:05.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f429be00640 10 client.7761 handle_command_reply: tid=0 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 client.7761 shutdown 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 msgr2=0x55d17c5990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c5990f0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f42ac02a250 tx=0x7f42ac004730 comp rx=0 tx=0).stop 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 shutdown_connections 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c583750 0x55d17c583ba0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c594590 0x55d17c594a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c56d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2682630398 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c569dd0 0x55d17c5990f0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 >> 172.21.15.165:0/2682630398 conn(0x55d17c53fab0 msgr2=0x55d17c53ed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f429be00640 0 client.7761 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.025+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 shutdown_connections 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.026+0000 7f42c49c1740 1 -- 172.21.15.165:0/2682630398 wait complete. 2024-04-25T10:12:05.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.026+0000 7f42c49c1740 20 client.7761 trim_cache size 0 max 16384 2024-04-25T10:12:05.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.042+0000 7f42c49c1740 1 Processor -- start 2024-04-25T10:12:05.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.042+0000 7f42c49c1740 1 -- start start 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c569dd0 0x55d17c56db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c587670 0x55d17c53fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c5403d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42c49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc200830 con 0x55d17c594590 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1ff9c0 con 0x55d17c569dd0 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1e84e0 con 0x55d17c587670 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c5403d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c5403d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36638/0 (socket says 172.21.15.165:36638) 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.043+0000 7f42bb400640 1 -- 172.21.15.165:0/2308187443 learned_addr learned my addr 172.21.15.165:0/2308187443 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42baa00640 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c569dd0 0x55d17c56db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42bb400640 1 -- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c587670 msgr2=0x55d17c53fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42bb400640 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c587670 0x55d17c53fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42bb400640 1 -- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c569dd0 msgr2=0x55d17c56db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42bb400640 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c569dd0 0x55d17c56db30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42bb400640 1 -- 172.21.15.165:0/2308187443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d17c150610 con 0x55d17c594590 2024-04-25T10:12:05.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42bb400640 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c5403d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f42b4001bf0 tx=0x7f42b40020b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acbd69f8dc10c56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42a6a00640 1 -- 172.21.15.165:0/2308187443 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b4003d80 con 0x55d17c594590 2024-04-25T10:12:05.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42a6a00640 1 -- 172.21.15.165:0/2308187443 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4018070 con 0x55d17c594590 2024-04-25T10:12:05.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.044+0000 7f42a6a00640 1 -- 172.21.15.165:0/2308187443 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b400fa50 con 0x55d17c594590 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 -- 172.21.15.165:0/2308187443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 msgr2=0x55d17c5403d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c5403d0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f42b4001bf0 tx=0x7f42b40020b0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 -- 172.21.15.165:0/2308187443 shutdown_connections 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c5403d0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c587670 0x55d17c53fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2308187443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c569dd0 0x55d17c56db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 -- 172.21.15.165:0/2308187443 >> 172.21.15.165:0/2308187443 conn(0x55d17c53fab0 msgr2=0x55d17c57f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 -- 172.21.15.165:0/2308187443 shutdown_connections 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.045+0000 7f42c49c1740 1 -- 172.21.15.165:0/2308187443 wait complete. 2024-04-25T10:12:05.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.046+0000 7f42c49c1740 1 -- start start 2024-04-25T10:12:05.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.046+0000 7f42c49c1740 1 Processor -- start 2024-04-25T10:12:05.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c569dd0 0x55d17c5f2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c588530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42bc1ff520 con 0x55d17c594590 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d17c599660 con 0x55d17c587670 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d17c557cb0 con 0x55d17c569dd0 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42c49c1740 10 client.0 _ll_register_callbacks cb 0x55d17c53aa80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c588530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c588530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36644/0 (socket says 172.21.15.165:36644) 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.047+0000 7f42bb400640 1 -- 172.21.15.165:0/2669078113 learned_addr learned my addr 172.21.15.165:0/2669078113 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42bb400640 1 -- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c569dd0 msgr2=0x55d17c5f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42bb400640 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c569dd0 0x55d17c5f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42bb400640 1 -- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 msgr2=0x55d17c587ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42ba000640 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42bb400640 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42bb400640 1 -- 172.21.15.165:0/2669078113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d17c567700 con 0x55d17c594590 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42bb400640 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c588530 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f42b400f600 tx=0x7f42b4001330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e57dee4f475d0d5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f429be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:05.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f429be00640 1 -- 172.21.15.165:0/2669078113 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b40050d0 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f429be00640 1 -- 172.21.15.165:0/2669078113 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f42b4005270 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f42b4018070 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f429be00640 1 -- 172.21.15.165:0/2669078113 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42b402f050 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.048+0000 7f42c49c1740 10 client.7767 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55d17c583b30 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f429be00640 1 -- 172.21.15.165:0/2669078113 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b4018070 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f429be00640 1 -- 172.21.15.165:0/2669078113 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f42b4032020 con 0x55d17c594590 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 10 client.7767 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 10 client.7767 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 20 client.7767 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c57bcf0 0x55d17c59fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 4 client.7767 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x55d17c4aa7e0 con 0x55d17c57bcf0 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.049+0000 7f42baa00640 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c57bcf0 0x55d17c59fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.050+0000 7f42baa00640 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c57bcf0 0x55d17c59fc70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=262e8b6edace83e7 server_cookie=5d8ea406892c23f1 in_seq=0 out_seq=0 2024-04-25T10:12:05.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.050+0000 7f429be00640 10 client.7767 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.052+0000 7f429be00640 1 -- 172.21.15.165:0/2669078113 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55d17c4aa7e0 con 0x55d17c57bcf0 2024-04-25T10:12:05.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.052+0000 7f429be00640 10 client.7767 handle_command_reply: tid=0 2024-04-25T10:12:05.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.052+0000 7f42c49c1740 1 client.7767 shutdown 2024-04-25T10:12:05.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.052+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 msgr2=0x55d17c588530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.052+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c588530 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f42b400f600 tx=0x7f42b4001330 comp rx=0 tx=0).stop 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 shutdown_connections 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55d17c57bcf0 0x55d17c59fc70 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d17c594590 0x55d17c588530 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d17c587670 0x55d17c587ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 --2- 172.21.15.165:0/2669078113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d17c569dd0 0x55d17c5f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 >> 172.21.15.165:0/2669078113 conn(0x55d17c53fab0 msgr2=0x55d17c57f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f429be00640 0 client.7767 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 shutdown_connections 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 1 -- 172.21.15.165:0/2669078113 wait complete. 2024-04-25T10:12:05.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.053+0000 7f42c49c1740 20 client.7767 trim_cache size 0 max 16384 2024-04-25T10:12:05.060 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:12:05.060 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:12:05.060 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "a90140f3-f69b-46cf-b6b7-4814e49f8e56", 2024-04-25T10:12:05.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:12:05.060 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f429c06b4d0 msgr2=0x7f429c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f429c06b4d0 0x7f429c06d990 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f42ac034000 tx=0x7f42ac02c000 comp rx=0 tx=0).stop 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 msgr2=0x7f42bc0cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0cc740 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f42a800b440 tx=0x7f42a800b900 comp rx=0 tx=0).stop 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 shutdown_connections 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f429c06b4d0 0x7f429c06d990 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42bc161890 0x7f42bc0ccc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42bc161240 0x7f42bc0cc740 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 --2- 172.21.15.165:0/873936479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42bc0b54a0 0x7f42bc0c9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 >> 172.21.15.165:0/873936479 conn(0x7f42bc0d04a0 msgr2=0x7f42bc0ce7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 shutdown_connections 2024-04-25T10:12:05.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.054+0000 7f42c1c00640 1 -- 172.21.15.165:0/873936479 wait complete. 2024-04-25T10:12:05.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:05.083 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.6:0 scrub abort 2024-04-25T10:12:05.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.297+0000 7f2ab9c00640 1 Processor -- start 2024-04-25T10:12:05.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.297+0000 7f2ab9c00640 1 -- start start 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.298+0000 7f2ab9c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2ab40b95c0 0x7f2ab40b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.298+0000 7f2ab9c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2ab40b5910 0x7f2ab40d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.298+0000 7f2ab9c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab4161e70 con 0x7f2ab415dbd0 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.298+0000 7f2ab9c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab40d1ce0 con 0x7f2ab40b5910 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.298+0000 7f2ab9c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab4162150 con 0x7f2ab40b95c0 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.298+0000 7f2ab9200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2ab40b5910 0x7f2ab40d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2ab415dbd0 0x7f2ab4161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48712/0 (socket says 172.21.15.165:48712) 2024-04-25T10:12:05.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab3e00640 1 -- 172.21.15.165:0/1746424422 learned_addr learned my addr 172.21.15.165:0/1746424422 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:05.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab3400640 1 --2- 172.21.15.165:0/1746424422 >> v2:172.21.15.172:3300/0 conn(0x7f2ab40b95c0 0x7f2ab40b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab2a00640 1 -- 172.21.15.165:0/1746424422 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149922344 0 0) 0x7f2ab4161e70 con 0x7f2ab415dbd0 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab2a00640 1 -- 172.21.15.165:0/1746424422 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a98003520 con 0x7f2ab415dbd0 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab9200640 1 -- 172.21.15.165:0/1746424422 >> v2:172.21.15.172:3300/0 conn(0x7f2ab40b95c0 msgr2=0x7f2ab40b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab9200640 1 --2- 172.21.15.165:0/1746424422 >> v2:172.21.15.172:3300/0 conn(0x7f2ab40b95c0 0x7f2ab40b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab9200640 1 -- 172.21.15.165:0/1746424422 >> v1:172.21.15.165:6789/0 conn(0x7f2ab415dbd0 legacy=0x7f2ab4161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.299+0000 7f2ab9200640 1 -- 172.21.15.165:0/1746424422 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4162430 con 0x7f2ab40b5910 2024-04-25T10:12:05.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.300+0000 7f2ab9200640 1 --2- 172.21.15.165:0/1746424422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab40d2170 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4161580 tx=0x7f2aac02a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5dc470f0a0ca331 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.300+0000 7f2ab2a00640 1 -- 172.21.15.165:0/1746424422 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac007740 con 0x7f2ab40b5910 2024-04-25T10:12:05.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab2a00640 1 -- 172.21.15.165:0/1746424422 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac0078e0 con 0x7f2ab40b5910 2024-04-25T10:12:05.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab2a00640 1 -- 172.21.15.165:0/1746424422 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac007cb0 con 0x7f2ab40b5910 2024-04-25T10:12:05.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 -- 172.21.15.165:0/1746424422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 msgr2=0x7f2ab40d2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/1746424422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab40d2170 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f2ab4161580 tx=0x7f2aac02a680 comp rx=0 tx=0).stop 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 -- 172.21.15.165:0/1746424422 shutdown_connections 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/1746424422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab40d2170 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/1746424422 >> v2:172.21.15.172:3300/0 conn(0x7f2ab40b95c0 0x7f2ab40b53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 -- 172.21.15.165:0/1746424422 >> 172.21.15.165:0/1746424422 conn(0x7f2ab40d03d0 msgr2=0x7f2ab40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 -- 172.21.15.165:0/1746424422 shutdown_connections 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.301+0000 7f2ab9c00640 1 -- 172.21.15.165:0/1746424422 wait complete. 2024-04-25T10:12:05.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.303+0000 7f2ab9c00640 1 Processor -- start 2024-04-25T10:12:05.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.303+0000 7f2ab9c00640 1 -- start start 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab415b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ab40b95c0 0x7f2ab415b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab415dbd0 0x7f2ab41f6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40d2830 con 0x7f2ab40b5910 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40d26b0 con 0x7f2ab415dbd0 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40d29b0 con 0x7f2ab40b95c0 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab415b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab415b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36662/0 (socket says 172.21.15.165:36662) 2024-04-25T10:12:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.304+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3882153750 learned_addr learned my addr 172.21.15.165:0/3882153750 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab9200640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab415dbd0 0x7f2ab41f6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3400640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ab40b95c0 0x7f2ab415b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ab40b95c0 msgr2=0x7f2ab415b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ab40b95c0 0x7f2ab415b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab415dbd0 msgr2=0x7f2ab41f6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab415dbd0 0x7f2ab41f6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3882153750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab4159c60 con 0x7f2ab40b5910 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab9200640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab415dbd0 0x7f2ab41f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab415b480 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400c390 tx=0x7f2aa400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2aabe00640 1 -- 172.21.15.165:0/3882153750 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4017070 con 0x7f2ab40b5910 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2aabe00640 1 -- 172.21.15.165:0/3882153750 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4005ce0 con 0x7f2ab40b5910 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2aabe00640 1 -- 172.21.15.165:0/3882153750 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40072f0 con 0x7f2ab40b5910 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.305+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab415c030 con 0x7f2ab40b5910 2024-04-25T10:12:05.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.306+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab4159f40 con 0x7f2ab40b5910 2024-04-25T10:12:05.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.308+0000 7f2aabe00640 1 -- 172.21.15.165:0/3882153750 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2aa401a070 con 0x7f2ab40b5910 2024-04-25T10:12:05.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.309+0000 7f2aabe00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a9406b4d0 0x7f2a9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.310+0000 7f2aabe00640 1 -- 172.21.15.165:0/3882153750 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2aa4089820 con 0x7f2ab40b5910 2024-04-25T10:12:05.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.310+0000 7f2ab3400640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a9406b4d0 0x7f2a9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.310+0000 7f2abe02a740 1 Processor -- start 2024-04-25T10:12:05.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.310+0000 7f2abe02a740 1 -- start start 2024-04-25T10:12:05.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2ab3400640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a9406b4d0 0x7f2a9406d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40d3350 tx=0x7f2aa0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4863ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4860efb0 0x557c48636cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2abe02a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c48577870 con 0x557c4863f290 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c48520d60 con 0x557c4863e870 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.311+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c48573910 con 0x557c4860efb0 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860ea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36678/0 (socket says 172.21.15.165:36678) 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 -- 172.21.15.165:0/2376705711 learned_addr learned my addr 172.21.15.165:0/2376705711 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4863ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 -- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4860efb0 msgr2=0x557c48636cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4860efb0 0x557c48636cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 -- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 msgr2=0x557c4863ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4863ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 -- 172.21.15.165:0/2376705711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c483567d0 con 0x557c4863f290 2024-04-25T10:12:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4863ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.312+0000 7f2ab3400640 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860ea70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000e920 tx=0x7f2aa0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edcf4795cf92885 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2aa8c00640 1 -- 172.21.15.165:0/2376705711 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0025070 con 0x557c4863f290 2024-04-25T10:12:05.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2aa8c00640 1 -- 172.21.15.165:0/2376705711 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00157c0 con 0x557c4863f290 2024-04-25T10:12:05.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2aa8c00640 1 -- 172.21.15.165:0/2376705711 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000f040 con 0x557c4863f290 2024-04-25T10:12:05.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 -- 172.21.15.165:0/2376705711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 msgr2=0x557c4860ea70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860ea70 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000e920 tx=0x7f2aa0011000 comp rx=0 tx=0).stop 2024-04-25T10:12:05.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 -- 172.21.15.165:0/2376705711 shutdown_connections 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4860efb0 0x557c48636cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860ea70 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2376705711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4863ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.313+0000 7f2abe02a740 1 -- 172.21.15.165:0/2376705711 >> 172.21.15.165:0/2376705711 conn(0x557c48610ef0 msgr2=0x557c48659680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.314+0000 7f2abe02a740 1 -- 172.21.15.165:0/2376705711 shutdown_connections 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.314+0000 7f2abe02a740 1 -- 172.21.15.165:0/2376705711 wait complete. 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.314+0000 7f2abe02a740 1 -- start start 2024-04-25T10:12:05.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.314+0000 7f2abe02a740 1 Processor -- start 2024-04-25T10:12:05.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.315+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 0x557c48635de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.315+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4863f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.315+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.315+0000 7f2abe02a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4863afc0 con 0x557c4863f290 2024-04-25T10:12:05.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c485341c0 con 0x557c4863d260 2024-04-25T10:12:05.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c4863b260 con 0x557c4863e870 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2abe02a740 10 client.0 _ll_register_callbacks cb 0x557c485728f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4863f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4863f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36694/0 (socket says 172.21.15.165:36694) 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 -- 172.21.15.165:0/2649336129 learned_addr learned my addr 172.21.15.165:0/2649336129 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 -- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 msgr2=0x557c48635de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 0x557c48635de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 -- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863d260 msgr2=0x557c4863d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab9200640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab3400640 1 -- 172.21.15.165:0/2649336129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c4824c0f0 con 0x557c4863f290 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.316+0000 7f2ab9200640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2ab3400640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4863f670 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0018040 tx=0x7f2aa0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53e79413c1fd21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2a9d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2a9d600640 1 -- 172.21.15.165:0/2649336129 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0030060 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2a9d600640 1 -- 172.21.15.165:0/2649336129 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2aa002b430 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aa00157c0 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2a9d600640 1 -- 172.21.15.165:0/2649336129 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aa0036050 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2abe02a740 10 client.7782 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.317+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x557c483b6f60 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2a9d600640 1 -- 172.21.15.165:0/2649336129 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa00157c0 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2a9d600640 1 -- 172.21.15.165:0/2649336129 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2aa0039020 con 0x557c4863f290 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2abe02a740 10 client.7782 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2abe02a740 10 client.7782 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2abe02a740 20 client.7782 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486696c0 0x557c48669aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2abe02a740 4 client.7782 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:05.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c4854d690 con 0x557c486696c0 2024-04-25T10:12:05.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.318+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486696c0 0x557c48669aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.319+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486696c0 0x557c48669aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8954c4ba932ea5d2 server_cookie=a57cc8f230e8a2e4 in_seq=0 out_seq=0 2024-04-25T10:12:05.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.319+0000 7f2a9d600640 10 client.7782 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.319+0000 7f2a9d600640 1 -- 172.21.15.165:0/2649336129 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557c4854d690 con 0x557c486696c0 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.319+0000 7f2a9d600640 10 client.7782 handle_command_reply: tid=0 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.319+0000 7f2abe02a740 1 client.7782 shutdown 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 msgr2=0x557c4863f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4863f670 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f2aa0018040 tx=0x7f2aa0015000 comp rx=0 tx=0).stop 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 shutdown_connections 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486696c0 0x557c48669aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4863f670 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 --2- 172.21.15.165:0/2649336129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 0x557c48635de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2a9d600640 0 client.7782 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 >> 172.21.15.165:0/2649336129 conn(0x557c4860efb0 msgr2=0x557c48613fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 shutdown_connections 2024-04-25T10:12:05.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 1 -- 172.21.15.165:0/2649336129 wait complete. 2024-04-25T10:12:05.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.320+0000 7f2abe02a740 20 client.7782 trim_cache size 0 max 16384 2024-04-25T10:12:05.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.336+0000 7f2abe02a740 1 Processor -- start 2024-04-25T10:12:05.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.336+0000 7f2abe02a740 1 -- start start 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 0x557c48614300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863f290 0x557c4863c750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2abe02a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab42002e0 con 0x557c4863d260 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab41ff440 con 0x557c4863f290 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40cc680 con 0x557c4863e870 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36702/0 (socket says 172.21.15.165:36702) 2024-04-25T10:12:05.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3919756106 learned_addr learned my addr 172.21.15.165:0/3919756106 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3919756106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 msgr2=0x557c48614300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 0x557c48614300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3919756106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863f290 msgr2=0x557c4863c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863f290 0x557c4863c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.337+0000 7f2ab3e00640 1 -- 172.21.15.165:0/3919756106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c48638d10 con 0x557c4863d260 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.338+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 0x557c4863d6e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400cd50 tx=0x7f2aa4087a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fda6ed413dacefc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.338+0000 7f2a9fe00640 1 -- 172.21.15.165:0/3919756106 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa4086070 con 0x557c4863d260 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.338+0000 7f2a9fe00640 1 -- 172.21.15.165:0/3919756106 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4087bc0 con 0x557c4863d260 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.338+0000 7f2a9fe00640 1 -- 172.21.15.165:0/3919756106 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa405e040 con 0x557c4863d260 2024-04-25T10:12:05.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 -- 172.21.15.165:0/3919756106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 msgr2=0x557c4863d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 0x557c4863d6e0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400cd50 tx=0x7f2aa4087a10 comp rx=0 tx=0).stop 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 -- 172.21.15.165:0/3919756106 shutdown_connections 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863f290 0x557c4863c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863e870 0x557c48614300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 --2- 172.21.15.165:0/3919756106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863d260 0x557c4863d6e0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 -- 172.21.15.165:0/3919756106 >> 172.21.15.165:0/3919756106 conn(0x557c4860efb0 msgr2=0x557c48614e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 -- 172.21.15.165:0/3919756106 shutdown_connections 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 -- 172.21.15.165:0/3919756106 wait complete. 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.339+0000 7f2abe02a740 1 -- start start 2024-04-25T10:12:05.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.340+0000 7f2abe02a740 1 Processor -- start 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863d260 0x557c4865bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4860d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c4863ac20 con 0x557c4863f290 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40ccc80 con 0x557c4863e870 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557c48545450 con 0x557c4863d260 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2abe02a740 10 client.0 _ll_register_callbacks cb 0x7f2aa0009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:05.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860dea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36716/0 (socket says 172.21.15.165:36716) 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 -- 172.21.15.165:0/141731749 learned_addr learned my addr 172.21.15.165:0/141731749 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 -- 172.21.15.165:0/141731749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863d260 msgr2=0x557c4865bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863d260 0x557c4865bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 -- 172.21.15.165:0/141731749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 msgr2=0x557c4860d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4860d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.341+0000 7f2ab9200640 1 -- 172.21.15.165:0/141731749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c482251b0 con 0x557c4863f290 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2ab9200640 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860dea0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009e50 tx=0x7f2aac007b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ea3301c5d08edc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2a9d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2a9d600640 1 -- 172.21.15.165:0/141731749 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac009450 con 0x557c4863f290 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2a9d600640 1 -- 172.21.15.165:0/141731749 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2aac03b410 con 0x557c4863f290 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2aa4087bc0 con 0x557c4863f290 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2a9d600640 1 -- 172.21.15.165:0/141731749 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2aac003990 con 0x557c4863f290 2024-04-25T10:12:05.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2abe02a740 10 client.7788 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x557c4865c4d0 con 0x557c4863f290 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.342+0000 7f2a9d600640 1 -- 172.21.15.165:0/141731749 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa4087bc0 con 0x557c4863f290 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2a9d600640 1 -- 172.21.15.165:0/141731749 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2aac043070 con 0x557c4863f290 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2abe02a740 10 client.7788 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2abe02a740 10 client.7788 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2abe02a740 20 client.7788 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2abe02a740 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486c3300 0x557c4865fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2abe02a740 4 client.7788 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x557c48584db0 con 0x557c486c3300 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486c3300 0x557c4865fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2a9d600640 10 client.7788 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.343+0000 7f2ab3e00640 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486c3300 0x557c4865fd10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=275502525ec34d58 server_cookie=2b0af1eb0af7eba in_seq=0 out_seq=0 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2a9d600640 1 -- 172.21.15.165:0/141731749 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x557c48584db0 con 0x557c486c3300 2024-04-25T10:12:05.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2a9d600640 10 client.7788 handle_command_reply: tid=0 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 client.7788 shutdown 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 msgr2=0x557c4860dea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860dea0 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f2aac009e50 tx=0x7f2aac007b50 comp rx=0 tx=0).stop 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 shutdown_connections 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x557c486c3300 0x557c4865fd10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557c4863f290 0x557c4860dea0 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557c4863e870 0x557c4860d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 --2- 172.21.15.165:0/141731749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557c4863d260 0x557c4865bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 >> 172.21.15.165:0/141731749 conn(0x557c4860efb0 msgr2=0x557c48614d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2a9d600640 0 client.7788 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 shutdown_connections 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 1 -- 172.21.15.165:0/141731749 wait complete. 2024-04-25T10:12:05.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.344+0000 7f2abe02a740 20 client.7788 trim_cache size 0 max 16384 2024-04-25T10:12:05.353 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:12:05.353 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:12:05.353 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a9406b4d0 msgr2=0x7f2a9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a9406b4d0 0x7f2a9406d990 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40d3350 tx=0x7f2aa0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 msgr2=0x7f2ab415b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab415b480 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400c390 tx=0x7f2aa400c850 comp rx=0 tx=0).stop 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 shutdown_connections 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a9406b4d0 0x7f2a9406d990 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2ab415dbd0 0x7f2ab41f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2ab40b95c0 0x7f2ab415b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 --2- 172.21.15.165:0/3882153750 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2ab40b5910 0x7f2ab415b480 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 >> 172.21.15.165:0/3882153750 conn(0x7f2ab40d03d0 msgr2=0x7f2ab40b7370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 shutdown_connections 2024-04-25T10:12:05.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.345+0000 7f2ab9c00640 1 -- 172.21.15.165:0/3882153750 wait complete. 2024-04-25T10:12:05.368 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:12:05.368 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:05.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.557+0000 7fc552400640 1 Processor -- start 2024-04-25T10:12:05.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.557+0000 7fc552400640 1 -- start start 2024-04-25T10:12:05.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.558+0000 7fc552400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c15dd30 0x7fc54c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.558+0000 7fc552400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 0x7fc54c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc552400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc54c15e110 con 0x7fc54c0b5970 2024-04-25T10:12:05.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc552400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc54c06d000 con 0x7fc54c06bff0 2024-04-25T10:12:05.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc552400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0b2fd0 con 0x7fc54c15dd30 2024-04-25T10:12:05.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc54be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 0x7fc54c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc551000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c15dd30 0x7fc54c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc54be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 0x7fc54c0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:60488/0 (socket says 172.21.15.165:60488) 2024-04-25T10:12:05.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.559+0000 7fc54be00640 1 -- 172.21.15.165:0/2622653979 learned_addr learned my addr 172.21.15.165:0/2622653979 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.560+0000 7fc551000640 1 -- 172.21.15.165:0/2622653979 >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 msgr2=0x7fc54c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.560+0000 7fc551000640 1 --2- 172.21.15.165:0/2622653979 >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 0x7fc54c0b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.560+0000 7fc551000640 1 -- 172.21.15.165:0/2622653979 >> v1:172.21.15.172:6789/0 conn(0x7fc54c0b5970 legacy=0x7fc54c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.560+0000 7fc551000640 1 -- 172.21.15.165:0/2622653979 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c06c480 con 0x7fc54c15dd30 2024-04-25T10:12:05.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.560+0000 7fc54be00640 1 --2- 172.21.15.165:0/2622653979 >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 0x7fc54c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:05.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.561+0000 7fc551000640 1 --2- 172.21.15.165:0/2622653979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c15dd30 0x7fc54c0cebd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097a0 tx=0x7fc54002a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db659860685adeee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.562+0000 7fc54b400640 1 -- 172.21.15.165:0/2622653979 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540037070 con 0x7fc54c15dd30 2024-04-25T10:12:05.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.562+0000 7fc54b400640 1 -- 172.21.15.165:0/2622653979 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540032440 con 0x7fc54c15dd30 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.562+0000 7fc54b400640 1 -- 172.21.15.165:0/2622653979 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540032810 con 0x7fc54c15dd30 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 -- 172.21.15.165:0/2622653979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c15dd30 msgr2=0x7fc54c0cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 --2- 172.21.15.165:0/2622653979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c15dd30 0x7fc54c0cebd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc5400097a0 tx=0x7fc54002a180 comp rx=0 tx=0).stop 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 -- 172.21.15.165:0/2622653979 shutdown_connections 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 --2- 172.21.15.165:0/2622653979 >> v2:172.21.15.172:3300/0 conn(0x7fc54c06bff0 0x7fc54c0b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 --2- 172.21.15.165:0/2622653979 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c15dd30 0x7fc54c0cebd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 -- 172.21.15.165:0/2622653979 >> 172.21.15.165:0/2622653979 conn(0x7fc54c0d0610 msgr2=0x7fc54c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 -- 172.21.15.165:0/2622653979 shutdown_connections 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.563+0000 7fc552400640 1 -- 172.21.15.165:0/2622653979 wait complete. 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.564+0000 7fc552400640 1 Processor -- start 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.565+0000 7fc552400640 1 -- start start 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.565+0000 7fc552400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 0x7fc54c0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc552400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15dd30 0x7fc54c15a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc552400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c157b60 0x7fc54c1f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc552400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c06cd20 con 0x7fc54c0b5970 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc552400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c06ca40 con 0x7fc54c15dd30 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc552400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0cf110 con 0x7fc54c157b60 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 0x7fc54c0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 0x7fc54c0b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36724/0 (socket says 172.21.15.165:36724) 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 -- 172.21.15.165:0/1163001868 learned_addr learned my addr 172.21.15.165:0/1163001868 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 -- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c157b60 msgr2=0x7fc54c1f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c157b60 0x7fc54c1f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 -- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15dd30 msgr2=0x7fc54c15a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc54be00640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15dd30 0x7fc54c15a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15dd30 0x7fc54c15a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.566+0000 7fc551000640 1 -- 172.21.15.165:0/1163001868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc54c0b4610 con 0x7fc54c0b5970 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.567+0000 7fc551000640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 0x7fc54c0b3ce0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c00 tx=0x7fc5400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.567+0000 7fc548c00640 1 -- 172.21.15.165:0/1163001868 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540037040 con 0x7fc54c0b5970 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.567+0000 7fc548c00640 1 -- 172.21.15.165:0/1163001868 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540042070 con 0x7fc54c0b5970 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.567+0000 7fc548c00640 1 -- 172.21.15.165:0/1163001868 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540030920 con 0x7fc54c0b5970 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.567+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c1f3840 con 0x7fc54c0b5970 2024-04-25T10:12:05.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.567+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc54c0b4220 con 0x7fc54c0b5970 2024-04-25T10:12:05.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.570+0000 7fc548c00640 1 -- 172.21.15.165:0/1163001868 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc540030ac0 con 0x7fc54c0b5970 2024-04-25T10:12:05.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.571+0000 7fc548c00640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc52c06b4d0 0x7fc52c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.571+0000 7fc548c00640 1 -- 172.21.15.165:0/1163001868 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc5400aade0 con 0x7fc54c0b5970 2024-04-25T10:12:05.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.571+0000 7fc54be00640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc52c06b4d0 0x7fc52c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.572+0000 7fc54be00640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc52c06b4d0 0x7fc52c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0b5450 tx=0x7fc538008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.573+0000 7fc5567e7740 1 Processor -- start 2024-04-25T10:12:05.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.573+0000 7fc5567e7740 1 -- start start 2024-04-25T10:12:05.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf2130d70 0x55aaf2132ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf216d440 0x55aaf2131150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 0x55aaf215bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc5567e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf2094c70 con 0x55aaf215b830 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf203dcf0 con 0x55aaf2130d70 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaf2090d10 con 0x55aaf216d440 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 0x55aaf215bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 0x55aaf215bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36736/0 (socket says 172.21.15.165:36736) 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 -- 172.21.15.165:0/2215030105 learned_addr learned my addr 172.21.15.165:0/2215030105 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551000640 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf2130d70 0x55aaf2132ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc54be00640 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf216d440 0x55aaf2131150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 -- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf216d440 msgr2=0x55aaf2131150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf216d440 0x55aaf2131150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 -- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf2130d70 msgr2=0x55aaf2132ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf2130d70 0x55aaf2132ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.574+0000 7fc551a00640 1 -- 172.21.15.165:0/2215030105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf1e737d0 con 0x55aaf215b830 2024-04-25T10:12:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.575+0000 7fc54be00640 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf216d440 0x55aaf2131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.575+0000 7fc551a00640 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 0x55aaf215bcb0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fc544001cd0 tx=0x7fc544002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42afa23826c8d730 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.575+0000 7fc53d600640 1 -- 172.21.15.165:0/2215030105 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544017070 con 0x55aaf215b830 2024-04-25T10:12:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.575+0000 7fc53d600640 1 -- 172.21.15.165:0/2215030105 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400cbf0 con 0x55aaf215b830 2024-04-25T10:12:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.575+0000 7fc53d600640 1 -- 172.21.15.165:0/2215030105 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc544003970 con 0x55aaf215b830 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 -- 172.21.15.165:0/2215030105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 msgr2=0x55aaf215bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 0x55aaf215bcb0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fc544001cd0 tx=0x7fc544002190 comp rx=0 tx=0).stop 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 -- 172.21.15.165:0/2215030105 shutdown_connections 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf215b830 0x55aaf215bcb0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf216d440 0x55aaf2131150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 --2- 172.21.15.165:0/2215030105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf2130d70 0x55aaf2132ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 -- 172.21.15.165:0/2215030105 >> 172.21.15.165:0/2215030105 conn(0x55aaf20f8a30 msgr2=0x55aaf2176830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 -- 172.21.15.165:0/2215030105 shutdown_connections 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.576+0000 7fc5567e7740 1 -- 172.21.15.165:0/2215030105 wait complete. 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.577+0000 7fc5567e7740 1 -- start start 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.577+0000 7fc5567e7740 1 Processor -- start 2024-04-25T10:12:05.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf2186360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf215eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf20333a0 con 0x55aaf2130d70 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf2159380 con 0x55aaf216d440 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaf215db40 con 0x55aaf215b830 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc5567e7740 10 client.0 _ll_register_callbacks cb 0x55aaf2050b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf2186360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf2186360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36740/0 (socket says 172.21.15.165:36740) 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.578+0000 7fc551000640 1 -- 172.21.15.165:0/1918577610 learned_addr learned my addr 172.21.15.165:0/1918577610 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551a00640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf215eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551000640 1 -- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 msgr2=0x55aaf216f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc54be00640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551000640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551000640 1 -- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 msgr2=0x55aaf215eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551000640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf215eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551000640 1 -- 172.21.15.165:0/1918577610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaf1da93f0 con 0x55aaf2130d70 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551a00640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf215eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc551000640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf2186360 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fc54002a1c0 tx=0x7fc540032440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f31b563d439629 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc532000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc532000640 1 -- 172.21.15.165:0/1918577610 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400a6020 con 0x55aaf2130d70 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc532000640 1 -- 172.21.15.165:0/1918577610 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc540081540 con 0x55aaf2130d70 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc54400cbf0 con 0x55aaf2130d70 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc532000640 1 -- 172.21.15.165:0/1918577610 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc54007d410 con 0x55aaf2130d70 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.579+0000 7fc5567e7740 10 client.7797 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:05.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55aaf1d690f0 con 0x55aaf2130d70 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc532000640 1 -- 172.21.15.165:0/1918577610 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54400cbf0 con 0x55aaf2130d70 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc532000640 1 -- 172.21.15.165:0/1918577610 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc54003a070 con 0x55aaf2130d70 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 10 client.7797 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 10 client.7797 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 20 client.7797 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf215f1f0 0x55aaf2170950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 4 client.7797 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aaf20a21b0 con 0x55aaf215f1f0 2024-04-25T10:12:05.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.580+0000 7fc54be00640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf215f1f0 0x55aaf2170950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.581+0000 7fc54be00640 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf215f1f0 0x55aaf2170950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf9702ed4829c694 server_cookie=8e9ea4d5e83785b2 in_seq=0 out_seq=0 2024-04-25T10:12:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.581+0000 7fc532000640 10 client.7797 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.581+0000 7fc532000640 1 -- 172.21.15.165:0/1918577610 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55aaf20a21b0 con 0x55aaf215f1f0 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc532000640 10 client.7797 handle_command_reply: tid=0 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 client.7797 shutdown 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 msgr2=0x55aaf2186360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf2186360 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7fc54002a1c0 tx=0x7fc540032440 comp rx=0 tx=0).stop 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 shutdown_connections 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf215f1f0 0x55aaf2170950 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf215eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1918577610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf2186360 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc532000640 0 client.7797 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 >> 172.21.15.165:0/1918577610 conn(0x55aaf20f8a30 msgr2=0x55aaf2179d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 shutdown_connections 2024-04-25T10:12:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 1 -- 172.21.15.165:0/1918577610 wait complete. 2024-04-25T10:12:05.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.582+0000 7fc5567e7740 20 client.7797 trim_cache size 0 max 16384 2024-04-25T10:12:05.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 Processor -- start 2024-04-25T10:12:05.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 -- start start 2024-04-25T10:12:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21554f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf2179f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf2184f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c200460 con 0x55aaf2130d70 2024-04-25T10:12:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1ff620 con 0x55aaf216d440 2024-04-25T10:12:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.599+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c0ccd30 con 0x55aaf215b830 2024-04-25T10:12:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21554f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21554f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36748/0 (socket says 172.21.15.165:36748) 2024-04-25T10:12:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 -- 172.21.15.165:0/1774137110 learned_addr learned my addr 172.21.15.165:0/1774137110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 -- 172.21.15.165:0/1774137110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 msgr2=0x55aaf2179f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf2179f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 -- 172.21.15.165:0/1774137110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 msgr2=0x55aaf2184f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf2184f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 -- 172.21.15.165:0/1774137110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aaf2188d70 con 0x55aaf2130d70 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.600+0000 7fc551000640 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21554f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc540077620 tx=0x7fc540090000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3436d15524f7ba6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc53cc00640 1 -- 172.21.15.165:0/1774137110 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400a6070 con 0x55aaf2130d70 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc53cc00640 1 -- 172.21.15.165:0/1774137110 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540095070 con 0x55aaf2130d70 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc53cc00640 1 -- 172.21.15.165:0/1774137110 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5400787c0 con 0x55aaf2130d70 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 -- 172.21.15.165:0/1774137110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 msgr2=0x55aaf21554f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21554f0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fc540077620 tx=0x7fc540090000 comp rx=0 tx=0).stop 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 -- 172.21.15.165:0/1774137110 shutdown_connections 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf2184f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf2179f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1774137110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21554f0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.601+0000 7fc5567e7740 1 -- 172.21.15.165:0/1774137110 >> 172.21.15.165:0/1774137110 conn(0x55aaf20f8a30 msgr2=0x55aaf2170740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.602+0000 7fc5567e7740 1 -- 172.21.15.165:0/1774137110 shutdown_connections 2024-04-25T10:12:05.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.602+0000 7fc5567e7740 1 -- 172.21.15.165:0/1774137110 wait complete. 2024-04-25T10:12:05.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.602+0000 7fc5567e7740 1 -- start start 2024-04-25T10:12:05.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.602+0000 7fc5567e7740 1 Processor -- start 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf2170240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc54c1ff020 con 0x55aaf2130d70 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aaf2155fb0 con 0x55aaf216d440 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aaf21c6840 con 0x55aaf215b830 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.603+0000 7fc5567e7740 10 client.0 _ll_register_callbacks cb 0x55aaf212bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21c5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36764/0 (socket says 172.21.15.165:36764) 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 -- 172.21.15.165:0/1760024580 learned_addr learned my addr 172.21.15.165:0/1760024580 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 -- 172.21.15.165:0/1760024580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 msgr2=0x55aaf216e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 -- 172.21.15.165:0/1760024580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 msgr2=0x55aaf2170240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf2170240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 -- 172.21.15.165:0/1760024580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aaf2151c90 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc551000640 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21c5de0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc54002a770 tx=0x7fc54007c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0a9b33087d5ac0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.604+0000 7fc532000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc532000640 1 -- 172.21.15.165:0/1760024580 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc540085030 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc532000640 1 -- 172.21.15.165:0/1760024580 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fc5400787c0 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc540095070 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc532000640 1 -- 172.21.15.165:0/1760024580 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5400a6040 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 10 client.7803 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55aaf2151c90 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc532000640 1 -- 172.21.15.165:0/1760024580 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc540095070 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc532000640 1 -- 172.21.15.165:0/1760024580 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc54003a070 con 0x55aaf2130d70 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 10 client.7803 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 10 client.7803 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 20 client.7803 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf2170780 0x55aaf21e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 4 client.7803 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.605+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55aaf209c500 con 0x55aaf2170780 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.606+0000 7fc54be00640 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf2170780 0x55aaf21e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.606+0000 7fc54be00640 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf2170780 0x55aaf21e3560 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=953111f9bc6484d8 server_cookie=ce7a01619dfcaef7 in_seq=0 out_seq=0 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.606+0000 7fc532000640 10 client.7803 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.606+0000 7fc532000640 1 -- 172.21.15.165:0/1760024580 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x55aaf209c500 con 0x55aaf2170780 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.606+0000 7fc532000640 10 client.7803 handle_command_reply: tid=0 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.606+0000 7fc5567e7740 1 client.7803 shutdown 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 msgr2=0x55aaf21c5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21c5de0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7fc54002a770 tx=0x7fc54007c070 comp rx=0 tx=0).stop 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 shutdown_connections 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55aaf2170780 0x55aaf21e3560 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55aaf216d440 0x55aaf2170240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55aaf215b830 0x55aaf216e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 --2- 172.21.15.165:0/1760024580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55aaf2130d70 0x55aaf21c5de0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc532000640 0 client.7803 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 >> 172.21.15.165:0/1760024580 conn(0x55aaf20f8a30 msgr2=0x55aaf215e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 shutdown_connections 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 1 -- 172.21.15.165:0/1760024580 wait complete. 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.607+0000 7fc5567e7740 20 client.7803 trim_cache size 0 max 16384 2024-04-25T10:12:05.616 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: "a90140f3-f69b-46cf-b6b7-4814e49f8e56": { 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "a90140f3-f69b-46cf-b6b7-4814e49f8e56", 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive" 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:05.617 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc52c06b4d0 msgr2=0x7fc52c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc52c06b4d0 0x7fc52c06d990 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fc54c0b5450 tx=0x7fc538008040 comp rx=0 tx=0).stop 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 msgr2=0x7fc54c0b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 0x7fc54c0b3ce0 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fc540000c00 tx=0x7fc5400046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 shutdown_connections 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc52c06b4d0 0x7fc52c06d990 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc54c157b60 0x7fc54c1f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc54c15dd30 0x7fc54c15a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 --2- 172.21.15.165:0/1163001868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc54c0b5970 0x7fc54c0b3ce0 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:05.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 >> 172.21.15.165:0/1163001868 conn(0x7fc54c0d0610 msgr2=0x7fc54c0d2060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:05.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 shutdown_connections 2024-04-25T10:12:05.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:05.608+0000 7fc552400640 1 -- 172.21.15.165:0/1163001868 wait complete. 2024-04-25T10:12:10.629 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:12:10.629 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.6:0 scrub status 2024-04-25T10:12:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.838+0000 7f5a7c600640 1 Processor -- start 2024-04-25T10:12:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.838+0000 7f5a7c600640 1 -- start start 2024-04-25T10:12:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a7c600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a741446d0 0x7f5a740b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a7c600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a74144d50 0x7f5a740b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a7c600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a740cccc0 con 0x7f5a741452a0 2024-04-25T10:12:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a7c600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a740b2b20 con 0x7f5a741446d0 2024-04-25T10:12:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a7c600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7406d400 con 0x7f5a74144d50 2024-04-25T10:12:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a79c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a741446d0 0x7f5a740b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.840+0000 7f5a79200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a74144d50 0x7f5a740b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.841+0000 7f5a79c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a741446d0 0x7f5a740b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:33850/0 (socket says 172.21.15.165:33850) 2024-04-25T10:12:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.841+0000 7f5a79c00640 1 -- 172.21.15.165:0/1185010491 learned_addr learned my addr 172.21.15.165:0/1185010491 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.841+0000 7f5a79c00640 1 -- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a74144d50 msgr2=0x7f5a740b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.841+0000 7f5a79c00640 1 --2- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a74144d50 0x7f5a740b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.841+0000 7f5a79c00640 1 -- 172.21.15.165:0/1185010491 >> v1:172.21.15.172:6789/0 conn(0x7f5a741452a0 legacy=0x7f5a740cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.841+0000 7f5a79c00640 1 -- 172.21.15.165:0/1185010491 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a740cc140 con 0x7f5a741446d0 2024-04-25T10:12:10.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.843+0000 7f5a79c00640 1 --2- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b3f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009870 tx=0x7f5a6802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7490ad07ba5edc20 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.843+0000 7f5a73e00640 1 -- 172.21.15.165:0/1185010491 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68037070 con 0x7f5a741446d0 2024-04-25T10:12:10.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.843+0000 7f5a73e00640 1 -- 172.21.15.165:0/1185010491 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a68032440 con 0x7f5a741446d0 2024-04-25T10:12:10.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a73e00640 1 -- 172.21.15.165:0/1185010491 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68032810 con 0x7f5a741446d0 2024-04-25T10:12:10.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 -- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 msgr2=0x7f5a740b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 --2- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b3f20 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f5a68009870 tx=0x7f5a6802a270 comp rx=0 tx=0).stop 2024-04-25T10:12:10.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 -- 172.21.15.165:0/1185010491 shutdown_connections 2024-04-25T10:12:10.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 --2- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a74144d50 0x7f5a740b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 --2- 172.21.15.165:0/1185010491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b3f20 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 -- 172.21.15.165:0/1185010491 >> 172.21.15.165:0/1185010491 conn(0x7f5a7406abe0 msgr2=0x7f5a7406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 -- 172.21.15.165:0/1185010491 shutdown_connections 2024-04-25T10:12:10.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.844+0000 7f5a7c600640 1 -- 172.21.15.165:0/1185010491 wait complete. 2024-04-25T10:12:10.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.846+0000 7f5a7c600640 1 Processor -- start 2024-04-25T10:12:10.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.846+0000 7f5a7c600640 1 -- start start 2024-04-25T10:12:10.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a7c600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a7c600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 0x7f5a741f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a7c600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a741452a0 0x7f5a741f3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a7c600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740cc700 con 0x7f5a74144d50 2024-04-25T10:12:10.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a7c600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740b49a0 con 0x7f5a741446d0 2024-04-25T10:12:10.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a7c600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a740cc9e0 con 0x7f5a741452a0 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a79200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 0x7f5a741f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a79200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 0x7f5a741f3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33170/0 (socket says 172.21.15.165:33170) 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.847+0000 7f5a79200640 1 -- 172.21.15.165:0/693460256 learned_addr learned my addr 172.21.15.165:0/693460256 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79c00640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a7a600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a741452a0 0x7f5a741f3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79200640 1 -- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a741452a0 msgr2=0x7f5a741f3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79200640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a741452a0 0x7f5a741f3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79200640 1 -- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 msgr2=0x7f5a740b2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79200640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79200640 1 -- 172.21.15.165:0/693460256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a74143180 con 0x7f5a74144d50 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a7a600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a741452a0 0x7f5a741f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:10.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a79200640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 0x7f5a741f3720 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e9b0 tx=0x7f5a6000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.848+0000 7f5a71600640 1 -- 172.21.15.165:0/693460256 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60017070 con 0x7f5a74144d50 2024-04-25T10:12:10.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.849+0000 7f5a71600640 1 -- 172.21.15.165:0/693460256 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6000c9a0 con 0x7f5a74144d50 2024-04-25T10:12:10.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.849+0000 7f5a71600640 1 -- 172.21.15.165:0/693460256 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000cd90 con 0x7f5a74144d50 2024-04-25T10:12:10.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.849+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a74142600 con 0x7f5a74144d50 2024-04-25T10:12:10.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.849+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a74142bc0 con 0x7f5a74144d50 2024-04-25T10:12:10.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.851+0000 7f5a71600640 1 -- 172.21.15.165:0/693460256 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a6001a070 con 0x7f5a74144d50 2024-04-25T10:12:10.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.852+0000 7f5a71600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a5406b4d0 0x7f5a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.852+0000 7f5a71600640 1 -- 172.21.15.165:0/693460256 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5a6008a400 con 0x7f5a74144d50 2024-04-25T10:12:10.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.852+0000 7f5a79c00640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a5406b4d0 0x7f5a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.852+0000 7f5a7f567740 1 Processor -- start 2024-04-25T10:12:10.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.853+0000 7f5a7f567740 1 -- start start 2024-04-25T10:12:10.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.853+0000 7f5a79c00640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a5406b4d0 0x7f5a5406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5a68034000 tx=0x7f5a6802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c32cbb0 0x55a73c32ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c32cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c357af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7f567740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73c290c70 con 0x55a73c357670 2024-04-25T10:12:10.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73c239cf0 con 0x55a73c369280 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a73c28cd10 con 0x55a73c32cbb0 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c357af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c357af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33186/0 (socket says 172.21.15.165:33186) 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 -- 172.21.15.165:0/763840890 learned_addr learned my addr 172.21.15.165:0/763840890 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a79200640 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c32cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 -- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c32cbb0 msgr2=0x55a73c32ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c32cbb0 0x55a73c32ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 -- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 msgr2=0x55a73c32cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c32cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a7a600640 1 -- 172.21.15.165:0/763840890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a73c06f7d0 con 0x55a73c357670 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.854+0000 7f5a79200640 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c32cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.855+0000 7f5a7a600640 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c357af0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c001da0 tx=0x7f5a6c00d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6eb4a28872c1af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.855+0000 7f5a66000640 1 -- 172.21.15.165:0/763840890 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c018070 con 0x55a73c357670 2024-04-25T10:12:10.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.855+0000 7f5a66000640 1 -- 172.21.15.165:0/763840890 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c003da0 con 0x55a73c357670 2024-04-25T10:12:10.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.855+0000 7f5a66000640 1 -- 172.21.15.165:0/763840890 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00db20 con 0x55a73c357670 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 -- 172.21.15.165:0/763840890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 msgr2=0x55a73c357af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c357af0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c001da0 tx=0x7f5a6c00d150 comp rx=0 tx=0).stop 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 -- 172.21.15.165:0/763840890 shutdown_connections 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c357af0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c32cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 --2- 172.21.15.165:0/763840890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c32cbb0 0x55a73c32ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 -- 172.21.15.165:0/763840890 >> 172.21.15.165:0/763840890 conn(0x55a73c2f4a30 msgr2=0x55a73c372670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 -- 172.21.15.165:0/763840890 shutdown_connections 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 -- 172.21.15.165:0/763840890 wait complete. 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.856+0000 7f5a7f567740 1 -- start start 2024-04-25T10:12:10.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.857+0000 7f5a7f567740 1 Processor -- start 2024-04-25T10:12:10.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c345f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c357670 0x55a73c359a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 0x55a73c36c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73c22f3a0 con 0x55a73c32cbb0 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73c3551c0 con 0x55a73c357670 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a73c3464a0 con 0x55a73c369280 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a7f567740 10 client.0 _ll_register_callbacks cb 0x55a73c24cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c345f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c345f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33192/0 (socket says 172.21.15.165:33192) 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79c00640 1 -- 172.21.15.165:0/2182196269 learned_addr learned my addr 172.21.15.165:0/2182196269 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79c00640 1 -- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 msgr2=0x55a73c36c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79c00640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 0x55a73c36c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79200640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c357670 0x55a73c359a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.858+0000 7f5a79c00640 1 -- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c357670 msgr2=0x55a73c359a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a79c00640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c357670 0x55a73c359a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a79c00640 1 -- 172.21.15.165:0/2182196269 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a73bfa53f0 con 0x55a73c32cbb0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a79200640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c357670 0x55a73c359a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a79c00640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c345f60 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5a68035040 tx=0x7f5a680046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12270512b6e0431e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2182196269 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6804e060 con 0x55a73c32cbb0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2182196269 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5a6803e520 con 0x55a73c32cbb0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a6c003da0 con 0x55a73c32cbb0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2182196269 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68054050 con 0x55a73c32cbb0 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a7f567740 10 client.7812 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:10.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.859+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55a73bf650f0 con 0x55a73c32cbb0 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2182196269 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6c003da0 con 0x55a73c32cbb0 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2182196269 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a68057020 con 0x55a73c32cbb0 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a7f567740 10 client.7812 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a7f567740 10 client.7812 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a7f567740 20 client.7812 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c359f90 0x55a73c385d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a7f567740 4 client.7812 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a73c29e1b0 con 0x55a73c359f90 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.860+0000 7f5a79200640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c359f90 0x55a73c385d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.861+0000 7f5a79200640 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c359f90 0x55a73c385d00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f3e305afae7a97 server_cookie=fd947efd49ec4406 in_seq=0 out_seq=0 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.861+0000 7f5a5aa00640 10 client.7812 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.861+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2182196269 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a73c29e1b0 con 0x55a73c359f90 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.861+0000 7f5a5aa00640 10 client.7812 handle_command_reply: tid=0 2024-04-25T10:12:10.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.861+0000 7f5a7f567740 1 client.7812 shutdown 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 msgr2=0x55a73c345f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c345f60 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f5a68035040 tx=0x7f5a680046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 shutdown_connections 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c359f90 0x55a73c385d00 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 0x55a73c36c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c357670 0x55a73c359a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2182196269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c345f60 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 >> 172.21.15.165:0/2182196269 conn(0x55a73c2f4a30 msgr2=0x55a73c354990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a5aa00640 0 client.7812 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 shutdown_connections 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 1 -- 172.21.15.165:0/2182196269 wait complete. 2024-04-25T10:12:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.862+0000 7f5a7f567740 20 client.7812 trim_cache size 0 max 16384 2024-04-25T10:12:10.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.878+0000 7f5a7f567740 1 Processor -- start 2024-04-25T10:12:10.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.878+0000 7f5a7f567740 1 -- start start 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c36d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c357670 0x55a73c354b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c36a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a7f567740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a742006c0 con 0x55a73c32cbb0 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741ff840 con 0x55a73c369280 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741fec70 con 0x55a73c357670 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c36d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a79c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c36d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33198/0 (socket says 172.21.15.165:33198) 2024-04-25T10:12:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.879+0000 7f5a79c00640 1 -- 172.21.15.165:0/274816022 learned_addr learned my addr 172.21.15.165:0/274816022 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:10.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a79c00640 1 -- 172.21.15.165:0/274816022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c357670 msgr2=0x55a73c354b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a79c00640 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c357670 0x55a73c354b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a79c00640 1 -- 172.21.15.165:0/274816022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 msgr2=0x55a73c36a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a79c00640 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c36a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a79c00640 1 -- 172.21.15.165:0/274816022 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a73c35a630 con 0x55a73c32cbb0 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a79c00640 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c36d7d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5a6803e970 tx=0x7f5a68043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df39a6c8070644ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a65600640 1 -- 172.21.15.165:0/274816022 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a68037040 con 0x55a73c32cbb0 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a65600640 1 -- 172.21.15.165:0/274816022 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6803ea00 con 0x55a73c32cbb0 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.880+0000 7f5a65600640 1 -- 172.21.15.165:0/274816022 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6803ecf0 con 0x55a73c32cbb0 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 -- 172.21.15.165:0/274816022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 msgr2=0x55a73c36d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c36d7d0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f5a6803e970 tx=0x7f5a68043000 comp rx=0 tx=0).stop 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 -- 172.21.15.165:0/274816022 shutdown_connections 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c369280 0x55a73c36a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c357670 0x55a73c354b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 --2- 172.21.15.165:0/274816022 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c32cbb0 0x55a73c36d7d0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 -- 172.21.15.165:0/274816022 >> 172.21.15.165:0/274816022 conn(0x55a73c2f4a30 msgr2=0x55a73c326a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 -- 172.21.15.165:0/274816022 shutdown_connections 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.881+0000 7f5a7f567740 1 -- 172.21.15.165:0/274816022 wait complete. 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.882+0000 7f5a7f567740 1 -- start start 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.882+0000 7f5a7f567740 1 Processor -- start 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c32cbb0 0x55a73c3dc5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c32c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 0x55a73c384b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a73c328ec0 con 0x55a73c357670 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a741ff240 con 0x55a73c32cbb0 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a73c386290 con 0x55a73c369280 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a7f567740 10 client.0 _ll_register_callbacks cb 0x7f5a68035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a79200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c32c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a79200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c32c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33214/0 (socket says 172.21.15.165:33214) 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.883+0000 7f5a79200640 1 -- 172.21.15.165:0/2725570851 learned_addr learned my addr 172.21.15.165:0/2725570851 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:10.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79c00640 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c32cbb0 0x55a73c3dc5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79200640 1 -- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 msgr2=0x55a73c384b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79200640 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 0x55a73c384b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79200640 1 -- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c32cbb0 msgr2=0x55a73c3dc5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79200640 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c32cbb0 0x55a73c3dc5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79200640 1 -- 172.21.15.165:0/2725570851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a73c371f10 con 0x55a73c357670 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a79200640 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c32c280 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5a60054dd0 tx=0x7f5a60009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=491bd91162220526 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a5aa00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2725570851 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a60070070 con 0x55a73c357670 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2725570851 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f5a60056c70 con 0x55a73c357670 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a6803ea00 con 0x55a73c357670 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2725570851 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a60021040 con 0x55a73c357670 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a7f567740 10 client.7818 fetch_fsmap learned FSMap version 65 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.884+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=2+}) v3 -- 0x55a73c36f4b0 con 0x55a73c357670 2024-04-25T10:12:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2725570851 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6803ea00 con 0x55a73c357670 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2725570851 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 65) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f5a60024060 con 0x55a73c357670 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a7f567740 10 client.7818 fetch_fsmap finished waiting for FSMap version 65 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a7f567740 10 client.7818 resolve_mds: resolved 6:0 to role '6:0' aka daemon mds.a 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a7f567740 20 client.7818 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c3850b0 0x55a73c368000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a7f567740 4 client.7818 mds_command: new command op to 7569 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 --> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a73c298500 con 0x55a73c3850b0 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a79c00640 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c3850b0 0x55a73c368000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a79c00640 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c3850b0 0x55a73c368000 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a982852449dea378 server_cookie=a2f5ed84f3a42a25 in_seq=0 out_seq=0 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.885+0000 7f5a5aa00640 10 client.7818 ms_handle_connect on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a5aa00640 1 -- 172.21.15.165:0/2725570851 <== mds.0 v2:172.21.15.165:6834/2500333822 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a73c298500 con 0x55a73c3850b0 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a5aa00640 10 client.7818 handle_command_reply: tid=0 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 client.7818 shutdown 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 msgr2=0x55a73c32c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c32c280 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f5a60054dd0 tx=0x7f5a60009670 comp rx=0 tx=0).stop 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 shutdown_connections 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:6834/2500333822,v1:172.21.15.165:6835/2500333822] conn(0x55a73c3850b0 0x55a73c368000 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a73c369280 0x55a73c384b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a73c357670 0x55a73c32c280 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 --2- 172.21.15.165:0/2725570851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a73c32cbb0 0x55a73c3dc5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a5aa00640 0 client.7818 ms_handle_reset on v2:172.21.15.165:6834/2500333822 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 >> 172.21.15.165:0/2725570851 conn(0x55a73c2f4a30 msgr2=0x55a73c344f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 shutdown_connections 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 1 -- 172.21.15.165:0/2725570851 wait complete. 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.886+0000 7f5a7f567740 20 client.7818 trim_cache size 0 max 16384 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:12:10.899 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:12:10.900 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:12:10.900 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:12:10.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a5406b4d0 msgr2=0x7f5a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a5406b4d0 0x7f5a5406d990 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f5a68034000 tx=0x7f5a6802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:10.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 msgr2=0x7f5a741f3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 0x7f5a741f3720 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000e9b0 tx=0x7f5a6000ee70 comp rx=0 tx=0).stop 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 shutdown_connections 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a5406b4d0 0x7f5a5406d990 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a741452a0 0x7f5a741f3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a74144d50 0x7f5a741f3720 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 --2- 172.21.15.165:0/693460256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a741446d0 0x7f5a740b2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 >> 172.21.15.165:0/693460256 conn(0x7f5a7406abe0 msgr2=0x7f5a740cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 shutdown_connections 2024-04-25T10:12:10.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:10.887+0000 7f5a7c600640 1 -- 172.21.15.165:0/693460256 wait complete. 2024-04-25T10:12:10.909 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:12:10.909 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:11.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.099+0000 7fce31400640 1 Processor -- start 2024-04-25T10:12:11.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.099+0000 7fce31400640 1 -- start start 2024-04-25T10:12:11.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce31400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce2c144440 0x7fce2c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce31400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fce2c071530 0x7fce2c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce31400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c0cd860 0x7fce2c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce31400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c0719a0 con 0x7fce2c144440 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce31400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce2c15c860 con 0x7fce2c071530 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce31400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c15bf30 con 0x7fce2c0cd860 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce2be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce2c144440 0x7fce2c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce2be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce2c144440 0x7fce2c070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33218/0 (socket says 172.21.15.165:33218) 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.101+0000 7fce2be00640 1 -- 172.21.15.165:0/167141610 learned_addr learned my addr 172.21.15.165:0/167141610 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce30a00640 1 --2- 172.21.15.165:0/167141610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c0cd860 0x7fce2c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2b400640 1 --2- 172.21.15.165:0/167141610 >> v2:172.21.15.172:3300/0 conn(0x7fce2c071530 0x7fce2c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2be00640 1 -- 172.21.15.165:0/167141610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c0cd860 msgr2=0x7fce2c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2be00640 1 --2- 172.21.15.165:0/167141610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c0cd860 0x7fce2c06d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2be00640 1 -- 172.21.15.165:0/167141610 >> v2:172.21.15.172:3300/0 conn(0x7fce2c071530 msgr2=0x7fce2c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2be00640 1 --2- 172.21.15.165:0/167141610 >> v2:172.21.15.172:3300/0 conn(0x7fce2c071530 0x7fce2c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2be00640 1 -- 172.21.15.165:0/167141610 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c15c210 con 0x7fce2c144440 2024-04-25T10:12:11.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.102+0000 7fce2b400640 1 --2- 172.21.15.165:0/167141610 >> v2:172.21.15.172:3300/0 conn(0x7fce2c071530 0x7fce2c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.103+0000 7fce2be00640 1 --2- 172.21.15.165:0/167141610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c144440 0x7fce2c070ff0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06b970 tx=0x7fce1c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f62503b01b48dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.103+0000 7fce2aa00640 1 -- 172.21.15.165:0/167141610 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c037070 con 0x7fce2c144440 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.103+0000 7fce2aa00640 1 -- 172.21.15.165:0/167141610 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c032440 con 0x7fce2c144440 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.103+0000 7fce2aa00640 1 -- 172.21.15.165:0/167141610 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c032810 con 0x7fce2c144440 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 -- 172.21.15.165:0/167141610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c144440 msgr2=0x7fce2c070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 --2- 172.21.15.165:0/167141610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c144440 0x7fce2c070ff0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fce2c06b970 tx=0x7fce1c02a270 comp rx=0 tx=0).stop 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 -- 172.21.15.165:0/167141610 shutdown_connections 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 --2- 172.21.15.165:0/167141610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c0cd860 0x7fce2c06d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 --2- 172.21.15.165:0/167141610 >> v2:172.21.15.172:3300/0 conn(0x7fce2c071530 0x7fce2c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 --2- 172.21.15.165:0/167141610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c144440 0x7fce2c070ff0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 -- 172.21.15.165:0/167141610 >> 172.21.15.165:0/167141610 conn(0x7fce2c06abe0 msgr2=0x7fce2c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 -- 172.21.15.165:0/167141610 shutdown_connections 2024-04-25T10:12:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.104+0000 7fce31400640 1 -- 172.21.15.165:0/167141610 wait complete. 2024-04-25T10:12:11.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.106+0000 7fce31400640 1 Processor -- start 2024-04-25T10:12:11.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.106+0000 7fce31400640 1 -- start start 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce31400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce2c071530 0x7fce2c0aa9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce31400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 0x7fce2c0a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce31400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c144440 0x7fce2c0a8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce31400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0cdcd0 con 0x7fce2c0cd860 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce31400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c15bc30 con 0x7fce2c071530 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce31400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c0cde50 con 0x7fce2c144440 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 0x7fce2c0a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 0x7fce2c0a8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33222/0 (socket says 172.21.15.165:33222) 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2b400640 1 -- 172.21.15.165:0/1165256108 learned_addr learned my addr 172.21.15.165:0/1165256108 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2be00640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce2c071530 0x7fce2c0aa9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2b400640 1 -- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c144440 msgr2=0x7fce2c0a8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2b400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c144440 0x7fce2c0a8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.107+0000 7fce2b400640 1 -- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce2c071530 msgr2=0x7fce2c0aa9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce2b400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce2c071530 0x7fce2c0aa9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce2b400640 1 -- 172.21.15.165:0/1165256108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c0a9080 con 0x7fce2c0cd860 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce2b400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 0x7fce2c0a8510 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c7c0 tx=0x7fce1800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18015070 con 0x7fce2c0cd860 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce18007a30 con 0x7fce2c0cd860 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce18007e00 con 0x7fce2c0cd860 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c0a9360 con 0x7fce2c0cd860 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.108+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c0a9920 con 0x7fce2c0cd860 2024-04-25T10:12:11.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.110+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c06e8f0 con 0x7fce2c0cd860 2024-04-25T10:12:11.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.111+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce18018070 con 0x7fce2c0cd860 2024-04-25T10:12:11.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.112+0000 7fce23e00640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce0c06b4d0 0x7fce0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.112+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fce1808a380 con 0x7fce2c0cd860 2024-04-25T10:12:11.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.112+0000 7fce2be00640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce0c06b4d0 0x7fce0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.113+0000 7fce2be00640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce0c06b4d0 0x7fce0c06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce1c034000 tx=0x7fce1c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.117+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce18058ff0 con 0x7fce2c0cd860 2024-04-25T10:12:11.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.346+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce2c073990 con 0x7fce2c0cd860 2024-04-25T10:12:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.347+0000 7fce23e00640 1 -- 172.21.15.165:0/1165256108 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fce180588f0 con 0x7fce2c0cd860 2024-04-25T10:12:11.353 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:11.353 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:11.353 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:11.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.349+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce0c06b4d0 msgr2=0x7fce0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.349+0000 7fce31400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce0c06b4d0 0x7fce0c06d990 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fce1c034000 tx=0x7fce1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.349+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 msgr2=0x7fce2c0a8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.349+0000 7fce31400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 0x7fce2c0a8510 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fce1800c7c0 tx=0x7fce1800cc80 comp rx=0 tx=0).stop 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 shutdown_connections 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce0c06b4d0 0x7fce0c06d990 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce2c144440 0x7fce2c0a8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce2c0cd860 0x7fce2c0a8510 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 --2- 172.21.15.165:0/1165256108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce2c071530 0x7fce2c0aa9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 >> 172.21.15.165:0/1165256108 conn(0x7fce2c06abe0 msgr2=0x7fce2c0c5cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 shutdown_connections 2024-04-25T10:12:11.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.350+0000 7fce31400640 1 -- 172.21.15.165:0/1165256108 wait complete. 2024-04-25T10:12:11.373 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:11.373 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:11.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.565+0000 7f696ca00640 1 Processor -- start 2024-04-25T10:12:11.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.565+0000 7f696ca00640 1 -- start start 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.566+0000 7f696ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f69680cfec0 0x7f69680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.566+0000 7f696ca00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6968071480 con 0x7f6968154ee0 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.566+0000 7f696ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6968071760 con 0x7f69680ce3c0 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.566+0000 7f696ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69680ce830 con 0x7f69680cfec0 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f6965e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6968154ee0 0x7f69680cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34790/0 (socket says 172.21.15.165:34790) 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f6965e00640 1 -- 172.21.15.165:0/1411590933 learned_addr learned my addr 172.21.15.165:0/1411590933 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f695d400640 1 --2- 172.21.15.165:0/1411590933 >> v2:172.21.15.165:3300/0 conn(0x7f69680cfec0 0x7f69680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f6965400640 1 -- 172.21.15.165:0/1411590933 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1055458504 0 0) 0x7f6968071480 con 0x7f6968154ee0 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f6965400640 1 -- 172.21.15.165:0/1411590933 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f694c003520 con 0x7f6968154ee0 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f695d400640 1 -- 172.21.15.165:0/1411590933 >> v1:172.21.15.172:6789/0 conn(0x7f69680ce3c0 legacy=0x7f696806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:11.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f695d400640 1 -- 172.21.15.165:0/1411590933 >> v1:172.21.15.165:6789/0 conn(0x7f6968154ee0 legacy=0x7f69680cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.567+0000 7f695d400640 1 -- 172.21.15.165:0/1411590933 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6968071a40 con 0x7f69680cfec0 2024-04-25T10:12:11.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.568+0000 7f695d400640 1 --2- 172.21.15.165:0/1411590933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69680cfec0 0x7f69680cde80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f69500097a0 tx=0x7f695002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7d36a3432790ef6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f6965400640 1 -- 172.21.15.165:0/1411590933 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6950037070 con 0x7f69680cfec0 2024-04-25T10:12:11.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f6965400640 1 -- 172.21.15.165:0/1411590933 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6950032440 con 0x7f69680cfec0 2024-04-25T10:12:11.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f6965400640 1 -- 172.21.15.165:0/1411590933 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69500327f0 con 0x7f69680cfec0 2024-04-25T10:12:11.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 -- 172.21.15.165:0/1411590933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69680cfec0 msgr2=0x7f69680cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 --2- 172.21.15.165:0/1411590933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69680cfec0 0x7f69680cde80 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f69500097a0 tx=0x7f695002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 -- 172.21.15.165:0/1411590933 shutdown_connections 2024-04-25T10:12:11.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 --2- 172.21.15.165:0/1411590933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69680cfec0 0x7f69680cde80 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 -- 172.21.15.165:0/1411590933 >> 172.21.15.165:0/1411590933 conn(0x7f696806abe0 msgr2=0x7f696806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 -- 172.21.15.165:0/1411590933 shutdown_connections 2024-04-25T10:12:11.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.569+0000 7f696ca00640 1 -- 172.21.15.165:0/1411590933 wait complete. 2024-04-25T10:12:11.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.571+0000 7f696ca00640 1 Processor -- start 2024-04-25T10:12:11.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.571+0000 7f696ca00640 1 -- start start 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f696ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69680ce3c0 0x7f696814f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f696ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69680cfec0 0x7f696814dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f696ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 0x7f696814e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f696ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6968071230 con 0x7f6968154ee0 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f696ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69681552c0 con 0x7f69680ce3c0 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f696ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6968155440 con 0x7f69680cfec0 2024-04-25T10:12:11.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f6966800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 0x7f696814e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f6966800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 0x7f696814e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33238/0 (socket says 172.21.15.165:33238) 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.572+0000 7f6966800640 1 -- 172.21.15.165:0/2488027425 learned_addr learned my addr 172.21.15.165:0/2488027425 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f695d400640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69680cfec0 0x7f696814dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6965e00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69680ce3c0 0x7f696814f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6966800640 1 -- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69680cfec0 msgr2=0x7f696814dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6966800640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69680cfec0 0x7f696814dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6966800640 1 -- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69680ce3c0 msgr2=0x7f696814f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6966800640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69680ce3c0 0x7f696814f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6966800640 1 -- 172.21.15.165:0/2488027425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6968168e00 con 0x7f6968154ee0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f6966800640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 0x7f696814e1c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f696000c780 tx=0x7f696000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.573+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6960017070 con 0x7f6968154ee0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.574+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6960003da0 con 0x7f6968154ee0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.574+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69600052b0 con 0x7f6968154ee0 2024-04-25T10:12:11.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.574+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69681690e0 con 0x7f6968154ee0 2024-04-25T10:12:11.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.574+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69681696a0 con 0x7f6968154ee0 2024-04-25T10:12:11.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.575+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6968073990 con 0x7f6968154ee0 2024-04-25T10:12:11.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.576+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f696001a070 con 0x7f6968154ee0 2024-04-25T10:12:11.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.577+0000 7f695ea00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f694406b4d0 0x7f694406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:11.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.577+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6960089a90 con 0x7f6968154ee0 2024-04-25T10:12:11.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.577+0000 7f6965e00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f694406b4d0 0x7f694406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:11.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.578+0000 7f6965e00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f694406b4d0 0x7f694406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f696814d3d0 tx=0x7f6958009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:11.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.582+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6960058700 con 0x7f6968154ee0 2024-04-25T10:12:11.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.850+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f6968168be0 con 0x7f6968154ee0 2024-04-25T10:12:11.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.851+0000 7f695ea00640 1 -- 172.21.15.165:0/2488027425 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7f6960058000 con 0x7f6968154ee0 2024-04-25T10:12:11.857 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:11.857 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":632,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076442369,"bytes_used":2583093248,"bytes_avail":770511020032,"bytes_total":773094113280,"write_bytes_sec":23914133,"read_op_per_sec":0,"write_op_per_sec":24},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7569}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:12:10.336282+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:11.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.854+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f694406b4d0 msgr2=0x7f694406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.854+0000 7f696ca00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f694406b4d0 0x7f694406d990 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f696814d3d0 tx=0x7f6958009040 comp rx=0 tx=0).stop 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.854+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 msgr2=0x7f696814e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.854+0000 7f696ca00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 0x7f696814e1c0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7f696000c780 tx=0x7f696000cc40 comp rx=0 tx=0).stop 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 shutdown_connections 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f694406b4d0 0x7f694406d990 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6968154ee0 0x7f696814e1c0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69680cfec0 0x7f696814dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 --2- 172.21.15.165:0/2488027425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69680ce3c0 0x7f696814f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 >> 172.21.15.165:0/2488027425 conn(0x7f696806abe0 msgr2=0x7f69680b1440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 shutdown_connections 2024-04-25T10:12:11.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:11.855+0000 7f696ca00640 1 -- 172.21.15.165:0/2488027425 wait complete. 2024-04-25T10:12:11.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:12:12.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.090+0000 7fbe9c400640 1 Processor -- start 2024-04-25T10:12:13.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.090+0000 7fbe9c400640 1 -- start start 2024-04-25T10:12:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.092+0000 7fbe9c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe940ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.092+0000 7fbe9c400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe9406d290 0x7fbe9406d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.092+0000 7fbe9c400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe94144b30 con 0x7fbe94144750 2024-04-25T10:12:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.092+0000 7fbe9c400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe940712d0 con 0x7fbe9406d290 2024-04-25T10:12:13.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.092+0000 7fbe9c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940715b0 con 0x7fbe940ce920 2024-04-25T10:12:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.093+0000 7fbe9a400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe9406d290 0x7fbe9406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.093+0000 7fbe99000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe940ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.093+0000 7fbe9a400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbe9406d290 0x7fbe9406d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:33934/0 (socket says 172.21.15.165:33934) 2024-04-25T10:12:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.093+0000 7fbe9a400640 1 -- 172.21.15.165:0/2689416335 learned_addr learned my addr 172.21.15.165:0/2689416335 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.094+0000 7fbe9a400640 1 -- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 msgr2=0x7fbe940ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:13.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.094+0000 7fbe9a400640 1 --2- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe940ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.094+0000 7fbe9a400640 1 -- 172.21.15.165:0/2689416335 >> v1:172.21.15.172:6789/0 conn(0x7fbe94144750 legacy=0x7fbe9406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.094+0000 7fbe9a400640 1 -- 172.21.15.165:0/2689416335 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94071890 con 0x7fbe9406d290 2024-04-25T10:12:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.095+0000 7fbe9a400640 1 --2- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe9406d720 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e50 tx=0x7fbe8c02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=17599c799afcd6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe93e00640 1 -- 172.21.15.165:0/2689416335 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c036070 con 0x7fbe9406d290 2024-04-25T10:12:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe93e00640 1 -- 172.21.15.165:0/2689416335 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c031600 con 0x7fbe9406d290 2024-04-25T10:12:13.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe93e00640 1 -- 172.21.15.165:0/2689416335 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c0319d0 con 0x7fbe9406d290 2024-04-25T10:12:13.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe9c400640 1 -- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 msgr2=0x7fbe9406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe9c400640 1 --2- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe9406d720 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009e50 tx=0x7fbe8c02a710 comp rx=0 tx=0).stop 2024-04-25T10:12:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe9c400640 1 -- 172.21.15.165:0/2689416335 shutdown_connections 2024-04-25T10:12:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe9c400640 1 --2- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe940ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe9c400640 1 --2- 172.21.15.165:0/2689416335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe9406d720 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.096+0000 7fbe9c400640 1 -- 172.21.15.165:0/2689416335 >> 172.21.15.165:0/2689416335 conn(0x7fbe9406abe0 msgr2=0x7fbe9406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.097+0000 7fbe9c400640 1 -- 172.21.15.165:0/2689416335 shutdown_connections 2024-04-25T10:12:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.097+0000 7fbe9c400640 1 -- 172.21.15.165:0/2689416335 wait complete. 2024-04-25T10:12:13.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.098+0000 7fbe9c400640 1 Processor -- start 2024-04-25T10:12:13.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.098+0000 7fbe9c400640 1 -- start start 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.099+0000 7fbe9c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe94156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.099+0000 7fbe9c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe9413cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.099+0000 7fbe9c400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 0x7fbe9413d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.099+0000 7fbe9c400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9406f020 con 0x7fbe94144750 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.099+0000 7fbe9c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9406ece0 con 0x7fbe9406d290 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.099+0000 7fbe9c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe94145450 con 0x7fbe940ce920 2024-04-25T10:12:13.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 0x7fbe9413d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 0x7fbe9413d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33250/0 (socket says 172.21.15.165:33250) 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 -- 172.21.15.165:0/27062317 learned_addr learned my addr 172.21.15.165:0/27062317 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe99a00640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe94156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe99000640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe9413cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 -- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 msgr2=0x7fbe9413cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe9413cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 -- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 msgr2=0x7fbe94156bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe94156bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.100+0000 7fbe9a400640 1 -- 172.21.15.165:0/27062317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe94141110 con 0x7fbe94144750 2024-04-25T10:12:13.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.101+0000 7fbe9a400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 0x7fbe9413d270 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c034040 tx=0x7fbe8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.101+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c036050 con 0x7fbe94144750 2024-04-25T10:12:13.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.101+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe8c042070 con 0x7fbe94144750 2024-04-25T10:12:13.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.101+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c009510 con 0x7fbe94144750 2024-04-25T10:12:13.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.101+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe941413f0 con 0x7fbe94144750 2024-04-25T10:12:13.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.101+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe94143090 con 0x7fbe94144750 2024-04-25T10:12:13.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.102+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe94142ba0 con 0x7fbe94144750 2024-04-25T10:12:13.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.103+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbe8c007bf0 con 0x7fbe94144750 2024-04-25T10:12:13.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.104+0000 7fbe91600640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe7806b4d0 0x7fbe7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.104+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fbe8c0ab2d0 con 0x7fbe94144750 2024-04-25T10:12:13.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.104+0000 7fbe99a00640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe7806b4d0 0x7fbe7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.105+0000 7fbe99a00640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe7806b4d0 0x7fbe7806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbe88001640 tx=0x7fbe88009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.108+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbe8c045360 con 0x7fbe94144750 2024-04-25T10:12:13.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.341+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbe94073990 con 0x7fbe94144750 2024-04-25T10:12:13.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.342+0000 7fbe91600640 1 -- 172.21.15.165:0/27062317 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fbe8c079eb0 con 0x7fbe94144750 2024-04-25T10:12:13.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:13.351 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:13.352 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:13.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.344+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe7806b4d0 msgr2=0x7fbe7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.344+0000 7fbe9c400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe7806b4d0 0x7fbe7806d990 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fbe88001640 tx=0x7fbe88009040 comp rx=0 tx=0).stop 2024-04-25T10:12:13.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.344+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 msgr2=0x7fbe9413d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.344+0000 7fbe9c400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 0x7fbe9413d270 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c034040 tx=0x7fbe8c004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 shutdown_connections 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbe7806b4d0 0x7fbe7806d990 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbe94144750 0x7fbe9413d270 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbe940ce920 0x7fbe9413cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 --2- 172.21.15.165:0/27062317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbe9406d290 0x7fbe94156bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 >> 172.21.15.165:0/27062317 conn(0x7fbe9406abe0 msgr2=0x7fbe940cd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 shutdown_connections 2024-04-25T10:12:13.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.345+0000 7fbe9c400640 1 -- 172.21.15.165:0/27062317 wait complete. 2024-04-25T10:12:13.369 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:13.369 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:13.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.563+0000 7f770c200640 1 Processor -- start 2024-04-25T10:12:13.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.563+0000 7f770c200640 1 -- start start 2024-04-25T10:12:13.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.564+0000 7f770c200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f770406abe0 0x7f77040cf780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.564+0000 7f770c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f770c200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f770406b500 con 0x7f77040cc1d0 2024-04-25T10:12:13.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f770c200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77040ce6e0 con 0x7f770406abe0 2024-04-25T10:12:13.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f770c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77040cfe10 con 0x7f770415c990 2024-04-25T10:12:13.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f7708e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f7708e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:40744/0 (socket says 172.21.15.165:40744) 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f7708e00640 1 -- 172.21.15.165:0/2279574676 learned_addr learned my addr 172.21.15.165:0/2279574676 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.565+0000 7f7709800640 1 --2- 172.21.15.165:0/2279574676 >> v2:172.21.15.172:3300/0 conn(0x7f770406abe0 0x7f77040cf780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.566+0000 7f7708e00640 1 -- 172.21.15.165:0/2279574676 >> v2:172.21.15.172:3300/0 conn(0x7f770406abe0 msgr2=0x7f77040cf780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.566+0000 7f7708e00640 1 --2- 172.21.15.165:0/2279574676 >> v2:172.21.15.172:3300/0 conn(0x7f770406abe0 0x7f77040cf780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.566+0000 7f7708e00640 1 -- 172.21.15.165:0/2279574676 >> v1:172.21.15.172:6789/0 conn(0x7f77040cc1d0 legacy=0x7f77040ceb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:13.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.566+0000 7f7708e00640 1 -- 172.21.15.165:0/2279574676 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040d00f0 con 0x7f770415c990 2024-04-25T10:12:13.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.567+0000 7f7708e00640 1 --2- 172.21.15.165:0/2279574676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770406afc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009870 tx=0x7f76f002a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd4d5210c95c41e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.568+0000 7f7703e00640 1 -- 172.21.15.165:0/2279574676 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0037070 con 0x7f770415c990 2024-04-25T10:12:13.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.568+0000 7f7703e00640 1 -- 172.21.15.165:0/2279574676 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f0032440 con 0x7f770415c990 2024-04-25T10:12:13.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.568+0000 7f7703e00640 1 -- 172.21.15.165:0/2279574676 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0032810 con 0x7f770415c990 2024-04-25T10:12:13.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 -- 172.21.15.165:0/2279574676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 msgr2=0x7f770406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 --2- 172.21.15.165:0/2279574676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770406afc0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009870 tx=0x7f76f002a200 comp rx=0 tx=0).stop 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 -- 172.21.15.165:0/2279574676 shutdown_connections 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 --2- 172.21.15.165:0/2279574676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770406afc0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 --2- 172.21.15.165:0/2279574676 >> v2:172.21.15.172:3300/0 conn(0x7f770406abe0 0x7f77040cf780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 -- 172.21.15.165:0/2279574676 >> 172.21.15.165:0/2279574676 conn(0x7f77040b2290 msgr2=0x7f77040b26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 -- 172.21.15.165:0/2279574676 shutdown_connections 2024-04-25T10:12:13.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.569+0000 7f770c200640 1 -- 172.21.15.165:0/2279574676 wait complete. 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.571+0000 7f770c200640 1 Processor -- start 2024-04-25T10:12:13.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.571+0000 7f770c200640 1 -- start start 2024-04-25T10:12:13.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770c200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 0x7f77040b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770c200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77040cc1d0 0x7f77041f3120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770c200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770409df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770c200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f770415cdd0 con 0x7f770406abe0 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770c200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77040ce370 con 0x7f77040cc1d0 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770c200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77040cf280 con 0x7f770415c990 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7709800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 0x7f77040b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7709800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 0x7f77040b3990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33262/0 (socket says 172.21.15.165:33262) 2024-04-25T10:12:13.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7709800640 1 -- 172.21.15.165:0/2132422381 learned_addr learned my addr 172.21.15.165:0/2132422381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f770a200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770409df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7708e00640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77040cc1d0 0x7f77041f3120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7709800640 1 -- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 msgr2=0x7f770409df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7709800640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770409df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.572+0000 7f7709800640 1 -- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77040cc1d0 msgr2=0x7f77041f3120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f7709800640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77040cc1d0 0x7f77041f3120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f7709800640 1 -- 172.21.15.165:0/2132422381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77040b3ed0 con 0x7f770406abe0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f7709800640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 0x7f77040b3990 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f76f800e9b0 tx=0x7f76f800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8017070 con 0x7f770406abe0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f800c9a0 con 0x7f770406abe0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f800cd90 con 0x7f770406abe0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77041f3660 con 0x7f770406abe0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.573+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77041f59e0 con 0x7f770406abe0 2024-04-25T10:12:13.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.574+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7704073990 con 0x7f770406abe0 2024-04-25T10:12:13.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.576+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76f801a070 con 0x7f770406abe0 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.576+0000 7f7701600640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76e406b4d0 0x7f76e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.577+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f76f808a450 con 0x7f770406abe0 2024-04-25T10:12:13.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.577+0000 7f7708e00640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76e406b4d0 0x7f76e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:13.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.577+0000 7f7708e00640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76e406b4d0 0x7f76e406d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:13.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.581+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76f80590c0 con 0x7f770406abe0 2024-04-25T10:12:13.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.851+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f77040b3060 con 0x7f770406abe0 2024-04-25T10:12:13.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.852+0000 7f7701600640 1 -- 172.21.15.165:0/2132422381 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1637 (secure 0 0 0) 0x7f76f80589c0 con 0x7f770406abe0 2024-04-25T10:12:13.858 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:13.858 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":634,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076581352,"bytes_used":2583371776,"bytes_avail":770510741504,"bytes_total":773094113280,"write_bytes_sec":23926797,"read_op_per_sec":0,"write_op_per_sec":25},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7569}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":22,"modified":"2024-04-25T10:12:10.336282+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"idle+waiting paths [/scrub_control_test_path]"}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:13.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.854+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76e406b4d0 msgr2=0x7f76e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.854+0000 7f770c200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76e406b4d0 0x7f76e406d990 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f76f0034000 tx=0x7f76f002e000 comp rx=0 tx=0).stop 2024-04-25T10:12:13.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.854+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 msgr2=0x7f77040b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.854+0000 7f770c200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 0x7f77040b3990 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f76f800e9b0 tx=0x7f76f800ee70 comp rx=0 tx=0).stop 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 shutdown_connections 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76e406b4d0 0x7f76e406d990 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f770415c990 0x7f770409df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77040cc1d0 0x7f77041f3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 --2- 172.21.15.165:0/2132422381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f770406abe0 0x7f77040b3990 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 >> 172.21.15.165:0/2132422381 conn(0x7f77040b2290 msgr2=0x7f770415e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 shutdown_connections 2024-04-25T10:12:13.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:13.855+0000 7f770c200640 1 -- 172.21.15.165:0/2132422381 wait complete. 2024-04-25T10:12:13.880 INFO:tasks.cephfs.filesystem.ceph_manager:{'a': 'idle+waiting paths [/scrub_control_test_path]'} 2024-04-25T10:12:14.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.089+0000 7f841d800640 1 Processor -- start 2024-04-25T10:12:15.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.090+0000 7f841d800640 1 -- start start 2024-04-25T10:12:15.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.091+0000 7f841d800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f841815db90 0x7f84180b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.091+0000 7f841d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84180d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.091+0000 7f841d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8418161df0 con 0x7f84181615a0 2024-04-25T10:12:15.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.091+0000 7f841d800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84180d1f50 con 0x7f841815db90 2024-04-25T10:12:15.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.091+0000 7f841d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84181620d0 con 0x7f84180d06c0 2024-04-25T10:12:15.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.092+0000 7f840fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f84181615a0 0x7f84180cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34802/0 (socket says 172.21.15.165:34802) 2024-04-25T10:12:15.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.092+0000 7f840fe00640 1 -- 172.21.15.165:0/3284550572 learned_addr learned my addr 172.21.15.165:0/3284550572 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.092+0000 7f8417e00640 1 --2- 172.21.15.165:0/3284550572 >> v2:172.21.15.172:3300/0 conn(0x7f841815db90 0x7f84180b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.092+0000 7f841ce00640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84180d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.092+0000 7f8417400640 1 -- 172.21.15.165:0/3284550572 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110563958 0 0) 0x7f8418161df0 con 0x7f84181615a0 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.092+0000 7f8417400640 1 -- 172.21.15.165:0/3284550572 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83fc003520 con 0x7f84181615a0 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f8417e00640 1 -- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 msgr2=0x7f84180d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f8417e00640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84180d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f8417e00640 1 -- 172.21.15.165:0/3284550572 >> v1:172.21.15.165:6789/0 conn(0x7f84181615a0 legacy=0x7f84180cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f8417400640 1 -- 172.21.15.165:0/3284550572 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827400523 0 0) 0x7f83fc003520 con 0x7f84181615a0 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f8417e00640 1 -- 172.21.15.165:0/3284550572 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84181623b0 con 0x7f841815db90 2024-04-25T10:12:15.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f841ce00640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84180d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.094+0000 7f8417e00640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180b93c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f84080097a0 tx=0x7f840802a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d9e19ed2ef8151f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.095+0000 7f8417400640 1 -- 172.21.15.165:0/3284550572 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408037070 con 0x7f841815db90 2024-04-25T10:12:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.095+0000 7f8417400640 1 -- 172.21.15.165:0/3284550572 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840802aba0 con 0x7f841815db90 2024-04-25T10:12:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.095+0000 7f8417400640 1 -- 172.21.15.165:0/3284550572 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400001670 con 0x7f841815db90 2024-04-25T10:12:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.095+0000 7f841d800640 1 -- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 msgr2=0x7f84180b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.095+0000 7f841d800640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180b93c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f84080097a0 tx=0x7f840802a110 comp rx=0 tx=0).stop 2024-04-25T10:12:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.096+0000 7f841d800640 1 -- 172.21.15.165:0/3284550572 shutdown_connections 2024-04-25T10:12:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.096+0000 7f841d800640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84180d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.096+0000 7f841d800640 1 --2- 172.21.15.165:0/3284550572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180b93c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.096+0000 7f841d800640 1 -- 172.21.15.165:0/3284550572 >> 172.21.15.165:0/3284550572 conn(0x7f84180b52b0 msgr2=0x7f84180b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.096+0000 7f841d800640 1 -- 172.21.15.165:0/3284550572 shutdown_connections 2024-04-25T10:12:15.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.096+0000 7f841d800640 1 -- 172.21.15.165:0/3284550572 wait complete. 2024-04-25T10:12:15.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.098+0000 7f841d800640 1 Processor -- start 2024-04-25T10:12:15.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.098+0000 7f841d800640 1 -- start start 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84181f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 0x7f84180ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418161ba0 con 0x7f84181615a0 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8418161a20 con 0x7f841815db90 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84180b68d0 con 0x7f84180d06c0 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 0x7f84180ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 0x7f84180ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33266/0 (socket says 172.21.15.165:33266) 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f841ce00640 1 -- 172.21.15.165:0/2692796886 learned_addr learned my addr 172.21.15.165:0/2692796886 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f8417e00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84181f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.099+0000 7f840fe00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841ce00640 1 -- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 msgr2=0x7f84181f4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841ce00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84181f4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841ce00640 1 -- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 msgr2=0x7f84180c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841ce00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841ce00640 1 -- 172.21.15.165:0/2692796886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8418160040 con 0x7f84181615a0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841ce00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 0x7f84180ccb00 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f841000efc0 tx=0x7f841000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410018020 con 0x7f84181615a0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8410014480 con 0x7f84181615a0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f841815fd60 con 0x7f84181615a0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.100+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8410014870 con 0x7f84181615a0 2024-04-25T10:12:15.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.101+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84180cabf0 con 0x7f84181615a0 2024-04-25T10:12:15.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.102+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8418073990 con 0x7f84181615a0 2024-04-25T10:12:15.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.103+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8410005d10 con 0x7f84181615a0 2024-04-25T10:12:15.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.104+0000 7f8414c00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f406b500 0x7f83f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.104+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f841008b0f0 con 0x7f84181615a0 2024-04-25T10:12:15.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.105+0000 7f8417e00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f406b500 0x7f83f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.106+0000 7f8417e00640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f406b500 0x7f83f406d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8408034000 tx=0x7f840802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.108+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8410022030 con 0x7f84181615a0 2024-04-25T10:12:15.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.345+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84180c6ff0 con 0x7f84181615a0 2024-04-25T10:12:15.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.346+0000 7f8414c00640 1 -- 172.21.15.165:0/2692796886 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f8410010040 con 0x7f84181615a0 2024-04-25T10:12:15.351 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:15.351 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:15.352 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.348+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f406b500 msgr2=0x7f83f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.348+0000 7f841d800640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f406b500 0x7f83f406d9c0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f8408034000 tx=0x7f840802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.348+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 msgr2=0x7f84180ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.348+0000 7f841d800640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 0x7f84180ccb00 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7f841000efc0 tx=0x7f841000c490 comp rx=0 tx=0).stop 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 shutdown_connections 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83f406b500 0x7f83f406d9c0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84181615a0 0x7f84180ccb00 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f841815db90 0x7f84180c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 --2- 172.21.15.165:0/2692796886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f84180d06c0 0x7f84181f4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 >> 172.21.15.165:0/2692796886 conn(0x7f84180b52b0 msgr2=0x7f84180b7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 shutdown_connections 2024-04-25T10:12:15.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.349+0000 7f841d800640 1 -- 172.21.15.165:0/2692796886 wait complete. 2024-04-25T10:12:15.373 DEBUG:tasks.cephfs.test_scrub_checks:current active=['a'] 2024-04-25T10:12:15.374 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.567+0000 7fe037400640 1 Processor -- start 2024-04-25T10:12:15.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.567+0000 7fe037400640 1 -- start start 2024-04-25T10:12:15.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.568+0000 7fe037400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe03015db70 0x7fe0300cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe037400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0300cfc20 0x7fe03006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe037400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe030071650 con 0x7fe03015df50 2024-04-25T10:12:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe037400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0300cbd80 con 0x7fe0300cfc20 2024-04-25T10:12:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe037400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe030161910 con 0x7fe03015db70 2024-04-25T10:12:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe035400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0300cfc20 0x7fe03006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe027e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe03015df50 0x7fe0300cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34814/0 (socket says 172.21.15.165:34814) 2024-04-25T10:12:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe027e00640 1 -- 172.21.15.165:0/2470785274 learned_addr learned my addr 172.21.15.165:0/2470785274 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe034a00640 1 --2- 172.21.15.165:0/2470785274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe03015db70 0x7fe0300cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe035400640 1 -- 172.21.15.165:0/2470785274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe03015db70 msgr2=0x7fe0300cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.569+0000 7fe035400640 1 --2- 172.21.15.165:0/2470785274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe03015db70 0x7fe0300cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.570+0000 7fe035400640 1 -- 172.21.15.165:0/2470785274 >> v1:172.21.15.165:6789/0 conn(0x7fe03015df50 legacy=0x7fe0300cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.570+0000 7fe035400640 1 -- 172.21.15.165:0/2470785274 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0300cf230 con 0x7fe0300cfc20 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.570+0000 7fe02fe00640 1 -- 172.21.15.165:0/2470785274 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3169106728 0 0) 0x7fe018001110 con 0x7fe03015df50 2024-04-25T10:12:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.571+0000 7fe035400640 1 --2- 172.21.15.165:0/2470785274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0300cfc20 0x7fe03006d290 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe028009d90 tx=0x7fe02802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45bcde904e7d53c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.571+0000 7fe02fe00640 1 -- 172.21.15.165:0/2470785274 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028007740 con 0x7fe0300cfc20 2024-04-25T10:12:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.571+0000 7fe02fe00640 1 -- 172.21.15.165:0/2470785274 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0280078e0 con 0x7fe0300cfc20 2024-04-25T10:12:15.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.571+0000 7fe02fe00640 1 -- 172.21.15.165:0/2470785274 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028007cb0 con 0x7fe0300cfc20 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 -- 172.21.15.165:0/2470785274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0300cfc20 msgr2=0x7fe03006d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 --2- 172.21.15.165:0/2470785274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0300cfc20 0x7fe03006d290 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fe028009d90 tx=0x7fe02802a660 comp rx=0 tx=0).stop 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 -- 172.21.15.165:0/2470785274 shutdown_connections 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 --2- 172.21.15.165:0/2470785274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0300cfc20 0x7fe03006d290 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 --2- 172.21.15.165:0/2470785274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe03015db70 0x7fe0300cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 -- 172.21.15.165:0/2470785274 >> 172.21.15.165:0/2470785274 conn(0x7fe03006abe0 msgr2=0x7fe03006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 -- 172.21.15.165:0/2470785274 shutdown_connections 2024-04-25T10:12:15.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.572+0000 7fe037400640 1 -- 172.21.15.165:0/2470785274 wait complete. 2024-04-25T10:12:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.574+0000 7fe037400640 1 Processor -- start 2024-04-25T10:12:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.574+0000 7fe037400640 1 -- start start 2024-04-25T10:12:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe037400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0300cfc20 0x7fe03015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe037400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03015db70 0x7fe0301f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe037400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 0x7fe030159810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe037400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030162120 con 0x7fe03015df50 2024-04-25T10:12:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe037400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030161e40 con 0x7fe03015db70 2024-04-25T10:12:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe037400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe030162640 con 0x7fe0300cfc20 2024-04-25T10:12:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe035400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 0x7fe030159810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe035400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 0x7fe030159810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33292/0 (socket says 172.21.15.165:33292) 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.575+0000 7fe035400640 1 -- 172.21.15.165:0/4162422284 learned_addr learned my addr 172.21.15.165:0/4162422284 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe034a00640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0300cfc20 0x7fe03015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe027e00640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03015db70 0x7fe0301f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe035400640 1 -- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0300cfc20 msgr2=0x7fe03015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe035400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0300cfc20 0x7fe03015e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe035400640 1 -- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03015db70 msgr2=0x7fe0301f66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe035400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03015db70 0x7fe0301f66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe035400640 1 -- 172.21.15.165:0/4162422284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0301f3b50 con 0x7fe03015df50 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe027e00640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03015db70 0x7fe0301f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe035400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 0x7fe030159810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe028009d90 tx=0x7fe028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe028040070 con 0x7fe03015df50 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe028031bc0 con 0x7fe03015df50 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02803c420 con 0x7fe03015df50 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.576+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0301f3e30 con 0x7fe03015df50 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.577+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0301f43f0 con 0x7fe03015df50 2024-04-25T10:12:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.578+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe030073990 con 0x7fe03015df50 2024-04-25T10:12:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.579+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe028039040 con 0x7fe03015df50 2024-04-25T10:12:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.579+0000 7fe02d600640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe00c06b4d0 0x7fe00c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.580+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fe0280ab9f0 con 0x7fe03015df50 2024-04-25T10:12:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.580+0000 7fe034a00640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe00c06b4d0 0x7fe00c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.580+0000 7fe034a00640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe00c06b4d0 0x7fe00c06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe020006fd0 tx=0x7fe020008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:15.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.585+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe02807a5d0 con 0x7fe03015df50 2024-04-25T10:12:15.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.848+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fe0300cda60 con 0x7fe03015df50 2024-04-25T10:12:15.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.849+0000 7fe02d600640 1 -- 172.21.15.165:0/4162422284 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7fe028079ed0 con 0x7fe03015df50 2024-04-25T10:12:15.854 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:15.854 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":636,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":100,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076581352,"bytes_used":2583371776,"bytes_avail":770510741504,"bytes_total":773094113280,"write_bytes_sec":8257562,"read_op_per_sec":0,"write_op_per_sec":9},"fsmap":{"epoch":65,"id":6,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":6,"rank":0,"name":"a","status":"up:active","gid":7569}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":23,"modified":"2024-04-25T10:12:14.338005+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:12:15.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe00c06b4d0 msgr2=0x7fe00c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe00c06b4d0 0x7fe00c06d990 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe020006fd0 tx=0x7fe020008040 comp rx=0 tx=0).stop 2024-04-25T10:12:15.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 msgr2=0x7fe030159810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:15.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 0x7fe030159810 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fe028009d90 tx=0x7fe028004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 shutdown_connections 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe00c06b4d0 0x7fe00c06d990 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe03015df50 0x7fe030159810 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe03015db70 0x7fe0301f66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 --2- 172.21.15.165:0/4162422284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0300cfc20 0x7fe03015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.851+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 >> 172.21.15.165:0/4162422284 conn(0x7fe03006abe0 msgr2=0x7fe030157f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.852+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 shutdown_connections 2024-04-25T10:12:15.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:15.852+0000 7fe037400640 1 -- 172.21.15.165:0/4162422284 wait complete. 2024-04-25T10:12:15.878 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:12:15.878 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:15.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:15.979 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:16.090 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:16.090 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:12:16.090 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:16.147 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:12:16.147 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:16.198 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.398+0000 7f275cc00640 1 Processor -- start 2024-04-25T10:12:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.398+0000 7f275cc00640 1 -- start start 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f275cc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f275815db90 0x7f27580b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f275cc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f27581615a0 0x7f27580cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f275cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27580d04f0 0x7f2758061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f275cc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580d0980 con 0x7f275815db90 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f275cc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27580d1f50 con 0x7f27581615a0 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f275cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2758161df0 con 0x7f27580d04f0 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f2757400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f275815db90 0x7f27580b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f2757400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f275815db90 0x7f27580b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33306/0 (socket says 172.21.15.165:33306) 2024-04-25T10:12:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.400+0000 7f2757400640 1 -- 172.21.15.165:0/2147247254 learned_addr learned my addr 172.21.15.165:0/2147247254 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:16.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f274ea00640 1 --2- 172.21.15.165:0/2147247254 >> v2:172.21.15.172:3300/0 conn(0x7f27581615a0 0x7f27580cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f2757e00640 1 --2- 172.21.15.165:0/2147247254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27580d04f0 0x7f2758061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f2757400640 1 -- 172.21.15.165:0/2147247254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27580d04f0 msgr2=0x7f2758061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f2757400640 1 --2- 172.21.15.165:0/2147247254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27580d04f0 0x7f2758061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f2757400640 1 -- 172.21.15.165:0/2147247254 >> v2:172.21.15.172:3300/0 conn(0x7f27581615a0 msgr2=0x7f27580cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f2757400640 1 --2- 172.21.15.165:0/2147247254 >> v2:172.21.15.172:3300/0 conn(0x7f27581615a0 0x7f27580cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f2757400640 1 -- 172.21.15.165:0/2147247254 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27581620d0 con 0x7f275815db90 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.401+0000 7f274ea00640 1 --2- 172.21.15.165:0/2147247254 >> v2:172.21.15.172:3300/0 conn(0x7f27581615a0 0x7f27580cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.402+0000 7f2757400640 1 --2- 172.21.15.165:0/2147247254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275815db90 0x7f27580b93c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f27580d1b30 tx=0x7f274802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f6f21d854ade55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.402+0000 7f2756a00640 1 -- 172.21.15.165:0/2147247254 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748037070 con 0x7f275815db90 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.402+0000 7f2756a00640 1 -- 172.21.15.165:0/2147247254 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2748032440 con 0x7f275815db90 2024-04-25T10:12:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.402+0000 7f2756a00640 1 -- 172.21.15.165:0/2147247254 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2748032810 con 0x7f275815db90 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 -- 172.21.15.165:0/2147247254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275815db90 msgr2=0x7f27580b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 --2- 172.21.15.165:0/2147247254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275815db90 0x7f27580b93c0 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f27580d1b30 tx=0x7f274802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 -- 172.21.15.165:0/2147247254 shutdown_connections 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 --2- 172.21.15.165:0/2147247254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27580d04f0 0x7f2758061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 --2- 172.21.15.165:0/2147247254 >> v2:172.21.15.172:3300/0 conn(0x7f27581615a0 0x7f27580cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 --2- 172.21.15.165:0/2147247254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f275815db90 0x7f27580b93c0 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 -- 172.21.15.165:0/2147247254 >> 172.21.15.165:0/2147247254 conn(0x7f27580b52b0 msgr2=0x7f27580b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 -- 172.21.15.165:0/2147247254 shutdown_connections 2024-04-25T10:12:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.403+0000 7f275cc00640 1 -- 172.21.15.165:0/2147247254 wait complete. 2024-04-25T10:12:16.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.405+0000 7f275cc00640 1 Processor -- start 2024-04-25T10:12:16.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.405+0000 7f275cc00640 1 -- start start 2024-04-25T10:12:16.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f275cc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27580d04f0 0x7f27580ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f275cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f275815db90 0x7f27580cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f275cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 0x7f27580cc740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f275cc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2758161a20 con 0x7f27581615a0 2024-04-25T10:12:16.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f275cc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27580d1be0 con 0x7f27580d04f0 2024-04-25T10:12:16.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f275cc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2758161ba0 con 0x7f275815db90 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f2757e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 0x7f27580cc740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f2757e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 0x7f27580cc740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33320/0 (socket says 172.21.15.165:33320) 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f2757e00640 1 -- 172.21.15.165:0/4036006133 learned_addr learned my addr 172.21.15.165:0/4036006133 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f274ea00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f275815db90 0x7f27580cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f2757e00640 1 -- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f275815db90 msgr2=0x7f27580cadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.406+0000 7f2757e00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f275815db90 0x7f27580cadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f2757e00640 1 -- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27580d04f0 msgr2=0x7f27580ca870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f2757e00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27580d04f0 0x7f27580ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f2757e00640 1 -- 172.21.15.165:0/4036006133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27581f0cf0 con 0x7f27581615a0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f2757e00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 0x7f27580cc740 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f275000cd20 tx=0x7f275000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750003b90 con 0x7f27581615a0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2750003d30 con 0x7f27581615a0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2750004240 con 0x7f27581615a0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27580cd420 con 0x7f27581615a0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.407+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27581e7d70 con 0x7f27581615a0 2024-04-25T10:12:16.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.409+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2758073990 con 0x7f27581615a0 2024-04-25T10:12:16.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.410+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27500043e0 con 0x7f27581615a0 2024-04-25T10:12:16.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.411+0000 7f274fe00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f273406b4d0 0x7f273406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.411+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f275008ae70 con 0x7f27581615a0 2024-04-25T10:12:16.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.411+0000 7f2757400640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f273406b4d0 0x7f273406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.412+0000 7f2757400640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f273406b4d0 0x7f273406d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2748034000 tx=0x7f274802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.415+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f275001d340 con 0x7f27581615a0 2024-04-25T10:12:16.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.611+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27580d14b0 con 0x7f27581615a0 2024-04-25T10:12:16.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.612+0000 7f274fe00640 1 -- 172.21.15.165:0/4036006133 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17114 (secure 0 0 0) 0x7f2750059a50 con 0x7f27581615a0 2024-04-25T10:12:16.618 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:16.618 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":100,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:47.812589+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":5}]},{"pgid":"14.a","mappings":[{"from":1,"to":2}]},{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f273406b4d0 msgr2=0x7f273406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f273406b4d0 0x7f273406d990 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f2748034000 tx=0x7f274802c000 comp rx=0 tx=0).stop 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 msgr2=0x7f27580cc740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 0x7f27580cc740 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f275000cd20 tx=0x7f275000a660 comp rx=0 tx=0).stop 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 shutdown_connections 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f273406b4d0 0x7f273406d990 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f27581615a0 0x7f27580cc740 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f275815db90 0x7f27580cadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 --2- 172.21.15.165:0/4036006133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f27580d04f0 0x7f27580ca870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 >> 172.21.15.165:0/4036006133 conn(0x7f27580b52b0 msgr2=0x7f27580b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 shutdown_connections 2024-04-25T10:12:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.615+0000 7f275cc00640 1 -- 172.21.15.165:0/4036006133 wait complete. 2024-04-25T10:12:16.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:20.638187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:21.928273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:16.641 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:16.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.843+0000 7f08aca00640 1 Processor -- start 2024-04-25T10:12:16.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.843+0000 7f08aca00640 1 -- start start 2024-04-25T10:12:16.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08aca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f08a806bff0 0x7f08a80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08aca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 0x7f08a80d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08aca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08a815e110 con 0x7f08a815dd30 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08aca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08a806d000 con 0x7f08a806bff0 2024-04-25T10:12:16.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08aca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80b2fd0 con 0x7f08a80b5970 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 0x7f08a80d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 0x7f08a80d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:35270/0 (socket says 172.21.15.165:35270) 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.845+0000 7f08a7e00640 1 -- 172.21.15.165:0/1483816590 learned_addr learned my addr 172.21.15.165:0/1483816590 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.846+0000 7f08a6a00640 1 --2- 172.21.15.165:0/1483816590 >> v2:172.21.15.172:3300/0 conn(0x7f08a806bff0 0x7f08a80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.846+0000 7f08a7e00640 1 -- 172.21.15.165:0/1483816590 >> v2:172.21.15.172:3300/0 conn(0x7f08a806bff0 msgr2=0x7f08a80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.846+0000 7f08a7e00640 1 --2- 172.21.15.165:0/1483816590 >> v2:172.21.15.172:3300/0 conn(0x7f08a806bff0 0x7f08a80b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.846+0000 7f08a7e00640 1 -- 172.21.15.165:0/1483816590 >> v1:172.21.15.172:6789/0 conn(0x7f08a815dd30 legacy=0x7f08a80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.846+0000 7f08a7e00640 1 -- 172.21.15.165:0/1483816590 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a806c480 con 0x7f08a80b5970 2024-04-25T10:12:16.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.847+0000 7f08a7e00640 1 --2- 172.21.15.165:0/1483816590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 0x7f08a80d2160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009e50 tx=0x7f08a002a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6d1de627d7a1cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.848+0000 7f08a6000640 1 -- 172.21.15.165:0/1483816590 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0036070 con 0x7f08a80b5970 2024-04-25T10:12:16.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.848+0000 7f08a6000640 1 -- 172.21.15.165:0/1483816590 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0031600 con 0x7f08a80b5970 2024-04-25T10:12:16.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.848+0000 7f08a6000640 1 -- 172.21.15.165:0/1483816590 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a00319d0 con 0x7f08a80b5970 2024-04-25T10:12:16.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.848+0000 7f08aca00640 1 -- 172.21.15.165:0/1483816590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 msgr2=0x7f08a80d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 --2- 172.21.15.165:0/1483816590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 0x7f08a80d2160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009e50 tx=0x7f08a002a710 comp rx=0 tx=0).stop 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 -- 172.21.15.165:0/1483816590 shutdown_connections 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 --2- 172.21.15.165:0/1483816590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a80b5970 0x7f08a80d2160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 --2- 172.21.15.165:0/1483816590 >> v2:172.21.15.172:3300/0 conn(0x7f08a806bff0 0x7f08a80b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 -- 172.21.15.165:0/1483816590 >> 172.21.15.165:0/1483816590 conn(0x7f08a80d0610 msgr2=0x7f08a80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 -- 172.21.15.165:0/1483816590 shutdown_connections 2024-04-25T10:12:16.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.849+0000 7f08aca00640 1 -- 172.21.15.165:0/1483816590 wait complete. 2024-04-25T10:12:16.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.850+0000 7f08aca00640 1 Processor -- start 2024-04-25T10:12:16.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 -- start start 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a80b5970 0x7f08a815bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a815dd30 0x7f08a80ac7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 0x7f08a80aa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a806cd20 con 0x7f08a80acd00 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08a806ca40 con 0x7f08a80b5970 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.851+0000 7f08aca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08a80d26a0 con 0x7f08a815dd30 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 0x7f08a80aa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 0x7f08a80aa2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48098/0 (socket says 172.21.15.165:48098) 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 -- 172.21.15.165:0/1613350133 learned_addr learned my addr 172.21.15.165:0/1613350133 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a6a00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a815dd30 0x7f08a80ac7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 -- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a815dd30 msgr2=0x7f08a80ac7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a815dd30 0x7f08a80ac7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 -- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a80b5970 msgr2=0x7f08a815bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:16.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a80b5970 0x7f08a815bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:16.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.852+0000 7f08a7e00640 1 -- 172.21.15.165:0/1613350133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a80aaf00 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.853+0000 7f08a7e00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 0x7f08a80aa2b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009e50 tx=0x7f08a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.853+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0036050 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.853+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a0042070 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.853+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08a80ab1e0 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.853+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a0009510 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.853+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08a80ab7a0 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.854+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08a8073990 con 0x7f08a80acd00 2024-04-25T10:12:16.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.855+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f08a0007bf0 con 0x7f08a80acd00 2024-04-25T10:12:16.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.856+0000 7f089f400640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f088806b4d0 0x7f088806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:16.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.856+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f08a00ab250 con 0x7f08a80acd00 2024-04-25T10:12:16.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.856+0000 7f08a7400640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f088806b4d0 0x7f088806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:16.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.857+0000 7f08a7400640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f088806b4d0 0x7f088806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0898000f60 tx=0x7f08980079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:16.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:16.860+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f08a0079ec0 con 0x7f08a80acd00 2024-04-25T10:12:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.059+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08a80d16c0 con 0x7f08a80acd00 2024-04-25T10:12:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.060+0000 7f089f400640 1 -- 172.21.15.165:0/1613350133 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f08a00797c0 con 0x7f08a80acd00 2024-04-25T10:12:17.066 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:12:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.062+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f088806b4d0 msgr2=0x7f088806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.062+0000 7f08aca00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f088806b4d0 0x7f088806d990 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0898000f60 tx=0x7f08980079b0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.062+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 msgr2=0x7f08a80aa2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.062+0000 7f08aca00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 0x7f08a80aa2b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f08a0009e50 tx=0x7f08a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 shutdown_connections 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f088806b4d0 0x7f088806d990 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f08a80acd00 0x7f08a80aa2b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f08a815dd30 0x7f08a80ac7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 --2- 172.21.15.165:0/1613350133 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f08a80b5970 0x7f08a815bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 >> 172.21.15.165:0/1613350133 conn(0x7f08a80d0610 msgr2=0x7f08a80b3410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 shutdown_connections 2024-04-25T10:12:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.063+0000 7f08aca00640 1 -- 172.21.15.165:0/1613350133 wait complete. 2024-04-25T10:12:17.087 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:17.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.286+0000 7f63a0600640 1 Processor -- start 2024-04-25T10:12:17.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.286+0000 7f63a0600640 1 -- start start 2024-04-25T10:12:17.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.288+0000 7f63a0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 0x7f63980cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.288+0000 7f63a0600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6398150b90 0x7f6398151020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.288+0000 7f63a0600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f639815e6e0 con 0x7f639815b1c0 2024-04-25T10:12:17.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.288+0000 7f63a0600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f639815b620 con 0x7f6398150b90 2024-04-25T10:12:17.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.288+0000 7f63a0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398151560 con 0x7f63980cd4c0 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 0x7f63980cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639e600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6398150b90 0x7f6398151020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 0x7f63980cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:35294/0 (socket says 172.21.15.165:35294) 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 -- 172.21.15.165:0/3946546025 learned_addr learned my addr 172.21.15.165:0/3946546025 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 -- 172.21.15.165:0/3946546025 >> v2:172.21.15.172:3300/0 conn(0x7f6398150b90 msgr2=0x7f6398151020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 --2- 172.21.15.165:0/3946546025 >> v2:172.21.15.172:3300/0 conn(0x7f6398150b90 0x7f6398151020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 -- 172.21.15.165:0/3946546025 >> v1:172.21.15.172:6789/0 conn(0x7f639815b1c0 legacy=0x7f639815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.289+0000 7f639dc00640 1 -- 172.21.15.165:0/3946546025 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639815e9c0 con 0x7f63980cd4c0 2024-04-25T10:12:17.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.291+0000 7f639dc00640 1 --2- 172.21.15.165:0/3946546025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 0x7f63980cd8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f639000bb00 tx=0x7f639002c6e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69334255a3c4a15e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.291+0000 7f6397e00640 1 -- 172.21.15.165:0/3946546025 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639003a070 con 0x7f63980cd4c0 2024-04-25T10:12:17.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.291+0000 7f6397e00640 1 -- 172.21.15.165:0/3946546025 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390032480 con 0x7f63980cd4c0 2024-04-25T10:12:17.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.291+0000 7f6397e00640 1 -- 172.21.15.165:0/3946546025 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390032850 con 0x7f63980cd4c0 2024-04-25T10:12:17.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 -- 172.21.15.165:0/3946546025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 msgr2=0x7f63980cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 --2- 172.21.15.165:0/3946546025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 0x7f63980cd8a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f639000bb00 tx=0x7f639002c6e0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 -- 172.21.15.165:0/3946546025 shutdown_connections 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 --2- 172.21.15.165:0/3946546025 >> v2:172.21.15.172:3300/0 conn(0x7f6398150b90 0x7f6398151020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 --2- 172.21.15.165:0/3946546025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63980cd4c0 0x7f63980cd8a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 -- 172.21.15.165:0/3946546025 >> 172.21.15.165:0/3946546025 conn(0x7f639806abe0 msgr2=0x7f639806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 -- 172.21.15.165:0/3946546025 shutdown_connections 2024-04-25T10:12:17.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.292+0000 7f63a0600640 1 -- 172.21.15.165:0/3946546025 wait complete. 2024-04-25T10:12:17.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.294+0000 7f63a0600640 1 Processor -- start 2024-04-25T10:12:17.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.294+0000 7f63a0600640 1 -- start start 2024-04-25T10:12:17.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f63a0600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 0x7f63981f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f63a0600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6398150b90 0x7f6398152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f63a0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f639815b1c0 0x7f63980add60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f63a0600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639815e370 con 0x7f63980cd4c0 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f63a0600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f639815a670 con 0x7f6398150b90 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f63a0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f639806eca0 con 0x7f639815b1c0 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f639dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 0x7f63981f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f639dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 0x7f63981f3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48104/0 (socket says 172.21.15.165:48104) 2024-04-25T10:12:17.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f639dc00640 1 -- 172.21.15.165:0/3673031970 learned_addr learned my addr 172.21.15.165:0/3673031970 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:17.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f639e600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f639815b1c0 0x7f63980add60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.295+0000 7f639d200640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6398150b90 0x7f6398152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f639dc00640 1 -- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f639815b1c0 msgr2=0x7f63980add60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f639dc00640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f639815b1c0 0x7f63980add60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f639dc00640 1 -- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6398150b90 msgr2=0x7f6398152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f639dc00640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6398150b90 0x7f6398152290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f639dc00640 1 -- 172.21.15.165:0/3673031970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63981f3d80 con 0x7f63980cd4c0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f639dc00640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 0x7f63981f3840 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6390038040 tx=0x7f63900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639003a070 con 0x7f63980cd4c0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6390035860 con 0x7f63980cd4c0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398153f40 con 0x7f63980cd4c0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390035c50 con 0x7f63980cd4c0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.296+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6398154500 con 0x7f63980cd4c0 2024-04-25T10:12:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.297+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6398073a00 con 0x7f63980cd4c0 2024-04-25T10:12:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.298+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6390047060 con 0x7f63980cd4c0 2024-04-25T10:12:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.299+0000 7f6395600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.299+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f63900aa910 con 0x7f63980cd4c0 2024-04-25T10:12:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.300+0000 7f639d200640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.300+0000 7f639d200640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6384007fd0 tx=0x7f6384009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.304+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6390033310 con 0x7f63980cd4c0 2024-04-25T10:12:17.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.496+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63980b0500 con 0x7f63980cd4c0 2024-04-25T10:12:17.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.497+0000 7f6395600640 1 -- 172.21.15.165:0/3673031970 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6390079580 con 0x7f63980cd4c0 2024-04-25T10:12:17.502 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 msgr2=0x7f637806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6384007fd0 tx=0x7f6384009040 comp rx=0 tx=0).stop 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 msgr2=0x7f63981f3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 0x7f63981f3840 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f6390038040 tx=0x7f63900046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 shutdown_connections 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f637806b4d0 0x7f637806d990 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f639815b1c0 0x7f63980add60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6398150b90 0x7f6398152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 --2- 172.21.15.165:0/3673031970 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63980cd4c0 0x7f63981f3840 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 >> 172.21.15.165:0/3673031970 conn(0x7f639806abe0 msgr2=0x7f63980b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 shutdown_connections 2024-04-25T10:12:17.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.499+0000 7f63a0600640 1 -- 172.21.15.165:0/3673031970 wait complete. 2024-04-25T10:12:17.522 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.720+0000 7f89dfe00640 1 Processor -- start 2024-04-25T10:12:17.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.720+0000 7f89dfe00640 1 -- start start 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.722+0000 7f89dfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89e013d9f0 0x7f89e013df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.722+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e00b0730 0x7f89e00b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.722+0000 7f89dfe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89e006d390 con 0x7f89e006e150 2024-04-25T10:12:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.722+0000 7f89dfe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89e00b10d0 con 0x7f89e013d9f0 2024-04-25T10:12:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.722+0000 7f89dfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e006d670 con 0x7f89e00b0730 2024-04-25T10:12:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89dea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89e013d9f0 0x7f89e013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89d7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e00b0730 0x7f89e00b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89dea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f89e013d9f0 0x7f89e013df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:35532/0 (socket says 172.21.15.165:35532) 2024-04-25T10:12:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89dea00640 1 -- 172.21.15.165:0/1776126497 learned_addr learned my addr 172.21.15.165:0/1776126497 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:17.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89d7e00640 1 -- 172.21.15.165:0/1776126497 >> v2:172.21.15.172:3300/0 conn(0x7f89e013d9f0 msgr2=0x7f89e013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:17.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89d7e00640 1 --2- 172.21.15.165:0/1776126497 >> v2:172.21.15.172:3300/0 conn(0x7f89e013d9f0 0x7f89e013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89d7e00640 1 -- 172.21.15.165:0/1776126497 >> v1:172.21.15.172:6789/0 conn(0x7f89e006e150 legacy=0x7f89e006e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.723+0000 7f89d7e00640 1 -- 172.21.15.165:0/1776126497 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e006d950 con 0x7f89e00b0730 2024-04-25T10:12:17.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.725+0000 7f89d7e00640 1 --2- 172.21.15.165:0/1776126497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e00b0730 0x7f89e00b0b90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f89c800ba40 tx=0x7f89c802c730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a3819c9698e263 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.725+0000 7f89de000640 1 -- 172.21.15.165:0/1776126497 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8031710 con 0x7f89e00b0730 2024-04-25T10:12:17.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.725+0000 7f89de000640 1 -- 172.21.15.165:0/1776126497 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89c80318b0 con 0x7f89e00b0730 2024-04-25T10:12:17.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.725+0000 7f89de000640 1 -- 172.21.15.165:0/1776126497 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89c8031c80 con 0x7f89e00b0730 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 -- 172.21.15.165:0/1776126497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e00b0730 msgr2=0x7f89e00b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 --2- 172.21.15.165:0/1776126497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e00b0730 0x7f89e00b0b90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f89c800ba40 tx=0x7f89c802c730 comp rx=0 tx=0).stop 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 -- 172.21.15.165:0/1776126497 shutdown_connections 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 --2- 172.21.15.165:0/1776126497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e00b0730 0x7f89e00b0b90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 --2- 172.21.15.165:0/1776126497 >> v2:172.21.15.172:3300/0 conn(0x7f89e013d9f0 0x7f89e013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 -- 172.21.15.165:0/1776126497 >> 172.21.15.165:0/1776126497 conn(0x7f89e0061cc0 msgr2=0x7f89e006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 -- 172.21.15.165:0/1776126497 shutdown_connections 2024-04-25T10:12:17.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.726+0000 7f89dfe00640 1 -- 172.21.15.165:0/1776126497 wait complete. 2024-04-25T10:12:17.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.728+0000 7f89dfe00640 1 Processor -- start 2024-04-25T10:12:17.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.728+0000 7f89dfe00640 1 -- start start 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e006e150 0x7f89e0137c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e00b0730 0x7f89e0138170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89dfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 0x7f89e013ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89dfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e013e5f0 con 0x7f89e013d9f0 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89dfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89e013e470 con 0x7f89e00b0730 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89dfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89e00ce880 con 0x7f89e006e150 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 0x7f89e013ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89df400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 0x7f89e013ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48116/0 (socket says 172.21.15.165:48116) 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89df400640 1 -- 172.21.15.165:0/3154154187 learned_addr learned my addr 172.21.15.165:0/3154154187 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:17.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.729+0000 7f89d7e00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e00b0730 0x7f89e0138170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89dea00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e006e150 0x7f89e0137c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89df400640 1 -- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e006e150 msgr2=0x7f89e0137c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89df400640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e006e150 0x7f89e0137c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89df400640 1 -- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e00b0730 msgr2=0x7f89e0138170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89df400640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e00b0730 0x7f89e0138170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89df400640 1 -- 172.21.15.165:0/3154154187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e01f6f40 con 0x7f89e013d9f0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89df400640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 0x7f89e013ad80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c7a0 tx=0x7f89d800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d8003b70 con 0x7f89e013d9f0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89d8003d30 con 0x7f89e013d9f0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89d800aa90 con 0x7f89e013d9f0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.730+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89e01f7220 con 0x7f89e013d9f0 2024-04-25T10:12:17.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.731+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89e01f77e0 con 0x7f89e013d9f0 2024-04-25T10:12:17.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.732+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89e0073990 con 0x7f89e013d9f0 2024-04-25T10:12:17.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.733+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f89d800ac30 con 0x7f89e013d9f0 2024-04-25T10:12:17.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.734+0000 7f89d6a00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f89bc06b4d0 0x7f89bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:17.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.734+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f89d808a350 con 0x7f89e013d9f0 2024-04-25T10:12:17.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.734+0000 7f89dea00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f89bc06b4d0 0x7f89bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:17.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.735+0000 7f89dea00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f89bc06b4d0 0x7f89bc06d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f89d00099e0 tx=0x7f89d0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:17.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.739+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f89d8058fd0 con 0x7f89e013d9f0 2024-04-25T10:12:17.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.935+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f89e013bb00 con 0x7f89e013d9f0 2024-04-25T10:12:17.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.936+0000 7f89d6a00640 1 -- 172.21.15.165:0/3154154187 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7f89d80588d0 con 0x7f89e013d9f0 2024-04-25T10:12:17.942 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f89bc06b4d0 msgr2=0x7f89bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f89bc06b4d0 0x7f89bc06d990 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f89d00099e0 tx=0x7f89d0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 msgr2=0x7f89e013ad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 0x7f89e013ad80 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f89d800c7a0 tx=0x7f89d800cc60 comp rx=0 tx=0).stop 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 shutdown_connections 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f89bc06b4d0 0x7f89bc06d990 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f89e013d9f0 0x7f89e013ad80 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f89e00b0730 0x7f89e0138170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 --2- 172.21.15.165:0/3154154187 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f89e006e150 0x7f89e0137c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 >> 172.21.15.165:0/3154154187 conn(0x7f89e0061cc0 msgr2=0x7f89e0071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:17.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 shutdown_connections 2024-04-25T10:12:17.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:17.939+0000 7f89dfe00640 1 -- 172.21.15.165:0/3154154187 wait complete. 2024-04-25T10:12:17.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:12:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.156+0000 7f42af800640 1 Processor -- start 2024-04-25T10:12:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.156+0000 7f42af800640 1 -- start start 2024-04-25T10:12:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.157+0000 7f42af800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42a80cc4f0 0x7f42a80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42af800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f42a80cdb20 0x7f42a80cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42af800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a8072280 con 0x7f42a815b270 2024-04-25T10:12:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42af800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a8071270 con 0x7f42a80cdb20 2024-04-25T10:12:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42af800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42a815bd80 con 0x7f42a80cc4f0 2024-04-25T10:12:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42a7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f42a80cdb20 0x7f42a80cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42a7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f42a80cdb20 0x7f42a80cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48128/0 (socket says 172.21.15.165:48128) 2024-04-25T10:12:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42a7e00640 1 -- 172.21.15.165:0/1770360523 learned_addr learned my addr 172.21.15.165:0/1770360523 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42ace00640 1 --2- 172.21.15.165:0/1770360523 >> v2:172.21.15.172:3300/0 conn(0x7f42a80cc4f0 0x7f42a80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.158+0000 7f42a7e00640 1 -- 172.21.15.165:0/1770360523 >> v2:172.21.15.172:3300/0 conn(0x7f42a80cc4f0 msgr2=0x7f42a80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.159+0000 7f42a7e00640 1 --2- 172.21.15.165:0/1770360523 >> v2:172.21.15.172:3300/0 conn(0x7f42a80cc4f0 0x7f42a80cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.159+0000 7f42a7e00640 1 -- 172.21.15.165:0/1770360523 >> v1:172.21.15.165:6789/0 conn(0x7f42a815b270 legacy=0x7f42a815b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.159+0000 7f42a7e00640 1 -- 172.21.15.165:0/1770360523 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80714c0 con 0x7f42a80cdb20 2024-04-25T10:12:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.160+0000 7f42a7e00640 1 --2- 172.21.15.165:0/1770360523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a80cdb20 0x7f42a80cdfb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f42a80cd290 tx=0x7f429402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b46bd62455c35a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.160+0000 7f42a7400640 1 -- 172.21.15.165:0/1770360523 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294037070 con 0x7f42a80cdb20 2024-04-25T10:12:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.160+0000 7f42a7400640 1 -- 172.21.15.165:0/1770360523 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4294032440 con 0x7f42a80cdb20 2024-04-25T10:12:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.160+0000 7f42a7400640 1 -- 172.21.15.165:0/1770360523 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4294032810 con 0x7f42a80cdb20 2024-04-25T10:12:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.160+0000 7f42af800640 1 -- 172.21.15.165:0/1770360523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a80cdb20 msgr2=0x7f42a80cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.160+0000 7f42af800640 1 --2- 172.21.15.165:0/1770360523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a80cdb20 0x7f42a80cdfb0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7f42a80cd290 tx=0x7f429402a270 comp rx=0 tx=0).stop 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.161+0000 7f42af800640 1 -- 172.21.15.165:0/1770360523 shutdown_connections 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.161+0000 7f42af800640 1 --2- 172.21.15.165:0/1770360523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a80cdb20 0x7f42a80cdfb0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.161+0000 7f42af800640 1 --2- 172.21.15.165:0/1770360523 >> v2:172.21.15.172:3300/0 conn(0x7f42a80cc4f0 0x7f42a80cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.161+0000 7f42af800640 1 -- 172.21.15.165:0/1770360523 >> 172.21.15.165:0/1770360523 conn(0x7f42a806abe0 msgr2=0x7f42a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.161+0000 7f42af800640 1 -- 172.21.15.165:0/1770360523 shutdown_connections 2024-04-25T10:12:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.161+0000 7f42af800640 1 -- 172.21.15.165:0/1770360523 wait complete. 2024-04-25T10:12:18.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.162+0000 7f42af800640 1 Processor -- start 2024-04-25T10:12:18.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.163+0000 7f42af800640 1 -- start start 2024-04-25T10:12:18.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.163+0000 7f42af800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a80cc4f0 0x7f42a8155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42af800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a80cdb20 0x7f42a8154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42af800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 0x7f42a81472c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42af800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8071d60 con 0x7f42a815b270 2024-04-25T10:12:18.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42af800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a8071a80 con 0x7f42a80cc4f0 2024-04-25T10:12:18.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42af800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80ae500 con 0x7f42a80cdb20 2024-04-25T10:12:18.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 0x7f42a81472c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 0x7f42a81472c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48138/0 (socket says 172.21.15.165:48138) 2024-04-25T10:12:18.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 -- 172.21.15.165:0/3349888402 learned_addr learned my addr 172.21.15.165:0/3349888402 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:18.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ace00640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a80cc4f0 0x7f42a8155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 -- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a80cdb20 msgr2=0x7f42a8154790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42a7e00640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a80cdb20 0x7f42a8154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a80cdb20 0x7f42a8154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 -- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a80cc4f0 msgr2=0x7f42a8155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a80cc4f0 0x7f42a8155590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ad800640 1 -- 172.21.15.165:0/3349888402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80af850 con 0x7f42a815b270 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.164+0000 7f42ace00640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a80cc4f0 0x7f42a8155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.165+0000 7f42ad800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 0x7f42a81472c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f42a000c7a0 tx=0x7f42a000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.165+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0003b70 con 0x7f42a815b270 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.165+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a0003d30 con 0x7f42a815b270 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.165+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a80afb30 con 0x7f42a815b270 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.165+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a000aa90 con 0x7f42a815b270 2024-04-25T10:12:18.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.165+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a80b00f0 con 0x7f42a815b270 2024-04-25T10:12:18.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.166+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a80b08f0 con 0x7f42a815b270 2024-04-25T10:12:18.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.167+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42a000ac30 con 0x7f42a815b270 2024-04-25T10:12:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.169+0000 7f42a4c00640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f428806b4d0 0x7f428806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.169+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f42a008a340 con 0x7f42a815b270 2024-04-25T10:12:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.169+0000 7f42ace00640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f428806b4d0 0x7f428806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:18.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.170+0000 7f42ace00640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f428806b4d0 0x7f428806d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f429c006fd0 tx=0x7f429c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:18.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.173+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42a80b08f0 con 0x7f42a815b270 2024-04-25T10:12:18.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.382+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f42a8073a00 con 0x7f42a815b270 2024-04-25T10:12:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.383+0000 7f42a4c00640 1 -- 172.21.15.165:0/3349888402 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v100) v1 ==== 80+0+0 (secure 0 0 0) 0x7f42a0058fc0 con 0x7f42a815b270 2024-04-25T10:12:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:12:18.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f428806b4d0 msgr2=0x7f428806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f428806b4d0 0x7f428806d990 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f429c006fd0 tx=0x7f429c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:18.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 msgr2=0x7f42a81472c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:18.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 0x7f42a81472c0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f42a000c7a0 tx=0x7f42a000cc60 comp rx=0 tx=0).stop 2024-04-25T10:12:18.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 shutdown_connections 2024-04-25T10:12:18.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f428806b4d0 0x7f428806d990 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42a815b270 0x7f42a81472c0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42a80cdb20 0x7f42a8154790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 --2- 172.21.15.165:0/3349888402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42a80cc4f0 0x7f42a8155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:18.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 >> 172.21.15.165:0/3349888402 conn(0x7f42a806abe0 msgr2=0x7f42a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:18.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 shutdown_connections 2024-04-25T10:12:18.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:18.386+0000 7f42af800640 1 -- 172.21.15.165:0/3349888402 wait complete. 2024-04-25T10:12:18.410 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:12:18.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:18.410 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:19.204 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:12:19.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:19.205 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:12:19.244 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:12:19.245 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:19.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:19.287 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:12:19.287 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:19.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:19.327 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:12:19.327 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:19.381 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:19.381 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:19.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.574+0000 7f3b21600640 1 Processor -- start 2024-04-25T10:12:19.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.574+0000 7f3b21600640 1 -- start start 2024-04-25T10:12:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b21600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b21600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3b1c06eb10 0x7f3b1c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b21600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b1c144b30 con 0x7f3b1c0cee60 2024-04-25T10:12:19.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b21600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b1c0712d0 con 0x7f3b1c06eb10 2024-04-25T10:12:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b21600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c0715b0 con 0x7f3b1c144750 2024-04-25T10:12:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3b1c06eb10 0x7f3b1c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3b1c06eb10 0x7f3b1c0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48148/0 (socket says 172.21.15.165:48148) 2024-04-25T10:12:19.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 -- 172.21.15.165:0/612173566 learned_addr learned my addr 172.21.15.165:0/612173566 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b1aa00640 1 --2- 172.21.15.165:0/612173566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 -- 172.21.15.165:0/612173566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 msgr2=0x7f3b1c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 --2- 172.21.15.165:0/612173566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 -- 172.21.15.165:0/612173566 >> v1:172.21.15.172:6789/0 conn(0x7f3b1c0cee60 legacy=0x7f3b1c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.575+0000 7f3b12000640 1 -- 172.21.15.165:0/612173566 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c071890 con 0x7f3b1c06eb10 2024-04-25T10:12:19.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.576+0000 7f3b12000640 1 --2- 172.21.15.165:0/612173566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c06eb10 0x7f3b1c0ce920 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009870 tx=0x7f3b0402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf1032e724be50a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b1a000640 1 -- 172.21.15.165:0/612173566 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04037070 con 0x7f3b1c06eb10 2024-04-25T10:12:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b1a000640 1 -- 172.21.15.165:0/612173566 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b04032440 con 0x7f3b1c06eb10 2024-04-25T10:12:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b1a000640 1 -- 172.21.15.165:0/612173566 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04032810 con 0x7f3b1c06eb10 2024-04-25T10:12:19.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b21600640 1 -- 172.21.15.165:0/612173566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c06eb10 msgr2=0x7f3b1c0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b21600640 1 --2- 172.21.15.165:0/612173566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c06eb10 0x7f3b1c0ce920 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f3b04009870 tx=0x7f3b0402a210 comp rx=0 tx=0).stop 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b21600640 1 -- 172.21.15.165:0/612173566 shutdown_connections 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b21600640 1 --2- 172.21.15.165:0/612173566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c06eb10 0x7f3b1c0ce920 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b21600640 1 --2- 172.21.15.165:0/612173566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.577+0000 7f3b21600640 1 -- 172.21.15.165:0/612173566 >> 172.21.15.165:0/612173566 conn(0x7f3b1c06abe0 msgr2=0x7f3b1c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.578+0000 7f3b21600640 1 -- 172.21.15.165:0/612173566 shutdown_connections 2024-04-25T10:12:19.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.578+0000 7f3b21600640 1 -- 172.21.15.165:0/612173566 wait complete. 2024-04-25T10:12:19.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.579+0000 7f3b21600640 1 Processor -- start 2024-04-25T10:12:19.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.579+0000 7f3b21600640 1 -- start start 2024-04-25T10:12:19.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.580+0000 7f3b21600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b1c06eb10 0x7f3b1c13e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.580+0000 7f3b21600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 0x7f3b1c15fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.580+0000 7f3b21600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.580+0000 7f3b21600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c145450 con 0x7f3b1c0cee60 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.580+0000 7f3b21600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c06f020 con 0x7f3b1c06eb10 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.580+0000 7f3b21600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b1c1455d0 con 0x7f3b1c144750 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 0x7f3b1c15fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 0x7f3b1c15fa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48160/0 (socket says 172.21.15.165:48160) 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 -- 172.21.15.165:0/140664036 learned_addr learned my addr 172.21.15.165:0/140664036 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 -- 172.21.15.165:0/140664036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 msgr2=0x7f3b1c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 -- 172.21.15.165:0/140664036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b1c06eb10 msgr2=0x7f3b1c13e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b1c06eb10 0x7f3b1c13e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 -- 172.21.15.165:0/140664036 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c0b4720 con 0x7f3b1c0cee60 2024-04-25T10:12:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.581+0000 7f3b12000640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 0x7f3b1c15fa90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.582+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04041020 con 0x7f3b1c0cee60 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.582+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b040306c0 con 0x7f3b1c0cee60 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.582+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b04037040 con 0x7f3b1c0cee60 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.582+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b1c1364d0 con 0x7f3b1c0cee60 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.582+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b1c0b3420 con 0x7f3b1c0cee60 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.583+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1c073990 con 0x7f3b1c0cee60 2024-04-25T10:12:19.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.584+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3b04044020 con 0x7f3b1c0cee60 2024-04-25T10:12:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.585+0000 7f3b13400640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3af806b4d0 0x7f3af806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.585+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f3b040aacc0 con 0x7f3b1c0cee60 2024-04-25T10:12:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.585+0000 7f3b1aa00640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3af806b4d0 0x7f3af806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.586+0000 7f3b1aa00640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3af806b4d0 0x7f3af806d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0070b0 tx=0x7f3b0c007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:19.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.590+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3b040b1080 con 0x7f3b1c0cee60 2024-04-25T10:12:19.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.821+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3b1c143f70 con 0x7f3b1c0cee60 2024-04-25T10:12:19.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.822+0000 7f3b13400640 1 -- 172.21.15.165:0/140664036 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f3b0404b030 con 0x7f3b1c0cee60 2024-04-25T10:12:19.828 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:19.828 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:19.828 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.824+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3af806b4d0 msgr2=0x7f3af806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.824+0000 7f3b21600640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3af806b4d0 0x7f3af806d990 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c0070b0 tx=0x7f3b0c007040 comp rx=0 tx=0).stop 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.824+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 msgr2=0x7f3b1c15fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.824+0000 7f3b21600640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 0x7f3b1c15fa90 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f3b04000c00 tx=0x7f3b040046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 shutdown_connections 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3af806b4d0 0x7f3af806d990 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3b1c144750 0x7f3b1c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3b1c0cee60 0x7f3b1c15fa90 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 --2- 172.21.15.165:0/140664036 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3b1c06eb10 0x7f3b1c13e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 >> 172.21.15.165:0/140664036 conn(0x7f3b1c06abe0 msgr2=0x7f3b1c143810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 shutdown_connections 2024-04-25T10:12:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:19.825+0000 7f3b21600640 1 -- 172.21.15.165:0/140664036 wait complete. 2024-04-25T10:12:19.848 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 Processor -- start 2024-04-25T10:12:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 -- start start 2024-04-25T10:12:20.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f712015d900 0x7f71200ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71200b2980 0x7f71200cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71200b6a40 con 0x7f71200b23c0 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71200cef60 con 0x7f71200b2980 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.041+0000 7f7125000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71200b7050 con 0x7f712015d900 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71200b2980 0x7f71200cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71200b2980 0x7f71200cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48168/0 (socket says 172.21.15.165:48168) 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 -- 172.21.15.165:0/271640598 learned_addr learned my addr 172.21.15.165:0/271640598 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f711f400640 1 --2- 172.21.15.165:0/271640598 >> v2:172.21.15.172:3300/0 conn(0x7f712015d900 0x7f71200ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 -- 172.21.15.165:0/271640598 >> v2:172.21.15.172:3300/0 conn(0x7f712015d900 msgr2=0x7f71200ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 --2- 172.21.15.165:0/271640598 >> v2:172.21.15.172:3300/0 conn(0x7f712015d900 0x7f71200ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 -- 172.21.15.165:0/271640598 >> v1:172.21.15.172:6789/0 conn(0x7f71200b23c0 legacy=0x7f71200b74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f7116a00640 1 -- 172.21.15.165:0/271640598 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200b9470 con 0x7f71200b2980 2024-04-25T10:12:20.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.042+0000 7f711f400640 1 --2- 172.21.15.165:0/271640598 >> v2:172.21.15.172:3300/0 conn(0x7f712015d900 0x7f71200ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.043+0000 7f7116a00640 1 --2- 172.21.15.165:0/271640598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71200b2980 0x7f71200cea20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f71201613d0 tx=0x7f710802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db21da052a5d475b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f711ea00640 1 -- 172.21.15.165:0/271640598 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108037070 con 0x7f71200b2980 2024-04-25T10:12:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f711ea00640 1 -- 172.21.15.165:0/271640598 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7108032440 con 0x7f71200b2980 2024-04-25T10:12:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f711ea00640 1 -- 172.21.15.165:0/271640598 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108032810 con 0x7f71200b2980 2024-04-25T10:12:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f7125000640 1 -- 172.21.15.165:0/271640598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71200b2980 msgr2=0x7f71200cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f7125000640 1 --2- 172.21.15.165:0/271640598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71200b2980 0x7f71200cea20 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f71201613d0 tx=0x7f710802a200 comp rx=0 tx=0).stop 2024-04-25T10:12:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f7125000640 1 -- 172.21.15.165:0/271640598 shutdown_connections 2024-04-25T10:12:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f7125000640 1 --2- 172.21.15.165:0/271640598 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71200b2980 0x7f71200cea20 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.044+0000 7f7125000640 1 --2- 172.21.15.165:0/271640598 >> v2:172.21.15.172:3300/0 conn(0x7f712015d900 0x7f71200ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.045+0000 7f7125000640 1 -- 172.21.15.165:0/271640598 >> 172.21.15.165:0/271640598 conn(0x7f71200619f0 msgr2=0x7f7120061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.045+0000 7f7125000640 1 -- 172.21.15.165:0/271640598 shutdown_connections 2024-04-25T10:12:20.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.045+0000 7f7125000640 1 -- 172.21.15.165:0/271640598 wait complete. 2024-04-25T10:12:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.046+0000 7f7125000640 1 Processor -- start 2024-04-25T10:12:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.046+0000 7f7125000640 1 -- start start 2024-04-25T10:12:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.047+0000 7f7125000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71200b23c0 0x7f7120159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.047+0000 7f7125000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71200b2980 0x7f71201574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.047+0000 7f7125000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 0x7f712015e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.047+0000 7f7125000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712015dce0 con 0x7f712015d900 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.047+0000 7f7125000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71200b9750 con 0x7f71200b2980 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.047+0000 7f7125000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71200b9090 con 0x7f71200b23c0 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 0x7f712015e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 0x7f712015e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48180/0 (socket says 172.21.15.165:48180) 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 -- 172.21.15.165:0/1707281476 learned_addr learned my addr 172.21.15.165:0/1707281476 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f7116a00640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71200b2980 0x7f71201574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 -- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71200b23c0 msgr2=0x7f7120159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711f400640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71200b23c0 0x7f7120159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71200b23c0 0x7f7120159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 -- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71200b2980 msgr2=0x7f71201574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71200b2980 0x7f71201574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 -- 172.21.15.165:0/1707281476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7120154d40 con 0x7f712015d900 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f7116a00640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71200b2980 0x7f71201574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.048+0000 7f711fe00640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 0x7f712015e830 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f711800b970 tx=0x7f711800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.049+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118019070 con 0x7f712015d900 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.049+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711800cb40 con 0x7f712015d900 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.049+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7118014440 con 0x7f712015d900 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.049+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7120155020 con 0x7f712015d900 2024-04-25T10:12:20.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.049+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71201555e0 con 0x7f712015d900 2024-04-25T10:12:20.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.050+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7120073990 con 0x7f712015d900 2024-04-25T10:12:20.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.051+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f711801c070 con 0x7f712015d900 2024-04-25T10:12:20.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.052+0000 7f7117e00640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.053+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7118089fe0 con 0x7f712015d900 2024-04-25T10:12:20.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.053+0000 7f711f400640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.053+0000 7f711f400640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7110008850 tx=0x7f7110008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.057+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7118058c90 con 0x7f712015d900 2024-04-25T10:12:20.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.255+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71200d42d0 con 0x7f712015d900 2024-04-25T10:12:20.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.258+0000 7f7117e00640 1 -- 172.21.15.165:0/1707281476 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17114 (secure 0 0 0) 0x7f7118058590 con 0x7f712015d900 2024-04-25T10:12:20.264 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:20.264 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":100,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:47.812589+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":5}]},{"pgid":"14.a","mappings":[{"from":1,"to":2}]},{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:20.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70fc06b4d0 msgr2=0x7f70fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70fc06b4d0 0x7f70fc06d990 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f7110008850 tx=0x7f7110008040 comp rx=0 tx=0).stop 2024-04-25T10:12:20.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 msgr2=0x7f712015e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 0x7f712015e830 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f711800b970 tx=0x7f711800be30 comp rx=0 tx=0).stop 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 shutdown_connections 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70fc06b4d0 0x7f70fc06d990 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f712015d900 0x7f712015e830 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71200b2980 0x7f71201574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 --2- 172.21.15.165:0/1707281476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71200b23c0 0x7f7120159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.261+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 >> 172.21.15.165:0/1707281476 conn(0x7f71200619f0 msgr2=0x7f71200d2b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.262+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 shutdown_connections 2024-04-25T10:12:20.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.262+0000 7f7125000640 1 -- 172.21.15.165:0/1707281476 wait complete. 2024-04-25T10:12:20.286 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:11:20.638187+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:11:21.928273+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '98', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:20.286 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.485+0000 7f9d3e400640 1 Processor -- start 2024-04-25T10:12:20.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.485+0000 7f9d3e400640 1 -- start start 2024-04-25T10:12:20.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3e400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9d3815cfc0 0x7f9d380c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3e400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d38145520 con 0x7f9d380ceb20 2024-04-25T10:12:20.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3e400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d38071270 con 0x7f9d38145140 2024-04-25T10:12:20.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3e400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d38071550 con 0x7f9d3815cfc0 2024-04-25T10:12:20.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3da00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9d3815cfc0 0x7f9d380c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d37e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9d380ceb20 0x7f9d380c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53434/0 (socket says 172.21.15.165:53434) 2024-04-25T10:12:20.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d37e00640 1 -- 172.21.15.165:0/1482426312 learned_addr learned my addr 172.21.15.165:0/1482426312 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:20.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3da00640 1 -- 172.21.15.165:0/1482426312 >> v1:172.21.15.172:6789/0 conn(0x7f9d38145140 legacy=0x7f9d380ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3da00640 1 -- 172.21.15.165:0/1482426312 >> v1:172.21.15.165:6789/0 conn(0x7f9d380ceb20 legacy=0x7f9d380c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d3da00640 1 -- 172.21.15.165:0/1482426312 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38071830 con 0x7f9d3815cfc0 2024-04-25T10:12:20.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.488+0000 7f9d37400640 1 -- 172.21.15.165:0/1482426312 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105220545 0 0) 0x7f9d38145520 con 0x7f9d380ceb20 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.489+0000 7f9d3da00640 1 --2- 172.21.15.165:0/1482426312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d3815cfc0 0x7f9d380c5200 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009d90 tx=0x7f9d3002a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=916378efeda6260d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.489+0000 7f9d37400640 1 -- 172.21.15.165:0/1482426312 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30031600 con 0x7f9d3815cfc0 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.489+0000 7f9d37400640 1 -- 172.21.15.165:0/1482426312 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d300317a0 con 0x7f9d3815cfc0 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.489+0000 7f9d37400640 1 -- 172.21.15.165:0/1482426312 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d30031b70 con 0x7f9d3815cfc0 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.489+0000 7f9d3e400640 1 -- 172.21.15.165:0/1482426312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d3815cfc0 msgr2=0x7f9d380c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.489+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1482426312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d3815cfc0 0x7f9d380c5200 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009d90 tx=0x7f9d3002a610 comp rx=0 tx=0).stop 2024-04-25T10:12:20.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.490+0000 7f9d3e400640 1 -- 172.21.15.165:0/1482426312 shutdown_connections 2024-04-25T10:12:20.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.490+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1482426312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d3815cfc0 0x7f9d380c5200 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.490+0000 7f9d3e400640 1 -- 172.21.15.165:0/1482426312 >> 172.21.15.165:0/1482426312 conn(0x7f9d38061cc0 msgr2=0x7f9d3806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.490+0000 7f9d3e400640 1 -- 172.21.15.165:0/1482426312 shutdown_connections 2024-04-25T10:12:20.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.490+0000 7f9d3e400640 1 -- 172.21.15.165:0/1482426312 wait complete. 2024-04-25T10:12:20.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.491+0000 7f9d3e400640 1 Processor -- start 2024-04-25T10:12:20.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.491+0000 7f9d3e400640 1 -- start start 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.492+0000 7f9d3e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 0x7f9d380cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.492+0000 7f9d3e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d38145140 0x7f9d3813ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.492+0000 7f9d3e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d3815cfc0 0x7f9d380b2620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.492+0000 7f9d3e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381665b0 con 0x7f9d380ceb20 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.492+0000 7f9d3e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38166430 con 0x7f9d3815cfc0 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.492+0000 7f9d3e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d38166730 con 0x7f9d38145140 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 0x7f9d380cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 0x7f9d380cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48194/0 (socket says 172.21.15.165:48194) 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 -- 172.21.15.165:0/1235926070 learned_addr learned my addr 172.21.15.165:0/1235926070 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3da00640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d3815cfc0 0x7f9d380b2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 -- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d38145140 msgr2=0x7f9d3813ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d38145140 0x7f9d3813ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 -- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d3815cfc0 msgr2=0x7f9d380b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d3815cfc0 0x7f9d380b2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 -- 172.21.15.165:0/1235926070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3813d520 con 0x7f9d380ceb20 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.493+0000 7f9d3d000640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 0x7f9d380cc8d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c430 tx=0x7f9d2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.494+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28011790 con 0x7f9d380ceb20 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.494+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d28017070 con 0x7f9d380ceb20 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.494+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d3813d800 con 0x7f9d380ceb20 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.494+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d28011bb0 con 0x7f9d380ceb20 2024-04-25T10:12:20.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.494+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d3813ddc0 con 0x7f9d380ceb20 2024-04-25T10:12:20.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.495+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d38073990 con 0x7f9d380ceb20 2024-04-25T10:12:20.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.496+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d2801c460 con 0x7f9d380ceb20 2024-04-25T10:12:20.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.497+0000 7f9d34c00640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d1406b4d0 0x7f9d1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.497+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9d2808a340 con 0x7f9d380ceb20 2024-04-25T10:12:20.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.498+0000 7f9d37e00640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d1406b4d0 0x7f9d1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.499+0000 7f9d37e00640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d1406b4d0 0x7f9d1406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007fd0 tx=0x7f9d20007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.504+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d28058fb0 con 0x7f9d380ceb20 2024-04-25T10:12:20.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.697+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9d38142890 con 0x7f9d380ceb20 2024-04-25T10:12:20.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.699+0000 7f9d34c00640 1 -- 172.21.15.165:0/1235926070 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9d280588b0 con 0x7f9d380ceb20 2024-04-25T10:12:20.704 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:12:20.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.700+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d1406b4d0 msgr2=0x7f9d1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.700+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d1406b4d0 0x7f9d1406d990 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f9d20007fd0 tx=0x7f9d20007920 comp rx=0 tx=0).stop 2024-04-25T10:12:20.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.700+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 msgr2=0x7f9d380cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.700+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 0x7f9d380cc8d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f9d2800c430 tx=0x7f9d2800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 shutdown_connections 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d1406b4d0 0x7f9d1406d990 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d3815cfc0 0x7f9d380b2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d38145140 0x7f9d3813ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 --2- 172.21.15.165:0/1235926070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d380ceb20 0x7f9d380cc8d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 >> 172.21.15.165:0/1235926070 conn(0x7f9d38061cc0 msgr2=0x7f9d3815ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 shutdown_connections 2024-04-25T10:12:20.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.702+0000 7f9d3e400640 1 -- 172.21.15.165:0/1235926070 wait complete. 2024-04-25T10:12:20.726 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:12:20.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.919+0000 7f78a6600640 1 Processor -- start 2024-04-25T10:12:20.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.919+0000 7f78a6600640 1 -- start start 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.920+0000 7f78a6600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f78a0161fc0 0x7f78a01623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.920+0000 7f78a6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 0x7f78a006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.920+0000 7f78a6600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a01628e0 con 0x7f78a006ea90 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.920+0000 7f78a6600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a0071b90 con 0x7f78a0161fc0 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.920+0000 7f78a6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a006dc80 con 0x7f78a006d2d0 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.921+0000 7f789ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f78a006ea90 0x7f78a0071480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53448/0 (socket says 172.21.15.165:53448) 2024-04-25T10:12:20.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.921+0000 7f789ea00640 1 -- 172.21.15.165:0/210993519 learned_addr learned my addr 172.21.15.165:0/210993519 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.921+0000 7f789fe00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 0x7f78a006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.921+0000 7f789f400640 1 --2- 172.21.15.165:0/210993519 >> v2:172.21.15.172:3300/0 conn(0x7f78a0161fc0 0x7f78a01623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.921+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125421966 0 0) 0x7f78a01628e0 con 0x7f78a006ea90 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.921+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7884003520 con 0x7f78a006ea90 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4166861166 0 0) 0x7f7884003520 con 0x7f78a006ea90 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 msgr2=0x7f78a006d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 0x7f78a006d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 >> v2:172.21.15.172:3300/0 conn(0x7f78a0161fc0 msgr2=0x7f78a01623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 --2- 172.21.15.165:0/210993519 >> v2:172.21.15.172:3300/0 conn(0x7f78a0161fc0 0x7f78a01623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0162ba0 con 0x7f78a006ea90 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f788c001690 con 0x7f78a006ea90 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 >> v1:172.21.15.165:6789/0 conn(0x7f78a006ea90 legacy=0x7f78a0071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 0x7f78840039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7884003f30 0x7f7884004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7884004d40 0x7f78840051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a0071b90 con 0x7f7884004d40 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a006dc80 con 0x7f7884003f30 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788400ba10 con 0x7f78a006d2d0 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.922+0000 7f789fe00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7884004d40 0x7f78840051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789f400640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 0x7f78840039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789ea00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7884003f30 0x7f7884004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789fe00640 1 -- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 msgr2=0x7f78840039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789fe00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78a006d2d0 0x7f78840039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789fe00640 1 -- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7884003f30 msgr2=0x7f7884004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:20.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789fe00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7884003f30 0x7f7884004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789fe00640 1 -- 172.21.15.165:0/210993519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884003520 con 0x7f7884004d40 2024-04-25T10:12:20.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.923+0000 7f789ea00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7884003f30 0x7f7884004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:20.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.924+0000 7f789fe00640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7884004d40 0x7f78840051c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f789800e170 tx=0x7f7884005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f0352d0ca8d2f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.924+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898034020 con 0x7f7884004d40 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.924+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78980305f0 con 0x7f7884004d40 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.924+0000 7f789e000640 1 -- 172.21.15.165:0/210993519 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7898009c10 con 0x7f7884004d40 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f78a6600640 1 -- 172.21.15.165:0/210993519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7884004d40 msgr2=0x7f78840051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f78a6600640 1 --2- 172.21.15.165:0/210993519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7884004d40 0x7f78840051c0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f789800e170 tx=0x7f7884005830 comp rx=0 tx=0).stop 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f789fe00640 1 -- 172.21.15.165:0/210993519 reap_dead start 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f78a6600640 1 -- 172.21.15.165:0/210993519 shutdown_connections 2024-04-25T10:12:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f78a6600640 1 -- 172.21.15.165:0/210993519 >> 172.21.15.165:0/210993519 conn(0x7f78a006abe0 msgr2=0x7f78a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f78a6600640 1 -- 172.21.15.165:0/210993519 shutdown_connections 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.925+0000 7f78a6600640 1 -- 172.21.15.165:0/210993519 wait complete. 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.927+0000 7f78a6600640 1 Processor -- start 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.927+0000 7f78a6600640 1 -- start start 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f78a6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7884004d40 0x7f78a0142a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f78a6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 0x7f78a0154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f78a6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78a0154590 0x7f78a0156460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f78a6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a0163160 con 0x7f78a0142270 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f78a6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a0163440 con 0x7f78a0154590 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f78a6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a006ef40 con 0x7f7884004d40 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 0x7f78a0154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 0x7f78a0154050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48218/0 (socket says 172.21.15.165:48218) 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 -- 172.21.15.165:0/1506883571 learned_addr learned my addr 172.21.15.165:0/1506883571 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 -- 172.21.15.165:0/1506883571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7884004d40 msgr2=0x7f78a0142a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7884004d40 0x7f78a0142a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 -- 172.21.15.165:0/1506883571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78a0154590 msgr2=0x7f78a0156460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78a0154590 0x7f78a0156460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.928+0000 7f789ea00640 1 -- 172.21.15.165:0/1506883571 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a0157230 con 0x7f78a0142270 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.929+0000 7f789ea00640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 0x7f78a0154050 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f788c000fd0 tx=0x7f788c00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.929+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c019070 con 0x7f78a0142270 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.929+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f788c0074e0 con 0x7f78a0142270 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.929+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f788c0049e0 con 0x7f78a0142270 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.929+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a00af0e0 con 0x7f78a0142270 2024-04-25T10:12:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.929+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a00af6a0 con 0x7f78a0142270 2024-04-25T10:12:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.930+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a01549b0 con 0x7f78a0142270 2024-04-25T10:12:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.931+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f788c004b80 con 0x7f78a0142270 2024-04-25T10:12:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.932+0000 7f7897400640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f788006b4d0 0x7f788006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.932+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f788c089e40 con 0x7f78a0142270 2024-04-25T10:12:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.932+0000 7f789f400640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f788006b4d0 0x7f788006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.933+0000 7f789f400640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f788006b4d0 0x7f788006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f78a0161460 tx=0x7f7890008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:20.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:20.937+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f788c058ab0 con 0x7f78a0142270 2024-04-25T10:12:21.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.124+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f78a0073a00 con 0x7f78a0142270 2024-04-25T10:12:21.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.125+0000 7f7897400640 1 -- 172.21.15.165:0/1506883571 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) v1 ==== 104+0+11 (secure 0 0 0) 0x7f788c0583b0 con 0x7f78a0142270 2024-04-25T10:12:21.130 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:12:21.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.127+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f788006b4d0 msgr2=0x7f788006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.127+0000 7f78a6600640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f788006b4d0 0x7f788006d990 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f78a0161460 tx=0x7f7890008040 comp rx=0 tx=0).stop 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 msgr2=0x7f78a0154050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 0x7f78a0154050 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7f788c000fd0 tx=0x7f788c00dd20 comp rx=0 tx=0).stop 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 shutdown_connections 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f788006b4d0 0x7f788006d990 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78a0154590 0x7f78a0156460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78a0142270 0x7f78a0154050 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 --2- 172.21.15.165:0/1506883571 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7884004d40 0x7f78a0142a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 >> 172.21.15.165:0/1506883571 conn(0x7f78a006abe0 msgr2=0x7f78a0153700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 shutdown_connections 2024-04-25T10:12:21.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.128+0000 7f78a6600640 1 -- 172.21.15.165:0/1506883571 wait complete. 2024-04-25T10:12:21.150 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:12:21.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.337+0000 7fd937000640 1 Processor -- start 2024-04-25T10:12:21.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.337+0000 7fd937000640 1 -- start start 2024-04-25T10:12:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.339+0000 7fd937000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd93015dd30 0x7fd9300cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.339+0000 7fd937000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd93015e110 con 0x7fd93006bff0 2024-04-25T10:12:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.339+0000 7fd937000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9300b2fd0 con 0x7fd9300b5b40 2024-04-25T10:12:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.339+0000 7fd937000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd93006d000 con 0x7fd93015dd30 2024-04-25T10:12:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.339+0000 7fd935200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd93006bff0 0x7fd9300b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53464/0 (socket says 172.21.15.165:53464) 2024-04-25T10:12:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.339+0000 7fd935200640 1 -- 172.21.15.165:0/2486291335 learned_addr learned my addr 172.21.15.165:0/2486291335 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1853555812 0 0) 0x7fd93015e110 con 0x7fd93006bff0 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd918003520 con 0x7fd93006bff0 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd935c00640 1 --2- 172.21.15.165:0/2486291335 >> v2:172.21.15.172:3300/0 conn(0x7fd93015dd30 0x7fd9300cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2185021490 0 0) 0x7fd918003520 con 0x7fd93006bff0 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 >> v2:172.21.15.172:3300/0 conn(0x7fd93015dd30 msgr2=0x7fd9300cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 --2- 172.21.15.165:0/2486291335 >> v2:172.21.15.172:3300/0 conn(0x7fd93015dd30 0x7fd9300cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 >> v1:172.21.15.172:6789/0 conn(0x7fd9300b5b40 legacy=0x7fd9300d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd93006c480 con 0x7fd93006bff0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd91c001670 con 0x7fd93006bff0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 >> v1:172.21.15.165:6789/0 conn(0x7fd93006bff0 legacy=0x7fd9300b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9180039b0 0x7fd918003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd918004800 0x7fd918004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9180051c0 0x7fd91800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93006d000 con 0x7fd918004800 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91800c0f0 con 0x7fd9180039b0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.340+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91800c2f0 con 0x7fd9180051c0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935200640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd918004800 0x7fd918004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935c00640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9180039b0 0x7fd918003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd936600640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9180051c0 0x7fd91800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935200640 1 -- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9180051c0 msgr2=0x7fd91800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935200640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9180051c0 0x7fd91800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935200640 1 -- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9180039b0 msgr2=0x7fd918003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935200640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9180039b0 0x7fd918003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd935200640 1 -- 172.21.15.165:0/2486291335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918003520 con 0x7fd918004800 2024-04-25T10:12:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd936600640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd9180051c0 0x7fd91800b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:21.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.341+0000 7fd936600640 1 -- 172.21.15.165:0/2486291335 reap_dead start 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.342+0000 7fd935200640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd918004800 0x7fd918004c80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c210 tx=0x7fd918005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa093e9081b3d9e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.342+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c007d30 con 0x7fd918004800 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd91c005af0 con 0x7fd918004800 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd92fe00640 1 -- 172.21.15.165:0/2486291335 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c007070 con 0x7fd918004800 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 -- 172.21.15.165:0/2486291335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd918004800 msgr2=0x7fd918004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd918004800 0x7fd918004c80 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7fd91c00c210 tx=0x7fd918005830 comp rx=0 tx=0).stop 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 -- 172.21.15.165:0/2486291335 shutdown_connections 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 --2- 172.21.15.165:0/2486291335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd918004800 0x7fd918004c80 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 -- 172.21.15.165:0/2486291335 >> 172.21.15.165:0/2486291335 conn(0x7fd9300d0610 msgr2=0x7fd9300d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 -- 172.21.15.165:0/2486291335 shutdown_connections 2024-04-25T10:12:21.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.343+0000 7fd937000640 1 -- 172.21.15.165:0/2486291335 wait complete. 2024-04-25T10:12:21.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.345+0000 7fd937000640 1 Processor -- start 2024-04-25T10:12:21.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.345+0000 7fd937000640 1 -- start start 2024-04-25T10:12:21.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd937000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd918004800 0x7fd930156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd937000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300cc010 0x7fd93015fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd937000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 0x7fd9301541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd937000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93006cd20 con 0x7fd930160220 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd937000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd93006ca40 con 0x7fd9300cc010 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd937000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9300b5430 con 0x7fd918004800 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd936600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 0x7fd9301541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd936600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 0x7fd9301541a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48242/0 (socket says 172.21.15.165:48242) 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.346+0000 7fd936600640 1 -- 172.21.15.165:0/1545677908 learned_addr learned my addr 172.21.15.165:0/1545677908 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd935200640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300cc010 0x7fd93015fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd935c00640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd918004800 0x7fd930156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd936600640 1 -- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd918004800 msgr2=0x7fd930156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd936600640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd918004800 0x7fd930156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd936600640 1 -- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300cc010 msgr2=0x7fd93015fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd936600640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300cc010 0x7fd93015fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd936600640 1 -- 172.21.15.165:0/1545677908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9301f6de0 con 0x7fd930160220 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd935c00640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd918004800 0x7fd930156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd936600640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 0x7fd9301541a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd9300b9aa0 tx=0x7fd92800d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd92801a070 con 0x7fd930160220 2024-04-25T10:12:21.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.347+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd928006d70 con 0x7fd930160220 2024-04-25T10:12:21.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.348+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9301f70c0 con 0x7fd930160220 2024-04-25T10:12:21.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.348+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd928012840 con 0x7fd930160220 2024-04-25T10:12:21.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.348+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9301f7680 con 0x7fd930160220 2024-04-25T10:12:21.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.349+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd930073990 con 0x7fd930160220 2024-04-25T10:12:21.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.350+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd92801d070 con 0x7fd930160220 2024-04-25T10:12:21.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.351+0000 7fd92d600640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd91006b500 0x7fd91006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.351+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fd928089630 con 0x7fd930160220 2024-04-25T10:12:21.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.351+0000 7fd935c00640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd91006b500 0x7fd91006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.352+0000 7fd935c00640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd91006b500 0x7fd91006d9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.355+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd9280582a0 con 0x7fd930160220 2024-04-25T10:12:21.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.549+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd9300b5210 con 0x7fd930160220 2024-04-25T10:12:21.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.550+0000 7fd92d600640 1 -- 172.21.15.165:0/1545677908 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd928057ba0 con 0x7fd930160220 2024-04-25T10:12:21.555 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:12:21.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.552+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd91006b500 msgr2=0x7fd91006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.552+0000 7fd937000640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd91006b500 0x7fd91006d9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fd924006fd0 tx=0x7fd924008040 comp rx=0 tx=0).stop 2024-04-25T10:12:21.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.552+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 msgr2=0x7fd9301541a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.552+0000 7fd937000640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 0x7fd9301541a0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7fd9300b9aa0 tx=0x7fd92800d9d0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 shutdown_connections 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd91006b500 0x7fd91006d9c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd930160220 0x7fd9301541a0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd9300cc010 0x7fd93015fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 --2- 172.21.15.165:0/1545677908 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd918004800 0x7fd930156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 >> 172.21.15.165:0/1545677908 conn(0x7fd9300d0610 msgr2=0x7fd9300b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 shutdown_connections 2024-04-25T10:12:21.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.553+0000 7fd937000640 1 -- 172.21.15.165:0/1545677908 wait complete. 2024-04-25T10:12:21.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:21.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.766+0000 7f7aec600640 1 Processor -- start 2024-04-25T10:12:21.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.766+0000 7f7aec600640 1 -- start start 2024-04-25T10:12:21.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.767+0000 7f7aec600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae40ce920 0x7f7ae40ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.767+0000 7f7aec600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7ae406d290 0x7f7ae406d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.767+0000 7f7aec600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ae4144b30 con 0x7f7ae4144750 2024-04-25T10:12:21.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.767+0000 7f7aec600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ae40712d0 con 0x7f7ae406d290 2024-04-25T10:12:21.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.767+0000 7f7aec600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae40715b0 con 0x7f7ae40ce920 2024-04-25T10:12:21.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7ae406d290 0x7f7ae406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7ae406d290 0x7f7ae406d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48248/0 (socket says 172.21.15.165:48248) 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 -- 172.21.15.165:0/2323247054 learned_addr learned my addr 172.21.15.165:0/2323247054 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7ae9200640 1 --2- 172.21.15.165:0/2323247054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae40ce920 0x7f7ae40ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 -- 172.21.15.165:0/2323247054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae40ce920 msgr2=0x7f7ae40ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 --2- 172.21.15.165:0/2323247054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae40ce920 0x7f7ae40ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 -- 172.21.15.165:0/2323247054 >> v1:172.21.15.172:6789/0 conn(0x7f7ae4144750 legacy=0x7f7ae406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.768+0000 7f7aea600640 1 -- 172.21.15.165:0/2323247054 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae4071890 con 0x7f7ae406d290 2024-04-25T10:12:21.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.769+0000 7f7aea600640 1 --2- 172.21.15.165:0/2323247054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae406d720 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009e50 tx=0x7f7adc02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7932757bdaee577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.770+0000 7f7ae3e00640 1 -- 172.21.15.165:0/2323247054 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc036070 con 0x7f7ae406d290 2024-04-25T10:12:21.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.770+0000 7f7ae3e00640 1 -- 172.21.15.165:0/2323247054 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7adc031600 con 0x7f7ae406d290 2024-04-25T10:12:21.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.770+0000 7f7ae3e00640 1 -- 172.21.15.165:0/2323247054 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc0319d0 con 0x7f7ae406d290 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 -- 172.21.15.165:0/2323247054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 msgr2=0x7f7ae406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 --2- 172.21.15.165:0/2323247054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae406d720 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009e50 tx=0x7f7adc02a710 comp rx=0 tx=0).stop 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 -- 172.21.15.165:0/2323247054 shutdown_connections 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 --2- 172.21.15.165:0/2323247054 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae406d720 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 --2- 172.21.15.165:0/2323247054 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae40ce920 0x7f7ae40ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 -- 172.21.15.165:0/2323247054 >> 172.21.15.165:0/2323247054 conn(0x7f7ae406abe0 msgr2=0x7f7ae406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 -- 172.21.15.165:0/2323247054 shutdown_connections 2024-04-25T10:12:21.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.771+0000 7f7aec600640 1 -- 172.21.15.165:0/2323247054 wait complete. 2024-04-25T10:12:21.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.772+0000 7f7aec600640 1 Processor -- start 2024-04-25T10:12:21.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 -- start start 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae414f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ae40ce920 0x7f7ae415c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae4144750 0x7f7ae40b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406f020 con 0x7f7ae406d290 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae406ece0 con 0x7f7ae40ce920 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.773+0000 7f7aec600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae4145450 con 0x7f7ae4144750 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae414f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae414f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48262/0 (socket says 172.21.15.165:48262) 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 -- 172.21.15.165:0/3193511137 learned_addr learned my addr 172.21.15.165:0/3193511137 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 -- 172.21.15.165:0/3193511137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae4144750 msgr2=0x7f7ae40b1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae4144750 0x7f7ae40b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 -- 172.21.15.165:0/3193511137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ae40ce920 msgr2=0x7f7ae415c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:21.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ae40ce920 0x7f7ae415c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 -- 172.21.15.165:0/3193511137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae414cd70 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae9c00640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae414f800 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c400 tx=0x7f7ad800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.774+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad80052d0 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.775+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad8005490 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.775+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad8007260 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.775+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae414d050 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.775+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae414d610 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.776+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae414d810 con 0x7f7ae406d290 2024-04-25T10:12:21.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.777+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7ad8007400 con 0x7f7ae406d290 2024-04-25T10:12:21.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.778+0000 7f7ae1600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7ac406b4d0 0x7f7ac406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:21.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.778+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f7ad8089990 con 0x7f7ae406d290 2024-04-25T10:12:21.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.778+0000 7f7ae9200640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7ac406b4d0 0x7f7ac406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:21.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.779+0000 7f7ae9200640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7ac406b4d0 0x7f7ac406d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7ae414cb10 tx=0x7f7ad0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:21.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:21.782+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7ad8058600 con 0x7f7ae406d290 2024-04-25T10:12:22.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.013+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ae4073990 con 0x7f7ae406d290 2024-04-25T10:12:22.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.014+0000 7f7ae1600640 1 -- 172.21.15.165:0/3193511137 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f7ad8057f00 con 0x7f7ae406d290 2024-04-25T10:12:22.019 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:22.019 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:22.019 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:22.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7ac406b4d0 msgr2=0x7f7ac406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7ac406b4d0 0x7f7ac406d990 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7ae414cb10 tx=0x7f7ad0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:22.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 msgr2=0x7f7ae414f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae414f800 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f7ad800c400 tx=0x7f7ad800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 shutdown_connections 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7ac406b4d0 0x7f7ac406d990 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7ae4144750 0x7f7ae40b1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7ae40ce920 0x7f7ae415c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 --2- 172.21.15.165:0/3193511137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7ae406d290 0x7f7ae414f800 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 >> 172.21.15.165:0/3193511137 conn(0x7f7ae406abe0 msgr2=0x7f7ae413e8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 shutdown_connections 2024-04-25T10:12:22.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.017+0000 7f7aec600640 1 -- 172.21.15.165:0/3193511137 wait complete. 2024-04-25T10:12:22.041 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:22.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.232+0000 7f81a6a00640 1 Processor -- start 2024-04-25T10:12:22.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.232+0000 7f81a6a00640 1 -- start start 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.233+0000 7f81a6a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81a0144440 0x7f81a0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.233+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81a00cda30 0x7f81a015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.233+0000 7f81a6a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81a015bf30 con 0x7f81a0071530 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.233+0000 7f81a6a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81a00719a0 con 0x7f81a0144440 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.233+0000 7f81a6a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a015c860 con 0x7f81a00cda30 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f81a4c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f81a0071530 0x7f81a00cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53468/0 (socket says 172.21.15.165:53468) 2024-04-25T10:12:22.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f81a4c00640 1 -- 172.21.15.165:0/3351203382 learned_addr learned my addr 172.21.15.165:0/3351203382 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f81a5600640 1 --2- 172.21.15.165:0/3351203382 >> v2:172.21.15.172:3300/0 conn(0x7f81a0144440 0x7f81a0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f81a6000640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81a00cda30 0x7f81a015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4278164216 0 0) 0x7f81a015bf30 con 0x7f81a0071530 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8188003520 con 0x7f81a0071530 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3945133361 0 0) 0x7f8188003520 con 0x7f81a0071530 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81a00cda30 msgr2=0x7f81a015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81a00cda30 0x7f81a015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 >> v2:172.21.15.172:3300/0 conn(0x7f81a0144440 msgr2=0x7f81a0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 --2- 172.21.15.165:0/3351203382 >> v2:172.21.15.172:3300/0 conn(0x7f81a0144440 0x7f81a0070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.234+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a015c210 con 0x7f81a0071530 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f818c001670 con 0x7f81a0071530 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 >> v1:172.21.15.165:6789/0 conn(0x7f81a0071530 legacy=0x7f81a00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81880039b0 0x7f8188003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00719a0 con 0x7f81880051c0 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a015c860 con 0x7f81880039b0 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f818800bf80 con 0x7f8188004800 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a6000640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81a00cda30 0x7f81a015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a6000640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a5600640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81880039b0 0x7f8188003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a4c00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a6000640 1 -- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8188004800 msgr2=0x7f8188004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a6000640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.235+0000 7f81a6000640 1 -- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81880039b0 msgr2=0x7f8188003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.236+0000 7f81a6000640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81880039b0 0x7f8188003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.236+0000 7f81a6000640 1 -- 172.21.15.165:0/3351203382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8188003520 con 0x7f81880051c0 2024-04-25T10:12:22.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.236+0000 7f81a4c00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8188004800 0x7f8188004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f81a6000640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8198009f00 tx=0x7f8188005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf888f9bbef928c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198009b50 con 0x7f81880051c0 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8198003990 con 0x7f81880051c0 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f819fe00640 1 -- 172.21.15.165:0/3351203382 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8198003d60 con 0x7f81880051c0 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f81a6a00640 1 -- 172.21.15.165:0/3351203382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81880051c0 msgr2=0x7f818800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f81a6a00640 1 --2- 172.21.15.165:0/3351203382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81880051c0 0x7f818800b9c0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f8198009f00 tx=0x7f8188005830 comp rx=0 tx=0).stop 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.237+0000 7f81a6000640 1 -- 172.21.15.165:0/3351203382 reap_dead start 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.238+0000 7f81a6a00640 1 -- 172.21.15.165:0/3351203382 shutdown_connections 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.238+0000 7f81a6a00640 1 -- 172.21.15.165:0/3351203382 >> 172.21.15.165:0/3351203382 conn(0x7f81a006abe0 msgr2=0x7f81a006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.238+0000 7f81a6a00640 1 -- 172.21.15.165:0/3351203382 shutdown_connections 2024-04-25T10:12:22.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.238+0000 7f81a6a00640 1 -- 172.21.15.165:0/3351203382 wait complete. 2024-04-25T10:12:22.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.239+0000 7f81a6a00640 1 Processor -- start 2024-04-25T10:12:22.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.240+0000 7f81a6a00640 1 -- start start 2024-04-25T10:12:22.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.240+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81880051c0 0x7f81a013dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 0x7f81a01f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a013ab10 0x7f81a013af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a6a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a015bc30 con 0x7f81a013ec70 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a6a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a00cdea0 con 0x7f81a013ab10 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a6a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81a015ccf0 con 0x7f81880051c0 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 0x7f81a01f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 0x7f81a01f3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48276/0 (socket says 172.21.15.165:48276) 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 -- 172.21.15.165:0/1853525741 learned_addr learned my addr 172.21.15.165:0/1853525741 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 -- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81880051c0 msgr2=0x7f81a013dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81880051c0 0x7f81a013dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 -- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a013ab10 msgr2=0x7f81a013af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a6000640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a013ab10 0x7f81a013af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a013ab10 0x7f81a013af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.241+0000 7f81a4c00640 1 -- 172.21.15.165:0/1853525741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a013e5c0 con 0x7f81a013ec70 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.242+0000 7f81a4c00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 0x7f81a01f3590 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f818c00d810 tx=0x7f818c00dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.242+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c019070 con 0x7f81a013ec70 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.242+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f818c0074e0 con 0x7f81a013ec70 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.242+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a013a030 con 0x7f81a013ec70 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.242+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0049e0 con 0x7f81a013ec70 2024-04-25T10:12:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.242+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a013a5f0 con 0x7f81a013ec70 2024-04-25T10:12:22.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.243+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a0073990 con 0x7f81a013ec70 2024-04-25T10:12:22.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.244+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f818c01c070 con 0x7f81a013ec70 2024-04-25T10:12:22.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.245+0000 7f819d600640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f818006b500 0x7f818006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.245+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f818c089fb0 con 0x7f81a013ec70 2024-04-25T10:12:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.245+0000 7f81a5600640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f818006b500 0x7f818006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.246+0000 7f81a5600640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f818006b500 0x7f818006d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8194009990 tx=0x7f8194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.249+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f818c058c50 con 0x7f81a013ec70 2024-04-25T10:12:22.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.430+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81a00cc6b0 con 0x7f81a013ec70 2024-04-25T10:12:22.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.433+0000 7f819d600640 1 -- 172.21.15.165:0/1853525741 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17114 (secure 0 0 0) 0x7f818c058550 con 0x7f81a013ec70 2024-04-25T10:12:22.439 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:22.439 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":100,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:47.812589+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":5}]},{"pgid":"14.a","mappings":[{"from":1,"to":2}]},{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f818006b500 msgr2=0x7f818006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f818006b500 0x7f818006d9c0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f8194009990 tx=0x7f8194008040 comp rx=0 tx=0).stop 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 msgr2=0x7f81a01f3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 0x7f81a01f3590 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f818c00d810 tx=0x7f818c00dcd0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 shutdown_connections 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f818006b500 0x7f818006d9c0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81a013ab10 0x7f81a013af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81a013ec70 0x7f81a01f3590 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 --2- 172.21.15.165:0/1853525741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81880051c0 0x7f81a013dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 >> 172.21.15.165:0/1853525741 conn(0x7f81a006abe0 msgr2=0x7f81a00cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 shutdown_connections 2024-04-25T10:12:22.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.436+0000 7f81a6a00640 1 -- 172.21.15.165:0/1853525741 wait complete. 2024-04-25T10:12:22.460 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:12:22.460 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:22.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.657+0000 7fcb69200640 1 Processor -- start 2024-04-25T10:12:22.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.657+0000 7fcb69200640 1 -- start start 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb69200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb64070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb69200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb64071530 0x7fcb640cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb69200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcb640cd860 0x7fcb6406d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb69200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6415c860 con 0x7fcb64071530 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb69200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb64071970 con 0x7fcb640cd860 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb69200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6415bf30 con 0x7fcb64144440 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb5aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb64071530 0x7fcb640cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb5aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb64071530 0x7fcb640cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48282/0 (socket says 172.21.15.165:48282) 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb5aa00640 1 -- 172.21.15.165:0/3199747534 learned_addr learned my addr 172.21.15.165:0/3199747534 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb63e00640 1 --2- 172.21.15.165:0/3199747534 >> v2:172.21.15.172:3300/0 conn(0x7fcb640cd860 0x7fcb6406d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.659+0000 7fcb63400640 1 --2- 172.21.15.165:0/3199747534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb64070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.660+0000 7fcb5aa00640 1 -- 172.21.15.165:0/3199747534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 msgr2=0x7fcb64070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.660+0000 7fcb5aa00640 1 --2- 172.21.15.165:0/3199747534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb64070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.660+0000 7fcb5aa00640 1 -- 172.21.15.165:0/3199747534 >> v2:172.21.15.172:3300/0 conn(0x7fcb640cd860 msgr2=0x7fcb6406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.660+0000 7fcb5aa00640 1 --2- 172.21.15.165:0/3199747534 >> v2:172.21.15.172:3300/0 conn(0x7fcb640cd860 0x7fcb6406d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.660+0000 7fcb5aa00640 1 -- 172.21.15.165:0/3199747534 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6415c210 con 0x7fcb64071530 2024-04-25T10:12:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.661+0000 7fcb5aa00640 1 --2- 172.21.15.165:0/3199747534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb640cd320 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcb64145360 tx=0x7fcb4c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f650e05a990935 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.661+0000 7fcb62a00640 1 -- 172.21.15.165:0/3199747534 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c037070 con 0x7fcb64071530 2024-04-25T10:12:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.661+0000 7fcb62a00640 1 -- 172.21.15.165:0/3199747534 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c032440 con 0x7fcb64071530 2024-04-25T10:12:22.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.661+0000 7fcb62a00640 1 -- 172.21.15.165:0/3199747534 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c032810 con 0x7fcb64071530 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 -- 172.21.15.165:0/3199747534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 msgr2=0x7fcb640cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 --2- 172.21.15.165:0/3199747534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb640cd320 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fcb64145360 tx=0x7fcb4c02a270 comp rx=0 tx=0).stop 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 -- 172.21.15.165:0/3199747534 shutdown_connections 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 --2- 172.21.15.165:0/3199747534 >> v2:172.21.15.172:3300/0 conn(0x7fcb640cd860 0x7fcb6406d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 --2- 172.21.15.165:0/3199747534 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb640cd320 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 --2- 172.21.15.165:0/3199747534 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb64070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 -- 172.21.15.165:0/3199747534 >> 172.21.15.165:0/3199747534 conn(0x7fcb6406abe0 msgr2=0x7fcb6406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 -- 172.21.15.165:0/3199747534 shutdown_connections 2024-04-25T10:12:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.662+0000 7fcb69200640 1 -- 172.21.15.165:0/3199747534 wait complete. 2024-04-25T10:12:22.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.664+0000 7fcb69200640 1 Processor -- start 2024-04-25T10:12:22.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.664+0000 7fcb69200640 1 -- start start 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb69200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb641f42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb69200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640cd860 0x7fcb640c7d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb69200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb640c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb69200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640cdcd0 con 0x7fcb64071530 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb69200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6415bc30 con 0x7fcb640cd860 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb69200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640cde50 con 0x7fcb64144440 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb63400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb641f42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb63400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb641f42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48294/0 (socket says 172.21.15.165:48294) 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.665+0000 7fcb63400640 1 -- 172.21.15.165:0/78926904 learned_addr learned my addr 172.21.15.165:0/78926904 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb5aa00640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640cd860 0x7fcb640c7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63e00640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb640c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63400640 1 -- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 msgr2=0x7fcb640c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63400640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb640c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63400640 1 -- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640cd860 msgr2=0x7fcb640c7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63400640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640cd860 0x7fcb640c7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63400640 1 -- 172.21.15.165:0/78926904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640c8760 con 0x7fcb64071530 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63e00640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb640c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb63400640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb641f42d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d5f0 tx=0x7fcb5400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540125f0 con 0x7fcb64071530 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54019070 con 0x7fcb64071530 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640c8a40 con 0x7fcb64071530 2024-04-25T10:12:22.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.666+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540129d0 con 0x7fcb64071530 2024-04-25T10:12:22.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.667+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640c9000 con 0x7fcb64071530 2024-04-25T10:12:22.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.668+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64073990 con 0x7fcb64071530 2024-04-25T10:12:22.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.669+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb54012b70 con 0x7fcb64071530 2024-04-25T10:12:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.669+0000 7fcb5be00640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.670+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fcb5408a860 con 0x7fcb64071530 2024-04-25T10:12:22.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.670+0000 7fcb5aa00640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:22.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.670+0000 7fcb5aa00640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c034000 tx=0x7fcb4c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:22.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.674+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb54059440 con 0x7fcb64071530 2024-04-25T10:12:22.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.903+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb64142590 con 0x7fcb64071530 2024-04-25T10:12:22.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.905+0000 7fcb5be00640 1 -- 172.21.15.165:0/78926904 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fcb54058d40 con 0x7fcb64071530 2024-04-25T10:12:22.910 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:22.910 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:22.910 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 msgr2=0x7fcb4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c034000 tx=0x7fcb4c02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 msgr2=0x7fcb641f42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb641f42d0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fcb5400d5f0 tx=0x7fcb5400dab0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 shutdown_connections 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64144440 0x7fcb640c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb640cd860 0x7fcb640c7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 --2- 172.21.15.165:0/78926904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071530 0x7fcb641f42d0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:22.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.907+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 >> 172.21.15.165:0/78926904 conn(0x7fcb6406abe0 msgr2=0x7fcb640cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:22.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.908+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 shutdown_connections 2024-04-25T10:12:22.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:22.908+0000 7fcb69200640 1 -- 172.21.15.165:0/78926904 wait complete. 2024-04-25T10:12:22.930 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:23.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.129+0000 7f2fb7e00640 1 Processor -- start 2024-04-25T10:12:23.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.129+0000 7f2fb7e00640 1 -- start start 2024-04-25T10:12:23.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fb7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2fb815db90 0x7f2fb80b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb80d06c0 0x7f2fb80d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fb7e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fb8161df0 con 0x7f2fb81615a0 2024-04-25T10:12:23.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fb7e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2fb80d1f50 con 0x7f2fb815db90 2024-04-25T10:12:23.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fb7e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb81620d0 con 0x7f2fb80d06c0 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fb6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2fb815db90 0x7f2fb80b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fae000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2fb81615a0 0x7f2fb80cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53476/0 (socket says 172.21.15.165:53476) 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.131+0000 7f2fae000640 1 -- 172.21.15.165:0/1438252244 learned_addr learned my addr 172.21.15.165:0/1438252244 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb7400640 1 --2- 172.21.15.165:0/1438252244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb80d06c0 0x7f2fb80d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb6000640 1 -- 172.21.15.165:0/1438252244 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661683967 0 0) 0x7f2fb8161df0 con 0x7f2fb81615a0 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb6000640 1 -- 172.21.15.165:0/1438252244 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f9c003520 con 0x7f2fb81615a0 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb6a00640 1 -- 172.21.15.165:0/1438252244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb80d06c0 msgr2=0x7f2fb80d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb6a00640 1 --2- 172.21.15.165:0/1438252244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb80d06c0 0x7f2fb80d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb6a00640 1 -- 172.21.15.165:0/1438252244 >> v1:172.21.15.165:6789/0 conn(0x7f2fb81615a0 legacy=0x7f2fb80cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.132+0000 7f2fb6a00640 1 -- 172.21.15.165:0/1438252244 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb81623b0 con 0x7f2fb815db90 2024-04-25T10:12:23.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.133+0000 7f2fb6a00640 1 --2- 172.21.15.165:0/1438252244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb815db90 0x7f2fb80b93c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80097a0 tx=0x7f2fa802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=108df30eec6941f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb6000640 1 -- 172.21.15.165:0/1438252244 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8037070 con 0x7f2fb815db90 2024-04-25T10:12:23.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb6000640 1 -- 172.21.15.165:0/1438252244 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8032440 con 0x7f2fb815db90 2024-04-25T10:12:23.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb6000640 1 -- 172.21.15.165:0/1438252244 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8032810 con 0x7f2fb815db90 2024-04-25T10:12:23.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb7e00640 1 -- 172.21.15.165:0/1438252244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb815db90 msgr2=0x7f2fb80b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/1438252244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb815db90 0x7f2fb80b93c0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7f2fa80097a0 tx=0x7f2fa802a180 comp rx=0 tx=0).stop 2024-04-25T10:12:23.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb7e00640 1 -- 172.21.15.165:0/1438252244 shutdown_connections 2024-04-25T10:12:23.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/1438252244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb80d06c0 0x7f2fb80d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/1438252244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb815db90 0x7f2fb80b93c0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.134+0000 7f2fb7e00640 1 -- 172.21.15.165:0/1438252244 >> 172.21.15.165:0/1438252244 conn(0x7f2fb80b52b0 msgr2=0x7f2fb80b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.135+0000 7f2fb7e00640 1 -- 172.21.15.165:0/1438252244 shutdown_connections 2024-04-25T10:12:23.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.135+0000 7f2fb7e00640 1 -- 172.21.15.165:0/1438252244 wait complete. 2024-04-25T10:12:23.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.136+0000 7f2fb7e00640 1 Processor -- start 2024-04-25T10:12:23.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.136+0000 7f2fb7e00640 1 -- start start 2024-04-25T10:12:23.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.137+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 0x7f2fb80d39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.137+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb815db90 0x7f2fb8157e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.137+0000 7f2fb7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fb81615a0 0x7f2fb81583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.137+0000 7f2fb7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8161ba0 con 0x7f2fb80d06c0 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.137+0000 7f2fb7e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb8161a20 con 0x7f2fb81615a0 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.137+0000 7f2fb7e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fb80b68d0 con 0x7f2fb815db90 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 0x7f2fb80d39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 0x7f2fb80d39f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48302/0 (socket says 172.21.15.165:48302) 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 -- 172.21.15.165:0/4109620136 learned_addr learned my addr 172.21.15.165:0/4109620136 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 -- 172.21.15.165:0/4109620136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb815db90 msgr2=0x7f2fb8157e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb815db90 0x7f2fb8157e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 -- 172.21.15.165:0/4109620136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fb81615a0 msgr2=0x7f2fb81583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fb81615a0 0x7f2fb81583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 -- 172.21.15.165:0/4109620136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fb815e110 con 0x7f2fb80d06c0 2024-04-25T10:12:23.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.138+0000 7f2fb6a00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 0x7f2fb80d39f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8035040 tx=0x7f2fa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.139+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8037040 con 0x7f2fb80d06c0 2024-04-25T10:12:23.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.139+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fa8030500 con 0x7f2fb80d06c0 2024-04-25T10:12:23.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.139+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fa8030910 con 0x7f2fb80d06c0 2024-04-25T10:12:23.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.139+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fb80cd270 con 0x7f2fb80d06c0 2024-04-25T10:12:23.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.139+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fb815e690 con 0x7f2fb80d06c0 2024-04-25T10:12:23.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.140+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fb8072e40 con 0x7f2fb80d06c0 2024-04-25T10:12:23.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.141+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2fa8044020 con 0x7f2fb80d06c0 2024-04-25T10:12:23.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.142+0000 7f2faf400640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f9406b4d0 0x7f2f9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.142+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f2fa80aa7a0 con 0x7f2fb80d06c0 2024-04-25T10:12:23.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.142+0000 7f2fae000640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f9406b4d0 0x7f2f9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.143+0000 7f2fae000640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f9406b4d0 0x7f2f9406d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0000f60 tx=0x7f2fa00078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.146+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2fb8072e40 con 0x7f2fb80d06c0 2024-04-25T10:12:23.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.379+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2fb80d24e0 con 0x7f2fb80d06c0 2024-04-25T10:12:23.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.380+0000 7f2faf400640 1 -- 172.21.15.165:0/4109620136 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 65 v65) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f2fb80d24e0 con 0x7f2fb80d06c0 2024-04-25T10:12:23.385 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:23.385 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":65,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":65,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:11:22.911237+0000","modified":"2024-04-25T10:11:25.971934+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":7569},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7569":{"gid":7569,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6835/2500333822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2500333822},{"type":"v1","addr":"172.21.15.165:6835","nonce":2500333822}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2024-04-25T10:12:23.385 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 65 2024-04-25T10:12:23.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f9406b4d0 msgr2=0x7f2f9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f9406b4d0 0x7f2f9406d990 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0000f60 tx=0x7f2fa00078f0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 msgr2=0x7f2fb80d39f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 0x7f2fb80d39f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f2fa8035040 tx=0x7f2fa80046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 shutdown_connections 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f9406b4d0 0x7f2f9406d990 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2fb81615a0 0x7f2fb81583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2fb815db90 0x7f2fb8157e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 --2- 172.21.15.165:0/4109620136 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2fb80d06c0 0x7f2fb80d39f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 >> 172.21.15.165:0/4109620136 conn(0x7f2fb80b52b0 msgr2=0x7f2fb80b7420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 shutdown_connections 2024-04-25T10:12:23.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.383+0000 7f2fb7e00640 1 -- 172.21.15.165:0/4109620136 wait complete. 2024-04-25T10:12:23.407 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:23.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.599+0000 7f9ea9800640 1 Processor -- start 2024-04-25T10:12:23.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.599+0000 7f9ea9800640 1 -- start start 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.601+0000 7f9ea9800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ea415c100 0x7f9ea4071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.601+0000 7f9ea9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 0x7f9ea406e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.601+0000 7f9ea9800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea40b3420 con 0x7f9ea406eac0 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.601+0000 7f9ea9800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea40b2810 con 0x7f9ea415c100 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.601+0000 7f9ea9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40b2af0 con 0x7f9ea40b3800 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea8e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9ea406eac0 0x7f9ea4165850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53490/0 (socket says 172.21.15.165:53490) 2024-04-25T10:12:23.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea8e00640 1 -- 172.21.15.165:0/2886790719 learned_addr learned my addr 172.21.15.165:0/2886790719 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/2886790719 >> v2:172.21.15.172:3300/0 conn(0x7f9ea415c100 0x7f9ea4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea3400640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 0x7f9ea406e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761591603 0 0) 0x7f9ea40b3420 con 0x7f9ea406eac0 2024-04-25T10:12:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e88003520 con 0x7f9ea406eac0 2024-04-25T10:12:23.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 516745393 0 0) 0x7f9e88003520 con 0x7f9ea406eac0 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 msgr2=0x7f9ea406e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.602+0000 7f9ea2a00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 0x7f9ea406e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 >> v2:172.21.15.172:3300/0 conn(0x7f9ea415c100 msgr2=0x7f9ea4071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 --2- 172.21.15.165:0/2886790719 >> v2:172.21.15.172:3300/0 conn(0x7f9ea415c100 0x7f9ea4071740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4071c80 con 0x7f9ea406eac0 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9e9c001670 con 0x7f9ea406eac0 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 >> v1:172.21.15.165:6789/0 conn(0x7f9ea406eac0 legacy=0x7f9ea4165850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e880039b0 0x7f9e88003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e88004800 0x7f9e88004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 0x7f9e880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40b2810 con 0x7f9e880039b0 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40b2af0 con 0x7f9e88004800 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8800b9c0 con 0x7f9ea40b3800 2024-04-25T10:12:23.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e880039b0 0x7f9e88003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea3400640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e88004800 0x7f9e88004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea8e00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 0x7f9e880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea3e00640 1 -- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 msgr2=0x7f9e880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea40b3800 0x7f9e880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.603+0000 7f9ea3e00640 1 -- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e88004800 msgr2=0x7f9e88004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.604+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e88004800 0x7f9e88004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.604+0000 7f9ea3e00640 1 -- 172.21.15.165:0/2886790719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e88003520 con 0x7f9e880039b0 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.604+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e880039b0 0x7f9e88003dd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ac0 tx=0x7f9e88005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02f22a90c5f41c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c036070 con 0x7f9e880039b0 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c031400 con 0x7f9e880039b0 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea2a00640 1 -- 172.21.15.165:0/2886790719 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c009690 con 0x7f9e880039b0 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea9800640 1 -- 172.21.15.165:0/2886790719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e880039b0 msgr2=0x7f9e88003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea9800640 1 --2- 172.21.15.165:0/2886790719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e880039b0 0x7f9e88003dd0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ac0 tx=0x7f9e88005830 comp rx=0 tx=0).stop 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea8e00640 1 -- 172.21.15.165:0/2886790719 reap_dead start 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea9800640 1 -- 172.21.15.165:0/2886790719 shutdown_connections 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.605+0000 7f9ea9800640 1 -- 172.21.15.165:0/2886790719 >> 172.21.15.165:0/2886790719 conn(0x7f9ea406abe0 msgr2=0x7f9ea406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.606+0000 7f9ea9800640 1 -- 172.21.15.165:0/2886790719 shutdown_connections 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.606+0000 7f9ea9800640 1 -- 172.21.15.165:0/2886790719 wait complete. 2024-04-25T10:12:23.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.607+0000 7f9ea9800640 1 Processor -- start 2024-04-25T10:12:23.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.607+0000 7f9ea9800640 1 -- start start 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.608+0000 7f9ea9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e880039b0 0x7f9ea4157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.608+0000 7f9ea9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea415a4f0 0x7f9ea415a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.608+0000 7f9ea9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 0x7f9ea4144890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.608+0000 7f9ea9800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40b24e0 con 0x7f9ea4144410 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.608+0000 7f9ea9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40b30b0 con 0x7f9e880039b0 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.608+0000 7f9ea9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea40711b0 con 0x7f9ea415a4f0 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 0x7f9ea4144890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 0x7f9ea4144890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48316/0 (socket says 172.21.15.165:48316) 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 -- 172.21.15.165:0/699567746 learned_addr learned my addr 172.21.15.165:0/699567746 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea3400640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea415a4f0 0x7f9ea415a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e880039b0 0x7f9ea4157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 -- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea415a4f0 msgr2=0x7f9ea415a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea415a4f0 0x7f9ea415a970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 -- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e880039b0 msgr2=0x7f9ea4157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e880039b0 0x7f9ea4157bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.609+0000 7f9ea8e00640 1 -- 172.21.15.165:0/699567746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4143370 con 0x7f9ea4144410 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.610+0000 7f9ea8e00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 0x7f9ea4144890 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406ba10 tx=0x7f9e9c00db80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.610+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c007230 con 0x7f9ea4144410 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.610+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9c0073f0 con 0x7f9ea4144410 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.610+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0047e0 con 0x7f9ea4144410 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.610+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ea4143650 con 0x7f9ea4144410 2024-04-25T10:12:23.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.610+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea4143c10 con 0x7f9ea4144410 2024-04-25T10:12:23.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.611+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea4073990 con 0x7f9ea4144410 2024-04-25T10:12:23.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.612+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e9c01b020 con 0x7f9ea4144410 2024-04-25T10:12:23.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.613+0000 7f9e9be00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e8006b500 0x7f9e8006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:23.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.613+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9e9c089f60 con 0x7f9ea4144410 2024-04-25T10:12:23.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.613+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e8006b500 0x7f9e8006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:23.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.614+0000 7f9ea3e00640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e8006b500 0x7f9e8006d9c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c033000 tx=0x7f9e8c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:23.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.617+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e9c01b310 con 0x7f9ea4144410 2024-04-25T10:12:23.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.815+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ea41655e0 con 0x7f9ea4144410 2024-04-25T10:12:23.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.818+0000 7f9e9be00640 1 -- 172.21.15.165:0/699567746 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+17114 (secure 0 0 0) 0x7f9e9c058b40 con 0x7f9ea4144410 2024-04-25T10:12:23.823 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:23.823 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":100,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:11:47.812589+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:11:20.638187+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_data","create_time":"2024-04-25T10:11:21.928273+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"98","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":1,"to":5}]},{"pgid":"14.a","mappings":[{"from":1,"to":2}]},{"pgid":"14.12","mappings":[{"from":1,"to":7}]},{"pgid":"14.32","mappings":[{"from":1,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e8006b500 msgr2=0x7f9e8006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e8006b500 0x7f9e8006d9c0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c033000 tx=0x7f9e8c008030 comp rx=0 tx=0).stop 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 msgr2=0x7f9ea4144890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 0x7f9ea4144890 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f9ea406ba10 tx=0x7f9e9c00db80 comp rx=0 tx=0).stop 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 shutdown_connections 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e8006b500 0x7f9e8006d9c0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ea4144410 0x7f9ea4144890 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ea415a4f0 0x7f9ea415a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 --2- 172.21.15.165:0/699567746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e880039b0 0x7f9ea4157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 >> 172.21.15.165:0/699567746 conn(0x7f9ea406abe0 msgr2=0x7f9ea4165de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:23.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 shutdown_connections 2024-04-25T10:12:23.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:23.821+0000 7f9ea9800640 1 -- 172.21.15.165:0/699567746 wait complete. 2024-04-25T10:12:23.844 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:12:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.033+0000 7fefad400640 1 Processor -- start 2024-04-25T10:12:24.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.034+0000 7fefad400640 1 -- start start 2024-04-25T10:12:24.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefad400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefa815dd30 0x7fefa80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefad400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fefa806bff0 0x7fefa80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefad400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa80b5970 0x7fefa8161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefad400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa806d000 con 0x7fefa815dd30 2024-04-25T10:12:24.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefad400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefa815e110 con 0x7fefa806bff0 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefad400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80b2fd0 con 0x7fefa80b5970 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefa7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefa815dd30 0x7fefa80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefa7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefa815dd30 0x7fefa80cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48320/0 (socket says 172.21.15.165:48320) 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.035+0000 7fefa7e00640 1 -- 172.21.15.165:0/1408363127 learned_addr learned my addr 172.21.15.165:0/1408363127 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fefaca00640 1 --2- 172.21.15.165:0/1408363127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa80b5970 0x7fefa8161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fef9fe00640 1 --2- 172.21.15.165:0/1408363127 >> v2:172.21.15.172:3300/0 conn(0x7fefa806bff0 0x7fefa80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fefa7e00640 1 -- 172.21.15.165:0/1408363127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa80b5970 msgr2=0x7fefa8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fefa7e00640 1 --2- 172.21.15.165:0/1408363127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa80b5970 0x7fefa8161820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fefa7e00640 1 -- 172.21.15.165:0/1408363127 >> v2:172.21.15.172:3300/0 conn(0x7fefa806bff0 msgr2=0x7fefa80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fefa7e00640 1 --2- 172.21.15.165:0/1408363127 >> v2:172.21.15.172:3300/0 conn(0x7fefa806bff0 0x7fefa80b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.036+0000 7fefa7e00640 1 -- 172.21.15.165:0/1408363127 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa806c480 con 0x7fefa815dd30 2024-04-25T10:12:24.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.037+0000 7fefa7e00640 1 --2- 172.21.15.165:0/1408363127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa815dd30 0x7fefa80cebd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fefa808c2c0 tx=0x7fef9802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3265baa71fec2044 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.037+0000 7fefa7400640 1 -- 172.21.15.165:0/1408363127 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98037070 con 0x7fefa815dd30 2024-04-25T10:12:24.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.037+0000 7fefa7400640 1 -- 172.21.15.165:0/1408363127 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef98032440 con 0x7fefa815dd30 2024-04-25T10:12:24.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.037+0000 7fefa7400640 1 -- 172.21.15.165:0/1408363127 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98032810 con 0x7fefa815dd30 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 -- 172.21.15.165:0/1408363127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa815dd30 msgr2=0x7fefa80cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 --2- 172.21.15.165:0/1408363127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa815dd30 0x7fefa80cebd0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fefa808c2c0 tx=0x7fef9802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 -- 172.21.15.165:0/1408363127 shutdown_connections 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 --2- 172.21.15.165:0/1408363127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa80b5970 0x7fefa8161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 --2- 172.21.15.165:0/1408363127 >> v2:172.21.15.172:3300/0 conn(0x7fefa806bff0 0x7fefa80b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 --2- 172.21.15.165:0/1408363127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa815dd30 0x7fefa80cebd0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 -- 172.21.15.165:0/1408363127 >> 172.21.15.165:0/1408363127 conn(0x7fefa80d0610 msgr2=0x7fefa80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 -- 172.21.15.165:0/1408363127 shutdown_connections 2024-04-25T10:12:24.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.038+0000 7fefad400640 1 -- 172.21.15.165:0/1408363127 wait complete. 2024-04-25T10:12:24.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.040+0000 7fefad400640 1 Processor -- start 2024-04-25T10:12:24.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.040+0000 7fefad400640 1 -- start start 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefad400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa80b5970 0x7fefa80b3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefad400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa815dd30 0x7fefa80caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefad400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 0x7fefa814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefad400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa806cd20 con 0x7fefa80cafc0 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefad400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefa806ca40 con 0x7fefa80b5970 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefad400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefa80cf110 con 0x7fefa815dd30 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefaca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 0x7fefa814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefaca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 0x7fefa814fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48336/0 (socket says 172.21.15.165:48336) 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.041+0000 7fefaca00640 1 -- 172.21.15.165:0/1789345344 learned_addr learned my addr 172.21.15.165:0/1789345344 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefa7e00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa80b5970 0x7fefa80b3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fef9fe00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa815dd30 0x7fefa80caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefaca00640 1 -- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa815dd30 msgr2=0x7fefa80caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefaca00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa815dd30 0x7fefa80caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefaca00640 1 -- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa80b5970 msgr2=0x7fefa80b3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefaca00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa80b5970 0x7fefa80b3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefaca00640 1 -- 172.21.15.165:0/1789345344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa80cbe70 con 0x7fefa80cafc0 2024-04-25T10:12:24.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefaca00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 0x7fefa814fe50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fefa0007e40 tx=0x7fefa000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.042+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0014620 con 0x7fefa80cafc0 2024-04-25T10:12:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.043+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa0019070 con 0x7fefa80cafc0 2024-04-25T10:12:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.043+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefa80cb520 con 0x7fefa80cafc0 2024-04-25T10:12:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.043+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0014a20 con 0x7fefa80cafc0 2024-04-25T10:12:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.043+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefa8150c70 con 0x7fefa80cafc0 2024-04-25T10:12:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.044+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefa80d0af0 con 0x7fefa80cafc0 2024-04-25T10:12:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.045+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefa0014bc0 con 0x7fefa80cafc0 2024-04-25T10:12:24.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.046+0000 7fefa4c00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef8406b4d0 0x7fef8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.046+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fefa008ade0 con 0x7fefa80cafc0 2024-04-25T10:12:24.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.046+0000 7fefa7e00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef8406b4d0 0x7fef8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.047+0000 7fefa7e00640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef8406b4d0 0x7fef8406d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fef98034000 tx=0x7fef9802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.050+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefa80d0af0 con 0x7fefa80cafc0 2024-04-25T10:12:24.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.283+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fefa8073990 con 0x7fefa80cafc0 2024-04-25T10:12:24.288 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:12:24.284+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:12:24.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.287+0000 7fefa4c00640 1 -- 172.21.15.165:0/1789345344 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v66) v1 ==== 161+0+0 (secure 0 0 0) 0x7fefa00599c0 con 0x7fefa80cafc0 2024-04-25T10:12:24.292 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:12:24.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.291+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef8406b4d0 msgr2=0x7fef8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.291+0000 7fefad400640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef8406b4d0 0x7fef8406d990 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fef98034000 tx=0x7fef9802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:24.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.291+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 msgr2=0x7fefa814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.291+0000 7fefad400640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 0x7fefa814fe50 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fefa0007e40 tx=0x7fefa000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 shutdown_connections 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fef8406b4d0 0x7fef8406d990 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefa80cafc0 0x7fefa814fe50 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefa815dd30 0x7fefa80caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 --2- 172.21.15.165:0/1789345344 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefa80b5970 0x7fefa80b3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 >> 172.21.15.165:0/1789345344 conn(0x7fefa80d0610 msgr2=0x7fefa815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 shutdown_connections 2024-04-25T10:12:24.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.292+0000 7fefad400640 1 -- 172.21.15.165:0/1789345344 wait complete. 2024-04-25T10:12:24.317 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:12:24.382 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:12:24.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.496+0000 7f7bd8e00640 1 Processor -- start 2024-04-25T10:12:24.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.496+0000 7f7bd8e00640 1 -- start start 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.498+0000 7f7bd8e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7bd415b980 0x7f7bd40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.498+0000 7f7bd8e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd4072280 con 0x7f7bd40ce230 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.498+0000 7f7bd8e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bd40714c0 con 0x7f7bd40cc4f0 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.498+0000 7f7bd8e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bd4071270 con 0x7f7bd415b980 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.498+0000 7f7bd1400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7bd40ce230 0x7f7bd415b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53498/0 (socket says 172.21.15.165:53498) 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.498+0000 7f7bd1400640 1 -- 172.21.15.165:0/4182810666 learned_addr learned my addr 172.21.15.165:0/4182810666 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:24.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd2800640 1 --2- 172.21.15.165:0/4182810666 >> v2:172.21.15.172:3300/0 conn(0x7f7bd415b980 0x7f7bd40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471396798 0 0) 0x7f7bd4072280 con 0x7f7bd40ce230 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb8003520 con 0x7f7bd40ce230 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 926793785 0 0) 0x7f7bb8003520 con 0x7f7bd40ce230 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 >> v2:172.21.15.172:3300/0 conn(0x7f7bd415b980 msgr2=0x7f7bd40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 --2- 172.21.15.165:0/4182810666 >> v2:172.21.15.172:3300/0 conn(0x7f7bd415b980 0x7f7bd40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 >> v1:172.21.15.172:6789/0 conn(0x7f7bd40cc4f0 legacy=0x7f7bd40cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd40717a0 con 0x7f7bd40ce230 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7bc0001670 con 0x7f7bd40ce230 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 >> v1:172.21.15.165:6789/0 conn(0x7f7bd40ce230 legacy=0x7f7bd415b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bb8003a00 0x7f7bb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bb8004800 0x7f7bb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd4071270 con 0x7f7bb80051c0 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800c0c0 con 0x7f7bb8003a00 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.499+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bb800c2c0 con 0x7f7bb8004800 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd2800640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd1400640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bb8004800 0x7f7bb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd1e00640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bb8003a00 0x7f7bb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd2800640 1 -- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bb8004800 msgr2=0x7f7bb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd2800640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bb8004800 0x7f7bb8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd2800640 1 -- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bb8003a00 msgr2=0x7f7bb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd2800640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bb8003a00 0x7f7bb8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.500+0000 7f7bd2800640 1 -- 172.21.15.165:0/4182810666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb8003520 con 0x7f7bb80051c0 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.501+0000 7f7bd2800640 1 -- 172.21.15.165:0/4182810666 reap_dead start 2024-04-25T10:12:24.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.501+0000 7f7bd2800640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bb800b9c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009fc0 tx=0x7f7bb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe2ab6d3d3eb2c21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc035020 con 0x7f7bb80051c0 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bcc003b90 con 0x7f7bb80051c0 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd0a00640 1 -- 172.21.15.165:0/4182810666 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bcc009c10 con 0x7f7bb80051c0 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 -- 172.21.15.165:0/4182810666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 msgr2=0x7f7bb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bb800b9c0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc009fc0 tx=0x7f7bb8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 -- 172.21.15.165:0/4182810666 shutdown_connections 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/4182810666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bb800b9c0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 -- 172.21.15.165:0/4182810666 >> 172.21.15.165:0/4182810666 conn(0x7f7bd406abe0 msgr2=0x7f7bd406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:24.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 -- 172.21.15.165:0/4182810666 shutdown_connections 2024-04-25T10:12:24.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.502+0000 7f7bd8e00640 1 -- 172.21.15.165:0/4182810666 wait complete. 2024-04-25T10:12:24.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.504+0000 7f7bd8e00640 1 Processor -- start 2024-04-25T10:12:24.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.504+0000 7f7bd8e00640 1 -- start start 2024-04-25T10:12:24.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bd413f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd4157490 0x7f7bd41e7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd8e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd41e7ab0 0x7f7bd415bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd40cc8d0 con 0x7f7bb80051c0 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd8e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd4071d60 con 0x7f7bd4157490 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd8e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd40cca50 con 0x7f7bd41e7ab0 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bd413f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bd413f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48360/0 (socket says 172.21.15.165:48360) 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.505+0000 7f7bd1e00640 1 -- 172.21.15.165:0/3547842645 learned_addr learned my addr 172.21.15.165:0/3547842645 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd2800640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd41e7ab0 0x7f7bd415bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1400640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd4157490 0x7f7bd41e7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1e00640 1 -- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd41e7ab0 msgr2=0x7f7bd415bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd41e7ab0 0x7f7bd415bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1e00640 1 -- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd4157490 msgr2=0x7f7bd41e7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:24.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd4157490 0x7f7bd41e7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1e00640 1 -- 172.21.15.165:0/3547842645 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bd41559e0 con 0x7f7bb80051c0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd1e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bd413f1a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400e530 tx=0x7f7bc400e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc40052e0 con 0x7f7bb80051c0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bc4018070 con 0x7f7bb80051c0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bc4003810 con 0x7f7bb80051c0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.506+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd4155cc0 con 0x7f7bb80051c0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.507+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd4156280 con 0x7f7bb80051c0 2024-04-25T10:12:24.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.508+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd4073a00 con 0x7f7bb80051c0 2024-04-25T10:12:24.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.509+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7bc40039b0 con 0x7f7bb80051c0 2024-04-25T10:12:24.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.510+0000 7f7bca000640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb406b500 0x7f7bb406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:24.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.510+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7bc408a0e0 con 0x7f7bb80051c0 2024-04-25T10:12:24.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.510+0000 7f7bd1400640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb406b500 0x7f7bb406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:24.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.511+0000 7f7bd1400640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb406b500 0x7f7bb406d9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00017e0 tx=0x7f7bc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:24.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.515+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7bc4058be0 con 0x7f7bb80051c0 2024-04-25T10:12:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:24.752+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7bd4151850 con 0x7f7bb80051c0 2024-04-25T10:12:25.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.297+0000 7f7bca000640 1 -- 172.21.15.165:0/3547842645 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v67) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7bc40584e0 con 0x7f7bb80051c0 2024-04-25T10:12:25.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb406b500 msgr2=0x7f7bb406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb406b500 0x7f7bb406d9c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f7bc00017e0 tx=0x7f7bc0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:25.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 msgr2=0x7f7bd413f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bd413f1a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f7bc400e530 tx=0x7f7bc400e9f0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 shutdown_connections 2024-04-25T10:12:25.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7bb406b500 0x7f7bb406d9c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7bd41e7ab0 0x7f7bd415bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7bd4157490 0x7f7bd41e7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 --2- 172.21.15.165:0/3547842645 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7bb80051c0 0x7f7bd413f1a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 >> 172.21.15.165:0/3547842645 conn(0x7f7bd406abe0 msgr2=0x7f7bd415b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:25.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 shutdown_connections 2024-04-25T10:12:25.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.301+0000 7f7bd8e00640 1 -- 172.21.15.165:0/3547842645 wait complete. 2024-04-25T10:12:25.327 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:12:25.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.513+0000 7f6e51c00640 1 Processor -- start 2024-04-25T10:12:25.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.513+0000 7f6e51c00640 1 -- start start 2024-04-25T10:12:25.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e51c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e4c168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e51c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c15e110 con 0x7f6e4c06bff0 2024-04-25T10:12:25.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e51c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e4c0b2fd0 con 0x7f6e4c15dd30 2024-04-25T10:12:25.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e51c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c06d000 con 0x7f6e4c0b5b40 2024-04-25T10:12:25.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e43400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6e4c06bff0 0x7f6e4c0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53508/0 (socket says 172.21.15.165:53508) 2024-04-25T10:12:25.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e43400640 1 -- 172.21.15.165:0/820332376 learned_addr learned my addr 172.21.15.165:0/820332376 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:25.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188135387 0 0) 0x7f6e4c15e110 con 0x7f6e4c06bff0 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e51200640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e4c168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.515+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e30003520 con 0x7f6e4c06bff0 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4005521824 0 0) 0x7f6e30003520 con 0x7f6e4c06bff0 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 msgr2=0x7f6e4c168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e4c168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 >> v1:172.21.15.172:6789/0 conn(0x7f6e4c15dd30 legacy=0x7f6e4c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c06c480 con 0x7f6e4c06bff0 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6e34001670 con 0x7f6e4c06bff0 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 >> v1:172.21.15.165:6789/0 conn(0x7f6e4c06bff0 legacy=0x7f6e4c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30004040 0x7f6e30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c06d000 con 0x7f6e30004d40 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3000bbb0 con 0x7f6e30004040 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e3000bdb0 con 0x7f6e4c0b5b40 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e51200640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e4be00640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.516+0000 7f6e51200640 1 -- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 msgr2=0x7f6e300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.517+0000 7f6e43400640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30004040 0x7f6e30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.517+0000 7f6e51200640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.517+0000 7f6e51200640 1 -- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30004040 msgr2=0x7f6e30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.517+0000 7f6e51200640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e30004040 0x7f6e30004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.517+0000 7f6e51200640 1 -- 172.21.15.165:0/820332376 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e30003520 con 0x7f6e30004d40 2024-04-25T10:12:25.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.517+0000 7f6e4be00640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c0b5b40 0x7f6e300039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e51200640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400ba10 tx=0x7f6e30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed5ea9b40e57fdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e44030070 con 0x7f6e30004d40 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4402adb0 con 0x7f6e30004d40 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e4b400640 1 -- 172.21.15.165:0/820332376 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4400b610 con 0x7f6e30004d40 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e51c00640 1 -- 172.21.15.165:0/820332376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 msgr2=0x7f6e300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e51c00640 1 --2- 172.21.15.165:0/820332376 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e300051c0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f6e4400ba10 tx=0x7f6e30005830 comp rx=0 tx=0).stop 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e51200640 1 -- 172.21.15.165:0/820332376 reap_dead start 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e51c00640 1 -- 172.21.15.165:0/820332376 shutdown_connections 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.518+0000 7f6e51c00640 1 -- 172.21.15.165:0/820332376 >> 172.21.15.165:0/820332376 conn(0x7f6e4c0d0610 msgr2=0x7f6e4c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.519+0000 7f6e51c00640 1 -- 172.21.15.165:0/820332376 shutdown_connections 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.519+0000 7f6e51c00640 1 -- 172.21.15.165:0/820332376 wait complete. 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.520+0000 7f6e51c00640 1 Processor -- start 2024-04-25T10:12:25.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.520+0000 7f6e51c00640 1 -- start start 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.521+0000 7f6e51c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e4c15a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.521+0000 7f6e51c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4c159f50 0x7f6e4c150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.521+0000 7f6e51c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c160d60 0x7f6e4c1611e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.521+0000 7f6e51c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c06cd20 con 0x7f6e30004d40 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.521+0000 7f6e51c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c06ca40 con 0x7f6e4c159f50 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.521+0000 7f6e51c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4c0d3170 con 0x7f6e4c160d60 2024-04-25T10:12:25.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e4c15a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e4c15a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48370/0 (socket says 172.21.15.165:48370) 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 -- 172.21.15.165:0/3690330228 learned_addr learned my addr 172.21.15.165:0/3690330228 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 -- 172.21.15.165:0/3690330228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c160d60 msgr2=0x7f6e4c1611e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c160d60 0x7f6e4c1611e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 -- 172.21.15.165:0/3690330228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4c159f50 msgr2=0x7f6e4c150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4c159f50 0x7f6e4c150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 -- 172.21.15.165:0/3690330228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c151230 con 0x7f6e30004d40 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e4be00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e4c15a4a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00d930 tx=0x7f6e3c00ddf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c016070 con 0x7f6e30004d40 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e3c006d50 con 0x7f6e30004d40 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.522+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e3c00bb40 con 0x7f6e30004d40 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.523+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4c15f9e0 con 0x7f6e30004d40 2024-04-25T10:12:25.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.523+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4c15ffa0 con 0x7f6e30004d40 2024-04-25T10:12:25.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.524+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e4c073990 con 0x7f6e30004d40 2024-04-25T10:12:25.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.525+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e3c00bce0 con 0x7f6e30004d40 2024-04-25T10:12:25.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.526+0000 7f6e48c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e2806b500 0x7f6e2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:25.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.526+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6e3c08a5e0 con 0x7f6e30004d40 2024-04-25T10:12:25.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.526+0000 7f6e43400640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e2806b500 0x7f6e2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:25.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.526+0000 7f6e43400640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e2806b500 0x7f6e2806d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0d4480 tx=0x7f6e34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:25.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.529+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e3c059170 con 0x7f6e30004d40 2024-04-25T10:12:25.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:25.724+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e4c0d16c0 con 0x7f6e30004d40 2024-04-25T10:12:26.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.301+0000 7f6e48c00640 1 -- 172.21.15.165:0/3690330228 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v102) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6e3c058a70 con 0x7f6e30004d40 2024-04-25T10:12:26.306 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:12:26.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.304+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e2806b500 msgr2=0x7f6e2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.304+0000 7f6e51c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e2806b500 0x7f6e2806d9c0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0d4480 tx=0x7f6e34009040 comp rx=0 tx=0).stop 2024-04-25T10:12:26.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.304+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 msgr2=0x7f6e4c15a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.304+0000 7f6e51c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e4c15a4a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f6e3c00d930 tx=0x7f6e3c00ddf0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 shutdown_connections 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e2806b500 0x7f6e2806d9c0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e4c160d60 0x7f6e4c1611e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e4c159f50 0x7f6e4c150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 --2- 172.21.15.165:0/3690330228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e30004d40 0x7f6e4c15a4a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 >> 172.21.15.165:0/3690330228 conn(0x7f6e4c0d0610 msgr2=0x7f6e4c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:26.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.305+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 shutdown_connections 2024-04-25T10:12:26.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.306+0000 7f6e51c00640 1 -- 172.21.15.165:0/3690330228 wait complete. 2024-04-25T10:12:26.333 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:12:26.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.518+0000 7fdf3d000640 1 Processor -- start 2024-04-25T10:12:26.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.518+0000 7fdf3d000640 1 -- start start 2024-04-25T10:12:26.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf3d000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdf3815b710 0x7fdf380cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf3d000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf380ac8b0 0x7fdf380acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf3d000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf3806f0e0 con 0x7fdf3806ec50 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf3d000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf3815e490 con 0x7fdf3815b710 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf3d000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf380cfd80 con 0x7fdf380ac8b0 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf35e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdf3815b710 0x7fdf380cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf35e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdf3815b710 0x7fdf380cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48384/0 (socket says 172.21.15.165:48384) 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf35e00640 1 -- 172.21.15.165:0/567352070 learned_addr learned my addr 172.21.15.165:0/567352070 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf35400640 1 --1- 172.21.15.165:0/567352070 >> v1:172.21.15.165:6789/0 conn(0x7fdf3806ec50 0x7fdf3815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53520/0 (socket says 172.21.15.165:53520) 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.520+0000 7fdf36800640 1 --2- 172.21.15.165:0/567352070 >> v2:172.21.15.172:3300/0 conn(0x7fdf380ac8b0 0x7fdf380acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.521+0000 7fdf35e00640 1 -- 172.21.15.165:0/567352070 >> v2:172.21.15.172:3300/0 conn(0x7fdf380ac8b0 msgr2=0x7fdf380acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.521+0000 7fdf35e00640 1 --2- 172.21.15.165:0/567352070 >> v2:172.21.15.172:3300/0 conn(0x7fdf380ac8b0 0x7fdf380acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.521+0000 7fdf35e00640 1 -- 172.21.15.165:0/567352070 >> v1:172.21.15.165:6789/0 conn(0x7fdf3806ec50 legacy=0x7fdf3815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.521+0000 7fdf35e00640 1 -- 172.21.15.165:0/567352070 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf380d0060 con 0x7fdf3815b710 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.522+0000 7fdf35e00640 1 --2- 172.21.15.165:0/567352070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf3815b710 0x7fdf380cf640 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdf380cd2e0 tx=0x7fdf2802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f50cafddd3ad3dab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.522+0000 7fdf34a00640 1 -- 172.21.15.165:0/567352070 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28037070 con 0x7fdf3815b710 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.522+0000 7fdf34a00640 1 -- 172.21.15.165:0/567352070 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf28032440 con 0x7fdf3815b710 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.522+0000 7fdf34a00640 1 -- 172.21.15.165:0/567352070 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf28032810 con 0x7fdf3815b710 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 -- 172.21.15.165:0/567352070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf3815b710 msgr2=0x7fdf380cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 --2- 172.21.15.165:0/567352070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf3815b710 0x7fdf380cf640 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fdf380cd2e0 tx=0x7fdf2802a270 comp rx=0 tx=0).stop 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 -- 172.21.15.165:0/567352070 shutdown_connections 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 --2- 172.21.15.165:0/567352070 >> v2:172.21.15.172:3300/0 conn(0x7fdf380ac8b0 0x7fdf380acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 --2- 172.21.15.165:0/567352070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf3815b710 0x7fdf380cf640 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 -- 172.21.15.165:0/567352070 >> 172.21.15.165:0/567352070 conn(0x7fdf3806abe0 msgr2=0x7fdf3806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 -- 172.21.15.165:0/567352070 shutdown_connections 2024-04-25T10:12:26.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.523+0000 7fdf3d000640 1 -- 172.21.15.165:0/567352070 wait complete. 2024-04-25T10:12:26.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.525+0000 7fdf3d000640 1 Processor -- start 2024-04-25T10:12:26.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.525+0000 7fdf3d000640 1 -- start start 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf3806ec50 0x7fdf380c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 0x7fdf38156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf3d000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf3815b710 0x7fdf381594c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf3d000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3815bc70 con 0x7fdf380ac8b0 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf3d000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3815baf0 con 0x7fdf3806ec50 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf3d000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf3815f030 con 0x7fdf3815b710 2024-04-25T10:12:26.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 0x7fdf38156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 0x7fdf38156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48386/0 (socket says 172.21.15.165:48386) 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 -- 172.21.15.165:0/2672768871 learned_addr learned my addr 172.21.15.165:0/2672768871 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf36800640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf3815b710 0x7fdf381594c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 -- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf3815b710 msgr2=0x7fdf381594c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35e00640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf3806ec50 0x7fdf380c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf3815b710 0x7fdf381594c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 -- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf3806ec50 msgr2=0x7fdf380c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf3806ec50 0x7fdf380c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.526+0000 7fdf35400640 1 -- 172.21.15.165:0/2672768871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf381f3900 con 0x7fdf380ac8b0 2024-04-25T10:12:26.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.527+0000 7fdf35400640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 0x7fdf38156630 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c400 tx=0x7fdf2400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:26.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.527+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf240074e0 con 0x7fdf380ac8b0 2024-04-25T10:12:26.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.527+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf24007680 con 0x7fdf380ac8b0 2024-04-25T10:12:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.527+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24004a00 con 0x7fdf380ac8b0 2024-04-25T10:12:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.527+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf381f3be0 con 0x7fdf380ac8b0 2024-04-25T10:12:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.527+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf381f41a0 con 0x7fdf380ac8b0 2024-04-25T10:12:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.528+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf38073a00 con 0x7fdf380ac8b0 2024-04-25T10:12:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.529+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf2401a020 con 0x7fdf380ac8b0 2024-04-25T10:12:26.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.530+0000 7fdf2e000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf1806b4d0 0x7fdf1806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:26.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.530+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5865+0+0 (secure 0 0 0) 0x7fdf24089af0 con 0x7fdf380ac8b0 2024-04-25T10:12:26.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.531+0000 7fdf35e00640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf1806b4d0 0x7fdf1806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:26.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.531+0000 7fdf35e00640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf1806b4d0 0x7fdf1806d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdf28034000 tx=0x7fdf2802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.534+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf24058830 con 0x7fdf380ac8b0 2024-04-25T10:12:26.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:26.732+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fdf3815eca0 con 0x7fdf380ac8b0 2024-04-25T10:12:27.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.311+0000 7fdf2e000640 1 -- 172.21.15.165:0/2672768871 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v103) v1 ==== 169+0+0 (secure 0 0 0) 0x7fdf24058130 con 0x7fdf380ac8b0 2024-04-25T10:12:27.317 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:12:27.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.316+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf1806b4d0 msgr2=0x7fdf1806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.316+0000 7fdf3d000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf1806b4d0 0x7fdf1806d990 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fdf28034000 tx=0x7fdf2802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:27.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.316+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 msgr2=0x7fdf38156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.316+0000 7fdf3d000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 0x7fdf38156630 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fdf2400c400 tx=0x7fdf2400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.318+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 shutdown_connections 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.318+0000 7fdf3d000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf1806b4d0 0x7fdf1806d990 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.318+0000 7fdf3d000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf3815b710 0x7fdf381594c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.318+0000 7fdf3d000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf380ac8b0 0x7fdf38156630 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.318+0000 7fdf3d000640 1 --2- 172.21.15.165:0/2672768871 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf3806ec50 0x7fdf380c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.319+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 >> 172.21.15.165:0/2672768871 conn(0x7fdf3806abe0 msgr2=0x7fdf3815c9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.319+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 shutdown_connections 2024-04-25T10:12:27.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.319+0000 7fdf3d000640 1 -- 172.21.15.165:0/2672768871 wait complete. 2024-04-25T10:12:27.352 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:12:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.574+0000 7f71f7a00640 1 Processor -- start 2024-04-25T10:12:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.574+0000 7f71f7a00640 1 -- start start 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.575+0000 7f71f7a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71f00ced80 0x7f71f00d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.575+0000 7f71f7a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71f0161710 0x7f71f00d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.575+0000 7f71f7a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f015e000 con 0x7f71f015dc20 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.575+0000 7f71f7a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f00d1de0 con 0x7f71f0161710 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.575+0000 7f71f7a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f0161c60 con 0x7f71f00ced80 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71f0161710 0x7f71f00d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71f0161710 0x7f71f00d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41424/0 (socket says 172.21.15.165:41424) 2024-04-25T10:12:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 -- 172.21.15.165:0/2848345749 learned_addr learned my addr 172.21.15.165:0/2848345749 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5000640 1 --1- 172.21.15.165:0/2848345749 >> v1:172.21.15.165:6789/0 conn(0x7f71f015dc20 0x7f71f00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44170/0 (socket says 172.21.15.165:44170) 2024-04-25T10:12:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71efe00640 1 --2- 172.21.15.165:0/2848345749 >> v2:172.21.15.172:3300/0 conn(0x7f71f00ced80 0x7f71f00d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 -- 172.21.15.165:0/2848345749 >> v2:172.21.15.172:3300/0 conn(0x7f71f00ced80 msgr2=0x7f71f00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 --2- 172.21.15.165:0/2848345749 >> v2:172.21.15.172:3300/0 conn(0x7f71f00ced80 0x7f71f00d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 -- 172.21.15.165:0/2848345749 >> v1:172.21.15.165:6789/0 conn(0x7f71f015dc20 legacy=0x7f71f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71f5a00640 1 -- 172.21.15.165:0/2848345749 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f0161f40 con 0x7f71f0161710 2024-04-25T10:12:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.576+0000 7f71efe00640 1 --2- 172.21.15.165:0/2848345749 >> v2:172.21.15.172:3300/0 conn(0x7f71f00ced80 0x7f71f00d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.577+0000 7f71f5a00640 1 --2- 172.21.15.165:0/2848345749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f0161710 0x7f71f00d2820 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f71f006da90 tx=0x7f71e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae2d0780e3ddee24 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.577+0000 7f71ef400640 1 -- 172.21.15.165:0/2848345749 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8037070 con 0x7f71f0161710 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.577+0000 7f71ef400640 1 -- 172.21.15.165:0/2848345749 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8032440 con 0x7f71f0161710 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.577+0000 7f71ef400640 1 -- 172.21.15.165:0/2848345749 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8032810 con 0x7f71f0161710 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 -- 172.21.15.165:0/2848345749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f0161710 msgr2=0x7f71f00d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2848345749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f0161710 0x7f71f00d2820 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f71f006da90 tx=0x7f71e802a270 comp rx=0 tx=0).stop 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 -- 172.21.15.165:0/2848345749 shutdown_connections 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2848345749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f0161710 0x7f71f00d2820 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2848345749 >> v2:172.21.15.172:3300/0 conn(0x7f71f00ced80 0x7f71f00d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 -- 172.21.15.165:0/2848345749 >> 172.21.15.165:0/2848345749 conn(0x7f71f00d0420 msgr2=0x7f71f00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 -- 172.21.15.165:0/2848345749 shutdown_connections 2024-04-25T10:12:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.578+0000 7f71f7a00640 1 -- 172.21.15.165:0/2848345749 wait complete. 2024-04-25T10:12:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.580+0000 7f71f7a00640 1 Processor -- start 2024-04-25T10:12:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.580+0000 7f71f7a00640 1 -- start start 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f7a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 0x7f71f00cd540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f7a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015dc20 0x7f71f0154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f7a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f0161710 0x7f71f015f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f7a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00b5e50 con 0x7f71f00ced80 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f7a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0162500 con 0x7f71f0161710 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f7a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00b5fd0 con 0x7f71f015dc20 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 0x7f71f00cd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 0x7f71f00cd540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41426/0 (socket says 172.21.15.165:41426) 2024-04-25T10:12:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.581+0000 7f71f5000640 1 -- 172.21.15.165:0/2376751130 learned_addr learned my addr 172.21.15.165:0/2376751130 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71efe00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015dc20 0x7f71f0154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f0161710 0x7f71f015f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5000640 1 -- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015dc20 msgr2=0x7f71f0154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5000640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015dc20 0x7f71f0154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5000640 1 -- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f0161710 msgr2=0x7f71f015f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5000640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f0161710 0x7f71f015f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5000640 1 -- 172.21.15.165:0/2376751130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00cc4a0 con 0x7f71f00ced80 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f5000640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 0x7f71f00cd540 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c760 tx=0x7f71e000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0017070 con 0x7f71f00ced80 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e000a640 con 0x7f71f00ced80 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000aa10 con 0x7f71f00ced80 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f00cc780 con 0x7f71f00ced80 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.582+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f00ccd40 con 0x7f71f00ced80 2024-04-25T10:12:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.584+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f0073a00 con 0x7f71f00ced80 2024-04-25T10:12:27.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.585+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71e001a070 con 0x7f71f00ced80 2024-04-25T10:12:27.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.585+0000 7f71ecc00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71d006b4d0 0x7f71d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:27.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.586+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f71e0089790 con 0x7f71f00ced80 2024-04-25T10:12:27.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.586+0000 7f71efe00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71d006b4d0 0x7f71d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:27.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.586+0000 7f71efe00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71d006b4d0 0x7f71d006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f71f00d3840 tx=0x7f71dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:27.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.590+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71e0058650 con 0x7f71f00ced80 2024-04-25T10:12:27.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.782+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f71f00d16b0 con 0x7f71f00ced80 2024-04-25T10:12:27.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.788+0000 7f71ecc00640 1 -- 172.21.15.165:0/2376751130 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f71e0089070 con 0x7f71f00ced80 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71d006b4d0 msgr2=0x7f71d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71d006b4d0 0x7f71d006d990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f71f00d3840 tx=0x7f71dc008040 comp rx=0 tx=0).stop 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 msgr2=0x7f71f00cd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 0x7f71f00cd540 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7f71e000c760 tx=0x7f71e000cc20 comp rx=0 tx=0).stop 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 shutdown_connections 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71d006b4d0 0x7f71d006d990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f0161710 0x7f71f015f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015dc20 0x7f71f0154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 --2- 172.21.15.165:0/2376751130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f00ced80 0x7f71f00cd540 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 >> 172.21.15.165:0/2376751130 conn(0x7f71f00d0420 msgr2=0x7f71f00d0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:27.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 shutdown_connections 2024-04-25T10:12:27.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:27.791+0000 7f71f7a00640 1 -- 172.21.15.165:0/2376751130 wait complete. 2024-04-25T10:12:27.813 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort' 2024-04-25T10:12:28.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.007+0000 7fece7000640 1 Processor -- start 2024-04-25T10:12:28.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.007+0000 7fece7000640 1 -- start start 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.009+0000 7fece7000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fece015dc00 0x7fece00c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.009+0000 7fece7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 0x7fece00d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.009+0000 7fece7000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece015dfe0 con 0x7fece00d1c00 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.009+0000 7fece7000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece00d2060 con 0x7fece015dc00 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.009+0000 7fece7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00d00c0 con 0x7fece00ceb20 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fece5000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fece00d1c00 0x7fece00ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44176/0 (socket says 172.21.15.165:44176) 2024-04-25T10:12:28.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fece5000640 1 -- 172.21.15.165:0/3126033780 learned_addr learned my addr 172.21.15.165:0/3126033780 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:28.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdfe00640 1 --2- 172.21.15.165:0/3126033780 >> v2:172.21.15.172:3300/0 conn(0x7fece015dc00 0x7fece00c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdf400640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 0x7fece00d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2983095676 0 0) 0x7fecd8001110 con 0x7fece00d1c00 2024-04-25T10:12:28.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fecc0003520 con 0x7fece00d1c00 2024-04-25T10:12:28.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 706192969 0 0) 0x7fecc0003520 con 0x7fece00d1c00 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 msgr2=0x7fece00d16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 0x7fece00d16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 >> v2:172.21.15.172:3300/0 conn(0x7fece015dc00 msgr2=0x7fece00c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.010+0000 7fecdea00640 1 --2- 172.21.15.165:0/3126033780 >> v2:172.21.15.172:3300/0 conn(0x7fece015dc00 0x7fece00c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00d03a0 con 0x7fece00d1c00 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fecd8001840 con 0x7fece00d1c00 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 >> v1:172.21.15.165:6789/0 conn(0x7fece00d1c00 legacy=0x7fece00ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecc00039b0 0x7fecc0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecc0004800 0x7fecc0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 0x7fecc00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00d2060 con 0x7fecc0004800 2024-04-25T10:12:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00d00c0 con 0x7fecc00039b0 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc000b9c0 con 0x7fece00ceb20 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdf400640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecc0004800 0x7fecc0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdf400640 1 -- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 msgr2=0x7fecc00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdf400640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece00ceb20 0x7fecc00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdf400640 1 -- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecc00039b0 msgr2=0x7fecc0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdf400640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fecc00039b0 0x7fecc0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.011+0000 7fecdf400640 1 -- 172.21.15.165:0/3126033780 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc0003520 con 0x7fecc0004800 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.012+0000 7fecdf400640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecc0004800 0x7fecc0004c80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009fc0 tx=0x7fecc0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=411edd359e1f9b5c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc8009c10 con 0x7fecc0004800 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc8036070 con 0x7fecc0004800 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fecdea00640 1 -- 172.21.15.165:0/3126033780 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc802d070 con 0x7fecc0004800 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece7000640 1 -- 172.21.15.165:0/3126033780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecc0004800 msgr2=0x7fecc0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece7000640 1 --2- 172.21.15.165:0/3126033780 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fecc0004800 0x7fecc0004c80 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fecc8009fc0 tx=0x7fecc0005830 comp rx=0 tx=0).stop 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece5000640 1 -- 172.21.15.165:0/3126033780 reap_dead start 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece7000640 1 -- 172.21.15.165:0/3126033780 shutdown_connections 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece7000640 1 -- 172.21.15.165:0/3126033780 >> 172.21.15.165:0/3126033780 conn(0x7fece006f5d0 msgr2=0x7fece006f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece7000640 1 -- 172.21.15.165:0/3126033780 shutdown_connections 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.013+0000 7fece7000640 1 -- 172.21.15.165:0/3126033780 wait complete. 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.015+0000 7fece7000640 1 Processor -- start 2024-04-25T10:12:28.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.015+0000 7fece7000640 1 -- start start 2024-04-25T10:12:28.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.016+0000 7fece7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecc0004800 0x7fece0147170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.016+0000 7fece7000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b6cf0 0x7fece00b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.016+0000 7fece7000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 0x7fece00b4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.016+0000 7fece7000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00cc130 con 0x7fece00b44e0 2024-04-25T10:12:28.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.016+0000 7fece7000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece00cc720 con 0x7fece00b6cf0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.016+0000 7fece7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00cfd90 con 0x7fecc0004800 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 0x7fece00b4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 0x7fece00b4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41450/0 (socket says 172.21.15.165:41450) 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 -- 172.21.15.165:0/3140037170 learned_addr learned my addr 172.21.15.165:0/3140037170 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fecdfe00640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecc0004800 0x7fece0147170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fecdf400640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b6cf0 0x7fece00b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 -- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecc0004800 msgr2=0x7fece0147170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecc0004800 0x7fece0147170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 -- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b6cf0 msgr2=0x7fece00b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b6cf0 0x7fece00b7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 -- 172.21.15.165:0/3140037170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece0153e10 con 0x7fece00b44e0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.017+0000 7fece5000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 0x7fece00b4960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fecd8001050 tx=0x7fecd800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.018+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8007230 con 0x7fece00b44e0 2024-04-25T10:12:28.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.018+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd80073d0 con 0x7fece00b44e0 2024-04-25T10:12:28.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.018+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd80047e0 con 0x7fece00b44e0 2024-04-25T10:12:28.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.018+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece01540f0 con 0x7fece00b44e0 2024-04-25T10:12:28.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.018+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece01546b0 con 0x7fece00b44e0 2024-04-25T10:12:28.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.019+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece0073990 con 0x7fece00b44e0 2024-04-25T10:12:28.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.020+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecd8011070 con 0x7fece00b44e0 2024-04-25T10:12:28.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.021+0000 7fecd7e00640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecbc06b4d0 0x7fecbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.021+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fecd8089450 con 0x7fece00b44e0 2024-04-25T10:12:28.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.021+0000 7fecdfe00640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecbc06b4d0 0x7fecbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.022+0000 7fecdfe00640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecbc06b4d0 0x7fecbc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fecd0004770 tx=0x7fecd0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.025+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fece015dfe0 con 0x7fece00b44e0 2024-04-25T10:12:28.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.289+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]} v 0) v1 -- 0x7fece00c66c0 con 0x7fece00b44e0 2024-04-25T10:12:28.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.314+0000 7fecd7e00640 1 -- 172.21.15.165:0/3140037170 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_abort"]}]=0 v479) v1 ==== 144+0+0 (secure 0 0 0) 0x7fecd80583a0 con 0x7fece00b44e0 2024-04-25T10:12:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecbc06b4d0 msgr2=0x7fecbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecbc06b4d0 0x7fecbc06d990 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fecd0004770 tx=0x7fecd0008040 comp rx=0 tx=0).stop 2024-04-25T10:12:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 msgr2=0x7fece00b4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 0x7fece00b4960 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fecd8001050 tx=0x7fecd800cb90 comp rx=0 tx=0).stop 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 shutdown_connections 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecbc06b4d0 0x7fecbc06d990 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b44e0 0x7fece00b4960 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b6cf0 0x7fece00b7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 --2- 172.21.15.165:0/3140037170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fecc0004800 0x7fece0147170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.318+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 >> 172.21.15.165:0/3140037170 conn(0x7fece006f5d0 msgr2=0x7fece00b2420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.319+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 shutdown_connections 2024-04-25T10:12:28.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.319+0000 7fece7000640 1 -- 172.21.15.165:0/3140037170 wait complete. 2024-04-25T10:12:28.342 INFO:tasks.cephfs_test_runner:test_scrub_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:12:28.343 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:12:28.343 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:28.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.541+0000 7f9224c00640 1 Processor -- start 2024-04-25T10:12:28.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.541+0000 7f9224c00640 1 -- start start 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.542+0000 7f9224c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f922006e870 0x7f92200cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.542+0000 7f9224c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f92200cc840 0x7f922015ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.542+0000 7f9224c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9220144240 con 0x7f9220143e60 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.542+0000 7f9224c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922015e220 con 0x7f92200cc840 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.542+0000 7f9224c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f922015e500 con 0x7f922006e870 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.543+0000 7f921fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f92200cc840 0x7f922015ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.543+0000 7f921fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f92200cc840 0x7f922015ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41458/0 (socket says 172.21.15.165:41458) 2024-04-25T10:12:28.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.543+0000 7f921fe00640 1 -- 172.21.15.165:0/9318559 learned_addr learned my addr 172.21.15.165:0/9318559 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:28.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.543+0000 7f9216a00640 1 --2- 172.21.15.165:0/9318559 >> v2:172.21.15.172:3300/0 conn(0x7f922006e870 0x7f92200cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.543+0000 7f921fe00640 1 -- 172.21.15.165:0/9318559 >> v2:172.21.15.172:3300/0 conn(0x7f922006e870 msgr2=0x7f92200cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.543+0000 7f921fe00640 1 --2- 172.21.15.165:0/9318559 >> v2:172.21.15.172:3300/0 conn(0x7f922006e870 0x7f92200cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.544+0000 7f921fe00640 1 -- 172.21.15.165:0/9318559 >> v1:172.21.15.172:6789/0 conn(0x7f9220143e60 legacy=0x7f922006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.544+0000 7f921fe00640 1 -- 172.21.15.165:0/9318559 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922015e7e0 con 0x7f92200cc840 2024-04-25T10:12:28.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.545+0000 7f921fe00640 1 --2- 172.21.15.165:0/9318559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92200cc840 0x7f922015ab40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f922006b970 tx=0x7f921802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9b24dea2b6cf514 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.545+0000 7f921ea00640 1 -- 172.21.15.165:0/9318559 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9218036070 con 0x7f92200cc840 2024-04-25T10:12:28.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.545+0000 7f921ea00640 1 -- 172.21.15.165:0/9318559 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9218031600 con 0x7f92200cc840 2024-04-25T10:12:28.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.545+0000 7f921ea00640 1 -- 172.21.15.165:0/9318559 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92180319d0 con 0x7f92200cc840 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.545+0000 7f9224c00640 1 -- 172.21.15.165:0/9318559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92200cc840 msgr2=0x7f922015ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.545+0000 7f9224c00640 1 --2- 172.21.15.165:0/9318559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92200cc840 0x7f922015ab40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f922006b970 tx=0x7f921802a730 comp rx=0 tx=0).stop 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.546+0000 7f9224c00640 1 -- 172.21.15.165:0/9318559 shutdown_connections 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.546+0000 7f9224c00640 1 --2- 172.21.15.165:0/9318559 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92200cc840 0x7f922015ab40 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.546+0000 7f9224c00640 1 --2- 172.21.15.165:0/9318559 >> v2:172.21.15.172:3300/0 conn(0x7f922006e870 0x7f92200cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.546+0000 7f9224c00640 1 -- 172.21.15.165:0/9318559 >> 172.21.15.165:0/9318559 conn(0x7f922006abe0 msgr2=0x7f922006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.546+0000 7f9224c00640 1 -- 172.21.15.165:0/9318559 shutdown_connections 2024-04-25T10:12:28.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.546+0000 7f9224c00640 1 -- 172.21.15.165:0/9318559 wait complete. 2024-04-25T10:12:28.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.547+0000 7f9224c00640 1 Processor -- start 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.547+0000 7f9224c00640 1 -- start start 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.548+0000 7f9224c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 0x7f922013da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.548+0000 7f9224c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92200cc840 0x7f9220140810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.548+0000 7f9224c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9220143e60 0x7f92200a8660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.548+0000 7f9224c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92200b2eb0 con 0x7f922006e870 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.548+0000 7f9224c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92200b2d30 con 0x7f9220143e60 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.548+0000 7f9224c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92200ccca0 con 0x7f92200cc840 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 0x7f922013da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 0x7f922013da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41462/0 (socket says 172.21.15.165:41462) 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 -- 172.21.15.165:0/2142247688 learned_addr learned my addr 172.21.15.165:0/2142247688 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 -- 172.21.15.165:0/2142247688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92200cc840 msgr2=0x7f9220140810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92200cc840 0x7f9220140810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 -- 172.21.15.165:0/2142247688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9220143e60 msgr2=0x7f92200a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9220143e60 0x7f92200a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 -- 172.21.15.165:0/2142247688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922013fa50 con 0x7f922006e870 2024-04-25T10:12:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.549+0000 7f921f400640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 0x7f922013da60 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f921000c3c0 tx=0x7f921000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.550+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92100052d0 con 0x7f922006e870 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.550+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9210005490 con 0x7f922006e870 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.550+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9210007260 con 0x7f922006e870 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.550+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f922013fd30 con 0x7f922006e870 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.550+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9220140d50 con 0x7f922006e870 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.551+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9220073990 con 0x7f922006e870 2024-04-25T10:12:28.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.552+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9210007400 con 0x7f922006e870 2024-04-25T10:12:28.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.553+0000 7f9217e00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91fc06b4d0 0x7f91fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.553+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f92100895a0 con 0x7f922006e870 2024-04-25T10:12:28.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.553+0000 7f9216a00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91fc06b4d0 0x7f91fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.554+0000 7f9216a00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91fc06b4d0 0x7f91fc06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9208006a90 tx=0x7f9208006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.557+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9210058580 con 0x7f922006e870 2024-04-25T10:12:28.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.755+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f922015dbc0 con 0x7f922006e870 2024-04-25T10:12:28.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.757+0000 7f9217e00640 1 -- 172.21.15.165:0/2142247688 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+13333 (secure 0 0 0) 0x7f9210057e80 con 0x7f922006e870 2024-04-25T10:12:28.762 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:28.763 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":103,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:27.302349+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6835/2500333822":"2024-04-26T10:12:24.284708+0000","172.21.15.165:6834/2500333822":"2024-04-26T10:12:24.284708+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91fc06b4d0 msgr2=0x7f91fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91fc06b4d0 0x7f91fc06d990 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f9208006a90 tx=0x7f9208006a20 comp rx=0 tx=0).stop 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 msgr2=0x7f922013da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 0x7f922013da60 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f921000c3c0 tx=0x7f921000c880 comp rx=0 tx=0).stop 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 shutdown_connections 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91fc06b4d0 0x7f91fc06d990 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9220143e60 0x7f92200a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92200cc840 0x7f9220140810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 --2- 172.21.15.165:0/2142247688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f922006e870 0x7f922013da60 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 >> 172.21.15.165:0/2142247688 conn(0x7f922006abe0 msgr2=0x7f92200af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.760+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 shutdown_connections 2024-04-25T10:12:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.761+0000 7f9224c00640 1 -- 172.21.15.165:0/2142247688 wait complete. 2024-04-25T10:12:28.784 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:28.784 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:28.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.982+0000 7f75e7c00640 1 Processor -- start 2024-04-25T10:12:28.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.982+0000 7f75e7c00640 1 -- start start 2024-04-25T10:12:28.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.983+0000 7f75e7c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75e0144d50 0x7f75e006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.983+0000 7f75e7c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e00cccc0 con 0x7f75e01452a0 2024-04-25T10:12:28.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75e7c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75e00cc140 con 0x7f75e01446d0 2024-04-25T10:12:28.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75e7c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75e00b2b20 con 0x7f75e0144d50 2024-04-25T10:12:28.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75e5c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f75e01452a0 0x7f75e006ddd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44184/0 (socket says 172.21.15.165:44184) 2024-04-25T10:12:28.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75e5c00640 1 -- 172.21.15.165:0/2557486788 learned_addr learned my addr 172.21.15.165:0/2557486788 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 792057622 0 0) 0x7f75e00cccc0 con 0x7f75e01452a0 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75c8003520 con 0x7f75e01452a0 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.984+0000 7f75dfe00640 1 --2- 172.21.15.165:0/2557486788 >> v2:172.21.15.172:3300/0 conn(0x7f75e0144d50 0x7f75e006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3651489285 0 0) 0x7f75c8003520 con 0x7f75e01452a0 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 >> v2:172.21.15.172:3300/0 conn(0x7f75e0144d50 msgr2=0x7f75e006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 --2- 172.21.15.165:0/2557486788 >> v2:172.21.15.172:3300/0 conn(0x7f75e0144d50 0x7f75e006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 >> v1:172.21.15.172:6789/0 conn(0x7f75e01446d0 legacy=0x7f75e00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00cc420 con 0x7f75e01452a0 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f75d80019f0 con 0x7f75e01452a0 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 >> v1:172.21.15.165:6789/0 conn(0x7f75e01452a0 legacy=0x7f75e006ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75c80039b0 0x7f75c8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75c8004800 0x7f75c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00b2b20 con 0x7f75c80051c0 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75c800c0f0 con 0x7f75c80039b0 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75c800c2f0 con 0x7f75c8004800 2024-04-25T10:12:28.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75e5c00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.985+0000 7f75e5200640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75c80039b0 0x7f75c8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.986+0000 7f75dfe00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75c8004800 0x7f75c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.986+0000 7f75e5c00640 1 -- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75c8004800 msgr2=0x7f75c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.986+0000 7f75e5c00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75c8004800 0x7f75c8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.986+0000 7f75e5c00640 1 -- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75c80039b0 msgr2=0x7f75c8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.986+0000 7f75e5c00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75c80039b0 0x7f75c8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.986+0000 7f75e5c00640 1 -- 172.21.15.165:0/2557486788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c8003520 con 0x7f75c80051c0 2024-04-25T10:12:28.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.987+0000 7f75e5c00640 1 -- 172.21.15.165:0/2557486788 reap_dead start 2024-04-25T10:12:28.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.987+0000 7f75e5c00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75c800b9c0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f75d800a830 tx=0x7f75c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93d7b8461fa9796d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.987+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d800a480 con 0x7f75c80051c0 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.987+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d802f070 con 0x7f75c80051c0 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.987+0000 7f75df400640 1 -- 172.21.15.165:0/2557486788 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d802bc50 con 0x7f75c80051c0 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 -- 172.21.15.165:0/2557486788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 msgr2=0x7f75c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75c800b9c0 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f75d800a830 tx=0x7f75c8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 -- 172.21.15.165:0/2557486788 shutdown_connections 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 --2- 172.21.15.165:0/2557486788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75c800b9c0 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 -- 172.21.15.165:0/2557486788 >> 172.21.15.165:0/2557486788 conn(0x7f75e006abe0 msgr2=0x7f75e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 -- 172.21.15.165:0/2557486788 shutdown_connections 2024-04-25T10:12:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.988+0000 7f75e7c00640 1 -- 172.21.15.165:0/2557486788 wait complete. 2024-04-25T10:12:28.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.990+0000 7f75e7c00640 1 Processor -- start 2024-04-25T10:12:28.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.990+0000 7f75e7c00640 1 -- start start 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e7c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75e015b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e7c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e00cb030 0x7f75e00b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e7c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00cb4b0 0x7f75e00b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e7c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e00cc9e0 con 0x7f75c80051c0 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e7c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e006d400 con 0x7f75e00cb030 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e7c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e0144ab0 con 0x7f75e00cb4b0 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75e015b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:28.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75e015b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41490/0 (socket says 172.21.15.165:41490) 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.991+0000 7f75e5200640 1 -- 172.21.15.165:0/4128856666 learned_addr learned my addr 172.21.15.165:0/4128856666 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e5200640 1 -- 172.21.15.165:0/4128856666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00cb4b0 msgr2=0x7f75e00b1120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e5200640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00cb4b0 0x7f75e00b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e5200640 1 -- 172.21.15.165:0/4128856666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e00cb030 msgr2=0x7f75e00b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e5200640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e00cb030 0x7f75e00b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:28.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e5200640 1 -- 172.21.15.165:0/4128856666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e00c5d00 con 0x7f75c80051c0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e5200640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75e015b160 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f75d400e460 tx=0x7f75d400e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40135f0 con 0x7f75c80051c0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75d401a070 con 0x7f75c80051c0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e00c5fe0 con 0x7f75c80051c0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75d40139d0 con 0x7f75c80051c0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.992+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e00c65a0 con 0x7f75c80051c0 2024-04-25T10:12:28.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.993+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e0073a00 con 0x7f75c80051c0 2024-04-25T10:12:28.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.995+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75d4013b70 con 0x7f75c80051c0 2024-04-25T10:12:29.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.996+0000 7f75dcc00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75c006b4d0 0x7f75c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.996+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f75d408a220 con 0x7f75c80051c0 2024-04-25T10:12:29.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.996+0000 7f75dfe00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75c006b4d0 0x7f75c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:28.997+0000 7f75dfe00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75c006b4d0 0x7f75c006d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f75e00c5b10 tx=0x7f75cc00d280 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.000+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75d4024020 con 0x7f75c80051c0 2024-04-25T10:12:29.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.192+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f75e00ae120 con 0x7f75c80051c0 2024-04-25T10:12:29.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.193+0000 7f75dcc00640 1 -- 172.21.15.165:0/4128856666 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7f75d4010300 con 0x7f75c80051c0 2024-04-25T10:12:29.198 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:12:29.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75c006b4d0 msgr2=0x7f75c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75c006b4d0 0x7f75c006d990 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f75e00c5b10 tx=0x7f75cc00d280 comp rx=0 tx=0).stop 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 msgr2=0x7f75e015b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75e015b160 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f75d400e460 tx=0x7f75d400e920 comp rx=0 tx=0).stop 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 shutdown_connections 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75c006b4d0 0x7f75c006d990 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00cb4b0 0x7f75e00b1120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e00cb030 0x7f75e00b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 --2- 172.21.15.165:0/4128856666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75c80051c0 0x7f75e015b160 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 >> 172.21.15.165:0/4128856666 conn(0x7f75e006abe0 msgr2=0x7f75e00ad250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 shutdown_connections 2024-04-25T10:12:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.196+0000 7f75e7c00640 1 -- 172.21.15.165:0/4128856666 wait complete. 2024-04-25T10:12:29.220 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:12:29.257 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:29.274 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.402+0000 7f1b9b200640 1 Processor -- start 2024-04-25T10:12:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.402+0000 7f1b9b200640 1 -- start start 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.404+0000 7f1b9b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 0x7f1b9c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.404+0000 7f1b9b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c161df0 con 0x7f1b9c15db90 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.404+0000 7f1b9b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9c1620d0 con 0x7f1b9c0d04f0 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.404+0000 7f1b9b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0d0980 con 0x7f1b9c0b93c0 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.404+0000 7f1b99e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1b9c15db90 0x7f1b9c1615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44198/0 (socket says 172.21.15.165:44198) 2024-04-25T10:12:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.404+0000 7f1b99e00640 1 -- 172.21.15.165:0/879081126 learned_addr learned my addr 172.21.15.165:0/879081126 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b91400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 0x7f1b9c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491780494 0 0) 0x7f1b9c161df0 con 0x7f1b9c15db90 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b80003520 con 0x7f1b9c15db90 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3580399194 0 0) 0x7f1b80003520 con 0x7f1b9c15db90 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 msgr2=0x7f1b9c0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 0x7f1b9c0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 >> v1:172.21.15.172:6789/0 conn(0x7f1b9c0d04f0 legacy=0x7f1b9c0d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c1623b0 con 0x7f1b9c15db90 2024-04-25T10:12:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1b8c0019f0 con 0x7f1b9c15db90 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 >> v1:172.21.15.165:6789/0 conn(0x7f1b9c15db90 legacy=0x7f1b9c1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 0x7f1b800039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80004d40 0x7f1b800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0d0980 con 0x7f1b80003f60 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000baa0 con 0x7f1b80004d40 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.405+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b8000bca0 con 0x7f1b9c0b93c0 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b91400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b99e00640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 0x7f1b800039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b9a800640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80004d40 0x7f1b800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b91400640 1 -- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 msgr2=0x7f1b800039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b91400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c0b93c0 0x7f1b800039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b91400640 1 -- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80004d40 msgr2=0x7f1b800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:29.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b91400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b80004d40 0x7f1b800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.406+0000 7f1b91400640 1 -- 172.21.15.165:0/879081126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b80003520 con 0x7f1b80003f60 2024-04-25T10:12:29.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.407+0000 7f1b91400640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b80004800 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009ac0 tx=0x7f1b80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b2e5894f2e2ed5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.407+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84036070 con 0x7f1b80003f60 2024-04-25T10:12:29.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.407+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b84031400 con 0x7f1b80003f60 2024-04-25T10:12:29.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.407+0000 7f1b99400640 1 -- 172.21.15.165:0/879081126 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84009690 con 0x7f1b80003f60 2024-04-25T10:12:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9b200640 1 -- 172.21.15.165:0/879081126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 msgr2=0x7f1b80004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9b200640 1 --2- 172.21.15.165:0/879081126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b80004800 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009ac0 tx=0x7f1b80005830 comp rx=0 tx=0).stop 2024-04-25T10:12:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9a800640 1 -- 172.21.15.165:0/879081126 reap_dead start 2024-04-25T10:12:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9b200640 1 -- 172.21.15.165:0/879081126 shutdown_connections 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9b200640 1 -- 172.21.15.165:0/879081126 >> 172.21.15.165:0/879081126 conn(0x7f1b9c0b52b0 msgr2=0x7f1b9c0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9b200640 1 -- 172.21.15.165:0/879081126 shutdown_connections 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.408+0000 7f1b9b200640 1 -- 172.21.15.165:0/879081126 wait complete. 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.410+0000 7f1b9b200640 1 Processor -- start 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.410+0000 7f1b9b200640 1 -- start start 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b9b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b9c15c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b9b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c1f49b0 0x7f1b9c15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b9b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c154800 0x7f1b9c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b9b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b68d0 con 0x7f1b80003f60 2024-04-25T10:12:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b9b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0d1f50 con 0x7f1b9c154800 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b9b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9c0b6a50 con 0x7f1b9c1f49b0 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b99e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b9c15c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b99e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b9c15c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41512/0 (socket says 172.21.15.165:41512) 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.411+0000 7f1b99e00640 1 -- 172.21.15.165:0/4156652081 learned_addr learned my addr 172.21.15.165:0/4156652081 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b91400640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c1f49b0 0x7f1b9c15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b9a800640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c154800 0x7f1b9c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b99e00640 1 -- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c1f49b0 msgr2=0x7f1b9c15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b99e00640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c1f49b0 0x7f1b9c15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b99e00640 1 -- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c154800 msgr2=0x7f1b9c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b99e00640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c154800 0x7f1b9c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b99e00640 1 -- 172.21.15.165:0/4156652081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9c1f4e30 con 0x7f1b80003f60 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b99e00640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b9c15c730 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c007ac0 tx=0x7f1b8c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c013420 con 0x7f1b80003f60 2024-04-25T10:12:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8c018070 con 0x7f1b80003f60 2024-04-25T10:12:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8c013800 con 0x7f1b80003f60 2024-04-25T10:12:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.412+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9c15cd10 con 0x7f1b80003f60 2024-04-25T10:12:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.413+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9c0cac40 con 0x7f1b80003f60 2024-04-25T10:12:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.414+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b9c072e40 con 0x7f1b80003f60 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.415+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b8c0139a0 con 0x7f1b80003f60 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.416+0000 7f1b92a00640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b7806b4d0 0x7f1b7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.416+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1b8c08a7c0 con 0x7f1b80003f60 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.416+0000 7f1b91400640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b7806b4d0 0x7f1b7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:29.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.417+0000 7f1b91400640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b7806b4d0 0x7f1b7806d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b84033000 tx=0x7f1b84008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:29.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.421+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b9c072e40 con 0x7f1b80003f60 2024-04-25T10:12:29.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:29.676+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7f1b9c073990 con 0x7f1b80003f60 2024-04-25T10:12:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.324+0000 7f1b92a00640 1 -- 172.21.15.165:0/4156652081 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v481) v1 ==== 158+0+0 (secure 0 0 0) 0x7f1b8c05f030 con 0x7f1b80003f60 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b7806b4d0 msgr2=0x7f1b7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b7806b4d0 0x7f1b7806d990 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1b84033000 tx=0x7f1b84008030 comp rx=0 tx=0).stop 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 msgr2=0x7f1b9c15c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b9c15c730 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f1b8c007ac0 tx=0x7f1b8c00e530 comp rx=0 tx=0).stop 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 shutdown_connections 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b7806b4d0 0x7f1b7806d990 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b9c154800 0x7f1b9c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b9c1f49b0 0x7f1b9c15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 --2- 172.21.15.165:0/4156652081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b80003f60 0x7f1b9c15c730 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 >> 172.21.15.165:0/4156652081 conn(0x7f1b9c0b52b0 msgr2=0x7f1b9c0ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.327+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 shutdown_connections 2024-04-25T10:12:30.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.328+0000 7f1b9b200640 1 -- 172.21.15.165:0/4156652081 wait complete. 2024-04-25T10:12:30.353 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:12:30.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.550+0000 7fc94ec00640 1 Processor -- start 2024-04-25T10:12:30.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.551+0000 7fc94ec00640 1 -- start start 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.552+0000 7fc94ec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9481446d0 0x7fc9480b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.552+0000 7fc94ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc948144d50 0x7fc9480b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.552+0000 7fc94ec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9480cccc0 con 0x7fc9481452a0 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.552+0000 7fc94ec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9480b2b20 con 0x7fc9481446d0 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.552+0000 7fc94ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc94806d400 con 0x7fc948144d50 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.552+0000 7fc947e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9481446d0 0x7fc9480b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc947e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9481446d0 0x7fc9480b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41524/0 (socket says 172.21.15.165:41524) 2024-04-25T10:12:30.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc947e00640 1 -- 172.21.15.165:0/911809142 learned_addr learned my addr 172.21.15.165:0/911809142 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc947e00640 1 -- 172.21.15.165:0/911809142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc948144d50 msgr2=0x7fc9480b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc93fe00640 1 --2- 172.21.15.165:0/911809142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc948144d50 0x7fc9480b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc947e00640 1 --2- 172.21.15.165:0/911809142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc948144d50 0x7fc9480b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc947e00640 1 -- 172.21.15.165:0/911809142 >> v1:172.21.15.172:6789/0 conn(0x7fc9481452a0 legacy=0x7fc9480cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.553+0000 7fc947e00640 1 -- 172.21.15.165:0/911809142 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480cc140 con 0x7fc9481446d0 2024-04-25T10:12:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.554+0000 7fc947e00640 1 --2- 172.21.15.165:0/911809142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc9480b3f20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc938009870 tx=0x7fc93802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18fe3ddf5353b7f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.554+0000 7fc947400640 1 -- 172.21.15.165:0/911809142 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938037070 con 0x7fc9481446d0 2024-04-25T10:12:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.554+0000 7fc947400640 1 -- 172.21.15.165:0/911809142 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938032440 con 0x7fc9481446d0 2024-04-25T10:12:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.554+0000 7fc947400640 1 -- 172.21.15.165:0/911809142 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938032810 con 0x7fc9481446d0 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 -- 172.21.15.165:0/911809142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 msgr2=0x7fc9480b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 --2- 172.21.15.165:0/911809142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc9480b3f20 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fc938009870 tx=0x7fc93802a210 comp rx=0 tx=0).stop 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 -- 172.21.15.165:0/911809142 shutdown_connections 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 --2- 172.21.15.165:0/911809142 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc948144d50 0x7fc9480b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 --2- 172.21.15.165:0/911809142 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc9480b3f20 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 -- 172.21.15.165:0/911809142 >> 172.21.15.165:0/911809142 conn(0x7fc94806abe0 msgr2=0x7fc94806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 -- 172.21.15.165:0/911809142 shutdown_connections 2024-04-25T10:12:30.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.555+0000 7fc94ec00640 1 -- 172.21.15.165:0/911809142 wait complete. 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.557+0000 7fc94ec00640 1 Processor -- start 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.557+0000 7fc94ec00640 1 -- start start 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc94ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc94813a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc94ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc948144d50 0x7fc948141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc94ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9481452a0 0x7fc948142020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc94ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cc700 con 0x7fc9481446d0 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc94ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480b49a0 con 0x7fc948144d50 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc94ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9480cc9e0 con 0x7fc9481452a0 2024-04-25T10:12:30.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc947e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc94813a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc947e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc94813a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41538/0 (socket says 172.21.15.165:41538) 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.558+0000 7fc947e00640 1 -- 172.21.15.165:0/1499840421 learned_addr learned my addr 172.21.15.165:0/1499840421 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc94cc00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9481452a0 0x7fc948142020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc947e00640 1 -- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9481452a0 msgr2=0x7fc948142020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc93fe00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc948144d50 0x7fc948141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc947e00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9481452a0 0x7fc948142020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc947e00640 1 -- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc948144d50 msgr2=0x7fc948141ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc947e00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc948144d50 0x7fc948141ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc947e00640 1 -- 172.21.15.165:0/1499840421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9480a91d0 con 0x7fc9481446d0 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc94cc00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9481452a0 0x7fc948142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc93fe00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc948144d50 0x7fc948141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc947e00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc94813a050 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc938000c00 tx=0x7fc9380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938037040 con 0x7fc9481446d0 2024-04-25T10:12:30.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc938042070 con 0x7fc9481446d0 2024-04-25T10:12:30.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc938030bd0 con 0x7fc9481446d0 2024-04-25T10:12:30.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9480a94b0 con 0x7fc9481446d0 2024-04-25T10:12:30.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.559+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9480a9a70 con 0x7fc9481446d0 2024-04-25T10:12:30.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.560+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc948073a00 con 0x7fc9481446d0 2024-04-25T10:12:30.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.561+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc938047460 con 0x7fc9481446d0 2024-04-25T10:12:30.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.562+0000 7fc944c00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc92406b4d0 0x7fc92406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:30.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.563+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc9380aad40 con 0x7fc9481446d0 2024-04-25T10:12:30.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.563+0000 7fc93fe00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc92406b4d0 0x7fc92406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:30.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.563+0000 7fc93fe00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc92406b4d0 0x7fc92406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc94815bbf0 tx=0x7fc930008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:30.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.567+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc938045360 con 0x7fc9481446d0 2024-04-25T10:12:30.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.750+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc948142700 con 0x7fc9481446d0 2024-04-25T10:12:30.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.755+0000 7fc944c00640 1 -- 172.21.15.165:0/1499840421 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc938079c90 con 0x7fc9481446d0 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc92406b4d0 msgr2=0x7fc92406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc92406b4d0 0x7fc92406d990 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc94815bbf0 tx=0x7fc930008040 comp rx=0 tx=0).stop 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 msgr2=0x7fc94813a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc94813a050 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fc938000c00 tx=0x7fc9380046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 shutdown_connections 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc92406b4d0 0x7fc92406d990 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9481452a0 0x7fc948142020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc948144d50 0x7fc948141ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 --2- 172.21.15.165:0/1499840421 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9481446d0 0x7fc94813a050 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 >> 172.21.15.165:0/1499840421 conn(0x7fc94806abe0 msgr2=0x7fc94806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.758+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 shutdown_connections 2024-04-25T10:12:30.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:30.759+0000 7fc94ec00640 1 -- 172.21.15.165:0/1499840421 wait complete. 2024-04-25T10:12:30.782 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:12:30.782 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:30.882 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:12:30.986 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:12:30.986 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:12:30.986 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:31.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:31.042 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:12:31.042 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:31.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:31.100 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:12:31.100 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:31.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:12:31.158 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:12:31.158 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:31.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:12:31.215 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:12:31.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:31.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.420+0000 7f9308c00640 1 Processor -- start 2024-04-25T10:12:31.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.420+0000 7f9308c00640 1 -- start start 2024-04-25T10:12:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.422+0000 7f9308c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 0x7f93041687e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.422+0000 7f9308c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930415e7d0 con 0x7f93040addc0 2024-04-25T10:12:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.422+0000 7f9308c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930415eab0 con 0x7f93040728a0 2024-04-25T10:12:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.422+0000 7f9308c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f930415ed90 con 0x7f930406c300 2024-04-25T10:12:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.422+0000 7f9303400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f93040addc0 0x7f930415de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44214/0 (socket says 172.21.15.165:44214) 2024-04-25T10:12:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.422+0000 7f9303400640 1 -- 172.21.15.165:0/2680051143 learned_addr learned my addr 172.21.15.165:0/2680051143 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9303e00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 0x7f93041687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924834669 0 0) 0x7f930415e7d0 con 0x7f93040addc0 2024-04-25T10:12:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e8003520 con 0x7f93040addc0 2024-04-25T10:12:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226135090 0 0) 0x7f92e8003520 con 0x7f93040addc0 2024-04-25T10:12:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 msgr2=0x7f93041687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 0x7f93041687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 >> v1:172.21.15.172:6789/0 conn(0x7f93040728a0 legacy=0x7f930406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040b2d60 con 0x7f93040addc0 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f92f4001670 con 0x7f93040addc0 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 >> v1:172.21.15.165:6789/0 conn(0x7f93040addc0 legacy=0x7f930415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 0x7f92e80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004060 0x7f92e8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8004d40 0x7f92e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930415ed90 con 0x7f92e8004d40 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e800bca0 con 0x7f92e8004060 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.423+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e800bea0 con 0x7f930406c300 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303e00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8004d40 0x7f92e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303400640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 0x7f92e80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9302a00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004060 0x7f92e8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303e00640 1 -- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 msgr2=0x7f92e80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303e00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930406c300 0x7f92e80039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303e00640 1 -- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004060 msgr2=0x7f92e8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303e00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004060 0x7f92e8004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.424+0000 7f9303e00640 1 -- 172.21.15.165:0/2680051143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8003520 con 0x7f92e8004d40 2024-04-25T10:12:31.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.425+0000 7f9303e00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8004d40 0x7f92e80051c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f92fc007330 tx=0x7f92e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de5329f4d2a9db3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.425+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc033070 con 0x7f92e8004d40 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.425+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc02f650 con 0x7f92e8004d40 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.425+0000 7f9302000640 1 -- 172.21.15.165:0/2680051143 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc009be0 con 0x7f92e8004d40 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9308c00640 1 -- 172.21.15.165:0/2680051143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8004d40 msgr2=0x7f92e80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9308c00640 1 --2- 172.21.15.165:0/2680051143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e8004d40 0x7f92e80051c0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f92fc007330 tx=0x7f92e8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9303e00640 1 -- 172.21.15.165:0/2680051143 reap_dead start 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9308c00640 1 -- 172.21.15.165:0/2680051143 shutdown_connections 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9308c00640 1 -- 172.21.15.165:0/2680051143 >> 172.21.15.165:0/2680051143 conn(0x7f93040718d0 msgr2=0x7f9304071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9308c00640 1 -- 172.21.15.165:0/2680051143 shutdown_connections 2024-04-25T10:12:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.426+0000 7f9308c00640 1 -- 172.21.15.165:0/2680051143 wait complete. 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.428+0000 7f9308c00640 1 Processor -- start 2024-04-25T10:12:31.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.428+0000 7f9308c00640 1 -- start start 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.429+0000 7f9308c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004d40 0x7f93041526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.429+0000 7f9308c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930414d300 0x7f93040b0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.429+0000 7f9308c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 0x7f930414f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.429+0000 7f9308c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93040ae1a0 con 0x7f930414f240 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.429+0000 7f9308c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930415e570 con 0x7f92e8004d40 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.429+0000 7f9308c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93040b4570 con 0x7f930414d300 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 0x7f930414f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 0x7f930414f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41558/0 (socket says 172.21.15.165:41558) 2024-04-25T10:12:31.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 -- 172.21.15.165:0/259190891 learned_addr learned my addr 172.21.15.165:0/259190891 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303400640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004d40 0x7f93041526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9302a00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930414d300 0x7f93040b0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 -- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930414d300 msgr2=0x7f93040b0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930414d300 0x7f93040b0900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 -- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004d40 msgr2=0x7f93041526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004d40 0x7f93041526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 -- 172.21.15.165:0/259190891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f930414d780 con 0x7f930414f240 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303400640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004d40 0x7f93041526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.430+0000 7f9303e00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 0x7f930414f6c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f92fc02b030 tx=0x7f92fc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.431+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc033040 con 0x7f930414f240 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.431+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92fc02f450 con 0x7f930414f240 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.431+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92fc009be0 con 0x7f930414f240 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.431+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930414eaa0 con 0x7f930414f240 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.431+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930414dce0 con 0x7f930414f240 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.432+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304073990 con 0x7f930414f240 2024-04-25T10:12:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.433+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92fc036070 con 0x7f930414f240 2024-04-25T10:12:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.434+0000 7f92fb400640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92e006b4d0 0x7f92e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.434+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f92fc0aa7b0 con 0x7f930414f240 2024-04-25T10:12:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.434+0000 7f9303400640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92e006b4d0 0x7f92e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.435+0000 7f9303400640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92e006b4d0 0x7f92e006d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f92f40017e0 tx=0x7f92f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.438+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f930414d780 con 0x7f930414f240 2024-04-25T10:12:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.673+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9304161bd0 con 0x7f930414f240 2024-04-25T10:12:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.674+0000 7f92fb400640 1 -- 172.21.15.165:0/259190891 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 67 v67) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f92fc079700 con 0x7f930414f240 2024-04-25T10:12:31.679 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:31.679 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":67,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5261,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/4074397071","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":4074397071},{"type":"v1","addr":"172.21.15.165:6837","nonce":4074397071}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5267,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3027350817","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3027350817},{"type":"v1","addr":"172.21.15.172:6833","nonce":3027350817}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":5273,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1988301696","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1988301696},{"type":"v1","addr":"172.21.15.172:6835","nonce":1988301696}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":61},{"gid":7956,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/2200209737","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":2200209737},{"type":"v1","addr":"172.21.15.165:6835","nonce":2200209737}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":67}],"filesystems":[]} 2024-04-25T10:12:31.679 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 67 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.676+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92e006b4d0 msgr2=0x7f92e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.676+0000 7f9308c00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92e006b4d0 0x7f92e006d990 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f92f40017e0 tx=0x7f92f4009040 comp rx=0 tx=0).stop 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.676+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 msgr2=0x7f930414f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.676+0000 7f9308c00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 0x7f930414f6c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f92fc02b030 tx=0x7f92fc004ab0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 shutdown_connections 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92e006b4d0 0x7f92e006d990 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f930414f240 0x7f930414f6c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f930414d300 0x7f93040b0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 --2- 172.21.15.165:0/259190891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8004d40 0x7f93041526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 >> 172.21.15.165:0/259190891 conn(0x7f93040718d0 msgr2=0x7f9304161df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 shutdown_connections 2024-04-25T10:12:31.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.677+0000 7f9308c00640 1 -- 172.21.15.165:0/259190891 wait complete. 2024-04-25T10:12:31.702 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:12:31.702 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:12:31.702 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:12:31.702 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:31.702 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:12:31.702 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:12:31.703 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:12:31.703 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:31.703 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:12:31.703 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:12:31.703 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:12:31.703 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:31.703 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:12:31.703 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:12:31.704 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:12:31.704 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:12:31.721 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:12:31.722 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:12:31.725 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:12:31.725 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:12:31.730 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:12:31.730 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:12:31.732 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:12:31.732 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:12:31.772 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:12:31.773 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:31.779 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:12:31.779 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:12:31.779 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:31.779 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:31.785 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:12:31.785 DEBUG:teuthology.parallel:result is None 2024-04-25T10:12:31.786 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:12:31.922 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:12:31.922 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:12:31.934 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:12:31.934 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:12:31.936 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:12:31.931+0000 7f7db7c00640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2024-04-25T10:12:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.959+0000 7f99d3e00640 1 Processor -- start 2024-04-25T10:12:31.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.960+0000 7f99d3e00640 1 -- start start 2024-04-25T10:12:31.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99cc15b5a0 0x7f99cc0b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99cc15b980 0x7f99cc0b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc162450 0x7f99cc0ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d3e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0cf320 con 0x7f99cc15b980 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99cc0cf040 con 0x7f99cc15b5a0 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc161970 con 0x7f99cc162450 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99cc15b980 0x7f99cc0b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99cc15b980 0x7f99cc0b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41588/0 (socket says 172.21.15.165:41588) 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 -- 172.21.15.165:0/87047837 learned_addr learned my addr 172.21.15.165:0/87047837 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d1400640 1 --2- 172.21.15.165:0/87047837 >> v2:172.21.15.172:3300/0 conn(0x7f99cc15b5a0 0x7f99cc0b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d1e00640 1 --2- 172.21.15.165:0/87047837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc162450 0x7f99cc0ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 -- 172.21.15.165:0/87047837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc162450 msgr2=0x7f99cc0ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 --2- 172.21.15.165:0/87047837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc162450 0x7f99cc0ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 -- 172.21.15.165:0/87047837 >> v2:172.21.15.172:3300/0 conn(0x7f99cc15b5a0 msgr2=0x7f99cc0b5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 --2- 172.21.15.165:0/87047837 >> v2:172.21.15.172:3300/0 conn(0x7f99cc15b5a0 0x7f99cc0b5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d0a00640 1 -- 172.21.15.165:0/87047837 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc161c50 con 0x7f99cc15b980 2024-04-25T10:12:31.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.961+0000 7f99d1400640 1 --2- 172.21.15.165:0/87047837 >> v2:172.21.15.172:3300/0 conn(0x7f99cc15b5a0 0x7f99cc0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.962+0000 7f99d0a00640 1 --2- 172.21.15.165:0/87047837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b980 0x7f99cc0b5a70 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f99cc08c0d0 tx=0x7f99b802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57aa4602790ce4be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99cbe00640 1 -- 172.21.15.165:0/87047837 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8004030 con 0x7f99cc15b980 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99cbe00640 1 -- 172.21.15.165:0/87047837 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b80041d0 con 0x7f99cc15b980 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99cbe00640 1 -- 172.21.15.165:0/87047837 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b8003610 con 0x7f99cc15b980 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 -- 172.21.15.165:0/87047837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b980 msgr2=0x7f99cc0b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 --2- 172.21.15.165:0/87047837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b980 0x7f99cc0b5a70 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7f99cc08c0d0 tx=0x7f99b802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 -- 172.21.15.165:0/87047837 shutdown_connections 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 --2- 172.21.15.165:0/87047837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc162450 0x7f99cc0ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 --2- 172.21.15.165:0/87047837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b980 0x7f99cc0b5a70 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 --2- 172.21.15.165:0/87047837 >> v2:172.21.15.172:3300/0 conn(0x7f99cc15b5a0 0x7f99cc0b5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.963+0000 7f99d3e00640 1 -- 172.21.15.165:0/87047837 >> 172.21.15.165:0/87047837 conn(0x7f99cc0d0530 msgr2=0x7f99cc0d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.964+0000 7f99d3e00640 1 -- 172.21.15.165:0/87047837 shutdown_connections 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.964+0000 7f99d3e00640 1 -- 172.21.15.165:0/87047837 wait complete. 2024-04-25T10:12:31.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.965+0000 7f99d3e00640 1 Processor -- start 2024-04-25T10:12:31.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.965+0000 7f99d3e00640 1 -- start start 2024-04-25T10:12:31.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 0x7f99cc0c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc15b980 0x7f99cc0c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc162450 0x7f99cc14ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0ce9d0 con 0x7f99cc15b5a0 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0ce850 con 0x7f99cc162450 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0ceb50 con 0x7f99cc15b980 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.966+0000 7f99d1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 0x7f99cc0c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 0x7f99cc0c8140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41592/0 (socket says 172.21.15.165:41592) 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 -- 172.21.15.165:0/2362009848 learned_addr learned my addr 172.21.15.165:0/2362009848 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc162450 0x7f99cc14ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d0a00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc15b980 0x7f99cc0c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 -- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc15b980 msgr2=0x7f99cc0c75e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc15b980 0x7f99cc0c75e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 -- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc162450 msgr2=0x7f99cc14ca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc162450 0x7f99cc14ca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 -- 172.21.15.165:0/2362009848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc150eb0 con 0x7f99cc15b5a0 2024-04-25T10:12:31.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.967+0000 7f99d1400640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 0x7f99cc0c8140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f99c000e810 tx=0x7f99c000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.968+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c000f040 con 0x7f99cc15b5a0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.968+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c0004030 con 0x7f99cc15b5a0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.968+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c0004400 con 0x7f99cc15b5a0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.968+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc151150 con 0x7f99cc15b5a0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.968+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc151710 con 0x7f99cc15b5a0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.969+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc073a00 con 0x7f99cc15b5a0 2024-04-25T10:12:31.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.970+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99c000c6d0 con 0x7f99cc15b5a0 2024-04-25T10:12:31.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.971+0000 7f99c9600640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99ac06b4d0 0x7f99ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:31.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.971+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f99c0089d10 con 0x7f99cc15b5a0 2024-04-25T10:12:31.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.971+0000 7f99d0a00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99ac06b4d0 0x7f99ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:31.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.972+0000 7f99d0a00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99ac06b4d0 0x7f99ac06d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99b8033000 tx=0x7f99b802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:31.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:31.975+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99c008f020 con 0x7f99cc15b5a0 2024-04-25T10:12:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.184+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f99cc0d17c0 con 0x7f99cc15b5a0 2024-04-25T10:12:32.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.358+0000 7f99c9600640 1 -- 172.21.15.165:0/2362009848 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v105) v1 ==== 97+0+0 (secure 0 0 0) 0x7f99c0017020 con 0x7f99cc15b5a0 2024-04-25T10:12:32.363 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99ac06b4d0 msgr2=0x7f99ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99ac06b4d0 0x7f99ac06d990 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f99b8033000 tx=0x7f99b802e000 comp rx=0 tx=0).stop 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 msgr2=0x7f99cc0c8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 0x7f99cc0c8140 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f99c000e810 tx=0x7f99c000ecd0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 shutdown_connections 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99ac06b4d0 0x7f99ac06d990 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99cc162450 0x7f99cc14ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99cc15b980 0x7f99cc0c75e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 --2- 172.21.15.165:0/2362009848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99cc15b5a0 0x7f99cc0c8140 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 >> 172.21.15.165:0/2362009848 conn(0x7f99cc0d0530 msgr2=0x7f99cc15d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.363+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 shutdown_connections 2024-04-25T10:12:32.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.364+0000 7f99d3e00640 1 -- 172.21.15.165:0/2362009848 wait complete. 2024-04-25T10:12:32.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:12:32.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.582+0000 7f544a200640 1 Processor -- start 2024-04-25T10:12:32.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.582+0000 7f544a200640 1 -- start start 2024-04-25T10:12:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.583+0000 7f544a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54440ced80 0x7f54440d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.583+0000 7f544a200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f544415e000 con 0x7f5444161710 2024-04-25T10:12:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.583+0000 7f544a200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54440d1de0 con 0x7f544415dc20 2024-04-25T10:12:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f544a200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5444161c60 con 0x7f54440ced80 2024-04-25T10:12:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5442a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54440ced80 0x7f54440d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5443e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5444161710 0x7f54440d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44230/0 (socket says 172.21.15.165:44230) 2024-04-25T10:12:32.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5442a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f54440ced80 0x7f54440d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41604/0 (socket says 172.21.15.165:41604) 2024-04-25T10:12:32.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5443e00640 1 -- 172.21.15.165:0/1818762815 learned_addr learned my addr 172.21.15.165:0/1818762815 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:32.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5442a00640 1 -- 172.21.15.165:0/1818762815 >> v1:172.21.15.172:6789/0 conn(0x7f544415dc20 legacy=0x7f54440ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:32.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5442a00640 1 -- 172.21.15.165:0/1818762815 >> v1:172.21.15.165:6789/0 conn(0x7f5444161710 legacy=0x7f54440d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5442a00640 1 -- 172.21.15.165:0/1818762815 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5444161f40 con 0x7f54440ced80 2024-04-25T10:12:32.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.584+0000 7f5442000640 1 -- 172.21.15.165:0/1818762815 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525300982 0 0) 0x7f544415e000 con 0x7f5444161710 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.585+0000 7f5442a00640 1 --2- 172.21.15.165:0/1818762815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54440ced80 0x7f54440d22e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f54300097a0 tx=0x7f543002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee09a3533d60754b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f5442000640 1 -- 172.21.15.165:0/1818762815 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430037070 con 0x7f54440ced80 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f5442000640 1 -- 172.21.15.165:0/1818762815 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5430032440 con 0x7f54440ced80 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f5442000640 1 -- 172.21.15.165:0/1818762815 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5430032810 con 0x7f54440ced80 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 -- 172.21.15.165:0/1818762815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54440ced80 msgr2=0x7f54440d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 --2- 172.21.15.165:0/1818762815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54440ced80 0x7f54440d22e0 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f54300097a0 tx=0x7f543002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 -- 172.21.15.165:0/1818762815 shutdown_connections 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 --2- 172.21.15.165:0/1818762815 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54440ced80 0x7f54440d22e0 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 -- 172.21.15.165:0/1818762815 >> 172.21.15.165:0/1818762815 conn(0x7f54440d0420 msgr2=0x7f54440d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 -- 172.21.15.165:0/1818762815 shutdown_connections 2024-04-25T10:12:32.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.586+0000 7f544a200640 1 -- 172.21.15.165:0/1818762815 wait complete. 2024-04-25T10:12:32.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.588+0000 7f544a200640 1 Processor -- start 2024-04-25T10:12:32.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.588+0000 7f544a200640 1 -- start start 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f544a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54440ced80 0x7f544415af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f544a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544415dc20 0x7f5444160630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f544a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 0x7f54440cd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f544a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54440b5e50 con 0x7f5444161710 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f544a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5444162500 con 0x7f544415dc20 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f544a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54440b5fd0 con 0x7f54440ced80 2024-04-25T10:12:32.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.589+0000 7f5443e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 0x7f54440cd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 0x7f54440cd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41608/0 (socket says 172.21.15.165:41608) 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 -- 172.21.15.165:0/3296781166 learned_addr learned my addr 172.21.15.165:0/3296781166 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443400640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54440ced80 0x7f544415af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5442a00640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544415dc20 0x7f5444160630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 -- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54440ced80 msgr2=0x7f544415af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54440ced80 0x7f544415af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 -- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544415dc20 msgr2=0x7f5444160630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544415dc20 0x7f5444160630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 -- 172.21.15.165:0/3296781166 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544415b460 con 0x7f5444161710 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5442a00640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544415dc20 0x7f5444160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.590+0000 7f5443e00640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 0x7f54440cd910 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f543c00c800 tx=0x7f543c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.591+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c018070 con 0x7f5444161710 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.591+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c004d80 con 0x7f5444161710 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.591+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c00a880 con 0x7f5444161710 2024-04-25T10:12:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.591+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f544415f590 con 0x7f5444161710 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.591+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f544415fb50 con 0x7f5444161710 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.592+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5444073a00 con 0x7f5444161710 2024-04-25T10:12:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.593+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f543c00aa20 con 0x7f5444161710 2024-04-25T10:12:32.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.594+0000 7f543b400640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:32.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.595+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f543c089680 con 0x7f5444161710 2024-04-25T10:12:32.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.595+0000 7f5443400640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:32.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.595+0000 7f5443400640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f542406b4d0 0x7f542406d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5434004ea0 tx=0x7f5434009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:32.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.599+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f543c0586e0 con 0x7f5444161710 2024-04-25T10:12:32.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.857+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f54440b56a0 con 0x7f5444161710 2024-04-25T10:12:32.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.858+0000 7f543b400640 1 -- 172.21.15.165:0/3296781166 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f543c057fe0 con 0x7f5444161710 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.863 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:12:32.864 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.865 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.866 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.867 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:12:32.868 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.869 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.870 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.871 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:12:32.872 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.873 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:12:32.874 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:12:32.875 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:12:32.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f542406b4d0 msgr2=0x7f542406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f542406b4d0 0x7f542406d990 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f5434004ea0 tx=0x7f5434009040 comp rx=0 tx=0).stop 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 msgr2=0x7f54440cd910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 0x7f54440cd910 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f543c00c800 tx=0x7f543c00ccc0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 shutdown_connections 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f542406b4d0 0x7f542406d990 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5444161710 0x7f54440cd910 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544415dc20 0x7f5444160630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 --2- 172.21.15.165:0/3296781166 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f54440ced80 0x7f544415af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 >> 172.21.15.165:0/3296781166 conn(0x7f54440d0420 msgr2=0x7f544415ce50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 shutdown_connections 2024-04-25T10:12:32.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:32.861+0000 7f544a200640 1 -- 172.21.15.165:0/3296781166 wait complete. 2024-04-25T10:12:32.883 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:33.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.076+0000 7ff751e00640 1 Processor -- start 2024-04-25T10:12:33.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.076+0000 7ff751e00640 1 -- start start 2024-04-25T10:12:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.078+0000 7ff751e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff74c144750 0x7ff74c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.078+0000 7ff751e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74c144b30 con 0x7ff74c06eb10 2024-04-25T10:12:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.078+0000 7ff751e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74c0712d0 con 0x7ff74c0cf030 2024-04-25T10:12:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.078+0000 7ff751e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff74c0715b0 con 0x7ff74c144750 2024-04-25T10:12:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.078+0000 7ff743e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff74c06eb10 0x7ff74c0ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44234/0 (socket says 172.21.15.165:44234) 2024-04-25T10:12:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.078+0000 7ff743e00640 1 -- 172.21.15.165:0/2051813633 learned_addr learned my addr 172.21.15.165:0/2051813633 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358212824 0 0) 0x7ff74c144b30 con 0x7ff74c06eb10 2024-04-25T10:12:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff730003520 con 0x7ff74c06eb10 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74aa00640 1 --2- 172.21.15.165:0/2051813633 >> v2:172.21.15.172:3300/0 conn(0x7ff74c144750 0x7ff74c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 718751886 0 0) 0x7ff730003520 con 0x7ff74c06eb10 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 >> v2:172.21.15.172:3300/0 conn(0x7ff74c144750 msgr2=0x7ff74c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 --2- 172.21.15.165:0/2051813633 >> v2:172.21.15.172:3300/0 conn(0x7ff74c144750 0x7ff74c06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 >> v1:172.21.15.172:6789/0 conn(0x7ff74c0cf030 legacy=0x7ff74c06d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c071890 con 0x7ff74c06eb10 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff734001670 con 0x7ff74c06eb10 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 >> v1:172.21.15.165:6789/0 conn(0x7ff74c06eb10 legacy=0x7ff74c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.079+0000 7ff74a000640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7300039b0 0x7ff730003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74a000640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730004800 0x7ff730004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74a000640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff73000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c0715b0 con 0x7ff7300051c0 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff73000c0f0 con 0x7ff730004800 2024-04-25T10:12:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff73000c2f0 con 0x7ff7300039b0 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74b400640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff73000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74aa00640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7300039b0 0x7ff730003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff743e00640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730004800 0x7ff730004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74b400640 1 -- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7300039b0 msgr2=0x7ff730003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74b400640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7300039b0 0x7ff730003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74b400640 1 -- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730004800 msgr2=0x7ff730004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74b400640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730004800 0x7ff730004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff74b400640 1 -- 172.21.15.165:0/2051813633 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff730003520 con 0x7ff7300051c0 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.080+0000 7ff743e00640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff730004800 0x7ff730004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.081+0000 7ff74b400640 1 -- 172.21.15.165:0/2051813633 reap_dead start 2024-04-25T10:12:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.081+0000 7ff74b400640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff73000b9c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff744004480 tx=0x7ff730005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c05f156c437b6f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.081+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744004080 con 0x7ff7300051c0 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.081+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74402ad80 con 0x7ff7300051c0 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff74a000640 1 -- 172.21.15.165:0/2051813633 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff744006d90 con 0x7ff7300051c0 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 -- 172.21.15.165:0/2051813633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 msgr2=0x7ff73000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff73000b9c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7ff744004480 tx=0x7ff730005830 comp rx=0 tx=0).stop 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 -- 172.21.15.165:0/2051813633 shutdown_connections 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 --2- 172.21.15.165:0/2051813633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff73000b9c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 -- 172.21.15.165:0/2051813633 >> 172.21.15.165:0/2051813633 conn(0x7ff74c06abe0 msgr2=0x7ff74c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 -- 172.21.15.165:0/2051813633 shutdown_connections 2024-04-25T10:12:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.082+0000 7ff751e00640 1 -- 172.21.15.165:0/2051813633 wait complete. 2024-04-25T10:12:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.084+0000 7ff751e00640 1 Processor -- start 2024-04-25T10:12:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.084+0000 7ff751e00640 1 -- start start 2024-04-25T10:12:33.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff751e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff74c15aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff751e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c13f9a0 0x7ff74c0b0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff751e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff74c0b10b0 0x7ff74c1f2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff751e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c1455d0 con 0x7ff7300051c0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff751e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c145450 con 0x7ff74c13f9a0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff751e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff74c15c000 con 0x7ff74c0b10b0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff74aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff74c15aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff74aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff74c15aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41630/0 (socket says 172.21.15.165:41630) 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.085+0000 7ff74aa00640 1 -- 172.21.15.165:0/4108236781 learned_addr learned my addr 172.21.15.165:0/4108236781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74b400640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff74c0b10b0 0x7ff74c1f2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff743e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c13f9a0 0x7ff74c0b0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74aa00640 1 -- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff74c0b10b0 msgr2=0x7ff74c1f2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74aa00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff74c0b10b0 0x7ff74c1f2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74aa00640 1 -- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c13f9a0 msgr2=0x7ff74c0b0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74aa00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c13f9a0 0x7ff74c0b0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74aa00640 1 -- 172.21.15.165:0/4108236781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff74c135fa0 con 0x7ff7300051c0 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff743e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c13f9a0 0x7ff74c0b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:33.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff74aa00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff74c15aa40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e7e0 tx=0x7ff73c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c017070 con 0x7ff7300051c0 2024-04-25T10:12:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73c00c6d0 con 0x7ff7300051c0 2024-04-25T10:12:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff73c00f040 con 0x7ff7300051c0 2024-04-25T10:12:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.086+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff74c136280 con 0x7ff7300051c0 2024-04-25T10:12:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.087+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff74c136840 con 0x7ff7300051c0 2024-04-25T10:12:33.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.088+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff74c15d480 con 0x7ff7300051c0 2024-04-25T10:12:33.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.089+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff73c01a070 con 0x7ff7300051c0 2024-04-25T10:12:33.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.089+0000 7ff743400640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff72806b500 0x7ff72806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.090+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff73c089d20 con 0x7ff7300051c0 2024-04-25T10:12:33.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.090+0000 7ff743e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff72806b500 0x7ff72806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.090+0000 7ff743e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff72806b500 0x7ff72806d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff734001eb0 tx=0x7ff734009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.093+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff73c058cc0 con 0x7ff7300051c0 2024-04-25T10:12:33.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.281+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff74c073990 con 0x7ff7300051c0 2024-04-25T10:12:33.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.283+0000 7ff743400640 1 -- 172.21.15.165:0/4108236781 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+13202 (secure 0 0 0) 0x7ff73c0585c0 con 0x7ff7300051c0 2024-04-25T10:12:33.288 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:33.288 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":107,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:32.374151+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":99,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":96,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff72806b500 msgr2=0x7ff72806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff72806b500 0x7ff72806d9c0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7ff734001eb0 tx=0x7ff734009040 comp rx=0 tx=0).stop 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 msgr2=0x7ff74c15aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff74c15aa40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7ff73c00e7e0 tx=0x7ff73c00eca0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 shutdown_connections 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff72806b500 0x7ff72806d9c0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff74c0b10b0 0x7ff74c1f2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff74c13f9a0 0x7ff74c0b0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 --2- 172.21.15.165:0/4108236781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7300051c0 0x7ff74c15aa40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.286+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 >> 172.21.15.165:0/4108236781 conn(0x7ff74c06abe0 msgr2=0x7ff74c0cf480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.287+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 shutdown_connections 2024-04-25T10:12:33.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.287+0000 7ff751e00640 1 -- 172.21.15.165:0/4108236781 wait complete. 2024-04-25T10:12:33.309 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:12:33.309 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:12:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.491+0000 7fd229400640 1 Processor -- start 2024-04-25T10:12:33.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.492+0000 7fd229400640 1 -- start start 2024-04-25T10:12:33.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.493+0000 7fd229400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd22415b710 0x7fd2240cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.493+0000 7fd229400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd22406ec50 0x7fd22415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.493+0000 7fd229400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd22406f0e0 con 0x7fd22415ee60 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.493+0000 7fd229400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22415e490 con 0x7fd22406ec50 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.493+0000 7fd229400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2240cfd80 con 0x7fd22415b710 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd22406ec50 0x7fd22415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd22406ec50 0x7fd22415e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41640/0 (socket says 172.21.15.165:41640) 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 -- 172.21.15.165:0/130197497 learned_addr learned my addr 172.21.15.165:0/130197497 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd222800640 1 --2- 172.21.15.165:0/130197497 >> v2:172.21.15.172:3300/0 conn(0x7fd22415b710 0x7fd2240cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 -- 172.21.15.165:0/130197497 >> v2:172.21.15.172:3300/0 conn(0x7fd22415b710 msgr2=0x7fd2240cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 --2- 172.21.15.165:0/130197497 >> v2:172.21.15.172:3300/0 conn(0x7fd22415b710 0x7fd2240cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 -- 172.21.15.165:0/130197497 >> v1:172.21.15.172:6789/0 conn(0x7fd22415ee60 legacy=0x7fd2240ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd219e00640 1 -- 172.21.15.165:0/130197497 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240d0060 con 0x7fd22406ec50 2024-04-25T10:12:33.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.494+0000 7fd222800640 1 --2- 172.21.15.165:0/130197497 >> v2:172.21.15.172:3300/0 conn(0x7fd22415b710 0x7fd2240cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.495+0000 7fd219e00640 1 --2- 172.21.15.165:0/130197497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd22415e920 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd22406b950 tx=0x7fd20c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=919b15eff3108846 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.495+0000 7fd221e00640 1 -- 172.21.15.165:0/130197497 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c037070 con 0x7fd22406ec50 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd221e00640 1 -- 172.21.15.165:0/130197497 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd20c032440 con 0x7fd22406ec50 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd221e00640 1 -- 172.21.15.165:0/130197497 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd20c032810 con 0x7fd22406ec50 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 -- 172.21.15.165:0/130197497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 msgr2=0x7fd22415e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 --2- 172.21.15.165:0/130197497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd22415e920 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fd22406b950 tx=0x7fd20c02a230 comp rx=0 tx=0).stop 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 -- 172.21.15.165:0/130197497 shutdown_connections 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 --2- 172.21.15.165:0/130197497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd22415e920 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 --2- 172.21.15.165:0/130197497 >> v2:172.21.15.172:3300/0 conn(0x7fd22415b710 0x7fd2240cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 -- 172.21.15.165:0/130197497 >> 172.21.15.165:0/130197497 conn(0x7fd22406abe0 msgr2=0x7fd22406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 -- 172.21.15.165:0/130197497 shutdown_connections 2024-04-25T10:12:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.496+0000 7fd229400640 1 -- 172.21.15.165:0/130197497 wait complete. 2024-04-25T10:12:33.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.498+0000 7fd229400640 1 Processor -- start 2024-04-25T10:12:33.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.498+0000 7fd229400640 1 -- start start 2024-04-25T10:12:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd229400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd2240b0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd229400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd22415b710 0x7fd224157340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd229400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd22415ee60 0x7fd2241f3440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd229400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415bc70 con 0x7fd22406ec50 2024-04-25T10:12:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd229400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd22415baf0 con 0x7fd22415ee60 2024-04-25T10:12:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd229400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2240acfc0 con 0x7fd22415b710 2024-04-25T10:12:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd2240b0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd2240b0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41652/0 (socket says 172.21.15.165:41652) 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 -- 172.21.15.165:0/3150351683 learned_addr learned my addr 172.21.15.165:0/3150351683 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 -- 172.21.15.165:0/3150351683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd22415b710 msgr2=0x7fd224157340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd22415b710 0x7fd224157340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 -- 172.21.15.165:0/3150351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd22415ee60 msgr2=0x7fd2241f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd22415ee60 0x7fd2241f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.499+0000 7fd222800640 1 -- 172.21.15.165:0/3150351683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2240b15a0 con 0x7fd22406ec50 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.500+0000 7fd222800640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd2240b0de0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd21400c7f0 tx=0x7fd21400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.500+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214015070 con 0x7fd22406ec50 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.500+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd214007a30 con 0x7fd22406ec50 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.500+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2240b1860 con 0x7fd22406ec50 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.500+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd214007e00 con 0x7fd22406ec50 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.500+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2240b1e20 con 0x7fd22406ec50 2024-04-25T10:12:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.501+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd224073a00 con 0x7fd22406ec50 2024-04-25T10:12:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.502+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd214018070 con 0x7fd22406ec50 2024-04-25T10:12:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.503+0000 7fd21b400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd20006b4d0 0x7fd20006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.503+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fd21408a0d0 con 0x7fd22406ec50 2024-04-25T10:12:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.503+0000 7fd219e00640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd20006b4d0 0x7fd20006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.504+0000 7fd219e00640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd20006b4d0 0x7fd20006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd20c034000 tx=0x7fd20c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.507+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd214058ff0 con 0x7fd22406ec50 2024-04-25T10:12:33.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.688+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd22406e210 con 0x7fd22406ec50 2024-04-25T10:12:33.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.689+0000 7fd21b400640 1 -- 172.21.15.165:0/3150351683 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v107) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2140588f0 con 0x7fd22406ec50 2024-04-25T10:12:33.693 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd20006b4d0 msgr2=0x7fd20006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd20006b4d0 0x7fd20006d990 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fd20c034000 tx=0x7fd20c02c000 comp rx=0 tx=0).stop 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 msgr2=0x7fd2240b0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd2240b0de0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7fd21400c7f0 tx=0x7fd21400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 shutdown_connections 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd20006b4d0 0x7fd20006d990 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd22415ee60 0x7fd2241f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd22415b710 0x7fd224157340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 --2- 172.21.15.165:0/3150351683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd22406ec50 0x7fd2240b0de0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.691+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 >> 172.21.15.165:0/3150351683 conn(0x7fd22406abe0 msgr2=0x7fd22415a510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.692+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 shutdown_connections 2024-04-25T10:12:33.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.692+0000 7fd229400640 1 -- 172.21.15.165:0/3150351683 wait complete. 2024-04-25T10:12:33.714 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:12:33.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.912+0000 7f0742000640 1 Processor -- start 2024-04-25T10:12:33.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.912+0000 7f0742000640 1 -- start start 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.913+0000 7f0742000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c15db10 0x7f073c162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.913+0000 7f0742000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f073c0b3680 0x7f073c0cfb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.913+0000 7f0742000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073c071670 con 0x7f073c0b3220 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.913+0000 7f0742000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c072280 con 0x7f073c0b3680 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.913+0000 7f0742000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c162770 con 0x7f073c15db10 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.914+0000 7f0732a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f073c0b3220 0x7f073c0cf3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44246/0 (socket says 172.21.15.165:44246) 2024-04-25T10:12:33.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.914+0000 7f0732a00640 1 -- 172.21.15.165:0/3375443625 learned_addr learned my addr 172.21.15.165:0/3375443625 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.914+0000 7f073be00640 1 --2- 172.21.15.165:0/3375443625 >> v2:172.21.15.165:3300/0 conn(0x7f073c0b3680 0x7f073c0cfb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.914+0000 7f073b400640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c15db10 0x7f073c162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.914+0000 7f073aa00640 1 -- 172.21.15.165:0/3375443625 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4018207444 0 0) 0x7f073c071670 con 0x7f073c0b3220 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.914+0000 7f073aa00640 1 -- 172.21.15.165:0/3375443625 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0720003520 con 0x7f073c0b3220 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.915+0000 7f073be00640 1 -- 172.21.15.165:0/3375443625 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c15db10 msgr2=0x7f073c162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.915+0000 7f073be00640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c15db10 0x7f073c162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.915+0000 7f073be00640 1 -- 172.21.15.165:0/3375443625 >> v1:172.21.15.165:6789/0 conn(0x7f073c0b3220 legacy=0x7f073c0cf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.915+0000 7f073aa00640 1 -- 172.21.15.165:0/3375443625 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 616581220 0 0) 0x7f0720003520 con 0x7f073c0b3220 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.915+0000 7f073be00640 1 -- 172.21.15.165:0/3375443625 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c071950 con 0x7f073c0b3680 2024-04-25T10:12:33.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.915+0000 7f073b400640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c15db10 0x7f073c162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:33.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.916+0000 7f073be00640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0cfb00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0734009d90 tx=0x7f073402a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c92117c012dedfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.916+0000 7f073aa00640 1 -- 172.21.15.165:0/3375443625 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724001670 con 0x7f073c0b3680 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.916+0000 7f073aa00640 1 -- 172.21.15.165:0/3375443625 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0734007540 con 0x7f073c0b3680 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.916+0000 7f073aa00640 1 -- 172.21.15.165:0/3375443625 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0734007910 con 0x7f073c0b3680 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.916+0000 7f0742000640 1 -- 172.21.15.165:0/3375443625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 msgr2=0x7f073c0cfb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.916+0000 7f0742000640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0cfb00 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f0734009d90 tx=0x7f073402a5f0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.917+0000 7f0742000640 1 -- 172.21.15.165:0/3375443625 shutdown_connections 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.917+0000 7f0742000640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0cfb00 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.917+0000 7f0742000640 1 --2- 172.21.15.165:0/3375443625 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c15db10 0x7f073c162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.917+0000 7f0742000640 1 -- 172.21.15.165:0/3375443625 >> 172.21.15.165:0/3375443625 conn(0x7f073c06abe0 msgr2=0x7f073c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:33.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.917+0000 7f0742000640 1 -- 172.21.15.165:0/3375443625 shutdown_connections 2024-04-25T10:12:33.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.917+0000 7f0742000640 1 -- 172.21.15.165:0/3375443625 wait complete. 2024-04-25T10:12:33.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.918+0000 7f0742000640 1 Processor -- start 2024-04-25T10:12:33.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.918+0000 7f0742000640 1 -- start start 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.919+0000 7f0742000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c0b3220 0x7f073c149580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.919+0000 7f0742000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0ce180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.919+0000 7f0742000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073c15db10 0x7f073c15ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.919+0000 7f0742000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c0b3b10 con 0x7f073c0b3680 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.919+0000 7f0742000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c071f10 con 0x7f073c15db10 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.919+0000 7f0742000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c0af0e0 con 0x7f073c0b3220 2024-04-25T10:12:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0ce180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0ce180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41670/0 (socket says 172.21.15.165:41670) 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 -- 172.21.15.165:0/1138292310 learned_addr learned my addr 172.21.15.165:0/1138292310 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 -- 172.21.15.165:0/1138292310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c0b3220 msgr2=0x7f073c149580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c0b3220 0x7f073c149580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 -- 172.21.15.165:0/1138292310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073c15db10 msgr2=0x7f073c15ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073c15db10 0x7f073c15ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 -- 172.21.15.165:0/1138292310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c0cc5d0 con 0x7f073c0b3680 2024-04-25T10:12:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.920+0000 7f0732a00640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0ce180 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f07240011e0 tx=0x7f072400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.921+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724017070 con 0x7f073c0b3680 2024-04-25T10:12:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.921+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0724005ad0 con 0x7f073c0b3680 2024-04-25T10:12:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.921+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c0cc8b0 con 0x7f073c0b3680 2024-04-25T10:12:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.921+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0724007070 con 0x7f073c0b3680 2024-04-25T10:12:33.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.921+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c0cce70 con 0x7f073c0b3680 2024-04-25T10:12:33.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.922+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c073a00 con 0x7f073c0b3680 2024-04-25T10:12:33.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.923+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f072401a070 con 0x7f073c0b3680 2024-04-25T10:12:33.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.924+0000 7f0733e00640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f071806b4d0 0x7f071806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:33.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.924+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f0724089620 con 0x7f073c0b3680 2024-04-25T10:12:33.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.924+0000 7f073b400640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f071806b4d0 0x7f071806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:33.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.925+0000 7f073b400640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f071806b4d0 0x7f071806d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f072c006fd0 tx=0x7f072c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:33.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:33.928+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07240585c0 con 0x7f073c0b3680 2024-04-25T10:12:34.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.126+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f073c158150 con 0x7f073c0b3680 2024-04-25T10:12:34.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.127+0000 7f0733e00640 1 -- 172.21.15.165:0/1138292310 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v107) v1 ==== 84+0+65 (secure 0 0 0) 0x7f0724057ec0 con 0x7f073c0b3680 2024-04-25T10:12:34.131 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:34.132 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:12:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:12:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:12:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:12:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:12:34.132 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.129+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f071806b4d0 msgr2=0x7f071806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f071806b4d0 0x7f071806d990 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f072c006fd0 tx=0x7f072c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 msgr2=0x7f073c0ce180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0ce180 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f07240011e0 tx=0x7f072400cbe0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 shutdown_connections 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f071806b4d0 0x7f071806d990 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f073c15db10 0x7f073c15ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f073c0b3680 0x7f073c0ce180 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 --2- 172.21.15.165:0/1138292310 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f073c0b3220 0x7f073c149580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 >> 172.21.15.165:0/1138292310 conn(0x7f073c06abe0 msgr2=0x7f073c0cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 shutdown_connections 2024-04-25T10:12:34.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.130+0000 7f0742000640 1 -- 172.21.15.165:0/1138292310 wait complete. 2024-04-25T10:12:34.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:12:34.155 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:12:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.349+0000 7fba79a00640 1 Processor -- start 2024-04-25T10:12:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.349+0000 7fba79a00640 1 -- start start 2024-04-25T10:12:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba79a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba740b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba79a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fba7406e150 0x7fba7406e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba79a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba7413e640 con 0x7fba7413d9f0 2024-04-25T10:12:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba79a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba740b10d0 con 0x7fba7406e150 2024-04-25T10:12:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba79a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba7406d390 con 0x7fba740b0730 2024-04-25T10:12:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba73e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fba7413d9f0 0x7fba7413df30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44248/0 (socket says 172.21.15.165:44248) 2024-04-25T10:12:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.351+0000 7fba73e00640 1 -- 172.21.15.165:0/3143597352 learned_addr learned my addr 172.21.15.165:0/3143597352 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089489418 0 0) 0x7fba7413e640 con 0x7fba7413d9f0 2024-04-25T10:12:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba58003520 con 0x7fba7413d9f0 2024-04-25T10:12:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba79000640 1 --2- 172.21.15.165:0/3143597352 >> v2:172.21.15.172:3300/0 conn(0x7fba7406e150 0x7fba7406e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba73400640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba740b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 151394138 0 0) 0x7fba58003520 con 0x7fba7413d9f0 2024-04-25T10:12:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 msgr2=0x7fba740b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba740b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 >> v2:172.21.15.172:3300/0 conn(0x7fba7406e150 msgr2=0x7fba7406e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 --2- 172.21.15.165:0/3143597352 >> v2:172.21.15.172:3300/0 conn(0x7fba7406e150 0x7fba7406e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7406d670 con 0x7fba7413d9f0 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fba5c001670 con 0x7fba7413d9f0 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 >> v1:172.21.15.165:6789/0 conn(0x7fba7413d9f0 legacy=0x7fba7413df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba580039b0 0x7fba58003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba58004d40 0x7fba580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7406d390 con 0x7fba58004d40 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba5800ddc0 con 0x7fba580039b0 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.352+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba5800dfc0 con 0x7fba740b0730 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba79000640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba58004d40 0x7fba580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba73e00640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba580039b0 0x7fba58003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba73400640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba79000640 1 -- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 msgr2=0x7fba58004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba79000640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba58004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba79000640 1 -- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba580039b0 msgr2=0x7fba58003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba79000640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba580039b0 0x7fba58003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba79000640 1 -- 172.21.15.165:0/3143597352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba58003520 con 0x7fba58004d40 2024-04-25T10:12:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.353+0000 7fba73400640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba740b0730 0x7fba58004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.354+0000 7fba79000640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba58004d40 0x7fba580051c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00ba00 tx=0x7fba58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3b90efe46583a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c030070 con 0x7fba58004d40 2024-04-25T10:12:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c02ade0 con 0x7fba58004d40 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba72a00640 1 -- 172.21.15.165:0/3143597352 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00b600 con 0x7fba58004d40 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79a00640 1 -- 172.21.15.165:0/3143597352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba58004d40 msgr2=0x7fba580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79a00640 1 --2- 172.21.15.165:0/3143597352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba58004d40 0x7fba580051c0 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7fba6c00ba00 tx=0x7fba58005830 comp rx=0 tx=0).stop 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79000640 1 -- 172.21.15.165:0/3143597352 reap_dead start 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79a00640 1 -- 172.21.15.165:0/3143597352 shutdown_connections 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79a00640 1 -- 172.21.15.165:0/3143597352 >> 172.21.15.165:0/3143597352 conn(0x7fba74061cc0 msgr2=0x7fba7406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79a00640 1 -- 172.21.15.165:0/3143597352 shutdown_connections 2024-04-25T10:12:34.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.355+0000 7fba79a00640 1 -- 172.21.15.165:0/3143597352 wait complete. 2024-04-25T10:12:34.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.357+0000 7fba79a00640 1 Processor -- start 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.357+0000 7fba79a00640 1 -- start start 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.358+0000 7fba79a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba58004d40 0x7fba74138290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.358+0000 7fba79a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 0x7fba7413bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.358+0000 7fba79a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba740b2700 0x7fba740b2b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.358+0000 7fba79a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7406eb20 con 0x7fba7413b840 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.358+0000 7fba79a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba7406dc30 con 0x7fba740b2700 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.358+0000 7fba79a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba740b0b90 con 0x7fba58004d40 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 0x7fba7413bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 0x7fba7413bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41678/0 (socket says 172.21.15.165:41678) 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 -- 172.21.15.165:0/3694049888 learned_addr learned my addr 172.21.15.165:0/3694049888 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73e00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba58004d40 0x7fba74138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 -- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba58004d40 msgr2=0x7fba74138290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba58004d40 0x7fba74138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 -- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba740b2700 msgr2=0x7fba740b2b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:34.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba740b2700 0x7fba740b2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:34.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 -- 172.21.15.165:0/3694049888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba740b3470 con 0x7fba7413b840 2024-04-25T10:12:34.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.359+0000 7fba73400640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 0x7fba7413bca0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fba64007f20 tx=0x7fba6400f5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.360+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba64018020 con 0x7fba7413b840 2024-04-25T10:12:34.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.361+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640145f0 con 0x7fba7413b840 2024-04-25T10:12:34.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.361+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba740b3750 con 0x7fba7413b840 2024-04-25T10:12:34.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.361+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640149c0 con 0x7fba7413b840 2024-04-25T10:12:34.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.361+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba740b3d10 con 0x7fba7413b840 2024-04-25T10:12:34.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.362+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba74073990 con 0x7fba7413b840 2024-04-25T10:12:34.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.363+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fba6401b020 con 0x7fba7413b840 2024-04-25T10:12:34.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.364+0000 7fba6be00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba5006b4d0 0x7fba5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:34.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.364+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fba6408ad00 con 0x7fba7413b840 2024-04-25T10:12:34.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.364+0000 7fba73e00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba5006b4d0 0x7fba5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:34.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.365+0000 7fba73e00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba5006b4d0 0x7fba5006d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fba5c001120 tx=0x7fba5c000f90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:34.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.369+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fba64022020 con 0x7fba7413b840 2024-04-25T10:12:34.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:34.567+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fba74159180 con 0x7fba7413b840 2024-04-25T10:12:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.385+0000 7fba6be00640 1 -- 172.21.15.165:0/3694049888 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v108) v1 ==== 138+0+0 (secure 0 0 0) 0x7fba64059ca0 con 0x7fba7413b840 2024-04-25T10:12:35.390 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:12:35.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.388+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba5006b4d0 msgr2=0x7fba5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.388+0000 7fba79a00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba5006b4d0 0x7fba5006d990 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fba5c001120 tx=0x7fba5c000f90 comp rx=0 tx=0).stop 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.388+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 msgr2=0x7fba7413bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.388+0000 7fba79a00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 0x7fba7413bca0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7fba64007f20 tx=0x7fba6400f5e0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 shutdown_connections 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fba5006b4d0 0x7fba5006d990 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fba740b2700 0x7fba740b2b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fba7413b840 0x7fba7413bca0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 --2- 172.21.15.165:0/3694049888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fba58004d40 0x7fba74138290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 >> 172.21.15.165:0/3694049888 conn(0x7fba74061cc0 msgr2=0x7fba740b1490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 shutdown_connections 2024-04-25T10:12:35.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.389+0000 7fba79a00640 1 -- 172.21.15.165:0/3694049888 wait complete. 2024-04-25T10:12:35.420 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:12:35.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.615+0000 7f5aca600640 1 Processor -- start 2024-04-25T10:12:35.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.615+0000 7f5aca600640 1 -- start start 2024-04-25T10:12:35.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.617+0000 7f5aca600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ac41446d0 0x7f5ac40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.617+0000 7f5aca600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.617+0000 7f5aca600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ac40cccc0 con 0x7f5ac41452a0 2024-04-25T10:12:35.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.617+0000 7f5aca600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ac40b2b20 con 0x7f5ac41446d0 2024-04-25T10:12:35.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.617+0000 7f5aca600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac406d400 con 0x7f5ac4144d50 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.617+0000 7f5ac3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ac41446d0 0x7f5ac40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5ac3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ac41446d0 0x7f5ac40b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41684/0 (socket says 172.21.15.165:41684) 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5ac3400640 1 -- 172.21.15.165:0/3422735359 learned_addr learned my addr 172.21.15.165:0/3422735359 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5abaa00640 1 --2- 172.21.15.165:0/3422735359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5ac3400640 1 -- 172.21.15.165:0/3422735359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 msgr2=0x7f5ac40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5ac3400640 1 --2- 172.21.15.165:0/3422735359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5ac3400640 1 -- 172.21.15.165:0/3422735359 >> v1:172.21.15.172:6789/0 conn(0x7f5ac41452a0 legacy=0x7f5ac40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:35.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.618+0000 7f5ac3400640 1 -- 172.21.15.165:0/3422735359 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac40cc140 con 0x7f5ac41446d0 2024-04-25T10:12:35.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.619+0000 7f5ac3400640 1 --2- 172.21.15.165:0/3422735359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac40b3f20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009870 tx=0x7f5ab402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a73a6ddf3d38b9c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5ac2a00640 1 -- 172.21.15.165:0/3422735359 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4037070 con 0x7f5ac41446d0 2024-04-25T10:12:35.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5ac2a00640 1 -- 172.21.15.165:0/3422735359 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab4032440 con 0x7f5ac41446d0 2024-04-25T10:12:35.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5ac2a00640 1 -- 172.21.15.165:0/3422735359 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4032810 con 0x7f5ac41446d0 2024-04-25T10:12:35.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5aca600640 1 -- 172.21.15.165:0/3422735359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 msgr2=0x7f5ac40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5aca600640 1 --2- 172.21.15.165:0/3422735359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac40b3f20 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4009870 tx=0x7f5ab402a210 comp rx=0 tx=0).stop 2024-04-25T10:12:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5aca600640 1 -- 172.21.15.165:0/3422735359 shutdown_connections 2024-04-25T10:12:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5aca600640 1 --2- 172.21.15.165:0/3422735359 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5aca600640 1 --2- 172.21.15.165:0/3422735359 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac40b3f20 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.620+0000 7f5aca600640 1 -- 172.21.15.165:0/3422735359 >> 172.21.15.165:0/3422735359 conn(0x7f5ac406abe0 msgr2=0x7f5ac406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.621+0000 7f5aca600640 1 -- 172.21.15.165:0/3422735359 shutdown_connections 2024-04-25T10:12:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.621+0000 7f5aca600640 1 -- 172.21.15.165:0/3422735359 wait complete. 2024-04-25T10:12:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.622+0000 7f5aca600640 1 Processor -- start 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 -- start start 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac413a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40ac270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ac41452a0 0x7f5ac413c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40cc700 con 0x7f5ac41446d0 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40b49a0 con 0x7f5ac41452a0 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.623+0000 7f5aca600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ac40cc9e0 con 0x7f5ac4144d50 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac413a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac413a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41690/0 (socket says 172.21.15.165:41690) 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 -- 172.21.15.165:0/1365461519 learned_addr learned my addr 172.21.15.165:0/1365461519 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 -- 172.21.15.165:0/1365461519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 msgr2=0x7f5ac40ac270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40ac270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 -- 172.21.15.165:0/1365461519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ac41452a0 msgr2=0x7f5ac413c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ac41452a0 0x7f5ac413c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.624+0000 7f5ac3400640 1 -- 172.21.15.165:0/1365461519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ac413a940 con 0x7f5ac41446d0 2024-04-25T10:12:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.625+0000 7f5ac3400640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac413a400 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.625+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4041020 con 0x7f5ac41446d0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.625+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab40306c0 con 0x7f5ac41446d0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.625+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ac413cf10 con 0x7f5ac41446d0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.625+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab4037040 con 0x7f5ac41446d0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.625+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ac40a8540 con 0x7f5ac41446d0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.626+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ac4073a00 con 0x7f5ac41446d0 2024-04-25T10:12:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.627+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ab4044020 con 0x7f5ac41446d0 2024-04-25T10:12:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.628+0000 7f5abbe00640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5aa006b4d0 0x7f5aa006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.628+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f5ab40aa5e0 con 0x7f5ac41446d0 2024-04-25T10:12:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.628+0000 7f5abaa00640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5aa006b4d0 0x7f5aa006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.629+0000 7f5abaa00640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5aa006b4d0 0x7f5aa006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40b02a0 tx=0x7f5aac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.633+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ab404b070 con 0x7f5ac41446d0 2024-04-25T10:12:35.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:35.827+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5ac4142930 con 0x7f5ac41446d0 2024-04-25T10:12:36.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.401+0000 7f5abbe00640 1 -- 172.21.15.165:0/1365461519 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v109) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5ab4079540 con 0x7f5ac41446d0 2024-04-25T10:12:36.406 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.405+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5aa006b4d0 msgr2=0x7f5aa006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.405+0000 7f5aca600640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5aa006b4d0 0x7f5aa006d990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40b02a0 tx=0x7f5aac008040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.405+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 msgr2=0x7f5ac413a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.405+0000 7f5aca600640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac413a400 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab40046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.405+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 shutdown_connections 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.406+0000 7f5aca600640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5aa006b4d0 0x7f5aa006d990 secure :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f5ac40b02a0 tx=0x7f5aac008040 comp rx=0 tx=0).stop 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.406+0000 7f5aca600640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ac41452a0 0x7f5ac413c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.406+0000 7f5aca600640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ac4144d50 0x7f5ac40ac270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.406+0000 7f5aca600640 1 --2- 172.21.15.165:0/1365461519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ac41446d0 0x7f5ac413a400 secure :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4000c00 tx=0x7f5ab40046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.406+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 >> 172.21.15.165:0/1365461519 conn(0x7f5ac406abe0 msgr2=0x7f5ac4143630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.407+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 shutdown_connections 2024-04-25T10:12:36.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.407+0000 7f5aca600640 1 -- 172.21.15.165:0/1365461519 wait complete. 2024-04-25T10:12:36.442 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:12:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.612+0000 7fb749e00640 1 Processor -- start 2024-04-25T10:12:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.612+0000 7fb749e00640 1 -- start start 2024-04-25T10:12:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb749e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb744165a80 0x7fb744165f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb749e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7440cdcd0 con 0x7fb7440ccf30 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb749e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7441632d0 con 0x7fb74415f950 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb749e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb74406f100 con 0x7fb744165a80 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb749400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb744165a80 0x7fb744165f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb73be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb7440ccf30 0x7fb744162bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44250/0 (socket says 172.21.15.165:44250) 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb749400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb744165a80 0x7fb744165f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41694/0 (socket says 172.21.15.165:41694) 2024-04-25T10:12:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.614+0000 7fb73be00640 1 -- 172.21.15.165:0/3026464539 learned_addr learned my addr 172.21.15.165:0/3026464539 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.615+0000 7fb749400640 1 -- 172.21.15.165:0/3026464539 >> v1:172.21.15.172:6789/0 conn(0x7fb74415f950 legacy=0x7fb74406e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.615+0000 7fb749400640 1 -- 172.21.15.165:0/3026464539 >> v1:172.21.15.165:6789/0 conn(0x7fb7440ccf30 legacy=0x7fb744162bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.615+0000 7fb743e00640 1 -- 172.21.15.165:0/3026464539 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3802793493 0 0) 0x7fb7440cdcd0 con 0x7fb7440ccf30 2024-04-25T10:12:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.615+0000 7fb749400640 1 -- 172.21.15.165:0/3026464539 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7440cd430 con 0x7fb744165a80 2024-04-25T10:12:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.616+0000 7fb749400640 1 --2- 172.21.15.165:0/3026464539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb744165f30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009d90 tx=0x7fb73c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df2938f31a128649 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.616+0000 7fb743e00640 1 -- 172.21.15.165:0/3026464539 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c031600 con 0x7fb744165a80 2024-04-25T10:12:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.616+0000 7fb743e00640 1 -- 172.21.15.165:0/3026464539 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c0317a0 con 0x7fb744165a80 2024-04-25T10:12:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.616+0000 7fb743e00640 1 -- 172.21.15.165:0/3026464539 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c031b70 con 0x7fb744165a80 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 -- 172.21.15.165:0/3026464539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 msgr2=0x7fb744165f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 --2- 172.21.15.165:0/3026464539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb744165f30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009d90 tx=0x7fb73c02a610 comp rx=0 tx=0).stop 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 -- 172.21.15.165:0/3026464539 shutdown_connections 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 --2- 172.21.15.165:0/3026464539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb744165f30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 -- 172.21.15.165:0/3026464539 >> 172.21.15.165:0/3026464539 conn(0x7fb744061cc0 msgr2=0x7fb74406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 -- 172.21.15.165:0/3026464539 shutdown_connections 2024-04-25T10:12:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.617+0000 7fb749e00640 1 -- 172.21.15.165:0/3026464539 wait complete. 2024-04-25T10:12:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.619+0000 7fb749e00640 1 Processor -- start 2024-04-25T10:12:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.619+0000 7fb749e00640 1 -- start start 2024-04-25T10:12:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7440ccf30 0x7fb744160eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb74415f950 0x7fb7441613f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb7441eb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7440cd9f0 con 0x7fb744165a80 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb744162440 con 0x7fb7440ccf30 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7441626a0 con 0x7fb74415f950 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb7441eb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb7441eb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41700/0 (socket says 172.21.15.165:41700) 2024-04-25T10:12:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749400640 1 -- 172.21.15.165:0/1651109719 learned_addr learned my addr 172.21.15.165:0/1651109719 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb73be00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb74415f950 0x7fb7441613f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749400640 1 -- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb74415f950 msgr2=0x7fb7441613f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb748a00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7440ccf30 0x7fb744160eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749400640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb74415f950 0x7fb7441613f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.620+0000 7fb749400640 1 -- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7440ccf30 msgr2=0x7fb744160eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb749400640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7440ccf30 0x7fb744160eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb749400640 1 -- 172.21.15.165:0/1651109719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb744147b00 con 0x7fb744165a80 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb749400640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb7441eb3d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009d90 tx=0x7fb73c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c040070 con 0x7fb744165a80 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb73c030b60 con 0x7fb744165a80 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb73c03c420 con 0x7fb744165a80 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb744147de0 con 0x7fb744165a80 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.621+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7441483a0 con 0x7fb744165a80 2024-04-25T10:12:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.622+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb744073990 con 0x7fb744165a80 2024-04-25T10:12:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.623+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb73c039040 con 0x7fb744165a80 2024-04-25T10:12:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.624+0000 7fb741600640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb72006b4d0 0x7fb72006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.625+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fb73c0ab370 con 0x7fb744165a80 2024-04-25T10:12:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.625+0000 7fb748a00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb72006b4d0 0x7fb72006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.625+0000 7fb748a00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb72006b4d0 0x7fb72006d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb734003d90 tx=0x7fb734009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.629+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb73c07a170 con 0x7fb744165a80 2024-04-25T10:12:36.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:36.856+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb7440b1290 con 0x7fb744165a80 2024-04-25T10:12:37.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.405+0000 7fb741600640 1 -- 172.21.15.165:0/1651109719 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v70) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb73c079a70 con 0x7fb744165a80 2024-04-25T10:12:37.410 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 15 and data pool 16 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb72006b4d0 msgr2=0x7fb72006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb72006b4d0 0x7fb72006d990 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fb734003d90 tx=0x7fb734009040 comp rx=0 tx=0).stop 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 msgr2=0x7fb7441eb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb7441eb3d0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fb73c009d90 tx=0x7fb73c004a40 comp rx=0 tx=0).stop 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 shutdown_connections 2024-04-25T10:12:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb72006b4d0 0x7fb72006d990 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb744165a80 0x7fb7441eb3d0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb74415f950 0x7fb7441613f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 --2- 172.21.15.165:0/1651109719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7440ccf30 0x7fb744160eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.409+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 >> 172.21.15.165:0/1651109719 conn(0x7fb744061cc0 msgr2=0x7fb7440b2890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.410+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 shutdown_connections 2024-04-25T10:12:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.410+0000 7fb749e00640 1 -- 172.21.15.165:0/1651109719 wait complete. 2024-04-25T10:12:37.456 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:37.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.624+0000 7f2e2f200640 1 Processor -- start 2024-04-25T10:12:37.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.624+0000 7f2e2f200640 1 -- start start 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.625+0000 7f2e2f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2e28144750 0x7f2e2806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.625+0000 7f2e2f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e2806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.625+0000 7f2e2f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e28144b30 con 0x7f2e2806eb10 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.625+0000 7f2e2f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e280712d0 con 0x7f2e28144750 2024-04-25T10:12:37.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.625+0000 7f2e2f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e280715b0 con 0x7f2e280cf030 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.626+0000 7f2e2d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e2806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.626+0000 7f2e2d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e2806d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:34682/0 (socket says 172.21.15.165:34682) 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.626+0000 7f2e2d200640 1 -- 172.21.15.165:0/3390626026 learned_addr learned my addr 172.21.15.165:0/3390626026 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:37.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.626+0000 7f2e27e00640 1 --2- 172.21.15.165:0/3390626026 >> v2:172.21.15.172:3300/0 conn(0x7f2e28144750 0x7f2e2806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.627+0000 7f2e27e00640 1 -- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 msgr2=0x7f2e2806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:37.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.627+0000 7f2e27e00640 1 --2- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e2806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.627+0000 7f2e27e00640 1 -- 172.21.15.165:0/3390626026 >> v1:172.21.15.172:6789/0 conn(0x7f2e2806eb10 legacy=0x7f2e280ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.627+0000 7f2e27e00640 1 -- 172.21.15.165:0/3390626026 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28071890 con 0x7f2e28144750 2024-04-25T10:12:37.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.629+0000 7f2e27e00640 1 --2- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 0x7f2e2806e5d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2e180097a0 tx=0x7f2e1802a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758f713f626bfbb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.629+0000 7f2e26a00640 1 -- 172.21.15.165:0/3390626026 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18037070 con 0x7f2e28144750 2024-04-25T10:12:37.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.629+0000 7f2e26a00640 1 -- 172.21.15.165:0/3390626026 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1802ae50 con 0x7f2e28144750 2024-04-25T10:12:37.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.629+0000 7f2e26a00640 1 -- 172.21.15.165:0/3390626026 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18032660 con 0x7f2e28144750 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 -- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 msgr2=0x7f2e2806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 --2- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 0x7f2e2806e5d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2e180097a0 tx=0x7f2e1802a100 comp rx=0 tx=0).stop 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 -- 172.21.15.165:0/3390626026 shutdown_connections 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 --2- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e2806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 --2- 172.21.15.165:0/3390626026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 0x7f2e2806e5d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 -- 172.21.15.165:0/3390626026 >> 172.21.15.165:0/3390626026 conn(0x7f2e2806abe0 msgr2=0x7f2e2806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 -- 172.21.15.165:0/3390626026 shutdown_connections 2024-04-25T10:12:37.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.630+0000 7f2e2f200640 1 -- 172.21.15.165:0/3390626026 wait complete. 2024-04-25T10:12:37.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.632+0000 7f2e2f200640 1 Processor -- start 2024-04-25T10:12:37.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.632+0000 7f2e2f200640 1 -- start start 2024-04-25T10:12:37.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e2f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 0x7f2e2815b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e2f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e280b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e2f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 0x7f2e28141b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e2f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e281455d0 con 0x7f2e2806eb10 2024-04-25T10:12:37.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e2f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e28145450 con 0x7f2e28144750 2024-04-25T10:12:37.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e2f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e2815c000 con 0x7f2e280cf030 2024-04-25T10:12:37.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 0x7f2e2815b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 0x7f2e2815b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55588/0 (socket says 172.21.15.165:55588) 2024-04-25T10:12:37.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 -- 172.21.15.165:0/2663138705 learned_addr learned my addr 172.21.15.165:0/2663138705 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 -- 172.21.15.165:0/2663138705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 msgr2=0x7f2e280b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e280b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 -- 172.21.15.165:0/2663138705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 msgr2=0x7f2e28141b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 0x7f2e28141b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.634+0000 7f2e27e00640 1 -- 172.21.15.165:0/2663138705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e28140980 con 0x7f2e2806eb10 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.635+0000 7f2e27e00640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 0x7f2e2815b3c0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2e18035040 tx=0x7f2e180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.635+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18041020 con 0x7f2e2806eb10 2024-04-25T10:12:37.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.635+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e1802ac50 con 0x7f2e2806eb10 2024-04-25T10:12:37.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.635+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e18037040 con 0x7f2e2806eb10 2024-04-25T10:12:37.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.635+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e28140c20 con 0x7f2e2806eb10 2024-04-25T10:12:37.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.635+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e280b3850 con 0x7f2e2806eb10 2024-04-25T10:12:37.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.636+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e28073990 con 0x7f2e2806eb10 2024-04-25T10:12:37.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.637+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2e18044020 con 0x7f2e2806eb10 2024-04-25T10:12:37.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.638+0000 7f2e1fe00640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2e0406b4d0 0x7f2e0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:37.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.638+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2e180aa990 con 0x7f2e2806eb10 2024-04-25T10:12:37.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.638+0000 7f2e27400640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2e0406b4d0 0x7f2e0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:37.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.639+0000 7f2e27400640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2e0406b4d0 0x7f2e0406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2e100012e0 tx=0x7f2e10007d50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:37.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.644+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2e1804b030 con 0x7f2e2806eb10 2024-04-25T10:12:37.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.834+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2e280d0510 con 0x7f2e2806eb10 2024-04-25T10:12:37.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.837+0000 7f2e1fe00640 1 -- 172.21.15.165:0/2663138705 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f2e180795e0 con 0x7f2e2806eb10 2024-04-25T10:12:37.842 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:37.843 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":110,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:36.857896+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2e0406b4d0 msgr2=0x7f2e0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2e0406b4d0 0x7f2e0406d990 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f2e100012e0 tx=0x7f2e10007d50 comp rx=0 tx=0).stop 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 msgr2=0x7f2e2815b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 0x7f2e2815b3c0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f2e18035040 tx=0x7f2e180046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 shutdown_connections 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2e0406b4d0 0x7f2e0406d990 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2e28144750 0x7f2e28141b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2e280cf030 0x7f2e280b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 --2- 172.21.15.165:0/2663138705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2e2806eb10 0x7f2e2815b3c0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:37.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 >> 172.21.15.165:0/2663138705 conn(0x7f2e2806abe0 msgr2=0x7f2e280cd630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:37.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.840+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 shutdown_connections 2024-04-25T10:12:37.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:37.841+0000 7f2e2f200640 1 -- 172.21.15.165:0/2663138705 wait complete. 2024-04-25T10:12:37.864 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:38.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.052+0000 7f2d4fe00640 1 Processor -- start 2024-04-25T10:12:38.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.052+0000 7f2d4fe00640 1 -- start start 2024-04-25T10:12:38.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d50071700 0x7f2d500cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 0x7f2d5015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d5015bf30 con 0x7f2d50144440 2024-04-25T10:12:38.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d5015c860 con 0x7f2d50071700 2024-04-25T10:12:38.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015c210 con 0x7f2d500cd860 2024-04-25T10:12:38.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2d50144440 0x7f2d50070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:38174/0 (socket says 172.21.15.165:38174) 2024-04-25T10:12:38.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4ea00640 1 -- 172.21.15.165:0/2003752206 learned_addr learned my addr 172.21.15.165:0/2003752206 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:38.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4f400640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 0x7f2d5015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589899608 0 0) 0x7f2d5015bf30 con 0x7f2d50144440 2024-04-25T10:12:38.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.054+0000 7f2d4e000640 1 --2- 172.21.15.165:0/2003752206 >> v2:172.21.15.172:3300/0 conn(0x7f2d50071700 0x7f2d500cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d34003520 con 0x7f2d50144440 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3516898132 0 0) 0x7f2d34003520 con 0x7f2d50144440 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 msgr2=0x7f2d5015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 0x7f2d5015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 >> v2:172.21.15.172:3300/0 conn(0x7f2d50071700 msgr2=0x7f2d500cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 --2- 172.21.15.165:0/2003752206 >> v2:172.21.15.172:3300/0 conn(0x7f2d50071700 0x7f2d500cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d5015c4f0 con 0x7f2d50144440 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2d400019f0 con 0x7f2d50144440 2024-04-25T10:12:38.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 >> v1:172.21.15.165:6789/0 conn(0x7f2d50144440 legacy=0x7f2d50070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d340039b0 0x7f2d34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d34004800 0x7f2d34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 0x7f2d340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015c860 con 0x7f2d340039b0 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015c210 con 0x7f2d34004800 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3400bb00 con 0x7f2d500cd860 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4e000640 1 --2- 172.21.15.165:0/2003752206 >> v2:172.21.15.172:3300/0 conn(0x7f2d50071700 0x7f2d500cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.055+0000 7f2d4ea00640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d340039b0 0x7f2d34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.056+0000 7f2d4ea00640 1 -- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 msgr2=0x7f2d340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.056+0000 7f2d4ea00640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d500cd860 0x7f2d340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.056+0000 7f2d4ea00640 1 -- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d34004800 msgr2=0x7f2d34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.056+0000 7f2d4ea00640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d34004800 0x7f2d34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.056+0000 7f2d4ea00640 1 -- 172.21.15.165:0/2003752206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d34003520 con 0x7f2d340039b0 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4ea00640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d340039b0 0x7f2d34003dd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000a8f0 tx=0x7f2d34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fd18f834f3dcb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4000a4c0 con 0x7f2d340039b0 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d4002f070 con 0x7f2d340039b0 2024-04-25T10:12:38.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4d600640 1 -- 172.21.15.165:0/2003752206 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4002bcb0 con 0x7f2d340039b0 2024-04-25T10:12:38.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4fe00640 1 -- 172.21.15.165:0/2003752206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d340039b0 msgr2=0x7f2d34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/2003752206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d340039b0 0x7f2d34003dd0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f2d4000a8f0 tx=0x7f2d34005830 comp rx=0 tx=0).stop 2024-04-25T10:12:38.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4f400640 1 -- 172.21.15.165:0/2003752206 reap_dead start 2024-04-25T10:12:38.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4fe00640 1 -- 172.21.15.165:0/2003752206 shutdown_connections 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.057+0000 7f2d4fe00640 1 -- 172.21.15.165:0/2003752206 >> 172.21.15.165:0/2003752206 conn(0x7f2d5006abe0 msgr2=0x7f2d5006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.058+0000 7f2d4fe00640 1 -- 172.21.15.165:0/2003752206 shutdown_connections 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.058+0000 7f2d4fe00640 1 -- 172.21.15.165:0/2003752206 wait complete. 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.059+0000 7f2d4fe00640 1 Processor -- start 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.059+0000 7f2d4fe00640 1 -- start start 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d340039b0 0x7f2d5015af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d5013f400 0x7f2d5013f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 0x7f2d500cb240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500cde40 con 0x7f2d500cadc0 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d500cdcc0 con 0x7f2d5013f400 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5015ccf0 con 0x7f2d340039b0 2024-04-25T10:12:38.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.060+0000 7f2d4f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 0x7f2d500cb240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 0x7f2d500cb240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55612/0 (socket says 172.21.15.165:55612) 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 -- 172.21.15.165:0/1936987415 learned_addr learned my addr 172.21.15.165:0/1936987415 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 -- 172.21.15.165:0/1936987415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d340039b0 msgr2=0x7f2d5015af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d340039b0 0x7f2d5015af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 -- 172.21.15.165:0/1936987415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d5013f400 msgr2=0x7f2d5013f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d5013f400 0x7f2d5013f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 -- 172.21.15.165:0/1936987415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d500cb780 con 0x7f2d500cadc0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4f400640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 0x7f2d500cb240 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2d500cef20 tx=0x7f2d4800d820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4801a070 con 0x7f2d500cadc0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d48004030 con 0x7f2d500cadc0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d48004440 con 0x7f2d500cadc0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.061+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d500cba10 con 0x7f2d500cadc0 2024-04-25T10:12:38.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.062+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d500cbfd0 con 0x7f2d500cadc0 2024-04-25T10:12:38.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.063+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d5013fdc0 con 0x7f2d500cadc0 2024-04-25T10:12:38.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.064+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d4801d070 con 0x7f2d500cadc0 2024-04-25T10:12:38.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.064+0000 7f2d46a00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d3006b4d0 0x7f2d3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.065+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2d480898d0 con 0x7f2d500cadc0 2024-04-25T10:12:38.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.065+0000 7f2d4ea00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d3006b4d0 0x7f2d3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.065+0000 7f2d4ea00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d3006b4d0 0x7f2d3006d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2d40033000 tx=0x7f2d40038040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.070+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d48058520 con 0x7f2d500cadc0 2024-04-25T10:12:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.254+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d50073990 con 0x7f2d500cadc0 2024-04-25T10:12:38.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.257+0000 7f2d46a00640 1 -- 172.21.15.165:0/1936987415 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f2d48057e20 con 0x7f2d500cadc0 2024-04-25T10:12:38.262 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:38.262 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":110,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:36.857896+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d3006b4d0 msgr2=0x7f2d3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d3006b4d0 0x7f2d3006d990 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f2d40033000 tx=0x7f2d40038040 comp rx=0 tx=0).stop 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 msgr2=0x7f2d500cb240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 0x7f2d500cb240 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f2d500cef20 tx=0x7f2d4800d820 comp rx=0 tx=0).stop 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 shutdown_connections 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2d3006b4d0 0x7f2d3006d990 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d500cadc0 0x7f2d500cb240 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d5013f400 0x7f2d5013f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 --2- 172.21.15.165:0/1936987415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d340039b0 0x7f2d5015af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 >> 172.21.15.165:0/1936987415 conn(0x7f2d5006abe0 msgr2=0x7f2d50070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 shutdown_connections 2024-04-25T10:12:38.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.259+0000 7f2d4fe00640 1 -- 172.21.15.165:0/1936987415 wait complete. 2024-04-25T10:12:38.282 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:12:38.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.485+0000 7fa161000640 1 Processor -- start 2024-04-25T10:12:38.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.486+0000 7fa161000640 1 -- start start 2024-04-25T10:12:38.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.487+0000 7fa161000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa15c1616f0 0x7fa15c161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.487+0000 7fa161000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c0ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.487+0000 7fa161000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa15c06c8d0 con 0x7fa15c1610f0 2024-04-25T10:12:38.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.487+0000 7fa161000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa15c1614d0 con 0x7fa15c1616f0 2024-04-25T10:12:38.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.487+0000 7fa161000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0c6960 con 0x7fa15c0ce0a0 2024-04-25T10:12:38.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.488+0000 7fa15a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c0ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.488+0000 7fa151400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa15c1616f0 0x7fa15c161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.488+0000 7fa159e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fa15c1610f0 0x7fa15c06c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:37060/0 (socket says 172.21.15.165:37060) 2024-04-25T10:12:38.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.488+0000 7fa159e00640 1 -- 172.21.15.165:0/3354391690 learned_addr learned my addr 172.21.15.165:0/3354391690 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:38.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.489+0000 7fa15a800640 1 -- 172.21.15.165:0/3354391690 >> v2:172.21.15.172:3300/0 conn(0x7fa15c1616f0 msgr2=0x7fa15c161b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:38.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.489+0000 7fa15a800640 1 --2- 172.21.15.165:0/3354391690 >> v2:172.21.15.172:3300/0 conn(0x7fa15c1616f0 0x7fa15c161b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.489+0000 7fa15a800640 1 -- 172.21.15.165:0/3354391690 >> v1:172.21.15.172:6789/0 conn(0x7fa15c1610f0 legacy=0x7fa15c06c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.489+0000 7fa15a800640 1 -- 172.21.15.165:0/3354391690 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c162260 con 0x7fa15c0ce0a0 2024-04-25T10:12:38.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.489+0000 7fa151400640 1 --2- 172.21.15.165:0/3354391690 >> v2:172.21.15.172:3300/0 conn(0x7fa15c1616f0 0x7fa15c161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:12:38.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.490+0000 7fa15a800640 1 --2- 172.21.15.165:0/3354391690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c0ce530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa154009d90 tx=0x7fa15402a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=400de3231f3fd96b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.491+0000 7fa159400640 1 -- 172.21.15.165:0/3354391690 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154007740 con 0x7fa15c0ce0a0 2024-04-25T10:12:38.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.491+0000 7fa159400640 1 -- 172.21.15.165:0/3354391690 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1540078e0 con 0x7fa15c0ce0a0 2024-04-25T10:12:38.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.491+0000 7fa159400640 1 -- 172.21.15.165:0/3354391690 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154007cb0 con 0x7fa15c0ce0a0 2024-04-25T10:12:38.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 -- 172.21.15.165:0/3354391690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 msgr2=0x7fa15c0ce530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 --2- 172.21.15.165:0/3354391690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c0ce530 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa154009d90 tx=0x7fa15402a660 comp rx=0 tx=0).stop 2024-04-25T10:12:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 -- 172.21.15.165:0/3354391690 shutdown_connections 2024-04-25T10:12:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 --2- 172.21.15.165:0/3354391690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c0ce530 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 --2- 172.21.15.165:0/3354391690 >> v2:172.21.15.172:3300/0 conn(0x7fa15c1616f0 0x7fa15c161b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 -- 172.21.15.165:0/3354391690 >> 172.21.15.165:0/3354391690 conn(0x7fa15c06f500 msgr2=0x7fa15c06f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 -- 172.21.15.165:0/3354391690 shutdown_connections 2024-04-25T10:12:38.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.492+0000 7fa161000640 1 -- 172.21.15.165:0/3354391690 wait complete. 2024-04-25T10:12:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.494+0000 7fa161000640 1 Processor -- start 2024-04-25T10:12:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.494+0000 7fa161000640 1 -- start start 2024-04-25T10:12:38.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa161000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa161000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 0x7fa15c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa161000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa15c1616f0 0x7fa15c157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa161000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0cfce0 con 0x7fa15c1610f0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa161000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0cfa00 con 0x7fa15c1616f0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa161000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa15c0cf690 con 0x7fa15c0ce0a0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa151400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 0x7fa15c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa151400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 0x7fa15c1550e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55614/0 (socket says 172.21.15.165:55614) 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.495+0000 7fa151400640 1 -- 172.21.15.165:0/766033583 learned_addr learned my addr 172.21.15.165:0/766033583 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa159e00640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa15a800640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa15c1616f0 0x7fa15c157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa151400640 1 -- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 msgr2=0x7fa15c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa151400640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa151400640 1 -- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa15c1616f0 msgr2=0x7fa15c157be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa151400640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa15c1616f0 0x7fa15c157be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa151400640 1 -- 172.21.15.165:0/766033583 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa15c155a50 con 0x7fa15c1610f0 2024-04-25T10:12:38.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa151400640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 0x7fa15c1550e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa14400e990 tx=0x7fa14400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa144017070 con 0x7fa15c1610f0 2024-04-25T10:12:38.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.496+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa14400c9e0 con 0x7fa15c1610f0 2024-04-25T10:12:38.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.497+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa15c1587c0 con 0x7fa15c1610f0 2024-04-25T10:12:38.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.497+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14400cdb0 con 0x7fa15c1610f0 2024-04-25T10:12:38.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.497+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa15c0b5350 con 0x7fa15c1610f0 2024-04-25T10:12:38.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.498+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa15c0b4e60 con 0x7fa15c1610f0 2024-04-25T10:12:38.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.499+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa14401a070 con 0x7fa15c1610f0 2024-04-25T10:12:38.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.500+0000 7fa152a00640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa13806b4d0 0x7fa13806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:38.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.500+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa14408a310 con 0x7fa15c1610f0 2024-04-25T10:12:38.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.500+0000 7fa159e00640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa13806b4d0 0x7fa13806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:38.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.501+0000 7fa159e00640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa13806b4d0 0x7fa13806d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0700b0 tx=0x7fa14c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:38.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.504+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa144058f60 con 0x7fa15c1610f0 2024-04-25T10:12:38.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:38.744+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa15c073a00 con 0x7fa15c1610f0 2024-04-25T10:12:39.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.455+0000 7fa152a00640 1 -- 172.21.15.165:0/766033583 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v73) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa144058860 con 0x7fa15c1610f0 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa13806b4d0 msgr2=0x7fa13806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa13806b4d0 0x7fa13806d990 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa15c0700b0 tx=0x7fa14c009040 comp rx=0 tx=0).stop 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 msgr2=0x7fa15c1550e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 0x7fa15c1550e0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fa14400e990 tx=0x7fa14400ee50 comp rx=0 tx=0).stop 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 shutdown_connections 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa13806b4d0 0x7fa13806d990 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa15c1616f0 0x7fa15c157be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa15c1610f0 0x7fa15c1550e0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 --2- 172.21.15.165:0/766033583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa15c0ce0a0 0x7fa15c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 >> 172.21.15.165:0/766033583 conn(0x7fa15c06f500 msgr2=0x7fa15c0b1850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 shutdown_connections 2024-04-25T10:12:39.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.460+0000 7fa161000640 1 -- 172.21.15.165:0/766033583 wait complete. 2024-04-25T10:12:39.487 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:12:39.487 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.693+0000 7fcd0aa00640 1 Processor -- start 2024-04-25T10:12:39.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.694+0000 7fcd0aa00640 1 -- start start 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd0406ea90 0x7fcd04071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd0aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcd0406ef10 0x7fcd0406d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd0aa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd04162ba0 con 0x7fcd04161fc0 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd0aa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd04165be0 con 0x7fcd0406ef10 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd0aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04162e80 con 0x7fcd0406ea90 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd03e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcd04161fc0 0x7fcd041623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:38184/0 (socket says 172.21.15.165:38184) 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd03e00640 1 -- 172.21.15.165:0/3856448089 learned_addr learned my addr 172.21.15.165:0/3856448089 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:39.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.695+0000 7fcd08a00640 1 --2- 172.21.15.165:0/3856448089 >> v2:172.21.15.165:3300/0 conn(0x7fcd0406ef10 0x7fcd0406d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fccfb400640 1 --2- 172.21.15.165:0/3856448089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd0406ea90 0x7fcd04071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fcd03400640 1 -- 172.21.15.165:0/3856448089 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1946740056 0 0) 0x7fcd04162ba0 con 0x7fcd04161fc0 2024-04-25T10:12:39.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fcd03400640 1 -- 172.21.15.165:0/3856448089 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcce8003520 con 0x7fcd04161fc0 2024-04-25T10:12:39.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fcd08a00640 1 -- 172.21.15.165:0/3856448089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd0406ea90 msgr2=0x7fcd04071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:39.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fcd08a00640 1 --2- 172.21.15.165:0/3856448089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd0406ea90 0x7fcd04071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fcd08a00640 1 -- 172.21.15.165:0/3856448089 >> v1:172.21.15.165:6789/0 conn(0x7fcd04161fc0 legacy=0x7fcd041623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.696+0000 7fcd08a00640 1 -- 172.21.15.165:0/3856448089 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd04163160 con 0x7fcd0406ef10 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.697+0000 7fcd08a00640 1 --2- 172.21.15.165:0/3856448089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ef10 0x7fcd0406d2d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009d90 tx=0x7fccfc02a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4786584d6c067ede server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.697+0000 7fcd03400640 1 -- 172.21.15.165:0/3856448089 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc031600 con 0x7fcd0406ef10 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd03400640 1 -- 172.21.15.165:0/3856448089 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccfc0317a0 con 0x7fcd0406ef10 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd03400640 1 -- 172.21.15.165:0/3856448089 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccfc031b70 con 0x7fcd0406ef10 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3856448089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ef10 msgr2=0x7fcd0406d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3856448089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ef10 0x7fcd0406d2d0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fccfc009d90 tx=0x7fccfc02a5d0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3856448089 shutdown_connections 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3856448089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ef10 0x7fcd0406d2d0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3856448089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd0406ea90 0x7fcd04071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3856448089 >> 172.21.15.165:0/3856448089 conn(0x7fcd0406abe0 msgr2=0x7fcd0406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.698+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3856448089 shutdown_connections 2024-04-25T10:12:39.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.699+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3856448089 wait complete. 2024-04-25T10:12:39.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.700+0000 7fcd0aa00640 1 Processor -- start 2024-04-25T10:12:39.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.700+0000 7fcd0aa00640 1 -- start start 2024-04-25T10:12:39.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 0x7fcd041fa980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd0406ef10 0x7fcd041faec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd0aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd04161fc0 0x7fcd041eb790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd0aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd04071b40 con 0x7fcd0406ea90 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd0aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd040719c0 con 0x7fcd0406ef10 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd0aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd0406d810 con 0x7fcd04161fc0 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.701+0000 7fcd03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 0x7fcd041fa980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 0x7fcd041fa980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55626/0 (socket says 172.21.15.165:55626) 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 -- 172.21.15.165:0/3853828327 learned_addr learned my addr 172.21.15.165:0/3853828327 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 -- 172.21.15.165:0/3853828327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd04161fc0 msgr2=0x7fcd041eb790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd04161fc0 0x7fcd041eb790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 -- 172.21.15.165:0/3853828327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd0406ef10 msgr2=0x7fcd041faec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd0406ef10 0x7fcd041faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 -- 172.21.15.165:0/3853828327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd040a8540 con 0x7fcd0406ea90 2024-04-25T10:12:39.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd03e00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 0x7fcd041fa980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c390 tx=0x7fccf400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf4017070 con 0x7fcd0406ea90 2024-04-25T10:12:39.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccf4005ce0 con 0x7fcd0406ea90 2024-04-25T10:12:39.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.702+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd041ec410 con 0x7fcd0406ea90 2024-04-25T10:12:39.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.703+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccf40072f0 con 0x7fcd0406ea90 2024-04-25T10:12:39.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.703+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd040a8a90 con 0x7fcd0406ea90 2024-04-25T10:12:39.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.704+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd040a8c90 con 0x7fcd0406ea90 2024-04-25T10:12:39.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.705+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fccf401a070 con 0x7fcd0406ea90 2024-04-25T10:12:39.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.706+0000 7fcd00c00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b4d0 0x7fcce006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:39.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.706+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fccf4089740 con 0x7fcd0406ea90 2024-04-25T10:12:39.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.706+0000 7fccfb400640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b4d0 0x7fcce006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:39.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.706+0000 7fccfb400640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b4d0 0x7fcce006d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcd040b0fc0 tx=0x7fccec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:39.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.710+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcd040a8c90 con 0x7fcd0406ea90 2024-04-25T10:12:39.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:39.946+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fcd04073a00 con 0x7fcd0406ea90 2024-04-25T10:12:40.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.459+0000 7fcd00c00640 1 -- 172.21.15.165:0/3853828327 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v74) v1 ==== 122+0+0 (secure 0 0 0) 0x7fccf4058420 con 0x7fcd0406ea90 2024-04-25T10:12:40.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.462+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b4d0 msgr2=0x7fcce006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.462+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b4d0 0x7fcce006d990 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fcd040b0fc0 tx=0x7fccec008040 comp rx=0 tx=0).stop 2024-04-25T10:12:40.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.462+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 msgr2=0x7fcd041fa980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.462+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 0x7fcd041fa980 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fccf400c390 tx=0x7fccf400c850 comp rx=0 tx=0).stop 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 shutdown_connections 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcce006b4d0 0x7fcce006d990 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcd04161fc0 0x7fcd041eb790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcd0406ef10 0x7fcd041faec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 --2- 172.21.15.165:0/3853828327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcd0406ea90 0x7fcd041fa980 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 >> 172.21.15.165:0/3853828327 conn(0x7fcd0406abe0 msgr2=0x7fcd040af740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 shutdown_connections 2024-04-25T10:12:40.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.463+0000 7fcd0aa00640 1 -- 172.21.15.165:0/3853828327 wait complete. 2024-04-25T10:12:40.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:12:40.488 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:40.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.683+0000 7fa23b200640 1 Processor -- start 2024-04-25T10:12:40.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.683+0000 7fa23b200640 1 -- start start 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa23b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa23c15e520 0x7fa23c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa23b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c15f040 0x7fa23c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa23b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa23c0ae0d0 0x7fa23c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa23b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23c0aee30 con 0x7fa23c15e520 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa23b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa23c0ae590 con 0x7fa23c0ae0d0 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa23b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c0ae870 con 0x7fa23c15f040 2024-04-25T10:12:40.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa239e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa23c15e520 0x7fa23c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa239e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa23c15e520 0x7fa23c15e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55628/0 (socket says 172.21.15.165:55628) 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.685+0000 7fa239e00640 1 -- 172.21.15.165:0/1875213494 learned_addr learned my addr 172.21.15.165:0/1875213494 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.686+0000 7fa239e00640 1 -- 172.21.15.165:0/1875213494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c15f040 msgr2=0x7fa23c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.686+0000 7fa239e00640 1 --2- 172.21.15.165:0/1875213494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c15f040 0x7fa23c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.686+0000 7fa239e00640 1 -- 172.21.15.165:0/1875213494 >> v2:172.21.15.172:3300/0 conn(0x7fa23c0ae0d0 msgr2=0x7fa23c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.686+0000 7fa239e00640 1 --2- 172.21.15.165:0/1875213494 >> v2:172.21.15.172:3300/0 conn(0x7fa23c0ae0d0 0x7fa23c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.686+0000 7fa239e00640 1 -- 172.21.15.165:0/1875213494 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0aeb50 con 0x7fa23c15e520 2024-04-25T10:12:40.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.687+0000 7fa239e00640 1 --2- 172.21.15.165:0/1875213494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15e520 0x7fa23c15e900 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0cf020 tx=0x7fa23402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdf4988e12c8db53 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.687+0000 7fa238a00640 1 -- 172.21.15.165:0/1875213494 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234037070 con 0x7fa23c15e520 2024-04-25T10:12:40.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.687+0000 7fa238a00640 1 -- 172.21.15.165:0/1875213494 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa234032440 con 0x7fa23c15e520 2024-04-25T10:12:40.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.687+0000 7fa238a00640 1 -- 172.21.15.165:0/1875213494 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa234032810 con 0x7fa23c15e520 2024-04-25T10:12:40.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 -- 172.21.15.165:0/1875213494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15e520 msgr2=0x7fa23c15e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 --2- 172.21.15.165:0/1875213494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15e520 0x7fa23c15e900 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fa23c0cf020 tx=0x7fa23402a1d0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 -- 172.21.15.165:0/1875213494 shutdown_connections 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 --2- 172.21.15.165:0/1875213494 >> v2:172.21.15.172:3300/0 conn(0x7fa23c0ae0d0 0x7fa23c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 --2- 172.21.15.165:0/1875213494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c15f040 0x7fa23c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 --2- 172.21.15.165:0/1875213494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15e520 0x7fa23c15e900 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 -- 172.21.15.165:0/1875213494 >> 172.21.15.165:0/1875213494 conn(0x7fa23c06abe0 msgr2=0x7fa23c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 -- 172.21.15.165:0/1875213494 shutdown_connections 2024-04-25T10:12:40.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.688+0000 7fa23b200640 1 -- 172.21.15.165:0/1875213494 wait complete. 2024-04-25T10:12:40.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.690+0000 7fa23b200640 1 Processor -- start 2024-04-25T10:12:40.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.690+0000 7fa23b200640 1 -- start start 2024-04-25T10:12:40.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c0ae0d0 0x7fa23c0ce4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa23c15e520 0x7fa23c1563b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 0x7fa23c15cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c15f620 con 0x7fa23c15f040 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c15f4a0 con 0x7fa23c15e520 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c071530 con 0x7fa23c0ae0d0 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 0x7fa23c15cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 0x7fa23c15cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55638/0 (socket says 172.21.15.165:55638) 2024-04-25T10:12:40.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.691+0000 7fa23a800640 1 -- 172.21.15.165:0/2113691221 learned_addr learned my addr 172.21.15.165:0/2113691221 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa239e00640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c0ae0d0 0x7fa23c0ce4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23a800640 1 -- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c0ae0d0 msgr2=0x7fa23c0ce4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23a800640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c0ae0d0 0x7fa23c0ce4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23a800640 1 -- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa23c15e520 msgr2=0x7fa23c1563b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23a800640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa23c15e520 0x7fa23c1563b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23a800640 1 -- 172.21.15.165:0/2113691221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c0b14c0 con 0x7fa23c15f040 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23a800640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 0x7fa23c15cb30 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00b9a0 tx=0x7fa22c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c019070 con 0x7fa23c15f040 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22c00cbc0 con 0x7fa23c15f040 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22c014440 con 0x7fa23c15f040 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa23c1568f0 con 0x7fa23c15f040 2024-04-25T10:12:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.692+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c0cca00 con 0x7fa23c15f040 2024-04-25T10:12:40.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.694+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa23c073990 con 0x7fa23c15f040 2024-04-25T10:12:40.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.695+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa22c01c070 con 0x7fa23c15f040 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.696+0000 7fa232000640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa21806b4d0 0x7fa21806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.696+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa22c08a1b0 con 0x7fa23c15f040 2024-04-25T10:12:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.696+0000 7fa239e00640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa21806b4d0 0x7fa21806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:40.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.697+0000 7fa239e00640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa21806b4d0 0x7fa21806d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa234034000 tx=0x7fa23402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.700+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa22c058e00 con 0x7fa23c15f040 2024-04-25T10:12:40.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.935+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa23c160500 con 0x7fa23c15f040 2024-04-25T10:12:40.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.936+0000 7fa232000640 1 -- 172.21.15.165:0/2113691221 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fa22c058700 con 0x7fa23c15f040 2024-04-25T10:12:40.941 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:40.941 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:40.942 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa21806b4d0 msgr2=0x7fa21806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa21806b4d0 0x7fa21806d990 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fa234034000 tx=0x7fa23402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 msgr2=0x7fa23c15cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 0x7fa23c15cb30 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7fa22c00b9a0 tx=0x7fa22c00be60 comp rx=0 tx=0).stop 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 shutdown_connections 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa21806b4d0 0x7fa21806d990 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa23c15f040 0x7fa23c15cb30 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa23c15e520 0x7fa23c1563b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 --2- 172.21.15.165:0/2113691221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa23c0ae0d0 0x7fa23c0ce4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 >> 172.21.15.165:0/2113691221 conn(0x7fa23c06abe0 msgr2=0x7fa23c15f830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 shutdown_connections 2024-04-25T10:12:40.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:40.939+0000 7fa23b200640 1 -- 172.21.15.165:0/2113691221 wait complete. 2024-04-25T10:12:40.963 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:12:41.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.158+0000 7fd82e600640 1 Processor -- start 2024-04-25T10:12:41.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.158+0000 7fd82e600640 1 -- start start 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd82e600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8280ce920 0x7fd8280cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd82e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 0x7fd82806d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd82e600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd828144b30 con 0x7fd828144750 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd82e600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8280712d0 con 0x7fd8280ce920 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd82e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280715b0 con 0x7fd82806d290 2024-04-25T10:12:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd827400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd828144750 0x7fd82806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:38188/0 (socket says 172.21.15.165:38188) 2024-04-25T10:12:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd827400640 1 -- 172.21.15.165:0/4138186254 learned_addr learned my addr 172.21.15.165:0/4138186254 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd826a00640 1 --2- 172.21.15.165:0/4138186254 >> v2:172.21.15.172:3300/0 conn(0x7fd8280ce920 0x7fd8280cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd827e00640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 0x7fd82806d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.160+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1504498512 0 0) 0x7fd828144b30 con 0x7fd828144750 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd80c003520 con 0x7fd828144750 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4123967964 0 0) 0x7fd80c003520 con 0x7fd828144750 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 msgr2=0x7fd82806d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 0x7fd82806d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 >> v2:172.21.15.172:3300/0 conn(0x7fd8280ce920 msgr2=0x7fd8280cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 --2- 172.21.15.165:0/4138186254 >> v2:172.21.15.172:3300/0 conn(0x7fd8280ce920 0x7fd8280cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd828071890 con 0x7fd828144750 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd818001670 con 0x7fd828144750 2024-04-25T10:12:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 >> v1:172.21.15.165:6789/0 conn(0x7fd828144750 legacy=0x7fd82806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 0x7fd80c0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c003f80 0x7fd80c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c004d40 0x7fd80c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280712d0 con 0x7fd80c003f80 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280715b0 con 0x7fd80c004d40 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.161+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80c00b9c0 con 0x7fd82806d290 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd826a00640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c003f80 0x7fd80c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd827400640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 0x7fd80c0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd827e00640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c004d40 0x7fd80c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd826a00640 1 -- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 msgr2=0x7fd80c0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd826a00640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd82806d290 0x7fd80c0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd826a00640 1 -- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c004d40 msgr2=0x7fd80c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd826a00640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c004d40 0x7fd80c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.162+0000 7fd826a00640 1 -- 172.21.15.165:0/4138186254 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd80c003520 con 0x7fd80c003f80 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.163+0000 7fd826a00640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c003f80 0x7fd80c004800 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd814009af0 tx=0x7fd80c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652e87fc0f0bb7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814036070 con 0x7fd80c003f80 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814031400 con 0x7fd80c003f80 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd826000640 1 -- 172.21.15.165:0/4138186254 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814009690 con 0x7fd80c003f80 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd82e600640 1 -- 172.21.15.165:0/4138186254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c003f80 msgr2=0x7fd80c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd82e600640 1 --2- 172.21.15.165:0/4138186254 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd80c003f80 0x7fd80c004800 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fd814009af0 tx=0x7fd80c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd827e00640 1 -- 172.21.15.165:0/4138186254 reap_dead start 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd82e600640 1 -- 172.21.15.165:0/4138186254 shutdown_connections 2024-04-25T10:12:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd82e600640 1 -- 172.21.15.165:0/4138186254 >> 172.21.15.165:0/4138186254 conn(0x7fd82806abe0 msgr2=0x7fd82806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd82e600640 1 -- 172.21.15.165:0/4138186254 shutdown_connections 2024-04-25T10:12:41.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.164+0000 7fd82e600640 1 -- 172.21.15.165:0/4138186254 wait complete. 2024-04-25T10:12:41.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.166+0000 7fd82e600640 1 Processor -- start 2024-04-25T10:12:41.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.166+0000 7fd82e600640 1 -- start start 2024-04-25T10:12:41.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.167+0000 7fd82e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c003f80 0x7fd828141550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.167+0000 7fd82e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 0x7fd8281421b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.167+0000 7fd82e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8281426f0 0x7fd8280b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.167+0000 7fd82e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82806f020 con 0x7fd82815b310 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.167+0000 7fd82e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd82806ece0 con 0x7fd80c003f80 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.167+0000 7fd82e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd828145450 con 0x7fd8281426f0 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 0x7fd8281421b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 0x7fd8281421b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55648/0 (socket says 172.21.15.165:55648) 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 -- 172.21.15.165:0/3724394467 learned_addr learned my addr 172.21.15.165:0/3724394467 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd827400640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c003f80 0x7fd828141550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 -- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8281426f0 msgr2=0x7fd8280b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8281426f0 0x7fd8280b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 -- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c003f80 msgr2=0x7fd828141550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c003f80 0x7fd828141550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 -- 172.21.15.165:0/3724394467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8280ad630 con 0x7fd82815b310 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.168+0000 7fd826a00640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 0x7fd8281421b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd814009c10 tx=0x7fd814004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.169+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814040020 con 0x7fd82815b310 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.169+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd814003810 con 0x7fd82815b310 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.169+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8280ad910 con 0x7fd82815b310 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.169+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd814036040 con 0x7fd82815b310 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.169+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8280aded0 con 0x7fd82815b310 2024-04-25T10:12:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.170+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd828142b10 con 0x7fd82815b310 2024-04-25T10:12:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.171+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd814043020 con 0x7fd82815b310 2024-04-25T10:12:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.172+0000 7fd81f400640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd80806b500 0x7fd80806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.172+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8140aa690 con 0x7fd82815b310 2024-04-25T10:12:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.172+0000 7fd827400640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd80806b500 0x7fd80806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.173+0000 7fd827400640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd80806b500 0x7fd80806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd818001850 tx=0x7fd818000fd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.176+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd81404a030 con 0x7fd82815b310 2024-04-25T10:12:41.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.370+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd8280ce220 con 0x7fd82815b310 2024-04-25T10:12:41.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.373+0000 7fd81f400640 1 -- 172.21.15.165:0/3724394467 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+16920 (secure 0 0 0) 0x7fd8280ce220 con 0x7fd82815b310 2024-04-25T10:12:41.378 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:41.378 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":111,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:38.406090+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd80806b500 msgr2=0x7fd80806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd80806b500 0x7fd80806d9c0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fd818001850 tx=0x7fd818000fd0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 msgr2=0x7fd8281421b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 0x7fd8281421b0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fd814009c10 tx=0x7fd814004730 comp rx=0 tx=0).stop 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 shutdown_connections 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd80806b500 0x7fd80806d9c0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8281426f0 0x7fd8280b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd82815b310 0x7fd8281421b0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 --2- 172.21.15.165:0/3724394467 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd80c003f80 0x7fd828141550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 >> 172.21.15.165:0/3724394467 conn(0x7fd82806abe0 msgr2=0x7fd82806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 shutdown_connections 2024-04-25T10:12:41.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.376+0000 7fd82e600640 1 -- 172.21.15.165:0/3724394467 wait complete. 2024-04-25T10:12:41.398 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:12:41.398 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.579+0000 7efdd5a00640 1 Processor -- start 2024-04-25T10:12:41.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.580+0000 7efdd5a00640 1 -- start start 2024-04-25T10:12:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdd5a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efdd00d2e90 0x7efdd00d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdd5a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00b9940 con 0x7efdd015dd30 2024-04-25T10:12:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdd5a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdd00d1df0 con 0x7efdd00d0900 2024-04-25T10:12:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdd5a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdd00d0d40 con 0x7efdd00d2e90 2024-04-25T10:12:41.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdce000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efdd00d2e90 0x7efdd00d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdce000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efdd00d2e90 0x7efdd00d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55664/0 (socket says 172.21.15.165:55664) 2024-04-25T10:12:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.581+0000 7efdce000640 1 -- 172.21.15.165:0/1031209370 learned_addr learned my addr 172.21.15.165:0/1031209370 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.582+0000 7efdce000640 1 -- 172.21.15.165:0/1031209370 >> v1:172.21.15.172:6789/0 conn(0x7efdd00d0900 legacy=0x7efdd00b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.582+0000 7efdce000640 1 -- 172.21.15.165:0/1031209370 >> v1:172.21.15.165:6789/0 conn(0x7efdd015dd30 legacy=0x7efdd00b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.582+0000 7efdce000640 1 -- 172.21.15.165:0/1031209370 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd00d20d0 con 0x7efdd00d2e90 2024-04-25T10:12:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.582+0000 7efdcd600640 1 -- 172.21.15.165:0/1031209370 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2918890264 0 0) 0x7efdd00b9940 con 0x7efdd015dd30 2024-04-25T10:12:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.583+0000 7efdce000640 1 --2- 172.21.15.165:0/1031209370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd00d03c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009850 tx=0x7efdb802a160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe298feb3cacf36b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.583+0000 7efdcd600640 1 -- 172.21.15.165:0/1031209370 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8004030 con 0x7efdd00d2e90 2024-04-25T10:12:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.583+0000 7efdcd600640 1 -- 172.21.15.165:0/1031209370 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb80041d0 con 0x7efdd00d2e90 2024-04-25T10:12:41.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.583+0000 7efdcd600640 1 -- 172.21.15.165:0/1031209370 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8003610 con 0x7efdd00d2e90 2024-04-25T10:12:41.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.583+0000 7efdd5a00640 1 -- 172.21.15.165:0/1031209370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 msgr2=0x7efdd00d03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.583+0000 7efdd5a00640 1 --2- 172.21.15.165:0/1031209370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd00d03c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efdb8009850 tx=0x7efdb802a160 comp rx=0 tx=0).stop 2024-04-25T10:12:41.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.584+0000 7efdd5a00640 1 -- 172.21.15.165:0/1031209370 shutdown_connections 2024-04-25T10:12:41.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.584+0000 7efdd5a00640 1 --2- 172.21.15.165:0/1031209370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd00d03c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.584+0000 7efdd5a00640 1 -- 172.21.15.165:0/1031209370 >> 172.21.15.165:0/1031209370 conn(0x7efdd0061c30 msgr2=0x7efdd006c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.584+0000 7efdd5a00640 1 -- 172.21.15.165:0/1031209370 shutdown_connections 2024-04-25T10:12:41.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.584+0000 7efdd5a00640 1 -- 172.21.15.165:0/1031209370 wait complete. 2024-04-25T10:12:41.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.585+0000 7efdd5a00640 1 Processor -- start 2024-04-25T10:12:41.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.585+0000 7efdd5a00640 1 -- start start 2024-04-25T10:12:41.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.586+0000 7efdd5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd00d0900 0x7efdd00b5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.586+0000 7efdd5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd0154720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.586+0000 7efdd5a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd015dd30 0x7efdd0149f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.586+0000 7efdd5a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00d2970 con 0x7efdd00d2e90 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.586+0000 7efdd5a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00d2690 con 0x7efdd015dd30 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.586+0000 7efdd5a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd00d1a10 con 0x7efdd00d0900 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd0154720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd0154720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55680/0 (socket says 172.21.15.165:55680) 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 -- 172.21.15.165:0/3727506125 learned_addr learned my addr 172.21.15.165:0/3727506125 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 -- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd00d0900 msgr2=0x7efdd00b5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdcea00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd00d0900 0x7efdd00b5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdcf400640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd015dd30 0x7efdd0149f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd00d0900 0x7efdd00b5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 -- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd015dd30 msgr2=0x7efdd0149f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd015dd30 0x7efdd0149f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdce000640 1 -- 172.21.15.165:0/3727506125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0154ec0 con 0x7efdd00d2e90 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdcea00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd00d0900 0x7efdd00b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:41.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.587+0000 7efdcf400640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd015dd30 0x7efdd0149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.588+0000 7efdce000640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd0154720 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efdb80090b0 tx=0x7efdb80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.588+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8041020 con 0x7efdd00d2e90 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.588+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb8003c50 con 0x7efdd00d2e90 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.588+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb8006040 con 0x7efdd00d2e90 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.588+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdd01551a0 con 0x7efdd00d2e90 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.588+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdd0155760 con 0x7efdd00d2e90 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.589+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdd0073990 con 0x7efdd00d2e90 2024-04-25T10:12:41.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.590+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdb8044020 con 0x7efdd00d2e90 2024-04-25T10:12:41.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.591+0000 7efdc6a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdac06b4d0 0x7efdac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:41.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.591+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efdb80ab170 con 0x7efdd00d2e90 2024-04-25T10:12:41.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.591+0000 7efdcea00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdac06b4d0 0x7efdac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:41.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.592+0000 7efdcea00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdac06b4d0 0x7efdac06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efdc0000f60 tx=0x7efdc0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.596+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdb804b030 con 0x7efdd00d2e90 2024-04-25T10:12:41.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.783+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdd0154c60 con 0x7efdac06b4d0 2024-04-25T10:12:41.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.799+0000 7efdc6a00640 1 -- 172.21.15.165:0/3727506125 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7efdc00013d0 con 0x7efdac06b4d0 2024-04-25T10:12:41.808 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:41.809 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdac06b4d0 msgr2=0x7efdac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdac06b4d0 0x7efdac06d990 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7efdc0000f60 tx=0x7efdc0009040 comp rx=0 tx=0).stop 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 msgr2=0x7efdd0154720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd0154720 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efdb80090b0 tx=0x7efdb80046c0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 shutdown_connections 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdac06b4d0 0x7efdac06d990 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efdd015dd30 0x7efdd0149f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efdd00d2e90 0x7efdd0154720 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 --2- 172.21.15.165:0/3727506125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efdd00d0900 0x7efdd00b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.805+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 >> 172.21.15.165:0/3727506125 conn(0x7efdd0061c30 msgr2=0x7efdd01603c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.806+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 shutdown_connections 2024-04-25T10:12:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:41.806+0000 7efdd5a00640 1 -- 172.21.15.165:0/3727506125 wait complete. 2024-04-25T10:12:41.827 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-04-25T10:12:40.344912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":426344,"kb_used_data":11708,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754548376,"statfs":{"total":773094113280,"available":772657537024,"internally_reserved":0,"allocated":11988992,"data_stored":6212053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":424463922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.955302"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.433601+0000","last_change":"2024-04-25T10:12:36.433601+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.433601+0000","last_undegraded":"2024-04-25T10:12:36.433601+0000","last_fullsized":"2024-04-25T10:12:36.433601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.420389+0000","last_change":"2024-04-25T10:12:36.420389+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.420389+0000","last_undegraded":"2024-04-25T10:12:36.420389+0000","last_fullsized":"2024-04-25T10:12:36.420389+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.436046+0000","last_change":"2024-04-25T10:12:36.436046+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.436046+0000","last_undegraded":"2024-04-25T10:12:36.436046+0000","last_fullsized":"2024-04-25T10:12:36.436046+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.431900+0000","last_change":"2024-04-25T10:12:36.431900+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.431900+0000","last_undegraded":"2024-04-25T10:12:36.431900+0000","last_fullsized":"2024-04-25T10:12:36.431900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426448+0000","last_change":"2024-04-25T10:12:36.426448+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426448+0000","last_undegraded":"2024-04-25T10:12:36.426448+0000","last_fullsized":"2024-04-25T10:12:36.426448+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.412404+0000","last_change":"2024-04-25T10:12:36.412404+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.412404+0000","last_undegraded":"2024-04-25T10:12:36.412404+0000","last_fullsized":"2024-04-25T10:12:36.412404+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.413363+0000","last_change":"2024-04-25T10:12:36.413363+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.413363+0000","last_undegraded":"2024-04-25T10:12:36.413363+0000","last_fullsized":"2024-04-25T10:12:36.413363+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426612+0000","last_change":"2024-04-25T10:12:36.426612+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426612+0000","last_undegraded":"2024-04-25T10:12:36.426612+0000","last_fullsized":"2024-04-25T10:12:36.426612+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.414396+0000","last_change":"2024-04-25T10:12:36.414396+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.414396+0000","last_undegraded":"2024-04-25T10:12:36.414396+0000","last_fullsized":"2024-04-25T10:12:36.414396+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.427360+0000","last_change":"2024-04-25T10:12:36.427360+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.427360+0000","last_undegraded":"2024-04-25T10:12:36.427360+0000","last_fullsized":"2024-04-25T10:12:36.427360+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428859+0000","last_change":"2024-04-25T10:12:36.428859+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428859+0000","last_undegraded":"2024-04-25T10:12:36.428859+0000","last_fullsized":"2024-04-25T10:12:36.428859+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.417088+0000","last_change":"2024-04-25T10:12:36.417088+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.417088+0000","last_undegraded":"2024-04-25T10:12:36.417088+0000","last_fullsized":"2024-04-25T10:12:36.417088+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.430707+0000","last_change":"2024-04-25T10:12:36.430707+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.430707+0000","last_undegraded":"2024-04-25T10:12:36.430707+0000","last_fullsized":"2024-04-25T10:12:36.430707+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425443+0000","last_change":"2024-04-25T10:12:36.425443+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425443+0000","last_undegraded":"2024-04-25T10:12:36.425443+0000","last_fullsized":"2024-04-25T10:12:36.425443+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423319+0000","last_change":"2024-04-25T10:12:36.423319+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423319+0000","last_undegraded":"2024-04-25T10:12:36.423319+0000","last_fullsized":"2024-04-25T10:12:36.423319+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422605+0000","last_change":"2024-04-25T10:12:36.422605+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422605+0000","last_undegraded":"2024-04-25T10:12:36.422605+0000","last_fullsized":"2024-04-25T10:12:36.422605+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426662+0000","last_change":"2024-04-25T10:12:36.426662+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426662+0000","last_undegraded":"2024-04-25T10:12:36.426662+0000","last_fullsized":"2024-04-25T10:12:36.426662+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.417057+0000","last_change":"2024-04-25T10:12:36.417057+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.417057+0000","last_undegraded":"2024-04-25T10:12:36.417057+0000","last_fullsized":"2024-04-25T10:12:36.417057+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.434665+0000","last_change":"2024-04-25T10:12:36.434665+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.434665+0000","last_undegraded":"2024-04-25T10:12:36.434665+0000","last_fullsized":"2024-04-25T10:12:36.434665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.416049+0000","last_change":"2024-04-25T10:12:36.416049+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.416049+0000","last_undegraded":"2024-04-25T10:12:36.416049+0000","last_fullsized":"2024-04-25T10:12:36.416049+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.427090+0000","last_change":"2024-04-25T10:12:36.427090+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.427090+0000","last_undegraded":"2024-04-25T10:12:36.427090+0000","last_fullsized":"2024-04-25T10:12:36.427090+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425039+0000","last_change":"2024-04-25T10:12:36.425039+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425039+0000","last_undegraded":"2024-04-25T10:12:36.425039+0000","last_fullsized":"2024-04-25T10:12:36.425039+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428265+0000","last_change":"2024-04-25T10:12:36.428265+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428265+0000","last_undegraded":"2024-04-25T10:12:36.428265+0000","last_fullsized":"2024-04-25T10:12:36.428265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.424969+0000","last_change":"2024-04-25T10:12:36.424969+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.424969+0000","last_undegraded":"2024-04-25T10:12:36.424969+0000","last_fullsized":"2024-04-25T10:12:36.424969+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.415694+0000","last_change":"2024-04-25T10:12:36.415694+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.415694+0000","last_undegraded":"2024-04-25T10:12:36.415694+0000","last_fullsized":"2024-04-25T10:12:36.415694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.414546+0000","last_change":"2024-04-25T10:12:36.414546+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.414546+0000","last_undegraded":"2024-04-25T10:12:36.414546+0000","last_fullsized":"2024-04-25T10:12:36.414546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428426+0000","last_change":"2024-04-25T10:12:36.428426+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428426+0000","last_undegraded":"2024-04-25T10:12:36.428426+0000","last_fullsized":"2024-04-25T10:12:36.428426+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.411138+0000","last_change":"2024-04-25T10:12:36.411138+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.411138+0000","last_undegraded":"2024-04-25T10:12:36.411138+0000","last_fullsized":"2024-04-25T10:12:36.411138+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.424024+0000","last_change":"2024-04-25T10:12:36.424024+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.424024+0000","last_undegraded":"2024-04-25T10:12:36.424024+0000","last_fullsized":"2024-04-25T10:12:36.424024+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.413939+0000","last_change":"2024-04-25T10:12:36.413939+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.413939+0000","last_undegraded":"2024-04-25T10:12:36.413939+0000","last_fullsized":"2024-04-25T10:12:36.413939+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.412188+0000","last_change":"2024-04-25T10:12:36.412188+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.412188+0000","last_undegraded":"2024-04-25T10:12:36.412188+0000","last_fullsized":"2024-04-25T10:12:36.412188+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426053+0000","last_change":"2024-04-25T10:12:36.426053+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426053+0000","last_undegraded":"2024-04-25T10:12:36.426053+0000","last_fullsized":"2024-04-25T10:12:36.426053+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423148+0000","last_change":"2024-04-25T10:12:36.423148+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423148+0000","last_undegraded":"2024-04-25T10:12:36.423148+0000","last_fullsized":"2024-04-25T10:12:36.423148+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433093+0000","last_change":"2024-04-25T10:12:36.428651+0000","last_active":"2024-04-25T10:12:36.433093+0000","last_peered":"2024-04-25T10:12:36.433093+0000","last_clean":"2024-04-25T10:12:36.433093+0000","last_became_active":"2024-04-25T10:12:36.428062+0000","last_became_peered":"2024-04-25T10:12:36.428062+0000","last_unstale":"2024-04-25T10:12:36.433093+0000","last_undegraded":"2024-04-25T10:12:36.433093+0000","last_fullsized":"2024-04-25T10:12:36.433093+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:38.055414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422457+0000","last_change":"2024-04-25T10:12:36.422457+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422457+0000","last_undegraded":"2024-04-25T10:12:36.422457+0000","last_fullsized":"2024-04-25T10:12:36.422457+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.435053+0000","last_change":"2024-04-25T10:12:36.432356+0000","last_active":"2024-04-25T10:12:36.435053+0000","last_peered":"2024-04-25T10:12:36.435053+0000","last_clean":"2024-04-25T10:12:36.435053+0000","last_became_active":"2024-04-25T10:12:36.432086+0000","last_became_peered":"2024-04-25T10:12:36.432086+0000","last_unstale":"2024-04-25T10:12:36.435053+0000","last_undegraded":"2024-04-25T10:12:36.435053+0000","last_fullsized":"2024-04-25T10:12:36.435053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:54.943354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.411887+0000","last_change":"2024-04-25T10:12:36.411887+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.411887+0000","last_undegraded":"2024-04-25T10:12:36.411887+0000","last_fullsized":"2024-04-25T10:12:36.411887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438917+0000","last_change":"2024-04-25T10:12:36.437597+0000","last_active":"2024-04-25T10:12:36.438917+0000","last_peered":"2024-04-25T10:12:36.438917+0000","last_clean":"2024-04-25T10:12:36.438917+0000","last_became_active":"2024-04-25T10:12:36.437214+0000","last_became_peered":"2024-04-25T10:12:36.437214+0000","last_unstale":"2024-04-25T10:12:36.438917+0000","last_undegraded":"2024-04-25T10:12:36.438917+0000","last_fullsized":"2024-04-25T10:12:36.438917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.420749+0000","last_change":"2024-04-25T10:12:36.420749+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.420749+0000","last_undegraded":"2024-04-25T10:12:36.420749+0000","last_fullsized":"2024-04-25T10:12:36.420749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.434471+0000","last_change":"2024-04-25T10:12:36.430378+0000","last_active":"2024-04-25T10:12:36.434471+0000","last_peered":"2024-04-25T10:12:36.434471+0000","last_clean":"2024-04-25T10:12:36.434471+0000","last_became_active":"2024-04-25T10:12:36.430043+0000","last_became_peered":"2024-04-25T10:12:36.430043+0000","last_unstale":"2024-04-25T10:12:36.434471+0000","last_undegraded":"2024-04-25T10:12:36.434471+0000","last_fullsized":"2024-04-25T10:12:36.434471+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428765+0000","last_change":"2024-04-25T10:12:36.428765+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428765+0000","last_undegraded":"2024-04-25T10:12:36.428765+0000","last_fullsized":"2024-04-25T10:12:36.428765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.435090+0000","last_change":"2024-04-25T10:12:36.432661+0000","last_active":"2024-04-25T10:12:36.435090+0000","last_peered":"2024-04-25T10:12:36.435090+0000","last_clean":"2024-04-25T10:12:36.435090+0000","last_became_active":"2024-04-25T10:12:36.432286+0000","last_became_peered":"2024-04-25T10:12:36.432286+0000","last_unstale":"2024-04-25T10:12:36.435090+0000","last_undegraded":"2024-04-25T10:12:36.435090+0000","last_fullsized":"2024-04-25T10:12:36.435090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:17.416609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423407+0000","last_change":"2024-04-25T10:12:36.423407+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423407+0000","last_undegraded":"2024-04-25T10:12:36.423407+0000","last_fullsized":"2024-04-25T10:12:36.423407+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.440008+0000","last_change":"2024-04-25T10:12:36.437498+0000","last_active":"2024-04-25T10:12:36.440008+0000","last_peered":"2024-04-25T10:12:36.440008+0000","last_clean":"2024-04-25T10:12:36.440008+0000","last_became_active":"2024-04-25T10:12:36.437164+0000","last_became_peered":"2024-04-25T10:12:36.437164+0000","last_unstale":"2024-04-25T10:12:36.440008+0000","last_undegraded":"2024-04-25T10:12:36.440008+0000","last_fullsized":"2024-04-25T10:12:36.440008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.416272+0000","last_change":"2024-04-25T10:12:36.416272+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.416272+0000","last_undegraded":"2024-04-25T10:12:36.416272+0000","last_fullsized":"2024-04-25T10:12:36.416272+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438102+0000","last_change":"2024-04-25T10:12:36.436140+0000","last_active":"2024-04-25T10:12:36.438102+0000","last_peered":"2024-04-25T10:12:36.438102+0000","last_clean":"2024-04-25T10:12:36.438102+0000","last_became_active":"2024-04-25T10:12:36.435843+0000","last_became_peered":"2024-04-25T10:12:36.435843+0000","last_unstale":"2024-04-25T10:12:36.438102+0000","last_undegraded":"2024-04-25T10:12:36.438102+0000","last_fullsized":"2024-04-25T10:12:36.438102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.986763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.429636+0000","last_change":"2024-04-25T10:12:36.429636+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.429636+0000","last_undegraded":"2024-04-25T10:12:36.429636+0000","last_fullsized":"2024-04-25T10:12:36.429636+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.435442+0000","last_change":"2024-04-25T10:12:36.433709+0000","last_active":"2024-04-25T10:12:36.435442+0000","last_peered":"2024-04-25T10:12:36.435442+0000","last_clean":"2024-04-25T10:12:36.435442+0000","last_became_active":"2024-04-25T10:12:36.433440+0000","last_became_peered":"2024-04-25T10:12:36.433440+0000","last_unstale":"2024-04-25T10:12:36.435442+0000","last_undegraded":"2024-04-25T10:12:36.435442+0000","last_fullsized":"2024-04-25T10:12:36.435442+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.503044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.424245+0000","last_change":"2024-04-25T10:12:36.424245+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.424245+0000","last_undegraded":"2024-04-25T10:12:36.424245+0000","last_fullsized":"2024-04-25T10:12:36.424245+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433721+0000","last_change":"2024-04-25T10:12:36.430969+0000","last_active":"2024-04-25T10:12:36.433721+0000","last_peered":"2024-04-25T10:12:36.433721+0000","last_clean":"2024-04-25T10:12:36.433721+0000","last_became_active":"2024-04-25T10:12:36.430681+0000","last_became_peered":"2024-04-25T10:12:36.430681+0000","last_unstale":"2024-04-25T10:12:36.433721+0000","last_undegraded":"2024-04-25T10:12:36.433721+0000","last_fullsized":"2024-04-25T10:12:36.433721+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:25.848233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426056+0000","last_change":"2024-04-25T10:12:36.426056+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426056+0000","last_undegraded":"2024-04-25T10:12:36.426056+0000","last_fullsized":"2024-04-25T10:12:36.426056+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437029+0000","last_change":"2024-04-25T10:12:36.435089+0000","last_active":"2024-04-25T10:12:36.437029+0000","last_peered":"2024-04-25T10:12:36.437029+0000","last_clean":"2024-04-25T10:12:36.437029+0000","last_became_active":"2024-04-25T10:12:36.434759+0000","last_became_peered":"2024-04-25T10:12:36.434759+0000","last_unstale":"2024-04-25T10:12:36.437029+0000","last_undegraded":"2024-04-25T10:12:36.437029+0000","last_fullsized":"2024-04-25T10:12:36.437029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:12.593794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422641+0000","last_change":"2024-04-25T10:12:36.422641+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422641+0000","last_undegraded":"2024-04-25T10:12:36.422641+0000","last_fullsized":"2024-04-25T10:12:36.422641+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.440408+0000","last_change":"2024-04-25T10:12:36.438390+0000","last_active":"2024-04-25T10:12:36.440408+0000","last_peered":"2024-04-25T10:12:36.440408+0000","last_clean":"2024-04-25T10:12:36.440408+0000","last_became_active":"2024-04-25T10:12:36.438100+0000","last_became_peered":"2024-04-25T10:12:36.438100+0000","last_unstale":"2024-04-25T10:12:36.440408+0000","last_undegraded":"2024-04-25T10:12:36.440408+0000","last_fullsized":"2024-04-25T10:12:36.440408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:42.723368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425357+0000","last_change":"2024-04-25T10:12:36.425357+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425357+0000","last_undegraded":"2024-04-25T10:12:36.425357+0000","last_fullsized":"2024-04-25T10:12:36.425357+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437664+0000","last_change":"2024-04-25T10:12:36.434095+0000","last_active":"2024-04-25T10:12:36.437664+0000","last_peered":"2024-04-25T10:12:36.437664+0000","last_clean":"2024-04-25T10:12:36.437664+0000","last_became_active":"2024-04-25T10:12:36.433803+0000","last_became_peered":"2024-04-25T10:12:36.433803+0000","last_unstale":"2024-04-25T10:12:36.437664+0000","last_undegraded":"2024-04-25T10:12:36.437664+0000","last_fullsized":"2024-04-25T10:12:36.437664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:37.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.430525+0000","last_change":"2024-04-25T10:12:36.430525+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.430525+0000","last_undegraded":"2024-04-25T10:12:36.430525+0000","last_fullsized":"2024-04-25T10:12:36.430525+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.429379+0000","last_change":"2024-04-25T10:12:36.420615+0000","last_active":"2024-04-25T10:12:36.429379+0000","last_peered":"2024-04-25T10:12:36.429379+0000","last_clean":"2024-04-25T10:12:36.429379+0000","last_became_active":"2024-04-25T10:12:36.420089+0000","last_became_peered":"2024-04-25T10:12:36.420089+0000","last_unstale":"2024-04-25T10:12:36.429379+0000","last_undegraded":"2024-04-25T10:12:36.429379+0000","last_fullsized":"2024-04-25T10:12:36.429379+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.417054+0000","last_change":"2024-04-25T10:12:36.417054+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.417054+0000","last_undegraded":"2024-04-25T10:12:36.417054+0000","last_fullsized":"2024-04-25T10:12:36.417054+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433939+0000","last_change":"2024-04-25T10:12:36.432744+0000","last_active":"2024-04-25T10:12:36.433939+0000","last_peered":"2024-04-25T10:12:36.433939+0000","last_clean":"2024-04-25T10:12:36.433939+0000","last_became_active":"2024-04-25T10:12:36.431971+0000","last_became_peered":"2024-04-25T10:12:36.431971+0000","last_unstale":"2024-04-25T10:12:36.433939+0000","last_undegraded":"2024-04-25T10:12:36.433939+0000","last_fullsized":"2024-04-25T10:12:36.433939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:18.232182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422041+0000","last_change":"2024-04-25T10:12:36.422041+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422041+0000","last_undegraded":"2024-04-25T10:12:36.422041+0000","last_fullsized":"2024-04-25T10:12:36.422041+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433117+0000","last_change":"2024-04-25T10:12:36.429891+0000","last_active":"2024-04-25T10:12:36.433117+0000","last_peered":"2024-04-25T10:12:36.433117+0000","last_clean":"2024-04-25T10:12:36.433117+0000","last_became_active":"2024-04-25T10:12:36.429336+0000","last_became_peered":"2024-04-25T10:12:36.429336+0000","last_unstale":"2024-04-25T10:12:36.433117+0000","last_undegraded":"2024-04-25T10:12:36.433117+0000","last_fullsized":"2024-04-25T10:12:36.433117+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:54.329258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423259+0000","last_change":"2024-04-25T10:12:36.423259+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423259+0000","last_undegraded":"2024-04-25T10:12:36.423259+0000","last_fullsized":"2024-04-25T10:12:36.423259+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437982+0000","last_change":"2024-04-25T10:12:36.434858+0000","last_active":"2024-04-25T10:12:36.437982+0000","last_peered":"2024-04-25T10:12:36.437982+0000","last_clean":"2024-04-25T10:12:36.437982+0000","last_became_active":"2024-04-25T10:12:36.434571+0000","last_became_peered":"2024-04-25T10:12:36.434571+0000","last_unstale":"2024-04-25T10:12:36.437982+0000","last_undegraded":"2024-04-25T10:12:36.437982+0000","last_fullsized":"2024-04-25T10:12:36.437982+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425482+0000","last_change":"2024-04-25T10:12:36.425482+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425482+0000","last_undegraded":"2024-04-25T10:12:36.425482+0000","last_fullsized":"2024-04-25T10:12:36.425482+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438927+0000","last_change":"2024-04-25T10:12:36.437073+0000","last_active":"2024-04-25T10:12:36.438927+0000","last_peered":"2024-04-25T10:12:36.438927+0000","last_clean":"2024-04-25T10:12:36.438927+0000","last_became_active":"2024-04-25T10:12:36.436575+0000","last_became_peered":"2024-04-25T10:12:36.436575+0000","last_unstale":"2024-04-25T10:12:36.438927+0000","last_undegraded":"2024-04-25T10:12:36.438927+0000","last_fullsized":"2024-04-25T10:12:36.438927+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.411286+0000","last_change":"2024-04-25T10:12:36.411286+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.411286+0000","last_undegraded":"2024-04-25T10:12:36.411286+0000","last_fullsized":"2024-04-25T10:12:36.411286+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433434+0000","last_change":"2024-04-25T10:12:36.430895+0000","last_active":"2024-04-25T10:12:36.433434+0000","last_peered":"2024-04-25T10:12:36.433434+0000","last_clean":"2024-04-25T10:12:36.433434+0000","last_became_active":"2024-04-25T10:12:36.430565+0000","last_became_peered":"2024-04-25T10:12:36.430565+0000","last_unstale":"2024-04-25T10:12:36.433434+0000","last_undegraded":"2024-04-25T10:12:36.433434+0000","last_fullsized":"2024-04-25T10:12:36.433434+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.503044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425911+0000","last_change":"2024-04-25T10:12:36.425911+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425911+0000","last_undegraded":"2024-04-25T10:12:36.425911+0000","last_fullsized":"2024-04-25T10:12:36.425911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.439907+0000","last_change":"2024-04-25T10:12:36.438473+0000","last_active":"2024-04-25T10:12:36.439907+0000","last_peered":"2024-04-25T10:12:36.439907+0000","last_clean":"2024-04-25T10:12:36.439907+0000","last_became_active":"2024-04-25T10:12:36.436411+0000","last_became_peered":"2024-04-25T10:12:36.436411+0000","last_unstale":"2024-04-25T10:12:36.439907+0000","last_undegraded":"2024-04-25T10:12:36.439907+0000","last_fullsized":"2024-04-25T10:12:36.439907+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:22.288167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425846+0000","last_change":"2024-04-25T10:12:36.425846+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425846+0000","last_undegraded":"2024-04-25T10:12:36.425846+0000","last_fullsized":"2024-04-25T10:12:36.425846+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.430981+0000","last_change":"2024-04-25T10:12:36.428322+0000","last_active":"2024-04-25T10:12:36.430981+0000","last_peered":"2024-04-25T10:12:36.430981+0000","last_clean":"2024-04-25T10:12:36.430981+0000","last_became_active":"2024-04-25T10:12:36.427541+0000","last_became_peered":"2024-04-25T10:12:36.427541+0000","last_unstale":"2024-04-25T10:12:36.430981+0000","last_undegraded":"2024-04-25T10:12:36.430981+0000","last_fullsized":"2024-04-25T10:12:36.430981+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.884287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.429110+0000","last_change":"2024-04-25T10:12:36.429110+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.429110+0000","last_undegraded":"2024-04-25T10:12:36.429110+0000","last_fullsized":"2024-04-25T10:12:36.429110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.424174+0000","last_change":"2024-04-25T10:12:36.420411+0000","last_active":"2024-04-25T10:12:36.424174+0000","last_peered":"2024-04-25T10:12:36.424174+0000","last_clean":"2024-04-25T10:12:36.424174+0000","last_became_active":"2024-04-25T10:12:36.419903+0000","last_became_peered":"2024-04-25T10:12:36.419903+0000","last_unstale":"2024-04-25T10:12:36.424174+0000","last_undegraded":"2024-04-25T10:12:36.424174+0000","last_fullsized":"2024-04-25T10:12:36.424174+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:27.226759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.429648+0000","last_change":"2024-04-25T10:12:36.429648+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.429648+0000","last_undegraded":"2024-04-25T10:12:36.429648+0000","last_fullsized":"2024-04-25T10:12:36.429648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437624+0000","last_change":"2024-04-25T10:12:36.435352+0000","last_active":"2024-04-25T10:12:36.437624+0000","last_peered":"2024-04-25T10:12:36.437624+0000","last_clean":"2024-04-25T10:12:36.437624+0000","last_became_active":"2024-04-25T10:12:36.434943+0000","last_became_peered":"2024-04-25T10:12:36.434943+0000","last_unstale":"2024-04-25T10:12:36.437624+0000","last_undegraded":"2024-04-25T10:12:36.437624+0000","last_fullsized":"2024-04-25T10:12:36.437624+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:25.848233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422862+0000","last_change":"2024-04-25T10:12:36.422862+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422862+0000","last_undegraded":"2024-04-25T10:12:36.422862+0000","last_fullsized":"2024-04-25T10:12:36.422862+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.427121+0000","last_change":"2024-04-25T10:12:36.419478+0000","last_active":"2024-04-25T10:12:36.427121+0000","last_peered":"2024-04-25T10:12:36.427121+0000","last_clean":"2024-04-25T10:12:36.427121+0000","last_became_active":"2024-04-25T10:12:36.419010+0000","last_became_peered":"2024-04-25T10:12:36.419010+0000","last_unstale":"2024-04-25T10:12:36.427121+0000","last_undegraded":"2024-04-25T10:12:36.427121+0000","last_fullsized":"2024-04-25T10:12:36.427121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:32.357148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426726+0000","last_change":"2024-04-25T10:12:36.426726+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426726+0000","last_undegraded":"2024-04-25T10:12:36.426726+0000","last_fullsized":"2024-04-25T10:12:36.426726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.434660+0000","last_change":"2024-04-25T10:12:36.424429+0000","last_active":"2024-04-25T10:12:36.434660+0000","last_peered":"2024-04-25T10:12:36.434660+0000","last_clean":"2024-04-25T10:12:36.434660+0000","last_became_active":"2024-04-25T10:12:36.423878+0000","last_became_peered":"2024-04-25T10:12:36.423878+0000","last_unstale":"2024-04-25T10:12:36.434660+0000","last_undegraded":"2024-04-25T10:12:36.434660+0000","last_fullsized":"2024-04-25T10:12:36.434660+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.413518+0000","last_change":"2024-04-25T10:12:36.413518+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.413518+0000","last_undegraded":"2024-04-25T10:12:36.413518+0000","last_fullsized":"2024-04-25T10:12:36.413518+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.441320+0000","last_change":"2024-04-25T10:12:36.440745+0000","last_active":"2024-04-25T10:12:36.441320+0000","last_peered":"2024-04-25T10:12:36.441320+0000","last_clean":"2024-04-25T10:12:36.441320+0000","last_became_active":"2024-04-25T10:12:36.440448+0000","last_became_peered":"2024-04-25T10:12:36.440448+0000","last_unstale":"2024-04-25T10:12:36.441320+0000","last_undegraded":"2024-04-25T10:12:36.441320+0000","last_fullsized":"2024-04-25T10:12:36.441320+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:47.209063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.430942+0000","last_change":"2024-04-25T10:12:36.430942+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.430942+0000","last_undegraded":"2024-04-25T10:12:36.430942+0000","last_fullsized":"2024-04-25T10:12:36.430942+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433298+0000","last_change":"2024-04-25T10:12:36.430380+0000","last_active":"2024-04-25T10:12:36.433298+0000","last_peered":"2024-04-25T10:12:36.433298+0000","last_clean":"2024-04-25T10:12:36.433298+0000","last_became_active":"2024-04-25T10:12:36.429347+0000","last_became_peered":"2024-04-25T10:12:36.429347+0000","last_unstale":"2024-04-25T10:12:36.433298+0000","last_undegraded":"2024-04-25T10:12:36.433298+0000","last_fullsized":"2024-04-25T10:12:36.433298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:31.241082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422992+0000","last_change":"2024-04-25T10:12:36.422992+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422992+0000","last_undegraded":"2024-04-25T10:12:36.422992+0000","last_fullsized":"2024-04-25T10:12:36.422992+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438473+0000","last_change":"2024-04-25T10:12:36.434613+0000","last_active":"2024-04-25T10:12:36.438473+0000","last_peered":"2024-04-25T10:12:36.438473+0000","last_clean":"2024-04-25T10:12:36.438473+0000","last_became_active":"2024-04-25T10:12:36.434287+0000","last_became_peered":"2024-04-25T10:12:36.434287+0000","last_unstale":"2024-04-25T10:12:36.438473+0000","last_undegraded":"2024-04-25T10:12:36.438473+0000","last_fullsized":"2024-04-25T10:12:36.438473+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422083+0000","last_change":"2024-04-25T10:12:36.422083+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422083+0000","last_undegraded":"2024-04-25T10:12:36.422083+0000","last_fullsized":"2024-04-25T10:12:36.422083+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433876+0000","last_change":"2024-04-25T10:12:36.429818+0000","last_active":"2024-04-25T10:12:36.433876+0000","last_peered":"2024-04-25T10:12:36.433876+0000","last_clean":"2024-04-25T10:12:36.433876+0000","last_became_active":"2024-04-25T10:12:36.429160+0000","last_became_peered":"2024-04-25T10:12:36.429160+0000","last_unstale":"2024-04-25T10:12:36.433876+0000","last_undegraded":"2024-04-25T10:12:36.433876+0000","last_fullsized":"2024-04-25T10:12:36.433876+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:37.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422773+0000","last_change":"2024-04-25T10:12:36.422773+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422773+0000","last_undegraded":"2024-04-25T10:12:36.422773+0000","last_fullsized":"2024-04-25T10:12:36.422773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.432528+0000","last_change":"2024-04-25T10:12:36.428404+0000","last_active":"2024-04-25T10:12:36.432528+0000","last_peered":"2024-04-25T10:12:36.432528+0000","last_clean":"2024-04-25T10:12:36.432528+0000","last_became_active":"2024-04-25T10:12:36.428103+0000","last_became_peered":"2024-04-25T10:12:36.428103+0000","last_unstale":"2024-04-25T10:12:36.432528+0000","last_undegraded":"2024-04-25T10:12:36.432528+0000","last_fullsized":"2024-04-25T10:12:36.432528+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.890535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423534+0000","last_change":"2024-04-25T10:12:36.423534+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423534+0000","last_undegraded":"2024-04-25T10:12:36.423534+0000","last_fullsized":"2024-04-25T10:12:36.423534+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.424932+0000","last_change":"2024-04-25T10:12:36.423055+0000","last_active":"2024-04-25T10:12:36.424932+0000","last_peered":"2024-04-25T10:12:36.424932+0000","last_clean":"2024-04-25T10:12:36.424932+0000","last_became_active":"2024-04-25T10:12:36.422751+0000","last_became_peered":"2024-04-25T10:12:36.422751+0000","last_unstale":"2024-04-25T10:12:36.424932+0000","last_undegraded":"2024-04-25T10:12:36.424932+0000","last_fullsized":"2024-04-25T10:12:36.424932+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:09.251063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423575+0000","last_change":"2024-04-25T10:12:36.423575+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423575+0000","last_undegraded":"2024-04-25T10:12:36.423575+0000","last_fullsized":"2024-04-25T10:12:36.423575+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.431150+0000","last_change":"2024-04-25T10:12:36.426977+0000","last_active":"2024-04-25T10:12:36.431150+0000","last_peered":"2024-04-25T10:12:36.431150+0000","last_clean":"2024-04-25T10:12:36.431150+0000","last_became_active":"2024-04-25T10:12:36.424814+0000","last_became_peered":"2024-04-25T10:12:36.424814+0000","last_unstale":"2024-04-25T10:12:36.431150+0000","last_undegraded":"2024-04-25T10:12:36.431150+0000","last_fullsized":"2024-04-25T10:12:36.431150+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:17.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426498+0000","last_change":"2024-04-25T10:12:36.426498+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426498+0000","last_undegraded":"2024-04-25T10:12:36.426498+0000","last_fullsized":"2024-04-25T10:12:36.426498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.421027+0000","last_change":"2024-04-25T10:12:36.419237+0000","last_active":"2024-04-25T10:12:36.421027+0000","last_peered":"2024-04-25T10:12:36.421027+0000","last_clean":"2024-04-25T10:12:36.421027+0000","last_became_active":"2024-04-25T10:12:36.418640+0000","last_became_peered":"2024-04-25T10:12:36.418640+0000","last_unstale":"2024-04-25T10:12:36.421027+0000","last_undegraded":"2024-04-25T10:12:36.421027+0000","last_fullsized":"2024-04-25T10:12:36.421027+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:43.632634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":339,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.422923+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:12:36.422923+0000","last_peered":"2024-04-25T10:12:36.422923+0000","last_clean":"2024-04-25T10:12:36.422923+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:12:36.422923+0000","last_undegraded":"2024-04-25T10:12:36.422923+0000","last_fullsized":"2024-04-25T10:12:36.422923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51132,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320708,"statfs":{"total":96636764160,"available":96584404992,"internally_reserved":0,"allocated":1945600,"data_stored":1220116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50548,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321292,"statfs":{"total":96636764160,"available":96585003008,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46388,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325452,"statfs":{"total":96636764160,"available":96589262848,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58868,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312972,"statfs":{"total":96636764160,"available":96576483328,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58872,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54708,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317132,"statfs":{"total":96636764160,"available":96580743168,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55276,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316564,"statfs":{"total":96636764160,"available":96580161536,"internally_reserved":0,"allocated":1933312,"data_stored":1213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:41.832 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:42.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.065+0000 7fc791c00640 1 Processor -- start 2024-04-25T10:12:42.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.065+0000 7fc791c00640 1 -- start start 2024-04-25T10:12:42.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.067+0000 7fc791c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.067+0000 7fc791c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc78c1615a0 0x7fc78c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.067+0000 7fc791c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc78c0d33f0 con 0x7fc78c0d04f0 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.067+0000 7fc791c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc78c0d0980 con 0x7fc78c1615a0 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.067+0000 7fc791c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0d1f50 con 0x7fc78c15db90 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.067+0000 7fc783400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc78c1615a0 0x7fc78c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc783400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc78c1615a0 0x7fc78c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55692/0 (socket says 172.21.15.165:55692) 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc783400640 1 -- 172.21.15.165:0/3424375274 learned_addr learned my addr 172.21.15.165:0/3424375274 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc791200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc78c0d04f0 0x7fc78c061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:38212/0 (socket says 172.21.15.165:38212) 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc78be00640 1 --2- 172.21.15.165:0/3424375274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc783400640 1 -- 172.21.15.165:0/3424375274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 msgr2=0x7fc78c0b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc783400640 1 --2- 172.21.15.165:0/3424375274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc783400640 1 -- 172.21.15.165:0/3424375274 >> v1:172.21.15.165:6789/0 conn(0x7fc78c0d04f0 legacy=0x7fc78c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.068+0000 7fc783400640 1 -- 172.21.15.165:0/3424375274 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c161df0 con 0x7fc78c1615a0 2024-04-25T10:12:42.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.069+0000 7fc783400640 1 --2- 172.21.15.165:0/3424375274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0cffb0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc774009870 tx=0x7fc77402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=885ad2fee43bd21f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.069+0000 7fc78b400640 1 -- 172.21.15.165:0/3424375274 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774037070 con 0x7fc78c1615a0 2024-04-25T10:12:42.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.069+0000 7fc78b400640 1 -- 172.21.15.165:0/3424375274 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774032440 con 0x7fc78c1615a0 2024-04-25T10:12:42.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.069+0000 7fc78b400640 1 -- 172.21.15.165:0/3424375274 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774032810 con 0x7fc78c1615a0 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 -- 172.21.15.165:0/3424375274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 msgr2=0x7fc78c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 --2- 172.21.15.165:0/3424375274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0cffb0 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7fc774009870 tx=0x7fc77402a250 comp rx=0 tx=0).stop 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 -- 172.21.15.165:0/3424375274 shutdown_connections 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 --2- 172.21.15.165:0/3424375274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0cffb0 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 --2- 172.21.15.165:0/3424375274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 -- 172.21.15.165:0/3424375274 >> 172.21.15.165:0/3424375274 conn(0x7fc78c0b52b0 msgr2=0x7fc78c0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 -- 172.21.15.165:0/3424375274 shutdown_connections 2024-04-25T10:12:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.070+0000 7fc791c00640 1 -- 172.21.15.165:0/3424375274 wait complete. 2024-04-25T10:12:42.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.072+0000 7fc791c00640 1 Processor -- start 2024-04-25T10:12:42.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.072+0000 7fc791c00640 1 -- start start 2024-04-25T10:12:42.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.073+0000 7fc791c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc78c0d04f0 0x7fc78c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.073+0000 7fc791c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.073+0000 7fc791c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0b81c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.073+0000 7fc791c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c0d1be0 con 0x7fc78c1615a0 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.073+0000 7fc791c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c1623b0 con 0x7fc78c0d04f0 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.073+0000 7fc791c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc78c161a20 con 0x7fc78c15db90 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0b81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0b81c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55700/0 (socket says 172.21.15.165:55700) 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 -- 172.21.15.165:0/440349898 learned_addr learned my addr 172.21.15.165:0/440349898 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 -- 172.21.15.165:0/440349898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 msgr2=0x7fc78c0b7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 -- 172.21.15.165:0/440349898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc78c0d04f0 msgr2=0x7fc78c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc78c0d04f0 0x7fc78c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 -- 172.21.15.165:0/440349898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc78c0cbc60 con 0x7fc78c1615a0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.074+0000 7fc791200640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0b81c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c730 tx=0x7fc78400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.075+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc784017070 con 0x7fc78c1615a0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.075+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78400ce40 con 0x7fc78c1615a0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.075+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc78400a7e0 con 0x7fc78c1615a0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.075+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc78c0cbf40 con 0x7fc78c1615a0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.075+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc78c0cc500 con 0x7fc78c1615a0 2024-04-25T10:12:42.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.076+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc78c073990 con 0x7fc78c1615a0 2024-04-25T10:12:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.077+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc78401a070 con 0x7fc78c1615a0 2024-04-25T10:12:42.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.078+0000 7fc788c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc76806b500 0x7fc76806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.078+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc784089920 con 0x7fc78c1615a0 2024-04-25T10:12:42.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.078+0000 7fc78be00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc76806b500 0x7fc76806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.079+0000 7fc78be00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc76806b500 0x7fc76806d9c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0098f0 tx=0x7fc77c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.082+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc784057f20 con 0x7fc78c1615a0 2024-04-25T10:12:42.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.279+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc78c0d14b0 con 0x7fc76806b500 2024-04-25T10:12:42.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.292+0000 7fc788c00640 1 -- 172.21.15.165:0/440349898 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247089 (secure 0 0 0) 0x7fc78c0d14b0 con 0x7fc76806b500 2024-04-25T10:12:42.300 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:42.301 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:12:42.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc76806b500 msgr2=0x7fc76806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc76806b500 0x7fc76806d9c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0098f0 tx=0x7fc77c008040 comp rx=0 tx=0).stop 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 msgr2=0x7fc78c0b81c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0b81c0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7fc78400c730 tx=0x7fc78400cbf0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 shutdown_connections 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc76806b500 0x7fc76806d9c0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc78c1615a0 0x7fc78c0b81c0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc78c15db90 0x7fc78c0b7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 --2- 172.21.15.165:0/440349898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc78c0d04f0 0x7fc78c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 >> 172.21.15.165:0/440349898 conn(0x7fc78c0b52b0 msgr2=0x7fc78c0c9bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 shutdown_connections 2024-04-25T10:12:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.298+0000 7fc791c00640 1 -- 172.21.15.165:0/440349898 wait complete. 2024-04-25T10:12:42.321 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":428,"stamp":"2024-04-25T10:12:40.344912+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":28,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":426344,"kb_used_data":11708,"kb_used_omap":12,"kb_used_meta":414515,"kb_avail":754548376,"statfs":{"total":773094113280,"available":772657537024,"internally_reserved":0,"allocated":11988992,"data_stored":6212053,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":424463922},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":19,"apply_latency_ms":19,"commit_latency_ns":19000000,"apply_latency_ns":19000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.955302"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.433601+0000","last_change":"2024-04-25T10:12:36.433601+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.433601+0000","last_undegraded":"2024-04-25T10:12:36.433601+0000","last_fullsized":"2024-04-25T10:12:36.433601+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.420389+0000","last_change":"2024-04-25T10:12:36.420389+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.420389+0000","last_undegraded":"2024-04-25T10:12:36.420389+0000","last_fullsized":"2024-04-25T10:12:36.420389+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.436046+0000","last_change":"2024-04-25T10:12:36.436046+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.436046+0000","last_undegraded":"2024-04-25T10:12:36.436046+0000","last_fullsized":"2024-04-25T10:12:36.436046+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.431900+0000","last_change":"2024-04-25T10:12:36.431900+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.431900+0000","last_undegraded":"2024-04-25T10:12:36.431900+0000","last_fullsized":"2024-04-25T10:12:36.431900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426448+0000","last_change":"2024-04-25T10:12:36.426448+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426448+0000","last_undegraded":"2024-04-25T10:12:36.426448+0000","last_fullsized":"2024-04-25T10:12:36.426448+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.412404+0000","last_change":"2024-04-25T10:12:36.412404+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.412404+0000","last_undegraded":"2024-04-25T10:12:36.412404+0000","last_fullsized":"2024-04-25T10:12:36.412404+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.413363+0000","last_change":"2024-04-25T10:12:36.413363+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.413363+0000","last_undegraded":"2024-04-25T10:12:36.413363+0000","last_fullsized":"2024-04-25T10:12:36.413363+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426612+0000","last_change":"2024-04-25T10:12:36.426612+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426612+0000","last_undegraded":"2024-04-25T10:12:36.426612+0000","last_fullsized":"2024-04-25T10:12:36.426612+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.414396+0000","last_change":"2024-04-25T10:12:36.414396+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.414396+0000","last_undegraded":"2024-04-25T10:12:36.414396+0000","last_fullsized":"2024-04-25T10:12:36.414396+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.427360+0000","last_change":"2024-04-25T10:12:36.427360+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.427360+0000","last_undegraded":"2024-04-25T10:12:36.427360+0000","last_fullsized":"2024-04-25T10:12:36.427360+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428859+0000","last_change":"2024-04-25T10:12:36.428859+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428859+0000","last_undegraded":"2024-04-25T10:12:36.428859+0000","last_fullsized":"2024-04-25T10:12:36.428859+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.417088+0000","last_change":"2024-04-25T10:12:36.417088+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.417088+0000","last_undegraded":"2024-04-25T10:12:36.417088+0000","last_fullsized":"2024-04-25T10:12:36.417088+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.430707+0000","last_change":"2024-04-25T10:12:36.430707+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.430707+0000","last_undegraded":"2024-04-25T10:12:36.430707+0000","last_fullsized":"2024-04-25T10:12:36.430707+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425443+0000","last_change":"2024-04-25T10:12:36.425443+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425443+0000","last_undegraded":"2024-04-25T10:12:36.425443+0000","last_fullsized":"2024-04-25T10:12:36.425443+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423319+0000","last_change":"2024-04-25T10:12:36.423319+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423319+0000","last_undegraded":"2024-04-25T10:12:36.423319+0000","last_fullsized":"2024-04-25T10:12:36.423319+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422605+0000","last_change":"2024-04-25T10:12:36.422605+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422605+0000","last_undegraded":"2024-04-25T10:12:36.422605+0000","last_fullsized":"2024-04-25T10:12:36.422605+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426662+0000","last_change":"2024-04-25T10:12:36.426662+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426662+0000","last_undegraded":"2024-04-25T10:12:36.426662+0000","last_fullsized":"2024-04-25T10:12:36.426662+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.417057+0000","last_change":"2024-04-25T10:12:36.417057+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.417057+0000","last_undegraded":"2024-04-25T10:12:36.417057+0000","last_fullsized":"2024-04-25T10:12:36.417057+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.434665+0000","last_change":"2024-04-25T10:12:36.434665+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.434665+0000","last_undegraded":"2024-04-25T10:12:36.434665+0000","last_fullsized":"2024-04-25T10:12:36.434665+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.416049+0000","last_change":"2024-04-25T10:12:36.416049+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.416049+0000","last_undegraded":"2024-04-25T10:12:36.416049+0000","last_fullsized":"2024-04-25T10:12:36.416049+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.427090+0000","last_change":"2024-04-25T10:12:36.427090+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.427090+0000","last_undegraded":"2024-04-25T10:12:36.427090+0000","last_fullsized":"2024-04-25T10:12:36.427090+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425039+0000","last_change":"2024-04-25T10:12:36.425039+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425039+0000","last_undegraded":"2024-04-25T10:12:36.425039+0000","last_fullsized":"2024-04-25T10:12:36.425039+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428265+0000","last_change":"2024-04-25T10:12:36.428265+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428265+0000","last_undegraded":"2024-04-25T10:12:36.428265+0000","last_fullsized":"2024-04-25T10:12:36.428265+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.424969+0000","last_change":"2024-04-25T10:12:36.424969+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.424969+0000","last_undegraded":"2024-04-25T10:12:36.424969+0000","last_fullsized":"2024-04-25T10:12:36.424969+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.415694+0000","last_change":"2024-04-25T10:12:36.415694+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.415694+0000","last_undegraded":"2024-04-25T10:12:36.415694+0000","last_fullsized":"2024-04-25T10:12:36.415694+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.414546+0000","last_change":"2024-04-25T10:12:36.414546+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.414546+0000","last_undegraded":"2024-04-25T10:12:36.414546+0000","last_fullsized":"2024-04-25T10:12:36.414546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428426+0000","last_change":"2024-04-25T10:12:36.428426+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428426+0000","last_undegraded":"2024-04-25T10:12:36.428426+0000","last_fullsized":"2024-04-25T10:12:36.428426+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.411138+0000","last_change":"2024-04-25T10:12:36.411138+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.411138+0000","last_undegraded":"2024-04-25T10:12:36.411138+0000","last_fullsized":"2024-04-25T10:12:36.411138+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.424024+0000","last_change":"2024-04-25T10:12:36.424024+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.424024+0000","last_undegraded":"2024-04-25T10:12:36.424024+0000","last_fullsized":"2024-04-25T10:12:36.424024+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.413939+0000","last_change":"2024-04-25T10:12:36.413939+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.413939+0000","last_undegraded":"2024-04-25T10:12:36.413939+0000","last_fullsized":"2024-04-25T10:12:36.413939+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.412188+0000","last_change":"2024-04-25T10:12:36.412188+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.412188+0000","last_undegraded":"2024-04-25T10:12:36.412188+0000","last_fullsized":"2024-04-25T10:12:36.412188+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426053+0000","last_change":"2024-04-25T10:12:36.426053+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426053+0000","last_undegraded":"2024-04-25T10:12:36.426053+0000","last_fullsized":"2024-04-25T10:12:36.426053+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423148+0000","last_change":"2024-04-25T10:12:36.423148+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423148+0000","last_undegraded":"2024-04-25T10:12:36.423148+0000","last_fullsized":"2024-04-25T10:12:36.423148+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433093+0000","last_change":"2024-04-25T10:12:36.428651+0000","last_active":"2024-04-25T10:12:36.433093+0000","last_peered":"2024-04-25T10:12:36.433093+0000","last_clean":"2024-04-25T10:12:36.433093+0000","last_became_active":"2024-04-25T10:12:36.428062+0000","last_became_peered":"2024-04-25T10:12:36.428062+0000","last_unstale":"2024-04-25T10:12:36.433093+0000","last_undegraded":"2024-04-25T10:12:36.433093+0000","last_fullsized":"2024-04-25T10:12:36.433093+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:38.055414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422457+0000","last_change":"2024-04-25T10:12:36.422457+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422457+0000","last_undegraded":"2024-04-25T10:12:36.422457+0000","last_fullsized":"2024-04-25T10:12:36.422457+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.435053+0000","last_change":"2024-04-25T10:12:36.432356+0000","last_active":"2024-04-25T10:12:36.435053+0000","last_peered":"2024-04-25T10:12:36.435053+0000","last_clean":"2024-04-25T10:12:36.435053+0000","last_became_active":"2024-04-25T10:12:36.432086+0000","last_became_peered":"2024-04-25T10:12:36.432086+0000","last_unstale":"2024-04-25T10:12:36.435053+0000","last_undegraded":"2024-04-25T10:12:36.435053+0000","last_fullsized":"2024-04-25T10:12:36.435053+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:54.943354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.411887+0000","last_change":"2024-04-25T10:12:36.411887+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.411887+0000","last_undegraded":"2024-04-25T10:12:36.411887+0000","last_fullsized":"2024-04-25T10:12:36.411887+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438917+0000","last_change":"2024-04-25T10:12:36.437597+0000","last_active":"2024-04-25T10:12:36.438917+0000","last_peered":"2024-04-25T10:12:36.438917+0000","last_clean":"2024-04-25T10:12:36.438917+0000","last_became_active":"2024-04-25T10:12:36.437214+0000","last_became_peered":"2024-04-25T10:12:36.437214+0000","last_unstale":"2024-04-25T10:12:36.438917+0000","last_undegraded":"2024-04-25T10:12:36.438917+0000","last_fullsized":"2024-04-25T10:12:36.438917+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.420749+0000","last_change":"2024-04-25T10:12:36.420749+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.420749+0000","last_undegraded":"2024-04-25T10:12:36.420749+0000","last_fullsized":"2024-04-25T10:12:36.420749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.434471+0000","last_change":"2024-04-25T10:12:36.430378+0000","last_active":"2024-04-25T10:12:36.434471+0000","last_peered":"2024-04-25T10:12:36.434471+0000","last_clean":"2024-04-25T10:12:36.434471+0000","last_became_active":"2024-04-25T10:12:36.430043+0000","last_became_peered":"2024-04-25T10:12:36.430043+0000","last_unstale":"2024-04-25T10:12:36.434471+0000","last_undegraded":"2024-04-25T10:12:36.434471+0000","last_fullsized":"2024-04-25T10:12:36.434471+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.428765+0000","last_change":"2024-04-25T10:12:36.428765+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.428765+0000","last_undegraded":"2024-04-25T10:12:36.428765+0000","last_fullsized":"2024-04-25T10:12:36.428765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.435090+0000","last_change":"2024-04-25T10:12:36.432661+0000","last_active":"2024-04-25T10:12:36.435090+0000","last_peered":"2024-04-25T10:12:36.435090+0000","last_clean":"2024-04-25T10:12:36.435090+0000","last_became_active":"2024-04-25T10:12:36.432286+0000","last_became_peered":"2024-04-25T10:12:36.432286+0000","last_unstale":"2024-04-25T10:12:36.435090+0000","last_undegraded":"2024-04-25T10:12:36.435090+0000","last_fullsized":"2024-04-25T10:12:36.435090+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:17.416609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423407+0000","last_change":"2024-04-25T10:12:36.423407+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423407+0000","last_undegraded":"2024-04-25T10:12:36.423407+0000","last_fullsized":"2024-04-25T10:12:36.423407+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.440008+0000","last_change":"2024-04-25T10:12:36.437498+0000","last_active":"2024-04-25T10:12:36.440008+0000","last_peered":"2024-04-25T10:12:36.440008+0000","last_clean":"2024-04-25T10:12:36.440008+0000","last_became_active":"2024-04-25T10:12:36.437164+0000","last_became_peered":"2024-04-25T10:12:36.437164+0000","last_unstale":"2024-04-25T10:12:36.440008+0000","last_undegraded":"2024-04-25T10:12:36.440008+0000","last_fullsized":"2024-04-25T10:12:36.440008+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:14.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.416272+0000","last_change":"2024-04-25T10:12:36.416272+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.416272+0000","last_undegraded":"2024-04-25T10:12:36.416272+0000","last_fullsized":"2024-04-25T10:12:36.416272+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438102+0000","last_change":"2024-04-25T10:12:36.436140+0000","last_active":"2024-04-25T10:12:36.438102+0000","last_peered":"2024-04-25T10:12:36.438102+0000","last_clean":"2024-04-25T10:12:36.438102+0000","last_became_active":"2024-04-25T10:12:36.435843+0000","last_became_peered":"2024-04-25T10:12:36.435843+0000","last_unstale":"2024-04-25T10:12:36.438102+0000","last_undegraded":"2024-04-25T10:12:36.438102+0000","last_fullsized":"2024-04-25T10:12:36.438102+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.986763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.429636+0000","last_change":"2024-04-25T10:12:36.429636+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.429636+0000","last_undegraded":"2024-04-25T10:12:36.429636+0000","last_fullsized":"2024-04-25T10:12:36.429636+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.435442+0000","last_change":"2024-04-25T10:12:36.433709+0000","last_active":"2024-04-25T10:12:36.435442+0000","last_peered":"2024-04-25T10:12:36.435442+0000","last_clean":"2024-04-25T10:12:36.435442+0000","last_became_active":"2024-04-25T10:12:36.433440+0000","last_became_peered":"2024-04-25T10:12:36.433440+0000","last_unstale":"2024-04-25T10:12:36.435442+0000","last_undegraded":"2024-04-25T10:12:36.435442+0000","last_fullsized":"2024-04-25T10:12:36.435442+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.503044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.424245+0000","last_change":"2024-04-25T10:12:36.424245+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.424245+0000","last_undegraded":"2024-04-25T10:12:36.424245+0000","last_fullsized":"2024-04-25T10:12:36.424245+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433721+0000","last_change":"2024-04-25T10:12:36.430969+0000","last_active":"2024-04-25T10:12:36.433721+0000","last_peered":"2024-04-25T10:12:36.433721+0000","last_clean":"2024-04-25T10:12:36.433721+0000","last_became_active":"2024-04-25T10:12:36.430681+0000","last_became_peered":"2024-04-25T10:12:36.430681+0000","last_unstale":"2024-04-25T10:12:36.433721+0000","last_undegraded":"2024-04-25T10:12:36.433721+0000","last_fullsized":"2024-04-25T10:12:36.433721+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:25.848233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426056+0000","last_change":"2024-04-25T10:12:36.426056+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426056+0000","last_undegraded":"2024-04-25T10:12:36.426056+0000","last_fullsized":"2024-04-25T10:12:36.426056+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437029+0000","last_change":"2024-04-25T10:12:36.435089+0000","last_active":"2024-04-25T10:12:36.437029+0000","last_peered":"2024-04-25T10:12:36.437029+0000","last_clean":"2024-04-25T10:12:36.437029+0000","last_became_active":"2024-04-25T10:12:36.434759+0000","last_became_peered":"2024-04-25T10:12:36.434759+0000","last_unstale":"2024-04-25T10:12:36.437029+0000","last_undegraded":"2024-04-25T10:12:36.437029+0000","last_fullsized":"2024-04-25T10:12:36.437029+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:12.593794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422641+0000","last_change":"2024-04-25T10:12:36.422641+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422641+0000","last_undegraded":"2024-04-25T10:12:36.422641+0000","last_fullsized":"2024-04-25T10:12:36.422641+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.440408+0000","last_change":"2024-04-25T10:12:36.438390+0000","last_active":"2024-04-25T10:12:36.440408+0000","last_peered":"2024-04-25T10:12:36.440408+0000","last_clean":"2024-04-25T10:12:36.440408+0000","last_became_active":"2024-04-25T10:12:36.438100+0000","last_became_peered":"2024-04-25T10:12:36.438100+0000","last_unstale":"2024-04-25T10:12:36.440408+0000","last_undegraded":"2024-04-25T10:12:36.440408+0000","last_fullsized":"2024-04-25T10:12:36.440408+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:42.723368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425357+0000","last_change":"2024-04-25T10:12:36.425357+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425357+0000","last_undegraded":"2024-04-25T10:12:36.425357+0000","last_fullsized":"2024-04-25T10:12:36.425357+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437664+0000","last_change":"2024-04-25T10:12:36.434095+0000","last_active":"2024-04-25T10:12:36.437664+0000","last_peered":"2024-04-25T10:12:36.437664+0000","last_clean":"2024-04-25T10:12:36.437664+0000","last_became_active":"2024-04-25T10:12:36.433803+0000","last_became_peered":"2024-04-25T10:12:36.433803+0000","last_unstale":"2024-04-25T10:12:36.437664+0000","last_undegraded":"2024-04-25T10:12:36.437664+0000","last_fullsized":"2024-04-25T10:12:36.437664+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:37.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.430525+0000","last_change":"2024-04-25T10:12:36.430525+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.430525+0000","last_undegraded":"2024-04-25T10:12:36.430525+0000","last_fullsized":"2024-04-25T10:12:36.430525+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.429379+0000","last_change":"2024-04-25T10:12:36.420615+0000","last_active":"2024-04-25T10:12:36.429379+0000","last_peered":"2024-04-25T10:12:36.429379+0000","last_clean":"2024-04-25T10:12:36.429379+0000","last_became_active":"2024-04-25T10:12:36.420089+0000","last_became_peered":"2024-04-25T10:12:36.420089+0000","last_unstale":"2024-04-25T10:12:36.429379+0000","last_undegraded":"2024-04-25T10:12:36.429379+0000","last_fullsized":"2024-04-25T10:12:36.429379+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.417054+0000","last_change":"2024-04-25T10:12:36.417054+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.417054+0000","last_undegraded":"2024-04-25T10:12:36.417054+0000","last_fullsized":"2024-04-25T10:12:36.417054+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433939+0000","last_change":"2024-04-25T10:12:36.432744+0000","last_active":"2024-04-25T10:12:36.433939+0000","last_peered":"2024-04-25T10:12:36.433939+0000","last_clean":"2024-04-25T10:12:36.433939+0000","last_became_active":"2024-04-25T10:12:36.431971+0000","last_became_peered":"2024-04-25T10:12:36.431971+0000","last_unstale":"2024-04-25T10:12:36.433939+0000","last_undegraded":"2024-04-25T10:12:36.433939+0000","last_fullsized":"2024-04-25T10:12:36.433939+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:18.232182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422041+0000","last_change":"2024-04-25T10:12:36.422041+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422041+0000","last_undegraded":"2024-04-25T10:12:36.422041+0000","last_fullsized":"2024-04-25T10:12:36.422041+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433117+0000","last_change":"2024-04-25T10:12:36.429891+0000","last_active":"2024-04-25T10:12:36.433117+0000","last_peered":"2024-04-25T10:12:36.433117+0000","last_clean":"2024-04-25T10:12:36.433117+0000","last_became_active":"2024-04-25T10:12:36.429336+0000","last_became_peered":"2024-04-25T10:12:36.429336+0000","last_unstale":"2024-04-25T10:12:36.433117+0000","last_undegraded":"2024-04-25T10:12:36.433117+0000","last_fullsized":"2024-04-25T10:12:36.433117+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:08:54.329258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423259+0000","last_change":"2024-04-25T10:12:36.423259+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423259+0000","last_undegraded":"2024-04-25T10:12:36.423259+0000","last_fullsized":"2024-04-25T10:12:36.423259+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437982+0000","last_change":"2024-04-25T10:12:36.434858+0000","last_active":"2024-04-25T10:12:36.437982+0000","last_peered":"2024-04-25T10:12:36.437982+0000","last_clean":"2024-04-25T10:12:36.437982+0000","last_became_active":"2024-04-25T10:12:36.434571+0000","last_became_peered":"2024-04-25T10:12:36.434571+0000","last_unstale":"2024-04-25T10:12:36.437982+0000","last_undegraded":"2024-04-25T10:12:36.437982+0000","last_fullsized":"2024-04-25T10:12:36.437982+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425482+0000","last_change":"2024-04-25T10:12:36.425482+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425482+0000","last_undegraded":"2024-04-25T10:12:36.425482+0000","last_fullsized":"2024-04-25T10:12:36.425482+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438927+0000","last_change":"2024-04-25T10:12:36.437073+0000","last_active":"2024-04-25T10:12:36.438927+0000","last_peered":"2024-04-25T10:12:36.438927+0000","last_clean":"2024-04-25T10:12:36.438927+0000","last_became_active":"2024-04-25T10:12:36.436575+0000","last_became_peered":"2024-04-25T10:12:36.436575+0000","last_unstale":"2024-04-25T10:12:36.438927+0000","last_undegraded":"2024-04-25T10:12:36.438927+0000","last_fullsized":"2024-04-25T10:12:36.438927+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.746043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.411286+0000","last_change":"2024-04-25T10:12:36.411286+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.411286+0000","last_undegraded":"2024-04-25T10:12:36.411286+0000","last_fullsized":"2024-04-25T10:12:36.411286+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433434+0000","last_change":"2024-04-25T10:12:36.430895+0000","last_active":"2024-04-25T10:12:36.433434+0000","last_peered":"2024-04-25T10:12:36.433434+0000","last_clean":"2024-04-25T10:12:36.433434+0000","last_became_active":"2024-04-25T10:12:36.430565+0000","last_became_peered":"2024-04-25T10:12:36.430565+0000","last_unstale":"2024-04-25T10:12:36.433434+0000","last_undegraded":"2024-04-25T10:12:36.433434+0000","last_fullsized":"2024-04-25T10:12:36.433434+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:34.503044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425911+0000","last_change":"2024-04-25T10:12:36.425911+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425911+0000","last_undegraded":"2024-04-25T10:12:36.425911+0000","last_fullsized":"2024-04-25T10:12:36.425911+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.439907+0000","last_change":"2024-04-25T10:12:36.438473+0000","last_active":"2024-04-25T10:12:36.439907+0000","last_peered":"2024-04-25T10:12:36.439907+0000","last_clean":"2024-04-25T10:12:36.439907+0000","last_became_active":"2024-04-25T10:12:36.436411+0000","last_became_peered":"2024-04-25T10:12:36.436411+0000","last_unstale":"2024-04-25T10:12:36.439907+0000","last_undegraded":"2024-04-25T10:12:36.439907+0000","last_fullsized":"2024-04-25T10:12:36.439907+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:22.288167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.425846+0000","last_change":"2024-04-25T10:12:36.425846+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.425846+0000","last_undegraded":"2024-04-25T10:12:36.425846+0000","last_fullsized":"2024-04-25T10:12:36.425846+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.430981+0000","last_change":"2024-04-25T10:12:36.428322+0000","last_active":"2024-04-25T10:12:36.430981+0000","last_peered":"2024-04-25T10:12:36.430981+0000","last_clean":"2024-04-25T10:12:36.430981+0000","last_became_active":"2024-04-25T10:12:36.427541+0000","last_became_peered":"2024-04-25T10:12:36.427541+0000","last_unstale":"2024-04-25T10:12:36.430981+0000","last_undegraded":"2024-04-25T10:12:36.430981+0000","last_fullsized":"2024-04-25T10:12:36.430981+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:47.884287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.429110+0000","last_change":"2024-04-25T10:12:36.429110+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.429110+0000","last_undegraded":"2024-04-25T10:12:36.429110+0000","last_fullsized":"2024-04-25T10:12:36.429110+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.424174+0000","last_change":"2024-04-25T10:12:36.420411+0000","last_active":"2024-04-25T10:12:36.424174+0000","last_peered":"2024-04-25T10:12:36.424174+0000","last_clean":"2024-04-25T10:12:36.424174+0000","last_became_active":"2024-04-25T10:12:36.419903+0000","last_became_peered":"2024-04-25T10:12:36.419903+0000","last_unstale":"2024-04-25T10:12:36.424174+0000","last_undegraded":"2024-04-25T10:12:36.424174+0000","last_fullsized":"2024-04-25T10:12:36.424174+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:27.226759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.429648+0000","last_change":"2024-04-25T10:12:36.429648+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.429648+0000","last_undegraded":"2024-04-25T10:12:36.429648+0000","last_fullsized":"2024-04-25T10:12:36.429648+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.437624+0000","last_change":"2024-04-25T10:12:36.435352+0000","last_active":"2024-04-25T10:12:36.437624+0000","last_peered":"2024-04-25T10:12:36.437624+0000","last_clean":"2024-04-25T10:12:36.437624+0000","last_became_active":"2024-04-25T10:12:36.434943+0000","last_became_peered":"2024-04-25T10:12:36.434943+0000","last_unstale":"2024-04-25T10:12:36.437624+0000","last_undegraded":"2024-04-25T10:12:36.437624+0000","last_fullsized":"2024-04-25T10:12:36.437624+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:56:25.848233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422862+0000","last_change":"2024-04-25T10:12:36.422862+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422862+0000","last_undegraded":"2024-04-25T10:12:36.422862+0000","last_fullsized":"2024-04-25T10:12:36.422862+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.427121+0000","last_change":"2024-04-25T10:12:36.419478+0000","last_active":"2024-04-25T10:12:36.427121+0000","last_peered":"2024-04-25T10:12:36.427121+0000","last_clean":"2024-04-25T10:12:36.427121+0000","last_became_active":"2024-04-25T10:12:36.419010+0000","last_became_peered":"2024-04-25T10:12:36.419010+0000","last_unstale":"2024-04-25T10:12:36.427121+0000","last_undegraded":"2024-04-25T10:12:36.427121+0000","last_fullsized":"2024-04-25T10:12:36.427121+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:10:32.357148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426726+0000","last_change":"2024-04-25T10:12:36.426726+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426726+0000","last_undegraded":"2024-04-25T10:12:36.426726+0000","last_fullsized":"2024-04-25T10:12:36.426726+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.434660+0000","last_change":"2024-04-25T10:12:36.424429+0000","last_active":"2024-04-25T10:12:36.434660+0000","last_peered":"2024-04-25T10:12:36.434660+0000","last_clean":"2024-04-25T10:12:36.434660+0000","last_became_active":"2024-04-25T10:12:36.423878+0000","last_became_peered":"2024-04-25T10:12:36.423878+0000","last_unstale":"2024-04-25T10:12:36.434660+0000","last_undegraded":"2024-04-25T10:12:36.434660+0000","last_fullsized":"2024-04-25T10:12:36.434660+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.413518+0000","last_change":"2024-04-25T10:12:36.413518+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.413518+0000","last_undegraded":"2024-04-25T10:12:36.413518+0000","last_fullsized":"2024-04-25T10:12:36.413518+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.441320+0000","last_change":"2024-04-25T10:12:36.440745+0000","last_active":"2024-04-25T10:12:36.441320+0000","last_peered":"2024-04-25T10:12:36.441320+0000","last_clean":"2024-04-25T10:12:36.441320+0000","last_became_active":"2024-04-25T10:12:36.440448+0000","last_became_peered":"2024-04-25T10:12:36.440448+0000","last_unstale":"2024-04-25T10:12:36.441320+0000","last_undegraded":"2024-04-25T10:12:36.441320+0000","last_fullsized":"2024-04-25T10:12:36.441320+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:47.209063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.430942+0000","last_change":"2024-04-25T10:12:36.430942+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.430942+0000","last_undegraded":"2024-04-25T10:12:36.430942+0000","last_fullsized":"2024-04-25T10:12:36.430942+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433298+0000","last_change":"2024-04-25T10:12:36.430380+0000","last_active":"2024-04-25T10:12:36.433298+0000","last_peered":"2024-04-25T10:12:36.433298+0000","last_clean":"2024-04-25T10:12:36.433298+0000","last_became_active":"2024-04-25T10:12:36.429347+0000","last_became_peered":"2024-04-25T10:12:36.429347+0000","last_unstale":"2024-04-25T10:12:36.433298+0000","last_undegraded":"2024-04-25T10:12:36.433298+0000","last_fullsized":"2024-04-25T10:12:36.433298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:37:31.241082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422992+0000","last_change":"2024-04-25T10:12:36.422992+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422992+0000","last_undegraded":"2024-04-25T10:12:36.422992+0000","last_fullsized":"2024-04-25T10:12:36.422992+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.438473+0000","last_change":"2024-04-25T10:12:36.434613+0000","last_active":"2024-04-25T10:12:36.438473+0000","last_peered":"2024-04-25T10:12:36.438473+0000","last_clean":"2024-04-25T10:12:36.438473+0000","last_became_active":"2024-04-25T10:12:36.434287+0000","last_became_peered":"2024-04-25T10:12:36.434287+0000","last_unstale":"2024-04-25T10:12:36.438473+0000","last_undegraded":"2024-04-25T10:12:36.438473+0000","last_fullsized":"2024-04-25T10:12:36.438473+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422083+0000","last_change":"2024-04-25T10:12:36.422083+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422083+0000","last_undegraded":"2024-04-25T10:12:36.422083+0000","last_fullsized":"2024-04-25T10:12:36.422083+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.433876+0000","last_change":"2024-04-25T10:12:36.429818+0000","last_active":"2024-04-25T10:12:36.433876+0000","last_peered":"2024-04-25T10:12:36.433876+0000","last_clean":"2024-04-25T10:12:36.433876+0000","last_became_active":"2024-04-25T10:12:36.429160+0000","last_became_peered":"2024-04-25T10:12:36.429160+0000","last_unstale":"2024-04-25T10:12:36.433876+0000","last_undegraded":"2024-04-25T10:12:36.433876+0000","last_fullsized":"2024-04-25T10:12:36.433876+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:37.927550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.422773+0000","last_change":"2024-04-25T10:12:36.422773+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.422773+0000","last_undegraded":"2024-04-25T10:12:36.422773+0000","last_fullsized":"2024-04-25T10:12:36.422773+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.432528+0000","last_change":"2024-04-25T10:12:36.428404+0000","last_active":"2024-04-25T10:12:36.432528+0000","last_peered":"2024-04-25T10:12:36.432528+0000","last_clean":"2024-04-25T10:12:36.432528+0000","last_became_active":"2024-04-25T10:12:36.428103+0000","last_became_peered":"2024-04-25T10:12:36.428103+0000","last_unstale":"2024-04-25T10:12:36.432528+0000","last_undegraded":"2024-04-25T10:12:36.432528+0000","last_fullsized":"2024-04-25T10:12:36.432528+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:17:43.890535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423534+0000","last_change":"2024-04-25T10:12:36.423534+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423534+0000","last_undegraded":"2024-04-25T10:12:36.423534+0000","last_fullsized":"2024-04-25T10:12:36.423534+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.424932+0000","last_change":"2024-04-25T10:12:36.423055+0000","last_active":"2024-04-25T10:12:36.424932+0000","last_peered":"2024-04-25T10:12:36.424932+0000","last_clean":"2024-04-25T10:12:36.424932+0000","last_became_active":"2024-04-25T10:12:36.422751+0000","last_became_peered":"2024-04-25T10:12:36.422751+0000","last_unstale":"2024-04-25T10:12:36.424932+0000","last_undegraded":"2024-04-25T10:12:36.424932+0000","last_fullsized":"2024-04-25T10:12:36.424932+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:49:09.251063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.423575+0000","last_change":"2024-04-25T10:12:36.423575+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.423575+0000","last_undegraded":"2024-04-25T10:12:36.423575+0000","last_fullsized":"2024-04-25T10:12:36.423575+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.431150+0000","last_change":"2024-04-25T10:12:36.426977+0000","last_active":"2024-04-25T10:12:36.431150+0000","last_peered":"2024-04-25T10:12:36.431150+0000","last_clean":"2024-04-25T10:12:36.431150+0000","last_became_active":"2024-04-25T10:12:36.424814+0000","last_became_peered":"2024-04-25T10:12:36.424814+0000","last_unstale":"2024-04-25T10:12:36.431150+0000","last_undegraded":"2024-04-25T10:12:36.431150+0000","last_fullsized":"2024-04-25T10:12:36.431150+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:18:17.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":1,"reported_epoch":109,"state":"creating+peering","last_fresh":"2024-04-25T10:12:36.426498+0000","last_change":"2024-04-25T10:12:36.426498+0000","last_active":"2024-04-25T10:12:36.386885+0000","last_peered":"2024-04-25T10:12:36.386885+0000","last_clean":"2024-04-25T10:12:36.386885+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:12:36.426498+0000","last_undegraded":"2024-04-25T10:12:36.426498+0000","last_fullsized":"2024-04-25T10:12:36.426498+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.421027+0000","last_change":"2024-04-25T10:12:36.419237+0000","last_active":"2024-04-25T10:12:36.421027+0000","last_peered":"2024-04-25T10:12:36.421027+0000","last_clean":"2024-04-25T10:12:36.421027+0000","last_became_active":"2024-04-25T10:12:36.418640+0000","last_became_peered":"2024-04-25T10:12:36.418640+0000","last_unstale":"2024-04-25T10:12:36.421027+0000","last_undegraded":"2024-04-25T10:12:36.421027+0000","last_fullsized":"2024-04-25T10:12:36.421027+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:58:43.632634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":339,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-25T10:12:36.422923+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:12:36.422923+0000","last_peered":"2024-04-25T10:12:36.422923+0000","last_clean":"2024-04-25T10:12:36.422923+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:12:36.422923+0000","last_undegraded":"2024-04-25T10:12:36.422923+0000","last_fullsized":"2024-04-25T10:12:36.422923+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51132,"kb_used_data":1900,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320708,"statfs":{"total":96636764160,"available":96584404992,"internally_reserved":0,"allocated":1945600,"data_stored":1220116,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50548,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321292,"statfs":{"total":96636764160,"available":96585003008,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46388,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325452,"statfs":{"total":96636764160,"available":96589262848,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":46135750},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574981,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58868,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312972,"statfs":{"total":96636764160,"available":96576483328,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574981,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58872,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312968,"statfs":{"total":96636764160,"available":96576479232,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574980,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54708,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317132,"statfs":{"total":96636764160,"available":96580743168,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574981,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50552,"kb_used_data":1320,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321288,"statfs":{"total":96636764160,"available":96584998912,"internally_reserved":0,"allocated":1351680,"data_stored":629748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574981,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55276,"kb_used_data":1888,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316564,"statfs":{"total":96636764160,"available":96580161536,"internally_reserved":0,"allocated":1933312,"data_stored":1213449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:42.325 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:42.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.561+0000 7ff2e6a00640 1 Processor -- start 2024-04-25T10:12:42.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.561+0000 7ff2e6a00640 1 -- start start 2024-04-25T10:12:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.563+0000 7ff2e6a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff2e015e520 0x7ff2e015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.563+0000 7ff2e6a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e00ae590 con 0x7ff2e015f040 2024-04-25T10:12:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.563+0000 7ff2e6a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e00ae870 con 0x7ff2e0070ff0 2024-04-25T10:12:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.563+0000 7ff2e6a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e00aee30 con 0x7ff2e015e520 2024-04-25T10:12:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.563+0000 7ff2e4c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff2e015f040 0x7ff2e00adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:38220/0 (socket says 172.21.15.165:38220) 2024-04-25T10:12:42.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.563+0000 7ff2e4c00640 1 -- 172.21.15.165:0/2644670105 learned_addr learned my addr 172.21.15.165:0/2644670105 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:42.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2644670105 >> v2:172.21.15.172:3300/0 conn(0x7ff2e015e520 0x7ff2e015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 156612985 0 0) 0x7ff2e00ae590 con 0x7ff2e015f040 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2c8003520 con 0x7ff2e015f040 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3952575685 0 0) 0x7ff2c8003520 con 0x7ff2e015f040 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 >> v2:172.21.15.172:3300/0 conn(0x7ff2e015e520 msgr2=0x7ff2e015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 --2- 172.21.15.165:0/2644670105 >> v2:172.21.15.172:3300/0 conn(0x7ff2e015e520 0x7ff2e015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 >> v1:172.21.15.172:6789/0 conn(0x7ff2e0070ff0 legacy=0x7ff2e006d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00aeb50 con 0x7ff2e015f040 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff2cc001670 con 0x7ff2e015f040 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 >> v1:172.21.15.165:6789/0 conn(0x7ff2e015f040 legacy=0x7ff2e00adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2c8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2c8004800 0x7ff2c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2c80051c0 0x7ff2c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00aee30 con 0x7ff2c80039b0 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c800c060 con 0x7ff2c80051c0 2024-04-25T10:12:42.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c800c260 con 0x7ff2c8004800 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.564+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2c8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e4c00640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2c8004800 0x7ff2c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e5600640 1 -- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2c8004800 msgr2=0x7ff2c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2c8004800 0x7ff2c8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e5600640 1 -- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2c80051c0 msgr2=0x7ff2c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2c80051c0 0x7ff2c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e5600640 1 -- 172.21.15.165:0/2644670105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c8003520 con 0x7ff2c80039b0 2024-04-25T10:12:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.565+0000 7ff2e6000640 1 -- 172.21.15.165:0/2644670105 reap_dead start 2024-04-25T10:12:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.566+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2c8003dd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009ac0 tx=0x7ff2c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8df0d34344eb1d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.566+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4036070 con 0x7ff2c80039b0 2024-04-25T10:12:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.566+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4031400 con 0x7ff2c80039b0 2024-04-25T10:12:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.566+0000 7ff2dfe00640 1 -- 172.21.15.165:0/2644670105 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4009690 con 0x7ff2c80039b0 2024-04-25T10:12:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2644670105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 msgr2=0x7ff2c8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2c8003dd0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009ac0 tx=0x7ff2c8005830 comp rx=0 tx=0).stop 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2644670105 shutdown_connections 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2644670105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2c8003dd0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2644670105 >> 172.21.15.165:0/2644670105 conn(0x7ff2e006abe0 msgr2=0x7ff2e006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2644670105 shutdown_connections 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.567+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2644670105 wait complete. 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.569+0000 7ff2e6a00640 1 Processor -- start 2024-04-25T10:12:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.569+0000 7ff2e6a00640 1 -- start start 2024-04-25T10:12:42.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2e013be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e00b3660 0x7ff2e00b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e6a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e00cd590 0x7ff2e00cda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015f650 con 0x7ff2c80039b0 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015f4d0 con 0x7ff2e00b3660 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e6a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015e900 con 0x7ff2e00cd590 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2e013be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2e013be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55728/0 (socket says 172.21.15.165:55728) 2024-04-25T10:12:42.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.570+0000 7ff2e5600640 1 -- 172.21.15.165:0/2693943995 learned_addr learned my addr 172.21.15.165:0/2693943995 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e6000640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e00cd590 0x7ff2e00cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e5600640 1 -- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e00cd590 msgr2=0x7ff2e00cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e4c00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e00b3660 0x7ff2e00b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e00cd590 0x7ff2e00cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e5600640 1 -- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e00b3660 msgr2=0x7ff2e00b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e00b3660 0x7ff2e00b3b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e5600640 1 -- 172.21.15.165:0/2693943995 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e00cbb30 con 0x7ff2c80039b0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e5600640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2e013be70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009be0 tx=0x7ff2d4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d403f020 con 0x7ff2c80039b0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2d4037040 con 0x7ff2c80039b0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4036070 con 0x7ff2c80039b0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2e00cbe10 con 0x7ff2c80039b0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.571+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e00cc3d0 con 0x7ff2c80039b0 2024-04-25T10:12:42.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.573+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e0073990 con 0x7ff2c80039b0 2024-04-25T10:12:42.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.574+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff2d4042020 con 0x7ff2c80039b0 2024-04-25T10:12:42.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.574+0000 7ff2dd600640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2c006b4d0 0x7ff2c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:42.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.575+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff2d40aa3c0 con 0x7ff2c80039b0 2024-04-25T10:12:42.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.575+0000 7ff2e4c00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2c006b4d0 0x7ff2c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:42.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.575+0000 7ff2e4c00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2c006b4d0 0x7ff2c006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00b3a40 tx=0x7ff2cc001470 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:42.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.579+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff2d4049030 con 0x7ff2c80039b0 2024-04-25T10:12:42.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.772+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff2e0071d30 con 0x7ff2c006b4d0 2024-04-25T10:12:42.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.783+0000 7ff2dd600640 1 -- 172.21.15.165:0/2693943995 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7ff2e0071d30 con 0x7ff2c006b4d0 2024-04-25T10:12:42.791 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:42.792 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2c006b4d0 msgr2=0x7ff2c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2c006b4d0 0x7ff2c006d990 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7ff2e00b3a40 tx=0x7ff2cc001470 comp rx=0 tx=0).stop 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 msgr2=0x7ff2e013be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2e013be70 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7ff2d4009be0 tx=0x7ff2d4004730 comp rx=0 tx=0).stop 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 shutdown_connections 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2c006b4d0 0x7ff2c006d990 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff2e00cd590 0x7ff2e00cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff2e00b3660 0x7ff2e00b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 --2- 172.21.15.165:0/2693943995 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff2c80039b0 0x7ff2e013be70 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 >> 172.21.15.165:0/2693943995 conn(0x7ff2e006abe0 msgr2=0x7ff2e00af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:42.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.789+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 shutdown_connections 2024-04-25T10:12:42.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:42.790+0000 7ff2e6a00640 1 -- 172.21.15.165:0/2693943995 wait complete. 2024-04-25T10:12:42.812 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-04-25T10:12:42.345456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430804,"kb_used_data":11976,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754543916,"statfs":{"total":773094113280,"available":772652969984,"internally_reserved":0,"allocated":12263424,"data_stored":6334412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":428723762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660408+0000","last_change":"2024-04-25T10:12:37.440175+0000","last_active":"2024-04-25T10:12:38.660408+0000","last_peered":"2024-04-25T10:12:38.660408+0000","last_clean":"2024-04-25T10:12:38.660408+0000","last_became_active":"2024-04-25T10:12:37.437927+0000","last_became_peered":"2024-04-25T10:12:37.437927+0000","last_unstale":"2024-04-25T10:12:38.660408+0000","last_undegraded":"2024-04-25T10:12:38.660408+0000","last_fullsized":"2024-04-25T10:12:38.660408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:47.423921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421866+0000","last_change":"2024-04-25T10:12:37.442081+0000","last_active":"2024-04-25T10:12:38.421866+0000","last_peered":"2024-04-25T10:12:38.421866+0000","last_clean":"2024-04-25T10:12:38.421866+0000","last_became_active":"2024-04-25T10:12:37.440679+0000","last_became_peered":"2024-04-25T10:12:37.440679+0000","last_unstale":"2024-04-25T10:12:38.421866+0000","last_undegraded":"2024-04-25T10:12:38.421866+0000","last_fullsized":"2024-04-25T10:12:38.421866+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:43.163133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660454+0000","last_change":"2024-04-25T10:12:37.443092+0000","last_active":"2024-04-25T10:12:38.660454+0000","last_peered":"2024-04-25T10:12:38.660454+0000","last_clean":"2024-04-25T10:12:38.660454+0000","last_became_active":"2024-04-25T10:12:37.442779+0000","last_became_peered":"2024-04-25T10:12:37.442779+0000","last_unstale":"2024-04-25T10:12:38.660454+0000","last_undegraded":"2024-04-25T10:12:38.660454+0000","last_fullsized":"2024-04-25T10:12:38.660454+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:43.318742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414964+0000","last_change":"2024-04-25T10:12:37.430317+0000","last_active":"2024-04-25T10:12:38.414964+0000","last_peered":"2024-04-25T10:12:38.414964+0000","last_clean":"2024-04-25T10:12:38.414964+0000","last_became_active":"2024-04-25T10:12:37.430009+0000","last_became_peered":"2024-04-25T10:12:37.430009+0000","last_unstale":"2024-04-25T10:12:38.414964+0000","last_undegraded":"2024-04-25T10:12:38.414964+0000","last_fullsized":"2024-04-25T10:12:38.414964+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:22.585606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657765+0000","last_change":"2024-04-25T10:12:37.443661+0000","last_active":"2024-04-25T10:12:38.657765+0000","last_peered":"2024-04-25T10:12:38.657765+0000","last_clean":"2024-04-25T10:12:38.657765+0000","last_became_active":"2024-04-25T10:12:37.443277+0000","last_became_peered":"2024-04-25T10:12:37.443277+0000","last_unstale":"2024-04-25T10:12:38.657765+0000","last_undegraded":"2024-04-25T10:12:38.657765+0000","last_fullsized":"2024-04-25T10:12:38.657765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421328+0000","last_change":"2024-04-25T10:12:37.446343+0000","last_active":"2024-04-25T10:12:38.421328+0000","last_peered":"2024-04-25T10:12:38.421328+0000","last_clean":"2024-04-25T10:12:38.421328+0000","last_became_active":"2024-04-25T10:12:37.444637+0000","last_became_peered":"2024-04-25T10:12:37.444637+0000","last_unstale":"2024-04-25T10:12:38.421328+0000","last_undegraded":"2024-04-25T10:12:38.421328+0000","last_fullsized":"2024-04-25T10:12:38.421328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418202+0000","last_change":"2024-04-25T10:12:37.443934+0000","last_active":"2024-04-25T10:12:38.418202+0000","last_peered":"2024-04-25T10:12:38.418202+0000","last_clean":"2024-04-25T10:12:38.418202+0000","last_became_active":"2024-04-25T10:12:37.443387+0000","last_became_peered":"2024-04-25T10:12:37.443387+0000","last_unstale":"2024-04-25T10:12:38.418202+0000","last_undegraded":"2024-04-25T10:12:38.418202+0000","last_fullsized":"2024-04-25T10:12:38.418202+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:56.964062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658734+0000","last_change":"2024-04-25T10:12:37.434890+0000","last_active":"2024-04-25T10:12:38.658734+0000","last_peered":"2024-04-25T10:12:38.658734+0000","last_clean":"2024-04-25T10:12:38.658734+0000","last_became_active":"2024-04-25T10:12:37.434607+0000","last_became_peered":"2024-04-25T10:12:37.434607+0000","last_unstale":"2024-04-25T10:12:38.658734+0000","last_undegraded":"2024-04-25T10:12:38.658734+0000","last_fullsized":"2024-04-25T10:12:38.658734+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:50.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421703+0000","last_change":"2024-04-25T10:12:37.440646+0000","last_active":"2024-04-25T10:12:38.421703+0000","last_peered":"2024-04-25T10:12:38.421703+0000","last_clean":"2024-04-25T10:12:38.421703+0000","last_became_active":"2024-04-25T10:12:37.440279+0000","last_became_peered":"2024-04-25T10:12:37.440279+0000","last_unstale":"2024-04-25T10:12:38.421703+0000","last_undegraded":"2024-04-25T10:12:38.421703+0000","last_fullsized":"2024-04-25T10:12:38.421703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658834+0000","last_change":"2024-04-25T10:12:37.451524+0000","last_active":"2024-04-25T10:12:38.658834+0000","last_peered":"2024-04-25T10:12:38.658834+0000","last_clean":"2024-04-25T10:12:38.658834+0000","last_became_active":"2024-04-25T10:12:37.450715+0000","last_became_peered":"2024-04-25T10:12:37.450715+0000","last_unstale":"2024-04-25T10:12:38.658834+0000","last_undegraded":"2024-04-25T10:12:38.658834+0000","last_fullsized":"2024-04-25T10:12:38.658834+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658039+0000","last_change":"2024-04-25T10:12:37.446411+0000","last_active":"2024-04-25T10:12:38.658039+0000","last_peered":"2024-04-25T10:12:38.658039+0000","last_clean":"2024-04-25T10:12:38.658039+0000","last_became_active":"2024-04-25T10:12:37.445773+0000","last_became_peered":"2024-04-25T10:12:37.445773+0000","last_unstale":"2024-04-25T10:12:38.658039+0000","last_undegraded":"2024-04-25T10:12:38.658039+0000","last_fullsized":"2024-04-25T10:12:38.658039+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:21.571051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418659+0000","last_change":"2024-04-25T10:12:37.449685+0000","last_active":"2024-04-25T10:12:38.418659+0000","last_peered":"2024-04-25T10:12:38.418659+0000","last_clean":"2024-04-25T10:12:38.418659+0000","last_became_active":"2024-04-25T10:12:37.448670+0000","last_became_peered":"2024-04-25T10:12:37.448670+0000","last_unstale":"2024-04-25T10:12:38.418659+0000","last_undegraded":"2024-04-25T10:12:38.418659+0000","last_fullsized":"2024-04-25T10:12:38.418659+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.413874+0000","last_change":"2024-04-25T10:12:37.441928+0000","last_active":"2024-04-25T10:12:38.413874+0000","last_peered":"2024-04-25T10:12:38.413874+0000","last_clean":"2024-04-25T10:12:38.413874+0000","last_became_active":"2024-04-25T10:12:37.441644+0000","last_became_peered":"2024-04-25T10:12:37.441644+0000","last_unstale":"2024-04-25T10:12:38.413874+0000","last_undegraded":"2024-04-25T10:12:38.413874+0000","last_fullsized":"2024-04-25T10:12:38.413874+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:12.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477504+0000","last_change":"2024-04-25T10:12:37.434637+0000","last_active":"2024-04-25T10:12:38.477504+0000","last_peered":"2024-04-25T10:12:38.477504+0000","last_clean":"2024-04-25T10:12:38.477504+0000","last_became_active":"2024-04-25T10:12:37.434374+0000","last_became_peered":"2024-04-25T10:12:37.434374+0000","last_unstale":"2024-04-25T10:12:38.477504+0000","last_undegraded":"2024-04-25T10:12:38.477504+0000","last_fullsized":"2024-04-25T10:12:38.477504+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:12.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477546+0000","last_change":"2024-04-25T10:12:37.448340+0000","last_active":"2024-04-25T10:12:38.477546+0000","last_peered":"2024-04-25T10:12:38.477546+0000","last_clean":"2024-04-25T10:12:38.477546+0000","last_became_active":"2024-04-25T10:12:37.446052+0000","last_became_peered":"2024-04-25T10:12:37.446052+0000","last_unstale":"2024-04-25T10:12:38.477546+0000","last_undegraded":"2024-04-25T10:12:38.477546+0000","last_fullsized":"2024-04-25T10:12:38.477546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:19.357372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477573+0000","last_change":"2024-04-25T10:12:37.424865+0000","last_active":"2024-04-25T10:12:38.477573+0000","last_peered":"2024-04-25T10:12:38.477573+0000","last_clean":"2024-04-25T10:12:38.477573+0000","last_became_active":"2024-04-25T10:12:37.424580+0000","last_became_peered":"2024-04-25T10:12:37.424580+0000","last_unstale":"2024-04-25T10:12:38.477573+0000","last_undegraded":"2024-04-25T10:12:38.477573+0000","last_fullsized":"2024-04-25T10:12:38.477573+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:34.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660749+0000","last_change":"2024-04-25T10:12:37.440407+0000","last_active":"2024-04-25T10:12:38.660749+0000","last_peered":"2024-04-25T10:12:38.660749+0000","last_clean":"2024-04-25T10:12:38.660749+0000","last_became_active":"2024-04-25T10:12:37.439466+0000","last_became_peered":"2024-04-25T10:12:37.439466+0000","last_unstale":"2024-04-25T10:12:38.660749+0000","last_undegraded":"2024-04-25T10:12:38.660749+0000","last_fullsized":"2024-04-25T10:12:38.660749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:16.348692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421858+0000","last_change":"2024-04-25T10:12:37.444108+0000","last_active":"2024-04-25T10:12:38.421858+0000","last_peered":"2024-04-25T10:12:38.421858+0000","last_clean":"2024-04-25T10:12:38.421858+0000","last_became_active":"2024-04-25T10:12:37.443514+0000","last_became_peered":"2024-04-25T10:12:37.443514+0000","last_unstale":"2024-04-25T10:12:38.421858+0000","last_undegraded":"2024-04-25T10:12:38.421858+0000","last_fullsized":"2024-04-25T10:12:38.421858+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660185+0000","last_change":"2024-04-25T10:12:37.451809+0000","last_active":"2024-04-25T10:12:38.660185+0000","last_peered":"2024-04-25T10:12:38.660185+0000","last_clean":"2024-04-25T10:12:38.660185+0000","last_became_active":"2024-04-25T10:12:37.449840+0000","last_became_peered":"2024-04-25T10:12:37.449840+0000","last_unstale":"2024-04-25T10:12:38.660185+0000","last_undegraded":"2024-04-25T10:12:38.660185+0000","last_fullsized":"2024-04-25T10:12:38.660185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:00.348275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419395+0000","last_change":"2024-04-25T10:12:37.425902+0000","last_active":"2024-04-25T10:12:38.419395+0000","last_peered":"2024-04-25T10:12:38.419395+0000","last_clean":"2024-04-25T10:12:38.419395+0000","last_became_active":"2024-04-25T10:12:37.425600+0000","last_became_peered":"2024-04-25T10:12:37.425600+0000","last_unstale":"2024-04-25T10:12:38.419395+0000","last_undegraded":"2024-04-25T10:12:38.419395+0000","last_fullsized":"2024-04-25T10:12:38.419395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656760+0000","last_change":"2024-04-25T10:12:37.432765+0000","last_active":"2024-04-25T10:12:38.656760+0000","last_peered":"2024-04-25T10:12:38.656760+0000","last_clean":"2024-04-25T10:12:38.656760+0000","last_became_active":"2024-04-25T10:12:37.432505+0000","last_became_peered":"2024-04-25T10:12:37.432505+0000","last_unstale":"2024-04-25T10:12:38.656760+0000","last_undegraded":"2024-04-25T10:12:38.656760+0000","last_fullsized":"2024-04-25T10:12:38.656760+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:00.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660318+0000","last_change":"2024-04-25T10:12:37.443138+0000","last_active":"2024-04-25T10:12:38.660318+0000","last_peered":"2024-04-25T10:12:38.660318+0000","last_clean":"2024-04-25T10:12:38.660318+0000","last_became_active":"2024-04-25T10:12:37.442308+0000","last_became_peered":"2024-04-25T10:12:37.442308+0000","last_unstale":"2024-04-25T10:12:38.660318+0000","last_undegraded":"2024-04-25T10:12:38.660318+0000","last_fullsized":"2024-04-25T10:12:38.660318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:43.163133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659462+0000","last_change":"2024-04-25T10:12:37.444200+0000","last_active":"2024-04-25T10:12:38.659462+0000","last_peered":"2024-04-25T10:12:38.659462+0000","last_clean":"2024-04-25T10:12:38.659462+0000","last_became_active":"2024-04-25T10:12:37.443940+0000","last_became_peered":"2024-04-25T10:12:37.443940+0000","last_unstale":"2024-04-25T10:12:38.659462+0000","last_undegraded":"2024-04-25T10:12:38.659462+0000","last_fullsized":"2024-04-25T10:12:38.659462+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.836901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659970+0000","last_change":"2024-04-25T10:12:37.448155+0000","last_active":"2024-04-25T10:12:38.659970+0000","last_peered":"2024-04-25T10:12:38.659970+0000","last_clean":"2024-04-25T10:12:38.659970+0000","last_became_active":"2024-04-25T10:12:37.447522+0000","last_became_peered":"2024-04-25T10:12:37.447522+0000","last_unstale":"2024-04-25T10:12:38.659970+0000","last_undegraded":"2024-04-25T10:12:38.659970+0000","last_fullsized":"2024-04-25T10:12:38.659970+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419980+0000","last_change":"2024-04-25T10:12:37.446108+0000","last_active":"2024-04-25T10:12:38.419980+0000","last_peered":"2024-04-25T10:12:38.419980+0000","last_clean":"2024-04-25T10:12:38.419980+0000","last_became_active":"2024-04-25T10:12:37.445831+0000","last_became_peered":"2024-04-25T10:12:37.445831+0000","last_unstale":"2024-04-25T10:12:38.419980+0000","last_undegraded":"2024-04-25T10:12:38.419980+0000","last_fullsized":"2024-04-25T10:12:38.419980+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418914+0000","last_change":"2024-04-25T10:12:37.441013+0000","last_active":"2024-04-25T10:12:38.418914+0000","last_peered":"2024-04-25T10:12:38.418914+0000","last_clean":"2024-04-25T10:12:38.418914+0000","last_became_active":"2024-04-25T10:12:37.440725+0000","last_became_peered":"2024-04-25T10:12:37.440725+0000","last_unstale":"2024-04-25T10:12:38.418914+0000","last_undegraded":"2024-04-25T10:12:38.418914+0000","last_fullsized":"2024-04-25T10:12:38.418914+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658244+0000","last_change":"2024-04-25T10:12:37.438780+0000","last_active":"2024-04-25T10:12:38.658244+0000","last_peered":"2024-04-25T10:12:38.658244+0000","last_clean":"2024-04-25T10:12:38.658244+0000","last_became_active":"2024-04-25T10:12:37.438484+0000","last_became_peered":"2024-04-25T10:12:37.438484+0000","last_unstale":"2024-04-25T10:12:38.658244+0000","last_undegraded":"2024-04-25T10:12:38.658244+0000","last_fullsized":"2024-04-25T10:12:38.658244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:54.229611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420777+0000","last_change":"2024-04-25T10:12:37.440617+0000","last_active":"2024-04-25T10:12:38.420777+0000","last_peered":"2024-04-25T10:12:38.420777+0000","last_clean":"2024-04-25T10:12:38.420777+0000","last_became_active":"2024-04-25T10:12:37.440310+0000","last_became_peered":"2024-04-25T10:12:37.440310+0000","last_unstale":"2024-04-25T10:12:38.420777+0000","last_undegraded":"2024-04-25T10:12:38.420777+0000","last_fullsized":"2024-04-25T10:12:38.420777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656959+0000","last_change":"2024-04-25T10:12:37.426018+0000","last_active":"2024-04-25T10:12:38.656959+0000","last_peered":"2024-04-25T10:12:38.656959+0000","last_clean":"2024-04-25T10:12:38.656959+0000","last_became_active":"2024-04-25T10:12:37.425637+0000","last_became_peered":"2024-04-25T10:12:37.425637+0000","last_unstale":"2024-04-25T10:12:38.656959+0000","last_undegraded":"2024-04-25T10:12:38.656959+0000","last_fullsized":"2024-04-25T10:12:38.656959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:00.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418921+0000","last_change":"2024-04-25T10:12:37.443856+0000","last_active":"2024-04-25T10:12:38.418921+0000","last_peered":"2024-04-25T10:12:38.418921+0000","last_clean":"2024-04-25T10:12:38.418921+0000","last_became_active":"2024-04-25T10:12:37.443421+0000","last_became_peered":"2024-04-25T10:12:37.443421+0000","last_unstale":"2024-04-25T10:12:38.418921+0000","last_undegraded":"2024-04-25T10:12:38.418921+0000","last_fullsized":"2024-04-25T10:12:38.418921+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418651+0000","last_change":"2024-04-25T10:12:37.433545+0000","last_active":"2024-04-25T10:12:38.418651+0000","last_peered":"2024-04-25T10:12:38.418651+0000","last_clean":"2024-04-25T10:12:38.418651+0000","last_became_active":"2024-04-25T10:12:37.433271+0000","last_became_peered":"2024-04-25T10:12:37.433271+0000","last_unstale":"2024-04-25T10:12:38.418651+0000","last_undegraded":"2024-04-25T10:12:38.418651+0000","last_fullsized":"2024-04-25T10:12:38.418651+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656933+0000","last_change":"2024-04-25T10:12:37.426624+0000","last_active":"2024-04-25T10:12:38.656933+0000","last_peered":"2024-04-25T10:12:38.656933+0000","last_clean":"2024-04-25T10:12:38.656933+0000","last_became_active":"2024-04-25T10:12:37.425873+0000","last_became_peered":"2024-04-25T10:12:37.425873+0000","last_unstale":"2024-04-25T10:12:38.656933+0000","last_undegraded":"2024-04-25T10:12:38.656933+0000","last_fullsized":"2024-04-25T10:12:38.656933+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658673+0000","last_change":"2024-04-25T10:12:37.451368+0000","last_active":"2024-04-25T10:12:38.658673+0000","last_peered":"2024-04-25T10:12:38.658673+0000","last_clean":"2024-04-25T10:12:38.658673+0000","last_became_active":"2024-04-25T10:12:37.451027+0000","last_became_peered":"2024-04-25T10:12:37.451027+0000","last_unstale":"2024-04-25T10:12:38.658673+0000","last_undegraded":"2024-04-25T10:12:38.658673+0000","last_fullsized":"2024-04-25T10:12:38.658673+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:08.472720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414586+0000","last_change":"2024-04-25T10:12:36.428651+0000","last_active":"2024-04-25T10:12:38.414586+0000","last_peered":"2024-04-25T10:12:38.414586+0000","last_clean":"2024-04-25T10:12:38.414586+0000","last_became_active":"2024-04-25T10:12:36.428062+0000","last_became_peered":"2024-04-25T10:12:36.428062+0000","last_unstale":"2024-04-25T10:12:38.414586+0000","last_undegraded":"2024-04-25T10:12:38.414586+0000","last_fullsized":"2024-04-25T10:12:38.414586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:42.723368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659663+0000","last_change":"2024-04-25T10:12:37.442206+0000","last_active":"2024-04-25T10:12:38.659663+0000","last_peered":"2024-04-25T10:12:38.659663+0000","last_clean":"2024-04-25T10:12:38.659663+0000","last_became_active":"2024-04-25T10:12:37.441474+0000","last_became_peered":"2024-04-25T10:12:37.441474+0000","last_unstale":"2024-04-25T10:12:38.659663+0000","last_undegraded":"2024-04-25T10:12:38.659663+0000","last_fullsized":"2024-04-25T10:12:38.659663+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:09.781858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659154+0000","last_change":"2024-04-25T10:12:36.432356+0000","last_active":"2024-04-25T10:12:38.659154+0000","last_peered":"2024-04-25T10:12:38.659154+0000","last_clean":"2024-04-25T10:12:38.659154+0000","last_became_active":"2024-04-25T10:12:36.432086+0000","last_became_peered":"2024-04-25T10:12:36.432086+0000","last_unstale":"2024-04-25T10:12:38.659154+0000","last_undegraded":"2024-04-25T10:12:38.659154+0000","last_fullsized":"2024-04-25T10:12:38.659154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:46.923030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419369+0000","last_change":"2024-04-25T10:12:37.447367+0000","last_active":"2024-04-25T10:12:38.419369+0000","last_peered":"2024-04-25T10:12:38.419369+0000","last_clean":"2024-04-25T10:12:38.419369+0000","last_became_active":"2024-04-25T10:12:37.447079+0000","last_became_peered":"2024-04-25T10:12:37.447079+0000","last_unstale":"2024-04-25T10:12:38.419369+0000","last_undegraded":"2024-04-25T10:12:38.419369+0000","last_fullsized":"2024-04-25T10:12:38.419369+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418183+0000","last_change":"2024-04-25T10:12:36.437597+0000","last_active":"2024-04-25T10:12:38.418183+0000","last_peered":"2024-04-25T10:12:38.418183+0000","last_clean":"2024-04-25T10:12:38.418183+0000","last_became_active":"2024-04-25T10:12:36.437214+0000","last_became_peered":"2024-04-25T10:12:36.437214+0000","last_unstale":"2024-04-25T10:12:38.418183+0000","last_undegraded":"2024-04-25T10:12:38.418183+0000","last_fullsized":"2024-04-25T10:12:38.418183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:54.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657727+0000","last_change":"2024-04-25T10:12:37.433634+0000","last_active":"2024-04-25T10:12:38.657727+0000","last_peered":"2024-04-25T10:12:38.657727+0000","last_clean":"2024-04-25T10:12:38.657727+0000","last_became_active":"2024-04-25T10:12:37.433334+0000","last_became_peered":"2024-04-25T10:12:37.433334+0000","last_unstale":"2024-04-25T10:12:38.657727+0000","last_undegraded":"2024-04-25T10:12:38.657727+0000","last_fullsized":"2024-04-25T10:12:38.657727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.027233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658433+0000","last_change":"2024-04-25T10:12:36.430378+0000","last_active":"2024-04-25T10:12:38.658433+0000","last_peered":"2024-04-25T10:12:38.658433+0000","last_clean":"2024-04-25T10:12:38.658433+0000","last_became_active":"2024-04-25T10:12:36.430043+0000","last_became_peered":"2024-04-25T10:12:36.430043+0000","last_unstale":"2024-04-25T10:12:38.658433+0000","last_undegraded":"2024-04-25T10:12:38.658433+0000","last_fullsized":"2024-04-25T10:12:38.658433+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:14.731959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414534+0000","last_change":"2024-04-25T10:12:37.438453+0000","last_active":"2024-04-25T10:12:38.414534+0000","last_peered":"2024-04-25T10:12:38.414534+0000","last_clean":"2024-04-25T10:12:38.414534+0000","last_became_active":"2024-04-25T10:12:37.438179+0000","last_became_peered":"2024-04-25T10:12:37.438179+0000","last_unstale":"2024-04-25T10:12:38.414534+0000","last_undegraded":"2024-04-25T10:12:38.414534+0000","last_fullsized":"2024-04-25T10:12:38.414534+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:56.927046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477763+0000","last_change":"2024-04-25T10:12:36.432661+0000","last_active":"2024-04-25T10:12:38.477763+0000","last_peered":"2024-04-25T10:12:38.477763+0000","last_clean":"2024-04-25T10:12:38.477763+0000","last_became_active":"2024-04-25T10:12:36.432286+0000","last_became_peered":"2024-04-25T10:12:36.432286+0000","last_unstale":"2024-04-25T10:12:38.477763+0000","last_undegraded":"2024-04-25T10:12:38.477763+0000","last_fullsized":"2024-04-25T10:12:38.477763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:35.057626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660604+0000","last_change":"2024-04-25T10:12:37.450831+0000","last_active":"2024-04-25T10:12:38.660604+0000","last_peered":"2024-04-25T10:12:38.660604+0000","last_clean":"2024-04-25T10:12:38.660604+0000","last_became_active":"2024-04-25T10:12:37.450537+0000","last_became_peered":"2024-04-25T10:12:37.450537+0000","last_unstale":"2024-04-25T10:12:38.660604+0000","last_undegraded":"2024-04-25T10:12:38.660604+0000","last_fullsized":"2024-04-25T10:12:38.660604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:13.308948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657298+0000","last_change":"2024-04-25T10:12:36.437498+0000","last_active":"2024-04-25T10:12:38.657298+0000","last_peered":"2024-04-25T10:12:38.657298+0000","last_clean":"2024-04-25T10:12:38.657298+0000","last_became_active":"2024-04-25T10:12:36.437164+0000","last_became_peered":"2024-04-25T10:12:36.437164+0000","last_unstale":"2024-04-25T10:12:38.657298+0000","last_undegraded":"2024-04-25T10:12:38.657298+0000","last_fullsized":"2024-04-25T10:12:38.657298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:06.982259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.422167+0000","last_change":"2024-04-25T10:12:37.442744+0000","last_active":"2024-04-25T10:12:38.422167+0000","last_peered":"2024-04-25T10:12:38.422167+0000","last_clean":"2024-04-25T10:12:38.422167+0000","last_became_active":"2024-04-25T10:12:37.441161+0000","last_became_peered":"2024-04-25T10:12:37.441161+0000","last_unstale":"2024-04-25T10:12:38.422167+0000","last_undegraded":"2024-04-25T10:12:38.422167+0000","last_fullsized":"2024-04-25T10:12:38.422167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:13.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417597+0000","last_change":"2024-04-25T10:12:36.436140+0000","last_active":"2024-04-25T10:12:38.417597+0000","last_peered":"2024-04-25T10:12:38.417597+0000","last_clean":"2024-04-25T10:12:38.417597+0000","last_became_active":"2024-04-25T10:12:36.435843+0000","last_became_peered":"2024-04-25T10:12:36.435843+0000","last_unstale":"2024-04-25T10:12:38.417597+0000","last_undegraded":"2024-04-25T10:12:38.417597+0000","last_fullsized":"2024-04-25T10:12:38.417597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414480+0000","last_change":"2024-04-25T10:12:37.436207+0000","last_active":"2024-04-25T10:12:38.414480+0000","last_peered":"2024-04-25T10:12:38.414480+0000","last_clean":"2024-04-25T10:12:38.414480+0000","last_became_active":"2024-04-25T10:12:37.435907+0000","last_became_peered":"2024-04-25T10:12:37.435907+0000","last_unstale":"2024-04-25T10:12:38.414480+0000","last_undegraded":"2024-04-25T10:12:38.414480+0000","last_fullsized":"2024-04-25T10:12:38.414480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:34.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414951+0000","last_change":"2024-04-25T10:12:36.433709+0000","last_active":"2024-04-25T10:12:38.414951+0000","last_peered":"2024-04-25T10:12:38.414951+0000","last_clean":"2024-04-25T10:12:38.414951+0000","last_became_active":"2024-04-25T10:12:36.433440+0000","last_became_peered":"2024-04-25T10:12:36.433440+0000","last_unstale":"2024-04-25T10:12:38.414951+0000","last_undegraded":"2024-04-25T10:12:38.414951+0000","last_fullsized":"2024-04-25T10:12:38.414951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:35.057626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658349+0000","last_change":"2024-04-25T10:12:37.439841+0000","last_active":"2024-04-25T10:12:38.658349+0000","last_peered":"2024-04-25T10:12:38.658349+0000","last_clean":"2024-04-25T10:12:38.658349+0000","last_became_active":"2024-04-25T10:12:37.439287+0000","last_became_peered":"2024-04-25T10:12:37.439287+0000","last_unstale":"2024-04-25T10:12:38.658349+0000","last_undegraded":"2024-04-25T10:12:38.658349+0000","last_fullsized":"2024-04-25T10:12:38.658349+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:20.343948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420040+0000","last_change":"2024-04-25T10:12:36.430969+0000","last_active":"2024-04-25T10:12:38.420040+0000","last_peered":"2024-04-25T10:12:38.420040+0000","last_clean":"2024-04-25T10:12:38.420040+0000","last_became_active":"2024-04-25T10:12:36.430681+0000","last_became_peered":"2024-04-25T10:12:36.430681+0000","last_unstale":"2024-04-25T10:12:38.420040+0000","last_undegraded":"2024-04-25T10:12:38.420040+0000","last_fullsized":"2024-04-25T10:12:38.420040+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:48.057318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657867+0000","last_change":"2024-04-25T10:12:37.435317+0000","last_active":"2024-04-25T10:12:38.657867+0000","last_peered":"2024-04-25T10:12:38.657867+0000","last_clean":"2024-04-25T10:12:38.657867+0000","last_became_active":"2024-04-25T10:12:37.435007+0000","last_became_peered":"2024-04-25T10:12:37.435007+0000","last_unstale":"2024-04-25T10:12:38.657867+0000","last_undegraded":"2024-04-25T10:12:38.657867+0000","last_fullsized":"2024-04-25T10:12:38.657867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659574+0000","last_change":"2024-04-25T10:12:36.435089+0000","last_active":"2024-04-25T10:12:38.659574+0000","last_peered":"2024-04-25T10:12:38.659574+0000","last_clean":"2024-04-25T10:12:38.659574+0000","last_became_active":"2024-04-25T10:12:36.434759+0000","last_became_peered":"2024-04-25T10:12:36.434759+0000","last_unstale":"2024-04-25T10:12:38.659574+0000","last_undegraded":"2024-04-25T10:12:38.659574+0000","last_fullsized":"2024-04-25T10:12:38.659574+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:37.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660287+0000","last_change":"2024-04-25T10:12:37.441207+0000","last_active":"2024-04-25T10:12:38.660287+0000","last_peered":"2024-04-25T10:12:38.660287+0000","last_clean":"2024-04-25T10:12:38.660287+0000","last_became_active":"2024-04-25T10:12:37.440925+0000","last_became_peered":"2024-04-25T10:12:37.440925+0000","last_unstale":"2024-04-25T10:12:38.660287+0000","last_undegraded":"2024-04-25T10:12:38.660287+0000","last_fullsized":"2024-04-25T10:12:38.660287+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:08.118583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477926+0000","last_change":"2024-04-25T10:12:36.438390+0000","last_active":"2024-04-25T10:12:38.477926+0000","last_peered":"2024-04-25T10:12:38.477926+0000","last_clean":"2024-04-25T10:12:38.477926+0000","last_became_active":"2024-04-25T10:12:36.438100+0000","last_became_peered":"2024-04-25T10:12:36.438100+0000","last_unstale":"2024-04-25T10:12:38.477926+0000","last_undegraded":"2024-04-25T10:12:38.477926+0000","last_fullsized":"2024-04-25T10:12:38.477926+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:30.172798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658108+0000","last_change":"2024-04-25T10:12:37.441135+0000","last_active":"2024-04-25T10:12:38.658108+0000","last_peered":"2024-04-25T10:12:38.658108+0000","last_clean":"2024-04-25T10:12:38.658108+0000","last_became_active":"2024-04-25T10:12:37.440844+0000","last_became_peered":"2024-04-25T10:12:37.440844+0000","last_unstale":"2024-04-25T10:12:38.658108+0000","last_undegraded":"2024-04-25T10:12:38.658108+0000","last_fullsized":"2024-04-25T10:12:38.658108+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420801+0000","last_change":"2024-04-25T10:12:36.434095+0000","last_active":"2024-04-25T10:12:38.420801+0000","last_peered":"2024-04-25T10:12:38.420801+0000","last_clean":"2024-04-25T10:12:38.420801+0000","last_became_active":"2024-04-25T10:12:36.433803+0000","last_became_peered":"2024-04-25T10:12:36.433803+0000","last_unstale":"2024-04-25T10:12:38.420801+0000","last_undegraded":"2024-04-25T10:12:38.420801+0000","last_fullsized":"2024-04-25T10:12:38.420801+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657583+0000","last_change":"2024-04-25T10:12:37.431496+0000","last_active":"2024-04-25T10:12:38.657583+0000","last_peered":"2024-04-25T10:12:38.657583+0000","last_clean":"2024-04-25T10:12:38.657583+0000","last_became_active":"2024-04-25T10:12:37.430520+0000","last_became_peered":"2024-04-25T10:12:37.430520+0000","last_unstale":"2024-04-25T10:12:38.657583+0000","last_undegraded":"2024-04-25T10:12:38.657583+0000","last_fullsized":"2024-04-25T10:12:38.657583+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:44.202217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658293+0000","last_change":"2024-04-25T10:12:36.420615+0000","last_active":"2024-04-25T10:12:38.658293+0000","last_peered":"2024-04-25T10:12:38.658293+0000","last_clean":"2024-04-25T10:12:38.658293+0000","last_became_active":"2024-04-25T10:12:36.420089+0000","last_became_peered":"2024-04-25T10:12:36.420089+0000","last_unstale":"2024-04-25T10:12:38.658293+0000","last_undegraded":"2024-04-25T10:12:38.658293+0000","last_fullsized":"2024-04-25T10:12:38.658293+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:54.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421871+0000","last_change":"2024-04-25T10:12:37.439710+0000","last_active":"2024-04-25T10:12:38.421871+0000","last_peered":"2024-04-25T10:12:38.421871+0000","last_clean":"2024-04-25T10:12:38.421871+0000","last_became_active":"2024-04-25T10:12:37.439422+0000","last_became_peered":"2024-04-25T10:12:37.439422+0000","last_unstale":"2024-04-25T10:12:38.421871+0000","last_undegraded":"2024-04-25T10:12:38.421871+0000","last_fullsized":"2024-04-25T10:12:38.421871+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:28.724635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414588+0000","last_change":"2024-04-25T10:12:36.432744+0000","last_active":"2024-04-25T10:12:38.414588+0000","last_peered":"2024-04-25T10:12:38.414588+0000","last_clean":"2024-04-25T10:12:38.414588+0000","last_became_active":"2024-04-25T10:12:36.431971+0000","last_became_peered":"2024-04-25T10:12:36.431971+0000","last_unstale":"2024-04-25T10:12:38.414588+0000","last_undegraded":"2024-04-25T10:12:38.414588+0000","last_fullsized":"2024-04-25T10:12:38.414588+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659025+0000","last_change":"2024-04-25T10:12:37.425673+0000","last_active":"2024-04-25T10:12:38.659025+0000","last_peered":"2024-04-25T10:12:38.659025+0000","last_clean":"2024-04-25T10:12:38.659025+0000","last_became_active":"2024-04-25T10:12:37.425265+0000","last_became_peered":"2024-04-25T10:12:37.425265+0000","last_unstale":"2024-04-25T10:12:38.659025+0000","last_undegraded":"2024-04-25T10:12:38.659025+0000","last_fullsized":"2024-04-25T10:12:38.659025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:48.545973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659783+0000","last_change":"2024-04-25T10:12:36.429891+0000","last_active":"2024-04-25T10:12:38.659783+0000","last_peered":"2024-04-25T10:12:38.659783+0000","last_clean":"2024-04-25T10:12:38.659783+0000","last_became_active":"2024-04-25T10:12:36.429336+0000","last_became_peered":"2024-04-25T10:12:36.429336+0000","last_unstale":"2024-04-25T10:12:38.659783+0000","last_undegraded":"2024-04-25T10:12:38.659783+0000","last_fullsized":"2024-04-25T10:12:38.659783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:54.333585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660854+0000","last_change":"2024-04-25T10:12:37.440609+0000","last_active":"2024-04-25T10:12:38.660854+0000","last_peered":"2024-04-25T10:12:38.660854+0000","last_clean":"2024-04-25T10:12:38.660854+0000","last_became_active":"2024-04-25T10:12:37.439917+0000","last_became_peered":"2024-04-25T10:12:37.439917+0000","last_unstale":"2024-04-25T10:12:38.660854+0000","last_undegraded":"2024-04-25T10:12:38.660854+0000","last_fullsized":"2024-04-25T10:12:38.660854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:03.858903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":22,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417179+0000","last_change":"2024-04-25T10:12:36.434858+0000","last_active":"2024-04-25T10:12:38.417179+0000","last_peered":"2024-04-25T10:12:38.417179+0000","last_clean":"2024-04-25T10:12:38.417179+0000","last_became_active":"2024-04-25T10:12:36.434571+0000","last_became_peered":"2024-04-25T10:12:36.434571+0000","last_unstale":"2024-04-25T10:12:38.417179+0000","last_undegraded":"2024-04-25T10:12:38.417179+0000","last_fullsized":"2024-04-25T10:12:38.417179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657357+0000","last_change":"2024-04-25T10:12:37.430884+0000","last_active":"2024-04-25T10:12:38.657357+0000","last_peered":"2024-04-25T10:12:38.657357+0000","last_clean":"2024-04-25T10:12:38.657357+0000","last_became_active":"2024-04-25T10:12:37.428118+0000","last_became_peered":"2024-04-25T10:12:37.428118+0000","last_unstale":"2024-04-25T10:12:38.657357+0000","last_undegraded":"2024-04-25T10:12:38.657357+0000","last_fullsized":"2024-04-25T10:12:38.657357+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:36.032449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421873+0000","last_change":"2024-04-25T10:12:36.437073+0000","last_active":"2024-04-25T10:12:38.421873+0000","last_peered":"2024-04-25T10:12:38.421873+0000","last_clean":"2024-04-25T10:12:38.421873+0000","last_became_active":"2024-04-25T10:12:36.436575+0000","last_became_peered":"2024-04-25T10:12:36.436575+0000","last_unstale":"2024-04-25T10:12:38.421873+0000","last_undegraded":"2024-04-25T10:12:38.421873+0000","last_fullsized":"2024-04-25T10:12:38.421873+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.986763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421317+0000","last_change":"2024-04-25T10:12:37.446967+0000","last_active":"2024-04-25T10:12:38.421317+0000","last_peered":"2024-04-25T10:12:38.421317+0000","last_clean":"2024-04-25T10:12:38.421317+0000","last_became_active":"2024-04-25T10:12:37.446664+0000","last_became_peered":"2024-04-25T10:12:37.446664+0000","last_unstale":"2024-04-25T10:12:38.421317+0000","last_undegraded":"2024-04-25T10:12:38.421317+0000","last_fullsized":"2024-04-25T10:12:38.421317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477978+0000","last_change":"2024-04-25T10:12:36.430895+0000","last_active":"2024-04-25T10:12:38.477978+0000","last_peered":"2024-04-25T10:12:38.477978+0000","last_clean":"2024-04-25T10:12:38.477978+0000","last_became_active":"2024-04-25T10:12:36.430565+0000","last_became_peered":"2024-04-25T10:12:36.430565+0000","last_unstale":"2024-04-25T10:12:38.477978+0000","last_undegraded":"2024-04-25T10:12:38.477978+0000","last_fullsized":"2024-04-25T10:12:38.477978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657946+0000","last_change":"2024-04-25T10:12:37.436317+0000","last_active":"2024-04-25T10:12:38.657946+0000","last_peered":"2024-04-25T10:12:38.657946+0000","last_clean":"2024-04-25T10:12:38.657946+0000","last_became_active":"2024-04-25T10:12:37.435976+0000","last_became_peered":"2024-04-25T10:12:37.435976+0000","last_unstale":"2024-04-25T10:12:38.657946+0000","last_undegraded":"2024-04-25T10:12:38.657946+0000","last_fullsized":"2024-04-25T10:12:38.657946+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657682+0000","last_change":"2024-04-25T10:12:36.438473+0000","last_active":"2024-04-25T10:12:38.657682+0000","last_peered":"2024-04-25T10:12:38.657682+0000","last_clean":"2024-04-25T10:12:38.657682+0000","last_became_active":"2024-04-25T10:12:36.436411+0000","last_became_peered":"2024-04-25T10:12:36.436411+0000","last_unstale":"2024-04-25T10:12:38.657682+0000","last_undegraded":"2024-04-25T10:12:38.657682+0000","last_fullsized":"2024-04-25T10:12:38.657682+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:52.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659307+0000","last_change":"2024-04-25T10:12:37.445297+0000","last_active":"2024-04-25T10:12:38.659307+0000","last_peered":"2024-04-25T10:12:38.659307+0000","last_clean":"2024-04-25T10:12:38.659307+0000","last_became_active":"2024-04-25T10:12:37.444851+0000","last_became_peered":"2024-04-25T10:12:37.444851+0000","last_unstale":"2024-04-25T10:12:38.659307+0000","last_undegraded":"2024-04-25T10:12:38.659307+0000","last_fullsized":"2024-04-25T10:12:38.659307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:24.622841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657071+0000","last_change":"2024-04-25T10:12:36.428322+0000","last_active":"2024-04-25T10:12:38.657071+0000","last_peered":"2024-04-25T10:12:38.657071+0000","last_clean":"2024-04-25T10:12:38.657071+0000","last_became_active":"2024-04-25T10:12:36.427541+0000","last_became_peered":"2024-04-25T10:12:36.427541+0000","last_unstale":"2024-04-25T10:12:38.657071+0000","last_undegraded":"2024-04-25T10:12:38.657071+0000","last_fullsized":"2024-04-25T10:12:38.657071+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:26.382466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660381+0000","last_change":"2024-04-25T10:12:37.438627+0000","last_active":"2024-04-25T10:12:38.660381+0000","last_peered":"2024-04-25T10:12:38.660381+0000","last_clean":"2024-04-25T10:12:38.660381+0000","last_became_active":"2024-04-25T10:12:37.438352+0000","last_became_peered":"2024-04-25T10:12:37.438352+0000","last_unstale":"2024-04-25T10:12:38.660381+0000","last_undegraded":"2024-04-25T10:12:38.660381+0000","last_fullsized":"2024-04-25T10:12:38.660381+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.577771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656597+0000","last_change":"2024-04-25T10:12:36.420411+0000","last_active":"2024-04-25T10:12:38.656597+0000","last_peered":"2024-04-25T10:12:38.656597+0000","last_clean":"2024-04-25T10:12:38.656597+0000","last_became_active":"2024-04-25T10:12:36.419903+0000","last_became_peered":"2024-04-25T10:12:36.419903+0000","last_unstale":"2024-04-25T10:12:38.656597+0000","last_undegraded":"2024-04-25T10:12:38.656597+0000","last_fullsized":"2024-04-25T10:12:38.656597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:43.819864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414447+0000","last_change":"2024-04-25T10:12:37.452031+0000","last_active":"2024-04-25T10:12:38.414447+0000","last_peered":"2024-04-25T10:12:38.414447+0000","last_clean":"2024-04-25T10:12:38.414447+0000","last_became_active":"2024-04-25T10:12:37.451729+0000","last_became_peered":"2024-04-25T10:12:37.451729+0000","last_unstale":"2024-04-25T10:12:38.414447+0000","last_undegraded":"2024-04-25T10:12:38.414447+0000","last_fullsized":"2024-04-25T10:12:38.414447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.311656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420290+0000","last_change":"2024-04-25T10:12:36.435352+0000","last_active":"2024-04-25T10:12:38.420290+0000","last_peered":"2024-04-25T10:12:38.420290+0000","last_clean":"2024-04-25T10:12:38.420290+0000","last_became_active":"2024-04-25T10:12:36.434943+0000","last_became_peered":"2024-04-25T10:12:36.434943+0000","last_unstale":"2024-04-25T10:12:38.420290+0000","last_undegraded":"2024-04-25T10:12:38.420290+0000","last_fullsized":"2024-04-25T10:12:38.420290+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657392+0000","last_change":"2024-04-25T10:12:37.437540+0000","last_active":"2024-04-25T10:12:38.657392+0000","last_peered":"2024-04-25T10:12:38.657392+0000","last_clean":"2024-04-25T10:12:38.657392+0000","last_became_active":"2024-04-25T10:12:37.437266+0000","last_became_peered":"2024-04-25T10:12:37.437266+0000","last_unstale":"2024-04-25T10:12:38.657392+0000","last_undegraded":"2024-04-25T10:12:38.657392+0000","last_fullsized":"2024-04-25T10:12:38.657392+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.631631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658976+0000","last_change":"2024-04-25T10:12:36.419478+0000","last_active":"2024-04-25T10:12:38.658976+0000","last_peered":"2024-04-25T10:12:38.658976+0000","last_clean":"2024-04-25T10:12:38.658976+0000","last_became_active":"2024-04-25T10:12:36.419010+0000","last_became_peered":"2024-04-25T10:12:36.419010+0000","last_unstale":"2024-04-25T10:12:38.658976+0000","last_undegraded":"2024-04-25T10:12:38.658976+0000","last_fullsized":"2024-04-25T10:12:38.658976+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:30.628822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658013+0000","last_change":"2024-04-25T10:12:37.439593+0000","last_active":"2024-04-25T10:12:38.658013+0000","last_peered":"2024-04-25T10:12:38.658013+0000","last_clean":"2024-04-25T10:12:38.658013+0000","last_became_active":"2024-04-25T10:12:37.438897+0000","last_became_peered":"2024-04-25T10:12:37.438897+0000","last_unstale":"2024-04-25T10:12:38.658013+0000","last_undegraded":"2024-04-25T10:12:38.658013+0000","last_fullsized":"2024-04-25T10:12:38.658013+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.701854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660240+0000","last_change":"2024-04-25T10:12:36.424429+0000","last_active":"2024-04-25T10:12:38.660240+0000","last_peered":"2024-04-25T10:12:38.660240+0000","last_clean":"2024-04-25T10:12:38.660240+0000","last_became_active":"2024-04-25T10:12:36.423878+0000","last_became_peered":"2024-04-25T10:12:36.423878+0000","last_unstale":"2024-04-25T10:12:38.660240+0000","last_undegraded":"2024-04-25T10:12:38.660240+0000","last_fullsized":"2024-04-25T10:12:38.660240+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:48.057318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417215+0000","last_change":"2024-04-25T10:12:37.448434+0000","last_active":"2024-04-25T10:12:38.417215+0000","last_peered":"2024-04-25T10:12:38.417215+0000","last_clean":"2024-04-25T10:12:38.417215+0000","last_became_active":"2024-04-25T10:12:37.447902+0000","last_became_peered":"2024-04-25T10:12:37.447902+0000","last_unstale":"2024-04-25T10:12:38.417215+0000","last_undegraded":"2024-04-25T10:12:38.417215+0000","last_fullsized":"2024-04-25T10:12:38.417215+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:56.964062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660034+0000","last_change":"2024-04-25T10:12:36.440745+0000","last_active":"2024-04-25T10:12:38.660034+0000","last_peered":"2024-04-25T10:12:38.660034+0000","last_clean":"2024-04-25T10:12:38.660034+0000","last_became_active":"2024-04-25T10:12:36.440448+0000","last_became_peered":"2024-04-25T10:12:36.440448+0000","last_unstale":"2024-04-25T10:12:38.660034+0000","last_undegraded":"2024-04-25T10:12:38.660034+0000","last_fullsized":"2024-04-25T10:12:38.660034+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:21.234143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.413900+0000","last_change":"2024-04-25T10:12:37.432087+0000","last_active":"2024-04-25T10:12:38.413900+0000","last_peered":"2024-04-25T10:12:38.413900+0000","last_clean":"2024-04-25T10:12:38.413900+0000","last_became_active":"2024-04-25T10:12:37.431814+0000","last_became_peered":"2024-04-25T10:12:37.431814+0000","last_unstale":"2024-04-25T10:12:38.413900+0000","last_undegraded":"2024-04-25T10:12:38.413900+0000","last_fullsized":"2024-04-25T10:12:38.413900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:19.357372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.422143+0000","last_change":"2024-04-25T10:12:36.430380+0000","last_active":"2024-04-25T10:12:38.422143+0000","last_peered":"2024-04-25T10:12:38.422143+0000","last_clean":"2024-04-25T10:12:38.422143+0000","last_became_active":"2024-04-25T10:12:36.429347+0000","last_became_peered":"2024-04-25T10:12:36.429347+0000","last_unstale":"2024-04-25T10:12:38.422143+0000","last_undegraded":"2024-04-25T10:12:38.422143+0000","last_fullsized":"2024-04-25T10:12:38.422143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657557+0000","last_change":"2024-04-25T10:12:37.436383+0000","last_active":"2024-04-25T10:12:38.657557+0000","last_peered":"2024-04-25T10:12:38.657557+0000","last_clean":"2024-04-25T10:12:38.657557+0000","last_became_active":"2024-04-25T10:12:37.436077+0000","last_became_peered":"2024-04-25T10:12:37.436077+0000","last_unstale":"2024-04-25T10:12:38.657557+0000","last_undegraded":"2024-04-25T10:12:38.657557+0000","last_fullsized":"2024-04-25T10:12:38.657557+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:53.082404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658706+0000","last_change":"2024-04-25T10:12:36.434613+0000","last_active":"2024-04-25T10:12:38.658706+0000","last_peered":"2024-04-25T10:12:38.658706+0000","last_clean":"2024-04-25T10:12:38.658706+0000","last_became_active":"2024-04-25T10:12:36.434287+0000","last_became_peered":"2024-04-25T10:12:36.434287+0000","last_unstale":"2024-04-25T10:12:38.658706+0000","last_undegraded":"2024-04-25T10:12:38.658706+0000","last_fullsized":"2024-04-25T10:12:38.658706+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:00.295480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656876+0000","last_change":"2024-04-25T10:12:37.442105+0000","last_active":"2024-04-25T10:12:38.656876+0000","last_peered":"2024-04-25T10:12:38.656876+0000","last_clean":"2024-04-25T10:12:38.656876+0000","last_became_active":"2024-04-25T10:12:37.440748+0000","last_became_peered":"2024-04-25T10:12:37.440748+0000","last_unstale":"2024-04-25T10:12:38.656876+0000","last_undegraded":"2024-04-25T10:12:38.656876+0000","last_fullsized":"2024-04-25T10:12:38.656876+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.046065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419799+0000","last_change":"2024-04-25T10:12:36.429818+0000","last_active":"2024-04-25T10:12:38.419799+0000","last_peered":"2024-04-25T10:12:38.419799+0000","last_clean":"2024-04-25T10:12:38.419799+0000","last_became_active":"2024-04-25T10:12:36.429160+0000","last_became_peered":"2024-04-25T10:12:36.429160+0000","last_unstale":"2024-04-25T10:12:38.419799+0000","last_undegraded":"2024-04-25T10:12:38.419799+0000","last_fullsized":"2024-04-25T10:12:38.419799+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659576+0000","last_change":"2024-04-25T10:12:37.434576+0000","last_active":"2024-04-25T10:12:38.659576+0000","last_peered":"2024-04-25T10:12:38.659576+0000","last_clean":"2024-04-25T10:12:38.659576+0000","last_became_active":"2024-04-25T10:12:37.434299+0000","last_became_peered":"2024-04-25T10:12:37.434299+0000","last_unstale":"2024-04-25T10:12:38.659576+0000","last_undegraded":"2024-04-25T10:12:38.659576+0000","last_fullsized":"2024-04-25T10:12:38.659576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:01.151422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656929+0000","last_change":"2024-04-25T10:12:36.428404+0000","last_active":"2024-04-25T10:12:38.656929+0000","last_peered":"2024-04-25T10:12:38.656929+0000","last_clean":"2024-04-25T10:12:38.656929+0000","last_became_active":"2024-04-25T10:12:36.428103+0000","last_became_peered":"2024-04-25T10:12:36.428103+0000","last_unstale":"2024-04-25T10:12:38.656929+0000","last_undegraded":"2024-04-25T10:12:38.656929+0000","last_fullsized":"2024-04-25T10:12:38.656929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:47.198001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.478234+0000","last_change":"2024-04-25T10:12:37.433986+0000","last_active":"2024-04-25T10:12:38.478234+0000","last_peered":"2024-04-25T10:12:38.478234+0000","last_clean":"2024-04-25T10:12:38.478234+0000","last_became_active":"2024-04-25T10:12:37.433467+0000","last_became_peered":"2024-04-25T10:12:37.433467+0000","last_unstale":"2024-04-25T10:12:38.478234+0000","last_undegraded":"2024-04-25T10:12:38.478234+0000","last_fullsized":"2024-04-25T10:12:38.478234+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.311656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420572+0000","last_change":"2024-04-25T10:12:36.423055+0000","last_active":"2024-04-25T10:12:38.420572+0000","last_peered":"2024-04-25T10:12:38.420572+0000","last_clean":"2024-04-25T10:12:38.420572+0000","last_became_active":"2024-04-25T10:12:36.422751+0000","last_became_peered":"2024-04-25T10:12:36.422751+0000","last_unstale":"2024-04-25T10:12:38.420572+0000","last_undegraded":"2024-04-25T10:12:38.420572+0000","last_fullsized":"2024-04-25T10:12:38.420572+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658454+0000","last_change":"2024-04-25T10:12:37.446201+0000","last_active":"2024-04-25T10:12:38.658454+0000","last_peered":"2024-04-25T10:12:38.658454+0000","last_clean":"2024-04-25T10:12:38.658454+0000","last_became_active":"2024-04-25T10:12:37.440114+0000","last_became_peered":"2024-04-25T10:12:37.440114+0000","last_unstale":"2024-04-25T10:12:38.658454+0000","last_undegraded":"2024-04-25T10:12:38.658454+0000","last_fullsized":"2024-04-25T10:12:38.658454+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:40.183842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659195+0000","last_change":"2024-04-25T10:12:36.426977+0000","last_active":"2024-04-25T10:12:38.659195+0000","last_peered":"2024-04-25T10:12:38.659195+0000","last_clean":"2024-04-25T10:12:38.659195+0000","last_became_active":"2024-04-25T10:12:36.424814+0000","last_became_peered":"2024-04-25T10:12:36.424814+0000","last_unstale":"2024-04-25T10:12:38.659195+0000","last_undegraded":"2024-04-25T10:12:38.659195+0000","last_fullsized":"2024-04-25T10:12:38.659195+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:28.338534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657375+0000","last_change":"2024-04-25T10:12:37.437126+0000","last_active":"2024-04-25T10:12:38.657375+0000","last_peered":"2024-04-25T10:12:38.657375+0000","last_clean":"2024-04-25T10:12:38.657375+0000","last_became_active":"2024-04-25T10:12:37.436835+0000","last_became_peered":"2024-04-25T10:12:37.436835+0000","last_unstale":"2024-04-25T10:12:38.657375+0000","last_undegraded":"2024-04-25T10:12:38.657375+0000","last_fullsized":"2024-04-25T10:12:38.657375+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:27.411214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659298+0000","last_change":"2024-04-25T10:12:36.419237+0000","last_active":"2024-04-25T10:12:38.659298+0000","last_peered":"2024-04-25T10:12:38.659298+0000","last_clean":"2024-04-25T10:12:38.659298+0000","last_became_active":"2024-04-25T10:12:36.418640+0000","last_became_peered":"2024-04-25T10:12:36.418640+0000","last_unstale":"2024-04-25T10:12:38.659298+0000","last_undegraded":"2024-04-25T10:12:38.659298+0000","last_fullsized":"2024-04-25T10:12:38.659298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.852848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477396+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:12:38.477396+0000","last_peered":"2024-04-25T10:12:38.477396+0000","last_clean":"2024-04-25T10:12:38.477396+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:12:38.477396+0000","last_undegraded":"2024-04-25T10:12:38.477396+0000","last_fullsized":"2024-04-25T10:12:38.477396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":1978368,"data_stored":1234135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50600,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321240,"statfs":{"total":96636764160,"available":96584949760,"internally_reserved":0,"allocated":1396736,"data_stored":645823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50580,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321260,"statfs":{"total":96636764160,"available":96584970240,"internally_reserved":0,"allocated":1376256,"data_stored":643607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1384448,"data_stored":643699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58896,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312944,"statfs":{"total":96636764160,"available":96576454656,"internally_reserved":0,"allocated":1380352,"data_stored":644189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54744,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317096,"statfs":{"total":96636764160,"available":96580706304,"internally_reserved":0,"allocated":1380352,"data_stored":643641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50576,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321264,"statfs":{"total":96636764160,"available":96584974336,"internally_reserved":0,"allocated":1380352,"data_stored":643641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":1986560,"data_stored":1235677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:42.817 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:12:43.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.051+0000 7f53b5800640 1 Processor -- start 2024-04-25T10:12:43.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.052+0000 7f53b5800640 1 -- start start 2024-04-25T10:12:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.053+0000 7f53b5800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.053+0000 7f53b5800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f53b01615a0 0x7f53b00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.053+0000 7f53b5800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b00d33f0 con 0x7f53b00d04f0 2024-04-25T10:12:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.053+0000 7f53b5800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b00d0960 con 0x7f53b01615a0 2024-04-25T10:12:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.053+0000 7f53b5800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b00d1f50 con 0x7f53b015db90 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f53b01615a0 0x7f53b00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53afe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f53b01615a0 0x7f53b00cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55744/0 (socket says 172.21.15.165:55744) 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 -- 172.21.15.165:0/3062319677 learned_addr learned my addr 172.21.15.165:0/3062319677 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 -- 172.21.15.165:0/3062319677 >> v2:172.21.15.172:3300/0 conn(0x7f53b015db90 msgr2=0x7f53b00b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 --2- 172.21.15.165:0/3062319677 >> v2:172.21.15.172:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 -- 172.21.15.165:0/3062319677 >> v1:172.21.15.172:6789/0 conn(0x7f53b00d04f0 legacy=0x7f53b0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.054+0000 7f53af400640 1 -- 172.21.15.165:0/3062319677 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b0161df0 con 0x7f53b01615a0 2024-04-25T10:12:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.056+0000 7f53af400640 1 --2- 172.21.15.165:0/3062319677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b01615a0 0x7f53b00cffb0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f53b006ceb0 tx=0x7f539802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b82330233e1dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.056+0000 7f53aea00640 1 -- 172.21.15.165:0/3062319677 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398037070 con 0x7f53b01615a0 2024-04-25T10:12:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.056+0000 7f53aea00640 1 -- 172.21.15.165:0/3062319677 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5398032440 con 0x7f53b01615a0 2024-04-25T10:12:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.056+0000 7f53aea00640 1 -- 172.21.15.165:0/3062319677 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398032810 con 0x7f53b01615a0 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.056+0000 7f53b5800640 1 -- 172.21.15.165:0/3062319677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b01615a0 msgr2=0x7f53b00cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.056+0000 7f53b5800640 1 --2- 172.21.15.165:0/3062319677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b01615a0 0x7f53b00cffb0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f53b006ceb0 tx=0x7f539802a270 comp rx=0 tx=0).stop 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.057+0000 7f53b5800640 1 -- 172.21.15.165:0/3062319677 shutdown_connections 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.057+0000 7f53b5800640 1 --2- 172.21.15.165:0/3062319677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b01615a0 0x7f53b00cffb0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.057+0000 7f53b5800640 1 --2- 172.21.15.165:0/3062319677 >> v2:172.21.15.172:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.057+0000 7f53b5800640 1 -- 172.21.15.165:0/3062319677 >> 172.21.15.165:0/3062319677 conn(0x7f53b00b52b0 msgr2=0x7f53b00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.057+0000 7f53b5800640 1 -- 172.21.15.165:0/3062319677 shutdown_connections 2024-04-25T10:12:43.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.057+0000 7f53b5800640 1 -- 172.21.15.165:0/3062319677 wait complete. 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.058+0000 7f53b5800640 1 Processor -- start 2024-04-25T10:12:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 -- start start 2024-04-25T10:12:43.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 0x7f53b015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b015db90 0x7f53b0151fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b01615a0 0x7f53b00cc940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00d1be0 con 0x7f53b00d04f0 2024-04-25T10:12:43.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b01623b0 con 0x7f53b015db90 2024-04-25T10:12:43.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.059+0000 7f53b5800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0161a20 con 0x7f53b01615a0 2024-04-25T10:12:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 0x7f53b015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 0x7f53b015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55756/0 (socket says 172.21.15.165:55756) 2024-04-25T10:12:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 -- 172.21.15.165:0/411485654 learned_addr learned my addr 172.21.15.165:0/411485654 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 -- 172.21.15.165:0/411485654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b01615a0 msgr2=0x7f53b00cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b01615a0 0x7f53b00cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 -- 172.21.15.165:0/411485654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b015db90 msgr2=0x7f53b0151fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b015db90 0x7f53b0151fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 -- 172.21.15.165:0/411485654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b01524f0 con 0x7f53b00d04f0 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.060+0000 7f53afe00640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 0x7f53b015e930 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f53a000efc0 tx=0x7f53a000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.061+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0018020 con 0x7f53b00d04f0 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.061+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0014480 con 0x7f53b00d04f0 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.061+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0014850 con 0x7f53b00d04f0 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.061+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b00cce80 con 0x7f53b00d04f0 2024-04-25T10:12:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.061+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b015f520 con 0x7f53b00d04f0 2024-04-25T10:12:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.062+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b0073990 con 0x7f53b00d04f0 2024-04-25T10:12:43.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.063+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53a0005d10 con 0x7f53b00d04f0 2024-04-25T10:12:43.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.064+0000 7f53a7e00640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.064+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53a008af20 con 0x7f53b00d04f0 2024-04-25T10:12:43.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.064+0000 7f53af400640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.065+0000 7f53af400640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5398034000 tx=0x7f539802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.068+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53a0022070 con 0x7f53b00d04f0 2024-04-25T10:12:43.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.336+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f53b00d14b0 con 0x7f53b00d04f0 2024-04-25T10:12:43.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.336+0000 7f53a7e00640 1 -- 172.21.15.165:0/411485654 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1015 (secure 0 0 0) 0x7f53a0010040 con 0x7f53b00d04f0 2024-04-25T10:12:43.341 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:43.341 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":664,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":111,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":441143296,"bytes_avail":772652969984,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:24.341144+0000","services":{}},"progress_events":{}} 2024-04-25T10:12:43.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 msgr2=0x7f538c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f5398034000 tx=0x7f539802c000 comp rx=0 tx=0).stop 2024-04-25T10:12:43.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 msgr2=0x7f53b015e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 0x7f53b015e930 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f53a000efc0 tx=0x7f53a000c490 comp rx=0 tx=0).stop 2024-04-25T10:12:43.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 shutdown_connections 2024-04-25T10:12:43.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b01615a0 0x7f53b00cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b015db90 0x7f53b0151fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 --2- 172.21.15.165:0/411485654 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00d04f0 0x7f53b015e930 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.339+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 >> 172.21.15.165:0/411485654 conn(0x7f53b00b52b0 msgr2=0x7f53b00b6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.340+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 shutdown_connections 2024-04-25T10:12:43.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.340+0000 7f53b5800640 1 -- 172.21.15.165:0/411485654 wait complete. 2024-04-25T10:12:43.365 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:12:43.365 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:43.374 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:12:43.370+0000 7f7db0400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:12:43.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.568+0000 7f6495000640 1 Processor -- start 2024-04-25T10:12:43.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.568+0000 7f6495000640 1 -- start start 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.570+0000 7f6495000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f649015db70 0x7f64900cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.570+0000 7f6495000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6490071650 con 0x7f649015df50 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.570+0000 7f6495000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64900cf230 con 0x7f64900cfc20 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.570+0000 7f6495000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64900cbd80 con 0x7f649015db70 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.570+0000 7f6487e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f649015df50 0x7f64900cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:38236/0 (socket says 172.21.15.165:38236) 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.570+0000 7f6487e00640 1 -- 172.21.15.165:0/784257365 learned_addr learned my addr 172.21.15.165:0/784257365 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648de00640 1 --2- 172.21.15.165:0/784257365 >> v2:172.21.15.172:3300/0 conn(0x7f649015db70 0x7f64900cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3970348269 0 0) 0x7f6478001110 con 0x7f649015df50 2024-04-25T10:12:43.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6474003520 con 0x7f649015df50 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251605132 0 0) 0x7f6474003520 con 0x7f649015df50 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 >> v2:172.21.15.172:3300/0 conn(0x7f649015db70 msgr2=0x7f64900cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 --2- 172.21.15.165:0/784257365 >> v2:172.21.15.172:3300/0 conn(0x7f649015db70 0x7f64900cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 >> v1:172.21.15.172:6789/0 conn(0x7f64900cfc20 legacy=0x7f649006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490161b60 con 0x7f649015df50 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6478001840 con 0x7f649015df50 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 >> v1:172.21.15.165:6789/0 conn(0x7f649015df50 legacy=0x7f64900cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6474003a00 0x7f6474003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f6474004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64740051c0 0x7f647400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64900cbd80 con 0x7f6474004800 2024-04-25T10:12:43.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647400c0c0 con 0x7f64740051c0 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.571+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647400c2c0 con 0x7f6474003a00 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f6487e00640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f6474004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f6487e00640 1 -- 172.21.15.165:0/784257365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6474003a00 msgr2=0x7f6474003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f6487e00640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6474003a00 0x7f6474003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f6487e00640 1 -- 172.21.15.165:0/784257365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64740051c0 msgr2=0x7f647400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f6487e00640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64740051c0 0x7f647400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f6487e00640 1 -- 172.21.15.165:0/784257365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474003520 con 0x7f6474004800 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.572+0000 7f648e800640 1 -- 172.21.15.165:0/784257365 reap_dead start 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.573+0000 7f6487e00640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f6474004c80 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f647800c320 tx=0x7f6474005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a149e7fa7114aee5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.573+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478036070 con 0x7f6474004800 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6478005af0 con 0x7f6474004800 2024-04-25T10:12:43.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f648d400640 1 -- 172.21.15.165:0/784257365 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6478007e00 con 0x7f6474004800 2024-04-25T10:12:43.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 -- 172.21.15.165:0/784257365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 msgr2=0x7f6474004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f6474004c80 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f647800c320 tx=0x7f6474005830 comp rx=0 tx=0).stop 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 -- 172.21.15.165:0/784257365 shutdown_connections 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 --2- 172.21.15.165:0/784257365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f6474004c80 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 -- 172.21.15.165:0/784257365 >> 172.21.15.165:0/784257365 conn(0x7f649006abe0 msgr2=0x7f649006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 -- 172.21.15.165:0/784257365 shutdown_connections 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.574+0000 7f6495000640 1 -- 172.21.15.165:0/784257365 wait complete. 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.576+0000 7f6495000640 1 Processor -- start 2024-04-25T10:12:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.576+0000 7f6495000640 1 -- start start 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.577+0000 7f6495000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f64901521c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.577+0000 7f6495000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64900b0610 0x7f64900b0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.577+0000 7f6495000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64901e79b0 0x7f64901e7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.577+0000 7f6495000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6490162120 con 0x7f6474004800 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.577+0000 7f6495000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6490161910 con 0x7f64901e79b0 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.577+0000 7f6495000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6490162640 con 0x7f64900b0610 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f64901521c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f64901521c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:55768/0 (socket says 172.21.15.165:55768) 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 -- 172.21.15.165:0/2596409647 learned_addr learned my addr 172.21.15.165:0/2596409647 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 -- 172.21.15.165:0/2596409647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64900b0610 msgr2=0x7f64900b0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64900b0610 0x7f64900b0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 -- 172.21.15.165:0/2596409647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64901e79b0 msgr2=0x7f64901e7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64901e79b0 0x7f64901e7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 -- 172.21.15.165:0/2596409647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6490152700 con 0x7f6474004800 2024-04-25T10:12:43.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.578+0000 7f648de00640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f64901521c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f648000e890 tx=0x7f648000ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.579+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6480017070 con 0x7f6474004800 2024-04-25T10:12:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.579+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f648000c940 con 0x7f6474004800 2024-04-25T10:12:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.579+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f648000cd10 con 0x7f6474004800 2024-04-25T10:12:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.579+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64900b10b0 con 0x7f6474004800 2024-04-25T10:12:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.579+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64900b1670 con 0x7f6474004800 2024-04-25T10:12:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.580+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6490073990 con 0x7f6474004800 2024-04-25T10:12:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.581+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f648001a070 con 0x7f6474004800 2024-04-25T10:12:43.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.582+0000 7f6486000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f646c06b500 0x7f646c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:43.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.583+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f648008a270 con 0x7f6474004800 2024-04-25T10:12:43.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.583+0000 7f6487e00640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f646c06b500 0x7f646c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:43.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.583+0000 7f6487e00640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f646c06b500 0x7f646c06d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6478033000 tx=0x7f6478007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:43.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.587+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6490152700 con 0x7f6474004800 2024-04-25T10:12:43.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.782+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64900b47e0 con 0x7f646c06b500 2024-04-25T10:12:43.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.800+0000 7f6486000640 1 -- 172.21.15.165:0/2596409647 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f64900b47e0 con 0x7f646c06b500 2024-04-25T10:12:43.807 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:43.810 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:12:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f646c06b500 msgr2=0x7f646c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f646c06b500 0x7f646c06d9c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f6478033000 tx=0x7f6478007000 comp rx=0 tx=0).stop 2024-04-25T10:12:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 msgr2=0x7f64901521c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:43.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f64901521c0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f648000e890 tx=0x7f648000ed50 comp rx=0 tx=0).stop 2024-04-25T10:12:43.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 shutdown_connections 2024-04-25T10:12:43.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f646c06b500 0x7f646c06d9c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64901e79b0 0x7f64901e7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64900b0610 0x7f64900b0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 --2- 172.21.15.165:0/2596409647 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6474004800 0x7f64901521c0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:43.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 >> 172.21.15.165:0/2596409647 conn(0x7f649006abe0 msgr2=0x7f64900ac970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:43.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.807+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 shutdown_connections 2024-04-25T10:12:43.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:43.808+0000 7f6495000640 1 -- 172.21.15.165:0/2596409647 wait complete. 2024-04-25T10:12:43.831 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":429,"stamp":"2024-04-25T10:12:42.345456+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430804,"kb_used_data":11976,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754543916,"statfs":{"total":773094113280,"available":772652969984,"internally_reserved":0,"allocated":12263424,"data_stored":6334412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":428723762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001569"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660408+0000","last_change":"2024-04-25T10:12:37.440175+0000","last_active":"2024-04-25T10:12:38.660408+0000","last_peered":"2024-04-25T10:12:38.660408+0000","last_clean":"2024-04-25T10:12:38.660408+0000","last_became_active":"2024-04-25T10:12:37.437927+0000","last_became_peered":"2024-04-25T10:12:37.437927+0000","last_unstale":"2024-04-25T10:12:38.660408+0000","last_undegraded":"2024-04-25T10:12:38.660408+0000","last_fullsized":"2024-04-25T10:12:38.660408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:47.423921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421866+0000","last_change":"2024-04-25T10:12:37.442081+0000","last_active":"2024-04-25T10:12:38.421866+0000","last_peered":"2024-04-25T10:12:38.421866+0000","last_clean":"2024-04-25T10:12:38.421866+0000","last_became_active":"2024-04-25T10:12:37.440679+0000","last_became_peered":"2024-04-25T10:12:37.440679+0000","last_unstale":"2024-04-25T10:12:38.421866+0000","last_undegraded":"2024-04-25T10:12:38.421866+0000","last_fullsized":"2024-04-25T10:12:38.421866+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:43.163133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660454+0000","last_change":"2024-04-25T10:12:37.443092+0000","last_active":"2024-04-25T10:12:38.660454+0000","last_peered":"2024-04-25T10:12:38.660454+0000","last_clean":"2024-04-25T10:12:38.660454+0000","last_became_active":"2024-04-25T10:12:37.442779+0000","last_became_peered":"2024-04-25T10:12:37.442779+0000","last_unstale":"2024-04-25T10:12:38.660454+0000","last_undegraded":"2024-04-25T10:12:38.660454+0000","last_fullsized":"2024-04-25T10:12:38.660454+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:43.318742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414964+0000","last_change":"2024-04-25T10:12:37.430317+0000","last_active":"2024-04-25T10:12:38.414964+0000","last_peered":"2024-04-25T10:12:38.414964+0000","last_clean":"2024-04-25T10:12:38.414964+0000","last_became_active":"2024-04-25T10:12:37.430009+0000","last_became_peered":"2024-04-25T10:12:37.430009+0000","last_unstale":"2024-04-25T10:12:38.414964+0000","last_undegraded":"2024-04-25T10:12:38.414964+0000","last_fullsized":"2024-04-25T10:12:38.414964+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:22.585606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657765+0000","last_change":"2024-04-25T10:12:37.443661+0000","last_active":"2024-04-25T10:12:38.657765+0000","last_peered":"2024-04-25T10:12:38.657765+0000","last_clean":"2024-04-25T10:12:38.657765+0000","last_became_active":"2024-04-25T10:12:37.443277+0000","last_became_peered":"2024-04-25T10:12:37.443277+0000","last_unstale":"2024-04-25T10:12:38.657765+0000","last_undegraded":"2024-04-25T10:12:38.657765+0000","last_fullsized":"2024-04-25T10:12:38.657765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421328+0000","last_change":"2024-04-25T10:12:37.446343+0000","last_active":"2024-04-25T10:12:38.421328+0000","last_peered":"2024-04-25T10:12:38.421328+0000","last_clean":"2024-04-25T10:12:38.421328+0000","last_became_active":"2024-04-25T10:12:37.444637+0000","last_became_peered":"2024-04-25T10:12:37.444637+0000","last_unstale":"2024-04-25T10:12:38.421328+0000","last_undegraded":"2024-04-25T10:12:38.421328+0000","last_fullsized":"2024-04-25T10:12:38.421328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418202+0000","last_change":"2024-04-25T10:12:37.443934+0000","last_active":"2024-04-25T10:12:38.418202+0000","last_peered":"2024-04-25T10:12:38.418202+0000","last_clean":"2024-04-25T10:12:38.418202+0000","last_became_active":"2024-04-25T10:12:37.443387+0000","last_became_peered":"2024-04-25T10:12:37.443387+0000","last_unstale":"2024-04-25T10:12:38.418202+0000","last_undegraded":"2024-04-25T10:12:38.418202+0000","last_fullsized":"2024-04-25T10:12:38.418202+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:56.964062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658734+0000","last_change":"2024-04-25T10:12:37.434890+0000","last_active":"2024-04-25T10:12:38.658734+0000","last_peered":"2024-04-25T10:12:38.658734+0000","last_clean":"2024-04-25T10:12:38.658734+0000","last_became_active":"2024-04-25T10:12:37.434607+0000","last_became_peered":"2024-04-25T10:12:37.434607+0000","last_unstale":"2024-04-25T10:12:38.658734+0000","last_undegraded":"2024-04-25T10:12:38.658734+0000","last_fullsized":"2024-04-25T10:12:38.658734+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:50.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421703+0000","last_change":"2024-04-25T10:12:37.440646+0000","last_active":"2024-04-25T10:12:38.421703+0000","last_peered":"2024-04-25T10:12:38.421703+0000","last_clean":"2024-04-25T10:12:38.421703+0000","last_became_active":"2024-04-25T10:12:37.440279+0000","last_became_peered":"2024-04-25T10:12:37.440279+0000","last_unstale":"2024-04-25T10:12:38.421703+0000","last_undegraded":"2024-04-25T10:12:38.421703+0000","last_fullsized":"2024-04-25T10:12:38.421703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658834+0000","last_change":"2024-04-25T10:12:37.451524+0000","last_active":"2024-04-25T10:12:38.658834+0000","last_peered":"2024-04-25T10:12:38.658834+0000","last_clean":"2024-04-25T10:12:38.658834+0000","last_became_active":"2024-04-25T10:12:37.450715+0000","last_became_peered":"2024-04-25T10:12:37.450715+0000","last_unstale":"2024-04-25T10:12:38.658834+0000","last_undegraded":"2024-04-25T10:12:38.658834+0000","last_fullsized":"2024-04-25T10:12:38.658834+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658039+0000","last_change":"2024-04-25T10:12:37.446411+0000","last_active":"2024-04-25T10:12:38.658039+0000","last_peered":"2024-04-25T10:12:38.658039+0000","last_clean":"2024-04-25T10:12:38.658039+0000","last_became_active":"2024-04-25T10:12:37.445773+0000","last_became_peered":"2024-04-25T10:12:37.445773+0000","last_unstale":"2024-04-25T10:12:38.658039+0000","last_undegraded":"2024-04-25T10:12:38.658039+0000","last_fullsized":"2024-04-25T10:12:38.658039+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:21.571051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418659+0000","last_change":"2024-04-25T10:12:37.449685+0000","last_active":"2024-04-25T10:12:38.418659+0000","last_peered":"2024-04-25T10:12:38.418659+0000","last_clean":"2024-04-25T10:12:38.418659+0000","last_became_active":"2024-04-25T10:12:37.448670+0000","last_became_peered":"2024-04-25T10:12:37.448670+0000","last_unstale":"2024-04-25T10:12:38.418659+0000","last_undegraded":"2024-04-25T10:12:38.418659+0000","last_fullsized":"2024-04-25T10:12:38.418659+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.413874+0000","last_change":"2024-04-25T10:12:37.441928+0000","last_active":"2024-04-25T10:12:38.413874+0000","last_peered":"2024-04-25T10:12:38.413874+0000","last_clean":"2024-04-25T10:12:38.413874+0000","last_became_active":"2024-04-25T10:12:37.441644+0000","last_became_peered":"2024-04-25T10:12:37.441644+0000","last_unstale":"2024-04-25T10:12:38.413874+0000","last_undegraded":"2024-04-25T10:12:38.413874+0000","last_fullsized":"2024-04-25T10:12:38.413874+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:12.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477504+0000","last_change":"2024-04-25T10:12:37.434637+0000","last_active":"2024-04-25T10:12:38.477504+0000","last_peered":"2024-04-25T10:12:38.477504+0000","last_clean":"2024-04-25T10:12:38.477504+0000","last_became_active":"2024-04-25T10:12:37.434374+0000","last_became_peered":"2024-04-25T10:12:37.434374+0000","last_unstale":"2024-04-25T10:12:38.477504+0000","last_undegraded":"2024-04-25T10:12:38.477504+0000","last_fullsized":"2024-04-25T10:12:38.477504+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:12.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477546+0000","last_change":"2024-04-25T10:12:37.448340+0000","last_active":"2024-04-25T10:12:38.477546+0000","last_peered":"2024-04-25T10:12:38.477546+0000","last_clean":"2024-04-25T10:12:38.477546+0000","last_became_active":"2024-04-25T10:12:37.446052+0000","last_became_peered":"2024-04-25T10:12:37.446052+0000","last_unstale":"2024-04-25T10:12:38.477546+0000","last_undegraded":"2024-04-25T10:12:38.477546+0000","last_fullsized":"2024-04-25T10:12:38.477546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:19.357372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477573+0000","last_change":"2024-04-25T10:12:37.424865+0000","last_active":"2024-04-25T10:12:38.477573+0000","last_peered":"2024-04-25T10:12:38.477573+0000","last_clean":"2024-04-25T10:12:38.477573+0000","last_became_active":"2024-04-25T10:12:37.424580+0000","last_became_peered":"2024-04-25T10:12:37.424580+0000","last_unstale":"2024-04-25T10:12:38.477573+0000","last_undegraded":"2024-04-25T10:12:38.477573+0000","last_fullsized":"2024-04-25T10:12:38.477573+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:34.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660749+0000","last_change":"2024-04-25T10:12:37.440407+0000","last_active":"2024-04-25T10:12:38.660749+0000","last_peered":"2024-04-25T10:12:38.660749+0000","last_clean":"2024-04-25T10:12:38.660749+0000","last_became_active":"2024-04-25T10:12:37.439466+0000","last_became_peered":"2024-04-25T10:12:37.439466+0000","last_unstale":"2024-04-25T10:12:38.660749+0000","last_undegraded":"2024-04-25T10:12:38.660749+0000","last_fullsized":"2024-04-25T10:12:38.660749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:16.348692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421858+0000","last_change":"2024-04-25T10:12:37.444108+0000","last_active":"2024-04-25T10:12:38.421858+0000","last_peered":"2024-04-25T10:12:38.421858+0000","last_clean":"2024-04-25T10:12:38.421858+0000","last_became_active":"2024-04-25T10:12:37.443514+0000","last_became_peered":"2024-04-25T10:12:37.443514+0000","last_unstale":"2024-04-25T10:12:38.421858+0000","last_undegraded":"2024-04-25T10:12:38.421858+0000","last_fullsized":"2024-04-25T10:12:38.421858+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660185+0000","last_change":"2024-04-25T10:12:37.451809+0000","last_active":"2024-04-25T10:12:38.660185+0000","last_peered":"2024-04-25T10:12:38.660185+0000","last_clean":"2024-04-25T10:12:38.660185+0000","last_became_active":"2024-04-25T10:12:37.449840+0000","last_became_peered":"2024-04-25T10:12:37.449840+0000","last_unstale":"2024-04-25T10:12:38.660185+0000","last_undegraded":"2024-04-25T10:12:38.660185+0000","last_fullsized":"2024-04-25T10:12:38.660185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:00.348275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419395+0000","last_change":"2024-04-25T10:12:37.425902+0000","last_active":"2024-04-25T10:12:38.419395+0000","last_peered":"2024-04-25T10:12:38.419395+0000","last_clean":"2024-04-25T10:12:38.419395+0000","last_became_active":"2024-04-25T10:12:37.425600+0000","last_became_peered":"2024-04-25T10:12:37.425600+0000","last_unstale":"2024-04-25T10:12:38.419395+0000","last_undegraded":"2024-04-25T10:12:38.419395+0000","last_fullsized":"2024-04-25T10:12:38.419395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656760+0000","last_change":"2024-04-25T10:12:37.432765+0000","last_active":"2024-04-25T10:12:38.656760+0000","last_peered":"2024-04-25T10:12:38.656760+0000","last_clean":"2024-04-25T10:12:38.656760+0000","last_became_active":"2024-04-25T10:12:37.432505+0000","last_became_peered":"2024-04-25T10:12:37.432505+0000","last_unstale":"2024-04-25T10:12:38.656760+0000","last_undegraded":"2024-04-25T10:12:38.656760+0000","last_fullsized":"2024-04-25T10:12:38.656760+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:00.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660318+0000","last_change":"2024-04-25T10:12:37.443138+0000","last_active":"2024-04-25T10:12:38.660318+0000","last_peered":"2024-04-25T10:12:38.660318+0000","last_clean":"2024-04-25T10:12:38.660318+0000","last_became_active":"2024-04-25T10:12:37.442308+0000","last_became_peered":"2024-04-25T10:12:37.442308+0000","last_unstale":"2024-04-25T10:12:38.660318+0000","last_undegraded":"2024-04-25T10:12:38.660318+0000","last_fullsized":"2024-04-25T10:12:38.660318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:43.163133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659462+0000","last_change":"2024-04-25T10:12:37.444200+0000","last_active":"2024-04-25T10:12:38.659462+0000","last_peered":"2024-04-25T10:12:38.659462+0000","last_clean":"2024-04-25T10:12:38.659462+0000","last_became_active":"2024-04-25T10:12:37.443940+0000","last_became_peered":"2024-04-25T10:12:37.443940+0000","last_unstale":"2024-04-25T10:12:38.659462+0000","last_undegraded":"2024-04-25T10:12:38.659462+0000","last_fullsized":"2024-04-25T10:12:38.659462+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.836901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659970+0000","last_change":"2024-04-25T10:12:37.448155+0000","last_active":"2024-04-25T10:12:38.659970+0000","last_peered":"2024-04-25T10:12:38.659970+0000","last_clean":"2024-04-25T10:12:38.659970+0000","last_became_active":"2024-04-25T10:12:37.447522+0000","last_became_peered":"2024-04-25T10:12:37.447522+0000","last_unstale":"2024-04-25T10:12:38.659970+0000","last_undegraded":"2024-04-25T10:12:38.659970+0000","last_fullsized":"2024-04-25T10:12:38.659970+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419980+0000","last_change":"2024-04-25T10:12:37.446108+0000","last_active":"2024-04-25T10:12:38.419980+0000","last_peered":"2024-04-25T10:12:38.419980+0000","last_clean":"2024-04-25T10:12:38.419980+0000","last_became_active":"2024-04-25T10:12:37.445831+0000","last_became_peered":"2024-04-25T10:12:37.445831+0000","last_unstale":"2024-04-25T10:12:38.419980+0000","last_undegraded":"2024-04-25T10:12:38.419980+0000","last_fullsized":"2024-04-25T10:12:38.419980+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418914+0000","last_change":"2024-04-25T10:12:37.441013+0000","last_active":"2024-04-25T10:12:38.418914+0000","last_peered":"2024-04-25T10:12:38.418914+0000","last_clean":"2024-04-25T10:12:38.418914+0000","last_became_active":"2024-04-25T10:12:37.440725+0000","last_became_peered":"2024-04-25T10:12:37.440725+0000","last_unstale":"2024-04-25T10:12:38.418914+0000","last_undegraded":"2024-04-25T10:12:38.418914+0000","last_fullsized":"2024-04-25T10:12:38.418914+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658244+0000","last_change":"2024-04-25T10:12:37.438780+0000","last_active":"2024-04-25T10:12:38.658244+0000","last_peered":"2024-04-25T10:12:38.658244+0000","last_clean":"2024-04-25T10:12:38.658244+0000","last_became_active":"2024-04-25T10:12:37.438484+0000","last_became_peered":"2024-04-25T10:12:37.438484+0000","last_unstale":"2024-04-25T10:12:38.658244+0000","last_undegraded":"2024-04-25T10:12:38.658244+0000","last_fullsized":"2024-04-25T10:12:38.658244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:54.229611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420777+0000","last_change":"2024-04-25T10:12:37.440617+0000","last_active":"2024-04-25T10:12:38.420777+0000","last_peered":"2024-04-25T10:12:38.420777+0000","last_clean":"2024-04-25T10:12:38.420777+0000","last_became_active":"2024-04-25T10:12:37.440310+0000","last_became_peered":"2024-04-25T10:12:37.440310+0000","last_unstale":"2024-04-25T10:12:38.420777+0000","last_undegraded":"2024-04-25T10:12:38.420777+0000","last_fullsized":"2024-04-25T10:12:38.420777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656959+0000","last_change":"2024-04-25T10:12:37.426018+0000","last_active":"2024-04-25T10:12:38.656959+0000","last_peered":"2024-04-25T10:12:38.656959+0000","last_clean":"2024-04-25T10:12:38.656959+0000","last_became_active":"2024-04-25T10:12:37.425637+0000","last_became_peered":"2024-04-25T10:12:37.425637+0000","last_unstale":"2024-04-25T10:12:38.656959+0000","last_undegraded":"2024-04-25T10:12:38.656959+0000","last_fullsized":"2024-04-25T10:12:38.656959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:00.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418921+0000","last_change":"2024-04-25T10:12:37.443856+0000","last_active":"2024-04-25T10:12:38.418921+0000","last_peered":"2024-04-25T10:12:38.418921+0000","last_clean":"2024-04-25T10:12:38.418921+0000","last_became_active":"2024-04-25T10:12:37.443421+0000","last_became_peered":"2024-04-25T10:12:37.443421+0000","last_unstale":"2024-04-25T10:12:38.418921+0000","last_undegraded":"2024-04-25T10:12:38.418921+0000","last_fullsized":"2024-04-25T10:12:38.418921+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418651+0000","last_change":"2024-04-25T10:12:37.433545+0000","last_active":"2024-04-25T10:12:38.418651+0000","last_peered":"2024-04-25T10:12:38.418651+0000","last_clean":"2024-04-25T10:12:38.418651+0000","last_became_active":"2024-04-25T10:12:37.433271+0000","last_became_peered":"2024-04-25T10:12:37.433271+0000","last_unstale":"2024-04-25T10:12:38.418651+0000","last_undegraded":"2024-04-25T10:12:38.418651+0000","last_fullsized":"2024-04-25T10:12:38.418651+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656933+0000","last_change":"2024-04-25T10:12:37.426624+0000","last_active":"2024-04-25T10:12:38.656933+0000","last_peered":"2024-04-25T10:12:38.656933+0000","last_clean":"2024-04-25T10:12:38.656933+0000","last_became_active":"2024-04-25T10:12:37.425873+0000","last_became_peered":"2024-04-25T10:12:37.425873+0000","last_unstale":"2024-04-25T10:12:38.656933+0000","last_undegraded":"2024-04-25T10:12:38.656933+0000","last_fullsized":"2024-04-25T10:12:38.656933+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658673+0000","last_change":"2024-04-25T10:12:37.451368+0000","last_active":"2024-04-25T10:12:38.658673+0000","last_peered":"2024-04-25T10:12:38.658673+0000","last_clean":"2024-04-25T10:12:38.658673+0000","last_became_active":"2024-04-25T10:12:37.451027+0000","last_became_peered":"2024-04-25T10:12:37.451027+0000","last_unstale":"2024-04-25T10:12:38.658673+0000","last_undegraded":"2024-04-25T10:12:38.658673+0000","last_fullsized":"2024-04-25T10:12:38.658673+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:08.472720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414586+0000","last_change":"2024-04-25T10:12:36.428651+0000","last_active":"2024-04-25T10:12:38.414586+0000","last_peered":"2024-04-25T10:12:38.414586+0000","last_clean":"2024-04-25T10:12:38.414586+0000","last_became_active":"2024-04-25T10:12:36.428062+0000","last_became_peered":"2024-04-25T10:12:36.428062+0000","last_unstale":"2024-04-25T10:12:38.414586+0000","last_undegraded":"2024-04-25T10:12:38.414586+0000","last_fullsized":"2024-04-25T10:12:38.414586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:42.723368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659663+0000","last_change":"2024-04-25T10:12:37.442206+0000","last_active":"2024-04-25T10:12:38.659663+0000","last_peered":"2024-04-25T10:12:38.659663+0000","last_clean":"2024-04-25T10:12:38.659663+0000","last_became_active":"2024-04-25T10:12:37.441474+0000","last_became_peered":"2024-04-25T10:12:37.441474+0000","last_unstale":"2024-04-25T10:12:38.659663+0000","last_undegraded":"2024-04-25T10:12:38.659663+0000","last_fullsized":"2024-04-25T10:12:38.659663+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:09.781858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659154+0000","last_change":"2024-04-25T10:12:36.432356+0000","last_active":"2024-04-25T10:12:38.659154+0000","last_peered":"2024-04-25T10:12:38.659154+0000","last_clean":"2024-04-25T10:12:38.659154+0000","last_became_active":"2024-04-25T10:12:36.432086+0000","last_became_peered":"2024-04-25T10:12:36.432086+0000","last_unstale":"2024-04-25T10:12:38.659154+0000","last_undegraded":"2024-04-25T10:12:38.659154+0000","last_fullsized":"2024-04-25T10:12:38.659154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:46.923030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419369+0000","last_change":"2024-04-25T10:12:37.447367+0000","last_active":"2024-04-25T10:12:38.419369+0000","last_peered":"2024-04-25T10:12:38.419369+0000","last_clean":"2024-04-25T10:12:38.419369+0000","last_became_active":"2024-04-25T10:12:37.447079+0000","last_became_peered":"2024-04-25T10:12:37.447079+0000","last_unstale":"2024-04-25T10:12:38.419369+0000","last_undegraded":"2024-04-25T10:12:38.419369+0000","last_fullsized":"2024-04-25T10:12:38.419369+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418183+0000","last_change":"2024-04-25T10:12:36.437597+0000","last_active":"2024-04-25T10:12:38.418183+0000","last_peered":"2024-04-25T10:12:38.418183+0000","last_clean":"2024-04-25T10:12:38.418183+0000","last_became_active":"2024-04-25T10:12:36.437214+0000","last_became_peered":"2024-04-25T10:12:36.437214+0000","last_unstale":"2024-04-25T10:12:38.418183+0000","last_undegraded":"2024-04-25T10:12:38.418183+0000","last_fullsized":"2024-04-25T10:12:38.418183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:54.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657727+0000","last_change":"2024-04-25T10:12:37.433634+0000","last_active":"2024-04-25T10:12:38.657727+0000","last_peered":"2024-04-25T10:12:38.657727+0000","last_clean":"2024-04-25T10:12:38.657727+0000","last_became_active":"2024-04-25T10:12:37.433334+0000","last_became_peered":"2024-04-25T10:12:37.433334+0000","last_unstale":"2024-04-25T10:12:38.657727+0000","last_undegraded":"2024-04-25T10:12:38.657727+0000","last_fullsized":"2024-04-25T10:12:38.657727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.027233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658433+0000","last_change":"2024-04-25T10:12:36.430378+0000","last_active":"2024-04-25T10:12:38.658433+0000","last_peered":"2024-04-25T10:12:38.658433+0000","last_clean":"2024-04-25T10:12:38.658433+0000","last_became_active":"2024-04-25T10:12:36.430043+0000","last_became_peered":"2024-04-25T10:12:36.430043+0000","last_unstale":"2024-04-25T10:12:38.658433+0000","last_undegraded":"2024-04-25T10:12:38.658433+0000","last_fullsized":"2024-04-25T10:12:38.658433+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:14.731959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414534+0000","last_change":"2024-04-25T10:12:37.438453+0000","last_active":"2024-04-25T10:12:38.414534+0000","last_peered":"2024-04-25T10:12:38.414534+0000","last_clean":"2024-04-25T10:12:38.414534+0000","last_became_active":"2024-04-25T10:12:37.438179+0000","last_became_peered":"2024-04-25T10:12:37.438179+0000","last_unstale":"2024-04-25T10:12:38.414534+0000","last_undegraded":"2024-04-25T10:12:38.414534+0000","last_fullsized":"2024-04-25T10:12:38.414534+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:56.927046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477763+0000","last_change":"2024-04-25T10:12:36.432661+0000","last_active":"2024-04-25T10:12:38.477763+0000","last_peered":"2024-04-25T10:12:38.477763+0000","last_clean":"2024-04-25T10:12:38.477763+0000","last_became_active":"2024-04-25T10:12:36.432286+0000","last_became_peered":"2024-04-25T10:12:36.432286+0000","last_unstale":"2024-04-25T10:12:38.477763+0000","last_undegraded":"2024-04-25T10:12:38.477763+0000","last_fullsized":"2024-04-25T10:12:38.477763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:35.057626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660604+0000","last_change":"2024-04-25T10:12:37.450831+0000","last_active":"2024-04-25T10:12:38.660604+0000","last_peered":"2024-04-25T10:12:38.660604+0000","last_clean":"2024-04-25T10:12:38.660604+0000","last_became_active":"2024-04-25T10:12:37.450537+0000","last_became_peered":"2024-04-25T10:12:37.450537+0000","last_unstale":"2024-04-25T10:12:38.660604+0000","last_undegraded":"2024-04-25T10:12:38.660604+0000","last_fullsized":"2024-04-25T10:12:38.660604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:13.308948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657298+0000","last_change":"2024-04-25T10:12:36.437498+0000","last_active":"2024-04-25T10:12:38.657298+0000","last_peered":"2024-04-25T10:12:38.657298+0000","last_clean":"2024-04-25T10:12:38.657298+0000","last_became_active":"2024-04-25T10:12:36.437164+0000","last_became_peered":"2024-04-25T10:12:36.437164+0000","last_unstale":"2024-04-25T10:12:38.657298+0000","last_undegraded":"2024-04-25T10:12:38.657298+0000","last_fullsized":"2024-04-25T10:12:38.657298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:06.982259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.422167+0000","last_change":"2024-04-25T10:12:37.442744+0000","last_active":"2024-04-25T10:12:38.422167+0000","last_peered":"2024-04-25T10:12:38.422167+0000","last_clean":"2024-04-25T10:12:38.422167+0000","last_became_active":"2024-04-25T10:12:37.441161+0000","last_became_peered":"2024-04-25T10:12:37.441161+0000","last_unstale":"2024-04-25T10:12:38.422167+0000","last_undegraded":"2024-04-25T10:12:38.422167+0000","last_fullsized":"2024-04-25T10:12:38.422167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:13.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417597+0000","last_change":"2024-04-25T10:12:36.436140+0000","last_active":"2024-04-25T10:12:38.417597+0000","last_peered":"2024-04-25T10:12:38.417597+0000","last_clean":"2024-04-25T10:12:38.417597+0000","last_became_active":"2024-04-25T10:12:36.435843+0000","last_became_peered":"2024-04-25T10:12:36.435843+0000","last_unstale":"2024-04-25T10:12:38.417597+0000","last_undegraded":"2024-04-25T10:12:38.417597+0000","last_fullsized":"2024-04-25T10:12:38.417597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414480+0000","last_change":"2024-04-25T10:12:37.436207+0000","last_active":"2024-04-25T10:12:38.414480+0000","last_peered":"2024-04-25T10:12:38.414480+0000","last_clean":"2024-04-25T10:12:38.414480+0000","last_became_active":"2024-04-25T10:12:37.435907+0000","last_became_peered":"2024-04-25T10:12:37.435907+0000","last_unstale":"2024-04-25T10:12:38.414480+0000","last_undegraded":"2024-04-25T10:12:38.414480+0000","last_fullsized":"2024-04-25T10:12:38.414480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:34.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414951+0000","last_change":"2024-04-25T10:12:36.433709+0000","last_active":"2024-04-25T10:12:38.414951+0000","last_peered":"2024-04-25T10:12:38.414951+0000","last_clean":"2024-04-25T10:12:38.414951+0000","last_became_active":"2024-04-25T10:12:36.433440+0000","last_became_peered":"2024-04-25T10:12:36.433440+0000","last_unstale":"2024-04-25T10:12:38.414951+0000","last_undegraded":"2024-04-25T10:12:38.414951+0000","last_fullsized":"2024-04-25T10:12:38.414951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:35.057626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658349+0000","last_change":"2024-04-25T10:12:37.439841+0000","last_active":"2024-04-25T10:12:38.658349+0000","last_peered":"2024-04-25T10:12:38.658349+0000","last_clean":"2024-04-25T10:12:38.658349+0000","last_became_active":"2024-04-25T10:12:37.439287+0000","last_became_peered":"2024-04-25T10:12:37.439287+0000","last_unstale":"2024-04-25T10:12:38.658349+0000","last_undegraded":"2024-04-25T10:12:38.658349+0000","last_fullsized":"2024-04-25T10:12:38.658349+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:20.343948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420040+0000","last_change":"2024-04-25T10:12:36.430969+0000","last_active":"2024-04-25T10:12:38.420040+0000","last_peered":"2024-04-25T10:12:38.420040+0000","last_clean":"2024-04-25T10:12:38.420040+0000","last_became_active":"2024-04-25T10:12:36.430681+0000","last_became_peered":"2024-04-25T10:12:36.430681+0000","last_unstale":"2024-04-25T10:12:38.420040+0000","last_undegraded":"2024-04-25T10:12:38.420040+0000","last_fullsized":"2024-04-25T10:12:38.420040+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:48.057318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657867+0000","last_change":"2024-04-25T10:12:37.435317+0000","last_active":"2024-04-25T10:12:38.657867+0000","last_peered":"2024-04-25T10:12:38.657867+0000","last_clean":"2024-04-25T10:12:38.657867+0000","last_became_active":"2024-04-25T10:12:37.435007+0000","last_became_peered":"2024-04-25T10:12:37.435007+0000","last_unstale":"2024-04-25T10:12:38.657867+0000","last_undegraded":"2024-04-25T10:12:38.657867+0000","last_fullsized":"2024-04-25T10:12:38.657867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659574+0000","last_change":"2024-04-25T10:12:36.435089+0000","last_active":"2024-04-25T10:12:38.659574+0000","last_peered":"2024-04-25T10:12:38.659574+0000","last_clean":"2024-04-25T10:12:38.659574+0000","last_became_active":"2024-04-25T10:12:36.434759+0000","last_became_peered":"2024-04-25T10:12:36.434759+0000","last_unstale":"2024-04-25T10:12:38.659574+0000","last_undegraded":"2024-04-25T10:12:38.659574+0000","last_fullsized":"2024-04-25T10:12:38.659574+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:37.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660287+0000","last_change":"2024-04-25T10:12:37.441207+0000","last_active":"2024-04-25T10:12:38.660287+0000","last_peered":"2024-04-25T10:12:38.660287+0000","last_clean":"2024-04-25T10:12:38.660287+0000","last_became_active":"2024-04-25T10:12:37.440925+0000","last_became_peered":"2024-04-25T10:12:37.440925+0000","last_unstale":"2024-04-25T10:12:38.660287+0000","last_undegraded":"2024-04-25T10:12:38.660287+0000","last_fullsized":"2024-04-25T10:12:38.660287+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:08.118583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477926+0000","last_change":"2024-04-25T10:12:36.438390+0000","last_active":"2024-04-25T10:12:38.477926+0000","last_peered":"2024-04-25T10:12:38.477926+0000","last_clean":"2024-04-25T10:12:38.477926+0000","last_became_active":"2024-04-25T10:12:36.438100+0000","last_became_peered":"2024-04-25T10:12:36.438100+0000","last_unstale":"2024-04-25T10:12:38.477926+0000","last_undegraded":"2024-04-25T10:12:38.477926+0000","last_fullsized":"2024-04-25T10:12:38.477926+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:30.172798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658108+0000","last_change":"2024-04-25T10:12:37.441135+0000","last_active":"2024-04-25T10:12:38.658108+0000","last_peered":"2024-04-25T10:12:38.658108+0000","last_clean":"2024-04-25T10:12:38.658108+0000","last_became_active":"2024-04-25T10:12:37.440844+0000","last_became_peered":"2024-04-25T10:12:37.440844+0000","last_unstale":"2024-04-25T10:12:38.658108+0000","last_undegraded":"2024-04-25T10:12:38.658108+0000","last_fullsized":"2024-04-25T10:12:38.658108+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420801+0000","last_change":"2024-04-25T10:12:36.434095+0000","last_active":"2024-04-25T10:12:38.420801+0000","last_peered":"2024-04-25T10:12:38.420801+0000","last_clean":"2024-04-25T10:12:38.420801+0000","last_became_active":"2024-04-25T10:12:36.433803+0000","last_became_peered":"2024-04-25T10:12:36.433803+0000","last_unstale":"2024-04-25T10:12:38.420801+0000","last_undegraded":"2024-04-25T10:12:38.420801+0000","last_fullsized":"2024-04-25T10:12:38.420801+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657583+0000","last_change":"2024-04-25T10:12:37.431496+0000","last_active":"2024-04-25T10:12:38.657583+0000","last_peered":"2024-04-25T10:12:38.657583+0000","last_clean":"2024-04-25T10:12:38.657583+0000","last_became_active":"2024-04-25T10:12:37.430520+0000","last_became_peered":"2024-04-25T10:12:37.430520+0000","last_unstale":"2024-04-25T10:12:38.657583+0000","last_undegraded":"2024-04-25T10:12:38.657583+0000","last_fullsized":"2024-04-25T10:12:38.657583+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:44.202217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658293+0000","last_change":"2024-04-25T10:12:36.420615+0000","last_active":"2024-04-25T10:12:38.658293+0000","last_peered":"2024-04-25T10:12:38.658293+0000","last_clean":"2024-04-25T10:12:38.658293+0000","last_became_active":"2024-04-25T10:12:36.420089+0000","last_became_peered":"2024-04-25T10:12:36.420089+0000","last_unstale":"2024-04-25T10:12:38.658293+0000","last_undegraded":"2024-04-25T10:12:38.658293+0000","last_fullsized":"2024-04-25T10:12:38.658293+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:54.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421871+0000","last_change":"2024-04-25T10:12:37.439710+0000","last_active":"2024-04-25T10:12:38.421871+0000","last_peered":"2024-04-25T10:12:38.421871+0000","last_clean":"2024-04-25T10:12:38.421871+0000","last_became_active":"2024-04-25T10:12:37.439422+0000","last_became_peered":"2024-04-25T10:12:37.439422+0000","last_unstale":"2024-04-25T10:12:38.421871+0000","last_undegraded":"2024-04-25T10:12:38.421871+0000","last_fullsized":"2024-04-25T10:12:38.421871+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:28.724635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414588+0000","last_change":"2024-04-25T10:12:36.432744+0000","last_active":"2024-04-25T10:12:38.414588+0000","last_peered":"2024-04-25T10:12:38.414588+0000","last_clean":"2024-04-25T10:12:38.414588+0000","last_became_active":"2024-04-25T10:12:36.431971+0000","last_became_peered":"2024-04-25T10:12:36.431971+0000","last_unstale":"2024-04-25T10:12:38.414588+0000","last_undegraded":"2024-04-25T10:12:38.414588+0000","last_fullsized":"2024-04-25T10:12:38.414588+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659025+0000","last_change":"2024-04-25T10:12:37.425673+0000","last_active":"2024-04-25T10:12:38.659025+0000","last_peered":"2024-04-25T10:12:38.659025+0000","last_clean":"2024-04-25T10:12:38.659025+0000","last_became_active":"2024-04-25T10:12:37.425265+0000","last_became_peered":"2024-04-25T10:12:37.425265+0000","last_unstale":"2024-04-25T10:12:38.659025+0000","last_undegraded":"2024-04-25T10:12:38.659025+0000","last_fullsized":"2024-04-25T10:12:38.659025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:48.545973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659783+0000","last_change":"2024-04-25T10:12:36.429891+0000","last_active":"2024-04-25T10:12:38.659783+0000","last_peered":"2024-04-25T10:12:38.659783+0000","last_clean":"2024-04-25T10:12:38.659783+0000","last_became_active":"2024-04-25T10:12:36.429336+0000","last_became_peered":"2024-04-25T10:12:36.429336+0000","last_unstale":"2024-04-25T10:12:38.659783+0000","last_undegraded":"2024-04-25T10:12:38.659783+0000","last_fullsized":"2024-04-25T10:12:38.659783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:54.333585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660854+0000","last_change":"2024-04-25T10:12:37.440609+0000","last_active":"2024-04-25T10:12:38.660854+0000","last_peered":"2024-04-25T10:12:38.660854+0000","last_clean":"2024-04-25T10:12:38.660854+0000","last_became_active":"2024-04-25T10:12:37.439917+0000","last_became_peered":"2024-04-25T10:12:37.439917+0000","last_unstale":"2024-04-25T10:12:38.660854+0000","last_undegraded":"2024-04-25T10:12:38.660854+0000","last_fullsized":"2024-04-25T10:12:38.660854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:03.858903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":22,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417179+0000","last_change":"2024-04-25T10:12:36.434858+0000","last_active":"2024-04-25T10:12:38.417179+0000","last_peered":"2024-04-25T10:12:38.417179+0000","last_clean":"2024-04-25T10:12:38.417179+0000","last_became_active":"2024-04-25T10:12:36.434571+0000","last_became_peered":"2024-04-25T10:12:36.434571+0000","last_unstale":"2024-04-25T10:12:38.417179+0000","last_undegraded":"2024-04-25T10:12:38.417179+0000","last_fullsized":"2024-04-25T10:12:38.417179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657357+0000","last_change":"2024-04-25T10:12:37.430884+0000","last_active":"2024-04-25T10:12:38.657357+0000","last_peered":"2024-04-25T10:12:38.657357+0000","last_clean":"2024-04-25T10:12:38.657357+0000","last_became_active":"2024-04-25T10:12:37.428118+0000","last_became_peered":"2024-04-25T10:12:37.428118+0000","last_unstale":"2024-04-25T10:12:38.657357+0000","last_undegraded":"2024-04-25T10:12:38.657357+0000","last_fullsized":"2024-04-25T10:12:38.657357+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:36.032449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421873+0000","last_change":"2024-04-25T10:12:36.437073+0000","last_active":"2024-04-25T10:12:38.421873+0000","last_peered":"2024-04-25T10:12:38.421873+0000","last_clean":"2024-04-25T10:12:38.421873+0000","last_became_active":"2024-04-25T10:12:36.436575+0000","last_became_peered":"2024-04-25T10:12:36.436575+0000","last_unstale":"2024-04-25T10:12:38.421873+0000","last_undegraded":"2024-04-25T10:12:38.421873+0000","last_fullsized":"2024-04-25T10:12:38.421873+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.986763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421317+0000","last_change":"2024-04-25T10:12:37.446967+0000","last_active":"2024-04-25T10:12:38.421317+0000","last_peered":"2024-04-25T10:12:38.421317+0000","last_clean":"2024-04-25T10:12:38.421317+0000","last_became_active":"2024-04-25T10:12:37.446664+0000","last_became_peered":"2024-04-25T10:12:37.446664+0000","last_unstale":"2024-04-25T10:12:38.421317+0000","last_undegraded":"2024-04-25T10:12:38.421317+0000","last_fullsized":"2024-04-25T10:12:38.421317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477978+0000","last_change":"2024-04-25T10:12:36.430895+0000","last_active":"2024-04-25T10:12:38.477978+0000","last_peered":"2024-04-25T10:12:38.477978+0000","last_clean":"2024-04-25T10:12:38.477978+0000","last_became_active":"2024-04-25T10:12:36.430565+0000","last_became_peered":"2024-04-25T10:12:36.430565+0000","last_unstale":"2024-04-25T10:12:38.477978+0000","last_undegraded":"2024-04-25T10:12:38.477978+0000","last_fullsized":"2024-04-25T10:12:38.477978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657946+0000","last_change":"2024-04-25T10:12:37.436317+0000","last_active":"2024-04-25T10:12:38.657946+0000","last_peered":"2024-04-25T10:12:38.657946+0000","last_clean":"2024-04-25T10:12:38.657946+0000","last_became_active":"2024-04-25T10:12:37.435976+0000","last_became_peered":"2024-04-25T10:12:37.435976+0000","last_unstale":"2024-04-25T10:12:38.657946+0000","last_undegraded":"2024-04-25T10:12:38.657946+0000","last_fullsized":"2024-04-25T10:12:38.657946+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657682+0000","last_change":"2024-04-25T10:12:36.438473+0000","last_active":"2024-04-25T10:12:38.657682+0000","last_peered":"2024-04-25T10:12:38.657682+0000","last_clean":"2024-04-25T10:12:38.657682+0000","last_became_active":"2024-04-25T10:12:36.436411+0000","last_became_peered":"2024-04-25T10:12:36.436411+0000","last_unstale":"2024-04-25T10:12:38.657682+0000","last_undegraded":"2024-04-25T10:12:38.657682+0000","last_fullsized":"2024-04-25T10:12:38.657682+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:52.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659307+0000","last_change":"2024-04-25T10:12:37.445297+0000","last_active":"2024-04-25T10:12:38.659307+0000","last_peered":"2024-04-25T10:12:38.659307+0000","last_clean":"2024-04-25T10:12:38.659307+0000","last_became_active":"2024-04-25T10:12:37.444851+0000","last_became_peered":"2024-04-25T10:12:37.444851+0000","last_unstale":"2024-04-25T10:12:38.659307+0000","last_undegraded":"2024-04-25T10:12:38.659307+0000","last_fullsized":"2024-04-25T10:12:38.659307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:24.622841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657071+0000","last_change":"2024-04-25T10:12:36.428322+0000","last_active":"2024-04-25T10:12:38.657071+0000","last_peered":"2024-04-25T10:12:38.657071+0000","last_clean":"2024-04-25T10:12:38.657071+0000","last_became_active":"2024-04-25T10:12:36.427541+0000","last_became_peered":"2024-04-25T10:12:36.427541+0000","last_unstale":"2024-04-25T10:12:38.657071+0000","last_undegraded":"2024-04-25T10:12:38.657071+0000","last_fullsized":"2024-04-25T10:12:38.657071+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:26.382466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660381+0000","last_change":"2024-04-25T10:12:37.438627+0000","last_active":"2024-04-25T10:12:38.660381+0000","last_peered":"2024-04-25T10:12:38.660381+0000","last_clean":"2024-04-25T10:12:38.660381+0000","last_became_active":"2024-04-25T10:12:37.438352+0000","last_became_peered":"2024-04-25T10:12:37.438352+0000","last_unstale":"2024-04-25T10:12:38.660381+0000","last_undegraded":"2024-04-25T10:12:38.660381+0000","last_fullsized":"2024-04-25T10:12:38.660381+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.577771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656597+0000","last_change":"2024-04-25T10:12:36.420411+0000","last_active":"2024-04-25T10:12:38.656597+0000","last_peered":"2024-04-25T10:12:38.656597+0000","last_clean":"2024-04-25T10:12:38.656597+0000","last_became_active":"2024-04-25T10:12:36.419903+0000","last_became_peered":"2024-04-25T10:12:36.419903+0000","last_unstale":"2024-04-25T10:12:38.656597+0000","last_undegraded":"2024-04-25T10:12:38.656597+0000","last_fullsized":"2024-04-25T10:12:38.656597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:43.819864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414447+0000","last_change":"2024-04-25T10:12:37.452031+0000","last_active":"2024-04-25T10:12:38.414447+0000","last_peered":"2024-04-25T10:12:38.414447+0000","last_clean":"2024-04-25T10:12:38.414447+0000","last_became_active":"2024-04-25T10:12:37.451729+0000","last_became_peered":"2024-04-25T10:12:37.451729+0000","last_unstale":"2024-04-25T10:12:38.414447+0000","last_undegraded":"2024-04-25T10:12:38.414447+0000","last_fullsized":"2024-04-25T10:12:38.414447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.311656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420290+0000","last_change":"2024-04-25T10:12:36.435352+0000","last_active":"2024-04-25T10:12:38.420290+0000","last_peered":"2024-04-25T10:12:38.420290+0000","last_clean":"2024-04-25T10:12:38.420290+0000","last_became_active":"2024-04-25T10:12:36.434943+0000","last_became_peered":"2024-04-25T10:12:36.434943+0000","last_unstale":"2024-04-25T10:12:38.420290+0000","last_undegraded":"2024-04-25T10:12:38.420290+0000","last_fullsized":"2024-04-25T10:12:38.420290+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657392+0000","last_change":"2024-04-25T10:12:37.437540+0000","last_active":"2024-04-25T10:12:38.657392+0000","last_peered":"2024-04-25T10:12:38.657392+0000","last_clean":"2024-04-25T10:12:38.657392+0000","last_became_active":"2024-04-25T10:12:37.437266+0000","last_became_peered":"2024-04-25T10:12:37.437266+0000","last_unstale":"2024-04-25T10:12:38.657392+0000","last_undegraded":"2024-04-25T10:12:38.657392+0000","last_fullsized":"2024-04-25T10:12:38.657392+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.631631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658976+0000","last_change":"2024-04-25T10:12:36.419478+0000","last_active":"2024-04-25T10:12:38.658976+0000","last_peered":"2024-04-25T10:12:38.658976+0000","last_clean":"2024-04-25T10:12:38.658976+0000","last_became_active":"2024-04-25T10:12:36.419010+0000","last_became_peered":"2024-04-25T10:12:36.419010+0000","last_unstale":"2024-04-25T10:12:38.658976+0000","last_undegraded":"2024-04-25T10:12:38.658976+0000","last_fullsized":"2024-04-25T10:12:38.658976+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:30.628822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658013+0000","last_change":"2024-04-25T10:12:37.439593+0000","last_active":"2024-04-25T10:12:38.658013+0000","last_peered":"2024-04-25T10:12:38.658013+0000","last_clean":"2024-04-25T10:12:38.658013+0000","last_became_active":"2024-04-25T10:12:37.438897+0000","last_became_peered":"2024-04-25T10:12:37.438897+0000","last_unstale":"2024-04-25T10:12:38.658013+0000","last_undegraded":"2024-04-25T10:12:38.658013+0000","last_fullsized":"2024-04-25T10:12:38.658013+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.701854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660240+0000","last_change":"2024-04-25T10:12:36.424429+0000","last_active":"2024-04-25T10:12:38.660240+0000","last_peered":"2024-04-25T10:12:38.660240+0000","last_clean":"2024-04-25T10:12:38.660240+0000","last_became_active":"2024-04-25T10:12:36.423878+0000","last_became_peered":"2024-04-25T10:12:36.423878+0000","last_unstale":"2024-04-25T10:12:38.660240+0000","last_undegraded":"2024-04-25T10:12:38.660240+0000","last_fullsized":"2024-04-25T10:12:38.660240+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:48.057318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417215+0000","last_change":"2024-04-25T10:12:37.448434+0000","last_active":"2024-04-25T10:12:38.417215+0000","last_peered":"2024-04-25T10:12:38.417215+0000","last_clean":"2024-04-25T10:12:38.417215+0000","last_became_active":"2024-04-25T10:12:37.447902+0000","last_became_peered":"2024-04-25T10:12:37.447902+0000","last_unstale":"2024-04-25T10:12:38.417215+0000","last_undegraded":"2024-04-25T10:12:38.417215+0000","last_fullsized":"2024-04-25T10:12:38.417215+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:56.964062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660034+0000","last_change":"2024-04-25T10:12:36.440745+0000","last_active":"2024-04-25T10:12:38.660034+0000","last_peered":"2024-04-25T10:12:38.660034+0000","last_clean":"2024-04-25T10:12:38.660034+0000","last_became_active":"2024-04-25T10:12:36.440448+0000","last_became_peered":"2024-04-25T10:12:36.440448+0000","last_unstale":"2024-04-25T10:12:38.660034+0000","last_undegraded":"2024-04-25T10:12:38.660034+0000","last_fullsized":"2024-04-25T10:12:38.660034+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:21.234143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.413900+0000","last_change":"2024-04-25T10:12:37.432087+0000","last_active":"2024-04-25T10:12:38.413900+0000","last_peered":"2024-04-25T10:12:38.413900+0000","last_clean":"2024-04-25T10:12:38.413900+0000","last_became_active":"2024-04-25T10:12:37.431814+0000","last_became_peered":"2024-04-25T10:12:37.431814+0000","last_unstale":"2024-04-25T10:12:38.413900+0000","last_undegraded":"2024-04-25T10:12:38.413900+0000","last_fullsized":"2024-04-25T10:12:38.413900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:19.357372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.422143+0000","last_change":"2024-04-25T10:12:36.430380+0000","last_active":"2024-04-25T10:12:38.422143+0000","last_peered":"2024-04-25T10:12:38.422143+0000","last_clean":"2024-04-25T10:12:38.422143+0000","last_became_active":"2024-04-25T10:12:36.429347+0000","last_became_peered":"2024-04-25T10:12:36.429347+0000","last_unstale":"2024-04-25T10:12:38.422143+0000","last_undegraded":"2024-04-25T10:12:38.422143+0000","last_fullsized":"2024-04-25T10:12:38.422143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657557+0000","last_change":"2024-04-25T10:12:37.436383+0000","last_active":"2024-04-25T10:12:38.657557+0000","last_peered":"2024-04-25T10:12:38.657557+0000","last_clean":"2024-04-25T10:12:38.657557+0000","last_became_active":"2024-04-25T10:12:37.436077+0000","last_became_peered":"2024-04-25T10:12:37.436077+0000","last_unstale":"2024-04-25T10:12:38.657557+0000","last_undegraded":"2024-04-25T10:12:38.657557+0000","last_fullsized":"2024-04-25T10:12:38.657557+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:53.082404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658706+0000","last_change":"2024-04-25T10:12:36.434613+0000","last_active":"2024-04-25T10:12:38.658706+0000","last_peered":"2024-04-25T10:12:38.658706+0000","last_clean":"2024-04-25T10:12:38.658706+0000","last_became_active":"2024-04-25T10:12:36.434287+0000","last_became_peered":"2024-04-25T10:12:36.434287+0000","last_unstale":"2024-04-25T10:12:38.658706+0000","last_undegraded":"2024-04-25T10:12:38.658706+0000","last_fullsized":"2024-04-25T10:12:38.658706+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:00.295480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656876+0000","last_change":"2024-04-25T10:12:37.442105+0000","last_active":"2024-04-25T10:12:38.656876+0000","last_peered":"2024-04-25T10:12:38.656876+0000","last_clean":"2024-04-25T10:12:38.656876+0000","last_became_active":"2024-04-25T10:12:37.440748+0000","last_became_peered":"2024-04-25T10:12:37.440748+0000","last_unstale":"2024-04-25T10:12:38.656876+0000","last_undegraded":"2024-04-25T10:12:38.656876+0000","last_fullsized":"2024-04-25T10:12:38.656876+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.046065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419799+0000","last_change":"2024-04-25T10:12:36.429818+0000","last_active":"2024-04-25T10:12:38.419799+0000","last_peered":"2024-04-25T10:12:38.419799+0000","last_clean":"2024-04-25T10:12:38.419799+0000","last_became_active":"2024-04-25T10:12:36.429160+0000","last_became_peered":"2024-04-25T10:12:36.429160+0000","last_unstale":"2024-04-25T10:12:38.419799+0000","last_undegraded":"2024-04-25T10:12:38.419799+0000","last_fullsized":"2024-04-25T10:12:38.419799+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659576+0000","last_change":"2024-04-25T10:12:37.434576+0000","last_active":"2024-04-25T10:12:38.659576+0000","last_peered":"2024-04-25T10:12:38.659576+0000","last_clean":"2024-04-25T10:12:38.659576+0000","last_became_active":"2024-04-25T10:12:37.434299+0000","last_became_peered":"2024-04-25T10:12:37.434299+0000","last_unstale":"2024-04-25T10:12:38.659576+0000","last_undegraded":"2024-04-25T10:12:38.659576+0000","last_fullsized":"2024-04-25T10:12:38.659576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:01.151422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656929+0000","last_change":"2024-04-25T10:12:36.428404+0000","last_active":"2024-04-25T10:12:38.656929+0000","last_peered":"2024-04-25T10:12:38.656929+0000","last_clean":"2024-04-25T10:12:38.656929+0000","last_became_active":"2024-04-25T10:12:36.428103+0000","last_became_peered":"2024-04-25T10:12:36.428103+0000","last_unstale":"2024-04-25T10:12:38.656929+0000","last_undegraded":"2024-04-25T10:12:38.656929+0000","last_fullsized":"2024-04-25T10:12:38.656929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:47.198001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.478234+0000","last_change":"2024-04-25T10:12:37.433986+0000","last_active":"2024-04-25T10:12:38.478234+0000","last_peered":"2024-04-25T10:12:38.478234+0000","last_clean":"2024-04-25T10:12:38.478234+0000","last_became_active":"2024-04-25T10:12:37.433467+0000","last_became_peered":"2024-04-25T10:12:37.433467+0000","last_unstale":"2024-04-25T10:12:38.478234+0000","last_undegraded":"2024-04-25T10:12:38.478234+0000","last_fullsized":"2024-04-25T10:12:38.478234+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.311656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420572+0000","last_change":"2024-04-25T10:12:36.423055+0000","last_active":"2024-04-25T10:12:38.420572+0000","last_peered":"2024-04-25T10:12:38.420572+0000","last_clean":"2024-04-25T10:12:38.420572+0000","last_became_active":"2024-04-25T10:12:36.422751+0000","last_became_peered":"2024-04-25T10:12:36.422751+0000","last_unstale":"2024-04-25T10:12:38.420572+0000","last_undegraded":"2024-04-25T10:12:38.420572+0000","last_fullsized":"2024-04-25T10:12:38.420572+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658454+0000","last_change":"2024-04-25T10:12:37.446201+0000","last_active":"2024-04-25T10:12:38.658454+0000","last_peered":"2024-04-25T10:12:38.658454+0000","last_clean":"2024-04-25T10:12:38.658454+0000","last_became_active":"2024-04-25T10:12:37.440114+0000","last_became_peered":"2024-04-25T10:12:37.440114+0000","last_unstale":"2024-04-25T10:12:38.658454+0000","last_undegraded":"2024-04-25T10:12:38.658454+0000","last_fullsized":"2024-04-25T10:12:38.658454+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:40.183842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659195+0000","last_change":"2024-04-25T10:12:36.426977+0000","last_active":"2024-04-25T10:12:38.659195+0000","last_peered":"2024-04-25T10:12:38.659195+0000","last_clean":"2024-04-25T10:12:38.659195+0000","last_became_active":"2024-04-25T10:12:36.424814+0000","last_became_peered":"2024-04-25T10:12:36.424814+0000","last_unstale":"2024-04-25T10:12:38.659195+0000","last_undegraded":"2024-04-25T10:12:38.659195+0000","last_fullsized":"2024-04-25T10:12:38.659195+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:28.338534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657375+0000","last_change":"2024-04-25T10:12:37.437126+0000","last_active":"2024-04-25T10:12:38.657375+0000","last_peered":"2024-04-25T10:12:38.657375+0000","last_clean":"2024-04-25T10:12:38.657375+0000","last_became_active":"2024-04-25T10:12:37.436835+0000","last_became_peered":"2024-04-25T10:12:37.436835+0000","last_unstale":"2024-04-25T10:12:38.657375+0000","last_undegraded":"2024-04-25T10:12:38.657375+0000","last_fullsized":"2024-04-25T10:12:38.657375+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:27.411214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659298+0000","last_change":"2024-04-25T10:12:36.419237+0000","last_active":"2024-04-25T10:12:38.659298+0000","last_peered":"2024-04-25T10:12:38.659298+0000","last_clean":"2024-04-25T10:12:38.659298+0000","last_became_active":"2024-04-25T10:12:36.418640+0000","last_became_peered":"2024-04-25T10:12:36.418640+0000","last_unstale":"2024-04-25T10:12:38.659298+0000","last_undegraded":"2024-04-25T10:12:38.659298+0000","last_fullsized":"2024-04-25T10:12:38.659298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.852848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477396+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:12:38.477396+0000","last_peered":"2024-04-25T10:12:38.477396+0000","last_clean":"2024-04-25T10:12:38.477396+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:12:38.477396+0000","last_undegraded":"2024-04-25T10:12:38.477396+0000","last_fullsized":"2024-04-25T10:12:38.477396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":1978368,"data_stored":1234135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50600,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321240,"statfs":{"total":96636764160,"available":96584949760,"internally_reserved":0,"allocated":1396736,"data_stored":645823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50580,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321260,"statfs":{"total":96636764160,"available":96584970240,"internally_reserved":0,"allocated":1376256,"data_stored":643607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1384448,"data_stored":643699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58896,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312944,"statfs":{"total":96636764160,"available":96576454656,"internally_reserved":0,"allocated":1380352,"data_stored":644189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54744,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317096,"statfs":{"total":96636764160,"available":96580706304,"internally_reserved":0,"allocated":1380352,"data_stored":643641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50576,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321264,"statfs":{"total":96636764160,"available":96584974336,"internally_reserved":0,"allocated":1380352,"data_stored":643641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":1986560,"data_stored":1235677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:46.835 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:12:47.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.042+0000 7f4763200640 1 Processor -- start 2024-04-25T10:12:47.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.043+0000 7f4763200640 1 -- start start 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4763200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f476415db90 0x7f47640b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4763200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47641615a0 0x7f47640cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4763200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f4764061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4763200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47640d0980 con 0x7f476415db90 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4763200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47640d1f50 con 0x7f47641615a0 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4763200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4764161df0 con 0x7f47640d04f0 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4761e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f476415db90 0x7f47640b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4761e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f476415db90 0x7f47640b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51538/0 (socket says 172.21.15.165:51538) 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4761e00640 1 -- 172.21.15.165:0/616722382 learned_addr learned my addr 172.21.15.165:0/616722382 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.044+0000 7f4761e00640 1 -- 172.21.15.165:0/616722382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 msgr2=0x7f4764061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.045+0000 7f4761e00640 1 --2- 172.21.15.165:0/616722382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f4764061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.045+0000 7f4761e00640 1 -- 172.21.15.165:0/616722382 >> v2:172.21.15.172:3300/0 conn(0x7f47641615a0 msgr2=0x7f47640cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.045+0000 7f4761e00640 1 --2- 172.21.15.165:0/616722382 >> v2:172.21.15.172:3300/0 conn(0x7f47641615a0 0x7f47640cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.045+0000 7f4761e00640 1 -- 172.21.15.165:0/616722382 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47641620d0 con 0x7f476415db90 2024-04-25T10:12:47.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.046+0000 7f4761e00640 1 --2- 172.21.15.165:0/616722382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640b93c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f47640d1b30 tx=0x7f475402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9518c338c854bb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.046+0000 7f4760a00640 1 -- 172.21.15.165:0/616722382 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754037070 con 0x7f476415db90 2024-04-25T10:12:47.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.046+0000 7f4760a00640 1 -- 172.21.15.165:0/616722382 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4754032440 con 0x7f476415db90 2024-04-25T10:12:47.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.046+0000 7f4760a00640 1 -- 172.21.15.165:0/616722382 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4754032810 con 0x7f476415db90 2024-04-25T10:12:47.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.046+0000 7f4763200640 1 -- 172.21.15.165:0/616722382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 msgr2=0x7f47640b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.046+0000 7f4763200640 1 --2- 172.21.15.165:0/616722382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640b93c0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f47640d1b30 tx=0x7f475402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 -- 172.21.15.165:0/616722382 shutdown_connections 2024-04-25T10:12:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 --2- 172.21.15.165:0/616722382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f4764061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 --2- 172.21.15.165:0/616722382 >> v2:172.21.15.172:3300/0 conn(0x7f47641615a0 0x7f47640cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 --2- 172.21.15.165:0/616722382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640b93c0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 -- 172.21.15.165:0/616722382 >> 172.21.15.165:0/616722382 conn(0x7f47640b52b0 msgr2=0x7f47640b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 -- 172.21.15.165:0/616722382 shutdown_connections 2024-04-25T10:12:47.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.047+0000 7f4763200640 1 -- 172.21.15.165:0/616722382 wait complete. 2024-04-25T10:12:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.048+0000 7f4763200640 1 Processor -- start 2024-04-25T10:12:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 -- start start 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f47640cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640d2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47641615a0 0x7f47641516f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4764161a20 con 0x7f476415db90 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47640d1be0 con 0x7f47641615a0 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.049+0000 7f4763200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4764161ba0 con 0x7f47640d04f0 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640d2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640d2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51540/0 (socket says 172.21.15.165:51540) 2024-04-25T10:12:47.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 -- 172.21.15.165:0/3007321156 learned_addr learned my addr 172.21.15.165:0/3007321156 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761e00640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f47640cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 -- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 msgr2=0x7f47640cc810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f47640cc810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 -- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47641615a0 msgr2=0x7f47641516f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47641615a0 0x7f47641516f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761400640 1 -- 172.21.15.165:0/3007321156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47640ccd50 con 0x7f476415db90 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.050+0000 7f4761e00640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f47640cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.051+0000 7f4761400640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640d2650 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f474c00b410 tx=0x7f474c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.051+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c015020 con 0x7f476415db90 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.051+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f474c011850 con 0x7f476415db90 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.051+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f474c011c40 con 0x7f476415db90 2024-04-25T10:12:47.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.051+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47640cb560 con 0x7f476415db90 2024-04-25T10:12:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.051+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47640cc010 con 0x7f476415db90 2024-04-25T10:12:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.052+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47640cbb20 con 0x7f476415db90 2024-04-25T10:12:47.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.053+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f474c018020 con 0x7f476415db90 2024-04-25T10:12:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.054+0000 7f475a000640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f474006b4d0 0x7f474006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.054+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f474c08a2d0 con 0x7f476415db90 2024-04-25T10:12:47.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.054+0000 7f4761e00640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f474006b4d0 0x7f474006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.055+0000 7f4761e00640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f474006b4d0 0x7f474006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4754034000 tx=0x7f475402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.059+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f474c01f020 con 0x7f476415db90 2024-04-25T10:12:47.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.250+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47640d14b0 con 0x7f474006b4d0 2024-04-25T10:12:47.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.259+0000 7f475a000640 1 -- 172.21.15.165:0/3007321156 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f47640d14b0 con 0x7f474006b4d0 2024-04-25T10:12:47.268 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:47.269 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f474006b4d0 msgr2=0x7f474006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f474006b4d0 0x7f474006d990 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4754034000 tx=0x7f475402e000 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 msgr2=0x7f47640d2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640d2650 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f474c00b410 tx=0x7f474c00b8d0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 shutdown_connections 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f474006b4d0 0x7f474006d990 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47641615a0 0x7f47641516f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f476415db90 0x7f47640d2650 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 --2- 172.21.15.165:0/3007321156 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47640d04f0 0x7f47640cc810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 >> 172.21.15.165:0/3007321156 conn(0x7f47640b52b0 msgr2=0x7f47640b5690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 shutdown_connections 2024-04-25T10:12:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.266+0000 7f4763200640 1 -- 172.21.15.165:0/3007321156 wait complete. 2024-04-25T10:12:47.290 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":431,"stamp":"2024-04-25T10:12:46.347195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":430804,"kb_used_data":11976,"kb_used_omap":12,"kb_used_meta":418675,"kb_avail":754543916,"statfs":{"total":773094113280,"available":772652969984,"internally_reserved":0,"allocated":12263424,"data_stored":6334412,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":428723762},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.937821"},"pg_stats":[{"pgid":"16.3f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660408+0000","last_change":"2024-04-25T10:12:37.440175+0000","last_active":"2024-04-25T10:12:38.660408+0000","last_peered":"2024-04-25T10:12:38.660408+0000","last_clean":"2024-04-25T10:12:38.660408+0000","last_became_active":"2024-04-25T10:12:37.437927+0000","last_became_peered":"2024-04-25T10:12:37.437927+0000","last_unstale":"2024-04-25T10:12:38.660408+0000","last_undegraded":"2024-04-25T10:12:38.660408+0000","last_fullsized":"2024-04-25T10:12:38.660408+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:22:47.423921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421866+0000","last_change":"2024-04-25T10:12:37.442081+0000","last_active":"2024-04-25T10:12:38.421866+0000","last_peered":"2024-04-25T10:12:38.421866+0000","last_clean":"2024-04-25T10:12:38.421866+0000","last_became_active":"2024-04-25T10:12:37.440679+0000","last_became_peered":"2024-04-25T10:12:37.440679+0000","last_unstale":"2024-04-25T10:12:38.421866+0000","last_undegraded":"2024-04-25T10:12:38.421866+0000","last_fullsized":"2024-04-25T10:12:38.421866+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:43.163133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660454+0000","last_change":"2024-04-25T10:12:37.443092+0000","last_active":"2024-04-25T10:12:38.660454+0000","last_peered":"2024-04-25T10:12:38.660454+0000","last_clean":"2024-04-25T10:12:38.660454+0000","last_became_active":"2024-04-25T10:12:37.442779+0000","last_became_peered":"2024-04-25T10:12:37.442779+0000","last_unstale":"2024-04-25T10:12:38.660454+0000","last_undegraded":"2024-04-25T10:12:38.660454+0000","last_fullsized":"2024-04-25T10:12:38.660454+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:43.318742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414964+0000","last_change":"2024-04-25T10:12:37.430317+0000","last_active":"2024-04-25T10:12:38.414964+0000","last_peered":"2024-04-25T10:12:38.414964+0000","last_clean":"2024-04-25T10:12:38.414964+0000","last_became_active":"2024-04-25T10:12:37.430009+0000","last_became_peered":"2024-04-25T10:12:37.430009+0000","last_unstale":"2024-04-25T10:12:38.414964+0000","last_undegraded":"2024-04-25T10:12:38.414964+0000","last_fullsized":"2024-04-25T10:12:38.414964+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:22.585606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657765+0000","last_change":"2024-04-25T10:12:37.443661+0000","last_active":"2024-04-25T10:12:38.657765+0000","last_peered":"2024-04-25T10:12:38.657765+0000","last_clean":"2024-04-25T10:12:38.657765+0000","last_became_active":"2024-04-25T10:12:37.443277+0000","last_became_peered":"2024-04-25T10:12:37.443277+0000","last_unstale":"2024-04-25T10:12:38.657765+0000","last_undegraded":"2024-04-25T10:12:38.657765+0000","last_fullsized":"2024-04-25T10:12:38.657765+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421328+0000","last_change":"2024-04-25T10:12:37.446343+0000","last_active":"2024-04-25T10:12:38.421328+0000","last_peered":"2024-04-25T10:12:38.421328+0000","last_clean":"2024-04-25T10:12:38.421328+0000","last_became_active":"2024-04-25T10:12:37.444637+0000","last_became_peered":"2024-04-25T10:12:37.444637+0000","last_unstale":"2024-04-25T10:12:38.421328+0000","last_undegraded":"2024-04-25T10:12:38.421328+0000","last_fullsized":"2024-04-25T10:12:38.421328+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418202+0000","last_change":"2024-04-25T10:12:37.443934+0000","last_active":"2024-04-25T10:12:38.418202+0000","last_peered":"2024-04-25T10:12:38.418202+0000","last_clean":"2024-04-25T10:12:38.418202+0000","last_became_active":"2024-04-25T10:12:37.443387+0000","last_became_peered":"2024-04-25T10:12:37.443387+0000","last_unstale":"2024-04-25T10:12:38.418202+0000","last_undegraded":"2024-04-25T10:12:38.418202+0000","last_fullsized":"2024-04-25T10:12:38.418202+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:56.964062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658734+0000","last_change":"2024-04-25T10:12:37.434890+0000","last_active":"2024-04-25T10:12:38.658734+0000","last_peered":"2024-04-25T10:12:38.658734+0000","last_clean":"2024-04-25T10:12:38.658734+0000","last_became_active":"2024-04-25T10:12:37.434607+0000","last_became_peered":"2024-04-25T10:12:37.434607+0000","last_unstale":"2024-04-25T10:12:38.658734+0000","last_undegraded":"2024-04-25T10:12:38.658734+0000","last_fullsized":"2024-04-25T10:12:38.658734+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:50.823364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421703+0000","last_change":"2024-04-25T10:12:37.440646+0000","last_active":"2024-04-25T10:12:38.421703+0000","last_peered":"2024-04-25T10:12:38.421703+0000","last_clean":"2024-04-25T10:12:38.421703+0000","last_became_active":"2024-04-25T10:12:37.440279+0000","last_became_peered":"2024-04-25T10:12:37.440279+0000","last_unstale":"2024-04-25T10:12:38.421703+0000","last_undegraded":"2024-04-25T10:12:38.421703+0000","last_fullsized":"2024-04-25T10:12:38.421703+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658834+0000","last_change":"2024-04-25T10:12:37.451524+0000","last_active":"2024-04-25T10:12:38.658834+0000","last_peered":"2024-04-25T10:12:38.658834+0000","last_clean":"2024-04-25T10:12:38.658834+0000","last_became_active":"2024-04-25T10:12:37.450715+0000","last_became_peered":"2024-04-25T10:12:37.450715+0000","last_unstale":"2024-04-25T10:12:38.658834+0000","last_undegraded":"2024-04-25T10:12:38.658834+0000","last_fullsized":"2024-04-25T10:12:38.658834+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658039+0000","last_change":"2024-04-25T10:12:37.446411+0000","last_active":"2024-04-25T10:12:38.658039+0000","last_peered":"2024-04-25T10:12:38.658039+0000","last_clean":"2024-04-25T10:12:38.658039+0000","last_became_active":"2024-04-25T10:12:37.445773+0000","last_became_peered":"2024-04-25T10:12:37.445773+0000","last_unstale":"2024-04-25T10:12:38.658039+0000","last_undegraded":"2024-04-25T10:12:38.658039+0000","last_fullsized":"2024-04-25T10:12:38.658039+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:21.571051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418659+0000","last_change":"2024-04-25T10:12:37.449685+0000","last_active":"2024-04-25T10:12:38.418659+0000","last_peered":"2024-04-25T10:12:38.418659+0000","last_clean":"2024-04-25T10:12:38.418659+0000","last_became_active":"2024-04-25T10:12:37.448670+0000","last_became_peered":"2024-04-25T10:12:37.448670+0000","last_unstale":"2024-04-25T10:12:38.418659+0000","last_undegraded":"2024-04-25T10:12:38.418659+0000","last_fullsized":"2024-04-25T10:12:38.418659+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.413874+0000","last_change":"2024-04-25T10:12:37.441928+0000","last_active":"2024-04-25T10:12:38.413874+0000","last_peered":"2024-04-25T10:12:38.413874+0000","last_clean":"2024-04-25T10:12:38.413874+0000","last_became_active":"2024-04-25T10:12:37.441644+0000","last_became_peered":"2024-04-25T10:12:37.441644+0000","last_unstale":"2024-04-25T10:12:38.413874+0000","last_undegraded":"2024-04-25T10:12:38.413874+0000","last_fullsized":"2024-04-25T10:12:38.413874+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:12.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477504+0000","last_change":"2024-04-25T10:12:37.434637+0000","last_active":"2024-04-25T10:12:38.477504+0000","last_peered":"2024-04-25T10:12:38.477504+0000","last_clean":"2024-04-25T10:12:38.477504+0000","last_became_active":"2024-04-25T10:12:37.434374+0000","last_became_peered":"2024-04-25T10:12:37.434374+0000","last_unstale":"2024-04-25T10:12:38.477504+0000","last_undegraded":"2024-04-25T10:12:38.477504+0000","last_fullsized":"2024-04-25T10:12:38.477504+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:41:12.973925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477546+0000","last_change":"2024-04-25T10:12:37.448340+0000","last_active":"2024-04-25T10:12:38.477546+0000","last_peered":"2024-04-25T10:12:38.477546+0000","last_clean":"2024-04-25T10:12:38.477546+0000","last_became_active":"2024-04-25T10:12:37.446052+0000","last_became_peered":"2024-04-25T10:12:37.446052+0000","last_unstale":"2024-04-25T10:12:38.477546+0000","last_undegraded":"2024-04-25T10:12:38.477546+0000","last_fullsized":"2024-04-25T10:12:38.477546+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:19.357372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477573+0000","last_change":"2024-04-25T10:12:37.424865+0000","last_active":"2024-04-25T10:12:38.477573+0000","last_peered":"2024-04-25T10:12:38.477573+0000","last_clean":"2024-04-25T10:12:38.477573+0000","last_became_active":"2024-04-25T10:12:37.424580+0000","last_became_peered":"2024-04-25T10:12:37.424580+0000","last_unstale":"2024-04-25T10:12:38.477573+0000","last_undegraded":"2024-04-25T10:12:38.477573+0000","last_fullsized":"2024-04-25T10:12:38.477573+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:34.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660749+0000","last_change":"2024-04-25T10:12:37.440407+0000","last_active":"2024-04-25T10:12:38.660749+0000","last_peered":"2024-04-25T10:12:38.660749+0000","last_clean":"2024-04-25T10:12:38.660749+0000","last_became_active":"2024-04-25T10:12:37.439466+0000","last_became_peered":"2024-04-25T10:12:37.439466+0000","last_unstale":"2024-04-25T10:12:38.660749+0000","last_undegraded":"2024-04-25T10:12:38.660749+0000","last_fullsized":"2024-04-25T10:12:38.660749+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:16.348692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421858+0000","last_change":"2024-04-25T10:12:37.444108+0000","last_active":"2024-04-25T10:12:38.421858+0000","last_peered":"2024-04-25T10:12:38.421858+0000","last_clean":"2024-04-25T10:12:38.421858+0000","last_became_active":"2024-04-25T10:12:37.443514+0000","last_became_peered":"2024-04-25T10:12:37.443514+0000","last_unstale":"2024-04-25T10:12:38.421858+0000","last_undegraded":"2024-04-25T10:12:38.421858+0000","last_fullsized":"2024-04-25T10:12:38.421858+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660185+0000","last_change":"2024-04-25T10:12:37.451809+0000","last_active":"2024-04-25T10:12:38.660185+0000","last_peered":"2024-04-25T10:12:38.660185+0000","last_clean":"2024-04-25T10:12:38.660185+0000","last_became_active":"2024-04-25T10:12:37.449840+0000","last_became_peered":"2024-04-25T10:12:37.449840+0000","last_unstale":"2024-04-25T10:12:38.660185+0000","last_undegraded":"2024-04-25T10:12:38.660185+0000","last_fullsized":"2024-04-25T10:12:38.660185+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:13:00.348275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419395+0000","last_change":"2024-04-25T10:12:37.425902+0000","last_active":"2024-04-25T10:12:38.419395+0000","last_peered":"2024-04-25T10:12:38.419395+0000","last_clean":"2024-04-25T10:12:38.419395+0000","last_became_active":"2024-04-25T10:12:37.425600+0000","last_became_peered":"2024-04-25T10:12:37.425600+0000","last_unstale":"2024-04-25T10:12:38.419395+0000","last_undegraded":"2024-04-25T10:12:38.419395+0000","last_fullsized":"2024-04-25T10:12:38.419395+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:22.480972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656760+0000","last_change":"2024-04-25T10:12:37.432765+0000","last_active":"2024-04-25T10:12:38.656760+0000","last_peered":"2024-04-25T10:12:38.656760+0000","last_clean":"2024-04-25T10:12:38.656760+0000","last_became_active":"2024-04-25T10:12:37.432505+0000","last_became_peered":"2024-04-25T10:12:37.432505+0000","last_unstale":"2024-04-25T10:12:38.656760+0000","last_undegraded":"2024-04-25T10:12:38.656760+0000","last_fullsized":"2024-04-25T10:12:38.656760+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:45:00.473984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660318+0000","last_change":"2024-04-25T10:12:37.443138+0000","last_active":"2024-04-25T10:12:38.660318+0000","last_peered":"2024-04-25T10:12:38.660318+0000","last_clean":"2024-04-25T10:12:38.660318+0000","last_became_active":"2024-04-25T10:12:37.442308+0000","last_became_peered":"2024-04-25T10:12:37.442308+0000","last_unstale":"2024-04-25T10:12:38.660318+0000","last_undegraded":"2024-04-25T10:12:38.660318+0000","last_fullsized":"2024-04-25T10:12:38.660318+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:32:43.163133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659462+0000","last_change":"2024-04-25T10:12:37.444200+0000","last_active":"2024-04-25T10:12:38.659462+0000","last_peered":"2024-04-25T10:12:38.659462+0000","last_clean":"2024-04-25T10:12:38.659462+0000","last_became_active":"2024-04-25T10:12:37.443940+0000","last_became_peered":"2024-04-25T10:12:37.443940+0000","last_unstale":"2024-04-25T10:12:38.659462+0000","last_undegraded":"2024-04-25T10:12:38.659462+0000","last_fullsized":"2024-04-25T10:12:38.659462+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:10.836901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659970+0000","last_change":"2024-04-25T10:12:37.448155+0000","last_active":"2024-04-25T10:12:38.659970+0000","last_peered":"2024-04-25T10:12:38.659970+0000","last_clean":"2024-04-25T10:12:38.659970+0000","last_became_active":"2024-04-25T10:12:37.447522+0000","last_became_peered":"2024-04-25T10:12:37.447522+0000","last_unstale":"2024-04-25T10:12:38.659970+0000","last_undegraded":"2024-04-25T10:12:38.659970+0000","last_fullsized":"2024-04-25T10:12:38.659970+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419980+0000","last_change":"2024-04-25T10:12:37.446108+0000","last_active":"2024-04-25T10:12:38.419980+0000","last_peered":"2024-04-25T10:12:38.419980+0000","last_clean":"2024-04-25T10:12:38.419980+0000","last_became_active":"2024-04-25T10:12:37.445831+0000","last_became_peered":"2024-04-25T10:12:37.445831+0000","last_unstale":"2024-04-25T10:12:38.419980+0000","last_undegraded":"2024-04-25T10:12:38.419980+0000","last_fullsized":"2024-04-25T10:12:38.419980+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:27.481817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418914+0000","last_change":"2024-04-25T10:12:37.441013+0000","last_active":"2024-04-25T10:12:38.418914+0000","last_peered":"2024-04-25T10:12:38.418914+0000","last_clean":"2024-04-25T10:12:38.418914+0000","last_became_active":"2024-04-25T10:12:37.440725+0000","last_became_peered":"2024-04-25T10:12:37.440725+0000","last_unstale":"2024-04-25T10:12:38.418914+0000","last_undegraded":"2024-04-25T10:12:38.418914+0000","last_fullsized":"2024-04-25T10:12:38.418914+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658244+0000","last_change":"2024-04-25T10:12:37.438780+0000","last_active":"2024-04-25T10:12:38.658244+0000","last_peered":"2024-04-25T10:12:38.658244+0000","last_clean":"2024-04-25T10:12:38.658244+0000","last_became_active":"2024-04-25T10:12:37.438484+0000","last_became_peered":"2024-04-25T10:12:37.438484+0000","last_unstale":"2024-04-25T10:12:38.658244+0000","last_undegraded":"2024-04-25T10:12:38.658244+0000","last_fullsized":"2024-04-25T10:12:38.658244+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:54.229611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420777+0000","last_change":"2024-04-25T10:12:37.440617+0000","last_active":"2024-04-25T10:12:38.420777+0000","last_peered":"2024-04-25T10:12:38.420777+0000","last_clean":"2024-04-25T10:12:38.420777+0000","last_became_active":"2024-04-25T10:12:37.440310+0000","last_became_peered":"2024-04-25T10:12:37.440310+0000","last_unstale":"2024-04-25T10:12:38.420777+0000","last_undegraded":"2024-04-25T10:12:38.420777+0000","last_fullsized":"2024-04-25T10:12:38.420777+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656959+0000","last_change":"2024-04-25T10:12:37.426018+0000","last_active":"2024-04-25T10:12:38.656959+0000","last_peered":"2024-04-25T10:12:38.656959+0000","last_clean":"2024-04-25T10:12:38.656959+0000","last_became_active":"2024-04-25T10:12:37.425637+0000","last_became_peered":"2024-04-25T10:12:37.425637+0000","last_unstale":"2024-04-25T10:12:38.656959+0000","last_undegraded":"2024-04-25T10:12:38.656959+0000","last_fullsized":"2024-04-25T10:12:38.656959+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:43:00.775595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418921+0000","last_change":"2024-04-25T10:12:37.443856+0000","last_active":"2024-04-25T10:12:38.418921+0000","last_peered":"2024-04-25T10:12:38.418921+0000","last_clean":"2024-04-25T10:12:38.418921+0000","last_became_active":"2024-04-25T10:12:37.443421+0000","last_became_peered":"2024-04-25T10:12:37.443421+0000","last_unstale":"2024-04-25T10:12:38.418921+0000","last_undegraded":"2024-04-25T10:12:38.418921+0000","last_fullsized":"2024-04-25T10:12:38.418921+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418651+0000","last_change":"2024-04-25T10:12:37.433545+0000","last_active":"2024-04-25T10:12:38.418651+0000","last_peered":"2024-04-25T10:12:38.418651+0000","last_clean":"2024-04-25T10:12:38.418651+0000","last_became_active":"2024-04-25T10:12:37.433271+0000","last_became_peered":"2024-04-25T10:12:37.433271+0000","last_unstale":"2024-04-25T10:12:38.418651+0000","last_undegraded":"2024-04-25T10:12:38.418651+0000","last_fullsized":"2024-04-25T10:12:38.418651+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:19:15.505817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.20","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656933+0000","last_change":"2024-04-25T10:12:37.426624+0000","last_active":"2024-04-25T10:12:38.656933+0000","last_peered":"2024-04-25T10:12:38.656933+0000","last_clean":"2024-04-25T10:12:38.656933+0000","last_became_active":"2024-04-25T10:12:37.425873+0000","last_became_peered":"2024-04-25T10:12:37.425873+0000","last_unstale":"2024-04-25T10:12:38.656933+0000","last_undegraded":"2024-04-25T10:12:38.656933+0000","last_fullsized":"2024-04-25T10:12:38.656933+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:31:08.735874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658673+0000","last_change":"2024-04-25T10:12:37.451368+0000","last_active":"2024-04-25T10:12:38.658673+0000","last_peered":"2024-04-25T10:12:38.658673+0000","last_clean":"2024-04-25T10:12:38.658673+0000","last_became_active":"2024-04-25T10:12:37.451027+0000","last_became_peered":"2024-04-25T10:12:37.451027+0000","last_unstale":"2024-04-25T10:12:38.658673+0000","last_undegraded":"2024-04-25T10:12:38.658673+0000","last_fullsized":"2024-04-25T10:12:38.658673+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:08.472720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"110'4","reported_seq":18,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414586+0000","last_change":"2024-04-25T10:12:36.428651+0000","last_active":"2024-04-25T10:12:38.414586+0000","last_peered":"2024-04-25T10:12:38.414586+0000","last_clean":"2024-04-25T10:12:38.414586+0000","last_became_active":"2024-04-25T10:12:36.428062+0000","last_became_peered":"2024-04-25T10:12:36.428062+0000","last_unstale":"2024-04-25T10:12:38.414586+0000","last_undegraded":"2024-04-25T10:12:38.414586+0000","last_fullsized":"2024-04-25T10:12:38.414586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:42.723368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659663+0000","last_change":"2024-04-25T10:12:37.442206+0000","last_active":"2024-04-25T10:12:38.659663+0000","last_peered":"2024-04-25T10:12:38.659663+0000","last_clean":"2024-04-25T10:12:38.659663+0000","last_became_active":"2024-04-25T10:12:37.441474+0000","last_became_peered":"2024-04-25T10:12:37.441474+0000","last_unstale":"2024-04-25T10:12:38.659663+0000","last_undegraded":"2024-04-25T10:12:38.659663+0000","last_fullsized":"2024-04-25T10:12:38.659663+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:33:09.781858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659154+0000","last_change":"2024-04-25T10:12:36.432356+0000","last_active":"2024-04-25T10:12:38.659154+0000","last_peered":"2024-04-25T10:12:38.659154+0000","last_clean":"2024-04-25T10:12:38.659154+0000","last_became_active":"2024-04-25T10:12:36.432086+0000","last_became_peered":"2024-04-25T10:12:36.432086+0000","last_unstale":"2024-04-25T10:12:38.659154+0000","last_undegraded":"2024-04-25T10:12:38.659154+0000","last_fullsized":"2024-04-25T10:12:38.659154+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:47:46.923030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419369+0000","last_change":"2024-04-25T10:12:37.447367+0000","last_active":"2024-04-25T10:12:38.419369+0000","last_peered":"2024-04-25T10:12:38.419369+0000","last_clean":"2024-04-25T10:12:38.419369+0000","last_became_active":"2024-04-25T10:12:37.447079+0000","last_became_peered":"2024-04-25T10:12:37.447079+0000","last_unstale":"2024-04-25T10:12:38.419369+0000","last_undegraded":"2024-04-25T10:12:38.419369+0000","last_fullsized":"2024-04-25T10:12:38.419369+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.418183+0000","last_change":"2024-04-25T10:12:36.437597+0000","last_active":"2024-04-25T10:12:38.418183+0000","last_peered":"2024-04-25T10:12:38.418183+0000","last_clean":"2024-04-25T10:12:38.418183+0000","last_became_active":"2024-04-25T10:12:36.437214+0000","last_became_peered":"2024-04-25T10:12:36.437214+0000","last_unstale":"2024-04-25T10:12:38.418183+0000","last_undegraded":"2024-04-25T10:12:38.418183+0000","last_fullsized":"2024-04-25T10:12:38.418183+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:54.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657727+0000","last_change":"2024-04-25T10:12:37.433634+0000","last_active":"2024-04-25T10:12:38.657727+0000","last_peered":"2024-04-25T10:12:38.657727+0000","last_clean":"2024-04-25T10:12:38.657727+0000","last_became_active":"2024-04-25T10:12:37.433334+0000","last_became_peered":"2024-04-25T10:12:37.433334+0000","last_unstale":"2024-04-25T10:12:38.657727+0000","last_undegraded":"2024-04-25T10:12:38.657727+0000","last_fullsized":"2024-04-25T10:12:38.657727+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:57:45.027233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658433+0000","last_change":"2024-04-25T10:12:36.430378+0000","last_active":"2024-04-25T10:12:38.658433+0000","last_peered":"2024-04-25T10:12:38.658433+0000","last_clean":"2024-04-25T10:12:38.658433+0000","last_became_active":"2024-04-25T10:12:36.430043+0000","last_became_peered":"2024-04-25T10:12:36.430043+0000","last_unstale":"2024-04-25T10:12:38.658433+0000","last_undegraded":"2024-04-25T10:12:38.658433+0000","last_fullsized":"2024-04-25T10:12:38.658433+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:14.731959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414534+0000","last_change":"2024-04-25T10:12:37.438453+0000","last_active":"2024-04-25T10:12:38.414534+0000","last_peered":"2024-04-25T10:12:38.414534+0000","last_clean":"2024-04-25T10:12:38.414534+0000","last_became_active":"2024-04-25T10:12:37.438179+0000","last_became_peered":"2024-04-25T10:12:37.438179+0000","last_unstale":"2024-04-25T10:12:38.414534+0000","last_undegraded":"2024-04-25T10:12:38.414534+0000","last_fullsized":"2024-04-25T10:12:38.414534+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:56.927046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477763+0000","last_change":"2024-04-25T10:12:36.432661+0000","last_active":"2024-04-25T10:12:38.477763+0000","last_peered":"2024-04-25T10:12:38.477763+0000","last_clean":"2024-04-25T10:12:38.477763+0000","last_became_active":"2024-04-25T10:12:36.432286+0000","last_became_peered":"2024-04-25T10:12:36.432286+0000","last_unstale":"2024-04-25T10:12:38.477763+0000","last_undegraded":"2024-04-25T10:12:38.477763+0000","last_fullsized":"2024-04-25T10:12:38.477763+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:35.057626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660604+0000","last_change":"2024-04-25T10:12:37.450831+0000","last_active":"2024-04-25T10:12:38.660604+0000","last_peered":"2024-04-25T10:12:38.660604+0000","last_clean":"2024-04-25T10:12:38.660604+0000","last_became_active":"2024-04-25T10:12:37.450537+0000","last_became_peered":"2024-04-25T10:12:37.450537+0000","last_unstale":"2024-04-25T10:12:38.660604+0000","last_undegraded":"2024-04-25T10:12:38.660604+0000","last_fullsized":"2024-04-25T10:12:38.660604+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:13.308948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657298+0000","last_change":"2024-04-25T10:12:36.437498+0000","last_active":"2024-04-25T10:12:38.657298+0000","last_peered":"2024-04-25T10:12:38.657298+0000","last_clean":"2024-04-25T10:12:38.657298+0000","last_became_active":"2024-04-25T10:12:36.437164+0000","last_became_peered":"2024-04-25T10:12:36.437164+0000","last_unstale":"2024-04-25T10:12:38.657298+0000","last_undegraded":"2024-04-25T10:12:38.657298+0000","last_fullsized":"2024-04-25T10:12:38.657298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:06.982259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.422167+0000","last_change":"2024-04-25T10:12:37.442744+0000","last_active":"2024-04-25T10:12:38.422167+0000","last_peered":"2024-04-25T10:12:38.422167+0000","last_clean":"2024-04-25T10:12:38.422167+0000","last_became_active":"2024-04-25T10:12:37.441161+0000","last_became_peered":"2024-04-25T10:12:37.441161+0000","last_unstale":"2024-04-25T10:12:38.422167+0000","last_undegraded":"2024-04-25T10:12:38.422167+0000","last_fullsized":"2024-04-25T10:12:38.422167+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:13.956579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417597+0000","last_change":"2024-04-25T10:12:36.436140+0000","last_active":"2024-04-25T10:12:38.417597+0000","last_peered":"2024-04-25T10:12:38.417597+0000","last_clean":"2024-04-25T10:12:38.417597+0000","last_became_active":"2024-04-25T10:12:36.435843+0000","last_became_peered":"2024-04-25T10:12:36.435843+0000","last_unstale":"2024-04-25T10:12:38.417597+0000","last_undegraded":"2024-04-25T10:12:38.417597+0000","last_fullsized":"2024-04-25T10:12:38.417597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414480+0000","last_change":"2024-04-25T10:12:37.436207+0000","last_active":"2024-04-25T10:12:38.414480+0000","last_peered":"2024-04-25T10:12:38.414480+0000","last_clean":"2024-04-25T10:12:38.414480+0000","last_became_active":"2024-04-25T10:12:37.435907+0000","last_became_peered":"2024-04-25T10:12:37.435907+0000","last_unstale":"2024-04-25T10:12:38.414480+0000","last_undegraded":"2024-04-25T10:12:38.414480+0000","last_fullsized":"2024-04-25T10:12:38.414480+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:48:34.118083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414951+0000","last_change":"2024-04-25T10:12:36.433709+0000","last_active":"2024-04-25T10:12:38.414951+0000","last_peered":"2024-04-25T10:12:38.414951+0000","last_clean":"2024-04-25T10:12:38.414951+0000","last_became_active":"2024-04-25T10:12:36.433440+0000","last_became_peered":"2024-04-25T10:12:36.433440+0000","last_unstale":"2024-04-25T10:12:38.414951+0000","last_undegraded":"2024-04-25T10:12:38.414951+0000","last_fullsized":"2024-04-25T10:12:38.414951+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:45:35.057626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658349+0000","last_change":"2024-04-25T10:12:37.439841+0000","last_active":"2024-04-25T10:12:38.658349+0000","last_peered":"2024-04-25T10:12:38.658349+0000","last_clean":"2024-04-25T10:12:38.658349+0000","last_became_active":"2024-04-25T10:12:37.439287+0000","last_became_peered":"2024-04-25T10:12:37.439287+0000","last_unstale":"2024-04-25T10:12:38.658349+0000","last_undegraded":"2024-04-25T10:12:38.658349+0000","last_fullsized":"2024-04-25T10:12:38.658349+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:20.343948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420040+0000","last_change":"2024-04-25T10:12:36.430969+0000","last_active":"2024-04-25T10:12:38.420040+0000","last_peered":"2024-04-25T10:12:38.420040+0000","last_clean":"2024-04-25T10:12:38.420040+0000","last_became_active":"2024-04-25T10:12:36.430681+0000","last_became_peered":"2024-04-25T10:12:36.430681+0000","last_unstale":"2024-04-25T10:12:38.420040+0000","last_undegraded":"2024-04-25T10:12:38.420040+0000","last_fullsized":"2024-04-25T10:12:38.420040+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:48.057318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657867+0000","last_change":"2024-04-25T10:12:37.435317+0000","last_active":"2024-04-25T10:12:38.657867+0000","last_peered":"2024-04-25T10:12:38.657867+0000","last_clean":"2024-04-25T10:12:38.657867+0000","last_became_active":"2024-04-25T10:12:37.435007+0000","last_became_peered":"2024-04-25T10:12:37.435007+0000","last_unstale":"2024-04-25T10:12:38.657867+0000","last_undegraded":"2024-04-25T10:12:38.657867+0000","last_fullsized":"2024-04-25T10:12:38.657867+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:23.304308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659574+0000","last_change":"2024-04-25T10:12:36.435089+0000","last_active":"2024-04-25T10:12:38.659574+0000","last_peered":"2024-04-25T10:12:38.659574+0000","last_clean":"2024-04-25T10:12:38.659574+0000","last_became_active":"2024-04-25T10:12:36.434759+0000","last_became_peered":"2024-04-25T10:12:36.434759+0000","last_unstale":"2024-04-25T10:12:38.659574+0000","last_undegraded":"2024-04-25T10:12:38.659574+0000","last_fullsized":"2024-04-25T10:12:38.659574+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:37.166230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660287+0000","last_change":"2024-04-25T10:12:37.441207+0000","last_active":"2024-04-25T10:12:38.660287+0000","last_peered":"2024-04-25T10:12:38.660287+0000","last_clean":"2024-04-25T10:12:38.660287+0000","last_became_active":"2024-04-25T10:12:37.440925+0000","last_became_peered":"2024-04-25T10:12:37.440925+0000","last_unstale":"2024-04-25T10:12:38.660287+0000","last_undegraded":"2024-04-25T10:12:38.660287+0000","last_fullsized":"2024-04-25T10:12:38.660287+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:08.118583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"110'3","reported_seq":17,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477926+0000","last_change":"2024-04-25T10:12:36.438390+0000","last_active":"2024-04-25T10:12:38.477926+0000","last_peered":"2024-04-25T10:12:38.477926+0000","last_clean":"2024-04-25T10:12:38.477926+0000","last_became_active":"2024-04-25T10:12:36.438100+0000","last_became_peered":"2024-04-25T10:12:36.438100+0000","last_unstale":"2024-04-25T10:12:38.477926+0000","last_undegraded":"2024-04-25T10:12:38.477926+0000","last_fullsized":"2024-04-25T10:12:38.477926+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:23:30.172798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658108+0000","last_change":"2024-04-25T10:12:37.441135+0000","last_active":"2024-04-25T10:12:38.658108+0000","last_peered":"2024-04-25T10:12:38.658108+0000","last_clean":"2024-04-25T10:12:38.658108+0000","last_became_active":"2024-04-25T10:12:37.440844+0000","last_became_peered":"2024-04-25T10:12:37.440844+0000","last_unstale":"2024-04-25T10:12:38.658108+0000","last_undegraded":"2024-04-25T10:12:38.658108+0000","last_fullsized":"2024-04-25T10:12:38.658108+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:53.420847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"110'6","reported_seq":20,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420801+0000","last_change":"2024-04-25T10:12:36.434095+0000","last_active":"2024-04-25T10:12:38.420801+0000","last_peered":"2024-04-25T10:12:38.420801+0000","last_clean":"2024-04-25T10:12:38.420801+0000","last_became_active":"2024-04-25T10:12:36.433803+0000","last_became_peered":"2024-04-25T10:12:36.433803+0000","last_unstale":"2024-04-25T10:12:38.420801+0000","last_undegraded":"2024-04-25T10:12:38.420801+0000","last_fullsized":"2024-04-25T10:12:38.420801+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:03.859392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657583+0000","last_change":"2024-04-25T10:12:37.431496+0000","last_active":"2024-04-25T10:12:38.657583+0000","last_peered":"2024-04-25T10:12:38.657583+0000","last_clean":"2024-04-25T10:12:38.657583+0000","last_became_active":"2024-04-25T10:12:37.430520+0000","last_became_peered":"2024-04-25T10:12:37.430520+0000","last_unstale":"2024-04-25T10:12:38.657583+0000","last_undegraded":"2024-04-25T10:12:38.657583+0000","last_fullsized":"2024-04-25T10:12:38.657583+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:44.202217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658293+0000","last_change":"2024-04-25T10:12:36.420615+0000","last_active":"2024-04-25T10:12:38.658293+0000","last_peered":"2024-04-25T10:12:38.658293+0000","last_clean":"2024-04-25T10:12:38.658293+0000","last_became_active":"2024-04-25T10:12:36.420089+0000","last_became_peered":"2024-04-25T10:12:36.420089+0000","last_unstale":"2024-04-25T10:12:38.658293+0000","last_undegraded":"2024-04-25T10:12:38.658293+0000","last_fullsized":"2024-04-25T10:12:38.658293+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:54.916063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421871+0000","last_change":"2024-04-25T10:12:37.439710+0000","last_active":"2024-04-25T10:12:38.421871+0000","last_peered":"2024-04-25T10:12:38.421871+0000","last_clean":"2024-04-25T10:12:38.421871+0000","last_became_active":"2024-04-25T10:12:37.439422+0000","last_became_peered":"2024-04-25T10:12:37.439422+0000","last_unstale":"2024-04-25T10:12:38.421871+0000","last_undegraded":"2024-04-25T10:12:38.421871+0000","last_fullsized":"2024-04-25T10:12:38.421871+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:36:28.724635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414588+0000","last_change":"2024-04-25T10:12:36.432744+0000","last_active":"2024-04-25T10:12:38.414588+0000","last_peered":"2024-04-25T10:12:38.414588+0000","last_clean":"2024-04-25T10:12:38.414588+0000","last_became_active":"2024-04-25T10:12:36.431971+0000","last_became_peered":"2024-04-25T10:12:36.431971+0000","last_unstale":"2024-04-25T10:12:38.414588+0000","last_undegraded":"2024-04-25T10:12:38.414588+0000","last_fullsized":"2024-04-25T10:12:38.414588+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659025+0000","last_change":"2024-04-25T10:12:37.425673+0000","last_active":"2024-04-25T10:12:38.659025+0000","last_peered":"2024-04-25T10:12:38.659025+0000","last_clean":"2024-04-25T10:12:38.659025+0000","last_became_active":"2024-04-25T10:12:37.425265+0000","last_became_peered":"2024-04-25T10:12:37.425265+0000","last_unstale":"2024-04-25T10:12:38.659025+0000","last_undegraded":"2024-04-25T10:12:38.659025+0000","last_fullsized":"2024-04-25T10:12:38.659025+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:07:48.545973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659783+0000","last_change":"2024-04-25T10:12:36.429891+0000","last_active":"2024-04-25T10:12:38.659783+0000","last_peered":"2024-04-25T10:12:38.659783+0000","last_clean":"2024-04-25T10:12:38.659783+0000","last_became_active":"2024-04-25T10:12:36.429336+0000","last_became_peered":"2024-04-25T10:12:36.429336+0000","last_unstale":"2024-04-25T10:12:38.659783+0000","last_undegraded":"2024-04-25T10:12:38.659783+0000","last_fullsized":"2024-04-25T10:12:38.659783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:54.333585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660854+0000","last_change":"2024-04-25T10:12:37.440609+0000","last_active":"2024-04-25T10:12:38.660854+0000","last_peered":"2024-04-25T10:12:38.660854+0000","last_clean":"2024-04-25T10:12:38.660854+0000","last_became_active":"2024-04-25T10:12:37.439917+0000","last_became_peered":"2024-04-25T10:12:37.439917+0000","last_unstale":"2024-04-25T10:12:38.660854+0000","last_undegraded":"2024-04-25T10:12:38.660854+0000","last_fullsized":"2024-04-25T10:12:38.660854+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:03.858903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"110'8","reported_seq":22,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417179+0000","last_change":"2024-04-25T10:12:36.434858+0000","last_active":"2024-04-25T10:12:38.417179+0000","last_peered":"2024-04-25T10:12:38.417179+0000","last_clean":"2024-04-25T10:12:38.417179+0000","last_became_active":"2024-04-25T10:12:36.434571+0000","last_became_peered":"2024-04-25T10:12:36.434571+0000","last_unstale":"2024-04-25T10:12:38.417179+0000","last_undegraded":"2024-04-25T10:12:38.417179+0000","last_fullsized":"2024-04-25T10:12:38.417179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657357+0000","last_change":"2024-04-25T10:12:37.430884+0000","last_active":"2024-04-25T10:12:38.657357+0000","last_peered":"2024-04-25T10:12:38.657357+0000","last_clean":"2024-04-25T10:12:38.657357+0000","last_became_active":"2024-04-25T10:12:37.428118+0000","last_became_peered":"2024-04-25T10:12:37.428118+0000","last_unstale":"2024-04-25T10:12:38.657357+0000","last_undegraded":"2024-04-25T10:12:38.657357+0000","last_fullsized":"2024-04-25T10:12:38.657357+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:36.032449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421873+0000","last_change":"2024-04-25T10:12:36.437073+0000","last_active":"2024-04-25T10:12:38.421873+0000","last_peered":"2024-04-25T10:12:38.421873+0000","last_clean":"2024-04-25T10:12:38.421873+0000","last_became_active":"2024-04-25T10:12:36.436575+0000","last_became_peered":"2024-04-25T10:12:36.436575+0000","last_unstale":"2024-04-25T10:12:38.421873+0000","last_undegraded":"2024-04-25T10:12:38.421873+0000","last_fullsized":"2024-04-25T10:12:38.421873+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:52.986763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.421317+0000","last_change":"2024-04-25T10:12:37.446967+0000","last_active":"2024-04-25T10:12:38.421317+0000","last_peered":"2024-04-25T10:12:38.421317+0000","last_clean":"2024-04-25T10:12:38.421317+0000","last_became_active":"2024-04-25T10:12:37.446664+0000","last_became_peered":"2024-04-25T10:12:37.446664+0000","last_unstale":"2024-04-25T10:12:38.421317+0000","last_undegraded":"2024-04-25T10:12:38.421317+0000","last_fullsized":"2024-04-25T10:12:38.421317+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"110'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477978+0000","last_change":"2024-04-25T10:12:36.430895+0000","last_active":"2024-04-25T10:12:38.477978+0000","last_peered":"2024-04-25T10:12:38.477978+0000","last_clean":"2024-04-25T10:12:38.477978+0000","last_became_active":"2024-04-25T10:12:36.430565+0000","last_became_peered":"2024-04-25T10:12:36.430565+0000","last_unstale":"2024-04-25T10:12:38.477978+0000","last_undegraded":"2024-04-25T10:12:38.477978+0000","last_fullsized":"2024-04-25T10:12:38.477978+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:28:38.946273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657946+0000","last_change":"2024-04-25T10:12:37.436317+0000","last_active":"2024-04-25T10:12:38.657946+0000","last_peered":"2024-04-25T10:12:38.657946+0000","last_clean":"2024-04-25T10:12:38.657946+0000","last_became_active":"2024-04-25T10:12:37.435976+0000","last_became_peered":"2024-04-25T10:12:37.435976+0000","last_unstale":"2024-04-25T10:12:38.657946+0000","last_undegraded":"2024-04-25T10:12:38.657946+0000","last_fullsized":"2024-04-25T10:12:38.657946+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:29:07.998400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657682+0000","last_change":"2024-04-25T10:12:36.438473+0000","last_active":"2024-04-25T10:12:38.657682+0000","last_peered":"2024-04-25T10:12:38.657682+0000","last_clean":"2024-04-25T10:12:38.657682+0000","last_became_active":"2024-04-25T10:12:36.436411+0000","last_became_peered":"2024-04-25T10:12:36.436411+0000","last_unstale":"2024-04-25T10:12:38.657682+0000","last_undegraded":"2024-04-25T10:12:38.657682+0000","last_fullsized":"2024-04-25T10:12:38.657682+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:52.404706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659307+0000","last_change":"2024-04-25T10:12:37.445297+0000","last_active":"2024-04-25T10:12:38.659307+0000","last_peered":"2024-04-25T10:12:38.659307+0000","last_clean":"2024-04-25T10:12:38.659307+0000","last_became_active":"2024-04-25T10:12:37.444851+0000","last_became_peered":"2024-04-25T10:12:37.444851+0000","last_unstale":"2024-04-25T10:12:38.659307+0000","last_undegraded":"2024-04-25T10:12:38.659307+0000","last_fullsized":"2024-04-25T10:12:38.659307+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:24.622841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657071+0000","last_change":"2024-04-25T10:12:36.428322+0000","last_active":"2024-04-25T10:12:38.657071+0000","last_peered":"2024-04-25T10:12:38.657071+0000","last_clean":"2024-04-25T10:12:38.657071+0000","last_became_active":"2024-04-25T10:12:36.427541+0000","last_became_peered":"2024-04-25T10:12:36.427541+0000","last_unstale":"2024-04-25T10:12:38.657071+0000","last_undegraded":"2024-04-25T10:12:38.657071+0000","last_fullsized":"2024-04-25T10:12:38.657071+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:26.382466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660381+0000","last_change":"2024-04-25T10:12:37.438627+0000","last_active":"2024-04-25T10:12:38.660381+0000","last_peered":"2024-04-25T10:12:38.660381+0000","last_clean":"2024-04-25T10:12:38.660381+0000","last_became_active":"2024-04-25T10:12:37.438352+0000","last_became_peered":"2024-04-25T10:12:37.438352+0000","last_unstale":"2024-04-25T10:12:38.660381+0000","last_undegraded":"2024-04-25T10:12:38.660381+0000","last_fullsized":"2024-04-25T10:12:38.660381+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:30:15.577771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656597+0000","last_change":"2024-04-25T10:12:36.420411+0000","last_active":"2024-04-25T10:12:38.656597+0000","last_peered":"2024-04-25T10:12:38.656597+0000","last_clean":"2024-04-25T10:12:38.656597+0000","last_became_active":"2024-04-25T10:12:36.419903+0000","last_became_peered":"2024-04-25T10:12:36.419903+0000","last_unstale":"2024-04-25T10:12:38.656597+0000","last_undegraded":"2024-04-25T10:12:38.656597+0000","last_fullsized":"2024-04-25T10:12:38.656597+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:43.819864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.414447+0000","last_change":"2024-04-25T10:12:37.452031+0000","last_active":"2024-04-25T10:12:38.414447+0000","last_peered":"2024-04-25T10:12:38.414447+0000","last_clean":"2024-04-25T10:12:38.414447+0000","last_became_active":"2024-04-25T10:12:37.451729+0000","last_became_peered":"2024-04-25T10:12:37.451729+0000","last_unstale":"2024-04-25T10:12:38.414447+0000","last_undegraded":"2024-04-25T10:12:38.414447+0000","last_fullsized":"2024-04-25T10:12:38.414447+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.311656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420290+0000","last_change":"2024-04-25T10:12:36.435352+0000","last_active":"2024-04-25T10:12:38.420290+0000","last_peered":"2024-04-25T10:12:38.420290+0000","last_clean":"2024-04-25T10:12:38.420290+0000","last_became_active":"2024-04-25T10:12:36.434943+0000","last_became_peered":"2024-04-25T10:12:36.434943+0000","last_unstale":"2024-04-25T10:12:38.420290+0000","last_undegraded":"2024-04-25T10:12:38.420290+0000","last_fullsized":"2024-04-25T10:12:38.420290+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657392+0000","last_change":"2024-04-25T10:12:37.437540+0000","last_active":"2024-04-25T10:12:38.657392+0000","last_peered":"2024-04-25T10:12:38.657392+0000","last_clean":"2024-04-25T10:12:38.657392+0000","last_became_active":"2024-04-25T10:12:37.437266+0000","last_became_peered":"2024-04-25T10:12:37.437266+0000","last_unstale":"2024-04-25T10:12:38.657392+0000","last_undegraded":"2024-04-25T10:12:38.657392+0000","last_fullsized":"2024-04-25T10:12:38.657392+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:52.631631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658976+0000","last_change":"2024-04-25T10:12:36.419478+0000","last_active":"2024-04-25T10:12:38.658976+0000","last_peered":"2024-04-25T10:12:38.658976+0000","last_clean":"2024-04-25T10:12:38.658976+0000","last_became_active":"2024-04-25T10:12:36.419010+0000","last_became_peered":"2024-04-25T10:12:36.419010+0000","last_unstale":"2024-04-25T10:12:38.658976+0000","last_undegraded":"2024-04-25T10:12:38.658976+0000","last_fullsized":"2024-04-25T10:12:38.658976+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:23:30.628822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658013+0000","last_change":"2024-04-25T10:12:37.439593+0000","last_active":"2024-04-25T10:12:38.658013+0000","last_peered":"2024-04-25T10:12:38.658013+0000","last_clean":"2024-04-25T10:12:38.658013+0000","last_became_active":"2024-04-25T10:12:37.438897+0000","last_became_peered":"2024-04-25T10:12:37.438897+0000","last_unstale":"2024-04-25T10:12:38.658013+0000","last_undegraded":"2024-04-25T10:12:38.658013+0000","last_fullsized":"2024-04-25T10:12:38.658013+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:50.701854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660240+0000","last_change":"2024-04-25T10:12:36.424429+0000","last_active":"2024-04-25T10:12:38.660240+0000","last_peered":"2024-04-25T10:12:38.660240+0000","last_clean":"2024-04-25T10:12:38.660240+0000","last_became_active":"2024-04-25T10:12:36.423878+0000","last_became_peered":"2024-04-25T10:12:36.423878+0000","last_unstale":"2024-04-25T10:12:38.660240+0000","last_undegraded":"2024-04-25T10:12:38.660240+0000","last_fullsized":"2024-04-25T10:12:38.660240+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:48.057318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.417215+0000","last_change":"2024-04-25T10:12:37.448434+0000","last_active":"2024-04-25T10:12:38.417215+0000","last_peered":"2024-04-25T10:12:38.417215+0000","last_clean":"2024-04-25T10:12:38.417215+0000","last_became_active":"2024-04-25T10:12:37.447902+0000","last_became_peered":"2024-04-25T10:12:37.447902+0000","last_unstale":"2024-04-25T10:12:38.417215+0000","last_undegraded":"2024-04-25T10:12:38.417215+0000","last_fullsized":"2024-04-25T10:12:38.417215+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:00:56.964062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"111'1","reported_seq":15,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.660034+0000","last_change":"2024-04-25T10:12:36.440745+0000","last_active":"2024-04-25T10:12:38.660034+0000","last_peered":"2024-04-25T10:12:38.660034+0000","last_clean":"2024-04-25T10:12:38.660034+0000","last_became_active":"2024-04-25T10:12:36.440448+0000","last_became_peered":"2024-04-25T10:12:36.440448+0000","last_unstale":"2024-04-25T10:12:38.660034+0000","last_undegraded":"2024-04-25T10:12:38.660034+0000","last_fullsized":"2024-04-25T10:12:38.660034+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:27:21.234143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.413900+0000","last_change":"2024-04-25T10:12:37.432087+0000","last_active":"2024-04-25T10:12:38.413900+0000","last_peered":"2024-04-25T10:12:38.413900+0000","last_clean":"2024-04-25T10:12:38.413900+0000","last_became_active":"2024-04-25T10:12:37.431814+0000","last_became_peered":"2024-04-25T10:12:37.431814+0000","last_unstale":"2024-04-25T10:12:38.413900+0000","last_undegraded":"2024-04-25T10:12:38.413900+0000","last_fullsized":"2024-04-25T10:12:38.413900+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:19.357372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.422143+0000","last_change":"2024-04-25T10:12:36.430380+0000","last_active":"2024-04-25T10:12:38.422143+0000","last_peered":"2024-04-25T10:12:38.422143+0000","last_clean":"2024-04-25T10:12:38.422143+0000","last_became_active":"2024-04-25T10:12:36.429347+0000","last_became_peered":"2024-04-25T10:12:36.429347+0000","last_unstale":"2024-04-25T10:12:38.422143+0000","last_undegraded":"2024-04-25T10:12:38.422143+0000","last_fullsized":"2024-04-25T10:12:38.422143+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:45.398157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657557+0000","last_change":"2024-04-25T10:12:37.436383+0000","last_active":"2024-04-25T10:12:38.657557+0000","last_peered":"2024-04-25T10:12:38.657557+0000","last_clean":"2024-04-25T10:12:38.657557+0000","last_became_active":"2024-04-25T10:12:37.436077+0000","last_became_peered":"2024-04-25T10:12:37.436077+0000","last_unstale":"2024-04-25T10:12:38.657557+0000","last_undegraded":"2024-04-25T10:12:38.657557+0000","last_fullsized":"2024-04-25T10:12:38.657557+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:53.082404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658706+0000","last_change":"2024-04-25T10:12:36.434613+0000","last_active":"2024-04-25T10:12:38.658706+0000","last_peered":"2024-04-25T10:12:38.658706+0000","last_clean":"2024-04-25T10:12:38.658706+0000","last_became_active":"2024-04-25T10:12:36.434287+0000","last_became_peered":"2024-04-25T10:12:36.434287+0000","last_unstale":"2024-04-25T10:12:38.658706+0000","last_undegraded":"2024-04-25T10:12:38.658706+0000","last_fullsized":"2024-04-25T10:12:38.658706+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:18:00.295480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656876+0000","last_change":"2024-04-25T10:12:37.442105+0000","last_active":"2024-04-25T10:12:38.656876+0000","last_peered":"2024-04-25T10:12:38.656876+0000","last_clean":"2024-04-25T10:12:38.656876+0000","last_became_active":"2024-04-25T10:12:37.440748+0000","last_became_peered":"2024-04-25T10:12:37.440748+0000","last_unstale":"2024-04-25T10:12:38.656876+0000","last_undegraded":"2024-04-25T10:12:38.656876+0000","last_fullsized":"2024-04-25T10:12:38.656876+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:05:39.046065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.419799+0000","last_change":"2024-04-25T10:12:36.429818+0000","last_active":"2024-04-25T10:12:38.419799+0000","last_peered":"2024-04-25T10:12:38.419799+0000","last_clean":"2024-04-25T10:12:38.419799+0000","last_became_active":"2024-04-25T10:12:36.429160+0000","last_became_peered":"2024-04-25T10:12:36.429160+0000","last_unstale":"2024-04-25T10:12:38.419799+0000","last_undegraded":"2024-04-25T10:12:38.419799+0000","last_fullsized":"2024-04-25T10:12:38.419799+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:27.946511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659576+0000","last_change":"2024-04-25T10:12:37.434576+0000","last_active":"2024-04-25T10:12:38.659576+0000","last_peered":"2024-04-25T10:12:38.659576+0000","last_clean":"2024-04-25T10:12:38.659576+0000","last_became_active":"2024-04-25T10:12:37.434299+0000","last_became_peered":"2024-04-25T10:12:37.434299+0000","last_unstale":"2024-04-25T10:12:38.659576+0000","last_undegraded":"2024-04-25T10:12:38.659576+0000","last_fullsized":"2024-04-25T10:12:38.659576+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:15:01.151422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.656929+0000","last_change":"2024-04-25T10:12:36.428404+0000","last_active":"2024-04-25T10:12:38.656929+0000","last_peered":"2024-04-25T10:12:38.656929+0000","last_clean":"2024-04-25T10:12:38.656929+0000","last_became_active":"2024-04-25T10:12:36.428103+0000","last_became_peered":"2024-04-25T10:12:36.428103+0000","last_unstale":"2024-04-25T10:12:38.656929+0000","last_undegraded":"2024-04-25T10:12:38.656929+0000","last_fullsized":"2024-04-25T10:12:38.656929+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:47.198001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.478234+0000","last_change":"2024-04-25T10:12:37.433986+0000","last_active":"2024-04-25T10:12:38.478234+0000","last_peered":"2024-04-25T10:12:38.478234+0000","last_clean":"2024-04-25T10:12:38.478234+0000","last_became_active":"2024-04-25T10:12:37.433467+0000","last_became_peered":"2024-04-25T10:12:37.433467+0000","last_unstale":"2024-04-25T10:12:38.478234+0000","last_undegraded":"2024-04-25T10:12:38.478234+0000","last_fullsized":"2024-04-25T10:12:38.478234+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:05.311656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.420572+0000","last_change":"2024-04-25T10:12:36.423055+0000","last_active":"2024-04-25T10:12:38.420572+0000","last_peered":"2024-04-25T10:12:38.420572+0000","last_clean":"2024-04-25T10:12:38.420572+0000","last_became_active":"2024-04-25T10:12:36.422751+0000","last_became_peered":"2024-04-25T10:12:36.422751+0000","last_unstale":"2024-04-25T10:12:38.420572+0000","last_undegraded":"2024-04-25T10:12:38.420572+0000","last_fullsized":"2024-04-25T10:12:38.420572+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:30.527352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.658454+0000","last_change":"2024-04-25T10:12:37.446201+0000","last_active":"2024-04-25T10:12:38.658454+0000","last_peered":"2024-04-25T10:12:38.658454+0000","last_clean":"2024-04-25T10:12:38.658454+0000","last_became_active":"2024-04-25T10:12:37.440114+0000","last_became_peered":"2024-04-25T10:12:37.440114+0000","last_unstale":"2024-04-25T10:12:38.658454+0000","last_undegraded":"2024-04-25T10:12:38.658454+0000","last_fullsized":"2024-04-25T10:12:38.658454+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:02:40.183842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":14,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659195+0000","last_change":"2024-04-25T10:12:36.426977+0000","last_active":"2024-04-25T10:12:38.659195+0000","last_peered":"2024-04-25T10:12:38.659195+0000","last_clean":"2024-04-25T10:12:38.659195+0000","last_became_active":"2024-04-25T10:12:36.424814+0000","last_became_peered":"2024-04-25T10:12:36.424814+0000","last_unstale":"2024-04-25T10:12:38.659195+0000","last_undegraded":"2024-04-25T10:12:38.659195+0000","last_fullsized":"2024-04-25T10:12:38.659195+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:08:28.338534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":12,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.657375+0000","last_change":"2024-04-25T10:12:37.437126+0000","last_active":"2024-04-25T10:12:38.657375+0000","last_peered":"2024-04-25T10:12:38.657375+0000","last_clean":"2024-04-25T10:12:38.657375+0000","last_became_active":"2024-04-25T10:12:37.436835+0000","last_became_peered":"2024-04-25T10:12:37.436835+0000","last_unstale":"2024-04-25T10:12:38.657375+0000","last_undegraded":"2024-04-25T10:12:38.657375+0000","last_fullsized":"2024-04-25T10:12:38.657375+0000","mapping_epoch":109,"log_start":"0'0","ondisk_log_start":"0'0","created":109,"last_epoch_clean":110,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:36.386885+0000","last_clean_scrub_stamp":"2024-04-25T10:12:36.386885+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:03:27.411214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"110'2","reported_seq":16,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.659298+0000","last_change":"2024-04-25T10:12:36.419237+0000","last_active":"2024-04-25T10:12:38.659298+0000","last_peered":"2024-04-25T10:12:38.659298+0000","last_clean":"2024-04-25T10:12:38.659298+0000","last_became_active":"2024-04-25T10:12:36.418640+0000","last_became_peered":"2024-04-25T10:12:36.418640+0000","last_unstale":"2024-04-25T10:12:38.659298+0000","last_undegraded":"2024-04-25T10:12:38.659298+0000","last_fullsized":"2024-04-25T10:12:38.659298+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:12:35.370744+0000","last_clean_scrub_stamp":"2024-04-25T10:12:35.370744+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:43.852848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":343,"reported_epoch":111,"state":"active+clean","last_fresh":"2024-04-25T10:12:38.477396+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:12:38.477396+0000","last_peered":"2024-04-25T10:12:38.477396+0000","last_clean":"2024-04-25T10:12:38.477396+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:12:38.477396+0000","last_undegraded":"2024-04-25T10:12:38.477396+0000","last_fullsized":"2024-04-25T10:12:38.477396+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":15,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574982,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51164,"kb_used_data":1932,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320676,"statfs":{"total":96636764160,"available":96584372224,"internally_reserved":0,"allocated":1978368,"data_stored":1234135,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574982,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50600,"kb_used_data":1364,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321240,"statfs":{"total":96636764160,"available":96584949760,"internally_reserved":0,"allocated":1396736,"data_stored":645823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50580,"kb_used_data":1344,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321260,"statfs":{"total":96636764160,"available":96584970240,"internally_reserved":0,"allocated":1376256,"data_stored":643607,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58908,"kb_used_data":1352,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312932,"statfs":{"total":96636764160,"available":96576442368,"internally_reserved":0,"allocated":1384448,"data_stored":643699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574982,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58896,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312944,"statfs":{"total":96636764160,"available":96576454656,"internally_reserved":0,"allocated":1380352,"data_stored":644189,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574981,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54744,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94317096,"statfs":{"total":96636764160,"available":96580706304,"internally_reserved":0,"allocated":1380352,"data_stored":643641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50576,"kb_used_data":1348,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321264,"statfs":{"total":96636764160,"available":96584974336,"internally_reserved":0,"allocated":1380352,"data_stored":643641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":50395589},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574982,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55336,"kb_used_data":1940,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316504,"statfs":{"total":96636764160,"available":96580100096,"internally_reserved":0,"allocated":1986560,"data_stored":1235677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:12:47.294 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:12:47.295 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:12:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.534+0000 7f5527e00640 1 Processor -- start 2024-04-25T10:12:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.534+0000 7f5527e00640 1 -- start start 2024-04-25T10:12:47.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f552813d9f0 0x7f552813df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 0x7f55280b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f552806d390 con 0x7f552806e150 2024-04-25T10:12:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55280b10d0 con 0x7f552813d9f0 2024-04-25T10:12:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552806d670 con 0x7f55280b0730 2024-04-25T10:12:47.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f552806e150 0x7f552806e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45304/0 (socket says 172.21.15.165:45304) 2024-04-25T10:12:47.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.536+0000 7f5527400640 1 -- 172.21.15.165:0/3437773482 learned_addr learned my addr 172.21.15.165:0/3437773482 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:47.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526a00640 1 --2- 172.21.15.165:0/3437773482 >> v2:172.21.15.172:3300/0 conn(0x7f552813d9f0 0x7f552813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f551fe00640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 0x7f55280b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221916283 0 0) 0x7f552806d390 con 0x7f552806e150 2024-04-25T10:12:47.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f550c003520 con 0x7f552806e150 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1460539547 0 0) 0x7f550c003520 con 0x7f552806e150 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 msgr2=0x7f55280b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 0x7f55280b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 >> v2:172.21.15.172:3300/0 conn(0x7f552813d9f0 msgr2=0x7f552813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 --2- 172.21.15.165:0/3437773482 >> v2:172.21.15.172:3300/0 conn(0x7f552813d9f0 0x7f552813df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552806d950 con 0x7f552806e150 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f55200019f0 con 0x7f552806e150 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 >> v1:172.21.15.165:6789/0 conn(0x7f552806e150 legacy=0x7f552806e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f550c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.537+0000 7f5526000640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 0x7f550c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526000640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c004d40 0x7f550c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552806d670 con 0x7f550c0039b0 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f550c00b9f0 con 0x7f550c004d40 2024-04-25T10:12:47.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550c00bbf0 con 0x7f55280b0730 2024-04-25T10:12:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526a00640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f550c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526a00640 1 -- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 msgr2=0x7f550c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526a00640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280b0730 0x7f550c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526a00640 1 -- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c004d40 msgr2=0x7f550c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526a00640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f550c004d40 0x7f550c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.538+0000 7f5526a00640 1 -- 172.21.15.165:0/3437773482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c003520 con 0x7f550c0039b0 2024-04-25T10:12:47.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.539+0000 7f5526a00640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f550c003dd0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5518009ac0 tx=0x7f550c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12f8175cefabacfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.539+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518036070 con 0x7f550c0039b0 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518031400 con 0x7f550c0039b0 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5526000640 1 -- 172.21.15.165:0/3437773482 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518009690 con 0x7f550c0039b0 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527e00640 1 -- 172.21.15.165:0/3437773482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 msgr2=0x7f550c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527e00640 1 --2- 172.21.15.165:0/3437773482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f550c003dd0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f5518009ac0 tx=0x7f550c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527400640 1 -- 172.21.15.165:0/3437773482 reap_dead start 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527e00640 1 -- 172.21.15.165:0/3437773482 shutdown_connections 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527e00640 1 -- 172.21.15.165:0/3437773482 >> 172.21.15.165:0/3437773482 conn(0x7f5528061cc0 msgr2=0x7f552806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527e00640 1 -- 172.21.15.165:0/3437773482 shutdown_connections 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.540+0000 7f5527e00640 1 -- 172.21.15.165:0/3437773482 wait complete. 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.542+0000 7f5527e00640 1 Processor -- start 2024-04-25T10:12:47.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.542+0000 7f5527e00640 1 -- start start 2024-04-25T10:12:47.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5527e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f55281368e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5527e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280ceab0 0x7f55281f4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5527e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5528137080 0x7f55280711d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5527e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552813e5f0 con 0x7f550c0039b0 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5527e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552813e470 con 0x7f5528137080 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5527e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55280b0b90 con 0x7f55280ceab0 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5526a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f55281368e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5526a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f55281368e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51562/0 (socket says 172.21.15.165:51562) 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.543+0000 7f5526a00640 1 -- 172.21.15.165:0/452934390 learned_addr learned my addr 172.21.15.165:0/452934390 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5527400640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5528137080 0x7f55280711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f551fe00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280ceab0 0x7f55281f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5526a00640 1 -- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280ceab0 msgr2=0x7f55281f4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5526a00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280ceab0 0x7f55281f4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5526a00640 1 -- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5528137080 msgr2=0x7f55280711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5526a00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5528137080 0x7f55280711d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5526a00640 1 -- 172.21.15.165:0/452934390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5528141aa0 con 0x7f550c0039b0 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5527400640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5528137080 0x7f55280711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.544+0000 7f5526a00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f55281368e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5518009ac0 tx=0x7f5518004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.545+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518040020 con 0x7f550c0039b0 2024-04-25T10:12:47.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.545+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518003b40 con 0x7f550c0039b0 2024-04-25T10:12:47.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.545+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55281f4f50 con 0x7f550c0039b0 2024-04-25T10:12:47.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.545+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518036040 con 0x7f550c0039b0 2024-04-25T10:12:47.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.545+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55281f5510 con 0x7f550c0039b0 2024-04-25T10:12:47.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.546+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5528073990 con 0x7f550c0039b0 2024-04-25T10:12:47.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.547+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5518043020 con 0x7f550c0039b0 2024-04-25T10:12:47.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.548+0000 7f551ea00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f550406b500 0x7f550406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:47.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.548+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f55180aa5d0 con 0x7f550c0039b0 2024-04-25T10:12:47.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.548+0000 7f551fe00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f550406b500 0x7f550406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:47.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.549+0000 7f551fe00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f550406b500 0x7f550406d9c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5528139af0 tx=0x7f5510008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:47.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.552+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f551804a070 con 0x7f550c0039b0 2024-04-25T10:12:47.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.800+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f5528136e20 con 0x7f550c0039b0 2024-04-25T10:12:47.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.805+0000 7f551ea00640 1 -- 172.21.15.165:0/452934390 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+147 (secure 0 0 0) 0x7f55180044e0 con 0x7f550c0039b0 2024-04-25T10:12:47.810 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:12:47.810 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:12:47.810 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:12:47.810 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:12:47.810 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:12:47.810 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:12:47.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.807+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f550406b500 msgr2=0x7f550406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.807+0000 7f5527e00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f550406b500 0x7f550406d9c0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f5528139af0 tx=0x7f5510008040 comp rx=0 tx=0).stop 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.807+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 msgr2=0x7f55281368e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.807+0000 7f5527e00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f55281368e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f5518009ac0 tx=0x7f5518004730 comp rx=0 tx=0).stop 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 shutdown_connections 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f550406b500 0x7f550406d9c0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5528137080 0x7f55280711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f55280ceab0 0x7f55281f4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 --2- 172.21.15.165:0/452934390 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f550c0039b0 0x7f55281368e0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 >> 172.21.15.165:0/452934390 conn(0x7f5528061cc0 msgr2=0x7f55280b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 shutdown_connections 2024-04-25T10:12:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:47.808+0000 7f5527e00640 1 -- 172.21.15.165:0/452934390 wait complete. 2024-04-25T10:12:47.831 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:12:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.021+0000 7f8461000640 1 Processor -- start 2024-04-25T10:12:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.021+0000 7f8461000640 1 -- start start 2024-04-25T10:12:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f8461000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f845c154ee0 0x7f845c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f8461000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f845c0ce590 0x7f845c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f8461000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f845c071480 con 0x7f845c0cfcf0 2024-04-25T10:12:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f8461000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c072090 con 0x7f845c0ce590 2024-04-25T10:12:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f8461000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f845c071230 con 0x7f845c154ee0 2024-04-25T10:12:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f845a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f845c0ce590 0x7f845c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f845a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f845c0ce590 0x7f845c06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51564/0 (socket says 172.21.15.165:51564) 2024-04-25T10:12:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.023+0000 7f845a800640 1 -- 172.21.15.165:0/4252300619 learned_addr learned my addr 172.21.15.165:0/4252300619 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.024+0000 7f845a800640 1 -- 172.21.15.165:0/4252300619 >> v2:172.21.15.172:3300/0 conn(0x7f845c154ee0 msgr2=0x7f845c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.024+0000 7f8459e00640 1 --2- 172.21.15.165:0/4252300619 >> v2:172.21.15.172:3300/0 conn(0x7f845c154ee0 0x7f845c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.024+0000 7f845a800640 1 --2- 172.21.15.165:0/4252300619 >> v2:172.21.15.172:3300/0 conn(0x7f845c154ee0 0x7f845c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.024+0000 7f845a800640 1 -- 172.21.15.165:0/4252300619 >> v1:172.21.15.172:6789/0 conn(0x7f845c0cfcf0 legacy=0x7f845c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.024+0000 7f845a800640 1 -- 172.21.15.165:0/4252300619 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c071760 con 0x7f845c0ce590 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.024+0000 7f8459e00640 1 --2- 172.21.15.165:0/4252300619 >> v2:172.21.15.172:3300/0 conn(0x7f845c154ee0 0x7f845c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.025+0000 7f845a800640 1 --2- 172.21.15.165:0/4252300619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d6f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f845c0d02e0 tx=0x7f845402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec280ccec2252390 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.025+0000 7f8458a00640 1 -- 172.21.15.165:0/4252300619 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8454036070 con 0x7f845c0ce590 2024-04-25T10:12:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.025+0000 7f8458a00640 1 -- 172.21.15.165:0/4252300619 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8454031600 con 0x7f845c0ce590 2024-04-25T10:12:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.025+0000 7f8458a00640 1 -- 172.21.15.165:0/4252300619 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84540319d0 con 0x7f845c0ce590 2024-04-25T10:12:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 -- 172.21.15.165:0/4252300619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 msgr2=0x7f845c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 --2- 172.21.15.165:0/4252300619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d6f0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f845c0d02e0 tx=0x7f845402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 -- 172.21.15.165:0/4252300619 shutdown_connections 2024-04-25T10:12:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 --2- 172.21.15.165:0/4252300619 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d6f0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 --2- 172.21.15.165:0/4252300619 >> v2:172.21.15.172:3300/0 conn(0x7f845c154ee0 0x7f845c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 -- 172.21.15.165:0/4252300619 >> 172.21.15.165:0/4252300619 conn(0x7f845c06abe0 msgr2=0x7f845c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 -- 172.21.15.165:0/4252300619 shutdown_connections 2024-04-25T10:12:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.026+0000 7f8461000640 1 -- 172.21.15.165:0/4252300619 wait complete. 2024-04-25T10:12:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.028+0000 7f8461000640 1 Processor -- start 2024-04-25T10:12:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.028+0000 7f8461000640 1 -- start start 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.029+0000 7f8461000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.029+0000 7f8461000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f845c0cfcf0 0x7f845c0ac290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8461000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f845c154ee0 0x7f845c0ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8461000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c1552c0 con 0x7f845c0ce590 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8461000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f845c071d20 con 0x7f845c0cfcf0 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8461000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f845c155440 con 0x7f845c154ee0 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51570/0 (socket says 172.21.15.165:51570) 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 -- 172.21.15.165:0/3372080885 learned_addr learned my addr 172.21.15.165:0/3372080885 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459400640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f845c0cfcf0 0x7f845c0ac290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f845a800640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f845c154ee0 0x7f845c0ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 -- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f845c154ee0 msgr2=0x7f845c0ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f845c154ee0 0x7f845c0ac7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 -- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f845c0cfcf0 msgr2=0x7f845c0ac290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f845c0cfcf0 0x7f845c0ac290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.030+0000 7f8459e00640 1 -- 172.21.15.165:0/3372080885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f845c0a94b0 con 0x7f845c0ce590 2024-04-25T10:12:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.031+0000 7f8459e00640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d7f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c7f0 tx=0x7f844c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.031+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c015070 con 0x7f845c0ce590 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.031+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f844c007820 con 0x7f845c0ce590 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.031+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f845c0a86f0 con 0x7f845c0ce590 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.031+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f844c007bd0 con 0x7f845c0ce590 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.031+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f845c0a8cb0 con 0x7f845c0ce590 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.032+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f845c073990 con 0x7f845c0ce590 2024-04-25T10:12:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.033+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f844c018070 con 0x7f845c0ce590 2024-04-25T10:12:48.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.034+0000 7f8452000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f843c06b500 0x7f843c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.034+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f844c08a3e0 con 0x7f845c0ce590 2024-04-25T10:12:48.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.034+0000 7f8459400640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f843c06b500 0x7f843c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.035+0000 7f8459400640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f843c06b500 0x7f843c06d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f845c0cc8a0 tx=0x7f8448009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.039+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f844c059130 con 0x7f845c0ce590 2024-04-25T10:12:48.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.257+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f845c06d360 con 0x7f845c0ce590 2024-04-25T10:12:48.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.258+0000 7f8452000640 1 -- 172.21.15.165:0/3372080885 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f844c058a30 con 0x7f845c0ce590 2024-04-25T10:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:12:48.263 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:12:48.263 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:12:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f843c06b500 msgr2=0x7f843c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f843c06b500 0x7f843c06d9c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f845c0cc8a0 tx=0x7f8448009040 comp rx=0 tx=0).stop 2024-04-25T10:12:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 msgr2=0x7f845c06d7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d7f0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f844c00c7f0 tx=0x7f844c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 shutdown_connections 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f843c06b500 0x7f843c06d9c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f845c154ee0 0x7f845c0ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f845c0cfcf0 0x7f845c0ac290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 --2- 172.21.15.165:0/3372080885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f845c0ce590 0x7f845c06d7f0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.260+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 >> 172.21.15.165:0/3372080885 conn(0x7f845c06abe0 msgr2=0x7f845c06e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.261+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 shutdown_connections 2024-04-25T10:12:48.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.261+0000 7f8461000640 1 -- 172.21.15.165:0/3372080885 wait complete. 2024-04-25T10:12:48.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 74, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:12:36.858581+0000', 'modified': '2024-04-25T10:12:39.947364+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8043}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8043': {'gid': 8043, 'name': 'd', 'rank': 0, 'incarnation': 70, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6835/437969658', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6833', 'nonce': 437969658}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 437969658}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:12:48.686 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:12:48.686 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:12:48.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.890+0000 7fea31600640 1 Processor -- start 2024-04-25T10:12:48.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.891+0000 7fea31600640 1 -- start start 2024-04-25T10:12:48.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.893+0000 7fea31600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fea2c145140 0x7fea2c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.893+0000 7fea31600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea2c0c5200 0x7fea2c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.893+0000 7fea31600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea2c0c5650 con 0x7fea2c0ce950 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.893+0000 7fea31600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c0ced90 con 0x7fea2c145140 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.893+0000 7fea31600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2c145520 con 0x7fea2c0c5200 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.893+0000 7fea2be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fea2c145140 0x7fea2c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fea2c0ce950 0x7fea2c0c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45320/0 (socket says 172.21.15.165:45320) 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2b400640 1 -- 172.21.15.165:0/387502226 learned_addr learned my addr 172.21.15.165:0/387502226 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2be00640 1 -- 172.21.15.165:0/387502226 >> v2:172.21.15.172:3300/0 conn(0x7fea2c0c5200 msgr2=0x7fea2c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2be00640 1 --2- 172.21.15.165:0/387502226 >> v2:172.21.15.172:3300/0 conn(0x7fea2c0c5200 0x7fea2c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2be00640 1 -- 172.21.15.165:0/387502226 >> v1:172.21.15.165:6789/0 conn(0x7fea2c0ce950 legacy=0x7fea2c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2be00640 1 -- 172.21.15.165:0/387502226 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c071270 con 0x7fea2c145140 2024-04-25T10:12:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.894+0000 7fea2aa00640 1 -- 172.21.15.165:0/387502226 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845169779 0 0) 0x7fea2c0c5650 con 0x7fea2c0ce950 2024-04-25T10:12:48.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.895+0000 7fea2be00640 1 --2- 172.21.15.165:0/387502226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0ce410 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fea2c06bb90 tx=0x7fea1c02a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a18e040c5f47f66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.895+0000 7fea2aa00640 1 -- 172.21.15.165:0/387502226 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c037070 con 0x7fea2c145140 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.895+0000 7fea2aa00640 1 -- 172.21.15.165:0/387502226 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c032440 con 0x7fea2c145140 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.895+0000 7fea2aa00640 1 -- 172.21.15.165:0/387502226 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c02ae60 con 0x7fea2c145140 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 -- 172.21.15.165:0/387502226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 msgr2=0x7fea2c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 --2- 172.21.15.165:0/387502226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0ce410 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fea2c06bb90 tx=0x7fea1c02a110 comp rx=0 tx=0).stop 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 -- 172.21.15.165:0/387502226 shutdown_connections 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 --2- 172.21.15.165:0/387502226 >> v2:172.21.15.172:3300/0 conn(0x7fea2c0c5200 0x7fea2c15bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 --2- 172.21.15.165:0/387502226 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0ce410 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 -- 172.21.15.165:0/387502226 >> 172.21.15.165:0/387502226 conn(0x7fea2c061cc0 msgr2=0x7fea2c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 -- 172.21.15.165:0/387502226 shutdown_connections 2024-04-25T10:12:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.896+0000 7fea31600640 1 -- 172.21.15.165:0/387502226 wait complete. 2024-04-25T10:12:48.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.898+0000 7fea31600640 1 Processor -- start 2024-04-25T10:12:48.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.898+0000 7fea31600640 1 -- start start 2024-04-25T10:12:48.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea31600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea2c0c5200 0x7fea2c13f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea31600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea2c0ce950 0x7fea2c1f22e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea31600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea31600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c071b10 con 0x7fea2c145140 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea31600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c071830 con 0x7fea2c0c5200 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea31600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c15c4f0 con 0x7fea2c0ce950 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea30c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea30c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0c6a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51582/0 (socket says 172.21.15.165:51582) 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.899+0000 7fea30c00640 1 -- 172.21.15.165:0/1657791591 learned_addr learned my addr 172.21.15.165:0/1657791591 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea30c00640 1 -- 172.21.15.165:0/1657791591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea2c0ce950 msgr2=0x7fea2c1f22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea30c00640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea2c0ce950 0x7fea2c1f22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea30c00640 1 -- 172.21.15.165:0/1657791591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea2c0c5200 msgr2=0x7fea2c13f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea30c00640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea2c0c5200 0x7fea2c13f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea30c00640 1 -- 172.21.15.165:0/1657791591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c0c7540 con 0x7fea2c145140 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea30c00640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0c6a80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fea2400b970 tx=0x7fea2400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea23e00640 1 -- 172.21.15.165:0/1657791591 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24019070 con 0x7fea2c145140 2024-04-25T10:12:48.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea23e00640 1 -- 172.21.15.165:0/1657791591 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2400cb60 con 0x7fea2c145140 2024-04-25T10:12:48.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea2c0c7820 con 0x7fea2c145140 2024-04-25T10:12:48.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea23e00640 1 -- 172.21.15.165:0/1657791591 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea24014440 con 0x7fea2c145140 2024-04-25T10:12:48.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.900+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea2c0c8da0 con 0x7fea2c145140 2024-04-25T10:12:48.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.902+0000 7fea23e00640 1 -- 172.21.15.165:0/1657791591 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fea2401c070 con 0x7fea2c145140 2024-04-25T10:12:48.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.904+0000 7fea23e00640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fea0c06b500 0x7fea0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.904+0000 7fea23e00640 1 -- 172.21.15.165:0/1657791591 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fea2408a100 con 0x7fea2c145140 2024-04-25T10:12:48.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.904+0000 7fea2be00640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fea0c06b500 0x7fea0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.905+0000 7fea359a0740 1 Processor -- start 2024-04-25T10:12:48.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.905+0000 7fea359a0740 1 -- start start 2024-04-25T10:12:48.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.905+0000 7fea2be00640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fea0c06b500 0x7fea0c06d9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fea1c034000 tx=0x7fea1c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea359a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1716d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 0x55d9c16efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16edfc0 0x55d9c1735880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea359a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c1652620 con 0x55d9c16eeb30 2024-04-25T10:12:48.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c15fbca0 con 0x55d9c16eef10 2024-04-25T10:12:48.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9c164e6c0 con 0x55d9c16edfc0 2024-04-25T10:12:48.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1716d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1716d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51584/0 (socket says 172.21.15.165:51584) 2024-04-25T10:12:48.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 -- 172.21.15.165:0/2773123842 learned_addr learned my addr 172.21.15.165:0/2773123842 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 -- 172.21.15.165:0/2773123842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16edfc0 msgr2=0x55d9c1735880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16edfc0 0x55d9c1735880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 -- 172.21.15.165:0/2773123842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 msgr2=0x55d9c16efc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 0x55d9c16efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.906+0000 7fea2be00640 1 -- 172.21.15.165:0/2773123842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9c1602f40 con 0x55d9c16eeb30 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.907+0000 7fea2be00640 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1716d50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02a700 tx=0x7fea1c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d73dfac2ef501c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.907+0000 7fea17e00640 1 -- 172.21.15.165:0/2773123842 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c037040 con 0x55d9c16eeb30 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.907+0000 7fea17e00640 1 -- 172.21.15.165:0/2773123842 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c042070 con 0x55d9c16eeb30 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.907+0000 7fea17e00640 1 -- 172.21.15.165:0/2773123842 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c03d6b0 con 0x55d9c16eeb30 2024-04-25T10:12:48.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 -- 172.21.15.165:0/2773123842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 msgr2=0x55d9c1716d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1716d50 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fea1c02a700 tx=0x7fea1c008030 comp rx=0 tx=0).stop 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 -- 172.21.15.165:0/2773123842 shutdown_connections 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16edfc0 0x55d9c1735880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 0x55d9c16efc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 --2- 172.21.15.165:0/2773123842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1716d50 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 -- 172.21.15.165:0/2773123842 >> 172.21.15.165:0/2773123842 conn(0x55d9c16ebd50 msgr2=0x55d9c1734730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 -- 172.21.15.165:0/2773123842 shutdown_connections 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 -- 172.21.15.165:0/2773123842 wait complete. 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.908+0000 7fea359a0740 1 -- start start 2024-04-25T10:12:48.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.909+0000 7fea359a0740 1 Processor -- start 2024-04-25T10:12:48.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1713fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 0x55d9c171a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c1737800 0x55d9c1718c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c16e9c60 con 0x55d9c16eeb30 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c15f11f0 con 0x55d9c16eef10 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9c17350f0 con 0x55d9c1737800 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea359a0740 10 client.0 _ll_register_callbacks cb 0x55d9c160ead0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1713fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1713fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51592/0 (socket says 172.21.15.165:51592) 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 -- 172.21.15.165:0/2263178373 learned_addr learned my addr 172.21.15.165:0/2263178373 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 -- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c1737800 msgr2=0x55d9c1718c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c1737800 0x55d9c1718c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 -- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 msgr2=0x55d9c171a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 0x55d9c171a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.910+0000 7fea2be00640 1 -- 172.21.15.165:0/2263178373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9c1610c50 con 0x55d9c16eeb30 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea2be00640 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1713fe0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c02c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=860a6aaa939a69d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea14c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:48.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea14c00640 1 -- 172.21.15.165:0/2263178373 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c05e060 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea14c00640 1 -- 172.21.15.165:0/2263178373 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fea1c059480 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea1c042070 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea14c00640 1 -- 172.21.15.165:0/2263178373 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea1c059680 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea359a0740 10 client.8202 fetch_fsmap learned FSMap version 74 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55d9c13672d0 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.911+0000 7fea14c00640 1 -- 172.21.15.165:0/2263178373 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c042070 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea14c00640 1 -- 172.21.15.165:0/2263178373 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fea1c067020 con 0x55d9c16eeb30 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea359a0740 10 client.8202 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea359a0740 10 client.8202 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea359a0740 20 client.8202 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea359a0740 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1735810 0x55d9c1735c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea359a0740 4 client.8202 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9c165fb60 con 0x55d9c1735810 2024-04-25T10:12:48.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.912+0000 7fea2b400640 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1735810 0x55d9c1735c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.913+0000 7fea2b400640 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1735810 0x55d9c1735c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3b6fadbb5462f31 server_cookie=a48a1618b93ba28c in_seq=0 out_seq=0 2024-04-25T10:12:48.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.913+0000 7fea14c00640 10 client.8202 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:12:48.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.915+0000 7fea14c00640 1 -- 172.21.15.165:0/2263178373 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55d9c165fb60 con 0x55d9c1735810 2024-04-25T10:12:48.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.915+0000 7fea14c00640 10 client.8202 handle_command_reply: tid=0 2024-04-25T10:12:48.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.915+0000 7fea359a0740 1 client.8202 shutdown 2024-04-25T10:12:48.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 msgr2=0x55d9c1713fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1713fe0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fea1c000c00 tx=0x7fea1c02c070 comp rx=0 tx=0).stop 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 shutdown_connections 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1735810 0x55d9c1735c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c1737800 0x55d9c1718c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c16eef10 0x55d9c171a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea14c00640 0 client.8202 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 --2- 172.21.15.165:0/2263178373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eeb30 0x55d9c1713fe0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 >> 172.21.15.165:0/2263178373 conn(0x55d9c16edfc0 msgr2=0x55d9c17337d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 shutdown_connections 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 1 -- 172.21.15.165:0/2263178373 wait complete. 2024-04-25T10:12:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.916+0000 7fea359a0740 20 client.8202 trim_cache size 0 max 16384 2024-04-25T10:12:48.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.933+0000 7fea359a0740 1 Processor -- start 2024-04-25T10:12:48.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.933+0000 7fea359a0740 1 -- start start 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c16e8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea359a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c1736cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c17371f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea359a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0cdd40 con 0x55d9c16eef10 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c1f5150 con 0x55d9c1737800 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea2c0cd740 con 0x55d9c16eeb30 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c1736cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c1736cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51602/0 (socket says 172.21.15.165:51602) 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 -- 172.21.15.165:0/3014718180 learned_addr learned my addr 172.21.15.165:0/3014718180 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 -- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 msgr2=0x55d9c16e8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2be00640 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c16e8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c16e8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea30c00640 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c17371f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 -- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 msgr2=0x55d9c17371f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c17371f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea2b400640 1 -- 172.21.15.165:0/3014718180 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9c14316b0 con 0x55d9c16eef10 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.934+0000 7fea30c00640 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c17371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:12:48.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.935+0000 7fea2b400640 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c1736cb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x55d9c16f1080 tx=0x7fea18007900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=790e746f63c47b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.935+0000 7fea17400640 1 -- 172.21.15.165:0/3014718180 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea180049e0 con 0x55d9c16eef10 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.935+0000 7fea17400640 1 -- 172.21.15.165:0/3014718180 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18004b80 con 0x55d9c16eef10 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.935+0000 7fea17400640 1 -- 172.21.15.165:0/3014718180 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18007640 con 0x55d9c16eef10 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 -- 172.21.15.165:0/3014718180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 msgr2=0x55d9c1736cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c1736cb0 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x55d9c16f1080 tx=0x7fea18007900 comp rx=0 tx=0).stop 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 -- 172.21.15.165:0/3014718180 shutdown_connections 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c17371f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c1736cb0 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 --2- 172.21.15.165:0/3014718180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c16e8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 -- 172.21.15.165:0/3014718180 >> 172.21.15.165:0/3014718180 conn(0x55d9c16edfc0 msgr2=0x55d9c1716670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 -- 172.21.15.165:0/3014718180 shutdown_connections 2024-04-25T10:12:48.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.936+0000 7fea359a0740 1 -- 172.21.15.165:0/3014718180 wait complete. 2024-04-25T10:12:48.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.937+0000 7fea359a0740 1 -- start start 2024-04-25T10:12:48.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.937+0000 7fea359a0740 1 Processor -- start 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c174ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c17370b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c1730a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c171b210 con 0x55d9c16eef10 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9c1718710 con 0x55d9c1737800 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d9c17176c0 con 0x55d9c16eeb30 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea359a0740 10 client.0 _ll_register_callbacks cb 0x7fea1c009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:12:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c17370b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c17370b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51606/0 (socket says 172.21.15.165:51606) 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 -- 172.21.15.165:0/1414891746 learned_addr learned my addr 172.21.15.165:0/1414891746 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 -- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 msgr2=0x55d9c174ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c174ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 -- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 msgr2=0x55d9c1730a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c1730a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.938+0000 7fea2b400640 1 -- 172.21.15.165:0/1414891746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9c174b050 con 0x55d9c16eef10 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea2b400640 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c17370b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0a8550 tx=0x7fea18001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa1b21b02e40e26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea14c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea14c00640 1 -- 172.21.15.165:0/1414891746 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea18007920 con 0x55d9c16eef10 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea14c00640 1 -- 172.21.15.165:0/1414891746 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fea180047e0 con 0x55d9c16eef10 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fea18004b80 con 0x55d9c16eef10 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea14c00640 1 -- 172.21.15.165:0/1414891746 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea18030050 con 0x55d9c16eef10 2024-04-25T10:12:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea359a0740 10 client.8208 fetch_fsmap learned FSMap version 74 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x55d9c1734ee0 con 0x55d9c16eef10 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea14c00640 1 -- 172.21.15.165:0/1414891746 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea18004b80 con 0x55d9c16eef10 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.939+0000 7fea14c00640 1 -- 172.21.15.165:0/1414891746 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fea18001650 con 0x55d9c16eef10 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea359a0740 10 client.8208 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea359a0740 10 client.8208 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea359a0740 20 client.8208 populate_metadata read hostname 'smithi165' 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea359a0740 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1711c20 0x55d9c1712000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea359a0740 4 client.8208 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:12:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d9c16ee790 con 0x55d9c1711c20 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.940+0000 7fea2be00640 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1711c20 0x55d9c1712000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.941+0000 7fea2be00640 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1711c20 0x55d9c1712000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c62b7d6089bdfb28 server_cookie=3a9b10594a3145c9 in_seq=0 out_seq=0 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.941+0000 7fea14c00640 10 client.8208 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea14c00640 1 -- 172.21.15.165:0/1414891746 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55d9c16ee790 con 0x55d9c1711c20 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea14c00640 10 client.8208 handle_command_reply: tid=0 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 client.8208 shutdown 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 msgr2=0x55d9c17370b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c17370b0 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0a8550 tx=0x7fea18001160 comp rx=0 tx=0).stop 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 shutdown_connections 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x55d9c1711c20 0x55d9c1712000 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55d9c1737800 0x55d9c1730a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55d9c16eef10 0x55d9c17370b0 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 --2- 172.21.15.165:0/1414891746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55d9c16eeb30 0x55d9c174ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea14c00640 0 client.8208 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 >> 172.21.15.165:0/1414891746 conn(0x55d9c16edfc0 msgr2=0x55d9c16ed270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 shutdown_connections 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 1 -- 172.21.15.165:0/1414891746 wait complete. 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.942+0000 7fea359a0740 20 client.8208 trim_cache size 0 max 16384 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 8043, 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:12:48.949 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 11.533989091, 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 74, 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 113, 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 111, 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 17.007349597000001 2024-04-25T10:12:48.950 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fea0c06b500 msgr2=0x7fea0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fea0c06b500 0x7fea0c06d9c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fea1c034000 tx=0x7fea1c02c000 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 msgr2=0x7fea2c0c6a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0c6a80 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fea2400b970 tx=0x7fea2400be30 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 shutdown_connections 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fea0c06b500 0x7fea0c06d9c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea2c145140 0x7fea2c0c6a80 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea2c0ce950 0x7fea2c1f22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 --2- 172.21.15.165:0/1657791591 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea2c0c5200 0x7fea2c13f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 >> 172.21.15.165:0/1657791591 conn(0x7fea2c061cc0 msgr2=0x7fea2c144300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 shutdown_connections 2024-04-25T10:12:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:48.943+0000 7fea31600640 1 -- 172.21.15.165:0/1657791591 wait complete. 2024-04-25T10:12:48.963 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:12:49.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.143+0000 7f6eb1c00640 1 Processor -- start 2024-04-25T10:12:49.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.143+0000 7f6eb1c00640 1 -- start start 2024-04-25T10:12:49.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.145+0000 7f6eb1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6eac0ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.145+0000 7f6eb1c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac144b30 con 0x7f6eac06d290 2024-04-25T10:12:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.145+0000 7f6eb1c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6eac0712d0 con 0x7f6eac144750 2024-04-25T10:12:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.145+0000 7f6eb1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0715b0 con 0x7f6eac0ce920 2024-04-25T10:12:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.145+0000 7f6eab400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6eac06d290 0x7f6eac06d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:45332/0 (socket says 172.21.15.165:45332) 2024-04-25T10:12:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.145+0000 7f6eab400640 1 -- 172.21.15.165:0/801182480 learned_addr learned my addr 172.21.15.165:0/801182480 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3714630174 0 0) 0x7f6eac144b30 con 0x7f6eac06d290 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e8c003520 con 0x7f6eac06d290 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6ea3e00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6eac0ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2268629531 0 0) 0x7f6e8c003520 con 0x7f6eac06d290 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 msgr2=0x7f6eac0ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6eac0ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 >> v1:172.21.15.172:6789/0 conn(0x7f6eac144750 legacy=0x7f6eac06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac071890 con 0x7f6eac06d290 2024-04-25T10:12:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6ea40019f0 con 0x7f6eac06d290 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 >> v1:172.21.15.165:6789/0 conn(0x7f6eac06d290 legacy=0x7f6eac06d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8c0039e0 0x7f6e8c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c004800 0x7f6e8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6e8c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac0715b0 con 0x7f6e8c0039e0 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c00bb50 con 0x7f6e8c004800 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.146+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8c00bd50 con 0x7f6eac0ce920 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8c0039e0 0x7f6e8c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eab400640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6e8c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6ea3e00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c004800 0x7f6e8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eaaa00640 1 -- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 msgr2=0x7f6e8c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6e8c005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eaaa00640 1 -- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c004800 msgr2=0x7f6e8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c004800 0x7f6e8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eaaa00640 1 -- 172.21.15.165:0/801182480 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8c003520 con 0x7f6e8c0039e0 2024-04-25T10:12:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6ea3e00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c004800 0x7f6e8c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.147+0000 7f6eab400640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac0ce920 0x7f6e8c005270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.149+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8c0039e0 0x7f6e8c003de0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c003dd0 tx=0x7f6e8c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aabff2629487344f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.149+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c037070 con 0x7f6e8c0039e0 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.149+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9c02aa20 con 0x7f6e8c0039e0 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.149+0000 7f6eaa000640 1 -- 172.21.15.165:0/801182480 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9c0039a0 con 0x7f6e8c0039e0 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eb1c00640 1 -- 172.21.15.165:0/801182480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8c0039e0 msgr2=0x7f6e8c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/801182480 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8c0039e0 0x7f6e8c003de0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c003dd0 tx=0x7f6e8c005830 comp rx=0 tx=0).stop 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eab400640 1 -- 172.21.15.165:0/801182480 reap_dead start 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eb1c00640 1 -- 172.21.15.165:0/801182480 shutdown_connections 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eb1c00640 1 -- 172.21.15.165:0/801182480 >> 172.21.15.165:0/801182480 conn(0x7f6eac06abe0 msgr2=0x7f6eac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eb1c00640 1 -- 172.21.15.165:0/801182480 shutdown_connections 2024-04-25T10:12:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.150+0000 7f6eb1c00640 1 -- 172.21.15.165:0/801182480 wait complete. 2024-04-25T10:12:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.152+0000 7f6eb1c00640 1 Processor -- start 2024-04-25T10:12:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.152+0000 7f6eb1c00640 1 -- start start 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eb1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c0039e0 0x7f6eac1e9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eb1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac13dde0 0x7f6eac1458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eb1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 0x7f6eac1462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eb1c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac145450 con 0x7f6eac145e30 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eb1c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac06f020 con 0x7f6e8c0039e0 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eb1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6eac1455d0 con 0x7f6eac13dde0 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 0x7f6eac1462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 0x7f6eac1462e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51634/0 (socket says 172.21.15.165:51634) 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 -- 172.21.15.165:0/915268540 learned_addr learned my addr 172.21.15.165:0/915268540 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6ea3e00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac13dde0 0x7f6eac1458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c0039e0 0x7f6eac1e9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 -- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac13dde0 msgr2=0x7f6eac1458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac13dde0 0x7f6eac1458f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 -- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c0039e0 msgr2=0x7f6eac1e9e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c0039e0 0x7f6eac1e9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.153+0000 7f6eab400640 1 -- 172.21.15.165:0/915268540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eac146920 con 0x7f6eac145e30 2024-04-25T10:12:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.154+0000 7f6eab400640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 0x7f6eac1462e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4001330 tx=0x7f6ea400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.154+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea400eb90 con 0x7f6eac145e30 2024-04-25T10:12:49.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.154+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4018070 con 0x7f6eac145e30 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.154+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4013420 con 0x7f6eac145e30 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.154+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eac146d60 con 0x7f6eac145e30 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.154+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eac147320 con 0x7f6eac145e30 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.155+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6eac06aff0 con 0x7f6eac145e30 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.156+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ea40135c0 con 0x7f6eac145e30 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.158+0000 7f6ea2a00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:12:49.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.158+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6ea408a6d0 con 0x7f6eac145e30 2024-04-25T10:12:49.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.159+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:12:49.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.160+0000 7f6eaaa00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:12:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.163+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6eac06aff0 con 0x7f6eac145e30 2024-04-25T10:12:49.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.398+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6eac073990 con 0x7f6eac145e30 2024-04-25T10:12:49.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.399+0000 7f6ea2a00640 1 -- 172.21.15.165:0/915268540 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v74) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6ea4059380 con 0x7f6eac145e30 2024-04-25T10:12:49.403 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:12:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 msgr2=0x7f6e8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c034000 tx=0x7f6e9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:12:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 msgr2=0x7f6eac1462e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:12:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 0x7f6eac1462e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4001330 tx=0x7f6ea400e530 comp rx=0 tx=0).stop 2024-04-25T10:12:49.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 shutdown_connections 2024-04-25T10:12:49.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e8806b4d0 0x7f6e8806d990 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6eac145e30 0x7f6eac1462e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6eac13dde0 0x7f6eac1458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 --2- 172.21.15.165:0/915268540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8c0039e0 0x7f6eac1e9e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:12:49.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.401+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 >> 172.21.15.165:0/915268540 conn(0x7f6eac06abe0 msgr2=0x7f6eac0cfab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:12:49.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.402+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 shutdown_connections 2024-04-25T10:12:49.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:12:49.402+0000 7f6eb1c00640 1 -- 172.21.15.165:0/915268540 wait complete. 2024-04-25T10:12:49.424 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:12:49.424 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:12:49.424 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:12:49.424 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:12:49.425 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:49.425 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:12:49.425 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:12:49.425 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:12:49.425 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:49.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:49.425 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:12:49.473 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:12:49.474 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41996sec preferred_lft 41996sec 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:49.475 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:12:49.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:49.476 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:12:49.596 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:12:49.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:49.600 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:12:49.650 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:12:49.651 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:12:49.651 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:12:49.651 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:12:49.651 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:12:49.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:49.651 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:12:49.651 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:49.651 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:12:49.651 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:12:49.652 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:12:49.652 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:49.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:49.795 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:12:49.844 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:12:49.844 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:12:49.844 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:49.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:49.844 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:12:49.926 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:49.926 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:49.981 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:12:49.981 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:12:49.981 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:50.032 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:12:50.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:50.033 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:50.163 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:50.163 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:50.164 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:12:50.184 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:50.184 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:50.184 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:12:50.184 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:12:50.184 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:12:50.211 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:12:50.213 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:12:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:12:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:12:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:12:50.213 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:12:50.218 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:12:50.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:50.218 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:50.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:50.293 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:12:50.364 INFO:teuthology.orchestra.run.smithi165.stdout:client8217 2024-04-25T10:12:50.365 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:12:50.365 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:12:50.417 INFO:tasks.cephfs.test_scrub_checks:mountpoint: /home/ubuntu/cephtest/mnt.0 2024-04-25T10:12:50.417 INFO:tasks.cephfs.test_scrub_checks:client_path: /home/ubuntu/cephtest/mnt.0/scrub_control_test_path 2024-04-25T10:12:50.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:50.417 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:50.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.535 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0305365 s, 34.3 MB/s 2024-04-25T10:12:50.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.561 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236918 s, 44.3 MB/s 2024-04-25T10:12:50.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244212 s, 42.9 MB/s 2024-04-25T10:12:50.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276263 s, 38.0 MB/s 2024-04-25T10:12:50.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239992 s, 43.7 MB/s 2024-04-25T10:12:50.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.684 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033205 s, 31.6 MB/s 2024-04-25T10:12:50.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222912 s, 47.0 MB/s 2024-04-25T10:12:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261676 s, 40.1 MB/s 2024-04-25T10:12:50.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.763 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193378 s, 54.2 MB/s 2024-04-25T10:12:50.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202545 s, 51.8 MB/s 2024-04-25T10:12:50.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192329 s, 54.5 MB/s 2024-04-25T10:12:50.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178848 s, 58.6 MB/s 2024-04-25T10:12:50.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208302 s, 50.3 MB/s 2024-04-25T10:12:50.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021327 s, 49.2 MB/s 2024-04-25T10:12:50.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.908 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211661 s, 49.5 MB/s 2024-04-25T10:12:50.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229236 s, 45.7 MB/s 2024-04-25T10:12:50.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021314 s, 49.2 MB/s 2024-04-25T10:12:50.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210937 s, 49.7 MB/s 2024-04-25T10:12:51.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.008 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198568 s, 52.8 MB/s 2024-04-25T10:12:51.029 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.029 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.029 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182739 s, 57.4 MB/s 2024-04-25T10:12:51.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.048 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157401 s, 66.6 MB/s 2024-04-25T10:12:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212264 s, 49.4 MB/s 2024-04-25T10:12:51.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.098 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207281 s, 50.6 MB/s 2024-04-25T10:12:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.121 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186607 s, 56.2 MB/s 2024-04-25T10:12:51.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.147 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235408 s, 44.5 MB/s 2024-04-25T10:12:51.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189371 s, 55.4 MB/s 2024-04-25T10:12:51.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189757 s, 55.3 MB/s 2024-04-25T10:12:51.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226989 s, 46.2 MB/s 2024-04-25T10:12:51.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.241 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192799 s, 54.4 MB/s 2024-04-25T10:12:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168766 s, 62.1 MB/s 2024-04-25T10:12:51.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.287 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229359 s, 45.7 MB/s 2024-04-25T10:12:51.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227278 s, 46.1 MB/s 2024-04-25T10:12:51.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:51.316 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:51.316 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:51.316 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:12:51.316 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:51.316 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:51.316 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:51.317 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:51.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.389 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180241 s, 58.2 MB/s 2024-04-25T10:12:51.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193811 s, 54.1 MB/s 2024-04-25T10:12:51.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202365 s, 51.8 MB/s 2024-04-25T10:12:51.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.464 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253667 s, 41.3 MB/s 2024-04-25T10:12:51.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177398 s, 59.1 MB/s 2024-04-25T10:12:51.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170934 s, 61.3 MB/s 2024-04-25T10:12:51.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205279 s, 51.1 MB/s 2024-04-25T10:12:51.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182089 s, 57.6 MB/s 2024-04-25T10:12:51.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.575 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191743 s, 54.7 MB/s 2024-04-25T10:12:51.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.605 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261965 s, 40.0 MB/s 2024-04-25T10:12:51.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019744 s, 53.1 MB/s 2024-04-25T10:12:51.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.654 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206697 s, 50.7 MB/s 2024-04-25T10:12:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202088 s, 51.9 MB/s 2024-04-25T10:12:51.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182469 s, 57.5 MB/s 2024-04-25T10:12:51.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.724 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.724 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206256 s, 50.8 MB/s 2024-04-25T10:12:51.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198925 s, 52.7 MB/s 2024-04-25T10:12:51.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.771 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200149 s, 52.4 MB/s 2024-04-25T10:12:51.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.795 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208332 s, 50.3 MB/s 2024-04-25T10:12:51.820 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.820 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.820 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209538 s, 50.0 MB/s 2024-04-25T10:12:51.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169885 s, 61.7 MB/s 2024-04-25T10:12:51.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169772 s, 61.8 MB/s 2024-04-25T10:12:51.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194481 s, 53.9 MB/s 2024-04-25T10:12:51.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.911 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222532 s, 47.1 MB/s 2024-04-25T10:12:51.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200716 s, 52.2 MB/s 2024-04-25T10:12:51.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222385 s, 47.2 MB/s 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:51.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:51.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222895 s, 47.0 MB/s 2024-04-25T10:12:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233744 s, 44.9 MB/s 2024-04-25T10:12:52.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209375 s, 50.1 MB/s 2024-04-25T10:12:52.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228645 s, 45.9 MB/s 2024-04-25T10:12:52.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.087 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222546 s, 47.1 MB/s 2024-04-25T10:12:52.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.112 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218802 s, 47.9 MB/s 2024-04-25T10:12:52.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.133 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181676 s, 57.7 MB/s 2024-04-25T10:12:52.136 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:52.136 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:52.136 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:52.136 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:12:52.136 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:52.136 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:52.136 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:52.137 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:52.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194765 s, 53.8 MB/s 2024-04-25T10:12:52.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274657 s, 38.2 MB/s 2024-04-25T10:12:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240754 s, 43.6 MB/s 2024-04-25T10:12:52.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259096 s, 40.5 MB/s 2024-04-25T10:12:52.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198152 s, 52.9 MB/s 2024-04-25T10:12:52.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.356 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222014 s, 47.2 MB/s 2024-04-25T10:12:52.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238904 s, 43.9 MB/s 2024-04-25T10:12:52.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217752 s, 48.2 MB/s 2024-04-25T10:12:52.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.432 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189843 s, 55.2 MB/s 2024-04-25T10:12:52.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.455 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199163 s, 52.6 MB/s 2024-04-25T10:12:52.480 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.480 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.480 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209941 s, 49.9 MB/s 2024-04-25T10:12:52.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.503 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206829 s, 50.7 MB/s 2024-04-25T10:12:52.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189827 s, 55.2 MB/s 2024-04-25T10:12:52.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.553 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245381 s, 42.7 MB/s 2024-04-25T10:12:52.579 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.579 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.579 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222638 s, 47.1 MB/s 2024-04-25T10:12:52.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.606 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220837 s, 47.5 MB/s 2024-04-25T10:12:52.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.630 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202603 s, 51.8 MB/s 2024-04-25T10:12:52.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.658 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239626 s, 43.8 MB/s 2024-04-25T10:12:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193432 s, 54.2 MB/s 2024-04-25T10:12:52.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.705 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198271 s, 52.9 MB/s 2024-04-25T10:12:52.725 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179266 s, 58.5 MB/s 2024-04-25T10:12:52.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192441 s, 54.5 MB/s 2024-04-25T10:12:52.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216377 s, 48.5 MB/s 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226564 s, 46.3 MB/s 2024-04-25T10:12:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206258 s, 50.8 MB/s 2024-04-25T10:12:52.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023563 s, 44.5 MB/s 2024-04-25T10:12:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.876 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202397 s, 51.8 MB/s 2024-04-25T10:12:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217326 s, 48.2 MB/s 2024-04-25T10:12:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244351 s, 42.9 MB/s 2024-04-25T10:12:52.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.957 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227821 s, 46.0 MB/s 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224831 s, 46.6 MB/s 2024-04-25T10:12:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241406 s, 43.4 MB/s 2024-04-25T10:12:53.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:53.017 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:53.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020866 s, 50.3 MB/s 2024-04-25T10:12:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209351 s, 50.1 MB/s 2024-04-25T10:12:53.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243072 s, 43.1 MB/s 2024-04-25T10:12:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214984 s, 48.8 MB/s 2024-04-25T10:12:53.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.214 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263434 s, 39.8 MB/s 2024-04-25T10:12:53.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.240 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.240 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022019 s, 47.6 MB/s 2024-04-25T10:12:53.268 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.268 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.268 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244285 s, 42.9 MB/s 2024-04-25T10:12:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226473 s, 46.3 MB/s 2024-04-25T10:12:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020469 s, 51.2 MB/s 2024-04-25T10:12:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251252 s, 41.7 MB/s 2024-04-25T10:12:53.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243243 s, 43.1 MB/s 2024-04-25T10:12:53.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223886 s, 46.8 MB/s 2024-04-25T10:12:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194709 s, 53.9 MB/s 2024-04-25T10:12:53.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024729 s, 42.4 MB/s 2024-04-25T10:12:53.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.479 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183099 s, 57.3 MB/s 2024-04-25T10:12:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227084 s, 46.2 MB/s 2024-04-25T10:12:53.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206431 s, 50.8 MB/s 2024-04-25T10:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022458 s, 46.7 MB/s 2024-04-25T10:12:53.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207023 s, 50.7 MB/s 2024-04-25T10:12:53.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193542 s, 54.2 MB/s 2024-04-25T10:12:53.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201855 s, 51.9 MB/s 2024-04-25T10:12:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231889 s, 45.2 MB/s 2024-04-25T10:12:53.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183644 s, 57.1 MB/s 2024-04-25T10:12:53.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.708 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270489 s, 38.8 MB/s 2024-04-25T10:12:53.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210214 s, 49.9 MB/s 2024-04-25T10:12:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017088 s, 61.4 MB/s 2024-04-25T10:12:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264856 s, 39.6 MB/s 2024-04-25T10:12:53.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.809 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210737 s, 49.8 MB/s 2024-04-25T10:12:53.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176868 s, 59.3 MB/s 2024-04-25T10:12:53.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204889 s, 51.2 MB/s 2024-04-25T10:12:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182281 s, 57.5 MB/s 2024-04-25T10:12:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200529 s, 52.3 MB/s 2024-04-25T10:12:53.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:53.903 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:53.903 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:53.903 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:12:53.903 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:53.903 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:53.904 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:53.904 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:53.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:53.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:53.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280398 s, 37.4 MB/s 2024-04-25T10:12:54.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251282 s, 41.7 MB/s 2024-04-25T10:12:54.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194509 s, 53.9 MB/s 2024-04-25T10:12:54.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209405 s, 50.1 MB/s 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.098 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233926 s, 44.8 MB/s 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.130 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272529 s, 38.5 MB/s 2024-04-25T10:12:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200573 s, 52.3 MB/s 2024-04-25T10:12:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222557 s, 47.1 MB/s 2024-04-25T10:12:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194421 s, 53.9 MB/s 2024-04-25T10:12:54.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.227 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211169 s, 49.7 MB/s 2024-04-25T10:12:54.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023301 s, 45.0 MB/s 2024-04-25T10:12:54.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.282 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022902 s, 45.8 MB/s 2024-04-25T10:12:54.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264457 s, 39.7 MB/s 2024-04-25T10:12:54.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207774 s, 50.5 MB/s 2024-04-25T10:12:54.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.364 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229933 s, 45.6 MB/s 2024-04-25T10:12:54.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.392 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231551 s, 45.3 MB/s 2024-04-25T10:12:54.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.418 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218924 s, 47.9 MB/s 2024-04-25T10:12:54.443 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.443 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.443 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209863 s, 50.0 MB/s 2024-04-25T10:12:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230966 s, 45.4 MB/s 2024-04-25T10:12:54.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240822 s, 43.5 MB/s 2024-04-25T10:12:54.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.520 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175658 s, 59.7 MB/s 2024-04-25T10:12:54.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.542 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187997 s, 55.8 MB/s 2024-04-25T10:12:54.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198829 s, 52.7 MB/s 2024-04-25T10:12:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214691 s, 48.8 MB/s 2024-04-25T10:12:54.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.616 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207657 s, 50.5 MB/s 2024-04-25T10:12:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182509 s, 57.5 MB/s 2024-04-25T10:12:54.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272618 s, 38.5 MB/s 2024-04-25T10:12:54.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.694 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213577 s, 49.1 MB/s 2024-04-25T10:12:54.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256687 s, 40.9 MB/s 2024-04-25T10:12:54.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.752 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252529 s, 41.5 MB/s 2024-04-25T10:12:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201913 s, 51.9 MB/s 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194983 s, 53.8 MB/s 2024-04-25T10:12:54.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:54.802 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:54.802 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:54.802 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:12:54.802 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:54.802 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:54.802 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:54.803 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:54.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.895 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288899 s, 36.3 MB/s 2024-04-25T10:12:54.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.922 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231899 s, 45.2 MB/s 2024-04-25T10:12:54.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.946 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209386 s, 50.1 MB/s 2024-04-25T10:12:54.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.971 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206719 s, 50.7 MB/s 2024-04-25T10:12:54.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:54.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:54.992 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180785 s, 58.0 MB/s 2024-04-25T10:12:55.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.018 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225096 s, 46.6 MB/s 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201197 s, 52.1 MB/s 2024-04-25T10:12:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018254 s, 57.4 MB/s 2024-04-25T10:12:55.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.091 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228464 s, 45.9 MB/s 2024-04-25T10:12:55.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.113 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185139 s, 56.6 MB/s 2024-04-25T10:12:55.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.145 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280941 s, 37.3 MB/s 2024-04-25T10:12:55.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264371 s, 39.7 MB/s 2024-04-25T10:12:55.203 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.203 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.203 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233073 s, 45.0 MB/s 2024-04-25T10:12:55.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.227 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199015 s, 52.7 MB/s 2024-04-25T10:12:55.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.250 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198669 s, 52.8 MB/s 2024-04-25T10:12:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.281 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264807 s, 39.6 MB/s 2024-04-25T10:12:55.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211017 s, 49.7 MB/s 2024-04-25T10:12:55.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207428 s, 50.6 MB/s 2024-04-25T10:12:55.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207531 s, 50.5 MB/s 2024-04-25T10:12:55.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261262 s, 40.1 MB/s 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.411 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222148 s, 47.2 MB/s 2024-04-25T10:12:55.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220524 s, 47.5 MB/s 2024-04-25T10:12:55.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.457 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166547 s, 63.0 MB/s 2024-04-25T10:12:55.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.485 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238431 s, 44.0 MB/s 2024-04-25T10:12:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225483 s, 46.5 MB/s 2024-04-25T10:12:55.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.537 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218136 s, 48.1 MB/s 2024-04-25T10:12:55.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.561 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204461 s, 51.3 MB/s 2024-04-25T10:12:55.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.586 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200407 s, 52.3 MB/s 2024-04-25T10:12:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.615 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242299 s, 43.3 MB/s 2024-04-25T10:12:55.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196457 s, 53.4 MB/s 2024-04-25T10:12:55.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.664 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205977 s, 50.9 MB/s 2024-04-25T10:12:55.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.687 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019227 s, 54.5 MB/s 2024-04-25T10:12:55.689 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:55.690 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:55.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225562 s, 46.5 MB/s 2024-04-25T10:12:55.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.801 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238929 s, 43.9 MB/s 2024-04-25T10:12:55.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.828 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232495 s, 45.1 MB/s 2024-04-25T10:12:55.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204843 s, 51.2 MB/s 2024-04-25T10:12:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.882 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260972 s, 40.2 MB/s 2024-04-25T10:12:55.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.905 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191538 s, 54.7 MB/s 2024-04-25T10:12:55.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182283 s, 57.5 MB/s 2024-04-25T10:12:55.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.950 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182084 s, 57.6 MB/s 2024-04-25T10:12:55.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:55.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:55.975 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213608 s, 49.1 MB/s 2024-04-25T10:12:56.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.004 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237463 s, 44.2 MB/s 2024-04-25T10:12:56.029 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.030 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212716 s, 49.3 MB/s 2024-04-25T10:12:56.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.057 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233373 s, 44.9 MB/s 2024-04-25T10:12:56.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.085 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236261 s, 44.4 MB/s 2024-04-25T10:12:56.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206833 s, 50.7 MB/s 2024-04-25T10:12:56.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238667 s, 43.9 MB/s 2024-04-25T10:12:56.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228768 s, 45.8 MB/s 2024-04-25T10:12:56.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258666 s, 40.5 MB/s 2024-04-25T10:12:56.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.228 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0295061 s, 35.5 MB/s 2024-04-25T10:12:56.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220156 s, 47.6 MB/s 2024-04-25T10:12:56.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222784 s, 47.1 MB/s 2024-04-25T10:12:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265709 s, 39.5 MB/s 2024-04-25T10:12:56.335 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.335 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.335 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213066 s, 49.2 MB/s 2024-04-25T10:12:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208674 s, 50.2 MB/s 2024-04-25T10:12:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197242 s, 53.2 MB/s 2024-04-25T10:12:56.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.407 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194686 s, 53.9 MB/s 2024-04-25T10:12:56.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174125 s, 60.2 MB/s 2024-04-25T10:12:56.453 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.453 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.453 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216205 s, 48.5 MB/s 2024-04-25T10:12:56.476 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.476 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.476 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194085 s, 54.0 MB/s 2024-04-25T10:12:56.502 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.502 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.502 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211351 s, 49.6 MB/s 2024-04-25T10:12:56.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.526 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205104 s, 51.1 MB/s 2024-04-25T10:12:56.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.556 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256667 s, 40.9 MB/s 2024-04-25T10:12:56.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208746 s, 50.2 MB/s 2024-04-25T10:12:56.585 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:56.585 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:56.586 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:56.586 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:12:56.586 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:56.586 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:56.586 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:56.586 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:56.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.676 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026001 s, 40.3 MB/s 2024-04-25T10:12:56.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.701 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210527 s, 49.8 MB/s 2024-04-25T10:12:56.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216412 s, 48.5 MB/s 2024-04-25T10:12:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212626 s, 49.3 MB/s 2024-04-25T10:12:56.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.775 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205323 s, 51.1 MB/s 2024-04-25T10:12:56.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.803 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238272 s, 44.0 MB/s 2024-04-25T10:12:56.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.834 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274452 s, 38.2 MB/s 2024-04-25T10:12:56.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233003 s, 45.0 MB/s 2024-04-25T10:12:56.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221036 s, 47.4 MB/s 2024-04-25T10:12:56.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.911 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204555 s, 51.3 MB/s 2024-04-25T10:12:56.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194202 s, 54.0 MB/s 2024-04-25T10:12:56.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229778 s, 45.6 MB/s 2024-04-25T10:12:56.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:56.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:56.992 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257537 s, 40.7 MB/s 2024-04-25T10:12:57.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.024 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278591 s, 37.6 MB/s 2024-04-25T10:12:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241206 s, 43.5 MB/s 2024-04-25T10:12:57.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.072 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169516 s, 61.9 MB/s 2024-04-25T10:12:57.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.098 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207131 s, 50.6 MB/s 2024-04-25T10:12:57.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228532 s, 45.9 MB/s 2024-04-25T10:12:57.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.151 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222825 s, 47.1 MB/s 2024-04-25T10:12:57.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213919 s, 49.0 MB/s 2024-04-25T10:12:57.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.199 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197396 s, 53.1 MB/s 2024-04-25T10:12:57.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.228 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246152 s, 42.6 MB/s 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.256 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236547 s, 44.3 MB/s 2024-04-25T10:12:57.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.285 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251121 s, 41.8 MB/s 2024-04-25T10:12:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199703 s, 52.5 MB/s 2024-04-25T10:12:57.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.334 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206937 s, 50.7 MB/s 2024-04-25T10:12:57.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220929 s, 47.5 MB/s 2024-04-25T10:12:57.384 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.384 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.384 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207927 s, 50.4 MB/s 2024-04-25T10:12:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226958 s, 46.2 MB/s 2024-04-25T10:12:57.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209146 s, 50.1 MB/s 2024-04-25T10:12:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.460 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206076 s, 50.9 MB/s 2024-04-25T10:12:57.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.487 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229001 s, 45.8 MB/s 2024-04-25T10:12:57.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:57.489 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:57.490 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:57.490 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:12:57.490 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:57.490 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:57.490 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:57.490 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:57.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220048 s, 47.7 MB/s 2024-04-25T10:12:57.600 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.601 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206408 s, 50.8 MB/s 2024-04-25T10:12:57.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211553 s, 49.6 MB/s 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202314 s, 51.8 MB/s 2024-04-25T10:12:57.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.676 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210843 s, 49.7 MB/s 2024-04-25T10:12:57.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.701 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208297 s, 50.3 MB/s 2024-04-25T10:12:57.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.727 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218022 s, 48.1 MB/s 2024-04-25T10:12:57.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.752 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021455 s, 48.9 MB/s 2024-04-25T10:12:57.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201384 s, 52.1 MB/s 2024-04-25T10:12:57.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.803 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227517 s, 46.1 MB/s 2024-04-25T10:12:57.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200442 s, 52.3 MB/s 2024-04-25T10:12:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229882 s, 45.6 MB/s 2024-04-25T10:12:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175277 s, 59.8 MB/s 2024-04-25T10:12:57.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177106 s, 59.2 MB/s 2024-04-25T10:12:57.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.918 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184917 s, 56.7 MB/s 2024-04-25T10:12:57.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.941 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196769 s, 53.3 MB/s 2024-04-25T10:12:57.963 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.964 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.964 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188655 s, 55.6 MB/s 2024-04-25T10:12:57.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:57.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:57.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219141 s, 47.8 MB/s 2024-04-25T10:12:58.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216728 s, 48.4 MB/s 2024-04-25T10:12:58.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.039 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205799 s, 51.0 MB/s 2024-04-25T10:12:58.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.065 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230746 s, 45.4 MB/s 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.091 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220604 s, 47.5 MB/s 2024-04-25T10:12:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222535 s, 47.1 MB/s 2024-04-25T10:12:58.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.147 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025896 s, 40.5 MB/s 2024-04-25T10:12:58.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.171 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200299 s, 52.4 MB/s 2024-04-25T10:12:58.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.199 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234169 s, 44.8 MB/s 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.224 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215421 s, 48.7 MB/s 2024-04-25T10:12:58.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203301 s, 51.6 MB/s 2024-04-25T10:12:58.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.274 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221709 s, 47.3 MB/s 2024-04-25T10:12:58.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212792 s, 49.3 MB/s 2024-04-25T10:12:58.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.325 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209437 s, 50.1 MB/s 2024-04-25T10:12:58.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260374 s, 40.3 MB/s 2024-04-25T10:12:58.356 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:58.357 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:58.442 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.442 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.442 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221806 s, 47.3 MB/s 2024-04-25T10:12:58.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208277 s, 50.3 MB/s 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.496 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252486 s, 41.5 MB/s 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214851 s, 48.8 MB/s 2024-04-25T10:12:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253468 s, 41.4 MB/s 2024-04-25T10:12:58.578 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.578 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.578 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212544 s, 49.3 MB/s 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.606 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239265 s, 43.8 MB/s 2024-04-25T10:12:58.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195635 s, 53.6 MB/s 2024-04-25T10:12:58.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.654 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019527 s, 53.7 MB/s 2024-04-25T10:12:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214044 s, 49.0 MB/s 2024-04-25T10:12:58.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170896 s, 61.4 MB/s 2024-04-25T10:12:58.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201548 s, 52.0 MB/s 2024-04-25T10:12:58.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211682 s, 49.5 MB/s 2024-04-25T10:12:58.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025265 s, 41.5 MB/s 2024-04-25T10:12:58.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196909 s, 53.3 MB/s 2024-04-25T10:12:58.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.829 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.829 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247073 s, 42.4 MB/s 2024-04-25T10:12:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023017 s, 45.6 MB/s 2024-04-25T10:12:58.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017948 s, 58.4 MB/s 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.900 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182813 s, 57.4 MB/s 2024-04-25T10:12:58.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.923 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190194 s, 55.1 MB/s 2024-04-25T10:12:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.950 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238178 s, 44.0 MB/s 2024-04-25T10:12:58.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:58.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:58.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025584 s, 41.0 MB/s 2024-04-25T10:12:59.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227623 s, 46.1 MB/s 2024-04-25T10:12:59.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.034 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.034 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220361 s, 47.6 MB/s 2024-04-25T10:12:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238352 s, 44.0 MB/s 2024-04-25T10:12:59.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.085 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192595 s, 54.4 MB/s 2024-04-25T10:12:59.111 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.111 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.111 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216873 s, 48.3 MB/s 2024-04-25T10:12:59.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234974 s, 44.6 MB/s 2024-04-25T10:12:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.161 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184891 s, 56.7 MB/s 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.184 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189911 s, 55.2 MB/s 2024-04-25T10:12:59.207 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.207 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.208 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199391 s, 52.6 MB/s 2024-04-25T10:12:59.236 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.236 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.236 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252524 s, 41.5 MB/s 2024-04-25T10:12:59.239 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:12:59.239 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:12:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205004 s, 51.1 MB/s 2024-04-25T10:12:59.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.346 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193449 s, 54.2 MB/s 2024-04-25T10:12:59.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.369 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208913 s, 50.2 MB/s 2024-04-25T10:12:59.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028336 s, 37.0 MB/s 2024-04-25T10:12:59.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:59.426 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:12:59.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211099 s, 49.7 MB/s 2024-04-25T10:12:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.451 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211698 s, 49.5 MB/s 2024-04-25T10:12:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208705 s, 50.2 MB/s 2024-04-25T10:12:59.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.495 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175992 s, 59.6 MB/s 2024-04-25T10:12:59.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169088 s, 62.0 MB/s 2024-04-25T10:12:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021139 s, 49.6 MB/s 2024-04-25T10:12:59.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.568 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254198 s, 41.3 MB/s 2024-04-25T10:12:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237428 s, 44.2 MB/s 2024-04-25T10:12:59.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022823 s, 45.9 MB/s 2024-04-25T10:12:59.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259368 s, 40.4 MB/s 2024-04-25T10:12:59.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.683 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025835 s, 40.6 MB/s 2024-04-25T10:12:59.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.708 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217544 s, 48.2 MB/s 2024-04-25T10:12:59.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203651 s, 51.5 MB/s 2024-04-25T10:12:59.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212125 s, 49.4 MB/s 2024-04-25T10:12:59.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.787 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250857 s, 41.8 MB/s 2024-04-25T10:12:59.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.811 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200984 s, 52.2 MB/s 2024-04-25T10:12:59.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254377 s, 41.2 MB/s 2024-04-25T10:12:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235887 s, 44.5 MB/s 2024-04-25T10:12:59.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177259 s, 59.2 MB/s 2024-04-25T10:12:59.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.914 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214076 s, 49.0 MB/s 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.940 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221442 s, 47.4 MB/s 2024-04-25T10:12:59.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.965 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214237 s, 48.9 MB/s 2024-04-25T10:12:59.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:12:59.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:12:59.991 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223316 s, 47.0 MB/s 2024-04-25T10:13:00.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180872 s, 58.0 MB/s 2024-04-25T10:13:00.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.040 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022489 s, 46.6 MB/s 2024-04-25T10:13:00.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213759 s, 49.1 MB/s 2024-04-25T10:13:00.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.095 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235754 s, 44.5 MB/s 2024-04-25T10:13:00.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021476 s, 48.8 MB/s 2024-04-25T10:13:00.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:00.123 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:00.123 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:00.123 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:13:00.123 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:00.123 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:00.123 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:00.124 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.259 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272512 s, 38.5 MB/s 2024-04-25T10:13:00.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.285 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221302 s, 47.4 MB/s 2024-04-25T10:13:00.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256016 s, 41.0 MB/s 2024-04-25T10:13:00.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.339 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205046 s, 51.1 MB/s 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.367 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236232 s, 44.4 MB/s 2024-04-25T10:13:00.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.392 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213825 s, 49.0 MB/s 2024-04-25T10:13:00.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.418 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224369 s, 46.7 MB/s 2024-04-25T10:13:00.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025281 s, 41.5 MB/s 2024-04-25T10:13:00.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205917 s, 50.9 MB/s 2024-04-25T10:13:00.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.500 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243511 s, 43.1 MB/s 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229885 s, 45.6 MB/s 2024-04-25T10:13:00.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210735 s, 49.8 MB/s 2024-04-25T10:13:00.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206043 s, 50.9 MB/s 2024-04-25T10:13:00.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.599 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193113 s, 54.3 MB/s 2024-04-25T10:13:00.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200673 s, 52.3 MB/s 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018782 s, 55.8 MB/s 2024-04-25T10:13:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197902 s, 53.0 MB/s 2024-04-25T10:13:00.691 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.692 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185472 s, 56.5 MB/s 2024-04-25T10:13:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208766 s, 50.2 MB/s 2024-04-25T10:13:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212749 s, 49.3 MB/s 2024-04-25T10:13:00.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.768 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217765 s, 48.2 MB/s 2024-04-25T10:13:00.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.795 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227095 s, 46.2 MB/s 2024-04-25T10:13:00.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198017 s, 53.0 MB/s 2024-04-25T10:13:00.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218079 s, 48.1 MB/s 2024-04-25T10:13:00.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202225 s, 51.9 MB/s 2024-04-25T10:13:00.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.893 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213374 s, 49.1 MB/s 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223999 s, 46.8 MB/s 2024-04-25T10:13:00.948 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.948 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.948 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247498 s, 42.4 MB/s 2024-04-25T10:13:00.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:00.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:00.977 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249512 s, 42.0 MB/s 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026448 s, 39.6 MB/s 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.032 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206569 s, 50.8 MB/s 2024-04-25T10:13:01.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238698 s, 43.9 MB/s 2024-04-25T10:13:01.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:01.062 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:01.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233448 s, 44.9 MB/s 2024-04-25T10:13:01.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232414 s, 45.1 MB/s 2024-04-25T10:13:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243333 s, 43.1 MB/s 2024-04-25T10:13:01.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.234 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253864 s, 41.3 MB/s 2024-04-25T10:13:01.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.259 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206616 s, 50.7 MB/s 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203419 s, 51.5 MB/s 2024-04-25T10:13:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212427 s, 49.4 MB/s 2024-04-25T10:13:01.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221802 s, 47.3 MB/s 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208325 s, 50.3 MB/s 2024-04-25T10:13:01.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.388 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250792 s, 41.8 MB/s 2024-04-25T10:13:01.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201068 s, 52.2 MB/s 2024-04-25T10:13:01.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.433 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175621 s, 59.7 MB/s 2024-04-25T10:13:01.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.463 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260597 s, 40.2 MB/s 2024-04-25T10:13:01.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.489 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218372 s, 48.0 MB/s 2024-04-25T10:13:01.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.516 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245092 s, 42.8 MB/s 2024-04-25T10:13:01.537 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.537 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.537 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017643 s, 59.4 MB/s 2024-04-25T10:13:01.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.559 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181601 s, 57.7 MB/s 2024-04-25T10:13:01.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.587 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245856 s, 42.7 MB/s 2024-04-25T10:13:01.613 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.613 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.613 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214231 s, 48.9 MB/s 2024-04-25T10:13:01.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308702 s, 34.0 MB/s 2024-04-25T10:13:01.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.673 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216768 s, 48.4 MB/s 2024-04-25T10:13:01.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.698 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216077 s, 48.5 MB/s 2024-04-25T10:13:01.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200186 s, 52.4 MB/s 2024-04-25T10:13:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233601 s, 44.9 MB/s 2024-04-25T10:13:01.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195126 s, 53.7 MB/s 2024-04-25T10:13:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222891 s, 47.0 MB/s 2024-04-25T10:13:01.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202797 s, 51.7 MB/s 2024-04-25T10:13:01.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241529 s, 43.4 MB/s 2024-04-25T10:13:01.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206353 s, 50.8 MB/s 2024-04-25T10:13:01.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211381 s, 49.6 MB/s 2024-04-25T10:13:01.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.930 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244164 s, 42.9 MB/s 2024-04-25T10:13:01.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:01.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:01.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224759 s, 46.7 MB/s 2024-04-25T10:13:01.958 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:01.958 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:01.959 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:01.959 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:13:01.959 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:01.959 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:01.959 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:01.959 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:02.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.052 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270871 s, 38.7 MB/s 2024-04-25T10:13:02.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219873 s, 47.7 MB/s 2024-04-25T10:13:02.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207565 s, 50.5 MB/s 2024-04-25T10:13:02.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212253 s, 49.4 MB/s 2024-04-25T10:13:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.145 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168776 s, 62.1 MB/s 2024-04-25T10:13:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172869 s, 60.7 MB/s 2024-04-25T10:13:02.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.197 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.197 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276267 s, 38.0 MB/s 2024-04-25T10:13:02.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187616 s, 55.9 MB/s 2024-04-25T10:13:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.247 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227706 s, 46.0 MB/s 2024-04-25T10:13:02.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211192 s, 49.7 MB/s 2024-04-25T10:13:02.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216493 s, 48.4 MB/s 2024-04-25T10:13:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.327 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238454 s, 44.0 MB/s 2024-04-25T10:13:02.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212372 s, 49.4 MB/s 2024-04-25T10:13:02.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257328 s, 40.7 MB/s 2024-04-25T10:13:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022646 s, 46.3 MB/s 2024-04-25T10:13:02.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022275 s, 47.1 MB/s 2024-04-25T10:13:02.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193714 s, 54.1 MB/s 2024-04-25T10:13:02.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.487 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024004 s, 43.7 MB/s 2024-04-25T10:13:02.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.512 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211892 s, 49.5 MB/s 2024-04-25T10:13:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230116 s, 45.6 MB/s 2024-04-25T10:13:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230049 s, 45.6 MB/s 2024-04-25T10:13:02.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.593 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235946 s, 44.4 MB/s 2024-04-25T10:13:02.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.619 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208578 s, 50.3 MB/s 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.643 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202922 s, 51.7 MB/s 2024-04-25T10:13:02.668 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205477 s, 51.0 MB/s 2024-04-25T10:13:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264788 s, 39.6 MB/s 2024-04-25T10:13:02.724 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.724 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.724 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020211 s, 51.9 MB/s 2024-04-25T10:13:02.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206836 s, 50.7 MB/s 2024-04-25T10:13:02.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214126 s, 49.0 MB/s 2024-04-25T10:13:02.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217638 s, 48.2 MB/s 2024-04-25T10:13:02.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.824 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019596 s, 53.5 MB/s 2024-04-25T10:13:02.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.847 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01966 s, 53.3 MB/s 2024-04-25T10:13:02.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:02.851 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:02.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.936 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.936 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225477 s, 46.5 MB/s 2024-04-25T10:13:02.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.959 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189482 s, 55.3 MB/s 2024-04-25T10:13:02.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:02.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:02.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239038 s, 43.9 MB/s 2024-04-25T10:13:03.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225 s, 46.6 MB/s 2024-04-25T10:13:03.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.039 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022586 s, 46.4 MB/s 2024-04-25T10:13:03.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.065 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220677 s, 47.5 MB/s 2024-04-25T10:13:03.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.091 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221209 s, 47.4 MB/s 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020388 s, 51.4 MB/s 2024-04-25T10:13:03.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.142 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021988 s, 47.7 MB/s 2024-04-25T10:13:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.166 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019997 s, 52.4 MB/s 2024-04-25T10:13:03.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.192 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220258 s, 47.6 MB/s 2024-04-25T10:13:03.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.219 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232133 s, 45.2 MB/s 2024-04-25T10:13:03.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200415 s, 52.3 MB/s 2024-04-25T10:13:03.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.268 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189411 s, 55.4 MB/s 2024-04-25T10:13:03.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.294 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229541 s, 45.7 MB/s 2024-04-25T10:13:03.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.322 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235491 s, 44.5 MB/s 2024-04-25T10:13:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023505 s, 44.6 MB/s 2024-04-25T10:13:03.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.377 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232704 s, 45.1 MB/s 2024-04-25T10:13:03.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.405 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244537 s, 42.9 MB/s 2024-04-25T10:13:03.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026049 s, 40.3 MB/s 2024-04-25T10:13:03.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.460 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211028 s, 49.7 MB/s 2024-04-25T10:13:03.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201821 s, 52.0 MB/s 2024-04-25T10:13:03.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.510 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219431 s, 47.8 MB/s 2024-04-25T10:13:03.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.533 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196478 s, 53.4 MB/s 2024-04-25T10:13:03.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.558 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208131 s, 50.4 MB/s 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.585 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230953 s, 45.4 MB/s 2024-04-25T10:13:03.613 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.613 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.613 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023594 s, 44.4 MB/s 2024-04-25T10:13:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.637 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.637 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196405 s, 53.4 MB/s 2024-04-25T10:13:03.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.662 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.662 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204113 s, 51.4 MB/s 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.683 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173739 s, 60.4 MB/s 2024-04-25T10:13:03.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186816 s, 56.1 MB/s 2024-04-25T10:13:03.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226885 s, 46.2 MB/s 2024-04-25T10:13:03.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:03.735 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:03.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256836 s, 40.8 MB/s 2024-04-25T10:13:03.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.847 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215643 s, 48.6 MB/s 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204512 s, 51.3 MB/s 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.895 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200098 s, 52.4 MB/s 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202449 s, 51.8 MB/s 2024-04-25T10:13:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223395 s, 46.9 MB/s 2024-04-25T10:13:03.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:03.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:03.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020878 s, 50.2 MB/s 2024-04-25T10:13:04.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.000 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026282 s, 39.9 MB/s 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.026 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223796 s, 46.9 MB/s 2024-04-25T10:13:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234529 s, 44.7 MB/s 2024-04-25T10:13:04.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229654 s, 45.7 MB/s 2024-04-25T10:13:04.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.108 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241052 s, 43.5 MB/s 2024-04-25T10:13:04.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249217 s, 42.1 MB/s 2024-04-25T10:13:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225384 s, 46.5 MB/s 2024-04-25T10:13:04.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.192 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242687 s, 43.2 MB/s 2024-04-25T10:13:04.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229538 s, 45.7 MB/s 2024-04-25T10:13:04.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233272 s, 45.0 MB/s 2024-04-25T10:13:04.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252153 s, 41.6 MB/s 2024-04-25T10:13:04.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.303 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212183 s, 49.4 MB/s 2024-04-25T10:13:04.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230663 s, 45.5 MB/s 2024-04-25T10:13:04.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.358 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222902 s, 47.0 MB/s 2024-04-25T10:13:04.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201418 s, 52.1 MB/s 2024-04-25T10:13:04.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223713 s, 46.9 MB/s 2024-04-25T10:13:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.432 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195297 s, 53.7 MB/s 2024-04-25T10:13:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242936 s, 43.2 MB/s 2024-04-25T10:13:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201095 s, 52.1 MB/s 2024-04-25T10:13:04.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207783 s, 50.5 MB/s 2024-04-25T10:13:04.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.538 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234713 s, 44.7 MB/s 2024-04-25T10:13:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232799 s, 45.0 MB/s 2024-04-25T10:13:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.590 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206131 s, 50.9 MB/s 2024-04-25T10:13:04.611 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.612 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188527 s, 55.6 MB/s 2024-04-25T10:13:04.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.634 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191746 s, 54.7 MB/s 2024-04-25T10:13:04.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:04.637 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:04.637 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:04.637 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:13:04.637 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:04.637 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:04.637 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:04.638 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:04.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223747 s, 46.9 MB/s 2024-04-25T10:13:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231694 s, 45.3 MB/s 2024-04-25T10:13:04.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0283801 s, 36.9 MB/s 2024-04-25T10:13:04.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237536 s, 44.1 MB/s 2024-04-25T10:13:04.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195588 s, 53.6 MB/s 2024-04-25T10:13:04.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206819 s, 50.7 MB/s 2024-04-25T10:13:04.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.884 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223898 s, 46.8 MB/s 2024-04-25T10:13:04.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216834 s, 48.4 MB/s 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209978 s, 49.9 MB/s 2024-04-25T10:13:04.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227853 s, 46.0 MB/s 2024-04-25T10:13:04.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:04.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:04.988 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235223 s, 44.6 MB/s 2024-04-25T10:13:05.017 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.018 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248315 s, 42.2 MB/s 2024-04-25T10:13:05.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211503 s, 49.6 MB/s 2024-04-25T10:13:05.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.072 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239511 s, 43.8 MB/s 2024-04-25T10:13:05.097 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.097 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.097 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214517 s, 48.9 MB/s 2024-04-25T10:13:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211863 s, 49.5 MB/s 2024-04-25T10:13:05.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233986 s, 44.8 MB/s 2024-04-25T10:13:05.174 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.175 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213121 s, 49.2 MB/s 2024-04-25T10:13:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236801 s, 44.3 MB/s 2024-04-25T10:13:05.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.233 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258604 s, 40.5 MB/s 2024-04-25T10:13:05.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.259 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222004 s, 47.2 MB/s 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.281 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182757 s, 57.4 MB/s 2024-04-25T10:13:05.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230422 s, 45.5 MB/s 2024-04-25T10:13:05.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192115 s, 54.6 MB/s 2024-04-25T10:13:05.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.357 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02221 s, 47.2 MB/s 2024-04-25T10:13:05.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223272 s, 47.0 MB/s 2024-04-25T10:13:05.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.410 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023196 s, 45.2 MB/s 2024-04-25T10:13:05.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224758 s, 46.7 MB/s 2024-04-25T10:13:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.462 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211816 s, 49.5 MB/s 2024-04-25T10:13:05.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202417 s, 51.8 MB/s 2024-04-25T10:13:05.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.512 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215345 s, 48.7 MB/s 2024-04-25T10:13:05.535 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.535 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194907 s, 53.8 MB/s 2024-04-25T10:13:05.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:05.538 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:05.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262355 s, 40.0 MB/s 2024-04-25T10:13:05.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222947 s, 47.0 MB/s 2024-04-25T10:13:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213644 s, 49.1 MB/s 2024-04-25T10:13:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221586 s, 47.3 MB/s 2024-04-25T10:13:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209411 s, 50.1 MB/s 2024-04-25T10:13:05.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209439 s, 50.1 MB/s 2024-04-25T10:13:05.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261816 s, 40.1 MB/s 2024-04-25T10:13:05.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.814 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225107 s, 46.6 MB/s 2024-04-25T10:13:05.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.840 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216976 s, 48.3 MB/s 2024-04-25T10:13:05.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.865 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209203 s, 50.1 MB/s 2024-04-25T10:13:05.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167515 s, 62.6 MB/s 2024-04-25T10:13:05.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206929 s, 50.7 MB/s 2024-04-25T10:13:05.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208807 s, 50.2 MB/s 2024-04-25T10:13:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022375 s, 46.9 MB/s 2024-04-25T10:13:05.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:05.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:05.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221366 s, 47.4 MB/s 2024-04-25T10:13:06.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.010 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201917 s, 51.9 MB/s 2024-04-25T10:13:06.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.038 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023077 s, 45.4 MB/s 2024-04-25T10:13:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191172 s, 54.8 MB/s 2024-04-25T10:13:06.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.084 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193727 s, 54.1 MB/s 2024-04-25T10:13:06.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209314 s, 50.1 MB/s 2024-04-25T10:13:06.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022565 s, 46.5 MB/s 2024-04-25T10:13:06.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.160 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019542 s, 53.7 MB/s 2024-04-25T10:13:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.186 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212258 s, 49.4 MB/s 2024-04-25T10:13:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234924 s, 44.6 MB/s 2024-04-25T10:13:06.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.239 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218419 s, 48.0 MB/s 2024-04-25T10:13:06.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218117 s, 48.1 MB/s 2024-04-25T10:13:06.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.289 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194971 s, 53.8 MB/s 2024-04-25T10:13:06.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227149 s, 46.2 MB/s 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221229 s, 47.4 MB/s 2024-04-25T10:13:06.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.363 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184545 s, 56.8 MB/s 2024-04-25T10:13:06.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186392 s, 56.3 MB/s 2024-04-25T10:13:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221543 s, 47.3 MB/s 2024-04-25T10:13:06.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:06.415 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240626 s, 43.6 MB/s 2024-04-25T10:13:06.531 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.531 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.531 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220452 s, 47.6 MB/s 2024-04-25T10:13:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.558 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232115 s, 45.2 MB/s 2024-04-25T10:13:06.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.588 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261977 s, 40.0 MB/s 2024-04-25T10:13:06.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.614 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215638 s, 48.6 MB/s 2024-04-25T10:13:06.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.639 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02013 s, 52.1 MB/s 2024-04-25T10:13:06.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.663 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201505 s, 52.0 MB/s 2024-04-25T10:13:06.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.688 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213954 s, 49.0 MB/s 2024-04-25T10:13:06.712 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.712 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.713 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205735 s, 51.0 MB/s 2024-04-25T10:13:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.739 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223969 s, 46.8 MB/s 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.765 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218412 s, 48.0 MB/s 2024-04-25T10:13:06.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199203 s, 52.6 MB/s 2024-04-25T10:13:06.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.815 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232149 s, 45.2 MB/s 2024-04-25T10:13:06.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223116 s, 47.0 MB/s 2024-04-25T10:13:06.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.866 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206342 s, 50.8 MB/s 2024-04-25T10:13:06.892 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.892 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.892 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230559 s, 45.5 MB/s 2024-04-25T10:13:06.917 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.917 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.917 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211836 s, 49.5 MB/s 2024-04-25T10:13:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.942 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206686 s, 50.7 MB/s 2024-04-25T10:13:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:06.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:06.971 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255989 s, 41.0 MB/s 2024-04-25T10:13:07.003 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.004 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287161 s, 36.5 MB/s 2024-04-25T10:13:07.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.035 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270375 s, 38.8 MB/s 2024-04-25T10:13:07.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212159 s, 49.4 MB/s 2024-04-25T10:13:07.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.086 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208058 s, 50.4 MB/s 2024-04-25T10:13:07.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.107 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182336 s, 57.5 MB/s 2024-04-25T10:13:07.132 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.132 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.132 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211538 s, 49.6 MB/s 2024-04-25T10:13:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.155 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191863 s, 54.7 MB/s 2024-04-25T10:13:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.182 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229725 s, 45.6 MB/s 2024-04-25T10:13:07.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.206 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200192 s, 52.4 MB/s 2024-04-25T10:13:07.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.233 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234271 s, 44.8 MB/s 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195297 s, 53.7 MB/s 2024-04-25T10:13:07.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254848 s, 41.1 MB/s 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229821 s, 45.6 MB/s 2024-04-25T10:13:07.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:07.316 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:07.316 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:07.316 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:13:07.317 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:07.317 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:07.317 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:07.317 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:07.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.404 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238609 s, 43.9 MB/s 2024-04-25T10:13:07.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019454 s, 53.9 MB/s 2024-04-25T10:13:07.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.454 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237951 s, 44.1 MB/s 2024-04-25T10:13:07.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.479 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219203 s, 47.8 MB/s 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218058 s, 48.1 MB/s 2024-04-25T10:13:07.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206716 s, 50.7 MB/s 2024-04-25T10:13:07.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.557 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235232 s, 44.6 MB/s 2024-04-25T10:13:07.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210713 s, 49.8 MB/s 2024-04-25T10:13:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.610 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239596 s, 43.8 MB/s 2024-04-25T10:13:07.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.636 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218851 s, 47.9 MB/s 2024-04-25T10:13:07.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.665 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255642 s, 41.0 MB/s 2024-04-25T10:13:07.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.693 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229936 s, 45.6 MB/s 2024-04-25T10:13:07.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256943 s, 40.8 MB/s 2024-04-25T10:13:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199011 s, 52.7 MB/s 2024-04-25T10:13:07.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246472 s, 42.5 MB/s 2024-04-25T10:13:07.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.802 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240694 s, 43.6 MB/s 2024-04-25T10:13:07.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.828 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217961 s, 48.1 MB/s 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.854 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225927 s, 46.4 MB/s 2024-04-25T10:13:07.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184377 s, 56.9 MB/s 2024-04-25T10:13:07.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212472 s, 49.4 MB/s 2024-04-25T10:13:07.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.931 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024427 s, 42.9 MB/s 2024-04-25T10:13:07.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206683 s, 50.7 MB/s 2024-04-25T10:13:07.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:07.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:07.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020504 s, 51.1 MB/s 2024-04-25T10:13:08.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.006 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212258 s, 49.4 MB/s 2024-04-25T10:13:08.031 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.031 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.031 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217315 s, 48.3 MB/s 2024-04-25T10:13:08.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245995 s, 42.6 MB/s 2024-04-25T10:13:08.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.083 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203721 s, 51.5 MB/s 2024-04-25T10:13:08.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217089 s, 48.3 MB/s 2024-04-25T10:13:08.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252355 s, 41.6 MB/s 2024-04-25T10:13:08.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.167 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250098 s, 41.9 MB/s 2024-04-25T10:13:08.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.198 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270237 s, 38.8 MB/s 2024-04-25T10:13:08.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017703 s, 59.2 MB/s 2024-04-25T10:13:08.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:08.223 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:08.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025895 s, 40.5 MB/s 2024-04-25T10:13:08.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.337 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235946 s, 44.4 MB/s 2024-04-25T10:13:08.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.365 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245131 s, 42.8 MB/s 2024-04-25T10:13:08.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.393 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231598 s, 45.3 MB/s 2024-04-25T10:13:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219958 s, 47.7 MB/s 2024-04-25T10:13:08.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.449 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257472 s, 40.7 MB/s 2024-04-25T10:13:08.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176152 s, 59.5 MB/s 2024-04-25T10:13:08.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179956 s, 58.3 MB/s 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205838 s, 50.9 MB/s 2024-04-25T10:13:08.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.543 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219878 s, 47.7 MB/s 2024-04-25T10:13:08.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.569 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.569 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215137 s, 48.7 MB/s 2024-04-25T10:13:08.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215333 s, 48.7 MB/s 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226082 s, 46.4 MB/s 2024-04-25T10:13:08.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.645 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200784 s, 52.2 MB/s 2024-04-25T10:13:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225001 s, 46.6 MB/s 2024-04-25T10:13:08.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235225 s, 44.6 MB/s 2024-04-25T10:13:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233888 s, 44.8 MB/s 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191275 s, 54.8 MB/s 2024-04-25T10:13:08.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.777 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238615 s, 43.9 MB/s 2024-04-25T10:13:08.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219669 s, 47.7 MB/s 2024-04-25T10:13:08.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192193 s, 54.6 MB/s 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.851 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213709 s, 49.1 MB/s 2024-04-25T10:13:08.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.875 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194117 s, 54.0 MB/s 2024-04-25T10:13:08.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017865 s, 58.7 MB/s 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.921 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204251 s, 51.3 MB/s 2024-04-25T10:13:08.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.946 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223505 s, 46.9 MB/s 2024-04-25T10:13:08.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.972 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213931 s, 49.0 MB/s 2024-04-25T10:13:08.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:08.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:08.998 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218538 s, 48.0 MB/s 2024-04-25T10:13:09.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209639 s, 50.0 MB/s 2024-04-25T10:13:09.049 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.049 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.049 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218473 s, 48.0 MB/s 2024-04-25T10:13:09.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026104 s, 40.2 MB/s 2024-04-25T10:13:09.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193117 s, 54.3 MB/s 2024-04-25T10:13:09.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:09.106 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:09.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.192 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220589 s, 47.5 MB/s 2024-04-25T10:13:09.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.215 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198954 s, 52.7 MB/s 2024-04-25T10:13:09.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248712 s, 42.2 MB/s 2024-04-25T10:13:09.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249922 s, 42.0 MB/s 2024-04-25T10:13:09.298 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.298 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.298 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210475 s, 49.8 MB/s 2024-04-25T10:13:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.324 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220974 s, 47.5 MB/s 2024-04-25T10:13:09.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241265 s, 43.5 MB/s 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290616 s, 36.1 MB/s 2024-04-25T10:13:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206207 s, 50.9 MB/s 2024-04-25T10:13:09.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.440 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257017 s, 40.8 MB/s 2024-04-25T10:13:09.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.469 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242574 s, 43.2 MB/s 2024-04-25T10:13:09.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.491 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183778 s, 57.1 MB/s 2024-04-25T10:13:09.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221392 s, 47.4 MB/s 2024-04-25T10:13:09.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249134 s, 42.1 MB/s 2024-04-25T10:13:09.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.571 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215945 s, 48.6 MB/s 2024-04-25T10:13:09.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227319 s, 46.1 MB/s 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021308 s, 49.2 MB/s 2024-04-25T10:13:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020133 s, 52.1 MB/s 2024-04-25T10:13:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231408 s, 45.3 MB/s 2024-04-25T10:13:09.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208458 s, 50.3 MB/s 2024-04-25T10:13:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.725 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225028 s, 46.6 MB/s 2024-04-25T10:13:09.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216589 s, 48.4 MB/s 2024-04-25T10:13:09.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019634 s, 53.4 MB/s 2024-04-25T10:13:09.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218446 s, 48.0 MB/s 2024-04-25T10:13:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232929 s, 45.0 MB/s 2024-04-25T10:13:09.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270929 s, 38.7 MB/s 2024-04-25T10:13:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.881 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188258 s, 55.7 MB/s 2024-04-25T10:13:09.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.908 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232356 s, 45.1 MB/s 2024-04-25T10:13:09.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213473 s, 49.1 MB/s 2024-04-25T10:13:09.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219264 s, 47.8 MB/s 2024-04-25T10:13:09.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:09.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:09.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216715 s, 48.4 MB/s 2024-04-25T10:13:10.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244017 s, 43.0 MB/s 2024-04-25T10:13:10.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:10.018 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223787 s, 46.9 MB/s 2024-04-25T10:13:10.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025381 s, 41.3 MB/s 2024-04-25T10:13:10.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.160 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228148 s, 46.0 MB/s 2024-04-25T10:13:10.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270645 s, 38.7 MB/s 2024-04-25T10:13:10.216 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.216 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216604 s, 48.4 MB/s 2024-04-25T10:13:10.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247632 s, 42.3 MB/s 2024-04-25T10:13:10.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256634 s, 40.9 MB/s 2024-04-25T10:13:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.300 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215374 s, 48.7 MB/s 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.328 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240505 s, 43.6 MB/s 2024-04-25T10:13:10.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165686 s, 63.3 MB/s 2024-04-25T10:13:10.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.375 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235348 s, 44.6 MB/s 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.401 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220211 s, 47.6 MB/s 2024-04-25T10:13:10.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209998 s, 49.9 MB/s 2024-04-25T10:13:10.453 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.454 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216673 s, 48.4 MB/s 2024-04-25T10:13:10.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.483 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255591 s, 41.0 MB/s 2024-04-25T10:13:10.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.508 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211918 s, 49.5 MB/s 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.534 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222767 s, 47.1 MB/s 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198147 s, 52.9 MB/s 2024-04-25T10:13:10.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.586 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231588 s, 45.3 MB/s 2024-04-25T10:13:10.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187998 s, 55.8 MB/s 2024-04-25T10:13:10.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189044 s, 55.5 MB/s 2024-04-25T10:13:10.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.655 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197611 s, 53.1 MB/s 2024-04-25T10:13:10.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205263 s, 51.1 MB/s 2024-04-25T10:13:10.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219931 s, 47.7 MB/s 2024-04-25T10:13:10.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.734 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238707 s, 43.9 MB/s 2024-04-25T10:13:10.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.760 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216789 s, 48.4 MB/s 2024-04-25T10:13:10.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241974 s, 43.3 MB/s 2024-04-25T10:13:10.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.813 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.813 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209215 s, 50.1 MB/s 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201664 s, 52.0 MB/s 2024-04-25T10:13:10.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255302 s, 41.1 MB/s 2024-04-25T10:13:10.892 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.893 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220006 s, 47.7 MB/s 2024-04-25T10:13:10.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:10.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:10.919 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225957 s, 46.4 MB/s 2024-04-25T10:13:10.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:10.921 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:11.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232086 s, 45.2 MB/s 2024-04-25T10:13:11.029 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.029 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.029 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180027 s, 58.2 MB/s 2024-04-25T10:13:11.054 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.054 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.054 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191094 s, 54.9 MB/s 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.077 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205533 s, 51.0 MB/s 2024-04-25T10:13:11.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233144 s, 45.0 MB/s 2024-04-25T10:13:11.128 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.128 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.129 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213271 s, 49.2 MB/s 2024-04-25T10:13:11.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242891 s, 43.2 MB/s 2024-04-25T10:13:11.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.182 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214887 s, 48.8 MB/s 2024-04-25T10:13:11.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020435 s, 51.3 MB/s 2024-04-25T10:13:11.229 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.230 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204534 s, 51.3 MB/s 2024-04-25T10:13:11.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205861 s, 50.9 MB/s 2024-04-25T10:13:11.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281812 s, 37.2 MB/s 2024-04-25T10:13:11.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.311 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209772 s, 50.0 MB/s 2024-04-25T10:13:11.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259784 s, 40.4 MB/s 2024-04-25T10:13:11.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.369 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239104 s, 43.9 MB/s 2024-04-25T10:13:11.397 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.398 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248002 s, 42.3 MB/s 2024-04-25T10:13:11.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.425 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.425 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226993 s, 46.2 MB/s 2024-04-25T10:13:11.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018608 s, 56.4 MB/s 2024-04-25T10:13:11.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.474 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233059 s, 45.0 MB/s 2024-04-25T10:13:11.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204166 s, 51.4 MB/s 2024-04-25T10:13:11.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199007 s, 52.7 MB/s 2024-04-25T10:13:11.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203574 s, 51.5 MB/s 2024-04-25T10:13:11.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218701 s, 47.9 MB/s 2024-04-25T10:13:11.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223817 s, 46.8 MB/s 2024-04-25T10:13:11.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.624 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221367 s, 47.4 MB/s 2024-04-25T10:13:11.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218174 s, 48.1 MB/s 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.674 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193598 s, 54.2 MB/s 2024-04-25T10:13:11.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178014 s, 58.9 MB/s 2024-04-25T10:13:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.719 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195554 s, 53.6 MB/s 2024-04-25T10:13:11.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255967 s, 41.0 MB/s 2024-04-25T10:13:11.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217761 s, 48.2 MB/s 2024-04-25T10:13:11.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215468 s, 48.7 MB/s 2024-04-25T10:13:11.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:11.802 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:11.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.881 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194737 s, 53.8 MB/s 2024-04-25T10:13:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.906 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218167 s, 48.1 MB/s 2024-04-25T10:13:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185207 s, 56.6 MB/s 2024-04-25T10:13:11.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206643 s, 50.7 MB/s 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:11.976 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208984 s, 50.2 MB/s 2024-04-25T10:13:12.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.010 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291581 s, 36.0 MB/s 2024-04-25T10:13:12.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.040 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250499 s, 41.9 MB/s 2024-04-25T10:13:12.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288729 s, 36.3 MB/s 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.102 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237677 s, 44.1 MB/s 2024-04-25T10:13:12.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.127 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020613 s, 50.9 MB/s 2024-04-25T10:13:12.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221304 s, 47.4 MB/s 2024-04-25T10:13:12.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258632 s, 40.5 MB/s 2024-04-25T10:13:12.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.212 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247567 s, 42.4 MB/s 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.239 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233796 s, 44.8 MB/s 2024-04-25T10:13:12.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.266 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234442 s, 44.7 MB/s 2024-04-25T10:13:12.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.293 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233011 s, 45.0 MB/s 2024-04-25T10:13:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.319 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207704 s, 50.5 MB/s 2024-04-25T10:13:12.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.345 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230826 s, 45.4 MB/s 2024-04-25T10:13:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245111 s, 42.8 MB/s 2024-04-25T10:13:12.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.400 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232871 s, 45.0 MB/s 2024-04-25T10:13:12.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.420 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165847 s, 63.2 MB/s 2024-04-25T10:13:12.445 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.446 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219068 s, 47.9 MB/s 2024-04-25T10:13:12.470 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.470 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.470 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200594 s, 52.3 MB/s 2024-04-25T10:13:12.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228404 s, 45.9 MB/s 2024-04-25T10:13:12.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233888 s, 44.8 MB/s 2024-04-25T10:13:12.550 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.550 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022308 s, 47.0 MB/s 2024-04-25T10:13:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.574 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199834 s, 52.5 MB/s 2024-04-25T10:13:12.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.599 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213011 s, 49.2 MB/s 2024-04-25T10:13:12.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231833 s, 45.2 MB/s 2024-04-25T10:13:12.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202999 s, 51.7 MB/s 2024-04-25T10:13:12.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209935 s, 49.9 MB/s 2024-04-25T10:13:12.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.701 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202544 s, 51.8 MB/s 2024-04-25T10:13:12.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:12.703 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:12.703 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:12.703 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:13:12.703 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:12.703 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:12.703 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:12.704 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:12.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219672 s, 47.7 MB/s 2024-04-25T10:13:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213843 s, 49.0 MB/s 2024-04-25T10:13:12.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189824 s, 55.2 MB/s 2024-04-25T10:13:12.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215804 s, 48.6 MB/s 2024-04-25T10:13:12.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173502 s, 60.4 MB/s 2024-04-25T10:13:12.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.912 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021282 s, 49.3 MB/s 2024-04-25T10:13:12.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.938 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216412 s, 48.5 MB/s 2024-04-25T10:13:12.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:12.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:12.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0294162 s, 35.6 MB/s 2024-04-25T10:13:13.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.000 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261409 s, 40.1 MB/s 2024-04-25T10:13:13.025 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.025 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.025 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020941 s, 50.1 MB/s 2024-04-25T10:13:13.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.052 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218939 s, 47.9 MB/s 2024-04-25T10:13:13.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.077 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213276 s, 49.2 MB/s 2024-04-25T10:13:13.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203335 s, 51.6 MB/s 2024-04-25T10:13:13.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.130 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251806 s, 41.6 MB/s 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.159 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251758 s, 41.7 MB/s 2024-04-25T10:13:13.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.186 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232721 s, 45.1 MB/s 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.210 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202518 s, 51.8 MB/s 2024-04-25T10:13:13.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.238 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221196 s, 47.4 MB/s 2024-04-25T10:13:13.265 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.265 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.265 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233936 s, 44.8 MB/s 2024-04-25T10:13:13.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.290 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204204 s, 51.3 MB/s 2024-04-25T10:13:13.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.320 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026899 s, 39.0 MB/s 2024-04-25T10:13:13.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.350 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250347 s, 41.9 MB/s 2024-04-25T10:13:13.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.381 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256897 s, 40.8 MB/s 2024-04-25T10:13:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224296 s, 46.7 MB/s 2024-04-25T10:13:13.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.432 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195376 s, 53.7 MB/s 2024-04-25T10:13:13.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.457 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207355 s, 50.6 MB/s 2024-04-25T10:13:13.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.488 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273041 s, 38.4 MB/s 2024-04-25T10:13:13.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.514 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209726 s, 50.0 MB/s 2024-04-25T10:13:13.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.538 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019867 s, 52.8 MB/s 2024-04-25T10:13:13.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.560 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177675 s, 59.0 MB/s 2024-04-25T10:13:13.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.581 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181579 s, 57.7 MB/s 2024-04-25T10:13:13.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224184 s, 46.8 MB/s 2024-04-25T10:13:13.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:13.610 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:13.610 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:13.610 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:13:13.610 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:13.611 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:13.611 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:13.611 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:13.690 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.886 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169958 s, 61.7 MB/s 2024-04-25T10:13:13.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220931 s, 47.5 MB/s 2024-04-25T10:13:13.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211947 s, 49.5 MB/s 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183561 s, 57.1 MB/s 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251508 s, 41.7 MB/s 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244008 s, 43.0 MB/s 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222544 s, 47.1 MB/s 2024-04-25T10:13:13.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238123 s, 44.0 MB/s 2024-04-25T10:13:13.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02161 s, 48.5 MB/s 2024-04-25T10:13:13.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217014 s, 48.3 MB/s 2024-04-25T10:13:13.952 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.952 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.952 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212107 s, 49.4 MB/s 2024-04-25T10:13:13.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:13.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:13.979 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219646 s, 47.7 MB/s 2024-04-25T10:13:14.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.008 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024112 s, 43.5 MB/s 2024-04-25T10:13:14.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.039 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265084 s, 39.6 MB/s 2024-04-25T10:13:14.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.065 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222381 s, 47.2 MB/s 2024-04-25T10:13:14.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198623 s, 52.8 MB/s 2024-04-25T10:13:14.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.114 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225128 s, 46.6 MB/s 2024-04-25T10:13:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.139 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210796 s, 49.7 MB/s 2024-04-25T10:13:14.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.163 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020821 s, 50.4 MB/s 2024-04-25T10:13:14.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.190 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023565 s, 44.5 MB/s 2024-04-25T10:13:14.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226188 s, 46.4 MB/s 2024-04-25T10:13:14.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250992 s, 41.8 MB/s 2024-04-25T10:13:14.270 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.270 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.270 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198509 s, 52.8 MB/s 2024-04-25T10:13:14.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.296 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.296 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214893 s, 48.8 MB/s 2024-04-25T10:13:14.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.321 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210248 s, 49.9 MB/s 2024-04-25T10:13:14.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.347 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208112 s, 50.4 MB/s 2024-04-25T10:13:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206323 s, 50.8 MB/s 2024-04-25T10:13:14.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.401 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256885 s, 40.8 MB/s 2024-04-25T10:13:14.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.429 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235886 s, 44.5 MB/s 2024-04-25T10:13:14.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.455 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229763 s, 45.6 MB/s 2024-04-25T10:13:14.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.479 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206978 s, 50.7 MB/s 2024-04-25T10:13:14.506 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.506 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.506 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238069 s, 44.0 MB/s 2024-04-25T10:13:14.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:14.509 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256394 s, 40.9 MB/s 2024-04-25T10:13:14.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227511 s, 46.1 MB/s 2024-04-25T10:13:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201233 s, 52.1 MB/s 2024-04-25T10:13:14.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253039 s, 41.4 MB/s 2024-04-25T10:13:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230707 s, 45.5 MB/s 2024-04-25T10:13:14.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.734 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237293 s, 44.2 MB/s 2024-04-25T10:13:14.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.766 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274306 s, 38.2 MB/s 2024-04-25T10:13:14.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.787 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182919 s, 57.3 MB/s 2024-04-25T10:13:14.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.813 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.813 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207787 s, 50.5 MB/s 2024-04-25T10:13:14.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203367 s, 51.6 MB/s 2024-04-25T10:13:14.859 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.859 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.859 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194185 s, 54.0 MB/s 2024-04-25T10:13:14.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200452 s, 52.3 MB/s 2024-04-25T10:13:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208739 s, 50.2 MB/s 2024-04-25T10:13:14.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.933 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198067 s, 52.9 MB/s 2024-04-25T10:13:14.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194342 s, 54.0 MB/s 2024-04-25T10:13:14.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:14.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:14.979 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192807 s, 54.4 MB/s 2024-04-25T10:13:15.001 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.001 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.001 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018665 s, 56.2 MB/s 2024-04-25T10:13:15.027 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.027 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.027 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220157 s, 47.6 MB/s 2024-04-25T10:13:15.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.056 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253482 s, 41.4 MB/s 2024-04-25T10:13:15.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202087 s, 51.9 MB/s 2024-04-25T10:13:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216127 s, 48.5 MB/s 2024-04-25T10:13:15.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.131 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213788 s, 49.0 MB/s 2024-04-25T10:13:15.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.159 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229229 s, 45.7 MB/s 2024-04-25T10:13:15.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.185 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222279 s, 47.2 MB/s 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.210 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216552 s, 48.4 MB/s 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.242 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264528 s, 39.6 MB/s 2024-04-25T10:13:15.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027979 s, 37.5 MB/s 2024-04-25T10:13:15.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205153 s, 51.1 MB/s 2024-04-25T10:13:15.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.324 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021185 s, 49.5 MB/s 2024-04-25T10:13:15.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.349 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207566 s, 50.5 MB/s 2024-04-25T10:13:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226058 s, 46.4 MB/s 2024-04-25T10:13:15.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.396 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176468 s, 59.4 MB/s 2024-04-25T10:13:15.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:15.399 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:15.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.487 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213382 s, 49.1 MB/s 2024-04-25T10:13:15.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021048 s, 49.8 MB/s 2024-04-25T10:13:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255505 s, 41.0 MB/s 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.563 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188688 s, 55.6 MB/s 2024-04-25T10:13:15.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238185 s, 44.0 MB/s 2024-04-25T10:13:15.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.618 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024481 s, 42.8 MB/s 2024-04-25T10:13:15.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.645 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227189 s, 46.2 MB/s 2024-04-25T10:13:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.672 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021636 s, 48.5 MB/s 2024-04-25T10:13:15.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.696 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197809 s, 53.0 MB/s 2024-04-25T10:13:15.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263901 s, 39.7 MB/s 2024-04-25T10:13:15.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200251 s, 52.4 MB/s 2024-04-25T10:13:15.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223766 s, 46.9 MB/s 2024-04-25T10:13:15.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.801 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019768 s, 53.0 MB/s 2024-04-25T10:13:15.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192288 s, 54.5 MB/s 2024-04-25T10:13:15.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212055 s, 49.4 MB/s 2024-04-25T10:13:15.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239369 s, 43.8 MB/s 2024-04-25T10:13:15.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196696 s, 53.3 MB/s 2024-04-25T10:13:15.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.928 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216684 s, 48.4 MB/s 2024-04-25T10:13:15.954 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.954 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.954 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020991 s, 50.0 MB/s 2024-04-25T10:13:15.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:15.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:15.978 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203134 s, 51.6 MB/s 2024-04-25T10:13:16.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.001 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187249 s, 56.0 MB/s 2024-04-25T10:13:16.027 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.027 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.027 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023356 s, 44.9 MB/s 2024-04-25T10:13:16.054 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.054 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.054 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233127 s, 45.0 MB/s 2024-04-25T10:13:16.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021311 s, 49.2 MB/s 2024-04-25T10:13:16.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218246 s, 48.0 MB/s 2024-04-25T10:13:16.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.133 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242219 s, 43.3 MB/s 2024-04-25T10:13:16.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.162 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249238 s, 42.1 MB/s 2024-04-25T10:13:16.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.182 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173035 s, 60.6 MB/s 2024-04-25T10:13:16.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.206 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199354 s, 52.6 MB/s 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.235 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250323 s, 41.9 MB/s 2024-04-25T10:13:16.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.261 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223857 s, 46.8 MB/s 2024-04-25T10:13:16.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.293 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279604 s, 37.5 MB/s 2024-04-25T10:13:16.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:16.296 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:16.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224009 s, 46.8 MB/s 2024-04-25T10:13:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233579 s, 44.9 MB/s 2024-04-25T10:13:16.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.428 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165174 s, 63.5 MB/s 2024-04-25T10:13:16.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240909 s, 43.5 MB/s 2024-04-25T10:13:16.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228303 s, 45.9 MB/s 2024-04-25T10:13:16.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.508 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.508 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220215 s, 47.6 MB/s 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.533 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210829 s, 49.7 MB/s 2024-04-25T10:13:16.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.560 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228136 s, 46.0 MB/s 2024-04-25T10:13:16.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262798 s, 39.9 MB/s 2024-04-25T10:13:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.624 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027477 s, 38.2 MB/s 2024-04-25T10:13:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019489 s, 53.8 MB/s 2024-04-25T10:13:16.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021718 s, 48.3 MB/s 2024-04-25T10:13:16.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020134 s, 52.1 MB/s 2024-04-25T10:13:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.720 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212577 s, 49.3 MB/s 2024-04-25T10:13:16.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236509 s, 44.3 MB/s 2024-04-25T10:13:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205027 s, 51.1 MB/s 2024-04-25T10:13:16.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212814 s, 49.3 MB/s 2024-04-25T10:13:16.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201841 s, 52.0 MB/s 2024-04-25T10:13:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.846 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213665 s, 49.1 MB/s 2024-04-25T10:13:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212473 s, 49.4 MB/s 2024-04-25T10:13:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216392 s, 48.5 MB/s 2024-04-25T10:13:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237995 s, 44.1 MB/s 2024-04-25T10:13:16.952 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250464 s, 41.9 MB/s 2024-04-25T10:13:16.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:16.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:16.981 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246172 s, 42.6 MB/s 2024-04-25T10:13:17.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.009 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.009 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231822 s, 45.2 MB/s 2024-04-25T10:13:17.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022359 s, 46.9 MB/s 2024-04-25T10:13:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220015 s, 47.7 MB/s 2024-04-25T10:13:17.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.086 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212441 s, 49.4 MB/s 2024-04-25T10:13:17.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.108 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184848 s, 56.7 MB/s 2024-04-25T10:13:17.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.135 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224515 s, 46.7 MB/s 2024-04-25T10:13:17.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.160 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212247 s, 49.4 MB/s 2024-04-25T10:13:17.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.186 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217914 s, 48.1 MB/s 2024-04-25T10:13:17.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:17.189 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:17.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.274 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230292 s, 45.5 MB/s 2024-04-25T10:13:17.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.300 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021757 s, 48.2 MB/s 2024-04-25T10:13:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023157 s, 45.3 MB/s 2024-04-25T10:13:17.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.354 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231363 s, 45.3 MB/s 2024-04-25T10:13:17.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243315 s, 43.1 MB/s 2024-04-25T10:13:17.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214868 s, 48.8 MB/s 2024-04-25T10:13:17.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222756 s, 47.1 MB/s 2024-04-25T10:13:17.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271694 s, 38.6 MB/s 2024-04-25T10:13:17.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.491 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204406 s, 51.3 MB/s 2024-04-25T10:13:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021954 s, 47.8 MB/s 2024-04-25T10:13:17.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186994 s, 56.1 MB/s 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.563 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205713 s, 51.0 MB/s 2024-04-25T10:13:17.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.589 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220177 s, 47.6 MB/s 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.615 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207357 s, 50.6 MB/s 2024-04-25T10:13:17.641 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.641 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.641 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228045 s, 46.0 MB/s 2024-04-25T10:13:17.666 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.666 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.666 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205131 s, 51.1 MB/s 2024-04-25T10:13:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243682 s, 43.0 MB/s 2024-04-25T10:13:17.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.720 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022103 s, 47.4 MB/s 2024-04-25T10:13:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206855 s, 50.7 MB/s 2024-04-25T10:13:17.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.769 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207047 s, 50.6 MB/s 2024-04-25T10:13:17.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019772 s, 53.0 MB/s 2024-04-25T10:13:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238602 s, 43.9 MB/s 2024-04-25T10:13:17.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.851 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248609 s, 42.2 MB/s 2024-04-25T10:13:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197381 s, 53.1 MB/s 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206614 s, 50.8 MB/s 2024-04-25T10:13:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.923 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193362 s, 54.2 MB/s 2024-04-25T10:13:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.948 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208781 s, 50.2 MB/s 2024-04-25T10:13:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223347 s, 46.9 MB/s 2024-04-25T10:13:17.999 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:17.999 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:17.999 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019552 s, 53.6 MB/s 2024-04-25T10:13:18.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198375 s, 52.9 MB/s 2024-04-25T10:13:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203969 s, 51.4 MB/s 2024-04-25T10:13:18.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.076 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237354 s, 44.2 MB/s 2024-04-25T10:13:18.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:13:18.079 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:13:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213245 s, 49.2 MB/s 2024-04-25T10:13:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023286 s, 45.0 MB/s 2024-04-25T10:13:18.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.215 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222228 s, 47.2 MB/s 2024-04-25T10:13:18.242 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238376 s, 44.0 MB/s 2024-04-25T10:13:18.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.266 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208924 s, 50.2 MB/s 2024-04-25T10:13:18.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.293 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233572 s, 44.9 MB/s 2024-04-25T10:13:18.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.318 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210565 s, 49.8 MB/s 2024-04-25T10:13:18.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.343 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.343 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206802 s, 50.7 MB/s 2024-04-25T10:13:18.368 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.368 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.368 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021291 s, 49.2 MB/s 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.392 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204422 s, 51.3 MB/s 2024-04-25T10:13:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205053 s, 51.1 MB/s 2024-04-25T10:13:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209517 s, 50.0 MB/s 2024-04-25T10:13:18.466 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218246 s, 48.0 MB/s 2024-04-25T10:13:18.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219978 s, 47.7 MB/s 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219616 s, 47.7 MB/s 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209145 s, 50.1 MB/s 2024-04-25T10:13:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.573 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219666 s, 47.7 MB/s 2024-04-25T10:13:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184181 s, 56.9 MB/s 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.620 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214295 s, 48.9 MB/s 2024-04-25T10:13:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.647 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023081 s, 45.4 MB/s 2024-04-25T10:13:18.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197771 s, 53.0 MB/s 2024-04-25T10:13:18.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.696 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224935 s, 46.6 MB/s 2024-04-25T10:13:18.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233715 s, 44.9 MB/s 2024-04-25T10:13:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236815 s, 44.3 MB/s 2024-04-25T10:13:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215741 s, 48.6 MB/s 2024-04-25T10:13:18.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.805 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237067 s, 44.2 MB/s 2024-04-25T10:13:18.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231243 s, 45.3 MB/s 2024-04-25T10:13:18.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220818 s, 47.5 MB/s 2024-04-25T10:13:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237611 s, 44.1 MB/s 2024-04-25T10:13:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023624 s, 44.4 MB/s 2024-04-25T10:13:18.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271681 s, 38.6 MB/s 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:13:18.971 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223787 s, 46.9 MB/s 2024-04-25T10:13:18.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:18.974 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:13:19.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.172+0000 7f3a30e00640 1 Processor -- start 2024-04-25T10:13:19.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.172+0000 7f3a30e00640 1 -- start start 2024-04-25T10:13:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.173+0000 7f3a30e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a2c099cb0 0x7f3a2c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.173+0000 7f3a30e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.173+0000 7f3a30e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c0d0080 con 0x7f3a2c09a090 2024-04-25T10:13:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.173+0000 7f3a30e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c06c4b0 con 0x7f3a2c099cb0 2024-04-25T10:13:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.173+0000 7f3a30e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0cfe30 con 0x7f3a2c0d1120 2024-04-25T10:13:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.174+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.174+0000 7f3a2b400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a2c099cb0 0x7f3a2c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.174+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0cf8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:47168/0 (socket says 172.21.15.165:47168) 2024-04-25T10:13:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.174+0000 7f3a2aa00640 1 -- 172.21.15.165:0/1081510990 learned_addr learned my addr 172.21.15.165:0/1081510990 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.175+0000 7f3a2b400640 1 -- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 msgr2=0x7f3a2c0cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.175+0000 7f3a2b400640 1 --2- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.175+0000 7f3a2b400640 1 -- 172.21.15.165:0/1081510990 >> v1:172.21.15.172:6789/0 conn(0x7f3a2c09a090 legacy=0x7f3a2c15d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.175+0000 7f3a2b400640 1 -- 172.21.15.165:0/1081510990 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0d0360 con 0x7f3a2c099cb0 2024-04-25T10:13:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.176+0000 7f3a2b400640 1 --2- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c06bf70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0097a0 tx=0x7f3a1c02a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=534d5e5baba774fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.177+0000 7f3a2a000640 1 -- 172.21.15.165:0/1081510990 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c037070 con 0x7f3a2c099cb0 2024-04-25T10:13:19.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.177+0000 7f3a2a000640 1 -- 172.21.15.165:0/1081510990 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c032440 con 0x7f3a2c099cb0 2024-04-25T10:13:19.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.177+0000 7f3a2a000640 1 -- 172.21.15.165:0/1081510990 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c032810 con 0x7f3a2c099cb0 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.177+0000 7f3a30e00640 1 -- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 msgr2=0x7f3a2c06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.177+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c06bf70 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c0097a0 tx=0x7f3a1c02a210 comp rx=0 tx=0).stop 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.178+0000 7f3a30e00640 1 -- 172.21.15.165:0/1081510990 shutdown_connections 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.178+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.178+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1081510990 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c06bf70 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.178+0000 7f3a30e00640 1 -- 172.21.15.165:0/1081510990 >> 172.21.15.165:0/1081510990 conn(0x7f3a2c0b3250 msgr2=0x7f3a2c0b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.178+0000 7f3a30e00640 1 -- 172.21.15.165:0/1081510990 shutdown_connections 2024-04-25T10:13:19.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.178+0000 7f3a30e00640 1 -- 172.21.15.165:0/1081510990 wait complete. 2024-04-25T10:13:19.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.179+0000 7f3a30e00640 1 Processor -- start 2024-04-25T10:13:19.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 -- start start 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 0x7f3a2c0cafb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0d0c00 con 0x7f3a2c09a090 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0d0920 con 0x7f3a2c099cb0 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.180+0000 7f3a30e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c15de10 con 0x7f3a2c0d1120 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 0x7f3a2c0cafb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 0x7f3a2c0cafb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35810/0 (socket says 172.21.15.165:35810) 2024-04-25T10:13:19.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 -- 172.21.15.165:0/1647511031 learned_addr learned my addr 172.21.15.165:0/1647511031 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2b400640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2be00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 -- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 msgr2=0x7f3a2c0b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 -- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 msgr2=0x7f3a2c0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c0caa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.181+0000 7f3a2aa00640 1 -- 172.21.15.165:0/1647511031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c15c1c0 con 0x7f3a2c09a090 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.182+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 0x7f3a2c0cafb0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e840 tx=0x7f3a1800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.182+0000 7f3a23400640 1 -- 172.21.15.165:0/1647511031 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18017070 con 0x7f3a2c09a090 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.182+0000 7f3a23400640 1 -- 172.21.15.165:0/1647511031 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1800c8e0 con 0x7f3a2c09a090 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.182+0000 7f3a23400640 1 -- 172.21.15.165:0/1647511031 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800ccd0 con 0x7f3a2c09a090 2024-04-25T10:13:19.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.182+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c0b4480 con 0x7f3a2c09a090 2024-04-25T10:13:19.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.182+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c0b4a40 con 0x7f3a2c09a090 2024-04-25T10:13:19.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.185+0000 7f3a23400640 1 -- 172.21.15.165:0/1647511031 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a1801a070 con 0x7f3a2c09a090 2024-04-25T10:13:19.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.185+0000 7f3a23400640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a0c06b4d0 0x7f3a0c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.186+0000 7f3a23400640 1 -- 172.21.15.165:0/1647511031 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3a1808a600 con 0x7f3a2c09a090 2024-04-25T10:13:19.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.186+0000 7f3a2b400640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a0c06b4d0 0x7f3a0c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.186+0000 7f3a353d8740 1 Processor -- start 2024-04-25T10:13:19.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.186+0000 7f3a353d8740 1 -- start start 2024-04-25T10:13:19.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a2b400640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a0c06b4d0 0x7f3a0c06d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c034000 tx=0x7f3a1c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfb4360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cfd1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cf89c60 0x56331cfcfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a353d8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331cee7f00 con 0x56331cfb3f80 2024-04-25T10:13:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331ceecf50 con 0x56331cf89c60 2024-04-25T10:13:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56331ce960b0 con 0x56331cfd1820 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.187+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfb4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfb4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35816/0 (socket says 172.21.15.165:35816) 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 -- 172.21.15.165:0/136566348 learned_addr learned my addr 172.21.15.165:0/136566348 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cfd1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 -- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 msgr2=0x56331cfd1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cfd1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 -- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cf89c60 msgr2=0x56331cfcfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cf89c60 0x56331cfcfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 -- 172.21.15.165:0/136566348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331cccdba0 con 0x56331cfb3f80 2024-04-25T10:13:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.188+0000 7f3a2b400640 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfb4360 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009560 tx=0x7f3a1c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a457496c2bd65a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a17e00640 1 -- 172.21.15.165:0/136566348 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c04d070 con 0x56331cfb3f80 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a17e00640 1 -- 172.21.15.165:0/136566348 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c032d40 con 0x56331cfb3f80 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a17e00640 1 -- 172.21.15.165:0/136566348 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c037040 con 0x56331cfb3f80 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 -- 172.21.15.165:0/136566348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 msgr2=0x56331cfb4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfb4360 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c009560 tx=0x7f3a1c02c000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 -- 172.21.15.165:0/136566348 shutdown_connections 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cf89c60 0x56331cfcfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cfd1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 --2- 172.21.15.165:0/136566348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfb4360 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 -- 172.21.15.165:0/136566348 >> 172.21.15.165:0/136566348 conn(0x56331cf86820 msgr2=0x56331cfced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 -- 172.21.15.165:0/136566348 shutdown_connections 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.189+0000 7f3a353d8740 1 -- 172.21.15.165:0/136566348 wait complete. 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.190+0000 7f3a353d8740 1 -- start start 2024-04-25T10:13:19.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.190+0000 7f3a353d8740 1 Processor -- start 2024-04-25T10:13:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.191+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfb3f80 0x56331cfdeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.191+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 0x56331cfb7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 0x56331cfc6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a353d8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331ce8b620 con 0x56331cfd1820 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331cee8ff0 con 0x56331cfaf320 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56331cf8aa00 con 0x56331cfb3f80 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a353d8740 10 client.0 _ll_register_callbacks cb 0x56331cc00f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 0x56331cfb7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 0x56331cfb7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35830/0 (socket says 172.21.15.165:35830) 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3347218010 learned_addr learned my addr 172.21.15.165:0/3347218010 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfb3f80 msgr2=0x56331cfdeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfb3f80 0x56331cfdeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 msgr2=0x56331cfc6f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 0x56331cfc6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3347218010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56331cbc1490 con 0x56331cfd1820 2024-04-25T10:13:19.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 0x56331cfb7c20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3a180588b0 tx=0x7f3a1800cfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a93774a24acfe8b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.192+0000 7f3a14c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a14c00640 1 -- 172.21.15.165:0/3347218010 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18021040 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a14c00640 1 -- 172.21.15.165:0/3347218010 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3a1806cd70 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a1c032d40 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a14c00640 1 -- 172.21.15.165:0/3347218010 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a180612c0 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 10 client.8226 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x56331cd2e330 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a14c00640 1 -- 172.21.15.165:0/3347218010 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c032d40 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a14c00640 1 -- 172.21.15.165:0/3347218010 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3a18058000 con 0x56331cfd1820 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 10 client.8226 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 10 client.8226 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 20 client.8226 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cfc74a0 0x56331cf882b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 4 client.8226 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:19.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.193+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56331cefa490 con 0x56331cfc74a0 2024-04-25T10:13:19.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.194+0000 7f3a2b400640 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cfc74a0 0x56331cf882b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.195+0000 7f3a2b400640 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cfc74a0 0x56331cf882b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=322528ff82bd27e server_cookie=13efccad790e48c1 in_seq=0 out_seq=0 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.195+0000 7f3a14c00640 10 client.8226 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.196+0000 7f3a14c00640 1 -- 172.21.15.165:0/3347218010 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56331cefa490 con 0x56331cfc74a0 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.196+0000 7f3a14c00640 10 client.8226 handle_command_reply: tid=0 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 client.8226 shutdown 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 msgr2=0x56331cfb7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 0x56331cfb7c20 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f3a180588b0 tx=0x7f3a1800cfb0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 shutdown_connections 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cfc74a0 0x56331cf882b0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 0x56331cfc6f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfd1820 0x56331cfb7c20 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3347218010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfb3f80 0x56331cfdeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a14c00640 0 client.8226 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 >> 172.21.15.165:0/3347218010 conn(0x56331cf89c60 msgr2=0x56331cf88f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 shutdown_connections 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 1 -- 172.21.15.165:0/3347218010 wait complete. 2024-04-25T10:13:19.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.197+0000 7f3a353d8740 20 client.8226 trim_cache size 0 max 16384 2024-04-25T10:13:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.214+0000 7f3a353d8740 1 Processor -- start 2024-04-25T10:13:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.214+0000 7f3a353d8740 1 -- start start 2024-04-25T10:13:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 0x56331cfcf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cf8a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cf8a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a353d8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56331cfc8370 con 0x56331cfb3f80 2024-04-25T10:13:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c2004f0 con 0x56331cfaf320 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c1ff6e0 con 0x56331cfd1820 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cf8a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cf8a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35838/0 (socket says 172.21.15.165:35838) 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3417525679 learned_addr learned my addr 172.21.15.165:0/3417525679 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3417525679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 msgr2=0x56331cf8a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cf8a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3417525679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 msgr2=0x56331cfcf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 0x56331cfcf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.215+0000 7f3a2aa00640 1 -- 172.21.15.165:0/3417525679 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56331cfc8880 con 0x56331cfb3f80 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.216+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cf8a040 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c520 tx=0x7f3a1808a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56db56f15910660 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.216+0000 7f3a17400640 1 -- 172.21.15.165:0/3417525679 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18021070 con 0x56331cfb3f80 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.216+0000 7f3a17400640 1 -- 172.21.15.165:0/3417525679 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1806cb70 con 0x56331cfb3f80 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.216+0000 7f3a17400640 1 -- 172.21.15.165:0/3417525679 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a18054900 con 0x56331cfb3f80 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.216+0000 7f3a353d8740 1 -- 172.21.15.165:0/3417525679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 msgr2=0x56331cf8a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.216+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cf8a040 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800c520 tx=0x7f3a1808a000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 -- 172.21.15.165:0/3417525679 shutdown_connections 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfd1820 0x56331cf8a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cf8a040 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 --2- 172.21.15.165:0/3417525679 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfaf320 0x56331cfcf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 -- 172.21.15.165:0/3417525679 >> 172.21.15.165:0/3417525679 conn(0x56331cf89c60 msgr2=0x56331cfd27d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 -- 172.21.15.165:0/3417525679 shutdown_connections 2024-04-25T10:13:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 -- 172.21.15.165:0/3417525679 wait complete. 2024-04-25T10:13:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.217+0000 7f3a353d8740 1 -- start start 2024-04-25T10:13:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.218+0000 7f3a353d8740 1 Processor -- start 2024-04-25T10:13:19.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfaf320 0x56331d028170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfc5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfd1820 0x56331cf8a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c09f460 con 0x56331cfb3f80 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c09ee60 con 0x56331cfd1820 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56331cfcef80 con 0x56331cfaf320 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a353d8740 10 client.0 _ll_register_callbacks cb 0x56331cf84c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfc5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfc5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35852/0 (socket says 172.21.15.165:35852) 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 -- 172.21.15.165:0/2550515018 learned_addr learned my addr 172.21.15.165:0/2550515018 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2b400640 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfaf320 0x56331d028170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 -- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfaf320 msgr2=0x56331d028170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfaf320 0x56331d028170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 -- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfd1820 msgr2=0x56331cf8a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfd1820 0x56331cf8a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.219+0000 7f3a2aa00640 1 -- 172.21.15.165:0/2550515018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56331cb9a610 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a2aa00640 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfc5b90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3a18058b40 tx=0x7f3a18088830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63dd4a699b0d2dad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a14c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a14c00640 1 -- 172.21.15.165:0/2550515018 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1807d030 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a14c00640 1 -- 172.21.15.165:0/2550515018 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3a1808a300 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a1806cb70 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a14c00640 1 -- 172.21.15.165:0/2550515018 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a18066050 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 10 client.8232 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x56331cb9a610 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a14c00640 1 -- 172.21.15.165:0/2550515018 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1806cb70 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a14c00640 1 -- 172.21.15.165:0/2550515018 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3a18069020 con 0x56331cfb3f80 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 10 client.8232 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 10 client.8232 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 20 client.8232 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cf8a580 0x56331cfb4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 4 client.8232 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:13:19.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.220+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x56331cef47e0 con 0x56331cf8a580 2024-04-25T10:13:19.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.221+0000 7f3a2b400640 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cf8a580 0x56331cfb4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.222+0000 7f3a2b400640 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cf8a580 0x56331cfb4c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f3acfb53f4a5d10 server_cookie=633bdcc05145bd88 in_seq=0 out_seq=0 2024-04-25T10:13:19.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.222+0000 7f3a14c00640 10 client.8232 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.232+0000 7f3a14c00640 1 -- 172.21.15.165:0/2550515018 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56331cef47e0 con 0x56331cf8a580 2024-04-25T10:13:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.232+0000 7f3a14c00640 10 client.8232 handle_command_reply: tid=0 2024-04-25T10:13:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.232+0000 7f3a353d8740 1 client.8232 shutdown 2024-04-25T10:13:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 msgr2=0x56331cfc5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfc5b90 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f3a18058b40 tx=0x7f3a18088830 comp rx=0 tx=0).stop 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 shutdown_connections 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x56331cf8a580 0x56331cfb4c40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56331cfd1820 0x56331cf8a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56331cfb3f80 0x56331cfc5b90 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 --2- 172.21.15.165:0/2550515018 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56331cfaf320 0x56331d028170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a14c00640 0 client.8232 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 >> 172.21.15.165:0/2550515018 conn(0x56331cf89c60 msgr2=0x56331cfd2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 shutdown_connections 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 1 -- 172.21.15.165:0/2550515018 wait complete. 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.233+0000 7f3a353d8740 20 client.8232 trim_cache size 0 max 16384 2024-04-25T10:13:19.239 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "acdac71d-901d-4929-9e9d-ae8453cf00b0", 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:13:19.240 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a0c06b4d0 msgr2=0x7f3a0c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a0c06b4d0 0x7f3a0c06d990 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c034000 tx=0x7f3a1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 msgr2=0x7f3a2c0cafb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 0x7f3a2c0cafb0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f3a1800e840 tx=0x7f3a1800ed00 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 shutdown_connections 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3a0c06b4d0 0x7f3a0c06d990 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a2c0d1120 0x7f3a2c0b4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a2c09a090 0x7f3a2c0cafb0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 --2- 172.21.15.165:0/1647511031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a2c099cb0 0x7f3a2c0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 >> 172.21.15.165:0/1647511031 conn(0x7f3a2c0b3250 msgr2=0x7f3a2c0cdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 shutdown_connections 2024-04-25T10:13:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.234+0000 7f3a30e00640 1 -- 172.21.15.165:0/1647511031 wait complete. 2024-04-25T10:13:19.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:19.257 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub pause 2024-04-25T10:13:19.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.475+0000 7f75b2600640 1 Processor -- start 2024-04-25T10:13:19.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.475+0000 7f75b2600640 1 -- start start 2024-04-25T10:13:19.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.476+0000 7f75b2600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac161fc0 0x7f75ac1623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.476+0000 7f75b2600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f75ac06ea90 0x7f75ac071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.476+0000 7f75b2600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ac1628e0 con 0x7f75ac06ef10 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.476+0000 7f75b2600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75ac165be0 con 0x7f75ac06ea90 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.476+0000 7f75b2600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac162ba0 con 0x7f75ac161fc0 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f75ac06ea90 0x7f75ac071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f75ac06ea90 0x7f75ac071480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35854/0 (socket says 172.21.15.165:35854) 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 -- 172.21.15.165:0/3113233457 learned_addr learned my addr 172.21.15.165:0/3113233457 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75ab400640 1 --2- 172.21.15.165:0/3113233457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac161fc0 0x7f75ac1623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 -- 172.21.15.165:0/3113233457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac161fc0 msgr2=0x7f75ac1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 --2- 172.21.15.165:0/3113233457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac161fc0 0x7f75ac1623a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 -- 172.21.15.165:0/3113233457 >> v1:172.21.15.172:6789/0 conn(0x7f75ac06ef10 legacy=0x7f75ac06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.477+0000 7f75aaa00640 1 -- 172.21.15.165:0/3113233457 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac162e80 con 0x7f75ac06ea90 2024-04-25T10:13:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.479+0000 7f75aaa00640 1 --2- 172.21.15.165:0/3113233457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ea90 0x7f75ac071480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7598009870 tx=0x7f759802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc1ba53f0590d5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.479+0000 7f75aa000640 1 -- 172.21.15.165:0/3113233457 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598037070 con 0x7f75ac06ea90 2024-04-25T10:13:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.479+0000 7f75aa000640 1 -- 172.21.15.165:0/3113233457 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7598032440 con 0x7f75ac06ea90 2024-04-25T10:13:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.479+0000 7f75aa000640 1 -- 172.21.15.165:0/3113233457 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598032810 con 0x7f75ac06ea90 2024-04-25T10:13:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.479+0000 7f75b2600640 1 -- 172.21.15.165:0/3113233457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ea90 msgr2=0x7f75ac071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.479+0000 7f75b2600640 1 --2- 172.21.15.165:0/3113233457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ea90 0x7f75ac071480 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f7598009870 tx=0x7f759802a180 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.480+0000 7f75b2600640 1 -- 172.21.15.165:0/3113233457 shutdown_connections 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.480+0000 7f75b2600640 1 --2- 172.21.15.165:0/3113233457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ea90 0x7f75ac071480 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.480+0000 7f75b2600640 1 --2- 172.21.15.165:0/3113233457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac161fc0 0x7f75ac1623a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.480+0000 7f75b2600640 1 -- 172.21.15.165:0/3113233457 >> 172.21.15.165:0/3113233457 conn(0x7f75ac06abe0 msgr2=0x7f75ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.480+0000 7f75b2600640 1 -- 172.21.15.165:0/3113233457 shutdown_connections 2024-04-25T10:13:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.480+0000 7f75b2600640 1 -- 172.21.15.165:0/3113233457 wait complete. 2024-04-25T10:13:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.481+0000 7f75b2600640 1 Processor -- start 2024-04-25T10:13:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.482+0000 7f75b2600640 1 -- start start 2024-04-25T10:13:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.482+0000 7f75b2600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac06ea90 0x7f75ac14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.482+0000 7f75b2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 0x7f75ac1f78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75b2600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75ac161fc0 0x7f75ac163f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75b2600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac0719c0 con 0x7f75ac06ef10 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75b2600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac163440 con 0x7f75ac161fc0 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75b2600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac071b40 con 0x7f75ac06ea90 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 0x7f75ac1f78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 0x7f75ac1f78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35864/0 (socket says 172.21.15.165:35864) 2024-04-25T10:13:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 -- 172.21.15.165:0/2315043551 learned_addr learned my addr 172.21.15.165:0/2315043551 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75ab400640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac06ea90 0x7f75ac14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 -- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac06ea90 msgr2=0x7f75ac14da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac06ea90 0x7f75ac14da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 -- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75ac161fc0 msgr2=0x7f75ac163f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75ac161fc0 0x7f75ac163f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.483+0000 7f75aaa00640 1 -- 172.21.15.165:0/2315043551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ac14df70 con 0x7f75ac06ef10 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.484+0000 7f75aaa00640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 0x7f75ac1f78e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7598009870 tx=0x7f75980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.484+0000 7f75a3400640 1 -- 172.21.15.165:0/2315043551 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598041020 con 0x7f75ac06ef10 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.484+0000 7f75a3400640 1 -- 172.21.15.165:0/2315043551 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75980306c0 con 0x7f75ac06ef10 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.484+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ac1f83d0 con 0x7f75ac06ef10 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.484+0000 7f75a3400640 1 -- 172.21.15.165:0/2315043551 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598037040 con 0x7f75ac06ef10 2024-04-25T10:13:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.484+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ac15f700 con 0x7f75ac06ef10 2024-04-25T10:13:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.486+0000 7f75a3400640 1 -- 172.21.15.165:0/2315043551 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7598044020 con 0x7f75ac06ef10 2024-04-25T10:13:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.487+0000 7f75a3400640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f758c06b500 0x7f758c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.487+0000 7f75a3400640 1 -- 172.21.15.165:0/2315043551 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f75980aa900 con 0x7f75ac06ef10 2024-04-25T10:13:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.488+0000 7f75ab400640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f758c06b500 0x7f758c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.488+0000 7f75b537a740 1 Processor -- start 2024-04-25T10:13:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.488+0000 7f75b537a740 1 -- start start 2024-04-25T10:13:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.488+0000 7f75ab400640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f758c06b500 0x7f758c06d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f759c006fd0 tx=0x7f759c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75b537a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc6a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc70b0 0x5647f9c96890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9c96dd0 0x5647f9cbeb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75b537a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9bff870 con 0x5647f9cc6690 2024-04-25T10:13:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9ba8d60 con 0x5647f9c96dd0 2024-04-25T10:13:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f9bfb910 con 0x5647f9cc70b0 2024-04-25T10:13:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35878/0 (socket says 172.21.15.165:35878) 2024-04-25T10:13:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.489+0000 7f75ab400640 1 -- 172.21.15.165:0/2681258232 learned_addr learned my addr 172.21.15.165:0/2681258232 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75aaa00640 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc70b0 0x5647f9c96890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75ab400640 1 -- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc70b0 msgr2=0x5647f9c96890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75abe00640 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9c96dd0 0x5647f9cbeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75ab400640 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc70b0 0x5647f9c96890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75ab400640 1 -- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9c96dd0 msgr2=0x5647f9cbeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75ab400640 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9c96dd0 0x5647f9cbeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75ab400640 1 -- 172.21.15.165:0/2681258232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f99de7d0 con 0x5647f9cc6690 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f75ab400640 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc6a70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e870 tx=0x7f759c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b62e200043cb856 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f7597e00640 1 -- 172.21.15.165:0/2681258232 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c025070 con 0x5647f9cc6690 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f7597e00640 1 -- 172.21.15.165:0/2681258232 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c015640 con 0x5647f9cc6690 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.490+0000 7f7597e00640 1 -- 172.21.15.165:0/2681258232 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759c00f040 con 0x5647f9cc6690 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 -- 172.21.15.165:0/2681258232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 msgr2=0x5647f9cc6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc6a70 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e870 tx=0x7f759c011000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 -- 172.21.15.165:0/2681258232 shutdown_connections 2024-04-25T10:13:19.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9c96dd0 0x5647f9cbeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc70b0 0x5647f9c96890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 --2- 172.21.15.165:0/2681258232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc6a70 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 -- 172.21.15.165:0/2681258232 >> 172.21.15.165:0/2681258232 conn(0x5647f9c98d10 msgr2=0x5647f9ce14a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 -- 172.21.15.165:0/2681258232 shutdown_connections 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.491+0000 7f75b537a740 1 -- 172.21.15.165:0/2681258232 wait complete. 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.492+0000 7f75b537a740 1 -- start start 2024-04-25T10:13:19.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.492+0000 7f75b537a740 1 Processor -- start 2024-04-25T10:13:19.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb5f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 0x5647f9cb8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 0x5647f9cc4a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9cc2de0 con 0x5647f9cc45c0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9bbc1c0 con 0x5647f9cc70b0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f9cc3080 con 0x5647f9cc6690 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.493+0000 7f75b537a740 10 client.0 _ll_register_callbacks cb 0x5647f9bfa8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 0x5647f9cc4a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 0x5647f9cc4a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35882/0 (socket says 172.21.15.165:35882) 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 -- 172.21.15.165:0/3746957858 learned_addr learned my addr 172.21.15.165:0/3746957858 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75ab400640 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 -- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 msgr2=0x5647f9cb5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 -- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 msgr2=0x5647f9cb8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 0x5647f9cb8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 -- 172.21.15.165:0/3746957858 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647f98d40f0 con 0x5647f9cc45c0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f75abe00640 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 0x5647f9cc4a00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d480 tx=0x7f75a400d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60219333f809923c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f7594c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:19.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f7594c00640 1 -- 172.21.15.165:0/3746957858 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4016030 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.494+0000 7f7594c00640 1 -- 172.21.15.165:0/3746957858 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f75a4006b50 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f759c015640 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f7594c00640 1 -- 172.21.15.165:0/3746957858 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75a401f050 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 10 client.8244 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5647f9a3ef60 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f7594c00640 1 -- 172.21.15.165:0/3746957858 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759c015640 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f7594c00640 1 -- 172.21.15.165:0/3746957858 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f75a4003970 con 0x5647f9cc45c0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 10 client.8244 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 10 client.8244 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 20 client.8244 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cb6dd0 0x5647f9cb71d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 4 client.8244 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.495+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5647f9bd5690 con 0x5647f9cb6dd0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.496+0000 7f75ab400640 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cb6dd0 0x5647f9cb71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.497+0000 7f75ab400640 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cb6dd0 0x5647f9cb71d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b2637f927b19a98 server_cookie=cf4dfcb4d4df1074 in_seq=0 out_seq=0 2024-04-25T10:13:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.497+0000 7f7594c00640 10 client.8244 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.498+0000 7f7594c00640 1 -- 172.21.15.165:0/3746957858 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5647f9bd5690 con 0x5647f9cb6dd0 2024-04-25T10:13:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.498+0000 7f7594c00640 10 client.8244 handle_command_reply: tid=0 2024-04-25T10:13:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.498+0000 7f75b537a740 1 client.8244 shutdown 2024-04-25T10:13:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 msgr2=0x5647f9cc4a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 0x5647f9cc4a00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f75a400d480 tx=0x7f75a400d940 comp rx=0 tx=0).stop 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 shutdown_connections 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cb6dd0 0x5647f9cb71d0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc45c0 0x5647f9cc4a00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 0x5647f9cb8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 --2- 172.21.15.165:0/3746957858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f7594c00640 0 client.8244 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 >> 172.21.15.165:0/3746957858 conn(0x5647f9c96dd0 msgr2=0x5647f9c956b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 shutdown_connections 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 1 -- 172.21.15.165:0/3746957858 wait complete. 2024-04-25T10:13:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.499+0000 7f75b537a740 20 client.8244 trim_cache size 0 max 16384 2024-04-25T10:13:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.515+0000 7f75b537a740 1 Processor -- start 2024-04-25T10:13:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.515+0000 7f75b537a740 1 -- start start 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc45c0 0x5647f9cfb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75b537a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 0x5647f9c9a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75b537a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac204b10 con 0x5647f9cc70b0 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac203cb0 con 0x5647f9cc45c0 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac203100 con 0x5647f9cc6690 2024-04-25T10:13:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 0x5647f9c9a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75abe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 0x5647f9c9a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35894/0 (socket says 172.21.15.165:35894) 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75abe00640 1 -- 172.21.15.165:0/1074727481 learned_addr learned my addr 172.21.15.165:0/1074727481 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.516+0000 7f75ab400640 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc45c0 0x5647f9cfb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75aaa00640 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75abe00640 1 -- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 msgr2=0x5647f9cb71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75abe00640 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75abe00640 1 -- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc45c0 msgr2=0x5647f9cfb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75abe00640 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc45c0 0x5647f9cfb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75abe00640 1 -- 172.21.15.165:0/1074727481 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5647f98ad1b0 con 0x5647f9cc70b0 2024-04-25T10:13:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75ab400640 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc45c0 0x5647f9cfb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f75abe00640 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 0x5647f9c9a550 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f75a4002070 tx=0x7f75a4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e77c454181e105 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f7597400640 1 -- 172.21.15.165:0/1074727481 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4027070 con 0x5647f9cc70b0 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f7597400640 1 -- 172.21.15.165:0/1074727481 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4003970 con 0x5647f9cc70b0 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.517+0000 7f7597400640 1 -- 172.21.15.165:0/1074727481 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4016040 con 0x5647f9cc70b0 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 -- 172.21.15.165:0/1074727481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 msgr2=0x5647f9c9a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 0x5647f9c9a550 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f75a4002070 tx=0x7f75a4012000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 -- 172.21.15.165:0/1074727481 shutdown_connections 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc70b0 0x5647f9c9a550 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc6690 0x5647f9cb71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 --2- 172.21.15.165:0/1074727481 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc45c0 0x5647f9cfb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 -- 172.21.15.165:0/1074727481 >> 172.21.15.165:0/1074727481 conn(0x5647f9c96dd0 msgr2=0x5647f9cc49a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 -- 172.21.15.165:0/1074727481 shutdown_connections 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.518+0000 7f75b537a740 1 -- 172.21.15.165:0/1074727481 wait complete. 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.519+0000 7f75b537a740 1 -- start start 2024-04-25T10:13:19.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.519+0000 7f75b537a740 1 Processor -- start 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc45c0 0x5647f9ce68a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc5480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 0x5647f9cc1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ac2036b0 con 0x5647f9cc6690 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5647f9cb85c0 con 0x5647f9cc70b0 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5647f9ce7390 con 0x5647f9cc45c0 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75b537a740 10 client.0 _ll_register_callbacks cb 0x7f759c0094e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:19.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc5480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75aaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc5480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35902/0 (socket says 172.21.15.165:35902) 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.520+0000 7f75aaa00640 1 -- 172.21.15.165:0/89444432 learned_addr learned my addr 172.21.15.165:0/89444432 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75ab400640 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc45c0 0x5647f9ce68a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75aaa00640 1 -- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc45c0 msgr2=0x5647f9ce68a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75aaa00640 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc45c0 0x5647f9ce68a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75aaa00640 1 -- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 msgr2=0x5647f9cc1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75aaa00640 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 0x5647f9cc1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75aaa00640 1 -- 172.21.15.165:0/89444432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5647f9cc49a0 con 0x5647f9cc6690 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75aaa00640 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc5480 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7598081000 tx=0x7f7598078840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cdae105d86d8a2c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f7594c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f7594c00640 1 -- 172.21.15.165:0/89444432 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7598078e20 con 0x5647f9cc6690 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f7594c00640 1 -- 172.21.15.165:0/89444432 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f75980810a0 con 0x5647f9cc6690 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75a4003970 con 0x5647f9cc6690 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f7594c00640 1 -- 172.21.15.165:0/89444432 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f759807cf70 con 0x5647f9cc6690 2024-04-25T10:13:19.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75b537a740 10 client.8250 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.521+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5647f9ce70d0 con 0x5647f9cc6690 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f7594c00640 1 -- 172.21.15.165:0/89444432 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a4003970 con 0x5647f9cc6690 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f7594c00640 1 -- 172.21.15.165:0/89444432 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f759803a030 con 0x5647f9cc6690 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75b537a740 10 client.8250 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75b537a740 10 client.8250 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75b537a740 20 client.8250 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75b537a740 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cfb0c0 0x5647f9cc23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75b537a740 4 client.8250 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x5647f9c0cdb0 con 0x5647f9cfb0c0 2024-04-25T10:13:19.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.522+0000 7f75ab400640 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cfb0c0 0x5647f9cc23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.523+0000 7f75ab400640 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cfb0c0 0x5647f9cc23b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71c4cbd31731a0ef server_cookie=648228a3286cd775 in_seq=0 out_seq=0 2024-04-25T10:13:19.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.523+0000 7f7594c00640 10 client.8250 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.524+0000 7f7594c00640 1 -- 172.21.15.165:0/89444432 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x5647f9c0cdb0 con 0x5647f9cfb0c0 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.524+0000 7f7594c00640 10 client.8250 handle_command_reply: tid=0 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.524+0000 7f75b537a740 1 client.8250 shutdown 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.524+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 msgr2=0x5647f9cc5480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.524+0000 7f75b537a740 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc5480 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7f7598081000 tx=0x7f7598078840 comp rx=0 tx=0).stop 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 shutdown_connections 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5647f9cfb0c0 0x5647f9cc23b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5647f9cc70b0 0x5647f9cc1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5647f9cc6690 0x5647f9cc5480 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 --2- 172.21.15.165:0/89444432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5647f9cc45c0 0x5647f9ce68a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f7594c00640 0 client.8250 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 >> 172.21.15.165:0/89444432 conn(0x5647f9c96dd0 msgr2=0x5647f9ce0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 shutdown_connections 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 1 -- 172.21.15.165:0/89444432 wait complete. 2024-04-25T10:13:19.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b537a740 20 client.8250 trim_cache size 0 max 16384 2024-04-25T10:13:19.533 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:13:19.533 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:13:19.533 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f758c06b500 msgr2=0x7f758c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b2600640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f758c06b500 0x7f758c06d9c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f759c006fd0 tx=0x7f759c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 msgr2=0x7f75ac1f78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.525+0000 7f75b2600640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 0x7f75ac1f78e0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f7598009870 tx=0x7f75980046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 shutdown_connections 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f758c06b500 0x7f758c06d9c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75ac161fc0 0x7f75ac163f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75ac06ef10 0x7f75ac1f78e0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 --2- 172.21.15.165:0/2315043551 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75ac06ea90 0x7f75ac14da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 >> 172.21.15.165:0/2315043551 conn(0x7f75ac06abe0 msgr2=0x7f75ac15ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 shutdown_connections 2024-04-25T10:13:19.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.526+0000 7f75b2600640 1 -- 172.21.15.165:0/2315043551 wait complete. 2024-04-25T10:13:19.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:19.547 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:13:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.729+0000 7fcfe0e00640 1 Processor -- start 2024-04-25T10:13:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.730+0000 7fcfe0e00640 1 -- start start 2024-04-25T10:13:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.731+0000 7fcfe0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcfdc0cadb0 0x7fcfdc0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.731+0000 7fcfe0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.731+0000 7fcfe0e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfdc0afc80 con 0x7fcfdc15de80 2024-04-25T10:13:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.731+0000 7fcfe0e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfdc0cb240 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfe0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0ce4d0 con 0x7fcfdc0ccfe0 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcfdc0cadb0 0x7fcfdc0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcfdc0cadb0 0x7fcfdc0cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35906/0 (socket says 172.21.15.165:35906) 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 -- 172.21.15.165:0/4184865044 learned_addr learned my addr 172.21.15.165:0/4184865044 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfda800640 1 --2- 172.21.15.165:0/4184865044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 -- 172.21.15.165:0/4184865044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 msgr2=0x7fcfdc0cd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 --2- 172.21.15.165:0/4184865044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc0cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 -- 172.21.15.165:0/4184865044 >> v1:172.21.15.172:6789/0 conn(0x7fcfdc15de80 legacy=0x7fcfdc0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.732+0000 7fcfd9400640 1 -- 172.21.15.165:0/4184865044 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0ce7b0 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.733+0000 7fcfd9400640 1 --2- 172.21.15.165:0/4184865044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc0cf6c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009e50 tx=0x7fcfc402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff23b4400592f3fe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.734+0000 7fcfd8a00640 1 -- 172.21.15.165:0/4184865044 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4036070 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.734+0000 7fcfd8a00640 1 -- 172.21.15.165:0/4184865044 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc4031600 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.734+0000 7fcfd8a00640 1 -- 172.21.15.165:0/4184865044 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc40319d0 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.734+0000 7fcfe0e00640 1 -- 172.21.15.165:0/4184865044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 msgr2=0x7fcfdc0cf6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.734+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/4184865044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc0cf6c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009e50 tx=0x7fcfc402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.735+0000 7fcfe0e00640 1 -- 172.21.15.165:0/4184865044 shutdown_connections 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.735+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/4184865044 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc0cd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.735+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/4184865044 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc0cf6c0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.735+0000 7fcfe0e00640 1 -- 172.21.15.165:0/4184865044 >> 172.21.15.165:0/4184865044 conn(0x7fcfdc0b2390 msgr2=0x7fcfdc0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.735+0000 7fcfe0e00640 1 -- 172.21.15.165:0/4184865044 shutdown_connections 2024-04-25T10:13:19.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.735+0000 7fcfe0e00640 1 -- 172.21.15.165:0/4184865044 wait complete. 2024-04-25T10:13:19.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.736+0000 7fcfe0e00640 1 Processor -- start 2024-04-25T10:13:19.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 -- start start 2024-04-25T10:13:19.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc1525d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc15a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc15de80 0x7fcfdc0b0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc15e260 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0cfc00 con 0x7fcfdc15de80 2024-04-25T10:13:19.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.737+0000 7fcfe0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0b2eb0 con 0x7fcfdc0ccfe0 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc1525d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc1525d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35918/0 (socket says 172.21.15.165:35918) 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1819107715 learned_addr learned my addr 172.21.15.165:0/1819107715 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9400640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc15a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfda800640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc15de80 0x7fcfdc0b0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 msgr2=0x7fcfdc15a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc15a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc15de80 msgr2=0x7fcfdc0b0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc15de80 0x7fcfdc0b0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.738+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1819107715 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0c78d0 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.739+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc1525d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c430 tx=0x7fcfcc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.739+0000 7fcfd2000640 1 -- 172.21.15.165:0/1819107715 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011790 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.739+0000 7fcfd2000640 1 -- 172.21.15.165:0/1819107715 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc017070 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.739+0000 7fcfd2000640 1 -- 172.21.15.165:0/1819107715 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc011bb0 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.739+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfdc0c7b40 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.739+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfdc152b10 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.741+0000 7fcfd2000640 1 -- 172.21.15.165:0/1819107715 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfcc01c460 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.742+0000 7fcfd2000640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcfb806b4d0 0x7fcfb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.742+0000 7fcfd2000640 1 -- 172.21.15.165:0/1819107715 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcfcc08a2c0 con 0x7fcfdc0cadb0 2024-04-25T10:13:19.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.742+0000 7fcfd9400640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcfb806b4d0 0x7fcfb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.743+0000 7fcfe3bdb740 1 Processor -- start 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.743+0000 7fcfe3bdb740 1 -- start start 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.743+0000 7fcfd9400640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcfb806b4d0 0x7fcfb806d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4033000 tx=0x7fcfc4009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3cf7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3cf5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d20af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c3c59c70 con 0x5598c3d32280 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c3c02cf0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598c3c55d10 con 0x5598c3d20670 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfd9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3cf5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfd9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3cf5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35930/0 (socket says 172.21.15.165:35930) 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfd9400640 1 -- 172.21.15.165:0/3725939961 learned_addr learned my addr 172.21.15.165:0/3725939961 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfda800640 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d20af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3cf7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfd9400640 1 -- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 msgr2=0x5598c3d20af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.744+0000 7fcfd9400640 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d20af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfd9400640 1 -- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 msgr2=0x5598c3cf7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfd9400640 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3cf7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfd9400640 1 -- 172.21.15.165:0/3725939961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c3a387d0 con 0x5598c3d32280 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3cf7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfd9400640 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3cf5f90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009f40 tx=0x7fcfc402e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a95da5ede101962 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfcaa00640 1 -- 172.21.15.165:0/3725939961 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc4036040 con 0x5598c3d32280 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfcaa00640 1 -- 172.21.15.165:0/3725939961 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc404c070 con 0x5598c3d32280 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.745+0000 7fcfcaa00640 1 -- 172.21.15.165:0/3725939961 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc403c870 con 0x5598c3d32280 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/3725939961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 msgr2=0x5598c3cf5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3cf5f90 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4009f40 tx=0x7fcfc402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/3725939961 shutdown_connections 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d20af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3cf5f90 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/3725939961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3cf7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/3725939961 >> 172.21.15.165:0/3725939961 conn(0x5598c3cbda30 msgr2=0x5598c3d3b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/3725939961 shutdown_connections 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.746+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/3725939961 wait complete. 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.747+0000 7fcfe3bdb740 1 -- start start 2024-04-25T10:13:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.747+0000 7fcfe3bdb740 1 Processor -- start 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d35b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d20670 0x5598c3d223a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d32280 0x5598c3d4b100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c3bf83a0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c3d1e1c0 con 0x5598c3d20670 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598c3d4c210 con 0x5598c3d32280 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfe3bdb740 10 client.0 _ll_register_callbacks cb 0x5598c3c15b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d35b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d35b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35932/0 (socket says 172.21.15.165:35932) 2024-04-25T10:13:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.748+0000 7fcfd9e00640 1 -- 172.21.15.165:0/2956386430 learned_addr learned my addr 172.21.15.165:0/2956386430 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfda800640 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d32280 0x5598c3d4b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfd9e00640 1 -- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d32280 msgr2=0x5598c3d4b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d32280 0x5598c3d4b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfd9e00640 1 -- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d20670 msgr2=0x5598c3d223a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d20670 0x5598c3d223a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfd9e00640 1 -- 172.21.15.165:0/2956386430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598c396e3f0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d35b80 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00cdf0 tx=0x7fcfcc08a1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1913d0a8662b8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfc3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfc3400640 1 -- 172.21.15.165:0/2956386430 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc060ef0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfc3400640 1 -- 172.21.15.165:0/2956386430 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcfcc054e50 con 0x5598c3cf5bb0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfc404c070 con 0x5598c3cf5bb0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfc3400640 1 -- 172.21.15.165:0/2956386430 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfcc070170 con 0x5598c3cf5bb0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfe3bdb740 10 client.8262 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.749+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5598c392e0f0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfc3400640 1 -- 172.21.15.165:0/2956386430 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc404c070 con 0x5598c3cf5bb0 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfc3400640 1 -- 172.21.15.165:0/2956386430 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcfcc057c30 con 0x5598c3cf5bb0 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfe3bdb740 10 client.8262 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfe3bdb740 10 client.8262 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfe3bdb740 20 client.8262 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d4b820 0x5598c3d339a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfe3bdb740 4 client.8262 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.750+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598c3c671b0 con 0x5598c3d4b820 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.751+0000 7fcfd9400640 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d4b820 0x5598c3d339a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.752+0000 7fcfd9400640 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d4b820 0x5598c3d339a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=36194ede96ad629b server_cookie=860a20e5edc8df0 in_seq=0 out_seq=0 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.752+0000 7fcfc3400640 10 client.8262 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfc3400640 1 -- 172.21.15.165:0/2956386430 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598c3c671b0 con 0x5598c3d4b820 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfc3400640 10 client.8262 handle_command_reply: tid=0 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 client.8262 shutdown 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 msgr2=0x5598c3d35b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d35b80 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00cdf0 tx=0x7fcfcc08a1f0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 shutdown_connections 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d4b820 0x5598c3d339a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d32280 0x5598c3d4b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.754+0000 7fcfc3400640 0 client.8262 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.753+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d20670 0x5598c3d223a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.754+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/2956386430 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d35b80 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.754+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 >> 172.21.15.165:0/2956386430 conn(0x5598c3cbda30 msgr2=0x5598c3d1d820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.754+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 shutdown_connections 2024-04-25T10:13:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.754+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/2956386430 wait complete. 2024-04-25T10:13:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.754+0000 7fcfe3bdb740 20 client.8262 trim_cache size 0 max 16384 2024-04-25T10:13:19.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.770+0000 7fcfe3bdb740 1 Processor -- start 2024-04-25T10:13:19.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.770+0000 7fcfe3bdb740 1 -- start start 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d232e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d23820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d32280 0x5598c3d34690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc200350 con 0x5598c3cf5bb0 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1ff550 con 0x5598c3d32280 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc0b4240 con 0x5598c3d20670 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d232e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfd9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d232e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35946/0 (socket says 172.21.15.165:35946) 2024-04-25T10:13:19.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.771+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1094464200 learned_addr learned my addr 172.21.15.165:0/1094464200 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1094464200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 msgr2=0x5598c3d23820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d23820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1094464200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d32280 msgr2=0x5598c3d34690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d32280 0x5598c3d34690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfd9e00640 1 -- 172.21.15.165:0/1094464200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598c3d3ea60 con 0x5598c3cf5bb0 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d232e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0037a0 tx=0x7fcfcc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f994421302aea4a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfca000640 1 -- 172.21.15.165:0/1094464200 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc08a0b0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfca000640 1 -- 172.21.15.165:0/1094464200 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc060ef0 con 0x5598c3cf5bb0 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.772+0000 7fcfca000640 1 -- 172.21.15.165:0/1094464200 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc057c30 con 0x5598c3cf5bb0 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1094464200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 msgr2=0x5598c3d232e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d232e0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0037a0 tx=0x7fcfcc014000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1094464200 shutdown_connections 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3d32280 0x5598c3d34690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3d23820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1094464200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3cf5bb0 0x5598c3d232e0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1094464200 >> 172.21.15.165:0/1094464200 conn(0x5598c3cbda30 msgr2=0x5598c3d1c560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1094464200 shutdown_connections 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.773+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1094464200 wait complete. 2024-04-25T10:13:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.774+0000 7fcfe3bdb740 1 -- start start 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.774+0000 7fcfe3bdb740 1 Processor -- start 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3dab900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3cf4370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3d54fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598c3c16fb0 con 0x5598c3d32280 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfdc1fef80 con 0x5598c3cf5bb0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598c3cf3ae0 con 0x5598c3d20670 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfe3bdb740 10 client.0 _ll_register_callbacks cb 0x7fcfcc054b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfda800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3d54fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfda800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3d54fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35962/0 (socket says 172.21.15.165:35962) 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.775+0000 7fcfda800640 1 -- 172.21.15.165:0/1918679519 learned_addr learned my addr 172.21.15.165:0/1918679519 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfd9400640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3cf4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3dab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfda800640 1 -- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 msgr2=0x5598c3cf4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfda800640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3cf4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfda800640 1 -- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 msgr2=0x5598c3dab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfda800640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3dab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfda800640 1 -- 172.21.15.165:0/1918679519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598c3d4b030 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfda800640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3d54fb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x5598c3d215d0 tx=0x7fcfd400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c4bdb8cff49099f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfc3400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfc3400640 1 -- 172.21.15.165:0/1918679519 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4012830 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfc3400640 1 -- 172.21.15.165:0/1918679519 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcfd40129d0 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.776+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfcc060ef0 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfc3400640 1 -- 172.21.15.165:0/1918679519 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfd4023050 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 10 client.8268 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5598c3daedb0 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfc3400640 1 -- 172.21.15.165:0/1918679519 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc060ef0 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfc3400640 1 -- 172.21.15.165:0/1918679519 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcfd4026020 con 0x5598c3d32280 2024-04-25T10:13:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 10 client.8268 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 10 client.8268 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 20 client.8268 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d20d70 0x5598c3d399b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 4 client.8268 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.777+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5598c3cf3e50 con 0x5598c3d20d70 2024-04-25T10:13:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.778+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d20d70 0x5598c3d399b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfd9e00640 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d20d70 0x5598c3d399b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a631479ac1e579d2 server_cookie=c5d8f1b3b46813d1 in_seq=0 out_seq=0 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfc3400640 10 client.8268 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfc3400640 1 -- 172.21.15.165:0/1918679519 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x5598c3cf3e50 con 0x5598c3d20d70 2024-04-25T10:13:19.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfc3400640 10 client.8268 handle_command_reply: tid=0 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfe3bdb740 1 client.8268 shutdown 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 msgr2=0x5598c3d54fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3d54fb0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x5598c3d215d0 tx=0x7fcfd400f670 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.779+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 shutdown_connections 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5598c3d20d70 0x5598c3d399b0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598c3d32280 0x5598c3d54fb0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598c3d20670 0x5598c3cf4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 --2- 172.21.15.165:0/1918679519 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598c3cf5bb0 0x5598c3dab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfc3400640 0 client.8268 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 >> 172.21.15.165:0/1918679519 conn(0x5598c3cbda30 msgr2=0x5598c3d232e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 shutdown_connections 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 1 -- 172.21.15.165:0/1918679519 wait complete. 2024-04-25T10:13:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe3bdb740 20 client.8268 trim_cache size 0 max 16384 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: "acdac71d-901d-4929-9e9d-ae8453cf00b0": { 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "acdac71d-901d-4929-9e9d-ae8453cf00b0", 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive" 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:19.788 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:13:19.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcfb806b4d0 msgr2=0x7fcfb806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcfb806b4d0 0x7fcfb806d990 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcfc4033000 tx=0x7fcfc4009000 comp rx=0 tx=0).stop 2024-04-25T10:13:19.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 msgr2=0x7fcfdc1525d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.780+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc1525d0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc00c430 tx=0x7fcfcc00c8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 shutdown_connections 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcfb806b4d0 0x7fcfb806d990 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfdc15de80 0x7fcfdc0b0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfdc0ccfe0 0x7fcfdc15a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 --2- 172.21.15.165:0/1819107715 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfdc0cadb0 0x7fcfdc1525d0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 >> 172.21.15.165:0/1819107715 conn(0x7fcfdc0b2390 msgr2=0x7fcfdc0cd450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 shutdown_connections 2024-04-25T10:13:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.781+0000 7fcfe0e00640 1 -- 172.21.15.165:0/1819107715 wait complete. 2024-04-25T10:13:19.802 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.983+0000 7f7e71c00640 1 Processor -- start 2024-04-25T10:13:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.983+0000 7f7e71c00640 1 -- start start 2024-04-25T10:13:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e71c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7e6c154ee0 0x7f7e6c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e71c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e71c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e6c071480 con 0x7f7e6c0cfcf0 2024-04-25T10:13:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e71c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e6c072090 con 0x7f7e6c154ee0 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e71c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c071230 con 0x7f7e6c0ce590 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e6aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7e6c154ee0 0x7f7e6c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e6aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7e6c154ee0 0x7f7e6c0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35968/0 (socket says 172.21.15.165:35968) 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e6aa00640 1 -- 172.21.15.165:0/573504240 learned_addr learned my addr 172.21.15.165:0/573504240 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.985+0000 7f7e6b400640 1 --2- 172.21.15.165:0/573504240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.986+0000 7f7e6aa00640 1 -- 172.21.15.165:0/573504240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 msgr2=0x7f7e6c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.986+0000 7f7e6aa00640 1 --2- 172.21.15.165:0/573504240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c06d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.986+0000 7f7e6aa00640 1 -- 172.21.15.165:0/573504240 >> v1:172.21.15.172:6789/0 conn(0x7f7e6c0cfcf0 legacy=0x7f7e6c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.986+0000 7f7e6aa00640 1 -- 172.21.15.165:0/573504240 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c071760 con 0x7f7e6c154ee0 2024-04-25T10:13:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.987+0000 7f7e6aa00640 1 --2- 172.21.15.165:0/573504240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0cf7b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00bb10 tx=0x7f7e5c02c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f9f607512a5b84 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.987+0000 7f7e6a000640 1 -- 172.21.15.165:0/573504240 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c036020 con 0x7f7e6c154ee0 2024-04-25T10:13:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.987+0000 7f7e6a000640 1 -- 172.21.15.165:0/573504240 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e5c032820 con 0x7f7e6c154ee0 2024-04-25T10:13:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.987+0000 7f7e6a000640 1 -- 172.21.15.165:0/573504240 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c032bf0 con 0x7f7e6c154ee0 2024-04-25T10:13:19.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 -- 172.21.15.165:0/573504240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 msgr2=0x7f7e6c0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 --2- 172.21.15.165:0/573504240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0cf7b0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c00bb10 tx=0x7f7e5c02c920 comp rx=0 tx=0).stop 2024-04-25T10:13:19.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 -- 172.21.15.165:0/573504240 shutdown_connections 2024-04-25T10:13:19.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 --2- 172.21.15.165:0/573504240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c06d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 --2- 172.21.15.165:0/573504240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0cf7b0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 -- 172.21.15.165:0/573504240 >> 172.21.15.165:0/573504240 conn(0x7f7e6c06abe0 msgr2=0x7f7e6c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:19.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 -- 172.21.15.165:0/573504240 shutdown_connections 2024-04-25T10:13:19.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.988+0000 7f7e71c00640 1 -- 172.21.15.165:0/573504240 wait complete. 2024-04-25T10:13:19.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.989+0000 7f7e71c00640 1 Processor -- start 2024-04-25T10:13:19.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.990+0000 7f7e71c00640 1 -- start start 2024-04-25T10:13:19.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.990+0000 7f7e71c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c0ad480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e71c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e6c0cfcf0 0x7f7e6c0abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e71c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:19.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e71c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c1552c0 con 0x7f7e6c154ee0 2024-04-25T10:13:19.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e71c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c071d20 con 0x7f7e6c0cfcf0 2024-04-25T10:13:19.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e71c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e6c155440 con 0x7f7e6c0ce590 2024-04-25T10:13:19.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0b4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35982/0 (socket says 172.21.15.165:35982) 2024-04-25T10:13:19.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 -- 172.21.15.165:0/1986700455 learned_addr learned my addr 172.21.15.165:0/1986700455 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:19.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 -- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 msgr2=0x7f7e6c0ad480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6aa00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c0ad480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e62000640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e6c0cfcf0 0x7f7e6c0abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:19.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c0ad480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 -- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e6c0cfcf0 msgr2=0x7f7e6c0abb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e6c0cfcf0 0x7f7e6c0abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.991+0000 7f7e6b400640 1 -- 172.21.15.165:0/1986700455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e6c0ac400 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.992+0000 7f7e6b400640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0b4070 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007e40 tx=0x7f7e6400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.992+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64014620 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.992+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e64019070 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.992+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e6c0ac6e0 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.992+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e64014a20 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.992+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e6c0acca0 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.993+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e6c073990 con 0x7f7e6c154ee0 2024-04-25T10:13:19.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.994+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7e64014bc0 con 0x7f7e6c154ee0 2024-04-25T10:13:20.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.995+0000 7f7e63400640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7e4806b4d0 0x7f7e4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.996+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7e6408acb0 con 0x7f7e6c154ee0 2024-04-25T10:13:20.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.996+0000 7f7e6aa00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7e4806b4d0 0x7f7e4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:19.996+0000 7f7e6aa00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7e4806b4d0 0x7f7e4806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c037000 tx=0x7f7e5c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.000+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7e64059960 con 0x7f7e6c154ee0 2024-04-25T10:13:20.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.234+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7e6c06d360 con 0x7f7e6c154ee0 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.235+0000 7f7e63400640 1 -- 172.21.15.165:0/1986700455 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f7e64059260 con 0x7f7e6c154ee0 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:20.240 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7e4806b4d0 msgr2=0x7f7e4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7e4806b4d0 0x7f7e4806d990 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c037000 tx=0x7f7e5c008040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 msgr2=0x7f7e6c0b4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0b4070 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7e64007e40 tx=0x7f7e6400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 shutdown_connections 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7e4806b4d0 0x7f7e4806d990 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e6c154ee0 0x7f7e6c0b4070 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e6c0cfcf0 0x7f7e6c0abb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 --2- 172.21.15.165:0/1986700455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e6c0ce590 0x7f7e6c0ad480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 >> 172.21.15.165:0/1986700455 conn(0x7f7e6c06abe0 msgr2=0x7f7e6c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 shutdown_connections 2024-04-25T10:13:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.238+0000 7f7e71c00640 1 -- 172.21.15.165:0/1986700455 wait complete. 2024-04-25T10:13:20.261 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:20.261 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:20.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.460+0000 7f19dba00640 1 Processor -- start 2024-04-25T10:13:20.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.461+0000 7f19dba00640 1 -- start start 2024-04-25T10:13:20.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19dba00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f19d415dc90 0x7f19d41625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19dba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40cc8e0 0x7f19d415ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19dba00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f19d40d1810 0x7f19d40d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19dba00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d415f390 con 0x7f19d40d1810 2024-04-25T10:13:20.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19dba00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19d40cce50 con 0x7f19d415dc90 2024-04-25T10:13:20.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19dba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d4161af0 con 0x7f19d40cc8e0 2024-04-25T10:13:20.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19d9a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f19d40d1810 0x7f19d40d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19d9a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f19d40d1810 0x7f19d40d1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35984/0 (socket says 172.21.15.165:35984) 2024-04-25T10:13:20.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.462+0000 7f19d9a00640 1 -- 172.21.15.165:0/610671778 learned_addr learned my addr 172.21.15.165:0/610671778 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:20.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d3e00640 1 --2- 172.21.15.165:0/610671778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40cc8e0 0x7f19d415ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d9000640 1 --2- 172.21.15.165:0/610671778 >> v2:172.21.15.172:3300/0 conn(0x7f19d415dc90 0x7f19d41625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d9a00640 1 -- 172.21.15.165:0/610671778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40cc8e0 msgr2=0x7f19d415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d9a00640 1 --2- 172.21.15.165:0/610671778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40cc8e0 0x7f19d415ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d9a00640 1 -- 172.21.15.165:0/610671778 >> v2:172.21.15.172:3300/0 conn(0x7f19d415dc90 msgr2=0x7f19d41625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:20.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d9a00640 1 --2- 172.21.15.165:0/610671778 >> v2:172.21.15.172:3300/0 conn(0x7f19d415dc90 0x7f19d41625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.463+0000 7f19d9a00640 1 -- 172.21.15.165:0/610671778 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d4161dd0 con 0x7f19d40d1810 2024-04-25T10:13:20.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.464+0000 7f19d9a00640 1 --2- 172.21.15.165:0/610671778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d40d1810 0x7f19d40d1ca0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f19d406b820 tx=0x7f19cc02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=365a6d95fc382a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19d3400640 1 -- 172.21.15.165:0/610671778 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc036070 con 0x7f19d40d1810 2024-04-25T10:13:20.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19d3400640 1 -- 172.21.15.165:0/610671778 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc031600 con 0x7f19d40d1810 2024-04-25T10:13:20.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19d3400640 1 -- 172.21.15.165:0/610671778 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc0319d0 con 0x7f19d40d1810 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 -- 172.21.15.165:0/610671778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d40d1810 msgr2=0x7f19d40d1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 --2- 172.21.15.165:0/610671778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d40d1810 0x7f19d40d1ca0 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f19d406b820 tx=0x7f19cc02a610 comp rx=0 tx=0).stop 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 -- 172.21.15.165:0/610671778 shutdown_connections 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 --2- 172.21.15.165:0/610671778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d40d1810 0x7f19d40d1ca0 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 --2- 172.21.15.165:0/610671778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40cc8e0 0x7f19d415ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 --2- 172.21.15.165:0/610671778 >> v2:172.21.15.172:3300/0 conn(0x7f19d415dc90 0x7f19d41625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 -- 172.21.15.165:0/610671778 >> 172.21.15.165:0/610671778 conn(0x7f19d406f590 msgr2=0x7f19d406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.465+0000 7f19dba00640 1 -- 172.21.15.165:0/610671778 shutdown_connections 2024-04-25T10:13:20.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.466+0000 7f19dba00640 1 -- 172.21.15.165:0/610671778 wait complete. 2024-04-25T10:13:20.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.467+0000 7f19dba00640 1 Processor -- start 2024-04-25T10:13:20.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.467+0000 7f19dba00640 1 -- start start 2024-04-25T10:13:20.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.468+0000 7f19dba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19d40cc8e0 0x7f19d414c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.468+0000 7f19dba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40d1810 0x7f19d406ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.468+0000 7f19dba00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 0x7f19d406cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.468+0000 7f19dba00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415e070 con 0x7f19d415dc90 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.468+0000 7f19dba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19d40cd130 con 0x7f19d40cc8e0 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.468+0000 7f19dba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19d415e1f0 con 0x7f19d40d1810 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 0x7f19d406cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 0x7f19d406cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35988/0 (socket says 172.21.15.165:35988) 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 -- 172.21.15.165:0/2131655188 learned_addr learned my addr 172.21.15.165:0/2131655188 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d3e00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40d1810 0x7f19d406ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9000640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19d40cc8e0 0x7f19d414c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 -- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40d1810 msgr2=0x7f19d406ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40d1810 0x7f19d406ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 -- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19d40cc8e0 msgr2=0x7f19d414c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19d40cc8e0 0x7f19d414c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.469+0000 7f19d9a00640 1 -- 172.21.15.165:0/2131655188 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19d41f52f0 con 0x7f19d415dc90 2024-04-25T10:13:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.470+0000 7f19d9a00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 0x7f19d406cfc0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009f40 tx=0x7f19cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.470+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc036050 con 0x7f19d415dc90 2024-04-25T10:13:20.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.470+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19cc042070 con 0x7f19d415dc90 2024-04-25T10:13:20.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.470+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19d414e9f0 con 0x7f19d415dc90 2024-04-25T10:13:20.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.470+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19cc009510 con 0x7f19d415dc90 2024-04-25T10:13:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.470+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19d414efb0 con 0x7f19d415dc90 2024-04-25T10:13:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.471+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19d4073990 con 0x7f19d415dc90 2024-04-25T10:13:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.472+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19cc007bf0 con 0x7f19d415dc90 2024-04-25T10:13:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.473+0000 7f19d0c00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19b006b500 0x7f19b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.473+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f19cc0ab2f0 con 0x7f19d415dc90 2024-04-25T10:13:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.473+0000 7f19d9000640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19b006b500 0x7f19b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:20.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.474+0000 7f19d9000640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19b006b500 0x7f19b006d9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006fd0 tx=0x7f19c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:20.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.478+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19cc079f10 con 0x7f19d415dc90 2024-04-25T10:13:20.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.744+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f19d40cf6b0 con 0x7f19d415dc90 2024-04-25T10:13:20.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.744+0000 7f19d0c00640 1 -- 172.21.15.165:0/2131655188 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1026 (secure 0 0 0) 0x7f19cc079810 con 0x7f19d415dc90 2024-04-25T10:13:20.750 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:20.750 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":701,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":985,"data_bytes":1010373368,"bytes_used":2438529024,"bytes_avail":770655584256,"bytes_total":773094113280,"write_bytes_sec":31076915,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:24.341144+0000","services":{}},"progress_events":{}} 2024-04-25T10:13:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.747+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19b006b500 msgr2=0x7f19b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.747+0000 7f19dba00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19b006b500 0x7f19b006d9c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f19c4006fd0 tx=0x7f19c4008040 comp rx=0 tx=0).stop 2024-04-25T10:13:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.747+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 msgr2=0x7f19d406cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.747+0000 7f19dba00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 0x7f19d406cfc0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f19cc009f40 tx=0x7f19cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:20.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 shutdown_connections 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19b006b500 0x7f19b006d9c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19d415dc90 0x7f19d406cfc0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19d40d1810 0x7f19d406ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 --2- 172.21.15.165:0/2131655188 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19d40cc8e0 0x7f19d414c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 >> 172.21.15.165:0/2131655188 conn(0x7f19d406f590 msgr2=0x7f19d406f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 shutdown_connections 2024-04-25T10:13:20.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:20.748+0000 7f19dba00640 1 -- 172.21.15.165:0/2131655188 wait complete. 2024-04-25T10:13:21.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:21.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.977+0000 7f53b5a00640 1 Processor -- start 2024-04-25T10:13:21.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.977+0000 7f53b5a00640 1 -- start start 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.978+0000 7f53b5a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53b0071700 0x7f53b00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.978+0000 7f53b5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 0x7f53b015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.978+0000 7f53b5a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b015bf30 con 0x7f53b0144440 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.978+0000 7f53b5a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b015c860 con 0x7f53b0071700 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.978+0000 7f53b5a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015c210 con 0x7f53b00cd860 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.978+0000 7f53afe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f53b0144440 0x7f53b0070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36682/0 (socket says 172.21.15.165:36682) 2024-04-25T10:13:21.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53afe00640 1 -- 172.21.15.165:0/4131035443 learned_addr learned my addr 172.21.15.165:0/4131035443 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53b5000640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 0x7f53b015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340118436 0 0) 0x7f53b015bf30 con 0x7f53b0144440 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5394003520 con 0x7f53b0144440 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53a7400640 1 --2- 172.21.15.165:0/4131035443 >> v2:172.21.15.172:3300/0 conn(0x7f53b0071700 0x7f53b00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740602502 0 0) 0x7f5394003520 con 0x7f53b0144440 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 msgr2=0x7f53b015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 0x7f53b015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 >> v2:172.21.15.172:3300/0 conn(0x7f53b0071700 msgr2=0x7f53b00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 --2- 172.21.15.165:0/4131035443 >> v2:172.21.15.172:3300/0 conn(0x7f53b0071700 0x7f53b00cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b015c4f0 con 0x7f53b0144440 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f53a0001670 con 0x7f53b0144440 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.979+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 >> v1:172.21.15.165:6789/0 conn(0x7f53b0144440 legacy=0x7f53b0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53af400640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53940039b0 0x7f5394003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53af400640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394004800 0x7f5394004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53af400640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 0x7f53940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015c860 con 0x7f53940039b0 2024-04-25T10:13:21.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015c210 con 0x7f5394004800 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f539400bb00 con 0x7f53b00cd860 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53afe00640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53940039b0 0x7f5394003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53a7400640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394004800 0x7f5394004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53afe00640 1 -- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 msgr2=0x7f53940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53afe00640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00cd860 0x7f53940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53afe00640 1 -- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394004800 msgr2=0x7f5394004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53afe00640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5394004800 0x7f5394004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.980+0000 7f53afe00640 1 -- 172.21.15.165:0/4131035443 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394003520 con 0x7f53940039b0 2024-04-25T10:13:21.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.981+0000 7f53afe00640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53940039b0 0x7f5394003dd0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f53a0007fd0 tx=0x7f5394005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60bd19d0bb1ac7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0007d50 con 0x7f53940039b0 2024-04-25T10:13:21.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a00058d0 con 0x7f53940039b0 2024-04-25T10:13:21.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53af400640 1 -- 172.21.15.165:0/4131035443 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0005ce0 con 0x7f53940039b0 2024-04-25T10:13:21.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5a00640 1 -- 172.21.15.165:0/4131035443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53940039b0 msgr2=0x7f5394003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5a00640 1 --2- 172.21.15.165:0/4131035443 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53940039b0 0x7f5394003dd0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f53a0007fd0 tx=0x7f5394005830 comp rx=0 tx=0).stop 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5000640 1 -- 172.21.15.165:0/4131035443 reap_dead start 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5a00640 1 -- 172.21.15.165:0/4131035443 shutdown_connections 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5a00640 1 -- 172.21.15.165:0/4131035443 >> 172.21.15.165:0/4131035443 conn(0x7f53b006abe0 msgr2=0x7f53b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5a00640 1 -- 172.21.15.165:0/4131035443 shutdown_connections 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.982+0000 7f53b5a00640 1 -- 172.21.15.165:0/4131035443 wait complete. 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.984+0000 7f53b5a00640 1 Processor -- start 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.984+0000 7f53b5a00640 1 -- start start 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53940039b0 0x7f53b0139420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b013f520 0x7f53b013f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 0x7f53b00c7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00cde40 con 0x7f53b00c7b70 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00cdcc0 con 0x7f53940039b0 2024-04-25T10:13:21.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b015ccf0 con 0x7f53b013f520 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 0x7f53b00c7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 0x7f53b00c7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36014/0 (socket says 172.21.15.165:36014) 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.985+0000 7f53b5000640 1 -- 172.21.15.165:0/3474369920 learned_addr learned my addr 172.21.15.165:0/3474369920 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53afe00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53940039b0 0x7f53b0139420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53a7400640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b013f520 0x7f53b013f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5000640 1 -- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b013f520 msgr2=0x7f53b013f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5000640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b013f520 0x7f53b013f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5000640 1 -- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53940039b0 msgr2=0x7f53b0139420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5000640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53940039b0 0x7f53b0139420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5000640 1 -- 172.21.15.165:0/3474369920 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b00c88e0 con 0x7f53b00c7b70 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5000640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 0x7f53b00c7fd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f53b00b2250 tx=0x7f53a800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a800ebc0 con 0x7f53b00c7b70 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a800ed80 con 0x7f53b00c7b70 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a8003bd0 con 0x7f53b00c7b70 2024-04-25T10:13:21.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b00c8bc0 con 0x7f53b00c7b70 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.986+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b00c9180 con 0x7f53b00c7b70 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.988+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b0073990 con 0x7f53b00c7b70 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.988+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53a801b020 con 0x7f53b00c7b70 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.989+0000 7f53acc00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.989+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f53a808a110 con 0x7f53b00c7b70 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.990+0000 7f53afe00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.990+0000 7f53afe00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f53a0033000 tx=0x7f53a0009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:21.994+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53a8058dc0 con 0x7f53b00c7b70 2024-04-25T10:13:22.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.222+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53b0142e80 con 0x7f53b00c7b70 2024-04-25T10:13:22.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.223+0000 7f53acc00640 1 -- 172.21.15.165:0/3474369920 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f53a80586c0 con 0x7f53b00c7b70 2024-04-25T10:13:22.228 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:22.229 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:22.229 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.225+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 msgr2=0x7f538c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.225+0000 7f53b5a00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f53a0033000 tx=0x7f53a0009030 comp rx=0 tx=0).stop 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.225+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 msgr2=0x7f53b00c7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.225+0000 7f53b5a00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 0x7f53b00c7fd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f53b00b2250 tx=0x7f53a800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 shutdown_connections 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b00c7b70 0x7f53b00c7fd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b013f520 0x7f53b013f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 --2- 172.21.15.165:0/3474369920 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53940039b0 0x7f53b0139420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 >> 172.21.15.165:0/3474369920 conn(0x7f53b006abe0 msgr2=0x7f53b006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 shutdown_connections 2024-04-25T10:13:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.226+0000 7f53b5a00640 1 -- 172.21.15.165:0/3474369920 wait complete. 2024-04-25T10:13:22.248 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:22.248 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:22.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.432+0000 7fd13be00640 1 Processor -- start 2024-04-25T10:13:22.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.432+0000 7fd13be00640 1 -- start start 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd13c0b93c0 0x7fd13c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd13c0d04f0 0x7fd13c0d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd13c161df0 con 0x7fd13c15db90 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13c0d1f50 con 0x7fd13c0b93c0 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd13c0d0940 con 0x7fd13c0d04f0 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd13c0b93c0 0x7fd13c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd13c0b93c0 0x7fd13c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36030/0 (socket says 172.21.15.165:36030) 2024-04-25T10:13:22.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.434+0000 7fd13a000640 1 -- 172.21.15.165:0/3350435235 learned_addr learned my addr 172.21.15.165:0/3350435235 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.435+0000 7fd13b400640 1 --2- 172.21.15.165:0/3350435235 >> v2:172.21.15.172:3300/0 conn(0x7fd13c0d04f0 0x7fd13c0d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.435+0000 7fd13a000640 1 -- 172.21.15.165:0/3350435235 >> v2:172.21.15.172:3300/0 conn(0x7fd13c0d04f0 msgr2=0x7fd13c0d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.435+0000 7fd13a000640 1 --2- 172.21.15.165:0/3350435235 >> v2:172.21.15.172:3300/0 conn(0x7fd13c0d04f0 0x7fd13c0d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.435+0000 7fd13a000640 1 -- 172.21.15.165:0/3350435235 >> v1:172.21.15.172:6789/0 conn(0x7fd13c15db90 legacy=0x7fd13c1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.435+0000 7fd13a000640 1 -- 172.21.15.165:0/3350435235 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c1620d0 con 0x7fd13c0b93c0 2024-04-25T10:13:22.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.435+0000 7fd13b400640 1 --2- 172.21.15.165:0/3350435235 >> v2:172.21.15.172:3300/0 conn(0x7fd13c0d04f0 0x7fd13c0d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.436+0000 7fd13a000640 1 --2- 172.21.15.165:0/3350435235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c0cffb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd13c08cc30 tx=0x7fd12402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f506386b02c4264 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.436+0000 7fd139600640 1 -- 172.21.15.165:0/3350435235 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124037070 con 0x7fd13c0b93c0 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.436+0000 7fd139600640 1 -- 172.21.15.165:0/3350435235 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd124032440 con 0x7fd13c0b93c0 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.436+0000 7fd139600640 1 -- 172.21.15.165:0/3350435235 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124032810 con 0x7fd13c0b93c0 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 -- 172.21.15.165:0/3350435235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 msgr2=0x7fd13c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 --2- 172.21.15.165:0/3350435235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c0cffb0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fd13c08cc30 tx=0x7fd12402a230 comp rx=0 tx=0).stop 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 -- 172.21.15.165:0/3350435235 shutdown_connections 2024-04-25T10:13:22.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 --2- 172.21.15.165:0/3350435235 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c0cffb0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 --2- 172.21.15.165:0/3350435235 >> v2:172.21.15.172:3300/0 conn(0x7fd13c0d04f0 0x7fd13c0d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 -- 172.21.15.165:0/3350435235 >> 172.21.15.165:0/3350435235 conn(0x7fd13c0b52b0 msgr2=0x7fd13c0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 -- 172.21.15.165:0/3350435235 shutdown_connections 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.437+0000 7fd13be00640 1 -- 172.21.15.165:0/3350435235 wait complete. 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.439+0000 7fd13be00640 1 Processor -- start 2024-04-25T10:13:22.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.439+0000 7fd13be00640 1 -- start start 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.440+0000 7fd13be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c14ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.440+0000 7fd13be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd13c0d04f0 0x7fd13c150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.440+0000 7fd13be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd13c15db90 0x7fd13c151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b68d0 con 0x7fd13c0b93c0 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0d1be0 con 0x7fd13c0d04f0 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd13c0b6a50 con 0x7fd13c15db90 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c14ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c14ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36040/0 (socket says 172.21.15.165:36040) 2024-04-25T10:13:22.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 -- 172.21.15.165:0/1555972639 learned_addr learned my addr 172.21.15.165:0/1555972639 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13a000640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd13c0d04f0 0x7fd13c150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13b400640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd13c15db90 0x7fd13c151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 -- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd13c15db90 msgr2=0x7fd13c151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd13c15db90 0x7fd13c151150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 -- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd13c0d04f0 msgr2=0x7fd13c150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd13c0d04f0 0x7fd13c150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.441+0000 7fd13aa00640 1 -- 172.21.15.165:0/1555972639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c14f230 con 0x7fd13c0b93c0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.442+0000 7fd13aa00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c14ecf0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b420 tx=0x7fd12c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.442+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c011830 con 0x7fd13c0b93c0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.442+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12c016070 con 0x7fd13c0b93c0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.442+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd13c15f280 con 0x7fd13c0b93c0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.442+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12c011c10 con 0x7fd13c0b93c0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.442+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd13c151a60 con 0x7fd13c0b93c0 2024-04-25T10:13:22.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.443+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd13c073990 con 0x7fd13c0b93c0 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.444+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd12c01b460 con 0x7fd13c0b93c0 2024-04-25T10:13:22.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.445+0000 7fd132a00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd11806b4d0 0x7fd11806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:22.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.445+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd12c089fc0 con 0x7fd13c0b93c0 2024-04-25T10:13:22.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.446+0000 7fd13a000640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd11806b4d0 0x7fd11806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:22.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.446+0000 7fd13a000640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd11806b4d0 0x7fd11806d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd124034000 tx=0x7fd12402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:22.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.450+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd12c020020 con 0x7fd13c0b93c0 2024-04-25T10:13:22.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.717+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd13c0d3320 con 0x7fd13c0b93c0 2024-04-25T10:13:22.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.718+0000 7fd132a00640 1 -- 172.21.15.165:0/1555972639 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1027 (secure 0 0 0) 0x7fd13c14f230 con 0x7fd13c0b93c0 2024-04-25T10:13:22.723 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:22.723 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":703,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580987,"bytes_used":2601459712,"bytes_avail":770492653568,"bytes_total":773094113280,"write_bytes_sec":36595014,"read_op_per_sec":0,"write_op_per_sec":37},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":25,"modified":"2024-04-25T10:12:24.341144+0000","services":{}},"progress_events":{}} 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd11806b4d0 msgr2=0x7fd11806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd11806b4d0 0x7fd11806d990 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fd124034000 tx=0x7fd12402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 msgr2=0x7fd13c14ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c14ecf0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fd12c00b420 tx=0x7fd12c00b8e0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 shutdown_connections 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd11806b4d0 0x7fd11806d990 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd13c15db90 0x7fd13c151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd13c0d04f0 0x7fd13c150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 --2- 172.21.15.165:0/1555972639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd13c0b93c0 0x7fd13c14ecf0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:22.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.720+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 >> 172.21.15.165:0/1555972639 conn(0x7fd13c0b52b0 msgr2=0x7fd13c0d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:22.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.721+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 shutdown_connections 2024-04-25T10:13:22.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:22.721+0000 7fd13be00640 1 -- 172.21.15.165:0/1555972639 wait complete. 2024-04-25T10:13:23.745 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:23.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.952+0000 7fd75a000640 1 Processor -- start 2024-04-25T10:13:23.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.952+0000 7fd75a000640 1 -- start start 2024-04-25T10:13:23.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd75a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd754154ee0 0x7fd7540cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd75a000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd754071480 con 0x7fd7540cfcf0 2024-04-25T10:13:23.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd75a000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd754071760 con 0x7fd7540ce590 2024-04-25T10:13:23.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd75a000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd754072090 con 0x7fd754154ee0 2024-04-25T10:13:23.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd753400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd754154ee0 0x7fd7540cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd752a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd7540cfcf0 0x7fd7540cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36694/0 (socket says 172.21.15.165:36694) 2024-04-25T10:13:23.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd753400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd754154ee0 0x7fd7540cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36044/0 (socket says 172.21.15.165:36044) 2024-04-25T10:13:23.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd752a00640 1 -- 172.21.15.165:0/3426505662 learned_addr learned my addr 172.21.15.165:0/3426505662 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:23.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd753400640 1 -- 172.21.15.165:0/3426505662 >> v1:172.21.15.172:6789/0 conn(0x7fd7540ce590 legacy=0x7fd75406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:23.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd753400640 1 -- 172.21.15.165:0/3426505662 >> v1:172.21.15.165:6789/0 conn(0x7fd7540cfcf0 legacy=0x7fd7540cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd753400640 1 -- 172.21.15.165:0/3426505662 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd754071a40 con 0x7fd754154ee0 2024-04-25T10:13:23.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.954+0000 7fd752000640 1 -- 172.21.15.165:0/3426505662 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 718885311 0 0) 0x7fd754071480 con 0x7fd7540cfcf0 2024-04-25T10:13:23.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.955+0000 7fd753400640 1 --2- 172.21.15.165:0/3426505662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd754154ee0 0x7fd7540cf7b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd74400ba40 tx=0x7fd74402c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e083f6b872ae6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.955+0000 7fd752000640 1 -- 172.21.15.165:0/3426505662 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744032770 con 0x7fd754154ee0 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.955+0000 7fd752000640 1 -- 172.21.15.165:0/3426505662 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd744038070 con 0x7fd754154ee0 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.955+0000 7fd752000640 1 -- 172.21.15.165:0/3426505662 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd744032b30 con 0x7fd754154ee0 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.955+0000 7fd75a000640 1 -- 172.21.15.165:0/3426505662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd754154ee0 msgr2=0x7fd7540cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.955+0000 7fd75a000640 1 --2- 172.21.15.165:0/3426505662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd754154ee0 0x7fd7540cf7b0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd74400ba40 tx=0x7fd74402c8b0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.956+0000 7fd75a000640 1 -- 172.21.15.165:0/3426505662 shutdown_connections 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.956+0000 7fd75a000640 1 --2- 172.21.15.165:0/3426505662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd754154ee0 0x7fd7540cf7b0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.956+0000 7fd75a000640 1 -- 172.21.15.165:0/3426505662 >> 172.21.15.165:0/3426505662 conn(0x7fd75406abe0 msgr2=0x7fd75406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.956+0000 7fd75a000640 1 -- 172.21.15.165:0/3426505662 shutdown_connections 2024-04-25T10:13:23.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.956+0000 7fd75a000640 1 -- 172.21.15.165:0/3426505662 wait complete. 2024-04-25T10:13:23.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.957+0000 7fd75a000640 1 Processor -- start 2024-04-25T10:13:23.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.957+0000 7fd75a000640 1 -- start start 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.958+0000 7fd75a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7540ce590 0x7fd7540a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.958+0000 7fd75a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 0x7fd7540aca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.958+0000 7fd75a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754154ee0 0x7fd7540aa4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.958+0000 7fd75a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7541552c0 con 0x7fd7540cfcf0 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.958+0000 7fd75a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd754071230 con 0x7fd754154ee0 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.958+0000 7fd75a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd754155440 con 0x7fd7540ce590 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 0x7fd7540aca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 0x7fd7540aca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36050/0 (socket says 172.21.15.165:36050) 2024-04-25T10:13:23.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 -- 172.21.15.165:0/3016217641 learned_addr learned my addr 172.21.15.165:0/3016217641 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd753400640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7540ce590 0x7fd7540a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 -- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7540ce590 msgr2=0x7fd7540a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd753e00640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754154ee0 0x7fd7540aa4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7540ce590 0x7fd7540a9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 -- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754154ee0 msgr2=0x7fd7540aa4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754154ee0 0x7fd7540aa4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd752a00640 1 -- 172.21.15.165:0/3016217641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7540ad1f0 con 0x7fd7540cfcf0 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.959+0000 7fd753e00640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754154ee0 0x7fd7540aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.960+0000 7fd752a00640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 0x7fd7540aca60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c4a0 tx=0x7fd74000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.960+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740018070 con 0x7fd7540cfcf0 2024-04-25T10:13:23.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.960+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7400074e0 con 0x7fd7540cfcf0 2024-04-25T10:13:23.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.960+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7540ad4d0 con 0x7fd7540cfcf0 2024-04-25T10:13:23.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.960+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd740004a20 con 0x7fd7540cfcf0 2024-04-25T10:13:23.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.960+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7540ada90 con 0x7fd7540cfcf0 2024-04-25T10:13:23.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.961+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd754073990 con 0x7fd7540cfcf0 2024-04-25T10:13:23.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.962+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd74001b070 con 0x7fd7540cfcf0 2024-04-25T10:13:23.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.963+0000 7fd74b400640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd73406b4d0 0x7fd73406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:23.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.963+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd740089700 con 0x7fd7540cfcf0 2024-04-25T10:13:23.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.963+0000 7fd753400640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd73406b4d0 0x7fd73406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:23.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.964+0000 7fd753400640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd73406b4d0 0x7fd73406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd744037000 tx=0x7fd744032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:23.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:23.967+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7400583b0 con 0x7fd7540cfcf0 2024-04-25T10:13:24.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.203+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7540a8750 con 0x7fd7540cfcf0 2024-04-25T10:13:24.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.204+0000 7fd74b400640 1 -- 172.21.15.165:0/3016217641 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fd740057cb0 con 0x7fd7540cfcf0 2024-04-25T10:13:24.210 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:24.210 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:24.210 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd73406b4d0 msgr2=0x7fd73406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd73406b4d0 0x7fd73406d990 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fd744037000 tx=0x7fd744032000 comp rx=0 tx=0).stop 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 msgr2=0x7fd7540aca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 0x7fd7540aca60 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fd74000c4a0 tx=0x7fd74000c960 comp rx=0 tx=0).stop 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 shutdown_connections 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd73406b4d0 0x7fd73406d990 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd754154ee0 0x7fd7540aa4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7540cfcf0 0x7fd7540aca60 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 --2- 172.21.15.165:0/3016217641 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7540ce590 0x7fd7540a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 >> 172.21.15.165:0/3016217641 conn(0x7fd75406abe0 msgr2=0x7fd7540aea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 shutdown_connections 2024-04-25T10:13:24.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.207+0000 7fd75a000640 1 -- 172.21.15.165:0/3016217641 wait complete. 2024-04-25T10:13:24.233 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:24.233 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:24.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.422+0000 7fcb69000640 1 Processor -- start 2024-04-25T10:13:24.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.423+0000 7fcb69000640 1 -- start start 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb69000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb64071700 0x7fcb640cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb69000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb640cd860 0x7fcb6415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb69000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb6415bf30 con 0x7fcb64144440 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb69000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb6415c860 con 0x7fcb64071700 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb69000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6415c210 con 0x7fcb640cd860 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb63400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcb64144440 0x7fcb64070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36702/0 (socket says 172.21.15.165:36702) 2024-04-25T10:13:24.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb63400640 1 -- 172.21.15.165:0/3466106733 learned_addr learned my addr 172.21.15.165:0/3466106733 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62a00640 1 --2- 172.21.15.165:0/3466106733 >> v2:172.21.15.165:3300/0 conn(0x7fcb64071700 0x7fcb640cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62000640 1 -- 172.21.15.165:0/3466106733 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301031542 0 0) 0x7fcb6415bf30 con 0x7fcb64144440 2024-04-25T10:13:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62000640 1 -- 172.21.15.165:0/3466106733 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb48003520 con 0x7fcb64144440 2024-04-25T10:13:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62a00640 1 -- 172.21.15.165:0/3466106733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb640cd860 msgr2=0x7fcb6415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62a00640 1 --2- 172.21.15.165:0/3466106733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb640cd860 0x7fcb6415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62a00640 1 -- 172.21.15.165:0/3466106733 >> v1:172.21.15.165:6789/0 conn(0x7fcb64144440 legacy=0x7fcb64070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.425+0000 7fcb62a00640 1 -- 172.21.15.165:0/3466106733 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb6415c4f0 con 0x7fcb64071700 2024-04-25T10:13:24.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.426+0000 7fcb62a00640 1 --2- 172.21.15.165:0/3466106733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071700 0x7fcb640cd320 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009d90 tx=0x7fcb4c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5267b912e5b31d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.427+0000 7fcb62000640 1 -- 172.21.15.165:0/3466106733 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c007740 con 0x7fcb64071700 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.427+0000 7fcb62000640 1 -- 172.21.15.165:0/3466106733 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c0078e0 con 0x7fcb64071700 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.427+0000 7fcb62000640 1 -- 172.21.15.165:0/3466106733 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c007cb0 con 0x7fcb64071700 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.427+0000 7fcb69000640 1 -- 172.21.15.165:0/3466106733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071700 msgr2=0x7fcb640cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.427+0000 7fcb69000640 1 --2- 172.21.15.165:0/3466106733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071700 0x7fcb640cd320 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009d90 tx=0x7fcb4c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.428+0000 7fcb69000640 1 -- 172.21.15.165:0/3466106733 shutdown_connections 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.428+0000 7fcb69000640 1 --2- 172.21.15.165:0/3466106733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb640cd860 0x7fcb6415ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.428+0000 7fcb69000640 1 --2- 172.21.15.165:0/3466106733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb64071700 0x7fcb640cd320 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.428+0000 7fcb69000640 1 -- 172.21.15.165:0/3466106733 >> 172.21.15.165:0/3466106733 conn(0x7fcb6406abe0 msgr2=0x7fcb6406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.428+0000 7fcb69000640 1 -- 172.21.15.165:0/3466106733 shutdown_connections 2024-04-25T10:13:24.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.428+0000 7fcb69000640 1 -- 172.21.15.165:0/3466106733 wait complete. 2024-04-25T10:13:24.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.429+0000 7fcb69000640 1 Processor -- start 2024-04-25T10:13:24.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.430+0000 7fcb69000640 1 -- start start 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb69000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64071700 0x7fcb640cb940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb69000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 0x7fcb6413b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb69000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb64144440 0x7fcb6413bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb69000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640cde40 con 0x7fcb640cd860 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb69000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb640cdcc0 con 0x7fcb64144440 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb69000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb6415d230 con 0x7fcb64071700 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 0x7fcb6413b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 0x7fcb6413b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36062/0 (socket says 172.21.15.165:36062) 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 -- 172.21.15.165:0/514598005 learned_addr learned my addr 172.21.15.165:0/514598005 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb63400640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64071700 0x7fcb640cb940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 -- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64071700 msgr2=0x7fcb640cb940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64071700 0x7fcb640cb940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 -- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb64144440 msgr2=0x7fcb6413bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb63e00640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb64144440 0x7fcb6413bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.431+0000 7fcb62a00640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb64144440 0x7fcb6413bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb62a00640 1 -- 172.21.15.165:0/514598005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640ca8a0 con 0x7fcb640cd860 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb63400640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64071700 0x7fcb640cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb62a00640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 0x7fcb6413b9d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009eb0 tx=0x7fcb4c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c040070 con 0x7fcb640cd860 2024-04-25T10:13:24.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb4c031c40 con 0x7fcb640cd860 2024-04-25T10:13:24.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb4c045460 con 0x7fcb640cd860 2024-04-25T10:13:24.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb640cab80 con 0x7fcb640cd860 2024-04-25T10:13:24.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.432+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb640cb140 con 0x7fcb640cd860 2024-04-25T10:13:24.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.434+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb64073990 con 0x7fcb640cd860 2024-04-25T10:13:24.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.435+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb4c039040 con 0x7fcb640cd860 2024-04-25T10:13:24.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.435+0000 7fcb5b400640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.436+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcb4c0ab600 con 0x7fcb640cd860 2024-04-25T10:13:24.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.436+0000 7fcb63400640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.436+0000 7fcb63400640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005880 tx=0x7fcb54009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.441+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcb4c07a220 con 0x7fcb640cd860 2024-04-25T10:13:24.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.687+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fcb6406bff0 con 0x7fcb640cd860 2024-04-25T10:13:24.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.688+0000 7fcb5b400640 1 -- 172.21.15.165:0/514598005 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fcb4c079b20 con 0x7fcb640cd860 2024-04-25T10:13:24.693 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:24.694 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":705,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580987,"bytes_used":2601459712,"bytes_avail":770492653568,"bytes_total":773094113280,"write_bytes_sec":21100689,"read_op_per_sec":0,"write_op_per_sec":21},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:13:24.361852+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 msgr2=0x7fcb4006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fcb54005880 tx=0x7fcb54009040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 msgr2=0x7fcb6413b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 0x7fcb6413b9d0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c009eb0 tx=0x7fcb4c004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:24.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 shutdown_connections 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb4006b500 0x7fcb4006d9c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb64144440 0x7fcb6413bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb640cd860 0x7fcb6413b9d0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 --2- 172.21.15.165:0/514598005 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb64071700 0x7fcb640cb940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 >> 172.21.15.165:0/514598005 conn(0x7fcb6406abe0 msgr2=0x7fcb64140fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 shutdown_connections 2024-04-25T10:13:24.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.691+0000 7fcb69000640 1 -- 172.21.15.165:0/514598005 wait complete. 2024-04-25T10:13:24.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:13:24.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:24.714 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub resume 2024-04-25T10:13:24.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.906+0000 7f8647000640 1 Processor -- start 2024-04-25T10:13:24.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.906+0000 7f8647000640 1 -- start start 2024-04-25T10:13:24.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8647000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8640145140 0x7f86400ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8647000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86400ce950 0x7f86400c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8647000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f86400c5030 0x7f864006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8647000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86400ced90 con 0x7f86400c5030 2024-04-25T10:13:24.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8647000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8640145520 con 0x7f8640145140 2024-04-25T10:13:24.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8647000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8640071270 con 0x7f86400ce950 2024-04-25T10:13:24.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f86400c5030 0x7f864006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f86400c5030 0x7f864006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36070/0 (socket says 172.21.15.165:36070) 2024-04-25T10:13:24.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 -- 172.21.15.165:0/1804238101 learned_addr learned my addr 172.21.15.165:0/1804238101 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8645c00640 1 --2- 172.21.15.165:0/1804238101 >> v2:172.21.15.172:3300/0 conn(0x7f8640145140 0x7f86400ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8645200640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86400ce950 0x7f86400c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 -- 172.21.15.165:0/1804238101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86400ce950 msgr2=0x7f86400c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86400ce950 0x7f86400c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 -- 172.21.15.165:0/1804238101 >> v2:172.21.15.172:3300/0 conn(0x7f8640145140 msgr2=0x7f86400ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 --2- 172.21.15.165:0/1804238101 >> v2:172.21.15.172:3300/0 conn(0x7f8640145140 0x7f86400ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.907+0000 7f8646600640 1 -- 172.21.15.165:0/1804238101 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8640071550 con 0x7f86400c5030 2024-04-25T10:13:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.908+0000 7f8645c00640 1 --2- 172.21.15.165:0/1804238101 >> v2:172.21.15.172:3300/0 conn(0x7f8640145140 0x7f86400ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.908+0000 7f8645200640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86400ce950 0x7f86400c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8646600640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400c5030 0x7f864006d1f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f864006f690 tx=0x7f863802a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a9baa71d96d4ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f863fe00640 1 -- 172.21.15.165:0/1804238101 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8638036070 con 0x7f86400c5030 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f863fe00640 1 -- 172.21.15.165:0/1804238101 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8638031600 con 0x7f86400c5030 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f863fe00640 1 -- 172.21.15.165:0/1804238101 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86380319d0 con 0x7f86400c5030 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 -- 172.21.15.165:0/1804238101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400c5030 msgr2=0x7f864006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400c5030 0x7f864006d1f0 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f864006f690 tx=0x7f863802a600 comp rx=0 tx=0).stop 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 -- 172.21.15.165:0/1804238101 shutdown_connections 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400c5030 0x7f864006d1f0 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 --2- 172.21.15.165:0/1804238101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86400ce950 0x7f86400c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 --2- 172.21.15.165:0/1804238101 >> v2:172.21.15.172:3300/0 conn(0x7f8640145140 0x7f86400ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.909+0000 7f8647000640 1 -- 172.21.15.165:0/1804238101 >> 172.21.15.165:0/1804238101 conn(0x7f8640061cc0 msgr2=0x7f864006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.911+0000 7f8647000640 1 -- 172.21.15.165:0/1804238101 shutdown_connections 2024-04-25T10:13:24.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.911+0000 7f8647000640 1 -- 172.21.15.165:0/1804238101 wait complete. 2024-04-25T10:13:24.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.912+0000 7f8647000640 1 Processor -- start 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.912+0000 7f8647000640 1 -- start start 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8647000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86400c5030 0x7f86400b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8647000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 0x7f86401f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8647000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8640145140 0x7f86400cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8647000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c54a0 con 0x7f86400ce950 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8647000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8640071b10 con 0x7f86400c5030 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8647000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86400c5620 con 0x7f8640145140 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8645200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 0x7f86401f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8645200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 0x7f86401f3b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36076/0 (socket says 172.21.15.165:36076) 2024-04-25T10:13:24.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.913+0000 7f8645200640 1 -- 172.21.15.165:0/2242371066 learned_addr learned my addr 172.21.15.165:0/2242371066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645c00640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86400c5030 0x7f86400b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8646600640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8640145140 0x7f86400cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645200640 1 -- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8640145140 msgr2=0x7f86400cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645200640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8640145140 0x7f86400cd340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645200640 1 -- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86400c5030 msgr2=0x7f86400b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645200640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86400c5030 0x7f86400b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645200640 1 -- 172.21.15.165:0/2242371066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86400cc2a0 con 0x7f86400ce950 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8645200640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 0x7f86401f3b30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f862c00c7c0 tx=0x7f862c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f863d600640 1 -- 172.21.15.165:0/2242371066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c015070 con 0x7f86400ce950 2024-04-25T10:13:24.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f863d600640 1 -- 172.21.15.165:0/2242371066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f862c007a30 con 0x7f86400ce950 2024-04-25T10:13:24.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86400cc580 con 0x7f86400ce950 2024-04-25T10:13:24.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f863d600640 1 -- 172.21.15.165:0/2242371066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c007e00 con 0x7f86400ce950 2024-04-25T10:13:24.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.914+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86400ccb40 con 0x7f86400ce950 2024-04-25T10:13:24.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.916+0000 7f863d600640 1 -- 172.21.15.165:0/2242371066 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f862c018070 con 0x7f86400ce950 2024-04-25T10:13:24.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.917+0000 7f863d600640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f862006b4d0 0x7f862006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.917+0000 7f863d600640 1 -- 172.21.15.165:0/2242371066 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f862c08a3d0 con 0x7f86400ce950 2024-04-25T10:13:24.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.917+0000 7f8645c00640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f862006b4d0 0x7f862006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.917+0000 7f864b56c740 1 Processor -- start 2024-04-25T10:13:24.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.917+0000 7f8645c00640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f862006b4d0 0x7f862006d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8640133db0 tx=0x7f8634008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.917+0000 7f864b56c740 1 -- start start 2024-04-25T10:13:24.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f864b56c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c408ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c407150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c431830 0x557a9c431cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f864b56c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c36ac70 con 0x557a9c406d70 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c313cf0 con 0x557a9c431830 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a9c366d10 con 0x557a9c443440 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f8645c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c408ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f8645c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c408ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36086/0 (socket says 172.21.15.165:36086) 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.919+0000 7f8645c00640 1 -- 172.21.15.165:0/1707975684 learned_addr learned my addr 172.21.15.165:0/1707975684 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645200640 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c407150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645c00640 1 -- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 msgr2=0x557a9c407150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645c00640 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c407150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8646600640 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c431830 0x557a9c431cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645c00640 1 -- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c431830 msgr2=0x557a9c431cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645c00640 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c431830 0x557a9c431cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645c00640 1 -- 172.21.15.165:0/1707975684 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9c1497d0 con 0x557a9c406d70 2024-04-25T10:13:24.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8645c00640 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c408ed0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f863400db90 tx=0x7f8634010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87f8f6c6b4413fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8632000640 1 -- 172.21.15.165:0/1707975684 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634019050 con 0x557a9c406d70 2024-04-25T10:13:24.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8632000640 1 -- 172.21.15.165:0/1707975684 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634023070 con 0x557a9c406d70 2024-04-25T10:13:24.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f8632000640 1 -- 172.21.15.165:0/1707975684 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634012de0 con 0x557a9c406d70 2024-04-25T10:13:24.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 -- 172.21.15.165:0/1707975684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 msgr2=0x557a9c408ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c408ed0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f863400db90 tx=0x7f8634010000 comp rx=0 tx=0).stop 2024-04-25T10:13:24.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 -- 172.21.15.165:0/1707975684 shutdown_connections 2024-04-25T10:13:24.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c431830 0x557a9c431cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c407150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 --2- 172.21.15.165:0/1707975684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c408ed0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 -- 172.21.15.165:0/1707975684 >> 172.21.15.165:0/1707975684 conn(0x557a9c3cea30 msgr2=0x557a9c44c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 -- 172.21.15.165:0/1707975684 shutdown_connections 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.920+0000 7f864b56c740 1 -- 172.21.15.165:0/1707975684 wait complete. 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 -- start start 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 Processor -- start 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c406d70 0x557a9c41d120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 0x557a9c44f670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c44fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c3093a0 con 0x557a9c431830 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c42f380 con 0x557a9c406d70 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a9c432b70 con 0x557a9c443440 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f864b56c740 10 client.0 _ll_register_callbacks cb 0x557a9c326b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f8645200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 0x557a9c44f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f8645200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 0x557a9c44f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36094/0 (socket says 172.21.15.165:36094) 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.922+0000 7f8645200640 1 -- 172.21.15.165:0/3170695891 learned_addr learned my addr 172.21.15.165:0/3170695891 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8645200640 1 -- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 msgr2=0x557a9c44fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8645200640 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c44fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8645200640 1 -- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c406d70 msgr2=0x557a9c41d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8645200640 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c406d70 0x557a9c41d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8645200640 1 -- 172.21.15.165:0/3170695891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a9c07f3f0 con 0x557a9c431830 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8645200640 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 0x557a9c44f670 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f862c007410 tx=0x7f862c059130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b07ecec8e87b6355 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8626a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8626a00640 1 -- 172.21.15.165:0/3170695891 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f862c01f040 con 0x557a9c431830 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8626a00640 1 -- 172.21.15.165:0/3170695891 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f862c0701b0 con 0x557a9c431830 2024-04-25T10:13:24.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8634023070 con 0x557a9c431830 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8626a00640 1 -- 172.21.15.165:0/3170695891 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f862c061000 con 0x557a9c431830 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f864b56c740 10 client.8322 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x557a9c03f0f0 con 0x557a9c431830 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.923+0000 7f8626a00640 1 -- 172.21.15.165:0/3170695891 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634023070 con 0x557a9c431830 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f8626a00640 1 -- 172.21.15.165:0/3170695891 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f862c088200 con 0x557a9c431830 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f864b56c740 10 client.8322 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f864b56c740 10 client.8322 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f864b56c740 20 client.8322 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f864b56c740 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4473c0 0x557a9c41e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f864b56c740 4 client.8322 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a9c3781b0 con 0x557a9c4473c0 2024-04-25T10:13:24.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.925+0000 7f8645c00640 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4473c0 0x557a9c41e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.926+0000 7f8645c00640 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4473c0 0x557a9c41e810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8fcbfd98f3f306f server_cookie=87aef800417e06b8 in_seq=0 out_seq=0 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.926+0000 7f8626a00640 10 client.8322 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f8626a00640 1 -- 172.21.15.165:0/3170695891 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557a9c3781b0 con 0x557a9c4473c0 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f8626a00640 10 client.8322 handle_command_reply: tid=0 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 client.8322 shutdown 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 msgr2=0x557a9c44f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 0x557a9c44f670 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f862c007410 tx=0x7f862c059130 comp rx=0 tx=0).stop 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 shutdown_connections 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4473c0 0x557a9c41e810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c443440 0x557a9c44fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c431830 0x557a9c44f670 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 --2- 172.21.15.165:0/3170695891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c406d70 0x557a9c41d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 >> 172.21.15.165:0/3170695891 conn(0x557a9c3cea30 msgr2=0x557a9c405610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f8626a00640 0 client.8322 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:24.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 shutdown_connections 2024-04-25T10:13:24.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.928+0000 7f864b56c740 1 -- 172.21.15.165:0/3170695891 wait complete. 2024-04-25T10:13:24.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.929+0000 7f864b56c740 20 client.8322 trim_cache size 0 max 16384 2024-04-25T10:13:24.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.943+0000 7f864b56c740 1 Processor -- start 2024-04-25T10:13:24.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.943+0000 7f864b56c740 1 -- start start 2024-04-25T10:13:24.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f864b56c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c442ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c42e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c431e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f864b56c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86402002c0 con 0x557a9c406d70 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401ff480 con 0x557a9c443440 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86401f7570 con 0x557a9c431830 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f8645c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c442ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f8645c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c442ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36100/0 (socket says 172.21.15.165:36100) 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f8645c00640 1 -- 172.21.15.165:0/3700218371 learned_addr learned my addr 172.21.15.165:0/3700218371 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.944+0000 7f8645c00640 1 -- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 msgr2=0x557a9c42e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645200640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c42e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8646600640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c431e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645c00640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c42e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645c00640 1 -- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 msgr2=0x557a9c431e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645c00640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c431e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645c00640 1 -- 172.21.15.165:0/3700218371 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9c403560 con 0x557a9c406d70 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645200640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c42e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8646600640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c431e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8645c00640 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c442ce0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f863400d590 tx=0x7f8634012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85934782fa2d096d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8631600640 1 -- 172.21.15.165:0/3700218371 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634037070 con 0x557a9c406d70 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8631600640 1 -- 172.21.15.165:0/3700218371 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634023040 con 0x557a9c406d70 2024-04-25T10:13:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.945+0000 7f8631600640 1 -- 172.21.15.165:0/3700218371 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634019040 con 0x557a9c406d70 2024-04-25T10:13:24.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 -- 172.21.15.165:0/3700218371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 msgr2=0x557a9c442ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c442ce0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f863400d590 tx=0x7f8634012000 comp rx=0 tx=0).stop 2024-04-25T10:13:24.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 -- 172.21.15.165:0/3700218371 shutdown_connections 2024-04-25T10:13:24.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c431e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c42e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 --2- 172.21.15.165:0/3700218371 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c442ce0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 -- 172.21.15.165:0/3700218371 >> 172.21.15.165:0/3700218371 conn(0x557a9c3cea30 msgr2=0x557a9c401200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 -- 172.21.15.165:0/3700218371 shutdown_connections 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.947+0000 7f864b56c740 1 -- 172.21.15.165:0/3700218371 wait complete. 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.948+0000 7f864b56c740 1 -- start start 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.948+0000 7f864b56c740 1 Processor -- start 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c4c31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c41ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c4bc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86401fee80 con 0x557a9c406d70 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c405a70 con 0x557a9c443440 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a9c442bf0 con 0x557a9c431830 2024-04-25T10:13:24.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f864b56c740 10 client.0 _ll_register_callbacks cb 0x7f862c00ce70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c4c31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c4c31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36106/0 (socket says 172.21.15.165:36106) 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 -- 172.21.15.165:0/2838438315 learned_addr learned my addr 172.21.15.165:0/2838438315 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 -- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 msgr2=0x557a9c41ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c41ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 -- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 msgr2=0x557a9c4bc1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c4bc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.949+0000 7f8645c00640 1 -- 172.21.15.165:0/2838438315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a9c41bfd0 con 0x557a9c406d70 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8645c00640 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c4c31d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f863400db60 tx=0x7f8634004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=465ec8ee58392d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8626a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8626a00640 1 -- 172.21.15.165:0/2838438315 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8634037040 con 0x557a9c406d70 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8634023040 con 0x557a9c406d70 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8626a00640 1 -- 172.21.15.165:0/2838438315 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8634032700 con 0x557a9c406d70 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8626a00640 1 -- 172.21.15.165:0/2838438315 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8634047050 con 0x557a9c406d70 2024-04-25T10:13:24.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f864b56c740 10 client.8328 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x557a9c445ff0 con 0x557a9c406d70 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8626a00640 1 -- 172.21.15.165:0/2838438315 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8634023040 con 0x557a9c406d70 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.950+0000 7f8626a00640 1 -- 172.21.15.165:0/2838438315 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f863404a020 con 0x557a9c406d70 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f864b56c740 10 client.8328 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f864b56c740 10 client.8328 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f864b56c740 20 client.8328 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f864b56c740 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4bc710 0x557a9c4bf4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f864b56c740 4 client.8328 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x557a9c372500 con 0x557a9c4bc710 2024-04-25T10:13:24.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.951+0000 7f8645200640 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4bc710 0x557a9c4bf4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.952+0000 7f8645200640 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4bc710 0x557a9c4bf4e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c856667a9f6dace5 server_cookie=a9199984faf82a17 in_seq=0 out_seq=0 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.952+0000 7f8626a00640 10 client.8328 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f8626a00640 1 -- 172.21.15.165:0/2838438315 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x557a9c372500 con 0x557a9c4bc710 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f8626a00640 10 client.8328 handle_command_reply: tid=0 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 client.8328 shutdown 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 msgr2=0x557a9c4c31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c4c31d0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f863400db60 tx=0x7f8634004730 comp rx=0 tx=0).stop 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 shutdown_connections 2024-04-25T10:13:24.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x557a9c4bc710 0x557a9c4bf4e0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557a9c443440 0x557a9c4bc1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557a9c431830 0x557a9c41ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 --2- 172.21.15.165:0/2838438315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557a9c406d70 0x557a9c4c31d0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f8626a00640 0 client.8328 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 >> 172.21.15.165:0/2838438315 conn(0x557a9c3cea30 msgr2=0x557a9c42e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 shutdown_connections 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 1 -- 172.21.15.165:0/2838438315 wait complete. 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.953+0000 7f864b56c740 20 client.8328 trim_cache size 0 max 16384 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:13:24.961 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:13:24.962 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:13:24.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f862006b4d0 msgr2=0x7f862006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f862006b4d0 0x7f862006d990 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f8640133db0 tx=0x7f8634008040 comp rx=0 tx=0).stop 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 msgr2=0x7f86401f3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 0x7f86401f3b30 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f862c00c7c0 tx=0x7f862c00cc80 comp rx=0 tx=0).stop 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 shutdown_connections 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f862006b4d0 0x7f862006d990 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8640145140 0x7f86400cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86400ce950 0x7f86401f3b30 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 --2- 172.21.15.165:0/2242371066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86400c5030 0x7f86400b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 >> 172.21.15.165:0/2242371066 conn(0x7f8640061cc0 msgr2=0x7f864015e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 shutdown_connections 2024-04-25T10:13:24.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:24.954+0000 7f8647000640 1 -- 172.21.15.165:0/2242371066 wait complete. 2024-04-25T10:13:24.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:24.974 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.7:0 scrub status 2024-04-25T10:13:25.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.158+0000 7fd4f3c00640 1 Processor -- start 2024-04-25T10:13:25.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.158+0000 7fd4f3c00640 1 -- start start 2024-04-25T10:13:25.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.160+0000 7fd4f3c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd4ec15b710 0x7fd4ec0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.160+0000 7fd4f3c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd4ec06ec50 0x7fd4ec15e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.160+0000 7fd4f3c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15ee60 0x7fd4ec0ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.160+0000 7fd4f3c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec15e490 con 0x7fd4ec15b710 2024-04-25T10:13:25.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.160+0000 7fd4f3c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ec06f0e0 con 0x7fd4ec06ec50 2024-04-25T10:13:25.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.160+0000 7fd4f3c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0cfd80 con 0x7fd4ec15ee60 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd4ec15b710 0x7fd4ec0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd4ec15b710 0x7fd4ec0cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36114/0 (socket says 172.21.15.165:36114) 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 -- 172.21.15.165:0/1989953477 learned_addr learned my addr 172.21.15.165:0/1989953477 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1989953477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15ee60 0x7fd4ec0ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4ebe00640 1 --2- 172.21.15.165:0/1989953477 >> v2:172.21.15.172:3300/0 conn(0x7fd4ec06ec50 0x7fd4ec15e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 -- 172.21.15.165:0/1989953477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15ee60 msgr2=0x7fd4ec0ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 --2- 172.21.15.165:0/1989953477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15ee60 0x7fd4ec0ac8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 -- 172.21.15.165:0/1989953477 >> v2:172.21.15.172:3300/0 conn(0x7fd4ec06ec50 msgr2=0x7fd4ec15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 --2- 172.21.15.165:0/1989953477 >> v2:172.21.15.172:3300/0 conn(0x7fd4ec06ec50 0x7fd4ec15e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.161+0000 7fd4f1200640 1 -- 172.21.15.165:0/1989953477 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0d0060 con 0x7fd4ec15b710 2024-04-25T10:13:25.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.162+0000 7fd4f1200640 1 --2- 172.21.15.165:0/1989953477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15b710 0x7fd4ec0cf640 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec06ba10 tx=0x7fd4dc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5de5468d778e918 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4eb400640 1 -- 172.21.15.165:0/1989953477 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc036070 con 0x7fd4ec15b710 2024-04-25T10:13:25.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4eb400640 1 -- 172.21.15.165:0/1989953477 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc031600 con 0x7fd4ec15b710 2024-04-25T10:13:25.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4eb400640 1 -- 172.21.15.165:0/1989953477 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc0319d0 con 0x7fd4ec15b710 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1989953477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15b710 msgr2=0x7fd4ec0cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1989953477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15b710 0x7fd4ec0cf640 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec06ba10 tx=0x7fd4dc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1989953477 shutdown_connections 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1989953477 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15ee60 0x7fd4ec0ac8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1989953477 >> v2:172.21.15.172:3300/0 conn(0x7fd4ec06ec50 0x7fd4ec15e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1989953477 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15b710 0x7fd4ec0cf640 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.163+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1989953477 >> 172.21.15.165:0/1989953477 conn(0x7fd4ec06abe0 msgr2=0x7fd4ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.164+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1989953477 shutdown_connections 2024-04-25T10:13:25.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.164+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1989953477 wait complete. 2024-04-25T10:13:25.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.165+0000 7fd4f3c00640 1 Processor -- start 2024-04-25T10:13:25.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.165+0000 7fd4f3c00640 1 -- start start 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.166+0000 7fd4f3c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4ec06ec50 0x7fd4ec0b2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.166+0000 7fd4f3c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15b710 0x7fd4ec14f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.166+0000 7fd4f3c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 0x7fd4ec0b4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.166+0000 7fd4f3c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec15bc70 con 0x7fd4ec15ee60 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.166+0000 7fd4f3c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec15baf0 con 0x7fd4ec06ec50 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.166+0000 7fd4f3c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0acdf0 con 0x7fd4ec15b710 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 0x7fd4ec0b4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 0x7fd4ec0b4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36122/0 (socket says 172.21.15.165:36122) 2024-04-25T10:13:25.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1416460916 learned_addr learned my addr 172.21.15.165:0/1416460916 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1416460916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15b710 msgr2=0x7fd4ec14f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15b710 0x7fd4ec14f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1416460916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4ec06ec50 msgr2=0x7fd4ec0b2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4ec06ec50 0x7fd4ec0b2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1416460916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ec0b2b30 con 0x7fd4ec15ee60 2024-04-25T10:13:25.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.167+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 0x7fd4ec0b4080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e7f0 tx=0x7fd4e400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.168+0000 7fd4e8c00640 1 -- 172.21.15.165:0/1416460916 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4017070 con 0x7fd4ec15ee60 2024-04-25T10:13:25.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.168+0000 7fd4e8c00640 1 -- 172.21.15.165:0/1416460916 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e400ee40 con 0x7fd4ec15ee60 2024-04-25T10:13:25.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.168+0000 7fd4e8c00640 1 -- 172.21.15.165:0/1416460916 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cb80 con 0x7fd4ec15ee60 2024-04-25T10:13:25.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.168+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ec0b2e10 con 0x7fd4ec15ee60 2024-04-25T10:13:25.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.168+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ec0b33d0 con 0x7fd4ec15ee60 2024-04-25T10:13:25.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.171+0000 7fd4e8c00640 1 -- 172.21.15.165:0/1416460916 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4e401a070 con 0x7fd4ec15ee60 2024-04-25T10:13:25.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.172+0000 7fd4e8c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4cc06b500 0x7fd4cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.172+0000 7fd4e8c00640 1 -- 172.21.15.165:0/1416460916 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd4e408a510 con 0x7fd4ec15ee60 2024-04-25T10:13:25.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.172+0000 7fd4f69a2740 1 Processor -- start 2024-04-25T10:13:25.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.172+0000 7fd4f1200640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4cc06b500 0x7fd4cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.172+0000 7fd4f69a2740 1 -- start start 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f1200640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4cc06b500 0x7fd4cc06d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc033000 tx=0x7fd4dc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb1c8d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 0x5589fb1c6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1f1670 0x5589fb1f1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fb12ac70 con 0x5589fb1c6bb0 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fb0d3cf0 con 0x5589fb1f1670 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.173+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589fb126d10 con 0x5589fb203280 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb1c8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb1c8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36138/0 (socket says 172.21.15.165:36138) 2024-04-25T10:13:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 -- 172.21.15.165:0/57312987 learned_addr learned my addr 172.21.15.165:0/57312987 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 -- 172.21.15.165:0/57312987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 msgr2=0x5589fb1c6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 0x5589fb1c6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 -- 172.21.15.165:0/57312987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1f1670 msgr2=0x5589fb1f1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1f1670 0x5589fb1f1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 -- 172.21.15.165:0/57312987 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589faf097d0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.174+0000 7fd4f1200640 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb1c8d10 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc02ac90 tx=0x7fd4dc02f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f726e526596b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.175+0000 7fd4e1600640 1 -- 172.21.15.165:0/57312987 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc03b6c0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.175+0000 7fd4e1600640 1 -- 172.21.15.165:0/57312987 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc04c070 con 0x5589fb1c6bb0 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.175+0000 7fd4e1600640 1 -- 172.21.15.165:0/57312987 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc03baa0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.175+0000 7fd4f69a2740 1 -- 172.21.15.165:0/57312987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 msgr2=0x5589fb1c8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.175+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb1c8d10 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc02ac90 tx=0x7fd4dc02f000 comp rx=0 tx=0).stop 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 -- 172.21.15.165:0/57312987 shutdown_connections 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1f1670 0x5589fb1f1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 0x5589fb1c6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/57312987 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb1c8d10 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 -- 172.21.15.165:0/57312987 >> 172.21.15.165:0/57312987 conn(0x5589fb18ea30 msgr2=0x5589fb20c670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 -- 172.21.15.165:0/57312987 shutdown_connections 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 -- 172.21.15.165:0/57312987 wait complete. 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.176+0000 7fd4f69a2740 1 -- start start 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.177+0000 7fd4f69a2740 1 Processor -- start 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb20f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 0x5589fb203680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb203280 0x5589fb20d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fb0c93a0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fb1ef1c0 con 0x5589fb203280 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589fb20e110 con 0x5589fb1f1670 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f69a2740 10 client.0 _ll_register_callbacks cb 0x5589fb0e6b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb20f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb20f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36146/0 (socket says 172.21.15.165:36146) 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 -- 172.21.15.165:0/2054979579 learned_addr learned my addr 172.21.15.165:0/2054979579 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 -- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 msgr2=0x5589fb203680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 0x5589fb203680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 -- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb203280 msgr2=0x5589fb20d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb203280 0x5589fb20d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.178+0000 7fd4f1200640 1 -- 172.21.15.165:0/2054979579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589fae3f3f0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4f1200640 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb20f210 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009f40 tx=0x7fd4dc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bdebcd009036f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4d6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4d6000640 1 -- 172.21.15.165:0/2054979579 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc004450 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4d6000640 1 -- 172.21.15.165:0/2054979579 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd4dc05cc80 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4dc04c070 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4d6000640 1 -- 172.21.15.165:0/2054979579 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4dc0045f0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4f69a2740 10 client.8340 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.179+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5589fadff0f0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4d6000640 1 -- 172.21.15.165:0/2054979579 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc04c070 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4d6000640 1 -- 172.21.15.165:0/2054979579 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4dc008040 con 0x5589fb1c6bb0 2024-04-25T10:13:25.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4f69a2740 10 client.8340 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4f69a2740 10 client.8340 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4f69a2740 20 client.8340 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb205f40 0x5589fb1f38f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4f69a2740 4 client.8340 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5589fb1381b0 con 0x5589fb205f40 2024-04-25T10:13:25.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.180+0000 7fd4ebe00640 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb205f40 0x5589fb1f38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.182+0000 7fd4ebe00640 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb205f40 0x5589fb1f38f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f875fd6532ca00d6 server_cookie=4d112e5c11b1d03e in_seq=0 out_seq=0 2024-04-25T10:13:25.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.182+0000 7fd4d6000640 10 client.8340 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:25.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4d6000640 1 -- 172.21.15.165:0/2054979579 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5589fb1381b0 con 0x5589fb205f40 2024-04-25T10:13:25.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4d6000640 10 client.8340 handle_command_reply: tid=0 2024-04-25T10:13:25.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 client.8340 shutdown 2024-04-25T10:13:25.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 msgr2=0x5589fb20f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb20f210 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009f40 tx=0x7fd4dc004ab0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 shutdown_connections 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb205f40 0x5589fb1f38f0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb203280 0x5589fb20d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 0x5589fb203680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/2054979579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1c6bb0 0x5589fb20f210 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4d6000640 0 client.8340 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.183+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 >> 172.21.15.165:0/2054979579 conn(0x5589fb18ea30 msgr2=0x5589fb1ec7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.184+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 shutdown_connections 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.184+0000 7fd4f69a2740 1 -- 172.21.15.165:0/2054979579 wait complete. 2024-04-25T10:13:25.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.184+0000 7fd4f69a2740 20 client.8340 trim_cache size 0 max 16384 2024-04-25T10:13:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.200+0000 7fd4f69a2740 1 Processor -- start 2024-04-25T10:13:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.200+0000 7fd4f69a2740 1 -- start start 2024-04-25T10:13:25.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 0x5589fb1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 0x5589fb1ee900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec200640 con 0x5589fb203280 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec1ff870 con 0x5589fb1c6bb0 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec0b0750 con 0x5589fb1f1670 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 0x5589fb1ee900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 0x5589fb1ee900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36162/0 (socket says 172.21.15.165:36162) 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.201+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1251661272 learned_addr learned my addr 172.21.15.165:0/1251661272 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1251661272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 msgr2=0x5589fb1f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1251661272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 msgr2=0x5589fb1f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 0x5589fb1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4f1c00640 1 -- 172.21.15.165:0/1251661272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5589fb1c67c0 con 0x5589fb203280 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4f1c00640 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 0x5589fb1ee900 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4058860 tx=0x7fd4e4059260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f797527e80f8642b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4e0c00640 1 -- 172.21.15.165:0/1251661272 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4021070 con 0x5589fb203280 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4e0c00640 1 -- 172.21.15.165:0/1251661272 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4061270 con 0x5589fb203280 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.202+0000 7fd4e0c00640 1 -- 172.21.15.165:0/1251661272 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cd20 con 0x5589fb203280 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 -- 172.21.15.165:0/1251661272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 msgr2=0x5589fb1ee900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 0x5589fb1ee900 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fd4e4058860 tx=0x7fd4e4059260 comp rx=0 tx=0).stop 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 -- 172.21.15.165:0/1251661272 shutdown_connections 2024-04-25T10:13:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb203280 0x5589fb1ee900 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/1251661272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 0x5589fb1f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 -- 172.21.15.165:0/1251661272 >> 172.21.15.165:0/1251661272 conn(0x5589fb18ea30 msgr2=0x5589fb2047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 -- 172.21.15.165:0/1251661272 shutdown_connections 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.203+0000 7fd4f69a2740 1 -- 172.21.15.165:0/1251661272 wait complete. 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.204+0000 7fd4f69a2740 1 -- start start 2024-04-25T10:13:25.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.204+0000 7fd4f69a2740 1 Processor -- start 2024-04-25T10:13:25.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 0x5589fb1f3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 0x5589fb1ed880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ec1ff270 con 0x5589fb1f1670 2024-04-25T10:13:25.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5589fb1f51e0 con 0x5589fb1c6bb0 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5589fb226800 con 0x5589fb203280 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4f69a2740 10 client.0 _ll_register_callbacks cb 0x5589fb210be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36174/0 (socket says 172.21.15.165:36174) 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 -- 172.21.15.165:0/3437048427 learned_addr learned my addr 172.21.15.165:0/3437048427 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 -- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 msgr2=0x5589fb1ed880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 0x5589fb1ed880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 -- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 msgr2=0x5589fb1f3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 0x5589fb1f3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.205+0000 7fd4ebe00640 1 -- 172.21.15.165:0/3437048427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5589fb264dd0 con 0x5589fb1f1670 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4ebe00640 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec06b5f0 tx=0x7fd4d8015780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765d0fe8560c545d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4d6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4d6000640 1 -- 172.21.15.165:0/3437048427 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d8015e40 con 0x5589fb1f1670 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4d6000640 1 -- 172.21.15.165:0/3437048427 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd4d8004030 con 0x5589fb1f1670 2024-04-25T10:13:25.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4e4061270 con 0x5589fb1f1670 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4d6000640 1 -- 172.21.15.165:0/3437048427 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4d8004230 con 0x5589fb1f1670 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4f69a2740 10 client.8346 fetch_fsmap learned FSMap version 74 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=2+}) v3 -- 0x5589fb27f870 con 0x5589fb1f1670 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.206+0000 7fd4d6000640 1 -- 172.21.15.165:0/3437048427 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4e4061270 con 0x5589fb1f1670 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4d6000640 1 -- 172.21.15.165:0/3437048427 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 74) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd4d80180e0 con 0x5589fb1f1670 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f69a2740 10 client.8346 fetch_fsmap finished waiting for FSMap version 74 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f69a2740 10 client.8346 resolve_mds: resolved 7:0 to role '7:0' aka daemon mds.d 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f69a2740 20 client.8346 populate_metadata read hostname 'smithi165' 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb20f020 0x5589fb1eddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f69a2740 4 client.8346 mds_command: new command op to 8043 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 --> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5589fb132500 con 0x5589fb20f020 2024-04-25T10:13:25.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.207+0000 7fd4f1200640 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb20f020 0x5589fb1eddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.208+0000 7fd4f1200640 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb20f020 0x5589fb1eddc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=47a361d2504a0cc8 server_cookie=be24ef115d05dfca in_seq=0 out_seq=0 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.208+0000 7fd4d6000640 10 client.8346 ms_handle_connect on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4d6000640 1 -- 172.21.15.165:0/3437048427 <== mds.0 v2:172.21.15.172:6833/437969658 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5589fb132500 con 0x5589fb20f020 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4d6000640 10 client.8346 handle_command_reply: tid=0 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 client.8346 shutdown 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 msgr2=0x5589fb1f3fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fd4ec06b5f0 tx=0x7fd4d8015780 comp rx=0 tx=0).stop 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 shutdown_connections 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:6833/437969658,v1:172.21.15.172:6835/437969658] conn(0x5589fb20f020 0x5589fb1eddc0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5589fb203280 0x5589fb1ed880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5589fb1f1670 0x5589fb1f3fe0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 --2- 172.21.15.165:0/3437048427 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5589fb1c6bb0 0x5589fb1f3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 >> 172.21.15.165:0/3437048427 conn(0x5589fb18ea30 msgr2=0x5589fb2044d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4d6000640 0 client.8346 ms_handle_reset on v2:172.21.15.172:6833/437969658 2024-04-25T10:13:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 shutdown_connections 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.209+0000 7fd4f69a2740 1 -- 172.21.15.165:0/3437048427 wait complete. 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f69a2740 20 client.8346 trim_cache size 0 max 16384 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:13:25.216 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4cc06b500 msgr2=0x7fd4cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4cc06b500 0x7fd4cc06d9c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc033000 tx=0x7fd4dc02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 msgr2=0x7fd4ec0b4080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 0x7fd4ec0b4080 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400e7f0 tx=0x7fd4e400ebf0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 shutdown_connections 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4cc06b500 0x7fd4cc06d9c0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4ec15ee60 0x7fd4ec0b4080 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4ec15b710 0x7fd4ec14f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 --2- 172.21.15.165:0/1416460916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4ec06ec50 0x7fd4ec0b2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 >> 172.21.15.165:0/1416460916 conn(0x7fd4ec06abe0 msgr2=0x7fd4ec0cf240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 shutdown_connections 2024-04-25T10:13:25.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.210+0000 7fd4f3c00640 1 -- 172.21.15.165:0/1416460916 wait complete. 2024-04-25T10:13:25.230 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:25.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.417+0000 7f0cff200640 1 Processor -- start 2024-04-25T10:13:25.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.417+0000 7f0cff200640 1 -- start start 2024-04-25T10:13:25.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.418+0000 7f0cff200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 0x7f0d000cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.418+0000 7f0cff200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d00161df0 con 0x7f0d0015db90 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.418+0000 7f0cff200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d001620d0 con 0x7f0d000d04f0 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.418+0000 7f0cff200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000d0980 con 0x7f0d000b93c0 2024-04-25T10:13:25.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.418+0000 7f0cfde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0d0015db90 0x7f0d001615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36716/0 (socket says 172.21.15.165:36716) 2024-04-25T10:13:25.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.418+0000 7f0cfde00640 1 -- 172.21.15.165:0/3269244007 learned_addr learned my addr 172.21.15.165:0/3269244007 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:25.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353193095 0 0) 0x7f0d00161df0 con 0x7f0d0015db90 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ce4003520 con 0x7f0d0015db90 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfd400640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 0x7f0d000cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163304925 0 0) 0x7f0ce4003520 con 0x7f0d0015db90 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 msgr2=0x7f0d000cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 0x7f0d000cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 >> v1:172.21.15.172:6789/0 conn(0x7f0d000d04f0 legacy=0x7f0d000d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d001623b0 con 0x7f0d0015db90 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f0cf0001670 con 0x7f0d0015db90 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 >> v1:172.21.15.165:6789/0 conn(0x7f0d0015db90 legacy=0x7f0d001615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ce40039e0 0x7f0ce4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ce4004800 0x7f0ce4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 0x7f0ce4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000d0980 con 0x7f0ce40039e0 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce400baf0 con 0x7f0ce4004800 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.419+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce400bcf0 con 0x7f0d000b93c0 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfde00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ce40039e0 0x7f0ce4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfe800640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 0x7f0ce4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfde00640 1 -- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 msgr2=0x7f0ce4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfde00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d000b93c0 0x7f0ce4005200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfde00640 1 -- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ce4004800 msgr2=0x7f0ce4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfde00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ce4004800 0x7f0ce4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.420+0000 7f0cfde00640 1 -- 172.21.15.165:0/3269244007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce4003520 con 0x7f0ce40039e0 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.421+0000 7f0cfde00640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ce40039e0 0x7f0ce4003e00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c210 tx=0x7f0ce4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f72ca7b74cbf3138 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.421+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0007d30 con 0x7f0ce40039e0 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.421+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0005ad0 con 0x7f0ce40039e0 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.421+0000 7f0cfca00640 1 -- 172.21.15.165:0/3269244007 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0007070 con 0x7f0ce40039e0 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cff200640 1 -- 172.21.15.165:0/3269244007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ce40039e0 msgr2=0x7f0ce4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cff200640 1 --2- 172.21.15.165:0/3269244007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ce40039e0 0x7f0ce4003e00 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000c210 tx=0x7f0ce4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cfe800640 1 -- 172.21.15.165:0/3269244007 reap_dead start 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cff200640 1 -- 172.21.15.165:0/3269244007 shutdown_connections 2024-04-25T10:13:25.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cff200640 1 -- 172.21.15.165:0/3269244007 >> 172.21.15.165:0/3269244007 conn(0x7f0d000b52b0 msgr2=0x7f0d000b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cff200640 1 -- 172.21.15.165:0/3269244007 shutdown_connections 2024-04-25T10:13:25.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.422+0000 7f0cff200640 1 -- 172.21.15.165:0/3269244007 wait complete. 2024-04-25T10:13:25.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.424+0000 7f0cff200640 1 Processor -- start 2024-04-25T10:13:25.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.424+0000 7f0cff200640 1 -- start start 2024-04-25T10:13:25.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cff200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ce40039e0 0x7f0d0014ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cff200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d000ccb40 0x7f0d001e8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cff200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 0x7f0d001e9480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cff200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b68d0 con 0x7f0d000ccfc0 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cff200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000d1f50 con 0x7f0d000ccb40 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cff200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d000b6a50 con 0x7f0ce40039e0 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cfe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 0x7f0d001e9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cfe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 0x7f0d001e9480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36190/0 (socket says 172.21.15.165:36190) 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.425+0000 7f0cfe800640 1 -- 172.21.15.165:0/2573917829 learned_addr learned my addr 172.21.15.165:0/2573917829 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfde00640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ce40039e0 0x7f0d0014ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfe800640 1 -- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ce40039e0 msgr2=0x7f0d0014ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfe800640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ce40039e0 0x7f0d0014ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfd400640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d000ccb40 0x7f0d001e8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfe800640 1 -- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d000ccb40 msgr2=0x7f0d001e8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfe800640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d000ccb40 0x7f0d001e8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfe800640 1 -- 172.21.15.165:0/2573917829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000cb230 con 0x7f0d000ccfc0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfd400640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d000ccb40 0x7f0d001e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cfe800640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 0x7f0d001e9480 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f0d000b8e80 tx=0x7f0cf800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.426+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8018070 con 0x7f0d000ccfc0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.427+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf8003b70 con 0x7f0d000ccfc0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.427+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf8005070 con 0x7f0d000ccfc0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.427+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d000cb4d0 con 0x7f0d000ccfc0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.427+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d000cba90 con 0x7f0d000ccfc0 2024-04-25T10:13:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.428+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d00161df0 con 0x7f0d000ccfc0 2024-04-25T10:13:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.429+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0cf801b070 con 0x7f0d000ccfc0 2024-04-25T10:13:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.430+0000 7f0cf6000640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0cdc06b500 0x7f0cdc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.430+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0cf80898a0 con 0x7f0d000ccfc0 2024-04-25T10:13:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.430+0000 7f0cfde00640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0cdc06b500 0x7f0cdc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.431+0000 7f0cfde00640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0cdc06b500 0x7f0cdc06d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0033000 tx=0x7f0cf002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.435+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d00161df0 con 0x7f0d000ccfc0 2024-04-25T10:13:25.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.671+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d00073990 con 0x7f0d000ccfc0 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.672+0000 7f0cf6000640 1 -- 172.21.15.165:0/2573917829 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f0cf8058620 con 0x7f0d000ccfc0 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:25.677 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.675+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0cdc06b500 msgr2=0x7f0cdc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.675+0000 7f0cff200640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0cdc06b500 0x7f0cdc06d9c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0cf0033000 tx=0x7f0cf002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.675+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 msgr2=0x7f0d001e9480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.675+0000 7f0cff200640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 0x7f0d001e9480 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f0d000b8e80 tx=0x7f0cf800dc00 comp rx=0 tx=0).stop 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 shutdown_connections 2024-04-25T10:13:25.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0cdc06b500 0x7f0cdc06d9c0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d000ccfc0 0x7f0d001e9480 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d000ccb40 0x7f0d001e8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 --2- 172.21.15.165:0/2573917829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ce40039e0 0x7f0d0014ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 >> 172.21.15.165:0/2573917829 conn(0x7f0d000b52b0 msgr2=0x7f0d000b8300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 shutdown_connections 2024-04-25T10:13:25.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.676+0000 7f0cff200640 1 -- 172.21.15.165:0/2573917829 wait complete. 2024-04-25T10:13:25.698 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:25.698 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:25.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.884+0000 7fc31de00640 1 Processor -- start 2024-04-25T10:13:25.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.884+0000 7fc31de00640 1 -- start start 2024-04-25T10:13:25.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.885+0000 7fc31de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 0x7fc318070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc31de00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31815bf30 con 0x7fc3180cda30 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc31de00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc31815c210 con 0x7fc318071530 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc31de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc31815c860 con 0x7fc318144440 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc31d400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc3180cda30 0x7fc31815ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36732/0 (socket says 172.21.15.165:36732) 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc31d400640 1 -- 172.21.15.165:0/3509701281 learned_addr learned my addr 172.21.15.165:0/3509701281 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 0x7fc318070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195723554 0 0) 0x7fc31815bf30 con 0x7fc3180cda30 2024-04-25T10:13:25.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.886+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc300003520 con 0x7fc3180cda30 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3894075588 0 0) 0x7fc300003520 con 0x7fc3180cda30 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 msgr2=0x7fc318070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 0x7fc318070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 >> v1:172.21.15.172:6789/0 conn(0x7fc318071530 legacy=0x7fc3180cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31815c4f0 con 0x7fc3180cda30 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc3100019f0 con 0x7fc3180cda30 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 >> v1:172.21.15.165:6789/0 conn(0x7fc3180cda30 legacy=0x7fc31815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc300003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 0x7fc300004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300004d40 0x7fc3000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc31815c860 con 0x7fc3000039e0 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc30000fff0 con 0x7fc300004d40 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3000101c0 con 0x7fc318144440 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.887+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc300003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc31d400640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300004d40 0x7fc3000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc317e00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 0x7fc300004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc31ca00640 1 -- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 msgr2=0x7fc300004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc318144440 0x7fc300004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc31ca00640 1 -- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300004d40 msgr2=0x7fc3000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc300004d40 0x7fc3000051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.888+0000 7fc31ca00640 1 -- 172.21.15.165:0/3509701281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc300003520 con 0x7fc3000039e0 2024-04-25T10:13:25.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.889+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc300003e00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00ba60 tx=0x7fc300005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26d720594fe8a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c030070 con 0x7fc3000039e0 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c02ae10 con 0x7fc3000039e0 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc317400640 1 -- 172.21.15.165:0/3509701281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00b600 con 0x7fc3000039e0 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31de00640 1 -- 172.21.15.165:0/3509701281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 msgr2=0x7fc300003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31de00640 1 --2- 172.21.15.165:0/3509701281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc300003e00 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00ba60 tx=0x7fc300005830 comp rx=0 tx=0).stop 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31d400640 1 -- 172.21.15.165:0/3509701281 reap_dead start 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31de00640 1 -- 172.21.15.165:0/3509701281 shutdown_connections 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31de00640 1 -- 172.21.15.165:0/3509701281 >> 172.21.15.165:0/3509701281 conn(0x7fc31806abe0 msgr2=0x7fc31806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31de00640 1 -- 172.21.15.165:0/3509701281 shutdown_connections 2024-04-25T10:13:25.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.890+0000 7fc31de00640 1 -- 172.21.15.165:0/3509701281 wait complete. 2024-04-25T10:13:25.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.892+0000 7fc31de00640 1 Processor -- start 2024-04-25T10:13:25.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.892+0000 7fc31de00640 1 -- start start 2024-04-25T10:13:25.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc31813add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31de00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3181e2b60 0x7fc3181588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31de00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31813cd90 0x7fc31813d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31de00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3180cdea0 con 0x7fc3000039e0 2024-04-25T10:13:25.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31de00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc318071970 con 0x7fc31813cd90 2024-04-25T10:13:25.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31de00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc318070ff0 con 0x7fc3181e2b60 2024-04-25T10:13:25.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.893+0000 7fc31ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc31813add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc317e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3181e2b60 0x7fc3181588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc31813add0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36212/0 (socket says 172.21.15.165:36212) 2024-04-25T10:13:25.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 -- 172.21.15.165:0/3338879593 learned_addr learned my addr 172.21.15.165:0/3338879593 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:25.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31d400640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31813cd90 0x7fc31813d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 -- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3181e2b60 msgr2=0x7fc3181588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3181e2b60 0x7fc3181588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 -- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31813cd90 msgr2=0x7fc31813d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31813cd90 0x7fc31813d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 -- 172.21.15.165:0/3338879593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3181591b0 con 0x7fc3000039e0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.894+0000 7fc31ca00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc31813add0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00ba60 tx=0x7fc30c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.895+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c043020 con 0x7fc3000039e0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.895+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc30c03f680 con 0x7fc3000039e0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.895+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc30c00b600 con 0x7fc3000039e0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.895+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc318159490 con 0x7fc3000039e0 2024-04-25T10:13:25.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.895+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc318159a50 con 0x7fc3000039e0 2024-04-25T10:13:25.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.896+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3180b2040 con 0x7fc3000039e0 2024-04-25T10:13:25.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.897+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc30c046020 con 0x7fc3000039e0 2024-04-25T10:13:25.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.898+0000 7fc314c00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:25.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.898+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc30c0aad90 con 0x7fc3000039e0 2024-04-25T10:13:25.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.898+0000 7fc317e00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:25.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.898+0000 7fc317e00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc3181567a0 tx=0x7fc308009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:25.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:25.902+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3180b2040 con 0x7fc3000039e0 2024-04-25T10:13:26.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.173+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc318073990 con 0x7fc3000039e0 2024-04-25T10:13:26.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.174+0000 7fc314c00640 1 -- 172.21.15.165:0/3338879593 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fc30c04d030 con 0x7fc3000039e0 2024-04-25T10:13:26.178 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:26.178 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":707,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580987,"bytes_used":2601459712,"bytes_avail":770492653568,"bytes_total":773094113280,"write_bytes_sec":21100689,"read_op_per_sec":0,"write_op_per_sec":21},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:13:24.361852+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2fc06b4d0 msgr2=0x7fc2fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc3181567a0 tx=0x7fc308009040 comp rx=0 tx=0).stop 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 msgr2=0x7fc31813add0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc31813add0 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fc30c00ba60 tx=0x7fc30c004730 comp rx=0 tx=0).stop 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 shutdown_connections 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2fc06b4d0 0x7fc2fc06d990 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc31813cd90 0x7fc31813d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3181e2b60 0x7fc3181588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 --2- 172.21.15.165:0/3338879593 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3000039e0 0x7fc31813add0 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.176+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 >> 172.21.15.165:0/3338879593 conn(0x7fc31806abe0 msgr2=0x7fc318141630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:26.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.177+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 shutdown_connections 2024-04-25T10:13:26.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:26.177+0000 7fc31de00640 1 -- 172.21.15.165:0/3338879593 wait complete. 2024-04-25T10:13:26.198 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:13:27.200 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:27.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.407+0000 7f7a97a00640 1 Processor -- start 2024-04-25T10:13:27.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.407+0000 7f7a97a00640 1 -- start start 2024-04-25T10:13:27.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.408+0000 7f7a97a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7a90154ee0 0x7f7a900cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a97a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a900ce590 0x7f7a9006d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a97a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a90071480 con 0x7f7a900cfcf0 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a97a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a90072090 con 0x7f7a90154ee0 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a97a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90071230 con 0x7f7a900ce590 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a95000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7a90154ee0 0x7f7a900cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a8fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7a900cfcf0 0x7f7a900cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34162/0 (socket says 172.21.15.165:34162) 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a8fe00640 1 -- 172.21.15.165:0/2494479312 learned_addr learned my addr 172.21.15.165:0/2494479312 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a95000640 1 -- 172.21.15.165:0/2494479312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a900ce590 msgr2=0x7f7a9006d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a95000640 1 --2- 172.21.15.165:0/2494479312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a900ce590 0x7f7a9006d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a95000640 1 -- 172.21.15.165:0/2494479312 >> v1:172.21.15.165:6789/0 conn(0x7f7a900cfcf0 legacy=0x7f7a900cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a95000640 1 -- 172.21.15.165:0/2494479312 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90071760 con 0x7f7a90154ee0 2024-04-25T10:13:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.409+0000 7f7a8f400640 1 -- 172.21.15.165:0/2494479312 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1604399681 0 0) 0x7f7a90071480 con 0x7f7a900cfcf0 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.410+0000 7f7a95000640 1 --2- 172.21.15.165:0/2494479312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a90154ee0 0x7f7a900cf7b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097a0 tx=0x7f7a8002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a80ce4552e0810 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a8f400640 1 -- 172.21.15.165:0/2494479312 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80037070 con 0x7f7a90154ee0 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a8f400640 1 -- 172.21.15.165:0/2494479312 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a80032440 con 0x7f7a90154ee0 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a8f400640 1 -- 172.21.15.165:0/2494479312 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80032810 con 0x7f7a90154ee0 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a97a00640 1 -- 172.21.15.165:0/2494479312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a90154ee0 msgr2=0x7f7a900cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a97a00640 1 --2- 172.21.15.165:0/2494479312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a90154ee0 0x7f7a900cf7b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7a800097a0 tx=0x7f7a8002a180 comp rx=0 tx=0).stop 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a97a00640 1 -- 172.21.15.165:0/2494479312 shutdown_connections 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.411+0000 7f7a97a00640 1 --2- 172.21.15.165:0/2494479312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a900ce590 0x7f7a9006d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.412+0000 7f7a97a00640 1 --2- 172.21.15.165:0/2494479312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a90154ee0 0x7f7a900cf7b0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.412+0000 7f7a97a00640 1 -- 172.21.15.165:0/2494479312 >> 172.21.15.165:0/2494479312 conn(0x7f7a9006abe0 msgr2=0x7f7a9006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.412+0000 7f7a97a00640 1 -- 172.21.15.165:0/2494479312 shutdown_connections 2024-04-25T10:13:27.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.412+0000 7f7a97a00640 1 -- 172.21.15.165:0/2494479312 wait complete. 2024-04-25T10:13:27.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.413+0000 7f7a97a00640 1 Processor -- start 2024-04-25T10:13:27.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.414+0000 7f7a97a00640 1 -- start start 2024-04-25T10:13:27.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.414+0000 7f7a97a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a900ce590 0x7f7a901400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.414+0000 7f7a97a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 0x7f7a901f3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a97a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a90154ee0 0x7f7a900b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a97a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a901552c0 con 0x7f7a900cfcf0 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a97a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90071d20 con 0x7f7a900ce590 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a97a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a90155440 con 0x7f7a90154ee0 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 0x7f7a901f3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 0x7f7a901f3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56644/0 (socket says 172.21.15.165:56644) 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 -- 172.21.15.165:0/4168150385 learned_addr learned my addr 172.21.15.165:0/4168150385 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 -- 172.21.15.165:0/4168150385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a90154ee0 msgr2=0x7f7a900b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a90154ee0 0x7f7a900b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 -- 172.21.15.165:0/4168150385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a900ce590 msgr2=0x7f7a901400a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a900ce590 0x7f7a901400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.415+0000 7f7a8fe00640 1 -- 172.21.15.165:0/4168150385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a90151a70 con 0x7f7a900cfcf0 2024-04-25T10:13:27.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.416+0000 7f7a8fe00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 0x7f7a901f3d90 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00c8c0 tx=0x7f7a7c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.416+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c018070 con 0x7f7a900cfcf0 2024-04-25T10:13:27.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.416+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c00a640 con 0x7f7a900cfcf0 2024-04-25T10:13:27.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.416+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a900b3260 con 0x7f7a900cfcf0 2024-04-25T10:13:27.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.416+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c00aa30 con 0x7f7a900cfcf0 2024-04-25T10:13:27.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.416+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a900b3820 con 0x7f7a900cfcf0 2024-04-25T10:13:27.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.417+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a90073990 con 0x7f7a900cfcf0 2024-04-25T10:13:27.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.418+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a7c01b070 con 0x7f7a900cfcf0 2024-04-25T10:13:27.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.419+0000 7f7a8cc00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a7006b4d0 0x7f7a7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.419+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f7a7c089b10 con 0x7f7a900cfcf0 2024-04-25T10:13:27.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.419+0000 7f7a95000640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a7006b4d0 0x7f7a7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.420+0000 7f7a95000640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a7006b4d0 0x7f7a7006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7a80034000 tx=0x7f7a8002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.424+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a7c058730 con 0x7f7a900cfcf0 2024-04-25T10:13:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.656+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a901538d0 con 0x7f7a900cfcf0 2024-04-25T10:13:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.658+0000 7f7a8cc00640 1 -- 172.21.15.165:0/4168150385 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f7a7c058030 con 0x7f7a900cfcf0 2024-04-25T10:13:27.662 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:27.662 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a7006b4d0 msgr2=0x7f7a7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a7006b4d0 0x7f7a7006d990 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f7a80034000 tx=0x7f7a8002c000 comp rx=0 tx=0).stop 2024-04-25T10:13:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 msgr2=0x7f7a901f3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 0x7f7a901f3d90 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00c8c0 tx=0x7f7a7c00cd80 comp rx=0 tx=0).stop 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 shutdown_connections 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a7006b4d0 0x7f7a7006d990 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a90154ee0 0x7f7a900b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a900cfcf0 0x7f7a901f3d90 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 --2- 172.21.15.165:0/4168150385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a900ce590 0x7f7a901400a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.660+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 >> 172.21.15.165:0/4168150385 conn(0x7f7a9006abe0 msgr2=0x7f7a9015c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.661+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 shutdown_connections 2024-04-25T10:13:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.661+0000 7f7a97a00640 1 -- 172.21.15.165:0/4168150385 wait complete. 2024-04-25T10:13:27.684 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:27.684 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:27.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.882+0000 7fefd1800640 1 Processor -- start 2024-04-25T10:13:27.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.882+0000 7fefd1800640 1 -- start start 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefd1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc0ce330 0x7fefcc06fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefd1800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefcc0700a0 0x7fefcc15d370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefd1800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefcc1626e0 con 0x7fefcc06abe0 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefd1800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefcc0cd270 con 0x7fefcc0700a0 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefd1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc0cd550 con 0x7fefcc0ce330 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefcb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fefcc0700a0 0x7fefcc15d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefcaa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fefcc06abe0 0x7fefcc06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34174/0 (socket says 172.21.15.165:34174) 2024-04-25T10:13:27.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefcaa00640 1 -- 172.21.15.165:0/3202753626 learned_addr learned my addr 172.21.15.165:0/3202753626 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:27.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefca000640 1 --2- 172.21.15.165:0/3202753626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc0ce330 0x7fefcc06fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefcb400640 1 -- 172.21.15.165:0/3202753626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc0ce330 msgr2=0x7fefcc06fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:27.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.884+0000 7fefcb400640 1 --2- 172.21.15.165:0/3202753626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc0ce330 0x7fefcc06fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.885+0000 7fefcb400640 1 -- 172.21.15.165:0/3202753626 >> v1:172.21.15.165:6789/0 conn(0x7fefcc06abe0 legacy=0x7fefcc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.885+0000 7fefcb400640 1 -- 172.21.15.165:0/3202753626 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc0cd830 con 0x7fefcc0700a0 2024-04-25T10:13:27.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.885+0000 7fefc9600640 1 -- 172.21.15.165:0/3202753626 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1319033914 0 0) 0x7fefcc1626e0 con 0x7fefcc06abe0 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.886+0000 7fefcb400640 1 --2- 172.21.15.165:0/3202753626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0700a0 0x7fefcc15d370 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72403764e2ea6fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.886+0000 7fefc9600640 1 -- 172.21.15.165:0/3202753626 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4031600 con 0x7fefcc0700a0 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.886+0000 7fefc9600640 1 -- 172.21.15.165:0/3202753626 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc40317a0 con 0x7fefcc0700a0 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.886+0000 7fefc9600640 1 -- 172.21.15.165:0/3202753626 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4031b70 con 0x7fefcc0700a0 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.886+0000 7fefd1800640 1 -- 172.21.15.165:0/3202753626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0700a0 msgr2=0x7fefcc15d370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.886+0000 7fefd1800640 1 --2- 172.21.15.165:0/3202753626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0700a0 0x7fefcc15d370 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.887+0000 7fefd1800640 1 -- 172.21.15.165:0/3202753626 shutdown_connections 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.887+0000 7fefd1800640 1 --2- 172.21.15.165:0/3202753626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0700a0 0x7fefcc15d370 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.887+0000 7fefd1800640 1 --2- 172.21.15.165:0/3202753626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc0ce330 0x7fefcc06fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.887+0000 7fefd1800640 1 -- 172.21.15.165:0/3202753626 >> 172.21.15.165:0/3202753626 conn(0x7fefcc0b2220 msgr2=0x7fefcc0b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.887+0000 7fefd1800640 1 -- 172.21.15.165:0/3202753626 shutdown_connections 2024-04-25T10:13:27.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.887+0000 7fefd1800640 1 -- 172.21.15.165:0/3202753626 wait complete. 2024-04-25T10:13:27.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.888+0000 7fefd1800640 1 Processor -- start 2024-04-25T10:13:27.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.888+0000 7fefd1800640 1 -- start start 2024-04-25T10:13:27.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.889+0000 7fefd1800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc06abe0 0x7fefcc15f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.889+0000 7fefd1800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefcc0700a0 0x7fefcc1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.889+0000 7fefd1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 0x7fefcc0b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.889+0000 7fefd1800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc15d8b0 con 0x7fefcc0ce330 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.889+0000 7fefd1800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc161780 con 0x7fefcc0700a0 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.889+0000 7fefd1800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefcc15da30 con 0x7fefcc06abe0 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 0x7fefcc0b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 0x7fefcc0b4a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56654/0 (socket says 172.21.15.165:56654) 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 -- 172.21.15.165:0/1047125836 learned_addr learned my addr 172.21.15.165:0/1047125836 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcaa00640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc06abe0 0x7fefcc15f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefca000640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefcc0700a0 0x7fefcc1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 -- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc06abe0 msgr2=0x7fefcc15f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc06abe0 0x7fefcc15f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 -- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefcc0700a0 msgr2=0x7fefcc1512b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefcc0700a0 0x7fefcc1512b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefcb400640 1 -- 172.21.15.165:0/1047125836 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefcc15fdc0 con 0x7fefcc0ce330 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.890+0000 7fefca000640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefcc0700a0 0x7fefcc1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:27.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.891+0000 7fefcb400640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 0x7fefcc0b4a70 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.891+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4040070 con 0x7fefcc0ce330 2024-04-25T10:13:27.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.891+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc4030be0 con 0x7fefcc0ce330 2024-04-25T10:13:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.891+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefcc1600a0 con 0x7fefcc0ce330 2024-04-25T10:13:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.891+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc4009450 con 0x7fefcc0ce330 2024-04-25T10:13:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.891+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefcc160660 con 0x7fefcc0ce330 2024-04-25T10:13:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.892+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefcc0afe50 con 0x7fefcc0ce330 2024-04-25T10:13:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.893+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fefc4039040 con 0x7fefcc0ce330 2024-04-25T10:13:27.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.894+0000 7fefc2a00640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fefac06b500 0x7fefac06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:27.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.894+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fefc40ab3e0 con 0x7fefcc0ce330 2024-04-25T10:13:27.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.894+0000 7fefcaa00640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fefac06b500 0x7fefac06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:27.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.895+0000 7fefcaa00640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fefac06b500 0x7fefac06d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fefbc000f60 tx=0x7fefbc007b50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:27.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:27.898+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fefc407a090 con 0x7fefcc0ce330 2024-04-25T10:13:28.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.160+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fefcc0ade40 con 0x7fefcc0ce330 2024-04-25T10:13:28.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.161+0000 7fefc2a00640 1 -- 172.21.15.165:0/1047125836 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7fefc4079990 con 0x7fefcc0ce330 2024-04-25T10:13:28.166 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:28.166 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":709,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580987,"bytes_used":2601459712,"bytes_avail":770492653568,"bytes_total":773094113280,"write_bytes_sec":21101232,"read_op_per_sec":0,"write_op_per_sec":21},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:13:24.361852+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:13:28.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fefac06b500 msgr2=0x7fefac06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fefac06b500 0x7fefac06d9c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fefbc000f60 tx=0x7fefbc007b50 comp rx=0 tx=0).stop 2024-04-25T10:13:28.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 msgr2=0x7fefcc0b4a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:28.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 0x7fefcc0b4a70 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fefc4009d90 tx=0x7fefc4004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 shutdown_connections 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fefac06b500 0x7fefac06d9c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fefcc0ce330 0x7fefcc0b4a70 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fefcc0700a0 0x7fefcc1512b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 --2- 172.21.15.165:0/1047125836 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fefcc06abe0 0x7fefcc15f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 >> 172.21.15.165:0/1047125836 conn(0x7fefcc0b2220 msgr2=0x7fefcc0cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 shutdown_connections 2024-04-25T10:13:28.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:28.164+0000 7fefd1800640 1 -- 172.21.15.165:0/1047125836 wait complete. 2024-04-25T10:13:28.186 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:13:29.187 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:29.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.394+0000 7f26cfe00640 1 Processor -- start 2024-04-25T10:13:29.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.394+0000 7f26cfe00640 1 -- start start 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f26d015e520 0x7f26d015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 0x7f26d00adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cfe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26d00ae590 con 0x7f26d00ae0d0 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cfe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26d00aee30 con 0x7f26d015e520 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00ae870 con 0x7f26d015f040 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cf400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f26d00ae0d0 0x7f26d0070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34180/0 (socket says 172.21.15.165:34180) 2024-04-25T10:13:29.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cf400640 1 -- 172.21.15.165:0/3566457790 learned_addr learned my addr 172.21.15.165:0/3566457790 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:29.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26cea00640 1 --2- 172.21.15.165:0/3566457790 >> v2:172.21.15.172:3300/0 conn(0x7f26d015e520 0x7f26d015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.396+0000 7f26ce000640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 0x7f26d00adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4100559546 0 0) 0x7f26d00ae590 con 0x7f26d00ae0d0 2024-04-25T10:13:29.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b4003520 con 0x7f26d00ae0d0 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1639407894 0 0) 0x7f26b4003520 con 0x7f26d00ae0d0 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 msgr2=0x7f26d00adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 0x7f26d00adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 >> v2:172.21.15.172:3300/0 conn(0x7f26d015e520 msgr2=0x7f26d015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 --2- 172.21.15.165:0/3566457790 >> v2:172.21.15.172:3300/0 conn(0x7f26d015e520 0x7f26d015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00aeb50 con 0x7f26d00ae0d0 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f26c80019f0 con 0x7f26d00ae0d0 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 >> v1:172.21.15.165:6789/0 conn(0x7f26d00ae0d0 legacy=0x7f26d0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26b40039b0 0x7f26b4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 0x7f26b4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b4004d40 0x7f26b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00aee30 con 0x7f26b40039b0 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d00ae870 con 0x7f26b4004d40 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26b400ba70 con 0x7f26d015f040 2024-04-25T10:13:29.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.397+0000 7f26cea00640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26b40039b0 0x7f26b4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cf400640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b4004d40 0x7f26b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cea00640 1 -- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 msgr2=0x7f26b4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26ce000640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 0x7f26b4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cea00640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d015f040 0x7f26b4004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cea00640 1 -- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b4004d40 msgr2=0x7f26b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cea00640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b4004d40 0x7f26b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cea00640 1 -- 172.21.15.165:0/3566457790 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26b4003520 con 0x7f26b40039b0 2024-04-25T10:13:29.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.398+0000 7f26cf400640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b4004d40 0x7f26b40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:29.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.399+0000 7f26cea00640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26b40039b0 0x7f26b4003dd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009ac0 tx=0x7f26b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c019bd22b3589cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.399+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0036070 con 0x7f26b40039b0 2024-04-25T10:13:29.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.399+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c0031400 con 0x7f26b40039b0 2024-04-25T10:13:29.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.399+0000 7f26cd600640 1 -- 172.21.15.165:0/3566457790 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c0009690 con 0x7f26b40039b0 2024-04-25T10:13:29.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cfe00640 1 -- 172.21.15.165:0/3566457790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26b40039b0 msgr2=0x7f26b4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cfe00640 1 --2- 172.21.15.165:0/3566457790 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26b40039b0 0x7f26b4003dd0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009ac0 tx=0x7f26b4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cf400640 1 -- 172.21.15.165:0/3566457790 reap_dead start 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cfe00640 1 -- 172.21.15.165:0/3566457790 shutdown_connections 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cfe00640 1 -- 172.21.15.165:0/3566457790 >> 172.21.15.165:0/3566457790 conn(0x7f26d006abe0 msgr2=0x7f26d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cfe00640 1 -- 172.21.15.165:0/3566457790 shutdown_connections 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.400+0000 7f26cfe00640 1 -- 172.21.15.165:0/3566457790 wait complete. 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.402+0000 7f26cfe00640 1 Processor -- start 2024-04-25T10:13:29.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.402+0000 7f26cfe00640 1 -- start start 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b40039b0 0x7f26d00c61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 0x7f26d00c6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d0145c30 0x7f26d01460b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d015f620 con 0x7f26d00c4450 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d015f4a0 con 0x7f26b40039b0 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d0071700 con 0x7f26d0145c30 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 0x7f26d00c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 0x7f26d00c6700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56672/0 (socket says 172.21.15.165:56672) 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 -- 172.21.15.165:0/2716503998 learned_addr learned my addr 172.21.15.165:0/2716503998 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cea00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b40039b0 0x7f26d00c61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cf400640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d0145c30 0x7f26d01460b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 -- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d0145c30 msgr2=0x7f26d01460b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d0145c30 0x7f26d01460b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 -- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b40039b0 msgr2=0x7f26d00c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b40039b0 0x7f26d00c61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26ce000640 1 -- 172.21.15.165:0/2716503998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d00c49a0 con 0x7f26d00c4450 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.403+0000 7f26cf400640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d0145c30 0x7f26d01460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.404+0000 7f26ce000640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 0x7f26d00c6700 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00e840 tx=0x7f26bc00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.404+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc017070 con 0x7f26d00c4450 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.404+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26bc00c8d0 con 0x7f26d00c4450 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.404+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26bc00f040 con 0x7f26d00c4450 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.404+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d00c4c80 con 0x7f26d00c4450 2024-04-25T10:13:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.404+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d00c5240 con 0x7f26d00c4450 2024-04-25T10:13:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.405+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d0073990 con 0x7f26d00c4450 2024-04-25T10:13:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.406+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f26bc01a070 con 0x7f26d00c4450 2024-04-25T10:13:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.407+0000 7f26c6a00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26b006b500 0x7f26b006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.407+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f26bc089f00 con 0x7f26d00c4450 2024-04-25T10:13:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.407+0000 7f26cea00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26b006b500 0x7f26b006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.408+0000 7f26cea00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26b006b500 0x7f26b006d9c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f26c0033000 tx=0x7f26c0008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.412+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f26bc058bb0 con 0x7f26d00c4450 2024-04-25T10:13:29.550 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:29.556 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.651+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f26d015e9a0 con 0x7f26d00c4450 2024-04-25T10:13:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.652+0000 7f26c6a00640 1 -- 172.21.15.165:0/2716503998 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f26bc0584b0 con 0x7f26d00c4450 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26b006b500 msgr2=0x7f26b006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26b006b500 0x7f26b006d9c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f26c0033000 tx=0x7f26c0008030 comp rx=0 tx=0).stop 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 msgr2=0x7f26d00c6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 0x7f26d00c6700 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7f26bc00e840 tx=0x7f26bc00ed00 comp rx=0 tx=0).stop 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 shutdown_connections 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f26b006b500 0x7f26b006d9c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26d0145c30 0x7f26d01460b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26d00c4450 0x7f26d00c6700 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 --2- 172.21.15.165:0/2716503998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26b40039b0 0x7f26d00c61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 >> 172.21.15.165:0/2716503998 conn(0x7f26d006abe0 msgr2=0x7f26d00b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.654+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 shutdown_connections 2024-04-25T10:13:29.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.655+0000 7f26cfe00640 1 -- 172.21.15.165:0/2716503998 wait complete. 2024-04-25T10:13:29.677 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:29.677 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.914+0000 7f018b400640 1 Processor -- start 2024-04-25T10:13:29.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.914+0000 7f018b400640 1 -- start start 2024-04-25T10:13:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.916+0000 7f018b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 0x7f01840cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.916+0000 7f018b400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0184071480 con 0x7f0184154ee0 2024-04-25T10:13:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.916+0000 7f018b400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0184071760 con 0x7f01840ce3c0 2024-04-25T10:13:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.916+0000 7f018b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0184072090 con 0x7f01840cfec0 2024-04-25T10:13:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.916+0000 7f0188a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0184154ee0 0x7f01840cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34184/0 (socket says 172.21.15.165:34184) 2024-04-25T10:13:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.916+0000 7f0188a00640 1 -- 172.21.15.165:0/2884743091 learned_addr learned my addr 172.21.15.165:0/2884743091 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:29.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183e00640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 0x7f01840cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139960461 0 0) 0x7f0184071480 con 0x7f0184154ee0 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0168003520 con 0x7f0184154ee0 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 15889942 0 0) 0x7f0168003520 con 0x7f0184154ee0 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 msgr2=0x7f01840cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 0x7f01840cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 >> v1:172.21.15.172:6789/0 conn(0x7f01840ce3c0 legacy=0x7f018406d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0184071a40 con 0x7f0184154ee0 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f01740019f0 con 0x7f0184154ee0 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 >> v1:172.21.15.165:6789/0 conn(0x7f0184154ee0 legacy=0x7f01840cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 0x7f01680039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168003f60 0x7f0168004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0168004d40 0x7f01680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016800ba70 con 0x7f0168004d40 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.917+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f016800bc20 con 0x7f0168003f60 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f016800be20 con 0x7f01840cfec0 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0189400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0168004d40 0x7f01680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0188a00640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 0x7f01680039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0189400640 1 -- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 msgr2=0x7f01680039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0189400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01840cfec0 0x7f01680039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0189400640 1 -- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168003f60 msgr2=0x7f0168004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0189400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168003f60 0x7f0168004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.918+0000 7f0189400640 1 -- 172.21.15.165:0/2884743091 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0168003520 con 0x7f0168004d40 2024-04-25T10:13:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.919+0000 7f0189400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0168004d40 0x7f01680051c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f017c001750 tx=0x7f0168005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b30f53109c3cb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.920+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c001370 con 0x7f0168004d40 2024-04-25T10:13:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.920+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017c0052d0 con 0x7f0168004d40 2024-04-25T10:13:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.920+0000 7f0183400640 1 -- 172.21.15.165:0/2884743091 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017c02abe0 con 0x7f0168004d40 2024-04-25T10:13:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f018b400640 1 -- 172.21.15.165:0/2884743091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0168004d40 msgr2=0x7f01680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f018b400640 1 --2- 172.21.15.165:0/2884743091 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0168004d40 0x7f01680051c0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7f017c001750 tx=0x7f0168005830 comp rx=0 tx=0).stop 2024-04-25T10:13:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f0189400640 1 -- 172.21.15.165:0/2884743091 reap_dead start 2024-04-25T10:13:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f018b400640 1 -- 172.21.15.165:0/2884743091 shutdown_connections 2024-04-25T10:13:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f018b400640 1 -- 172.21.15.165:0/2884743091 >> 172.21.15.165:0/2884743091 conn(0x7f018406abe0 msgr2=0x7f018406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f018b400640 1 -- 172.21.15.165:0/2884743091 shutdown_connections 2024-04-25T10:13:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.921+0000 7f018b400640 1 -- 172.21.15.165:0/2884743091 wait complete. 2024-04-25T10:13:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.923+0000 7f018b400640 1 Processor -- start 2024-04-25T10:13:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.923+0000 7f018b400640 1 -- start start 2024-04-25T10:13:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f018b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168004d40 0x7f01841e3620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f018b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 0x7f01841f3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f018b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01841f4240 0x7f01840ad8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f018b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01840ce800 con 0x7f01840aa030 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f018b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0184071230 con 0x7f0168004d40 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f018b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01841552c0 con 0x7f01841f4240 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 0x7f01841f3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 0x7f01841f3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56698/0 (socket says 172.21.15.165:56698) 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 -- 172.21.15.165:0/887538921 learned_addr learned my addr 172.21.15.165:0/887538921 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 -- 172.21.15.165:0/887538921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01841f4240 msgr2=0x7f01840ad8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01841f4240 0x7f01840ad8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 -- 172.21.15.165:0/887538921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168004d40 msgr2=0x7f01841e3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168004d40 0x7f01841e3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.924+0000 7f0183e00640 1 -- 172.21.15.165:0/887538921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01840aa7b0 con 0x7f01840aa030 2024-04-25T10:13:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.925+0000 7f0183e00640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 0x7f01841f3d00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f017000e7c0 tx=0x7f017000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.925+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0170017070 con 0x7f01840aa030 2024-04-25T10:13:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.925+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017000c8c0 con 0x7f01840aa030 2024-04-25T10:13:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.925+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017000ccb0 con 0x7f01840aa030 2024-04-25T10:13:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.925+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01840aaa90 con 0x7f01840aa030 2024-04-25T10:13:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.925+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01840ab050 con 0x7f01840aa030 2024-04-25T10:13:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.926+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0184073990 con 0x7f01840aa030 2024-04-25T10:13:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.927+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f017001a070 con 0x7f01840aa030 2024-04-25T10:13:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.928+0000 7f0180c00640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f016406b4d0 0x7f016406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.928+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f017008a6b0 con 0x7f01840aa030 2024-04-25T10:13:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.929+0000 7f0188a00640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f016406b4d0 0x7f016406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.929+0000 7f0188a00640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f016406b4d0 0x7f016406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0174001370 tx=0x7f0174009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:29.933+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01700592d0 con 0x7f01840aa030 2024-04-25T10:13:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.198+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f018406d360 con 0x7f01840aa030 2024-04-25T10:13:30.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.199+0000 7f0180c00640 1 -- 172.21.15.165:0/887538921 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1631 (secure 0 0 0) 0x7f0170058bd0 con 0x7f01840aa030 2024-04-25T10:13:30.204 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:30.204 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":711,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076669268,"bytes_used":2601754624,"bytes_avail":770492358656,"bytes_total":773094113280,"write_bytes_sec":21109181,"read_op_per_sec":0,"write_op_per_sec":22},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":26,"modified":"2024-04-25T10:13:24.361852+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused paths [/scrub_control_test_path]"}}}}}},"progress_events":{}} 2024-04-25T10:13:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f016406b4d0 msgr2=0x7f016406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f016406b4d0 0x7f016406d990 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0174001370 tx=0x7f0174009040 comp rx=0 tx=0).stop 2024-04-25T10:13:30.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 msgr2=0x7f01841f3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 0x7f01841f3d00 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f017000e7c0 tx=0x7f017000ec80 comp rx=0 tx=0).stop 2024-04-25T10:13:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 shutdown_connections 2024-04-25T10:13:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f016406b4d0 0x7f016406d990 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01841f4240 0x7f01840ad8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01840aa030 0x7f01841f3d00 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 --2- 172.21.15.165:0/887538921 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0168004d40 0x7f01841e3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:30.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 >> 172.21.15.165:0/887538921 conn(0x7f018406abe0 msgr2=0x7f018406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:30.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 shutdown_connections 2024-04-25T10:13:30.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:30.202+0000 7f018b400640 1 -- 172.21.15.165:0/887538921 wait complete. 2024-04-25T10:13:30.224 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused paths [/scrub_control_test_path]'} 2024-04-25T10:13:31.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.429+0000 7f388b800640 1 Processor -- start 2024-04-25T10:13:31.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.430+0000 7f388b800640 1 -- start start 2024-04-25T10:13:31.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.431+0000 7f388b800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3884165680 0x7f38840aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.431+0000 7f388b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38840b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.431+0000 7f388b800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38840cf330 con 0x7f38840cc6c0 2024-04-25T10:13:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.431+0000 7f388b800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3884072340 con 0x7f3884165680 2024-04-25T10:13:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.431+0000 7f388b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38840b2b90 con 0x7f38840cec70 2024-04-25T10:13:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.431+0000 7f3888e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3884165680 0x7f38840aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3888e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3884165680 0x7f38840aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56700/0 (socket says 172.21.15.165:56700) 2024-04-25T10:13:31.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3888e00640 1 -- 172.21.15.165:0/703069881 learned_addr learned my addr 172.21.15.165:0/703069881 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3888e00640 1 -- 172.21.15.165:0/703069881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 msgr2=0x7f38840b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3883e00640 1 --2- 172.21.15.165:0/703069881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38840b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3888e00640 1 --2- 172.21.15.165:0/703069881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38840b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3888e00640 1 -- 172.21.15.165:0/703069881 >> v1:172.21.15.172:6789/0 conn(0x7f38840cc6c0 legacy=0x7f38840cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.432+0000 7f3888e00640 1 -- 172.21.15.165:0/703069881 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3884071d80 con 0x7f3884165680 2024-04-25T10:13:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.433+0000 7f3888e00640 1 --2- 172.21.15.165:0/703069881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840aecf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3878009870 tx=0x7f387802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b8227aaee25f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.434+0000 7f3883400640 1 -- 172.21.15.165:0/703069881 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878037070 con 0x7f3884165680 2024-04-25T10:13:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.434+0000 7f3883400640 1 -- 172.21.15.165:0/703069881 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3878032440 con 0x7f3884165680 2024-04-25T10:13:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.434+0000 7f3883400640 1 -- 172.21.15.165:0/703069881 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3878032810 con 0x7f3884165680 2024-04-25T10:13:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.434+0000 7f388b800640 1 -- 172.21.15.165:0/703069881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 msgr2=0x7f38840aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.434+0000 7f388b800640 1 --2- 172.21.15.165:0/703069881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840aecf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f3878009870 tx=0x7f387802a180 comp rx=0 tx=0).stop 2024-04-25T10:13:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.435+0000 7f388b800640 1 -- 172.21.15.165:0/703069881 shutdown_connections 2024-04-25T10:13:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.435+0000 7f388b800640 1 --2- 172.21.15.165:0/703069881 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38840b2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.435+0000 7f388b800640 1 --2- 172.21.15.165:0/703069881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840aecf0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.435+0000 7f388b800640 1 -- 172.21.15.165:0/703069881 >> 172.21.15.165:0/703069881 conn(0x7f388406abe0 msgr2=0x7f388406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.435+0000 7f388b800640 1 -- 172.21.15.165:0/703069881 shutdown_connections 2024-04-25T10:13:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.435+0000 7f388b800640 1 -- 172.21.15.165:0/703069881 wait complete. 2024-04-25T10:13:31.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.437+0000 7f388b800640 1 Processor -- start 2024-04-25T10:13:31.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.437+0000 7f388b800640 1 -- start start 2024-04-25T10:13:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.438+0000 7f388b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38840cc6c0 0x7f3884157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.438+0000 7f388b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38841f82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.438+0000 7f388b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.438+0000 7f388b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38840cf0d0 con 0x7f3884165680 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.438+0000 7f388b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3884071670 con 0x7f38840cc6c0 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.438+0000 7f388b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388406ea30 con 0x7f38840cec70 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840cbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56702/0 (socket says 172.21.15.165:56702) 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 -- 172.21.15.165:0/3135755170 learned_addr learned my addr 172.21.15.165:0/3135755170 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3888e00640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38840cc6c0 0x7f3884157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3883e00640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38841f82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 -- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 msgr2=0x7f38841f82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38841f82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 -- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38840cc6c0 msgr2=0x7f3884157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38840cc6c0 0x7f3884157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3889800640 1 -- 172.21.15.165:0/3135755170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38841537f0 con 0x7f3884165680 2024-04-25T10:13:31.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.439+0000 7f3888e00640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38840cc6c0 0x7f3884157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.440+0000 7f3889800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840cbc90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f387c00b970 tx=0x7f387c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.440+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c019070 con 0x7f3884165680 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.440+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f387c00cb40 con 0x7f3884165680 2024-04-25T10:13:31.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.440+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3884153ad0 con 0x7f3884165680 2024-04-25T10:13:31.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.440+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f387c014440 con 0x7f3884165680 2024-04-25T10:13:31.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.440+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3884154090 con 0x7f3884165680 2024-04-25T10:13:31.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.441+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3884154290 con 0x7f3884165680 2024-04-25T10:13:31.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.442+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f387c01c070 con 0x7f3884165680 2024-04-25T10:13:31.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.443+0000 7f3880c00640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f386406b4d0 0x7f386406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.443+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f387c08a080 con 0x7f3884165680 2024-04-25T10:13:31.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.443+0000 7f3888e00640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f386406b4d0 0x7f386406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.444+0000 7f3888e00640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f386406b4d0 0x7f386406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3878034000 tx=0x7f387802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.447+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3884154290 con 0x7f3884165680 2024-04-25T10:13:31.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.669+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3884073990 con 0x7f3884165680 2024-04-25T10:13:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.670+0000 7f3880c00640 1 -- 172.21.15.165:0/3135755170 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f387c058ca0 con 0x7f3884165680 2024-04-25T10:13:31.674 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:31.674 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:31.674 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.672+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f386406b4d0 msgr2=0x7f386406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.672+0000 7f388b800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f386406b4d0 0x7f386406d990 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f3878034000 tx=0x7f387802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 msgr2=0x7f38840cbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840cbc90 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f387c00b970 tx=0x7f387c00be30 comp rx=0 tx=0).stop 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 shutdown_connections 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f386406b4d0 0x7f386406d990 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3884165680 0x7f38840cbc90 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f38840cec70 0x7f38841f82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 --2- 172.21.15.165:0/3135755170 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38840cc6c0 0x7f3884157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 >> 172.21.15.165:0/3135755170 conn(0x7f388406abe0 msgr2=0x7f38840b4340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 shutdown_connections 2024-04-25T10:13:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.673+0000 7f388b800640 1 -- 172.21.15.165:0/3135755170 wait complete. 2024-04-25T10:13:31.695 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:13:31.696 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:13:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.888+0000 7f6949800640 1 Processor -- start 2024-04-25T10:13:31.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.889+0000 7f6949800640 1 -- start start 2024-04-25T10:13:31.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.890+0000 7f6949800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f694415e520 0x7f694415e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6949800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f694415f040 0x7f69440adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6949800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69440ae0d0 0x7f6944070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6949800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440aee30 con 0x7f694415e520 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6949800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69440ae590 con 0x7f694415f040 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6949800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69440ae870 con 0x7f69440ae0d0 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f694415e520 0x7f694415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f694415e520 0x7f694415e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56706/0 (socket says 172.21.15.165:56706) 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 -- 172.21.15.165:0/2194954361 learned_addr learned my addr 172.21.15.165:0/2194954361 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 -- 172.21.15.165:0/2194954361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69440ae0d0 msgr2=0x7f6944070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 --2- 172.21.15.165:0/2194954361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69440ae0d0 0x7f6944070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 -- 172.21.15.165:0/2194954361 >> v2:172.21.15.172:3300/0 conn(0x7f694415f040 msgr2=0x7f69440adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 --2- 172.21.15.165:0/2194954361 >> v2:172.21.15.172:3300/0 conn(0x7f694415f040 0x7f69440adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.891+0000 7f6943e00640 1 -- 172.21.15.165:0/2194954361 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440aeb50 con 0x7f694415e520 2024-04-25T10:13:31.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.892+0000 7f6943e00640 1 --2- 172.21.15.165:0/2194954361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f694415e900 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f69440cf050 tx=0x7f693402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a28a03fe1bf2d556 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6942a00640 1 -- 172.21.15.165:0/2194954361 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934037070 con 0x7f694415e520 2024-04-25T10:13:31.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6942a00640 1 -- 172.21.15.165:0/2194954361 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6934032440 con 0x7f694415e520 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6942a00640 1 -- 172.21.15.165:0/2194954361 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934032810 con 0x7f694415e520 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 -- 172.21.15.165:0/2194954361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 msgr2=0x7f694415e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 --2- 172.21.15.165:0/2194954361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f694415e900 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f69440cf050 tx=0x7f693402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 -- 172.21.15.165:0/2194954361 shutdown_connections 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 --2- 172.21.15.165:0/2194954361 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69440ae0d0 0x7f6944070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 --2- 172.21.15.165:0/2194954361 >> v2:172.21.15.172:3300/0 conn(0x7f694415f040 0x7f69440adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 --2- 172.21.15.165:0/2194954361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f694415e900 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 -- 172.21.15.165:0/2194954361 >> 172.21.15.165:0/2194954361 conn(0x7f694406abe0 msgr2=0x7f694406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 -- 172.21.15.165:0/2194954361 shutdown_connections 2024-04-25T10:13:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.893+0000 7f6949800640 1 -- 172.21.15.165:0/2194954361 wait complete. 2024-04-25T10:13:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.895+0000 7f6949800640 1 Processor -- start 2024-04-25T10:13:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.895+0000 7f6949800640 1 -- start start 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6949800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69440ae0d0 0x7f694414d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6949800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f6944158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6949800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f694415f040 0x7f69440cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6949800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694415f650 con 0x7f694415e520 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6949800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694415f4d0 con 0x7f69440ae0d0 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6949800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6944071530 con 0x7f694415f040 2024-04-25T10:13:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6943400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f6944158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6943400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f6944158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56718/0 (socket says 172.21.15.165:56718) 2024-04-25T10:13:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.896+0000 7f6943400640 1 -- 172.21.15.165:0/3989081916 learned_addr learned my addr 172.21.15.165:0/3989081916 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943e00640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69440ae0d0 0x7f694414d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6948e00640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f694415f040 0x7f69440cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943400640 1 -- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f694415f040 msgr2=0x7f69440cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943400640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f694415f040 0x7f69440cdae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943400640 1 -- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69440ae0d0 msgr2=0x7f694414d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943400640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69440ae0d0 0x7f694414d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943400640 1 -- 172.21.15.165:0/3989081916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69440cc7a0 con 0x7f694415e520 2024-04-25T10:13:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943e00640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69440ae0d0 0x7f694414d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6943400640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f6944158f50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f693000b410 tx=0x7f693000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930015020 con 0x7f694415e520 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6930011850 con 0x7f694415e520 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6930011c40 con 0x7f694415e520 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.897+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69440cca40 con 0x7f694415e520 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.898+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69440cd000 con 0x7f694415e520 2024-04-25T10:13:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.899+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6944073990 con 0x7f694415e520 2024-04-25T10:13:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.900+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6930018020 con 0x7f694415e520 2024-04-25T10:13:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.901+0000 7f693be00640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.901+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f693008a2f0 con 0x7f694415e520 2024-04-25T10:13:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.901+0000 7f6943e00640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.901+0000 7f6943e00640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f692406b4d0 0x7f692406d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6934034000 tx=0x7f693402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:31.905+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f693001f020 con 0x7f694415e520 2024-04-25T10:13:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.170+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f694415c4e0 con 0x7f694415e520 2024-04-25T10:13:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.171+0000 7f693be00640 1 -- 172.21.15.165:0/3989081916 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1573 (secure 0 0 0) 0x7f69300172c0 con 0x7f694415e520 2024-04-25T10:13:32.175 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:32.175 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":713,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":113,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076669268,"bytes_used":2601754624,"bytes_avail":770492358656,"bytes_total":773094113280,"write_bytes_sec":5524637,"read_op_per_sec":0,"write_op_per_sec":6},"fsmap":{"epoch":74,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"d","status":"up:active","gid":8043}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":27,"modified":"2024-04-25T10:13:30.364343+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f692406b4d0 msgr2=0x7f692406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f692406b4d0 0x7f692406d990 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f6934034000 tx=0x7f693402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 msgr2=0x7f6944158f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f6944158f50 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f693000b410 tx=0x7f693000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 shutdown_connections 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f692406b4d0 0x7f692406d990 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f694415f040 0x7f69440cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f694415e520 0x7f6944158f50 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 --2- 172.21.15.165:0/3989081916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69440ae0d0 0x7f694414d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.173+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 >> 172.21.15.165:0/3989081916 conn(0x7f694406abe0 msgr2=0x7f694406d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.174+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 shutdown_connections 2024-04-25T10:13:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.174+0000 7f6949800640 1 -- 172.21.15.165:0/3989081916 wait complete. 2024-04-25T10:13:32.196 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:13:32.196 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:32.196 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:32.298 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:32.402 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:32.402 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:13:32.402 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:32.461 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:13:32.461 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:32.516 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.716+0000 7f936a800640 1 Processor -- start 2024-04-25T10:13:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.716+0000 7f936a800640 1 -- start start 2024-04-25T10:13:32.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.717+0000 7f936a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f93640cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.717+0000 7f936a800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936415bf30 con 0x7f93640cd860 2024-04-25T10:13:32.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.717+0000 7f936a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f936415c210 con 0x7f9364144440 2024-04-25T10:13:32.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.717+0000 7f936a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f936415c860 con 0x7f9364071700 2024-04-25T10:13:32.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.717+0000 7f9369e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f93640cd860 0x7f936415ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34194/0 (socket says 172.21.15.165:34194) 2024-04-25T10:13:32.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.717+0000 7f9369e00640 1 -- 172.21.15.165:0/2761787222 learned_addr learned my addr 172.21.15.165:0/2761787222 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9368a00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f93640cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248413464 0 0) 0x7f936415bf30 con 0x7f93640cd860 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f934c003520 con 0x7f93640cd860 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2546192047 0 0) 0x7f934c003520 con 0x7f93640cd860 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 msgr2=0x7f93640cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f93640cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 >> v1:172.21.15.172:6789/0 conn(0x7f9364144440 legacy=0x7f9364070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f936415c4f0 con 0x7f93640cd860 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f935c0019f0 con 0x7f93640cd860 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 >> v1:172.21.15.165:6789/0 conn(0x7f93640cd860 legacy=0x7f936415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f934c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004010 0x7f934c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f936415c860 con 0x7f934c004d40 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934c00bc10 con 0x7f934c004010 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.718+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934c00be10 con 0x7f9364071700 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369400640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f934c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369e00640 1 -- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 msgr2=0x7f934c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f934c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369e00640 1 -- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004010 msgr2=0x7f934c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004010 0x7f934c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369e00640 1 -- 172.21.15.165:0/2761787222 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f934c003520 con 0x7f934c004d40 2024-04-25T10:13:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.719+0000 7f9369400640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9364071700 0x7f934c0039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:32.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.720+0000 7f9369e00640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f935c00a860 tx=0x7f934c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93db1e6dde21119 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.720+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00a480 con 0x7f934c004d40 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.720+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c02f070 con 0x7f934c004d40 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.720+0000 7f9363e00640 1 -- 172.21.15.165:0/2761787222 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c02bd40 con 0x7f934c004d40 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f936a800640 1 -- 172.21.15.165:0/2761787222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f934c004d40 msgr2=0x7f934c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f936a800640 1 --2- 172.21.15.165:0/2761787222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f934c004d40 0x7f934c0051c0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f935c00a860 tx=0x7f934c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f9369e00640 1 -- 172.21.15.165:0/2761787222 reap_dead start 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f936a800640 1 -- 172.21.15.165:0/2761787222 shutdown_connections 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f936a800640 1 -- 172.21.15.165:0/2761787222 >> 172.21.15.165:0/2761787222 conn(0x7f936406abe0 msgr2=0x7f936406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f936a800640 1 -- 172.21.15.165:0/2761787222 shutdown_connections 2024-04-25T10:13:32.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.721+0000 7f936a800640 1 -- 172.21.15.165:0/2761787222 wait complete. 2024-04-25T10:13:32.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.724+0000 7f936a800640 1 Processor -- start 2024-04-25T10:13:32.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.724+0000 7f936a800640 1 -- start start 2024-04-25T10:13:32.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.725+0000 7f936a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004d40 0x7f93641f30b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.725+0000 7f936a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93641e46a0 0x7f93641e4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.725+0000 7f936a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 0x7f93641e2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.725+0000 7f936a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640cde70 con 0x7f93641e2720 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.725+0000 7f936a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93640cdcf0 con 0x7f934c004d40 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.725+0000 7f936a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93640cd320 con 0x7f93641e46a0 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 0x7f93641e2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 0x7f93641e2bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56742/0 (socket says 172.21.15.165:56742) 2024-04-25T10:13:32.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 -- 172.21.15.165:0/2437310731 learned_addr learned my addr 172.21.15.165:0/2437310731 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:32.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9368a00640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93641e46a0 0x7f93641e4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369400640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004d40 0x7f93641f30b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 -- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93641e46a0 msgr2=0x7f93641e4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93641e46a0 0x7f93641e4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 -- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004d40 msgr2=0x7f93641f30b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004d40 0x7f93641f30b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 -- 172.21.15.165:0/2437310731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93641e34a0 con 0x7f93641e2720 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.726+0000 7f9369e00640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 0x7f93641e2bd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f935c02b530 tx=0x7f936415c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.727+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c00a480 con 0x7f93641e2720 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.727+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935c02bc90 con 0x7f93641e2720 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.727+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935c034690 con 0x7f93641e2720 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.727+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93641e3780 con 0x7f93641e2720 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.727+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93641e3d40 con 0x7f93641e2720 2024-04-25T10:13:32.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.728+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9364073990 con 0x7f93641e2720 2024-04-25T10:13:32.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.729+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f935c034830 con 0x7f93641e2720 2024-04-25T10:13:32.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.730+0000 7f9361600640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f934406b4d0 0x7f934406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:32.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.730+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f935c0aa780 con 0x7f93641e2720 2024-04-25T10:13:32.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.730+0000 7f9369400640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f934406b4d0 0x7f934406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:32.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.731+0000 7f9369400640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f934406b4d0 0x7f934406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9358000c00 tx=0x7f9358009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.734+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f935c079310 con 0x7f93641e2720 2024-04-25T10:13:32.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.933+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f936406afc0 con 0x7f93641e2720 2024-04-25T10:13:32.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.935+0000 7f9361600640 1 -- 172.21.15.165:0/2437310731 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17018 (secure 0 0 0) 0x7f935c078c10 con 0x7f93641e2720 2024-04-25T10:13:32.939 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:32.939 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:48.513977+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1c","mappings":[{"from":1,"to":7}]},{"pgid":"16.31","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.937+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f934406b4d0 msgr2=0x7f934406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.937+0000 7f936a800640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f934406b4d0 0x7f934406d990 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f9358000c00 tx=0x7f9358009040 comp rx=0 tx=0).stop 2024-04-25T10:13:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.937+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 msgr2=0x7f93641e2bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.937+0000 7f936a800640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 0x7f93641e2bd0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f935c02b530 tx=0x7f936415c100 comp rx=0 tx=0).stop 2024-04-25T10:13:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 shutdown_connections 2024-04-25T10:13:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f934406b4d0 0x7f934406d990 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93641e2720 0x7f93641e2bd0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f93641e46a0 0x7f93641e4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 --2- 172.21.15.165:0/2437310731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f934c004d40 0x7f93641f30b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:32.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 >> 172.21.15.165:0/2437310731 conn(0x7f936406abe0 msgr2=0x7f93640b2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:32.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 shutdown_connections 2024-04-25T10:13:32.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:32.938+0000 7f936a800640 1 -- 172.21.15.165:0/2437310731 wait complete. 2024-04-25T10:13:32.961 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:34.618582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:35.883096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:32.962 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:33.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.153+0000 7f48bf200640 1 Processor -- start 2024-04-25T10:13:33.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.153+0000 7f48bf200640 1 -- start start 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.154+0000 7f48bf200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f48c0145140 0x7f48c00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.154+0000 7f48bf200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00c5200 0x7f48c015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.154+0000 7f48bf200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00c5650 con 0x7f48c00ce950 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.154+0000 7f48bf200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c00ced90 con 0x7f48c0145140 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.154+0000 7f48bf200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0145520 con 0x7f48c00c5200 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f48c0145140 0x7f48c00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f48c0145140 0x7f48c00ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56752/0 (socket says 172.21.15.165:56752) 2024-04-25T10:13:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 -- 172.21.15.165:0/390109170 learned_addr learned my addr 172.21.15.165:0/390109170 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:33.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 -- 172.21.15.165:0/390109170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00c5200 msgr2=0x7f48c015bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 --2- 172.21.15.165:0/390109170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00c5200 0x7f48c015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 -- 172.21.15.165:0/390109170 >> v1:172.21.15.172:6789/0 conn(0x7f48c00ce950 legacy=0x7f48c00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.155+0000 7f48bde00640 1 -- 172.21.15.165:0/390109170 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0071270 con 0x7f48c0145140 2024-04-25T10:13:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.156+0000 7f48bde00640 1 --2- 172.21.15.165:0/390109170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c0145140 0x7f48c00ce410 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009e50 tx=0x7f48b002a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=558c648092042cfe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.157+0000 7f48bd400640 1 -- 172.21.15.165:0/390109170 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0036070 con 0x7f48c0145140 2024-04-25T10:13:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.157+0000 7f48bd400640 1 -- 172.21.15.165:0/390109170 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0031600 con 0x7f48c0145140 2024-04-25T10:13:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.157+0000 7f48bd400640 1 -- 172.21.15.165:0/390109170 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b00319d0 con 0x7f48c0145140 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.157+0000 7f48bf200640 1 -- 172.21.15.165:0/390109170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c0145140 msgr2=0x7f48c00ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.157+0000 7f48bf200640 1 --2- 172.21.15.165:0/390109170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c0145140 0x7f48c00ce410 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009e50 tx=0x7f48b002a630 comp rx=0 tx=0).stop 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.158+0000 7f48bf200640 1 -- 172.21.15.165:0/390109170 shutdown_connections 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.158+0000 7f48bf200640 1 --2- 172.21.15.165:0/390109170 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00c5200 0x7f48c015bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.158+0000 7f48bf200640 1 --2- 172.21.15.165:0/390109170 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c0145140 0x7f48c00ce410 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.158+0000 7f48bf200640 1 -- 172.21.15.165:0/390109170 >> 172.21.15.165:0/390109170 conn(0x7f48c0061cc0 msgr2=0x7f48c006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.158+0000 7f48bf200640 1 -- 172.21.15.165:0/390109170 shutdown_connections 2024-04-25T10:13:33.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.158+0000 7f48bf200640 1 -- 172.21.15.165:0/390109170 wait complete. 2024-04-25T10:13:33.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.159+0000 7f48bf200640 1 Processor -- start 2024-04-25T10:13:33.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.160+0000 7f48bf200640 1 -- start start 2024-04-25T10:13:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.160+0000 7f48bf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 0x7f48c0140cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.160+0000 7f48bf200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ce950 0x7f48c00ca090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bf200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c0145140 0x7f48c01f4990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bf200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0071b10 con 0x7f48c00c5200 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bf200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0071830 con 0x7f48c0145140 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bf200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c015c4f0 con 0x7f48c00ce950 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 0x7f48c0140cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 0x7f48c0140cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56754/0 (socket says 172.21.15.165:56754) 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 -- 172.21.15.165:0/1274426023 learned_addr learned my addr 172.21.15.165:0/1274426023 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48be800640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c0145140 0x7f48c01f4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 -- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ce950 msgr2=0x7f48c00ca090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ce950 0x7f48c00ca090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 -- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c0145140 msgr2=0x7f48c01f4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c0145140 0x7f48c01f4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.161+0000 7f48bde00640 1 -- 172.21.15.165:0/1274426023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c01411f0 con 0x7f48c00c5200 2024-04-25T10:13:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.162+0000 7f48bde00640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 0x7f48c0140cb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009e50 tx=0x7f48b0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.162+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0041020 con 0x7f48c00c5200 2024-04-25T10:13:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.162+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0007800 con 0x7f48c00c5200 2024-04-25T10:13:33.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.162+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c00caa70 con 0x7f48c00c5200 2024-04-25T10:13:33.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.162+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0036040 con 0x7f48c00c5200 2024-04-25T10:13:33.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.162+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c00cb030 con 0x7f48c00c5200 2024-04-25T10:13:33.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.163+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c0073990 con 0x7f48c00c5200 2024-04-25T10:13:33.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.164+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f48b0044020 con 0x7f48c00c5200 2024-04-25T10:13:33.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.165+0000 7f48b6a00640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f489c06b4d0 0x7f489c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.165+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f48b00ab1a0 con 0x7f48c00c5200 2024-04-25T10:13:33.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.165+0000 7f48b5400640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f489c06b4d0 0x7f489c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.166+0000 7f48b5400640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f489c06b4d0 0x7f489c06d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f48c01e8490 tx=0x7f48a80079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.170+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f48b004b030 con 0x7f48c00c5200 2024-04-25T10:13:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.365+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f48c006ec00 con 0x7f48c00c5200 2024-04-25T10:13:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.366+0000 7f48b6a00640 1 -- 172.21.15.165:0/1274426023 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f48c01411f0 con 0x7f48c00c5200 2024-04-25T10:13:33.371 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:13:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.369+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f489c06b4d0 msgr2=0x7f489c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.369+0000 7f48bf200640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f489c06b4d0 0x7f489c06d990 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f48c01e8490 tx=0x7f48a80079b0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.369+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 msgr2=0x7f48c0140cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.369+0000 7f48bf200640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 0x7f48c0140cb0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f48b0009e50 tx=0x7f48b0004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.369+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 shutdown_connections 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f489c06b4d0 0x7f489c06d990 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f48c0145140 0x7f48c01f4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f48c00ce950 0x7f48c00ca090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 --2- 172.21.15.165:0/1274426023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f48c00c5200 0x7f48c0140cb0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 >> 172.21.15.165:0/1274426023 conn(0x7f48c0061cc0 msgr2=0x7f48c00b14c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 shutdown_connections 2024-04-25T10:13:33.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.370+0000 7f48bf200640 1 -- 172.21.15.165:0/1274426023 wait complete. 2024-04-25T10:13:33.392 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:33.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.593+0000 7fcdf1a00640 1 Processor -- start 2024-04-25T10:13:33.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.594+0000 7fcdf1a00640 1 -- start start 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.595+0000 7fcdf1a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcdec0ce230 0x7fcdec0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.595+0000 7fcdf1a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcdec15b270 0x7fcdec15b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.595+0000 7fcdf1a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdec072280 con 0x7fcdec0cc4f0 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.595+0000 7fcdf1a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec071270 con 0x7fcdec0ce230 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.595+0000 7fcdf1a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdec0714c0 con 0x7fcdec15b270 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcdeaa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcdec0cc4f0 0x7fcdec0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34198/0 (socket says 172.21.15.165:34198) 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcdeaa00640 1 -- 172.21.15.165:0/2826831941 learned_addr learned my addr 172.21.15.165:0/2826831941 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:33.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcde2000640 1 --2- 172.21.15.165:0/2826831941 >> v2:172.21.15.165:3300/0 conn(0x7fcdec0ce230 0x7fcdec0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcdeb400640 1 --2- 172.21.15.165:0/2826831941 >> v2:172.21.15.172:3300/0 conn(0x7fcdec15b270 0x7fcdec15b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcdea000640 1 -- 172.21.15.165:0/2826831941 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3507269270 0 0) 0x7fcdec072280 con 0x7fcdec0cc4f0 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcdea000640 1 -- 172.21.15.165:0/2826831941 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd0003520 con 0x7fcdec0cc4f0 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcde2000640 1 -- 172.21.15.165:0/2826831941 >> v2:172.21.15.172:3300/0 conn(0x7fcdec15b270 msgr2=0x7fcdec15b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcde2000640 1 --2- 172.21.15.165:0/2826831941 >> v2:172.21.15.172:3300/0 conn(0x7fcdec15b270 0x7fcdec15b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcde2000640 1 -- 172.21.15.165:0/2826831941 >> v1:172.21.15.165:6789/0 conn(0x7fcdec0cc4f0 legacy=0x7fcdec0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcde2000640 1 -- 172.21.15.165:0/2826831941 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec0717a0 con 0x7fcdec0ce230 2024-04-25T10:13:33.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.596+0000 7fcdeb400640 1 --2- 172.21.15.165:0/2826831941 >> v2:172.21.15.172:3300/0 conn(0x7fcdec15b270 0x7fcdec15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.597+0000 7fcde2000640 1 --2- 172.21.15.165:0/2826831941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec0ce230 0x7fcdec0cc8d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcdec15af60 tx=0x7fcdd402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=301fc1c429560351 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdea000640 1 -- 172.21.15.165:0/2826831941 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4037070 con 0x7fcdec0ce230 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdea000640 1 -- 172.21.15.165:0/2826831941 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd4032440 con 0x7fcdec0ce230 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdea000640 1 -- 172.21.15.165:0/2826831941 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd4032810 con 0x7fcdec0ce230 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2826831941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec0ce230 msgr2=0x7fcdec0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2826831941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec0ce230 0x7fcdec0cc8d0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7fcdec15af60 tx=0x7fcdd402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2826831941 shutdown_connections 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2826831941 >> v2:172.21.15.172:3300/0 conn(0x7fcdec15b270 0x7fcdec15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.598+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2826831941 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec0ce230 0x7fcdec0cc8d0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.599+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2826831941 >> 172.21.15.165:0/2826831941 conn(0x7fcdec06abe0 msgr2=0x7fcdec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.599+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2826831941 shutdown_connections 2024-04-25T10:13:33.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.599+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2826831941 wait complete. 2024-04-25T10:13:33.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.600+0000 7fcdf1a00640 1 Processor -- start 2024-04-25T10:13:33.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.601+0000 7fcdf1a00640 1 -- start start 2024-04-25T10:13:33.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.601+0000 7fcdf1a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcdec0cc4f0 0x7fcdec142030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.601+0000 7fcdf1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdec0ce230 0x7fcdec1e5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdf1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 0x7fcdec1e6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdf1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec15bc20 con 0x7fcdec15b270 2024-04-25T10:13:33.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdf1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec071d60 con 0x7fcdec0ce230 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdf1a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec15bda0 con 0x7fcdec0cc4f0 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 0x7fcdec1e6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 0x7fcdec1e6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56780/0 (socket says 172.21.15.165:56780) 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 -- 172.21.15.165:0/2703376143 learned_addr learned my addr 172.21.15.165:0/2703376143 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeaa00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcdec0cc4f0 0x7fcdec142030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcde2000640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdec0ce230 0x7fcdec1e5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 -- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcdec0cc4f0 msgr2=0x7fcdec142030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcdec0cc4f0 0x7fcdec142030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 -- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdec0ce230 msgr2=0x7fcdec1e5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdec0ce230 0x7fcdec1e5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.602+0000 7fcdeb400640 1 -- 172.21.15.165:0/2703376143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdec154010 con 0x7fcdec15b270 2024-04-25T10:13:33.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.603+0000 7fcdeb400640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 0x7fcdec1e6020 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c820 tx=0x7fcde400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.603+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4015070 con 0x7fcdec15b270 2024-04-25T10:13:33.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.603+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4007a30 con 0x7fcdec15b270 2024-04-25T10:13:33.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.603+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4007e20 con 0x7fcdec15b270 2024-04-25T10:13:33.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.603+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec1542f0 con 0x7fcdec15b270 2024-04-25T10:13:33.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.603+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1548b0 con 0x7fcdec15b270 2024-04-25T10:13:33.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.604+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdec073a00 con 0x7fcdec15b270 2024-04-25T10:13:33.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.605+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcde4018070 con 0x7fcdec15b270 2024-04-25T10:13:33.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.606+0000 7fcde3400640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc806b4d0 0x7fcdc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:33.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.607+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fcde408a350 con 0x7fcdec15b270 2024-04-25T10:13:33.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.607+0000 7fcdeaa00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc806b4d0 0x7fcdc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:33.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.608+0000 7fcdeaa00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc806b4d0 0x7fcdc806d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0012e0 tx=0x7fcddc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:33.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.611+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcde4059010 con 0x7fcdec15b270 2024-04-25T10:13:33.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.804+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcdec0b43f0 con 0x7fcdec15b270 2024-04-25T10:13:33.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.804+0000 7fcde3400640 1 -- 172.21.15.165:0/2703376143 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcde4058910 con 0x7fcdec15b270 2024-04-25T10:13:33.809 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:13:33.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc806b4d0 msgr2=0x7fcdc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc806b4d0 0x7fcdc806d990 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcddc0012e0 tx=0x7fcddc009040 comp rx=0 tx=0).stop 2024-04-25T10:13:33.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 msgr2=0x7fcdec1e6020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 0x7fcdec1e6020 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c820 tx=0x7fcde400cce0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 shutdown_connections 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdc806b4d0 0x7fcdc806d990 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcdec15b270 0x7fcdec1e6020 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcdec0ce230 0x7fcdec1e5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 --2- 172.21.15.165:0/2703376143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcdec0cc4f0 0x7fcdec142030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 >> 172.21.15.165:0/2703376143 conn(0x7fcdec06abe0 msgr2=0x7fcdec158030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 shutdown_connections 2024-04-25T10:13:33.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:33.807+0000 7fcdf1a00640 1 -- 172.21.15.165:0/2703376143 wait complete. 2024-04-25T10:13:33.831 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.032+0000 7f546d800640 1 Processor -- start 2024-04-25T10:13:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.033+0000 7f546d800640 1 -- start start 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.034+0000 7f546d800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f546815da90 0x7f5468161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.034+0000 7f546d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5468162040 con 0x7f5468161ba0 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.034+0000 7f546d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5468162320 con 0x7f54680b9750 2024-04-25T10:13:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.034+0000 7f546d800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5468162600 con 0x7f546815da90 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.034+0000 7f5467400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5468161ba0 0x7f54680d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34210/0 (socket says 172.21.15.165:34210) 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.034+0000 7f5467400640 1 -- 172.21.15.165:0/3040475691 learned_addr learned my addr 172.21.15.165:0/3040475691 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5466a00640 1 --2- 172.21.15.165:0/3040475691 >> v2:172.21.15.172:3300/0 conn(0x7f546815da90 0x7f5468161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 522477484 0 0) 0x7f5468162040 con 0x7f5468161ba0 2024-04-25T10:13:34.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f544c003520 con 0x7f5468161ba0 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3746108000 0 0) 0x7f544c003520 con 0x7f5468161ba0 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 >> v2:172.21.15.172:3300/0 conn(0x7f546815da90 msgr2=0x7f5468161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 --2- 172.21.15.165:0/3040475691 >> v2:172.21.15.172:3300/0 conn(0x7f546815da90 0x7f5468161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 >> v1:172.21.15.172:6789/0 conn(0x7f54680b9750 legacy=0x7f54680ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54680d1b10 con 0x7f5468161ba0 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.035+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5460001670 con 0x7f5468161ba0 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 >> v1:172.21.15.165:6789/0 conn(0x7f5468161ba0 legacy=0x7f54680d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f544c0039b0 0x7f544c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c004800 0x7f544c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f544c0051c0 0x7f544c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468162600 con 0x7f544c0051c0 2024-04-25T10:13:34.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544c00c160 con 0x7f544c004800 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544c00c360 con 0x7f544c0039b0 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5467400640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f544c0051c0 0x7f544c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5466a00640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f544c0039b0 0x7f544c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.036+0000 7f5466000640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c004800 0x7f544c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.037+0000 7f5467400640 1 -- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f544c0039b0 msgr2=0x7f544c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.037+0000 7f5467400640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f544c0039b0 0x7f544c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.037+0000 7f5467400640 1 -- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c004800 msgr2=0x7f544c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.037+0000 7f5467400640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c004800 0x7f544c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.037+0000 7f5467400640 1 -- 172.21.15.165:0/3040475691 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f544c003520 con 0x7f544c0051c0 2024-04-25T10:13:34.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.038+0000 7f5467400640 1 -- 172.21.15.165:0/3040475691 reap_dead start 2024-04-25T10:13:34.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.038+0000 7f5467400640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f544c0051c0 0x7f544c00b9c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f546000c240 tx=0x7f544c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6a3d83c0305ed77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.038+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460007d10 con 0x7f544c0051c0 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.038+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460005ad0 con 0x7f544c0051c0 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.038+0000 7f5465600640 1 -- 172.21.15.165:0/3040475691 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460007070 con 0x7f544c0051c0 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 -- 172.21.15.165:0/3040475691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f544c0051c0 msgr2=0x7f544c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f544c0051c0 0x7f544c00b9c0 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7f546000c240 tx=0x7f544c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 -- 172.21.15.165:0/3040475691 shutdown_connections 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 --2- 172.21.15.165:0/3040475691 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f544c0051c0 0x7f544c00b9c0 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 -- 172.21.15.165:0/3040475691 >> 172.21.15.165:0/3040475691 conn(0x7f5468061a20 msgr2=0x7f5468061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 -- 172.21.15.165:0/3040475691 shutdown_connections 2024-04-25T10:13:34.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.039+0000 7f546d800640 1 -- 172.21.15.165:0/3040475691 wait complete. 2024-04-25T10:13:34.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.041+0000 7f546d800640 1 Processor -- start 2024-04-25T10:13:34.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.041+0000 7f546d800640 1 -- start start 2024-04-25T10:13:34.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f546d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c0051c0 0x7f54681517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f546d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f546815e380 0x7f54680c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f546d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 0x7f546815a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f546d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5468161660 con 0x7f54680c7a10 2024-04-25T10:13:34.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f546d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f546815de70 con 0x7f544c0051c0 2024-04-25T10:13:34.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f546d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54681617e0 con 0x7f546815e380 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f5467400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 0x7f546815a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f5467400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 0x7f546815a5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56792/0 (socket says 172.21.15.165:56792) 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.042+0000 7f5467400640 1 -- 172.21.15.165:0/3892448867 learned_addr learned my addr 172.21.15.165:0/3892448867 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5466000640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f546815e380 0x7f54680c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5466a00640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c0051c0 0x7f54681517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5467400640 1 -- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f546815e380 msgr2=0x7f54680c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5467400640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f546815e380 0x7f54680c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5467400640 1 -- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c0051c0 msgr2=0x7f54681517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5467400640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c0051c0 0x7f54681517a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5467400640 1 -- 172.21.15.165:0/3892448867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5468152160 con 0x7f54680c7a10 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f5467400640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 0x7f546815a5c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f546000c360 tx=0x7f54681624f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460007d10 con 0x7f54680c7a10 2024-04-25T10:13:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.043+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5460000ed0 con 0x7f54680c7a10 2024-04-25T10:13:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.044+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5460007070 con 0x7f54680c7a10 2024-04-25T10:13:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.044+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54680c8600 con 0x7f54680c7a10 2024-04-25T10:13:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.044+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54680c8320 con 0x7f54680c7a10 2024-04-25T10:13:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.045+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5468073a00 con 0x7f54680c7a10 2024-04-25T10:13:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.046+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5460044020 con 0x7f54680c7a10 2024-04-25T10:13:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.047+0000 7f545ea00640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f544406b4d0 0x7f544406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.047+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f54600aa120 con 0x7f54680c7a10 2024-04-25T10:13:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.047+0000 7f5466a00640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f544406b4d0 0x7f544406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.048+0000 7f5466a00640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f544406b4d0 0x7f544406d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5458009fc0 tx=0x7f545800b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.051+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5460078dc0 con 0x7f54680c7a10 2024-04-25T10:13:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.247+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f54680d1590 con 0x7f54680c7a10 2024-04-25T10:13:34.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.248+0000 7f545ea00640 1 -- 172.21.15.165:0/3892448867 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f54600786c0 con 0x7f54680c7a10 2024-04-25T10:13:34.252 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f544406b4d0 msgr2=0x7f544406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f544406b4d0 0x7f544406d990 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f5458009fc0 tx=0x7f545800b040 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 msgr2=0x7f546815a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 0x7f546815a5c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7f546000c360 tx=0x7f54681624f0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 shutdown_connections 2024-04-25T10:13:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f544406b4d0 0x7f544406d990 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f54680c7a10 0x7f546815a5c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f546815e380 0x7f54680c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 --2- 172.21.15.165:0/3892448867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f544c0051c0 0x7f54681517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 >> 172.21.15.165:0/3892448867 conn(0x7f5468061a20 msgr2=0x7f546806c080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 shutdown_connections 2024-04-25T10:13:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.251+0000 7f546d800640 1 -- 172.21.15.165:0/3892448867 wait complete. 2024-04-25T10:13:34.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:13:34.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.466+0000 7f8aa6400640 1 Processor -- start 2024-04-25T10:13:34.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.466+0000 7f8aa6400640 1 -- start start 2024-04-25T10:13:34.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8aa6400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8aa015b710 0x7f8aa00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8aa6400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa006f0e0 con 0x7f8aa00ac8b0 2024-04-25T10:13:34.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8aa6400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa00cfd80 con 0x7f8aa006ec50 2024-04-25T10:13:34.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8aa6400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa015e490 con 0x7f8aa015b710 2024-04-25T10:13:34.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8a9fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8aa00ac8b0 0x7f8aa0071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34214/0 (socket says 172.21.15.165:34214) 2024-04-25T10:13:34.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8a9fe00640 1 -- 172.21.15.165:0/2487236643 learned_addr learned my addr 172.21.15.165:0/2487236643 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8a9f400640 1 --2- 172.21.15.165:0/2487236643 >> v2:172.21.15.172:3300/0 conn(0x7f8aa015b710 0x7f8aa00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.468+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2175587089 0 0) 0x7f8aa006f0e0 con 0x7f8aa00ac8b0 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a80003520 con 0x7f8aa00ac8b0 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306471677 0 0) 0x7f8a80003520 con 0x7f8aa00ac8b0 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 >> v2:172.21.15.172:3300/0 conn(0x7f8aa015b710 msgr2=0x7f8aa00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 --2- 172.21.15.165:0/2487236643 >> v2:172.21.15.172:3300/0 conn(0x7f8aa015b710 0x7f8aa00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 >> v1:172.21.15.172:6789/0 conn(0x7f8aa006ec50 legacy=0x7f8aa015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa00d0060 con 0x7f8aa00ac8b0 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8a980019f0 con 0x7f8aa00ac8b0 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 >> v1:172.21.15.165:6789/0 conn(0x7f8aa00ac8b0 legacy=0x7f8aa0071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a800039b0 0x7f8a80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800051c0 0x7f8a8000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015e490 con 0x7f8a800039b0 2024-04-25T10:13:34.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8000c060 con 0x7f8a80004800 2024-04-25T10:13:34.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8000c260 con 0x7f8a800051c0 2024-04-25T10:13:34.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.469+0000 7f8a9f400640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a800039b0 0x7f8a80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9f400640 1 -- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800051c0 msgr2=0x7f8a8000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9fe00640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800051c0 0x7f8a8000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a97e00640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9f400640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800051c0 0x7f8a8000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9f400640 1 -- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a80004800 msgr2=0x7f8a80004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9f400640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9f400640 1 -- 172.21.15.165:0/2487236643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a80003520 con 0x7f8a800039b0 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a97e00640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a80004800 0x7f8a80004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.470+0000 7f8a9fe00640 1 -- 172.21.15.165:0/2487236643 reap_dead start 2024-04-25T10:13:34.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.471+0000 7f8a9f400640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a800039b0 0x7f8a80003dd0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009ac0 tx=0x7f8a80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c1917b41662a690 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.471+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90036070 con 0x7f8a800039b0 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.471+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a90031400 con 0x7f8a800039b0 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.471+0000 7f8a9ea00640 1 -- 172.21.15.165:0/2487236643 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90009690 con 0x7f8a800039b0 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 -- 172.21.15.165:0/2487236643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a800039b0 msgr2=0x7f8a80003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a800039b0 0x7f8a80003dd0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009ac0 tx=0x7f8a80005830 comp rx=0 tx=0).stop 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 -- 172.21.15.165:0/2487236643 shutdown_connections 2024-04-25T10:13:34.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 --2- 172.21.15.165:0/2487236643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a800039b0 0x7f8a80003dd0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 -- 172.21.15.165:0/2487236643 >> 172.21.15.165:0/2487236643 conn(0x7f8aa006abe0 msgr2=0x7f8aa006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 -- 172.21.15.165:0/2487236643 shutdown_connections 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.472+0000 7f8aa6400640 1 -- 172.21.15.165:0/2487236643 wait complete. 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.474+0000 7f8aa6400640 1 Processor -- start 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.474+0000 7f8aa6400640 1 -- start start 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8aa6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800039b0 0x7f8aa0156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8aa6400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa0152360 0x7f8aa015bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8aa6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 0x7f8aa015c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8aa6400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015bc70 con 0x7f8aa01527e0 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8aa6400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa015baf0 con 0x7f8aa0152360 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8aa6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa00acd00 con 0x7f8a800039b0 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 0x7f8aa015c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 0x7f8aa015c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56822/0 (socket says 172.21.15.165:56822) 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 -- 172.21.15.165:0/3741972804 learned_addr learned my addr 172.21.15.165:0/3741972804 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:34.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9f400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800039b0 0x7f8aa0156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 -- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800039b0 msgr2=0x7f8aa0156190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800039b0 0x7f8aa0156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 -- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa0152360 msgr2=0x7f8aa015bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa0152360 0x7f8aa015bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.475+0000 7f8a9fe00640 1 -- 172.21.15.165:0/3741972804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa01f6c70 con 0x7f8aa01527e0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.476+0000 7f8a9fe00640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 0x7f8aa015c330 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00af380 tx=0x7f8a9800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.476+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800b870 con 0x7f8aa01527e0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.476+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9800ba50 con 0x7f8aa01527e0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.476+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800be40 con 0x7f8aa01527e0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.476+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa01f6f50 con 0x7f8aa01527e0 2024-04-25T10:13:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.476+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa01f7510 con 0x7f8aa01527e0 2024-04-25T10:13:34.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.477+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa0073a00 con 0x7f8aa01527e0 2024-04-25T10:13:34.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.478+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a9801b020 con 0x7f8aa01527e0 2024-04-25T10:13:34.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.479+0000 7f8a97400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:34.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.479+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8a9808a4d0 con 0x7f8aa01527e0 2024-04-25T10:13:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.480+0000 7f8a9f400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.480+0000 7f8a9f400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8a90033000 tx=0x7f8a90008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:34.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.484+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8a980590f0 con 0x7f8aa01527e0 2024-04-25T10:13:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.690+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8aa01f7880 con 0x7f8aa01527e0 2024-04-25T10:13:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.691+0000 7f8a97400640 1 -- 172.21.15.165:0/3741972804 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v113) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8a980589f0 con 0x7f8aa01527e0 2024-04-25T10:13:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:13:34.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a7c06b4d0 msgr2=0x7f8a7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f8a90033000 tx=0x7f8a90008030 comp rx=0 tx=0).stop 2024-04-25T10:13:34.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 msgr2=0x7f8aa015c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:34.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 0x7f8aa015c330 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f8aa00af380 tx=0x7f8a9800b0d0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 shutdown_connections 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a7c06b4d0 0x7f8a7c06d990 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa01527e0 0x7f8aa015c330 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa0152360 0x7f8aa015bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 --2- 172.21.15.165:0/3741972804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a800039b0 0x7f8aa0156190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 >> 172.21.15.165:0/3741972804 conn(0x7f8aa006abe0 msgr2=0x7f8aa006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 shutdown_connections 2024-04-25T10:13:34.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:34.694+0000 7f8aa6400640 1 -- 172.21.15.165:0/3741972804 wait complete. 2024-04-25T10:13:34.717 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:13:34.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:34.718 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:13:35.588 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:13:35.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:13:35.589 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:13:35.628 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:13:35.629 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:35.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:35.670 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:13:35.670 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:35.714 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:35.715 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:13:35.715 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:35.762 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:35.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.965+0000 7f6cf4000640 1 Processor -- start 2024-04-25T10:13:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.965+0000 7f6cf4000640 1 -- start start 2024-04-25T10:13:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.967+0000 7f6cf4000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6cec0d0ad0 0x7f6cec168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.967+0000 7f6cf4000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec0b9940 con 0x7f6cec15dd30 2024-04-25T10:13:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.967+0000 7f6cf4000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec0d1df0 con 0x7f6cec0d2e90 2024-04-25T10:13:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.967+0000 7f6cf4000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cec0b7a50 con 0x7f6cec0d0ad0 2024-04-25T10:13:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.967+0000 7f6cf1600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6cec15dd30 0x7f6cec0b9230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34218/0 (socket says 172.21.15.165:34218) 2024-04-25T10:13:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.967+0000 7f6cf1600640 1 -- 172.21.15.165:0/3062004327 learned_addr learned my addr 172.21.15.165:0/3062004327 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3062004327 >> v2:172.21.15.172:3300/0 conn(0x7f6cec0d0ad0 0x7f6cec168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837973919 0 0) 0x7f6cec0b9940 con 0x7f6cec15dd30 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cd0003520 con 0x7f6cec15dd30 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895064575 0 0) 0x7f6cd0003520 con 0x7f6cec15dd30 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 >> v2:172.21.15.172:3300/0 conn(0x7f6cec0d0ad0 msgr2=0x7f6cec168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 --2- 172.21.15.165:0/3062004327 >> v2:172.21.15.172:3300/0 conn(0x7f6cec0d0ad0 0x7f6cec168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 >> v1:172.21.15.172:6789/0 conn(0x7f6cec0d2e90 legacy=0x7f6cec0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0d20d0 con 0x7f6cec15dd30 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6ce40019f0 con 0x7f6cec15dd30 2024-04-25T10:13:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 >> v1:172.21.15.165:6789/0 conn(0x7f6cec15dd30 legacy=0x7f6cec0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd00039b0 0x7f6cd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd0004800 0x7f6cd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd00052e0 0x7f6cd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0b7a50 con 0x7f6cd00052e0 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd000c1b0 con 0x7f6cd00039b0 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.968+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd000c3b0 con 0x7f6cd0004800 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd00052e0 0x7f6cd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf2000640 1 -- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd0004800 msgr2=0x7f6cd0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd0004800 0x7f6cd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf1600640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd00039b0 0x7f6cd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf2000640 1 -- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd00039b0 msgr2=0x7f6cd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cd00039b0 0x7f6cd0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.969+0000 7f6cf2000640 1 -- 172.21.15.165:0/3062004327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd0003520 con 0x7f6cd00052e0 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.970+0000 7f6cf2000640 1 -- 172.21.15.165:0/3062004327 reap_dead start 2024-04-25T10:13:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.970+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd00052e0 0x7f6cd000b9c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009a90 tx=0x7f6cd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278dbabdf587e0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.970+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc009690 con 0x7f6cd00052e0 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.970+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc004020 con 0x7f6cd00052e0 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.970+0000 7f6cebe00640 1 -- 172.21.15.165:0/3062004327 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc004410 con 0x7f6cd00052e0 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 -- 172.21.15.165:0/3062004327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd00052e0 msgr2=0x7f6cd000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd00052e0 0x7f6cd000b9c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc009a90 tx=0x7f6cd0005830 comp rx=0 tx=0).stop 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 -- 172.21.15.165:0/3062004327 shutdown_connections 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3062004327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cd00052e0 0x7f6cd000b9c0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 -- 172.21.15.165:0/3062004327 >> 172.21.15.165:0/3062004327 conn(0x7f6cec061c30 msgr2=0x7f6cec06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 -- 172.21.15.165:0/3062004327 shutdown_connections 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.971+0000 7f6cf4000640 1 -- 172.21.15.165:0/3062004327 wait complete. 2024-04-25T10:13:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.973+0000 7f6cf4000640 1 Processor -- start 2024-04-25T10:13:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.973+0000 7f6cf4000640 1 -- start start 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf4000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd00052e0 0x7f6cec1550f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf4000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cec0b3150 0x7f6cec0b35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf4000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 0x7f6cec0cbe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf4000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0d2970 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf4000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0d2690 con 0x7f6cec0b3150 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf4000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec0d1a10 con 0x7f6cd00052e0 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 0x7f6cec0cbe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 0x7f6cec0cbe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56834/0 (socket says 172.21.15.165:56834) 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.974+0000 7f6cf2000640 1 -- 172.21.15.165:0/3766175939 learned_addr learned my addr 172.21.15.165:0/3766175939 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf1600640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd00052e0 0x7f6cec1550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf0c00640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cec0b3150 0x7f6cec0b35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf2000640 1 -- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd00052e0 msgr2=0x7f6cec1550f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd00052e0 0x7f6cec1550f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf2000640 1 -- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cec0b3150 msgr2=0x7f6cec0b35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cec0b3150 0x7f6cec0b35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf2000640 1 -- 172.21.15.165:0/3766175939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec0cc5a0 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf1600640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd00052e0 0x7f6cec1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf0c00640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cec0b3150 0x7f6cec0b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf2000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 0x7f6cec0cbe40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02abb0 tx=0x7f6cdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc009690 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cdc03e070 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cdc003e20 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.975+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec0cc880 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.976+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec0cce40 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.977+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec073990 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.978+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cdc043460 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.978+0000 7f6ce9600640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ccc06b500 0x7f6ccc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.979+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6cdc0aa030 con 0x7f6cec0cb9c0 2024-04-25T10:13:35.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.979+0000 7f6cf1600640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ccc06b500 0x7f6ccc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:35.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.980+0000 7f6cf1600640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ccc06b500 0x7f6ccc06d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4001580 tx=0x7f6ce4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:35.983+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cdc078d80 con 0x7f6cec0cb9c0 2024-04-25T10:13:36.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.216+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cec160c00 con 0x7f6cec0cb9c0 2024-04-25T10:13:36.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.217+0000 7f6ce9600640 1 -- 172.21.15.165:0/3766175939 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f6cdc078680 con 0x7f6cec0cb9c0 2024-04-25T10:13:36.223 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:36.223 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ccc06b500 msgr2=0x7f6ccc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ccc06b500 0x7f6ccc06d9c0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4001580 tx=0x7f6ce4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 msgr2=0x7f6cec0cbe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 0x7f6cec0cbe40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7f6cdc02abb0 tx=0x7f6cdc004730 comp rx=0 tx=0).stop 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 shutdown_connections 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ccc06b500 0x7f6ccc06d9c0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cec0cb9c0 0x7f6cec0cbe40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cec0b3150 0x7f6cec0b35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 --2- 172.21.15.165:0/3766175939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cd00052e0 0x7f6cec1550f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.219+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 >> 172.21.15.165:0/3766175939 conn(0x7f6cec061c30 msgr2=0x7f6cec06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.220+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 shutdown_connections 2024-04-25T10:13:36.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.220+0000 7f6cf4000640 1 -- 172.21.15.165:0/3766175939 wait complete. 2024-04-25T10:13:36.243 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:36.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.429+0000 7f680f600640 1 Processor -- start 2024-04-25T10:13:36.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.429+0000 7f680f600640 1 -- start start 2024-04-25T10:13:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.431+0000 7f680f600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f68080cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.431+0000 7f680f600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6808071480 con 0x7f68080cfcf0 2024-04-25T10:13:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.431+0000 7f680f600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6808071760 con 0x7f68080ce590 2024-04-25T10:13:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.431+0000 7f680f600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808072090 con 0x7f6808154ee0 2024-04-25T10:13:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.431+0000 7f6807e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f68080cfcf0 0x7f68080cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:34222/0 (socket says 172.21.15.165:34222) 2024-04-25T10:13:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.431+0000 7f6807e00640 1 -- 172.21.15.165:0/2597249300 learned_addr learned my addr 172.21.15.165:0/2597249300 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f680cc00640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f68080cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2292505492 0 0) 0x7f6808071480 con 0x7f68080cfcf0 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67ec003520 con 0x7f68080cfcf0 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1208441180 0 0) 0x7f67ec003520 con 0x7f68080cfcf0 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 msgr2=0x7f68080cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f68080cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 >> v1:172.21.15.172:6789/0 conn(0x7f68080ce590 legacy=0x7f680806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6808071a40 con 0x7f68080cfcf0 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f67f4001670 con 0x7f68080cfcf0 2024-04-25T10:13:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 >> v1:172.21.15.165:6789/0 conn(0x7f68080cfcf0 legacy=0x7f68080cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec0039e0 0x7f67ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67ec004800 0x7f67ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f67ec005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec00bac0 con 0x7f67ec0039e0 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec00bc70 con 0x7f67ec004800 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.432+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67ec00be70 con 0x7f6808154ee0 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680cc00640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec0039e0 0x7f67ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f6807e00640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67ec004800 0x7f67ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680d600640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f67ec005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680cc00640 1 -- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 msgr2=0x7f67ec005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680cc00640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f67ec005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680cc00640 1 -- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67ec004800 msgr2=0x7f67ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680cc00640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67ec004800 0x7f67ec004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680cc00640 1 -- 172.21.15.165:0/2597249300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67ec003520 con 0x7f67ec0039e0 2024-04-25T10:13:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.433+0000 7f680d600640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6808154ee0 0x7f67ec005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.434+0000 7f680cc00640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec0039e0 0x7f67ec003e00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f67f800dd40 tx=0x7f67ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30ce1bf893e4f4b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.434+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f8033070 con 0x7f67ec0039e0 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.434+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67f802d940 con 0x7f67ec0039e0 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.434+0000 7f6807400640 1 -- 172.21.15.165:0/2597249300 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67f800d910 con 0x7f67ec0039e0 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680f600640 1 -- 172.21.15.165:0/2597249300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec0039e0 msgr2=0x7f67ec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680f600640 1 --2- 172.21.15.165:0/2597249300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67ec0039e0 0x7f67ec003e00 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f67f800dd40 tx=0x7f67ec005830 comp rx=0 tx=0).stop 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680d600640 1 -- 172.21.15.165:0/2597249300 reap_dead start 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680f600640 1 -- 172.21.15.165:0/2597249300 shutdown_connections 2024-04-25T10:13:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680f600640 1 -- 172.21.15.165:0/2597249300 >> 172.21.15.165:0/2597249300 conn(0x7f680806abe0 msgr2=0x7f680806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680f600640 1 -- 172.21.15.165:0/2597249300 shutdown_connections 2024-04-25T10:13:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.435+0000 7f680f600640 1 -- 172.21.15.165:0/2597249300 wait complete. 2024-04-25T10:13:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.437+0000 7f680f600640 1 Processor -- start 2024-04-25T10:13:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.437+0000 7f680f600640 1 -- start start 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680f600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67ec0039e0 0x7f68081f3a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680f600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808153af0 0x7f680815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680f600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 0x7f68081f7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680f600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6808071230 con 0x7f68081f7670 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680f600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68081552c0 con 0x7f6808153af0 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680f600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6808155440 con 0x7f67ec0039e0 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 0x7f68081f7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 0x7f68081f7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56858/0 (socket says 172.21.15.165:56858) 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.438+0000 7f680d600640 1 -- 172.21.15.165:0/1890877718 learned_addr learned my addr 172.21.15.165:0/1890877718 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f6807e00640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808153af0 0x7f680815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680cc00640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67ec0039e0 0x7f68081f3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680d600640 1 -- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67ec0039e0 msgr2=0x7f68081f3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680d600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67ec0039e0 0x7f68081f3a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680d600640 1 -- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808153af0 msgr2=0x7f680815d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680d600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808153af0 0x7f680815d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680d600640 1 -- 172.21.15.165:0/1890877718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68081f4990 con 0x7f68081f7670 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680cc00640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67ec0039e0 0x7f68081f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f680d600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 0x7f68081f7b00 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f68080620a0 tx=0x7f680000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.439+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800006d50 con 0x7f68081f7670 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.440+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6800003970 con 0x7f68081f7670 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.440+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6800003d60 con 0x7f68081f7670 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.440+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6808153f70 con 0x7f68081f7670 2024-04-25T10:13:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.440+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68081f40f0 con 0x7f68081f7670 2024-04-25T10:13:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.441+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6808073990 con 0x7f68081f7670 2024-04-25T10:13:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.442+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f680001b020 con 0x7f68081f7670 2024-04-25T10:13:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.443+0000 7f6804c00640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67e806b500 0x7f67e806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.443+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f6800089fd0 con 0x7f68081f7670 2024-04-25T10:13:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.443+0000 7f680cc00640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67e806b500 0x7f67e806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.444+0000 7f680cc00640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67e806b500 0x7f67e806d9c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f67f800dd10 tx=0x7f67f8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.447+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f680001ce40 con 0x7f68081f7670 2024-04-25T10:13:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.642+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f680806d360 con 0x7f68081f7670 2024-04-25T10:13:36.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.645+0000 7f6804c00640 1 -- 172.21.15.165:0/1890877718 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17018 (secure 0 0 0) 0x7f6800058c80 con 0x7f68081f7670 2024-04-25T10:13:36.650 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:36.650 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:48.513977+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1c","mappings":[{"from":1,"to":7}]},{"pgid":"16.31","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67e806b500 msgr2=0x7f67e806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67e806b500 0x7f67e806d9c0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f67f800dd10 tx=0x7f67f8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 msgr2=0x7f68081f7b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 0x7f68081f7b00 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f68080620a0 tx=0x7f680000dc00 comp rx=0 tx=0).stop 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 shutdown_connections 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67e806b500 0x7f67e806d9c0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68081f7670 0x7f68081f7b00 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6808153af0 0x7f680815d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 --2- 172.21.15.165:0/1890877718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67ec0039e0 0x7f68081f3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 >> 172.21.15.165:0/1890877718 conn(0x7f680806abe0 msgr2=0x7f68080b4470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 shutdown_connections 2024-04-25T10:13:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.648+0000 7f680f600640 1 -- 172.21.15.165:0/1890877718 wait complete. 2024-04-25T10:13:36.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:12:34.618582+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:12:35.883096+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '111', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:36.671 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:36.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.858+0000 7f43a1e00640 1 Processor -- start 2024-04-25T10:13:36.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.858+0000 7f43a1e00640 1 -- start start 2024-04-25T10:13:36.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.859+0000 7f43a1e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f439c15dd30 0x7f439c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.859+0000 7f43a1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f439c06bff0 0x7f439c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.859+0000 7f43a1e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f439c15e110 con 0x7f439c0b5970 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.859+0000 7f43a1e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c06d000 con 0x7f439c06bff0 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.859+0000 7f43a1e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439c0b2fd0 con 0x7f439c15dd30 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f439c06bff0 0x7f439c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f439c06bff0 0x7f439c0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40396/0 (socket says 172.21.15.165:40396) 2024-04-25T10:13:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 -- 172.21.15.165:0/500837275 learned_addr learned my addr 172.21.15.165:0/500837275 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:36.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 -- 172.21.15.165:0/500837275 >> v2:172.21.15.172:3300/0 conn(0x7f439c15dd30 msgr2=0x7f439c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 --2- 172.21.15.165:0/500837275 >> v2:172.21.15.172:3300/0 conn(0x7f439c15dd30 0x7f439c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 -- 172.21.15.165:0/500837275 >> v1:172.21.15.172:6789/0 conn(0x7f439c0b5970 legacy=0x7f439c161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:36.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.860+0000 7f439be00640 1 -- 172.21.15.165:0/500837275 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c06c480 con 0x7f439c06bff0 2024-04-25T10:13:36.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.861+0000 7f439be00640 1 --2- 172.21.15.165:0/500837275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c06bff0 0x7f439c0b5430 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f439c08bfe0 tx=0x7f438402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84ff1dc57dfcff8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.861+0000 7f439b400640 1 -- 172.21.15.165:0/500837275 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384037070 con 0x7f439c06bff0 2024-04-25T10:13:36.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f439b400640 1 -- 172.21.15.165:0/500837275 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4384032440 con 0x7f439c06bff0 2024-04-25T10:13:36.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f439b400640 1 -- 172.21.15.165:0/500837275 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4384032810 con 0x7f439c06bff0 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 -- 172.21.15.165:0/500837275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c06bff0 msgr2=0x7f439c0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 --2- 172.21.15.165:0/500837275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c06bff0 0x7f439c0b5430 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f439c08bfe0 tx=0x7f438402a230 comp rx=0 tx=0).stop 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 -- 172.21.15.165:0/500837275 shutdown_connections 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 --2- 172.21.15.165:0/500837275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c06bff0 0x7f439c0b5430 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 --2- 172.21.15.165:0/500837275 >> v2:172.21.15.172:3300/0 conn(0x7f439c15dd30 0x7f439c0cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 -- 172.21.15.165:0/500837275 >> 172.21.15.165:0/500837275 conn(0x7f439c0d0610 msgr2=0x7f439c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 -- 172.21.15.165:0/500837275 shutdown_connections 2024-04-25T10:13:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.862+0000 7f43a1e00640 1 -- 172.21.15.165:0/500837275 wait complete. 2024-04-25T10:13:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.864+0000 7f43a1e00640 1 Processor -- start 2024-04-25T10:13:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.864+0000 7f43a1e00640 1 -- start start 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.865+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 0x7f439c158620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.865+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f439c15dd30 0x7f439c0c9e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.865+0000 7f43a1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f439c153ff0 0x7f439c0b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.865+0000 7f43a1e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c06cd20 con 0x7f439c0b5970 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.865+0000 7f43a1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c06ca40 con 0x7f439c153ff0 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.865+0000 7f43a1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f439c0cf110 con 0x7f439c15dd30 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 0x7f439c158620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 0x7f439c158620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40408/0 (socket says 172.21.15.165:40408) 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 -- 172.21.15.165:0/3695364675 learned_addr learned my addr 172.21.15.165:0/3695364675 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f439be00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f439c15dd30 0x7f439c0c9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a1400640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f439c153ff0 0x7f439c0b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 -- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f439c15dd30 msgr2=0x7f439c0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f439c15dd30 0x7f439c0c9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 -- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f439c153ff0 msgr2=0x7f439c0b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f439c153ff0 0x7f439c0b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 -- 172.21.15.165:0/3695364675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c158c90 con 0x7f439c0b5970 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.866+0000 7f43a0a00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 0x7f439c158620 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f438c00ba50 tx=0x7f438c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.867+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c019070 con 0x7f439c0b5970 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.867+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438c00cc00 con 0x7f439c0b5970 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.867+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438c014440 con 0x7f439c0b5970 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.867+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c158f70 con 0x7f439c0b5970 2024-04-25T10:13:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.867+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c159530 con 0x7f439c0b5970 2024-04-25T10:13:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.868+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c073990 con 0x7f439c0b5970 2024-04-25T10:13:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.869+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f438c01c070 con 0x7f439c0b5970 2024-04-25T10:13:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.870+0000 7f4398c00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f437806b4d0 0x7f437806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.870+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f438c08a1b0 con 0x7f439c0b5970 2024-04-25T10:13:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.870+0000 7f439be00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f437806b4d0 0x7f437806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.871+0000 7f439be00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f437806b4d0 0x7f437806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4384034000 tx=0x7f438402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:36.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:36.874+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f438c058e60 con 0x7f439c0b5970 2024-04-25T10:13:37.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.068+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f439c0d16c0 con 0x7f439c0b5970 2024-04-25T10:13:37.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.069+0000 7f4398c00640 1 -- 172.21.15.165:0/3695364675 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7f438c058760 con 0x7f439c0b5970 2024-04-25T10:13:37.073 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f437806b4d0 msgr2=0x7f437806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f437806b4d0 0x7f437806d990 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f4384034000 tx=0x7f438402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 msgr2=0x7f439c158620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 0x7f439c158620 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f438c00ba50 tx=0x7f438c00bf10 comp rx=0 tx=0).stop 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 shutdown_connections 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f437806b4d0 0x7f437806d990 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f439c153ff0 0x7f439c0b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f439c15dd30 0x7f439c0c9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 --2- 172.21.15.165:0/3695364675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f439c0b5970 0x7f439c158620 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 >> 172.21.15.165:0/3695364675 conn(0x7f439c0d0610 msgr2=0x7f439c0b8610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 shutdown_connections 2024-04-25T10:13:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.071+0000 7f43a1e00640 1 -- 172.21.15.165:0/3695364675 wait complete. 2024-04-25T10:13:37.096 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:13:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.295+0000 7fb798c00640 1 Processor -- start 2024-04-25T10:13:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.295+0000 7fb798c00640 1 -- start start 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb798c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb7941452a0 0x7fb79406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb798c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7940cccc0 con 0x7fb794144d50 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb798c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7940cc140 con 0x7fb7941446d0 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb798c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7940b2b20 con 0x7fb7941452a0 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb792800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb7941452a0 0x7fb79406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb792800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb7941452a0 0x7fb79406dfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40422/0 (socket says 172.21.15.165:40422) 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb792800640 1 -- 172.21.15.165:0/463255525 learned_addr learned my addr 172.21.15.165:0/463255525 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:37.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb791400640 1 --1- 172.21.15.165:0/463255525 >> v1:172.21.15.165:6789/0 conn(0x7fb794144d50 0x7fb79406d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48184/0 (socket says 172.21.15.165:48184) 2024-04-25T10:13:37.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb792800640 1 -- 172.21.15.165:0/463255525 >> v1:172.21.15.172:6789/0 conn(0x7fb7941446d0 legacy=0x7fb7940b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.297+0000 7fb792800640 1 -- 172.21.15.165:0/463255525 >> v1:172.21.15.165:6789/0 conn(0x7fb794144d50 legacy=0x7fb79406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.298+0000 7fb792800640 1 -- 172.21.15.165:0/463255525 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7940cc420 con 0x7fb7941452a0 2024-04-25T10:13:37.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.299+0000 7fb792800640 1 --2- 172.21.15.165:0/463255525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7941452a0 0x7fb79406dfa0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009e50 tx=0x7fb78c02a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af4aebde02609d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.299+0000 7fb790a00640 1 -- 172.21.15.165:0/463255525 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c036070 con 0x7fb7941452a0 2024-04-25T10:13:37.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.299+0000 7fb790a00640 1 -- 172.21.15.165:0/463255525 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb78c031600 con 0x7fb7941452a0 2024-04-25T10:13:37.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.299+0000 7fb790a00640 1 -- 172.21.15.165:0/463255525 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb78c0319d0 con 0x7fb7941452a0 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.299+0000 7fb798c00640 1 -- 172.21.15.165:0/463255525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7941452a0 msgr2=0x7fb79406dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.299+0000 7fb798c00640 1 --2- 172.21.15.165:0/463255525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7941452a0 0x7fb79406dfa0 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7fb78c009e50 tx=0x7fb78c02a760 comp rx=0 tx=0).stop 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.300+0000 7fb798c00640 1 -- 172.21.15.165:0/463255525 shutdown_connections 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.300+0000 7fb798c00640 1 --2- 172.21.15.165:0/463255525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7941452a0 0x7fb79406dfa0 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.300+0000 7fb798c00640 1 -- 172.21.15.165:0/463255525 >> 172.21.15.165:0/463255525 conn(0x7fb79406abe0 msgr2=0x7fb79406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.300+0000 7fb798c00640 1 -- 172.21.15.165:0/463255525 shutdown_connections 2024-04-25T10:13:37.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.300+0000 7fb798c00640 1 -- 172.21.15.165:0/463255525 wait complete. 2024-04-25T10:13:37.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.301+0000 7fb798c00640 1 Processor -- start 2024-04-25T10:13:37.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.302+0000 7fb798c00640 1 -- start start 2024-04-25T10:13:37.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.302+0000 7fb798c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7941446d0 0x7fb794142cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.302+0000 7fb798c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 0x7fb79415ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb798c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7941452a0 0x7fb79415b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb798c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7940cc9e0 con 0x7fb794144d50 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb798c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79406d400 con 0x7fb7941446d0 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb798c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb794144ab0 con 0x7fb7941452a0 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 0x7fb79415ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 0x7fb79415ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40424/0 (socket says 172.21.15.165:40424) 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 -- 172.21.15.165:0/3634878805 learned_addr learned my addr 172.21.15.165:0/3634878805 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791e00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7941446d0 0x7fb794142cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 -- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7941452a0 msgr2=0x7fb79415b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7941452a0 0x7fb79415b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 -- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7941446d0 msgr2=0x7fb794142cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7941446d0 0x7fb794142cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.303+0000 7fb791400640 1 -- 172.21.15.165:0/3634878805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7941f5330 con 0x7fb794144d50 2024-04-25T10:13:37.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.304+0000 7fb791400640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 0x7fb79415ada0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c3f0 tx=0x7fb78000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.304+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7800074e0 con 0x7fb794144d50 2024-04-25T10:13:37.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.304+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb780007680 con 0x7fb794144d50 2024-04-25T10:13:37.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.304+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb780004a00 con 0x7fb794144d50 2024-04-25T10:13:37.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.304+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7941f5610 con 0x7fb794144d50 2024-04-25T10:13:37.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.304+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7941f5bd0 con 0x7fb794144d50 2024-04-25T10:13:37.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.305+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb794073a00 con 0x7fb794144d50 2024-04-25T10:13:37.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.306+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb78001a020 con 0x7fb794144d50 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.308+0000 7fb78a000640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb77406b4d0 0x7fb77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.308+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fb780089b30 con 0x7fb794144d50 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.308+0000 7fb791e00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb77406b4d0 0x7fb77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.309+0000 7fb791e00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb77406b4d0 0x7fb77406d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.312+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb780010300 con 0x7fb794144d50 2024-04-25T10:13:37.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.505+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb7940b43f0 con 0x7fb794144d50 2024-04-25T10:13:37.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.506+0000 7fb78a000640 1 -- 172.21.15.165:0/3634878805 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb7800587e0 con 0x7fb794144d50 2024-04-25T10:13:37.511 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:13:37.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb77406b4d0 msgr2=0x7fb77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb77406b4d0 0x7fb77406d990 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fb784006fd0 tx=0x7fb784008040 comp rx=0 tx=0).stop 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 msgr2=0x7fb79415ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 0x7fb79415ada0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7fb78000c3f0 tx=0x7fb78000c8b0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 shutdown_connections 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb77406b4d0 0x7fb77406d990 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7941452a0 0x7fb79415b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb794144d50 0x7fb79415ada0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 --2- 172.21.15.165:0/3634878805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7941446d0 0x7fb794142cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 >> 172.21.15.165:0/3634878805 conn(0x7fb79406abe0 msgr2=0x7fb7940af3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.509+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 shutdown_connections 2024-04-25T10:13:37.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.510+0000 7fb798c00640 1 -- 172.21.15.165:0/3634878805 wait complete. 2024-04-25T10:13:37.533 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:13:37.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.730+0000 7f79f8400640 1 Processor -- start 2024-04-25T10:13:37.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.730+0000 7f79f8400640 1 -- start start 2024-04-25T10:13:37.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.731+0000 7f79f8400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79f0144d50 0x7f79f006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.731+0000 7f79f8400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f79f01452a0 0x7f79f006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.731+0000 7f79f8400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00cccc0 con 0x7f79f01446d0 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.731+0000 7f79f8400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f00b2b20 con 0x7f79f01452a0 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.731+0000 7f79f8400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f006d400 con 0x7f79f0144d50 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f5a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f79f01446d0 0x7f79f00b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48192/0 (socket says 172.21.15.165:48192) 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f5a00640 1 -- 172.21.15.165:0/3989564838 learned_addr learned my addr 172.21.15.165:0/3989564838 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:37.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f6400640 1 --2- 172.21.15.165:0/3989564838 >> v2:172.21.15.165:3300/0 conn(0x7f79f01452a0 0x7f79f006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f5000640 1 --2- 172.21.15.165:0/3989564838 >> v2:172.21.15.172:3300/0 conn(0x7f79f0144d50 0x7f79f006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79efe00640 1 -- 172.21.15.165:0/3989564838 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1254081261 0 0) 0x7f79f00cccc0 con 0x7f79f01446d0 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79efe00640 1 -- 172.21.15.165:0/3989564838 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79d8003520 con 0x7f79f01446d0 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f6400640 1 -- 172.21.15.165:0/3989564838 >> v2:172.21.15.172:3300/0 conn(0x7f79f0144d50 msgr2=0x7f79f006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f6400640 1 --2- 172.21.15.165:0/3989564838 >> v2:172.21.15.172:3300/0 conn(0x7f79f0144d50 0x7f79f006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f6400640 1 -- 172.21.15.165:0/3989564838 >> v1:172.21.15.165:6789/0 conn(0x7f79f01446d0 legacy=0x7f79f00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f6400640 1 -- 172.21.15.165:0/3989564838 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00cc140 con 0x7f79f01452a0 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.732+0000 7f79f5000640 1 --2- 172.21.15.165:0/3989564838 >> v2:172.21.15.172:3300/0 conn(0x7f79f0144d50 0x7f79f006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.733+0000 7f79f6400640 1 --2- 172.21.15.165:0/3989564838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f01452a0 0x7f79f006ddd0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f79f0071c70 tx=0x7f79e802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8d586aeb1a7465f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.733+0000 7f79efe00640 1 -- 172.21.15.165:0/3989564838 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8007740 con 0x7f79f01452a0 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79efe00640 1 -- 172.21.15.165:0/3989564838 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e80078e0 con 0x7f79f01452a0 2024-04-25T10:13:37.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79efe00640 1 -- 172.21.15.165:0/3989564838 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e8007cb0 con 0x7f79f01452a0 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 -- 172.21.15.165:0/3989564838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f01452a0 msgr2=0x7f79f006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 --2- 172.21.15.165:0/3989564838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f01452a0 0x7f79f006ddd0 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f79f0071c70 tx=0x7f79e802a680 comp rx=0 tx=0).stop 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 -- 172.21.15.165:0/3989564838 shutdown_connections 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 --2- 172.21.15.165:0/3989564838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f01452a0 0x7f79f006ddd0 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 --2- 172.21.15.165:0/3989564838 >> v2:172.21.15.172:3300/0 conn(0x7f79f0144d50 0x7f79f006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 -- 172.21.15.165:0/3989564838 >> 172.21.15.165:0/3989564838 conn(0x7f79f006abe0 msgr2=0x7f79f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 -- 172.21.15.165:0/3989564838 shutdown_connections 2024-04-25T10:13:37.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.734+0000 7f79f8400640 1 -- 172.21.15.165:0/3989564838 wait complete. 2024-04-25T10:13:37.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.736+0000 7f79f8400640 1 Processor -- start 2024-04-25T10:13:37.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.736+0000 7f79f8400640 1 -- start start 2024-04-25T10:13:37.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f8400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79f01446d0 0x7f79f00cee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f8400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 0x7f79f013de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f8400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79f01452a0 0x7f79f01f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f8400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00cc9e0 con 0x7f79f0144d50 2024-04-25T10:13:37.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f8400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00cc700 con 0x7f79f01446d0 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f8400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f0144ab0 con 0x7f79f01452a0 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.737+0000 7f79f5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 0x7f79f013de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 0x7f79f013de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40440/0 (socket says 172.21.15.165:40440) 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 -- 172.21.15.165:0/3892409157 learned_addr learned my addr 172.21.15.165:0/3892409157 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f6400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79f01452a0 0x7f79f01f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5a00640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79f01446d0 0x7f79f00cee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 -- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79f01452a0 msgr2=0x7f79f01f6ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79f01452a0 0x7f79f01f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 -- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79f01446d0 msgr2=0x7f79f00cee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79f01446d0 0x7f79f00cee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 -- 172.21.15.165:0/3892409157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f01f7a30 con 0x7f79f0144d50 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5a00640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79f01446d0 0x7f79f00cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.738+0000 7f79f5000640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 0x7f79f013de20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c7f0 tx=0x7f79dc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.739+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc015070 con 0x7f79f0144d50 2024-04-25T10:13:37.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.739+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc007a30 con 0x7f79f0144d50 2024-04-25T10:13:37.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.739+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc007e00 con 0x7f79f0144d50 2024-04-25T10:13:37.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.739+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f01f7d10 con 0x7f79f0144d50 2024-04-25T10:13:37.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.739+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f00ce610 con 0x7f79f0144d50 2024-04-25T10:13:37.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.740+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f0073a00 con 0x7f79f0144d50 2024-04-25T10:13:37.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.741+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f79dc018070 con 0x7f79f0144d50 2024-04-25T10:13:37.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.742+0000 7f79ed600640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79d006b4d0 0x7f79d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:37.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.742+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f79dc08a380 con 0x7f79f0144d50 2024-04-25T10:13:37.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.742+0000 7f79f5a00640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79d006b4d0 0x7f79d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:37.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.743+0000 7f79f5a00640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79d006b4d0 0x7f79d006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005880 tx=0x7f79e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:37.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.746+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f79dc059030 con 0x7f79f0144d50 2024-04-25T10:13:37.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.940+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79f0142b90 con 0x7f79f0144d50 2024-04-25T10:13:37.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.941+0000 7f79ed600640 1 -- 172.21.15.165:0/3892409157 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79dc058930 con 0x7f79f0144d50 2024-04-25T10:13:37.945 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.943+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79d006b4d0 msgr2=0x7f79d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.943+0000 7f79f8400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79d006b4d0 0x7f79d006d990 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f79e4005880 tx=0x7f79e4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.943+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 msgr2=0x7f79f013de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.943+0000 7f79f8400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 0x7f79f013de20 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f79dc00c7f0 tx=0x7f79dc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 shutdown_connections 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f79d006b4d0 0x7f79d006d990 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f79f01452a0 0x7f79f01f6ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79f0144d50 0x7f79f013de20 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 --2- 172.21.15.165:0/3892409157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79f01446d0 0x7f79f00cee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:37.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 >> 172.21.15.165:0/3892409157 conn(0x7f79f006abe0 msgr2=0x7f79f006e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:37.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 shutdown_connections 2024-04-25T10:13:37.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:37.944+0000 7f79f8400640 1 -- 172.21.15.165:0/3892409157 wait complete. 2024-04-25T10:13:37.968 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:38.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.156+0000 7ff905800640 1 Processor -- start 2024-04-25T10:13:38.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.156+0000 7ff905800640 1 -- start start 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff905800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff900144440 0x7ff900070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff905800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff900071530 0x7ff9000cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff905800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff90015bf30 con 0x7ff9000cd860 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff905800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff90015c860 con 0x7ff900071530 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff905800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff900071970 con 0x7ff900144440 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff8ff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff900071530 0x7ff9000cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff8ff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff900071530 0x7ff9000cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40452/0 (socket says 172.21.15.165:40452) 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.158+0000 7ff8ff400640 1 -- 172.21.15.165:0/3959591607 learned_addr learned my addr 172.21.15.165:0/3959591607 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.159+0000 7ff8ffe00640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff900144440 0x7ff900070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.159+0000 7ff8ff400640 1 -- 172.21.15.165:0/3959591607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff900144440 msgr2=0x7ff900070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.159+0000 7ff8ff400640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff900144440 0x7ff900070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.159+0000 7ff8ff400640 1 -- 172.21.15.165:0/3959591607 >> v1:172.21.15.172:6789/0 conn(0x7ff9000cd860 legacy=0x7ff90006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.159+0000 7ff8ff400640 1 -- 172.21.15.165:0/3959591607 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90015c210 con 0x7ff900071530 2024-04-25T10:13:38.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.159+0000 7ff8ffe00640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff900144440 0x7ff900070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:38.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.160+0000 7ff8ff400640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000cd320 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba4ef3dac19e5a9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.160+0000 7ff8fea00640 1 -- 172.21.15.165:0/3959591607 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec037070 con 0x7ff900071530 2024-04-25T10:13:38.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.160+0000 7ff8fea00640 1 -- 172.21.15.165:0/3959591607 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8ec032440 con 0x7ff900071530 2024-04-25T10:13:38.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.160+0000 7ff8fea00640 1 -- 172.21.15.165:0/3959591607 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec032810 con 0x7ff900071530 2024-04-25T10:13:38.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 -- 172.21.15.165:0/3959591607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 msgr2=0x7ff9000cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000cd320 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec009870 tx=0x7ff8ec02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 -- 172.21.15.165:0/3959591607 shutdown_connections 2024-04-25T10:13:38.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000cd320 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 --2- 172.21.15.165:0/3959591607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff900144440 0x7ff900070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 -- 172.21.15.165:0/3959591607 >> 172.21.15.165:0/3959591607 conn(0x7ff90006abe0 msgr2=0x7ff90006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 -- 172.21.15.165:0/3959591607 shutdown_connections 2024-04-25T10:13:38.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.161+0000 7ff905800640 1 -- 172.21.15.165:0/3959591607 wait complete. 2024-04-25T10:13:38.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.163+0000 7ff905800640 1 Processor -- start 2024-04-25T10:13:38.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.163+0000 7ff905800640 1 -- start start 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff905800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000acfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff905800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9000cd860 0x7ff9000ad520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff905800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff900144440 0x7ff9001f32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff905800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000cdcd0 con 0x7ff900071530 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff905800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff90015bc30 con 0x7ff900144440 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff905800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9000cde50 con 0x7ff9000cd860 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff8ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000acfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff8ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000acfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40466/0 (socket says 172.21.15.165:40466) 2024-04-25T10:13:38.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.164+0000 7ff8ffe00640 1 -- 172.21.15.165:0/1361299662 learned_addr learned my addr 172.21.15.165:0/1361299662 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ffe00640 1 -- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9000cd860 msgr2=0x7ff9000ad520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ff400640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9000cd860 0x7ff9000ad520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ffe00640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9000cd860 0x7ff9000ad520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ffe00640 1 -- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff900144440 msgr2=0x7ff9001f32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ffe00640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff900144440 0x7ff9001f32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ffe00640 1 -- 172.21.15.165:0/1361299662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9000b2ba0 con 0x7ff900071530 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8ffe00640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000acfe0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c7f0 tx=0x7ff8f000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0015070 con 0x7ff900071530 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8f00077c0 con 0x7ff900071530 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8f0007b70 con 0x7ff900071530 2024-04-25T10:13:38.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9000b2e80 con 0x7ff900071530 2024-04-25T10:13:38.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.165+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9000b3440 con 0x7ff900071530 2024-04-25T10:13:38.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.166+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900073990 con 0x7ff900071530 2024-04-25T10:13:38.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.167+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8f0018070 con 0x7ff900071530 2024-04-25T10:13:38.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.168+0000 7ff8f7e00640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8e006b4d0 0x7ff8e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.168+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff8f008a450 con 0x7ff900071530 2024-04-25T10:13:38.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.169+0000 7ff8ff400640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8e006b4d0 0x7ff8e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.169+0000 7ff8ff400640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8e006b4d0 0x7ff8e006d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec034000 tx=0x7ff8ec02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.173+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8f0059180 con 0x7ff900071530 2024-04-25T10:13:38.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.407+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff9000b2370 con 0x7ff900071530 2024-04-25T10:13:38.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.408+0000 7ff8f7e00640 1 -- 172.21.15.165:0/1361299662 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7ff8f0058a80 con 0x7ff900071530 2024-04-25T10:13:38.413 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:38.413 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:38.413 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:38.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8e006b4d0 msgr2=0x7ff8e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8e006b4d0 0x7ff8e006d990 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec034000 tx=0x7ff8ec02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:38.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 msgr2=0x7ff9000acfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000acfe0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7ff8f000c7f0 tx=0x7ff8f000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 shutdown_connections 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8e006b4d0 0x7ff8e006d990 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff900144440 0x7ff9001f32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9000cd860 0x7ff9000ad520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 --2- 172.21.15.165:0/1361299662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff900071530 0x7ff9000acfe0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.411+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 >> 172.21.15.165:0/1361299662 conn(0x7ff90006abe0 msgr2=0x7ff9000c58c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.412+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 shutdown_connections 2024-04-25T10:13:38.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.412+0000 7ff905800640 1 -- 172.21.15.165:0/1361299662 wait complete. 2024-04-25T10:13:38.435 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.631+0000 7efe13a00640 1 Processor -- start 2024-04-25T10:13:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.631+0000 7efe13a00640 1 -- start start 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe13a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efe0c0ce920 0x7efe0c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe13a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c06d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe13a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0c144b30 con 0x7efe0c144750 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe13a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0c0712d0 con 0x7efe0c0ce920 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe13a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c0715b0 con 0x7efe0c06d290 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe0be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efe0c0ce920 0x7efe0c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe0be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efe0c0ce920 0x7efe0c0cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40478/0 (socket says 172.21.15.165:40478) 2024-04-25T10:13:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe0be00640 1 -- 172.21.15.165:0/3637008285 learned_addr learned my addr 172.21.15.165:0/3637008285 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.633+0000 7efe11a00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c06d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.634+0000 7efe0be00640 1 -- 172.21.15.165:0/3637008285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 msgr2=0x7efe0c06d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.634+0000 7efe0be00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c06d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.634+0000 7efe0be00640 1 -- 172.21.15.165:0/3637008285 >> v1:172.21.15.172:6789/0 conn(0x7efe0c144750 legacy=0x7efe0c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.634+0000 7efe0be00640 1 -- 172.21.15.165:0/3637008285 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c071890 con 0x7efe0c0ce920 2024-04-25T10:13:38.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.634+0000 7efe11a00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.635+0000 7efe0be00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c0ce920 0x7efe0c0cedb0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009870 tx=0x7efdf802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817af108d6e35650 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.635+0000 7efe0b400640 1 -- 172.21.15.165:0/3637008285 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8037070 con 0x7efe0c0ce920 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.635+0000 7efe0b400640 1 -- 172.21.15.165:0/3637008285 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8032440 con 0x7efe0c0ce920 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.635+0000 7efe0b400640 1 -- 172.21.15.165:0/3637008285 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8032810 con 0x7efe0c0ce920 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 -- 172.21.15.165:0/3637008285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c0ce920 msgr2=0x7efe0c0cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c0ce920 0x7efe0c0cedb0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7efdf8009870 tx=0x7efdf802a210 comp rx=0 tx=0).stop 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 -- 172.21.15.165:0/3637008285 shutdown_connections 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c06d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 --2- 172.21.15.165:0/3637008285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c0ce920 0x7efe0c0cedb0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 -- 172.21.15.165:0/3637008285 >> 172.21.15.165:0/3637008285 conn(0x7efe0c06abe0 msgr2=0x7efe0c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 -- 172.21.15.165:0/3637008285 shutdown_connections 2024-04-25T10:13:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.636+0000 7efe13a00640 1 -- 172.21.15.165:0/3637008285 wait complete. 2024-04-25T10:13:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.637+0000 7efe13a00640 1 Processor -- start 2024-04-25T10:13:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.638+0000 7efe13a00640 1 -- start start 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.638+0000 7efe13a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe13a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe0c0ce920 0x7efe0c15d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe13a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 0x7efe0c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe13a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c06f020 con 0x7efe0c144750 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe13a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c06ece0 con 0x7efe0c0ce920 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe13a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe0c145450 con 0x7efe0c06d290 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 0x7efe0c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 0x7efe0c15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40492/0 (socket says 172.21.15.165:40492) 2024-04-25T10:13:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 -- 172.21.15.165:0/3702676232 learned_addr learned my addr 172.21.15.165:0/3702676232 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11000640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe0be00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe0c0ce920 0x7efe0c15d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 -- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 msgr2=0x7efe0c0afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c0afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 -- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe0c0ce920 msgr2=0x7efe0c15d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe0c0ce920 0x7efe0c15d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.639+0000 7efe11a00640 1 -- 172.21.15.165:0/3702676232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0c0b05b0 con 0x7efe0c144750 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.640+0000 7efe11a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 0x7efe0c15dd80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7efe0400cd50 tx=0x7efe0400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.640+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400f680 con 0x7efe0c144750 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.640+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe0400f840 con 0x7efe0c144750 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.640+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe0400fc30 con 0x7efe0c144750 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.640+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe0c0b0890 con 0x7efe0c144750 2024-04-25T10:13:38.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.640+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0c1f1850 con 0x7efe0c144750 2024-04-25T10:13:38.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.641+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe0c0b0db0 con 0x7efe0c144750 2024-04-25T10:13:38.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.642+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe0400e3e0 con 0x7efe0c144750 2024-04-25T10:13:38.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.643+0000 7efe08c00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdec06b4d0 0x7efdec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:38.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.643+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7efe0408b1b0 con 0x7efe0c144750 2024-04-25T10:13:38.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.643+0000 7efe11000640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdec06b4d0 0x7efdec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:38.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.644+0000 7efe11000640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdec06b4d0 0x7efdec06d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efe00003d90 tx=0x7efe00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:38.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.647+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe0c0b0db0 con 0x7efe0c144750 2024-04-25T10:13:38.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.845+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe0c073990 con 0x7efe0c144750 2024-04-25T10:13:38.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.847+0000 7efe08c00640 1 -- 172.21.15.165:0/3702676232 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17018 (secure 0 0 0) 0x7efe04014030 con 0x7efe0c144750 2024-04-25T10:13:38.852 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:38.852 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:48.513977+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1c","mappings":[{"from":1,"to":7}]},{"pgid":"16.31","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:38.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdec06b4d0 msgr2=0x7efdec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdec06b4d0 0x7efdec06d990 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efe00003d90 tx=0x7efe00009040 comp rx=0 tx=0).stop 2024-04-25T10:13:38.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 msgr2=0x7efe0c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:38.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 0x7efe0c15dd80 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7efe0400cd50 tx=0x7efe0400a650 comp rx=0 tx=0).stop 2024-04-25T10:13:38.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 shutdown_connections 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdec06b4d0 0x7efdec06d990 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe0c144750 0x7efe0c15dd80 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe0c0ce920 0x7efe0c15d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 --2- 172.21.15.165:0/3702676232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe0c06d290 0x7efe0c0afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 >> 172.21.15.165:0/3702676232 conn(0x7efe0c06abe0 msgr2=0x7efe0c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 shutdown_connections 2024-04-25T10:13:38.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:38.850+0000 7efe13a00640 1 -- 172.21.15.165:0/3702676232 wait complete. 2024-04-25T10:13:38.878 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:13:38.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.072+0000 7fc19aa00640 1 Processor -- start 2024-04-25T10:13:39.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.072+0000 7fc19aa00640 1 -- start start 2024-04-25T10:13:39.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.073+0000 7fc19aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc194144750 0x7fc19406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc19aa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc194144b30 con 0x7fc1940cf030 2024-04-25T10:13:39.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc19aa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1940712d0 con 0x7fc19406eb10 2024-04-25T10:13:39.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc19aa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1940715b0 con 0x7fc194144750 2024-04-25T10:13:39.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc198a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc1940cf030 0x7fc19406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48198/0 (socket says 172.21.15.165:48198) 2024-04-25T10:13:39.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc198a00640 1 -- 172.21.15.165:0/1509296221 learned_addr learned my addr 172.21.15.165:0/1509296221 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:39.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc193e00640 1 --2- 172.21.15.165:0/1509296221 >> v2:172.21.15.172:3300/0 conn(0x7fc194144750 0x7fc19406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.074+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3695920464 0 0) 0x7fc194144b30 con 0x7fc1940cf030 2024-04-25T10:13:39.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc178003520 con 0x7fc1940cf030 2024-04-25T10:13:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3174957755 0 0) 0x7fc178003520 con 0x7fc1940cf030 2024-04-25T10:13:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 >> v2:172.21.15.172:3300/0 conn(0x7fc194144750 msgr2=0x7fc19406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 --2- 172.21.15.165:0/1509296221 >> v2:172.21.15.172:3300/0 conn(0x7fc194144750 0x7fc19406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 >> v1:172.21.15.172:6789/0 conn(0x7fc19406eb10 legacy=0x7fc1940ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc194071890 con 0x7fc1940cf030 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc18c0019f0 con 0x7fc1940cf030 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 >> v1:172.21.15.165:6789/0 conn(0x7fc1940cf030 legacy=0x7fc19406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc178003a00 0x7fc178003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc178004800 0x7fc178004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1780051c0 0x7fc17800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1940715b0 con 0x7fc1780051c0 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc17800c0c0 con 0x7fc178003a00 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc17800c2c0 con 0x7fc178004800 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.075+0000 7fc198a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1780051c0 0x7fc17800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.076+0000 7fc193400640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc178004800 0x7fc178004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.076+0000 7fc198a00640 1 -- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc178004800 msgr2=0x7fc178004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.076+0000 7fc198a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc178004800 0x7fc178004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.076+0000 7fc198a00640 1 -- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc178003a00 msgr2=0x7fc178003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.076+0000 7fc198a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc178003a00 0x7fc178003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.076+0000 7fc198a00640 1 -- 172.21.15.165:0/1509296221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc178003520 con 0x7fc1780051c0 2024-04-25T10:13:39.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.077+0000 7fc198a00640 1 -- 172.21.15.165:0/1509296221 reap_dead start 2024-04-25T10:13:39.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.077+0000 7fc198a00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1780051c0 0x7fc17800b9c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00a790 tx=0x7fc178005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d4d72dc7a2c65b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.077+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c00a3b0 con 0x7fc1780051c0 2024-04-25T10:13:39.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.077+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c02f070 con 0x7fc1780051c0 2024-04-25T10:13:39.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.077+0000 7fc192a00640 1 -- 172.21.15.165:0/1509296221 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02bc20 con 0x7fc1780051c0 2024-04-25T10:13:39.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 -- 172.21.15.165:0/1509296221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1780051c0 msgr2=0x7fc17800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1780051c0 0x7fc17800b9c0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fc18c00a790 tx=0x7fc178005830 comp rx=0 tx=0).stop 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 -- 172.21.15.165:0/1509296221 shutdown_connections 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 --2- 172.21.15.165:0/1509296221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc1780051c0 0x7fc17800b9c0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 -- 172.21.15.165:0/1509296221 >> 172.21.15.165:0/1509296221 conn(0x7fc19406abe0 msgr2=0x7fc19406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 -- 172.21.15.165:0/1509296221 shutdown_connections 2024-04-25T10:13:39.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.078+0000 7fc19aa00640 1 -- 172.21.15.165:0/1509296221 wait complete. 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.080+0000 7fc19aa00640 1 Processor -- start 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.080+0000 7fc19aa00640 1 -- start start 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1780051c0 0x7fc1941ef870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19415acb0 0x7fc194139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc19aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 0x7fc19413a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc19aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1941455d0 con 0x7fc19415b130 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc19aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc194145450 con 0x7fc19415acb0 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc19aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19415c000 con 0x7fc1780051c0 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc198a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 0x7fc19413a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc198a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 0x7fc19413a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40508/0 (socket says 172.21.15.165:40508) 2024-04-25T10:13:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc198a00640 1 -- 172.21.15.165:0/2339973578 learned_addr learned my addr 172.21.15.165:0/2339973578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.081+0000 7fc193e00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1780051c0 0x7fc1941ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc193400640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19415acb0 0x7fc194139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc198a00640 1 -- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1780051c0 msgr2=0x7fc1941ef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc198a00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1780051c0 0x7fc1941ef870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc198a00640 1 -- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19415acb0 msgr2=0x7fc194139db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc198a00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19415acb0 0x7fc194139db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc198a00640 1 -- 172.21.15.165:0/2339973578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1941efdb0 con 0x7fc19415b130 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc193e00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1780051c0 0x7fc1941ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc198a00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 0x7fc19413a2f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc18c02b450 tx=0x7fc18c02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02b7a0 con 0x7fc19415b130 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c02b940 con 0x7fc19415b130 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc194135040 con 0x7fc19415b130 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c02bcf0 con 0x7fc19415b130 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.082+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1941355e0 con 0x7fc19415b130 2024-04-25T10:13:39.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.084+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc194073990 con 0x7fc19415b130 2024-04-25T10:13:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.085+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc18c034470 con 0x7fc19415b130 2024-04-25T10:13:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.086+0000 7fc18be00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17406b4d0 0x7fc17406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.086+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc18c0aa340 con 0x7fc19415b130 2024-04-25T10:13:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.086+0000 7fc193e00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17406b4d0 0x7fc17406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.087+0000 7fc193e00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17406b4d0 0x7fc17406d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc1941f2d00 tx=0x7fc184008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.090+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc1941efdb0 con 0x7fc19415b130 2024-04-25T10:13:39.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.314+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1940d0510 con 0x7fc19415b130 2024-04-25T10:13:39.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.315+0000 7fc18be00640 1 -- 172.21.15.165:0/2339973578 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7fc18c078f60 con 0x7fc19415b130 2024-04-25T10:13:39.319 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:39.319 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:39.319 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:39.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17406b4d0 msgr2=0x7fc17406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17406b4d0 0x7fc17406d990 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc1941f2d00 tx=0x7fc184008040 comp rx=0 tx=0).stop 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 msgr2=0x7fc19413a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 0x7fc19413a2f0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7fc18c02b450 tx=0x7fc18c02d070 comp rx=0 tx=0).stop 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 shutdown_connections 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17406b4d0 0x7fc17406d990 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19415b130 0x7fc19413a2f0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19415acb0 0x7fc194139db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 --2- 172.21.15.165:0/2339973578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc1780051c0 0x7fc1941ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.317+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 >> 172.21.15.165:0/2339973578 conn(0x7fc19406abe0 msgr2=0x7fc1940b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.318+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 shutdown_connections 2024-04-25T10:13:39.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.318+0000 7fc19aa00640 1 -- 172.21.15.165:0/2339973578 wait complete. 2024-04-25T10:13:39.340 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:39.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.530+0000 7f4725200640 1 Processor -- start 2024-04-25T10:13:39.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.530+0000 7f4725200640 1 -- start start 2024-04-25T10:13:39.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f4725200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f472015db90 0x7f47200b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f4725200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47201615a0 0x7f47200cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f4725200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f4720061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f4725200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47200d0980 con 0x7f472015db90 2024-04-25T10:13:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f4725200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47200d1f50 con 0x7f47201615a0 2024-04-25T10:13:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f4725200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4720161df0 con 0x7f47200d04f0 2024-04-25T10:13:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f471f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f472015db90 0x7f47200b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f471f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f472015db90 0x7f47200b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40516/0 (socket says 172.21.15.165:40516) 2024-04-25T10:13:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f471f400640 1 -- 172.21.15.165:0/1848736977 learned_addr learned my addr 172.21.15.165:0/1848736977 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:39.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f471ea00640 1 --2- 172.21.15.165:0/1848736977 >> v2:172.21.15.172:3300/0 conn(0x7f47201615a0 0x7f47200cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.532+0000 7f471fe00640 1 --2- 172.21.15.165:0/1848736977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f4720061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.533+0000 7f471f400640 1 -- 172.21.15.165:0/1848736977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 msgr2=0x7f4720061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.533+0000 7f471f400640 1 --2- 172.21.15.165:0/1848736977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f4720061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.533+0000 7f471f400640 1 -- 172.21.15.165:0/1848736977 >> v2:172.21.15.172:3300/0 conn(0x7f47201615a0 msgr2=0x7f47200cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.533+0000 7f471f400640 1 --2- 172.21.15.165:0/1848736977 >> v2:172.21.15.172:3300/0 conn(0x7f47201615a0 0x7f47200cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.533+0000 7f471f400640 1 -- 172.21.15.165:0/1848736977 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47201620d0 con 0x7f472015db90 2024-04-25T10:13:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.533+0000 7f471ea00640 1 --2- 172.21.15.165:0/1848736977 >> v2:172.21.15.172:3300/0 conn(0x7f47201615a0 0x7f47200cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.534+0000 7f471f400640 1 --2- 172.21.15.165:0/1848736977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f47200b93c0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f47200d1b30 tx=0x7f471002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866d7970e0cc0173 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.534+0000 7f471e000640 1 -- 172.21.15.165:0/1848736977 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710037070 con 0x7f472015db90 2024-04-25T10:13:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.534+0000 7f471e000640 1 -- 172.21.15.165:0/1848736977 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710032440 con 0x7f472015db90 2024-04-25T10:13:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.534+0000 7f471e000640 1 -- 172.21.15.165:0/1848736977 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710032810 con 0x7f472015db90 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 -- 172.21.15.165:0/1848736977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 msgr2=0x7f47200b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 --2- 172.21.15.165:0/1848736977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f47200b93c0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f47200d1b30 tx=0x7f471002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 -- 172.21.15.165:0/1848736977 shutdown_connections 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 --2- 172.21.15.165:0/1848736977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f4720061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 --2- 172.21.15.165:0/1848736977 >> v2:172.21.15.172:3300/0 conn(0x7f47201615a0 0x7f47200cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 --2- 172.21.15.165:0/1848736977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f47200b93c0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 -- 172.21.15.165:0/1848736977 >> 172.21.15.165:0/1848736977 conn(0x7f47200b52b0 msgr2=0x7f47200b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 -- 172.21.15.165:0/1848736977 shutdown_connections 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.535+0000 7f4725200640 1 -- 172.21.15.165:0/1848736977 wait complete. 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.537+0000 7f4725200640 1 Processor -- start 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.537+0000 7f4725200640 1 -- start start 2024-04-25T10:13:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f4725200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f472014cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f4725200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f472015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f4725200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47201615a0 0x7f4720158d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f4725200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4720161a20 con 0x7f472015db90 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f4725200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47200d1be0 con 0x7f47201615a0 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f4725200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4720161ba0 con 0x7f47200d04f0 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f471ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f472015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f471ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f472015f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40522/0 (socket says 172.21.15.165:40522) 2024-04-25T10:13:39.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f471ea00640 1 -- 172.21.15.165:0/4065488844 learned_addr learned my addr 172.21.15.165:0/4065488844 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f471ea00640 1 -- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 msgr2=0x7f472014cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.538+0000 7f471f400640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f472014cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f471fe00640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47201615a0 0x7f4720158d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f471ea00640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f472014cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f471ea00640 1 -- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47201615a0 msgr2=0x7f4720158d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f471ea00640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47201615a0 0x7f4720158d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f471ea00640 1 -- 172.21.15.165:0/4065488844 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f472015fec0 con 0x7f472015db90 2024-04-25T10:13:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f471ea00640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f472015f490 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f470c00c7f0 tx=0x7f470c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c015070 con 0x7f472015db90 2024-04-25T10:13:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c0077c0 con 0x7f472015db90 2024-04-25T10:13:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47201601a0 con 0x7f472015db90 2024-04-25T10:13:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c007b70 con 0x7f472015db90 2024-04-25T10:13:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.539+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4720160760 con 0x7f472015db90 2024-04-25T10:13:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.540+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4720073990 con 0x7f472015db90 2024-04-25T10:13:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.541+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f470c018070 con 0x7f472015db90 2024-04-25T10:13:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.542+0000 7f4717400640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f470006b4d0 0x7f470006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.542+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f470c08a4d0 con 0x7f472015db90 2024-04-25T10:13:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.542+0000 7f471f400640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f470006b4d0 0x7f470006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.543+0000 7f471f400640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f470006b4d0 0x7f470006d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4710034000 tx=0x7f471002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.547+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f470c059180 con 0x7f472015db90 2024-04-25T10:13:39.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.778+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47200d14b0 con 0x7f472015db90 2024-04-25T10:13:39.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.779+0000 7f4717400640 1 -- 172.21.15.165:0/4065488844 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 74 v74) v1 ==== 94+0+4949 (secure 0 0 0) 0x7f470c058a80 con 0x7f472015db90 2024-04-25T10:13:39.783 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:39.783 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":74,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[{"mdsmap":{"epoch":74,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:12:36.858581+0000","modified":"2024-04-25T10:12:39.947364+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":8043},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8043":{"gid":8043,"name":"d","rank":0,"incarnation":70,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/437969658","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":437969658},{"type":"v1","addr":"172.21.15.172:6835","nonce":437969658}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2024-04-25T10:13:39.784 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 74 2024-04-25T10:13:39.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f470006b4d0 msgr2=0x7f470006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f470006b4d0 0x7f470006d990 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4710034000 tx=0x7f471002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:39.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 msgr2=0x7f472015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f472015f490 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f470c00c7f0 tx=0x7f470c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 shutdown_connections 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f470006b4d0 0x7f470006d990 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47201615a0 0x7f4720158d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f472015db90 0x7f472015f490 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 --2- 172.21.15.165:0/4065488844 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47200d04f0 0x7f472014cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 >> 172.21.15.165:0/4065488844 conn(0x7f47200b52b0 msgr2=0x7f47200d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 shutdown_connections 2024-04-25T10:13:39.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.782+0000 7f4725200640 1 -- 172.21.15.165:0/4065488844 wait complete. 2024-04-25T10:13:39.805 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:39.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.983+0000 7f3efac00640 1 Processor -- start 2024-04-25T10:13:39.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.983+0000 7f3efac00640 1 -- start start 2024-04-25T10:13:39.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.984+0000 7f3efac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3ef406e870 0x7f3ef40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.984+0000 7f3efac00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef4144240 con 0x7f3ef40cc840 2024-04-25T10:13:39.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.984+0000 7f3efac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ef415e220 con 0x7f3ef4143e60 2024-04-25T10:13:39.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.984+0000 7f3efac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ef415e500 con 0x7f3ef406e870 2024-04-25T10:13:39.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3ef8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3ef406e870 0x7f3ef40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3efa200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3ef40cc840 0x7f3ef415ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48208/0 (socket says 172.21.15.165:48208) 2024-04-25T10:13:39.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3efa200640 1 -- 172.21.15.165:0/1086824291 learned_addr learned my addr 172.21.15.165:0/1086824291 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:39.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3ef8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3ef406e870 0x7f3ef40cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40528/0 (socket says 172.21.15.165:40528) 2024-04-25T10:13:39.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3ef8e00640 1 -- 172.21.15.165:0/1086824291 >> v1:172.21.15.172:6789/0 conn(0x7f3ef4143e60 legacy=0x7f3ef406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:39.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3ef8e00640 1 -- 172.21.15.165:0/1086824291 >> v1:172.21.15.165:6789/0 conn(0x7f3ef40cc840 legacy=0x7f3ef415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.985+0000 7f3ef8e00640 1 -- 172.21.15.165:0/1086824291 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef415e7e0 con 0x7f3ef406e870 2024-04-25T10:13:39.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.986+0000 7f3ef8e00640 1 --2- 172.21.15.165:0/1086824291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef406e870 0x7f3ef40cc300 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00097a0 tx=0x7f3ee002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99b05305a7a0caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3ef3e00640 1 -- 172.21.15.165:0/1086824291 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0037070 con 0x7f3ef406e870 2024-04-25T10:13:39.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3ef3e00640 1 -- 172.21.15.165:0/1086824291 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ee0032440 con 0x7f3ef406e870 2024-04-25T10:13:39.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3ef3e00640 1 -- 172.21.15.165:0/1086824291 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee0032810 con 0x7f3ef406e870 2024-04-25T10:13:39.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 -- 172.21.15.165:0/1086824291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef406e870 msgr2=0x7f3ef40cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 --2- 172.21.15.165:0/1086824291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef406e870 0x7f3ef40cc300 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f3ee00097a0 tx=0x7f3ee002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 -- 172.21.15.165:0/1086824291 shutdown_connections 2024-04-25T10:13:39.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 --2- 172.21.15.165:0/1086824291 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef406e870 0x7f3ef40cc300 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 -- 172.21.15.165:0/1086824291 >> 172.21.15.165:0/1086824291 conn(0x7f3ef406abe0 msgr2=0x7f3ef406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:39.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 -- 172.21.15.165:0/1086824291 shutdown_connections 2024-04-25T10:13:39.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.987+0000 7f3efac00640 1 -- 172.21.15.165:0/1086824291 wait complete. 2024-04-25T10:13:39.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.989+0000 7f3efac00640 1 Processor -- start 2024-04-25T10:13:39.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.989+0000 7f3efac00640 1 -- start start 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef406e870 0x7f3ef40c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cc840 0x7f3ef415a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 0x7f3ef40cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40b2eb0 con 0x7f3ef4143e60 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40b2d30 con 0x7f3ef406e870 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ef40ccca0 con 0x7f3ef40cc840 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efa200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 0x7f3ef40cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efa200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 0x7f3ef40cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40532/0 (socket says 172.21.15.165:40532) 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.990+0000 7f3efa200640 1 -- 172.21.15.165:0/2178386653 learned_addr learned my addr 172.21.15.165:0/2178386653 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3ef8e00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cc840 0x7f3ef415a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3ef9800640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef406e870 0x7f3ef40c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3efa200640 1 -- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cc840 msgr2=0x7f3ef415a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3efa200640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cc840 0x7f3ef415a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3efa200640 1 -- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef406e870 msgr2=0x7f3ef40c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3efa200640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef406e870 0x7f3ef40c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3efa200640 1 -- 172.21.15.165:0/2178386653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef41f7bc0 con 0x7f3ef4143e60 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.991+0000 7f3efa200640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 0x7f3ef40cac10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00cbf0 tx=0x7f3eec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.992+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec017070 con 0x7f3ef4143e60 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.992+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3eec00ace0 con 0x7f3ef4143e60 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.992+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ef41f6e20 con 0x7f3ef4143e60 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.992+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3eec012600 con 0x7f3ef4143e60 2024-04-25T10:13:39.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.992+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ef41f73c0 con 0x7f3ef4143e60 2024-04-25T10:13:39.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.993+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ef4073990 con 0x7f3ef4143e60 2024-04-25T10:13:39.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.994+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3eec01a070 con 0x7f3ef4143e60 2024-04-25T10:13:39.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.995+0000 7f3ef1600640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ed406b500 0x7f3ed406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:39.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.995+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f3eec089fb0 con 0x7f3ef4143e60 2024-04-25T10:13:39.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.995+0000 7f3ef9800640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ed406b500 0x7f3ed406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.996+0000 7f3ef9800640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ed406b500 0x7f3ed406d9c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8006fd0 tx=0x7f3ee8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:39.999+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3eec058610 con 0x7f3ef4143e60 2024-04-25T10:13:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.181+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ef40ad480 con 0x7f3ef4143e60 2024-04-25T10:13:40.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.183+0000 7f3ef1600640 1 -- 172.21.15.165:0/2178386653 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17018 (secure 0 0 0) 0x7f3eec05c6b0 con 0x7f3ef4143e60 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:40.188 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:12:48.513977+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:12:34.618582+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_data","create_time":"2024-04-25T10:12:35.883096+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"111","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.1c","mappings":[{"from":1,"to":7}]},{"pgid":"16.31","mappings":[{"from":5,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ed406b500 msgr2=0x7f3ed406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ed406b500 0x7f3ed406d9c0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8006fd0 tx=0x7f3ee8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 msgr2=0x7f3ef40cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 0x7f3ef40cac10 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f3eec00cbf0 tx=0x7f3eec00a430 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 shutdown_connections 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ed406b500 0x7f3ed406d9c0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ef4143e60 0x7f3ef40cac10 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ef40cc840 0x7f3ef415a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 --2- 172.21.15.165:0/2178386653 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ef406e870 0x7f3ef40c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 >> 172.21.15.165:0/2178386653 conn(0x7f3ef406abe0 msgr2=0x7f3ef40ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 shutdown_connections 2024-04-25T10:13:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.186+0000 7f3efac00640 1 -- 172.21.15.165:0/2178386653 wait complete. 2024-04-25T10:13:40.209 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:13:40.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.404+0000 7f4bb4a00640 1 Processor -- start 2024-04-25T10:13:40.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.404+0000 7f4bb4a00640 1 -- start start 2024-04-25T10:13:40.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4bb4a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4bb00ceb20 0x7f4bb00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4bb4a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4bb00c5030 0x7f4bb015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4bb4a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bb0145520 con 0x7f4bb0145140 2024-04-25T10:13:40.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4bb4a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb0071270 con 0x7f4bb00c5030 2024-04-25T10:13:40.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4bb4a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bb0071550 con 0x7f4bb00ceb20 2024-04-25T10:13:40.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4bb00c5030 0x7f4bb015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4bb00c5030 0x7f4bb015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40544/0 (socket says 172.21.15.165:40544) 2024-04-25T10:13:40.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 -- 172.21.15.165:0/1511045721 learned_addr learned my addr 172.21.15.165:0/1511045721 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:40.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 -- 172.21.15.165:0/1511045721 >> v2:172.21.15.172:3300/0 conn(0x7f4bb00ceb20 msgr2=0x7f4bb00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 --2- 172.21.15.165:0/1511045721 >> v2:172.21.15.172:3300/0 conn(0x7f4bb00ceb20 0x7f4bb00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 -- 172.21.15.165:0/1511045721 >> v1:172.21.15.172:6789/0 conn(0x7f4bb0145140 legacy=0x7f4bb00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.406+0000 7f4baf200640 1 -- 172.21.15.165:0/1511045721 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb0071830 con 0x7f4bb00c5030 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.407+0000 7f4baf200640 1 --2- 172.21.15.165:0/1511045721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb015bfb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00b0c00 tx=0x7f4ba802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d5d7e65f4bb11a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bade00640 1 -- 172.21.15.165:0/1511045721 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba8036070 con 0x7f4bb00c5030 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bade00640 1 -- 172.21.15.165:0/1511045721 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba8031600 con 0x7f4bb00c5030 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bade00640 1 -- 172.21.15.165:0/1511045721 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba80319d0 con 0x7f4bb00c5030 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bb4a00640 1 -- 172.21.15.165:0/1511045721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 msgr2=0x7f4bb015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/1511045721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb015bfb0 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f4bb00b0c00 tx=0x7f4ba802a660 comp rx=0 tx=0).stop 2024-04-25T10:13:40.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bb4a00640 1 -- 172.21.15.165:0/1511045721 shutdown_connections 2024-04-25T10:13:40.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/1511045721 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb015bfb0 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/1511045721 >> v2:172.21.15.172:3300/0 conn(0x7f4bb00ceb20 0x7f4bb00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.408+0000 7f4bb4a00640 1 -- 172.21.15.165:0/1511045721 >> 172.21.15.165:0/1511045721 conn(0x7f4bb0061cc0 msgr2=0x7f4bb006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.409+0000 7f4bb4a00640 1 -- 172.21.15.165:0/1511045721 shutdown_connections 2024-04-25T10:13:40.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.409+0000 7f4bb4a00640 1 -- 172.21.15.165:0/1511045721 wait complete. 2024-04-25T10:13:40.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.410+0000 7f4bb4a00640 1 Processor -- start 2024-04-25T10:13:40.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.411+0000 7f4bb4a00640 1 -- start start 2024-04-25T10:13:40.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.411+0000 7f4bb4a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb01e43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.411+0000 7f4bb4a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bb00ceb20 0x7f4bb013fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bb4a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0145140 0x7f4bb01f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bb4a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00c5620 con 0x7f4bb00c5030 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bb4a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb00c54a0 con 0x7f4bb0145140 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bb4a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bb015c4f0 con 0x7f4bb00ceb20 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb01e43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb01e43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40558/0 (socket says 172.21.15.165:40558) 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 -- 172.21.15.165:0/2768443095 learned_addr learned my addr 172.21.15.165:0/2768443095 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 -- 172.21.15.165:0/2768443095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bb00ceb20 msgr2=0x7f4bb013fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bb00ceb20 0x7f4bb013fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 -- 172.21.15.165:0/2768443095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0145140 msgr2=0x7f4bb01f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0145140 0x7f4bb01f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 -- 172.21.15.165:0/2768443095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb01f3d60 con 0x7f4bb00c5030 2024-04-25T10:13:40.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.412+0000 7f4bae800640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb01e43e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000b3e0 tx=0x7f4ba000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.413+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0015020 con 0x7f4bb00c5030 2024-04-25T10:13:40.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.413+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ba0011850 con 0x7f4bb00c5030 2024-04-25T10:13:40.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.413+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0011c40 con 0x7f4bb00c5030 2024-04-25T10:13:40.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.413+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bb01f4040 con 0x7f4bb00c5030 2024-04-25T10:13:40.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.413+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bb0140310 con 0x7f4bb00c5030 2024-04-25T10:13:40.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.414+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bb00c4720 con 0x7f4bb00c5030 2024-04-25T10:13:40.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.415+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ba0018020 con 0x7f4bb00c5030 2024-04-25T10:13:40.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.416+0000 7f4ba7400640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b8c06b4d0 0x7f4b8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.416+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f4ba008a2d0 con 0x7f4bb00c5030 2024-04-25T10:13:40.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.416+0000 7f4ba5e00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b8c06b4d0 0x7f4b8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.417+0000 7f4ba5e00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b8c06b4d0 0x7f4b8c06d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0144470 tx=0x7f4b98006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.420+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4bb00c4720 con 0x7f4bb00c5030 2024-04-25T10:13:40.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.657+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4bb013d890 con 0x7f4bb00c5030 2024-04-25T10:13:40.662 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:13:40.658+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:13:40.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.662+0000 7f4ba7400640 1 -- 172.21.15.165:0/2768443095 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v75) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4ba001f020 con 0x7f4bb00c5030 2024-04-25T10:13:40.666 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:13:40.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b8c06b4d0 msgr2=0x7f4b8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b8c06b4d0 0x7f4b8c06d990 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4bb0144470 tx=0x7f4b98006c90 comp rx=0 tx=0).stop 2024-04-25T10:13:40.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 msgr2=0x7f4bb01e43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb01e43e0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000b3e0 tx=0x7f4ba000b8a0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 shutdown_connections 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4b8c06b4d0 0x7f4b8c06d990 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4bb0145140 0x7f4bb01f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4bb00ceb20 0x7f4bb013fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 --2- 172.21.15.165:0/2768443095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4bb00c5030 0x7f4bb01e43e0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.665+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 >> 172.21.15.165:0/2768443095 conn(0x7f4bb0061cc0 msgr2=0x7f4bb00b3c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.666+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 shutdown_connections 2024-04-25T10:13:40.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.666+0000 7f4bb4a00640 1 -- 172.21.15.165:0/2768443095 wait complete. 2024-04-25T10:13:40.689 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:13:40.747 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:13:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.888+0000 7f86e5e00640 1 Processor -- start 2024-04-25T10:13:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.888+0000 7f86e5e00640 1 -- start start 2024-04-25T10:13:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86e5e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f86e006bff0 0x7f86e00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86e5e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e015e110 con 0x7f86e00b5970 2024-04-25T10:13:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86e5e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e00b2fd0 con 0x7f86e015dd30 2024-04-25T10:13:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86e5e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e006d000 con 0x7f86e006bff0 2024-04-25T10:13:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86e5400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f86e00b5970 0x7f86e00d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48222/0 (socket says 172.21.15.165:48222) 2024-04-25T10:13:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86e5400640 1 -- 172.21.15.165:0/971881122 learned_addr learned my addr 172.21.15.165:0/971881122 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86dfe00640 1 --2- 172.21.15.165:0/971881122 >> v2:172.21.15.172:3300/0 conn(0x7f86e006bff0 0x7f86e00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.890+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993429352 0 0) 0x7f86e015e110 con 0x7f86e00b5970 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86c8003520 con 0x7f86e00b5970 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3570425011 0 0) 0x7f86c8003520 con 0x7f86e00b5970 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 >> v2:172.21.15.172:3300/0 conn(0x7f86e006bff0 msgr2=0x7f86e00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 --2- 172.21.15.165:0/971881122 >> v2:172.21.15.172:3300/0 conn(0x7f86e006bff0 0x7f86e00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 >> v1:172.21.15.172:6789/0 conn(0x7f86e015dd30 legacy=0x7f86e00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e006c480 con 0x7f86e00b5970 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f86d80019f0 con 0x7f86e00b5970 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 >> v1:172.21.15.165:6789/0 conn(0x7f86e00b5970 legacy=0x7f86e00d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80039b0 0x7f86c8003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c80043d0 0x7f86c80048a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c80050a0 0x7f86c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e006d000 con 0x7f86c80043d0 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c800bfe0 con 0x7f86c80039b0 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c800c1e0 con 0x7f86c80050a0 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.891+0000 7f86dfe00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c80043d0 0x7f86c80048a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86e4a00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80039b0 0x7f86c8003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86e5400640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c80050a0 0x7f86c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86dfe00640 1 -- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c80050a0 msgr2=0x7f86c800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86dfe00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86c80050a0 0x7f86c800b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86dfe00640 1 -- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80039b0 msgr2=0x7f86c8003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:40.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86dfe00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80039b0 0x7f86c8003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86dfe00640 1 -- 172.21.15.165:0/971881122 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c8003520 con 0x7f86c80043d0 2024-04-25T10:13:40.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.892+0000 7f86e5400640 1 -- 172.21.15.165:0/971881122 reap_dead start 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.893+0000 7f86dfe00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c80043d0 0x7f86c80048a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009ac0 tx=0x7f86c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201c09f2a1329bd5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.893+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc036070 con 0x7f86c80043d0 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.893+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc031400 con 0x7f86c80043d0 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.893+0000 7f86df400640 1 -- 172.21.15.165:0/971881122 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc009690 con 0x7f86c80043d0 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 -- 172.21.15.165:0/971881122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c80043d0 msgr2=0x7f86c80048a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c80043d0 0x7f86c80048a0 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f86cc009ac0 tx=0x7f86c8005830 comp rx=0 tx=0).stop 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 -- 172.21.15.165:0/971881122 shutdown_connections 2024-04-25T10:13:40.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 --2- 172.21.15.165:0/971881122 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86c80043d0 0x7f86c80048a0 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 -- 172.21.15.165:0/971881122 >> 172.21.15.165:0/971881122 conn(0x7f86e00d0610 msgr2=0x7f86e00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:40.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 -- 172.21.15.165:0/971881122 shutdown_connections 2024-04-25T10:13:40.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.894+0000 7f86e5e00640 1 -- 172.21.15.165:0/971881122 wait complete. 2024-04-25T10:13:40.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.896+0000 7f86e5e00640 1 Processor -- start 2024-04-25T10:13:40.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.896+0000 7f86e5e00640 1 -- start start 2024-04-25T10:13:40.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80043d0 0x7f86e0154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86e0152de0 0x7f86e00c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 0x7f86e00cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e006cd20 con 0x7f86e00cbfe0 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e006ca40 con 0x7f86c80043d0 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e00b5430 con 0x7f86e0152de0 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 0x7f86e00cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 0x7f86e00cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40578/0 (socket says 172.21.15.165:40578) 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5400640 1 -- 172.21.15.165:0/1076342876 learned_addr learned my addr 172.21.15.165:0/1076342876 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86dfe00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86e0152de0 0x7f86e00c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e4a00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80043d0 0x7f86e0154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5400640 1 -- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86e0152de0 msgr2=0x7f86e00c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5400640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86e0152de0 0x7f86e00c9ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.897+0000 7f86e5400640 1 -- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80043d0 msgr2=0x7f86e0154290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86e5400640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80043d0 0x7f86e0154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:40.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86e5400640 1 -- 172.21.15.165:0/1076342876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e015e570 con 0x7f86e00cbfe0 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86dfe00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86e0152de0 0x7f86e00c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86e5400640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 0x7f86e00cc460 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b93c0 tx=0x7f86d800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d800ee20 con 0x7f86e00cbfe0 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8014430 con 0x7f86e00cbfe0 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8014820 con 0x7f86e00cbfe0 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e015e850 con 0x7f86e00cbfe0 2024-04-25T10:13:40.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.898+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e015ee10 con 0x7f86e00cbfe0 2024-04-25T10:13:40.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.899+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e0073990 con 0x7f86e00cbfe0 2024-04-25T10:13:40.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.901+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f86d80149c0 con 0x7f86e00cbfe0 2024-04-25T10:13:40.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.901+0000 7f86dcc00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f86c006b4d0 0x7f86c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:40.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.901+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f86d808a3b0 con 0x7f86e00cbfe0 2024-04-25T10:13:40.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.902+0000 7f86e4a00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f86c006b4d0 0x7f86c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:40.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.902+0000 7f86e4a00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f86c006b4d0 0x7f86c006d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f86d4007fd0 tx=0x7f86d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:40.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:40.906+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f86d8058f70 con 0x7f86e00cbfe0 2024-04-25T10:13:41.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.136+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f86e00d16c0 con 0x7f86e00cbfe0 2024-04-25T10:13:41.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.665+0000 7f86dcc00640 1 -- 172.21.15.165:0/1076342876 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v76) v1 ==== 104+0+0 (secure 0 0 0) 0x7f86d8058870 con 0x7f86e00cbfe0 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.669+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f86c006b4d0 msgr2=0x7f86c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.669+0000 7f86e5e00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f86c006b4d0 0x7f86c006d990 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f86d4007fd0 tx=0x7f86d4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.669+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 msgr2=0x7f86e00cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.669+0000 7f86e5e00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 0x7f86e00cc460 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f86e00b93c0 tx=0x7f86d800e530 comp rx=0 tx=0).stop 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 shutdown_connections 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f86c006b4d0 0x7f86c006d990 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86e00cbfe0 0x7f86e00cc460 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86e0152de0 0x7f86e00c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 --2- 172.21.15.165:0/1076342876 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86c80043d0 0x7f86e0154290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 >> 172.21.15.165:0/1076342876 conn(0x7f86e00d0610 msgr2=0x7f86e015ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 shutdown_connections 2024-04-25T10:13:41.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.670+0000 7f86e5e00640 1 -- 172.21.15.165:0/1076342876 wait complete. 2024-04-25T10:13:41.696 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:13:41.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.888+0000 7f710b000640 1 Processor -- start 2024-04-25T10:13:41.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.889+0000 7f710b000640 1 -- start start 2024-04-25T10:13:41.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.890+0000 7f710b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 0x7f710406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.890+0000 7f710b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71041452a0 0x7f710406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.890+0000 7f710b000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71040cccc0 con 0x7f71041446d0 2024-04-25T10:13:41.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.890+0000 7f710b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71040b2b20 con 0x7f71041452a0 2024-04-25T10:13:41.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.890+0000 7f710b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710406d400 con 0x7f7104144d50 2024-04-25T10:13:41.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f7103e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f71041446d0 0x7f71040b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48226/0 (socket says 172.21.15.165:48226) 2024-04-25T10:13:41.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f7103e00640 1 -- 172.21.15.165:0/3873923019 learned_addr learned my addr 172.21.15.165:0/3873923019 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:41.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f70fb400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 0x7f710406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f7109000640 1 --2- 172.21.15.165:0/3873923019 >> v2:172.21.15.172:3300/0 conn(0x7f71041452a0 0x7f710406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704844740 0 0) 0x7f71040cccc0 con 0x7f71041446d0 2024-04-25T10:13:41.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70e4003520 con 0x7f71041446d0 2024-04-25T10:13:41.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.891+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2173651110 0 0) 0x7f70e4003520 con 0x7f71041446d0 2024-04-25T10:13:41.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 msgr2=0x7f710406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 0x7f710406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 >> v2:172.21.15.172:3300/0 conn(0x7f71041452a0 msgr2=0x7f710406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 --2- 172.21.15.165:0/3873923019 >> v2:172.21.15.172:3300/0 conn(0x7f71041452a0 0x7f710406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040cc140 con 0x7f71041446d0 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f70f4001670 con 0x7f71041446d0 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 >> v1:172.21.15.165:6789/0 conn(0x7f71041446d0 legacy=0x7f71040b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70e4004800 0x7f70e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 0x7f70e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040b2b20 con 0x7f70e4004800 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710406d400 con 0x7f70e40039b0 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70e400b9c0 con 0x7f7104144d50 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f70fb400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70e4004800 0x7f70e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f70fb400640 1 -- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 msgr2=0x7f70e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7109000640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 0x7f70e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f70fb400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7104144d50 0x7f70e40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103e00640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f70fb400640 1 -- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e40039b0 msgr2=0x7f70e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f70fb400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f70fb400640 1 -- 172.21.15.165:0/3873923019 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70e4003520 con 0x7f70e4004800 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.892+0000 7f7103e00640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f70e40039b0 0x7f70e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:41.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.893+0000 7f70fb400640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70e4004800 0x7f70e4004c80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009ac0 tx=0x7f70e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6802e4ab8bdb8f02 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec036070 con 0x7f70e4004800 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec031400 con 0x7f70e4004800 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f7103400640 1 -- 172.21.15.165:0/3873923019 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec009690 con 0x7f70e4004800 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f710b000640 1 -- 172.21.15.165:0/3873923019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70e4004800 msgr2=0x7f70e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f710b000640 1 --2- 172.21.15.165:0/3873923019 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70e4004800 0x7f70e4004c80 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f70ec009ac0 tx=0x7f70e4005830 comp rx=0 tx=0).stop 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f7109000640 1 -- 172.21.15.165:0/3873923019 reap_dead start 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f710b000640 1 -- 172.21.15.165:0/3873923019 shutdown_connections 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f710b000640 1 -- 172.21.15.165:0/3873923019 >> 172.21.15.165:0/3873923019 conn(0x7f710406abe0 msgr2=0x7f710406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f710b000640 1 -- 172.21.15.165:0/3873923019 shutdown_connections 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.894+0000 7f710b000640 1 -- 172.21.15.165:0/3873923019 wait complete. 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.896+0000 7f710b000640 1 Processor -- start 2024-04-25T10:13:41.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.896+0000 7f710b000640 1 -- start start 2024-04-25T10:13:41.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f710b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e4004800 0x7f7104140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f710b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71040c63a0 0x7f71040c6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f710b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 0x7f71040ab070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f710b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040cc9e0 con 0x7f71040aabf0 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f710b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71040cc700 con 0x7f71040c63a0 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f710b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7104144ab0 con 0x7f70e4004800 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f7109000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 0x7f71040ab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f7109000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 0x7f71040ab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40590/0 (socket says 172.21.15.165:40590) 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.897+0000 7f7109000640 1 -- 172.21.15.165:0/3840842742 learned_addr learned my addr 172.21.15.165:0/3840842742 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7103e00640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e4004800 0x7f7104140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7109000640 1 -- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e4004800 msgr2=0x7f7104140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7109000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e4004800 0x7f7104140560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f70fb400640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71040c63a0 0x7f71040c6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7109000640 1 -- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71040c63a0 msgr2=0x7f71040c6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7109000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71040c63a0 0x7f71040c6820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:41.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7109000640 1 -- 172.21.15.165:0/3840842742 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71040ab5b0 con 0x7f71040aabf0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7109000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 0x7f71040ab070 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f71040cbb70 tx=0x7f70fc00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc016620 con 0x7f71040aabf0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc01b070 con 0x7f71040aabf0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc016a20 con 0x7f71040aabf0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.898+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71040ab870 con 0x7f71040aabf0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.899+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71040abe30 con 0x7f71040aabf0 2024-04-25T10:13:41.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.899+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7104073a00 con 0x7f71040aabf0 2024-04-25T10:13:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.900+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70fc005d10 con 0x7f71040aabf0 2024-04-25T10:13:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.901+0000 7f7100c00640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70e006b500 0x7f70e006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.901+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f70fc08b060 con 0x7f71040aabf0 2024-04-25T10:13:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.902+0000 7f7103e00640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70e006b500 0x7f70e006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:41.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.902+0000 7f7103e00640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70e006b500 0x7f70e006d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f70f4001200 tx=0x7f70f4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:41.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:41.906+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f70fc025020 con 0x7f71040aabf0 2024-04-25T10:13:42.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.098+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71040afd80 con 0x7f71040aabf0 2024-04-25T10:13:42.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.814+0000 7f7100c00640 1 -- 172.21.15.165:0/3840842742 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v115) v1 ==== 181+0+0 (secure 0 0 0) 0x7f70fc059c20 con 0x7f71040aabf0 2024-04-25T10:13:42.819 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:13:42.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.818+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70e006b500 msgr2=0x7f70e006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.818+0000 7f710b000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70e006b500 0x7f70e006d9c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f70f4001200 tx=0x7f70f4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.818+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 msgr2=0x7f71040ab070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:42.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.818+0000 7f710b000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 0x7f71040ab070 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f71040cbb70 tx=0x7f70fc00f520 comp rx=0 tx=0).stop 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.819+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 shutdown_connections 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.819+0000 7f710b000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f70e006b500 0x7f70e006d9c0 secure :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f70f4001200 tx=0x7f70f4009040 comp rx=0 tx=0).stop 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.819+0000 7f710b000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71040aabf0 0x7f71040ab070 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.819+0000 7f710b000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71040c63a0 0x7f71040c6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.819+0000 7f710b000640 1 --2- 172.21.15.165:0/3840842742 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70e4004800 0x7f7104140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.819+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 >> 172.21.15.165:0/3840842742 conn(0x7f710406abe0 msgr2=0x7f7104141970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:42.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.820+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 shutdown_connections 2024-04-25T10:13:42.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:42.820+0000 7f710b000640 1 -- 172.21.15.165:0/3840842742 wait complete. 2024-04-25T10:13:42.849 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:13:43.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.051+0000 7f0bc3c00640 1 Processor -- start 2024-04-25T10:13:43.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.051+0000 7f0bc3c00640 1 -- start start 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bc3c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bc3c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0bbc0cdb20 0x7f0bbc0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bc3c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bbc072280 con 0x7f0bbc15b270 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bc3c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bbc0ce4c0 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bc3c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc071270 con 0x7f0bbc0cc4f0 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0bbc0cdb20 0x7f0bbc0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0bbc0cdb20 0x7f0bbc0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40602/0 (socket says 172.21.15.165:40602) 2024-04-25T10:13:43.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 -- 172.21.15.165:0/3564008053 learned_addr learned my addr 172.21.15.165:0/3564008053 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 -- 172.21.15.165:0/3564008053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 msgr2=0x7f0bbc0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 --2- 172.21.15.165:0/3564008053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 -- 172.21.15.165:0/3564008053 >> v1:172.21.15.172:6789/0 conn(0x7f0bbc15b270 legacy=0x7f0bbc15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:43.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.053+0000 7f0bbbe00640 1 -- 172.21.15.165:0/3564008053 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc0714c0 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.054+0000 7f0bbbe00640 1 --2- 172.21.15.165:0/3564008053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc0cdf80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009870 tx=0x7f0ba402c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eb45faacd19e23b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.055+0000 7f0bbb400640 1 -- 172.21.15.165:0/3564008053 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4038070 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.055+0000 7f0bbb400640 1 -- 172.21.15.165:0/3564008053 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4031710 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.055+0000 7f0bbb400640 1 -- 172.21.15.165:0/3564008053 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4031ae0 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.055+0000 7f0bc3c00640 1 -- 172.21.15.165:0/3564008053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 msgr2=0x7f0bbc0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.055+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/3564008053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc0cdf80 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009870 tx=0x7f0ba402c420 comp rx=0 tx=0).stop 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.056+0000 7f0bc3c00640 1 -- 172.21.15.165:0/3564008053 shutdown_connections 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.056+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/3564008053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc0cdf80 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.056+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/3564008053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc0cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.056+0000 7f0bc3c00640 1 -- 172.21.15.165:0/3564008053 >> 172.21.15.165:0/3564008053 conn(0x7f0bbc06abe0 msgr2=0x7f0bbc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.056+0000 7f0bc3c00640 1 -- 172.21.15.165:0/3564008053 shutdown_connections 2024-04-25T10:13:43.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.056+0000 7f0bc3c00640 1 -- 172.21.15.165:0/3564008053 wait complete. 2024-04-25T10:13:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.057+0000 7f0bc3c00640 1 Processor -- start 2024-04-25T10:13:43.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.058+0000 7f0bc3c00640 1 -- start start 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.058+0000 7f0bc3c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc1ea1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc3c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc154400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc3c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bbc15b270 0x7f0bbc155590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc3c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc071a80 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc3c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc15bd90 con 0x7f0bbc15b270 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc3c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bbc071d60 con 0x7f0bbc0cc4f0 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc154400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc154400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40604/0 (socket says 172.21.15.165:40604) 2024-04-25T10:13:43.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 -- 172.21.15.165:0/4087779812 learned_addr learned my addr 172.21.15.165:0/4087779812 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc1c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bbc15b270 0x7f0bbc155590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bc1200640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc1ea1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 -- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 msgr2=0x7f0bbc1ea1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc1ea1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 -- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bbc15b270 msgr2=0x7f0bbc155590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bbc15b270 0x7f0bbc155590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.059+0000 7f0bbbe00640 1 -- 172.21.15.165:0/4087779812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bbc0ca1a0 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.060+0000 7f0bbbe00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc154400 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009870 tx=0x7f0ba40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.060+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4042020 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.060+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba4033830 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.060+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba4038040 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.060+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bbc0ca480 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.060+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bbc0caa40 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.061+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bbc073a00 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.062+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0ba4045020 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.063+0000 7f0bb8c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b9806b4d0 0x7f0b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.063+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5807+0+0 (secure 0 0 0) 0x7f0ba40aaf00 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.064+0000 7f0bc1200640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b9806b4d0 0x7f0b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:43.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.064+0000 7f0bc1200640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b9806b4d0 0x7f0b9806d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0bac007dc0 tx=0x7f0bac0072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:43.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.067+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0ba404c030 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.262+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0bbc06d3a0 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.818+0000 7f0bb8c00640 1 -- 172.21.15.165:0/4087779812 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v116) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0ba4079d10 con 0x7f0bbc0cdb20 2024-04-25T10:13:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:13:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b9806b4d0 msgr2=0x7f0b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b9806b4d0 0x7f0b9806d990 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0bac007dc0 tx=0x7f0bac0072f0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 msgr2=0x7f0bbc154400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc154400 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f0ba4009870 tx=0x7f0ba40046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 shutdown_connections 2024-04-25T10:13:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b9806b4d0 0x7f0b9806d990 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0bbc15b270 0x7f0bbc155590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0bbc0cdb20 0x7f0bbc154400 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 --2- 172.21.15.165:0/4087779812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0bbc0cc4f0 0x7f0bbc1ea1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.822+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 >> 172.21.15.165:0/4087779812 conn(0x7f0bbc06abe0 msgr2=0x7f0bbc0b2e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.824+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 shutdown_connections 2024-04-25T10:13:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:43.825+0000 7f0bc3c00640 1 -- 172.21.15.165:0/4087779812 wait complete. 2024-04-25T10:13:43.854 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:13:44.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.050+0000 7f4aae600640 1 Processor -- start 2024-04-25T10:13:44.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.050+0000 7f4aae600640 1 -- start start 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.053+0000 7f4aae600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4aa8161100 0x7f4aa806e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.053+0000 7f4aae600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa80cc8b0 0x7f4aa806ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.053+0000 7f4aae600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aa8161a50 con 0x7f4aa8071ec0 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.053+0000 7f4aae600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aa8161800 con 0x7f4aa8161100 2024-04-25T10:13:44.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.053+0000 7f4aae600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8161d30 con 0x7f4aa80cc8b0 2024-04-25T10:13:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.054+0000 7f4aa7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4aa8071ec0 0x7f4aa806d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48242/0 (socket says 172.21.15.165:48242) 2024-04-25T10:13:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.054+0000 7f4aa7e00640 1 -- 172.21.15.165:0/842170123 learned_addr learned my addr 172.21.15.165:0/842170123 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa7400640 1 --2- 172.21.15.165:0/842170123 >> v2:172.21.15.172:3300/0 conn(0x7f4aa8161100 0x7f4aa806e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161058792 0 0) 0x7f4aa8161a50 con 0x7f4aa8071ec0 2024-04-25T10:13:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a8c003520 con 0x7f4aa8071ec0 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833398905 0 0) 0x7f4a8c003520 con 0x7f4aa8071ec0 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa80cc8b0 msgr2=0x7f4aa806ea30 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa80cc8b0 0x7f4aa806ea30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 >> v2:172.21.15.172:3300/0 conn(0x7f4aa8161100 msgr2=0x7f4aa806e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 --2- 172.21.15.165:0/842170123 >> v2:172.21.15.172:3300/0 conn(0x7f4aa8161100 0x7f4aa806e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa8162010 con 0x7f4aa8071ec0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4aa00019f0 con 0x7f4aa8071ec0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 >> v1:172.21.15.165:6789/0 conn(0x7f4aa8071ec0 legacy=0x7f4aa806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0039b0 0x7f4a8c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.055+0000 7f4aa6000640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a8c004800 0x7f4a8c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.056+0000 7f4aa6000640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a8c0051c0 0x7f4a8c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.056+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8161800 con 0x7f4a8c0039b0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.056+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa8161d30 con 0x7f4a8c004800 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.056+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8c00c0b0 con 0x7f4a8c0051c0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.056+0000 7f4aa7400640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0039b0 0x7f4a8c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.056+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a8c0051c0 0x7f4a8c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.057+0000 7f4aa6a00640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a8c004800 0x7f4a8c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.058+0000 7f4aa7e00640 1 -- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a8c004800 msgr2=0x7f4a8c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.058+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a8c004800 0x7f4a8c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.058+0000 7f4aa7e00640 1 -- 172.21.15.165:0/842170123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0039b0 msgr2=0x7f4a8c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.058+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0039b0 0x7f4a8c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.058+0000 7f4aa7e00640 1 -- 172.21.15.165:0/842170123 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8c003520 con 0x7f4a8c0051c0 2024-04-25T10:13:44.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.059+0000 7f4aa7400640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0039b0 0x7f4a8c003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.061+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a8c0051c0 0x7f4a8c00b9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000a8a0 tx=0x7f4a8c005830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2fc139cdaea193b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.063+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000a4c0 con 0x7f4a8c0051c0 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.063+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa002f070 con 0x7f4a8c0051c0 2024-04-25T10:13:44.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.063+0000 7f4aa6000640 1 -- 172.21.15.165:0/842170123 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa002bd30 con 0x7f4a8c0051c0 2024-04-25T10:13:44.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.065+0000 7f4aae600640 1 -- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a8c0051c0 msgr2=0x7f4a8c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.065+0000 7f4aae600640 1 --2- 172.21.15.165:0/842170123 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4a8c0051c0 0x7f4a8c00b9c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000a8a0 tx=0x7f4a8c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:44.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.065+0000 7f4aa7e00640 1 -- 172.21.15.165:0/842170123 reap_dead start 2024-04-25T10:13:44.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.066+0000 7f4aae600640 1 -- 172.21.15.165:0/842170123 shutdown_connections 2024-04-25T10:13:44.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.066+0000 7f4aae600640 1 -- 172.21.15.165:0/842170123 >> 172.21.15.165:0/842170123 conn(0x7f4aa806abe0 msgr2=0x7f4aa806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.066+0000 7f4aae600640 1 -- 172.21.15.165:0/842170123 shutdown_connections 2024-04-25T10:13:44.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.066+0000 7f4aae600640 1 -- 172.21.15.165:0/842170123 wait complete. 2024-04-25T10:13:44.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.073+0000 7f4aae600640 1 Processor -- start 2024-04-25T10:13:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.075+0000 7f4aae600640 1 -- start start 2024-04-25T10:13:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.077+0000 7f4aae600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0051c0 0x7f4aa80adca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.077+0000 7f4aae600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa8144330 0x7f4aa81447b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.077+0000 7f4aae600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 0x7f4aa8141f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.077+0000 7f4aae600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa81614e0 con 0x7f4a8c0051c0 2024-04-25T10:13:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.077+0000 7f4aae600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80cd5a0 con 0x7f4aa8141aa0 2024-04-25T10:13:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.077+0000 7f4aae600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aa80cfd80 con 0x7f4aa8144330 2024-04-25T10:13:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.078+0000 7f4aa7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 0x7f4aa8141f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.078+0000 7f4aa7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 0x7f4aa8141f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:36230/0 (socket says 172.21.15.165:36230) 2024-04-25T10:13:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.078+0000 7f4aa7e00640 1 -- 172.21.15.165:0/2908592988 learned_addr learned my addr 172.21.15.165:0/2908592988 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:44.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.078+0000 7f4aa7400640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0051c0 0x7f4aa80adca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.079+0000 7f4aa6a00640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa8144330 0x7f4aa81447b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.082+0000 7f4aa7e00640 1 -- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa8144330 msgr2=0x7f4aa81447b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.082+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa8144330 0x7f4aa81447b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.082+0000 7f4aa7e00640 1 -- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0051c0 msgr2=0x7f4aa80adca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.082+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0051c0 0x7f4aa80adca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.083+0000 7f4aa7e00640 1 -- 172.21.15.165:0/2908592988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aa815cda0 con 0x7f4aa8141aa0 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.083+0000 7f4aa7400640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0051c0 0x7f4aa80adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.083+0000 7f4aa6a00640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa8144330 0x7f4aa81447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:44.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.083+0000 7f4aa7e00640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 0x7f4aa8141f50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000a8a0 tx=0x7f4aa0001dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.084+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0044070 con 0x7f4aa8141aa0 2024-04-25T10:13:44.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.084+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa00019e0 con 0x7f4aa8141aa0 2024-04-25T10:13:44.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.084+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000a4c0 con 0x7f4aa8141aa0 2024-04-25T10:13:44.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.084+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aa815d040 con 0x7f4aa8141aa0 2024-04-25T10:13:44.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.084+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aa815d600 con 0x7f4aa8141aa0 2024-04-25T10:13:44.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.087+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aa8073a00 con 0x7f4aa8141aa0 2024-04-25T10:13:44.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.094+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4aa0034870 con 0x7f4aa8141aa0 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.095+0000 7f4a9f400640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8806b5a0 0x7f4a8806da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.095+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f4aa00aa7d0 con 0x7f4aa8141aa0 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.096+0000 7f4aa7400640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8806b5a0 0x7f4a8806da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.098+0000 7f4aa7400640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8806b5a0 0x7f4a8806da60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4aa806f190 tx=0x7f4a98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.100+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4aa002ce20 con 0x7f4aa8141aa0 2024-04-25T10:13:44.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.304+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f4aa80cf560 con 0x7f4aa8141aa0 2024-04-25T10:13:44.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.316+0000 7f4a9f400640 1 -- 172.21.15.165:0/2908592988 <== mon.1 v2:172.21.15.172:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4aa0079150 con 0x7f4aa8141aa0 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8806b5a0 msgr2=0x7f4a8806da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8806b5a0 0x7f4a8806da60 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f4aa806f190 tx=0x7f4a98008040 comp rx=0 tx=0).stop 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 msgr2=0x7f4aa8141f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 0x7f4aa8141f50 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000a8a0 tx=0x7f4aa0001dc0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 shutdown_connections 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8806b5a0 0x7f4a8806da60 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aa8141aa0 0x7f4aa8141f50 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aa8144330 0x7f4aa81447b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 --2- 172.21.15.165:0/2908592988 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a8c0051c0 0x7f4aa80adca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.318+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 >> 172.21.15.165:0/2908592988 conn(0x7f4aa806abe0 msgr2=0x7f4aa80b2930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.319+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 shutdown_connections 2024-04-25T10:13:44.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.319+0000 7f4aae600640 1 -- 172.21.15.165:0/2908592988 wait complete. 2024-04-25T10:13:44.341 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume' 2024-04-25T10:13:44.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.536+0000 7fb817800640 1 Processor -- start 2024-04-25T10:13:44.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.536+0000 7fb817800640 1 -- start start 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb817800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8100cfc20 0x7fb8100cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb817800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb81015df50 0x7fb81006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb817800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb810071650 con 0x7fb81015db70 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb817800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8100cbd80 con 0x7fb8100cfc20 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb817800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb810161910 con 0x7fb81015df50 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb80fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8100cfc20 0x7fb8100cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb814e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb81015db70 0x7fb8100cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48252/0 (socket says 172.21.15.165:48252) 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.538+0000 7fb814e00640 1 -- 172.21.15.165:0/2544277630 learned_addr learned my addr 172.21.15.165:0/2544277630 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.539+0000 7fb815800640 1 --2- 172.21.15.165:0/2544277630 >> v2:172.21.15.172:3300/0 conn(0x7fb81015df50 0x7fb81006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.539+0000 7fb80fe00640 1 -- 172.21.15.165:0/2544277630 >> v2:172.21.15.172:3300/0 conn(0x7fb81015df50 msgr2=0x7fb81006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.539+0000 7fb80fe00640 1 --2- 172.21.15.165:0/2544277630 >> v2:172.21.15.172:3300/0 conn(0x7fb81015df50 0x7fb81006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.539+0000 7fb80fe00640 1 -- 172.21.15.165:0/2544277630 >> v1:172.21.15.165:6789/0 conn(0x7fb81015db70 legacy=0x7fb8100cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.539+0000 7fb80f400640 1 -- 172.21.15.165:0/2544277630 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1717043621 0 0) 0x7fb804001110 con 0x7fb81015db70 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.539+0000 7fb80fe00640 1 -- 172.21.15.165:0/2544277630 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100cf230 con 0x7fb8100cfc20 2024-04-25T10:13:44.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.540+0000 7fb80fe00640 1 --2- 172.21.15.165:0/2544277630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8100cb840 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fb8100b4d10 tx=0x7fb7fc02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87d3b6a199a9c60d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb80f400640 1 -- 172.21.15.165:0/2544277630 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc037070 con 0x7fb8100cfc20 2024-04-25T10:13:44.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb80f400640 1 -- 172.21.15.165:0/2544277630 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7fc032440 con 0x7fb8100cfc20 2024-04-25T10:13:44.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb80f400640 1 -- 172.21.15.165:0/2544277630 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc032810 con 0x7fb8100cfc20 2024-04-25T10:13:44.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb817800640 1 -- 172.21.15.165:0/2544277630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 msgr2=0x7fb8100cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb817800640 1 --2- 172.21.15.165:0/2544277630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8100cb840 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fb8100b4d10 tx=0x7fb7fc02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb817800640 1 -- 172.21.15.165:0/2544277630 shutdown_connections 2024-04-25T10:13:44.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb817800640 1 --2- 172.21.15.165:0/2544277630 >> v2:172.21.15.172:3300/0 conn(0x7fb81015df50 0x7fb81006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb817800640 1 --2- 172.21.15.165:0/2544277630 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8100cb840 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.541+0000 7fb817800640 1 -- 172.21.15.165:0/2544277630 >> 172.21.15.165:0/2544277630 conn(0x7fb81006abe0 msgr2=0x7fb81006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.542+0000 7fb817800640 1 -- 172.21.15.165:0/2544277630 shutdown_connections 2024-04-25T10:13:44.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.542+0000 7fb817800640 1 -- 172.21.15.165:0/2544277630 wait complete. 2024-04-25T10:13:44.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.543+0000 7fb817800640 1 Processor -- start 2024-04-25T10:13:44.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.543+0000 7fb817800640 1 -- start start 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb817800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8101542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb817800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb81015db70 0x7fb8100af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb817800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81015df50 0x7fb8100adea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb817800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810162120 con 0x7fb8100cfc20 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb817800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb810161e40 con 0x7fb81015df50 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb817800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb810162640 con 0x7fb81015db70 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.544+0000 7fb814e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8101542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8101542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40650/0 (socket says 172.21.15.165:40650) 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 -- 172.21.15.165:0/1186190731 learned_addr learned my addr 172.21.15.165:0/1186190731 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb815800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81015df50 0x7fb8100adea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb80fe00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb81015db70 0x7fb8100af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 -- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb81015db70 msgr2=0x7fb8100af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb81015db70 0x7fb8100af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 -- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81015df50 msgr2=0x7fb8100adea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81015df50 0x7fb8100adea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 -- 172.21.15.165:0/1186190731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8100afd40 con 0x7fb8100cfc20 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb815800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81015df50 0x7fb8100adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.545+0000 7fb814e00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8101542e0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fb80400c780 tx=0x7fb80400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.546+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb804018070 con 0x7fb8100cfc20 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.546+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8040072d0 con 0x7fb8100cfc20 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.546+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8040047e0 con 0x7fb8100cfc20 2024-04-25T10:13:44.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.546+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8100b0020 con 0x7fb8100cfc20 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.546+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8100b05e0 con 0x7fb8100cfc20 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.547+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8100b01f0 con 0x7fb8100cfc20 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.548+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb804004980 con 0x7fb8100cfc20 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.549+0000 7fb80cc00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb7f006b4d0 0x7fb7f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.549+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb804089580 con 0x7fb8100cfc20 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.549+0000 7fb80fe00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb7f006b4d0 0x7fb7f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.550+0000 7fb80fe00640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb7f006b4d0 0x7fb7f006d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc034000 tx=0x7fb7fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:44.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.554+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb810071650 con 0x7fb8100cfc20 2024-04-25T10:13:44.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.822+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]} v 0) v1 -- 0x7fb810073990 con 0x7fb8100cfc20 2024-04-25T10:13:44.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.833+0000 7fb80cc00640 1 -- 172.21.15.165:0/1186190731 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume"]}]=0 v539) v1 ==== 155+0+0 (secure 0 0 0) 0x7fb804004c70 con 0x7fb8100cfc20 2024-04-25T10:13:44.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb7f006b4d0 msgr2=0x7fb7f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb7f006b4d0 0x7fb7f006d990 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc034000 tx=0x7fb7fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:44.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 msgr2=0x7fb8101542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8101542e0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fb80400c780 tx=0x7fb80400cc40 comp rx=0 tx=0).stop 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 shutdown_connections 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb7f006b4d0 0x7fb7f006d990 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb81015df50 0x7fb8100adea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb81015db70 0x7fb8100af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 --2- 172.21.15.165:0/1186190731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8100cfc20 0x7fb8101542e0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:44.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.836+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 >> 172.21.15.165:0/1186190731 conn(0x7fb81006abe0 msgr2=0x7fb81015b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:44.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.837+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 shutdown_connections 2024-04-25T10:13:44.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:44.837+0000 7fb817800640 1 -- 172.21.15.165:0/1186190731 wait complete. 2024-04-25T10:13:44.859 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:13:44.859 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:13:44.860 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:45.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.065+0000 7f6c96a00640 1 Processor -- start 2024-04-25T10:13:45.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.065+0000 7f6c96a00640 1 -- start start 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.066+0000 7f6c96a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6c90154ee0 0x7f6c900cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.066+0000 7f6c96a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 0x7f6c900cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.066+0000 7f6c96a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c90071480 con 0x7f6c900ce3c0 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.066+0000 7f6c96a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c900ce830 con 0x7f6c90154ee0 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.066+0000 7f6c96a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c90072090 con 0x7f6c900cfcf0 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c94a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6c900ce3c0 0x7f6c9015e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48264/0 (socket says 172.21.15.165:48264) 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c94a00640 1 -- 172.21.15.165:0/76375090 learned_addr learned my addr 172.21.15.165:0/76375090 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:45.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8fe00640 1 --2- 172.21.15.165:0/76375090 >> v2:172.21.15.172:3300/0 conn(0x7f6c90154ee0 0x7f6c900cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c87400640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 0x7f6c900cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 773365281 0 0) 0x7f6c90071480 con 0x7f6c900ce3c0 2024-04-25T10:13:45.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c74003520 con 0x7f6c900ce3c0 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2776230899 0 0) 0x7f6c74003520 con 0x7f6c900ce3c0 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 msgr2=0x7f6c900cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8f400640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 0x7f6c900cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.067+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 >> v2:172.21.15.172:3300/0 conn(0x7f6c90154ee0 msgr2=0x7f6c900cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 --2- 172.21.15.165:0/76375090 >> v2:172.21.15.172:3300/0 conn(0x7f6c90154ee0 0x7f6c900cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90071760 con 0x7f6c900ce3c0 2024-04-25T10:13:45.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6c880019f0 con 0x7f6c900ce3c0 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 >> v1:172.21.15.165:6789/0 conn(0x7f6c900ce3c0 legacy=0x7f6c9015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c740039b0 0x7f6c74003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 0x7f6c74004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c900ce830 con 0x7f6c74004d40 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7400dbd0 con 0x7f6c740039b0 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7400dd50 con 0x7f6c900cfcf0 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c94a00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8fe00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c740039b0 0x7f6c74003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c94a00640 1 -- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 msgr2=0x7f6c74004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c87400640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 0x7f6c74004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c94a00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c900cfcf0 0x7f6c74004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c94a00640 1 -- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c740039b0 msgr2=0x7f6c74003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c94a00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c740039b0 0x7f6c74003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c94a00640 1 -- 172.21.15.165:0/76375090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c74003520 con 0x7f6c74004d40 2024-04-25T10:13:45.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.068+0000 7f6c8fe00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c740039b0 0x7f6c74003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.069+0000 7f6c94a00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800a7a0 tx=0x7f6c74005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615ec7ff25bef8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8800a3f0 con 0x7f6c74004d40 2024-04-25T10:13:45.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8802f070 con 0x7f6c74004d40 2024-04-25T10:13:45.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c8f400640 1 -- 172.21.15.165:0/76375090 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8802bc20 con 0x7f6c74004d40 2024-04-25T10:13:45.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c96a00640 1 -- 172.21.15.165:0/76375090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 msgr2=0x7f6c740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c96a00640 1 --2- 172.21.15.165:0/76375090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c740051c0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800a7a0 tx=0x7f6c74005830 comp rx=0 tx=0).stop 2024-04-25T10:13:45.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c94a00640 1 -- 172.21.15.165:0/76375090 reap_dead start 2024-04-25T10:13:45.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c96a00640 1 -- 172.21.15.165:0/76375090 shutdown_connections 2024-04-25T10:13:45.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.070+0000 7f6c96a00640 1 -- 172.21.15.165:0/76375090 >> 172.21.15.165:0/76375090 conn(0x7f6c9006abe0 msgr2=0x7f6c9006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.071+0000 7f6c96a00640 1 -- 172.21.15.165:0/76375090 shutdown_connections 2024-04-25T10:13:45.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.071+0000 7f6c96a00640 1 -- 172.21.15.165:0/76375090 wait complete. 2024-04-25T10:13:45.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.072+0000 7f6c96a00640 1 Processor -- start 2024-04-25T10:13:45.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.072+0000 7f6c96a00640 1 -- start start 2024-04-25T10:13:45.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.073+0000 7f6c96a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c9015d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.073+0000 7f6c96a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c90142f60 0x7f6c90144810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.073+0000 7f6c96a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9014def0 0x7f6c9014e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.073+0000 7f6c96a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c90071230 con 0x7f6c74004d40 2024-04-25T10:13:45.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.073+0000 7f6c96a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c90071d20 con 0x7f6c9014def0 2024-04-25T10:13:45.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.073+0000 7f6c96a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c901552c0 con 0x7f6c90142f60 2024-04-25T10:13:45.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c9015d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c9015d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40660/0 (socket says 172.21.15.165:40660) 2024-04-25T10:13:45.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 -- 172.21.15.165:0/121696502 learned_addr learned my addr 172.21.15.165:0/121696502 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:45.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 -- 172.21.15.165:0/121696502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c90142f60 msgr2=0x7f6c90144810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c90142f60 0x7f6c90144810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 -- 172.21.15.165:0/121696502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9014def0 msgr2=0x7f6c9014e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9014def0 0x7f6c9014e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 -- 172.21.15.165:0/121696502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c901433e0 con 0x7f6c74004d40 2024-04-25T10:13:45.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.074+0000 7f6c8fe00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c9015d710 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f6c80010a30 tx=0x7f6c80010ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.075+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8000f040 con 0x7f6c74004d40 2024-04-25T10:13:45.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.075+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8000ec00 con 0x7f6c74004d40 2024-04-25T10:13:45.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.075+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8001c400 con 0x7f6c74004d40 2024-04-25T10:13:45.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.075+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c901436b0 con 0x7f6c74004d40 2024-04-25T10:13:45.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.075+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c90143c70 con 0x7f6c74004d40 2024-04-25T10:13:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.076+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c90073990 con 0x7f6c74004d40 2024-04-25T10:13:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.077+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c8001c5a0 con 0x7f6c74004d40 2024-04-25T10:13:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.078+0000 7f6c8cc00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c6c06b4d0 0x7f6c6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.079+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6c8008a770 con 0x7f6c74004d40 2024-04-25T10:13:45.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.079+0000 7f6c87400640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c6c06b4d0 0x7f6c6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.079+0000 7f6c87400640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c6c06b4d0 0x7f6c6c06d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c901f6ea0 tx=0x7f6c78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.083+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c80018030 con 0x7f6c74004d40 2024-04-25T10:13:45.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.282+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c9006d360 con 0x7f6c74004d40 2024-04-25T10:13:45.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.284+0000 7f6c8cc00640 1 -- 172.21.15.165:0/121696502 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+13339 (secure 0 0 0) 0x7f6c800596c0 con 0x7f6c74004d40 2024-04-25T10:13:45.288 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:45.288 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":116,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:43.814918+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/437969658":"2024-04-26T10:13:40.658933+0000","172.21.15.172:6833/437969658":"2024-04-26T10:13:40.658933+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.286+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c6c06b4d0 msgr2=0x7f6c6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.286+0000 7f6c96a00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c6c06b4d0 0x7f6c6c06d990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c901f6ea0 tx=0x7f6c78008040 comp rx=0 tx=0).stop 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.286+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 msgr2=0x7f6c9015d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.286+0000 7f6c96a00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c9015d710 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f6c80010a30 tx=0x7f6c80010ef0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.286+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 shutdown_connections 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.286+0000 7f6c96a00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c6c06b4d0 0x7f6c6c06d990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.287+0000 7f6c96a00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c9014def0 0x7f6c9014e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.287+0000 7f6c96a00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c90142f60 0x7f6c90144810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.287+0000 7f6c96a00640 1 --2- 172.21.15.165:0/121696502 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c74004d40 0x7f6c9015d710 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.287+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 >> 172.21.15.165:0/121696502 conn(0x7f6c9006abe0 msgr2=0x7f6c900cb840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.287+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 shutdown_connections 2024-04-25T10:13:45.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.287+0000 7f6c96a00640 1 -- 172.21.15.165:0/121696502 wait complete. 2024-04-25T10:13:45.309 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:45.310 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:45.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.512+0000 7f070ee00640 1 Processor -- start 2024-04-25T10:13:45.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.513+0000 7f070ee00640 1 -- start start 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f070ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f070806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f070ee00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f07081452a0 0x7f070806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f070ee00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07080cccc0 con 0x7f07081446d0 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f070ee00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07080b2b20 con 0x7f07081452a0 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f070ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f070806d400 con 0x7f0708144d50 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f070ce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f07081452a0 0x7f070806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f0707e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f07081446d0 0x7f07080b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:48274/0 (socket says 172.21.15.165:48274) 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.514+0000 7f0707e00640 1 -- 172.21.15.165:0/3967012029 learned_addr learned my addr 172.21.15.165:0/3967012029 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f0707400640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f070806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f070ce00640 1 -- 172.21.15.165:0/3967012029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 msgr2=0x7f070806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f070ce00640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f070806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f070ce00640 1 -- 172.21.15.165:0/3967012029 >> v1:172.21.15.165:6789/0 conn(0x7f07081446d0 legacy=0x7f07080b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f0706a00640 1 -- 172.21.15.165:0/3967012029 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4294014913 0 0) 0x7f07080cccc0 con 0x7f07081446d0 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f070ce00640 1 -- 172.21.15.165:0/3967012029 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080cc140 con 0x7f07081452a0 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.515+0000 7f0707400640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f070806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.516+0000 7f070ce00640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f070806ddd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f07000097a0 tx=0x7f070002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1be718d5d2830e22 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.516+0000 7f0706a00640 1 -- 172.21.15.165:0/3967012029 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700037070 con 0x7f07081452a0 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.516+0000 7f0706a00640 1 -- 172.21.15.165:0/3967012029 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700032440 con 0x7f07081452a0 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.516+0000 7f0706a00640 1 -- 172.21.15.165:0/3967012029 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700032810 con 0x7f07081452a0 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 -- 172.21.15.165:0/3967012029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 msgr2=0x7f070806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f070806ddd0 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7f07000097a0 tx=0x7f070002a180 comp rx=0 tx=0).stop 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 -- 172.21.15.165:0/3967012029 shutdown_connections 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f070806ddd0 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 --2- 172.21.15.165:0/3967012029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f070806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 -- 172.21.15.165:0/3967012029 >> 172.21.15.165:0/3967012029 conn(0x7f070806abe0 msgr2=0x7f070806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 -- 172.21.15.165:0/3967012029 shutdown_connections 2024-04-25T10:13:45.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.517+0000 7f070ee00640 1 -- 172.21.15.165:0/3967012029 wait complete. 2024-04-25T10:13:45.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.518+0000 7f070ee00640 1 Processor -- start 2024-04-25T10:13:45.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.519+0000 7f070ee00640 1 -- start start 2024-04-25T10:13:45.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07081446d0 0x7f07080b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f07080afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f07081400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080cc9e0 con 0x7f07081452a0 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07080cc700 con 0x7f07081446d0 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0708144ab0 con 0x7f0708144d50 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f07081400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f07081400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40682/0 (socket says 172.21.15.165:40682) 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 -- 172.21.15.165:0/599738331 learned_addr learned my addr 172.21.15.165:0/599738331 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 -- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 msgr2=0x7f07080afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f07080afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 -- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07081446d0 msgr2=0x7f07080b1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f0707e00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07081446d0 0x7f07080b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07081446d0 0x7f07080b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.520+0000 7f070ce00640 1 -- 172.21.15.165:0/599738331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07080b0360 con 0x7f07081452a0 2024-04-25T10:13:45.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.521+0000 7f070ce00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f07081400a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f0700035040 tx=0x7f07000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.521+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700037040 con 0x7f07081452a0 2024-04-25T10:13:45.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.521+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0700042070 con 0x7f07081452a0 2024-04-25T10:13:45.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.521+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0700030b50 con 0x7f07081452a0 2024-04-25T10:13:45.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.521+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07080c4850 con 0x7f07081452a0 2024-04-25T10:13:45.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.521+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07080b08e0 con 0x7f07081452a0 2024-04-25T10:13:45.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.522+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0708073a00 con 0x7f07081452a0 2024-04-25T10:13:45.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.523+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0700030cf0 con 0x7f07081452a0 2024-04-25T10:13:45.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.525+0000 7f06ffe00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06e406b4d0 0x7f06e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.525+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f07000aac70 con 0x7f07081452a0 2024-04-25T10:13:45.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.525+0000 7f0707e00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06e406b4d0 0x7f06e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.525+0000 7f0707e00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06e406b4d0 0x7f06e406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f06f80012e0 tx=0x7f06f8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.529+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0700032e20 con 0x7f07081452a0 2024-04-25T10:13:45.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.726+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f070806eab0 con 0x7f07081452a0 2024-04-25T10:13:45.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.727+0000 7f06ffe00640 1 -- 172.21.15.165:0/599738331 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v116) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0700079c50 con 0x7f07081452a0 2024-04-25T10:13:45.732 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:13:45.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06e406b4d0 msgr2=0x7f06e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06e406b4d0 0x7f06e406d990 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f06f80012e0 tx=0x7f06f8009040 comp rx=0 tx=0).stop 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 msgr2=0x7f07081400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f07081400a0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7f0700035040 tx=0x7f07000046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 shutdown_connections 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f06e406b4d0 0x7f06e406d990 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07081452a0 0x7f07081400a0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0708144d50 0x7f07080afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 --2- 172.21.15.165:0/599738331 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07081446d0 0x7f07080b1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 >> 172.21.15.165:0/599738331 conn(0x7f070806abe0 msgr2=0x7f07080af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 shutdown_connections 2024-04-25T10:13:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.730+0000 7f070ee00640 1 -- 172.21.15.165:0/599738331 wait complete. 2024-04-25T10:13:45.754 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:13:45.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.950+0000 7f99f5c00640 1 Processor -- start 2024-04-25T10:13:45.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.951+0000 7f99f5c00640 1 -- start start 2024-04-25T10:13:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99f5c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99f00d13f0 0x7f99f00d1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99f5c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99f00d1dc0 0x7f99f00b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99f5c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99f00d0350 con 0x7f99f015bd30 2024-04-25T10:13:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99f5c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00d0630 con 0x7f99f00d1dc0 2024-04-25T10:13:45.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99f5c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99f00d0910 con 0x7f99f00d13f0 2024-04-25T10:13:45.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99ef400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99f00d1dc0 0x7f99f00b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.952+0000 7f99ef400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99f00d1dc0 0x7f99f00b4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40696/0 (socket says 172.21.15.165:40696) 2024-04-25T10:13:45.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.953+0000 7f99ef400640 1 -- 172.21.15.165:0/1396956445 learned_addr learned my addr 172.21.15.165:0/1396956445 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:45.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.953+0000 7f99e6000640 1 --2- 172.21.15.165:0/1396956445 >> v2:172.21.15.172:3300/0 conn(0x7f99f00d13f0 0x7f99f00d1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.953+0000 7f99ef400640 1 -- 172.21.15.165:0/1396956445 >> v2:172.21.15.172:3300/0 conn(0x7f99f00d13f0 msgr2=0x7f99f00d1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.953+0000 7f99ef400640 1 --2- 172.21.15.165:0/1396956445 >> v2:172.21.15.172:3300/0 conn(0x7f99f00d13f0 0x7f99f00d1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.953+0000 7f99ef400640 1 -- 172.21.15.165:0/1396956445 >> v1:172.21.15.172:6789/0 conn(0x7f99f015bd30 legacy=0x7f99f00cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:45.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.953+0000 7f99ef400640 1 -- 172.21.15.165:0/1396956445 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f00d0bf0 con 0x7f99f00d1dc0 2024-04-25T10:13:45.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.954+0000 7f99ef400640 1 --2- 172.21.15.165:0/1396956445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f00d1dc0 0x7f99f00b4a30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f99f015f7f0 tx=0x7f99e802a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3abfe4d12dc4275 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.954+0000 7f99ee000640 1 -- 172.21.15.165:0/1396956445 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8036070 con 0x7f99f00d1dc0 2024-04-25T10:13:45.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.954+0000 7f99ee000640 1 -- 172.21.15.165:0/1396956445 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8031600 con 0x7f99f00d1dc0 2024-04-25T10:13:45.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.954+0000 7f99ee000640 1 -- 172.21.15.165:0/1396956445 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80319d0 con 0x7f99f00d1dc0 2024-04-25T10:13:45.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 -- 172.21.15.165:0/1396956445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f00d1dc0 msgr2=0x7f99f00b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1396956445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f00d1dc0 0x7f99f00b4a30 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7f99f015f7f0 tx=0x7f99e802a780 comp rx=0 tx=0).stop 2024-04-25T10:13:45.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 -- 172.21.15.165:0/1396956445 shutdown_connections 2024-04-25T10:13:45.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1396956445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f00d1dc0 0x7f99f00b4a30 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1396956445 >> v2:172.21.15.172:3300/0 conn(0x7f99f00d13f0 0x7f99f00d1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 -- 172.21.15.165:0/1396956445 >> 172.21.15.165:0/1396956445 conn(0x7f99f006f590 msgr2=0x7f99f006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:45.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 -- 172.21.15.165:0/1396956445 shutdown_connections 2024-04-25T10:13:45.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.955+0000 7f99f5c00640 1 -- 172.21.15.165:0/1396956445 wait complete. 2024-04-25T10:13:45.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.957+0000 7f99f5c00640 1 Processor -- start 2024-04-25T10:13:45.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.957+0000 7f99f5c00640 1 -- start start 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99f5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f00d13f0 0x7f99f00c8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99f5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00d1dc0 0x7f99f00cb2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99f5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 0x7f99f00cb820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99f5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00d0010 con 0x7f99f015bd30 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99f5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f015c110 con 0x7f99f00d1dc0 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99f5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99f00b73b0 con 0x7f99f00d13f0 2024-04-25T10:13:45.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 0x7f99f00cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 0x7f99f00cb820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40704/0 (socket says 172.21.15.165:40704) 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.958+0000 7f99ef400640 1 -- 172.21.15.165:0/1465107938 learned_addr learned my addr 172.21.15.165:0/1465107938 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99e6000640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00d1dc0 0x7f99f00cb2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99eea00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f00d13f0 0x7f99f00c8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99ef400640 1 -- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f00d13f0 msgr2=0x7f99f00c8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99ef400640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f00d13f0 0x7f99f00c8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99ef400640 1 -- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00d1dc0 msgr2=0x7f99f00cb2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99ef400640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00d1dc0 0x7f99f00cb2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99ef400640 1 -- 172.21.15.165:0/1465107938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f006c9e0 con 0x7f99f015bd30 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99eea00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f00d13f0 0x7f99f00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:45.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.959+0000 7f99ef400640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 0x7f99f00cb820 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f99e802ad70 tx=0x7f99e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.960+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8036050 con 0x7f99f015bd30 2024-04-25T10:13:45.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.960+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e8042070 con 0x7f99f015bd30 2024-04-25T10:13:45.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.960+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f006ccc0 con 0x7f99f015bd30 2024-04-25T10:13:45.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.960+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e8009510 con 0x7f99f015bd30 2024-04-25T10:13:45.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.960+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f006d280 con 0x7f99f015bd30 2024-04-25T10:13:45.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.961+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99f0073990 con 0x7f99f015bd30 2024-04-25T10:13:45.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.962+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99e8007bf0 con 0x7f99f015bd30 2024-04-25T10:13:45.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.963+0000 7f99e7400640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99cc06b4d0 0x7f99cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:45.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.964+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f99e80ab050 con 0x7f99f015bd30 2024-04-25T10:13:45.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.964+0000 7f99eea00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99cc06b4d0 0x7f99cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:45.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.964+0000 7f99eea00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99cc06b4d0 0x7f99cc06d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f99e000a810 tx=0x7f99e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:45.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:45.968+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99e807a0c0 con 0x7f99f015bd30 2024-04-25T10:13:46.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.225+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f99f00b1570 con 0x7f99f015bd30 2024-04-25T10:13:46.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.847+0000 7f99e7400640 1 -- 172.21.15.165:0/1465107938 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v541) v1 ==== 169+0+0 (secure 0 0 0) 0x7f99e80799c0 con 0x7f99f015bd30 2024-04-25T10:13:46.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.850+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99cc06b4d0 msgr2=0x7f99cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.850+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99cc06b4d0 0x7f99cc06d990 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f99e000a810 tx=0x7f99e0008040 comp rx=0 tx=0).stop 2024-04-25T10:13:46.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.850+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 msgr2=0x7f99f00cb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:46.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.850+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 0x7f99f00cb820 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7f99e802ad70 tx=0x7f99e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:13:46.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 shutdown_connections 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99cc06b4d0 0x7f99cc06d990 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99f015bd30 0x7f99f00cb820 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99f00d1dc0 0x7f99f00cb2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 --2- 172.21.15.165:0/1465107938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99f00d13f0 0x7f99f00c8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 >> 172.21.15.165:0/1465107938 conn(0x7f99f006f590 msgr2=0x7f99f00cc100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 shutdown_connections 2024-04-25T10:13:46.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:46.851+0000 7f99f5c00640 1 -- 172.21.15.165:0/1465107938 wait complete. 2024-04-25T10:13:46.875 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:13:47.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.071+0000 7f8fbfe00640 1 Processor -- start 2024-04-25T10:13:47.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.071+0000 7f8fbfe00640 1 -- start start 2024-04-25T10:13:47.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbfe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8fb80cc4f0 0x7f8fb80cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 0x7f8fb80cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbfe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb8072280 con 0x7f8fb815b270 2024-04-25T10:13:47.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbfe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb80ce4c0 con 0x7f8fb80cc4f0 2024-04-25T10:13:47.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8071270 con 0x7f8fb80cdb20 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8fb815b270 0x7f8fb815b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58352/0 (socket says 172.21.15.165:58352) 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.073+0000 7f8fbde00640 1 -- 172.21.15.165:0/2951106069 learned_addr learned my addr 172.21.15.165:0/2951106069 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fbca00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 0x7f8fb80cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2951106069 >> v2:172.21.15.172:3300/0 conn(0x7f8fb80cc4f0 0x7f8fb80cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 496964301 0 0) 0x7f8fb8072280 con 0x7f8fb815b270 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fa0005730 con 0x7f8fb815b270 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2008569475 0 0) 0x7f8fa0005730 con 0x7f8fb815b270 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 msgr2=0x7f8fb80cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 0x7f8fb80cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 >> v2:172.21.15.172:3300/0 conn(0x7f8fb80cc4f0 msgr2=0x7f8fb80cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 --2- 172.21.15.165:0/2951106069 >> v2:172.21.15.172:3300/0 conn(0x7f8fb80cc4f0 0x7f8fb80cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80714c0 con 0x7f8fb815b270 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8fb00019f0 con 0x7f8fb815b270 2024-04-25T10:13:47.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 >> v1:172.21.15.165:6789/0 conn(0x7f8fb815b270 legacy=0x7f8fb815b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fa0005bc0 0x7f8fa0005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fa0006a10 0x7f8fa0006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 0x7f8fa00073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8071270 con 0x7f8fa0005bc0 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa000dc00 con 0x7f8fa0006a10 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa000de00 con 0x7f8fb80cdb20 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.074+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fa0005bc0 0x7f8fa0005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbca00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fa0006a10 0x7f8fa0006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbde00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 0x7f8fa00073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbd400640 1 -- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 msgr2=0x7f8fa00073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fb80cdb20 0x7f8fa00073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbd400640 1 -- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fa0006a10 msgr2=0x7f8fa0006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fa0006a10 0x7f8fa0006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbd400640 1 -- 172.21.15.165:0/2951106069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0005730 con 0x7f8fa0005bc0 2024-04-25T10:13:47.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.075+0000 7f8fbca00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fa0006a10 0x7f8fa0006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.076+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fa0005bc0 0x7f8fa0005fe0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ba00 tx=0x7f8fa0007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5a9176da9e12964 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.076+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac030070 con 0x7f8fa0005bc0 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.076+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac02adb0 con 0x7f8fa0005bc0 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.076+0000 7f8fb7e00640 1 -- 172.21.15.165:0/2951106069 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00b600 con 0x7f8fa0005bc0 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2951106069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fa0005bc0 msgr2=0x7f8fa0005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2951106069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fa0005bc0 0x7f8fa0005fe0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00ba00 tx=0x7f8fa0007a40 comp rx=0 tx=0).stop 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbde00640 1 -- 172.21.15.165:0/2951106069 reap_dead start 2024-04-25T10:13:47.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2951106069 shutdown_connections 2024-04-25T10:13:47.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2951106069 >> 172.21.15.165:0/2951106069 conn(0x7f8fb806abe0 msgr2=0x7f8fb806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2951106069 shutdown_connections 2024-04-25T10:13:47.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.077+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2951106069 wait complete. 2024-04-25T10:13:47.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.079+0000 7f8fbfe00640 1 Processor -- start 2024-04-25T10:13:47.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.079+0000 7f8fbfe00640 1 -- start start 2024-04-25T10:13:47.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fa0005bc0 0x7f8fb81f3250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 0x7f8fb80c7510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb80c7a50 0x7f8fb81f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8071a80 con 0x7f8fb80c4b60 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb815bd90 con 0x7f8fb80c7a50 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb8071d60 con 0x7f8fa0005bc0 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 0x7f8fb80c7510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 0x7f8fb80c7510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39924/0 (socket says 172.21.15.165:39924) 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 -- 172.21.15.165:0/2262632552 learned_addr learned my addr 172.21.15.165:0/2262632552 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 -- 172.21.15.165:0/2262632552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fa0005bc0 msgr2=0x7f8fb81f3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fa0005bc0 0x7f8fb81f3250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 -- 172.21.15.165:0/2262632552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb80c7a50 msgr2=0x7f8fb81f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb80c7a50 0x7f8fb81f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.080+0000 7f8fbca00640 1 -- 172.21.15.165:0/2262632552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb80c5860 con 0x7f8fb80c4b60 2024-04-25T10:13:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.081+0000 7f8fbca00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 0x7f8fb80c7510 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400e7e0 tx=0x7f8fa400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.081+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa4017070 con 0x7f8fb80c4b60 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.081+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa400c6d0 con 0x7f8fb80c4b60 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.081+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa400f040 con 0x7f8fb80c4b60 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.081+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb80c5b40 con 0x7f8fb80c4b60 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.081+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb80c6100 con 0x7f8fb80c4b60 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.082+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb8073a00 con 0x7f8fb80c4b60 2024-04-25T10:13:47.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.083+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fa401a070 con 0x7f8fb80c4b60 2024-04-25T10:13:47.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.084+0000 7f8fb5600640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9806b500 0x7f8f9806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.084+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8fa4089c50 con 0x7f8fb80c4b60 2024-04-25T10:13:47.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.084+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9806b500 0x7f8f9806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.085+0000 7f8fbd400640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9806b500 0x7f8f9806d9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00079f0 tx=0x7f8fac02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.088+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fa4058c30 con 0x7f8fb80c4b60 2024-04-25T10:13:47.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.270+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8fb806d3a0 con 0x7f8fb80c4b60 2024-04-25T10:13:47.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.276+0000 7f8fb5600640 1 -- 172.21.15.165:0/2262632552 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8fa4058530 con 0x7f8fb80c4b60 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.279+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9806b500 msgr2=0x7f8f9806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.279+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9806b500 0x7f8f9806d9c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8fa00079f0 tx=0x7f8fac02c000 comp rx=0 tx=0).stop 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.279+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 msgr2=0x7f8fb80c7510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.279+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 0x7f8fb80c7510 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f8fa400e7e0 tx=0x7f8fa400eca0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 shutdown_connections 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8f9806b500 0x7f8f9806d9c0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fb80c7a50 0x7f8fb81f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fb80c4b60 0x7f8fb80c7510 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 --2- 172.21.15.165:0/2262632552 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fa0005bc0 0x7f8fb81f3250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 >> 172.21.15.165:0/2262632552 conn(0x7f8fb806abe0 msgr2=0x7f8fb815b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 shutdown_connections 2024-04-25T10:13:47.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.280+0000 7f8fbfe00640 1 -- 172.21.15.165:0/2262632552 wait complete. 2024-04-25T10:13:47.302 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:13:47.302 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:47.404 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:13:47.510 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:13:47.510 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:13:47.511 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:47.567 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:47.567 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:13:47.567 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:47.625 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:47.625 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:13:47.625 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:47.681 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:13:47.681 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:13:47.681 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:13:47.738 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:13:47.738 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:13:47.738 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:47.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.946+0000 7efe50400640 1 Processor -- start 2024-04-25T10:13:47.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.946+0000 7efe50400640 1 -- start start 2024-04-25T10:13:47.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.947+0000 7efe50400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 0x7efe480ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe50400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe48144b30 con 0x7efe4806d290 2024-04-25T10:13:47.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe50400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe480712d0 con 0x7efe48144750 2024-04-25T10:13:47.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe50400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe480715b0 con 0x7efe480ce920 2024-04-25T10:13:47.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe4e400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efe4806d290 0x7efe4806d720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58366/0 (socket says 172.21.15.165:58366) 2024-04-25T10:13:47.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe4e400640 1 -- 172.21.15.165:0/2360224550 learned_addr learned my addr 172.21.15.165:0/2360224550 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:47.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe4d000640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 0x7efe480ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1635767496 0 0) 0x7efe48144b30 con 0x7efe4806d290 2024-04-25T10:13:47.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.948+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe30003520 con 0x7efe4806d290 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3560023244 0 0) 0x7efe30003520 con 0x7efe4806d290 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 msgr2=0x7efe480ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 0x7efe480ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 >> v1:172.21.15.172:6789/0 conn(0x7efe48144750 legacy=0x7efe4806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe48071890 con 0x7efe4806d290 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7efe400019f0 con 0x7efe4806d290 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 >> v1:172.21.15.165:6789/0 conn(0x7efe4806d290 legacy=0x7efe4806d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe30003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 0x7efe30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe30004d40 0x7efe300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480715b0 con 0x7efe300039e0 2024-04-25T10:13:47.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe3000bb50 con 0x7efe30004d40 2024-04-25T10:13:47.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe3000bd50 con 0x7efe480ce920 2024-04-25T10:13:47.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.949+0000 7efe4da00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe30003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.950+0000 7efe4da00640 1 -- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 msgr2=0x7efe30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.950+0000 7efe4da00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480ce920 0x7efe30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.950+0000 7efe4da00640 1 -- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe30004d40 msgr2=0x7efe300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:47.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.950+0000 7efe4da00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe30004d40 0x7efe300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.950+0000 7efe4da00640 1 -- 172.21.15.165:0/2360224550 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe30003520 con 0x7efe300039e0 2024-04-25T10:13:47.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe4da00640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe30003e00 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7efe3c001750 tx=0x7efe30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b73c2ef9437209c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c001370 con 0x7efe300039e0 2024-04-25T10:13:47.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c0052d0 con 0x7efe300039e0 2024-04-25T10:13:47.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe47e00640 1 -- 172.21.15.165:0/2360224550 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c02abe0 con 0x7efe300039e0 2024-04-25T10:13:47.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe50400640 1 -- 172.21.15.165:0/2360224550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 msgr2=0x7efe30003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe50400640 1 --2- 172.21.15.165:0/2360224550 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe30003e00 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7efe3c001750 tx=0x7efe30005830 comp rx=0 tx=0).stop 2024-04-25T10:13:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.951+0000 7efe4e400640 1 -- 172.21.15.165:0/2360224550 reap_dead start 2024-04-25T10:13:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.952+0000 7efe50400640 1 -- 172.21.15.165:0/2360224550 shutdown_connections 2024-04-25T10:13:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.952+0000 7efe50400640 1 -- 172.21.15.165:0/2360224550 >> 172.21.15.165:0/2360224550 conn(0x7efe4806abe0 msgr2=0x7efe4806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.952+0000 7efe50400640 1 -- 172.21.15.165:0/2360224550 shutdown_connections 2024-04-25T10:13:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.952+0000 7efe50400640 1 -- 172.21.15.165:0/2360224550 wait complete. 2024-04-25T10:13:47.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.953+0000 7efe50400640 1 Processor -- start 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.954+0000 7efe50400640 1 -- start start 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.954+0000 7efe50400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe4815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.954+0000 7efe50400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480afa80 0x7efe480aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe50400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe480a9240 0x7efe480a96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe50400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe48145450 con 0x7efe300039e0 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe50400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe4806f020 con 0x7efe480a9240 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe50400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe481455d0 con 0x7efe480afa80 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe4815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe4815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39948/0 (socket says 172.21.15.165:39948) 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 -- 172.21.15.165:0/2146032145 learned_addr learned my addr 172.21.15.165:0/2146032145 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 -- 172.21.15.165:0/2146032145 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480afa80 msgr2=0x7efe480aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480afa80 0x7efe480aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 -- 172.21.15.165:0/2146032145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe480a9240 msgr2=0x7efe480a96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe480a9240 0x7efe480a96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 -- 172.21.15.165:0/2146032145 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe480aa880 con 0x7efe300039e0 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.955+0000 7efe4da00640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe4815b2c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7efe3c02a590 tx=0x7efe3c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.956+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c040020 con 0x7efe300039e0 2024-04-25T10:13:47.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.956+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c007680 con 0x7efe300039e0 2024-04-25T10:13:47.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.956+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c007a30 con 0x7efe300039e0 2024-04-25T10:13:47.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.956+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe480a9fe0 con 0x7efe300039e0 2024-04-25T10:13:47.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.956+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe480aa5a0 con 0x7efe300039e0 2024-04-25T10:13:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.957+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe48073990 con 0x7efe300039e0 2024-04-25T10:13:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.958+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe3c043020 con 0x7efe300039e0 2024-04-25T10:13:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.959+0000 7efe45600640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe2806b500 0x7efe2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.959+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7efe3c0aa2e0 con 0x7efe300039e0 2024-04-25T10:13:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.959+0000 7efe4d000640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe2806b500 0x7efe2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:47.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.960+0000 7efe4d000640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe2806b500 0x7efe2806d9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efe34006ce0 tx=0x7efe34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:47.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:47.963+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe3c04a020 con 0x7efe300039e0 2024-04-25T10:13:48.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.201+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efe480d0510 con 0x7efe300039e0 2024-04-25T10:13:48.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.202+0000 7efe45600640 1 -- 172.21.15.165:0/2146032145 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+3711 (secure 0 0 0) 0x7efe3c0793e0 con 0x7efe300039e0 2024-04-25T10:13:48.207 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:48.207 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":76,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5938,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1850810266","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1850810266},{"type":"v1","addr":"172.21.15.165:6837","nonce":1850810266}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69},{"gid":6091,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1904396914","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1904396914},{"type":"v1","addr":"172.21.15.172:6835","nonce":1904396914}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":76},{"gid":8031,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3417013964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3417013964},{"type":"v1","addr":"172.21.15.165:6836","nonce":3417013964}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":68},{"gid":8040,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1658114913","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1658114913},{"type":"v1","addr":"172.21.15.172:6834","nonce":1658114913}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":69}],"filesystems":[]} 2024-04-25T10:13:48.207 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 76 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe2806b500 msgr2=0x7efe2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe2806b500 0x7efe2806d9c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7efe34006ce0 tx=0x7efe34008040 comp rx=0 tx=0).stop 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 msgr2=0x7efe4815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe4815b2c0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7efe3c02a590 tx=0x7efe3c001120 comp rx=0 tx=0).stop 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 shutdown_connections 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe2806b500 0x7efe2806d9c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe480a9240 0x7efe480a96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe480afa80 0x7efe480aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 --2- 172.21.15.165:0/2146032145 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe300039e0 0x7efe4815b2c0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.205+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 >> 172.21.15.165:0/2146032145 conn(0x7efe4806abe0 msgr2=0x7efe4806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.206+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 shutdown_connections 2024-04-25T10:13:48.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.206+0000 7efe50400640 1 -- 172.21.15.165:0/2146032145 wait complete. 2024-04-25T10:13:48.233 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:13:48.233 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:13:48.233 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:13:48.233 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:48.234 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:13:48.234 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:13:48.234 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:13:48.234 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:48.234 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:13:48.234 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:13:48.235 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:13:48.235 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:48.235 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:13:48.235 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:13:48.235 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:13:48.235 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:13:48.254 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:13:48.255 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:13:48.256 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:13:48.256 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:13:48.258 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:13:48.258 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:13:48.259 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:13:48.259 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:13:48.303 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:13:48.304 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:48.305 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:13:48.306 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:48.310 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:13:48.310 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:48.311 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:13:48.311 DEBUG:teuthology.parallel:result is None 2024-04-25T10:13:48.311 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:13:48.453 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:13:48.453 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:13:48.460 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:13:48.461 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:13:48.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.482+0000 7fd59e000640 1 Processor -- start 2024-04-25T10:13:48.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.483+0000 7fd59e000640 1 -- start start 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd59e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd59815b7d0 0x7fd5980cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd59e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5980b5620 0x7fd5980b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd59e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd59806cf50 con 0x7fd59806c2d0 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd59e000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980ce530 con 0x7fd59815b7d0 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd59e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5980ce810 con 0x7fd5980b5620 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd597400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd59815b7d0 0x7fd5980cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd597400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd59815b7d0 0x7fd5980cf010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39974/0 (socket says 172.21.15.165:39974) 2024-04-25T10:13:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd597400640 1 -- 172.21.15.165:0/3838878829 learned_addr learned my addr 172.21.15.165:0/3838878829 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.484+0000 7fd58fe00640 1 --2- 172.21.15.165:0/3838878829 >> v2:172.21.15.172:3300/0 conn(0x7fd5980b5620 0x7fd5980b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.485+0000 7fd597400640 1 -- 172.21.15.165:0/3838878829 >> v2:172.21.15.172:3300/0 conn(0x7fd5980b5620 msgr2=0x7fd5980b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.485+0000 7fd597400640 1 --2- 172.21.15.165:0/3838878829 >> v2:172.21.15.172:3300/0 conn(0x7fd5980b5620 0x7fd5980b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.485+0000 7fd597400640 1 -- 172.21.15.165:0/3838878829 >> v1:172.21.15.172:6789/0 conn(0x7fd59806c2d0 legacy=0x7fd59806c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.485+0000 7fd597400640 1 -- 172.21.15.165:0/3838878829 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5980ceaf0 con 0x7fd59815b7d0 2024-04-25T10:13:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.485+0000 7fd58fe00640 1 --2- 172.21.15.165:0/3838878829 >> v2:172.21.15.172:3300/0 conn(0x7fd5980b5620 0x7fd5980b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:13:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.485+0000 7fd596a00640 1 -- 172.21.15.165:0/3838878829 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3023842239 0 0) 0x7fd59806cf50 con 0x7fd59806c2d0 2024-04-25T10:13:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.486+0000 7fd597400640 1 --2- 172.21.15.165:0/3838878829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980cf010 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd5980ce180 tx=0x7fd58802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d89f00c9fb877e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.486+0000 7fd596a00640 1 -- 172.21.15.165:0/3838878829 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588037070 con 0x7fd59815b7d0 2024-04-25T10:13:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.486+0000 7fd596a00640 1 -- 172.21.15.165:0/3838878829 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd588032440 con 0x7fd59815b7d0 2024-04-25T10:13:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.486+0000 7fd596a00640 1 -- 172.21.15.165:0/3838878829 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd588032810 con 0x7fd59815b7d0 2024-04-25T10:13:48.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 -- 172.21.15.165:0/3838878829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 msgr2=0x7fd5980cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 --2- 172.21.15.165:0/3838878829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980cf010 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7fd5980ce180 tx=0x7fd58802a270 comp rx=0 tx=0).stop 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 -- 172.21.15.165:0/3838878829 shutdown_connections 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 --2- 172.21.15.165:0/3838878829 >> v2:172.21.15.172:3300/0 conn(0x7fd5980b5620 0x7fd5980b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 --2- 172.21.15.165:0/3838878829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980cf010 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 -- 172.21.15.165:0/3838878829 >> 172.21.15.165:0/3838878829 conn(0x7fd5980d0800 msgr2=0x7fd5980d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 -- 172.21.15.165:0/3838878829 shutdown_connections 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.487+0000 7fd59e000640 1 -- 172.21.15.165:0/3838878829 wait complete. 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.488+0000 7fd59e000640 1 Processor -- start 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.489+0000 7fd59e000640 1 -- start start 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.489+0000 7fd59e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd59806c2d0 0x7fd598159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd59e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5980b5620 0x7fd5980cbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd59e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd59e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980cf6d0 con 0x7fd59815b7d0 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd59e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5980cf550 con 0x7fd59806c2d0 2024-04-25T10:13:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd59e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd59815bbb0 con 0x7fd5980b5620 2024-04-25T10:13:48.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980b3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39990/0 (socket says 172.21.15.165:39990) 2024-04-25T10:13:48.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 -- 172.21.15.165:0/402483327 learned_addr learned my addr 172.21.15.165:0/402483327 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:48.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597400640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd59806c2d0 0x7fd598159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd58fe00640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5980b5620 0x7fd5980cbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 -- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5980b5620 msgr2=0x7fd5980cbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5980b5620 0x7fd5980cbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 -- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd59806c2d0 msgr2=0x7fd598159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd59806c2d0 0x7fd598159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.490+0000 7fd597e00640 1 -- 172.21.15.165:0/402483327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5981f7c40 con 0x7fd59815b7d0 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.491+0000 7fd597e00640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980b3500 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c700 tx=0x7fd59000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.491+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd590017070 con 0x7fd59815b7d0 2024-04-25T10:13:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.491+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd590003dc0 con 0x7fd59815b7d0 2024-04-25T10:13:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.491+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5981f3710 con 0x7fd59815b7d0 2024-04-25T10:13:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.491+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5900052b0 con 0x7fd59815b7d0 2024-04-25T10:13:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.491+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5981f73a0 con 0x7fd59815b7d0 2024-04-25T10:13:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.492+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd598073a00 con 0x7fd59815b7d0 2024-04-25T10:13:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.493+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd59001a070 con 0x7fd59815b7d0 2024-04-25T10:13:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.494+0000 7fd58f400640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd57406b520 0x7fd57406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.494+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd590089560 con 0x7fd59815b7d0 2024-04-25T10:13:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.494+0000 7fd597400640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd57406b520 0x7fd57406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.495+0000 7fd597400640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd57406b520 0x7fd57406d9e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd588034000 tx=0x7fd58802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:48.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.499+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd590057fe0 con 0x7fd59815b7d0 2024-04-25T10:13:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.709+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd5980b9010 con 0x7fd59815b7d0 2024-04-25T10:13:48.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.872+0000 7fd58f400640 1 -- 172.21.15.165:0/402483327 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v118) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd59005c080 con 0x7fd59815b7d0 2024-04-25T10:13:48.876 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:13:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd57406b520 msgr2=0x7fd57406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd57406b520 0x7fd57406d9e0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fd588034000 tx=0x7fd58802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 msgr2=0x7fd5980b3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980b3500 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fd59000c700 tx=0x7fd59000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 shutdown_connections 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd57406b520 0x7fd57406d9e0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd59815b7d0 0x7fd5980b3500 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5980b5620 0x7fd5980cbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 --2- 172.21.15.165:0/402483327 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd59806c2d0 0x7fd598159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.876+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 >> 172.21.15.165:0/402483327 conn(0x7fd5980d0800 msgr2=0x7fd5981609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.877+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 shutdown_connections 2024-04-25T10:13:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:48.877+0000 7fd59e000640 1 -- 172.21.15.165:0/402483327 wait complete. 2024-04-25T10:13:48.905 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:13:49.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.098+0000 7f4e61200640 1 Processor -- start 2024-04-25T10:13:49.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.098+0000 7f4e61200640 1 -- start start 2024-04-25T10:13:49.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.100+0000 7f4e61200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 0x7f4e5c06dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.100+0000 7f4e61200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c072560 con 0x7f4e5c0b3960 2024-04-25T10:13:49.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.100+0000 7f4e61200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5c1625c0 con 0x7f4e5c06e020 2024-04-25T10:13:49.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.100+0000 7f4e61200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c06ec10 con 0x7f4e5c072180 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e5a800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4e5c0b3960 0x7f4e5c163040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58402/0 (socket says 172.21.15.165:58402) 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e5a800640 1 -- 172.21.15.165:0/3201630717 learned_addr learned my addr 172.21.15.165:0/3201630717 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e59e00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 0x7f4e5c06dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132232466 0 0) 0x7f4e5c072560 con 0x7f4e5c0b3960 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e3c003520 con 0x7f4e5c0b3960 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2017364019 0 0) 0x7f4e3c003520 con 0x7f4e5c0b3960 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 msgr2=0x7f4e5c06dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 0x7f4e5c06dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 >> v1:172.21.15.172:6789/0 conn(0x7f4e5c06e020 legacy=0x7f4e5c0b3250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c071590 con 0x7f4e5c0b3960 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4e54001670 con 0x7f4e5c0b3960 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 >> v1:172.21.15.165:6789/0 conn(0x7f4e5c0b3960 legacy=0x7f4e5c163040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.101+0000 7f4e58a00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e3c0039e0 0x7f4e3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e58a00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 0x7f4e3c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e58a00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e3c004d40 0x7f4e3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c06ec10 con 0x7f4e3c0039e0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c00ba70 con 0x7f4e3c004d40 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c00bc70 con 0x7f4e5c072180 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e59e00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e3c0039e0 0x7f4e3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e5a800640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e3c004d40 0x7f4e3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e59e00640 1 -- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 msgr2=0x7f4e3c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e59e00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e5c072180 0x7f4e3c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e59e00640 1 -- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e3c004d40 msgr2=0x7f4e3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e59e00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e3c004d40 0x7f4e3c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.102+0000 7f4e59e00640 1 -- 172.21.15.165:0/3201630717 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c003520 con 0x7f4e3c0039e0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.103+0000 7f4e59e00640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e3c0039e0 0x7f4e3c003e00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009a90 tx=0x7f4e3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476e20e400ab7dc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c036070 con 0x7f4e3c0039e0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e4c031400 con 0x7f4e3c0039e0 2024-04-25T10:13:49.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e58a00640 1 -- 172.21.15.165:0/3201630717 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e4c009690 con 0x7f4e3c0039e0 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e61200640 1 -- 172.21.15.165:0/3201630717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e3c0039e0 msgr2=0x7f4e3c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e61200640 1 --2- 172.21.15.165:0/3201630717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e3c0039e0 0x7f4e3c003e00 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c009a90 tx=0x7f4e3c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e5a800640 1 -- 172.21.15.165:0/3201630717 reap_dead start 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e61200640 1 -- 172.21.15.165:0/3201630717 shutdown_connections 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e61200640 1 -- 172.21.15.165:0/3201630717 >> 172.21.15.165:0/3201630717 conn(0x7f4e5c06abe0 msgr2=0x7f4e5c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e61200640 1 -- 172.21.15.165:0/3201630717 shutdown_connections 2024-04-25T10:13:49.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.104+0000 7f4e61200640 1 -- 172.21.15.165:0/3201630717 wait complete. 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.106+0000 7f4e61200640 1 Processor -- start 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.106+0000 7f4e61200640 1 -- start start 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e61200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e3c0039e0 0x7f4e5c155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e61200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e5c155b80 0x7f4e5c0ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e61200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 0x7f4e5c150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e61200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c162840 con 0x7f4e5c0aced0 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e61200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c06e930 con 0x7f4e5c155b80 2024-04-25T10:13:49.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e61200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5c162b20 con 0x7f4e3c0039e0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e5a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 0x7f4e5c150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e5a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 0x7f4e5c150820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40006/0 (socket says 172.21.15.165:40006) 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.107+0000 7f4e5a800640 1 -- 172.21.15.165:0/2483869144 learned_addr learned my addr 172.21.15.165:0/2483869144 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e59e00640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e3c0039e0 0x7f4e5c155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e59400640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e5c155b80 0x7f4e5c0ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e5a800640 1 -- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e3c0039e0 msgr2=0x7f4e5c155640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e5a800640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e3c0039e0 0x7f4e5c155640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e5a800640 1 -- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e5c155b80 msgr2=0x7f4e5c0ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e5a800640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e5c155b80 0x7f4e5c0ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e5a800640 1 -- 172.21.15.165:0/2483869144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5c1515a0 con 0x7f4e5c0aced0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e5a800640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 0x7f4e5c150820 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f4e54000f60 tx=0x7f4e5400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e540072d0 con 0x7f4e5c0aced0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e54007490 con 0x7f4e5c0aced0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e54004a20 con 0x7f4e5c0aced0 2024-04-25T10:13:49.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.108+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5c152d60 con 0x7f4e5c0aced0 2024-04-25T10:13:49.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.109+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5c153320 con 0x7f4e5c0aced0 2024-04-25T10:13:49.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.110+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5c073a00 con 0x7f4e5c0aced0 2024-04-25T10:13:49.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.111+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e5401b020 con 0x7f4e5c0aced0 2024-04-25T10:13:49.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.112+0000 7f4e52000640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e3806b4d0 0x7f4e3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.112+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4e54089c50 con 0x7f4e5c0aced0 2024-04-25T10:13:49.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.113+0000 7f4e59e00640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e3806b4d0 0x7f4e3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.113+0000 7f4e59e00640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e3806b4d0 0x7f4e3806d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c033000 tx=0x7f4e4c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.117+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e5401b310 con 0x7f4e5c0aced0 2024-04-25T10:13:49.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.386+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4e5c0cb950 con 0x7f4e5c0aced0 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.387+0000 7f4e52000640 1 -- 172.21.15.165:0/2483869144 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4e54058c80 con 0x7f4e5c0aced0 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:13:49.391 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.392 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:13:49.393 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.394 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.395 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:13:49.396 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.397 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.398 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:13:49.399 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:49.400 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.401 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:13:49.402 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.403 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:13:49.404 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:13:49.405 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:13:49.405 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:13:49.405 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.405 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:49.405 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:13:49.405 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e3806b4d0 msgr2=0x7f4e3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e3806b4d0 0x7f4e3806d990 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f4e4c033000 tx=0x7f4e4c008030 comp rx=0 tx=0).stop 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 msgr2=0x7f4e5c150820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 0x7f4e5c150820 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f4e54000f60 tx=0x7f4e5400dc00 comp rx=0 tx=0).stop 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 shutdown_connections 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e3806b4d0 0x7f4e3806d990 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e5c0aced0 0x7f4e5c150820 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e5c155b80 0x7f4e5c0ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 --2- 172.21.15.165:0/2483869144 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e3c0039e0 0x7f4e5c155640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 >> 172.21.15.165:0/2483869144 conn(0x7f4e5c06abe0 msgr2=0x7f4e5c0b1780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 shutdown_connections 2024-04-25T10:13:49.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.390+0000 7f4e61200640 1 -- 172.21.15.165:0/2483869144 wait complete. 2024-04-25T10:13:49.412 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:49.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.606+0000 7f939f800640 1 Processor -- start 2024-04-25T10:13:49.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.607+0000 7f939f800640 1 -- start start 2024-04-25T10:13:49.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.608+0000 7f939f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.608+0000 7f939f800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f93980b92a0 0x7f93980d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.608+0000 7f939f800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980b97a0 con 0x7f93980b2cb0 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.608+0000 7f939f800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f939815dfc0 con 0x7f93980b92a0 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.608+0000 7f939f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93980d1a00 con 0x7f939815dbe0 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f93980b92a0 0x7f93980d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f93980b92a0 0x7f93980d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40020/0 (socket says 172.21.15.165:40020) 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 -- 172.21.15.165:0/2741862578 learned_addr learned my addr 172.21.15.165:0/2741862578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:49.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f938fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f93980b2cb0 0x7f93980b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58416/0 (socket says 172.21.15.165:58416) 2024-04-25T10:13:49.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 -- 172.21.15.165:0/2741862578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 msgr2=0x7f93980d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:49.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939ce00640 1 --2- 172.21.15.165:0/2741862578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 --2- 172.21.15.165:0/2741862578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 -- 172.21.15.165:0/2741862578 >> v1:172.21.15.165:6789/0 conn(0x7f93980b2cb0 legacy=0x7f93980b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.609+0000 7f939d800640 1 -- 172.21.15.165:0/2741862578 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980d1c50 con 0x7f93980b92a0 2024-04-25T10:13:49.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.610+0000 7f939d800640 1 --2- 172.21.15.165:0/2741862578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b92a0 0x7f93980d0e50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f9390009e50 tx=0x7f939002a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3661d2e3c7325cfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.610+0000 7f9397e00640 1 -- 172.21.15.165:0/2741862578 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9390036070 con 0x7f93980b92a0 2024-04-25T10:13:49.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.610+0000 7f9397e00640 1 -- 172.21.15.165:0/2741862578 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9390031600 con 0x7f93980b92a0 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.610+0000 7f9397e00640 1 -- 172.21.15.165:0/2741862578 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93900319d0 con 0x7f93980b92a0 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 -- 172.21.15.165:0/2741862578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b92a0 msgr2=0x7f93980d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 --2- 172.21.15.165:0/2741862578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b92a0 0x7f93980d0e50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f9390009e50 tx=0x7f939002a710 comp rx=0 tx=0).stop 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 -- 172.21.15.165:0/2741862578 shutdown_connections 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 --2- 172.21.15.165:0/2741862578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b92a0 0x7f93980d0e50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 --2- 172.21.15.165:0/2741862578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 -- 172.21.15.165:0/2741862578 >> 172.21.15.165:0/2741862578 conn(0x7f9398061a20 msgr2=0x7f9398061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 -- 172.21.15.165:0/2741862578 shutdown_connections 2024-04-25T10:13:49.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.611+0000 7f939f800640 1 -- 172.21.15.165:0/2741862578 wait complete. 2024-04-25T10:13:49.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.613+0000 7f939f800640 1 Processor -- start 2024-04-25T10:13:49.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.613+0000 7f939f800640 1 -- start start 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939f800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 0x7f93980cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939f800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93980b92a0 0x7f9398149d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939f800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939f800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980b6ab0 con 0x7f93980b2cb0 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939f800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980d2210 con 0x7f93980b92a0 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939f800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93980b6c30 con 0x7f939815dbe0 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 0x7f93980cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 0x7f93980cc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40032/0 (socket says 172.21.15.165:40032) 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.614+0000 7f939ce00640 1 -- 172.21.15.165:0/2904548693 learned_addr learned my addr 172.21.15.165:0/2904548693 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939d800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939ce00640 1 -- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 msgr2=0x7f93980cacc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939ce00640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f938fe00640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93980b92a0 0x7f9398149d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939ce00640 1 -- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93980b92a0 msgr2=0x7f9398149d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939ce00640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93980b92a0 0x7f9398149d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939ce00640 1 -- 172.21.15.165:0/2904548693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980cd0b0 con 0x7f93980b2cb0 2024-04-25T10:13:49.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f939ce00640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 0x7f93980cc9d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f938800c7f0 tx=0x7f938800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388015070 con 0x7f93980b2cb0 2024-04-25T10:13:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.615+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9388004220 con 0x7f93980b2cb0 2024-04-25T10:13:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.616+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93980cd390 con 0x7f93980b2cb0 2024-04-25T10:13:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.616+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9388007620 con 0x7f93980b2cb0 2024-04-25T10:13:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.616+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93980cd950 con 0x7f93980b2cb0 2024-04-25T10:13:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.617+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93980d3640 con 0x7f93980b2cb0 2024-04-25T10:13:49.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.618+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f93880079d0 con 0x7f93980b2cb0 2024-04-25T10:13:49.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.619+0000 7f9395600640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f937406b4d0 0x7f937406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:49.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.619+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f938808a4a0 con 0x7f93980b2cb0 2024-04-25T10:13:49.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.619+0000 7f938fe00640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f937406b4d0 0x7f937406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:49.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.620+0000 7f938fe00640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f937406b4d0 0x7f937406d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f93980b64a0 tx=0x7f9380009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:49.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.623+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f93980d3640 con 0x7f93980b2cb0 2024-04-25T10:13:49.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.816+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9398073990 con 0x7f93980b2cb0 2024-04-25T10:13:49.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.819+0000 7f9395600640 1 -- 172.21.15.165:0/2904548693 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) v1 ==== 74+0+13210 (secure 0 0 0) 0x7f9388059440 con 0x7f93980b2cb0 2024-04-25T10:13:49.823 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:49.823 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":120,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:48.885163+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:49.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f937406b4d0 msgr2=0x7f937406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f937406b4d0 0x7f937406d990 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f93980b64a0 tx=0x7f9380009040 comp rx=0 tx=0).stop 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 msgr2=0x7f93980cc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 0x7f93980cc9d0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f938800c7f0 tx=0x7f938800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 shutdown_connections 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f937406b4d0 0x7f937406d990 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f939815dbe0 0x7f93980cacc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f93980b92a0 0x7f9398149d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 --2- 172.21.15.165:0/2904548693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f93980b2cb0 0x7f93980cc9d0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.821+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 >> 172.21.15.165:0/2904548693 conn(0x7f9398061a20 msgr2=0x7f93980ce200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.822+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 shutdown_connections 2024-04-25T10:13:49.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:49.822+0000 7f939f800640 1 -- 172.21.15.165:0/2904548693 wait complete. 2024-04-25T10:13:49.844 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:13:49.844 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:13:50.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.022+0000 7fbdaf600640 1 Processor -- start 2024-04-25T10:13:50.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.022+0000 7fbdaf600640 1 -- start start 2024-04-25T10:13:50.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdaf600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbda81446d0 0x7fbda80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdaf600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbda806e8a0 0x7fbda80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdaf600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbda80cccc0 con 0x7fbda8144d50 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdaf600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda80b2b20 con 0x7fbda806e8a0 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdaf600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbda806d400 con 0x7fbda81446d0 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdad600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbda806e8a0 0x7fbda80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdad600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbda806e8a0 0x7fbda80b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40036/0 (socket says 172.21.15.165:40036) 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbd9fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbda8144d50 0x7fbda80cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58418/0 (socket says 172.21.15.165:58418) 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.024+0000 7fbdad600640 1 -- 172.21.15.165:0/3437801872 learned_addr learned my addr 172.21.15.165:0/3437801872 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.025+0000 7fbdacc00640 1 --2- 172.21.15.165:0/3437801872 >> v2:172.21.15.172:3300/0 conn(0x7fbda81446d0 0x7fbda80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.025+0000 7fbdad600640 1 -- 172.21.15.165:0/3437801872 >> v2:172.21.15.172:3300/0 conn(0x7fbda81446d0 msgr2=0x7fbda80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.025+0000 7fbdad600640 1 --2- 172.21.15.165:0/3437801872 >> v2:172.21.15.172:3300/0 conn(0x7fbda81446d0 0x7fbda80b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.025+0000 7fbdad600640 1 -- 172.21.15.165:0/3437801872 >> v1:172.21.15.165:6789/0 conn(0x7fbda8144d50 legacy=0x7fbda80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.025+0000 7fbdad600640 1 -- 172.21.15.165:0/3437801872 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80cc140 con 0x7fbda806e8a0 2024-04-25T10:13:50.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.025+0000 7fbda7e00640 1 -- 172.21.15.165:0/3437801872 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661551076 0 0) 0x7fbda80cccc0 con 0x7fbda8144d50 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.026+0000 7fbdad600640 1 --2- 172.21.15.165:0/3437801872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda80b4460 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbda80cf560 tx=0x7fbda002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ee5724397cc9779 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.026+0000 7fbda7e00640 1 -- 172.21.15.165:0/3437801872 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda0036070 con 0x7fbda806e8a0 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.026+0000 7fbda7e00640 1 -- 172.21.15.165:0/3437801872 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbda0031600 con 0x7fbda806e8a0 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.026+0000 7fbda7e00640 1 -- 172.21.15.165:0/3437801872 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbda00319d0 con 0x7fbda806e8a0 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 -- 172.21.15.165:0/3437801872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 msgr2=0x7fbda80b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 --2- 172.21.15.165:0/3437801872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda80b4460 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7fbda80cf560 tx=0x7fbda002a730 comp rx=0 tx=0).stop 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 -- 172.21.15.165:0/3437801872 shutdown_connections 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 --2- 172.21.15.165:0/3437801872 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda80b4460 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 --2- 172.21.15.165:0/3437801872 >> v2:172.21.15.172:3300/0 conn(0x7fbda81446d0 0x7fbda80b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 -- 172.21.15.165:0/3437801872 >> 172.21.15.165:0/3437801872 conn(0x7fbda806abe0 msgr2=0x7fbda806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 -- 172.21.15.165:0/3437801872 shutdown_connections 2024-04-25T10:13:50.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.027+0000 7fbdaf600640 1 -- 172.21.15.165:0/3437801872 wait complete. 2024-04-25T10:13:50.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.029+0000 7fbdaf600640 1 Processor -- start 2024-04-25T10:13:50.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.029+0000 7fbdaf600640 1 -- start start 2024-04-25T10:13:50.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdaf600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda8156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdaf600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda81446d0 0x7fbda80cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdaf600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbda8144d50 0x7fbda81f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdaf600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80cc700 con 0x7fbda806e8a0 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdaf600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80b49a0 con 0x7fbda81446d0 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdaf600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbda80cc9e0 con 0x7fbda8144d50 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda8156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda8156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40040/0 (socket says 172.21.15.165:40040) 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 -- 172.21.15.165:0/194144991 learned_addr learned my addr 172.21.15.165:0/194144991 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbd9fe00640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda81446d0 0x7fbda80cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 -- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbda8144d50 msgr2=0x7fbda81f6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbda8144d50 0x7fbda81f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 -- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda81446d0 msgr2=0x7fbda80cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda81446d0 0x7fbda80cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.030+0000 7fbdacc00640 1 -- 172.21.15.165:0/194144991 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbda80ce7c0 con 0x7fbda806e8a0 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.031+0000 7fbdacc00640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda8156d30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800d830 tx=0x7fbd9800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.031+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800b6c0 con 0x7fbda806e8a0 2024-04-25T10:13:50.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.031+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd98019070 con 0x7fbda806e8a0 2024-04-25T10:13:50.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.031+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbda80ce2d0 con 0x7fbda806e8a0 2024-04-25T10:13:50.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.031+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd9800ba80 con 0x7fbda806e8a0 2024-04-25T10:13:50.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.031+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbda80c5240 con 0x7fbda806e8a0 2024-04-25T10:13:50.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.032+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbda8073a00 con 0x7fbda806e8a0 2024-04-25T10:13:50.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.033+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbd9801e460 con 0x7fbda806e8a0 2024-04-25T10:13:50.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.034+0000 7fbda5600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd8406b520 0x7fbd8406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.034+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fbd9808a6a0 con 0x7fbda806e8a0 2024-04-25T10:13:50.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.034+0000 7fbd9fe00640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd8406b520 0x7fbd8406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.035+0000 7fbd9fe00640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd8406b520 0x7fbd8406d9e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbda80b2340 tx=0x7fbd90009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.038+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbd98059750 con 0x7fbda806e8a0 2024-04-25T10:13:50.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.232+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbda8140af0 con 0x7fbda806e8a0 2024-04-25T10:13:50.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.233+0000 7fbda5600640 1 -- 172.21.15.165:0/194144991 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v120) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbd98059050 con 0x7fbda806e8a0 2024-04-25T10:13:50.237 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:13:50.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.235+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd8406b520 msgr2=0x7fbd8406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.235+0000 7fbdaf600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd8406b520 0x7fbd8406d9e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fbda80b2340 tx=0x7fbd90009040 comp rx=0 tx=0).stop 2024-04-25T10:13:50.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.235+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 msgr2=0x7fbda8156d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda8156d30 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7fbd9800d830 tx=0x7fbd9800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 shutdown_connections 2024-04-25T10:13:50.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbd8406b520 0x7fbd8406d9e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbda8144d50 0x7fbda81f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbda81446d0 0x7fbda80cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 --2- 172.21.15.165:0/194144991 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbda806e8a0 0x7fbda8156d30 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 >> 172.21.15.165:0/194144991 conn(0x7fbda806abe0 msgr2=0x7fbda8141ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 shutdown_connections 2024-04-25T10:13:50.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.236+0000 7fbdaf600640 1 -- 172.21.15.165:0/194144991 wait complete. 2024-04-25T10:13:50.258 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:13:50.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.461+0000 7ff824400640 1 Processor -- start 2024-04-25T10:13:50.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.461+0000 7ff824400640 1 -- start start 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff824400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff81c0cc930 0x7ff81c06d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff824400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff81c0ce950 0x7ff81c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff824400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff81c15c150 con 0x7ff81c144d40 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff824400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c06d850 con 0x7ff81c0cc930 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff824400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff81c0cd4b0 con 0x7ff81c0ce950 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff81c0cc930 0x7ff81c06d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff81c0cc930 0x7ff81c06d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40052/0 (socket says 172.21.15.165:40052) 2024-04-25T10:13:50.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 -- 172.21.15.165:0/3729927768 learned_addr learned my addr 172.21.15.165:0/3729927768 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff822400640 1 --2- 172.21.15.165:0/3729927768 >> v2:172.21.15.172:3300/0 conn(0x7ff81c0ce950 0x7ff81c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 -- 172.21.15.165:0/3729927768 >> v2:172.21.15.172:3300/0 conn(0x7ff81c0ce950 msgr2=0x7ff81c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 --2- 172.21.15.165:0/3729927768 >> v2:172.21.15.172:3300/0 conn(0x7ff81c0ce950 0x7ff81c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 -- 172.21.15.165:0/3729927768 >> v1:172.21.15.172:6789/0 conn(0x7ff81c144d40 legacy=0x7ff81c15c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff821000640 1 -- 172.21.15.165:0/3729927768 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c0ccef0 con 0x7ff81c0cc930 2024-04-25T10:13:50.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.463+0000 7ff822400640 1 --2- 172.21.15.165:0/3729927768 >> v2:172.21.15.172:3300/0 conn(0x7ff81c0ce950 0x7ff81c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:50.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.465+0000 7ff821000640 1 --2- 172.21.15.165:0/3729927768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c0cc930 0x7ff81c06d310 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0ccdf0 tx=0x7ff80802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77e430d66cf5a4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.465+0000 7ff81be00640 1 -- 172.21.15.165:0/3729927768 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808037070 con 0x7ff81c0cc930 2024-04-25T10:13:50.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.465+0000 7ff81be00640 1 -- 172.21.15.165:0/3729927768 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80802ae20 con 0x7ff81c0cc930 2024-04-25T10:13:50.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.465+0000 7ff81be00640 1 -- 172.21.15.165:0/3729927768 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff808032660 con 0x7ff81c0cc930 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.465+0000 7ff824400640 1 -- 172.21.15.165:0/3729927768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c0cc930 msgr2=0x7ff81c06d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.465+0000 7ff824400640 1 --2- 172.21.15.165:0/3729927768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c0cc930 0x7ff81c06d310 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7ff81c0ccdf0 tx=0x7ff80802a110 comp rx=0 tx=0).stop 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.466+0000 7ff824400640 1 -- 172.21.15.165:0/3729927768 shutdown_connections 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.466+0000 7ff824400640 1 --2- 172.21.15.165:0/3729927768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c0cc930 0x7ff81c06d310 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.466+0000 7ff824400640 1 --2- 172.21.15.165:0/3729927768 >> v2:172.21.15.172:3300/0 conn(0x7ff81c0ce950 0x7ff81c0cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.466+0000 7ff824400640 1 -- 172.21.15.165:0/3729927768 >> 172.21.15.165:0/3729927768 conn(0x7ff81c06abe0 msgr2=0x7ff81c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.466+0000 7ff824400640 1 -- 172.21.15.165:0/3729927768 shutdown_connections 2024-04-25T10:13:50.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.466+0000 7ff824400640 1 -- 172.21.15.165:0/3729927768 wait complete. 2024-04-25T10:13:50.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.467+0000 7ff824400640 1 Processor -- start 2024-04-25T10:13:50.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.467+0000 7ff824400640 1 -- start start 2024-04-25T10:13:50.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.468+0000 7ff824400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff81c0cc930 0x7ff81c13c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.468+0000 7ff824400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff81c0ce950 0x7ff81c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.468+0000 7ff824400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 0x7ff81c0b07c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.468+0000 7ff824400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c0cf2f0 con 0x7ff81c144d40 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.468+0000 7ff824400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c15c3f0 con 0x7ff81c0ce950 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.468+0000 7ff824400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c06da60 con 0x7ff81c0cc930 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 0x7ff81c0b07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 0x7ff81c0b07c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40064/0 (socket says 172.21.15.165:40064) 2024-04-25T10:13:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 -- 172.21.15.165:0/3371036765 learned_addr learned my addr 172.21.15.165:0/3371036765 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:50.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff821000640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff81c0ce950 0x7ff81c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff821a00640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff81c0cc930 0x7ff81c13c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 -- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff81c0cc930 msgr2=0x7ff81c13c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff81c0cc930 0x7ff81c13c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 -- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff81c0ce950 msgr2=0x7ff81c0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff81c0ce950 0x7ff81c0a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 -- 172.21.15.165:0/3371036765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff81c13c620 con 0x7ff81c144d40 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff821a00640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff81c0cc930 0x7ff81c13c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff821000640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff81c0ce950 0x7ff81c0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.469+0000 7ff822400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 0x7ff81c0b07c0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7ff81400cf20 tx=0x7ff81400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.470+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81401a070 con 0x7ff81c144d40 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.470+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400d840 con 0x7ff81c144d40 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.470+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400dc30 con 0x7ff81c144d40 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.470+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c0a8f40 con 0x7ff81c144d40 2024-04-25T10:13:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.470+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c0a9500 con 0x7ff81c144d40 2024-04-25T10:13:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.471+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c073a00 con 0x7ff81c144d40 2024-04-25T10:13:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.472+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff81401d070 con 0x7ff81c144d40 2024-04-25T10:13:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.473+0000 7ff819600640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7fc06b500 0x7ff7fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.473+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff81408a770 con 0x7ff81c144d40 2024-04-25T10:13:50.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.473+0000 7ff821a00640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7fc06b500 0x7ff7fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.474+0000 7ff821a00640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7fc06b500 0x7ff7fc06d9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff81000a810 tx=0x7ff810008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.477+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff814059900 con 0x7ff81c144d40 2024-04-25T10:13:50.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.671+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7ff81c072240 con 0x7ff81c144d40 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.672+0000 7ff819600640 1 -- 172.21.15.165:0/3371036765 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v120) v1 ==== 84+0+65 (secure 0 0 0) 0x7ff814059200 con 0x7ff81c144d40 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:13:50.676 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.674+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7fc06b500 msgr2=0x7ff7fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.674+0000 7ff824400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7fc06b500 0x7ff7fc06d9c0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff81000a810 tx=0x7ff810008040 comp rx=0 tx=0).stop 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.674+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 msgr2=0x7ff81c0b07c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.674+0000 7ff824400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 0x7ff81c0b07c0 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7ff81400cf20 tx=0x7ff81400a7d0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 shutdown_connections 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7fc06b500 0x7ff7fc06d9c0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff81c144d40 0x7ff81c0b07c0 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff81c0ce950 0x7ff81c0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 --2- 172.21.15.165:0/3371036765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff81c0cc930 0x7ff81c13c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 >> 172.21.15.165:0/3371036765 conn(0x7ff81c06abe0 msgr2=0x7ff81c15c910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 shutdown_connections 2024-04-25T10:13:50.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.675+0000 7ff824400640 1 -- 172.21.15.165:0/3371036765 wait complete. 2024-04-25T10:13:50.697 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:13:50.698 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:13:50.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.892+0000 7f6910e00640 1 Processor -- start 2024-04-25T10:13:50.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.892+0000 7f6910e00640 1 -- start start 2024-04-25T10:13:50.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.893+0000 7f6910e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 0x7f690c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.893+0000 7f6910e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f690c0cdb20 0x7f690c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.893+0000 7f6910e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f690c072280 con 0x7f690c15b270 2024-04-25T10:13:50.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.893+0000 7f6910e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690c0ce4c0 con 0x7f690c0cdb20 2024-04-25T10:13:50.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.893+0000 7f6910e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c071270 con 0x7f690c0cc4f0 2024-04-25T10:13:50.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f690a800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f690c15b270 0x7f690c15b680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58426/0 (socket says 172.21.15.165:58426) 2024-04-25T10:13:50.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f690a800640 1 -- 172.21.15.165:0/3525801698 learned_addr learned my addr 172.21.15.165:0/3525801698 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6909400640 1 --2- 172.21.15.165:0/3525801698 >> v2:172.21.15.172:3300/0 conn(0x7f690c0cdb20 0x7f690c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281641762 0 0) 0x7f690c072280 con 0x7f690c15b270 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6909e00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 0x7f690c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f0003520 con 0x7f690c15b270 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2184963795 0 0) 0x7f68f0003520 con 0x7f690c15b270 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 msgr2=0x7f690c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 0x7f690c0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 >> v2:172.21.15.172:3300/0 conn(0x7f690c0cdb20 msgr2=0x7f690c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.894+0000 7f6908a00640 1 --2- 172.21.15.165:0/3525801698 >> v2:172.21.15.172:3300/0 conn(0x7f690c0cdb20 0x7f690c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c0714c0 con 0x7f690c15b270 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f69040019f0 con 0x7f690c15b270 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909400640 1 --2- 172.21.15.165:0/3525801698 >> v2:172.21.15.172:3300/0 conn(0x7f690c0cdb20 0x7f690c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 >> v1:172.21.15.165:6789/0 conn(0x7f690c15b270 legacy=0x7f690c15b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f68f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 0x7f68f0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c071270 con 0x7f68f00039b0 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000ba70 con 0x7f68f0004d40 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f000bbf0 con 0x7f690c0cc4f0 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909e00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f68f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f690a800640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909400640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 0x7f68f0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909e00640 1 -- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 msgr2=0x7f68f0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909e00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0cc4f0 0x7f68f0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909e00640 1 -- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68f0004d40 msgr2=0x7f68f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:50.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909e00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f6909e00640 1 -- 172.21.15.165:0/3525801698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f0003520 con 0x7f68f00039b0 2024-04-25T10:13:50.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.895+0000 7f690a800640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68f0004d40 0x7f68f00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:50.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.896+0000 7f6909e00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f68f0003dd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00ba60 tx=0x7f68f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf2231155f0395ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc030070 con 0x7f68f00039b0 2024-04-25T10:13:50.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc02ae10 con 0x7f68f00039b0 2024-04-25T10:13:50.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6908a00640 1 -- 172.21.15.165:0/3525801698 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00b600 con 0x7f68f00039b0 2024-04-25T10:13:50.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6910e00640 1 -- 172.21.15.165:0/3525801698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 msgr2=0x7f68f0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6910e00640 1 --2- 172.21.15.165:0/3525801698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f68f0003dd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f68fc00ba60 tx=0x7f68f0005830 comp rx=0 tx=0).stop 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f690a800640 1 -- 172.21.15.165:0/3525801698 reap_dead start 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6910e00640 1 -- 172.21.15.165:0/3525801698 shutdown_connections 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.897+0000 7f6910e00640 1 -- 172.21.15.165:0/3525801698 >> 172.21.15.165:0/3525801698 conn(0x7f690c06abe0 msgr2=0x7f690c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.898+0000 7f6910e00640 1 -- 172.21.15.165:0/3525801698 shutdown_connections 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.898+0000 7f6910e00640 1 -- 172.21.15.165:0/3525801698 wait complete. 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.899+0000 7f6910e00640 1 Processor -- start 2024-04-25T10:13:50.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.899+0000 7f6910e00640 1 -- start start 2024-04-25T10:13:50.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.900+0000 7f6910e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f690c144640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.900+0000 7f6910e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f690c1f39e0 0x7f690c0c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.900+0000 7f6910e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0c9230 0x7f690c149690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.900+0000 7f6910e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c071a80 con 0x7f68f00039b0 2024-04-25T10:13:50.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.900+0000 7f6910e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690c15bd90 con 0x7f690c1f39e0 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.900+0000 7f6910e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690c071d60 con 0x7f690c0c9230 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f690c144640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f690c144640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40092/0 (socket says 172.21.15.165:40092) 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 -- 172.21.15.165:0/4178619595 learned_addr learned my addr 172.21.15.165:0/4178619595 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 -- 172.21.15.165:0/4178619595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0c9230 msgr2=0x7f690c149690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0c9230 0x7f690c149690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 -- 172.21.15.165:0/4178619595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f690c1f39e0 msgr2=0x7f690c0c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f690c1f39e0 0x7f690c0c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 -- 172.21.15.165:0/4178619595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690c148910 con 0x7f68f00039b0 2024-04-25T10:13:50.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.901+0000 7f6909e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f690c144640 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f68fc02bfd0 tx=0x7f68fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.902+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc043020 con 0x7f68f00039b0 2024-04-25T10:13:50.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.902+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc03f680 con 0x7f68f00039b0 2024-04-25T10:13:50.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.902+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc00b600 con 0x7f68f00039b0 2024-04-25T10:13:50.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.902+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690c14fef0 con 0x7f68f00039b0 2024-04-25T10:13:50.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.902+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690c148e90 con 0x7f68f00039b0 2024-04-25T10:13:50.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.903+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690c0c9650 con 0x7f68f00039b0 2024-04-25T10:13:50.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.904+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f68fc046020 con 0x7f68f00039b0 2024-04-25T10:13:50.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.905+0000 7f6902000640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f68ec06b500 0x7f68ec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:50.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.905+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f68fc0aacf0 con 0x7f68f00039b0 2024-04-25T10:13:50.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.905+0000 7f6909400640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f68ec06b500 0x7f68ec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:50.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.906+0000 7f6909400640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f68ec06b500 0x7f68ec06d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f68f8006e80 tx=0x7f68f8006e10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:50.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:50.909+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68fc04d030 con 0x7f68f00039b0 2024-04-25T10:13:51.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.108+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f690c15bbb0 con 0x7f68f00039b0 2024-04-25T10:13:51.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.905+0000 7f6902000640 1 -- 172.21.15.165:0/4178619595 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v121) v1 ==== 138+0+0 (secure 0 0 0) 0x7f690c15bbb0 con 0x7f68f00039b0 2024-04-25T10:13:51.909 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.909+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f68ec06b500 msgr2=0x7f68ec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.909+0000 7f6910e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f68ec06b500 0x7f68ec06d9c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f68f8006e80 tx=0x7f68f8006e10 comp rx=0 tx=0).stop 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.909+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 msgr2=0x7f690c144640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.909+0000 7f6910e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f690c144640 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f68fc02bfd0 tx=0x7f68fc004730 comp rx=0 tx=0).stop 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 shutdown_connections 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f68ec06b500 0x7f68ec06d9c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f690c0c9230 0x7f690c149690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f690c1f39e0 0x7f690c0c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 --2- 172.21.15.165:0/4178619595 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68f00039b0 0x7f690c144640 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 >> 172.21.15.165:0/4178619595 conn(0x7f690c06abe0 msgr2=0x7f690c0b1c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 shutdown_connections 2024-04-25T10:13:51.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:51.910+0000 7f6910e00640 1 -- 172.21.15.165:0/4178619595 wait complete. 2024-04-25T10:13:51.940 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:13:52.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.131+0000 7f8bbdc00640 1 Processor -- start 2024-04-25T10:13:52.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.131+0000 7f8bbdc00640 1 -- start start 2024-04-25T10:13:52.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.133+0000 7f8bbdc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bb80b5b40 0x7f8bb8168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.133+0000 7f8bbdc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb815e110 con 0x7f8bb806bff0 2024-04-25T10:13:52.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.133+0000 7f8bbdc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb80b2fd0 con 0x7f8bb815dd30 2024-04-25T10:13:52.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.133+0000 7f8bbdc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb806d000 con 0x7f8bb80b5b40 2024-04-25T10:13:52.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.133+0000 7f8baf400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8bb806bff0 0x7f8bb80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58428/0 (socket says 172.21.15.165:58428) 2024-04-25T10:13:52.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.133+0000 7f8baf400640 1 -- 172.21.15.165:0/441519603 learned_addr learned my addr 172.21.15.165:0/441519603 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:52.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bbd200640 1 --2- 172.21.15.165:0/441519603 >> v2:172.21.15.172:3300/0 conn(0x7f8bb80b5b40 0x7f8bb8168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 955672055 0 0) 0x7f8bb815e110 con 0x7f8bb806bff0 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b9c003520 con 0x7f8bb806bff0 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3842732574 0 0) 0x7f8b9c003520 con 0x7f8bb806bff0 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 >> v2:172.21.15.172:3300/0 conn(0x7f8bb80b5b40 msgr2=0x7f8bb8168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 --2- 172.21.15.165:0/441519603 >> v2:172.21.15.172:3300/0 conn(0x7f8bb80b5b40 0x7f8bb8168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 >> v1:172.21.15.172:6789/0 conn(0x7f8bb815dd30 legacy=0x7f8bb80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb806c480 con 0x7f8bb806bff0 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8ba0001670 con 0x7f8bb806bff0 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 >> v1:172.21.15.165:6789/0 conn(0x7f8bb806bff0 legacy=0x7f8bb80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.134+0000 7f8bb7400640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b9c004800 0x7f8b9c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7400640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb806d000 con 0x7f8b9c0039b0 2024-04-25T10:13:52.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c00c060 con 0x7f8b9c0051c0 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b9c00c260 con 0x7f8b9c004800 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7e00640 1 -- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b9c004800 msgr2=0x7f8b9c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8baf400640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b9c004800 0x7f8b9c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bbd200640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b9c004800 0x7f8b9c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7e00640 1 -- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b9c0051c0 msgr2=0x7f8b9c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b9c0051c0 0x7f8b9c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bb7e00640 1 -- 172.21.15.165:0/441519603 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b9c003520 con 0x7f8b9c0039b0 2024-04-25T10:13:52.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.135+0000 7f8bbd200640 1 -- 172.21.15.165:0/441519603 reap_dead start 2024-04-25T10:13:52.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.136+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001750 tx=0x7f8b9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260f2528c065218b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8001370 con 0x7f8b9c0039b0 2024-04-25T10:13:52.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba80052d0 con 0x7f8b9c0039b0 2024-04-25T10:13:52.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bb7400640 1 -- 172.21.15.165:0/441519603 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba802abc0 con 0x7f8b9c0039b0 2024-04-25T10:13:52.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bbdc00640 1 -- 172.21.15.165:0/441519603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9c0039b0 msgr2=0x7f8b9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8001750 tx=0x7f8b9c005830 comp rx=0 tx=0).stop 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bbdc00640 1 -- 172.21.15.165:0/441519603 shutdown_connections 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/441519603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b9c0039b0 0x7f8b9c003dd0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.137+0000 7f8bbdc00640 1 -- 172.21.15.165:0/441519603 >> 172.21.15.165:0/441519603 conn(0x7f8bb80d0610 msgr2=0x7f8bb80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.138+0000 7f8bbdc00640 1 -- 172.21.15.165:0/441519603 shutdown_connections 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.138+0000 7f8bbdc00640 1 -- 172.21.15.165:0/441519603 wait complete. 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.139+0000 7f8bbdc00640 1 Processor -- start 2024-04-25T10:13:52.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.139+0000 7f8bbdc00640 1 -- start start 2024-04-25T10:13:52.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.140+0000 7f8bbdc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb8157f90 0x7f8bb815f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.140+0000 7f8bbdc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bb8160a80 0x7f8bb80b8880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.140+0000 7f8bbdc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 0x7f8bb80ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.140+0000 7f8bbdc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb806cd20 con 0x7f8bb8158370 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.140+0000 7f8bbdc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb806ca40 con 0x7f8bb8157f90 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.140+0000 7f8bbdc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb80b5430 con 0x7f8bb8160a80 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 0x7f8bb80ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 0x7f8bb80ccd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40120/0 (socket says 172.21.15.165:40120) 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 -- 172.21.15.165:0/2891814864 learned_addr learned my addr 172.21.15.165:0/2891814864 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8baf400640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bb8160a80 0x7f8bb80b8880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb8157f90 0x7f8bb815f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 -- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bb8160a80 msgr2=0x7f8bb80b8880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bb8160a80 0x7f8bb80b8880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 -- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb8157f90 msgr2=0x7f8bb815f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb8157f90 0x7f8bb815f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bbd200640 1 -- 172.21.15.165:0/2891814864 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb815f9e0 con 0x7f8bb8158370 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.141+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb8157f90 0x7f8bb815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.142+0000 7f8bbd200640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 0x7f8bb80ccd70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8bb808c1f0 tx=0x7f8bb000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.142+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0016420 con 0x7f8bb8158370 2024-04-25T10:13:52.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.142+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb001b070 con 0x7f8bb8158370 2024-04-25T10:13:52.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.142+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0016820 con 0x7f8bb8158370 2024-04-25T10:13:52.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.142+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb815fcc0 con 0x7f8bb8158370 2024-04-25T10:13:52.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.142+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb8160280 con 0x7f8bb8158370 2024-04-25T10:13:52.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.143+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb8073990 con 0x7f8bb8158370 2024-04-25T10:13:52.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.145+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bb0005d10 con 0x7f8bb8158370 2024-04-25T10:13:52.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.145+0000 7f8bb4c00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b9406b4d0 0x7f8b9406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:52.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.146+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f8bb008ada0 con 0x7f8bb8158370 2024-04-25T10:13:52.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.146+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b9406b4d0 0x7f8b9406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:52.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.147+0000 7f8bb7e00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b9406b4d0 0x7f8b9406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8032000 tx=0x7f8ba802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:52.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.151+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bb0025020 con 0x7f8bb8158370 2024-04-25T10:13:52.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.350+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8bb80d16c0 con 0x7f8bb8158370 2024-04-25T10:13:52.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.920+0000 7f8bb4c00640 1 -- 172.21.15.165:0/2891814864 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v122) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8bb0059bd0 con 0x7f8bb8158370 2024-04-25T10:13:52.924 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:13:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.924+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b9406b4d0 msgr2=0x7f8b9406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.924+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b9406b4d0 0x7f8b9406d990 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8032000 tx=0x7f8ba802e000 comp rx=0 tx=0).stop 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.924+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 msgr2=0x7f8bb80ccd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.924+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 0x7f8bb80ccd70 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7f8bb808c1f0 tx=0x7f8bb000f520 comp rx=0 tx=0).stop 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 shutdown_connections 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b9406b4d0 0x7f8b9406d990 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bb8158370 0x7f8bb80ccd70 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bb8160a80 0x7f8bb80b8880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 --2- 172.21.15.165:0/2891814864 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bb8157f90 0x7f8bb815f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:52.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 >> 172.21.15.165:0/2891814864 conn(0x7f8bb80d0610 msgr2=0x7f8bb80b6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 shutdown_connections 2024-04-25T10:13:52.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:52.925+0000 7f8bbdc00640 1 -- 172.21.15.165:0/2891814864 wait complete. 2024-04-25T10:13:52.963 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:13:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.137+0000 7fad22200640 1 Processor -- start 2024-04-25T10:13:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.137+0000 7fad22200640 1 -- start start 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad22200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 0x7fad1c06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad22200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fad1c1452a0 0x7fad1c06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad22200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad1c0cccc0 con 0x7fad1c1446d0 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad22200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad1c0b2b20 con 0x7fad1c1452a0 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad22200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c06d400 con 0x7fad1c144d50 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad1b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fad1c1446d0 0x7fad1c0b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58430/0 (socket says 172.21.15.165:58430) 2024-04-25T10:13:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.139+0000 7fad1b400640 1 -- 172.21.15.165:0/254273352 learned_addr learned my addr 172.21.15.165:0/254273352 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1be00640 1 --2- 172.21.15.165:0/254273352 >> v2:172.21.15.172:3300/0 conn(0x7fad1c1452a0 0x7fad1c06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1aa00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 0x7fad1c06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135608645 0 0) 0x7fad1c0cccc0 con 0x7fad1c1446d0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad00003520 con 0x7fad1c1446d0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 404278157 0 0) 0x7fad00003520 con 0x7fad1c1446d0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 msgr2=0x7fad1c06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 0x7fad1c06d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 >> v2:172.21.15.172:3300/0 conn(0x7fad1c1452a0 msgr2=0x7fad1c06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 --2- 172.21.15.165:0/254273352 >> v2:172.21.15.172:3300/0 conn(0x7fad1c1452a0 0x7fad1c06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c0cc140 con 0x7fad1c1446d0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fad0c001670 con 0x7fad1c1446d0 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 >> v1:172.21.15.165:6789/0 conn(0x7fad1c1446d0 legacy=0x7fad1c0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 0x7fad000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad00003f30 0x7fad00004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad00004d40 0x7fad000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0b2b20 con 0x7fad00004d40 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c06d400 con 0x7fad00003f30 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.140+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0000b9c0 con 0x7fad1c144d50 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1be00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad00004d40 0x7fad000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1b400640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 0x7fad000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1aa00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad00003f30 0x7fad00004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1be00640 1 -- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 msgr2=0x7fad000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1be00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad1c144d50 0x7fad000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1be00640 1 -- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad00003f30 msgr2=0x7fad00004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1be00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad00003f30 0x7fad00004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1be00640 1 -- 172.21.15.165:0/254273352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad00003520 con 0x7fad00004d40 2024-04-25T10:13:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.141+0000 7fad1aa00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad00003f30 0x7fad00004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.142+0000 7fad1be00640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad00004d40 0x7fad000051c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fad14009fc0 tx=0x7fad00005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd676e9aad829ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14009c10 con 0x7fad00004d40 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad14036070 con 0x7fad00004d40 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad1a000640 1 -- 172.21.15.165:0/254273352 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad1402d070 con 0x7fad00004d40 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad22200640 1 -- 172.21.15.165:0/254273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad00004d40 msgr2=0x7fad000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad22200640 1 --2- 172.21.15.165:0/254273352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad00004d40 0x7fad000051c0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fad14009fc0 tx=0x7fad00005830 comp rx=0 tx=0).stop 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad1be00640 1 -- 172.21.15.165:0/254273352 reap_dead start 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad22200640 1 -- 172.21.15.165:0/254273352 shutdown_connections 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad22200640 1 -- 172.21.15.165:0/254273352 >> 172.21.15.165:0/254273352 conn(0x7fad1c06abe0 msgr2=0x7fad1c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad22200640 1 -- 172.21.15.165:0/254273352 shutdown_connections 2024-04-25T10:13:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.143+0000 7fad22200640 1 -- 172.21.15.165:0/254273352 wait complete. 2024-04-25T10:13:53.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.145+0000 7fad22200640 1 Processor -- start 2024-04-25T10:13:53.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.145+0000 7fad22200640 1 -- start start 2024-04-25T10:13:53.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad22200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad00004d40 0x7fad1c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad22200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 0x7fad1c0a9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad22200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c0b0c30 0x7fad1c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad22200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0cc9e0 con 0x7fad1c1f4d90 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad22200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c0cc700 con 0x7fad1c0b0c30 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad22200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad1c144ab0 con 0x7fad00004d40 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 0x7fad1c0a9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad1aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 0x7fad1c0a9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40132/0 (socket says 172.21.15.165:40132) 2024-04-25T10:13:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.146+0000 7fad1aa00640 1 -- 172.21.15.165:0/3189220636 learned_addr learned my addr 172.21.15.165:0/3189220636 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1b400640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad00004d40 0x7fad1c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1aa00640 1 -- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad00004d40 msgr2=0x7fad1c0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1aa00640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad00004d40 0x7fad1c0a84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1be00640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c0b0c30 0x7fad1c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1aa00640 1 -- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c0b0c30 msgr2=0x7fad1c0b10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1aa00640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c0b0c30 0x7fad1c0b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1aa00640 1 -- 172.21.15.165:0/3189220636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad1c138830 con 0x7fad1c1f4d90 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad1aa00640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 0x7fad1c0a9500 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fad08010890 tx=0x7fad08010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.147+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800f040 con 0x7fad1c1f4d90 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.148+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800eb00 con 0x7fad1c1f4d90 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.148+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad1c1f5330 con 0x7fad1c1f4d90 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.148+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0801c400 con 0x7fad1c1f4d90 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.148+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad1c1f58f0 con 0x7fad1c1f4d90 2024-04-25T10:13:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.149+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad1c0b15f0 con 0x7fad1c1f4d90 2024-04-25T10:13:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.150+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fad0801c5a0 con 0x7fad1c1f4d90 2024-04-25T10:13:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.151+0000 7fad13400640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7facfc06b500 0x7facfc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.151+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fad0808a850 con 0x7fad1c1f4d90 2024-04-25T10:13:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.151+0000 7fad1b400640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7facfc06b500 0x7facfc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:53.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.152+0000 7fad1b400640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7facfc06b500 0x7facfc06d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0058b0 tx=0x7fad0c001d90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:53.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.156+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fad08018030 con 0x7fad1c1f4d90 2024-04-25T10:13:53.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.387+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fad1c073a00 con 0x7fad1c1f4d90 2024-04-25T10:13:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.924+0000 7fad13400640 1 -- 172.21.15.165:0/3189220636 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v79) v1 ==== 174+0+0 (secure 0 0 0) 0x7fad08059590 con 0x7fad1c1f4d90 2024-04-25T10:13:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 17 and data pool 18 2024-04-25T10:13:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7facfc06b500 msgr2=0x7facfc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7facfc06b500 0x7facfc06d9c0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fad0c0058b0 tx=0x7fad0c001d90 comp rx=0 tx=0).stop 2024-04-25T10:13:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 msgr2=0x7fad1c0a9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 0x7fad1c0a9500 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fad08010890 tx=0x7fad08010d50 comp rx=0 tx=0).stop 2024-04-25T10:13:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 shutdown_connections 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7facfc06b500 0x7facfc06d9c0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fad1c0b0c30 0x7fad1c0b10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fad1c1f4d90 0x7fad1c0a9500 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 --2- 172.21.15.165:0/3189220636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fad00004d40 0x7fad1c0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 >> 172.21.15.165:0/3189220636 conn(0x7fad1c06abe0 msgr2=0x7fad1c143780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 shutdown_connections 2024-04-25T10:13:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:53.928+0000 7fad22200640 1 -- 172.21.15.165:0/3189220636 wait complete. 2024-04-25T10:13:53.965 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:54.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.131+0000 7f6a32000640 1 Processor -- start 2024-04-25T10:13:54.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.131+0000 7f6a32000640 1 -- start start 2024-04-25T10:13:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.132+0000 7f6a32000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15e520 0x7f6a2c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.132+0000 7f6a32000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6a2c15f040 0x7f6a2c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a32000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6a2c0ae0d0 0x7f6a2c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a32000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2c0aee30 con 0x7f6a2c15f040 2024-04-25T10:13:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a32000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a2c0ae590 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a32000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c0ae870 con 0x7f6a2c15e520 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6a2c15f040 0x7f6a2c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6a2c15f040 0x7f6a2c0adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40134/0 (socket says 172.21.15.165:40134) 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 -- 172.21.15.165:0/3195724124 learned_addr learned my addr 172.21.15.165:0/3195724124 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 -- 172.21.15.165:0/3195724124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15e520 msgr2=0x7f6a2c15e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 --2- 172.21.15.165:0/3195724124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15e520 0x7f6a2c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 -- 172.21.15.165:0/3195724124 >> v2:172.21.15.172:3300/0 conn(0x7f6a2c0ae0d0 msgr2=0x7f6a2c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 --2- 172.21.15.165:0/3195724124 >> v2:172.21.15.172:3300/0 conn(0x7f6a2c0ae0d0 0x7f6a2c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.133+0000 7f6a2be00640 1 -- 172.21.15.165:0/3195724124 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c0aeb50 con 0x7f6a2c15f040 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.134+0000 7f6a2be00640 1 --2- 172.21.15.165:0/3195724124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c15f040 0x7f6a2c0adb90 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0cf020 tx=0x7f6a1c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71564927ea9668c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.135+0000 7f6a2b400640 1 -- 172.21.15.165:0/3195724124 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c037070 con 0x7f6a2c15f040 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.135+0000 7f6a2b400640 1 -- 172.21.15.165:0/3195724124 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a1c032440 con 0x7f6a2c15f040 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.135+0000 7f6a2b400640 1 -- 172.21.15.165:0/3195724124 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1c032810 con 0x7f6a2c15f040 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.135+0000 7f6a32000640 1 -- 172.21.15.165:0/3195724124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c15f040 msgr2=0x7f6a2c0adb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.135+0000 7f6a32000640 1 --2- 172.21.15.165:0/3195724124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c15f040 0x7f6a2c0adb90 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f6a2c0cf020 tx=0x7f6a1c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 -- 172.21.15.165:0/3195724124 shutdown_connections 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 --2- 172.21.15.165:0/3195724124 >> v2:172.21.15.172:3300/0 conn(0x7f6a2c0ae0d0 0x7f6a2c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 --2- 172.21.15.165:0/3195724124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c15f040 0x7f6a2c0adb90 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 --2- 172.21.15.165:0/3195724124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15e520 0x7f6a2c15e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 -- 172.21.15.165:0/3195724124 >> 172.21.15.165:0/3195724124 conn(0x7f6a2c06abe0 msgr2=0x7f6a2c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 -- 172.21.15.165:0/3195724124 shutdown_connections 2024-04-25T10:13:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.136+0000 7f6a32000640 1 -- 172.21.15.165:0/3195724124 wait complete. 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.137+0000 7f6a32000640 1 Processor -- start 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.137+0000 7f6a32000640 1 -- start start 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 0x7f6a2c0ce680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a32000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a2c15e520 0x7f6a2c145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a32000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15f040 0x7f6a2c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a32000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c15f620 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a32000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c15f4a0 con 0x7f6a2c15e520 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a32000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a2c071530 con 0x7f6a2c15f040 2024-04-25T10:13:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.138+0000 7f6a30c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 0x7f6a2c0ce680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 0x7f6a2c0ce680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40136/0 (socket says 172.21.15.165:40136) 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 -- 172.21.15.165:0/460559874 learned_addr learned my addr 172.21.15.165:0/460559874 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a2be00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a2c15e520 0x7f6a2c145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a31600640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15f040 0x7f6a2c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 -- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15f040 msgr2=0x7f6a2c15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15f040 0x7f6a2c15a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 -- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a2c15e520 msgr2=0x7f6a2c145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a2c15e520 0x7f6a2c145cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 -- 172.21.15.165:0/460559874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a2c1545f0 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.139+0000 7f6a30c00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 0x7f6a2c0ce680 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b4d0 tx=0x7f6a2000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.140+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20014070 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.140+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a20010870 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.140+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a20010c60 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.140+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2c0b15a0 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.140+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2c0b4240 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.141+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a2c073990 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.142+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6a20017070 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.143+0000 7f6a28c00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a1006b4d0 0x7f6a1006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.143+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6a2008a240 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.143+0000 7f6a2be00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a1006b4d0 0x7f6a1006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.144+0000 7f6a2be00640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a1006b4d0 0x7f6a1006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c034000 tx=0x7f6a1c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.147+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6a20058e90 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.341+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6a2c071190 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.343+0000 7f6a28c00640 1 -- 172.21.15.165:0/460559874 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17001 (secure 0 0 0) 0x7f6a20058790 con 0x7f6a2c0ae0d0 2024-04-25T10:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:54.347 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":123,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:53.388981+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a1006b4d0 msgr2=0x7f6a1006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a1006b4d0 0x7f6a1006d990 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f6a1c034000 tx=0x7f6a1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:13:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 msgr2=0x7f6a2c0ce680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 0x7f6a2c0ce680 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f6a2000b4d0 tx=0x7f6a2000b990 comp rx=0 tx=0).stop 2024-04-25T10:13:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 shutdown_connections 2024-04-25T10:13:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6a1006b4d0 0x7f6a1006d990 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6a2c15f040 0x7f6a2c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6a2c15e520 0x7f6a2c145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 --2- 172.21.15.165:0/460559874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6a2c0ae0d0 0x7f6a2c0ce680 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 >> 172.21.15.165:0/460559874 conn(0x7f6a2c06abe0 msgr2=0x7f6a2c15cca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 shutdown_connections 2024-04-25T10:13:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.346+0000 7f6a32000640 1 -- 172.21.15.165:0/460559874 wait complete. 2024-04-25T10:13:54.369 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:54.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.567+0000 7faadba00640 1 Processor -- start 2024-04-25T10:13:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.567+0000 7faadba00640 1 -- start start 2024-04-25T10:13:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faadba00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faad415dbe0 0x7faad40d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faadba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad40b2cb0 0x7faad40d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faadba00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad40b97a0 con 0x7faad40b92a0 2024-04-25T10:13:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faadba00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad415dfc0 con 0x7faad415dbe0 2024-04-25T10:13:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faadba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad40d1a00 con 0x7faad40b2cb0 2024-04-25T10:13:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faad9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faad415dbe0 0x7faad40d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faad9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faad415dbe0 0x7faad40d0910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40148/0 (socket says 172.21.15.165:40148) 2024-04-25T10:13:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faad9000640 1 -- 172.21.15.165:0/2853624484 learned_addr learned my addr 172.21.15.165:0/2853624484 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.569+0000 7faad9a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7faad40b92a0 0x7faad40b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58442/0 (socket says 172.21.15.165:58442) 2024-04-25T10:13:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.570+0000 7faad3e00640 1 --2- 172.21.15.165:0/2853624484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad40b2cb0 0x7faad40d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.570+0000 7faad9000640 1 -- 172.21.15.165:0/2853624484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad40b2cb0 msgr2=0x7faad40d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.570+0000 7faad9000640 1 --2- 172.21.15.165:0/2853624484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad40b2cb0 0x7faad40d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.570+0000 7faad9000640 1 -- 172.21.15.165:0/2853624484 >> v1:172.21.15.165:6789/0 conn(0x7faad40b92a0 legacy=0x7faad40b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.570+0000 7faad9000640 1 -- 172.21.15.165:0/2853624484 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad40d1c50 con 0x7faad415dbe0 2024-04-25T10:13:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.571+0000 7faad9000640 1 --2- 172.21.15.165:0/2853624484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad415dbe0 0x7faad40d0910 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7faac4009870 tx=0x7faac402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f9eec07d6b9aa95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.571+0000 7faad3400640 1 -- 172.21.15.165:0/2853624484 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac4037070 con 0x7faad415dbe0 2024-04-25T10:13:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.571+0000 7faad3400640 1 -- 172.21.15.165:0/2853624484 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac4032440 con 0x7faad415dbe0 2024-04-25T10:13:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.571+0000 7faad3400640 1 -- 172.21.15.165:0/2853624484 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac4032810 con 0x7faad415dbe0 2024-04-25T10:13:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 -- 172.21.15.165:0/2853624484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad415dbe0 msgr2=0x7faad40d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 --2- 172.21.15.165:0/2853624484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad415dbe0 0x7faad40d0910 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7faac4009870 tx=0x7faac402a250 comp rx=0 tx=0).stop 2024-04-25T10:13:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 -- 172.21.15.165:0/2853624484 shutdown_connections 2024-04-25T10:13:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 --2- 172.21.15.165:0/2853624484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad40b2cb0 0x7faad40d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 --2- 172.21.15.165:0/2853624484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad415dbe0 0x7faad40d0910 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 -- 172.21.15.165:0/2853624484 >> 172.21.15.165:0/2853624484 conn(0x7faad4061a20 msgr2=0x7faad4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 -- 172.21.15.165:0/2853624484 shutdown_connections 2024-04-25T10:13:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.572+0000 7faadba00640 1 -- 172.21.15.165:0/2853624484 wait complete. 2024-04-25T10:13:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.575+0000 7faadba00640 1 Processor -- start 2024-04-25T10:13:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.575+0000 7faadba00640 1 -- start start 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faadba00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad40b2cb0 0x7faad4145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faadba00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 0x7faad40cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faadba00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad415dbe0 0x7faad40c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faadba00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad40b6ab0 con 0x7faad40b92a0 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faadba00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad40d2210 con 0x7faad40b2cb0 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faadba00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad40b6c30 con 0x7faad415dbe0 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faad3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 0x7faad40cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faad3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 0x7faad40cab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40162/0 (socket says 172.21.15.165:40162) 2024-04-25T10:13:54.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.576+0000 7faad3e00640 1 -- 172.21.15.165:0/3348356596 learned_addr learned my addr 172.21.15.165:0/3348356596 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad9000640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad40b2cb0 0x7faad4145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad3e00640 1 -- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad415dbe0 msgr2=0x7faad40c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad3e00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad415dbe0 0x7faad40c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad3e00640 1 -- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad40b2cb0 msgr2=0x7faad4145660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad3e00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad40b2cb0 0x7faad4145660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad3e00640 1 -- 172.21.15.165:0/3348356596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad40b46a0 con 0x7faad40b92a0 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad3e00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 0x7faad40cab70 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7faac000d830 tx=0x7faac000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000b6c0 con 0x7faad40b92a0 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac0019070 con 0x7faad40b92a0 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac000ba80 con 0x7faad40b92a0 2024-04-25T10:13:54.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.577+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad40b4980 con 0x7faad40b92a0 2024-04-25T10:13:54.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.578+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad40b4f40 con 0x7faad40b92a0 2024-04-25T10:13:54.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.579+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad4073990 con 0x7faad40b92a0 2024-04-25T10:13:54.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.580+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faac001e460 con 0x7faad40b92a0 2024-04-25T10:13:54.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.580+0000 7faad0c00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab406b4d0 0x7faab406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:54.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.581+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faac008aae0 con 0x7faad40b92a0 2024-04-25T10:13:54.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.581+0000 7faad9000640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab406b4d0 0x7faab406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:54.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.581+0000 7faad9000640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab406b4d0 0x7faab406d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faac4034000 tx=0x7faac402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:54.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.585+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faac0059730 con 0x7faad40b92a0 2024-04-25T10:13:54.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.782+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faad406c8e0 con 0x7faad40b92a0 2024-04-25T10:13:54.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.785+0000 7faad0c00640 1 -- 172.21.15.165:0/3348356596 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+17001 (secure 0 0 0) 0x7faac0059030 con 0x7faad40b92a0 2024-04-25T10:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:54.790 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":123,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:53.388981+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab406b4d0 msgr2=0x7faab406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab406b4d0 0x7faab406d990 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7faac4034000 tx=0x7faac402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 msgr2=0x7faad40cab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 0x7faad40cab70 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7faac000d830 tx=0x7faac000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 shutdown_connections 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab406b4d0 0x7faab406d990 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad415dbe0 0x7faad40c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad40b92a0 0x7faad40cab70 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 --2- 172.21.15.165:0/3348356596 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad40b2cb0 0x7faad4145660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 >> 172.21.15.165:0/3348356596 conn(0x7faad4061a20 msgr2=0x7faad415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 shutdown_connections 2024-04-25T10:13:54.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:54.788+0000 7faadba00640 1 -- 172.21.15.165:0/3348356596 wait complete. 2024-04-25T10:13:54.812 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:13:54.895 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:13:54.889+0000 7f115e400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:13:55.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.015+0000 7ff788c00640 1 Processor -- start 2024-04-25T10:13:55.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.015+0000 7ff788c00640 1 -- start start 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.016+0000 7ff788c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7841446d0 0x7ff7840b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff788c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7840cccc0 con 0x7ff784144d50 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff788c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7840d0460 con 0x7ff78406e8a0 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff788c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7840b2b20 con 0x7ff7841446d0 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff781400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff784144d50 0x7ff7840cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58452/0 (socket says 172.21.15.165:58452) 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff781400640 1 -- 172.21.15.165:0/2594779949 learned_addr learned my addr 172.21.15.165:0/2594779949 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff781e00640 1 --2- 172.21.15.165:0/2594779949 >> v2:172.21.15.172:3300/0 conn(0x7ff7841446d0 0x7ff7840b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2992580635 0 0) 0x7ff7840cccc0 con 0x7ff784144d50 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.017+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff768003520 con 0x7ff784144d50 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2723772108 0 0) 0x7ff768003520 con 0x7ff784144d50 2024-04-25T10:13:55.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 >> v2:172.21.15.172:3300/0 conn(0x7ff7841446d0 msgr2=0x7ff7840b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 --2- 172.21.15.165:0/2594779949 >> v2:172.21.15.172:3300/0 conn(0x7ff7841446d0 0x7ff7840b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 >> v1:172.21.15.172:6789/0 conn(0x7ff78406e8a0 legacy=0x7ff7840cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840cc140 con 0x7ff784144d50 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff770001670 con 0x7ff784144d50 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 >> v1:172.21.15.165:6789/0 conn(0x7ff784144d50 legacy=0x7ff7840cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7680039b0 0x7ff768003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768004800 0x7ff768004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7680051c0 0x7ff76800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840b2b20 con 0x7ff768004800 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff76800c110 con 0x7ff7680039b0 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff76800c310 con 0x7ff7680051c0 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff781e00640 1 --2- 172.21.15.165:0/2594779949 >> v2:172.21.15.172:3300/0 conn(0x7ff7841446d0 0x7ff7840b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.018+0000 7ff781400640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768004800 0x7ff768004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.019+0000 7ff781400640 1 -- 172.21.15.165:0/2594779949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7680051c0 msgr2=0x7ff76800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.019+0000 7ff781400640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7680051c0 0x7ff76800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.019+0000 7ff781400640 1 -- 172.21.15.165:0/2594779949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7680039b0 msgr2=0x7ff768003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.019+0000 7ff781400640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7680039b0 0x7ff768003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.019+0000 7ff781400640 1 -- 172.21.15.165:0/2594779949 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768003520 con 0x7ff768004800 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.019+0000 7ff782800640 1 -- 172.21.15.165:0/2594779949 reap_dead start 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.020+0000 7ff781400640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768004800 0x7ff768004c80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff77000c240 tx=0x7ff768005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d776f49827902b16 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.020+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770007d10 con 0x7ff768004800 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.020+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770005ad0 con 0x7ff768004800 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.020+0000 7ff780a00640 1 -- 172.21.15.165:0/2594779949 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770007070 con 0x7ff768004800 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 -- 172.21.15.165:0/2594779949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768004800 msgr2=0x7ff768004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768004800 0x7ff768004c80 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7ff77000c240 tx=0x7ff768005830 comp rx=0 tx=0).stop 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 -- 172.21.15.165:0/2594779949 shutdown_connections 2024-04-25T10:13:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 --2- 172.21.15.165:0/2594779949 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff768004800 0x7ff768004c80 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 -- 172.21.15.165:0/2594779949 >> 172.21.15.165:0/2594779949 conn(0x7ff78406abe0 msgr2=0x7ff78406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:55.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 -- 172.21.15.165:0/2594779949 shutdown_connections 2024-04-25T10:13:55.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.021+0000 7ff788c00640 1 -- 172.21.15.165:0/2594779949 wait complete. 2024-04-25T10:13:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.023+0000 7ff788c00640 1 Processor -- start 2024-04-25T10:13:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.023+0000 7ff788c00640 1 -- start start 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff788c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff768004800 0x7ff7841595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff788c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 0x7ff7840acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff788c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78415a300 0x7ff7840a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff788c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840cc700 con 0x7ff784159db0 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff788c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff78406d400 con 0x7ff768004800 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff788c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7840cc9e0 con 0x7ff78415a300 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 0x7ff7840acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 0x7ff7840acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40176/0 (socket says 172.21.15.165:40176) 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 -- 172.21.15.165:0/3888840297 learned_addr learned my addr 172.21.15.165:0/3888840297 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781e00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff768004800 0x7ff7841595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff782800640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78415a300 0x7ff7840a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 -- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78415a300 msgr2=0x7ff7840a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78415a300 0x7ff7840a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 -- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff768004800 msgr2=0x7ff7841595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:55.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff768004800 0x7ff7841595f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781400640 1 -- 172.21.15.165:0/3888840297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7840a86f0 con 0x7ff784159db0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.024+0000 7ff781e00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff768004800 0x7ff7841595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff782800640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78415a300 0x7ff7840a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff781400640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 0x7ff7840acdf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7ff77000c240 tx=0x7ff78406d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770041070 con 0x7ff784159db0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff770004da0 con 0x7ff784159db0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff770007d10 con 0x7ff784159db0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7840adcd0 con 0x7ff784159db0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.025+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff784159b30 con 0x7ff784159db0 2024-04-25T10:13:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.026+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff784073a00 con 0x7ff784159db0 2024-04-25T10:13:55.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.027+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff770044070 con 0x7ff784159db0 2024-04-25T10:13:55.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.029+0000 7ff77a000640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff76406b4d0 0x7ff76406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:55.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.029+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7700aacf0 con 0x7ff784159db0 2024-04-25T10:13:55.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.029+0000 7ff781e00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff76406b4d0 0x7ff76406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:55.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.030+0000 7ff781e00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff76406b4d0 0x7ff76406d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff774006a90 tx=0x7ff774008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:55.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.033+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff770079ae0 con 0x7ff784159db0 2024-04-25T10:13:55.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:55.265+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff7840cccc0 con 0x7ff784159db0 2024-04-25T10:13:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.167+0000 7ff77a000640 1 -- 172.21.15.165:0/3888840297 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v81) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff7700793e0 con 0x7ff784159db0 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.170+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff76406b4d0 msgr2=0x7ff76406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.170+0000 7ff788c00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff76406b4d0 0x7ff76406d990 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff774006a90 tx=0x7ff774008040 comp rx=0 tx=0).stop 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.170+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 msgr2=0x7ff7840acdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.170+0000 7ff788c00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 0x7ff7840acdf0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7ff77000c240 tx=0x7ff78406d370 comp rx=0 tx=0).stop 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 shutdown_connections 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff76406b4d0 0x7ff76406d990 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff78415a300 0x7ff7840a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff784159db0 0x7ff7840acdf0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 --2- 172.21.15.165:0/3888840297 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff768004800 0x7ff7841595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 >> 172.21.15.165:0/3888840297 conn(0x7ff78406abe0 msgr2=0x7ff78415b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 shutdown_connections 2024-04-25T10:13:56.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.171+0000 7ff788c00640 1 -- 172.21.15.165:0/3888840297 wait complete. 2024-04-25T10:13:56.194 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:13:56.194 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:13:56.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.370+0000 7f4ab2200640 1 Processor -- start 2024-04-25T10:13:56.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.370+0000 7f4ab2200640 1 -- start start 2024-04-25T10:13:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 0x7f4aac0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab2200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4aac0cc150 0x7f4aac15e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab2200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4aac0b0b80 con 0x7f4aac15b3c0 2024-04-25T10:13:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab2200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4aac0cd650 con 0x7f4aac0cc150 2024-04-25T10:13:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cc5b0 con 0x7f4aac15b7a0 2024-04-25T10:13:56.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab0e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4aac15b3c0 0x7f4aac0cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58454/0 (socket says 172.21.15.165:58454) 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.372+0000 7f4ab0e00640 1 -- 172.21.15.165:0/2790572580 learned_addr learned my addr 172.21.15.165:0/2790572580 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4ab1800640 1 --2- 172.21.15.165:0/2790572580 >> v2:172.21.15.172:3300/0 conn(0x7f4aac0cc150 0x7f4aac15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aabe00640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 0x7f4aac0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208842647 0 0) 0x7f4aa0001110 con 0x7f4aac15b3c0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a94003520 con 0x7f4aac15b3c0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2980739944 0 0) 0x7f4a94003520 con 0x7f4aac15b3c0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 msgr2=0x7f4aac0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 0x7f4aac0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 >> v2:172.21.15.172:3300/0 conn(0x7f4aac0cc150 msgr2=0x7f4aac15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 --2- 172.21.15.165:0/2790572580 >> v2:172.21.15.172:3300/0 conn(0x7f4aac0cc150 0x7f4aac15e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac0cc800 con 0x7f4aac15b3c0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4aa0001840 con 0x7f4aac15b3c0 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 >> v1:172.21.15.165:6789/0 conn(0x7f4aac15b3c0 legacy=0x7f4aac0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.373+0000 7f4aab400640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4a94003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4aab400640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a94004800 0x7f4a94004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4aab400640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 0x7f4a940051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cd650 con 0x7f4a940039b0 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0cc5b0 con 0x7f4a94004800 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9400b9c0 con 0x7f4aac15b7a0 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4a94003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4ab0e00640 1 -- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 msgr2=0x7f4a940051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac15b7a0 0x7f4a940051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4ab0e00640 1 -- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a94004800 msgr2=0x7f4a94004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4a94004800 0x7f4a94004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.374+0000 7f4ab0e00640 1 -- 172.21.15.165:0/2790572580 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a94003520 con 0x7f4a940039b0 2024-04-25T10:13:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.375+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4a94003dd0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c3e0 tx=0x7f4a94005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2e1202648a41cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.375+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0032020 con 0x7f4a940039b0 2024-04-25T10:13:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.375+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0005af0 con 0x7f4a940039b0 2024-04-25T10:13:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.375+0000 7f4aab400640 1 -- 172.21.15.165:0/2790572580 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa000c0e0 con 0x7f4a940039b0 2024-04-25T10:13:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab2200640 1 -- 172.21.15.165:0/2790572580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 msgr2=0x7f4a94003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab2200640 1 --2- 172.21.15.165:0/2790572580 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4a94003dd0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c3e0 tx=0x7f4a94005830 comp rx=0 tx=0).stop 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab1800640 1 -- 172.21.15.165:0/2790572580 reap_dead start 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab2200640 1 -- 172.21.15.165:0/2790572580 shutdown_connections 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab2200640 1 -- 172.21.15.165:0/2790572580 >> 172.21.15.165:0/2790572580 conn(0x7f4aac061cc0 msgr2=0x7f4aac06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab2200640 1 -- 172.21.15.165:0/2790572580 shutdown_connections 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.376+0000 7f4ab2200640 1 -- 172.21.15.165:0/2790572580 wait complete. 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.378+0000 7f4ab2200640 1 Processor -- start 2024-04-25T10:13:56.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.378+0000 7f4ab2200640 1 -- start start 2024-04-25T10:13:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab2200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4aac1f4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aac14b480 0x7f4aac149450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b37c0 0x7f4aac148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab2200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0ce1f0 con 0x7f4a940039b0 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0ccdc0 con 0x7f4aac14b480 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4aac0ce370 con 0x7f4aac0b37c0 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4aac1f4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4aac1f4470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40202/0 (socket says 172.21.15.165:40202) 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.379+0000 7f4ab0e00640 1 -- 172.21.15.165:0/804624677 learned_addr learned my addr 172.21.15.165:0/804624677 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4aabe00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aac14b480 0x7f4aac149450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab0e00640 1 -- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b37c0 msgr2=0x7f4aac148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b37c0 0x7f4aac148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab0e00640 1 -- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aac14b480 msgr2=0x7f4aac149450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aac14b480 0x7f4aac149450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab0e00640 1 -- 172.21.15.165:0/804624677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4aac149990 con 0x7f4a940039b0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab0e00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4aac1f4470 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c3e0 tx=0x7f4aac0cc9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0032030 con 0x7f4a940039b0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4aa0004bf0 con 0x7f4a940039b0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aa0007070 con 0x7f4a940039b0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aac14b970 con 0x7f4a940039b0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.380+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4aac14bf30 con 0x7f4a940039b0 2024-04-25T10:13:56.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.381+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4aac073990 con 0x7f4a940039b0 2024-04-25T10:13:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.382+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4aa0035020 con 0x7f4a940039b0 2024-04-25T10:13:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.383+0000 7f4aa8c00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8c06b4d0 0x7f4a8c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.383+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4aa00aa4c0 con 0x7f4a940039b0 2024-04-25T10:13:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.384+0000 7f4aabe00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8c06b4d0 0x7f4a8c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.384+0000 7f4aabe00640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8c06b4d0 0x7f4a8c06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4aac14b8a0 tx=0x7f4a98008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.388+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4aac0b0b80 con 0x7f4a940039b0 2024-04-25T10:13:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:56.617+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4aac1e61d0 con 0x7f4a940039b0 2024-04-25T10:13:57.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.178+0000 7f4aa8c00640 1 -- 172.21.15.165:0/804624677 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4aa0079110 con 0x7f4a940039b0 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.181+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8c06b4d0 msgr2=0x7f4a8c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.181+0000 7f4ab2200640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8c06b4d0 0x7f4a8c06d990 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f4aac14b8a0 tx=0x7f4a98008040 comp rx=0 tx=0).stop 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.181+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 msgr2=0x7f4aac1f4470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.181+0000 7f4ab2200640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4aac1f4470 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f4aa000c3e0 tx=0x7f4aac0cc9d0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 shutdown_connections 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4a8c06b4d0 0x7f4a8c06d990 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4aac0b37c0 0x7f4aac148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4aac14b480 0x7f4aac149450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 --2- 172.21.15.165:0/804624677 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4a940039b0 0x7f4aac1f4470 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 >> 172.21.15.165:0/804624677 conn(0x7f4aac061cc0 msgr2=0x7f4aac0cb020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 shutdown_connections 2024-04-25T10:13:57.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.182+0000 7f4ab2200640 1 -- 172.21.15.165:0/804624677 wait complete. 2024-04-25T10:13:57.207 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:13:57.207 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:57.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.398+0000 7f508a200640 1 Processor -- start 2024-04-25T10:13:57.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.399+0000 7f508a200640 1 -- start start 2024-04-25T10:13:57.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.400+0000 7f508a200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f508415e520 0x7f508415e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.400+0000 7f508a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508415f040 0x7f50840adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.400+0000 7f508a200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50840ae590 con 0x7f50840ae0d0 2024-04-25T10:13:57.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.400+0000 7f508a200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50840aee30 con 0x7f508415e520 2024-04-25T10:13:57.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.400+0000 7f508a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50840ae870 con 0x7f508415f040 2024-04-25T10:13:57.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5089800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f50840ae0d0 0x7f5084070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35582/0 (socket says 172.21.15.165:35582) 2024-04-25T10:13:57.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5089800640 1 -- 172.21.15.165:0/2808082414 learned_addr learned my addr 172.21.15.165:0/2808082414 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f507be00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508415f040 0x7f50840adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5088e00640 1 --2- 172.21.15.165:0/2808082414 >> v2:172.21.15.172:3300/0 conn(0x7f508415e520 0x7f508415e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296321932 0 0) 0x7f50840ae590 con 0x7f50840ae0d0 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5068003520 con 0x7f50840ae0d0 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1518587622 0 0) 0x7f5068003520 con 0x7f50840ae0d0 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508415f040 msgr2=0x7f50840adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508415f040 0x7f50840adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 >> v2:172.21.15.172:3300/0 conn(0x7f508415e520 msgr2=0x7f508415e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 --2- 172.21.15.165:0/2808082414 >> v2:172.21.15.172:3300/0 conn(0x7f508415e520 0x7f508415e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50840aeb50 con 0x7f50840ae0d0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f507c0019f0 con 0x7f50840ae0d0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 >> v1:172.21.15.165:6789/0 conn(0x7f50840ae0d0 legacy=0x7f5084070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50680039b0 0x7f5068003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.401+0000 7f5083e00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5068004800 0x7f5068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f5083e00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50680051c0 0x7f506800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840aee30 con 0x7f5068004800 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50840ae870 con 0x7f50680039b0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506800c030 con 0x7f50680051c0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f508415f040 0x7f50840adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5068004800 0x7f5068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f5088e00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50680039b0 0x7f5068003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f5089800640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50680051c0 0x7f506800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 -- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50680051c0 msgr2=0x7f506800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50680051c0 0x7f506800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 -- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50680039b0 msgr2=0x7f5068003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50680039b0 0x7f5068003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.402+0000 7f507be00640 1 -- 172.21.15.165:0/2808082414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5068003520 con 0x7f5068004800 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.403+0000 7f507be00640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5068004800 0x7f5068004c80 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f506c006fd0 tx=0x7f5068005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51b4640eed59f3be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c037070 con 0x7f5068004800 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f506c02ab10 con 0x7f5068004800 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f5083e00640 1 -- 172.21.15.165:0/2808082414 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f506c00f040 con 0x7f5068004800 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f508a200640 1 -- 172.21.15.165:0/2808082414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5068004800 msgr2=0x7f5068004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f508a200640 1 --2- 172.21.15.165:0/2808082414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5068004800 0x7f5068004c80 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f506c006fd0 tx=0x7f5068005830 comp rx=0 tx=0).stop 2024-04-25T10:13:57.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f5089800640 1 -- 172.21.15.165:0/2808082414 reap_dead start 2024-04-25T10:13:57.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f508a200640 1 -- 172.21.15.165:0/2808082414 shutdown_connections 2024-04-25T10:13:57.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f508a200640 1 -- 172.21.15.165:0/2808082414 >> 172.21.15.165:0/2808082414 conn(0x7f508406abe0 msgr2=0x7f508406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f508a200640 1 -- 172.21.15.165:0/2808082414 shutdown_connections 2024-04-25T10:13:57.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.404+0000 7f508a200640 1 -- 172.21.15.165:0/2808082414 wait complete. 2024-04-25T10:13:57.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.406+0000 7f508a200640 1 Processor -- start 2024-04-25T10:13:57.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.406+0000 7f508a200640 1 -- start start 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f508a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5068004800 0x7f5084133dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f508a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50840caea0 0x7f5084135570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f508a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 0x7f5084137730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f508a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508415f620 con 0x7f50841372b0 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f508a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508415f4a0 con 0x7f50840caea0 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f508a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5084071700 con 0x7f5068004800 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f5089800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 0x7f5084137730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f5089800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 0x7f5084137730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38108/0 (socket says 172.21.15.165:38108) 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.407+0000 7f5089800640 1 -- 172.21.15.165:0/1293219321 learned_addr learned my addr 172.21.15.165:0/1293219321 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5088e00640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5068004800 0x7f5084133dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f507be00640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50840caea0 0x7f5084135570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5089800640 1 -- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5068004800 msgr2=0x7f5084133dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5089800640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5068004800 0x7f5084133dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5089800640 1 -- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50840caea0 msgr2=0x7f5084135570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5089800640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50840caea0 0x7f5084135570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5089800640 1 -- 172.21.15.165:0/1293219321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50841344d0 con 0x7f50841372b0 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5089800640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 0x7f5084137730 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f50840b4bd0 tx=0x7f507c00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c00bdb0 con 0x7f50841372b0 2024-04-25T10:13:57.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c004b60 con 0x7f50841372b0 2024-04-25T10:13:57.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50841347b0 con 0x7f50841372b0 2024-04-25T10:13:57.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c007070 con 0x7f50841372b0 2024-04-25T10:13:57.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.408+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5084134d70 con 0x7f50841372b0 2024-04-25T10:13:57.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.409+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5084073990 con 0x7f50841372b0 2024-04-25T10:13:57.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.410+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f507c01b020 con 0x7f50841372b0 2024-04-25T10:13:57.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.411+0000 7f5081600640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f506006b4d0 0x7f506006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.411+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f507c08a550 con 0x7f50841372b0 2024-04-25T10:13:57.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.412+0000 7f5088e00640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f506006b4d0 0x7f506006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.412+0000 7f5088e00640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f506006b4d0 0x7f506006d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.416+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f507c0591a0 con 0x7f50841372b0 2024-04-25T10:13:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.647+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f508415b300 con 0x7f50841372b0 2024-04-25T10:13:57.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.648+0000 7f5081600640 1 -- 172.21.15.165:0/1293219321 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f507c058aa0 con 0x7f50841372b0 2024-04-25T10:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:57.652 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:13:57.652 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f506006b4d0 msgr2=0x7f506006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f506006b4d0 0x7f506006d990 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5074006fd0 tx=0x7f5074008040 comp rx=0 tx=0).stop 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 msgr2=0x7f5084137730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 0x7f5084137730 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f50840b4bd0 tx=0x7f507c00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 shutdown_connections 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f506006b4d0 0x7f506006d990 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50841372b0 0x7f5084137730 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f50840caea0 0x7f5084135570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 --2- 172.21.15.165:0/1293219321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5068004800 0x7f5084133dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 >> 172.21.15.165:0/1293219321 conn(0x7f508406abe0 msgr2=0x7f50840af100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 shutdown_connections 2024-04-25T10:13:57.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.651+0000 7f508a200640 1 -- 172.21.15.165:0/1293219321 wait complete. 2024-04-25T10:13:57.673 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:13:57.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.865+0000 7f5ae6400640 1 Processor -- start 2024-04-25T10:13:57.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.865+0000 7f5ae6400640 1 -- start start 2024-04-25T10:13:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae6400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ae015db90 0x7f5ae00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae6400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00d33f0 con 0x7f5ae00d04f0 2024-04-25T10:13:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae6400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ae00d0980 con 0x7f5ae015db90 2024-04-25T10:13:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00d1f50 con 0x7f5ae01615a0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae5000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ae015db90 0x7f5ae00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae5000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ae015db90 0x7f5ae00b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38116/0 (socket says 172.21.15.165:38116) 2024-04-25T10:13:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.867+0000 7f5ae5000640 1 -- 172.21.15.165:0/39309734 learned_addr learned my addr 172.21.15.165:0/39309734 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.868+0000 7f5ad7e00640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.868+0000 7f5ae5000640 1 -- 172.21.15.165:0/39309734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 msgr2=0x7f5ae00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.868+0000 7f5ae5000640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.868+0000 7f5ae5000640 1 -- 172.21.15.165:0/39309734 >> v1:172.21.15.172:6789/0 conn(0x7f5ae00d04f0 legacy=0x7f5ae0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.868+0000 7f5ae5000640 1 -- 172.21.15.165:0/39309734 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0161df0 con 0x7f5ae015db90 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.868+0000 7f5ad7e00640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:57.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.869+0000 7f5ae5000640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae015db90 0x7f5ae00b93c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009870 tx=0x7f5ad002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7156c9f22c6ef274 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.869+0000 7f5adfe00640 1 -- 172.21.15.165:0/39309734 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0037070 con 0x7f5ae015db90 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.869+0000 7f5adfe00640 1 -- 172.21.15.165:0/39309734 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad0032440 con 0x7f5ae015db90 2024-04-25T10:13:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.869+0000 7f5adfe00640 1 -- 172.21.15.165:0/39309734 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0032810 con 0x7f5ae015db90 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 -- 172.21.15.165:0/39309734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae015db90 msgr2=0x7f5ae00b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae015db90 0x7f5ae00b93c0 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0009870 tx=0x7f5ad002a210 comp rx=0 tx=0).stop 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 -- 172.21.15.165:0/39309734 shutdown_connections 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 --2- 172.21.15.165:0/39309734 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae015db90 0x7f5ae00b93c0 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 -- 172.21.15.165:0/39309734 >> 172.21.15.165:0/39309734 conn(0x7f5ae00b52b0 msgr2=0x7f5ae00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 -- 172.21.15.165:0/39309734 shutdown_connections 2024-04-25T10:13:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.870+0000 7f5ae6400640 1 -- 172.21.15.165:0/39309734 wait complete. 2024-04-25T10:13:57.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.872+0000 7f5ae6400640 1 Processor -- start 2024-04-25T10:13:57.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.872+0000 7f5ae6400640 1 -- start start 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.873+0000 7f5ae6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 0x7f5ae014ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.873+0000 7f5ae6400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ae015db90 0x7f5ae00cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.873+0000 7f5ae6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00c9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.873+0000 7f5ae6400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae00d1be0 con 0x7f5ae00d04f0 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.873+0000 7f5ae6400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae01623b0 con 0x7f5ae015db90 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.873+0000 7f5ae6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0161a20 con 0x7f5ae01615a0 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 0x7f5ae014ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 0x7f5ae014ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38124/0 (socket says 172.21.15.165:38124) 2024-04-25T10:13:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 -- 172.21.15.165:0/1006881503 learned_addr learned my addr 172.21.15.165:0/1006881503 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 -- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 msgr2=0x7f5ae00c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00c9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 -- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ae015db90 msgr2=0x7f5ae00cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ad7e00640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ae015db90 0x7f5ae00cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ae015db90 0x7f5ae00cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 -- 172.21.15.165:0/1006881503 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0160040 con 0x7f5ae00d04f0 2024-04-25T10:13:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.874+0000 7f5ae5000640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 0x7f5ae014ec60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.875+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0037040 con 0x7f5ae00d04f0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.875+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad0042070 con 0x7f5ae00d04f0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.875+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0030bd0 con 0x7f5ae00d04f0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.875+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae0160320 con 0x7f5ae00d04f0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.875+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae01608e0 con 0x7f5ae00d04f0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.876+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae0073990 con 0x7f5ae00d04f0 2024-04-25T10:13:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.877+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ad0047460 con 0x7f5ae00d04f0 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.878+0000 7f5add600640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5abc06b500 0x7f5abc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.878+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ad00aafc0 con 0x7f5ae00d04f0 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.878+0000 7f5ad7e00640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5abc06b500 0x7f5abc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.879+0000 7f5ad7e00640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5abc06b500 0x7f5abc06d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ae015fe30 tx=0x7f5ac8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:57.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:57.882+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ad0045360 con 0x7f5ae00d04f0 2024-04-25T10:13:58.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.077+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae00d14b0 con 0x7f5ae00d04f0 2024-04-25T10:13:58.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.079+0000 7f5add600640 1 -- 172.21.15.165:0/1006881503 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f5ad0079ca0 con 0x7f5ae00d04f0 2024-04-25T10:13:58.083 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:58.084 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":124,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:54.924428+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:13:58.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5abc06b500 msgr2=0x7f5abc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5abc06b500 0x7f5abc06d9c0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ae015fe30 tx=0x7f5ac8008040 comp rx=0 tx=0).stop 2024-04-25T10:13:58.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 msgr2=0x7f5ae014ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 0x7f5ae014ec60 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f5ad0000c00 tx=0x7f5ad00046c0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 shutdown_connections 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5abc06b500 0x7f5abc06d9c0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ae01615a0 0x7f5ae00c9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ae015db90 0x7f5ae00cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 --2- 172.21.15.165:0/1006881503 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ae00d04f0 0x7f5ae014ec60 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 >> 172.21.15.165:0/1006881503 conn(0x7f5ae00b52b0 msgr2=0x7f5ae00b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 shutdown_connections 2024-04-25T10:13:58.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.082+0000 7f5ae6400640 1 -- 172.21.15.165:0/1006881503 wait complete. 2024-04-25T10:13:58.105 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:13:58.105 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:58.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.305+0000 7fc333400640 1 Processor -- start 2024-04-25T10:13:58.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.306+0000 7fc333400640 1 -- start start 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.307+0000 7fc333400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc32c165680 0x7fc32c0aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.307+0000 7fc333400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c0cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.307+0000 7fc333400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc32c0cf330 con 0x7fc32c0cec70 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.307+0000 7fc333400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc32c0cf100 con 0x7fc32c165680 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.307+0000 7fc333400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c072340 con 0x7fc32c0b2650 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.307+0000 7fc330a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc32c165680 0x7fc32c0aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc330a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc32c165680 0x7fc32c0aecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38138/0 (socket says 172.21.15.165:38138) 2024-04-25T10:13:58.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc330a00640 1 -- 172.21.15.165:0/4200093775 learned_addr learned my addr 172.21.15.165:0/4200093775 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:58.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc331400640 1 --2- 172.21.15.165:0/4200093775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c0cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc330a00640 1 -- 172.21.15.165:0/4200093775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 msgr2=0x7fc32c0cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc330a00640 1 --2- 172.21.15.165:0/4200093775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c0cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc330a00640 1 -- 172.21.15.165:0/4200093775 >> v1:172.21.15.172:6789/0 conn(0x7fc32c0cec70 legacy=0x7fc32c0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.308+0000 7fc330a00640 1 -- 172.21.15.165:0/4200093775 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c071d80 con 0x7fc32c165680 2024-04-25T10:13:58.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.309+0000 7fc330a00640 1 --2- 172.21.15.165:0/4200093775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c0aecf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc314009870 tx=0x7fc31402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f117a788cb9c3e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.309+0000 7fc32be00640 1 -- 172.21.15.165:0/4200093775 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314037070 con 0x7fc32c165680 2024-04-25T10:13:58.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.309+0000 7fc32be00640 1 -- 172.21.15.165:0/4200093775 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc314032440 con 0x7fc32c165680 2024-04-25T10:13:58.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.309+0000 7fc32be00640 1 -- 172.21.15.165:0/4200093775 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc314032810 con 0x7fc32c165680 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 -- 172.21.15.165:0/4200093775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 msgr2=0x7fc32c0aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 --2- 172.21.15.165:0/4200093775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c0aecf0 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7fc314009870 tx=0x7fc31402a180 comp rx=0 tx=0).stop 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 -- 172.21.15.165:0/4200093775 shutdown_connections 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 --2- 172.21.15.165:0/4200093775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c0cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 --2- 172.21.15.165:0/4200093775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c0aecf0 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 -- 172.21.15.165:0/4200093775 >> 172.21.15.165:0/4200093775 conn(0x7fc32c06abe0 msgr2=0x7fc32c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 -- 172.21.15.165:0/4200093775 shutdown_connections 2024-04-25T10:13:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.310+0000 7fc333400640 1 -- 172.21.15.165:0/4200093775 wait complete. 2024-04-25T10:13:58.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.312+0000 7fc333400640 1 Processor -- start 2024-04-25T10:13:58.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.312+0000 7fc333400640 1 -- start start 2024-04-25T10:13:58.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc333400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc333400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc32c0cec70 0x7fc32c15ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc333400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c15f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc333400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c06ea30 con 0x7fc32c165680 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc333400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c071670 con 0x7fc32c0cec70 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc333400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32c165a60 con 0x7fc32c0b2650 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc331400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c15f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc331400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c15f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38150/0 (socket says 172.21.15.165:38150) 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.313+0000 7fc331400640 1 -- 172.21.15.165:0/136120004 learned_addr learned my addr 172.21.15.165:0/136120004 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc330a00640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc323e00640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc32c0cec70 0x7fc32c15ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc331400640 1 -- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 msgr2=0x7fc32c157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc331400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc331400640 1 -- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc32c0cec70 msgr2=0x7fc32c15ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc331400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc32c0cec70 0x7fc32c15ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc331400640 1 -- 172.21.15.165:0/136120004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c160fb0 con 0x7fc32c165680 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc323e00640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc32c0cec70 0x7fc32c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc331400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c15f170 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc324007e60 tx=0x7fc32400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324014420 con 0x7fc32c165680 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc324019070 con 0x7fc32c165680 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc324014820 con 0x7fc32c165680 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.314+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc32c161290 con 0x7fc32c165680 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.315+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc32c161850 con 0x7fc32c165680 2024-04-25T10:13:58.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.316+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32c073990 con 0x7fc32c165680 2024-04-25T10:13:58.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.317+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc3240149c0 con 0x7fc32c165680 2024-04-25T10:13:58.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.318+0000 7fc329600640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc30806b4d0 0x7fc30806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.318+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc32408ac90 con 0x7fc32c165680 2024-04-25T10:13:58.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.318+0000 7fc330a00640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc30806b4d0 0x7fc30806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.319+0000 7fc330a00640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc30806b4d0 0x7fc30806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc31402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.322+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc3240598e0 con 0x7fc32c165680 2024-04-25T10:13:58.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.515+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc32c15cd80 con 0x7fc30806b4d0 2024-04-25T10:13:58.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.534+0000 7fc329600640 1 -- 172.21.15.165:0/136120004 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7fc31407fa60 con 0x7fc30806b4d0 2024-04-25T10:13:58.542 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:58.544 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc30806b4d0 msgr2=0x7fc30806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc30806b4d0 0x7fc30806d990 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fc314034000 tx=0x7fc31402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 msgr2=0x7fc32c15f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c15f170 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fc324007e60 tx=0x7fc32400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 shutdown_connections 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc30806b4d0 0x7fc30806d990 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc32c165680 0x7fc32c15f170 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc32c0cec70 0x7fc32c15ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 --2- 172.21.15.165:0/136120004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc32c0b2650 0x7fc32c157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.541+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 >> 172.21.15.165:0/136120004 conn(0x7fc32c06abe0 msgr2=0x7fc32c164120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.542+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 shutdown_connections 2024-04-25T10:13:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.542+0000 7fc333400640 1 -- 172.21.15.165:0/136120004 wait complete. 2024-04-25T10:13:58.564 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-04-25T10:13:58.373107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444568,"kb_used_data":13288,"kb_used_omap":12,"kb_used_meta":431155,"kb_avail":754530152,"statfs":{"total":773094113280,"available":772638875648,"internally_reserved":0,"allocated":13606912,"data_stored":6978044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":441503282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001505"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703632+0000","last_change":"2024-04-25T10:13:53.938397+0000","last_active":"2024-04-25T10:13:55.703632+0000","last_peered":"2024-04-25T10:13:55.703632+0000","last_clean":"2024-04-25T10:13:55.703632+0000","last_became_active":"2024-04-25T10:13:53.938046+0000","last_became_peered":"2024-04-25T10:13:53.938046+0000","last_unstale":"2024-04-25T10:13:55.703632+0000","last_undegraded":"2024-04-25T10:13:55.703632+0000","last_fullsized":"2024-04-25T10:13:55.703632+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:05.909155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703644+0000","last_change":"2024-04-25T10:13:53.974401+0000","last_active":"2024-04-25T10:13:55.703644+0000","last_peered":"2024-04-25T10:13:55.703644+0000","last_clean":"2024-04-25T10:13:55.703644+0000","last_became_active":"2024-04-25T10:13:53.973601+0000","last_became_peered":"2024-04-25T10:13:53.973601+0000","last_unstale":"2024-04-25T10:13:55.703644+0000","last_undegraded":"2024-04-25T10:13:55.703644+0000","last_fullsized":"2024-04-25T10:13:55.703644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:22.949190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704042+0000","last_change":"2024-04-25T10:13:53.963140+0000","last_active":"2024-04-25T10:13:55.704042+0000","last_peered":"2024-04-25T10:13:55.704042+0000","last_clean":"2024-04-25T10:13:55.704042+0000","last_became_active":"2024-04-25T10:13:53.962832+0000","last_became_peered":"2024-04-25T10:13:53.962832+0000","last_unstale":"2024-04-25T10:13:55.704042+0000","last_undegraded":"2024-04-25T10:13:55.704042+0000","last_fullsized":"2024-04-25T10:13:55.704042+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:37.819231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703811+0000","last_change":"2024-04-25T10:13:53.963887+0000","last_active":"2024-04-25T10:13:55.703811+0000","last_peered":"2024-04-25T10:13:55.703811+0000","last_clean":"2024-04-25T10:13:55.703811+0000","last_became_active":"2024-04-25T10:13:53.961528+0000","last_became_peered":"2024-04-25T10:13:53.961528+0000","last_unstale":"2024-04-25T10:13:55.703811+0000","last_undegraded":"2024-04-25T10:13:55.703811+0000","last_fullsized":"2024-04-25T10:13:55.703811+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:49.121305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703032+0000","last_change":"2024-04-25T10:13:53.942190+0000","last_active":"2024-04-25T10:13:55.703032+0000","last_peered":"2024-04-25T10:13:55.703032+0000","last_clean":"2024-04-25T10:13:55.703032+0000","last_became_active":"2024-04-25T10:13:53.940783+0000","last_became_peered":"2024-04-25T10:13:53.940783+0000","last_unstale":"2024-04-25T10:13:55.703032+0000","last_undegraded":"2024-04-25T10:13:55.703032+0000","last_fullsized":"2024-04-25T10:13:55.703032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:27.688942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944266+0000","last_change":"2024-04-25T10:13:53.968179+0000","last_active":"2024-04-25T10:13:54.944266+0000","last_peered":"2024-04-25T10:13:54.944266+0000","last_clean":"2024-04-25T10:13:54.944266+0000","last_became_active":"2024-04-25T10:13:53.967888+0000","last_became_peered":"2024-04-25T10:13:53.967888+0000","last_unstale":"2024-04-25T10:13:54.944266+0000","last_undegraded":"2024-04-25T10:13:54.944266+0000","last_fullsized":"2024-04-25T10:13:54.944266+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:55.521986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943783+0000","last_change":"2024-04-25T10:13:53.950947+0000","last_active":"2024-04-25T10:13:54.943783+0000","last_peered":"2024-04-25T10:13:54.943783+0000","last_clean":"2024-04-25T10:13:54.943783+0000","last_became_active":"2024-04-25T10:13:53.950654+0000","last_became_peered":"2024-04-25T10:13:53.950654+0000","last_unstale":"2024-04-25T10:13:54.943783+0000","last_undegraded":"2024-04-25T10:13:54.943783+0000","last_fullsized":"2024-04-25T10:13:54.943783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:15.539574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703615+0000","last_change":"2024-04-25T10:13:53.965105+0000","last_active":"2024-04-25T10:13:55.703615+0000","last_peered":"2024-04-25T10:13:55.703615+0000","last_clean":"2024-04-25T10:13:55.703615+0000","last_became_active":"2024-04-25T10:13:53.963012+0000","last_became_peered":"2024-04-25T10:13:53.963012+0000","last_unstale":"2024-04-25T10:13:55.703615+0000","last_undegraded":"2024-04-25T10:13:55.703615+0000","last_fullsized":"2024-04-25T10:13:55.703615+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:05.733395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704758+0000","last_change":"2024-04-25T10:13:53.966264+0000","last_active":"2024-04-25T10:13:55.704758+0000","last_peered":"2024-04-25T10:13:55.704758+0000","last_clean":"2024-04-25T10:13:55.704758+0000","last_became_active":"2024-04-25T10:13:53.965946+0000","last_became_peered":"2024-04-25T10:13:53.965946+0000","last_unstale":"2024-04-25T10:13:55.704758+0000","last_undegraded":"2024-04-25T10:13:55.704758+0000","last_fullsized":"2024-04-25T10:13:55.704758+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:53.911639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933761+0000","last_change":"2024-04-25T10:13:53.955704+0000","last_active":"2024-04-25T10:13:54.933761+0000","last_peered":"2024-04-25T10:13:54.933761+0000","last_clean":"2024-04-25T10:13:54.933761+0000","last_became_active":"2024-04-25T10:13:53.949523+0000","last_became_peered":"2024-04-25T10:13:53.949523+0000","last_unstale":"2024-04-25T10:13:54.933761+0000","last_undegraded":"2024-04-25T10:13:54.933761+0000","last_fullsized":"2024-04-25T10:13:54.933761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:12.419457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703006+0000","last_change":"2024-04-25T10:13:53.949693+0000","last_active":"2024-04-25T10:13:55.703006+0000","last_peered":"2024-04-25T10:13:55.703006+0000","last_clean":"2024-04-25T10:13:55.703006+0000","last_became_active":"2024-04-25T10:13:53.948556+0000","last_became_peered":"2024-04-25T10:13:53.948556+0000","last_unstale":"2024-04-25T10:13:55.703006+0000","last_undegraded":"2024-04-25T10:13:55.703006+0000","last_fullsized":"2024-04-25T10:13:55.703006+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:42.566879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944941+0000","last_change":"2024-04-25T10:13:53.945609+0000","last_active":"2024-04-25T10:13:54.944941+0000","last_peered":"2024-04-25T10:13:54.944941+0000","last_clean":"2024-04-25T10:13:54.944941+0000","last_became_active":"2024-04-25T10:13:53.940951+0000","last_became_peered":"2024-04-25T10:13:53.940951+0000","last_unstale":"2024-04-25T10:13:54.944941+0000","last_undegraded":"2024-04-25T10:13:54.944941+0000","last_fullsized":"2024-04-25T10:13:54.944941+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:00.452282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933423+0000","last_change":"2024-04-25T10:13:53.948512+0000","last_active":"2024-04-25T10:13:54.933423+0000","last_peered":"2024-04-25T10:13:54.933423+0000","last_clean":"2024-04-25T10:13:54.933423+0000","last_became_active":"2024-04-25T10:13:53.948170+0000","last_became_peered":"2024-04-25T10:13:53.948170+0000","last_unstale":"2024-04-25T10:13:54.933423+0000","last_undegraded":"2024-04-25T10:13:54.933423+0000","last_fullsized":"2024-04-25T10:13:54.933423+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:25.935473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703117+0000","last_change":"2024-04-25T10:13:53.965655+0000","last_active":"2024-04-25T10:13:55.703117+0000","last_peered":"2024-04-25T10:13:55.703117+0000","last_clean":"2024-04-25T10:13:55.703117+0000","last_became_active":"2024-04-25T10:13:53.963844+0000","last_became_peered":"2024-04-25T10:13:53.963844+0000","last_unstale":"2024-04-25T10:13:55.703117+0000","last_undegraded":"2024-04-25T10:13:55.703117+0000","last_fullsized":"2024-04-25T10:13:55.703117+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:58.988897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.701959+0000","last_change":"2024-04-25T10:13:53.947215+0000","last_active":"2024-04-25T10:13:55.701959+0000","last_peered":"2024-04-25T10:13:55.701959+0000","last_clean":"2024-04-25T10:13:55.701959+0000","last_became_active":"2024-04-25T10:13:53.946710+0000","last_became_peered":"2024-04-25T10:13:53.946710+0000","last_unstale":"2024-04-25T10:13:55.701959+0000","last_undegraded":"2024-04-25T10:13:55.701959+0000","last_fullsized":"2024-04-25T10:13:55.701959+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:38.769942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943388+0000","last_change":"2024-04-25T10:13:53.956248+0000","last_active":"2024-04-25T10:13:54.943388+0000","last_peered":"2024-04-25T10:13:54.943388+0000","last_clean":"2024-04-25T10:13:54.943388+0000","last_became_active":"2024-04-25T10:13:53.955567+0000","last_became_peered":"2024-04-25T10:13:53.955567+0000","last_unstale":"2024-04-25T10:13:54.943388+0000","last_undegraded":"2024-04-25T10:13:54.943388+0000","last_fullsized":"2024-04-25T10:13:54.943388+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:02.712721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933637+0000","last_change":"2024-04-25T10:13:53.948244+0000","last_active":"2024-04-25T10:13:54.933637+0000","last_peered":"2024-04-25T10:13:54.933637+0000","last_clean":"2024-04-25T10:13:54.933637+0000","last_became_active":"2024-04-25T10:13:53.947829+0000","last_became_peered":"2024-04-25T10:13:53.947829+0000","last_unstale":"2024-04-25T10:13:54.933637+0000","last_undegraded":"2024-04-25T10:13:54.933637+0000","last_fullsized":"2024-04-25T10:13:54.933637+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.041456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704082+0000","last_change":"2024-04-25T10:13:53.973596+0000","last_active":"2024-04-25T10:13:55.704082+0000","last_peered":"2024-04-25T10:13:55.704082+0000","last_clean":"2024-04-25T10:13:55.704082+0000","last_became_active":"2024-04-25T10:13:53.973355+0000","last_became_peered":"2024-04-25T10:13:53.973355+0000","last_unstale":"2024-04-25T10:13:55.704082+0000","last_undegraded":"2024-04-25T10:13:55.704082+0000","last_fullsized":"2024-04-25T10:13:55.704082+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:11.185126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938428+0000","last_change":"2024-04-25T10:13:53.960666+0000","last_active":"2024-04-25T10:13:54.938428+0000","last_peered":"2024-04-25T10:13:54.938428+0000","last_clean":"2024-04-25T10:13:54.938428+0000","last_became_active":"2024-04-25T10:13:53.960313+0000","last_became_peered":"2024-04-25T10:13:53.960313+0000","last_unstale":"2024-04-25T10:13:54.938428+0000","last_undegraded":"2024-04-25T10:13:54.938428+0000","last_fullsized":"2024-04-25T10:13:54.938428+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.074055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160444+0000","last_change":"2024-04-25T10:13:53.953933+0000","last_active":"2024-04-25T10:13:55.160444+0000","last_peered":"2024-04-25T10:13:55.160444+0000","last_clean":"2024-04-25T10:13:55.160444+0000","last_became_active":"2024-04-25T10:13:53.953645+0000","last_became_peered":"2024-04-25T10:13:53.953645+0000","last_unstale":"2024-04-25T10:13:55.160444+0000","last_undegraded":"2024-04-25T10:13:55.160444+0000","last_fullsized":"2024-04-25T10:13:55.160444+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:32.025475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703412+0000","last_change":"2024-04-25T10:13:53.964523+0000","last_active":"2024-04-25T10:13:55.703412+0000","last_peered":"2024-04-25T10:13:55.703412+0000","last_clean":"2024-04-25T10:13:55.703412+0000","last_became_active":"2024-04-25T10:13:53.964140+0000","last_became_peered":"2024-04-25T10:13:53.964140+0000","last_unstale":"2024-04-25T10:13:55.703412+0000","last_undegraded":"2024-04-25T10:13:55.703412+0000","last_fullsized":"2024-04-25T10:13:55.703412+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:41.776706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702642+0000","last_change":"2024-04-25T10:13:53.965556+0000","last_active":"2024-04-25T10:13:55.702642+0000","last_peered":"2024-04-25T10:13:55.702642+0000","last_clean":"2024-04-25T10:13:55.702642+0000","last_became_active":"2024-04-25T10:13:53.965284+0000","last_became_peered":"2024-04-25T10:13:53.965284+0000","last_unstale":"2024-04-25T10:13:55.702642+0000","last_undegraded":"2024-04-25T10:13:55.702642+0000","last_fullsized":"2024-04-25T10:13:55.702642+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:12.544755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702641+0000","last_change":"2024-04-25T10:13:53.965243+0000","last_active":"2024-04-25T10:13:55.702641+0000","last_peered":"2024-04-25T10:13:55.702641+0000","last_clean":"2024-04-25T10:13:55.702641+0000","last_became_active":"2024-04-25T10:13:53.964917+0000","last_became_peered":"2024-04-25T10:13:53.964917+0000","last_unstale":"2024-04-25T10:13:55.702641+0000","last_undegraded":"2024-04-25T10:13:55.702641+0000","last_fullsized":"2024-04-25T10:13:55.702641+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:25.959967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.936302+0000","last_change":"2024-04-25T10:13:53.951530+0000","last_active":"2024-04-25T10:13:54.936302+0000","last_peered":"2024-04-25T10:13:54.936302+0000","last_clean":"2024-04-25T10:13:54.936302+0000","last_became_active":"2024-04-25T10:13:53.951220+0000","last_became_peered":"2024-04-25T10:13:53.951220+0000","last_unstale":"2024-04-25T10:13:54.936302+0000","last_undegraded":"2024-04-25T10:13:54.936302+0000","last_fullsized":"2024-04-25T10:13:54.936302+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:04.728318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939131+0000","last_change":"2024-04-25T10:13:53.953306+0000","last_active":"2024-04-25T10:13:54.939131+0000","last_peered":"2024-04-25T10:13:54.939131+0000","last_clean":"2024-04-25T10:13:54.939131+0000","last_became_active":"2024-04-25T10:13:53.952897+0000","last_became_peered":"2024-04-25T10:13:53.952897+0000","last_unstale":"2024-04-25T10:13:54.939131+0000","last_undegraded":"2024-04-25T10:13:54.939131+0000","last_fullsized":"2024-04-25T10:13:54.939131+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:32.097428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160827+0000","last_change":"2024-04-25T10:13:53.960383+0000","last_active":"2024-04-25T10:13:55.160827+0000","last_peered":"2024-04-25T10:13:55.160827+0000","last_clean":"2024-04-25T10:13:55.160827+0000","last_became_active":"2024-04-25T10:13:53.960120+0000","last_became_peered":"2024-04-25T10:13:53.960120+0000","last_unstale":"2024-04-25T10:13:55.160827+0000","last_undegraded":"2024-04-25T10:13:55.160827+0000","last_fullsized":"2024-04-25T10:13:55.160827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:39.823966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938473+0000","last_change":"2024-04-25T10:13:53.957550+0000","last_active":"2024-04-25T10:13:54.938473+0000","last_peered":"2024-04-25T10:13:54.938473+0000","last_clean":"2024-04-25T10:13:54.938473+0000","last_became_active":"2024-04-25T10:13:53.957267+0000","last_became_peered":"2024-04-25T10:13:53.957267+0000","last_unstale":"2024-04-25T10:13:54.938473+0000","last_undegraded":"2024-04-25T10:13:54.938473+0000","last_fullsized":"2024-04-25T10:13:54.938473+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:05.909155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945288+0000","last_change":"2024-04-25T10:13:53.950224+0000","last_active":"2024-04-25T10:13:54.945288+0000","last_peered":"2024-04-25T10:13:54.945288+0000","last_clean":"2024-04-25T10:13:54.945288+0000","last_became_active":"2024-04-25T10:13:53.949626+0000","last_became_peered":"2024-04-25T10:13:53.949626+0000","last_unstale":"2024-04-25T10:13:54.945288+0000","last_undegraded":"2024-04-25T10:13:54.945288+0000","last_fullsized":"2024-04-25T10:13:54.945288+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.661911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160346+0000","last_change":"2024-04-25T10:13:53.963506+0000","last_active":"2024-04-25T10:13:55.160346+0000","last_peered":"2024-04-25T10:13:55.160346+0000","last_clean":"2024-04-25T10:13:55.160346+0000","last_became_active":"2024-04-25T10:13:53.963245+0000","last_became_peered":"2024-04-25T10:13:53.963245+0000","last_unstale":"2024-04-25T10:13:55.160346+0000","last_undegraded":"2024-04-25T10:13:55.160346+0000","last_fullsized":"2024-04-25T10:13:55.160346+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:13.483720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703188+0000","last_change":"2024-04-25T10:13:53.966377+0000","last_active":"2024-04-25T10:13:55.703188+0000","last_peered":"2024-04-25T10:13:55.703188+0000","last_clean":"2024-04-25T10:13:55.703188+0000","last_became_active":"2024-04-25T10:13:53.966071+0000","last_became_peered":"2024-04-25T10:13:53.966071+0000","last_unstale":"2024-04-25T10:13:55.703188+0000","last_undegraded":"2024-04-25T10:13:55.703188+0000","last_fullsized":"2024-04-25T10:13:55.703188+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:53.911639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938028+0000","last_change":"2024-04-25T10:13:53.958828+0000","last_active":"2024-04-25T10:13:54.938028+0000","last_peered":"2024-04-25T10:13:54.938028+0000","last_clean":"2024-04-25T10:13:54.938028+0000","last_became_active":"2024-04-25T10:13:53.958530+0000","last_became_peered":"2024-04-25T10:13:53.958530+0000","last_unstale":"2024-04-25T10:13:54.938028+0000","last_undegraded":"2024-04-25T10:13:54.938028+0000","last_fullsized":"2024-04-25T10:13:54.938028+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.142499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.936722+0000","last_change":"2024-04-25T10:13:53.951325+0000","last_active":"2024-04-25T10:13:54.936722+0000","last_peered":"2024-04-25T10:13:54.936722+0000","last_clean":"2024-04-25T10:13:54.936722+0000","last_became_active":"2024-04-25T10:13:53.951015+0000","last_became_peered":"2024-04-25T10:13:53.951015+0000","last_unstale":"2024-04-25T10:13:54.936722+0000","last_undegraded":"2024-04-25T10:13:54.936722+0000","last_fullsized":"2024-04-25T10:13:54.936722+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.221512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933258+0000","last_change":"2024-04-25T10:13:53.957838+0000","last_active":"2024-04-25T10:13:54.933258+0000","last_peered":"2024-04-25T10:13:54.933258+0000","last_clean":"2024-04-25T10:13:54.933258+0000","last_became_active":"2024-04-25T10:13:53.949077+0000","last_became_peered":"2024-04-25T10:13:53.949077+0000","last_unstale":"2024-04-25T10:13:54.933258+0000","last_undegraded":"2024-04-25T10:13:54.933258+0000","last_fullsized":"2024-04-25T10:13:54.933258+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:30.373592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"123'4","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938619+0000","last_change":"2024-04-25T10:13:52.952957+0000","last_active":"2024-04-25T10:13:54.938619+0000","last_peered":"2024-04-25T10:13:54.938619+0000","last_clean":"2024-04-25T10:13:54.938619+0000","last_became_active":"2024-04-25T10:13:52.952649+0000","last_became_peered":"2024-04-25T10:13:52.952649+0000","last_unstale":"2024-04-25T10:13:54.938619+0000","last_undegraded":"2024-04-25T10:13:54.938619+0000","last_fullsized":"2024-04-25T10:13:54.938619+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:29.112347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944658+0000","last_change":"2024-04-25T10:13:53.972068+0000","last_active":"2024-04-25T10:13:54.944658+0000","last_peered":"2024-04-25T10:13:54.944658+0000","last_clean":"2024-04-25T10:13:54.944658+0000","last_became_active":"2024-04-25T10:13:53.971723+0000","last_became_peered":"2024-04-25T10:13:53.971723+0000","last_unstale":"2024-04-25T10:13:54.944658+0000","last_undegraded":"2024-04-25T10:13:54.944658+0000","last_fullsized":"2024-04-25T10:13:54.944658+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:59.790942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933566+0000","last_change":"2024-04-25T10:13:52.949136+0000","last_active":"2024-04-25T10:13:54.933566+0000","last_peered":"2024-04-25T10:13:54.933566+0000","last_clean":"2024-04-25T10:13:54.933566+0000","last_became_active":"2024-04-25T10:13:52.945369+0000","last_became_peered":"2024-04-25T10:13:52.945369+0000","last_unstale":"2024-04-25T10:13:54.933566+0000","last_undegraded":"2024-04-25T10:13:54.933566+0000","last_fullsized":"2024-04-25T10:13:54.933566+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.859783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938032+0000","last_change":"2024-04-25T10:13:53.948958+0000","last_active":"2024-04-25T10:13:54.938032+0000","last_peered":"2024-04-25T10:13:54.938032+0000","last_clean":"2024-04-25T10:13:54.938032+0000","last_became_active":"2024-04-25T10:13:53.948652+0000","last_became_peered":"2024-04-25T10:13:53.948652+0000","last_unstale":"2024-04-25T10:13:54.938032+0000","last_undegraded":"2024-04-25T10:13:54.938032+0000","last_fullsized":"2024-04-25T10:13:54.938032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:24.992378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704579+0000","last_change":"2024-04-25T10:13:52.959482+0000","last_active":"2024-04-25T10:13:55.704579+0000","last_peered":"2024-04-25T10:13:55.704579+0000","last_clean":"2024-04-25T10:13:55.704579+0000","last_became_active":"2024-04-25T10:13:52.959158+0000","last_became_peered":"2024-04-25T10:13:52.959158+0000","last_unstale":"2024-04-25T10:13:55.704579+0000","last_undegraded":"2024-04-25T10:13:55.704579+0000","last_fullsized":"2024-04-25T10:13:55.704579+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:15.010081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704853+0000","last_change":"2024-04-25T10:13:53.968149+0000","last_active":"2024-04-25T10:13:55.704853+0000","last_peered":"2024-04-25T10:13:55.704853+0000","last_clean":"2024-04-25T10:13:55.704853+0000","last_became_active":"2024-04-25T10:13:53.967768+0000","last_became_peered":"2024-04-25T10:13:53.967768+0000","last_unstale":"2024-04-25T10:13:55.704853+0000","last_undegraded":"2024-04-25T10:13:55.704853+0000","last_fullsized":"2024-04-25T10:13:55.704853+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.370452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933566+0000","last_change":"2024-04-25T10:13:52.947627+0000","last_active":"2024-04-25T10:13:54.933566+0000","last_peered":"2024-04-25T10:13:54.933566+0000","last_clean":"2024-04-25T10:13:54.933566+0000","last_became_active":"2024-04-25T10:13:52.946659+0000","last_became_peered":"2024-04-25T10:13:52.946659+0000","last_unstale":"2024-04-25T10:13:54.933566+0000","last_undegraded":"2024-04-25T10:13:54.933566+0000","last_fullsized":"2024-04-25T10:13:54.933566+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:20.814067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933045+0000","last_change":"2024-04-25T10:13:53.943413+0000","last_active":"2024-04-25T10:13:54.933045+0000","last_peered":"2024-04-25T10:13:54.933045+0000","last_clean":"2024-04-25T10:13:54.933045+0000","last_became_active":"2024-04-25T10:13:53.942619+0000","last_became_peered":"2024-04-25T10:13:53.942619+0000","last_unstale":"2024-04-25T10:13:54.933045+0000","last_undegraded":"2024-04-25T10:13:54.933045+0000","last_fullsized":"2024-04-25T10:13:54.933045+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:42.566879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702153+0000","last_change":"2024-04-25T10:13:52.951325+0000","last_active":"2024-04-25T10:13:55.702153+0000","last_peered":"2024-04-25T10:13:55.702153+0000","last_clean":"2024-04-25T10:13:55.702153+0000","last_became_active":"2024-04-25T10:13:52.950036+0000","last_became_peered":"2024-04-25T10:13:52.950036+0000","last_unstale":"2024-04-25T10:13:55.702153+0000","last_undegraded":"2024-04-25T10:13:55.702153+0000","last_fullsized":"2024-04-25T10:13:55.702153+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:28.476337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702616+0000","last_change":"2024-04-25T10:13:53.946993+0000","last_active":"2024-04-25T10:13:55.702616+0000","last_peered":"2024-04-25T10:13:55.702616+0000","last_clean":"2024-04-25T10:13:55.702616+0000","last_became_active":"2024-04-25T10:13:53.946306+0000","last_became_peered":"2024-04-25T10:13:53.946306+0000","last_unstale":"2024-04-25T10:13:55.702616+0000","last_undegraded":"2024-04-25T10:13:55.702616+0000","last_fullsized":"2024-04-25T10:13:55.702616+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:49.929009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943982+0000","last_change":"2024-04-25T10:13:52.944454+0000","last_active":"2024-04-25T10:13:54.943982+0000","last_peered":"2024-04-25T10:13:54.943982+0000","last_clean":"2024-04-25T10:13:54.943982+0000","last_became_active":"2024-04-25T10:13:52.944146+0000","last_became_peered":"2024-04-25T10:13:52.944146+0000","last_unstale":"2024-04-25T10:13:54.943982+0000","last_undegraded":"2024-04-25T10:13:54.943982+0000","last_fullsized":"2024-04-25T10:13:54.943982+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:49.745429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703585+0000","last_change":"2024-04-25T10:13:53.973096+0000","last_active":"2024-04-25T10:13:55.703585+0000","last_peered":"2024-04-25T10:13:55.703585+0000","last_clean":"2024-04-25T10:13:55.703585+0000","last_became_active":"2024-04-25T10:13:53.971075+0000","last_became_peered":"2024-04-25T10:13:53.971075+0000","last_unstale":"2024-04-25T10:13:55.703585+0000","last_undegraded":"2024-04-25T10:13:55.703585+0000","last_fullsized":"2024-04-25T10:13:55.703585+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.370452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.937796+0000","last_change":"2024-04-25T10:13:52.949231+0000","last_active":"2024-04-25T10:13:54.937796+0000","last_peered":"2024-04-25T10:13:54.937796+0000","last_clean":"2024-04-25T10:13:54.937796+0000","last_became_active":"2024-04-25T10:13:52.948931+0000","last_became_peered":"2024-04-25T10:13:52.948931+0000","last_unstale":"2024-04-25T10:13:54.937796+0000","last_undegraded":"2024-04-25T10:13:54.937796+0000","last_fullsized":"2024-04-25T10:13:54.937796+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:53.684783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933013+0000","last_change":"2024-04-25T10:13:53.965985+0000","last_active":"2024-04-25T10:13:54.933013+0000","last_peered":"2024-04-25T10:13:54.933013+0000","last_clean":"2024-04-25T10:13:54.933013+0000","last_became_active":"2024-04-25T10:13:53.964930+0000","last_became_peered":"2024-04-25T10:13:53.964930+0000","last_unstale":"2024-04-25T10:13:54.933013+0000","last_undegraded":"2024-04-25T10:13:54.933013+0000","last_fullsized":"2024-04-25T10:13:54.933013+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:12.544755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943617+0000","last_change":"2024-04-25T10:13:52.951999+0000","last_active":"2024-04-25T10:13:54.943617+0000","last_peered":"2024-04-25T10:13:54.943617+0000","last_clean":"2024-04-25T10:13:54.943617+0000","last_became_active":"2024-04-25T10:13:52.951631+0000","last_became_peered":"2024-04-25T10:13:52.951631+0000","last_unstale":"2024-04-25T10:13:54.943617+0000","last_undegraded":"2024-04-25T10:13:54.943617+0000","last_fullsized":"2024-04-25T10:13:54.943617+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:49.401614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.937861+0000","last_change":"2024-04-25T10:13:53.951524+0000","last_active":"2024-04-25T10:13:54.937861+0000","last_peered":"2024-04-25T10:13:54.937861+0000","last_clean":"2024-04-25T10:13:54.937861+0000","last_became_active":"2024-04-25T10:13:53.951209+0000","last_became_peered":"2024-04-25T10:13:53.951209+0000","last_unstale":"2024-04-25T10:13:54.937861+0000","last_undegraded":"2024-04-25T10:13:54.937861+0000","last_fullsized":"2024-04-25T10:13:54.937861+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:42.566879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939045+0000","last_change":"2024-04-25T10:13:52.956823+0000","last_active":"2024-04-25T10:13:54.939045+0000","last_peered":"2024-04-25T10:13:54.939045+0000","last_clean":"2024-04-25T10:13:54.939045+0000","last_became_active":"2024-04-25T10:13:52.955291+0000","last_became_peered":"2024-04-25T10:13:52.955291+0000","last_unstale":"2024-04-25T10:13:54.939045+0000","last_undegraded":"2024-04-25T10:13:54.939045+0000","last_fullsized":"2024-04-25T10:13:54.939045+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:03.727616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938405+0000","last_change":"2024-04-25T10:13:53.972282+0000","last_active":"2024-04-25T10:13:54.938405+0000","last_peered":"2024-04-25T10:13:54.938405+0000","last_clean":"2024-04-25T10:13:54.938405+0000","last_became_active":"2024-04-25T10:13:53.971854+0000","last_became_peered":"2024-04-25T10:13:53.971854+0000","last_unstale":"2024-04-25T10:13:54.938405+0000","last_undegraded":"2024-04-25T10:13:54.938405+0000","last_fullsized":"2024-04-25T10:13:54.938405+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:27.356559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938863+0000","last_change":"2024-04-25T10:13:52.946859+0000","last_active":"2024-04-25T10:13:54.938863+0000","last_peered":"2024-04-25T10:13:54.938863+0000","last_clean":"2024-04-25T10:13:54.938863+0000","last_became_active":"2024-04-25T10:13:52.946562+0000","last_became_peered":"2024-04-25T10:13:52.946562+0000","last_unstale":"2024-04-25T10:13:54.938863+0000","last_undegraded":"2024-04-25T10:13:54.938863+0000","last_fullsized":"2024-04-25T10:13:54.938863+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:37.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704412+0000","last_change":"2024-04-25T10:13:53.966753+0000","last_active":"2024-04-25T10:13:55.704412+0000","last_peered":"2024-04-25T10:13:55.704412+0000","last_clean":"2024-04-25T10:13:55.704412+0000","last_became_active":"2024-04-25T10:13:53.966490+0000","last_became_peered":"2024-04-25T10:13:53.966490+0000","last_unstale":"2024-04-25T10:13:55.704412+0000","last_undegraded":"2024-04-25T10:13:55.704412+0000","last_fullsized":"2024-04-25T10:13:55.704412+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:57.777532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"123'3","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938599+0000","last_change":"2024-04-25T10:13:52.953111+0000","last_active":"2024-04-25T10:13:54.938599+0000","last_peered":"2024-04-25T10:13:54.938599+0000","last_clean":"2024-04-25T10:13:54.938599+0000","last_became_active":"2024-04-25T10:13:52.952801+0000","last_became_peered":"2024-04-25T10:13:52.952801+0000","last_unstale":"2024-04-25T10:13:54.938599+0000","last_undegraded":"2024-04-25T10:13:54.938599+0000","last_fullsized":"2024-04-25T10:13:54.938599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:48.911762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939414+0000","last_change":"2024-04-25T10:13:53.963311+0000","last_active":"2024-04-25T10:13:54.939414+0000","last_peered":"2024-04-25T10:13:54.939414+0000","last_clean":"2024-04-25T10:13:54.939414+0000","last_became_active":"2024-04-25T10:13:53.962886+0000","last_became_peered":"2024-04-25T10:13:53.962886+0000","last_unstale":"2024-04-25T10:13:54.939414+0000","last_undegraded":"2024-04-25T10:13:54.939414+0000","last_fullsized":"2024-04-25T10:13:54.939414+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:59.838400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"123'6","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703122+0000","last_change":"2024-04-25T10:13:52.949538+0000","last_active":"2024-04-25T10:13:55.703122+0000","last_peered":"2024-04-25T10:13:55.703122+0000","last_clean":"2024-04-25T10:13:55.703122+0000","last_became_active":"2024-04-25T10:13:52.949263+0000","last_became_peered":"2024-04-25T10:13:52.949263+0000","last_unstale":"2024-04-25T10:13:55.703122+0000","last_undegraded":"2024-04-25T10:13:55.703122+0000","last_fullsized":"2024-04-25T10:13:55.703122+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:19.361315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938031+0000","last_change":"2024-04-25T10:13:53.965432+0000","last_active":"2024-04-25T10:13:54.938031+0000","last_peered":"2024-04-25T10:13:54.938031+0000","last_clean":"2024-04-25T10:13:54.938031+0000","last_became_active":"2024-04-25T10:13:53.964752+0000","last_became_peered":"2024-04-25T10:13:53.964752+0000","last_unstale":"2024-04-25T10:13:54.938031+0000","last_undegraded":"2024-04-25T10:13:54.938031+0000","last_fullsized":"2024-04-25T10:13:54.938031+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:27.688942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703621+0000","last_change":"2024-04-25T10:13:52.953619+0000","last_active":"2024-04-25T10:13:55.703621+0000","last_peered":"2024-04-25T10:13:55.703621+0000","last_clean":"2024-04-25T10:13:55.703621+0000","last_became_active":"2024-04-25T10:13:52.952867+0000","last_became_peered":"2024-04-25T10:13:52.952867+0000","last_unstale":"2024-04-25T10:13:55.703621+0000","last_undegraded":"2024-04-25T10:13:55.703621+0000","last_fullsized":"2024-04-25T10:13:55.703621+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:15.010081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933081+0000","last_change":"2024-04-25T10:13:53.950372+0000","last_active":"2024-04-25T10:13:54.933081+0000","last_peered":"2024-04-25T10:13:54.933081+0000","last_clean":"2024-04-25T10:13:54.933081+0000","last_became_active":"2024-04-25T10:13:53.948847+0000","last_became_peered":"2024-04-25T10:13:53.948847+0000","last_unstale":"2024-04-25T10:13:54.933081+0000","last_undegraded":"2024-04-25T10:13:54.933081+0000","last_fullsized":"2024-04-25T10:13:54.933081+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:27.688942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703821+0000","last_change":"2024-04-25T10:13:52.951277+0000","last_active":"2024-04-25T10:13:55.703821+0000","last_peered":"2024-04-25T10:13:55.703821+0000","last_clean":"2024-04-25T10:13:55.703821+0000","last_became_active":"2024-04-25T10:13:52.949938+0000","last_became_peered":"2024-04-25T10:13:52.949938+0000","last_unstale":"2024-04-25T10:13:55.703821+0000","last_undegraded":"2024-04-25T10:13:55.703821+0000","last_fullsized":"2024-04-25T10:13:55.703821+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.233132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160238+0000","last_change":"2024-04-25T10:13:53.955064+0000","last_active":"2024-04-25T10:13:55.160238+0000","last_peered":"2024-04-25T10:13:55.160238+0000","last_clean":"2024-04-25T10:13:55.160238+0000","last_became_active":"2024-04-25T10:13:53.954769+0000","last_became_peered":"2024-04-25T10:13:53.954769+0000","last_unstale":"2024-04-25T10:13:55.160238+0000","last_undegraded":"2024-04-25T10:13:55.160238+0000","last_fullsized":"2024-04-25T10:13:55.160238+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:49.129447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703830+0000","last_change":"2024-04-25T10:13:52.945225+0000","last_active":"2024-04-25T10:13:55.703830+0000","last_peered":"2024-04-25T10:13:55.703830+0000","last_clean":"2024-04-25T10:13:55.703830+0000","last_became_active":"2024-04-25T10:13:52.944298+0000","last_became_peered":"2024-04-25T10:13:52.944298+0000","last_unstale":"2024-04-25T10:13:55.703830+0000","last_undegraded":"2024-04-25T10:13:55.703830+0000","last_fullsized":"2024-04-25T10:13:55.703830+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:15.136996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938524+0000","last_change":"2024-04-25T10:13:53.956113+0000","last_active":"2024-04-25T10:13:54.938524+0000","last_peered":"2024-04-25T10:13:54.938524+0000","last_clean":"2024-04-25T10:13:54.938524+0000","last_became_active":"2024-04-25T10:13:53.955756+0000","last_became_peered":"2024-04-25T10:13:53.955756+0000","last_unstale":"2024-04-25T10:13:54.938524+0000","last_undegraded":"2024-04-25T10:13:54.938524+0000","last_fullsized":"2024-04-25T10:13:54.938524+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:16.867933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933780+0000","last_change":"2024-04-25T10:13:52.954949+0000","last_active":"2024-04-25T10:13:54.933780+0000","last_peered":"2024-04-25T10:13:54.933780+0000","last_clean":"2024-04-25T10:13:54.933780+0000","last_became_active":"2024-04-25T10:13:52.954523+0000","last_became_peered":"2024-04-25T10:13:52.954523+0000","last_unstale":"2024-04-25T10:13:54.933780+0000","last_undegraded":"2024-04-25T10:13:54.933780+0000","last_fullsized":"2024-04-25T10:13:54.933780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:49.620494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703355+0000","last_change":"2024-04-25T10:13:53.964942+0000","last_active":"2024-04-25T10:13:55.703355+0000","last_peered":"2024-04-25T10:13:55.703355+0000","last_clean":"2024-04-25T10:13:55.703355+0000","last_became_active":"2024-04-25T10:13:53.964657+0000","last_became_peered":"2024-04-25T10:13:53.964657+0000","last_unstale":"2024-04-25T10:13:55.703355+0000","last_undegraded":"2024-04-25T10:13:55.703355+0000","last_fullsized":"2024-04-25T10:13:55.703355+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.074055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160957+0000","last_change":"2024-04-25T10:13:52.942066+0000","last_active":"2024-04-25T10:13:55.160957+0000","last_peered":"2024-04-25T10:13:55.160957+0000","last_clean":"2024-04-25T10:13:55.160957+0000","last_became_active":"2024-04-25T10:13:52.941765+0000","last_became_peered":"2024-04-25T10:13:52.941765+0000","last_unstale":"2024-04-25T10:13:55.160957+0000","last_undegraded":"2024-04-25T10:13:55.160957+0000","last_fullsized":"2024-04-25T10:13:55.160957+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:16.814033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945286+0000","last_change":"2024-04-25T10:13:53.954544+0000","last_active":"2024-04-25T10:13:54.945286+0000","last_peered":"2024-04-25T10:13:54.945286+0000","last_clean":"2024-04-25T10:13:54.945286+0000","last_became_active":"2024-04-25T10:13:53.954282+0000","last_became_peered":"2024-04-25T10:13:53.954282+0000","last_unstale":"2024-04-25T10:13:54.945286+0000","last_undegraded":"2024-04-25T10:13:54.945286+0000","last_fullsized":"2024-04-25T10:13:54.945286+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:13.800570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704890+0000","last_change":"2024-04-25T10:13:52.959185+0000","last_active":"2024-04-25T10:13:55.704890+0000","last_peered":"2024-04-25T10:13:55.704890+0000","last_clean":"2024-04-25T10:13:55.704890+0000","last_became_active":"2024-04-25T10:13:52.958904+0000","last_became_peered":"2024-04-25T10:13:52.958904+0000","last_unstale":"2024-04-25T10:13:55.704890+0000","last_undegraded":"2024-04-25T10:13:55.704890+0000","last_fullsized":"2024-04-25T10:13:55.704890+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:26.993885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939231+0000","last_change":"2024-04-25T10:13:53.967716+0000","last_active":"2024-04-25T10:13:54.939231+0000","last_peered":"2024-04-25T10:13:54.939231+0000","last_clean":"2024-04-25T10:13:54.939231+0000","last_became_active":"2024-04-25T10:13:53.967029+0000","last_became_peered":"2024-04-25T10:13:53.967029+0000","last_unstale":"2024-04-25T10:13:54.939231+0000","last_undegraded":"2024-04-25T10:13:54.939231+0000","last_fullsized":"2024-04-25T10:13:54.939231+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:03.943579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704039+0000","last_change":"2024-04-25T10:13:52.958671+0000","last_active":"2024-04-25T10:13:55.704039+0000","last_peered":"2024-04-25T10:13:55.704039+0000","last_clean":"2024-04-25T10:13:55.704039+0000","last_became_active":"2024-04-25T10:13:52.958352+0000","last_became_peered":"2024-04-25T10:13:52.958352+0000","last_unstale":"2024-04-25T10:13:55.704039+0000","last_undegraded":"2024-04-25T10:13:55.704039+0000","last_fullsized":"2024-04-25T10:13:55.704039+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:31.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943684+0000","last_change":"2024-04-25T10:13:53.947883+0000","last_active":"2024-04-25T10:13:54.943684+0000","last_peered":"2024-04-25T10:13:54.943684+0000","last_clean":"2024-04-25T10:13:54.943684+0000","last_became_active":"2024-04-25T10:13:53.940997+0000","last_became_peered":"2024-04-25T10:13:53.940997+0000","last_unstale":"2024-04-25T10:13:54.943684+0000","last_undegraded":"2024-04-25T10:13:54.943684+0000","last_fullsized":"2024-04-25T10:13:54.943684+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:29.369969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944071+0000","last_change":"2024-04-25T10:13:52.952949+0000","last_active":"2024-04-25T10:13:54.944071+0000","last_peered":"2024-04-25T10:13:54.944071+0000","last_clean":"2024-04-25T10:13:54.944071+0000","last_became_active":"2024-04-25T10:13:52.952656+0000","last_became_peered":"2024-04-25T10:13:52.952656+0000","last_unstale":"2024-04-25T10:13:54.944071+0000","last_undegraded":"2024-04-25T10:13:54.944071+0000","last_fullsized":"2024-04-25T10:13:54.944071+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:22.863698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702881+0000","last_change":"2024-04-25T10:13:53.961093+0000","last_active":"2024-04-25T10:13:55.702881+0000","last_peered":"2024-04-25T10:13:55.702881+0000","last_clean":"2024-04-25T10:13:55.702881+0000","last_became_active":"2024-04-25T10:13:53.960111+0000","last_became_peered":"2024-04-25T10:13:53.960111+0000","last_unstale":"2024-04-25T10:13:55.702881+0000","last_undegraded":"2024-04-25T10:13:55.702881+0000","last_fullsized":"2024-04-25T10:13:55.702881+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:49.121305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702906+0000","last_change":"2024-04-25T10:13:52.956176+0000","last_active":"2024-04-25T10:13:55.702906+0000","last_peered":"2024-04-25T10:13:55.702906+0000","last_clean":"2024-04-25T10:13:55.702906+0000","last_became_active":"2024-04-25T10:13:52.955794+0000","last_became_peered":"2024-04-25T10:13:52.955794+0000","last_unstale":"2024-04-25T10:13:55.702906+0000","last_undegraded":"2024-04-25T10:13:55.702906+0000","last_fullsized":"2024-04-25T10:13:55.702906+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:19.361315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703659+0000","last_change":"2024-04-25T10:13:53.967211+0000","last_active":"2024-04-25T10:13:55.703659+0000","last_peered":"2024-04-25T10:13:55.703659+0000","last_clean":"2024-04-25T10:13:55.703659+0000","last_became_active":"2024-04-25T10:13:53.965126+0000","last_became_peered":"2024-04-25T10:13:53.965126+0000","last_unstale":"2024-04-25T10:13:55.703659+0000","last_undegraded":"2024-04-25T10:13:55.703659+0000","last_fullsized":"2024-04-25T10:13:55.703659+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:23.327516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703583+0000","last_change":"2024-04-25T10:13:52.954482+0000","last_active":"2024-04-25T10:13:55.703583+0000","last_peered":"2024-04-25T10:13:55.703583+0000","last_clean":"2024-04-25T10:13:55.703583+0000","last_became_active":"2024-04-25T10:13:52.954195+0000","last_became_peered":"2024-04-25T10:13:52.954195+0000","last_unstale":"2024-04-25T10:13:55.703583+0000","last_undegraded":"2024-04-25T10:13:55.703583+0000","last_fullsized":"2024-04-25T10:13:55.703583+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.096531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.937509+0000","last_change":"2024-04-25T10:13:53.958857+0000","last_active":"2024-04-25T10:13:54.937509+0000","last_peered":"2024-04-25T10:13:54.937509+0000","last_clean":"2024-04-25T10:13:54.937509+0000","last_became_active":"2024-04-25T10:13:53.958579+0000","last_became_peered":"2024-04-25T10:13:53.958579+0000","last_unstale":"2024-04-25T10:13:54.937509+0000","last_undegraded":"2024-04-25T10:13:54.937509+0000","last_fullsized":"2024-04-25T10:13:54.937509+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:36.863606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.701549+0000","last_change":"2024-04-25T10:13:52.959587+0000","last_active":"2024-04-25T10:13:55.701549+0000","last_peered":"2024-04-25T10:13:55.701549+0000","last_clean":"2024-04-25T10:13:55.701549+0000","last_became_active":"2024-04-25T10:13:52.958324+0000","last_became_peered":"2024-04-25T10:13:52.958324+0000","last_unstale":"2024-04-25T10:13:55.701549+0000","last_undegraded":"2024-04-25T10:13:55.701549+0000","last_fullsized":"2024-04-25T10:13:55.701549+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:58.821153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945338+0000","last_change":"2024-04-25T10:13:53.951670+0000","last_active":"2024-04-25T10:13:54.945338+0000","last_peered":"2024-04-25T10:13:54.945338+0000","last_clean":"2024-04-25T10:13:54.945338+0000","last_became_active":"2024-04-25T10:13:53.946757+0000","last_became_peered":"2024-04-25T10:13:53.946757+0000","last_unstale":"2024-04-25T10:13:54.945338+0000","last_undegraded":"2024-04-25T10:13:54.945338+0000","last_fullsized":"2024-04-25T10:13:54.945338+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.326360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944997+0000","last_change":"2024-04-25T10:13:52.944932+0000","last_active":"2024-04-25T10:13:54.944997+0000","last_peered":"2024-04-25T10:13:54.944997+0000","last_clean":"2024-04-25T10:13:54.944997+0000","last_became_active":"2024-04-25T10:13:52.943685+0000","last_became_peered":"2024-04-25T10:13:52.943685+0000","last_unstale":"2024-04-25T10:13:54.944997+0000","last_undegraded":"2024-04-25T10:13:54.944997+0000","last_fullsized":"2024-04-25T10:13:54.944997+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:56.619463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933237+0000","last_change":"2024-04-25T10:13:53.964895+0000","last_active":"2024-04-25T10:13:54.933237+0000","last_peered":"2024-04-25T10:13:54.933237+0000","last_clean":"2024-04-25T10:13:54.933237+0000","last_became_active":"2024-04-25T10:13:53.950760+0000","last_became_peered":"2024-04-25T10:13:53.950760+0000","last_unstale":"2024-04-25T10:13:54.933237+0000","last_undegraded":"2024-04-25T10:13:54.933237+0000","last_fullsized":"2024-04-25T10:13:54.933237+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:05.909155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"124'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938402+0000","last_change":"2024-04-25T10:13:52.956533+0000","last_active":"2024-04-25T10:13:54.938402+0000","last_peered":"2024-04-25T10:13:54.938402+0000","last_clean":"2024-04-25T10:13:54.938402+0000","last_became_active":"2024-04-25T10:13:52.956203+0000","last_became_peered":"2024-04-25T10:13:52.956203+0000","last_unstale":"2024-04-25T10:13:54.938402+0000","last_undegraded":"2024-04-25T10:13:54.938402+0000","last_fullsized":"2024-04-25T10:13:54.938402+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:45.869471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704782+0000","last_change":"2024-04-25T10:13:53.956469+0000","last_active":"2024-04-25T10:13:55.704782+0000","last_peered":"2024-04-25T10:13:55.704782+0000","last_clean":"2024-04-25T10:13:55.704782+0000","last_became_active":"2024-04-25T10:13:53.955971+0000","last_became_peered":"2024-04-25T10:13:53.955971+0000","last_unstale":"2024-04-25T10:13:55.704782+0000","last_undegraded":"2024-04-25T10:13:55.704782+0000","last_fullsized":"2024-04-25T10:13:55.704782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.634240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938563+0000","last_change":"2024-04-25T10:13:52.958822+0000","last_active":"2024-04-25T10:13:54.938563+0000","last_peered":"2024-04-25T10:13:54.938563+0000","last_clean":"2024-04-25T10:13:54.938563+0000","last_became_active":"2024-04-25T10:13:52.957856+0000","last_became_peered":"2024-04-25T10:13:52.957856+0000","last_unstale":"2024-04-25T10:13:54.938563+0000","last_undegraded":"2024-04-25T10:13:54.938563+0000","last_fullsized":"2024-04-25T10:13:54.938563+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:47.032097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.161291+0000","last_change":"2024-04-25T10:13:53.961947+0000","last_active":"2024-04-25T10:13:55.161291+0000","last_peered":"2024-04-25T10:13:55.161291+0000","last_clean":"2024-04-25T10:13:55.161291+0000","last_became_active":"2024-04-25T10:13:53.961028+0000","last_became_peered":"2024-04-25T10:13:53.961028+0000","last_unstale":"2024-04-25T10:13:55.161291+0000","last_undegraded":"2024-04-25T10:13:55.161291+0000","last_fullsized":"2024-04-25T10:13:55.161291+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:24.518057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945424+0000","last_change":"2024-04-25T10:13:52.950621+0000","last_active":"2024-04-25T10:13:54.945424+0000","last_peered":"2024-04-25T10:13:54.945424+0000","last_clean":"2024-04-25T10:13:54.945424+0000","last_became_active":"2024-04-25T10:13:52.950352+0000","last_became_peered":"2024-04-25T10:13:52.950352+0000","last_unstale":"2024-04-25T10:13:54.945424+0000","last_undegraded":"2024-04-25T10:13:54.945424+0000","last_fullsized":"2024-04-25T10:13:54.945424+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:09.806566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943559+0000","last_change":"2024-04-25T10:13:53.939586+0000","last_active":"2024-04-25T10:13:54.943559+0000","last_peered":"2024-04-25T10:13:54.943559+0000","last_clean":"2024-04-25T10:13:54.943559+0000","last_became_active":"2024-04-25T10:13:53.939317+0000","last_became_peered":"2024-04-25T10:13:53.939317+0000","last_unstale":"2024-04-25T10:13:54.943559+0000","last_undegraded":"2024-04-25T10:13:54.943559+0000","last_fullsized":"2024-04-25T10:13:54.943559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:25.479339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702561+0000","last_change":"2024-04-25T10:13:52.948064+0000","last_active":"2024-04-25T10:13:55.702561+0000","last_peered":"2024-04-25T10:13:55.702561+0000","last_clean":"2024-04-25T10:13:55.702561+0000","last_became_active":"2024-04-25T10:13:52.947756+0000","last_became_peered":"2024-04-25T10:13:52.947756+0000","last_unstale":"2024-04-25T10:13:55.702561+0000","last_undegraded":"2024-04-25T10:13:55.702561+0000","last_fullsized":"2024-04-25T10:13:55.702561+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:49.620494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703352+0000","last_change":"2024-04-25T10:13:53.961501+0000","last_active":"2024-04-25T10:13:55.703352+0000","last_peered":"2024-04-25T10:13:55.703352+0000","last_clean":"2024-04-25T10:13:55.703352+0000","last_became_active":"2024-04-25T10:13:53.961045+0000","last_became_peered":"2024-04-25T10:13:53.961045+0000","last_unstale":"2024-04-25T10:13:55.703352+0000","last_undegraded":"2024-04-25T10:13:55.703352+0000","last_fullsized":"2024-04-25T10:13:55.703352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.634240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702370+0000","last_change":"2024-04-25T10:13:52.948077+0000","last_active":"2024-04-25T10:13:55.702370+0000","last_peered":"2024-04-25T10:13:55.702370+0000","last_clean":"2024-04-25T10:13:55.702370+0000","last_became_active":"2024-04-25T10:13:52.947666+0000","last_became_peered":"2024-04-25T10:13:52.947666+0000","last_unstale":"2024-04-25T10:13:55.702370+0000","last_undegraded":"2024-04-25T10:13:55.702370+0000","last_fullsized":"2024-04-25T10:13:55.702370+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.859783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702590+0000","last_change":"2024-04-25T10:13:53.956397+0000","last_active":"2024-04-25T10:13:55.702590+0000","last_peered":"2024-04-25T10:13:55.702590+0000","last_clean":"2024-04-25T10:13:55.702590+0000","last_became_active":"2024-04-25T10:13:53.950697+0000","last_became_peered":"2024-04-25T10:13:53.950697+0000","last_unstale":"2024-04-25T10:13:55.702590+0000","last_undegraded":"2024-04-25T10:13:55.702590+0000","last_fullsized":"2024-04-25T10:13:55.702590+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:11.654276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943571+0000","last_change":"2024-04-25T10:13:52.947940+0000","last_active":"2024-04-25T10:13:54.943571+0000","last_peered":"2024-04-25T10:13:54.943571+0000","last_clean":"2024-04-25T10:13:54.943571+0000","last_became_active":"2024-04-25T10:13:52.947672+0000","last_became_peered":"2024-04-25T10:13:52.947672+0000","last_unstale":"2024-04-25T10:13:54.943571+0000","last_undegraded":"2024-04-25T10:13:54.943571+0000","last_fullsized":"2024-04-25T10:13:54.943571+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:35.167270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933175+0000","last_change":"2024-04-25T10:13:53.956924+0000","last_active":"2024-04-25T10:13:54.933175+0000","last_peered":"2024-04-25T10:13:54.933175+0000","last_clean":"2024-04-25T10:13:54.933175+0000","last_became_active":"2024-04-25T10:13:53.956597+0000","last_became_peered":"2024-04-25T10:13:53.956597+0000","last_unstale":"2024-04-25T10:13:54.933175+0000","last_undegraded":"2024-04-25T10:13:54.933175+0000","last_fullsized":"2024-04-25T10:13:54.933175+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.074055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.701604+0000","last_change":"2024-04-25T10:13:52.959362+0000","last_active":"2024-04-25T10:13:55.701604+0000","last_peered":"2024-04-25T10:13:55.701604+0000","last_clean":"2024-04-25T10:13:55.701604+0000","last_became_active":"2024-04-25T10:13:52.957216+0000","last_became_peered":"2024-04-25T10:13:52.957216+0000","last_unstale":"2024-04-25T10:13:55.701604+0000","last_undegraded":"2024-04-25T10:13:55.701604+0000","last_fullsized":"2024-04-25T10:13:55.701604+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:28.811360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703810+0000","last_change":"2024-04-25T10:13:53.962969+0000","last_active":"2024-04-25T10:13:55.703810+0000","last_peered":"2024-04-25T10:13:55.703810+0000","last_clean":"2024-04-25T10:13:55.703810+0000","last_became_active":"2024-04-25T10:13:53.962553+0000","last_became_peered":"2024-04-25T10:13:53.962553+0000","last_unstale":"2024-04-25T10:13:55.703810+0000","last_undegraded":"2024-04-25T10:13:55.703810+0000","last_fullsized":"2024-04-25T10:13:55.703810+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:58.988897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"123'8","reported_seq":22,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702757+0000","last_change":"2024-04-25T10:13:52.953569+0000","last_active":"2024-04-25T10:13:55.702757+0000","last_peered":"2024-04-25T10:13:55.702757+0000","last_clean":"2024-04-25T10:13:55.702757+0000","last_became_active":"2024-04-25T10:13:52.953276+0000","last_became_peered":"2024-04-25T10:13:52.953276+0000","last_unstale":"2024-04-25T10:13:55.702757+0000","last_undegraded":"2024-04-25T10:13:55.702757+0000","last_fullsized":"2024-04-25T10:13:55.702757+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:13.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":368,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702090+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:13:55.702090+0000","last_peered":"2024-04-25T10:13:55.702090+0000","last_clean":"2024-04-25T10:13:55.702090+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:13:55.702090+0000","last_undegraded":"2024-04-25T10:13:55.702090+0000","last_fullsized":"2024-04-25T10:13:55.702090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":2154496,"data_stored":1315205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54900,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316940,"statfs":{"total":96636764160,"available":96580546560,"internally_reserved":0,"allocated":1548288,"data_stored":724643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50744,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321096,"statfs":{"total":96636764160,"available":96584802304,"internally_reserved":0,"allocated":1552384,"data_stored":724221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59068,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1556480,"data_stored":725673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59080,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312760,"statfs":{"total":96636764160,"available":96576266240,"internally_reserved":0,"allocated":1560576,"data_stored":725707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54900,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316940,"statfs":{"total":96636764160,"available":96580546560,"internally_reserved":0,"allocated":1544192,"data_stored":724061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54904,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316936,"statfs":{"total":96636764160,"available":96580542464,"internally_reserved":0,"allocated":1548288,"data_stored":724083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59640,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312200,"statfs":{"total":96636764160,"available":96575692800,"internally_reserved":0,"allocated":2142208,"data_stored":1314451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:58.569 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:13:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.803+0000 7f4559400640 1 Processor -- start 2024-04-25T10:13:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.803+0000 7f4559400640 1 -- start start 2024-04-25T10:13:58.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.804+0000 7f4559400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4554143e60 0x7f455406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.804+0000 7f4559400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4559400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455415bb50 con 0x7f45540cc840 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4559400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4554144240 con 0x7f4554143e60 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4559400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455415e220 con 0x7f455406e6a0 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4553e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4554143e60 0x7f455406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4553400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4558a00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f45540cc840 0x7f4554070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:44268/0 (socket says 172.21.15.165:44268) 2024-04-25T10:13:58.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4558a00640 1 -- 172.21.15.165:0/2501015482 learned_addr learned my addr 172.21.15.165:0/2501015482 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:58.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.805+0000 7f4553e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4554143e60 0x7f455406e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:53646/0 (socket says 172.21.15.165:53646) 2024-04-25T10:13:58.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.806+0000 7f4553e00640 1 -- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 msgr2=0x7f45540cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:58.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.806+0000 7f4553e00640 1 --2- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.806+0000 7f4553e00640 1 -- 172.21.15.165:0/2501015482 >> v1:172.21.15.172:6789/0 conn(0x7f45540cc840 legacy=0x7f4554070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.806+0000 7f4553e00640 1 -- 172.21.15.165:0/2501015482 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455415e500 con 0x7f4554143e60 2024-04-25T10:13:58.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.807+0000 7f4553e00640 1 --2- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f455406e160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402c390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fdefa8c20aa62656 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.808+0000 7f4552a00640 1 -- 172.21.15.165:0/2501015482 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544039070 con 0x7f4554143e60 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.808+0000 7f4552a00640 1 -- 172.21.15.165:0/2501015482 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4544034640 con 0x7f4554143e60 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.808+0000 7f4552a00640 1 -- 172.21.15.165:0/2501015482 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4544034a10 con 0x7f4554143e60 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 -- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 msgr2=0x7f455406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 --2- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f455406e160 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f45440097a0 tx=0x7f454402c390 comp rx=0 tx=0).stop 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 -- 172.21.15.165:0/2501015482 shutdown_connections 2024-04-25T10:13:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 --2- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 --2- 172.21.15.165:0/2501015482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f455406e160 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 -- 172.21.15.165:0/2501015482 >> 172.21.15.165:0/2501015482 conn(0x7f455406abe0 msgr2=0x7f455406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 -- 172.21.15.165:0/2501015482 shutdown_connections 2024-04-25T10:13:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.809+0000 7f4559400640 1 -- 172.21.15.165:0/2501015482 wait complete. 2024-04-25T10:13:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.811+0000 7f4559400640 1 Processor -- start 2024-04-25T10:13:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.811+0000 7f4559400640 1 -- start start 2024-04-25T10:13:58.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4559400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4559400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 0x7f45541f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4559400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f4554140fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4559400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45540cccd0 con 0x7f45540cc840 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4559400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455415eac0 con 0x7f4554143e60 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4559400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45540cce50 con 0x7f455406e6a0 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 0x7f45541f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 0x7f45541f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38160/0 (socket says 172.21.15.165:38160) 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 -- 172.21.15.165:0/3248931912 learned_addr learned my addr 172.21.15.165:0/3248931912 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4558a00640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f4554140fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 -- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 msgr2=0x7f45540cbfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 -- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 msgr2=0x7f4554140fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:58.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f4554140fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.812+0000 7f4553400640 1 -- 172.21.15.165:0/3248931912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45540accb0 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.813+0000 7f4553400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 0x7f45541f45b0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f453c00e9b0 tx=0x7f453c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.813+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c017070 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.813+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453c00c9a0 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.813+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45540acf90 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.813+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453c00cd90 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.813+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45540ad550 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.814+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4554073990 con 0x7f45540cc840 2024-04-25T10:13:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.815+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f453c01a070 con 0x7f45540cc840 2024-04-25T10:13:58.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.816+0000 7f454be00640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f453006b4d0 0x7f453006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:58.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.816+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f453c08a440 con 0x7f45540cc840 2024-04-25T10:13:58.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.817+0000 7f4553e00640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f453006b4d0 0x7f453006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:58.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.817+0000 7f4553e00640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f453006b4d0 0x7f453006d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4544036000 tx=0x7f454402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:58.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:58.821+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f453c059090 con 0x7f45540cc840 2024-04-25T10:13:59.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.012+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45540aca60 con 0x7f453006b4d0 2024-04-25T10:13:59.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.031+0000 7f454be00640 1 -- 172.21.15.165:0/3248931912 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7f4544080a60 con 0x7f453006b4d0 2024-04-25T10:13:59.038 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:13:59.040 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f453006b4d0 msgr2=0x7f453006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f453006b4d0 0x7f453006d990 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f4544036000 tx=0x7f454402e000 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 msgr2=0x7f45541f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 0x7f45541f45b0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f453c00e9b0 tx=0x7f453c00ee70 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 shutdown_connections 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f453006b4d0 0x7f453006d990 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4554143e60 0x7f4554140fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f45540cc840 0x7f45541f45b0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 --2- 172.21.15.165:0/3248931912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f455406e6a0 0x7f45540cbfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.037+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 >> 172.21.15.165:0/3248931912 conn(0x7f455406abe0 msgr2=0x7f455415c820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.038+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 shutdown_connections 2024-04-25T10:13:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.038+0000 7f4559400640 1 -- 172.21.15.165:0/3248931912 wait complete. 2024-04-25T10:13:59.060 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":480,"stamp":"2024-04-25T10:13:58.373107+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":184,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":444568,"kb_used_data":13288,"kb_used_omap":12,"kb_used_meta":431155,"kb_avail":754530152,"statfs":{"total":773094113280,"available":772638875648,"internally_reserved":0,"allocated":13606912,"data_stored":6978044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":441503282},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":26,"apply_latency_ms":26,"commit_latency_ns":26000000,"apply_latency_ns":26000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001505"},"pg_stats":[{"pgid":"18.3f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703632+0000","last_change":"2024-04-25T10:13:53.938397+0000","last_active":"2024-04-25T10:13:55.703632+0000","last_peered":"2024-04-25T10:13:55.703632+0000","last_clean":"2024-04-25T10:13:55.703632+0000","last_became_active":"2024-04-25T10:13:53.938046+0000","last_became_peered":"2024-04-25T10:13:53.938046+0000","last_unstale":"2024-04-25T10:13:55.703632+0000","last_undegraded":"2024-04-25T10:13:55.703632+0000","last_fullsized":"2024-04-25T10:13:55.703632+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:05.909155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703644+0000","last_change":"2024-04-25T10:13:53.974401+0000","last_active":"2024-04-25T10:13:55.703644+0000","last_peered":"2024-04-25T10:13:55.703644+0000","last_clean":"2024-04-25T10:13:55.703644+0000","last_became_active":"2024-04-25T10:13:53.973601+0000","last_became_peered":"2024-04-25T10:13:53.973601+0000","last_unstale":"2024-04-25T10:13:55.703644+0000","last_undegraded":"2024-04-25T10:13:55.703644+0000","last_fullsized":"2024-04-25T10:13:55.703644+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:22.949190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704042+0000","last_change":"2024-04-25T10:13:53.963140+0000","last_active":"2024-04-25T10:13:55.704042+0000","last_peered":"2024-04-25T10:13:55.704042+0000","last_clean":"2024-04-25T10:13:55.704042+0000","last_became_active":"2024-04-25T10:13:53.962832+0000","last_became_peered":"2024-04-25T10:13:53.962832+0000","last_unstale":"2024-04-25T10:13:55.704042+0000","last_undegraded":"2024-04-25T10:13:55.704042+0000","last_fullsized":"2024-04-25T10:13:55.704042+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:57:37.819231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.3c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703811+0000","last_change":"2024-04-25T10:13:53.963887+0000","last_active":"2024-04-25T10:13:55.703811+0000","last_peered":"2024-04-25T10:13:55.703811+0000","last_clean":"2024-04-25T10:13:55.703811+0000","last_became_active":"2024-04-25T10:13:53.961528+0000","last_became_peered":"2024-04-25T10:13:53.961528+0000","last_unstale":"2024-04-25T10:13:55.703811+0000","last_undegraded":"2024-04-25T10:13:55.703811+0000","last_fullsized":"2024-04-25T10:13:55.703811+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:49.121305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703032+0000","last_change":"2024-04-25T10:13:53.942190+0000","last_active":"2024-04-25T10:13:55.703032+0000","last_peered":"2024-04-25T10:13:55.703032+0000","last_clean":"2024-04-25T10:13:55.703032+0000","last_became_active":"2024-04-25T10:13:53.940783+0000","last_became_peered":"2024-04-25T10:13:53.940783+0000","last_unstale":"2024-04-25T10:13:55.703032+0000","last_undegraded":"2024-04-25T10:13:55.703032+0000","last_fullsized":"2024-04-25T10:13:55.703032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:27.688942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944266+0000","last_change":"2024-04-25T10:13:53.968179+0000","last_active":"2024-04-25T10:13:54.944266+0000","last_peered":"2024-04-25T10:13:54.944266+0000","last_clean":"2024-04-25T10:13:54.944266+0000","last_became_active":"2024-04-25T10:13:53.967888+0000","last_became_peered":"2024-04-25T10:13:53.967888+0000","last_unstale":"2024-04-25T10:13:54.944266+0000","last_undegraded":"2024-04-25T10:13:54.944266+0000","last_fullsized":"2024-04-25T10:13:54.944266+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:55.521986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943783+0000","last_change":"2024-04-25T10:13:53.950947+0000","last_active":"2024-04-25T10:13:54.943783+0000","last_peered":"2024-04-25T10:13:54.943783+0000","last_clean":"2024-04-25T10:13:54.943783+0000","last_became_active":"2024-04-25T10:13:53.950654+0000","last_became_peered":"2024-04-25T10:13:53.950654+0000","last_unstale":"2024-04-25T10:13:54.943783+0000","last_undegraded":"2024-04-25T10:13:54.943783+0000","last_fullsized":"2024-04-25T10:13:54.943783+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:13:15.539574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703615+0000","last_change":"2024-04-25T10:13:53.965105+0000","last_active":"2024-04-25T10:13:55.703615+0000","last_peered":"2024-04-25T10:13:55.703615+0000","last_clean":"2024-04-25T10:13:55.703615+0000","last_became_active":"2024-04-25T10:13:53.963012+0000","last_became_peered":"2024-04-25T10:13:53.963012+0000","last_unstale":"2024-04-25T10:13:55.703615+0000","last_undegraded":"2024-04-25T10:13:55.703615+0000","last_fullsized":"2024-04-25T10:13:55.703615+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:05.733395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704758+0000","last_change":"2024-04-25T10:13:53.966264+0000","last_active":"2024-04-25T10:13:55.704758+0000","last_peered":"2024-04-25T10:13:55.704758+0000","last_clean":"2024-04-25T10:13:55.704758+0000","last_became_active":"2024-04-25T10:13:53.965946+0000","last_became_peered":"2024-04-25T10:13:53.965946+0000","last_unstale":"2024-04-25T10:13:55.704758+0000","last_undegraded":"2024-04-25T10:13:55.704758+0000","last_fullsized":"2024-04-25T10:13:55.704758+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:53.911639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933761+0000","last_change":"2024-04-25T10:13:53.955704+0000","last_active":"2024-04-25T10:13:54.933761+0000","last_peered":"2024-04-25T10:13:54.933761+0000","last_clean":"2024-04-25T10:13:54.933761+0000","last_became_active":"2024-04-25T10:13:53.949523+0000","last_became_peered":"2024-04-25T10:13:53.949523+0000","last_unstale":"2024-04-25T10:13:54.933761+0000","last_undegraded":"2024-04-25T10:13:54.933761+0000","last_fullsized":"2024-04-25T10:13:54.933761+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:47:12.419457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703006+0000","last_change":"2024-04-25T10:13:53.949693+0000","last_active":"2024-04-25T10:13:55.703006+0000","last_peered":"2024-04-25T10:13:55.703006+0000","last_clean":"2024-04-25T10:13:55.703006+0000","last_became_active":"2024-04-25T10:13:53.948556+0000","last_became_peered":"2024-04-25T10:13:53.948556+0000","last_unstale":"2024-04-25T10:13:55.703006+0000","last_undegraded":"2024-04-25T10:13:55.703006+0000","last_fullsized":"2024-04-25T10:13:55.703006+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:42.566879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944941+0000","last_change":"2024-04-25T10:13:53.945609+0000","last_active":"2024-04-25T10:13:54.944941+0000","last_peered":"2024-04-25T10:13:54.944941+0000","last_clean":"2024-04-25T10:13:54.944941+0000","last_became_active":"2024-04-25T10:13:53.940951+0000","last_became_peered":"2024-04-25T10:13:53.940951+0000","last_unstale":"2024-04-25T10:13:54.944941+0000","last_undegraded":"2024-04-25T10:13:54.944941+0000","last_fullsized":"2024-04-25T10:13:54.944941+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:00.452282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933423+0000","last_change":"2024-04-25T10:13:53.948512+0000","last_active":"2024-04-25T10:13:54.933423+0000","last_peered":"2024-04-25T10:13:54.933423+0000","last_clean":"2024-04-25T10:13:54.933423+0000","last_became_active":"2024-04-25T10:13:53.948170+0000","last_became_peered":"2024-04-25T10:13:53.948170+0000","last_unstale":"2024-04-25T10:13:54.933423+0000","last_undegraded":"2024-04-25T10:13:54.933423+0000","last_fullsized":"2024-04-25T10:13:54.933423+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:25.935473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703117+0000","last_change":"2024-04-25T10:13:53.965655+0000","last_active":"2024-04-25T10:13:55.703117+0000","last_peered":"2024-04-25T10:13:55.703117+0000","last_clean":"2024-04-25T10:13:55.703117+0000","last_became_active":"2024-04-25T10:13:53.963844+0000","last_became_peered":"2024-04-25T10:13:53.963844+0000","last_unstale":"2024-04-25T10:13:55.703117+0000","last_undegraded":"2024-04-25T10:13:55.703117+0000","last_fullsized":"2024-04-25T10:13:55.703117+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:58.988897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.701959+0000","last_change":"2024-04-25T10:13:53.947215+0000","last_active":"2024-04-25T10:13:55.701959+0000","last_peered":"2024-04-25T10:13:55.701959+0000","last_clean":"2024-04-25T10:13:55.701959+0000","last_became_active":"2024-04-25T10:13:53.946710+0000","last_became_peered":"2024-04-25T10:13:53.946710+0000","last_unstale":"2024-04-25T10:13:55.701959+0000","last_undegraded":"2024-04-25T10:13:55.701959+0000","last_fullsized":"2024-04-25T10:13:55.701959+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:38.769942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943388+0000","last_change":"2024-04-25T10:13:53.956248+0000","last_active":"2024-04-25T10:13:54.943388+0000","last_peered":"2024-04-25T10:13:54.943388+0000","last_clean":"2024-04-25T10:13:54.943388+0000","last_became_active":"2024-04-25T10:13:53.955567+0000","last_became_peered":"2024-04-25T10:13:53.955567+0000","last_unstale":"2024-04-25T10:13:54.943388+0000","last_undegraded":"2024-04-25T10:13:54.943388+0000","last_fullsized":"2024-04-25T10:13:54.943388+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:52:02.712721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933637+0000","last_change":"2024-04-25T10:13:53.948244+0000","last_active":"2024-04-25T10:13:54.933637+0000","last_peered":"2024-04-25T10:13:54.933637+0000","last_clean":"2024-04-25T10:13:54.933637+0000","last_became_active":"2024-04-25T10:13:53.947829+0000","last_became_peered":"2024-04-25T10:13:53.947829+0000","last_unstale":"2024-04-25T10:13:54.933637+0000","last_undegraded":"2024-04-25T10:13:54.933637+0000","last_fullsized":"2024-04-25T10:13:54.933637+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:05.041456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704082+0000","last_change":"2024-04-25T10:13:53.973596+0000","last_active":"2024-04-25T10:13:55.704082+0000","last_peered":"2024-04-25T10:13:55.704082+0000","last_clean":"2024-04-25T10:13:55.704082+0000","last_became_active":"2024-04-25T10:13:53.973355+0000","last_became_peered":"2024-04-25T10:13:53.973355+0000","last_unstale":"2024-04-25T10:13:55.704082+0000","last_undegraded":"2024-04-25T10:13:55.704082+0000","last_fullsized":"2024-04-25T10:13:55.704082+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:16:11.185126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938428+0000","last_change":"2024-04-25T10:13:53.960666+0000","last_active":"2024-04-25T10:13:54.938428+0000","last_peered":"2024-04-25T10:13:54.938428+0000","last_clean":"2024-04-25T10:13:54.938428+0000","last_became_active":"2024-04-25T10:13:53.960313+0000","last_became_peered":"2024-04-25T10:13:53.960313+0000","last_unstale":"2024-04-25T10:13:54.938428+0000","last_undegraded":"2024-04-25T10:13:54.938428+0000","last_fullsized":"2024-04-25T10:13:54.938428+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.074055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160444+0000","last_change":"2024-04-25T10:13:53.953933+0000","last_active":"2024-04-25T10:13:55.160444+0000","last_peered":"2024-04-25T10:13:55.160444+0000","last_clean":"2024-04-25T10:13:55.160444+0000","last_became_active":"2024-04-25T10:13:53.953645+0000","last_became_peered":"2024-04-25T10:13:53.953645+0000","last_unstale":"2024-04-25T10:13:55.160444+0000","last_undegraded":"2024-04-25T10:13:55.160444+0000","last_fullsized":"2024-04-25T10:13:55.160444+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:20:32.025475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703412+0000","last_change":"2024-04-25T10:13:53.964523+0000","last_active":"2024-04-25T10:13:55.703412+0000","last_peered":"2024-04-25T10:13:55.703412+0000","last_clean":"2024-04-25T10:13:55.703412+0000","last_became_active":"2024-04-25T10:13:53.964140+0000","last_became_peered":"2024-04-25T10:13:53.964140+0000","last_unstale":"2024-04-25T10:13:55.703412+0000","last_undegraded":"2024-04-25T10:13:55.703412+0000","last_fullsized":"2024-04-25T10:13:55.703412+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:05:41.776706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702642+0000","last_change":"2024-04-25T10:13:53.965556+0000","last_active":"2024-04-25T10:13:55.702642+0000","last_peered":"2024-04-25T10:13:55.702642+0000","last_clean":"2024-04-25T10:13:55.702642+0000","last_became_active":"2024-04-25T10:13:53.965284+0000","last_became_peered":"2024-04-25T10:13:53.965284+0000","last_unstale":"2024-04-25T10:13:55.702642+0000","last_undegraded":"2024-04-25T10:13:55.702642+0000","last_fullsized":"2024-04-25T10:13:55.702642+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:12.544755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702641+0000","last_change":"2024-04-25T10:13:53.965243+0000","last_active":"2024-04-25T10:13:55.702641+0000","last_peered":"2024-04-25T10:13:55.702641+0000","last_clean":"2024-04-25T10:13:55.702641+0000","last_became_active":"2024-04-25T10:13:53.964917+0000","last_became_peered":"2024-04-25T10:13:53.964917+0000","last_unstale":"2024-04-25T10:13:55.702641+0000","last_undegraded":"2024-04-25T10:13:55.702641+0000","last_fullsized":"2024-04-25T10:13:55.702641+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:25.959967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.936302+0000","last_change":"2024-04-25T10:13:53.951530+0000","last_active":"2024-04-25T10:13:54.936302+0000","last_peered":"2024-04-25T10:13:54.936302+0000","last_clean":"2024-04-25T10:13:54.936302+0000","last_became_active":"2024-04-25T10:13:53.951220+0000","last_became_peered":"2024-04-25T10:13:53.951220+0000","last_unstale":"2024-04-25T10:13:54.936302+0000","last_undegraded":"2024-04-25T10:13:54.936302+0000","last_fullsized":"2024-04-25T10:13:54.936302+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:15:04.728318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939131+0000","last_change":"2024-04-25T10:13:53.953306+0000","last_active":"2024-04-25T10:13:54.939131+0000","last_peered":"2024-04-25T10:13:54.939131+0000","last_clean":"2024-04-25T10:13:54.939131+0000","last_became_active":"2024-04-25T10:13:53.952897+0000","last_became_peered":"2024-04-25T10:13:53.952897+0000","last_unstale":"2024-04-25T10:13:54.939131+0000","last_undegraded":"2024-04-25T10:13:54.939131+0000","last_fullsized":"2024-04-25T10:13:54.939131+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:32.097428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160827+0000","last_change":"2024-04-25T10:13:53.960383+0000","last_active":"2024-04-25T10:13:55.160827+0000","last_peered":"2024-04-25T10:13:55.160827+0000","last_clean":"2024-04-25T10:13:55.160827+0000","last_became_active":"2024-04-25T10:13:53.960120+0000","last_became_peered":"2024-04-25T10:13:53.960120+0000","last_unstale":"2024-04-25T10:13:55.160827+0000","last_undegraded":"2024-04-25T10:13:55.160827+0000","last_fullsized":"2024-04-25T10:13:55.160827+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:45:39.823966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938473+0000","last_change":"2024-04-25T10:13:53.957550+0000","last_active":"2024-04-25T10:13:54.938473+0000","last_peered":"2024-04-25T10:13:54.938473+0000","last_clean":"2024-04-25T10:13:54.938473+0000","last_became_active":"2024-04-25T10:13:53.957267+0000","last_became_peered":"2024-04-25T10:13:53.957267+0000","last_unstale":"2024-04-25T10:13:54.938473+0000","last_undegraded":"2024-04-25T10:13:54.938473+0000","last_fullsized":"2024-04-25T10:13:54.938473+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:05.909155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945288+0000","last_change":"2024-04-25T10:13:53.950224+0000","last_active":"2024-04-25T10:13:54.945288+0000","last_peered":"2024-04-25T10:13:54.945288+0000","last_clean":"2024-04-25T10:13:54.945288+0000","last_became_active":"2024-04-25T10:13:53.949626+0000","last_became_peered":"2024-04-25T10:13:53.949626+0000","last_unstale":"2024-04-25T10:13:54.945288+0000","last_undegraded":"2024-04-25T10:13:54.945288+0000","last_fullsized":"2024-04-25T10:13:54.945288+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:28:04.661911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160346+0000","last_change":"2024-04-25T10:13:53.963506+0000","last_active":"2024-04-25T10:13:55.160346+0000","last_peered":"2024-04-25T10:13:55.160346+0000","last_clean":"2024-04-25T10:13:55.160346+0000","last_became_active":"2024-04-25T10:13:53.963245+0000","last_became_peered":"2024-04-25T10:13:53.963245+0000","last_unstale":"2024-04-25T10:13:55.160346+0000","last_undegraded":"2024-04-25T10:13:55.160346+0000","last_fullsized":"2024-04-25T10:13:55.160346+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:13.483720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703188+0000","last_change":"2024-04-25T10:13:53.966377+0000","last_active":"2024-04-25T10:13:55.703188+0000","last_peered":"2024-04-25T10:13:55.703188+0000","last_clean":"2024-04-25T10:13:55.703188+0000","last_became_active":"2024-04-25T10:13:53.966071+0000","last_became_peered":"2024-04-25T10:13:53.966071+0000","last_unstale":"2024-04-25T10:13:55.703188+0000","last_undegraded":"2024-04-25T10:13:55.703188+0000","last_fullsized":"2024-04-25T10:13:55.703188+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:53.911639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938028+0000","last_change":"2024-04-25T10:13:53.958828+0000","last_active":"2024-04-25T10:13:54.938028+0000","last_peered":"2024-04-25T10:13:54.938028+0000","last_clean":"2024-04-25T10:13:54.938028+0000","last_became_active":"2024-04-25T10:13:53.958530+0000","last_became_peered":"2024-04-25T10:13:53.958530+0000","last_unstale":"2024-04-25T10:13:54.938028+0000","last_undegraded":"2024-04-25T10:13:54.938028+0000","last_fullsized":"2024-04-25T10:13:54.938028+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:15:41.142499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.936722+0000","last_change":"2024-04-25T10:13:53.951325+0000","last_active":"2024-04-25T10:13:54.936722+0000","last_peered":"2024-04-25T10:13:54.936722+0000","last_clean":"2024-04-25T10:13:54.936722+0000","last_became_active":"2024-04-25T10:13:53.951015+0000","last_became_peered":"2024-04-25T10:13:53.951015+0000","last_unstale":"2024-04-25T10:13:54.936722+0000","last_undegraded":"2024-04-25T10:13:54.936722+0000","last_fullsized":"2024-04-25T10:13:54.936722+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:07.221512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933258+0000","last_change":"2024-04-25T10:13:53.957838+0000","last_active":"2024-04-25T10:13:54.933258+0000","last_peered":"2024-04-25T10:13:54.933258+0000","last_clean":"2024-04-25T10:13:54.933258+0000","last_became_active":"2024-04-25T10:13:53.949077+0000","last_became_peered":"2024-04-25T10:13:53.949077+0000","last_unstale":"2024-04-25T10:13:54.933258+0000","last_undegraded":"2024-04-25T10:13:54.933258+0000","last_fullsized":"2024-04-25T10:13:54.933258+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:26:30.373592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"123'4","reported_seq":18,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938619+0000","last_change":"2024-04-25T10:13:52.952957+0000","last_active":"2024-04-25T10:13:54.938619+0000","last_peered":"2024-04-25T10:13:54.938619+0000","last_clean":"2024-04-25T10:13:54.938619+0000","last_became_active":"2024-04-25T10:13:52.952649+0000","last_became_peered":"2024-04-25T10:13:52.952649+0000","last_unstale":"2024-04-25T10:13:54.938619+0000","last_undegraded":"2024-04-25T10:13:54.938619+0000","last_fullsized":"2024-04-25T10:13:54.938619+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:43:29.112347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944658+0000","last_change":"2024-04-25T10:13:53.972068+0000","last_active":"2024-04-25T10:13:54.944658+0000","last_peered":"2024-04-25T10:13:54.944658+0000","last_clean":"2024-04-25T10:13:54.944658+0000","last_became_active":"2024-04-25T10:13:53.971723+0000","last_became_peered":"2024-04-25T10:13:53.971723+0000","last_unstale":"2024-04-25T10:13:54.944658+0000","last_undegraded":"2024-04-25T10:13:54.944658+0000","last_fullsized":"2024-04-25T10:13:54.944658+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:47:59.790942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933566+0000","last_change":"2024-04-25T10:13:52.949136+0000","last_active":"2024-04-25T10:13:54.933566+0000","last_peered":"2024-04-25T10:13:54.933566+0000","last_clean":"2024-04-25T10:13:54.933566+0000","last_became_active":"2024-04-25T10:13:52.945369+0000","last_became_peered":"2024-04-25T10:13:52.945369+0000","last_unstale":"2024-04-25T10:13:54.933566+0000","last_undegraded":"2024-04-25T10:13:54.933566+0000","last_fullsized":"2024-04-25T10:13:54.933566+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.859783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938032+0000","last_change":"2024-04-25T10:13:53.948958+0000","last_active":"2024-04-25T10:13:54.938032+0000","last_peered":"2024-04-25T10:13:54.938032+0000","last_clean":"2024-04-25T10:13:54.938032+0000","last_became_active":"2024-04-25T10:13:53.948652+0000","last_became_peered":"2024-04-25T10:13:53.948652+0000","last_unstale":"2024-04-25T10:13:54.938032+0000","last_undegraded":"2024-04-25T10:13:54.938032+0000","last_fullsized":"2024-04-25T10:13:54.938032+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:24.992378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704579+0000","last_change":"2024-04-25T10:13:52.959482+0000","last_active":"2024-04-25T10:13:55.704579+0000","last_peered":"2024-04-25T10:13:55.704579+0000","last_clean":"2024-04-25T10:13:55.704579+0000","last_became_active":"2024-04-25T10:13:52.959158+0000","last_became_peered":"2024-04-25T10:13:52.959158+0000","last_unstale":"2024-04-25T10:13:55.704579+0000","last_undegraded":"2024-04-25T10:13:55.704579+0000","last_fullsized":"2024-04-25T10:13:55.704579+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:15.010081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704853+0000","last_change":"2024-04-25T10:13:53.968149+0000","last_active":"2024-04-25T10:13:55.704853+0000","last_peered":"2024-04-25T10:13:55.704853+0000","last_clean":"2024-04-25T10:13:55.704853+0000","last_became_active":"2024-04-25T10:13:53.967768+0000","last_became_peered":"2024-04-25T10:13:53.967768+0000","last_unstale":"2024-04-25T10:13:55.704853+0000","last_undegraded":"2024-04-25T10:13:55.704853+0000","last_fullsized":"2024-04-25T10:13:55.704853+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.370452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933566+0000","last_change":"2024-04-25T10:13:52.947627+0000","last_active":"2024-04-25T10:13:54.933566+0000","last_peered":"2024-04-25T10:13:54.933566+0000","last_clean":"2024-04-25T10:13:54.933566+0000","last_became_active":"2024-04-25T10:13:52.946659+0000","last_became_peered":"2024-04-25T10:13:52.946659+0000","last_unstale":"2024-04-25T10:13:54.933566+0000","last_undegraded":"2024-04-25T10:13:54.933566+0000","last_fullsized":"2024-04-25T10:13:54.933566+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:06:20.814067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933045+0000","last_change":"2024-04-25T10:13:53.943413+0000","last_active":"2024-04-25T10:13:54.933045+0000","last_peered":"2024-04-25T10:13:54.933045+0000","last_clean":"2024-04-25T10:13:54.933045+0000","last_became_active":"2024-04-25T10:13:53.942619+0000","last_became_peered":"2024-04-25T10:13:53.942619+0000","last_unstale":"2024-04-25T10:13:54.933045+0000","last_undegraded":"2024-04-25T10:13:54.933045+0000","last_fullsized":"2024-04-25T10:13:54.933045+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:42.566879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702153+0000","last_change":"2024-04-25T10:13:52.951325+0000","last_active":"2024-04-25T10:13:55.702153+0000","last_peered":"2024-04-25T10:13:55.702153+0000","last_clean":"2024-04-25T10:13:55.702153+0000","last_became_active":"2024-04-25T10:13:52.950036+0000","last_became_peered":"2024-04-25T10:13:52.950036+0000","last_unstale":"2024-04-25T10:13:55.702153+0000","last_undegraded":"2024-04-25T10:13:55.702153+0000","last_fullsized":"2024-04-25T10:13:55.702153+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:42:28.476337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702616+0000","last_change":"2024-04-25T10:13:53.946993+0000","last_active":"2024-04-25T10:13:55.702616+0000","last_peered":"2024-04-25T10:13:55.702616+0000","last_clean":"2024-04-25T10:13:55.702616+0000","last_became_active":"2024-04-25T10:13:53.946306+0000","last_became_peered":"2024-04-25T10:13:53.946306+0000","last_unstale":"2024-04-25T10:13:55.702616+0000","last_undegraded":"2024-04-25T10:13:55.702616+0000","last_fullsized":"2024-04-25T10:13:55.702616+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:49.929009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943982+0000","last_change":"2024-04-25T10:13:52.944454+0000","last_active":"2024-04-25T10:13:54.943982+0000","last_peered":"2024-04-25T10:13:54.943982+0000","last_clean":"2024-04-25T10:13:54.943982+0000","last_became_active":"2024-04-25T10:13:52.944146+0000","last_became_peered":"2024-04-25T10:13:52.944146+0000","last_unstale":"2024-04-25T10:13:54.943982+0000","last_undegraded":"2024-04-25T10:13:54.943982+0000","last_fullsized":"2024-04-25T10:13:54.943982+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:01:49.745429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703585+0000","last_change":"2024-04-25T10:13:53.973096+0000","last_active":"2024-04-25T10:13:55.703585+0000","last_peered":"2024-04-25T10:13:55.703585+0000","last_clean":"2024-04-25T10:13:55.703585+0000","last_became_active":"2024-04-25T10:13:53.971075+0000","last_became_peered":"2024-04-25T10:13:53.971075+0000","last_unstale":"2024-04-25T10:13:55.703585+0000","last_undegraded":"2024-04-25T10:13:55.703585+0000","last_fullsized":"2024-04-25T10:13:55.703585+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:39.370452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.937796+0000","last_change":"2024-04-25T10:13:52.949231+0000","last_active":"2024-04-25T10:13:54.937796+0000","last_peered":"2024-04-25T10:13:54.937796+0000","last_clean":"2024-04-25T10:13:54.937796+0000","last_became_active":"2024-04-25T10:13:52.948931+0000","last_became_peered":"2024-04-25T10:13:52.948931+0000","last_unstale":"2024-04-25T10:13:54.937796+0000","last_undegraded":"2024-04-25T10:13:54.937796+0000","last_fullsized":"2024-04-25T10:13:54.937796+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:53.684783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933013+0000","last_change":"2024-04-25T10:13:53.965985+0000","last_active":"2024-04-25T10:13:54.933013+0000","last_peered":"2024-04-25T10:13:54.933013+0000","last_clean":"2024-04-25T10:13:54.933013+0000","last_became_active":"2024-04-25T10:13:53.964930+0000","last_became_peered":"2024-04-25T10:13:53.964930+0000","last_unstale":"2024-04-25T10:13:54.933013+0000","last_undegraded":"2024-04-25T10:13:54.933013+0000","last_fullsized":"2024-04-25T10:13:54.933013+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:39:12.544755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943617+0000","last_change":"2024-04-25T10:13:52.951999+0000","last_active":"2024-04-25T10:13:54.943617+0000","last_peered":"2024-04-25T10:13:54.943617+0000","last_clean":"2024-04-25T10:13:54.943617+0000","last_became_active":"2024-04-25T10:13:52.951631+0000","last_became_peered":"2024-04-25T10:13:52.951631+0000","last_unstale":"2024-04-25T10:13:54.943617+0000","last_undegraded":"2024-04-25T10:13:54.943617+0000","last_fullsized":"2024-04-25T10:13:54.943617+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:49.401614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.937861+0000","last_change":"2024-04-25T10:13:53.951524+0000","last_active":"2024-04-25T10:13:54.937861+0000","last_peered":"2024-04-25T10:13:54.937861+0000","last_clean":"2024-04-25T10:13:54.937861+0000","last_became_active":"2024-04-25T10:13:53.951209+0000","last_became_peered":"2024-04-25T10:13:53.951209+0000","last_unstale":"2024-04-25T10:13:54.937861+0000","last_undegraded":"2024-04-25T10:13:54.937861+0000","last_fullsized":"2024-04-25T10:13:54.937861+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:42.566879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939045+0000","last_change":"2024-04-25T10:13:52.956823+0000","last_active":"2024-04-25T10:13:54.939045+0000","last_peered":"2024-04-25T10:13:54.939045+0000","last_clean":"2024-04-25T10:13:54.939045+0000","last_became_active":"2024-04-25T10:13:52.955291+0000","last_became_peered":"2024-04-25T10:13:52.955291+0000","last_unstale":"2024-04-25T10:13:54.939045+0000","last_undegraded":"2024-04-25T10:13:54.939045+0000","last_fullsized":"2024-04-25T10:13:54.939045+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:03.727616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938405+0000","last_change":"2024-04-25T10:13:53.972282+0000","last_active":"2024-04-25T10:13:54.938405+0000","last_peered":"2024-04-25T10:13:54.938405+0000","last_clean":"2024-04-25T10:13:54.938405+0000","last_became_active":"2024-04-25T10:13:53.971854+0000","last_became_peered":"2024-04-25T10:13:53.971854+0000","last_unstale":"2024-04-25T10:13:54.938405+0000","last_undegraded":"2024-04-25T10:13:54.938405+0000","last_fullsized":"2024-04-25T10:13:54.938405+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:27.356559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938863+0000","last_change":"2024-04-25T10:13:52.946859+0000","last_active":"2024-04-25T10:13:54.938863+0000","last_peered":"2024-04-25T10:13:54.938863+0000","last_clean":"2024-04-25T10:13:54.938863+0000","last_became_active":"2024-04-25T10:13:52.946562+0000","last_became_peered":"2024-04-25T10:13:52.946562+0000","last_unstale":"2024-04-25T10:13:54.938863+0000","last_undegraded":"2024-04-25T10:13:54.938863+0000","last_fullsized":"2024-04-25T10:13:54.938863+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:37.752696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704412+0000","last_change":"2024-04-25T10:13:53.966753+0000","last_active":"2024-04-25T10:13:55.704412+0000","last_peered":"2024-04-25T10:13:55.704412+0000","last_clean":"2024-04-25T10:13:55.704412+0000","last_became_active":"2024-04-25T10:13:53.966490+0000","last_became_peered":"2024-04-25T10:13:53.966490+0000","last_unstale":"2024-04-25T10:13:55.704412+0000","last_undegraded":"2024-04-25T10:13:55.704412+0000","last_fullsized":"2024-04-25T10:13:55.704412+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:53:57.777532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"123'3","reported_seq":17,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938599+0000","last_change":"2024-04-25T10:13:52.953111+0000","last_active":"2024-04-25T10:13:54.938599+0000","last_peered":"2024-04-25T10:13:54.938599+0000","last_clean":"2024-04-25T10:13:54.938599+0000","last_became_active":"2024-04-25T10:13:52.952801+0000","last_became_peered":"2024-04-25T10:13:52.952801+0000","last_unstale":"2024-04-25T10:13:54.938599+0000","last_undegraded":"2024-04-25T10:13:54.938599+0000","last_fullsized":"2024-04-25T10:13:54.938599+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:48.911762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939414+0000","last_change":"2024-04-25T10:13:53.963311+0000","last_active":"2024-04-25T10:13:54.939414+0000","last_peered":"2024-04-25T10:13:54.939414+0000","last_clean":"2024-04-25T10:13:54.939414+0000","last_became_active":"2024-04-25T10:13:53.962886+0000","last_became_peered":"2024-04-25T10:13:53.962886+0000","last_unstale":"2024-04-25T10:13:54.939414+0000","last_undegraded":"2024-04-25T10:13:54.939414+0000","last_fullsized":"2024-04-25T10:13:54.939414+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:59.838400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"123'6","reported_seq":20,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703122+0000","last_change":"2024-04-25T10:13:52.949538+0000","last_active":"2024-04-25T10:13:55.703122+0000","last_peered":"2024-04-25T10:13:55.703122+0000","last_clean":"2024-04-25T10:13:55.703122+0000","last_became_active":"2024-04-25T10:13:52.949263+0000","last_became_peered":"2024-04-25T10:13:52.949263+0000","last_unstale":"2024-04-25T10:13:55.703122+0000","last_undegraded":"2024-04-25T10:13:55.703122+0000","last_fullsized":"2024-04-25T10:13:55.703122+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:19.361315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938031+0000","last_change":"2024-04-25T10:13:53.965432+0000","last_active":"2024-04-25T10:13:54.938031+0000","last_peered":"2024-04-25T10:13:54.938031+0000","last_clean":"2024-04-25T10:13:54.938031+0000","last_became_active":"2024-04-25T10:13:53.964752+0000","last_became_peered":"2024-04-25T10:13:53.964752+0000","last_unstale":"2024-04-25T10:13:54.938031+0000","last_undegraded":"2024-04-25T10:13:54.938031+0000","last_fullsized":"2024-04-25T10:13:54.938031+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:27.688942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703621+0000","last_change":"2024-04-25T10:13:52.953619+0000","last_active":"2024-04-25T10:13:55.703621+0000","last_peered":"2024-04-25T10:13:55.703621+0000","last_clean":"2024-04-25T10:13:55.703621+0000","last_became_active":"2024-04-25T10:13:52.952867+0000","last_became_peered":"2024-04-25T10:13:52.952867+0000","last_unstale":"2024-04-25T10:13:55.703621+0000","last_undegraded":"2024-04-25T10:13:55.703621+0000","last_fullsized":"2024-04-25T10:13:55.703621+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:46:15.010081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933081+0000","last_change":"2024-04-25T10:13:53.950372+0000","last_active":"2024-04-25T10:13:54.933081+0000","last_peered":"2024-04-25T10:13:54.933081+0000","last_clean":"2024-04-25T10:13:54.933081+0000","last_became_active":"2024-04-25T10:13:53.948847+0000","last_became_peered":"2024-04-25T10:13:53.948847+0000","last_unstale":"2024-04-25T10:13:54.933081+0000","last_undegraded":"2024-04-25T10:13:54.933081+0000","last_fullsized":"2024-04-25T10:13:54.933081+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:27.688942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703821+0000","last_change":"2024-04-25T10:13:52.951277+0000","last_active":"2024-04-25T10:13:55.703821+0000","last_peered":"2024-04-25T10:13:55.703821+0000","last_clean":"2024-04-25T10:13:55.703821+0000","last_became_active":"2024-04-25T10:13:52.949938+0000","last_became_peered":"2024-04-25T10:13:52.949938+0000","last_unstale":"2024-04-25T10:13:55.703821+0000","last_undegraded":"2024-04-25T10:13:55.703821+0000","last_fullsized":"2024-04-25T10:13:55.703821+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:15:52.233132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160238+0000","last_change":"2024-04-25T10:13:53.955064+0000","last_active":"2024-04-25T10:13:55.160238+0000","last_peered":"2024-04-25T10:13:55.160238+0000","last_clean":"2024-04-25T10:13:55.160238+0000","last_became_active":"2024-04-25T10:13:53.954769+0000","last_became_peered":"2024-04-25T10:13:53.954769+0000","last_unstale":"2024-04-25T10:13:55.160238+0000","last_undegraded":"2024-04-25T10:13:55.160238+0000","last_fullsized":"2024-04-25T10:13:55.160238+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:06:49.129447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703830+0000","last_change":"2024-04-25T10:13:52.945225+0000","last_active":"2024-04-25T10:13:55.703830+0000","last_peered":"2024-04-25T10:13:55.703830+0000","last_clean":"2024-04-25T10:13:55.703830+0000","last_became_active":"2024-04-25T10:13:52.944298+0000","last_became_peered":"2024-04-25T10:13:52.944298+0000","last_unstale":"2024-04-25T10:13:55.703830+0000","last_undegraded":"2024-04-25T10:13:55.703830+0000","last_fullsized":"2024-04-25T10:13:55.703830+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:32:15.136996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938524+0000","last_change":"2024-04-25T10:13:53.956113+0000","last_active":"2024-04-25T10:13:54.938524+0000","last_peered":"2024-04-25T10:13:54.938524+0000","last_clean":"2024-04-25T10:13:54.938524+0000","last_became_active":"2024-04-25T10:13:53.955756+0000","last_became_peered":"2024-04-25T10:13:53.955756+0000","last_unstale":"2024-04-25T10:13:54.938524+0000","last_undegraded":"2024-04-25T10:13:54.938524+0000","last_fullsized":"2024-04-25T10:13:54.938524+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:14:16.867933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933780+0000","last_change":"2024-04-25T10:13:52.954949+0000","last_active":"2024-04-25T10:13:54.933780+0000","last_peered":"2024-04-25T10:13:54.933780+0000","last_clean":"2024-04-25T10:13:54.933780+0000","last_became_active":"2024-04-25T10:13:52.954523+0000","last_became_peered":"2024-04-25T10:13:52.954523+0000","last_unstale":"2024-04-25T10:13:54.933780+0000","last_undegraded":"2024-04-25T10:13:54.933780+0000","last_fullsized":"2024-04-25T10:13:54.933780+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:49.620494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703355+0000","last_change":"2024-04-25T10:13:53.964942+0000","last_active":"2024-04-25T10:13:55.703355+0000","last_peered":"2024-04-25T10:13:55.703355+0000","last_clean":"2024-04-25T10:13:55.703355+0000","last_became_active":"2024-04-25T10:13:53.964657+0000","last_became_peered":"2024-04-25T10:13:53.964657+0000","last_unstale":"2024-04-25T10:13:55.703355+0000","last_undegraded":"2024-04-25T10:13:55.703355+0000","last_fullsized":"2024-04-25T10:13:55.703355+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.074055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.160957+0000","last_change":"2024-04-25T10:13:52.942066+0000","last_active":"2024-04-25T10:13:55.160957+0000","last_peered":"2024-04-25T10:13:55.160957+0000","last_clean":"2024-04-25T10:13:55.160957+0000","last_became_active":"2024-04-25T10:13:52.941765+0000","last_became_peered":"2024-04-25T10:13:52.941765+0000","last_unstale":"2024-04-25T10:13:55.160957+0000","last_undegraded":"2024-04-25T10:13:55.160957+0000","last_fullsized":"2024-04-25T10:13:55.160957+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:19:16.814033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945286+0000","last_change":"2024-04-25T10:13:53.954544+0000","last_active":"2024-04-25T10:13:54.945286+0000","last_peered":"2024-04-25T10:13:54.945286+0000","last_clean":"2024-04-25T10:13:54.945286+0000","last_became_active":"2024-04-25T10:13:53.954282+0000","last_became_peered":"2024-04-25T10:13:53.954282+0000","last_unstale":"2024-04-25T10:13:54.945286+0000","last_undegraded":"2024-04-25T10:13:54.945286+0000","last_fullsized":"2024-04-25T10:13:54.945286+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:13.800570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"123'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704890+0000","last_change":"2024-04-25T10:13:52.959185+0000","last_active":"2024-04-25T10:13:55.704890+0000","last_peered":"2024-04-25T10:13:55.704890+0000","last_clean":"2024-04-25T10:13:55.704890+0000","last_became_active":"2024-04-25T10:13:52.958904+0000","last_became_peered":"2024-04-25T10:13:52.958904+0000","last_unstale":"2024-04-25T10:13:55.704890+0000","last_undegraded":"2024-04-25T10:13:55.704890+0000","last_fullsized":"2024-04-25T10:13:55.704890+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:09:26.993885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.939231+0000","last_change":"2024-04-25T10:13:53.967716+0000","last_active":"2024-04-25T10:13:54.939231+0000","last_peered":"2024-04-25T10:13:54.939231+0000","last_clean":"2024-04-25T10:13:54.939231+0000","last_became_active":"2024-04-25T10:13:53.967029+0000","last_became_peered":"2024-04-25T10:13:53.967029+0000","last_unstale":"2024-04-25T10:13:54.939231+0000","last_undegraded":"2024-04-25T10:13:54.939231+0000","last_fullsized":"2024-04-25T10:13:54.939231+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:24:03.943579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704039+0000","last_change":"2024-04-25T10:13:52.958671+0000","last_active":"2024-04-25T10:13:55.704039+0000","last_peered":"2024-04-25T10:13:55.704039+0000","last_clean":"2024-04-25T10:13:55.704039+0000","last_became_active":"2024-04-25T10:13:52.958352+0000","last_became_peered":"2024-04-25T10:13:52.958352+0000","last_unstale":"2024-04-25T10:13:55.704039+0000","last_undegraded":"2024-04-25T10:13:55.704039+0000","last_fullsized":"2024-04-25T10:13:55.704039+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:31.851104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943684+0000","last_change":"2024-04-25T10:13:53.947883+0000","last_active":"2024-04-25T10:13:54.943684+0000","last_peered":"2024-04-25T10:13:54.943684+0000","last_clean":"2024-04-25T10:13:54.943684+0000","last_became_active":"2024-04-25T10:13:53.940997+0000","last_became_peered":"2024-04-25T10:13:53.940997+0000","last_unstale":"2024-04-25T10:13:54.943684+0000","last_undegraded":"2024-04-25T10:13:54.943684+0000","last_fullsized":"2024-04-25T10:13:54.943684+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:29.369969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944071+0000","last_change":"2024-04-25T10:13:52.952949+0000","last_active":"2024-04-25T10:13:54.944071+0000","last_peered":"2024-04-25T10:13:54.944071+0000","last_clean":"2024-04-25T10:13:54.944071+0000","last_became_active":"2024-04-25T10:13:52.952656+0000","last_became_peered":"2024-04-25T10:13:52.952656+0000","last_unstale":"2024-04-25T10:13:54.944071+0000","last_undegraded":"2024-04-25T10:13:54.944071+0000","last_fullsized":"2024-04-25T10:13:54.944071+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:22.863698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702881+0000","last_change":"2024-04-25T10:13:53.961093+0000","last_active":"2024-04-25T10:13:55.702881+0000","last_peered":"2024-04-25T10:13:55.702881+0000","last_clean":"2024-04-25T10:13:55.702881+0000","last_became_active":"2024-04-25T10:13:53.960111+0000","last_became_peered":"2024-04-25T10:13:53.960111+0000","last_unstale":"2024-04-25T10:13:55.702881+0000","last_undegraded":"2024-04-25T10:13:55.702881+0000","last_fullsized":"2024-04-25T10:13:55.702881+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:49.121305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702906+0000","last_change":"2024-04-25T10:13:52.956176+0000","last_active":"2024-04-25T10:13:55.702906+0000","last_peered":"2024-04-25T10:13:55.702906+0000","last_clean":"2024-04-25T10:13:55.702906+0000","last_became_active":"2024-04-25T10:13:52.955794+0000","last_became_peered":"2024-04-25T10:13:52.955794+0000","last_unstale":"2024-04-25T10:13:55.702906+0000","last_undegraded":"2024-04-25T10:13:55.702906+0000","last_fullsized":"2024-04-25T10:13:55.702906+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:19.361315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703659+0000","last_change":"2024-04-25T10:13:53.967211+0000","last_active":"2024-04-25T10:13:55.703659+0000","last_peered":"2024-04-25T10:13:55.703659+0000","last_clean":"2024-04-25T10:13:55.703659+0000","last_became_active":"2024-04-25T10:13:53.965126+0000","last_became_peered":"2024-04-25T10:13:53.965126+0000","last_unstale":"2024-04-25T10:13:55.703659+0000","last_undegraded":"2024-04-25T10:13:55.703659+0000","last_fullsized":"2024-04-25T10:13:55.703659+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:12:23.327516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703583+0000","last_change":"2024-04-25T10:13:52.954482+0000","last_active":"2024-04-25T10:13:55.703583+0000","last_peered":"2024-04-25T10:13:55.703583+0000","last_clean":"2024-04-25T10:13:55.703583+0000","last_became_active":"2024-04-25T10:13:52.954195+0000","last_became_peered":"2024-04-25T10:13:52.954195+0000","last_unstale":"2024-04-25T10:13:55.703583+0000","last_undegraded":"2024-04-25T10:13:55.703583+0000","last_fullsized":"2024-04-25T10:13:55.703583+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:38.096531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.937509+0000","last_change":"2024-04-25T10:13:53.958857+0000","last_active":"2024-04-25T10:13:54.937509+0000","last_peered":"2024-04-25T10:13:54.937509+0000","last_clean":"2024-04-25T10:13:54.937509+0000","last_became_active":"2024-04-25T10:13:53.958579+0000","last_became_peered":"2024-04-25T10:13:53.958579+0000","last_unstale":"2024-04-25T10:13:54.937509+0000","last_undegraded":"2024-04-25T10:13:54.937509+0000","last_fullsized":"2024-04-25T10:13:54.937509+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:07:36.863606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.701549+0000","last_change":"2024-04-25T10:13:52.959587+0000","last_active":"2024-04-25T10:13:55.701549+0000","last_peered":"2024-04-25T10:13:55.701549+0000","last_clean":"2024-04-25T10:13:55.701549+0000","last_became_active":"2024-04-25T10:13:52.958324+0000","last_became_peered":"2024-04-25T10:13:52.958324+0000","last_unstale":"2024-04-25T10:13:55.701549+0000","last_undegraded":"2024-04-25T10:13:55.701549+0000","last_fullsized":"2024-04-25T10:13:55.701549+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:06:58.821153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945338+0000","last_change":"2024-04-25T10:13:53.951670+0000","last_active":"2024-04-25T10:13:54.945338+0000","last_peered":"2024-04-25T10:13:54.945338+0000","last_clean":"2024-04-25T10:13:54.945338+0000","last_became_active":"2024-04-25T10:13:53.946757+0000","last_became_peered":"2024-04-25T10:13:53.946757+0000","last_unstale":"2024-04-25T10:13:54.945338+0000","last_undegraded":"2024-04-25T10:13:54.945338+0000","last_fullsized":"2024-04-25T10:13:54.945338+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:41:09.326360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"123'2","reported_seq":16,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.944997+0000","last_change":"2024-04-25T10:13:52.944932+0000","last_active":"2024-04-25T10:13:54.944997+0000","last_peered":"2024-04-25T10:13:54.944997+0000","last_clean":"2024-04-25T10:13:54.944997+0000","last_became_active":"2024-04-25T10:13:52.943685+0000","last_became_peered":"2024-04-25T10:13:52.943685+0000","last_unstale":"2024-04-25T10:13:54.944997+0000","last_undegraded":"2024-04-25T10:13:54.944997+0000","last_fullsized":"2024-04-25T10:13:54.944997+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:56.619463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933237+0000","last_change":"2024-04-25T10:13:53.964895+0000","last_active":"2024-04-25T10:13:54.933237+0000","last_peered":"2024-04-25T10:13:54.933237+0000","last_clean":"2024-04-25T10:13:54.933237+0000","last_became_active":"2024-04-25T10:13:53.950760+0000","last_became_peered":"2024-04-25T10:13:53.950760+0000","last_unstale":"2024-04-25T10:13:54.933237+0000","last_undegraded":"2024-04-25T10:13:54.933237+0000","last_fullsized":"2024-04-25T10:13:54.933237+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:44:05.909155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"124'1","reported_seq":15,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938402+0000","last_change":"2024-04-25T10:13:52.956533+0000","last_active":"2024-04-25T10:13:54.938402+0000","last_peered":"2024-04-25T10:13:54.938402+0000","last_clean":"2024-04-25T10:13:54.938402+0000","last_became_active":"2024-04-25T10:13:52.956203+0000","last_became_peered":"2024-04-25T10:13:52.956203+0000","last_unstale":"2024-04-25T10:13:54.938402+0000","last_undegraded":"2024-04-25T10:13:54.938402+0000","last_fullsized":"2024-04-25T10:13:54.938402+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:12:45.869471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.704782+0000","last_change":"2024-04-25T10:13:53.956469+0000","last_active":"2024-04-25T10:13:55.704782+0000","last_peered":"2024-04-25T10:13:55.704782+0000","last_clean":"2024-04-25T10:13:55.704782+0000","last_became_active":"2024-04-25T10:13:53.955971+0000","last_became_peered":"2024-04-25T10:13:53.955971+0000","last_unstale":"2024-04-25T10:13:55.704782+0000","last_undegraded":"2024-04-25T10:13:55.704782+0000","last_fullsized":"2024-04-25T10:13:55.704782+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.634240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.938563+0000","last_change":"2024-04-25T10:13:52.958822+0000","last_active":"2024-04-25T10:13:54.938563+0000","last_peered":"2024-04-25T10:13:54.938563+0000","last_clean":"2024-04-25T10:13:54.938563+0000","last_became_active":"2024-04-25T10:13:52.957856+0000","last_became_peered":"2024-04-25T10:13:52.957856+0000","last_unstale":"2024-04-25T10:13:54.938563+0000","last_undegraded":"2024-04-25T10:13:54.938563+0000","last_fullsized":"2024-04-25T10:13:54.938563+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:47.032097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.161291+0000","last_change":"2024-04-25T10:13:53.961947+0000","last_active":"2024-04-25T10:13:55.161291+0000","last_peered":"2024-04-25T10:13:55.161291+0000","last_clean":"2024-04-25T10:13:55.161291+0000","last_became_active":"2024-04-25T10:13:53.961028+0000","last_became_peered":"2024-04-25T10:13:53.961028+0000","last_unstale":"2024-04-25T10:13:55.161291+0000","last_undegraded":"2024-04-25T10:13:55.161291+0000","last_fullsized":"2024-04-25T10:13:55.161291+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:30:24.518057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.945424+0000","last_change":"2024-04-25T10:13:52.950621+0000","last_active":"2024-04-25T10:13:54.945424+0000","last_peered":"2024-04-25T10:13:54.945424+0000","last_clean":"2024-04-25T10:13:54.945424+0000","last_became_active":"2024-04-25T10:13:52.950352+0000","last_became_peered":"2024-04-25T10:13:52.950352+0000","last_unstale":"2024-04-25T10:13:54.945424+0000","last_undegraded":"2024-04-25T10:13:54.945424+0000","last_fullsized":"2024-04-25T10:13:54.945424+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:09.806566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943559+0000","last_change":"2024-04-25T10:13:53.939586+0000","last_active":"2024-04-25T10:13:54.943559+0000","last_peered":"2024-04-25T10:13:54.943559+0000","last_clean":"2024-04-25T10:13:54.943559+0000","last_became_active":"2024-04-25T10:13:53.939317+0000","last_became_peered":"2024-04-25T10:13:53.939317+0000","last_unstale":"2024-04-25T10:13:54.943559+0000","last_undegraded":"2024-04-25T10:13:54.943559+0000","last_fullsized":"2024-04-25T10:13:54.943559+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:25.479339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702561+0000","last_change":"2024-04-25T10:13:52.948064+0000","last_active":"2024-04-25T10:13:55.702561+0000","last_peered":"2024-04-25T10:13:55.702561+0000","last_clean":"2024-04-25T10:13:55.702561+0000","last_became_active":"2024-04-25T10:13:52.947756+0000","last_became_peered":"2024-04-25T10:13:52.947756+0000","last_unstale":"2024-04-25T10:13:55.702561+0000","last_undegraded":"2024-04-25T10:13:55.702561+0000","last_fullsized":"2024-04-25T10:13:55.702561+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:49:49.620494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703352+0000","last_change":"2024-04-25T10:13:53.961501+0000","last_active":"2024-04-25T10:13:55.703352+0000","last_peered":"2024-04-25T10:13:55.703352+0000","last_clean":"2024-04-25T10:13:55.703352+0000","last_became_active":"2024-04-25T10:13:53.961045+0000","last_became_peered":"2024-04-25T10:13:53.961045+0000","last_unstale":"2024-04-25T10:13:55.703352+0000","last_undegraded":"2024-04-25T10:13:55.703352+0000","last_fullsized":"2024-04-25T10:13:55.703352+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:30:08.634240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702370+0000","last_change":"2024-04-25T10:13:52.948077+0000","last_active":"2024-04-25T10:13:55.702370+0000","last_peered":"2024-04-25T10:13:55.702370+0000","last_clean":"2024-04-25T10:13:55.702370+0000","last_became_active":"2024-04-25T10:13:52.947666+0000","last_became_peered":"2024-04-25T10:13:52.947666+0000","last_unstale":"2024-04-25T10:13:55.702370+0000","last_undegraded":"2024-04-25T10:13:55.702370+0000","last_fullsized":"2024-04-25T10:13:55.702370+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:34.859783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702590+0000","last_change":"2024-04-25T10:13:53.956397+0000","last_active":"2024-04-25T10:13:55.702590+0000","last_peered":"2024-04-25T10:13:55.702590+0000","last_clean":"2024-04-25T10:13:55.702590+0000","last_became_active":"2024-04-25T10:13:53.950697+0000","last_became_peered":"2024-04-25T10:13:53.950697+0000","last_unstale":"2024-04-25T10:13:55.702590+0000","last_undegraded":"2024-04-25T10:13:55.702590+0000","last_fullsized":"2024-04-25T10:13:55.702590+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:44:11.654276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.943571+0000","last_change":"2024-04-25T10:13:52.947940+0000","last_active":"2024-04-25T10:13:54.943571+0000","last_peered":"2024-04-25T10:13:54.943571+0000","last_clean":"2024-04-25T10:13:54.943571+0000","last_became_active":"2024-04-25T10:13:52.947672+0000","last_became_peered":"2024-04-25T10:13:52.947672+0000","last_unstale":"2024-04-25T10:13:54.943571+0000","last_undegraded":"2024-04-25T10:13:54.943571+0000","last_fullsized":"2024-04-25T10:13:54.943571+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:35.167270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:54.933175+0000","last_change":"2024-04-25T10:13:53.956924+0000","last_active":"2024-04-25T10:13:54.933175+0000","last_peered":"2024-04-25T10:13:54.933175+0000","last_clean":"2024-04-25T10:13:54.933175+0000","last_became_active":"2024-04-25T10:13:53.956597+0000","last_became_peered":"2024-04-25T10:13:53.956597+0000","last_unstale":"2024-04-25T10:13:54.933175+0000","last_undegraded":"2024-04-25T10:13:54.933175+0000","last_fullsized":"2024-04-25T10:13:54.933175+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:43.074055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":14,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.701604+0000","last_change":"2024-04-25T10:13:52.959362+0000","last_active":"2024-04-25T10:13:55.701604+0000","last_peered":"2024-04-25T10:13:55.701604+0000","last_clean":"2024-04-25T10:13:55.701604+0000","last_became_active":"2024-04-25T10:13:52.957216+0000","last_became_peered":"2024-04-25T10:13:52.957216+0000","last_unstale":"2024-04-25T10:13:55.701604+0000","last_undegraded":"2024-04-25T10:13:55.701604+0000","last_fullsized":"2024-04-25T10:13:55.701604+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:28.811360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":12,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.703810+0000","last_change":"2024-04-25T10:13:53.962969+0000","last_active":"2024-04-25T10:13:55.703810+0000","last_peered":"2024-04-25T10:13:55.703810+0000","last_clean":"2024-04-25T10:13:55.703810+0000","last_became_active":"2024-04-25T10:13:53.962553+0000","last_became_peered":"2024-04-25T10:13:53.962553+0000","last_unstale":"2024-04-25T10:13:55.703810+0000","last_undegraded":"2024-04-25T10:13:55.703810+0000","last_fullsized":"2024-04-25T10:13:55.703810+0000","mapping_epoch":122,"log_start":"0'0","ondisk_log_start":"0'0","created":122,"last_epoch_clean":123,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:52.906543+0000","last_clean_scrub_stamp":"2024-04-25T10:13:52.906543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:58.988897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"123'8","reported_seq":22,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702757+0000","last_change":"2024-04-25T10:13:52.953569+0000","last_active":"2024-04-25T10:13:55.702757+0000","last_peered":"2024-04-25T10:13:55.702757+0000","last_clean":"2024-04-25T10:13:55.702757+0000","last_became_active":"2024-04-25T10:13:52.953276+0000","last_became_peered":"2024-04-25T10:13:52.953276+0000","last_unstale":"2024-04-25T10:13:55.702757+0000","last_undegraded":"2024-04-25T10:13:55.702757+0000","last_fullsized":"2024-04-25T10:13:55.702757+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:13:51.889296+0000","last_clean_scrub_stamp":"2024-04-25T10:13:51.889296+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:13.341509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":368,"reported_epoch":124,"state":"active+clean","last_fresh":"2024-04-25T10:13:55.702090+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:13:55.702090+0000","last_peered":"2024-04-25T10:13:55.702090+0000","last_clean":"2024-04-25T10:13:55.702090+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:13:55.702090+0000","last_undegraded":"2024-04-25T10:13:55.702090+0000","last_fullsized":"2024-04-25T10:13:55.702090+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51332,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320508,"statfs":{"total":96636764160,"available":96584200192,"internally_reserved":0,"allocated":2154496,"data_stored":1315205,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":50395595},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574997,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54900,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316940,"statfs":{"total":96636764160,"available":96580546560,"internally_reserved":0,"allocated":1548288,"data_stored":724643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574997,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50744,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94321096,"statfs":{"total":96636764160,"available":96584802304,"internally_reserved":0,"allocated":1552384,"data_stored":724221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59068,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312772,"statfs":{"total":96636764160,"available":96576278528,"internally_reserved":0,"allocated":1556480,"data_stored":725673,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834574997,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59080,"kb_used_data":1524,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312760,"statfs":{"total":96636764160,"available":96576266240,"internally_reserved":0,"allocated":1560576,"data_stored":725707,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574996,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54900,"kb_used_data":1508,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316940,"statfs":{"total":96636764160,"available":96580546560,"internally_reserved":0,"allocated":1544192,"data_stored":724061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54904,"kb_used_data":1512,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316936,"statfs":{"total":96636764160,"available":96580542464,"internally_reserved":0,"allocated":1548288,"data_stored":724083,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834574997,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59640,"kb_used_data":2092,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312200,"statfs":{"total":96636764160,"available":96575692800,"internally_reserved":0,"allocated":2142208,"data_stored":1314451,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:13:59.064 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:13:59.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:13:59.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.311+0000 7f212f200640 1 Processor -- start 2024-04-25T10:13:59.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.311+0000 7f212f200640 1 -- start start 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.312+0000 7f212f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f212806ea90 0x7f2128071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.312+0000 7f212f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f212806d2d0 0x7f212806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.312+0000 7f212f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2128162ba0 con 0x7f2128161fc0 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.312+0000 7f212f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f212806dc80 con 0x7f212806ea90 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.312+0000 7f212f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2128165be0 con 0x7f212806d2d0 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f211f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f212806ea90 0x7f2128071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f2127e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2128161fc0 0x7f21281623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35586/0 (socket says 172.21.15.165:35586) 2024-04-25T10:13:59.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f2127e00640 1 -- 172.21.15.165:0/4044372936 learned_addr learned my addr 172.21.15.165:0/4044372936 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:59.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f212d200640 1 --2- 172.21.15.165:0/4044372936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f212806d2d0 0x7f212806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f211f400640 1 -- 172.21.15.165:0/4044372936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f212806d2d0 msgr2=0x7f212806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f211f400640 1 --2- 172.21.15.165:0/4044372936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f212806d2d0 0x7f212806d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f211f400640 1 -- 172.21.15.165:0/4044372936 >> v1:172.21.15.165:6789/0 conn(0x7f2128161fc0 legacy=0x7f21281623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f211f400640 1 -- 172.21.15.165:0/4044372936 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2128162e80 con 0x7f212806ea90 2024-04-25T10:13:59.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.313+0000 7f2127400640 1 -- 172.21.15.165:0/4044372936 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 155723579 0 0) 0x7f2128162ba0 con 0x7f2128161fc0 2024-04-25T10:13:59.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.315+0000 7f211f400640 1 --2- 172.21.15.165:0/4044372936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806ea90 0x7f2128071480 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f21100097d0 tx=0x7f211002a130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=866f1002314dfc04 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.315+0000 7f2127400640 1 -- 172.21.15.165:0/4044372936 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110037070 con 0x7f212806ea90 2024-04-25T10:13:59.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.315+0000 7f2127400640 1 -- 172.21.15.165:0/4044372936 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f211002ae40 con 0x7f212806ea90 2024-04-25T10:13:59.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.315+0000 7f2127400640 1 -- 172.21.15.165:0/4044372936 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110032660 con 0x7f212806ea90 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 -- 172.21.15.165:0/4044372936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806ea90 msgr2=0x7f2128071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 --2- 172.21.15.165:0/4044372936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806ea90 0x7f2128071480 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f21100097d0 tx=0x7f211002a130 comp rx=0 tx=0).stop 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 -- 172.21.15.165:0/4044372936 shutdown_connections 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 --2- 172.21.15.165:0/4044372936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f212806d2d0 0x7f212806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 --2- 172.21.15.165:0/4044372936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806ea90 0x7f2128071480 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 -- 172.21.15.165:0/4044372936 >> 172.21.15.165:0/4044372936 conn(0x7f212806abe0 msgr2=0x7f212806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 -- 172.21.15.165:0/4044372936 shutdown_connections 2024-04-25T10:13:59.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.316+0000 7f212f200640 1 -- 172.21.15.165:0/4044372936 wait complete. 2024-04-25T10:13:59.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.318+0000 7f212f200640 1 Processor -- start 2024-04-25T10:13:59.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.318+0000 7f212f200640 1 -- start start 2024-04-25T10:13:59.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f212f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 0x7f21281f7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f212f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212806ea90 0x7f21280ae120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f212f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2128161fc0 0x7f21280b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f212f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21280719c0 con 0x7f212806d2d0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f212f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2128163440 con 0x7f212806ea90 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f212f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2128071b40 con 0x7f2128161fc0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f2127e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 0x7f21281f7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f2127e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 0x7f21281f7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38190/0 (socket says 172.21.15.165:38190) 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.319+0000 7f2127e00640 1 -- 172.21.15.165:0/2397314452 learned_addr learned my addr 172.21.15.165:0/2397314452 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f211f400640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212806ea90 0x7f21280ae120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f212d200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2128161fc0 0x7f21280b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2127e00640 1 -- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2128161fc0 msgr2=0x7f21280b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2127e00640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2128161fc0 0x7f21280b06f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2127e00640 1 -- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212806ea90 msgr2=0x7f21280ae120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2127e00640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212806ea90 0x7f21280ae120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2127e00640 1 -- 172.21.15.165:0/2397314452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21281f8110 con 0x7f212806d2d0 2024-04-25T10:13:59.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2127e00640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 0x7f21281f7bd0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f211800c430 tx=0x7f211800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21180074e0 con 0x7f212806d2d0 2024-04-25T10:13:59.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21180047e0 con 0x7f212806d2d0 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21280ab570 con 0x7f212806d2d0 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2118004bd0 con 0x7f212806d2d0 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.320+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21280abe10 con 0x7f212806d2d0 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.322+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2128073a00 con 0x7f212806d2d0 2024-04-25T10:13:59.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.323+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2118011070 con 0x7f212806d2d0 2024-04-25T10:13:59.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.323+0000 7f2124c00640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.324+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2118089fc0 con 0x7f212806d2d0 2024-04-25T10:13:59.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.324+0000 7f211f400640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.324+0000 7f211f400640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2110034000 tx=0x7f211002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.328+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2118058ca0 con 0x7f212806d2d0 2024-04-25T10:13:59.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.595+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f21280aea50 con 0x7f212806d2d0 2024-04-25T10:13:59.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.601+0000 7f2124c00640 1 -- 172.21.15.165:0/2397314452 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+147 (secure 0 0 0) 0x7f21180585a0 con 0x7f212806d2d0 2024-04-25T10:13:59.605 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:13:59.606 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:13:59.606 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:13:59.606 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:13:59.606 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:13:59.606 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 msgr2=0x7f210406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2110034000 tx=0x7f211002e000 comp rx=0 tx=0).stop 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 msgr2=0x7f21281f7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 0x7f21281f7bd0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7f211800c430 tx=0x7f211800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 shutdown_connections 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f210406b4d0 0x7f210406d990 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2128161fc0 0x7f21280b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f212806ea90 0x7f21280ae120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 --2- 172.21.15.165:0/2397314452 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f212806d2d0 0x7f21281f7bd0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.604+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 >> 172.21.15.165:0/2397314452 conn(0x7f212806abe0 msgr2=0x7f21280b3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.605+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 shutdown_connections 2024-04-25T10:13:59.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.605+0000 7f212f200640 1 -- 172.21.15.165:0/2397314452 wait complete. 2024-04-25T10:13:59.628 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:13:59.671 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:59.680 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:13:59.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.817+0000 7f8c0a200640 1 Processor -- start 2024-04-25T10:13:59.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.817+0000 7f8c0a200640 1 -- start start 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c0a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8c04165680 0x7f8c040aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c0a200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c040b2650 0x7f8c040cf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c0a200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040cf330 con 0x7f8c040cec70 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c0a200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040cf100 con 0x7f8c04165680 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c0a200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c04072340 con 0x7f8c040b2650 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c02a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8c040cec70 0x7f8c040cc6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35592/0 (socket says 172.21.15.165:35592) 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c02a00640 1 -- 172.21.15.165:0/2793628833 learned_addr learned my addr 172.21.15.165:0/2793628833 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:13:59.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.819+0000 7f8c03400640 1 --2- 172.21.15.165:0/2793628833 >> v2:172.21.15.165:3300/0 conn(0x7f8c04165680 0x7f8c040aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c03e00640 1 --2- 172.21.15.165:0/2793628833 >> v2:172.21.15.172:3300/0 conn(0x7f8c040b2650 0x7f8c040cf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c02000640 1 -- 172.21.15.165:0/2793628833 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694268825 0 0) 0x7f8c040cf330 con 0x7f8c040cec70 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c02000640 1 -- 172.21.15.165:0/2793628833 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8be8003520 con 0x7f8c040cec70 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c03400640 1 -- 172.21.15.165:0/2793628833 >> v2:172.21.15.172:3300/0 conn(0x7f8c040b2650 msgr2=0x7f8c040cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c03400640 1 --2- 172.21.15.165:0/2793628833 >> v2:172.21.15.172:3300/0 conn(0x7f8c040b2650 0x7f8c040cf580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c03400640 1 -- 172.21.15.165:0/2793628833 >> v1:172.21.15.165:6789/0 conn(0x7f8c040cec70 legacy=0x7f8c040cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c03400640 1 -- 172.21.15.165:0/2793628833 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04071d80 con 0x7f8c04165680 2024-04-25T10:13:59.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.820+0000 7f8c03e00640 1 --2- 172.21.15.165:0/2793628833 >> v2:172.21.15.172:3300/0 conn(0x7f8c040b2650 0x7f8c040cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.821+0000 7f8c03400640 1 --2- 172.21.15.165:0/2793628833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c04165680 0x7f8c040aecf0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8c040724c0 tx=0x7f8bf402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c015bed0a6f9a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.822+0000 7f8c02000640 1 -- 172.21.15.165:0/2793628833 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4037070 con 0x7f8c04165680 2024-04-25T10:13:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.822+0000 7f8c02000640 1 -- 172.21.15.165:0/2793628833 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf4032440 con 0x7f8c04165680 2024-04-25T10:13:59.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.822+0000 7f8c02000640 1 -- 172.21.15.165:0/2793628833 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf402ae60 con 0x7f8c04165680 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.822+0000 7f8c0a200640 1 -- 172.21.15.165:0/2793628833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c04165680 msgr2=0x7f8c040aecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.822+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2793628833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c04165680 0x7f8c040aecf0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8c040724c0 tx=0x7f8bf402a110 comp rx=0 tx=0).stop 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.823+0000 7f8c0a200640 1 -- 172.21.15.165:0/2793628833 shutdown_connections 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.823+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2793628833 >> v2:172.21.15.172:3300/0 conn(0x7f8c040b2650 0x7f8c040cf580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.823+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2793628833 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c04165680 0x7f8c040aecf0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.823+0000 7f8c0a200640 1 -- 172.21.15.165:0/2793628833 >> 172.21.15.165:0/2793628833 conn(0x7f8c0406abe0 msgr2=0x7f8c0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.823+0000 7f8c0a200640 1 -- 172.21.15.165:0/2793628833 shutdown_connections 2024-04-25T10:13:59.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.823+0000 7f8c0a200640 1 -- 172.21.15.165:0/2793628833 wait complete. 2024-04-25T10:13:59.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.824+0000 7f8c0a200640 1 Processor -- start 2024-04-25T10:13:59.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 -- start start 2024-04-25T10:13:59.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c040b2650 0x7f8c041f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 0x7f8c041531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c04165680 0x7f8c04151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0406ea30 con 0x7f8c040cec70 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04071670 con 0x7f8c04165680 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.825+0000 7f8c0a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c04165a60 con 0x7f8c040b2650 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 0x7f8c041531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 0x7f8c041531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38218/0 (socket says 172.21.15.165:38218) 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 -- 172.21.15.165:0/2545806880 learned_addr learned my addr 172.21.15.165:0/2545806880 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c03e00640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c04165680 0x7f8c04151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 -- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c040b2650 msgr2=0x7f8c041f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c040b2650 0x7f8c041f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 -- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c04165680 msgr2=0x7f8c04151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c04165680 0x7f8c04151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:13:59.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 -- 172.21.15.165:0/2545806880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c04153920 con 0x7f8c040cec70 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c03e00640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c04165680 0x7f8c04151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.826+0000 7f8c02a00640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 0x7f8c041531b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c460 tx=0x7f8bf000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.827+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0017070 con 0x7f8c040cec70 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.827+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf0005ce0 con 0x7f8c040cec70 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.827+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c04153c00 con 0x7f8c040cec70 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.827+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0007290 con 0x7f8c040cec70 2024-04-25T10:13:59.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.827+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c04157030 con 0x7f8c040cec70 2024-04-25T10:13:59.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.828+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c04157230 con 0x7f8c040cec70 2024-04-25T10:13:59.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.829+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bf001a070 con 0x7f8c040cec70 2024-04-25T10:13:59.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.830+0000 7f8bfb400640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8be406b4d0 0x7f8be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:13:59.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.831+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8bf00897b0 con 0x7f8c040cec70 2024-04-25T10:13:59.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.831+0000 7f8c03400640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8be406b4d0 0x7f8be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:13:59.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.831+0000 7f8c03400640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8be406b4d0 0x7f8be406d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4034000 tx=0x7f8bf402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:13:59.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:13:59.835+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c04157230 con 0x7f8c040cec70 2024-04-25T10:14:00.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.068+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c04154120 con 0x7f8c040cec70 2024-04-25T10:14:00.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.069+0000 7f8bfb400640 1 -- 172.21.15.165:0/2545806880 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f8bf0058400 con 0x7f8c040cec70 2024-04-25T10:14:00.074 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:00.074 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:00.074 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:00.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8be406b4d0 msgr2=0x7f8be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8be406b4d0 0x7f8be406d990 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4034000 tx=0x7f8bf402c000 comp rx=0 tx=0).stop 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 msgr2=0x7f8c041531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 0x7f8c041531b0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000c460 tx=0x7f8bf000c920 comp rx=0 tx=0).stop 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 shutdown_connections 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8be406b4d0 0x7f8be406d990 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c04165680 0x7f8c04151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c040cec70 0x7f8c041531b0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 --2- 172.21.15.165:0/2545806880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c040b2650 0x7f8c041f81d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.072+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 >> 172.21.15.165:0/2545806880 conn(0x7f8c0406abe0 msgr2=0x7f8c04163290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.073+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 shutdown_connections 2024-04-25T10:14:00.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.073+0000 7f8c0a200640 1 -- 172.21.15.165:0/2545806880 wait complete. 2024-04-25T10:14:00.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 82, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:13:53.389326+0000', 'modified': '2024-04-25T10:13:56.618933+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6136}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6136': {'gid': 6136, 'name': 'd', 'rank': 0, 'incarnation': 79, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6834/3899404305', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 3899404305}, {'type': 'v1', 'addr': '172.21.15.172:6834', 'nonce': 3899404305}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:14:00.498 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:14:00.498 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:14:00.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.701+0000 7fe7ce800640 1 Processor -- start 2024-04-25T10:14:00.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.702+0000 7fe7ce800640 1 -- start start 2024-04-25T10:14:00.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.703+0000 7fe7ce800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe7c80b5ae0 0x7fe7c8168950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.703+0000 7fe7ce800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c8161e70 con 0x7fe7c815dbd0 2024-04-25T10:14:00.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.703+0000 7fe7ce800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7c8162150 con 0x7fe7c80b95c0 2024-04-25T10:14:00.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.703+0000 7fe7ce800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7c80d1ce0 con 0x7fe7c80b5ae0 2024-04-25T10:14:00.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.703+0000 7fe7cd400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe7c815dbd0 0x7fe7c8161620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35608/0 (socket says 172.21.15.165:35608) 2024-04-25T10:14:00.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.703+0000 7fe7cd400640 1 -- 172.21.15.165:0/2388039976 learned_addr learned my addr 172.21.15.165:0/2388039976 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:00.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7cde00640 1 --2- 172.21.15.165:0/2388039976 >> v2:172.21.15.172:3300/0 conn(0x7fe7c80b5ae0 0x7fe7c8168950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967727098 0 0) 0x7fe7c8161e70 con 0x7fe7c815dbd0 2024-04-25T10:14:00.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7b0003520 con 0x7fe7c815dbd0 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 450933846 0 0) 0x7fe7b0003520 con 0x7fe7c815dbd0 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 >> v2:172.21.15.172:3300/0 conn(0x7fe7c80b5ae0 msgr2=0x7fe7c8168950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 --2- 172.21.15.165:0/2388039976 >> v2:172.21.15.172:3300/0 conn(0x7fe7c80b5ae0 0x7fe7c8168950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 >> v1:172.21.15.172:6789/0 conn(0x7fe7c80b95c0 legacy=0x7fe7c80b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8162430 con 0x7fe7c815dbd0 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe7bc001670 con 0x7fe7c815dbd0 2024-04-25T10:14:00.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 >> v1:172.21.15.165:6789/0 conn(0x7fe7c815dbd0 legacy=0x7fe7c8161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7b0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7b0004800 0x7fe7b0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7b00051c0 0x7fe7b000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80d1ce0 con 0x7fe7b00039b0 2024-04-25T10:14:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b000c060 con 0x7fe7b00051c0 2024-04-25T10:14:00.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.704+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b000c260 con 0x7fe7b0004800 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7b0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cd400640 1 -- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7b0004800 msgr2=0x7fe7b0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7b0004800 0x7fe7b0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cd400640 1 -- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7b00051c0 msgr2=0x7fe7b000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cde00640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7b00051c0 0x7fe7b000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7b00051c0 0x7fe7b000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cde00640 1 -- 172.21.15.165:0/2388039976 reap_dead start 2024-04-25T10:14:00.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.705+0000 7fe7cd400640 1 -- 172.21.15.165:0/2388039976 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b0003520 con 0x7fe7b00039b0 2024-04-25T10:14:00.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.706+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7b0003dd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c240 tx=0x7fe7b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29e22e20d8fce6cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.706+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc007d10 con 0x7fe7b00039b0 2024-04-25T10:14:00.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.706+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc005ad0 con 0x7fe7b00039b0 2024-04-25T10:14:00.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.706+0000 7fe7c7e00640 1 -- 172.21.15.165:0/2388039976 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc007070 con 0x7fe7b00039b0 2024-04-25T10:14:00.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 -- 172.21.15.165:0/2388039976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 msgr2=0x7fe7b0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7b0003dd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c240 tx=0x7fe7b0005830 comp rx=0 tx=0).stop 2024-04-25T10:14:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 -- 172.21.15.165:0/2388039976 shutdown_connections 2024-04-25T10:14:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2388039976 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7b0003dd0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 -- 172.21.15.165:0/2388039976 >> 172.21.15.165:0/2388039976 conn(0x7fe7c80d03d0 msgr2=0x7fe7c80d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 -- 172.21.15.165:0/2388039976 shutdown_connections 2024-04-25T10:14:00.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.707+0000 7fe7ce800640 1 -- 172.21.15.165:0/2388039976 wait complete. 2024-04-25T10:14:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.709+0000 7fe7ce800640 1 Processor -- start 2024-04-25T10:14:00.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.709+0000 7fe7ce800640 1 -- start start 2024-04-25T10:14:00.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7ce800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7c8148c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7ce800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7c8159a40 0x7fe7c8159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7ce800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7c814ad00 0x7fe7c814b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7ce800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80b5550 con 0x7fe7b00039b0 2024-04-25T10:14:00.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7ce800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80b53d0 con 0x7fe7c814ad00 2024-04-25T10:14:00.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7ce800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c80b56d0 con 0x7fe7c8159a40 2024-04-25T10:14:00.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7c8148c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7c8148c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38234/0 (socket says 172.21.15.165:38234) 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 -- 172.21.15.165:0/2461058318 learned_addr learned my addr 172.21.15.165:0/2461058318 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cde00640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7c814ad00 0x7fe7c814b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cca00640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7c8159a40 0x7fe7c8159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 -- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7c8159a40 msgr2=0x7fe7c8159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7c8159a40 0x7fe7c8159ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 -- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7c814ad00 msgr2=0x7fe7c814b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7c814ad00 0x7fe7c814b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.711+0000 7fe7cd400640 1 -- 172.21.15.165:0/2461058318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7c8149180 con 0x7fe7b00039b0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.712+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7c8148c40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c240 tx=0x7fe7c80b6230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.712+0000 7fe7c5600640 1 -- 172.21.15.165:0/2461058318 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc03f020 con 0x7fe7b00039b0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.712+0000 7fe7c5600640 1 -- 172.21.15.165:0/2461058318 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc001430 con 0x7fe7b00039b0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.712+0000 7fe7c5600640 1 -- 172.21.15.165:0/2461058318 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0047e0 con 0x7fe7b00039b0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.712+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7c814bab0 con 0x7fe7b00039b0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.712+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7c80cbfa0 con 0x7fe7b00039b0 2024-04-25T10:14:00.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.714+0000 7fe7c5600640 1 -- 172.21.15.165:0/2461058318 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7bc042020 con 0x7fe7b00039b0 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.716+0000 7fe7c5600640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7a806b500 0x7fe7a806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.716+0000 7fe7c5600640 1 -- 172.21.15.165:0/2461058318 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7bc0a9f30 con 0x7fe7b00039b0 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.716+0000 7fe7cca00640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7a806b500 0x7fe7a806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.717+0000 7fe7d2c71740 1 Processor -- start 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.717+0000 7fe7d2c71740 1 -- start start 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.717+0000 7fe7cca00640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7a806b500 0x7fe7a806d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80c9ba0 tx=0x7fe7b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0e8cb30 0x55a2c0eb4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0e8cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0e8b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0df0620 con 0x55a2c0ed57a0 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0d99ca0 con 0x55a2c0e8cb30 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0dec6c0 con 0x55a2c0eb4eb0 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0e8b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0e8b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38238/0 (socket says 172.21.15.165:38238) 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 -- 172.21.15.165:0/1895989849 learned_addr learned my addr 172.21.15.165:0/1895989849 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 -- 172.21.15.165:0/1895989849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 msgr2=0x55a2c0e8cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0e8cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 -- 172.21.15.165:0/1895989849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0e8cb30 msgr2=0x55a2c0eb4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0e8cb30 0x55a2c0eb4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.718+0000 7fe7cde00640 1 -- 172.21.15.165:0/1895989849 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c0da0f40 con 0x55a2c0ed57a0 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.719+0000 7fe7cde00640 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0e8b2f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe7c806b850 tx=0x7fe7c000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bc13f94a96660aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.719+0000 7fe7b9c00640 1 -- 172.21.15.165:0/1895989849 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0003e20 con 0x55a2c0ed57a0 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.719+0000 7fe7b9c00640 1 -- 172.21.15.165:0/1895989849 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c001a070 con 0x55a2c0ed57a0 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.719+0000 7fe7b9c00640 1 -- 172.21.15.165:0/1895989849 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0015420 con 0x55a2c0ed57a0 2024-04-25T10:14:00.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.719+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1895989849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 msgr2=0x55a2c0e8b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.719+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0e8b2f0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fe7c806b850 tx=0x7fe7c000b040 comp rx=0 tx=0).stop 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1895989849 shutdown_connections 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0e8b2f0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0e8cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1895989849 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0e8cb30 0x55a2c0eb4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1895989849 >> 172.21.15.165:0/1895989849 conn(0x55a2c0e89d50 msgr2=0x55a2c0ed2730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1895989849 shutdown_connections 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1895989849 wait complete. 2024-04-25T10:14:00.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 -- start start 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.720+0000 7fe7d2c71740 1 Processor -- start 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.721+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0eb2d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.721+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0eb56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.721+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb84c0 0x55a2c0eb8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.721+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0eb44e0 con 0x55a2c0ed57a0 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.721+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0d8f1f0 con 0x55a2c0eb84c0 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.721+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0eb3ed0 con 0x55a2c0eb4eb0 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7d2c71740 10 client.0 _ll_register_callbacks cb 0x55a2c0dacad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:00.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0eb56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0eb56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38250/0 (socket says 172.21.15.165:38250) 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 -- 172.21.15.165:0/2621963197 learned_addr learned my addr 172.21.15.165:0/2621963197 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 -- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 msgr2=0x55a2c0eb2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0eb2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cde00640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb84c0 0x55a2c0eb8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0eb2d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 -- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb84c0 msgr2=0x55a2c0eb8940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb84c0 0x55a2c0eb8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 -- 172.21.15.165:0/2621963197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2c0daec50 con 0x55a2c0ed57a0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.722+0000 7fe7cca00640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0eb56d0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4014ae0 tx=0x7fe7b4014ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a20753fa1619e7a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7ae000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7ae000640 1 -- 172.21.15.165:0/2621963197 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b400dac0 con 0x55a2c0ed57a0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7ae000640 1 -- 172.21.15.165:0/2621963197 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7b4022c60 con 0x55a2c0ed57a0 2024-04-25T10:14:00.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7c001a070 con 0x55a2c0ed57a0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7ae000640 1 -- 172.21.15.165:0/2621963197 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7b4022e60 con 0x55a2c0ed57a0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7d2c71740 10 client.8742 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a2c0b052d0 con 0x55a2c0ed57a0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7ae000640 1 -- 172.21.15.165:0/2621963197 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7c001a070 con 0x55a2c0ed57a0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7ae000640 1 -- 172.21.15.165:0/2621963197 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe7b4029050 con 0x55a2c0ed57a0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7d2c71740 10 client.8742 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.723+0000 7fe7d2c71740 10 client.8742 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.724+0000 7fe7d2c71740 20 client.8742 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.724+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0eee420 0x55a2c0ef0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.724+0000 7fe7d2c71740 4 client.8742 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.724+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2c0dfdb60 con 0x55a2c0eee420 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.724+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0eee420 0x55a2c0ef0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.725+0000 7fe7cd400640 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0eee420 0x55a2c0ef0810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e9f4b6d0da4d8d88 server_cookie=f471305594778616 in_seq=0 out_seq=0 2024-04-25T10:14:00.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.725+0000 7fe7ae000640 10 client.8742 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7ae000640 1 -- 172.21.15.165:0/2621963197 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a2c0dfdb60 con 0x55a2c0eee420 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7ae000640 10 client.8742 handle_command_reply: tid=0 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 client.8742 shutdown 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 msgr2=0x55a2c0eb56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0eb56d0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fe7b4014ae0 tx=0x7fe7b4014ee0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 shutdown_connections 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0eee420 0x55a2c0ef0810 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb84c0 0x55a2c0eb8940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0eb56d0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7ae000640 0 client.8742 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/2621963197 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0eb2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 >> 172.21.15.165:0/2621963197 conn(0x55a2c0e8cb30 msgr2=0x55a2c0ed2530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.727+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 shutdown_connections 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.728+0000 7fe7d2c71740 1 -- 172.21.15.165:0/2621963197 wait complete. 2024-04-25T10:14:00.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.728+0000 7fe7d2c71740 20 client.8742 trim_cache size 0 max 16384 2024-04-25T10:14:00.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 Processor -- start 2024-04-25T10:14:00.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 -- start start 2024-04-25T10:14:00.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0ed1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 0x55a2c0e8b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ee52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81ffd90 con 0x55a2c0eb84c0 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c815c680 con 0x55a2c0ed57a0 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.745+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7c81ff790 con 0x55a2c0eb4eb0 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 0x55a2c0e8b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 0x55a2c0e8b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38260/0 (socket says 172.21.15.165:38260) 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 -- 172.21.15.165:0/1436371007 learned_addr learned my addr 172.21.15.165:0/1436371007 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cde00640 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ee52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 -- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 msgr2=0x55a2c0ed1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0ed1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 -- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0ed57a0 msgr2=0x55a2c0ee52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ee52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cca00640 1 -- 172.21.15.165:0/1436371007 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2c0eb8cb0 con 0x55a2c0eb84c0 2024-04-25T10:14:00.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.746+0000 7fe7cde00640 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ee52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:00.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.747+0000 7fe7cca00640 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 0x55a2c0e8b4e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40054e0 tx=0x7fe7b401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6ae3211f1fd7e5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.747+0000 7fe7b9200640 1 -- 172.21.15.165:0/1436371007 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b40035f0 con 0x55a2c0eb84c0 2024-04-25T10:14:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.747+0000 7fe7b9200640 1 -- 172.21.15.165:0/1436371007 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b4003790 con 0x55a2c0eb84c0 2024-04-25T10:14:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.747+0000 7fe7b9200640 1 -- 172.21.15.165:0/1436371007 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4003b40 con 0x55a2c0eb84c0 2024-04-25T10:14:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.747+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1436371007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 msgr2=0x55a2c0e8b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.747+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 0x55a2c0e8b4e0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fe7b40054e0 tx=0x7fe7b401c000 comp rx=0 tx=0).stop 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1436371007 shutdown_connections 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ee52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0eb84c0 0x55a2c0e8b4e0 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/1436371007 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb4eb0 0x55a2c0ed1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1436371007 >> 172.21.15.165:0/1436371007 conn(0x55a2c0e8cb30 msgr2=0x55a2c0ed29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1436371007 shutdown_connections 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 -- 172.21.15.165:0/1436371007 wait complete. 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 -- start start 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.748+0000 7fe7d2c71740 1 Processor -- start 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb4eb0 0x55a2c0e9fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb84c0 0x55a2c0ea0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ed5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0eb9540 con 0x55a2c0ed57a0 2024-04-25T10:14:00.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0eb6400 con 0x55a2c0eb4eb0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2c0ea1e80 con 0x55a2c0eb84c0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.749+0000 7fe7d2c71740 10 client.0 _ll_register_callbacks cb 0x7fe7b4014760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ed5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ed5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38264/0 (socket says 172.21.15.165:38264) 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 -- 172.21.15.165:0/481424578 learned_addr learned my addr 172.21.15.165:0/481424578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cd400640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb4eb0 0x55a2c0e9fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 -- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb84c0 msgr2=0x55a2c0ea0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cca00640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb84c0 0x55a2c0ea0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb84c0 0x55a2c0ea0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 -- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb4eb0 msgr2=0x55a2c0e9fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb4eb0 0x55a2c0e9fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 -- 172.21.15.165:0/481424578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2c0bcf6b0 con 0x55a2c0ed57a0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7cde00640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ed5b80 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0002230 tx=0x7fe7c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b8e244fb3d7a6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.750+0000 7fe7ae000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:00.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7ae000640 1 -- 172.21.15.165:0/481424578 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7c0028470 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7ae000640 1 -- 172.21.15.165:0/481424578 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe7c0028610 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe7b4003790 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7ae000640 1 -- 172.21.15.165:0/481424578 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7c0030050 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 10 client.8748 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a2c0ebb030 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7ae000640 1 -- 172.21.15.165:0/481424578 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b4003790 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7ae000640 1 -- 172.21.15.165:0/481424578 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe7c0033020 con 0x55a2c0ed57a0 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 10 client.8748 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 10 client.8748 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 20 client.8748 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0ed60c0 0x55a2c0e868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 4 client.8748 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:14:00.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.751+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2c0df7eb0 con 0x55a2c0ed60c0 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.752+0000 7fe7cd400640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0ed60c0 0x55a2c0e868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.753+0000 7fe7cd400640 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0ed60c0 0x55a2c0e868b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f63dabd0bcd49aa server_cookie=fab1aea663bf82ea in_seq=0 out_seq=0 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.753+0000 7fe7ae000640 10 client.8748 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.753+0000 7fe7ae000640 1 -- 172.21.15.165:0/481424578 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55a2c0df7eb0 con 0x55a2c0ed60c0 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.753+0000 7fe7ae000640 10 client.8748 handle_command_reply: tid=0 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 client.8748 shutdown 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 msgr2=0x55a2c0ed5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ed5b80 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fe7c0002230 tx=0x7fe7c0004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 shutdown_connections 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a2c0ed60c0 0x55a2c0e868b0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a2c0ed57a0 0x55a2c0ed5b80 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a2c0eb84c0 0x55a2c0ea0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 --2- 172.21.15.165:0/481424578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a2c0eb4eb0 0x55a2c0e9fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7ae000640 0 client.8748 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 >> 172.21.15.165:0/481424578 conn(0x55a2c0e8cb30 msgr2=0x55a2c0eb96c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 shutdown_connections 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 1 -- 172.21.15.165:0/481424578 wait complete. 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.754+0000 7fe7d2c71740 20 client.8748 trim_cache size 0 max 16384 2024-04-25T10:14:00.761 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 6136, 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.8256871840000004, 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 82, 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 124, 2024-04-25T10:14:00.762 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 124, 2024-04-25T10:14:00.763 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 12.293401728999999 2024-04-25T10:14:00.763 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:00.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7a806b500 msgr2=0x7fe7a806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7a806b500 0x7fe7a806d9c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fe7c80c9ba0 tx=0x7fe7b4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 msgr2=0x7fe7c8148c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7c8148c40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00c240 tx=0x7fe7c80b6230 comp rx=0 tx=0).stop 2024-04-25T10:14:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 shutdown_connections 2024-04-25T10:14:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7a806b500 0x7fe7a806d9c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7c814ad00 0x7fe7c814b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7c8159a40 0x7fe7c8159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 --2- 172.21.15.165:0/2461058318 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7b00039b0 0x7fe7c8148c40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 >> 172.21.15.165:0/2461058318 conn(0x7fe7c80d03d0 msgr2=0x7fe7c80b4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 shutdown_connections 2024-04-25T10:14:00.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.755+0000 7fe7ce800640 1 -- 172.21.15.165:0/2461058318 wait complete. 2024-04-25T10:14:00.776 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:14:00.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.959+0000 7f11a9000640 1 Processor -- start 2024-04-25T10:14:00.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.960+0000 7f11a9000640 1 -- start start 2024-04-25T10:14:00.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.961+0000 7f11a9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a415b2c0 0x7f11a406fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.961+0000 7f11a9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11a415b6a0 0x7f11a40b2da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.961+0000 7f11a9000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11a4070170 con 0x7f11a40b32e0 2024-04-25T10:14:00.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a9000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11a40adbe0 con 0x7f11a415b6a0 2024-04-25T10:14:00.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40adec0 con 0x7f11a415b2c0 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11a415b6a0 0x7f11a40b2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11a415b6a0 0x7f11a40b2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38270/0 (socket says 172.21.15.165:38270) 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 -- 172.21.15.165:0/2018991537 learned_addr learned my addr 172.21.15.165:0/2018991537 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a3e00640 1 --1- 172.21.15.165:0/2018991537 >> v1:172.21.15.165:6789/0 conn(0x7f11a40b32e0 0x7f11a40ae6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:35622/0 (socket says 172.21.15.165:35622) 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a3400640 1 --2- 172.21.15.165:0/2018991537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a415b2c0 0x7f11a406fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 -- 172.21.15.165:0/2018991537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a415b2c0 msgr2=0x7f11a406fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 --2- 172.21.15.165:0/2018991537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a415b2c0 0x7f11a406fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 -- 172.21.15.165:0/2018991537 >> v1:172.21.15.165:6789/0 conn(0x7f11a40b32e0 legacy=0x7f11a40ae6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2a00640 1 -- 172.21.15.165:0/2018991537 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a415bb00 con 0x7f11a415b6a0 2024-04-25T10:14:00.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.962+0000 7f11a2000640 1 -- 172.21.15.165:0/2018991537 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3514504605 0 0) 0x7f11a4070170 con 0x7f11a40b32e0 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.963+0000 7f11a2a00640 1 --2- 172.21.15.165:0/2018991537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b6a0 0x7f11a40b2da0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1190009870 tx=0x7f119002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f472ac5159ace server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a2000640 1 -- 172.21.15.165:0/2018991537 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190037070 con 0x7f11a415b6a0 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a2000640 1 -- 172.21.15.165:0/2018991537 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1190032440 con 0x7f11a415b6a0 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a2000640 1 -- 172.21.15.165:0/2018991537 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190032810 con 0x7f11a415b6a0 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a9000640 1 -- 172.21.15.165:0/2018991537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b6a0 msgr2=0x7f11a40b2da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a9000640 1 --2- 172.21.15.165:0/2018991537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b6a0 0x7f11a40b2da0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f1190009870 tx=0x7f119002a270 comp rx=0 tx=0).stop 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a9000640 1 -- 172.21.15.165:0/2018991537 shutdown_connections 2024-04-25T10:14:00.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a9000640 1 --2- 172.21.15.165:0/2018991537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b6a0 0x7f11a40b2da0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a9000640 1 --2- 172.21.15.165:0/2018991537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a415b2c0 0x7f11a406fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.964+0000 7f11a9000640 1 -- 172.21.15.165:0/2018991537 >> 172.21.15.165:0/2018991537 conn(0x7f11a40cc7f0 msgr2=0x7f11a40ccbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:00.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.965+0000 7f11a9000640 1 -- 172.21.15.165:0/2018991537 shutdown_connections 2024-04-25T10:14:00.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.965+0000 7f11a9000640 1 -- 172.21.15.165:0/2018991537 wait complete. 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.966+0000 7f11a9000640 1 Processor -- start 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.967+0000 7f11a9000640 1 -- start start 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.967+0000 7f11a9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a40b32e0 0x7f11a41531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.967+0000 7f11a9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 0x7f11a414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a9000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a415b6a0 0x7f11a40cb500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a9000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40aef50 con 0x7f11a415b2c0 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a9000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a40aedd0 con 0x7f11a415b6a0 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a406abe0 con 0x7f11a40b32e0 2024-04-25T10:14:00.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 0x7f11a414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 0x7f11a414e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38280/0 (socket says 172.21.15.165:38280) 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 -- 172.21.15.165:0/527112408 learned_addr learned my addr 172.21.15.165:0/527112408 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 -- 172.21.15.165:0/527112408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a40b32e0 msgr2=0x7f11a41531b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a40b32e0 0x7f11a41531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 -- 172.21.15.165:0/527112408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a415b6a0 msgr2=0x7f11a40cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a415b6a0 0x7f11a40cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.968+0000 7f11a2a00640 1 -- 172.21.15.165:0/527112408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a40c9b30 con 0x7f11a415b2c0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.969+0000 7f11a2a00640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 0x7f11a414e8f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f1190000c00 tx=0x7f11900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.969+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190041020 con 0x7f11a415b2c0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.969+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11900306c0 con 0x7f11a415b2c0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.969+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1190037040 con 0x7f11a415b2c0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.969+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11a40c91c0 con 0x7f11a415b2c0 2024-04-25T10:14:00.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.969+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11a40c94a0 con 0x7f11a415b2c0 2024-04-25T10:14:00.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.970+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11a4152e90 con 0x7f11a415b2c0 2024-04-25T10:14:00.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.971+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1190044020 con 0x7f11a415b2c0 2024-04-25T10:14:00.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.972+0000 7f119b400640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118406b4d0 0x7f118406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:00.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.972+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f11900aab50 con 0x7f11a415b2c0 2024-04-25T10:14:00.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.972+0000 7f11a3400640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118406b4d0 0x7f118406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:00.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.973+0000 7f11a3400640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118406b4d0 0x7f118406d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f11940098f0 tx=0x7f1194008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:00.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:00.977+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11a4152e90 con 0x7f11a415b2c0 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.214+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f11a4073990 con 0x7f11a415b2c0 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.215+0000 7f119b400640 1 -- 172.21.15.165:0/527112408 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v82) v1 ==== 53+0+73 (secure 0 0 0) 0x7f119004b030 con 0x7f11a415b2c0 2024-04-25T10:14:01.219 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:14:01.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118406b4d0 msgr2=0x7f118406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118406b4d0 0x7f118406d990 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f11940098f0 tx=0x7f1194008040 comp rx=0 tx=0).stop 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 msgr2=0x7f11a414e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 0x7f11a414e8f0 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f1190000c00 tx=0x7f11900046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 shutdown_connections 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118406b4d0 0x7f118406d990 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a415b6a0 0x7f11a40cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a415b2c0 0x7f11a414e8f0 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 --2- 172.21.15.165:0/527112408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a40b32e0 0x7f11a41531b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 >> 172.21.15.165:0/527112408 conn(0x7f11a40cc7f0 msgr2=0x7f11a415e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:01.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 shutdown_connections 2024-04-25T10:14:01.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:01.218+0000 7f11a9000640 1 -- 172.21.15.165:0/527112408 wait complete. 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:14:01.242 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:01.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.243 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:14:01.293 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:14:01.293 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:14:01.293 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:14:01.293 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.293 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:14:01.294 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41924sec preferred_lft 41924sec 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:01.295 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:14:01.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.296 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:14:01.373 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:14:01.377 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.377 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:14:01.430 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:14:01.430 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:14:01.430 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:14:01.430 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:14:01.430 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:14:01.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.431 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:14:01.431 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:01.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:14:01.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:14:01.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:14:01.431 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:01.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.581 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:14:01.631 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:14:01.632 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:14:01.632 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:01.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.632 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:14:01.714 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:01.714 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:01.769 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:14:01.770 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:14:01.770 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:01.822 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:14:01.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:01.823 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:01.954 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:01.954 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:01.955 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:14:01.979 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:01.979 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:01.979 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:14:01.979 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:14:01.979 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:02.005 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:14:02.005 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:14:02.005 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:14:02.005 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:14:02.005 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:14:02.005 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:14:02.006 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:14:02.006 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:14:02.008 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:14:02.009 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:14:02.009 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:14:02.009 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:14:02.009 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:14:02.013 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:14:02.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:02.014 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:02.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:02.084 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:14:02.158 INFO:teuthology.orchestra.run.smithi165.stdout:client8760 2024-04-25T10:14:02.159 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:14:02.159 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:02.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:02.210 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:02.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0332844 s, 31.5 MB/s 2024-04-25T10:14:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0428346 s, 24.5 MB/s 2024-04-25T10:14:02.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.471 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0454748 s, 23.1 MB/s 2024-04-25T10:14:02.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0290883 s, 36.0 MB/s 2024-04-25T10:14:02.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367269 s, 28.6 MB/s 2024-04-25T10:14:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231566 s, 45.3 MB/s 2024-04-25T10:14:02.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.612 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.03549 s, 29.5 MB/s 2024-04-25T10:14:02.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.643 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.643 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025843 s, 40.6 MB/s 2024-04-25T10:14:02.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.668 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.668 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211878 s, 49.5 MB/s 2024-04-25T10:14:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235373 s, 44.5 MB/s 2024-04-25T10:14:02.717 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.717 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.717 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185252 s, 56.6 MB/s 2024-04-25T10:14:02.746 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.746 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.746 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247981 s, 42.3 MB/s 2024-04-25T10:14:02.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.771 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199961 s, 52.4 MB/s 2024-04-25T10:14:02.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.803 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277577 s, 37.8 MB/s 2024-04-25T10:14:02.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0367204 s, 28.6 MB/s 2024-04-25T10:14:02.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.870 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022973 s, 45.6 MB/s 2024-04-25T10:14:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215331 s, 48.7 MB/s 2024-04-25T10:14:02.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203183 s, 51.6 MB/s 2024-04-25T10:14:02.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.947 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224089 s, 46.8 MB/s 2024-04-25T10:14:02.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:02.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:02.975 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242369 s, 43.3 MB/s 2024-04-25T10:14:03.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.008 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0293081 s, 35.8 MB/s 2024-04-25T10:14:03.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.037 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241182 s, 43.5 MB/s 2024-04-25T10:14:03.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.060 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192503 s, 54.5 MB/s 2024-04-25T10:14:03.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.089 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251552 s, 41.7 MB/s 2024-04-25T10:14:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.115 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222795 s, 47.1 MB/s 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021819 s, 48.1 MB/s 2024-04-25T10:14:03.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205748 s, 51.0 MB/s 2024-04-25T10:14:03.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218704 s, 47.9 MB/s 2024-04-25T10:14:03.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.212 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180078 s, 58.2 MB/s 2024-04-25T10:14:03.237 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.237 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.237 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215695 s, 48.6 MB/s 2024-04-25T10:14:03.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.262 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199137 s, 52.7 MB/s 2024-04-25T10:14:03.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204588 s, 51.3 MB/s 2024-04-25T10:14:03.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:03.288 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:03.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.377 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245395 s, 42.7 MB/s 2024-04-25T10:14:03.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.404 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237137 s, 44.2 MB/s 2024-04-25T10:14:03.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.431 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.431 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230365 s, 45.5 MB/s 2024-04-25T10:14:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279378 s, 37.5 MB/s 2024-04-25T10:14:03.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260112 s, 40.3 MB/s 2024-04-25T10:14:03.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259182 s, 40.5 MB/s 2024-04-25T10:14:03.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.557 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307852 s, 34.1 MB/s 2024-04-25T10:14:03.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.587 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261631 s, 40.1 MB/s 2024-04-25T10:14:03.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.611 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200631 s, 52.3 MB/s 2024-04-25T10:14:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.636 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210578 s, 49.8 MB/s 2024-04-25T10:14:03.660 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.660 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.660 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207399 s, 50.6 MB/s 2024-04-25T10:14:03.685 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.685 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.685 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211711 s, 49.5 MB/s 2024-04-25T10:14:03.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206675 s, 50.7 MB/s 2024-04-25T10:14:03.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251476 s, 41.7 MB/s 2024-04-25T10:14:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236827 s, 44.3 MB/s 2024-04-25T10:14:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199557 s, 52.5 MB/s 2024-04-25T10:14:03.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.814 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189431 s, 55.4 MB/s 2024-04-25T10:14:03.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021709 s, 48.3 MB/s 2024-04-25T10:14:03.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254531 s, 41.2 MB/s 2024-04-25T10:14:03.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.894 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198136 s, 52.9 MB/s 2024-04-25T10:14:03.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.917 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188057 s, 55.8 MB/s 2024-04-25T10:14:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227611 s, 46.1 MB/s 2024-04-25T10:14:03.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.968 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186373 s, 56.3 MB/s 2024-04-25T10:14:03.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:03.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:03.993 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215901 s, 48.6 MB/s 2024-04-25T10:14:04.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.020 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022732 s, 46.1 MB/s 2024-04-25T10:14:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205687 s, 51.0 MB/s 2024-04-25T10:14:04.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.070 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211589 s, 49.6 MB/s 2024-04-25T10:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.096 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219363 s, 47.8 MB/s 2024-04-25T10:14:04.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.119 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195338 s, 53.7 MB/s 2024-04-25T10:14:04.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.146 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235324 s, 44.6 MB/s 2024-04-25T10:14:04.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.171 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202771 s, 51.7 MB/s 2024-04-25T10:14:04.197 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.197 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.197 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223383 s, 46.9 MB/s 2024-04-25T10:14:04.200 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:04.200 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:04.200 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:04.200 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:14:04.200 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:04.200 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:04.200 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:04.201 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:04.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229809 s, 45.6 MB/s 2024-04-25T10:14:04.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.308 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181657 s, 57.7 MB/s 2024-04-25T10:14:04.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.332 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214084 s, 49.0 MB/s 2024-04-25T10:14:04.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.356 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209652 s, 50.0 MB/s 2024-04-25T10:14:04.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.381 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217982 s, 48.1 MB/s 2024-04-25T10:14:04.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227982 s, 46.0 MB/s 2024-04-25T10:14:04.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257224 s, 40.8 MB/s 2024-04-25T10:14:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.464 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209866 s, 50.0 MB/s 2024-04-25T10:14:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164497 s, 63.7 MB/s 2024-04-25T10:14:04.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.512 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024033 s, 43.6 MB/s 2024-04-25T10:14:04.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201938 s, 51.9 MB/s 2024-04-25T10:14:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.565 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245516 s, 42.7 MB/s 2024-04-25T10:14:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213171 s, 49.2 MB/s 2024-04-25T10:14:04.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.619 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.619 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249031 s, 42.1 MB/s 2024-04-25T10:14:04.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022353 s, 46.9 MB/s 2024-04-25T10:14:04.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.670 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207103 s, 50.6 MB/s 2024-04-25T10:14:04.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.697 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02356 s, 44.5 MB/s 2024-04-25T10:14:04.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.724 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226213 s, 46.4 MB/s 2024-04-25T10:14:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.750 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225091 s, 46.6 MB/s 2024-04-25T10:14:04.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203426 s, 51.5 MB/s 2024-04-25T10:14:04.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202319 s, 51.8 MB/s 2024-04-25T10:14:04.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198618 s, 52.8 MB/s 2024-04-25T10:14:04.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213947 s, 49.0 MB/s 2024-04-25T10:14:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.874 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203148 s, 51.6 MB/s 2024-04-25T10:14:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.900 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219018 s, 47.9 MB/s 2024-04-25T10:14:04.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237821 s, 44.1 MB/s 2024-04-25T10:14:04.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.959 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284722 s, 36.8 MB/s 2024-04-25T10:14:04.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:04.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:04.984 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211955 s, 49.5 MB/s 2024-04-25T10:14:05.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233782 s, 44.9 MB/s 2024-04-25T10:14:05.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.040 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233044 s, 45.0 MB/s 2024-04-25T10:14:05.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232668 s, 45.1 MB/s 2024-04-25T10:14:05.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209476 s, 50.1 MB/s 2024-04-25T10:14:05.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:05.095 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:05.095 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:05.095 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:14:05.095 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:05.096 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:05.096 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:05.096 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:05.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.182 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236351 s, 44.4 MB/s 2024-04-25T10:14:05.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.208 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022374 s, 46.9 MB/s 2024-04-25T10:14:05.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.233 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020382 s, 51.4 MB/s 2024-04-25T10:14:05.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.258 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218908 s, 47.9 MB/s 2024-04-25T10:14:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205903 s, 50.9 MB/s 2024-04-25T10:14:05.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251336 s, 41.7 MB/s 2024-04-25T10:14:05.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.337 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207355 s, 50.6 MB/s 2024-04-25T10:14:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.364 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231918 s, 45.2 MB/s 2024-04-25T10:14:05.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232693 s, 45.1 MB/s 2024-04-25T10:14:05.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181004 s, 57.9 MB/s 2024-04-25T10:14:05.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198891 s, 52.7 MB/s 2024-04-25T10:14:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209138 s, 50.1 MB/s 2024-04-25T10:14:05.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.482 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172647 s, 60.7 MB/s 2024-04-25T10:14:05.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190385 s, 55.1 MB/s 2024-04-25T10:14:05.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197869 s, 53.0 MB/s 2024-04-25T10:14:05.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.554 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197211 s, 53.2 MB/s 2024-04-25T10:14:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.581 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237926 s, 44.1 MB/s 2024-04-25T10:14:05.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.606 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211224 s, 49.6 MB/s 2024-04-25T10:14:05.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.630 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206321 s, 50.8 MB/s 2024-04-25T10:14:05.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224941 s, 46.6 MB/s 2024-04-25T10:14:05.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.682 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213414 s, 49.1 MB/s 2024-04-25T10:14:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234901 s, 44.6 MB/s 2024-04-25T10:14:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187067 s, 56.1 MB/s 2024-04-25T10:14:05.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020497 s, 51.2 MB/s 2024-04-25T10:14:05.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.781 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196876 s, 53.3 MB/s 2024-04-25T10:14:05.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207572 s, 50.5 MB/s 2024-04-25T10:14:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201856 s, 51.9 MB/s 2024-04-25T10:14:05.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.852 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173959 s, 60.3 MB/s 2024-04-25T10:14:05.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229768 s, 45.6 MB/s 2024-04-25T10:14:05.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.906 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226647 s, 46.3 MB/s 2024-04-25T10:14:05.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.928 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186171 s, 56.3 MB/s 2024-04-25T10:14:05.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:05.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:05.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021457 s, 48.9 MB/s 2024-04-25T10:14:05.956 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:05.956 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:06.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197745 s, 53.0 MB/s 2024-04-25T10:14:06.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.064 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254042 s, 41.3 MB/s 2024-04-25T10:14:06.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.087 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192006 s, 54.6 MB/s 2024-04-25T10:14:06.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.114 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227294 s, 46.1 MB/s 2024-04-25T10:14:06.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.142 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231318 s, 45.3 MB/s 2024-04-25T10:14:06.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.169 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223958 s, 46.8 MB/s 2024-04-25T10:14:06.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208503 s, 50.3 MB/s 2024-04-25T10:14:06.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.224 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277454 s, 37.8 MB/s 2024-04-25T10:14:06.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.250 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215191 s, 48.7 MB/s 2024-04-25T10:14:06.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025342 s, 41.4 MB/s 2024-04-25T10:14:06.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213773 s, 49.1 MB/s 2024-04-25T10:14:06.328 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.329 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019923 s, 52.6 MB/s 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021404 s, 49.0 MB/s 2024-04-25T10:14:06.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201925 s, 51.9 MB/s 2024-04-25T10:14:06.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238193 s, 44.0 MB/s 2024-04-25T10:14:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182627 s, 57.4 MB/s 2024-04-25T10:14:06.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.452 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.452 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202788 s, 51.7 MB/s 2024-04-25T10:14:06.478 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.478 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.478 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216889 s, 48.3 MB/s 2024-04-25T10:14:06.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211661 s, 49.5 MB/s 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220624 s, 47.5 MB/s 2024-04-25T10:14:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.556 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217466 s, 48.2 MB/s 2024-04-25T10:14:06.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.584 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233655 s, 44.9 MB/s 2024-04-25T10:14:06.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.609 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208581 s, 50.3 MB/s 2024-04-25T10:14:06.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.641 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.641 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278622 s, 37.6 MB/s 2024-04-25T10:14:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247509 s, 42.4 MB/s 2024-04-25T10:14:06.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.697 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235184 s, 44.6 MB/s 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207978 s, 50.4 MB/s 2024-04-25T10:14:06.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210722 s, 49.8 MB/s 2024-04-25T10:14:06.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211762 s, 49.5 MB/s 2024-04-25T10:14:06.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018381 s, 57.0 MB/s 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198762 s, 52.8 MB/s 2024-04-25T10:14:06.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:06.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:06.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018268 s, 57.4 MB/s 2024-04-25T10:14:06.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:06.844 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:07.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.031031 s, 33.8 MB/s 2024-04-25T10:14:07.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226914 s, 46.2 MB/s 2024-04-25T10:14:07.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245683 s, 42.7 MB/s 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241937 s, 43.3 MB/s 2024-04-25T10:14:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.274 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227458 s, 46.1 MB/s 2024-04-25T10:14:07.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.302 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023066 s, 45.5 MB/s 2024-04-25T10:14:07.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241709 s, 43.4 MB/s 2024-04-25T10:14:07.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197685 s, 53.0 MB/s 2024-04-25T10:14:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209818 s, 50.0 MB/s 2024-04-25T10:14:07.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239623 s, 43.8 MB/s 2024-04-25T10:14:07.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.440 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268029 s, 39.1 MB/s 2024-04-25T10:14:07.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.468 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239188 s, 43.8 MB/s 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214529 s, 48.9 MB/s 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194458 s, 53.9 MB/s 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.543 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226277 s, 46.3 MB/s 2024-04-25T10:14:07.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195069 s, 53.8 MB/s 2024-04-25T10:14:07.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251145 s, 41.8 MB/s 2024-04-25T10:14:07.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.624 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227972 s, 46.0 MB/s 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.649 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213943 s, 49.0 MB/s 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.676 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228465 s, 45.9 MB/s 2024-04-25T10:14:07.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204427 s, 51.3 MB/s 2024-04-25T10:14:07.725 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.725 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.725 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224132 s, 46.8 MB/s 2024-04-25T10:14:07.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.752 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215406 s, 48.7 MB/s 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222465 s, 47.1 MB/s 2024-04-25T10:14:07.804 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.805 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225134 s, 46.6 MB/s 2024-04-25T10:14:07.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227234 s, 46.1 MB/s 2024-04-25T10:14:07.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222651 s, 47.1 MB/s 2024-04-25T10:14:07.880 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.881 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188593 s, 55.6 MB/s 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224547 s, 46.7 MB/s 2024-04-25T10:14:07.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.931 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196223 s, 53.4 MB/s 2024-04-25T10:14:07.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.954 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.954 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194737 s, 53.8 MB/s 2024-04-25T10:14:07.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:07.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:07.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219086 s, 47.9 MB/s 2024-04-25T10:14:07.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:07.982 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:07.982 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:07.982 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:14:07.982 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:07.982 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:07.982 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:07.983 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:08.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217616 s, 48.2 MB/s 2024-04-25T10:14:08.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023163 s, 45.3 MB/s 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.119 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219959 s, 47.7 MB/s 2024-04-25T10:14:08.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.146 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223857 s, 46.8 MB/s 2024-04-25T10:14:08.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022013 s, 47.6 MB/s 2024-04-25T10:14:08.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.197 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207611 s, 50.5 MB/s 2024-04-25T10:14:08.223 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.223 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.223 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228621 s, 45.9 MB/s 2024-04-25T10:14:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.252 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238434 s, 44.0 MB/s 2024-04-25T10:14:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198866 s, 52.7 MB/s 2024-04-25T10:14:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.304 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246956 s, 42.5 MB/s 2024-04-25T10:14:08.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250693 s, 41.8 MB/s 2024-04-25T10:14:08.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.357 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206019 s, 50.9 MB/s 2024-04-25T10:14:08.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254241 s, 41.2 MB/s 2024-04-25T10:14:08.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215926 s, 48.6 MB/s 2024-04-25T10:14:08.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219627 s, 47.7 MB/s 2024-04-25T10:14:08.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.465 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231114 s, 45.4 MB/s 2024-04-25T10:14:08.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.492 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224002 s, 46.8 MB/s 2024-04-25T10:14:08.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272885 s, 38.4 MB/s 2024-04-25T10:14:08.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.548 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195018 s, 53.8 MB/s 2024-04-25T10:14:08.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.574 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236943 s, 44.3 MB/s 2024-04-25T10:14:08.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020319 s, 51.6 MB/s 2024-04-25T10:14:08.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.622 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197748 s, 53.0 MB/s 2024-04-25T10:14:08.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200601 s, 52.3 MB/s 2024-04-25T10:14:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021003 s, 49.9 MB/s 2024-04-25T10:14:08.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.696 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209938 s, 49.9 MB/s 2024-04-25T10:14:08.724 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.725 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233638 s, 44.9 MB/s 2024-04-25T10:14:08.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203192 s, 51.6 MB/s 2024-04-25T10:14:08.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198945 s, 52.7 MB/s 2024-04-25T10:14:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023692 s, 44.3 MB/s 2024-04-25T10:14:08.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.828 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236874 s, 44.3 MB/s 2024-04-25T10:14:08.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.852 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188496 s, 55.6 MB/s 2024-04-25T10:14:08.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225106 s, 46.6 MB/s 2024-04-25T10:14:08.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:08.880 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:08.881 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:08.881 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:14:08.881 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:08.881 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:08.881 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:08.881 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:08.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.966 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244529 s, 42.9 MB/s 2024-04-25T10:14:08.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:08.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:08.988 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175848 s, 59.6 MB/s 2024-04-25T10:14:09.011 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.011 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.011 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197748 s, 53.0 MB/s 2024-04-25T10:14:09.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.038 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230903 s, 45.4 MB/s 2024-04-25T10:14:09.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.063 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214039 s, 49.0 MB/s 2024-04-25T10:14:09.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.087 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207668 s, 50.5 MB/s 2024-04-25T10:14:09.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.113 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212769 s, 49.3 MB/s 2024-04-25T10:14:09.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219514 s, 47.8 MB/s 2024-04-25T10:14:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215045 s, 48.8 MB/s 2024-04-25T10:14:09.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233267 s, 45.0 MB/s 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.219 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227525 s, 46.1 MB/s 2024-04-25T10:14:09.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199538 s, 52.6 MB/s 2024-04-25T10:14:09.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241889 s, 43.3 MB/s 2024-04-25T10:14:09.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.298 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.298 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021087 s, 49.7 MB/s 2024-04-25T10:14:09.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.321 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191894 s, 54.6 MB/s 2024-04-25T10:14:09.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.346 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207541 s, 50.5 MB/s 2024-04-25T10:14:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217666 s, 48.2 MB/s 2024-04-25T10:14:09.394 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.394 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.394 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193127 s, 54.3 MB/s 2024-04-25T10:14:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.417 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182999 s, 57.3 MB/s 2024-04-25T10:14:09.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.440 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185512 s, 56.5 MB/s 2024-04-25T10:14:09.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.465 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218108 s, 48.1 MB/s 2024-04-25T10:14:09.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.490 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212004 s, 49.5 MB/s 2024-04-25T10:14:09.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235516 s, 44.5 MB/s 2024-04-25T10:14:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.544 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226694 s, 46.3 MB/s 2024-04-25T10:14:09.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.570 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226685 s, 46.3 MB/s 2024-04-25T10:14:09.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.596 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222147 s, 47.2 MB/s 2024-04-25T10:14:09.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.627 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027467 s, 38.2 MB/s 2024-04-25T10:14:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201229 s, 52.1 MB/s 2024-04-25T10:14:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021285 s, 49.3 MB/s 2024-04-25T10:14:09.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228676 s, 45.9 MB/s 2024-04-25T10:14:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231296 s, 45.3 MB/s 2024-04-25T10:14:09.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229008 s, 45.8 MB/s 2024-04-25T10:14:09.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:09.759 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:09.759 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:09.759 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:14:09.760 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:09.760 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:09.760 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:09.760 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:09.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.843 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209655 s, 50.0 MB/s 2024-04-25T10:14:09.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205235 s, 51.1 MB/s 2024-04-25T10:14:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180574 s, 58.1 MB/s 2024-04-25T10:14:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205884 s, 50.9 MB/s 2024-04-25T10:14:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192839 s, 54.4 MB/s 2024-04-25T10:14:09.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260595 s, 40.2 MB/s 2024-04-25T10:14:09.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:09.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:09.991 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207817 s, 50.5 MB/s 2024-04-25T10:14:10.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195693 s, 53.6 MB/s 2024-04-25T10:14:10.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235515 s, 44.5 MB/s 2024-04-25T10:14:10.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.066 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187604 s, 55.9 MB/s 2024-04-25T10:14:10.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.092 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228354 s, 45.9 MB/s 2024-04-25T10:14:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240933 s, 43.5 MB/s 2024-04-25T10:14:10.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.147 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237537 s, 44.1 MB/s 2024-04-25T10:14:10.173 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.173 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.173 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226288 s, 46.3 MB/s 2024-04-25T10:14:10.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.196 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192968 s, 54.3 MB/s 2024-04-25T10:14:10.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.225 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236018 s, 44.4 MB/s 2024-04-25T10:14:10.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.250 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020969 s, 50.0 MB/s 2024-04-25T10:14:10.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024714 s, 42.4 MB/s 2024-04-25T10:14:10.304 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.304 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.304 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220652 s, 47.5 MB/s 2024-04-25T10:14:10.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217301 s, 48.3 MB/s 2024-04-25T10:14:10.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.356 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219775 s, 47.7 MB/s 2024-04-25T10:14:10.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.380 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207213 s, 50.6 MB/s 2024-04-25T10:14:10.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020901 s, 50.2 MB/s 2024-04-25T10:14:10.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.433 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214179 s, 49.0 MB/s 2024-04-25T10:14:10.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.458 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200687 s, 52.2 MB/s 2024-04-25T10:14:10.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.483 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211495 s, 49.6 MB/s 2024-04-25T10:14:10.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191488 s, 54.8 MB/s 2024-04-25T10:14:10.532 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.532 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.532 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220448 s, 47.6 MB/s 2024-04-25T10:14:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.557 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021187 s, 49.5 MB/s 2024-04-25T10:14:10.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201266 s, 52.1 MB/s 2024-04-25T10:14:10.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228179 s, 46.0 MB/s 2024-04-25T10:14:10.633 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.633 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.633 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206268 s, 50.8 MB/s 2024-04-25T10:14:10.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:10.635 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:10.635 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:10.635 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:14:10.635 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:10.636 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:10.636 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:10.636 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:10.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.720 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210966 s, 49.7 MB/s 2024-04-25T10:14:10.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.745 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021469 s, 48.8 MB/s 2024-04-25T10:14:10.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.769 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199917 s, 52.5 MB/s 2024-04-25T10:14:10.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212181 s, 49.4 MB/s 2024-04-25T10:14:10.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183649 s, 57.1 MB/s 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.843 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021796 s, 48.1 MB/s 2024-04-25T10:14:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248314 s, 42.2 MB/s 2024-04-25T10:14:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222841 s, 47.1 MB/s 2024-04-25T10:14:10.925 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.925 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.926 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235018 s, 44.6 MB/s 2024-04-25T10:14:10.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199839 s, 52.5 MB/s 2024-04-25T10:14:10.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:10.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:10.977 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235172 s, 44.6 MB/s 2024-04-25T10:14:11.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.005 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024458 s, 42.9 MB/s 2024-04-25T10:14:11.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.032 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225424 s, 46.5 MB/s 2024-04-25T10:14:11.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.060 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247369 s, 42.4 MB/s 2024-04-25T10:14:11.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.085 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206656 s, 50.7 MB/s 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212672 s, 49.3 MB/s 2024-04-25T10:14:11.135 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.135 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.135 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231124 s, 45.4 MB/s 2024-04-25T10:14:11.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.160 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203738 s, 51.5 MB/s 2024-04-25T10:14:11.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198199 s, 52.9 MB/s 2024-04-25T10:14:11.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.209 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212405 s, 49.4 MB/s 2024-04-25T10:14:11.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189106 s, 55.4 MB/s 2024-04-25T10:14:11.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.258 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229592 s, 45.7 MB/s 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219153 s, 47.8 MB/s 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.311 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233876 s, 44.8 MB/s 2024-04-25T10:14:11.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228494 s, 45.9 MB/s 2024-04-25T10:14:11.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.367 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259969 s, 40.3 MB/s 2024-04-25T10:14:11.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019578 s, 53.6 MB/s 2024-04-25T10:14:11.417 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.417 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.418 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222249 s, 47.2 MB/s 2024-04-25T10:14:11.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.441 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.441 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199637 s, 52.5 MB/s 2024-04-25T10:14:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.468 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226842 s, 46.2 MB/s 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.495 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229203 s, 45.7 MB/s 2024-04-25T10:14:11.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.524 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254413 s, 41.2 MB/s 2024-04-25T10:14:11.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:11.527 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:11.528 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:11.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.618 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249345 s, 42.1 MB/s 2024-04-25T10:14:11.644 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.644 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.644 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236812 s, 44.3 MB/s 2024-04-25T10:14:11.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.670 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211994 s, 49.5 MB/s 2024-04-25T10:14:11.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.694 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200249 s, 52.4 MB/s 2024-04-25T10:14:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.718 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190976 s, 54.9 MB/s 2024-04-25T10:14:11.743 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.743 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.743 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214517 s, 48.9 MB/s 2024-04-25T10:14:11.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.771 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231392 s, 45.3 MB/s 2024-04-25T10:14:11.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232826 s, 45.0 MB/s 2024-04-25T10:14:11.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205056 s, 51.1 MB/s 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.848 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220463 s, 47.6 MB/s 2024-04-25T10:14:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211264 s, 49.6 MB/s 2024-04-25T10:14:11.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.898 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210194 s, 49.9 MB/s 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.922 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186374 s, 56.3 MB/s 2024-04-25T10:14:11.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.942 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174888 s, 60.0 MB/s 2024-04-25T10:14:11.968 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.968 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.968 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021378 s, 49.0 MB/s 2024-04-25T10:14:11.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:11.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:11.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209402 s, 50.1 MB/s 2024-04-25T10:14:12.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.021 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229854 s, 45.6 MB/s 2024-04-25T10:14:12.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026164 s, 40.1 MB/s 2024-04-25T10:14:12.077 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227623 s, 46.1 MB/s 2024-04-25T10:14:12.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.102 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197825 s, 53.0 MB/s 2024-04-25T10:14:12.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197133 s, 53.2 MB/s 2024-04-25T10:14:12.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234032 s, 44.8 MB/s 2024-04-25T10:14:12.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.178 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218323 s, 48.0 MB/s 2024-04-25T10:14:12.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214722 s, 48.8 MB/s 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.230 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221787 s, 47.3 MB/s 2024-04-25T10:14:12.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.252 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019388 s, 54.1 MB/s 2024-04-25T10:14:12.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.279 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226184 s, 46.4 MB/s 2024-04-25T10:14:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235008 s, 44.6 MB/s 2024-04-25T10:14:12.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.332 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206398 s, 50.8 MB/s 2024-04-25T10:14:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215142 s, 48.7 MB/s 2024-04-25T10:14:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254394 s, 41.2 MB/s 2024-04-25T10:14:12.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189287 s, 55.4 MB/s 2024-04-25T10:14:12.410 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:12.411 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:12.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.499 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022166 s, 47.3 MB/s 2024-04-25T10:14:12.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022338 s, 46.9 MB/s 2024-04-25T10:14:12.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.553 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250064 s, 41.9 MB/s 2024-04-25T10:14:12.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.577 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197016 s, 53.2 MB/s 2024-04-25T10:14:12.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236713 s, 44.3 MB/s 2024-04-25T10:14:12.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.630 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228704 s, 45.8 MB/s 2024-04-25T10:14:12.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220012 s, 47.7 MB/s 2024-04-25T10:14:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204593 s, 51.3 MB/s 2024-04-25T10:14:12.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218125 s, 48.1 MB/s 2024-04-25T10:14:12.731 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206954 s, 50.7 MB/s 2024-04-25T10:14:12.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020887 s, 50.2 MB/s 2024-04-25T10:14:12.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215425 s, 48.7 MB/s 2024-04-25T10:14:12.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210964 s, 49.7 MB/s 2024-04-25T10:14:12.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.830 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188402 s, 55.7 MB/s 2024-04-25T10:14:12.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209543 s, 50.0 MB/s 2024-04-25T10:14:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197104 s, 53.2 MB/s 2024-04-25T10:14:12.901 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.901 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191762 s, 54.7 MB/s 2024-04-25T10:14:12.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230947 s, 45.4 MB/s 2024-04-25T10:14:12.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211439 s, 49.6 MB/s 2024-04-25T10:14:12.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:12.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:12.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224315 s, 46.7 MB/s 2024-04-25T10:14:13.003 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.003 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.004 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020163 s, 52.0 MB/s 2024-04-25T10:14:13.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280134 s, 37.4 MB/s 2024-04-25T10:14:13.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.066 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240987 s, 43.5 MB/s 2024-04-25T10:14:13.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.094 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023231 s, 45.1 MB/s 2024-04-25T10:14:13.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262356 s, 40.0 MB/s 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024336 s, 43.1 MB/s 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.178 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209044 s, 50.2 MB/s 2024-04-25T10:14:13.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.202 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191059 s, 54.9 MB/s 2024-04-25T10:14:13.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.227 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209546 s, 50.0 MB/s 2024-04-25T10:14:13.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.255 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227348 s, 46.1 MB/s 2024-04-25T10:14:13.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207566 s, 50.5 MB/s 2024-04-25T10:14:13.310 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.310 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.310 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255403 s, 41.1 MB/s 2024-04-25T10:14:13.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:13.312 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243549 s, 43.1 MB/s 2024-04-25T10:14:13.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219479 s, 47.8 MB/s 2024-04-25T10:14:13.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196278 s, 53.4 MB/s 2024-04-25T10:14:13.476 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.476 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.476 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213731 s, 49.1 MB/s 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.499 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183099 s, 57.3 MB/s 2024-04-25T10:14:13.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191903 s, 54.6 MB/s 2024-04-25T10:14:13.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262976 s, 39.9 MB/s 2024-04-25T10:14:13.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.577 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211108 s, 49.7 MB/s 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237077 s, 44.2 MB/s 2024-04-25T10:14:13.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192274 s, 54.5 MB/s 2024-04-25T10:14:13.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245823 s, 42.7 MB/s 2024-04-25T10:14:13.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201716 s, 52.0 MB/s 2024-04-25T10:14:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.705 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217933 s, 48.1 MB/s 2024-04-25T10:14:13.731 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.731 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.731 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220381 s, 47.6 MB/s 2024-04-25T10:14:13.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234962 s, 44.6 MB/s 2024-04-25T10:14:13.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.787 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243835 s, 43.0 MB/s 2024-04-25T10:14:13.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.811 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197156 s, 53.2 MB/s 2024-04-25T10:14:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209043 s, 50.2 MB/s 2024-04-25T10:14:13.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.860 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194817 s, 53.8 MB/s 2024-04-25T10:14:13.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018936 s, 55.4 MB/s 2024-04-25T10:14:13.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.910 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222967 s, 47.0 MB/s 2024-04-25T10:14:13.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210034 s, 49.9 MB/s 2024-04-25T10:14:13.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.962 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.962 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239487 s, 43.8 MB/s 2024-04-25T10:14:13.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:13.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:13.990 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243013 s, 43.1 MB/s 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.020 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265409 s, 39.5 MB/s 2024-04-25T10:14:14.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219525 s, 47.8 MB/s 2024-04-25T10:14:14.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211029 s, 49.7 MB/s 2024-04-25T10:14:14.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259189 s, 40.5 MB/s 2024-04-25T10:14:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.131 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266582 s, 39.3 MB/s 2024-04-25T10:14:14.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.159 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024189 s, 43.3 MB/s 2024-04-25T10:14:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207905 s, 50.4 MB/s 2024-04-25T10:14:14.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.214 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260673 s, 40.2 MB/s 2024-04-25T10:14:14.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:14.216 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:14.216 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:14.216 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:14:14.216 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:14.216 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:14.217 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:14.217 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:14.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022595 s, 46.4 MB/s 2024-04-25T10:14:14.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.325 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.325 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223396 s, 46.9 MB/s 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.350 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211808 s, 49.5 MB/s 2024-04-25T10:14:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182362 s, 57.5 MB/s 2024-04-25T10:14:14.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.400 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251504 s, 41.7 MB/s 2024-04-25T10:14:14.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231213 s, 45.4 MB/s 2024-04-25T10:14:14.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.454 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228585 s, 45.9 MB/s 2024-04-25T10:14:14.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.483 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255566 s, 41.0 MB/s 2024-04-25T10:14:14.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.507 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195741 s, 53.6 MB/s 2024-04-25T10:14:14.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.534 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217963 s, 48.1 MB/s 2024-04-25T10:14:14.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.559 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217601 s, 48.2 MB/s 2024-04-25T10:14:14.584 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.584 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.584 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210061 s, 49.9 MB/s 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.609 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216044 s, 48.5 MB/s 2024-04-25T10:14:14.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.634 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020962 s, 50.0 MB/s 2024-04-25T10:14:14.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.664 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255764 s, 41.0 MB/s 2024-04-25T10:14:14.690 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.690 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.690 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226064 s, 46.4 MB/s 2024-04-25T10:14:14.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.718 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246841 s, 42.5 MB/s 2024-04-25T10:14:14.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.744 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209185 s, 50.1 MB/s 2024-04-25T10:14:14.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242702 s, 43.2 MB/s 2024-04-25T10:14:14.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217952 s, 48.1 MB/s 2024-04-25T10:14:14.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224131 s, 46.8 MB/s 2024-04-25T10:14:14.848 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189283 s, 55.4 MB/s 2024-04-25T10:14:14.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199823 s, 52.5 MB/s 2024-04-25T10:14:14.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192989 s, 54.3 MB/s 2024-04-25T10:14:14.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.923 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233708 s, 44.9 MB/s 2024-04-25T10:14:14.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238875 s, 43.9 MB/s 2024-04-25T10:14:14.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:14.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:14.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247173 s, 42.4 MB/s 2024-04-25T10:14:15.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233152 s, 45.0 MB/s 2024-04-25T10:14:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.032 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208168 s, 50.4 MB/s 2024-04-25T10:14:15.057 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.058 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.058 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213682 s, 49.1 MB/s 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.084 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214619 s, 48.9 MB/s 2024-04-25T10:14:15.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217138 s, 48.3 MB/s 2024-04-25T10:14:15.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:15.114 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:15.114 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:15.114 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:14:15.114 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:15.114 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:15.115 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:15.115 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:15.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.201 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233259 s, 45.0 MB/s 2024-04-25T10:14:15.229 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.229 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.229 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021924 s, 47.8 MB/s 2024-04-25T10:14:15.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.255 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02218 s, 47.3 MB/s 2024-04-25T10:14:15.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190803 s, 55.0 MB/s 2024-04-25T10:14:15.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234045 s, 44.8 MB/s 2024-04-25T10:14:15.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221731 s, 47.3 MB/s 2024-04-25T10:14:15.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.516 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236137 s, 44.4 MB/s 2024-04-25T10:14:15.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221022 s, 47.4 MB/s 2024-04-25T10:14:15.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242713 s, 43.2 MB/s 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238251 s, 44.0 MB/s 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284725 s, 36.8 MB/s 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232115 s, 45.2 MB/s 2024-04-25T10:14:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231382 s, 45.3 MB/s 2024-04-25T10:14:15.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202803 s, 51.7 MB/s 2024-04-25T10:14:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205228 s, 51.1 MB/s 2024-04-25T10:14:15.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239117 s, 43.9 MB/s 2024-04-25T10:14:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237072 s, 44.2 MB/s 2024-04-25T10:14:15.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205538 s, 51.0 MB/s 2024-04-25T10:14:15.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.683 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228992 s, 45.8 MB/s 2024-04-25T10:14:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.714 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267962 s, 39.1 MB/s 2024-04-25T10:14:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194418 s, 53.9 MB/s 2024-04-25T10:14:15.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.763 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211642 s, 49.5 MB/s 2024-04-25T10:14:15.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.787 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020515 s, 51.1 MB/s 2024-04-25T10:14:15.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180743 s, 58.0 MB/s 2024-04-25T10:14:15.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180459 s, 58.1 MB/s 2024-04-25T10:14:15.856 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.856 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.856 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216124 s, 48.5 MB/s 2024-04-25T10:14:15.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229699 s, 45.6 MB/s 2024-04-25T10:14:15.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.912 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244051 s, 43.0 MB/s 2024-04-25T10:14:15.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.937 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219533 s, 47.8 MB/s 2024-04-25T10:14:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020202 s, 51.9 MB/s 2024-04-25T10:14:15.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:15.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:15.985 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199047 s, 52.7 MB/s 2024-04-25T10:14:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.010 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213816 s, 49.0 MB/s 2024-04-25T10:14:16.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:16.013 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:16.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.098 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.098 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240637 s, 43.6 MB/s 2024-04-25T10:14:16.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.124 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223176 s, 47.0 MB/s 2024-04-25T10:14:16.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.151 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211611 s, 49.6 MB/s 2024-04-25T10:14:16.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.175 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208838 s, 50.2 MB/s 2024-04-25T10:14:16.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.198 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191348 s, 54.8 MB/s 2024-04-25T10:14:16.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.221 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196827 s, 53.3 MB/s 2024-04-25T10:14:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239254 s, 43.8 MB/s 2024-04-25T10:14:16.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234006 s, 44.8 MB/s 2024-04-25T10:14:16.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246693 s, 42.5 MB/s 2024-04-25T10:14:16.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232461 s, 45.1 MB/s 2024-04-25T10:14:16.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.361 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232105 s, 45.2 MB/s 2024-04-25T10:14:16.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218954 s, 47.9 MB/s 2024-04-25T10:14:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022571 s, 46.5 MB/s 2024-04-25T10:14:16.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.439 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217835 s, 48.1 MB/s 2024-04-25T10:14:16.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023735 s, 44.2 MB/s 2024-04-25T10:14:16.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224001 s, 46.8 MB/s 2024-04-25T10:14:16.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.514 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177854 s, 59.0 MB/s 2024-04-25T10:14:16.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.542 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244586 s, 42.9 MB/s 2024-04-25T10:14:16.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.563 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180284 s, 58.2 MB/s 2024-04-25T10:14:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207067 s, 50.6 MB/s 2024-04-25T10:14:16.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.618 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262068 s, 40.0 MB/s 2024-04-25T10:14:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214235 s, 48.9 MB/s 2024-04-25T10:14:16.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.665 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177889 s, 58.9 MB/s 2024-04-25T10:14:16.689 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.689 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.689 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201934 s, 51.9 MB/s 2024-04-25T10:14:16.715 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.716 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.716 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229381 s, 45.7 MB/s 2024-04-25T10:14:16.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.742 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020572 s, 51.0 MB/s 2024-04-25T10:14:16.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019915 s, 52.7 MB/s 2024-04-25T10:14:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020615 s, 50.9 MB/s 2024-04-25T10:14:16.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.811 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197604 s, 53.1 MB/s 2024-04-25T10:14:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218347 s, 48.0 MB/s 2024-04-25T10:14:16.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209237 s, 50.1 MB/s 2024-04-25T10:14:16.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.891 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254758 s, 41.2 MB/s 2024-04-25T10:14:16.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:16.894 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:16.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:16.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:16.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312666 s, 33.5 MB/s 2024-04-25T10:14:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221703 s, 47.3 MB/s 2024-04-25T10:14:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193563 s, 54.2 MB/s 2024-04-25T10:14:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232907 s, 45.0 MB/s 2024-04-25T10:14:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227857 s, 46.0 MB/s 2024-04-25T10:14:17.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.124 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232201 s, 45.2 MB/s 2024-04-25T10:14:17.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202583 s, 51.8 MB/s 2024-04-25T10:14:17.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192361 s, 54.5 MB/s 2024-04-25T10:14:17.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208122 s, 50.4 MB/s 2024-04-25T10:14:17.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212144 s, 49.4 MB/s 2024-04-25T10:14:17.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274037 s, 38.3 MB/s 2024-04-25T10:14:17.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210383 s, 49.8 MB/s 2024-04-25T10:14:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243657 s, 43.0 MB/s 2024-04-25T10:14:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233664 s, 44.9 MB/s 2024-04-25T10:14:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226178 s, 46.4 MB/s 2024-04-25T10:14:17.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226217 s, 46.4 MB/s 2024-04-25T10:14:17.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227669 s, 46.1 MB/s 2024-04-25T10:14:17.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216243 s, 48.5 MB/s 2024-04-25T10:14:17.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250515 s, 41.9 MB/s 2024-04-25T10:14:17.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229048 s, 45.8 MB/s 2024-04-25T10:14:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239507 s, 43.8 MB/s 2024-04-25T10:14:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232139 s, 45.2 MB/s 2024-04-25T10:14:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201047 s, 52.2 MB/s 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.601 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232451 s, 45.1 MB/s 2024-04-25T10:14:17.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.630 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250682 s, 41.8 MB/s 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.658 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246182 s, 42.6 MB/s 2024-04-25T10:14:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213384 s, 49.1 MB/s 2024-04-25T10:14:17.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.709 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.709 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209251 s, 50.1 MB/s 2024-04-25T10:14:17.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203996 s, 51.4 MB/s 2024-04-25T10:14:17.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.756 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191203 s, 54.8 MB/s 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.779 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195969 s, 53.5 MB/s 2024-04-25T10:14:17.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197049 s, 53.2 MB/s 2024-04-25T10:14:17.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:17.805 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:17.805 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:17.805 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:14:17.805 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:17.805 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:17.805 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:17.806 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204495 s, 51.3 MB/s 2024-04-25T10:14:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022473 s, 46.7 MB/s 2024-04-25T10:14:17.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.941 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020705 s, 50.6 MB/s 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.965 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207364 s, 50.6 MB/s 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:17.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237631 s, 44.1 MB/s 2024-04-25T10:14:18.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.021 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023813 s, 44.0 MB/s 2024-04-25T10:14:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.047 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210632 s, 49.8 MB/s 2024-04-25T10:14:18.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235936 s, 44.4 MB/s 2024-04-25T10:14:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179923 s, 58.3 MB/s 2024-04-25T10:14:18.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196953 s, 53.2 MB/s 2024-04-25T10:14:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.147 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231205 s, 45.4 MB/s 2024-04-25T10:14:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.175 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243739 s, 43.0 MB/s 2024-04-25T10:14:18.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.201 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223067 s, 47.0 MB/s 2024-04-25T10:14:18.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.228 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225827 s, 46.4 MB/s 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.255 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228638 s, 45.9 MB/s 2024-04-25T10:14:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209609 s, 50.0 MB/s 2024-04-25T10:14:18.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219901 s, 47.7 MB/s 2024-04-25T10:14:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262524 s, 39.9 MB/s 2024-04-25T10:14:18.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.366 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249884 s, 42.0 MB/s 2024-04-25T10:14:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205615 s, 51.0 MB/s 2024-04-25T10:14:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215048 s, 48.8 MB/s 2024-04-25T10:14:18.442 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.442 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.442 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218026 s, 48.1 MB/s 2024-04-25T10:14:18.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254805 s, 41.2 MB/s 2024-04-25T10:14:18.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277399 s, 37.8 MB/s 2024-04-25T10:14:18.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270321 s, 38.8 MB/s 2024-04-25T10:14:18.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262114 s, 40.0 MB/s 2024-04-25T10:14:18.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.593 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204221 s, 51.3 MB/s 2024-04-25T10:14:18.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.619 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.619 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219799 s, 47.7 MB/s 2024-04-25T10:14:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236563 s, 44.3 MB/s 2024-04-25T10:14:18.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.673 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227518 s, 46.1 MB/s 2024-04-25T10:14:18.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.697 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201912 s, 51.9 MB/s 2024-04-25T10:14:18.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020966 s, 50.0 MB/s 2024-04-25T10:14:18.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:18.724 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:18.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021745 s, 48.2 MB/s 2024-04-25T10:14:18.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206362 s, 50.8 MB/s 2024-04-25T10:14:18.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205977 s, 50.9 MB/s 2024-04-25T10:14:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242483 s, 43.2 MB/s 2024-04-25T10:14:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211357 s, 49.6 MB/s 2024-04-25T10:14:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235226 s, 44.6 MB/s 2024-04-25T10:14:18.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.968 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.968 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248319 s, 42.2 MB/s 2024-04-25T10:14:18.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:18.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:18.995 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224043 s, 46.8 MB/s 2024-04-25T10:14:19.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.021 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225741 s, 46.5 MB/s 2024-04-25T10:14:19.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264276 s, 39.7 MB/s 2024-04-25T10:14:19.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246499 s, 42.5 MB/s 2024-04-25T10:14:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238281 s, 44.0 MB/s 2024-04-25T10:14:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266956 s, 39.3 MB/s 2024-04-25T10:14:19.163 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220013 s, 47.7 MB/s 2024-04-25T10:14:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259251 s, 40.4 MB/s 2024-04-25T10:14:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020445 s, 51.3 MB/s 2024-04-25T10:14:19.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249847 s, 42.0 MB/s 2024-04-25T10:14:19.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207974 s, 50.4 MB/s 2024-04-25T10:14:19.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.301 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253148 s, 41.4 MB/s 2024-04-25T10:14:19.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215037 s, 48.8 MB/s 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221772 s, 47.3 MB/s 2024-04-25T10:14:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227956 s, 46.0 MB/s 2024-04-25T10:14:19.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247949 s, 42.3 MB/s 2024-04-25T10:14:19.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233144 s, 45.0 MB/s 2024-04-25T10:14:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275322 s, 38.1 MB/s 2024-04-25T10:14:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214976 s, 48.8 MB/s 2024-04-25T10:14:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225267 s, 46.5 MB/s 2024-04-25T10:14:19.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.547 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227802 s, 46.0 MB/s 2024-04-25T10:14:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212735 s, 49.3 MB/s 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219214 s, 47.8 MB/s 2024-04-25T10:14:19.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208553 s, 50.3 MB/s 2024-04-25T10:14:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213938 s, 49.0 MB/s 2024-04-25T10:14:19.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:19.651 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:19.651 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:19.651 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:14:19.651 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:19.651 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:19.651 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:19.652 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022607 s, 46.4 MB/s 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021038 s, 49.8 MB/s 2024-04-25T10:14:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214523 s, 48.9 MB/s 2024-04-25T10:14:19.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.815 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231659 s, 45.3 MB/s 2024-04-25T10:14:19.839 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.840 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209586 s, 50.0 MB/s 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.866 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221012 s, 47.4 MB/s 2024-04-25T10:14:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210985 s, 49.7 MB/s 2024-04-25T10:14:19.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.914 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187664 s, 55.9 MB/s 2024-04-25T10:14:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.939 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210402 s, 49.8 MB/s 2024-04-25T10:14:19.964 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.964 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.964 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221769 s, 47.3 MB/s 2024-04-25T10:14:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197526 s, 53.1 MB/s 2024-04-25T10:14:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204995 s, 51.2 MB/s 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207874 s, 50.4 MB/s 2024-04-25T10:14:20.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.063 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197796 s, 53.0 MB/s 2024-04-25T10:14:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268302 s, 39.1 MB/s 2024-04-25T10:14:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.121 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236684 s, 44.3 MB/s 2024-04-25T10:14:20.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.146 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.146 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203152 s, 51.6 MB/s 2024-04-25T10:14:20.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.171 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207411 s, 50.6 MB/s 2024-04-25T10:14:20.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.196 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212688 s, 49.3 MB/s 2024-04-25T10:14:20.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.222 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209475 s, 50.1 MB/s 2024-04-25T10:14:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184307 s, 56.9 MB/s 2024-04-25T10:14:20.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242693 s, 43.2 MB/s 2024-04-25T10:14:20.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.297 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203464 s, 51.5 MB/s 2024-04-25T10:14:20.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.324 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228786 s, 45.8 MB/s 2024-04-25T10:14:20.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204081 s, 51.4 MB/s 2024-04-25T10:14:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199312 s, 52.6 MB/s 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.396 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207049 s, 50.6 MB/s 2024-04-25T10:14:20.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.420 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198089 s, 52.9 MB/s 2024-04-25T10:14:20.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.448 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234655 s, 44.7 MB/s 2024-04-25T10:14:20.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.482 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292324 s, 35.9 MB/s 2024-04-25T10:14:20.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.510 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246281 s, 42.6 MB/s 2024-04-25T10:14:20.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214078 s, 49.0 MB/s 2024-04-25T10:14:20.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:20.538 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:20.538 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:20.539 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:14:20.539 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:20.539 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:20.539 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:20.539 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:20.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257825 s, 40.7 MB/s 2024-04-25T10:14:20.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.659 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.659 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237125 s, 44.2 MB/s 2024-04-25T10:14:20.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.687 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244761 s, 42.8 MB/s 2024-04-25T10:14:20.712 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.713 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212079 s, 49.4 MB/s 2024-04-25T10:14:20.738 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.738 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.739 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022122 s, 47.4 MB/s 2024-04-25T10:14:20.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.764 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.764 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212434 s, 49.4 MB/s 2024-04-25T10:14:20.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210082 s, 49.9 MB/s 2024-04-25T10:14:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213492 s, 49.1 MB/s 2024-04-25T10:14:20.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191011 s, 54.9 MB/s 2024-04-25T10:14:20.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.865 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238044 s, 44.0 MB/s 2024-04-25T10:14:20.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212282 s, 49.4 MB/s 2024-04-25T10:14:20.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224663 s, 46.7 MB/s 2024-04-25T10:14:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205302 s, 51.1 MB/s 2024-04-25T10:14:20.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.965 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200938 s, 52.2 MB/s 2024-04-25T10:14:20.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:20.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:20.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202184 s, 51.9 MB/s 2024-04-25T10:14:21.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.016 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.016 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231426 s, 45.3 MB/s 2024-04-25T10:14:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022003 s, 47.7 MB/s 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213358 s, 49.1 MB/s 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216127 s, 48.5 MB/s 2024-04-25T10:14:21.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193985 s, 54.1 MB/s 2024-04-25T10:14:21.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203524 s, 51.5 MB/s 2024-04-25T10:14:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225523 s, 46.5 MB/s 2024-04-25T10:14:21.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205256 s, 51.1 MB/s 2024-04-25T10:14:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.213 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184557 s, 56.8 MB/s 2024-04-25T10:14:21.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280362 s, 37.4 MB/s 2024-04-25T10:14:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223001 s, 47.0 MB/s 2024-04-25T10:14:21.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.297 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220062 s, 47.6 MB/s 2024-04-25T10:14:21.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.320 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192948 s, 54.3 MB/s 2024-04-25T10:14:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.350 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267151 s, 39.3 MB/s 2024-04-25T10:14:21.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210864 s, 49.7 MB/s 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.400 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209803 s, 50.0 MB/s 2024-04-25T10:14:21.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.429 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250133 s, 41.9 MB/s 2024-04-25T10:14:21.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:21.432 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:21.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256576 s, 40.9 MB/s 2024-04-25T10:14:21.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.548 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218113 s, 48.1 MB/s 2024-04-25T10:14:21.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246458 s, 42.5 MB/s 2024-04-25T10:14:21.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.605 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244738 s, 42.8 MB/s 2024-04-25T10:14:21.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207254 s, 50.6 MB/s 2024-04-25T10:14:21.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.658 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232174 s, 45.2 MB/s 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243327 s, 43.1 MB/s 2024-04-25T10:14:21.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179153 s, 58.5 MB/s 2024-04-25T10:14:21.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.734 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.734 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231022 s, 45.4 MB/s 2024-04-25T10:14:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202947 s, 51.7 MB/s 2024-04-25T10:14:21.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023653 s, 44.3 MB/s 2024-04-25T10:14:21.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.812 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228794 s, 45.8 MB/s 2024-04-25T10:14:21.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.838 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216419 s, 48.5 MB/s 2024-04-25T10:14:21.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.863 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.863 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210325 s, 49.9 MB/s 2024-04-25T10:14:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222634 s, 47.1 MB/s 2024-04-25T10:14:21.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230478 s, 45.5 MB/s 2024-04-25T10:14:21.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.945 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247064 s, 42.4 MB/s 2024-04-25T10:14:21.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.971 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211644 s, 49.5 MB/s 2024-04-25T10:14:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:21.998 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217593 s, 48.2 MB/s 2024-04-25T10:14:22.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.025 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.025 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022286 s, 47.1 MB/s 2024-04-25T10:14:22.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215123 s, 48.7 MB/s 2024-04-25T10:14:22.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022688 s, 46.2 MB/s 2024-04-25T10:14:22.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206767 s, 50.7 MB/s 2024-04-25T10:14:22.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.127 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204226 s, 51.3 MB/s 2024-04-25T10:14:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218334 s, 48.0 MB/s 2024-04-25T10:14:22.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.180 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226021 s, 46.4 MB/s 2024-04-25T10:14:22.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.209 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246831 s, 42.5 MB/s 2024-04-25T10:14:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190875 s, 54.9 MB/s 2024-04-25T10:14:22.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.258 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216692 s, 48.4 MB/s 2024-04-25T10:14:22.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208862 s, 50.2 MB/s 2024-04-25T10:14:22.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.311 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217201 s, 48.3 MB/s 2024-04-25T10:14:22.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226927 s, 46.2 MB/s 2024-04-25T10:14:22.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:22.341 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:22.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245918 s, 42.6 MB/s 2024-04-25T10:14:22.452 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.452 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.452 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216095 s, 48.5 MB/s 2024-04-25T10:14:22.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.478 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.478 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212458 s, 49.4 MB/s 2024-04-25T10:14:22.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.499 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174867 s, 60.0 MB/s 2024-04-25T10:14:22.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.526 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.526 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224108 s, 46.8 MB/s 2024-04-25T10:14:22.553 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.554 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249301 s, 42.1 MB/s 2024-04-25T10:14:22.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.580 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222038 s, 47.2 MB/s 2024-04-25T10:14:22.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.602 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185238 s, 56.6 MB/s 2024-04-25T10:14:22.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217897 s, 48.1 MB/s 2024-04-25T10:14:22.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.655 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236509 s, 44.3 MB/s 2024-04-25T10:14:22.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.683 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235584 s, 44.5 MB/s 2024-04-25T10:14:22.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245126 s, 42.8 MB/s 2024-04-25T10:14:22.736 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.737 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220184 s, 47.6 MB/s 2024-04-25T10:14:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.762 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198055 s, 52.9 MB/s 2024-04-25T10:14:22.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.787 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207279 s, 50.6 MB/s 2024-04-25T10:14:22.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.813 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.813 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217478 s, 48.2 MB/s 2024-04-25T10:14:22.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.838 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.838 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199038 s, 52.7 MB/s 2024-04-25T10:14:22.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203729 s, 51.5 MB/s 2024-04-25T10:14:22.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193185 s, 54.3 MB/s 2024-04-25T10:14:22.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.908 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019689 s, 53.3 MB/s 2024-04-25T10:14:22.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216465 s, 48.4 MB/s 2024-04-25T10:14:22.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.962 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236652 s, 44.3 MB/s 2024-04-25T10:14:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233407 s, 44.9 MB/s 2024-04-25T10:14:23.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188967 s, 55.5 MB/s 2024-04-25T10:14:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.038 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205699 s, 51.0 MB/s 2024-04-25T10:14:23.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.062 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201915 s, 51.9 MB/s 2024-04-25T10:14:23.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220128 s, 47.6 MB/s 2024-04-25T10:14:23.117 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.117 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.117 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024655 s, 42.5 MB/s 2024-04-25T10:14:23.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.145 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232876 s, 45.0 MB/s 2024-04-25T10:14:23.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.175 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265584 s, 39.5 MB/s 2024-04-25T10:14:23.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.201 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223199 s, 47.0 MB/s 2024-04-25T10:14:23.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.228 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228765 s, 45.8 MB/s 2024-04-25T10:14:23.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:23.231 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:23.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.322 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255692 s, 41.0 MB/s 2024-04-25T10:14:23.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.345 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019895 s, 52.7 MB/s 2024-04-25T10:14:23.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251147 s, 41.8 MB/s 2024-04-25T10:14:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208452 s, 50.3 MB/s 2024-04-25T10:14:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.430 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264219 s, 39.7 MB/s 2024-04-25T10:14:23.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.454 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.454 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210453 s, 49.8 MB/s 2024-04-25T10:14:23.481 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.482 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237143 s, 44.2 MB/s 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.510 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245772 s, 42.7 MB/s 2024-04-25T10:14:23.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238568 s, 44.0 MB/s 2024-04-25T10:14:23.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225416 s, 46.5 MB/s 2024-04-25T10:14:23.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.594 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236685 s, 44.3 MB/s 2024-04-25T10:14:23.619 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.619 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.619 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215747 s, 48.6 MB/s 2024-04-25T10:14:23.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230577 s, 45.5 MB/s 2024-04-25T10:14:23.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196501 s, 53.4 MB/s 2024-04-25T10:14:23.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.693 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201151 s, 52.1 MB/s 2024-04-25T10:14:23.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.719 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220143 s, 47.6 MB/s 2024-04-25T10:14:23.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024273 s, 43.2 MB/s 2024-04-25T10:14:23.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224376 s, 46.7 MB/s 2024-04-25T10:14:23.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196487 s, 53.4 MB/s 2024-04-25T10:14:23.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.822 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020941 s, 50.1 MB/s 2024-04-25T10:14:23.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.846 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198196 s, 52.9 MB/s 2024-04-25T10:14:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020666 s, 50.7 MB/s 2024-04-25T10:14:23.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.895 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.895 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205956 s, 50.9 MB/s 2024-04-25T10:14:23.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.922 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221126 s, 47.4 MB/s 2024-04-25T10:14:23.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253932 s, 41.3 MB/s 2024-04-25T10:14:23.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:23.977 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222259 s, 47.2 MB/s 2024-04-25T10:14:23.999 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:23.999 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.000 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187525 s, 55.9 MB/s 2024-04-25T10:14:24.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.026 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226187 s, 46.4 MB/s 2024-04-25T10:14:24.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.055 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.055 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249162 s, 42.1 MB/s 2024-04-25T10:14:24.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197408 s, 53.1 MB/s 2024-04-25T10:14:24.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231012 s, 45.4 MB/s 2024-04-25T10:14:24.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268558 s, 39.0 MB/s 2024-04-25T10:14:24.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:24.139 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:24.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.227 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.227 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206966 s, 50.7 MB/s 2024-04-25T10:14:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.253 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218563 s, 48.0 MB/s 2024-04-25T10:14:24.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221337 s, 47.4 MB/s 2024-04-25T10:14:24.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.302 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214456 s, 48.9 MB/s 2024-04-25T10:14:24.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023896 s, 43.9 MB/s 2024-04-25T10:14:24.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.361 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247385 s, 42.4 MB/s 2024-04-25T10:14:24.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.388 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235761 s, 44.5 MB/s 2024-04-25T10:14:24.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.411 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191946 s, 54.6 MB/s 2024-04-25T10:14:24.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196161 s, 53.5 MB/s 2024-04-25T10:14:24.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202032 s, 51.9 MB/s 2024-04-25T10:14:24.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.491 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0285394 s, 36.7 MB/s 2024-04-25T10:14:24.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251797 s, 41.6 MB/s 2024-04-25T10:14:24.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.545 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201932 s, 51.9 MB/s 2024-04-25T10:14:24.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.571 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224033 s, 46.8 MB/s 2024-04-25T10:14:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.599 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239828 s, 43.7 MB/s 2024-04-25T10:14:24.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.625 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214938 s, 48.8 MB/s 2024-04-25T10:14:24.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222192 s, 47.2 MB/s 2024-04-25T10:14:24.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224648 s, 46.7 MB/s 2024-04-25T10:14:24.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243138 s, 43.1 MB/s 2024-04-25T10:14:24.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.729 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195786 s, 53.6 MB/s 2024-04-25T10:14:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256282 s, 40.9 MB/s 2024-04-25T10:14:24.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222415 s, 47.1 MB/s 2024-04-25T10:14:24.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215517 s, 48.7 MB/s 2024-04-25T10:14:24.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.836 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021631 s, 48.5 MB/s 2024-04-25T10:14:24.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219799 s, 47.7 MB/s 2024-04-25T10:14:24.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216294 s, 48.5 MB/s 2024-04-25T10:14:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.915 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232142 s, 45.2 MB/s 2024-04-25T10:14:24.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.938 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196615 s, 53.3 MB/s 2024-04-25T10:14:24.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250315 s, 41.9 MB/s 2024-04-25T10:14:24.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:24.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:24.990 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189071 s, 55.5 MB/s 2024-04-25T10:14:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197244 s, 53.2 MB/s 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.040 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232098 s, 45.2 MB/s 2024-04-25T10:14:25.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:25.043 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:25.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191681 s, 54.7 MB/s 2024-04-25T10:14:25.154 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.154 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.154 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242475 s, 43.2 MB/s 2024-04-25T10:14:25.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251877 s, 41.6 MB/s 2024-04-25T10:14:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.208 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216294 s, 48.5 MB/s 2024-04-25T10:14:25.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.233 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206879 s, 50.7 MB/s 2024-04-25T10:14:25.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.262 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234155 s, 44.8 MB/s 2024-04-25T10:14:25.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.287 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210735 s, 49.8 MB/s 2024-04-25T10:14:25.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226909 s, 46.2 MB/s 2024-04-25T10:14:25.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239534 s, 43.8 MB/s 2024-04-25T10:14:25.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246019 s, 42.6 MB/s 2024-04-25T10:14:25.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.396 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211588 s, 49.6 MB/s 2024-04-25T10:14:25.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.422 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227719 s, 46.0 MB/s 2024-04-25T10:14:25.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201445 s, 52.1 MB/s 2024-04-25T10:14:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221624 s, 47.3 MB/s 2024-04-25T10:14:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.496 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195034 s, 53.8 MB/s 2024-04-25T10:14:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216957 s, 48.3 MB/s 2024-04-25T10:14:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.547 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219453 s, 47.8 MB/s 2024-04-25T10:14:25.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.577 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.577 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253915 s, 41.3 MB/s 2024-04-25T10:14:25.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.606 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253428 s, 41.4 MB/s 2024-04-25T10:14:25.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210952 s, 49.7 MB/s 2024-04-25T10:14:25.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210773 s, 49.7 MB/s 2024-04-25T10:14:25.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.684 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239558 s, 43.8 MB/s 2024-04-25T10:14:25.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235819 s, 44.5 MB/s 2024-04-25T10:14:25.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235787 s, 44.5 MB/s 2024-04-25T10:14:25.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.762 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192156 s, 54.6 MB/s 2024-04-25T10:14:25.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208541 s, 50.3 MB/s 2024-04-25T10:14:25.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230157 s, 45.6 MB/s 2024-04-25T10:14:25.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209702 s, 50.0 MB/s 2024-04-25T10:14:25.864 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.865 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200696 s, 52.2 MB/s 2024-04-25T10:14:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236723 s, 44.3 MB/s 2024-04-25T10:14:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213627 s, 49.1 MB/s 2024-04-25T10:14:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:25.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:25.945 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023118 s, 45.4 MB/s 2024-04-25T10:14:25.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:25.947 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:26.045 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300655 s, 34.9 MB/s 2024-04-25T10:14:26.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.076 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268516 s, 39.1 MB/s 2024-04-25T10:14:26.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219238 s, 47.8 MB/s 2024-04-25T10:14:26.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.126 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199237 s, 52.6 MB/s 2024-04-25T10:14:26.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020745 s, 50.5 MB/s 2024-04-25T10:14:26.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022474 s, 46.7 MB/s 2024-04-25T10:14:26.200 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.200 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.200 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196927 s, 53.2 MB/s 2024-04-25T10:14:26.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.225 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199207 s, 52.6 MB/s 2024-04-25T10:14:26.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.250 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214708 s, 48.8 MB/s 2024-04-25T10:14:26.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.278 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.278 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240648 s, 43.6 MB/s 2024-04-25T10:14:26.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252916 s, 41.5 MB/s 2024-04-25T10:14:26.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227657 s, 46.1 MB/s 2024-04-25T10:14:26.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217574 s, 48.2 MB/s 2024-04-25T10:14:26.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021237 s, 49.4 MB/s 2024-04-25T10:14:26.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.411 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.411 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225945 s, 46.4 MB/s 2024-04-25T10:14:26.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021471 s, 48.8 MB/s 2024-04-25T10:14:26.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.463 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213066 s, 49.2 MB/s 2024-04-25T10:14:26.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189305 s, 55.4 MB/s 2024-04-25T10:14:26.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.512 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224709 s, 46.7 MB/s 2024-04-25T10:14:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234131 s, 44.8 MB/s 2024-04-25T10:14:26.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236891 s, 44.3 MB/s 2024-04-25T10:14:26.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.593 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.593 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215774 s, 48.6 MB/s 2024-04-25T10:14:26.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.620 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230767 s, 45.4 MB/s 2024-04-25T10:14:26.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240471 s, 43.6 MB/s 2024-04-25T10:14:26.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.674 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223617 s, 46.9 MB/s 2024-04-25T10:14:26.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218547 s, 48.0 MB/s 2024-04-25T10:14:26.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.724 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.724 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204685 s, 51.2 MB/s 2024-04-25T10:14:26.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233035 s, 45.0 MB/s 2024-04-25T10:14:26.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203521 s, 51.5 MB/s 2024-04-25T10:14:26.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.801 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214121 s, 49.0 MB/s 2024-04-25T10:14:26.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258153 s, 40.6 MB/s 2024-04-25T10:14:26.856 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.856 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.856 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216019 s, 48.5 MB/s 2024-04-25T10:14:26.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:26.859 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:26.859 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:26.859 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:14:26.859 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:26.859 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:26.859 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:26.860 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.947 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228136 s, 46.0 MB/s 2024-04-25T10:14:26.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:26.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:26.985 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0334006 s, 31.4 MB/s 2024-04-25T10:14:27.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026973 s, 38.9 MB/s 2024-04-25T10:14:27.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266602 s, 39.3 MB/s 2024-04-25T10:14:27.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.072 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222318 s, 47.2 MB/s 2024-04-25T10:14:27.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023937 s, 43.8 MB/s 2024-04-25T10:14:27.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175094 s, 59.9 MB/s 2024-04-25T10:14:27.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228712 s, 45.8 MB/s 2024-04-25T10:14:27.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.178 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242554 s, 43.2 MB/s 2024-04-25T10:14:27.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.202 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203516 s, 51.5 MB/s 2024-04-25T10:14:27.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249854 s, 42.0 MB/s 2024-04-25T10:14:27.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.262 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278402 s, 37.7 MB/s 2024-04-25T10:14:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.289 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022496 s, 46.6 MB/s 2024-04-25T10:14:27.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.311 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194949 s, 53.8 MB/s 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.332 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171938 s, 61.0 MB/s 2024-04-25T10:14:27.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174723 s, 60.0 MB/s 2024-04-25T10:14:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199209 s, 52.6 MB/s 2024-04-25T10:14:27.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027855 s, 37.6 MB/s 2024-04-25T10:14:27.434 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.434 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.434 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224111 s, 46.8 MB/s 2024-04-25T10:14:27.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221888 s, 47.3 MB/s 2024-04-25T10:14:27.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.482 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198427 s, 52.8 MB/s 2024-04-25T10:14:27.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.510 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240091 s, 43.7 MB/s 2024-04-25T10:14:27.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251878 s, 41.6 MB/s 2024-04-25T10:14:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213953 s, 49.0 MB/s 2024-04-25T10:14:27.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.594 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242948 s, 43.2 MB/s 2024-04-25T10:14:27.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.620 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228078 s, 46.0 MB/s 2024-04-25T10:14:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212237 s, 49.4 MB/s 2024-04-25T10:14:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219446 s, 47.8 MB/s 2024-04-25T10:14:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236923 s, 44.3 MB/s 2024-04-25T10:14:27.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.724 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.724 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208286 s, 50.3 MB/s 2024-04-25T10:14:27.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214758 s, 48.8 MB/s 2024-04-25T10:14:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219736 s, 47.7 MB/s 2024-04-25T10:14:27.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:27.779 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:27.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273892 s, 38.3 MB/s 2024-04-25T10:14:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216268 s, 48.5 MB/s 2024-04-25T10:14:27.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0164944 s, 63.6 MB/s 2024-04-25T10:14:27.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.946 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255998 s, 41.0 MB/s 2024-04-25T10:14:27.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197672 s, 53.0 MB/s 2024-04-25T10:14:27.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:27.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:27.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199791 s, 52.5 MB/s 2024-04-25T10:14:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.017 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195352 s, 53.7 MB/s 2024-04-25T10:14:28.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210006 s, 49.9 MB/s 2024-04-25T10:14:28.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0315033 s, 33.3 MB/s 2024-04-25T10:14:28.106 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.106 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.106 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247224 s, 42.4 MB/s 2024-04-25T10:14:28.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235551 s, 44.5 MB/s 2024-04-25T10:14:28.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204391 s, 51.3 MB/s 2024-04-25T10:14:28.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182922 s, 57.3 MB/s 2024-04-25T10:14:28.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.206 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230327 s, 45.5 MB/s 2024-04-25T10:14:28.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.234 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226508 s, 46.3 MB/s 2024-04-25T10:14:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234429 s, 44.7 MB/s 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.288 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229441 s, 45.7 MB/s 2024-04-25T10:14:28.317 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.317 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.317 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249654 s, 42.0 MB/s 2024-04-25T10:14:28.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.346 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238253 s, 44.0 MB/s 2024-04-25T10:14:28.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243089 s, 43.1 MB/s 2024-04-25T10:14:28.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236447 s, 44.3 MB/s 2024-04-25T10:14:28.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211685 s, 49.5 MB/s 2024-04-25T10:14:28.453 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.453 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.453 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212014 s, 49.5 MB/s 2024-04-25T10:14:28.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261646 s, 40.1 MB/s 2024-04-25T10:14:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194282 s, 54.0 MB/s 2024-04-25T10:14:28.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.538 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268086 s, 39.1 MB/s 2024-04-25T10:14:28.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.571 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028932 s, 36.2 MB/s 2024-04-25T10:14:28.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.599 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243835 s, 43.0 MB/s 2024-04-25T10:14:28.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200139 s, 52.4 MB/s 2024-04-25T10:14:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232441 s, 45.1 MB/s 2024-04-25T10:14:28.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250852 s, 41.8 MB/s 2024-04-25T10:14:28.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236128 s, 44.4 MB/s 2024-04-25T10:14:28.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:28.709 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:28.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223476 s, 46.9 MB/s 2024-04-25T10:14:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.821 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02512 s, 41.7 MB/s 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251022 s, 41.8 MB/s 2024-04-25T10:14:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205336 s, 51.1 MB/s 2024-04-25T10:14:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224793 s, 46.6 MB/s 2024-04-25T10:14:28.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245402 s, 42.7 MB/s 2024-04-25T10:14:28.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209737 s, 50.0 MB/s 2024-04-25T10:14:28.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:28.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:28.984 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272548 s, 38.5 MB/s 2024-04-25T10:14:29.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239291 s, 43.8 MB/s 2024-04-25T10:14:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275179 s, 38.1 MB/s 2024-04-25T10:14:29.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238165 s, 44.0 MB/s 2024-04-25T10:14:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020263 s, 51.7 MB/s 2024-04-25T10:14:29.118 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.119 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.119 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200654 s, 52.3 MB/s 2024-04-25T10:14:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270044 s, 38.8 MB/s 2024-04-25T10:14:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197209 s, 53.2 MB/s 2024-04-25T10:14:29.200 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.201 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239778 s, 43.7 MB/s 2024-04-25T10:14:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181112 s, 57.9 MB/s 2024-04-25T10:14:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.248 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224605 s, 46.7 MB/s 2024-04-25T10:14:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219694 s, 47.7 MB/s 2024-04-25T10:14:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246708 s, 42.5 MB/s 2024-04-25T10:14:29.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.329 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222343 s, 47.2 MB/s 2024-04-25T10:14:29.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.358 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241164 s, 43.5 MB/s 2024-04-25T10:14:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02558 s, 41.0 MB/s 2024-04-25T10:14:29.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024651 s, 42.5 MB/s 2024-04-25T10:14:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251438 s, 41.7 MB/s 2024-04-25T10:14:29.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.471 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229291 s, 45.7 MB/s 2024-04-25T10:14:29.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227 s, 46.2 MB/s 2024-04-25T10:14:29.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228493 s, 45.9 MB/s 2024-04-25T10:14:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251231 s, 41.7 MB/s 2024-04-25T10:14:29.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.585 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.585 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274054 s, 38.3 MB/s 2024-04-25T10:14:29.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.609 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.609 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200067 s, 52.4 MB/s 2024-04-25T10:14:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255331 s, 41.1 MB/s 2024-04-25T10:14:29.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:29.642 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221863 s, 47.3 MB/s 2024-04-25T10:14:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222617 s, 47.1 MB/s 2024-04-25T10:14:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246635 s, 42.5 MB/s 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216339 s, 48.5 MB/s 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022585 s, 46.4 MB/s 2024-04-25T10:14:29.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:29.841 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:29.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.861 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218977 s, 47.9 MB/s 2024-04-25T10:14:29.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.882 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187823 s, 55.8 MB/s 2024-04-25T10:14:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195725 s, 53.6 MB/s 2024-04-25T10:14:29.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0167596 s, 62.6 MB/s 2024-04-25T10:14:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208245 s, 50.4 MB/s 2024-04-25T10:14:29.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.972 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203297 s, 51.6 MB/s 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:29.996 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020637 s, 50.8 MB/s 2024-04-25T10:14:30.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233314 s, 44.9 MB/s 2024-04-25T10:14:30.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237278 s, 44.2 MB/s 2024-04-25T10:14:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200821 s, 52.2 MB/s 2024-04-25T10:14:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021756 s, 48.2 MB/s 2024-04-25T10:14:30.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.128 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.128 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230911 s, 45.4 MB/s 2024-04-25T10:14:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183739 s, 57.1 MB/s 2024-04-25T10:14:30.180 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269591 s, 38.9 MB/s 2024-04-25T10:14:30.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.206 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207877 s, 50.4 MB/s 2024-04-25T10:14:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238771 s, 43.9 MB/s 2024-04-25T10:14:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235256 s, 44.6 MB/s 2024-04-25T10:14:30.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.289 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230709 s, 45.5 MB/s 2024-04-25T10:14:30.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.320 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264328 s, 39.7 MB/s 2024-04-25T10:14:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229739 s, 45.6 MB/s 2024-04-25T10:14:30.368 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197339 s, 53.1 MB/s 2024-04-25T10:14:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230405 s, 45.5 MB/s 2024-04-25T10:14:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244747 s, 42.8 MB/s 2024-04-25T10:14:30.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.452 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230844 s, 45.4 MB/s 2024-04-25T10:14:30.476 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.476 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.476 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205272 s, 51.1 MB/s 2024-04-25T10:14:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243824 s, 43.0 MB/s 2024-04-25T10:14:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205417 s, 51.0 MB/s 2024-04-25T10:14:30.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:14:30.533 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:14:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02613 s, 40.1 MB/s 2024-04-25T10:14:30.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.693 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223369 s, 46.9 MB/s 2024-04-25T10:14:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021336 s, 49.1 MB/s 2024-04-25T10:14:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228493 s, 45.9 MB/s 2024-04-25T10:14:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210508 s, 49.8 MB/s 2024-04-25T10:14:30.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.795 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200028 s, 52.4 MB/s 2024-04-25T10:14:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269457 s, 38.9 MB/s 2024-04-25T10:14:30.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.852 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022279 s, 47.1 MB/s 2024-04-25T10:14:30.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236031 s, 44.4 MB/s 2024-04-25T10:14:30.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246712 s, 42.5 MB/s 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242769 s, 43.2 MB/s 2024-04-25T10:14:30.963 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.963 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236152 s, 44.4 MB/s 2024-04-25T10:14:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023804 s, 44.1 MB/s 2024-04-25T10:14:31.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.019 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228724 s, 45.8 MB/s 2024-04-25T10:14:31.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195653 s, 53.6 MB/s 2024-04-25T10:14:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234249 s, 44.8 MB/s 2024-04-25T10:14:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.099 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239874 s, 43.7 MB/s 2024-04-25T10:14:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217643 s, 48.2 MB/s 2024-04-25T10:14:31.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.151 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223534 s, 46.9 MB/s 2024-04-25T10:14:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.175 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202198 s, 51.9 MB/s 2024-04-25T10:14:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258168 s, 40.6 MB/s 2024-04-25T10:14:31.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231645 s, 45.3 MB/s 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208908 s, 50.2 MB/s 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212341 s, 49.4 MB/s 2024-04-25T10:14:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202913 s, 51.7 MB/s 2024-04-25T10:14:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200427 s, 52.3 MB/s 2024-04-25T10:14:31.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205685 s, 51.0 MB/s 2024-04-25T10:14:31.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.380 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213495 s, 49.1 MB/s 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.405 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213543 s, 49.1 MB/s 2024-04-25T10:14:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210012 s, 49.9 MB/s 2024-04-25T10:14:31.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.458 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252816 s, 41.5 MB/s 2024-04-25T10:14:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:14:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:14:31.482 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195873 s, 53.5 MB/s 2024-04-25T10:14:31.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:31.485 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:14:31.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.688+0000 7f20bf400640 1 Processor -- start 2024-04-25T10:14:31.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.688+0000 7f20bf400640 1 -- start start 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f20b80cf640 0x7f20b806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b80ac8b0 0x7f20b80accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bf400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20b80cfad0 con 0x7f20b815b710 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bf400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20b806f190 con 0x7f20b80cf640 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bf400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815e490 con 0x7f20b80ac8b0 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20b7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f20b80cf640 0x7f20b806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bca00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f20b815b710 0x7f20b815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42846/0 (socket says 172.21.15.165:42846) 2024-04-25T10:14:31.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.690+0000 7f20bca00640 1 -- 172.21.15.165:0/2128894632 learned_addr learned my addr 172.21.15.165:0/2128894632 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.691+0000 7f20bd400640 1 --2- 172.21.15.165:0/2128894632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b80ac8b0 0x7f20b80accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.691+0000 7f20b7e00640 1 -- 172.21.15.165:0/2128894632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b80ac8b0 msgr2=0x7f20b80accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.691+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2128894632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b80ac8b0 0x7f20b80accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.691+0000 7f20b7e00640 1 -- 172.21.15.165:0/2128894632 >> v1:172.21.15.165:6789/0 conn(0x7f20b815b710 legacy=0x7f20b815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.691+0000 7f20b7e00640 1 -- 172.21.15.165:0/2128894632 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80cfd80 con 0x7f20b80cf640 2024-04-25T10:14:31.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.692+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2128894632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b806ec50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f20a40097a0 tx=0x7f20a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f6ddb778d14343 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.692+0000 7f20b7400640 1 -- 172.21.15.165:0/2128894632 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4037070 con 0x7f20b80cf640 2024-04-25T10:14:31.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.692+0000 7f20b7400640 1 -- 172.21.15.165:0/2128894632 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4032440 con 0x7f20b80cf640 2024-04-25T10:14:31.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.692+0000 7f20b7400640 1 -- 172.21.15.165:0/2128894632 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4032810 con 0x7f20b80cf640 2024-04-25T10:14:31.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 -- 172.21.15.165:0/2128894632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 msgr2=0x7f20b806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 --2- 172.21.15.165:0/2128894632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b806ec50 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f20a40097a0 tx=0x7f20a402a180 comp rx=0 tx=0).stop 2024-04-25T10:14:31.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 -- 172.21.15.165:0/2128894632 shutdown_connections 2024-04-25T10:14:31.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 --2- 172.21.15.165:0/2128894632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b80ac8b0 0x7f20b80accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 --2- 172.21.15.165:0/2128894632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b806ec50 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 -- 172.21.15.165:0/2128894632 >> 172.21.15.165:0/2128894632 conn(0x7f20b806abe0 msgr2=0x7f20b806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 -- 172.21.15.165:0/2128894632 shutdown_connections 2024-04-25T10:14:31.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.693+0000 7f20bf400640 1 -- 172.21.15.165:0/2128894632 wait complete. 2024-04-25T10:14:31.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.695+0000 7f20bf400640 1 Processor -- start 2024-04-25T10:14:31.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.696+0000 7f20bf400640 1 -- start start 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20bf400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20b80ac8b0 0x7f20b80b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b80b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20bf400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b815b710 0x7f20b80b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20bf400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815baf0 con 0x7f20b80cf640 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20bf400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b80d0340 con 0x7f20b80ac8b0 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20bf400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b815bc70 con 0x7f20b815b710 2024-04-25T10:14:31.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b80b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b80b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51100/0 (socket says 172.21.15.165:51100) 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 -- 172.21.15.165:0/1665552830 learned_addr learned my addr 172.21.15.165:0/1665552830 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 -- 172.21.15.165:0/1665552830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b815b710 msgr2=0x7f20b80b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b815b710 0x7f20b80b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 -- 172.21.15.165:0/1665552830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20b80ac8b0 msgr2=0x7f20b80b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20b80ac8b0 0x7f20b80b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.697+0000 7f20b7e00640 1 -- 172.21.15.165:0/1665552830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20b80b25b0 con 0x7f20b80cf640 2024-04-25T10:14:31.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.698+0000 7f20b7e00640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b80b3b90 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f20a4035040 tx=0x7f20a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.698+0000 7f20b4c00640 1 -- 172.21.15.165:0/1665552830 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4037040 con 0x7f20b80cf640 2024-04-25T10:14:31.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.698+0000 7f20b4c00640 1 -- 172.21.15.165:0/1665552830 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a4030770 con 0x7f20b80cf640 2024-04-25T10:14:31.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.698+0000 7f20b4c00640 1 -- 172.21.15.165:0/1665552830 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4030b80 con 0x7f20b80cf640 2024-04-25T10:14:31.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.698+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b80b2890 con 0x7f20b80cf640 2024-04-25T10:14:31.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.698+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b80b2e50 con 0x7f20b80cf640 2024-04-25T10:14:31.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.701+0000 7f20b4c00640 1 -- 172.21.15.165:0/1665552830 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f20a4044020 con 0x7f20b80cf640 2024-04-25T10:14:31.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.702+0000 7f20b4c00640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f209806b4d0 0x7f209806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.702+0000 7f20b4c00640 1 -- 172.21.15.165:0/1665552830 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20a40aa910 con 0x7f20b80cf640 2024-04-25T10:14:31.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.702+0000 7f20bca00640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f209806b4d0 0x7f209806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.702+0000 7f20c23c5740 1 Processor -- start 2024-04-25T10:14:31.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.703+0000 7f20c23c5740 1 -- start start 2024-04-25T10:14:31.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.703+0000 7f20bca00640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f209806b4d0 0x7f209806d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f20ac000f60 tx=0x7f20ac007a80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e611b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e7ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e36ab0 0x5633f9e7cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20c23c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633f9d94f00 con 0x5633f9e7e670 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633f9d99f50 con 0x5633f9e60dd0 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633f9d430b0 con 0x5633f9e36ab0 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e7ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e7ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51104/0 (socket says 172.21.15.165:51104) 2024-04-25T10:14:31.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 -- 172.21.15.165:0/477418937 learned_addr learned my addr 172.21.15.165:0/477418937 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20bca00640 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e611b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20bd400640 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e36ab0 0x5633f9e7cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 -- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e36ab0 msgr2=0x5633f9e7cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e36ab0 0x5633f9e7cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 -- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 msgr2=0x5633f9e611b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e611b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20b7e00640 1 -- 172.21.15.165:0/477418937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633f9b7aba0 con 0x5633f9e7e670 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.704+0000 7f20bca00640 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.705+0000 7f20b7e00640 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e7ebc0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f20a40810d0 tx=0x7f20a4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24aa343e31ef7cd4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.705+0000 7f20a9200640 1 -- 172.21.15.165:0/477418937 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4078ef0 con 0x5633f9e7e670 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.705+0000 7f20a9200640 1 -- 172.21.15.165:0/477418937 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a407d040 con 0x5633f9e7e670 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.705+0000 7f20a9200640 1 -- 172.21.15.165:0/477418937 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4081600 con 0x5633f9e7e670 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 -- 172.21.15.165:0/477418937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 msgr2=0x5633f9e7ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e7ebc0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f20a40810d0 tx=0x7f20a4009190 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 -- 172.21.15.165:0/477418937 shutdown_connections 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e36ab0 0x5633f9e7cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e7ebc0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 --2- 172.21.15.165:0/477418937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e611b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 -- 172.21.15.165:0/477418937 >> 172.21.15.165:0/477418937 conn(0x5633f9e33670 msgr2=0x5633f9e7bbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 -- 172.21.15.165:0/477418937 shutdown_connections 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.706+0000 7f20c23c5740 1 -- 172.21.15.165:0/477418937 wait complete. 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.707+0000 7f20c23c5740 1 -- start start 2024-04-25T10:14:31.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.707+0000 7f20c23c5740 1 Processor -- start 2024-04-25T10:14:31.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e8ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e5a880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9e63ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633f9d38620 con 0x5633f9e7e670 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633f9d95ff0 con 0x5633f9e60dd0 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633f9d94f00 con 0x5633f9e63a70 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20c23c5740 10 client.0 _ll_register_callbacks cb 0x5633f9aadf80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e5a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e5a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51112/0 (socket says 172.21.15.165:51112) 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 -- 172.21.15.165:0/2486446875 learned_addr learned my addr 172.21.15.165:0/2486446875 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 -- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 msgr2=0x5633f9e63ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9e63ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 -- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 msgr2=0x5633f9e8ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.708+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e8ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f20b7e00640 1 -- 172.21.15.165:0/2486446875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633f9a6e490 con 0x5633f9e7e670 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e5a880 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f20a40097a0 tx=0x7f20a402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bf503f7f5dc729c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f209d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:31.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f209d600640 1 -- 172.21.15.165:0/2486446875 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4078ef0 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f209d600640 1 -- 172.21.15.165:0/2486446875 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20a40771f0 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20a407d040 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f209d600640 1 -- 172.21.15.165:0/2486446875 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4030d20 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f20c23c5740 10 client.8775 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5633f9bdb330 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.709+0000 7f209d600640 1 -- 172.21.15.165:0/2486446875 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20a407d040 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f209d600640 1 -- 172.21.15.165:0/2486446875 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20a403a030 con 0x5633f9e7e670 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f20c23c5740 10 client.8775 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f20c23c5740 10 client.8775 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f20c23c5740 20 client.8775 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e8fbf0 0x5633f9e90010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f20c23c5740 4 client.8775 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.710+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633f9da7490 con 0x5633f9e8fbf0 2024-04-25T10:14:31.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.711+0000 7f20bca00640 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e8fbf0 0x5633f9e90010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.712+0000 7f209d600640 10 client.8775 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:31.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.712+0000 7f20bca00640 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e8fbf0 0x5633f9e90010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f88d25cc16b84a server_cookie=bdf72cb5e556dea8 in_seq=0 out_seq=0 2024-04-25T10:14:31.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f209d600640 1 -- 172.21.15.165:0/2486446875 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5633f9da7490 con 0x5633f9e8fbf0 2024-04-25T10:14:31.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f209d600640 10 client.8775 handle_command_reply: tid=0 2024-04-25T10:14:31.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 client.8775 shutdown 2024-04-25T10:14:31.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 msgr2=0x5633f9e5a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e5a880 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f20a40097a0 tx=0x7f20a402a180 comp rx=0 tx=0).stop 2024-04-25T10:14:31.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 shutdown_connections 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e8fbf0 0x5633f9e90010 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9e63ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e7e670 0x5633f9e5a880 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2486446875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9e8ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 >> 172.21.15.165:0/2486446875 conn(0x5633f9e36ab0 msgr2=0x5633f9e35fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f209d600640 0 client.8775 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 shutdown_connections 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 1 -- 172.21.15.165:0/2486446875 wait complete. 2024-04-25T10:14:31.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.714+0000 7f20c23c5740 20 client.8775 trim_cache size 0 max 16384 2024-04-25T10:14:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 Processor -- start 2024-04-25T10:14:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 -- start start 2024-04-25T10:14:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 0x5633f9e306d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9ee66e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e7e670 0x5633f9e8aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b82005a0 con 0x5633f9e60dd0 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81ff7a0 con 0x5633f9e7e670 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.731+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81febd0 con 0x5633f9e63a70 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 0x5633f9e306d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 0x5633f9e306d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51124/0 (socket says 172.21.15.165:51124) 2024-04-25T10:14:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 -- 172.21.15.165:0/1704135585 learned_addr learned my addr 172.21.15.165:0/1704135585 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20b7e00640 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9ee66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bd400640 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e7e670 0x5633f9e8aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 -- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 msgr2=0x5633f9ee66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9ee66e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 -- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e7e670 msgr2=0x5633f9e8aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e7e670 0x5633f9e8aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.732+0000 7f20bca00640 1 -- 172.21.15.165:0/1704135585 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633f9a47610 con 0x5633f9e60dd0 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.733+0000 7f20bca00640 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 0x5633f9e306d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007fd0 tx=0x7f20ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f196e0f20ad378a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.733+0000 7f209fe00640 1 -- 172.21.15.165:0/1704135585 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac0203c0 con 0x5633f9e60dd0 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.733+0000 7f209fe00640 1 -- 172.21.15.165:0/1704135585 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac020560 con 0x5633f9e60dd0 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.733+0000 7f209fe00640 1 -- 172.21.15.165:0/1704135585 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac014430 con 0x5633f9e60dd0 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.733+0000 7f20c23c5740 1 -- 172.21.15.165:0/1704135585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 msgr2=0x5633f9e306d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.733+0000 7f20c23c5740 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 0x5633f9e306d0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007fd0 tx=0x7f20ac0057f0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 -- 172.21.15.165:0/1704135585 shutdown_connections 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e7e670 0x5633f9e8aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e63a70 0x5633f9ee66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 --2- 172.21.15.165:0/1704135585 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e60dd0 0x5633f9e306d0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 -- 172.21.15.165:0/1704135585 >> 172.21.15.165:0/1704135585 conn(0x5633f9e36ab0 msgr2=0x5633f9e645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 -- 172.21.15.165:0/1704135585 shutdown_connections 2024-04-25T10:14:31.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 -- 172.21.15.165:0/1704135585 wait complete. 2024-04-25T10:14:31.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.734+0000 7f20c23c5740 1 -- start start 2024-04-25T10:14:31.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.735+0000 7f20c23c5740 1 Processor -- start 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.735+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9ee26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.735+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 0x5633f9ee5c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20c23c5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e7e670 0x5633f9e60220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20c23c5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633f9e62f30 con 0x5633f9e63a70 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b81ff1a0 con 0x5633f9e60dd0 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20c23c5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633f9e333f0 con 0x5633f9e7e670 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20c23c5740 10 client.0 _ll_register_callbacks cb 0x5633f9e31a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 0x5633f9ee5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 0x5633f9ee5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51136/0 (socket says 172.21.15.165:51136) 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 -- 172.21.15.165:0/2554371512 learned_addr learned my addr 172.21.15.165:0/2554371512 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20bd400640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e7e670 0x5633f9e60220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 -- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e7e670 msgr2=0x5633f9e60220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e7e670 0x5633f9e60220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 -- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 msgr2=0x5633f9ee26c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20bca00640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9ee26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9ee26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20b7e00640 1 -- 172.21.15.165:0/2554371512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5633f9e8ec40 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.736+0000 7f20bd400640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e7e670 0x5633f9e60220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20b7e00640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 0x5633f9ee5c10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f20a4075300 tx=0x7f20a4047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34ce6380e0cef7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f209d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f209d600640 1 -- 172.21.15.165:0/2554371512 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a407d040 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f209d600640 1 -- 172.21.15.165:0/2554371512 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f20a4081170 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20ac020560 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f209d600640 1 -- 172.21.15.165:0/2554371512 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4078ef0 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20c23c5740 10 client.8781 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x5633f9e8ec40 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f209d600640 1 -- 172.21.15.165:0/2554371512 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20ac020560 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f209d600640 1 -- 172.21.15.165:0/2554371512 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f20a4078340 con 0x5633f9e63a70 2024-04-25T10:14:31.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20c23c5740 10 client.8781 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20c23c5740 10 client.8781 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.737+0000 7f20c23c5740 20 client.8781 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.738+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e60760 0x5633f9e8aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.738+0000 7f20c23c5740 4 client.8781 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.738+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5633f9e5c850 con 0x5633f9e60760 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.738+0000 7f20bca00640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e60760 0x5633f9e8aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.739+0000 7f20bca00640 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e60760 0x5633f9e8aaf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14f70dc47b75df75 server_cookie=e111db17b24a144b in_seq=0 out_seq=0 2024-04-25T10:14:31.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.739+0000 7f209d600640 10 client.8781 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f209d600640 1 -- 172.21.15.165:0/2554371512 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5633f9e5c850 con 0x5633f9e60760 2024-04-25T10:14:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f209d600640 10 client.8781 handle_command_reply: tid=0 2024-04-25T10:14:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 client.8781 shutdown 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 msgr2=0x5633f9ee5c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 0x5633f9ee5c10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f20a4075300 tx=0x7f20a4047000 comp rx=0 tx=0).stop 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 shutdown_connections 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x5633f9e60760 0x5633f9e8aaf0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5633f9e7e670 0x5633f9e60220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5633f9e63a70 0x5633f9ee5c10 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 --2- 172.21.15.165:0/2554371512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5633f9e60dd0 0x5633f9ee26c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 >> 172.21.15.165:0/2554371512 conn(0x5633f9e36ab0 msgr2=0x5633f9e5b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f209d600640 0 client.8781 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 shutdown_connections 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.746+0000 7f20c23c5740 1 -- 172.21.15.165:0/2554371512 wait complete. 2024-04-25T10:14:31.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.747+0000 7f20c23c5740 20 client.8781 trim_cache size 0 max 16384 2024-04-25T10:14:31.752 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:31.752 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:14:31.752 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "a47cc4f9-ad62-4fd7-8e1a-1eece39733c7", 2024-04-25T10:14:31.752 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:14:31.752 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.747+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f209806b4d0 msgr2=0x7f209806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.747+0000 7f20bf400640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f209806b4d0 0x7f209806d990 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f20ac000f60 tx=0x7f20ac007a80 comp rx=0 tx=0).stop 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.747+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 msgr2=0x7f20b80b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.747+0000 7f20bf400640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b80b3b90 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f20a4035040 tx=0x7f20a40046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 shutdown_connections 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f209806b4d0 0x7f209806d990 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20b815b710 0x7f20b80b40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20b80cf640 0x7f20b80b3b90 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 --2- 172.21.15.165:0/1665552830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f20b80ac8b0 0x7f20b80b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 >> 172.21.15.165:0/1665552830 conn(0x7f20b806abe0 msgr2=0x7f20b815d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 shutdown_connections 2024-04-25T10:14:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.748+0000 7f20bf400640 1 -- 172.21.15.165:0/1665552830 wait complete. 2024-04-25T10:14:31.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:31.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub pause 2024-04-25T10:14:31.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.979+0000 7fcb37200640 1 Processor -- start 2024-04-25T10:14:31.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.979+0000 7fcb37200640 1 -- start start 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.980+0000 7fcb37200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb3815d9c0 0x7fcb3806c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.980+0000 7fcb37200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380cecb0 0x7fcb380d21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.980+0000 7fcb37200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3806c880 con 0x7fcb380b9260 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.980+0000 7fcb37200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb380d1a30 con 0x7fcb3815d9c0 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.980+0000 7fcb37200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3806cb60 con 0x7fcb380cecb0 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb3815d9c0 0x7fcb3806c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcb3815d9c0 0x7fcb3806c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51146/0 (socket says 172.21.15.165:51146) 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 -- 172.21.15.165:0/1655078211 learned_addr learned my addr 172.21.15.165:0/1655078211 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb36800640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380cecb0 0x7fcb380d21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 -- 172.21.15.165:0/1655078211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380cecb0 msgr2=0x7fcb380d21a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380cecb0 0x7fcb380d21a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 -- 172.21.15.165:0/1655078211 >> v1:172.21.15.165:6789/0 conn(0x7fcb380b9260 legacy=0x7fcb380ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb35e00640 1 -- 172.21.15.165:0/1655078211 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3806ce40 con 0x7fcb3815d9c0 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb34a00640 1 -- 172.21.15.165:0/1655078211 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1176589612 0 0) 0x7fcb3806c880 con 0x7fcb380b9260 2024-04-25T10:14:31.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.981+0000 7fcb36800640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380cecb0 0x7fcb380d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:31.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.982+0000 7fcb35e00640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb3806c030 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009e50 tx=0x7fcb3002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3150d53e9e7d5573 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb34a00640 1 -- 172.21.15.165:0/1655078211 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30036070 con 0x7fcb3815d9c0 2024-04-25T10:14:31.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb34a00640 1 -- 172.21.15.165:0/1655078211 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb30031600 con 0x7fcb3815d9c0 2024-04-25T10:14:31.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb34a00640 1 -- 172.21.15.165:0/1655078211 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb300319d0 con 0x7fcb3815d9c0 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 -- 172.21.15.165:0/1655078211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 msgr2=0x7fcb3806c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb3806c030 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7fcb30009e50 tx=0x7fcb3002a7a0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 -- 172.21.15.165:0/1655078211 shutdown_connections 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380cecb0 0x7fcb380d21a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 --2- 172.21.15.165:0/1655078211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb3806c030 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 -- 172.21.15.165:0/1655078211 >> 172.21.15.165:0/1655078211 conn(0x7fcb380d0770 msgr2=0x7fcb380d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 -- 172.21.15.165:0/1655078211 shutdown_connections 2024-04-25T10:14:31.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.983+0000 7fcb37200640 1 -- 172.21.15.165:0/1655078211 wait complete. 2024-04-25T10:14:31.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.985+0000 7fcb37200640 1 Processor -- start 2024-04-25T10:14:31.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.985+0000 7fcb37200640 1 -- start start 2024-04-25T10:14:31.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb37200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380b9260 0x7fcb381497f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb37200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb380cecb0 0x7fcb380ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb37200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb380c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb37200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3815dda0 con 0x7fcb3815d9c0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb37200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3806c570 con 0x7fcb380cecb0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb37200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3815df20 con 0x7fcb380b9260 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb36800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb380c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb36800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb380c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51148/0 (socket says 172.21.15.165:51148) 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.986+0000 7fcb36800640 1 -- 172.21.15.165:0/1270015248 learned_addr learned my addr 172.21.15.165:0/1270015248 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb35e00640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380b9260 0x7fcb381497f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb35400640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb380cecb0 0x7fcb380ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb36800640 1 -- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380b9260 msgr2=0x7fcb381497f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb36800640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380b9260 0x7fcb381497f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb36800640 1 -- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb380cecb0 msgr2=0x7fcb380ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb36800640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb380cecb0 0x7fcb380ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb36800640 1 -- 172.21.15.165:0/1270015248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3814a2f0 con 0x7fcb3815d9c0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb36800640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb380c9420 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800c820 tx=0x7fcb2800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.987+0000 7fcb2e000640 1 -- 172.21.15.165:0/1270015248 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28015070 con 0x7fcb3815d9c0 2024-04-25T10:14:31.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.988+0000 7fcb2e000640 1 -- 172.21.15.165:0/1270015248 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28007a30 con 0x7fcb3815d9c0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.988+0000 7fcb2e000640 1 -- 172.21.15.165:0/1270015248 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28007e20 con 0x7fcb3815d9c0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.988+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3814a5d0 con 0x7fcb3815d9c0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.988+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3814ab90 con 0x7fcb3815d9c0 2024-04-25T10:14:31.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.990+0000 7fcb2e000640 1 -- 172.21.15.165:0/1270015248 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcb28018070 con 0x7fcb3815d9c0 2024-04-25T10:14:31.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.991+0000 7fcb2e000640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb1806b500 0x7fcb1806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.991+0000 7fcb2e000640 1 -- 172.21.15.165:0/1270015248 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb2808a2d0 con 0x7fcb3815d9c0 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.991+0000 7fcb35e00640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb1806b500 0x7fcb1806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.992+0000 7fcb3f3fd740 1 Processor -- start 2024-04-25T10:14:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.992+0000 7fcb3f3fd740 1 -- start start 2024-04-25T10:14:31.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.992+0000 7fcb35e00640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb1806b500 0x7fcb1806d9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcb30033000 tx=0x7fcb30009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883979c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a883949a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883949fb0 0x55a883971cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a8838b2870 con 0x55a883979870 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88385bd60 con 0x55a88397a290 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a8838ae910 con 0x55a883949fb0 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883979c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883979c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51154/0 (socket says 172.21.15.165:51154) 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 -- 172.21.15.165:0/3964873831 learned_addr learned my addr 172.21.15.165:0/3964873831 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb36800640 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883949fb0 0x55a883971cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35400640 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a883949a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 -- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883949fb0 msgr2=0x55a883971cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883949fb0 0x55a883971cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 -- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 msgr2=0x55a883949a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a883949a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.993+0000 7fcb35e00640 1 -- 172.21.15.165:0/3964873831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8836917d0 con 0x55a883979870 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.994+0000 7fcb35e00640 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883979c50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fcb30034040 tx=0x7fcb3002e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdfade394a3e792f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.994+0000 7fcb22600640 1 -- 172.21.15.165:0/3964873831 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb3004c070 con 0x55a883979870 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.994+0000 7fcb22600640 1 -- 172.21.15.165:0/3964873831 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3003c990 con 0x55a883979870 2024-04-25T10:14:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.994+0000 7fcb22600640 1 -- 172.21.15.165:0/3964873831 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30036040 con 0x55a883979870 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/3964873831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 msgr2=0x55a883979c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883979c50 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7fcb30034040 tx=0x7fcb3002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/3964873831 shutdown_connections 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883949fb0 0x55a883971cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a883949a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/3964873831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883979c50 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/3964873831 >> 172.21.15.165:0/3964873831 conn(0x55a88394bef0 msgr2=0x55a883994680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/3964873831 shutdown_connections 2024-04-25T10:14:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.995+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/3964873831 wait complete. 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.996+0000 7fcb3f3fd740 1 -- start start 2024-04-25T10:14:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.996+0000 7fcb3f3fd740 1 Processor -- start 2024-04-25T10:14:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883993b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a8839a7850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a8839a7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a883975fc0 con 0x55a883979870 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88386f1c0 con 0x55a88397a290 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a883976260 con 0x55a88396bb60 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb3f3fd740 10 client.0 _ll_register_callbacks cb 0x55a8838ad8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883993b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb35e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883993b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51158/0 (socket says 172.21.15.165:51158) 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb35e00640 1 -- 172.21.15.165:0/2676137152 learned_addr learned my addr 172.21.15.165:0/2676137152 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb35400640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a8839a7850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.997+0000 7fcb36800640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a8839a7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb35e00640 1 -- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 msgr2=0x55a8839a7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb35e00640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a8839a7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb35e00640 1 -- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 msgr2=0x55a8839a7850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb35e00640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a8839a7850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb35e00640 1 -- 172.21.15.165:0/2676137152 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a8835870f0 con 0x55a883979870 2024-04-25T10:14:32.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb36800640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a8839a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb35e00640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883993b80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fcb30034040 tx=0x7fcb3002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66480c20ef0a15b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb16a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb16a00640 1 -- 172.21.15.165:0/2676137152 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb30055060 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb16a00640 1 -- 172.21.15.165:0/2676137152 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb300478d0 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb3003c990 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb16a00640 1 -- 172.21.15.165:0/2676137152 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb3005b050 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb3f3fd740 10 client.8796 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.998+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a8836f1f60 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb16a00640 1 -- 172.21.15.165:0/2676137152 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb3003c990 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb16a00640 1 -- 172.21.15.165:0/2676137152 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb30008030 con 0x55a883979870 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb3f3fd740 10 client.8796 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb3f3fd740 10 client.8796 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb3f3fd740 20 client.8796 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:32.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88396e2b0 0x55a88396af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb3f3fd740 4 client.8796 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a883888690 con 0x55a88396e2b0 2024-04-25T10:14:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:31.999+0000 7fcb35400640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88396e2b0 0x55a88396af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.000+0000 7fcb35400640 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88396e2b0 0x55a88396af70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=604dc86e240de4d3 server_cookie=3325935f83472fde in_seq=0 out_seq=0 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.000+0000 7fcb16a00640 10 client.8796 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb16a00640 1 -- 172.21.15.165:0/2676137152 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a883888690 con 0x55a88396e2b0 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb16a00640 10 client.8796 handle_command_reply: tid=0 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 client.8796 shutdown 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 msgr2=0x55a883993b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883993b80 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7fcb30034040 tx=0x7fcb3002c000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 shutdown_connections 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88396e2b0 0x55a88396af70 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a8839a7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a88397a290 0x55a8839a7850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2676137152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883993b80 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 >> 172.21.15.165:0/2676137152 conn(0x55a883949fb0 msgr2=0x55a883994480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb16a00640 0 client.8796 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 shutdown_connections 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2676137152 wait complete. 2024-04-25T10:14:32.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.002+0000 7fcb3f3fd740 20 client.8796 trim_cache size 0 max 16384 2024-04-25T10:14:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.018+0000 7fcb3f3fd740 1 Processor -- start 2024-04-25T10:14:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.018+0000 7fcb3f3fd740 1 -- start start 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a88396af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 0x55a8839a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883979870 0x55a883948c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb38200050 con 0x55a88396bb60 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb381ff1f0 con 0x55a883949fb0 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3814cc50 con 0x55a883979870 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 0x55a8839a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb35400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 0x55a8839a79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51172/0 (socket says 172.21.15.165:51172) 2024-04-25T10:14:32.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.019+0000 7fcb35400640 1 -- 172.21.15.165:0/2999141515 learned_addr learned my addr 172.21.15.165:0/2999141515 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb36800640 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883979870 0x55a883948c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35400640 1 -- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883979870 msgr2=0x55a883948c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35400640 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883979870 0x55a883948c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35e00640 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a88396af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35400640 1 -- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 msgr2=0x55a88396af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35400640 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a88396af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35400640 1 -- 172.21.15.165:0/2999141515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a8835601b0 con 0x55a88396bb60 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb35400640 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 0x55a8839a79d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fcb24000f60 tx=0x7fcb24001c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1cc31318241f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb21c00640 1 -- 172.21.15.165:0/2999141515 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb240053d0 con 0x55a88396bb60 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb21c00640 1 -- 172.21.15.165:0/2999141515 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24018e50 con 0x55a88396bb60 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.020+0000 7fcb21c00640 1 -- 172.21.15.165:0/2999141515 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb24018710 con 0x55a88396bb60 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2999141515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 msgr2=0x55a8839a79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 0x55a8839a79d0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7fcb24000f60 tx=0x7fcb24001c80 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2999141515 shutdown_connections 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a883979870 0x55a883948c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a88396bb60 0x55a8839a79d0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/2999141515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a88396af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2999141515 >> 172.21.15.165:0/2999141515 conn(0x55a883950e30 msgr2=0x55a8839a6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2999141515 shutdown_connections 2024-04-25T10:14:32.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.021+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/2999141515 wait complete. 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.022+0000 7fcb3f3fd740 1 -- start start 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.022+0000 7fcb3f3fd740 1 Processor -- start 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a883948960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a883948ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883992050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb381fec20 con 0x55a883979870 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a88394fde0 con 0x55a883949fb0 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a88396b2e0 con 0x55a88396bb60 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb3f3fd740 10 client.0 _ll_register_callbacks cb 0x7fcb30034070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:32.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb36800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883992050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb36800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883992050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51176/0 (socket says 172.21.15.165:51176) 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb36800640 1 -- 172.21.15.165:0/512957479 learned_addr learned my addr 172.21.15.165:0/512957479 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb35e00640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a883948960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.023+0000 7fcb35400640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a883948ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb36800640 1 -- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 msgr2=0x55a883948ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb36800640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a883948ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb36800640 1 -- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 msgr2=0x55a883948960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb36800640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a883948960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb36800640 1 -- 172.21.15.165:0/512957479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a883997b00 con 0x55a883979870 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb36800640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883992050 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fcb28054c80 tx=0x7fcb28058f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef74a22866b560f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb16a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:32.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb16a00640 1 -- 172.21.15.165:0/512957479 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2801f070 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb16a00640 1 -- 172.21.15.165:0/512957479 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcb28059b50 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcb24018e50 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb16a00640 1 -- 172.21.15.165:0/512957479 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb28088050 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb3f3fd740 10 client.8802 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a88394f770 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.024+0000 7fcb16a00640 1 -- 172.21.15.165:0/512957479 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb24018e50 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb16a00640 1 -- 172.21.15.165:0/512957479 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fcb28057c70 con 0x55a883979870 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb3f3fd740 10 client.8802 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb3f3fd740 10 client.8802 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb3f3fd740 20 client.8802 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88397a290 0x55a883992590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb3f3fd740 4 client.8802 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55a8838bfdb0 con 0x55a88397a290 2024-04-25T10:14:32.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.025+0000 7fcb35e00640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88397a290 0x55a883992590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.026+0000 7fcb35e00640 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88397a290 0x55a883992590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25e37251573f2893 server_cookie=ca7fd6187385ccc9 in_seq=0 out_seq=0 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.026+0000 7fcb16a00640 10 client.8802 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb16a00640 1 -- 172.21.15.165:0/512957479 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55a8838bfdb0 con 0x55a88397a290 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb16a00640 10 client.8802 handle_command_reply: tid=0 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 client.8802 shutdown 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 msgr2=0x55a883992050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883992050 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7fcb28054c80 tx=0x7fcb28058f20 comp rx=0 tx=0).stop 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 shutdown_connections 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a88397a290 0x55a883992590 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a883979870 0x55a883992050 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a88396bb60 0x55a883948ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 --2- 172.21.15.165:0/512957479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a883949fb0 0x55a883948960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 >> 172.21.15.165:0/512957479 conn(0x55a883950e30 msgr2=0x55a88399c4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb16a00640 0 client.8802 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 shutdown_connections 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 1 -- 172.21.15.165:0/512957479 wait complete. 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.027+0000 7fcb3f3fd740 20 client.8802 trim_cache size 0 max 16384 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:14:32.034 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb1806b500 msgr2=0x7fcb1806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb1806b500 0x7fcb1806d9c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fcb30033000 tx=0x7fcb30009000 comp rx=0 tx=0).stop 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 msgr2=0x7fcb380c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb380c9420 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800c820 tx=0x7fcb2800cce0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 shutdown_connections 2024-04-25T10:14:32.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb1806b500 0x7fcb1806d9c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb3815d9c0 0x7fcb380c9420 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb380cecb0 0x7fcb380ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 --2- 172.21.15.165:0/1270015248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcb380b9260 0x7fcb381497f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 >> 172.21.15.165:0/1270015248 conn(0x7fcb380d0770 msgr2=0x7fcb380b3330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 shutdown_connections 2024-04-25T10:14:32.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.028+0000 7fcb37200640 1 -- 172.21.15.165:0/1270015248 wait complete. 2024-04-25T10:14:32.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:32.049 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:14:32.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.229+0000 7ff338a00640 1 Processor -- start 2024-04-25T10:14:32.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.229+0000 7ff338a00640 1 -- start start 2024-04-25T10:14:32.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.230+0000 7ff338a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3340ced80 0x7ff3340d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff338a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff33415e000 con 0x7ff334161710 2024-04-25T10:14:32.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff338a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3340d1de0 con 0x7ff33415dc20 2024-04-25T10:14:32.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff338a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff334161c60 con 0x7ff3340ced80 2024-04-25T10:14:32.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff332800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff334161710 0x7ff3340d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42858/0 (socket says 172.21.15.165:42858) 2024-04-25T10:14:32.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff332800640 1 -- 172.21.15.165:0/2894067300 learned_addr learned my addr 172.21.15.165:0/2894067300 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:32.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff32be00640 1 --2- 172.21.15.165:0/2894067300 >> v2:172.21.15.172:3300/0 conn(0x7ff3340ced80 0x7ff3340d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.231+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2394909939 0 0) 0x7ff33415e000 con 0x7ff334161710 2024-04-25T10:14:32.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff318003520 con 0x7ff334161710 2024-04-25T10:14:32.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3262536927 0 0) 0x7ff318003520 con 0x7ff334161710 2024-04-25T10:14:32.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 >> v2:172.21.15.172:3300/0 conn(0x7ff3340ced80 msgr2=0x7ff3340d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 --2- 172.21.15.165:0/2894067300 >> v2:172.21.15.172:3300/0 conn(0x7ff3340ced80 0x7ff3340d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 >> v1:172.21.15.172:6789/0 conn(0x7ff33415dc20 legacy=0x7ff3340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff334161f40 con 0x7ff334161710 2024-04-25T10:14:32.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.232+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff32c0019f0 con 0x7ff334161710 2024-04-25T10:14:32.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 >> v1:172.21.15.165:6789/0 conn(0x7ff334161710 legacy=0x7ff3340d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3180039b0 0x7ff318003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff318004800 0x7ff318004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3180051c0 0x7ff31800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334161c60 con 0x7ff3180051c0 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31800c010 con 0x7ff3180039b0 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31800c210 con 0x7ff318004800 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.233+0000 7ff332800640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3180051c0 0x7ff31800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff331e00640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3180039b0 0x7ff318003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff32be00640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff318004800 0x7ff318004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff332800640 1 -- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff318004800 msgr2=0x7ff318004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff332800640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff318004800 0x7ff318004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff332800640 1 -- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3180039b0 msgr2=0x7ff318003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff332800640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3180039b0 0x7ff318003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff332800640 1 -- 172.21.15.165:0/2894067300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff318003520 con 0x7ff3180051c0 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.234+0000 7ff331e00640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3180039b0 0x7ff318003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.235+0000 7ff332800640 1 -- 172.21.15.165:0/2894067300 reap_dead start 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.235+0000 7ff332800640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3180051c0 0x7ff31800b9c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005a80 tx=0x7ff318005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78b7cbe3f2785ffa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.235+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c005650 con 0x7ff3180051c0 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.235+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c0082d0 con 0x7ff3180051c0 2024-04-25T10:14:32.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.235+0000 7ff331400640 1 -- 172.21.15.165:0/2894067300 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0086c0 con 0x7ff3180051c0 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 -- 172.21.15.165:0/2894067300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3180051c0 msgr2=0x7ff31800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3180051c0 0x7ff31800b9c0 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005a80 tx=0x7ff318005830 comp rx=0 tx=0).stop 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 -- 172.21.15.165:0/2894067300 shutdown_connections 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 --2- 172.21.15.165:0/2894067300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3180051c0 0x7ff31800b9c0 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 -- 172.21.15.165:0/2894067300 >> 172.21.15.165:0/2894067300 conn(0x7ff3340d0420 msgr2=0x7ff3340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 -- 172.21.15.165:0/2894067300 shutdown_connections 2024-04-25T10:14:32.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.236+0000 7ff338a00640 1 -- 172.21.15.165:0/2894067300 wait complete. 2024-04-25T10:14:32.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.238+0000 7ff338a00640 1 Processor -- start 2024-04-25T10:14:32.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.238+0000 7ff338a00640 1 -- start start 2024-04-25T10:14:32.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff338a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3180051c0 0x7ff3340cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff338a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff33415c630 0x7ff3341f3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff338a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 0x7ff3340b86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff338a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340b5e50 con 0x7ff3340b8250 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff338a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334162500 con 0x7ff33415c630 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff338a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3340b5fd0 con 0x7ff3180051c0 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 0x7ff3340b86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 0x7ff3340b86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51186/0 (socket says 172.21.15.165:51186) 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.239+0000 7ff332800640 1 -- 172.21.15.165:0/3014624499 learned_addr learned my addr 172.21.15.165:0/3014624499 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff32be00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff33415c630 0x7ff3341f3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff331e00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3180051c0 0x7ff3340cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff332800640 1 -- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3180051c0 msgr2=0x7ff3340cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff332800640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3180051c0 0x7ff3340cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff332800640 1 -- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff33415c630 msgr2=0x7ff3341f3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff332800640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff33415c630 0x7ff3341f3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff332800640 1 -- 172.21.15.165:0/3014624499 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff33415cab0 con 0x7ff3340b8250 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff331e00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3180051c0 0x7ff3340cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.240+0000 7ff332800640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 0x7ff3340b86d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005b70 tx=0x7ff32c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.241+0000 7ff32a000640 1 -- 172.21.15.165:0/3014624499 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c044070 con 0x7ff3340b8250 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.241+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3341f41f0 con 0x7ff3340b8250 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.241+0000 7ff32a000640 1 -- 172.21.15.165:0/3014624499 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c001c50 con 0x7ff3340b8250 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.241+0000 7ff32a000640 1 -- 172.21.15.165:0/3014624499 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c005650 con 0x7ff3340b8250 2024-04-25T10:14:32.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.241+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3341f47b0 con 0x7ff3340b8250 2024-04-25T10:14:32.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.245+0000 7ff32a000640 1 -- 172.21.15.165:0/3014624499 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff32c047070 con 0x7ff3340b8250 2024-04-25T10:14:32.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.245+0000 7ff33b837740 1 Processor -- start 2024-04-25T10:14:32.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.245+0000 7ff32a000640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff31006b4d0 0x7ff31006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.245+0000 7ff33b837740 1 -- start start 2024-04-25T10:14:32.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.246+0000 7ff32a000640 1 -- 172.21.15.165:0/3014624499 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff32c0aa500 con 0x7ff3340b8250 2024-04-25T10:14:32.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.246+0000 7ff32be00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff31006b4d0 0x7ff31006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff33b837740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 0x556ef864dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8688280 0x556ef864bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8676670 0x556ef8676af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff33b837740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef85afc70 con 0x556ef864bbb0 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef8558cf0 con 0x556ef8688280 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ef85abd10 con 0x556ef8676670 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff32be00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff31006b4d0 0x7ff31006d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff334160830 tx=0x7ff31c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff331e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 0x556ef864dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff331e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 0x556ef864dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51190/0 (socket says 172.21.15.165:51190) 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff331e00640 1 -- 172.21.15.165:0/2880330964 learned_addr learned my addr 172.21.15.165:0/2880330964 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff332800640 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8676670 0x556ef8676af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.247+0000 7ff32be00640 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8688280 0x556ef864bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.248+0000 7ff331e00640 1 -- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8676670 msgr2=0x556ef8676af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.248+0000 7ff331e00640 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8676670 0x556ef8676af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.248+0000 7ff331e00640 1 -- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8688280 msgr2=0x556ef864bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.248+0000 7ff331e00640 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8688280 0x556ef864bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.248+0000 7ff331e00640 1 -- 172.21.15.165:0/2880330964 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ef838e7d0 con 0x556ef864bbb0 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.248+0000 7ff332800640 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8676670 0x556ef8676af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.249+0000 7ff331e00640 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 0x556ef864dd10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff324001830 tx=0x7ff324001cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a616522fee85096e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.249+0000 7ff322600640 1 -- 172.21.15.165:0/2880330964 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff324007440 con 0x556ef864bbb0 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.249+0000 7ff322600640 1 -- 172.21.15.165:0/2880330964 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff324018070 con 0x556ef864bbb0 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.249+0000 7ff322600640 1 -- 172.21.15.165:0/2880330964 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3240067d0 con 0x556ef864bbb0 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 -- 172.21.15.165:0/2880330964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 msgr2=0x556ef864dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 0x556ef864dd10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7ff324001830 tx=0x7ff324001cf0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 -- 172.21.15.165:0/2880330964 shutdown_connections 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8676670 0x556ef8676af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8688280 0x556ef864bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 --2- 172.21.15.165:0/2880330964 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef864bbb0 0x556ef864dd10 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 -- 172.21.15.165:0/2880330964 >> 172.21.15.165:0/2880330964 conn(0x556ef8613a30 msgr2=0x556ef8691670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 -- 172.21.15.165:0/2880330964 shutdown_connections 2024-04-25T10:14:32.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.250+0000 7ff33b837740 1 -- 172.21.15.165:0/2880330964 wait complete. 2024-04-25T10:14:32.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.251+0000 7ff33b837740 1 -- start start 2024-04-25T10:14:32.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.251+0000 7ff33b837740 1 Processor -- start 2024-04-25T10:14:32.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 0x556ef8691e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef86734b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef86785d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef854e3a0 con 0x556ef8676670 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef86741c0 con 0x556ef8678170 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ef8677b00 con 0x556ef8688280 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff33b837740 10 client.0 _ll_register_callbacks cb 0x556ef856bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.252+0000 7ff331e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 0x556ef8691e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 0x556ef8691e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51200/0 (socket says 172.21.15.165:51200) 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 -- 172.21.15.165:0/3897113066 learned_addr learned my addr 172.21.15.165:0/3897113066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff32be00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef86734b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff332800640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef86785d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 -- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 msgr2=0x556ef86734b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef86734b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 -- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 msgr2=0x556ef86785d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef86785d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff331e00640 1 -- 172.21.15.165:0/3897113066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ef82c43f0 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.253+0000 7ff32be00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef86734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.254+0000 7ff331e00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 0x556ef8691e30 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff324001800 tx=0x7ff324001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e8d1c3dca3ea762 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.254+0000 7ff316a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff316a00640 1 -- 172.21.15.165:0/3897113066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32400ea00 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff316a00640 1 -- 172.21.15.165:0/3897113066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff324028c70 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff324018070 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff316a00640 1 -- 172.21.15.165:0/3897113066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff32400eba0 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff33b837740 10 client.8814 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x556ef82840f0 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.255+0000 7ff316a00640 1 -- 172.21.15.165:0/3897113066 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff324018070 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff316a00640 1 -- 172.21.15.165:0/3897113066 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff3240067d0 con 0x556ef8676670 2024-04-25T10:14:32.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff33b837740 10 client.8814 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff33b837740 10 client.8814 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff33b837740 20 client.8814 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff33b837740 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8676fc0 0x556ef8677410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff33b837740 4 client.8814 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.256+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ef85bd1b0 con 0x556ef8676fc0 2024-04-25T10:14:32.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.257+0000 7ff32be00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8676fc0 0x556ef8677410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.258+0000 7ff32be00640 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8676fc0 0x556ef8677410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2edcc004824b2e8 server_cookie=25145cfc237a082 in_seq=0 out_seq=0 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.258+0000 7ff316a00640 10 client.8814 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.259+0000 7ff316a00640 1 -- 172.21.15.165:0/3897113066 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x556ef85bd1b0 con 0x556ef8676fc0 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.259+0000 7ff316a00640 10 client.8814 handle_command_reply: tid=0 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.259+0000 7ff33b837740 1 client.8814 shutdown 2024-04-25T10:14:32.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.259+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 msgr2=0x556ef8691e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.259+0000 7ff33b837740 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 0x556ef8691e30 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7ff324001800 tx=0x7ff324001190 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 shutdown_connections 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8676fc0 0x556ef8677410 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef86785d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef86734b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 --2- 172.21.15.165:0/3897113066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8676670 0x556ef8691e30 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 >> 172.21.15.165:0/3897113066 conn(0x556ef8613a30 msgr2=0x556ef86743a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff316a00640 0 client.8814 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 shutdown_connections 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 1 -- 172.21.15.165:0/3897113066 wait complete. 2024-04-25T10:14:32.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.260+0000 7ff33b837740 20 client.8814 trim_cache size 0 max 16384 2024-04-25T10:14:32.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 Processor -- start 2024-04-25T10:14:32.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 -- start start 2024-04-25T10:14:32.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef8678790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef8674060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 0x556ef86745a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff334200b30 con 0x556ef8649cc0 2024-04-25T10:14:32.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341ffcd0 con 0x556ef8678170 2024-04-25T10:14:32.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.277+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341ff0d0 con 0x556ef8688280 2024-04-25T10:14:32.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 0x556ef86745a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 0x556ef86745a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51214/0 (socket says 172.21.15.165:51214) 2024-04-25T10:14:32.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 -- 172.21.15.165:0/1237308222 learned_addr learned my addr 172.21.15.165:0/1237308222 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 -- 172.21.15.165:0/1237308222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 msgr2=0x556ef8674060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef8674060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 -- 172.21.15.165:0/1237308222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 msgr2=0x556ef8678790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef8678790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 -- 172.21.15.165:0/1237308222 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ef8689d10 con 0x556ef8649cc0 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.278+0000 7ff332800640 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 0x556ef86745a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff32c078a10 tx=0x7ff32c02baf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12ce51a6836c2292 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff321c00640 1 -- 172.21.15.165:0/1237308222 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c07cb20 con 0x556ef8649cc0 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff321c00640 1 -- 172.21.15.165:0/1237308222 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c090020 con 0x556ef8649cc0 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff321c00640 1 -- 172.21.15.165:0/1237308222 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0810e0 con 0x556ef8649cc0 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 -- 172.21.15.165:0/1237308222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 msgr2=0x556ef86745a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 0x556ef86745a0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff32c078a10 tx=0x7ff32c02baf0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 -- 172.21.15.165:0/1237308222 shutdown_connections 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8649cc0 0x556ef86745a0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef8674060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 --2- 172.21.15.165:0/1237308222 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef8678790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 -- 172.21.15.165:0/1237308222 >> 172.21.15.165:0/1237308222 conn(0x556ef8613a30 msgr2=0x556ef86732c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 -- 172.21.15.165:0/1237308222 shutdown_connections 2024-04-25T10:14:32.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.279+0000 7ff33b837740 1 -- 172.21.15.165:0/1237308222 wait complete. 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.280+0000 7ff33b837740 1 -- start start 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.280+0000 7ff33b837740 1 Processor -- start 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef866c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef8671c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 0x556ef86721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3341ff6d0 con 0x556ef8679290 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ef86fd440 con 0x556ef8678170 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ef866caa0 con 0x556ef8688280 2024-04-25T10:14:32.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.281+0000 7ff33b837740 10 client.0 _ll_register_callbacks cb 0x556ef8679d70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 0x556ef86721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 0x556ef86721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51224/0 (socket says 172.21.15.165:51224) 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 -- 172.21.15.165:0/862995013 learned_addr learned my addr 172.21.15.165:0/862995013 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 -- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 msgr2=0x556ef8671c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef8671c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 -- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 msgr2=0x556ef866c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef866c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 -- 172.21.15.165:0/862995013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ef8687130 con 0x556ef8679290 2024-04-25T10:14:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff332800640 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 0x556ef86721a0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff32c074ee0 tx=0x7ff32c0051b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54c0242252097701 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.282+0000 7ff316a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.283+0000 7ff316a00640 1 -- 172.21.15.165:0/862995013 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff32c0750f0 con 0x556ef8679290 2024-04-25T10:14:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.283+0000 7ff316a00640 1 -- 172.21.15.165:0/862995013 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff32c0aa070 con 0x556ef8679290 2024-04-25T10:14:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.283+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff32c090020 con 0x556ef8679290 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff316a00640 1 -- 172.21.15.165:0/862995013 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff32c075290 con 0x556ef8679290 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 10 client.8820 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x556ef86e56f0 con 0x556ef8679290 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff316a00640 1 -- 172.21.15.165:0/862995013 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff32c090020 con 0x556ef8679290 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff316a00640 1 -- 172.21.15.165:0/862995013 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff32c032020 con 0x556ef8679290 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 10 client.8820 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 10 client.8820 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 20 client.8820 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8688660 0x556ef8688a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 4 client.8820 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.286+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x556ef85b7500 con 0x556ef8688660 2024-04-25T10:14:32.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.287+0000 7ff331e00640 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8688660 0x556ef8688a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.288+0000 7ff331e00640 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8688660 0x556ef8688a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4e3dcc623da8e18 server_cookie=e439371ebfc2468d in_seq=0 out_seq=0 2024-04-25T10:14:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.288+0000 7ff316a00640 10 client.8820 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff316a00640 1 -- 172.21.15.165:0/862995013 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x556ef85b7500 con 0x556ef8688660 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff316a00640 10 client.8820 handle_command_reply: tid=0 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 client.8820 shutdown 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 msgr2=0x556ef86721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 0x556ef86721a0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff32c074ee0 tx=0x7ff32c0051b0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 shutdown_connections 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x556ef8688660 0x556ef8688a70 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ef8679290 0x556ef86721a0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ef8688280 0x556ef8671c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 --2- 172.21.15.165:0/862995013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ef8678170 0x556ef866c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 >> 172.21.15.165:0/862995013 conn(0x556ef8613a30 msgr2=0x556ef8678820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.289+0000 7ff316a00640 0 client.8820 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:32.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 shutdown_connections 2024-04-25T10:14:32.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff33b837740 1 -- 172.21.15.165:0/862995013 wait complete. 2024-04-25T10:14:32.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff33b837740 20 client.8820 trim_cache size 0 max 16384 2024-04-25T10:14:32.295 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: "a47cc4f9-ad62-4fd7-8e1a-1eece39733c7": { 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "a47cc4f9-ad62-4fd7-8e1a-1eece39733c7", 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive" 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:14:32.296 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff31006b4d0 msgr2=0x7ff31006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff338a00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff31006b4d0 0x7ff31006d990 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7ff334160830 tx=0x7ff31c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 msgr2=0x7ff3340b86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.290+0000 7ff338a00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 0x7ff3340b86d0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7ff32c005b70 tx=0x7ff32c004d00 comp rx=0 tx=0).stop 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 shutdown_connections 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff31006b4d0 0x7ff31006d990 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3340b8250 0x7ff3340b86d0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff33415c630 0x7ff3341f3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 --2- 172.21.15.165:0/3014624499 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3180051c0 0x7ff3340cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 >> 172.21.15.165:0/3014624499 conn(0x7ff3340d0420 msgr2=0x7ff3340b3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 shutdown_connections 2024-04-25T10:14:32.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.291+0000 7ff338a00640 1 -- 172.21.15.165:0/3014624499 wait complete. 2024-04-25T10:14:32.311 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:32.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.496+0000 7fd7c5600640 1 Processor -- start 2024-04-25T10:14:32.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.496+0000 7fd7c5600640 1 -- start start 2024-04-25T10:14:32.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.497+0000 7fd7c5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd7c015e520 0x7fd7c015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.497+0000 7fd7c5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd7c0070ff0 0x7fd7c0071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c5600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7c00ae590 con 0x7fd7c015f040 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c5600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c00aee30 con 0x7fd7c0070ff0 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7c0071980 con 0x7fd7c015e520 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7b7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd7c015f040 0x7fd7c00adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42870/0 (socket says 172.21.15.165:42870) 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7b7e00640 1 -- 172.21.15.165:0/3716879754 learned_addr learned my addr 172.21.15.165:0/3716879754 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:32.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c4c00640 1 --2- 172.21.15.165:0/3716879754 >> v2:172.21.15.165:3300/0 conn(0x7fd7c0070ff0 0x7fd7c0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/3716879754 >> v2:172.21.15.172:3300/0 conn(0x7fd7c015e520 0x7fd7c015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7bf400640 1 -- 172.21.15.165:0/3716879754 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903886250 0 0) 0x7fd7c00ae590 con 0x7fd7c015f040 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7bf400640 1 -- 172.21.15.165:0/3716879754 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7a4003520 con 0x7fd7c015f040 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c4c00640 1 -- 172.21.15.165:0/3716879754 >> v2:172.21.15.172:3300/0 conn(0x7fd7c015e520 msgr2=0x7fd7c015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c4c00640 1 --2- 172.21.15.165:0/3716879754 >> v2:172.21.15.172:3300/0 conn(0x7fd7c015e520 0x7fd7c015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.498+0000 7fd7c4c00640 1 -- 172.21.15.165:0/3716879754 >> v1:172.21.15.165:6789/0 conn(0x7fd7c015f040 legacy=0x7fd7c00adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.499+0000 7fd7c4c00640 1 -- 172.21.15.165:0/3716879754 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00ae870 con 0x7fd7c0070ff0 2024-04-25T10:14:32.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.500+0000 7fd7c4c00640 1 --2- 172.21.15.165:0/3716879754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c0070ff0 0x7fd7c0071440 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd7c006ba10 tx=0x7fd7b802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b6ec78dfe6cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.500+0000 7fd7bf400640 1 -- 172.21.15.165:0/3716879754 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8007740 con 0x7fd7c0070ff0 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.500+0000 7fd7bf400640 1 -- 172.21.15.165:0/3716879754 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7b80078e0 con 0x7fd7c0070ff0 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.500+0000 7fd7bf400640 1 -- 172.21.15.165:0/3716879754 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7b8007c90 con 0x7fd7c0070ff0 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.500+0000 7fd7c5600640 1 -- 172.21.15.165:0/3716879754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c0070ff0 msgr2=0x7fd7c0071440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.500+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3716879754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c0070ff0 0x7fd7c0071440 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7fd7c006ba10 tx=0x7fd7b802a680 comp rx=0 tx=0).stop 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.501+0000 7fd7c5600640 1 -- 172.21.15.165:0/3716879754 shutdown_connections 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.501+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3716879754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c0070ff0 0x7fd7c0071440 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.501+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3716879754 >> v2:172.21.15.172:3300/0 conn(0x7fd7c015e520 0x7fd7c015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.501+0000 7fd7c5600640 1 -- 172.21.15.165:0/3716879754 >> 172.21.15.165:0/3716879754 conn(0x7fd7c006abe0 msgr2=0x7fd7c006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.501+0000 7fd7c5600640 1 -- 172.21.15.165:0/3716879754 shutdown_connections 2024-04-25T10:14:32.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.501+0000 7fd7c5600640 1 -- 172.21.15.165:0/3716879754 wait complete. 2024-04-25T10:14:32.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.502+0000 7fd7c5600640 1 Processor -- start 2024-04-25T10:14:32.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.503+0000 7fd7c5600640 1 -- start start 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7c5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c0070ff0 0x7fd7c01f2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7c5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 0x7fd7c00b2860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7c5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c015f040 0x7fd7c015ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7c5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c00ae2a0 con 0x7fd7c015e520 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7c5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c006d1f0 con 0x7fd7c015f040 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7c5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c015f4d0 con 0x7fd7c0070ff0 2024-04-25T10:14:32.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 0x7fd7c00b2860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 0x7fd7c00b2860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51244/0 (socket says 172.21.15.165:51244) 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 -- 172.21.15.165:0/3752268134 learned_addr learned my addr 172.21.15.165:0/3752268134 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c0070ff0 0x7fd7c01f2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 -- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c0070ff0 msgr2=0x7fd7c01f2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c0070ff0 0x7fd7c01f2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 -- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c015f040 msgr2=0x7fd7c015ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c015f040 0x7fd7c015ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.504+0000 7fd7b7e00640 1 -- 172.21.15.165:0/3752268134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c0146760 con 0x7fd7c015e520 2024-04-25T10:14:32.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.505+0000 7fd7b7e00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 0x7fd7c00b2860 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800c390 tx=0x7fd7a800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.505+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8017070 con 0x7fd7c015e520 2024-04-25T10:14:32.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.505+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a8005ce0 con 0x7fd7c015e520 2024-04-25T10:14:32.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.505+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a80072f0 con 0x7fd7c015e520 2024-04-25T10:14:32.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.505+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c0143b10 con 0x7fd7c015e520 2024-04-25T10:14:32.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.505+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c01440d0 con 0x7fd7c015e520 2024-04-25T10:14:32.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.506+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c0073990 con 0x7fd7c015e520 2024-04-25T10:14:32.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.507+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd7a801a070 con 0x7fd7c015e520 2024-04-25T10:14:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.508+0000 7fd7bcc00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd79c06b4d0 0x7fd79c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.508+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd7a8089830 con 0x7fd7c015e520 2024-04-25T10:14:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.508+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd79c06b4d0 0x7fd79c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.509+0000 7fd7bfe00640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd79c06b4d0 0x7fd79c06d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.513+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd7a8058480 con 0x7fd7c015e520 2024-04-25T10:14:32.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.753+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7c0162080 con 0x7fd7c015e520 2024-04-25T10:14:32.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.754+0000 7fd7bcc00640 1 -- 172.21.15.165:0/3752268134 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fd7a8057d80 con 0x7fd7c015e520 2024-04-25T10:14:32.758 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:32.759 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:32.759 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:32.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd79c06b4d0 msgr2=0x7fd79c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd79c06b4d0 0x7fd79c06d990 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0006fd0 tx=0x7fd7b0008040 comp rx=0 tx=0).stop 2024-04-25T10:14:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 msgr2=0x7fd7c00b2860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 0x7fd7c00b2860 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7fd7a800c390 tx=0x7fd7a800c850 comp rx=0 tx=0).stop 2024-04-25T10:14:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 shutdown_connections 2024-04-25T10:14:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd79c06b4d0 0x7fd79c06d990 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd7c015f040 0x7fd7c015ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd7c015e520 0x7fd7c00b2860 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 --2- 172.21.15.165:0/3752268134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd7c0070ff0 0x7fd7c01f2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 >> 172.21.15.165:0/3752268134 conn(0x7fd7c006abe0 msgr2=0x7fd7c006d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.757+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 shutdown_connections 2024-04-25T10:14:32.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.758+0000 7fd7c5600640 1 -- 172.21.15.165:0/3752268134 wait complete. 2024-04-25T10:14:32.780 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:32.780 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:32.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.973+0000 7f71cc000640 1 Processor -- start 2024-04-25T10:14:32.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.973+0000 7f71cc000640 1 -- start start 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71cc000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71cc000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71c415df50 0x7f71c40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71cc000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71c40cec70 0x7f71c40cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71cc000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c40cbd80 con 0x7f71c415df50 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71cc000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71c4161910 con 0x7f71c40cec70 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71cc000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4071650 con 0x7f71c415db70 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71c8c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71c415df50 0x7f71c40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71c8c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71c415df50 0x7f71c40cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51252/0 (socket says 172.21.15.165:51252) 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.975+0000 7f71c8c00640 1 -- 172.21.15.165:0/2411054574 learned_addr learned my addr 172.21.15.165:0/2411054574 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71c9600640 1 --2- 172.21.15.165:0/2411054574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71ca000640 1 --2- 172.21.15.165:0/2411054574 >> v2:172.21.15.172:3300/0 conn(0x7f71c40cec70 0x7f71c40cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71c8c00640 1 -- 172.21.15.165:0/2411054574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 msgr2=0x7f71c40cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71c8c00640 1 --2- 172.21.15.165:0/2411054574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c40cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71c8c00640 1 -- 172.21.15.165:0/2411054574 >> v2:172.21.15.172:3300/0 conn(0x7f71c40cec70 msgr2=0x7f71c40cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71c8c00640 1 --2- 172.21.15.165:0/2411054574 >> v2:172.21.15.172:3300/0 conn(0x7f71c40cec70 0x7f71c40cfa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.976+0000 7f71c8c00640 1 -- 172.21.15.165:0/2411054574 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c40cf230 con 0x7f71c415df50 2024-04-25T10:14:32.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.977+0000 7f71c8c00640 1 --2- 172.21.15.165:0/2411054574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c415df50 0x7f71c40cf510 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f71c40af270 tx=0x7f71b002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a69ca3125102d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71c3e00640 1 -- 172.21.15.165:0/2411054574 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0037070 con 0x7f71c415df50 2024-04-25T10:14:32.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71c3e00640 1 -- 172.21.15.165:0/2411054574 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b0032440 con 0x7f71c415df50 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71c3e00640 1 -- 172.21.15.165:0/2411054574 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b0032810 con 0x7f71c415df50 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 -- 172.21.15.165:0/2411054574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c415df50 msgr2=0x7f71c40cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 --2- 172.21.15.165:0/2411054574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c415df50 0x7f71c40cf510 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f71c40af270 tx=0x7f71b002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 -- 172.21.15.165:0/2411054574 shutdown_connections 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 --2- 172.21.15.165:0/2411054574 >> v2:172.21.15.172:3300/0 conn(0x7f71c40cec70 0x7f71c40cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 --2- 172.21.15.165:0/2411054574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c415df50 0x7f71c40cf510 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 --2- 172.21.15.165:0/2411054574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c40cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.978+0000 7f71cc000640 1 -- 172.21.15.165:0/2411054574 >> 172.21.15.165:0/2411054574 conn(0x7f71c406abe0 msgr2=0x7f71c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.979+0000 7f71cc000640 1 -- 172.21.15.165:0/2411054574 shutdown_connections 2024-04-25T10:14:32.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.979+0000 7f71cc000640 1 -- 172.21.15.165:0/2411054574 wait complete. 2024-04-25T10:14:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.980+0000 7f71cc000640 1 Processor -- start 2024-04-25T10:14:32.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.980+0000 7f71cc000640 1 -- start start 2024-04-25T10:14:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71cc000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 0x7f71c4152760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71cc000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c4160530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71cc000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71c415df50 0x7f71c4159730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71cc000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4162120 con 0x7f71c40cec70 2024-04-25T10:14:32.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71cc000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4161e40 con 0x7f71c415df50 2024-04-25T10:14:32.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71cc000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4162640 con 0x7f71c415db70 2024-04-25T10:14:32.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71c9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 0x7f71c4152760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71c9600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 0x7f71c4152760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51260/0 (socket says 172.21.15.165:51260) 2024-04-25T10:14:32.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.981+0000 7f71c9600640 1 -- 172.21.15.165:0/3663323507 learned_addr learned my addr 172.21.15.165:0/3663323507 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:32.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c9600640 1 -- 172.21.15.165:0/3663323507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 msgr2=0x7f71c4160530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c9600640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c4160530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c9600640 1 -- 172.21.15.165:0/3663323507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71c415df50 msgr2=0x7f71c4159730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:32.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c9600640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71c415df50 0x7f71c4159730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:32.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c9600640 1 -- 172.21.15.165:0/3663323507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c415c5c0 con 0x7f71c40cec70 2024-04-25T10:14:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c9600640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 0x7f71c4152760 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d800 tx=0x7f71b800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.982+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800b6c0 con 0x7f71c40cec70 2024-04-25T10:14:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.983+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b8019070 con 0x7f71c40cec70 2024-04-25T10:14:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.983+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b800ba80 con 0x7f71c40cec70 2024-04-25T10:14:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.983+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c415c8a0 con 0x7f71c40cec70 2024-04-25T10:14:32.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.983+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c4160010 con 0x7f71c40cec70 2024-04-25T10:14:32.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.984+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c4073990 con 0x7f71c40cec70 2024-04-25T10:14:32.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.985+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71b800bc20 con 0x7f71c40cec70 2024-04-25T10:14:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.986+0000 7f71c1600640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71a406b4d0 0x7f71a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.986+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f71b808a7e0 con 0x7f71c40cec70 2024-04-25T10:14:32.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.987+0000 7f71c8c00640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71a406b4d0 0x7f71a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:32.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.987+0000 7f71c8c00640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71a406b4d0 0x7f71a406d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f71b0034000 tx=0x7f71b002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:32.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:32.991+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71b80594c0 con 0x7f71c40cec70 2024-04-25T10:14:33.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.258+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f71c40af040 con 0x7f71c40cec70 2024-04-25T10:14:33.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.259+0000 7f71c1600640 1 -- 172.21.15.165:0/3663323507 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1186 (secure 0 0 0) 0x7f71b8058dc0 con 0x7f71c40cec70 2024-04-25T10:14:33.263 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:33.263 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":774,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":900,"data_bytes":920776242,"bytes_used":2276810752,"bytes_avail":770817302528,"bytes_total":773094113280,"write_bytes_sec":30816442,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T10:13:34.366141+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71a406b4d0 msgr2=0x7f71a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71a406b4d0 0x7f71a406d990 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f71b0034000 tx=0x7f71b002c000 comp rx=0 tx=0).stop 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 msgr2=0x7f71c4152760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 0x7f71c4152760 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f71b800d800 tx=0x7f71b800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 shutdown_connections 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71a406b4d0 0x7f71a406d990 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71c415df50 0x7f71c4159730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71c415db70 0x7f71c4160530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 --2- 172.21.15.165:0/3663323507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71c40cec70 0x7f71c4152760 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:33.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 >> 172.21.15.165:0/3663323507 conn(0x7f71c406abe0 msgr2=0x7f71c40ac950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:33.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 shutdown_connections 2024-04-25T10:14:33.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:33.262+0000 7f71cc000640 1 -- 172.21.15.165:0/3663323507 wait complete. 2024-04-25T10:14:33.285 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:14:34.286 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:34.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.492+0000 7ffb67000640 1 Processor -- start 2024-04-25T10:14:34.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.493+0000 7ffb67000640 1 -- start start 2024-04-25T10:14:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.494+0000 7ffb67000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffb60144750 0x7ffb6006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.494+0000 7ffb67000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb60144b30 con 0x7ffb6006eb10 2024-04-25T10:14:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.494+0000 7ffb67000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb600712d0 con 0x7ffb600cf030 2024-04-25T10:14:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.494+0000 7ffb67000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb600715b0 con 0x7ffb60144750 2024-04-25T10:14:34.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb57400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ffb6006eb10 0x7ffb600ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42880/0 (socket says 172.21.15.165:42880) 2024-04-25T10:14:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb57400640 1 -- 172.21.15.165:0/296112729 learned_addr learned my addr 172.21.15.165:0/296112729 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb5fe00640 1 --2- 172.21.15.165:0/296112729 >> v2:172.21.15.172:3300/0 conn(0x7ffb60144750 0x7ffb6006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593863875 0 0) 0x7ffb60144b30 con 0x7ffb6006eb10 2024-04-25T10:14:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb44003520 con 0x7ffb6006eb10 2024-04-25T10:14:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1605819590 0 0) 0x7ffb44003520 con 0x7ffb6006eb10 2024-04-25T10:14:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 >> v2:172.21.15.172:3300/0 conn(0x7ffb60144750 msgr2=0x7ffb6006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.495+0000 7ffb5f400640 1 --2- 172.21.15.165:0/296112729 >> v2:172.21.15.172:3300/0 conn(0x7ffb60144750 0x7ffb6006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 >> v1:172.21.15.172:6789/0 conn(0x7ffb600cf030 legacy=0x7ffb6006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb60071890 con 0x7ffb6006eb10 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ffb48001670 con 0x7ffb6006eb10 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 >> v1:172.21.15.165:6789/0 conn(0x7ffb6006eb10 legacy=0x7ffb600ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb440039b0 0x7ffb44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb440051c0 0x7ffb4400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb600715b0 con 0x7ffb44004800 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4400c060 con 0x7ffb440051c0 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb4400c260 con 0x7ffb440039b0 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb57400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb57400640 1 -- 172.21.15.165:0/296112729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb440039b0 msgr2=0x7ffb44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb57400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb440039b0 0x7ffb44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb57400640 1 -- 172.21.15.165:0/296112729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb440051c0 msgr2=0x7ffb4400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb57400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb440051c0 0x7ffb4400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb57400640 1 -- 172.21.15.165:0/296112729 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb44003520 con 0x7ffb44004800 2024-04-25T10:14:34.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.496+0000 7ffb65000640 1 -- 172.21.15.165:0/296112729 reap_dead start 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.497+0000 7ffb57400640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c240 tx=0x7ffb44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42345b8d6dfce843 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48007d10 con 0x7ffb44004800 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb48005ad0 con 0x7ffb44004800 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb5f400640 1 -- 172.21.15.165:0/296112729 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb48007070 con 0x7ffb44004800 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 -- 172.21.15.165:0/296112729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 msgr2=0x7ffb44004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800c240 tx=0x7ffb44005830 comp rx=0 tx=0).stop 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 -- 172.21.15.165:0/296112729 shutdown_connections 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 --2- 172.21.15.165:0/296112729 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb44004c80 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 -- 172.21.15.165:0/296112729 >> 172.21.15.165:0/296112729 conn(0x7ffb6006abe0 msgr2=0x7ffb6006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 -- 172.21.15.165:0/296112729 shutdown_connections 2024-04-25T10:14:34.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.498+0000 7ffb67000640 1 -- 172.21.15.165:0/296112729 wait complete. 2024-04-25T10:14:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.500+0000 7ffb67000640 1 Processor -- start 2024-04-25T10:14:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.500+0000 7ffb67000640 1 -- start start 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.501+0000 7ffb67000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb6015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.501+0000 7ffb67000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6015ef30 0x7ffb6015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.501+0000 7ffb67000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb60142d70 0x7ffb601431f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.501+0000 7ffb67000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb601455d0 con 0x7ffb44004800 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.501+0000 7ffb67000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb60145450 con 0x7ffb6015ef30 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.501+0000 7ffb67000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb6015c000 con 0x7ffb60142d70 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb6015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb6015d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51276/0 (socket says 172.21.15.165:51276) 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 -- 172.21.15.165:0/3057202932 learned_addr learned my addr 172.21.15.165:0/3057202932 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 -- 172.21.15.165:0/3057202932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb60142d70 msgr2=0x7ffb601431f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb60142d70 0x7ffb601431f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 -- 172.21.15.165:0/3057202932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6015ef30 msgr2=0x7ffb6015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6015ef30 0x7ffb6015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 -- 172.21.15.165:0/3057202932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb6015fa80 con 0x7ffb44004800 2024-04-25T10:14:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.502+0000 7ffb5fe00640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb6015d520 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000e800 tx=0x7ffb5000ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.503+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50017070 con 0x7ffb44004800 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.503+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5000ee50 con 0x7ffb44004800 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.503+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5000cb00 con 0x7ffb44004800 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.503+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb6015fd60 con 0x7ffb44004800 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.503+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb60160320 con 0x7ffb44004800 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.504+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb60073990 con 0x7ffb44004800 2024-04-25T10:14:34.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.505+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb5001a070 con 0x7ffb44004800 2024-04-25T10:14:34.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.506+0000 7ffb5cc00640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3c06b4d0 0x7ffb3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.506+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffb5008a590 con 0x7ffb44004800 2024-04-25T10:14:34.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.506+0000 7ffb57400640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3c06b4d0 0x7ffb3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.507+0000 7ffb57400640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3c06b4d0 0x7ffb3c06d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffb48033000 tx=0x7ffb48009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.511+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ffb500591e0 con 0x7ffb44004800 2024-04-25T10:14:34.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.744+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffb6006b490 con 0x7ffb44004800 2024-04-25T10:14:34.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.745+0000 7ffb5cc00640 1 -- 172.21.15.165:0/3057202932 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7ffb50058ae0 con 0x7ffb44004800 2024-04-25T10:14:34.749 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:34.749 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:34.749 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.747+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3c06b4d0 msgr2=0x7ffb3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.747+0000 7ffb67000640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3c06b4d0 0x7ffb3c06d990 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7ffb48033000 tx=0x7ffb48009030 comp rx=0 tx=0).stop 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.747+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 msgr2=0x7ffb6015d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.747+0000 7ffb67000640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb6015d520 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000e800 tx=0x7ffb5000ec00 comp rx=0 tx=0).stop 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 shutdown_connections 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb3c06b4d0 0x7ffb3c06d990 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb60142d70 0x7ffb601431f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb6015ef30 0x7ffb6015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 --2- 172.21.15.165:0/3057202932 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb44004800 0x7ffb6015d520 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 >> 172.21.15.165:0/3057202932 conn(0x7ffb6006abe0 msgr2=0x7ffb600b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 shutdown_connections 2024-04-25T10:14:34.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.748+0000 7ffb67000640 1 -- 172.21.15.165:0/3057202932 wait complete. 2024-04-25T10:14:34.770 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:34.770 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:34.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.959+0000 7f81da200640 1 Processor -- start 2024-04-25T10:14:34.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.960+0000 7f81da200640 1 -- start start 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.961+0000 7f81da200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81d415db70 0x7f81d40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.961+0000 7f81da200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 0x7f81d406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.961+0000 7f81da200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d4071650 con 0x7f81d415df50 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.961+0000 7f81da200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d40cbd80 con 0x7f81d415db70 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.961+0000 7f81da200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d4161910 con 0x7f81d40cfc20 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.962+0000 7f81d3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 0x7f81d406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.962+0000 7f81d3400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f81d415db70 0x7f81d40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.962+0000 7f81d3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 0x7f81d406d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:52078/0 (socket says 172.21.15.165:52078) 2024-04-25T10:14:34.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.962+0000 7f81d3e00640 1 -- 172.21.15.165:0/1329903362 learned_addr learned my addr 172.21.15.165:0/1329903362 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:34.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.963+0000 7f81d3e00640 1 -- 172.21.15.165:0/1329903362 >> v2:172.21.15.172:3300/0 conn(0x7f81d415db70 msgr2=0x7f81d40cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:34.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.963+0000 7f81d3e00640 1 --2- 172.21.15.165:0/1329903362 >> v2:172.21.15.172:3300/0 conn(0x7f81d415db70 0x7f81d40cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.963+0000 7f81d3e00640 1 -- 172.21.15.165:0/1329903362 >> v1:172.21.15.172:6789/0 conn(0x7f81d415df50 legacy=0x7f81d40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.963+0000 7f81d3e00640 1 -- 172.21.15.165:0/1329903362 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d40cf230 con 0x7f81d40cfc20 2024-04-25T10:14:34.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.964+0000 7f81d3e00640 1 --2- 172.21.15.165:0/1329903362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 0x7f81d406d290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009e50 tx=0x7f81cc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e57c3299e3ceea66 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.965+0000 7f81d2a00640 1 -- 172.21.15.165:0/1329903362 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc036070 con 0x7f81d40cfc20 2024-04-25T10:14:34.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.965+0000 7f81d2a00640 1 -- 172.21.15.165:0/1329903362 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc031600 con 0x7f81d40cfc20 2024-04-25T10:14:34.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.965+0000 7f81d2a00640 1 -- 172.21.15.165:0/1329903362 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc0319d0 con 0x7f81d40cfc20 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 -- 172.21.15.165:0/1329903362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 msgr2=0x7f81d406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 --2- 172.21.15.165:0/1329903362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 0x7f81d406d290 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81cc009e50 tx=0x7f81cc02a680 comp rx=0 tx=0).stop 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 -- 172.21.15.165:0/1329903362 shutdown_connections 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 --2- 172.21.15.165:0/1329903362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d40cfc20 0x7f81d406d290 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 --2- 172.21.15.165:0/1329903362 >> v2:172.21.15.172:3300/0 conn(0x7f81d415db70 0x7f81d40cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 -- 172.21.15.165:0/1329903362 >> 172.21.15.165:0/1329903362 conn(0x7f81d406abe0 msgr2=0x7f81d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:34.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 -- 172.21.15.165:0/1329903362 shutdown_connections 2024-04-25T10:14:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.966+0000 7f81da200640 1 -- 172.21.15.165:0/1329903362 wait complete. 2024-04-25T10:14:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.968+0000 7f81da200640 1 Processor -- start 2024-04-25T10:14:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.968+0000 7f81da200640 1 -- start start 2024-04-25T10:14:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81da200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 0x7f81d41490a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81da200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d415db70 0x7f81d40b39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81da200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81d415df50 0x7f81d40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81da200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d4162120 con 0x7f81d40cfc20 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81da200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d4161e40 con 0x7f81d415df50 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81da200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d4162640 con 0x7f81d415db70 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81d3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 0x7f81d41490a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81d3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 0x7f81d41490a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51292/0 (socket says 172.21.15.165:51292) 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.969+0000 7f81d3400640 1 -- 172.21.15.165:0/3337707777 learned_addr learned my addr 172.21.15.165:0/3337707777 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3e00640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81d415df50 0x7f81d40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81cbe00640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d415db70 0x7f81d40b39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3400640 1 -- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d415db70 msgr2=0x7f81d40b39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3400640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d415db70 0x7f81d40b39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3400640 1 -- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81d415df50 msgr2=0x7f81d40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3400640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81d415df50 0x7f81d40b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3400640 1 -- 172.21.15.165:0/3337707777 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d41495e0 con 0x7f81d40cfc20 2024-04-25T10:14:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3e00640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81d415df50 0x7f81d40b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81cbe00640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d415db70 0x7f81d40b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81d3400640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 0x7f81d41490a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e9b0 tx=0x7f81c400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c4017070 con 0x7f81d40cfc20 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c400c9a0 con 0x7f81d40cfc20 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.970+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c400cd90 con 0x7f81d40cfc20 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.971+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d41f7190 con 0x7f81d40cfc20 2024-04-25T10:14:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.971+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d41f7750 con 0x7f81d40cfc20 2024-04-25T10:14:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.972+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d4073990 con 0x7f81d40cfc20 2024-04-25T10:14:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.973+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81c401a070 con 0x7f81d40cfc20 2024-04-25T10:14:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.974+0000 7f81cb400640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81b006b4d0 0x7f81b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.974+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81c408a810 con 0x7f81d40cfc20 2024-04-25T10:14:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.974+0000 7f81cbe00640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81b006b4d0 0x7f81b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.975+0000 7f81cbe00640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81b006b4d0 0x7f81b006d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f81d41f6d10 tx=0x7f81bc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:34.978+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81c4059460 con 0x7f81d40cfc20 2024-04-25T10:14:35.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.237+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f81d41f7af0 con 0x7f81d40cfc20 2024-04-25T10:14:35.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.238+0000 7f81cb400640 1 -- 172.21.15.165:0/3337707777 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f81c4058d60 con 0x7f81d40cfc20 2024-04-25T10:14:35.242 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:35.242 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":776,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076362760,"bytes_used":2615246848,"bytes_avail":770478866432,"bytes_total":773094113280,"write_bytes_sec":43781449,"read_op_per_sec":0,"write_op_per_sec":45},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T10:13:34.366141+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:14:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81b006b4d0 msgr2=0x7f81b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81b006b4d0 0x7f81b006d990 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f81d41f6d10 tx=0x7f81bc009040 comp rx=0 tx=0).stop 2024-04-25T10:14:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 msgr2=0x7f81d41490a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:35.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 0x7f81d41490a0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f81c400e9b0 tx=0x7f81c400ee70 comp rx=0 tx=0).stop 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 shutdown_connections 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81b006b4d0 0x7f81b006d990 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81d415df50 0x7f81d40b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81d415db70 0x7f81d40b39e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 --2- 172.21.15.165:0/3337707777 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81d40cfc20 0x7f81d41490a0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 >> 172.21.15.165:0/3337707777 conn(0x7f81d406abe0 msgr2=0x7f81d4158d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.240+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 shutdown_connections 2024-04-25T10:14:35.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:35.241+0000 7f81da200640 1 -- 172.21.15.165:0/3337707777 wait complete. 2024-04-25T10:14:35.261 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:14:36.263 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:36.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.468+0000 7f4ea1c00640 1 Processor -- start 2024-04-25T10:14:36.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.468+0000 7f4ea1c00640 1 -- start start 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.470+0000 7f4ea1c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4e9c0d0ad0 0x7f4e9c168760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.470+0000 7f4ea1c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0b9940 con 0x7f4e9c0d2e90 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.470+0000 7f4ea1c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e9c0d1df0 con 0x7f4e9c15dd30 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.470+0000 7f4ea1c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e9c0b7a50 con 0x7f4e9c0d0ad0 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.470+0000 7f4e9a000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4e9c0d2e90 0x7f4e9c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42894/0 (socket says 172.21.15.165:42894) 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.470+0000 7f4e9a000640 1 -- 172.21.15.165:0/841819759 learned_addr learned my addr 172.21.15.165:0/841819759 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:36.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e9b400640 1 --2- 172.21.15.165:0/841819759 >> v2:172.21.15.172:3300/0 conn(0x7f4e9c0d0ad0 0x7f4e9c168760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1395972890 0 0) 0x7f4e9c0b9940 con 0x7f4e9c0d2e90 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e7c003520 con 0x7f4e9c0d2e90 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3850490190 0 0) 0x7f4e7c003520 con 0x7f4e9c0d2e90 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 >> v2:172.21.15.172:3300/0 conn(0x7f4e9c0d0ad0 msgr2=0x7f4e9c168760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 --2- 172.21.15.165:0/841819759 >> v2:172.21.15.172:3300/0 conn(0x7f4e9c0d0ad0 0x7f4e9c168760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 >> v1:172.21.15.172:6789/0 conn(0x7f4e9c15dd30 legacy=0x7f4e9c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c0d20d0 con 0x7f4e9c0d2e90 2024-04-25T10:14:36.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4e84001b70 con 0x7f4e9c0d2e90 2024-04-25T10:14:36.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 >> v1:172.21.15.165:6789/0 conn(0x7f4e9c0d2e90 legacy=0x7f4e9c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e7c0039b0 0x7f4e7c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c004800 0x7f4e7c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e7c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0b7a50 con 0x7f4e7c0052e0 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c00c1b0 con 0x7f4e7c004800 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.471+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e7c00c3b0 con 0x7f4e7c0039b0 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9b400640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e7c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9a000640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c004800 0x7f4e7c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9aa00640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e7c0039b0 0x7f4e7c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9b400640 1 -- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e7c0039b0 msgr2=0x7f4e7c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9b400640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e7c0039b0 0x7f4e7c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9b400640 1 -- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c004800 msgr2=0x7f4e7c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9b400640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e7c004800 0x7f4e7c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.472+0000 7f4e9b400640 1 -- 172.21.15.165:0/841819759 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e7c003520 con 0x7f4e7c0052e0 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.473+0000 7f4e9b400640 1 -- 172.21.15.165:0/841819759 reap_dead start 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.473+0000 7f4e9b400640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e7c00b9c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009a90 tx=0x7f4e7c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e4efc2a0de0bfa8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.473+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94009690 con 0x7f4e7c0052e0 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.473+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e94004020 con 0x7f4e7c0052e0 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.473+0000 7f4e99600640 1 -- 172.21.15.165:0/841819759 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e94004410 con 0x7f4e7c0052e0 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 -- 172.21.15.165:0/841819759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 msgr2=0x7f4e7c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e7c00b9c0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f4e94009a90 tx=0x7f4e7c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 -- 172.21.15.165:0/841819759 shutdown_connections 2024-04-25T10:14:36.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/841819759 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e7c00b9c0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 -- 172.21.15.165:0/841819759 >> 172.21.15.165:0/841819759 conn(0x7f4e9c061c30 msgr2=0x7f4e9c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 -- 172.21.15.165:0/841819759 shutdown_connections 2024-04-25T10:14:36.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.474+0000 7f4ea1c00640 1 -- 172.21.15.165:0/841819759 wait complete. 2024-04-25T10:14:36.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.476+0000 7f4ea1c00640 1 Processor -- start 2024-04-25T10:14:36.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.476+0000 7f4ea1c00640 1 -- start start 2024-04-25T10:14:36.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4ea1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e9c152fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4ea1c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e9c160170 0x7f4e9c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4ea1c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e9c15b9e0 0x7f4e9c15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4ea1c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0d2970 con 0x7f4e7c0052e0 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4ea1c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0d2690 con 0x7f4e9c15b9e0 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4ea1c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e9c0d1a10 con 0x7f4e9c160170 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4e9aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e9c152fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4e9aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e9c152fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51300/0 (socket says 172.21.15.165:51300) 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.477+0000 7f4e9aa00640 1 -- 172.21.15.165:0/3068522898 learned_addr learned my addr 172.21.15.165:0/3068522898 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9a000640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e9c160170 0x7f4e9c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9b400640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e9c15b9e0 0x7f4e9c15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9aa00640 1 -- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e9c160170 msgr2=0x7f4e9c158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9aa00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e9c160170 0x7f4e9c158f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9aa00640 1 -- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e9c15b9e0 msgr2=0x7f4e9c15be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9aa00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e9c15b9e0 0x7f4e9c15be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9aa00640 1 -- 172.21.15.165:0/3068522898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e9c157eb0 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9b400640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e9c15b9e0 0x7f4e9c15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9a000640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e9c160170 0x7f4e9c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.478+0000 7f4e9aa00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e9c152fd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00e480 tx=0x7f4e8c00e940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.479+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c0052e0 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.479+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e8c018070 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.479+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e8c003830 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.479+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e9c158190 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.479+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e9c158750 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.480+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e9c073990 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.481+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4e8c0039d0 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.482+0000 7f4e92a00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e7806b500 0x7f4e7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.482+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4e8c08a0b0 con 0x7f4e7c0052e0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.482+0000 7f4e9a000640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e7806b500 0x7f4e7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.483+0000 7f4e9a000640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e7806b500 0x7f4e7806d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4e84001ce0 tx=0x7f4e84009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.486+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4e8c058d90 con 0x7f4e7c0052e0 2024-04-25T10:14:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.713+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e9c0c82f0 con 0x7f4e7c0052e0 2024-04-25T10:14:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.714+0000 7f4e92a00640 1 -- 172.21.15.165:0/3068522898 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f4e8c058690 con 0x7f4e7c0052e0 2024-04-25T10:14:36.718 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:36.718 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e7806b500 msgr2=0x7f4e7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e7806b500 0x7f4e7806d9c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4e84001ce0 tx=0x7f4e84009040 comp rx=0 tx=0).stop 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 msgr2=0x7f4e9c152fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e9c152fd0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f4e8c00e480 tx=0x7f4e8c00e940 comp rx=0 tx=0).stop 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 shutdown_connections 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4e7806b500 0x7f4e7806d9c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4e9c15b9e0 0x7f4e9c15be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4e9c160170 0x7f4e9c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 --2- 172.21.15.165:0/3068522898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4e7c0052e0 0x7f4e9c152fd0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 >> 172.21.15.165:0/3068522898 conn(0x7f4e9c061c30 msgr2=0x7f4e9c0c9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 shutdown_connections 2024-04-25T10:14:36.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.717+0000 7f4ea1c00640 1 -- 172.21.15.165:0/3068522898 wait complete. 2024-04-25T10:14:36.740 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:36.740 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.933+0000 7f3faea00640 1 Processor -- start 2024-04-25T10:14:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.933+0000 7f3faea00640 1 -- start start 2024-04-25T10:14:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3fa8154ee0 0x7f3fa80cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3fa80ce590 0x7f3fa806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fa8071480 con 0x7f3fa80cfcf0 2024-04-25T10:14:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faea00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa8072090 con 0x7f3fa80ce590 2024-04-25T10:14:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fa8071230 con 0x7f3fa8154ee0 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3fa80ce590 0x7f3fa806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3fa80ce590 0x7f3fa806d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37662/0 (socket says 172.21.15.165:37662) 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.935+0000 7f3faca00640 1 -- 172.21.15.165:0/2616898558 learned_addr learned my addr 172.21.15.165:0/2616898558 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.936+0000 7f3fa7e00640 1 --2- 172.21.15.165:0/2616898558 >> v2:172.21.15.172:3300/0 conn(0x7f3fa8154ee0 0x7f3fa80cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.936+0000 7f3faca00640 1 -- 172.21.15.165:0/2616898558 >> v2:172.21.15.172:3300/0 conn(0x7f3fa8154ee0 msgr2=0x7f3fa80cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.936+0000 7f3faca00640 1 --2- 172.21.15.165:0/2616898558 >> v2:172.21.15.172:3300/0 conn(0x7f3fa8154ee0 0x7f3fa80cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.936+0000 7f3faca00640 1 -- 172.21.15.165:0/2616898558 >> v1:172.21.15.172:6789/0 conn(0x7f3fa80cfcf0 legacy=0x7f3fa80cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.936+0000 7f3faca00640 1 -- 172.21.15.165:0/2616898558 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8071760 con 0x7f3fa80ce590 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.937+0000 7f3faca00640 1 --2- 172.21.15.165:0/2616898558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa806d6f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80d02e0 tx=0x7f3fa002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99197a104b0f942 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.937+0000 7f3fa6a00640 1 -- 172.21.15.165:0/2616898558 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa0036070 con 0x7f3fa80ce590 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.937+0000 7f3fa6a00640 1 -- 172.21.15.165:0/2616898558 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fa0031600 con 0x7f3fa80ce590 2024-04-25T10:14:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.937+0000 7f3fa6a00640 1 -- 172.21.15.165:0/2616898558 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00319d0 con 0x7f3fa80ce590 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 -- 172.21.15.165:0/2616898558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 msgr2=0x7f3fa806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 --2- 172.21.15.165:0/2616898558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa806d6f0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80d02e0 tx=0x7f3fa002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 -- 172.21.15.165:0/2616898558 shutdown_connections 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 --2- 172.21.15.165:0/2616898558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa806d6f0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 --2- 172.21.15.165:0/2616898558 >> v2:172.21.15.172:3300/0 conn(0x7f3fa8154ee0 0x7f3fa80cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 -- 172.21.15.165:0/2616898558 >> 172.21.15.165:0/2616898558 conn(0x7f3fa806abe0 msgr2=0x7f3fa806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 -- 172.21.15.165:0/2616898558 shutdown_connections 2024-04-25T10:14:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.938+0000 7f3faea00640 1 -- 172.21.15.165:0/2616898558 wait complete. 2024-04-25T10:14:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.940+0000 7f3faea00640 1 Processor -- start 2024-04-25T10:14:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.940+0000 7f3faea00640 1 -- start start 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.941+0000 7f3faea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.941+0000 7f3faea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fa80cfcf0 0x7f3fa80b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.941+0000 7f3faea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fa8154ee0 0x7f3fa80b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.941+0000 7f3faea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa81552c0 con 0x7f3fa80ce590 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.941+0000 7f3faea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8071d20 con 0x7f3fa80cfcf0 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.941+0000 7f3faea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa8155440 con 0x7f3fa8154ee0 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa815a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37664/0 (socket says 172.21.15.165:37664) 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 -- 172.21.15.165:0/3297838915 learned_addr learned my addr 172.21.15.165:0/3297838915 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7400640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fa80cfcf0 0x7f3fa80b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3faca00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fa8154ee0 0x7f3fa80b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 -- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fa8154ee0 msgr2=0x7f3fa80b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fa8154ee0 0x7f3fa80b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 -- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fa80cfcf0 msgr2=0x7f3fa80b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fa80cfcf0 0x7f3fa80b3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.942+0000 7f3fa7e00640 1 -- 172.21.15.165:0/3297838915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa8158c40 con 0x7f3fa80ce590 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.943+0000 7f3fa7e00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa815a4d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800d5f0 tx=0x7f3f9800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.943+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980125f0 con 0x7f3fa80ce590 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.943+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f98019070 con 0x7f3fa80ce590 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.943+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980129d0 con 0x7f3fa80ce590 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.943+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa8158f20 con 0x7f3fa80ce590 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.943+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa81594e0 con 0x7f3fa80ce590 2024-04-25T10:14:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.944+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa8073990 con 0x7f3fa80ce590 2024-04-25T10:14:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.945+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f98012b70 con 0x7f3fa80ce590 2024-04-25T10:14:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.946+0000 7f3f9fe00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f8806b4d0 0x7f3f8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.946+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f9808a7f0 con 0x7f3fa80ce590 2024-04-25T10:14:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.946+0000 7f3fa7400640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f8806b4d0 0x7f3f8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.947+0000 7f3fa7400640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f8806b4d0 0x7f3f8806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80ae6d0 tx=0x7f3f94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:36.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:36.952+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f980594d0 con 0x7f3fa80ce590 2024-04-25T10:14:37.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.197+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3fa8156720 con 0x7f3fa80ce590 2024-04-25T10:14:37.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.198+0000 7f3f9fe00640 1 -- 172.21.15.165:0/3297838915 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f3f98058dd0 con 0x7f3fa80ce590 2024-04-25T10:14:37.202 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:37.202 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":778,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076362760,"bytes_used":2615246848,"bytes_avail":770478866432,"bytes_total":773094113280,"write_bytes_sec":28196605,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":29,"modified":"2024-04-25T10:13:34.366141+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:14:37.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.200+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f8806b4d0 msgr2=0x7f3f8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.200+0000 7f3faea00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f8806b4d0 0x7f3f8806d990 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3fa80ae6d0 tx=0x7f3f94009040 comp rx=0 tx=0).stop 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.200+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 msgr2=0x7f3fa815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.200+0000 7f3faea00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa815a4d0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f3f9800d5f0 tx=0x7f3f9800dab0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.200+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 shutdown_connections 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.200+0000 7f3faea00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f8806b4d0 0x7f3f8806d990 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.201+0000 7f3faea00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fa8154ee0 0x7f3fa80b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.201+0000 7f3faea00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fa80cfcf0 0x7f3fa80b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.201+0000 7f3faea00640 1 --2- 172.21.15.165:0/3297838915 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fa80ce590 0x7f3fa815a4d0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.201+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 >> 172.21.15.165:0/3297838915 conn(0x7f3fa806abe0 msgr2=0x7f3fa806e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.201+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 shutdown_connections 2024-04-25T10:14:37.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:37.201+0000 7f3faea00640 1 -- 172.21.15.165:0/3297838915 wait complete. 2024-04-25T10:14:37.223 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:14:38.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:38.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.431+0000 7fda32e00640 1 Processor -- start 2024-04-25T10:14:38.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.431+0000 7fda32e00640 1 -- start start 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda32e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda2c164970 0x7fda2c166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda32e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c144b30 con 0x7fda2c0ce920 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda32e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda2c0712d0 con 0x7fda2c144750 2024-04-25T10:14:38.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda32e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda2c0715b0 con 0x7fda2c164970 2024-04-25T10:14:38.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda30e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda2c164970 0x7fda2c166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda30e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda2c164970 0x7fda2c166d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37678/0 (socket says 172.21.15.165:37678) 2024-04-25T10:14:38.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda30e00640 1 -- 172.21.15.165:0/1700773187 learned_addr learned my addr 172.21.15.165:0/1700773187 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda23400640 1 --1- 172.21.15.165:0/1700773187 >> v1:172.21.15.165:6789/0 conn(0x7fda2c0ce920 0x7fda2c0cedb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47250/0 (socket says 172.21.15.165:47250) 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda30e00640 1 -- 172.21.15.165:0/1700773187 >> v1:172.21.15.172:6789/0 conn(0x7fda2c144750 legacy=0x7fda2c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda30e00640 1 -- 172.21.15.165:0/1700773187 >> v1:172.21.15.165:6789/0 conn(0x7fda2c0ce920 legacy=0x7fda2c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.434+0000 7fda30e00640 1 -- 172.21.15.165:0/1700773187 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c071890 con 0x7fda2c164970 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.433+0000 7fda2b400640 1 -- 172.21.15.165:0/1700773187 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359116210 0 0) 0x7fda2c144b30 con 0x7fda2c0ce920 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.434+0000 7fda30e00640 1 --2- 172.21.15.165:0/1700773187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c164970 0x7fda2c166d60 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fda24009e50 tx=0x7fda2402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23304d8d63a63926 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.435+0000 7fda2b400640 1 -- 172.21.15.165:0/1700773187 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda24036070 con 0x7fda2c164970 2024-04-25T10:14:38.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.435+0000 7fda2b400640 1 -- 172.21.15.165:0/1700773187 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda24031600 con 0x7fda2c164970 2024-04-25T10:14:38.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.435+0000 7fda2b400640 1 -- 172.21.15.165:0/1700773187 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda240319d0 con 0x7fda2c164970 2024-04-25T10:14:38.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.435+0000 7fda32e00640 1 -- 172.21.15.165:0/1700773187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c164970 msgr2=0x7fda2c166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.435+0000 7fda32e00640 1 --2- 172.21.15.165:0/1700773187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c164970 0x7fda2c166d60 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fda24009e50 tx=0x7fda2402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.436+0000 7fda32e00640 1 -- 172.21.15.165:0/1700773187 shutdown_connections 2024-04-25T10:14:38.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.436+0000 7fda32e00640 1 --2- 172.21.15.165:0/1700773187 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c164970 0x7fda2c166d60 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.436+0000 7fda32e00640 1 -- 172.21.15.165:0/1700773187 >> 172.21.15.165:0/1700773187 conn(0x7fda2c06abe0 msgr2=0x7fda2c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.436+0000 7fda32e00640 1 -- 172.21.15.165:0/1700773187 shutdown_connections 2024-04-25T10:14:38.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.436+0000 7fda32e00640 1 -- 172.21.15.165:0/1700773187 wait complete. 2024-04-25T10:14:38.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.437+0000 7fda32e00640 1 Processor -- start 2024-04-25T10:14:38.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.437+0000 7fda32e00640 1 -- start start 2024-04-25T10:14:38.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.438+0000 7fda32e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2c0ce920 0x7fda2c15a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.438+0000 7fda32e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 0x7fda2c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.438+0000 7fda32e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2c164970 0x7fda2c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.438+0000 7fda32e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c145450 con 0x7fda2c144750 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.438+0000 7fda32e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c06f020 con 0x7fda2c164970 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.438+0000 7fda32e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2c1455d0 con 0x7fda2c0ce920 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 0x7fda2c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 0x7fda2c15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37682/0 (socket says 172.21.15.165:37682) 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 -- 172.21.15.165:0/784548768 learned_addr learned my addr 172.21.15.165:0/784548768 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 -- 172.21.15.165:0/784548768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2c0ce920 msgr2=0x7fda2c15a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2c0ce920 0x7fda2c15a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 -- 172.21.15.165:0/784548768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2c164970 msgr2=0x7fda2c15b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2c164970 0x7fda2c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 -- 172.21.15.165:0/784548768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c15ad00 con 0x7fda2c144750 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.439+0000 7fda23400640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 0x7fda2c15af10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c460 tx=0x7fda1400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.440+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda140074e0 con 0x7fda2c144750 2024-04-25T10:14:38.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.440+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda140047e0 con 0x7fda2c144750 2024-04-25T10:14:38.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.440+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda14004bd0 con 0x7fda2c144750 2024-04-25T10:14:38.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.440+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2c15dec0 con 0x7fda2c144750 2024-04-25T10:14:38.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.440+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2c15e480 con 0x7fda2c144750 2024-04-25T10:14:38.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.441+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda2c073990 con 0x7fda2c144750 2024-04-25T10:14:38.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.442+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda14011070 con 0x7fda2c144750 2024-04-25T10:14:38.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.443+0000 7fda28c00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0806b4d0 0x7fda0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.443+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda14089f50 con 0x7fda2c144750 2024-04-25T10:14:38.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.443+0000 7fda2be00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0806b4d0 0x7fda0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.444+0000 7fda2be00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0806b4d0 0x7fda0806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0cc780 tx=0x7fda1c0079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.447+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda14058c40 con 0x7fda2c144750 2024-04-25T10:14:38.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.665+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda2c0cee50 con 0x7fda2c144750 2024-04-25T10:14:38.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.666+0000 7fda28c00640 1 -- 172.21.15.165:0/784548768 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fda14058540 con 0x7fda2c144750 2024-04-25T10:14:38.670 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:38.670 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:38.671 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0806b4d0 msgr2=0x7fda0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0806b4d0 0x7fda0806d990 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fda2c0cc780 tx=0x7fda1c0079b0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 msgr2=0x7fda2c15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 0x7fda2c15af10 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fda1400c460 tx=0x7fda1400c920 comp rx=0 tx=0).stop 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 shutdown_connections 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0806b4d0 0x7fda0806d990 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda2c164970 0x7fda2c15b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2c144750 0x7fda2c15af10 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 --2- 172.21.15.165:0/784548768 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2c0ce920 0x7fda2c15a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 >> 172.21.15.165:0/784548768 conn(0x7fda2c06abe0 msgr2=0x7fda2c140b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 shutdown_connections 2024-04-25T10:14:38.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.669+0000 7fda32e00640 1 -- 172.21.15.165:0/784548768 wait complete. 2024-04-25T10:14:38.691 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:38.691 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:38.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.882+0000 7f9abb200640 1 Processor -- start 2024-04-25T10:14:38.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.882+0000 7f9abb200640 1 -- start start 2024-04-25T10:14:38.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9abb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab4165680 0x7f9ab40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9abb200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9ab40cec70 0x7f9ab40b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9abb200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ab40cc6c0 0x7f9ab40ccaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9abb200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab4072340 con 0x7f9ab40cec70 2024-04-25T10:14:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9abb200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab40b2b90 con 0x7f9ab40cc6c0 2024-04-25T10:14:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9abb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4071670 con 0x7f9ab4165680 2024-04-25T10:14:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9aab400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9ab40cec70 0x7f9ab40b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.884+0000 7f9aab400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9ab40cec70 0x7f9ab40b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37690/0 (socket says 172.21.15.165:37690) 2024-04-25T10:14:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9aab400640 1 -- 172.21.15.165:0/638348169 learned_addr learned my addr 172.21.15.165:0/638348169 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9ab3e00640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab4165680 0x7f9ab40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9ab9200640 1 --2- 172.21.15.165:0/638348169 >> v2:172.21.15.172:3300/0 conn(0x7f9ab40cc6c0 0x7f9ab40ccaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9aab400640 1 -- 172.21.15.165:0/638348169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab4165680 msgr2=0x7f9ab40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9aab400640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab4165680 0x7f9ab40aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9aab400640 1 -- 172.21.15.165:0/638348169 >> v2:172.21.15.172:3300/0 conn(0x7f9ab40cc6c0 msgr2=0x7f9ab40ccaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9aab400640 1 --2- 172.21.15.165:0/638348169 >> v2:172.21.15.172:3300/0 conn(0x7f9ab40cc6c0 0x7f9ab40ccaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9aab400640 1 -- 172.21.15.165:0/638348169 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab40cf330 con 0x7f9ab40cec70 2024-04-25T10:14:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9ab9200640 1 --2- 172.21.15.165:0/638348169 >> v2:172.21.15.172:3300/0 conn(0x7f9ab40cc6c0 0x7f9ab40ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.885+0000 7f9ab3e00640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab4165680 0x7f9ab40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.886+0000 7f9aab400640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab40cec70 0x7f9ab40b2650 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40718c0 tx=0x7f9a9c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53a3e7310cea382 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.886+0000 7f9ab3400640 1 -- 172.21.15.165:0/638348169 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c037070 con 0x7f9ab40cec70 2024-04-25T10:14:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.886+0000 7f9ab3400640 1 -- 172.21.15.165:0/638348169 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c032440 con 0x7f9ab40cec70 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.886+0000 7f9ab3400640 1 -- 172.21.15.165:0/638348169 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c032810 con 0x7f9ab40cec70 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 -- 172.21.15.165:0/638348169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab40cec70 msgr2=0x7f9ab40b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab40cec70 0x7f9ab40b2650 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9ab40718c0 tx=0x7f9a9c02a140 comp rx=0 tx=0).stop 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 -- 172.21.15.165:0/638348169 shutdown_connections 2024-04-25T10:14:38.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 --2- 172.21.15.165:0/638348169 >> v2:172.21.15.172:3300/0 conn(0x7f9ab40cc6c0 0x7f9ab40ccaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab40cec70 0x7f9ab40b2650 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 --2- 172.21.15.165:0/638348169 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab4165680 0x7f9ab40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 -- 172.21.15.165:0/638348169 >> 172.21.15.165:0/638348169 conn(0x7f9ab406abe0 msgr2=0x7f9ab406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:38.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 -- 172.21.15.165:0/638348169 shutdown_connections 2024-04-25T10:14:38.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.887+0000 7f9abb200640 1 -- 172.21.15.165:0/638348169 wait complete. 2024-04-25T10:14:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.889+0000 7f9abb200640 1 Processor -- start 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.889+0000 7f9abb200640 1 -- start start 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9abb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ab40cc6c0 0x7f9ab414f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9abb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab40cec70 0x7f9ab414b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9abb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 0x7f9ab414bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9abb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab4072060 con 0x7f9ab4165680 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9abb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab40cf0d0 con 0x7f9ab40cc6c0 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9abb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab406ea30 con 0x7f9ab40cec70 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9ab9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 0x7f9ab414bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9ab9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 0x7f9ab414bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37692/0 (socket says 172.21.15.165:37692) 2024-04-25T10:14:38.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9ab9200640 1 -- 172.21.15.165:0/856081208 learned_addr learned my addr 172.21.15.165:0/856081208 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:38.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.890+0000 7f9ab3e00640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ab40cc6c0 0x7f9ab414f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9aab400640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab40cec70 0x7f9ab414b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab9200640 1 -- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab40cec70 msgr2=0x7f9ab414b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab9200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab40cec70 0x7f9ab414b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab9200640 1 -- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ab40cc6c0 msgr2=0x7f9ab414f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:38.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab9200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ab40cc6c0 0x7f9ab414f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:38.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab9200640 1 -- 172.21.15.165:0/856081208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab414a610 con 0x7f9ab4165680 2024-04-25T10:14:38.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab9200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 0x7f9ab414bc40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00cf20 tx=0x7f9aac00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac01a070 con 0x7f9ab4165680 2024-04-25T10:14:38.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac00d840 con 0x7f9ab4165680 2024-04-25T10:14:38.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab414a8f0 con 0x7f9ab4165680 2024-04-25T10:14:38.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.891+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac00dc30 con 0x7f9ab4165680 2024-04-25T10:14:38.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.892+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab414aeb0 con 0x7f9ab4165680 2024-04-25T10:14:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.893+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab4073990 con 0x7f9ab4165680 2024-04-25T10:14:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.894+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9aac01d070 con 0x7f9ab4165680 2024-04-25T10:14:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.894+0000 7f9ab0c00640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.895+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9aac08acc0 con 0x7f9ab4165680 2024-04-25T10:14:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.895+0000 7f9ab3e00640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:38.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.896+0000 7f9ab3e00640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a9006b4d0 0x7f9a9006d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4161620 tx=0x7f9aa4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:38.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:38.900+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9aac091080 con 0x7f9ab4165680 2024-04-25T10:14:39.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.158+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9ab40aedd0 con 0x7f9ab4165680 2024-04-25T10:14:39.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.159+0000 7f9ab0c00640 1 -- 172.21.15.165:0/856081208 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f9aac059910 con 0x7f9ab4165680 2024-04-25T10:14:39.163 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:39.163 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":780,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076362760,"bytes_used":2615246848,"bytes_avail":770478866432,"bytes_total":773094113280,"write_bytes_sec":28196481,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:14:36.387123+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:14:39.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a9006b4d0 msgr2=0x7f9a9006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a9006b4d0 0x7f9a9006d990 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4161620 tx=0x7f9aa4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:39.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 msgr2=0x7f9ab414bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 0x7f9ab414bc40 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00cf20 tx=0x7f9aac00a710 comp rx=0 tx=0).stop 2024-04-25T10:14:39.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 shutdown_connections 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a9006b4d0 0x7f9a9006d990 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9ab4165680 0x7f9ab414bc40 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ab40cec70 0x7f9ab414b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 --2- 172.21.15.165:0/856081208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ab40cc6c0 0x7f9ab414f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 >> 172.21.15.165:0/856081208 conn(0x7f9ab406abe0 msgr2=0x7f9ab4159380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 shutdown_connections 2024-04-25T10:14:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.162+0000 7f9abb200640 1 -- 172.21.15.165:0/856081208 wait complete. 2024-04-25T10:14:39.183 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:14:39.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.184 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub abort 2024-04-25T10:14:39.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.378+0000 7fb2db200640 1 Processor -- start 2024-04-25T10:14:39.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.378+0000 7fb2db200640 1 -- start start 2024-04-25T10:14:39.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.379+0000 7fb2db200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb2dc145140 0x7fb2dc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2db200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2dc0ce950 0x7fb2dc0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2db200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2dc145520 con 0x7fb2dc0c5030 2024-04-25T10:14:39.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2db200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc0ced90 con 0x7fb2dc0ce950 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2db200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2dc071270 con 0x7fb2dc145140 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d9400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb2dc0ce950 0x7fb2dc0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2da800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb2dc0c5030 0x7fb2dc06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47256/0 (socket says 172.21.15.165:47256) 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2da800640 1 -- 172.21.15.165:0/1477833939 learned_addr learned my addr 172.21.15.165:0/1477833939 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d9400640 1 -- 172.21.15.165:0/1477833939 >> v2:172.21.15.172:3300/0 conn(0x7fb2dc145140 msgr2=0x7fb2dc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1477833939 >> v2:172.21.15.172:3300/0 conn(0x7fb2dc145140 0x7fb2dc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1477833939 >> v2:172.21.15.172:3300/0 conn(0x7fb2dc145140 0x7fb2dc0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d9400640 1 -- 172.21.15.165:0/1477833939 >> v1:172.21.15.165:6789/0 conn(0x7fb2dc0c5030 legacy=0x7fb2dc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d8a00640 1 -- 172.21.15.165:0/1477833939 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2370577273 0 0) 0x7fb2dc145520 con 0x7fb2dc0c5030 2024-04-25T10:14:39.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.380+0000 7fb2d9400640 1 -- 172.21.15.165:0/1477833939 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc071550 con 0x7fb2dc0ce950 2024-04-25T10:14:39.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.381+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1477833939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc0ce950 0x7fb2dc0c4af0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06f690 tx=0x7fb2c802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49012e82b559996 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2d8a00640 1 -- 172.21.15.165:0/1477833939 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8037070 con 0x7fb2dc0ce950 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2d8a00640 1 -- 172.21.15.165:0/1477833939 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8032440 con 0x7fb2dc0ce950 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2d8a00640 1 -- 172.21.15.165:0/1477833939 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c802ae60 con 0x7fb2dc0ce950 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 -- 172.21.15.165:0/1477833939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc0ce950 msgr2=0x7fb2dc0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 --2- 172.21.15.165:0/1477833939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc0ce950 0x7fb2dc0c4af0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc06f690 tx=0x7fb2c802a110 comp rx=0 tx=0).stop 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 -- 172.21.15.165:0/1477833939 shutdown_connections 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 --2- 172.21.15.165:0/1477833939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc0ce950 0x7fb2dc0c4af0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 --2- 172.21.15.165:0/1477833939 >> v2:172.21.15.172:3300/0 conn(0x7fb2dc145140 0x7fb2dc0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 -- 172.21.15.165:0/1477833939 >> 172.21.15.165:0/1477833939 conn(0x7fb2dc061cc0 msgr2=0x7fb2dc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 -- 172.21.15.165:0/1477833939 shutdown_connections 2024-04-25T10:14:39.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.382+0000 7fb2db200640 1 -- 172.21.15.165:0/1477833939 wait complete. 2024-04-25T10:14:39.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.384+0000 7fb2db200640 1 Processor -- start 2024-04-25T10:14:39.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.384+0000 7fb2db200640 1 -- start start 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0c5030 0x7fb2dc0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2db200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0ce950 0x7fb2dc1f0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 0x7fb2dc1f4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2db200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c5480 con 0x7fb2dc145140 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2db200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc071b10 con 0x7fb2dc0ce950 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0c5600 con 0x7fb2dc0c5030 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2da800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 0x7fb2dc1f4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2da800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 0x7fb2dc1f4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37722/0 (socket says 172.21.15.165:37722) 2024-04-25T10:14:39.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.385+0000 7fb2da800640 1 -- 172.21.15.165:0/2828647299 learned_addr learned my addr 172.21.15.165:0/2828647299 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2da800640 1 -- 172.21.15.165:0/2828647299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0c5030 msgr2=0x7fb2dc0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2da800640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0c5030 0x7fb2dc0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2da800640 1 -- 172.21.15.165:0/2828647299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0ce950 msgr2=0x7fb2dc1f0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2da800640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0ce950 0x7fb2dc1f0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2da800640 1 -- 172.21.15.165:0/2828647299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0c94f0 con 0x7fb2dc145140 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2da800640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 0x7fb2dc1f4610 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c7a0 tx=0x7fb2d400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2d2000640 1 -- 172.21.15.165:0/2828647299 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d4018070 con 0x7fb2dc145140 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2d2000640 1 -- 172.21.15.165:0/2828647299 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d400a640 con 0x7fb2dc145140 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2d2000640 1 -- 172.21.15.165:0/2828647299 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d400aa70 con 0x7fb2dc145140 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2dc0c8c50 con 0x7fb2dc145140 2024-04-25T10:14:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.386+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2dc0c9210 con 0x7fb2dc145140 2024-04-25T10:14:39.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.388+0000 7fb2d2000640 1 -- 172.21.15.165:0/2828647299 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb2d400ac10 con 0x7fb2dc145140 2024-04-25T10:14:39.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.389+0000 7fb2d2000640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.389+0000 7fb2d2000640 1 -- 172.21.15.165:0/2828647299 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2d4089ac0 con 0x7fb2dc145140 2024-04-25T10:14:39.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.389+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.390+0000 7fb2e3bfd740 1 Processor -- start 2024-04-25T10:14:39.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.390+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00a810 tx=0x7fb2cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.390+0000 7fb2e3bfd740 1 -- start start 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f8c80 0x55a9e77f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77c93c0 0x55a9e77f1100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e772d870 con 0x55a9e77f96a0 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e76d6d60 con 0x55a9e77f8c80 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9e7729910 con 0x55a9e77c93c0 2024-04-25T10:14:39.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37736/0 (socket says 172.21.15.165:37736) 2024-04-25T10:14:39.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.391+0000 7fb2d9400640 1 -- 172.21.15.165:0/612320578 learned_addr learned my addr 172.21.15.165:0/612320578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2d9400640 1 -- 172.21.15.165:0/612320578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77c93c0 msgr2=0x55a9e77f1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2d9400640 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77c93c0 0x55a9e77f1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2d9400640 1 -- 172.21.15.165:0/612320578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f8c80 msgr2=0x55a9e77f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2d9400640 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f8c80 0x55a9e77f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2d9400640 1 -- 172.21.15.165:0/612320578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9e750c7d0 con 0x55a9e77f96a0 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2d9400640 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77c8e80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009890 tx=0x7fb2c802c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7a6e3477ca0cb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2c6a00640 1 -- 172.21.15.165:0/612320578 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8037050 con 0x55a9e77f96a0 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2c6a00640 1 -- 172.21.15.165:0/612320578 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8043070 con 0x55a9e77f96a0 2024-04-25T10:14:39.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.392+0000 7fb2c6a00640 1 -- 172.21.15.165:0/612320578 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c803e870 con 0x55a9e77f96a0 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/612320578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 msgr2=0x55a9e77c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77c8e80 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8009890 tx=0x7fb2c802c070 comp rx=0 tx=0).stop 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/612320578 shutdown_connections 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77c93c0 0x55a9e77f1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77c8e80 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/612320578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f8c80 0x55a9e77f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/612320578 >> 172.21.15.165:0/612320578 conn(0x55a9e77cb300 msgr2=0x55a9e7813a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/612320578 shutdown_connections 2024-04-25T10:14:39.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.393+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/612320578 wait complete. 2024-04-25T10:14:39.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.394+0000 7fb2e3bfd740 1 -- start start 2024-04-25T10:14:39.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.394+0000 7fb2e3bfd740 1 Processor -- start 2024-04-25T10:14:39.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f8c80 0x55a9e77f4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77fc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77f79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e77f53d0 con 0x55a9e77f96a0 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e76ea1c0 con 0x55a9e77fb760 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9e77f5670 con 0x55a9e77f8c80 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2e3bfd740 10 client.0 _ll_register_callbacks cb 0x55a9e77288f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77fc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77fc8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37750/0 (socket says 172.21.15.165:37750) 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.395+0000 7fb2d9400640 1 -- 172.21.15.165:0/1821736939 learned_addr learned my addr 172.21.15.165:0/1821736939 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f8c80 0x55a9e77f4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2da800640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77f79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9400640 1 -- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f8c80 msgr2=0x55a9e77f4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f8c80 0x55a9e77f4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9400640 1 -- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 msgr2=0x55a9e77f79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77f79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9400640 1 -- 172.21.15.165:0/1821736939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9e74020f0 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2da800640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77fc8f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8040040 tx=0x7fb2c8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554e566cbee0209c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2bb400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2bb400640 1 -- 172.21.15.165:0/1821736939 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8032640 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2bb400640 1 -- 172.21.15.165:0/1821736939 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2c80327e0 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.396+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2c8043070 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2bb400640 1 -- 172.21.15.165:0/1821736939 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2c8057050 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 10 client.8886 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a9e756cf60 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2bb400640 1 -- 172.21.15.165:0/1821736939 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c8043070 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2bb400640 1 -- 172.21.15.165:0/1821736939 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb2c805a020 con 0x55a9e77f96a0 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 10 client.8886 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 10 client.8886 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:39.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 20 client.8886 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:39.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9f50 0x55a9e77fa330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 4 client.8886 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:39.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.397+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a9e7703690 con 0x55a9e77f9f50 2024-04-25T10:14:39.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.398+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9f50 0x55a9e77fa330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.399+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9f50 0x55a9e77fa330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efc8b9c107173ce2 server_cookie=3cba54db67499b8a in_seq=0 out_seq=0 2024-04-25T10:14:39.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.399+0000 7fb2bb400640 10 client.8886 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2bb400640 1 -- 172.21.15.165:0/1821736939 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a9e7703690 con 0x55a9e77f9f50 2024-04-25T10:14:39.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2bb400640 10 client.8886 handle_command_reply: tid=0 2024-04-25T10:14:39.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 client.8886 shutdown 2024-04-25T10:14:39.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 msgr2=0x55a9e77fc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77fc8f0 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8040040 tx=0x7fb2c8008030 comp rx=0 tx=0).stop 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 shutdown_connections 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9f50 0x55a9e77fa330 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77f79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f96a0 0x55a9e77fc8f0 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1821736939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f8c80 0x55a9e77f4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 >> 172.21.15.165:0/1821736939 conn(0x55a9e77c93c0 msgr2=0x55a9e77ce3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2bb400640 0 client.8886 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 shutdown_connections 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1821736939 wait complete. 2024-04-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.401+0000 7fb2e3bfd740 20 client.8886 trim_cache size 0 max 16384 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.417+0000 7fb2e3bfd740 1 Processor -- start 2024-04-25T10:14:39.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.417+0000 7fb2e3bfd740 1 -- start start 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e77c3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f96a0 0x55a9e77f9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc1fbc10 con 0x55a9e77f8c80 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0cda60 con 0x55a9e77fb760 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0a99b0 con 0x55a9e77f96a0 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e77c3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e77c3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37754/0 (socket says 172.21.15.165:37754) 2024-04-25T10:14:39.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.418+0000 7fb2d9e00640 1 -- 172.21.15.165:0/3054348445 learned_addr learned my addr 172.21.15.165:0/3054348445 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2d9e00640 1 -- 172.21.15.165:0/3054348445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f96a0 msgr2=0x55a9e77f9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f96a0 0x55a9e77f9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2d9e00640 1 -- 172.21.15.165:0/3054348445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 msgr2=0x55a9e77cfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2d9e00640 1 -- 172.21.15.165:0/3054348445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a9e73db1b0 con 0x55a9e77f8c80 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e77c3960 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00c520 tx=0x7fb2cc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82bebf8d672a95d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2c6000640 1 -- 172.21.15.165:0/3054348445 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc013ba0 con 0x55a9e77f8c80 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2c6000640 1 -- 172.21.15.165:0/3054348445 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc026030 con 0x55a9e77f8c80 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.419+0000 7fb2c6000640 1 -- 172.21.15.165:0/3054348445 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc0220e0 con 0x55a9e77f8c80 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/3054348445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 msgr2=0x55a9e77c3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e77c3960 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00c520 tx=0x7fb2cc00f000 comp rx=0 tx=0).stop 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/3054348445 shutdown_connections 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77fb760 0x55a9e77cfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77f96a0 0x55a9e77f9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/3054348445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e77c3960 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/3054348445 >> 172.21.15.165:0/3054348445 conn(0x55a9e77c93c0 msgr2=0x55a9e77f2e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/3054348445 shutdown_connections 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.420+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/3054348445 wait complete. 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.421+0000 7fb2e3bfd740 1 -- start start 2024-04-25T10:14:39.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.421+0000 7fb2e3bfd740 1 Processor -- start 2024-04-25T10:14:39.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e787c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f96a0 0x55a9e77f2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77fb760 0x55a9e78135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9e77f1a70 con 0x55a9e77f8c80 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2dc0cd460 con 0x55a9e77f96a0 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a9e77f3510 con 0x55a9e77fb760 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2e3bfd740 10 client.0 _ll_register_callbacks cb 0x7fb2c8035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e787c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e787c430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37756/0 (socket says 172.21.15.165:37756) 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 -- 172.21.15.165:0/1793752608 learned_addr learned my addr 172.21.15.165:0/1793752608 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f96a0 0x55a9e77f2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2da800640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77fb760 0x55a9e78135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 -- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77fb760 msgr2=0x55a9e78135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77fb760 0x55a9e78135f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 -- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f96a0 msgr2=0x55a9e77f2fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f96a0 0x55a9e77f2fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.422+0000 7fb2d9e00640 1 -- 172.21.15.165:0/1793752608 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a9e7880aa0 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2d9e00640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e787c430 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006eb0 tx=0x7fb2cc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970d2bce673bbbad server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2bb400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2bb400640 1 -- 172.21.15.165:0/1793752608 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc013db0 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2bb400640 1 -- 172.21.15.165:0/1793752608 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb2cc035c50 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2cc026030 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2bb400640 1 -- 172.21.15.165:0/1793752608 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2cc03c050 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2e3bfd740 10 client.8892 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.423+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55a9e77cdde0 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2bb400640 1 -- 172.21.15.165:0/1793752608 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc026030 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2bb400640 1 -- 172.21.15.165:0/1793752608 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb2cc03f020 con 0x55a9e77f8c80 2024-04-25T10:14:39.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2e3bfd740 10 client.8892 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:39.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2e3bfd740 10 client.8892 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:39.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2e3bfd740 20 client.8892 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:39.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9060 0x55a9e77cfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2e3bfd740 4 client.8892 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:14:39.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x55a9e773adb0 con 0x55a9e77f9060 2024-04-25T10:14:39.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.424+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9060 0x55a9e77cfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.425+0000 7fb2bb400640 10 client.8892 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.425+0000 7fb2d9400640 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9060 0x55a9e77cfa90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b48021b3be50c16 server_cookie=b27878ef3cb847fd in_seq=0 out_seq=0 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2bb400640 1 -- 172.21.15.165:0/1793752608 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x55a9e773adb0 con 0x55a9e77f9060 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2bb400640 10 client.8892 handle_command_reply: tid=0 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 client.8892 shutdown 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 msgr2=0x55a9e787c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e787c430 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc006eb0 tx=0x7fb2cc011000 comp rx=0 tx=0).stop 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 shutdown_connections 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55a9e77f9060 0x55a9e77cfa90 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a9e77fb760 0x55a9e78135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a9e77f96a0 0x55a9e77f2fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 --2- 172.21.15.165:0/1793752608 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a9e77f8c80 0x55a9e787c430 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2bb400640 0 client.8892 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 >> 172.21.15.165:0/1793752608 conn(0x55a9e77c93c0 msgr2=0x55a9e77f9f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 shutdown_connections 2024-04-25T10:14:39.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 1 -- 172.21.15.165:0/1793752608 wait complete. 2024-04-25T10:14:39.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.426+0000 7fb2e3bfd740 20 client.8892 trim_cache size 0 max 16384 2024-04-25T10:14:39.433 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:39.433 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:14:39.433 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:39.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2bc06b4d0 msgr2=0x7fb2bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc00a810 tx=0x7fb2cc008040 comp rx=0 tx=0).stop 2024-04-25T10:14:39.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 msgr2=0x7fb2dc1f4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 0x7fb2dc1f4610 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fb2d400c7a0 tx=0x7fb2d400cc60 comp rx=0 tx=0).stop 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 shutdown_connections 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2bc06b4d0 0x7fb2bc06d990 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2dc145140 0x7fb2dc1f4610 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2dc0ce950 0x7fb2dc1f0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 --2- 172.21.15.165:0/2828647299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb2dc0c5030 0x7fb2dc0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 >> 172.21.15.165:0/2828647299 conn(0x7fb2dc061cc0 msgr2=0x7fb2dc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 shutdown_connections 2024-04-25T10:14:39.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.427+0000 7fb2db200640 1 -- 172.21.15.165:0/2828647299 wait complete. 2024-04-25T10:14:39.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:39.448 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:14:39.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.634+0000 7f637fa00640 1 Processor -- start 2024-04-25T10:14:39.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.634+0000 7f637fa00640 1 -- start start 2024-04-25T10:14:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637fa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6378161fc0 0x7f63781623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637fa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f637806ea90 0x7f6378071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637fa00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63781628e0 con 0x7f637806d2d0 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637fa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6378165be0 con 0x7f6378161fc0 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637fa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6378162ba0 con 0x7f637806ea90 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637d000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6378161fc0 0x7f63781623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637d000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6378161fc0 0x7f63781623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37764/0 (socket says 172.21.15.165:37764) 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f637d000640 1 -- 172.21.15.165:0/1063761389 learned_addr learned my addr 172.21.15.165:0/1063761389 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.636+0000 7f6377e00640 1 --2- 172.21.15.165:0/1063761389 >> v2:172.21.15.172:3300/0 conn(0x7f637806ea90 0x7f6378071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.637+0000 7f637d000640 1 -- 172.21.15.165:0/1063761389 >> v2:172.21.15.172:3300/0 conn(0x7f637806ea90 msgr2=0x7f6378071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.637+0000 7f637d000640 1 --2- 172.21.15.165:0/1063761389 >> v2:172.21.15.172:3300/0 conn(0x7f637806ea90 0x7f6378071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.637+0000 7f637d000640 1 -- 172.21.15.165:0/1063761389 >> v1:172.21.15.172:6789/0 conn(0x7f637806d2d0 legacy=0x7f637806d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.637+0000 7f637d000640 1 -- 172.21.15.165:0/1063761389 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6378162e80 con 0x7f6378161fc0 2024-04-25T10:14:39.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.638+0000 7f637d000640 1 --2- 172.21.15.165:0/1063761389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6378161fc0 0x7f63781623a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f63780713b0 tx=0x7f636c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c032ab9c7c79c4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.638+0000 7f6377400640 1 -- 172.21.15.165:0/1063761389 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c037070 con 0x7f6378161fc0 2024-04-25T10:14:39.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.638+0000 7f6377400640 1 -- 172.21.15.165:0/1063761389 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c032440 con 0x7f6378161fc0 2024-04-25T10:14:39.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.638+0000 7f6377400640 1 -- 172.21.15.165:0/1063761389 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c032810 con 0x7f6378161fc0 2024-04-25T10:14:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 -- 172.21.15.165:0/1063761389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6378161fc0 msgr2=0x7f63781623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 --2- 172.21.15.165:0/1063761389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6378161fc0 0x7f63781623a0 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f63780713b0 tx=0x7f636c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 -- 172.21.15.165:0/1063761389 shutdown_connections 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 --2- 172.21.15.165:0/1063761389 >> v2:172.21.15.172:3300/0 conn(0x7f637806ea90 0x7f6378071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 --2- 172.21.15.165:0/1063761389 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6378161fc0 0x7f63781623a0 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 -- 172.21.15.165:0/1063761389 >> 172.21.15.165:0/1063761389 conn(0x7f637806abe0 msgr2=0x7f637806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 -- 172.21.15.165:0/1063761389 shutdown_connections 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.639+0000 7f637fa00640 1 -- 172.21.15.165:0/1063761389 wait complete. 2024-04-25T10:14:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.641+0000 7f637fa00640 1 Processor -- start 2024-04-25T10:14:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.641+0000 7f637fa00640 1 -- start start 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f637fa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637806d2d0 0x7f63780b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f637fa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 0x7f63781ead80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f637fa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6378161fc0 0x7f63781eb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f637fa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63780719c0 con 0x7f637806ea90 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f637fa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378163440 con 0x7f637806d2d0 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f637fa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6378071b40 con 0x7f6378161fc0 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f6377e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 0x7f63781ead80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f6377e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 0x7f63781ead80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37772/0 (socket says 172.21.15.165:37772) 2024-04-25T10:14:39.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.642+0000 7f6377e00640 1 -- 172.21.15.165:0/3211575893 learned_addr learned my addr 172.21.15.165:0/3211575893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f637da00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6378161fc0 0x7f63781eb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f637d000640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637806d2d0 0x7f63780b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6377e00640 1 -- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6378161fc0 msgr2=0x7f63781eb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6377e00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6378161fc0 0x7f63781eb2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6377e00640 1 -- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637806d2d0 msgr2=0x7f63780b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6377e00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637806d2d0 0x7f63780b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6377e00640 1 -- 172.21.15.165:0/3211575893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63781eb9d0 con 0x7f637806ea90 2024-04-25T10:14:39.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6377e00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 0x7f63781ead80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f636400d800 tx=0x7f636400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6374c00640 1 -- 172.21.15.165:0/3211575893 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400b6c0 con 0x7f637806ea90 2024-04-25T10:14:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6374c00640 1 -- 172.21.15.165:0/3211575893 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6364019070 con 0x7f637806ea90 2024-04-25T10:14:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f6374c00640 1 -- 172.21.15.165:0/3211575893 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636400ba80 con 0x7f637806ea90 2024-04-25T10:14:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63781ebcb0 con 0x7f637806ea90 2024-04-25T10:14:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.643+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63780b0760 con 0x7f637806ea90 2024-04-25T10:14:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.645+0000 7f6374c00640 1 -- 172.21.15.165:0/3211575893 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f636400bc20 con 0x7f637806ea90 2024-04-25T10:14:39.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.646+0000 7f6374c00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635806b4d0 0x7f635806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.646+0000 7f6374c00640 1 -- 172.21.15.165:0/3211575893 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f636408a7b0 con 0x7f637806ea90 2024-04-25T10:14:39.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.647+0000 7f637d000640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635806b4d0 0x7f635806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.647+0000 7f637d000640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635806b4d0 0x7f635806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f636c034000 tx=0x7f636c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.647+0000 7f63827f0740 1 Processor -- start 2024-04-25T10:14:39.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 -- start start 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0ed1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ecff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad0e33c70 con 0x558ad0ecfbb0 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad0ddccf0 con 0x558ad0efa670 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.648+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ad0e2fd10 con 0x558ad0f0c280 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0ed1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0ed1d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37784/0 (socket says 172.21.15.165:37784) 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 -- 172.21.15.165:0/2572993046 learned_addr learned my addr 172.21.15.165:0/2572993046 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 -- 172.21.15.165:0/2572993046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 msgr2=0x558ad0ecff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ecff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 -- 172.21.15.165:0/2572993046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 msgr2=0x558ad0efaaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 -- 172.21.15.165:0/2572993046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad0c127d0 con 0x558ad0ecfbb0 2024-04-25T10:14:39.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.649+0000 7f637d000640 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0ed1d10 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f636c02a790 tx=0x7f636c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5998903377e56be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f6369600640 1 -- 172.21.15.165:0/2572993046 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c04d070 con 0x558ad0ecfbb0 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f6369600640 1 -- 172.21.15.165:0/2572993046 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c032d60 con 0x558ad0ecfbb0 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f6369600640 1 -- 172.21.15.165:0/2572993046 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c037040 con 0x558ad0ecfbb0 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 -- 172.21.15.165:0/2572993046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 msgr2=0x558ad0ed1d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0ed1d10 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f636c02a790 tx=0x7f636c02c000 comp rx=0 tx=0).stop 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 -- 172.21.15.165:0/2572993046 shutdown_connections 2024-04-25T10:14:39.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efaaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ecff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 --2- 172.21.15.165:0/2572993046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0ed1d10 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 -- 172.21.15.165:0/2572993046 >> 172.21.15.165:0/2572993046 conn(0x558ad0e97a30 msgr2=0x558ad0f15670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.650+0000 7f63827f0740 1 -- 172.21.15.165:0/2572993046 shutdown_connections 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.651+0000 7f63827f0740 1 -- 172.21.15.165:0/2572993046 wait complete. 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.651+0000 7f63827f0740 1 -- start start 2024-04-25T10:14:39.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.651+0000 7f63827f0740 1 Processor -- start 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.652+0000 7f63827f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f17070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.652+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0efa670 0x558ad0f0ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.652+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0f0c280 0x558ad0ee78a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.652+0000 7f63827f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad0dd23a0 con 0x558ad0ecfbb0 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad0ef81c0 con 0x558ad0f0c280 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ad0f17970 con 0x558ad0efa670 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f63827f0740 10 client.0 _ll_register_callbacks cb 0x558ad0defb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f17070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f17070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37792/0 (socket says 172.21.15.165:37792) 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 -- 172.21.15.165:0/295714578 learned_addr learned my addr 172.21.15.165:0/295714578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 -- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0efa670 msgr2=0x558ad0f0ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0efa670 0x558ad0f0ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 -- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0f0c280 msgr2=0x558ad0ee78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0f0c280 0x558ad0ee78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 -- 172.21.15.165:0/295714578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ad0b483f0 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f637d000640 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f17070 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f636c009960 tx=0x7f636c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78dd649eba7e372f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.653+0000 7f635e000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f635e000640 1 -- 172.21.15.165:0/295714578 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c058060 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f635e000640 1 -- 172.21.15.165:0/295714578 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f636c0489c0 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f636c032d60 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f635e000640 1 -- 172.21.15.165:0/295714578 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636c05e050 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 10 client.8904 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x558ad0b080f0 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f635e000640 1 -- 172.21.15.165:0/295714578 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c032d60 con 0x558ad0ecfbb0 2024-04-25T10:14:39.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f635e000640 1 -- 172.21.15.165:0/295714578 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f636c061020 con 0x558ad0ecfbb0 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 10 client.8904 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 10 client.8904 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 20 client.8904 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0ee5d50 0x558ad0f0fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 4 client.8904 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.654+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ad0e411b0 con 0x558ad0ee5d50 2024-04-25T10:14:39.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.655+0000 7f6377e00640 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0ee5d50 0x558ad0f0fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.656+0000 7f6377e00640 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0ee5d50 0x558ad0f0fb80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a05b51bc953bc7aa server_cookie=f3544e5d7a7c4b42 in_seq=0 out_seq=0 2024-04-25T10:14:39.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.656+0000 7f635e000640 10 client.8904 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f635e000640 1 -- 172.21.15.165:0/295714578 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558ad0e411b0 con 0x558ad0ee5d50 2024-04-25T10:14:39.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f635e000640 10 client.8904 handle_command_reply: tid=0 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 client.8904 shutdown 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 msgr2=0x558ad0f17070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f17070 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f636c009960 tx=0x7f636c004730 comp rx=0 tx=0).stop 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 shutdown_connections 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0ee5d50 0x558ad0f0fb80 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0f0c280 0x558ad0ee78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0efa670 0x558ad0f0ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f635e000640 0 client.8904 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 --2- 172.21.15.165:0/295714578 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f17070 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 >> 172.21.15.165:0/295714578 conn(0x558ad0e97a30 msgr2=0x558ad0ef7990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 shutdown_connections 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 1 -- 172.21.15.165:0/295714578 wait complete. 2024-04-25T10:14:39.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.658+0000 7f63827f0740 20 client.8904 trim_cache size 0 max 16384 2024-04-25T10:14:39.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.674+0000 7f63827f0740 1 Processor -- start 2024-04-25T10:14:39.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.674+0000 7f63827f0740 1 -- start start 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.675+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0ecfbb0 0x558ad0ee7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.675+0000 7f63827f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 0x558ad0f19150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.675+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ec9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.675+0000 7f63827f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378204640 con 0x558ad0efa670 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.675+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6378203a60 con 0x558ad0ecfbb0 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.675+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63781eda10 con 0x558ad0f0c280 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 0x558ad0f19150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 0x558ad0f19150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37802/0 (socket says 172.21.15.165:37802) 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 -- 172.21.15.165:0/2597234743 learned_addr learned my addr 172.21.15.165:0/2597234743 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f637d000640 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0ecfbb0 0x558ad0ee7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f637da00640 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ec9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 -- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 msgr2=0x558ad0ec9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ec9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 -- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0ecfbb0 msgr2=0x558ad0ee7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0ecfbb0 0x558ad0ee7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 -- 172.21.15.165:0/2597234743 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ad0ee5b60 con 0x558ad0efa670 2024-04-25T10:14:39.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6377e00640 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 0x558ad0f19150 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f636400d7d0 tx=0x7f6364006d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66955ba115119102 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6368c00640 1 -- 172.21.15.165:0/2597234743 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364061540 con 0x558ad0efa670 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6368c00640 1 -- 172.21.15.165:0/2597234743 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6364087070 con 0x558ad0efa670 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.676+0000 7f6368c00640 1 -- 172.21.15.165:0/2597234743 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364056ed0 con 0x558ad0efa670 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 -- 172.21.15.165:0/2597234743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 msgr2=0x558ad0f19150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 0x558ad0f19150 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f636400d7d0 tx=0x7f6364006d50 comp rx=0 tx=0).stop 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 -- 172.21.15.165:0/2597234743 shutdown_connections 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0ec9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0efa670 0x558ad0f19150 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 --2- 172.21.15.165:0/2597234743 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0ecfbb0 0x558ad0ee7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 -- 172.21.15.165:0/2597234743 >> 172.21.15.165:0/2597234743 conn(0x558ad0e97a30 msgr2=0x558ad0f10c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 -- 172.21.15.165:0/2597234743 shutdown_connections 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.677+0000 7f63827f0740 1 -- 172.21.15.165:0/2597234743 wait complete. 2024-04-25T10:14:39.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.678+0000 7f63827f0740 1 -- start start 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.678+0000 7f63827f0740 1 Processor -- start 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f7edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efd170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0efd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ad0f0de40 con 0x558ad0ecfbb0 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637815fbd0 con 0x558ad0efa670 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ad0f27ef0 con 0x558ad0f0c280 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f63827f0740 10 client.0 _ll_register_callbacks cb 0x7f636c04b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f637d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f7edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f637d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f7edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37808/0 (socket says 172.21.15.165:37808) 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f637d000640 1 -- 172.21.15.165:0/4023269469 learned_addr learned my addr 172.21.15.165:0/4023269469 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f637d000640 1 -- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 msgr2=0x558ad0efd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f637d000640 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0efd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f637d000640 1 -- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 msgr2=0x558ad0efd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.679+0000 7f6377e00640 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f637d000640 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efd170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f637d000640 1 -- 172.21.15.165:0/4023269469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558ad0f18690 con 0x558ad0ecfbb0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f637d000640 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f7edc0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f636c009960 tx=0x7f636c008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb80add10c38f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f635e000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f635e000640 1 -- 172.21.15.165:0/4023269469 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c067060 con 0x558ad0ecfbb0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f635e000640 1 -- 172.21.15.165:0/4023269469 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f636c03ca50 con 0x558ad0ecfbb0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6364087070 con 0x558ad0ecfbb0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f635e000640 1 -- 172.21.15.165:0/4023269469 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f636c06d050 con 0x558ad0ecfbb0 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f63827f0740 10 client.8910 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:39.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x558ad0ef9fb0 con 0x558ad0ecfbb0 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.680+0000 7f635e000640 1 -- 172.21.15.165:0/4023269469 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6364087070 con 0x558ad0ecfbb0 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f635e000640 1 -- 172.21.15.165:0/4023269469 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f636c070020 con 0x558ad0ecfbb0 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f63827f0740 10 client.8910 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f63827f0740 10 client.8910 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f63827f0740 20 client.8910 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f63827f0740 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0f0b000 0x558ad0ec9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f63827f0740 4 client.8910 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558ad0e3b500 con 0x558ad0f0b000 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.681+0000 7f6377e00640 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0f0b000 0x558ad0ec9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.682+0000 7f6377e00640 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0f0b000 0x558ad0ec9a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d904060fca4f113 server_cookie=7c25bd4332c50e4 in_seq=0 out_seq=0 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.682+0000 7f635e000640 10 client.8910 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f635e000640 1 -- 172.21.15.165:0/4023269469 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+69 (crc 0 0 0) 0x558ad0e3b500 con 0x558ad0f0b000 2024-04-25T10:14:39.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f635e000640 10 client.8910 handle_command_reply: tid=0 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 client.8910 shutdown 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 msgr2=0x558ad0f7edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f7edc0 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7f636c009960 tx=0x7f636c008030 comp rx=0 tx=0).stop 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 shutdown_connections 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x558ad0f0b000 0x558ad0ec9a30 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558ad0f0c280 0x558ad0efd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558ad0efa670 0x558ad0efd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 --2- 172.21.15.165:0/4023269469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558ad0ecfbb0 0x558ad0f7edc0 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f635e000640 0 client.8910 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 >> 172.21.15.165:0/4023269469 conn(0x558ad0e97a30 msgr2=0x558ad0f10dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 shutdown_connections 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 1 -- 172.21.15.165:0/4023269469 wait complete. 2024-04-25T10:14:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.683+0000 7f63827f0740 20 client.8910 trim_cache size 0 max 16384 2024-04-25T10:14:39.695 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:39.695 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:14:39.695 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:14:39.695 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635806b4d0 msgr2=0x7f635806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635806b4d0 0x7f635806d990 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f636c034000 tx=0x7f636c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 msgr2=0x7f63781ead80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 0x7f63781ead80 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f636400d800 tx=0x7f636400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 shutdown_connections 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635806b4d0 0x7f635806d990 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6378161fc0 0x7f63781eb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637806ea90 0x7f63781ead80 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 --2- 172.21.15.165:0/3211575893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637806d2d0 0x7f63780b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 >> 172.21.15.165:0/3211575893 conn(0x7f637806abe0 msgr2=0x7f637806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 shutdown_connections 2024-04-25T10:14:39.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.684+0000 7f637fa00640 1 -- 172.21.15.165:0/3211575893 wait complete. 2024-04-25T10:14:39.704 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.885+0000 7fb4aa800640 1 Processor -- start 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.885+0000 7fb4aa800640 1 -- start start 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.886+0000 7fb4aa800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb4a4144750 0x7fb4a406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb4aa800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a4144b30 con 0x7fb4a406eb10 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb4aa800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4a40712d0 con 0x7fb4a40cf030 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb4aa800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4a40715b0 con 0x7fb4a4144750 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb49aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb4a406eb10 0x7fb4a40ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47268/0 (socket says 172.21.15.165:47268) 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb49aa00640 1 -- 172.21.15.165:0/2392351065 learned_addr learned my addr 172.21.15.165:0/2392351065 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb4a3400640 1 --2- 172.21.15.165:0/2392351065 >> v2:172.21.15.172:3300/0 conn(0x7fb4a4144750 0x7fb4a406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140669632 0 0) 0x7fb4a4144b30 con 0x7fb4a406eb10 2024-04-25T10:14:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.887+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb488003520 con 0x7fb4a406eb10 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1408245244 0 0) 0x7fb488003520 con 0x7fb4a406eb10 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 >> v2:172.21.15.172:3300/0 conn(0x7fb4a4144750 msgr2=0x7fb4a406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 --2- 172.21.15.165:0/2392351065 >> v2:172.21.15.172:3300/0 conn(0x7fb4a4144750 0x7fb4a406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 >> v1:172.21.15.172:6789/0 conn(0x7fb4a40cf030 legacy=0x7fb4a406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a4071890 con 0x7fb4a406eb10 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb48c001670 con 0x7fb4a406eb10 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 >> v1:172.21.15.165:6789/0 conn(0x7fb4a406eb10 legacy=0x7fb4a40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4880039b0 0x7fb488003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb488004800 0x7fb488004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4880051c0 0x7fb48800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a40715b0 con 0x7fb4880051c0 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb48800c060 con 0x7fb4880039b0 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48800c260 con 0x7fb488004800 2024-04-25T10:14:39.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.888+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4880051c0 0x7fb48800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb4a3400640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4880039b0 0x7fb488003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb49aa00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb488004800 0x7fb488004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb4a3e00640 1 -- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb488004800 msgr2=0x7fb488004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb488004800 0x7fb488004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb4a3e00640 1 -- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4880039b0 msgr2=0x7fb488003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4880039b0 0x7fb488003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.889+0000 7fb4a3e00640 1 -- 172.21.15.165:0/2392351065 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb488003520 con 0x7fb4880051c0 2024-04-25T10:14:39.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4a3e00640 1 -- 172.21.15.165:0/2392351065 reap_dead start 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4880051c0 0x7fb48800b9c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb49c001e50 tx=0x7fb488005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5cc3d41708c2cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c001a70 con 0x7fb4880051c0 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c003970 con 0x7fb4880051c0 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4a2a00640 1 -- 172.21.15.165:0/2392351065 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c003d60 con 0x7fb4880051c0 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4aa800640 1 -- 172.21.15.165:0/2392351065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4880051c0 msgr2=0x7fb48800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.890+0000 7fb4aa800640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4880051c0 0x7fb48800b9c0 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fb49c001e50 tx=0x7fb488005830 comp rx=0 tx=0).stop 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.891+0000 7fb4aa800640 1 -- 172.21.15.165:0/2392351065 shutdown_connections 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.891+0000 7fb4aa800640 1 --2- 172.21.15.165:0/2392351065 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4880051c0 0x7fb48800b9c0 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.891+0000 7fb4aa800640 1 -- 172.21.15.165:0/2392351065 >> 172.21.15.165:0/2392351065 conn(0x7fb4a406abe0 msgr2=0x7fb4a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.891+0000 7fb4aa800640 1 -- 172.21.15.165:0/2392351065 shutdown_connections 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.891+0000 7fb4aa800640 1 -- 172.21.15.165:0/2392351065 wait complete. 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.892+0000 7fb4aa800640 1 Processor -- start 2024-04-25T10:14:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 -- start start 2024-04-25T10:14:39.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4880051c0 0x7fb4a41f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40b3010 0x7fb4a415b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 0x7fb4a41427b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a41455d0 con 0x7fb4a4142330 2024-04-25T10:14:39.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a4145450 con 0x7fb4a40b3010 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.893+0000 7fb4aa800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a415c000 con 0x7fb4880051c0 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 0x7fb4a41427b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 0x7fb4a41427b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37822/0 (socket says 172.21.15.165:37822) 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 -- 172.21.15.165:0/1491387382 learned_addr learned my addr 172.21.15.165:0/1491387382 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3400640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4880051c0 0x7fb4a41f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb49aa00640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40b3010 0x7fb4a415b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 -- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4880051c0 msgr2=0x7fb4a41f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4880051c0 0x7fb4a41f35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 -- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40b3010 msgr2=0x7fb4a415b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40b3010 0x7fb4a415b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3e00640 1 -- 172.21.15.165:0/1491387382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a40b3860 con 0x7fb4a4142330 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.894+0000 7fb4a3400640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4880051c0 0x7fb4a41f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.895+0000 7fb4a3e00640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 0x7fb4a41427b0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb49c02a920 tx=0x7fb49c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.895+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c007db0 con 0x7fb4a4142330 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.895+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb49c02ac70 con 0x7fb4a4142330 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.895+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a40b3b40 con 0x7fb4a4142330 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.895+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb49c006d70 con 0x7fb4a4142330 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.895+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a40b4100 con 0x7fb4a4142330 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.896+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a4073990 con 0x7fb4a4142330 2024-04-25T10:14:39.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.897+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb49c007030 con 0x7fb4a4142330 2024-04-25T10:14:39.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.898+0000 7fb49be00640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb48006b4d0 0x7fb48006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:39.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.898+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb49c0a9fc0 con 0x7fb4a4142330 2024-04-25T10:14:39.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.898+0000 7fb4a3400640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb48006b4d0 0x7fb48006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:39.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.899+0000 7fb4a3400640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb48006b4d0 0x7fb48006d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb494009980 tx=0x7fb494008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:39.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:39.903+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb49c043060 con 0x7fb4a4142330 2024-04-25T10:14:40.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.139+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4a40d0510 con 0x7fb4a4142330 2024-04-25T10:14:40.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.140+0000 7fb49be00640 1 -- 172.21.15.165:0/1491387382 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fb49c078c10 con 0x7fb4a4142330 2024-04-25T10:14:40.144 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:40.144 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:40.144 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb48006b4d0 msgr2=0x7fb48006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb48006b4d0 0x7fb48006d990 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb494009980 tx=0x7fb494008040 comp rx=0 tx=0).stop 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 msgr2=0x7fb4a41427b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 0x7fb4a41427b0 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7fb49c02a920 tx=0x7fb49c0014a0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 shutdown_connections 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb48006b4d0 0x7fb48006d990 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4142330 0x7fb4a41427b0 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40b3010 0x7fb4a415b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 --2- 172.21.15.165:0/1491387382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4880051c0 0x7fb4a41f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 >> 172.21.15.165:0/1491387382 conn(0x7fb4a406abe0 msgr2=0x7fb4a415c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 shutdown_connections 2024-04-25T10:14:40.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.143+0000 7fb4aa800640 1 -- 172.21.15.165:0/1491387382 wait complete. 2024-04-25T10:14:40.164 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:40.165 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:40.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.364+0000 7f34d6600640 1 Processor -- start 2024-04-25T10:14:40.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.365+0000 7f34d6600640 1 -- start start 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.366+0000 7f34d6600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f34d0071700 0x7f34d00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.366+0000 7f34d6600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34d00cd860 0x7f34d015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.366+0000 7f34d6600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d015bf30 con 0x7f34d0144440 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.366+0000 7f34d6600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d015c860 con 0x7f34d00cd860 2024-04-25T10:14:40.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.366+0000 7f34d6600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d015c210 con 0x7f34d0071700 2024-04-25T10:14:40.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34d00cd860 0x7f34d015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f34d00cd860 0x7f34d015ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37824/0 (socket says 172.21.15.165:37824) 2024-04-25T10:14:40.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 -- 172.21.15.165:0/1994188173 learned_addr learned my addr 172.21.15.165:0/1994188173 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34c7e00640 1 --2- 172.21.15.165:0/1994188173 >> v2:172.21.15.172:3300/0 conn(0x7f34d0071700 0x7f34d00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 -- 172.21.15.165:0/1994188173 >> v2:172.21.15.172:3300/0 conn(0x7f34d0071700 msgr2=0x7f34d00cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 --2- 172.21.15.165:0/1994188173 >> v2:172.21.15.172:3300/0 conn(0x7f34d0071700 0x7f34d00cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 -- 172.21.15.165:0/1994188173 >> v1:172.21.15.172:6789/0 conn(0x7f34d0144440 legacy=0x7f34d0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34d5c00640 1 -- 172.21.15.165:0/1994188173 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d015c4f0 con 0x7f34d00cd860 2024-04-25T10:14:40.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.367+0000 7f34c7e00640 1 --2- 172.21.15.165:0/1994188173 >> v2:172.21.15.172:3300/0 conn(0x7f34d0071700 0x7f34d00cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:40.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.368+0000 7f34d5c00640 1 --2- 172.21.15.165:0/1994188173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d00cd860 0x7f34d015ccf0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f34d006b970 tx=0x7f34c802a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e2ced53966fab6c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.369+0000 7f34cfe00640 1 -- 172.21.15.165:0/1994188173 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8036070 con 0x7f34d00cd860 2024-04-25T10:14:40.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.369+0000 7f34cfe00640 1 -- 172.21.15.165:0/1994188173 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c8031600 con 0x7f34d00cd860 2024-04-25T10:14:40.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.369+0000 7f34cfe00640 1 -- 172.21.15.165:0/1994188173 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c80319f0 con 0x7f34d00cd860 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.369+0000 7f34d6600640 1 -- 172.21.15.165:0/1994188173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d00cd860 msgr2=0x7f34d015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.369+0000 7f34d6600640 1 --2- 172.21.15.165:0/1994188173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d00cd860 0x7f34d015ccf0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f34d006b970 tx=0x7f34c802a7c0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.370+0000 7f34d6600640 1 -- 172.21.15.165:0/1994188173 shutdown_connections 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.370+0000 7f34d6600640 1 --2- 172.21.15.165:0/1994188173 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d00cd860 0x7f34d015ccf0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.370+0000 7f34d6600640 1 --2- 172.21.15.165:0/1994188173 >> v2:172.21.15.172:3300/0 conn(0x7f34d0071700 0x7f34d00cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.370+0000 7f34d6600640 1 -- 172.21.15.165:0/1994188173 >> 172.21.15.165:0/1994188173 conn(0x7f34d006abe0 msgr2=0x7f34d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.370+0000 7f34d6600640 1 -- 172.21.15.165:0/1994188173 shutdown_connections 2024-04-25T10:14:40.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.370+0000 7f34d6600640 1 -- 172.21.15.165:0/1994188173 wait complete. 2024-04-25T10:14:40.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.371+0000 7f34d6600640 1 Processor -- start 2024-04-25T10:14:40.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.372+0000 7f34d6600640 1 -- start start 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.372+0000 7f34d6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d0071700 0x7f34d013dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.372+0000 7f34d6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d00cd860 0x7f34d00cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 0x7f34d00c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d00cde70 con 0x7f34d0144440 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d00cdcf0 con 0x7f34d0071700 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d015d230 con 0x7f34d00cd860 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 0x7f34d00c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 0x7f34d00c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37834/0 (socket says 172.21.15.165:37834) 2024-04-25T10:14:40.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 -- 172.21.15.165:0/856234270 learned_addr learned my addr 172.21.15.165:0/856234270 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 -- 172.21.15.165:0/856234270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d00cd860 msgr2=0x7f34d00cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d00cd860 0x7f34d00cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 -- 172.21.15.165:0/856234270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d0071700 msgr2=0x7f34d013dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d0071700 0x7f34d013dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.373+0000 7f34d5c00640 1 -- 172.21.15.165:0/856234270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d01e65b0 con 0x7f34d0144440 2024-04-25T10:14:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.374+0000 7f34d5c00640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 0x7f34d00c8b70 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f34c802a800 tx=0x7f34c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.374+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8036050 con 0x7f34d0144440 2024-04-25T10:14:40.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.374+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c8042070 con 0x7f34d0144440 2024-04-25T10:14:40.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.374+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8009510 con 0x7f34d0144440 2024-04-25T10:14:40.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.374+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d01e2c60 con 0x7f34d0144440 2024-04-25T10:14:40.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.374+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d00cc1a0 con 0x7f34d0144440 2024-04-25T10:14:40.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.375+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d0073990 con 0x7f34d0144440 2024-04-25T10:14:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.376+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34c8004470 con 0x7f34d0144440 2024-04-25T10:14:40.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.378+0000 7f34cd600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34ac06b4d0 0x7f34ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.378+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34c80ab3a0 con 0x7f34d0144440 2024-04-25T10:14:40.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.378+0000 7f34d5200640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34ac06b4d0 0x7f34ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.379+0000 7f34d5200640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34ac06b4d0 0x7f34ac06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f34c0000f60 tx=0x7f34c00079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.382+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34c807a090 con 0x7f34d0144440 2024-04-25T10:14:40.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.651+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f34d0071190 con 0x7f34d0144440 2024-04-25T10:14:40.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.652+0000 7f34cd600640 1 -- 172.21.15.165:0/856234270 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f34c8079990 con 0x7f34d0144440 2024-04-25T10:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:40.657 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":781,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076579851,"bytes_used":2615681024,"bytes_avail":770478432256,"bytes_total":773094113280,"write_bytes_sec":28215439,"read_op_per_sec":0,"write_op_per_sec":29},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:14:36.387123+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34ac06b4d0 msgr2=0x7f34ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34ac06b4d0 0x7f34ac06d990 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f34c0000f60 tx=0x7f34c00079d0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 msgr2=0x7f34d00c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 0x7f34d00c8b70 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f34c802a800 tx=0x7f34c8004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 shutdown_connections 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34ac06b4d0 0x7f34ac06d990 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d0144440 0x7f34d00c8b70 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d00cd860 0x7f34d00cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 --2- 172.21.15.165:0/856234270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d0071700 0x7f34d013dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 >> 172.21.15.165:0/856234270 conn(0x7f34d006abe0 msgr2=0x7f34d00ccd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 shutdown_connections 2024-04-25T10:14:40.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.655+0000 7f34d6600640 1 -- 172.21.15.165:0/856234270 wait complete. 2024-04-25T10:14:40.676 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:14:40.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:40.676 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.8:0 scrub resume 2024-04-25T10:14:40.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.864+0000 7fa4d2c00640 1 Processor -- start 2024-04-25T10:14:40.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.865+0000 7fa4d2c00640 1 -- start start 2024-04-25T10:14:40.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.866+0000 7fa4d2c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 0x7fa4cc0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.866+0000 7fa4d2c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc072280 con 0x7fa4cc0cc4f0 2024-04-25T10:14:40.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.866+0000 7fa4d2c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4cc0714c0 con 0x7fa4cc0ce230 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.866+0000 7fa4d2c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc15bdc0 con 0x7fa4cc15b980 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.867+0000 7fa4cbe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa4cc0cc4f0 0x7fa4cc0cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47270/0 (socket says 172.21.15.165:47270) 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.867+0000 7fa4cbe00640 1 -- 172.21.15.165:0/1372323827 learned_addr learned my addr 172.21.15.165:0/1372323827 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.867+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 0x7fa4cc0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.867+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1595700500 0 0) 0x7fa4cc072280 con 0x7fa4cc0cc4f0 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.867+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa4b0003520 con 0x7fa4cc0cc4f0 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2247589634 0 0) 0x7fa4b0003520 con 0x7fa4cc0cc4f0 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 msgr2=0x7fa4cc0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 0x7fa4cc0cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 >> v1:172.21.15.172:6789/0 conn(0x7fa4cc0ce230 legacy=0x7fa4cc15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0717a0 con 0x7fa4cc0cc4f0 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa4bc001670 con 0x7fa4cc0cc4f0 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 >> v1:172.21.15.165:6789/0 conn(0x7fa4cc0cc4f0 legacy=0x7fa4cc0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 0x7fa4b00039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0003f60 0x7fa4b0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4b00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc15bdc0 con 0x7fa4b0004d40 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000baa0 con 0x7fa4b0003f60 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4b000bca0 con 0x7fa4cc15b980 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4b00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 0x7fa4b00039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.868+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0003f60 0x7fa4b0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.869+0000 7fa4d0c00640 1 -- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 msgr2=0x7fa4b00039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.869+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc15b980 0x7fa4b00039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.869+0000 7fa4d0c00640 1 -- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0003f60 msgr2=0x7fa4b0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.869+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0003f60 0x7fa4b0004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.869+0000 7fa4d0c00640 1 -- 172.21.15.165:0/1372323827 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4b0003520 con 0x7fa4b0004d40 2024-04-25T10:14:40.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.869+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4b0003f60 0x7fa4b0004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4b00051c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009fc0 tx=0x7fa4b0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58c5e3c8ecf9e86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4009c10 con 0x7fa4b0004d40 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4036070 con 0x7fa4b0004d40 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4cb400640 1 -- 172.21.15.165:0/1372323827 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4031880 con 0x7fa4b0004d40 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1372323827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 msgr2=0x7fa4b00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1372323827 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4b00051c0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009fc0 tx=0x7fa4b0005830 comp rx=0 tx=0).stop 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4d0c00640 1 -- 172.21.15.165:0/1372323827 reap_dead start 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.870+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1372323827 shutdown_connections 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.871+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1372323827 >> 172.21.15.165:0/1372323827 conn(0x7fa4cc06abe0 msgr2=0x7fa4cc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.871+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1372323827 shutdown_connections 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.871+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1372323827 wait complete. 2024-04-25T10:14:40.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.872+0000 7fa4d2c00640 1 Processor -- start 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 -- start start 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4cc14d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4cc0c46d0 0x7fa4cc0cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc1e7570 0x7fa4cc1e79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc071d60 con 0x7fa4b0004d40 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc071270 con 0x7fa4cc0c46d0 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.873+0000 7fa4d2c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc06e8a0 con 0x7fa4cc1e7570 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4cc14d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4cc14d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37852/0 (socket says 172.21.15.165:37852) 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 -- 172.21.15.165:0/1537953137 learned_addr learned my addr 172.21.15.165:0/1537953137 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4cc0c46d0 0x7fa4cc0cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 -- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc1e7570 msgr2=0x7fa4cc1e79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc1e7570 0x7fa4cc1e79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc1e7570 0x7fa4cc1e79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 -- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4cc0c46d0 msgr2=0x7fa4cc0cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4cc0c46d0 0x7fa4cc0cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4cbe00640 1 -- 172.21.15.165:0/1537953137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4cc0c4d20 con 0x7fa4b0004d40 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.874+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4cc0c46d0 0x7fa4cc0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.875+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4cc14d6b0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00d800 tx=0x7fa4bc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.875+0000 7fa4c8c00640 1 -- 172.21.15.165:0/1537953137 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc019070 con 0x7fa4b0004d40 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.875+0000 7fa4c8c00640 1 -- 172.21.15.165:0/1537953137 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc0074e0 con 0x7fa4b0004d40 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.875+0000 7fa4c8c00640 1 -- 172.21.15.165:0/1537953137 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0049e0 con 0x7fa4b0004d40 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.875+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4cc0c5000 con 0x7fa4b0004d40 2024-04-25T10:14:40.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.875+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4cc0c55c0 con 0x7fa4b0004d40 2024-04-25T10:14:40.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.878+0000 7fa4c8c00640 1 -- 172.21.15.165:0/1537953137 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4bc01c070 con 0x7fa4b0004d40 2024-04-25T10:14:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.879+0000 7fa4c8c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4a806b4d0 0x7fa4a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.879+0000 7fa4c8c00640 1 -- 172.21.15.165:0/1537953137 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4bc089d40 con 0x7fa4b0004d40 2024-04-25T10:14:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.879+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4a806b4d0 0x7fa4a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.880+0000 7fa4d5ba7740 1 Processor -- start 2024-04-25T10:14:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.880+0000 7fa4d5ba7740 1 -- start start 2024-04-25T10:14:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.880+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4a806b4d0 0x7fa4a806d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc1f7000 tx=0x7fa4b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4cdd4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4cdd2f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdfdaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4cd36c70 con 0x557d4cdfd670 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4ccdfcf0 con 0x557d4ce0f280 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d4cd32d10 con 0x557d4cdd2bb0 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdfdaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdfdaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37860/0 (socket says 172.21.15.165:37860) 2024-04-25T10:14:40.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 -- 172.21.15.165:0/394039074 learned_addr learned my addr 172.21.15.165:0/394039074 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4cdd4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4c3400640 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4cdd2f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 -- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 msgr2=0x557d4cdd4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4cdd4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 -- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 msgr2=0x557d4cdd2f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4cdd2f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.881+0000 7fa4d0c00640 1 -- 172.21.15.165:0/394039074 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4cb157d0 con 0x557d4cdfd670 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.882+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdfdaf0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e200 tx=0x7fa4c402d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec388e58b9343385 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.882+0000 7fa4bbe00640 1 -- 172.21.15.165:0/394039074 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4003b90 con 0x557d4cdfd670 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.882+0000 7fa4bbe00640 1 -- 172.21.15.165:0/394039074 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4036040 con 0x557d4cdfd670 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.882+0000 7fa4bbe00640 1 -- 172.21.15.165:0/394039074 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4004030 con 0x557d4cdfd670 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/394039074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 msgr2=0x557d4cdfdaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdfdaf0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fa4c400e200 tx=0x7fa4c402d000 comp rx=0 tx=0).stop 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/394039074 shutdown_connections 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdfdaf0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4cdd2f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/394039074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4cdd4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/394039074 >> 172.21.15.165:0/394039074 conn(0x557d4cd9aa30 msgr2=0x557d4ce18670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/394039074 shutdown_connections 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.883+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/394039074 wait complete. 2024-04-25T10:14:40.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.884+0000 7fa4d5ba7740 1 -- start start 2024-04-25T10:14:40.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.884+0000 7fa4d5ba7740 1 Processor -- start 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdfd670 0x557d4ce1c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 0x557d4ce1adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4ccd53a0 con 0x557d4ce0f280 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4cdfb1c0 con 0x557d4cdd2bb0 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d4ce28a30 con 0x557d4cdfd670 2024-04-25T10:14:40.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d5ba7740 10 client.0 _ll_register_callbacks cb 0x557d4ccf2b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 0x557d4ce1adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 0x557d4ce1adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37864/0 (socket says 172.21.15.165:37864) 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 -- 172.21.15.165:0/3531178999 learned_addr learned my addr 172.21.15.165:0/3531178999 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 -- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdfd670 msgr2=0x557d4ce1c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdfd670 0x557d4ce1c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 -- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdd2bb0 msgr2=0x557d4cdf9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.885+0000 7fa4d0c00640 1 -- 172.21.15.165:0/3531178999 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d4ca4b3f0 con 0x557d4ce0f280 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 0x557d4ce1adc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009e60 tx=0x7fa4c4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47584afacde52bb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4b8c00640 1 -- 172.21.15.165:0/3531178999 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4c4003e00 con 0x557d4ce0f280 2024-04-25T10:14:40.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4b8c00640 1 -- 172.21.15.165:0/3531178999 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4c4004030 con 0x557d4ce0f280 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4c4036040 con 0x557d4ce0f280 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4b8c00640 1 -- 172.21.15.165:0/3531178999 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4c4058050 con 0x557d4ce0f280 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4d5ba7740 10 client.8934 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x557d4ca0b0f0 con 0x557d4ce0f280 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.886+0000 7fa4b8c00640 1 -- 172.21.15.165:0/3531178999 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4c4036040 con 0x557d4ce0f280 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4b8c00640 1 -- 172.21.15.165:0/3531178999 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4c405b020 con 0x557d4ce0f280 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4d5ba7740 10 client.8934 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4d5ba7740 10 client.8934 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4d5ba7740 20 client.8934 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1c6c0 0x557d4cdff290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4d5ba7740 4 client.8934 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d4cd441b0 con 0x557d4ce1c6c0 2024-04-25T10:14:40.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.887+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1c6c0 0x557d4cdff290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.888+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1c6c0 0x557d4cdff290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695171da9ecf6cef server_cookie=843704555f2112bb in_seq=0 out_seq=0 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.888+0000 7fa4b8c00640 10 client.8934 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4b8c00640 1 -- 172.21.15.165:0/3531178999 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557d4cd441b0 con 0x557d4ce1c6c0 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4b8c00640 10 client.8934 handle_command_reply: tid=0 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 client.8934 shutdown 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 msgr2=0x557d4ce1adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 0x557d4ce1adc0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fa4c4009e60 tx=0x7fa4c4004ab0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 shutdown_connections 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1c6c0 0x557d4cdff290 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4ce0f280 0x557d4ce1adc0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdfd670 0x557d4ce1c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3531178999 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 >> 172.21.15.165:0/3531178999 conn(0x557d4cd9aa30 msgr2=0x557d4cdfa840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4b8c00640 0 client.8934 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 shutdown_connections 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.890+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3531178999 wait complete. 2024-04-25T10:14:40.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.891+0000 7fa4d5ba7740 20 client.8934 trim_cache size 0 max 16384 2024-04-25T10:14:40.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 Processor -- start 2024-04-25T10:14:40.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 -- start start 2024-04-25T10:14:40.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdfd670 0x557d4cdfda70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4ce0f280 0x557d4ce39a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc200b70 con 0x557d4cdd2bb0 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc1ffda0 con 0x557d4cdfd670 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.910+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc1ff2f0 con 0x557d4ce0f280 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37870/0 (socket says 172.21.15.165:37870) 2024-04-25T10:14:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 -- 172.21.15.165:0/3394792111 learned_addr learned my addr 172.21.15.165:0/3394792111 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 -- 172.21.15.165:0/3394792111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4ce0f280 msgr2=0x557d4ce39a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4ce0f280 0x557d4ce39a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 -- 172.21.15.165:0/3394792111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdfd670 msgr2=0x557d4cdfda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdfd670 0x557d4cdfda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 -- 172.21.15.165:0/3394792111 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d4ccda0d0 con 0x557d4cdd2bb0 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.911+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9b20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc058100 tx=0x7fa4bc007030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0d49b5ea8f9ab83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4bb400640 1 -- 172.21.15.165:0/3394792111 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc088150 con 0x557d4cdd2bb0 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4bb400640 1 -- 172.21.15.165:0/3394792111 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc056700 con 0x557d4cdd2bb0 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4bb400640 1 -- 172.21.15.165:0/3394792111 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc054a70 con 0x557d4cdd2bb0 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3394792111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 msgr2=0x557d4cdf9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9b20 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc058100 tx=0x7fa4bc007030 comp rx=0 tx=0).stop 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3394792111 shutdown_connections 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4ce0f280 0x557d4ce39a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4cdfd670 0x557d4cdfda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/3394792111 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdd2bb0 0x557d4cdf9b20 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3394792111 >> 172.21.15.165:0/3394792111 conn(0x557d4cd9aa30 msgr2=0x557d4cdfa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3394792111 shutdown_connections 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.912+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/3394792111 wait complete. 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.913+0000 7fa4d5ba7740 1 -- start start 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.913+0000 7fa4d5ba7740 1 Processor -- start 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4ce89d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdf7550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4ce1b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4cc1ff7a0 con 0x557d4cdfd670 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d4cdcd210 con 0x557d4ce0f280 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557d4ce12da0 con 0x557d4cdd2bb0 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4d5ba7740 10 client.0 _ll_register_callbacks cb 0x7fa4b0004580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4c3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdf7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.914+0000 7fa4c3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdf7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37874/0 (socket says 172.21.15.165:37874) 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 -- 172.21.15.165:0/1085882540 learned_addr learned my addr 172.21.15.165:0/1085882540 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4ce1b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4ce89d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 -- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 msgr2=0x557d4ce89d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4ce89d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 -- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 msgr2=0x557d4ce1b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4ce1b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 -- 172.21.15.165:0/1085882540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557d4cdf8a50 con 0x557d4cdfd670 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4d0c00640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4ce1b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4c3400640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdf7550 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0cd090 tx=0x7fa4b4001e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b6c623ae3ccc0e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.915+0000 7fa4b8c00640 1 -- 172.21.15.165:0/1085882540 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b4025060 con 0x557d4cdfd670 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4b8c00640 1 -- 172.21.15.165:0/1085882540 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa4b40067d0 con 0x557d4cdfd670 2024-04-25T10:14:40.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa4bc056700 con 0x557d4cdfd670 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4b8c00640 1 -- 172.21.15.165:0/1085882540 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa4b402b050 con 0x557d4cdfd670 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 10 client.8940 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x557d4ce2b2e0 con 0x557d4cdfd670 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4b8c00640 1 -- 172.21.15.165:0/1085882540 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc056700 con 0x557d4cdfd670 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4b8c00640 1 -- 172.21.15.165:0/1085882540 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa4b402e020 con 0x557d4cdfd670 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 10 client.8940 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 10 client.8940 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 20 client.8940 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1bf00 0x557d4ce13230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 4 client.8940 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.916+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x557d4cd3e500 con 0x557d4ce1bf00 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.917+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1bf00 0x557d4ce13230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.918+0000 7fa4cbe00640 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1bf00 0x557d4ce13230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2258985b439341d5 server_cookie=f8b8a172600013dc in_seq=0 out_seq=0 2024-04-25T10:14:40.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.918+0000 7fa4b8c00640 10 client.8940 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:40.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4b8c00640 1 -- 172.21.15.165:0/1085882540 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x557d4cd3e500 con 0x557d4ce1bf00 2024-04-25T10:14:40.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4b8c00640 10 client.8940 handle_command_reply: tid=0 2024-04-25T10:14:40.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 client.8940 shutdown 2024-04-25T10:14:40.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 msgr2=0x557d4cdf7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdf7550 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc0cd090 tx=0x7fa4b4001e30 comp rx=0 tx=0).stop 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 shutdown_connections 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x557d4ce1bf00 0x557d4ce13230 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557d4ce0f280 0x557d4ce1b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4b8c00640 0 client.8940 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557d4cdfd670 0x557d4cdf7550 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 --2- 172.21.15.165:0/1085882540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557d4cdd2bb0 0x557d4ce89d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 >> 172.21.15.165:0/1085882540 conn(0x557d4cd9aa30 msgr2=0x557d4ce2abd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 shutdown_connections 2024-04-25T10:14:40.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 1 -- 172.21.15.165:0/1085882540 wait complete. 2024-04-25T10:14:40.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.921+0000 7fa4d5ba7740 20 client.8940 trim_cache size 0 max 16384 2024-04-25T10:14:40.931 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:40.931 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:14:40.931 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:40.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4a806b4d0 msgr2=0x7fa4a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4a806b4d0 0x7fa4a806d990 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc1f7000 tx=0x7fa4b4009040 comp rx=0 tx=0).stop 2024-04-25T10:14:40.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 msgr2=0x7fa4cc14d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:40.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4cc14d6b0 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc00d800 tx=0x7fa4bc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 shutdown_connections 2024-04-25T10:14:40.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4a806b4d0 0x7fa4a806d990 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4cc1e7570 0x7fa4cc1e79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4cc0c46d0 0x7fa4cc0cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 --2- 172.21.15.165:0/1537953137 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4b0004d40 0x7fa4cc14d6b0 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:40.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 >> 172.21.15.165:0/1537953137 conn(0x7fa4cc06abe0 msgr2=0x7fa4cc15d960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:40.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 shutdown_connections 2024-04-25T10:14:40.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:40.922+0000 7fa4d2c00640 1 -- 172.21.15.165:0/1537953137 wait complete. 2024-04-25T10:14:40.943 INFO:teuthology.orchestra.run:Running command with timeout 30 2024-04-25T10:14:40.943 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph tell mds.8:0 scrub status 2024-04-25T10:14:41.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.139+0000 7f5246600640 1 Processor -- start 2024-04-25T10:14:41.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.139+0000 7f5246600640 1 -- start start 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.141+0000 7f5246600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524013d9f0 0x7f524013df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.141+0000 7f5246600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f524006e860 0x7f52400b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.141+0000 7f5246600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f524006d390 con 0x7f52400b0730 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.141+0000 7f5246600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52400b10d0 con 0x7f524006e860 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.141+0000 7f5246600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f524006d670 con 0x7f524013d9f0 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.141+0000 7f5245c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f524006e860 0x7f52400b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f524006e860 0x7f52400b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37882/0 (socket says 172.21.15.165:37882) 2024-04-25T10:14:41.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245c00640 1 -- 172.21.15.165:0/498725038 learned_addr learned my addr 172.21.15.165:0/498725038 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245200640 1 --2- 172.21.15.165:0/498725038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524013d9f0 0x7f524013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245c00640 1 -- 172.21.15.165:0/498725038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524013d9f0 msgr2=0x7f524013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245c00640 1 --2- 172.21.15.165:0/498725038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524013d9f0 0x7f524013df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245c00640 1 -- 172.21.15.165:0/498725038 >> v1:172.21.15.172:6789/0 conn(0x7f52400b0730 legacy=0x7f524006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.142+0000 7f5245c00640 1 -- 172.21.15.165:0/498725038 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524006d950 con 0x7f524006e860 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.143+0000 7f5245c00640 1 --2- 172.21.15.165:0/498725038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524006e860 0x7f52400b0b90 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5238009e50 tx=0x7f523802a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb36d2fa4bd24b33 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.143+0000 7f523f400640 1 -- 172.21.15.165:0/498725038 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238036070 con 0x7f524006e860 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.143+0000 7f523f400640 1 -- 172.21.15.165:0/498725038 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238031600 con 0x7f524006e860 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.143+0000 7f523f400640 1 -- 172.21.15.165:0/498725038 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380319d0 con 0x7f524006e860 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 -- 172.21.15.165:0/498725038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524006e860 msgr2=0x7f52400b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 --2- 172.21.15.165:0/498725038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524006e860 0x7f52400b0b90 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f5238009e50 tx=0x7f523802a5b0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 -- 172.21.15.165:0/498725038 shutdown_connections 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 --2- 172.21.15.165:0/498725038 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524006e860 0x7f52400b0b90 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 --2- 172.21.15.165:0/498725038 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524013d9f0 0x7f524013df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 -- 172.21.15.165:0/498725038 >> 172.21.15.165:0/498725038 conn(0x7f5240061cc0 msgr2=0x7f524006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 -- 172.21.15.165:0/498725038 shutdown_connections 2024-04-25T10:14:41.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.144+0000 7f5246600640 1 -- 172.21.15.165:0/498725038 wait complete. 2024-04-25T10:14:41.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.146+0000 7f5246600640 1 Processor -- start 2024-04-25T10:14:41.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.146+0000 7f5246600640 1 -- start start 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5246600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524006e860 0x7f5240138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5246600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52400b0730 0x7f524013a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5246600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 0x7f524013a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5246600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524013e5f0 con 0x7f524013d9f0 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5246600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524013e470 con 0x7f52400b0730 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5246600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52400ce880 con 0x7f524006e860 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 0x7f524013a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 0x7f524013a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37886/0 (socket says 172.21.15.165:37886) 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 -- 172.21.15.165:0/3250237013 learned_addr learned my addr 172.21.15.165:0/3250237013 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 -- 172.21.15.165:0/3250237013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524006e860 msgr2=0x7f5240138190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524006e860 0x7f5240138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 -- 172.21.15.165:0/3250237013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52400b0730 msgr2=0x7f524013a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52400b0730 0x7f524013a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.147+0000 7f5245c00640 1 -- 172.21.15.165:0/3250237013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52400cd870 con 0x7f524013d9f0 2024-04-25T10:14:41.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.148+0000 7f5245c00640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 0x7f524013a590 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5238009e50 tx=0x7f5238004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.148+0000 7f523cc00640 1 -- 172.21.15.165:0/3250237013 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238036050 con 0x7f524013d9f0 2024-04-25T10:14:41.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.148+0000 7f523cc00640 1 -- 172.21.15.165:0/3250237013 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238042070 con 0x7f524013d9f0 2024-04-25T10:14:41.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.148+0000 7f523cc00640 1 -- 172.21.15.165:0/3250237013 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238007bd0 con 0x7f524013d9f0 2024-04-25T10:14:41.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.148+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52400cdb50 con 0x7f524013d9f0 2024-04-25T10:14:41.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.148+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52400ce110 con 0x7f524013d9f0 2024-04-25T10:14:41.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.150+0000 7f523cc00640 1 -- 172.21.15.165:0/3250237013 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5238047460 con 0x7f524013d9f0 2024-04-25T10:14:41.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.151+0000 7f523cc00640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522006b4d0 0x7f522006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.151+0000 7f523cc00640 1 -- 172.21.15.165:0/3250237013 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f52380ab390 con 0x7f524013d9f0 2024-04-25T10:14:41.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.151+0000 7f5245200640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522006b4d0 0x7f522006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.152+0000 7f524aa46740 1 Processor -- start 2024-04-25T10:14:41.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.152+0000 7f5245200640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522006b4d0 0x7f522006d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f523400a810 tx=0x7f5234008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.152+0000 7f524aa46740 1 -- start start 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4a9d70 0x55f54f4abed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4e6440 0x55f54f4aa150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f524aa46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f524aa46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f54f40dc70 con 0x55f54f4d4830 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f54f3b6cf0 con 0x55f54f4e6440 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f54f409d10 con 0x55f54f4a9d70 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f5245c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f5245c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d4cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37888/0 (socket says 172.21.15.165:37888) 2024-04-25T10:14:41.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.153+0000 7f5245c00640 1 -- 172.21.15.165:0/3576541029 learned_addr learned my addr 172.21.15.165:0/3576541029 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245200640 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4a9d70 0x55f54f4abed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f523fe00640 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4e6440 0x55f54f4aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245c00640 1 -- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4a9d70 msgr2=0x55f54f4abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245c00640 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4a9d70 0x55f54f4abed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245c00640 1 -- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4e6440 msgr2=0x55f54f4aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245c00640 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4e6440 0x55f54f4aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245c00640 1 -- 172.21.15.165:0/3576541029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f54f1ec7d0 con 0x55f54f4d4830 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f523fe00640 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4e6440 0x55f54f4aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5245c00640 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d4cb0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f5238075dd0 tx=0x7f52380792d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ad8bd588bc7954 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5231600640 1 -- 172.21.15.165:0/3576541029 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380a6070 con 0x55f54f4d4830 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5231600640 1 -- 172.21.15.165:0/3576541029 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238037040 con 0x55f54f4d4830 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.154+0000 7f5231600640 1 -- 172.21.15.165:0/3576541029 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523807dac0 con 0x55f54f4d4830 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 -- 172.21.15.165:0/3576541029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 msgr2=0x55f54f4d4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d4cb0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f5238075dd0 tx=0x7f52380792d0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 -- 172.21.15.165:0/3576541029 shutdown_connections 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d4cb0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4e6440 0x55f54f4aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 --2- 172.21.15.165:0/3576541029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4a9d70 0x55f54f4abed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 -- 172.21.15.165:0/3576541029 >> 172.21.15.165:0/3576541029 conn(0x55f54f471a30 msgr2=0x55f54f4ef830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 -- 172.21.15.165:0/3576541029 shutdown_connections 2024-04-25T10:14:41.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.155+0000 7f524aa46740 1 -- 172.21.15.165:0/3576541029 wait complete. 2024-04-25T10:14:41.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.156+0000 7f524aa46740 1 -- start start 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.156+0000 7f524aa46740 1 Processor -- start 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4c42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4d0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f54f3ac3a0 con 0x55f54f4d4830 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f54f4d2380 con 0x55f54f4a9d70 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f54f4d6620 con 0x55f54f4e6440 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f524aa46740 10 client.0 _ll_register_callbacks cb 0x55f54f3c9b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f523fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f523fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37902/0 (socket says 172.21.15.165:37902) 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.157+0000 7f523fe00640 1 -- 172.21.15.165:0/2731735399 learned_addr learned my addr 172.21.15.165:0/2731735399 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f5245c00640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4d0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f5245200640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4c42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f523fe00640 1 -- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 msgr2=0x55f54f4d0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f523fe00640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4d0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f523fe00640 1 -- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 msgr2=0x55f54f4c42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f523fe00640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4c42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f523fe00640 1 -- 172.21.15.165:0/2731735399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f54f1223f0 con 0x55f54f4d4830 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f5245200640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f5245c00640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f523fe00640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d1c80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f522c004500 tx=0x7f522c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81c8ee4ad60751b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f5226000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:41.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.158+0000 7f5226000640 1 -- 172.21.15.165:0/2731735399 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c005230 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f5226000640 1 -- 172.21.15.165:0/2731735399 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f522c0053d0 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5238037040 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f5226000640 1 -- 172.21.15.165:0/2731735399 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f522c01f050 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 10 client.8952 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55f54f0e20f0 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f5226000640 1 -- 172.21.15.165:0/2731735399 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238037040 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f5226000640 1 -- 172.21.15.165:0/2731735399 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f522c022020 con 0x55f54f4d4830 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 10 client.8952 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 10 client.8952 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 20 client.8952 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4d52c0 0x55f54f4d5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 4 client.8952 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.159+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f54f41b1b0 con 0x55f54f4d52c0 2024-04-25T10:14:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.160+0000 7f5245200640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4d52c0 0x55f54f4d5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.161+0000 7f5245200640 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4d52c0 0x55f54f4d5710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccbfe9f964866c server_cookie=49b4dfa55cbba88b in_seq=0 out_seq=0 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.161+0000 7f5226000640 10 client.8952 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.162+0000 7f5226000640 1 -- 172.21.15.165:0/2731735399 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f54f41b1b0 con 0x55f54f4d52c0 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.162+0000 7f5226000640 10 client.8952 handle_command_reply: tid=0 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.162+0000 7f524aa46740 1 client.8952 shutdown 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 msgr2=0x55f54f4d1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d1c80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f522c004500 tx=0x7f522c00d520 comp rx=0 tx=0).stop 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 shutdown_connections 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4d52c0 0x55f54f4d5710 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4d0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4d1c80 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f5226000640 0 client.8952 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 --2- 172.21.15.165:0/2731735399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4c42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 >> 172.21.15.165:0/2731735399 conn(0x55f54f471a30 msgr2=0x55f54f4d2560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 shutdown_connections 2024-04-25T10:14:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 1 -- 172.21.15.165:0/2731735399 wait complete. 2024-04-25T10:14:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.163+0000 7f524aa46740 20 client.8952 trim_cache size 0 max 16384 2024-04-25T10:14:41.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.179+0000 7f524aa46740 1 Processor -- start 2024-04-25T10:14:41.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.179+0000 7f524aa46740 1 -- start start 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4d4830 0x55f54f4d3630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f524aa46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 0x55f54f4ff770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f524aa46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52402002a0 con 0x55f54f4e6440 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401ff400 con 0x55f54f4a9d70 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5240157970 con 0x55f54f4d4830 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f5245c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 0x55f54f4ff770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f5245c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 0x55f54f4ff770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37906/0 (socket says 172.21.15.165:37906) 2024-04-25T10:14:41.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.180+0000 7f5245c00640 1 -- 172.21.15.165:0/653580817 learned_addr learned my addr 172.21.15.165:0/653580817 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5245c00640 1 -- 172.21.15.165:0/653580817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4d4830 msgr2=0x55f54f4d3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5245c00640 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4d4830 0x55f54f4d3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5245c00640 1 -- 172.21.15.165:0/653580817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 msgr2=0x55f54f4d30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5245c00640 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5245c00640 1 -- 172.21.15.165:0/653580817 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f54f4d4c70 con 0x55f54f4e6440 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5245c00640 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 0x55f54f4ff770 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f523808f040 tx=0x7f52380ab000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2162d207b80b3236 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5230c00640 1 -- 172.21.15.165:0/653580817 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5238095070 con 0x55f54f4e6440 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5230c00640 1 -- 172.21.15.165:0/653580817 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238082080 con 0x55f54f4e6440 2024-04-25T10:14:41.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.181+0000 7f5230c00640 1 -- 172.21.15.165:0/653580817 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52380a6070 con 0x55f54f4e6440 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 -- 172.21.15.165:0/653580817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 msgr2=0x55f54f4ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 0x55f54f4ff770 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f523808f040 tx=0x7f52380ab000 comp rx=0 tx=0).stop 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 -- 172.21.15.165:0/653580817 shutdown_connections 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4e6440 0x55f54f4ff770 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4d4830 0x55f54f4d3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 --2- 172.21.15.165:0/653580817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4d30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 -- 172.21.15.165:0/653580817 >> 172.21.15.165:0/653580817 conn(0x55f54f471a30 msgr2=0x55f54f4e9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 -- 172.21.15.165:0/653580817 shutdown_connections 2024-04-25T10:14:41.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.182+0000 7f524aa46740 1 -- 172.21.15.165:0/653580817 wait complete. 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.183+0000 7f524aa46740 1 -- start start 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.183+0000 7f524aa46740 1 Processor -- start 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4ff720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4ce940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4a8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f54f4a7030 con 0x55f54f4d4830 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52401fef20 con 0x55f54f4a9d70 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f54f4f2290 con 0x55f54f4e6440 2024-04-25T10:14:41.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f524aa46740 10 client.0 _ll_register_callbacks cb 0x7f522c007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f523fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f523fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4ce940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37922/0 (socket says 172.21.15.165:37922) 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.184+0000 7f523fe00640 1 -- 172.21.15.165:0/1803751414 learned_addr learned my addr 172.21.15.165:0/1803751414 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f5245200640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4ff720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f523fe00640 1 -- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 msgr2=0x55f54f4a8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f523fe00640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4a8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f523fe00640 1 -- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 msgr2=0x55f54f4ff720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f523fe00640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4ff720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f523fe00640 1 -- 172.21.15.165:0/1803751414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f54f4ab7f0 con 0x55f54f4d4830 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f5245200640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f523fe00640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4ce940 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f522c00d560 tx=0x7f522c0037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba27804c0614c61b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f5226000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f5226000640 1 -- 172.21.15.165:0/1803751414 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c029020 con 0x55f54f4d4830 2024-04-25T10:14:41.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f5226000640 1 -- 172.21.15.165:0/1803751414 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f522c007440 con 0x55f54f4d4830 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.185+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5238082080 con 0x55f54f4d4830 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f5226000640 1 -- 172.21.15.165:0/1803751414 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=82) v2 ==== 24+0+0 (secure 0 0 0) 0x7f522c007660 con 0x55f54f4d4830 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 10 client.8958 fetch_fsmap learned FSMap version 82 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=82,monmap=2+}) v3 -- 0x55f54f5095a0 con 0x55f54f4d4830 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f5226000640 1 -- 172.21.15.165:0/1803751414 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5238082080 con 0x55f54f4d4830 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f5226000640 1 -- 172.21.15.165:0/1803751414 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 82) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f522c022040 con 0x55f54f4d4830 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 10 client.8958 fetch_fsmap finished waiting for FSMap version 82 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 10 client.8958 resolve_mds: resolved 8:0 to role '8:0' aka daemon mds.d 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 20 client.8958 populate_metadata read hostname 'smithi165' 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4a0150 0x55f54f4a8a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 4 client.8958 mds_command: new command op to 6136 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:14:41.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.186+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 --> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f54f4a66a0 con 0x55f54f4a0150 2024-04-25T10:14:41.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.187+0000 7f5245200640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4a0150 0x55f54f4a8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f5245200640 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4a0150 0x55f54f4a8a70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14cda9b8c14d1178 server_cookie=69040ad5ecb857d3 in_seq=0 out_seq=0 2024-04-25T10:14:41.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f5226000640 10 client.8958 ms_handle_connect on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:41.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f5226000640 1 -- 172.21.15.165:0/1803751414 <== mds.0 v2:172.21.15.172:6832/3899404305 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55f54f4a66a0 con 0x55f54f4a0150 2024-04-25T10:14:41.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f5226000640 10 client.8958 handle_command_reply: tid=0 2024-04-25T10:14:41.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f524aa46740 1 client.8958 shutdown 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 msgr2=0x55f54f4ce940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.188+0000 7f524aa46740 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4ce940 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f522c00d560 tx=0x7f522c0037d0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 shutdown_connections 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:6832/3899404305,v1:172.21.15.172:6834/3899404305] conn(0x55f54f4a0150 0x55f54f4a8a70 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f54f4e6440 0x55f54f4a8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f54f4d4830 0x55f54f4ce940 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f5226000640 0 client.8958 ms_handle_reset on v2:172.21.15.172:6832/3899404305 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 --2- 172.21.15.165:0/1803751414 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f54f4a9d70 0x55f54f4ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 >> 172.21.15.165:0/1803751414 conn(0x55f54f471a30 msgr2=0x55f54f5121c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 shutdown_connections 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 1 -- 172.21.15.165:0/1803751414 wait complete. 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f524aa46740 20 client.8958 trim_cache size 0 max 16384 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:14:41.196 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:14:41.197 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:14:41.197 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:14:41.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522006b4d0 msgr2=0x7f522006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f5246600640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522006b4d0 0x7f522006d990 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f523400a810 tx=0x7f5234008040 comp rx=0 tx=0).stop 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 msgr2=0x7f524013a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.189+0000 7f5246600640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 0x7f524013a590 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5238009e50 tx=0x7f5238004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 shutdown_connections 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f522006b4d0 0x7f522006d990 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f524013d9f0 0x7f524013a590 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f52400b0730 0x7f524013a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 --2- 172.21.15.165:0/3250237013 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f524006e860 0x7f5240138190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 >> 172.21.15.165:0/3250237013 conn(0x7f5240061cc0 msgr2=0x7f52400711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 shutdown_connections 2024-04-25T10:14:41.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.190+0000 7f5246600640 1 -- 172.21.15.165:0/3250237013 wait complete. 2024-04-25T10:14:41.210 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:14:41.210 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.387+0000 7f56f5600640 1 Processor -- start 2024-04-25T10:14:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.387+0000 7f56f5600640 1 -- start start 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56f5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f56f0144750 0x7f56f006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56f5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f56f00cf030 0x7f56f006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56f5600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56f0144b30 con 0x7f56f006eb10 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56f5600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00712d0 con 0x7f56f0144750 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56f5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56f00715b0 con 0x7f56f00cf030 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f56f0144750 0x7f56f006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f56f0144750 0x7f56f006e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37938/0 (socket says 172.21.15.165:37938) 2024-04-25T10:14:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56eea00640 1 -- 172.21.15.165:0/2323362185 learned_addr learned my addr 172.21.15.165:0/2323362185 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.389+0000 7f56eea00640 1 -- 172.21.15.165:0/2323362185 >> v2:172.21.15.172:3300/0 conn(0x7f56f00cf030 msgr2=0x7f56f006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.390+0000 7f56ef400640 1 --2- 172.21.15.165:0/2323362185 >> v2:172.21.15.172:3300/0 conn(0x7f56f00cf030 0x7f56f006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.390+0000 7f56eea00640 1 --2- 172.21.15.165:0/2323362185 >> v2:172.21.15.172:3300/0 conn(0x7f56f00cf030 0x7f56f006d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.390+0000 7f56eea00640 1 -- 172.21.15.165:0/2323362185 >> v1:172.21.15.172:6789/0 conn(0x7f56f006eb10 legacy=0x7f56f00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.390+0000 7f56eea00640 1 -- 172.21.15.165:0/2323362185 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0071890 con 0x7f56f0144750 2024-04-25T10:14:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.391+0000 7f56eea00640 1 --2- 172.21.15.165:0/2323362185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f0144750 0x7f56f006e5d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f56f006f720 tx=0x7f56e002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fd8141ac6c2104f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.391+0000 7f56ee000640 1 -- 172.21.15.165:0/2323362185 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0037070 con 0x7f56f0144750 2024-04-25T10:14:41.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.391+0000 7f56ee000640 1 -- 172.21.15.165:0/2323362185 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0032440 con 0x7f56f0144750 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.391+0000 7f56ee000640 1 -- 172.21.15.165:0/2323362185 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0032810 con 0x7f56f0144750 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 -- 172.21.15.165:0/2323362185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f0144750 msgr2=0x7f56f006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 --2- 172.21.15.165:0/2323362185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f0144750 0x7f56f006e5d0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f56f006f720 tx=0x7f56e002a230 comp rx=0 tx=0).stop 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 -- 172.21.15.165:0/2323362185 shutdown_connections 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 --2- 172.21.15.165:0/2323362185 >> v2:172.21.15.172:3300/0 conn(0x7f56f00cf030 0x7f56f006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 --2- 172.21.15.165:0/2323362185 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f0144750 0x7f56f006e5d0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 -- 172.21.15.165:0/2323362185 >> 172.21.15.165:0/2323362185 conn(0x7f56f006abe0 msgr2=0x7f56f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 -- 172.21.15.165:0/2323362185 shutdown_connections 2024-04-25T10:14:41.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.392+0000 7f56f5600640 1 -- 172.21.15.165:0/2323362185 wait complete. 2024-04-25T10:14:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.393+0000 7f56f5600640 1 Processor -- start 2024-04-25T10:14:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.394+0000 7f56f5600640 1 -- start start 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.394+0000 7f56f5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 0x7f56f01421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.394+0000 7f56f5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f00cf030 0x7f56f015de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56f5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f0144750 0x7f56f015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56f5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f01455d0 con 0x7f56f006eb10 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56f5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56f0145450 con 0x7f56f00cf030 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56f5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56f015c000 con 0x7f56f0144750 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 0x7f56f01421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 0x7f56f01421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37940/0 (socket says 172.21.15.165:37940) 2024-04-25T10:14:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 -- 172.21.15.165:0/1705546298 learned_addr learned my addr 172.21.15.165:0/1705546298 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 -- 172.21.15.165:0/1705546298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f0144750 msgr2=0x7f56f015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f0144750 0x7f56f015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 -- 172.21.15.165:0/1705546298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f00cf030 msgr2=0x7f56f015de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f00cf030 0x7f56f015de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 -- 172.21.15.165:0/1705546298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f0143780 con 0x7f56f006eb10 2024-04-25T10:14:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.395+0000 7f56eea00640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 0x7f56f01421a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000c00 tx=0x7f56e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.396+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0041020 con 0x7f56f006eb10 2024-04-25T10:14:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.396+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56e0030970 con 0x7f56f006eb10 2024-04-25T10:14:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.396+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56f0142c00 con 0x7f56f006eb10 2024-04-25T10:14:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.396+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0037040 con 0x7f56f006eb10 2024-04-25T10:14:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.396+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f01431c0 con 0x7f56f006eb10 2024-04-25T10:14:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.397+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56f0073990 con 0x7f56f006eb10 2024-04-25T10:14:41.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.398+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f56e0044020 con 0x7f56f006eb10 2024-04-25T10:14:41.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.399+0000 7f56e7400640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.399+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f56e00aad00 con 0x7f56f006eb10 2024-04-25T10:14:41.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.399+0000 7f56e6000640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.400+0000 7f56e6000640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f56f0140650 tx=0x7f56d8006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.404+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f56e004b030 con 0x7f56f006eb10 2024-04-25T10:14:41.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.646+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56f00d0510 con 0x7f56f006eb10 2024-04-25T10:14:41.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.647+0000 7f56e7400640 1 -- 172.21.15.165:0/1705546298 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f56e0079a80 con 0x7f56f006eb10 2024-04-25T10:14:41.651 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:41.651 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:41.651 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:41.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56cc06b4d0 msgr2=0x7f56cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56cc06b4d0 0x7f56cc06d990 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f56f0140650 tx=0x7f56d8006ba0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 msgr2=0x7f56f01421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 0x7f56f01421a0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000c00 tx=0x7f56e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 shutdown_connections 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f56cc06b4d0 0x7f56cc06d990 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56f0144750 0x7f56f015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f56f00cf030 0x7f56f015de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 --2- 172.21.15.165:0/1705546298 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56f006eb10 0x7f56f01421a0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.649+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 >> 172.21.15.165:0/1705546298 conn(0x7f56f006abe0 msgr2=0x7f56f013eda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.650+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 shutdown_connections 2024-04-25T10:14:41.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.650+0000 7f56f5600640 1 -- 172.21.15.165:0/1705546298 wait complete. 2024-04-25T10:14:41.671 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:41.671 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:41.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.862+0000 7f69e7200640 1 Processor -- start 2024-04-25T10:14:41.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.863+0000 7f69e7200640 1 -- start start 2024-04-25T10:14:41.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.864+0000 7f69e7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e815e520 0x7f69e815e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.864+0000 7f69e7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f69e8070ff0 0x7f69e8071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.864+0000 7f69e7200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69e8071950 con 0x7f69e815f040 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.864+0000 7f69e7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69e80aee30 con 0x7f69e8070ff0 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.864+0000 7f69e7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80ae590 con 0x7f69e815e520 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69e6800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f69e8070ff0 0x7f69e8071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69dd400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f69e815f040 0x7f69e80adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47282/0 (socket says 172.21.15.165:47282) 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69dd400640 1 -- 172.21.15.165:0/3943929518 learned_addr learned my addr 172.21.15.165:0/3943929518 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69e5e00640 1 --2- 172.21.15.165:0/3943929518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e815e520 0x7f69e815e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69e6800640 1 -- 172.21.15.165:0/3943929518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e815e520 msgr2=0x7f69e815e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69e6800640 1 --2- 172.21.15.165:0/3943929518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e815e520 0x7f69e815e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69e6800640 1 -- 172.21.15.165:0/3943929518 >> v1:172.21.15.165:6789/0 conn(0x7f69e815f040 legacy=0x7f69e80adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.865+0000 7f69e6800640 1 -- 172.21.15.165:0/3943929518 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e80ae870 con 0x7f69e8070ff0 2024-04-25T10:14:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.866+0000 7f69e6800640 1 --2- 172.21.15.165:0/3943929518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e8070ff0 0x7f69e8071410 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009d90 tx=0x7f69e002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbe382625b7f799a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.867+0000 7f69e5400640 1 -- 172.21.15.165:0/3943929518 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0007740 con 0x7f69e8070ff0 2024-04-25T10:14:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.867+0000 7f69e5400640 1 -- 172.21.15.165:0/3943929518 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e00078e0 con 0x7f69e8070ff0 2024-04-25T10:14:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.867+0000 7f69e5400640 1 -- 172.21.15.165:0/3943929518 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0007cb0 con 0x7f69e8070ff0 2024-04-25T10:14:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.867+0000 7f69e7200640 1 -- 172.21.15.165:0/3943929518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e8070ff0 msgr2=0x7f69e8071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.867+0000 7f69e7200640 1 --2- 172.21.15.165:0/3943929518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e8070ff0 0x7f69e8071410 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009d90 tx=0x7f69e002a660 comp rx=0 tx=0).stop 2024-04-25T10:14:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.868+0000 7f69e7200640 1 -- 172.21.15.165:0/3943929518 shutdown_connections 2024-04-25T10:14:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.868+0000 7f69e7200640 1 --2- 172.21.15.165:0/3943929518 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e8070ff0 0x7f69e8071410 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.868+0000 7f69e7200640 1 --2- 172.21.15.165:0/3943929518 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e815e520 0x7f69e815e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.868+0000 7f69e7200640 1 -- 172.21.15.165:0/3943929518 >> 172.21.15.165:0/3943929518 conn(0x7f69e806abe0 msgr2=0x7f69e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.868+0000 7f69e7200640 1 -- 172.21.15.165:0/3943929518 shutdown_connections 2024-04-25T10:14:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.868+0000 7f69e7200640 1 -- 172.21.15.165:0/3943929518 wait complete. 2024-04-25T10:14:41.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.869+0000 7f69e7200640 1 Processor -- start 2024-04-25T10:14:41.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.869+0000 7f69e7200640 1 -- start start 2024-04-25T10:14:41.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.870+0000 7f69e7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e8070ff0 0x7f69e8157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.870+0000 7f69e7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e815e520 0x7f69e80b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.870+0000 7f69e7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 0x7f69e80b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.870+0000 7f69e7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e80ae2a0 con 0x7f69e815f040 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.870+0000 7f69e7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e806d1f0 con 0x7f69e815e520 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.870+0000 7f69e7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69e815f4a0 con 0x7f69e8070ff0 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 0x7f69e80b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 0x7f69e80b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37962/0 (socket says 172.21.15.165:37962) 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 -- 172.21.15.165:0/1828010147 learned_addr learned my addr 172.21.15.165:0/1828010147 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 -- 172.21.15.165:0/1828010147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e8070ff0 msgr2=0x7f69e8157300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e8070ff0 0x7f69e8157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 -- 172.21.15.165:0/1828010147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e815e520 msgr2=0x7f69e80b1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e815e520 0x7f69e80b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 -- 172.21.15.165:0/1828010147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69e81f7840 con 0x7f69e815f040 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.871+0000 7f69e6800640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 0x7f69e80b1eb0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009eb0 tx=0x7f69e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.872+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e003f070 con 0x7f69e815f040 2024-04-25T10:14:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.872+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69e0031bc0 con 0x7f69e815f040 2024-04-25T10:14:41.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.872+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69e0044460 con 0x7f69e815f040 2024-04-25T10:14:41.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.872+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e80b24e0 con 0x7f69e815f040 2024-04-25T10:14:41.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.872+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e80b2aa0 con 0x7f69e815f040 2024-04-25T10:14:41.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.873+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e8073990 con 0x7f69e815f040 2024-04-25T10:14:41.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.874+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f69e0007790 con 0x7f69e815f040 2024-04-25T10:14:41.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.875+0000 7f69dea00640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69c406b4d0 0x7f69c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:41.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.875+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f69e00abbd0 con 0x7f69e815f040 2024-04-25T10:14:41.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.875+0000 7f69e5e00640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69c406b4d0 0x7f69c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:41.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.876+0000 7f69e5e00640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69c406b4d0 0x7f69c406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f69e81365a0 tx=0x7f69d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:41.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:41.879+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f69e007a820 con 0x7f69e815f040 2024-04-25T10:14:42.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.132+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f69e815db20 con 0x7f69e815f040 2024-04-25T10:14:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.133+0000 7f69dea00640 1 -- 172.21.15.165:0/1828010147 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f69e007a120 con 0x7f69e815f040 2024-04-25T10:14:42.137 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:42.137 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":783,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076579851,"bytes_used":2615681024,"bytes_avail":770478432256,"bytes_total":773094113280,"write_bytes_sec":12982178,"read_op_per_sec":0,"write_op_per_sec":14},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:14:36.387123+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:14:42.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.135+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69c406b4d0 msgr2=0x7f69c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.135+0000 7f69e7200640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69c406b4d0 0x7f69c406d990 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f69e81365a0 tx=0x7f69d8008040 comp rx=0 tx=0).stop 2024-04-25T10:14:42.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.135+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 msgr2=0x7f69e80b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.135+0000 7f69e7200640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 0x7f69e80b1eb0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f69e0009eb0 tx=0x7f69e0004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.135+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 shutdown_connections 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f69c406b4d0 0x7f69c406d990 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f69e815f040 0x7f69e80b1eb0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f69e815e520 0x7f69e80b1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 --2- 172.21.15.165:0/1828010147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f69e8070ff0 0x7f69e8157300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:42.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 >> 172.21.15.165:0/1828010147 conn(0x7f69e806abe0 msgr2=0x7f69e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:42.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 shutdown_connections 2024-04-25T10:14:42.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:42.136+0000 7f69e7200640 1 -- 172.21.15.165:0/1828010147 wait complete. 2024-04-25T10:14:42.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:14:43.158 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:43.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.366+0000 7fe821e00640 1 Processor -- start 2024-04-25T10:14:43.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.366+0000 7fe821e00640 1 -- start start 2024-04-25T10:14:43.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.367+0000 7fe821e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe81c0cfec0 0x7fe81c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.367+0000 7fe821e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe81c0ce3c0 0x7fe81c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.367+0000 7fe821e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c071480 con 0x7fe81c154ee0 2024-04-25T10:14:43.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.367+0000 7fe821e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c0ce830 con 0x7fe81c0ce3c0 2024-04-25T10:14:43.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.367+0000 7fe821e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c072090 con 0x7fe81c0cfec0 2024-04-25T10:14:43.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe81c154ee0 0x7fe81c0cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47286/0 (socket says 172.21.15.165:47286) 2024-04-25T10:14:43.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81aa00640 1 -- 172.21.15.165:0/2699555592 learned_addr learned my addr 172.21.15.165:0/2699555592 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:43.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81b400640 1 --2- 172.21.15.165:0/2699555592 >> v2:172.21.15.165:3300/0 conn(0x7fe81c0ce3c0 0x7fe81c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81a000640 1 --2- 172.21.15.165:0/2699555592 >> v2:172.21.15.172:3300/0 conn(0x7fe81c0cfec0 0x7fe81c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe819600640 1 -- 172.21.15.165:0/2699555592 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644161208 0 0) 0x7fe81c071480 con 0x7fe81c154ee0 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81b400640 1 -- 172.21.15.165:0/2699555592 >> v2:172.21.15.172:3300/0 conn(0x7fe81c0cfec0 msgr2=0x7fe81c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81b400640 1 --2- 172.21.15.165:0/2699555592 >> v2:172.21.15.172:3300/0 conn(0x7fe81c0cfec0 0x7fe81c0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81b400640 1 -- 172.21.15.165:0/2699555592 >> v1:172.21.15.165:6789/0 conn(0x7fe81c154ee0 legacy=0x7fe81c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.368+0000 7fe81b400640 1 -- 172.21.15.165:0/2699555592 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c071760 con 0x7fe81c0ce3c0 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.369+0000 7fe81b400640 1 --2- 172.21.15.165:0/2699555592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c06d6f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0cf560 tx=0x7fe81402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93b2e70a3df38b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.369+0000 7fe819600640 1 -- 172.21.15.165:0/2699555592 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814007740 con 0x7fe81c0ce3c0 2024-04-25T10:14:43.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.369+0000 7fe819600640 1 -- 172.21.15.165:0/2699555592 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8140078e0 con 0x7fe81c0ce3c0 2024-04-25T10:14:43.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.369+0000 7fe819600640 1 -- 172.21.15.165:0/2699555592 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814007cb0 con 0x7fe81c0ce3c0 2024-04-25T10:14:43.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 -- 172.21.15.165:0/2699555592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c0ce3c0 msgr2=0x7fe81c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 --2- 172.21.15.165:0/2699555592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c06d6f0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fe81c0cf560 tx=0x7fe81402a680 comp rx=0 tx=0).stop 2024-04-25T10:14:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 -- 172.21.15.165:0/2699555592 shutdown_connections 2024-04-25T10:14:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 --2- 172.21.15.165:0/2699555592 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c06d6f0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 --2- 172.21.15.165:0/2699555592 >> v2:172.21.15.172:3300/0 conn(0x7fe81c0cfec0 0x7fe81c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 -- 172.21.15.165:0/2699555592 >> 172.21.15.165:0/2699555592 conn(0x7fe81c06abe0 msgr2=0x7fe81c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 -- 172.21.15.165:0/2699555592 shutdown_connections 2024-04-25T10:14:43.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.370+0000 7fe821e00640 1 -- 172.21.15.165:0/2699555592 wait complete. 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.372+0000 7fe821e00640 1 Processor -- start 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.372+0000 7fe821e00640 1 -- start start 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe821e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c144020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe821e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe81c0cfec0 0x7fe81c146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe821e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 0x7fe81c143780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe821e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c071d20 con 0x7fe81c154ee0 2024-04-25T10:14:43.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe821e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c071230 con 0x7fe81c0ce3c0 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe821e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1552c0 con 0x7fe81c0cfec0 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe81b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 0x7fe81c143780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe81b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 0x7fe81c143780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37982/0 (socket says 172.21.15.165:37982) 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.373+0000 7fe81b400640 1 -- 172.21.15.165:0/640141590 learned_addr learned my addr 172.21.15.165:0/640141590 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81aa00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c144020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81b400640 1 -- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe81c0cfec0 msgr2=0x7fe81c146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81a000640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe81c0cfec0 0x7fe81c146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81b400640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe81c0cfec0 0x7fe81c146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81b400640 1 -- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe81c0ce3c0 msgr2=0x7fe81c144020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81b400640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c144020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81b400640 1 -- 172.21.15.165:0/640141590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c1446e0 con 0x7fe81c154ee0 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81aa00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81a000640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe81c0cfec0 0x7fe81c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe81b400640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 0x7fe81c143780 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe81402ac70 tx=0x7fe814004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814040070 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814031c90 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814045680 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.374+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c1449c0 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.375+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c144f80 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.376+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c073990 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.376+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe814039040 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.377+0000 7fe813400640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.378+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8140abd20 con 0x7fe81c154ee0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.378+0000 7fe81aa00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.378+0000 7fe81aa00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe80c000f60 tx=0x7fe80c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.382+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe81407aac0 con 0x7fe81c154ee0 2024-04-25T10:14:43.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.606+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe81c06d360 con 0x7fe81c154ee0 2024-04-25T10:14:43.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.607+0000 7fe813400640 1 -- 172.21.15.165:0/640141590 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fe814043e40 con 0x7fe81c154ee0 2024-04-25T10:14:43.611 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:43.611 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7fc06b500 msgr2=0x7fe7fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fe80c000f60 tx=0x7fe80c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 msgr2=0x7fe81c143780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 0x7fe81c143780 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fe81402ac70 tx=0x7fe814004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 shutdown_connections 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7fc06b500 0x7fe7fc06d9c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe81c154ee0 0x7fe81c143780 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe81c0cfec0 0x7fe81c146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 --2- 172.21.15.165:0/640141590 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe81c0ce3c0 0x7fe81c144020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 >> 172.21.15.165:0/640141590 conn(0x7fe81c06abe0 msgr2=0x7fe81c0cdae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 shutdown_connections 2024-04-25T10:14:43.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.610+0000 7fe821e00640 1 -- 172.21.15.165:0/640141590 wait complete. 2024-04-25T10:14:43.631 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:43.632 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:43.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.817+0000 7f0aabc00640 1 Processor -- start 2024-04-25T10:14:43.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.817+0000 7f0aabc00640 1 -- start start 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aabc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0aa40ce920 0x7f0aa40cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aabc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa4144b30 con 0x7f0aa406d290 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aabc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0aa40712d0 con 0x7f0aa4144750 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aabc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0aa40715b0 con 0x7f0aa40ce920 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aa3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0aa40ce920 0x7f0aa40cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aa3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0aa40ce920 0x7f0aa40cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:37998/0 (socket says 172.21.15.165:37998) 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aa3e00640 1 -- 172.21.15.165:0/189006332 learned_addr learned my addr 172.21.15.165:0/189006332 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.819+0000 7f0aa9c00640 1 --1- 172.21.15.165:0/189006332 >> v1:172.21.15.165:6789/0 conn(0x7f0aa406d290 0x7f0aa406d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47302/0 (socket says 172.21.15.165:47302) 2024-04-25T10:14:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.820+0000 7f0aa3e00640 1 -- 172.21.15.165:0/189006332 >> v1:172.21.15.172:6789/0 conn(0x7f0aa4144750 legacy=0x7f0aa406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.820+0000 7f0aa3e00640 1 -- 172.21.15.165:0/189006332 >> v1:172.21.15.165:6789/0 conn(0x7f0aa406d290 legacy=0x7f0aa406d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.820+0000 7f0aa3e00640 1 -- 172.21.15.165:0/189006332 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4071890 con 0x7f0aa40ce920 2024-04-25T10:14:43.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.821+0000 7f0aa3e00640 1 --2- 172.21.15.165:0/189006332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa40ce920 0x7f0aa40cedb0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009870 tx=0x7f0a9002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2718583cc2326ab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.821+0000 7f0aa3400640 1 -- 172.21.15.165:0/189006332 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90037070 con 0x7f0aa40ce920 2024-04-25T10:14:43.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.821+0000 7f0aa3400640 1 -- 172.21.15.165:0/189006332 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a90032440 con 0x7f0aa40ce920 2024-04-25T10:14:43.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.821+0000 7f0aa3400640 1 -- 172.21.15.165:0/189006332 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a90032810 con 0x7f0aa40ce920 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 -- 172.21.15.165:0/189006332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa40ce920 msgr2=0x7f0aa40cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 --2- 172.21.15.165:0/189006332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa40ce920 0x7f0aa40cedb0 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f0a90009870 tx=0x7f0a9002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 -- 172.21.15.165:0/189006332 shutdown_connections 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 --2- 172.21.15.165:0/189006332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa40ce920 0x7f0aa40cedb0 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 -- 172.21.15.165:0/189006332 >> 172.21.15.165:0/189006332 conn(0x7f0aa406abe0 msgr2=0x7f0aa406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 -- 172.21.15.165:0/189006332 shutdown_connections 2024-04-25T10:14:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.822+0000 7f0aabc00640 1 -- 172.21.15.165:0/189006332 wait complete. 2024-04-25T10:14:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.824+0000 7f0aabc00640 1 Processor -- start 2024-04-25T10:14:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.824+0000 7f0aabc00640 1 -- start start 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aabc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 0x7f0aa40af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aabc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa40ce920 0x7f0aa413d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aabc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa4144750 0x7f0aa40b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aabc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa406f020 con 0x7f0aa406d290 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aabc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa406ece0 con 0x7f0aa40ce920 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aabc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa4145450 con 0x7f0aa4144750 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.825+0000 7f0aa9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 0x7f0aa40af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 0x7f0aa40af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38010/0 (socket says 172.21.15.165:38010) 2024-04-25T10:14:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 -- 172.21.15.165:0/4029862105 learned_addr learned my addr 172.21.15.165:0/4029862105 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa3e00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa40ce920 0x7f0aa413d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9c00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa4144750 0x7f0aa40b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 -- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa4144750 msgr2=0x7f0aa40b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa4144750 0x7f0aa40b38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 -- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa40ce920 msgr2=0x7f0aa413d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa40ce920 0x7f0aa413d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 -- 172.21.15.165:0/4029862105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa40b4480 con 0x7f0aa406d290 2024-04-25T10:14:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.826+0000 7f0aa9200640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 0x7f0aa40af2c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c430 tx=0x7f0a9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.827+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98011790 con 0x7f0aa406d290 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.827+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a98017070 con 0x7f0aa406d290 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.827+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a98011bb0 con 0x7f0aa406d290 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.827+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa40acac0 con 0x7f0aa406d290 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.827+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa40ad080 con 0x7f0aa406d290 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.828+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa40ac8b0 con 0x7f0aa406d290 2024-04-25T10:14:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.829+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a9801c460 con 0x7f0aa406d290 2024-04-25T10:14:43.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.830+0000 7f0aa0c00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8406b4d0 0x7f0a8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.830+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a9808a280 con 0x7f0aa406d290 2024-04-25T10:14:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.830+0000 7f0aa3e00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8406b4d0 0x7f0a8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.831+0000 7f0aa3e00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8406b4d0 0x7f0a8406d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0a90034000 tx=0x7f0a9002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:43.834+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a98058f70 con 0x7f0aa406d290 2024-04-25T10:14:44.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.102+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0aa4073990 con 0x7f0aa406d290 2024-04-25T10:14:44.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.103+0000 7f0aa0c00640 1 -- 172.21.15.165:0/4029862105 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f0a98058870 con 0x7f0aa406d290 2024-04-25T10:14:44.107 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:44.107 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":785,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076647530,"bytes_used":2615812096,"bytes_avail":770478301184,"bytes_total":773094113280,"write_bytes_sec":12988506,"read_op_per_sec":0,"write_op_per_sec":14},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:14:36.387123+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:14:44.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.105+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8406b4d0 msgr2=0x7f0a8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.105+0000 7f0aabc00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8406b4d0 0x7f0a8406d990 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f0a90034000 tx=0x7f0a9002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 msgr2=0x7f0aa40af2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 0x7f0aa40af2c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800c430 tx=0x7f0a9800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 shutdown_connections 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a8406b4d0 0x7f0a8406d990 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0aa4144750 0x7f0aa40b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0aa40ce920 0x7f0aa413d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 --2- 172.21.15.165:0/4029862105 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0aa406d290 0x7f0aa40af2c0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 >> 172.21.15.165:0/4029862105 conn(0x7f0aa406abe0 msgr2=0x7f0aa406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 shutdown_connections 2024-04-25T10:14:44.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:44.106+0000 7f0aabc00640 1 -- 172.21.15.165:0/4029862105 wait complete. 2024-04-25T10:14:44.128 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:14:45.129 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:45.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.339+0000 7f254fe00640 1 Processor -- start 2024-04-25T10:14:45.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.339+0000 7f254fe00640 1 -- start start 2024-04-25T10:14:45.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f25500b84f0 0x7f2550168bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500b6a40 con 0x7f255015d900 2024-04-25T10:14:45.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25500b9470 con 0x7f25500b2980 2024-04-25T10:14:45.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25500b7050 con 0x7f25500b84f0 2024-04-25T10:14:45.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f255015d900 0x7f25500ce4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47304/0 (socket says 172.21.15.165:47304) 2024-04-25T10:14:45.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254ea00640 1 -- 172.21.15.165:0/1144878469 learned_addr learned my addr 172.21.15.165:0/1144878469 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:45.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3264684526 0 0) 0x7f25500b6a40 con 0x7f255015d900 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2534003520 con 0x7f255015d900 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.341+0000 7f254f400640 1 --2- 172.21.15.165:0/1144878469 >> v2:172.21.15.172:3300/0 conn(0x7f25500b84f0 0x7f2550168bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 95793753 0 0) 0x7f2534003520 con 0x7f255015d900 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 >> v2:172.21.15.172:3300/0 conn(0x7f25500b84f0 msgr2=0x7f2550168bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 --2- 172.21.15.165:0/1144878469 >> v2:172.21.15.172:3300/0 conn(0x7f25500b84f0 0x7f2550168bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 >> v1:172.21.15.172:6789/0 conn(0x7f25500b2980 legacy=0x7f2550165490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500b6ce0 con 0x7f255015d900 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2540001690 con 0x7f255015d900 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 >> v1:172.21.15.165:6789/0 conn(0x7f255015d900 legacy=0x7f25500ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25340039b0 0x7f2534003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2534004800 0x7f2534004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253400c0c0 con 0x7f25340051c0 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253400c270 con 0x7f25340039b0 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253400c470 con 0x7f2534004800 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254f400640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254ea00640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25340039b0 0x7f2534003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.342+0000 7f254e000640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2534004800 0x7f2534004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.343+0000 7f254f400640 1 -- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2534004800 msgr2=0x7f2534004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.343+0000 7f254f400640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2534004800 0x7f2534004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.343+0000 7f254f400640 1 -- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25340039b0 msgr2=0x7f2534003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.343+0000 7f254f400640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25340039b0 0x7f2534003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.343+0000 7f254f400640 1 -- 172.21.15.165:0/1144878469 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534003520 con 0x7f25340051c0 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.343+0000 7f254f400640 1 -- 172.21.15.165:0/1144878469 reap_dead start 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.344+0000 7f254f400640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f2548007330 tx=0x7f2534005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9bc13546e449fa2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.344+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548009be0 con 0x7f25340051c0 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.344+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2548003b90 con 0x7f25340051c0 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.344+0000 7f254d600640 1 -- 172.21.15.165:0/1144878469 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2548004020 con 0x7f25340051c0 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.344+0000 7f254fe00640 1 -- 172.21.15.165:0/1144878469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 msgr2=0x7f253400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.344+0000 7f254fe00640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f2548007330 tx=0x7f2534005830 comp rx=0 tx=0).stop 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.345+0000 7f254fe00640 1 -- 172.21.15.165:0/1144878469 shutdown_connections 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.345+0000 7f254fe00640 1 --2- 172.21.15.165:0/1144878469 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f253400b9c0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.345+0000 7f254fe00640 1 -- 172.21.15.165:0/1144878469 >> 172.21.15.165:0/1144878469 conn(0x7f25500619f0 msgr2=0x7f2550061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.345+0000 7f254fe00640 1 -- 172.21.15.165:0/1144878469 shutdown_connections 2024-04-25T10:14:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.345+0000 7f254fe00640 1 -- 172.21.15.165:0/1144878469 wait complete. 2024-04-25T10:14:45.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.346+0000 7f254fe00640 1 Processor -- start 2024-04-25T10:14:45.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.347+0000 7f254fe00640 1 -- start start 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f255015eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255014eaa0 0x7f25500ccab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25500ccff0 0x7f255015ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25500b9090 con 0x7f25340051c0 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f255015dce0 con 0x7f255014eaa0 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25500b9210 con 0x7f25500ccff0 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f255015eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f255015eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38032/0 (socket says 172.21.15.165:38032) 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 -- 172.21.15.165:0/3527141959 learned_addr learned my addr 172.21.15.165:0/3527141959 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 -- 172.21.15.165:0/3527141959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25500ccff0 msgr2=0x7f255015ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25500ccff0 0x7f255015ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 -- 172.21.15.165:0/3527141959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255014eaa0 msgr2=0x7f25500ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255014eaa0 0x7f25500ccab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.348+0000 7f254ea00640 1 -- 172.21.15.165:0/3527141959 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25500cd570 con 0x7f25340051c0 2024-04-25T10:14:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.349+0000 7f254ea00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f255015eac0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f254000e760 tx=0x7f254000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.349+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000c6c0 con 0x7f25340051c0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.349+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2540018070 con 0x7f25340051c0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.349+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f255015dfe0 con 0x7f25340051c0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.349+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254000caa0 con 0x7f25340051c0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.349+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f255015e5a0 con 0x7f25340051c0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.350+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2550073990 con 0x7f25340051c0 2024-04-25T10:14:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.351+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f254000cc40 con 0x7f25340051c0 2024-04-25T10:14:45.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.352+0000 7f2547400640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f253006b500 0x7f253006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.352+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f254008a240 con 0x7f25340051c0 2024-04-25T10:14:45.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.352+0000 7f254e000640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f253006b500 0x7f253006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.353+0000 7f254e000640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f253006b500 0x7f253006d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f25500cba90 tx=0x7f253c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.357+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2540058e00 con 0x7f25340051c0 2024-04-25T10:14:45.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.594+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25500ce950 con 0x7f25340051c0 2024-04-25T10:14:45.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.596+0000 7f2547400640 1 -- 172.21.15.165:0/3527141959 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f2540058700 con 0x7f25340051c0 2024-04-25T10:14:45.600 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:45.600 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:45.600 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:45.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f253006b500 msgr2=0x7f253006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f253006b500 0x7f253006d9c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f25500cba90 tx=0x7f253c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:45.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 msgr2=0x7f255015eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f255015eac0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f254000e760 tx=0x7f254000ec20 comp rx=0 tx=0).stop 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 shutdown_connections 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f253006b500 0x7f253006d9c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25500ccff0 0x7f255015ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f255014eaa0 0x7f25500ccab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 --2- 172.21.15.165:0/3527141959 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25340051c0 0x7f255015eac0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.598+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 >> 172.21.15.165:0/3527141959 conn(0x7f25500619f0 msgr2=0x7f25500b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.599+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 shutdown_connections 2024-04-25T10:14:45.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.599+0000 7f254fe00640 1 -- 172.21.15.165:0/3527141959 wait complete. 2024-04-25T10:14:45.620 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:45.621 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:45.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.814+0000 7f074ca00640 1 Processor -- start 2024-04-25T10:14:45.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.815+0000 7f074ca00640 1 -- start start 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f074ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f074815db90 0x7f07480b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f074ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07481615a0 0x7f07480cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f074ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f07480d04f0 0x7f0748061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f074ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07480d0980 con 0x7f07480d04f0 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f074ca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07480d1f50 con 0x7f074815db90 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f074ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0748161df0 con 0x7f07481615a0 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.816+0000 7f0747e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f07480d04f0 0x7f0748061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f07480d04f0 0x7f0748061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38044/0 (socket says 172.21.15.165:38044) 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 -- 172.21.15.165:0/4183393786 learned_addr learned my addr 172.21.15.165:0/4183393786 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0746a00640 1 --2- 172.21.15.165:0/4183393786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07481615a0 0x7f07480cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747400640 1 --2- 172.21.15.165:0/4183393786 >> v2:172.21.15.172:3300/0 conn(0x7f074815db90 0x7f07480b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 -- 172.21.15.165:0/4183393786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07481615a0 msgr2=0x7f07480cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 --2- 172.21.15.165:0/4183393786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07481615a0 0x7f07480cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 -- 172.21.15.165:0/4183393786 >> v2:172.21.15.172:3300/0 conn(0x7f074815db90 msgr2=0x7f07480b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:45.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 --2- 172.21.15.165:0/4183393786 >> v2:172.21.15.172:3300/0 conn(0x7f074815db90 0x7f07480b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.817+0000 7f0747e00640 1 -- 172.21.15.165:0/4183393786 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07481620d0 con 0x7f07480d04f0 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.818+0000 7f0747e00640 1 --2- 172.21.15.165:0/4183393786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07480d04f0 0x7f0748061600 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f074806ceb0 tx=0x7f074002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3296971a2edab2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.818+0000 7f0746000640 1 -- 172.21.15.165:0/4183393786 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740036070 con 0x7f07480d04f0 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.818+0000 7f0746000640 1 -- 172.21.15.165:0/4183393786 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0740031600 con 0x7f07480d04f0 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.818+0000 7f0746000640 1 -- 172.21.15.165:0/4183393786 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07400319d0 con 0x7f07480d04f0 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 -- 172.21.15.165:0/4183393786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07480d04f0 msgr2=0x7f0748061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 --2- 172.21.15.165:0/4183393786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07480d04f0 0x7f0748061600 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f074806ceb0 tx=0x7f074002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 -- 172.21.15.165:0/4183393786 shutdown_connections 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 --2- 172.21.15.165:0/4183393786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07480d04f0 0x7f0748061600 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 --2- 172.21.15.165:0/4183393786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07481615a0 0x7f07480cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 --2- 172.21.15.165:0/4183393786 >> v2:172.21.15.172:3300/0 conn(0x7f074815db90 0x7f07480b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 -- 172.21.15.165:0/4183393786 >> 172.21.15.165:0/4183393786 conn(0x7f07480b52b0 msgr2=0x7f07480b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:45.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 -- 172.21.15.165:0/4183393786 shutdown_connections 2024-04-25T10:14:45.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.819+0000 7f074ca00640 1 -- 172.21.15.165:0/4183393786 wait complete. 2024-04-25T10:14:45.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.821+0000 7f074ca00640 1 Processor -- start 2024-04-25T10:14:45.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.822+0000 7f074ca00640 1 -- start start 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f074ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07480d04f0 0x7f0748159b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f074ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074815db90 0x7f074814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f074ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 0x7f07480c8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f074ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0748161a20 con 0x7f07481615a0 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f074ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07480d1be0 con 0x7f074815db90 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f074ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0748161ba0 con 0x7f07480d04f0 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f0747e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 0x7f07480c8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f0747e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 0x7f07480c8cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:38054/0 (socket says 172.21.15.165:38054) 2024-04-25T10:14:45.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.823+0000 7f0747e00640 1 -- 172.21.15.165:0/2670152153 learned_addr learned my addr 172.21.15.165:0/2670152153 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:45.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747e00640 1 -- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07480d04f0 msgr2=0x7f0748159b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0746a00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074815db90 0x7f074814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747400640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07480d04f0 0x7f0748159b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747e00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07480d04f0 0x7f0748159b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747e00640 1 -- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074815db90 msgr2=0x7f074814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747e00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074815db90 0x7f074814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747e00640 1 -- 172.21.15.165:0/2670152153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074815c830 con 0x7f07481615a0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0746a00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074815db90 0x7f074814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f0747e00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 0x7f07480c8cc0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0740034040 tx=0x7f0740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740036050 con 0x7f07481615a0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0740042070 con 0x7f07481615a0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0740009510 con 0x7f07481615a0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.824+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f074815cb10 con 0x7f07481615a0 2024-04-25T10:14:45.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.825+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07480cbe60 con 0x7f07481615a0 2024-04-25T10:14:45.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.826+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07480cbc50 con 0x7f07481615a0 2024-04-25T10:14:45.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.827+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0740007bf0 con 0x7f07481615a0 2024-04-25T10:14:45.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.828+0000 7f073f400640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f072806b4d0 0x7f072806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:45.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.828+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07400ab3e0 con 0x7f07481615a0 2024-04-25T10:14:45.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.828+0000 7f0747400640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f072806b4d0 0x7f072806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:45.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.829+0000 7f0747400640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f072806b4d0 0x7f072806d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f073800b7e0 tx=0x7f0738008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:45.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:45.832+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0740079fa0 con 0x7f07481615a0 2024-04-25T10:14:46.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.101+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0748073990 con 0x7f07481615a0 2024-04-25T10:14:46.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.101+0000 7f073f400640 1 -- 172.21.15.165:0/2670152153 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1594 (secure 0 0 0) 0x7f07400798a0 con 0x7f07481615a0 2024-04-25T10:14:46.105 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:46.105 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":787,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076647530,"bytes_used":2615812096,"bytes_avail":770478301184,"bytes_total":773094113280,"write_bytes_sec":24567,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":30,"modified":"2024-04-25T10:14:36.387123+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}}}}}},"progress_events":{}} 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f072806b4d0 msgr2=0x7f072806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f072806b4d0 0x7f072806d990 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f073800b7e0 tx=0x7f0738008040 comp rx=0 tx=0).stop 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 msgr2=0x7f07480c8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 0x7f07480c8cc0 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f0740034040 tx=0x7f0740004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 shutdown_connections 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f072806b4d0 0x7f072806d990 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07481615a0 0x7f07480c8cc0 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f074815db90 0x7f074814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 --2- 172.21.15.165:0/2670152153 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07480d04f0 0x7f0748159b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 >> 172.21.15.165:0/2670152153 conn(0x7f07480b52b0 msgr2=0x7f07480b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 shutdown_connections 2024-04-25T10:14:46.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:46.104+0000 7f074ca00640 1 -- 172.21.15.165:0/2670152153 wait complete. 2024-04-25T10:14:46.126 INFO:tasks.cephfs.filesystem.ceph_manager:{'d': 'paused'} 2024-04-25T10:14:47.127 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:47.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.332+0000 7f0a8d800640 1 Processor -- start 2024-04-25T10:14:47.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.333+0000 7f0a8d800640 1 -- start start 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a8d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 0x7f0a8806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a8d800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0a8806eb10 0x7f0a880ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a8d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a88144b30 con 0x7f0a880cee60 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a8d800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a880712d0 con 0x7f0a8806eb10 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a8d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880715b0 con 0x7f0a88144750 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a87400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0a880cee60 0x7f0a8815f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52906/0 (socket says 172.21.15.165:52906) 2024-04-25T10:14:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.334+0000 7f0a87400640 1 -- 172.21.15.165:0/3391851370 learned_addr learned my addr 172.21.15.165:0/3391851370 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86a00640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 0x7f0a8806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3391851370 >> v2:172.21.15.172:3300/0 conn(0x7f0a8806eb10 0x7f0a880ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940096577 0 0) 0x7f0a88144b30 con 0x7f0a880cee60 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a6c003520 con 0x7f0a880cee60 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2154543654 0 0) 0x7f0a6c003520 con 0x7f0a880cee60 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 msgr2=0x7f0a8806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 0x7f0a8806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 >> v2:172.21.15.172:3300/0 conn(0x7f0a8806eb10 msgr2=0x7f0a880ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 --2- 172.21.15.165:0/3391851370 >> v2:172.21.15.172:3300/0 conn(0x7f0a8806eb10 0x7f0a880ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a88071890 con 0x7f0a880cee60 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.335+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f0a800019f0 con 0x7f0a880cee60 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 >> v1:172.21.15.165:6789/0 conn(0x7f0a880cee60 legacy=0x7f0a8815f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a6c0039b0 0x7f0a6c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 0x7f0a6c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880712d0 con 0x7f0a6c004800 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a880715b0 con 0x7f0a6c0039b0 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a6c00b9c0 con 0x7f0a88144750 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a86a00640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a6c0039b0 0x7f0a6c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 msgr2=0x7f0a6c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a88144750 0x7f0a6c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a6c0039b0 msgr2=0x7f0a6c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a6c0039b0 0x7f0a6c003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.336+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3391851370 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a6c003520 con 0x7f0a6c004800 2024-04-25T10:14:47.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.337+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009a90 tx=0x7f0a6c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a27cb27b47633c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.338+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70036070 con 0x7f0a6c004800 2024-04-25T10:14:47.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.338+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a70031400 con 0x7f0a6c004800 2024-04-25T10:14:47.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.338+0000 7f0a86000640 1 -- 172.21.15.165:0/3391851370 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70009690 con 0x7f0a6c004800 2024-04-25T10:14:47.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.338+0000 7f0a8d800640 1 -- 172.21.15.165:0/3391851370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a6c004800 msgr2=0x7f0a6c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.338+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3391851370 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a6c004800 0x7f0a6c004c80 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7f0a70009a90 tx=0x7f0a6c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.338+0000 7f0a87400640 1 -- 172.21.15.165:0/3391851370 reap_dead start 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.339+0000 7f0a8d800640 1 -- 172.21.15.165:0/3391851370 shutdown_connections 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.339+0000 7f0a8d800640 1 -- 172.21.15.165:0/3391851370 >> 172.21.15.165:0/3391851370 conn(0x7f0a8806abe0 msgr2=0x7f0a8806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.339+0000 7f0a8d800640 1 -- 172.21.15.165:0/3391851370 shutdown_connections 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.339+0000 7f0a8d800640 1 -- 172.21.15.165:0/3391851370 wait complete. 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.340+0000 7f0a8d800640 1 Processor -- start 2024-04-25T10:14:47.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.340+0000 7f0a8d800640 1 -- start start 2024-04-25T10:14:47.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.341+0000 7f0a8d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6c004800 0x7f0a8815da90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.341+0000 7f0a8d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 0x7f0a880a8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.341+0000 7f0a8d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815f3a0 0x7f0a880a9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.341+0000 7f0a8d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a88145450 con 0x7f0a8815ef20 2024-04-25T10:14:47.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.341+0000 7f0a8d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a8806f020 con 0x7f0a8815f3a0 2024-04-25T10:14:47.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.341+0000 7f0a8d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a881455d0 con 0x7f0a6c004800 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 0x7f0a880a8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 0x7f0a880a8dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50376/0 (socket says 172.21.15.165:50376) 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3625039046 learned_addr learned my addr 172.21.15.165:0/3625039046 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a86a00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6c004800 0x7f0a8815da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a87400640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815f3a0 0x7f0a880a9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6c004800 msgr2=0x7f0a8815da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6c004800 0x7f0a8815da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815f3a0 msgr2=0x7f0a880a9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815f3a0 0x7f0a880a9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 -- 172.21.15.165:0/3625039046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a8815f970 con 0x7f0a8815ef20 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.342+0000 7f0a7fe00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 0x7f0a880a8dd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0a7002abb0 tx=0x7f0a70004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.343+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70040020 con 0x7f0a8815ef20 2024-04-25T10:14:47.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.343+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a70003950 con 0x7f0a8815ef20 2024-04-25T10:14:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.343+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a8815fc50 con 0x7f0a8815ef20 2024-04-25T10:14:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.343+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a70036040 con 0x7f0a8815ef20 2024-04-25T10:14:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.343+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a88160210 con 0x7f0a8815ef20 2024-04-25T10:14:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.344+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a88073990 con 0x7f0a8815ef20 2024-04-25T10:14:47.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.345+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0a70043020 con 0x7f0a8815ef20 2024-04-25T10:14:47.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.346+0000 7f0a7ea00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a6406b4d0 0x7f0a6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.346+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0a700aa8b0 con 0x7f0a8815ef20 2024-04-25T10:14:47.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.347+0000 7f0a86a00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a6406b4d0 0x7f0a6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.347+0000 7f0a86a00640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a6406b4d0 0x7f0a6406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009830 tx=0x7f0a78008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.351+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0a7004a030 con 0x7f0a8815ef20 2024-04-25T10:14:47.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.568+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a8813ed20 con 0x7f0a8815ef20 2024-04-25T10:14:47.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.570+0000 7f0a7ea00640 1 -- 172.21.15.165:0/3625039046 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f0a70079540 con 0x7f0a8815ef20 2024-04-25T10:14:47.573 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:47.574 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:47.574 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a6406b4d0 msgr2=0x7f0a6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a6406b4d0 0x7f0a6406d990 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f0a78009830 tx=0x7f0a78008040 comp rx=0 tx=0).stop 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 msgr2=0x7f0a880a8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 0x7f0a880a8dd0 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f0a7002abb0 tx=0x7f0a70004730 comp rx=0 tx=0).stop 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 shutdown_connections 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0a6406b4d0 0x7f0a6406d990 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0a8815f3a0 0x7f0a880a9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0a8815ef20 0x7f0a880a8dd0 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 --2- 172.21.15.165:0/3625039046 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0a6c004800 0x7f0a8815da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 >> 172.21.15.165:0/3625039046 conn(0x7f0a8806abe0 msgr2=0x7f0a880cca20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 shutdown_connections 2024-04-25T10:14:47.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.572+0000 7f0a8d800640 1 -- 172.21.15.165:0/3625039046 wait complete. 2024-04-25T10:14:47.594 DEBUG:tasks.cephfs.test_scrub_checks:current active=['d'] 2024-04-25T10:14:47.595 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:14:47.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.780+0000 7fa302e00640 1 Processor -- start 2024-04-25T10:14:47.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.780+0000 7fa302e00640 1 -- start start 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.782+0000 7fa302e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa2fc0d06c0 0x7fa2fc168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.782+0000 7fa302e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc0d33f0 con 0x7fa2fc15db90 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.782+0000 7fa302e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc161df0 con 0x7fa2fc0b93c0 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.782+0000 7fa302e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc0d1f50 con 0x7fa2fc0d06c0 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.782+0000 7fa301a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa2fc15db90 0x7fa2fc1615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52922/0 (socket says 172.21.15.165:52922) 2024-04-25T10:14:47.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.782+0000 7fa301a00640 1 -- 172.21.15.165:0/700708510 learned_addr learned my addr 172.21.15.165:0/700708510 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:47.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa302400640 1 --2- 172.21.15.165:0/700708510 >> v2:172.21.15.172:3300/0 conn(0x7fa2fc0d06c0 0x7fa2fc168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636395129 0 0) 0x7fa2fc0d33f0 con 0x7fa2fc15db90 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e4003520 con 0x7fa2fc15db90 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3759950740 0 0) 0x7fa2e4003520 con 0x7fa2fc15db90 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 >> v2:172.21.15.172:3300/0 conn(0x7fa2fc0d06c0 msgr2=0x7fa2fc168930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 --2- 172.21.15.165:0/700708510 >> v2:172.21.15.172:3300/0 conn(0x7fa2fc0d06c0 0x7fa2fc168930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 >> v1:172.21.15.172:6789/0 conn(0x7fa2fc0b93c0 legacy=0x7fa2fc0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc1620d0 con 0x7fa2fc15db90 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa2f0001670 con 0x7fa2fc15db90 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 >> v1:172.21.15.165:6789/0 conn(0x7fa2fc15db90 legacy=0x7fa2fc1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0d1f50 con 0x7fa2e4004800 2024-04-25T10:14:47.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e400c110 con 0x7fa2e40051c0 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e400c310 con 0x7fa2e40039b0 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.783+0000 7fa301000640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa302400640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301a00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301000640 1 -- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2e40039b0 msgr2=0x7fa2e4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301000640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301000640 1 -- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2e40051c0 msgr2=0x7fa2e400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301000640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2e40051c0 0x7fa2e400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301000640 1 -- 172.21.15.165:0/700708510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e4003520 con 0x7fa2e4004800 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa302400640 1 -- 172.21.15.165:0/700708510 reap_dead start 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.784+0000 7fa301a00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2e40039b0 0x7fa2e4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.785+0000 7fa301000640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8001750 tx=0x7fa2e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41a8c8c059859172 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.785+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8001370 con 0x7fa2e4004800 2024-04-25T10:14:47.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.785+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e80052d0 con 0x7fa2e4004800 2024-04-25T10:14:47.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.785+0000 7fa2fbe00640 1 -- 172.21.15.165:0/700708510 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e802abc0 con 0x7fa2e4004800 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 -- 172.21.15.165:0/700708510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 msgr2=0x7fa2e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8001750 tx=0x7fa2e4005830 comp rx=0 tx=0).stop 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 -- 172.21.15.165:0/700708510 shutdown_connections 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 --2- 172.21.15.165:0/700708510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2e4004c80 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 -- 172.21.15.165:0/700708510 >> 172.21.15.165:0/700708510 conn(0x7fa2fc0b52b0 msgr2=0x7fa2fc0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 -- 172.21.15.165:0/700708510 shutdown_connections 2024-04-25T10:14:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.786+0000 7fa302e00640 1 -- 172.21.15.165:0/700708510 wait complete. 2024-04-25T10:14:47.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.788+0000 7fa302e00640 1 Processor -- start 2024-04-25T10:14:47.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.788+0000 7fa302e00640 1 -- start start 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa302e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2fc14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa302e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2fc0cb280 0x7fa2fc0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa302e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2fc0ccee0 0x7fa2fc15e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa302e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0b68d0 con 0x7fa2e4004800 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa302e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0d1be0 con 0x7fa2fc0cb280 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa302e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0b6a50 con 0x7fa2fc0ccee0 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2fc14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2fc14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50386/0 (socket says 172.21.15.165:50386) 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.789+0000 7fa301a00640 1 -- 172.21.15.165:0/2044707272 learned_addr learned my addr 172.21.15.165:0/2044707272 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa302400640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2fc0ccee0 0x7fa2fc15e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301a00640 1 -- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2fc0ccee0 msgr2=0x7fa2fc15e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301000640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2fc0cb280 0x7fa2fc0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301a00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2fc0ccee0 0x7fa2fc15e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301a00640 1 -- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2fc0cb280 msgr2=0x7fa2fc0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301a00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2fc0cb280 0x7fa2fc0cc9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301a00640 1 -- 172.21.15.165:0/2044707272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc0cb9f0 con 0x7fa2e4004800 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa301a00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2fc14e970 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d810 tx=0x7fa2f000dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f0019070 con 0x7fa2e4004800 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f00074e0 con 0x7fa2e4004800 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f00049e0 con 0x7fa2e4004800 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc0cbcd0 con 0x7fa2e4004800 2024-04-25T10:14:47.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.790+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc0cb700 con 0x7fa2e4004800 2024-04-25T10:14:47.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.791+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc073990 con 0x7fa2e4004800 2024-04-25T10:14:47.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.792+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa2f001c070 con 0x7fa2e4004800 2024-04-25T10:14:47.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.793+0000 7fa2f9600640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa2dc06b4d0 0x7fa2dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:47.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.793+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa2f0089ea0 con 0x7fa2e4004800 2024-04-25T10:14:47.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.793+0000 7fa301000640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa2dc06b4d0 0x7fa2dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:47.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.794+0000 7fa301000640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa2dc06b4d0 0x7fa2dc06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8032000 tx=0x7fa2e802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:47.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:47.797+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa2f0058af0 con 0x7fa2e4004800 2024-04-25T10:14:48.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.064+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fa2fc15d340 con 0x7fa2e4004800 2024-04-25T10:14:48.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.065+0000 7fa2f9600640 1 -- 172.21.15.165:0/2044707272 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1571 (secure 0 0 0) 0x7fa2f00583f0 con 0x7fa2e4004800 2024-04-25T10:14:48.068 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:48.069 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":789,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":124,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076647530,"bytes_used":2615812096,"bytes_avail":770478301184,"bytes_total":773094113280,"write_bytes_sec":24567,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":82,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"d","status":"up:active","gid":6136}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":31,"modified":"2024-04-25T10:14:46.390466+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa2dc06b4d0 msgr2=0x7fa2dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa2dc06b4d0 0x7fa2dc06d990 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8032000 tx=0x7fa2e802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 msgr2=0x7fa2fc14e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2fc14e970 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7fa2f000d810 tx=0x7fa2f000dcd0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 shutdown_connections 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa2dc06b4d0 0x7fa2dc06d990 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa2fc0ccee0 0x7fa2fc15e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa2fc0cb280 0x7fa2fc0cc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 --2- 172.21.15.165:0/2044707272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2e4004800 0x7fa2fc14e970 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 >> 172.21.15.165:0/2044707272 conn(0x7fa2fc0b52b0 msgr2=0x7fa2fc0d2c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 shutdown_connections 2024-04-25T10:14:48.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.068+0000 7fa302e00640 1 -- 172.21.15.165:0/2044707272 wait complete. 2024-04-25T10:14:48.091 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:14:48.091 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:14:48.091 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:48.204 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:14:48.320 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:14:48.321 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:14:48.321 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:48.378 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:14:48.379 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:48.430 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:48.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.624+0000 7fa7ee000640 1 Processor -- start 2024-04-25T10:14:48.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.624+0000 7fa7ee000640 1 -- start start 2024-04-25T10:14:48.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.625+0000 7fa7ee000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 0x7fa7e80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7ee000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e815bf30 con 0x7fa7e8071700 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7ee000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e815c210 con 0x7fa7e8144440 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7ee000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815c860 con 0x7fa7e815dd00 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7dfe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa7e8071700 0x7fa7e80cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52934/0 (socket says 172.21.15.165:52934) 2024-04-25T10:14:48.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7dfe00640 1 -- 172.21.15.165:0/2576105912 learned_addr learned my addr 172.21.15.165:0/2576105912 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3885928815 0 0) 0x7fa7e815bf30 con 0x7fa7e8071700 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7ed600640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 0x7fa7e80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.626+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7cc003520 con 0x7fa7e8071700 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2850153772 0 0) 0x7fa7cc003520 con 0x7fa7e8071700 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 msgr2=0x7fa7e80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 0x7fa7e80cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 >> v1:172.21.15.172:6789/0 conn(0x7fa7e8144440 legacy=0x7fa7e8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e815c4f0 con 0x7fa7e8071700 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa7d0001670 con 0x7fa7e8071700 2024-04-25T10:14:48.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 >> v1:172.21.15.165:6789/0 conn(0x7fa7e8071700 legacy=0x7fa7e80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7cc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 0x7fa7cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7cc004d40 0x7fa7cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e815c860 con 0x7fa7cc0039e0 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc00bbe0 con 0x7fa7cc004d40 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7cc00bde0 con 0x7fa7e815dd00 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7cc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.627+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 0x7fa7cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.628+0000 7fa7ed600640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7cc004d40 0x7fa7cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.628+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 msgr2=0x7fa7cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.628+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e815dd00 0x7fa7cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.628+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7cc004d40 msgr2=0x7fa7cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:48.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.628+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7cc004d40 0x7fa7cc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.628+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2576105912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7cc003520 con 0x7fa7cc0039e0 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.629+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7cc003e00 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8001750 tx=0x7fa7cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c674801afb3e34dc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.629+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8001370 con 0x7fa7cc0039e0 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.629+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d80052d0 con 0x7fa7cc0039e0 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.629+0000 7fa7e7e00640 1 -- 172.21.15.165:0/2576105912 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d802abe0 con 0x7fa7cc0039e0 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ee000640 1 -- 172.21.15.165:0/2576105912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 msgr2=0x7fa7cc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2576105912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7cc003e00 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8001750 tx=0x7fa7cc005830 comp rx=0 tx=0).stop 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ed600640 1 -- 172.21.15.165:0/2576105912 reap_dead start 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ee000640 1 -- 172.21.15.165:0/2576105912 shutdown_connections 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ee000640 1 -- 172.21.15.165:0/2576105912 >> 172.21.15.165:0/2576105912 conn(0x7fa7e806abe0 msgr2=0x7fa7e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ee000640 1 -- 172.21.15.165:0/2576105912 shutdown_connections 2024-04-25T10:14:48.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.630+0000 7fa7ee000640 1 -- 172.21.15.165:0/2576105912 wait complete. 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.632+0000 7fa7ee000640 1 Processor -- start 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.632+0000 7fa7ee000640 1 -- start start 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ee000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7e81f23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ee000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e81502a0 0x7fa7e813c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ee000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e813c940 0x7fa7e80caa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ee000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80cdbb0 con 0x7fa7cc0039e0 2024-04-25T10:14:48.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ee000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80cda30 con 0x7fa7e81502a0 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ee000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e80cdd30 con 0x7fa7e813c940 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7e81f23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7e81f23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50400/0 (socket says 172.21.15.165:50400) 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2912100914 learned_addr learned my addr 172.21.15.165:0/2912100914 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2912100914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e813c940 msgr2=0x7fa7e80caa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e813c940 0x7fa7e80caa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2912100914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e81502a0 msgr2=0x7fa7e813c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e81502a0 0x7fa7e813c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.633+0000 7fa7ecc00640 1 -- 172.21.15.165:0/2912100914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e81f2910 con 0x7fa7cc0039e0 2024-04-25T10:14:48.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.634+0000 7fa7ecc00640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7e81f23d0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8008fd0 tx=0x7fa7d8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.634+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8042070 con 0x7fa7cc0039e0 2024-04-25T10:14:48.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.634+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d802be50 con 0x7fa7cc0039e0 2024-04-25T10:14:48.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.634+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8037040 con 0x7fa7cc0039e0 2024-04-25T10:14:48.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.634+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e81f2b90 con 0x7fa7cc0039e0 2024-04-25T10:14:48.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.634+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e81f3150 con 0x7fa7cc0039e0 2024-04-25T10:14:48.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.635+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e80c54a0 con 0x7fa7cc0039e0 2024-04-25T10:14:48.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.636+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa7d80052d0 con 0x7fa7cc0039e0 2024-04-25T10:14:48.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.637+0000 7fa7e5600640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:48.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.637+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa7d80a9b30 con 0x7fa7cc0039e0 2024-04-25T10:14:48.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.637+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:48.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.638+0000 7fa7dfe00640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7c406b4d0 0x7fa7c406d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0001eb0 tx=0x7fa7d0001430 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:48.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.642+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa7e81f2910 con 0x7fa7cc0039e0 2024-04-25T10:14:48.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.837+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa7e8073990 con 0x7fa7cc0039e0 2024-04-25T10:14:48.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.840+0000 7fa7e5600640 1 -- 172.21.15.165:0/2912100914 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16988 (secure 0 0 0) 0x7fa7d8078810 con 0x7fa7cc0039e0 2024-04-25T10:14:48.844 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:48.845 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":124,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:54.924428+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7c406b4d0 msgr2=0x7fa7c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7c406b4d0 0x7fa7c406d990 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fa7d0001eb0 tx=0x7fa7d0001430 comp rx=0 tx=0).stop 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 msgr2=0x7fa7e81f23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7e81f23d0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7fa7d8008fd0 tx=0x7fa7d8001120 comp rx=0 tx=0).stop 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 shutdown_connections 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa7c406b4d0 0x7fa7c406d990 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa7e813c940 0x7fa7e80caa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa7e81502a0 0x7fa7e813c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 --2- 172.21.15.165:0/2912100914 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa7cc0039e0 0x7fa7e81f23d0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:48.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 >> 172.21.15.165:0/2912100914 conn(0x7fa7e806abe0 msgr2=0x7fa7e813faf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:48.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 shutdown_connections 2024-04-25T10:14:48.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:48.843+0000 7fa7ee000640 1 -- 172.21.15.165:0/2912100914 wait complete. 2024-04-25T10:14:48.867 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:51.159277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:52.405294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:48.868 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:49.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.065+0000 7f5c42a00640 1 Processor -- start 2024-04-25T10:14:49.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.065+0000 7f5c42a00640 1 -- start start 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.066+0000 7f5c42a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5c3c13d9f0 0x7f5c3c13df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.066+0000 7f5c42a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5c3c0b0730 0x7f5c3c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c42a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3c06d390 con 0x7f5c3c06e150 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c42a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3c06d670 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c42a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3c06d950 con 0x7f5c3c0b0730 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c41600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5c3c13d9f0 0x7f5c3c13df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c42000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5c3c06e150 0x7f5c3c06e560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52950/0 (socket says 172.21.15.165:52950) 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c41600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5c3c13d9f0 0x7f5c3c13df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50404/0 (socket says 172.21.15.165:50404) 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c42000640 1 -- 172.21.15.165:0/1384951726 learned_addr learned my addr 172.21.15.165:0/1384951726 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c40c00640 1 --2- 172.21.15.165:0/1384951726 >> v2:172.21.15.172:3300/0 conn(0x7f5c3c0b0730 0x7f5c3c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c41600640 1 -- 172.21.15.165:0/1384951726 >> v2:172.21.15.172:3300/0 conn(0x7f5c3c0b0730 msgr2=0x7f5c3c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c41600640 1 --2- 172.21.15.165:0/1384951726 >> v2:172.21.15.172:3300/0 conn(0x7f5c3c0b0730 0x7f5c3c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c41600640 1 -- 172.21.15.165:0/1384951726 >> v1:172.21.15.165:6789/0 conn(0x7f5c3c06e150 legacy=0x7f5c3c06e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c41600640 1 -- 172.21.15.165:0/1384951726 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c06dc30 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.067+0000 7f5c3be00640 1 -- 172.21.15.165:0/1384951726 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505471888 0 0) 0x7f5c3c06d390 con 0x7f5c3c06e150 2024-04-25T10:14:49.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.068+0000 7f5c41600640 1 --2- 172.21.15.165:0/1384951726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13df30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c06ef50 tx=0x7f5c3002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7900a4f44ec72f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.069+0000 7f5c3be00640 1 -- 172.21.15.165:0/1384951726 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30037070 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.069+0000 7f5c3be00640 1 -- 172.21.15.165:0/1384951726 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3002add0 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.069+0000 7f5c3be00640 1 -- 172.21.15.165:0/1384951726 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c30032660 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.069+0000 7f5c42a00640 1 -- 172.21.15.165:0/1384951726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 msgr2=0x7f5c3c13df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.069+0000 7f5c42a00640 1 --2- 172.21.15.165:0/1384951726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13df30 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f5c3c06ef50 tx=0x7f5c3002a080 comp rx=0 tx=0).stop 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.070+0000 7f5c42a00640 1 -- 172.21.15.165:0/1384951726 shutdown_connections 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.070+0000 7f5c42a00640 1 --2- 172.21.15.165:0/1384951726 >> v2:172.21.15.172:3300/0 conn(0x7f5c3c0b0730 0x7f5c3c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.070+0000 7f5c42a00640 1 --2- 172.21.15.165:0/1384951726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13df30 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.070+0000 7f5c42a00640 1 -- 172.21.15.165:0/1384951726 >> 172.21.15.165:0/1384951726 conn(0x7f5c3c061cc0 msgr2=0x7f5c3c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.070+0000 7f5c42a00640 1 -- 172.21.15.165:0/1384951726 shutdown_connections 2024-04-25T10:14:49.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.070+0000 7f5c42a00640 1 -- 172.21.15.165:0/1384951726 wait complete. 2024-04-25T10:14:49.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.071+0000 7f5c42a00640 1 Processor -- start 2024-04-25T10:14:49.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.071+0000 7f5c42a00640 1 -- start start 2024-04-25T10:14:49.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.072+0000 7f5c42a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c3c06e150 0x7f5c3c139910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.072+0000 7f5c42a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c3c0b0730 0x7f5c3c0b3640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.072+0000 7f5c42a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13ff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.072+0000 7f5c42a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c0ce880 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.072+0000 7f5c42a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c13e680 con 0x7f5c3c0b0730 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.072+0000 7f5c42a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c0cea00 con 0x7f5c3c06e150 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50418/0 (socket says 172.21.15.165:50418) 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 -- 172.21.15.165:0/2847409116 learned_addr learned my addr 172.21.15.165:0/2847409116 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c41600640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c3c06e150 0x7f5c3c139910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c40c00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c3c0b0730 0x7f5c3c0b3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 -- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c3c06e150 msgr2=0x7f5c3c139910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c3c06e150 0x7f5c3c139910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 -- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c3c0b0730 msgr2=0x7f5c3c0b3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c3c0b0730 0x7f5c3c0b3640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.073+0000 7f5c42000640 1 -- 172.21.15.165:0/2847409116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c0b25a0 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.074+0000 7f5c42000640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13ff70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c7d0 tx=0x7f5c3400cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.074+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c34018070 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.074+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c3400a640 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.074+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c3400aa70 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.074+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c0b2880 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.074+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c0b2e40 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.075+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c073990 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.076+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5c3400ac10 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.077+0000 7f5c39600640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c1c06b4d0 0x7f5c1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.077+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5c34089a70 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.078+0000 7f5c41600640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c1c06b4d0 0x7f5c1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.078+0000 7f5c41600640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c1c06b4d0 0x7f5c1c06d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5c30034000 tx=0x7f5c3002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.082+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5c340586c0 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.276+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5c3c15b780 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.277+0000 7f5c39600640 1 -- 172.21.15.165:0/2847409116 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5c34057fc0 con 0x7f5c3c13d9f0 2024-04-25T10:14:49.281 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c1c06b4d0 msgr2=0x7f5c1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c1c06b4d0 0x7f5c1c06d990 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f5c30034000 tx=0x7f5c3002e000 comp rx=0 tx=0).stop 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 msgr2=0x7f5c3c13ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13ff70 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f5c3400c7d0 tx=0x7f5c3400cc90 comp rx=0 tx=0).stop 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 shutdown_connections 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c1c06b4d0 0x7f5c1c06d990 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5c3c13d9f0 0x7f5c3c13ff70 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5c3c0b0730 0x7f5c3c0b3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 --2- 172.21.15.165:0/2847409116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5c3c06e150 0x7f5c3c139910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 >> 172.21.15.165:0/2847409116 conn(0x7f5c3c061cc0 msgr2=0x7f5c3c13e830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 shutdown_connections 2024-04-25T10:14:49.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.280+0000 7f5c42a00640 1 -- 172.21.15.165:0/2847409116 wait complete. 2024-04-25T10:14:49.303 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.507+0000 7f631b200640 1 Processor -- start 2024-04-25T10:14:49.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.508+0000 7f631b200640 1 -- start start 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.509+0000 7f631b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c13d9f0 0x7f631c13df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.509+0000 7f631b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f631c0b0730 0x7f631c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.509+0000 7f631b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f631c06e150 0x7f631c06e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.509+0000 7f631b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c0b10d0 con 0x7f631c06e150 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.509+0000 7f631b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f631c06d390 con 0x7f631c0b0730 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.509+0000 7f631b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c06d670 con 0x7f631c13d9f0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f631c06e150 0x7f631c06e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f631c06e150 0x7f631c06e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50428/0 (socket says 172.21.15.165:50428) 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 -- 172.21.15.165:0/2334660190 learned_addr learned my addr 172.21.15.165:0/2334660190 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:49.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f6319e00640 1 --2- 172.21.15.165:0/2334660190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c13d9f0 0x7f631c13df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f6319400640 1 --2- 172.21.15.165:0/2334660190 >> v2:172.21.15.172:3300/0 conn(0x7f631c0b0730 0x7f631c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 -- 172.21.15.165:0/2334660190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c13d9f0 msgr2=0x7f631c13df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 --2- 172.21.15.165:0/2334660190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c13d9f0 0x7f631c13df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 -- 172.21.15.165:0/2334660190 >> v2:172.21.15.172:3300/0 conn(0x7f631c0b0730 msgr2=0x7f631c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 --2- 172.21.15.165:0/2334660190 >> v2:172.21.15.172:3300/0 conn(0x7f631c0b0730 0x7f631c0b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.510+0000 7f631a800640 1 -- 172.21.15.165:0/2334660190 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c06d950 con 0x7f631c06e150 2024-04-25T10:14:49.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.511+0000 7f631a800640 1 --2- 172.21.15.165:0/2334660190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c06e150 0x7f631c06e5a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f631c072390 tx=0x7f631402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92a3cb84348003d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f6318a00640 1 -- 172.21.15.165:0/2334660190 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314037070 con 0x7f631c06e150 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f6318a00640 1 -- 172.21.15.165:0/2334660190 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631402ae00 con 0x7f631c06e150 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f6318a00640 1 -- 172.21.15.165:0/2334660190 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314032660 con 0x7f631c06e150 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 -- 172.21.15.165:0/2334660190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c06e150 msgr2=0x7f631c06e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 --2- 172.21.15.165:0/2334660190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c06e150 0x7f631c06e5a0 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f631c072390 tx=0x7f631402a0b0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 -- 172.21.15.165:0/2334660190 shutdown_connections 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 --2- 172.21.15.165:0/2334660190 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c06e150 0x7f631c06e5a0 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 --2- 172.21.15.165:0/2334660190 >> v2:172.21.15.172:3300/0 conn(0x7f631c0b0730 0x7f631c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 --2- 172.21.15.165:0/2334660190 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c13d9f0 0x7f631c13df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.512+0000 7f631b200640 1 -- 172.21.15.165:0/2334660190 >> 172.21.15.165:0/2334660190 conn(0x7f631c061cc0 msgr2=0x7f631c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.513+0000 7f631b200640 1 -- 172.21.15.165:0/2334660190 shutdown_connections 2024-04-25T10:14:49.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.513+0000 7f631b200640 1 -- 172.21.15.165:0/2334660190 wait complete. 2024-04-25T10:14:49.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.514+0000 7f631b200640 1 Processor -- start 2024-04-25T10:14:49.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.514+0000 7f631b200640 1 -- start start 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.515+0000 7f631b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c06e150 0x7f631c15dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.515+0000 7f631b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c0b0730 0x7f631c13ea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.515+0000 7f631b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 0x7f631c13efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.515+0000 7f631b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c13e470 con 0x7f631c13d9f0 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.515+0000 7f631b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f631c06eae0 con 0x7f631c06e150 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.515+0000 7f631b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f631c13e5f0 con 0x7f631c0b0730 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 0x7f631c13efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 0x7f631c13efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50440/0 (socket says 172.21.15.165:50440) 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 -- 172.21.15.165:0/1603714299 learned_addr learned my addr 172.21.15.165:0/1603714299 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f6319e00640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c06e150 0x7f631c15dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f6319400640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c0b0730 0x7f631c13ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 -- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c0b0730 msgr2=0x7f631c13ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c0b0730 0x7f631c13ea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 -- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c06e150 msgr2=0x7f631c15dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c06e150 0x7f631c15dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 -- 172.21.15.165:0/1603714299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f631c0ce600 con 0x7f631c13d9f0 2024-04-25T10:14:49.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.516+0000 7f631a800640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 0x7f631c13efa0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f63140090b0 tx=0x7f63140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.517+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314041070 con 0x7f631c13d9f0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.517+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f631403c460 con 0x7f631c13d9f0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.517+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6314037040 con 0x7f631c13d9f0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.517+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f631c0cc990 con 0x7f631c13d9f0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.517+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f631c0ccf50 con 0x7f631c13d9f0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.518+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f631c073990 con 0x7f631c13d9f0 2024-04-25T10:14:49.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.519+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f631403a070 con 0x7f631c13d9f0 2024-04-25T10:14:49.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.520+0000 7f6312000640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62fc06b500 0x7f62fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.520+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f631402ae00 con 0x7f631c13d9f0 2024-04-25T10:14:49.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.521+0000 7f6319e00640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62fc06b500 0x7f62fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.521+0000 7f6319e00640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62fc06b500 0x7f62fc06d9c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f630c009f30 tx=0x7f630c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.525+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63140790d0 con 0x7f631c13d9f0 2024-04-25T10:14:49.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.717+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f631c0ce090 con 0x7f631c13d9f0 2024-04-25T10:14:49.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.718+0000 7f6312000640 1 -- 172.21.15.165:0/1603714299 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63140789d0 con 0x7f631c13d9f0 2024-04-25T10:14:49.722 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.720+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62fc06b500 msgr2=0x7f62fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.720+0000 7f631b200640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62fc06b500 0x7f62fc06d9c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f630c009f30 tx=0x7f630c00b040 comp rx=0 tx=0).stop 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 msgr2=0x7f631c13efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 0x7f631c13efa0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f63140090b0 tx=0x7f63140046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 shutdown_connections 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62fc06b500 0x7f62fc06d9c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f631c13d9f0 0x7f631c13efa0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f631c0b0730 0x7f631c13ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 --2- 172.21.15.165:0/1603714299 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f631c06e150 0x7f631c15dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 >> 172.21.15.165:0/1603714299 conn(0x7f631c061cc0 msgr2=0x7f631c143ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 shutdown_connections 2024-04-25T10:14:49.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.721+0000 7f631b200640 1 -- 172.21.15.165:0/1603714299 wait complete. 2024-04-25T10:14:49.745 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:49.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.945+0000 7f3846200640 1 Processor -- start 2024-04-25T10:14:49.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.946+0000 7f3846200640 1 -- start start 2024-04-25T10:14:49.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.947+0000 7f3846200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f384015bcc0 0x7f384009a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.947+0000 7f3846200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f38401601e0 0x7f384015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.947+0000 7f3846200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3840099ae0 con 0x7f384015b8e0 2024-04-25T10:14:49.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f3846200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3840099dc0 con 0x7f384015bcc0 2024-04-25T10:14:49.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f3846200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f384009a0a0 con 0x7f38401601e0 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f3844e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f384015b8e0 0x7f38400cfd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52964/0 (socket says 172.21.15.165:52964) 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f383fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f384015bcc0 0x7f384009a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f3844e00640 1 -- 172.21.15.165:0/3109635002 learned_addr learned my addr 172.21.15.165:0/3109635002 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f3845800640 1 --2- 172.21.15.165:0/3109635002 >> v2:172.21.15.172:3300/0 conn(0x7f38401601e0 0x7f384015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f383fe00640 1 -- 172.21.15.165:0/3109635002 >> v2:172.21.15.172:3300/0 conn(0x7f38401601e0 msgr2=0x7f384015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f383fe00640 1 --2- 172.21.15.165:0/3109635002 >> v2:172.21.15.172:3300/0 conn(0x7f38401601e0 0x7f384015f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f383fe00640 1 -- 172.21.15.165:0/3109635002 >> v1:172.21.15.165:6789/0 conn(0x7f384015b8e0 legacy=0x7f38400cfd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f383f400640 1 -- 172.21.15.165:0/3109635002 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168196369 0 0) 0x7f3840099ae0 con 0x7f384015b8e0 2024-04-25T10:14:49.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.948+0000 7f383fe00640 1 -- 172.21.15.165:0/3109635002 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f384009a380 con 0x7f384015bcc0 2024-04-25T10:14:49.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.949+0000 7f3845800640 1 --2- 172.21.15.165:0/3109635002 >> v2:172.21.15.172:3300/0 conn(0x7f38401601e0 0x7f384015f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:49.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.950+0000 7f383fe00640 1 --2- 172.21.15.165:0/3109635002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015bcc0 0x7f384009a8a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f384009b2e0 tx=0x7f382802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c856ab15c29cd2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.950+0000 7f383f400640 1 -- 172.21.15.165:0/3109635002 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828037070 con 0x7f384015bcc0 2024-04-25T10:14:49.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.950+0000 7f383f400640 1 -- 172.21.15.165:0/3109635002 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3828032440 con 0x7f384015bcc0 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.950+0000 7f383f400640 1 -- 172.21.15.165:0/3109635002 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3828032810 con 0x7f384015bcc0 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 -- 172.21.15.165:0/3109635002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015bcc0 msgr2=0x7f384009a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 --2- 172.21.15.165:0/3109635002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015bcc0 0x7f384009a8a0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f384009b2e0 tx=0x7f382802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 -- 172.21.15.165:0/3109635002 shutdown_connections 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 --2- 172.21.15.165:0/3109635002 >> v2:172.21.15.172:3300/0 conn(0x7f38401601e0 0x7f384015f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 --2- 172.21.15.165:0/3109635002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015bcc0 0x7f384009a8a0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 -- 172.21.15.165:0/3109635002 >> 172.21.15.165:0/3109635002 conn(0x7f38400b3250 msgr2=0x7f38400b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 -- 172.21.15.165:0/3109635002 shutdown_connections 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.951+0000 7f3846200640 1 -- 172.21.15.165:0/3109635002 wait complete. 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.953+0000 7f3846200640 1 Processor -- start 2024-04-25T10:14:49.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.953+0000 7f3846200640 1 -- start start 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3846200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 0x7f384006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3846200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f384015bcc0 0x7f38401fac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3846200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38401601e0 0x7f3840155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3846200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400d0710 con 0x7f384015b8e0 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3846200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38400d0590 con 0x7f38401601e0 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3846200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f384015fb10 con 0x7f384015bcc0 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3844e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 0x7f384006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3844e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 0x7f384006f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50456/0 (socket says 172.21.15.165:50456) 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f3844e00640 1 -- 172.21.15.165:0/3281137968 learned_addr learned my addr 172.21.15.165:0/3281137968 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:49.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.954+0000 7f383fe00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f384015bcc0 0x7f38401fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3844e00640 1 -- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f384015bcc0 msgr2=0x7f38401fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3845800640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38401601e0 0x7f3840155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3844e00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f384015bcc0 0x7f38401fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3844e00640 1 -- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38401601e0 msgr2=0x7f3840155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3844e00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38401601e0 0x7f3840155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3844e00640 1 -- 172.21.15.165:0/3281137968 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38401fb680 con 0x7f384015b8e0 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f383fe00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f384015bcc0 0x7f38401fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3845800640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38401601e0 0x7f3840155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f3844e00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 0x7f384006f6f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f383000c4d0 tx=0x7f383000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830018070 con 0x7f384015b8e0 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3830007500 con 0x7f384015b8e0 2024-04-25T10:14:49.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.955+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3830004a20 con 0x7f384015b8e0 2024-04-25T10:14:49.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.956+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38401fb960 con 0x7f384015b8e0 2024-04-25T10:14:49.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.956+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38401f8c60 con 0x7f384015b8e0 2024-04-25T10:14:49.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.957+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38400b4010 con 0x7f384015b8e0 2024-04-25T10:14:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.958+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3830004bc0 con 0x7f384015b8e0 2024-04-25T10:14:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.959+0000 7f383cc00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f381c06b4d0 0x7f381c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.959+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3830089710 con 0x7f384015b8e0 2024-04-25T10:14:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.959+0000 7f383fe00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f381c06b4d0 0x7f381c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.960+0000 7f383fe00640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f381c06b4d0 0x7f381c06d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3828034000 tx=0x7f382802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:49.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:49.964+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3830058400 con 0x7f384015b8e0 2024-04-25T10:14:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.159+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f38400ce830 con 0x7f384015b8e0 2024-04-25T10:14:50.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.160+0000 7f383cc00640 1 -- 172.21.15.165:0/3281137968 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3830057d00 con 0x7f384015b8e0 2024-04-25T10:14:50.164 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.162+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f381c06b4d0 msgr2=0x7f381c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.162+0000 7f3846200640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f381c06b4d0 0x7f381c06d990 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3828034000 tx=0x7f382802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.162+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 msgr2=0x7f384006f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.162+0000 7f3846200640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 0x7f384006f6f0 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f383000c4d0 tx=0x7f383000c990 comp rx=0 tx=0).stop 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 shutdown_connections 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f381c06b4d0 0x7f381c06d990 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f38401601e0 0x7f3840155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f384015bcc0 0x7f38401fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 --2- 172.21.15.165:0/3281137968 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f384015b8e0 0x7f384006f6f0 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 >> 172.21.15.165:0/3281137968 conn(0x7f38400b3250 msgr2=0x7f384015acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 shutdown_connections 2024-04-25T10:14:50.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.163+0000 7f3846200640 1 -- 172.21.15.165:0/3281137968 wait complete. 2024-04-25T10:14:50.186 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:14:50.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.385+0000 7f07ef200640 1 Processor -- start 2024-04-25T10:14:50.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.386+0000 7f07ef200640 1 -- start start 2024-04-25T10:14:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.387+0000 7f07ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07f00c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.387+0000 7f07ef200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f0145520 con 0x7f07f00ceb20 2024-04-25T10:14:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.387+0000 7f07ef200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07f0071270 con 0x7f07f0145140 2024-04-25T10:14:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.387+0000 7f07ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0071550 con 0x7f07f015cfc0 2024-04-25T10:14:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.387+0000 7f07e7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f07f00ceb20 0x7f07f00c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52970/0 (socket says 172.21.15.165:52970) 2024-04-25T10:14:50.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07e7e00640 1 -- 172.21.15.165:0/4056712381 learned_addr learned my addr 172.21.15.165:0/4056712381 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 121119287 0 0) 0x7f07f0145520 con 0x7f07f00ceb20 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ee800640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07f00c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d4003520 con 0x7f07f00ceb20 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3691142733 0 0) 0x7f07d4003520 con 0x7f07f00ceb20 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 msgr2=0x7f07f00c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07f00c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 >> v1:172.21.15.172:6789/0 conn(0x7f07f0145140 legacy=0x7f07f00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f0071830 con 0x7f07f00ceb20 2024-04-25T10:14:50.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f07d8001690 con 0x7f07f00ceb20 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 >> v1:172.21.15.165:6789/0 conn(0x7f07f00ceb20 legacy=0x7f07f00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.388+0000 7f07ed400640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07d40039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ed400640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07d4004010 0x7f07d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ed400640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d4004d40 0x7f07d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0071550 con 0x7f07d4004010 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07d400bc00 con 0x7f07d4004d40 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07d400be00 con 0x7f07f015cfc0 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07e7e00640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07d4004010 0x7f07d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ede00640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07d40039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ee800640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d4004d40 0x7f07d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07e7e00640 1 -- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 msgr2=0x7f07d40039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07e7e00640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07d40039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07e7e00640 1 -- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d4004d40 msgr2=0x7f07d40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:50.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07e7e00640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07d4004d40 0x7f07d40051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07e7e00640 1 -- 172.21.15.165:0/4056712381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d4003520 con 0x7f07d4004010 2024-04-25T10:14:50.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.389+0000 7f07ede00640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07f015cfc0 0x7f07d40039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.390+0000 7f07e7e00640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07d4004010 0x7f07d4004800 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c210 tx=0x7f07d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ed0c5c4bdc5844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8007d10 con 0x7f07d4004010 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8005af0 con 0x7f07d4004010 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ed400640 1 -- 172.21.15.165:0/4056712381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8007070 con 0x7f07d4004010 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ef200640 1 -- 172.21.15.165:0/4056712381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07d4004010 msgr2=0x7f07d4004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ef200640 1 --2- 172.21.15.165:0/4056712381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07d4004010 0x7f07d4004800 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f07d800c210 tx=0x7f07d4005830 comp rx=0 tx=0).stop 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ee800640 1 -- 172.21.15.165:0/4056712381 reap_dead start 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ef200640 1 -- 172.21.15.165:0/4056712381 shutdown_connections 2024-04-25T10:14:50.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ef200640 1 -- 172.21.15.165:0/4056712381 >> 172.21.15.165:0/4056712381 conn(0x7f07f0061cc0 msgr2=0x7f07f006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ef200640 1 -- 172.21.15.165:0/4056712381 shutdown_connections 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.391+0000 7f07ef200640 1 -- 172.21.15.165:0/4056712381 wait complete. 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.393+0000 7f07ef200640 1 Processor -- start 2024-04-25T10:14:50.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.393+0000 7f07ef200640 1 -- start start 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d4004010 0x7f07f01e4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07ef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 0x7f07f0142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07f01e7e10 0x7f07f01432b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07ef200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c5380 con 0x7f07f01e79b0 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c5200 con 0x7f07f01e7e10 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00c5500 con 0x7f07d4004010 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.394+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 0x7f07f0142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 0x7f07f0142d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50478/0 (socket says 172.21.15.165:50478) 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 -- 172.21.15.165:0/3550311909 learned_addr learned my addr 172.21.15.165:0/3550311909 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 -- 172.21.15.165:0/3550311909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d4004010 msgr2=0x7f07f01e4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d4004010 0x7f07f01e4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 -- 172.21.15.165:0/3550311909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07f01e7e10 msgr2=0x7f07f01432b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07f01e7e10 0x7f07f01432b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 -- 172.21.15.165:0/3550311909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f012d1a0 con 0x7f07f01e79b0 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e7e00640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 0x7f07f0142d70 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f07d8001990 tx=0x7f07f0071440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.395+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8001650 con 0x7f07f01e79b0 2024-04-25T10:14:50.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.396+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d8038040 con 0x7f07f01e79b0 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.396+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f012d480 con 0x7f07f01e79b0 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.396+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d8007270 con 0x7f07f01e79b0 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.396+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f012da40 con 0x7f07f01e79b0 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.397+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f0073990 con 0x7f07f01e79b0 2024-04-25T10:14:50.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.398+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f07d8007410 con 0x7f07f01e79b0 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.399+0000 7f07e6000640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f07cc06b4d0 0x7f07cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.399+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f07d80aa870 con 0x7f07f01e79b0 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.399+0000 7f07ede00640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f07cc06b4d0 0x7f07cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:50.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.400+0000 7f07ede00640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f07cc06b4d0 0x7f07cc06d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f07e00016d0 tx=0x7f07e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:50.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.404+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f07d8079430 con 0x7f07f01e79b0 2024-04-25T10:14:50.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.619+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f07f00b2c90 con 0x7f07f01e79b0 2024-04-25T10:14:50.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.620+0000 7f07e6000640 1 -- 172.21.15.165:0/3550311909 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v124) v1 ==== 80+0+0 (secure 0 0 0) 0x7f07d8078d30 con 0x7f07f01e79b0 2024-04-25T10:14:50.624 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f07cc06b4d0 msgr2=0x7f07cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f07cc06b4d0 0x7f07cc06d990 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f07e00016d0 tx=0x7f07e0009040 comp rx=0 tx=0).stop 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 msgr2=0x7f07f0142d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 0x7f07f0142d70 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f07d8001990 tx=0x7f07f0071440 comp rx=0 tx=0).stop 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 shutdown_connections 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f07cc06b4d0 0x7f07cc06d990 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f07f01e7e10 0x7f07f01432b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f07f01e79b0 0x7f07f0142d70 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 --2- 172.21.15.165:0/3550311909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f07d4004010 0x7f07f01e4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 >> 172.21.15.165:0/3550311909 conn(0x7f07f0061cc0 msgr2=0x7f07f00b2500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 shutdown_connections 2024-04-25T10:14:50.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:50.623+0000 7f07ef200640 1 -- 172.21.15.165:0/3550311909 wait complete. 2024-04-25T10:14:50.645 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:14:50.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:50.646 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:14:51.418 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:14:51.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:14:51.418 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:14:51.457 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:14:51.457 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:51.498 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:51.498 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:14:51.499 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:51.543 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:14:51.544 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:14:51.544 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:14:51.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:14:51.593 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:51.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.793+0000 7f94c4800640 1 Processor -- start 2024-04-25T10:14:51.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.793+0000 7f94c4800640 1 -- start start 2024-04-25T10:14:51.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.794+0000 7f94c4800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.794+0000 7f94c4800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f94bc0ce590 0x7f94bc06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.794+0000 7f94c4800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94bc071480 con 0x7f94bc0cfcf0 2024-04-25T10:14:51.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.794+0000 7f94c4800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94bc072090 con 0x7f94bc0ce590 2024-04-25T10:14:51.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.794+0000 7f94c4800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc0d0150 con 0x7f94bc154ee0 2024-04-25T10:14:51.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f94bc0ce590 0x7f94bc06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f94bc0ce590 0x7f94bc06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50480/0 (socket says 172.21.15.165:50480) 2024-04-25T10:14:51.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c1400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f94bc0cfcf0 0x7f94bc0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52978/0 (socket says 172.21.15.165:52978) 2024-04-25T10:14:51.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 -- 172.21.15.165:0/1836380694 learned_addr learned my addr 172.21.15.165:0/1836380694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:51.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 -- 172.21.15.165:0/1836380694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 msgr2=0x7f94bc0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c1e00640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 -- 172.21.15.165:0/1836380694 >> v1:172.21.15.165:6789/0 conn(0x7f94bc0cfcf0 legacy=0x7f94bc0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c2800640 1 -- 172.21.15.165:0/1836380694 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc071760 con 0x7f94bc0ce590 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c0a00640 1 -- 172.21.15.165:0/1836380694 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436018343 0 0) 0x7f94bc071480 con 0x7f94bc0cfcf0 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.795+0000 7f94c1e00640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.796+0000 7f94c2800640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06d6f0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009e50 tx=0x7f94b802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77ea56df94333e5a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c0a00640 1 -- 172.21.15.165:0/1836380694 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8036070 con 0x7f94bc0ce590 2024-04-25T10:14:51.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c0a00640 1 -- 172.21.15.165:0/1836380694 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b8031600 con 0x7f94bc0ce590 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c0a00640 1 -- 172.21.15.165:0/1836380694 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b80319d0 con 0x7f94bc0ce590 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 -- 172.21.15.165:0/1836380694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 msgr2=0x7f94bc06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06d6f0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f94b8009e50 tx=0x7f94b802a710 comp rx=0 tx=0).stop 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 -- 172.21.15.165:0/1836380694 shutdown_connections 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06d6f0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 --2- 172.21.15.165:0/1836380694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 -- 172.21.15.165:0/1836380694 >> 172.21.15.165:0/1836380694 conn(0x7f94bc06abe0 msgr2=0x7f94bc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 -- 172.21.15.165:0/1836380694 shutdown_connections 2024-04-25T10:14:51.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.797+0000 7f94c4800640 1 -- 172.21.15.165:0/1836380694 wait complete. 2024-04-25T10:14:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.799+0000 7f94c4800640 1 Processor -- start 2024-04-25T10:14:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.799+0000 7f94c4800640 1 -- start start 2024-04-25T10:14:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c4800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c4800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94bc0cfcf0 0x7f94bc144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c4800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c4800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc071d20 con 0x7f94bc0ce590 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c4800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc071230 con 0x7f94bc0cfcf0 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c4800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc1552c0 con 0x7f94bc154ee0 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50484/0 (socket says 172.21.15.165:50484) 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.800+0000 7f94c1e00640 1 -- 172.21.15.165:0/1247946483 learned_addr learned my addr 172.21.15.165:0/1247946483 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c2800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c1e00640 1 -- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 msgr2=0x7f94bc145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c1e00640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c1e00640 1 -- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94bc0cfcf0 msgr2=0x7f94bc144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c1e00640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94bc0cfcf0 0x7f94bc144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c1e00640 1 -- 172.21.15.165:0/1247946483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94bc1f3850 con 0x7f94bc0ce590 2024-04-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c1e00640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06dd40 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c7f0 tx=0x7f94b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0015070 con 0x7f94bc0ce590 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b0007820 con 0x7f94bc0ce590 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b0007bd0 con 0x7f94bc0ce590 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.801+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc1f3b30 con 0x7f94bc0ce590 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.802+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc06e280 con 0x7f94bc0ce590 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.803+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc073990 con 0x7f94bc0ce590 2024-04-25T10:14:51.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.804+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f94b0018070 con 0x7f94bc0ce590 2024-04-25T10:14:51.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.804+0000 7f94b6000640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f94a006b4d0 0x7f94a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:51.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.805+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94b008a500 con 0x7f94bc0ce590 2024-04-25T10:14:51.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.805+0000 7f94c1400640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f94a006b4d0 0x7f94a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:51.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.805+0000 7f94c1400640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f94a006b4d0 0x7f94a006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f94ac007fd0 tx=0x7f94ac007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:51.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:51.809+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f94b00591f0 con 0x7f94bc0ce590 2024-04-25T10:14:52.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.040+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94bc06d360 con 0x7f94bc0ce590 2024-04-25T10:14:52.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.041+0000 7f94b6000640 1 -- 172.21.15.165:0/1247946483 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f94b0059e20 con 0x7f94bc0ce590 2024-04-25T10:14:52.045 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:52.045 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:52.045 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f94a006b4d0 msgr2=0x7f94a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f94a006b4d0 0x7f94a006d990 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f94ac007fd0 tx=0x7f94ac007920 comp rx=0 tx=0).stop 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 msgr2=0x7f94bc06dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06dd40 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f94b000c7f0 tx=0x7f94b000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 shutdown_connections 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f94a006b4d0 0x7f94a006d990 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f94bc154ee0 0x7f94bc145120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f94bc0cfcf0 0x7f94bc144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 --2- 172.21.15.165:0/1247946483 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f94bc0ce590 0x7f94bc06dd40 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 >> 172.21.15.165:0/1247946483 conn(0x7f94bc06abe0 msgr2=0x7f94bc15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 shutdown_connections 2024-04-25T10:14:52.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.044+0000 7f94c4800640 1 -- 172.21.15.165:0/1247946483 wait complete. 2024-04-25T10:14:52.066 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:52.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.260+0000 7f42f8000640 1 Processor -- start 2024-04-25T10:14:52.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.260+0000 7f42f8000640 1 -- start start 2024-04-25T10:14:52.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f8000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f42f0144750 0x7f42f006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f8000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 0x7f42f006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f8000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42f0144b30 con 0x7f42f006eb10 2024-04-25T10:14:52.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f8000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42f00712d0 con 0x7f42f0144750 2024-04-25T10:14:52.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f8000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00715b0 con 0x7f42f00cf030 2024-04-25T10:14:52.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f4c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f42f006eb10 0x7f42f00ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52988/0 (socket says 172.21.15.165:52988) 2024-04-25T10:14:52.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f4c00640 1 -- 172.21.15.165:0/956289967 learned_addr learned my addr 172.21.15.165:0/956289967 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f5600640 1 --2- 172.21.15.165:0/956289967 >> v2:172.21.15.172:3300/0 conn(0x7f42f0144750 0x7f42f006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42f6000640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 0x7f42f006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.262+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097281387 0 0) 0x7f42f0144b30 con 0x7f42f006eb10 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d8003520 con 0x7f42f006eb10 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 175397107 0 0) 0x7f42d8003520 con 0x7f42f006eb10 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 msgr2=0x7f42f006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 0x7f42f006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 >> v2:172.21.15.172:3300/0 conn(0x7f42f0144750 msgr2=0x7f42f006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 --2- 172.21.15.165:0/956289967 >> v2:172.21.15.172:3300/0 conn(0x7f42f0144750 0x7f42f006e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f0071890 con 0x7f42f006eb10 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f42dc001670 con 0x7f42f006eb10 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42f5600640 1 --2- 172.21.15.165:0/956289967 >> v2:172.21.15.172:3300/0 conn(0x7f42f0144750 0x7f42f006e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 >> v1:172.21.15.165:6789/0 conn(0x7f42f006eb10 legacy=0x7f42f00ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 0x7f42d80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8003f30 0x7f42d8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00712d0 con 0x7f42d8004d40 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f00715b0 con 0x7f42d8003f30 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d800ba10 con 0x7f42f00cf030 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.263+0000 7f42f6000640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f5600640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 0x7f42d80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f4c00640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8003f30 0x7f42d8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f6000640 1 -- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 msgr2=0x7f42d80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f6000640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00cf030 0x7f42d80039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f6000640 1 -- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8003f30 msgr2=0x7f42d8004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f6000640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8003f30 0x7f42d8004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f6000640 1 -- 172.21.15.165:0/956289967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8003520 con 0x7f42d8004d40 2024-04-25T10:14:52.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.264+0000 7f42f4c00640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8003f30 0x7f42d8004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:52.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.265+0000 7f42f6000640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f42e800e170 tx=0x7f42d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac381a69fdbe9e7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.265+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8034020 con 0x7f42d8004d40 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.265+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e80305f0 con 0x7f42d8004d40 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.265+0000 7f42efe00640 1 -- 172.21.15.165:0/956289967 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e8009c10 con 0x7f42d8004d40 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f8000640 1 -- 172.21.15.165:0/956289967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42d8004d40 msgr2=0x7f42d80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f8000640 1 --2- 172.21.15.165:0/956289967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42d8004d40 0x7f42d80051c0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7f42e800e170 tx=0x7f42d8005830 comp rx=0 tx=0).stop 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f6000640 1 -- 172.21.15.165:0/956289967 reap_dead start 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f8000640 1 -- 172.21.15.165:0/956289967 shutdown_connections 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f8000640 1 -- 172.21.15.165:0/956289967 >> 172.21.15.165:0/956289967 conn(0x7f42f006abe0 msgr2=0x7f42f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f8000640 1 -- 172.21.15.165:0/956289967 shutdown_connections 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.266+0000 7f42f8000640 1 -- 172.21.15.165:0/956289967 wait complete. 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.268+0000 7f42f8000640 1 Processor -- start 2024-04-25T10:14:52.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.268+0000 7f42f8000640 1 -- start start 2024-04-25T10:14:52.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f8000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8004d40 0x7f42f01435d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f8000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 0x7f42f01414e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f8000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00acf80 0x7f42f00a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f8000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f01455d0 con 0x7f42f00acb00 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f8000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f0145450 con 0x7f42d8004d40 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f8000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f015c000 con 0x7f42f00acf80 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 0x7f42f01414e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 0x7f42f01414e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50506/0 (socket says 172.21.15.165:50506) 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f4c00640 1 -- 172.21.15.165:0/2299653208 learned_addr learned my addr 172.21.15.165:0/2299653208 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.269+0000 7f42f4c00640 1 -- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00acf80 msgr2=0x7f42f00a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f6000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00acf80 0x7f42f00a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f5600640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8004d40 0x7f42f01435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f4c00640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00acf80 0x7f42f00a94b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f4c00640 1 -- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8004d40 msgr2=0x7f42f01435d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f4c00640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8004d40 0x7f42f01435d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f4c00640 1 -- 172.21.15.165:0/2299653208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00a86f0 con 0x7f42f00acb00 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f5600640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8004d40 0x7f42f01435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f4c00640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 0x7f42f01414e0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00d8b0 tx=0x7f42dc00dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc019070 con 0x7f42f00acb00 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42dc0074e0 con 0x7f42f00acb00 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42dc0049e0 con 0x7f42f00acb00 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f00a89d0 con 0x7f42f00acb00 2024-04-25T10:14:52.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.270+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f00a8f90 con 0x7f42f00acb00 2024-04-25T10:14:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.272+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f0073990 con 0x7f42f00acb00 2024-04-25T10:14:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.272+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f42dc01c070 con 0x7f42f00acb00 2024-04-25T10:14:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.273+0000 7f42ed600640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42d006b500 0x7f42d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.273+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f42dc089ea0 con 0x7f42f00acb00 2024-04-25T10:14:52.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.274+0000 7f42f5600640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42d006b500 0x7f42d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.274+0000 7f42f5600640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42d006b500 0x7f42d006d9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009c10 tx=0x7f42e4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.278+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f42dc058c20 con 0x7f42f00acb00 2024-04-25T10:14:52.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.477+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f00d0510 con 0x7f42f00acb00 2024-04-25T10:14:52.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.479+0000 7f42ed600640 1 -- 172.21.15.165:0/2299653208 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f42dc058520 con 0x7f42f00acb00 2024-04-25T10:14:52.484 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:52.484 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":124,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:54.924428+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42d006b500 msgr2=0x7f42d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42d006b500 0x7f42d006d9c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f42e4009c10 tx=0x7f42e4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 msgr2=0x7f42f01414e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 0x7f42f01414e0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f42dc00d8b0 tx=0x7f42dc00dd70 comp rx=0 tx=0).stop 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 shutdown_connections 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f42d006b500 0x7f42d006d9c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f42f00acf80 0x7f42f00a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f42f00acb00 0x7f42f01414e0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 --2- 172.21.15.165:0/2299653208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f42d8004d40 0x7f42f01435d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 >> 172.21.15.165:0/2299653208 conn(0x7f42f006abe0 msgr2=0x7f42f00b0ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 shutdown_connections 2024-04-25T10:14:52.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.483+0000 7f42f8000640 1 -- 172.21.15.165:0/2299653208 wait complete. 2024-04-25T10:14:52.505 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:13:51.159277+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:13:52.405294+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '124', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:14:52.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:14:52.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.692+0000 7f6ba0600640 1 Processor -- start 2024-04-25T10:14:52.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.693+0000 7f6ba0600640 1 -- start start 2024-04-25T10:14:52.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.694+0000 7f6ba0600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6b9815dc20 0x7f6b980ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.694+0000 7f6ba0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b980cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.694+0000 7f6ba0600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b9815e000 con 0x7f6b980b9330 2024-04-25T10:14:52.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.694+0000 7f6ba0600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b980d1de0 con 0x7f6b9815dc20 2024-04-25T10:14:52.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.694+0000 7f6ba0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98161c60 con 0x7f6b98161710 2024-04-25T10:14:52.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.695+0000 7f6b9dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6b9815dc20 0x7f6b980ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.695+0000 7f6b9dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6b9815dc20 0x7f6b980ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51374/0 (socket says 172.21.15.165:51374) 2024-04-25T10:14:52.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.695+0000 7f6b9dc00640 1 -- 172.21.15.165:0/2289817112 learned_addr learned my addr 172.21.15.165:0/2289817112 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:52.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.695+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2289817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b980cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.696+0000 7f6b9d200640 1 -- 172.21.15.165:0/2289817112 >> v2:172.21.15.172:3300/0 conn(0x7f6b9815dc20 msgr2=0x7f6b980ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:52.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.696+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2289817112 >> v2:172.21.15.172:3300/0 conn(0x7f6b9815dc20 0x7f6b980ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.696+0000 7f6b9d200640 1 -- 172.21.15.165:0/2289817112 >> v1:172.21.15.172:6789/0 conn(0x7f6b980b9330 legacy=0x7f6b9806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.696+0000 7f6b9d200640 1 -- 172.21.15.165:0/2289817112 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98161f40 con 0x7f6b98161710 2024-04-25T10:14:52.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.697+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2289817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b980cebb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b840097a0 tx=0x7f6b8402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5216ecf875e50422 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6b97e00640 1 -- 172.21.15.165:0/2289817112 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84037070 con 0x7f6b98161710 2024-04-25T10:14:52.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6b97e00640 1 -- 172.21.15.165:0/2289817112 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b84032440 con 0x7f6b98161710 2024-04-25T10:14:52.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6b97e00640 1 -- 172.21.15.165:0/2289817112 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b84032810 con 0x7f6b98161710 2024-04-25T10:14:52.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 -- 172.21.15.165:0/2289817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 msgr2=0x7f6b980cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2289817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b980cebb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f6b840097a0 tx=0x7f6b8402a180 comp rx=0 tx=0).stop 2024-04-25T10:14:52.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 -- 172.21.15.165:0/2289817112 shutdown_connections 2024-04-25T10:14:52.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2289817112 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b980cebb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2289817112 >> v2:172.21.15.172:3300/0 conn(0x7f6b9815dc20 0x7f6b980ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 -- 172.21.15.165:0/2289817112 >> 172.21.15.165:0/2289817112 conn(0x7f6b980d0420 msgr2=0x7f6b980d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.698+0000 7f6ba0600640 1 -- 172.21.15.165:0/2289817112 shutdown_connections 2024-04-25T10:14:52.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.699+0000 7f6ba0600640 1 -- 172.21.15.165:0/2289817112 wait complete. 2024-04-25T10:14:52.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.700+0000 7f6ba0600640 1 Processor -- start 2024-04-25T10:14:52.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.700+0000 7f6ba0600640 1 -- start start 2024-04-25T10:14:52.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.701+0000 7f6ba0600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 0x7f6b980b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.701+0000 7f6ba0600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b9815dc20 0x7f6b9815f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.701+0000 7f6ba0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b9815fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.701+0000 7f6ba0600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980b5e50 con 0x7f6b980b9330 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.701+0000 7f6ba0600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b98162500 con 0x7f6b9815dc20 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.701+0000 7f6ba0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b980b5fd0 con 0x7f6b98161710 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 0x7f6b980b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 0x7f6b980b1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50508/0 (socket says 172.21.15.165:50508) 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 -- 172.21.15.165:0/2939479786 learned_addr learned my addr 172.21.15.165:0/2939479786 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b9815dc20 0x7f6b9815f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9e600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b9815fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 -- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 msgr2=0x7f6b9815fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b9815fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 -- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b9815dc20 msgr2=0x7f6b9815f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b9815dc20 0x7f6b9815f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9dc00640 1 -- 172.21.15.165:0/2939479786 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b98157200 con 0x7f6b980b9330 2024-04-25T10:14:52.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.702+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b9815dc20 0x7f6b9815f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.703+0000 7f6b9dc00640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 0x7f6b980b1c80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009cf0 tx=0x7f6b8800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.703+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b8800cda0 con 0x7f6b980b9330 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.703+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b88019070 con 0x7f6b980b9330 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.703+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b980b2380 con 0x7f6b980b9330 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.703+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b880146a0 con 0x7f6b980b9330 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.703+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b9815a140 con 0x7f6b980b9330 2024-04-25T10:14:52.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.704+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b98073a00 con 0x7f6b980b9330 2024-04-25T10:14:52.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.705+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6b88005d10 con 0x7f6b980b9330 2024-04-25T10:14:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.706+0000 7f6b95600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b7806b500 0x7f6b7806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.706+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6b8808a830 con 0x7f6b980b9330 2024-04-25T10:14:52.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.706+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b7806b500 0x7f6b7806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:52.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.707+0000 7f6b9d200640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b7806b500 0x7f6b7806d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6b84034000 tx=0x7f6b8402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:52.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.711+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6b98157200 con 0x7f6b980b9330 2024-04-25T10:14:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.905+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b980d16b0 con 0x7f6b980b9330 2024-04-25T10:14:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.906+0000 7f6b95600640 1 -- 172.21.15.165:0/2939479786 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b88010040 con 0x7f6b980b9330 2024-04-25T10:14:52.909 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:14:52.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b7806b500 msgr2=0x7f6b7806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b7806b500 0x7f6b7806d9c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6b84034000 tx=0x7f6b8402e000 comp rx=0 tx=0).stop 2024-04-25T10:14:52.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 msgr2=0x7f6b980b1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:52.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 0x7f6b980b1c80 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f6b88009cf0 tx=0x7f6b8800c490 comp rx=0 tx=0).stop 2024-04-25T10:14:52.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 shutdown_connections 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6b7806b500 0x7f6b7806d9c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6b98161710 0x7f6b9815fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6b9815dc20 0x7f6b9815f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 --2- 172.21.15.165:0/2939479786 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6b980b9330 0x7f6b980b1c80 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 >> 172.21.15.165:0/2939479786 conn(0x7f6b980d0420 msgr2=0x7f6b980b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 shutdown_connections 2024-04-25T10:14:52.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:52.908+0000 7f6ba0600640 1 -- 172.21.15.165:0/2939479786 wait complete. 2024-04-25T10:14:52.932 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:14:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.135+0000 7f1e51000640 1 Processor -- start 2024-04-25T10:14:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.135+0000 7f1e51000640 1 -- start start 2024-04-25T10:14:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.137+0000 7f1e51000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c15baf0 0x7f1e4c071690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.137+0000 7f1e51000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e4c0cc730 0x7f1e4c0d04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.137+0000 7f1e51000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e4c0cf430 con 0x7f1e4c0d2400 2024-04-25T10:14:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.137+0000 7f1e51000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e4c15cee0 con 0x7f1e4c0cc730 2024-04-25T10:14:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.137+0000 7f1e51000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0d0040 con 0x7f1e4c15baf0 2024-04-25T10:14:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.138+0000 7f1e4a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e4c0cc730 0x7f1e4c0d04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.138+0000 7f1e49e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c15baf0 0x7f1e4c071690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.138+0000 7f1e41400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1e4c0d2400 0x7f1e4c0cc020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:58356/0 (socket says 172.21.15.165:58356) 2024-04-25T10:14:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.138+0000 7f1e4a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1e4c0cc730 0x7f1e4c0d04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:51404/0 (socket says 172.21.15.165:51404) 2024-04-25T10:14:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.138+0000 7f1e41400640 1 -- 172.21.15.165:0/2092500058 learned_addr learned my addr 172.21.15.165:0/2092500058 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.139+0000 7f1e4a800640 1 -- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c15baf0 msgr2=0x7f1e4c071690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.139+0000 7f1e4a800640 1 --2- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c15baf0 0x7f1e4c071690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.139+0000 7f1e4a800640 1 -- 172.21.15.165:0/2092500058 >> v1:172.21.15.172:6789/0 conn(0x7f1e4c0d2400 legacy=0x7f1e4c0cc020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.139+0000 7f1e4a800640 1 -- 172.21.15.165:0/2092500058 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c0cf710 con 0x7f1e4c0cc730 2024-04-25T10:14:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.139+0000 7f1e49e00640 1 --2- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c15baf0 0x7f1e4c071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:14:53.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.140+0000 7f1e4a800640 1 --2- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c0cc730 0x7f1e4c0d04d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009d90 tx=0x7f1e4402a5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae3fb9bafa7920fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.141+0000 7f1e49400640 1 -- 172.21.15.165:0/2092500058 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44031600 con 0x7f1e4c0cc730 2024-04-25T10:14:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.141+0000 7f1e49400640 1 -- 172.21.15.165:0/2092500058 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e440317a0 con 0x7f1e4c0cc730 2024-04-25T10:14:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.141+0000 7f1e49400640 1 -- 172.21.15.165:0/2092500058 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44031b70 con 0x7f1e4c0cc730 2024-04-25T10:14:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 -- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c0cc730 msgr2=0x7f1e4c0d04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 --2- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c0cc730 0x7f1e4c0d04d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009d90 tx=0x7f1e4402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 -- 172.21.15.165:0/2092500058 shutdown_connections 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 --2- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c0cc730 0x7f1e4c0d04d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 --2- 172.21.15.165:0/2092500058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c15baf0 0x7f1e4c071690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 -- 172.21.15.165:0/2092500058 >> 172.21.15.165:0/2092500058 conn(0x7f1e4c0d17c0 msgr2=0x7f1e4c0d1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 -- 172.21.15.165:0/2092500058 shutdown_connections 2024-04-25T10:14:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.142+0000 7f1e51000640 1 -- 172.21.15.165:0/2092500058 wait complete. 2024-04-25T10:14:53.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.144+0000 7f1e51000640 1 Processor -- start 2024-04-25T10:14:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.144+0000 7f1e51000640 1 -- start start 2024-04-25T10:14:53.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e51000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c0cc730 0x7f1e4c14a060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e51000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c15baf0 0x7f1e4c149720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e51000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 0x7f1e4c14b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e51000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c071bd0 con 0x7f1e4c1540d0 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e51000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c0cfcd0 con 0x7f1e4c15baf0 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e51000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c071d50 con 0x7f1e4c0cc730 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e4a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 0x7f1e4c14b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e4a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 0x7f1e4c14b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50520/0 (socket says 172.21.15.165:50520) 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.145+0000 7f1e4a800640 1 -- 172.21.15.165:0/2018078875 learned_addr learned my addr 172.21.15.165:0/2018078875 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e41400640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c15baf0 0x7f1e4c149720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e49e00640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c0cc730 0x7f1e4c14a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e4a800640 1 -- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c0cc730 msgr2=0x7f1e4c14a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e4a800640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c0cc730 0x7f1e4c14a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e4a800640 1 -- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c15baf0 msgr2=0x7f1e4c149720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e4a800640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c15baf0 0x7f1e4c149720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e4a800640 1 -- 172.21.15.165:0/2018078875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e4c14a740 con 0x7f1e4c1540d0 2024-04-25T10:14:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.146+0000 7f1e4a800640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 0x7f1e4c14b7e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009eb0 tx=0x7f1e44004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.147+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44040070 con 0x7f1e4c1540d0 2024-04-25T10:14:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.147+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e44030b60 con 0x7f1e4c1540d0 2024-04-25T10:14:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.147+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e44009450 con 0x7f1e4c1540d0 2024-04-25T10:14:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.147+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c14aa20 con 0x7f1e4c1540d0 2024-04-25T10:14:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.147+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c14afe0 con 0x7f1e4c1540d0 2024-04-25T10:14:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.148+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c072ce0 con 0x7f1e4c1540d0 2024-04-25T10:14:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.149+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1e44039040 con 0x7f1e4c1540d0 2024-04-25T10:14:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.150+0000 7f1e42a00640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e2806b500 0x7f1e2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.150+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1e440abb10 con 0x7f1e4c1540d0 2024-04-25T10:14:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.150+0000 7f1e49e00640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e2806b500 0x7f1e2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.151+0000 7f1e49e00640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e2806b500 0x7f1e2806d9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009a30 tx=0x7f1e3c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.154+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1e4c072ce0 con 0x7f1e4c1540d0 2024-04-25T10:14:53.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.352+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e4c0cce60 con 0x7f1e4c1540d0 2024-04-25T10:14:53.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.353+0000 7f1e42a00640 1 -- 172.21.15.165:0/2018078875 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e4407a6d0 con 0x7f1e4c1540d0 2024-04-25T10:14:53.357 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.355+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e2806b500 msgr2=0x7f1e2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.355+0000 7f1e51000640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e2806b500 0x7f1e2806d9c0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009a30 tx=0x7f1e3c008040 comp rx=0 tx=0).stop 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.355+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 msgr2=0x7f1e4c14b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.355+0000 7f1e51000640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 0x7f1e4c14b7e0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7f1e44009eb0 tx=0x7f1e44004a40 comp rx=0 tx=0).stop 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 shutdown_connections 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1e2806b500 0x7f1e2806d9c0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1e4c1540d0 0x7f1e4c14b7e0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1e4c15baf0 0x7f1e4c149720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 --2- 172.21.15.165:0/2018078875 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1e4c0cc730 0x7f1e4c14a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 >> 172.21.15.165:0/2018078875 conn(0x7f1e4c0d17c0 msgr2=0x7f1e4c0cb2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 shutdown_connections 2024-04-25T10:14:53.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.356+0000 7f1e51000640 1 -- 172.21.15.165:0/2018078875 wait complete. 2024-04-25T10:14:53.378 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:14:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.573+0000 7fbaf9000640 1 Processor -- start 2024-04-25T10:14:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.573+0000 7fbaf9000640 1 -- start start 2024-04-25T10:14:53.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbaf40cf640 0x7fbaf406ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf40accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf9000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaf40cfad0 con 0x7fbaf415b710 2024-04-25T10:14:53.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf9000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaf406f190 con 0x7fbaf40cf640 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf415e490 con 0x7fbaf40ac8b0 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbaf40cf640 0x7fbaf406ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbaf40cf640 0x7fbaf406ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50524/0 (socket says 172.21.15.165:50524) 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.575+0000 7fbaf1400640 1 -- 172.21.15.165:0/4177384835 learned_addr learned my addr 172.21.15.165:0/4177384835 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.576+0000 7fbaf2800640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf40accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.576+0000 7fbaf1400640 1 -- 172.21.15.165:0/4177384835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 msgr2=0x7fbaf40accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.576+0000 7fbaf1400640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf40accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.576+0000 7fbaf1400640 1 -- 172.21.15.165:0/4177384835 >> v1:172.21.15.172:6789/0 conn(0x7fbaf415b710 legacy=0x7fbaf415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.576+0000 7fbaf1400640 1 -- 172.21.15.165:0/4177384835 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf40cfd80 con 0x7fbaf40cf640 2024-04-25T10:14:53.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.576+0000 7fbaf2800640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:53.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.577+0000 7fbaf1400640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf40cf640 0x7fbaf406ec50 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009870 tx=0x7fbae002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a091932ce87d51b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.577+0000 7fbaf0a00640 1 -- 172.21.15.165:0/4177384835 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0037070 con 0x7fbaf40cf640 2024-04-25T10:14:53.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.577+0000 7fbaf0a00640 1 -- 172.21.15.165:0/4177384835 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0032440 con 0x7fbaf40cf640 2024-04-25T10:14:53.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.577+0000 7fbaf0a00640 1 -- 172.21.15.165:0/4177384835 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0032810 con 0x7fbaf40cf640 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 -- 172.21.15.165:0/4177384835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf40cf640 msgr2=0x7fbaf406ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf40cf640 0x7fbaf406ec50 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009870 tx=0x7fbae002a200 comp rx=0 tx=0).stop 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 -- 172.21.15.165:0/4177384835 shutdown_connections 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf40accd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 --2- 172.21.15.165:0/4177384835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf40cf640 0x7fbaf406ec50 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 -- 172.21.15.165:0/4177384835 >> 172.21.15.165:0/4177384835 conn(0x7fbaf406abe0 msgr2=0x7fbaf406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 -- 172.21.15.165:0/4177384835 shutdown_connections 2024-04-25T10:14:53.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.578+0000 7fbaf9000640 1 -- 172.21.15.165:0/4177384835 wait complete. 2024-04-25T10:14:53.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.580+0000 7fbaf9000640 1 Processor -- start 2024-04-25T10:14:53.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.580+0000 7fbaf9000640 1 -- start start 2024-04-25T10:14:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf41f77a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf9000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaf40cf640 0x7fbaf40c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 0x7fbaf4146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf9000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf415baf0 con 0x7fbaf415b710 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf9000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf40d0340 con 0x7fbaf40cf640 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaf415bc70 con 0x7fbaf40ac8b0 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 0x7fbaf4146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 0x7fbaf4146fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50526/0 (socket says 172.21.15.165:50526) 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.581+0000 7fbaf2800640 1 -- 172.21.15.165:0/248156445 learned_addr learned my addr 172.21.15.165:0/248156445 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf1400640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaf40cf640 0x7fbaf40c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf1e00640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf41f77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf2800640 1 -- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 msgr2=0x7fbaf41f77a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf2800640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf41f77a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf2800640 1 -- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaf40cf640 msgr2=0x7fbaf40c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf2800640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaf40cf640 0x7fbaf40c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf2800640 1 -- 172.21.15.165:0/248156445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf414b1b0 con 0x7fbaf415b710 2024-04-25T10:14:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf2800640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 0x7fbaf4146fb0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00cef0 tx=0x7fbaec00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec01a070 con 0x7fbaf415b710 2024-04-25T10:14:53.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbaec00d860 con 0x7fbaf415b710 2024-04-25T10:14:53.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaf414b490 con 0x7fbaf415b710 2024-04-25T10:14:53.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.582+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaec00dc50 con 0x7fbaf415b710 2024-04-25T10:14:53.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.583+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf414ba50 con 0x7fbaf415b710 2024-04-25T10:14:53.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.584+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaf4073a00 con 0x7fbaf415b710 2024-04-25T10:14:53.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.585+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbaec01d070 con 0x7fbaf415b710 2024-04-25T10:14:53.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.585+0000 7fbaea000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbad406b4d0 0x7fbad406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:53.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.586+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbaec08ac70 con 0x7fbaf415b710 2024-04-25T10:14:53.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.586+0000 7fbaf1e00640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbad406b4d0 0x7fbad406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:53.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.586+0000 7fbaf1e00640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbad406b4d0 0x7fbad406d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006fd0 tx=0x7fbae4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:53.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.590+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbaec059a10 con 0x7fbaf415b710 2024-04-25T10:14:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.783+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbaf415d880 con 0x7fbaf415b710 2024-04-25T10:14:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.784+0000 7fbaea000640 1 -- 172.21.15.165:0/248156445 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbaec059310 con 0x7fbaf415b710 2024-04-25T10:14:53.788 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbad406b4d0 msgr2=0x7fbad406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbad406b4d0 0x7fbad406d990 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fbae4006fd0 tx=0x7fbae4008040 comp rx=0 tx=0).stop 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 msgr2=0x7fbaf4146fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 0x7fbaf4146fb0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fbaec00cef0 tx=0x7fbaec00a710 comp rx=0 tx=0).stop 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 shutdown_connections 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbad406b4d0 0x7fbad406d990 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbaf415b710 0x7fbaf4146fb0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbaf40cf640 0x7fbaf40c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 --2- 172.21.15.165:0/248156445 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbaf40ac8b0 0x7fbaf41f77a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:53.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 >> 172.21.15.165:0/248156445 conn(0x7fbaf406abe0 msgr2=0x7fbaf415c680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:53.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 shutdown_connections 2024-04-25T10:14:53.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:53.787+0000 7fbaf9000640 1 -- 172.21.15.165:0/248156445 wait complete. 2024-04-25T10:14:53.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:54.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.005+0000 7fc1a2200640 1 Processor -- start 2024-04-25T10:14:54.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.005+0000 7fc1a2200640 1 -- start start 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.006+0000 7fc1a2200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc19c15e520 0x7fc19c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc1a2200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc19c070ff0 0x7fc19c071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc1a2200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc19c0ae590 con 0x7fc19c15f040 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc1a2200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c0aee30 con 0x7fc19c15e520 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc1a2200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc19c071980 con 0x7fc19c070ff0 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc1a0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc19c15e520 0x7fc19c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc193e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc19c15f040 0x7fc19c0adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52998/0 (socket says 172.21.15.165:52998) 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc193e00640 1 -- 172.21.15.165:0/683741151 learned_addr learned my addr 172.21.15.165:0/683741151 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc1a1800640 1 --2- 172.21.15.165:0/683741151 >> v2:172.21.15.172:3300/0 conn(0x7fc19c070ff0 0x7fc19c071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc19be00640 1 -- 172.21.15.165:0/683741151 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2689782036 0 0) 0x7fc19c0ae590 con 0x7fc19c15f040 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.007+0000 7fc19be00640 1 -- 172.21.15.165:0/683741151 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc180003520 con 0x7fc19c15f040 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.008+0000 7fc1a0e00640 1 -- 172.21.15.165:0/683741151 >> v2:172.21.15.172:3300/0 conn(0x7fc19c070ff0 msgr2=0x7fc19c071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.008+0000 7fc1a0e00640 1 --2- 172.21.15.165:0/683741151 >> v2:172.21.15.172:3300/0 conn(0x7fc19c070ff0 0x7fc19c071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.008+0000 7fc1a0e00640 1 -- 172.21.15.165:0/683741151 >> v1:172.21.15.165:6789/0 conn(0x7fc19c15f040 legacy=0x7fc19c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.008+0000 7fc1a0e00640 1 -- 172.21.15.165:0/683741151 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c0ae870 con 0x7fc19c15e520 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.009+0000 7fc1a0e00640 1 --2- 172.21.15.165:0/683741151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c15e520 0x7fc19c15e900 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc19c06f450 tx=0x7fc18c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c76f689baa5b46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.009+0000 7fc19be00640 1 -- 172.21.15.165:0/683741151 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037070 con 0x7fc19c15e520 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.009+0000 7fc19be00640 1 -- 172.21.15.165:0/683741151 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c032440 con 0x7fc19c15e520 2024-04-25T10:14:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.009+0000 7fc19be00640 1 -- 172.21.15.165:0/683741151 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c032810 con 0x7fc19c15e520 2024-04-25T10:14:54.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.009+0000 7fc1a2200640 1 -- 172.21.15.165:0/683741151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c15e520 msgr2=0x7fc19c15e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.009+0000 7fc1a2200640 1 --2- 172.21.15.165:0/683741151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c15e520 0x7fc19c15e900 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fc19c06f450 tx=0x7fc18c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.010+0000 7fc1a2200640 1 -- 172.21.15.165:0/683741151 shutdown_connections 2024-04-25T10:14:54.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.010+0000 7fc1a2200640 1 --2- 172.21.15.165:0/683741151 >> v2:172.21.15.172:3300/0 conn(0x7fc19c070ff0 0x7fc19c071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.010+0000 7fc1a2200640 1 --2- 172.21.15.165:0/683741151 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c15e520 0x7fc19c15e900 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.010+0000 7fc1a2200640 1 -- 172.21.15.165:0/683741151 >> 172.21.15.165:0/683741151 conn(0x7fc19c06abe0 msgr2=0x7fc19c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.010+0000 7fc1a2200640 1 -- 172.21.15.165:0/683741151 shutdown_connections 2024-04-25T10:14:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.010+0000 7fc1a2200640 1 -- 172.21.15.165:0/683741151 wait complete. 2024-04-25T10:14:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.011+0000 7fc1a2200640 1 Processor -- start 2024-04-25T10:14:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.012+0000 7fc1a2200640 1 -- start start 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a2200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 0x7fc19c15b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19c15e520 0x7fc19c0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc19c15f040 0x7fc19c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a2200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c0ae2a0 con 0x7fc19c070ff0 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c06d1f0 con 0x7fc19c15e520 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc19c15f4d0 con 0x7fc19c15f040 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 0x7fc19c15b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 0x7fc19c15b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50548/0 (socket says 172.21.15.165:50548) 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 -- 172.21.15.165:0/2407078680 learned_addr learned my addr 172.21.15.165:0/2407078680 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 -- 172.21.15.165:0/2407078680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc19c15f040 msgr2=0x7fc19c0b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc19c15f040 0x7fc19c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 -- 172.21.15.165:0/2407078680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19c15e520 msgr2=0x7fc19c0b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19c15e520 0x7fc19c0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.013+0000 7fc1a0e00640 1 -- 172.21.15.165:0/2407078680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc19c1594f0 con 0x7fc19c070ff0 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.014+0000 7fc1a0e00640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 0x7fc19c15b050 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009890 tx=0x7fc18c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.014+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c037040 con 0x7fc19c070ff0 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.014+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc18c042070 con 0x7fc19c070ff0 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.014+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc19c1597d0 con 0x7fc19c070ff0 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.014+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc18c030960 con 0x7fc19c070ff0 2024-04-25T10:14:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.014+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc19c159d90 con 0x7fc19c070ff0 2024-04-25T10:14:54.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.015+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc19c073990 con 0x7fc19c070ff0 2024-04-25T10:14:54.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.016+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc18c030b00 con 0x7fc19c070ff0 2024-04-25T10:14:54.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.017+0000 7fc199600640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17806b4d0 0x7fc17806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.017+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc18c0aacb0 con 0x7fc19c070ff0 2024-04-25T10:14:54.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.017+0000 7fc193e00640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17806b4d0 0x7fc17806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.018+0000 7fc193e00640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17806b4d0 0x7fc17806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc184000f60 tx=0x7fc1840078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.021+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc18c079990 con 0x7fc19c070ff0 2024-04-25T10:14:54.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.253+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc19c0afaa0 con 0x7fc19c070ff0 2024-04-25T10:14:54.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.254+0000 7fc199600640 1 -- 172.21.15.165:0/2407078680 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fc18c079290 con 0x7fc19c070ff0 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:54.258 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.256+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17806b4d0 msgr2=0x7fc17806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.256+0000 7fc1a2200640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17806b4d0 0x7fc17806d990 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fc184000f60 tx=0x7fc1840078f0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 msgr2=0x7fc19c15b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 0x7fc19c15b050 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fc18c009890 tx=0x7fc18c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 shutdown_connections 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc17806b4d0 0x7fc17806d990 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc19c15f040 0x7fc19c0b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc19c15e520 0x7fc19c0b3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 --2- 172.21.15.165:0/2407078680 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc19c070ff0 0x7fc19c15b050 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 >> 172.21.15.165:0/2407078680 conn(0x7fc19c06abe0 msgr2=0x7fc19c0b0510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 shutdown_connections 2024-04-25T10:14:54.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.257+0000 7fc1a2200640 1 -- 172.21.15.165:0/2407078680 wait complete. 2024-04-25T10:14:54.279 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:54.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.472+0000 7f91e7e00640 1 Processor -- start 2024-04-25T10:14:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.473+0000 7f91e7e00640 1 -- start start 2024-04-25T10:14:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.474+0000 7f91e7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91e8144440 0x7f91e8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.474+0000 7f91e7e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e815bf30 con 0x7f91e80cda30 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.474+0000 7f91e7e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91e815c210 con 0x7f91e8071530 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.474+0000 7f91e7e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91e80719a0 con 0x7f91e8144440 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.475+0000 7f91e6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91e8144440 0x7f91e8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.475+0000 7f91e6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91e8144440 0x7f91e8070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50554/0 (socket says 172.21.15.165:50554) 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.475+0000 7f91e6a00640 1 -- 172.21.15.165:0/688598881 learned_addr learned my addr 172.21.15.165:0/688598881 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.475+0000 7f91e6a00640 1 -- 172.21.15.165:0/688598881 >> v1:172.21.15.172:6789/0 conn(0x7f91e8071530 legacy=0x7f91e80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.475+0000 7f91e6a00640 1 -- 172.21.15.165:0/688598881 >> v1:172.21.15.165:6789/0 conn(0x7f91e80cda30 legacy=0x7f91e815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.475+0000 7f91e6a00640 1 -- 172.21.15.165:0/688598881 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e815c4f0 con 0x7f91e8144440 2024-04-25T10:14:54.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.476+0000 7f91e6a00640 1 --2- 172.21.15.165:0/688598881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e8070ff0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009e50 tx=0x7f91d802a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=590d2149f5d5df55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.476+0000 7f91e5600640 1 -- 172.21.15.165:0/688598881 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d8036070 con 0x7f91e8144440 2024-04-25T10:14:54.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.476+0000 7f91e5600640 1 -- 172.21.15.165:0/688598881 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91d8031600 con 0x7f91e8144440 2024-04-25T10:14:54.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.476+0000 7f91e5600640 1 -- 172.21.15.165:0/688598881 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91d80319d0 con 0x7f91e8144440 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 -- 172.21.15.165:0/688598881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 msgr2=0x7f91e8070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 --2- 172.21.15.165:0/688598881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e8070ff0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f91d8009e50 tx=0x7f91d802a7f0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 -- 172.21.15.165:0/688598881 shutdown_connections 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 --2- 172.21.15.165:0/688598881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e8070ff0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 -- 172.21.15.165:0/688598881 >> 172.21.15.165:0/688598881 conn(0x7f91e806abe0 msgr2=0x7f91e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 -- 172.21.15.165:0/688598881 shutdown_connections 2024-04-25T10:14:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.477+0000 7f91e7e00640 1 -- 172.21.15.165:0/688598881 wait complete. 2024-04-25T10:14:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.479+0000 7f91e7e00640 1 Processor -- start 2024-04-25T10:14:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.479+0000 7f91e7e00640 1 -- start start 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91e8071530 0x7f91e81f1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91e80cda30 0x7f91e80cc400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e80c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e80cdea0 con 0x7f91e8144440 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e815c860 con 0x7f91e8071530 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91e815d400 con 0x7f91e80cda30 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e80c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e80c7ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50568/0 (socket says 172.21.15.165:50568) 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 -- 172.21.15.165:0/3113736587 learned_addr learned my addr 172.21.15.165:0/3113736587 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 -- 172.21.15.165:0/3113736587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91e80cda30 msgr2=0x7f91e80cc400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91e80cda30 0x7f91e80cc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 -- 172.21.15.165:0/3113736587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91e8071530 msgr2=0x7f91e81f1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91e8071530 0x7f91e81f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.480+0000 7f91e7400640 1 -- 172.21.15.165:0/3113736587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91e80cb360 con 0x7f91e8144440 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.481+0000 7f91e7400640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e80c7ad0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f91e0001eb0 tx=0x7f91e000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.481+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0017070 con 0x7f91e8144440 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.481+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91e00052d0 con 0x7f91e8144440 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.481+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91e0003830 con 0x7f91e8144440 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.481+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e80cb640 con 0x7f91e8144440 2024-04-25T10:14:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.481+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e80cbc00 con 0x7f91e8144440 2024-04-25T10:14:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.482+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e8073990 con 0x7f91e8144440 2024-04-25T10:14:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.483+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91e001a070 con 0x7f91e8144440 2024-04-25T10:14:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.484+0000 7f91dea00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91c406b4d0 0x7f91c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.484+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91e0089770 con 0x7f91e8144440 2024-04-25T10:14:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.484+0000 7f91e6a00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91c406b4d0 0x7f91c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.485+0000 7f91e6a00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91c406b4d0 0x7f91c406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f91e80b2280 tx=0x7f91d802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.488+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91e0058450 con 0x7f91e8144440 2024-04-25T10:14:54.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.669+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91e806b4b0 con 0x7f91e8144440 2024-04-25T10:14:54.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.670+0000 7f91dea00640 1 -- 172.21.15.165:0/3113736587 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f91e0057d50 con 0x7f91e8144440 2024-04-25T10:14:54.675 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:54.675 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":124,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:54.924428+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:54.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.673+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91c406b4d0 msgr2=0x7f91c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.673+0000 7f91e7e00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91c406b4d0 0x7f91c406d990 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f91e80b2280 tx=0x7f91d802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:54.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.673+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 msgr2=0x7f91e80c7ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.673+0000 7f91e7e00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e80c7ad0 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f91e0001eb0 tx=0x7f91e000c890 comp rx=0 tx=0).stop 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 shutdown_connections 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f91c406b4d0 0x7f91c406d990 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91e8144440 0x7f91e80c7ad0 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91e80cda30 0x7f91e80cc400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 --2- 172.21.15.165:0/3113736587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f91e8071530 0x7f91e81f1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 >> 172.21.15.165:0/3113736587 conn(0x7f91e806abe0 msgr2=0x7f91e80c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 shutdown_connections 2024-04-25T10:14:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.674+0000 7f91e7e00640 1 -- 172.21.15.165:0/3113736587 wait complete. 2024-04-25T10:14:54.696 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:14:54.696 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:54.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.884+0000 7f9c5a600640 1 Processor -- start 2024-04-25T10:14:54.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.885+0000 7f9c5a600640 1 -- start start 2024-04-25T10:14:54.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.886+0000 7f9c5a600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9c5415dd00 0x7f9c540cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.886+0000 7f9c5a600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5415bf30 con 0x7f9c54144440 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.886+0000 7f9c5a600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c5415c210 con 0x7f9c54071700 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.886+0000 7f9c5a600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c5415c860 con 0x7f9c5415dd00 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.886+0000 7f9c59c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9c5415dd00 0x7f9c540cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.887+0000 7f9c59c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9c5415dd00 0x7f9c540cda30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50580/0 (socket says 172.21.15.165:50580) 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.887+0000 7f9c59c00640 1 -- 172.21.15.165:0/3950187574 learned_addr learned my addr 172.21.15.165:0/3950187574 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.887+0000 7f9c59200640 1 --1- 172.21.15.165:0/3950187574 >> v1:172.21.15.165:6789/0 conn(0x7f9c54144440 0x7f9c54070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53012/0 (socket says 172.21.15.165:53012) 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.887+0000 7f9c59c00640 1 -- 172.21.15.165:0/3950187574 >> v1:172.21.15.172:6789/0 conn(0x7f9c54071700 legacy=0x7f9c540cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.887+0000 7f9c59c00640 1 -- 172.21.15.165:0/3950187574 >> v1:172.21.15.165:6789/0 conn(0x7f9c54144440 legacy=0x7f9c54070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.887+0000 7f9c59c00640 1 -- 172.21.15.165:0/3950187574 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5415c4f0 con 0x7f9c5415dd00 2024-04-25T10:14:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.888+0000 7f9c59c00640 1 --2- 172.21.15.165:0/3950187574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c5415dd00 0x7f9c540cda30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f9c5415c9e0 tx=0x7f9c4c02a7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d2698c0ed5808e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.888+0000 7f9c53e00640 1 -- 172.21.15.165:0/3950187574 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c036070 con 0x7f9c5415dd00 2024-04-25T10:14:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.888+0000 7f9c53e00640 1 -- 172.21.15.165:0/3950187574 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c4c031600 con 0x7f9c5415dd00 2024-04-25T10:14:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.888+0000 7f9c53e00640 1 -- 172.21.15.165:0/3950187574 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c4c0319d0 con 0x7f9c5415dd00 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 -- 172.21.15.165:0/3950187574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c5415dd00 msgr2=0x7f9c540cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 --2- 172.21.15.165:0/3950187574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c5415dd00 0x7f9c540cda30 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f9c5415c9e0 tx=0x7f9c4c02a7f0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 -- 172.21.15.165:0/3950187574 shutdown_connections 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 --2- 172.21.15.165:0/3950187574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c5415dd00 0x7f9c540cda30 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 -- 172.21.15.165:0/3950187574 >> 172.21.15.165:0/3950187574 conn(0x7f9c5406abe0 msgr2=0x7f9c5406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 -- 172.21.15.165:0/3950187574 shutdown_connections 2024-04-25T10:14:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.889+0000 7f9c5a600640 1 -- 172.21.15.165:0/3950187574 wait complete. 2024-04-25T10:14:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.891+0000 7f9c5a600640 1 Processor -- start 2024-04-25T10:14:54.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.891+0000 7f9c5a600640 1 -- start start 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c5a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c54071700 0x7f9c54163bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c5a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 0x7f9c541640f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c5a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c5415dd00 0x7f9c5415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c5a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54166400 con 0x7f9c54144440 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c5a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54166280 con 0x7f9c54071700 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c5a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54166580 con 0x7f9c5415dd00 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 0x7f9c541640f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 0x7f9c541640f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50594/0 (socket says 172.21.15.165:50594) 2024-04-25T10:14:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.892+0000 7f9c4be00640 1 -- 172.21.15.165:0/2575733039 learned_addr learned my addr 172.21.15.165:0/2575733039 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c59200640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c54071700 0x7f9c54163bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c4be00640 1 -- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c5415dd00 msgr2=0x7f9c5415a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c4be00640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c5415dd00 0x7f9c5415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c4be00640 1 -- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c54071700 msgr2=0x7f9c54163bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c4be00640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c54071700 0x7f9c54163bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c4be00640 1 -- 172.21.15.165:0/2575733039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c5413fba0 con 0x7f9c54144440 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c59200640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c54071700 0x7f9c54163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c4be00640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 0x7f9c541640f0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c460 tx=0x7f9c3c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c016020 con 0x7f9c54144440 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c012870 con 0x7f9c54144440 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.893+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c012c60 con 0x7f9c54144440 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.894+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5413fe80 con 0x7f9c54144440 2024-04-25T10:14:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.894+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c54140440 con 0x7f9c54144440 2024-04-25T10:14:54.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.895+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c54073990 con 0x7f9c54144440 2024-04-25T10:14:54.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.896+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9c3c019020 con 0x7f9c54144440 2024-04-25T10:14:54.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.897+0000 7f9c51600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9c3006b4d0 0x7f9c3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:54.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.897+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9c3c08a150 con 0x7f9c54144440 2024-04-25T10:14:54.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.897+0000 7f9c59200640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9c3006b4d0 0x7f9c3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:54.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.898+0000 7f9c59200640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9c3006b4d0 0x7f9c3006d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9c540b46b0 tx=0x7f9c44009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:54.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:54.901+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9c3c020030 con 0x7f9c54144440 2024-04-25T10:14:55.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.134+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9c54140b60 con 0x7f9c54144440 2024-04-25T10:14:55.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.135+0000 7f9c51600640 1 -- 172.21.15.165:0/2575733039 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f9c3c058da0 con 0x7f9c54144440 2024-04-25T10:14:55.139 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:55.139 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:55.139 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9c3006b4d0 msgr2=0x7f9c3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9c3006b4d0 0x7f9c3006d990 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f9c540b46b0 tx=0x7f9c44009040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 msgr2=0x7f9c541640f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 0x7f9c541640f0 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f9c3c00c460 tx=0x7f9c3c00c920 comp rx=0 tx=0).stop 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 shutdown_connections 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9c3006b4d0 0x7f9c3006d990 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9c5415dd00 0x7f9c5415a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9c54144440 0x7f9c541640f0 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 --2- 172.21.15.165:0/2575733039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9c54071700 0x7f9c54163bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 >> 172.21.15.165:0/2575733039 conn(0x7f9c5406abe0 msgr2=0x7f9c5415f1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 shutdown_connections 2024-04-25T10:14:55.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.138+0000 7f9c5a600640 1 -- 172.21.15.165:0/2575733039 wait complete. 2024-04-25T10:14:55.161 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:14:55.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.351+0000 7f72ec200640 1 Processor -- start 2024-04-25T10:14:55.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.351+0000 7f72ec200640 1 -- start start 2024-04-25T10:14:55.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72ec200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f72e40cfc20 0x7f72e40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72ec200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415df50 0x7f72e406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72ec200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72e4071650 con 0x7f72e415db70 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72ec200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72e40cbd80 con 0x7f72e40cfc20 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72ec200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4161910 con 0x7f72e415df50 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72e8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f72e40cfc20 0x7f72e40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72e8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f72e40cfc20 0x7f72e40cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50610/0 (socket says 172.21.15.165:50610) 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72e8e00640 1 -- 172.21.15.165:0/3558342717 learned_addr learned my addr 172.21.15.165:0/3558342717 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.353+0000 7f72e9800640 1 --1- 172.21.15.165:0/3558342717 >> v1:172.21.15.165:6789/0 conn(0x7f72e415db70 0x7f72e40cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53020/0 (socket says 172.21.15.165:53020) 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72ea200640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415df50 0x7f72e406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72e8e00640 1 -- 172.21.15.165:0/3558342717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415df50 msgr2=0x7f72e406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72e8e00640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415df50 0x7f72e406d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72e8e00640 1 -- 172.21.15.165:0/3558342717 >> v1:172.21.15.165:6789/0 conn(0x7f72e415db70 legacy=0x7f72e40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72e8e00640 1 -- 172.21.15.165:0/3558342717 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e40cf230 con 0x7f72e40cfc20 2024-04-25T10:14:55.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72ea200640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415df50 0x7f72e406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:55.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.354+0000 7f72e3e00640 1 -- 172.21.15.165:0/3558342717 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1181370379 0 0) 0x7f72d8001110 con 0x7f72e415db70 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.355+0000 7f72e8e00640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e40cfc20 0x7f72e40cb840 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009870 tx=0x7f72d002a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18b1a9fb0d56ee89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.355+0000 7f72e3e00640 1 -- 172.21.15.165:0/3558342717 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0037070 con 0x7f72e40cfc20 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.355+0000 7f72e3e00640 1 -- 172.21.15.165:0/3558342717 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72d0032440 con 0x7f72e40cfc20 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.355+0000 7f72e3e00640 1 -- 172.21.15.165:0/3558342717 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72d0032810 con 0x7f72e40cfc20 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 -- 172.21.15.165:0/3558342717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e40cfc20 msgr2=0x7f72e40cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e40cfc20 0x7f72e40cb840 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f72d0009870 tx=0x7f72d002a1c0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 -- 172.21.15.165:0/3558342717 shutdown_connections 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415df50 0x7f72e406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 --2- 172.21.15.165:0/3558342717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e40cfc20 0x7f72e40cb840 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 -- 172.21.15.165:0/3558342717 >> 172.21.15.165:0/3558342717 conn(0x7f72e406abe0 msgr2=0x7f72e406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 -- 172.21.15.165:0/3558342717 shutdown_connections 2024-04-25T10:14:55.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.356+0000 7f72ec200640 1 -- 172.21.15.165:0/3558342717 wait complete. 2024-04-25T10:14:55.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.358+0000 7f72ec200640 1 Processor -- start 2024-04-25T10:14:55.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.358+0000 7f72ec200640 1 -- start start 2024-04-25T10:14:55.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ec200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72e40cfc20 0x7f72e4147ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ec200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415db70 0x7f72e4144790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ec200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 0x7f72e4158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ec200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4162120 con 0x7f72e415df50 2024-04-25T10:14:55.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ec200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4161e40 con 0x7f72e40cfc20 2024-04-25T10:14:55.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ec200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72e4162640 con 0x7f72e415db70 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ea200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 0x7f72e4158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ea200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 0x7f72e4158240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50626/0 (socket says 172.21.15.165:50626) 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ea200640 1 -- 172.21.15.165:0/1295108131 learned_addr learned my addr 172.21.15.165:0/1295108131 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72e9800640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72e40cfc20 0x7f72e4147ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ea200640 1 -- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415db70 msgr2=0x7f72e4144790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.359+0000 7f72ea200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415db70 0x7f72e4144790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72ea200640 1 -- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72e40cfc20 msgr2=0x7f72e4147ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72ea200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72e40cfc20 0x7f72e4147ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72ea200640 1 -- 172.21.15.165:0/1295108131 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e4148010 con 0x7f72e415df50 2024-04-25T10:14:55.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72ea200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 0x7f72e4158240 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00cd50 tx=0x7f72dc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00f680 con 0x7f72e415df50 2024-04-25T10:14:55.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc00f840 con 0x7f72e415df50 2024-04-25T10:14:55.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc00fc30 con 0x7f72e415df50 2024-04-25T10:14:55.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e4155960 con 0x7f72e415df50 2024-04-25T10:14:55.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.360+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e40b3b90 con 0x7f72e415df50 2024-04-25T10:14:55.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.361+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e40cdfb0 con 0x7f72e415df50 2024-04-25T10:14:55.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.362+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f72dc00e3e0 con 0x7f72e415df50 2024-04-25T10:14:55.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.363+0000 7f72e1600640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72c406b4d0 0x7f72c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.363+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f72dc08b180 con 0x7f72e415df50 2024-04-25T10:14:55.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.363+0000 7f72e9800640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72c406b4d0 0x7f72c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.364+0000 7f72e9800640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72c406b4d0 0x7f72c406d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000f60 tx=0x7f72d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.367+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f72e40cdfb0 con 0x7f72e415df50 2024-04-25T10:14:55.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.583+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72e4073990 con 0x7f72e415df50 2024-04-25T10:14:55.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.584+0000 7f72e1600640 1 -- 172.21.15.165:0/1295108131 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 82 v82) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f72e4148010 con 0x7f72e415df50 2024-04-25T10:14:55.588 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:55.588 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":82,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[{"mdsmap":{"epoch":82,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:13:53.389326+0000","modified":"2024-04-25T10:13:56.618933+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6136},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6136":{"gid":6136,"name":"d","rank":0,"incarnation":79,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/3899404305","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3899404305},{"type":"v1","addr":"172.21.15.172:6834","nonce":3899404305}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2024-04-25T10:14:55.588 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 82 2024-04-25T10:14:55.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72c406b4d0 msgr2=0x7f72c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72c406b4d0 0x7f72c406d990 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f72d8000f60 tx=0x7f72d8009040 comp rx=0 tx=0).stop 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 msgr2=0x7f72e4158240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 0x7f72e4158240 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00cd50 tx=0x7f72dc00a650 comp rx=0 tx=0).stop 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 shutdown_connections 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f72c406b4d0 0x7f72c406d990 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f72e415df50 0x7f72e4158240 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f72e415db70 0x7f72e4144790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 --2- 172.21.15.165:0/1295108131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f72e40cfc20 0x7f72e4147ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.586+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 >> 172.21.15.165:0/1295108131 conn(0x7f72e406abe0 msgr2=0x7f72e415c690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.587+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 shutdown_connections 2024-04-25T10:14:55.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.587+0000 7f72ec200640 1 -- 172.21.15.165:0/1295108131 wait complete. 2024-04-25T10:14:55.607 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:14:55.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.794+0000 7ff7d2800640 1 Processor -- start 2024-04-25T10:14:55.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.794+0000 7ff7d2800640 1 -- start start 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7d2800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0d0b10 0x7ff7cc0ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7d2800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0b5f00 0x7ff7cc0d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7d2800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0cec00 con 0x7ff7cc15db60 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7d2800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc0ce930 con 0x7ff7cc0d0b10 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7d2800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0b6450 con 0x7ff7cc0b5f00 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7cb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff7cc15db60 0x7ff7cc0d0400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53034/0 (socket says 172.21.15.165:53034) 2024-04-25T10:14:55.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.795+0000 7ff7cb400640 1 -- 172.21.15.165:0/2058819168 learned_addr learned my addr 172.21.15.165:0/2058819168 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:55.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7caa00640 1 --2- 172.21.15.165:0/2058819168 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0d0b10 0x7ff7cc0ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 535196499 0 0) 0x7ff7cc0cec00 con 0x7ff7cc15db60 2024-04-25T10:14:55.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7ac003520 con 0x7ff7cc15db60 2024-04-25T10:14:55.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7cbe00640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0b5f00 0x7ff7cc0d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4194552398 0 0) 0x7ff7ac003520 con 0x7ff7cc15db60 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0b5f00 msgr2=0x7ff7cc0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0b5f00 0x7ff7cc0d2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0d0b10 msgr2=0x7ff7cc0ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 --2- 172.21.15.165:0/2058819168 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0d0b10 0x7ff7cc0ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0ceee0 con 0x7ff7cc15db60 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff7bc001670 con 0x7ff7cc15db60 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.796+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 >> v1:172.21.15.165:6789/0 conn(0x7ff7cc15db60 legacy=0x7ff7cc0d0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7ca000640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7ac0039e0 0x7ff7ac003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7ca000640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7ca000640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7ac0051c0 0x7ff7ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0ce930 con 0x7ff7ac004800 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0b6450 con 0x7ff7ac0039e0 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7ac00c170 con 0x7ff7ac0051c0 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7cbe00640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc0b5f00 0x7ff7cc0d2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:55.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 --2- 172.21.15.165:0/2058819168 >> v2:172.21.15.172:3300/0 conn(0x7ff7cc0d0b10 0x7ff7cc0ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 -- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7ac0051c0 msgr2=0x7ff7ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7ac0051c0 0x7ff7ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 -- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7ac0039e0 msgr2=0x7ff7ac003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7cb400640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7ac0039e0 0x7ff7ac003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7ac0039e0 0x7ff7ac003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.797+0000 7ff7caa00640 1 -- 172.21.15.165:0/2058819168 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ac003520 con 0x7ff7ac004800 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.798+0000 7ff7caa00640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7ac004c80 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0d1830 tx=0x7ff7ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be898009f046f186 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4037070 con 0x7ff7ac004800 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b402aca0 con 0x7ff7ac004800 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7ca000640 1 -- 172.21.15.165:0/2058819168 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400f040 con 0x7ff7ac004800 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7d2800640 1 -- 172.21.15.165:0/2058819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 msgr2=0x7ff7ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7d2800640 1 --2- 172.21.15.165:0/2058819168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7ac004c80 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0d1830 tx=0x7ff7ac005830 comp rx=0 tx=0).stop 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7cbe00640 1 -- 172.21.15.165:0/2058819168 reap_dead start 2024-04-25T10:14:55.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7d2800640 1 -- 172.21.15.165:0/2058819168 shutdown_connections 2024-04-25T10:14:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7d2800640 1 -- 172.21.15.165:0/2058819168 >> 172.21.15.165:0/2058819168 conn(0x7ff7cc061600 msgr2=0x7ff7cc061a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.799+0000 7ff7d2800640 1 -- 172.21.15.165:0/2058819168 shutdown_connections 2024-04-25T10:14:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.800+0000 7ff7d2800640 1 -- 172.21.15.165:0/2058819168 wait complete. 2024-04-25T10:14:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.801+0000 7ff7d2800640 1 Processor -- start 2024-04-25T10:14:55.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.801+0000 7ff7d2800640 1 -- start start 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.802+0000 7ff7d2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7cc155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.802+0000 7ff7d2800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc152ce0 0x7ff7cc0c9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.802+0000 7ff7d2800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba00 0x7ff7cc15be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.802+0000 7ff7d2800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc15df40 con 0x7ff7ac004800 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.802+0000 7ff7d2800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0d1d50 con 0x7ff7cc15ba00 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.802+0000 7ff7d2800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc15e0c0 con 0x7ff7cc152ce0 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7cc155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7cc155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50654/0 (socket says 172.21.15.165:50654) 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 -- 172.21.15.165:0/1097652508 learned_addr learned my addr 172.21.15.165:0/1097652508 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7caa00640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc152ce0 0x7ff7cc0c9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 -- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc152ce0 msgr2=0x7ff7cc0c9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc152ce0 0x7ff7cc0c9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 -- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba00 msgr2=0x7ff7cc15be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba00 0x7ff7cc15be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 -- 172.21.15.165:0/1097652508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc153510 con 0x7ff7ac004800 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.803+0000 7ff7cb400640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7cc155cc0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00d7d0 tx=0x7ff7bc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.804+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc019070 con 0x7ff7ac004800 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.804+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7bc0074e0 con 0x7ff7ac004800 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.804+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc1537f0 con 0x7ff7ac004800 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.804+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7bc0049e0 con 0x7ff7ac004800 2024-04-25T10:14:55.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.804+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc153db0 con 0x7ff7ac004800 2024-04-25T10:14:55.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.805+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc073270 con 0x7ff7ac004800 2024-04-25T10:14:55.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.806+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7bc01c070 con 0x7ff7ac004800 2024-04-25T10:14:55.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.807+0000 7ff7c3400640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7a806b4d0 0x7ff7a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:55.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.808+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff7bc089e20 con 0x7ff7ac004800 2024-04-25T10:14:55.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.808+0000 7ff7caa00640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7a806b4d0 0x7ff7a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:55.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.808+0000 7ff7caa00640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7a806b4d0 0x7ff7a806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4034000 tx=0x7ff7b402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:55.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:55.812+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7bc058a70 con 0x7ff7ac004800 2024-04-25T10:14:56.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.009+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7cc073990 con 0x7ff7ac004800 2024-04-25T10:14:56.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.011+0000 7ff7c3400640 1 -- 172.21.15.165:0/1097652508 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16988 (secure 0 0 0) 0x7ff7bc058370 con 0x7ff7ac004800 2024-04-25T10:14:56.015 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:14:56.015 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":124,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:13:54.924428+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:13:51.159277+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_data","create_time":"2024-04-25T10:13:52.405294+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"124","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7a806b4d0 msgr2=0x7ff7a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7a806b4d0 0x7ff7a806d990 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4034000 tx=0x7ff7b402e000 comp rx=0 tx=0).stop 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 msgr2=0x7ff7cc155cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7cc155cc0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc00d7d0 tx=0x7ff7bc00dc90 comp rx=0 tx=0).stop 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 shutdown_connections 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7a806b4d0 0x7ff7a806d990 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7cc15ba00 0x7ff7cc15be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7cc152ce0 0x7ff7cc0c9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 --2- 172.21.15.165:0/1097652508 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7ac004800 0x7ff7cc155cc0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 >> 172.21.15.165:0/1097652508 conn(0x7ff7cc061600 msgr2=0x7ff7cc0d2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 shutdown_connections 2024-04-25T10:14:56.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.014+0000 7ff7d2800640 1 -- 172.21.15.165:0/1097652508 wait complete. 2024-04-25T10:14:56.036 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:14:56.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.224+0000 7f2527200640 1 Processor -- start 2024-04-25T10:14:56.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.224+0000 7f2527200640 1 -- start start 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2527200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252015dd30 0x7f25200cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2527200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f25200b5b40 0x7f25200d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2527200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f252015e110 con 0x7f252006bff0 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2527200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f252006d000 con 0x7f25200b5b40 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2527200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25200b2fd0 con 0x7f252015dd30 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2526800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f25200b5b40 0x7f25200d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2525400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f252006bff0 0x7f25200b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53038/0 (socket says 172.21.15.165:53038) 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2526800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f25200b5b40 0x7f25200d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50666/0 (socket says 172.21.15.165:50666) 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2525400640 1 -- 172.21.15.165:0/94609512 learned_addr learned my addr 172.21.15.165:0/94609512 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:56.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.226+0000 7f2525e00640 1 --2- 172.21.15.165:0/94609512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252015dd30 0x7f25200cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.227+0000 7f2526800640 1 -- 172.21.15.165:0/94609512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252015dd30 msgr2=0x7f25200cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.227+0000 7f2526800640 1 --2- 172.21.15.165:0/94609512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252015dd30 0x7f25200cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.227+0000 7f2526800640 1 -- 172.21.15.165:0/94609512 >> v1:172.21.15.165:6789/0 conn(0x7f252006bff0 legacy=0x7f25200b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.227+0000 7f2526800640 1 -- 172.21.15.165:0/94609512 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252006c480 con 0x7f25200b5b40 2024-04-25T10:14:56.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.227+0000 7f2524a00640 1 -- 172.21.15.165:0/94609512 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2455756907 0 0) 0x7f252015e110 con 0x7f252006bff0 2024-04-25T10:14:56.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.228+0000 7f2526800640 1 --2- 172.21.15.165:0/94609512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25200b5b40 0x7f25200d2160 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f251c009d90 tx=0x7f251c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d423fc13bfc5a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.228+0000 7f2524a00640 1 -- 172.21.15.165:0/94609512 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c007740 con 0x7f25200b5b40 2024-04-25T10:14:56.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.228+0000 7f2524a00640 1 -- 172.21.15.165:0/94609512 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f251c0078e0 con 0x7f25200b5b40 2024-04-25T10:14:56.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.228+0000 7f2524a00640 1 -- 172.21.15.165:0/94609512 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c007cb0 con 0x7f25200b5b40 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.228+0000 7f2527200640 1 -- 172.21.15.165:0/94609512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25200b5b40 msgr2=0x7f25200d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.228+0000 7f2527200640 1 --2- 172.21.15.165:0/94609512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25200b5b40 0x7f25200d2160 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f251c009d90 tx=0x7f251c02a660 comp rx=0 tx=0).stop 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.229+0000 7f2527200640 1 -- 172.21.15.165:0/94609512 shutdown_connections 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.229+0000 7f2527200640 1 --2- 172.21.15.165:0/94609512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f25200b5b40 0x7f25200d2160 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.229+0000 7f2527200640 1 --2- 172.21.15.165:0/94609512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f252015dd30 0x7f25200cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.229+0000 7f2527200640 1 -- 172.21.15.165:0/94609512 >> 172.21.15.165:0/94609512 conn(0x7f25200d0610 msgr2=0x7f25200d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.229+0000 7f2527200640 1 -- 172.21.15.165:0/94609512 shutdown_connections 2024-04-25T10:14:56.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.229+0000 7f2527200640 1 -- 172.21.15.165:0/94609512 wait complete. 2024-04-25T10:14:56.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.230+0000 7f2527200640 1 Processor -- start 2024-04-25T10:14:56.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.231+0000 7f2527200640 1 -- start start 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.231+0000 7f2527200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25200b5b40 0x7f252015fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.231+0000 7f2527200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 0x7f25200b4970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.231+0000 7f2527200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25200b4eb0 0x7f2520144500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2527200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252006cd20 con 0x7f252015dd30 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2527200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252006ca40 con 0x7f25200b4eb0 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2527200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25200cf110 con 0x7f25200b5b40 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 0x7f25200b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 0x7f25200b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50678/0 (socket says 172.21.15.165:50678) 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 -- 172.21.15.165:0/2141099101 learned_addr learned my addr 172.21.15.165:0/2141099101 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:56.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525e00640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25200b5b40 0x7f252015fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2526800640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25200b4eb0 0x7f2520144500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 -- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25200b5b40 msgr2=0x7f252015fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25200b5b40 0x7f252015fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 -- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25200b4eb0 msgr2=0x7f2520144500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25200b4eb0 0x7f2520144500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.232+0000 7f2525400640 1 -- 172.21.15.165:0/2141099101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2520160400 con 0x7f252015dd30 2024-04-25T10:14:56.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.233+0000 7f2525400640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 0x7f25200b4970 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f251000c4d0 tx=0x7f251000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.233+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510018070 con 0x7f252015dd30 2024-04-25T10:14:56.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.233+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2510007500 con 0x7f252015dd30 2024-04-25T10:14:56.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.233+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2510004a20 con 0x7f252015dd30 2024-04-25T10:14:56.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.233+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25201606e0 con 0x7f252015dd30 2024-04-25T10:14:56.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.233+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2520160ca0 con 0x7f252015dd30 2024-04-25T10:14:56.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.234+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2520073990 con 0x7f252015dd30 2024-04-25T10:14:56.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.235+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2510004bc0 con 0x7f252015dd30 2024-04-25T10:14:56.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.236+0000 7f251a000640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f250406b4d0 0x7f250406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.236+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2510089610 con 0x7f252015dd30 2024-04-25T10:14:56.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.236+0000 7f2525e00640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f250406b4d0 0x7f250406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.237+0000 7f2525e00640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f250406b4d0 0x7f250406d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2514006fd0 tx=0x7f2514008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.240+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2510058390 con 0x7f252015dd30 2024-04-25T10:14:56.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.477+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f25200d16c0 con 0x7f252015dd30 2024-04-25T10:14:56.482 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:14:56.478+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:14:56.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.482+0000 7f251a000640 1 -- 172.21.15.165:0/2141099101 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v83) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2510057c90 con 0x7f252015dd30 2024-04-25T10:14:56.486 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:14:56.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.485+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f250406b4d0 msgr2=0x7f250406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.485+0000 7f2527200640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f250406b4d0 0x7f250406d990 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f2514006fd0 tx=0x7f2514008040 comp rx=0 tx=0).stop 2024-04-25T10:14:56.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.485+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 msgr2=0x7f25200b4970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.485+0000 7f2527200640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 0x7f25200b4970 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f251000c4d0 tx=0x7f251000c990 comp rx=0 tx=0).stop 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 shutdown_connections 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f250406b4d0 0x7f250406d990 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f25200b4eb0 0x7f2520144500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f252015dd30 0x7f25200b4970 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 --2- 172.21.15.165:0/2141099101 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f25200b5b40 0x7f252015fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 >> 172.21.15.165:0/2141099101 conn(0x7f25200d0610 msgr2=0x7f25200b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 shutdown_connections 2024-04-25T10:14:56.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.486+0000 7f2527200640 1 -- 172.21.15.165:0/2141099101 wait complete. 2024-04-25T10:14:56.511 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:14:56.623 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:14:56.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.697+0000 7fbc3a800640 1 Processor -- start 2024-04-25T10:14:56.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.697+0000 7fbc3a800640 1 -- start start 2024-04-25T10:14:56.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.699+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 0x7fbc34161660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.699+0000 7fbc3a800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc34162040 con 0x7fbc34161ba0 2024-04-25T10:14:56.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.699+0000 7fbc3a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc34162320 con 0x7fbc340b9750 2024-04-25T10:14:56.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.699+0000 7fbc3a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc34162600 con 0x7fbc3415da90 2024-04-25T10:14:56.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.699+0000 7fbc33e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbc34161ba0 0x7fbc340d2310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:53052/0 (socket says 172.21.15.165:53052) 2024-04-25T10:14:56.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.699+0000 7fbc33e00640 1 -- 172.21.15.165:0/3162478194 learned_addr learned my addr 172.21.15.165:0/3162478194 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:56.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc33400640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 0x7fbc34161660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3323172740 0 0) 0x7fbc34162040 con 0x7fbc34161ba0 2024-04-25T10:14:56.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbc18003520 con 0x7fbc34161ba0 2024-04-25T10:14:56.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1638853444 0 0) 0x7fbc18003520 con 0x7fbc34161ba0 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 msgr2=0x7fbc34161660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 0x7fbc34161660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 >> v1:172.21.15.172:6789/0 conn(0x7fbc340b9750 legacy=0x7fbc340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340d1b10 con 0x7fbc34161ba0 2024-04-25T10:14:56.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.700+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fbc240019f0 con 0x7fbc34161ba0 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 >> v1:172.21.15.165:6789/0 conn(0x7fbc34161ba0 legacy=0x7fbc340d2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 0x7fbc18003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc18003f80 0x7fbc18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc34162600 con 0x7fbc18004d40 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1800bb80 con 0x7fbc18003f80 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1800bd80 con 0x7fbc3415da90 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33e00640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33400640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 0x7fbc18003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32a00640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc18003f80 0x7fbc18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33e00640 1 -- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 msgr2=0x7fbc18003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33e00640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc3415da90 0x7fbc18003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33e00640 1 -- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc18003f80 msgr2=0x7fbc18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33e00640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc18003f80 0x7fbc18004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc33e00640 1 -- 172.21.15.165:0/3162478194 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc18003520 con 0x7fbc18004d40 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.701+0000 7fbc32a00640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc18003f80 0x7fbc18004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.702+0000 7fbc33e00640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc180051c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400a830 tx=0x7fbc18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e1ef9118ea248b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2400a480 con 0x7fbc18004d40 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2402f070 con 0x7fbc18004d40 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc32000640 1 -- 172.21.15.165:0/3162478194 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2402bcf0 con 0x7fbc18004d40 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc3a800640 1 -- 172.21.15.165:0/3162478194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 msgr2=0x7fbc180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc3a800640 1 --2- 172.21.15.165:0/3162478194 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc180051c0 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7fbc2400a830 tx=0x7fbc18005830 comp rx=0 tx=0).stop 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc33e00640 1 -- 172.21.15.165:0/3162478194 reap_dead start 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc3a800640 1 -- 172.21.15.165:0/3162478194 shutdown_connections 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc3a800640 1 -- 172.21.15.165:0/3162478194 >> 172.21.15.165:0/3162478194 conn(0x7fbc34061a20 msgr2=0x7fbc34061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.703+0000 7fbc3a800640 1 -- 172.21.15.165:0/3162478194 shutdown_connections 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.704+0000 7fbc3a800640 1 -- 172.21.15.165:0/3162478194 wait complete. 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.705+0000 7fbc3a800640 1 Processor -- start 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.705+0000 7fbc3a800640 1 -- start start 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.706+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc340ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.706+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc340c74d0 0x7fbc340c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.706+0000 7fbc3a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc34158640 0x7fbc34158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.706+0000 7fbc3a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340ced80 con 0x7fbc18004d40 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.706+0000 7fbc3a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc3415de70 con 0x7fbc340c74d0 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.706+0000 7fbc3a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc340cef00 con 0x7fbc34158640 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc340ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc340ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50686/0 (socket says 172.21.15.165:50686) 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 -- 172.21.15.165:0/860020506 learned_addr learned my addr 172.21.15.165:0/860020506 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 -- 172.21.15.165:0/860020506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc34158640 msgr2=0x7fbc34158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc34158640 0x7fbc34158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 -- 172.21.15.165:0/860020506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc340c74d0 msgr2=0x7fbc340c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc340c74d0 0x7fbc340c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 -- 172.21.15.165:0/860020506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc340c90e0 con 0x7fbc18004d40 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.707+0000 7fbc33400640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc340ce0b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c011630 tx=0x7fbc2c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.708+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c015870 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.708+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc2c015a10 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.708+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc2c015dc0 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.708+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc34159460 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.708+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc340c8320 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.709+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc34073a00 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.710+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc2c00f490 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.711+0000 7fbc2b400640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbc1006b500 0x7fbc1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.711+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbc2c08ad70 con 0x7fbc18004d40 2024-04-25T10:14:56.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.711+0000 7fbc32a00640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbc1006b500 0x7fbc1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:56.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.712+0000 7fbc32a00640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbc1006b500 0x7fbc1006d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbc341f6d30 tx=0x7fbc1c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:56.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.715+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc2c019030 con 0x7fbc18004d40 2024-04-25T10:14:56.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:56.955+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbc340b87a0 con 0x7fbc18004d40 2024-04-25T10:14:57.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.486+0000 7fbc2b400640 1 -- 172.21.15.165:0/860020506 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v84) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbc2c059970 con 0x7fbc18004d40 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbc1006b500 msgr2=0x7fbc1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbc1006b500 0x7fbc1006d9c0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fbc341f6d30 tx=0x7fbc1c009040 comp rx=0 tx=0).stop 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 msgr2=0x7fbc340ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc340ce0b0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fbc2c011630 tx=0x7fbc2c011af0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 shutdown_connections 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbc1006b500 0x7fbc1006d9c0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc34158640 0x7fbc34158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc340c74d0 0x7fbc340c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 --2- 172.21.15.165:0/860020506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc18004d40 0x7fbc340ce0b0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 >> 172.21.15.165:0/860020506 conn(0x7fbc34061a20 msgr2=0x7fbc340b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 shutdown_connections 2024-04-25T10:14:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.492+0000 7fbc3a800640 1 -- 172.21.15.165:0/860020506 wait complete. 2024-04-25T10:14:57.519 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:14:57.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.722+0000 7fd14da00640 1 Processor -- start 2024-04-25T10:14:57.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.722+0000 7fd14da00640 1 -- start start 2024-04-25T10:14:57.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd14da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd14815eec0 0x7fd1481689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd14da00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1480d1770 con 0x7fd14806ccd0 2024-04-25T10:14:57.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd14da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd148162740 con 0x7fd148070dc0 2024-04-25T10:14:57.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd14da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1480b0eb0 con 0x7fd14815eec0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd13fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd14806ccd0 0x7fd1480b07a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44714/0 (socket says 172.21.15.165:44714) 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd13fe00640 1 -- 172.21.15.165:0/2703619902 learned_addr learned my addr 172.21.15.165:0/2703619902 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd147400640 1 --2- 172.21.15.165:0/2703619902 >> v2:172.21.15.172:3300/0 conn(0x7fd14815eec0 0x7fd1481689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2593655040 0 0) 0x7fd1480d1770 con 0x7fd14806ccd0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.724+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd12c003520 con 0x7fd14806ccd0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1932702089 0 0) 0x7fd12c003520 con 0x7fd14806ccd0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 >> v2:172.21.15.172:3300/0 conn(0x7fd14815eec0 msgr2=0x7fd1481689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 --2- 172.21.15.165:0/2703619902 >> v2:172.21.15.172:3300/0 conn(0x7fd14815eec0 0x7fd1481689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 >> v1:172.21.15.172:6789/0 conn(0x7fd148070dc0 legacy=0x7fd1480711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd148161980 con 0x7fd14806ccd0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd130001670 con 0x7fd14806ccd0 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 >> v1:172.21.15.165:6789/0 conn(0x7fd14806ccd0 legacy=0x7fd1480b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd12c0052e0 0x7fd12c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480b0eb0 con 0x7fd12c0039b0 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c00c1b0 con 0x7fd12c004800 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd12c00c3b0 con 0x7fd12c0052e0 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.725+0000 7fd146a00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd13fe00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd147400640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd12c0052e0 0x7fd12c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd146a00640 1 -- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd12c0052e0 msgr2=0x7fd12c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd146a00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd12c0052e0 0x7fd12c00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd146a00640 1 -- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c004800 msgr2=0x7fd12c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd146a00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c004800 0x7fd12c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd146a00640 1 -- 172.21.15.165:0/2703619902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd12c003520 con 0x7fd12c0039b0 2024-04-25T10:14:57.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.726+0000 7fd147400640 1 -- 172.21.15.165:0/2703619902 reap_dead start 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.727+0000 7fd146a00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd138001720 tx=0x7fd12c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37a8ebcf87aae83 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.727+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd138001370 con 0x7fd12c0039b0 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.727+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1380052d0 con 0x7fd12c0039b0 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.727+0000 7fd146000640 1 -- 172.21.15.165:0/2703619902 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13802ab50 con 0x7fd12c0039b0 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 -- 172.21.15.165:0/2703619902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c0039b0 msgr2=0x7fd12c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd138001720 tx=0x7fd12c005830 comp rx=0 tx=0).stop 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 -- 172.21.15.165:0/2703619902 shutdown_connections 2024-04-25T10:14:57.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 --2- 172.21.15.165:0/2703619902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd12c0039b0 0x7fd12c003dd0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 -- 172.21.15.165:0/2703619902 >> 172.21.15.165:0/2703619902 conn(0x7fd1480b3530 msgr2=0x7fd1480b3940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 -- 172.21.15.165:0/2703619902 shutdown_connections 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.728+0000 7fd14da00640 1 -- 172.21.15.165:0/2703619902 wait complete. 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.730+0000 7fd14da00640 1 Processor -- start 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.730+0000 7fd14da00640 1 -- start start 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd14da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c0039b0 0x7fd1481585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd14da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 0x7fd1480cf960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd14da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1481f4ff0 0x7fd1480cfea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd14da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1480b10f0 con 0x7fd1481f4b70 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd14da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1481622b0 con 0x7fd12c0039b0 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd14da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd148161f40 con 0x7fd1481f4ff0 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd13fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 0x7fd1480cf960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd13fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 0x7fd1480cf960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49822/0 (socket says 172.21.15.165:49822) 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd13fe00640 1 -- 172.21.15.165:0/1882909784 learned_addr learned my addr 172.21.15.165:0/1882909784 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:57.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.731+0000 7fd146a00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c0039b0 0x7fd1481585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd147400640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1481f4ff0 0x7fd1480cfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13fe00640 1 -- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1481f4ff0 msgr2=0x7fd1480cfea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13fe00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1481f4ff0 0x7fd1480cfea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13fe00640 1 -- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c0039b0 msgr2=0x7fd1481585b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13fe00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c0039b0 0x7fd1481585b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13fe00640 1 -- 172.21.15.165:0/1882909784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1480cae00 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13fe00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 0x7fd1480cf960 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e730 tx=0x7fd13000ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13000ee40 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd130018070 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd13000ca90 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.732+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1480cb0e0 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.733+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1480cb6a0 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.734+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd148073a00 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.734+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd13000cc30 con 0x7fd1481f4b70 2024-04-25T10:14:57.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.735+0000 7fd13ea00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd12406b500 0x7fd12406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:57.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.736+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fd13008a1b0 con 0x7fd1481f4b70 2024-04-25T10:14:57.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.736+0000 7fd146a00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd12406b500 0x7fd12406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:57.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.737+0000 7fd146a00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd12406b500 0x7fd12406d9c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd138032000 tx=0x7fd13802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:57.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.740+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd130058e40 con 0x7fd1481f4b70 2024-04-25T10:14:57.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:57.937+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd14815c820 con 0x7fd1481f4b70 2024-04-25T10:14:58.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.490+0000 7fd13ea00640 1 -- 172.21.15.165:0/1882909784 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v126) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd130058740 con 0x7fd1481f4b70 2024-04-25T10:14:58.494 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:14:58.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd12406b500 msgr2=0x7fd12406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd12406b500 0x7fd12406d9c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fd138032000 tx=0x7fd13802e000 comp rx=0 tx=0).stop 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 msgr2=0x7fd1480cf960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 0x7fd1480cf960 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd13000e730 tx=0x7fd13000ebf0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 shutdown_connections 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd12406b500 0x7fd12406d9c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd1481f4ff0 0x7fd1480cfea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd1481f4b70 0x7fd1480cf960 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 --2- 172.21.15.165:0/1882909784 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd12c0039b0 0x7fd1481585b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 >> 172.21.15.165:0/1882909784 conn(0x7fd1480b3530 msgr2=0x7fd14815d1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 shutdown_connections 2024-04-25T10:14:58.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.494+0000 7fd14da00640 1 -- 172.21.15.165:0/1882909784 wait complete. 2024-04-25T10:14:58.524 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:14:58.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.729+0000 7f8b4fe00640 1 Processor -- start 2024-04-25T10:14:58.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.729+0000 7f8b4fe00640 1 -- start start 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8b481446d0 0x7f8b480b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b48144d50 0x7f8b480b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480cccc0 con 0x7f8b481452a0 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4fe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b480b2b20 con 0x7f8b481446d0 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4806d400 con 0x7f8b48144d50 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4d400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8b481446d0 0x7f8b480b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8b481452a0 0x7f8b480cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44716/0 (socket says 172.21.15.165:44716) 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4de00640 1 -- 172.21.15.165:0/373583461 learned_addr learned my addr 172.21.15.165:0/373583461 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:58.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.731+0000 7f8b4d400640 1 --2- 172.21.15.165:0/373583461 >> v2:172.21.15.165:3300/0 conn(0x7f8b481446d0 0x7f8b480b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49838/0 (socket says 172.21.15.165:49838) 2024-04-25T10:14:58.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.732+0000 7f8b4ca00640 1 --2- 172.21.15.165:0/373583461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b48144d50 0x7f8b480b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.732+0000 7f8b4d400640 1 -- 172.21.15.165:0/373583461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b48144d50 msgr2=0x7f8b480b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:58.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.732+0000 7f8b4d400640 1 --2- 172.21.15.165:0/373583461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b48144d50 0x7f8b480b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.732+0000 7f8b4d400640 1 -- 172.21.15.165:0/373583461 >> v1:172.21.15.165:6789/0 conn(0x7f8b481452a0 legacy=0x7f8b480cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.732+0000 7f8b4d400640 1 -- 172.21.15.165:0/373583461 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480cc140 con 0x7f8b481446d0 2024-04-25T10:14:58.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.732+0000 7f8b47e00640 1 -- 172.21.15.165:0/373583461 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1771122351 0 0) 0x7f8b480cccc0 con 0x7f8b481452a0 2024-04-25T10:14:58.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.733+0000 7f8b4d400640 1 --2- 172.21.15.165:0/373583461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481446d0 0x7f8b480b3f20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b8c98e128e15ca9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.733+0000 7f8b47e00640 1 -- 172.21.15.165:0/373583461 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c037070 con 0x7f8b481446d0 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.733+0000 7f8b47e00640 1 -- 172.21.15.165:0/373583461 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c032440 con 0x7f8b481446d0 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.733+0000 7f8b47e00640 1 -- 172.21.15.165:0/373583461 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c032810 con 0x7f8b481446d0 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 -- 172.21.15.165:0/373583461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481446d0 msgr2=0x7f8b480b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/373583461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481446d0 0x7f8b480b3f20 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c0097a0 tx=0x7f8b3c02a180 comp rx=0 tx=0).stop 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 -- 172.21.15.165:0/373583461 shutdown_connections 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/373583461 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b48144d50 0x7f8b480b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/373583461 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481446d0 0x7f8b480b3f20 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 -- 172.21.15.165:0/373583461 >> 172.21.15.165:0/373583461 conn(0x7f8b4806abe0 msgr2=0x7f8b4806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:58.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 -- 172.21.15.165:0/373583461 shutdown_connections 2024-04-25T10:14:58.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.734+0000 7f8b4fe00640 1 -- 172.21.15.165:0/373583461 wait complete. 2024-04-25T10:14:58.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.736+0000 7f8b4fe00640 1 Processor -- start 2024-04-25T10:14:58.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.736+0000 7f8b4fe00640 1 -- start start 2024-04-25T10:14:58.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b481446d0 0x7f8b480acc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b48144d50 0x7f8b480ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 0x7f8b481f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480cc700 con 0x7f8b481452a0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480b49a0 con 0x7f8b48144d50 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480cc9e0 con 0x7f8b481446d0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 0x7f8b481f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4de00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 0x7f8b481f38d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49840/0 (socket says 172.21.15.165:49840) 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.737+0000 7f8b4de00640 1 -- 172.21.15.165:0/4041582907 learned_addr learned my addr 172.21.15.165:0/4041582907 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4d400640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b481446d0 0x7f8b480acc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4ca00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b48144d50 0x7f8b480ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4de00640 1 -- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b481446d0 msgr2=0x7f8b480acc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4de00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b481446d0 0x7f8b480acc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4de00640 1 -- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b48144d50 msgr2=0x7f8b480ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4de00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b48144d50 0x7f8b480ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4de00640 1 -- 172.21.15.165:0/4041582907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480ae200 con 0x7f8b481452a0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4ca00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b48144d50 0x7f8b480ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4de00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 0x7f8b481f38d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c700 tx=0x7f8b4000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40018070 con 0x7f8b481452a0 2024-04-25T10:14:58.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4000ce10 con 0x7f8b481452a0 2024-04-25T10:14:58.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4000a860 con 0x7f8b481452a0 2024-04-25T10:14:58.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.738+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4813ce70 con 0x7f8b481452a0 2024-04-25T10:14:58.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.739+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b480adc40 con 0x7f8b481452a0 2024-04-25T10:14:58.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.740+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4813d040 con 0x7f8b481452a0 2024-04-25T10:14:58.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.741+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8b4000aa00 con 0x7f8b481452a0 2024-04-25T10:14:58.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.742+0000 7f8b45600640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:58.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.742+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f8b40089840 con 0x7f8b481452a0 2024-04-25T10:14:58.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.742+0000 7f8b4d400640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b2806b4d0 0x7f8b2806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:58.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.743+0000 7f8b4d400640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c034000 tx=0x7f8b3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:58.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.746+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8b40058680 con 0x7f8b481452a0 2024-04-25T10:14:58.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:58.944+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b48073a00 con 0x7f8b481452a0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.508+0000 7f8b45600640 1 -- 172.21.15.165:0/4041582907 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v127) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8b40057f80 con 0x7f8b481452a0 2024-04-25T10:14:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.512+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b2806b4d0 msgr2=0x7f8b2806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.512+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c034000 tx=0x7f8b3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.512+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 msgr2=0x7f8b481f38d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.512+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 0x7f8b481f38d0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f8b4000c700 tx=0x7f8b4000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 shutdown_connections 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8b2806b4d0 0x7f8b2806d990 secure :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c034000 tx=0x7f8b3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8b481452a0 0x7f8b481f38d0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8b48144d50 0x7f8b480ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 --2- 172.21.15.165:0/4041582907 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8b481446d0 0x7f8b480acc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 >> 172.21.15.165:0/4041582907 conn(0x7f8b4806abe0 msgr2=0x7f8b480b1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 shutdown_connections 2024-04-25T10:14:59.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.513+0000 7f8b4fe00640 1 -- 172.21.15.165:0/4041582907 wait complete. 2024-04-25T10:14:59.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:14:59.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.743+0000 7fe7d2000640 1 Processor -- start 2024-04-25T10:14:59.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.743+0000 7fe7d2000640 1 -- start start 2024-04-25T10:14:59.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.745+0000 7fe7d2000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe7cc1655b0 0x7fe7cc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.745+0000 7fe7d2000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe7cc0ccbe0 0x7fe7cc071d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.745+0000 7fe7d2000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7cc0cecd0 con 0x7fe7cc0d02b0 2024-04-25T10:14:59.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.745+0000 7fe7d2000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc165990 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.745+0000 7fe7d2000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7cc0cefb0 con 0x7fe7cc1655b0 2024-04-25T10:14:59.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.746+0000 7fe7caa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe7cc0ccbe0 0x7fe7cc071d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.746+0000 7fe7cbe00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe7cc0d02b0 0x7fe7cc06e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:58744/0 (socket says 172.21.15.165:58744) 2024-04-25T10:14:59.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.746+0000 7fe7cbe00640 1 -- 172.21.15.165:0/33196383 learned_addr learned my addr 172.21.15.165:0/33196383 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:14:59.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.746+0000 7fe7cb400640 1 --2- 172.21.15.165:0/33196383 >> v2:172.21.15.172:3300/0 conn(0x7fe7cc1655b0 0x7fe7cc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.747+0000 7fe7caa00640 1 -- 172.21.15.165:0/33196383 >> v2:172.21.15.172:3300/0 conn(0x7fe7cc1655b0 msgr2=0x7fe7cc0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.747+0000 7fe7caa00640 1 --2- 172.21.15.165:0/33196383 >> v2:172.21.15.172:3300/0 conn(0x7fe7cc1655b0 0x7fe7cc0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.747+0000 7fe7caa00640 1 -- 172.21.15.165:0/33196383 >> v1:172.21.15.172:6789/0 conn(0x7fe7cc0d02b0 legacy=0x7fe7cc06e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:14:59.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.747+0000 7fe7caa00640 1 -- 172.21.15.165:0/33196383 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc0cf290 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.750+0000 7fe7caa00640 1 --2- 172.21.15.165:0/33196383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc071d90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0d07b0 tx=0x7fe7b802a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8f1c6f88ef0977 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.751+0000 7fe7ca000640 1 -- 172.21.15.165:0/33196383 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8037070 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.751+0000 7fe7ca000640 1 -- 172.21.15.165:0/33196383 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b802ae20 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.751+0000 7fe7ca000640 1 -- 172.21.15.165:0/33196383 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8032660 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.753+0000 7fe7d2000640 1 -- 172.21.15.165:0/33196383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 msgr2=0x7fe7cc071d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.753+0000 7fe7d2000640 1 --2- 172.21.15.165:0/33196383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc071d90 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe7cc0d07b0 tx=0x7fe7b802a320 comp rx=0 tx=0).stop 2024-04-25T10:14:59.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.753+0000 7fe7d2000640 1 -- 172.21.15.165:0/33196383 shutdown_connections 2024-04-25T10:14:59.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.753+0000 7fe7d2000640 1 --2- 172.21.15.165:0/33196383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc071d90 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.753+0000 7fe7d2000640 1 --2- 172.21.15.165:0/33196383 >> v2:172.21.15.172:3300/0 conn(0x7fe7cc1655b0 0x7fe7cc0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.753+0000 7fe7d2000640 1 -- 172.21.15.165:0/33196383 >> 172.21.15.165:0/33196383 conn(0x7fe7cc06abe0 msgr2=0x7fe7cc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:14:59.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.754+0000 7fe7d2000640 1 -- 172.21.15.165:0/33196383 shutdown_connections 2024-04-25T10:14:59.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.754+0000 7fe7d2000640 1 -- 172.21.15.165:0/33196383 wait complete. 2024-04-25T10:14:59.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.757+0000 7fe7d2000640 1 Processor -- start 2024-04-25T10:14:59.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.757+0000 7fe7d2000640 1 -- start start 2024-04-25T10:14:59.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.759+0000 7fe7d2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc162c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.759+0000 7fe7d2000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7cc1655b0 0x7fe7cc071f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.759+0000 7fe7d2000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7cc161680 0x7fe7cc15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.759+0000 7fe7d2000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06ecb0 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.759+0000 7fe7d2000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc0cf850 con 0x7fe7cc1655b0 2024-04-25T10:14:59.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.759+0000 7fe7d2000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7cc06ee30 con 0x7fe7cc161680 2024-04-25T10:14:59.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.760+0000 7fe7cb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc162c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.760+0000 7fe7cb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc162c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49860/0 (socket says 172.21.15.165:49860) 2024-04-25T10:14:59.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.760+0000 7fe7cb400640 1 -- 172.21.15.165:0/2493688252 learned_addr learned my addr 172.21.15.165:0/2493688252 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:14:59.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7cb400640 1 -- 172.21.15.165:0/2493688252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7cc161680 msgr2=0x7fe7cc15c3e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-25T10:14:59.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7cb400640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7cc161680 0x7fe7cc15c3e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7cb400640 1 -- 172.21.15.165:0/2493688252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7cc1655b0 msgr2=0x7fe7cc071f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:14:59.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7cb400640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7cc1655b0 0x7fe7cc071f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:14:59.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7cb400640 1 -- 172.21.15.165:0/2493688252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc161b80 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7cb400640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc162c20 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00d5f0 tx=0x7fe7bc00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.761+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0125f0 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.762+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7bc019070 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.762+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7cc161e60 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.762+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7bc0129b0 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.762+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7cc162420 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.764+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7cc073a00 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.767+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe7bc012b50 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.770+0000 7fe7c3400640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7ac06b5a0 0x7fe7ac06da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:14:59.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.770+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fe7bc08b090 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.771+0000 7fe7caa00640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7ac06b5a0 0x7fe7ac06da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:14:59.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.772+0000 7fe7caa00640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7ac06b5a0 0x7fe7ac06da60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8034000 tx=0x7fe7b802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:14:59.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.774+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe7bc059fd0 con 0x7fe7cc0ccbe0 2024-04-25T10:14:59.955 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:59.961 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:14:59.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.993+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe7cc0b3500 con 0x7fe7cc0ccbe0 2024-04-25T10:15:00.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:14:59.998+0000 7fe7c3400640 1 -- 172.21.15.165:0/2493688252 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v23) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe7bc0598d0 con 0x7fe7cc0ccbe0 2024-04-25T10:15:00.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.000+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7ac06b5a0 msgr2=0x7fe7ac06da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.000+0000 7fe7d2000640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7ac06b5a0 0x7fe7ac06da60 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fe7b8034000 tx=0x7fe7b802c000 comp rx=0 tx=0).stop 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 msgr2=0x7fe7cc162c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc162c20 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe7bc00d5f0 tx=0x7fe7bc00dab0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 shutdown_connections 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe7ac06b5a0 0x7fe7ac06da60 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7cc161680 0x7fe7cc15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7cc1655b0 0x7fe7cc071f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 --2- 172.21.15.165:0/2493688252 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe7cc0ccbe0 0x7fe7cc162c20 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 >> 172.21.15.165:0/2493688252 conn(0x7fe7cc06abe0 msgr2=0x7fe7cc1647d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 shutdown_connections 2024-04-25T10:15:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.001+0000 7fe7d2000640 1 -- 172.21.15.165:0/2493688252 wait complete. 2024-04-25T10:15:00.022 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort' 2024-04-25T10:15:00.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.294+0000 7f0ff4c00640 1 Processor -- start 2024-04-25T10:15:00.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.295+0000 7f0ff4c00640 1 -- start start 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0ff4c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ff015db90 0x7f0ff00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0ff4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff01615a0 0x7f0ff00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0ff4c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0ff00d04f0 0x7f0ff0061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0ff4c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00d0980 con 0x7f0ff015db90 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0ff4c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff00d1f50 con 0x7f0ff00d04f0 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0ff4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0161df0 con 0x7f0ff01615a0 2024-04-25T10:15:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.296+0000 7f0fef400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ff015db90 0x7f0ff00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0ff015db90 0x7f0ff00b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49868/0 (socket says 172.21.15.165:49868) 2024-04-25T10:15:00.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 -- 172.21.15.165:0/2935390306 learned_addr learned my addr 172.21.15.165:0/2935390306 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fe6a00640 1 --2- 172.21.15.165:0/2935390306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff01615a0 0x7f0ff00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fefe00640 1 --2- 172.21.15.165:0/2935390306 >> v2:172.21.15.172:3300/0 conn(0x7f0ff00d04f0 0x7f0ff0061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 -- 172.21.15.165:0/2935390306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff01615a0 msgr2=0x7f0ff00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 --2- 172.21.15.165:0/2935390306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff01615a0 0x7f0ff00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 -- 172.21.15.165:0/2935390306 >> v2:172.21.15.172:3300/0 conn(0x7f0ff00d04f0 msgr2=0x7f0ff0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 --2- 172.21.15.165:0/2935390306 >> v2:172.21.15.172:3300/0 conn(0x7f0ff00d04f0 0x7f0ff0061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.297+0000 7f0fef400640 1 -- 172.21.15.165:0/2935390306 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff01620d0 con 0x7f0ff015db90 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.299+0000 7f0fef400640 1 --2- 172.21.15.165:0/2935390306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff015db90 0x7f0ff00b93c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0ff008bcb0 tx=0x7f0fe002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bafd14c58edf9d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.299+0000 7f0feea00640 1 -- 172.21.15.165:0/2935390306 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0037070 con 0x7f0ff015db90 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.299+0000 7f0feea00640 1 -- 172.21.15.165:0/2935390306 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe0032440 con 0x7f0ff015db90 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.299+0000 7f0feea00640 1 -- 172.21.15.165:0/2935390306 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe0032810 con 0x7f0ff015db90 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2935390306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff015db90 msgr2=0x7f0ff00b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2935390306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff015db90 0x7f0ff00b93c0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f0ff008bcb0 tx=0x7f0fe002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2935390306 shutdown_connections 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2935390306 >> v2:172.21.15.172:3300/0 conn(0x7f0ff00d04f0 0x7f0ff0061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2935390306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff01615a0 0x7f0ff00cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2935390306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff015db90 0x7f0ff00b93c0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2935390306 >> 172.21.15.165:0/2935390306 conn(0x7f0ff00b52b0 msgr2=0x7f0ff00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2935390306 shutdown_connections 2024-04-25T10:15:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.300+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2935390306 wait complete. 2024-04-25T10:15:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.302+0000 7f0ff4c00640 1 Processor -- start 2024-04-25T10:15:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.302+0000 7f0ff4c00640 1 -- start start 2024-04-25T10:15:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.303+0000 7f0ff4c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff00d04f0 0x7f0ff00cd060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.303+0000 7f0ff4c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff015db90 0x7f0ff00cbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.303+0000 7f0ff4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 0x7f0ff015a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.303+0000 7f0ff4c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0161a20 con 0x7f0ff01615a0 2024-04-25T10:15:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.303+0000 7f0ff4c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff00d1be0 con 0x7f0ff00d04f0 2024-04-25T10:15:00.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.303+0000 7f0ff4c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff0161ba0 con 0x7f0ff015db90 2024-04-25T10:15:00.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 0x7f0ff015a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 0x7f0ff015a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49882/0 (socket says 172.21.15.165:49882) 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 -- 172.21.15.165:0/2909481002 learned_addr learned my addr 172.21.15.165:0/2909481002 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fef400640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff00d04f0 0x7f0ff00cd060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fe6a00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff015db90 0x7f0ff00cbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 -- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff015db90 msgr2=0x7f0ff00cbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff015db90 0x7f0ff00cbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 -- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff00d04f0 msgr2=0x7f0ff00cd060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff00d04f0 0x7f0ff00cd060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 -- 172.21.15.165:0/2909481002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff015aea0 con 0x7f0ff01615a0 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fe6a00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff015db90 0x7f0ff00cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:00.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.304+0000 7f0fefe00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 0x7f0ff015a960 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8007d10 tx=0x7f0fe800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.305+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8014420 con 0x7f0ff01615a0 2024-04-25T10:15:00.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.305+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe8019070 con 0x7f0ff01615a0 2024-04-25T10:15:00.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.305+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff00cd640 con 0x7f0ff01615a0 2024-04-25T10:15:00.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.305+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8014820 con 0x7f0ff01615a0 2024-04-25T10:15:00.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.305+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff00cb7b0 con 0x7f0ff01615a0 2024-04-25T10:15:00.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.306+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff0073990 con 0x7f0ff01615a0 2024-04-25T10:15:00.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.307+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0fe80149c0 con 0x7f0ff01615a0 2024-04-25T10:15:00.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.308+0000 7f0fe7e00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0fcc06b4d0 0x7f0fcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:00.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.308+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0fe808a760 con 0x7f0ff01615a0 2024-04-25T10:15:00.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.309+0000 7f0fef400640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0fcc06b4d0 0x7f0fcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:00.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.309+0000 7f0fef400640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0fcc06b4d0 0x7f0fcc06d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0034000 tx=0x7f0fe002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:00.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.312+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0fe80597e0 con 0x7f0ff01615a0 2024-04-25T10:15:00.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:00.580+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]} v 0) v1 -- 0x7f0ff00d14b0 con 0x7f0ff01615a0 2024-04-25T10:15:01.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.514+0000 7f0fe7e00640 1 -- 172.21.15.165:0/2909481002 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_pause_and_resume_with_abort"]}]=0 v598) v1 ==== 166+0+0 (secure 0 0 0) 0x7f0fe80590e0 con 0x7f0ff01615a0 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0fcc06b4d0 msgr2=0x7f0fcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0fcc06b4d0 0x7f0fcc06d990 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0034000 tx=0x7f0fe002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 msgr2=0x7f0ff015a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 0x7f0ff015a960 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8007d10 tx=0x7f0fe800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 shutdown_connections 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0fcc06b4d0 0x7f0fcc06d990 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0ff01615a0 0x7f0ff015a960 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0ff015db90 0x7f0ff00cbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 --2- 172.21.15.165:0/2909481002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0ff00d04f0 0x7f0ff00cd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 >> 172.21.15.165:0/2909481002 conn(0x7f0ff00b52b0 msgr2=0x7f0ff00d23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 shutdown_connections 2024-04-25T10:15:01.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.518+0000 7f0ff4c00640 1 -- 172.21.15.165:0/2909481002 wait complete. 2024-04-25T10:15:01.542 INFO:tasks.cephfs_test_runner:test_scrub_pause_and_resume_with_abort (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:15:01.543 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:15:01.543 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:01.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.741+0000 7fc343200640 1 Processor -- start 2024-04-25T10:15:01.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.741+0000 7fc343200640 1 -- start start 2024-04-25T10:15:01.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.742+0000 7fc343200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 0x7fc344070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc343200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34415bf30 con 0x7fc344071530 2024-04-25T10:15:01.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc343200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34415c210 con 0x7fc3440cda30 2024-04-25T10:15:01.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc343200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415c860 con 0x7fc344144440 2024-04-25T10:15:01.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc33be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc344071530 0x7fc3440cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44720/0 (socket says 172.21.15.165:44720) 2024-04-25T10:15:01.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc33be00640 1 -- 172.21.15.165:0/2432282059 learned_addr learned my addr 172.21.15.165:0/2432282059 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc341e00640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 0x7fc344070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392061281 0 0) 0x7fc34415bf30 con 0x7fc344071530 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.743+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc328003520 con 0x7fc344071530 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 37833534 0 0) 0x7fc328003520 con 0x7fc344071530 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 msgr2=0x7fc344070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 0x7fc344070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 >> v1:172.21.15.172:6789/0 conn(0x7fc3440cda30 legacy=0x7fc34415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34415c4f0 con 0x7fc344071530 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc32c001670 con 0x7fc344071530 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 >> v1:172.21.15.165:6789/0 conn(0x7fc344071530 legacy=0x7fc3440cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3280039e0 0x7fc328003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328004800 0x7fc328004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 0x7fc3280051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34415c860 con 0x7fc3280039e0 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc32800bb30 con 0x7fc328004800 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc32800bd30 con 0x7fc344144440 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.744+0000 7fc341e00640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3280039e0 0x7fc328003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.745+0000 7fc33be00640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328004800 0x7fc328004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.745+0000 7fc341e00640 1 -- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 msgr2=0x7fc3280051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.745+0000 7fc341e00640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc344144440 0x7fc3280051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.745+0000 7fc341e00640 1 -- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328004800 msgr2=0x7fc328004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.745+0000 7fc341e00640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc328004800 0x7fc328004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.745+0000 7fc341e00640 1 -- 172.21.15.165:0/2432282059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc328003520 con 0x7fc3280039e0 2024-04-25T10:15:01.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc341e00640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3280039e0 0x7fc328003e00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fc334009a90 tx=0x7fc328005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95d874d6145dad1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334036070 con 0x7fc3280039e0 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc334031400 con 0x7fc3280039e0 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc341400640 1 -- 172.21.15.165:0/2432282059 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc334009690 con 0x7fc3280039e0 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc343200640 1 -- 172.21.15.165:0/2432282059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3280039e0 msgr2=0x7fc328003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc343200640 1 --2- 172.21.15.165:0/2432282059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc3280039e0 0x7fc328003e00 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7fc334009a90 tx=0x7fc328005830 comp rx=0 tx=0).stop 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.746+0000 7fc342800640 1 -- 172.21.15.165:0/2432282059 reap_dead start 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.747+0000 7fc343200640 1 -- 172.21.15.165:0/2432282059 shutdown_connections 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.747+0000 7fc343200640 1 -- 172.21.15.165:0/2432282059 >> 172.21.15.165:0/2432282059 conn(0x7fc34406abe0 msgr2=0x7fc34406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.747+0000 7fc343200640 1 -- 172.21.15.165:0/2432282059 shutdown_connections 2024-04-25T10:15:01.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.747+0000 7fc343200640 1 -- 172.21.15.165:0/2432282059 wait complete. 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.748+0000 7fc343200640 1 Processor -- start 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.748+0000 7fc343200640 1 -- start start 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.749+0000 7fc343200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3280039e0 0x7fc344156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.749+0000 7fc343200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34413b1e0 0x7fc34413fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.749+0000 7fc343200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 0x7fc344140040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.749+0000 7fc343200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3440cdea0 con 0x7fc34415b370 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.749+0000 7fc343200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc344071970 con 0x7fc34413b1e0 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.749+0000 7fc343200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc344070ff0 con 0x7fc3280039e0 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 0x7fc344140040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 0x7fc344140040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49892/0 (socket says 172.21.15.165:49892) 2024-04-25T10:15:01.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 -- 172.21.15.165:0/3324809945 learned_addr learned my addr 172.21.15.165:0/3324809945 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc33be00640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34413b1e0 0x7fc34413fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc341e00640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3280039e0 0x7fc344156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 -- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3280039e0 msgr2=0x7fc344156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3280039e0 0x7fc344156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 -- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34413b1e0 msgr2=0x7fc34413fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34413b1e0 0x7fc34413fb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.750+0000 7fc342800640 1 -- 172.21.15.165:0/3324809945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34413b670 con 0x7fc34415b370 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.751+0000 7fc342800640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 0x7fc344140040 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fc344072490 tx=0x7fc33c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.751+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c017070 con 0x7fc34415b370 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.751+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33c005d60 con 0x7fc34415b370 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.751+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc33c00a850 con 0x7fc34415b370 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.751+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34413b950 con 0x7fc34415b370 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.751+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34413bf10 con 0x7fc34415b370 2024-04-25T10:15:01.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.752+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344073990 con 0x7fc34415b370 2024-04-25T10:15:01.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.753+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc33c01a070 con 0x7fc34415b370 2024-04-25T10:15:01.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.754+0000 7fc33a000640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc32006b4d0 0x7fc32006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:01.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.754+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc33c089490 con 0x7fc34415b370 2024-04-25T10:15:01.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.754+0000 7fc341e00640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc32006b4d0 0x7fc32006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:01.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.755+0000 7fc341e00640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc32006b4d0 0x7fc32006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc334033000 tx=0x7fc334008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:01.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.759+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc33c0583e0 con 0x7fc34415b370 2024-04-25T10:15:01.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.953+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc344142850 con 0x7fc34415b370 2024-04-25T10:15:01.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.955+0000 7fc33a000640 1 -- 172.21.15.165:0/3324809945 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+13341 (secure 0 0 0) 0x7fc33c057ce0 con 0x7fc34415b370 2024-04-25T10:15:01.959 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:01.959 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":127,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:14:59.505021+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6834/3899404305":"2024-04-26T10:14:56.479095+0000","172.21.15.172:6832/3899404305":"2024-04-26T10:14:56.479095+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc32006b4d0 msgr2=0x7fc32006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc32006b4d0 0x7fc32006d990 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc334033000 tx=0x7fc334008030 comp rx=0 tx=0).stop 2024-04-25T10:15:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 msgr2=0x7fc344140040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:01.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 0x7fc344140040 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fc344072490 tx=0x7fc33c00cb90 comp rx=0 tx=0).stop 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 shutdown_connections 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc32006b4d0 0x7fc32006d990 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc34415b370 0x7fc344140040 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc34413b1e0 0x7fc34413fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 --2- 172.21.15.165:0/3324809945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc3280039e0 0x7fc344156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 >> 172.21.15.165:0/3324809945 conn(0x7fc34406abe0 msgr2=0x7fc344143050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 shutdown_connections 2024-04-25T10:15:01.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:01.958+0000 7fc343200640 1 -- 172.21.15.165:0/3324809945 wait complete. 2024-04-25T10:15:01.980 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:01.980 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:02.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.165+0000 7ff682200640 1 Processor -- start 2024-04-25T10:15:02.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.165+0000 7ff682200640 1 -- start start 2024-04-25T10:15:02.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.166+0000 7ff682200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0ceb20 0x7ff67c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.166+0000 7ff682200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff67c0c5030 0x7ff67c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.166+0000 7ff682200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c145520 con 0x7ff67c145140 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.166+0000 7ff682200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c071270 con 0x7ff67c0c5030 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.166+0000 7ff682200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c071550 con 0x7ff67c0ceb20 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff680e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff67c145140 0x7ff67c0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44730/0 (socket says 172.21.15.165:44730) 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff680e00640 1 -- 172.21.15.165:0/672170828 learned_addr learned my addr 172.21.15.165:0/672170828 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff681800640 1 --2- 172.21.15.165:0/672170828 >> v2:172.21.15.165:3300/0 conn(0x7ff67c0c5030 0x7ff67c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff67be00640 1 -- 172.21.15.165:0/672170828 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 900745296 0 0) 0x7ff67c145520 con 0x7ff67c145140 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff67be00640 1 -- 172.21.15.165:0/672170828 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff660003520 con 0x7ff67c145140 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff673e00640 1 --2- 172.21.15.165:0/672170828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0ceb20 0x7ff67c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff681800640 1 -- 172.21.15.165:0/672170828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0ceb20 msgr2=0x7ff67c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff681800640 1 --2- 172.21.15.165:0/672170828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0ceb20 0x7ff67c0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff681800640 1 -- 172.21.15.165:0/672170828 >> v1:172.21.15.165:6789/0 conn(0x7ff67c145140 legacy=0x7ff67c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.167+0000 7ff681800640 1 -- 172.21.15.165:0/672170828 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c071830 con 0x7ff67c0c5030 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.168+0000 7ff681800640 1 --2- 172.21.15.165:0/672170828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c15bfb0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff674009d90 tx=0x7ff67402a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e7d46f4ebad7f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.168+0000 7ff67be00640 1 -- 172.21.15.165:0/672170828 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674031600 con 0x7ff67c0c5030 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.168+0000 7ff67be00640 1 -- 172.21.15.165:0/672170828 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6740317a0 con 0x7ff67c0c5030 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.168+0000 7ff67be00640 1 -- 172.21.15.165:0/672170828 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674031b70 con 0x7ff67c0c5030 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 -- 172.21.15.165:0/672170828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 msgr2=0x7ff67c15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 --2- 172.21.15.165:0/672170828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c15bfb0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7ff674009d90 tx=0x7ff67402a5d0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 -- 172.21.15.165:0/672170828 shutdown_connections 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 --2- 172.21.15.165:0/672170828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c15bfb0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 --2- 172.21.15.165:0/672170828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c0ceb20 0x7ff67c0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 -- 172.21.15.165:0/672170828 >> 172.21.15.165:0/672170828 conn(0x7ff67c061cc0 msgr2=0x7ff67c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 -- 172.21.15.165:0/672170828 shutdown_connections 2024-04-25T10:15:02.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.169+0000 7ff682200640 1 -- 172.21.15.165:0/672170828 wait complete. 2024-04-25T10:15:02.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.171+0000 7ff682200640 1 Processor -- start 2024-04-25T10:15:02.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.171+0000 7ff682200640 1 -- start start 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.171+0000 7ff682200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c13e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.171+0000 7ff682200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c0ceb20 0x7ff67c13c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff682200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c145140 0x7ff67c15adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff682200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c5640 con 0x7ff67c0c5030 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff682200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c0c54c0 con 0x7ff67c0ceb20 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff682200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c15c4f0 con 0x7ff67c145140 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c13e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c13e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49916/0 (socket says 172.21.15.165:49916) 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 -- 172.21.15.165:0/1180754410 learned_addr learned my addr 172.21.15.165:0/1180754410 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff681800640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c145140 0x7ff67c15adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 -- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c145140 msgr2=0x7ff67c15adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c145140 0x7ff67c15adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 -- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c0ceb20 msgr2=0x7ff67c13c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff673e00640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c0ceb20 0x7ff67c13c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c0ceb20 0x7ff67c13c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.172+0000 7ff680e00640 1 -- 172.21.15.165:0/1180754410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c13d0c0 con 0x7ff67c0c5030 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.173+0000 7ff680e00640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c13e160 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c390 tx=0x7ff66c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.173+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c017070 con 0x7ff67c0c5030 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.173+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c005ce0 con 0x7ff67c0c5030 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.173+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0072f0 con 0x7ff67c0c5030 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.173+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c13d3a0 con 0x7ff67c0c5030 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.173+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c13d960 con 0x7ff67c0c5030 2024-04-25T10:15:02.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.174+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c073990 con 0x7ff67c0c5030 2024-04-25T10:15:02.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.175+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff66c01a070 con 0x7ff67c0c5030 2024-04-25T10:15:02.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.176+0000 7ff679600640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff65806b4d0 0x7ff65806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.176+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff66c089540 con 0x7ff67c0c5030 2024-04-25T10:15:02.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.177+0000 7ff673e00640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff65806b4d0 0x7ff65806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.177+0000 7ff673e00640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff65806b4d0 0x7ff65806d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff664006fd0 tx=0x7ff664008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.180+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff66c058490 con 0x7ff67c0c5030 2024-04-25T10:15:02.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.360+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff67c15e8c0 con 0x7ff67c0c5030 2024-04-25T10:15:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.361+0000 7ff679600640 1 -- 172.21.15.165:0/1180754410 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v127) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff66c057d90 con 0x7ff67c0c5030 2024-04-25T10:15:02.365 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:15:02.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff65806b4d0 msgr2=0x7ff65806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff65806b4d0 0x7ff65806d990 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7ff664006fd0 tx=0x7ff664008040 comp rx=0 tx=0).stop 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 msgr2=0x7ff67c13e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c13e160 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7ff66c00c390 tx=0x7ff66c00c850 comp rx=0 tx=0).stop 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 shutdown_connections 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff65806b4d0 0x7ff65806d990 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff67c145140 0x7ff67c15adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff67c0ceb20 0x7ff67c13c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 --2- 172.21.15.165:0/1180754410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff67c0c5030 0x7ff67c13e160 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 >> 172.21.15.165:0/1180754410 conn(0x7ff67c061cc0 msgr2=0x7ff67c0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 shutdown_connections 2024-04-25T10:15:02.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.364+0000 7ff682200640 1 -- 172.21.15.165:0/1180754410 wait complete. 2024-04-25T10:15:02.386 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:15:02.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.581+0000 7fcfc6600640 1 Processor -- start 2024-04-25T10:15:02.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.581+0000 7fcfc6600640 1 -- start start 2024-04-25T10:15:02.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.583+0000 7fcfc6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.583+0000 7fcfc6600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfc0071530 0x7fcfc00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.583+0000 7fcfc6600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfc015bf30 con 0x7fcfc00cd860 2024-04-25T10:15:02.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.583+0000 7fcfc6600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfc015c860 con 0x7fcfc0071530 2024-04-25T10:15:02.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.583+0000 7fcfc6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc0071970 con 0x7fcfc0144440 2024-04-25T10:15:02.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.584+0000 7fcfc5200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.584+0000 7fcfb7e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcfc0071530 0x7fcfc00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.584+0000 7fcfc5200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc0070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37476/0 (socket says 172.21.15.165:37476) 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.584+0000 7fcfc5200640 1 -- 172.21.15.165:0/1033958296 learned_addr learned my addr 172.21.15.165:0/1033958296 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.585+0000 7fcfb7e00640 1 -- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 msgr2=0x7fcfc0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.585+0000 7fcfb7e00640 1 --2- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc0070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.585+0000 7fcfb7e00640 1 -- 172.21.15.165:0/1033958296 >> v1:172.21.15.172:6789/0 conn(0x7fcfc00cd860 legacy=0x7fcfc006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.585+0000 7fcfb7e00640 1 -- 172.21.15.165:0/1033958296 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc015c210 con 0x7fcfc0071530 2024-04-25T10:15:02.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.585+0000 7fcfc5200640 1 --2- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc0070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:02.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.586+0000 7fcfb7e00640 1 --2- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc00cd320 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097a0 tx=0x7fcfa802a210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=552cf4441ffec59f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.587+0000 7fcfbfe00640 1 -- 172.21.15.165:0/1033958296 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8037070 con 0x7fcfc0071530 2024-04-25T10:15:02.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.587+0000 7fcfbfe00640 1 -- 172.21.15.165:0/1033958296 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8032440 con 0x7fcfc0071530 2024-04-25T10:15:02.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.587+0000 7fcfbfe00640 1 -- 172.21.15.165:0/1033958296 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8032810 con 0x7fcfc0071530 2024-04-25T10:15:02.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.587+0000 7fcfc6600640 1 -- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 msgr2=0x7fcfc00cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.587+0000 7fcfc6600640 1 --2- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc00cd320 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fcfa80097a0 tx=0x7fcfa802a210 comp rx=0 tx=0).stop 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.588+0000 7fcfc6600640 1 -- 172.21.15.165:0/1033958296 shutdown_connections 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.588+0000 7fcfc6600640 1 --2- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc00cd320 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.588+0000 7fcfc6600640 1 --2- 172.21.15.165:0/1033958296 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc0070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.588+0000 7fcfc6600640 1 -- 172.21.15.165:0/1033958296 >> 172.21.15.165:0/1033958296 conn(0x7fcfc006abe0 msgr2=0x7fcfc006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.588+0000 7fcfc6600640 1 -- 172.21.15.165:0/1033958296 shutdown_connections 2024-04-25T10:15:02.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.588+0000 7fcfc6600640 1 -- 172.21.15.165:0/1033958296 wait complete. 2024-04-25T10:15:02.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.589+0000 7fcfc6600640 1 Processor -- start 2024-04-25T10:15:02.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 -- start start 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc013ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 0x7fcfc01f3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc01e6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cdcd0 con 0x7fcfc00cd860 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc015bc30 con 0x7fcfc0071530 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.590+0000 7fcfc6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfc00cde50 con 0x7fcfc0144440 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 0x7fcfc01f3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 0x7fcfc01f3780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49922/0 (socket says 172.21.15.165:49922) 2024-04-25T10:15:02.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 -- 172.21.15.165:0/2257494315 learned_addr learned my addr 172.21.15.165:0/2257494315 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfc5200640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc013ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 -- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 msgr2=0x7fcfc01e6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc01e6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 -- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 msgr2=0x7fcfc013ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc013ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.591+0000 7fcfb7e00640 1 -- 172.21.15.165:0/2257494315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc01e1d70 con 0x7fcfc00cd860 2024-04-25T10:15:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.592+0000 7fcfb7e00640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 0x7fcfc01f3780 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcfa802a250 tx=0x7fcfa80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.592+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8041020 con 0x7fcfc00cd860 2024-04-25T10:15:02.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.592+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfa8030700 con 0x7fcfc00cd860 2024-04-25T10:15:02.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.592+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8037040 con 0x7fcfc00cd860 2024-04-25T10:15:02.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.592+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc01e2050 con 0x7fcfc00cd860 2024-04-25T10:15:02.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.592+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfc01e2610 con 0x7fcfc00cd860 2024-04-25T10:15:02.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.593+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfc0073990 con 0x7fcfc00cd860 2024-04-25T10:15:02.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.594+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcfa8044020 con 0x7fcfc00cd860 2024-04-25T10:15:02.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.595+0000 7fcfbd600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf9c06b4d0 0x7fcf9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:02.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.596+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcfa80aa7e0 con 0x7fcfc00cd860 2024-04-25T10:15:02.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.596+0000 7fcfc5200640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf9c06b4d0 0x7fcf9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:02.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.596+0000 7fcfc5200640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf9c06b4d0 0x7fcf9c06d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00099e0 tx=0x7fcfb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:02.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.600+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcfa804b030 con 0x7fcfc00cd860 2024-04-25T10:15:02.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:02.871+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7fcfc006eaa0 con 0x7fcfc00cd860 2024-04-25T10:15:03.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.528+0000 7fcfbd600640 1 -- 172.21.15.165:0/2257494315 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v600) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcfa8079860 con 0x7fcfc00cd860 2024-04-25T10:15:03.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf9c06b4d0 msgr2=0x7fcf9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf9c06b4d0 0x7fcf9c06d990 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fcfb00099e0 tx=0x7fcfb0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:03.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 msgr2=0x7fcfc01f3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 0x7fcfc01f3780 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7fcfa802a250 tx=0x7fcfa80046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 shutdown_connections 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf9c06b4d0 0x7fcf9c06d990 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcfc0144440 0x7fcfc01e6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcfc00cd860 0x7fcfc01f3780 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 --2- 172.21.15.165:0/2257494315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcfc0071530 0x7fcfc013ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.531+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 >> 172.21.15.165:0/2257494315 conn(0x7fcfc006abe0 msgr2=0x7fcfc0142d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.532+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 shutdown_connections 2024-04-25T10:15:03.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.532+0000 7fcfc6600640 1 -- 172.21.15.165:0/2257494315 wait complete. 2024-04-25T10:15:03.557 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:15:03.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.757+0000 7f5d48400640 1 Processor -- start 2024-04-25T10:15:03.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.757+0000 7f5d48400640 1 -- start start 2024-04-25T10:15:03.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d48400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5d400cc4f0 0x7f5d400cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d48400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5d4015b980 0x7f5d400cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d48400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d40072280 con 0x7f5d400cdb20 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d48400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d40071270 con 0x7f5d4015b980 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d48400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d4015bd80 con 0x7f5d400cc4f0 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d45000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5d400cdb20 0x7f5d4015b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44738/0 (socket says 172.21.15.165:44738) 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d45000640 1 -- 172.21.15.165:0/687877765 learned_addr learned my addr 172.21.15.165:0/687877765 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:03.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d46400640 1 --2- 172.21.15.165:0/687877765 >> v2:172.21.15.165:3300/0 conn(0x7f5d4015b980 0x7f5d400cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.759+0000 7f5d45a00640 1 --2- 172.21.15.165:0/687877765 >> v2:172.21.15.172:3300/0 conn(0x7f5d400cc4f0 0x7f5d400cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.760+0000 7f5d3fe00640 1 -- 172.21.15.165:0/687877765 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3523983612 0 0) 0x7f5d40072280 con 0x7f5d400cdb20 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.760+0000 7f5d3fe00640 1 -- 172.21.15.165:0/687877765 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d28003520 con 0x7f5d400cdb20 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.760+0000 7f5d46400640 1 -- 172.21.15.165:0/687877765 >> v2:172.21.15.172:3300/0 conn(0x7f5d400cc4f0 msgr2=0x7f5d400cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.760+0000 7f5d46400640 1 --2- 172.21.15.165:0/687877765 >> v2:172.21.15.172:3300/0 conn(0x7f5d400cc4f0 0x7f5d400cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.760+0000 7f5d46400640 1 -- 172.21.15.165:0/687877765 >> v1:172.21.15.165:6789/0 conn(0x7f5d400cdb20 legacy=0x7f5d4015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.760+0000 7f5d46400640 1 -- 172.21.15.165:0/687877765 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400714c0 con 0x7f5d4015b980 2024-04-25T10:15:03.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.761+0000 7f5d46400640 1 --2- 172.21.15.165:0/687877765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400cdfb0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f5d400cd290 tx=0x7f5d3802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab524b42224f4400 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.761+0000 7f5d3fe00640 1 -- 172.21.15.165:0/687877765 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007740 con 0x7f5d4015b980 2024-04-25T10:15:03.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.761+0000 7f5d3fe00640 1 -- 172.21.15.165:0/687877765 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d380078e0 con 0x7f5d4015b980 2024-04-25T10:15:03.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.761+0000 7f5d3fe00640 1 -- 172.21.15.165:0/687877765 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38007cb0 con 0x7f5d4015b980 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 -- 172.21.15.165:0/687877765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 msgr2=0x7f5d400cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 --2- 172.21.15.165:0/687877765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400cdfb0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f5d400cd290 tx=0x7f5d3802a680 comp rx=0 tx=0).stop 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 -- 172.21.15.165:0/687877765 shutdown_connections 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 --2- 172.21.15.165:0/687877765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400cdfb0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 --2- 172.21.15.165:0/687877765 >> v2:172.21.15.172:3300/0 conn(0x7f5d400cc4f0 0x7f5d400cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 -- 172.21.15.165:0/687877765 >> 172.21.15.165:0/687877765 conn(0x7f5d4006abe0 msgr2=0x7f5d4006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 -- 172.21.15.165:0/687877765 shutdown_connections 2024-04-25T10:15:03.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.762+0000 7f5d48400640 1 -- 172.21.15.165:0/687877765 wait complete. 2024-04-25T10:15:03.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.764+0000 7f5d48400640 1 Processor -- start 2024-04-25T10:15:03.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.764+0000 7f5d48400640 1 -- start start 2024-04-25T10:15:03.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d48400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d400cc4f0 0x7f5d4014b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d48400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d400cdb20 0x7f5d400afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d48400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400b03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d48400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40071d60 con 0x7f5d4015b980 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d48400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d40071a80 con 0x7f5d400cdb20 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d48400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d400ae500 con 0x7f5d400cc4f0 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d46400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400b03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d46400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400b03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49940/0 (socket says 172.21.15.165:49940) 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.765+0000 7f5d46400640 1 -- 172.21.15.165:0/2155085985 learned_addr learned my addr 172.21.15.165:0/2155085985 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d45000640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d400cdb20 0x7f5d400afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d45a00640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d400cc4f0 0x7f5d4014b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d46400640 1 -- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d400cc4f0 msgr2=0x7f5d4014b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d46400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d400cc4f0 0x7f5d4014b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d46400640 1 -- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d400cdb20 msgr2=0x7f5d400afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d46400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d400cdb20 0x7f5d400afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d46400640 1 -- 172.21.15.165:0/2155085985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d400b0cd0 con 0x7f5d4015b980 2024-04-25T10:15:03.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d45000640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d400cdb20 0x7f5d400afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:03.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d46400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400b03f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f5d3802ac70 tx=0x7f5d38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38040070 con 0x7f5d4015b980 2024-04-25T10:15:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d38031bc0 con 0x7f5d4015b980 2024-04-25T10:15:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.766+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d38009450 con 0x7f5d4015b980 2024-04-25T10:15:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.767+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d400b0fb0 con 0x7f5d4015b980 2024-04-25T10:15:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.767+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d400b1570 con 0x7f5d4015b980 2024-04-25T10:15:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.768+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d40073a00 con 0x7f5d4015b980 2024-04-25T10:15:03.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.769+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5d38039040 con 0x7f5d4015b980 2024-04-25T10:15:03.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.771+0000 7f5d3d600640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5d2406b4d0 0x7f5d2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:03.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.771+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5d380ab8a0 con 0x7f5d4015b980 2024-04-25T10:15:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.771+0000 7f5d45a00640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5d2406b4d0 0x7f5d2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:03.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.772+0000 7f5d45a00640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5d2406b4d0 0x7f5d2406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009970 tx=0x7f5d34008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:03.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.775+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5d38043360 con 0x7f5d4015b980 2024-04-25T10:15:03.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.972+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5d400cd220 con 0x7f5d4015b980 2024-04-25T10:15:03.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.977+0000 7f5d3d600640 1 -- 172.21.15.165:0/2155085985 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v24)=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5d3807a9a0 con 0x7f5d4015b980 2024-04-25T10:15:03.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.979+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5d2406b4d0 msgr2=0x7f5d2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.979+0000 7f5d48400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5d2406b4d0 0x7f5d2406d990 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f5d34009970 tx=0x7f5d34008040 comp rx=0 tx=0).stop 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.979+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 msgr2=0x7f5d400b03f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.979+0000 7f5d48400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400b03f0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f5d3802ac70 tx=0x7f5d38004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 shutdown_connections 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5d2406b4d0 0x7f5d2406d990 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5d4015b980 0x7f5d400b03f0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5d400cdb20 0x7f5d400afeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 --2- 172.21.15.165:0/2155085985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5d400cc4f0 0x7f5d4014b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 >> 172.21.15.165:0/2155085985 conn(0x7f5d4006abe0 msgr2=0x7f5d400ac8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 shutdown_connections 2024-04-25T10:15:03.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:03.980+0000 7f5d48400640 1 -- 172.21.15.165:0/2155085985 wait complete. 2024-04-25T10:15:04.005 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:15:04.005 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:04.107 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:15:04.212 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:15:04.212 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:15:04.212 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:04.270 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:04.270 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:15:04.270 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:04.326 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:04.326 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:15:04.326 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:04.383 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:15:04.383 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:15:04.383 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:04.440 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:15:04.441 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:15:04.441 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.639+0000 7f765e200640 1 Processor -- start 2024-04-25T10:15:04.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.639+0000 7f765e200640 1 -- start start 2024-04-25T10:15:04.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.640+0000 7f765e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7658154ee0 0x7f76580cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.640+0000 7f765e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f76580cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.640+0000 7f765e200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7658071480 con 0x7f76580ce3c0 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.640+0000 7f765e200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76580ce830 con 0x7f7658154ee0 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.640+0000 7f765e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7658072090 con 0x7f76580cfcf0 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7658154ee0 0x7f76580cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f76580ce3c0 0x7f765815e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44740/0 (socket says 172.21.15.165:44740) 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657e00640 1 -- 172.21.15.165:0/4198519909 learned_addr learned my addr 172.21.15.165:0/4198519909 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:04.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f764fe00640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f76580cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657400640 1 -- 172.21.15.165:0/4198519909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 msgr2=0x7f76580cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7656a00640 1 -- 172.21.15.165:0/4198519909 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987367272 0 0) 0x7f7658071480 con 0x7f76580ce3c0 2024-04-25T10:15:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657400640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f76580cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657400640 1 -- 172.21.15.165:0/4198519909 >> v1:172.21.15.165:6789/0 conn(0x7f76580ce3c0 legacy=0x7f765815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f7657400640 1 -- 172.21.15.165:0/4198519909 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658071760 con 0x7f7658154ee0 2024-04-25T10:15:04.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.641+0000 7f764fe00640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f76580cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:04.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.642+0000 7f7657400640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7658154ee0 0x7f76580cf7b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f76480097a0 tx=0x7f764802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98fdbc0910935e13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f7656a00640 1 -- 172.21.15.165:0/4198519909 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648037070 con 0x7f7658154ee0 2024-04-25T10:15:04.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f7656a00640 1 -- 172.21.15.165:0/4198519909 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648032440 con 0x7f7658154ee0 2024-04-25T10:15:04.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f7656a00640 1 -- 172.21.15.165:0/4198519909 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648032810 con 0x7f7658154ee0 2024-04-25T10:15:04.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 -- 172.21.15.165:0/4198519909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7658154ee0 msgr2=0x7f76580cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7658154ee0 0x7f76580cf7b0 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f76480097a0 tx=0x7f764802a180 comp rx=0 tx=0).stop 2024-04-25T10:15:04.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 -- 172.21.15.165:0/4198519909 shutdown_connections 2024-04-25T10:15:04.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f76580cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 --2- 172.21.15.165:0/4198519909 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7658154ee0 0x7f76580cf7b0 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 -- 172.21.15.165:0/4198519909 >> 172.21.15.165:0/4198519909 conn(0x7f765806abe0 msgr2=0x7f765806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 -- 172.21.15.165:0/4198519909 shutdown_connections 2024-04-25T10:15:04.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.643+0000 7f765e200640 1 -- 172.21.15.165:0/4198519909 wait complete. 2024-04-25T10:15:04.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.645+0000 7f765e200640 1 Processor -- start 2024-04-25T10:15:04.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.645+0000 7f765e200640 1 -- start start 2024-04-25T10:15:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.646+0000 7f765e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 0x7f765815c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.646+0000 7f765e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f765815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.646+0000 7f765e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658154ee0 0x7f76580ada20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.646+0000 7f765e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658071230 con 0x7f76580ce3c0 2024-04-25T10:15:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.646+0000 7f765e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7658071d20 con 0x7f7658154ee0 2024-04-25T10:15:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.646+0000 7f765e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76581552c0 con 0x7f76580cfcf0 2024-04-25T10:15:04.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 0x7f765815c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 0x7f765815c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49952/0 (socket says 172.21.15.165:49952) 2024-04-25T10:15:04.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 -- 172.21.15.165:0/2776630498 learned_addr learned my addr 172.21.15.165:0/2776630498 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:04.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 -- 172.21.15.165:0/2776630498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 msgr2=0x7f765815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f765815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 -- 172.21.15.165:0/2776630498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658154ee0 msgr2=0x7f76580ada20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658154ee0 0x7f76580ada20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 -- 172.21.15.165:0/2776630498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658147410 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.647+0000 7f7657400640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 0x7f765815c140 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7648035040 tx=0x7f76480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.648+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648037040 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.648+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648030770 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.648+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648030b80 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.648+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76581476f0 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.648+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7658147cb0 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.649+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7658073990 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.650+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7648044020 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.651+0000 7f764f400640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f763406b4d0 0x7f763406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.651+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f76480aa750 con 0x7f76580ce3c0 2024-04-25T10:15:04.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.651+0000 7f764fe00640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f763406b4d0 0x7f763406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:04.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.652+0000 7f764fe00640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f763406b4d0 0x7f763406d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7640006fd0 tx=0x7f7640008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:04.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.656+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7648032e20 con 0x7f76580ce3c0 2024-04-25T10:15:04.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.897+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f765806d360 con 0x7f76580ce3c0 2024-04-25T10:15:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.899+0000 7f764f400640 1 -- 172.21.15.165:0/2776630498 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 84 v84) v1 ==== 94+0+3711 (secure 0 0 0) 0x7f76480313a0 con 0x7f76580ce3c0 2024-04-25T10:15:04.903 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:04.903 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":84,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5579,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3776585402","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3776585402},{"type":"v1","addr":"172.21.15.165:6836","nonce":3776585402}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":77},{"gid":5588,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3317765253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3317765253},{"type":"v1","addr":"172.21.15.172:6835","nonce":3317765253}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78},{"gid":5702,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1431589539","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1431589539},{"type":"v1","addr":"172.21.15.172:6834","nonce":1431589539}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":84},{"gid":6133,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1316507568","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1316507568},{"type":"v1","addr":"172.21.15.165:6837","nonce":1316507568}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":78}],"filesystems":[]} 2024-04-25T10:15:04.903 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 84 2024-04-25T10:15:04.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.901+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f763406b4d0 msgr2=0x7f763406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.901+0000 7f765e200640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f763406b4d0 0x7f763406d990 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f7640006fd0 tx=0x7f7640008040 comp rx=0 tx=0).stop 2024-04-25T10:15:04.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.901+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 msgr2=0x7f765815c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:04.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.901+0000 7f765e200640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 0x7f765815c140 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f7648035040 tx=0x7f76480046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 shutdown_connections 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f763406b4d0 0x7f763406d990 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7658154ee0 0x7f76580ada20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76580cfcf0 0x7f765815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 --2- 172.21.15.165:0/2776630498 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76580ce3c0 0x7f765815c140 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 >> 172.21.15.165:0/2776630498 conn(0x7f765806abe0 msgr2=0x7f76580cc850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 shutdown_connections 2024-04-25T10:15:04.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:04.902+0000 7f765e200640 1 -- 172.21.15.165:0/2776630498 wait complete. 2024-04-25T10:15:04.925 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:15:04.925 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:15:04.926 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:15:04.926 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:04.926 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:15:04.926 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:15:04.926 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:15:04.926 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:04.926 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:15:04.926 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:15:04.927 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:04.927 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:04.927 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:15:04.927 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:15:04.927 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:15:04.927 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:04.944 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:15:04.945 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:15:04.949 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:15:04.950 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:15:04.950 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:04.951 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:15:04.953 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:15:04.954 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:15:04.998 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:15:04.998 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:05.000 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:15:05.001 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:05.003 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:15:05.003 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:05.008 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:15:05.008 DEBUG:teuthology.parallel:result is None 2024-04-25T10:15:05.008 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:15:05.148 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:15:05.149 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:15:05.153 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:15:05.160 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:15:05.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.189+0000 7f1acd600640 1 Processor -- start 2024-04-25T10:15:05.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.189+0000 7f1acd600640 1 -- start start 2024-04-25T10:15:05.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.190+0000 7f1acd600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ac80b5b40 0x7f1ac8168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.190+0000 7f1acd600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac815e110 con 0x7f1ac806bff0 2024-04-25T10:15:05.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.190+0000 7f1acd600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac80b2fd0 con 0x7f1ac815dd30 2024-04-25T10:15:05.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.190+0000 7f1acd600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac806d000 con 0x7f1ac80b5b40 2024-04-25T10:15:05.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1ac7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1ac806bff0 0x7f1ac80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44776/0 (socket says 172.21.15.165:44776) 2024-04-25T10:15:05.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1ac7400640 1 -- 172.21.15.165:0/4213339513 learned_addr learned my addr 172.21.15.165:0/4213339513 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:05.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1accc00640 1 --2- 172.21.15.165:0/4213339513 >> v2:172.21.15.172:3300/0 conn(0x7f1ac80b5b40 0x7f1ac8168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1ac6a00640 1 -- 172.21.15.165:0/4213339513 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1086389708 0 0) 0x7f1ac815e110 con 0x7f1ac806bff0 2024-04-25T10:15:05.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1ac6a00640 1 -- 172.21.15.165:0/4213339513 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aac003520 con 0x7f1ac806bff0 2024-04-25T10:15:05.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1accc00640 1 -- 172.21.15.165:0/4213339513 >> v1:172.21.15.172:6789/0 conn(0x7f1ac815dd30 legacy=0x7f1ac80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1accc00640 1 -- 172.21.15.165:0/4213339513 >> v1:172.21.15.165:6789/0 conn(0x7f1ac806bff0 legacy=0x7f1ac80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.191+0000 7f1accc00640 1 -- 172.21.15.165:0/4213339513 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac806c480 con 0x7f1ac80b5b40 2024-04-25T10:15:05.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.192+0000 7f1accc00640 1 --2- 172.21.15.165:0/4213339513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac80b5b40 0x7f1ac8168b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009d90 tx=0x7f1ac002a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16db2406bec8581f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.193+0000 7f1ac6a00640 1 -- 172.21.15.165:0/4213339513 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007740 con 0x7f1ac80b5b40 2024-04-25T10:15:05.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.193+0000 7f1ac6a00640 1 -- 172.21.15.165:0/4213339513 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ac00078e0 con 0x7f1ac80b5b40 2024-04-25T10:15:05.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.193+0000 7f1ac6a00640 1 -- 172.21.15.165:0/4213339513 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac0007cb0 con 0x7f1ac80b5b40 2024-04-25T10:15:05.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.193+0000 7f1acd600640 1 -- 172.21.15.165:0/4213339513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac80b5b40 msgr2=0x7f1ac8168b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.193+0000 7f1acd600640 1 --2- 172.21.15.165:0/4213339513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac80b5b40 0x7f1ac8168b50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0009d90 tx=0x7f1ac002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.194+0000 7f1acd600640 1 -- 172.21.15.165:0/4213339513 shutdown_connections 2024-04-25T10:15:05.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.194+0000 7f1acd600640 1 --2- 172.21.15.165:0/4213339513 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac80b5b40 0x7f1ac8168b50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.194+0000 7f1acd600640 1 -- 172.21.15.165:0/4213339513 >> 172.21.15.165:0/4213339513 conn(0x7f1ac80d0610 msgr2=0x7f1ac80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:05.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.194+0000 7f1acd600640 1 -- 172.21.15.165:0/4213339513 shutdown_connections 2024-04-25T10:15:05.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.194+0000 7f1acd600640 1 -- 172.21.15.165:0/4213339513 wait complete. 2024-04-25T10:15:05.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.195+0000 7f1acd600640 1 Processor -- start 2024-04-25T10:15:05.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.196+0000 7f1acd600640 1 -- start start 2024-04-25T10:15:05.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.196+0000 7f1acd600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 0x7f1ac815a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.196+0000 7f1acd600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815dd30 0x7f1ac814dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1acd600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac8158c70 0x7f1ac81f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1acd600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac806cd20 con 0x7f1ac80b5b40 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1acd600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac806ca40 con 0x7f1ac815dd30 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1acd600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac80d3170 con 0x7f1ac8158c70 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 0x7f1ac815a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 0x7f1ac815a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49988/0 (socket says 172.21.15.165:49988) 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 -- 172.21.15.165:0/4087412060 learned_addr learned my addr 172.21.15.165:0/4087412060 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 -- 172.21.15.165:0/4087412060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac8158c70 msgr2=0x7f1ac81f39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac8158c70 0x7f1ac81f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 -- 172.21.15.165:0/4087412060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815dd30 msgr2=0x7f1ac814dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815dd30 0x7f1ac814dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 -- 172.21.15.165:0/4087412060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac8160180 con 0x7f1ac80b5b40 2024-04-25T10:15:05.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.197+0000 7f1ac7e00640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 0x7f1ac815a4d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800c3c0 tx=0x7f1ab800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.198+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab80052d0 con 0x7f1ac80b5b40 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.198+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8005490 con 0x7f1ac80b5b40 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.198+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8007260 con 0x7f1ac80b5b40 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.198+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac815f810 con 0x7f1ac80b5b40 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.198+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac8157140 con 0x7f1ac80b5b40 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.199+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac8073990 con 0x7f1ac80b5b40 2024-04-25T10:15:05.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.200+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ab8007400 con 0x7f1ac80b5b40 2024-04-25T10:15:05.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.201+0000 7f1abfe00640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1aa806b4d0 0x7f1aa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:05.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.201+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1ab8089660 con 0x7f1ac80b5b40 2024-04-25T10:15:05.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.201+0000 7f1ac7400640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1aa806b4d0 0x7f1aa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:05.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.202+0000 7f1ac7400640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1aa806b4d0 0x7f1aa806d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80cfd80 tx=0x7f1ab4007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:05.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.205+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ab8058520 con 0x7f1ac80b5b40 2024-04-25T10:15:05.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:05.419+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1ac80d16c0 con 0x7f1ac80b5b40 2024-04-25T10:15:06.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.568+0000 7f1abfe00640 1 -- 172.21.15.165:0/4087412060 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v132) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1ab8057e20 con 0x7f1ac80b5b40 2024-04-25T10:15:06.572 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1aa806b4d0 msgr2=0x7f1aa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1aa806b4d0 0x7f1aa806d990 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f1ac80cfd80 tx=0x7f1ab4007930 comp rx=0 tx=0).stop 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 msgr2=0x7f1ac815a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 0x7f1ac815a4d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f1ab800c3c0 tx=0x7f1ab800c880 comp rx=0 tx=0).stop 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 shutdown_connections 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1aa806b4d0 0x7f1aa806d990 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ac8158c70 0x7f1ac81f39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ac815dd30 0x7f1ac814dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 --2- 172.21.15.165:0/4087412060 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ac80b5b40 0x7f1ac815a4d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 >> 172.21.15.165:0/4087412060 conn(0x7f1ac80d0610 msgr2=0x7f1ac815cf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 shutdown_connections 2024-04-25T10:15:06.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.572+0000 7f1acd600640 1 -- 172.21.15.165:0/4087412060 wait complete. 2024-04-25T10:15:06.596 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:15:06.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.791+0000 7fc24fe00640 1 Processor -- start 2024-04-25T10:15:06.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.792+0000 7fc24fe00640 1 -- start start 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc24fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc2500ceb80 0x7fc2500cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc24fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2500b8360 0x7fc25006c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc24fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500d2730 con 0x7fc2500b96e0 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc24fe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500d2a10 con 0x7fc2500ceb80 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc24fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500d2cf0 con 0x7fc2500b8360 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc246000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc2500b96e0 0x7fc25006c110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44778/0 (socket says 172.21.15.165:44778) 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc246000640 1 -- 172.21.15.165:0/731043929 learned_addr learned my addr 172.21.15.165:0/731043929 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:06.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.793+0000 7fc24ea00640 1 --2- 172.21.15.165:0/731043929 >> v2:172.21.15.165:3300/0 conn(0x7fc2500ceb80 0x7fc2500cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.794+0000 7fc24f400640 1 --2- 172.21.15.165:0/731043929 >> v2:172.21.15.172:3300/0 conn(0x7fc2500b8360 0x7fc25006c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.794+0000 7fc24ea00640 1 -- 172.21.15.165:0/731043929 >> v2:172.21.15.172:3300/0 conn(0x7fc2500b8360 msgr2=0x7fc25006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.794+0000 7fc24e000640 1 -- 172.21.15.165:0/731043929 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3927488890 0 0) 0x7fc2500d2730 con 0x7fc2500b96e0 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.794+0000 7fc24ea00640 1 --2- 172.21.15.165:0/731043929 >> v2:172.21.15.172:3300/0 conn(0x7fc2500b8360 0x7fc25006c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.794+0000 7fc24ea00640 1 -- 172.21.15.165:0/731043929 >> v1:172.21.15.165:6789/0 conn(0x7fc2500b96e0 legacy=0x7fc25006c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.794+0000 7fc24ea00640 1 -- 172.21.15.165:0/731043929 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500d2fd0 con 0x7fc2500ceb80 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.795+0000 7fc24ea00640 1 --2- 172.21.15.165:0/731043929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500ceb80 0x7fc2500cef60 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc25008cd80 tx=0x7fc24002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=120be1774bcfa735 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.795+0000 7fc24e000640 1 -- 172.21.15.165:0/731043929 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc240037070 con 0x7fc2500ceb80 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.795+0000 7fc24e000640 1 -- 172.21.15.165:0/731043929 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc240032440 con 0x7fc2500ceb80 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.795+0000 7fc24e000640 1 -- 172.21.15.165:0/731043929 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc24002ae60 con 0x7fc2500ceb80 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 -- 172.21.15.165:0/731043929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500ceb80 msgr2=0x7fc2500cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 --2- 172.21.15.165:0/731043929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500ceb80 0x7fc2500cef60 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc25008cd80 tx=0x7fc24002a110 comp rx=0 tx=0).stop 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 -- 172.21.15.165:0/731043929 shutdown_connections 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 --2- 172.21.15.165:0/731043929 >> v2:172.21.15.172:3300/0 conn(0x7fc2500b8360 0x7fc25006c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 --2- 172.21.15.165:0/731043929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500ceb80 0x7fc2500cef60 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 -- 172.21.15.165:0/731043929 >> 172.21.15.165:0/731043929 conn(0x7fc2500d0470 msgr2=0x7fc2500d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 -- 172.21.15.165:0/731043929 shutdown_connections 2024-04-25T10:15:06.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.796+0000 7fc24fe00640 1 -- 172.21.15.165:0/731043929 wait complete. 2024-04-25T10:15:06.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.798+0000 7fc24fe00640 1 Processor -- start 2024-04-25T10:15:06.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.798+0000 7fc24fe00640 1 -- start start 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2500b8360 0x7fc2500c9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 0x7fc2501e90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc24fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2500ceb80 0x7fc2501e95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc24fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500cf4a0 con 0x7fc2500b96e0 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc24fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500d2420 con 0x7fc2500b8360 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc24fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2501e5750 con 0x7fc2500ceb80 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc246000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 0x7fc2501e90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc246000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 0x7fc2501e90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50004/0 (socket says 172.21.15.165:50004) 2024-04-25T10:15:06.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.799+0000 7fc246000640 1 -- 172.21.15.165:0/4245867372 learned_addr learned my addr 172.21.15.165:0/4245867372 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:06.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc246000640 1 -- 172.21.15.165:0/4245867372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2500ceb80 msgr2=0x7fc2501e95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc246000640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2500ceb80 0x7fc2501e95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc246000640 1 -- 172.21.15.165:0/4245867372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2500b8360 msgr2=0x7fc2500c9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc246000640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2500b8360 0x7fc2500c9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc246000640 1 -- 172.21.15.165:0/4245867372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2501e59f0 con 0x7fc2500b96e0 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc246000640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 0x7fc2501e90a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c4e0 tx=0x7fc23800c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238018070 con 0x7fc2500b96e0 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238007500 con 0x7fc2500b96e0 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238004a20 con 0x7fc2500b96e0 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.800+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2501e5cd0 con 0x7fc2500b96e0 2024-04-25T10:15:06.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.801+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2500aa470 con 0x7fc2500b96e0 2024-04-25T10:15:06.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.802+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc250073990 con 0x7fc2500b96e0 2024-04-25T10:15:06.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.802+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc23801b070 con 0x7fc2500b96e0 2024-04-25T10:15:06.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.803+0000 7fc247400640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc22c06b520 0x7fc22c06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:06.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.803+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc238089470 con 0x7fc2500b96e0 2024-04-25T10:15:06.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.804+0000 7fc24ea00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc22c06b520 0x7fc22c06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:06.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.804+0000 7fc24ea00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc22c06b520 0x7fc22c06d9e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc240034000 tx=0x7fc24002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:06.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:06.808+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2380585e0 con 0x7fc2500b96e0 2024-04-25T10:15:07.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.083+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc2500ce370 con 0x7fc2500b96e0 2024-04-25T10:15:07.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.084+0000 7fc247400640 1 -- 172.21.15.165:0/4245867372 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc238057ee0 con 0x7fc2500b96e0 2024-04-25T10:15:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:07.088 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:15:07.088 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:15:07.089 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.090 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.091 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:15:07.092 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.093 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.094 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:15:07.095 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.096 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.097 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.098 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.099 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.100 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:15:07.101 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:15:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:15:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:15:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:07.102 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:15:07.102 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc22c06b520 msgr2=0x7fc22c06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc22c06b520 0x7fc22c06d9e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fc240034000 tx=0x7fc24002c000 comp rx=0 tx=0).stop 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 msgr2=0x7fc2501e90a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 0x7fc2501e90a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c4e0 tx=0x7fc23800c9a0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 shutdown_connections 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc22c06b520 0x7fc22c06d9e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2500ceb80 0x7fc2501e95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2500b96e0 0x7fc2501e90a0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 --2- 172.21.15.165:0/4245867372 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2500b8360 0x7fc2500c9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.087+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 >> 172.21.15.165:0/4245867372 conn(0x7fc2500d0470 msgr2=0x7fc25006c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.088+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 shutdown_connections 2024-04-25T10:15:07.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.088+0000 7fc24fe00640 1 -- 172.21.15.165:0/4245867372 wait complete. 2024-04-25T10:15:07.110 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:07.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.297+0000 7f592dc00640 1 Processor -- start 2024-04-25T10:15:07.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.298+0000 7f592dc00640 1 -- start start 2024-04-25T10:15:07.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.299+0000 7f592dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59280cfc20 0x7f59280cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.299+0000 7f592dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f592806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.299+0000 7f592dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5928071650 con 0x7f592815db70 2024-04-25T10:15:07.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.299+0000 7f592dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59280cbd80 con 0x7f59280cfc20 2024-04-25T10:15:07.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.299+0000 7f592dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5928161910 con 0x7f592815df50 2024-04-25T10:15:07.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.300+0000 7f5926000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59280cfc20 0x7f59280cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.300+0000 7f5926000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f59280cfc20 0x7f59280cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:38242/0 (socket says 172.21.15.165:38242) 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.300+0000 7f5926000640 1 -- 172.21.15.165:0/2870449336 learned_addr learned my addr 172.21.15.165:0/2870449336 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.300+0000 7f5927400640 1 --2- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f592806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.301+0000 7f5926000640 1 -- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 msgr2=0x7f592806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.301+0000 7f5926000640 1 --2- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f592806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.301+0000 7f5926000640 1 -- 172.21.15.165:0/2870449336 >> v1:172.21.15.172:6789/0 conn(0x7f592815db70 legacy=0x7f59280cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.301+0000 7f5926000640 1 -- 172.21.15.165:0/2870449336 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59280cf230 con 0x7f59280cfc20 2024-04-25T10:15:07.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.302+0000 7f5926000640 1 --2- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59280cfc20 0x7f59280cb840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5914009870 tx=0x7f591402a1e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58a69f9063d272cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.303+0000 7f5925600640 1 -- 172.21.15.165:0/2870449336 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914037070 con 0x7f59280cfc20 2024-04-25T10:15:07.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.303+0000 7f5925600640 1 -- 172.21.15.165:0/2870449336 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914032440 con 0x7f59280cfc20 2024-04-25T10:15:07.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.303+0000 7f5925600640 1 -- 172.21.15.165:0/2870449336 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914032810 con 0x7f59280cfc20 2024-04-25T10:15:07.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.304+0000 7f592dc00640 1 -- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59280cfc20 msgr2=0x7f59280cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.304+0000 7f592dc00640 1 --2- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59280cfc20 0x7f59280cb840 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f5914009870 tx=0x7f591402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.305+0000 7f592dc00640 1 -- 172.21.15.165:0/2870449336 shutdown_connections 2024-04-25T10:15:07.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.305+0000 7f592dc00640 1 --2- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f592806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.305+0000 7f592dc00640 1 --2- 172.21.15.165:0/2870449336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59280cfc20 0x7f59280cb840 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.305+0000 7f592dc00640 1 -- 172.21.15.165:0/2870449336 >> 172.21.15.165:0/2870449336 conn(0x7f592806abe0 msgr2=0x7f592806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.305+0000 7f592dc00640 1 -- 172.21.15.165:0/2870449336 shutdown_connections 2024-04-25T10:15:07.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.305+0000 7f592dc00640 1 -- 172.21.15.165:0/2870449336 wait complete. 2024-04-25T10:15:07.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.307+0000 7f592dc00640 1 Processor -- start 2024-04-25T10:15:07.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.307+0000 7f592dc00640 1 -- start start 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.308+0000 7f592dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 0x7f5928146f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.308+0000 7f592dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f592815db70 0x7f5928160b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.308+0000 7f592dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f5928155f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.308+0000 7f592dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928162120 con 0x7f59280cfc20 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.308+0000 7f592dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5928161e40 con 0x7f592815db70 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.308+0000 7f592dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5928162640 con 0x7f592815df50 2024-04-25T10:15:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 0x7f5928146f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 0x7f5928146f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35658/0 (socket says 172.21.15.165:35658) 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 -- 172.21.15.165:0/25503285 learned_addr learned my addr 172.21.15.165:0/25503285 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926000640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f592815db70 0x7f5928160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5927400640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f5928155f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 -- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 msgr2=0x7f5928155f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f5928155f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 -- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f592815db70 msgr2=0x7f5928160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f592815db70 0x7f5928160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5926a00640 1 -- 172.21.15.165:0/25503285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592815fae0 con 0x7f59280cfc20 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.309+0000 7f5927400640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f5928155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.310+0000 7f5926a00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 0x7f5928146f70 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f591800c4d0 tx=0x7f591800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.310+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59180052d0 con 0x7f59280cfc20 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.310+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5918005470 con 0x7f59280cfc20 2024-04-25T10:15:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.310+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592815fd80 con 0x7f59280cfc20 2024-04-25T10:15:07.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.310+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5918012850 con 0x7f59280cfc20 2024-04-25T10:15:07.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.310+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5928160340 con 0x7f59280cfc20 2024-04-25T10:15:07.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.311+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5928073990 con 0x7f59280cfc20 2024-04-25T10:15:07.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.312+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59180129f0 con 0x7f59280cfc20 2024-04-25T10:15:07.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.313+0000 7f591ea00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.313+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5918089c40 con 0x7f59280cfc20 2024-04-25T10:15:07.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.313+0000 7f5926000640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.314+0000 7f5926000640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f590806b4d0 0x7f590806d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5914034000 tx=0x7f591402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.317+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5928071650 con 0x7f59280cfc20 2024-04-25T10:15:07.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.508+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f59280cdaa0 con 0x7f59280cfc20 2024-04-25T10:15:07.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.510+0000 7f591ea00640 1 -- 172.21.15.165:0/25503285 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) v1 ==== 74+0+13210 (secure 0 0 0) 0x7f5918058c70 con 0x7f59280cfc20 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:07.514 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":132,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:15:06.565635+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":122,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f590806b4d0 msgr2=0x7f590806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f590806b4d0 0x7f590806d990 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5914034000 tx=0x7f591402e000 comp rx=0 tx=0).stop 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 msgr2=0x7f5928146f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 0x7f5928146f70 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f591800c4d0 tx=0x7f591800c990 comp rx=0 tx=0).stop 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 shutdown_connections 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f590806b4d0 0x7f590806d990 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f592815df50 0x7f5928155f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f592815db70 0x7f5928160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 --2- 172.21.15.165:0/25503285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59280cfc20 0x7f5928146f70 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 >> 172.21.15.165:0/25503285 conn(0x7f592806abe0 msgr2=0x7f5928157fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 shutdown_connections 2024-04-25T10:15:07.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.513+0000 7f592dc00640 1 -- 172.21.15.165:0/25503285 wait complete. 2024-04-25T10:15:07.534 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:15:07.535 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:15:07.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.731+0000 7fdd65600640 1 Processor -- start 2024-04-25T10:15:07.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.732+0000 7fdd65600640 1 -- start start 2024-04-25T10:15:07.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd65600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdd6015dc20 0x7fdd600ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd65600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd600d32f0 0x7fdd600cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd65600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd6015e000 con 0x7fdd60161710 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd65600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd600d1de0 con 0x7fdd6015dc20 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd65600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60161c60 con 0x7fdd600d32f0 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd5ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdd6015dc20 0x7fdd600ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd5ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdd6015dc20 0x7fdd600ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35670/0 (socket says 172.21.15.165:35670) 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.733+0000 7fdd5ea00640 1 -- 172.21.15.165:0/3123728410 learned_addr learned my addr 172.21.15.165:0/3123728410 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.734+0000 7fdd5ea00640 1 -- 172.21.15.165:0/3123728410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd600d32f0 msgr2=0x7fdd600cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.734+0000 7fdd5ea00640 1 --2- 172.21.15.165:0/3123728410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd600d32f0 0x7fdd600cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.734+0000 7fdd5ea00640 1 -- 172.21.15.165:0/3123728410 >> v1:172.21.15.172:6789/0 conn(0x7fdd60161710 legacy=0x7fdd6006bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:07.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.734+0000 7fdd5ea00640 1 -- 172.21.15.165:0/3123728410 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd60161f40 con 0x7fdd6015dc20 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.735+0000 7fdd5ea00640 1 --2- 172.21.15.165:0/3123728410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ce670 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009870 tx=0x7fdd5002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66526e48c244aba8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.735+0000 7fdd5d600640 1 -- 172.21.15.165:0/3123728410 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50037070 con 0x7fdd6015dc20 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.735+0000 7fdd5d600640 1 -- 172.21.15.165:0/3123728410 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd50032440 con 0x7fdd6015dc20 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.735+0000 7fdd5d600640 1 -- 172.21.15.165:0/3123728410 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd50032810 con 0x7fdd6015dc20 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 -- 172.21.15.165:0/3123728410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 msgr2=0x7fdd600ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 --2- 172.21.15.165:0/3123728410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ce670 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fdd50009870 tx=0x7fdd5002a210 comp rx=0 tx=0).stop 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 -- 172.21.15.165:0/3123728410 shutdown_connections 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 --2- 172.21.15.165:0/3123728410 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd600d32f0 0x7fdd600cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 --2- 172.21.15.165:0/3123728410 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ce670 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 -- 172.21.15.165:0/3123728410 >> 172.21.15.165:0/3123728410 conn(0x7fdd600d0420 msgr2=0x7fdd600d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 -- 172.21.15.165:0/3123728410 shutdown_connections 2024-04-25T10:15:07.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.736+0000 7fdd65600640 1 -- 172.21.15.165:0/3123728410 wait complete. 2024-04-25T10:15:07.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.738+0000 7fdd65600640 1 Processor -- start 2024-04-25T10:15:07.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.738+0000 7fdd65600640 1 -- start start 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd65600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd600d32f0 0x7fdd601589b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd65600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ccf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd65600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd60161710 0x7fdd6015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd65600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600b5e50 con 0x7fdd6015dc20 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd65600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd60162500 con 0x7fdd600d32f0 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd65600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd600b5fd0 con 0x7fdd60161710 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ccf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ccf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35684/0 (socket says 172.21.15.165:35684) 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 -- 172.21.15.165:0/2382320314 learned_addr learned my addr 172.21.15.165:0/2382320314 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5ea00640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd600d32f0 0x7fdd601589b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5f400640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd60161710 0x7fdd6015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 -- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd60161710 msgr2=0x7fdd6015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd60161710 0x7fdd6015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 -- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd600d32f0 msgr2=0x7fdd601589b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd600d32f0 0x7fdd601589b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.739+0000 7fdd5e000640 1 -- 172.21.15.165:0/2382320314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600cb5b0 con 0x7fdd6015dc20 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd5ea00640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd600d32f0 0x7fdd601589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd5f400640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd60161710 0x7fdd6015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd5e000640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ccf30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ba50 tx=0x7fdd4c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c019070 con 0x7fdd6015dc20 2024-04-25T10:15:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c00cc20 con 0x7fdd6015dc20 2024-04-25T10:15:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c014440 con 0x7fdd6015dc20 2024-04-25T10:15:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd600cb850 con 0x7fdd6015dc20 2024-04-25T10:15:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.740+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd600cbe10 con 0x7fdd6015dc20 2024-04-25T10:15:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.741+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd60073a00 con 0x7fdd6015dc20 2024-04-25T10:15:07.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.742+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdd4c01c070 con 0x7fdd6015dc20 2024-04-25T10:15:07.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.743+0000 7fdd56a00640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd4006b4d0 0x7fdd4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:07.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.743+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fdd4c089cf0 con 0x7fdd6015dc20 2024-04-25T10:15:07.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.744+0000 7fdd5ea00640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd4006b4d0 0x7fdd4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:07.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.744+0000 7fdd5ea00640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd4006b4d0 0x7fdd4006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdd50034000 tx=0x7fdd5002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:07.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.747+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdd4c0586d0 con 0x7fdd6015dc20 2024-04-25T10:15:07.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.929+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd600d16b0 con 0x7fdd6015dc20 2024-04-25T10:15:07.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.930+0000 7fdd56a00640 1 -- 172.21.15.165:0/2382320314 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v132) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd4c05c770 con 0x7fdd6015dc20 2024-04-25T10:15:07.933 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:15:07.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.932+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd4006b4d0 msgr2=0x7fdd4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.932+0000 7fdd65600640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd4006b4d0 0x7fdd4006d990 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fdd50034000 tx=0x7fdd5002e000 comp rx=0 tx=0).stop 2024-04-25T10:15:07.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.932+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 msgr2=0x7fdd600ccf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:07.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.932+0000 7fdd65600640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ccf30 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c00ba50 tx=0x7fdd4c00bf10 comp rx=0 tx=0).stop 2024-04-25T10:15:07.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 shutdown_connections 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdd4006b4d0 0x7fdd4006d990 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdd60161710 0x7fdd6015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdd6015dc20 0x7fdd600ccf30 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 --2- 172.21.15.165:0/2382320314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdd600d32f0 0x7fdd601589b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 >> 172.21.15.165:0/2382320314 conn(0x7fdd600d0420 msgr2=0x7fdd600b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 shutdown_connections 2024-04-25T10:15:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:07.933+0000 7fdd65600640 1 -- 172.21.15.165:0/2382320314 wait complete. 2024-04-25T10:15:07.954 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:15:08.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.137+0000 7f515fe00640 1 Processor -- start 2024-04-25T10:15:08.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.137+0000 7f515fe00640 1 -- start start 2024-04-25T10:15:08.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.138+0000 7f515fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5160144ae0 0x7f51600ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.138+0000 7f515fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 0x7f516015f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f515fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516015c520 con 0x7f516015fef0 2024-04-25T10:15:08.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f515fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516015c800 con 0x7f5160144ae0 2024-04-25T10:15:08.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f515fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516015cae0 con 0x7f516015bcf0 2024-04-25T10:15:08.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f5157e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 0x7f516015f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f515ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5160144ae0 0x7f51600ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f5157e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 0x7f516015f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:57416/0 (socket says 172.21.15.165:57416) 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.139+0000 7f5157e00640 1 -- 172.21.15.165:0/3859505486 learned_addr learned my addr 172.21.15.165:0/3859505486 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.140+0000 7f515ea00640 1 -- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 msgr2=0x7f516015f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.140+0000 7f515ea00640 1 --2- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 0x7f516015f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.140+0000 7f515ea00640 1 -- 172.21.15.165:0/3859505486 >> v1:172.21.15.172:6789/0 conn(0x7f516015fef0 legacy=0x7f516006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.140+0000 7f515ea00640 1 -- 172.21.15.165:0/3859505486 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516015c150 con 0x7f5160144ae0 2024-04-25T10:15:08.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.140+0000 7f5157e00640 1 --2- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 0x7f516015f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:15:08.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.141+0000 7f515ea00640 1 --2- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5160144ae0 0x7f51600ccb00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f51500097a0 tx=0x7f515002a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97c675aced97f23c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.142+0000 7f515e000640 1 -- 172.21.15.165:0/3859505486 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150037070 con 0x7f5160144ae0 2024-04-25T10:15:08.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.142+0000 7f515e000640 1 -- 172.21.15.165:0/3859505486 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150032440 con 0x7f5160144ae0 2024-04-25T10:15:08.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.142+0000 7f515e000640 1 -- 172.21.15.165:0/3859505486 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150032810 con 0x7f5160144ae0 2024-04-25T10:15:08.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 -- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5160144ae0 msgr2=0x7f51600ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 --2- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5160144ae0 0x7f51600ccb00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f51500097a0 tx=0x7f515002a180 comp rx=0 tx=0).stop 2024-04-25T10:15:08.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 -- 172.21.15.165:0/3859505486 shutdown_connections 2024-04-25T10:15:08.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 --2- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015bcf0 0x7f516015f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 --2- 172.21.15.165:0/3859505486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5160144ae0 0x7f51600ccb00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 -- 172.21.15.165:0/3859505486 >> 172.21.15.165:0/3859505486 conn(0x7f5160061c30 msgr2=0x7f516006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 -- 172.21.15.165:0/3859505486 shutdown_connections 2024-04-25T10:15:08.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.143+0000 7f515fe00640 1 -- 172.21.15.165:0/3859505486 wait complete. 2024-04-25T10:15:08.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.145+0000 7f515fe00640 1 Processor -- start 2024-04-25T10:15:08.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.145+0000 7f515fe00640 1 -- start start 2024-04-25T10:15:08.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.146+0000 7f515fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 0x7f516013e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.146+0000 7f515fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516015bcf0 0x7f51600b3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015fef0 0x7f51600b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51600712c0 con 0x7f5160144ae0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5160071140 con 0x7f516015bcf0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5160071440 con 0x7f516015fef0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 0x7f516013e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 0x7f516013e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35688/0 (socket says 172.21.15.165:35688) 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 -- 172.21.15.165:0/2138275961 learned_addr learned my addr 172.21.15.165:0/2138275961 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 -- 172.21.15.165:0/2138275961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015fef0 msgr2=0x7f51600b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015fef0 0x7f51600b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 -- 172.21.15.165:0/2138275961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516015bcf0 msgr2=0x7f51600b3310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516015bcf0 0x7f51600b3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.147+0000 7f515ea00640 1 -- 172.21.15.165:0/2138275961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160140a20 con 0x7f5160144ae0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.148+0000 7f515ea00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 0x7f516013e0f0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f51500098c0 tx=0x7f51500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.148+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150037040 con 0x7f5160144ae0 2024-04-25T10:15:08.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.148+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5150030500 con 0x7f5160144ae0 2024-04-25T10:15:08.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.148+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51500308d0 con 0x7f5160144ae0 2024-04-25T10:15:08.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.148+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5160140d00 con 0x7f5160144ae0 2024-04-25T10:15:08.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.148+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51601412c0 con 0x7f5160144ae0 2024-04-25T10:15:08.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.149+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5160073990 con 0x7f5160144ae0 2024-04-25T10:15:08.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.150+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5150044020 con 0x7f5160144ae0 2024-04-25T10:15:08.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.151+0000 7f5156a00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f513c06b4d0 0x7f513c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.151+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f51500aa340 con 0x7f5160144ae0 2024-04-25T10:15:08.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.151+0000 7f5157e00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f513c06b4d0 0x7f513c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.152+0000 7f5157e00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f513c06b4d0 0x7f513c06d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5148009630 tx=0x7f5148006e90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.155+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5150079370 con 0x7f5160144ae0 2024-04-25T10:15:08.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.337+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f51600718a0 con 0x7f5160144ae0 2024-04-25T10:15:08.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.338+0000 7f5156a00640 1 -- 172.21.15.165:0/2138275961 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v132) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5150078c70 con 0x7f5160144ae0 2024-04-25T10:15:08.341 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:08.341 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:15:08.341 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:15:08.342 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:15:08.342 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:15:08.342 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:08.342 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:15:08.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f513c06b4d0 msgr2=0x7f513c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f513c06b4d0 0x7f513c06d990 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f5148009630 tx=0x7f5148006e90 comp rx=0 tx=0).stop 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 msgr2=0x7f516013e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 0x7f516013e0f0 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f51500098c0 tx=0x7f51500046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 shutdown_connections 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f513c06b4d0 0x7f513c06d990 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f516015fef0 0x7f51600b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f516015bcf0 0x7f51600b3310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 --2- 172.21.15.165:0/2138275961 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5160144ae0 0x7f516013e0f0 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 >> 172.21.15.165:0/2138275961 conn(0x7f5160061c30 msgr2=0x7f516006db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.340+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 shutdown_connections 2024-04-25T10:15:08.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.341+0000 7f515fe00640 1 -- 172.21.15.165:0/2138275961 wait complete. 2024-04-25T10:15:08.362 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:15:08.362 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:15:08.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.558+0000 7f9e40e00640 1 Processor -- start 2024-04-25T10:15:08.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.559+0000 7f9e40e00640 1 -- start start 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e40e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9e3c0ced80 0x7f9e3c0d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e40e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9e3c161710 0x7f9e3c0d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e40e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c15e000 con 0x7f9e3c15dc20 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e40e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0d1de0 con 0x7f9e3c0ced80 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e40e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c161c60 con 0x7f9e3c161710 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e39e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9e3c15dc20 0x7f9e3c0ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39412/0 (socket says 172.21.15.165:39412) 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e39e00640 1 -- 172.21.15.165:0/3990829533 learned_addr learned my addr 172.21.15.165:0/3990829533 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:08.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.560+0000 7f9e39400640 1 --2- 172.21.15.165:0/3990829533 >> v2:172.21.15.165:3300/0 conn(0x7f9e3c0ced80 0x7f9e3c0d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e3a800640 1 --2- 172.21.15.165:0/3990829533 >> v2:172.21.15.172:3300/0 conn(0x7f9e3c161710 0x7f9e3c0d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e38a00640 1 -- 172.21.15.165:0/3990829533 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3533407210 0 0) 0x7f9e3c15e000 con 0x7f9e3c15dc20 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e38a00640 1 -- 172.21.15.165:0/3990829533 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e20003520 con 0x7f9e3c15dc20 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e39400640 1 -- 172.21.15.165:0/3990829533 >> v2:172.21.15.172:3300/0 conn(0x7f9e3c161710 msgr2=0x7f9e3c0d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e39400640 1 --2- 172.21.15.165:0/3990829533 >> v2:172.21.15.172:3300/0 conn(0x7f9e3c161710 0x7f9e3c0d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e39400640 1 -- 172.21.15.165:0/3990829533 >> v1:172.21.15.165:6789/0 conn(0x7f9e3c15dc20 legacy=0x7f9e3c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.561+0000 7f9e39400640 1 -- 172.21.15.165:0/3990829533 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c161f40 con 0x7f9e3c0ced80 2024-04-25T10:15:08.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.562+0000 7f9e39400640 1 --2- 172.21.15.165:0/3990829533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c0ced80 0x7f9e3c0d22e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0b6570 tx=0x7f9e2802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac21c97b0c086a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.562+0000 7f9e38a00640 1 -- 172.21.15.165:0/3990829533 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28037070 con 0x7f9e3c0ced80 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.562+0000 7f9e38a00640 1 -- 172.21.15.165:0/3990829533 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e28032440 con 0x7f9e3c0ced80 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.562+0000 7f9e38a00640 1 -- 172.21.15.165:0/3990829533 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e28032810 con 0x7f9e3c0ced80 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 -- 172.21.15.165:0/3990829533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c0ced80 msgr2=0x7f9e3c0d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 --2- 172.21.15.165:0/3990829533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c0ced80 0x7f9e3c0d22e0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0b6570 tx=0x7f9e2802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 -- 172.21.15.165:0/3990829533 shutdown_connections 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 --2- 172.21.15.165:0/3990829533 >> v2:172.21.15.172:3300/0 conn(0x7f9e3c161710 0x7f9e3c0d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 --2- 172.21.15.165:0/3990829533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c0ced80 0x7f9e3c0d22e0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 -- 172.21.15.165:0/3990829533 >> 172.21.15.165:0/3990829533 conn(0x7f9e3c0d0420 msgr2=0x7f9e3c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 -- 172.21.15.165:0/3990829533 shutdown_connections 2024-04-25T10:15:08.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.563+0000 7f9e40e00640 1 -- 172.21.15.165:0/3990829533 wait complete. 2024-04-25T10:15:08.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.565+0000 7f9e40e00640 1 Processor -- start 2024-04-25T10:15:08.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.565+0000 7f9e40e00640 1 -- start start 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e40e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e3c0ced80 0x7f9e3c14ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e40e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e3c15dc20 0x7f9e3c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e40e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 0x7f9e3c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e40e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0b5e50 con 0x7f9e3c161710 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e40e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c162500 con 0x7f9e3c15dc20 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e40e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0b5fd0 con 0x7f9e3c0ced80 2024-04-25T10:15:08.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 0x7f9e3c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 0x7f9e3c0c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35712/0 (socket says 172.21.15.165:35712) 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 -- 172.21.15.165:0/87306212 learned_addr learned my addr 172.21.15.165:0/87306212 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e39e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e3c0ced80 0x7f9e3c14ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 -- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e3c0ced80 msgr2=0x7f9e3c14ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e39400640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e3c15dc20 0x7f9e3c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e3c0ced80 0x7f9e3c14ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 -- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e3c15dc20 msgr2=0x7f9e3c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e3c15dc20 0x7f9e3c0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.566+0000 7f9e3a800640 1 -- 172.21.15.165:0/87306212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c14f3a0 con 0x7f9e3c161710 2024-04-25T10:15:08.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.567+0000 7f9e3a800640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 0x7f9e3c0c9900 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9e34007d20 tx=0x7f9e3400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.567+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34014420 con 0x7f9e3c161710 2024-04-25T10:15:08.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.567+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e34019070 con 0x7f9e3c161710 2024-04-25T10:15:08.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.567+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c1e82f0 con 0x7f9e3c161710 2024-04-25T10:15:08.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.567+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e34014820 con 0x7f9e3c161710 2024-04-25T10:15:08.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.567+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c1e88b0 con 0x7f9e3c161710 2024-04-25T10:15:08.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.568+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c0cf660 con 0x7f9e3c161710 2024-04-25T10:15:08.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.569+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9e340149c0 con 0x7f9e3c161710 2024-04-25T10:15:08.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.570+0000 7f9e32000640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:08.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.570+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f9e3408a970 con 0x7f9e3c161710 2024-04-25T10:15:08.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.570+0000 7f9e39e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:08.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.571+0000 7f9e39e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0ca4f0 tx=0x7f9e2c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:08.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.574+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9e3c0cf660 con 0x7f9e3c161710 2024-04-25T10:15:08.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:08.771+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9e3c073a00 con 0x7f9e3c161710 2024-04-25T10:15:09.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.599+0000 7f9e32000640 1 -- 172.21.15.165:0/87306212 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v133) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9e34011260 con 0x7f9e3c161710 2024-04-25T10:15:09.605 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.602+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e1c06b4d0 msgr2=0x7f9e1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.602+0000 7f9e40e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0ca4f0 tx=0x7f9e2c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.602+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 msgr2=0x7f9e3c0c9900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.602+0000 7f9e40e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 0x7f9e3c0c9900 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9e34007d20 tx=0x7f9e3400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 shutdown_connections 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9e1c06b4d0 0x7f9e1c06d990 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9e3c161710 0x7f9e3c0c9900 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9e3c15dc20 0x7f9e3c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 --2- 172.21.15.165:0/87306212 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9e3c0ced80 0x7f9e3c14ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 >> 172.21.15.165:0/87306212 conn(0x7f9e3c0d0420 msgr2=0x7f9e3c06c6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 shutdown_connections 2024-04-25T10:15:09.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.603+0000 7f9e40e00640 1 -- 172.21.15.165:0/87306212 wait complete. 2024-04-25T10:15:09.633 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:15:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.823+0000 7fa886c00640 1 Processor -- start 2024-04-25T10:15:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.823+0000 7fa886c00640 1 -- start start 2024-04-25T10:15:09.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa886c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa880144d50 0x7fa88006d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa886c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa8801452a0 0x7fa88006ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa886c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8800cccc0 con 0x7fa8801446d0 2024-04-25T10:15:09.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa886c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8800b2b20 con 0x7fa8801452a0 2024-04-25T10:15:09.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa886c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88006d400 con 0x7fa880144d50 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa87fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa8801446d0 0x7fa8800b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39428/0 (socket says 172.21.15.165:39428) 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa87fe00640 1 -- 172.21.15.165:0/3721974095 learned_addr learned my addr 172.21.15.165:0/3721974095 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.825+0000 7fa884c00640 1 --2- 172.21.15.165:0/3721974095 >> v2:172.21.15.165:3300/0 conn(0x7fa8801452a0 0x7fa88006ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa87f400640 1 --2- 172.21.15.165:0/3721974095 >> v2:172.21.15.172:3300/0 conn(0x7fa880144d50 0x7fa88006d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa87ea00640 1 -- 172.21.15.165:0/3721974095 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3845479516 0 0) 0x7fa8800cccc0 con 0x7fa8801446d0 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa87ea00640 1 -- 172.21.15.165:0/3721974095 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa864003520 con 0x7fa8801446d0 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa884c00640 1 -- 172.21.15.165:0/3721974095 >> v2:172.21.15.172:3300/0 conn(0x7fa880144d50 msgr2=0x7fa88006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa884c00640 1 --2- 172.21.15.165:0/3721974095 >> v2:172.21.15.172:3300/0 conn(0x7fa880144d50 0x7fa88006d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa884c00640 1 -- 172.21.15.165:0/3721974095 >> v1:172.21.15.165:6789/0 conn(0x7fa8801446d0 legacy=0x7fa8800b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa884c00640 1 -- 172.21.15.165:0/3721974095 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800cc140 con 0x7fa8801452a0 2024-04-25T10:15:09.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.826+0000 7fa87f400640 1 --2- 172.21.15.165:0/3721974095 >> v2:172.21.15.172:3300/0 conn(0x7fa880144d50 0x7fa88006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.827+0000 7fa884c00640 1 --2- 172.21.15.165:0/3721974095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa88006ddd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fa880071c70 tx=0x7fa87802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30b415f43b4ae9c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.827+0000 7fa87ea00640 1 -- 172.21.15.165:0/3721974095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878007740 con 0x7fa8801452a0 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.827+0000 7fa87ea00640 1 -- 172.21.15.165:0/3721974095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8780078e0 con 0x7fa8801452a0 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.827+0000 7fa87ea00640 1 -- 172.21.15.165:0/3721974095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878007cb0 con 0x7fa8801452a0 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 -- 172.21.15.165:0/3721974095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 msgr2=0x7fa88006ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 --2- 172.21.15.165:0/3721974095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa88006ddd0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fa880071c70 tx=0x7fa87802a680 comp rx=0 tx=0).stop 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 -- 172.21.15.165:0/3721974095 shutdown_connections 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 --2- 172.21.15.165:0/3721974095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa88006ddd0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 --2- 172.21.15.165:0/3721974095 >> v2:172.21.15.172:3300/0 conn(0x7fa880144d50 0x7fa88006d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 -- 172.21.15.165:0/3721974095 >> 172.21.15.165:0/3721974095 conn(0x7fa88006abe0 msgr2=0x7fa88006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:09.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 -- 172.21.15.165:0/3721974095 shutdown_connections 2024-04-25T10:15:09.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.828+0000 7fa886c00640 1 -- 172.21.15.165:0/3721974095 wait complete. 2024-04-25T10:15:09.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.830+0000 7fa886c00640 1 Processor -- start 2024-04-25T10:15:09.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.830+0000 7fa886c00640 1 -- start start 2024-04-25T10:15:09.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa886c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8801446d0 0x7fa8800b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa886c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa880144d50 0x7fa880142670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa886c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa880142bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa886c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800cc9e0 con 0x7fa8801452a0 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa886c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8800cc700 con 0x7fa8801446d0 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa886c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa880144ab0 con 0x7fa880144d50 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa884c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa880142bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa884c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa880142bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35726/0 (socket says 172.21.15.165:35726) 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.831+0000 7fa884c00640 1 -- 172.21.15.165:0/2023753521 learned_addr learned my addr 172.21.15.165:0/2023753521 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa87fe00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8801446d0 0x7fa8800b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa87f400640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa880144d50 0x7fa880142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa884c00640 1 -- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa880144d50 msgr2=0x7fa880142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa884c00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa880144d50 0x7fa880142670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa884c00640 1 -- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8801446d0 msgr2=0x7fa8800b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa884c00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8801446d0 0x7fa8800b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:09.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa884c00640 1 -- 172.21.15.165:0/2023753521 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8800b0890 con 0x7fa8801452a0 2024-04-25T10:15:09.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa884c00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa880142bb0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fa878009e50 tx=0x7fa878004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.832+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878040070 con 0x7fa8801452a0 2024-04-25T10:15:09.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.833+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa878031bf0 con 0x7fa8801452a0 2024-04-25T10:15:09.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.833+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa878009450 con 0x7fa8801452a0 2024-04-25T10:15:09.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.833+0000 7fa886c00640 1 -- 172.21.15.165:0/2023753521 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8800b0b70 con 0x7fa8801452a0 2024-04-25T10:15:09.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.833+0000 7fa886c00640 1 -- 172.21.15.165:0/2023753521 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8800b1130 con 0x7fa8801452a0 2024-04-25T10:15:09.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.834+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa854005290 con 0x7fa8801452a0 2024-04-25T10:15:09.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.835+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa878039040 con 0x7fa8801452a0 2024-04-25T10:15:09.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.836+0000 7fa877e00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa86006b500 0x7fa86006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:09.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.836+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fa8780abac0 con 0x7fa8801452a0 2024-04-25T10:15:09.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.836+0000 7fa87fe00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa86006b500 0x7fa86006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:09.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.837+0000 7fa87fe00640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa86006b500 0x7fa86006d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa870005880 tx=0x7fa870009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:09.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:09.841+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa87807aa10 con 0x7fa8801452a0 2024-04-25T10:15:10.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.043+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa854005580 con 0x7fa8801452a0 2024-04-25T10:15:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.605+0000 7fa877e00640 1 -- 172.21.15.165:0/2023753521 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v134) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa87807a310 con 0x7fa8801452a0 2024-04-25T10:15:10.609 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:15:10.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa86006b500 msgr2=0x7fa86006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa86006b500 0x7fa86006d9c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fa870005880 tx=0x7fa870009040 comp rx=0 tx=0).stop 2024-04-25T10:15:10.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 msgr2=0x7fa880142bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa880142bb0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7fa878009e50 tx=0x7fa878004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:10.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 shutdown_connections 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa86006b500 0x7fa86006d9c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8801452a0 0x7fa880142bb0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa880144d50 0x7fa880142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 --2- 172.21.15.165:0/2023753521 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8801446d0 0x7fa8800b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.610+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 >> 172.21.15.165:0/2023753521 conn(0x7fa88006abe0 msgr2=0x7fa8800b3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.611+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 shutdown_connections 2024-04-25T10:15:10.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.611+0000 7fa875600640 1 -- 172.21.15.165:0/2023753521 wait complete. 2024-04-25T10:15:10.649 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:15:10.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.822+0000 7fa9f9c00640 1 Processor -- start 2024-04-25T10:15:10.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.823+0000 7fa9f9c00640 1 -- start start 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9f9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 0x7fa9f4161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9f9c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa9f40b5ae0 0x7fa9f40d2170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9f9c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9f4161e70 con 0x7fa9f40b95c0 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9f9c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9f40d1ce0 con 0x7fa9f40b5ae0 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9f9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4162150 con 0x7fa9f415dbd0 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9eb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa9f40b95c0 0x7fa9f40b53d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39436/0 (socket says 172.21.15.165:39436) 2024-04-25T10:15:10.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.824+0000 7fa9eb400640 1 -- 172.21.15.165:0/3006228437 learned_addr learned my addr 172.21.15.165:0/3006228437 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 0x7fa9f4161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f9200640 1 --2- 172.21.15.165:0/3006228437 >> v2:172.21.15.172:3300/0 conn(0x7fa9f40b5ae0 0x7fa9f40d2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633460267 0 0) 0x7fa9f4161e70 con 0x7fa9f40b95c0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9d8003520 con 0x7fa9f40b95c0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 928692136 0 0) 0x7fa9d8003520 con 0x7fa9f40b95c0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 msgr2=0x7fa9f4161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 0x7fa9f4161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 >> v2:172.21.15.172:3300/0 conn(0x7fa9f40b5ae0 msgr2=0x7fa9f40d2170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 --2- 172.21.15.165:0/3006228437 >> v2:172.21.15.172:3300/0 conn(0x7fa9f40b5ae0 0x7fa9f40d2170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f4162430 con 0x7fa9f40b95c0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa9dc001670 con 0x7fa9f40b95c0 2024-04-25T10:15:10.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 >> v1:172.21.15.165:6789/0 conn(0x7fa9f40b95c0 legacy=0x7fa9f40b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d80039b0 0x7fa9d8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d8004800 0x7fa9d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 0x7fa9d80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f40d1ce0 con 0x7fa9d80039b0 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4162150 con 0x7fa9d8004800 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.825+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d800ba70 con 0x7fa9f415dbd0 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d80039b0 0x7fa9d8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f9200640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 0x7fa9d80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9eb400640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d8004800 0x7fa9d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f3e00640 1 -- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 msgr2=0x7fa9d80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9f415dbd0 0x7fa9d80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f3e00640 1 -- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d8004800 msgr2=0x7fa9d8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9d8004800 0x7fa9d8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.826+0000 7fa9f3e00640 1 -- 172.21.15.165:0/3006228437 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8003520 con 0x7fa9d80039b0 2024-04-25T10:15:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.827+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d80039b0 0x7fa9d8003dd0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4007330 tx=0x7fa9d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a96f5635734f06f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.827+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4033070 con 0x7fa9d80039b0 2024-04-25T10:15:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.827+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9e402f630 con 0x7fa9d80039b0 2024-04-25T10:15:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.827+0000 7fa9f3400640 1 -- 172.21.15.165:0/3006228437 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9e4009be0 con 0x7fa9d80039b0 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9c00640 1 -- 172.21.15.165:0/3006228437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d80039b0 msgr2=0x7fa9d8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/3006228437 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9d80039b0 0x7fa9d8003dd0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4007330 tx=0x7fa9d8005830 comp rx=0 tx=0).stop 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9200640 1 -- 172.21.15.165:0/3006228437 reap_dead start 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9c00640 1 -- 172.21.15.165:0/3006228437 shutdown_connections 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9c00640 1 -- 172.21.15.165:0/3006228437 >> 172.21.15.165:0/3006228437 conn(0x7fa9f40d03d0 msgr2=0x7fa9f40d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9c00640 1 -- 172.21.15.165:0/3006228437 shutdown_connections 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.828+0000 7fa9f9c00640 1 -- 172.21.15.165:0/3006228437 wait complete. 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.830+0000 7fa9f9c00640 1 Processor -- start 2024-04-25T10:15:10.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.830+0000 7fa9f9c00640 1 -- start start 2024-04-25T10:15:10.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d80039b0 0x7fa9f40b84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f414bde0 0x7fa9f40ccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 0x7fa9f415ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f41617a0 con 0x7fa9f40cd130 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4161620 con 0x7fa9f414bde0 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9f4161920 con 0x7fa9d80039b0 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 0x7fa9f415ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 0x7fa9f415ee10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35730/0 (socket says 172.21.15.165:35730) 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.831+0000 7fa9f9200640 1 -- 172.21.15.165:0/949228348 learned_addr learned my addr 172.21.15.165:0/949228348 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d80039b0 0x7fa9f40b84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9eb400640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f414bde0 0x7fa9f40ccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f9200640 1 -- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d80039b0 msgr2=0x7fa9f40b84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f9200640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d80039b0 0x7fa9f40b84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f9200640 1 -- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f414bde0 msgr2=0x7fa9f40ccbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f9200640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f414bde0 0x7fa9f40ccbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f9200640 1 -- 172.21.15.165:0/949228348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9f415e050 con 0x7fa9f40cd130 2024-04-25T10:15:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9eb400640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f414bde0 0x7fa9f40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:10.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d80039b0 0x7fa9f40b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f9200640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 0x7fa9f415ee10 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40ca770 tx=0x7fa9ec00e830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.832+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec017030 con 0x7fa9f40cd130 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.833+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ec004020 con 0x7fa9f40cd130 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.833+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9f415e330 con 0x7fa9f40cd130 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.833+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00f040 con 0x7fa9f40cd130 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.833+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9f415d200 con 0x7fa9f40cd130 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.834+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9f40ca060 con 0x7fa9f40cd130 2024-04-25T10:15:10.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.835+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa9ec01a070 con 0x7fa9f40cd130 2024-04-25T10:15:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.836+0000 7fa9f0c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9d006b4d0 0x7fa9d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.836+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fa9ec0894a0 con 0x7fa9f40cd130 2024-04-25T10:15:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.836+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9d006b4d0 0x7fa9d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:10.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.837+0000 7fa9f3e00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9d006b4d0 0x7fa9d006d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4035000 tx=0x7fa9e400b030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:10.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:10.841+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa9ec058150 con 0x7fa9f40cd130 2024-04-25T10:15:11.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.075+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa9f40d1520 con 0x7fa9f40cd130 2024-04-25T10:15:11.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.648+0000 7fa9f0c00640 1 -- 172.21.15.165:0/949228348 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v86) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa9ec057a50 con 0x7fa9f40cd130 2024-04-25T10:15:11.652 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 19 and data pool 20 2024-04-25T10:15:11.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.655+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9d006b4d0 msgr2=0x7fa9d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.655+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9d006b4d0 0x7fa9d006d990 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fa9e4035000 tx=0x7fa9e400b030 comp rx=0 tx=0).stop 2024-04-25T10:15:11.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.655+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 msgr2=0x7fa9f415ee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.655+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 0x7fa9f415ee10 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fa9f40ca770 tx=0x7fa9ec00e830 comp rx=0 tx=0).stop 2024-04-25T10:15:11.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.657+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 shutdown_connections 2024-04-25T10:15:11.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.657+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa9d006b4d0 0x7fa9d006d990 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.657+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa9f40cd130 0x7fa9f415ee10 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.657+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa9f414bde0 0x7fa9f40ccbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.657+0000 7fa9f9c00640 1 --2- 172.21.15.165:0/949228348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa9d80039b0 0x7fa9f40b84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.657+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 >> 172.21.15.165:0/949228348 conn(0x7fa9f40d03d0 msgr2=0x7fa9f40b6b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.658+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 shutdown_connections 2024-04-25T10:15:11.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.658+0000 7fa9f9c00640 1 -- 172.21.15.165:0/949228348 wait complete. 2024-04-25T10:15:11.692 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:11.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.859+0000 7fa205a00640 1 Processor -- start 2024-04-25T10:15:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.860+0000 7fa205a00640 1 -- start start 2024-04-25T10:15:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.861+0000 7fa205a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 0x7fa200070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.861+0000 7fa205a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20015bf30 con 0x7fa2000cda30 2024-04-25T10:15:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.861+0000 7fa205a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa20015c210 con 0x7fa200071530 2024-04-25T10:15:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.861+0000 7fa205a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015c860 con 0x7fa200144440 2024-04-25T10:15:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.862+0000 7fa205000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa2000cda30 0x7fa20015ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39448/0 (socket says 172.21.15.165:39448) 2024-04-25T10:15:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.862+0000 7fa205000640 1 -- 172.21.15.165:0/2589961781 learned_addr learned my addr 172.21.15.165:0/2589961781 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:11.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.862+0000 7fa1ffe00640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 0x7fa200070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.862+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3128528612 0 0) 0x7fa20015bf30 con 0x7fa2000cda30 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.862+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1e4003520 con 0x7fa2000cda30 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.862+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 447178703 0 0) 0x7fa1e4003520 con 0x7fa2000cda30 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 msgr2=0x7fa200070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 0x7fa200070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 >> v1:172.21.15.172:6789/0 conn(0x7fa200071530 legacy=0x7fa2000cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa20015c4f0 con 0x7fa2000cda30 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa1f80019f0 con 0x7fa2000cda30 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 >> v1:172.21.15.165:6789/0 conn(0x7fa2000cda30 legacy=0x7fa20015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e40039e0 0x7fa1e4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e4004800 0x7fa1e4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 0x7fa1e40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa20015c860 con 0x7fa1e4004800 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e400bb00 con 0x7fa1e40039e0 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e400bd00 con 0x7fa200144440 2024-04-25T10:15:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1f7400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e4004800 0x7fa1e4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa1ffe00640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e40039e0 0x7fa1e4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.863+0000 7fa205000640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 0x7fa1e40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.864+0000 7fa1f7400640 1 -- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 msgr2=0x7fa1e40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.864+0000 7fa1f7400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa200144440 0x7fa1e40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.864+0000 7fa1f7400640 1 -- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e40039e0 msgr2=0x7fa1e4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.864+0000 7fa1f7400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1e40039e0 0x7fa1e4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.864+0000 7fa1f7400640 1 -- 172.21.15.165:0/2589961781 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4003520 con 0x7fa1e4004800 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.865+0000 7fa1f7400640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e4004800 0x7fa1e4004c80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001720 tx=0x7fa1e4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7341c0bdf6a4af64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.865+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8001370 con 0x7fa1e4004800 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.865+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e80052d0 con 0x7fa1e4004800 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.865+0000 7fa1ff400640 1 -- 172.21.15.165:0/2589961781 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e802ab90 con 0x7fa1e4004800 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205a00640 1 -- 172.21.15.165:0/2589961781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e4004800 msgr2=0x7fa1e4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205a00640 1 --2- 172.21.15.165:0/2589961781 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1e4004800 0x7fa1e4004c80 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8001720 tx=0x7fa1e4005830 comp rx=0 tx=0).stop 2024-04-25T10:15:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205000640 1 -- 172.21.15.165:0/2589961781 reap_dead start 2024-04-25T10:15:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205a00640 1 -- 172.21.15.165:0/2589961781 shutdown_connections 2024-04-25T10:15:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205a00640 1 -- 172.21.15.165:0/2589961781 >> 172.21.15.165:0/2589961781 conn(0x7fa20006abe0 msgr2=0x7fa20006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205a00640 1 -- 172.21.15.165:0/2589961781 shutdown_connections 2024-04-25T10:15:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.866+0000 7fa205a00640 1 -- 172.21.15.165:0/2589961781 wait complete. 2024-04-25T10:15:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.868+0000 7fa205a00640 1 Processor -- start 2024-04-25T10:15:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.868+0000 7fa205a00640 1 -- start start 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e4004800 0x7fa2001e6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 0x7fa2001e31e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa200140120 0x7fa2000cb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2000cdea0 con 0x7fa2001e4ab0 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa200071970 con 0x7fa200140120 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa200070ff0 con 0x7fa1e4004800 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 0x7fa2001e31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 0x7fa2001e31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35748/0 (socket says 172.21.15.165:35748) 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 -- 172.21.15.165:0/3160441290 learned_addr learned my addr 172.21.15.165:0/3160441290 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205000640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa200140120 0x7fa2000cb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 -- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e4004800 msgr2=0x7fa2001e6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1ffe00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e4004800 0x7fa2001e6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e4004800 0x7fa2001e6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 -- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa200140120 msgr2=0x7fa2000cb1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa200140120 0x7fa2000cb1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa1f7400640 1 -- 172.21.15.165:0/3160441290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2001e3720 con 0x7fa2001e4ab0 2024-04-25T10:15:11.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.869+0000 7fa205000640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa200140120 0x7fa2000cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:11.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.870+0000 7fa1f7400640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 0x7fa2001e31e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8008fd0 tx=0x7fa1e8001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.870+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8042070 con 0x7fa2001e4ab0 2024-04-25T10:15:11.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.870+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e8005480 con 0x7fa2001e4ab0 2024-04-25T10:15:11.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.870+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e8037040 con 0x7fa2001e4ab0 2024-04-25T10:15:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.870+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2001e3cf0 con 0x7fa2001e4ab0 2024-04-25T10:15:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.870+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2001e42b0 con 0x7fa2001e4ab0 2024-04-25T10:15:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.871+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa200073990 con 0x7fa2001e4ab0 2024-04-25T10:15:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.872+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1e8045070 con 0x7fa2001e4ab0 2024-04-25T10:15:11.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.873+0000 7fa1fcc00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1dc06b500 0x7fa1dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:11.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.873+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1e80a9c00 con 0x7fa2001e4ab0 2024-04-25T10:15:11.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.874+0000 7fa1ffe00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1dc06b500 0x7fa1dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:11.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.875+0000 7fa1ffe00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1dc06b500 0x7fa1dc06d9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa200145090 tx=0x7fa1f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:11.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:11.878+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1e8078850 con 0x7fa2001e4ab0 2024-04-25T10:15:12.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.074+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa20006bff0 con 0x7fa2001e4ab0 2024-04-25T10:15:12.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.077+0000 7fa1fcc00640 1 -- 172.21.15.165:0/3160441290 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16949 (secure 0 0 0) 0x7fa1e8078150 con 0x7fa2001e4ab0 2024-04-25T10:15:12.081 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:12.081 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":135,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:15:11.076855+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:12.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1dc06b500 msgr2=0x7fa1dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1dc06b500 0x7fa1dc06d9c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fa200145090 tx=0x7fa1f0008040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 msgr2=0x7fa2001e31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 0x7fa2001e31e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8008fd0 tx=0x7fa1e8001120 comp rx=0 tx=0).stop 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 shutdown_connections 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa1dc06b500 0x7fa1dc06d9c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa200140120 0x7fa2000cb1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa2001e4ab0 0x7fa2001e31e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 --2- 172.21.15.165:0/3160441290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1e4004800 0x7fa2001e6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 >> 172.21.15.165:0/3160441290 conn(0x7fa20006abe0 msgr2=0x7fa2000c5f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 shutdown_connections 2024-04-25T10:15:12.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.080+0000 7fa205a00640 1 -- 172.21.15.165:0/3160441290 wait complete. 2024-04-25T10:15:12.102 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:12.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.298+0000 7f0c66000640 1 Processor -- start 2024-04-25T10:15:12.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.298+0000 7f0c66000640 1 -- start start 2024-04-25T10:15:12.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c66000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0c60143e60 0x7f0c6006e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c66000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0c6006e6a0 0x7f0c600cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c66000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6015bb50 con 0x7f0c600cc840 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c66000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c60144240 con 0x7f0c60143e60 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c66000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6015e220 con 0x7f0c6006e6a0 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c64c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0c60143e60 0x7f0c6006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c64c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0c60143e60 0x7f0c6006e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35752/0 (socket says 172.21.15.165:35752) 2024-04-25T10:15:12.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.300+0000 7f0c64c00640 1 -- 172.21.15.165:0/2114894414 learned_addr learned my addr 172.21.15.165:0/2114894414 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:12.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.301+0000 7f0c64c00640 1 -- 172.21.15.165:0/2114894414 >> v2:172.21.15.172:3300/0 conn(0x7f0c6006e6a0 msgr2=0x7f0c600cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.301+0000 7f0c64c00640 1 --2- 172.21.15.165:0/2114894414 >> v2:172.21.15.172:3300/0 conn(0x7f0c6006e6a0 0x7f0c600cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.301+0000 7f0c64c00640 1 -- 172.21.15.165:0/2114894414 >> v1:172.21.15.172:6789/0 conn(0x7f0c600cc840 legacy=0x7f0c60070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.301+0000 7f0c64c00640 1 -- 172.21.15.165:0/2114894414 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6015e500 con 0x7f0c60143e60 2024-04-25T10:15:12.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.302+0000 7f0c64c00640 1 --2- 172.21.15.165:0/2114894414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c6006e160 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0c6006b970 tx=0x7f0c5402c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=141f5c45dd48d487 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.302+0000 7f0c5f400640 1 -- 172.21.15.165:0/2114894414 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c540325a0 con 0x7f0c60143e60 2024-04-25T10:15:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.302+0000 7f0c5f400640 1 -- 172.21.15.165:0/2114894414 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c54038070 con 0x7f0c60143e60 2024-04-25T10:15:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.302+0000 7f0c5f400640 1 -- 172.21.15.165:0/2114894414 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c54032960 con 0x7f0c60143e60 2024-04-25T10:15:12.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 -- 172.21.15.165:0/2114894414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 msgr2=0x7f0c6006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 --2- 172.21.15.165:0/2114894414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c6006e160 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0c6006b970 tx=0x7f0c5402c8e0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 -- 172.21.15.165:0/2114894414 shutdown_connections 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 --2- 172.21.15.165:0/2114894414 >> v2:172.21.15.172:3300/0 conn(0x7f0c6006e6a0 0x7f0c600cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 --2- 172.21.15.165:0/2114894414 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c6006e160 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 -- 172.21.15.165:0/2114894414 >> 172.21.15.165:0/2114894414 conn(0x7f0c6006abe0 msgr2=0x7f0c6006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 -- 172.21.15.165:0/2114894414 shutdown_connections 2024-04-25T10:15:12.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.303+0000 7f0c66000640 1 -- 172.21.15.165:0/2114894414 wait complete. 2024-04-25T10:15:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.304+0000 7f0c66000640 1 Processor -- start 2024-04-25T10:15:12.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 -- start start 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6006e6a0 0x7f0c60142e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600cc840 0x7f0c60142300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c601e5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600cccd0 con 0x7f0c60143e60 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6015eac0 con 0x7f0c6006e6a0 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.305+0000 7f0c66000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c600cce50 con 0x7f0c600cc840 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c601e5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c601e5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35756/0 (socket says 172.21.15.165:35756) 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 -- 172.21.15.165:0/3934469274 learned_addr learned my addr 172.21.15.165:0/3934469274 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c64c00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6006e6a0 0x7f0c60142e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600cc840 0x7f0c60142300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 -- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600cc840 msgr2=0x7f0c60142300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600cc840 0x7f0c60142300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 -- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6006e6a0 msgr2=0x7f0c60142e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6006e6a0 0x7f0c60142e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c65600640 1 -- 172.21.15.165:0/3934469274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c601e47f0 con 0x7f0c60143e60 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c5fe00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600cc840 0x7f0c60142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.306+0000 7f0c64c00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6006e6a0 0x7f0c60142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.307+0000 7f0c65600640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c601e5890 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800c780 tx=0x7f0c5800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.307+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c58017070 con 0x7f0c60143e60 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.307+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c58003da0 con 0x7f0c60143e60 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.307+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580052b0 con 0x7f0c60143e60 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.307+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c601e4ad0 con 0x7f0c60143e60 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.307+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c601e5090 con 0x7f0c60143e60 2024-04-25T10:15:12.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.308+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c60073990 con 0x7f0c60143e60 2024-04-25T10:15:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.309+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0c5801a070 con 0x7f0c60143e60 2024-04-25T10:15:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.310+0000 7f0c5cc00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c4006b4d0 0x7f0c4006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.310+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0c58089ab0 con 0x7f0c60143e60 2024-04-25T10:15:12.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.310+0000 7f0c64c00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c4006b4d0 0x7f0c4006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.311+0000 7f0c64c00640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c4006b4d0 0x7f0c4006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0c54037000 tx=0x7f0c54008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.314+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0c58058700 con 0x7f0c60143e60 2024-04-25T10:15:12.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.513+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c600b1df0 con 0x7f0c60143e60 2024-04-25T10:15:12.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.515+0000 7f0c5cc00640 1 -- 172.21.15.165:0/3934469274 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16949 (secure 0 0 0) 0x7f0c58058000 con 0x7f0c60143e60 2024-04-25T10:15:12.519 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:12.519 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":135,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:15:11.076855+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.517+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c4006b4d0 msgr2=0x7f0c4006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.517+0000 7f0c66000640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c4006b4d0 0x7f0c4006d990 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0c54037000 tx=0x7f0c54008040 comp rx=0 tx=0).stop 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.517+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 msgr2=0x7f0c601e5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.517+0000 7f0c66000640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c601e5890 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f0c5800c780 tx=0x7f0c5800cc40 comp rx=0 tx=0).stop 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 shutdown_connections 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0c4006b4d0 0x7f0c4006d990 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0c60143e60 0x7f0c601e5890 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0c600cc840 0x7f0c60142300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 --2- 172.21.15.165:0/3934469274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0c6006e6a0 0x7f0c60142e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 >> 172.21.15.165:0/3934469274 conn(0x7f0c6006abe0 msgr2=0x7f0c600b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 shutdown_connections 2024-04-25T10:15:12.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.518+0000 7f0c66000640 1 -- 172.21.15.165:0/3934469274 wait complete. 2024-04-25T10:15:12.541 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:15:12.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.749+0000 7fe4c8c00640 1 Processor -- start 2024-04-25T10:15:12.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.749+0000 7fe4c8c00640 1 -- start start 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.751+0000 7fe4c8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cc4f0 0x7fe4c40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.751+0000 7fe4c8c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4c40cdb20 0x7fe4c40cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.751+0000 7fe4c8c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c4072280 con 0x7fe4c415b270 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.751+0000 7fe4c8c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c40ce4c0 con 0x7fe4c40cdb20 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.751+0000 7fe4c8c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4071270 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.751+0000 7fe4c1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4c40cdb20 0x7fe4c40cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cc4f0 0x7fe4c40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4c40cdb20 0x7fe4c40cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:38364/0 (socket says 172.21.15.165:38364) 2024-04-25T10:15:12.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1400640 1 -- 172.21.15.165:0/4036206624 learned_addr learned my addr 172.21.15.165:0/4036206624 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:12.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1400640 1 -- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cc4f0 msgr2=0x7fe4c40cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:12.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1400640 1 --2- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cc4f0 0x7fe4c40cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1400640 1 -- 172.21.15.165:0/4036206624 >> v1:172.21.15.172:6789/0 conn(0x7fe4c415b270 legacy=0x7fe4c415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.752+0000 7fe4c1400640 1 -- 172.21.15.165:0/4036206624 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c40714c0 con 0x7fe4c40cdb20 2024-04-25T10:15:12.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.754+0000 7fe4c1400640 1 --2- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c40cdb20 0x7fe4c40cdf80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009e50 tx=0x7fe4ac02a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfdfc0421612becb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.754+0000 7fe4c0a00640 1 -- 172.21.15.165:0/4036206624 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac036070 con 0x7fe4c40cdb20 2024-04-25T10:15:12.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.754+0000 7fe4c0a00640 1 -- 172.21.15.165:0/4036206624 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ac031600 con 0x7fe4c40cdb20 2024-04-25T10:15:12.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.754+0000 7fe4c0a00640 1 -- 172.21.15.165:0/4036206624 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ac0319d0 con 0x7fe4c40cdb20 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 -- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c40cdb20 msgr2=0x7fe4c40cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c40cdb20 0x7fe4c40cdf80 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac009e50 tx=0x7fe4ac02a710 comp rx=0 tx=0).stop 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 -- 172.21.15.165:0/4036206624 shutdown_connections 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c40cdb20 0x7fe4c40cdf80 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/4036206624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cc4f0 0x7fe4c40cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 -- 172.21.15.165:0/4036206624 >> 172.21.15.165:0/4036206624 conn(0x7fe4c406abe0 msgr2=0x7fe4c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 -- 172.21.15.165:0/4036206624 shutdown_connections 2024-04-25T10:15:12.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.755+0000 7fe4c8c00640 1 -- 172.21.15.165:0/4036206624 wait complete. 2024-04-25T10:15:12.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.757+0000 7fe4c8c00640 1 Processor -- start 2024-04-25T10:15:12.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.757+0000 7fe4c8c00640 1 -- start start 2024-04-25T10:15:12.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c8c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 0x7fe4c4146180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c8c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cdb20 0x7fe4c40ca8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c8c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c415b270 0x7fe4c4142d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c8c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4071a80 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c8c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c415bd90 con 0x7fe4c415b270 2024-04-25T10:15:12.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c8c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c4071d60 con 0x7fe4c40cdb20 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 0x7fe4c4146180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 0x7fe4c4146180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35760/0 (socket says 172.21.15.165:35760) 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c1e00640 1 -- 172.21.15.165:0/3942206273 learned_addr learned my addr 172.21.15.165:0/3942206273 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c1e00640 1 -- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cdb20 msgr2=0x7fe4c40ca8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c1e00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cdb20 0x7fe4c40ca8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c1e00640 1 -- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c415b270 msgr2=0x7fe4c4142d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.758+0000 7fe4c2800640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c415b270 0x7fe4c4142d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4c1e00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c415b270 0x7fe4c4142d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4c1e00640 1 -- 172.21.15.165:0/3942206273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c40cbc00 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4c2800640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c415b270 0x7fe4c4142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:12.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4c1e00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 0x7fe4c4146180 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e8a0 tx=0x7fe4b400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b4017070 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b400c940 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400cd30 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c40cae00 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.759+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c40cb360 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.761+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c4073a00 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.762+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4b401a070 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.762+0000 7fe4ba000640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4a006b4d0 0x7fe4a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:12.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.763+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4b408a350 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.763+0000 7fe4c1400640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4a006b4d0 0x7fe4a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:12.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.763+0000 7fe4c1400640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4a006b4d0 0x7fe4a006d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac033000 tx=0x7fe4ac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:12.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.767+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4c40cae00 con 0x7fe4c40cc4f0 2024-04-25T10:15:12.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:12.986+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fe4c406d3a0 con 0x7fe4c40cc4f0 2024-04-25T10:15:13.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.672+0000 7fe4ba000640 1 -- 172.21.15.165:0/3942206273 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v89) v1 ==== 118+0+0 (secure 0 0 0) 0x7fe4b4058f10 con 0x7fe4c40cc4f0 2024-04-25T10:15:13.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.675+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4a006b4d0 msgr2=0x7fe4a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.675+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4a006b4d0 0x7fe4a006d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fe4ac033000 tx=0x7fe4ac02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:13.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.675+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 msgr2=0x7fe4c4146180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.675+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 0x7fe4c4146180 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7fe4b400e8a0 tx=0x7fe4b400ed60 comp rx=0 tx=0).stop 2024-04-25T10:15:13.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 shutdown_connections 2024-04-25T10:15:13.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4a006b4d0 0x7fe4a006d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4c415b270 0x7fe4c4142d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4c40cdb20 0x7fe4c40ca8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 --2- 172.21.15.165:0/3942206273 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4c40cc4f0 0x7fe4c4146180 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 >> 172.21.15.165:0/3942206273 conn(0x7fe4c406abe0 msgr2=0x7fe4c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 shutdown_connections 2024-04-25T10:15:13.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.676+0000 7fe4c8c00640 1 -- 172.21.15.165:0/3942206273 wait complete. 2024-04-25T10:15:13.701 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:15:13.702 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:15:13.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.905+0000 7fbc07200640 1 Processor -- start 2024-04-25T10:15:13.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.905+0000 7fbc07200640 1 -- start start 2024-04-25T10:15:13.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.906+0000 7fbc07200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbc080b0730 0x7fbc080b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.906+0000 7fbc07200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc0806e150 0x7fbc0806e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbc07200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc0813e640 con 0x7fbc0813d9f0 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbc07200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc0806d390 con 0x7fbc080b0730 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbc07200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0806d670 con 0x7fbc0806e150 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbc080b0730 0x7fbc080b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fbc080b0730 0x7fbc080b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35776/0 (socket says 172.21.15.165:35776) 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 -- 172.21.15.165:0/1747539520 learned_addr learned my addr 172.21.15.165:0/1747539520 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbc06800640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc0806e150 0x7fbc0806e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 -- 172.21.15.165:0/1747539520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc0806e150 msgr2=0x7fbc0806e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc0806e150 0x7fbc0806e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 -- 172.21.15.165:0/1747539520 >> v1:172.21.15.172:6789/0 conn(0x7fbc0813d9f0 legacy=0x7fbc0813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbbffe00640 1 -- 172.21.15.165:0/1747539520 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0806d950 con 0x7fbc080b0730 2024-04-25T10:15:13.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.907+0000 7fbc06800640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc0806e150 0x7fbc0806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.908+0000 7fbbffe00640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc080b0730 0x7fbc080b0bc0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009870 tx=0x7fbbf002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec24fd90b07601a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.909+0000 7fbc05400640 1 -- 172.21.15.165:0/1747539520 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0037070 con 0x7fbc080b0730 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.909+0000 7fbc05400640 1 -- 172.21.15.165:0/1747539520 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf002ae10 con 0x7fbc080b0730 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.909+0000 7fbc05400640 1 -- 172.21.15.165:0/1747539520 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0032660 con 0x7fbc080b0730 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.909+0000 7fbc07200640 1 -- 172.21.15.165:0/1747539520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc080b0730 msgr2=0x7fbc080b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc080b0730 0x7fbc080b0bc0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009870 tx=0x7fbbf002a100 comp rx=0 tx=0).stop 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 -- 172.21.15.165:0/1747539520 shutdown_connections 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc0806e150 0x7fbc0806e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 --2- 172.21.15.165:0/1747539520 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc080b0730 0x7fbc080b0bc0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 -- 172.21.15.165:0/1747539520 >> 172.21.15.165:0/1747539520 conn(0x7fbc08061cc0 msgr2=0x7fbc0806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 -- 172.21.15.165:0/1747539520 shutdown_connections 2024-04-25T10:15:13.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.910+0000 7fbc07200640 1 -- 172.21.15.165:0/1747539520 wait complete. 2024-04-25T10:15:13.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.912+0000 7fbc07200640 1 Processor -- start 2024-04-25T10:15:13.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.912+0000 7fbc07200640 1 -- start start 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc07200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc0806e150 0x7fbc0813a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc07200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc080b0730 0x7fbc08137a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc07200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 0x7fbc08138d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc07200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080ce880 con 0x7fbc0813d9f0 2024-04-25T10:15:13.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc07200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc0806eb00 con 0x7fbc0806e150 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc07200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc080cea00 con 0x7fbc080b0730 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.913+0000 7fbc06800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 0x7fbc08138d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 0x7fbc08138d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35778/0 (socket says 172.21.15.165:35778) 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 -- 172.21.15.165:0/1363320612 learned_addr learned my addr 172.21.15.165:0/1363320612 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbbffe00640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc080b0730 0x7fbc08137a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc05e00640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc0806e150 0x7fbc0813a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 -- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc080b0730 msgr2=0x7fbc08137a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc080b0730 0x7fbc08137a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 -- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc0806e150 msgr2=0x7fbc0813a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc0806e150 0x7fbc0813a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 -- 172.21.15.165:0/1363320612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08137fb0 con 0x7fbc0813d9f0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.914+0000 7fbc06800640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 0x7fbc08138d10 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000cef0 tx=0x7fbc0000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.915+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0001a070 con 0x7fbc0813d9f0 2024-04-25T10:15:13.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.915+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc0000d860 con 0x7fbc0813d9f0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.915+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0000dc50 con 0x7fbc0813d9f0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.915+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc08139540 con 0x7fbc0813d9f0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.915+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc08139b00 con 0x7fbc0813d9f0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.916+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc08073990 con 0x7fbc0813d9f0 2024-04-25T10:15:13.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.917+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbc0001d070 con 0x7fbc0813d9f0 2024-04-25T10:15:13.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.918+0000 7fbbfe000640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbbe406b500 0x7fbbe406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:13.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.918+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbc0008ac20 con 0x7fbc0813d9f0 2024-04-25T10:15:13.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.918+0000 7fbc05e00640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbbe406b500 0x7fbbe406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:13.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.919+0000 7fbc05e00640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbbe406b500 0x7fbbe406d9c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800a750 tx=0x7fbbf8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:13.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:13.922+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbc000599c0 con 0x7fbc0813d9f0 2024-04-25T10:15:14.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.136+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbc080b1240 con 0x7fbc0813d9f0 2024-04-25T10:15:14.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.684+0000 7fbbfe000640 1 -- 172.21.15.165:0/1363320612 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v90) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbc000592c0 con 0x7fbc0813d9f0 2024-04-25T10:15:14.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.687+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbbe406b500 msgr2=0x7fbbe406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.687+0000 7fbc07200640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbbe406b500 0x7fbbe406d9c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fbbf800a750 tx=0x7fbbf8008040 comp rx=0 tx=0).stop 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.687+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 msgr2=0x7fbc08138d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.687+0000 7fbc07200640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 0x7fbc08138d10 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fbc0000cef0 tx=0x7fbc0000a710 comp rx=0 tx=0).stop 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 shutdown_connections 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbbe406b500 0x7fbbe406d9c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbc0813d9f0 0x7fbc08138d10 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbc080b0730 0x7fbc08137a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 --2- 172.21.15.165:0/1363320612 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbc0806e150 0x7fbc0813a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 >> 172.21.15.165:0/1363320612 conn(0x7fbc08061cc0 msgr2=0x7fbc0813c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 shutdown_connections 2024-04-25T10:15:14.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.688+0000 7fbc07200640 1 -- 172.21.15.165:0/1363320612 wait complete. 2024-04-25T10:15:14.712 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:15:14.713 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:14.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.902+0000 7f613fe00640 1 Processor -- start 2024-04-25T10:15:14.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.903+0000 7f613fe00640 1 -- start start 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.904+0000 7f613fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f6140070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.904+0000 7f613fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61400cda30 0x7f614015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.904+0000 7f613fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f614015bf30 con 0x7f6140071530 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.904+0000 7f613fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614015c860 con 0x7f61400cda30 2024-04-25T10:15:14.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.904+0000 7f613fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6140071970 con 0x7f6140144440 2024-04-25T10:15:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.905+0000 7f613ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f6140070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.905+0000 7f613f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61400cda30 0x7f614015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.905+0000 7f613ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f6140070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:57552/0 (socket says 172.21.15.165:57552) 2024-04-25T10:15:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.905+0000 7f613ea00640 1 -- 172.21.15.165:0/3436018732 learned_addr learned my addr 172.21.15.165:0/3436018732 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:14.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.905+0000 7f613f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f61400cda30 0x7f614015ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:38390/0 (socket says 172.21.15.165:38390) 2024-04-25T10:15:14.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.906+0000 7f613f400640 1 -- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 msgr2=0x7f6140070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:14.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.906+0000 7f613f400640 1 --2- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f6140070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.906+0000 7f613f400640 1 -- 172.21.15.165:0/3436018732 >> v1:172.21.15.172:6789/0 conn(0x7f6140071530 legacy=0x7f61400cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.906+0000 7f613f400640 1 -- 172.21.15.165:0/3436018732 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614015c210 con 0x7f61400cda30 2024-04-25T10:15:14.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.908+0000 7f613f400640 1 --2- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61400cda30 0x7f614015ccf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6138009d90 tx=0x7f613802a6f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad8dc74537f6a828 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.908+0000 7f613e000640 1 -- 172.21.15.165:0/3436018732 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138007740 con 0x7f61400cda30 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613e000640 1 -- 172.21.15.165:0/3436018732 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61380078e0 con 0x7f61400cda30 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613e000640 1 -- 172.21.15.165:0/3436018732 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138007cb0 con 0x7f61400cda30 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 -- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61400cda30 msgr2=0x7f614015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 --2- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61400cda30 0x7f614015ccf0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f6138009d90 tx=0x7f613802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 -- 172.21.15.165:0/3436018732 shutdown_connections 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 --2- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f61400cda30 0x7f614015ccf0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 --2- 172.21.15.165:0/3436018732 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f6140070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 -- 172.21.15.165:0/3436018732 >> 172.21.15.165:0/3436018732 conn(0x7f614006abe0 msgr2=0x7f614006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 -- 172.21.15.165:0/3436018732 shutdown_connections 2024-04-25T10:15:14.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.909+0000 7f613fe00640 1 -- 172.21.15.165:0/3436018732 wait complete. 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.911+0000 7f613fe00640 1 Processor -- start 2024-04-25T10:15:14.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.911+0000 7f613fe00640 1 -- start start 2024-04-25T10:15:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.912+0000 7f613fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6140071530 0x7f61401f5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.912+0000 7f613fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 0x7f614013f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.912+0000 7f613fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f61400aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.912+0000 7f613fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614015bc30 con 0x7f61400cda30 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.912+0000 7f613fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61400cdea0 con 0x7f6140071530 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.912+0000 7f613fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614015d230 con 0x7f6140144440 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 0x7f614013f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 0x7f614013f0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35794/0 (socket says 172.21.15.165:35794) 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 -- 172.21.15.165:0/4233429811 learned_addr learned my addr 172.21.15.165:0/4233429811 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f613ea00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6140071530 0x7f61401f5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f613f400640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f61400aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 -- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 msgr2=0x7f61400aa380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f61400aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 -- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6140071530 msgr2=0x7f61401f5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6140071530 0x7f61401f5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.913+0000 7f6136000640 1 -- 172.21.15.165:0/4233429811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61401f5ca0 con 0x7f61400cda30 2024-04-25T10:15:14.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.914+0000 7f6136000640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 0x7f614013f0a0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f612800c400 tx=0x7f612800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.914+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61280052d0 con 0x7f61400cda30 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.914+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6128005490 con 0x7f61400cda30 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.914+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6128007260 con 0x7f61400cda30 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.914+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614013f5e0 con 0x7f61400cda30 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.914+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61400a8d60 con 0x7f61400cda30 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.915+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61400a9650 con 0x7f61400cda30 2024-04-25T10:15:14.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.916+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6128007400 con 0x7f61400cda30 2024-04-25T10:15:14.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.917+0000 7f6137e00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f611c06b500 0x7f611c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:14.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.917+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6128089880 con 0x7f61400cda30 2024-04-25T10:15:14.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.917+0000 7f613ea00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f611c06b500 0x7f611c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:14.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.918+0000 7f613ea00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f611c06b500 0x7f611c06d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6130009ac0 tx=0x7f6130008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:14.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:14.921+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f61401f5ca0 con 0x7f61400cda30 2024-04-25T10:15:15.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.155+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61400712e0 con 0x7f61400cda30 2024-04-25T10:15:15.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.156+0000 7f6137e00640 1 -- 172.21.15.165:0/4233429811 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f6128058560 con 0x7f61400cda30 2024-04-25T10:15:15.160 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:15.160 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:15.160 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f611c06b500 msgr2=0x7f611c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f611c06b500 0x7f611c06d9c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6130009ac0 tx=0x7f6130008040 comp rx=0 tx=0).stop 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 msgr2=0x7f614013f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 0x7f614013f0a0 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f612800c400 tx=0x7f612800c8c0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 shutdown_connections 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f611c06b500 0x7f611c06d9c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6140144440 0x7f61400aa380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f61400cda30 0x7f614013f0a0 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 --2- 172.21.15.165:0/4233429811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6140071530 0x7f61401f5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 >> 172.21.15.165:0/4233429811 conn(0x7f614006abe0 msgr2=0x7f6140140e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 shutdown_connections 2024-04-25T10:15:15.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.159+0000 7f613fe00640 1 -- 172.21.15.165:0/4233429811 wait complete. 2024-04-25T10:15:15.182 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:15:15.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.367+0000 7fa8a9000640 1 Processor -- start 2024-04-25T10:15:15.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.367+0000 7fa8a9000640 1 -- start start 2024-04-25T10:15:15.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 0x7fa8a406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a9000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa8a40cf030 0x7fa8a406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a9000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8a4144b30 con 0x7fa8a406eb10 2024-04-25T10:15:15.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a9000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8a40712d0 con 0x7fa8a40cf030 2024-04-25T10:15:15.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40715b0 con 0x7fa8a4144750 2024-04-25T10:15:15.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a1400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa8a406eb10 0x7fa8a40ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39450/0 (socket says 172.21.15.165:39450) 2024-04-25T10:15:15.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.369+0000 7fa8a1400640 1 -- 172.21.15.165:0/570913733 learned_addr learned my addr 172.21.15.165:0/570913733 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 0x7fa8a406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781432698 0 0) 0x7fa8a4144b30 con 0x7fa8a406eb10 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a2800640 1 --2- 172.21.15.165:0/570913733 >> v2:172.21.15.172:3300/0 conn(0x7fa8a40cf030 0x7fa8a406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa888003520 con 0x7fa8a406eb10 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1304580556 0 0) 0x7fa888003520 con 0x7fa8a406eb10 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 msgr2=0x7fa8a406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 0x7fa8a406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 >> v2:172.21.15.172:3300/0 conn(0x7fa8a40cf030 msgr2=0x7fa8a406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 --2- 172.21.15.165:0/570913733 >> v2:172.21.15.172:3300/0 conn(0x7fa8a40cf030 0x7fa8a406d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a4071890 con 0x7fa8a406eb10 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa890001670 con 0x7fa8a406eb10 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a2800640 1 --2- 172.21.15.165:0/570913733 >> v2:172.21.15.172:3300/0 conn(0x7fa8a40cf030 0x7fa8a406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 >> v1:172.21.15.165:6789/0 conn(0x7fa8a406eb10 legacy=0x7fa8a40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8880039b0 0x7fa888003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa888004800 0x7fa888004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 0x7fa8880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40712d0 con 0x7fa8880039b0 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a40715b0 con 0x7fa888004800 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.370+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88800bac0 con 0x7fa8a4144750 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8880039b0 0x7fa888003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1400640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa888004800 0x7fa888004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a2800640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 0x7fa8880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1e00640 1 -- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 msgr2=0x7fa8880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a4144750 0x7fa8880051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1e00640 1 -- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa888004800 msgr2=0x7fa888004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa888004800 0x7fa888004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.371+0000 7fa8a1e00640 1 -- 172.21.15.165:0/570913733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa888003520 con 0x7fa8880039b0 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.372+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8880039b0 0x7fa888003dd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa894009fc0 tx=0x7fa888005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd63d5f075594c30 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.372+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894009c10 con 0x7fa8880039b0 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.372+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa894036070 con 0x7fa8880039b0 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.372+0000 7fa8a0a00640 1 -- 172.21.15.165:0/570913733 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa894031850 con 0x7fa8880039b0 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a9000640 1 -- 172.21.15.165:0/570913733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8880039b0 msgr2=0x7fa888003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a9000640 1 --2- 172.21.15.165:0/570913733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8880039b0 0x7fa888003dd0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7fa894009fc0 tx=0x7fa888005830 comp rx=0 tx=0).stop 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a2800640 1 -- 172.21.15.165:0/570913733 reap_dead start 2024-04-25T10:15:15.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a9000640 1 -- 172.21.15.165:0/570913733 shutdown_connections 2024-04-25T10:15:15.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a9000640 1 -- 172.21.15.165:0/570913733 >> 172.21.15.165:0/570913733 conn(0x7fa8a406abe0 msgr2=0x7fa8a406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a9000640 1 -- 172.21.15.165:0/570913733 shutdown_connections 2024-04-25T10:15:15.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.373+0000 7fa8a9000640 1 -- 172.21.15.165:0/570913733 wait complete. 2024-04-25T10:15:15.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.375+0000 7fa8a9000640 1 Processor -- start 2024-04-25T10:15:15.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.375+0000 7fa8a9000640 1 -- start start 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a9000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880039b0 0x7fa8a415b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 0x7fa8a4142fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a415c400 0x7fa8a415c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a9000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a41455d0 con 0x7fa8a4142b40 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a9000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a4145450 con 0x7fa8880039b0 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a415c000 con 0x7fa8a415c400 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 0x7fa8a4142fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 0x7fa8a4142fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35822/0 (socket says 172.21.15.165:35822) 2024-04-25T10:15:15.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.376+0000 7fa8a1400640 1 -- 172.21.15.165:0/1873330457 learned_addr learned my addr 172.21.15.165:0/1873330457 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1400640 1 -- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a415c400 msgr2=0x7fa8a415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880039b0 0x7fa8a415b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a2800640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a415c400 0x7fa8a415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1400640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a415c400 0x7fa8a415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1400640 1 -- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880039b0 msgr2=0x7fa8a415b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1400640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880039b0 0x7fa8a415b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1400640 1 -- 172.21.15.165:0/1873330457 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a415d150 con 0x7fa8a4142b40 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880039b0 0x7fa8a415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a2800640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a415c400 0x7fa8a415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa8a1400640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 0x7fa8a4142fc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fa89000d7a0 tx=0x7fa89000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.377+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa890019070 con 0x7fa8a4142b40 2024-04-25T10:15:15.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.378+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa89000de50 con 0x7fa8a4142b40 2024-04-25T10:15:15.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.378+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8a4131d50 con 0x7fa8a4142b40 2024-04-25T10:15:15.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.378+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8900047e0 con 0x7fa8a4142b40 2024-04-25T10:15:15.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.378+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8a415cdc0 con 0x7fa8a4142b40 2024-04-25T10:15:15.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.379+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8a4143500 con 0x7fa8a4142b40 2024-04-25T10:15:15.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.380+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa89001c070 con 0x7fa8a4142b40 2024-04-25T10:15:15.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.381+0000 7fa89a000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa88406b500 0x7fa88406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.381+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa890089d80 con 0x7fa8a4142b40 2024-04-25T10:15:15.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.381+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa88406b500 0x7fa88406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.382+0000 7fa8a1e00640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa88406b500 0x7fa88406d9c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa894035000 tx=0x7fa89402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.385+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa890058a60 con 0x7fa8a4142b40 2024-04-25T10:15:15.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.565+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8a4073990 con 0x7fa8a4142b40 2024-04-25T10:15:15.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.568+0000 7fa89a000640 1 -- 172.21.15.165:0/1873330457 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+16936 (secure 0 0 0) 0x7fa890058360 con 0x7fa8a4142b40 2024-04-25T10:15:15.572 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:15.573 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":136,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:15:12.644781+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa88406b500 msgr2=0x7fa88406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa88406b500 0x7fa88406d9c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fa894035000 tx=0x7fa89402f000 comp rx=0 tx=0).stop 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 msgr2=0x7fa8a4142fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 0x7fa8a4142fc0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7fa89000d7a0 tx=0x7fa89000dc60 comp rx=0 tx=0).stop 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 shutdown_connections 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa88406b500 0x7fa88406d9c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a415c400 0x7fa8a415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a4142b40 0x7fa8a4142fc0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 --2- 172.21.15.165:0/1873330457 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8880039b0 0x7fa8a415b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 >> 172.21.15.165:0/1873330457 conn(0x7fa8a406abe0 msgr2=0x7fa8a40b2630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.571+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 shutdown_connections 2024-04-25T10:15:15.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.572+0000 7fa8a9000640 1 -- 172.21.15.165:0/1873330457 wait complete. 2024-04-25T10:15:15.592 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:15:15.593 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:15.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.777+0000 7f58dbe00640 1 Processor -- start 2024-04-25T10:15:15.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.777+0000 7f58dbe00640 1 -- start start 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.778+0000 7f58dbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f58dc071700 0x7f58dc0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.778+0000 7f58dbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f58dc0cd860 0x7f58dc15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.778+0000 7f58dbe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc15bf30 con 0x7f58dc144440 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.778+0000 7f58dbe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc15c860 con 0x7f58dc071700 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.778+0000 7f58dbe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc15c210 con 0x7f58dc0cd860 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58da000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f58dc071700 0x7f58dc0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58daa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f58dc144440 0x7f58dc070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39464/0 (socket says 172.21.15.165:39464) 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58daa00640 1 -- 172.21.15.165:0/1469325050 learned_addr learned my addr 172.21.15.165:0/1469325050 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58da000640 1 -- 172.21.15.165:0/1469325050 >> v2:172.21.15.172:3300/0 conn(0x7f58dc0cd860 msgr2=0x7f58dc15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58db400640 1 --2- 172.21.15.165:0/1469325050 >> v2:172.21.15.172:3300/0 conn(0x7f58dc0cd860 0x7f58dc15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58da000640 1 --2- 172.21.15.165:0/1469325050 >> v2:172.21.15.172:3300/0 conn(0x7f58dc0cd860 0x7f58dc15ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58da000640 1 -- 172.21.15.165:0/1469325050 >> v1:172.21.15.165:6789/0 conn(0x7f58dc144440 legacy=0x7f58dc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58da000640 1 -- 172.21.15.165:0/1469325050 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc15c4f0 con 0x7f58dc071700 2024-04-25T10:15:15.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58d9600640 1 -- 172.21.15.165:0/1469325050 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4763982 0 0) 0x7f58dc15bf30 con 0x7f58dc144440 2024-04-25T10:15:15.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.779+0000 7f58db400640 1 --2- 172.21.15.165:0/1469325050 >> v2:172.21.15.172:3300/0 conn(0x7f58dc0cd860 0x7f58dc15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.780+0000 7f58da000640 1 --2- 172.21.15.165:0/1469325050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc071700 0x7f58dc0cd320 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f58dc145460 tx=0x7f58c802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aadae75d67b8f99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.780+0000 7f58d9600640 1 -- 172.21.15.165:0/1469325050 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8037070 con 0x7f58dc071700 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.780+0000 7f58d9600640 1 -- 172.21.15.165:0/1469325050 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58c8032440 con 0x7f58dc071700 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.780+0000 7f58d9600640 1 -- 172.21.15.165:0/1469325050 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58c8032810 con 0x7f58dc071700 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 -- 172.21.15.165:0/1469325050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc071700 msgr2=0x7f58dc0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1469325050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc071700 0x7f58dc0cd320 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f58dc145460 tx=0x7f58c802a250 comp rx=0 tx=0).stop 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 -- 172.21.15.165:0/1469325050 shutdown_connections 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1469325050 >> v2:172.21.15.172:3300/0 conn(0x7f58dc0cd860 0x7f58dc15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1469325050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc071700 0x7f58dc0cd320 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 -- 172.21.15.165:0/1469325050 >> 172.21.15.165:0/1469325050 conn(0x7f58dc06abe0 msgr2=0x7f58dc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 -- 172.21.15.165:0/1469325050 shutdown_connections 2024-04-25T10:15:15.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.781+0000 7f58dbe00640 1 -- 172.21.15.165:0/1469325050 wait complete. 2024-04-25T10:15:15.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.783+0000 7f58dbe00640 1 Processor -- start 2024-04-25T10:15:15.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.783+0000 7f58dbe00640 1 -- start start 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58dc071700 0x7f58dc1f47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58dc0cd860 0x7f58dc0a8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58dbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 0x7f58dc1e42b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58dbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0cde70 con 0x7f58dc144440 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58dbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0cdcf0 con 0x7f58dc071700 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58dbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc15d230 con 0x7f58dc0cd860 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 0x7f58dc1e42b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 0x7f58dc1e42b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35848/0 (socket says 172.21.15.165:35848) 2024-04-25T10:15:15.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 -- 172.21.15.165:0/1127957315 learned_addr learned my addr 172.21.15.165:0/1127957315 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:15.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 -- 172.21.15.165:0/1127957315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58dc0cd860 msgr2=0x7f58dc0a8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58dc0cd860 0x7f58dc0a8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 -- 172.21.15.165:0/1127957315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58dc071700 msgr2=0x7f58dc1f47b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58dc071700 0x7f58dc1f47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.784+0000 7f58db400640 1 -- 172.21.15.165:0/1127957315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc1f4d90 con 0x7f58dc144440 2024-04-25T10:15:15.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.785+0000 7f58db400640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 0x7f58dc1e42b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f58d400cef0 tx=0x7f58d400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.785+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d401a070 con 0x7f58dc144440 2024-04-25T10:15:15.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.785+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d400d860 con 0x7f58dc144440 2024-04-25T10:15:15.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.785+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d400dc50 con 0x7f58dc144440 2024-04-25T10:15:15.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.785+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc1f5070 con 0x7f58dc144440 2024-04-25T10:15:15.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.785+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc0a8c40 con 0x7f58dc144440 2024-04-25T10:15:15.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.786+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc073990 con 0x7f58dc144440 2024-04-25T10:15:15.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.787+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58d401d070 con 0x7f58dc144440 2024-04-25T10:15:15.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.788+0000 7f58d2a00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58bc06b4d0 0x7f58bc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:15.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.789+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f58d408ad00 con 0x7f58dc144440 2024-04-25T10:15:15.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.789+0000 7f58daa00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58bc06b4d0 0x7f58bc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:15.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.789+0000 7f58daa00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58bc06b4d0 0x7f58bc06d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f58cc007fd0 tx=0x7f58cc007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:15.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.793+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58d4059a10 con 0x7f58dc144440 2024-04-25T10:15:15.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.970+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f58dc1e5090 con 0x7f58bc06b4d0 2024-04-25T10:15:15.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.978+0000 7f58d2a00640 1 -- 172.21.15.165:0/1127957315 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252414 (secure 0 0 0) 0x7f58dc1e5090 con 0x7f58bc06b4d0 2024-04-25T10:15:15.985 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:15.987 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:15:15.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.983+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58bc06b4d0 msgr2=0x7f58bc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.983+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58bc06b4d0 0x7f58bc06d990 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f58cc007fd0 tx=0x7f58cc007960 comp rx=0 tx=0).stop 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.983+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 msgr2=0x7f58dc1e42b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.983+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 0x7f58dc1e42b0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f58d400cef0 tx=0x7f58d400a710 comp rx=0 tx=0).stop 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 shutdown_connections 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58bc06b4d0 0x7f58bc06d990 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58dc144440 0x7f58dc1e42b0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58dc0cd860 0x7f58dc0a8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 --2- 172.21.15.165:0/1127957315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58dc071700 0x7f58dc1f47b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 >> 172.21.15.165:0/1127957315 conn(0x7f58dc06abe0 msgr2=0x7f58dc143770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 shutdown_connections 2024-04-25T10:15:15.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:15.984+0000 7f58dbe00640 1 -- 172.21.15.165:0/1127957315 wait complete. 2024-04-25T10:15:16.002 INFO:tasks.ceph.mds.b.smithi172.stderr:2024-04-25T10:15:15.998+0000 7fb7fb400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:15:16.006 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":530,"stamp":"2024-04-25T10:15:14.398540+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":441524,"kb_used_data":14388,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754533196,"statfs":{"total":773094113280,"available":772641992704,"internally_reserved":0,"allocated":14733312,"data_stored":7502357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.796623"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.667173+0000","last_change":"2024-04-25T10:15:11.664563+0000","last_active":"2024-04-25T10:15:11.667173+0000","last_peered":"2024-04-25T10:15:11.667173+0000","last_clean":"2024-04-25T10:15:11.667173+0000","last_became_active":"2024-04-25T10:15:11.664262+0000","last_became_peered":"2024-04-25T10:15:11.664262+0000","last_unstale":"2024-04-25T10:15:11.667173+0000","last_undegraded":"2024-04-25T10:15:11.667173+0000","last_fullsized":"2024-04-25T10:15:11.667173+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:47.822436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.653491+0000","last_change":"2024-04-25T10:15:11.652505+0000","last_active":"2024-04-25T10:15:11.653491+0000","last_peered":"2024-04-25T10:15:11.653491+0000","last_clean":"2024-04-25T10:15:11.653491+0000","last_became_active":"2024-04-25T10:15:11.651525+0000","last_became_peered":"2024-04-25T10:15:11.651525+0000","last_unstale":"2024-04-25T10:15:11.653491+0000","last_undegraded":"2024-04-25T10:15:11.653491+0000","last_fullsized":"2024-04-25T10:15:11.653491+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:53.877359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677752+0000","last_change":"2024-04-25T10:15:11.671894+0000","last_active":"2024-04-25T10:15:11.677752+0000","last_peered":"2024-04-25T10:15:11.677752+0000","last_clean":"2024-04-25T10:15:11.677752+0000","last_became_active":"2024-04-25T10:15:11.671607+0000","last_became_peered":"2024-04-25T10:15:11.671607+0000","last_unstale":"2024-04-25T10:15:11.677752+0000","last_undegraded":"2024-04-25T10:15:11.677752+0000","last_fullsized":"2024-04-25T10:15:11.677752+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:55.512074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.660968+0000","last_change":"2024-04-25T10:15:11.656244+0000","last_active":"2024-04-25T10:15:11.660968+0000","last_peered":"2024-04-25T10:15:11.660968+0000","last_clean":"2024-04-25T10:15:11.660968+0000","last_became_active":"2024-04-25T10:15:11.654467+0000","last_became_peered":"2024-04-25T10:15:11.654467+0000","last_unstale":"2024-04-25T10:15:11.660968+0000","last_undegraded":"2024-04-25T10:15:11.660968+0000","last_fullsized":"2024-04-25T10:15:11.660968+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.655238+0000","last_change":"2024-04-25T10:15:11.652489+0000","last_active":"2024-04-25T10:15:11.655238+0000","last_peered":"2024-04-25T10:15:11.655238+0000","last_clean":"2024-04-25T10:15:11.655238+0000","last_became_active":"2024-04-25T10:15:11.651876+0000","last_became_peered":"2024-04-25T10:15:11.651876+0000","last_unstale":"2024-04-25T10:15:11.655238+0000","last_undegraded":"2024-04-25T10:15:11.655238+0000","last_fullsized":"2024-04-25T10:15:11.655238+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:03.354415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663503+0000","last_change":"2024-04-25T10:15:11.661891+0000","last_active":"2024-04-25T10:15:11.663503+0000","last_peered":"2024-04-25T10:15:11.663503+0000","last_clean":"2024-04-25T10:15:11.663503+0000","last_became_active":"2024-04-25T10:15:11.657224+0000","last_became_peered":"2024-04-25T10:15:11.657224+0000","last_unstale":"2024-04-25T10:15:11.663503+0000","last_undegraded":"2024-04-25T10:15:11.663503+0000","last_fullsized":"2024-04-25T10:15:11.663503+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:06.469199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.682613+0000","last_change":"2024-04-25T10:15:11.677635+0000","last_active":"2024-04-25T10:15:11.682613+0000","last_peered":"2024-04-25T10:15:11.682613+0000","last_clean":"2024-04-25T10:15:11.682613+0000","last_became_active":"2024-04-25T10:15:11.677338+0000","last_became_peered":"2024-04-25T10:15:11.677338+0000","last_unstale":"2024-04-25T10:15:11.682613+0000","last_undegraded":"2024-04-25T10:15:11.682613+0000","last_fullsized":"2024-04-25T10:15:11.682613+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670212+0000","last_change":"2024-04-25T10:15:11.666715+0000","last_active":"2024-04-25T10:15:11.670212+0000","last_peered":"2024-04-25T10:15:11.670212+0000","last_clean":"2024-04-25T10:15:11.670212+0000","last_became_active":"2024-04-25T10:15:11.666409+0000","last_became_peered":"2024-04-25T10:15:11.666409+0000","last_unstale":"2024-04-25T10:15:11.670212+0000","last_undegraded":"2024-04-25T10:15:11.670212+0000","last_fullsized":"2024-04-25T10:15:11.670212+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:26.327083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.662947+0000","last_change":"2024-04-25T10:15:11.655868+0000","last_active":"2024-04-25T10:15:11.662947+0000","last_peered":"2024-04-25T10:15:11.662947+0000","last_clean":"2024-04-25T10:15:11.662947+0000","last_became_active":"2024-04-25T10:15:11.655586+0000","last_became_peered":"2024-04-25T10:15:11.655586+0000","last_unstale":"2024-04-25T10:15:11.662947+0000","last_undegraded":"2024-04-25T10:15:11.662947+0000","last_fullsized":"2024-04-25T10:15:11.662947+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:08.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674743+0000","last_change":"2024-04-25T10:15:11.672787+0000","last_active":"2024-04-25T10:15:11.674743+0000","last_peered":"2024-04-25T10:15:11.674743+0000","last_clean":"2024-04-25T10:15:11.674743+0000","last_became_active":"2024-04-25T10:15:11.672520+0000","last_became_peered":"2024-04-25T10:15:11.672520+0000","last_unstale":"2024-04-25T10:15:11.674743+0000","last_undegraded":"2024-04-25T10:15:11.674743+0000","last_fullsized":"2024-04-25T10:15:11.674743+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:40.148654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.678273+0000","last_change":"2024-04-25T10:15:11.672484+0000","last_active":"2024-04-25T10:15:11.678273+0000","last_peered":"2024-04-25T10:15:11.678273+0000","last_clean":"2024-04-25T10:15:11.678273+0000","last_became_active":"2024-04-25T10:15:11.671871+0000","last_became_peered":"2024-04-25T10:15:11.671871+0000","last_unstale":"2024-04-25T10:15:11.678273+0000","last_undegraded":"2024-04-25T10:15:11.678273+0000","last_fullsized":"2024-04-25T10:15:11.678273+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:08.455518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661167+0000","last_change":"2024-04-25T10:15:11.659933+0000","last_active":"2024-04-25T10:15:11.661167+0000","last_peered":"2024-04-25T10:15:11.661167+0000","last_clean":"2024-04-25T10:15:11.661167+0000","last_became_active":"2024-04-25T10:15:11.658771+0000","last_became_peered":"2024-04-25T10:15:11.658771+0000","last_unstale":"2024-04-25T10:15:11.661167+0000","last_undegraded":"2024-04-25T10:15:11.661167+0000","last_fullsized":"2024-04-25T10:15:11.661167+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:05.742186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670770+0000","last_change":"2024-04-25T10:15:11.663501+0000","last_active":"2024-04-25T10:15:11.670770+0000","last_peered":"2024-04-25T10:15:11.670770+0000","last_clean":"2024-04-25T10:15:11.670770+0000","last_became_active":"2024-04-25T10:15:11.663199+0000","last_became_peered":"2024-04-25T10:15:11.663199+0000","last_unstale":"2024-04-25T10:15:11.670770+0000","last_undegraded":"2024-04-25T10:15:11.670770+0000","last_fullsized":"2024-04-25T10:15:11.670770+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661111+0000","last_change":"2024-04-25T10:15:11.659633+0000","last_active":"2024-04-25T10:15:11.661111+0000","last_peered":"2024-04-25T10:15:11.661111+0000","last_clean":"2024-04-25T10:15:11.661111+0000","last_became_active":"2024-04-25T10:15:11.659088+0000","last_became_peered":"2024-04-25T10:15:11.659088+0000","last_unstale":"2024-04-25T10:15:11.661111+0000","last_undegraded":"2024-04-25T10:15:11.661111+0000","last_fullsized":"2024-04-25T10:15:11.661111+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:31.139547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672894+0000","last_change":"2024-04-25T10:15:11.668220+0000","last_active":"2024-04-25T10:15:11.672894+0000","last_peered":"2024-04-25T10:15:11.672894+0000","last_clean":"2024-04-25T10:15:11.672894+0000","last_became_active":"2024-04-25T10:15:11.667072+0000","last_became_peered":"2024-04-25T10:15:11.667072+0000","last_unstale":"2024-04-25T10:15:11.672894+0000","last_undegraded":"2024-04-25T10:15:11.672894+0000","last_fullsized":"2024-04-25T10:15:11.672894+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:56.462785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664094+0000","last_change":"2024-04-25T10:15:11.662542+0000","last_active":"2024-04-25T10:15:11.664094+0000","last_peered":"2024-04-25T10:15:11.664094+0000","last_clean":"2024-04-25T10:15:11.664094+0000","last_became_active":"2024-04-25T10:15:11.662282+0000","last_became_peered":"2024-04-25T10:15:11.662282+0000","last_unstale":"2024-04-25T10:15:11.664094+0000","last_undegraded":"2024-04-25T10:15:11.664094+0000","last_fullsized":"2024-04-25T10:15:11.664094+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:44.990918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664153+0000","last_change":"2024-04-25T10:15:11.663043+0000","last_active":"2024-04-25T10:15:11.664153+0000","last_peered":"2024-04-25T10:15:11.664153+0000","last_clean":"2024-04-25T10:15:11.664153+0000","last_became_active":"2024-04-25T10:15:11.662554+0000","last_became_peered":"2024-04-25T10:15:11.662554+0000","last_unstale":"2024-04-25T10:15:11.664153+0000","last_undegraded":"2024-04-25T10:15:11.664153+0000","last_fullsized":"2024-04-25T10:15:11.664153+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:53.877359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.657067+0000","last_change":"2024-04-25T10:15:11.653941+0000","last_active":"2024-04-25T10:15:11.657067+0000","last_peered":"2024-04-25T10:15:11.657067+0000","last_clean":"2024-04-25T10:15:11.657067+0000","last_became_active":"2024-04-25T10:15:11.653430+0000","last_became_peered":"2024-04-25T10:15:11.653430+0000","last_unstale":"2024-04-25T10:15:11.657067+0000","last_undegraded":"2024-04-25T10:15:11.657067+0000","last_fullsized":"2024-04-25T10:15:11.657067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:26.103092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677023+0000","last_change":"2024-04-25T10:15:11.671635+0000","last_active":"2024-04-25T10:15:11.677023+0000","last_peered":"2024-04-25T10:15:11.677023+0000","last_clean":"2024-04-25T10:15:11.677023+0000","last_became_active":"2024-04-25T10:15:11.671330+0000","last_became_peered":"2024-04-25T10:15:11.671330+0000","last_unstale":"2024-04-25T10:15:11.677023+0000","last_undegraded":"2024-04-25T10:15:11.677023+0000","last_fullsized":"2024-04-25T10:15:11.677023+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:38.262780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.668361+0000","last_change":"2024-04-25T10:15:11.667567+0000","last_active":"2024-04-25T10:15:11.668361+0000","last_peered":"2024-04-25T10:15:11.668361+0000","last_clean":"2024-04-25T10:15:11.668361+0000","last_became_active":"2024-04-25T10:15:11.666723+0000","last_became_peered":"2024-04-25T10:15:11.666723+0000","last_unstale":"2024-04-25T10:15:11.668361+0000","last_undegraded":"2024-04-25T10:15:11.668361+0000","last_fullsized":"2024-04-25T10:15:11.668361+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:23.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.652216+0000","last_change":"2024-04-25T10:15:11.649577+0000","last_active":"2024-04-25T10:15:11.652216+0000","last_peered":"2024-04-25T10:15:11.652216+0000","last_clean":"2024-04-25T10:15:11.652216+0000","last_became_active":"2024-04-25T10:15:11.647923+0000","last_became_peered":"2024-04-25T10:15:11.647923+0000","last_unstale":"2024-04-25T10:15:11.652216+0000","last_undegraded":"2024-04-25T10:15:11.652216+0000","last_fullsized":"2024-04-25T10:15:11.652216+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:06.469199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672059+0000","last_change":"2024-04-25T10:15:11.666744+0000","last_active":"2024-04-25T10:15:11.672059+0000","last_peered":"2024-04-25T10:15:11.672059+0000","last_clean":"2024-04-25T10:15:11.672059+0000","last_became_active":"2024-04-25T10:15:11.666467+0000","last_became_peered":"2024-04-25T10:15:11.666467+0000","last_unstale":"2024-04-25T10:15:11.672059+0000","last_undegraded":"2024-04-25T10:15:11.672059+0000","last_fullsized":"2024-04-25T10:15:11.672059+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:12.394872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674721+0000","last_change":"2024-04-25T10:15:11.667699+0000","last_active":"2024-04-25T10:15:11.674721+0000","last_peered":"2024-04-25T10:15:11.674721+0000","last_clean":"2024-04-25T10:15:11.674721+0000","last_became_active":"2024-04-25T10:15:11.667393+0000","last_became_peered":"2024-04-25T10:15:11.667393+0000","last_unstale":"2024-04-25T10:15:11.674721+0000","last_undegraded":"2024-04-25T10:15:11.674721+0000","last_fullsized":"2024-04-25T10:15:11.674721+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:15.329552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663246+0000","last_change":"2024-04-25T10:15:11.654118+0000","last_active":"2024-04-25T10:15:11.663246+0000","last_peered":"2024-04-25T10:15:11.663246+0000","last_clean":"2024-04-25T10:15:11.663246+0000","last_became_active":"2024-04-25T10:15:11.653815+0000","last_became_peered":"2024-04-25T10:15:11.653815+0000","last_unstale":"2024-04-25T10:15:11.663246+0000","last_undegraded":"2024-04-25T10:15:11.663246+0000","last_fullsized":"2024-04-25T10:15:11.663246+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:15.470375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676464+0000","last_change":"2024-04-25T10:15:11.669879+0000","last_active":"2024-04-25T10:15:11.676464+0000","last_peered":"2024-04-25T10:15:11.676464+0000","last_clean":"2024-04-25T10:15:11.676464+0000","last_became_active":"2024-04-25T10:15:11.668631+0000","last_became_peered":"2024-04-25T10:15:11.668631+0000","last_unstale":"2024-04-25T10:15:11.676464+0000","last_undegraded":"2024-04-25T10:15:11.676464+0000","last_fullsized":"2024-04-25T10:15:11.676464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:41.020359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.679757+0000","last_change":"2024-04-25T10:15:11.676218+0000","last_active":"2024-04-25T10:15:11.679757+0000","last_peered":"2024-04-25T10:15:11.679757+0000","last_clean":"2024-04-25T10:15:11.679757+0000","last_became_active":"2024-04-25T10:15:11.670407+0000","last_became_peered":"2024-04-25T10:15:11.670407+0000","last_unstale":"2024-04-25T10:15:11.679757+0000","last_undegraded":"2024-04-25T10:15:11.679757+0000","last_fullsized":"2024-04-25T10:15:11.679757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.473484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.678825+0000","last_change":"2024-04-25T10:15:11.673208+0000","last_active":"2024-04-25T10:15:11.678825+0000","last_peered":"2024-04-25T10:15:11.678825+0000","last_clean":"2024-04-25T10:15:11.678825+0000","last_became_active":"2024-04-25T10:15:11.672901+0000","last_became_peered":"2024-04-25T10:15:11.672901+0000","last_unstale":"2024-04-25T10:15:11.678825+0000","last_undegraded":"2024-04-25T10:15:11.678825+0000","last_fullsized":"2024-04-25T10:15:11.678825+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:03.354415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674551+0000","last_change":"2024-04-25T10:15:11.672019+0000","last_active":"2024-04-25T10:15:11.674551+0000","last_peered":"2024-04-25T10:15:11.674551+0000","last_clean":"2024-04-25T10:15:11.674551+0000","last_became_active":"2024-04-25T10:15:11.671743+0000","last_became_peered":"2024-04-25T10:15:11.671743+0000","last_unstale":"2024-04-25T10:15:11.674551+0000","last_undegraded":"2024-04-25T10:15:11.674551+0000","last_fullsized":"2024-04-25T10:15:11.674551+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:16.828048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677177+0000","last_change":"2024-04-25T10:15:11.674602+0000","last_active":"2024-04-25T10:15:11.677177+0000","last_peered":"2024-04-25T10:15:11.677177+0000","last_clean":"2024-04-25T10:15:11.677177+0000","last_became_active":"2024-04-25T10:15:11.674331+0000","last_became_peered":"2024-04-25T10:15:11.674331+0000","last_unstale":"2024-04-25T10:15:11.677177+0000","last_undegraded":"2024-04-25T10:15:11.677177+0000","last_fullsized":"2024-04-25T10:15:11.677177+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:15.470375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676655+0000","last_change":"2024-04-25T10:15:11.668072+0000","last_active":"2024-04-25T10:15:11.676655+0000","last_peered":"2024-04-25T10:15:11.676655+0000","last_clean":"2024-04-25T10:15:11.676655+0000","last_became_active":"2024-04-25T10:15:11.652010+0000","last_became_peered":"2024-04-25T10:15:11.652010+0000","last_unstale":"2024-04-25T10:15:11.676655+0000","last_undegraded":"2024-04-25T10:15:11.676655+0000","last_fullsized":"2024-04-25T10:15:11.676655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:26.327083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.668914+0000","last_change":"2024-04-25T10:15:11.660019+0000","last_active":"2024-04-25T10:15:11.668914+0000","last_peered":"2024-04-25T10:15:11.668914+0000","last_clean":"2024-04-25T10:15:11.668914+0000","last_became_active":"2024-04-25T10:15:11.659666+0000","last_became_peered":"2024-04-25T10:15:11.659666+0000","last_unstale":"2024-04-25T10:15:11.668914+0000","last_undegraded":"2024-04-25T10:15:11.668914+0000","last_fullsized":"2024-04-25T10:15:11.668914+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:11.604482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670550+0000","last_change":"2024-04-25T10:15:11.666920+0000","last_active":"2024-04-25T10:15:11.670550+0000","last_peered":"2024-04-25T10:15:11.670550+0000","last_clean":"2024-04-25T10:15:11.670550+0000","last_became_active":"2024-04-25T10:15:11.666314+0000","last_became_peered":"2024-04-25T10:15:11.666314+0000","last_unstale":"2024-04-25T10:15:11.670550+0000","last_undegraded":"2024-04-25T10:15:11.670550+0000","last_fullsized":"2024-04-25T10:15:11.670550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:38.694735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.668936+0000","last_change":"2024-04-25T10:15:11.657725+0000","last_active":"2024-04-25T10:15:11.668936+0000","last_peered":"2024-04-25T10:15:11.668936+0000","last_clean":"2024-04-25T10:15:11.668936+0000","last_became_active":"2024-04-25T10:15:11.657432+0000","last_became_peered":"2024-04-25T10:15:11.657432+0000","last_unstale":"2024-04-25T10:15:11.668936+0000","last_undegraded":"2024-04-25T10:15:11.668936+0000","last_fullsized":"2024-04-25T10:15:11.668936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:16.681740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"135'4","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.685546+0000","last_change":"2024-04-25T10:15:10.635806+0000","last_active":"2024-04-25T10:15:11.685546+0000","last_peered":"2024-04-25T10:15:11.685546+0000","last_clean":"2024-04-25T10:15:11.685546+0000","last_became_active":"2024-04-25T10:15:10.635481+0000","last_became_peered":"2024-04-25T10:15:10.635481+0000","last_unstale":"2024-04-25T10:15:11.685546+0000","last_undegraded":"2024-04-25T10:15:11.685546+0000","last_fullsized":"2024-04-25T10:15:11.685546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:05.798975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672874+0000","last_change":"2024-04-25T10:15:11.666378+0000","last_active":"2024-04-25T10:15:11.672874+0000","last_peered":"2024-04-25T10:15:11.672874+0000","last_clean":"2024-04-25T10:15:11.672874+0000","last_became_active":"2024-04-25T10:15:11.666107+0000","last_became_peered":"2024-04-25T10:15:11.666107+0000","last_unstale":"2024-04-25T10:15:11.672874+0000","last_undegraded":"2024-04-25T10:15:11.672874+0000","last_fullsized":"2024-04-25T10:15:11.672874+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:22.437705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677579+0000","last_change":"2024-04-25T10:15:10.639733+0000","last_active":"2024-04-25T10:15:11.677579+0000","last_peered":"2024-04-25T10:15:11.677579+0000","last_clean":"2024-04-25T10:15:11.677579+0000","last_became_active":"2024-04-25T10:15:10.639407+0000","last_became_peered":"2024-04-25T10:15:10.639407+0000","last_unstale":"2024-04-25T10:15:11.677579+0000","last_undegraded":"2024-04-25T10:15:11.677579+0000","last_fullsized":"2024-04-25T10:15:11.677579+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:41.809965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664970+0000","last_change":"2024-04-25T10:15:11.662724+0000","last_active":"2024-04-25T10:15:11.664970+0000","last_peered":"2024-04-25T10:15:11.664970+0000","last_clean":"2024-04-25T10:15:11.664970+0000","last_became_active":"2024-04-25T10:15:11.662349+0000","last_became_peered":"2024-04-25T10:15:11.662349+0000","last_unstale":"2024-04-25T10:15:11.664970+0000","last_undegraded":"2024-04-25T10:15:11.664970+0000","last_fullsized":"2024-04-25T10:15:11.664970+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:41.573787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.681904+0000","last_change":"2024-04-25T10:15:10.639834+0000","last_active":"2024-04-25T10:15:11.681904+0000","last_peered":"2024-04-25T10:15:11.681904+0000","last_clean":"2024-04-25T10:15:11.681904+0000","last_became_active":"2024-04-25T10:15:10.638986+0000","last_became_peered":"2024-04-25T10:15:10.638986+0000","last_unstale":"2024-04-25T10:15:11.681904+0000","last_undegraded":"2024-04-25T10:15:11.681904+0000","last_fullsized":"2024-04-25T10:15:11.681904+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.129952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661069+0000","last_change":"2024-04-25T10:15:11.659786+0000","last_active":"2024-04-25T10:15:11.661069+0000","last_peered":"2024-04-25T10:15:11.661069+0000","last_clean":"2024-04-25T10:15:11.661069+0000","last_became_active":"2024-04-25T10:15:11.659050+0000","last_became_peered":"2024-04-25T10:15:11.659050+0000","last_unstale":"2024-04-25T10:15:11.661069+0000","last_undegraded":"2024-04-25T10:15:11.661069+0000","last_fullsized":"2024-04-25T10:15:11.661069+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:56.798107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683010+0000","last_change":"2024-04-25T10:15:10.641648+0000","last_active":"2024-04-25T10:15:11.683010+0000","last_peered":"2024-04-25T10:15:11.683010+0000","last_clean":"2024-04-25T10:15:11.683010+0000","last_became_active":"2024-04-25T10:15:10.640951+0000","last_became_peered":"2024-04-25T10:15:10.640951+0000","last_unstale":"2024-04-25T10:15:11.683010+0000","last_undegraded":"2024-04-25T10:15:11.683010+0000","last_fullsized":"2024-04-25T10:15:11.683010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:21.339581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670526+0000","last_change":"2024-04-25T10:15:11.667364+0000","last_active":"2024-04-25T10:15:11.670526+0000","last_peered":"2024-04-25T10:15:11.670526+0000","last_clean":"2024-04-25T10:15:11.670526+0000","last_became_active":"2024-04-25T10:15:11.666127+0000","last_became_peered":"2024-04-25T10:15:11.666127+0000","last_unstale":"2024-04-25T10:15:11.670526+0000","last_undegraded":"2024-04-25T10:15:11.670526+0000","last_fullsized":"2024-04-25T10:15:11.670526+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.426238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683625+0000","last_change":"2024-04-25T10:15:10.640576+0000","last_active":"2024-04-25T10:15:11.683625+0000","last_peered":"2024-04-25T10:15:11.683625+0000","last_clean":"2024-04-25T10:15:11.683625+0000","last_became_active":"2024-04-25T10:15:10.639671+0000","last_became_peered":"2024-04-25T10:15:10.639671+0000","last_unstale":"2024-04-25T10:15:11.683625+0000","last_undegraded":"2024-04-25T10:15:11.683625+0000","last_fullsized":"2024-04-25T10:15:11.683625+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:28.331945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670330+0000","last_change":"2024-04-25T10:15:11.667115+0000","last_active":"2024-04-25T10:15:11.670330+0000","last_peered":"2024-04-25T10:15:11.670330+0000","last_clean":"2024-04-25T10:15:11.670330+0000","last_became_active":"2024-04-25T10:15:11.666840+0000","last_became_peered":"2024-04-25T10:15:11.666840+0000","last_unstale":"2024-04-25T10:15:11.670330+0000","last_undegraded":"2024-04-25T10:15:11.670330+0000","last_fullsized":"2024-04-25T10:15:11.670330+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.063295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670993+0000","last_change":"2024-04-25T10:15:10.640096+0000","last_active":"2024-04-25T10:15:11.670993+0000","last_peered":"2024-04-25T10:15:11.670993+0000","last_clean":"2024-04-25T10:15:11.670993+0000","last_became_active":"2024-04-25T10:15:10.639752+0000","last_became_peered":"2024-04-25T10:15:10.639752+0000","last_unstale":"2024-04-25T10:15:11.670993+0000","last_undegraded":"2024-04-25T10:15:11.670993+0000","last_fullsized":"2024-04-25T10:15:11.670993+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:37.247607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664754+0000","last_change":"2024-04-25T10:15:11.663571+0000","last_active":"2024-04-25T10:15:11.664754+0000","last_peered":"2024-04-25T10:15:11.664754+0000","last_clean":"2024-04-25T10:15:11.664754+0000","last_became_active":"2024-04-25T10:15:11.663265+0000","last_became_peered":"2024-04-25T10:15:11.663265+0000","last_unstale":"2024-04-25T10:15:11.664754+0000","last_undegraded":"2024-04-25T10:15:11.664754+0000","last_fullsized":"2024-04-25T10:15:11.664754+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:58.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.632141+0000","last_change":"2024-04-25T10:15:10.632606+0000","last_active":"2024-04-25T10:15:11.632141+0000","last_peered":"2024-04-25T10:15:11.632141+0000","last_clean":"2024-04-25T10:15:11.632141+0000","last_became_active":"2024-04-25T10:15:10.631808+0000","last_became_peered":"2024-04-25T10:15:10.631808+0000","last_unstale":"2024-04-25T10:15:11.632141+0000","last_undegraded":"2024-04-25T10:15:11.632141+0000","last_fullsized":"2024-04-25T10:15:11.632141+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:56.048122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.681550+0000","last_change":"2024-04-25T10:15:11.674233+0000","last_active":"2024-04-25T10:15:11.681550+0000","last_peered":"2024-04-25T10:15:11.681550+0000","last_clean":"2024-04-25T10:15:11.681550+0000","last_became_active":"2024-04-25T10:15:11.673911+0000","last_became_peered":"2024-04-25T10:15:11.673911+0000","last_unstale":"2024-04-25T10:15:11.681550+0000","last_undegraded":"2024-04-25T10:15:11.681550+0000","last_fullsized":"2024-04-25T10:15:11.681550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:34.132593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.641428+0000","last_change":"2024-04-25T10:15:10.631224+0000","last_active":"2024-04-25T10:15:11.641428+0000","last_peered":"2024-04-25T10:15:11.641428+0000","last_clean":"2024-04-25T10:15:11.641428+0000","last_became_active":"2024-04-25T10:15:10.630886+0000","last_became_peered":"2024-04-25T10:15:10.630886+0000","last_unstale":"2024-04-25T10:15:11.641428+0000","last_undegraded":"2024-04-25T10:15:11.641428+0000","last_fullsized":"2024-04-25T10:15:11.641428+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.637636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672204+0000","last_change":"2024-04-25T10:15:11.670178+0000","last_active":"2024-04-25T10:15:11.672204+0000","last_peered":"2024-04-25T10:15:11.672204+0000","last_clean":"2024-04-25T10:15:11.672204+0000","last_became_active":"2024-04-25T10:15:11.669880+0000","last_became_peered":"2024-04-25T10:15:11.669880+0000","last_unstale":"2024-04-25T10:15:11.672204+0000","last_undegraded":"2024-04-25T10:15:11.672204+0000","last_fullsized":"2024-04-25T10:15:11.672204+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:07.621852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.646871+0000","last_change":"2024-04-25T10:15:10.639564+0000","last_active":"2024-04-25T10:15:11.646871+0000","last_peered":"2024-04-25T10:15:11.646871+0000","last_clean":"2024-04-25T10:15:11.646871+0000","last_became_active":"2024-04-25T10:15:10.639264+0000","last_became_peered":"2024-04-25T10:15:10.639264+0000","last_unstale":"2024-04-25T10:15:11.646871+0000","last_undegraded":"2024-04-25T10:15:11.646871+0000","last_fullsized":"2024-04-25T10:15:11.646871+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:25.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661391+0000","last_change":"2024-04-25T10:15:11.655786+0000","last_active":"2024-04-25T10:15:11.661391+0000","last_peered":"2024-04-25T10:15:11.661391+0000","last_clean":"2024-04-25T10:15:11.661391+0000","last_became_active":"2024-04-25T10:15:11.655532+0000","last_became_peered":"2024-04-25T10:15:11.655532+0000","last_unstale":"2024-04-25T10:15:11.661391+0000","last_undegraded":"2024-04-25T10:15:11.661391+0000","last_fullsized":"2024-04-25T10:15:11.661391+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.473484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.644906+0000","last_change":"2024-04-25T10:15:10.637636+0000","last_active":"2024-04-25T10:15:11.644906+0000","last_peered":"2024-04-25T10:15:11.644906+0000","last_clean":"2024-04-25T10:15:11.644906+0000","last_became_active":"2024-04-25T10:15:10.637307+0000","last_became_peered":"2024-04-25T10:15:10.637307+0000","last_unstale":"2024-04-25T10:15:11.644906+0000","last_undegraded":"2024-04-25T10:15:11.644906+0000","last_fullsized":"2024-04-25T10:15:11.644906+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:46.171253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676679+0000","last_change":"2024-04-25T10:15:11.669397+0000","last_active":"2024-04-25T10:15:11.676679+0000","last_peered":"2024-04-25T10:15:11.676679+0000","last_clean":"2024-04-25T10:15:11.676679+0000","last_became_active":"2024-04-25T10:15:11.651998+0000","last_became_peered":"2024-04-25T10:15:11.651998+0000","last_unstale":"2024-04-25T10:15:11.676679+0000","last_undegraded":"2024-04-25T10:15:11.676679+0000","last_fullsized":"2024-04-25T10:15:11.676679+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.426238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"135'3","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683582+0000","last_change":"2024-04-25T10:15:10.644006+0000","last_active":"2024-04-25T10:15:11.683582+0000","last_peered":"2024-04-25T10:15:11.683582+0000","last_clean":"2024-04-25T10:15:11.683582+0000","last_became_active":"2024-04-25T10:15:10.643277+0000","last_became_peered":"2024-04-25T10:15:10.643277+0000","last_unstale":"2024-04-25T10:15:11.683582+0000","last_undegraded":"2024-04-25T10:15:11.683582+0000","last_fullsized":"2024-04-25T10:15:11.683582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.478239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676774+0000","last_change":"2024-04-25T10:15:11.667847+0000","last_active":"2024-04-25T10:15:11.676774+0000","last_peered":"2024-04-25T10:15:11.676774+0000","last_clean":"2024-04-25T10:15:11.676774+0000","last_became_active":"2024-04-25T10:15:11.667506+0000","last_became_peered":"2024-04-25T10:15:11.667506+0000","last_unstale":"2024-04-25T10:15:11.676774+0000","last_undegraded":"2024-04-25T10:15:11.676774+0000","last_fullsized":"2024-04-25T10:15:11.676774+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:11.604482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"135'6","reported_seq":18,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.686365+0000","last_change":"2024-04-25T10:15:10.642085+0000","last_active":"2024-04-25T10:15:11.686365+0000","last_peered":"2024-04-25T10:15:11.686365+0000","last_clean":"2024-04-25T10:15:11.686365+0000","last_became_active":"2024-04-25T10:15:10.641323+0000","last_became_peered":"2024-04-25T10:15:10.641323+0000","last_unstale":"2024-04-25T10:15:11.686365+0000","last_undegraded":"2024-04-25T10:15:11.686365+0000","last_fullsized":"2024-04-25T10:15:11.686365+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:26.004030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663479+0000","last_change":"2024-04-25T10:15:11.660391+0000","last_active":"2024-04-25T10:15:11.663479+0000","last_peered":"2024-04-25T10:15:11.663479+0000","last_clean":"2024-04-25T10:15:11.663479+0000","last_became_active":"2024-04-25T10:15:11.660024+0000","last_became_peered":"2024-04-25T10:15:11.660024+0000","last_unstale":"2024-04-25T10:15:11.663479+0000","last_undegraded":"2024-04-25T10:15:11.663479+0000","last_fullsized":"2024-04-25T10:15:11.663479+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:58.464571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.675052+0000","last_change":"2024-04-25T10:15:10.637439+0000","last_active":"2024-04-25T10:15:11.675052+0000","last_peered":"2024-04-25T10:15:11.675052+0000","last_clean":"2024-04-25T10:15:11.675052+0000","last_became_active":"2024-04-25T10:15:10.635798+0000","last_became_peered":"2024-04-25T10:15:10.635798+0000","last_unstale":"2024-04-25T10:15:11.675052+0000","last_undegraded":"2024-04-25T10:15:11.675052+0000","last_fullsized":"2024-04-25T10:15:11.675052+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:07.315410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.681992+0000","last_change":"2024-04-25T10:15:11.674455+0000","last_active":"2024-04-25T10:15:11.681992+0000","last_peered":"2024-04-25T10:15:11.681992+0000","last_clean":"2024-04-25T10:15:11.681992+0000","last_became_active":"2024-04-25T10:15:11.674118+0000","last_became_peered":"2024-04-25T10:15:11.674118+0000","last_unstale":"2024-04-25T10:15:11.681992+0000","last_undegraded":"2024-04-25T10:15:11.681992+0000","last_fullsized":"2024-04-25T10:15:11.681992+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:26.103092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.650459+0000","last_change":"2024-04-25T10:15:10.641150+0000","last_active":"2024-04-25T10:15:11.650459+0000","last_peered":"2024-04-25T10:15:11.650459+0000","last_clean":"2024-04-25T10:15:11.650459+0000","last_became_active":"2024-04-25T10:15:10.639971+0000","last_became_peered":"2024-04-25T10:15:10.639971+0000","last_unstale":"2024-04-25T10:15:11.650459+0000","last_undegraded":"2024-04-25T10:15:11.650459+0000","last_fullsized":"2024-04-25T10:15:11.650459+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:58.454376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.656937+0000","last_change":"2024-04-25T10:15:11.653269+0000","last_active":"2024-04-25T10:15:11.656937+0000","last_peered":"2024-04-25T10:15:11.656937+0000","last_clean":"2024-04-25T10:15:11.656937+0000","last_became_active":"2024-04-25T10:15:11.652757+0000","last_became_peered":"2024-04-25T10:15:11.652757+0000","last_unstale":"2024-04-25T10:15:11.656937+0000","last_undegraded":"2024-04-25T10:15:11.656937+0000","last_fullsized":"2024-04-25T10:15:11.656937+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:34.132593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"135'8","reported_seq":20,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.684179+0000","last_change":"2024-04-25T10:15:10.634313+0000","last_active":"2024-04-25T10:15:11.684179+0000","last_peered":"2024-04-25T10:15:11.684179+0000","last_clean":"2024-04-25T10:15:11.684179+0000","last_became_active":"2024-04-25T10:15:10.633822+0000","last_became_peered":"2024-04-25T10:15:10.633822+0000","last_unstale":"2024-04-25T10:15:11.684179+0000","last_undegraded":"2024-04-25T10:15:11.684179+0000","last_fullsized":"2024-04-25T10:15:11.684179+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:54.496900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676974+0000","last_change":"2024-04-25T10:15:11.673131+0000","last_active":"2024-04-25T10:15:11.676974+0000","last_peered":"2024-04-25T10:15:11.676974+0000","last_clean":"2024-04-25T10:15:11.676974+0000","last_became_active":"2024-04-25T10:15:11.672392+0000","last_became_peered":"2024-04-25T10:15:11.672392+0000","last_unstale":"2024-04-25T10:15:11.676974+0000","last_undegraded":"2024-04-25T10:15:11.676974+0000","last_fullsized":"2024-04-25T10:15:11.676974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:28.877969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.675362+0000","last_change":"2024-04-25T10:15:10.639953+0000","last_active":"2024-04-25T10:15:11.675362+0000","last_peered":"2024-04-25T10:15:11.675362+0000","last_clean":"2024-04-25T10:15:11.675362+0000","last_became_active":"2024-04-25T10:15:10.639660+0000","last_became_peered":"2024-04-25T10:15:10.639660+0000","last_unstale":"2024-04-25T10:15:11.675362+0000","last_undegraded":"2024-04-25T10:15:11.675362+0000","last_fullsized":"2024-04-25T10:15:11.675362+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:43.975745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.673270+0000","last_change":"2024-04-25T10:15:11.669118+0000","last_active":"2024-04-25T10:15:11.673270+0000","last_peered":"2024-04-25T10:15:11.673270+0000","last_clean":"2024-04-25T10:15:11.673270+0000","last_became_active":"2024-04-25T10:15:11.668766+0000","last_became_peered":"2024-04-25T10:15:11.668766+0000","last_unstale":"2024-04-25T10:15:11.673270+0000","last_undegraded":"2024-04-25T10:15:11.673270+0000","last_fullsized":"2024-04-25T10:15:11.673270+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:28.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.685273+0000","last_change":"2024-04-25T10:15:10.635510+0000","last_active":"2024-04-25T10:15:11.685273+0000","last_peered":"2024-04-25T10:15:11.685273+0000","last_clean":"2024-04-25T10:15:11.685273+0000","last_became_active":"2024-04-25T10:15:10.634925+0000","last_became_peered":"2024-04-25T10:15:10.634925+0000","last_unstale":"2024-04-25T10:15:11.685273+0000","last_undegraded":"2024-04-25T10:15:11.685273+0000","last_fullsized":"2024-04-25T10:15:11.685273+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:15.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.679051+0000","last_change":"2024-04-25T10:15:11.673192+0000","last_active":"2024-04-25T10:15:11.679051+0000","last_peered":"2024-04-25T10:15:11.679051+0000","last_clean":"2024-04-25T10:15:11.679051+0000","last_became_active":"2024-04-25T10:15:11.672805+0000","last_became_peered":"2024-04-25T10:15:11.672805+0000","last_unstale":"2024-04-25T10:15:11.679051+0000","last_undegraded":"2024-04-25T10:15:11.679051+0000","last_fullsized":"2024-04-25T10:15:11.679051+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:42.825139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.680051+0000","last_change":"2024-04-25T10:15:10.631726+0000","last_active":"2024-04-25T10:15:11.680051+0000","last_peered":"2024-04-25T10:15:11.680051+0000","last_clean":"2024-04-25T10:15:11.680051+0000","last_became_active":"2024-04-25T10:15:10.631403+0000","last_became_peered":"2024-04-25T10:15:10.631403+0000","last_unstale":"2024-04-25T10:15:11.680051+0000","last_undegraded":"2024-04-25T10:15:11.680051+0000","last_fullsized":"2024-04-25T10:15:11.680051+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:40.005185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674131+0000","last_change":"2024-04-25T10:15:11.668937+0000","last_active":"2024-04-25T10:15:11.674131+0000","last_peered":"2024-04-25T10:15:11.674131+0000","last_clean":"2024-04-25T10:15:11.674131+0000","last_became_active":"2024-04-25T10:15:11.668585+0000","last_became_peered":"2024-04-25T10:15:11.668585+0000","last_unstale":"2024-04-25T10:15:11.674131+0000","last_undegraded":"2024-04-25T10:15:11.674131+0000","last_fullsized":"2024-04-25T10:15:11.674131+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:07.922921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.647009+0000","last_change":"2024-04-25T10:15:10.631234+0000","last_active":"2024-04-25T10:15:11.647009+0000","last_peered":"2024-04-25T10:15:11.647009+0000","last_clean":"2024-04-25T10:15:11.647009+0000","last_became_active":"2024-04-25T10:15:10.630930+0000","last_became_peered":"2024-04-25T10:15:10.630930+0000","last_unstale":"2024-04-25T10:15:11.647009+0000","last_undegraded":"2024-04-25T10:15:11.647009+0000","last_fullsized":"2024-04-25T10:15:11.647009+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:26.004030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.658240+0000","last_change":"2024-04-25T10:15:11.657444+0000","last_active":"2024-04-25T10:15:11.658240+0000","last_peered":"2024-04-25T10:15:11.658240+0000","last_clean":"2024-04-25T10:15:11.658240+0000","last_became_active":"2024-04-25T10:15:11.656056+0000","last_became_peered":"2024-04-25T10:15:11.656056+0000","last_unstale":"2024-04-25T10:15:11.658240+0000","last_undegraded":"2024-04-25T10:15:11.658240+0000","last_fullsized":"2024-04-25T10:15:11.658240+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.106975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.684084+0000","last_change":"2024-04-25T10:15:10.639880+0000","last_active":"2024-04-25T10:15:11.684084+0000","last_peered":"2024-04-25T10:15:11.684084+0000","last_clean":"2024-04-25T10:15:11.684084+0000","last_became_active":"2024-04-25T10:15:10.638722+0000","last_became_peered":"2024-04-25T10:15:10.638722+0000","last_unstale":"2024-04-25T10:15:11.684084+0000","last_undegraded":"2024-04-25T10:15:11.684084+0000","last_fullsized":"2024-04-25T10:15:11.684084+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.637636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.665151+0000","last_change":"2024-04-25T10:15:11.662653+0000","last_active":"2024-04-25T10:15:11.665151+0000","last_peered":"2024-04-25T10:15:11.665151+0000","last_clean":"2024-04-25T10:15:11.665151+0000","last_became_active":"2024-04-25T10:15:11.662375+0000","last_became_peered":"2024-04-25T10:15:11.662375+0000","last_unstale":"2024-04-25T10:15:11.665151+0000","last_undegraded":"2024-04-25T10:15:11.665151+0000","last_fullsized":"2024-04-25T10:15:11.665151+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:04.579560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.646824+0000","last_change":"2024-04-25T10:15:10.639010+0000","last_active":"2024-04-25T10:15:11.646824+0000","last_peered":"2024-04-25T10:15:11.646824+0000","last_clean":"2024-04-25T10:15:11.646824+0000","last_became_active":"2024-04-25T10:15:10.638623+0000","last_became_peered":"2024-04-25T10:15:10.638623+0000","last_unstale":"2024-04-25T10:15:11.646824+0000","last_undegraded":"2024-04-25T10:15:11.646824+0000","last_fullsized":"2024-04-25T10:15:11.646824+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.554699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.657678+0000","last_change":"2024-04-25T10:15:11.655272+0000","last_active":"2024-04-25T10:15:11.657678+0000","last_peered":"2024-04-25T10:15:11.657678+0000","last_clean":"2024-04-25T10:15:11.657678+0000","last_became_active":"2024-04-25T10:15:11.654780+0000","last_became_peered":"2024-04-25T10:15:11.654780+0000","last_unstale":"2024-04-25T10:15:11.657678+0000","last_undegraded":"2024-04-25T10:15:11.657678+0000","last_fullsized":"2024-04-25T10:15:11.657678+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661178+0000","last_change":"2024-04-25T10:15:10.638090+0000","last_active":"2024-04-25T10:15:11.661178+0000","last_peered":"2024-04-25T10:15:11.661178+0000","last_clean":"2024-04-25T10:15:11.661178+0000","last_became_active":"2024-04-25T10:15:10.637747+0000","last_became_peered":"2024-04-25T10:15:10.637747+0000","last_unstale":"2024-04-25T10:15:11.661178+0000","last_undegraded":"2024-04-25T10:15:11.661178+0000","last_fullsized":"2024-04-25T10:15:11.661178+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:32.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.678592+0000","last_change":"2024-04-25T10:15:11.673350+0000","last_active":"2024-04-25T10:15:11.678592+0000","last_peered":"2024-04-25T10:15:11.678592+0000","last_clean":"2024-04-25T10:15:11.678592+0000","last_became_active":"2024-04-25T10:15:11.671893+0000","last_became_peered":"2024-04-25T10:15:11.671893+0000","last_unstale":"2024-04-25T10:15:11.678592+0000","last_undegraded":"2024-04-25T10:15:11.678592+0000","last_fullsized":"2024-04-25T10:15:11.678592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:28.516655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.633558+0000","last_change":"2024-04-25T10:15:10.628254+0000","last_active":"2024-04-25T10:15:11.633558+0000","last_peered":"2024-04-25T10:15:11.633558+0000","last_clean":"2024-04-25T10:15:11.633558+0000","last_became_active":"2024-04-25T10:15:10.627969+0000","last_became_peered":"2024-04-25T10:15:10.627969+0000","last_unstale":"2024-04-25T10:15:11.633558+0000","last_undegraded":"2024-04-25T10:15:11.633558+0000","last_fullsized":"2024-04-25T10:15:11.633558+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:27.862796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.659467+0000","last_change":"2024-04-25T10:15:11.657988+0000","last_active":"2024-04-25T10:15:11.659467+0000","last_peered":"2024-04-25T10:15:11.659467+0000","last_clean":"2024-04-25T10:15:11.659467+0000","last_became_active":"2024-04-25T10:15:11.657688+0000","last_became_peered":"2024-04-25T10:15:11.657688+0000","last_unstale":"2024-04-25T10:15:11.659467+0000","last_undegraded":"2024-04-25T10:15:11.659467+0000","last_fullsized":"2024-04-25T10:15:11.659467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:30.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677359+0000","last_change":"2024-04-25T10:15:10.638662+0000","last_active":"2024-04-25T10:15:11.677359+0000","last_peered":"2024-04-25T10:15:11.677359+0000","last_clean":"2024-04-25T10:15:11.677359+0000","last_became_active":"2024-04-25T10:15:10.638251+0000","last_became_peered":"2024-04-25T10:15:10.638251+0000","last_unstale":"2024-04-25T10:15:11.677359+0000","last_undegraded":"2024-04-25T10:15:11.677359+0000","last_fullsized":"2024-04-25T10:15:11.677359+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:38.508983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676812+0000","last_change":"2024-04-25T10:15:11.669906+0000","last_active":"2024-04-25T10:15:11.676812+0000","last_peered":"2024-04-25T10:15:11.676812+0000","last_clean":"2024-04-25T10:15:11.676812+0000","last_became_active":"2024-04-25T10:15:11.665312+0000","last_became_peered":"2024-04-25T10:15:11.665312+0000","last_unstale":"2024-04-25T10:15:11.676812+0000","last_undegraded":"2024-04-25T10:15:11.676812+0000","last_fullsized":"2024-04-25T10:15:11.676812+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.642033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.647607+0000","last_change":"2024-04-25T10:15:10.640772+0000","last_active":"2024-04-25T10:15:11.647607+0000","last_peered":"2024-04-25T10:15:11.647607+0000","last_clean":"2024-04-25T10:15:11.647607+0000","last_became_active":"2024-04-25T10:15:10.638297+0000","last_became_peered":"2024-04-25T10:15:10.638297+0000","last_unstale":"2024-04-25T10:15:11.647607+0000","last_undegraded":"2024-04-25T10:15:11.647607+0000","last_fullsized":"2024-04-25T10:15:11.647607+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:05.798975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683031+0000","last_change":"2024-04-25T10:15:11.678918+0000","last_active":"2024-04-25T10:15:11.683031+0000","last_peered":"2024-04-25T10:15:11.683031+0000","last_clean":"2024-04-25T10:15:11.683031+0000","last_became_active":"2024-04-25T10:15:11.678647+0000","last_became_peered":"2024-04-25T10:15:11.678647+0000","last_unstale":"2024-04-25T10:15:11.683031+0000","last_undegraded":"2024-04-25T10:15:11.683031+0000","last_fullsized":"2024-04-25T10:15:11.683031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.106975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.647336+0000","last_change":"2024-04-25T10:15:10.636463+0000","last_active":"2024-04-25T10:15:11.647336+0000","last_peered":"2024-04-25T10:15:11.647336+0000","last_clean":"2024-04-25T10:15:11.647336+0000","last_became_active":"2024-04-25T10:15:10.636167+0000","last_became_peered":"2024-04-25T10:15:10.636167+0000","last_unstale":"2024-04-25T10:15:11.647336+0000","last_undegraded":"2024-04-25T10:15:11.647336+0000","last_fullsized":"2024-04-25T10:15:11.647336+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:58.454376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674215+0000","last_change":"2024-04-25T10:15:11.672702+0000","last_active":"2024-04-25T10:15:11.674215+0000","last_peered":"2024-04-25T10:15:11.674215+0000","last_clean":"2024-04-25T10:15:11.674215+0000","last_became_active":"2024-04-25T10:15:11.672372+0000","last_became_peered":"2024-04-25T10:15:11.672372+0000","last_unstale":"2024-04-25T10:15:11.674215+0000","last_undegraded":"2024-04-25T10:15:11.674215+0000","last_fullsized":"2024-04-25T10:15:11.674215+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:06.358130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.633335+0000","last_change":"2024-04-25T10:15:10.635111+0000","last_active":"2024-04-25T10:15:11.633335+0000","last_peered":"2024-04-25T10:15:11.633335+0000","last_clean":"2024-04-25T10:15:11.633335+0000","last_became_active":"2024-04-25T10:15:10.634846+0000","last_became_peered":"2024-04-25T10:15:10.634846+0000","last_unstale":"2024-04-25T10:15:11.633335+0000","last_undegraded":"2024-04-25T10:15:11.633335+0000","last_fullsized":"2024-04-25T10:15:11.633335+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.411065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663126+0000","last_change":"2024-04-25T10:15:11.656733+0000","last_active":"2024-04-25T10:15:11.663126+0000","last_peered":"2024-04-25T10:15:11.663126+0000","last_clean":"2024-04-25T10:15:11.663126+0000","last_became_active":"2024-04-25T10:15:11.656333+0000","last_became_peered":"2024-04-25T10:15:11.656333+0000","last_unstale":"2024-04-25T10:15:11.663126+0000","last_undegraded":"2024-04-25T10:15:11.663126+0000","last_fullsized":"2024-04-25T10:15:11.663126+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:26.405321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.643106+0000","last_change":"2024-04-25T10:15:10.640786+0000","last_active":"2024-04-25T10:15:11.643106+0000","last_peered":"2024-04-25T10:15:11.643106+0000","last_clean":"2024-04-25T10:15:11.643106+0000","last_became_active":"2024-04-25T10:15:10.640460+0000","last_became_peered":"2024-04-25T10:15:10.640460+0000","last_unstale":"2024-04-25T10:15:11.643106+0000","last_undegraded":"2024-04-25T10:15:11.643106+0000","last_fullsized":"2024-04-25T10:15:11.643106+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.492238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661827+0000","last_change":"2024-04-25T10:15:11.656621+0000","last_active":"2024-04-25T10:15:11.661827+0000","last_peered":"2024-04-25T10:15:11.661827+0000","last_clean":"2024-04-25T10:15:11.661827+0000","last_became_active":"2024-04-25T10:15:11.656201+0000","last_became_peered":"2024-04-25T10:15:11.656201+0000","last_unstale":"2024-04-25T10:15:11.661827+0000","last_undegraded":"2024-04-25T10:15:11.661827+0000","last_fullsized":"2024-04-25T10:15:11.661827+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:26.507412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663186+0000","last_change":"2024-04-25T10:15:10.644685+0000","last_active":"2024-04-25T10:15:11.663186+0000","last_peered":"2024-04-25T10:15:11.663186+0000","last_clean":"2024-04-25T10:15:11.663186+0000","last_became_active":"2024-04-25T10:15:10.644106+0000","last_became_peered":"2024-04-25T10:15:10.644106+0000","last_unstale":"2024-04-25T10:15:11.663186+0000","last_undegraded":"2024-04-25T10:15:11.663186+0000","last_fullsized":"2024-04-25T10:15:11.663186+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:16.468611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672467+0000","last_change":"2024-04-25T10:15:11.670437+0000","last_active":"2024-04-25T10:15:11.672467+0000","last_peered":"2024-04-25T10:15:11.672467+0000","last_clean":"2024-04-25T10:15:11.672467+0000","last_became_active":"2024-04-25T10:15:11.670161+0000","last_became_peered":"2024-04-25T10:15:11.670161+0000","last_unstale":"2024-04-25T10:15:11.672467+0000","last_undegraded":"2024-04-25T10:15:11.672467+0000","last_fullsized":"2024-04-25T10:15:11.672467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:41.573787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.632566+0000","last_change":"2024-04-25T10:15:10.636624+0000","last_active":"2024-04-25T10:15:11.632566+0000","last_peered":"2024-04-25T10:15:11.632566+0000","last_clean":"2024-04-25T10:15:11.632566+0000","last_became_active":"2024-04-25T10:15:10.636230+0000","last_became_peered":"2024-04-25T10:15:10.636230+0000","last_unstale":"2024-04-25T10:15:11.632566+0000","last_undegraded":"2024-04-25T10:15:11.632566+0000","last_fullsized":"2024-04-25T10:15:11.632566+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.626859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683047+0000","last_change":"2024-04-25T10:15:11.680243+0000","last_active":"2024-04-25T10:15:11.683047+0000","last_peered":"2024-04-25T10:15:11.683047+0000","last_clean":"2024-04-25T10:15:11.683047+0000","last_became_active":"2024-04-25T10:15:11.679869+0000","last_became_peered":"2024-04-25T10:15:11.679869+0000","last_unstale":"2024-04-25T10:15:11.683047+0000","last_undegraded":"2024-04-25T10:15:11.683047+0000","last_fullsized":"2024-04-25T10:15:11.683047+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:30.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.648850+0000","last_change":"2024-04-25T10:15:10.623523+0000","last_active":"2024-04-25T10:15:11.648850+0000","last_peered":"2024-04-25T10:15:11.648850+0000","last_clean":"2024-04-25T10:15:11.648850+0000","last_became_active":"2024-04-25T10:15:10.623248+0000","last_became_peered":"2024-04-25T10:15:10.623248+0000","last_unstale":"2024-04-25T10:15:11.648850+0000","last_undegraded":"2024-04-25T10:15:11.648850+0000","last_fullsized":"2024-04-25T10:15:11.648850+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:27.947709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.675969+0000","last_change":"2024-04-25T10:15:11.669025+0000","last_active":"2024-04-25T10:15:11.675969+0000","last_peered":"2024-04-25T10:15:11.675969+0000","last_clean":"2024-04-25T10:15:11.675969+0000","last_became_active":"2024-04-25T10:15:11.668609+0000","last_became_peered":"2024-04-25T10:15:11.668609+0000","last_unstale":"2024-04-25T10:15:11.675969+0000","last_undegraded":"2024-04-25T10:15:11.675969+0000","last_fullsized":"2024-04-25T10:15:11.675969+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.063295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.662658+0000","last_change":"2024-04-25T10:15:10.641533+0000","last_active":"2024-04-25T10:15:11.662658+0000","last_peered":"2024-04-25T10:15:11.662658+0000","last_clean":"2024-04-25T10:15:11.662658+0000","last_became_active":"2024-04-25T10:15:10.641138+0000","last_became_peered":"2024-04-25T10:15:10.641138+0000","last_unstale":"2024-04-25T10:15:11.662658+0000","last_undegraded":"2024-04-25T10:15:11.662658+0000","last_fullsized":"2024-04-25T10:15:11.662658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:12.199656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":387,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.658287+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:15:11.658287+0000","last_peered":"2024-04-25T10:15:11.658287+0000","last_clean":"2024-04-25T10:15:11.658287+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:15:11.658287+0000","last_undegraded":"2024-04-25T10:15:11.658287+0000","last_fullsized":"2024-04-25T10:15:11.658287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2297856,"data_stored":1382650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1675264,"data_stored":783413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50884,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320956,"statfs":{"total":96636764160,"available":96584658944,"internally_reserved":0,"allocated":1691648,"data_stored":790602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46720,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325120,"statfs":{"total":96636764160,"available":96588922880,"internally_reserved":0,"allocated":1687552,"data_stored":790580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1703936,"data_stored":792262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1695744,"data_stored":791196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1691648,"data_stored":790650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2289664,"data_stored":1381004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:16.014 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:15:16.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.244+0000 7ff313e00640 1 Processor -- start 2024-04-25T10:15:16.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.245+0000 7ff313e00640 1 -- start start 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff313e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff31415db90 0x7ff3140b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff313e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3141615a0 0x7ff3140cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff313e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3140d33f0 con 0x7ff3140d04f0 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff313e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3140d0980 con 0x7ff31415db90 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff313e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140d1f50 con 0x7ff3141615a0 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff312a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff31415db90 0x7ff3140b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff312a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff31415db90 0x7ff3140b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35862/0 (socket says 172.21.15.165:35862) 2024-04-25T10:15:16.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.246+0000 7ff312a00640 1 -- 172.21.15.165:0/2903703160 learned_addr learned my addr 172.21.15.165:0/2903703160 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.247+0000 7ff312000640 1 --2- 172.21.15.165:0/2903703160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3141615a0 0x7ff3140cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.247+0000 7ff312a00640 1 -- 172.21.15.165:0/2903703160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3141615a0 msgr2=0x7ff3140cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.247+0000 7ff312a00640 1 --2- 172.21.15.165:0/2903703160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3141615a0 0x7ff3140cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.247+0000 7ff312a00640 1 -- 172.21.15.165:0/2903703160 >> v1:172.21.15.172:6789/0 conn(0x7ff3140d04f0 legacy=0x7ff314061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.247+0000 7ff312a00640 1 -- 172.21.15.165:0/2903703160 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff314161df0 con 0x7ff31415db90 2024-04-25T10:15:16.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.248+0000 7ff312a00640 1 --2- 172.21.15.165:0/2903703160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31415db90 0x7ff3140b93c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff304009870 tx=0x7ff30402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e8b64d0890e41e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff311600640 1 -- 172.21.15.165:0/2903703160 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304037070 con 0x7ff31415db90 2024-04-25T10:15:16.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff311600640 1 -- 172.21.15.165:0/2903703160 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff304032440 con 0x7ff31415db90 2024-04-25T10:15:16.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff311600640 1 -- 172.21.15.165:0/2903703160 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304032810 con 0x7ff31415db90 2024-04-25T10:15:16.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 -- 172.21.15.165:0/2903703160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31415db90 msgr2=0x7ff3140b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 --2- 172.21.15.165:0/2903703160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31415db90 0x7ff3140b93c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff304009870 tx=0x7ff30402a270 comp rx=0 tx=0).stop 2024-04-25T10:15:16.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 -- 172.21.15.165:0/2903703160 shutdown_connections 2024-04-25T10:15:16.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 --2- 172.21.15.165:0/2903703160 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3141615a0 0x7ff3140cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 --2- 172.21.15.165:0/2903703160 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31415db90 0x7ff3140b93c0 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 -- 172.21.15.165:0/2903703160 >> 172.21.15.165:0/2903703160 conn(0x7ff3140b52b0 msgr2=0x7ff3140b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.249+0000 7ff313e00640 1 -- 172.21.15.165:0/2903703160 shutdown_connections 2024-04-25T10:15:16.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.250+0000 7ff313e00640 1 -- 172.21.15.165:0/2903703160 wait complete. 2024-04-25T10:15:16.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.251+0000 7ff313e00640 1 Processor -- start 2024-04-25T10:15:16.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.251+0000 7ff313e00640 1 -- start start 2024-04-25T10:15:16.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.252+0000 7ff313e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3140d04f0 0x7ff3141f46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.252+0000 7ff313e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff31415db90 0x7ff3140c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.252+0000 7ff313e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 0x7ff31414e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.252+0000 7ff313e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3140d1be0 con 0x7ff3141615a0 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.252+0000 7ff313e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3141623b0 con 0x7ff3140d04f0 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.252+0000 7ff313e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff314161a20 con 0x7ff31415db90 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 0x7ff31414e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 0x7ff31414e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35874/0 (socket says 172.21.15.165:35874) 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 -- 172.21.15.165:0/3132915744 learned_addr learned my addr 172.21.15.165:0/3132915744 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff312a00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3140d04f0 0x7ff3141f46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff312000640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff31415db90 0x7ff3140c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 -- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff31415db90 msgr2=0x7ff3140c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff31415db90 0x7ff3140c93f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 -- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3140d04f0 msgr2=0x7ff3141f46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3140d04f0 0x7ff3141f46f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 -- 172.21.15.165:0/3132915744 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3141f77b0 con 0x7ff3141615a0 2024-04-25T10:15:16.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff313400640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 0x7ff31414e850 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c730 tx=0x7ff30c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.253+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c016020 con 0x7ff3141615a0 2024-04-25T10:15:16.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.254+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff30c006d50 con 0x7ff3141615a0 2024-04-25T10:15:16.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.254+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30c003bb0 con 0x7ff3141615a0 2024-04-25T10:15:16.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.254+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3141f7a90 con 0x7ff3141615a0 2024-04-25T10:15:16.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.254+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3141f8050 con 0x7ff3141615a0 2024-04-25T10:15:16.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.255+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3140b8ca0 con 0x7ff3141615a0 2024-04-25T10:15:16.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.256+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff30c0050d0 con 0x7ff3141615a0 2024-04-25T10:15:16.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.257+0000 7ff30aa00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2f006b4d0 0x7ff2f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.257+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff30c08a220 con 0x7ff3141615a0 2024-04-25T10:15:16.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.257+0000 7ff312a00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2f006b4d0 0x7ff2f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.258+0000 7ff312a00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2f006b4d0 0x7ff2f006d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff304034000 tx=0x7ff30402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.261+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3140b8ca0 con 0x7ff3141615a0 2024-04-25T10:15:16.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.455+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3140d1880 con 0x7ff2f006b4d0 2024-04-25T10:15:16.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.473+0000 7ff30aa00640 1 -- 172.21.15.165:0/3132915744 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252414 (secure 0 0 0) 0x7ff3140d1880 con 0x7ff2f006b4d0 2024-04-25T10:15:16.481 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:16.482 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2f006b4d0 msgr2=0x7ff2f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2f006b4d0 0x7ff2f006d990 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7ff304034000 tx=0x7ff30402e000 comp rx=0 tx=0).stop 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 msgr2=0x7ff31414e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 0x7ff31414e850 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff30c00c730 tx=0x7ff30c00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 shutdown_connections 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2f006b4d0 0x7ff2f006d990 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3141615a0 0x7ff31414e850 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff31415db90 0x7ff3140c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 --2- 172.21.15.165:0/3132915744 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3140d04f0 0x7ff3141f46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 >> 172.21.15.165:0/3132915744 conn(0x7ff3140b52b0 msgr2=0x7ff3140b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 shutdown_connections 2024-04-25T10:15:16.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.479+0000 7ff313e00640 1 -- 172.21.15.165:0/3132915744 wait complete. 2024-04-25T10:15:16.501 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":531,"stamp":"2024-04-25T10:15:16.399142+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":441524,"kb_used_data":14388,"kb_used_omap":12,"kb_used_meta":426995,"kb_avail":754533196,"statfs":{"total":773094113280,"available":772641992704,"internally_reserved":0,"allocated":14733312,"data_stored":7502357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":437243442},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001862"},"pg_stats":[{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.667173+0000","last_change":"2024-04-25T10:15:11.664563+0000","last_active":"2024-04-25T10:15:11.667173+0000","last_peered":"2024-04-25T10:15:11.667173+0000","last_clean":"2024-04-25T10:15:11.667173+0000","last_became_active":"2024-04-25T10:15:11.664262+0000","last_became_peered":"2024-04-25T10:15:11.664262+0000","last_unstale":"2024-04-25T10:15:11.667173+0000","last_undegraded":"2024-04-25T10:15:11.667173+0000","last_fullsized":"2024-04-25T10:15:11.667173+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:44:47.822436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.653491+0000","last_change":"2024-04-25T10:15:11.652505+0000","last_active":"2024-04-25T10:15:11.653491+0000","last_peered":"2024-04-25T10:15:11.653491+0000","last_clean":"2024-04-25T10:15:11.653491+0000","last_became_active":"2024-04-25T10:15:11.651525+0000","last_became_peered":"2024-04-25T10:15:11.651525+0000","last_unstale":"2024-04-25T10:15:11.653491+0000","last_undegraded":"2024-04-25T10:15:11.653491+0000","last_fullsized":"2024-04-25T10:15:11.653491+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:53.877359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677752+0000","last_change":"2024-04-25T10:15:11.671894+0000","last_active":"2024-04-25T10:15:11.677752+0000","last_peered":"2024-04-25T10:15:11.677752+0000","last_clean":"2024-04-25T10:15:11.677752+0000","last_became_active":"2024-04-25T10:15:11.671607+0000","last_became_peered":"2024-04-25T10:15:11.671607+0000","last_unstale":"2024-04-25T10:15:11.677752+0000","last_undegraded":"2024-04-25T10:15:11.677752+0000","last_fullsized":"2024-04-25T10:15:11.677752+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:55.512074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.660968+0000","last_change":"2024-04-25T10:15:11.656244+0000","last_active":"2024-04-25T10:15:11.660968+0000","last_peered":"2024-04-25T10:15:11.660968+0000","last_clean":"2024-04-25T10:15:11.660968+0000","last_became_active":"2024-04-25T10:15:11.654467+0000","last_became_peered":"2024-04-25T10:15:11.654467+0000","last_unstale":"2024-04-25T10:15:11.660968+0000","last_undegraded":"2024-04-25T10:15:11.660968+0000","last_fullsized":"2024-04-25T10:15:11.660968+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.655238+0000","last_change":"2024-04-25T10:15:11.652489+0000","last_active":"2024-04-25T10:15:11.655238+0000","last_peered":"2024-04-25T10:15:11.655238+0000","last_clean":"2024-04-25T10:15:11.655238+0000","last_became_active":"2024-04-25T10:15:11.651876+0000","last_became_peered":"2024-04-25T10:15:11.651876+0000","last_unstale":"2024-04-25T10:15:11.655238+0000","last_undegraded":"2024-04-25T10:15:11.655238+0000","last_fullsized":"2024-04-25T10:15:11.655238+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:03.354415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663503+0000","last_change":"2024-04-25T10:15:11.661891+0000","last_active":"2024-04-25T10:15:11.663503+0000","last_peered":"2024-04-25T10:15:11.663503+0000","last_clean":"2024-04-25T10:15:11.663503+0000","last_became_active":"2024-04-25T10:15:11.657224+0000","last_became_peered":"2024-04-25T10:15:11.657224+0000","last_unstale":"2024-04-25T10:15:11.663503+0000","last_undegraded":"2024-04-25T10:15:11.663503+0000","last_fullsized":"2024-04-25T10:15:11.663503+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:06.469199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.682613+0000","last_change":"2024-04-25T10:15:11.677635+0000","last_active":"2024-04-25T10:15:11.682613+0000","last_peered":"2024-04-25T10:15:11.682613+0000","last_clean":"2024-04-25T10:15:11.682613+0000","last_became_active":"2024-04-25T10:15:11.677338+0000","last_became_peered":"2024-04-25T10:15:11.677338+0000","last_unstale":"2024-04-25T10:15:11.682613+0000","last_undegraded":"2024-04-25T10:15:11.682613+0000","last_fullsized":"2024-04-25T10:15:11.682613+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670212+0000","last_change":"2024-04-25T10:15:11.666715+0000","last_active":"2024-04-25T10:15:11.670212+0000","last_peered":"2024-04-25T10:15:11.670212+0000","last_clean":"2024-04-25T10:15:11.670212+0000","last_became_active":"2024-04-25T10:15:11.666409+0000","last_became_peered":"2024-04-25T10:15:11.666409+0000","last_unstale":"2024-04-25T10:15:11.670212+0000","last_undegraded":"2024-04-25T10:15:11.670212+0000","last_fullsized":"2024-04-25T10:15:11.670212+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:26.327083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.662947+0000","last_change":"2024-04-25T10:15:11.655868+0000","last_active":"2024-04-25T10:15:11.662947+0000","last_peered":"2024-04-25T10:15:11.662947+0000","last_clean":"2024-04-25T10:15:11.662947+0000","last_became_active":"2024-04-25T10:15:11.655586+0000","last_became_peered":"2024-04-25T10:15:11.655586+0000","last_unstale":"2024-04-25T10:15:11.662947+0000","last_undegraded":"2024-04-25T10:15:11.662947+0000","last_fullsized":"2024-04-25T10:15:11.662947+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:08.068855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674743+0000","last_change":"2024-04-25T10:15:11.672787+0000","last_active":"2024-04-25T10:15:11.674743+0000","last_peered":"2024-04-25T10:15:11.674743+0000","last_clean":"2024-04-25T10:15:11.674743+0000","last_became_active":"2024-04-25T10:15:11.672520+0000","last_became_peered":"2024-04-25T10:15:11.672520+0000","last_unstale":"2024-04-25T10:15:11.674743+0000","last_undegraded":"2024-04-25T10:15:11.674743+0000","last_fullsized":"2024-04-25T10:15:11.674743+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:06:40.148654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.678273+0000","last_change":"2024-04-25T10:15:11.672484+0000","last_active":"2024-04-25T10:15:11.678273+0000","last_peered":"2024-04-25T10:15:11.678273+0000","last_clean":"2024-04-25T10:15:11.678273+0000","last_became_active":"2024-04-25T10:15:11.671871+0000","last_became_peered":"2024-04-25T10:15:11.671871+0000","last_unstale":"2024-04-25T10:15:11.678273+0000","last_undegraded":"2024-04-25T10:15:11.678273+0000","last_fullsized":"2024-04-25T10:15:11.678273+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:08.455518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661167+0000","last_change":"2024-04-25T10:15:11.659933+0000","last_active":"2024-04-25T10:15:11.661167+0000","last_peered":"2024-04-25T10:15:11.661167+0000","last_clean":"2024-04-25T10:15:11.661167+0000","last_became_active":"2024-04-25T10:15:11.658771+0000","last_became_peered":"2024-04-25T10:15:11.658771+0000","last_unstale":"2024-04-25T10:15:11.661167+0000","last_undegraded":"2024-04-25T10:15:11.661167+0000","last_fullsized":"2024-04-25T10:15:11.661167+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:05.742186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670770+0000","last_change":"2024-04-25T10:15:11.663501+0000","last_active":"2024-04-25T10:15:11.670770+0000","last_peered":"2024-04-25T10:15:11.670770+0000","last_clean":"2024-04-25T10:15:11.670770+0000","last_became_active":"2024-04-25T10:15:11.663199+0000","last_became_peered":"2024-04-25T10:15:11.663199+0000","last_unstale":"2024-04-25T10:15:11.670770+0000","last_undegraded":"2024-04-25T10:15:11.670770+0000","last_fullsized":"2024-04-25T10:15:11.670770+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:01:08.111703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661111+0000","last_change":"2024-04-25T10:15:11.659633+0000","last_active":"2024-04-25T10:15:11.661111+0000","last_peered":"2024-04-25T10:15:11.661111+0000","last_clean":"2024-04-25T10:15:11.661111+0000","last_became_active":"2024-04-25T10:15:11.659088+0000","last_became_peered":"2024-04-25T10:15:11.659088+0000","last_unstale":"2024-04-25T10:15:11.661111+0000","last_undegraded":"2024-04-25T10:15:11.661111+0000","last_fullsized":"2024-04-25T10:15:11.661111+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:28:31.139547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672894+0000","last_change":"2024-04-25T10:15:11.668220+0000","last_active":"2024-04-25T10:15:11.672894+0000","last_peered":"2024-04-25T10:15:11.672894+0000","last_clean":"2024-04-25T10:15:11.672894+0000","last_became_active":"2024-04-25T10:15:11.667072+0000","last_became_peered":"2024-04-25T10:15:11.667072+0000","last_unstale":"2024-04-25T10:15:11.672894+0000","last_undegraded":"2024-04-25T10:15:11.672894+0000","last_fullsized":"2024-04-25T10:15:11.672894+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:56.462785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664094+0000","last_change":"2024-04-25T10:15:11.662542+0000","last_active":"2024-04-25T10:15:11.664094+0000","last_peered":"2024-04-25T10:15:11.664094+0000","last_clean":"2024-04-25T10:15:11.664094+0000","last_became_active":"2024-04-25T10:15:11.662282+0000","last_became_peered":"2024-04-25T10:15:11.662282+0000","last_unstale":"2024-04-25T10:15:11.664094+0000","last_undegraded":"2024-04-25T10:15:11.664094+0000","last_fullsized":"2024-04-25T10:15:11.664094+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:44.990918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664153+0000","last_change":"2024-04-25T10:15:11.663043+0000","last_active":"2024-04-25T10:15:11.664153+0000","last_peered":"2024-04-25T10:15:11.664153+0000","last_clean":"2024-04-25T10:15:11.664153+0000","last_became_active":"2024-04-25T10:15:11.662554+0000","last_became_peered":"2024-04-25T10:15:11.662554+0000","last_unstale":"2024-04-25T10:15:11.664153+0000","last_undegraded":"2024-04-25T10:15:11.664153+0000","last_fullsized":"2024-04-25T10:15:11.664153+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:53.877359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.657067+0000","last_change":"2024-04-25T10:15:11.653941+0000","last_active":"2024-04-25T10:15:11.657067+0000","last_peered":"2024-04-25T10:15:11.657067+0000","last_clean":"2024-04-25T10:15:11.657067+0000","last_became_active":"2024-04-25T10:15:11.653430+0000","last_became_peered":"2024-04-25T10:15:11.653430+0000","last_unstale":"2024-04-25T10:15:11.657067+0000","last_undegraded":"2024-04-25T10:15:11.657067+0000","last_fullsized":"2024-04-25T10:15:11.657067+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:26.103092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677023+0000","last_change":"2024-04-25T10:15:11.671635+0000","last_active":"2024-04-25T10:15:11.677023+0000","last_peered":"2024-04-25T10:15:11.677023+0000","last_clean":"2024-04-25T10:15:11.677023+0000","last_became_active":"2024-04-25T10:15:11.671330+0000","last_became_peered":"2024-04-25T10:15:11.671330+0000","last_unstale":"2024-04-25T10:15:11.677023+0000","last_undegraded":"2024-04-25T10:15:11.677023+0000","last_fullsized":"2024-04-25T10:15:11.677023+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:38.262780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.668361+0000","last_change":"2024-04-25T10:15:11.667567+0000","last_active":"2024-04-25T10:15:11.668361+0000","last_peered":"2024-04-25T10:15:11.668361+0000","last_clean":"2024-04-25T10:15:11.668361+0000","last_became_active":"2024-04-25T10:15:11.666723+0000","last_became_peered":"2024-04-25T10:15:11.666723+0000","last_unstale":"2024-04-25T10:15:11.668361+0000","last_undegraded":"2024-04-25T10:15:11.668361+0000","last_fullsized":"2024-04-25T10:15:11.668361+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:23.024245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.652216+0000","last_change":"2024-04-25T10:15:11.649577+0000","last_active":"2024-04-25T10:15:11.652216+0000","last_peered":"2024-04-25T10:15:11.652216+0000","last_clean":"2024-04-25T10:15:11.652216+0000","last_became_active":"2024-04-25T10:15:11.647923+0000","last_became_peered":"2024-04-25T10:15:11.647923+0000","last_unstale":"2024-04-25T10:15:11.652216+0000","last_undegraded":"2024-04-25T10:15:11.652216+0000","last_fullsized":"2024-04-25T10:15:11.652216+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:06.469199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672059+0000","last_change":"2024-04-25T10:15:11.666744+0000","last_active":"2024-04-25T10:15:11.672059+0000","last_peered":"2024-04-25T10:15:11.672059+0000","last_clean":"2024-04-25T10:15:11.672059+0000","last_became_active":"2024-04-25T10:15:11.666467+0000","last_became_peered":"2024-04-25T10:15:11.666467+0000","last_unstale":"2024-04-25T10:15:11.672059+0000","last_undegraded":"2024-04-25T10:15:11.672059+0000","last_fullsized":"2024-04-25T10:15:11.672059+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:12.394872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674721+0000","last_change":"2024-04-25T10:15:11.667699+0000","last_active":"2024-04-25T10:15:11.674721+0000","last_peered":"2024-04-25T10:15:11.674721+0000","last_clean":"2024-04-25T10:15:11.674721+0000","last_became_active":"2024-04-25T10:15:11.667393+0000","last_became_peered":"2024-04-25T10:15:11.667393+0000","last_unstale":"2024-04-25T10:15:11.674721+0000","last_undegraded":"2024-04-25T10:15:11.674721+0000","last_fullsized":"2024-04-25T10:15:11.674721+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:15.329552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663246+0000","last_change":"2024-04-25T10:15:11.654118+0000","last_active":"2024-04-25T10:15:11.663246+0000","last_peered":"2024-04-25T10:15:11.663246+0000","last_clean":"2024-04-25T10:15:11.663246+0000","last_became_active":"2024-04-25T10:15:11.653815+0000","last_became_peered":"2024-04-25T10:15:11.653815+0000","last_unstale":"2024-04-25T10:15:11.663246+0000","last_undegraded":"2024-04-25T10:15:11.663246+0000","last_fullsized":"2024-04-25T10:15:11.663246+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:15.470375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676464+0000","last_change":"2024-04-25T10:15:11.669879+0000","last_active":"2024-04-25T10:15:11.676464+0000","last_peered":"2024-04-25T10:15:11.676464+0000","last_clean":"2024-04-25T10:15:11.676464+0000","last_became_active":"2024-04-25T10:15:11.668631+0000","last_became_peered":"2024-04-25T10:15:11.668631+0000","last_unstale":"2024-04-25T10:15:11.676464+0000","last_undegraded":"2024-04-25T10:15:11.676464+0000","last_fullsized":"2024-04-25T10:15:11.676464+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:41.020359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.679757+0000","last_change":"2024-04-25T10:15:11.676218+0000","last_active":"2024-04-25T10:15:11.679757+0000","last_peered":"2024-04-25T10:15:11.679757+0000","last_clean":"2024-04-25T10:15:11.679757+0000","last_became_active":"2024-04-25T10:15:11.670407+0000","last_became_peered":"2024-04-25T10:15:11.670407+0000","last_unstale":"2024-04-25T10:15:11.679757+0000","last_undegraded":"2024-04-25T10:15:11.679757+0000","last_fullsized":"2024-04-25T10:15:11.679757+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.473484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.678825+0000","last_change":"2024-04-25T10:15:11.673208+0000","last_active":"2024-04-25T10:15:11.678825+0000","last_peered":"2024-04-25T10:15:11.678825+0000","last_clean":"2024-04-25T10:15:11.678825+0000","last_became_active":"2024-04-25T10:15:11.672901+0000","last_became_peered":"2024-04-25T10:15:11.672901+0000","last_unstale":"2024-04-25T10:15:11.678825+0000","last_undegraded":"2024-04-25T10:15:11.678825+0000","last_fullsized":"2024-04-25T10:15:11.678825+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:51:03.354415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674551+0000","last_change":"2024-04-25T10:15:11.672019+0000","last_active":"2024-04-25T10:15:11.674551+0000","last_peered":"2024-04-25T10:15:11.674551+0000","last_clean":"2024-04-25T10:15:11.674551+0000","last_became_active":"2024-04-25T10:15:11.671743+0000","last_became_peered":"2024-04-25T10:15:11.671743+0000","last_unstale":"2024-04-25T10:15:11.674551+0000","last_undegraded":"2024-04-25T10:15:11.674551+0000","last_fullsized":"2024-04-25T10:15:11.674551+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:16.828048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677177+0000","last_change":"2024-04-25T10:15:11.674602+0000","last_active":"2024-04-25T10:15:11.677177+0000","last_peered":"2024-04-25T10:15:11.677177+0000","last_clean":"2024-04-25T10:15:11.677177+0000","last_became_active":"2024-04-25T10:15:11.674331+0000","last_became_peered":"2024-04-25T10:15:11.674331+0000","last_unstale":"2024-04-25T10:15:11.677177+0000","last_undegraded":"2024-04-25T10:15:11.677177+0000","last_fullsized":"2024-04-25T10:15:11.677177+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:15.470375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676655+0000","last_change":"2024-04-25T10:15:11.668072+0000","last_active":"2024-04-25T10:15:11.676655+0000","last_peered":"2024-04-25T10:15:11.676655+0000","last_clean":"2024-04-25T10:15:11.676655+0000","last_became_active":"2024-04-25T10:15:11.652010+0000","last_became_peered":"2024-04-25T10:15:11.652010+0000","last_unstale":"2024-04-25T10:15:11.676655+0000","last_undegraded":"2024-04-25T10:15:11.676655+0000","last_fullsized":"2024-04-25T10:15:11.676655+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:26.327083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.668914+0000","last_change":"2024-04-25T10:15:11.660019+0000","last_active":"2024-04-25T10:15:11.668914+0000","last_peered":"2024-04-25T10:15:11.668914+0000","last_clean":"2024-04-25T10:15:11.668914+0000","last_became_active":"2024-04-25T10:15:11.659666+0000","last_became_peered":"2024-04-25T10:15:11.659666+0000","last_unstale":"2024-04-25T10:15:11.668914+0000","last_undegraded":"2024-04-25T10:15:11.668914+0000","last_fullsized":"2024-04-25T10:15:11.668914+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:11.604482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670550+0000","last_change":"2024-04-25T10:15:11.666920+0000","last_active":"2024-04-25T10:15:11.670550+0000","last_peered":"2024-04-25T10:15:11.670550+0000","last_clean":"2024-04-25T10:15:11.670550+0000","last_became_active":"2024-04-25T10:15:11.666314+0000","last_became_peered":"2024-04-25T10:15:11.666314+0000","last_unstale":"2024-04-25T10:15:11.670550+0000","last_undegraded":"2024-04-25T10:15:11.670550+0000","last_fullsized":"2024-04-25T10:15:11.670550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:49:38.694735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.668936+0000","last_change":"2024-04-25T10:15:11.657725+0000","last_active":"2024-04-25T10:15:11.668936+0000","last_peered":"2024-04-25T10:15:11.668936+0000","last_clean":"2024-04-25T10:15:11.668936+0000","last_became_active":"2024-04-25T10:15:11.657432+0000","last_became_peered":"2024-04-25T10:15:11.657432+0000","last_unstale":"2024-04-25T10:15:11.668936+0000","last_undegraded":"2024-04-25T10:15:11.668936+0000","last_fullsized":"2024-04-25T10:15:11.668936+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:16.681740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"135'4","reported_seq":16,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.685546+0000","last_change":"2024-04-25T10:15:10.635806+0000","last_active":"2024-04-25T10:15:11.685546+0000","last_peered":"2024-04-25T10:15:11.685546+0000","last_clean":"2024-04-25T10:15:11.685546+0000","last_became_active":"2024-04-25T10:15:10.635481+0000","last_became_peered":"2024-04-25T10:15:10.635481+0000","last_unstale":"2024-04-25T10:15:11.685546+0000","last_undegraded":"2024-04-25T10:15:11.685546+0000","last_fullsized":"2024-04-25T10:15:11.685546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:05.798975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672874+0000","last_change":"2024-04-25T10:15:11.666378+0000","last_active":"2024-04-25T10:15:11.672874+0000","last_peered":"2024-04-25T10:15:11.672874+0000","last_clean":"2024-04-25T10:15:11.672874+0000","last_became_active":"2024-04-25T10:15:11.666107+0000","last_became_peered":"2024-04-25T10:15:11.666107+0000","last_unstale":"2024-04-25T10:15:11.672874+0000","last_undegraded":"2024-04-25T10:15:11.672874+0000","last_fullsized":"2024-04-25T10:15:11.672874+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:22.437705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677579+0000","last_change":"2024-04-25T10:15:10.639733+0000","last_active":"2024-04-25T10:15:11.677579+0000","last_peered":"2024-04-25T10:15:11.677579+0000","last_clean":"2024-04-25T10:15:11.677579+0000","last_became_active":"2024-04-25T10:15:10.639407+0000","last_became_peered":"2024-04-25T10:15:10.639407+0000","last_unstale":"2024-04-25T10:15:11.677579+0000","last_undegraded":"2024-04-25T10:15:11.677579+0000","last_fullsized":"2024-04-25T10:15:11.677579+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:41.809965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664970+0000","last_change":"2024-04-25T10:15:11.662724+0000","last_active":"2024-04-25T10:15:11.664970+0000","last_peered":"2024-04-25T10:15:11.664970+0000","last_clean":"2024-04-25T10:15:11.664970+0000","last_became_active":"2024-04-25T10:15:11.662349+0000","last_became_peered":"2024-04-25T10:15:11.662349+0000","last_unstale":"2024-04-25T10:15:11.664970+0000","last_undegraded":"2024-04-25T10:15:11.664970+0000","last_fullsized":"2024-04-25T10:15:11.664970+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:41.573787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.681904+0000","last_change":"2024-04-25T10:15:10.639834+0000","last_active":"2024-04-25T10:15:11.681904+0000","last_peered":"2024-04-25T10:15:11.681904+0000","last_clean":"2024-04-25T10:15:11.681904+0000","last_became_active":"2024-04-25T10:15:10.638986+0000","last_became_peered":"2024-04-25T10:15:10.638986+0000","last_unstale":"2024-04-25T10:15:11.681904+0000","last_undegraded":"2024-04-25T10:15:11.681904+0000","last_fullsized":"2024-04-25T10:15:11.681904+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:58:17.129952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661069+0000","last_change":"2024-04-25T10:15:11.659786+0000","last_active":"2024-04-25T10:15:11.661069+0000","last_peered":"2024-04-25T10:15:11.661069+0000","last_clean":"2024-04-25T10:15:11.661069+0000","last_became_active":"2024-04-25T10:15:11.659050+0000","last_became_peered":"2024-04-25T10:15:11.659050+0000","last_unstale":"2024-04-25T10:15:11.661069+0000","last_undegraded":"2024-04-25T10:15:11.661069+0000","last_fullsized":"2024-04-25T10:15:11.661069+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:53:56.798107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683010+0000","last_change":"2024-04-25T10:15:10.641648+0000","last_active":"2024-04-25T10:15:11.683010+0000","last_peered":"2024-04-25T10:15:11.683010+0000","last_clean":"2024-04-25T10:15:11.683010+0000","last_became_active":"2024-04-25T10:15:10.640951+0000","last_became_peered":"2024-04-25T10:15:10.640951+0000","last_unstale":"2024-04-25T10:15:11.683010+0000","last_undegraded":"2024-04-25T10:15:11.683010+0000","last_fullsized":"2024-04-25T10:15:11.683010+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:29:21.339581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670526+0000","last_change":"2024-04-25T10:15:11.667364+0000","last_active":"2024-04-25T10:15:11.670526+0000","last_peered":"2024-04-25T10:15:11.670526+0000","last_clean":"2024-04-25T10:15:11.670526+0000","last_became_active":"2024-04-25T10:15:11.666127+0000","last_became_peered":"2024-04-25T10:15:11.666127+0000","last_unstale":"2024-04-25T10:15:11.670526+0000","last_undegraded":"2024-04-25T10:15:11.670526+0000","last_fullsized":"2024-04-25T10:15:11.670526+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.426238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683625+0000","last_change":"2024-04-25T10:15:10.640576+0000","last_active":"2024-04-25T10:15:11.683625+0000","last_peered":"2024-04-25T10:15:11.683625+0000","last_clean":"2024-04-25T10:15:11.683625+0000","last_became_active":"2024-04-25T10:15:10.639671+0000","last_became_peered":"2024-04-25T10:15:10.639671+0000","last_unstale":"2024-04-25T10:15:11.683625+0000","last_undegraded":"2024-04-25T10:15:11.683625+0000","last_fullsized":"2024-04-25T10:15:11.683625+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:45:28.331945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670330+0000","last_change":"2024-04-25T10:15:11.667115+0000","last_active":"2024-04-25T10:15:11.670330+0000","last_peered":"2024-04-25T10:15:11.670330+0000","last_clean":"2024-04-25T10:15:11.670330+0000","last_became_active":"2024-04-25T10:15:11.666840+0000","last_became_peered":"2024-04-25T10:15:11.666840+0000","last_unstale":"2024-04-25T10:15:11.670330+0000","last_undegraded":"2024-04-25T10:15:11.670330+0000","last_fullsized":"2024-04-25T10:15:11.670330+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.063295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.670993+0000","last_change":"2024-04-25T10:15:10.640096+0000","last_active":"2024-04-25T10:15:11.670993+0000","last_peered":"2024-04-25T10:15:11.670993+0000","last_clean":"2024-04-25T10:15:11.670993+0000","last_became_active":"2024-04-25T10:15:10.639752+0000","last_became_peered":"2024-04-25T10:15:10.639752+0000","last_unstale":"2024-04-25T10:15:11.670993+0000","last_undegraded":"2024-04-25T10:15:11.670993+0000","last_fullsized":"2024-04-25T10:15:11.670993+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:19:37.247607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.664754+0000","last_change":"2024-04-25T10:15:11.663571+0000","last_active":"2024-04-25T10:15:11.664754+0000","last_peered":"2024-04-25T10:15:11.664754+0000","last_clean":"2024-04-25T10:15:11.664754+0000","last_became_active":"2024-04-25T10:15:11.663265+0000","last_became_peered":"2024-04-25T10:15:11.663265+0000","last_unstale":"2024-04-25T10:15:11.664754+0000","last_undegraded":"2024-04-25T10:15:11.664754+0000","last_fullsized":"2024-04-25T10:15:11.664754+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:24:58.722845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.632141+0000","last_change":"2024-04-25T10:15:10.632606+0000","last_active":"2024-04-25T10:15:11.632141+0000","last_peered":"2024-04-25T10:15:11.632141+0000","last_clean":"2024-04-25T10:15:11.632141+0000","last_became_active":"2024-04-25T10:15:10.631808+0000","last_became_peered":"2024-04-25T10:15:10.631808+0000","last_unstale":"2024-04-25T10:15:11.632141+0000","last_undegraded":"2024-04-25T10:15:11.632141+0000","last_fullsized":"2024-04-25T10:15:11.632141+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:56.048122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.681550+0000","last_change":"2024-04-25T10:15:11.674233+0000","last_active":"2024-04-25T10:15:11.681550+0000","last_peered":"2024-04-25T10:15:11.681550+0000","last_clean":"2024-04-25T10:15:11.681550+0000","last_became_active":"2024-04-25T10:15:11.673911+0000","last_became_peered":"2024-04-25T10:15:11.673911+0000","last_unstale":"2024-04-25T10:15:11.681550+0000","last_undegraded":"2024-04-25T10:15:11.681550+0000","last_fullsized":"2024-04-25T10:15:11.681550+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:34.132593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.641428+0000","last_change":"2024-04-25T10:15:10.631224+0000","last_active":"2024-04-25T10:15:11.641428+0000","last_peered":"2024-04-25T10:15:11.641428+0000","last_clean":"2024-04-25T10:15:11.641428+0000","last_became_active":"2024-04-25T10:15:10.630886+0000","last_became_peered":"2024-04-25T10:15:10.630886+0000","last_unstale":"2024-04-25T10:15:11.641428+0000","last_undegraded":"2024-04-25T10:15:11.641428+0000","last_fullsized":"2024-04-25T10:15:11.641428+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.637636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672204+0000","last_change":"2024-04-25T10:15:11.670178+0000","last_active":"2024-04-25T10:15:11.672204+0000","last_peered":"2024-04-25T10:15:11.672204+0000","last_clean":"2024-04-25T10:15:11.672204+0000","last_became_active":"2024-04-25T10:15:11.669880+0000","last_became_peered":"2024-04-25T10:15:11.669880+0000","last_unstale":"2024-04-25T10:15:11.672204+0000","last_undegraded":"2024-04-25T10:15:11.672204+0000","last_fullsized":"2024-04-25T10:15:11.672204+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:51:07.621852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.646871+0000","last_change":"2024-04-25T10:15:10.639564+0000","last_active":"2024-04-25T10:15:11.646871+0000","last_peered":"2024-04-25T10:15:11.646871+0000","last_clean":"2024-04-25T10:15:11.646871+0000","last_became_active":"2024-04-25T10:15:10.639264+0000","last_became_peered":"2024-04-25T10:15:10.639264+0000","last_unstale":"2024-04-25T10:15:11.646871+0000","last_undegraded":"2024-04-25T10:15:11.646871+0000","last_fullsized":"2024-04-25T10:15:11.646871+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:25.390148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661391+0000","last_change":"2024-04-25T10:15:11.655786+0000","last_active":"2024-04-25T10:15:11.661391+0000","last_peered":"2024-04-25T10:15:11.661391+0000","last_clean":"2024-04-25T10:15:11.661391+0000","last_became_active":"2024-04-25T10:15:11.655532+0000","last_became_peered":"2024-04-25T10:15:11.655532+0000","last_unstale":"2024-04-25T10:15:11.661391+0000","last_undegraded":"2024-04-25T10:15:11.661391+0000","last_fullsized":"2024-04-25T10:15:11.661391+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:59:19.473484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.644906+0000","last_change":"2024-04-25T10:15:10.637636+0000","last_active":"2024-04-25T10:15:11.644906+0000","last_peered":"2024-04-25T10:15:11.644906+0000","last_clean":"2024-04-25T10:15:11.644906+0000","last_became_active":"2024-04-25T10:15:10.637307+0000","last_became_peered":"2024-04-25T10:15:10.637307+0000","last_unstale":"2024-04-25T10:15:11.644906+0000","last_undegraded":"2024-04-25T10:15:11.644906+0000","last_fullsized":"2024-04-25T10:15:11.644906+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:43:46.171253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676679+0000","last_change":"2024-04-25T10:15:11.669397+0000","last_active":"2024-04-25T10:15:11.676679+0000","last_peered":"2024-04-25T10:15:11.676679+0000","last_clean":"2024-04-25T10:15:11.676679+0000","last_became_active":"2024-04-25T10:15:11.651998+0000","last_became_peered":"2024-04-25T10:15:11.651998+0000","last_unstale":"2024-04-25T10:15:11.676679+0000","last_undegraded":"2024-04-25T10:15:11.676679+0000","last_fullsized":"2024-04-25T10:15:11.676679+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:23.426238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"135'3","reported_seq":15,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683582+0000","last_change":"2024-04-25T10:15:10.644006+0000","last_active":"2024-04-25T10:15:11.683582+0000","last_peered":"2024-04-25T10:15:11.683582+0000","last_clean":"2024-04-25T10:15:11.683582+0000","last_became_active":"2024-04-25T10:15:10.643277+0000","last_became_peered":"2024-04-25T10:15:10.643277+0000","last_unstale":"2024-04-25T10:15:11.683582+0000","last_undegraded":"2024-04-25T10:15:11.683582+0000","last_fullsized":"2024-04-25T10:15:11.683582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:30.478239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676774+0000","last_change":"2024-04-25T10:15:11.667847+0000","last_active":"2024-04-25T10:15:11.676774+0000","last_peered":"2024-04-25T10:15:11.676774+0000","last_clean":"2024-04-25T10:15:11.676774+0000","last_became_active":"2024-04-25T10:15:11.667506+0000","last_became_peered":"2024-04-25T10:15:11.667506+0000","last_unstale":"2024-04-25T10:15:11.676774+0000","last_undegraded":"2024-04-25T10:15:11.676774+0000","last_fullsized":"2024-04-25T10:15:11.676774+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:35:11.604482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"135'6","reported_seq":18,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.686365+0000","last_change":"2024-04-25T10:15:10.642085+0000","last_active":"2024-04-25T10:15:11.686365+0000","last_peered":"2024-04-25T10:15:11.686365+0000","last_clean":"2024-04-25T10:15:11.686365+0000","last_became_active":"2024-04-25T10:15:10.641323+0000","last_became_peered":"2024-04-25T10:15:10.641323+0000","last_unstale":"2024-04-25T10:15:11.686365+0000","last_undegraded":"2024-04-25T10:15:11.686365+0000","last_fullsized":"2024-04-25T10:15:11.686365+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:26.004030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663479+0000","last_change":"2024-04-25T10:15:11.660391+0000","last_active":"2024-04-25T10:15:11.663479+0000","last_peered":"2024-04-25T10:15:11.663479+0000","last_clean":"2024-04-25T10:15:11.663479+0000","last_became_active":"2024-04-25T10:15:11.660024+0000","last_became_peered":"2024-04-25T10:15:11.660024+0000","last_unstale":"2024-04-25T10:15:11.663479+0000","last_undegraded":"2024-04-25T10:15:11.663479+0000","last_fullsized":"2024-04-25T10:15:11.663479+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:58.464571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.675052+0000","last_change":"2024-04-25T10:15:10.637439+0000","last_active":"2024-04-25T10:15:11.675052+0000","last_peered":"2024-04-25T10:15:11.675052+0000","last_clean":"2024-04-25T10:15:11.675052+0000","last_became_active":"2024-04-25T10:15:10.635798+0000","last_became_peered":"2024-04-25T10:15:10.635798+0000","last_unstale":"2024-04-25T10:15:11.675052+0000","last_undegraded":"2024-04-25T10:15:11.675052+0000","last_fullsized":"2024-04-25T10:15:11.675052+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:51:07.315410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.681992+0000","last_change":"2024-04-25T10:15:11.674455+0000","last_active":"2024-04-25T10:15:11.681992+0000","last_peered":"2024-04-25T10:15:11.681992+0000","last_clean":"2024-04-25T10:15:11.681992+0000","last_became_active":"2024-04-25T10:15:11.674118+0000","last_became_peered":"2024-04-25T10:15:11.674118+0000","last_unstale":"2024-04-25T10:15:11.681992+0000","last_undegraded":"2024-04-25T10:15:11.681992+0000","last_fullsized":"2024-04-25T10:15:11.681992+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:26.103092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.650459+0000","last_change":"2024-04-25T10:15:10.641150+0000","last_active":"2024-04-25T10:15:11.650459+0000","last_peered":"2024-04-25T10:15:11.650459+0000","last_clean":"2024-04-25T10:15:11.650459+0000","last_became_active":"2024-04-25T10:15:10.639971+0000","last_became_peered":"2024-04-25T10:15:10.639971+0000","last_unstale":"2024-04-25T10:15:11.650459+0000","last_undegraded":"2024-04-25T10:15:11.650459+0000","last_fullsized":"2024-04-25T10:15:11.650459+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:58.454376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.656937+0000","last_change":"2024-04-25T10:15:11.653269+0000","last_active":"2024-04-25T10:15:11.656937+0000","last_peered":"2024-04-25T10:15:11.656937+0000","last_clean":"2024-04-25T10:15:11.656937+0000","last_became_active":"2024-04-25T10:15:11.652757+0000","last_became_peered":"2024-04-25T10:15:11.652757+0000","last_unstale":"2024-04-25T10:15:11.656937+0000","last_undegraded":"2024-04-25T10:15:11.656937+0000","last_fullsized":"2024-04-25T10:15:11.656937+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:12:34.132593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"135'8","reported_seq":20,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.684179+0000","last_change":"2024-04-25T10:15:10.634313+0000","last_active":"2024-04-25T10:15:11.684179+0000","last_peered":"2024-04-25T10:15:11.684179+0000","last_clean":"2024-04-25T10:15:11.684179+0000","last_became_active":"2024-04-25T10:15:10.633822+0000","last_became_peered":"2024-04-25T10:15:10.633822+0000","last_unstale":"2024-04-25T10:15:11.684179+0000","last_undegraded":"2024-04-25T10:15:11.684179+0000","last_fullsized":"2024-04-25T10:15:11.684179+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:54.496900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676974+0000","last_change":"2024-04-25T10:15:11.673131+0000","last_active":"2024-04-25T10:15:11.676974+0000","last_peered":"2024-04-25T10:15:11.676974+0000","last_clean":"2024-04-25T10:15:11.676974+0000","last_became_active":"2024-04-25T10:15:11.672392+0000","last_became_peered":"2024-04-25T10:15:11.672392+0000","last_unstale":"2024-04-25T10:15:11.676974+0000","last_undegraded":"2024-04-25T10:15:11.676974+0000","last_fullsized":"2024-04-25T10:15:11.676974+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:28.877969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.675362+0000","last_change":"2024-04-25T10:15:10.639953+0000","last_active":"2024-04-25T10:15:11.675362+0000","last_peered":"2024-04-25T10:15:11.675362+0000","last_clean":"2024-04-25T10:15:11.675362+0000","last_became_active":"2024-04-25T10:15:10.639660+0000","last_became_peered":"2024-04-25T10:15:10.639660+0000","last_unstale":"2024-04-25T10:15:11.675362+0000","last_undegraded":"2024-04-25T10:15:11.675362+0000","last_fullsized":"2024-04-25T10:15:11.675362+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:28:43.975745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.673270+0000","last_change":"2024-04-25T10:15:11.669118+0000","last_active":"2024-04-25T10:15:11.673270+0000","last_peered":"2024-04-25T10:15:11.673270+0000","last_clean":"2024-04-25T10:15:11.673270+0000","last_became_active":"2024-04-25T10:15:11.668766+0000","last_became_peered":"2024-04-25T10:15:11.668766+0000","last_unstale":"2024-04-25T10:15:11.673270+0000","last_undegraded":"2024-04-25T10:15:11.673270+0000","last_fullsized":"2024-04-25T10:15:11.673270+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:28.962882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.685273+0000","last_change":"2024-04-25T10:15:10.635510+0000","last_active":"2024-04-25T10:15:11.685273+0000","last_peered":"2024-04-25T10:15:11.685273+0000","last_clean":"2024-04-25T10:15:11.685273+0000","last_became_active":"2024-04-25T10:15:10.634925+0000","last_became_peered":"2024-04-25T10:15:10.634925+0000","last_unstale":"2024-04-25T10:15:11.685273+0000","last_undegraded":"2024-04-25T10:15:11.685273+0000","last_fullsized":"2024-04-25T10:15:11.685273+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:15.666567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.679051+0000","last_change":"2024-04-25T10:15:11.673192+0000","last_active":"2024-04-25T10:15:11.679051+0000","last_peered":"2024-04-25T10:15:11.679051+0000","last_clean":"2024-04-25T10:15:11.679051+0000","last_became_active":"2024-04-25T10:15:11.672805+0000","last_became_peered":"2024-04-25T10:15:11.672805+0000","last_unstale":"2024-04-25T10:15:11.679051+0000","last_undegraded":"2024-04-25T10:15:11.679051+0000","last_fullsized":"2024-04-25T10:15:11.679051+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:19:42.825139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"135'1","reported_seq":13,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.680051+0000","last_change":"2024-04-25T10:15:10.631726+0000","last_active":"2024-04-25T10:15:11.680051+0000","last_peered":"2024-04-25T10:15:11.680051+0000","last_clean":"2024-04-25T10:15:11.680051+0000","last_became_active":"2024-04-25T10:15:10.631403+0000","last_became_peered":"2024-04-25T10:15:10.631403+0000","last_unstale":"2024-04-25T10:15:11.680051+0000","last_undegraded":"2024-04-25T10:15:11.680051+0000","last_fullsized":"2024-04-25T10:15:11.680051+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:40.005185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674131+0000","last_change":"2024-04-25T10:15:11.668937+0000","last_active":"2024-04-25T10:15:11.674131+0000","last_peered":"2024-04-25T10:15:11.674131+0000","last_clean":"2024-04-25T10:15:11.674131+0000","last_became_active":"2024-04-25T10:15:11.668585+0000","last_became_peered":"2024-04-25T10:15:11.668585+0000","last_unstale":"2024-04-25T10:15:11.674131+0000","last_undegraded":"2024-04-25T10:15:11.674131+0000","last_fullsized":"2024-04-25T10:15:11.674131+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:02:07.922921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.647009+0000","last_change":"2024-04-25T10:15:10.631234+0000","last_active":"2024-04-25T10:15:11.647009+0000","last_peered":"2024-04-25T10:15:11.647009+0000","last_clean":"2024-04-25T10:15:11.647009+0000","last_became_active":"2024-04-25T10:15:10.630930+0000","last_became_peered":"2024-04-25T10:15:10.630930+0000","last_unstale":"2024-04-25T10:15:11.647009+0000","last_undegraded":"2024-04-25T10:15:11.647009+0000","last_fullsized":"2024-04-25T10:15:11.647009+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:26.004030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.658240+0000","last_change":"2024-04-25T10:15:11.657444+0000","last_active":"2024-04-25T10:15:11.658240+0000","last_peered":"2024-04-25T10:15:11.658240+0000","last_clean":"2024-04-25T10:15:11.658240+0000","last_became_active":"2024-04-25T10:15:11.656056+0000","last_became_peered":"2024-04-25T10:15:11.656056+0000","last_unstale":"2024-04-25T10:15:11.658240+0000","last_undegraded":"2024-04-25T10:15:11.658240+0000","last_fullsized":"2024-04-25T10:15:11.658240+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.106975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.684084+0000","last_change":"2024-04-25T10:15:10.639880+0000","last_active":"2024-04-25T10:15:11.684084+0000","last_peered":"2024-04-25T10:15:11.684084+0000","last_clean":"2024-04-25T10:15:11.684084+0000","last_became_active":"2024-04-25T10:15:10.638722+0000","last_became_peered":"2024-04-25T10:15:10.638722+0000","last_unstale":"2024-04-25T10:15:11.684084+0000","last_undegraded":"2024-04-25T10:15:11.684084+0000","last_fullsized":"2024-04-25T10:15:11.684084+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:42.637636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.665151+0000","last_change":"2024-04-25T10:15:11.662653+0000","last_active":"2024-04-25T10:15:11.665151+0000","last_peered":"2024-04-25T10:15:11.665151+0000","last_clean":"2024-04-25T10:15:11.665151+0000","last_became_active":"2024-04-25T10:15:11.662375+0000","last_became_peered":"2024-04-25T10:15:11.662375+0000","last_unstale":"2024-04-25T10:15:11.665151+0000","last_undegraded":"2024-04-25T10:15:11.665151+0000","last_fullsized":"2024-04-25T10:15:11.665151+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:14:04.579560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.646824+0000","last_change":"2024-04-25T10:15:10.639010+0000","last_active":"2024-04-25T10:15:11.646824+0000","last_peered":"2024-04-25T10:15:11.646824+0000","last_clean":"2024-04-25T10:15:11.646824+0000","last_became_active":"2024-04-25T10:15:10.638623+0000","last_became_peered":"2024-04-25T10:15:10.638623+0000","last_unstale":"2024-04-25T10:15:11.646824+0000","last_undegraded":"2024-04-25T10:15:11.646824+0000","last_fullsized":"2024-04-25T10:15:11.646824+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:07:52.554699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.657678+0000","last_change":"2024-04-25T10:15:11.655272+0000","last_active":"2024-04-25T10:15:11.657678+0000","last_peered":"2024-04-25T10:15:11.657678+0000","last_clean":"2024-04-25T10:15:11.657678+0000","last_became_active":"2024-04-25T10:15:11.654780+0000","last_became_peered":"2024-04-25T10:15:11.654780+0000","last_unstale":"2024-04-25T10:15:11.657678+0000","last_undegraded":"2024-04-25T10:15:11.657678+0000","last_fullsized":"2024-04-25T10:15:11.657678+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:00.866713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661178+0000","last_change":"2024-04-25T10:15:10.638090+0000","last_active":"2024-04-25T10:15:11.661178+0000","last_peered":"2024-04-25T10:15:11.661178+0000","last_clean":"2024-04-25T10:15:11.661178+0000","last_became_active":"2024-04-25T10:15:10.637747+0000","last_became_peered":"2024-04-25T10:15:10.637747+0000","last_unstale":"2024-04-25T10:15:11.661178+0000","last_undegraded":"2024-04-25T10:15:11.661178+0000","last_fullsized":"2024-04-25T10:15:11.661178+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:14:32.217244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.678592+0000","last_change":"2024-04-25T10:15:11.673350+0000","last_active":"2024-04-25T10:15:11.678592+0000","last_peered":"2024-04-25T10:15:11.678592+0000","last_clean":"2024-04-25T10:15:11.678592+0000","last_became_active":"2024-04-25T10:15:11.671893+0000","last_became_peered":"2024-04-25T10:15:11.671893+0000","last_unstale":"2024-04-25T10:15:11.678592+0000","last_undegraded":"2024-04-25T10:15:11.678592+0000","last_fullsized":"2024-04-25T10:15:11.678592+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:28.516655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.633558+0000","last_change":"2024-04-25T10:15:10.628254+0000","last_active":"2024-04-25T10:15:11.633558+0000","last_peered":"2024-04-25T10:15:11.633558+0000","last_clean":"2024-04-25T10:15:11.633558+0000","last_became_active":"2024-04-25T10:15:10.627969+0000","last_became_peered":"2024-04-25T10:15:10.627969+0000","last_unstale":"2024-04-25T10:15:11.633558+0000","last_undegraded":"2024-04-25T10:15:11.633558+0000","last_fullsized":"2024-04-25T10:15:11.633558+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:17:27.862796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.659467+0000","last_change":"2024-04-25T10:15:11.657988+0000","last_active":"2024-04-25T10:15:11.659467+0000","last_peered":"2024-04-25T10:15:11.659467+0000","last_clean":"2024-04-25T10:15:11.659467+0000","last_became_active":"2024-04-25T10:15:11.657688+0000","last_became_peered":"2024-04-25T10:15:11.657688+0000","last_unstale":"2024-04-25T10:15:11.659467+0000","last_undegraded":"2024-04-25T10:15:11.659467+0000","last_fullsized":"2024-04-25T10:15:11.659467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:30.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"135'2","reported_seq":14,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.677359+0000","last_change":"2024-04-25T10:15:10.638662+0000","last_active":"2024-04-25T10:15:11.677359+0000","last_peered":"2024-04-25T10:15:11.677359+0000","last_clean":"2024-04-25T10:15:11.677359+0000","last_became_active":"2024-04-25T10:15:10.638251+0000","last_became_peered":"2024-04-25T10:15:10.638251+0000","last_unstale":"2024-04-25T10:15:11.677359+0000","last_undegraded":"2024-04-25T10:15:11.677359+0000","last_fullsized":"2024-04-25T10:15:11.677359+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:07:38.508983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.676812+0000","last_change":"2024-04-25T10:15:11.669906+0000","last_active":"2024-04-25T10:15:11.676812+0000","last_peered":"2024-04-25T10:15:11.676812+0000","last_clean":"2024-04-25T10:15:11.676812+0000","last_became_active":"2024-04-25T10:15:11.665312+0000","last_became_peered":"2024-04-25T10:15:11.665312+0000","last_unstale":"2024-04-25T10:15:11.676812+0000","last_undegraded":"2024-04-25T10:15:11.676812+0000","last_fullsized":"2024-04-25T10:15:11.676812+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:40.642033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.647607+0000","last_change":"2024-04-25T10:15:10.640772+0000","last_active":"2024-04-25T10:15:11.647607+0000","last_peered":"2024-04-25T10:15:11.647607+0000","last_clean":"2024-04-25T10:15:11.647607+0000","last_became_active":"2024-04-25T10:15:10.638297+0000","last_became_peered":"2024-04-25T10:15:10.638297+0000","last_unstale":"2024-04-25T10:15:11.647607+0000","last_undegraded":"2024-04-25T10:15:11.647607+0000","last_fullsized":"2024-04-25T10:15:11.647607+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:58:05.798975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683031+0000","last_change":"2024-04-25T10:15:11.678918+0000","last_active":"2024-04-25T10:15:11.683031+0000","last_peered":"2024-04-25T10:15:11.683031+0000","last_clean":"2024-04-25T10:15:11.683031+0000","last_became_active":"2024-04-25T10:15:11.678647+0000","last_became_peered":"2024-04-25T10:15:11.678647+0000","last_unstale":"2024-04-25T10:15:11.683031+0000","last_undegraded":"2024-04-25T10:15:11.683031+0000","last_fullsized":"2024-04-25T10:15:11.683031+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:01:05.106975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.647336+0000","last_change":"2024-04-25T10:15:10.636463+0000","last_active":"2024-04-25T10:15:11.647336+0000","last_peered":"2024-04-25T10:15:11.647336+0000","last_clean":"2024-04-25T10:15:11.647336+0000","last_became_active":"2024-04-25T10:15:10.636167+0000","last_became_peered":"2024-04-25T10:15:10.636167+0000","last_unstale":"2024-04-25T10:15:11.647336+0000","last_undegraded":"2024-04-25T10:15:11.647336+0000","last_fullsized":"2024-04-25T10:15:11.647336+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:58.454376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.674215+0000","last_change":"2024-04-25T10:15:11.672702+0000","last_active":"2024-04-25T10:15:11.674215+0000","last_peered":"2024-04-25T10:15:11.674215+0000","last_clean":"2024-04-25T10:15:11.674215+0000","last_became_active":"2024-04-25T10:15:11.672372+0000","last_became_peered":"2024-04-25T10:15:11.672372+0000","last_unstale":"2024-04-25T10:15:11.674215+0000","last_undegraded":"2024-04-25T10:15:11.674215+0000","last_fullsized":"2024-04-25T10:15:11.674215+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:54:06.358130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.633335+0000","last_change":"2024-04-25T10:15:10.635111+0000","last_active":"2024-04-25T10:15:11.633335+0000","last_peered":"2024-04-25T10:15:11.633335+0000","last_clean":"2024-04-25T10:15:11.633335+0000","last_became_active":"2024-04-25T10:15:10.634846+0000","last_became_peered":"2024-04-25T10:15:10.634846+0000","last_unstale":"2024-04-25T10:15:11.633335+0000","last_undegraded":"2024-04-25T10:15:11.633335+0000","last_fullsized":"2024-04-25T10:15:11.633335+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:22.411065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663126+0000","last_change":"2024-04-25T10:15:11.656733+0000","last_active":"2024-04-25T10:15:11.663126+0000","last_peered":"2024-04-25T10:15:11.663126+0000","last_clean":"2024-04-25T10:15:11.663126+0000","last_became_active":"2024-04-25T10:15:11.656333+0000","last_became_peered":"2024-04-25T10:15:11.656333+0000","last_unstale":"2024-04-25T10:15:11.663126+0000","last_undegraded":"2024-04-25T10:15:11.663126+0000","last_fullsized":"2024-04-25T10:15:11.663126+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:26.405321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.643106+0000","last_change":"2024-04-25T10:15:10.640786+0000","last_active":"2024-04-25T10:15:11.643106+0000","last_peered":"2024-04-25T10:15:11.643106+0000","last_clean":"2024-04-25T10:15:11.643106+0000","last_became_active":"2024-04-25T10:15:10.640460+0000","last_became_peered":"2024-04-25T10:15:10.640460+0000","last_unstale":"2024-04-25T10:15:11.643106+0000","last_undegraded":"2024-04-25T10:15:11.643106+0000","last_fullsized":"2024-04-25T10:15:11.643106+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.492238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.661827+0000","last_change":"2024-04-25T10:15:11.656621+0000","last_active":"2024-04-25T10:15:11.661827+0000","last_peered":"2024-04-25T10:15:11.661827+0000","last_clean":"2024-04-25T10:15:11.661827+0000","last_became_active":"2024-04-25T10:15:11.656201+0000","last_became_peered":"2024-04-25T10:15:11.656201+0000","last_unstale":"2024-04-25T10:15:11.661827+0000","last_undegraded":"2024-04-25T10:15:11.661827+0000","last_fullsized":"2024-04-25T10:15:11.661827+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:26.507412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.663186+0000","last_change":"2024-04-25T10:15:10.644685+0000","last_active":"2024-04-25T10:15:11.663186+0000","last_peered":"2024-04-25T10:15:11.663186+0000","last_clean":"2024-04-25T10:15:11.663186+0000","last_became_active":"2024-04-25T10:15:10.644106+0000","last_became_peered":"2024-04-25T10:15:10.644106+0000","last_unstale":"2024-04-25T10:15:11.663186+0000","last_undegraded":"2024-04-25T10:15:11.663186+0000","last_fullsized":"2024-04-25T10:15:11.663186+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:49:16.468611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.672467+0000","last_change":"2024-04-25T10:15:11.670437+0000","last_active":"2024-04-25T10:15:11.672467+0000","last_peered":"2024-04-25T10:15:11.672467+0000","last_clean":"2024-04-25T10:15:11.672467+0000","last_became_active":"2024-04-25T10:15:11.670161+0000","last_became_peered":"2024-04-25T10:15:11.670161+0000","last_unstale":"2024-04-25T10:15:11.672467+0000","last_undegraded":"2024-04-25T10:15:11.672467+0000","last_fullsized":"2024-04-25T10:15:11.672467+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:41.573787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.632566+0000","last_change":"2024-04-25T10:15:10.636624+0000","last_active":"2024-04-25T10:15:11.632566+0000","last_peered":"2024-04-25T10:15:11.632566+0000","last_clean":"2024-04-25T10:15:11.632566+0000","last_became_active":"2024-04-25T10:15:10.636230+0000","last_became_peered":"2024-04-25T10:15:10.636230+0000","last_unstale":"2024-04-25T10:15:11.632566+0000","last_undegraded":"2024-04-25T10:15:11.632566+0000","last_fullsized":"2024-04-25T10:15:11.632566+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:39.626859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.683047+0000","last_change":"2024-04-25T10:15:11.680243+0000","last_active":"2024-04-25T10:15:11.683047+0000","last_peered":"2024-04-25T10:15:11.683047+0000","last_clean":"2024-04-25T10:15:11.683047+0000","last_became_active":"2024-04-25T10:15:11.679869+0000","last_became_peered":"2024-04-25T10:15:11.679869+0000","last_unstale":"2024-04-25T10:15:11.683047+0000","last_undegraded":"2024-04-25T10:15:11.683047+0000","last_fullsized":"2024-04-25T10:15:11.683047+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:30.833239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.648850+0000","last_change":"2024-04-25T10:15:10.623523+0000","last_active":"2024-04-25T10:15:11.648850+0000","last_peered":"2024-04-25T10:15:11.648850+0000","last_clean":"2024-04-25T10:15:11.648850+0000","last_became_active":"2024-04-25T10:15:10.623248+0000","last_became_peered":"2024-04-25T10:15:10.623248+0000","last_unstale":"2024-04-25T10:15:11.648850+0000","last_undegraded":"2024-04-25T10:15:11.648850+0000","last_fullsized":"2024-04-25T10:15:11.648850+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:37:27.947709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.675969+0000","last_change":"2024-04-25T10:15:11.669025+0000","last_active":"2024-04-25T10:15:11.675969+0000","last_peered":"2024-04-25T10:15:11.675969+0000","last_clean":"2024-04-25T10:15:11.675969+0000","last_became_active":"2024-04-25T10:15:11.668609+0000","last_became_peered":"2024-04-25T10:15:11.668609+0000","last_unstale":"2024-04-25T10:15:11.675969+0000","last_undegraded":"2024-04-25T10:15:11.675969+0000","last_fullsized":"2024-04-25T10:15:11.675969+0000","mapping_epoch":134,"log_start":"0'0","ondisk_log_start":"0'0","created":134,"last_epoch_clean":135,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:10.599386+0000","last_clean_scrub_stamp":"2024-04-25T10:15:10.599386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:19:57.063295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.662658+0000","last_change":"2024-04-25T10:15:10.641533+0000","last_active":"2024-04-25T10:15:11.662658+0000","last_peered":"2024-04-25T10:15:11.662658+0000","last_clean":"2024-04-25T10:15:11.662658+0000","last_became_active":"2024-04-25T10:15:10.641138+0000","last_became_peered":"2024-04-25T10:15:10.641138+0000","last_unstale":"2024-04-25T10:15:11.662658+0000","last_undegraded":"2024-04-25T10:15:11.662658+0000","last_fullsized":"2024-04-25T10:15:11.662658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:15:09.584212+0000","last_clean_scrub_stamp":"2024-04-25T10:15:09.584212+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:12.199656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":387,"reported_epoch":135,"state":"active+clean","last_fresh":"2024-04-25T10:15:11.658287+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:15:11.658287+0000","last_peered":"2024-04-25T10:15:11.658287+0000","last_clean":"2024-04-25T10:15:11.658287+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:15:11.658287+0000","last_undegraded":"2024-04-25T10:15:11.658287+0000","last_fullsized":"2024-04-25T10:15:11.658287+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2297856,"data_stored":1382650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":54655435},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55028,"kb_used_data":1636,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316812,"statfs":{"total":96636764160,"available":96580415488,"internally_reserved":0,"allocated":1675264,"data_stored":783413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50884,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320956,"statfs":{"total":96636764160,"available":96584658944,"internally_reserved":0,"allocated":1691648,"data_stored":790602,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46720,"kb_used_data":1648,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94325120,"statfs":{"total":96636764160,"available":96588922880,"internally_reserved":0,"allocated":1687552,"data_stored":790580,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1664,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1703936,"data_stored":792262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":58915271},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59212,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312628,"statfs":{"total":96636764160,"available":96576131072,"internally_reserved":0,"allocated":1695744,"data_stored":791196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55040,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316800,"statfs":{"total":96636764160,"available":96580403200,"internally_reserved":0,"allocated":1691648,"data_stored":790650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2236,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2289664,"data_stored":1381004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:15:16.505 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:15:16.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:15:16.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.752+0000 7fe907e00640 1 Processor -- start 2024-04-25T10:15:16.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.752+0000 7fe907e00640 1 -- start start 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.753+0000 7fe907e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe908154350 0x7fe90806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.753+0000 7fe907e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90815ae90 con 0x7fe908154730 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe907e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe90815b170 con 0x7fe9080b0e80 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe907e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe90815b450 con 0x7fe908154350 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe8fe000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe908154730 0x7fe90815a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:39480/0 (socket says 172.21.15.165:39480) 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe8fe000640 1 -- 172.21.15.165:0/2498190337 learned_addr learned my addr 172.21.15.165:0/2498190337 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe906a00640 1 --2- 172.21.15.165:0/2498190337 >> v2:172.21.15.165:3300/0 conn(0x7fe908154350 0x7fe90806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe906000640 1 -- 172.21.15.165:0/2498190337 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741973712 0 0) 0x7fe90815ae90 con 0x7fe908154730 2024-04-25T10:15:16.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe906a00640 1 -- 172.21.15.165:0/2498190337 >> v1:172.21.15.172:6789/0 conn(0x7fe9080b0e80 legacy=0x7fe9080711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:16.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe906a00640 1 -- 172.21.15.165:0/2498190337 >> v1:172.21.15.165:6789/0 conn(0x7fe908154730 legacy=0x7fe90815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.754+0000 7fe906a00640 1 -- 172.21.15.165:0/2498190337 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe90806d320 con 0x7fe908154350 2024-04-25T10:15:16.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe906a00640 1 --2- 172.21.15.165:0/2498190337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90806db20 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097a0 tx=0x7fe8f802a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b24fc2efdbd1ed78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe906000640 1 -- 172.21.15.165:0/2498190337 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8037070 con 0x7fe908154350 2024-04-25T10:15:16.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe906000640 1 -- 172.21.15.165:0/2498190337 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f802adc0 con 0x7fe908154350 2024-04-25T10:15:16.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe906000640 1 -- 172.21.15.165:0/2498190337 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8032660 con 0x7fe908154350 2024-04-25T10:15:16.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe907e00640 1 -- 172.21.15.165:0/2498190337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 msgr2=0x7fe90806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe907e00640 1 --2- 172.21.15.165:0/2498190337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90806db20 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80097a0 tx=0x7fe8f802a0b0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe907e00640 1 -- 172.21.15.165:0/2498190337 shutdown_connections 2024-04-25T10:15:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.756+0000 7fe907e00640 1 --2- 172.21.15.165:0/2498190337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90806db20 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.757+0000 7fe907e00640 1 -- 172.21.15.165:0/2498190337 >> 172.21.15.165:0/2498190337 conn(0x7fe908061cc0 msgr2=0x7fe90806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.757+0000 7fe907e00640 1 -- 172.21.15.165:0/2498190337 shutdown_connections 2024-04-25T10:15:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.757+0000 7fe907e00640 1 -- 172.21.15.165:0/2498190337 wait complete. 2024-04-25T10:15:16.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.758+0000 7fe907e00640 1 Processor -- start 2024-04-25T10:15:16.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 -- start start 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9080b0e80 0x7fe90815e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90814c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe908154730 0x7fe9081534e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90806e060 con 0x7fe908154350 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90815ac30 con 0x7fe9080b0e80 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.759+0000 7fe907e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9080718e0 con 0x7fe908154730 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90814c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90814c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35896/0 (socket says 172.21.15.165:35896) 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 -- 172.21.15.165:0/4256727049 learned_addr learned my addr 172.21.15.165:0/4256727049 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe906a00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9080b0e80 0x7fe90815e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 -- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe908154730 msgr2=0x7fe9081534e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe908154730 0x7fe9081534e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 -- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9080b0e80 msgr2=0x7fe90815e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9080b0e80 0x7fe90815e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.760+0000 7fe8fe000640 1 -- 172.21.15.165:0/4256727049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe908157930 con 0x7fe908154350 2024-04-25T10:15:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.761+0000 7fe8fe000640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90814c2f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c3e0 tx=0x7fe8f000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.761+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f00074e0 con 0x7fe908154350 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.761+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f00047e0 con 0x7fe908154350 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.761+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004bd0 con 0x7fe908154350 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.761+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe908157650 con 0x7fe908154350 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.761+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90814cbe0 con 0x7fe908154350 2024-04-25T10:15:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.762+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe908073990 con 0x7fe908154350 2024-04-25T10:15:16.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.763+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe8f0011070 con 0x7fe908154350 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.764+0000 7fe8ff400640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.764+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe8f0089ff0 con 0x7fe908154350 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.765+0000 7fe906a00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:16.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.765+0000 7fe906a00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8034000 tx=0x7fe8f802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:16.769+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe8f0058cd0 con 0x7fe908154350 2024-04-25T10:15:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.035+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9080b2960 con 0x7fe908154350 2024-04-25T10:15:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.040+0000 7fe8ff400640 1 -- 172.21.15.165:0/4256727049 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+147 (secure 0 0 0) 0x7fe8f00585d0 con 0x7fe908154350 2024-04-25T10:15:17.044 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:15:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:15:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:15:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:15:17.044 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:15:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 msgr2=0x7fe8e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8034000 tx=0x7fe8f802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 msgr2=0x7fe90814c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90814c2f0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7fe8f000c3e0 tx=0x7fe8f000c8a0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 shutdown_connections 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe8e406b4d0 0x7fe8e406d990 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe908154730 0x7fe9081534e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe908154350 0x7fe90814c2f0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 --2- 172.21.15.165:0/4256727049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9080b0e80 0x7fe90815e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 >> 172.21.15.165:0/4256727049 conn(0x7fe908061cc0 msgr2=0x7fe9080b2be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 shutdown_connections 2024-04-25T10:15:17.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.043+0000 7fe907e00640 1 -- 172.21.15.165:0/4256727049 wait complete. 2024-04-25T10:15:17.065 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.254+0000 7f4c9a400640 1 Processor -- start 2024-04-25T10:15:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.254+0000 7f4c9a400640 1 -- start start 2024-04-25T10:15:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c9a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c9406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c9a400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4c9406eb10 0x7f4c940ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c9a400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c94144b30 con 0x7f4c940cee60 2024-04-25T10:15:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c9a400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940712d0 con 0x7f4c9406eb10 2024-04-25T10:15:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c9a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c940715b0 con 0x7f4c94144750 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c93e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4c940cee60 0x7f4c9415f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54916/0 (socket says 172.21.15.165:54916) 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c8be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4c9406eb10 0x7f4c940ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.255+0000 7f4c93e00640 1 -- 172.21.15.165:0/1335355811 learned_addr learned my addr 172.21.15.165:0/1335355811 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.256+0000 7f4c93400640 1 --2- 172.21.15.165:0/1335355811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c9406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.256+0000 7f4c8be00640 1 -- 172.21.15.165:0/1335355811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 msgr2=0x7f4c9406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.256+0000 7f4c8be00640 1 --2- 172.21.15.165:0/1335355811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c9406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.256+0000 7f4c8be00640 1 -- 172.21.15.165:0/1335355811 >> v1:172.21.15.165:6789/0 conn(0x7f4c940cee60 legacy=0x7f4c9415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.256+0000 7f4c8be00640 1 -- 172.21.15.165:0/1335355811 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94071890 con 0x7f4c9406eb10 2024-04-25T10:15:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.256+0000 7f4c92a00640 1 -- 172.21.15.165:0/1335355811 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160370557 0 0) 0x7f4c94144b30 con 0x7f4c940cee60 2024-04-25T10:15:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.257+0000 7f4c8be00640 1 --2- 172.21.15.165:0/1335355811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c940ce920 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0097a0 tx=0x7f4c7c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b27ba88ee14ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.257+0000 7f4c92a00640 1 -- 172.21.15.165:0/1335355811 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c037070 con 0x7f4c9406eb10 2024-04-25T10:15:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.257+0000 7f4c92a00640 1 -- 172.21.15.165:0/1335355811 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c7c032440 con 0x7f4c9406eb10 2024-04-25T10:15:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.257+0000 7f4c92a00640 1 -- 172.21.15.165:0/1335355811 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c7c032810 con 0x7f4c9406eb10 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 -- 172.21.15.165:0/1335355811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 msgr2=0x7f4c940ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 --2- 172.21.15.165:0/1335355811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c940ce920 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c0097a0 tx=0x7f4c7c02a180 comp rx=0 tx=0).stop 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 -- 172.21.15.165:0/1335355811 shutdown_connections 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 --2- 172.21.15.165:0/1335355811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c940ce920 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 --2- 172.21.15.165:0/1335355811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c9406e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 -- 172.21.15.165:0/1335355811 >> 172.21.15.165:0/1335355811 conn(0x7f4c9406abe0 msgr2=0x7f4c9406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 -- 172.21.15.165:0/1335355811 shutdown_connections 2024-04-25T10:15:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.258+0000 7f4c9a400640 1 -- 172.21.15.165:0/1335355811 wait complete. 2024-04-25T10:15:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.260+0000 7f4c9a400640 1 Processor -- start 2024-04-25T10:15:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.260+0000 7f4c9a400640 1 -- start start 2024-04-25T10:15:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c9a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c94157260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c9a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c940cee60 0x7f4c941e5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c9a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c941e63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c9a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94145450 con 0x7f4c9406eb10 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c9a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9406f020 con 0x7f4c940cee60 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c9a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c941455d0 con 0x7f4c94144750 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c93400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c94157260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c93400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c94157260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57530/0 (socket says 172.21.15.165:57530) 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.261+0000 7f4c93400640 1 -- 172.21.15.165:0/3326127762 learned_addr learned my addr 172.21.15.165:0/3326127762 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c8be00640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c940cee60 0x7f4c941e5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93e00640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c941e63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93400640 1 -- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 msgr2=0x7f4c941e63c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c941e63c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93400640 1 -- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c940cee60 msgr2=0x7f4c941e5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c940cee60 0x7f4c941e5e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93400640 1 -- 172.21.15.165:0/3326127762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c940af2c0 con 0x7f4c9406eb10 2024-04-25T10:15:17.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c93400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c94157260 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d5f0 tx=0x7f4c8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.262+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840125f0 con 0x7f4c9406eb10 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.263+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c84019070 con 0x7f4c9406eb10 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.263+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9412fb90 con 0x7f4c9406eb10 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.263+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c840129d0 con 0x7f4c9406eb10 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.263+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c94130150 con 0x7f4c9406eb10 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.264+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94073990 con 0x7f4c9406eb10 2024-04-25T10:15:17.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.265+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c84012b70 con 0x7f4c9406eb10 2024-04-25T10:15:17.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.266+0000 7f4c8b400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c7006b4d0 0x7f4c7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:17.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.266+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c8408a7c0 con 0x7f4c9406eb10 2024-04-25T10:15:17.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.266+0000 7f4c8be00640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c7006b4d0 0x7f4c7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:17.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.267+0000 7f4c8be00640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c7006b4d0 0x7f4c7006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c034000 tx=0x7f4c7c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:17.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.271+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c84059490 con 0x7f4c9406eb10 2024-04-25T10:15:17.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.504+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c940d0510 con 0x7f4c9406eb10 2024-04-25T10:15:17.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.505+0000 7f4c8b400640 1 -- 172.21.15.165:0/3326127762 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f4c84058d90 con 0x7f4c9406eb10 2024-04-25T10:15:17.510 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:17.510 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:17.511 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.508+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c7006b4d0 msgr2=0x7f4c7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.508+0000 7f4c9a400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c7006b4d0 0x7f4c7006d990 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c034000 tx=0x7f4c7c02e000 comp rx=0 tx=0).stop 2024-04-25T10:15:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.508+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 msgr2=0x7f4c94157260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.508+0000 7f4c9a400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c94157260 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400d5f0 tx=0x7f4c8400dab0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 shutdown_connections 2024-04-25T10:15:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c7006b4d0 0x7f4c7006d990 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c94144750 0x7f4c941e63c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c940cee60 0x7f4c941e5e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 --2- 172.21.15.165:0/3326127762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c9406eb10 0x7f4c94157260 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 >> 172.21.15.165:0/3326127762 conn(0x7f4c9406abe0 msgr2=0x7f4c9406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 shutdown_connections 2024-04-25T10:15:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:17.509+0000 7f4c9a400640 1 -- 172.21.15.165:0/3326127762 wait complete. 2024-04-25T10:15:17.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 90, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:15:11.077981+0000', 'modified': '2024-04-25T10:15:14.138367+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9234}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9234': {'gid': 9234, 'name': 'b', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6835/2186062364', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6834', 'nonce': 2186062364}, {'type': 'v1', 'addr': '172.21.15.172:6835', 'nonce': 2186062364}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:15:17.934 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:15:17.934 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-25T10:15:18.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.136+0000 7f228a000640 1 Processor -- start 2024-04-25T10:15:18.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.137+0000 7f228a000640 1 -- start start 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f228a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f228406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f228a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f22841452a0 0x7f228406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f228a000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22840cccc0 con 0x7f22841446d0 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f228a000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22840b2b20 con 0x7f22841452a0 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f228a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228406d400 con 0x7f2284144d50 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f2283400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f22841446d0 0x7f22840b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54922/0 (socket says 172.21.15.165:54922) 2024-04-25T10:15:18.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.138+0000 7f2283400640 1 -- 172.21.15.165:0/3238787362 learned_addr learned my addr 172.21.15.165:0/3238787362 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282a00640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f228406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125185721 0 0) 0x7f22840cccc0 con 0x7f22841446d0 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2268003520 con 0x7f22841446d0 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2283e00640 1 --2- 172.21.15.165:0/3238787362 >> v2:172.21.15.172:3300/0 conn(0x7f22841452a0 0x7f228406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525876270 0 0) 0x7f2268003520 con 0x7f22841446d0 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 msgr2=0x7f228406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f228406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 >> v2:172.21.15.172:3300/0 conn(0x7f22841452a0 msgr2=0x7f228406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 --2- 172.21.15.165:0/3238787362 >> v2:172.21.15.172:3300/0 conn(0x7f22841452a0 0x7f228406ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840cc140 con 0x7f22841446d0 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f22740019f0 con 0x7f22841446d0 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 >> v1:172.21.15.165:6789/0 conn(0x7f22841446d0 legacy=0x7f22840b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.139+0000 7f2282000640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f22680039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2282000640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2268004030 0x7f2268004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2282000640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f22680051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840b2b20 con 0x7f2268004d40 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228406d400 con 0x7f2268004030 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226800bc00 con 0x7f2284144d50 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 --2- 172.21.15.165:0/3238787362 >> v2:172.21.15.172:3300/0 conn(0x7f22841452a0 0x7f228406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f22680051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2282a00640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2268004030 0x7f2268004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283400640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f22680039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 -- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 msgr2=0x7f22680039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f22680039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 -- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2268004030 msgr2=0x7f2268004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2268004030 0x7f2268004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283e00640 1 -- 172.21.15.165:0/3238787362 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2268003520 con 0x7f2268004d40 2024-04-25T10:15:18.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.140+0000 7f2283400640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2284144d50 0x7f22680039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.141+0000 7f2283e00640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f22680051c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f227c00aa70 tx=0x7f2268005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f662fdbbe41f14 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c00a640 con 0x7f2268004d40 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f227c02ac70 con 0x7f2268004d40 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f2282000640 1 -- 172.21.15.165:0/3238787362 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c004240 con 0x7f2268004d40 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f228a000640 1 -- 172.21.15.165:0/3238787362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 msgr2=0x7f22680051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f228a000640 1 --2- 172.21.15.165:0/3238787362 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f22680051c0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f227c00aa70 tx=0x7f2268005830 comp rx=0 tx=0).stop 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f2283e00640 1 -- 172.21.15.165:0/3238787362 reap_dead start 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f228a000640 1 -- 172.21.15.165:0/3238787362 shutdown_connections 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.142+0000 7f228a000640 1 -- 172.21.15.165:0/3238787362 >> 172.21.15.165:0/3238787362 conn(0x7f228406abe0 msgr2=0x7f228406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.143+0000 7f228a000640 1 -- 172.21.15.165:0/3238787362 shutdown_connections 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.143+0000 7f228a000640 1 -- 172.21.15.165:0/3238787362 wait complete. 2024-04-25T10:15:18.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.144+0000 7f228a000640 1 Processor -- start 2024-04-25T10:15:18.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.144+0000 7f228a000640 1 -- start start 2024-04-25T10:15:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.145+0000 7f228a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f228415b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.145+0000 7f228a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22841f4f20 0x7f22841f5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.145+0000 7f228a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22841e25a0 0x7f22841e2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.145+0000 7f228a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840cc9e0 con 0x7f2268004d40 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.145+0000 7f228a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840cc700 con 0x7f22841e25a0 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.145+0000 7f228a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2284144ab0 con 0x7f22841f4f20 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f228415b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f228415b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57548/0 (socket says 172.21.15.165:57548) 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 -- 172.21.15.165:0/3450441874 learned_addr learned my addr 172.21.15.165:0/3450441874 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283e00640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22841e25a0 0x7f22841e2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 -- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22841f4f20 msgr2=0x7f22841f5730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22841f4f20 0x7f22841f5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 -- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22841e25a0 msgr2=0x7f22841e2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22841e25a0 0x7f22841e2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 -- 172.21.15.165:0/3450441874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22840ad610 con 0x7f2268004d40 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.146+0000 7f2283400640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f228415b510 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2274002230 tx=0x7f227400b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.147+0000 7f227b400640 1 -- 172.21.15.165:0/3450441874 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274019070 con 0x7f2268004d40 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.147+0000 7f227b400640 1 -- 172.21.15.165:0/3450441874 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2274004d60 con 0x7f2268004d40 2024-04-25T10:15:18.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.147+0000 7f227b400640 1 -- 172.21.15.165:0/3450441874 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740072d0 con 0x7f2268004d40 2024-04-25T10:15:18.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.147+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22840ad8f0 con 0x7f2268004d40 2024-04-25T10:15:18.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.147+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22840adeb0 con 0x7f2268004d40 2024-04-25T10:15:18.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.149+0000 7f227b400640 1 -- 172.21.15.165:0/3450441874 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f227401c070 con 0x7f2268004d40 2024-04-25T10:15:18.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.150+0000 7f227b400640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f226006b4d0 0x7f226006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.150+0000 7f227b400640 1 -- 172.21.15.165:0/3450441874 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f227408a650 con 0x7f2268004d40 2024-04-25T10:15:18.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.150+0000 7f2282a00640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f226006b4d0 0x7f226006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.150+0000 7f228cd8a740 1 Processor -- start 2024-04-25T10:15:18.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.151+0000 7f228cd8a740 1 -- start start 2024-04-25T10:15:18.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.151+0000 7f2282a00640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f226006b4d0 0x7f226006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f22841f5340 tx=0x7f226c00d280 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964950 0x55cdcf98cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 0x55cdcf965a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf963de0 0x55cdcf9ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f228cd8a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf8c8620 con 0x55cdcf964d30 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf871ca0 con 0x55cdcf964950 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf8c46c0 con 0x55cdcf963de0 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 0x55cdcf965a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 0x55cdcf965a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57560/0 (socket says 172.21.15.165:57560) 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 -- 172.21.15.165:0/797908131 learned_addr learned my addr 172.21.15.165:0/797908131 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2283400640 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964950 0x55cdcf98cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2283e00640 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf963de0 0x55cdcf9ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 -- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf963de0 msgr2=0x55cdcf9ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf963de0 0x55cdcf9ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 -- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964950 msgr2=0x55cdcf98cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964950 0x55cdcf98cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2282a00640 1 -- 172.21.15.165:0/797908131 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdcf878f40 con 0x55cdcf964d30 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.152+0000 7f2283e00640 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf963de0 0x55cdcf9ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.153+0000 7f2282a00640 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 0x55cdcf965a80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f226c00f960 tx=0x7f226c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9215da6b692140 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.153+0000 7f2273400640 1 -- 172.21.15.165:0/797908131 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c01b050 con 0x55cdcf964d30 2024-04-25T10:15:18.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.153+0000 7f2273400640 1 -- 172.21.15.165:0/797908131 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c025070 con 0x55cdcf964d30 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.153+0000 7f2273400640 1 -- 172.21.15.165:0/797908131 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226c014e00 con 0x55cdcf964d30 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 -- 172.21.15.165:0/797908131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 msgr2=0x55cdcf965a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 0x55cdcf965a80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f226c00f960 tx=0x7f226c012000 comp rx=0 tx=0).stop 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 -- 172.21.15.165:0/797908131 shutdown_connections 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf963de0 0x55cdcf9ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964d30 0x55cdcf965a80 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 --2- 172.21.15.165:0/797908131 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964950 0x55cdcf98cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 -- 172.21.15.165:0/797908131 >> 172.21.15.165:0/797908131 conn(0x55cdcf961b70 msgr2=0x55cdcf9aa550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 -- 172.21.15.165:0/797908131 shutdown_connections 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.154+0000 7f228cd8a740 1 -- 172.21.15.165:0/797908131 wait complete. 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.155+0000 7f228cd8a740 1 -- start start 2024-04-25T10:15:18.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.155+0000 7f228cd8a740 1 Processor -- start 2024-04-25T10:15:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf964950 0x55cdcf989e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf977ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 0x55cdcf9add70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf95fa80 con 0x55cdcf9ac350 2024-04-25T10:15:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf8671f0 con 0x55cdcf964d30 2024-04-25T10:15:18.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf9aaf10 con 0x55cdcf964950 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f228cd8a740 10 client.0 _ll_register_callbacks cb 0x55cdcf884ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f2283e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 0x55cdcf9add70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f2283e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 0x55cdcf9add70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57568/0 (socket says 172.21.15.165:57568) 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f2283e00640 1 -- 172.21.15.165:0/1874022757 learned_addr learned my addr 172.21.15.165:0/1874022757 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f2283400640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf964950 0x55cdcf989e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f2283e00640 1 -- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf964950 msgr2=0x55cdcf989e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.156+0000 7f2283e00640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf964950 0x55cdcf989e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2283e00640 1 -- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 msgr2=0x55cdcf977ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2282a00640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf977ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2283e00640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf977ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2283e00640 1 -- 172.21.15.165:0/1874022757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdcf886c50 con 0x55cdcf9ac350 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2283e00640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 0x55cdcf9add70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f227c034040 tx=0x7f227c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35350e1c878f639d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2267e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2267e00640 1 -- 172.21.15.165:0/1874022757 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227c032910 con 0x55cdcf9ac350 2024-04-25T10:15:18.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2267e00640 1 -- 172.21.15.165:0/1874022757 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f227c046c30 con 0x55cdcf9ac350 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f226c025070 con 0x55cdcf9ac350 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.157+0000 7f2267e00640 1 -- 172.21.15.165:0/1874022757 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f227c046e30 con 0x55cdcf9ac350 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 10 client.9360 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55cdcf5dd2d0 con 0x55cdcf9ac350 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f2267e00640 1 -- 172.21.15.165:0/1874022757 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226c025070 con 0x55cdcf9ac350 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f2267e00640 1 -- 172.21.15.165:0/1874022757 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f227c032ab0 con 0x55cdcf9ac350 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 10 client.9360 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 10 client.9360 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 20 client.9360 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf9c2110 0x55cdcf9c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 4 client.9360 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.158+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdcf8d5b60 con 0x55cdcf9c2110 2024-04-25T10:15:18.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.159+0000 7f2283400640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf9c2110 0x55cdcf9c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.159+0000 7f2283400640 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf9c2110 0x55cdcf9c24f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7677ba8d0313d585 server_cookie=e0a214aadf70f156 in_seq=0 out_seq=0 2024-04-25T10:15:18.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.159+0000 7f2267e00640 10 client.9360 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f2267e00640 1 -- 172.21.15.165:0/1874022757 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55cdcf8d5b60 con 0x55cdcf9c2110 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f2267e00640 10 client.9360 handle_command_reply: tid=0 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 client.9360 shutdown 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 msgr2=0x55cdcf9add70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 0x55cdcf9add70 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f227c034040 tx=0x7f227c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 shutdown_connections 2024-04-25T10:15:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf9ac350 0x55cdcf9add70 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf9c2110 0x55cdcf9c24f0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf977ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1874022757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf964950 0x55cdcf989e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 >> 172.21.15.165:0/1874022757 conn(0x55cdcf963de0 msgr2=0x55cdcf98c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.162+0000 7f2267e00640 0 client.9360 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.163+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 shutdown_connections 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.163+0000 7f228cd8a740 1 -- 172.21.15.165:0/1874022757 wait complete. 2024-04-25T10:15:18.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.163+0000 7f228cd8a740 20 client.9360 trim_cache size 0 max 16384 2024-04-25T10:15:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.180+0000 7f228cd8a740 1 Processor -- start 2024-04-25T10:15:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.180+0000 7f228cd8a740 1 -- start start 2024-04-25T10:15:18.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcf9c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf988b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f228cd8a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22840cedd0 con 0x55cdcf964950 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22841e1f90 con 0x55cdcf964d30 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22840ce7d0 con 0x55cdcf9ac350 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f2283400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcf9c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f2283400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcf9c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57578/0 (socket says 172.21.15.165:57578) 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.181+0000 7f2283400640 1 -- 172.21.15.165:0/1749280495 learned_addr learned my addr 172.21.15.165:0/1749280495 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2282a00640 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283e00640 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf988b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283400640 1 -- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 msgr2=0x55cdcf988b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283400640 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf988b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283400640 1 -- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 msgr2=0x55cdcf9c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283400640 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283400640 1 -- 172.21.15.165:0/1749280495 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdcf6a76b0 con 0x55cdcf964950 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283e00640 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf988b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2283400640 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcf9c8520 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f227400ba60 tx=0x7f227400bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db57c0214ec975b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2272a00640 1 -- 172.21.15.165:0/1749280495 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274056f30 con 0x55cdcf964950 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2272a00640 1 -- 172.21.15.165:0/1749280495 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740552a0 con 0x55cdcf964950 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.182+0000 7f2272a00640 1 -- 172.21.15.165:0/1749280495 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22740582a0 con 0x55cdcf964950 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 -- 172.21.15.165:0/1749280495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 msgr2=0x55cdcf9c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcf9c8520 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f227400ba60 tx=0x7f227400bf40 comp rx=0 tx=0).stop 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 -- 172.21.15.165:0/1749280495 shutdown_connections 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf988b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 --2- 172.21.15.165:0/1749280495 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcf9c8520 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 -- 172.21.15.165:0/1749280495 >> 172.21.15.165:0/1749280495 conn(0x55cdcf963de0 msgr2=0x55cdcf9c12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 -- 172.21.15.165:0/1749280495 shutdown_connections 2024-04-25T10:15:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.183+0000 7f228cd8a740 1 -- 172.21.15.165:0/1749280495 wait complete. 2024-04-25T10:15:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.184+0000 7f228cd8a740 1 -- start start 2024-04-25T10:15:18.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.184+0000 7f228cd8a740 1 Processor -- start 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcfa194c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf9c8a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf990c80 con 0x55cdcf964950 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf9ad570 con 0x55cdcf964d30 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55cdcf95e8b0 con 0x55cdcf9ac350 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f228cd8a740 10 client.0 _ll_register_callbacks cb 0x7f22680036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f2283400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcfa194c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f2283400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcfa194c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57594/0 (socket says 172.21.15.165:57594) 2024-04-25T10:15:18.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.185+0000 7f2283400640 1 -- 172.21.15.165:0/3221934751 learned_addr learned my addr 172.21.15.165:0/3221934751 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283e00640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf9c8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2282a00640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283400640 1 -- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 msgr2=0x55cdcf9c8a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283400640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf9c8a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283400640 1 -- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 msgr2=0x55cdcf9c8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283400640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283400640 1 -- 172.21.15.165:0/3221934751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdcf989840 con 0x55cdcf964950 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283e00640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf9c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2282a00640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2283400640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcfa194c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f227400ba60 tx=0x7f22740888b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf854ba59240a1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2267e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2267e00640 1 -- 172.21.15.165:0/3221934751 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2274076060 con 0x55cdcf964950 2024-04-25T10:15:18.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2267e00640 1 -- 172.21.15.165:0/3221934751 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2274058280 con 0x55cdcf964950 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f22740552a0 con 0x55cdcf964950 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f2267e00640 1 -- 172.21.15.165:0/3221934751 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2274054980 con 0x55cdcf964950 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f228cd8a740 10 client.9366 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.186+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x55cdcfa19e20 con 0x55cdcf964950 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f2267e00640 1 -- 172.21.15.165:0/3221934751 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22740552a0 con 0x55cdcf964950 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f2267e00640 1 -- 172.21.15.165:0/3221934751 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f2274054b20 con 0x55cdcf964950 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f228cd8a740 10 client.9366 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f228cd8a740 10 client.9366 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f228cd8a740 20 client.9366 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f228cd8a740 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf991670 0x55cdcf991a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f228cd8a740 4 client.9366 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55cdcf9622d0 con 0x55cdcf991670 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.187+0000 7f2282a00640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf991670 0x55cdcf991a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.188+0000 7f2282a00640 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf991670 0x55cdcf991a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7223363782edcfdd server_cookie=82d5afc12ab6e289 in_seq=0 out_seq=0 2024-04-25T10:15:18.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.188+0000 7f2267e00640 10 client.9366 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f2267e00640 1 -- 172.21.15.165:0/3221934751 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55cdcf9622d0 con 0x55cdcf991670 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f2267e00640 10 client.9366 handle_command_reply: tid=0 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 client.9366 shutdown 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 msgr2=0x55cdcfa194c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcfa194c0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f227400ba60 tx=0x7f22740888b0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 shutdown_connections 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55cdcf9ac350 0x55cdcf9c8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x55cdcf991670 0x55cdcf991a50 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55cdcf964d30 0x55cdcf9c8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 --2- 172.21.15.165:0/3221934751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55cdcf964950 0x55cdcfa194c0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f2267e00640 0 client.9366 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 >> 172.21.15.165:0/3221934751 conn(0x55cdcf963de0 msgr2=0x55cdcf9c12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.189+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 shutdown_connections 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228cd8a740 1 -- 172.21.15.165:0/3221934751 wait complete. 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228cd8a740 20 client.9366 trim_cache size 0 max 16384 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 9234, 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:15:18.197 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.5386439059999999, 2024-04-25T10:15:18.198 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 90, 2024-04-25T10:15:18.198 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 136, 2024-04-25T10:15:18.198 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 136, 2024-04-25T10:15:18.198 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 13.028552306 2024-04-25T10:15:18.198 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:15:18.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f226006b4d0 msgr2=0x7f226006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f226006b4d0 0x7f226006d990 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f22841f5340 tx=0x7f226c00d280 comp rx=0 tx=0).stop 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 msgr2=0x7f228415b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f228415b510 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f2274002230 tx=0x7f227400b560 comp rx=0 tx=0).stop 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 shutdown_connections 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f226006b4d0 0x7f226006d990 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22841e25a0 0x7f22841e2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22841f4f20 0x7f22841f5730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 --2- 172.21.15.165:0/3450441874 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2268004d40 0x7f228415b510 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.190+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 >> 172.21.15.165:0/3450441874 conn(0x7f228406abe0 msgr2=0x7f22840b19d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.191+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 shutdown_connections 2024-04-25T10:15:18.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.191+0000 7f228a000640 1 -- 172.21.15.165:0/3450441874 wait complete. 2024-04-25T10:15:18.212 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:15:18.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.404+0000 7f4621c00640 1 Processor -- start 2024-04-25T10:15:18.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.404+0000 7f4621c00640 1 -- start start 2024-04-25T10:15:18.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.406+0000 7f4621c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 0x7f461c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f4621c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c15bf30 con 0x7f461c144440 2024-04-25T10:15:18.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f4621c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f461c15c210 con 0x7f461c071700 2024-04-25T10:15:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f4621c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c15c860 con 0x7f461c15dd00 2024-04-25T10:15:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f461be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f461c144440 0x7f461c070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54936/0 (socket says 172.21.15.165:54936) 2024-04-25T10:15:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f461be00640 1 -- 172.21.15.165:0/3891643889 learned_addr learned my addr 172.21.15.165:0/3891643889 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:18.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f4621200640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 0x7f461c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.407+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749715437 0 0) 0x7f461c15bf30 con 0x7f461c144440 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4600003520 con 0x7f461c144440 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1561534339 0 0) 0x7f4600003520 con 0x7f461c144440 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 msgr2=0x7f461c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 0x7f461c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 >> v1:172.21.15.172:6789/0 conn(0x7f461c071700 legacy=0x7f461c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c15c4f0 con 0x7f461c144440 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f460c001670 con 0x7f461c144440 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 >> v1:172.21.15.165:6789/0 conn(0x7f461c144440 legacy=0x7f461c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46000039e0 0x7f4600003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4600004800 0x7f4600004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 0x7f46000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c15c860 con 0x7f46000039e0 2024-04-25T10:15:18.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f460000bb30 con 0x7f4600004800 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f460000bd30 con 0x7f461c15dd00 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.408+0000 7f461be00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46000039e0 0x7f4600003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f461b400640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4600004800 0x7f4600004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f4621200640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 0x7f46000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f461be00640 1 -- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 msgr2=0x7f46000051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f461be00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f461c15dd00 0x7f46000051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f461be00640 1 -- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4600004800 msgr2=0x7f4600004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f461be00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4600004800 0x7f4600004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.409+0000 7f461be00640 1 -- 172.21.15.165:0/3891643889 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4600003520 con 0x7f46000039e0 2024-04-25T10:15:18.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.410+0000 7f461be00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46000039e0 0x7f4600003e00 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c210 tx=0x7f4600005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa78fa21aefa2e3c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.410+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c007d10 con 0x7f46000039e0 2024-04-25T10:15:18.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.410+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460c005ad0 con 0x7f46000039e0 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.410+0000 7f461aa00640 1 -- 172.21.15.165:0/3891643889 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460c007070 con 0x7f46000039e0 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621c00640 1 -- 172.21.15.165:0/3891643889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46000039e0 msgr2=0x7f4600003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621c00640 1 --2- 172.21.15.165:0/3891643889 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46000039e0 0x7f4600003e00 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f460c00c210 tx=0x7f4600005830 comp rx=0 tx=0).stop 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621200640 1 -- 172.21.15.165:0/3891643889 reap_dead start 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621c00640 1 -- 172.21.15.165:0/3891643889 shutdown_connections 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621c00640 1 -- 172.21.15.165:0/3891643889 >> 172.21.15.165:0/3891643889 conn(0x7f461c06abe0 msgr2=0x7f461c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621c00640 1 -- 172.21.15.165:0/3891643889 shutdown_connections 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.411+0000 7f4621c00640 1 -- 172.21.15.165:0/3891643889 wait complete. 2024-04-25T10:15:18.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.413+0000 7f4621c00640 1 Processor -- start 2024-04-25T10:15:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.413+0000 7f4621c00640 1 -- start start 2024-04-25T10:15:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000039e0 0x7f461c13b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c139400 0x7f461c139880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 0x7f461c138530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cdbb0 con 0x7f461c138080 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cda30 con 0x7f461c139400 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461c0cdd30 con 0x7f46000039e0 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.414+0000 7f4621200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 0x7f461c138530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 0x7f461c138530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57624/0 (socket says 172.21.15.165:57624) 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 -- 172.21.15.165:0/451411215 learned_addr learned my addr 172.21.15.165:0/451411215 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f461b400640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c139400 0x7f461c139880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f461be00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000039e0 0x7f461c13b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 -- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000039e0 msgr2=0x7f461c13b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000039e0 0x7f461c13b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 -- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c139400 msgr2=0x7f461c139880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c139400 0x7f461c139880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 -- 172.21.15.165:0/451411215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461c1f34e0 con 0x7f461c138080 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.415+0000 7f4621200640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 0x7f461c138530 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f461c0b2250 tx=0x7f461400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.416+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f461401c070 con 0x7f461c138080 2024-04-25T10:15:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.416+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614017400 con 0x7f461c138080 2024-04-25T10:15:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.416+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46140177f0 con 0x7f461c138080 2024-04-25T10:15:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.416+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461c13d240 con 0x7f461c138080 2024-04-25T10:15:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.416+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461c13d800 con 0x7f461c138080 2024-04-25T10:15:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.417+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461c073990 con 0x7f461c138080 2024-04-25T10:15:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.418+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4614005d10 con 0x7f461c138080 2024-04-25T10:15:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.419+0000 7f4613e00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f45fc06b500 0x7f45fc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.419+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f461408a510 con 0x7f461c138080 2024-04-25T10:15:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.420+0000 7f461be00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f45fc06b500 0x7f45fc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.420+0000 7f461be00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f45fc06b500 0x7f45fc06d9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f460c032000 tx=0x7f460c02d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.424+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4614012070 con 0x7f461c138080 2024-04-25T10:15:18.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.659+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f461c141340 con 0x7f461c138080 2024-04-25T10:15:18.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.660+0000 7f4613e00640 1 -- 172.21.15.165:0/451411215 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v90) v1 ==== 53+0+73 (secure 0 0 0) 0x7f46140591f0 con 0x7f461c138080 2024-04-25T10:15:18.663 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.662+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f45fc06b500 msgr2=0x7f45fc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.662+0000 7f4621c00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f45fc06b500 0x7f45fc06d9c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f460c032000 tx=0x7f460c02d000 comp rx=0 tx=0).stop 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 msgr2=0x7f461c138530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 0x7f461c138530 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f461c0b2250 tx=0x7f461400f4a0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 shutdown_connections 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f45fc06b500 0x7f45fc06d9c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f461c138080 0x7f461c138530 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f461c139400 0x7f461c139880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 --2- 172.21.15.165:0/451411215 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46000039e0 0x7f461c13b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:18.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 >> 172.21.15.165:0/451411215 conn(0x7f461c06abe0 msgr2=0x7f461c13f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:18.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 shutdown_connections 2024-04-25T10:15:18.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:18.663+0000 7f4621c00640 1 -- 172.21.15.165:0/451411215 wait complete. 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:15:18.686 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:18.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.687 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:15:18.737 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:15:18.737 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:15:18.737 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:15:18.737 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:15:18.738 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41847sec preferred_lft 41847sec 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:15:18.739 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.740 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:15:18.740 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:15:18.740 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:15:18.740 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:15:18.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.741 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:15:18.820 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:15:18.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.824 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:15:18.876 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:15:18.876 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:15:18.877 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:15:18.877 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:15:18.877 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:15:18.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:18.877 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:15:18.877 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:18.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:15:18.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:15:18.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:15:18.878 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:19.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.029 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:15:19.079 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:15:19.079 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:15:19.080 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:19.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.080 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:15:19.157 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:19.157 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:19.209 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:15:19.209 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:15:19.210 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:19.257 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:15:19.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.257 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:19.430 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:15:19.451 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:15:19.477 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:15:19.478 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:15:19.479 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:15:19.480 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:15:19.484 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:15:19.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.485 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:15:19.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:19.560 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:15:19.635 INFO:teuthology.orchestra.run.smithi165.stdout:client9375 2024-04-25T10:15:19.637 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:15:19.637 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:15:19.693 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:19.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.879+0000 7fed96c00640 1 Processor -- start 2024-04-25T10:15:19.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.879+0000 7fed96c00640 1 -- start start 2024-04-25T10:15:19.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed96c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fed90143e60 0x7fed9006e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed96c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fed9006e6a0 0x7fed900cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed96c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9015bb50 con 0x7fed900cc840 2024-04-25T10:15:19.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed96c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed90144240 con 0x7fed9006e6a0 2024-04-25T10:15:19.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed96c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9015e220 con 0x7fed90143e60 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fed9006e6a0 0x7fed900cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fed9006e6a0 0x7fed900cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57632/0 (socket says 172.21.15.165:57632) 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 -- 172.21.15.165:0/732956117 learned_addr learned my addr 172.21.15.165:0/732956117 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed95800640 1 --2- 172.21.15.165:0/732956117 >> v2:172.21.15.172:3300/0 conn(0x7fed90143e60 0x7fed9006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 -- 172.21.15.165:0/732956117 >> v2:172.21.15.172:3300/0 conn(0x7fed90143e60 msgr2=0x7fed9006e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 --2- 172.21.15.165:0/732956117 >> v2:172.21.15.172:3300/0 conn(0x7fed90143e60 0x7fed9006e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 -- 172.21.15.165:0/732956117 >> v1:172.21.15.172:6789/0 conn(0x7fed900cc840 legacy=0x7fed90070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:19.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.881+0000 7fed94e00640 1 -- 172.21.15.165:0/732956117 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9015e500 con 0x7fed9006e6a0 2024-04-25T10:15:19.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.882+0000 7fed94e00640 1 --2- 172.21.15.165:0/732956117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900cc300 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fed900b2770 tx=0x7fed7c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0073d6df6897b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.883+0000 7fed8fe00640 1 -- 172.21.15.165:0/732956117 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c037070 con 0x7fed9006e6a0 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.883+0000 7fed8fe00640 1 -- 172.21.15.165:0/732956117 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c032440 con 0x7fed9006e6a0 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.883+0000 7fed8fe00640 1 -- 172.21.15.165:0/732956117 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c032810 con 0x7fed9006e6a0 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.883+0000 7fed96c00640 1 -- 172.21.15.165:0/732956117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 msgr2=0x7fed900cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.883+0000 7fed96c00640 1 --2- 172.21.15.165:0/732956117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900cc300 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fed900b2770 tx=0x7fed7c02a230 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.884+0000 7fed96c00640 1 -- 172.21.15.165:0/732956117 shutdown_connections 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.884+0000 7fed96c00640 1 --2- 172.21.15.165:0/732956117 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900cc300 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.884+0000 7fed96c00640 1 --2- 172.21.15.165:0/732956117 >> v2:172.21.15.172:3300/0 conn(0x7fed90143e60 0x7fed9006e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.884+0000 7fed96c00640 1 -- 172.21.15.165:0/732956117 >> 172.21.15.165:0/732956117 conn(0x7fed9006abe0 msgr2=0x7fed9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.884+0000 7fed96c00640 1 -- 172.21.15.165:0/732956117 shutdown_connections 2024-04-25T10:15:19.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.884+0000 7fed96c00640 1 -- 172.21.15.165:0/732956117 wait complete. 2024-04-25T10:15:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.885+0000 7fed96c00640 1 Processor -- start 2024-04-25T10:15:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.886+0000 7fed96c00640 1 -- start start 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.886+0000 7fed96c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed96c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed900cc840 0x7fed90140de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed96c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed90143e60 0x7fed90141320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed96c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed900cccd0 con 0x7fed9006e6a0 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed96c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9015eac0 con 0x7fed900cc840 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed96c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed900cce50 con 0x7fed90143e60 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900b0140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57646/0 (socket says 172.21.15.165:57646) 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 -- 172.21.15.165:0/252836746 learned_addr learned my addr 172.21.15.165:0/252836746 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 -- 172.21.15.165:0/252836746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed90143e60 msgr2=0x7fed90141320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed90143e60 0x7fed90141320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 -- 172.21.15.165:0/252836746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed900cc840 msgr2=0x7fed90140de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed900cc840 0x7fed90140de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.887+0000 7fed95800640 1 -- 172.21.15.165:0/252836746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed900b0ab0 con 0x7fed9006e6a0 2024-04-25T10:15:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.888+0000 7fed95800640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900b0140 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fed8400d620 tx=0x7fed8400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.888+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8400b620 con 0x7fed9006e6a0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.888+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed84019070 con 0x7fed9006e6a0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.888+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8400ba00 con 0x7fed9006e6a0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.888+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed900b0d90 con 0x7fed9006e6a0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.888+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed900b1350 con 0x7fed9006e6a0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.889+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed90073990 con 0x7fed9006e6a0 2024-04-25T10:15:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.890+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed8400bba0 con 0x7fed9006e6a0 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.891+0000 7fed8d600640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed7006b500 0x7fed7006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.891+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed8408a830 con 0x7fed9006e6a0 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.891+0000 7fed94e00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed7006b500 0x7fed7006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.892+0000 7fed94e00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed7006b500 0x7fed7006d9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fed7c034000 tx=0x7fed7c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:19.896+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed84059480 con 0x7fed9006e6a0 2024-04-25T10:15:20.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.131+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed90071c10 con 0x7fed9006e6a0 2024-04-25T10:15:20.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.132+0000 7fed8d600640 1 -- 172.21.15.165:0/252836746 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fed84058d80 con 0x7fed9006e6a0 2024-04-25T10:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:20.136 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:20.136 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed7006b500 msgr2=0x7fed7006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed7006b500 0x7fed7006d9c0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fed7c034000 tx=0x7fed7c02c000 comp rx=0 tx=0).stop 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 msgr2=0x7fed900b0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900b0140 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fed8400d620 tx=0x7fed8400dae0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 shutdown_connections 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed7006b500 0x7fed7006d9c0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed90143e60 0x7fed90141320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed900cc840 0x7fed90140de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 --2- 172.21.15.165:0/252836746 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed9006e6a0 0x7fed900b0140 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 >> 172.21.15.165:0/252836746 conn(0x7fed9006abe0 msgr2=0x7fed9015bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 shutdown_connections 2024-04-25T10:15:20.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.135+0000 7fed96c00640 1 -- 172.21.15.165:0/252836746 wait complete. 2024-04-25T10:15:20.158 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:20.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.359+0000 7f80b2400640 1 Processor -- start 2024-04-25T10:15:20.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.359+0000 7f80b2400640 1 -- start start 2024-04-25T10:15:20.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.360+0000 7f80b2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80ac071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.360+0000 7f80b2400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac15ed90 con 0x7f80ac15e900 2024-04-25T10:15:20.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.360+0000 7f80b2400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80ac0ae590 con 0x7f80ac15e520 2024-04-25T10:15:20.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.360+0000 7f80b2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac0aee30 con 0x7f80ac070ff0 2024-04-25T10:15:20.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.361+0000 7f80abe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f80ac15e900 0x7f80ac0adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54952/0 (socket says 172.21.15.165:54952) 2024-04-25T10:15:20.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.361+0000 7f80abe00640 1 -- 172.21.15.165:0/723154782 learned_addr learned my addr 172.21.15.165:0/723154782 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:20.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.361+0000 7f80b1a00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80ac071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.361+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2140985443 0 0) 0x7f80ac15ed90 con 0x7f80ac15e900 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.361+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8090003520 con 0x7f80ac15e900 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3393739792 0 0) 0x7f8090003520 con 0x7f80ac15e900 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 msgr2=0x7f80ac071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80ac071440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 >> v1:172.21.15.172:6789/0 conn(0x7f80ac15e520 legacy=0x7f80ac15f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0ae870 con 0x7f80ac15e900 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8098001690 con 0x7f80ac15e900 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 >> v1:172.21.15.165:6789/0 conn(0x7f80ac15e900 legacy=0x7f80ac0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80900039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090003f60 0x7f8090004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8090004d40 0x7f80900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac0aee30 con 0x7f8090004d40 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f809000baa0 con 0x7f8090003f60 2024-04-25T10:15:20.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f809000bca0 con 0x7f80ac070ff0 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80b1a00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8090004d40 0x7f80900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.362+0000 7f80abe00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090003f60 0x7f8090004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1a00640 1 -- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 msgr2=0x7f80900039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1000640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80900039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1a00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80900039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1a00640 1 -- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090003f60 msgr2=0x7f8090004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1a00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090003f60 0x7f8090004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1a00640 1 -- 172.21.15.165:0/723154782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8090003520 con 0x7f8090004d40 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80b1000640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac070ff0 0x7f80900039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:20.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.363+0000 7f80abe00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090003f60 0x7f8090004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:20.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.364+0000 7f80b1a00640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8090004d40 0x7f80900051c0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009fc0 tx=0x7f8090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392b47e5ac4e386d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.364+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4009c10 con 0x7f8090004d40 2024-04-25T10:15:20.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.364+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4036070 con 0x7f8090004d40 2024-04-25T10:15:20.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.364+0000 7f80ab400640 1 -- 172.21.15.165:0/723154782 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4031860 con 0x7f8090004d40 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b2400640 1 -- 172.21.15.165:0/723154782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8090004d40 msgr2=0x7f80900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b2400640 1 --2- 172.21.15.165:0/723154782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8090004d40 0x7f80900051c0 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f80a4009fc0 tx=0x7f8090005830 comp rx=0 tx=0).stop 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b1a00640 1 -- 172.21.15.165:0/723154782 reap_dead start 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b2400640 1 -- 172.21.15.165:0/723154782 shutdown_connections 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b2400640 1 -- 172.21.15.165:0/723154782 >> 172.21.15.165:0/723154782 conn(0x7f80ac06abe0 msgr2=0x7f80ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b2400640 1 -- 172.21.15.165:0/723154782 shutdown_connections 2024-04-25T10:15:20.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.365+0000 7f80b2400640 1 -- 172.21.15.165:0/723154782 wait complete. 2024-04-25T10:15:20.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.366+0000 7f80b2400640 1 Processor -- start 2024-04-25T10:15:20.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.367+0000 7f80b2400640 1 -- start start 2024-04-25T10:15:20.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.367+0000 7f80b2400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090004d40 0x7f80ac151c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b2400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac0cd540 0x7f80ac0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b2400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 0x7f80ac146e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b2400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac0ae2a0 con 0x7f80ac1469f0 2024-04-25T10:15:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b2400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac06d1f0 con 0x7f8090004d40 2024-04-25T10:15:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b2400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80ac071440 con 0x7f80ac0cd540 2024-04-25T10:15:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 0x7f80ac146e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 0x7f80ac146e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:57674/0 (socket says 172.21.15.165:57674) 2024-04-25T10:15:20.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 -- 172.21.15.165:0/1529897893 learned_addr learned my addr 172.21.15.165:0/1529897893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1000640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090004d40 0x7f80ac151c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80abe00640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac0cd540 0x7f80ac0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 -- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac0cd540 msgr2=0x7f80ac0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac0cd540 0x7f80ac0cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 -- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090004d40 msgr2=0x7f80ac151c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090004d40 0x7f80ac151c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.368+0000 7f80b1a00640 1 -- 172.21.15.165:0/1529897893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80ac0ce090 con 0x7f80ac1469f0 2024-04-25T10:15:20.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.369+0000 7f80b1a00640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 0x7f80ac146e70 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f80a400e230 tx=0x7f80a4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.369+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4009c10 con 0x7f80ac1469f0 2024-04-25T10:15:20.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.369+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a4031bc0 con 0x7f80ac1469f0 2024-04-25T10:15:20.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.369+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a4045430 con 0x7f80ac1469f0 2024-04-25T10:15:20.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.369+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80ac0ce370 con 0x7f80ac1469f0 2024-04-25T10:15:20.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.369+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80ac159270 con 0x7f80ac1469f0 2024-04-25T10:15:20.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.370+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80ac159470 con 0x7f80ac1469f0 2024-04-25T10:15:20.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.371+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f80a4005d10 con 0x7f80ac1469f0 2024-04-25T10:15:20.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.372+0000 7f80a8c00640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f808c06b4d0 0x7f808c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:20.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.372+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f80a40aaea0 con 0x7f80ac1469f0 2024-04-25T10:15:20.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.372+0000 7f80b1000640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f808c06b4d0 0x7f808c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:20.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.373+0000 7f80b1000640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f808c06b4d0 0x7f808c06d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f809c001630 tx=0x7f809c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:20.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.376+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f80ac159470 con 0x7f80ac1469f0 2024-04-25T10:15:20.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.610+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80ac073990 con 0x7f80ac1469f0 2024-04-25T10:15:20.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.611+0000 7f80a8c00640 1 -- 172.21.15.165:0/1529897893 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f80a4079af0 con 0x7f80ac1469f0 2024-04-25T10:15:20.616 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:20.617 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:20.617 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:20.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f808c06b4d0 msgr2=0x7f808c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f808c06b4d0 0x7f808c06d990 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f809c001630 tx=0x7f809c009040 comp rx=0 tx=0).stop 2024-04-25T10:15:20.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 msgr2=0x7f80ac146e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:20.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 0x7f80ac146e70 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f80a400e230 tx=0x7f80a4009750 comp rx=0 tx=0).stop 2024-04-25T10:15:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 shutdown_connections 2024-04-25T10:15:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f808c06b4d0 0x7f808c06d990 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f80ac1469f0 0x7f80ac146e70 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f80ac0cd540 0x7f80ac0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 --2- 172.21.15.165:0/1529897893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8090004d40 0x7f80ac151c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.614+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 >> 172.21.15.165:0/1529897893 conn(0x7f80ac06abe0 msgr2=0x7f80ac0b06c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.615+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 shutdown_connections 2024-04-25T10:15:20.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:20.615+0000 7f80b2400640 1 -- 172.21.15.165:0/1529897893 wait complete. 2024-04-25T10:15:20.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:20.638 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:20.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0405915 s, 25.8 MB/s 2024-04-25T10:15:20.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.784 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0308193 s, 34.0 MB/s 2024-04-25T10:15:20.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.830 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0422466 s, 24.8 MB/s 2024-04-25T10:15:20.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.873 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.873 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0381626 s, 27.5 MB/s 2024-04-25T10:15:20.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.903 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260661 s, 40.2 MB/s 2024-04-25T10:15:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.938 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0317206 s, 33.1 MB/s 2024-04-25T10:15:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249481 s, 42.0 MB/s 2024-04-25T10:15:20.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:20.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:20.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181251 s, 57.9 MB/s 2024-04-25T10:15:21.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195317 s, 53.7 MB/s 2024-04-25T10:15:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.041 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245126 s, 42.8 MB/s 2024-04-25T10:15:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203328 s, 51.6 MB/s 2024-04-25T10:15:21.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.094 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237589 s, 44.1 MB/s 2024-04-25T10:15:21.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218702 s, 47.9 MB/s 2024-04-25T10:15:21.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.144 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195032 s, 53.8 MB/s 2024-04-25T10:15:21.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.169 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222844 s, 47.1 MB/s 2024-04-25T10:15:21.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214995 s, 48.8 MB/s 2024-04-25T10:15:21.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.222 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233037 s, 45.0 MB/s 2024-04-25T10:15:21.249 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.249 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.249 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239978 s, 43.7 MB/s 2024-04-25T10:15:21.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.277 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.277 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241749 s, 43.4 MB/s 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.320 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.039187 s, 26.8 MB/s 2024-04-25T10:15:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.344 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.345 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194982 s, 53.8 MB/s 2024-04-25T10:15:21.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.372 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236209 s, 44.4 MB/s 2024-04-25T10:15:21.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.399 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213502 s, 49.1 MB/s 2024-04-25T10:15:21.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.424 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219651 s, 47.7 MB/s 2024-04-25T10:15:21.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321834 s, 32.6 MB/s 2024-04-25T10:15:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196391 s, 53.4 MB/s 2024-04-25T10:15:21.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230411 s, 45.5 MB/s 2024-04-25T10:15:21.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.544 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301843 s, 34.7 MB/s 2024-04-25T10:15:21.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232459 s, 45.1 MB/s 2024-04-25T10:15:21.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217214 s, 48.3 MB/s 2024-04-25T10:15:21.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.622 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213779 s, 49.0 MB/s 2024-04-25T10:15:21.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.649 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023358 s, 44.9 MB/s 2024-04-25T10:15:21.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:21.651 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:21.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313539 s, 33.4 MB/s 2024-04-25T10:15:21.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.779 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284534 s, 36.9 MB/s 2024-04-25T10:15:21.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.801 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018982 s, 55.2 MB/s 2024-04-25T10:15:21.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174154 s, 60.2 MB/s 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.852 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237184 s, 44.2 MB/s 2024-04-25T10:15:21.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.875 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01944 s, 53.9 MB/s 2024-04-25T10:15:21.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.906 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025945 s, 40.4 MB/s 2024-04-25T10:15:21.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248291 s, 42.2 MB/s 2024-04-25T10:15:21.962 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.962 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.962 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239158 s, 43.8 MB/s 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239367 s, 43.8 MB/s 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021127 s, 49.6 MB/s 2024-04-25T10:15:22.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231882 s, 45.2 MB/s 2024-04-25T10:15:22.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.073 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260354 s, 40.3 MB/s 2024-04-25T10:15:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235149 s, 44.6 MB/s 2024-04-25T10:15:22.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.127 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223883 s, 46.8 MB/s 2024-04-25T10:15:22.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.248 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.249 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214068 s, 49.0 MB/s 2024-04-25T10:15:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227407 s, 46.1 MB/s 2024-04-25T10:15:22.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219944 s, 47.7 MB/s 2024-04-25T10:15:22.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233747 s, 44.9 MB/s 2024-04-25T10:15:22.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.258 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214868 s, 48.8 MB/s 2024-04-25T10:15:22.281 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.281 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.281 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018632 s, 56.3 MB/s 2024-04-25T10:15:22.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213665 s, 49.1 MB/s 2024-04-25T10:15:22.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.329 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192756 s, 54.4 MB/s 2024-04-25T10:15:22.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.358 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244407 s, 42.9 MB/s 2024-04-25T10:15:22.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210578 s, 49.8 MB/s 2024-04-25T10:15:22.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227793 s, 46.0 MB/s 2024-04-25T10:15:22.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237055 s, 44.2 MB/s 2024-04-25T10:15:22.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.464 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02414 s, 43.4 MB/s 2024-04-25T10:15:22.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.491 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220172 s, 47.6 MB/s 2024-04-25T10:15:22.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.520 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.520 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248976 s, 42.1 MB/s 2024-04-25T10:15:22.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.548 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240241 s, 43.6 MB/s 2024-04-25T10:15:22.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.575 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233033 s, 45.0 MB/s 2024-04-25T10:15:22.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:22.578 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.667 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238239 s, 44.0 MB/s 2024-04-25T10:15:22.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.693 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223112 s, 47.0 MB/s 2024-04-25T10:15:22.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240767 s, 43.6 MB/s 2024-04-25T10:15:22.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236929 s, 44.3 MB/s 2024-04-25T10:15:22.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235849 s, 44.5 MB/s 2024-04-25T10:15:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188518 s, 55.6 MB/s 2024-04-25T10:15:22.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233132 s, 45.0 MB/s 2024-04-25T10:15:22.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211871 s, 49.5 MB/s 2024-04-25T10:15:22.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238617 s, 43.9 MB/s 2024-04-25T10:15:22.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.906 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226004 s, 46.4 MB/s 2024-04-25T10:15:22.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195039 s, 53.8 MB/s 2024-04-25T10:15:22.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.955 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217247 s, 48.3 MB/s 2024-04-25T10:15:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:22.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:22.982 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224563 s, 46.7 MB/s 2024-04-25T10:15:23.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.006 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195561 s, 53.6 MB/s 2024-04-25T10:15:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.034 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023761 s, 44.1 MB/s 2024-04-25T10:15:23.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224564 s, 46.7 MB/s 2024-04-25T10:15:23.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.083 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194356 s, 54.0 MB/s 2024-04-25T10:15:23.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201519 s, 52.0 MB/s 2024-04-25T10:15:23.132 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.132 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.132 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212342 s, 49.4 MB/s 2024-04-25T10:15:23.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203529 s, 51.5 MB/s 2024-04-25T10:15:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270318 s, 38.8 MB/s 2024-04-25T10:15:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.213 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211009 s, 49.7 MB/s 2024-04-25T10:15:23.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.239 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224136 s, 46.8 MB/s 2024-04-25T10:15:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.266 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223732 s, 46.9 MB/s 2024-04-25T10:15:23.292 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.292 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.292 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218151 s, 48.1 MB/s 2024-04-25T10:15:23.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.319 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021889 s, 47.9 MB/s 2024-04-25T10:15:23.343 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.343 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.343 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204776 s, 51.2 MB/s 2024-04-25T10:15:23.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.368 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.368 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205099 s, 51.1 MB/s 2024-04-25T10:15:23.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.392 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203252 s, 51.6 MB/s 2024-04-25T10:15:23.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.423 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270461 s, 38.8 MB/s 2024-04-25T10:15:23.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.448 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216401 s, 48.5 MB/s 2024-04-25T10:15:23.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220159 s, 47.6 MB/s 2024-04-25T10:15:23.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:23.478 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:23.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.563 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192198 s, 54.6 MB/s 2024-04-25T10:15:23.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.590 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245958 s, 42.6 MB/s 2024-04-25T10:15:23.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.619 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248981 s, 42.1 MB/s 2024-04-25T10:15:23.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.642 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196216 s, 53.4 MB/s 2024-04-25T10:15:23.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.672 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259998 s, 40.3 MB/s 2024-04-25T10:15:23.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.694 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187072 s, 56.1 MB/s 2024-04-25T10:15:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027498 s, 38.1 MB/s 2024-04-25T10:15:23.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242269 s, 43.3 MB/s 2024-04-25T10:15:23.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.781 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022805 s, 46.0 MB/s 2024-04-25T10:15:23.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229805 s, 45.6 MB/s 2024-04-25T10:15:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217842 s, 48.1 MB/s 2024-04-25T10:15:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022829 s, 45.9 MB/s 2024-04-25T10:15:23.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238066 s, 44.0 MB/s 2024-04-25T10:15:23.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.915 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229218 s, 45.7 MB/s 2024-04-25T10:15:23.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.941 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215214 s, 48.7 MB/s 2024-04-25T10:15:23.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.965 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205949 s, 50.9 MB/s 2024-04-25T10:15:23.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:23.992 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:23.992 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233098 s, 45.0 MB/s 2024-04-25T10:15:24.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.020 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234692 s, 44.7 MB/s 2024-04-25T10:15:24.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.049 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239723 s, 43.7 MB/s 2024-04-25T10:15:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.073 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021099 s, 49.7 MB/s 2024-04-25T10:15:24.097 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.097 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.097 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211092 s, 49.7 MB/s 2024-04-25T10:15:24.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208409 s, 50.3 MB/s 2024-04-25T10:15:24.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0169513 s, 61.9 MB/s 2024-04-25T10:15:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.168 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02045 s, 51.3 MB/s 2024-04-25T10:15:24.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221712 s, 47.3 MB/s 2024-04-25T10:15:24.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.221 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.221 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02277 s, 46.1 MB/s 2024-04-25T10:15:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204977 s, 51.2 MB/s 2024-04-25T10:15:24.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.271 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217769 s, 48.2 MB/s 2024-04-25T10:15:24.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.301 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256729 s, 40.8 MB/s 2024-04-25T10:15:24.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.329 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228108 s, 46.0 MB/s 2024-04-25T10:15:24.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.357 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234289 s, 44.8 MB/s 2024-04-25T10:15:24.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213414 s, 49.1 MB/s 2024-04-25T10:15:24.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:24.385 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:24.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.471 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.471 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243262 s, 43.1 MB/s 2024-04-25T10:15:24.496 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.496 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.496 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210432 s, 49.8 MB/s 2024-04-25T10:15:24.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172336 s, 60.8 MB/s 2024-04-25T10:15:24.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240783 s, 43.5 MB/s 2024-04-25T10:15:24.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225629 s, 46.5 MB/s 2024-04-25T10:15:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231822 s, 45.2 MB/s 2024-04-25T10:15:24.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193752 s, 54.1 MB/s 2024-04-25T10:15:24.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252057 s, 41.6 MB/s 2024-04-25T10:15:24.676 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222746 s, 47.1 MB/s 2024-04-25T10:15:24.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185982 s, 56.4 MB/s 2024-04-25T10:15:24.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.720 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174567 s, 60.1 MB/s 2024-04-25T10:15:24.743 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.743 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.743 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194198 s, 54.0 MB/s 2024-04-25T10:15:24.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.773 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.773 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256033 s, 41.0 MB/s 2024-04-25T10:15:24.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229616 s, 45.7 MB/s 2024-04-25T10:15:24.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.827 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239302 s, 43.8 MB/s 2024-04-25T10:15:24.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235443 s, 44.5 MB/s 2024-04-25T10:15:24.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.886 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276456 s, 37.9 MB/s 2024-04-25T10:15:24.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.915 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241723 s, 43.4 MB/s 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.943 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245532 s, 42.7 MB/s 2024-04-25T10:15:24.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.973 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026302 s, 39.9 MB/s 2024-04-25T10:15:24.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:24.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:24.998 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212677 s, 49.3 MB/s 2024-04-25T10:15:25.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.020 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017872 s, 58.7 MB/s 2024-04-25T10:15:25.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.048 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224429 s, 46.7 MB/s 2024-04-25T10:15:25.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.073 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021165 s, 49.5 MB/s 2024-04-25T10:15:25.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229492 s, 45.7 MB/s 2024-04-25T10:15:25.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.124 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194414 s, 53.9 MB/s 2024-04-25T10:15:25.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227871 s, 46.0 MB/s 2024-04-25T10:15:25.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.178 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023545 s, 44.5 MB/s 2024-04-25T10:15:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222854 s, 47.1 MB/s 2024-04-25T10:15:25.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235403 s, 44.5 MB/s 2024-04-25T10:15:25.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.259 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024089 s, 43.5 MB/s 2024-04-25T10:15:25.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.285 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220675 s, 47.5 MB/s 2024-04-25T10:15:25.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:25.288 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:25.288 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:25.288 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:15:25.288 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:25.289 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:25.289 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:25.289 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:25.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281118 s, 37.3 MB/s 2024-04-25T10:15:25.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.410 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235603 s, 44.5 MB/s 2024-04-25T10:15:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021641 s, 48.5 MB/s 2024-04-25T10:15:25.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206981 s, 50.7 MB/s 2024-04-25T10:15:25.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.490 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262906 s, 39.9 MB/s 2024-04-25T10:15:25.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.516 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209803 s, 50.0 MB/s 2024-04-25T10:15:25.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202177 s, 51.9 MB/s 2024-04-25T10:15:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187227 s, 56.0 MB/s 2024-04-25T10:15:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.586 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211381 s, 49.6 MB/s 2024-04-25T10:15:25.611 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.611 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.611 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021547 s, 48.7 MB/s 2024-04-25T10:15:25.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.635 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198245 s, 52.9 MB/s 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.661 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022772 s, 46.0 MB/s 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.684 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195786 s, 53.6 MB/s 2024-04-25T10:15:25.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228156 s, 46.0 MB/s 2024-04-25T10:15:25.735 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.735 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.735 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203707 s, 51.5 MB/s 2024-04-25T10:15:25.762 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.763 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233874 s, 44.8 MB/s 2024-04-25T10:15:25.791 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.791 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.791 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246631 s, 42.5 MB/s 2024-04-25T10:15:25.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213522 s, 49.1 MB/s 2024-04-25T10:15:25.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.843 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231318 s, 45.3 MB/s 2024-04-25T10:15:25.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204878 s, 51.2 MB/s 2024-04-25T10:15:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.893 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207456 s, 50.5 MB/s 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.918 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211575 s, 49.6 MB/s 2024-04-25T10:15:25.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.947 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253164 s, 41.4 MB/s 2024-04-25T10:15:25.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.972 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209556 s, 50.0 MB/s 2024-04-25T10:15:25.999 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:25.999 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:25.999 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231071 s, 45.4 MB/s 2024-04-25T10:15:26.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205796 s, 51.0 MB/s 2024-04-25T10:15:26.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.052 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245106 s, 42.8 MB/s 2024-04-25T10:15:26.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178828 s, 58.6 MB/s 2024-04-25T10:15:26.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212033 s, 49.5 MB/s 2024-04-25T10:15:26.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.123 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019966 s, 52.5 MB/s 2024-04-25T10:15:26.147 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200377 s, 52.3 MB/s 2024-04-25T10:15:26.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245924 s, 42.6 MB/s 2024-04-25T10:15:26.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:26.178 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:26.178 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:26.178 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:15:26.179 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:26.179 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:26.179 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:26.179 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:26.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023715 s, 44.2 MB/s 2024-04-25T10:15:26.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.295 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249978 s, 41.9 MB/s 2024-04-25T10:15:26.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.324 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244255 s, 42.9 MB/s 2024-04-25T10:15:26.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.349 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211183 s, 49.7 MB/s 2024-04-25T10:15:26.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219832 s, 47.7 MB/s 2024-04-25T10:15:26.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247176 s, 42.4 MB/s 2024-04-25T10:15:26.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.430 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213307 s, 49.2 MB/s 2024-04-25T10:15:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223888 s, 46.8 MB/s 2024-04-25T10:15:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.488 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277576 s, 37.8 MB/s 2024-04-25T10:15:26.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.513 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210328 s, 49.9 MB/s 2024-04-25T10:15:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222279 s, 47.2 MB/s 2024-04-25T10:15:26.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240389 s, 43.6 MB/s 2024-04-25T10:15:26.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255829 s, 41.0 MB/s 2024-04-25T10:15:26.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.622 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205369 s, 51.1 MB/s 2024-04-25T10:15:26.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262397 s, 40.0 MB/s 2024-04-25T10:15:26.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237654 s, 44.1 MB/s 2024-04-25T10:15:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223295 s, 47.0 MB/s 2024-04-25T10:15:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019657 s, 53.3 MB/s 2024-04-25T10:15:26.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235944 s, 44.4 MB/s 2024-04-25T10:15:26.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.786 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245262 s, 42.8 MB/s 2024-04-25T10:15:26.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.815 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254672 s, 41.2 MB/s 2024-04-25T10:15:26.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.840 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203647 s, 51.5 MB/s 2024-04-25T10:15:26.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.870 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253168 s, 41.4 MB/s 2024-04-25T10:15:26.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.894 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198425 s, 52.8 MB/s 2024-04-25T10:15:26.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.921 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228743 s, 45.8 MB/s 2024-04-25T10:15:26.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238199 s, 44.0 MB/s 2024-04-25T10:15:26.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:26.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:26.979 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257389 s, 40.7 MB/s 2024-04-25T10:15:27.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.006 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230704 s, 45.5 MB/s 2024-04-25T10:15:27.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.036 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.036 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261892 s, 40.0 MB/s 2024-04-25T10:15:27.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.063 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227735 s, 46.0 MB/s 2024-04-25T10:15:27.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214166 s, 49.0 MB/s 2024-04-25T10:15:27.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.113 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210867 s, 49.7 MB/s 2024-04-25T10:15:27.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:27.116 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:27.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.198 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207967 s, 50.4 MB/s 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.226 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236076 s, 44.4 MB/s 2024-04-25T10:15:27.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251662 s, 41.7 MB/s 2024-04-25T10:15:27.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224209 s, 46.8 MB/s 2024-04-25T10:15:27.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021707 s, 48.3 MB/s 2024-04-25T10:15:27.335 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.335 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.335 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239897 s, 43.7 MB/s 2024-04-25T10:15:27.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208916 s, 50.2 MB/s 2024-04-25T10:15:27.386 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227961 s, 46.0 MB/s 2024-04-25T10:15:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.414 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023528 s, 44.6 MB/s 2024-04-25T10:15:27.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205479 s, 51.0 MB/s 2024-04-25T10:15:27.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.465 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022739 s, 46.1 MB/s 2024-04-25T10:15:27.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024314 s, 43.1 MB/s 2024-04-25T10:15:27.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234237 s, 44.8 MB/s 2024-04-25T10:15:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023887 s, 43.9 MB/s 2024-04-25T10:15:27.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233859 s, 44.8 MB/s 2024-04-25T10:15:27.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.601 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207767 s, 50.5 MB/s 2024-04-25T10:15:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020458 s, 51.3 MB/s 2024-04-25T10:15:27.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181505 s, 57.8 MB/s 2024-04-25T10:15:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267935 s, 39.1 MB/s 2024-04-25T10:15:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175929 s, 59.6 MB/s 2024-04-25T10:15:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238781 s, 43.9 MB/s 2024-04-25T10:15:27.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.753 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.753 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020675 s, 50.7 MB/s 2024-04-25T10:15:27.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.776 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017934 s, 58.5 MB/s 2024-04-25T10:15:27.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204558 s, 51.3 MB/s 2024-04-25T10:15:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.830 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259636 s, 40.4 MB/s 2024-04-25T10:15:27.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234791 s, 44.7 MB/s 2024-04-25T10:15:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226976 s, 46.2 MB/s 2024-04-25T10:15:27.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.910 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222657 s, 47.1 MB/s 2024-04-25T10:15:27.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.938 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02444 s, 42.9 MB/s 2024-04-25T10:15:27.963 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.963 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.963 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217651 s, 48.2 MB/s 2024-04-25T10:15:27.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:27.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:27.988 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218449 s, 48.0 MB/s 2024-04-25T10:15:28.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.019 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267675 s, 39.2 MB/s 2024-04-25T10:15:28.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:28.021 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:28.021 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:28.021 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:15:28.021 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:28.021 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:28.022 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:28.022 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:28.104 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.104 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203272 s, 51.6 MB/s 2024-04-25T10:15:28.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.131 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225657 s, 46.5 MB/s 2024-04-25T10:15:28.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.156 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218997 s, 47.9 MB/s 2024-04-25T10:15:28.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202756 s, 51.7 MB/s 2024-04-25T10:15:28.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217494 s, 48.2 MB/s 2024-04-25T10:15:28.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226138 s, 46.4 MB/s 2024-04-25T10:15:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193074 s, 54.3 MB/s 2024-04-25T10:15:28.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.282 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225583 s, 46.5 MB/s 2024-04-25T10:15:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023296 s, 45.0 MB/s 2024-04-25T10:15:28.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.339 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.339 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254237 s, 41.2 MB/s 2024-04-25T10:15:28.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.364 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210504 s, 49.8 MB/s 2024-04-25T10:15:28.390 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.390 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.390 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221575 s, 47.3 MB/s 2024-04-25T10:15:28.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.414 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202942 s, 51.7 MB/s 2024-04-25T10:15:28.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.438 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.438 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201034 s, 52.2 MB/s 2024-04-25T10:15:28.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.468 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248226 s, 42.2 MB/s 2024-04-25T10:15:28.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211926 s, 49.5 MB/s 2024-04-25T10:15:28.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227167 s, 46.2 MB/s 2024-04-25T10:15:28.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.547 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213686 s, 49.1 MB/s 2024-04-25T10:15:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.573 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226331 s, 46.3 MB/s 2024-04-25T10:15:28.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.601 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024777 s, 42.3 MB/s 2024-04-25T10:15:28.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209908 s, 50.0 MB/s 2024-04-25T10:15:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251794 s, 41.6 MB/s 2024-04-25T10:15:28.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.686 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252349 s, 41.6 MB/s 2024-04-25T10:15:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241094 s, 43.5 MB/s 2024-04-25T10:15:28.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226465 s, 46.3 MB/s 2024-04-25T10:15:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.766 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219069 s, 47.9 MB/s 2024-04-25T10:15:28.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215385 s, 48.7 MB/s 2024-04-25T10:15:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231071 s, 45.4 MB/s 2024-04-25T10:15:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.842 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188078 s, 55.8 MB/s 2024-04-25T10:15:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213225 s, 49.2 MB/s 2024-04-25T10:15:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.893 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220923 s, 47.5 MB/s 2024-04-25T10:15:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:28.928 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0301778 s, 34.7 MB/s 2024-04-25T10:15:28.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:28.930 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:28.930 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:28.930 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:15:28.930 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:28.930 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:28.931 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:28.931 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:29.018 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.019 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249887 s, 42.0 MB/s 2024-04-25T10:15:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220738 s, 47.5 MB/s 2024-04-25T10:15:29.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223129 s, 47.0 MB/s 2024-04-25T10:15:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229006 s, 45.8 MB/s 2024-04-25T10:15:29.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.124 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222202 s, 47.2 MB/s 2024-04-25T10:15:29.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211538 s, 49.6 MB/s 2024-04-25T10:15:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023989 s, 43.7 MB/s 2024-04-25T10:15:29.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.206 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248294 s, 42.2 MB/s 2024-04-25T10:15:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220267 s, 47.6 MB/s 2024-04-25T10:15:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.262 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244831 s, 42.8 MB/s 2024-04-25T10:15:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.288 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021947 s, 47.8 MB/s 2024-04-25T10:15:29.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232683 s, 45.1 MB/s 2024-04-25T10:15:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.342 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241431 s, 43.4 MB/s 2024-04-25T10:15:29.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.366 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194126 s, 54.0 MB/s 2024-04-25T10:15:29.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.393 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227414 s, 46.1 MB/s 2024-04-25T10:15:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209805 s, 50.0 MB/s 2024-04-25T10:15:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244097 s, 43.0 MB/s 2024-04-25T10:15:29.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216304 s, 48.5 MB/s 2024-04-25T10:15:29.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264091 s, 39.7 MB/s 2024-04-25T10:15:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.533 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243476 s, 43.1 MB/s 2024-04-25T10:15:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240498 s, 43.6 MB/s 2024-04-25T10:15:29.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.591 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.591 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025756 s, 40.7 MB/s 2024-04-25T10:15:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192018 s, 54.6 MB/s 2024-04-25T10:15:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198853 s, 52.7 MB/s 2024-04-25T10:15:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212469 s, 49.4 MB/s 2024-04-25T10:15:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200472 s, 52.3 MB/s 2024-04-25T10:15:29.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.710 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186106 s, 56.3 MB/s 2024-04-25T10:15:29.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.739 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.739 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251207 s, 41.7 MB/s 2024-04-25T10:15:29.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.768 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248234 s, 42.2 MB/s 2024-04-25T10:15:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022204 s, 47.2 MB/s 2024-04-25T10:15:29.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.819 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206058 s, 50.9 MB/s 2024-04-25T10:15:29.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.846 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225425 s, 46.5 MB/s 2024-04-25T10:15:29.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:29.848 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:29.848 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:29.848 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:15:29.848 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:29.848 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:29.848 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:29.849 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226071 s, 46.4 MB/s 2024-04-25T10:15:29.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.960 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218154 s, 48.1 MB/s 2024-04-25T10:15:29.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:29.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:29.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252327 s, 41.6 MB/s 2024-04-25T10:15:30.017 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.017 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.017 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246281 s, 42.6 MB/s 2024-04-25T10:15:30.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222802 s, 47.1 MB/s 2024-04-25T10:15:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231207 s, 45.4 MB/s 2024-04-25T10:15:30.093 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:30.100 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:15:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226672 s, 46.3 MB/s 2024-04-25T10:15:30.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212053 s, 49.4 MB/s 2024-04-25T10:15:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180248 s, 58.2 MB/s 2024-04-25T10:15:30.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.167 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212004 s, 49.5 MB/s 2024-04-25T10:15:30.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236251 s, 44.4 MB/s 2024-04-25T10:15:30.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189983 s, 55.2 MB/s 2024-04-25T10:15:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217327 s, 48.2 MB/s 2024-04-25T10:15:30.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.271 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236918 s, 44.3 MB/s 2024-04-25T10:15:30.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.297 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223896 s, 46.8 MB/s 2024-04-25T10:15:30.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.323 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225334 s, 46.5 MB/s 2024-04-25T10:15:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204285 s, 51.3 MB/s 2024-04-25T10:15:30.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.371 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200449 s, 52.3 MB/s 2024-04-25T10:15:30.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184436 s, 56.9 MB/s 2024-04-25T10:15:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187832 s, 55.8 MB/s 2024-04-25T10:15:30.441 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.441 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.441 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206616 s, 50.8 MB/s 2024-04-25T10:15:30.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.466 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.466 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207022 s, 50.7 MB/s 2024-04-25T10:15:30.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229085 s, 45.8 MB/s 2024-04-25T10:15:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026311 s, 39.9 MB/s 2024-04-25T10:15:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185593 s, 56.5 MB/s 2024-04-25T10:15:30.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.570 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201155 s, 52.1 MB/s 2024-04-25T10:15:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206469 s, 50.8 MB/s 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253854 s, 41.3 MB/s 2024-04-25T10:15:30.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.652 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241824 s, 43.4 MB/s 2024-04-25T10:15:30.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228075 s, 46.0 MB/s 2024-04-25T10:15:30.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.708 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252697 s, 41.5 MB/s 2024-04-25T10:15:30.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.738 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250425 s, 41.9 MB/s 2024-04-25T10:15:30.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:30.740 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:30.740 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:30.740 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:15:30.740 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:30.741 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:30.741 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:30.741 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:30.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209877 s, 50.0 MB/s 2024-04-25T10:15:30.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.894 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208056 s, 50.4 MB/s 2024-04-25T10:15:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238377 s, 44.0 MB/s 2024-04-25T10:15:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018006 s, 58.2 MB/s 2024-04-25T10:15:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251879 s, 41.6 MB/s 2024-04-25T10:15:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:30.995 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228258 s, 45.9 MB/s 2024-04-25T10:15:31.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238022 s, 44.1 MB/s 2024-04-25T10:15:31.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.048 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.048 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192642 s, 54.4 MB/s 2024-04-25T10:15:31.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.075 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.075 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227344 s, 46.1 MB/s 2024-04-25T10:15:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225009 s, 46.6 MB/s 2024-04-25T10:15:31.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.129 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235666 s, 44.5 MB/s 2024-04-25T10:15:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244351 s, 42.9 MB/s 2024-04-25T10:15:31.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215812 s, 48.6 MB/s 2024-04-25T10:15:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.210 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226157 s, 46.4 MB/s 2024-04-25T10:15:31.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.232 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.232 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187806 s, 55.8 MB/s 2024-04-25T10:15:31.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205378 s, 51.1 MB/s 2024-04-25T10:15:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223894 s, 46.8 MB/s 2024-04-25T10:15:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224256 s, 46.8 MB/s 2024-04-25T10:15:31.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279345 s, 37.5 MB/s 2024-04-25T10:15:31.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.369 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225318 s, 46.5 MB/s 2024-04-25T10:15:31.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.395 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222216 s, 47.2 MB/s 2024-04-25T10:15:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243542 s, 43.1 MB/s 2024-04-25T10:15:31.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.449 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216316 s, 48.5 MB/s 2024-04-25T10:15:31.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218755 s, 47.9 MB/s 2024-04-25T10:15:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232714 s, 45.1 MB/s 2024-04-25T10:15:31.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.528 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215898 s, 48.6 MB/s 2024-04-25T10:15:31.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.556 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233872 s, 44.8 MB/s 2024-04-25T10:15:31.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222346 s, 47.2 MB/s 2024-04-25T10:15:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.605 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196063 s, 53.5 MB/s 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187643 s, 55.9 MB/s 2024-04-25T10:15:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215505 s, 48.7 MB/s 2024-04-25T10:15:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195289 s, 53.7 MB/s 2024-04-25T10:15:31.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:31.679 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:31.679 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:31.679 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:15:31.680 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:31.680 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:31.680 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:31.680 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.767 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219398 s, 47.8 MB/s 2024-04-25T10:15:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229594 s, 45.7 MB/s 2024-04-25T10:15:31.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.821 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227489 s, 46.1 MB/s 2024-04-25T10:15:31.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.847 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222995 s, 47.0 MB/s 2024-04-25T10:15:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219763 s, 47.7 MB/s 2024-04-25T10:15:31.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198028 s, 53.0 MB/s 2024-04-25T10:15:31.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222869 s, 47.0 MB/s 2024-04-25T10:15:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226226 s, 46.4 MB/s 2024-04-25T10:15:31.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:31.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:31.978 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234388 s, 44.7 MB/s 2024-04-25T10:15:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241071 s, 43.5 MB/s 2024-04-25T10:15:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0281148 s, 37.3 MB/s 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249557 s, 42.0 MB/s 2024-04-25T10:15:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282618 s, 37.1 MB/s 2024-04-25T10:15:32.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.129 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.129 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230543 s, 45.5 MB/s 2024-04-25T10:15:32.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.158 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252005 s, 41.6 MB/s 2024-04-25T10:15:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02343 s, 44.8 MB/s 2024-04-25T10:15:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.218 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287721 s, 36.4 MB/s 2024-04-25T10:15:32.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.244 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021627 s, 48.5 MB/s 2024-04-25T10:15:32.270 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.271 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219639 s, 47.7 MB/s 2024-04-25T10:15:32.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246333 s, 42.6 MB/s 2024-04-25T10:15:32.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.323 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193296 s, 54.2 MB/s 2024-04-25T10:15:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274891 s, 38.1 MB/s 2024-04-25T10:15:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161879 s, 64.8 MB/s 2024-04-25T10:15:32.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.397 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.397 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.016703 s, 62.8 MB/s 2024-04-25T10:15:32.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.423 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221008 s, 47.4 MB/s 2024-04-25T10:15:32.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.448 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213532 s, 49.1 MB/s 2024-04-25T10:15:32.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.474 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225233 s, 46.6 MB/s 2024-04-25T10:15:32.500 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.500 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.500 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222647 s, 47.1 MB/s 2024-04-25T10:15:32.527 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.527 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.527 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225629 s, 46.5 MB/s 2024-04-25T10:15:32.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.556 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256803 s, 40.8 MB/s 2024-04-25T10:15:32.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.588 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274461 s, 38.2 MB/s 2024-04-25T10:15:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217649 s, 48.2 MB/s 2024-04-25T10:15:32.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:32.616 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023262 s, 45.1 MB/s 2024-04-25T10:15:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247957 s, 42.3 MB/s 2024-04-25T10:15:32.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.765 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232252 s, 45.1 MB/s 2024-04-25T10:15:32.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.784 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0166914 s, 62.8 MB/s 2024-04-25T10:15:32.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.811 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213007 s, 49.2 MB/s 2024-04-25T10:15:32.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.834 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197388 s, 53.1 MB/s 2024-04-25T10:15:32.859 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.859 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.859 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020865 s, 50.3 MB/s 2024-04-25T10:15:32.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241239 s, 43.5 MB/s 2024-04-25T10:15:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231007 s, 45.4 MB/s 2024-04-25T10:15:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.942 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239988 s, 43.7 MB/s 2024-04-25T10:15:32.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.969 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231572 s, 45.3 MB/s 2024-04-25T10:15:32.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:32.994 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:32.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206935 s, 50.7 MB/s 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250164 s, 41.9 MB/s 2024-04-25T10:15:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234462 s, 44.7 MB/s 2024-04-25T10:15:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247988 s, 42.3 MB/s 2024-04-25T10:15:33.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204737 s, 51.2 MB/s 2024-04-25T10:15:33.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.126 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193842 s, 54.1 MB/s 2024-04-25T10:15:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220038 s, 47.7 MB/s 2024-04-25T10:15:33.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.180 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232632 s, 45.1 MB/s 2024-04-25T10:15:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249736 s, 42.0 MB/s 2024-04-25T10:15:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.233 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207982 s, 50.4 MB/s 2024-04-25T10:15:33.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.263 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.263 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255207 s, 41.1 MB/s 2024-04-25T10:15:33.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.289 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216742 s, 48.4 MB/s 2024-04-25T10:15:33.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229628 s, 45.7 MB/s 2024-04-25T10:15:33.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223762 s, 46.9 MB/s 2024-04-25T10:15:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247003 s, 42.5 MB/s 2024-04-25T10:15:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229803 s, 45.6 MB/s 2024-04-25T10:15:33.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.424 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220081 s, 47.6 MB/s 2024-04-25T10:15:33.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217003 s, 48.3 MB/s 2024-04-25T10:15:33.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.477 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214273 s, 48.9 MB/s 2024-04-25T10:15:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.501 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.502 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206506 s, 50.8 MB/s 2024-04-25T10:15:33.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233515 s, 44.9 MB/s 2024-04-25T10:15:33.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:33.531 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:33.531 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:33.531 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:15:33.531 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:33.531 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:33.532 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:33.532 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:33.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262279 s, 40.0 MB/s 2024-04-25T10:15:33.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258204 s, 40.6 MB/s 2024-04-25T10:15:33.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216544 s, 48.4 MB/s 2024-04-25T10:15:33.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188815 s, 55.5 MB/s 2024-04-25T10:15:33.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243602 s, 43.0 MB/s 2024-04-25T10:15:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207011 s, 50.7 MB/s 2024-04-25T10:15:33.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.777 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209744 s, 50.0 MB/s 2024-04-25T10:15:33.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.801 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.801 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205863 s, 50.9 MB/s 2024-04-25T10:15:33.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.827 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.827 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215724 s, 48.6 MB/s 2024-04-25T10:15:33.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243449 s, 43.1 MB/s 2024-04-25T10:15:33.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188281 s, 55.7 MB/s 2024-04-25T10:15:33.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.900 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193591 s, 54.2 MB/s 2024-04-25T10:15:33.925 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.926 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212981 s, 49.2 MB/s 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.954 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023732 s, 44.2 MB/s 2024-04-25T10:15:33.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:33.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:33.980 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221506 s, 47.3 MB/s 2024-04-25T10:15:34.009 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.009 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.009 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237778 s, 44.1 MB/s 2024-04-25T10:15:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191396 s, 54.8 MB/s 2024-04-25T10:15:34.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.064 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268514 s, 39.1 MB/s 2024-04-25T10:15:34.090 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.091 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226211 s, 46.4 MB/s 2024-04-25T10:15:34.117 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.117 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.117 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021157 s, 49.6 MB/s 2024-04-25T10:15:34.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.144 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229892 s, 45.6 MB/s 2024-04-25T10:15:34.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.170 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.170 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179718 s, 58.3 MB/s 2024-04-25T10:15:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228963 s, 45.8 MB/s 2024-04-25T10:15:34.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.223 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.223 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244184 s, 42.9 MB/s 2024-04-25T10:15:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233436 s, 44.9 MB/s 2024-04-25T10:15:34.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234237 s, 44.8 MB/s 2024-04-25T10:15:34.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237833 s, 44.1 MB/s 2024-04-25T10:15:34.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.334 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226467 s, 46.3 MB/s 2024-04-25T10:15:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216673 s, 48.4 MB/s 2024-04-25T10:15:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233485 s, 44.9 MB/s 2024-04-25T10:15:34.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.414 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222804 s, 47.1 MB/s 2024-04-25T10:15:34.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.440 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223306 s, 47.0 MB/s 2024-04-25T10:15:34.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:34.442 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:34.442 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:34.442 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:15:34.442 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:34.443 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:34.443 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:34.443 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0303926 s, 34.5 MB/s 2024-04-25T10:15:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208793 s, 50.2 MB/s 2024-04-25T10:15:34.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232856 s, 45.0 MB/s 2024-04-25T10:15:34.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.616 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018331 s, 57.2 MB/s 2024-04-25T10:15:34.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297594 s, 35.2 MB/s 2024-04-25T10:15:34.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.675 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217684 s, 48.2 MB/s 2024-04-25T10:15:34.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.701 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.701 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206882 s, 50.7 MB/s 2024-04-25T10:15:34.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221681 s, 47.3 MB/s 2024-04-25T10:15:34.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226239 s, 46.3 MB/s 2024-04-25T10:15:34.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235516 s, 44.5 MB/s 2024-04-25T10:15:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242426 s, 43.3 MB/s 2024-04-25T10:15:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224231 s, 46.8 MB/s 2024-04-25T10:15:34.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.861 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204659 s, 51.2 MB/s 2024-04-25T10:15:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.885 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190775 s, 55.0 MB/s 2024-04-25T10:15:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.911 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231239 s, 45.3 MB/s 2024-04-25T10:15:34.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.938 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.939 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023168 s, 45.3 MB/s 2024-04-25T10:15:34.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.969 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269515 s, 38.9 MB/s 2024-04-25T10:15:34.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:34.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:34.996 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226414 s, 46.3 MB/s 2024-04-25T10:15:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.023 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229324 s, 45.7 MB/s 2024-04-25T10:15:35.045 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.045 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.045 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188588 s, 55.6 MB/s 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.075 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252106 s, 41.6 MB/s 2024-04-25T10:15:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208349 s, 50.3 MB/s 2024-04-25T10:15:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177991 s, 58.9 MB/s 2024-04-25T10:15:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199478 s, 52.6 MB/s 2024-04-25T10:15:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233442 s, 44.9 MB/s 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0299835 s, 35.0 MB/s 2024-04-25T10:15:35.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190841 s, 54.9 MB/s 2024-04-25T10:15:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233509 s, 44.9 MB/s 2024-04-25T10:15:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218463 s, 48.0 MB/s 2024-04-25T10:15:35.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196916 s, 53.2 MB/s 2024-04-25T10:15:35.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.332 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218946 s, 47.9 MB/s 2024-04-25T10:15:35.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.359 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227396 s, 46.1 MB/s 2024-04-25T10:15:35.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:35.361 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:35.361 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:35.361 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:15:35.361 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:35.362 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:35.362 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:35.362 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229761 s, 45.6 MB/s 2024-04-25T10:15:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243365 s, 43.1 MB/s 2024-04-25T10:15:35.502 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.503 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208849 s, 50.2 MB/s 2024-04-25T10:15:35.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.529 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227927 s, 46.0 MB/s 2024-04-25T10:15:35.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.554 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.554 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199503 s, 52.6 MB/s 2024-04-25T10:15:35.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235969 s, 44.4 MB/s 2024-04-25T10:15:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.610 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213206 s, 49.2 MB/s 2024-04-25T10:15:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240153 s, 43.7 MB/s 2024-04-25T10:15:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249873 s, 42.0 MB/s 2024-04-25T10:15:35.693 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.694 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264785 s, 39.6 MB/s 2024-04-25T10:15:35.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239415 s, 43.8 MB/s 2024-04-25T10:15:35.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020484 s, 51.2 MB/s 2024-04-25T10:15:35.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.775 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235626 s, 44.5 MB/s 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196768 s, 53.3 MB/s 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212362 s, 49.4 MB/s 2024-04-25T10:15:35.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.846 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019625 s, 53.4 MB/s 2024-04-25T10:15:35.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208695 s, 50.2 MB/s 2024-04-25T10:15:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022358 s, 46.9 MB/s 2024-04-25T10:15:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211107 s, 49.7 MB/s 2024-04-25T10:15:35.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.950 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214437 s, 48.9 MB/s 2024-04-25T10:15:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214892 s, 48.8 MB/s 2024-04-25T10:15:36.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265454 s, 39.5 MB/s 2024-04-25T10:15:36.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.038 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261296 s, 40.1 MB/s 2024-04-25T10:15:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227213 s, 46.1 MB/s 2024-04-25T10:15:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247525 s, 42.4 MB/s 2024-04-25T10:15:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263451 s, 39.8 MB/s 2024-04-25T10:15:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258018 s, 40.6 MB/s 2024-04-25T10:15:36.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.176 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.176 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197319 s, 53.1 MB/s 2024-04-25T10:15:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188567 s, 55.6 MB/s 2024-04-25T10:15:36.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.226 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242644 s, 43.2 MB/s 2024-04-25T10:15:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210833 s, 49.7 MB/s 2024-04-25T10:15:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184873 s, 56.7 MB/s 2024-04-25T10:15:36.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:36.276 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266583 s, 39.3 MB/s 2024-04-25T10:15:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023587 s, 44.5 MB/s 2024-04-25T10:15:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233536 s, 44.9 MB/s 2024-04-25T10:15:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241692 s, 43.4 MB/s 2024-04-25T10:15:36.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216546 s, 48.4 MB/s 2024-04-25T10:15:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210232 s, 49.9 MB/s 2024-04-25T10:15:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236113 s, 44.4 MB/s 2024-04-25T10:15:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213678 s, 49.1 MB/s 2024-04-25T10:15:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226314 s, 46.3 MB/s 2024-04-25T10:15:36.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.602 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225512 s, 46.5 MB/s 2024-04-25T10:15:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222481 s, 47.1 MB/s 2024-04-25T10:15:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237282 s, 44.2 MB/s 2024-04-25T10:15:36.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198677 s, 52.8 MB/s 2024-04-25T10:15:36.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.708 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232228 s, 45.2 MB/s 2024-04-25T10:15:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213731 s, 49.1 MB/s 2024-04-25T10:15:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234248 s, 44.8 MB/s 2024-04-25T10:15:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227251 s, 46.1 MB/s 2024-04-25T10:15:36.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.815 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233202 s, 45.0 MB/s 2024-04-25T10:15:36.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241591 s, 43.4 MB/s 2024-04-25T10:15:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209557 s, 50.0 MB/s 2024-04-25T10:15:36.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.896 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.896 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230654 s, 45.5 MB/s 2024-04-25T10:15:36.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194757 s, 53.8 MB/s 2024-04-25T10:15:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231169 s, 45.4 MB/s 2024-04-25T10:15:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.969 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189203 s, 55.4 MB/s 2024-04-25T10:15:36.997 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:36.997 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:36.997 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239927 s, 43.7 MB/s 2024-04-25T10:15:37.025 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.026 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245414 s, 42.7 MB/s 2024-04-25T10:15:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196749 s, 53.3 MB/s 2024-04-25T10:15:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.076 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225856 s, 46.4 MB/s 2024-04-25T10:15:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288515 s, 36.3 MB/s 2024-04-25T10:15:37.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214934 s, 48.8 MB/s 2024-04-25T10:15:37.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.159 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203009 s, 51.7 MB/s 2024-04-25T10:15:37.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208347 s, 50.3 MB/s 2024-04-25T10:15:37.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:37.186 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220299 s, 47.6 MB/s 2024-04-25T10:15:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221424 s, 47.4 MB/s 2024-04-25T10:15:37.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.321 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224846 s, 46.6 MB/s 2024-04-25T10:15:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232326 s, 45.1 MB/s 2024-04-25T10:15:37.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.370 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183649 s, 57.1 MB/s 2024-04-25T10:15:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.397 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021781 s, 48.1 MB/s 2024-04-25T10:15:37.425 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.425 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.425 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246411 s, 42.6 MB/s 2024-04-25T10:15:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218524 s, 48.0 MB/s 2024-04-25T10:15:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023277 s, 45.0 MB/s 2024-04-25T10:15:37.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216156 s, 48.5 MB/s 2024-04-25T10:15:37.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210423 s, 49.8 MB/s 2024-04-25T10:15:37.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.556 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216648 s, 48.4 MB/s 2024-04-25T10:15:37.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221396 s, 47.4 MB/s 2024-04-25T10:15:37.602 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.603 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0176433 s, 59.4 MB/s 2024-04-25T10:15:37.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.634 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273469 s, 38.3 MB/s 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.664 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260251 s, 40.3 MB/s 2024-04-25T10:15:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.692 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232678 s, 45.1 MB/s 2024-04-25T10:15:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.721 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024304 s, 43.1 MB/s 2024-04-25T10:15:37.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216893 s, 48.3 MB/s 2024-04-25T10:15:37.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.772 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.772 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203075 s, 51.6 MB/s 2024-04-25T10:15:37.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.797 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206578 s, 50.8 MB/s 2024-04-25T10:15:37.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.824 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236519 s, 44.3 MB/s 2024-04-25T10:15:37.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254004 s, 41.3 MB/s 2024-04-25T10:15:37.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.884 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270916 s, 38.7 MB/s 2024-04-25T10:15:37.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.910 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217553 s, 48.2 MB/s 2024-04-25T10:15:37.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.937 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.937 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222795 s, 47.1 MB/s 2024-04-25T10:15:37.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257048 s, 40.8 MB/s 2024-04-25T10:15:37.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:37.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:37.993 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218137 s, 48.1 MB/s 2024-04-25T10:15:38.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.019 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.019 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219948 s, 47.7 MB/s 2024-04-25T10:15:38.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.047 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242253 s, 43.3 MB/s 2024-04-25T10:15:38.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023088 s, 45.4 MB/s 2024-04-25T10:15:38.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216391 s, 48.5 MB/s 2024-04-25T10:15:38.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:38.103 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:38.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205933 s, 50.9 MB/s 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178352 s, 58.8 MB/s 2024-04-25T10:15:38.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.234 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.235 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025348 s, 41.4 MB/s 2024-04-25T10:15:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226688 s, 46.3 MB/s 2024-04-25T10:15:38.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.285 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190733 s, 55.0 MB/s 2024-04-25T10:15:38.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.311 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.311 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022278 s, 47.1 MB/s 2024-04-25T10:15:38.336 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.336 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.336 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203131 s, 51.6 MB/s 2024-04-25T10:15:38.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.369 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.369 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292834 s, 35.8 MB/s 2024-04-25T10:15:38.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.398 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244444 s, 42.9 MB/s 2024-04-25T10:15:38.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.422 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214367 s, 48.9 MB/s 2024-04-25T10:15:38.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.448 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217647 s, 48.2 MB/s 2024-04-25T10:15:38.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211345 s, 49.6 MB/s 2024-04-25T10:15:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203775 s, 51.5 MB/s 2024-04-25T10:15:38.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212288 s, 49.4 MB/s 2024-04-25T10:15:38.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243979 s, 43.0 MB/s 2024-04-25T10:15:38.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211019 s, 49.7 MB/s 2024-04-25T10:15:38.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.603 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230482 s, 45.5 MB/s 2024-04-25T10:15:38.630 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229772 s, 45.6 MB/s 2024-04-25T10:15:38.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.658 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.658 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222258 s, 47.2 MB/s 2024-04-25T10:15:38.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.684 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229579 s, 45.7 MB/s 2024-04-25T10:15:38.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.713 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251102 s, 41.8 MB/s 2024-04-25T10:15:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226088 s, 46.4 MB/s 2024-04-25T10:15:38.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.767 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233776 s, 44.9 MB/s 2024-04-25T10:15:38.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213705 s, 49.1 MB/s 2024-04-25T10:15:38.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210056 s, 49.9 MB/s 2024-04-25T10:15:38.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.846 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.846 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245527 s, 42.7 MB/s 2024-04-25T10:15:38.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215986 s, 48.5 MB/s 2024-04-25T10:15:38.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209872 s, 50.0 MB/s 2024-04-25T10:15:38.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.921 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.921 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208388 s, 50.3 MB/s 2024-04-25T10:15:38.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242563 s, 43.2 MB/s 2024-04-25T10:15:38.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.975 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211619 s, 49.6 MB/s 2024-04-25T10:15:38.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:38.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:38.998 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203007 s, 51.7 MB/s 2024-04-25T10:15:39.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:39.001 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:39.001 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:39.001 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:15:39.001 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:39.001 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:39.002 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:39.002 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:39.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237777 s, 44.1 MB/s 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.115 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224889 s, 46.6 MB/s 2024-04-25T10:15:39.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220574 s, 47.5 MB/s 2024-04-25T10:15:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.167 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233401 s, 44.9 MB/s 2024-04-25T10:15:39.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.189 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185536 s, 56.5 MB/s 2024-04-25T10:15:39.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.209 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160365 s, 65.4 MB/s 2024-04-25T10:15:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235437 s, 44.5 MB/s 2024-04-25T10:15:39.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.261 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.261 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218116 s, 48.1 MB/s 2024-04-25T10:15:39.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210762 s, 49.8 MB/s 2024-04-25T10:15:39.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226237 s, 46.3 MB/s 2024-04-25T10:15:39.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.340 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216766 s, 48.4 MB/s 2024-04-25T10:15:39.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.366 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213422 s, 49.1 MB/s 2024-04-25T10:15:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210668 s, 49.8 MB/s 2024-04-25T10:15:39.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.418 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231346 s, 45.3 MB/s 2024-04-25T10:15:39.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244824 s, 42.8 MB/s 2024-04-25T10:15:39.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212225 s, 49.4 MB/s 2024-04-25T10:15:39.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.503 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267168 s, 39.2 MB/s 2024-04-25T10:15:39.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.534 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270268 s, 38.8 MB/s 2024-04-25T10:15:39.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.560 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222722 s, 47.1 MB/s 2024-04-25T10:15:39.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.582 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189062 s, 55.5 MB/s 2024-04-25T10:15:39.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218646 s, 48.0 MB/s 2024-04-25T10:15:39.634 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.635 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225139 s, 46.6 MB/s 2024-04-25T10:15:39.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.664 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261084 s, 40.2 MB/s 2024-04-25T10:15:39.694 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266911 s, 39.3 MB/s 2024-04-25T10:15:39.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243938 s, 43.0 MB/s 2024-04-25T10:15:39.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.752 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245352 s, 42.7 MB/s 2024-04-25T10:15:39.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.780 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236925 s, 44.3 MB/s 2024-04-25T10:15:39.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241763 s, 43.4 MB/s 2024-04-25T10:15:39.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.834 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226045 s, 46.4 MB/s 2024-04-25T10:15:39.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.862 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235262 s, 44.6 MB/s 2024-04-25T10:15:39.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214936 s, 48.8 MB/s 2024-04-25T10:15:39.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:39.914 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:39.914 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221839 s, 47.3 MB/s 2024-04-25T10:15:39.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:39.917 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:40.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.002 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228976 s, 45.8 MB/s 2024-04-25T10:15:40.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.030 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.031 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237782 s, 44.1 MB/s 2024-04-25T10:15:40.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.062 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274053 s, 38.3 MB/s 2024-04-25T10:15:40.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.092 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267137 s, 39.3 MB/s 2024-04-25T10:15:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188726 s, 55.6 MB/s 2024-04-25T10:15:40.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195374 s, 53.7 MB/s 2024-04-25T10:15:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232181 s, 45.2 MB/s 2024-04-25T10:15:40.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216075 s, 48.5 MB/s 2024-04-25T10:15:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.214 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189866 s, 55.2 MB/s 2024-04-25T10:15:40.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255508 s, 41.0 MB/s 2024-04-25T10:15:40.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229431 s, 45.7 MB/s 2024-04-25T10:15:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218993 s, 47.9 MB/s 2024-04-25T10:15:40.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282386 s, 37.1 MB/s 2024-04-25T10:15:40.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.358 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022581 s, 46.4 MB/s 2024-04-25T10:15:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237146 s, 44.2 MB/s 2024-04-25T10:15:40.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.410 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202314 s, 51.8 MB/s 2024-04-25T10:15:40.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.433 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019497 s, 53.8 MB/s 2024-04-25T10:15:40.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01934 s, 54.2 MB/s 2024-04-25T10:15:40.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265559 s, 39.5 MB/s 2024-04-25T10:15:40.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.514 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247628 s, 42.3 MB/s 2024-04-25T10:15:40.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.542 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220467 s, 47.6 MB/s 2024-04-25T10:15:40.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.571 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244607 s, 42.9 MB/s 2024-04-25T10:15:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233497 s, 44.9 MB/s 2024-04-25T10:15:40.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237652 s, 44.1 MB/s 2024-04-25T10:15:40.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.654 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250074 s, 41.9 MB/s 2024-04-25T10:15:40.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230682 s, 45.5 MB/s 2024-04-25T10:15:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200037 s, 52.4 MB/s 2024-04-25T10:15:40.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213878 s, 49.0 MB/s 2024-04-25T10:15:40.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235453 s, 44.5 MB/s 2024-04-25T10:15:40.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200465 s, 52.3 MB/s 2024-04-25T10:15:40.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214166 s, 49.0 MB/s 2024-04-25T10:15:40.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214426 s, 48.9 MB/s 2024-04-25T10:15:40.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:40.836 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:40.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.923 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027261 s, 38.5 MB/s 2024-04-25T10:15:40.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265043 s, 39.6 MB/s 2024-04-25T10:15:40.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.976 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.976 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201069 s, 52.2 MB/s 2024-04-25T10:15:40.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:40.998 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:40.998 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018015 s, 58.2 MB/s 2024-04-25T10:15:41.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.022 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020363 s, 51.5 MB/s 2024-04-25T10:15:41.049 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.049 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.049 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235632 s, 44.5 MB/s 2024-04-25T10:15:41.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.078 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246956 s, 42.5 MB/s 2024-04-25T10:15:41.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.103 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.103 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218324 s, 48.0 MB/s 2024-04-25T10:15:41.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.133 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262824 s, 39.9 MB/s 2024-04-25T10:15:41.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.162 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024025 s, 43.6 MB/s 2024-04-25T10:15:41.188 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.188 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.188 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235574 s, 44.5 MB/s 2024-04-25T10:15:41.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.212 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.212 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199812 s, 52.5 MB/s 2024-04-25T10:15:41.236 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.236 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.236 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197749 s, 53.0 MB/s 2024-04-25T10:15:41.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.260 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.260 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206121 s, 50.9 MB/s 2024-04-25T10:15:41.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.283 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189445 s, 55.3 MB/s 2024-04-25T10:15:41.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217251 s, 48.3 MB/s 2024-04-25T10:15:41.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254309 s, 41.2 MB/s 2024-04-25T10:15:41.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.364 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.365 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214373 s, 48.9 MB/s 2024-04-25T10:15:41.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.389 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020735 s, 50.6 MB/s 2024-04-25T10:15:41.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.415 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217427 s, 48.2 MB/s 2024-04-25T10:15:41.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028171 s, 37.2 MB/s 2024-04-25T10:15:41.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.474 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.474 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221459 s, 47.3 MB/s 2024-04-25T10:15:41.501 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.501 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.501 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237641 s, 44.1 MB/s 2024-04-25T10:15:41.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203772 s, 51.5 MB/s 2024-04-25T10:15:41.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.544 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0160609 s, 65.3 MB/s 2024-04-25T10:15:41.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.570 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021957 s, 47.8 MB/s 2024-04-25T10:15:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270173 s, 38.8 MB/s 2024-04-25T10:15:41.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.628 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234671 s, 44.7 MB/s 2024-04-25T10:15:41.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.656 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234795 s, 44.7 MB/s 2024-04-25T10:15:41.685 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.685 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.685 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238948 s, 43.9 MB/s 2024-04-25T10:15:41.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217562 s, 48.2 MB/s 2024-04-25T10:15:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254759 s, 41.2 MB/s 2024-04-25T10:15:41.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:41.747 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:41.747 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:41.747 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:15:41.747 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:41.747 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:41.748 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:41.748 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:41.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.838 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.838 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234073 s, 44.8 MB/s 2024-04-25T10:15:41.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.861 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199158 s, 52.7 MB/s 2024-04-25T10:15:41.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219798 s, 47.7 MB/s 2024-04-25T10:15:41.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239173 s, 43.8 MB/s 2024-04-25T10:15:41.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.948 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276619 s, 37.9 MB/s 2024-04-25T10:15:41.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:41.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:41.977 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261435 s, 40.1 MB/s 2024-04-25T10:15:42.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.009 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0275541 s, 38.1 MB/s 2024-04-25T10:15:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.035 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205578 s, 51.0 MB/s 2024-04-25T10:15:42.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.061 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225882 s, 46.4 MB/s 2024-04-25T10:15:42.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.086 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206639 s, 50.7 MB/s 2024-04-25T10:15:42.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.112 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.112 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217742 s, 48.2 MB/s 2024-04-25T10:15:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226648 s, 46.3 MB/s 2024-04-25T10:15:42.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218061 s, 48.1 MB/s 2024-04-25T10:15:42.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.187 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.187 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181408 s, 57.8 MB/s 2024-04-25T10:15:42.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.218 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269447 s, 38.9 MB/s 2024-04-25T10:15:42.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243377 s, 43.1 MB/s 2024-04-25T10:15:42.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212594 s, 49.3 MB/s 2024-04-25T10:15:42.298 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.299 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224898 s, 46.6 MB/s 2024-04-25T10:15:42.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.327 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252154 s, 41.6 MB/s 2024-04-25T10:15:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239035 s, 43.9 MB/s 2024-04-25T10:15:42.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262923 s, 39.9 MB/s 2024-04-25T10:15:42.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.412 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.412 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227517 s, 46.1 MB/s 2024-04-25T10:15:42.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213838 s, 49.0 MB/s 2024-04-25T10:15:42.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.463 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217508 s, 48.2 MB/s 2024-04-25T10:15:42.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.485 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188886 s, 55.5 MB/s 2024-04-25T10:15:42.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.514 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251021 s, 41.8 MB/s 2024-04-25T10:15:42.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.545 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264474 s, 39.6 MB/s 2024-04-25T10:15:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.573 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236234 s, 44.4 MB/s 2024-04-25T10:15:42.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205387 s, 51.1 MB/s 2024-04-25T10:15:42.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.626 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.626 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248057 s, 42.3 MB/s 2024-04-25T10:15:42.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.654 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.654 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244469 s, 42.9 MB/s 2024-04-25T10:15:42.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.682 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238281 s, 44.0 MB/s 2024-04-25T10:15:42.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:42.684 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:42.684 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:42.684 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:15:42.684 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:42.684 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:42.685 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:42.685 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:42.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.767 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214018 s, 49.0 MB/s 2024-04-25T10:15:42.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222714 s, 47.1 MB/s 2024-04-25T10:15:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226537 s, 46.3 MB/s 2024-04-25T10:15:42.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.843 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222319 s, 47.2 MB/s 2024-04-25T10:15:42.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234406 s, 44.7 MB/s 2024-04-25T10:15:42.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.900 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.900 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260022 s, 40.3 MB/s 2024-04-25T10:15:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211036 s, 49.7 MB/s 2024-04-25T10:15:42.954 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.955 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252854 s, 41.5 MB/s 2024-04-25T10:15:42.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:42.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:42.981 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225143 s, 46.6 MB/s 2024-04-25T10:15:43.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262862 s, 39.9 MB/s 2024-04-25T10:15:43.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.040 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245727 s, 42.7 MB/s 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230642 s, 45.5 MB/s 2024-04-25T10:15:43.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.096 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255684 s, 41.0 MB/s 2024-04-25T10:15:43.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.123 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218198 s, 48.1 MB/s 2024-04-25T10:15:43.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258366 s, 40.6 MB/s 2024-04-25T10:15:43.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220474 s, 47.6 MB/s 2024-04-25T10:15:43.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.198 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.198 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0165664 s, 63.3 MB/s 2024-04-25T10:15:43.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.225 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224406 s, 46.7 MB/s 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.253 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253466 s, 41.4 MB/s 2024-04-25T10:15:43.281 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.281 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.281 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022989 s, 45.6 MB/s 2024-04-25T10:15:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208065 s, 50.4 MB/s 2024-04-25T10:15:43.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222561 s, 47.1 MB/s 2024-04-25T10:15:43.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.361 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247472 s, 42.4 MB/s 2024-04-25T10:15:43.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200848 s, 52.2 MB/s 2024-04-25T10:15:43.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.410 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.410 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211306 s, 49.6 MB/s 2024-04-25T10:15:43.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218537 s, 48.0 MB/s 2024-04-25T10:15:43.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.464 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246881 s, 42.5 MB/s 2024-04-25T10:15:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229846 s, 45.6 MB/s 2024-04-25T10:15:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023273 s, 45.1 MB/s 2024-04-25T10:15:43.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0296564 s, 35.4 MB/s 2024-04-25T10:15:43.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.581 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245952 s, 42.6 MB/s 2024-04-25T10:15:43.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.607 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023193 s, 45.2 MB/s 2024-04-25T10:15:43.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:43.610 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:43.610 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:43.610 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:15:43.610 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:43.611 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:43.611 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:43.611 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:43.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.696 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184883 s, 56.7 MB/s 2024-04-25T10:15:43.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240376 s, 43.6 MB/s 2024-04-25T10:15:43.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.749 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.749 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231465 s, 45.3 MB/s 2024-04-25T10:15:43.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.774 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021612 s, 48.5 MB/s 2024-04-25T10:15:43.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210534 s, 49.8 MB/s 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232562 s, 45.1 MB/s 2024-04-25T10:15:43.854 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217777 s, 48.1 MB/s 2024-04-25T10:15:43.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.884 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241026 s, 43.5 MB/s 2024-04-25T10:15:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216827 s, 48.4 MB/s 2024-04-25T10:15:43.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222398 s, 47.1 MB/s 2024-04-25T10:15:43.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.966 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266994 s, 39.3 MB/s 2024-04-25T10:15:43.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:43.993 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:43.994 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229577 s, 45.7 MB/s 2024-04-25T10:15:44.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.026 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0288315 s, 36.4 MB/s 2024-04-25T10:15:44.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.053 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218993 s, 47.9 MB/s 2024-04-25T10:15:44.078 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223592 s, 46.9 MB/s 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229221 s, 45.7 MB/s 2024-04-25T10:15:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.134 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.134 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232627 s, 45.1 MB/s 2024-04-25T10:15:44.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.162 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246239 s, 42.6 MB/s 2024-04-25T10:15:44.188 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.188 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.188 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215955 s, 48.6 MB/s 2024-04-25T10:15:44.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.222 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0307958 s, 34.0 MB/s 2024-04-25T10:15:44.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.253 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264035 s, 39.7 MB/s 2024-04-25T10:15:44.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287976 s, 36.4 MB/s 2024-04-25T10:15:44.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237256 s, 44.2 MB/s 2024-04-25T10:15:44.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208011 s, 50.4 MB/s 2024-04-25T10:15:44.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.365 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234545 s, 44.7 MB/s 2024-04-25T10:15:44.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.389 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203202 s, 51.6 MB/s 2024-04-25T10:15:44.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227754 s, 46.0 MB/s 2024-04-25T10:15:44.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.441 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.441 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202191 s, 51.9 MB/s 2024-04-25T10:15:44.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.468 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240469 s, 43.6 MB/s 2024-04-25T10:15:44.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.495 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.495 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227149 s, 46.2 MB/s 2024-04-25T10:15:44.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.523 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.523 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221111 s, 47.4 MB/s 2024-04-25T10:15:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212308 s, 49.4 MB/s 2024-04-25T10:15:44.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:44.550 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216135 s, 48.5 MB/s 2024-04-25T10:15:44.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0278571 s, 37.6 MB/s 2024-04-25T10:15:44.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286317 s, 36.6 MB/s 2024-04-25T10:15:44.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242524 s, 43.2 MB/s 2024-04-25T10:15:44.761 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.761 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.761 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242713 s, 43.2 MB/s 2024-04-25T10:15:44.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.788 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228885 s, 45.8 MB/s 2024-04-25T10:15:44.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.819 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271028 s, 38.7 MB/s 2024-04-25T10:15:44.845 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.845 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.845 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02171 s, 48.3 MB/s 2024-04-25T10:15:44.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.872 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.872 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226704 s, 46.3 MB/s 2024-04-25T10:15:44.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.898 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022135 s, 47.4 MB/s 2024-04-25T10:15:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215728 s, 48.6 MB/s 2024-04-25T10:15:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023407 s, 44.8 MB/s 2024-04-25T10:15:44.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:44.979 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:44.979 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242699 s, 43.2 MB/s 2024-04-25T10:15:45.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.007 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.007 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222938 s, 47.0 MB/s 2024-04-25T10:15:45.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.035 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.035 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248824 s, 42.1 MB/s 2024-04-25T10:15:45.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.066 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262313 s, 40.0 MB/s 2024-04-25T10:15:45.090 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.090 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.090 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214109 s, 49.0 MB/s 2024-04-25T10:15:45.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.115 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213022 s, 49.2 MB/s 2024-04-25T10:15:45.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236869 s, 44.3 MB/s 2024-04-25T10:15:45.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.169 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223304 s, 47.0 MB/s 2024-04-25T10:15:45.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.202 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.202 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272738 s, 38.4 MB/s 2024-04-25T10:15:45.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.230 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242681 s, 43.2 MB/s 2024-04-25T10:15:45.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.256 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.256 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02148 s, 48.8 MB/s 2024-04-25T10:15:45.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.284 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.284 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022406 s, 46.8 MB/s 2024-04-25T10:15:45.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.305 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185575 s, 56.5 MB/s 2024-04-25T10:15:45.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.333 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.333 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236803 s, 44.3 MB/s 2024-04-25T10:15:45.362 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.362 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.362 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252318 s, 41.6 MB/s 2024-04-25T10:15:45.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.388 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220716 s, 47.5 MB/s 2024-04-25T10:15:45.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.424 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313433 s, 33.5 MB/s 2024-04-25T10:15:45.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227095 s, 46.2 MB/s 2024-04-25T10:15:45.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.489 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0348651 s, 30.1 MB/s 2024-04-25T10:15:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213816 s, 49.0 MB/s 2024-04-25T10:15:45.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:45.517 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:45.518 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:45.518 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:15:45.518 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:45.518 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:45.518 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:45.518 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:45.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.604 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.604 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219369 s, 47.8 MB/s 2024-04-25T10:15:45.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.635 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271477 s, 38.6 MB/s 2024-04-25T10:15:45.660 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.661 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202882 s, 51.7 MB/s 2024-04-25T10:15:45.685 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.685 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.685 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205313 s, 51.1 MB/s 2024-04-25T10:15:45.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018014 s, 58.2 MB/s 2024-04-25T10:15:45.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200836 s, 52.2 MB/s 2024-04-25T10:15:45.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229899 s, 45.6 MB/s 2024-04-25T10:15:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247267 s, 42.4 MB/s 2024-04-25T10:15:45.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263789 s, 39.8 MB/s 2024-04-25T10:15:45.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.841 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.841 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193263 s, 54.3 MB/s 2024-04-25T10:15:45.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227442 s, 46.1 MB/s 2024-04-25T10:15:45.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.899 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.899 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260871 s, 40.2 MB/s 2024-04-25T10:15:45.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.930 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272026 s, 38.5 MB/s 2024-04-25T10:15:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211968 s, 49.5 MB/s 2024-04-25T10:15:45.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:45.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:45.981 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216481 s, 48.4 MB/s 2024-04-25T10:15:46.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.011 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.011 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255731 s, 41.0 MB/s 2024-04-25T10:15:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.038 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023061 s, 45.5 MB/s 2024-04-25T10:15:46.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244793 s, 42.8 MB/s 2024-04-25T10:15:46.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.096 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249103 s, 42.1 MB/s 2024-04-25T10:15:46.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.124 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241856 s, 43.4 MB/s 2024-04-25T10:15:46.155 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.155 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.155 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274728 s, 38.2 MB/s 2024-04-25T10:15:46.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.184 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245745 s, 42.7 MB/s 2024-04-25T10:15:46.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.210 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225155 s, 46.6 MB/s 2024-04-25T10:15:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022191 s, 47.3 MB/s 2024-04-25T10:15:46.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228768 s, 45.8 MB/s 2024-04-25T10:15:46.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.294 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254267 s, 41.2 MB/s 2024-04-25T10:15:46.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.319 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.319 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218783 s, 47.9 MB/s 2024-04-25T10:15:46.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246925 s, 42.5 MB/s 2024-04-25T10:15:46.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.374 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.374 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222794 s, 47.1 MB/s 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.401 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228563 s, 45.9 MB/s 2024-04-25T10:15:46.421 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.421 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.421 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171069 s, 61.3 MB/s 2024-04-25T10:15:46.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.448 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.448 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022707 s, 46.2 MB/s 2024-04-25T10:15:46.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:46.451 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:46.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198182 s, 52.9 MB/s 2024-04-25T10:15:46.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.560 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021418 s, 49.0 MB/s 2024-04-25T10:15:46.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.587 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.587 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227192 s, 46.2 MB/s 2024-04-25T10:15:46.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.617 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257209 s, 40.8 MB/s 2024-04-25T10:15:46.641 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.642 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021464 s, 48.9 MB/s 2024-04-25T10:15:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0178889 s, 58.6 MB/s 2024-04-25T10:15:46.696 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.697 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.027192 s, 38.6 MB/s 2024-04-25T10:15:46.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0287486 s, 36.5 MB/s 2024-04-25T10:15:46.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233907 s, 44.8 MB/s 2024-04-25T10:15:46.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.786 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251379 s, 41.7 MB/s 2024-04-25T10:15:46.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.818 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280392 s, 37.4 MB/s 2024-04-25T10:15:46.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.847 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.847 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249526 s, 42.0 MB/s 2024-04-25T10:15:46.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.879 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279098 s, 37.6 MB/s 2024-04-25T10:15:46.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025259 s, 41.5 MB/s 2024-04-25T10:15:46.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.930 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.017881 s, 58.6 MB/s 2024-04-25T10:15:46.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.959 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02507 s, 41.8 MB/s 2024-04-25T10:15:46.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:46.988 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:46.988 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244276 s, 42.9 MB/s 2024-04-25T10:15:47.023 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.024 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0321736 s, 32.6 MB/s 2024-04-25T10:15:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.053 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250432 s, 41.9 MB/s 2024-04-25T10:15:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.086 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265546 s, 39.5 MB/s 2024-04-25T10:15:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.113 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025373 s, 41.3 MB/s 2024-04-25T10:15:47.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.138 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.138 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208308 s, 50.3 MB/s 2024-04-25T10:15:47.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.162 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210955 s, 49.7 MB/s 2024-04-25T10:15:47.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025295 s, 41.5 MB/s 2024-04-25T10:15:47.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.215 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.215 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201635 s, 52.0 MB/s 2024-04-25T10:15:47.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.239 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201951 s, 51.9 MB/s 2024-04-25T10:15:47.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.266 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236219 s, 44.4 MB/s 2024-04-25T10:15:47.298 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.298 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.298 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0279608 s, 37.5 MB/s 2024-04-25T10:15:47.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.332 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.332 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0291105 s, 36.0 MB/s 2024-04-25T10:15:47.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.357 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.357 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208787 s, 50.2 MB/s 2024-04-25T10:15:47.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.379 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185154 s, 56.6 MB/s 2024-04-25T10:15:47.400 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.401 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186322 s, 56.3 MB/s 2024-04-25T10:15:47.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:47.403 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:47.404 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:47.404 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:15:47.404 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:47.404 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:47.404 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:47.404 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:47.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240526 s, 43.6 MB/s 2024-04-25T10:15:47.524 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233255 s, 45.0 MB/s 2024-04-25T10:15:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229967 s, 45.6 MB/s 2024-04-25T10:15:47.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.580 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236192 s, 44.4 MB/s 2024-04-25T10:15:47.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.607 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023378 s, 44.9 MB/s 2024-04-25T10:15:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.633 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219464 s, 47.8 MB/s 2024-04-25T10:15:47.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.661 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023997 s, 43.7 MB/s 2024-04-25T10:15:47.689 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.689 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.690 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242264 s, 43.3 MB/s 2024-04-25T10:15:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220571 s, 47.5 MB/s 2024-04-25T10:15:47.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.741 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213129 s, 49.2 MB/s 2024-04-25T10:15:47.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.769 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234511 s, 44.7 MB/s 2024-04-25T10:15:47.793 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.793 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.793 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206184 s, 50.9 MB/s 2024-04-25T10:15:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01855 s, 56.5 MB/s 2024-04-25T10:15:47.843 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223385 s, 46.9 MB/s 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.869 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212354 s, 49.4 MB/s 2024-04-25T10:15:47.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.894 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.894 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220996 s, 47.4 MB/s 2024-04-25T10:15:47.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.922 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240168 s, 43.7 MB/s 2024-04-25T10:15:47.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230505 s, 45.5 MB/s 2024-04-25T10:15:47.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:47.980 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:47.981 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273794 s, 38.3 MB/s 2024-04-25T10:15:48.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.010 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253238 s, 41.4 MB/s 2024-04-25T10:15:48.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0318379 s, 32.9 MB/s 2024-04-25T10:15:48.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.073 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232156 s, 45.2 MB/s 2024-04-25T10:15:48.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222415 s, 47.1 MB/s 2024-04-25T10:15:48.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.132 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.132 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277539 s, 37.8 MB/s 2024-04-25T10:15:48.160 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.161 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.161 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252464 s, 41.5 MB/s 2024-04-25T10:15:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214098 s, 49.0 MB/s 2024-04-25T10:15:48.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.213 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.213 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231667 s, 45.3 MB/s 2024-04-25T10:15:48.237 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.237 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.237 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208563 s, 50.3 MB/s 2024-04-25T10:15:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.265 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235304 s, 44.6 MB/s 2024-04-25T10:15:48.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.288 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209028 s, 50.2 MB/s 2024-04-25T10:15:48.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.318 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249499 s, 42.0 MB/s 2024-04-25T10:15:48.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.345 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.345 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227743 s, 46.0 MB/s 2024-04-25T10:15:48.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:48.348 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:48.348 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:48.348 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:15:48.348 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:48.348 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:48.349 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:48.349 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:48.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.435 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.435 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230514 s, 45.5 MB/s 2024-04-25T10:15:48.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.464 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247126 s, 42.4 MB/s 2024-04-25T10:15:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219071 s, 47.9 MB/s 2024-04-25T10:15:48.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.515 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215075 s, 48.8 MB/s 2024-04-25T10:15:48.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.542 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.542 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233024 s, 45.0 MB/s 2024-04-25T10:15:48.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.568 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226004 s, 46.4 MB/s 2024-04-25T10:15:48.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.596 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230333 s, 45.5 MB/s 2024-04-25T10:15:48.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214189 s, 49.0 MB/s 2024-04-25T10:15:48.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223721 s, 46.9 MB/s 2024-04-25T10:15:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203524 s, 51.5 MB/s 2024-04-25T10:15:48.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240954 s, 43.5 MB/s 2024-04-25T10:15:48.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223799 s, 46.9 MB/s 2024-04-25T10:15:48.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211452 s, 49.6 MB/s 2024-04-25T10:15:48.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.780 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024575 s, 42.7 MB/s 2024-04-25T10:15:48.804 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.805 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.805 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212466 s, 49.4 MB/s 2024-04-25T10:15:48.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.836 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272477 s, 38.5 MB/s 2024-04-25T10:15:48.866 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.867 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266094 s, 39.4 MB/s 2024-04-25T10:15:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019546 s, 53.6 MB/s 2024-04-25T10:15:48.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.916 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.916 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220264 s, 47.6 MB/s 2024-04-25T10:15:48.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.943 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228451 s, 45.9 MB/s 2024-04-25T10:15:48.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233406 s, 44.9 MB/s 2024-04-25T10:15:48.997 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:48.997 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:48.997 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235596 s, 44.5 MB/s 2024-04-25T10:15:49.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.024 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.024 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231088 s, 45.4 MB/s 2024-04-25T10:15:49.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.052 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236208 s, 44.4 MB/s 2024-04-25T10:15:49.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.084 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.084 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280438 s, 37.4 MB/s 2024-04-25T10:15:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228409 s, 45.9 MB/s 2024-04-25T10:15:49.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.136 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219227 s, 47.8 MB/s 2024-04-25T10:15:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265042 s, 39.6 MB/s 2024-04-25T10:15:49.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195844 s, 53.5 MB/s 2024-04-25T10:15:49.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.217 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.217 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221567 s, 47.3 MB/s 2024-04-25T10:15:49.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218698 s, 47.9 MB/s 2024-04-25T10:15:49.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277272 s, 37.8 MB/s 2024-04-25T10:15:49.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:49.278 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:15:49.278 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:15:49.278 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:15:49.278 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:15:49.279 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:15:49.279 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:15:49.279 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:15:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268279 s, 39.1 MB/s 2024-04-25T10:15:49.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.399 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231417 s, 45.3 MB/s 2024-04-25T10:15:49.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.424 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.424 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223074 s, 47.0 MB/s 2024-04-25T10:15:49.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.455 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.455 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263004 s, 39.9 MB/s 2024-04-25T10:15:49.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249012 s, 42.1 MB/s 2024-04-25T10:15:49.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.512 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236412 s, 44.4 MB/s 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023637 s, 44.4 MB/s 2024-04-25T10:15:49.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.570 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025447 s, 41.2 MB/s 2024-04-25T10:15:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023853 s, 44.0 MB/s 2024-04-25T10:15:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211483 s, 49.6 MB/s 2024-04-25T10:15:49.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.652 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.652 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255926 s, 41.0 MB/s 2024-04-25T10:15:49.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228211 s, 45.9 MB/s 2024-04-25T10:15:49.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.710 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267678 s, 39.2 MB/s 2024-04-25T10:15:49.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187186 s, 56.0 MB/s 2024-04-25T10:15:49.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.756 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179809 s, 58.3 MB/s 2024-04-25T10:15:49.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.786 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258141 s, 40.6 MB/s 2024-04-25T10:15:49.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.811 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213315 s, 49.2 MB/s 2024-04-25T10:15:49.839 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.839 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.839 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232757 s, 45.1 MB/s 2024-04-25T10:15:49.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.865 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.865 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220676 s, 47.5 MB/s 2024-04-25T10:15:49.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.887 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.887 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0171164 s, 61.3 MB/s 2024-04-25T10:15:49.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.913 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.913 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021842 s, 48.0 MB/s 2024-04-25T10:15:49.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.942 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024806 s, 42.3 MB/s 2024-04-25T10:15:49.964 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.965 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192666 s, 54.4 MB/s 2024-04-25T10:15:49.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:49.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:49.995 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267996 s, 39.1 MB/s 2024-04-25T10:15:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211933 s, 49.5 MB/s 2024-04-25T10:15:50.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.046 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.046 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215336 s, 48.7 MB/s 2024-04-25T10:15:50.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023037 s, 45.5 MB/s 2024-04-25T10:15:50.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216739 s, 48.4 MB/s 2024-04-25T10:15:50.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.131 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265712 s, 39.5 MB/s 2024-04-25T10:15:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225699 s, 46.5 MB/s 2024-04-25T10:15:50.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.182 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218329 s, 48.0 MB/s 2024-04-25T10:15:50.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:15:50.210 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:15:50.210 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232624 s, 45.1 MB/s 2024-04-25T10:15:50.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:50.213 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub start /scrub_control_test_path recursive 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.425+0000 7ff0d2200640 1 Processor -- start 2024-04-25T10:15:50.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.426+0000 7ff0d2200640 1 -- start start 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d2200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff0cc15dd30 0x7ff0cc0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d2200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc15e110 con 0x7ff0cc0b5b40 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d2200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0cc0b2fd0 con 0x7ff0cc06bff0 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d2200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc06d000 con 0x7ff0cc15dd30 2024-04-25T10:15:50.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d1800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff0cc0b5b40 0x7ff0cc0d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36540/0 (socket says 172.21.15.165:36540) 2024-04-25T10:15:50.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d1800640 1 -- 172.21.15.165:0/2863388693 learned_addr learned my addr 172.21.15.165:0/2863388693 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.428+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/2863388693 >> v2:172.21.15.172:3300/0 conn(0x7ff0cc15dd30 0x7ff0cc0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1183354615 0 0) 0x7ff0cc15e110 con 0x7ff0cc0b5b40 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0ac003520 con 0x7ff0cc0b5b40 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4167565542 0 0) 0x7ff0ac003520 con 0x7ff0cc0b5b40 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 >> v2:172.21.15.172:3300/0 conn(0x7ff0cc15dd30 msgr2=0x7ff0cc0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 --2- 172.21.15.165:0/2863388693 >> v2:172.21.15.172:3300/0 conn(0x7ff0cc15dd30 0x7ff0cc0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 >> v1:172.21.15.172:6789/0 conn(0x7ff0cc06bff0 legacy=0x7ff0cc0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc06c480 con 0x7ff0cc0b5b40 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff0c40019f0 con 0x7ff0cc0b5b40 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 >> v1:172.21.15.165:6789/0 conn(0x7ff0cc0b5b40 legacy=0x7ff0cc0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac003a00 0x7ff0ac003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0ac004800 0x7ff0ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc06d000 con 0x7ff0ac004800 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00c0c0 con 0x7ff0ac0051c0 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac00c2c0 con 0x7ff0ac003a00 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.429+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0ac004800 0x7ff0ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac003a00 0x7ff0ac003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0d1800640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0cbe00640 1 -- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac003a00 msgr2=0x7ff0ac003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac003a00 0x7ff0ac003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0cbe00640 1 -- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac0051c0 msgr2=0x7ff0ac00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0ac0051c0 0x7ff0ac00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0d1800640 1 -- 172.21.15.165:0/2863388693 reap_dead start 2024-04-25T10:15:50.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.430+0000 7ff0cbe00640 1 -- 172.21.15.165:0/2863388693 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac003520 con 0x7ff0ac004800 2024-04-25T10:15:50.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.431+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0ac004800 0x7ff0ac004c80 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4001750 tx=0x7ff0ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c01875980b5a367 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.431+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4001370 con 0x7ff0ac004800 2024-04-25T10:15:50.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.431+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b40052d0 con 0x7ff0ac004800 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.431+0000 7ff0cb400640 1 -- 172.21.15.165:0/2863388693 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b402abc0 con 0x7ff0ac004800 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 -- 172.21.15.165:0/2863388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0ac004800 msgr2=0x7ff0ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0ac004800 0x7ff0ac004c80 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4001750 tx=0x7ff0ac005830 comp rx=0 tx=0).stop 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 -- 172.21.15.165:0/2863388693 shutdown_connections 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 --2- 172.21.15.165:0/2863388693 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0ac004800 0x7ff0ac004c80 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 -- 172.21.15.165:0/2863388693 >> 172.21.15.165:0/2863388693 conn(0x7ff0cc0d0610 msgr2=0x7ff0cc0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 -- 172.21.15.165:0/2863388693 shutdown_connections 2024-04-25T10:15:50.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.432+0000 7ff0d2200640 1 -- 172.21.15.165:0/2863388693 wait complete. 2024-04-25T10:15:50.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.434+0000 7ff0d2200640 1 Processor -- start 2024-04-25T10:15:50.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.434+0000 7ff0d2200640 1 -- start start 2024-04-25T10:15:50.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac004800 0x7ff0cc1576c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d2200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 0x7ff0cc14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0cc0cd6d0 0x7ff0cc160310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d2200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc06cd20 con 0x7ff0cc156e20 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc06ca40 con 0x7ff0cc0cd6d0 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0b5430 con 0x7ff0ac004800 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 0x7ff0cc14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 0x7ff0cc14dd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35562/0 (socket says 172.21.15.165:35562) 2024-04-25T10:15:50.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 -- 172.21.15.165:0/881001208 learned_addr learned my addr 172.21.15.165:0/881001208 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 -- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac004800 msgr2=0x7ff0cc1576c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d1800640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0cc0cd6d0 0x7ff0cc160310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac004800 0x7ff0cc1576c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac004800 0x7ff0cc1576c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 -- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0cc0cd6d0 msgr2=0x7ff0cc160310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.435+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0cc0cd6d0 0x7ff0cc160310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0cbe00640 1 -- 172.21.15.165:0/881001208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0cc630 con 0x7ff0cc156e20 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 0x7ff0cc14dd60 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff0b402a570 tx=0x7ff0b4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0c8c00640 1 -- 172.21.15.165:0/881001208 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4037040 con 0x7ff0cc156e20 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0c8c00640 1 -- 172.21.15.165:0/881001208 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0b4040070 con 0x7ff0cc156e20 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0c8c00640 1 -- 172.21.15.165:0/881001208 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4007a30 con 0x7ff0cc156e20 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc0cc910 con 0x7ff0cc156e20 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.436+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc0cced0 con 0x7ff0cc156e20 2024-04-25T10:15:50.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.438+0000 7ff0c8c00640 1 -- 172.21.15.165:0/881001208 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff0b40052d0 con 0x7ff0cc156e20 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.439+0000 7ff0c8c00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0a806b4d0 0x7ff0a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.439+0000 7ff0c8c00640 1 -- 172.21.15.165:0/881001208 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0b40a9ff0 con 0x7ff0cc156e20 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.440+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0a806b4d0 0x7ff0a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.440+0000 7ff0d658e740 1 Processor -- start 2024-04-25T10:15:50.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.440+0000 7ff0d658e740 1 -- start start 2024-04-25T10:15:50.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.440+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0a806b4d0 0x7ff0a806d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0692f0 tx=0x7ff0bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 0x5592ca948360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca965d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca91dc60 0x5592ca963ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d658e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ca87bf00 con 0x5592ca947f80 2024-04-25T10:15:50.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ca880f50 con 0x5592ca965820 2024-04-25T10:15:50.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ca82a0b0 con 0x5592ca91dc60 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 0x5592ca948360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 0x5592ca948360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35570/0 (socket says 172.21.15.165:35570) 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.441+0000 7ff0d0e00640 1 -- 172.21.15.165:0/3139773927 learned_addr learned my addr 172.21.15.165:0/3139773927 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca965d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d1800640 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca91dc60 0x5592ca963ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d0e00640 1 -- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca91dc60 msgr2=0x5592ca963ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca91dc60 0x5592ca963ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d0e00640 1 -- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 msgr2=0x5592ca965d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca965d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d0e00640 1 -- 172.21.15.165:0/3139773927 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ca661ba0 con 0x5592ca947f80 2024-04-25T10:15:50.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.442+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 0x5592ca948360 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc012be0 tx=0x7ff0bc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a21d763ff0e318 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0c1600640 1 -- 172.21.15.165:0/3139773927 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc022440 con 0x5592ca947f80 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0c1600640 1 -- 172.21.15.165:0/3139773927 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc027070 con 0x5592ca947f80 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0c1600640 1 -- 172.21.15.165:0/3139773927 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0227e0 con 0x5592ca947f80 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 -- 172.21.15.165:0/3139773927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 msgr2=0x5592ca948360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 0x5592ca948360 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc012be0 tx=0x7ff0bc015000 comp rx=0 tx=0).stop 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 -- 172.21.15.165:0/3139773927 shutdown_connections 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca91dc60 0x5592ca963ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca965d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3139773927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca947f80 0x5592ca948360 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 -- 172.21.15.165:0/3139773927 >> 172.21.15.165:0/3139773927 conn(0x5592ca91a820 msgr2=0x5592ca962d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 -- 172.21.15.165:0/3139773927 shutdown_connections 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.443+0000 7ff0d658e740 1 -- 172.21.15.165:0/3139773927 wait complete. 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.444+0000 7ff0d658e740 1 -- start start 2024-04-25T10:15:50.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.444+0000 7ff0d658e740 1 Processor -- start 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca963490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca95cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ca81f620 con 0x5592ca9760e0 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ca87cff0 con 0x5592ca965820 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ca91ea00 con 0x5592ca947f80 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.445+0000 7ff0d658e740 10 client.0 _ll_register_callbacks cb 0x5592ca594f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35574/0 (socket says 172.21.15.165:35574) 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 -- 172.21.15.165:0/3143022213 learned_addr learned my addr 172.21.15.165:0/3143022213 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca95cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 -- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 msgr2=0x5592ca963490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca963490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca963490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 -- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 msgr2=0x5592ca95cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca95cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 -- 172.21.15.165:0/3143022213 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592ca555490 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca95cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0d1800640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95d520 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0b9aa0 tx=0x7ff0c400fbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbad2df1de94eba7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.446+0000 7ff0ba000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0ba000640 1 -- 172.21.15.165:0/3143022213 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4004b60 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0ba000640 1 -- 172.21.15.165:0/3143022213 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0c4004d00 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0bc027070 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0ba000640 1 -- 172.21.15.165:0/3143022213 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0c400d790 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 10 client.9399 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5592ca6c2330 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0ba000640 1 -- 172.21.15.165:0/3143022213 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc027070 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0ba000640 1 -- 172.21.15.165:0/3143022213 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff0c400d930 con 0x5592ca9760e0 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 10 client.9399 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 10 client.9399 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.b 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 20 client.9399 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:50.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca976a20 0x5592ca976e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 4 client.9399 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:50.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.447+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5592ca88e490 con 0x5592ca976a20 2024-04-25T10:15:50.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.448+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca976a20 0x5592ca976e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.449+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca976a20 0x5592ca976e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ebbff18f5e5702 server_cookie=36b199bc3102a164 in_seq=0 out_seq=0 2024-04-25T10:15:50.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.449+0000 7ff0ba000640 10 client.9399 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.451+0000 7ff0ba000640 1 -- 172.21.15.165:0/3143022213 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5592ca88e490 con 0x5592ca976a20 2024-04-25T10:15:50.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.451+0000 7ff0ba000640 10 client.9399 handle_command_reply: tid=0 2024-04-25T10:15:50.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.451+0000 7ff0d658e740 1 client.9399 shutdown 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 msgr2=0x5592ca95d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95d520 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0b9aa0 tx=0x7ff0c400fbb0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 shutdown_connections 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95d520 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca976a20 0x5592ca976e30 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca965820 0x5592ca95cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 --2- 172.21.15.165:0/3143022213 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca963490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0ba000640 0 client.9399 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 >> 172.21.15.165:0/3143022213 conn(0x5592ca91dc60 msgr2=0x5592ca917cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 shutdown_connections 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 1 -- 172.21.15.165:0/3143022213 wait complete. 2024-04-25T10:15:50.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.452+0000 7ff0d658e740 20 client.9399 trim_cache size 0 max 16384 2024-04-25T10:15:50.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.468+0000 7ff0d658e740 1 Processor -- start 2024-04-25T10:15:50.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.468+0000 7ff0d658e740 1 -- start start 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.469+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca947f80 0x5592ca95cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.469+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca965820 0x5592ca9735b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.469+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.469+0000 7ff0d658e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc200ba0 con 0x5592ca9760e0 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.469+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1ffd40 con 0x5592ca947f80 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.469+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1ff290 con 0x5592ca965820 2024-04-25T10:15:50.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35576/0 (socket says 172.21.15.165:35576) 2024-04-25T10:15:50.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 -- 172.21.15.165:0/1402726417 learned_addr learned my addr 172.21.15.165:0/1402726417 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 -- 172.21.15.165:0/1402726417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca965820 msgr2=0x5592ca9735b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca965820 0x5592ca9735b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 -- 172.21.15.165:0/1402726417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca947f80 msgr2=0x5592ca95cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca947f80 0x5592ca95cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.470+0000 7ff0d1800640 1 -- 172.21.15.165:0/1402726417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5592ca52e610 con 0x5592ca9760e0 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.471+0000 7ff0d1800640 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95dcd0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40019b0 tx=0x7ff0c4006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd4b531ec351b352 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.471+0000 7ff0c0c00640 1 -- 172.21.15.165:0/1402726417 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400d930 con 0x5592ca9760e0 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.471+0000 7ff0c0c00640 1 -- 172.21.15.165:0/1402726417 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4004b60 con 0x5592ca9760e0 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.471+0000 7ff0c0c00640 1 -- 172.21.15.165:0/1402726417 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4007350 con 0x5592ca9760e0 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.471+0000 7ff0d658e740 1 -- 172.21.15.165:0/1402726417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 msgr2=0x5592ca95dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.471+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95dcd0 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7ff0c40019b0 tx=0x7ff0c4006000 comp rx=0 tx=0).stop 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 -- 172.21.15.165:0/1402726417 shutdown_connections 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca9760e0 0x5592ca95dcd0 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca965820 0x5592ca9735b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1402726417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca947f80 0x5592ca95cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 -- 172.21.15.165:0/1402726417 >> 172.21.15.165:0/1402726417 conn(0x5592ca91dc60 msgr2=0x5592ca91cd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 -- 172.21.15.165:0/1402726417 shutdown_connections 2024-04-25T10:15:50.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 -- 172.21.15.165:0/1402726417 wait complete. 2024-04-25T10:15:50.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.472+0000 7ff0d658e740 1 -- start start 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 Processor -- start 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca9c8720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 0x5592ca9c2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca9760e0 0x5592ca9c2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5592ca95d6d0 con 0x5592ca965820 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1ff740 con 0x5592ca9760e0 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.473+0000 7ff0d658e740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5592ca975ef0 con 0x5592ca947f80 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0d658e740 10 client.0 _ll_register_callbacks cb 0x5592ca918c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 0x5592ca9c2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 0x5592ca9c2140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35588/0 (socket says 172.21.15.165:35588) 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 -- 172.21.15.165:0/1530546634 learned_addr learned my addr 172.21.15.165:0/1530546634 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 -- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 msgr2=0x5592ca9c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca9c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 -- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca9760e0 msgr2=0x5592ca9c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca9760e0 0x5592ca9c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 -- 172.21.15.165:0/1530546634 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5592ca973bf0 con 0x5592ca965820 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0cbe00640 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 0x5592ca9c2140 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4080720 tx=0x7ff0b4078540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b09aa571af5fa9a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.474+0000 7ff0ba000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0ba000640 1 -- 172.21.15.165:0/1530546634 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b4090020 con 0x5592ca965820 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0ba000640 1 -- 172.21.15.165:0/1530546634 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff0b4076840 con 0x5592ca965820 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0c4004b60 con 0x5592ca965820 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0ba000640 1 -- 172.21.15.165:0/1530546634 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0b402be10 con 0x5592ca965820 2024-04-25T10:15:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 10 client.9405 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x5592ca971830 con 0x5592ca965820 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0ba000640 1 -- 172.21.15.165:0/1530546634 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c4004b60 con 0x5592ca965820 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0ba000640 1 -- 172.21.15.165:0/1530546634 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ff0b403a070 con 0x5592ca965820 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 10 client.9405 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 10 client.9405 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.b 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 20 client.9405 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca9c8c60 0x5592ca9cefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 4 client.9405 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.475+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "scrub start", "path": "/scrub_control_test_path", "scrubops": ["recursive"]}) v1 -- 0x5592ca95ece0 con 0x5592ca9c8c60 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.476+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca9c8c60 0x5592ca9cefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.477+0000 7ff0d0e00640 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca9c8c60 0x5592ca9cefe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a822bc4a6088deb server_cookie=8fa0d2b0504f9764 in_seq=0 out_seq=0 2024-04-25T10:15:50.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.477+0000 7ff0ba000640 10 client.9405 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0ba000640 1 -- 172.21.15.165:0/1530546634 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5592ca95ece0 con 0x5592ca9c8c60 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0ba000640 10 client.9405 handle_command_reply: tid=0 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 client.9405 shutdown 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 msgr2=0x5592ca9c2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 0x5592ca9c2140 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4080720 tx=0x7ff0b4078540 comp rx=0 tx=0).stop 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 shutdown_connections 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x5592ca9c8c60 0x5592ca9cefe0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5592ca9760e0 0x5592ca9c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5592ca965820 0x5592ca9c2140 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 --2- 172.21.15.165:0/1530546634 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5592ca947f80 0x5592ca9c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0ba000640 0 client.9405 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 >> 172.21.15.165:0/1530546634 conn(0x5592ca91dc60 msgr2=0x5592ca95c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 shutdown_connections 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 1 -- 172.21.15.165:0/1530546634 wait complete. 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.483+0000 7ff0d658e740 20 client.9405 trim_cache size 0 max 16384 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "36ce1c07-d7b3-41e8-b95f-48be7776a489", 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:15:50.488 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:15:50.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0a806b4d0 msgr2=0x7ff0a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0a806b4d0 0x7ff0a806d990 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0692f0 tx=0x7ff0bc008040 comp rx=0 tx=0).stop 2024-04-25T10:15:50.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 msgr2=0x7ff0cc14dd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 0x7ff0cc14dd60 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7ff0b402a570 tx=0x7ff0b4001120 comp rx=0 tx=0).stop 2024-04-25T10:15:50.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 shutdown_connections 2024-04-25T10:15:50.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff0a806b4d0 0x7ff0a806d990 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff0cc0cd6d0 0x7ff0cc160310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0cc156e20 0x7ff0cc14dd60 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 --2- 172.21.15.165:0/881001208 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0ac004800 0x7ff0cc1576c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 >> 172.21.15.165:0/881001208 conn(0x7ff0cc0d0610 msgr2=0x7ff0cc0ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 shutdown_connections 2024-04-25T10:15:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.484+0000 7ff0d2200640 1 -- 172.21.15.165:0/881001208 wait complete. 2024-04-25T10:15:50.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:50.506 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub pause 2024-04-25T10:15:50.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.722+0000 7fedf6400640 1 Processor -- start 2024-04-25T10:15:50.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.722+0000 7fedf6400640 1 -- start start 2024-04-25T10:15:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedf6400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fedf01616f0 0x7fedf0161b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedf6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf00ce0a0 0x7fedf00ce530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedf6400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedf006c8d0 con 0x7fedf01610f0 2024-04-25T10:15:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedf6400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedf01614d0 con 0x7fedf01616f0 2024-04-25T10:15:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedf6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00c6960 con 0x7fedf00ce0a0 2024-04-25T10:15:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedef400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fedf01610f0 0x7fedf006c1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36552/0 (socket says 172.21.15.165:36552) 2024-04-25T10:15:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedef400640 1 -- 172.21.15.165:0/1605443572 learned_addr learned my addr 172.21.15.165:0/1605443572 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fede6a00640 1 --2- 172.21.15.165:0/1605443572 >> v2:172.21.15.165:3300/0 conn(0x7fedf01616f0 0x7fedf0161b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.724+0000 7fedefe00640 1 --2- 172.21.15.165:0/1605443572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf00ce0a0 0x7fedf00ce530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.725+0000 7fede6a00640 1 -- 172.21.15.165:0/1605443572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf00ce0a0 msgr2=0x7fedf00ce530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.725+0000 7fede6a00640 1 --2- 172.21.15.165:0/1605443572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf00ce0a0 0x7fedf00ce530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.725+0000 7fede6a00640 1 -- 172.21.15.165:0/1605443572 >> v1:172.21.15.165:6789/0 conn(0x7fedf01610f0 legacy=0x7fedf006c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.725+0000 7fede6a00640 1 -- 172.21.15.165:0/1605443572 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf0162260 con 0x7fedf01616f0 2024-04-25T10:15:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.726+0000 7fede6a00640 1 --2- 172.21.15.165:0/1605443572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf01616f0 0x7fedf0161b50 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fedd80097a0 tx=0x7fedd802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7704de99e554b47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.726+0000 7fedeea00640 1 -- 172.21.15.165:0/1605443572 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8037070 con 0x7fedf01616f0 2024-04-25T10:15:50.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.726+0000 7fedeea00640 1 -- 172.21.15.165:0/1605443572 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8032440 con 0x7fedf01616f0 2024-04-25T10:15:50.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.726+0000 7fedeea00640 1 -- 172.21.15.165:0/1605443572 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8032810 con 0x7fedf01616f0 2024-04-25T10:15:50.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 -- 172.21.15.165:0/1605443572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf01616f0 msgr2=0x7fedf0161b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 --2- 172.21.15.165:0/1605443572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf01616f0 0x7fedf0161b50 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7fedd80097a0 tx=0x7fedd802a180 comp rx=0 tx=0).stop 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 -- 172.21.15.165:0/1605443572 shutdown_connections 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 --2- 172.21.15.165:0/1605443572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf00ce0a0 0x7fedf00ce530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 --2- 172.21.15.165:0/1605443572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf01616f0 0x7fedf0161b50 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 -- 172.21.15.165:0/1605443572 >> 172.21.15.165:0/1605443572 conn(0x7fedf006f500 msgr2=0x7fedf006f910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 -- 172.21.15.165:0/1605443572 shutdown_connections 2024-04-25T10:15:50.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.727+0000 7fedf6400640 1 -- 172.21.15.165:0/1605443572 wait complete. 2024-04-25T10:15:50.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.728+0000 7fedf6400640 1 Processor -- start 2024-04-25T10:15:50.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 -- start start 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 0x7fedf0155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf01610f0 0x7fedf015dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedf01616f0 0x7fedf00b6c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00cfce0 con 0x7fedf00ce0a0 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00cfa00 con 0x7fedf01616f0 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.729+0000 7fedf6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf00cf690 con 0x7fedf01610f0 2024-04-25T10:15:50.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 0x7fedf0155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 0x7fedf0155050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35608/0 (socket says 172.21.15.165:35608) 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 -- 172.21.15.165:0/490286765 learned_addr learned my addr 172.21.15.165:0/490286765 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fede6a00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf01610f0 0x7fedf015dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedefe00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedf01616f0 0x7fedf00b6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 -- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf01610f0 msgr2=0x7fedf015dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf01610f0 0x7fedf015dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 -- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedf01616f0 msgr2=0x7fedf00b6c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedf01616f0 0x7fedf00b6c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedef400640 1 -- 172.21.15.165:0/490286765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf00b5c10 con 0x7fedf00ce0a0 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fede6a00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf01610f0 0x7fedf015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.730+0000 7fedefe00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedf01616f0 0x7fedf00b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.731+0000 7fedef400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 0x7fedf0155050 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fede000c400 tx=0x7fede000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.731+0000 7fede7e00640 1 -- 172.21.15.165:0/490286765 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede0017070 con 0x7fedf00ce0a0 2024-04-25T10:15:50.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.731+0000 7fede7e00640 1 -- 172.21.15.165:0/490286765 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede0005ce0 con 0x7fedf00ce0a0 2024-04-25T10:15:50.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.731+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf00b5ed0 con 0x7fedf00ce0a0 2024-04-25T10:15:50.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.731+0000 7fede7e00640 1 -- 172.21.15.165:0/490286765 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede00072f0 con 0x7fedf00ce0a0 2024-04-25T10:15:50.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.731+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedf00b6490 con 0x7fedf00ce0a0 2024-04-25T10:15:50.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.733+0000 7fede7e00640 1 -- 172.21.15.165:0/490286765 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fede001a070 con 0x7fedf00ce0a0 2024-04-25T10:15:50.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.734+0000 7fede7e00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedcc06b4d0 0x7fedcc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.734+0000 7fede7e00640 1 -- 172.21.15.165:0/490286765 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fede0089870 con 0x7fedf00ce0a0 2024-04-25T10:15:50.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.735+0000 7fede6a00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedcc06b4d0 0x7fedcc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.735+0000 7fedf9266740 1 Processor -- start 2024-04-25T10:15:50.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.735+0000 7fedf9266740 1 -- start start 2024-04-25T10:15:50.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.735+0000 7fede6a00640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedcc06b4d0 0x7fedcc06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fedd8034000 tx=0x7fedd802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff4690 0x564b01ff4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff50b0 0x564b01fc4890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedf9266740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 0x564b01fecb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedf9266740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b01f2d870 con 0x564b01fc4dd0 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b01ed6d60 con 0x564b01ff4690 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b01f29910 con 0x564b01ff50b0 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 0x564b01fecb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 0x564b01fecb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35620/0 (socket says 172.21.15.165:35620) 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.736+0000 7fedefe00640 1 -- 172.21.15.165:0/119216511 learned_addr learned my addr 172.21.15.165:0/119216511 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fede6a00640 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff50b0 0x564b01fc4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedef400640 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff4690 0x564b01ff4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedefe00640 1 -- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff50b0 msgr2=0x564b01fc4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedefe00640 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff50b0 0x564b01fc4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedefe00640 1 -- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff4690 msgr2=0x564b01ff4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedefe00640 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff4690 0x564b01ff4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedefe00640 1 -- 172.21.15.165:0/119216511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b01d0c7d0 con 0x564b01fc4dd0 2024-04-25T10:15:50.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7fedefe00640 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 0x564b01fecb10 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fede80073e0 tx=0x7fede80078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fdbbdeca7b58fd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7feddfe00640 1 -- 172.21.15.165:0/119216511 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede801a070 con 0x564b01fc4dd0 2024-04-25T10:15:50.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7feddfe00640 1 -- 172.21.15.165:0/119216511 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8003da0 con 0x564b01fc4dd0 2024-04-25T10:15:50.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.737+0000 7feddfe00640 1 -- 172.21.15.165:0/119216511 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8004230 con 0x564b01fc4dd0 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 -- 172.21.15.165:0/119216511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 msgr2=0x564b01fecb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 0x564b01fecb10 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fede80073e0 tx=0x7fede80078a0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 -- 172.21.15.165:0/119216511 shutdown_connections 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01fc4dd0 0x564b01fecb10 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff50b0 0x564b01fc4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 --2- 172.21.15.165:0/119216511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff4690 0x564b01ff4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 -- 172.21.15.165:0/119216511 >> 172.21.15.165:0/119216511 conn(0x564b01fc6d10 msgr2=0x564b0200f4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 -- 172.21.15.165:0/119216511 shutdown_connections 2024-04-25T10:15:50.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.738+0000 7fedf9266740 1 -- 172.21.15.165:0/119216511 wait complete. 2024-04-25T10:15:50.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.739+0000 7fedf9266740 1 -- start start 2024-04-25T10:15:50.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.739+0000 7fedf9266740 1 Processor -- start 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b01ff6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b01ff72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 0x564b01ff3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b01ff0de0 con 0x564b01febc00 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b01eea1c0 con 0x564b01ff50b0 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b01ff1080 con 0x564b01ff4690 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.740+0000 7fedf9266740 10 client.0 _ll_register_callbacks cb 0x564b01f288f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:50.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 0x564b01ff3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 0x564b01ff3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35628/0 (socket says 172.21.15.165:35628) 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 -- 172.21.15.165:0/2995406944 learned_addr learned my addr 172.21.15.165:0/2995406944 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 -- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 msgr2=0x564b01ff6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedef400640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b01ff6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fede6a00640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b01ff72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b01ff6d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 -- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 msgr2=0x564b01ff72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b01ff72d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 -- 172.21.15.165:0/2995406944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b01c020f0 con 0x564b01febc00 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7fedefe00640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 0x564b01ff3080 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fede8014000 tx=0x7fede8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f366d739fcbd7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.741+0000 7feddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7feddcc00640 1 -- 172.21.15.165:0/2995406944 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede801a070 con 0x564b01febc00 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7feddcc00640 1 -- 172.21.15.165:0/2995406944 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fede8004650 con 0x564b01febc00 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fede8003da0 con 0x564b01febc00 2024-04-25T10:15:50.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7feddcc00640 1 -- 172.21.15.165:0/2995406944 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede802d050 con 0x564b01febc00 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 10 client.9420 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x564b01d6cf60 con 0x564b01febc00 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7feddcc00640 1 -- 172.21.15.165:0/2995406944 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fede8003da0 con 0x564b01febc00 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7feddcc00640 1 -- 172.21.15.165:0/2995406944 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fede8030020 con 0x564b01febc00 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 10 client.9420 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 10 client.9420 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.b 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 20 client.9420 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b01ff5980 0x564b01ff5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 4 client.9420 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.742+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564b01f03690 con 0x564b01ff5980 2024-04-25T10:15:50.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.743+0000 7fedef400640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b01ff5980 0x564b01ff5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.744+0000 7fedef400640 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b01ff5980 0x564b01ff5d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0852e3b4d2ae405 server_cookie=2a79d95fa17562f9 in_seq=0 out_seq=0 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.744+0000 7feddcc00640 10 client.9420 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7feddcc00640 1 -- 172.21.15.165:0/2995406944 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x564b01f03690 con 0x564b01ff5980 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7feddcc00640 10 client.9420 handle_command_reply: tid=0 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 client.9420 shutdown 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 msgr2=0x564b01ff3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 0x564b01ff3080 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7fede8014000 tx=0x7fede8004ab0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 shutdown_connections 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01febc00 0x564b01ff3080 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b01ff5980 0x564b01ff5d60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b01ff72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 --2- 172.21.15.165:0/2995406944 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b01ff6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 >> 172.21.15.165:0/2995406944 conn(0x564b01fc4dd0 msgr2=0x564b02013240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7feddcc00640 0 client.9420 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.745+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 shutdown_connections 2024-04-25T10:15:50.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.746+0000 7fedf9266740 1 -- 172.21.15.165:0/2995406944 wait complete. 2024-04-25T10:15:50.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.746+0000 7fedf9266740 20 client.9420 trim_cache size 0 max 16384 2024-04-25T10:15:50.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.762+0000 7fedf9266740 1 Processor -- start 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.762+0000 7fedf9266740 1 -- start start 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.762+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01febc00 0x564b01ff72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.762+0000 7fedf9266740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 0x564b02017230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b02017770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fedf9266740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x564b01ff4de0 con 0x564b01ff4690 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0200580 con 0x564b01ff50b0 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedf015a5c0 con 0x564b01febc00 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 0x564b02017230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 0x564b02017230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35638/0 (socket says 172.21.15.165:35638) 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 -- 172.21.15.165:0/1631108086 learned_addr learned my addr 172.21.15.165:0/1631108086 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fedef400640 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01febc00 0x564b01ff72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 -- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01febc00 msgr2=0x564b01ff72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01febc00 0x564b01ff72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 -- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 msgr2=0x564b02017770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b02017770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.763+0000 7fede6a00640 1 -- 172.21.15.165:0/1631108086 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564b0200e610 con 0x564b01ff4690 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fede6a00640 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 0x564b02017230 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fedd802a770 tx=0x7fedd8008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67894c022f96b106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7feddf400640 1 -- 172.21.15.165:0/1631108086 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd804e070 con 0x564b01ff4690 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7feddf400640 1 -- 172.21.15.165:0/1631108086 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8049660 con 0x564b01ff4690 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7feddf400640 1 -- 172.21.15.165:0/1631108086 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedd8037040 con 0x564b01ff4690 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 -- 172.21.15.165:0/1631108086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 msgr2=0x564b02017230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 0x564b02017230 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7fedd802a770 tx=0x7fedd8008030 comp rx=0 tx=0).stop 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 -- 172.21.15.165:0/1631108086 shutdown_connections 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01ff50b0 0x564b02017770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff4690 0x564b02017230 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 --2- 172.21.15.165:0/1631108086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01febc00 0x564b01ff72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.764+0000 7fedf9266740 1 -- 172.21.15.165:0/1631108086 >> 172.21.15.165:0/1631108086 conn(0x564b01fc4dd0 msgr2=0x564b01ff6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.765+0000 7fedf9266740 1 -- 172.21.15.165:0/1631108086 shutdown_connections 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.765+0000 7fedf9266740 1 -- 172.21.15.165:0/1631108086 wait complete. 2024-04-25T10:15:50.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.765+0000 7fedf9266740 1 -- start start 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.765+0000 7fedf9266740 1 Processor -- start 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.766+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01febc00 0x564b0205fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.766+0000 7fedf9266740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b02004e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.766+0000 7fedf9266740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 0x564b01feabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.766+0000 7fedf9266740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf0159fc0 con 0x564b01ff50b0 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.766+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedf01599c0 con 0x564b01febc00 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.766+0000 7fedf9266740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x564b01fec400 con 0x564b01ff4690 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedf9266740 10 client.0 _ll_register_callbacks cb 0x7fede80098a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 0x564b01feabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 0x564b01feabd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35652/0 (socket says 172.21.15.165:35652) 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 -- 172.21.15.165:0/1122863651 learned_addr learned my addr 172.21.15.165:0/1122863651 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 -- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 msgr2=0x564b02004e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b02004e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 -- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01febc00 msgr2=0x564b0205fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01febc00 0x564b0205fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 -- 172.21.15.165:0/1122863651 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564b01bdb1b0 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7fedefe00640 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 0x564b01feabd0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fede80078e0 tx=0x7fede8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72c3ce395f22c716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.767+0000 7feddcc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7feddcc00640 1 -- 172.21.15.165:0/1122863651 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fede8035020 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7feddcc00640 1 -- 172.21.15.165:0/1122863651 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fede80044b0 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fedd8049660 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7feddcc00640 1 -- 172.21.15.165:0/1122863651 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7fede803c050 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 10 client.9426 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x564b0207de60 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7feddcc00640 1 -- 172.21.15.165:0/1122863651 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedd8049660 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7feddcc00640 1 -- 172.21.15.165:0/1122863651 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fede803f020 con 0x564b01ff50b0 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 10 client.9426 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 10 client.9426 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.b 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 20 client.9426 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:50.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b02015300 0x564b02015710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 4 client.9426 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.768+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x564b01f3adb0 con 0x564b02015300 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.769+0000 7fedef400640 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b02015300 0x564b02015710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7fedef400640 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b02015300 0x564b02015710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd2f594c0a6516ed server_cookie=9369ef60d22037cb in_seq=0 out_seq=0 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7feddcc00640 10 client.9426 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7feddcc00640 1 -- 172.21.15.165:0/1122863651 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+25 (crc 0 0 0) 0x564b01f3adb0 con 0x564b02015300 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7feddcc00640 10 client.9426 handle_command_reply: tid=0 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7fedf9266740 1 client.9426 shutdown 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 msgr2=0x564b01feabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.770+0000 7fedf9266740 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 0x564b01feabd0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7fede80078e0 tx=0x7fede8004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 shutdown_connections 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x564b01ff50b0 0x564b01feabd0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x564b01ff4690 0x564b02004e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x564b02015300 0x564b02015710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 --2- 172.21.15.165:0/1122863651 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x564b01febc00 0x564b0205fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 >> 172.21.15.165:0/1122863651 conn(0x564b01fc4dd0 msgr2=0x564b01ff00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7feddcc00640 0 client.9426 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 shutdown_connections 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 1 -- 172.21.15.165:0/1122863651 wait complete. 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf9266740 20 client.9426 trim_cache size 0 max 16384 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0 2024-04-25T10:15:50.781 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:15:50.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedcc06b4d0 msgr2=0x7fedcc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf6400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedcc06b4d0 0x7fedcc06d990 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fedd8034000 tx=0x7fedd802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 msgr2=0x7fedf0155050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.771+0000 7fedf6400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 0x7fedf0155050 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fede000c400 tx=0x7fede000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 shutdown_connections 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fedcc06b4d0 0x7fedcc06d990 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fedf01616f0 0x7fedf00b6c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fedf01610f0 0x7fedf015dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 --2- 172.21.15.165:0/490286765 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fedf00ce0a0 0x7fedf0155050 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 >> 172.21.15.165:0/490286765 conn(0x7fedf006f500 msgr2=0x7fedf00c6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 shutdown_connections 2024-04-25T10:15:50.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.772+0000 7fedf6400640 1 -- 172.21.15.165:0/490286765 wait complete. 2024-04-25T10:15:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:15:50.792 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.9:0 scrub status 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.978+0000 7f06a7200640 1 Processor -- start 2024-04-25T10:15:50.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.979+0000 7f06a7200640 1 -- start start 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.980+0000 7f06a7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f06a806ebf0 0x7f06a80726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f06a806c300 0x7f06a80b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a7200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06a815e710 con 0x7f06a815dfa0 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a815e9f0 con 0x7f06a806ebf0 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06a815ecd0 con 0x7f06a806c300 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f06a806ebf0 0x7f06a80726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f06a806ebf0 0x7f06a80726c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35660/0 (socket says 172.21.15.165:35660) 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a5e00640 1 -- 172.21.15.165:0/3409947479 learned_addr learned my addr 172.21.15.165:0/3409947479 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a5400640 1 --1- 172.21.15.165:0/3409947479 >> v1:172.21.15.165:6789/0 conn(0x7f06a815dfa0 0x7f06a806bbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36562/0 (socket says 172.21.15.165:36562) 2024-04-25T10:15:50.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a5e00640 1 -- 172.21.15.165:0/3409947479 >> v2:172.21.15.172:3300/0 conn(0x7f06a806c300 msgr2=0x7f06a80b3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.981+0000 7f06a6800640 1 --2- 172.21.15.165:0/3409947479 >> v2:172.21.15.172:3300/0 conn(0x7f06a806c300 0x7f06a80b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.982+0000 7f06a5e00640 1 --2- 172.21.15.165:0/3409947479 >> v2:172.21.15.172:3300/0 conn(0x7f06a806c300 0x7f06a80b3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.982+0000 7f06a5e00640 1 -- 172.21.15.165:0/3409947479 >> v1:172.21.15.165:6789/0 conn(0x7f06a815dfa0 legacy=0x7f06a806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.982+0000 7f06a4a00640 1 -- 172.21.15.165:0/3409947479 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199877650 0 0) 0x7f06a815e710 con 0x7f06a815dfa0 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.982+0000 7f06a5e00640 1 -- 172.21.15.165:0/3409947479 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a80b2f20 con 0x7f06a806ebf0 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.982+0000 7f06a6800640 1 --2- 172.21.15.165:0/3409947479 >> v2:172.21.15.172:3300/0 conn(0x7f06a806c300 0x7f06a80b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a5e00640 1 --2- 172.21.15.165:0/3409947479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a806ebf0 0x7f06a80726c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b4f90 tx=0x7f069802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=349f5419be1856cb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a4a00640 1 -- 172.21.15.165:0/3409947479 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0698036070 con 0x7f06a806ebf0 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a4a00640 1 -- 172.21.15.165:0/3409947479 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0698031600 con 0x7f06a806ebf0 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a4a00640 1 -- 172.21.15.165:0/3409947479 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06980319d0 con 0x7f06a806ebf0 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a7200640 1 -- 172.21.15.165:0/3409947479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a806ebf0 msgr2=0x7f06a80726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a7200640 1 --2- 172.21.15.165:0/3409947479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a806ebf0 0x7f06a80726c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f06a80b4f90 tx=0x7f069802a730 comp rx=0 tx=0).stop 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a7200640 1 -- 172.21.15.165:0/3409947479 shutdown_connections 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a7200640 1 --2- 172.21.15.165:0/3409947479 >> v2:172.21.15.172:3300/0 conn(0x7f06a806c300 0x7f06a80b3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a7200640 1 --2- 172.21.15.165:0/3409947479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a806ebf0 0x7f06a80726c0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.983+0000 7f06a7200640 1 -- 172.21.15.165:0/3409947479 >> 172.21.15.165:0/3409947479 conn(0x7f06a8071af0 msgr2=0x7f06a8071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.984+0000 7f06a7200640 1 -- 172.21.15.165:0/3409947479 shutdown_connections 2024-04-25T10:15:50.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.984+0000 7f06a7200640 1 -- 172.21.15.165:0/3409947479 wait complete. 2024-04-25T10:15:50.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.985+0000 7f06a7200640 1 Processor -- start 2024-04-25T10:15:50.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.985+0000 7f06a7200640 1 -- start start 2024-04-25T10:15:50.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.986+0000 7f06a7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06a806c300 0x7f06a814b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.986+0000 7f06a7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06a806ebf0 0x7f06a80cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.986+0000 7f06a7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 0x7f06a80b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:50.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.986+0000 7f06a7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a806f150 con 0x7f06a815dfa0 2024-04-25T10:15:50.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.986+0000 7f06a7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a806efd0 con 0x7f06a806c300 2024-04-25T10:15:50.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.986+0000 7f06a7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a80b3980 con 0x7f06a806ebf0 2024-04-25T10:15:50.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 0x7f06a80b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 0x7f06a80b0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35662/0 (socket says 172.21.15.165:35662) 2024-04-25T10:15:50.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 -- 172.21.15.165:0/1533763039 learned_addr learned my addr 172.21.15.165:0/1533763039 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:50.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a5e00640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06a806c300 0x7f06a814b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a5400640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06a806ebf0 0x7f06a80cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:50.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 -- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06a806ebf0 msgr2=0x7f06a80cc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06a806ebf0 0x7f06a80cc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 -- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06a806c300 msgr2=0x7f06a814b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:50.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06a806c300 0x7f06a814b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:50.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.987+0000 7f06a6800640 1 -- 172.21.15.165:0/1533763039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a81f7a90 con 0x7f06a815dfa0 2024-04-25T10:15:50.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.988+0000 7f06a6800640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 0x7f06a80b0be0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c7f0 tx=0x7f06a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:50.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.988+0000 7f069e000640 1 -- 172.21.15.165:0/1533763039 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0015070 con 0x7f06a815dfa0 2024-04-25T10:15:50.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.988+0000 7f069e000640 1 -- 172.21.15.165:0/1533763039 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00077c0 con 0x7f06a815dfa0 2024-04-25T10:15:50.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.988+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a815a020 con 0x7f06a815dfa0 2024-04-25T10:15:50.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.988+0000 7f069e000640 1 -- 172.21.15.165:0/1533763039 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0007bf0 con 0x7f06a815dfa0 2024-04-25T10:15:50.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.988+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06a815a5e0 con 0x7f06a815dfa0 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.991+0000 7f069e000640 1 -- 172.21.15.165:0/1533763039 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f06a0018070 con 0x7f06a815dfa0 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.992+0000 7f069e000640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.992+0000 7f069e000640 1 -- 172.21.15.165:0/1533763039 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f06a008a4f0 con 0x7f06a815dfa0 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.992+0000 7f06a5e00640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.993+0000 7f06af97d740 1 Processor -- start 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.993+0000 7f06af97d740 1 -- start start 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.993+0000 7f06a5e00640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f068806b4d0 0x7f068806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0698033000 tx=0x7f0698009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5c6150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06af97d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b5f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06af97d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193b529c70 con 0x56193b5f0830 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193b4d2cf0 con 0x56193b5c5d70 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56193b525d10 con 0x56193b602440 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b5f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b5f0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35668/0 (socket says 172.21.15.165:35668) 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 -- 172.21.15.165:0/3091123775 learned_addr learned my addr 172.21.15.165:0/3091123775 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 -- 172.21.15.165:0/3091123775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 msgr2=0x56193b5c6150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 -- 172.21.15.165:0/3091123775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 msgr2=0x56193b5c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.994+0000 7f06a6800640 1 -- 172.21.15.165:0/3091123775 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56193b3087d0 con 0x56193b5f0830 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.995+0000 7f06a6800640 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b5f0cb0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f06a0088730 tx=0x7f06a005cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6202d0abc59ac78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.995+0000 7f0692600640 1 -- 172.21.15.165:0/3091123775 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a001f040 con 0x56193b5f0830 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.995+0000 7f0692600640 1 -- 172.21.15.165:0/3091123775 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00550b0 con 0x56193b5f0830 2024-04-25T10:15:51.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.995+0000 7f0692600640 1 -- 172.21.15.165:0/3091123775 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0057e90 con 0x56193b5f0830 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 -- 172.21.15.165:0/3091123775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 msgr2=0x56193b5f0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b5f0cb0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7f06a0088730 tx=0x7f06a005cb90 comp rx=0 tx=0).stop 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 -- 172.21.15.165:0/3091123775 shutdown_connections 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b5f0cb0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5c6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 --2- 172.21.15.165:0/3091123775 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 -- 172.21.15.165:0/3091123775 >> 172.21.15.165:0/3091123775 conn(0x56193b58da30 msgr2=0x56193b60b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 -- 172.21.15.165:0/3091123775 shutdown_connections 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.996+0000 7f06af97d740 1 -- 172.21.15.165:0/3091123775 wait complete. 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.997+0000 7f06af97d740 1 -- start start 2024-04-25T10:15:51.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.997+0000 7f06af97d740 1 Processor -- start 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b6049c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b61c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193b4c83a0 con 0x56193b5f0830 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193b5ee380 con 0x56193b5c5d70 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56193b61d420 con 0x56193b602440 2024-04-25T10:15:51.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06af97d740 10 client.0 _ll_register_callbacks cb 0x56193b4e5b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b6049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b6049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35678/0 (socket says 172.21.15.165:35678) 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06a5400640 1 -- 172.21.15.165:0/3049363355 learned_addr learned my addr 172.21.15.165:0/3049363355 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06a5e00640 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.998+0000 7f06a5400640 1 -- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 msgr2=0x56193b61c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06a5400640 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b61c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06a5400640 1 -- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 msgr2=0x56193b5e22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06a5400640 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e22f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06a5400640 1 -- 172.21.15.165:0/3049363355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56193b23e3f0 con 0x56193b5f0830 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06a5400640 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b6049c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f06940010c0 tx=0x7f0694001520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=824d3768adaecee8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f0686a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:51.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f0686a00640 1 -- 172.21.15.165:0/3049363355 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694019060 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f0686a00640 1 -- 172.21.15.165:0/3049363355 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0694014480 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06a00550b0 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f0686a00640 1 -- 172.21.15.165:0/3049363355 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f069401f050 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06af97d740 10 client.9441 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:50.999+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x56193b1fe0f0 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f0686a00640 1 -- 172.21.15.165:0/3049363355 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a00550b0 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f0686a00640 1 -- 172.21.15.165:0/3049363355 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0694022020 con 0x56193b5f0830 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f06af97d740 10 client.9441 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f06af97d740 10 client.9441 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.b 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f06af97d740 20 client.9441 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f06af97d740 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e7a80 0x56193b5eeef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f06af97d740 4 client.9441 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.000+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56193b5371b0 con 0x56193b5e7a80 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.001+0000 7f06a5e00640 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e7a80 0x56193b5eeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.002+0000 7f06a5e00640 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e7a80 0x56193b5eeef0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cea070ca541b9e79 server_cookie=21ddee405b124489 in_seq=0 out_seq=0 2024-04-25T10:15:51.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.002+0000 7f0686a00640 10 client.9441 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:51.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.003+0000 7f0686a00640 1 -- 172.21.15.165:0/3049363355 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x56193b5371b0 con 0x56193b5e7a80 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.003+0000 7f0686a00640 10 client.9441 handle_command_reply: tid=0 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.003+0000 7f06af97d740 1 client.9441 shutdown 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.003+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 msgr2=0x56193b6049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.003+0000 7f06af97d740 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b6049c0 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f06940010c0 tx=0x7f0694001520 comp rx=0 tx=0).stop 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.003+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 shutdown_connections 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b61c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b6049c0 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e7a80 0x56193b5eeef0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 --2- 172.21.15.165:0/3049363355 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 >> 172.21.15.165:0/3049363355 conn(0x56193b58da30 msgr2=0x56193b60edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f0686a00640 0 client.9441 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 shutdown_connections 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 1 -- 172.21.15.165:0/3049363355 wait complete. 2024-04-25T10:15:51.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.004+0000 7f06af97d740 20 client.9441 trim_cache size 0 max 16384 2024-04-25T10:15:51.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.020+0000 7f06af97d740 1 Processor -- start 2024-04-25T10:15:51.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.020+0000 7f06af97d740 1 -- start start 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.021+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b5c5d70 0x56193b61bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.021+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5f0830 0x56193b624c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.021+0000 7f06af97d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 0x56193b625160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.021+0000 7f06af97d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8200050 con 0x56193b602440 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.021+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81ff230 con 0x56193b5f0830 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.021+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06a8156c90 con 0x56193b5c5d70 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 0x56193b625160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 0x56193b625160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35680/0 (socket says 172.21.15.165:35680) 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 -- 172.21.15.165:0/643514754 learned_addr learned my addr 172.21.15.165:0/643514754 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a5e00640 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b5c5d70 0x56193b61bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 -- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b5c5d70 msgr2=0x56193b61bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b5c5d70 0x56193b61bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 -- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5f0830 msgr2=0x56193b624c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a5400640 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5f0830 0x56193b624c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5f0830 0x56193b624c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 -- 172.21.15.165:0/643514754 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56193b5bc9c0 con 0x56193b602440 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.022+0000 7f06a6800640 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 0x56193b625160 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f06a0088730 tx=0x7f06a001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f317ec6582e61154 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f0691c00640 1 -- 172.21.15.165:0/643514754 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0075070 con 0x56193b602440 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f0691c00640 1 -- 172.21.15.165:0/643514754 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0086070 con 0x56193b602440 2024-04-25T10:15:51.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f0691c00640 1 -- 172.21.15.165:0/643514754 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a001f040 con 0x56193b602440 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 -- 172.21.15.165:0/643514754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 msgr2=0x56193b625160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 0x56193b625160 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f06a0088730 tx=0x7f06a001b000 comp rx=0 tx=0).stop 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 -- 172.21.15.165:0/643514754 shutdown_connections 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b602440 0x56193b625160 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5f0830 0x56193b624c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 --2- 172.21.15.165:0/643514754 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b5c5d70 0x56193b61bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 -- 172.21.15.165:0/643514754 >> 172.21.15.165:0/643514754 conn(0x56193b58da30 msgr2=0x56193b5c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 -- 172.21.15.165:0/643514754 shutdown_connections 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.023+0000 7f06af97d740 1 -- 172.21.15.165:0/643514754 wait complete. 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.024+0000 7f06af97d740 1 -- start start 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.024+0000 7f06af97d740 1 Processor -- start 2024-04-25T10:15:51.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e6f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b602820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5eee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x56193b5e2e90 con 0x56193b5f0830 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06a81feed0 con 0x56193b5c5d70 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x56193b61d7d0 con 0x56193b602440 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.025+0000 7f06af97d740 10 client.0 _ll_register_callbacks cb 0x7f0694007da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b602820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b602820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35694/0 (socket says 172.21.15.165:35694) 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 -- 172.21.15.165:0/596618694 learned_addr learned my addr 172.21.15.165:0/596618694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a6800640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5eee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5e00640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 -- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 msgr2=0x56193b5eee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5eee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 -- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 msgr2=0x56193b5e6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e6f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 -- 172.21.15.165:0/596618694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56193b625100 con 0x56193b5f0830 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f06a5400640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b602820 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0694001eb0 tx=0x7f0694007ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c69522cf37ef5905 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.026+0000 7f0686a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f0686a00640 1 -- 172.21.15.165:0/596618694 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0694014a50 con 0x56193b5f0830 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f0686a00640 1 -- 172.21.15.165:0/596618694 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0694014bf0 con 0x56193b5f0830 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06a0086070 con 0x56193b5f0830 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f0686a00640 1 -- 172.21.15.165:0/596618694 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=90) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0694030050 con 0x56193b5f0830 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 10 client.9447 fetch_fsmap learned FSMap version 90 2024-04-25T10:15:51.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=90,monmap=2+}) v3 -- 0x56193b5e6740 con 0x56193b5f0830 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f0686a00640 1 -- 172.21.15.165:0/596618694 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a0086070 con 0x56193b5f0830 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f0686a00640 1 -- 172.21.15.165:0/596618694 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 90) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0694033020 con 0x56193b5f0830 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 10 client.9447 fetch_fsmap finished waiting for FSMap version 90 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 10 client.9447 resolve_mds: resolved 9:0 to role '9:0' aka daemon mds.b 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 20 client.9447 populate_metadata read hostname 'smithi165' 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e10d0 0x56193b5ef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 4 client.9447 mds_command: new command op to 9234 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.027+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 --> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56193b531500 con 0x56193b5e10d0 2024-04-25T10:15:51.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.028+0000 7f06a5e00640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e10d0 0x56193b5ef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.029+0000 7f06a5e00640 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e10d0 0x56193b5ef370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41ea9b721b5d8f2a server_cookie=db40acc979208022 in_seq=0 out_seq=0 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.029+0000 7f0686a00640 10 client.9447 ms_handle_connect on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f0686a00640 1 -- 172.21.15.165:0/596618694 <== mds.0 v2:172.21.15.172:6834/2186062364 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+276 (crc 0 0 0) 0x56193b531500 con 0x56193b5e10d0 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f0686a00640 10 client.9447 handle_command_reply: tid=0 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 client.9447 shutdown 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 msgr2=0x56193b602820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b602820 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f0694001eb0 tx=0x7f0694007ac0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 shutdown_connections 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x56193b602440 0x56193b5eee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x56193b5f0830 0x56193b602820 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:6834/2186062364,v1:172.21.15.172:6835/2186062364] conn(0x56193b5e10d0 0x56193b5ef370 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 --2- 172.21.15.165:0/596618694 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x56193b5c5d70 0x56193b5e6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 >> 172.21.15.165:0/596618694 conn(0x56193b58da30 msgr2=0x56193b5f3f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f0686a00640 0 client.9447 ms_handle_reset on v2:172.21.15.172:6834/2186062364 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 shutdown_connections 2024-04-25T10:15:51.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 1 -- 172.21.15.165:0/596618694 wait complete. 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.030+0000 7f06af97d740 20 client.9447 trim_cache size 0 max 16384 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "PAUSED (0 inodes in the stack)", 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: "36ce1c07-d7b3-41e8-b95f-48be7776a489": { 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/scrub_control_test_path", 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "36ce1c07-d7b3-41e8-b95f-48be7776a489", 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive" 2024-04-25T10:15:51.038 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:51.039 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:15:51.039 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f068806b4d0 msgr2=0x7f068806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f068806b4d0 0x7f068806d990 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f0698033000 tx=0x7f0698009000 comp rx=0 tx=0).stop 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 msgr2=0x7f06a80b0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 0x7f06a80b0be0 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f06a000c7f0 tx=0x7f06a000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 shutdown_connections 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f068806b4d0 0x7f068806d990 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f06a815dfa0 0x7f06a80b0be0 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f06a806ebf0 0x7f06a80cc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 --2- 172.21.15.165:0/1533763039 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f06a806c300 0x7f06a814b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 >> 172.21.15.165:0/1533763039 conn(0x7f06a8071af0 msgr2=0x7f06a815ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 shutdown_connections 2024-04-25T10:15:51.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.031+0000 7f06a7200640 1 -- 172.21.15.165:0/1533763039 wait complete. 2024-04-25T10:15:51.051 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:51.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.245+0000 7fe167400640 1 Processor -- start 2024-04-25T10:15:51.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.246+0000 7fe167400640 1 -- start start 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe167400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe160164970 0x7fe160166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe167400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe160144b30 con 0x7fe160144750 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe167400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1600712d0 con 0x7fe1600ce920 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe167400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1600715b0 con 0x7fe160164970 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe165400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe160164970 0x7fe160166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe164a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe160144750 0x7fe16006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36566/0 (socket says 172.21.15.165:36566) 2024-04-25T10:15:51.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.247+0000 7fe164a00640 1 -- 172.21.15.165:0/120239945 learned_addr learned my addr 172.21.15.165:0/120239945 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:51.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.248+0000 7fe165400640 1 -- 172.21.15.165:0/120239945 >> v1:172.21.15.172:6789/0 conn(0x7fe1600ce920 legacy=0x7fe1600cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.248+0000 7fe165400640 1 -- 172.21.15.165:0/120239945 >> v1:172.21.15.165:6789/0 conn(0x7fe160144750 legacy=0x7fe16006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.248+0000 7fe165400640 1 -- 172.21.15.165:0/120239945 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe160071890 con 0x7fe160164970 2024-04-25T10:15:51.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.249+0000 7fe165400640 1 --2- 172.21.15.165:0/120239945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe160166d60 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fe158009d90 tx=0x7fe15802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f27d34342a0b8c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe15f400640 1 -- 172.21.15.165:0/120239945 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158007740 con 0x7fe160164970 2024-04-25T10:15:51.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe15f400640 1 -- 172.21.15.165:0/120239945 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1580078e0 con 0x7fe160164970 2024-04-25T10:15:51.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe15f400640 1 -- 172.21.15.165:0/120239945 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158007cb0 con 0x7fe160164970 2024-04-25T10:15:51.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe167400640 1 -- 172.21.15.165:0/120239945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 msgr2=0x7fe160166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe167400640 1 --2- 172.21.15.165:0/120239945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe160166d60 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7fe158009d90 tx=0x7fe15802a660 comp rx=0 tx=0).stop 2024-04-25T10:15:51.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe167400640 1 -- 172.21.15.165:0/120239945 shutdown_connections 2024-04-25T10:15:51.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.250+0000 7fe167400640 1 --2- 172.21.15.165:0/120239945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe160166d60 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.251+0000 7fe167400640 1 -- 172.21.15.165:0/120239945 >> 172.21.15.165:0/120239945 conn(0x7fe16006abe0 msgr2=0x7fe16006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.251+0000 7fe167400640 1 -- 172.21.15.165:0/120239945 shutdown_connections 2024-04-25T10:15:51.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.251+0000 7fe167400640 1 -- 172.21.15.165:0/120239945 wait complete. 2024-04-25T10:15:51.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.252+0000 7fe167400640 1 Processor -- start 2024-04-25T10:15:51.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.252+0000 7fe167400640 1 -- start start 2024-04-25T10:15:51.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.253+0000 7fe167400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600ce920 0x7fe160154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.253+0000 7fe167400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe160144750 0x7fe1601555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.253+0000 7fe167400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe16013d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.253+0000 7fe167400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe160145450 con 0x7fe160164970 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.253+0000 7fe167400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16006f020 con 0x7fe1600ce920 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.253+0000 7fe167400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1601455d0 con 0x7fe160144750 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe16013d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe16013d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35722/0 (socket says 172.21.15.165:35722) 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 -- 172.21.15.165:0/3349148690 learned_addr learned my addr 172.21.15.165:0/3349148690 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe164a00640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600ce920 0x7fe160154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 -- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe160144750 msgr2=0x7fe1601555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe160144750 0x7fe1601555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 -- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600ce920 msgr2=0x7fe160154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600ce920 0x7fe160154e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 -- 172.21.15.165:0/3349148690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1601388e0 con 0x7fe160164970 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.254+0000 7fe165400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe16013d750 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fe158009eb0 tx=0x7fe158004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.255+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158040070 con 0x7fe160164970 2024-04-25T10:15:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.255+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe158031c90 con 0x7fe160164970 2024-04-25T10:15:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.255+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe158045680 con 0x7fe160164970 2024-04-25T10:15:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.255+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe160138bc0 con 0x7fe160164970 2024-04-25T10:15:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.255+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe160139180 con 0x7fe160164970 2024-04-25T10:15:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.256+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe160073990 con 0x7fe160164970 2024-04-25T10:15:51.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.257+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe158039040 con 0x7fe160164970 2024-04-25T10:15:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.258+0000 7fe15cc00640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe13c06b4d0 0x7fe13c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.258+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe1580abea0 con 0x7fe160164970 2024-04-25T10:15:51.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.258+0000 7fe164a00640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe13c06b4d0 0x7fe13c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.259+0000 7fe164a00640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe13c06b4d0 0x7fe13c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe150005880 tx=0x7fe150009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.262+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe158043360 con 0x7fe160164970 2024-04-25T10:15:51.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.501+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe160139610 con 0x7fe160164970 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.503+0000 7fe15cc00640 1 -- 172.21.15.165:0/3349148690 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fe15807ac40 con 0x7fe160164970 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:51.506 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:51.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.505+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe13c06b4d0 msgr2=0x7fe13c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.505+0000 7fe167400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe13c06b4d0 0x7fe13c06d990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fe150005880 tx=0x7fe150009040 comp rx=0 tx=0).stop 2024-04-25T10:15:51.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 msgr2=0x7fe16013d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe16013d750 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7fe158009eb0 tx=0x7fe158004a40 comp rx=0 tx=0).stop 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 shutdown_connections 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe13c06b4d0 0x7fe13c06d990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe160164970 0x7fe16013d750 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe160144750 0x7fe1601555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 --2- 172.21.15.165:0/3349148690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1600ce920 0x7fe160154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 >> 172.21.15.165:0/3349148690 conn(0x7fe16006abe0 msgr2=0x7fe16006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 shutdown_connections 2024-04-25T10:15:51.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.506+0000 7fe167400640 1 -- 172.21.15.165:0/3349148690 wait complete. 2024-04-25T10:15:51.528 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:15:51.528 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:15:51.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.725+0000 7fc86e200640 1 Processor -- start 2024-04-25T10:15:51.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.725+0000 7fc86e200640 1 -- start start 2024-04-25T10:15:51.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc86e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc868154ee0 0x7fc8680cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc86e200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc8680cfcf0 0x7fc8680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc86e200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc868071480 con 0x7fc8680ce3c0 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc86e200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8680ce830 con 0x7fc868154ee0 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc86e200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc868072090 con 0x7fc8680cfcf0 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc867400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc868154ee0 0x7fc8680cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc867400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc868154ee0 0x7fc8680cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35734/0 (socket says 172.21.15.165:35734) 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.727+0000 7fc867400640 1 -- 172.21.15.165:0/354812926 learned_addr learned my addr 172.21.15.165:0/354812926 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.728+0000 7fc85ea00640 1 --2- 172.21.15.165:0/354812926 >> v2:172.21.15.172:3300/0 conn(0x7fc8680cfcf0 0x7fc8680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.728+0000 7fc867400640 1 -- 172.21.15.165:0/354812926 >> v2:172.21.15.172:3300/0 conn(0x7fc8680cfcf0 msgr2=0x7fc8680cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.728+0000 7fc867400640 1 --2- 172.21.15.165:0/354812926 >> v2:172.21.15.172:3300/0 conn(0x7fc8680cfcf0 0x7fc8680cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.728+0000 7fc867400640 1 -- 172.21.15.165:0/354812926 >> v1:172.21.15.165:6789/0 conn(0x7fc8680ce3c0 legacy=0x7fc86815e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.728+0000 7fc867400640 1 -- 172.21.15.165:0/354812926 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868071760 con 0x7fc868154ee0 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.729+0000 7fc867400640 1 --2- 172.21.15.165:0/354812926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc868154ee0 0x7fc8680cf7b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cf560 tx=0x7fc85802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b048908418ad42 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.729+0000 7fc866a00640 1 -- 172.21.15.165:0/354812926 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858037070 con 0x7fc868154ee0 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.729+0000 7fc866a00640 1 -- 172.21.15.165:0/354812926 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc858032440 con 0x7fc868154ee0 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.729+0000 7fc866a00640 1 -- 172.21.15.165:0/354812926 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc858032810 con 0x7fc868154ee0 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 -- 172.21.15.165:0/354812926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc868154ee0 msgr2=0x7fc8680cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 --2- 172.21.15.165:0/354812926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc868154ee0 0x7fc8680cf7b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7fc8680cf560 tx=0x7fc85802a270 comp rx=0 tx=0).stop 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 -- 172.21.15.165:0/354812926 shutdown_connections 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 --2- 172.21.15.165:0/354812926 >> v2:172.21.15.172:3300/0 conn(0x7fc8680cfcf0 0x7fc8680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 --2- 172.21.15.165:0/354812926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc868154ee0 0x7fc8680cf7b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 -- 172.21.15.165:0/354812926 >> 172.21.15.165:0/354812926 conn(0x7fc86806abe0 msgr2=0x7fc86806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 -- 172.21.15.165:0/354812926 shutdown_connections 2024-04-25T10:15:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.730+0000 7fc86e200640 1 -- 172.21.15.165:0/354812926 wait complete. 2024-04-25T10:15:51.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.732+0000 7fc86e200640 1 Processor -- start 2024-04-25T10:15:51.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.732+0000 7fc86e200640 1 -- start start 2024-04-25T10:15:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc86e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8680ce3c0 0x7fc8681409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc86e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 0x7fc8680a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc86e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc868154ee0 0x7fc868159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc86e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868071d20 con 0x7fc8680cfcf0 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc86e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc868071230 con 0x7fc868154ee0 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc86e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8681552c0 con 0x7fc8680ce3c0 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc85ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 0x7fc8680a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc85ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 0x7fc8680a89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35746/0 (socket says 172.21.15.165:35746) 2024-04-25T10:15:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.733+0000 7fc85ea00640 1 -- 172.21.15.165:0/1102920029 learned_addr learned my addr 172.21.15.165:0/1102920029 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc867e00640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc868154ee0 0x7fc868159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc867400640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8680ce3c0 0x7fc8681409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85ea00640 1 -- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8680ce3c0 msgr2=0x7fc8681409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85ea00640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8680ce3c0 0x7fc8681409c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85ea00640 1 -- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc868154ee0 msgr2=0x7fc868159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85ea00640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc868154ee0 0x7fc868159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85ea00640 1 -- 172.21.15.165:0/1102920029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868141230 con 0x7fc8680cfcf0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc867e00640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc868154ee0 0x7fc868159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85ea00640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 0x7fc8680a89f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d800 tx=0x7fc85000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000b6c0 con 0x7fc8680cfcf0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc850019070 con 0x7fc8680cfcf0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85000ba80 con 0x7fc8680cfcf0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.734+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc868140f00 con 0x7fc8680cfcf0 2024-04-25T10:15:51.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.735+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8681f3f60 con 0x7fc8680cfcf0 2024-04-25T10:15:51.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.736+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc868074290 con 0x7fc8680cfcf0 2024-04-25T10:15:51.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.736+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc85000bc20 con 0x7fc8680cfcf0 2024-04-25T10:15:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.738+0000 7fc85fe00640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b500 0x7fc84406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.738+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc85008a860 con 0x7fc8680cfcf0 2024-04-25T10:15:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.738+0000 7fc867400640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b500 0x7fc84406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.739+0000 7fc867400640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b500 0x7fc84406d9c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc858034000 tx=0x7fc85802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:51.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:51.743+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc868074290 con 0x7fc8680cfcf0 2024-04-25T10:15:52.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.004+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fc868073990 con 0x7fc8680cfcf0 2024-04-25T10:15:52.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.005+0000 7fc85fe00640 1 -- 172.21.15.165:0/1102920029 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1186 (secure 0 0 0) 0x7fc85001c360 con 0x7fc8680cfcf0 2024-04-25T10:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:52.009 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":853,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":948,"data_bytes":971343703,"bytes_used":2371715072,"bytes_avail":770722398208,"bytes_total":773094113280,"write_bytes_sec":30291514,"read_op_per_sec":0,"write_op_per_sec":31},"fsmap":{"epoch":90,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":9234}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-25T10:14:50.391716+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b500 msgr2=0x7fc84406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b500 0x7fc84406d9c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fc858034000 tx=0x7fc85802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 msgr2=0x7fc8680a89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 0x7fc8680a89f0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fc85000d800 tx=0x7fc85000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 shutdown_connections 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc84406b500 0x7fc84406d9c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc868154ee0 0x7fc868159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8680cfcf0 0x7fc8680a89f0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 --2- 172.21.15.165:0/1102920029 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8680ce3c0 0x7fc8681409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 >> 172.21.15.165:0/1102920029 conn(0x7fc86806abe0 msgr2=0x7fc8680b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 shutdown_connections 2024-04-25T10:15:52.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:52.008+0000 7fc86e200640 1 -- 172.21.15.165:0/1102920029 wait complete. 2024-04-25T10:15:52.029 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:15:53.031 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:53.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.232+0000 7f3bbe800640 1 Processor -- start 2024-04-25T10:15:53.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.233+0000 7f3bbe800640 1 -- start start 2024-04-25T10:15:53.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.234+0000 7f3bbe800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3bb8144440 0x7f3bb8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.234+0000 7f3bbe800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.234+0000 7f3bbe800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bb815bf30 con 0x7f3bb80cd860 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.234+0000 7f3bbe800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bb815c860 con 0x7f3bb8144440 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.234+0000 7f3bbe800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb8071970 con 0x7f3bb8071530 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbd400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3bb8144440 0x7f3bb8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3bb80cd860 0x7f3bb806d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36588/0 (socket says 172.21.15.165:36588) 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbde00640 1 -- 172.21.15.165:0/2103774731 learned_addr learned my addr 172.21.15.165:0/2103774731 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbca00640 1 --2- 172.21.15.165:0/2103774731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbd400640 1 -- 172.21.15.165:0/2103774731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 msgr2=0x7f3bb80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbd400640 1 --2- 172.21.15.165:0/2103774731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb80cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbd400640 1 -- 172.21.15.165:0/2103774731 >> v1:172.21.15.165:6789/0 conn(0x7f3bb80cd860 legacy=0x7f3bb806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bbd400640 1 -- 172.21.15.165:0/2103774731 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb815c210 con 0x7f3bb8144440 2024-04-25T10:15:53.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.235+0000 7f3bb7e00640 1 -- 172.21.15.165:0/2103774731 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1210043667 0 0) 0x7f3bb815bf30 con 0x7f3bb80cd860 2024-04-25T10:15:53.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.236+0000 7f3bbd400640 1 --2- 172.21.15.165:0/2103774731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb8144440 0x7f3bb8070ff0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009d90 tx=0x7f3bac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be69e4953ab2515 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.236+0000 7f3bb7e00640 1 -- 172.21.15.165:0/2103774731 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac007740 con 0x7f3bb8144440 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.236+0000 7f3bb7e00640 1 -- 172.21.15.165:0/2103774731 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bac0078e0 con 0x7f3bb8144440 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.236+0000 7f3bb7e00640 1 -- 172.21.15.165:0/2103774731 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bac007cb0 con 0x7f3bb8144440 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 -- 172.21.15.165:0/2103774731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb8144440 msgr2=0x7f3bb8070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 --2- 172.21.15.165:0/2103774731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb8144440 0x7f3bb8070ff0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f3bac009d90 tx=0x7f3bac02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 -- 172.21.15.165:0/2103774731 shutdown_connections 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 --2- 172.21.15.165:0/2103774731 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb80cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 --2- 172.21.15.165:0/2103774731 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb8144440 0x7f3bb8070ff0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 -- 172.21.15.165:0/2103774731 >> 172.21.15.165:0/2103774731 conn(0x7f3bb806abe0 msgr2=0x7f3bb806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 -- 172.21.15.165:0/2103774731 shutdown_connections 2024-04-25T10:15:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.237+0000 7f3bbe800640 1 -- 172.21.15.165:0/2103774731 wait complete. 2024-04-25T10:15:53.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.239+0000 7f3bbe800640 1 Processor -- start 2024-04-25T10:15:53.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.239+0000 7f3bbe800640 1 -- start start 2024-04-25T10:15:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbe800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb81e1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbe800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 0x7f3bb815b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbe800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bb8144440 0x7f3bb81e2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbe800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80cdcd0 con 0x7f3bb80cd860 2024-04-25T10:15:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbe800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb815bc30 con 0x7f3bb8144440 2024-04-25T10:15:53.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbe800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb80cde50 con 0x7f3bb8071530 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 0x7f3bb815b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 0x7f3bb815b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35762/0 (socket says 172.21.15.165:35762) 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbca00640 1 -- 172.21.15.165:0/653131403 learned_addr learned my addr 172.21.15.165:0/653131403 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.240+0000 7f3bbd400640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb81e1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbde00640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bb8144440 0x7f3bb81e2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbca00640 1 -- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 msgr2=0x7f3bb81e1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbca00640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb81e1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbca00640 1 -- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bb8144440 msgr2=0x7f3bb81e2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbca00640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bb8144440 0x7f3bb81e2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbca00640 1 -- 172.21.15.165:0/653131403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb815b550 con 0x7f3bb80cd860 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbde00640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bb8144440 0x7f3bb81e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbca00640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 0x7f3bb815b010 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c7f0 tx=0x7f3ba400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4015070 con 0x7f3bb80cd860 2024-04-25T10:15:53.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ba4004240 con 0x7f3bb80cd860 2024-04-25T10:15:53.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb80cc2b0 con 0x7f3bb80cd860 2024-04-25T10:15:53.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4007620 con 0x7f3bb80cd860 2024-04-25T10:15:53.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.241+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb80cc870 con 0x7f3bb80cd860 2024-04-25T10:15:53.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.242+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb806e270 con 0x7f3bb80cd860 2024-04-25T10:15:53.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.243+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ba40079d0 con 0x7f3bb80cd860 2024-04-25T10:15:53.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.244+0000 7f3bb5600640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b9806b4d0 0x7f3b9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.244+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ba408a490 con 0x7f3bb80cd860 2024-04-25T10:15:53.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.245+0000 7f3bbd400640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b9806b4d0 0x7f3b9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.245+0000 7f3bbd400640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b9806b4d0 0x7f3b9806d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3bac034000 tx=0x7f3bac033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.248+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bb806e270 con 0x7f3bb80cd860 2024-04-25T10:15:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.467+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bb806ea10 con 0x7f3bb80cd860 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.469+0000 7f3bb5600640 1 -- 172.21.15.165:0/653131403 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f3bb815b550 con 0x7f3bb80cd860 2024-04-25T10:15:53.472 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:53.473 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:53.473 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b9806b4d0 msgr2=0x7f3b9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b9806b4d0 0x7f3b9806d990 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3bac034000 tx=0x7f3bac033040 comp rx=0 tx=0).stop 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 msgr2=0x7f3bb815b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 0x7f3bb815b010 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400c7f0 tx=0x7f3ba400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 shutdown_connections 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3b9806b4d0 0x7f3b9806d990 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bb8144440 0x7f3bb81e2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bb80cd860 0x7f3bb815b010 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 --2- 172.21.15.165:0/653131403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bb8071530 0x7f3bb81e1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 >> 172.21.15.165:0/653131403 conn(0x7f3bb806abe0 msgr2=0x7f3bb80c60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 shutdown_connections 2024-04-25T10:15:53.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.472+0000 7f3bbe800640 1 -- 172.21.15.165:0/653131403 wait complete. 2024-04-25T10:15:53.494 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:15:53.494 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:15:53.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.688+0000 7f9920a00640 1 Processor -- start 2024-04-25T10:15:53.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.688+0000 7f9920a00640 1 -- start start 2024-04-25T10:15:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.689+0000 7f9920a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 0x7f991c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.689+0000 7f9920a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f991c071530 0x7f991c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.689+0000 7f9920a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991c15bf30 con 0x7f991c0cd860 2024-04-25T10:15:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.689+0000 7f9920a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991c15c860 con 0x7f991c071530 2024-04-25T10:15:53.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.689+0000 7f9920a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c071970 con 0x7f991c144440 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.689+0000 7f991b200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f991c0cd860 0x7f991c06d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36598/0 (socket says 172.21.15.165:36598) 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f991b200640 1 -- 172.21.15.165:0/951812404 learned_addr learned my addr 172.21.15.165:0/951812404 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9911e00640 1 --2- 172.21.15.165:0/951812404 >> v2:172.21.15.172:3300/0 conn(0x7f991c071530 0x7f991c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f991a800640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 0x7f991c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335159279 0 0) 0x7f991c15bf30 con 0x7f991c0cd860 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9900003520 con 0x7f991c0cd860 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2633158131 0 0) 0x7f9900003520 con 0x7f991c0cd860 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 msgr2=0x7f991c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 0x7f991c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 >> v2:172.21.15.172:3300/0 conn(0x7f991c071530 msgr2=0x7f991c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 --2- 172.21.15.165:0/951812404 >> v2:172.21.15.172:3300/0 conn(0x7f991c071530 0x7f991c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c15c210 con 0x7f991c0cd860 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f99140019f0 con 0x7f991c0cd860 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.690+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 >> v1:172.21.15.165:6789/0 conn(0x7f991c0cd860 legacy=0x7f991c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9919e00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 0x7f99000039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9919e00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9900003f30 0x7f9900004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9919e00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f99000051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c15c860 con 0x7f9900004d40 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c071970 con 0x7f9900003f30 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990000b9f0 con 0x7f991c144440 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991b200640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f99000051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991a800640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 0x7f99000039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9911e00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9900003f30 0x7f9900004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991b200640 1 -- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 msgr2=0x7f99000039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991b200640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c144440 0x7f99000039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991b200640 1 -- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9900003f30 msgr2=0x7f9900004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991b200640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9900003f30 0x7f9900004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f991b200640 1 -- 172.21.15.165:0/951812404 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9900003520 con 0x7f9900004d40 2024-04-25T10:15:53.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.691+0000 7f9911e00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9900003f30 0x7f9900004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.692+0000 7f991b200640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f99000051c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f991400a920 tx=0x7f9900005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c0e049c4871fdf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400a4c0 con 0x7f9900004d40 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991402f070 con 0x7f9900004d40 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9919e00640 1 -- 172.21.15.165:0/951812404 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991402bd40 con 0x7f9900004d40 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9920a00640 1 -- 172.21.15.165:0/951812404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 msgr2=0x7f99000051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9920a00640 1 --2- 172.21.15.165:0/951812404 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f99000051c0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f991400a920 tx=0x7f9900005830 comp rx=0 tx=0).stop 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f991b200640 1 -- 172.21.15.165:0/951812404 reap_dead start 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9920a00640 1 -- 172.21.15.165:0/951812404 shutdown_connections 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9920a00640 1 -- 172.21.15.165:0/951812404 >> 172.21.15.165:0/951812404 conn(0x7f991c06abe0 msgr2=0x7f991c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9920a00640 1 -- 172.21.15.165:0/951812404 shutdown_connections 2024-04-25T10:15:53.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.693+0000 7f9920a00640 1 -- 172.21.15.165:0/951812404 wait complete. 2024-04-25T10:15:53.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.695+0000 7f9920a00640 1 Processor -- start 2024-04-25T10:15:53.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.695+0000 7f9920a00640 1 -- start start 2024-04-25T10:15:53.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.696+0000 7f9920a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f991c1e2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.696+0000 7f9920a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c1e4c30 0x7f991c1e50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.696+0000 7f9920a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c12cb40 0x7f991c12cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.696+0000 7f9920a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cdcd0 con 0x7f9900004d40 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.696+0000 7f9920a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991c15bc30 con 0x7f991c12cb40 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.696+0000 7f9920a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f991c0cde50 con 0x7f991c1e4c30 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f991c1e2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f991c1e2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35780/0 (socket says 172.21.15.165:35780) 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 -- 172.21.15.165:0/1845911025 learned_addr learned my addr 172.21.15.165:0/1845911025 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 -- 172.21.15.165:0/1845911025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c1e4c30 msgr2=0x7f991c1e50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c1e4c30 0x7f991c1e50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 -- 172.21.15.165:0/1845911025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c12cb40 msgr2=0x7f991c12cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c12cb40 0x7f991c12cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 -- 172.21.15.165:0/1845911025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991c12baa0 con 0x7f9900004d40 2024-04-25T10:15:53.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f991a800640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f991c1e2240 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f990c011630 tx=0x7f990c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.697+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c015870 con 0x7f9900004d40 2024-04-25T10:15:53.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.698+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c015a10 con 0x7f9900004d40 2024-04-25T10:15:53.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.698+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c015dc0 con 0x7f9900004d40 2024-04-25T10:15:53.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.698+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991c12bd80 con 0x7f9900004d40 2024-04-25T10:15:53.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.698+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f991c12c340 con 0x7f9900004d40 2024-04-25T10:15:53.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.699+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991c073990 con 0x7f9900004d40 2024-04-25T10:15:53.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.700+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f990c00f490 con 0x7f9900004d40 2024-04-25T10:15:53.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.701+0000 7f9913400640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b4d0 0x7f98f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:53.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.701+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f990c08ace0 con 0x7f9900004d40 2024-04-25T10:15:53.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.701+0000 7f9911e00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b4d0 0x7f98f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:53.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.702+0000 7f9911e00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b4d0 0x7f98f806d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f991c1e1fa0 tx=0x7f9904008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:53.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.705+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f990c019030 con 0x7f9900004d40 2024-04-25T10:15:53.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.973+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f991c0b2010 con 0x7f9900004d40 2024-04-25T10:15:53.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.974+0000 7f9913400640 1 -- 172.21.15.165:0/1845911025 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7f990c0599d0 con 0x7f9900004d40 2024-04-25T10:15:53.977 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:53.978 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":854,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580426,"bytes_used":2612666368,"bytes_avail":770481446912,"bytes_total":773094113280,"write_bytes_sec":39061297,"read_op_per_sec":0,"write_op_per_sec":40},"fsmap":{"epoch":90,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":9234}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-25T10:14:50.391716+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b4d0 msgr2=0x7f98f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b4d0 0x7f98f806d990 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f991c1e1fa0 tx=0x7f9904008040 comp rx=0 tx=0).stop 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 msgr2=0x7f991c1e2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f991c1e2240 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f990c011630 tx=0x7f990c011af0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 shutdown_connections 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98f806b4d0 0x7f98f806d990 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f991c12cb40 0x7f991c12cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f991c1e4c30 0x7f991c1e50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 --2- 172.21.15.165:0/1845911025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9900004d40 0x7f991c1e2240 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:53.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 >> 172.21.15.165:0/1845911025 conn(0x7f991c06abe0 msgr2=0x7f991c0c4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:53.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 shutdown_connections 2024-04-25T10:15:53.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:53.977+0000 7f9920a00640 1 -- 172.21.15.165:0/1845911025 wait complete. 2024-04-25T10:15:53.998 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:15:54.999 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:55.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.211+0000 7ff172200640 1 Processor -- start 2024-04-25T10:15:55.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.211+0000 7ff172200640 1 -- start start 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.212+0000 7ff172200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff16c143e60 0x7ff16c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.212+0000 7ff172200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.212+0000 7ff172200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff16c0cc840 0x7ff16c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.212+0000 7ff172200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c144240 con 0x7ff16c143e60 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.212+0000 7ff172200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff16c15e220 con 0x7ff16c0cc840 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.212+0000 7ff172200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c15e500 con 0x7ff16c06e6a0 2024-04-25T10:15:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff16c143e60 0x7ff16c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff16c143e60 0x7ff16c06e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35784/0 (socket says 172.21.15.165:35784) 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 -- 172.21.15.165:0/1759182221 learned_addr learned my addr 172.21.15.165:0/1759182221 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff171800640 1 --2- 172.21.15.165:0/1759182221 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0cc840 0x7ff16c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 -- 172.21.15.165:0/1759182221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 msgr2=0x7ff16c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff16be00640 1 --2- 172.21.15.165:0/1759182221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 --2- 172.21.15.165:0/1759182221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 -- 172.21.15.165:0/1759182221 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0cc840 msgr2=0x7ff16c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 --2- 172.21.15.165:0/1759182221 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0cc840 0x7ff16c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff170e00640 1 -- 172.21.15.165:0/1759182221 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c15e7e0 con 0x7ff16c143e60 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.213+0000 7ff171800640 1 --2- 172.21.15.165:0/1759182221 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0cc840 0x7ff16c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.214+0000 7ff170e00640 1 --2- 172.21.15.165:0/1759182221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c143e60 0x7ff16c06e160 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0b2740 tx=0x7ff15c02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1657e768f0d3f7ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff16b400640 1 -- 172.21.15.165:0/1759182221 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00b2e0 con 0x7ff16c143e60 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff16b400640 1 -- 172.21.15.165:0/1759182221 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c009600 con 0x7ff16c143e60 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff16b400640 1 -- 172.21.15.165:0/1759182221 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c00b480 con 0x7ff16c143e60 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 -- 172.21.15.165:0/1759182221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c143e60 msgr2=0x7ff16c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 --2- 172.21.15.165:0/1759182221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c143e60 0x7ff16c06e160 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0b2740 tx=0x7ff15c02c8a0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 -- 172.21.15.165:0/1759182221 shutdown_connections 2024-04-25T10:15:55.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 --2- 172.21.15.165:0/1759182221 >> v2:172.21.15.172:3300/0 conn(0x7ff16c0cc840 0x7ff16c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 --2- 172.21.15.165:0/1759182221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 --2- 172.21.15.165:0/1759182221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c143e60 0x7ff16c06e160 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.215+0000 7ff172200640 1 -- 172.21.15.165:0/1759182221 >> 172.21.15.165:0/1759182221 conn(0x7ff16c06abe0 msgr2=0x7ff16c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.216+0000 7ff172200640 1 -- 172.21.15.165:0/1759182221 shutdown_connections 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.216+0000 7ff172200640 1 -- 172.21.15.165:0/1759182221 wait complete. 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.217+0000 7ff172200640 1 Processor -- start 2024-04-25T10:15:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.217+0000 7ff172200640 1 -- start start 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff172200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff172200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 0x7ff16c15a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff172200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c143e60 0x7ff16c0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff172200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0cce50 con 0x7ff16c0cc840 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff172200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0cccd0 con 0x7ff16c143e60 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff172200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff16c0b2d30 con 0x7ff16c06e6a0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.218+0000 7ff16be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 0x7ff16c15a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 0x7ff16c15a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35800/0 (socket says 172.21.15.165:35800) 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 -- 172.21.15.165:0/1356486358 learned_addr learned my addr 172.21.15.165:0/1356486358 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff170e00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 -- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 msgr2=0x7ff16c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 -- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c143e60 msgr2=0x7ff16c0c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c143e60 0x7ff16c0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 -- 172.21.15.165:0/1356486358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff16c0b1780 con 0x7ff16c0cc840 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff170e00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff16be00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 0x7ff16c15a020 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c570 tx=0x7ff15800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.219+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158015070 con 0x7ff16c0cc840 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.220+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff158011800 con 0x7ff16c0cc840 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.220+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff158011bf0 con 0x7ff16c0cc840 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.220+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff16c0c9e00 con 0x7ff16c0cc840 2024-04-25T10:15:55.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.220+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff16c0c9ab0 con 0x7ff16c0cc840 2024-04-25T10:15:55.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.221+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff16c073990 con 0x7ff16c0cc840 2024-04-25T10:15:55.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.222+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff158018070 con 0x7ff16c0cc840 2024-04-25T10:15:55.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.223+0000 7ff168c00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff14c06b4d0 0x7ff14c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.223+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff15808a260 con 0x7ff16c0cc840 2024-04-25T10:15:55.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.223+0000 7ff170e00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff14c06b4d0 0x7ff14c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.224+0000 7ff170e00640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff14c06b4d0 0x7ff14c06d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff15c036000 tx=0x7ff15c00b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.228+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff158058eb0 con 0x7ff16c0cc840 2024-04-25T10:15:55.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.444+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff16c06b3d0 con 0x7ff16c0cc840 2024-04-25T10:15:55.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.445+0000 7ff168c00640 1 -- 172.21.15.165:0/1356486358 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7ff1580587b0 con 0x7ff16c0cc840 2024-04-25T10:15:55.448 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:55.448 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:55.449 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:55.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.447+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff14c06b4d0 msgr2=0x7ff14c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.447+0000 7ff172200640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff14c06b4d0 0x7ff14c06d990 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7ff15c036000 tx=0x7ff15c00b000 comp rx=0 tx=0).stop 2024-04-25T10:15:55.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.447+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 msgr2=0x7ff16c15a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.447+0000 7ff172200640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 0x7ff16c15a020 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7ff15800c570 tx=0x7ff15800ca30 comp rx=0 tx=0).stop 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 shutdown_connections 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff14c06b4d0 0x7ff14c06d990 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff16c143e60 0x7ff16c0c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff16c0cc840 0x7ff16c15a020 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 --2- 172.21.15.165:0/1356486358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff16c06e6a0 0x7ff16c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 >> 172.21.15.165:0/1356486358 conn(0x7ff16c06abe0 msgr2=0x7ff16c15ccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 shutdown_connections 2024-04-25T10:15:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.448+0000 7ff172200640 1 -- 172.21.15.165:0/1356486358 wait complete. 2024-04-25T10:15:55.472 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:15:55.472 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:15:55.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.664+0000 7fd518000640 1 Processor -- start 2024-04-25T10:15:55.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.664+0000 7fd518000640 1 -- start start 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd518000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd51015db70 0x7fd5100cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd518000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5100cfc20 0x7fd51006d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd518000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd510071650 con 0x7fd51015df50 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd518000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100cbd80 con 0x7fd51015db70 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd518000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd510161910 con 0x7fd5100cfc20 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd515600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd51015db70 0x7fd5100cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd515600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd51015db70 0x7fd5100cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35808/0 (socket says 172.21.15.165:35808) 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd515600640 1 -- 172.21.15.165:0/2081027881 learned_addr learned my addr 172.21.15.165:0/2081027881 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:55.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.666+0000 7fd514c00640 1 --1- 172.21.15.165:0/2081027881 >> v1:172.21.15.165:6789/0 conn(0x7fd51015df50 0x7fd5100cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36606/0 (socket says 172.21.15.165:36606) 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.667+0000 7fd516000640 1 --2- 172.21.15.165:0/2081027881 >> v2:172.21.15.172:3300/0 conn(0x7fd5100cfc20 0x7fd51006d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.667+0000 7fd515600640 1 -- 172.21.15.165:0/2081027881 >> v2:172.21.15.172:3300/0 conn(0x7fd5100cfc20 msgr2=0x7fd51006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.667+0000 7fd515600640 1 --2- 172.21.15.165:0/2081027881 >> v2:172.21.15.172:3300/0 conn(0x7fd5100cfc20 0x7fd51006d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.667+0000 7fd515600640 1 -- 172.21.15.165:0/2081027881 >> v1:172.21.15.165:6789/0 conn(0x7fd51015df50 legacy=0x7fd5100cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.667+0000 7fd515600640 1 -- 172.21.15.165:0/2081027881 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100cf230 con 0x7fd51015db70 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.667+0000 7fd50fe00640 1 -- 172.21.15.165:0/2081027881 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952485539 0 0) 0x7fd4fc001110 con 0x7fd51015df50 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.668+0000 7fd515600640 1 --2- 172.21.15.165:0/2081027881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015db70 0x7fd5100cb840 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd5100b4d10 tx=0x7fd50402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38d889b33d4895a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.668+0000 7fd50fe00640 1 -- 172.21.15.165:0/2081027881 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504037070 con 0x7fd51015db70 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.668+0000 7fd50fe00640 1 -- 172.21.15.165:0/2081027881 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd504032440 con 0x7fd51015db70 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.668+0000 7fd50fe00640 1 -- 172.21.15.165:0/2081027881 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd504032810 con 0x7fd51015db70 2024-04-25T10:15:55.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 -- 172.21.15.165:0/2081027881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015db70 msgr2=0x7fd5100cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 --2- 172.21.15.165:0/2081027881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015db70 0x7fd5100cb840 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fd5100b4d10 tx=0x7fd50402a270 comp rx=0 tx=0).stop 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 -- 172.21.15.165:0/2081027881 shutdown_connections 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 --2- 172.21.15.165:0/2081027881 >> v2:172.21.15.172:3300/0 conn(0x7fd5100cfc20 0x7fd51006d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 --2- 172.21.15.165:0/2081027881 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015db70 0x7fd5100cb840 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 -- 172.21.15.165:0/2081027881 >> 172.21.15.165:0/2081027881 conn(0x7fd51006abe0 msgr2=0x7fd51006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 -- 172.21.15.165:0/2081027881 shutdown_connections 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.669+0000 7fd518000640 1 -- 172.21.15.165:0/2081027881 wait complete. 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.671+0000 7fd518000640 1 Processor -- start 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.671+0000 7fd518000640 1 -- start start 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd518000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5100cfc20 0x7fd51013fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd518000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd51015db70 0x7fd5101e9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd518000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 0x7fd510140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd518000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510162120 con 0x7fd51015df50 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd518000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd510161e40 con 0x7fd5100cfc20 2024-04-25T10:15:55.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd518000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd510162640 con 0x7fd51015db70 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd516000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 0x7fd510140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd516000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 0x7fd510140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35824/0 (socket says 172.21.15.165:35824) 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd516000640 1 -- 172.21.15.165:0/2117892047 learned_addr learned my addr 172.21.15.165:0/2117892047 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd516000640 1 -- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd51015db70 msgr2=0x7fd5101e9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd514c00640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd51015db70 0x7fd5101e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.672+0000 7fd515600640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5100cfc20 0x7fd51013fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd516000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd51015db70 0x7fd5101e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd516000640 1 -- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5100cfc20 msgr2=0x7fd51013fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd516000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5100cfc20 0x7fd51013fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd516000640 1 -- 172.21.15.165:0/2117892047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5101e8680 con 0x7fd51015df50 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd516000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 0x7fd510140290 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd508007e60 tx=0x7fd50800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508014420 con 0x7fd51015df50 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd508019070 con 0x7fd51015df50 2024-04-25T10:15:55.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd508014820 con 0x7fd51015df50 2024-04-25T10:15:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5101e8960 con 0x7fd51015df50 2024-04-25T10:15:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.673+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5101e8f20 con 0x7fd51015df50 2024-04-25T10:15:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.674+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd510073990 con 0x7fd51015df50 2024-04-25T10:15:55.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.675+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5080149c0 con 0x7fd51015df50 2024-04-25T10:15:55.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.676+0000 7fd50d600640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4f006b500 0x7fd4f006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:55.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.676+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd50808ac60 con 0x7fd51015df50 2024-04-25T10:15:55.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.676+0000 7fd515600640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4f006b500 0x7fd4f006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:55.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.677+0000 7fd515600640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4f006b500 0x7fd4f006d9c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd504034000 tx=0x7fd50402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:55.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.680+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5080598b0 con 0x7fd51015df50 2024-04-25T10:15:55.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.950+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd5100cbb40 con 0x7fd51015df50 2024-04-25T10:15:55.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.950+0000 7fd50d600640 1 -- 172.21.15.165:0/2117892047 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1188 (secure 0 0 0) 0x7fd5080591b0 con 0x7fd51015df50 2024-04-25T10:15:55.954 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:55.954 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":856,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580426,"bytes_used":2612666368,"bytes_avail":770481446912,"bytes_total":773094113280,"write_bytes_sec":23650226,"read_op_per_sec":0,"write_op_per_sec":24},"fsmap":{"epoch":90,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":9234}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":32,"modified":"2024-04-25T10:14:50.391716+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4f006b500 msgr2=0x7fd4f006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4f006b500 0x7fd4f006d9c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fd504034000 tx=0x7fd50402e000 comp rx=0 tx=0).stop 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 msgr2=0x7fd510140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 0x7fd510140290 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fd508007e60 tx=0x7fd50800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 shutdown_connections 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4f006b500 0x7fd4f006d9c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd51015df50 0x7fd510140290 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd51015db70 0x7fd5101e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 --2- 172.21.15.165:0/2117892047 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5100cfc20 0x7fd51013fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:55.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.953+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 >> 172.21.15.165:0/2117892047 conn(0x7fd51006abe0 msgr2=0x7fd51015e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:55.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.954+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 shutdown_connections 2024-04-25T10:15:55.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:55.954+0000 7fd518000640 1 -- 172.21.15.165:0/2117892047 wait complete. 2024-04-25T10:15:55.976 INFO:tasks.cephfs.filesystem.ceph_manager:{} 2024-04-25T10:15:56.978 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:57.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.188+0000 7f143e000640 1 Processor -- start 2024-04-25T10:15:57.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.189+0000 7f143e000640 1 -- start start 2024-04-25T10:15:57.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.190+0000 7f143e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f143815db70 0x7f1438162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.190+0000 7f143e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14380b6cb0 0x7f14380b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.190+0000 7f143e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1438162690 con 0x7f14380b5a60 2024-04-25T10:15:57.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.190+0000 7f143e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14380d3800 con 0x7f14380b6cb0 2024-04-25T10:15:57.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.190+0000 7f143e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380ceee0 con 0x7f143815db70 2024-04-25T10:15:57.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.191+0000 7f142ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14380b6cb0 0x7f14380b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.191+0000 7f1437400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f143815db70 0x7f1438162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.191+0000 7f142ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14380b6cb0 0x7f14380b5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:54844/0 (socket says 172.21.15.165:54844) 2024-04-25T10:15:57.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.191+0000 7f142ea00640 1 -- 172.21.15.165:0/3237976082 learned_addr learned my addr 172.21.15.165:0/3237976082 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:57.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.192+0000 7f1437400640 1 -- 172.21.15.165:0/3237976082 >> v2:172.21.15.172:3300/0 conn(0x7f14380b6cb0 msgr2=0x7f14380b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.192+0000 7f1437400640 1 --2- 172.21.15.165:0/3237976082 >> v2:172.21.15.172:3300/0 conn(0x7f14380b6cb0 0x7f14380b5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.192+0000 7f1437400640 1 -- 172.21.15.165:0/3237976082 >> v1:172.21.15.172:6789/0 conn(0x7f14380b5a60 legacy=0x7f143806c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.192+0000 7f1437400640 1 -- 172.21.15.165:0/3237976082 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380d1d10 con 0x7f143815db70 2024-04-25T10:15:57.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.193+0000 7f1437400640 1 --2- 172.21.15.165:0/3237976082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f143815db70 0x7f1438162150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14280097a0 tx=0x7f142802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bc59ed9676dd0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.193+0000 7f1436a00640 1 -- 172.21.15.165:0/3237976082 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428037070 con 0x7f143815db70 2024-04-25T10:15:57.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.193+0000 7f1436a00640 1 -- 172.21.15.165:0/3237976082 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428032440 con 0x7f143815db70 2024-04-25T10:15:57.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.193+0000 7f1436a00640 1 -- 172.21.15.165:0/3237976082 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428032810 con 0x7f143815db70 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 -- 172.21.15.165:0/3237976082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f143815db70 msgr2=0x7f1438162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 --2- 172.21.15.165:0/3237976082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f143815db70 0x7f1438162150 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f14280097a0 tx=0x7f142802a180 comp rx=0 tx=0).stop 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 -- 172.21.15.165:0/3237976082 shutdown_connections 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 --2- 172.21.15.165:0/3237976082 >> v2:172.21.15.172:3300/0 conn(0x7f14380b6cb0 0x7f14380b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 --2- 172.21.15.165:0/3237976082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f143815db70 0x7f1438162150 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 -- 172.21.15.165:0/3237976082 >> 172.21.15.165:0/3237976082 conn(0x7f14380d0520 msgr2=0x7f14380d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 -- 172.21.15.165:0/3237976082 shutdown_connections 2024-04-25T10:15:57.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.194+0000 7f143e000640 1 -- 172.21.15.165:0/3237976082 wait complete. 2024-04-25T10:15:57.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.196+0000 7f143e000640 1 Processor -- start 2024-04-25T10:15:57.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.196+0000 7f143e000640 1 -- start start 2024-04-25T10:15:57.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f143e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14380b5a60 0x7f1438158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f143e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14380b6cb0 0x7f14380cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f143e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 0x7f143806c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f143e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143815df50 con 0x7f143815db70 2024-04-25T10:15:57.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f143e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380d22d0 con 0x7f14380b6cb0 2024-04-25T10:15:57.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f143e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380ce530 con 0x7f14380b5a60 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f1437e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 0x7f143806c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f1437e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 0x7f143806c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53486/0 (socket says 172.21.15.165:53486) 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.197+0000 7f1437e00640 1 -- 172.21.15.165:0/3601330178 learned_addr learned my addr 172.21.15.165:0/3601330178 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437400640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14380b5a60 0x7f1438158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f142ea00640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14380b6cb0 0x7f14380cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437e00640 1 -- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14380b5a60 msgr2=0x7f1438158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437e00640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14380b5a60 0x7f1438158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437e00640 1 -- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14380b6cb0 msgr2=0x7f14380cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437e00640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14380b6cb0 0x7f14380cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437e00640 1 -- 172.21.15.165:0/3601330178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380cb2c0 con 0x7f143815db70 2024-04-25T10:15:57.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f1437e00640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 0x7f143806c2d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f143000c6e0 tx=0x7f143000cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430003b70 con 0x7f143815db70 2024-04-25T10:15:57.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1430003d30 con 0x7f143815db70 2024-04-25T10:15:57.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1430005310 con 0x7f143815db70 2024-04-25T10:15:57.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.198+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14380cb5a0 con 0x7f143815db70 2024-04-25T10:15:57.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.199+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14380cbb60 con 0x7f143815db70 2024-04-25T10:15:57.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.200+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1438073990 con 0x7f143815db70 2024-04-25T10:15:57.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.201+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14300054b0 con 0x7f143815db70 2024-04-25T10:15:57.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.202+0000 7f142fe00640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f141406b4d0 0x7f141406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.202+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f143008a270 con 0x7f143815db70 2024-04-25T10:15:57.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.202+0000 7f1437400640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f141406b4d0 0x7f141406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.203+0000 7f1437400640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f141406b4d0 0x7f141406d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1428034000 tx=0x7f142802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.207+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1430058ec0 con 0x7f143815db70 2024-04-25T10:15:57.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.433+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14380d1820 con 0x7f143815db70 2024-04-25T10:15:57.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.434+0000 7f142fe00640 1 -- 172.21.15.165:0/3601330178 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f14300587c0 con 0x7f143815db70 2024-04-25T10:15:57.437 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:57.437 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:57.438 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:57.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.436+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f141406b4d0 msgr2=0x7f141406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.436+0000 7f143e000640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f141406b4d0 0x7f141406d990 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f1428034000 tx=0x7f142802e000 comp rx=0 tx=0).stop 2024-04-25T10:15:57.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.436+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 msgr2=0x7f143806c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.436+0000 7f143e000640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 0x7f143806c2d0 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7f143000c6e0 tx=0x7f143000cba0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 shutdown_connections 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f141406b4d0 0x7f141406d990 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f143815db70 0x7f143806c2d0 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14380b6cb0 0x7f14380cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 --2- 172.21.15.165:0/3601330178 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14380b5a60 0x7f1438158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 >> 172.21.15.165:0/3601330178 conn(0x7f14380d0520 msgr2=0x7f14380b8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 shutdown_connections 2024-04-25T10:15:57.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.437+0000 7f143e000640 1 -- 172.21.15.165:0/3601330178 wait complete. 2024-04-25T10:15:57.457 DEBUG:tasks.cephfs.test_scrub_checks:current active=['b'] 2024-04-25T10:15:57.457 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:15:57.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.640+0000 7f564ec00640 1 Processor -- start 2024-04-25T10:15:57.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.641+0000 7f564ec00640 1 -- start start 2024-04-25T10:15:57.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5648144440 0x7f5648070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 0x7f564815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ec00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f564815bf30 con 0x7f5648071530 2024-04-25T10:15:57.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56480719a0 con 0x7f5648144440 2024-04-25T10:15:57.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564815c860 con 0x7f56480cda30 2024-04-25T10:15:57.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ce00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5648071530 0x7f56480cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41044/0 (socket says 172.21.15.165:41044) 2024-04-25T10:15:57.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.642+0000 7f564ce00640 1 -- 172.21.15.165:0/938703527 learned_addr learned my addr 172.21.15.165:0/938703527 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f564d800640 1 --2- 172.21.15.165:0/938703527 >> v2:172.21.15.172:3300/0 conn(0x7f5648144440 0x7f5648070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f564e200640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 0x7f564815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3283978586 0 0) 0x7f564815bf30 con 0x7f5648071530 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5630003520 con 0x7f5648071530 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3662545475 0 0) 0x7f5630003520 con 0x7f5648071530 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 msgr2=0x7f564815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 0x7f564815ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 >> v2:172.21.15.172:3300/0 conn(0x7f5648144440 msgr2=0x7f5648070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 --2- 172.21.15.165:0/938703527 >> v2:172.21.15.172:3300/0 conn(0x7f5648144440 0x7f5648070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f564815c210 con 0x7f5648071530 2024-04-25T10:15:57.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5634001670 con 0x7f5648071530 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 >> v1:172.21.15.165:6789/0 conn(0x7f5648071530 legacy=0x7f56480cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56300039b0 0x7f5630003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5630004800 0x7f5630004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 0x7f56300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480719a0 con 0x7f56300039b0 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564815c860 con 0x7f5630004800 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.643+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f563000ba40 con 0x7f56480cda30 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564d800640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56300039b0 0x7f5630003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564ce00640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5630004800 0x7f5630004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564d800640 1 -- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 msgr2=0x7f56300051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564d800640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56480cda30 0x7f56300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564d800640 1 -- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5630004800 msgr2=0x7f5630004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564d800640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5630004800 0x7f5630004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.644+0000 7f564d800640 1 -- 172.21.15.165:0/938703527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5630003520 con 0x7f56300039b0 2024-04-25T10:15:57.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.645+0000 7f564d800640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56300039b0 0x7f5630003dd0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f563c009ac0 tx=0x7f5630005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99510a58c9edf9a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.645+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c036070 con 0x7f56300039b0 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.645+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f563c031400 con 0x7f56300039b0 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.645+0000 7f5647e00640 1 -- 172.21.15.165:0/938703527 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563c009690 con 0x7f56300039b0 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564ec00640 1 -- 172.21.15.165:0/938703527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56300039b0 msgr2=0x7f5630003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564ec00640 1 --2- 172.21.15.165:0/938703527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56300039b0 0x7f5630003dd0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f563c009ac0 tx=0x7f5630005830 comp rx=0 tx=0).stop 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564e200640 1 -- 172.21.15.165:0/938703527 reap_dead start 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564ec00640 1 -- 172.21.15.165:0/938703527 shutdown_connections 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564ec00640 1 -- 172.21.15.165:0/938703527 >> 172.21.15.165:0/938703527 conn(0x7f564806abe0 msgr2=0x7f564806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564ec00640 1 -- 172.21.15.165:0/938703527 shutdown_connections 2024-04-25T10:15:57.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.646+0000 7f564ec00640 1 -- 172.21.15.165:0/938703527 wait complete. 2024-04-25T10:15:57.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.648+0000 7f564ec00640 1 Processor -- start 2024-04-25T10:15:57.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.648+0000 7f564ec00640 1 -- start start 2024-04-25T10:15:57.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56300039b0 0x7f56480c9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 0x7f56481f69e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f564815a620 0x7f564815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f564815bc30 con 0x7f56481f73c0 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56480cdea0 con 0x7f564815a620 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f564815ccf0 con 0x7f56300039b0 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 0x7f56481f69e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 0x7f56481f69e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53506/0 (socket says 172.21.15.165:53506) 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.649+0000 7f564ce00640 1 -- 172.21.15.165:0/352189838 learned_addr learned my addr 172.21.15.165:0/352189838 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564d800640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56300039b0 0x7f56480c9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564e200640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f564815a620 0x7f564815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564ce00640 1 -- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56300039b0 msgr2=0x7f56480c9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564ce00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56300039b0 0x7f56480c9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564ce00640 1 -- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f564815a620 msgr2=0x7f564815aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564ce00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f564815a620 0x7f564815aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564ce00640 1 -- 172.21.15.165:0/352189838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56480ca0c0 con 0x7f56481f73c0 2024-04-25T10:15:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564e200640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f564815a620 0x7f564815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f564ce00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 0x7f56481f69e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f563400d7e0 tx=0x7f563400dca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.650+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5634019070 con 0x7f56481f73c0 2024-04-25T10:15:57.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.651+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56340074e0 con 0x7f56481f73c0 2024-04-25T10:15:57.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.651+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56480c7690 con 0x7f56481f73c0 2024-04-25T10:15:57.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.651+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56340049e0 con 0x7f56481f73c0 2024-04-25T10:15:57.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.651+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56480c7c50 con 0x7f56481f73c0 2024-04-25T10:15:57.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.652+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5648073990 con 0x7f56481f73c0 2024-04-25T10:15:57.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.653+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f563401c070 con 0x7f56481f73c0 2024-04-25T10:15:57.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.654+0000 7f5645600640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f562806b4d0 0x7f562806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:57.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.654+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5634089ee0 con 0x7f56481f73c0 2024-04-25T10:15:57.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.654+0000 7f564d800640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f562806b4d0 0x7f562806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:57.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.655+0000 7f564d800640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f562806b4d0 0x7f562806d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f563c033000 tx=0x7f563c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:57.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.658+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5634058b30 con 0x7f56481f73c0 2024-04-25T10:15:57.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.922+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f564806bff0 con 0x7f56481f73c0 2024-04-25T10:15:57.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.923+0000 7f5645600640 1 -- 172.21.15.165:0/352189838 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1598 (secure 0 0 0) 0x7f5634058430 con 0x7f56481f73c0 2024-04-25T10:15:57.927 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:57.927 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":858,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":136,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076580426,"bytes_used":2612666368,"bytes_avail":770481446912,"bytes_total":773094113280,"write_bytes_sec":23649767,"read_op_per_sec":0,"write_op_per_sec":24},"fsmap":{"epoch":90,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"b","status":"up:active","gid":9234}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":33,"modified":"2024-04-25T10:15:56.412283+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{"scrub status":"paused"}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:15:57.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f562806b4d0 msgr2=0x7f562806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f562806b4d0 0x7f562806d990 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f563c033000 tx=0x7f563c008030 comp rx=0 tx=0).stop 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 msgr2=0x7f56481f69e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 0x7f56481f69e0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f563400d7e0 tx=0x7f563400dca0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 shutdown_connections 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f562806b4d0 0x7f562806d990 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f564815a620 0x7f564815aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f56481f73c0 0x7f56481f69e0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 --2- 172.21.15.165:0/352189838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f56300039b0 0x7f56480c9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 >> 172.21.15.165:0/352189838 conn(0x7f564806abe0 msgr2=0x7f5648141400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 shutdown_connections 2024-04-25T10:15:57.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:57.926+0000 7f564ec00640 1 -- 172.21.15.165:0/352189838 wait complete. 2024-04-25T10:15:57.947 INFO:tasks.cephfs.filesystem.ceph_manager:{'b': 'paused'} 2024-04-25T10:15:57.947 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:15:57.947 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:15:57.976 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:15:57.977 INFO:tasks.cephfs.test_scrub_checks:Waiting for promotion of one of the original standbys {'c', 'a', 'd'} 2024-04-25T10:15:57.977 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mds_beacon_grace 2024-04-25T10:15:58.187 INFO:teuthology.orchestra.run.smithi165.stdout:{"mds_beacon_grace":"15.000000"} 2024-04-25T10:15:58.187 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mds_beacon_grace": "15.000000" } 2024-04-25T10:15:58.187 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:15:58.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.387+0000 7fc301a00640 1 Processor -- start 2024-04-25T10:15:58.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.387+0000 7fc301a00640 1 -- start start 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.389+0000 7fc301a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc2fc0b3800 0x7fc2fc06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.389+0000 7fc301a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc0b3420 con 0x7fc2fc15c100 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.389+0000 7fc301a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2fc0b2810 con 0x7fc2fc06eac0 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.389+0000 7fc301a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b2af0 con 0x7fc2fc0b3800 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.389+0000 7fc2fbe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc2fc15c100 0x7fc2fc071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41052/0 (socket says 172.21.15.165:41052) 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.390+0000 7fc2fbe00640 1 -- 172.21.15.165:0/3765589817 learned_addr learned my addr 172.21.15.165:0/3765589817 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.390+0000 7fc2fb400640 1 --2- 172.21.15.165:0/3765589817 >> v2:172.21.15.172:3300/0 conn(0x7fc2fc0b3800 0x7fc2fc06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.390+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 825222410 0 0) 0x7fc2fc0b3420 con 0x7fc2fc15c100 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.390+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2e0003520 con 0x7fc2fc15c100 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.390+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3720104804 0 0) 0x7fc2e0003520 con 0x7fc2fc15c100 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 >> v2:172.21.15.172:3300/0 conn(0x7fc2fc0b3800 msgr2=0x7fc2fc06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 --2- 172.21.15.165:0/3765589817 >> v2:172.21.15.172:3300/0 conn(0x7fc2fc0b3800 0x7fc2fc06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 >> v1:172.21.15.172:6789/0 conn(0x7fc2fc06eac0 legacy=0x7fc2fc06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc0b2dd0 con 0x7fc2fc15c100 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc2e4001670 con 0x7fc2fc15c100 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 >> v1:172.21.15.165:6789/0 conn(0x7fc2fc15c100 legacy=0x7fc2fc071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e00051c0 0x7fc2e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b2af0 con 0x7fc2e0004800 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e000c060 con 0x7fc2e00039b0 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e000c260 con 0x7fc2e00051c0 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2fb400640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2fb400640 1 -- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e00051c0 msgr2=0x7fc2e000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2fb400640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2e00051c0 0x7fc2e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2fbe00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.391+0000 7fc2fb400640 1 -- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00039b0 msgr2=0x7fc2e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.392+0000 7fc2fb400640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.392+0000 7fc2fb400640 1 -- 172.21.15.165:0/3765589817 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0003520 con 0x7fc2e0004800 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.392+0000 7fc301000640 1 -- 172.21.15.165:0/3765589817 reap_dead start 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.392+0000 7fc2fbe00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2e00039b0 0x7fc2e0003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.393+0000 7fc2fb400640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2e0004c80 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009f90 tx=0x7fc2e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3bd1789b7b8380b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.393+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec009c10 con 0x7fc2e0004800 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.393+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec036070 con 0x7fc2e0004800 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.393+0000 7fc2faa00640 1 -- 172.21.15.165:0/3765589817 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec031850 con 0x7fc2e0004800 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.393+0000 7fc301a00640 1 -- 172.21.15.165:0/3765589817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 msgr2=0x7fc2e0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.393+0000 7fc301a00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2e0004c80 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec009f90 tx=0x7fc2e0005830 comp rx=0 tx=0).stop 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.394+0000 7fc301a00640 1 -- 172.21.15.165:0/3765589817 shutdown_connections 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.394+0000 7fc301a00640 1 --2- 172.21.15.165:0/3765589817 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2e0004c80 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.394+0000 7fc301a00640 1 -- 172.21.15.165:0/3765589817 >> 172.21.15.165:0/3765589817 conn(0x7fc2fc06abe0 msgr2=0x7fc2fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.394+0000 7fc301a00640 1 -- 172.21.15.165:0/3765589817 shutdown_connections 2024-04-25T10:15:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.394+0000 7fc301a00640 1 -- 172.21.15.165:0/3765589817 wait complete. 2024-04-25T10:15:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.395+0000 7fc301a00640 1 Processor -- start 2024-04-25T10:15:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.396+0000 7fc301a00640 1 -- start start 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.396+0000 7fc301a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2fc146350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.396+0000 7fc301a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2fc157ed0 0x7fc2fc0b02d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.396+0000 7fc301a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2fc158350 0x7fc2fc0b0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc301a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0711b0 con 0x7fc2e0004800 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc301a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc0b24e0 con 0x7fc2fc157ed0 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc301a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2fc071330 con 0x7fc2fc158350 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2fc146350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2fc146350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53522/0 (socket says 172.21.15.165:53522) 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 -- 172.21.15.165:0/2947567315 learned_addr learned my addr 172.21.15.165:0/2947567315 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:15:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 -- 172.21.15.165:0/2947567315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2fc158350 msgr2=0x7fc2fc0b0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2fc158350 0x7fc2fc0b0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 -- 172.21.15.165:0/2947567315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2fc157ed0 msgr2=0x7fc2fc0b02d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2fc157ed0 0x7fc2fc0b02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.397+0000 7fc2fbe00640 1 -- 172.21.15.165:0/2947567315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2fc158890 con 0x7fc2e0004800 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.398+0000 7fc2fbe00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2fc146350 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400d800 tx=0x7fc2e400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.398+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4019070 con 0x7fc2e0004800 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.398+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e40074e0 con 0x7fc2e0004800 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.398+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2fc158b70 con 0x7fc2e0004800 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.398+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e4004a20 con 0x7fc2e0004800 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.398+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2fc159130 con 0x7fc2e0004800 2024-04-25T10:15:58.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.399+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2fc073990 con 0x7fc2e0004800 2024-04-25T10:15:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.400+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc2e401c070 con 0x7fc2e0004800 2024-04-25T10:15:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.401+0000 7fc2f3e00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:15:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.401+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2e4089e40 con 0x7fc2e0004800 2024-04-25T10:15:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.401+0000 7fc2fb400640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:15:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.402+0000 7fc2fb400640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2d806b4d0 0x7fc2d806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec035000 tx=0x7fc2ec02d000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:15:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.405+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc2e4058a90 con 0x7fc2e0004800 2024-04-25T10:15:58.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.644+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2fc156eb0 con 0x7fc2e0004800 2024-04-25T10:15:58.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.645+0000 7fc2f3e00640 1 -- 172.21.15.165:0/2947567315 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7fc2e4058390 con 0x7fc2e0004800 2024-04-25T10:15:58.649 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:15:58.649 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:15:58.649 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:15:58.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.648+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2d806b4d0 msgr2=0x7fc2d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.648+0000 7fc301a00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2d806b4d0 0x7fc2d806d990 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec035000 tx=0x7fc2ec02d000 comp rx=0 tx=0).stop 2024-04-25T10:15:58.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 msgr2=0x7fc2fc146350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2fc146350 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fc2e400d800 tx=0x7fc2e400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 shutdown_connections 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc2d806b4d0 0x7fc2d806d990 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2fc158350 0x7fc2fc0b0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2fc157ed0 0x7fc2fc0b02d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 --2- 172.21.15.165:0/2947567315 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2e0004800 0x7fc2fc146350 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 >> 172.21.15.165:0/2947567315 conn(0x7fc2fc06abe0 msgr2=0x7fc2fc162b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 shutdown_connections 2024-04-25T10:15:58.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:15:58.649+0000 7fc301a00640 1 -- 172.21.15.165:0/2947567315 wait complete. 2024-04-25T10:15:58.669 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:16:00.215 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:00.223 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:03.670 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:03.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.876+0000 7f246d800640 1 Processor -- start 2024-04-25T10:16:03.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.877+0000 7f246d800640 1 -- start start 2024-04-25T10:16:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f246d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f246d800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f24680ce3c0 0x7f246806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f246d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2468071480 con 0x7f2468154ee0 2024-04-25T10:16:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f246d800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2468072090 con 0x7f24680ce3c0 2024-04-25T10:16:03.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f246d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2468071230 con 0x7f24680cfec0 2024-04-25T10:16:03.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f2466a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2468154ee0 0x7f24680cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41064/0 (socket says 172.21.15.165:41064) 2024-04-25T10:16:03.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f2466a00640 1 -- 172.21.15.165:0/2666234413 learned_addr learned my addr 172.21.15.165:0/2666234413 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:03.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.878+0000 7f2467400640 1 --2- 172.21.15.165:0/2666234413 >> v2:172.21.15.165:3300/0 conn(0x7f24680ce3c0 0x7f246806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f245e000640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f2466000640 1 -- 172.21.15.165:0/2666234413 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1020347019 0 0) 0x7f2468071480 con 0x7f2468154ee0 2024-04-25T10:16:03.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f2467400640 1 -- 172.21.15.165:0/2666234413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 msgr2=0x7f24680cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:03.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f2467400640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f2467400640 1 -- 172.21.15.165:0/2666234413 >> v1:172.21.15.165:6789/0 conn(0x7f2468154ee0 legacy=0x7f24680cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f2467400640 1 -- 172.21.15.165:0/2666234413 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2468071760 con 0x7f24680ce3c0 2024-04-25T10:16:03.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.879+0000 7f245e000640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.880+0000 7f2467400640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24680ce3c0 0x7f246806d6f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d90 tx=0x7f246002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5514de5ec5d44255 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.880+0000 7f2466000640 1 -- 172.21.15.165:0/2666234413 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460007740 con 0x7f24680ce3c0 2024-04-25T10:16:03.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.880+0000 7f2466000640 1 -- 172.21.15.165:0/2666234413 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24600078e0 con 0x7f24680ce3c0 2024-04-25T10:16:03.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.880+0000 7f2466000640 1 -- 172.21.15.165:0/2666234413 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460007cb0 con 0x7f24680ce3c0 2024-04-25T10:16:03.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 -- 172.21.15.165:0/2666234413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24680ce3c0 msgr2=0x7f246806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24680ce3c0 0x7f246806d6f0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d90 tx=0x7f246002a660 comp rx=0 tx=0).stop 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 -- 172.21.15.165:0/2666234413 shutdown_connections 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f24680ce3c0 0x7f246806d6f0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 --2- 172.21.15.165:0/2666234413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 -- 172.21.15.165:0/2666234413 >> 172.21.15.165:0/2666234413 conn(0x7f246806abe0 msgr2=0x7f246806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 -- 172.21.15.165:0/2666234413 shutdown_connections 2024-04-25T10:16:03.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.881+0000 7f246d800640 1 -- 172.21.15.165:0/2666234413 wait complete. 2024-04-25T10:16:03.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.883+0000 7f246d800640 1 Processor -- start 2024-04-25T10:16:03.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.883+0000 7f246d800640 1 -- start start 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f246d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680ce3c0 0x7f24680af800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f246d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f246d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 0x7f24680a9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f246d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2468071d20 con 0x7f2468154ee0 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f246d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24680ce800 con 0x7f24680ce3c0 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f246d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24681552c0 con 0x7f24680cfec0 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f2467400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 0x7f24680a9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f2467400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 0x7f24680a9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53550/0 (socket says 172.21.15.165:53550) 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f2467400640 1 -- 172.21.15.165:0/2895388757 learned_addr learned my addr 172.21.15.165:0/2895388757 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f245e000640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.884+0000 7f2466a00640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680ce3c0 0x7f24680af800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2467400640 1 -- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 msgr2=0x7f24680aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2467400640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2467400640 1 -- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680ce3c0 msgr2=0x7f24680af800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2467400640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680ce3c0 0x7f24680af800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2467400640 1 -- 172.21.15.165:0/2895388757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246814f9d0 con 0x7f2468154ee0 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2466a00640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680ce3c0 0x7f24680af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f2467400640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 0x7f24680a9950 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d90 tx=0x7f2460004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460040070 con 0x7f2468154ee0 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2460031c40 con 0x7f2468154ee0 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460003990 con 0x7f2468154ee0 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.885+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24681f6dd0 con 0x7f2468154ee0 2024-04-25T10:16:03.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.886+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24681f4010 con 0x7f2468154ee0 2024-04-25T10:16:03.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.887+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2468073990 con 0x7f2468154ee0 2024-04-25T10:16:03.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.887+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2460039040 con 0x7f2468154ee0 2024-04-25T10:16:03.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.888+0000 7f245f400640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f244406b4d0 0x7f244406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:03.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.888+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f24600ab410 con 0x7f2468154ee0 2024-04-25T10:16:03.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.889+0000 7f2466a00640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f244406b4d0 0x7f244406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:03.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.889+0000 7f2466a00640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f244406b4d0 0x7f244406d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2458000f60 tx=0x7f2458009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:03.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:03.893+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f246007a060 con 0x7f2468154ee0 2024-04-25T10:16:04.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.131+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f246806d360 con 0x7f2468154ee0 2024-04-25T10:16:04.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.132+0000 7f245f400640 1 -- 172.21.15.165:0/2895388757 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f2460079960 con 0x7f2468154ee0 2024-04-25T10:16:04.135 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:04.135 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:04.135 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:16:04.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f244406b4d0 msgr2=0x7f244406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f244406b4d0 0x7f244406d990 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f2458000f60 tx=0x7f2458009040 comp rx=0 tx=0).stop 2024-04-25T10:16:04.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 msgr2=0x7f24680a9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:04.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 0x7f24680a9950 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f2460009d90 tx=0x7f2460004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:04.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 shutdown_connections 2024-04-25T10:16:04.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f244406b4d0 0x7f244406d990 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.134+0000 7f246d800640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2468154ee0 0x7f24680a9950 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.135+0000 7f246d800640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f24680cfec0 0x7f24680aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.135+0000 7f246d800640 1 --2- 172.21.15.165:0/2895388757 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f24680ce3c0 0x7f24680af800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:04.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.135+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 >> 172.21.15.165:0/2895388757 conn(0x7f246806abe0 msgr2=0x7f24680b45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:04.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.135+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 shutdown_connections 2024-04-25T10:16:04.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:04.135+0000 7f246d800640 1 -- 172.21.15.165:0/2895388757 wait complete. 2024-04-25T10:16:04.157 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:16:09.158 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:09.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.359+0000 7f0071a00640 1 Processor -- start 2024-04-25T10:16:09.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.359+0000 7f0071a00640 1 -- start start 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f0071a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 0x7f006c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f0071a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c071480 con 0x7f006c0ce3c0 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f0071a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f006c071760 con 0x7f006c154ee0 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f0071a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c072090 con 0x7f006c0cfec0 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f006b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f006c0ce3c0 0x7f006c06d6f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50276/0 (socket says 172.21.15.165:50276) 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f006b400640 1 -- 172.21.15.165:0/393158482 learned_addr learned my addr 172.21.15.165:0/393158482 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:09.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.361+0000 7f0063e00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 0x7f006c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565729065 0 0) 0x7f006c071480 con 0x7f006c0ce3c0 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0050003520 con 0x7f006c0ce3c0 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2013860752 0 0) 0x7f0050003520 con 0x7f006c0ce3c0 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 msgr2=0x7f006c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 0x7f006c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 >> v1:172.21.15.172:6789/0 conn(0x7f006c154ee0 legacy=0x7f006c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c071a40 con 0x7f006c0ce3c0 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f00640019f0 con 0x7f006c0ce3c0 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 >> v1:172.21.15.165:6789/0 conn(0x7f006c0ce3c0 legacy=0x7f006c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f0050003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0050004800 0x7f0050004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 0x7f0050005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005000bb50 con 0x7f00500039e0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f005000bcd0 con 0x7f0050004800 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f005000bed0 con 0x7f006c0cfec0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.362+0000 7f006aa00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f0050003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.363+0000 7f0063e00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0050004800 0x7f0050004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.363+0000 7f006aa00640 1 -- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 msgr2=0x7f0050005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.363+0000 7f006aa00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0cfec0 0x7f0050005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.363+0000 7f006aa00640 1 -- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0050004800 msgr2=0x7f0050004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.363+0000 7f006aa00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0050004800 0x7f0050004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.363+0000 7f006aa00640 1 -- 172.21.15.165:0/393158482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0050003520 con 0x7f00500039e0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.364+0000 7f006aa00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f0050003e00 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f005c001750 tx=0x7f0050005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fcbc90cf8cc3436 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.364+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c001370 con 0x7f00500039e0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.364+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c0052d0 con 0x7f00500039e0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.364+0000 7f006a000640 1 -- 172.21.15.165:0/393158482 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c02abe0 con 0x7f00500039e0 2024-04-25T10:16:09.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f0071a00640 1 -- 172.21.15.165:0/393158482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 msgr2=0x7f0050003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f0071a00640 1 --2- 172.21.15.165:0/393158482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f0050003e00 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7f005c001750 tx=0x7f0050005830 comp rx=0 tx=0).stop 2024-04-25T10:16:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f006b400640 1 -- 172.21.15.165:0/393158482 reap_dead start 2024-04-25T10:16:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f0071a00640 1 -- 172.21.15.165:0/393158482 shutdown_connections 2024-04-25T10:16:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f0071a00640 1 -- 172.21.15.165:0/393158482 >> 172.21.15.165:0/393158482 conn(0x7f006c06abe0 msgr2=0x7f006c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f0071a00640 1 -- 172.21.15.165:0/393158482 shutdown_connections 2024-04-25T10:16:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.365+0000 7f0071a00640 1 -- 172.21.15.165:0/393158482 wait complete. 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.367+0000 7f0071a00640 1 Processor -- start 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.367+0000 7f0071a00640 1 -- start start 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0071a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f006c0ad3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0071a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f006c0ab950 0x7f006c0b36b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0071a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0abdd0 0x7f006c0b3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0071a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0ce800 con 0x7f00500039e0 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0071a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c071230 con 0x7f006c0ab950 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0071a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006c1552c0 con 0x7f006c0abdd0 2024-04-25T10:16:09.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f006c0ad3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f006c0ad3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56014/0 (socket says 172.21.15.165:56014) 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 -- 172.21.15.165:0/1427802332 learned_addr learned my addr 172.21.15.165:0/1427802332 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0063e00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f006c0ab950 0x7f006c0b36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006b400640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0abdd0 0x7f006c0b3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 -- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0abdd0 msgr2=0x7f006c0b3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0abdd0 0x7f006c0b3bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 -- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f006c0ab950 msgr2=0x7f006c0b36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f006c0ab950 0x7f006c0b36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f006aa00640 1 -- 172.21.15.165:0/1427802332 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c1f4a10 con 0x7f00500039e0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.369+0000 7f0063e00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f006c0ab950 0x7f006c0b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.370+0000 7f006aa00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f006c0ad3b0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f005c02a590 tx=0x7f005c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.370+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c03f020 con 0x7f00500039e0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.370+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f005c005480 con 0x7f00500039e0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.370+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f005c037040 con 0x7f00500039e0 2024-04-25T10:16:09.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.370+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c0aae70 con 0x7f00500039e0 2024-04-25T10:16:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.370+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c0ab430 con 0x7f00500039e0 2024-04-25T10:16:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.371+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c073990 con 0x7f00500039e0 2024-04-25T10:16:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.372+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f005c042020 con 0x7f00500039e0 2024-04-25T10:16:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.373+0000 7f0062a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f004806b500 0x7f004806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:09.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.373+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f005c0aa300 con 0x7f00500039e0 2024-04-25T10:16:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.373+0000 7f0063e00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f004806b500 0x7f004806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:09.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.374+0000 7f0063e00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f004806b500 0x7f004806d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f006c153660 tx=0x7f0054008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:09.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.378+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f005c049020 con 0x7f00500039e0 2024-04-25T10:16:09.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.613+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f006c0aaab0 con 0x7f00500039e0 2024-04-25T10:16:09.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.614+0000 7f0062a00640 1 -- 172.21.15.165:0/1427802332 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) v1 ==== 94+0+4952 (secure 0 0 0) 0x7f005c078ec0 con 0x7f00500039e0 2024-04-25T10:16:09.618 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:09.618 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":90,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9225,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":90,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:15:14.138367+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9234},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9234":{"gid":9234,"name":"b","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6835/2186062364","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":2186062364},{"type":"v1","addr":"172.21.15.172:6835","nonce":2186062364}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:09.619 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 90 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f004806b500 msgr2=0x7f004806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f004806b500 0x7f004806d9c0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f006c153660 tx=0x7f0054008040 comp rx=0 tx=0).stop 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 msgr2=0x7f006c0ad3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f006c0ad3b0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f005c02a590 tx=0x7f005c001120 comp rx=0 tx=0).stop 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 shutdown_connections 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f004806b500 0x7f004806d9c0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f006c0abdd0 0x7f006c0b3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f006c0ab950 0x7f006c0b36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 --2- 172.21.15.165:0/1427802332 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f00500039e0 0x7f006c0ad3b0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 >> 172.21.15.165:0/1427802332 conn(0x7f006c06abe0 msgr2=0x7f006c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:09.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.617+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 shutdown_connections 2024-04-25T10:16:09.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:09.618+0000 7f0071a00640 1 -- 172.21.15.165:0/1427802332 wait complete. 2024-04-25T10:16:09.640 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:16:14.641 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:14.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.843+0000 7f304ae00640 1 Processor -- start 2024-04-25T10:16:14.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.843+0000 7f304ae00640 1 -- start start 2024-04-25T10:16:14.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.844+0000 7f304ae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f3044070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.844+0000 7f304ae00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3044071530 0x7f30440cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.844+0000 7f304ae00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f304415bf30 con 0x7f30440cd860 2024-04-25T10:16:14.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.844+0000 7f304ae00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f304415c860 con 0x7f3044071530 2024-04-25T10:16:14.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.844+0000 7f304ae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3044071970 con 0x7f3044144440 2024-04-25T10:16:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3044071530 0x7f30440cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3044071530 0x7f30440cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56022/0 (socket says 172.21.15.165:56022) 2024-04-25T10:16:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049000640 1 -- 172.21.15.165:0/4248232336 learned_addr learned my addr 172.21.15.165:0/4248232336 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f304a400640 1 --1- 172.21.15.165:0/4248232336 >> v1:172.21.15.165:6789/0 conn(0x7f30440cd860 0x7f304406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50280/0 (socket says 172.21.15.165:50280) 2024-04-25T10:16:14.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049a00640 1 --2- 172.21.15.165:0/4248232336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f3044070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3043e00640 1 -- 172.21.15.165:0/4248232336 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3050590534 0 0) 0x7f304415bf30 con 0x7f30440cd860 2024-04-25T10:16:14.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049000640 1 -- 172.21.15.165:0/4248232336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 msgr2=0x7f3044070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:14.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049000640 1 --2- 172.21.15.165:0/4248232336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f3044070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.845+0000 7f3049000640 1 -- 172.21.15.165:0/4248232336 >> v1:172.21.15.165:6789/0 conn(0x7f30440cd860 legacy=0x7f304406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.846+0000 7f3049000640 1 -- 172.21.15.165:0/4248232336 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f304415c210 con 0x7f3044071530 2024-04-25T10:16:14.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.847+0000 7f3049000640 1 --2- 172.21.15.165:0/4248232336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044071530 0x7f30440cd320 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3030009870 tx=0x7f303002a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af5735be4fdd716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.847+0000 7f3043e00640 1 -- 172.21.15.165:0/4248232336 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030037070 con 0x7f3044071530 2024-04-25T10:16:14.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.847+0000 7f3043e00640 1 -- 172.21.15.165:0/4248232336 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f303002ae50 con 0x7f3044071530 2024-04-25T10:16:14.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.847+0000 7f3043e00640 1 -- 172.21.15.165:0/4248232336 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030032860 con 0x7f3044071530 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 -- 172.21.15.165:0/4248232336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044071530 msgr2=0x7f30440cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 --2- 172.21.15.165:0/4248232336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044071530 0x7f30440cd320 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f3030009870 tx=0x7f303002a300 comp rx=0 tx=0).stop 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 -- 172.21.15.165:0/4248232336 shutdown_connections 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 --2- 172.21.15.165:0/4248232336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3044071530 0x7f30440cd320 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 --2- 172.21.15.165:0/4248232336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f3044070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 -- 172.21.15.165:0/4248232336 >> 172.21.15.165:0/4248232336 conn(0x7f304406abe0 msgr2=0x7f304406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 -- 172.21.15.165:0/4248232336 shutdown_connections 2024-04-25T10:16:14.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.848+0000 7f304ae00640 1 -- 172.21.15.165:0/4248232336 wait complete. 2024-04-25T10:16:14.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.850+0000 7f304ae00640 1 Processor -- start 2024-04-25T10:16:14.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.850+0000 7f304ae00640 1 -- start start 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f304ae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3044071530 0x7f30441f41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f304ae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 0x7f30440b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f304ae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f30441e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f304ae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30440cdcd0 con 0x7f30440cd860 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f304ae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f304415bc30 con 0x7f3044071530 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f304ae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30440cde50 con 0x7f3044144440 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 0x7f30440b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 0x7f30440b25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:56038/0 (socket says 172.21.15.165:56038) 2024-04-25T10:16:14.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 -- 172.21.15.165:0/3679501501 learned_addr learned my addr 172.21.15.165:0/3679501501 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049a00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3044071530 0x7f30441f41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 -- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 msgr2=0x7f30441e4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f30441e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 -- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3044071530 msgr2=0x7f30441f41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3044071530 0x7f30441f41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.851+0000 7f3049000640 1 -- 172.21.15.165:0/3679501501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30440b2b20 con 0x7f30440cd860 2024-04-25T10:16:14.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.852+0000 7f3049000640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 0x7f30440b25e0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3030035040 tx=0x7f30300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.852+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030037040 con 0x7f30440cd860 2024-04-25T10:16:14.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.852+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030004230 con 0x7f30440cd860 2024-04-25T10:16:14.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.852+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030032e20 con 0x7f30440cd860 2024-04-25T10:16:14.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.852+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30440b2df0 con 0x7f30440cd860 2024-04-25T10:16:14.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.852+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30440b33b0 con 0x7f30440cd860 2024-04-25T10:16:14.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.854+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f304413ddd0 con 0x7f30440cd860 2024-04-25T10:16:14.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.855+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f303003a070 con 0x7f30440cd860 2024-04-25T10:16:14.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.856+0000 7f3041600640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f302406b4d0 0x7f302406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:14.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.856+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f30300aaac0 con 0x7f30440cd860 2024-04-25T10:16:14.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.857+0000 7f3049a00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f302406b4d0 0x7f302406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:14.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.857+0000 7f3049a00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f302406b4d0 0x7f302406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3038006fd0 tx=0x7f3038008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:14.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:14.862+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f304413ddd0 con 0x7f30440cd860 2024-04-25T10:16:15.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.096+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f30440c6850 con 0x7f30440cd860 2024-04-25T10:16:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.098+0000 7f3041600640 1 -- 172.21.15.165:0/3679501501 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 91 v91) v1 ==== 94+0+4160 (secure 0 0 0) 0x7f3030041020 con 0x7f30440cd860 2024-04-25T10:16:15.101 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:15.101 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":91,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":91,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:14.197041+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:replay","state_seq":1,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:15.101 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 91 2024-04-25T10:16:15.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.100+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f302406b4d0 msgr2=0x7f302406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:15.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.100+0000 7f304ae00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f302406b4d0 0x7f302406d990 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f3038006fd0 tx=0x7f3038008040 comp rx=0 tx=0).stop 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 msgr2=0x7f30440b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 0x7f30440b25e0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f3030035040 tx=0x7f30300046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 shutdown_connections 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f302406b4d0 0x7f302406d990 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3044144440 0x7f30441e4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30440cd860 0x7f30440b25e0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 --2- 172.21.15.165:0/3679501501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3044071530 0x7f30441f41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 >> 172.21.15.165:0/3679501501 conn(0x7f304406abe0 msgr2=0x7f304413d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 shutdown_connections 2024-04-25T10:16:15.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:15.101+0000 7f304ae00640 1 -- 172.21.15.165:0/3679501501 wait complete. 2024-04-25T10:16:15.122 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=45.0 retry_count=0)... 2024-04-25T10:16:20.123 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:20.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.329+0000 7ff283200640 1 Processor -- start 2024-04-25T10:16:20.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.330+0000 7ff283200640 1 -- start start 2024-04-25T10:16:20.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff283200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff27c15dd30 0x7ff27c0b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff283200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 0x7ff27c0b6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff283200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27c0d1df0 con 0x7ff27c0b9770 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff283200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff27c0d20d0 con 0x7ff27c15dd30 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff283200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0d23b0 con 0x7ff27c0d0ad0 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff27b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff27c0b9770 0x7ff27c0d03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33888/0 (socket says 172.21.15.165:33888) 2024-04-25T10:16:20.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.331+0000 7ff27b400640 1 -- 172.21.15.165:0/2805132706 learned_addr learned my addr 172.21.15.165:0/2805132706 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27be00640 1 --2- 172.21.15.165:0/2805132706 >> v2:172.21.15.172:3300/0 conn(0x7ff27c15dd30 0x7ff27c0b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1523086597 0 0) 0x7ff27c0d1df0 con 0x7ff27c0b9770 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff281200640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 0x7ff27c0b6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff260003520 con 0x7ff27c0b9770 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3055207366 0 0) 0x7ff260003520 con 0x7ff27c0b9770 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 msgr2=0x7ff27c0b6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 0x7ff27c0b6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 >> v2:172.21.15.172:3300/0 conn(0x7ff27c15dd30 msgr2=0x7ff27c0b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 --2- 172.21.15.165:0/2805132706 >> v2:172.21.15.172:3300/0 conn(0x7ff27c15dd30 0x7ff27c0b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c0d2690 con 0x7ff27c0b9770 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff2680017f0 con 0x7ff27c0b9770 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 >> v1:172.21.15.165:6789/0 conn(0x7ff27c0b9770 legacy=0x7ff27c0d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 0x7ff2600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260003f80 0x7ff260004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff260004d40 0x7ff2600051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0d20d0 con 0x7ff260004d40 2024-04-25T10:16:20.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0d23b0 con 0x7ff260003f80 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.332+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26000bb20 con 0x7ff27c0d0ad0 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff281200640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff260004d40 0x7ff2600051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff27be00640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 0x7ff2600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff27b400640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260003f80 0x7ff260004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff281200640 1 -- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 msgr2=0x7ff2600039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff281200640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0d0ad0 0x7ff2600039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff281200640 1 -- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260003f80 msgr2=0x7ff260004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff281200640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260003f80 0x7ff260004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff281200640 1 -- 172.21.15.165:0/2805132706 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260003520 con 0x7ff260004d40 2024-04-25T10:16:20.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.333+0000 7ff27b400640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260003f80 0x7ff260004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.334+0000 7ff281200640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff260004d40 0x7ff2600051c0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7ff274007360 tx=0x7ff260005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce873d00675247fb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff274009be0 con 0x7ff260004d40 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff274036600 con 0x7ff260004d40 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff27aa00640 1 -- 172.21.15.165:0/2805132706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2740369d0 con 0x7ff260004d40 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff283200640 1 -- 172.21.15.165:0/2805132706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff260004d40 msgr2=0x7ff2600051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff283200640 1 --2- 172.21.15.165:0/2805132706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff260004d40 0x7ff2600051c0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7ff274007360 tx=0x7ff260005830 comp rx=0 tx=0).stop 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff281200640 1 -- 172.21.15.165:0/2805132706 reap_dead start 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff283200640 1 -- 172.21.15.165:0/2805132706 shutdown_connections 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff283200640 1 -- 172.21.15.165:0/2805132706 >> 172.21.15.165:0/2805132706 conn(0x7ff27c061c30 msgr2=0x7ff27c06c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff283200640 1 -- 172.21.15.165:0/2805132706 shutdown_connections 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.335+0000 7ff283200640 1 -- 172.21.15.165:0/2805132706 wait complete. 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.337+0000 7ff283200640 1 Processor -- start 2024-04-25T10:16:20.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.337+0000 7ff283200640 1 -- start start 2024-04-25T10:16:20.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff283200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260004d40 0x7ff27c155f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff283200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 0x7ff27c154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff283200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0cc180 0x7ff27c156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff283200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0d1b90 con 0x7ff27c1549b0 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff283200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0d1a10 con 0x7ff260004d40 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff283200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27c0d2e90 con 0x7ff27c0cc180 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff27b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 0x7ff27c154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff27b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 0x7ff27c154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50488/0 (socket says 172.21.15.165:50488) 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.338+0000 7ff27b400640 1 -- 172.21.15.165:0/3876845527 learned_addr learned my addr 172.21.15.165:0/3876845527 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27be00640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260004d40 0x7ff27c155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff281200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0cc180 0x7ff27c156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27b400640 1 -- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0cc180 msgr2=0x7ff27c156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27b400640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0cc180 0x7ff27c156e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27b400640 1 -- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260004d40 msgr2=0x7ff27c155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27b400640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260004d40 0x7ff27c155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27b400640 1 -- 172.21.15.165:0/3876845527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27c154ee0 con 0x7ff27c1549b0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff281200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0cc180 0x7ff27c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff27b400640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 0x7ff27c154290 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7ff26800d870 tx=0x7ff26800dd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.339+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff268007480 con 0x7ff27c1549b0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.340+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff268017070 con 0x7ff27c1549b0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.340+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680059a0 con 0x7ff27c1549b0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.340+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff27c1551c0 con 0x7ff27c1549b0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.340+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff27c155780 con 0x7ff27c1549b0 2024-04-25T10:16:20.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.341+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff27c073990 con 0x7ff27c1549b0 2024-04-25T10:16:20.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.342+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff268005b40 con 0x7ff27c1549b0 2024-04-25T10:16:20.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.343+0000 7ff273e00640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff25c06b4d0 0x7ff25c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.343+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff26808a610 con 0x7ff27c1549b0 2024-04-25T10:16:20.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.343+0000 7ff27be00640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff25c06b4d0 0x7ff25c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.344+0000 7ff27be00640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff25c06b4d0 0x7ff25c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009a90 tx=0x7ff26c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.347+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff268007620 con 0x7ff27c1549b0 2024-04-25T10:16:20.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.581+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff27c15d1c0 con 0x7ff27c1549b0 2024-04-25T10:16:20.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.582+0000 7ff273e00640 1 -- 172.21.15.165:0/3876845527 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4161 (secure 0 0 0) 0x7ff268059210 con 0x7ff27c1549b0 2024-04-25T10:16:20.585 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:20.586 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:16.251260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:20.586 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 94 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff25c06b4d0 msgr2=0x7ff25c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff25c06b4d0 0x7ff25c06d990 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7ff26c009a90 tx=0x7ff26c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 msgr2=0x7ff27c154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 0x7ff27c154290 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7ff26800d870 tx=0x7ff26800dd30 comp rx=0 tx=0).stop 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 shutdown_connections 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff25c06b4d0 0x7ff25c06d990 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff27c0cc180 0x7ff27c156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff27c1549b0 0x7ff27c154290 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 --2- 172.21.15.165:0/3876845527 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff260004d40 0x7ff27c155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 >> 172.21.15.165:0/3876845527 conn(0x7ff27c061c30 msgr2=0x7ff27c0b8010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.585+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 shutdown_connections 2024-04-25T10:16:20.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.586+0000 7ff283200640 1 -- 172.21.15.165:0/3876845527 wait complete. 2024-04-25T10:16:20.607 DEBUG:tasks.ceph_test_case:wait_until_true: success in 20s and 0 retries 2024-04-25T10:16:20.607 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:20.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.798+0000 7fb502400640 1 Processor -- start 2024-04-25T10:16:20.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.798+0000 7fb502400640 1 -- start start 2024-04-25T10:16:20.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.799+0000 7fb502400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4fc06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.799+0000 7fb502400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0b3420 con 0x7fb4fc06eac0 2024-04-25T10:16:20.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb502400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4fc0b2810 con 0x7fb4fc15c100 2024-04-25T10:16:20.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb502400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b2af0 con 0x7fb4fc0b3800 2024-04-25T10:16:20.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb501a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb4fc06eac0 0x7fb4fc06d2f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33896/0 (socket says 172.21.15.165:33896) 2024-04-25T10:16:20.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb501a00640 1 -- 172.21.15.165:0/2933958609 learned_addr learned my addr 172.21.15.165:0/2933958609 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb4f3e00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4fc06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 259288588 0 0) 0x7fb4fc0b3420 con 0x7fb4fc06eac0 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.800+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e0003520 con 0x7fb4fc06eac0 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2665628282 0 0) 0x7fb4e0003520 con 0x7fb4fc06eac0 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 msgr2=0x7fb4fc06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4fc06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 >> v1:172.21.15.172:6789/0 conn(0x7fb4fc15c100 legacy=0x7fb4fc071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc0b2dd0 con 0x7fb4fc06eac0 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb4f40019f0 con 0x7fb4fc06eac0 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4f3e00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4fc06e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 >> v1:172.21.15.165:6789/0 conn(0x7fb4fc06eac0 legacy=0x7fb4fc06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e00039e0 0x7fb4e0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4e0004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4e0004d40 0x7fb4e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b2af0 con 0x7fb4e0004d40 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e000bb50 con 0x7fb4e00039e0 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4e000bd20 con 0x7fb4fc0b3800 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.801+0000 7fb501a00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4e0004d40 0x7fb4e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb501000640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e00039e0 0x7fb4e0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb4f3e00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4e0004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb501a00640 1 -- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 msgr2=0x7fb4e0004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb501a00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc0b3800 0x7fb4e0004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb501a00640 1 -- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e00039e0 msgr2=0x7fb4e0003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:20.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb501a00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e00039e0 0x7fb4e0003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.802+0000 7fb501a00640 1 -- 172.21.15.165:0/2933958609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4e0003520 con 0x7fb4e0004d40 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.803+0000 7fb501a00640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4e0004d40 0x7fb4e00051c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400a790 tx=0x7fb4e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c7c18a2dcaaab server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.803+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f400a3b0 con 0x7fb4e0004d40 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.803+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f402f070 con 0x7fb4e0004d40 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.803+0000 7fb4fbe00640 1 -- 172.21.15.165:0/2933958609 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f402bbc0 con 0x7fb4e0004d40 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.803+0000 7fb502400640 1 -- 172.21.15.165:0/2933958609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4e0004d40 msgr2=0x7fb4e00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.803+0000 7fb502400640 1 --2- 172.21.15.165:0/2933958609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4e0004d40 0x7fb4e00051c0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400a790 tx=0x7fb4e0005830 comp rx=0 tx=0).stop 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.804+0000 7fb501a00640 1 -- 172.21.15.165:0/2933958609 reap_dead start 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.804+0000 7fb502400640 1 -- 172.21.15.165:0/2933958609 shutdown_connections 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.804+0000 7fb502400640 1 -- 172.21.15.165:0/2933958609 >> 172.21.15.165:0/2933958609 conn(0x7fb4fc06abe0 msgr2=0x7fb4fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.804+0000 7fb502400640 1 -- 172.21.15.165:0/2933958609 shutdown_connections 2024-04-25T10:16:20.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.804+0000 7fb502400640 1 -- 172.21.15.165:0/2933958609 wait complete. 2024-04-25T10:16:20.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.805+0000 7fb502400640 1 Processor -- start 2024-04-25T10:16:20.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.806+0000 7fb502400640 1 -- start start 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb502400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e0004d40 0x7fb4fc155120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb502400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc15c4e0 0x7fb4fc15c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb502400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 0x7fb4fc0b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb502400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0711b0 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb502400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc0b24e0 con 0x7fb4e0004d40 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb502400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4fc071330 con 0x7fb4fc15c4e0 2024-04-25T10:16:20.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 0x7fb4fc0b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 0x7fb4fc0b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50494/0 (socket says 172.21.15.165:50494) 2024-04-25T10:16:20.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 -- 172.21.15.165:0/1131089476 learned_addr learned my addr 172.21.15.165:0/1131089476 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:20.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501000640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e0004d40 0x7fb4fc155120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 -- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc15c4e0 msgr2=0x7fb4fc15c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb4f3e00640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc15c4e0 0x7fb4fc15c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc15c4e0 0x7fb4fc15c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 -- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e0004d40 msgr2=0x7fb4fc155120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e0004d40 0x7fb4fc155120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501a00640 1 -- 172.21.15.165:0/1131089476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4fc15e850 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.807+0000 7fb501000640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e0004d40 0x7fb4fc155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.808+0000 7fb501a00640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 0x7fb4fc0b0560 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400a8b0 tx=0x7fb4fc0b2fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.808+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f4004d70 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.808+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f402b760 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.808+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4fc0b0de0 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.808+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f402bb10 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.808+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4fc0b13a0 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.809+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4fc15ea20 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.810+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4f402bcb0 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.811+0000 7fb4f9600640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb4d806b500 0x7fb4d806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.811+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb4f40aa880 con 0x7fb4fc0b00b0 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.812+0000 7fb501000640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb4d806b500 0x7fb4d806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:20.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.812+0000 7fb501000640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb4d806b500 0x7fb4d806d9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0016b0 tx=0x7fb4ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:20.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:20.815+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4f4046060 con 0x7fb4fc0b00b0 2024-04-25T10:16:21.003 INFO:tasks.ceph.mds.c.smithi165.stderr:2024-04-25T10:16:20.999+0000 7fd091e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:21.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.049+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb4fc073990 con 0x7fb4fc0b00b0 2024-04-25T10:16:21.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.050+0000 7fb4f9600640 1 -- 172.21.15.165:0/1131089476 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4161 (secure 0 0 0) 0x7fb4f40793f0 con 0x7fb4fc0b00b0 2024-04-25T10:16:21.053 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:21.054 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:16.251260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:21.054 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 94 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb4d806b500 msgr2=0x7fb4d806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb4d806b500 0x7fb4d806d9c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec0016b0 tx=0x7fb4ec009040 comp rx=0 tx=0).stop 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 msgr2=0x7fb4fc0b0560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 0x7fb4fc0b0560 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fb4f400a8b0 tx=0x7fb4fc0b2fa0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 shutdown_connections 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb4d806b500 0x7fb4d806d9c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4fc0b00b0 0x7fb4fc0b0560 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4fc15c4e0 0x7fb4fc15c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 --2- 172.21.15.165:0/1131089476 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4e0004d40 0x7fb4fc155120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 >> 172.21.15.165:0/1131089476 conn(0x7fb4fc06abe0 msgr2=0x7fb4fc157190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 shutdown_connections 2024-04-25T10:16:21.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.053+0000 7fb502400640 1 -- 172.21.15.165:0/1131089476 wait complete. 2024-04-25T10:16:21.075 DEBUG:tasks.cephfs.test_scrub_checks:current active=['c'] 2024-04-25T10:16:21.075 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:16:21.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.270+0000 7f2d1e200640 1 Processor -- start 2024-04-25T10:16:21.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.271+0000 7f2d1e200640 1 -- start start 2024-04-25T10:16:21.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.272+0000 7f2d1e200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2d18143e60 0x7f2d1806e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.272+0000 7f2d1e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2d1806e6a0 0x7f2d180cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.272+0000 7f2d1e200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d1815bb50 con 0x7f2d180cc840 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.272+0000 7f2d1e200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d18144240 con 0x7f2d1806e6a0 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.272+0000 7f2d1e200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d1815e220 con 0x7f2d18143e60 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d0fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2d1806e6a0 0x7f2d180cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d1d800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2d180cc840 0x7f2d18070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33910/0 (socket says 172.21.15.165:33910) 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d1d800640 1 -- 172.21.15.165:0/393586168 learned_addr learned my addr 172.21.15.165:0/393586168 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d1ce00640 1 --2- 172.21.15.165:0/393586168 >> v2:172.21.15.172:3300/0 conn(0x7f2d18143e60 0x7f2d1806e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d0fe00640 1 -- 172.21.15.165:0/393586168 >> v2:172.21.15.172:3300/0 conn(0x7f2d18143e60 msgr2=0x7f2d1806e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d0fe00640 1 --2- 172.21.15.165:0/393586168 >> v2:172.21.15.172:3300/0 conn(0x7f2d18143e60 0x7f2d1806e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d0fe00640 1 -- 172.21.15.165:0/393586168 >> v1:172.21.15.165:6789/0 conn(0x7f2d180cc840 legacy=0x7f2d18070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d17e00640 1 -- 172.21.15.165:0/393586168 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2170922338 0 0) 0x7f2d10001490 con 0x7f2d180cc840 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.273+0000 7f2d0fe00640 1 -- 172.21.15.165:0/393586168 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1815e500 con 0x7f2d1806e6a0 2024-04-25T10:16:21.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.274+0000 7f2d0fe00640 1 --2- 172.21.15.165:0/393586168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d1806e6a0 0x7f2d180cc300 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b2770 tx=0x7f2d0002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8280d4d3a665196 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d17e00640 1 -- 172.21.15.165:0/393586168 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00037070 con 0x7f2d1806e6a0 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d17e00640 1 -- 172.21.15.165:0/393586168 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00032440 con 0x7f2d1806e6a0 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d17e00640 1 -- 172.21.15.165:0/393586168 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00032810 con 0x7f2d1806e6a0 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 -- 172.21.15.165:0/393586168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d1806e6a0 msgr2=0x7f2d180cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 --2- 172.21.15.165:0/393586168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d1806e6a0 0x7f2d180cc300 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f2d180b2770 tx=0x7f2d0002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 -- 172.21.15.165:0/393586168 shutdown_connections 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 --2- 172.21.15.165:0/393586168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d1806e6a0 0x7f2d180cc300 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 --2- 172.21.15.165:0/393586168 >> v2:172.21.15.172:3300/0 conn(0x7f2d18143e60 0x7f2d1806e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 -- 172.21.15.165:0/393586168 >> 172.21.15.165:0/393586168 conn(0x7f2d1806abe0 msgr2=0x7f2d1806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 -- 172.21.15.165:0/393586168 shutdown_connections 2024-04-25T10:16:21.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.275+0000 7f2d1e200640 1 -- 172.21.15.165:0/393586168 wait complete. 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.277+0000 7f2d1e200640 1 Processor -- start 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.277+0000 7f2d1e200640 1 -- start start 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d1e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d1806e6a0 0x7f2d1813d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d1e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 0x7f2d181f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d1e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d18143e60 0x7f2d1813b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d1e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180cccd0 con 0x7f2d180cc840 2024-04-25T10:16:21.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d1e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1815eac0 con 0x7f2d18143e60 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d1e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d180cce50 con 0x7f2d1806e6a0 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 0x7f2d181f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d0fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 0x7f2d181f66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50502/0 (socket says 172.21.15.165:50502) 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.278+0000 7f2d0fe00640 1 -- 172.21.15.165:0/1513275820 learned_addr learned my addr 172.21.15.165:0/1513275820 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d1ce00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d1806e6a0 0x7f2d1813d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d0fe00640 1 -- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d1806e6a0 msgr2=0x7f2d1813d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d1d800640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d18143e60 0x7f2d1813b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d0fe00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d1806e6a0 0x7f2d1813d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d0fe00640 1 -- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d18143e60 msgr2=0x7f2d1813b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d0fe00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d18143e60 0x7f2d1813b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d0fe00640 1 -- 172.21.15.165:0/1513275820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d1813dbc0 con 0x7f2d180cc840 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d1ce00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d1806e6a0 0x7f2d1813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.279+0000 7f2d0fe00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 0x7f2d181f66a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f2d00035040 tx=0x7f2d000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.280+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00037050 con 0x7f2d180cc840 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.280+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d00038040 con 0x7f2d180cc840 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.280+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d00004450 con 0x7f2d180cc840 2024-04-25T10:16:21.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.280+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1813dea0 con 0x7f2d180cc840 2024-04-25T10:16:21.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.280+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d18137d90 con 0x7f2d180cc840 2024-04-25T10:16:21.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.281+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d18073990 con 0x7f2d180cc840 2024-04-25T10:16:21.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.282+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2d00030980 con 0x7f2d180cc840 2024-04-25T10:16:21.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.283+0000 7f2d15600640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2cf406b4d0 0x7f2cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:21.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.283+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f2d000ab210 con 0x7f2d180cc840 2024-04-25T10:16:21.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.283+0000 7f2d1ce00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2cf406b4d0 0x7f2cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:21.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.284+0000 7f2d1ce00640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2cf406b4d0 0x7f2cf406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800b820 tx=0x7f2d08008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:21.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.287+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2d00079f30 con 0x7f2d180cc840 2024-04-25T10:16:21.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.556+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2d1815dea0 con 0x7f2d180cc840 2024-04-25T10:16:21.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.557+0000 7f2d15600640 1 -- 172.21.15.165:0/1513275820 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1045 (secure 0 0 0) 0x7f2d00079830 con 0x7f2d180cc840 2024-04-25T10:16:21.561 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:21.561 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":882,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":137,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":1048,"data_bytes":1076641813,"bytes_used":2612916224,"bytes_avail":770481197056,"bytes_total":773094113280,"read_bytes_sec":238499,"write_bytes_sec":204,"read_op_per_sec":1,"write_op_per_sec":0},"fsmap":{"epoch":94,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":9225}],"up:standby":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":35,"modified":"2024-04-25T10:16:14.419490+0000","services":{}},"progress_events":{}} 2024-04-25T10:16:21.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2cf406b4d0 msgr2=0x7f2cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2cf406b4d0 0x7f2cf406d990 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800b820 tx=0x7f2d08008040 comp rx=0 tx=0).stop 2024-04-25T10:16:21.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 msgr2=0x7f2d181f66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:21.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 0x7f2d181f66a0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f2d00035040 tx=0x7f2d000046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 shutdown_connections 2024-04-25T10:16:21.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2cf406b4d0 0x7f2cf406d990 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2d18143e60 0x7f2d1813b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2d180cc840 0x7f2d181f66a0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 --2- 172.21.15.165:0/1513275820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2d1806e6a0 0x7f2d1813d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 >> 172.21.15.165:0/1513275820 conn(0x7f2d1806abe0 msgr2=0x7f2d1806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 shutdown_connections 2024-04-25T10:16:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:21.560+0000 7f2d1e200640 1 -- 172.21.15.165:0/1513275820 wait complete. 2024-04-25T10:16:21.583 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:21.583 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:21.667 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:21.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:21.774 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:16:21.774 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:21.831 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:16:21.831 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:21.884 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:22.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.086+0000 7fb955e00640 1 Processor -- start 2024-04-25T10:16:22.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.086+0000 7fb955e00640 1 -- start start 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.087+0000 7fb955e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb950143e60 0x7fb95006e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb955e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb95006e6a0 0x7fb9500cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb955e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb95015bb50 con 0x7fb9500cc840 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb955e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb950144240 con 0x7fb950143e60 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb955e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95015e220 con 0x7fb95006e6a0 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb950143e60 0x7fb95006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb950143e60 0x7fb95006e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50504/0 (socket says 172.21.15.165:50504) 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 -- 172.21.15.165:0/239384806 learned_addr learned my addr 172.21.15.165:0/239384806 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:22.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb94fe00640 1 --2- 172.21.15.165:0/239384806 >> v2:172.21.15.172:3300/0 conn(0x7fb95006e6a0 0x7fb9500cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 -- 172.21.15.165:0/239384806 >> v2:172.21.15.172:3300/0 conn(0x7fb95006e6a0 msgr2=0x7fb9500cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 --2- 172.21.15.165:0/239384806 >> v2:172.21.15.172:3300/0 conn(0x7fb95006e6a0 0x7fb9500cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 -- 172.21.15.165:0/239384806 >> v1:172.21.15.172:6789/0 conn(0x7fb9500cc840 legacy=0x7fb950070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.088+0000 7fb954a00640 1 -- 172.21.15.165:0/239384806 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95015e500 con 0x7fb950143e60 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.089+0000 7fb954a00640 1 --2- 172.21.15.165:0/239384806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950143e60 0x7fb95006e160 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fb95006b970 tx=0x7fb94402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51e078290861116f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.090+0000 7fb94f400640 1 -- 172.21.15.165:0/239384806 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944037070 con 0x7fb950143e60 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.090+0000 7fb94f400640 1 -- 172.21.15.165:0/239384806 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944032440 con 0x7fb950143e60 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.090+0000 7fb94f400640 1 -- 172.21.15.165:0/239384806 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944032810 con 0x7fb950143e60 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.090+0000 7fb955e00640 1 -- 172.21.15.165:0/239384806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950143e60 msgr2=0x7fb95006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.090+0000 7fb955e00640 1 --2- 172.21.15.165:0/239384806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950143e60 0x7fb95006e160 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fb95006b970 tx=0x7fb94402a230 comp rx=0 tx=0).stop 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.091+0000 7fb955e00640 1 -- 172.21.15.165:0/239384806 shutdown_connections 2024-04-25T10:16:22.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.091+0000 7fb955e00640 1 --2- 172.21.15.165:0/239384806 >> v2:172.21.15.172:3300/0 conn(0x7fb95006e6a0 0x7fb9500cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.091+0000 7fb955e00640 1 --2- 172.21.15.165:0/239384806 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb950143e60 0x7fb95006e160 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.091+0000 7fb955e00640 1 -- 172.21.15.165:0/239384806 >> 172.21.15.165:0/239384806 conn(0x7fb95006abe0 msgr2=0x7fb95006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.091+0000 7fb955e00640 1 -- 172.21.15.165:0/239384806 shutdown_connections 2024-04-25T10:16:22.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.091+0000 7fb955e00640 1 -- 172.21.15.165:0/239384806 wait complete. 2024-04-25T10:16:22.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.092+0000 7fb955e00640 1 Processor -- start 2024-04-25T10:16:22.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.093+0000 7fb955e00640 1 -- start start 2024-04-25T10:16:22.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.093+0000 7fb955e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95006e6a0 0x7fb9501f2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb955e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 0x7fb9501e3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb955e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb950143e60 0x7fb95006d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb955e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500cccd0 con 0x7fb9500cc840 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb955e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb95015eac0 con 0x7fb95006e6a0 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb955e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9500cce50 con 0x7fb950143e60 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 0x7fb9501e3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 0x7fb9501e3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50520/0 (socket says 172.21.15.165:50520) 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 -- 172.21.15.165:0/2002729491 learned_addr learned my addr 172.21.15.165:0/2002729491 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb954a00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95006e6a0 0x7fb9501f2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 -- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb950143e60 msgr2=0x7fb95006d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb950143e60 0x7fb95006d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 -- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95006e6a0 msgr2=0x7fb9501f2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95006e6a0 0x7fb9501f2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.094+0000 7fb94fe00640 1 -- 172.21.15.165:0/2002729491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9501f2760 con 0x7fb9500cc840 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.095+0000 7fb94fe00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 0x7fb9501e3fc0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c820 tx=0x7fb93c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.095+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c015070 con 0x7fb9500cc840 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.095+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb93c004240 con 0x7fb9500cc840 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.095+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00ce10 con 0x7fb9500cc840 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.095+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9501e47f0 con 0x7fb9500cc840 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.095+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9501f2cb0 con 0x7fb9500cc840 2024-04-25T10:16:22.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.096+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9501e4bf0 con 0x7fb9500cc840 2024-04-25T10:16:22.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.097+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb93c007830 con 0x7fb9500cc840 2024-04-25T10:16:22.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.098+0000 7fb94cc00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb93006b4d0 0x7fb93006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.098+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fb93c08a360 con 0x7fb9500cc840 2024-04-25T10:16:22.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.098+0000 7fb954a00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb93006b4d0 0x7fb93006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.099+0000 7fb954a00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb93006b4d0 0x7fb93006d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb944034000 tx=0x7fb94402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.103+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb93c058f60 con 0x7fb9500cc840 2024-04-25T10:16:22.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.299+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb950073990 con 0x7fb9500cc840 2024-04-25T10:16:22.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.303+0000 7fb94cc00640 1 -- 172.21.15.165:0/2002729491 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17067 (secure 0 0 0) 0x7fb93c058860 con 0x7fb9500cc840 2024-04-25T10:16:22.306 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:22.307 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":137,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:14.197143+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/2186062364":"2024-04-26T10:16:14.196999+0000","172.21.15.172:6834/2186062364":"2024-04-26T10:16:14.196999+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:22.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.305+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb93006b4d0 msgr2=0x7fb93006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.305+0000 7fb955e00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb93006b4d0 0x7fb93006d990 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fb944034000 tx=0x7fb94402e000 comp rx=0 tx=0).stop 2024-04-25T10:16:22.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 msgr2=0x7fb9501e3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 0x7fb9501e3fc0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00c820 tx=0x7fb93c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 shutdown_connections 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb93006b4d0 0x7fb93006d990 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb950143e60 0x7fb95006d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb9500cc840 0x7fb9501e3fc0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 --2- 172.21.15.165:0/2002729491 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb95006e6a0 0x7fb9501f2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 >> 172.21.15.165:0/2002729491 conn(0x7fb95006abe0 msgr2=0x7fb950070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 shutdown_connections 2024-04-25T10:16:22.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.306+0000 7fb955e00640 1 -- 172.21.15.165:0/2002729491 wait complete. 2024-04-25T10:16:22.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:08.823693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:10.098970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:22.330 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:22.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.530+0000 7ff7e8200640 1 Processor -- start 2024-04-25T10:16:22.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.531+0000 7ff7e8200640 1 -- start start 2024-04-25T10:16:22.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.532+0000 7ff7e8200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff7e0161710 0x7ff7e00d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.532+0000 7ff7e8200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e015e000 con 0x7ff7e00ced80 2024-04-25T10:16:22.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.532+0000 7ff7e8200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e00d1de0 con 0x7ff7e015dc20 2024-04-25T10:16:22.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.532+0000 7ff7e8200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e0161c60 con 0x7ff7e0161710 2024-04-25T10:16:22.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.532+0000 7ff7e4e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff7e00ced80 0x7ff7e00d22e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33918/0 (socket says 172.21.15.165:33918) 2024-04-25T10:16:22.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.532+0000 7ff7e4e00640 1 -- 172.21.15.165:0/4011841441 learned_addr learned my addr 172.21.15.165:0/4011841441 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7e6200640 1 --2- 172.21.15.165:0/4011841441 >> v2:172.21.15.165:3300/0 conn(0x7ff7e0161710 0x7ff7e00d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7dfe00640 1 -- 172.21.15.165:0/4011841441 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762024168 0 0) 0x7ff7e015e000 con 0x7ff7e00ced80 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7dfe00640 1 -- 172.21.15.165:0/4011841441 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7c8003520 con 0x7ff7e00ced80 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7dfe00640 1 -- 172.21.15.165:0/4011841441 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 252174223 0 0) 0x7ff7c8003520 con 0x7ff7e00ced80 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7e6200640 1 -- 172.21.15.165:0/4011841441 >> v1:172.21.15.172:6789/0 conn(0x7ff7e015dc20 legacy=0x7ff7e00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7e6200640 1 -- 172.21.15.165:0/4011841441 >> v1:172.21.15.165:6789/0 conn(0x7ff7e00ced80 legacy=0x7ff7e00d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.533+0000 7ff7e6200640 1 -- 172.21.15.165:0/4011841441 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0161f40 con 0x7ff7e0161710 2024-04-25T10:16:22.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.534+0000 7ff7e6200640 1 --2- 172.21.15.165:0/4011841441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e0161710 0x7ff7e00d29f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80097a0 tx=0x7ff7d802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8fd284644d6a3f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.535+0000 7ff7dfe00640 1 -- 172.21.15.165:0/4011841441 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d8037070 con 0x7ff7e0161710 2024-04-25T10:16:22.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.535+0000 7ff7dfe00640 1 -- 172.21.15.165:0/4011841441 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d802ac50 con 0x7ff7e0161710 2024-04-25T10:16:22.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.535+0000 7ff7dfe00640 1 -- 172.21.15.165:0/4011841441 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc001670 con 0x7ff7e0161710 2024-04-25T10:16:22.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.535+0000 7ff7e8200640 1 -- 172.21.15.165:0/4011841441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e0161710 msgr2=0x7ff7e00d29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.535+0000 7ff7e8200640 1 --2- 172.21.15.165:0/4011841441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e0161710 0x7ff7e00d29f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7ff7d80097a0 tx=0x7ff7d802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.536+0000 7ff7e8200640 1 -- 172.21.15.165:0/4011841441 shutdown_connections 2024-04-25T10:16:22.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.536+0000 7ff7e8200640 1 --2- 172.21.15.165:0/4011841441 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e0161710 0x7ff7e00d29f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.536+0000 7ff7e8200640 1 -- 172.21.15.165:0/4011841441 >> 172.21.15.165:0/4011841441 conn(0x7ff7e00d0420 msgr2=0x7ff7e00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.536+0000 7ff7e8200640 1 -- 172.21.15.165:0/4011841441 shutdown_connections 2024-04-25T10:16:22.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.536+0000 7ff7e8200640 1 -- 172.21.15.165:0/4011841441 wait complete. 2024-04-25T10:16:22.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.537+0000 7ff7e8200640 1 Processor -- start 2024-04-25T10:16:22.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.538+0000 7ff7e8200640 1 -- start start 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.538+0000 7ff7e8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7e00ced80 0x7ff7e0152040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.538+0000 7ff7e8200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 0x7ff7e00c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.538+0000 7ff7e8200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7e0161710 0x7ff7e00b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e8200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00b5e50 con 0x7ff7e015dc20 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e8200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e0162500 con 0x7ff7e0161710 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e00b5fd0 con 0x7ff7e00ced80 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 0x7ff7e00c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 0x7ff7e00c8810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50542/0 (socket says 172.21.15.165:50542) 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 -- 172.21.15.165:0/3387330776 learned_addr learned my addr 172.21.15.165:0/3387330776 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 -- 172.21.15.165:0/3387330776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7e00ced80 msgr2=0x7ff7e0152040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7e00ced80 0x7ff7e0152040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 -- 172.21.15.165:0/3387330776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7e0161710 msgr2=0x7ff7e00b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7e0161710 0x7ff7e00b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 -- 172.21.15.165:0/3387330776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e00c9040 con 0x7ff7e015dc20 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.539+0000 7ff7e4e00640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 0x7ff7e00c8810 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c5b0 tx=0x7ff7cc00ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.540+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc017070 con 0x7ff7e015dc20 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.540+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7cc005ce0 con 0x7ff7e015dc20 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.540+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc007330 con 0x7ff7e015dc20 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.540+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e00c9320 con 0x7ff7e015dc20 2024-04-25T10:16:22.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.540+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e00c98e0 con 0x7ff7e015dc20 2024-04-25T10:16:22.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.541+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e0073a00 con 0x7ff7e015dc20 2024-04-25T10:16:22.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.542+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff7cc01a070 con 0x7ff7e015dc20 2024-04-25T10:16:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.543+0000 7ff7dd600640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7c006b4d0 0x7ff7c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.543+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff7cc089920 con 0x7ff7e015dc20 2024-04-25T10:16:22.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.543+0000 7ff7e5800640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7c006b4d0 0x7ff7c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.544+0000 7ff7e5800640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7c006b4d0 0x7ff7c006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40012e0 tx=0x7ff7d4007d30 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.547+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff7cc0585b0 con 0x7ff7e015dc20 2024-04-25T10:16:22.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.745+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7e00ceb40 con 0x7ff7e015dc20 2024-04-25T10:16:22.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.746+0000 7ff7dd600640 1 -- 172.21.15.165:0/3387330776 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7cc057eb0 con 0x7ff7e015dc20 2024-04-25T10:16:22.749 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:16:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.748+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7c006b4d0 msgr2=0x7ff7c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.748+0000 7ff7e8200640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7c006b4d0 0x7ff7c006d990 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40012e0 tx=0x7ff7d4007d30 comp rx=0 tx=0).stop 2024-04-25T10:16:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.748+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 msgr2=0x7ff7e00c8810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.748+0000 7ff7e8200640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 0x7ff7e00c8810 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc00c5b0 tx=0x7ff7cc00ca70 comp rx=0 tx=0).stop 2024-04-25T10:16:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 shutdown_connections 2024-04-25T10:16:22.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff7c006b4d0 0x7ff7c006d990 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff7e0161710 0x7ff7e00b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff7e015dc20 0x7ff7e00c8810 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 --2- 172.21.15.165:0/3387330776 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff7e00ced80 0x7ff7e0152040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 >> 172.21.15.165:0/3387330776 conn(0x7ff7e00d0420 msgr2=0x7ff7e015b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 shutdown_connections 2024-04-25T10:16:22.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.749+0000 7ff7e8200640 1 -- 172.21.15.165:0/3387330776 wait complete. 2024-04-25T10:16:22.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:22.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.966+0000 7f924be00640 1 Processor -- start 2024-04-25T10:16:22.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.967+0000 7f924be00640 1 -- start start 2024-04-25T10:16:22.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.968+0000 7f924be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f924c15e720 0x7f924c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.968+0000 7f924be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c0b0b80 con 0x7f924c15b7a0 2024-04-25T10:16:22.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.968+0000 7f924be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c0cc800 con 0x7f924c15b3c0 2024-04-25T10:16:22.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.968+0000 7f924be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c0cd650 con 0x7f924c15e720 2024-04-25T10:16:22.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.969+0000 7f9242000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f924c15b7a0 0x7f924c15e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33926/0 (socket says 172.21.15.165:33926) 2024-04-25T10:16:22.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.969+0000 7f9242000640 1 -- 172.21.15.165:0/3466286179 learned_addr learned my addr 172.21.15.165:0/3466286179 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:22.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.969+0000 7f924b400640 1 --2- 172.21.15.165:0/3466286179 >> v2:172.21.15.172:3300/0 conn(0x7f924c15e720 0x7f924c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.969+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348949207 0 0) 0x7f9234001490 con 0x7f924c15b7a0 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9230003520 con 0x7f924c15b7a0 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3989106855 0 0) 0x7f9230003520 con 0x7f924c15b7a0 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 >> v2:172.21.15.172:3300/0 conn(0x7f924c15e720 msgr2=0x7f924c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 --2- 172.21.15.165:0/3466286179 >> v2:172.21.15.172:3300/0 conn(0x7f924c15e720 0x7f924c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 >> v1:172.21.15.172:6789/0 conn(0x7f924c15b3c0 legacy=0x7f924c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0ccae0 con 0x7f924c15b7a0 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9234001bc0 con 0x7f924c15b7a0 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 >> v1:172.21.15.165:6789/0 conn(0x7f924c15b7a0 legacy=0x7f924c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92300039b0 0x7f9230003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f9230004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0cd650 con 0x7f9230004800 2024-04-25T10:16:22.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923000c160 con 0x7f92300039b0 2024-04-25T10:16:22.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923000c360 con 0x7f92300051c0 2024-04-25T10:16:22.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.970+0000 7f9242000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f9230004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f924aa00640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92300039b0 0x7f9230003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f924b400640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f9242000640 1 -- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92300051c0 msgr2=0x7f923000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f9242000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92300051c0 0x7f923000b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f9242000640 1 -- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92300039b0 msgr2=0x7f9230003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f9242000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92300039b0 0x7f9230003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f9242000640 1 -- 172.21.15.165:0/3466286179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230003520 con 0x7f9230004800 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.971+0000 7f924b400640 1 -- 172.21.15.165:0/3466286179 reap_dead start 2024-04-25T10:16:22.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.972+0000 7f9242000640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f9230004c80 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f923400aa00 tx=0x7f9230005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66ba58ad0d3fdaa0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.974+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400a650 con 0x7f9230004800 2024-04-25T10:16:22.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.974+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923402f070 con 0x7f9230004800 2024-04-25T10:16:22.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.974+0000 7f924a000640 1 -- 172.21.15.165:0/3466286179 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923402be40 con 0x7f9230004800 2024-04-25T10:16:22.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.974+0000 7f924be00640 1 -- 172.21.15.165:0/3466286179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 msgr2=0x7f9230004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.974+0000 7f924be00640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f9230004c80 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f923400aa00 tx=0x7f9230005830 comp rx=0 tx=0).stop 2024-04-25T10:16:22.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.975+0000 7f924be00640 1 -- 172.21.15.165:0/3466286179 shutdown_connections 2024-04-25T10:16:22.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.975+0000 7f924be00640 1 --2- 172.21.15.165:0/3466286179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f9230004c80 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.975+0000 7f924be00640 1 -- 172.21.15.165:0/3466286179 >> 172.21.15.165:0/3466286179 conn(0x7f924c061cc0 msgr2=0x7f924c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:22.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.975+0000 7f924be00640 1 -- 172.21.15.165:0/3466286179 shutdown_connections 2024-04-25T10:16:22.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.975+0000 7f924be00640 1 -- 172.21.15.165:0/3466286179 wait complete. 2024-04-25T10:16:22.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.976+0000 7f924be00640 1 Processor -- start 2024-04-25T10:16:22.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.977+0000 7f924be00640 1 -- start start 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f924c14d050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c0abe20 0x7f924c0ac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924c0b3550 0x7f924c0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0b10e0 con 0x7f9230004800 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0b0f60 con 0x7f924c0abe20 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0b1260 con 0x7f924c0b3550 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f924c14d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f924c14d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50560/0 (socket says 172.21.15.165:50560) 2024-04-25T10:16:22.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 -- 172.21.15.165:0/2737962884 learned_addr learned my addr 172.21.15.165:0/2737962884 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:22.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f9242000640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c0abe20 0x7f924c0ac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 -- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924c0b3550 msgr2=0x7f924c0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924b400640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924c0b3550 0x7f924c0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924c0b3550 0x7f924c0b39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 -- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c0abe20 msgr2=0x7f924c0ac2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c0abe20 0x7f924c0ac2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f924aa00640 1 -- 172.21.15.165:0/2737962884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0b4160 con 0x7f9230004800 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.978+0000 7f9242000640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c0abe20 0x7f924c0ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:22.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.979+0000 7f924aa00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f924c14d050 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e500 tx=0x7f923c00e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.980+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c0052e0 con 0x7f9230004800 2024-04-25T10:16:22.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.980+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c018070 con 0x7f9230004800 2024-04-25T10:16:22.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.980+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c003810 con 0x7f9230004800 2024-04-25T10:16:22.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.980+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c0b4440 con 0x7f9230004800 2024-04-25T10:16:22.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.981+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c0a9a40 con 0x7f9230004800 2024-04-25T10:16:22.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.982+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c073990 con 0x7f9230004800 2024-04-25T10:16:22.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.983+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f923c0039b0 con 0x7f9230004800 2024-04-25T10:16:22.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.984+0000 7f9243e00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922806b500 0x7f922806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:22.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.984+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f923c089fa0 con 0x7f9230004800 2024-04-25T10:16:22.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.984+0000 7f9242000640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922806b500 0x7f922806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:22.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.985+0000 7f9242000640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922806b500 0x7f922806d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9234033000 tx=0x7f9234009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:22.988+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f923c058c50 con 0x7f9230004800 2024-04-25T10:16:23.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.190+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f924c072360 con 0x7f9230004800 2024-04-25T10:16:23.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.191+0000 7f9243e00640 1 -- 172.21.15.165:0/2737962884 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7f923c058550 con 0x7f9230004800 2024-04-25T10:16:23.195 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:16:23.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922806b500 msgr2=0x7f922806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922806b500 0x7f922806d9c0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f9234033000 tx=0x7f9234009030 comp rx=0 tx=0).stop 2024-04-25T10:16:23.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 msgr2=0x7f924c14d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f924c14d050 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f923c00e500 tx=0x7f923c00e9c0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 shutdown_connections 2024-04-25T10:16:23.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922806b500 0x7f922806d9c0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f924c0b3550 0x7f924c0b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f924c0abe20 0x7f924c0ac2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 --2- 172.21.15.165:0/2737962884 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9230004800 0x7f924c14d050 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 >> 172.21.15.165:0/2737962884 conn(0x7f924c061cc0 msgr2=0x7f924c06e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:23.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 shutdown_connections 2024-04-25T10:16:23.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.194+0000 7f924be00640 1 -- 172.21.15.165:0/2737962884 wait complete. 2024-04-25T10:16:23.216 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:23.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.418+0000 7f47f8400640 1 Processor -- start 2024-04-25T10:16:23.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.418+0000 7f47f8400640 1 -- start start 2024-04-25T10:16:23.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.420+0000 7f47f8400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f47f00d3440 0x7f47f00d1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.420+0000 7f47f8400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f00d23a0 con 0x7f47f015dc20 2024-04-25T10:16:23.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.420+0000 7f47f8400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47f00d2680 con 0x7f47f00ced80 2024-04-25T10:16:23.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.420+0000 7f47f8400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47f00d2960 con 0x7f47f00d3440 2024-04-25T10:16:23.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.420+0000 7f47f5a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f47f015dc20 0x7f47f00ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33928/0 (socket says 172.21.15.165:33928) 2024-04-25T10:16:23.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.420+0000 7f47f5a00640 1 -- 172.21.15.165:0/186756431 learned_addr learned my addr 172.21.15.165:0/186756431 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:23.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2715038830 0 0) 0x7f47f00d23a0 con 0x7f47f015dc20 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47d4003520 con 0x7f47f015dc20 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47f6400640 1 --2- 172.21.15.165:0/186756431 >> v2:172.21.15.172:3300/0 conn(0x7f47f00d3440 0x7f47f00d1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3315623666 0 0) 0x7f47d4003520 con 0x7f47f015dc20 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 >> v2:172.21.15.172:3300/0 conn(0x7f47f00d3440 msgr2=0x7f47f00d1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 --2- 172.21.15.165:0/186756431 >> v2:172.21.15.172:3300/0 conn(0x7f47f00d3440 0x7f47f00d1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 >> v1:172.21.15.172:6789/0 conn(0x7f47f00ced80 legacy=0x7f47f00b6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f00d2c40 con 0x7f47f015dc20 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f47e0001670 con 0x7f47f015dc20 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 >> v1:172.21.15.165:6789/0 conn(0x7f47f015dc20 legacy=0x7f47f00ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47d4003a00 0x7f47d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4004800 0x7f47d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d4005290 0x7f47d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00d2960 con 0x7f47d4005290 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d400c1e0 con 0x7f47d4003a00 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.421+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d400c3e0 con 0x7f47d4004800 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f6400640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d4005290 0x7f47d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f5000640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4004800 0x7f47d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f5a00640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47d4003a00 0x7f47d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f6400640 1 -- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4004800 msgr2=0x7f47d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f6400640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4004800 0x7f47d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f6400640 1 -- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47d4003a00 msgr2=0x7f47d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f6400640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47d4003a00 0x7f47d4003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.422+0000 7f47f6400640 1 -- 172.21.15.165:0/186756431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d4003520 con 0x7f47d4005290 2024-04-25T10:16:23.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.423+0000 7f47f6400640 1 -- 172.21.15.165:0/186756431 reap_dead start 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.423+0000 7f47f6400640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d4005290 0x7f47d400b9c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f47e800ba30 tx=0x7f47d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=671acf8405f23a29 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8030070 con 0x7f47d4005290 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e802ae50 con 0x7f47d4005290 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47efe00640 1 -- 172.21.15.165:0/186756431 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800b600 con 0x7f47d4005290 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 -- 172.21.15.165:0/186756431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d4005290 msgr2=0x7f47d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d4005290 0x7f47d400b9c0 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x7f47e800ba30 tx=0x7f47d4005830 comp rx=0 tx=0).stop 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 -- 172.21.15.165:0/186756431 shutdown_connections 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 --2- 172.21.15.165:0/186756431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47d4005290 0x7f47d400b9c0 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 -- 172.21.15.165:0/186756431 >> 172.21.15.165:0/186756431 conn(0x7f47f0061a20 msgr2=0x7f47f0061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 -- 172.21.15.165:0/186756431 shutdown_connections 2024-04-25T10:16:23.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.424+0000 7f47f8400640 1 -- 172.21.15.165:0/186756431 wait complete. 2024-04-25T10:16:23.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.426+0000 7f47f8400640 1 Processor -- start 2024-04-25T10:16:23.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.426+0000 7f47f8400640 1 -- start start 2024-04-25T10:16:23.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f8400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4005290 0x7f47f015a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f8400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f015e000 0x7f47f0157090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f8400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 0x7f47f015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f8400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00d1e60 con 0x7f47f015b340 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f8400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00d1ce0 con 0x7f47f015e000 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f8400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47f00d1fe0 con 0x7f47d4005290 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.427+0000 7f47f6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 0x7f47f015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 0x7f47f015b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50588/0 (socket says 172.21.15.165:50588) 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 -- 172.21.15.165:0/3568465581 learned_addr learned my addr 172.21.15.165:0/3568465581 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 -- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4005290 msgr2=0x7f47f015a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4005290 0x7f47f015a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 -- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f015e000 msgr2=0x7f47f0157090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f5000640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f015e000 0x7f47f0157090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f015e000 0x7f47f0157090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 -- 172.21.15.165:0/3568465581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47f015e850 con 0x7f47f015b340 2024-04-25T10:16:23.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.428+0000 7f47f6400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 0x7f47f015b7c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f47e802a900 tx=0x7f47e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.429+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e800b600 con 0x7f47f015b340 2024-04-25T10:16:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.429+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47e802ac10 con 0x7f47f015b340 2024-04-25T10:16:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.429+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47e8004020 con 0x7f47f015b340 2024-04-25T10:16:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.429+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47f015eb30 con 0x7f47f015b340 2024-04-25T10:16:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.429+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47f00b3100 con 0x7f47f015b340 2024-04-25T10:16:23.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.430+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47f015bd00 con 0x7f47f015b340 2024-04-25T10:16:23.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.431+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f47e8044020 con 0x7f47f015b340 2024-04-25T10:16:23.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.432+0000 7f47ed600640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d006b4d0 0x7f47d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.432+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f47e80a9c20 con 0x7f47f015b340 2024-04-25T10:16:23.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.432+0000 7f47f5a00640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d006b4d0 0x7f47d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.433+0000 7f47f5a00640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d006b4d0 0x7f47d006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f47e0001eb0 tx=0x7f47e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:23.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.437+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f47e804b020 con 0x7f47f015b340 2024-04-25T10:16:23.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.632+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f47f0073a00 con 0x7f47f015b340 2024-04-25T10:16:23.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.633+0000 7f47ed600640 1 -- 172.21.15.165:0/3568465581 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7f47e8044310 con 0x7f47f015b340 2024-04-25T10:16:23.637 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:16:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d006b4d0 msgr2=0x7f47d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d006b4d0 0x7f47d006d990 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f47e0001eb0 tx=0x7f47e0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 msgr2=0x7f47f015b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 0x7f47f015b7c0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7f47e802a900 tx=0x7f47e8004730 comp rx=0 tx=0).stop 2024-04-25T10:16:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 shutdown_connections 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f47d006b4d0 0x7f47d006d990 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f47f015b340 0x7f47f015b7c0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f47f015e000 0x7f47f0157090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 --2- 172.21.15.165:0/3568465581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f47d4005290 0x7f47f015a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 >> 172.21.15.165:0/3568465581 conn(0x7f47f0061a20 msgr2=0x7f47f00b69f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 shutdown_connections 2024-04-25T10:16:23.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.636+0000 7f47f8400640 1 -- 172.21.15.165:0/3568465581 wait complete. 2024-04-25T10:16:23.659 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:16:23.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.856+0000 7f0597e00640 1 Processor -- start 2024-04-25T10:16:23.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.856+0000 7f0597e00640 1 -- start start 2024-04-25T10:16:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.857+0000 7f0597e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0598145140 0x7f05980ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.858+0000 7f0597e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980ce950 0x7f05980c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.858+0000 7f0597e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0598145520 con 0x7f05980c5030 2024-04-25T10:16:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.858+0000 7f0597e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980ced90 con 0x7f0598145140 2024-04-25T10:16:23.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.858+0000 7f0597e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0598071270 con 0x7f05980ce950 2024-04-25T10:16:23.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.858+0000 7f0596a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0598145140 0x7f05980ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.859+0000 7f0596a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0598145140 0x7f05980ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:39568/0 (socket says 172.21.15.165:39568) 2024-04-25T10:16:23.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.859+0000 7f0596a00640 1 -- 172.21.15.165:0/538211887 learned_addr learned my addr 172.21.15.165:0/538211887 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:23.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.858+0000 7f0596000640 1 --2- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980ce950 0x7f05980c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.859+0000 7f0597400640 1 --1- 172.21.15.165:0/538211887 >> v1:172.21.15.172:6789/0 conn(0x7f05980c5030 0x7f059806d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:34694/0 (socket says 172.21.15.165:34694) 2024-04-25T10:16:23.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.859+0000 7f0596a00640 1 -- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980ce950 msgr2=0x7f05980c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:23.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.859+0000 7f0596a00640 1 --2- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980ce950 0x7f05980c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.859+0000 7f0596a00640 1 -- 172.21.15.165:0/538211887 >> v1:172.21.15.172:6789/0 conn(0x7f05980c5030 legacy=0x7f059806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.860+0000 7f0596a00640 1 -- 172.21.15.165:0/538211887 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0598071550 con 0x7f0598145140 2024-04-25T10:16:23.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.861+0000 7f0596a00640 1 --2- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0598145140 0x7f05980ce410 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0588009870 tx=0x7f058802a1d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35a52def08414e69 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:23.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.861+0000 7f0595600640 1 -- 172.21.15.165:0/538211887 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588037070 con 0x7f0598145140 2024-04-25T10:16:23.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.861+0000 7f0595600640 1 -- 172.21.15.165:0/538211887 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588032440 con 0x7f0598145140 2024-04-25T10:16:23.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.861+0000 7f0595600640 1 -- 172.21.15.165:0/538211887 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588032810 con 0x7f0598145140 2024-04-25T10:16:23.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 -- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0598145140 msgr2=0x7f05980ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 --2- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0598145140 0x7f05980ce410 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0588009870 tx=0x7f058802a1d0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 -- 172.21.15.165:0/538211887 shutdown_connections 2024-04-25T10:16:23.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 --2- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980ce950 0x7f05980c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 --2- 172.21.15.165:0/538211887 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0598145140 0x7f05980ce410 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 -- 172.21.15.165:0/538211887 >> 172.21.15.165:0/538211887 conn(0x7f0598061cc0 msgr2=0x7f059806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:23.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 -- 172.21.15.165:0/538211887 shutdown_connections 2024-04-25T10:16:23.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.862+0000 7f0597e00640 1 -- 172.21.15.165:0/538211887 wait complete. 2024-04-25T10:16:23.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.864+0000 7f0597e00640 1 Processor -- start 2024-04-25T10:16:23.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.864+0000 7f0597e00640 1 -- start start 2024-04-25T10:16:23.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980c5030 0x7f05980a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980ce950 0x7f059815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 0x7f059815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980c54a0 con 0x7f0598145140 2024-04-25T10:16:23.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0598071b10 con 0x7f05980ce950 2024-04-25T10:16:23.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980c5620 con 0x7f05980c5030 2024-04-25T10:16:23.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 0x7f059815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 0x7f059815b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50596/0 (socket says 172.21.15.165:50596) 2024-04-25T10:16:23.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0597400640 1 -- 172.21.15.165:0/2043871828 learned_addr learned my addr 172.21.15.165:0/2043871828 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0596000640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980ce950 0x7f059815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.865+0000 7f0596a00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980c5030 0x7f05980a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597400640 1 -- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980c5030 msgr2=0x7f05980a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597400640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980c5030 0x7f05980a8710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597400640 1 -- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980ce950 msgr2=0x7f059815afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597400640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980ce950 0x7f059815afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597400640 1 -- 172.21.15.165:0/2043871828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059813d640 con 0x7f0598145140 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0596000640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980ce950 0x7f059815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:23.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597400640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 0x7f059815b500 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f059000c780 tx=0x7f059000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590017070 con 0x7f0598145140 2024-04-25T10:16:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059000a640 con 0x7f0598145140 2024-04-25T10:16:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000aa30 con 0x7f0598145140 2024-04-25T10:16:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059813d920 con 0x7f0598145140 2024-04-25T10:16:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.866+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059813dee0 con 0x7f0598145140 2024-04-25T10:16:23.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.868+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0598073990 con 0x7f0598145140 2024-04-25T10:16:23.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.869+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f059001a070 con 0x7f0598145140 2024-04-25T10:16:23.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.870+0000 7f058ea00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f057806b4d0 0x7f057806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:23.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.870+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f05900899c0 con 0x7f0598145140 2024-04-25T10:16:23.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.870+0000 7f0596a00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f057806b4d0 0x7f057806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:23.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.871+0000 7f0596a00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f057806b4d0 0x7f057806d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0588034000 tx=0x7f058802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:23.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:23.874+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0590057f80 con 0x7f0598145140 2024-04-25T10:16:24.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.085+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f059815d440 con 0x7f0598145140 2024-04-25T10:16:24.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.086+0000 7f058ea00640 1 -- 172.21.15.165:0/2043871828 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v137) v1 ==== 80+0+124 (secure 0 0 0) 0x7f059005c020 con 0x7f0598145140 2024-04-25T10:16:24.089 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.172:6835/2186062364 2024-04-26T10:16:14.196999+0000 2024-04-25T10:16:24.089 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.15.172:6834/2186062364 2024-04-26T10:16:14.196999+0000 2024-04-25T10:16:24.090 INFO:teuthology.orchestra.run.smithi165.stderr:listed 2 entries 2024-04-25T10:16:24.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f057806b4d0 msgr2=0x7f057806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f057806b4d0 0x7f057806d990 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f0588034000 tx=0x7f058802e000 comp rx=0 tx=0).stop 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 msgr2=0x7f059815b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 0x7f059815b500 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7f059000c780 tx=0x7f059000cc40 comp rx=0 tx=0).stop 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 shutdown_connections 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f057806b4d0 0x7f057806d990 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0598145140 0x7f059815b500 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f05980ce950 0x7f059815afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 --2- 172.21.15.165:0/2043871828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f05980c5030 0x7f05980a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 >> 172.21.15.165:0/2043871828 conn(0x7f0598061cc0 msgr2=0x7f05980b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 shutdown_connections 2024-04-25T10:16:24.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.089+0000 7f0597e00640 1 -- 172.21.15.165:0/2043871828 wait complete. 2024-04-25T10:16:24.112 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:16:24.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:24.112 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:24.205 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:16:24.205 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:24.205 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:16:24.258 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:16:24.258 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:24.316 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:24.317 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:16:24.317 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:24.371 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:24.371 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:16:24.372 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:24.429 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:24.429 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:24.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.633+0000 7f8fcac00640 1 Processor -- start 2024-04-25T10:16:24.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.633+0000 7f8fcac00640 1 -- start start 2024-04-25T10:16:24.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fcac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8fc4143e60 0x7f8fc406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fcac00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc4144240 con 0x7f8fc406e6a0 2024-04-25T10:16:24.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fcac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc415e220 con 0x7f8fc40cca10 2024-04-25T10:16:24.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fcac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc415e500 con 0x7f8fc4143e60 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fc9800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8fc4143e60 0x7f8fc406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fc9800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8fc4143e60 0x7f8fc406e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50598/0 (socket says 172.21.15.165:50598) 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fc9800640 1 -- 172.21.15.165:0/3026596361 learned_addr learned my addr 172.21.15.165:0/3026596361 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fc8e00640 1 --1- 172.21.15.165:0/3026596361 >> v1:172.21.15.165:6789/0 conn(0x7f8fc406e6a0 0x7f8fc40cc300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33944/0 (socket says 172.21.15.165:33944) 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fc9800640 1 -- 172.21.15.165:0/3026596361 >> v1:172.21.15.172:6789/0 conn(0x7f8fc40cca10 legacy=0x7f8fc415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.635+0000 7f8fc9800640 1 -- 172.21.15.165:0/3026596361 >> v1:172.21.15.165:6789/0 conn(0x7f8fc406e6a0 legacy=0x7f8fc40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.636+0000 7f8fc9800640 1 -- 172.21.15.165:0/3026596361 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc415e7e0 con 0x7f8fc4143e60 2024-04-25T10:16:24.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.637+0000 7f8fc9800640 1 --2- 172.21.15.165:0/3026596361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc4143e60 0x7f8fc406e160 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009870 tx=0x7f8fb402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=551763fc90ee7a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:24.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.637+0000 7f8fc3e00640 1 -- 172.21.15.165:0/3026596361 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4037070 con 0x7f8fc4143e60 2024-04-25T10:16:24.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.637+0000 7f8fc3e00640 1 -- 172.21.15.165:0/3026596361 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb4032440 con 0x7f8fc4143e60 2024-04-25T10:16:24.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.637+0000 7f8fc3e00640 1 -- 172.21.15.165:0/3026596361 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb4032810 con 0x7f8fc4143e60 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.637+0000 7f8fcac00640 1 -- 172.21.15.165:0/3026596361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc4143e60 msgr2=0x7f8fc406e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.637+0000 7f8fcac00640 1 --2- 172.21.15.165:0/3026596361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc4143e60 0x7f8fc406e160 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4009870 tx=0x7f8fb402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.638+0000 7f8fcac00640 1 -- 172.21.15.165:0/3026596361 shutdown_connections 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.638+0000 7f8fcac00640 1 --2- 172.21.15.165:0/3026596361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc4143e60 0x7f8fc406e160 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.638+0000 7f8fcac00640 1 -- 172.21.15.165:0/3026596361 >> 172.21.15.165:0/3026596361 conn(0x7f8fc406abe0 msgr2=0x7f8fc406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.638+0000 7f8fcac00640 1 -- 172.21.15.165:0/3026596361 shutdown_connections 2024-04-25T10:16:24.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.638+0000 7f8fcac00640 1 -- 172.21.15.165:0/3026596361 wait complete. 2024-04-25T10:16:24.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.639+0000 7f8fcac00640 1 Processor -- start 2024-04-25T10:16:24.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.639+0000 7f8fcac00640 1 -- start start 2024-04-25T10:16:24.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.640+0000 7f8fcac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fc406e6a0 0x7f8fc413c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.640+0000 7f8fcac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 0x7f8fc413df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.640+0000 7f8fcac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc4143e60 0x7f8fc413e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.640+0000 7f8fcac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b2eb0 con 0x7f8fc40cca10 2024-04-25T10:16:24.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.640+0000 7f8fcac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc40b2d30 con 0x7f8fc4143e60 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.640+0000 7f8fcac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc415b250 con 0x7f8fc406e6a0 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 0x7f8fc413df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 0x7f8fc413df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50608/0 (socket says 172.21.15.165:50608) 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 -- 172.21.15.165:0/2616671980 learned_addr learned my addr 172.21.15.165:0/2616671980 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 -- 172.21.15.165:0/2616671980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fc406e6a0 msgr2=0x7f8fc413c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fc406e6a0 0x7f8fc413c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 -- 172.21.15.165:0/2616671980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc4143e60 msgr2=0x7f8fc413e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc4143e60 0x7f8fc413e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 -- 172.21.15.165:0/2616671980 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc413c5d0 con 0x7f8fc40cca10 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.641+0000 7f8fc8e00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 0x7f8fc413df60 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000ca70 tx=0x7f8fb000cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.642+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb0017070 con 0x7f8fc40cca10 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.642+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fb000a760 con 0x7f8fc40cca10 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.642+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fb000ab30 con 0x7f8fc40cca10 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.642+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc40ac8b0 con 0x7f8fc40cca10 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.642+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc413d8d0 con 0x7f8fc40cca10 2024-04-25T10:16:24.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.643+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc4073990 con 0x7f8fc40cca10 2024-04-25T10:16:24.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.644+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8fb001a070 con 0x7f8fc40cca10 2024-04-25T10:16:24.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.645+0000 7f8fc1600640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8fa406b500 0x7f8fa406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:24.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.645+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8fb0089dd0 con 0x7f8fc40cca10 2024-04-25T10:16:24.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.645+0000 7f8fc9800640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8fa406b500 0x7f8fa406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:24.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.646+0000 7f8fc9800640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8fa406b500 0x7f8fa406d9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4034000 tx=0x7f8fb402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:24.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.650+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8fb0058940 con 0x7f8fc40cca10 2024-04-25T10:16:24.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.885+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fc40cc680 con 0x7f8fc40cca10 2024-04-25T10:16:24.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.885+0000 7f8fc1600640 1 -- 172.21.15.165:0/2616671980 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4161 (secure 0 0 0) 0x7f8fb0058240 con 0x7f8fc40cca10 2024-04-25T10:16:24.890 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:24.890 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:16.251260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:24.890 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 94 2024-04-25T10:16:24.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8fa406b500 msgr2=0x7f8fa406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8fa406b500 0x7f8fa406d9c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f8fb4034000 tx=0x7f8fb402c000 comp rx=0 tx=0).stop 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 msgr2=0x7f8fc413df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 0x7f8fc413df60 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000ca70 tx=0x7f8fb000cf30 comp rx=0 tx=0).stop 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 shutdown_connections 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8fa406b500 0x7f8fa406d9c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8fc4143e60 0x7f8fc413e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8fc40cca10 0x7f8fc413df60 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 --2- 172.21.15.165:0/2616671980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8fc406e6a0 0x7f8fc413c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:24.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.888+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 >> 172.21.15.165:0/2616671980 conn(0x7f8fc406abe0 msgr2=0x7f8fc415b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:24.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.889+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 shutdown_connections 2024-04-25T10:16:24.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:24.889+0000 7f8fcac00640 1 -- 172.21.15.165:0/2616671980 wait complete. 2024-04-25T10:16:24.912 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:25.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.098+0000 7f9ed8a00640 1 Processor -- start 2024-04-25T10:16:25.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.098+0000 7f9ed8a00640 1 -- start start 2024-04-25T10:16:25.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.099+0000 7f9ed8a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9ed4143e60 0x7f9ed406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.099+0000 7f9ed8a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed4144240 con 0x7f9ed40cca10 2024-04-25T10:16:25.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.099+0000 7f9ed8a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ed415e220 con 0x7f9ed406e6a0 2024-04-25T10:16:25.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.099+0000 7f9ed8a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ed415e500 con 0x7f9ed4143e60 2024-04-25T10:16:25.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.100+0000 7f9ed3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9ed40cca10 0x7f9ed415ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33958/0 (socket says 172.21.15.165:33958) 2024-04-25T10:16:25.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.100+0000 7f9ed3e00640 1 -- 172.21.15.165:0/399210420 learned_addr learned my addr 172.21.15.165:0/399210420 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:25.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.100+0000 7f9ed3400640 1 --2- 172.21.15.165:0/399210420 >> v2:172.21.15.172:3300/0 conn(0x7f9ed4143e60 0x7f9ed406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.100+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540418846 0 0) 0x7f9ed4144240 con 0x7f9ed40cca10 2024-04-25T10:16:25.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.100+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9eb8003520 con 0x7f9ed40cca10 2024-04-25T10:16:25.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 645131995 0 0) 0x7f9eb8003520 con 0x7f9ed40cca10 2024-04-25T10:16:25.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 >> v2:172.21.15.172:3300/0 conn(0x7f9ed4143e60 msgr2=0x7f9ed406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 --2- 172.21.15.165:0/399210420 >> v2:172.21.15.172:3300/0 conn(0x7f9ed4143e60 0x7f9ed406e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 >> v1:172.21.15.172:6789/0 conn(0x7f9ed406e6a0 legacy=0x7f9ed40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed415e7e0 con 0x7f9ed40cca10 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9ecc0019f0 con 0x7f9ed40cca10 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 >> v1:172.21.15.165:6789/0 conn(0x7f9ed40cca10 legacy=0x7f9ed415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb8003a00 0x7f9eb8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb8004800 0x7f9eb8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9eb800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed415e500 con 0x7f9eb80051c0 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb800c0c0 con 0x7f9eb8003a00 2024-04-25T10:16:25.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb800c2c0 con 0x7f9eb8004800 2024-04-25T10:16:25.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.101+0000 7f9ed3e00640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9eb800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.102+0000 7f9ed3e00640 1 -- 172.21.15.165:0/399210420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb8004800 msgr2=0x7f9eb8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.102+0000 7f9ed3e00640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9eb8004800 0x7f9eb8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.102+0000 7f9ed3e00640 1 -- 172.21.15.165:0/399210420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb8003a00 msgr2=0x7f9eb8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.102+0000 7f9ed3e00640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9eb8003a00 0x7f9eb8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.102+0000 7f9ed3e00640 1 -- 172.21.15.165:0/399210420 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb8003520 con 0x7f9eb80051c0 2024-04-25T10:16:25.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.102+0000 7f9ed3e00640 1 -- 172.21.15.165:0/399210420 reap_dead start 2024-04-25T10:16:25.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.103+0000 7f9ed3e00640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9eb800b9c0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00a790 tx=0x7f9eb8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f4de5f8896f174 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.103+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc00a3b0 con 0x7f9eb80051c0 2024-04-25T10:16:25.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.103+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ecc02f070 con 0x7f9eb80051c0 2024-04-25T10:16:25.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.103+0000 7f9ed2000640 1 -- 172.21.15.165:0/399210420 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ecc02bc20 con 0x7f9eb80051c0 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.103+0000 7f9ed8a00640 1 -- 172.21.15.165:0/399210420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 msgr2=0x7f9eb800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.103+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9eb800b9c0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7f9ecc00a790 tx=0x7f9eb8005830 comp rx=0 tx=0).stop 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.104+0000 7f9ed8a00640 1 -- 172.21.15.165:0/399210420 shutdown_connections 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.104+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/399210420 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9eb800b9c0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.104+0000 7f9ed8a00640 1 -- 172.21.15.165:0/399210420 >> 172.21.15.165:0/399210420 conn(0x7f9ed406abe0 msgr2=0x7f9ed406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.104+0000 7f9ed8a00640 1 -- 172.21.15.165:0/399210420 shutdown_connections 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.104+0000 7f9ed8a00640 1 -- 172.21.15.165:0/399210420 wait complete. 2024-04-25T10:16:25.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.105+0000 7f9ed8a00640 1 Processor -- start 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.105+0000 7f9ed8a00640 1 -- start start 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.106+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9ed41576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.106+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed40cae30 0x7f9ed41564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.106+0000 7f9ed8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ed413b000 0x7f9ed413b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.106+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b2eb0 con 0x7f9eb80051c0 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.106+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40b2d30 con 0x7f9ed40cae30 2024-04-25T10:16:25.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.106+0000 7f9ed8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ed40cc300 con 0x7f9ed413b000 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9ed41576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9ed41576b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50620/0 (socket says 172.21.15.165:50620) 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 -- 172.21.15.165:0/1109183672 learned_addr learned my addr 172.21.15.165:0/1109183672 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 -- 172.21.15.165:0/1109183672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ed413b000 msgr2=0x7f9ed413b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ed413b000 0x7f9ed413b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 -- 172.21.15.165:0/1109183672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed40cae30 msgr2=0x7f9ed41564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed40cae30 0x7f9ed41564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 -- 172.21.15.165:0/1109183672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed40cb2b0 con 0x7f9eb80051c0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ed3400640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9ed41576b0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e7a0 tx=0x7f9ec400ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.107+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec4017070 con 0x7f9eb80051c0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.108+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ec400c940 con 0x7f9eb80051c0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.108+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ec400cd10 con 0x7f9eb80051c0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.108+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ed4157db0 con 0x7f9eb80051c0 2024-04-25T10:16:25.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.108+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ed4158370 con 0x7f9eb80051c0 2024-04-25T10:16:25.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.109+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ed4073990 con 0x7f9eb80051c0 2024-04-25T10:16:25.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.110+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9ec401a070 con 0x7f9eb80051c0 2024-04-25T10:16:25.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.111+0000 7f9ecb400640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9eb406b4d0 0x7f9eb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.111+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f9ec408a670 con 0x7f9eb80051c0 2024-04-25T10:16:25.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.111+0000 7f9ed2a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9eb406b4d0 0x7f9eb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.112+0000 7f9ed2a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9eb406b4d0 0x7f9eb406d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9ed41675a0 tx=0x7f9ec00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.116+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9ed40cb2b0 con 0x7f9eb80051c0 2024-04-25T10:16:25.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.329+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ed415b500 con 0x7f9eb80051c0 2024-04-25T10:16:25.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.331+0000 7f9ecb400640 1 -- 172.21.15.165:0/1109183672 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17067 (secure 0 0 0) 0x7f9ec4059300 con 0x7f9eb80051c0 2024-04-25T10:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:25.336 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":137,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:14.197143+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/2186062364":"2024-04-26T10:16:14.196999+0000","172.21.15.172:6834/2186062364":"2024-04-26T10:16:14.196999+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:25.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.334+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9eb406b4d0 msgr2=0x7f9eb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9eb406b4d0 0x7f9eb406d990 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f9ed41675a0 tx=0x7f9ec00079b0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 msgr2=0x7f9ed41576b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9ed41576b0 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7f9ec400e7a0 tx=0x7f9ec400ec60 comp rx=0 tx=0).stop 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 shutdown_connections 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9eb406b4d0 0x7f9eb406d990 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9ed413b000 0x7f9ed413b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9ed40cae30 0x7f9ed41564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 --2- 172.21.15.165:0/1109183672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9eb80051c0 0x7f9ed41576b0 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 >> 172.21.15.165:0/1109183672 conn(0x7f9ed406abe0 msgr2=0x7f9ed415aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 shutdown_connections 2024-04-25T10:16:25.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.335+0000 7f9ed8a00640 1 -- 172.21.15.165:0/1109183672 wait complete. 2024-04-25T10:16:25.357 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:15:08.823693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '135', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:15:10.098970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:25.357 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:25.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.551+0000 7fc87a800640 1 Processor -- start 2024-04-25T10:16:25.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.552+0000 7fc87a800640 1 -- start start 2024-04-25T10:16:25.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.553+0000 7fc87a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc874144d50 0x7fc87406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.553+0000 7fc87a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc8741452a0 0x7fc87406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.553+0000 7fc87a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8740cccc0 con 0x7fc8741446d0 2024-04-25T10:16:25.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.553+0000 7fc87a800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8740b2b20 con 0x7fc8741452a0 2024-04-25T10:16:25.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.553+0000 7fc87a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87406d400 con 0x7fc874144d50 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc8741452a0 0x7fc87406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc8741452a0 0x7fc87406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50632/0 (socket says 172.21.15.165:50632) 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 -- 172.21.15.165:0/951924279 learned_addr learned my addr 172.21.15.165:0/951924279 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc86be00640 1 --2- 172.21.15.165:0/951924279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc874144d50 0x7fc87406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 -- 172.21.15.165:0/951924279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc874144d50 msgr2=0x7fc87406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 --2- 172.21.15.165:0/951924279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc874144d50 0x7fc87406d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 -- 172.21.15.165:0/951924279 >> v1:172.21.15.172:6789/0 conn(0x7fc8741446d0 legacy=0x7fc8740b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.554+0000 7fc873e00640 1 -- 172.21.15.165:0/951924279 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740cc140 con 0x7fc8741452a0 2024-04-25T10:16:25.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.556+0000 7fc873e00640 1 --2- 172.21.15.165:0/951924279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8741452a0 0x7fc87406ddd0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009e50 tx=0x7fc86c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe1d8c16933e1a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.556+0000 7fc872a00640 1 -- 172.21.15.165:0/951924279 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c036070 con 0x7fc8741452a0 2024-04-25T10:16:25.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.556+0000 7fc872a00640 1 -- 172.21.15.165:0/951924279 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c031600 con 0x7fc8741452a0 2024-04-25T10:16:25.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.556+0000 7fc872a00640 1 -- 172.21.15.165:0/951924279 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0319d0 con 0x7fc8741452a0 2024-04-25T10:16:25.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 -- 172.21.15.165:0/951924279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8741452a0 msgr2=0x7fc87406ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 --2- 172.21.15.165:0/951924279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8741452a0 0x7fc87406ddd0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7fc86c009e50 tx=0x7fc86c02a710 comp rx=0 tx=0).stop 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 -- 172.21.15.165:0/951924279 shutdown_connections 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 --2- 172.21.15.165:0/951924279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8741452a0 0x7fc87406ddd0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 --2- 172.21.15.165:0/951924279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc874144d50 0x7fc87406d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 -- 172.21.15.165:0/951924279 >> 172.21.15.165:0/951924279 conn(0x7fc87406abe0 msgr2=0x7fc87406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 -- 172.21.15.165:0/951924279 shutdown_connections 2024-04-25T10:16:25.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.557+0000 7fc87a800640 1 -- 172.21.15.165:0/951924279 wait complete. 2024-04-25T10:16:25.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.559+0000 7fc87a800640 1 Processor -- start 2024-04-25T10:16:25.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.559+0000 7fc87a800640 1 -- start start 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc87a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8741446d0 0x7fc8741f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc87a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 0x7fc8740ad250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc87a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8741452a0 0x7fc8740ce1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc87a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740cc9e0 con 0x7fc874144d50 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc87a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8740cc700 con 0x7fc8741452a0 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc87a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc874144ab0 con 0x7fc8741446d0 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc86be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 0x7fc8740ad250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc86be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 0x7fc8740ad250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50638/0 (socket says 172.21.15.165:50638) 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.560+0000 7fc86be00640 1 -- 172.21.15.165:0/2111011819 learned_addr learned my addr 172.21.15.165:0/2111011819 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc873400640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8741446d0 0x7fc8741f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86be00640 1 -- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8741446d0 msgr2=0x7fc8741f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86be00640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8741446d0 0x7fc8741f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86be00640 1 -- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8741452a0 msgr2=0x7fc8740ce1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86be00640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8741452a0 0x7fc8740ce1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86be00640 1 -- 172.21.15.165:0/2111011819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8740adca0 con 0x7fc874144d50 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc873400640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8741446d0 0x7fc8741f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86be00640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 0x7fc8740ad250 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9e0 tx=0x7fc85c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c017070 con 0x7fc874144d50 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c00c9d0 con 0x7fc874144d50 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00cdc0 con 0x7fc874144d50 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.561+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8740adf80 con 0x7fc874144d50 2024-04-25T10:16:25.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.562+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8741575f0 con 0x7fc874144d50 2024-04-25T10:16:25.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.563+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc874073a00 con 0x7fc874144d50 2024-04-25T10:16:25.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.564+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc85c01a070 con 0x7fc874144d50 2024-04-25T10:16:25.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.565+0000 7fc86b400640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85006b500 0x7fc85006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:25.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.566+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fc85c08a730 con 0x7fc874144d50 2024-04-25T10:16:25.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.566+0000 7fc873400640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85006b500 0x7fc85006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:25.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.566+0000 7fc873400640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85006b500 0x7fc85006d9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc864001680 tx=0x7fc864009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:25.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.570+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc85c0593d0 con 0x7fc874144d50 2024-04-25T10:16:25.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.769+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc87406e8a0 con 0x7fc874144d50 2024-04-25T10:16:25.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.770+0000 7fc86b400640 1 -- 172.21.15.165:0/2111011819 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc85c058cd0 con 0x7fc874144d50 2024-04-25T10:16:25.774 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:16:25.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85006b500 msgr2=0x7fc85006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85006b500 0x7fc85006d9c0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fc864001680 tx=0x7fc864009040 comp rx=0 tx=0).stop 2024-04-25T10:16:25.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 msgr2=0x7fc8740ad250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:25.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 0x7fc8740ad250 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00e9e0 tx=0x7fc85c00eea0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 shutdown_connections 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85006b500 0x7fc85006d9c0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8741452a0 0x7fc8740ce1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874144d50 0x7fc8740ad250 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 --2- 172.21.15.165:0/2111011819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8741446d0 0x7fc8741f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.772+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 >> 172.21.15.165:0/2111011819 conn(0x7fc87406abe0 msgr2=0x7fc874071230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.773+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 shutdown_connections 2024-04-25T10:16:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.773+0000 7fc87a800640 1 -- 172.21.15.165:0/2111011819 wait complete. 2024-04-25T10:16:25.795 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:16:25.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.994+0000 7f58ef200640 1 Processor -- start 2024-04-25T10:16:25.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.994+0000 7f58ef200640 1 -- start start 2024-04-25T10:16:26.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.996+0000 7f58ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 0x7f58f00b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.996+0000 7f58ef200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00b5490 con 0x7f58f00b1fb0 2024-04-25T10:16:26.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.996+0000 7f58ef200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00b5730 con 0x7f58f00d0d40 2024-04-25T10:16:26.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.996+0000 7f58ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f015ea60 con 0x7f58f00b1140 2024-04-25T10:16:26.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.996+0000 7f58ed400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f58f00b1fb0 0x7f58f00d0630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33972/0 (socket says 172.21.15.165:33972) 2024-04-25T10:16:26.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.996+0000 7f58ed400640 1 -- 172.21.15.165:0/3428800033 learned_addr learned my addr 172.21.15.165:0/3428800033 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:26.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58ede00640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 0x7f58f00b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277151312 0 0) 0x7f58f00b5490 con 0x7f58f00b1fb0 2024-04-25T10:16:26.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58d4003520 con 0x7f58f00b1fb0 2024-04-25T10:16:26.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 68404908 0 0) 0x7f58d4003520 con 0x7f58f00b1fb0 2024-04-25T10:16:26.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 msgr2=0x7f58f00b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 0x7f58f00b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 >> v1:172.21.15.172:6789/0 conn(0x7f58f00d0d40 legacy=0x7f58f0146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f00b5a10 con 0x7f58f00b1fb0 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f58dc001670 con 0x7f58f00b1fb0 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 >> v1:172.21.15.165:6789/0 conn(0x7f58f00b1fb0 legacy=0x7f58f00d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d40039e0 0x7f58d4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 0x7f58d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d4004d40 0x7f58d40051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f015ea60 con 0x7f58d4004d40 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58d400bbd0 con 0x7f58d40039e0 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.997+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58d400bdd0 con 0x7f58f00b1140 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ee800640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d4004d40 0x7f58d40051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ede00640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d40039e0 0x7f58d4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ed400640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 0x7f58d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ee800640 1 -- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 msgr2=0x7f58d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ee800640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f00b1140 0x7f58d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ee800640 1 -- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d40039e0 msgr2=0x7f58d4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ee800640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d40039e0 0x7f58d4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.998+0000 7f58ee800640 1 -- 172.21.15.165:0/3428800033 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4003520 con 0x7f58d4004d40 2024-04-25T10:16:26.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:25.999+0000 7f58ee800640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d4004d40 0x7f58d40051a0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f58e8001e20 tx=0x7f58d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf733706cc794e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.000+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8001a70 con 0x7f58d4004d40 2024-04-25T10:16:26.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.000+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e802ae40 con 0x7f58d4004d40 2024-04-25T10:16:26.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.000+0000 7f58eca00640 1 -- 172.21.15.165:0/3428800033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e8003bb0 con 0x7f58d4004d40 2024-04-25T10:16:26.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.000+0000 7f58ef200640 1 -- 172.21.15.165:0/3428800033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d4004d40 msgr2=0x7f58d40051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.000+0000 7f58ef200640 1 --2- 172.21.15.165:0/3428800033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58d4004d40 0x7f58d40051a0 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7f58e8001e20 tx=0x7f58d4005830 comp rx=0 tx=0).stop 2024-04-25T10:16:26.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.000+0000 7f58ee800640 1 -- 172.21.15.165:0/3428800033 reap_dead start 2024-04-25T10:16:26.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.001+0000 7f58ef200640 1 -- 172.21.15.165:0/3428800033 shutdown_connections 2024-04-25T10:16:26.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.001+0000 7f58ef200640 1 -- 172.21.15.165:0/3428800033 >> 172.21.15.165:0/3428800033 conn(0x7f58f00728b0 msgr2=0x7f58f0099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.001+0000 7f58ef200640 1 -- 172.21.15.165:0/3428800033 shutdown_connections 2024-04-25T10:16:26.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.001+0000 7f58ef200640 1 -- 172.21.15.165:0/3428800033 wait complete. 2024-04-25T10:16:26.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.002+0000 7f58ef200640 1 Processor -- start 2024-04-25T10:16:26.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.003+0000 7f58ef200640 1 -- start start 2024-04-25T10:16:26.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4004d40 0x7f58f00ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 0x7f58f00b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f015bda0 0x7f58f0146d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ef200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0160360 con 0x7f58f0147a50 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f01601e0 con 0x7f58d4004d40 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f01604e0 con 0x7f58f015bda0 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 0x7f58f00b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 0x7f58f00b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50664/0 (socket says 172.21.15.165:50664) 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 -- 172.21.15.165:0/3923362031 learned_addr learned my addr 172.21.15.165:0/3923362031 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 -- 172.21.15.165:0/3923362031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f015bda0 msgr2=0x7f58f0146d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f015bda0 0x7f58f0146d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 -- 172.21.15.165:0/3923362031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4004d40 msgr2=0x7f58f00ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4004d40 0x7f58f00ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.004+0000 7f58ed400640 1 -- 172.21.15.165:0/3923362031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0158ff0 con 0x7f58f0147a50 2024-04-25T10:16:26.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.005+0000 7f58ed400640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 0x7f58f00b02a0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e760 tx=0x7f58dc00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.005+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00c640 con 0x7f58f0147a50 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.005+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58dc018070 con 0x7f58f0147a50 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.005+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f01592d0 con 0x7f58f0147a50 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.005+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58dc00ca00 con 0x7f58f0147a50 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.005+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f0159890 con 0x7f58f0147a50 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.006+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f009a1e0 con 0x7f58f0147a50 2024-04-25T10:16:26.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.007+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58dc00cba0 con 0x7f58f0147a50 2024-04-25T10:16:26.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.008+0000 7f58e6000640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58d006b4d0 0x7f58d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.009+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f58dc08a280 con 0x7f58f0147a50 2024-04-25T10:16:26.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.009+0000 7f58ede00640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58d006b4d0 0x7f58d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.009+0000 7f58ede00640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58d006b4d0 0x7f58d006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009910 tx=0x7f58e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.013+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58f009a1e0 con 0x7f58f0147a50 2024-04-25T10:16:26.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.206+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f58f0159a90 con 0x7f58f0147a50 2024-04-25T10:16:26.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.207+0000 7f58e6000640 1 -- 172.21.15.165:0/3923362031 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v137) v1 ==== 104+0+11 (secure 0 0 0) 0x7f58f0159a90 con 0x7f58f0147a50 2024-04-25T10:16:26.211 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:16:26.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.209+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58d006b4d0 msgr2=0x7f58d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.209+0000 7f58ef200640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58d006b4d0 0x7f58d006d990 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f58e0009910 tx=0x7f58e0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:26.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.209+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 msgr2=0x7f58f00b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.209+0000 7f58ef200640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 0x7f58f00b02a0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00e760 tx=0x7f58dc00ec20 comp rx=0 tx=0).stop 2024-04-25T10:16:26.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 shutdown_connections 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58d006b4d0 0x7f58d006d990 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58f015bda0 0x7f58f0146d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58f0147a50 0x7f58f00b02a0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 --2- 172.21.15.165:0/3923362031 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58d4004d40 0x7f58f00ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 >> 172.21.15.165:0/3923362031 conn(0x7f58f00728b0 msgr2=0x7f58f0160b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 shutdown_connections 2024-04-25T10:16:26.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.210+0000 7f58ef200640 1 -- 172.21.15.165:0/3923362031 wait complete. 2024-04-25T10:16:26.234 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:16:26.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.432+0000 7fbb22800640 1 Processor -- start 2024-04-25T10:16:26.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.432+0000 7fbb22800640 1 -- start start 2024-04-25T10:16:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.434+0000 7fbb22800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb1c15b3c0 0x7fbb1c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.434+0000 7fbb22800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 0x7fbb1c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.434+0000 7fbb22800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0b0b80 con 0x7fbb1c0cdae0 2024-04-25T10:16:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.434+0000 7fbb22800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cd650 con 0x7fbb1c15b3c0 2024-04-25T10:16:26.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.434+0000 7fbb22800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cc800 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.435+0000 7fbb20a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 0x7fbb1c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.435+0000 7fbb21400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb1c15b3c0 0x7fbb1c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.435+0000 7fbb20a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 0x7fbb1c0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:48262/0 (socket says 172.21.15.165:48262) 2024-04-25T10:16:26.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.435+0000 7fbb20a00640 1 -- 172.21.15.165:0/2792524901 learned_addr learned my addr 172.21.15.165:0/2792524901 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.436+0000 7fbb20a00640 1 -- 172.21.15.165:0/2792524901 >> v2:172.21.15.172:3300/0 conn(0x7fbb1c15b3c0 msgr2=0x7fbb1c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.436+0000 7fbb20a00640 1 --2- 172.21.15.165:0/2792524901 >> v2:172.21.15.172:3300/0 conn(0x7fbb1c15b3c0 0x7fbb1c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.436+0000 7fbb1be00640 1 -- 172.21.15.165:0/2792524901 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2505178979 0 0) 0x7fbb14001110 con 0x7fbb1c0cdae0 2024-04-25T10:16:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.436+0000 7fbb20a00640 1 -- 172.21.15.165:0/2792524901 >> v1:172.21.15.172:6789/0 conn(0x7fbb1c0cdae0 legacy=0x7fbb1c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.436+0000 7fbb20a00640 1 -- 172.21.15.165:0/2792524901 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0ccae0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.437+0000 7fbb20a00640 1 --2- 172.21.15.165:0/2792524901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 0x7fbb1c0cc150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009870 tx=0x7fbb0802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=597b2d00e5a3e7ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.437+0000 7fbb1be00640 1 -- 172.21.15.165:0/2792524901 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037070 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.437+0000 7fbb1be00640 1 -- 172.21.15.165:0/2792524901 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb08032440 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.437+0000 7fbb1be00640 1 -- 172.21.15.165:0/2792524901 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb08032810 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 -- 172.21.15.165:0/2792524901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 msgr2=0x7fbb1c0cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 --2- 172.21.15.165:0/2792524901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 0x7fbb1c0cc150 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009870 tx=0x7fbb0802a270 comp rx=0 tx=0).stop 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 -- 172.21.15.165:0/2792524901 shutdown_connections 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 --2- 172.21.15.165:0/2792524901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b7a0 0x7fbb1c0cc150 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 --2- 172.21.15.165:0/2792524901 >> v2:172.21.15.172:3300/0 conn(0x7fbb1c15b3c0 0x7fbb1c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 -- 172.21.15.165:0/2792524901 >> 172.21.15.165:0/2792524901 conn(0x7fbb1c061cc0 msgr2=0x7fbb1c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 -- 172.21.15.165:0/2792524901 shutdown_connections 2024-04-25T10:16:26.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.438+0000 7fbb22800640 1 -- 172.21.15.165:0/2792524901 wait complete. 2024-04-25T10:16:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.440+0000 7fbb22800640 1 Processor -- start 2024-04-25T10:16:26.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.440+0000 7fbb22800640 1 -- start start 2024-04-25T10:16:26.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb22800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1c0cdae0 0x7fbb1c14a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb22800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b3c0 0x7fbb1c0b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb22800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 0x7fbb1c0b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb22800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c15e8a0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb22800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c15e720 con 0x7fbb1c0cdae0 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb22800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cdf40 con 0x7fbb1c15b3c0 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 0x7fbb1c0b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb21e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 0x7fbb1c0b3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50680/0 (socket says 172.21.15.165:50680) 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.441+0000 7fbb21e00640 1 -- 172.21.15.165:0/624793831 learned_addr learned my addr 172.21.15.165:0/624793831 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb20a00640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b3c0 0x7fbb1c0b3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21400640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1c0cdae0 0x7fbb1c14a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21e00640 1 -- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b3c0 msgr2=0x7fbb1c0b3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21e00640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b3c0 0x7fbb1c0b3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21e00640 1 -- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1c0cdae0 msgr2=0x7fbb1c14a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21e00640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1c0cdae0 0x7fbb1c14a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21e00640 1 -- 172.21.15.165:0/624793831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c14adc0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb20a00640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b3c0 0x7fbb1c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb21e00640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 0x7fbb1c0b3ab0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c3e0 tx=0x7fbb1400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14017070 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb140035f0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.442+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14003a20 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.443+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c0a92f0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.443+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c14b340 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.444+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c073990 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.445+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb14003bc0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.445+0000 7fbb19600640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.446+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fbb140898a0 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.446+0000 7fbb21400640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.447+0000 7fbb21400640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbafc06b4d0 0x7fbafc06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbb100074a0 tx=0x7fbb1000b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.450+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb1c0b0b80 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.647+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbb1c072360 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.647+0000 7fbb19600640 1 -- 172.21.15.165:0/624793831 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v137) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbb14058410 con 0x7fbb1c15b7a0 2024-04-25T10:16:26.652 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:16:26.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbafc06b4d0 msgr2=0x7fbafc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbafc06b4d0 0x7fbafc06d990 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fbb100074a0 tx=0x7fbb1000b040 comp rx=0 tx=0).stop 2024-04-25T10:16:26.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 msgr2=0x7fbb1c0b3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 0x7fbb1c0b3ab0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7fbb1400c3e0 tx=0x7fbb1400c8a0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 shutdown_connections 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbafc06b4d0 0x7fbafc06d990 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb1c15b7a0 0x7fbb1c0b3ab0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb1c15b3c0 0x7fbb1c0b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 --2- 172.21.15.165:0/624793831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb1c0cdae0 0x7fbb1c14a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 >> 172.21.15.165:0/624793831 conn(0x7fbb1c061cc0 msgr2=0x7fbb1c15bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 shutdown_connections 2024-04-25T10:16:26.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.650+0000 7fbb22800640 1 -- 172.21.15.165:0/624793831 wait complete. 2024-04-25T10:16:26.674 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:26.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.871+0000 7fea07200640 1 Processor -- start 2024-04-25T10:16:26.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.871+0000 7fea07200640 1 -- start start 2024-04-25T10:16:26.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea07200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fea00154ee0 0x7fea000cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea07200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea000cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea07200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fea000ce3c0 0x7fea0015e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea07200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea000ce830 con 0x7fea00154ee0 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea07200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea00072090 con 0x7fea000ce3c0 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea07200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea000d0150 con 0x7fea000cfcf0 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fe9ffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fea00154ee0 0x7fea000cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fe9ffe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fea00154ee0 0x7fea000cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51788/0 (socket says 172.21.15.165:51788) 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fe9ffe00640 1 -- 172.21.15.165:0/793299108 learned_addr learned my addr 172.21.15.165:0/793299108 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.873+0000 7fea05200640 1 --2- 172.21.15.165:0/793299108 >> v2:172.21.15.172:3300/0 conn(0x7fea000ce3c0 0x7fea0015e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.874+0000 7fe9ff400640 1 --2- 172.21.15.165:0/793299108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea000cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.874+0000 7fe9ffe00640 1 -- 172.21.15.165:0/793299108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 msgr2=0x7fea000cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.874+0000 7fe9ffe00640 1 --2- 172.21.15.165:0/793299108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea000cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.874+0000 7fe9ffe00640 1 -- 172.21.15.165:0/793299108 >> v2:172.21.15.172:3300/0 conn(0x7fea000ce3c0 msgr2=0x7fea0015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:26.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.874+0000 7fe9ffe00640 1 --2- 172.21.15.165:0/793299108 >> v2:172.21.15.172:3300/0 conn(0x7fea000ce3c0 0x7fea0015e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.874+0000 7fe9ffe00640 1 -- 172.21.15.165:0/793299108 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea00071480 con 0x7fea00154ee0 2024-04-25T10:16:26.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.875+0000 7fe9ffe00640 1 --2- 172.21.15.165:0/793299108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea00154ee0 0x7fea000cf7b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fea0006f6b0 tx=0x7fe9f002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f26b7bf91392194 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.875+0000 7fe9fea00640 1 -- 172.21.15.165:0/793299108 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0037070 con 0x7fea00154ee0 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fe9fea00640 1 -- 172.21.15.165:0/793299108 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f0032440 con 0x7fea00154ee0 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fe9fea00640 1 -- 172.21.15.165:0/793299108 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0032810 con 0x7fea00154ee0 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 -- 172.21.15.165:0/793299108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea00154ee0 msgr2=0x7fea000cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 --2- 172.21.15.165:0/793299108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea00154ee0 0x7fea000cf7b0 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fea0006f6b0 tx=0x7fe9f002a210 comp rx=0 tx=0).stop 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 -- 172.21.15.165:0/793299108 shutdown_connections 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 --2- 172.21.15.165:0/793299108 >> v2:172.21.15.172:3300/0 conn(0x7fea000ce3c0 0x7fea0015e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 --2- 172.21.15.165:0/793299108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea000cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 --2- 172.21.15.165:0/793299108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea00154ee0 0x7fea000cf7b0 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 -- 172.21.15.165:0/793299108 >> 172.21.15.165:0/793299108 conn(0x7fea0006abe0 msgr2=0x7fea0006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:26.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.876+0000 7fea07200640 1 -- 172.21.15.165:0/793299108 shutdown_connections 2024-04-25T10:16:26.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.877+0000 7fea07200640 1 -- 172.21.15.165:0/793299108 wait complete. 2024-04-25T10:16:26.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.878+0000 7fea07200640 1 Processor -- start 2024-04-25T10:16:26.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.878+0000 7fea07200640 1 -- start start 2024-04-25T10:16:26.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.879+0000 7fea07200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 0x7fea001e7be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.879+0000 7fea07200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea00143880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.879+0000 7fea07200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea00154ee0 0x7fea000b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.879+0000 7fea07200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00071a40 con 0x7fea000ce3c0 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.879+0000 7fea07200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea00071230 con 0x7fea00154ee0 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.879+0000 7fea07200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea00071d20 con 0x7fea000cfcf0 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 0x7fea001e7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 0x7fea001e7be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51802/0 (socket says 172.21.15.165:51802) 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 -- 172.21.15.165:0/1735457141 learned_addr learned my addr 172.21.15.165:0/1735457141 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 -- 172.21.15.165:0/1735457141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 msgr2=0x7fea00143880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea00143880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 -- 172.21.15.165:0/1735457141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea00154ee0 msgr2=0x7fea000b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea00154ee0 0x7fea000b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 -- 172.21.15.165:0/1735457141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea001e3f60 con 0x7fea000ce3c0 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.880+0000 7fe9ffe00640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 0x7fea001e7be0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000c00 tx=0x7fe9f00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.881+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0041020 con 0x7fea000ce3c0 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.881+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f00306c0 con 0x7fea000ce3c0 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.881+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f0037040 con 0x7fea000ce3c0 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.881+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea001e5570 con 0x7fea000ce3c0 2024-04-25T10:16:26.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.881+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea001e5b30 con 0x7fea000ce3c0 2024-04-25T10:16:26.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.882+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea00073990 con 0x7fea000ce3c0 2024-04-25T10:16:26.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.883+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9f0044020 con 0x7fea000ce3c0 2024-04-25T10:16:26.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.884+0000 7fe9f7e00640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9e006b4d0 0x7fe9e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:26.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.884+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe9f00aabb0 con 0x7fea000ce3c0 2024-04-25T10:16:26.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.884+0000 7fe9ff400640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9e006b4d0 0x7fe9e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:26.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.885+0000 7fe9ff400640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9e006b4d0 0x7fe9e006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fea00146aa0 tx=0x7fe9ec008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:26.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:26.888+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9f004b030 con 0x7fea000ce3c0 2024-04-25T10:16:27.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.125+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fea0006d360 con 0x7fea000ce3c0 2024-04-25T10:16:27.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.126+0000 7fe9f7e00640 1 -- 172.21.15.165:0/1735457141 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4161 (secure 0 0 0) 0x7fe9f007f030 con 0x7fea000ce3c0 2024-04-25T10:16:27.131 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:27.131 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:16.251260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:27.131 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 94 2024-04-25T10:16:27.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9e006b4d0 msgr2=0x7fe9e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9e006b4d0 0x7fe9e006d990 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7fea00146aa0 tx=0x7fe9ec008040 comp rx=0 tx=0).stop 2024-04-25T10:16:27.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 msgr2=0x7fea001e7be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 0x7fea001e7be0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7fe9f0000c00 tx=0x7fe9f00046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 shutdown_connections 2024-04-25T10:16:27.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9e006b4d0 0x7fe9e006d990 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fea00154ee0 0x7fea000b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.129+0000 7fea07200640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fea000cfcf0 0x7fea00143880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.130+0000 7fea07200640 1 --2- 172.21.15.165:0/1735457141 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fea000ce3c0 0x7fea001e7be0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.130+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 >> 172.21.15.165:0/1735457141 conn(0x7fea0006abe0 msgr2=0x7fea0015c520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.130+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 shutdown_connections 2024-04-25T10:16:27.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.130+0000 7fea07200640 1 -- 172.21.15.165:0/1735457141 wait complete. 2024-04-25T10:16:27.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:27.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.347+0000 7f9036000640 1 Processor -- start 2024-04-25T10:16:27.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.347+0000 7f9036000640 1 -- start start 2024-04-25T10:16:27.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.348+0000 7f9036000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f903015dbe0 0x7f90300d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.348+0000 7f9036000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90300b92a0 0x7f90300d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.348+0000 7f9036000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90300b97a0 con 0x7f90300b2cb0 2024-04-25T10:16:27.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.348+0000 7f9036000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f903015dfc0 con 0x7f90300b92a0 2024-04-25T10:16:27.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.348+0000 7f9036000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90300d1a00 con 0x7f903015dbe0 2024-04-25T10:16:27.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.349+0000 7f902fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90300b92a0 0x7f90300d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.349+0000 7f902fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f90300b92a0 0x7f90300d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51818/0 (socket says 172.21.15.165:51818) 2024-04-25T10:16:27.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.349+0000 7f902fe00640 1 -- 172.21.15.165:0/1382003625 learned_addr learned my addr 172.21.15.165:0/1382003625 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.349+0000 7f902f400640 1 --2- 172.21.15.165:0/1382003625 >> v2:172.21.15.172:3300/0 conn(0x7f903015dbe0 0x7f90300d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.350+0000 7f902fe00640 1 -- 172.21.15.165:0/1382003625 >> v2:172.21.15.172:3300/0 conn(0x7f903015dbe0 msgr2=0x7f90300d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.350+0000 7f902fe00640 1 --2- 172.21.15.165:0/1382003625 >> v2:172.21.15.172:3300/0 conn(0x7f903015dbe0 0x7f90300d0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.350+0000 7f902fe00640 1 -- 172.21.15.165:0/1382003625 >> v1:172.21.15.172:6789/0 conn(0x7f90300b2cb0 legacy=0x7f90300b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.350+0000 7f902fe00640 1 -- 172.21.15.165:0/1382003625 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300d1c50 con 0x7f90300b92a0 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.350+0000 7f902f400640 1 --2- 172.21.15.165:0/1382003625 >> v2:172.21.15.172:3300/0 conn(0x7f903015dbe0 0x7f90300d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:27.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.351+0000 7f902fe00640 1 --2- 172.21.15.165:0/1382003625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90300b92a0 0x7f90300d0e50 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f903006c140 tx=0x7f902802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d31d9dc92e6877d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.351+0000 7f902e000640 1 -- 172.21.15.165:0/1382003625 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028036070 con 0x7f90300b92a0 2024-04-25T10:16:27.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.351+0000 7f902e000640 1 -- 172.21.15.165:0/1382003625 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9028031600 con 0x7f90300b92a0 2024-04-25T10:16:27.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.351+0000 7f902e000640 1 -- 172.21.15.165:0/1382003625 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90280319d0 con 0x7f90300b92a0 2024-04-25T10:16:27.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 -- 172.21.15.165:0/1382003625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90300b92a0 msgr2=0x7f90300d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 --2- 172.21.15.165:0/1382003625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90300b92a0 0x7f90300d0e50 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f903006c140 tx=0x7f902802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 -- 172.21.15.165:0/1382003625 shutdown_connections 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 --2- 172.21.15.165:0/1382003625 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90300b92a0 0x7f90300d0e50 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 --2- 172.21.15.165:0/1382003625 >> v2:172.21.15.172:3300/0 conn(0x7f903015dbe0 0x7f90300d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 -- 172.21.15.165:0/1382003625 >> 172.21.15.165:0/1382003625 conn(0x7f9030061a20 msgr2=0x7f9030061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 -- 172.21.15.165:0/1382003625 shutdown_connections 2024-04-25T10:16:27.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.352+0000 7f9036000640 1 -- 172.21.15.165:0/1382003625 wait complete. 2024-04-25T10:16:27.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.354+0000 7f9036000640 1 Processor -- start 2024-04-25T10:16:27.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.354+0000 7f9036000640 1 -- start start 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f9036000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90300b2cb0 0x7f9030159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f9036000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90300b92a0 0x7f9030155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f9036000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 0x7f9030156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f9036000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300b6ab0 con 0x7f903015dbe0 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f9036000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90300d2210 con 0x7f90300b92a0 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f9036000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90300b6c30 con 0x7f90300b2cb0 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f902fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 0x7f9030156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f902fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 0x7f9030156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51820/0 (socket says 172.21.15.165:51820) 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f902fe00640 1 -- 172.21.15.165:0/1689587859 learned_addr learned my addr 172.21.15.165:0/1689587859 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f902f400640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90300b2cb0 0x7f9030159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.355+0000 7f902ea00640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90300b92a0 0x7f9030155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f902fe00640 1 -- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90300b2cb0 msgr2=0x7f9030159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f902fe00640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90300b2cb0 0x7f9030159fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f902fe00640 1 -- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90300b92a0 msgr2=0x7f9030155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f902fe00640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90300b92a0 0x7f9030155b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f902fe00640 1 -- 172.21.15.165:0/1689587859 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90300cc1c0 con 0x7f903015dbe0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f902fe00640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 0x7f9030156090 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f902802ac90 tx=0x7f9028004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902803f020 con 0x7f903015dbe0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9028037040 con 0x7f903015dbe0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9028036070 con 0x7f903015dbe0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.356+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90300cc4a0 con 0x7f903015dbe0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.357+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90300cca60 con 0x7f903015dbe0 2024-04-25T10:16:27.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.358+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9030073990 con 0x7f903015dbe0 2024-04-25T10:16:27.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.359+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9028042020 con 0x7f903015dbe0 2024-04-25T10:16:27.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.359+0000 7f9027400640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f901006b4d0 0x7f901006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.360+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f90280aad00 con 0x7f903015dbe0 2024-04-25T10:16:27.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.360+0000 7f902f400640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f901006b4d0 0x7f901006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.361+0000 7f902f400640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f901006b4d0 0x7f901006d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f902000a950 tx=0x7f9020008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.364+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9028049020 con 0x7f903015dbe0 2024-04-25T10:16:27.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.558+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f903006c6d0 con 0x7f903015dbe0 2024-04-25T10:16:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.561+0000 7f9027400640 1 -- 172.21.15.165:0/1689587859 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17067 (secure 0 0 0) 0x7f9028079990 con 0x7f903015dbe0 2024-04-25T10:16:27.566 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:27.566 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":137,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:14.197143+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/2186062364":"2024-04-26T10:16:14.196999+0000","172.21.15.172:6834/2186062364":"2024-04-26T10:16:14.196999+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.563+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f901006b4d0 msgr2=0x7f901006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.563+0000 7f9036000640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f901006b4d0 0x7f901006d990 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f902000a950 tx=0x7f9020008040 comp rx=0 tx=0).stop 2024-04-25T10:16:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 msgr2=0x7f9030156090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 0x7f9030156090 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f902802ac90 tx=0x7f9028004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 shutdown_connections 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f901006b4d0 0x7f901006d990 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f903015dbe0 0x7f9030156090 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90300b92a0 0x7f9030155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 --2- 172.21.15.165:0/1689587859 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f90300b2cb0 0x7f9030159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 >> 172.21.15.165:0/1689587859 conn(0x7f9030061a20 msgr2=0x7f90300b7320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 shutdown_connections 2024-04-25T10:16:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.564+0000 7f9036000640 1 -- 172.21.15.165:0/1689587859 wait complete. 2024-04-25T10:16:27.586 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:16:27.586 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:27.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.781+0000 7fce90a00640 1 Processor -- start 2024-04-25T10:16:27.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.782+0000 7fce90a00640 1 -- start start 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.783+0000 7fce90a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c15db70 0x7fce8c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.783+0000 7fce90a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce8c0cfc20 0x7fce8c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.783+0000 7fce90a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce8c071650 con 0x7fce8c15df50 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.783+0000 7fce90a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce8c0cbd80 con 0x7fce8c0cfc20 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.783+0000 7fce90a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c161910 con 0x7fce8c15db70 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce8c0cfc20 0x7fce8c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fce8c0cfc20 0x7fce8c06d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51824/0 (socket says 172.21.15.165:51824) 2024-04-25T10:16:27.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 -- 172.21.15.165:0/3617004324 learned_addr learned my addr 172.21.15.165:0/3617004324 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce89e00640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c15db70 0x7fce8c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 -- 172.21.15.165:0/3617004324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c15db70 msgr2=0x7fce8c0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c15db70 0x7fce8c0cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 -- 172.21.15.165:0/3617004324 >> v1:172.21.15.172:6789/0 conn(0x7fce8c15df50 legacy=0x7fce8c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce8a800640 1 -- 172.21.15.165:0/3617004324 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0cf230 con 0x7fce8c0cfc20 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.784+0000 7fce89e00640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c15db70 0x7fce8c0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.785+0000 7fce8a800640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c0cfc20 0x7fce8c06d290 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fce84009e50 tx=0x7fce8402a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be92af6b262d883f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.785+0000 7fce89400640 1 -- 172.21.15.165:0/3617004324 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84036070 con 0x7fce8c0cfc20 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce89400640 1 -- 172.21.15.165:0/3617004324 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84031600 con 0x7fce8c0cfc20 2024-04-25T10:16:27.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce89400640 1 -- 172.21.15.165:0/3617004324 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce840319d0 con 0x7fce8c0cfc20 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 -- 172.21.15.165:0/3617004324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c0cfc20 msgr2=0x7fce8c06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c0cfc20 0x7fce8c06d290 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fce84009e50 tx=0x7fce8402a6d0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 -- 172.21.15.165:0/3617004324 shutdown_connections 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c0cfc20 0x7fce8c06d290 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 --2- 172.21.15.165:0/3617004324 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c15db70 0x7fce8c0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 -- 172.21.15.165:0/3617004324 >> 172.21.15.165:0/3617004324 conn(0x7fce8c06abe0 msgr2=0x7fce8c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 -- 172.21.15.165:0/3617004324 shutdown_connections 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.786+0000 7fce90a00640 1 -- 172.21.15.165:0/3617004324 wait complete. 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.788+0000 7fce90a00640 1 Processor -- start 2024-04-25T10:16:27.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.788+0000 7fce90a00640 1 -- start start 2024-04-25T10:16:27.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce90a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c0cfc20 0x7fce8c147130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce90a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 0x7fce8c1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce90a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8c15df50 0x7fce8c1f3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce90a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c162120 con 0x7fce8c15db70 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce90a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c161e40 con 0x7fce8c15df50 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce90a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce8c162640 con 0x7fce8c0cfc20 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce81400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 0x7fce8c1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce81400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 0x7fce8c1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51828/0 (socket says 172.21.15.165:51828) 2024-04-25T10:16:27.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.789+0000 7fce81400640 1 -- 172.21.15.165:0/273982378 learned_addr learned my addr 172.21.15.165:0/273982378 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:27.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce81400640 1 -- 172.21.15.165:0/273982378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c0cfc20 msgr2=0x7fce8c147130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce81400640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c0cfc20 0x7fce8c147130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce81400640 1 -- 172.21.15.165:0/273982378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8c15df50 msgr2=0x7fce8c1f3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:27.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce81400640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8c15df50 0x7fce8c1f3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:27.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce81400640 1 -- 172.21.15.165:0/273982378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c15f8d0 con 0x7fce8c15db70 2024-04-25T10:16:27.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce81400640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 0x7fce8c1593c0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b3f0 tx=0x7fce7400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74015020 con 0x7fce8c15db70 2024-04-25T10:16:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce74011830 con 0x7fce8c15db70 2024-04-25T10:16:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8c15fbb0 con 0x7fce8c15db70 2024-04-25T10:16:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.790+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce74011c20 con 0x7fce8c15db70 2024-04-25T10:16:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.791+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8c160170 con 0x7fce8c15db70 2024-04-25T10:16:27.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.792+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce8c073990 con 0x7fce8c15db70 2024-04-25T10:16:27.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.793+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fce74018020 con 0x7fce8c15db70 2024-04-25T10:16:27.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.794+0000 7fce82a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6806b500 0x7fce6806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:27.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.794+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fce74089f80 con 0x7fce8c15db70 2024-04-25T10:16:27.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.794+0000 7fce89e00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6806b500 0x7fce6806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:27.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.795+0000 7fce89e00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6806b500 0x7fce6806d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fce7c006fd0 tx=0x7fce7c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:27.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:27.798+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fce7401f030 con 0x7fce8c15db70 2024-04-25T10:16:28.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.028+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce8c0723c0 con 0x7fce8c15db70 2024-04-25T10:16:28.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.029+0000 7fce82a00640 1 -- 172.21.15.165:0/273982378 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4161 (secure 0 0 0) 0x7fce740172c0 con 0x7fce8c15db70 2024-04-25T10:16:28.033 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:28.034 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:16.251260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:28.034 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 94 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.031+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6806b500 msgr2=0x7fce6806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.031+0000 7fce90a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6806b500 0x7fce6806d9c0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fce7c006fd0 tx=0x7fce7c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.031+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 msgr2=0x7fce8c1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.031+0000 7fce90a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 0x7fce8c1593c0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7fce7400b3f0 tx=0x7fce7400b8b0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 shutdown_connections 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fce6806b500 0x7fce6806d9c0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fce8c15df50 0x7fce8c1f3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fce8c15db70 0x7fce8c1593c0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 --2- 172.21.15.165:0/273982378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fce8c0cfc20 0x7fce8c147130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 >> 172.21.15.165:0/273982378 conn(0x7fce8c06abe0 msgr2=0x7fce8c15c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 shutdown_connections 2024-04-25T10:16:28.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.032+0000 7fce90a00640 1 -- 172.21.15.165:0/273982378 wait complete. 2024-04-25T10:16:28.055 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:28.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.246+0000 7f5ed0a00640 1 Processor -- start 2024-04-25T10:16:28.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.246+0000 7f5ed0a00640 1 -- start start 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ed0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 0x7f5ecc06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ed0a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ecc06c130 0x7f5ecc0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ed0a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ecc15e7d0 con 0x7f5ecc0addc0 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ed0a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ecc06c5a0 con 0x7f5ecc06c130 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ed0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15ed90 con 0x7f5ecc0728a0 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ecb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5ecc0addc0 0x7f5ecc15de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40458/0 (socket says 172.21.15.165:40458) 2024-04-25T10:16:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ecb400640 1 -- 172.21.15.165:0/3888052417 learned_addr learned my addr 172.21.15.165:0/3888052417 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925240848 0 0) 0x7f5ecc15e7d0 con 0x7f5ecc0addc0 2024-04-25T10:16:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ecbe00640 1 --2- 172.21.15.165:0/3888052417 >> v2:172.21.15.172:3300/0 conn(0x7f5ecc06c130 0x7f5ecc0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5ecaa00640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 0x7f5ecc06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.248+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5eb0003520 con 0x7f5ecc0addc0 2024-04-25T10:16:28.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456690420 0 0) 0x7f5eb0003520 con 0x7f5ecc0addc0 2024-04-25T10:16:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 msgr2=0x7f5ecc06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 0x7f5ecc06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 >> v2:172.21.15.172:3300/0 conn(0x7f5ecc06c130 msgr2=0x7f5ecc0b3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 --2- 172.21.15.165:0/3888052417 >> v2:172.21.15.172:3300/0 conn(0x7f5ecc06c130 0x7f5ecc0b3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc15eab0 con 0x7f5ecc0addc0 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5ebc001670 con 0x7f5ecc0addc0 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 >> v1:172.21.15.165:6789/0 conn(0x7f5ecc0addc0 legacy=0x7f5ecc15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0004800 0x7f5eb0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 0x7f5eb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc06c5a0 con 0x7f5eb00039b0 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15ed90 con 0x7f5eb0004800 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb000bb20 con 0x7f5ecc0728a0 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5ecb400640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5ecb400640 1 -- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 msgr2=0x7f5eb00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5ecb400640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc0728a0 0x7f5eb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5ecb400640 1 -- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0004800 msgr2=0x7f5eb0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5ecb400640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5eb0004800 0x7f5eb0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.249+0000 7f5ecb400640 1 -- 172.21.15.165:0/3888052417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb0003520 con 0x7f5eb00039b0 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.250+0000 7f5ecb400640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007fd0 tx=0x7f5eb0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e766ac146e5adf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc007d50 con 0x7f5eb00039b0 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc005ad0 con 0x7f5eb00039b0 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5eca000640 1 -- 172.21.15.165:0/3888052417 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc007070 con 0x7f5eb00039b0 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ed0a00640 1 -- 172.21.15.165:0/3888052417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 msgr2=0x7f5eb0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/3888052417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5eb0003dd0 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc007fd0 tx=0x7f5eb0005830 comp rx=0 tx=0).stop 2024-04-25T10:16:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ecbe00640 1 -- 172.21.15.165:0/3888052417 reap_dead start 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ed0a00640 1 -- 172.21.15.165:0/3888052417 shutdown_connections 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ed0a00640 1 -- 172.21.15.165:0/3888052417 >> 172.21.15.165:0/3888052417 conn(0x7f5ecc0718d0 msgr2=0x7f5ecc071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ed0a00640 1 -- 172.21.15.165:0/3888052417 shutdown_connections 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.251+0000 7f5ed0a00640 1 -- 172.21.15.165:0/3888052417 wait complete. 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.253+0000 7f5ed0a00640 1 Processor -- start 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.253+0000 7f5ed0a00640 1 -- start start 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ed0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5ecc1401f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ed0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc1f3b20 0x7f5ecc1f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ed0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ecc155970 0x7f5ecc155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ed0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0b3040 con 0x7f5eb00039b0 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ed0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc15e570 con 0x7f5ecc155970 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ed0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc0ae1a0 con 0x7f5ecc1f3b20 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ecb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5ecc1401f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ecb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5ecc1401f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51856/0 (socket says 172.21.15.165:51856) 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.254+0000 7f5ecb400640 1 -- 172.21.15.165:0/410596350 learned_addr learned my addr 172.21.15.165:0/410596350 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecaa00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc1f3b20 0x7f5ecc1f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecbe00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ecc155970 0x7f5ecc155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecb400640 1 -- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc1f3b20 msgr2=0x7f5ecc1f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecb400640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc1f3b20 0x7f5ecc1f3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecb400640 1 -- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ecc155970 msgr2=0x7f5ecc155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecb400640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ecc155970 0x7f5ecc155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecb400640 1 -- 172.21.15.165:0/410596350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ecc1f44e0 con 0x7f5eb00039b0 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecbe00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ecc155970 0x7f5ecc155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ecb400640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5ecc1401f0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc001970 tx=0x7f5ebc001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc040020 con 0x7f5eb00039b0 2024-04-25T10:16:28.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ebc001430 con 0x7f5eb00039b0 2024-04-25T10:16:28.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ebc0047e0 con 0x7f5eb00039b0 2024-04-25T10:16:28.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.255+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc1566e0 con 0x7f5eb00039b0 2024-04-25T10:16:28.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.256+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc140a20 con 0x7f5eb00039b0 2024-04-25T10:16:28.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.257+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc074290 con 0x7f5eb00039b0 2024-04-25T10:16:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.258+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ebc043020 con 0x7f5eb00039b0 2024-04-25T10:16:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.258+0000 7f5ec3400640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5eac06b4d0 0x7f5eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.259+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f5ebc0a9ee0 con 0x7f5eb00039b0 2024-04-25T10:16:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.259+0000 7f5ecaa00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5eac06b4d0 0x7f5eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.259+0000 7f5ecaa00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5eac06b4d0 0x7f5eac06d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8006a90 tx=0x7f5eb8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.263+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ecc074290 con 0x7f5eb00039b0 2024-04-25T10:16:28.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.503+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ecc0af260 con 0x7f5eb00039b0 2024-04-25T10:16:28.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.504+0000 7f5ec3400640 1 -- 172.21.15.165:0/410596350 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 94 v94) v1 ==== 94+0+4161 (secure 0 0 0) 0x7f5ecc0af260 con 0x7f5eb00039b0 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":94,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":94,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:15:11.077981+0000","modified":"2024-04-25T10:16:16.251260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":137,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9225},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9225":{"gid":9225,"name":"c","rank":0,"incarnation":91,"state":"up:active","state_seq":21,"addr":"172.21.15.165:6837/2696658600","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2696658600},{"type":"v1","addr":"172.21.15.165:6837","nonce":2696658600}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2024-04-25T10:16:28.509 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 94 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5eac06b4d0 msgr2=0x7f5eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5eac06b4d0 0x7f5eac06d990 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f5eb8006a90 tx=0x7f5eb8008040 comp rx=0 tx=0).stop 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 msgr2=0x7f5ecc1401f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5ecc1401f0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc001970 tx=0x7f5ebc001a40 comp rx=0 tx=0).stop 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 shutdown_connections 2024-04-25T10:16:28.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5eac06b4d0 0x7f5eac06d990 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ecc155970 0x7f5ecc155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ecc1f3b20 0x7f5ecc1f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 --2- 172.21.15.165:0/410596350 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5eb00039b0 0x7f5ecc1401f0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 >> 172.21.15.165:0/410596350 conn(0x7f5ecc0718d0 msgr2=0x7f5ecc153290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 shutdown_connections 2024-04-25T10:16:28.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.507+0000 7f5ed0a00640 1 -- 172.21.15.165:0/410596350 wait complete. 2024-04-25T10:16:28.530 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:28.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.713+0000 7f8898a00640 1 Processor -- start 2024-04-25T10:16:28.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.713+0000 7f8898a00640 1 -- start start 2024-04-25T10:16:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8898a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8894144750 0x7f889406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8898a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f88940ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8898a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8894144b30 con 0x7f88940cee60 2024-04-25T10:16:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8898a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88940712d0 con 0x7f8894144750 2024-04-25T10:16:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8898a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88940715b0 con 0x7f889406eb10 2024-04-25T10:16:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8892800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f88940cee60 0x7f889415f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40474/0 (socket says 172.21.15.165:40474) 2024-04-25T10:16:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.714+0000 7f8891e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8894144750 0x7f889406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8892800640 1 -- 172.21.15.165:0/4124762113 learned_addr learned my addr 172.21.15.165:0/4124762113 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8891400640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f88940ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8890a00640 1 -- 172.21.15.165:0/4124762113 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487517070 0 0) 0x7f8894144b30 con 0x7f88940cee60 2024-04-25T10:16:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8890a00640 1 -- 172.21.15.165:0/4124762113 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8878003520 con 0x7f88940cee60 2024-04-25T10:16:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8891e00640 1 -- 172.21.15.165:0/4124762113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 msgr2=0x7f88940ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8891e00640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f88940ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8891e00640 1 -- 172.21.15.165:0/4124762113 >> v1:172.21.15.165:6789/0 conn(0x7f88940cee60 legacy=0x7f889415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8891e00640 1 -- 172.21.15.165:0/4124762113 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8894071890 con 0x7f8894144750 2024-04-25T10:16:28.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.715+0000 7f8891400640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f88940ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.716+0000 7f8891e00640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8894144750 0x7f889406e5d0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f88840097a0 tx=0x7f888402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f1dc3f99a2aef server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.716+0000 7f8890a00640 1 -- 172.21.15.165:0/4124762113 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884037070 con 0x7f8894144750 2024-04-25T10:16:28.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.716+0000 7f8890a00640 1 -- 172.21.15.165:0/4124762113 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8884032440 con 0x7f8894144750 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.716+0000 7f8890a00640 1 -- 172.21.15.165:0/4124762113 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884032810 con 0x7f8894144750 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 -- 172.21.15.165:0/4124762113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8894144750 msgr2=0x7f889406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8894144750 0x7f889406e5d0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f88840097a0 tx=0x7f888402a180 comp rx=0 tx=0).stop 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 -- 172.21.15.165:0/4124762113 shutdown_connections 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f88940ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 --2- 172.21.15.165:0/4124762113 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8894144750 0x7f889406e5d0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 -- 172.21.15.165:0/4124762113 >> 172.21.15.165:0/4124762113 conn(0x7f889406abe0 msgr2=0x7f889406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 -- 172.21.15.165:0/4124762113 shutdown_connections 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.717+0000 7f8898a00640 1 -- 172.21.15.165:0/4124762113 wait complete. 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.719+0000 7f8898a00640 1 Processor -- start 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.719+0000 7f8898a00640 1 -- start start 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8898a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f889413ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8898a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 0x7f88941f4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8898a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8894144750 0x7f8894140ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8898a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8894145450 con 0x7f88940cee60 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8898a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889406f020 con 0x7f8894144750 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8898a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88941455d0 con 0x7f889406eb10 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8891400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 0x7f88941f4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8891400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 0x7f88941f4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51868/0 (socket says 172.21.15.165:51868) 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.720+0000 7f8891400640 1 -- 172.21.15.165:0/1705795262 learned_addr learned my addr 172.21.15.165:0/1705795262 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891e00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f889413ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8892800640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8894144750 0x7f8894140ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891400640 1 -- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 msgr2=0x7f889413ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891400640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f889413ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891400640 1 -- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8894144750 msgr2=0x7f8894140ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891400640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8894144750 0x7f8894140ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891400640 1 -- 172.21.15.165:0/1705795262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f889413b350 con 0x7f88940cee60 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891e00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f889413ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:28.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8891400640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 0x7f88941f4280 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c7f0 tx=0x7f887c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c015070 con 0x7f88940cee60 2024-04-25T10:16:28.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c007a30 con 0x7f88940cee60 2024-04-25T10:16:28.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88941514e0 con 0x7f88940cee60 2024-04-25T10:16:28.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c007e00 con 0x7f88940cee60 2024-04-25T10:16:28.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.721+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8894151aa0 con 0x7f88940cee60 2024-04-25T10:16:28.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.723+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8894073990 con 0x7f88940cee60 2024-04-25T10:16:28.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.724+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f887c018070 con 0x7f88940cee60 2024-04-25T10:16:28.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.724+0000 7f888a000640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f887006b4d0 0x7f887006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:28.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.725+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f887c08a2d0 con 0x7f88940cee60 2024-04-25T10:16:28.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.725+0000 7f8891e00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f887006b4d0 0x7f887006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:28.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.725+0000 7f8891e00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f887006b4d0 0x7f887006d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8884034000 tx=0x7f888402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:28.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.729+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f887c058fa0 con 0x7f88940cee60 2024-04-25T10:16:28.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.924+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88940d0510 con 0x7f88940cee60 2024-04-25T10:16:28.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.927+0000 7f888a000640 1 -- 172.21.15.165:0/1705795262 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+17067 (secure 0 0 0) 0x7f887c0588a0 con 0x7f88940cee60 2024-04-25T10:16:28.932 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:28.932 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":137,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:14.197143+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:15:08.823693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_data","create_time":"2024-04-25T10:15:10.098970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6835/2186062364":"2024-04-26T10:16:14.196999+0000","172.21.15.172:6834/2186062364":"2024-04-26T10:16:14.196999+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.929+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f887006b4d0 msgr2=0x7f887006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.929+0000 7f8898a00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f887006b4d0 0x7f887006d990 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f8884034000 tx=0x7f888402e000 comp rx=0 tx=0).stop 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.929+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 msgr2=0x7f88941f4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.929+0000 7f8898a00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 0x7f88941f4280 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c7f0 tx=0x7f887c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 shutdown_connections 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f887006b4d0 0x7f887006d990 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8894144750 0x7f8894140ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88940cee60 0x7f88941f4280 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 --2- 172.21.15.165:0/1705795262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f889406eb10 0x7f889413ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:28.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 >> 172.21.15.165:0/1705795262 conn(0x7f889406abe0 msgr2=0x7f88940b19e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:28.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 shutdown_connections 2024-04-25T10:16:28.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:28.930+0000 7f8898a00640 1 -- 172.21.15.165:0/1705795262 wait complete. 2024-04-25T10:16:28.953 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:16:29.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.148+0000 7fe3ef000640 1 Processor -- start 2024-04-25T10:16:29.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.148+0000 7fe3ef000640 1 -- start start 2024-04-25T10:16:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3ef000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3e80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3ef000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80cfd80 con 0x7fe3e806ec50 2024-04-25T10:16:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3ef000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e80d0060 con 0x7fe3e80ac8b0 2024-04-25T10:16:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3ef000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e815e490 con 0x7fe3e815b710 2024-04-25T10:16:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3e7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe3e806ec50 0x7fe3e815e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40488/0 (socket says 172.21.15.165:40488) 2024-04-25T10:16:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3e7400640 1 -- 172.21.15.165:0/4199271536 learned_addr learned my addr 172.21.15.165:0/4199271536 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.150+0000 7fe3e7e00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3e80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 901644262 0 0) 0x7fe3e80cfd80 con 0x7fe3e806ec50 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3cc003520 con 0x7fe3e806ec50 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 782022965 0 0) 0x7fe3cc003520 con 0x7fe3e806ec50 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 msgr2=0x7fe3e80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3e80cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 >> v1:172.21.15.172:6789/0 conn(0x7fe3e80ac8b0 legacy=0x7fe3e8071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80d0340 con 0x7fe3e806ec50 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe3d4001670 con 0x7fe3e806ec50 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 >> v1:172.21.15.165:6789/0 conn(0x7fe3e806ec50 legacy=0x7fe3e815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3cc003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3cc004d40 0x7fe3cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e815e490 con 0x7fe3cc004030 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc00bb80 con 0x7fe3cc004d40 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.151+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc00bd80 con 0x7fe3e815b710 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7400640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7e00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3cc003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3ed000640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3cc004d40 0x7fe3cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7400640 1 -- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 msgr2=0x7fe3cc003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7400640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3cc003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7400640 1 -- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3cc004d40 msgr2=0x7fe3cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7400640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3cc004d40 0x7fe3cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7400640 1 -- 172.21.15.165:0/4199271536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc003520 con 0x7fe3cc004030 2024-04-25T10:16:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3ed000640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3cc004d40 0x7fe3cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.152+0000 7fe3e7e00640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e815b710 0x7fe3cc003a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.153+0000 7fe3e7400640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3cc004800 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c240 tx=0x7fe3cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dadbfe0fd6c0752 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.153+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4007cf0 con 0x7fe3cc004030 2024-04-25T10:16:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.153+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d40058d0 con 0x7fe3cc004030 2024-04-25T10:16:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.153+0000 7fe3e6a00640 1 -- 172.21.15.165:0/4199271536 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d4005d20 con 0x7fe3cc004030 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ef000640 1 -- 172.21.15.165:0/4199271536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 msgr2=0x7fe3cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ef000640 1 --2- 172.21.15.165:0/4199271536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3cc004800 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fe3d400c240 tx=0x7fe3cc005830 comp rx=0 tx=0).stop 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ed000640 1 -- 172.21.15.165:0/4199271536 reap_dead start 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ef000640 1 -- 172.21.15.165:0/4199271536 shutdown_connections 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ef000640 1 -- 172.21.15.165:0/4199271536 >> 172.21.15.165:0/4199271536 conn(0x7fe3e806abe0 msgr2=0x7fe3e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ef000640 1 -- 172.21.15.165:0/4199271536 shutdown_connections 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.154+0000 7fe3ef000640 1 -- 172.21.15.165:0/4199271536 wait complete. 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.156+0000 7fe3ef000640 1 Processor -- start 2024-04-25T10:16:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.156+0000 7fe3ef000640 1 -- start start 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3ef000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3e8155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3ef000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e814e0f0 0x7fe3e8159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3ef000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3e8159d60 0x7fe3e80c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3ef000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e815bc70 con 0x7fe3cc004030 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3ef000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e815baf0 con 0x7fe3e8159d60 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3ef000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e815f030 con 0x7fe3e814e0f0 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.157+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3e8155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3e8155650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51886/0 (socket says 172.21.15.165:51886) 2024-04-25T10:16:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 -- 172.21.15.165:0/1978456904 learned_addr learned my addr 172.21.15.165:0/1978456904 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3ed000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3e8159d60 0x7fe3e80c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 -- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e814e0f0 msgr2=0x7fe3e8159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7400640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e814e0f0 0x7fe3e8159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e814e0f0 0x7fe3e8159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 -- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3e8159d60 msgr2=0x7fe3e80c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3e8159d60 0x7fe3e80c4de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 -- 172.21.15.165:0/1978456904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e80b02f0 con 0x7fe3cc004030 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.158+0000 7fe3e7e00640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3e8155650 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e810 tx=0x7fe3d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.159+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8017070 con 0x7fe3cc004030 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.159+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d800c6d0 con 0x7fe3cc004030 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.159+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d800f040 con 0x7fe3cc004030 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.159+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e80b05d0 con 0x7fe3cc004030 2024-04-25T10:16:29.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.159+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e80b0b90 con 0x7fe3cc004030 2024-04-25T10:16:29.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.160+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e8073a00 con 0x7fe3cc004030 2024-04-25T10:16:29.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.161+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe3d801a070 con 0x7fe3cc004030 2024-04-25T10:16:29.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.162+0000 7fe3dfe00640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3c806b500 0x7fe3c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.162+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 6187+0+0 (secure 0 0 0) 0x7fe3d8089fa0 con 0x7fe3cc004030 2024-04-25T10:16:29.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.162+0000 7fe3e7400640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3c806b500 0x7fe3c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.163+0000 7fe3e7400640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3c806b500 0x7fe3c806d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4033000 tx=0x7fe3d4009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.166+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe3d8058c30 con 0x7fe3cc004030 2024-04-25T10:16:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.406+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe3e80b3650 con 0x7fe3cc004030 2024-04-25T10:16:29.412 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:16:29.407+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:16:29.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.411+0000 7fe3dfe00640 1 -- 172.21.15.165:0/1978456904 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v95) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe3d8058530 con 0x7fe3cc004030 2024-04-25T10:16:29.415 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:16:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.414+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3c806b500 msgr2=0x7fe3c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.414+0000 7fe3ef000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3c806b500 0x7fe3c806d9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe3d4033000 tx=0x7fe3d4009030 comp rx=0 tx=0).stop 2024-04-25T10:16:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.414+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 msgr2=0x7fe3e8155650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.414+0000 7fe3ef000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3e8155650 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fe3d800e810 tx=0x7fe3d800ecd0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 shutdown_connections 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe3c806b500 0x7fe3c806d9c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe3e8159d60 0x7fe3e80c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe3e814e0f0 0x7fe3e8159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 --2- 172.21.15.165:0/1978456904 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe3cc004030 0x7fe3e8155650 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 >> 172.21.15.165:0/1978456904 conn(0x7fe3e806abe0 msgr2=0x7fe3e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.415+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 shutdown_connections 2024-04-25T10:16:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.416+0000 7fe3ef000640 1 -- 172.21.15.165:0/1978456904 wait complete. 2024-04-25T10:16:29.438 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:16:29.495 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:16:29.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.622+0000 7f3490e00640 1 Processor -- start 2024-04-25T10:16:29.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.623+0000 7f3490e00640 1 -- start start 2024-04-25T10:16:29.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f3490e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f348c145140 0x7f348c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f3490e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f3490e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f348c0c5030 0x7f348c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f3490e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c0ced90 con 0x7f348c145140 2024-04-25T10:16:29.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f3490e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348c145520 con 0x7f348c0c5030 2024-04-25T10:16:29.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f3490e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c071270 con 0x7f348c0ce950 2024-04-25T10:16:29.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f348b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f348c145140 0x7f348c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f348b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f348c145140 0x7f348c0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51892/0 (socket says 172.21.15.165:51892) 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.624+0000 7f348b400640 1 -- 172.21.15.165:0/2662067821 learned_addr learned my addr 172.21.15.165:0/2662067821 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.625+0000 7f348b400640 1 -- 172.21.15.165:0/2662067821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 msgr2=0x7f348c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.625+0000 7f348b400640 1 --2- 172.21.15.165:0/2662067821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.625+0000 7f348b400640 1 -- 172.21.15.165:0/2662067821 >> v2:172.21.15.172:3300/0 conn(0x7f348c0c5030 msgr2=0x7f348c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.625+0000 7f348b400640 1 --2- 172.21.15.165:0/2662067821 >> v2:172.21.15.172:3300/0 conn(0x7f348c0c5030 0x7f348c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.625+0000 7f348b400640 1 -- 172.21.15.165:0/2662067821 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c071550 con 0x7f348c145140 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.626+0000 7f348b400640 1 --2- 172.21.15.165:0/2662067821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0ce410 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f348c0b2860 tx=0x7f347c02a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acf1090b55660a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.626+0000 7f348aa00640 1 -- 172.21.15.165:0/2662067821 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c037070 con 0x7f348c145140 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.626+0000 7f348aa00640 1 -- 172.21.15.165:0/2662067821 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347c032440 con 0x7f348c145140 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.626+0000 7f348aa00640 1 -- 172.21.15.165:0/2662067821 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347c032810 con 0x7f348c145140 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 -- 172.21.15.165:0/2662067821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 msgr2=0x7f348c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 --2- 172.21.15.165:0/2662067821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0ce410 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f348c0b2860 tx=0x7f347c02a140 comp rx=0 tx=0).stop 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 -- 172.21.15.165:0/2662067821 shutdown_connections 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 --2- 172.21.15.165:0/2662067821 >> v2:172.21.15.172:3300/0 conn(0x7f348c0c5030 0x7f348c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 --2- 172.21.15.165:0/2662067821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 --2- 172.21.15.165:0/2662067821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0ce410 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 -- 172.21.15.165:0/2662067821 >> 172.21.15.165:0/2662067821 conn(0x7f348c061cc0 msgr2=0x7f348c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:29.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 -- 172.21.15.165:0/2662067821 shutdown_connections 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.627+0000 7f3490e00640 1 -- 172.21.15.165:0/2662067821 wait complete. 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.629+0000 7f3490e00640 1 Processor -- start 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.629+0000 7f3490e00640 1 -- start start 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f3490e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f348c0c5030 0x7f348c0cdb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f3490e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f3490e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0b1090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f3490e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0c54a0 con 0x7f348c145140 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f3490e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f348c071b10 con 0x7f348c0c5030 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f3490e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f348c0c5620 con 0x7f348c0ce950 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f348be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0b1090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f348be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0b1090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51908/0 (socket says 172.21.15.165:51908) 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.630+0000 7f348be00640 1 -- 172.21.15.165:0/553310345 learned_addr learned my addr 172.21.15.165:0/553310345 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348b400640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f348c0c5030 0x7f348c0cdb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f3483e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348be00640 1 -- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 msgr2=0x7f348c0cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348be00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0cb100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348be00640 1 -- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f348c0c5030 msgr2=0x7f348c0cdb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348be00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f348c0c5030 0x7f348c0cdb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348be00640 1 -- 172.21.15.165:0/553310345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f348c0a87c0 con 0x7f348c145140 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348b400640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f348c0c5030 0x7f348c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f3483e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f348be00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0b1090 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f348400b9a0 tx=0x7f348400be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484019070 con 0x7f348c145140 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f348400cbc0 con 0x7f348c145140 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3484014440 con 0x7f348c145140 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.631+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f348c142750 con 0x7f348c145140 2024-04-25T10:16:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.632+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f348c142d10 con 0x7f348c145140 2024-04-25T10:16:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.633+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348c073990 con 0x7f348c145140 2024-04-25T10:16:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.634+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f348401c070 con 0x7f348c145140 2024-04-25T10:16:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.635+0000 7f3483400640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f346806b4d0 0x7f346806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.635+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f348408a500 con 0x7f348c145140 2024-04-25T10:16:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.635+0000 7f348b400640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f346806b4d0 0x7f346806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.636+0000 7f348b400640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f346806b4d0 0x7f346806d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f347c034000 tx=0x7f347c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.639+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3484023020 con 0x7f348c145140 2024-04-25T10:16:29.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:29.865+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f348c067f10 con 0x7f348c145140 2024-04-25T10:16:30.345 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:30.353 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:16:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.414+0000 7f3483400640 1 -- 172.21.15.165:0/553310345 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v96) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3484059220 con 0x7f348c145140 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f346806b4d0 msgr2=0x7f346806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f346806b4d0 0x7f346806d990 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f347c034000 tx=0x7f347c02e000 comp rx=0 tx=0).stop 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 msgr2=0x7f348c0b1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0b1090 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f348400b9a0 tx=0x7f348400be60 comp rx=0 tx=0).stop 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 shutdown_connections 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f346806b4d0 0x7f346806d990 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f348c145140 0x7f348c0b1090 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f348c0ce950 0x7f348c0cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 --2- 172.21.15.165:0/553310345 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f348c0c5030 0x7f348c0cdb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 >> 172.21.15.165:0/553310345 conn(0x7f348c061cc0 msgr2=0x7f348c13fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.417+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 shutdown_connections 2024-04-25T10:16:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.418+0000 7f3490e00640 1 -- 172.21.15.165:0/553310345 wait complete. 2024-04-25T10:16:30.443 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:16:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.628+0000 7f7146400640 1 Processor -- start 2024-04-25T10:16:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.628+0000 7f7146400640 1 -- start start 2024-04-25T10:16:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7146400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7146400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71400c5030 0x7f714015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7146400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7140145520 con 0x7f7140145140 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7146400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7140071270 con 0x7f71400c5030 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7146400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7140071550 con 0x7f71400ceb20 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7145a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71400c5030 0x7f714015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7145a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71400c5030 0x7f714015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51916/0 (socket says 172.21.15.165:51916) 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f7145a00640 1 -- 172.21.15.165:0/2475214938 learned_addr learned my addr 172.21.15.165:0/2475214938 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.630+0000 7f713fe00640 1 --2- 172.21.15.165:0/2475214938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.631+0000 7f7145a00640 1 -- 172.21.15.165:0/2475214938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 msgr2=0x7f71400c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.631+0000 7f7145a00640 1 --2- 172.21.15.165:0/2475214938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.631+0000 7f7145a00640 1 -- 172.21.15.165:0/2475214938 >> v1:172.21.15.172:6789/0 conn(0x7f7140145140 legacy=0x7f71400ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.631+0000 7f7145a00640 1 -- 172.21.15.165:0/2475214938 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7140071830 con 0x7f71400c5030 2024-04-25T10:16:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.632+0000 7f7145a00640 1 --2- 172.21.15.165:0/2475214938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f714015bfb0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7138009e50 tx=0x7f713802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a62b3a50276af98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.632+0000 7f713f400640 1 -- 172.21.15.165:0/2475214938 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7138036070 con 0x7f71400c5030 2024-04-25T10:16:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.632+0000 7f713f400640 1 -- 172.21.15.165:0/2475214938 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7138031600 con 0x7f71400c5030 2024-04-25T10:16:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.632+0000 7f713f400640 1 -- 172.21.15.165:0/2475214938 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71380319d0 con 0x7f71400c5030 2024-04-25T10:16:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 -- 172.21.15.165:0/2475214938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 msgr2=0x7f714015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 --2- 172.21.15.165:0/2475214938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f714015bfb0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f7138009e50 tx=0x7f713802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 -- 172.21.15.165:0/2475214938 shutdown_connections 2024-04-25T10:16:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 --2- 172.21.15.165:0/2475214938 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f714015bfb0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 --2- 172.21.15.165:0/2475214938 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 -- 172.21.15.165:0/2475214938 >> 172.21.15.165:0/2475214938 conn(0x7f7140061cc0 msgr2=0x7f714006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 -- 172.21.15.165:0/2475214938 shutdown_connections 2024-04-25T10:16:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.633+0000 7f7146400640 1 -- 172.21.15.165:0/2475214938 wait complete. 2024-04-25T10:16:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.635+0000 7f7146400640 1 Processor -- start 2024-04-25T10:16:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.635+0000 7f7146400640 1 -- start start 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.636+0000 7f7146400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f71400aaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.636+0000 7f7146400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400cdae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.636+0000 7f7146400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7140145140 0x7f71401e84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.636+0000 7f7146400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c5640 con 0x7f71400c5030 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.636+0000 7f7146400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71400c54c0 con 0x7f7140145140 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.636+0000 7f7146400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f714015c4f0 con 0x7f71400ceb20 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f71400aaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f71400aaaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51922/0 (socket says 172.21.15.165:51922) 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 -- 172.21.15.165:0/3092831600 learned_addr learned my addr 172.21.15.165:0/3092831600 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f713fe00640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400cdae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:30.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145a00640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7140145140 0x7f71401e84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:30.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 -- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 msgr2=0x7f71400cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:30.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400cdae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 -- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7140145140 msgr2=0x7f71401e84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:30.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7140145140 0x7f71401e84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:30.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.637+0000 7f7145000640 1 -- 172.21.15.165:0/3092831600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71400cc330 con 0x7f71400c5030 2024-04-25T10:16:30.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.638+0000 7f7145000640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f71400aaaf0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f713400c400 tx=0x7f713400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.638+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134017070 con 0x7f71400c5030 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.638+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71340052d0 con 0x7f71400c5030 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.638+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7134003830 con 0x7f71400c5030 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.638+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71400cc610 con 0x7f71400c5030 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.638+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71400ccbd0 con 0x7f71400c5030 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.639+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7140073990 con 0x7f71400c5030 2024-04-25T10:16:30.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.640+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f713401a070 con 0x7f71400c5030 2024-04-25T10:16:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.641+0000 7f713cc00640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f712006b4d0 0x7f712006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.641+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6273+0+0 (secure 0 0 0) 0x7f7134089710 con 0x7f71400c5030 2024-04-25T10:16:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.642+0000 7f713fe00640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f712006b4d0 0x7f712006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.642+0000 7f713fe00640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f712006b4d0 0x7f712006d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7140137750 tx=0x7f712c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.646+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71340583e0 con 0x7f71400c5030 2024-04-25T10:16:30.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:30.829+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7140067ef0 con 0x7f71400c5030 2024-04-25T10:16:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.427+0000 7f713cc00640 1 -- 172.21.15.165:0/3092831600 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v139) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7134087e20 con 0x7f71400c5030 2024-04-25T10:16:31.432 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:16:31.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.432+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f712006b4d0 msgr2=0x7f712006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.432+0000 7f7146400640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f712006b4d0 0x7f712006d990 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7140137750 tx=0x7f712c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.432+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 msgr2=0x7f71400aaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.432+0000 7f7146400640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f71400aaaf0 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f713400c400 tx=0x7f713400c8c0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.433+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 shutdown_connections 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.433+0000 7f7146400640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f712006b4d0 0x7f712006d990 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.433+0000 7f7146400640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7140145140 0x7f71401e84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.433+0000 7f7146400640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71400ceb20 0x7f71400cdae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.433+0000 7f7146400640 1 --2- 172.21.15.165:0/3092831600 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71400c5030 0x7f71400aaaf0 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.433+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 >> 172.21.15.165:0/3092831600 conn(0x7f7140061cc0 msgr2=0x7f71400b3460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:31.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.435+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 shutdown_connections 2024-04-25T10:16:31.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.435+0000 7f7146400640 1 -- 172.21.15.165:0/3092831600 wait complete. 2024-04-25T10:16:31.464 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:16:31.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.662+0000 7f5e4be00640 1 Processor -- start 2024-04-25T10:16:31.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.662+0000 7f5e4be00640 1 -- start start 2024-04-25T10:16:31.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5e4c0ceb20 0x7f5e4c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e4c145520 con 0x7f5e4c145140 2024-04-25T10:16:31.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e4c071270 con 0x7f5e4c0ceb20 2024-04-25T10:16:31.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c071550 con 0x7f5e4c0c5030 2024-04-25T10:16:31.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5e4c0ceb20 0x7f5e4c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5e4c145140 0x7f5e4c0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40490/0 (socket says 172.21.15.165:40490) 2024-04-25T10:16:31.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.664+0000 7f5e4aa00640 1 -- 172.21.15.165:0/4249106981 learned_addr learned my addr 172.21.15.165:0/4249106981 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:31.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e4b400640 1 --2- 172.21.15.165:0/4249106981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e49600640 1 -- 172.21.15.165:0/4249106981 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2114101189 0 0) 0x7f5e4c145520 con 0x7f5e4c145140 2024-04-25T10:16:31.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e49600640 1 -- 172.21.15.165:0/4249106981 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e30003520 con 0x7f5e4c145140 2024-04-25T10:16:31.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e4a000640 1 -- 172.21.15.165:0/4249106981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 msgr2=0x7f5e4c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:31.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e4a000640 1 --2- 172.21.15.165:0/4249106981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c15bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e4a000640 1 -- 172.21.15.165:0/4249106981 >> v1:172.21.15.165:6789/0 conn(0x7f5e4c145140 legacy=0x7f5e4c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.665+0000 7f5e4a000640 1 -- 172.21.15.165:0/4249106981 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c071830 con 0x7f5e4c0ceb20 2024-04-25T10:16:31.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.666+0000 7f5e4a000640 1 --2- 172.21.15.165:0/4249106981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c0c4af0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f5e380097a0 tx=0x7f5e3802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83c61ab47721a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:31.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.666+0000 7f5e49600640 1 -- 172.21.15.165:0/4249106981 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38037070 con 0x7f5e4c0ceb20 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.666+0000 7f5e49600640 1 -- 172.21.15.165:0/4249106981 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3802ae50 con 0x7f5e4c0ceb20 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.666+0000 7f5e49600640 1 -- 172.21.15.165:0/4249106981 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38032660 con 0x7f5e4c0ceb20 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 -- 172.21.15.165:0/4249106981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c0ceb20 msgr2=0x7f5e4c0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 --2- 172.21.15.165:0/4249106981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c0c4af0 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f5e380097a0 tx=0x7f5e3802a100 comp rx=0 tx=0).stop 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 -- 172.21.15.165:0/4249106981 shutdown_connections 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 --2- 172.21.15.165:0/4249106981 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c15bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 --2- 172.21.15.165:0/4249106981 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c0c4af0 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 -- 172.21.15.165:0/4249106981 >> 172.21.15.165:0/4249106981 conn(0x7f5e4c061cc0 msgr2=0x7f5e4c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 -- 172.21.15.165:0/4249106981 shutdown_connections 2024-04-25T10:16:31.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.667+0000 7f5e4be00640 1 -- 172.21.15.165:0/4249106981 wait complete. 2024-04-25T10:16:31.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.669+0000 7f5e4be00640 1 Processor -- start 2024-04-25T10:16:31.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.669+0000 7f5e4be00640 1 -- start start 2024-04-25T10:16:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c0a9c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c1f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 0x7f5e4c133ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c5620 con 0x7f5e4c145140 2024-04-25T10:16:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c0c54a0 con 0x7f5e4c0ceb20 2024-04-25T10:16:31.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e4c15c4f0 con 0x7f5e4c0c5030 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 0x7f5e4c133ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 0x7f5e4c133ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51936/0 (socket says 172.21.15.165:51936) 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.670+0000 7f5e4b400640 1 -- 172.21.15.165:0/1017895895 learned_addr learned my addr 172.21.15.165:0/1017895895 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4aa00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c0a9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4b400640 1 -- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 msgr2=0x7f5e4c0a9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4b400640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c0a9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4b400640 1 -- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c0ceb20 msgr2=0x7f5e4c1f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4a000640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c1f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4b400640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c1f07b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4b400640 1 -- 172.21.15.165:0/1017895895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c134680 con 0x7f5e4c145140 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4b400640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 0x7f5e4c133ee0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f5e44007e70 tx=0x7f5e4400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44014620 con 0x7f5e4c145140 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e44019070 con 0x7f5e4c145140 2024-04-25T10:16:31.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e4c134960 con 0x7f5e4c145140 2024-04-25T10:16:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.671+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e44014a20 con 0x7f5e4c145140 2024-04-25T10:16:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.672+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e4c134f20 con 0x7f5e4c145140 2024-04-25T10:16:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.673+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e4c073990 con 0x7f5e4c145140 2024-04-25T10:16:31.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.674+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e44014bc0 con 0x7f5e4c145140 2024-04-25T10:16:31.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.675+0000 7f5e42a00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2c06b500 0x7f5e2c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:31.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.675+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f5e4408acb0 con 0x7f5e4c145140 2024-04-25T10:16:31.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.675+0000 7f5e4aa00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2c06b500 0x7f5e2c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:31.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.676+0000 7f5e4aa00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2c06b500 0x7f5e2c06d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c005880 tx=0x7f5e3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:31.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.679+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e44059980 con 0x7f5e4c145140 2024-04-25T10:16:31.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:31.873+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5e4c06c220 con 0x7f5e4c145140 2024-04-25T10:16:32.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.431+0000 7f5e42a00640 1 -- 172.21.15.165:0/1017895895 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v140) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5e44059280 con 0x7f5e4c145140 2024-04-25T10:16:32.436 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:16:32.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2c06b500 msgr2=0x7f5e2c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2c06b500 0x7f5e2c06d9c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f5e3c005880 tx=0x7f5e3c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:32.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 msgr2=0x7f5e4c133ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 0x7f5e4c133ee0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f5e44007e70 tx=0x7f5e4400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 shutdown_connections 2024-04-25T10:16:32.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2c06b500 0x7f5e2c06d9c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4c145140 0x7f5e4c133ee0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4c0ceb20 0x7f5e4c1f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 --2- 172.21.15.165:0/1017895895 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e4c0c5030 0x7f5e4c0a9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.435+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 >> 172.21.15.165:0/1017895895 conn(0x7f5e4c061cc0 msgr2=0x7f5e4c0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:32.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.436+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 shutdown_connections 2024-04-25T10:16:32.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.437+0000 7f5e4be00640 1 -- 172.21.15.165:0/1017895895 wait complete. 2024-04-25T10:16:32.465 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:16:32.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.665+0000 7f7a67200640 1 Processor -- start 2024-04-25T10:16:32.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.665+0000 7f7a67200640 1 -- start start 2024-04-25T10:16:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.666+0000 7f7a67200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 0x7f7a680cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a67200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6815e110 con 0x7f7a6806bff0 2024-04-25T10:16:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a67200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a680b2fd0 con 0x7f7a680b5b40 2024-04-25T10:16:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a67200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806d000 con 0x7f7a6815dd30 2024-04-25T10:16:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a5fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7a6806bff0 0x7f7a680b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40498/0 (socket says 172.21.15.165:40498) 2024-04-25T10:16:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a5fe00640 1 -- 172.21.15.165:0/1809649335 learned_addr learned my addr 172.21.15.165:0/1809649335 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a65e00640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 0x7f7a680cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219114579 0 0) 0x7f7a6815e110 con 0x7f7a6806bff0 2024-04-25T10:16:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.667+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a4c003520 con 0x7f7a6806bff0 2024-04-25T10:16:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2104107891 0 0) 0x7f7a4c003520 con 0x7f7a6806bff0 2024-04-25T10:16:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 msgr2=0x7f7a680cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 0x7f7a680cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 >> v1:172.21.15.172:6789/0 conn(0x7f7a680b5b40 legacy=0x7f7a680d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6806c480 con 0x7f7a6806bff0 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f7a50001670 con 0x7f7a6806bff0 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 >> v1:172.21.15.165:6789/0 conn(0x7f7a6806bff0 legacy=0x7f7a680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a4c0039e0 0x7f7a4c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 0x7f7a4c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a4c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806d000 con 0x7f7a4c004d40 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c00ba70 con 0x7f7a4c0039e0 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c00bc70 con 0x7f7a6815dd30 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.668+0000 7f7a66800640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a4c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a5fe00640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 0x7f7a4c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a65e00640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a4c0039e0 0x7f7a4c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a66800640 1 -- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 msgr2=0x7f7a4c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a66800640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a6815dd30 0x7f7a4c004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a66800640 1 -- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a4c0039e0 msgr2=0x7f7a4c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a66800640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a4c0039e0 0x7f7a4c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a66800640 1 -- 172.21.15.165:0/1809649335 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c003520 con 0x7f7a4c004d40 2024-04-25T10:16:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.669+0000 7f7a65e00640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a4c0039e0 0x7f7a4c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:32.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.670+0000 7f7a66800640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a4c0051c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f7a60001e20 tx=0x7f7a4c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a88e588d912a72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.670+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60001a70 con 0x7f7a4c004d40 2024-04-25T10:16:32.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.670+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6002ae40 con 0x7f7a4c004d40 2024-04-25T10:16:32.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.670+0000 7f7a65400640 1 -- 172.21.15.165:0/1809649335 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60003bb0 con 0x7f7a4c004d40 2024-04-25T10:16:32.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a67200640 1 -- 172.21.15.165:0/1809649335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 msgr2=0x7f7a4c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a67200640 1 --2- 172.21.15.165:0/1809649335 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a4c0051c0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f7a60001e20 tx=0x7f7a4c005830 comp rx=0 tx=0).stop 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a66800640 1 -- 172.21.15.165:0/1809649335 reap_dead start 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a67200640 1 -- 172.21.15.165:0/1809649335 shutdown_connections 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a67200640 1 -- 172.21.15.165:0/1809649335 >> 172.21.15.165:0/1809649335 conn(0x7f7a680d0610 msgr2=0x7f7a680d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a67200640 1 -- 172.21.15.165:0/1809649335 shutdown_connections 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.671+0000 7f7a67200640 1 -- 172.21.15.165:0/1809649335 wait complete. 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.673+0000 7f7a67200640 1 Processor -- start 2024-04-25T10:16:32.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.673+0000 7f7a67200640 1 -- start start 2024-04-25T10:16:32.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a67200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a6814ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a67200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a681f43c0 0x7f7a680caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a67200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a680ac420 0x7f7a680ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a67200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806cd20 con 0x7f7a4c004d40 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a67200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6806ca40 con 0x7f7a681f43c0 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a67200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a680cebd0 con 0x7f7a680ac420 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a65e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a6814ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a65e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a6814ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51948/0 (socket says 172.21.15.165:51948) 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a65e00640 1 -- 172.21.15.165:0/3923849228 learned_addr learned my addr 172.21.15.165:0/3923849228 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.674+0000 7f7a66800640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a680ac420 0x7f7a680ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a5fe00640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a681f43c0 0x7f7a680caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a65e00640 1 -- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a680ac420 msgr2=0x7f7a680ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a65e00640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a680ac420 0x7f7a680ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a65e00640 1 -- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a681f43c0 msgr2=0x7f7a680caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a65e00640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a681f43c0 0x7f7a680caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a65e00640 1 -- 172.21.15.165:0/3923849228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a680cb3f0 con 0x7f7a4c004d40 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a65e00640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a6814ee70 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7a58011630 tx=0x7f7a58011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800f920 con 0x7f7a4c004d40 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a5800fac0 con 0x7f7a4c004d40 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a5800fe90 con 0x7f7a4c004d40 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.675+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a680cb6d0 con 0x7f7a4c004d40 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.676+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a680cbc90 con 0x7f7a4c004d40 2024-04-25T10:16:32.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.677+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a68073990 con 0x7f7a4c004d40 2024-04-25T10:16:32.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.678+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7a58015460 con 0x7f7a4c004d40 2024-04-25T10:16:32.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.678+0000 7f7a5e000640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a4406b670 0x7f7a4406db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:32.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.679+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7a5808aa10 con 0x7f7a4c004d40 2024-04-25T10:16:32.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.679+0000 7f7a5fe00640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a4406b670 0x7f7a4406db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:32.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.679+0000 7f7a5fe00640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a4406b670 0x7f7a4406db30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7a680ac280 tx=0x7f7a50009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:32.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.683+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7a58019030 con 0x7f7a4c004d40 2024-04-25T10:16:32.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.880+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7a680ca310 con 0x7f7a4c004d40 2024-04-25T10:16:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.886+0000 7f7a5e000640 1 -- 172.21.15.165:0/3923849228 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a58059a40 con 0x7f7a4c004d40 2024-04-25T10:16:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a4406b670 msgr2=0x7f7a4406db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a4406b670 0x7f7a4406db30 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f7a680ac280 tx=0x7f7a50009040 comp rx=0 tx=0).stop 2024-04-25T10:16:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 msgr2=0x7f7a6814ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a6814ee70 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f7a58011630 tx=0x7f7a58011af0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 shutdown_connections 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7a4406b670 0x7f7a4406db30 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7a680ac420 0x7f7a680ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7a681f43c0 0x7f7a680caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 --2- 172.21.15.165:0/3923849228 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7a4c004d40 0x7f7a6814ee70 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 >> 172.21.15.165:0/3923849228 conn(0x7f7a680d0610 msgr2=0x7f7a680b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 shutdown_connections 2024-04-25T10:16:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:32.889+0000 7f7a67200640 1 -- 172.21.15.165:0/3923849228 wait complete. 2024-04-25T10:16:32.912 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover' 2024-04-25T10:16:33.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.131+0000 7f7f4e800640 1 Processor -- start 2024-04-25T10:16:33.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.131+0000 7f7f4e800640 1 -- start start 2024-04-25T10:16:33.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f4e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f480cc8e0 0x7f7f481625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f4e800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7f480d1810 0x7f7f480d1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f4e800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f480cce50 con 0x7f7f4815dc90 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f4e800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f48161af0 con 0x7f7f480d1810 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f4e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f48161dd0 con 0x7f7f480cc8e0 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f47400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7f4815dc90 0x7f7f4815ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40506/0 (socket says 172.21.15.165:40506) 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f47400640 1 -- 172.21.15.165:0/286584302 learned_addr learned my addr 172.21.15.165:0/286584302 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f47e00640 1 --2- 172.21.15.165:0/286584302 >> v2:172.21.15.165:3300/0 conn(0x7f7f480d1810 0x7f7f480d1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.133+0000 7f7f46a00640 1 --2- 172.21.15.165:0/286584302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f480cc8e0 0x7f7f481625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.134+0000 7f7f46a00640 1 -- 172.21.15.165:0/286584302 >> v2:172.21.15.165:3300/0 conn(0x7f7f480d1810 msgr2=0x7f7f480d1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.134+0000 7f7f46a00640 1 --2- 172.21.15.165:0/286584302 >> v2:172.21.15.165:3300/0 conn(0x7f7f480d1810 0x7f7f480d1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.134+0000 7f7f46a00640 1 -- 172.21.15.165:0/286584302 >> v1:172.21.15.165:6789/0 conn(0x7f7f4815dc90 legacy=0x7f7f4815ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.134+0000 7f7f46a00640 1 -- 172.21.15.165:0/286584302 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f481620b0 con 0x7f7f480cc8e0 2024-04-25T10:16:33.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.136+0000 7f7f46a00640 1 --2- 172.21.15.165:0/286584302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f480cc8e0 0x7f7f481625d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097a0 tx=0x7f7f3802a110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e972f88c506f3e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.138+0000 7f7f46000640 1 -- 172.21.15.165:0/286584302 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38037070 con 0x7f7f480cc8e0 2024-04-25T10:16:33.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.138+0000 7f7f46000640 1 -- 172.21.15.165:0/286584302 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f38032440 con 0x7f7f480cc8e0 2024-04-25T10:16:33.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.138+0000 7f7f46000640 1 -- 172.21.15.165:0/286584302 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f3802ae60 con 0x7f7f480cc8e0 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 -- 172.21.15.165:0/286584302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f480cc8e0 msgr2=0x7f7f481625d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 --2- 172.21.15.165:0/286584302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f480cc8e0 0x7f7f481625d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f7f380097a0 tx=0x7f7f3802a110 comp rx=0 tx=0).stop 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 -- 172.21.15.165:0/286584302 shutdown_connections 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 --2- 172.21.15.165:0/286584302 >> v2:172.21.15.165:3300/0 conn(0x7f7f480d1810 0x7f7f480d1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 --2- 172.21.15.165:0/286584302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f480cc8e0 0x7f7f481625d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 -- 172.21.15.165:0/286584302 >> 172.21.15.165:0/286584302 conn(0x7f7f4806f590 msgr2=0x7f7f4806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 -- 172.21.15.165:0/286584302 shutdown_connections 2024-04-25T10:16:33.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.139+0000 7f7f4e800640 1 -- 172.21.15.165:0/286584302 wait complete. 2024-04-25T10:16:33.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.141+0000 7f7f4e800640 1 Processor -- start 2024-04-25T10:16:33.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.142+0000 7f7f4e800640 1 -- start start 2024-04-25T10:16:33.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.143+0000 7f7f4e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f480cc8e0 0x7f7f480ceac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.143+0000 7f7f4e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 0x7f7f480cb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.143+0000 7f7f4e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f4815dc90 0x7f7f4806d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.143+0000 7f7f4e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4815e1f0 con 0x7f7f480d1810 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.143+0000 7f7f4e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f4815e070 con 0x7f7f480cc8e0 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.143+0000 7f7f4e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f480cfde0 con 0x7f7f4815dc90 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 0x7f7f480cb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f47400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f480cc8e0 0x7f7f480ceac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 0x7f7f480cb2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51968/0 (socket says 172.21.15.165:51968) 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 -- 172.21.15.165:0/1710900939 learned_addr learned my addr 172.21.15.165:0/1710900939 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f47e00640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f4815dc90 0x7f7f4806d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 -- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f4815dc90 msgr2=0x7f7f4806d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f4815dc90 0x7f7f4806d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 -- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f480cc8e0 msgr2=0x7f7f480ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f480cc8e0 0x7f7f480ceac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.144+0000 7f7f46a00640 1 -- 172.21.15.165:0/1710900939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f480cb830 con 0x7f7f480d1810 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.145+0000 7f7f46a00640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 0x7f7f480cb2f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009890 tx=0x7f7f380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.146+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38040020 con 0x7f7f480d1810 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.146+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f38038040 con 0x7f7f480d1810 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.146+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f38037070 con 0x7f7f480d1810 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.146+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f481f41d0 con 0x7f7f480d1810 2024-04-25T10:16:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.146+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f481f4790 con 0x7f7f480d1810 2024-04-25T10:16:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.148+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f48073990 con 0x7f7f480d1810 2024-04-25T10:16:33.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.149+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7f38043020 con 0x7f7f480d1810 2024-04-25T10:16:33.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.150+0000 7f7f3f400640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f2406b520 0x7f7f2406d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.151+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7f380a9ff0 con 0x7f7f480d1810 2024-04-25T10:16:33.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.151+0000 7f7f47400640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f2406b520 0x7f7f2406d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.151+0000 7f7f47400640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f2406b520 0x7f7f2406d9e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f30007fd0 tx=0x7f7f30007e80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.155+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7f3804a070 con 0x7f7f480d1810 2024-04-25T10:16:33.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.423+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]} v 0) v1 -- 0x7f7f480cf6b0 con 0x7f7f480d1810 2024-04-25T10:16:33.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.434+0000 7f7f3f400640 1 -- 172.21.15.165:0/1710900939 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_task_status_on_mds_failover"]}]=0 v667) v1 ==== 166+0+0 (secure 0 0 0) 0x7f7f38078940 con 0x7f7f480d1810 2024-04-25T10:16:33.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f2406b520 msgr2=0x7f7f2406d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f2406b520 0x7f7f2406d9e0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f7f30007fd0 tx=0x7f7f30007e80 comp rx=0 tx=0).stop 2024-04-25T10:16:33.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 msgr2=0x7f7f480cb2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 0x7f7f480cb2f0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f7f38009890 tx=0x7f7f380046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 shutdown_connections 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f2406b520 0x7f7f2406d9e0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7f4815dc90 0x7f7f4806d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7f480d1810 0x7f7f480cb2f0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 --2- 172.21.15.165:0/1710900939 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7f480cc8e0 0x7f7f480ceac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.437+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 >> 172.21.15.165:0/1710900939 conn(0x7f7f4806f590 msgr2=0x7f7f4806f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.438+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 shutdown_connections 2024-04-25T10:16:33.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.438+0000 7f7f4e800640 1 -- 172.21.15.165:0/1710900939 wait complete. 2024-04-25T10:16:33.461 INFO:tasks.cephfs_test_runner:test_scrub_task_status_on_mds_failover (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:16:33.461 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) 2024-04-25T10:16:33.462 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:33.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.661+0000 7f7d84400640 1 Processor -- start 2024-04-25T10:16:33.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.661+0000 7f7d84400640 1 -- start start 2024-04-25T10:16:33.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d84400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7d7c144750 0x7f7d7c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d84400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c06eb10 0x7f7d7c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d84400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d7c144b30 con 0x7f7d7c0cee60 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d84400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d7c0712d0 con 0x7f7d7c144750 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d84400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c0715b0 con 0x7f7d7c06eb10 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d81a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7d7c144750 0x7f7d7c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d82400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7d7c0cee60 0x7f7d7c15f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40508/0 (socket says 172.21.15.165:40508) 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d82400640 1 -- 172.21.15.165:0/2886056465 learned_addr learned my addr 172.21.15.165:0/2886056465 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.663+0000 7f7d81000640 1 --2- 172.21.15.165:0/2886056465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c06eb10 0x7f7d7c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.664+0000 7f7d81a00640 1 -- 172.21.15.165:0/2886056465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c06eb10 msgr2=0x7f7d7c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.664+0000 7f7d81a00640 1 --2- 172.21.15.165:0/2886056465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c06eb10 0x7f7d7c0ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.664+0000 7f7d81a00640 1 -- 172.21.15.165:0/2886056465 >> v1:172.21.15.165:6789/0 conn(0x7f7d7c0cee60 legacy=0x7f7d7c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.664+0000 7f7d81a00640 1 -- 172.21.15.165:0/2886056465 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c071890 con 0x7f7d7c144750 2024-04-25T10:16:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.664+0000 7f7d7be00640 1 -- 172.21.15.165:0/2886056465 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 174774926 0 0) 0x7f7d7c144b30 con 0x7f7d7c0cee60 2024-04-25T10:16:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.665+0000 7f7d81a00640 1 --2- 172.21.15.165:0/2886056465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c144750 0x7f7d7c06e5d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7d700097a0 tx=0x7f7d7002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ddb86d3f78c323 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.665+0000 7f7d7be00640 1 -- 172.21.15.165:0/2886056465 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70037070 con 0x7f7d7c144750 2024-04-25T10:16:33.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d7be00640 1 -- 172.21.15.165:0/2886056465 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d70032440 con 0x7f7d7c144750 2024-04-25T10:16:33.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d7be00640 1 -- 172.21.15.165:0/2886056465 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70032810 con 0x7f7d7c144750 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 -- 172.21.15.165:0/2886056465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c144750 msgr2=0x7f7d7c06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 --2- 172.21.15.165:0/2886056465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c144750 0x7f7d7c06e5d0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f7d700097a0 tx=0x7f7d7002a180 comp rx=0 tx=0).stop 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 -- 172.21.15.165:0/2886056465 shutdown_connections 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 --2- 172.21.15.165:0/2886056465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c06eb10 0x7f7d7c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 --2- 172.21.15.165:0/2886056465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c144750 0x7f7d7c06e5d0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 -- 172.21.15.165:0/2886056465 >> 172.21.15.165:0/2886056465 conn(0x7f7d7c06abe0 msgr2=0x7f7d7c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 -- 172.21.15.165:0/2886056465 shutdown_connections 2024-04-25T10:16:33.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.666+0000 7f7d84400640 1 -- 172.21.15.165:0/2886056465 wait complete. 2024-04-25T10:16:33.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.668+0000 7f7d84400640 1 Processor -- start 2024-04-25T10:16:33.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.668+0000 7f7d84400640 1 -- start start 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.669+0000 7f7d84400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 0x7f7d7c0aff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.669+0000 7f7d84400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d7c0cee60 0x7f7d7c1e53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.669+0000 7f7d84400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c144750 0x7f7d7c1f2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.669+0000 7f7d84400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c145450 con 0x7f7d7c06eb10 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.669+0000 7f7d84400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c06f020 con 0x7f7d7c0cee60 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.669+0000 7f7d84400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c1455d0 con 0x7f7d7c144750 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 0x7f7d7c0aff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 0x7f7d7c0aff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51978/0 (socket says 172.21.15.165:51978) 2024-04-25T10:16:33.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 -- 172.21.15.165:0/2738641098 learned_addr learned my addr 172.21.15.165:0/2738641098 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:33.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d82400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c144750 0x7f7d7c1f2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 -- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c144750 msgr2=0x7f7d7c1f2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c144750 0x7f7d7c1f2d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 -- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d7c0cee60 msgr2=0x7f7d7c1e53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d7c0cee60 0x7f7d7c1e53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 -- 172.21.15.165:0/2738641098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d7c1e5ea0 con 0x7f7d7c06eb10 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.670+0000 7f7d81a00640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 0x7f7d7c0aff40 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f7d700097a0 tx=0x7f7d700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.671+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70037050 con 0x7f7d7c06eb10 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.671+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d70038040 con 0x7f7d7c06eb10 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.671+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70004470 con 0x7f7d7c06eb10 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.671+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c1e6180 con 0x7f7d7c06eb10 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.671+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c1e6740 con 0x7f7d7c06eb10 2024-04-25T10:16:33.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.672+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d7c0af240 con 0x7f7d7c06eb10 2024-04-25T10:16:33.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.673+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7d70030900 con 0x7f7d7c06eb10 2024-04-25T10:16:33.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.674+0000 7f7d79600640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d5c06b4d0 0x7f7d5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:33.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.674+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f7d700aac60 con 0x7f7d7c06eb10 2024-04-25T10:16:33.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.674+0000 7f7d81000640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d5c06b4d0 0x7f7d5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:33.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.675+0000 7f7d81000640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d5c06b4d0 0x7f7d5c06d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c1f6dc0 tx=0x7f7d68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:33.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.679+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7d70079b60 con 0x7f7d7c06eb10 2024-04-25T10:16:33.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.876+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7d7c073990 con 0x7f7d7c06eb10 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.877+0000 7f7d79600640 1 -- 172.21.15.165:0/2738641098 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+13473 (secure 0 0 0) 0x7f7d70079460 con 0x7f7d7c06eb10 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:33.882 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":140,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:32.428537+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/2696658600":"2024-04-26T10:16:29.408973+0000","172.21.15.172:6835/2186062364":"2024-04-26T10:16:14.196999+0000","172.21.15.165:6835/2696658600":"2024-04-26T10:16:29.408973+0000","172.21.15.172:6834/2186062364":"2024-04-26T10:16:14.196999+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.880+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d5c06b4d0 msgr2=0x7f7d5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.880+0000 7f7d84400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d5c06b4d0 0x7f7d5c06d990 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f7d7c1f6dc0 tx=0x7f7d68008040 comp rx=0 tx=0).stop 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.880+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 msgr2=0x7f7d7c0aff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.880+0000 7f7d84400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 0x7f7d7c0aff40 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f7d700097a0 tx=0x7f7d700046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 shutdown_connections 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7d5c06b4d0 0x7f7d5c06d990 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7d7c144750 0x7f7d7c1f2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7d7c0cee60 0x7f7d7c1e53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 --2- 172.21.15.165:0/2738641098 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7d7c06eb10 0x7f7d7c0aff40 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:33.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 >> 172.21.15.165:0/2738641098 conn(0x7f7d7c06abe0 msgr2=0x7f7d7c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:33.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 shutdown_connections 2024-04-25T10:16:33.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:33.881+0000 7f7d84400640 1 -- 172.21.15.165:0/2738641098 wait complete. 2024-04-25T10:16:33.904 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:33.905 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:34.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.104+0000 7febb1a00640 1 Processor -- start 2024-04-25T10:16:34.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.104+0000 7febb1a00640 1 -- start start 2024-04-25T10:16:34.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.106+0000 7febb1a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7febac1452a0 0x7febac06dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.106+0000 7febb1a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0cccc0 con 0x7febac144d50 2024-04-25T10:16:34.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.106+0000 7febb1a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febac0cc140 con 0x7febac1446d0 2024-04-25T10:16:34.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.106+0000 7febb1a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febac0b2b20 con 0x7febac1452a0 2024-04-25T10:16:34.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.106+0000 7febaa000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7febac144d50 0x7febac06d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40514/0 (socket says 172.21.15.165:40514) 2024-04-25T10:16:34.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.106+0000 7febaa000640 1 -- 172.21.15.165:0/879684555 learned_addr learned my addr 172.21.15.165:0/879684555 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2555134045 0 0) 0x7febac0cccc0 con 0x7febac144d50 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb90003520 con 0x7febac144d50 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7febab400640 1 --2- 172.21.15.165:0/879684555 >> v2:172.21.15.172:3300/0 conn(0x7febac1452a0 0x7febac06dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 810531533 0 0) 0x7feb90003520 con 0x7febac144d50 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 >> v2:172.21.15.172:3300/0 conn(0x7febac1452a0 msgr2=0x7febac06dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 --2- 172.21.15.165:0/879684555 >> v2:172.21.15.172:3300/0 conn(0x7febac1452a0 0x7febac06dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 >> v1:172.21.15.172:6789/0 conn(0x7febac1446d0 legacy=0x7febac0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0cc420 con 0x7febac144d50 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7feb94001670 con 0x7febac144d50 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 >> v1:172.21.15.165:6789/0 conn(0x7febac144d50 legacy=0x7febac06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb900039b0 0x7feb90003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb90004800 0x7feb90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900051c0 0x7feb9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0b2b20 con 0x7feb900039b0 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb9000c110 con 0x7feb90004800 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.107+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9000c310 con 0x7feb900051c0 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaaa00640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb900039b0 0x7feb90003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaa000640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb90004800 0x7feb90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaaa00640 1 -- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900051c0 msgr2=0x7feb9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaaa00640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900051c0 0x7feb9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaaa00640 1 -- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb90004800 msgr2=0x7feb90004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaaa00640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb90004800 0x7feb90004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febaaa00640 1 -- 172.21.15.165:0/879684555 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90003520 con 0x7feb900039b0 2024-04-25T10:16:34.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.108+0000 7febab400640 1 -- 172.21.15.165:0/879684555 reap_dead start 2024-04-25T10:16:34.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.109+0000 7febaaa00640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb900039b0 0x7feb90003dd0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7feb9c001750 tx=0x7feb90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afdb50ff0fdaa849 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.109+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c001370 con 0x7feb900039b0 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.109+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9c0052d0 con 0x7feb900039b0 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.109+0000 7feba9600640 1 -- 172.21.15.165:0/879684555 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9c02abc0 con 0x7feb900039b0 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 -- 172.21.15.165:0/879684555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb900039b0 msgr2=0x7feb90003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb900039b0 0x7feb90003dd0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7feb9c001750 tx=0x7feb90005830 comp rx=0 tx=0).stop 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 -- 172.21.15.165:0/879684555 shutdown_connections 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 --2- 172.21.15.165:0/879684555 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb900039b0 0x7feb90003dd0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 -- 172.21.15.165:0/879684555 >> 172.21.15.165:0/879684555 conn(0x7febac06abe0 msgr2=0x7febac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 -- 172.21.15.165:0/879684555 shutdown_connections 2024-04-25T10:16:34.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.110+0000 7febb1a00640 1 -- 172.21.15.165:0/879684555 wait complete. 2024-04-25T10:16:34.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.112+0000 7febb1a00640 1 Processor -- start 2024-04-25T10:16:34.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.112+0000 7febb1a00640 1 -- start start 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febb1a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900039b0 0x7febac1555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febb1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febac0ac490 0x7febac0ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febb1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 0x7febac153010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febb1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac0cc9e0 con 0x7febac152b90 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febb1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac06d400 con 0x7febac0ac490 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febb1a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febac144ab0 con 0x7feb900039b0 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 0x7febac153010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 0x7febac153010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51994/0 (socket says 172.21.15.165:51994) 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.113+0000 7febab400640 1 -- 172.21.15.165:0/738634442 learned_addr learned my addr 172.21.15.165:0/738634442 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febaa000640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febac0ac490 0x7febac0ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febaaa00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900039b0 0x7febac1555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febab400640 1 -- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900039b0 msgr2=0x7febac1555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febab400640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900039b0 0x7febac1555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febab400640 1 -- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febac0ac490 msgr2=0x7febac0ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febab400640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febac0ac490 0x7febac0ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febab400640 1 -- 172.21.15.165:0/738634442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febac0adfd0 con 0x7febac152b90 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7febab400640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 0x7febac153010 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7febac06f710 tx=0x7feba400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.114+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba400ed90 con 0x7febac152b90 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.115+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba4019070 con 0x7febac152b90 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.115+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4014660 con 0x7febac152b90 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.115+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac0acf30 con 0x7febac152b90 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.115+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac0ad4f0 con 0x7febac152b90 2024-04-25T10:16:34.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.116+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac073a00 con 0x7febac152b90 2024-04-25T10:16:34.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.117+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feba4005d10 con 0x7febac152b90 2024-04-25T10:16:34.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.118+0000 7feba2a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb8806b500 0x7feb8806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.118+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7feba408a370 con 0x7febac152b90 2024-04-25T10:16:34.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.118+0000 7febaaa00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb8806b500 0x7feb8806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.119+0000 7febaaa00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb8806b500 0x7feb8806d9c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7feb9c032000 tx=0x7feb9c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.122+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feba4023020 con 0x7febac152b90 2024-04-25T10:16:34.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.318+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7febac143050 con 0x7febac152b90 2024-04-25T10:16:34.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.319+0000 7feba2a00640 1 -- 172.21.15.165:0/738634442 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7feba4059270 con 0x7febac152b90 2024-04-25T10:16:34.323 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:16:34.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb8806b500 msgr2=0x7feb8806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb8806b500 0x7feb8806d9c0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7feb9c032000 tx=0x7feb9c02e000 comp rx=0 tx=0).stop 2024-04-25T10:16:34.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 msgr2=0x7febac153010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 0x7febac153010 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7febac06f710 tx=0x7feba400e4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 shutdown_connections 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb8806b500 0x7feb8806d9c0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7febac152b90 0x7febac153010 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7febac0ac490 0x7febac0ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 --2- 172.21.15.165:0/738634442 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb900039b0 0x7febac1555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 >> 172.21.15.165:0/738634442 conn(0x7febac06abe0 msgr2=0x7febac0af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 shutdown_connections 2024-04-25T10:16:34.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.322+0000 7febb1a00640 1 -- 172.21.15.165:0/738634442 wait complete. 2024-04-25T10:16:34.346 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:16:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.537+0000 7fa38ea00640 1 Processor -- start 2024-04-25T10:16:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.538+0000 7fa38ea00640 1 -- start start 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.539+0000 7fa38ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa38815db70 0x7fa3880cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.539+0000 7fa38ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3880cfc20 0x7fa38806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.539+0000 7fa38ea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa388071650 con 0x7fa38815df50 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.539+0000 7fa38ea00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3880cbd80 con 0x7fa38815db70 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.539+0000 7fa38ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa388161910 con 0x7fa3880cfc20 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa387e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa38815db70 0x7fa3880cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa37f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa38815df50 0x7fa3880cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:40520/0 (socket says 172.21.15.165:40520) 2024-04-25T10:16:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa37f400640 1 -- 172.21.15.165:0/3406335525 learned_addr learned my addr 172.21.15.165:0/3406335525 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa387e00640 1 -- 172.21.15.165:0/3406335525 >> v2:172.21.15.172:3300/0 conn(0x7fa3880cfc20 msgr2=0x7fa38806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa387e00640 1 --2- 172.21.15.165:0/3406335525 >> v2:172.21.15.172:3300/0 conn(0x7fa3880cfc20 0x7fa38806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa387e00640 1 -- 172.21.15.165:0/3406335525 >> v1:172.21.15.165:6789/0 conn(0x7fa38815df50 legacy=0x7fa3880cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.540+0000 7fa387e00640 1 -- 172.21.15.165:0/3406335525 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880cf230 con 0x7fa38815db70 2024-04-25T10:16:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.541+0000 7fa387e00640 1 --2- 172.21.15.165:0/3406335525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880cb840 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fa3880b4d10 tx=0x7fa37802a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1ba6fee472e29f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.541+0000 7fa387400640 1 -- 172.21.15.165:0/3406335525 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378037070 con 0x7fa38815db70 2024-04-25T10:16:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.541+0000 7fa387400640 1 -- 172.21.15.165:0/3406335525 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37802ad30 con 0x7fa38815db70 2024-04-25T10:16:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.541+0000 7fa387400640 1 -- 172.21.15.165:0/3406335525 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378032660 con 0x7fa38815db70 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 -- 172.21.15.165:0/3406335525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 msgr2=0x7fa3880cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 --2- 172.21.15.165:0/3406335525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880cb840 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fa3880b4d10 tx=0x7fa37802a0a0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 -- 172.21.15.165:0/3406335525 shutdown_connections 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 --2- 172.21.15.165:0/3406335525 >> v2:172.21.15.172:3300/0 conn(0x7fa3880cfc20 0x7fa38806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 --2- 172.21.15.165:0/3406335525 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880cb840 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 -- 172.21.15.165:0/3406335525 >> 172.21.15.165:0/3406335525 conn(0x7fa38806abe0 msgr2=0x7fa38806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 -- 172.21.15.165:0/3406335525 shutdown_connections 2024-04-25T10:16:34.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.542+0000 7fa38ea00640 1 -- 172.21.15.165:0/3406335525 wait complete. 2024-04-25T10:16:34.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.544+0000 7fa38ea00640 1 Processor -- start 2024-04-25T10:16:34.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.544+0000 7fa38ea00640 1 -- start start 2024-04-25T10:16:34.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.545+0000 7fa38ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3880cfc20 0x7fa3880af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.545+0000 7fa38ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.545+0000 7fa38ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38815df50 0x7fa3880b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.545+0000 7fa38ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388162120 con 0x7fa38815db70 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.545+0000 7fa38ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388161e40 con 0x7fa3880cfc20 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.545+0000 7fa38ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa388162640 con 0x7fa38815df50 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880b17b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52008/0 (socket says 172.21.15.165:52008) 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 -- 172.21.15.165:0/726667089 learned_addr learned my addr 172.21.15.165:0/726667089 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa387e00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3880cfc20 0x7fa3880af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa38ca00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38815df50 0x7fa3880b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 -- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38815df50 msgr2=0x7fa3880b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38815df50 0x7fa3880b4050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 -- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3880cfc20 msgr2=0x7fa3880af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3880cfc20 0x7fa3880af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 -- 172.21.15.165:0/726667089 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388159b30 con 0x7fa38815db70 2024-04-25T10:16:34.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.546+0000 7fa37f400640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880b17b0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c7f0 tx=0x7fa37000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.547+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370018070 con 0x7fa38815db70 2024-04-25T10:16:34.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.547+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370007500 con 0x7fa38815db70 2024-04-25T10:16:34.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.547+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa370004a20 con 0x7fa38815db70 2024-04-25T10:16:34.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.547+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa388159e10 con 0x7fa38815db70 2024-04-25T10:16:34.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.547+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38815a3d0 con 0x7fa38815db70 2024-04-25T10:16:34.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.548+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa388073990 con 0x7fa38815db70 2024-04-25T10:16:34.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.549+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa370004bc0 con 0x7fa38815db70 2024-04-25T10:16:34.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.550+0000 7fa384c00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa36406b4d0 0x7fa36406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:34.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.550+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fa370089800 con 0x7fa38815db70 2024-04-25T10:16:34.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.550+0000 7fa387e00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa36406b4d0 0x7fa36406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:34.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.551+0000 7fa387e00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa36406b4d0 0x7fa36406d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa378034000 tx=0x7fa37802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:34.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.555+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa388071650 con 0x7fa38815db70 2024-04-25T10:16:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:34.829+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7fa38815a5a0 con 0x7fa38815db70 2024-04-25T10:16:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.448+0000 7fa384c00640 1 -- 172.21.15.165:0/726667089 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v669) v1 ==== 162+0+0 (secure 0 0 0) 0x7fa370058700 con 0x7fa38815db70 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa36406b4d0 msgr2=0x7fa36406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa36406b4d0 0x7fa36406d990 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa378034000 tx=0x7fa37802e000 comp rx=0 tx=0).stop 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 msgr2=0x7fa3880b17b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880b17b0 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7fa37000c7f0 tx=0x7fa37000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 shutdown_connections 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa36406b4d0 0x7fa36406d990 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa38815df50 0x7fa3880b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa38815db70 0x7fa3880b17b0 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 --2- 172.21.15.165:0/726667089 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3880cfc20 0x7fa3880af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 >> 172.21.15.165:0/726667089 conn(0x7fa38806abe0 msgr2=0x7fa3880cc030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 shutdown_connections 2024-04-25T10:16:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.452+0000 7fa38ea00640 1 -- 172.21.15.165:0/726667089 wait complete. 2024-04-25T10:16:35.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:16:35.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.675+0000 7fd641000640 1 Processor -- start 2024-04-25T10:16:35.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.675+0000 7fd641000640 1 -- start start 2024-04-25T10:16:35.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.676+0000 7fd641000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c06b780 0x7fd63c15d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.676+0000 7fd641000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd63c06abe0 0x7fd63c06b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.676+0000 7fd641000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c0d0d40 con 0x7fd63c145420 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.676+0000 7fd641000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0d1020 con 0x7fd63c06abe0 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.676+0000 7fd641000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0d1300 con 0x7fd63c06b780 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.676+0000 7fd63be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd63c06abe0 0x7fd63c06b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd63c06abe0 0x7fd63c06b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52022/0 (socket says 172.21.15.165:52022) 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63be00640 1 -- 172.21.15.165:0/408155181 learned_addr learned my addr 172.21.15.165:0/408155181 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd633e00640 1 --2- 172.21.15.165:0/408155181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c06b780 0x7fd63c15d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63be00640 1 -- 172.21.15.165:0/408155181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c06b780 msgr2=0x7fd63c15d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63be00640 1 --2- 172.21.15.165:0/408155181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c06b780 0x7fd63c15d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63be00640 1 -- 172.21.15.165:0/408155181 >> v1:172.21.15.165:6789/0 conn(0x7fd63c145420 legacy=0x7fd63c0d0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63be00640 1 -- 172.21.15.165:0/408155181 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c15bd20 con 0x7fd63c06abe0 2024-04-25T10:16:35.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.677+0000 7fd63aa00640 1 -- 172.21.15.165:0/408155181 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383731503 0 0) 0x7fd63c0d0d40 con 0x7fd63c145420 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.678+0000 7fd63be00640 1 --2- 172.21.15.165:0/408155181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06abe0 0x7fd63c06b090 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fd634009e50 tx=0x7fd63402a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d57dae600692f33a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.678+0000 7fd63aa00640 1 -- 172.21.15.165:0/408155181 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd634036070 con 0x7fd63c06abe0 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.678+0000 7fd63aa00640 1 -- 172.21.15.165:0/408155181 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd634031600 con 0x7fd63c06abe0 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.678+0000 7fd63aa00640 1 -- 172.21.15.165:0/408155181 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6340319d0 con 0x7fd63c06abe0 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 -- 172.21.15.165:0/408155181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06abe0 msgr2=0x7fd63c06b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 --2- 172.21.15.165:0/408155181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06abe0 0x7fd63c06b090 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7fd634009e50 tx=0x7fd63402a710 comp rx=0 tx=0).stop 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 -- 172.21.15.165:0/408155181 shutdown_connections 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 --2- 172.21.15.165:0/408155181 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06abe0 0x7fd63c06b090 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 --2- 172.21.15.165:0/408155181 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c06b780 0x7fd63c15d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 -- 172.21.15.165:0/408155181 >> 172.21.15.165:0/408155181 conn(0x7fd63c0728b0 msgr2=0x7fd63c099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 -- 172.21.15.165:0/408155181 shutdown_connections 2024-04-25T10:16:35.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.679+0000 7fd641000640 1 -- 172.21.15.165:0/408155181 wait complete. 2024-04-25T10:16:35.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.681+0000 7fd641000640 1 Processor -- start 2024-04-25T10:16:35.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.681+0000 7fd641000640 1 -- start start 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd641000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd63c06abe0 0x7fd63c0b3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd641000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 0x7fd63c0b4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd641000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c145420 0x7fd63c1446c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd641000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15d930 con 0x7fd63c06b780 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd641000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15d7b0 con 0x7fd63c06abe0 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd641000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c15c520 con 0x7fd63c145420 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.682+0000 7fd633e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 0x7fd63c0b4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 0x7fd63c0b4ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52030/0 (socket says 172.21.15.165:52030) 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 -- 172.21.15.165:0/3149505093 learned_addr learned my addr 172.21.15.165:0/3149505093 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 -- 172.21.15.165:0/3149505093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c145420 msgr2=0x7fd63c1446c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c145420 0x7fd63c1446c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 -- 172.21.15.165:0/3149505093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd63c06abe0 msgr2=0x7fd63c0b3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd63c06abe0 0x7fd63c0b3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 -- 172.21.15.165:0/3149505093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c0cae30 con 0x7fd63c06b780 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633e00640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 0x7fd63c0b4ef0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fd62400d800 tx=0x7fd62400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624004020 con 0x7fd63c06b780 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.683+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6240041c0 con 0x7fd63c06b780 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.684+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c0b3e20 con 0x7fd63c06b780 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.684+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624004390 con 0x7fd63c06b780 2024-04-25T10:16:35.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.684+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c0b43e0 con 0x7fd63c06b780 2024-04-25T10:16:35.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.685+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c0b45e0 con 0x7fd63c06b780 2024-04-25T10:16:35.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.686+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd62401b020 con 0x7fd63c06b780 2024-04-25T10:16:35.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.687+0000 7fd633400640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd61806b4d0 0x7fd61806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:35.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.687+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd624089f80 con 0x7fd63c06b780 2024-04-25T10:16:35.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.687+0000 7fd63b400640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd61806b4d0 0x7fd61806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:35.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.688+0000 7fd63b400640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd61806b4d0 0x7fd61806d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd62c001560 tx=0x7fd62c007930 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:35.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.691+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd63c0b45e0 con 0x7fd63c06b780 2024-04-25T10:16:35.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.884+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd63c073990 con 0x7fd63c06b780 2024-04-25T10:16:35.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.891+0000 7fd633400640 1 -- 172.21.15.165:0/3149505093 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd624058e80 con 0x7fd63c06b780 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd61806b4d0 msgr2=0x7fd61806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd61806b4d0 0x7fd61806d990 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fd62c001560 tx=0x7fd62c007930 comp rx=0 tx=0).stop 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 msgr2=0x7fd63c0b4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 0x7fd63c0b4ef0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7fd62400d800 tx=0x7fd62400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 shutdown_connections 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd61806b4d0 0x7fd61806d990 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd63c145420 0x7fd63c1446c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd63c06b780 0x7fd63c0b4ef0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 --2- 172.21.15.165:0/3149505093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd63c06abe0 0x7fd63c0b3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:35.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 >> 172.21.15.165:0/3149505093 conn(0x7fd63c0728b0 msgr2=0x7fd63c15db00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:35.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 shutdown_connections 2024-04-25T10:16:35.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:35.894+0000 7fd641000640 1 -- 172.21.15.165:0/3149505093 wait complete. 2024-04-25T10:16:35.917 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:16:35.918 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:36.044 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:16:36.164 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:16:36.164 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:16:36.164 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:36.221 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:36.221 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:16:36.221 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:36.280 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:36.280 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:16:36.280 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:36.338 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:16:36.338 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:16:36.338 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:36.396 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:16:36.396 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:16:36.396 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:36.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.605+0000 7fd121e00640 1 Processor -- start 2024-04-25T10:16:36.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.605+0000 7fd121e00640 1 -- start start 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd121e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd11c144750 0x7fd11c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd121e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd11c06eb10 0x7fd11c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd121e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c0cee60 0x7fd11c15f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd121e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11c144b30 con 0x7fd11c06eb10 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd121e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd11c0712d0 con 0x7fd11c144750 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd121e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c0715b0 con 0x7fd11c0cee60 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd11c06eb10 0x7fd11c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd11c06eb10 0x7fd11c0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52046/0 (socket says 172.21.15.165:52046) 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 -- 172.21.15.165:0/2508038703 learned_addr learned my addr 172.21.15.165:0/2508038703 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11b400640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c0cee60 0x7fd11c15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11aa00640 1 --2- 172.21.15.165:0/2508038703 >> v2:172.21.15.172:3300/0 conn(0x7fd11c144750 0x7fd11c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 -- 172.21.15.165:0/2508038703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c0cee60 msgr2=0x7fd11c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c0cee60 0x7fd11c15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 -- 172.21.15.165:0/2508038703 >> v2:172.21.15.172:3300/0 conn(0x7fd11c144750 msgr2=0x7fd11c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 --2- 172.21.15.165:0/2508038703 >> v2:172.21.15.172:3300/0 conn(0x7fd11c144750 0x7fd11c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11a000640 1 -- 172.21.15.165:0/2508038703 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c071890 con 0x7fd11c06eb10 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.607+0000 7fd11b400640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c0cee60 0x7fd11c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.609+0000 7fd11a000640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c06eb10 0x7fd11c0ce920 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fd11c06f5d0 tx=0x7fd10802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ba48c6bb16c393a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.609+0000 7fd119600640 1 -- 172.21.15.165:0/2508038703 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108037070 con 0x7fd11c06eb10 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.609+0000 7fd119600640 1 -- 172.21.15.165:0/2508038703 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd108032440 con 0x7fd11c06eb10 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.609+0000 7fd119600640 1 -- 172.21.15.165:0/2508038703 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd108032810 con 0x7fd11c06eb10 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 -- 172.21.15.165:0/2508038703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c06eb10 msgr2=0x7fd11c0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c06eb10 0x7fd11c0ce920 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7fd11c06f5d0 tx=0x7fd10802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 -- 172.21.15.165:0/2508038703 shutdown_connections 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c0cee60 0x7fd11c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 --2- 172.21.15.165:0/2508038703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c06eb10 0x7fd11c0ce920 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 --2- 172.21.15.165:0/2508038703 >> v2:172.21.15.172:3300/0 conn(0x7fd11c144750 0x7fd11c06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 -- 172.21.15.165:0/2508038703 >> 172.21.15.165:0/2508038703 conn(0x7fd11c06abe0 msgr2=0x7fd11c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 -- 172.21.15.165:0/2508038703 shutdown_connections 2024-04-25T10:16:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.610+0000 7fd121e00640 1 -- 172.21.15.165:0/2508038703 wait complete. 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.612+0000 7fd121e00640 1 Processor -- start 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.612+0000 7fd121e00640 1 -- start start 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.613+0000 7fd121e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c06eb10 0x7fd11c0b4170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.613+0000 7fd121e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd11c0cee60 0x7fd11c142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.613+0000 7fd121e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 0x7fd11c0b16f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.613+0000 7fd121e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c145450 con 0x7fd11c144750 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.613+0000 7fd121e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c06f020 con 0x7fd11c0cee60 2024-04-25T10:16:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.613+0000 7fd121e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11c1455d0 con 0x7fd11c06eb10 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 0x7fd11c0b16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 0x7fd11c0b16f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52052/0 (socket says 172.21.15.165:52052) 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 -- 172.21.15.165:0/4179582373 learned_addr learned my addr 172.21.15.165:0/4179582373 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11aa00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c06eb10 0x7fd11c0b4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11a000640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd11c0cee60 0x7fd11c142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 -- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c06eb10 msgr2=0x7fd11c0b4170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c06eb10 0x7fd11c0b4170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 -- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd11c0cee60 msgr2=0x7fd11c142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd11c0cee60 0x7fd11c142b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 -- 172.21.15.165:0/4179582373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd11c0b30d0 con 0x7fd11c144750 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11aa00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c06eb10 0x7fd11c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.614+0000 7fd11b400640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 0x7fd11c0b16f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fd11400cd40 tx=0x7fd11400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.615+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11400f680 con 0x7fd11c144750 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.615+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11400f840 con 0x7fd11c144750 2024-04-25T10:16:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.615+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11400fc30 con 0x7fd11c144750 2024-04-25T10:16:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.615+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd11c0b33b0 con 0x7fd11c144750 2024-04-25T10:16:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.615+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd11c0b3970 con 0x7fd11c144750 2024-04-25T10:16:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.616+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd11c073990 con 0x7fd11c144750 2024-04-25T10:16:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.617+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd1140213f0 con 0x7fd11c144750 2024-04-25T10:16:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.618+0000 7fd113400640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd0fc06b4d0 0x7fd0fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.618+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd11408af20 con 0x7fd11c144750 2024-04-25T10:16:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.618+0000 7fd11aa00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd0fc06b4d0 0x7fd0fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.619+0000 7fd11aa00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd0fc06b4d0 0x7fd0fc06d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd10c008890 tx=0x7fd10c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.622+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd114014030 con 0x7fd11c144750 2024-04-25T10:16:36.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.859+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd11c06e8f0 con 0x7fd11c144750 2024-04-25T10:16:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.859+0000 7fd113400640 1 -- 172.21.15.165:0/4179582373 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+2914 (secure 0 0 0) 0x7fd114059860 con 0x7fd11c144750 2024-04-25T10:16:36.864 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:36.864 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":96,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6346,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3864385879","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3864385879},{"type":"v1","addr":"172.21.15.165:6836","nonce":3864385879}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":6355,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2682868927","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2682868927},{"type":"v1","addr":"172.21.15.172:6833","nonce":2682868927}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":85},{"gid":9657,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/513525724","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":513525724},{"type":"v1","addr":"172.21.15.165:6837","nonce":513525724}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":96}],"filesystems":[]} 2024-04-25T10:16:36.864 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 96 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd0fc06b4d0 msgr2=0x7fd0fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd0fc06b4d0 0x7fd0fc06d990 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fd10c008890 tx=0x7fd10c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 msgr2=0x7fd11c0b16f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 0x7fd11c0b16f0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7fd11400cd40 tx=0x7fd11400a650 comp rx=0 tx=0).stop 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 shutdown_connections 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd0fc06b4d0 0x7fd0fc06d990 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd11c144750 0x7fd11c0b16f0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd11c0cee60 0x7fd11c142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 --2- 172.21.15.165:0/4179582373 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd11c06eb10 0x7fd11c0b4170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 >> 172.21.15.165:0/4179582373 conn(0x7fd11c06abe0 msgr2=0x7fd11c13f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.862+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 shutdown_connections 2024-04-25T10:16:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:36.863+0000 7fd121e00640 1 -- 172.21.15.165:0/4179582373 wait complete. 2024-04-25T10:16:36.886 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:16:36.886 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:16:36.886 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:16:36.886 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:36.887 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:16:36.887 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:16:36.887 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:16:36.887 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:36.887 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:16:36.887 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:16:36.888 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:16:36.888 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:16:36.888 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:16:36.888 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:16:36.906 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:16:36.906 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:16:36.907 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:16:36.907 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:16:36.914 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:16:36.914 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:16:36.919 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:16:36.920 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:36.926 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:16:36.926 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:36.954 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:16:36.954 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:36.959 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:16:36.959 DEBUG:teuthology.parallel:result is None 2024-04-25T10:16:36.959 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:16:37.068 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:16:37.074 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:16:37.099 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:16:37.100 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:16:37.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.133+0000 7f4efe400640 1 Processor -- start 2024-04-25T10:16:37.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.133+0000 7f4efe400640 1 -- start start 2024-04-25T10:16:37.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.134+0000 7f4efe400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4ef8161fc0 0x7f4ef81623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.134+0000 7f4efe400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef806ea90 0x7f4ef8071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.134+0000 7f4efe400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4ef806ef10 0x7f4ef806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.134+0000 7f4efe400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef8165be0 con 0x7f4ef8161fc0 2024-04-25T10:16:37.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.134+0000 7f4efe400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef81628e0 con 0x7f4ef806ef10 2024-04-25T10:16:37.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.134+0000 7f4efe400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8162ba0 con 0x7f4ef806ea90 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4ef8161fc0 0x7f4ef81623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4ef8161fc0 0x7f4ef81623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43380/0 (socket says 172.21.15.165:43380) 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 -- 172.21.15.165:0/1402927567 learned_addr learned my addr 172.21.15.165:0/1402927567 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7e00640 1 --2- 172.21.15.165:0/1402927567 >> v2:172.21.15.172:3300/0 conn(0x7f4ef806ef10 0x7f4ef806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef806ea90 0x7f4ef8071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 -- 172.21.15.165:0/1402927567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef806ea90 msgr2=0x7f4ef8071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef806ea90 0x7f4ef8071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 -- 172.21.15.165:0/1402927567 >> v2:172.21.15.172:3300/0 conn(0x7f4ef806ef10 msgr2=0x7f4ef806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1402927567 >> v2:172.21.15.172:3300/0 conn(0x7f4ef806ef10 0x7f4ef806d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef7400640 1 -- 172.21.15.165:0/1402927567 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef8162e80 con 0x7f4ef8161fc0 2024-04-25T10:16:37.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.135+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef806ea90 0x7f4ef8071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:37.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.136+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef8161fc0 0x7f4ef81623a0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f4ef806e4f0 tx=0x7f4ee802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb2d6e443ab27d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4ef6000640 1 -- 172.21.15.165:0/1402927567 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8037070 con 0x7f4ef8161fc0 2024-04-25T10:16:37.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4ef6000640 1 -- 172.21.15.165:0/1402927567 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee8032440 con 0x7f4ef8161fc0 2024-04-25T10:16:37.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4ef6000640 1 -- 172.21.15.165:0/1402927567 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8032810 con 0x7f4ef8161fc0 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4efe400640 1 -- 172.21.15.165:0/1402927567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef8161fc0 msgr2=0x7f4ef81623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4efe400640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef8161fc0 0x7f4ef81623a0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7f4ef806e4f0 tx=0x7f4ee802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4efe400640 1 -- 172.21.15.165:0/1402927567 shutdown_connections 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.137+0000 7f4efe400640 1 --2- 172.21.15.165:0/1402927567 >> v2:172.21.15.172:3300/0 conn(0x7f4ef806ef10 0x7f4ef806d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.138+0000 7f4efe400640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef806ea90 0x7f4ef8071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.138+0000 7f4efe400640 1 --2- 172.21.15.165:0/1402927567 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef8161fc0 0x7f4ef81623a0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.138+0000 7f4efe400640 1 -- 172.21.15.165:0/1402927567 >> 172.21.15.165:0/1402927567 conn(0x7f4ef806abe0 msgr2=0x7f4ef806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.138+0000 7f4efe400640 1 -- 172.21.15.165:0/1402927567 shutdown_connections 2024-04-25T10:16:37.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.138+0000 7f4efe400640 1 -- 172.21.15.165:0/1402927567 wait complete. 2024-04-25T10:16:37.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.139+0000 7f4efe400640 1 Processor -- start 2024-04-25T10:16:37.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.140+0000 7f4efe400640 1 -- start start 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4efe400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef806ea90 0x7f4ef8159eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4efe400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 0x7f4ef8153b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4efe400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef8161fc0 0x7f4ef81577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4efe400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef80719c0 con 0x7f4ef806ef10 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4efe400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8163440 con 0x7f4ef806ea90 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4efe400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef8071b40 con 0x7f4ef8161fc0 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 0x7f4ef8153b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 0x7f4ef8153b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43388/0 (socket says 172.21.15.165:43388) 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 -- 172.21.15.165:0/1553223059 learned_addr learned my addr 172.21.15.165:0/1553223059 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 -- 172.21.15.165:0/1553223059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef8161fc0 msgr2=0x7f4ef81577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef8161fc0 0x7f4ef81577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 -- 172.21.15.165:0/1553223059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef806ea90 msgr2=0x7f4ef8159eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef806ea90 0x7f4ef8159eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:37.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.141+0000 7f4ef6a00640 1 -- 172.21.15.165:0/1553223059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef81f7ce0 con 0x7f4ef806ef10 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.142+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 0x7f4ef8153b90 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e810 tx=0x7f4ee400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.142+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee400f040 con 0x7f4ef806ef10 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.142+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4004030 con 0x7f4ef806ef10 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.142+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4004400 con 0x7f4ef806ef10 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.142+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef815a4b0 con 0x7f4ef806ef10 2024-04-25T10:16:37.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.142+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef81f7f50 con 0x7f4ef806ef10 2024-04-25T10:16:37.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.143+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef8073a00 con 0x7f4ef806ef10 2024-04-25T10:16:37.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.144+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ee400c6d0 con 0x7f4ef806ef10 2024-04-25T10:16:37.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.145+0000 7f4eef400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:37.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.145+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4ee4089c80 con 0x7f4ef806ef10 2024-04-25T10:16:37.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.145+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:37.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.146+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b4d0 0x7f4ed806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8034000 tx=0x7f4ee802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:37.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.149+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ee4017020 con 0x7f4ef806ef10 2024-04-25T10:16:37.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:37.359+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f4ef80d0260 con 0x7f4ef806ef10 2024-04-25T10:16:38.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.477+0000 7f4eef400640 1 -- 172.21.15.165:0/1553223059 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v144) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4ee4058cb0 con 0x7f4ef806ef10 2024-04-25T10:16:38.481 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b4d0 msgr2=0x7f4ed806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b4d0 0x7f4ed806d990 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8034000 tx=0x7f4ee802c000 comp rx=0 tx=0).stop 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 msgr2=0x7f4ef8153b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 0x7f4ef8153b90 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400e810 tx=0x7f4ee400ecd0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 shutdown_connections 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b4d0 0x7f4ed806d990 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef8161fc0 0x7f4ef81577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef806ef10 0x7f4ef8153b90 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 --2- 172.21.15.165:0/1553223059 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4ef806ea90 0x7f4ef8159eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.481+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 >> 172.21.15.165:0/1553223059 conn(0x7f4ef806abe0 msgr2=0x7f4ef806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.482+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 shutdown_connections 2024-04-25T10:16:38.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.482+0000 7f4efe400640 1 -- 172.21.15.165:0/1553223059 wait complete. 2024-04-25T10:16:38.505 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:16:38.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.684+0000 7f4efea00640 1 Processor -- start 2024-04-25T10:16:38.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.684+0000 7f4efea00640 1 -- start start 2024-04-25T10:16:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.685+0000 7f4efea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 0x7f4ef806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.685+0000 7f4efea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef815c150 con 0x7f4ef8144d40 2024-04-25T10:16:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.685+0000 7f4efea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef80cd4b0 con 0x7f4ef80cc930 2024-04-25T10:16:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.685+0000 7f4efea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef806d850 con 0x7f4ef80cf060 2024-04-25T10:16:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4ef8144d40 0x7f4ef815c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47852/0 (socket says 172.21.15.165:47852) 2024-04-25T10:16:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef7e00640 1 -- 172.21.15.165:0/4083140341 learned_addr learned my addr 172.21.15.165:0/4083140341 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 463656812 0 0) 0x7f4ef815c150 con 0x7f4ef8144d40 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4edc003520 con 0x7f4ef8144d40 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4efca00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 0x7f4ef806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2429656263 0 0) 0x7f4edc003520 con 0x7f4ef8144d40 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 msgr2=0x7f4ef806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 0x7f4ef806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 >> v1:172.21.15.172:6789/0 conn(0x7f4ef80cc930 legacy=0x7f4ef80ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80ccef0 con 0x7f4ef8144d40 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.686+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4ee8001690 con 0x7f4ef8144d40 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 >> v1:172.21.15.165:6789/0 conn(0x7f4ef8144d40 legacy=0x7f4ef815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 0x7f4edc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004060 0x7f4edc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4edc004e60 0x7f4edc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc005650 con 0x7f4edc004e60 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc00bf00 con 0x7f4edc004060 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc00c080 con 0x7f4ef80cf060 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4efca00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4edc004e60 0x7f4edc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4ef7e00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 0x7f4edc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4efca00640 1 -- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 msgr2=0x7f4edc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4efca00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80cf060 0x7f4edc0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4efca00640 1 -- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004060 msgr2=0x7f4edc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4efca00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004060 0x7f4edc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.687+0000 7f4efca00640 1 -- 172.21.15.165:0/4083140341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc003520 con 0x7f4edc004e60 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.688+0000 7f4efca00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4edc004e60 0x7f4edc00b9c0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0007390 tx=0x7f4edc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d0d26b8a09f4286 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0009be0 con 0x7f4edc004e60 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef002e620 con 0x7f4edc004e60 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4ef6a00640 1 -- 172.21.15.165:0/4083140341 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef002e9f0 con 0x7f4edc004e60 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efea00640 1 -- 172.21.15.165:0/4083140341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4edc004e60 msgr2=0x7f4edc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efea00640 1 --2- 172.21.15.165:0/4083140341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4edc004e60 0x7f4edc00b9c0 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7f4ef0007390 tx=0x7f4edc005830 comp rx=0 tx=0).stop 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efca00640 1 -- 172.21.15.165:0/4083140341 reap_dead start 2024-04-25T10:16:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efea00640 1 -- 172.21.15.165:0/4083140341 shutdown_connections 2024-04-25T10:16:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efea00640 1 -- 172.21.15.165:0/4083140341 >> 172.21.15.165:0/4083140341 conn(0x7f4ef806abe0 msgr2=0x7f4ef806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efea00640 1 -- 172.21.15.165:0/4083140341 shutdown_connections 2024-04-25T10:16:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.689+0000 7f4efea00640 1 -- 172.21.15.165:0/4083140341 wait complete. 2024-04-25T10:16:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.691+0000 7f4efea00640 1 Processor -- start 2024-04-25T10:16:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.691+0000 7f4efea00640 1 -- start start 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004e60 0x7f4ef81f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80ac8b0 0x7f4ef80ad760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 0x7f4ef813dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef806d310 con 0x7f4ef80b2d10 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef815c3f0 con 0x7f4edc004e60 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef806d490 con 0x7f4ef80ac8b0 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 0x7f4ef813dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 0x7f4ef813dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43406/0 (socket says 172.21.15.165:43406) 2024-04-25T10:16:38.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.692+0000 7f4efca00640 1 -- 172.21.15.165:0/1246212636 learned_addr learned my addr 172.21.15.165:0/1246212636 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4ef7e00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004e60 0x7f4ef81f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80ac8b0 0x7f4ef80ad760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efca00640 1 -- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80ac8b0 msgr2=0x7f4ef80ad760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efca00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80ac8b0 0x7f4ef80ad760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efca00640 1 -- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004e60 msgr2=0x7f4ef81f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efca00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004e60 0x7f4ef81f40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efca00640 1 -- 172.21.15.165:0/1246212636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef80af370 con 0x7f4ef80b2d10 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4ef7400640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80ac8b0 0x7f4ef80ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efca00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 0x7f4ef813dd80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00074b0 tx=0x7f4ef0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0009be0 con 0x7f4ef80b2d10 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ef002e420 con 0x7f4ef80b2d10 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0038280 con 0x7f4ef80b2d10 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.693+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef80af650 con 0x7f4ef80b2d10 2024-04-25T10:16:38.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.694+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef81f4620 con 0x7f4ef80b2d10 2024-04-25T10:16:38.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.695+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef80cedd0 con 0x7f4ef80b2d10 2024-04-25T10:16:38.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.695+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4ef0038870 con 0x7f4ef80b2d10 2024-04-25T10:16:38.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.697+0000 7f4eefe00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b500 0x7f4ed806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:38.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.697+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4ef00aa8e0 con 0x7f4ef80b2d10 2024-04-25T10:16:38.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.697+0000 7f4ef7e00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b500 0x7f4ed806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:38.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.698+0000 7f4ef7e00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b500 0x7f4ed806d9c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80010e0 tx=0x7f4ee8000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:38.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.701+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4ef80cedd0 con 0x7f4ef80b2d10 2024-04-25T10:16:38.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.967+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f4ef8067df0 con 0x7f4ef80b2d10 2024-04-25T10:16:38.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.968+0000 7f4eefe00640 1 -- 172.21.15.165:0/1246212636 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f4ef0032030 con 0x7f4ef80b2d10 2024-04-25T10:16:38.972 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:38.972 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:16:38.972 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:16:38.972 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:16:38.973 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.974 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.975 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:16:38.976 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.977 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.978 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.979 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:16:38.980 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.981 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:16:38.982 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:38.983 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:16:38.984 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.971+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b500 msgr2=0x7f4ed806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.971+0000 7f4efea00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b500 0x7f4ed806d9c0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f4ee80010e0 tx=0x7f4ee8000f50 comp rx=0 tx=0).stop 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.971+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 msgr2=0x7f4ef813dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.971+0000 7f4efea00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 0x7f4ef813dd80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f4ef00074b0 tx=0x7f4ef0004ab0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 shutdown_connections 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4ed806b500 0x7f4ed806d9c0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4ef80b2d10 0x7f4ef813dd80 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4ef80ac8b0 0x7f4ef80ad760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 --2- 172.21.15.165:0/1246212636 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4edc004e60 0x7f4ef81f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:38.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 >> 172.21.15.165:0/1246212636 conn(0x7f4ef806abe0 msgr2=0x7f4ef815e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:38.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 shutdown_connections 2024-04-25T10:16:38.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:38.972+0000 7f4efea00640 1 -- 172.21.15.165:0/1246212636 wait complete. 2024-04-25T10:16:38.995 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:39.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.187+0000 7ff089200640 1 Processor -- start 2024-04-25T10:16:39.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.187+0000 7ff089200640 1 -- start start 2024-04-25T10:16:39.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.188+0000 7ff089200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.188+0000 7ff089200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff084071530 0x7ff0840cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.188+0000 7ff089200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff08415bf30 con 0x7ff0840cd860 2024-04-25T10:16:39.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.188+0000 7ff089200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff08415c860 con 0x7ff084071530 2024-04-25T10:16:39.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.188+0000 7ff089200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff084071970 con 0x7ff084144440 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff083e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff0840cd860 0x7ff08406d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47866/0 (socket says 172.21.15.165:47866) 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff083e00640 1 -- 172.21.15.165:0/2353869769 learned_addr learned my addr 172.21.15.165:0/2353869769 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082a00640 1 --2- 172.21.15.165:0/2353869769 >> v2:172.21.15.165:3300/0 conn(0x7ff084071530 0x7ff0840cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff083400640 1 --2- 172.21.15.165:0/2353869769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082000640 1 -- 172.21.15.165:0/2353869769 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149144525 0 0) 0x7ff08415bf30 con 0x7ff0840cd860 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082000640 1 -- 172.21.15.165:0/2353869769 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff068003520 con 0x7ff0840cd860 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082a00640 1 -- 172.21.15.165:0/2353869769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 msgr2=0x7ff084070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082a00640 1 --2- 172.21.15.165:0/2353869769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082a00640 1 -- 172.21.15.165:0/2353869769 >> v1:172.21.15.165:6789/0 conn(0x7ff0840cd860 legacy=0x7ff08406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.189+0000 7ff082a00640 1 -- 172.21.15.165:0/2353869769 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff08415c210 con 0x7ff084071530 2024-04-25T10:16:39.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.190+0000 7ff082a00640 1 --2- 172.21.15.165:0/2353869769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff084071530 0x7ff0840cd320 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0097a0 tx=0x7ff06c02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2db8704298c32ceb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff082000640 1 -- 172.21.15.165:0/2353869769 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c037070 con 0x7ff084071530 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff082000640 1 -- 172.21.15.165:0/2353869769 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c032440 con 0x7ff084071530 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff082000640 1 -- 172.21.15.165:0/2353869769 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c032810 con 0x7ff084071530 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 -- 172.21.15.165:0/2353869769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff084071530 msgr2=0x7ff0840cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 --2- 172.21.15.165:0/2353869769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff084071530 0x7ff0840cd320 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0097a0 tx=0x7ff06c02a210 comp rx=0 tx=0).stop 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 -- 172.21.15.165:0/2353869769 shutdown_connections 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 --2- 172.21.15.165:0/2353869769 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff084071530 0x7ff0840cd320 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 --2- 172.21.15.165:0/2353869769 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 -- 172.21.15.165:0/2353869769 >> 172.21.15.165:0/2353869769 conn(0x7ff08406abe0 msgr2=0x7ff08406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 -- 172.21.15.165:0/2353869769 shutdown_connections 2024-04-25T10:16:39.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.191+0000 7ff089200640 1 -- 172.21.15.165:0/2353869769 wait complete. 2024-04-25T10:16:39.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.193+0000 7ff089200640 1 Processor -- start 2024-04-25T10:16:39.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.193+0000 7ff089200640 1 -- start start 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff089200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff084071530 0x7ff084153730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff089200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 0x7ff08413c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff089200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff089200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840cdcd0 con 0x7ff0840cd860 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff089200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff08415bc30 con 0x7ff084071530 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff089200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0840cde50 con 0x7ff084144440 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff082a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 0x7ff08413c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff082a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 0x7ff08413c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43420/0 (socket says 172.21.15.165:43420) 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.194+0000 7ff082a00640 1 -- 172.21.15.165:0/1357035563 learned_addr learned my addr 172.21.15.165:0/1357035563 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff082a00640 1 -- 172.21.15.165:0/1357035563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 msgr2=0x7ff084156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff082a00640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff082a00640 1 -- 172.21.15.165:0/1357035563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff084071530 msgr2=0x7ff084153730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff082a00640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff084071530 0x7ff084153730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff082a00640 1 -- 172.21.15.165:0/1357035563 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff084154f20 con 0x7ff0840cd860 2024-04-25T10:16:39.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff082a00640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 0x7ff08413c490 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0098c0 tx=0x7ff06c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c037040 con 0x7ff0840cd860 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff06c030500 con 0x7ff0840cd860 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff06c0308d0 con 0x7ff0840cd860 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.195+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff084154190 con 0x7ff0840cd860 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.196+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff084154bb0 con 0x7ff0840cd860 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.197+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff084073990 con 0x7ff0840cd860 2024-04-25T10:16:39.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.198+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff06c044020 con 0x7ff0840cd860 2024-04-25T10:16:39.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.198+0000 7ff07b400640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff06006b4d0 0x7ff06006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.199+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7ff06c0aa2e0 con 0x7ff0840cd860 2024-04-25T10:16:39.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.199+0000 7ff083400640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff06006b4d0 0x7ff06006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.199+0000 7ff083400640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff06006b4d0 0x7ff06006d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff0740070b0 tx=0x7ff074007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.203+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff06c079310 con 0x7ff0840cd860 2024-04-25T10:16:39.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.400+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff0840cc4b0 con 0x7ff0840cd860 2024-04-25T10:16:39.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.401+0000 7ff07b400640 1 -- 172.21.15.165:0/1357035563 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) v1 ==== 74+0+13210 (secure 0 0 0) 0x7ff06c078c10 con 0x7ff0840cd860 2024-04-25T10:16:39.404 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:39.404 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":144,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:38.474656+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":134,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:39.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.403+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff06006b4d0 msgr2=0x7ff06006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.403+0000 7ff089200640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff06006b4d0 0x7ff06006d990 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff0740070b0 tx=0x7ff074007040 comp rx=0 tx=0).stop 2024-04-25T10:16:39.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.403+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 msgr2=0x7ff08413c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.403+0000 7ff089200640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 0x7ff08413c490 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7ff06c0098c0 tx=0x7ff06c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 shutdown_connections 2024-04-25T10:16:39.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff06006b4d0 0x7ff06006d990 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff084144440 0x7ff084156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0840cd860 0x7ff08413c490 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 --2- 172.21.15.165:0/1357035563 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff084071530 0x7ff084153730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 >> 172.21.15.165:0/1357035563 conn(0x7ff08406abe0 msgr2=0x7ff0840c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 shutdown_connections 2024-04-25T10:16:39.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.404+0000 7ff089200640 1 -- 172.21.15.165:0/1357035563 wait complete. 2024-04-25T10:16:39.426 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:16:39.426 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:16:39.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.632+0000 7f13db200640 1 Processor -- start 2024-04-25T10:16:39.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.632+0000 7f13db200640 1 -- start start 2024-04-25T10:16:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13db200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f13d415b710 0x7f13d40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13db200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f13d415ee60 0x7f13d40ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13db200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d415e490 con 0x7f13d415ee60 2024-04-25T10:16:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13db200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13d40cfd80 con 0x7f13d415b710 2024-04-25T10:16:39.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40d0060 con 0x7f13d406ec50 2024-04-25T10:16:39.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13d9200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f13d415ee60 0x7f13d40ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13d9200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f13d415ee60 0x7f13d40ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43434/0 (socket says 172.21.15.165:43434) 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.634+0000 7f13d9200640 1 -- 172.21.15.165:0/559785652 learned_addr learned my addr 172.21.15.165:0/559785652 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d3400640 1 --2- 172.21.15.165:0/559785652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d3e00640 1 --2- 172.21.15.165:0/559785652 >> v2:172.21.15.172:3300/0 conn(0x7f13d415b710 0x7f13d40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d9200640 1 -- 172.21.15.165:0/559785652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 msgr2=0x7f13d415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d9200640 1 --2- 172.21.15.165:0/559785652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d9200640 1 -- 172.21.15.165:0/559785652 >> v2:172.21.15.172:3300/0 conn(0x7f13d415b710 msgr2=0x7f13d40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d9200640 1 --2- 172.21.15.165:0/559785652 >> v2:172.21.15.172:3300/0 conn(0x7f13d415b710 0x7f13d40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d9200640 1 -- 172.21.15.165:0/559785652 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d40d0340 con 0x7f13d415ee60 2024-04-25T10:16:39.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.635+0000 7f13d3e00640 1 --2- 172.21.15.165:0/559785652 >> v2:172.21.15.172:3300/0 conn(0x7f13d415b710 0x7f13d40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.636+0000 7f13d9200640 1 --2- 172.21.15.165:0/559785652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d40ac8b0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f13d406b950 tx=0x7f13cc02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ad379067846cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.636+0000 7f13d2a00640 1 -- 172.21.15.165:0/559785652 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc036070 con 0x7f13d415ee60 2024-04-25T10:16:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13d2a00640 1 -- 172.21.15.165:0/559785652 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc031600 con 0x7f13d415ee60 2024-04-25T10:16:39.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13d2a00640 1 -- 172.21.15.165:0/559785652 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc0319d0 con 0x7f13d415ee60 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 -- 172.21.15.165:0/559785652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 msgr2=0x7f13d40ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 --2- 172.21.15.165:0/559785652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d40ac8b0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f13d406b950 tx=0x7f13cc02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 -- 172.21.15.165:0/559785652 shutdown_connections 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 --2- 172.21.15.165:0/559785652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d40ac8b0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 --2- 172.21.15.165:0/559785652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 --2- 172.21.15.165:0/559785652 >> v2:172.21.15.172:3300/0 conn(0x7f13d415b710 0x7f13d40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 -- 172.21.15.165:0/559785652 >> 172.21.15.165:0/559785652 conn(0x7f13d406abe0 msgr2=0x7f13d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 -- 172.21.15.165:0/559785652 shutdown_connections 2024-04-25T10:16:39.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.637+0000 7f13db200640 1 -- 172.21.15.165:0/559785652 wait complete. 2024-04-25T10:16:39.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.639+0000 7f13db200640 1 Processor -- start 2024-04-25T10:16:39.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.639+0000 7f13db200640 1 -- start start 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.640+0000 7f13db200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d4147220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.640+0000 7f13db200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d415b710 0x7f13d4159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.640+0000 7f13db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d4159760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.640+0000 7f13db200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d415bc70 con 0x7f13d415ee60 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.640+0000 7f13db200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d415baf0 con 0x7f13d415b710 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.640+0000 7f13db200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d40acdf0 con 0x7f13d406ec50 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d4159760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d4159760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43450/0 (socket says 172.21.15.165:43450) 2024-04-25T10:16:39.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 -- 172.21.15.165:0/395161538 learned_addr learned my addr 172.21.15.165:0/395161538 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d3400640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d415b710 0x7f13d4159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d3e00640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d4147220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 -- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 msgr2=0x7f13d4147220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d4147220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 -- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d415b710 msgr2=0x7f13d4159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d415b710 0x7f13d4159220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 -- 172.21.15.165:0/395161538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d414dc30 con 0x7f13d415ee60 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.641+0000 7f13d9200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d4159760 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009f40 tx=0x7f13cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.642+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc041020 con 0x7f13d415ee60 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.642+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13cc007780 con 0x7f13d415ee60 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.642+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc036040 con 0x7f13d415ee60 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.642+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13d40af3b0 con 0x7f13d415ee60 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.642+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13d40af970 con 0x7f13d415ee60 2024-04-25T10:16:39.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.643+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13d4073a00 con 0x7f13d415ee60 2024-04-25T10:16:39.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.644+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13cc044020 con 0x7f13d415ee60 2024-04-25T10:16:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.645+0000 7f13cbe00640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13b406b4d0 0x7f13b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.646+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f13cc0aad30 con 0x7f13d415ee60 2024-04-25T10:16:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.646+0000 7f13d3e00640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13b406b4d0 0x7f13b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:39.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.646+0000 7f13d3e00640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13b406b4d0 0x7f13b406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f13c4008850 tx=0x7f13c4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:39.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.650+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13cc0b0020 con 0x7f13d415ee60 2024-04-25T10:16:39.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.850+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13d406df30 con 0x7f13d415ee60 2024-04-25T10:16:39.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.851+0000 7f13cbe00640 1 -- 172.21.15.165:0/395161538 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13cc04b030 con 0x7f13d415ee60 2024-04-25T10:16:39.855 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:16:39.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.853+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13b406b4d0 msgr2=0x7f13b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.853+0000 7f13db200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13b406b4d0 0x7f13b406d990 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f13c4008850 tx=0x7f13c4008040 comp rx=0 tx=0).stop 2024-04-25T10:16:39.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.853+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 msgr2=0x7f13d4159760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.853+0000 7f13db200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d4159760 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f13cc009f40 tx=0x7f13cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 shutdown_connections 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13b406b4d0 0x7f13b406d990 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13d415ee60 0x7f13d4159760 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13d415b710 0x7f13d4159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 --2- 172.21.15.165:0/395161538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13d406ec50 0x7f13d4147220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 >> 172.21.15.165:0/395161538 conn(0x7f13d406abe0 msgr2=0x7f13d415c700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 shutdown_connections 2024-04-25T10:16:39.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:39.854+0000 7f13db200640 1 -- 172.21.15.165:0/395161538 wait complete. 2024-04-25T10:16:39.877 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:16:40.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.077+0000 7f5cc7600640 1 Processor -- start 2024-04-25T10:16:40.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.077+0000 7f5cc7600640 1 -- start start 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc7600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5cc015db10 0x7f5cc0162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc7600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5cc00b3220 0x7f5cc00b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc00cf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc7600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc00b3bc0 con 0x7f5cc00b3220 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc7600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cc0072280 con 0x7f5cc015db10 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0162770 con 0x7f5cc00cf3f0 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cb7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5cc00b3220 0x7f5cc00b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cb7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5cc00b3220 0x7f5cc00b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43466/0 (socket says 172.21.15.165:43466) 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cb7e00640 1 -- 172.21.15.165:0/360087280 learned_addr learned my addr 172.21.15.165:0/360087280 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc4c00640 1 --2- 172.21.15.165:0/360087280 >> v2:172.21.15.172:3300/0 conn(0x7f5cc015db10 0x7f5cc0162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.079+0000 7f5cc5600640 1 --2- 172.21.15.165:0/360087280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc00cf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.080+0000 7f5cb7e00640 1 -- 172.21.15.165:0/360087280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 msgr2=0x7f5cc00cf860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.080+0000 7f5cb7e00640 1 --2- 172.21.15.165:0/360087280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc00cf860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.080+0000 7f5cb7e00640 1 -- 172.21.15.165:0/360087280 >> v2:172.21.15.172:3300/0 conn(0x7f5cc015db10 msgr2=0x7f5cc0162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.080+0000 7f5cb7e00640 1 --2- 172.21.15.165:0/360087280 >> v2:172.21.15.172:3300/0 conn(0x7f5cc015db10 0x7f5cc0162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.080+0000 7f5cb7e00640 1 -- 172.21.15.165:0/360087280 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc0071670 con 0x7f5cc00b3220 2024-04-25T10:16:40.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.081+0000 7f5cb7e00640 1 --2- 172.21.15.165:0/360087280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc00b3220 0x7f5cc00b3680 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5cc006f500 tx=0x7f5ca802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f15fc8754947dc31 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.081+0000 7f5cbfe00640 1 -- 172.21.15.165:0/360087280 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8037070 con 0x7f5cc00b3220 2024-04-25T10:16:40.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.081+0000 7f5cbfe00640 1 -- 172.21.15.165:0/360087280 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca8032440 con 0x7f5cc00b3220 2024-04-25T10:16:40.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.081+0000 7f5cbfe00640 1 -- 172.21.15.165:0/360087280 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca8032810 con 0x7f5cc00b3220 2024-04-25T10:16:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.081+0000 7f5cc7600640 1 -- 172.21.15.165:0/360087280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc00b3220 msgr2=0x7f5cc00b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.081+0000 7f5cc7600640 1 --2- 172.21.15.165:0/360087280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc00b3220 0x7f5cc00b3680 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f5cc006f500 tx=0x7f5ca802a210 comp rx=0 tx=0).stop 2024-04-25T10:16:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 -- 172.21.15.165:0/360087280 shutdown_connections 2024-04-25T10:16:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 --2- 172.21.15.165:0/360087280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc00cf860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 --2- 172.21.15.165:0/360087280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc00b3220 0x7f5cc00b3680 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 --2- 172.21.15.165:0/360087280 >> v2:172.21.15.172:3300/0 conn(0x7f5cc015db10 0x7f5cc0162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 -- 172.21.15.165:0/360087280 >> 172.21.15.165:0/360087280 conn(0x7f5cc006abe0 msgr2=0x7f5cc006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 -- 172.21.15.165:0/360087280 shutdown_connections 2024-04-25T10:16:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.082+0000 7f5cc7600640 1 -- 172.21.15.165:0/360087280 wait complete. 2024-04-25T10:16:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.083+0000 7f5cc7600640 1 Processor -- start 2024-04-25T10:16:40.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 -- start start 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc00b3220 0x7f5cc01446d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc0144da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 0x7f5cc01452e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0071f10 con 0x7f5cc015db10 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc0071c30 con 0x7f5cc00b3220 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.084+0000 7f5cc7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cc00af0e0 con 0x7f5cc00cf3f0 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 0x7f5cc01452e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 0x7f5cc01452e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43478/0 (socket says 172.21.15.165:43478) 2024-04-25T10:16:40.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 -- 172.21.15.165:0/858701312 learned_addr learned my addr 172.21.15.165:0/858701312 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:40.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cb7e00640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc0144da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc4c00640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc00b3220 0x7f5cc01446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 -- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 msgr2=0x7f5cc0144da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc0144da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 -- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc00b3220 msgr2=0x7f5cc01446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc00b3220 0x7f5cc01446d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 -- 172.21.15.165:0/858701312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc00cf090 con 0x7f5cc015db10 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.085+0000 7f5cc5600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 0x7f5cc01452e0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8007e60 tx=0x7f5cb800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.086+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8014620 con 0x7f5cc015db10 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.086+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb8019070 con 0x7f5cc015db10 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.086+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cc015efd0 con 0x7f5cc015db10 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.086+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb8014a20 con 0x7f5cc015db10 2024-04-25T10:16:40.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.086+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cc015f590 con 0x7f5cc015db10 2024-04-25T10:16:40.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.087+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cc00712b0 con 0x7f5cc015db10 2024-04-25T10:16:40.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.088+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5cb8014bc0 con 0x7f5cc015db10 2024-04-25T10:16:40.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.089+0000 7f5cbd600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.089+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5cb808a910 con 0x7f5cc015db10 2024-04-25T10:16:40.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.089+0000 7f5cc4c00640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.090+0000 7f5cc4c00640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006fd0 tx=0x7f5cb0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.093+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5cc00712b0 con 0x7f5cc015db10 2024-04-25T10:16:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.291+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5cc0073a00 con 0x7f5cc015db10 2024-04-25T10:16:40.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.292+0000 7f5cbd600640 1 -- 172.21.15.165:0/858701312 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v144) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5cb8059940 con 0x7f5cc015db10 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:16:40.296 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c9c06b4d0 msgr2=0x7f5c9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f5cb0006fd0 tx=0x7f5cb0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 msgr2=0x7f5cc01452e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 0x7f5cc01452e0 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f5cb8007e60 tx=0x7f5cb800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 shutdown_connections 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5c9c06b4d0 0x7f5c9c06d990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5cc015db10 0x7f5cc01452e0 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5cc00cf3f0 0x7f5cc0144da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 --2- 172.21.15.165:0/858701312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5cc00b3220 0x7f5cc01446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.295+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 >> 172.21.15.165:0/858701312 conn(0x7f5cc006abe0 msgr2=0x7f5cc006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.296+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 shutdown_connections 2024-04-25T10:16:40.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.296+0000 7f5cc7600640 1 -- 172.21.15.165:0/858701312 wait complete. 2024-04-25T10:16:40.318 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:16:40.318 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:16:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.524+0000 7fc291e00640 1 Processor -- start 2024-04-25T10:16:40.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.524+0000 7fc291e00640 1 -- start start 2024-04-25T10:16:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.526+0000 7fc291e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc28c15b3c0 0x7fc28c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.526+0000 7fc291e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc28c0cc150 0x7fc28c15e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.526+0000 7fc291e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc28c0b0b80 con 0x7fc28c15b7a0 2024-04-25T10:16:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.526+0000 7fc291e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c0cc5e0 con 0x7fc28c0cc150 2024-04-25T10:16:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.526+0000 7fc291e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc28c0cd650 con 0x7fc28c15b3c0 2024-04-25T10:16:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.526+0000 7fc291400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc28c0cc150 0x7fc28c15e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc291400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc28c0cc150 0x7fc28c15e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43480/0 (socket says 172.21.15.165:43480) 2024-04-25T10:16:40.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc291400640 1 -- 172.21.15.165:0/3401629586 learned_addr learned my addr 172.21.15.165:0/3401629586 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc290a00640 1 --2- 172.21.15.165:0/3401629586 >> v2:172.21.15.172:3300/0 conn(0x7fc28c15b3c0 0x7fc28c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc291400640 1 -- 172.21.15.165:0/3401629586 >> v2:172.21.15.172:3300/0 conn(0x7fc28c15b3c0 msgr2=0x7fc28c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc291400640 1 --2- 172.21.15.165:0/3401629586 >> v2:172.21.15.172:3300/0 conn(0x7fc28c15b3c0 0x7fc28c0b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc291400640 1 -- 172.21.15.165:0/3401629586 >> v1:172.21.15.172:6789/0 conn(0x7fc28c15b7a0 legacy=0x7fc28c0cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.527+0000 7fc291400640 1 -- 172.21.15.165:0/3401629586 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c0cc800 con 0x7fc28c0cc150 2024-04-25T10:16:40.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.528+0000 7fc291400640 1 --2- 172.21.15.165:0/3401629586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c0cc150 0x7fc28c15e010 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fc28c06f690 tx=0x7fc28402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d65c1341186671 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc28be00640 1 -- 172.21.15.165:0/3401629586 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284036070 con 0x7fc28c0cc150 2024-04-25T10:16:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc28be00640 1 -- 172.21.15.165:0/3401629586 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284031600 con 0x7fc28c0cc150 2024-04-25T10:16:40.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc28be00640 1 -- 172.21.15.165:0/3401629586 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2840319d0 con 0x7fc28c0cc150 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc291e00640 1 -- 172.21.15.165:0/3401629586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c0cc150 msgr2=0x7fc28c15e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc291e00640 1 --2- 172.21.15.165:0/3401629586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c0cc150 0x7fc28c15e010 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7fc28c06f690 tx=0x7fc28402a660 comp rx=0 tx=0).stop 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc291e00640 1 -- 172.21.15.165:0/3401629586 shutdown_connections 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc291e00640 1 --2- 172.21.15.165:0/3401629586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c0cc150 0x7fc28c15e010 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc291e00640 1 --2- 172.21.15.165:0/3401629586 >> v2:172.21.15.172:3300/0 conn(0x7fc28c15b3c0 0x7fc28c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.529+0000 7fc291e00640 1 -- 172.21.15.165:0/3401629586 >> 172.21.15.165:0/3401629586 conn(0x7fc28c061cc0 msgr2=0x7fc28c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.530+0000 7fc291e00640 1 -- 172.21.15.165:0/3401629586 shutdown_connections 2024-04-25T10:16:40.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.530+0000 7fc291e00640 1 -- 172.21.15.165:0/3401629586 wait complete. 2024-04-25T10:16:40.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.531+0000 7fc291e00640 1 Processor -- start 2024-04-25T10:16:40.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.531+0000 7fc291e00640 1 -- start start 2024-04-25T10:16:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.532+0000 7fc291e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c0cc150 0x7fc28c0a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.532+0000 7fc291e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28c15b3c0 0x7fc28c1e9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.532+0000 7fc291e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 0x7fc28c0ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.532+0000 7fc291e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0ce1f0 con 0x7fc28c15b7a0 2024-04-25T10:16:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.532+0000 7fc291e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0ccdc0 con 0x7fc28c0cc150 2024-04-25T10:16:40.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.532+0000 7fc291e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28c0ce370 con 0x7fc28c15b3c0 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 0x7fc28c0ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 0x7fc28c0ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43488/0 (socket says 172.21.15.165:43488) 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 -- 172.21.15.165:0/634079348 learned_addr learned my addr 172.21.15.165:0/634079348 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc290a00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c0cc150 0x7fc28c0a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 -- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28c15b3c0 msgr2=0x7fc28c1e9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28c15b3c0 0x7fc28c1e9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 -- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c0cc150 msgr2=0x7fc28c0a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c0cc150 0x7fc28c0a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:40.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.533+0000 7fc291400640 1 -- 172.21.15.165:0/634079348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc28c1e85d0 con 0x7fc28c15b7a0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.534+0000 7fc291400640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 0x7fc28c0ca330 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fc284009f40 tx=0x7fc284004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.534+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284036050 con 0x7fc28c15b7a0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.534+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284042070 con 0x7fc28c15b7a0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.534+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284009510 con 0x7fc28c15b7a0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.534+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc28c1e88b0 con 0x7fc28c15b7a0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.534+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc28c1e8e70 con 0x7fc28c15b7a0 2024-04-25T10:16:40.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.535+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc28c073990 con 0x7fc28c15b7a0 2024-04-25T10:16:40.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.536+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc284007c30 con 0x7fc28c15b7a0 2024-04-25T10:16:40.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.537+0000 7fc289600640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc26806b4d0 0x7fc26806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:40.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.537+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc2840aaf70 con 0x7fc28c15b7a0 2024-04-25T10:16:40.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.537+0000 7fc290a00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc26806b4d0 0x7fc26806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:40.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.538+0000 7fc290a00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc26806b4d0 0x7fc26806d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc27c006fd0 tx=0x7fc27c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:40.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.542+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc28c0b0b80 con 0x7fc28c15b7a0 2024-04-25T10:16:40.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:40.736+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc28c072360 con 0x7fc28c15b7a0 2024-04-25T10:16:41.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.509+0000 7fc289600640 1 -- 172.21.15.165:0/634079348 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v145) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc284079fa0 con 0x7fc28c15b7a0 2024-04-25T10:16:41.513 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc26806b4d0 msgr2=0x7fc26806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc26806b4d0 0x7fc26806d990 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fc27c006fd0 tx=0x7fc27c008040 comp rx=0 tx=0).stop 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 msgr2=0x7fc28c0ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 0x7fc28c0ca330 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7fc284009f40 tx=0x7fc284004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 shutdown_connections 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc26806b4d0 0x7fc26806d990 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc28c15b7a0 0x7fc28c0ca330 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc28c15b3c0 0x7fc28c1e9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 --2- 172.21.15.165:0/634079348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc28c0cc150 0x7fc28c0a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.513+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 >> 172.21.15.165:0/634079348 conn(0x7fc28c061cc0 msgr2=0x7fc28c160630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.514+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 shutdown_connections 2024-04-25T10:16:41.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.514+0000 7fc291e00640 1 -- 172.21.15.165:0/634079348 wait complete. 2024-04-25T10:16:41.540 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:16:41.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.732+0000 7fc97f000640 1 Processor -- start 2024-04-25T10:16:41.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.732+0000 7fc97f000640 1 -- start start 2024-04-25T10:16:41.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.733+0000 7fc97f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780ceb20 0x7fc9780c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.733+0000 7fc97f000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9780c5030 0x7fc97815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.733+0000 7fc97f000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc978145520 con 0x7fc978145140 2024-04-25T10:16:41.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.733+0000 7fc97f000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc978071270 con 0x7fc9780c5030 2024-04-25T10:16:41.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.733+0000 7fc97f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc978071550 con 0x7fc9780ceb20 2024-04-25T10:16:41.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97e600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9780c5030 0x7fc97815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97e600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc9780c5030 0x7fc97815bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43498/0 (socket says 172.21.15.165:43498) 2024-04-25T10:16:41.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97e600640 1 -- 172.21.15.165:0/3523804486 learned_addr learned my addr 172.21.15.165:0/3523804486 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:41.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97d200640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780ceb20 0x7fc9780c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97e600640 1 -- 172.21.15.165:0/3523804486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780ceb20 msgr2=0x7fc9780c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:41.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97e600640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780ceb20 0x7fc9780c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.734+0000 7fc97e600640 1 -- 172.21.15.165:0/3523804486 >> v1:172.21.15.172:6789/0 conn(0x7fc978145140 legacy=0x7fc9780ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.735+0000 7fc97e600640 1 -- 172.21.15.165:0/3523804486 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978071830 con 0x7fc9780c5030 2024-04-25T10:16:41.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.735+0000 7fc97d200640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780ceb20 0x7fc9780c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.736+0000 7fc97e600640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780c5030 0x7fc97815bfb0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fc970009e50 tx=0x7fc97002a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d567196171ef3e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.736+0000 7fc977e00640 1 -- 172.21.15.165:0/3523804486 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc970036070 con 0x7fc9780c5030 2024-04-25T10:16:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.736+0000 7fc977e00640 1 -- 172.21.15.165:0/3523804486 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc970031600 con 0x7fc9780c5030 2024-04-25T10:16:41.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.736+0000 7fc977e00640 1 -- 172.21.15.165:0/3523804486 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9700319d0 con 0x7fc9780c5030 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 -- 172.21.15.165:0/3523804486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780c5030 msgr2=0x7fc97815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780c5030 0x7fc97815bfb0 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7fc970009e50 tx=0x7fc97002a680 comp rx=0 tx=0).stop 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 -- 172.21.15.165:0/3523804486 shutdown_connections 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780c5030 0x7fc97815bfb0 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 --2- 172.21.15.165:0/3523804486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780ceb20 0x7fc9780c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 -- 172.21.15.165:0/3523804486 >> 172.21.15.165:0/3523804486 conn(0x7fc978061cc0 msgr2=0x7fc97806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:41.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 -- 172.21.15.165:0/3523804486 shutdown_connections 2024-04-25T10:16:41.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.737+0000 7fc97f000640 1 -- 172.21.15.165:0/3523804486 wait complete. 2024-04-25T10:16:41.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.739+0000 7fc97f000640 1 Processor -- start 2024-04-25T10:16:41.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.739+0000 7fc97f000640 1 -- start start 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780c5030 0x7fc9780acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97f000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 0x7fc9781f1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc978145140 0x7fc9780a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97f000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9780c5640 con 0x7fc9780ceb20 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9780c54c0 con 0x7fc978145140 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97815c4f0 con 0x7fc9780c5030 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 0x7fc9781f1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 0x7fc9781f1ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43514/0 (socket says 172.21.15.165:43514) 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97d200640 1 -- 172.21.15.165:0/3851774096 learned_addr learned my addr 172.21.15.165:0/3851774096 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97dc00640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780c5030 0x7fc9780acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.740+0000 7fc97e600640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc978145140 0x7fc9780a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97d200640 1 -- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780c5030 msgr2=0x7fc9780acc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97d200640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780c5030 0x7fc9780acc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97d200640 1 -- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc978145140 msgr2=0x7fc9780a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:41.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97d200640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc978145140 0x7fc9780a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:41.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97d200640 1 -- 172.21.15.165:0/3851774096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9780a89e0 con 0x7fc9780ceb20 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97d200640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 0x7fc9781f1ac0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fc96400e7e0 tx=0x7fc96400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc96400f040 con 0x7fc9780ceb20 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc964004030 con 0x7fc9780ceb20 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc964004400 con 0x7fc9780ceb20 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.741+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9780a8c80 con 0x7fc9780ceb20 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.742+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9780a9240 con 0x7fc9780ceb20 2024-04-25T10:16:41.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.743+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc978073990 con 0x7fc9780ceb20 2024-04-25T10:16:41.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.744+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc96400c6d0 con 0x7fc9780ceb20 2024-04-25T10:16:41.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.745+0000 7fc975600640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc95806b500 0x7fc95806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:41.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.746+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 5619+0+0 (secure 0 0 0) 0x7fc964089f20 con 0x7fc9780ceb20 2024-04-25T10:16:41.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.746+0000 7fc97dc00640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc95806b500 0x7fc95806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:41.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.746+0000 7fc97dc00640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc95806b500 0x7fc95806d9c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007fd0 tx=0x7fc96c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:41.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.750+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc964017020 con 0x7fc9780ceb20 2024-04-25T10:16:41.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:41.947+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fc9780aa2d0 con 0x7fc9780ceb20 2024-04-25T10:16:42.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.529+0000 7fc975600640 1 -- 172.21.15.165:0/3851774096 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v146) v1 ==== 170+0+0 (secure 0 0 0) 0x7fc964058de0 con 0x7fc9780ceb20 2024-04-25T10:16:42.533 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc95806b500 msgr2=0x7fc95806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc95806b500 0x7fc95806d9c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fc96c007fd0 tx=0x7fc96c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 msgr2=0x7fc9781f1ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 0x7fc9781f1ac0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7fc96400e7e0 tx=0x7fc96400eca0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 shutdown_connections 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc95806b500 0x7fc95806d9c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc978145140 0x7fc9780a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9780ceb20 0x7fc9781f1ac0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 --2- 172.21.15.165:0/3851774096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9780c5030 0x7fc9780acc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 >> 172.21.15.165:0/3851774096 conn(0x7fc978061cc0 msgr2=0x7fc97806eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 shutdown_connections 2024-04-25T10:16:42.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.533+0000 7fc97f000640 1 -- 172.21.15.165:0/3851774096 wait complete. 2024-04-25T10:16:42.567 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:16:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.739+0000 7fa3f4800640 1 Processor -- start 2024-04-25T10:16:42.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.739+0000 7fa3f4800640 1 -- start start 2024-04-25T10:16:42.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.740+0000 7fa3f4800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3ec1446d0 0x7fa3ec0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.740+0000 7fa3f4800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec0cccc0 con 0x7fa3ec144d50 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.740+0000 7fa3f4800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3ec0d0460 con 0x7fa3ec06e8a0 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.740+0000 7fa3f4800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0b2b20 con 0x7fa3ec1446d0 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.740+0000 7fa3f1400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa3ec144d50 0x7fa3ec0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47878/0 (socket says 172.21.15.165:47878) 2024-04-25T10:16:42.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f1400640 1 -- 172.21.15.165:0/11098762 learned_addr learned my addr 172.21.15.165:0/11098762 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f1e00640 1 --2- 172.21.15.165:0/11098762 >> v2:172.21.15.172:3300/0 conn(0x7fa3ec1446d0 0x7fa3ec0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750640235 0 0) 0x7fa3ec0cccc0 con 0x7fa3ec144d50 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3d4003520 con 0x7fa3ec144d50 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 689688679 0 0) 0x7fa3d4003520 con 0x7fa3ec144d50 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 >> v2:172.21.15.172:3300/0 conn(0x7fa3ec1446d0 msgr2=0x7fa3ec0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 --2- 172.21.15.165:0/11098762 >> v2:172.21.15.172:3300/0 conn(0x7fa3ec1446d0 0x7fa3ec0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 >> v1:172.21.15.172:6789/0 conn(0x7fa3ec06e8a0 legacy=0x7fa3ec0cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec0cc140 con 0x7fa3ec144d50 2024-04-25T10:16:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.741+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa3dc001670 con 0x7fa3ec144d50 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 >> v1:172.21.15.165:6789/0 conn(0x7fa3ec144d50 legacy=0x7fa3ec0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3d4004800 0x7fa3d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d40051c0 0x7fa3d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0b2b20 con 0x7fa3d4004800 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d400c060 con 0x7fa3d40039b0 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d400c260 con 0x7fa3d40051c0 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1400640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3d4004800 0x7fa3d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1e00640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1400640 1 -- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d40051c0 msgr2=0x7fa3d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1400640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3d40051c0 0x7fa3d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1400640 1 -- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d40039b0 msgr2=0x7fa3d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1400640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d40039b0 0x7fa3d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.742+0000 7fa3f1400640 1 -- 172.21.15.165:0/11098762 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d4003520 con 0x7fa3d4004800 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.743+0000 7fa3f2800640 1 -- 172.21.15.165:0/11098762 reap_dead start 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.744+0000 7fa3f1400640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3d4004800 0x7fa3d4004c80 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c240 tx=0x7fa3d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d2c3a7150b2c1f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.744+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc007d10 con 0x7fa3d4004800 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.744+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc005ad0 con 0x7fa3d4004800 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.744+0000 7fa3f0a00640 1 -- 172.21.15.165:0/11098762 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc007070 con 0x7fa3d4004800 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.744+0000 7fa3f4800640 1 -- 172.21.15.165:0/11098762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3d4004800 msgr2=0x7fa3d4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.744+0000 7fa3f4800640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3d4004800 0x7fa3d4004c80 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c240 tx=0x7fa3d4005830 comp rx=0 tx=0).stop 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.745+0000 7fa3f4800640 1 -- 172.21.15.165:0/11098762 shutdown_connections 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.745+0000 7fa3f4800640 1 --2- 172.21.15.165:0/11098762 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3d4004800 0x7fa3d4004c80 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.745+0000 7fa3f4800640 1 -- 172.21.15.165:0/11098762 >> 172.21.15.165:0/11098762 conn(0x7fa3ec06abe0 msgr2=0x7fa3ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.745+0000 7fa3f4800640 1 -- 172.21.15.165:0/11098762 shutdown_connections 2024-04-25T10:16:42.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.745+0000 7fa3f4800640 1 -- 172.21.15.165:0/11098762 wait complete. 2024-04-25T10:16:42.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.746+0000 7fa3f4800640 1 Processor -- start 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.747+0000 7fa3f4800640 1 -- start start 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f4800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d4004800 0x7fa3ec13e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f4800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 0x7fa3ec1e9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f4800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3ec1f3a30 0x7fa3ec0ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f4800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0cc700 con 0x7fa3ec13eb40 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f4800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec06d400 con 0x7fa3d4004800 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f4800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0cc9e0 con 0x7fa3ec1f3a30 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 0x7fa3ec1e9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 0x7fa3ec1e9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43532/0 (socket says 172.21.15.165:43532) 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 -- 172.21.15.165:0/2417562675 learned_addr learned my addr 172.21.15.165:0/2417562675 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1e00640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d4004800 0x7fa3ec13e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f2800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3ec1f3a30 0x7fa3ec0ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 -- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3ec1f3a30 msgr2=0x7fa3ec0ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3ec1f3a30 0x7fa3ec0ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 -- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d4004800 msgr2=0x7fa3ec13e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d4004800 0x7fa3ec13e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.748+0000 7fa3f1400640 1 -- 172.21.15.165:0/2417562675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec13efd0 con 0x7fa3ec13eb40 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3f2800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3ec1f3a30 0x7fa3ec0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3f1400640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 0x7fa3ec1e9170 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c240 tx=0x7fa3ec06d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc041070 con 0x7fa3ec13eb40 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3dc004da0 con 0x7fa3ec13eb40 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3dc007d10 con 0x7fa3ec13eb40 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec13f2b0 con 0x7fa3ec13eb40 2024-04-25T10:16:42.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.749+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec13f870 con 0x7fa3ec13eb40 2024-04-25T10:16:42.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.750+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec073a00 con 0x7fa3ec13eb40 2024-04-25T10:16:42.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.752+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa3dc044070 con 0x7fa3ec13eb40 2024-04-25T10:16:42.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.752+0000 7fa3e6000640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa3d006b4d0 0x7fa3d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:42.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.753+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fa3dc0aab60 con 0x7fa3ec13eb40 2024-04-25T10:16:42.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.753+0000 7fa3f1e00640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa3d006b4d0 0x7fa3d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:42.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.753+0000 7fa3f1e00640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa3d006b4d0 0x7fa3d006d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec068c30 tx=0x7fa3e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:42.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.757+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa3dc079a40 con 0x7fa3ec13eb40 2024-04-25T10:16:42.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:42.982+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa3ec0cccc0 con 0x7fa3ec13eb40 2024-04-25T10:16:43.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.546+0000 7fa3e6000640 1 -- 172.21.15.165:0/2417562675 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v98) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa3dc079340 con 0x7fa3ec13eb40 2024-04-25T10:16:43.551 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 21 and data pool 22 2024-04-25T10:16:43.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.554+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa3d006b4d0 msgr2=0x7fa3d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.554+0000 7fa3f4800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa3d006b4d0 0x7fa3d006d990 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec068c30 tx=0x7fa3e0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:43.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 msgr2=0x7fa3ec1e9170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 0x7fa3ec1e9170 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc00c240 tx=0x7fa3ec06d370 comp rx=0 tx=0).stop 2024-04-25T10:16:43.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 shutdown_connections 2024-04-25T10:16:43.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa3d006b4d0 0x7fa3d006d990 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3ec1f3a30 0x7fa3ec0ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3ec13eb40 0x7fa3ec1e9170 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 --2- 172.21.15.165:0/2417562675 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa3d4004800 0x7fa3ec13e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.555+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 >> 172.21.15.165:0/2417562675 conn(0x7fa3ec06abe0 msgr2=0x7fa3ec0af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:43.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.557+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 shutdown_connections 2024-04-25T10:16:43.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.558+0000 7fa3f4800640 1 -- 172.21.15.165:0/2417562675 wait complete. 2024-04-25T10:16:43.587 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:43.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.751+0000 7faa54e00640 1 Processor -- start 2024-04-25T10:16:43.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.751+0000 7faa54e00640 1 -- start start 2024-04-25T10:16:43.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.753+0000 7faa54e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faa5015db70 0x7faa500cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.753+0000 7faa54e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa50071650 con 0x7faa5015df50 2024-04-25T10:16:43.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.753+0000 7faa54e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa500cf230 con 0x7faa500cfc20 2024-04-25T10:16:43.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.753+0000 7faa54e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa500cbd80 con 0x7faa5015db70 2024-04-25T10:16:43.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.753+0000 7faa47e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7faa5015df50 0x7faa500cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47884/0 (socket says 172.21.15.165:47884) 2024-04-25T10:16:43.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.753+0000 7faa47e00640 1 -- 172.21.15.165:0/774974239 learned_addr learned my addr 172.21.15.165:0/774974239 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:43.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3609267739 0 0) 0x7faa38001110 con 0x7faa5015df50 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa34003520 con 0x7faa5015df50 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4de00640 1 --2- 172.21.15.165:0/774974239 >> v2:172.21.15.172:3300/0 conn(0x7faa5015db70 0x7faa500cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2693603309 0 0) 0x7faa34003520 con 0x7faa5015df50 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 >> v2:172.21.15.172:3300/0 conn(0x7faa5015db70 msgr2=0x7faa500cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 --2- 172.21.15.165:0/774974239 >> v2:172.21.15.172:3300/0 conn(0x7faa5015db70 0x7faa500cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 >> v1:172.21.15.172:6789/0 conn(0x7faa500cfc20 legacy=0x7faa5006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa50161b60 con 0x7faa5015df50 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7faa38001840 con 0x7faa5015df50 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 >> v1:172.21.15.165:6789/0 conn(0x7faa5015df50 legacy=0x7faa500cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340051c0 0x7faa3400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa500cbd80 con 0x7faa340039b0 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa3400c110 con 0x7faa340051c0 2024-04-25T10:16:43.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.754+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa3400c310 con 0x7faa34004800 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4de00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa47e00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4de00640 1 -- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa34004800 msgr2=0x7faa34004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4de00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa34004800 0x7faa34004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4de00640 1 -- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340051c0 msgr2=0x7faa3400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4de00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340051c0 0x7faa3400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4de00640 1 -- 172.21.15.165:0/774974239 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa34003520 con 0x7faa340039b0 2024-04-25T10:16:43.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.755+0000 7faa4e800640 1 -- 172.21.15.165:0/774974239 reap_dead start 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.756+0000 7faa4de00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ba30 tx=0x7faa34005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39dbcffd34f9966d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.756+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa40030070 con 0x7faa340039b0 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.756+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa4002ae10 con 0x7faa340039b0 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.756+0000 7faa4d400640 1 -- 172.21.15.165:0/774974239 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa4000b600 con 0x7faa340039b0 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 -- 172.21.15.165:0/774974239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa340039b0 msgr2=0x7faa34003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ba30 tx=0x7faa34005830 comp rx=0 tx=0).stop 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 -- 172.21.15.165:0/774974239 shutdown_connections 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 --2- 172.21.15.165:0/774974239 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa340039b0 0x7faa34003dd0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 -- 172.21.15.165:0/774974239 >> 172.21.15.165:0/774974239 conn(0x7faa5006abe0 msgr2=0x7faa5006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:43.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 -- 172.21.15.165:0/774974239 shutdown_connections 2024-04-25T10:16:43.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.757+0000 7faa54e00640 1 -- 172.21.15.165:0/774974239 wait complete. 2024-04-25T10:16:43.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.759+0000 7faa54e00640 1 Processor -- start 2024-04-25T10:16:43.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.759+0000 7faa54e00640 1 -- start start 2024-04-25T10:16:43.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa54e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340039b0 0x7faa50141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa54e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa501448d0 0x7faa5015e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa54e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 0x7faa5015e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa54e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50162120 con 0x7faa50144d50 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa54e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa50161910 con 0x7faa340039b0 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa54e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa50162640 con 0x7faa501448d0 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 0x7faa5015e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 0x7faa5015e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43560/0 (socket says 172.21.15.165:43560) 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 -- 172.21.15.165:0/517086460 learned_addr learned my addr 172.21.15.165:0/517086460 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4de00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340039b0 0x7faa50141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 -- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa501448d0 msgr2=0x7faa5015e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa501448d0 0x7faa5015e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 -- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340039b0 msgr2=0x7faa50141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340039b0 0x7faa50141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.760+0000 7faa4e800640 1 -- 172.21.15.165:0/517086460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa501424d0 con 0x7faa50144d50 2024-04-25T10:16:43.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.761+0000 7faa4e800640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 0x7faa5015e930 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7faa500ce1f0 tx=0x7faa4800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:43.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.761+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa48017070 con 0x7faa50144d50 2024-04-25T10:16:43.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.761+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa501611c0 con 0x7faa50144d50 2024-04-25T10:16:43.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.761+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa480070d0 con 0x7faa50144d50 2024-04-25T10:16:43.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.761+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa480074c0 con 0x7faa50144d50 2024-04-25T10:16:43.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.761+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa5015b160 con 0x7faa50144d50 2024-04-25T10:16:43.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.762+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa50073990 con 0x7faa50144d50 2024-04-25T10:16:43.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.763+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faa4801a070 con 0x7faa50144d50 2024-04-25T10:16:43.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.764+0000 7faa46000640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faa2c06b4d0 0x7faa2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:43.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.764+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faa48089730 con 0x7faa50144d50 2024-04-25T10:16:43.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.764+0000 7faa4de00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faa2c06b4d0 0x7faa2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:43.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.765+0000 7faa4de00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faa2c06b4d0 0x7faa2c06d990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ba00 tx=0x7faa40030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:43.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.769+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faa48058300 con 0x7faa50144d50 2024-04-25T10:16:43.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.964+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faa500aeaf0 con 0x7faa50144d50 2024-04-25T10:16:43.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.966+0000 7faa46000640 1 -- 172.21.15.165:0/517086460 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16941 (secure 0 0 0) 0x7faa48057c00 con 0x7faa50144d50 2024-04-25T10:16:43.970 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:43.970 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":147,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:42.983510+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:43.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faa2c06b4d0 msgr2=0x7faa2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faa2c06b4d0 0x7faa2c06d990 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7faa4000ba00 tx=0x7faa40030000 comp rx=0 tx=0).stop 2024-04-25T10:16:43.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 msgr2=0x7faa5015e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:43.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 0x7faa5015e930 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7faa500ce1f0 tx=0x7faa4800c980 comp rx=0 tx=0).stop 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 shutdown_connections 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faa2c06b4d0 0x7faa2c06d990 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faa50144d50 0x7faa5015e930 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faa501448d0 0x7faa5015e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 --2- 172.21.15.165:0/517086460 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faa340039b0 0x7faa50141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 >> 172.21.15.165:0/517086460 conn(0x7faa5006abe0 msgr2=0x7faa500acd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 shutdown_connections 2024-04-25T10:16:43.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:43.970+0000 7faa54e00640 1 -- 172.21.15.165:0/517086460 wait complete. 2024-04-25T10:16:43.992 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:44.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.189+0000 7fcc83e00640 1 Processor -- start 2024-04-25T10:16:44.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.189+0000 7fcc83e00640 1 -- start start 2024-04-25T10:16:44.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.190+0000 7fcc83e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc84144440 0x7fcc84070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.190+0000 7fcc83e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8415bf30 con 0x7fcc84071530 2024-04-25T10:16:44.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.190+0000 7fcc83e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc8415c210 con 0x7fcc840cda30 2024-04-25T10:16:44.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.190+0000 7fcc83e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc840719a0 con 0x7fcc84144440 2024-04-25T10:16:44.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc7be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcc84071530 0x7fcc840cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47886/0 (socket says 172.21.15.165:47886) 2024-04-25T10:16:44.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc7be00640 1 -- 172.21.15.165:0/2446980356 learned_addr learned my addr 172.21.15.165:0/2446980356 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:44.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc82a00640 1 --2- 172.21.15.165:0/2446980356 >> v2:172.21.15.165:3300/0 conn(0x7fcc84144440 0x7fcc84070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc82a00640 1 -- 172.21.15.165:0/2446980356 >> v1:172.21.15.172:6789/0 conn(0x7fcc840cda30 legacy=0x7fcc8415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc82a00640 1 -- 172.21.15.165:0/2446980356 >> v1:172.21.15.165:6789/0 conn(0x7fcc84071530 legacy=0x7fcc840cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc82a00640 1 -- 172.21.15.165:0/2446980356 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc8415c4f0 con 0x7fcc84144440 2024-04-25T10:16:44.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.191+0000 7fcc82000640 1 -- 172.21.15.165:0/2446980356 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 852886997 0 0) 0x7fcc8415bf30 con 0x7fcc84071530 2024-04-25T10:16:44.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.192+0000 7fcc82a00640 1 --2- 172.21.15.165:0/2446980356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84144440 0x7fcc84070ff0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcc740097a0 tx=0x7fcc7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24985034051273de server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc82000640 1 -- 172.21.15.165:0/2446980356 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74037070 con 0x7fcc84144440 2024-04-25T10:16:44.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc82000640 1 -- 172.21.15.165:0/2446980356 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc74032440 con 0x7fcc84144440 2024-04-25T10:16:44.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc82000640 1 -- 172.21.15.165:0/2446980356 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74032810 con 0x7fcc84144440 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 -- 172.21.15.165:0/2446980356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84144440 msgr2=0x7fcc84070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 --2- 172.21.15.165:0/2446980356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84144440 0x7fcc84070ff0 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7fcc740097a0 tx=0x7fcc7402a270 comp rx=0 tx=0).stop 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 -- 172.21.15.165:0/2446980356 shutdown_connections 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 --2- 172.21.15.165:0/2446980356 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84144440 0x7fcc84070ff0 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 -- 172.21.15.165:0/2446980356 >> 172.21.15.165:0/2446980356 conn(0x7fcc8406abe0 msgr2=0x7fcc8406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 -- 172.21.15.165:0/2446980356 shutdown_connections 2024-04-25T10:16:44.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.193+0000 7fcc83e00640 1 -- 172.21.15.165:0/2446980356 wait complete. 2024-04-25T10:16:44.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.195+0000 7fcc83e00640 1 Processor -- start 2024-04-25T10:16:44.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.195+0000 7fcc83e00640 1 -- start start 2024-04-25T10:16:44.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc83e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 0x7fcc8413b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc83e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc840cda30 0x7fcc8415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc83e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc84144440 0x7fcc840cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc83e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc840cdea0 con 0x7fcc84071530 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc83e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415c860 con 0x7fcc84144440 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc83e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc8415d400 con 0x7fcc840cda30 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.196+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 0x7fcc8413b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 0x7fcc8413b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43582/0 (socket says 172.21.15.165:43582) 2024-04-25T10:16:44.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 -- 172.21.15.165:0/3518259842 learned_addr learned my addr 172.21.15.165:0/3518259842 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 -- 172.21.15.165:0/3518259842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc840cda30 msgr2=0x7fcc8415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc840cda30 0x7fcc8415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 -- 172.21.15.165:0/3518259842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc84144440 msgr2=0x7fcc840cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc84144440 0x7fcc840cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 -- 172.21.15.165:0/3518259842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc840b2760 con 0x7fcc84071530 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.197+0000 7fcc82a00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 0x7fcc8413b540 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcc740097a0 tx=0x7fcc740046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.198+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74041070 con 0x7fcc84071530 2024-04-25T10:16:44.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.198+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc7403c870 con 0x7fcc84071530 2024-04-25T10:16:44.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.198+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc74037040 con 0x7fcc84071530 2024-04-25T10:16:44.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.198+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc841f4b40 con 0x7fcc84071530 2024-04-25T10:16:44.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.198+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc840b2d00 con 0x7fcc84071530 2024-04-25T10:16:44.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.199+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc84073990 con 0x7fcc84071530 2024-04-25T10:16:44.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.200+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc74044070 con 0x7fcc84071530 2024-04-25T10:16:44.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.201+0000 7fcc7aa00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc6006b4d0 0x7fcc6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.201+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc740ab2e0 con 0x7fcc84071530 2024-04-25T10:16:44.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.201+0000 7fcc7be00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc6006b4d0 0x7fcc6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.202+0000 7fcc7be00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc6006b4d0 0x7fcc6006d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c007fd0 tx=0x7fcc6c007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.205+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc7407a050 con 0x7fcc84071530 2024-04-25T10:16:44.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.397+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc841e2d80 con 0x7fcc84071530 2024-04-25T10:16:44.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.400+0000 7fcc7aa00640 1 -- 172.21.15.165:0/3518259842 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+16941 (secure 0 0 0) 0x7fcc74079950 con 0x7fcc84071530 2024-04-25T10:16:44.404 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:44.404 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":147,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:42.983510+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:44.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc6006b4d0 msgr2=0x7fcc6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc6006b4d0 0x7fcc6006d990 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c007fd0 tx=0x7fcc6c007920 comp rx=0 tx=0).stop 2024-04-25T10:16:44.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 msgr2=0x7fcc8413b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 0x7fcc8413b540 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7fcc740097a0 tx=0x7fcc740046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 shutdown_connections 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc6006b4d0 0x7fcc6006d990 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc84144440 0x7fcc840cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc840cda30 0x7fcc8415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 --2- 172.21.15.165:0/3518259842 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc84071530 0x7fcc8413b540 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 >> 172.21.15.165:0/3518259842 conn(0x7fcc8406abe0 msgr2=0x7fcc84140dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 shutdown_connections 2024-04-25T10:16:44.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.403+0000 7fcc83e00640 1 -- 172.21.15.165:0/3518259842 wait complete. 2024-04-25T10:16:44.426 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:16:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.626+0000 7f97c6000640 1 Processor -- start 2024-04-25T10:16:44.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.626+0000 7f97c6000640 1 -- start start 2024-04-25T10:16:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.628+0000 7f97c6000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f97c00cecb0 0x7f97c0168970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.628+0000 7f97c6000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c006c880 con 0x7f97c00b9260 2024-04-25T10:16:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.628+0000 7f97c6000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97c006cb60 con 0x7f97c015d9c0 2024-04-25T10:16:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.628+0000 7f97c6000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c00d1a30 con 0x7f97c00cecb0 2024-04-25T10:16:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.628+0000 7f97b7e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f97c00b9260 0x7f97c00ce5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47896/0 (socket says 172.21.15.165:47896) 2024-04-25T10:16:44.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.628+0000 7f97b7e00640 1 -- 172.21.15.165:0/4248876004 learned_addr learned my addr 172.21.15.165:0/4248876004 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97c5600640 1 --2- 172.21.15.165:0/4248876004 >> v2:172.21.15.172:3300/0 conn(0x7f97c00cecb0 0x7f97c0168970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277098913 0 0) 0x7f97c006c880 con 0x7f97c00b9260 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a0003520 con 0x7f97c00b9260 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3292427823 0 0) 0x7f97a0003520 con 0x7f97c00b9260 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 >> v2:172.21.15.172:3300/0 conn(0x7f97c00cecb0 msgr2=0x7f97c0168970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 --2- 172.21.15.165:0/4248876004 >> v2:172.21.15.172:3300/0 conn(0x7f97c00cecb0 0x7f97c0168970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 >> v1:172.21.15.172:6789/0 conn(0x7f97c015d9c0 legacy=0x7f97c006c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c006ce40 con 0x7f97c00b9260 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f97b0001670 con 0x7f97c00b9260 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 >> v1:172.21.15.165:6789/0 conn(0x7f97c00b9260 legacy=0x7f97c00ce5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97a0003a00 0x7f97a0003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0005210 0x7f97a000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00d1a30 con 0x7f97a0004800 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a000c150 con 0x7f97a0003a00 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.629+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a000c350 con 0x7f97a0005210 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97b7e00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97c4c00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97a0003a00 0x7f97a0003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97c5600640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0005210 0x7f97a000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97b7e00640 1 -- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0005210 msgr2=0x7f97a000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97b7e00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0005210 0x7f97a000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97b7e00640 1 -- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97a0003a00 msgr2=0x7f97a0003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97b7e00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97a0003a00 0x7f97a0003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97b7e00640 1 -- 172.21.15.165:0/4248876004 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a0003520 con 0x7f97a0004800 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97c5600640 1 -- 172.21.15.165:0/4248876004 reap_dead start 2024-04-25T10:16:44.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.630+0000 7f97c4c00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97a0003a00 0x7f97a0003e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:16:44.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.631+0000 7f97b7e00640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c110 tx=0x7f97a0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aeff063d1767d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.631+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0007c40 con 0x7f97a0004800 2024-04-25T10:16:44.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.631+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b0005ad0 con 0x7f97a0004800 2024-04-25T10:16:44.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.631+0000 7f97bfe00640 1 -- 172.21.15.165:0/4248876004 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0007070 con 0x7f97a0004800 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 -- 172.21.15.165:0/4248876004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a0004800 msgr2=0x7f97a0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c110 tx=0x7f97a0005830 comp rx=0 tx=0).stop 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 -- 172.21.15.165:0/4248876004 shutdown_connections 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 --2- 172.21.15.165:0/4248876004 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97a0004800 0x7f97a0004c80 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 -- 172.21.15.165:0/4248876004 >> 172.21.15.165:0/4248876004 conn(0x7f97c00d0770 msgr2=0x7f97c00d0b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 -- 172.21.15.165:0/4248876004 shutdown_connections 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.632+0000 7f97c6000640 1 -- 172.21.15.165:0/4248876004 wait complete. 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.634+0000 7f97c6000640 1 Processor -- start 2024-04-25T10:16:44.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.634+0000 7f97c6000640 1 -- start start 2024-04-25T10:16:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97c6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0004800 0x7f97c0155010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97c6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 0x7f97c00c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97c6000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97c00c9d20 0x7f97c015fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97c6000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015df20 con 0x7f97c01f4550 2024-04-25T10:16:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97c6000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c015dda0 con 0x7f97c00c9d20 2024-04-25T10:16:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97c6000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c00ce5a0 con 0x7f97a0004800 2024-04-25T10:16:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 0x7f97c00c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97b7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 0x7f97c00c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43602/0 (socket says 172.21.15.165:43602) 2024-04-25T10:16:44.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.635+0000 7f97b7e00640 1 -- 172.21.15.165:0/2930735394 learned_addr learned my addr 172.21.15.165:0/2930735394 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97c4c00640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0004800 0x7f97c0155010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97c5600640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97c00c9d20 0x7f97c015fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97b7e00640 1 -- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0004800 msgr2=0x7f97c0155010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97b7e00640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0004800 0x7f97c0155010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97b7e00640 1 -- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97c00c9d20 msgr2=0x7f97c015fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:44.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97b7e00640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97c00c9d20 0x7f97c015fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97b7e00640 1 -- 172.21.15.165:0/2930735394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00ca1a0 con 0x7f97c01f4550 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.636+0000 7f97b7e00640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 0x7f97c00c97e0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c230 tx=0x7f97c00b96d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.637+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b003f070 con 0x7f97c01f4550 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.637+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b0001370 con 0x7f97c01f4550 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.637+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b0007070 con 0x7f97c01f4550 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.637+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c01606b0 con 0x7f97c01f4550 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.637+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c00c76f0 con 0x7f97c01f4550 2024-04-25T10:16:44.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.638+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c0073990 con 0x7f97c01f4550 2024-04-25T10:16:44.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.639+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97b0042070 con 0x7f97c01f4550 2024-04-25T10:16:44.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.640+0000 7f97bd600640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f979c06b500 0x7f979c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:44.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.640+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97b00aa4d0 con 0x7f97c01f4550 2024-04-25T10:16:44.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.640+0000 7f97c4c00640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f979c06b500 0x7f979c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:44.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.641+0000 7f97c4c00640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f979c06b500 0x7f979c06d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f97c0050820 tx=0x7f97a8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:44.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.645+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f97b0049020 con 0x7f97c01f4550 2024-04-25T10:16:44.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:44.881+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f97c00d15e0 con 0x7f97c01f4550 2024-04-25T10:16:45.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.532+0000 7f97bd600640 1 -- 172.21.15.165:0/2930735394 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v100) v1 ==== 118+0+0 (secure 0 0 0) 0x7f97b0078bf0 con 0x7f97c01f4550 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f979c06b500 msgr2=0x7f979c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f979c06b500 0x7f979c06d9c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f97c0050820 tx=0x7f97a8009040 comp rx=0 tx=0).stop 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 msgr2=0x7f97c00c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 0x7f97c00c97e0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f97b000c230 tx=0x7f97c00b96d0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 shutdown_connections 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f979c06b500 0x7f979c06d9c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97c00c9d20 0x7f97c015fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97c01f4550 0x7f97c00c97e0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 --2- 172.21.15.165:0/2930735394 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97a0004800 0x7f97c0155010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.536+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 >> 172.21.15.165:0/2930735394 conn(0x7f97c00d0770 msgr2=0x7f97c00b6a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:45.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.537+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 shutdown_connections 2024-04-25T10:16:45.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.537+0000 7f97c6000640 1 -- 172.21.15.165:0/2930735394 wait complete. 2024-04-25T10:16:45.563 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:16:45.563 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:16:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.770+0000 7f9a24a00640 1 Processor -- start 2024-04-25T10:16:45.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.770+0000 7f9a24a00640 1 -- start start 2024-04-25T10:16:45.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.771+0000 7f9a24a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a2013d9f0 0x7f9a2013df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.771+0000 7f9a24a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2006d390 con 0x7f9a200b0730 2024-04-25T10:16:45.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.771+0000 7f9a24a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a2006d670 con 0x7f9a2006e860 2024-04-25T10:16:45.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.771+0000 7f9a24a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a2006d950 con 0x7f9a2013d9f0 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.772+0000 7f9a16a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9a200b0730 0x7f9a2006e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47898/0 (socket says 172.21.15.165:47898) 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.772+0000 7f9a16a00640 1 -- 172.21.15.165:0/494522501 learned_addr learned my addr 172.21.15.165:0/494522501 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.772+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2714732221 0 0) 0x7f9a2006d390 con 0x7f9a200b0730 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.772+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a04003520 con 0x7f9a200b0730 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.772+0000 7f9a1f400640 1 --2- 172.21.15.165:0/494522501 >> v2:172.21.15.172:3300/0 conn(0x7f9a2013d9f0 0x7f9a2013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.772+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2250522831 0 0) 0x7f9a04003520 con 0x7f9a200b0730 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 >> v2:172.21.15.172:3300/0 conn(0x7f9a2013d9f0 msgr2=0x7f9a2013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 --2- 172.21.15.165:0/494522501 >> v2:172.21.15.172:3300/0 conn(0x7f9a2013d9f0 0x7f9a2013df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 >> v1:172.21.15.172:6789/0 conn(0x7f9a2006e860 legacy=0x7f9a200ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2006dc30 con 0x7f9a200b0730 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9a08001670 con 0x7f9a200b0730 2024-04-25T10:16:45.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 >> v1:172.21.15.165:6789/0 conn(0x7f9a200b0730 legacy=0x7f9a2006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a04003a00 0x7f9a04003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a04004800 0x7f9a04004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040053c0 0x7f9a0400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2006d950 con 0x7f9a04004800 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0400c2a0 con 0x7f9a04003a00 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0400c4a0 con 0x7f9a040053c0 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a16a00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a04004800 0x7f9a04004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.773+0000 7f9a1f400640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a04003a00 0x7f9a04003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.774+0000 7f9a16a00640 1 -- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040053c0 msgr2=0x7f9a0400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.774+0000 7f9a16a00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040053c0 0x7f9a0400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.774+0000 7f9a16a00640 1 -- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a04003a00 msgr2=0x7f9a04003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.774+0000 7f9a16a00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a04003a00 0x7f9a04003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.774+0000 7f9a16a00640 1 -- 172.21.15.165:0/494522501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04003520 con 0x7f9a04004800 2024-04-25T10:16:45.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.774+0000 7f9a1fe00640 1 -- 172.21.15.165:0/494522501 reap_dead start 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a16a00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a04004800 0x7f9a04004c80 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c110 tx=0x7f9a04005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e5771e0848d2d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08007c40 con 0x7f9a04004800 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08005af0 con 0x7f9a04004800 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a1ea00640 1 -- 172.21.15.165:0/494522501 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08007070 con 0x7f9a04004800 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a24a00640 1 -- 172.21.15.165:0/494522501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a04004800 msgr2=0x7f9a04004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a24a00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a04004800 0x7f9a04004c80 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f9a0800c110 tx=0x7f9a04005830 comp rx=0 tx=0).stop 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.775+0000 7f9a24a00640 1 -- 172.21.15.165:0/494522501 shutdown_connections 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.776+0000 7f9a24a00640 1 --2- 172.21.15.165:0/494522501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a04004800 0x7f9a04004c80 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.776+0000 7f9a24a00640 1 -- 172.21.15.165:0/494522501 >> 172.21.15.165:0/494522501 conn(0x7f9a20061cc0 msgr2=0x7f9a2006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.776+0000 7f9a24a00640 1 -- 172.21.15.165:0/494522501 shutdown_connections 2024-04-25T10:16:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.776+0000 7f9a24a00640 1 -- 172.21.15.165:0/494522501 wait complete. 2024-04-25T10:16:45.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.778+0000 7f9a24a00640 1 Processor -- start 2024-04-25T10:16:45.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.778+0000 7f9a24a00640 1 -- start start 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a24a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a04004800 0x7f9a2013ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a24a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 0x7f9a20158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a24a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a201554e0 0x7f9a20155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a24a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200b0f50 con 0x7f9a20157ca0 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a24a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a200b0dd0 con 0x7f9a201554e0 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a24a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2013c630 con 0x7f9a04004800 2024-04-25T10:16:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a16a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 0x7f9a20158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a16a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 0x7f9a20158120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43608/0 (socket says 172.21.15.165:43608) 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.779+0000 7f9a16a00640 1 -- 172.21.15.165:0/3501240628 learned_addr learned my addr 172.21.15.165:0/3501240628 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a1f400640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a04004800 0x7f9a2013ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a1fe00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a201554e0 0x7f9a20155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a16a00640 1 -- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a04004800 msgr2=0x7f9a2013ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a16a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a04004800 0x7f9a2013ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a16a00640 1 -- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a201554e0 msgr2=0x7f9a20155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a16a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a201554e0 0x7f9a20155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a16a00640 1 -- 172.21.15.165:0/3501240628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a2013b590 con 0x7f9a20157ca0 2024-04-25T10:16:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a16a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 0x7f9a20158120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9a0802ad60 tx=0x7f9a200ce7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08037040 con 0x7f9a20157ca0 2024-04-25T10:16:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.780+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a08041070 con 0x7f9a20157ca0 2024-04-25T10:16:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.781+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a2013b870 con 0x7f9a20157ca0 2024-04-25T10:16:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.781+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a08001670 con 0x7f9a20157ca0 2024-04-25T10:16:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.781+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a2013be30 con 0x7f9a20157ca0 2024-04-25T10:16:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.782+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a20073990 con 0x7f9a20157ca0 2024-04-25T10:16:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.783+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a08004a20 con 0x7f9a20157ca0 2024-04-25T10:16:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.784+0000 7f9a17e00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99fc06b4d0 0x7f99fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.784+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a080aa080 con 0x7f9a20157ca0 2024-04-25T10:16:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.784+0000 7f9a1f400640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99fc06b4d0 0x7f99fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.785+0000 7f9a1f400640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99fc06b4d0 0x7f99fc06d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9a10006fd0 tx=0x7f9a10008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:45.788+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a08005e20 con 0x7f9a20157ca0 2024-04-25T10:16:46.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.026+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9a200c53b0 con 0x7f9a20157ca0 2024-04-25T10:16:46.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.554+0000 7f9a17e00640 1 -- 172.21.15.165:0/3501240628 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v101) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9a08078c40 con 0x7f9a20157ca0 2024-04-25T10:16:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.557+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99fc06b4d0 msgr2=0x7f99fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.557+0000 7f9a24a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99fc06b4d0 0x7f99fc06d990 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9a10006fd0 tx=0x7f9a10008040 comp rx=0 tx=0).stop 2024-04-25T10:16:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.557+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 msgr2=0x7f9a20158120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.557+0000 7f9a24a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 0x7f9a20158120 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f9a0802ad60 tx=0x7f9a200ce7a0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 shutdown_connections 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99fc06b4d0 0x7f99fc06d990 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a201554e0 0x7f9a20155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a20157ca0 0x7f9a20158120 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 --2- 172.21.15.165:0/3501240628 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a04004800 0x7f9a2013ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 >> 172.21.15.165:0/3501240628 conn(0x7f9a20061cc0 msgr2=0x7f9a20158ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 shutdown_connections 2024-04-25T10:16:46.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.558+0000 7f9a24a00640 1 -- 172.21.15.165:0/3501240628 wait complete. 2024-04-25T10:16:46.583 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:16:46.583 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:46.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.785+0000 7f2974e00640 1 Processor -- start 2024-04-25T10:16:46.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.785+0000 7f2974e00640 1 -- start start 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.786+0000 7f2974e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29700b93c0 0x7f29700cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.786+0000 7f2974e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f29700d04f0 0x7f29700d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.786+0000 7f2974e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2970161df0 con 0x7f297015db90 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.786+0000 7f2974e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29700d1f50 con 0x7f29700b93c0 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.786+0000 7f2974e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29700d0940 con 0x7f29700d04f0 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f296f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f297015db90 0x7f29701615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:47908/0 (socket says 172.21.15.165:47908) 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f296f400640 1 -- 172.21.15.165:0/4282311785 learned_addr learned my addr 172.21.15.165:0/4282311785 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f2966a00640 1 --2- 172.21.15.165:0/4282311785 >> v2:172.21.15.165:3300/0 conn(0x7f29700b93c0 0x7f29700cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f296fe00640 1 --2- 172.21.15.165:0/4282311785 >> v2:172.21.15.172:3300/0 conn(0x7f29700d04f0 0x7f29700d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f2966a00640 1 -- 172.21.15.165:0/4282311785 >> v2:172.21.15.172:3300/0 conn(0x7f29700d04f0 msgr2=0x7f29700d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:46.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f2966a00640 1 --2- 172.21.15.165:0/4282311785 >> v2:172.21.15.172:3300/0 conn(0x7f29700d04f0 0x7f29700d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f2966a00640 1 -- 172.21.15.165:0/4282311785 >> v1:172.21.15.165:6789/0 conn(0x7f297015db90 legacy=0x7f29701615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f2966a00640 1 -- 172.21.15.165:0/4282311785 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29701620d0 con 0x7f29700b93c0 2024-04-25T10:16:46.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f296ea00640 1 -- 172.21.15.165:0/4282311785 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3210453729 0 0) 0x7f2970161df0 con 0x7f297015db90 2024-04-25T10:16:46.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.787+0000 7f296fe00640 1 --2- 172.21.15.165:0/4282311785 >> v2:172.21.15.172:3300/0 conn(0x7f29700d04f0 0x7f29700d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.788+0000 7f2966a00640 1 --2- 172.21.15.165:0/4282311785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29700b93c0 0x7f29700cffb0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f297008cc30 tx=0x7f295802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=597cd1fd3d8ace2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.789+0000 7f296ea00640 1 -- 172.21.15.165:0/4282311785 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958037070 con 0x7f29700b93c0 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.789+0000 7f296ea00640 1 -- 172.21.15.165:0/4282311785 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2958032440 con 0x7f29700b93c0 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.789+0000 7f296ea00640 1 -- 172.21.15.165:0/4282311785 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2958032810 con 0x7f29700b93c0 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.789+0000 7f2974e00640 1 -- 172.21.15.165:0/4282311785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29700b93c0 msgr2=0x7f29700cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.789+0000 7f2974e00640 1 --2- 172.21.15.165:0/4282311785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29700b93c0 0x7f29700cffb0 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f297008cc30 tx=0x7f295802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.790+0000 7f2974e00640 1 -- 172.21.15.165:0/4282311785 shutdown_connections 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.790+0000 7f2974e00640 1 --2- 172.21.15.165:0/4282311785 >> v2:172.21.15.172:3300/0 conn(0x7f29700d04f0 0x7f29700d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.790+0000 7f2974e00640 1 --2- 172.21.15.165:0/4282311785 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29700b93c0 0x7f29700cffb0 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.790+0000 7f2974e00640 1 -- 172.21.15.165:0/4282311785 >> 172.21.15.165:0/4282311785 conn(0x7f29700b52b0 msgr2=0x7f29700b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:46.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.790+0000 7f2974e00640 1 -- 172.21.15.165:0/4282311785 shutdown_connections 2024-04-25T10:16:46.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.790+0000 7f2974e00640 1 -- 172.21.15.165:0/4282311785 wait complete. 2024-04-25T10:16:46.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.791+0000 7f2974e00640 1 Processor -- start 2024-04-25T10:16:46.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 -- start start 2024-04-25T10:16:46.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29700b93c0 0x7f297014eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29700d04f0 0x7f297015b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 0x7f29700cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700b68d0 con 0x7f297015db90 2024-04-25T10:16:46.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29700d1be0 con 0x7f29700d04f0 2024-04-25T10:16:46.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.792+0000 7f2974e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29700b6a50 con 0x7f29700b93c0 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 0x7f29700cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 0x7f29700cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:43626/0 (socket says 172.21.15.165:43626) 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 -- 172.21.15.165:0/3817575014 learned_addr learned my addr 172.21.15.165:0/3817575014 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 -- 172.21.15.165:0/3817575014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29700b93c0 msgr2=0x7f297014eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29700b93c0 0x7f297014eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 -- 172.21.15.165:0/3817575014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29700d04f0 msgr2=0x7f297015b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29700d04f0 0x7f297015b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:46.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 -- 172.21.15.165:0/3817575014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29701599a0 con 0x7f297015db90 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.793+0000 7f296fe00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 0x7f29700cc880 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f296800ce40 tx=0x7f296800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.794+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296801a070 con 0x7f297015db90 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.794+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f296800d840 con 0x7f297015db90 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.794+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f296800dc30 con 0x7f297015db90 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.794+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2970159c80 con 0x7f297015db90 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.794+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f297015a240 con 0x7f297015db90 2024-04-25T10:16:46.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.795+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2970073990 con 0x7f297015db90 2024-04-25T10:16:46.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.796+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f296801d070 con 0x7f297015db90 2024-04-25T10:16:46.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.797+0000 7f2967e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f294c06b4d0 0x7f294c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:46.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.797+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f296808ac60 con 0x7f297015db90 2024-04-25T10:16:46.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.797+0000 7f296f400640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f294c06b4d0 0x7f294c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:46.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.798+0000 7f296f400640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f294c06b4d0 0x7f294c06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2960007fd0 tx=0x7f2960007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:46.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:46.801+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29680598b0 con 0x7f297015db90 2024-04-25T10:16:47.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.032+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f29700d14b0 con 0x7f297015db90 2024-04-25T10:16:47.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.033+0000 7f2967e00640 1 -- 172.21.15.165:0/3817575014 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4956 (secure 0 0 0) 0x7f29680591b0 con 0x7f297015db90 2024-04-25T10:16:47.037 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:47.037 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":101,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":9762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:16:46.028556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:16:47.037 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 101 2024-04-25T10:16:47.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f294c06b4d0 msgr2=0x7f294c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f294c06b4d0 0x7f294c06d990 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f2960007fd0 tx=0x7f2960007960 comp rx=0 tx=0).stop 2024-04-25T10:16:47.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 msgr2=0x7f29700cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 0x7f29700cc880 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7f296800ce40 tx=0x7f296800a710 comp rx=0 tx=0).stop 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 shutdown_connections 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f294c06b4d0 0x7f294c06d990 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f297015db90 0x7f29700cc880 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29700d04f0 0x7f297015b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 --2- 172.21.15.165:0/3817575014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29700b93c0 0x7f297014eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 >> 172.21.15.165:0/3817575014 conn(0x7f29700b52b0 msgr2=0x7f29700ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 shutdown_connections 2024-04-25T10:16:47.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.036+0000 7f2974e00640 1 -- 172.21.15.165:0/3817575014 wait complete. 2024-04-25T10:16:47.060 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:16:47.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.259+0000 7f29fa600640 1 Processor -- start 2024-04-25T10:16:47.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.259+0000 7f29fa600640 1 -- start start 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29fa600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29f415c100 0x7f29f4071740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29fa600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f29f406ec90 0x7f29f406d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29fa600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29f40b3420 con 0x7f29f40b3800 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29fa600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f40b2810 con 0x7f29f415c100 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29fa600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29f40b2af0 con 0x7f29f406ec90 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29f9200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29f415c100 0x7f29f4071740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29f9200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f29f415c100 0x7f29f4071740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52668/0 (socket says 172.21.15.165:52668) 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.261+0000 7f29f9200640 1 -- 172.21.15.165:0/3233980257 learned_addr learned my addr 172.21.15.165:0/3233980257 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.262+0000 7f29f9c00640 1 --2- 172.21.15.165:0/3233980257 >> v2:172.21.15.172:3300/0 conn(0x7f29f406ec90 0x7f29f406d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.262+0000 7f29f9200640 1 -- 172.21.15.165:0/3233980257 >> v2:172.21.15.172:3300/0 conn(0x7f29f406ec90 msgr2=0x7f29f406d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.262+0000 7f29f9200640 1 --2- 172.21.15.165:0/3233980257 >> v2:172.21.15.172:3300/0 conn(0x7f29f406ec90 0x7f29f406d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.262+0000 7f29f9200640 1 -- 172.21.15.165:0/3233980257 >> v1:172.21.15.172:6789/0 conn(0x7f29f40b3800 legacy=0x7f29f406e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.262+0000 7f29f9200640 1 -- 172.21.15.165:0/3233980257 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f4071c80 con 0x7f29f415c100 2024-04-25T10:16:47.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.263+0000 7f29f9200640 1 --2- 172.21.15.165:0/3233980257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f415c100 0x7f29f4071740 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009870 tx=0x7f29e802a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff07eecb4781010f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29f3400640 1 -- 172.21.15.165:0/3233980257 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8037070 con 0x7f29f415c100 2024-04-25T10:16:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29f3400640 1 -- 172.21.15.165:0/3233980257 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8032440 con 0x7f29f415c100 2024-04-25T10:16:47.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29f3400640 1 -- 172.21.15.165:0/3233980257 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8032810 con 0x7f29f415c100 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 -- 172.21.15.165:0/3233980257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f415c100 msgr2=0x7f29f4071740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 --2- 172.21.15.165:0/3233980257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f415c100 0x7f29f4071740 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009870 tx=0x7f29e802a1c0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 -- 172.21.15.165:0/3233980257 shutdown_connections 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 --2- 172.21.15.165:0/3233980257 >> v2:172.21.15.172:3300/0 conn(0x7f29f406ec90 0x7f29f406d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 --2- 172.21.15.165:0/3233980257 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f415c100 0x7f29f4071740 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 -- 172.21.15.165:0/3233980257 >> 172.21.15.165:0/3233980257 conn(0x7f29f406abe0 msgr2=0x7f29f406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.264+0000 7f29fa600640 1 -- 172.21.15.165:0/3233980257 shutdown_connections 2024-04-25T10:16:47.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.265+0000 7f29fa600640 1 -- 172.21.15.165:0/3233980257 wait complete. 2024-04-25T10:16:47.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.266+0000 7f29fa600640 1 Processor -- start 2024-04-25T10:16:47.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.266+0000 7f29fa600640 1 -- start start 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.267+0000 7f29fa600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 0x7f29f41f8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.267+0000 7f29fa600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29f40b3800 0x7f29f4160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.267+0000 7f29fa600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29f415c100 0x7f29f4157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.267+0000 7f29fa600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40b24e0 con 0x7f29f406ec90 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.267+0000 7f29fa600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40b30b0 con 0x7f29f40b3800 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.267+0000 7f29fa600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29f40711b0 con 0x7f29f415c100 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 0x7f29f41f8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 0x7f29f41f8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52674/0 (socket says 172.21.15.165:52674) 2024-04-25T10:16:47.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 -- 172.21.15.165:0/86844401 learned_addr learned my addr 172.21.15.165:0/86844401 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 -- 172.21.15.165:0/86844401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29f415c100 msgr2=0x7f29f4157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29f415c100 0x7f29f4157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 -- 172.21.15.165:0/86844401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29f40b3800 msgr2=0x7f29f4160ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29f40b3800 0x7f29f4160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.268+0000 7f29f9200640 1 -- 172.21.15.165:0/86844401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29f41613e0 con 0x7f29f406ec90 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.269+0000 7f29f9200640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 0x7f29f41f8490 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009870 tx=0x7f29e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.269+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e8037040 con 0x7f29f406ec90 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.269+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29e8004230 con 0x7f29f406ec90 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.269+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29f415ce90 con 0x7f29f406ec90 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.269+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29e803c670 con 0x7f29f406ec90 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.269+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29f415d730 con 0x7f29f406ec90 2024-04-25T10:16:47.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.270+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29f4073990 con 0x7f29f406ec90 2024-04-25T10:16:47.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.271+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f29e803a070 con 0x7f29f406ec90 2024-04-25T10:16:47.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.272+0000 7f29f0c00640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29d406b4d0 0x7f29d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.272+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f29e80aad00 con 0x7f29f406ec90 2024-04-25T10:16:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.273+0000 7f29f3e00640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29d406b4d0 0x7f29d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.273+0000 7f29f3e00640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29d406b4d0 0x7f29d406d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f29f406e920 tx=0x7f29e00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.277+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f29e8041020 con 0x7f29f406ec90 2024-04-25T10:16:47.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.476+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29f406b060 con 0x7f29f406ec90 2024-04-25T10:16:47.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.478+0000 7f29f0c00640 1 -- 172.21.15.165:0/86844401 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f29e8079950 con 0x7f29f406ec90 2024-04-25T10:16:47.481 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:47.482 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":148,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:44.518826+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29d406b4d0 msgr2=0x7f29d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29d406b4d0 0x7f29d406d990 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f29f406e920 tx=0x7f29e00079b0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 msgr2=0x7f29f41f8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 0x7f29f41f8490 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7f29e8009870 tx=0x7f29e80046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 shutdown_connections 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f29d406b4d0 0x7f29d406d990 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f29f415c100 0x7f29f4157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f29f40b3800 0x7f29f4160ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 --2- 172.21.15.165:0/86844401 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f29f406ec90 0x7f29f41f8490 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 >> 172.21.15.165:0/86844401 conn(0x7f29f406abe0 msgr2=0x7f29f40affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 shutdown_connections 2024-04-25T10:16:47.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.481+0000 7f29fa600640 1 -- 172.21.15.165:0/86844401 wait complete. 2024-04-25T10:16:47.504 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:16:47.504 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:47.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.695+0000 7fc935000640 1 Processor -- start 2024-04-25T10:16:47.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.695+0000 7fc935000640 1 -- start start 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.696+0000 7fc935000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc930164970 0x7fc930166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.696+0000 7fc935000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc930144b30 con 0x7fc930144750 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc935000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9300712d0 con 0x7fc9300ce920 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc935000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9300715b0 con 0x7fc930164970 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc930144750 0x7fc93006e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50632/0 (socket says 172.21.15.165:50632) 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92de00640 1 -- 172.21.15.165:0/192943342 learned_addr learned my addr 172.21.15.165:0/192943342 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:47.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92e800640 1 --2- 172.21.15.165:0/192943342 >> v2:172.21.15.165:3300/0 conn(0x7fc930164970 0x7fc930166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92d400640 1 -- 172.21.15.165:0/192943342 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4025771108 0 0) 0x7fc930144b30 con 0x7fc930144750 2024-04-25T10:16:47.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92d400640 1 -- 172.21.15.165:0/192943342 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc914003520 con 0x7fc930144750 2024-04-25T10:16:47.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92e800640 1 -- 172.21.15.165:0/192943342 >> v1:172.21.15.172:6789/0 conn(0x7fc9300ce920 legacy=0x7fc9300cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:47.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92e800640 1 -- 172.21.15.165:0/192943342 >> v1:172.21.15.165:6789/0 conn(0x7fc930144750 legacy=0x7fc93006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.697+0000 7fc92e800640 1 -- 172.21.15.165:0/192943342 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc930071890 con 0x7fc930164970 2024-04-25T10:16:47.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.698+0000 7fc92e800640 1 --2- 172.21.15.165:0/192943342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc930166d60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92802a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47bfc7cc60ae7529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc92d400640 1 -- 172.21.15.165:0/192943342 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928031600 con 0x7fc930164970 2024-04-25T10:16:47.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc92d400640 1 -- 172.21.15.165:0/192943342 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9280317a0 con 0x7fc930164970 2024-04-25T10:16:47.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc92d400640 1 -- 172.21.15.165:0/192943342 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928031b70 con 0x7fc930164970 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc935000640 1 -- 172.21.15.165:0/192943342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 msgr2=0x7fc930166d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc935000640 1 --2- 172.21.15.165:0/192943342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc930166d60 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc92802a610 comp rx=0 tx=0).stop 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc935000640 1 -- 172.21.15.165:0/192943342 shutdown_connections 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc935000640 1 --2- 172.21.15.165:0/192943342 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc930166d60 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.699+0000 7fc935000640 1 -- 172.21.15.165:0/192943342 >> 172.21.15.165:0/192943342 conn(0x7fc93006abe0 msgr2=0x7fc93006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.700+0000 7fc935000640 1 -- 172.21.15.165:0/192943342 shutdown_connections 2024-04-25T10:16:47.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.700+0000 7fc935000640 1 -- 172.21.15.165:0/192943342 wait complete. 2024-04-25T10:16:47.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.701+0000 7fc935000640 1 Processor -- start 2024-04-25T10:16:47.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.701+0000 7fc935000640 1 -- start start 2024-04-25T10:16:47.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc935000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9300ce920 0x7fc9300b1460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc935000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc930144750 0x7fc93015b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc935000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc93013ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc935000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc930145450 con 0x7fc930164970 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc935000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc93006f020 con 0x7fc930144750 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc935000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9301455d0 con 0x7fc9300ce920 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.702+0000 7fc92e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc93013ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc93013ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52692/0 (socket says 172.21.15.165:52692) 2024-04-25T10:16:47.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 -- 172.21.15.165:0/589328148 learned_addr learned my addr 172.21.15.165:0/589328148 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc927e00640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc930144750 0x7fc93015b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 -- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9300ce920 msgr2=0x7fc9300b1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9300ce920 0x7fc9300b1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 -- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc930144750 msgr2=0x7fc93015b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc930144750 0x7fc93015b490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 -- 172.21.15.165:0/589328148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9300af7f0 con 0x7fc930164970 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.703+0000 7fc92e800640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc93013ed80 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc928004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.704+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928040070 con 0x7fc930164970 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.704+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc928030b60 con 0x7fc930164970 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.704+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9300afad0 con 0x7fc930164970 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.704+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc928009450 con 0x7fc930164970 2024-04-25T10:16:47.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.704+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93015caf0 con 0x7fc930164970 2024-04-25T10:16:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.705+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9300afca0 con 0x7fc930164970 2024-04-25T10:16:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.706+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc928039040 con 0x7fc930164970 2024-04-25T10:16:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.707+0000 7fc926000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc90c06b4d0 0x7fc90c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.707+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc9280ab790 con 0x7fc930164970 2024-04-25T10:16:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.707+0000 7fc92de00640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc90c06b4d0 0x7fc90c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.708+0000 7fc92de00640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc90c06b4d0 0x7fc90c06d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc920000f60 tx=0x7fc9200078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.711+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc92804be20 con 0x7fc930164970 2024-04-25T10:16:47.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.908+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc93015ccf0 con 0x7fc90c06b4d0 2024-04-25T10:16:47.912 INFO:tasks.ceph.mds.c.smithi165.stderr:2024-04-25T10:16:47.908+0000 7f7cf1000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:16:47.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.917+0000 7fc926000640 1 -- 172.21.15.165:0/589328148 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240297 (secure 0 0 0) 0x7fc93015ccf0 con 0x7fc90c06b4d0 2024-04-25T10:16:47.925 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:47.926 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.923+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc90c06b4d0 msgr2=0x7fc90c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.923+0000 7fc935000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc90c06b4d0 0x7fc90c06d990 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fc920000f60 tx=0x7fc9200078f0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.923+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 msgr2=0x7fc93013ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.923+0000 7fc935000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc93013ed80 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7fc928009d90 tx=0x7fc928004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 shutdown_connections 2024-04-25T10:16:47.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc90c06b4d0 0x7fc90c06d990 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc930164970 0x7fc93013ed80 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc930144750 0x7fc93015b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 --2- 172.21.15.165:0/589328148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc9300ce920 0x7fc9300b1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 >> 172.21.15.165:0/589328148 conn(0x7fc93006abe0 msgr2=0x7fc930159930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 shutdown_connections 2024-04-25T10:16:47.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:47.924+0000 7fc935000640 1 -- 172.21.15.165:0/589328148 wait complete. 2024-04-25T10:16:47.945 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":588,"stamp":"2024-04-25T10:16:46.429655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514532,"kb_used_data":15332,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754460188,"statfs":{"total":773094113280,"available":772567232512,"internally_reserved":0,"allocated":15699968,"data_stored":7974021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.916960"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771483+0000","last_change":"2024-04-25T10:16:41.771483+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771483+0000","last_undegraded":"2024-04-25T10:16:41.771483+0000","last_fullsized":"2024-04-25T10:16:41.771483+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.529372+0000","last_change":"2024-04-25T10:16:41.529372+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.529372+0000","last_undegraded":"2024-04-25T10:16:41.529372+0000","last_fullsized":"2024-04-25T10:16:41.529372+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527978+0000","last_change":"2024-04-25T10:16:41.527978+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527978+0000","last_undegraded":"2024-04-25T10:16:41.527978+0000","last_fullsized":"2024-04-25T10:16:41.527978+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.515512+0000","last_change":"2024-04-25T10:16:41.515512+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.515512+0000","last_undegraded":"2024-04-25T10:16:41.515512+0000","last_fullsized":"2024-04-25T10:16:41.515512+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.520010+0000","last_change":"2024-04-25T10:16:41.520010+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.520010+0000","last_undegraded":"2024-04-25T10:16:41.520010+0000","last_fullsized":"2024-04-25T10:16:41.520010+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771391+0000","last_change":"2024-04-25T10:16:41.771391+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771391+0000","last_undegraded":"2024-04-25T10:16:41.771391+0000","last_fullsized":"2024-04-25T10:16:41.771391+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.533726+0000","last_change":"2024-04-25T10:16:41.533726+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.533726+0000","last_undegraded":"2024-04-25T10:16:41.533726+0000","last_fullsized":"2024-04-25T10:16:41.533726+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771742+0000","last_change":"2024-04-25T10:16:41.771742+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771742+0000","last_undegraded":"2024-04-25T10:16:41.771742+0000","last_fullsized":"2024-04-25T10:16:41.771742+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527761+0000","last_change":"2024-04-25T10:16:41.527761+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527761+0000","last_undegraded":"2024-04-25T10:16:41.527761+0000","last_fullsized":"2024-04-25T10:16:41.527761+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532558+0000","last_change":"2024-04-25T10:16:41.532558+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532558+0000","last_undegraded":"2024-04-25T10:16:41.532558+0000","last_fullsized":"2024-04-25T10:16:41.532558+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.523221+0000","last_change":"2024-04-25T10:16:41.523221+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.523221+0000","last_undegraded":"2024-04-25T10:16:41.523221+0000","last_fullsized":"2024-04-25T10:16:41.523221+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771737+0000","last_change":"2024-04-25T10:16:41.771737+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771737+0000","last_undegraded":"2024-04-25T10:16:41.771737+0000","last_fullsized":"2024-04-25T10:16:41.771737+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518116+0000","last_change":"2024-04-25T10:16:41.518116+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518116+0000","last_undegraded":"2024-04-25T10:16:41.518116+0000","last_fullsized":"2024-04-25T10:16:41.518116+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518416+0000","last_change":"2024-04-25T10:16:41.518416+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518416+0000","last_undegraded":"2024-04-25T10:16:41.518416+0000","last_fullsized":"2024-04-25T10:16:41.518416+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532462+0000","last_change":"2024-04-25T10:16:41.532462+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532462+0000","last_undegraded":"2024-04-25T10:16:41.532462+0000","last_fullsized":"2024-04-25T10:16:41.532462+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.523312+0000","last_change":"2024-04-25T10:16:41.523312+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.523312+0000","last_undegraded":"2024-04-25T10:16:41.523312+0000","last_fullsized":"2024-04-25T10:16:41.523312+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527903+0000","last_change":"2024-04-25T10:16:41.527903+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527903+0000","last_undegraded":"2024-04-25T10:16:41.527903+0000","last_fullsized":"2024-04-25T10:16:41.527903+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.516348+0000","last_change":"2024-04-25T10:16:41.516348+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.516348+0000","last_undegraded":"2024-04-25T10:16:41.516348+0000","last_fullsized":"2024-04-25T10:16:41.516348+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518027+0000","last_change":"2024-04-25T10:16:41.518027+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518027+0000","last_undegraded":"2024-04-25T10:16:41.518027+0000","last_fullsized":"2024-04-25T10:16:41.518027+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.522514+0000","last_change":"2024-04-25T10:16:41.522514+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.522514+0000","last_undegraded":"2024-04-25T10:16:41.522514+0000","last_fullsized":"2024-04-25T10:16:41.522514+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.519283+0000","last_change":"2024-04-25T10:16:41.519283+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.519283+0000","last_undegraded":"2024-04-25T10:16:41.519283+0000","last_fullsized":"2024-04-25T10:16:41.519283+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.522470+0000","last_change":"2024-04-25T10:16:41.522470+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.522470+0000","last_undegraded":"2024-04-25T10:16:41.522470+0000","last_fullsized":"2024-04-25T10:16:41.522470+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.519328+0000","last_change":"2024-04-25T10:16:41.519328+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.519328+0000","last_undegraded":"2024-04-25T10:16:41.519328+0000","last_fullsized":"2024-04-25T10:16:41.519328+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518380+0000","last_change":"2024-04-25T10:16:41.518380+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518380+0000","last_undegraded":"2024-04-25T10:16:41.518380+0000","last_fullsized":"2024-04-25T10:16:41.518380+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.533665+0000","last_change":"2024-04-25T10:16:41.533665+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.533665+0000","last_undegraded":"2024-04-25T10:16:41.533665+0000","last_fullsized":"2024-04-25T10:16:41.533665+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.515619+0000","last_change":"2024-04-25T10:16:41.515619+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.515619+0000","last_undegraded":"2024-04-25T10:16:41.515619+0000","last_fullsized":"2024-04-25T10:16:41.515619+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771499+0000","last_change":"2024-04-25T10:16:41.771499+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771499+0000","last_undegraded":"2024-04-25T10:16:41.771499+0000","last_fullsized":"2024-04-25T10:16:41.771499+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532414+0000","last_change":"2024-04-25T10:16:41.532414+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532414+0000","last_undegraded":"2024-04-25T10:16:41.532414+0000","last_fullsized":"2024-04-25T10:16:41.532414+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.524128+0000","last_change":"2024-04-25T10:16:41.524128+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.524128+0000","last_undegraded":"2024-04-25T10:16:41.524128+0000","last_fullsized":"2024-04-25T10:16:41.524128+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532462+0000","last_change":"2024-04-25T10:16:41.532462+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532462+0000","last_undegraded":"2024-04-25T10:16:41.532462+0000","last_fullsized":"2024-04-25T10:16:41.532462+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527919+0000","last_change":"2024-04-25T10:16:41.527919+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527919+0000","last_undegraded":"2024-04-25T10:16:41.527919+0000","last_fullsized":"2024-04-25T10:16:41.527919+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.522480+0000","last_change":"2024-04-25T10:16:41.522480+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.522480+0000","last_undegraded":"2024-04-25T10:16:41.522480+0000","last_fullsized":"2024-04-25T10:16:41.522480+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":408,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:16:41.530080+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:16:41.530080+0000","last_peered":"2024-04-25T10:16:41.530080+0000","last_clean":"2024-04-25T10:16:41.530080+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:16:41.530080+0000","last_undegraded":"2024-04-25T10:16:41.530080+0000","last_fullsized":"2024-04-25T10:16:41.530080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59904,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311936,"statfs":{"total":96636764160,"available":96575422464,"internally_reserved":0,"allocated":2400256,"data_stored":1435648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575030,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82876,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288964,"statfs":{"total":96636764160,"available":96551899136,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82876,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288964,"statfs":{"total":96636764160,"available":96551899136,"internally_reserved":0,"allocated":1806336,"data_stored":845280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83032519},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59324,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312516,"statfs":{"total":96636764160,"available":96576016384,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55160,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316680,"statfs":{"total":96636764160,"available":96580280320,"internally_reserved":0,"allocated":1806336,"data_stored":845280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64064,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307776,"statfs":{"total":96636764160,"available":96571162624,"internally_reserved":0,"allocated":2412544,"data_stored":1441857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:47.986 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:48.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.181+0000 7f3e77200640 1 Processor -- start 2024-04-25T10:16:48.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.181+0000 7f3e77200640 1 -- start start 2024-04-25T10:16:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e77200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3e70144440 0x7f3e70070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e77200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3e70071530 0x7f3e700cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e77200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e7015bf30 con 0x7f3e700cd860 2024-04-25T10:16:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e77200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e700719a0 con 0x7f3e70144440 2024-04-25T10:16:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e77200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e7015c860 con 0x7f3e70071530 2024-04-25T10:16:48.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e75e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3e70144440 0x7f3e70070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e76800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3e700cd860 0x7f3e7006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50634/0 (socket says 172.21.15.165:50634) 2024-04-25T10:16:48.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e76800640 1 -- 172.21.15.165:0/3434211200 learned_addr learned my addr 172.21.15.165:0/3434211200 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:48.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.183+0000 7f3e75400640 1 --2- 172.21.15.165:0/3434211200 >> v2:172.21.15.172:3300/0 conn(0x7f3e70071530 0x7f3e700cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.184+0000 7f3e75e00640 1 -- 172.21.15.165:0/3434211200 >> v2:172.21.15.172:3300/0 conn(0x7f3e70071530 msgr2=0x7f3e700cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.184+0000 7f3e75e00640 1 --2- 172.21.15.165:0/3434211200 >> v2:172.21.15.172:3300/0 conn(0x7f3e70071530 0x7f3e700cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.184+0000 7f3e75e00640 1 -- 172.21.15.165:0/3434211200 >> v1:172.21.15.165:6789/0 conn(0x7f3e700cd860 legacy=0x7f3e7006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.184+0000 7f3e75e00640 1 -- 172.21.15.165:0/3434211200 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7015c210 con 0x7f3e70144440 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.184+0000 7f3e74a00640 1 -- 172.21.15.165:0/3434211200 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645078686 0 0) 0x7f3e7015bf30 con 0x7f3e700cd860 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.185+0000 7f3e75e00640 1 --2- 172.21.15.165:0/3434211200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e70070ff0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f3e7006b970 tx=0x7f3e6402a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=325e360144df17e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.185+0000 7f3e74a00640 1 -- 172.21.15.165:0/3434211200 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64037070 con 0x7f3e70144440 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.185+0000 7f3e74a00640 1 -- 172.21.15.165:0/3434211200 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e64032440 con 0x7f3e70144440 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.185+0000 7f3e74a00640 1 -- 172.21.15.165:0/3434211200 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e64032810 con 0x7f3e70144440 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 -- 172.21.15.165:0/3434211200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 msgr2=0x7f3e70070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 --2- 172.21.15.165:0/3434211200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e70070ff0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f3e7006b970 tx=0x7f3e6402a250 comp rx=0 tx=0).stop 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 -- 172.21.15.165:0/3434211200 shutdown_connections 2024-04-25T10:16:48.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 --2- 172.21.15.165:0/3434211200 >> v2:172.21.15.172:3300/0 conn(0x7f3e70071530 0x7f3e700cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 --2- 172.21.15.165:0/3434211200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e70070ff0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 -- 172.21.15.165:0/3434211200 >> 172.21.15.165:0/3434211200 conn(0x7f3e7006abe0 msgr2=0x7f3e7006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 -- 172.21.15.165:0/3434211200 shutdown_connections 2024-04-25T10:16:48.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.186+0000 7f3e77200640 1 -- 172.21.15.165:0/3434211200 wait complete. 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.188+0000 7f3e77200640 1 Processor -- start 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.188+0000 7f3e77200640 1 -- start start 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e77200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e70071530 0x7f3e701f61b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e77200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e700cd860 0x7f3e701e40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e77200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e701e45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e77200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700cdcd0 con 0x7f3e70144440 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e77200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7015bc30 con 0x7f3e700cd860 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e77200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e700cde50 con 0x7f3e70071530 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e701e45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e701e45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52720/0 (socket says 172.21.15.165:52720) 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 -- 172.21.15.165:0/2114326278 learned_addr learned my addr 172.21.15.165:0/2114326278 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e75400640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e700cd860 0x7f3e701e40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e75e00640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e70071530 0x7f3e701f61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 -- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e70071530 msgr2=0x7f3e701f61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e70071530 0x7f3e701f61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 -- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e700cd860 msgr2=0x7f3e701e40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e700cd860 0x7f3e701e40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.189+0000 7f3e76800640 1 -- 172.21.15.165:0/2114326278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e701f66f0 con 0x7f3e70144440 2024-04-25T10:16:48.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.190+0000 7f3e76800640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e701e45f0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c7a0 tx=0x7f3e6c00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.190+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c018070 con 0x7f3e70144440 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.190+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e6c00a640 con 0x7f3e70144440 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.190+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e6c00aab0 con 0x7f3e70144440 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.190+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e700ca310 con 0x7f3e70144440 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.190+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e700ca8d0 con 0x7f3e70144440 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.192+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e70073990 con 0x7f3e70144440 2024-04-25T10:16:48.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.192+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3e6c01b070 con 0x7f3e70144440 2024-04-25T10:16:48.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.193+0000 7f3e6a000640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3e5406b4d0 0x7f3e5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.193+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3e6c089910 con 0x7f3e70144440 2024-04-25T10:16:48.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.194+0000 7f3e75e00640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3e5406b4d0 0x7f3e5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.194+0000 7f3e75e00640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3e5406b4d0 0x7f3e5406d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3e64034000 tx=0x7f3e6402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.197+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3e6c058690 con 0x7f3e70144440 2024-04-25T10:16:48.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.387+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e7006eaa0 con 0x7f3e5406b4d0 2024-04-25T10:16:48.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.400+0000 7f3e6a000640 1 -- 172.21.15.165:0/2114326278 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+240297 (secure 0 0 0) 0x7f3e7006eaa0 con 0x7f3e5406b4d0 2024-04-25T10:16:48.406 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:48.407 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.404+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3e5406b4d0 msgr2=0x7f3e5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.404+0000 7f3e77200640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3e5406b4d0 0x7f3e5406d990 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f3e64034000 tx=0x7f3e6402e000 comp rx=0 tx=0).stop 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.404+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 msgr2=0x7f3e701e45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.404+0000 7f3e77200640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e701e45f0 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c00c7a0 tx=0x7f3e6c00cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 shutdown_connections 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3e5406b4d0 0x7f3e5406d990 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3e70144440 0x7f3e701e45f0 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3e700cd860 0x7f3e701e40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 --2- 172.21.15.165:0/2114326278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3e70071530 0x7f3e701f61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 >> 172.21.15.165:0/2114326278 conn(0x7f3e7006abe0 msgr2=0x7f3e700b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 shutdown_connections 2024-04-25T10:16:48.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.405+0000 7f3e77200640 1 -- 172.21.15.165:0/2114326278 wait complete. 2024-04-25T10:16:48.427 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":588,"stamp":"2024-04-25T10:16:46.429655+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":514532,"kb_used_data":15332,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754460188,"statfs":{"total":773094113280,"available":772567232512,"internally_reserved":0,"allocated":15699968,"data_stored":7974021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.916960"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771483+0000","last_change":"2024-04-25T10:16:41.771483+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771483+0000","last_undegraded":"2024-04-25T10:16:41.771483+0000","last_fullsized":"2024-04-25T10:16:41.771483+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.529372+0000","last_change":"2024-04-25T10:16:41.529372+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.529372+0000","last_undegraded":"2024-04-25T10:16:41.529372+0000","last_fullsized":"2024-04-25T10:16:41.529372+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527978+0000","last_change":"2024-04-25T10:16:41.527978+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527978+0000","last_undegraded":"2024-04-25T10:16:41.527978+0000","last_fullsized":"2024-04-25T10:16:41.527978+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.515512+0000","last_change":"2024-04-25T10:16:41.515512+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.515512+0000","last_undegraded":"2024-04-25T10:16:41.515512+0000","last_fullsized":"2024-04-25T10:16:41.515512+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.520010+0000","last_change":"2024-04-25T10:16:41.520010+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.520010+0000","last_undegraded":"2024-04-25T10:16:41.520010+0000","last_fullsized":"2024-04-25T10:16:41.520010+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771391+0000","last_change":"2024-04-25T10:16:41.771391+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771391+0000","last_undegraded":"2024-04-25T10:16:41.771391+0000","last_fullsized":"2024-04-25T10:16:41.771391+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.533726+0000","last_change":"2024-04-25T10:16:41.533726+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.533726+0000","last_undegraded":"2024-04-25T10:16:41.533726+0000","last_fullsized":"2024-04-25T10:16:41.533726+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771742+0000","last_change":"2024-04-25T10:16:41.771742+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771742+0000","last_undegraded":"2024-04-25T10:16:41.771742+0000","last_fullsized":"2024-04-25T10:16:41.771742+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527761+0000","last_change":"2024-04-25T10:16:41.527761+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527761+0000","last_undegraded":"2024-04-25T10:16:41.527761+0000","last_fullsized":"2024-04-25T10:16:41.527761+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532558+0000","last_change":"2024-04-25T10:16:41.532558+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532558+0000","last_undegraded":"2024-04-25T10:16:41.532558+0000","last_fullsized":"2024-04-25T10:16:41.532558+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.523221+0000","last_change":"2024-04-25T10:16:41.523221+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.523221+0000","last_undegraded":"2024-04-25T10:16:41.523221+0000","last_fullsized":"2024-04-25T10:16:41.523221+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771737+0000","last_change":"2024-04-25T10:16:41.771737+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771737+0000","last_undegraded":"2024-04-25T10:16:41.771737+0000","last_fullsized":"2024-04-25T10:16:41.771737+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518116+0000","last_change":"2024-04-25T10:16:41.518116+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518116+0000","last_undegraded":"2024-04-25T10:16:41.518116+0000","last_fullsized":"2024-04-25T10:16:41.518116+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518416+0000","last_change":"2024-04-25T10:16:41.518416+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518416+0000","last_undegraded":"2024-04-25T10:16:41.518416+0000","last_fullsized":"2024-04-25T10:16:41.518416+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532462+0000","last_change":"2024-04-25T10:16:41.532462+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532462+0000","last_undegraded":"2024-04-25T10:16:41.532462+0000","last_fullsized":"2024-04-25T10:16:41.532462+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.523312+0000","last_change":"2024-04-25T10:16:41.523312+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.523312+0000","last_undegraded":"2024-04-25T10:16:41.523312+0000","last_fullsized":"2024-04-25T10:16:41.523312+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527903+0000","last_change":"2024-04-25T10:16:41.527903+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527903+0000","last_undegraded":"2024-04-25T10:16:41.527903+0000","last_fullsized":"2024-04-25T10:16:41.527903+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.516348+0000","last_change":"2024-04-25T10:16:41.516348+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.516348+0000","last_undegraded":"2024-04-25T10:16:41.516348+0000","last_fullsized":"2024-04-25T10:16:41.516348+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518027+0000","last_change":"2024-04-25T10:16:41.518027+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518027+0000","last_undegraded":"2024-04-25T10:16:41.518027+0000","last_fullsized":"2024-04-25T10:16:41.518027+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.522514+0000","last_change":"2024-04-25T10:16:41.522514+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.522514+0000","last_undegraded":"2024-04-25T10:16:41.522514+0000","last_fullsized":"2024-04-25T10:16:41.522514+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.519283+0000","last_change":"2024-04-25T10:16:41.519283+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.519283+0000","last_undegraded":"2024-04-25T10:16:41.519283+0000","last_fullsized":"2024-04-25T10:16:41.519283+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.522470+0000","last_change":"2024-04-25T10:16:41.522470+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.522470+0000","last_undegraded":"2024-04-25T10:16:41.522470+0000","last_fullsized":"2024-04-25T10:16:41.522470+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.519328+0000","last_change":"2024-04-25T10:16:41.519328+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.519328+0000","last_undegraded":"2024-04-25T10:16:41.519328+0000","last_fullsized":"2024-04-25T10:16:41.519328+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.518380+0000","last_change":"2024-04-25T10:16:41.518380+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.518380+0000","last_undegraded":"2024-04-25T10:16:41.518380+0000","last_fullsized":"2024-04-25T10:16:41.518380+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.533665+0000","last_change":"2024-04-25T10:16:41.533665+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.533665+0000","last_undegraded":"2024-04-25T10:16:41.533665+0000","last_fullsized":"2024-04-25T10:16:41.533665+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.515619+0000","last_change":"2024-04-25T10:16:41.515619+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.515619+0000","last_undegraded":"2024-04-25T10:16:41.515619+0000","last_fullsized":"2024-04-25T10:16:41.515619+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.771499+0000","last_change":"2024-04-25T10:16:41.771499+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.771499+0000","last_undegraded":"2024-04-25T10:16:41.771499+0000","last_fullsized":"2024-04-25T10:16:41.771499+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532414+0000","last_change":"2024-04-25T10:16:41.532414+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532414+0000","last_undegraded":"2024-04-25T10:16:41.532414+0000","last_fullsized":"2024-04-25T10:16:41.532414+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.524128+0000","last_change":"2024-04-25T10:16:41.524128+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.524128+0000","last_undegraded":"2024-04-25T10:16:41.524128+0000","last_fullsized":"2024-04-25T10:16:41.524128+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.532462+0000","last_change":"2024-04-25T10:16:41.532462+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.532462+0000","last_undegraded":"2024-04-25T10:16:41.532462+0000","last_fullsized":"2024-04-25T10:16:41.532462+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.527919+0000","last_change":"2024-04-25T10:16:41.527919+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.527919+0000","last_undegraded":"2024-04-25T10:16:41.527919+0000","last_fullsized":"2024-04-25T10:16:41.527919+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-04-25T10:16:42.510632+0000","last_change":"2024-04-25T10:16:42.510632+0000","last_active":"2024-04-25T10:16:42.510632+0000","last_peered":"2024-04-25T10:16:42.510632+0000","last_clean":"2024-04-25T10:16:42.510632+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:42.510632+0000","last_undegraded":"2024-04-25T10:16:42.510632+0000","last_fullsized":"2024-04-25T10:16:42.510632+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":1,"reported_epoch":145,"state":"creating+peering","last_fresh":"2024-04-25T10:16:41.522480+0000","last_change":"2024-04-25T10:16:41.522480+0000","last_active":"2024-04-25T10:16:41.494096+0000","last_peered":"2024-04-25T10:16:41.494096+0000","last_clean":"2024-04-25T10:16:41.494096+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:16:41.522480+0000","last_undegraded":"2024-04-25T10:16:41.522480+0000","last_fullsized":"2024-04-25T10:16:41.522480+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":408,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-04-25T10:16:41.530080+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:16:41.530080+0000","last_peered":"2024-04-25T10:16:41.530080+0000","last_clean":"2024-04-25T10:16:41.530080+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:16:41.530080+0000","last_undegraded":"2024-04-25T10:16:41.530080+0000","last_fullsized":"2024-04-25T10:16:41.530080+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59904,"kb_used_data":2344,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311936,"statfs":{"total":96636764160,"available":96575422464,"internally_reserved":0,"allocated":2400256,"data_stored":1435648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575030,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55164,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316676,"statfs":{"total":96636764160,"available":96580276224,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82876,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288964,"statfs":{"total":96636764160,"available":96551899136,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82876,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288964,"statfs":{"total":96636764160,"available":96551899136,"internally_reserved":0,"allocated":1806336,"data_stored":845280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83032519},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59324,"kb_used_data":1776,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312516,"statfs":{"total":96636764160,"available":96576016384,"internally_reserved":0,"allocated":1818624,"data_stored":851489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55160,"kb_used_data":1764,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316680,"statfs":{"total":96636764160,"available":96580280320,"internally_reserved":0,"allocated":1806336,"data_stored":845280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64064,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307776,"statfs":{"total":96636764160,"available":96571162624,"internally_reserved":0,"allocated":2412544,"data_stored":1441857,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:48.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.666+0000 7fc0a0200640 1 Processor -- start 2024-04-25T10:16:48.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.666+0000 7fc0a0200640 1 -- start start 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.667+0000 7fc0a0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980cfec0 0x7fc0980cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.667+0000 7fc0a0200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc0980ce3c0 0x7fc09806d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc0a0200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc098071480 con 0x7fc098154ee0 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc0a0200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc098072090 con 0x7fc0980ce3c0 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc0a0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc098071230 con 0x7fc0980cfec0 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09d800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc098154ee0 0x7fc0980cf7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50648/0 (socket says 172.21.15.165:50648) 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09d800640 1 -- 172.21.15.165:0/2066417360 learned_addr learned my addr 172.21.15.165:0/2066417360 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:48.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09e200640 1 --2- 172.21.15.165:0/2066417360 >> v2:172.21.15.165:3300/0 conn(0x7fc0980ce3c0 0x7fc09806d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc097e00640 1 -- 172.21.15.165:0/2066417360 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768831952 0 0) 0x7fc098071480 con 0x7fc098154ee0 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09ce00640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980cfec0 0x7fc0980cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc097e00640 1 -- 172.21.15.165:0/2066417360 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc080005730 con 0x7fc098154ee0 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09e200640 1 -- 172.21.15.165:0/2066417360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980cfec0 msgr2=0x7fc0980cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:48.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09e200640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980cfec0 0x7fc0980cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09e200640 1 -- 172.21.15.165:0/2066417360 >> v1:172.21.15.165:6789/0 conn(0x7fc098154ee0 legacy=0x7fc0980cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09e200640 1 -- 172.21.15.165:0/2066417360 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc098071760 con 0x7fc0980ce3c0 2024-04-25T10:16:48.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.668+0000 7fc09ce00640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980cfec0 0x7fc0980cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.669+0000 7fc09e200640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0980ce3c0 0x7fc09806d6f0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fc090009d90 tx=0x7fc09002a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cab3253443f60d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.670+0000 7fc097e00640 1 -- 172.21.15.165:0/2066417360 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090007740 con 0x7fc0980ce3c0 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.670+0000 7fc097e00640 1 -- 172.21.15.165:0/2066417360 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0900078e0 con 0x7fc0980ce3c0 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.670+0000 7fc097e00640 1 -- 172.21.15.165:0/2066417360 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090007cb0 con 0x7fc0980ce3c0 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.670+0000 7fc0a0200640 1 -- 172.21.15.165:0/2066417360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0980ce3c0 msgr2=0x7fc09806d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.670+0000 7fc0a0200640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0980ce3c0 0x7fc09806d6f0 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7fc090009d90 tx=0x7fc09002a660 comp rx=0 tx=0).stop 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.671+0000 7fc0a0200640 1 -- 172.21.15.165:0/2066417360 shutdown_connections 2024-04-25T10:16:48.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.671+0000 7fc0a0200640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc0980ce3c0 0x7fc09806d6f0 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.671+0000 7fc0a0200640 1 --2- 172.21.15.165:0/2066417360 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980cfec0 0x7fc0980cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.671+0000 7fc0a0200640 1 -- 172.21.15.165:0/2066417360 >> 172.21.15.165:0/2066417360 conn(0x7fc09806abe0 msgr2=0x7fc09806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.671+0000 7fc0a0200640 1 -- 172.21.15.165:0/2066417360 shutdown_connections 2024-04-25T10:16:48.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.671+0000 7fc0a0200640 1 -- 172.21.15.165:0/2066417360 wait complete. 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.672+0000 7fc0a0200640 1 Processor -- start 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.672+0000 7fc0a0200640 1 -- start start 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.673+0000 7fc0a0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980ce3c0 0x7fc0980add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.673+0000 7fc0a0200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0980cfec0 0x7fc098159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.673+0000 7fc0a0200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 0x7fc0981f3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.673+0000 7fc0a0200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc098071d20 con 0x7fc098154ee0 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.673+0000 7fc0a0200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0980ce800 con 0x7fc0980cfec0 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.673+0000 7fc0a0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0981552c0 con 0x7fc0980ce3c0 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 0x7fc0981f3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 0x7fc0981f3750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52736/0 (socket says 172.21.15.165:52736) 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 -- 172.21.15.165:0/348350801 learned_addr learned my addr 172.21.15.165:0/348350801 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09ce00640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0980cfec0 0x7fc098159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09d800640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980ce3c0 0x7fc0980add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 -- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980ce3c0 msgr2=0x7fc0980add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980ce3c0 0x7fc0980add30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 -- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0980cfec0 msgr2=0x7fc098159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0980cfec0 0x7fc098159c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 -- 172.21.15.165:0/348350801 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0980b04b0 con 0x7fc098154ee0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09ce00640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0980cfec0 0x7fc098159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.674+0000 7fc09e200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 0x7fc0981f3750 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fc090009eb0 tx=0x7fc090004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.675+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090040070 con 0x7fc098154ee0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.675+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc090031bc0 con 0x7fc098154ee0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.675+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc090009450 con 0x7fc098154ee0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.675+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0980a9ba0 con 0x7fc098154ee0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.675+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0980aa3e0 con 0x7fc098154ee0 2024-04-25T10:16:48.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.676+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc098073990 con 0x7fc098154ee0 2024-04-25T10:16:48.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.677+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc090039040 con 0x7fc098154ee0 2024-04-25T10:16:48.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.678+0000 7fc095600640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc07806b4d0 0x7fc07806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:48.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.678+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc0900abc70 con 0x7fc098154ee0 2024-04-25T10:16:48.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.678+0000 7fc09d800640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc07806b4d0 0x7fc07806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:48.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.679+0000 7fc09d800640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc07806b4d0 0x7fc07806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000f60 tx=0x7fc08c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:48.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.682+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc0980a9ba0 con 0x7fc098154ee0 2024-04-25T10:16:48.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.873+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc09806d360 con 0x7fc07806b4d0 2024-04-25T10:16:48.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.890+0000 7fc095600640 1 -- 172.21.15.165:0/348350801 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7fc09806d360 con 0x7fc07806b4d0 2024-04-25T10:16:48.898 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:48.899 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc07806b4d0 msgr2=0x7fc07806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc07806b4d0 0x7fc07806d990 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fc08c000f60 tx=0x7fc08c009040 comp rx=0 tx=0).stop 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 msgr2=0x7fc0981f3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 0x7fc0981f3750 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7fc090009eb0 tx=0x7fc090004a40 comp rx=0 tx=0).stop 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 shutdown_connections 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc07806b4d0 0x7fc07806d990 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc098154ee0 0x7fc0981f3750 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc0980cfec0 0x7fc098159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 --2- 172.21.15.165:0/348350801 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc0980ce3c0 0x7fc0980add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 >> 172.21.15.165:0/348350801 conn(0x7fc09806abe0 msgr2=0x7fc09806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 shutdown_connections 2024-04-25T10:16:48.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:48.897+0000 7fc0a0200640 1 -- 172.21.15.165:0/348350801 wait complete. 2024-04-25T10:16:48.919 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2024-04-25T10:16:48.430234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498284,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":482419,"kb_avail":754476436,"statfs":{"total":773094113280,"available":772583870464,"internally_reserved":0,"allocated":16097280,"data_stored":8161676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":493997618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001884"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562120+0000","last_change":"2024-04-25T10:16:43.558546+0000","last_active":"2024-04-25T10:16:44.562120+0000","last_peered":"2024-04-25T10:16:44.562120+0000","last_clean":"2024-04-25T10:16:44.562120+0000","last_became_active":"2024-04-25T10:16:43.557592+0000","last_became_peered":"2024-04-25T10:16:43.557592+0000","last_unstale":"2024-04-25T10:16:44.562120+0000","last_undegraded":"2024-04-25T10:16:44.562120+0000","last_fullsized":"2024-04-25T10:16:44.562120+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.763212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644312+0000","last_change":"2024-04-25T10:16:43.542077+0000","last_active":"2024-04-25T10:16:44.644312+0000","last_peered":"2024-04-25T10:16:44.644312+0000","last_clean":"2024-04-25T10:16:44.644312+0000","last_became_active":"2024-04-25T10:16:43.541331+0000","last_became_peered":"2024-04-25T10:16:43.541331+0000","last_unstale":"2024-04-25T10:16:44.644312+0000","last_undegraded":"2024-04-25T10:16:44.644312+0000","last_fullsized":"2024-04-25T10:16:44.644312+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:18.077257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644302+0000","last_change":"2024-04-25T10:16:43.554057+0000","last_active":"2024-04-25T10:16:44.644302+0000","last_peered":"2024-04-25T10:16:44.644302+0000","last_clean":"2024-04-25T10:16:44.644302+0000","last_became_active":"2024-04-25T10:16:43.553688+0000","last_became_peered":"2024-04-25T10:16:43.553688+0000","last_unstale":"2024-04-25T10:16:44.644302+0000","last_undegraded":"2024-04-25T10:16:44.644302+0000","last_fullsized":"2024-04-25T10:16:44.644302+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:23.042495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536340+0000","last_change":"2024-04-25T10:16:43.553869+0000","last_active":"2024-04-25T10:16:44.536340+0000","last_peered":"2024-04-25T10:16:44.536340+0000","last_clean":"2024-04-25T10:16:44.536340+0000","last_became_active":"2024-04-25T10:16:43.552593+0000","last_became_peered":"2024-04-25T10:16:43.552593+0000","last_unstale":"2024-04-25T10:16:44.536340+0000","last_undegraded":"2024-04-25T10:16:44.536340+0000","last_fullsized":"2024-04-25T10:16:44.536340+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:38.269376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.260200+0000","last_change":"2024-04-25T10:16:43.558696+0000","last_active":"2024-04-25T10:16:45.260200+0000","last_peered":"2024-04-25T10:16:45.260200+0000","last_clean":"2024-04-25T10:16:45.260200+0000","last_became_active":"2024-04-25T10:16:43.554924+0000","last_became_peered":"2024-04-25T10:16:43.554924+0000","last_unstale":"2024-04-25T10:16:45.260200+0000","last_undegraded":"2024-04-25T10:16:45.260200+0000","last_fullsized":"2024-04-25T10:16:45.260200+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:27.953723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259886+0000","last_change":"2024-04-25T10:16:43.553964+0000","last_active":"2024-04-25T10:16:45.259886+0000","last_peered":"2024-04-25T10:16:45.259886+0000","last_clean":"2024-04-25T10:16:45.259886+0000","last_became_active":"2024-04-25T10:16:43.553667+0000","last_became_peered":"2024-04-25T10:16:43.553667+0000","last_unstale":"2024-04-25T10:16:45.259886+0000","last_undegraded":"2024-04-25T10:16:45.259886+0000","last_fullsized":"2024-04-25T10:16:45.259886+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:48.946817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561293+0000","last_change":"2024-04-25T10:16:43.554632+0000","last_active":"2024-04-25T10:16:44.561293+0000","last_peered":"2024-04-25T10:16:44.561293+0000","last_clean":"2024-04-25T10:16:44.561293+0000","last_became_active":"2024-04-25T10:16:43.554371+0000","last_became_peered":"2024-04-25T10:16:43.554371+0000","last_unstale":"2024-04-25T10:16:44.561293+0000","last_undegraded":"2024-04-25T10:16:44.561293+0000","last_fullsized":"2024-04-25T10:16:44.561293+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.084637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537189+0000","last_change":"2024-04-25T10:16:43.563238+0000","last_active":"2024-04-25T10:16:44.537189+0000","last_peered":"2024-04-25T10:16:44.537189+0000","last_clean":"2024-04-25T10:16:44.537189+0000","last_became_active":"2024-04-25T10:16:43.562888+0000","last_became_peered":"2024-04-25T10:16:43.562888+0000","last_unstale":"2024-04-25T10:16:44.537189+0000","last_undegraded":"2024-04-25T10:16:44.537189+0000","last_fullsized":"2024-04-25T10:16:44.537189+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:34.293550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259043+0000","last_change":"2024-04-25T10:16:43.557253+0000","last_active":"2024-04-25T10:16:45.259043+0000","last_peered":"2024-04-25T10:16:45.259043+0000","last_clean":"2024-04-25T10:16:45.259043+0000","last_became_active":"2024-04-25T10:16:43.556951+0000","last_became_peered":"2024-04-25T10:16:43.556951+0000","last_unstale":"2024-04-25T10:16:45.259043+0000","last_undegraded":"2024-04-25T10:16:45.259043+0000","last_fullsized":"2024-04-25T10:16:45.259043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.380192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259383+0000","last_change":"2024-04-25T10:16:43.558625+0000","last_active":"2024-04-25T10:16:45.259383+0000","last_peered":"2024-04-25T10:16:45.259383+0000","last_clean":"2024-04-25T10:16:45.259383+0000","last_became_active":"2024-04-25T10:16:43.558255+0000","last_became_peered":"2024-04-25T10:16:43.558255+0000","last_unstale":"2024-04-25T10:16:45.259383+0000","last_undegraded":"2024-04-25T10:16:45.259383+0000","last_fullsized":"2024-04-25T10:16:45.259383+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.033238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536630+0000","last_change":"2024-04-25T10:16:43.557356+0000","last_active":"2024-04-25T10:16:44.536630+0000","last_peered":"2024-04-25T10:16:44.536630+0000","last_clean":"2024-04-25T10:16:44.536630+0000","last_became_active":"2024-04-25T10:16:43.557009+0000","last_became_peered":"2024-04-25T10:16:43.557009+0000","last_unstale":"2024-04-25T10:16:44.536630+0000","last_undegraded":"2024-04-25T10:16:44.536630+0000","last_fullsized":"2024-04-25T10:16:44.536630+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:40.022950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537537+0000","last_change":"2024-04-25T10:16:43.559117+0000","last_active":"2024-04-25T10:16:44.537537+0000","last_peered":"2024-04-25T10:16:44.537537+0000","last_clean":"2024-04-25T10:16:44.537537+0000","last_became_active":"2024-04-25T10:16:43.558792+0000","last_became_peered":"2024-04-25T10:16:43.558792+0000","last_unstale":"2024-04-25T10:16:44.537537+0000","last_undegraded":"2024-04-25T10:16:44.537537+0000","last_fullsized":"2024-04-25T10:16:44.537537+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:41.494324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259928+0000","last_change":"2024-04-25T10:16:43.559623+0000","last_active":"2024-04-25T10:16:45.259928+0000","last_peered":"2024-04-25T10:16:45.259928+0000","last_clean":"2024-04-25T10:16:45.259928+0000","last_became_active":"2024-04-25T10:16:43.559334+0000","last_became_peered":"2024-04-25T10:16:43.559334+0000","last_unstale":"2024-04-25T10:16:45.259928+0000","last_undegraded":"2024-04-25T10:16:45.259928+0000","last_fullsized":"2024-04-25T10:16:45.259928+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:43.288992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536382+0000","last_change":"2024-04-25T10:16:43.557501+0000","last_active":"2024-04-25T10:16:44.536382+0000","last_peered":"2024-04-25T10:16:44.536382+0000","last_clean":"2024-04-25T10:16:44.536382+0000","last_became_active":"2024-04-25T10:16:43.557212+0000","last_became_peered":"2024-04-25T10:16:43.557212+0000","last_unstale":"2024-04-25T10:16:44.536382+0000","last_undegraded":"2024-04-25T10:16:44.536382+0000","last_fullsized":"2024-04-25T10:16:44.536382+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:12.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644124+0000","last_change":"2024-04-25T10:16:43.547840+0000","last_active":"2024-04-25T10:16:44.644124+0000","last_peered":"2024-04-25T10:16:44.644124+0000","last_clean":"2024-04-25T10:16:44.644124+0000","last_became_active":"2024-04-25T10:16:43.544427+0000","last_became_peered":"2024-04-25T10:16:43.544427+0000","last_unstale":"2024-04-25T10:16:44.644124+0000","last_undegraded":"2024-04-25T10:16:44.644124+0000","last_fullsized":"2024-04-25T10:16:44.644124+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.009160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258182+0000","last_change":"2024-04-25T10:16:43.562196+0000","last_active":"2024-04-25T10:16:45.258182+0000","last_peered":"2024-04-25T10:16:45.258182+0000","last_clean":"2024-04-25T10:16:45.258182+0000","last_became_active":"2024-04-25T10:16:43.561890+0000","last_became_peered":"2024-04-25T10:16:43.561890+0000","last_unstale":"2024-04-25T10:16:45.258182+0000","last_undegraded":"2024-04-25T10:16:45.258182+0000","last_fullsized":"2024-04-25T10:16:45.258182+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:40.034886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539779+0000","last_change":"2024-04-25T10:16:43.555511+0000","last_active":"2024-04-25T10:16:44.539779+0000","last_peered":"2024-04-25T10:16:44.539779+0000","last_clean":"2024-04-25T10:16:44.539779+0000","last_became_active":"2024-04-25T10:16:43.555209+0000","last_became_peered":"2024-04-25T10:16:43.555209+0000","last_unstale":"2024-04-25T10:16:44.539779+0000","last_undegraded":"2024-04-25T10:16:44.539779+0000","last_fullsized":"2024-04-25T10:16:44.539779+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:27.563607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536915+0000","last_change":"2024-04-25T10:16:43.557716+0000","last_active":"2024-04-25T10:16:44.536915+0000","last_peered":"2024-04-25T10:16:44.536915+0000","last_clean":"2024-04-25T10:16:44.536915+0000","last_became_active":"2024-04-25T10:16:43.557442+0000","last_became_peered":"2024-04-25T10:16:43.557442+0000","last_unstale":"2024-04-25T10:16:44.536915+0000","last_undegraded":"2024-04-25T10:16:44.536915+0000","last_fullsized":"2024-04-25T10:16:44.536915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.240799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567976+0000","last_change":"2024-04-25T10:16:43.559780+0000","last_active":"2024-04-25T10:16:44.567976+0000","last_peered":"2024-04-25T10:16:44.567976+0000","last_clean":"2024-04-25T10:16:44.567976+0000","last_became_active":"2024-04-25T10:16:43.559507+0000","last_became_peered":"2024-04-25T10:16:43.559507+0000","last_unstale":"2024-04-25T10:16:44.567976+0000","last_undegraded":"2024-04-25T10:16:44.567976+0000","last_fullsized":"2024-04-25T10:16:44.567976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:23.418603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258576+0000","last_change":"2024-04-25T10:16:43.544613+0000","last_active":"2024-04-25T10:16:45.258576+0000","last_peered":"2024-04-25T10:16:45.258576+0000","last_clean":"2024-04-25T10:16:45.258576+0000","last_became_active":"2024-04-25T10:16:43.544181+0000","last_became_peered":"2024-04-25T10:16:43.544181+0000","last_unstale":"2024-04-25T10:16:45.258576+0000","last_undegraded":"2024-04-25T10:16:45.258576+0000","last_fullsized":"2024-04-25T10:16:45.258576+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:43.288992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259936+0000","last_change":"2024-04-25T10:16:43.565132+0000","last_active":"2024-04-25T10:16:45.259936+0000","last_peered":"2024-04-25T10:16:45.259936+0000","last_clean":"2024-04-25T10:16:45.259936+0000","last_became_active":"2024-04-25T10:16:43.564843+0000","last_became_peered":"2024-04-25T10:16:43.564843+0000","last_unstale":"2024-04-25T10:16:45.259936+0000","last_undegraded":"2024-04-25T10:16:45.259936+0000","last_fullsized":"2024-04-25T10:16:45.259936+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:57.505598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561287+0000","last_change":"2024-04-25T10:16:43.552626+0000","last_active":"2024-04-25T10:16:44.561287+0000","last_peered":"2024-04-25T10:16:44.561287+0000","last_clean":"2024-04-25T10:16:44.561287+0000","last_became_active":"2024-04-25T10:16:43.551212+0000","last_became_peered":"2024-04-25T10:16:43.551212+0000","last_unstale":"2024-04-25T10:16:44.561287+0000","last_undegraded":"2024-04-25T10:16:44.561287+0000","last_fullsized":"2024-04-25T10:16:44.561287+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.189603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538152+0000","last_change":"2024-04-25T10:16:43.556110+0000","last_active":"2024-04-25T10:16:44.538152+0000","last_peered":"2024-04-25T10:16:44.538152+0000","last_clean":"2024-04-25T10:16:44.538152+0000","last_became_active":"2024-04-25T10:16:43.555755+0000","last_became_peered":"2024-04-25T10:16:43.555755+0000","last_unstale":"2024-04-25T10:16:44.538152+0000","last_undegraded":"2024-04-25T10:16:44.538152+0000","last_fullsized":"2024-04-25T10:16:44.538152+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.966214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536433+0000","last_change":"2024-04-25T10:16:43.561869+0000","last_active":"2024-04-25T10:16:44.536433+0000","last_peered":"2024-04-25T10:16:44.536433+0000","last_clean":"2024-04-25T10:16:44.536433+0000","last_became_active":"2024-04-25T10:16:43.561583+0000","last_became_peered":"2024-04-25T10:16:43.561583+0000","last_unstale":"2024-04-25T10:16:44.536433+0000","last_undegraded":"2024-04-25T10:16:44.536433+0000","last_fullsized":"2024-04-25T10:16:44.536433+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:25.788606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562314+0000","last_change":"2024-04-25T10:16:43.546831+0000","last_active":"2024-04-25T10:16:44.562314+0000","last_peered":"2024-04-25T10:16:44.562314+0000","last_clean":"2024-04-25T10:16:44.562314+0000","last_became_active":"2024-04-25T10:16:43.546419+0000","last_became_peered":"2024-04-25T10:16:43.546419+0000","last_unstale":"2024-04-25T10:16:44.562314+0000","last_undegraded":"2024-04-25T10:16:44.562314+0000","last_fullsized":"2024-04-25T10:16:44.562314+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:51.069413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561846+0000","last_change":"2024-04-25T10:16:43.553554+0000","last_active":"2024-04-25T10:16:44.561846+0000","last_peered":"2024-04-25T10:16:44.561846+0000","last_clean":"2024-04-25T10:16:44.561846+0000","last_became_active":"2024-04-25T10:16:43.553131+0000","last_became_peered":"2024-04-25T10:16:43.553131+0000","last_unstale":"2024-04-25T10:16:44.561846+0000","last_undegraded":"2024-04-25T10:16:44.561846+0000","last_fullsized":"2024-04-25T10:16:44.561846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:18.077257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258531+0000","last_change":"2024-04-25T10:16:43.560703+0000","last_active":"2024-04-25T10:16:45.258531+0000","last_peered":"2024-04-25T10:16:45.258531+0000","last_clean":"2024-04-25T10:16:45.258531+0000","last_became_active":"2024-04-25T10:16:43.560398+0000","last_became_peered":"2024-04-25T10:16:43.560398+0000","last_unstale":"2024-04-25T10:16:45.258531+0000","last_undegraded":"2024-04-25T10:16:45.258531+0000","last_fullsized":"2024-04-25T10:16:45.258531+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:10.354287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536983+0000","last_change":"2024-04-25T10:16:43.554470+0000","last_active":"2024-04-25T10:16:44.536983+0000","last_peered":"2024-04-25T10:16:44.536983+0000","last_clean":"2024-04-25T10:16:44.536983+0000","last_became_active":"2024-04-25T10:16:43.554142+0000","last_became_peered":"2024-04-25T10:16:43.554142+0000","last_unstale":"2024-04-25T10:16:44.536983+0000","last_undegraded":"2024-04-25T10:16:44.536983+0000","last_fullsized":"2024-04-25T10:16:44.536983+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:40.366765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538948+0000","last_change":"2024-04-25T10:16:43.568131+0000","last_active":"2024-04-25T10:16:44.538948+0000","last_peered":"2024-04-25T10:16:44.538948+0000","last_clean":"2024-04-25T10:16:44.538948+0000","last_became_active":"2024-04-25T10:16:43.567824+0000","last_became_peered":"2024-04-25T10:16:43.567824+0000","last_unstale":"2024-04-25T10:16:44.538948+0000","last_undegraded":"2024-04-25T10:16:44.538948+0000","last_fullsized":"2024-04-25T10:16:44.538948+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537321+0000","last_change":"2024-04-25T10:16:43.547282+0000","last_active":"2024-04-25T10:16:44.537321+0000","last_peered":"2024-04-25T10:16:44.537321+0000","last_clean":"2024-04-25T10:16:44.537321+0000","last_became_active":"2024-04-25T10:16:43.545939+0000","last_became_peered":"2024-04-25T10:16:43.545939+0000","last_unstale":"2024-04-25T10:16:44.537321+0000","last_undegraded":"2024-04-25T10:16:44.537321+0000","last_fullsized":"2024-04-25T10:16:44.537321+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:10.488478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258369+0000","last_change":"2024-04-25T10:16:43.558141+0000","last_active":"2024-04-25T10:16:45.258369+0000","last_peered":"2024-04-25T10:16:45.258369+0000","last_clean":"2024-04-25T10:16:45.258369+0000","last_became_active":"2024-04-25T10:16:43.557866+0000","last_became_peered":"2024-04-25T10:16:43.557866+0000","last_unstale":"2024-04-25T10:16:45.258369+0000","last_undegraded":"2024-04-25T10:16:45.258369+0000","last_fullsized":"2024-04-25T10:16:45.258369+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:20.806187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537245+0000","last_change":"2024-04-25T10:16:43.558128+0000","last_active":"2024-04-25T10:16:44.537245+0000","last_peered":"2024-04-25T10:16:44.537245+0000","last_clean":"2024-04-25T10:16:44.537245+0000","last_became_active":"2024-04-25T10:16:43.557811+0000","last_became_peered":"2024-04-25T10:16:43.557811+0000","last_unstale":"2024-04-25T10:16:44.537245+0000","last_undegraded":"2024-04-25T10:16:44.537245+0000","last_fullsized":"2024-04-25T10:16:44.537245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:15.502788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644518+0000","last_change":"2024-04-25T10:16:43.557078+0000","last_active":"2024-04-25T10:16:44.644518+0000","last_peered":"2024-04-25T10:16:44.644518+0000","last_clean":"2024-04-25T10:16:44.644518+0000","last_became_active":"2024-04-25T10:16:43.556763+0000","last_became_peered":"2024-04-25T10:16:43.556763+0000","last_unstale":"2024-04-25T10:16:44.644518+0000","last_undegraded":"2024-04-25T10:16:44.644518+0000","last_fullsized":"2024-04-25T10:16:44.644518+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:51.069413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"147'4","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259562+0000","last_change":"2024-04-25T10:16:42.551717+0000","last_active":"2024-04-25T10:16:45.259562+0000","last_peered":"2024-04-25T10:16:45.259562+0000","last_clean":"2024-04-25T10:16:45.259562+0000","last_became_active":"2024-04-25T10:16:42.551437+0000","last_became_peered":"2024-04-25T10:16:42.551437+0000","last_unstale":"2024-04-25T10:16:45.259562+0000","last_undegraded":"2024-04-25T10:16:45.259562+0000","last_fullsized":"2024-04-25T10:16:45.259562+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:40.073529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644386+0000","last_change":"2024-04-25T10:16:43.543406+0000","last_active":"2024-04-25T10:16:44.644386+0000","last_peered":"2024-04-25T10:16:44.644386+0000","last_clean":"2024-04-25T10:16:44.644386+0000","last_became_active":"2024-04-25T10:16:43.542934+0000","last_became_peered":"2024-04-25T10:16:43.542934+0000","last_unstale":"2024-04-25T10:16:44.644386+0000","last_undegraded":"2024-04-25T10:16:44.644386+0000","last_fullsized":"2024-04-25T10:16:44.644386+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.763212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567363+0000","last_change":"2024-04-25T10:16:42.571780+0000","last_active":"2024-04-25T10:16:44.567363+0000","last_peered":"2024-04-25T10:16:44.567363+0000","last_clean":"2024-04-25T10:16:44.567363+0000","last_became_active":"2024-04-25T10:16:42.571355+0000","last_became_peered":"2024-04-25T10:16:42.571355+0000","last_unstale":"2024-04-25T10:16:44.567363+0000","last_undegraded":"2024-04-25T10:16:44.567363+0000","last_fullsized":"2024-04-25T10:16:44.567363+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.102371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538914+0000","last_change":"2024-04-25T10:16:43.559864+0000","last_active":"2024-04-25T10:16:44.538914+0000","last_peered":"2024-04-25T10:16:44.538914+0000","last_clean":"2024-04-25T10:16:44.538914+0000","last_became_active":"2024-04-25T10:16:43.559490+0000","last_became_peered":"2024-04-25T10:16:43.559490+0000","last_unstale":"2024-04-25T10:16:44.538914+0000","last_undegraded":"2024-04-25T10:16:44.538914+0000","last_fullsized":"2024-04-25T10:16:44.538914+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:09.481790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568117+0000","last_change":"2024-04-25T10:16:42.571044+0000","last_active":"2024-04-25T10:16:44.568117+0000","last_peered":"2024-04-25T10:16:44.568117+0000","last_clean":"2024-04-25T10:16:44.568117+0000","last_became_active":"2024-04-25T10:16:42.570593+0000","last_became_peered":"2024-04-25T10:16:42.570593+0000","last_unstale":"2024-04-25T10:16:44.568117+0000","last_undegraded":"2024-04-25T10:16:44.568117+0000","last_fullsized":"2024-04-25T10:16:44.568117+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:40.073529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567879+0000","last_change":"2024-04-25T10:16:43.550656+0000","last_active":"2024-04-25T10:16:44.567879+0000","last_peered":"2024-04-25T10:16:44.567879+0000","last_clean":"2024-04-25T10:16:44.567879+0000","last_became_active":"2024-04-25T10:16:43.550317+0000","last_became_peered":"2024-04-25T10:16:43.550317+0000","last_unstale":"2024-04-25T10:16:44.567879+0000","last_undegraded":"2024-04-25T10:16:44.567879+0000","last_fullsized":"2024-04-25T10:16:44.567879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.020831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644492+0000","last_change":"2024-04-25T10:16:42.553654+0000","last_active":"2024-04-25T10:16:44.644492+0000","last_peered":"2024-04-25T10:16:44.644492+0000","last_clean":"2024-04-25T10:16:44.644492+0000","last_became_active":"2024-04-25T10:16:42.553115+0000","last_became_peered":"2024-04-25T10:16:42.553115+0000","last_unstale":"2024-04-25T10:16:44.644492+0000","last_undegraded":"2024-04-25T10:16:44.644492+0000","last_fullsized":"2024-04-25T10:16:44.644492+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:57.834058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.257865+0000","last_change":"2024-04-25T10:16:43.558036+0000","last_active":"2024-04-25T10:16:45.257865+0000","last_peered":"2024-04-25T10:16:45.257865+0000","last_clean":"2024-04-25T10:16:45.257865+0000","last_became_active":"2024-04-25T10:16:43.557754+0000","last_became_peered":"2024-04-25T10:16:43.557754+0000","last_unstale":"2024-04-25T10:16:45.257865+0000","last_undegraded":"2024-04-25T10:16:45.257865+0000","last_fullsized":"2024-04-25T10:16:45.257865+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:14.630997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538931+0000","last_change":"2024-04-25T10:16:42.556137+0000","last_active":"2024-04-25T10:16:44.538931+0000","last_peered":"2024-04-25T10:16:44.538931+0000","last_clean":"2024-04-25T10:16:44.538931+0000","last_became_active":"2024-04-25T10:16:42.555806+0000","last_became_peered":"2024-04-25T10:16:42.555806+0000","last_unstale":"2024-04-25T10:16:44.538931+0000","last_undegraded":"2024-04-25T10:16:44.538931+0000","last_fullsized":"2024-04-25T10:16:44.538931+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:31.761423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.535934+0000","last_change":"2024-04-25T10:16:43.554154+0000","last_active":"2024-04-25T10:16:44.535934+0000","last_peered":"2024-04-25T10:16:44.535934+0000","last_clean":"2024-04-25T10:16:44.535934+0000","last_became_active":"2024-04-25T10:16:43.552635+0000","last_became_peered":"2024-04-25T10:16:43.552635+0000","last_unstale":"2024-04-25T10:16:44.535934+0000","last_undegraded":"2024-04-25T10:16:44.535934+0000","last_fullsized":"2024-04-25T10:16:44.535934+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:39.834167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259987+0000","last_change":"2024-04-25T10:16:42.557166+0000","last_active":"2024-04-25T10:16:45.259987+0000","last_peered":"2024-04-25T10:16:45.259987+0000","last_clean":"2024-04-25T10:16:45.259987+0000","last_became_active":"2024-04-25T10:16:42.556841+0000","last_became_peered":"2024-04-25T10:16:42.556841+0000","last_unstale":"2024-04-25T10:16:45.259987+0000","last_undegraded":"2024-04-25T10:16:45.259987+0000","last_fullsized":"2024-04-25T10:16:45.259987+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:28.174820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644268+0000","last_change":"2024-04-25T10:16:43.541816+0000","last_active":"2024-04-25T10:16:44.644268+0000","last_peered":"2024-04-25T10:16:44.644268+0000","last_clean":"2024-04-25T10:16:44.644268+0000","last_became_active":"2024-04-25T10:16:43.541356+0000","last_became_peered":"2024-04-25T10:16:43.541356+0000","last_unstale":"2024-04-25T10:16:44.644268+0000","last_undegraded":"2024-04-25T10:16:44.644268+0000","last_fullsized":"2024-04-25T10:16:44.644268+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.189603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.560999+0000","last_change":"2024-04-25T10:16:42.562767+0000","last_active":"2024-04-25T10:16:44.560999+0000","last_peered":"2024-04-25T10:16:44.560999+0000","last_clean":"2024-04-25T10:16:44.560999+0000","last_became_active":"2024-04-25T10:16:42.562417+0000","last_became_peered":"2024-04-25T10:16:42.562417+0000","last_unstale":"2024-04-25T10:16:44.560999+0000","last_undegraded":"2024-04-25T10:16:44.560999+0000","last_fullsized":"2024-04-25T10:16:44.560999+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.722759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561472+0000","last_change":"2024-04-25T10:16:43.558917+0000","last_active":"2024-04-25T10:16:44.561472+0000","last_peered":"2024-04-25T10:16:44.561472+0000","last_clean":"2024-04-25T10:16:44.561472+0000","last_became_active":"2024-04-25T10:16:43.558613+0000","last_became_peered":"2024-04-25T10:16:43.558613+0000","last_unstale":"2024-04-25T10:16:44.561472+0000","last_undegraded":"2024-04-25T10:16:44.561472+0000","last_fullsized":"2024-04-25T10:16:44.561472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:57.980294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258330+0000","last_change":"2024-04-25T10:16:42.566952+0000","last_active":"2024-04-25T10:16:45.258330+0000","last_peered":"2024-04-25T10:16:45.258330+0000","last_clean":"2024-04-25T10:16:45.258330+0000","last_became_active":"2024-04-25T10:16:42.566676+0000","last_became_peered":"2024-04-25T10:16:42.566676+0000","last_unstale":"2024-04-25T10:16:45.258330+0000","last_undegraded":"2024-04-25T10:16:45.258330+0000","last_fullsized":"2024-04-25T10:16:45.258330+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:59.508460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539287+0000","last_change":"2024-04-25T10:16:43.561379+0000","last_active":"2024-04-25T10:16:44.539287+0000","last_peered":"2024-04-25T10:16:44.539287+0000","last_clean":"2024-04-25T10:16:44.539287+0000","last_became_active":"2024-04-25T10:16:43.561087+0000","last_became_peered":"2024-04-25T10:16:43.561087+0000","last_unstale":"2024-04-25T10:16:44.539287+0000","last_undegraded":"2024-04-25T10:16:44.539287+0000","last_fullsized":"2024-04-25T10:16:44.539287+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:23.414369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568011+0000","last_change":"2024-04-25T10:16:42.569373+0000","last_active":"2024-04-25T10:16:44.568011+0000","last_peered":"2024-04-25T10:16:44.568011+0000","last_clean":"2024-04-25T10:16:44.568011+0000","last_became_active":"2024-04-25T10:16:42.568629+0000","last_became_peered":"2024-04-25T10:16:42.568629+0000","last_unstale":"2024-04-25T10:16:44.568011+0000","last_undegraded":"2024-04-25T10:16:44.568011+0000","last_fullsized":"2024-04-25T10:16:44.568011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:59.508460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536837+0000","last_change":"2024-04-25T10:16:43.551780+0000","last_active":"2024-04-25T10:16:44.536837+0000","last_peered":"2024-04-25T10:16:44.536837+0000","last_clean":"2024-04-25T10:16:44.536837+0000","last_became_active":"2024-04-25T10:16:43.551458+0000","last_became_peered":"2024-04-25T10:16:43.551458+0000","last_unstale":"2024-04-25T10:16:44.536837+0000","last_undegraded":"2024-04-25T10:16:44.536837+0000","last_fullsized":"2024-04-25T10:16:44.536837+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:13.485034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562222+0000","last_change":"2024-04-25T10:16:42.566623+0000","last_active":"2024-04-25T10:16:44.562222+0000","last_peered":"2024-04-25T10:16:44.562222+0000","last_clean":"2024-04-25T10:16:44.562222+0000","last_became_active":"2024-04-25T10:16:42.566064+0000","last_became_peered":"2024-04-25T10:16:42.566064+0000","last_unstale":"2024-04-25T10:16:44.562222+0000","last_undegraded":"2024-04-25T10:16:44.562222+0000","last_fullsized":"2024-04-25T10:16:44.562222+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:37.252841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537203+0000","last_change":"2024-04-25T10:16:43.557534+0000","last_active":"2024-04-25T10:16:44.537203+0000","last_peered":"2024-04-25T10:16:44.537203+0000","last_clean":"2024-04-25T10:16:44.537203+0000","last_became_active":"2024-04-25T10:16:43.557189+0000","last_became_peered":"2024-04-25T10:16:43.557189+0000","last_unstale":"2024-04-25T10:16:44.537203+0000","last_undegraded":"2024-04-25T10:16:44.537203+0000","last_fullsized":"2024-04-25T10:16:44.537203+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:00.427901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"147'3","reported_seq":17,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536733+0000","last_change":"2024-04-25T10:16:42.549331+0000","last_active":"2024-04-25T10:16:44.536733+0000","last_peered":"2024-04-25T10:16:44.536733+0000","last_clean":"2024-04-25T10:16:44.536733+0000","last_became_active":"2024-04-25T10:16:42.549027+0000","last_became_peered":"2024-04-25T10:16:42.549027+0000","last_unstale":"2024-04-25T10:16:44.536733+0000","last_undegraded":"2024-04-25T10:16:44.536733+0000","last_fullsized":"2024-04-25T10:16:44.536733+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:26.406784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537764+0000","last_change":"2024-04-25T10:16:43.555391+0000","last_active":"2024-04-25T10:16:44.537764+0000","last_peered":"2024-04-25T10:16:44.537764+0000","last_clean":"2024-04-25T10:16:44.537764+0000","last_became_active":"2024-04-25T10:16:43.554763+0000","last_became_peered":"2024-04-25T10:16:43.554763+0000","last_unstale":"2024-04-25T10:16:44.537764+0000","last_undegraded":"2024-04-25T10:16:44.537764+0000","last_fullsized":"2024-04-25T10:16:44.537764+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:53.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.260077+0000","last_change":"2024-04-25T10:16:42.556479+0000","last_active":"2024-04-25T10:16:45.260077+0000","last_peered":"2024-04-25T10:16:45.260077+0000","last_clean":"2024-04-25T10:16:45.260077+0000","last_became_active":"2024-04-25T10:16:42.556163+0000","last_became_peered":"2024-04-25T10:16:42.556163+0000","last_unstale":"2024-04-25T10:16:45.260077+0000","last_undegraded":"2024-04-25T10:16:45.260077+0000","last_fullsized":"2024-04-25T10:16:45.260077+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:15.848421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644123+0000","last_change":"2024-04-25T10:16:43.548087+0000","last_active":"2024-04-25T10:16:44.644123+0000","last_peered":"2024-04-25T10:16:44.644123+0000","last_clean":"2024-04-25T10:16:44.644123+0000","last_became_active":"2024-04-25T10:16:43.545124+0000","last_became_peered":"2024-04-25T10:16:43.545124+0000","last_unstale":"2024-04-25T10:16:44.644123+0000","last_undegraded":"2024-04-25T10:16:44.644123+0000","last_fullsized":"2024-04-25T10:16:44.644123+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:52.071878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537171+0000","last_change":"2024-04-25T10:16:42.562159+0000","last_active":"2024-04-25T10:16:44.537171+0000","last_peered":"2024-04-25T10:16:44.537171+0000","last_clean":"2024-04-25T10:16:44.537171+0000","last_became_active":"2024-04-25T10:16:42.561883+0000","last_became_peered":"2024-04-25T10:16:42.561883+0000","last_unstale":"2024-04-25T10:16:44.537171+0000","last_undegraded":"2024-04-25T10:16:44.537171+0000","last_fullsized":"2024-04-25T10:16:44.537171+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.486253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259922+0000","last_change":"2024-04-25T10:16:43.553742+0000","last_active":"2024-04-25T10:16:45.259922+0000","last_peered":"2024-04-25T10:16:45.259922+0000","last_clean":"2024-04-25T10:16:45.259922+0000","last_became_active":"2024-04-25T10:16:43.553465+0000","last_became_peered":"2024-04-25T10:16:43.553465+0000","last_unstale":"2024-04-25T10:16:45.259922+0000","last_undegraded":"2024-04-25T10:16:45.259922+0000","last_fullsized":"2024-04-25T10:16:45.259922+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:14.630997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536152+0000","last_change":"2024-04-25T10:16:42.552024+0000","last_active":"2024-04-25T10:16:44.536152+0000","last_peered":"2024-04-25T10:16:44.536152+0000","last_clean":"2024-04-25T10:16:44.536152+0000","last_became_active":"2024-04-25T10:16:42.550839+0000","last_became_peered":"2024-04-25T10:16:42.550839+0000","last_unstale":"2024-04-25T10:16:44.536152+0000","last_undegraded":"2024-04-25T10:16:44.536152+0000","last_fullsized":"2024-04-25T10:16:44.536152+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:05.849811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644086+0000","last_change":"2024-04-25T10:16:43.560677+0000","last_active":"2024-04-25T10:16:44.644086+0000","last_peered":"2024-04-25T10:16:44.644086+0000","last_clean":"2024-04-25T10:16:44.644086+0000","last_became_active":"2024-04-25T10:16:43.560405+0000","last_became_peered":"2024-04-25T10:16:43.560405+0000","last_unstale":"2024-04-25T10:16:44.644086+0000","last_undegraded":"2024-04-25T10:16:44.644086+0000","last_fullsized":"2024-04-25T10:16:44.644086+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.735033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"147'8","reported_seq":22,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.560671+0000","last_change":"2024-04-25T10:16:42.566140+0000","last_active":"2024-04-25T10:16:44.560671+0000","last_peered":"2024-04-25T10:16:44.560671+0000","last_clean":"2024-04-25T10:16:44.560671+0000","last_became_active":"2024-04-25T10:16:42.565757+0000","last_became_peered":"2024-04-25T10:16:42.565757+0000","last_unstale":"2024-04-25T10:16:44.560671+0000","last_undegraded":"2024-04-25T10:16:44.560671+0000","last_fullsized":"2024-04-25T10:16:44.560671+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:09.589446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567743+0000","last_change":"2024-04-25T10:16:43.564528+0000","last_active":"2024-04-25T10:16:44.567743+0000","last_peered":"2024-04-25T10:16:44.567743+0000","last_clean":"2024-04-25T10:16:44.567743+0000","last_became_active":"2024-04-25T10:16:43.563011+0000","last_became_peered":"2024-04-25T10:16:43.563011+0000","last_unstale":"2024-04-25T10:16:44.567743+0000","last_undegraded":"2024-04-25T10:16:44.567743+0000","last_fullsized":"2024-04-25T10:16:44.567743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:00.010983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537170+0000","last_change":"2024-04-25T10:16:42.561495+0000","last_active":"2024-04-25T10:16:44.537170+0000","last_peered":"2024-04-25T10:16:44.537170+0000","last_clean":"2024-04-25T10:16:44.537170+0000","last_became_active":"2024-04-25T10:16:42.560241+0000","last_became_peered":"2024-04-25T10:16:42.560241+0000","last_unstale":"2024-04-25T10:16:44.537170+0000","last_undegraded":"2024-04-25T10:16:44.537170+0000","last_fullsized":"2024-04-25T10:16:44.537170+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:11.915069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568090+0000","last_change":"2024-04-25T10:16:43.557795+0000","last_active":"2024-04-25T10:16:44.568090+0000","last_peered":"2024-04-25T10:16:44.568090+0000","last_clean":"2024-04-25T10:16:44.568090+0000","last_became_active":"2024-04-25T10:16:43.557521+0000","last_became_peered":"2024-04-25T10:16:43.557521+0000","last_unstale":"2024-04-25T10:16:44.568090+0000","last_undegraded":"2024-04-25T10:16:44.568090+0000","last_fullsized":"2024-04-25T10:16:44.568090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:13.810505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567675+0000","last_change":"2024-04-25T10:16:42.564071+0000","last_active":"2024-04-25T10:16:44.567675+0000","last_peered":"2024-04-25T10:16:44.567675+0000","last_clean":"2024-04-25T10:16:44.567675+0000","last_became_active":"2024-04-25T10:16:42.563824+0000","last_became_peered":"2024-04-25T10:16:42.563824+0000","last_unstale":"2024-04-25T10:16:44.567675+0000","last_undegraded":"2024-04-25T10:16:44.567675+0000","last_fullsized":"2024-04-25T10:16:44.567675+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:28.174820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562102+0000","last_change":"2024-04-25T10:16:43.557697+0000","last_active":"2024-04-25T10:16:44.562102+0000","last_peered":"2024-04-25T10:16:44.562102+0000","last_clean":"2024-04-25T10:16:44.562102+0000","last_became_active":"2024-04-25T10:16:43.555480+0000","last_became_peered":"2024-04-25T10:16:43.555480+0000","last_unstale":"2024-04-25T10:16:44.562102+0000","last_undegraded":"2024-04-25T10:16:44.562102+0000","last_fullsized":"2024-04-25T10:16:44.562102+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:45.108391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644365+0000","last_change":"2024-04-25T10:16:42.555771+0000","last_active":"2024-04-25T10:16:44.644365+0000","last_peered":"2024-04-25T10:16:44.644365+0000","last_clean":"2024-04-25T10:16:44.644365+0000","last_became_active":"2024-04-25T10:16:42.555336+0000","last_became_peered":"2024-04-25T10:16:42.555336+0000","last_unstale":"2024-04-25T10:16:44.644365+0000","last_undegraded":"2024-04-25T10:16:44.644365+0000","last_fullsized":"2024-04-25T10:16:44.644365+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:22.397834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562488+0000","last_change":"2024-04-25T10:16:43.540694+0000","last_active":"2024-04-25T10:16:44.562488+0000","last_peered":"2024-04-25T10:16:44.562488+0000","last_clean":"2024-04-25T10:16:44.562488+0000","last_became_active":"2024-04-25T10:16:43.540044+0000","last_became_peered":"2024-04-25T10:16:43.540044+0000","last_unstale":"2024-04-25T10:16:44.562488+0000","last_undegraded":"2024-04-25T10:16:44.562488+0000","last_fullsized":"2024-04-25T10:16:44.562488+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.118906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537658+0000","last_change":"2024-04-25T10:16:42.561604+0000","last_active":"2024-04-25T10:16:44.537658+0000","last_peered":"2024-04-25T10:16:44.537658+0000","last_clean":"2024-04-25T10:16:44.537658+0000","last_became_active":"2024-04-25T10:16:42.561278+0000","last_became_peered":"2024-04-25T10:16:42.561278+0000","last_unstale":"2024-04-25T10:16:44.537658+0000","last_undegraded":"2024-04-25T10:16:44.537658+0000","last_fullsized":"2024-04-25T10:16:44.537658+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:52.219063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567427+0000","last_change":"2024-04-25T10:16:43.560700+0000","last_active":"2024-04-25T10:16:44.567427+0000","last_peered":"2024-04-25T10:16:44.567427+0000","last_clean":"2024-04-25T10:16:44.567427+0000","last_became_active":"2024-04-25T10:16:43.560410+0000","last_became_peered":"2024-04-25T10:16:43.560410+0000","last_unstale":"2024-04-25T10:16:44.567427+0000","last_undegraded":"2024-04-25T10:16:44.567427+0000","last_fullsized":"2024-04-25T10:16:44.567427+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:34.167604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258171+0000","last_change":"2024-04-25T10:16:42.552123+0000","last_active":"2024-04-25T10:16:45.258171+0000","last_peered":"2024-04-25T10:16:45.258171+0000","last_clean":"2024-04-25T10:16:45.258171+0000","last_became_active":"2024-04-25T10:16:42.551840+0000","last_became_peered":"2024-04-25T10:16:42.551840+0000","last_unstale":"2024-04-25T10:16:45.258171+0000","last_undegraded":"2024-04-25T10:16:45.258171+0000","last_fullsized":"2024-04-25T10:16:45.258171+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:11.016703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644307+0000","last_change":"2024-04-25T10:16:43.535347+0000","last_active":"2024-04-25T10:16:44.644307+0000","last_peered":"2024-04-25T10:16:44.644307+0000","last_clean":"2024-04-25T10:16:44.644307+0000","last_became_active":"2024-04-25T10:16:43.533702+0000","last_became_peered":"2024-04-25T10:16:43.533702+0000","last_unstale":"2024-04-25T10:16:44.644307+0000","last_undegraded":"2024-04-25T10:16:44.644307+0000","last_fullsized":"2024-04-25T10:16:44.644307+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:57.980294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538061+0000","last_change":"2024-04-25T10:16:42.555853+0000","last_active":"2024-04-25T10:16:44.538061+0000","last_peered":"2024-04-25T10:16:44.538061+0000","last_clean":"2024-04-25T10:16:44.538061+0000","last_became_active":"2024-04-25T10:16:42.555550+0000","last_became_peered":"2024-04-25T10:16:42.555550+0000","last_unstale":"2024-04-25T10:16:44.538061+0000","last_undegraded":"2024-04-25T10:16:44.538061+0000","last_fullsized":"2024-04-25T10:16:44.538061+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:01.727950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567403+0000","last_change":"2024-04-25T10:16:43.556539+0000","last_active":"2024-04-25T10:16:44.567403+0000","last_peered":"2024-04-25T10:16:44.567403+0000","last_clean":"2024-04-25T10:16:44.567403+0000","last_became_active":"2024-04-25T10:16:43.556219+0000","last_became_peered":"2024-04-25T10:16:43.556219+0000","last_unstale":"2024-04-25T10:16:44.567403+0000","last_undegraded":"2024-04-25T10:16:44.567403+0000","last_fullsized":"2024-04-25T10:16:44.567403+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.257333+0000","last_change":"2024-04-25T10:16:42.551277+0000","last_active":"2024-04-25T10:16:45.257333+0000","last_peered":"2024-04-25T10:16:45.257333+0000","last_clean":"2024-04-25T10:16:45.257333+0000","last_became_active":"2024-04-25T10:16:42.550967+0000","last_became_peered":"2024-04-25T10:16:42.550967+0000","last_unstale":"2024-04-25T10:16:45.257333+0000","last_undegraded":"2024-04-25T10:16:45.257333+0000","last_fullsized":"2024-04-25T10:16:45.257333+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:19.550214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644311+0000","last_change":"2024-04-25T10:16:43.546966+0000","last_active":"2024-04-25T10:16:44.644311+0000","last_peered":"2024-04-25T10:16:44.644311+0000","last_clean":"2024-04-25T10:16:44.644311+0000","last_became_active":"2024-04-25T10:16:43.546529+0000","last_became_peered":"2024-04-25T10:16:43.546529+0000","last_unstale":"2024-04-25T10:16:44.644311+0000","last_undegraded":"2024-04-25T10:16:44.644311+0000","last_fullsized":"2024-04-25T10:16:44.644311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:45.108391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537963+0000","last_change":"2024-04-25T10:16:42.561957+0000","last_active":"2024-04-25T10:16:44.537963+0000","last_peered":"2024-04-25T10:16:44.537963+0000","last_clean":"2024-04-25T10:16:44.537963+0000","last_became_active":"2024-04-25T10:16:42.561460+0000","last_became_peered":"2024-04-25T10:16:42.561460+0000","last_unstale":"2024-04-25T10:16:44.537963+0000","last_undegraded":"2024-04-25T10:16:44.537963+0000","last_fullsized":"2024-04-25T10:16:44.537963+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:36.001686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258883+0000","last_change":"2024-04-25T10:16:43.555830+0000","last_active":"2024-04-25T10:16:45.258883+0000","last_peered":"2024-04-25T10:16:45.258883+0000","last_clean":"2024-04-25T10:16:45.258883+0000","last_became_active":"2024-04-25T10:16:43.554690+0000","last_became_peered":"2024-04-25T10:16:43.554690+0000","last_unstale":"2024-04-25T10:16:45.258883+0000","last_undegraded":"2024-04-25T10:16:45.258883+0000","last_fullsized":"2024-04-25T10:16:45.258883+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:20.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539527+0000","last_change":"2024-04-25T10:16:42.561522+0000","last_active":"2024-04-25T10:16:44.539527+0000","last_peered":"2024-04-25T10:16:44.539527+0000","last_clean":"2024-04-25T10:16:44.539527+0000","last_became_active":"2024-04-25T10:16:42.561181+0000","last_became_peered":"2024-04-25T10:16:42.561181+0000","last_unstale":"2024-04-25T10:16:44.539527+0000","last_undegraded":"2024-04-25T10:16:44.539527+0000","last_fullsized":"2024-04-25T10:16:44.539527+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:33.277014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538833+0000","last_change":"2024-04-25T10:16:43.559320+0000","last_active":"2024-04-25T10:16:44.538833+0000","last_peered":"2024-04-25T10:16:44.538833+0000","last_clean":"2024-04-25T10:16:44.538833+0000","last_became_active":"2024-04-25T10:16:43.559034+0000","last_became_peered":"2024-04-25T10:16:43.559034+0000","last_unstale":"2024-04-25T10:16:44.538833+0000","last_undegraded":"2024-04-25T10:16:44.538833+0000","last_fullsized":"2024-04-25T10:16:44.538833+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:01.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"148'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562091+0000","last_change":"2024-04-25T10:16:42.570360+0000","last_active":"2024-04-25T10:16:44.562091+0000","last_peered":"2024-04-25T10:16:44.562091+0000","last_clean":"2024-04-25T10:16:44.562091+0000","last_became_active":"2024-04-25T10:16:42.569941+0000","last_became_peered":"2024-04-25T10:16:42.569941+0000","last_unstale":"2024-04-25T10:16:44.562091+0000","last_undegraded":"2024-04-25T10:16:44.562091+0000","last_fullsized":"2024-04-25T10:16:44.562091+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:38.817632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562408+0000","last_change":"2024-04-25T10:16:43.557368+0000","last_active":"2024-04-25T10:16:44.562408+0000","last_peered":"2024-04-25T10:16:44.562408+0000","last_clean":"2024-04-25T10:16:44.562408+0000","last_became_active":"2024-04-25T10:16:43.557070+0000","last_became_peered":"2024-04-25T10:16:43.557070+0000","last_unstale":"2024-04-25T10:16:44.562408+0000","last_undegraded":"2024-04-25T10:16:44.562408+0000","last_fullsized":"2024-04-25T10:16:44.562408+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.833357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644415+0000","last_change":"2024-04-25T10:16:42.553757+0000","last_active":"2024-04-25T10:16:44.644415+0000","last_peered":"2024-04-25T10:16:44.644415+0000","last_clean":"2024-04-25T10:16:44.644415+0000","last_became_active":"2024-04-25T10:16:42.552414+0000","last_became_peered":"2024-04-25T10:16:42.552414+0000","last_unstale":"2024-04-25T10:16:44.644415+0000","last_undegraded":"2024-04-25T10:16:44.644415+0000","last_fullsized":"2024-04-25T10:16:44.644415+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:29.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259472+0000","last_change":"2024-04-25T10:16:43.551110+0000","last_active":"2024-04-25T10:16:45.259472+0000","last_peered":"2024-04-25T10:16:45.259472+0000","last_clean":"2024-04-25T10:16:45.259472+0000","last_became_active":"2024-04-25T10:16:43.550776+0000","last_became_peered":"2024-04-25T10:16:43.550776+0000","last_unstale":"2024-04-25T10:16:45.259472+0000","last_undegraded":"2024-04-25T10:16:45.259472+0000","last_fullsized":"2024-04-25T10:16:45.259472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:36.036392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259025+0000","last_change":"2024-04-25T10:16:42.553623+0000","last_active":"2024-04-25T10:16:45.259025+0000","last_peered":"2024-04-25T10:16:45.259025+0000","last_clean":"2024-04-25T10:16:45.259025+0000","last_became_active":"2024-04-25T10:16:42.553344+0000","last_became_peered":"2024-04-25T10:16:42.553344+0000","last_unstale":"2024-04-25T10:16:45.259025+0000","last_undegraded":"2024-04-25T10:16:45.259025+0000","last_fullsized":"2024-04-25T10:16:45.259025+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:09.078537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537775+0000","last_change":"2024-04-25T10:16:43.557497+0000","last_active":"2024-04-25T10:16:44.537775+0000","last_peered":"2024-04-25T10:16:44.537775+0000","last_clean":"2024-04-25T10:16:44.537775+0000","last_became_active":"2024-04-25T10:16:43.557138+0000","last_became_peered":"2024-04-25T10:16:43.557138+0000","last_unstale":"2024-04-25T10:16:44.537775+0000","last_undegraded":"2024-04-25T10:16:44.537775+0000","last_fullsized":"2024-04-25T10:16:44.537775+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:06.866347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561372+0000","last_change":"2024-04-25T10:16:42.569306+0000","last_active":"2024-04-25T10:16:44.561372+0000","last_peered":"2024-04-25T10:16:44.561372+0000","last_clean":"2024-04-25T10:16:44.561372+0000","last_became_active":"2024-04-25T10:16:42.568916+0000","last_became_peered":"2024-04-25T10:16:42.568916+0000","last_unstale":"2024-04-25T10:16:44.561372+0000","last_undegraded":"2024-04-25T10:16:44.561372+0000","last_fullsized":"2024-04-25T10:16:44.561372+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:11.043365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536335+0000","last_change":"2024-04-25T10:16:43.558958+0000","last_active":"2024-04-25T10:16:44.536335+0000","last_peered":"2024-04-25T10:16:44.536335+0000","last_clean":"2024-04-25T10:16:44.536335+0000","last_became_active":"2024-04-25T10:16:43.558167+0000","last_became_peered":"2024-04-25T10:16:43.558167+0000","last_unstale":"2024-04-25T10:16:44.536335+0000","last_undegraded":"2024-04-25T10:16:44.536335+0000","last_fullsized":"2024-04-25T10:16:44.536335+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.739294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536161+0000","last_change":"2024-04-25T10:16:42.560964+0000","last_active":"2024-04-25T10:16:44.536161+0000","last_peered":"2024-04-25T10:16:44.536161+0000","last_clean":"2024-04-25T10:16:44.536161+0000","last_became_active":"2024-04-25T10:16:42.560653+0000","last_became_peered":"2024-04-25T10:16:42.560653+0000","last_unstale":"2024-04-25T10:16:44.536161+0000","last_undegraded":"2024-04-25T10:16:44.536161+0000","last_fullsized":"2024-04-25T10:16:44.536161+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.536743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644262+0000","last_change":"2024-04-25T10:16:43.558553+0000","last_active":"2024-04-25T10:16:44.644262+0000","last_peered":"2024-04-25T10:16:44.644262+0000","last_clean":"2024-04-25T10:16:44.644262+0000","last_became_active":"2024-04-25T10:16:43.558286+0000","last_became_peered":"2024-04-25T10:16:43.558286+0000","last_unstale":"2024-04-25T10:16:44.644262+0000","last_undegraded":"2024-04-25T10:16:44.644262+0000","last_fullsized":"2024-04-25T10:16:44.644262+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.084637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561386+0000","last_change":"2024-04-25T10:16:42.567149+0000","last_active":"2024-04-25T10:16:44.561386+0000","last_peered":"2024-04-25T10:16:44.561386+0000","last_clean":"2024-04-25T10:16:44.561386+0000","last_became_active":"2024-04-25T10:16:42.566786+0000","last_became_peered":"2024-04-25T10:16:42.566786+0000","last_unstale":"2024-04-25T10:16:44.561386+0000","last_undegraded":"2024-04-25T10:16:44.561386+0000","last_fullsized":"2024-04-25T10:16:44.561386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:40.477788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538795+0000","last_change":"2024-04-25T10:16:43.568156+0000","last_active":"2024-04-25T10:16:44.538795+0000","last_peered":"2024-04-25T10:16:44.538795+0000","last_clean":"2024-04-25T10:16:44.538795+0000","last_became_active":"2024-04-25T10:16:43.567851+0000","last_became_peered":"2024-04-25T10:16:43.567851+0000","last_unstale":"2024-04-25T10:16:44.538795+0000","last_undegraded":"2024-04-25T10:16:44.538795+0000","last_fullsized":"2024-04-25T10:16:44.538795+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:15.868630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567095+0000","last_change":"2024-04-25T10:16:42.564913+0000","last_active":"2024-04-25T10:16:44.567095+0000","last_peered":"2024-04-25T10:16:44.567095+0000","last_clean":"2024-04-25T10:16:44.567095+0000","last_became_active":"2024-04-25T10:16:42.563056+0000","last_became_peered":"2024-04-25T10:16:42.563056+0000","last_unstale":"2024-04-25T10:16:44.567095+0000","last_undegraded":"2024-04-25T10:16:44.567095+0000","last_fullsized":"2024-04-25T10:16:44.567095+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:15.848421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539580+0000","last_change":"2024-04-25T10:16:43.555075+0000","last_active":"2024-04-25T10:16:44.539580+0000","last_peered":"2024-04-25T10:16:44.539580+0000","last_clean":"2024-04-25T10:16:44.539580+0000","last_became_active":"2024-04-25T10:16:43.554782+0000","last_became_peered":"2024-04-25T10:16:43.554782+0000","last_unstale":"2024-04-25T10:16:44.539580+0000","last_undegraded":"2024-04-25T10:16:44.539580+0000","last_fullsized":"2024-04-25T10:16:44.539580+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:30.375818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"147'6","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258556+0000","last_change":"2024-04-25T10:16:42.553024+0000","last_active":"2024-04-25T10:16:45.258556+0000","last_peered":"2024-04-25T10:16:45.258556+0000","last_clean":"2024-04-25T10:16:45.258556+0000","last_became_active":"2024-04-25T10:16:42.552733+0000","last_became_peered":"2024-04-25T10:16:42.552733+0000","last_unstale":"2024-04-25T10:16:45.258556+0000","last_undegraded":"2024-04-25T10:16:45.258556+0000","last_fullsized":"2024-04-25T10:16:45.258556+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:12.363657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567417+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:16:44.567417+0000","last_peered":"2024-04-25T10:16:44.567417+0000","last_clean":"2024-04-25T10:16:44.567417+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:16:44.567417+0000","last_undegraded":"2024-04-25T10:16:44.567417+0000","last_fullsized":"2024-04-25T10:16:44.567417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575031,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59948,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311892,"statfs":{"total":96636764160,"available":96575377408,"internally_reserved":0,"allocated":2453504,"data_stored":1462405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575031,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55208,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1859584,"data_stored":872085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55204,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1855488,"data_stored":872015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82924,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288916,"statfs":{"total":96636764160,"available":96551849984,"internally_reserved":0,"allocated":1867776,"data_stored":873627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575031,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82928,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288912,"statfs":{"total":96636764160,"available":96551845888,"internally_reserved":0,"allocated":1863680,"data_stored":872631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83032519},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55212,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316628,"statfs":{"total":96636764160,"available":96580227072,"internally_reserved":0,"allocated":1859584,"data_stored":872085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47484,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324356,"statfs":{"total":96636764160,"available":96588140544,"internally_reserved":0,"allocated":2473984,"data_stored":1464701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:48.923 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:16:49.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.160+0000 7f778d000640 1 Processor -- start 2024-04-25T10:16:49.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.161+0000 7f778d000640 1 -- start start 2024-04-25T10:16:49.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.162+0000 7f778d000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f778815dd30 0x7f77880b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.162+0000 7f778d000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77880b9770 0x7f77880d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.162+0000 7f778d000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77880d1df0 con 0x7f77880d2e90 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.162+0000 7f778d000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77880b7a50 con 0x7f778815dd30 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.162+0000 7f778d000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77880d20d0 con 0x7f77880b9770 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7785e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f778815dd30 0x7f77880b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7786800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f77880d2e90 0x7f77880b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50650/0 (socket says 172.21.15.165:50650) 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7786800640 1 -- 172.21.15.165:0/1306302411 learned_addr learned my addr 172.21.15.165:0/1306302411 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f777d400640 1 --2- 172.21.15.165:0/1306302411 >> v2:172.21.15.172:3300/0 conn(0x7f77880b9770 0x7f77880d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7785e00640 1 -- 172.21.15.165:0/1306302411 >> v2:172.21.15.172:3300/0 conn(0x7f77880b9770 msgr2=0x7f77880d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7785e00640 1 --2- 172.21.15.165:0/1306302411 >> v2:172.21.15.172:3300/0 conn(0x7f77880b9770 0x7f77880d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7785e00640 1 -- 172.21.15.165:0/1306302411 >> v1:172.21.15.165:6789/0 conn(0x7f77880d2e90 legacy=0x7f77880b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7785e00640 1 -- 172.21.15.165:0/1306302411 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77880d23b0 con 0x7f778815dd30 2024-04-25T10:16:49.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.163+0000 7f7785400640 1 -- 172.21.15.165:0/1306302411 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4122503980 0 0) 0x7f77880d1df0 con 0x7f77880d2e90 2024-04-25T10:16:49.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.164+0000 7f7785e00640 1 --2- 172.21.15.165:0/1306302411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f778815dd30 0x7f77880b9230 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f77880b6540 tx=0x7f777002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efebf72f99b68a77 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.164+0000 7f7785400640 1 -- 172.21.15.165:0/1306302411 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770037070 con 0x7f778815dd30 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.164+0000 7f7785400640 1 -- 172.21.15.165:0/1306302411 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f777002add0 con 0x7f778815dd30 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.164+0000 7f7785400640 1 -- 172.21.15.165:0/1306302411 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7770032660 con 0x7f778815dd30 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 -- 172.21.15.165:0/1306302411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f778815dd30 msgr2=0x7f77880b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 --2- 172.21.15.165:0/1306302411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f778815dd30 0x7f77880b9230 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f77880b6540 tx=0x7f777002a080 comp rx=0 tx=0).stop 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 -- 172.21.15.165:0/1306302411 shutdown_connections 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 --2- 172.21.15.165:0/1306302411 >> v2:172.21.15.172:3300/0 conn(0x7f77880b9770 0x7f77880d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 --2- 172.21.15.165:0/1306302411 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f778815dd30 0x7f77880b9230 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 -- 172.21.15.165:0/1306302411 >> 172.21.15.165:0/1306302411 conn(0x7f7788061c30 msgr2=0x7f778806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 -- 172.21.15.165:0/1306302411 shutdown_connections 2024-04-25T10:16:49.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.165+0000 7f778d000640 1 -- 172.21.15.165:0/1306302411 wait complete. 2024-04-25T10:16:49.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.167+0000 7f778d000640 1 Processor -- start 2024-04-25T10:16:49.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.167+0000 7f778d000640 1 -- start start 2024-04-25T10:16:49.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f778d000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77880b9770 0x7f7788151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f778d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 0x7f77880cce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f778d000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f778815dd30 0x7f77880cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f778d000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880d1a10 con 0x7f77880d2e90 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f778d000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77880d2970 con 0x7f77880b9770 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f778d000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77880d1b90 con 0x7f778815dd30 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f777d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 0x7f77880cce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f777d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 0x7f77880cce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52756/0 (socket says 172.21.15.165:52756) 2024-04-25T10:16:49.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.168+0000 7f777d400640 1 -- 172.21.15.165:0/1795608100 learned_addr learned my addr 172.21.15.165:0/1795608100 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:49.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f7785e00640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77880b9770 0x7f7788151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777d400640 1 -- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f778815dd30 msgr2=0x7f77880cd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777d400640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f778815dd30 0x7f77880cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777d400640 1 -- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77880b9770 msgr2=0x7f7788151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777d400640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77880b9770 0x7f7788151300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777d400640 1 -- 172.21.15.165:0/1795608100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7788151840 con 0x7f77880d2e90 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777d400640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 0x7f77880cce90 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f777800d830 tx=0x7f777800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777800b6c0 con 0x7f77880d2e90 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7778019070 con 0x7f77880d2e90 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f777800baa0 con 0x7f77880d2e90 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.169+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77881f4d50 con 0x7f77880d2e90 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.170+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f778815ff60 con 0x7f77880d2e90 2024-04-25T10:16:49.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.171+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7788073990 con 0x7f77880d2e90 2024-04-25T10:16:49.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.172+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f777801e460 con 0x7f77880d2e90 2024-04-25T10:16:49.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.172+0000 7f777ea00640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f776406b4d0 0x7f776406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.173+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f777808aa40 con 0x7f77880d2e90 2024-04-25T10:16:49.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.173+0000 7f7785e00640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f776406b4d0 0x7f776406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.173+0000 7f7785e00640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f776406b4d0 0x7f776406d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7770034000 tx=0x7f777002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.177+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7778059720 con 0x7f77880d2e90 2024-04-25T10:16:49.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.451+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f77880b8750 con 0x7f77880d2e90 2024-04-25T10:16:49.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.452+0000 7f777ea00640 1 -- 172.21.15.165:0/1795608100 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1018 (secure 0 0 0) 0x7f7778059020 con 0x7f77880d2e90 2024-04-25T10:16:49.455 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:49.456 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":910,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":148,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":510242816,"bytes_avail":772583870464,"bytes_total":773094113280,"write_bytes_sec":2388,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":101,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"c","status":"up:active","gid":9765}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":35,"modified":"2024-04-25T10:16:14.419490+0000","services":{}},"progress_events":{}} 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f776406b4d0 msgr2=0x7f776406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f776406b4d0 0x7f776406d990 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f7770034000 tx=0x7f777002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 msgr2=0x7f77880cce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 0x7f77880cce90 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f777800d830 tx=0x7f777800dcf0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 shutdown_connections 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f776406b4d0 0x7f776406d990 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.455+0000 7f778d000640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f778815dd30 0x7f77880cd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.456+0000 7f778d000640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77880d2e90 0x7f77880cce90 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.456+0000 7f778d000640 1 --2- 172.21.15.165:0/1795608100 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77880b9770 0x7f7788151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.456+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 >> 172.21.15.165:0/1795608100 conn(0x7f7788061c30 msgr2=0x7f77880d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.456+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 shutdown_connections 2024-04-25T10:16:49.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.456+0000 7f778d000640 1 -- 172.21.15.165:0/1795608100 wait complete. 2024-04-25T10:16:49.479 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:16:49.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:49.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.684+0000 7f280ec00640 1 Processor -- start 2024-04-25T10:16:49.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.684+0000 7f280ec00640 1 -- start start 2024-04-25T10:16:49.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.685+0000 7f280ec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f280815db70 0x7f28080cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.685+0000 7f280ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28080cfc20 0x7f280806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.685+0000 7f280ec00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2808071650 con 0x7f280815df50 2024-04-25T10:16:49.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.686+0000 7f280ec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28080cbd80 con 0x7f280815db70 2024-04-25T10:16:49.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.686+0000 7f280ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2808161910 con 0x7f28080cfc20 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.686+0000 7f27ff400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f280815df50 0x7f28080cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50658/0 (socket says 172.21.15.165:50658) 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.686+0000 7f2807e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f280815db70 0x7f28080cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.686+0000 7f27ff400640 1 -- 172.21.15.165:0/679036033 learned_addr learned my addr 172.21.15.165:0/679036033 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.686+0000 7f280cc00640 1 --2- 172.21.15.165:0/679036033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28080cfc20 0x7f280806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.687+0000 7f2807400640 1 -- 172.21.15.165:0/679036033 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675500253 0 0) 0x7f27f0001110 con 0x7f280815df50 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.687+0000 7f2807400640 1 -- 172.21.15.165:0/679036033 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27ec003520 con 0x7f280815df50 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.687+0000 7f2807e00640 1 -- 172.21.15.165:0/679036033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28080cfc20 msgr2=0x7f280806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.687+0000 7f2807e00640 1 --2- 172.21.15.165:0/679036033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28080cfc20 0x7f280806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.687+0000 7f2807e00640 1 -- 172.21.15.165:0/679036033 >> v1:172.21.15.165:6789/0 conn(0x7f280815df50 legacy=0x7f28080cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.687+0000 7f2807e00640 1 -- 172.21.15.165:0/679036033 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28080cf230 con 0x7f280815db70 2024-04-25T10:16:49.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.689+0000 7f2807e00640 1 --2- 172.21.15.165:0/679036033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f280815db70 0x7f28080cb840 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097a0 tx=0x7f27f802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ab3be0b927707 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.689+0000 7f2807400640 1 -- 172.21.15.165:0/679036033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8037070 con 0x7f280815db70 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.689+0000 7f2807400640 1 -- 172.21.15.165:0/679036033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f802ae50 con 0x7f280815db70 2024-04-25T10:16:49.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.689+0000 7f2807400640 1 -- 172.21.15.165:0/679036033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8032660 con 0x7f280815db70 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 -- 172.21.15.165:0/679036033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f280815db70 msgr2=0x7f28080cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 --2- 172.21.15.165:0/679036033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f280815db70 0x7f28080cb840 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f27f80097a0 tx=0x7f27f802a100 comp rx=0 tx=0).stop 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 -- 172.21.15.165:0/679036033 shutdown_connections 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 --2- 172.21.15.165:0/679036033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f28080cfc20 0x7f280806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 --2- 172.21.15.165:0/679036033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f280815db70 0x7f28080cb840 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 -- 172.21.15.165:0/679036033 >> 172.21.15.165:0/679036033 conn(0x7f280806abe0 msgr2=0x7f280806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 -- 172.21.15.165:0/679036033 shutdown_connections 2024-04-25T10:16:49.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.690+0000 7f280ec00640 1 -- 172.21.15.165:0/679036033 wait complete. 2024-04-25T10:16:49.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.692+0000 7f280ec00640 1 Processor -- start 2024-04-25T10:16:49.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.692+0000 7f280ec00640 1 -- start start 2024-04-25T10:16:49.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 0x7f28081423e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280815db70 0x7f28081f3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280815df50 0x7f2808142ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2808162120 con 0x7f28080cfc20 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2808161e40 con 0x7f280815df50 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2808162640 con 0x7f280815db70 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 0x7f28081423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 0x7f28081423e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52770/0 (socket says 172.21.15.165:52770) 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 -- 172.21.15.165:0/1447689659 learned_addr learned my addr 172.21.15.165:0/1447689659 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280cc00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280815df50 0x7f2808142ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f27ff400640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280815db70 0x7f28081f3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 -- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280815db70 msgr2=0x7f28081f3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280815db70 0x7f28081f3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 -- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280815df50 msgr2=0x7f2808142ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280815df50 0x7f2808142ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f2807e00640 1 -- 172.21.15.165:0/1447689659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808154a80 con 0x7f28080cfc20 2024-04-25T10:16:49.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f27ff400640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280815db70 0x7f28081f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:49.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.693+0000 7f280cc00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280815df50 0x7f2808142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.694+0000 7f2807e00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 0x7f28081423e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f27f8035040 tx=0x7f27f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.694+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8041070 con 0x7f28080cfc20 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.694+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f803c460 con 0x7f28080cfc20 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.694+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f8037040 con 0x7f28080cfc20 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.694+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2808154d60 con 0x7f28080cfc20 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.694+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2808155320 con 0x7f28080cfc20 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.696+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2808073990 con 0x7f28080cfc20 2024-04-25T10:16:49.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.696+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f27f803a070 con 0x7f28080cfc20 2024-04-25T10:16:49.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.697+0000 7f2804c00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.697+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f27f80aa600 con 0x7f28080cfc20 2024-04-25T10:16:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.698+0000 7f27ff400640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:49.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.698+0000 7f27ff400640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f28081f1400 tx=0x7f27f0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:49.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.702+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f27f8079250 con 0x7f28080cfc20 2024-04-25T10:16:49.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.901+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28080722b0 con 0x7f27e406b4d0 2024-04-25T10:16:49.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.919+0000 7f2804c00640 1 -- 172.21.15.165:0/1447689659 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252486 (secure 0 0 0) 0x7f28080722b0 con 0x7f27e406b4d0 2024-04-25T10:16:49.926 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:49.927 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:16:49.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 msgr2=0x7f27e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f28081f1400 tx=0x7f27f0009040 comp rx=0 tx=0).stop 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 msgr2=0x7f28081423e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 0x7f28081423e0 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x7f27f8035040 tx=0x7f27f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 shutdown_connections 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f27e406b4d0 0x7f27e406d990 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f280815df50 0x7f2808142ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f280815db70 0x7f28081f3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 --2- 172.21.15.165:0/1447689659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f28080cfc20 0x7f28081423e0 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 >> 172.21.15.165:0/1447689659 conn(0x7f280806abe0 msgr2=0x7f280815e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 shutdown_connections 2024-04-25T10:16:49.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:49.926+0000 7f280ec00640 1 -- 172.21.15.165:0/1447689659 wait complete. 2024-04-25T10:16:49.949 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":589,"stamp":"2024-04-25T10:16:48.430234+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498284,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":482419,"kb_avail":754476436,"statfs":{"total":773094113280,"available":772583870464,"internally_reserved":0,"allocated":16097280,"data_stored":8161676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":493997618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":21,"apply_latency_ms":21,"commit_latency_ns":21000000,"apply_latency_ns":21000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001884"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562120+0000","last_change":"2024-04-25T10:16:43.558546+0000","last_active":"2024-04-25T10:16:44.562120+0000","last_peered":"2024-04-25T10:16:44.562120+0000","last_clean":"2024-04-25T10:16:44.562120+0000","last_became_active":"2024-04-25T10:16:43.557592+0000","last_became_peered":"2024-04-25T10:16:43.557592+0000","last_unstale":"2024-04-25T10:16:44.562120+0000","last_undegraded":"2024-04-25T10:16:44.562120+0000","last_fullsized":"2024-04-25T10:16:44.562120+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.763212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644312+0000","last_change":"2024-04-25T10:16:43.542077+0000","last_active":"2024-04-25T10:16:44.644312+0000","last_peered":"2024-04-25T10:16:44.644312+0000","last_clean":"2024-04-25T10:16:44.644312+0000","last_became_active":"2024-04-25T10:16:43.541331+0000","last_became_peered":"2024-04-25T10:16:43.541331+0000","last_unstale":"2024-04-25T10:16:44.644312+0000","last_undegraded":"2024-04-25T10:16:44.644312+0000","last_fullsized":"2024-04-25T10:16:44.644312+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:18.077257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644302+0000","last_change":"2024-04-25T10:16:43.554057+0000","last_active":"2024-04-25T10:16:44.644302+0000","last_peered":"2024-04-25T10:16:44.644302+0000","last_clean":"2024-04-25T10:16:44.644302+0000","last_became_active":"2024-04-25T10:16:43.553688+0000","last_became_peered":"2024-04-25T10:16:43.553688+0000","last_unstale":"2024-04-25T10:16:44.644302+0000","last_undegraded":"2024-04-25T10:16:44.644302+0000","last_fullsized":"2024-04-25T10:16:44.644302+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:23.042495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536340+0000","last_change":"2024-04-25T10:16:43.553869+0000","last_active":"2024-04-25T10:16:44.536340+0000","last_peered":"2024-04-25T10:16:44.536340+0000","last_clean":"2024-04-25T10:16:44.536340+0000","last_became_active":"2024-04-25T10:16:43.552593+0000","last_became_peered":"2024-04-25T10:16:43.552593+0000","last_unstale":"2024-04-25T10:16:44.536340+0000","last_undegraded":"2024-04-25T10:16:44.536340+0000","last_fullsized":"2024-04-25T10:16:44.536340+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:38.269376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.260200+0000","last_change":"2024-04-25T10:16:43.558696+0000","last_active":"2024-04-25T10:16:45.260200+0000","last_peered":"2024-04-25T10:16:45.260200+0000","last_clean":"2024-04-25T10:16:45.260200+0000","last_became_active":"2024-04-25T10:16:43.554924+0000","last_became_peered":"2024-04-25T10:16:43.554924+0000","last_unstale":"2024-04-25T10:16:45.260200+0000","last_undegraded":"2024-04-25T10:16:45.260200+0000","last_fullsized":"2024-04-25T10:16:45.260200+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:27.953723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259886+0000","last_change":"2024-04-25T10:16:43.553964+0000","last_active":"2024-04-25T10:16:45.259886+0000","last_peered":"2024-04-25T10:16:45.259886+0000","last_clean":"2024-04-25T10:16:45.259886+0000","last_became_active":"2024-04-25T10:16:43.553667+0000","last_became_peered":"2024-04-25T10:16:43.553667+0000","last_unstale":"2024-04-25T10:16:45.259886+0000","last_undegraded":"2024-04-25T10:16:45.259886+0000","last_fullsized":"2024-04-25T10:16:45.259886+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:48.946817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561293+0000","last_change":"2024-04-25T10:16:43.554632+0000","last_active":"2024-04-25T10:16:44.561293+0000","last_peered":"2024-04-25T10:16:44.561293+0000","last_clean":"2024-04-25T10:16:44.561293+0000","last_became_active":"2024-04-25T10:16:43.554371+0000","last_became_peered":"2024-04-25T10:16:43.554371+0000","last_unstale":"2024-04-25T10:16:44.561293+0000","last_undegraded":"2024-04-25T10:16:44.561293+0000","last_fullsized":"2024-04-25T10:16:44.561293+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.084637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537189+0000","last_change":"2024-04-25T10:16:43.563238+0000","last_active":"2024-04-25T10:16:44.537189+0000","last_peered":"2024-04-25T10:16:44.537189+0000","last_clean":"2024-04-25T10:16:44.537189+0000","last_became_active":"2024-04-25T10:16:43.562888+0000","last_became_peered":"2024-04-25T10:16:43.562888+0000","last_unstale":"2024-04-25T10:16:44.537189+0000","last_undegraded":"2024-04-25T10:16:44.537189+0000","last_fullsized":"2024-04-25T10:16:44.537189+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:34.293550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259043+0000","last_change":"2024-04-25T10:16:43.557253+0000","last_active":"2024-04-25T10:16:45.259043+0000","last_peered":"2024-04-25T10:16:45.259043+0000","last_clean":"2024-04-25T10:16:45.259043+0000","last_became_active":"2024-04-25T10:16:43.556951+0000","last_became_peered":"2024-04-25T10:16:43.556951+0000","last_unstale":"2024-04-25T10:16:45.259043+0000","last_undegraded":"2024-04-25T10:16:45.259043+0000","last_fullsized":"2024-04-25T10:16:45.259043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.380192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259383+0000","last_change":"2024-04-25T10:16:43.558625+0000","last_active":"2024-04-25T10:16:45.259383+0000","last_peered":"2024-04-25T10:16:45.259383+0000","last_clean":"2024-04-25T10:16:45.259383+0000","last_became_active":"2024-04-25T10:16:43.558255+0000","last_became_peered":"2024-04-25T10:16:43.558255+0000","last_unstale":"2024-04-25T10:16:45.259383+0000","last_undegraded":"2024-04-25T10:16:45.259383+0000","last_fullsized":"2024-04-25T10:16:45.259383+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.033238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536630+0000","last_change":"2024-04-25T10:16:43.557356+0000","last_active":"2024-04-25T10:16:44.536630+0000","last_peered":"2024-04-25T10:16:44.536630+0000","last_clean":"2024-04-25T10:16:44.536630+0000","last_became_active":"2024-04-25T10:16:43.557009+0000","last_became_peered":"2024-04-25T10:16:43.557009+0000","last_unstale":"2024-04-25T10:16:44.536630+0000","last_undegraded":"2024-04-25T10:16:44.536630+0000","last_fullsized":"2024-04-25T10:16:44.536630+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:40.022950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537537+0000","last_change":"2024-04-25T10:16:43.559117+0000","last_active":"2024-04-25T10:16:44.537537+0000","last_peered":"2024-04-25T10:16:44.537537+0000","last_clean":"2024-04-25T10:16:44.537537+0000","last_became_active":"2024-04-25T10:16:43.558792+0000","last_became_peered":"2024-04-25T10:16:43.558792+0000","last_unstale":"2024-04-25T10:16:44.537537+0000","last_undegraded":"2024-04-25T10:16:44.537537+0000","last_fullsized":"2024-04-25T10:16:44.537537+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:41.494324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259928+0000","last_change":"2024-04-25T10:16:43.559623+0000","last_active":"2024-04-25T10:16:45.259928+0000","last_peered":"2024-04-25T10:16:45.259928+0000","last_clean":"2024-04-25T10:16:45.259928+0000","last_became_active":"2024-04-25T10:16:43.559334+0000","last_became_peered":"2024-04-25T10:16:43.559334+0000","last_unstale":"2024-04-25T10:16:45.259928+0000","last_undegraded":"2024-04-25T10:16:45.259928+0000","last_fullsized":"2024-04-25T10:16:45.259928+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:43.288992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536382+0000","last_change":"2024-04-25T10:16:43.557501+0000","last_active":"2024-04-25T10:16:44.536382+0000","last_peered":"2024-04-25T10:16:44.536382+0000","last_clean":"2024-04-25T10:16:44.536382+0000","last_became_active":"2024-04-25T10:16:43.557212+0000","last_became_peered":"2024-04-25T10:16:43.557212+0000","last_unstale":"2024-04-25T10:16:44.536382+0000","last_undegraded":"2024-04-25T10:16:44.536382+0000","last_fullsized":"2024-04-25T10:16:44.536382+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:12.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644124+0000","last_change":"2024-04-25T10:16:43.547840+0000","last_active":"2024-04-25T10:16:44.644124+0000","last_peered":"2024-04-25T10:16:44.644124+0000","last_clean":"2024-04-25T10:16:44.644124+0000","last_became_active":"2024-04-25T10:16:43.544427+0000","last_became_peered":"2024-04-25T10:16:43.544427+0000","last_unstale":"2024-04-25T10:16:44.644124+0000","last_undegraded":"2024-04-25T10:16:44.644124+0000","last_fullsized":"2024-04-25T10:16:44.644124+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.009160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258182+0000","last_change":"2024-04-25T10:16:43.562196+0000","last_active":"2024-04-25T10:16:45.258182+0000","last_peered":"2024-04-25T10:16:45.258182+0000","last_clean":"2024-04-25T10:16:45.258182+0000","last_became_active":"2024-04-25T10:16:43.561890+0000","last_became_peered":"2024-04-25T10:16:43.561890+0000","last_unstale":"2024-04-25T10:16:45.258182+0000","last_undegraded":"2024-04-25T10:16:45.258182+0000","last_fullsized":"2024-04-25T10:16:45.258182+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:40.034886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539779+0000","last_change":"2024-04-25T10:16:43.555511+0000","last_active":"2024-04-25T10:16:44.539779+0000","last_peered":"2024-04-25T10:16:44.539779+0000","last_clean":"2024-04-25T10:16:44.539779+0000","last_became_active":"2024-04-25T10:16:43.555209+0000","last_became_peered":"2024-04-25T10:16:43.555209+0000","last_unstale":"2024-04-25T10:16:44.539779+0000","last_undegraded":"2024-04-25T10:16:44.539779+0000","last_fullsized":"2024-04-25T10:16:44.539779+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:27.563607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536915+0000","last_change":"2024-04-25T10:16:43.557716+0000","last_active":"2024-04-25T10:16:44.536915+0000","last_peered":"2024-04-25T10:16:44.536915+0000","last_clean":"2024-04-25T10:16:44.536915+0000","last_became_active":"2024-04-25T10:16:43.557442+0000","last_became_peered":"2024-04-25T10:16:43.557442+0000","last_unstale":"2024-04-25T10:16:44.536915+0000","last_undegraded":"2024-04-25T10:16:44.536915+0000","last_fullsized":"2024-04-25T10:16:44.536915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.240799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567976+0000","last_change":"2024-04-25T10:16:43.559780+0000","last_active":"2024-04-25T10:16:44.567976+0000","last_peered":"2024-04-25T10:16:44.567976+0000","last_clean":"2024-04-25T10:16:44.567976+0000","last_became_active":"2024-04-25T10:16:43.559507+0000","last_became_peered":"2024-04-25T10:16:43.559507+0000","last_unstale":"2024-04-25T10:16:44.567976+0000","last_undegraded":"2024-04-25T10:16:44.567976+0000","last_fullsized":"2024-04-25T10:16:44.567976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:23.418603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258576+0000","last_change":"2024-04-25T10:16:43.544613+0000","last_active":"2024-04-25T10:16:45.258576+0000","last_peered":"2024-04-25T10:16:45.258576+0000","last_clean":"2024-04-25T10:16:45.258576+0000","last_became_active":"2024-04-25T10:16:43.544181+0000","last_became_peered":"2024-04-25T10:16:43.544181+0000","last_unstale":"2024-04-25T10:16:45.258576+0000","last_undegraded":"2024-04-25T10:16:45.258576+0000","last_fullsized":"2024-04-25T10:16:45.258576+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:43.288992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259936+0000","last_change":"2024-04-25T10:16:43.565132+0000","last_active":"2024-04-25T10:16:45.259936+0000","last_peered":"2024-04-25T10:16:45.259936+0000","last_clean":"2024-04-25T10:16:45.259936+0000","last_became_active":"2024-04-25T10:16:43.564843+0000","last_became_peered":"2024-04-25T10:16:43.564843+0000","last_unstale":"2024-04-25T10:16:45.259936+0000","last_undegraded":"2024-04-25T10:16:45.259936+0000","last_fullsized":"2024-04-25T10:16:45.259936+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:57.505598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561287+0000","last_change":"2024-04-25T10:16:43.552626+0000","last_active":"2024-04-25T10:16:44.561287+0000","last_peered":"2024-04-25T10:16:44.561287+0000","last_clean":"2024-04-25T10:16:44.561287+0000","last_became_active":"2024-04-25T10:16:43.551212+0000","last_became_peered":"2024-04-25T10:16:43.551212+0000","last_unstale":"2024-04-25T10:16:44.561287+0000","last_undegraded":"2024-04-25T10:16:44.561287+0000","last_fullsized":"2024-04-25T10:16:44.561287+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.189603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538152+0000","last_change":"2024-04-25T10:16:43.556110+0000","last_active":"2024-04-25T10:16:44.538152+0000","last_peered":"2024-04-25T10:16:44.538152+0000","last_clean":"2024-04-25T10:16:44.538152+0000","last_became_active":"2024-04-25T10:16:43.555755+0000","last_became_peered":"2024-04-25T10:16:43.555755+0000","last_unstale":"2024-04-25T10:16:44.538152+0000","last_undegraded":"2024-04-25T10:16:44.538152+0000","last_fullsized":"2024-04-25T10:16:44.538152+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.966214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536433+0000","last_change":"2024-04-25T10:16:43.561869+0000","last_active":"2024-04-25T10:16:44.536433+0000","last_peered":"2024-04-25T10:16:44.536433+0000","last_clean":"2024-04-25T10:16:44.536433+0000","last_became_active":"2024-04-25T10:16:43.561583+0000","last_became_peered":"2024-04-25T10:16:43.561583+0000","last_unstale":"2024-04-25T10:16:44.536433+0000","last_undegraded":"2024-04-25T10:16:44.536433+0000","last_fullsized":"2024-04-25T10:16:44.536433+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:25.788606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562314+0000","last_change":"2024-04-25T10:16:43.546831+0000","last_active":"2024-04-25T10:16:44.562314+0000","last_peered":"2024-04-25T10:16:44.562314+0000","last_clean":"2024-04-25T10:16:44.562314+0000","last_became_active":"2024-04-25T10:16:43.546419+0000","last_became_peered":"2024-04-25T10:16:43.546419+0000","last_unstale":"2024-04-25T10:16:44.562314+0000","last_undegraded":"2024-04-25T10:16:44.562314+0000","last_fullsized":"2024-04-25T10:16:44.562314+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:51.069413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561846+0000","last_change":"2024-04-25T10:16:43.553554+0000","last_active":"2024-04-25T10:16:44.561846+0000","last_peered":"2024-04-25T10:16:44.561846+0000","last_clean":"2024-04-25T10:16:44.561846+0000","last_became_active":"2024-04-25T10:16:43.553131+0000","last_became_peered":"2024-04-25T10:16:43.553131+0000","last_unstale":"2024-04-25T10:16:44.561846+0000","last_undegraded":"2024-04-25T10:16:44.561846+0000","last_fullsized":"2024-04-25T10:16:44.561846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:18.077257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258531+0000","last_change":"2024-04-25T10:16:43.560703+0000","last_active":"2024-04-25T10:16:45.258531+0000","last_peered":"2024-04-25T10:16:45.258531+0000","last_clean":"2024-04-25T10:16:45.258531+0000","last_became_active":"2024-04-25T10:16:43.560398+0000","last_became_peered":"2024-04-25T10:16:43.560398+0000","last_unstale":"2024-04-25T10:16:45.258531+0000","last_undegraded":"2024-04-25T10:16:45.258531+0000","last_fullsized":"2024-04-25T10:16:45.258531+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:10.354287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536983+0000","last_change":"2024-04-25T10:16:43.554470+0000","last_active":"2024-04-25T10:16:44.536983+0000","last_peered":"2024-04-25T10:16:44.536983+0000","last_clean":"2024-04-25T10:16:44.536983+0000","last_became_active":"2024-04-25T10:16:43.554142+0000","last_became_peered":"2024-04-25T10:16:43.554142+0000","last_unstale":"2024-04-25T10:16:44.536983+0000","last_undegraded":"2024-04-25T10:16:44.536983+0000","last_fullsized":"2024-04-25T10:16:44.536983+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:40.366765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538948+0000","last_change":"2024-04-25T10:16:43.568131+0000","last_active":"2024-04-25T10:16:44.538948+0000","last_peered":"2024-04-25T10:16:44.538948+0000","last_clean":"2024-04-25T10:16:44.538948+0000","last_became_active":"2024-04-25T10:16:43.567824+0000","last_became_peered":"2024-04-25T10:16:43.567824+0000","last_unstale":"2024-04-25T10:16:44.538948+0000","last_undegraded":"2024-04-25T10:16:44.538948+0000","last_fullsized":"2024-04-25T10:16:44.538948+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537321+0000","last_change":"2024-04-25T10:16:43.547282+0000","last_active":"2024-04-25T10:16:44.537321+0000","last_peered":"2024-04-25T10:16:44.537321+0000","last_clean":"2024-04-25T10:16:44.537321+0000","last_became_active":"2024-04-25T10:16:43.545939+0000","last_became_peered":"2024-04-25T10:16:43.545939+0000","last_unstale":"2024-04-25T10:16:44.537321+0000","last_undegraded":"2024-04-25T10:16:44.537321+0000","last_fullsized":"2024-04-25T10:16:44.537321+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:10.488478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258369+0000","last_change":"2024-04-25T10:16:43.558141+0000","last_active":"2024-04-25T10:16:45.258369+0000","last_peered":"2024-04-25T10:16:45.258369+0000","last_clean":"2024-04-25T10:16:45.258369+0000","last_became_active":"2024-04-25T10:16:43.557866+0000","last_became_peered":"2024-04-25T10:16:43.557866+0000","last_unstale":"2024-04-25T10:16:45.258369+0000","last_undegraded":"2024-04-25T10:16:45.258369+0000","last_fullsized":"2024-04-25T10:16:45.258369+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:20.806187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537245+0000","last_change":"2024-04-25T10:16:43.558128+0000","last_active":"2024-04-25T10:16:44.537245+0000","last_peered":"2024-04-25T10:16:44.537245+0000","last_clean":"2024-04-25T10:16:44.537245+0000","last_became_active":"2024-04-25T10:16:43.557811+0000","last_became_peered":"2024-04-25T10:16:43.557811+0000","last_unstale":"2024-04-25T10:16:44.537245+0000","last_undegraded":"2024-04-25T10:16:44.537245+0000","last_fullsized":"2024-04-25T10:16:44.537245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:15.502788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644518+0000","last_change":"2024-04-25T10:16:43.557078+0000","last_active":"2024-04-25T10:16:44.644518+0000","last_peered":"2024-04-25T10:16:44.644518+0000","last_clean":"2024-04-25T10:16:44.644518+0000","last_became_active":"2024-04-25T10:16:43.556763+0000","last_became_peered":"2024-04-25T10:16:43.556763+0000","last_unstale":"2024-04-25T10:16:44.644518+0000","last_undegraded":"2024-04-25T10:16:44.644518+0000","last_fullsized":"2024-04-25T10:16:44.644518+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:51.069413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"147'4","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259562+0000","last_change":"2024-04-25T10:16:42.551717+0000","last_active":"2024-04-25T10:16:45.259562+0000","last_peered":"2024-04-25T10:16:45.259562+0000","last_clean":"2024-04-25T10:16:45.259562+0000","last_became_active":"2024-04-25T10:16:42.551437+0000","last_became_peered":"2024-04-25T10:16:42.551437+0000","last_unstale":"2024-04-25T10:16:45.259562+0000","last_undegraded":"2024-04-25T10:16:45.259562+0000","last_fullsized":"2024-04-25T10:16:45.259562+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:40.073529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644386+0000","last_change":"2024-04-25T10:16:43.543406+0000","last_active":"2024-04-25T10:16:44.644386+0000","last_peered":"2024-04-25T10:16:44.644386+0000","last_clean":"2024-04-25T10:16:44.644386+0000","last_became_active":"2024-04-25T10:16:43.542934+0000","last_became_peered":"2024-04-25T10:16:43.542934+0000","last_unstale":"2024-04-25T10:16:44.644386+0000","last_undegraded":"2024-04-25T10:16:44.644386+0000","last_fullsized":"2024-04-25T10:16:44.644386+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.763212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567363+0000","last_change":"2024-04-25T10:16:42.571780+0000","last_active":"2024-04-25T10:16:44.567363+0000","last_peered":"2024-04-25T10:16:44.567363+0000","last_clean":"2024-04-25T10:16:44.567363+0000","last_became_active":"2024-04-25T10:16:42.571355+0000","last_became_peered":"2024-04-25T10:16:42.571355+0000","last_unstale":"2024-04-25T10:16:44.567363+0000","last_undegraded":"2024-04-25T10:16:44.567363+0000","last_fullsized":"2024-04-25T10:16:44.567363+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.102371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538914+0000","last_change":"2024-04-25T10:16:43.559864+0000","last_active":"2024-04-25T10:16:44.538914+0000","last_peered":"2024-04-25T10:16:44.538914+0000","last_clean":"2024-04-25T10:16:44.538914+0000","last_became_active":"2024-04-25T10:16:43.559490+0000","last_became_peered":"2024-04-25T10:16:43.559490+0000","last_unstale":"2024-04-25T10:16:44.538914+0000","last_undegraded":"2024-04-25T10:16:44.538914+0000","last_fullsized":"2024-04-25T10:16:44.538914+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:09.481790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568117+0000","last_change":"2024-04-25T10:16:42.571044+0000","last_active":"2024-04-25T10:16:44.568117+0000","last_peered":"2024-04-25T10:16:44.568117+0000","last_clean":"2024-04-25T10:16:44.568117+0000","last_became_active":"2024-04-25T10:16:42.570593+0000","last_became_peered":"2024-04-25T10:16:42.570593+0000","last_unstale":"2024-04-25T10:16:44.568117+0000","last_undegraded":"2024-04-25T10:16:44.568117+0000","last_fullsized":"2024-04-25T10:16:44.568117+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:40.073529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567879+0000","last_change":"2024-04-25T10:16:43.550656+0000","last_active":"2024-04-25T10:16:44.567879+0000","last_peered":"2024-04-25T10:16:44.567879+0000","last_clean":"2024-04-25T10:16:44.567879+0000","last_became_active":"2024-04-25T10:16:43.550317+0000","last_became_peered":"2024-04-25T10:16:43.550317+0000","last_unstale":"2024-04-25T10:16:44.567879+0000","last_undegraded":"2024-04-25T10:16:44.567879+0000","last_fullsized":"2024-04-25T10:16:44.567879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.020831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644492+0000","last_change":"2024-04-25T10:16:42.553654+0000","last_active":"2024-04-25T10:16:44.644492+0000","last_peered":"2024-04-25T10:16:44.644492+0000","last_clean":"2024-04-25T10:16:44.644492+0000","last_became_active":"2024-04-25T10:16:42.553115+0000","last_became_peered":"2024-04-25T10:16:42.553115+0000","last_unstale":"2024-04-25T10:16:44.644492+0000","last_undegraded":"2024-04-25T10:16:44.644492+0000","last_fullsized":"2024-04-25T10:16:44.644492+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:57.834058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.257865+0000","last_change":"2024-04-25T10:16:43.558036+0000","last_active":"2024-04-25T10:16:45.257865+0000","last_peered":"2024-04-25T10:16:45.257865+0000","last_clean":"2024-04-25T10:16:45.257865+0000","last_became_active":"2024-04-25T10:16:43.557754+0000","last_became_peered":"2024-04-25T10:16:43.557754+0000","last_unstale":"2024-04-25T10:16:45.257865+0000","last_undegraded":"2024-04-25T10:16:45.257865+0000","last_fullsized":"2024-04-25T10:16:45.257865+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:14.630997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538931+0000","last_change":"2024-04-25T10:16:42.556137+0000","last_active":"2024-04-25T10:16:44.538931+0000","last_peered":"2024-04-25T10:16:44.538931+0000","last_clean":"2024-04-25T10:16:44.538931+0000","last_became_active":"2024-04-25T10:16:42.555806+0000","last_became_peered":"2024-04-25T10:16:42.555806+0000","last_unstale":"2024-04-25T10:16:44.538931+0000","last_undegraded":"2024-04-25T10:16:44.538931+0000","last_fullsized":"2024-04-25T10:16:44.538931+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:31.761423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.535934+0000","last_change":"2024-04-25T10:16:43.554154+0000","last_active":"2024-04-25T10:16:44.535934+0000","last_peered":"2024-04-25T10:16:44.535934+0000","last_clean":"2024-04-25T10:16:44.535934+0000","last_became_active":"2024-04-25T10:16:43.552635+0000","last_became_peered":"2024-04-25T10:16:43.552635+0000","last_unstale":"2024-04-25T10:16:44.535934+0000","last_undegraded":"2024-04-25T10:16:44.535934+0000","last_fullsized":"2024-04-25T10:16:44.535934+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:39.834167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259987+0000","last_change":"2024-04-25T10:16:42.557166+0000","last_active":"2024-04-25T10:16:45.259987+0000","last_peered":"2024-04-25T10:16:45.259987+0000","last_clean":"2024-04-25T10:16:45.259987+0000","last_became_active":"2024-04-25T10:16:42.556841+0000","last_became_peered":"2024-04-25T10:16:42.556841+0000","last_unstale":"2024-04-25T10:16:45.259987+0000","last_undegraded":"2024-04-25T10:16:45.259987+0000","last_fullsized":"2024-04-25T10:16:45.259987+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:28.174820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644268+0000","last_change":"2024-04-25T10:16:43.541816+0000","last_active":"2024-04-25T10:16:44.644268+0000","last_peered":"2024-04-25T10:16:44.644268+0000","last_clean":"2024-04-25T10:16:44.644268+0000","last_became_active":"2024-04-25T10:16:43.541356+0000","last_became_peered":"2024-04-25T10:16:43.541356+0000","last_unstale":"2024-04-25T10:16:44.644268+0000","last_undegraded":"2024-04-25T10:16:44.644268+0000","last_fullsized":"2024-04-25T10:16:44.644268+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.189603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.560999+0000","last_change":"2024-04-25T10:16:42.562767+0000","last_active":"2024-04-25T10:16:44.560999+0000","last_peered":"2024-04-25T10:16:44.560999+0000","last_clean":"2024-04-25T10:16:44.560999+0000","last_became_active":"2024-04-25T10:16:42.562417+0000","last_became_peered":"2024-04-25T10:16:42.562417+0000","last_unstale":"2024-04-25T10:16:44.560999+0000","last_undegraded":"2024-04-25T10:16:44.560999+0000","last_fullsized":"2024-04-25T10:16:44.560999+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.722759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561472+0000","last_change":"2024-04-25T10:16:43.558917+0000","last_active":"2024-04-25T10:16:44.561472+0000","last_peered":"2024-04-25T10:16:44.561472+0000","last_clean":"2024-04-25T10:16:44.561472+0000","last_became_active":"2024-04-25T10:16:43.558613+0000","last_became_peered":"2024-04-25T10:16:43.558613+0000","last_unstale":"2024-04-25T10:16:44.561472+0000","last_undegraded":"2024-04-25T10:16:44.561472+0000","last_fullsized":"2024-04-25T10:16:44.561472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:57.980294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258330+0000","last_change":"2024-04-25T10:16:42.566952+0000","last_active":"2024-04-25T10:16:45.258330+0000","last_peered":"2024-04-25T10:16:45.258330+0000","last_clean":"2024-04-25T10:16:45.258330+0000","last_became_active":"2024-04-25T10:16:42.566676+0000","last_became_peered":"2024-04-25T10:16:42.566676+0000","last_unstale":"2024-04-25T10:16:45.258330+0000","last_undegraded":"2024-04-25T10:16:45.258330+0000","last_fullsized":"2024-04-25T10:16:45.258330+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:59.508460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539287+0000","last_change":"2024-04-25T10:16:43.561379+0000","last_active":"2024-04-25T10:16:44.539287+0000","last_peered":"2024-04-25T10:16:44.539287+0000","last_clean":"2024-04-25T10:16:44.539287+0000","last_became_active":"2024-04-25T10:16:43.561087+0000","last_became_peered":"2024-04-25T10:16:43.561087+0000","last_unstale":"2024-04-25T10:16:44.539287+0000","last_undegraded":"2024-04-25T10:16:44.539287+0000","last_fullsized":"2024-04-25T10:16:44.539287+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:23.414369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568011+0000","last_change":"2024-04-25T10:16:42.569373+0000","last_active":"2024-04-25T10:16:44.568011+0000","last_peered":"2024-04-25T10:16:44.568011+0000","last_clean":"2024-04-25T10:16:44.568011+0000","last_became_active":"2024-04-25T10:16:42.568629+0000","last_became_peered":"2024-04-25T10:16:42.568629+0000","last_unstale":"2024-04-25T10:16:44.568011+0000","last_undegraded":"2024-04-25T10:16:44.568011+0000","last_fullsized":"2024-04-25T10:16:44.568011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:59.508460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536837+0000","last_change":"2024-04-25T10:16:43.551780+0000","last_active":"2024-04-25T10:16:44.536837+0000","last_peered":"2024-04-25T10:16:44.536837+0000","last_clean":"2024-04-25T10:16:44.536837+0000","last_became_active":"2024-04-25T10:16:43.551458+0000","last_became_peered":"2024-04-25T10:16:43.551458+0000","last_unstale":"2024-04-25T10:16:44.536837+0000","last_undegraded":"2024-04-25T10:16:44.536837+0000","last_fullsized":"2024-04-25T10:16:44.536837+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:13.485034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562222+0000","last_change":"2024-04-25T10:16:42.566623+0000","last_active":"2024-04-25T10:16:44.562222+0000","last_peered":"2024-04-25T10:16:44.562222+0000","last_clean":"2024-04-25T10:16:44.562222+0000","last_became_active":"2024-04-25T10:16:42.566064+0000","last_became_peered":"2024-04-25T10:16:42.566064+0000","last_unstale":"2024-04-25T10:16:44.562222+0000","last_undegraded":"2024-04-25T10:16:44.562222+0000","last_fullsized":"2024-04-25T10:16:44.562222+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:37.252841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537203+0000","last_change":"2024-04-25T10:16:43.557534+0000","last_active":"2024-04-25T10:16:44.537203+0000","last_peered":"2024-04-25T10:16:44.537203+0000","last_clean":"2024-04-25T10:16:44.537203+0000","last_became_active":"2024-04-25T10:16:43.557189+0000","last_became_peered":"2024-04-25T10:16:43.557189+0000","last_unstale":"2024-04-25T10:16:44.537203+0000","last_undegraded":"2024-04-25T10:16:44.537203+0000","last_fullsized":"2024-04-25T10:16:44.537203+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:00.427901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"147'3","reported_seq":17,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536733+0000","last_change":"2024-04-25T10:16:42.549331+0000","last_active":"2024-04-25T10:16:44.536733+0000","last_peered":"2024-04-25T10:16:44.536733+0000","last_clean":"2024-04-25T10:16:44.536733+0000","last_became_active":"2024-04-25T10:16:42.549027+0000","last_became_peered":"2024-04-25T10:16:42.549027+0000","last_unstale":"2024-04-25T10:16:44.536733+0000","last_undegraded":"2024-04-25T10:16:44.536733+0000","last_fullsized":"2024-04-25T10:16:44.536733+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:26.406784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537764+0000","last_change":"2024-04-25T10:16:43.555391+0000","last_active":"2024-04-25T10:16:44.537764+0000","last_peered":"2024-04-25T10:16:44.537764+0000","last_clean":"2024-04-25T10:16:44.537764+0000","last_became_active":"2024-04-25T10:16:43.554763+0000","last_became_peered":"2024-04-25T10:16:43.554763+0000","last_unstale":"2024-04-25T10:16:44.537764+0000","last_undegraded":"2024-04-25T10:16:44.537764+0000","last_fullsized":"2024-04-25T10:16:44.537764+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:53.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.260077+0000","last_change":"2024-04-25T10:16:42.556479+0000","last_active":"2024-04-25T10:16:45.260077+0000","last_peered":"2024-04-25T10:16:45.260077+0000","last_clean":"2024-04-25T10:16:45.260077+0000","last_became_active":"2024-04-25T10:16:42.556163+0000","last_became_peered":"2024-04-25T10:16:42.556163+0000","last_unstale":"2024-04-25T10:16:45.260077+0000","last_undegraded":"2024-04-25T10:16:45.260077+0000","last_fullsized":"2024-04-25T10:16:45.260077+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:15.848421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644123+0000","last_change":"2024-04-25T10:16:43.548087+0000","last_active":"2024-04-25T10:16:44.644123+0000","last_peered":"2024-04-25T10:16:44.644123+0000","last_clean":"2024-04-25T10:16:44.644123+0000","last_became_active":"2024-04-25T10:16:43.545124+0000","last_became_peered":"2024-04-25T10:16:43.545124+0000","last_unstale":"2024-04-25T10:16:44.644123+0000","last_undegraded":"2024-04-25T10:16:44.644123+0000","last_fullsized":"2024-04-25T10:16:44.644123+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:52.071878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537171+0000","last_change":"2024-04-25T10:16:42.562159+0000","last_active":"2024-04-25T10:16:44.537171+0000","last_peered":"2024-04-25T10:16:44.537171+0000","last_clean":"2024-04-25T10:16:44.537171+0000","last_became_active":"2024-04-25T10:16:42.561883+0000","last_became_peered":"2024-04-25T10:16:42.561883+0000","last_unstale":"2024-04-25T10:16:44.537171+0000","last_undegraded":"2024-04-25T10:16:44.537171+0000","last_fullsized":"2024-04-25T10:16:44.537171+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.486253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259922+0000","last_change":"2024-04-25T10:16:43.553742+0000","last_active":"2024-04-25T10:16:45.259922+0000","last_peered":"2024-04-25T10:16:45.259922+0000","last_clean":"2024-04-25T10:16:45.259922+0000","last_became_active":"2024-04-25T10:16:43.553465+0000","last_became_peered":"2024-04-25T10:16:43.553465+0000","last_unstale":"2024-04-25T10:16:45.259922+0000","last_undegraded":"2024-04-25T10:16:45.259922+0000","last_fullsized":"2024-04-25T10:16:45.259922+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:14.630997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536152+0000","last_change":"2024-04-25T10:16:42.552024+0000","last_active":"2024-04-25T10:16:44.536152+0000","last_peered":"2024-04-25T10:16:44.536152+0000","last_clean":"2024-04-25T10:16:44.536152+0000","last_became_active":"2024-04-25T10:16:42.550839+0000","last_became_peered":"2024-04-25T10:16:42.550839+0000","last_unstale":"2024-04-25T10:16:44.536152+0000","last_undegraded":"2024-04-25T10:16:44.536152+0000","last_fullsized":"2024-04-25T10:16:44.536152+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:05.849811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644086+0000","last_change":"2024-04-25T10:16:43.560677+0000","last_active":"2024-04-25T10:16:44.644086+0000","last_peered":"2024-04-25T10:16:44.644086+0000","last_clean":"2024-04-25T10:16:44.644086+0000","last_became_active":"2024-04-25T10:16:43.560405+0000","last_became_peered":"2024-04-25T10:16:43.560405+0000","last_unstale":"2024-04-25T10:16:44.644086+0000","last_undegraded":"2024-04-25T10:16:44.644086+0000","last_fullsized":"2024-04-25T10:16:44.644086+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.735033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"147'8","reported_seq":22,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.560671+0000","last_change":"2024-04-25T10:16:42.566140+0000","last_active":"2024-04-25T10:16:44.560671+0000","last_peered":"2024-04-25T10:16:44.560671+0000","last_clean":"2024-04-25T10:16:44.560671+0000","last_became_active":"2024-04-25T10:16:42.565757+0000","last_became_peered":"2024-04-25T10:16:42.565757+0000","last_unstale":"2024-04-25T10:16:44.560671+0000","last_undegraded":"2024-04-25T10:16:44.560671+0000","last_fullsized":"2024-04-25T10:16:44.560671+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:09.589446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567743+0000","last_change":"2024-04-25T10:16:43.564528+0000","last_active":"2024-04-25T10:16:44.567743+0000","last_peered":"2024-04-25T10:16:44.567743+0000","last_clean":"2024-04-25T10:16:44.567743+0000","last_became_active":"2024-04-25T10:16:43.563011+0000","last_became_peered":"2024-04-25T10:16:43.563011+0000","last_unstale":"2024-04-25T10:16:44.567743+0000","last_undegraded":"2024-04-25T10:16:44.567743+0000","last_fullsized":"2024-04-25T10:16:44.567743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:00.010983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537170+0000","last_change":"2024-04-25T10:16:42.561495+0000","last_active":"2024-04-25T10:16:44.537170+0000","last_peered":"2024-04-25T10:16:44.537170+0000","last_clean":"2024-04-25T10:16:44.537170+0000","last_became_active":"2024-04-25T10:16:42.560241+0000","last_became_peered":"2024-04-25T10:16:42.560241+0000","last_unstale":"2024-04-25T10:16:44.537170+0000","last_undegraded":"2024-04-25T10:16:44.537170+0000","last_fullsized":"2024-04-25T10:16:44.537170+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:11.915069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568090+0000","last_change":"2024-04-25T10:16:43.557795+0000","last_active":"2024-04-25T10:16:44.568090+0000","last_peered":"2024-04-25T10:16:44.568090+0000","last_clean":"2024-04-25T10:16:44.568090+0000","last_became_active":"2024-04-25T10:16:43.557521+0000","last_became_peered":"2024-04-25T10:16:43.557521+0000","last_unstale":"2024-04-25T10:16:44.568090+0000","last_undegraded":"2024-04-25T10:16:44.568090+0000","last_fullsized":"2024-04-25T10:16:44.568090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:13.810505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567675+0000","last_change":"2024-04-25T10:16:42.564071+0000","last_active":"2024-04-25T10:16:44.567675+0000","last_peered":"2024-04-25T10:16:44.567675+0000","last_clean":"2024-04-25T10:16:44.567675+0000","last_became_active":"2024-04-25T10:16:42.563824+0000","last_became_peered":"2024-04-25T10:16:42.563824+0000","last_unstale":"2024-04-25T10:16:44.567675+0000","last_undegraded":"2024-04-25T10:16:44.567675+0000","last_fullsized":"2024-04-25T10:16:44.567675+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:28.174820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562102+0000","last_change":"2024-04-25T10:16:43.557697+0000","last_active":"2024-04-25T10:16:44.562102+0000","last_peered":"2024-04-25T10:16:44.562102+0000","last_clean":"2024-04-25T10:16:44.562102+0000","last_became_active":"2024-04-25T10:16:43.555480+0000","last_became_peered":"2024-04-25T10:16:43.555480+0000","last_unstale":"2024-04-25T10:16:44.562102+0000","last_undegraded":"2024-04-25T10:16:44.562102+0000","last_fullsized":"2024-04-25T10:16:44.562102+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:45.108391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644365+0000","last_change":"2024-04-25T10:16:42.555771+0000","last_active":"2024-04-25T10:16:44.644365+0000","last_peered":"2024-04-25T10:16:44.644365+0000","last_clean":"2024-04-25T10:16:44.644365+0000","last_became_active":"2024-04-25T10:16:42.555336+0000","last_became_peered":"2024-04-25T10:16:42.555336+0000","last_unstale":"2024-04-25T10:16:44.644365+0000","last_undegraded":"2024-04-25T10:16:44.644365+0000","last_fullsized":"2024-04-25T10:16:44.644365+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:22.397834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562488+0000","last_change":"2024-04-25T10:16:43.540694+0000","last_active":"2024-04-25T10:16:44.562488+0000","last_peered":"2024-04-25T10:16:44.562488+0000","last_clean":"2024-04-25T10:16:44.562488+0000","last_became_active":"2024-04-25T10:16:43.540044+0000","last_became_peered":"2024-04-25T10:16:43.540044+0000","last_unstale":"2024-04-25T10:16:44.562488+0000","last_undegraded":"2024-04-25T10:16:44.562488+0000","last_fullsized":"2024-04-25T10:16:44.562488+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.118906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537658+0000","last_change":"2024-04-25T10:16:42.561604+0000","last_active":"2024-04-25T10:16:44.537658+0000","last_peered":"2024-04-25T10:16:44.537658+0000","last_clean":"2024-04-25T10:16:44.537658+0000","last_became_active":"2024-04-25T10:16:42.561278+0000","last_became_peered":"2024-04-25T10:16:42.561278+0000","last_unstale":"2024-04-25T10:16:44.537658+0000","last_undegraded":"2024-04-25T10:16:44.537658+0000","last_fullsized":"2024-04-25T10:16:44.537658+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:52.219063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567427+0000","last_change":"2024-04-25T10:16:43.560700+0000","last_active":"2024-04-25T10:16:44.567427+0000","last_peered":"2024-04-25T10:16:44.567427+0000","last_clean":"2024-04-25T10:16:44.567427+0000","last_became_active":"2024-04-25T10:16:43.560410+0000","last_became_peered":"2024-04-25T10:16:43.560410+0000","last_unstale":"2024-04-25T10:16:44.567427+0000","last_undegraded":"2024-04-25T10:16:44.567427+0000","last_fullsized":"2024-04-25T10:16:44.567427+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:34.167604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258171+0000","last_change":"2024-04-25T10:16:42.552123+0000","last_active":"2024-04-25T10:16:45.258171+0000","last_peered":"2024-04-25T10:16:45.258171+0000","last_clean":"2024-04-25T10:16:45.258171+0000","last_became_active":"2024-04-25T10:16:42.551840+0000","last_became_peered":"2024-04-25T10:16:42.551840+0000","last_unstale":"2024-04-25T10:16:45.258171+0000","last_undegraded":"2024-04-25T10:16:45.258171+0000","last_fullsized":"2024-04-25T10:16:45.258171+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:11.016703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644307+0000","last_change":"2024-04-25T10:16:43.535347+0000","last_active":"2024-04-25T10:16:44.644307+0000","last_peered":"2024-04-25T10:16:44.644307+0000","last_clean":"2024-04-25T10:16:44.644307+0000","last_became_active":"2024-04-25T10:16:43.533702+0000","last_became_peered":"2024-04-25T10:16:43.533702+0000","last_unstale":"2024-04-25T10:16:44.644307+0000","last_undegraded":"2024-04-25T10:16:44.644307+0000","last_fullsized":"2024-04-25T10:16:44.644307+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:57.980294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538061+0000","last_change":"2024-04-25T10:16:42.555853+0000","last_active":"2024-04-25T10:16:44.538061+0000","last_peered":"2024-04-25T10:16:44.538061+0000","last_clean":"2024-04-25T10:16:44.538061+0000","last_became_active":"2024-04-25T10:16:42.555550+0000","last_became_peered":"2024-04-25T10:16:42.555550+0000","last_unstale":"2024-04-25T10:16:44.538061+0000","last_undegraded":"2024-04-25T10:16:44.538061+0000","last_fullsized":"2024-04-25T10:16:44.538061+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:01.727950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567403+0000","last_change":"2024-04-25T10:16:43.556539+0000","last_active":"2024-04-25T10:16:44.567403+0000","last_peered":"2024-04-25T10:16:44.567403+0000","last_clean":"2024-04-25T10:16:44.567403+0000","last_became_active":"2024-04-25T10:16:43.556219+0000","last_became_peered":"2024-04-25T10:16:43.556219+0000","last_unstale":"2024-04-25T10:16:44.567403+0000","last_undegraded":"2024-04-25T10:16:44.567403+0000","last_fullsized":"2024-04-25T10:16:44.567403+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.257333+0000","last_change":"2024-04-25T10:16:42.551277+0000","last_active":"2024-04-25T10:16:45.257333+0000","last_peered":"2024-04-25T10:16:45.257333+0000","last_clean":"2024-04-25T10:16:45.257333+0000","last_became_active":"2024-04-25T10:16:42.550967+0000","last_became_peered":"2024-04-25T10:16:42.550967+0000","last_unstale":"2024-04-25T10:16:45.257333+0000","last_undegraded":"2024-04-25T10:16:45.257333+0000","last_fullsized":"2024-04-25T10:16:45.257333+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:19.550214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644311+0000","last_change":"2024-04-25T10:16:43.546966+0000","last_active":"2024-04-25T10:16:44.644311+0000","last_peered":"2024-04-25T10:16:44.644311+0000","last_clean":"2024-04-25T10:16:44.644311+0000","last_became_active":"2024-04-25T10:16:43.546529+0000","last_became_peered":"2024-04-25T10:16:43.546529+0000","last_unstale":"2024-04-25T10:16:44.644311+0000","last_undegraded":"2024-04-25T10:16:44.644311+0000","last_fullsized":"2024-04-25T10:16:44.644311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:45.108391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537963+0000","last_change":"2024-04-25T10:16:42.561957+0000","last_active":"2024-04-25T10:16:44.537963+0000","last_peered":"2024-04-25T10:16:44.537963+0000","last_clean":"2024-04-25T10:16:44.537963+0000","last_became_active":"2024-04-25T10:16:42.561460+0000","last_became_peered":"2024-04-25T10:16:42.561460+0000","last_unstale":"2024-04-25T10:16:44.537963+0000","last_undegraded":"2024-04-25T10:16:44.537963+0000","last_fullsized":"2024-04-25T10:16:44.537963+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:36.001686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258883+0000","last_change":"2024-04-25T10:16:43.555830+0000","last_active":"2024-04-25T10:16:45.258883+0000","last_peered":"2024-04-25T10:16:45.258883+0000","last_clean":"2024-04-25T10:16:45.258883+0000","last_became_active":"2024-04-25T10:16:43.554690+0000","last_became_peered":"2024-04-25T10:16:43.554690+0000","last_unstale":"2024-04-25T10:16:45.258883+0000","last_undegraded":"2024-04-25T10:16:45.258883+0000","last_fullsized":"2024-04-25T10:16:45.258883+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:20.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539527+0000","last_change":"2024-04-25T10:16:42.561522+0000","last_active":"2024-04-25T10:16:44.539527+0000","last_peered":"2024-04-25T10:16:44.539527+0000","last_clean":"2024-04-25T10:16:44.539527+0000","last_became_active":"2024-04-25T10:16:42.561181+0000","last_became_peered":"2024-04-25T10:16:42.561181+0000","last_unstale":"2024-04-25T10:16:44.539527+0000","last_undegraded":"2024-04-25T10:16:44.539527+0000","last_fullsized":"2024-04-25T10:16:44.539527+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:33.277014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538833+0000","last_change":"2024-04-25T10:16:43.559320+0000","last_active":"2024-04-25T10:16:44.538833+0000","last_peered":"2024-04-25T10:16:44.538833+0000","last_clean":"2024-04-25T10:16:44.538833+0000","last_became_active":"2024-04-25T10:16:43.559034+0000","last_became_peered":"2024-04-25T10:16:43.559034+0000","last_unstale":"2024-04-25T10:16:44.538833+0000","last_undegraded":"2024-04-25T10:16:44.538833+0000","last_fullsized":"2024-04-25T10:16:44.538833+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:01.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"148'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562091+0000","last_change":"2024-04-25T10:16:42.570360+0000","last_active":"2024-04-25T10:16:44.562091+0000","last_peered":"2024-04-25T10:16:44.562091+0000","last_clean":"2024-04-25T10:16:44.562091+0000","last_became_active":"2024-04-25T10:16:42.569941+0000","last_became_peered":"2024-04-25T10:16:42.569941+0000","last_unstale":"2024-04-25T10:16:44.562091+0000","last_undegraded":"2024-04-25T10:16:44.562091+0000","last_fullsized":"2024-04-25T10:16:44.562091+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:38.817632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562408+0000","last_change":"2024-04-25T10:16:43.557368+0000","last_active":"2024-04-25T10:16:44.562408+0000","last_peered":"2024-04-25T10:16:44.562408+0000","last_clean":"2024-04-25T10:16:44.562408+0000","last_became_active":"2024-04-25T10:16:43.557070+0000","last_became_peered":"2024-04-25T10:16:43.557070+0000","last_unstale":"2024-04-25T10:16:44.562408+0000","last_undegraded":"2024-04-25T10:16:44.562408+0000","last_fullsized":"2024-04-25T10:16:44.562408+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.833357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644415+0000","last_change":"2024-04-25T10:16:42.553757+0000","last_active":"2024-04-25T10:16:44.644415+0000","last_peered":"2024-04-25T10:16:44.644415+0000","last_clean":"2024-04-25T10:16:44.644415+0000","last_became_active":"2024-04-25T10:16:42.552414+0000","last_became_peered":"2024-04-25T10:16:42.552414+0000","last_unstale":"2024-04-25T10:16:44.644415+0000","last_undegraded":"2024-04-25T10:16:44.644415+0000","last_fullsized":"2024-04-25T10:16:44.644415+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:29.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259472+0000","last_change":"2024-04-25T10:16:43.551110+0000","last_active":"2024-04-25T10:16:45.259472+0000","last_peered":"2024-04-25T10:16:45.259472+0000","last_clean":"2024-04-25T10:16:45.259472+0000","last_became_active":"2024-04-25T10:16:43.550776+0000","last_became_peered":"2024-04-25T10:16:43.550776+0000","last_unstale":"2024-04-25T10:16:45.259472+0000","last_undegraded":"2024-04-25T10:16:45.259472+0000","last_fullsized":"2024-04-25T10:16:45.259472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:36.036392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259025+0000","last_change":"2024-04-25T10:16:42.553623+0000","last_active":"2024-04-25T10:16:45.259025+0000","last_peered":"2024-04-25T10:16:45.259025+0000","last_clean":"2024-04-25T10:16:45.259025+0000","last_became_active":"2024-04-25T10:16:42.553344+0000","last_became_peered":"2024-04-25T10:16:42.553344+0000","last_unstale":"2024-04-25T10:16:45.259025+0000","last_undegraded":"2024-04-25T10:16:45.259025+0000","last_fullsized":"2024-04-25T10:16:45.259025+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:09.078537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537775+0000","last_change":"2024-04-25T10:16:43.557497+0000","last_active":"2024-04-25T10:16:44.537775+0000","last_peered":"2024-04-25T10:16:44.537775+0000","last_clean":"2024-04-25T10:16:44.537775+0000","last_became_active":"2024-04-25T10:16:43.557138+0000","last_became_peered":"2024-04-25T10:16:43.557138+0000","last_unstale":"2024-04-25T10:16:44.537775+0000","last_undegraded":"2024-04-25T10:16:44.537775+0000","last_fullsized":"2024-04-25T10:16:44.537775+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:06.866347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561372+0000","last_change":"2024-04-25T10:16:42.569306+0000","last_active":"2024-04-25T10:16:44.561372+0000","last_peered":"2024-04-25T10:16:44.561372+0000","last_clean":"2024-04-25T10:16:44.561372+0000","last_became_active":"2024-04-25T10:16:42.568916+0000","last_became_peered":"2024-04-25T10:16:42.568916+0000","last_unstale":"2024-04-25T10:16:44.561372+0000","last_undegraded":"2024-04-25T10:16:44.561372+0000","last_fullsized":"2024-04-25T10:16:44.561372+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:11.043365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536335+0000","last_change":"2024-04-25T10:16:43.558958+0000","last_active":"2024-04-25T10:16:44.536335+0000","last_peered":"2024-04-25T10:16:44.536335+0000","last_clean":"2024-04-25T10:16:44.536335+0000","last_became_active":"2024-04-25T10:16:43.558167+0000","last_became_peered":"2024-04-25T10:16:43.558167+0000","last_unstale":"2024-04-25T10:16:44.536335+0000","last_undegraded":"2024-04-25T10:16:44.536335+0000","last_fullsized":"2024-04-25T10:16:44.536335+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.739294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536161+0000","last_change":"2024-04-25T10:16:42.560964+0000","last_active":"2024-04-25T10:16:44.536161+0000","last_peered":"2024-04-25T10:16:44.536161+0000","last_clean":"2024-04-25T10:16:44.536161+0000","last_became_active":"2024-04-25T10:16:42.560653+0000","last_became_peered":"2024-04-25T10:16:42.560653+0000","last_unstale":"2024-04-25T10:16:44.536161+0000","last_undegraded":"2024-04-25T10:16:44.536161+0000","last_fullsized":"2024-04-25T10:16:44.536161+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.536743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644262+0000","last_change":"2024-04-25T10:16:43.558553+0000","last_active":"2024-04-25T10:16:44.644262+0000","last_peered":"2024-04-25T10:16:44.644262+0000","last_clean":"2024-04-25T10:16:44.644262+0000","last_became_active":"2024-04-25T10:16:43.558286+0000","last_became_peered":"2024-04-25T10:16:43.558286+0000","last_unstale":"2024-04-25T10:16:44.644262+0000","last_undegraded":"2024-04-25T10:16:44.644262+0000","last_fullsized":"2024-04-25T10:16:44.644262+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.084637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561386+0000","last_change":"2024-04-25T10:16:42.567149+0000","last_active":"2024-04-25T10:16:44.561386+0000","last_peered":"2024-04-25T10:16:44.561386+0000","last_clean":"2024-04-25T10:16:44.561386+0000","last_became_active":"2024-04-25T10:16:42.566786+0000","last_became_peered":"2024-04-25T10:16:42.566786+0000","last_unstale":"2024-04-25T10:16:44.561386+0000","last_undegraded":"2024-04-25T10:16:44.561386+0000","last_fullsized":"2024-04-25T10:16:44.561386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:40.477788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538795+0000","last_change":"2024-04-25T10:16:43.568156+0000","last_active":"2024-04-25T10:16:44.538795+0000","last_peered":"2024-04-25T10:16:44.538795+0000","last_clean":"2024-04-25T10:16:44.538795+0000","last_became_active":"2024-04-25T10:16:43.567851+0000","last_became_peered":"2024-04-25T10:16:43.567851+0000","last_unstale":"2024-04-25T10:16:44.538795+0000","last_undegraded":"2024-04-25T10:16:44.538795+0000","last_fullsized":"2024-04-25T10:16:44.538795+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:15.868630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567095+0000","last_change":"2024-04-25T10:16:42.564913+0000","last_active":"2024-04-25T10:16:44.567095+0000","last_peered":"2024-04-25T10:16:44.567095+0000","last_clean":"2024-04-25T10:16:44.567095+0000","last_became_active":"2024-04-25T10:16:42.563056+0000","last_became_peered":"2024-04-25T10:16:42.563056+0000","last_unstale":"2024-04-25T10:16:44.567095+0000","last_undegraded":"2024-04-25T10:16:44.567095+0000","last_fullsized":"2024-04-25T10:16:44.567095+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:15.848421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539580+0000","last_change":"2024-04-25T10:16:43.555075+0000","last_active":"2024-04-25T10:16:44.539580+0000","last_peered":"2024-04-25T10:16:44.539580+0000","last_clean":"2024-04-25T10:16:44.539580+0000","last_became_active":"2024-04-25T10:16:43.554782+0000","last_became_peered":"2024-04-25T10:16:43.554782+0000","last_unstale":"2024-04-25T10:16:44.539580+0000","last_undegraded":"2024-04-25T10:16:44.539580+0000","last_fullsized":"2024-04-25T10:16:44.539580+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:30.375818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"147'6","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258556+0000","last_change":"2024-04-25T10:16:42.553024+0000","last_active":"2024-04-25T10:16:45.258556+0000","last_peered":"2024-04-25T10:16:45.258556+0000","last_clean":"2024-04-25T10:16:45.258556+0000","last_became_active":"2024-04-25T10:16:42.552733+0000","last_became_peered":"2024-04-25T10:16:42.552733+0000","last_unstale":"2024-04-25T10:16:45.258556+0000","last_undegraded":"2024-04-25T10:16:45.258556+0000","last_fullsized":"2024-04-25T10:16:45.258556+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:12.363657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567417+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:16:44.567417+0000","last_peered":"2024-04-25T10:16:44.567417+0000","last_clean":"2024-04-25T10:16:44.567417+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:16:44.567417+0000","last_undegraded":"2024-04-25T10:16:44.567417+0000","last_fullsized":"2024-04-25T10:16:44.567417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575031,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59948,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311892,"statfs":{"total":96636764160,"available":96575377408,"internally_reserved":0,"allocated":2453504,"data_stored":1462405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575031,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55208,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1859584,"data_stored":872085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55204,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1855488,"data_stored":872015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575031,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82924,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288916,"statfs":{"total":96636764160,"available":96551849984,"internally_reserved":0,"allocated":1867776,"data_stored":873627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575031,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82928,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288912,"statfs":{"total":96636764160,"available":96551845888,"internally_reserved":0,"allocated":1863680,"data_stored":872631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83032519},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575030,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575031,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55212,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316628,"statfs":{"total":96636764160,"available":96580227072,"internally_reserved":0,"allocated":1859584,"data_stored":872085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575031,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47484,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324356,"statfs":{"total":96636764160,"available":96588140544,"internally_reserved":0,"allocated":2473984,"data_stored":1464701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:52.953 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:16:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.161+0000 7fe5f1e00640 1 Processor -- start 2024-04-25T10:16:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.161+0000 7fe5f1e00640 1 -- start start 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.162+0000 7fe5f1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5ec06e870 0x7fe5ec0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.162+0000 7fe5f1e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe5ec0cc840 0x7fe5ec15ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.162+0000 7fe5f1e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5ec144240 con 0x7fe5ec143e60 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.162+0000 7fe5f1e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec15e220 con 0x7fe5ec06e870 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.162+0000 7fe5f1e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5ec15e500 con 0x7fe5ec0cc840 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5f0a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe5ec143e60 0x7fe5ec06e160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50674/0 (socket says 172.21.15.165:50674) 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5f0a00640 1 -- 172.21.15.165:0/1266494669 learned_addr learned my addr 172.21.15.165:0/1266494669 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5e3e00640 1 --2- 172.21.15.165:0/1266494669 >> v2:172.21.15.165:3300/0 conn(0x7fe5ec06e870 0x7fe5ec0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5f1400640 1 --2- 172.21.15.165:0/1266494669 >> v2:172.21.15.172:3300/0 conn(0x7fe5ec0cc840 0x7fe5ec15ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5ebe00640 1 -- 172.21.15.165:0/1266494669 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716610746 0 0) 0x7fe5ec144240 con 0x7fe5ec143e60 2024-04-25T10:16:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5ebe00640 1 -- 172.21.15.165:0/1266494669 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5d0003520 con 0x7fe5ec143e60 2024-04-25T10:16:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5e3e00640 1 -- 172.21.15.165:0/1266494669 >> v2:172.21.15.172:3300/0 conn(0x7fe5ec0cc840 msgr2=0x7fe5ec15ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5e3e00640 1 --2- 172.21.15.165:0/1266494669 >> v2:172.21.15.172:3300/0 conn(0x7fe5ec0cc840 0x7fe5ec15ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5e3e00640 1 -- 172.21.15.165:0/1266494669 >> v1:172.21.15.165:6789/0 conn(0x7fe5ec143e60 legacy=0x7fe5ec06e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.163+0000 7fe5e3e00640 1 -- 172.21.15.165:0/1266494669 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec15e7e0 con 0x7fe5ec06e870 2024-04-25T10:16:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.164+0000 7fe5e3e00640 1 --2- 172.21.15.165:0/1266494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec06e870 0x7fe5ec0cc300 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0b2710 tx=0x7fe5d402a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61bcce96de4c6780 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5ebe00640 1 -- 172.21.15.165:0/1266494669 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4037070 con 0x7fe5ec06e870 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5ebe00640 1 -- 172.21.15.165:0/1266494669 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4032440 con 0x7fe5ec06e870 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5ebe00640 1 -- 172.21.15.165:0/1266494669 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4032810 con 0x7fe5ec06e870 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5f1e00640 1 -- 172.21.15.165:0/1266494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec06e870 msgr2=0x7fe5ec0cc300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/1266494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec06e870 0x7fe5ec0cc300 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fe5ec0b2710 tx=0x7fe5d402a1a0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5f1e00640 1 -- 172.21.15.165:0/1266494669 shutdown_connections 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/1266494669 >> v2:172.21.15.172:3300/0 conn(0x7fe5ec0cc840 0x7fe5ec15ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.165+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/1266494669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec06e870 0x7fe5ec0cc300 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.166+0000 7fe5f1e00640 1 -- 172.21.15.165:0/1266494669 >> 172.21.15.165:0/1266494669 conn(0x7fe5ec06abe0 msgr2=0x7fe5ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.166+0000 7fe5f1e00640 1 -- 172.21.15.165:0/1266494669 shutdown_connections 2024-04-25T10:16:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.166+0000 7fe5f1e00640 1 -- 172.21.15.165:0/1266494669 wait complete. 2024-04-25T10:16:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.167+0000 7fe5f1e00640 1 Processor -- start 2024-04-25T10:16:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.167+0000 7fe5f1e00640 1 -- start start 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.168+0000 7fe5f1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ec06e870 0x7fe5ec140bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.168+0000 7fe5f1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 0x7fe5ec06d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.168+0000 7fe5f1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5ec143e60 0x7fe5ec159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.168+0000 7fe5f1e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0b2eb0 con 0x7fe5ec0cc840 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.168+0000 7fe5f1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0b2d30 con 0x7fe5ec06e870 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.168+0000 7fe5f1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5ec0ccca0 con 0x7fe5ec143e60 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 0x7fe5ec06d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 0x7fe5ec06d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52790/0 (socket says 172.21.15.165:52790) 2024-04-25T10:16:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 -- 172.21.15.165:0/4020054482 learned_addr learned my addr 172.21.15.165:0/4020054482 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5f0a00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ec06e870 0x7fe5ec140bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 -- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5ec143e60 msgr2=0x7fe5ec159c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5ec143e60 0x7fe5ec159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 -- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ec06e870 msgr2=0x7fe5ec140bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ec06e870 0x7fe5ec140bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.169+0000 7fe5e3e00640 1 -- 172.21.15.165:0/4020054482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5ec1f6700 con 0x7fe5ec0cc840 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.170+0000 7fe5e3e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 0x7fe5ec06d6a0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009890 tx=0x7fe5d40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.170+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4037040 con 0x7fe5ec0cc840 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.170+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d4042070 con 0x7fe5ec0cc840 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.170+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d4030960 con 0x7fe5ec0cc840 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.170+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5ec1f69e0 con 0x7fe5ec0cc840 2024-04-25T10:16:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.170+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5ec1f6fa0 con 0x7fe5ec0cc840 2024-04-25T10:16:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.171+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5ec073990 con 0x7fe5ec0cc840 2024-04-25T10:16:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.172+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe5d4030b00 con 0x7fe5ec0cc840 2024-04-25T10:16:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.173+0000 7fe5e9600640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5c806b4d0 0x7fe5c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.173+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe5d40aacb0 con 0x7fe5ec0cc840 2024-04-25T10:16:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.173+0000 7fe5f0a00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5c806b4d0 0x7fe5c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.174+0000 7fe5f0a00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5c806b4d0 0x7fe5c806d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc000f60 tx=0x7fe5dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.178+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5d4079990 con 0x7fe5ec0cc840 2024-04-25T10:16:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.368+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe5ec15bcf0 con 0x7fe5c806b4d0 2024-04-25T10:16:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.386+0000 7fe5e9600640 1 -- 172.21.15.165:0/4020054482 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252374 (secure 0 0 0) 0x7fe5ec15bcf0 con 0x7fe5c806b4d0 2024-04-25T10:16:53.393 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:53.394 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5c806b4d0 msgr2=0x7fe5c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5c806b4d0 0x7fe5c806d990 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc000f60 tx=0x7fe5dc009040 comp rx=0 tx=0).stop 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 msgr2=0x7fe5ec06d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 0x7fe5ec06d6a0 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fe5d4009890 tx=0x7fe5d40046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 shutdown_connections 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe5c806b4d0 0x7fe5c806d990 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5ec143e60 0x7fe5ec159c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5ec0cc840 0x7fe5ec06d6a0 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 --2- 172.21.15.165:0/4020054482 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe5ec06e870 0x7fe5ec140bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 >> 172.21.15.165:0/4020054482 conn(0x7fe5ec06abe0 msgr2=0x7fe5ec0cb770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 shutdown_connections 2024-04-25T10:16:53.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.391+0000 7fe5f1e00640 1 -- 172.21.15.165:0/4020054482 wait complete. 2024-04-25T10:16:53.413 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":591,"stamp":"2024-04-25T10:16:52.431536+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":498284,"kb_used_data":15720,"kb_used_omap":12,"kb_used_meta":482419,"kb_avail":754476436,"statfs":{"total":773094113280,"available":772583870464,"internally_reserved":0,"allocated":16097280,"data_stored":8161676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":493997618},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.910465"},"pg_stats":[{"pgid":"22.3f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562120+0000","last_change":"2024-04-25T10:16:43.558546+0000","last_active":"2024-04-25T10:16:44.562120+0000","last_peered":"2024-04-25T10:16:44.562120+0000","last_clean":"2024-04-25T10:16:44.562120+0000","last_became_active":"2024-04-25T10:16:43.557592+0000","last_became_peered":"2024-04-25T10:16:43.557592+0000","last_unstale":"2024-04-25T10:16:44.562120+0000","last_undegraded":"2024-04-25T10:16:44.562120+0000","last_fullsized":"2024-04-25T10:16:44.562120+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.763212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644312+0000","last_change":"2024-04-25T10:16:43.542077+0000","last_active":"2024-04-25T10:16:44.644312+0000","last_peered":"2024-04-25T10:16:44.644312+0000","last_clean":"2024-04-25T10:16:44.644312+0000","last_became_active":"2024-04-25T10:16:43.541331+0000","last_became_peered":"2024-04-25T10:16:43.541331+0000","last_unstale":"2024-04-25T10:16:44.644312+0000","last_undegraded":"2024-04-25T10:16:44.644312+0000","last_fullsized":"2024-04-25T10:16:44.644312+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:18.077257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644302+0000","last_change":"2024-04-25T10:16:43.554057+0000","last_active":"2024-04-25T10:16:44.644302+0000","last_peered":"2024-04-25T10:16:44.644302+0000","last_clean":"2024-04-25T10:16:44.644302+0000","last_became_active":"2024-04-25T10:16:43.553688+0000","last_became_peered":"2024-04-25T10:16:43.553688+0000","last_unstale":"2024-04-25T10:16:44.644302+0000","last_undegraded":"2024-04-25T10:16:44.644302+0000","last_fullsized":"2024-04-25T10:16:44.644302+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:23:23.042495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536340+0000","last_change":"2024-04-25T10:16:43.553869+0000","last_active":"2024-04-25T10:16:44.536340+0000","last_peered":"2024-04-25T10:16:44.536340+0000","last_clean":"2024-04-25T10:16:44.536340+0000","last_became_active":"2024-04-25T10:16:43.552593+0000","last_became_peered":"2024-04-25T10:16:43.552593+0000","last_unstale":"2024-04-25T10:16:44.536340+0000","last_undegraded":"2024-04-25T10:16:44.536340+0000","last_fullsized":"2024-04-25T10:16:44.536340+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:38.269376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.260200+0000","last_change":"2024-04-25T10:16:43.558696+0000","last_active":"2024-04-25T10:16:45.260200+0000","last_peered":"2024-04-25T10:16:45.260200+0000","last_clean":"2024-04-25T10:16:45.260200+0000","last_became_active":"2024-04-25T10:16:43.554924+0000","last_became_peered":"2024-04-25T10:16:43.554924+0000","last_unstale":"2024-04-25T10:16:45.260200+0000","last_undegraded":"2024-04-25T10:16:45.260200+0000","last_fullsized":"2024-04-25T10:16:45.260200+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:27.953723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259886+0000","last_change":"2024-04-25T10:16:43.553964+0000","last_active":"2024-04-25T10:16:45.259886+0000","last_peered":"2024-04-25T10:16:45.259886+0000","last_clean":"2024-04-25T10:16:45.259886+0000","last_became_active":"2024-04-25T10:16:43.553667+0000","last_became_peered":"2024-04-25T10:16:43.553667+0000","last_unstale":"2024-04-25T10:16:45.259886+0000","last_undegraded":"2024-04-25T10:16:45.259886+0000","last_fullsized":"2024-04-25T10:16:45.259886+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:09:48.946817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561293+0000","last_change":"2024-04-25T10:16:43.554632+0000","last_active":"2024-04-25T10:16:44.561293+0000","last_peered":"2024-04-25T10:16:44.561293+0000","last_clean":"2024-04-25T10:16:44.561293+0000","last_became_active":"2024-04-25T10:16:43.554371+0000","last_became_peered":"2024-04-25T10:16:43.554371+0000","last_unstale":"2024-04-25T10:16:44.561293+0000","last_undegraded":"2024-04-25T10:16:44.561293+0000","last_fullsized":"2024-04-25T10:16:44.561293+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.084637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537189+0000","last_change":"2024-04-25T10:16:43.563238+0000","last_active":"2024-04-25T10:16:44.537189+0000","last_peered":"2024-04-25T10:16:44.537189+0000","last_clean":"2024-04-25T10:16:44.537189+0000","last_became_active":"2024-04-25T10:16:43.562888+0000","last_became_peered":"2024-04-25T10:16:43.562888+0000","last_unstale":"2024-04-25T10:16:44.537189+0000","last_undegraded":"2024-04-25T10:16:44.537189+0000","last_fullsized":"2024-04-25T10:16:44.537189+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:34.293550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259043+0000","last_change":"2024-04-25T10:16:43.557253+0000","last_active":"2024-04-25T10:16:45.259043+0000","last_peered":"2024-04-25T10:16:45.259043+0000","last_clean":"2024-04-25T10:16:45.259043+0000","last_became_active":"2024-04-25T10:16:43.556951+0000","last_became_peered":"2024-04-25T10:16:43.556951+0000","last_unstale":"2024-04-25T10:16:45.259043+0000","last_undegraded":"2024-04-25T10:16:45.259043+0000","last_fullsized":"2024-04-25T10:16:45.259043+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:13.380192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259383+0000","last_change":"2024-04-25T10:16:43.558625+0000","last_active":"2024-04-25T10:16:45.259383+0000","last_peered":"2024-04-25T10:16:45.259383+0000","last_clean":"2024-04-25T10:16:45.259383+0000","last_became_active":"2024-04-25T10:16:43.558255+0000","last_became_peered":"2024-04-25T10:16:43.558255+0000","last_unstale":"2024-04-25T10:16:45.259383+0000","last_undegraded":"2024-04-25T10:16:45.259383+0000","last_fullsized":"2024-04-25T10:16:45.259383+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:12.033238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536630+0000","last_change":"2024-04-25T10:16:43.557356+0000","last_active":"2024-04-25T10:16:44.536630+0000","last_peered":"2024-04-25T10:16:44.536630+0000","last_clean":"2024-04-25T10:16:44.536630+0000","last_became_active":"2024-04-25T10:16:43.557009+0000","last_became_peered":"2024-04-25T10:16:43.557009+0000","last_unstale":"2024-04-25T10:16:44.536630+0000","last_undegraded":"2024-04-25T10:16:44.536630+0000","last_fullsized":"2024-04-25T10:16:44.536630+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:02:40.022950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537537+0000","last_change":"2024-04-25T10:16:43.559117+0000","last_active":"2024-04-25T10:16:44.537537+0000","last_peered":"2024-04-25T10:16:44.537537+0000","last_clean":"2024-04-25T10:16:44.537537+0000","last_became_active":"2024-04-25T10:16:43.558792+0000","last_became_peered":"2024-04-25T10:16:43.558792+0000","last_unstale":"2024-04-25T10:16:44.537537+0000","last_undegraded":"2024-04-25T10:16:44.537537+0000","last_fullsized":"2024-04-25T10:16:44.537537+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:41.494324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259928+0000","last_change":"2024-04-25T10:16:43.559623+0000","last_active":"2024-04-25T10:16:45.259928+0000","last_peered":"2024-04-25T10:16:45.259928+0000","last_clean":"2024-04-25T10:16:45.259928+0000","last_became_active":"2024-04-25T10:16:43.559334+0000","last_became_peered":"2024-04-25T10:16:43.559334+0000","last_unstale":"2024-04-25T10:16:45.259928+0000","last_undegraded":"2024-04-25T10:16:45.259928+0000","last_fullsized":"2024-04-25T10:16:45.259928+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:43.288992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536382+0000","last_change":"2024-04-25T10:16:43.557501+0000","last_active":"2024-04-25T10:16:44.536382+0000","last_peered":"2024-04-25T10:16:44.536382+0000","last_clean":"2024-04-25T10:16:44.536382+0000","last_became_active":"2024-04-25T10:16:43.557212+0000","last_became_peered":"2024-04-25T10:16:43.557212+0000","last_unstale":"2024-04-25T10:16:44.536382+0000","last_undegraded":"2024-04-25T10:16:44.536382+0000","last_fullsized":"2024-04-25T10:16:44.536382+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:12.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644124+0000","last_change":"2024-04-25T10:16:43.547840+0000","last_active":"2024-04-25T10:16:44.644124+0000","last_peered":"2024-04-25T10:16:44.644124+0000","last_clean":"2024-04-25T10:16:44.644124+0000","last_became_active":"2024-04-25T10:16:43.544427+0000","last_became_peered":"2024-04-25T10:16:43.544427+0000","last_unstale":"2024-04-25T10:16:44.644124+0000","last_undegraded":"2024-04-25T10:16:44.644124+0000","last_fullsized":"2024-04-25T10:16:44.644124+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:36.009160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258182+0000","last_change":"2024-04-25T10:16:43.562196+0000","last_active":"2024-04-25T10:16:45.258182+0000","last_peered":"2024-04-25T10:16:45.258182+0000","last_clean":"2024-04-25T10:16:45.258182+0000","last_became_active":"2024-04-25T10:16:43.561890+0000","last_became_peered":"2024-04-25T10:16:43.561890+0000","last_unstale":"2024-04-25T10:16:45.258182+0000","last_undegraded":"2024-04-25T10:16:45.258182+0000","last_fullsized":"2024-04-25T10:16:45.258182+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:32:40.034886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539779+0000","last_change":"2024-04-25T10:16:43.555511+0000","last_active":"2024-04-25T10:16:44.539779+0000","last_peered":"2024-04-25T10:16:44.539779+0000","last_clean":"2024-04-25T10:16:44.539779+0000","last_became_active":"2024-04-25T10:16:43.555209+0000","last_became_peered":"2024-04-25T10:16:43.555209+0000","last_unstale":"2024-04-25T10:16:44.539779+0000","last_undegraded":"2024-04-25T10:16:44.539779+0000","last_fullsized":"2024-04-25T10:16:44.539779+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:56:27.563607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536915+0000","last_change":"2024-04-25T10:16:43.557716+0000","last_active":"2024-04-25T10:16:44.536915+0000","last_peered":"2024-04-25T10:16:44.536915+0000","last_clean":"2024-04-25T10:16:44.536915+0000","last_became_active":"2024-04-25T10:16:43.557442+0000","last_became_peered":"2024-04-25T10:16:43.557442+0000","last_unstale":"2024-04-25T10:16:44.536915+0000","last_undegraded":"2024-04-25T10:16:44.536915+0000","last_fullsized":"2024-04-25T10:16:44.536915+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:05:47.240799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567976+0000","last_change":"2024-04-25T10:16:43.559780+0000","last_active":"2024-04-25T10:16:44.567976+0000","last_peered":"2024-04-25T10:16:44.567976+0000","last_clean":"2024-04-25T10:16:44.567976+0000","last_became_active":"2024-04-25T10:16:43.559507+0000","last_became_peered":"2024-04-25T10:16:43.559507+0000","last_unstale":"2024-04-25T10:16:44.567976+0000","last_undegraded":"2024-04-25T10:16:44.567976+0000","last_fullsized":"2024-04-25T10:16:44.567976+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:23.418603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258576+0000","last_change":"2024-04-25T10:16:43.544613+0000","last_active":"2024-04-25T10:16:45.258576+0000","last_peered":"2024-04-25T10:16:45.258576+0000","last_clean":"2024-04-25T10:16:45.258576+0000","last_became_active":"2024-04-25T10:16:43.544181+0000","last_became_peered":"2024-04-25T10:16:43.544181+0000","last_unstale":"2024-04-25T10:16:45.258576+0000","last_undegraded":"2024-04-25T10:16:45.258576+0000","last_fullsized":"2024-04-25T10:16:45.258576+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:43.288992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259936+0000","last_change":"2024-04-25T10:16:43.565132+0000","last_active":"2024-04-25T10:16:45.259936+0000","last_peered":"2024-04-25T10:16:45.259936+0000","last_clean":"2024-04-25T10:16:45.259936+0000","last_became_active":"2024-04-25T10:16:43.564843+0000","last_became_peered":"2024-04-25T10:16:43.564843+0000","last_unstale":"2024-04-25T10:16:45.259936+0000","last_undegraded":"2024-04-25T10:16:45.259936+0000","last_fullsized":"2024-04-25T10:16:45.259936+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:57.505598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561287+0000","last_change":"2024-04-25T10:16:43.552626+0000","last_active":"2024-04-25T10:16:44.561287+0000","last_peered":"2024-04-25T10:16:44.561287+0000","last_clean":"2024-04-25T10:16:44.561287+0000","last_became_active":"2024-04-25T10:16:43.551212+0000","last_became_peered":"2024-04-25T10:16:43.551212+0000","last_unstale":"2024-04-25T10:16:44.561287+0000","last_undegraded":"2024-04-25T10:16:44.561287+0000","last_fullsized":"2024-04-25T10:16:44.561287+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.189603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538152+0000","last_change":"2024-04-25T10:16:43.556110+0000","last_active":"2024-04-25T10:16:44.538152+0000","last_peered":"2024-04-25T10:16:44.538152+0000","last_clean":"2024-04-25T10:16:44.538152+0000","last_became_active":"2024-04-25T10:16:43.555755+0000","last_became_peered":"2024-04-25T10:16:43.555755+0000","last_unstale":"2024-04-25T10:16:44.538152+0000","last_undegraded":"2024-04-25T10:16:44.538152+0000","last_fullsized":"2024-04-25T10:16:44.538152+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:51.966214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536433+0000","last_change":"2024-04-25T10:16:43.561869+0000","last_active":"2024-04-25T10:16:44.536433+0000","last_peered":"2024-04-25T10:16:44.536433+0000","last_clean":"2024-04-25T10:16:44.536433+0000","last_became_active":"2024-04-25T10:16:43.561583+0000","last_became_peered":"2024-04-25T10:16:43.561583+0000","last_unstale":"2024-04-25T10:16:44.536433+0000","last_undegraded":"2024-04-25T10:16:44.536433+0000","last_fullsized":"2024-04-25T10:16:44.536433+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:00:25.788606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562314+0000","last_change":"2024-04-25T10:16:43.546831+0000","last_active":"2024-04-25T10:16:44.562314+0000","last_peered":"2024-04-25T10:16:44.562314+0000","last_clean":"2024-04-25T10:16:44.562314+0000","last_became_active":"2024-04-25T10:16:43.546419+0000","last_became_peered":"2024-04-25T10:16:43.546419+0000","last_unstale":"2024-04-25T10:16:44.562314+0000","last_undegraded":"2024-04-25T10:16:44.562314+0000","last_fullsized":"2024-04-25T10:16:44.562314+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:51.069413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561846+0000","last_change":"2024-04-25T10:16:43.553554+0000","last_active":"2024-04-25T10:16:44.561846+0000","last_peered":"2024-04-25T10:16:44.561846+0000","last_clean":"2024-04-25T10:16:44.561846+0000","last_became_active":"2024-04-25T10:16:43.553131+0000","last_became_peered":"2024-04-25T10:16:43.553131+0000","last_unstale":"2024-04-25T10:16:44.561846+0000","last_undegraded":"2024-04-25T10:16:44.561846+0000","last_fullsized":"2024-04-25T10:16:44.561846+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:34:18.077257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258531+0000","last_change":"2024-04-25T10:16:43.560703+0000","last_active":"2024-04-25T10:16:45.258531+0000","last_peered":"2024-04-25T10:16:45.258531+0000","last_clean":"2024-04-25T10:16:45.258531+0000","last_became_active":"2024-04-25T10:16:43.560398+0000","last_became_peered":"2024-04-25T10:16:43.560398+0000","last_unstale":"2024-04-25T10:16:45.258531+0000","last_undegraded":"2024-04-25T10:16:45.258531+0000","last_fullsized":"2024-04-25T10:16:45.258531+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:10.354287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536983+0000","last_change":"2024-04-25T10:16:43.554470+0000","last_active":"2024-04-25T10:16:44.536983+0000","last_peered":"2024-04-25T10:16:44.536983+0000","last_clean":"2024-04-25T10:16:44.536983+0000","last_became_active":"2024-04-25T10:16:43.554142+0000","last_became_peered":"2024-04-25T10:16:43.554142+0000","last_unstale":"2024-04-25T10:16:44.536983+0000","last_undegraded":"2024-04-25T10:16:44.536983+0000","last_fullsized":"2024-04-25T10:16:44.536983+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:40.366765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538948+0000","last_change":"2024-04-25T10:16:43.568131+0000","last_active":"2024-04-25T10:16:44.538948+0000","last_peered":"2024-04-25T10:16:44.538948+0000","last_clean":"2024-04-25T10:16:44.538948+0000","last_became_active":"2024-04-25T10:16:43.567824+0000","last_became_peered":"2024-04-25T10:16:43.567824+0000","last_unstale":"2024-04-25T10:16:44.538948+0000","last_undegraded":"2024-04-25T10:16:44.538948+0000","last_fullsized":"2024-04-25T10:16:44.538948+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:58.850593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537321+0000","last_change":"2024-04-25T10:16:43.547282+0000","last_active":"2024-04-25T10:16:44.537321+0000","last_peered":"2024-04-25T10:16:44.537321+0000","last_clean":"2024-04-25T10:16:44.537321+0000","last_became_active":"2024-04-25T10:16:43.545939+0000","last_became_peered":"2024-04-25T10:16:43.545939+0000","last_unstale":"2024-04-25T10:16:44.537321+0000","last_undegraded":"2024-04-25T10:16:44.537321+0000","last_fullsized":"2024-04-25T10:16:44.537321+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:10.488478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258369+0000","last_change":"2024-04-25T10:16:43.558141+0000","last_active":"2024-04-25T10:16:45.258369+0000","last_peered":"2024-04-25T10:16:45.258369+0000","last_clean":"2024-04-25T10:16:45.258369+0000","last_became_active":"2024-04-25T10:16:43.557866+0000","last_became_peered":"2024-04-25T10:16:43.557866+0000","last_unstale":"2024-04-25T10:16:45.258369+0000","last_undegraded":"2024-04-25T10:16:45.258369+0000","last_fullsized":"2024-04-25T10:16:45.258369+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:20.806187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537245+0000","last_change":"2024-04-25T10:16:43.558128+0000","last_active":"2024-04-25T10:16:44.537245+0000","last_peered":"2024-04-25T10:16:44.537245+0000","last_clean":"2024-04-25T10:16:44.537245+0000","last_became_active":"2024-04-25T10:16:43.557811+0000","last_became_peered":"2024-04-25T10:16:43.557811+0000","last_unstale":"2024-04-25T10:16:44.537245+0000","last_undegraded":"2024-04-25T10:16:44.537245+0000","last_fullsized":"2024-04-25T10:16:44.537245+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:15.502788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644518+0000","last_change":"2024-04-25T10:16:43.557078+0000","last_active":"2024-04-25T10:16:44.644518+0000","last_peered":"2024-04-25T10:16:44.644518+0000","last_clean":"2024-04-25T10:16:44.644518+0000","last_became_active":"2024-04-25T10:16:43.556763+0000","last_became_peered":"2024-04-25T10:16:43.556763+0000","last_unstale":"2024-04-25T10:16:44.644518+0000","last_undegraded":"2024-04-25T10:16:44.644518+0000","last_fullsized":"2024-04-25T10:16:44.644518+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:51.069413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"147'4","reported_seq":18,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259562+0000","last_change":"2024-04-25T10:16:42.551717+0000","last_active":"2024-04-25T10:16:45.259562+0000","last_peered":"2024-04-25T10:16:45.259562+0000","last_clean":"2024-04-25T10:16:45.259562+0000","last_became_active":"2024-04-25T10:16:42.551437+0000","last_became_peered":"2024-04-25T10:16:42.551437+0000","last_unstale":"2024-04-25T10:16:45.259562+0000","last_undegraded":"2024-04-25T10:16:45.259562+0000","last_fullsized":"2024-04-25T10:16:45.259562+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:40.073529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644386+0000","last_change":"2024-04-25T10:16:43.543406+0000","last_active":"2024-04-25T10:16:44.644386+0000","last_peered":"2024-04-25T10:16:44.644386+0000","last_clean":"2024-04-25T10:16:44.644386+0000","last_became_active":"2024-04-25T10:16:43.542934+0000","last_became_peered":"2024-04-25T10:16:43.542934+0000","last_unstale":"2024-04-25T10:16:44.644386+0000","last_undegraded":"2024-04-25T10:16:44.644386+0000","last_fullsized":"2024-04-25T10:16:44.644386+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:49.763212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567363+0000","last_change":"2024-04-25T10:16:42.571780+0000","last_active":"2024-04-25T10:16:44.567363+0000","last_peered":"2024-04-25T10:16:44.567363+0000","last_clean":"2024-04-25T10:16:44.567363+0000","last_became_active":"2024-04-25T10:16:42.571355+0000","last_became_peered":"2024-04-25T10:16:42.571355+0000","last_unstale":"2024-04-25T10:16:44.567363+0000","last_undegraded":"2024-04-25T10:16:44.567363+0000","last_fullsized":"2024-04-25T10:16:44.567363+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:13.102371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538914+0000","last_change":"2024-04-25T10:16:43.559864+0000","last_active":"2024-04-25T10:16:44.538914+0000","last_peered":"2024-04-25T10:16:44.538914+0000","last_clean":"2024-04-25T10:16:44.538914+0000","last_became_active":"2024-04-25T10:16:43.559490+0000","last_became_peered":"2024-04-25T10:16:43.559490+0000","last_unstale":"2024-04-25T10:16:44.538914+0000","last_undegraded":"2024-04-25T10:16:44.538914+0000","last_fullsized":"2024-04-25T10:16:44.538914+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:37:09.481790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568117+0000","last_change":"2024-04-25T10:16:42.571044+0000","last_active":"2024-04-25T10:16:44.568117+0000","last_peered":"2024-04-25T10:16:44.568117+0000","last_clean":"2024-04-25T10:16:44.568117+0000","last_became_active":"2024-04-25T10:16:42.570593+0000","last_became_peered":"2024-04-25T10:16:42.570593+0000","last_unstale":"2024-04-25T10:16:44.568117+0000","last_undegraded":"2024-04-25T10:16:44.568117+0000","last_fullsized":"2024-04-25T10:16:44.568117+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:40.073529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567879+0000","last_change":"2024-04-25T10:16:43.550656+0000","last_active":"2024-04-25T10:16:44.567879+0000","last_peered":"2024-04-25T10:16:44.567879+0000","last_clean":"2024-04-25T10:16:44.567879+0000","last_became_active":"2024-04-25T10:16:43.550317+0000","last_became_peered":"2024-04-25T10:16:43.550317+0000","last_unstale":"2024-04-25T10:16:44.567879+0000","last_undegraded":"2024-04-25T10:16:44.567879+0000","last_fullsized":"2024-04-25T10:16:44.567879+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:22.020831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644492+0000","last_change":"2024-04-25T10:16:42.553654+0000","last_active":"2024-04-25T10:16:44.644492+0000","last_peered":"2024-04-25T10:16:44.644492+0000","last_clean":"2024-04-25T10:16:44.644492+0000","last_became_active":"2024-04-25T10:16:42.553115+0000","last_became_peered":"2024-04-25T10:16:42.553115+0000","last_unstale":"2024-04-25T10:16:44.644492+0000","last_undegraded":"2024-04-25T10:16:44.644492+0000","last_fullsized":"2024-04-25T10:16:44.644492+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:36:57.834058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.257865+0000","last_change":"2024-04-25T10:16:43.558036+0000","last_active":"2024-04-25T10:16:45.257865+0000","last_peered":"2024-04-25T10:16:45.257865+0000","last_clean":"2024-04-25T10:16:45.257865+0000","last_became_active":"2024-04-25T10:16:43.557754+0000","last_became_peered":"2024-04-25T10:16:43.557754+0000","last_unstale":"2024-04-25T10:16:45.257865+0000","last_undegraded":"2024-04-25T10:16:45.257865+0000","last_fullsized":"2024-04-25T10:16:45.257865+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:14.630997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538931+0000","last_change":"2024-04-25T10:16:42.556137+0000","last_active":"2024-04-25T10:16:44.538931+0000","last_peered":"2024-04-25T10:16:44.538931+0000","last_clean":"2024-04-25T10:16:44.538931+0000","last_became_active":"2024-04-25T10:16:42.555806+0000","last_became_peered":"2024-04-25T10:16:42.555806+0000","last_unstale":"2024-04-25T10:16:44.538931+0000","last_undegraded":"2024-04-25T10:16:44.538931+0000","last_fullsized":"2024-04-25T10:16:44.538931+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:31.761423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.535934+0000","last_change":"2024-04-25T10:16:43.554154+0000","last_active":"2024-04-25T10:16:44.535934+0000","last_peered":"2024-04-25T10:16:44.535934+0000","last_clean":"2024-04-25T10:16:44.535934+0000","last_became_active":"2024-04-25T10:16:43.552635+0000","last_became_peered":"2024-04-25T10:16:43.552635+0000","last_unstale":"2024-04-25T10:16:44.535934+0000","last_undegraded":"2024-04-25T10:16:44.535934+0000","last_fullsized":"2024-04-25T10:16:44.535934+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:39.834167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259987+0000","last_change":"2024-04-25T10:16:42.557166+0000","last_active":"2024-04-25T10:16:45.259987+0000","last_peered":"2024-04-25T10:16:45.259987+0000","last_clean":"2024-04-25T10:16:45.259987+0000","last_became_active":"2024-04-25T10:16:42.556841+0000","last_became_peered":"2024-04-25T10:16:42.556841+0000","last_unstale":"2024-04-25T10:16:45.259987+0000","last_undegraded":"2024-04-25T10:16:45.259987+0000","last_fullsized":"2024-04-25T10:16:45.259987+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:28.174820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644268+0000","last_change":"2024-04-25T10:16:43.541816+0000","last_active":"2024-04-25T10:16:44.644268+0000","last_peered":"2024-04-25T10:16:44.644268+0000","last_clean":"2024-04-25T10:16:44.644268+0000","last_became_active":"2024-04-25T10:16:43.541356+0000","last_became_peered":"2024-04-25T10:16:43.541356+0000","last_unstale":"2024-04-25T10:16:44.644268+0000","last_undegraded":"2024-04-25T10:16:44.644268+0000","last_fullsized":"2024-04-25T10:16:44.644268+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:04:44.189603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.560999+0000","last_change":"2024-04-25T10:16:42.562767+0000","last_active":"2024-04-25T10:16:44.560999+0000","last_peered":"2024-04-25T10:16:44.560999+0000","last_clean":"2024-04-25T10:16:44.560999+0000","last_became_active":"2024-04-25T10:16:42.562417+0000","last_became_peered":"2024-04-25T10:16:42.562417+0000","last_unstale":"2024-04-25T10:16:44.560999+0000","last_undegraded":"2024-04-25T10:16:44.560999+0000","last_fullsized":"2024-04-25T10:16:44.560999+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:47.722759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561472+0000","last_change":"2024-04-25T10:16:43.558917+0000","last_active":"2024-04-25T10:16:44.561472+0000","last_peered":"2024-04-25T10:16:44.561472+0000","last_clean":"2024-04-25T10:16:44.561472+0000","last_became_active":"2024-04-25T10:16:43.558613+0000","last_became_peered":"2024-04-25T10:16:43.558613+0000","last_unstale":"2024-04-25T10:16:44.561472+0000","last_undegraded":"2024-04-25T10:16:44.561472+0000","last_fullsized":"2024-04-25T10:16:44.561472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:57.980294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258330+0000","last_change":"2024-04-25T10:16:42.566952+0000","last_active":"2024-04-25T10:16:45.258330+0000","last_peered":"2024-04-25T10:16:45.258330+0000","last_clean":"2024-04-25T10:16:45.258330+0000","last_became_active":"2024-04-25T10:16:42.566676+0000","last_became_peered":"2024-04-25T10:16:42.566676+0000","last_unstale":"2024-04-25T10:16:45.258330+0000","last_undegraded":"2024-04-25T10:16:45.258330+0000","last_fullsized":"2024-04-25T10:16:45.258330+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:59.508460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539287+0000","last_change":"2024-04-25T10:16:43.561379+0000","last_active":"2024-04-25T10:16:44.539287+0000","last_peered":"2024-04-25T10:16:44.539287+0000","last_clean":"2024-04-25T10:16:44.539287+0000","last_became_active":"2024-04-25T10:16:43.561087+0000","last_became_peered":"2024-04-25T10:16:43.561087+0000","last_unstale":"2024-04-25T10:16:44.539287+0000","last_undegraded":"2024-04-25T10:16:44.539287+0000","last_fullsized":"2024-04-25T10:16:44.539287+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:23.414369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568011+0000","last_change":"2024-04-25T10:16:42.569373+0000","last_active":"2024-04-25T10:16:44.568011+0000","last_peered":"2024-04-25T10:16:44.568011+0000","last_clean":"2024-04-25T10:16:44.568011+0000","last_became_active":"2024-04-25T10:16:42.568629+0000","last_became_peered":"2024-04-25T10:16:42.568629+0000","last_unstale":"2024-04-25T10:16:44.568011+0000","last_undegraded":"2024-04-25T10:16:44.568011+0000","last_fullsized":"2024-04-25T10:16:44.568011+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:59.508460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536837+0000","last_change":"2024-04-25T10:16:43.551780+0000","last_active":"2024-04-25T10:16:44.536837+0000","last_peered":"2024-04-25T10:16:44.536837+0000","last_clean":"2024-04-25T10:16:44.536837+0000","last_became_active":"2024-04-25T10:16:43.551458+0000","last_became_peered":"2024-04-25T10:16:43.551458+0000","last_unstale":"2024-04-25T10:16:44.536837+0000","last_undegraded":"2024-04-25T10:16:44.536837+0000","last_fullsized":"2024-04-25T10:16:44.536837+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:05:13.485034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562222+0000","last_change":"2024-04-25T10:16:42.566623+0000","last_active":"2024-04-25T10:16:44.562222+0000","last_peered":"2024-04-25T10:16:44.562222+0000","last_clean":"2024-04-25T10:16:44.562222+0000","last_became_active":"2024-04-25T10:16:42.566064+0000","last_became_peered":"2024-04-25T10:16:42.566064+0000","last_unstale":"2024-04-25T10:16:44.562222+0000","last_undegraded":"2024-04-25T10:16:44.562222+0000","last_fullsized":"2024-04-25T10:16:44.562222+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:55:37.252841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537203+0000","last_change":"2024-04-25T10:16:43.557534+0000","last_active":"2024-04-25T10:16:44.537203+0000","last_peered":"2024-04-25T10:16:44.537203+0000","last_clean":"2024-04-25T10:16:44.537203+0000","last_became_active":"2024-04-25T10:16:43.557189+0000","last_became_peered":"2024-04-25T10:16:43.557189+0000","last_unstale":"2024-04-25T10:16:44.537203+0000","last_undegraded":"2024-04-25T10:16:44.537203+0000","last_fullsized":"2024-04-25T10:16:44.537203+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:38:00.427901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"147'3","reported_seq":17,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536733+0000","last_change":"2024-04-25T10:16:42.549331+0000","last_active":"2024-04-25T10:16:44.536733+0000","last_peered":"2024-04-25T10:16:44.536733+0000","last_clean":"2024-04-25T10:16:44.536733+0000","last_became_active":"2024-04-25T10:16:42.549027+0000","last_became_peered":"2024-04-25T10:16:42.549027+0000","last_unstale":"2024-04-25T10:16:44.536733+0000","last_undegraded":"2024-04-25T10:16:44.536733+0000","last_fullsized":"2024-04-25T10:16:44.536733+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:26.406784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537764+0000","last_change":"2024-04-25T10:16:43.555391+0000","last_active":"2024-04-25T10:16:44.537764+0000","last_peered":"2024-04-25T10:16:44.537764+0000","last_clean":"2024-04-25T10:16:44.537764+0000","last_became_active":"2024-04-25T10:16:43.554763+0000","last_became_peered":"2024-04-25T10:16:43.554763+0000","last_unstale":"2024-04-25T10:16:44.537764+0000","last_undegraded":"2024-04-25T10:16:44.537764+0000","last_fullsized":"2024-04-25T10:16:44.537764+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:53.235598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.260077+0000","last_change":"2024-04-25T10:16:42.556479+0000","last_active":"2024-04-25T10:16:45.260077+0000","last_peered":"2024-04-25T10:16:45.260077+0000","last_clean":"2024-04-25T10:16:45.260077+0000","last_became_active":"2024-04-25T10:16:42.556163+0000","last_became_peered":"2024-04-25T10:16:42.556163+0000","last_unstale":"2024-04-25T10:16:45.260077+0000","last_undegraded":"2024-04-25T10:16:45.260077+0000","last_fullsized":"2024-04-25T10:16:45.260077+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:15.848421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644123+0000","last_change":"2024-04-25T10:16:43.548087+0000","last_active":"2024-04-25T10:16:44.644123+0000","last_peered":"2024-04-25T10:16:44.644123+0000","last_clean":"2024-04-25T10:16:44.644123+0000","last_became_active":"2024-04-25T10:16:43.545124+0000","last_became_peered":"2024-04-25T10:16:43.545124+0000","last_unstale":"2024-04-25T10:16:44.644123+0000","last_undegraded":"2024-04-25T10:16:44.644123+0000","last_fullsized":"2024-04-25T10:16:44.644123+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:52.071878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537171+0000","last_change":"2024-04-25T10:16:42.562159+0000","last_active":"2024-04-25T10:16:44.537171+0000","last_peered":"2024-04-25T10:16:44.537171+0000","last_clean":"2024-04-25T10:16:44.537171+0000","last_became_active":"2024-04-25T10:16:42.561883+0000","last_became_peered":"2024-04-25T10:16:42.561883+0000","last_unstale":"2024-04-25T10:16:44.537171+0000","last_undegraded":"2024-04-25T10:16:44.537171+0000","last_fullsized":"2024-04-25T10:16:44.537171+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:14.486253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259922+0000","last_change":"2024-04-25T10:16:43.553742+0000","last_active":"2024-04-25T10:16:45.259922+0000","last_peered":"2024-04-25T10:16:45.259922+0000","last_clean":"2024-04-25T10:16:45.259922+0000","last_became_active":"2024-04-25T10:16:43.553465+0000","last_became_peered":"2024-04-25T10:16:43.553465+0000","last_unstale":"2024-04-25T10:16:45.259922+0000","last_undegraded":"2024-04-25T10:16:45.259922+0000","last_fullsized":"2024-04-25T10:16:45.259922+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:14.630997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536152+0000","last_change":"2024-04-25T10:16:42.552024+0000","last_active":"2024-04-25T10:16:44.536152+0000","last_peered":"2024-04-25T10:16:44.536152+0000","last_clean":"2024-04-25T10:16:44.536152+0000","last_became_active":"2024-04-25T10:16:42.550839+0000","last_became_peered":"2024-04-25T10:16:42.550839+0000","last_unstale":"2024-04-25T10:16:44.536152+0000","last_undegraded":"2024-04-25T10:16:44.536152+0000","last_fullsized":"2024-04-25T10:16:44.536152+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:05.849811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644086+0000","last_change":"2024-04-25T10:16:43.560677+0000","last_active":"2024-04-25T10:16:44.644086+0000","last_peered":"2024-04-25T10:16:44.644086+0000","last_clean":"2024-04-25T10:16:44.644086+0000","last_became_active":"2024-04-25T10:16:43.560405+0000","last_became_peered":"2024-04-25T10:16:43.560405+0000","last_unstale":"2024-04-25T10:16:44.644086+0000","last_undegraded":"2024-04-25T10:16:44.644086+0000","last_fullsized":"2024-04-25T10:16:44.644086+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:46:08.735033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"147'8","reported_seq":22,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.560671+0000","last_change":"2024-04-25T10:16:42.566140+0000","last_active":"2024-04-25T10:16:44.560671+0000","last_peered":"2024-04-25T10:16:44.560671+0000","last_clean":"2024-04-25T10:16:44.560671+0000","last_became_active":"2024-04-25T10:16:42.565757+0000","last_became_peered":"2024-04-25T10:16:42.565757+0000","last_unstale":"2024-04-25T10:16:44.560671+0000","last_undegraded":"2024-04-25T10:16:44.560671+0000","last_fullsized":"2024-04-25T10:16:44.560671+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:09.589446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567743+0000","last_change":"2024-04-25T10:16:43.564528+0000","last_active":"2024-04-25T10:16:44.567743+0000","last_peered":"2024-04-25T10:16:44.567743+0000","last_clean":"2024-04-25T10:16:44.567743+0000","last_became_active":"2024-04-25T10:16:43.563011+0000","last_became_peered":"2024-04-25T10:16:43.563011+0000","last_unstale":"2024-04-25T10:16:44.567743+0000","last_undegraded":"2024-04-25T10:16:44.567743+0000","last_fullsized":"2024-04-25T10:16:44.567743+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:41:00.010983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537170+0000","last_change":"2024-04-25T10:16:42.561495+0000","last_active":"2024-04-25T10:16:44.537170+0000","last_peered":"2024-04-25T10:16:44.537170+0000","last_clean":"2024-04-25T10:16:44.537170+0000","last_became_active":"2024-04-25T10:16:42.560241+0000","last_became_peered":"2024-04-25T10:16:42.560241+0000","last_unstale":"2024-04-25T10:16:44.537170+0000","last_undegraded":"2024-04-25T10:16:44.537170+0000","last_fullsized":"2024-04-25T10:16:44.537170+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:11.915069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.568090+0000","last_change":"2024-04-25T10:16:43.557795+0000","last_active":"2024-04-25T10:16:44.568090+0000","last_peered":"2024-04-25T10:16:44.568090+0000","last_clean":"2024-04-25T10:16:44.568090+0000","last_became_active":"2024-04-25T10:16:43.557521+0000","last_became_peered":"2024-04-25T10:16:43.557521+0000","last_unstale":"2024-04-25T10:16:44.568090+0000","last_undegraded":"2024-04-25T10:16:44.568090+0000","last_fullsized":"2024-04-25T10:16:44.568090+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:03:13.810505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567675+0000","last_change":"2024-04-25T10:16:42.564071+0000","last_active":"2024-04-25T10:16:44.567675+0000","last_peered":"2024-04-25T10:16:44.567675+0000","last_clean":"2024-04-25T10:16:44.567675+0000","last_became_active":"2024-04-25T10:16:42.563824+0000","last_became_peered":"2024-04-25T10:16:42.563824+0000","last_unstale":"2024-04-25T10:16:44.567675+0000","last_undegraded":"2024-04-25T10:16:44.567675+0000","last_fullsized":"2024-04-25T10:16:44.567675+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:17:28.174820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562102+0000","last_change":"2024-04-25T10:16:43.557697+0000","last_active":"2024-04-25T10:16:44.562102+0000","last_peered":"2024-04-25T10:16:44.562102+0000","last_clean":"2024-04-25T10:16:44.562102+0000","last_became_active":"2024-04-25T10:16:43.555480+0000","last_became_peered":"2024-04-25T10:16:43.555480+0000","last_unstale":"2024-04-25T10:16:44.562102+0000","last_undegraded":"2024-04-25T10:16:44.562102+0000","last_fullsized":"2024-04-25T10:16:44.562102+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:45.108391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"147'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644365+0000","last_change":"2024-04-25T10:16:42.555771+0000","last_active":"2024-04-25T10:16:44.644365+0000","last_peered":"2024-04-25T10:16:44.644365+0000","last_clean":"2024-04-25T10:16:44.644365+0000","last_became_active":"2024-04-25T10:16:42.555336+0000","last_became_peered":"2024-04-25T10:16:42.555336+0000","last_unstale":"2024-04-25T10:16:44.644365+0000","last_undegraded":"2024-04-25T10:16:44.644365+0000","last_fullsized":"2024-04-25T10:16:44.644365+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:36:22.397834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562488+0000","last_change":"2024-04-25T10:16:43.540694+0000","last_active":"2024-04-25T10:16:44.562488+0000","last_peered":"2024-04-25T10:16:44.562488+0000","last_clean":"2024-04-25T10:16:44.562488+0000","last_became_active":"2024-04-25T10:16:43.540044+0000","last_became_peered":"2024-04-25T10:16:43.540044+0000","last_unstale":"2024-04-25T10:16:44.562488+0000","last_undegraded":"2024-04-25T10:16:44.562488+0000","last_fullsized":"2024-04-25T10:16:44.562488+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:42:14.118906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537658+0000","last_change":"2024-04-25T10:16:42.561604+0000","last_active":"2024-04-25T10:16:44.537658+0000","last_peered":"2024-04-25T10:16:44.537658+0000","last_clean":"2024-04-25T10:16:44.537658+0000","last_became_active":"2024-04-25T10:16:42.561278+0000","last_became_peered":"2024-04-25T10:16:42.561278+0000","last_unstale":"2024-04-25T10:16:44.537658+0000","last_undegraded":"2024-04-25T10:16:44.537658+0000","last_fullsized":"2024-04-25T10:16:44.537658+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:52.219063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567427+0000","last_change":"2024-04-25T10:16:43.560700+0000","last_active":"2024-04-25T10:16:44.567427+0000","last_peered":"2024-04-25T10:16:44.567427+0000","last_clean":"2024-04-25T10:16:44.567427+0000","last_became_active":"2024-04-25T10:16:43.560410+0000","last_became_peered":"2024-04-25T10:16:43.560410+0000","last_unstale":"2024-04-25T10:16:44.567427+0000","last_undegraded":"2024-04-25T10:16:44.567427+0000","last_fullsized":"2024-04-25T10:16:44.567427+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:31:34.167604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258171+0000","last_change":"2024-04-25T10:16:42.552123+0000","last_active":"2024-04-25T10:16:45.258171+0000","last_peered":"2024-04-25T10:16:45.258171+0000","last_clean":"2024-04-25T10:16:45.258171+0000","last_became_active":"2024-04-25T10:16:42.551840+0000","last_became_peered":"2024-04-25T10:16:42.551840+0000","last_unstale":"2024-04-25T10:16:45.258171+0000","last_undegraded":"2024-04-25T10:16:45.258171+0000","last_fullsized":"2024-04-25T10:16:45.258171+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:02:11.016703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644307+0000","last_change":"2024-04-25T10:16:43.535347+0000","last_active":"2024-04-25T10:16:44.644307+0000","last_peered":"2024-04-25T10:16:44.644307+0000","last_clean":"2024-04-25T10:16:44.644307+0000","last_became_active":"2024-04-25T10:16:43.533702+0000","last_became_peered":"2024-04-25T10:16:43.533702+0000","last_unstale":"2024-04-25T10:16:44.644307+0000","last_undegraded":"2024-04-25T10:16:44.644307+0000","last_fullsized":"2024-04-25T10:16:44.644307+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:00:57.980294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538061+0000","last_change":"2024-04-25T10:16:42.555853+0000","last_active":"2024-04-25T10:16:44.538061+0000","last_peered":"2024-04-25T10:16:44.538061+0000","last_clean":"2024-04-25T10:16:44.538061+0000","last_became_active":"2024-04-25T10:16:42.555550+0000","last_became_peered":"2024-04-25T10:16:42.555550+0000","last_unstale":"2024-04-25T10:16:44.538061+0000","last_undegraded":"2024-04-25T10:16:44.538061+0000","last_fullsized":"2024-04-25T10:16:44.538061+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:01.727950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567403+0000","last_change":"2024-04-25T10:16:43.556539+0000","last_active":"2024-04-25T10:16:44.567403+0000","last_peered":"2024-04-25T10:16:44.567403+0000","last_clean":"2024-04-25T10:16:44.567403+0000","last_became_active":"2024-04-25T10:16:43.556219+0000","last_became_peered":"2024-04-25T10:16:43.556219+0000","last_unstale":"2024-04-25T10:16:44.567403+0000","last_undegraded":"2024-04-25T10:16:44.567403+0000","last_fullsized":"2024-04-25T10:16:44.567403+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:15:36.967856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.257333+0000","last_change":"2024-04-25T10:16:42.551277+0000","last_active":"2024-04-25T10:16:45.257333+0000","last_peered":"2024-04-25T10:16:45.257333+0000","last_clean":"2024-04-25T10:16:45.257333+0000","last_became_active":"2024-04-25T10:16:42.550967+0000","last_became_peered":"2024-04-25T10:16:42.550967+0000","last_unstale":"2024-04-25T10:16:45.257333+0000","last_undegraded":"2024-04-25T10:16:45.257333+0000","last_fullsized":"2024-04-25T10:16:45.257333+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:19.550214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644311+0000","last_change":"2024-04-25T10:16:43.546966+0000","last_active":"2024-04-25T10:16:44.644311+0000","last_peered":"2024-04-25T10:16:44.644311+0000","last_clean":"2024-04-25T10:16:44.644311+0000","last_became_active":"2024-04-25T10:16:43.546529+0000","last_became_peered":"2024-04-25T10:16:43.546529+0000","last_unstale":"2024-04-25T10:16:44.644311+0000","last_undegraded":"2024-04-25T10:16:44.644311+0000","last_fullsized":"2024-04-25T10:16:44.644311+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:03:45.108391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537963+0000","last_change":"2024-04-25T10:16:42.561957+0000","last_active":"2024-04-25T10:16:44.537963+0000","last_peered":"2024-04-25T10:16:44.537963+0000","last_clean":"2024-04-25T10:16:44.537963+0000","last_became_active":"2024-04-25T10:16:42.561460+0000","last_became_peered":"2024-04-25T10:16:42.561460+0000","last_unstale":"2024-04-25T10:16:44.537963+0000","last_undegraded":"2024-04-25T10:16:44.537963+0000","last_fullsized":"2024-04-25T10:16:44.537963+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:36.001686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258883+0000","last_change":"2024-04-25T10:16:43.555830+0000","last_active":"2024-04-25T10:16:45.258883+0000","last_peered":"2024-04-25T10:16:45.258883+0000","last_clean":"2024-04-25T10:16:45.258883+0000","last_became_active":"2024-04-25T10:16:43.554690+0000","last_became_peered":"2024-04-25T10:16:43.554690+0000","last_unstale":"2024-04-25T10:16:45.258883+0000","last_undegraded":"2024-04-25T10:16:45.258883+0000","last_fullsized":"2024-04-25T10:16:45.258883+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:20.566750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"147'2","reported_seq":16,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539527+0000","last_change":"2024-04-25T10:16:42.561522+0000","last_active":"2024-04-25T10:16:44.539527+0000","last_peered":"2024-04-25T10:16:44.539527+0000","last_clean":"2024-04-25T10:16:44.539527+0000","last_became_active":"2024-04-25T10:16:42.561181+0000","last_became_peered":"2024-04-25T10:16:42.561181+0000","last_unstale":"2024-04-25T10:16:44.539527+0000","last_undegraded":"2024-04-25T10:16:44.539527+0000","last_fullsized":"2024-04-25T10:16:44.539527+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:37:33.277014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538833+0000","last_change":"2024-04-25T10:16:43.559320+0000","last_active":"2024-04-25T10:16:44.538833+0000","last_peered":"2024-04-25T10:16:44.538833+0000","last_clean":"2024-04-25T10:16:44.538833+0000","last_became_active":"2024-04-25T10:16:43.559034+0000","last_became_peered":"2024-04-25T10:16:43.559034+0000","last_unstale":"2024-04-25T10:16:44.538833+0000","last_undegraded":"2024-04-25T10:16:44.538833+0000","last_fullsized":"2024-04-25T10:16:44.538833+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:53:01.802584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"148'1","reported_seq":15,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562091+0000","last_change":"2024-04-25T10:16:42.570360+0000","last_active":"2024-04-25T10:16:44.562091+0000","last_peered":"2024-04-25T10:16:44.562091+0000","last_clean":"2024-04-25T10:16:44.562091+0000","last_became_active":"2024-04-25T10:16:42.569941+0000","last_became_peered":"2024-04-25T10:16:42.569941+0000","last_unstale":"2024-04-25T10:16:44.562091+0000","last_undegraded":"2024-04-25T10:16:44.562091+0000","last_fullsized":"2024-04-25T10:16:44.562091+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:03:38.817632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.562408+0000","last_change":"2024-04-25T10:16:43.557368+0000","last_active":"2024-04-25T10:16:44.562408+0000","last_peered":"2024-04-25T10:16:44.562408+0000","last_clean":"2024-04-25T10:16:44.562408+0000","last_became_active":"2024-04-25T10:16:43.557070+0000","last_became_peered":"2024-04-25T10:16:43.557070+0000","last_unstale":"2024-04-25T10:16:44.562408+0000","last_undegraded":"2024-04-25T10:16:44.562408+0000","last_fullsized":"2024-04-25T10:16:44.562408+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:40:55.833357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644415+0000","last_change":"2024-04-25T10:16:42.553757+0000","last_active":"2024-04-25T10:16:44.644415+0000","last_peered":"2024-04-25T10:16:44.644415+0000","last_clean":"2024-04-25T10:16:44.644415+0000","last_became_active":"2024-04-25T10:16:42.552414+0000","last_became_peered":"2024-04-25T10:16:42.552414+0000","last_unstale":"2024-04-25T10:16:44.644415+0000","last_undegraded":"2024-04-25T10:16:44.644415+0000","last_fullsized":"2024-04-25T10:16:44.644415+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:29.359282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259472+0000","last_change":"2024-04-25T10:16:43.551110+0000","last_active":"2024-04-25T10:16:45.259472+0000","last_peered":"2024-04-25T10:16:45.259472+0000","last_clean":"2024-04-25T10:16:45.259472+0000","last_became_active":"2024-04-25T10:16:43.550776+0000","last_became_peered":"2024-04-25T10:16:43.550776+0000","last_unstale":"2024-04-25T10:16:45.259472+0000","last_undegraded":"2024-04-25T10:16:45.259472+0000","last_fullsized":"2024-04-25T10:16:45.259472+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:23:36.036392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.259025+0000","last_change":"2024-04-25T10:16:42.553623+0000","last_active":"2024-04-25T10:16:45.259025+0000","last_peered":"2024-04-25T10:16:45.259025+0000","last_clean":"2024-04-25T10:16:45.259025+0000","last_became_active":"2024-04-25T10:16:42.553344+0000","last_became_peered":"2024-04-25T10:16:42.553344+0000","last_unstale":"2024-04-25T10:16:45.259025+0000","last_undegraded":"2024-04-25T10:16:45.259025+0000","last_fullsized":"2024-04-25T10:16:45.259025+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:37:09.078537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.537775+0000","last_change":"2024-04-25T10:16:43.557497+0000","last_active":"2024-04-25T10:16:44.537775+0000","last_peered":"2024-04-25T10:16:44.537775+0000","last_clean":"2024-04-25T10:16:44.537775+0000","last_became_active":"2024-04-25T10:16:43.557138+0000","last_became_peered":"2024-04-25T10:16:43.557138+0000","last_unstale":"2024-04-25T10:16:44.537775+0000","last_undegraded":"2024-04-25T10:16:44.537775+0000","last_fullsized":"2024-04-25T10:16:44.537775+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:37:06.866347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561372+0000","last_change":"2024-04-25T10:16:42.569306+0000","last_active":"2024-04-25T10:16:44.561372+0000","last_peered":"2024-04-25T10:16:44.561372+0000","last_clean":"2024-04-25T10:16:44.561372+0000","last_became_active":"2024-04-25T10:16:42.568916+0000","last_became_peered":"2024-04-25T10:16:42.568916+0000","last_unstale":"2024-04-25T10:16:44.561372+0000","last_undegraded":"2024-04-25T10:16:44.561372+0000","last_fullsized":"2024-04-25T10:16:44.561372+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:11.043365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536335+0000","last_change":"2024-04-25T10:16:43.558958+0000","last_active":"2024-04-25T10:16:44.536335+0000","last_peered":"2024-04-25T10:16:44.536335+0000","last_clean":"2024-04-25T10:16:44.536335+0000","last_became_active":"2024-04-25T10:16:43.558167+0000","last_became_peered":"2024-04-25T10:16:43.558167+0000","last_unstale":"2024-04-25T10:16:44.536335+0000","last_undegraded":"2024-04-25T10:16:44.536335+0000","last_fullsized":"2024-04-25T10:16:44.536335+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:48.739294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.536161+0000","last_change":"2024-04-25T10:16:42.560964+0000","last_active":"2024-04-25T10:16:44.536161+0000","last_peered":"2024-04-25T10:16:44.536161+0000","last_clean":"2024-04-25T10:16:44.536161+0000","last_became_active":"2024-04-25T10:16:42.560653+0000","last_became_peered":"2024-04-25T10:16:42.560653+0000","last_unstale":"2024-04-25T10:16:44.536161+0000","last_undegraded":"2024-04-25T10:16:44.536161+0000","last_fullsized":"2024-04-25T10:16:44.536161+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:17:11.536743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.644262+0000","last_change":"2024-04-25T10:16:43.558553+0000","last_active":"2024-04-25T10:16:44.644262+0000","last_peered":"2024-04-25T10:16:44.644262+0000","last_clean":"2024-04-25T10:16:44.644262+0000","last_became_active":"2024-04-25T10:16:43.558286+0000","last_became_peered":"2024-04-25T10:16:43.558286+0000","last_unstale":"2024-04-25T10:16:44.644262+0000","last_undegraded":"2024-04-25T10:16:44.644262+0000","last_fullsized":"2024-04-25T10:16:44.644262+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:35.084637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.561386+0000","last_change":"2024-04-25T10:16:42.567149+0000","last_active":"2024-04-25T10:16:44.561386+0000","last_peered":"2024-04-25T10:16:44.561386+0000","last_clean":"2024-04-25T10:16:44.561386+0000","last_became_active":"2024-04-25T10:16:42.566786+0000","last_became_peered":"2024-04-25T10:16:42.566786+0000","last_unstale":"2024-04-25T10:16:44.561386+0000","last_undegraded":"2024-04-25T10:16:44.561386+0000","last_fullsized":"2024-04-25T10:16:44.561386+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:37:40.477788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.538795+0000","last_change":"2024-04-25T10:16:43.568156+0000","last_active":"2024-04-25T10:16:44.538795+0000","last_peered":"2024-04-25T10:16:44.538795+0000","last_clean":"2024-04-25T10:16:44.538795+0000","last_became_active":"2024-04-25T10:16:43.567851+0000","last_became_peered":"2024-04-25T10:16:43.567851+0000","last_unstale":"2024-04-25T10:16:44.538795+0000","last_undegraded":"2024-04-25T10:16:44.538795+0000","last_fullsized":"2024-04-25T10:16:44.538795+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:15.868630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":14,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567095+0000","last_change":"2024-04-25T10:16:42.564913+0000","last_active":"2024-04-25T10:16:44.567095+0000","last_peered":"2024-04-25T10:16:44.567095+0000","last_clean":"2024-04-25T10:16:44.567095+0000","last_became_active":"2024-04-25T10:16:42.563056+0000","last_became_peered":"2024-04-25T10:16:42.563056+0000","last_unstale":"2024-04-25T10:16:44.567095+0000","last_undegraded":"2024-04-25T10:16:44.567095+0000","last_fullsized":"2024-04-25T10:16:44.567095+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:51:15.848421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":12,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.539580+0000","last_change":"2024-04-25T10:16:43.555075+0000","last_active":"2024-04-25T10:16:44.539580+0000","last_peered":"2024-04-25T10:16:44.539580+0000","last_clean":"2024-04-25T10:16:44.539580+0000","last_became_active":"2024-04-25T10:16:43.554782+0000","last_became_peered":"2024-04-25T10:16:43.554782+0000","last_unstale":"2024-04-25T10:16:44.539580+0000","last_undegraded":"2024-04-25T10:16:44.539580+0000","last_fullsized":"2024-04-25T10:16:44.539580+0000","mapping_epoch":146,"log_start":"0'0","ondisk_log_start":"0'0","created":146,"last_epoch_clean":147,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:42.510632+0000","last_clean_scrub_stamp":"2024-04-25T10:16:42.510632+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:39:30.375818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"147'6","reported_seq":20,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:45.258556+0000","last_change":"2024-04-25T10:16:42.553024+0000","last_active":"2024-04-25T10:16:45.258556+0000","last_peered":"2024-04-25T10:16:45.258556+0000","last_clean":"2024-04-25T10:16:45.258556+0000","last_became_active":"2024-04-25T10:16:42.552733+0000","last_became_peered":"2024-04-25T10:16:42.552733+0000","last_unstale":"2024-04-25T10:16:45.258556+0000","last_undegraded":"2024-04-25T10:16:45.258556+0000","last_fullsized":"2024-04-25T10:16:45.258556+0000","mapping_epoch":145,"log_start":"0'0","ondisk_log_start":"0'0","created":145,"last_epoch_clean":146,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:16:41.494096+0000","last_clean_scrub_stamp":"2024-04-25T10:16:41.494096+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:52:12.363657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":414,"reported_epoch":148,"state":"active+clean","last_fresh":"2024-04-25T10:16:44.567417+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:16:44.567417+0000","last_peered":"2024-04-25T10:16:44.567417+0000","last_clean":"2024-04-25T10:16:44.567417+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:16:44.567417+0000","last_undegraded":"2024-04-25T10:16:44.567417+0000","last_fullsized":"2024-04-25T10:16:44.567417+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":21,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59948,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311892,"statfs":{"total":96636764160,"available":96575377408,"internally_reserved":0,"allocated":2453504,"data_stored":1462405,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55208,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316632,"statfs":{"total":96636764160,"available":96580231168,"internally_reserved":0,"allocated":1859584,"data_stored":872085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575032,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55204,"kb_used_data":1812,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316636,"statfs":{"total":96636764160,"available":96580235264,"internally_reserved":0,"allocated":1855488,"data_stored":872015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575032,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82924,"kb_used_data":1824,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288916,"statfs":{"total":96636764160,"available":96551849984,"internally_reserved":0,"allocated":1867776,"data_stored":873627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":83032517},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575032,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":82928,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":81086,"kb_avail":94288912,"statfs":{"total":96636764160,"available":96551845888,"internally_reserved":0,"allocated":1863680,"data_stored":872631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":83032519},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575031,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59376,"kb_used_data":1820,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312464,"statfs":{"total":96636764160,"available":96575963136,"internally_reserved":0,"allocated":1863680,"data_stored":872127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575032,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55212,"kb_used_data":1816,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316628,"statfs":{"total":96636764160,"available":96580227072,"internally_reserved":0,"allocated":1859584,"data_stored":872085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":54655429},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575032,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47484,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324356,"statfs":{"total":96636764160,"available":96588140544,"internally_reserved":0,"allocated":2473984,"data_stored":1464701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:16:53.416 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:16:53.417 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:16:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.651+0000 7f129e600640 1 Processor -- start 2024-04-25T10:16:53.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.652+0000 7f129e600640 1 -- start start 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f129e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f129815dd30 0x7f12980cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f129e600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f129806bff0 0x7f12980b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f129e600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129815e110 con 0x7f12980b5970 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f129e600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f129806d000 con 0x7f129806bff0 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f129e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980b2fd0 con 0x7f129815dd30 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f1297e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f129806bff0 0x7f12980b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f1297e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f129806bff0 0x7f12980b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52792/0 (socket says 172.21.15.165:52792) 2024-04-25T10:16:53.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.653+0000 7f1297e00640 1 -- 172.21.15.165:0/286738484 learned_addr learned my addr 172.21.15.165:0/286738484 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.654+0000 7f129d200640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f129815dd30 0x7f12980cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.654+0000 7f1297e00640 1 -- 172.21.15.165:0/286738484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f129815dd30 msgr2=0x7f12980cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.654+0000 7f1297e00640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f129815dd30 0x7f12980cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.654+0000 7f1297e00640 1 -- 172.21.15.165:0/286738484 >> v1:172.21.15.172:6789/0 conn(0x7f12980b5970 legacy=0x7f1298161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.654+0000 7f1297e00640 1 -- 172.21.15.165:0/286738484 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f129806c480 con 0x7f129806bff0 2024-04-25T10:16:53.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.654+0000 7f129d200640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f129815dd30 0x7f12980cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.655+0000 7f1297e00640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f129806bff0 0x7f12980b5430 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1280009e50 tx=0x7f128002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82e0ea846d07a7f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.655+0000 7f1297400640 1 -- 172.21.15.165:0/286738484 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1280036070 con 0x7f129806bff0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.655+0000 7f1297400640 1 -- 172.21.15.165:0/286738484 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1280031600 con 0x7f129806bff0 2024-04-25T10:16:53.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.655+0000 7f1297400640 1 -- 172.21.15.165:0/286738484 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800319d0 con 0x7f129806bff0 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 -- 172.21.15.165:0/286738484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f129806bff0 msgr2=0x7f12980b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f129806bff0 0x7f12980b5430 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f1280009e50 tx=0x7f128002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 -- 172.21.15.165:0/286738484 shutdown_connections 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f129806bff0 0x7f12980b5430 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 --2- 172.21.15.165:0/286738484 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f129815dd30 0x7f12980cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 -- 172.21.15.165:0/286738484 >> 172.21.15.165:0/286738484 conn(0x7f12980d0610 msgr2=0x7f12980d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 -- 172.21.15.165:0/286738484 shutdown_connections 2024-04-25T10:16:53.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.656+0000 7f129e600640 1 -- 172.21.15.165:0/286738484 wait complete. 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.658+0000 7f129e600640 1 Processor -- start 2024-04-25T10:16:53.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.658+0000 7f129e600640 1 -- start start 2024-04-25T10:16:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 0x7f1298153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f129815dd30 0x7f1298152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12980cbad0 0x7f12980cbf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129806cd20 con 0x7f12980b5970 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f129806ca40 con 0x7f129815dd30 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12980cf110 con 0x7f12980cbad0 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 0x7f1298153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 0x7f1298153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52802/0 (socket says 172.21.15.165:52802) 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 -- 172.21.15.165:0/1644255639 learned_addr learned my addr 172.21.15.165:0/1644255639 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 -- 172.21.15.165:0/1644255639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12980cbad0 msgr2=0x7f12980cbf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12980cbad0 0x7f12980cbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 -- 172.21.15.165:0/1644255639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f129815dd30 msgr2=0x7f1298152cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f129815dd30 0x7f1298152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.659+0000 7f129d200640 1 -- 172.21.15.165:0/1644255639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1298156c50 con 0x7f12980b5970 2024-04-25T10:16:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.660+0000 7f129d200640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 0x7f1298153f90 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f128800c7f0 tx=0x7f128800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.660+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288015070 con 0x7f12980b5970 2024-04-25T10:16:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.660+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1288007820 con 0x7f12980b5970 2024-04-25T10:16:53.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.660+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288007bd0 con 0x7f12980b5970 2024-04-25T10:16:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.660+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f129815e9e0 con 0x7f12980b5970 2024-04-25T10:16:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.660+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12981f3770 con 0x7f12980b5970 2024-04-25T10:16:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.661+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1298073990 con 0x7f12980b5970 2024-04-25T10:16:53.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.662+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1288018070 con 0x7f12980b5970 2024-04-25T10:16:53.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.663+0000 7f1294c00640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f127406b4d0 0x7f127406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:53.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.663+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f128808a4d0 con 0x7f12980b5970 2024-04-25T10:16:53.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.663+0000 7f1297e00640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f127406b4d0 0x7f127406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:53.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.664+0000 7f1297e00640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f127406b4d0 0x7f127406d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1280033000 tx=0x7f128002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:53.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.667+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f12880591b0 con 0x7f12980b5970 2024-04-25T10:16:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.933+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f12980d16c0 con 0x7f12980b5970 2024-04-25T10:16:53.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.937+0000 7f1294c00640 1 -- 172.21.15.165:0/1644255639 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+147 (secure 0 0 0) 0x7f1288058ab0 con 0x7f12980b5970 2024-04-25T10:16:53.940 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:16:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:16:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:16:53.940 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:16:53.941 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:16:53.941 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.939+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f127406b4d0 msgr2=0x7f127406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.939+0000 7f129e600640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f127406b4d0 0x7f127406d990 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f1280033000 tx=0x7f128002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.939+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 msgr2=0x7f1298153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.939+0000 7f129e600640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 0x7f1298153f90 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f128800c7f0 tx=0x7f128800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 shutdown_connections 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f127406b4d0 0x7f127406d990 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f12980cbad0 0x7f12980cbf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f129815dd30 0x7f1298152cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 --2- 172.21.15.165:0/1644255639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f12980b5970 0x7f1298153f90 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:53.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 >> 172.21.15.165:0/1644255639 conn(0x7f12980d0610 msgr2=0x7f12980b8890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:53.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 shutdown_connections 2024-04-25T10:16:53.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:53.940+0000 7f129e600640 1 -- 172.21.15.165:0/1644255639 wait complete. 2024-04-25T10:16:53.961 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:16:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.156+0000 7f4d6ea00640 1 Processor -- start 2024-04-25T10:16:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.156+0000 7f4d6ea00640 1 -- start start 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4d68144440 0x7f4d68070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4d68071530 0x7f4d680cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6ea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d6815bf30 con 0x7f4d680cd860 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6ea00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d680719a0 con 0x7f4d68144440 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d6815c860 con 0x7f4d68071530 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6d600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4d68144440 0x7f4d68070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6e000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4d680cd860 0x7f4d6806d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50676/0 (socket says 172.21.15.165:50676) 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6e000640 1 -- 172.21.15.165:0/1254557924 learned_addr learned my addr 172.21.15.165:0/1254557924 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.158+0000 7f4d6cc00640 1 --2- 172.21.15.165:0/1254557924 >> v2:172.21.15.172:3300/0 conn(0x7f4d68071530 0x7f4d680cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.159+0000 7f4d6d600640 1 -- 172.21.15.165:0/1254557924 >> v2:172.21.15.172:3300/0 conn(0x7f4d68071530 msgr2=0x7f4d680cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.159+0000 7f4d6d600640 1 --2- 172.21.15.165:0/1254557924 >> v2:172.21.15.172:3300/0 conn(0x7f4d68071530 0x7f4d680cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.159+0000 7f4d6d600640 1 -- 172.21.15.165:0/1254557924 >> v1:172.21.15.165:6789/0 conn(0x7f4d680cd860 legacy=0x7f4d6806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.159+0000 7f4d6d600640 1 -- 172.21.15.165:0/1254557924 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6815c210 con 0x7f4d68144440 2024-04-25T10:16:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.159+0000 7f4d67e00640 1 -- 172.21.15.165:0/1254557924 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738260695 0 0) 0x7f4d6815bf30 con 0x7f4d680cd860 2024-04-25T10:16:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.160+0000 7f4d6d600640 1 --2- 172.21.15.165:0/1254557924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d68070ff0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f4d6806b970 tx=0x7f4d5c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6834cf63a5659065 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.160+0000 7f4d67e00640 1 -- 172.21.15.165:0/1254557924 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c037070 con 0x7f4d68144440 2024-04-25T10:16:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.160+0000 7f4d67e00640 1 -- 172.21.15.165:0/1254557924 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d5c032440 con 0x7f4d68144440 2024-04-25T10:16:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.160+0000 7f4d67e00640 1 -- 172.21.15.165:0/1254557924 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5c032810 con 0x7f4d68144440 2024-04-25T10:16:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 -- 172.21.15.165:0/1254557924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 msgr2=0x7f4d68070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/1254557924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d68070ff0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7f4d6806b970 tx=0x7f4d5c02a230 comp rx=0 tx=0).stop 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 -- 172.21.15.165:0/1254557924 shutdown_connections 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/1254557924 >> v2:172.21.15.172:3300/0 conn(0x7f4d68071530 0x7f4d680cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/1254557924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d68070ff0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 -- 172.21.15.165:0/1254557924 >> 172.21.15.165:0/1254557924 conn(0x7f4d6806abe0 msgr2=0x7f4d6806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 -- 172.21.15.165:0/1254557924 shutdown_connections 2024-04-25T10:16:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.161+0000 7f4d6ea00640 1 -- 172.21.15.165:0/1254557924 wait complete. 2024-04-25T10:16:54.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.163+0000 7f4d6ea00640 1 Processor -- start 2024-04-25T10:16:54.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.163+0000 7f4d6ea00640 1 -- start start 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d68071530 0x7f4d681f56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d680cd860 0x7f4d680c8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d680c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680cdcd0 con 0x7f4d68144440 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6815bc30 con 0x7f4d68071530 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d680cde50 con 0x7f4d680cd860 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d680c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d680c8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52816/0 (socket says 172.21.15.165:52816) 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 -- 172.21.15.165:0/3957211240 learned_addr learned my addr 172.21.15.165:0/3957211240 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6d600640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d68071530 0x7f4d681f56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6cc00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d680cd860 0x7f4d680c8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 -- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d680cd860 msgr2=0x7f4d680c8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d680cd860 0x7f4d680c8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 -- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d68071530 msgr2=0x7f4d681f56c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d68071530 0x7f4d681f56c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6e000640 1 -- 172.21.15.165:0/3957211240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d681f1b40 con 0x7f4d68144440 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6d600640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d68071530 0x7f4d681f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.164+0000 7f4d6cc00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d680cd860 0x7f4d680c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.165+0000 7f4d6e000640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d680c8cb0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c7a0 tx=0x7f4d6000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.165+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d60018070 con 0x7f4d68144440 2024-04-25T10:16:54.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.165+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d6000a640 con 0x7f4d68144440 2024-04-25T10:16:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.165+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6000aab0 con 0x7f4d68144440 2024-04-25T10:16:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.165+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d681f1e20 con 0x7f4d68144440 2024-04-25T10:16:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.165+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681f23e0 con 0x7f4d68144440 2024-04-25T10:16:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.166+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6806b4f0 con 0x7f4d68144440 2024-04-25T10:16:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.167+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4d6000ac50 con 0x7f4d68144440 2024-04-25T10:16:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.168+0000 7f4d65600640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4d4806b4d0 0x7f4d4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.168+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4d60089a30 con 0x7f4d68144440 2024-04-25T10:16:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.168+0000 7f4d6d600640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4d4806b4d0 0x7f4d4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.169+0000 7f4d6d600640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4d4806b4d0 0x7f4d4806d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c034000 tx=0x7f4d5c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:54.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.172+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4d6806b4f0 con 0x7f4d68144440 2024-04-25T10:16:54.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.409+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d68073990 con 0x7f4d68144440 2024-04-25T10:16:54.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.410+0000 7f4d65600640 1 -- 172.21.15.165:0/3957211240 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 101 v101) v1 ==== 95+0+4956 (secure 0 0 0) 0x7f4d60058710 con 0x7f4d68144440 2024-04-25T10:16:54.415 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:16:54.415 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":101,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":9762,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":101,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:16:46.028556+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:16:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 101 2024-04-25T10:16:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.412+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4d4806b4d0 msgr2=0x7f4d4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.412+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4d4806b4d0 0x7f4d4806d990 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c034000 tx=0x7f4d5c02e000 comp rx=0 tx=0).stop 2024-04-25T10:16:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.412+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 msgr2=0x7f4d680c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.412+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d680c8cb0 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f4d6000c7a0 tx=0x7f4d6000cc60 comp rx=0 tx=0).stop 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 shutdown_connections 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4d4806b4d0 0x7f4d4806d990 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4d68144440 0x7f4d680c8cb0 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4d680cd860 0x7f4d680c8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 --2- 172.21.15.165:0/3957211240 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4d68071530 0x7f4d681f56c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 >> 172.21.15.165:0/3957211240 conn(0x7f4d6806abe0 msgr2=0x7f4d680c4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 shutdown_connections 2024-04-25T10:16:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:54.413+0000 7f4d6ea00640 1 -- 172.21.15.165:0/3957211240 wait complete. 2024-04-25T10:16:54.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 101, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:16:42.983841+0000', 'modified': '2024-04-25T10:16:46.028556+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9765}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9765': {'gid': 9765, 'name': 'c', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6837/2515540447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2515540447}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2515540447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:16:54.838 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:16:54.838 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:16:55.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.052+0000 7f53f3000640 1 Processor -- start 2024-04-25T10:16:55.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.052+0000 7f53f3000640 1 -- start start 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f3000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 0x7f53ec0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f3000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53ec0b5b40 0x7f53ec0d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f3000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53ec15e110 con 0x7f53ec06bff0 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f3000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53ec06d000 con 0x7f53ec0b5b40 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f3000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0b2fd0 con 0x7f53ec15dd30 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f1200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f53ec06bff0 0x7f53ec0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50688/0 (socket says 172.21.15.165:50688) 2024-04-25T10:16:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.054+0000 7f53f1200640 1 -- 172.21.15.165:0/323119282 learned_addr learned my addr 172.21.15.165:0/323119282 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:16:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194194288 0 0) 0x7f53ec15e110 con 0x7f53ec06bff0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53d4003520 con 0x7f53ec06bff0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53f1c00640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 0x7f53ec0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53f2600640 1 --2- 172.21.15.165:0/323119282 >> v2:172.21.15.172:3300/0 conn(0x7f53ec0b5b40 0x7f53ec0d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3884851561 0 0) 0x7f53d4003520 con 0x7f53ec06bff0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 msgr2=0x7f53ec0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 0x7f53ec0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 >> v2:172.21.15.172:3300/0 conn(0x7f53ec0b5b40 msgr2=0x7f53ec0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 --2- 172.21.15.165:0/323119282 >> v2:172.21.15.172:3300/0 conn(0x7f53ec0b5b40 0x7f53ec0d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec06c480 con 0x7f53ec06bff0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f53dc0019f0 con 0x7f53ec06bff0 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 >> v1:172.21.15.165:6789/0 conn(0x7f53ec06bff0 legacy=0x7f53ec0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 0x7f53d40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004030 0x7f53d4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec06d000 con 0x7f53d4004d40 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0b2fd0 con 0x7f53d4004030 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d400bb20 con 0x7f53ec15dd30 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.055+0000 7f53f2600640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f1c00640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 0x7f53d40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f1200640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004030 0x7f53d4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f2600640 1 -- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 msgr2=0x7f53d40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f2600640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec15dd30 0x7f53d40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f2600640 1 -- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004030 msgr2=0x7f53d4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f2600640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004030 0x7f53d4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f2600640 1 -- 172.21.15.165:0/323119282 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4003520 con 0x7f53d4004d40 2024-04-25T10:16:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.056+0000 7f53f1200640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004030 0x7f53d4004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.057+0000 7f53f2600640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f53e400ba30 tx=0x7f53d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f266f8b0c32bd8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.057+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4030070 con 0x7f53d4004d40 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.057+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e402ae10 con 0x7f53d4004d40 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.057+0000 7f53ebe00640 1 -- 172.21.15.165:0/323119282 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400b600 con 0x7f53d4004d40 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f3000640 1 -- 172.21.15.165:0/323119282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53d4004d40 msgr2=0x7f53d40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f3000640 1 --2- 172.21.15.165:0/323119282 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53d4004d40 0x7f53d40051c0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7f53e400ba30 tx=0x7f53d4005830 comp rx=0 tx=0).stop 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f2600640 1 -- 172.21.15.165:0/323119282 reap_dead start 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f3000640 1 -- 172.21.15.165:0/323119282 shutdown_connections 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f3000640 1 -- 172.21.15.165:0/323119282 >> 172.21.15.165:0/323119282 conn(0x7f53ec0d0610 msgr2=0x7f53ec0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f3000640 1 -- 172.21.15.165:0/323119282 shutdown_connections 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.058+0000 7f53f3000640 1 -- 172.21.15.165:0/323119282 wait complete. 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.060+0000 7f53f3000640 1 Processor -- start 2024-04-25T10:16:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.060+0000 7f53f3000640 1 -- start start 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f3000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004d40 0x7f53ec0cb490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f3000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec0cd3d0 0x7f53ec0ac560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 0x7f53ec0aa8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec06cd20 con 0x7f53ec0aa460 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f3000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec06ca40 con 0x7f53d4004d40 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f3000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec0cebd0 con 0x7f53ec0cd3d0 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 0x7f53ec0aa8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 0x7f53ec0aa8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52830/0 (socket says 172.21.15.165:52830) 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 -- 172.21.15.165:0/3794608624 learned_addr learned my addr 172.21.15.165:0/3794608624 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f1200640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec0cd3d0 0x7f53ec0ac560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 -- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec0cd3d0 msgr2=0x7f53ec0ac560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f1c00640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004d40 0x7f53ec0cb490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec0cd3d0 0x7f53ec0ac560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 -- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004d40 msgr2=0x7f53ec0cb490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004d40 0x7f53ec0cb490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.061+0000 7f53f2600640 1 -- 172.21.15.165:0/3794608624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53ec0cba70 con 0x7f53ec0aa460 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.062+0000 7f53f2600640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 0x7f53ec0aa8e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f53e402a900 tx=0x7f53e4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.062+0000 7f53e9600640 1 -- 172.21.15.165:0/3794608624 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e400b600 con 0x7f53ec0aa460 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.062+0000 7f53e9600640 1 -- 172.21.15.165:0/3794608624 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e402ac10 con 0x7f53ec0aa460 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.062+0000 7f53e9600640 1 -- 172.21.15.165:0/3794608624 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4004260 con 0x7f53ec0aa460 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.062+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53ec0cbd50 con 0x7f53ec0aa460 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.062+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53ec0cd830 con 0x7f53ec0aa460 2024-04-25T10:16:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.064+0000 7f53e9600640 1 -- 172.21.15.165:0/3794608624 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f53e403d650 con 0x7f53ec0aa460 2024-04-25T10:16:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.065+0000 7f53e9600640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.065+0000 7f53e9600640 1 -- 172.21.15.165:0/3794608624 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53e40aa420 con 0x7f53ec0aa460 2024-04-25T10:16:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.066+0000 7f53f1c00640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.066+0000 7f53f7572740 1 Processor -- start 2024-04-25T10:16:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.066+0000 7f53f7572740 1 -- start start 2024-04-25T10:16:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.066+0000 7f53f1c00640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f7572740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 0x55866575b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665733f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866577c7a0 0x5586657322f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f7572740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558665697620 con 0x558665733b30 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558665640ca0 con 0x55866575beb0 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5586656936c0 con 0x55866577c7a0 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 0x55866575b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 0x55866575b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52832/0 (socket says 172.21.15.165:52832) 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.067+0000 7f53f1c00640 1 -- 172.21.15.165:0/975555695 learned_addr learned my addr 172.21.15.165:0/975555695 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1200640 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665733f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f2600640 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866577c7a0 0x5586657322f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1c00640 1 -- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866577c7a0 msgr2=0x5586657322f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1c00640 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866577c7a0 0x5586657322f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1c00640 1 -- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 msgr2=0x558665733f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1c00640 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665733f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1c00640 1 -- 172.21.15.165:0/975555695 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558665647f40 con 0x558665733b30 2024-04-25T10:16:55.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1200640 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665733f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:55.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53f1c00640 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 0x55866575b970 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f53e0010970 tx=0x7f53e0013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57658b9839cac3d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53da000640 1 -- 172.21.15.165:0/975555695 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e001c050 con 0x558665733b30 2024-04-25T10:16:55.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53da000640 1 -- 172.21.15.165:0/975555695 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0026070 con 0x558665733b30 2024-04-25T10:16:55.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.068+0000 7f53da000640 1 -- 172.21.15.165:0/975555695 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e000f040 con 0x558665733b30 2024-04-25T10:16:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 -- 172.21.15.165:0/975555695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 msgr2=0x55866575b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 0x55866575b970 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f53e0010970 tx=0x7f53e0013000 comp rx=0 tx=0).stop 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 -- 172.21.15.165:0/975555695 shutdown_connections 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866577c7a0 0x5586657322f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665733f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 --2- 172.21.15.165:0/975555695 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665733b30 0x55866575b970 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 -- 172.21.15.165:0/975555695 >> 172.21.15.165:0/975555695 conn(0x558665730d50 msgr2=0x558665779730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 -- 172.21.15.165:0/975555695 shutdown_connections 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.069+0000 7f53f7572740 1 -- 172.21.15.165:0/975555695 wait complete. 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.070+0000 7f53f7572740 1 -- start start 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.070+0000 7f53f7572740 1 Processor -- start 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 0x55866575d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x55866575dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x5586657461b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55866575b4e0 con 0x55866575beb0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586656361f0 con 0x55866577c7a0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55866575aed0 con 0x558665745c20 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f7572740 10 client.0 _ll_register_callbacks cb 0x558665653ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 0x55866575d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 0x55866575d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52840/0 (socket says 172.21.15.165:52840) 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.071+0000 7f53f1c00640 1 -- 172.21.15.165:0/1564364365 learned_addr learned my addr 172.21.15.165:0/1564364365 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f2600640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x5586657461b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1c00640 1 -- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 msgr2=0x5586657461b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1200640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x55866575dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1c00640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x5586657461b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1c00640 1 -- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 msgr2=0x55866575dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1c00640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x55866575dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1c00640 1 -- 172.21.15.165:0/1564364365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558665655c50 con 0x55866575beb0 2024-04-25T10:16:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f1c00640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 0x55866575d310 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f53e0008010 tx=0x7f53e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fef8865a609afd55 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53cea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53cea00640 1 -- 172.21.15.165:0/1564364365 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0021bc0 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e0026070 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.072+0000 7f53cea00640 1 -- 172.21.15.165:0/1564364365 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53e0021d60 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53cea00640 1 -- 172.21.15.165:0/1564364365 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e001c020 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 10 client.9939 fetch_fsmap learned FSMap version 101 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x5586653ac2d0 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53cea00640 1 -- 172.21.15.165:0/1564364365 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0026070 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53cea00640 1 -- 172.21.15.165:0/1564364365 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 101) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53e0016420 con 0x55866575beb0 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 10 client.9939 fetch_fsmap finished waiting for FSMap version 101 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 10 client.9939 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 20 client.9939 populate_metadata read hostname 'smithi165' 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657482d0 0x5586657486b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 4 client.9939 mds_command: new command op to 9765 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:16:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 --> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5586656a4b60 con 0x5586657482d0 2024-04-25T10:16:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.073+0000 7f53f1200640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657482d0 0x5586657486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.074+0000 7f53f1200640 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657482d0 0x5586657486b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a23afe4316bafe server_cookie=6784065a7b6bd816 in_seq=0 out_seq=0 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.074+0000 7f53cea00640 10 client.9939 ms_handle_connect on v2:172.21.15.165:6836/2515540447 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53cea00640 1 -- 172.21.15.165:0/1564364365 <== mds.0 v2:172.21.15.165:6836/2515540447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5586656a4b60 con 0x5586657482d0 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53cea00640 10 client.9939 handle_command_reply: tid=0 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 client.9939 shutdown 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 msgr2=0x55866575d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 0x55866575d310 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f53e0008010 tx=0x7f53e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 shutdown_connections 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x5586657461b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657482d0 0x5586657486b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x55866575dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 --2- 172.21.15.165:0/1564364365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866575beb0 0x55866575d310 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 >> 172.21.15.165:0/1564364365 conn(0x558665733b30 msgr2=0x558665779530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53cea00640 0 client.9939 ms_handle_reset on v2:172.21.15.165:6836/2515540447 2024-04-25T10:16:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 shutdown_connections 2024-04-25T10:16:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 1 -- 172.21.15.165:0/1564364365 wait complete. 2024-04-25T10:16:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.075+0000 7f53f7572740 20 client.9939 trim_cache size 0 max 16384 2024-04-25T10:16:55.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.092+0000 7f53f7572740 1 Processor -- start 2024-04-25T10:16:55.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.092+0000 7f53f7572740 1 -- start start 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f7572740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 0x5586657323e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866575beb0 0x558665778630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x558665778b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f7572740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1ffae0 con 0x558665745c20 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1f4ee0 con 0x55866577c7a0 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53ec1ff540 con 0x55866575beb0 2024-04-25T10:16:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 0x5586657323e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 0x5586657323e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52856/0 (socket says 172.21.15.165:52856) 2024-04-25T10:16:55.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 -- 172.21.15.165:0/1625680139 learned_addr learned my addr 172.21.15.165:0/1625680139 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 -- 172.21.15.165:0/1625680139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866575beb0 msgr2=0x558665778630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866575beb0 0x558665778630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 -- 172.21.15.165:0/1625680139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 msgr2=0x558665778b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x558665778b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.093+0000 7f53f1c00640 1 -- 172.21.15.165:0/1625680139 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586654766b0 con 0x558665745c20 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.094+0000 7f53f1c00640 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 0x5586657323e0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bed0 tx=0x7f53e002b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2aabfd40587b3bd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.094+0000 7f53d9600640 1 -- 172.21.15.165:0/1625680139 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e00169f0 con 0x558665745c20 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.094+0000 7f53d9600640 1 -- 172.21.15.165:0/1625680139 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0026040 con 0x558665745c20 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.094+0000 7f53d9600640 1 -- 172.21.15.165:0/1625680139 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e0016420 con 0x558665745c20 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 -- 172.21.15.165:0/1625680139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 msgr2=0x5586657323e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 0x5586657323e0 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f53e000bed0 tx=0x7f53e002b000 comp rx=0 tx=0).stop 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 -- 172.21.15.165:0/1625680139 shutdown_connections 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866577c7a0 0x558665778b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55866575beb0 0x558665778630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 --2- 172.21.15.165:0/1625680139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558665745c20 0x5586657323e0 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 -- 172.21.15.165:0/1625680139 >> 172.21.15.165:0/1625680139 conn(0x558665733b30 msgr2=0x558665756490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 -- 172.21.15.165:0/1625680139 shutdown_connections 2024-04-25T10:16:55.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.095+0000 7f53f7572740 1 -- 172.21.15.165:0/1625680139 wait complete. 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.096+0000 7f53f7572740 1 -- start start 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.096+0000 7f53f7572740 1 Processor -- start 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x558665747bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665778cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 0x5586657ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55866574c2c0 con 0x55866577c7a0 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55866575ce50 con 0x55866575beb0 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55866577ce00 con 0x558665745c20 2024-04-25T10:16:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f7572740 10 client.0 _ll_register_callbacks cb 0x7f53e0010870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 0x5586657ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 0x5586657ebe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52858/0 (socket says 172.21.15.165:52858) 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 -- 172.21.15.165:0/2036556317 learned_addr learned my addr 172.21.15.165:0/2036556317 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f1200640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665778cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 -- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 msgr2=0x558665747bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f1c00640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x558665747bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x558665747bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 -- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 msgr2=0x558665778cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.097+0000 7f53f2600640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665778cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53f2600640 1 -- 172.21.15.165:0/2036556317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55866574b3d0 con 0x55866577c7a0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53f1200640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665778cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53f2600640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 0x5586657ebe70 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f53e4074dd0 tx=0x7f53e4046e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb9571f63286e25c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53cea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:16:55.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53cea00640 1 -- 172.21.15.165:0/2036556317 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4081080 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53cea00640 1 -- 172.21.15.165:0/2036556317 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f53e40aa070 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f53e0026040 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53cea00640 1 -- 172.21.15.165:0/2036556317 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=101) v2 ==== 24+0+0 (secure 0 0 0) 0x7f53e4080bf0 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53f7572740 10 client.9945 fetch_fsmap learned FSMap version 101 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=101,monmap=2+}) v3 -- 0x558665744e90 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53cea00640 1 -- 172.21.15.165:0/2036556317 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e0026040 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.098+0000 7f53cea00640 1 -- 172.21.15.165:0/2036556317 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 101) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f53e40a8b40 con 0x55866577c7a0 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f7572740 10 client.9945 fetch_fsmap finished waiting for FSMap version 101 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f7572740 10 client.9945 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f7572740 20 client.9945 populate_metadata read hostname 'smithi165' 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f7572740 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657567c0 0x55866575ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f7572740 4 client.9945 mds_command: new command op to 9765 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:16:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 --> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55866569eeb0 con 0x5586657567c0 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f1c00640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657567c0 0x55866575ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53f1c00640 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657567c0 0x55866575ce50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560a2f5b152ea028 server_cookie=2de2702422d788c5 in_seq=0 out_seq=0 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.099+0000 7f53cea00640 10 client.9945 ms_handle_connect on v2:172.21.15.165:6836/2515540447 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53cea00640 1 -- 172.21.15.165:0/2036556317 <== mds.0 v2:172.21.15.165:6836/2515540447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55866569eeb0 con 0x5586657567c0 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53cea00640 10 client.9945 handle_command_reply: tid=0 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 client.9945 shutdown 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 msgr2=0x5586657ebe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 0x5586657ebe70 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f53e4074dd0 tx=0x7f53e4046e30 comp rx=0 tx=0).stop 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 shutdown_connections 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55866577c7a0 0x5586657ebe70 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x5586657567c0 0x55866575ce50 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55866575beb0 0x558665778cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 --2- 172.21.15.165:0/2036556317 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558665745c20 0x558665747bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 >> 172.21.15.165:0/2036556317 conn(0x558665733b30 msgr2=0x55866577bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53cea00640 0 client.9945 ms_handle_reset on v2:172.21.15.165:6836/2515540447 2024-04-25T10:16:55.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 shutdown_connections 2024-04-25T10:16:55.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 1 -- 172.21.15.165:0/2036556317 wait complete. 2024-04-25T10:16:55.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.100+0000 7f53f7572740 20 client.9945 trim_cache size 0 max 16384 2024-04-25T10:16:55.108 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:16:55.108 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:16:55.108 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 9765, 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 11.552207732999999, 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 101, 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 148, 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 148, 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 17.995903382000002 2024-04-25T10:16:55.109 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:16:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53d006b4d0 msgr2=0x7f53d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53d006b4d0 0x7f53d006d990 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f53e0006fd0 tx=0x7f53e0008040 comp rx=0 tx=0).stop 2024-04-25T10:16:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 msgr2=0x7f53ec0aa8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 0x7f53ec0aa8e0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7f53e402a900 tx=0x7f53e4004730 comp rx=0 tx=0).stop 2024-04-25T10:16:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 shutdown_connections 2024-04-25T10:16:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f53d006b4d0 0x7f53d006d990 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53ec0aa460 0x7f53ec0aa8e0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53ec0cd3d0 0x7f53ec0ac560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 --2- 172.21.15.165:0/3794608624 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53d4004d40 0x7f53ec0cb490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 >> 172.21.15.165:0/3794608624 conn(0x7f53ec0d0610 msgr2=0x7f53ec0b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 shutdown_connections 2024-04-25T10:16:55.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.101+0000 7f53f3000640 1 -- 172.21.15.165:0/3794608624 wait complete. 2024-04-25T10:16:55.120 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:16:55.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.301+0000 7ff5f3000640 1 Processor -- start 2024-04-25T10:16:55.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.302+0000 7ff5f3000640 1 -- start start 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.303+0000 7ff5f3000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5ec144440 0x7ff5ec070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.303+0000 7ff5f3000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff5ec0cda30 0x7ff5ec15ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.303+0000 7ff5f3000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5ec15bf30 con 0x7ff5ec071530 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.303+0000 7ff5f3000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec0719a0 con 0x7ff5ec144440 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.303+0000 7ff5f3000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5ec15c860 con 0x7ff5ec0cda30 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5ec144440 0x7ff5ec070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff5ec144440 0x7ff5ec070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52862/0 (socket says 172.21.15.165:52862) 2024-04-25T10:16:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 -- 172.21.15.165:0/2050711903 learned_addr learned my addr 172.21.15.165:0/2050711903 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 -- 172.21.15.165:0/2050711903 >> v2:172.21.15.172:3300/0 conn(0x7ff5ec0cda30 msgr2=0x7ff5ec15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 --2- 172.21.15.165:0/2050711903 >> v2:172.21.15.172:3300/0 conn(0x7ff5ec0cda30 0x7ff5ec15ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 -- 172.21.15.165:0/2050711903 >> v1:172.21.15.172:6789/0 conn(0x7ff5ec071530 legacy=0x7ff5ec0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:16:55.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.304+0000 7ff5f1c00640 1 -- 172.21.15.165:0/2050711903 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec15c210 con 0x7ff5ec144440 2024-04-25T10:16:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.305+0000 7ff5f1c00640 1 --2- 172.21.15.165:0/2050711903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec144440 0x7ff5ec070ff0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0b2280 tx=0x7ff5e002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d421ecfeaf7111 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.305+0000 7ff5ebe00640 1 -- 172.21.15.165:0/2050711903 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0037070 con 0x7ff5ec144440 2024-04-25T10:16:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5ebe00640 1 -- 172.21.15.165:0/2050711903 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5e0032440 con 0x7ff5ec144440 2024-04-25T10:16:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5ebe00640 1 -- 172.21.15.165:0/2050711903 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5e0032810 con 0x7ff5ec144440 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 -- 172.21.15.165:0/2050711903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec144440 msgr2=0x7ff5ec070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 --2- 172.21.15.165:0/2050711903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec144440 0x7ff5ec070ff0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec0b2280 tx=0x7ff5e002a2c0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 -- 172.21.15.165:0/2050711903 shutdown_connections 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 --2- 172.21.15.165:0/2050711903 >> v2:172.21.15.172:3300/0 conn(0x7ff5ec0cda30 0x7ff5ec15ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 --2- 172.21.15.165:0/2050711903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec144440 0x7ff5ec070ff0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 -- 172.21.15.165:0/2050711903 >> 172.21.15.165:0/2050711903 conn(0x7ff5ec06abe0 msgr2=0x7ff5ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 -- 172.21.15.165:0/2050711903 shutdown_connections 2024-04-25T10:16:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.306+0000 7ff5f3000640 1 -- 172.21.15.165:0/2050711903 wait complete. 2024-04-25T10:16:55.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.308+0000 7ff5f3000640 1 Processor -- start 2024-04-25T10:16:55.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.308+0000 7ff5f3000640 1 -- start start 2024-04-25T10:16:55.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f3000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ec071530 0x7ff5ec1e41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f3000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 0x7ff5ec12c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f3000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ec144440 0x7ff5ec1f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f3000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec15bc30 con 0x7ff5ec0cda30 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f3000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec0cdea0 con 0x7ff5ec144440 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f3000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5ec15d230 con 0x7ff5ec071530 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 0x7ff5ec12c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f1200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 0x7ff5ec12c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52868/0 (socket says 172.21.15.165:52868) 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.309+0000 7ff5f1200640 1 -- 172.21.15.165:0/3616994002 learned_addr learned my addr 172.21.15.165:0/3616994002 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f2600640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ec144440 0x7ff5ec1f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1c00640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ec071530 0x7ff5ec1e41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1200640 1 -- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ec071530 msgr2=0x7ff5ec1e41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1200640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ec071530 0x7ff5ec1e41d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1200640 1 -- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ec144440 msgr2=0x7ff5ec1f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1200640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ec144440 0x7ff5ec1f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1200640 1 -- 172.21.15.165:0/3616994002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ec12c810 con 0x7ff5ec0cda30 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f1200640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 0x7ff5ec12c070 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c410 tx=0x7ff5d800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8017070 con 0x7ff5ec0cda30 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5d80035f0 con 0x7ff5ec0cda30 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5ec12caf0 con 0x7ff5ec0cda30 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.310+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5d8003a20 con 0x7ff5ec0cda30 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.311+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5ec12d0b0 con 0x7ff5ec0cda30 2024-04-25T10:16:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.312+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5ec06e270 con 0x7ff5ec0cda30 2024-04-25T10:16:55.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.313+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff5d801a070 con 0x7ff5ec0cda30 2024-04-25T10:16:55.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.313+0000 7ff5e9600640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:16:55.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.314+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff5d8089790 con 0x7ff5ec0cda30 2024-04-25T10:16:55.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.314+0000 7ff5f1c00640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:16:55.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.315+0000 7ff5f1c00640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0034000 tx=0x7ff5e002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:16:55.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.318+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff5ec06e270 con 0x7ff5ec0cda30 2024-04-25T10:16:55.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.550+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff5ec073990 con 0x7ff5ec0cda30 2024-04-25T10:16:55.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.551+0000 7ff5e9600640 1 -- 172.21.15.165:0/3616994002 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v101) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff5d8058480 con 0x7ff5ec0cda30 2024-04-25T10:16:55.555 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:16:55.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 msgr2=0x7ff5cc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7ff5e0034000 tx=0x7ff5e002e000 comp rx=0 tx=0).stop 2024-04-25T10:16:55.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 msgr2=0x7ff5ec12c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:16:55.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 0x7ff5ec12c070 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7ff5d800c410 tx=0x7ff5d800c8d0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 shutdown_connections 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5cc06b4d0 0x7ff5cc06d990 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5ec144440 0x7ff5ec1f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff5ec0cda30 0x7ff5ec12c070 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 --2- 172.21.15.165:0/3616994002 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff5ec071530 0x7ff5ec1e41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 >> 172.21.15.165:0/3616994002 conn(0x7ff5ec06abe0 msgr2=0x7ff5ec140fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 shutdown_connections 2024-04-25T10:16:55.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:16:55.554+0000 7ff5f3000640 1 -- 172.21.15.165:0/3616994002 wait complete. 2024-04-25T10:16:55.576 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:16:55.576 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:16:55.577 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:55.577 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.577 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:16:55.628 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:16:55.628 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:16:55.628 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:16:55.628 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.628 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:16:55.629 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41750sec preferred_lft 41750sec 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:55.630 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:16:55.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.631 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:16:55.705 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:16:55.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.710 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:16:55.762 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:16:55.762 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:16:55.762 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:16:55.762 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:16:55.763 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:16:55.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.763 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:16:55.763 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:16:55.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:16:55.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:16:55.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:16:55.763 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:16:55.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.906 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:16:55.956 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:16:55.957 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:55.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:55.957 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:16:56.040 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:56.040 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.092 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:16:56.092 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:16:56.092 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.138 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:16:56.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.138 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:56.315 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:16:56.337 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.338 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:56.338 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:16:56.338 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:16:56.338 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:16:56.363 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:16:56.367 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:16:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:16:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:16:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:16:56.367 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:16:56.371 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:16:56.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.371 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:16:56.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.447 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:16:56.515 INFO:teuthology.orchestra.run.smithi165.stdout:client9954 2024-04-25T10:16:56.515 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:16:56.515 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:16:56.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.0 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.0/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:16:56.568 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:16:56.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.670 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261894 s, 40.0 MB/s 2024-04-25T10:16:56.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025529 s, 41.1 MB/s 2024-04-25T10:16:56.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230942 s, 45.4 MB/s 2024-04-25T10:16:56.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0396049 s, 26.5 MB/s 2024-04-25T10:16:56.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.795 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.796 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226118 s, 46.4 MB/s 2024-04-25T10:16:56.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.822 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232823 s, 45.0 MB/s 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.851 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225506 s, 46.5 MB/s 2024-04-25T10:16:56.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216097 s, 48.5 MB/s 2024-04-25T10:16:56.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.944 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0624818 s, 16.8 MB/s 2024-04-25T10:16:56.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:56.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:56.984 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0347549 s, 30.2 MB/s 2024-04-25T10:16:57.016 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.016 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.016 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277565 s, 37.8 MB/s 2024-04-25T10:16:57.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.047 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.047 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261415 s, 40.1 MB/s 2024-04-25T10:16:57.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.074 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235418 s, 44.5 MB/s 2024-04-25T10:16:57.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.102 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241054 s, 43.5 MB/s 2024-04-25T10:16:57.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.136 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.136 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0300474 s, 34.9 MB/s 2024-04-25T10:16:57.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235964 s, 44.4 MB/s 2024-04-25T10:16:57.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.192 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236409 s, 44.4 MB/s 2024-04-25T10:16:57.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.222 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269155 s, 39.0 MB/s 2024-04-25T10:16:57.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255487 s, 41.0 MB/s 2024-04-25T10:16:57.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203285 s, 51.6 MB/s 2024-04-25T10:16:57.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.301 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214819 s, 48.8 MB/s 2024-04-25T10:16:57.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253063 s, 41.4 MB/s 2024-04-25T10:16:57.354 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020457 s, 51.3 MB/s 2024-04-25T10:16:57.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230298 s, 45.5 MB/s 2024-04-25T10:16:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215828 s, 48.6 MB/s 2024-04-25T10:16:57.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257201 s, 40.8 MB/s 2024-04-25T10:16:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.463 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217525 s, 48.2 MB/s 2024-04-25T10:16:57.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.490 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196846 s, 53.3 MB/s 2024-04-25T10:16:57.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.515 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235918 s, 44.4 MB/s 2024-04-25T10:16:57.541 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.541 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.541 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221747 s, 47.3 MB/s 2024-04-25T10:16:57.567 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.568 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226837 s, 46.2 MB/s 2024-04-25T10:16:57.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252892 s, 41.5 MB/s 2024-04-25T10:16:57.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.1 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.1/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:16:57.600 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:16:57.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.688 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253784 s, 41.3 MB/s 2024-04-25T10:16:57.715 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.715 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.715 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225272 s, 46.5 MB/s 2024-04-25T10:16:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.744 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02544 s, 41.2 MB/s 2024-04-25T10:16:57.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.769 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.769 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213463 s, 49.1 MB/s 2024-04-25T10:16:57.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192004 s, 54.6 MB/s 2024-04-25T10:16:57.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.814 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.814 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193907 s, 54.1 MB/s 2024-04-25T10:16:57.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.840 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217377 s, 48.2 MB/s 2024-04-25T10:16:57.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.870 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267638 s, 39.2 MB/s 2024-04-25T10:16:57.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.903 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.903 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270954 s, 38.7 MB/s 2024-04-25T10:16:57.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.928 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233353 s, 44.9 MB/s 2024-04-25T10:16:57.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247828 s, 42.3 MB/s 2024-04-25T10:16:57.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:57.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:57.982 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220841 s, 47.5 MB/s 2024-04-25T10:16:58.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.008 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.008 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021935 s, 47.8 MB/s 2024-04-25T10:16:58.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.032 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197287 s, 53.1 MB/s 2024-04-25T10:16:58.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.056 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207735 s, 50.5 MB/s 2024-04-25T10:16:58.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.083 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.083 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232633 s, 45.1 MB/s 2024-04-25T10:16:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212344 s, 49.4 MB/s 2024-04-25T10:16:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212764 s, 49.3 MB/s 2024-04-25T10:16:58.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.158 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207718 s, 50.5 MB/s 2024-04-25T10:16:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.183 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213781 s, 49.0 MB/s 2024-04-25T10:16:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209282 s, 50.1 MB/s 2024-04-25T10:16:58.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.236 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236971 s, 44.2 MB/s 2024-04-25T10:16:58.265 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.265 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.265 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249308 s, 42.1 MB/s 2024-04-25T10:16:58.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.290 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212264 s, 49.4 MB/s 2024-04-25T10:16:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212449 s, 49.4 MB/s 2024-04-25T10:16:58.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211946 s, 49.5 MB/s 2024-04-25T10:16:58.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.365 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.366 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020799 s, 50.4 MB/s 2024-04-25T10:16:58.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.391 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217436 s, 48.2 MB/s 2024-04-25T10:16:58.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.418 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.418 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231637 s, 45.3 MB/s 2024-04-25T10:16:58.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0286406 s, 36.6 MB/s 2024-04-25T10:16:58.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196184 s, 53.4 MB/s 2024-04-25T10:16:58.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.499 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213959 s, 49.0 MB/s 2024-04-25T10:16:58.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.2 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.2/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:16:58.502 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:16:58.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.590 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234601 s, 44.7 MB/s 2024-04-25T10:16:58.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.616 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.616 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219581 s, 47.8 MB/s 2024-04-25T10:16:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200513 s, 52.3 MB/s 2024-04-25T10:16:58.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.664 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.664 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220013 s, 47.7 MB/s 2024-04-25T10:16:58.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.692 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237783 s, 44.1 MB/s 2024-04-25T10:16:58.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.718 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.718 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206794 s, 50.7 MB/s 2024-04-25T10:16:58.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192888 s, 54.4 MB/s 2024-04-25T10:16:58.766 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.767 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.767 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225959 s, 46.4 MB/s 2024-04-25T10:16:58.789 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.790 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.790 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191372 s, 54.8 MB/s 2024-04-25T10:16:58.815 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.816 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220292 s, 47.6 MB/s 2024-04-25T10:16:58.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232098 s, 45.2 MB/s 2024-04-25T10:16:58.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.871 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.871 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221662 s, 47.3 MB/s 2024-04-25T10:16:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.901 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267752 s, 39.2 MB/s 2024-04-25T10:16:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.928 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223921 s, 46.8 MB/s 2024-04-25T10:16:58.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245345 s, 42.7 MB/s 2024-04-25T10:16:58.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:58.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:58.982 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216577 s, 48.4 MB/s 2024-04-25T10:16:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.014 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274465 s, 38.2 MB/s 2024-04-25T10:16:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241612 s, 43.4 MB/s 2024-04-25T10:16:59.072 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.073 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254961 s, 41.1 MB/s 2024-04-25T10:16:59.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.095 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193885 s, 54.1 MB/s 2024-04-25T10:16:59.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.122 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233097 s, 45.0 MB/s 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022276 s, 47.1 MB/s 2024-04-25T10:16:59.171 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192485 s, 54.5 MB/s 2024-04-25T10:16:59.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.200 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225742 s, 46.5 MB/s 2024-04-25T10:16:59.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.228 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.228 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248614 s, 42.2 MB/s 2024-04-25T10:16:59.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216433 s, 48.4 MB/s 2024-04-25T10:16:59.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.279 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.279 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021998 s, 47.7 MB/s 2024-04-25T10:16:59.304 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.304 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.305 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212627 s, 49.3 MB/s 2024-04-25T10:16:59.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.334 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.334 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245518 s, 42.7 MB/s 2024-04-25T10:16:59.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.361 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242336 s, 43.3 MB/s 2024-04-25T10:16:59.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.386 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210169 s, 49.9 MB/s 2024-04-25T10:16:59.421 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.421 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.421 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0313883 s, 33.4 MB/s 2024-04-25T10:16:59.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:16:59.423 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:16:59.424 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:16:59.424 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.3 2024-04-25T10:16:59.424 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:16:59.424 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.3/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:16:59.424 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:16:59.424 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:16:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200013 s, 52.4 MB/s 2024-04-25T10:16:59.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233512 s, 44.9 MB/s 2024-04-25T10:16:59.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.552 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0188297 s, 55.7 MB/s 2024-04-25T10:16:59.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.583 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026688 s, 39.3 MB/s 2024-04-25T10:16:59.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.612 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.612 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252634 s, 41.5 MB/s 2024-04-25T10:16:59.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.636 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.636 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202942 s, 51.7 MB/s 2024-04-25T10:16:59.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.663 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234347 s, 44.7 MB/s 2024-04-25T10:16:59.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.684 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177758 s, 59.0 MB/s 2024-04-25T10:16:59.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.708 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.708 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201563 s, 52.0 MB/s 2024-04-25T10:16:59.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214791 s, 48.8 MB/s 2024-04-25T10:16:59.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207485 s, 50.5 MB/s 2024-04-25T10:16:59.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.781 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201019 s, 52.2 MB/s 2024-04-25T10:16:59.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.809 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232986 s, 45.0 MB/s 2024-04-25T10:16:59.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204705 s, 51.2 MB/s 2024-04-25T10:16:59.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204827 s, 51.2 MB/s 2024-04-25T10:16:59.882 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.883 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.883 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208851 s, 50.2 MB/s 2024-04-25T10:16:59.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.908 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.908 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212206 s, 49.4 MB/s 2024-04-25T10:16:59.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.933 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.933 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02115 s, 49.6 MB/s 2024-04-25T10:16:59.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.955 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.956 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189959 s, 55.2 MB/s 2024-04-25T10:16:59.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:16:59.982 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:16:59.982 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222193 s, 47.2 MB/s 2024-04-25T10:17:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195917 s, 53.5 MB/s 2024-04-25T10:17:00.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.033 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226498 s, 46.3 MB/s 2024-04-25T10:17:00.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.056 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.056 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196121 s, 53.5 MB/s 2024-04-25T10:17:00.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192171 s, 54.6 MB/s 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.105 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214447 s, 48.9 MB/s 2024-04-25T10:17:00.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.133 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.133 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242126 s, 43.3 MB/s 2024-04-25T10:17:00.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.162 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.162 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249732 s, 42.0 MB/s 2024-04-25T10:17:00.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269584 s, 38.9 MB/s 2024-04-25T10:17:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234535 s, 44.7 MB/s 2024-04-25T10:17:00.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.243 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.243 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197567 s, 53.1 MB/s 2024-04-25T10:17:00.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025707 s, 40.8 MB/s 2024-04-25T10:17:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.297 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209597 s, 50.0 MB/s 2024-04-25T10:17:00.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.4 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.4/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:00.300 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:00.394 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0309172 s, 33.9 MB/s 2024-04-25T10:17:00.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.420 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212524 s, 49.3 MB/s 2024-04-25T10:17:00.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230436 s, 45.5 MB/s 2024-04-25T10:17:00.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020247 s, 51.8 MB/s 2024-04-25T10:17:00.473 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:00.480 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:00.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.498 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.498 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223303 s, 47.0 MB/s 2024-04-25T10:17:00.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.521 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.521 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197432 s, 53.1 MB/s 2024-04-25T10:17:00.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.544 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204444 s, 51.3 MB/s 2024-04-25T10:17:00.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0180625 s, 58.1 MB/s 2024-04-25T10:17:00.592 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.592 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.593 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240358 s, 43.6 MB/s 2024-04-25T10:17:00.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.620 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237034 s, 44.2 MB/s 2024-04-25T10:17:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228689 s, 45.9 MB/s 2024-04-25T10:17:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250563 s, 41.8 MB/s 2024-04-25T10:17:00.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208893 s, 50.2 MB/s 2024-04-25T10:17:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.727 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221617 s, 47.3 MB/s 2024-04-25T10:17:00.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.755 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235035 s, 44.6 MB/s 2024-04-25T10:17:00.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.777 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184157 s, 56.9 MB/s 2024-04-25T10:17:00.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.802 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212085 s, 49.4 MB/s 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249049 s, 42.1 MB/s 2024-04-25T10:17:00.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208689 s, 50.2 MB/s 2024-04-25T10:17:00.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.884 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241344 s, 43.4 MB/s 2024-04-25T10:17:00.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204177 s, 51.4 MB/s 2024-04-25T10:17:00.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223946 s, 46.8 MB/s 2024-04-25T10:17:00.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222439 s, 47.1 MB/s 2024-04-25T10:17:00.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:00.984 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:00.985 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186388 s, 56.3 MB/s 2024-04-25T10:17:01.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02273 s, 46.1 MB/s 2024-04-25T10:17:01.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.038 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.038 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215595 s, 48.6 MB/s 2024-04-25T10:17:01.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.064 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239412 s, 43.8 MB/s 2024-04-25T10:17:01.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234152 s, 44.8 MB/s 2024-04-25T10:17:01.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.115 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191526 s, 54.7 MB/s 2024-04-25T10:17:01.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.141 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.141 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221304 s, 47.4 MB/s 2024-04-25T10:17:01.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.164 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182546 s, 57.4 MB/s 2024-04-25T10:17:01.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.189 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021072 s, 49.8 MB/s 2024-04-25T10:17:01.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:01.191 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:01.191 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:01.191 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.5 2024-04-25T10:17:01.191 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:01.191 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.5/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:01.191 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:01.192 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:01.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.322 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262366 s, 40.0 MB/s 2024-04-25T10:17:01.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221184 s, 47.4 MB/s 2024-04-25T10:17:01.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.370 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.370 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018529 s, 56.6 MB/s 2024-04-25T10:17:01.394 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.394 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.394 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195701 s, 53.6 MB/s 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.419 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209356 s, 50.1 MB/s 2024-04-25T10:17:01.445 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.445 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.445 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212809 s, 49.3 MB/s 2024-04-25T10:17:01.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0265475 s, 39.5 MB/s 2024-04-25T10:17:01.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.499 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.499 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208669 s, 50.3 MB/s 2024-04-25T10:17:01.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270506 s, 38.8 MB/s 2024-04-25T10:17:01.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.558 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226343 s, 46.3 MB/s 2024-04-25T10:17:01.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.586 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214758 s, 48.8 MB/s 2024-04-25T10:17:01.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207035 s, 50.6 MB/s 2024-04-25T10:17:01.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.632 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202673 s, 51.7 MB/s 2024-04-25T10:17:01.656 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02075 s, 50.5 MB/s 2024-04-25T10:17:01.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.686 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252849 s, 41.5 MB/s 2024-04-25T10:17:01.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204033 s, 51.4 MB/s 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.742 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266527 s, 39.3 MB/s 2024-04-25T10:17:01.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.768 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.769 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226972 s, 46.2 MB/s 2024-04-25T10:17:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220053 s, 47.7 MB/s 2024-04-25T10:17:01.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026058 s, 40.2 MB/s 2024-04-25T10:17:01.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210098 s, 49.9 MB/s 2024-04-25T10:17:01.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.878 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241464 s, 43.4 MB/s 2024-04-25T10:17:01.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243088 s, 43.1 MB/s 2024-04-25T10:17:01.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023553 s, 44.5 MB/s 2024-04-25T10:17:01.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.959 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.959 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208285 s, 50.3 MB/s 2024-04-25T10:17:01.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:01.989 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:01.989 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264484 s, 39.6 MB/s 2024-04-25T10:17:02.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.012 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191537 s, 54.7 MB/s 2024-04-25T10:17:02.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.037 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208996 s, 50.2 MB/s 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.064 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237078 s, 44.2 MB/s 2024-04-25T10:17:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.088 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200164 s, 52.4 MB/s 2024-04-25T10:17:02.108 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.109 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.109 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179492 s, 58.4 MB/s 2024-04-25T10:17:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273032 s, 38.4 MB/s 2024-04-25T10:17:02.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:02.148 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:02.148 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:02.148 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.6 2024-04-25T10:17:02.148 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:02.148 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.6/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:02.149 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:02.149 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:02.283 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.284 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.284 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224237 s, 46.8 MB/s 2024-04-25T10:17:02.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.313 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245045 s, 42.8 MB/s 2024-04-25T10:17:02.336 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.336 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.336 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203405 s, 51.6 MB/s 2024-04-25T10:17:02.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.367 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252735 s, 41.5 MB/s 2024-04-25T10:17:02.391 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.392 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.392 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208513 s, 50.3 MB/s 2024-04-25T10:17:02.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200847 s, 52.2 MB/s 2024-04-25T10:17:02.443 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.444 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246284 s, 42.6 MB/s 2024-04-25T10:17:02.468 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.469 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.469 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210218 s, 49.9 MB/s 2024-04-25T10:17:02.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.489 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173258 s, 60.5 MB/s 2024-04-25T10:17:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262634 s, 39.9 MB/s 2024-04-25T10:17:02.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.547 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.547 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234932 s, 44.6 MB/s 2024-04-25T10:17:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205415 s, 51.0 MB/s 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.596 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205524 s, 51.0 MB/s 2024-04-25T10:17:02.622 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227031 s, 46.2 MB/s 2024-04-25T10:17:02.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219913 s, 47.7 MB/s 2024-04-25T10:17:02.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.674 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204989 s, 51.2 MB/s 2024-04-25T10:17:02.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.705 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262542 s, 39.9 MB/s 2024-04-25T10:17:02.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241444 s, 43.4 MB/s 2024-04-25T10:17:02.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206981 s, 50.7 MB/s 2024-04-25T10:17:02.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.787 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.788 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261961 s, 40.0 MB/s 2024-04-25T10:17:02.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.811 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.811 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185201 s, 56.6 MB/s 2024-04-25T10:17:02.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208963 s, 50.2 MB/s 2024-04-25T10:17:02.863 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.863 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.863 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224886 s, 46.6 MB/s 2024-04-25T10:17:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259645 s, 40.4 MB/s 2024-04-25T10:17:02.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.920 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.920 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217846 s, 48.1 MB/s 2024-04-25T10:17:02.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.946 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221323 s, 47.4 MB/s 2024-04-25T10:17:02.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:02.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:02.974 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236962 s, 44.3 MB/s 2024-04-25T10:17:03.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.005 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269579 s, 38.9 MB/s 2024-04-25T10:17:03.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.033 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241003 s, 43.5 MB/s 2024-04-25T10:17:03.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.062 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.062 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247243 s, 42.4 MB/s 2024-04-25T10:17:03.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.089 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232204 s, 45.2 MB/s 2024-04-25T10:17:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229911 s, 45.6 MB/s 2024-04-25T10:17:03.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.7 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.7/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:03.119 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:03.203 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.203 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.203 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206892 s, 50.7 MB/s 2024-04-25T10:17:03.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.226 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198002 s, 53.0 MB/s 2024-04-25T10:17:03.252 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.253 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229032 s, 45.8 MB/s 2024-04-25T10:17:03.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.280 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.280 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233282 s, 44.9 MB/s 2024-04-25T10:17:03.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.309 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.309 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251663 s, 41.7 MB/s 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.340 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269148 s, 39.0 MB/s 2024-04-25T10:17:03.362 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.363 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189011 s, 55.5 MB/s 2024-04-25T10:17:03.388 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.389 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217517 s, 48.2 MB/s 2024-04-25T10:17:03.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.413 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.413 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204127 s, 51.4 MB/s 2024-04-25T10:17:03.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.436 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.436 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193168 s, 54.3 MB/s 2024-04-25T10:17:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223388 s, 46.9 MB/s 2024-04-25T10:17:03.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.490 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240551 s, 43.6 MB/s 2024-04-25T10:17:03.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.515 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203785 s, 51.5 MB/s 2024-04-25T10:17:03.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.543 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240349 s, 43.6 MB/s 2024-04-25T10:17:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262045 s, 40.0 MB/s 2024-04-25T10:17:03.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.598 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.598 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197515 s, 53.1 MB/s 2024-04-25T10:17:03.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209184 s, 50.1 MB/s 2024-04-25T10:17:03.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.647 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218214 s, 48.1 MB/s 2024-04-25T10:17:03.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.672 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213835 s, 49.0 MB/s 2024-04-25T10:17:03.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.698 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225256 s, 46.6 MB/s 2024-04-25T10:17:03.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206438 s, 50.8 MB/s 2024-04-25T10:17:03.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.756 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267866 s, 39.1 MB/s 2024-04-25T10:17:03.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.784 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224433 s, 46.7 MB/s 2024-04-25T10:17:03.804 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.804 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.804 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0175327 s, 59.8 MB/s 2024-04-25T10:17:03.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241208 s, 43.5 MB/s 2024-04-25T10:17:03.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019829 s, 52.9 MB/s 2024-04-25T10:17:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.881 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208948 s, 50.2 MB/s 2024-04-25T10:17:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213906 s, 49.0 MB/s 2024-04-25T10:17:03.930 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.931 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203846 s, 51.4 MB/s 2024-04-25T10:17:03.956 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.957 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208204 s, 50.4 MB/s 2024-04-25T10:17:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:03.991 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.030224 s, 34.7 MB/s 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201457 s, 52.0 MB/s 2024-04-25T10:17:04.018 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.8 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.8/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:04.018 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:04.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212531 s, 49.3 MB/s 2024-04-25T10:17:04.128 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.128 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.128 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231776 s, 45.2 MB/s 2024-04-25T10:17:04.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.150 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.150 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183375 s, 57.2 MB/s 2024-04-25T10:17:04.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.177 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.177 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241788 s, 43.4 MB/s 2024-04-25T10:17:04.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221044 s, 47.4 MB/s 2024-04-25T10:17:04.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232749 s, 45.1 MB/s 2024-04-25T10:17:04.258 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.259 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023806 s, 44.0 MB/s 2024-04-25T10:17:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238873 s, 43.9 MB/s 2024-04-25T10:17:04.310 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.310 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.310 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192418 s, 54.5 MB/s 2024-04-25T10:17:04.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.337 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.338 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231694 s, 45.3 MB/s 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.361 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193391 s, 54.2 MB/s 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.393 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274283 s, 38.2 MB/s 2024-04-25T10:17:04.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.416 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200385 s, 52.3 MB/s 2024-04-25T10:17:04.442 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.443 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222165 s, 47.2 MB/s 2024-04-25T10:17:04.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.466 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213916 s, 49.0 MB/s 2024-04-25T10:17:04.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.492 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.492 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210361 s, 49.8 MB/s 2024-04-25T10:17:04.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.522 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.522 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262874 s, 39.9 MB/s 2024-04-25T10:17:04.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.552 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246385 s, 42.6 MB/s 2024-04-25T10:17:04.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.580 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.580 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230104 s, 45.6 MB/s 2024-04-25T10:17:04.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.608 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.608 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024112 s, 43.5 MB/s 2024-04-25T10:17:04.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.631 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194342 s, 54.0 MB/s 2024-04-25T10:17:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219966 s, 47.7 MB/s 2024-04-25T10:17:04.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.686 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.686 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251038 s, 41.8 MB/s 2024-04-25T10:17:04.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.714 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234907 s, 44.6 MB/s 2024-04-25T10:17:04.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.741 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.741 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228811 s, 45.8 MB/s 2024-04-25T10:17:04.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.765 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.765 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200239 s, 52.4 MB/s 2024-04-25T10:17:04.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.792 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.792 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237557 s, 44.1 MB/s 2024-04-25T10:17:04.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.819 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.819 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216065 s, 48.5 MB/s 2024-04-25T10:17:04.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268584 s, 39.0 MB/s 2024-04-25T10:17:04.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248072 s, 42.3 MB/s 2024-04-25T10:17:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197924 s, 53.0 MB/s 2024-04-25T10:17:04.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:04.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:04.931 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245149 s, 42.8 MB/s 2024-04-25T10:17:04.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:04.933 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:04.934 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:04.934 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.9 2024-04-25T10:17:04.934 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:04.934 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.9/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:04.934 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:04.934 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:05.021 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.022 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226764 s, 46.2 MB/s 2024-04-25T10:17:05.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185723 s, 56.5 MB/s 2024-04-25T10:17:05.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.071 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246855 s, 42.5 MB/s 2024-04-25T10:17:05.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255797 s, 41.0 MB/s 2024-04-25T10:17:05.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.127 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219826 s, 47.7 MB/s 2024-04-25T10:17:05.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253328 s, 41.4 MB/s 2024-04-25T10:17:05.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.181 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206931 s, 50.7 MB/s 2024-04-25T10:17:05.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.209 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240321 s, 43.6 MB/s 2024-04-25T10:17:05.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.235 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223759 s, 46.9 MB/s 2024-04-25T10:17:05.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.266 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258352 s, 40.6 MB/s 2024-04-25T10:17:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207337 s, 50.6 MB/s 2024-04-25T10:17:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.318 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243059 s, 43.1 MB/s 2024-04-25T10:17:05.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.346 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.346 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242782 s, 43.2 MB/s 2024-04-25T10:17:05.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.373 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234763 s, 44.7 MB/s 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.399 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222118 s, 47.2 MB/s 2024-04-25T10:17:05.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.427 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.427 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234107 s, 44.8 MB/s 2024-04-25T10:17:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196974 s, 53.2 MB/s 2024-04-25T10:17:05.478 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.478 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.478 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240347 s, 43.6 MB/s 2024-04-25T10:17:05.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.505 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.505 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225116 s, 46.6 MB/s 2024-04-25T10:17:05.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.534 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.534 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246873 s, 42.5 MB/s 2024-04-25T10:17:05.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.558 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.558 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202606 s, 51.8 MB/s 2024-04-25T10:17:05.583 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.584 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.584 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217816 s, 48.1 MB/s 2024-04-25T10:17:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.614 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260266 s, 40.3 MB/s 2024-04-25T10:17:05.641 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.641 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.641 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232409 s, 45.1 MB/s 2024-04-25T10:17:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202478 s, 51.8 MB/s 2024-04-25T10:17:05.687 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.688 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019519 s, 53.7 MB/s 2024-04-25T10:17:05.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.713 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.713 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215774 s, 48.6 MB/s 2024-04-25T10:17:05.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.737 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203943 s, 51.4 MB/s 2024-04-25T10:17:05.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028082 s, 37.3 MB/s 2024-04-25T10:17:05.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205113 s, 51.1 MB/s 2024-04-25T10:17:05.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269086 s, 39.0 MB/s 2024-04-25T10:17:05.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243211 s, 43.1 MB/s 2024-04-25T10:17:05.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.10 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.10/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:05.856 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:05.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.944 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.944 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239358 s, 43.8 MB/s 2024-04-25T10:17:05.972 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:05.973 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:05.973 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241464 s, 43.4 MB/s 2024-04-25T10:17:06.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.002 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256963 s, 40.8 MB/s 2024-04-25T10:17:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.028 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219868 s, 47.7 MB/s 2024-04-25T10:17:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199492 s, 52.6 MB/s 2024-04-25T10:17:06.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.073 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.073 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0183036 s, 57.3 MB/s 2024-04-25T10:17:06.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.100 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.100 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236396 s, 44.4 MB/s 2024-04-25T10:17:06.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.125 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.125 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204631 s, 51.2 MB/s 2024-04-25T10:17:06.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.153 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.153 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235382 s, 44.5 MB/s 2024-04-25T10:17:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227916 s, 46.0 MB/s 2024-04-25T10:17:06.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216048 s, 48.5 MB/s 2024-04-25T10:17:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191716 s, 54.7 MB/s 2024-04-25T10:17:06.253 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.254 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.254 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237162 s, 44.2 MB/s 2024-04-25T10:17:06.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.274 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0170615 s, 61.5 MB/s 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.307 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0292937 s, 35.8 MB/s 2024-04-25T10:17:06.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.331 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.331 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201245 s, 52.1 MB/s 2024-04-25T10:17:06.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023076 s, 45.4 MB/s 2024-04-25T10:17:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253841 s, 41.3 MB/s 2024-04-25T10:17:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0273433 s, 38.3 MB/s 2024-04-25T10:17:06.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230901 s, 45.4 MB/s 2024-04-25T10:17:06.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.472 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.472 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209204 s, 50.1 MB/s 2024-04-25T10:17:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.500 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232325 s, 45.1 MB/s 2024-04-25T10:17:06.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.525 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.525 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020977 s, 50.0 MB/s 2024-04-25T10:17:06.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.551 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.551 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223563 s, 46.9 MB/s 2024-04-25T10:17:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201189 s, 52.1 MB/s 2024-04-25T10:17:06.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.605 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.605 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249194 s, 42.1 MB/s 2024-04-25T10:17:06.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203011 s, 51.7 MB/s 2024-04-25T10:17:06.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.655 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.655 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223597 s, 46.9 MB/s 2024-04-25T10:17:06.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.680 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.680 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210234 s, 49.9 MB/s 2024-04-25T10:17:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218457 s, 48.0 MB/s 2024-04-25T10:17:06.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.729 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.729 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194304 s, 54.0 MB/s 2024-04-25T10:17:06.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024205 s, 43.3 MB/s 2024-04-25T10:17:06.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.11 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.11/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:06.760 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:06.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.844 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.844 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233349 s, 44.9 MB/s 2024-04-25T10:17:06.873 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.873 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.873 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249163 s, 42.1 MB/s 2024-04-25T10:17:06.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252849 s, 41.5 MB/s 2024-04-25T10:17:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211645 s, 49.5 MB/s 2024-04-25T10:17:06.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210086 s, 49.9 MB/s 2024-04-25T10:17:06.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:06.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:06.978 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235115 s, 44.6 MB/s 2024-04-25T10:17:07.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.004 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223409 s, 46.9 MB/s 2024-04-25T10:17:07.028 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.028 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.028 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202915 s, 51.7 MB/s 2024-04-25T10:17:07.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.051 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.051 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201336 s, 52.1 MB/s 2024-04-25T10:17:07.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.075 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202921 s, 51.7 MB/s 2024-04-25T10:17:07.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.102 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229594 s, 45.7 MB/s 2024-04-25T10:17:07.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.130 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023944 s, 43.8 MB/s 2024-04-25T10:17:07.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.159 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.159 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244717 s, 42.8 MB/s 2024-04-25T10:17:07.184 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209756 s, 50.0 MB/s 2024-04-25T10:17:07.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.209 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.209 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209387 s, 50.1 MB/s 2024-04-25T10:17:07.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.235 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224276 s, 46.8 MB/s 2024-04-25T10:17:07.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253961 s, 41.3 MB/s 2024-04-25T10:17:07.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.288 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.288 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196058 s, 53.5 MB/s 2024-04-25T10:17:07.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.315 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232343 s, 45.1 MB/s 2024-04-25T10:17:07.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.338 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.339 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199011 s, 52.7 MB/s 2024-04-25T10:17:07.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.361 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.362 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193621 s, 54.2 MB/s 2024-04-25T10:17:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191442 s, 54.8 MB/s 2024-04-25T10:17:07.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.409 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.409 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206516 s, 50.8 MB/s 2024-04-25T10:17:07.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.432 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.432 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195424 s, 53.7 MB/s 2024-04-25T10:17:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.462 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.462 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257959 s, 40.6 MB/s 2024-04-25T10:17:07.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.488 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225224 s, 46.6 MB/s 2024-04-25T10:17:07.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.515 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.515 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023659 s, 44.3 MB/s 2024-04-25T10:17:07.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.540 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.540 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020435 s, 51.3 MB/s 2024-04-25T10:17:07.569 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.569 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.569 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249531 s, 42.0 MB/s 2024-04-25T10:17:07.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.594 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.594 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215594 s, 48.6 MB/s 2024-04-25T10:17:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.617 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193841 s, 54.1 MB/s 2024-04-25T10:17:07.643 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.644 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.644 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226352 s, 46.3 MB/s 2024-04-25T10:17:07.646 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:07.646 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:07.647 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:07.647 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.12 2024-04-25T10:17:07.647 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:07.647 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.12/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:07.647 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:07.647 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:07.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209814 s, 50.0 MB/s 2024-04-25T10:17:07.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215417 s, 48.7 MB/s 2024-04-25T10:17:07.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.777 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.777 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229077 s, 45.8 MB/s 2024-04-25T10:17:07.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.799 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185253 s, 56.6 MB/s 2024-04-25T10:17:07.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.824 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.824 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207171 s, 50.6 MB/s 2024-04-25T10:17:07.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.851 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.851 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232315 s, 45.1 MB/s 2024-04-25T10:17:07.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212016 s, 49.5 MB/s 2024-04-25T10:17:07.906 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259907 s, 40.3 MB/s 2024-04-25T10:17:07.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.932 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204711 s, 51.2 MB/s 2024-04-25T10:17:07.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.959 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222745 s, 47.1 MB/s 2024-04-25T10:17:07.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:07.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:07.983 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213271 s, 49.2 MB/s 2024-04-25T10:17:08.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.011 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.011 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021686 s, 48.4 MB/s 2024-04-25T10:17:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245426 s, 42.7 MB/s 2024-04-25T10:17:08.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.063 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.063 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021048 s, 49.8 MB/s 2024-04-25T10:17:08.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.092 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245982 s, 42.6 MB/s 2024-04-25T10:17:08.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200497 s, 52.3 MB/s 2024-04-25T10:17:08.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230048 s, 45.6 MB/s 2024-04-25T10:17:08.164 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191056 s, 54.9 MB/s 2024-04-25T10:17:08.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221666 s, 47.3 MB/s 2024-04-25T10:17:08.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.214 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.215 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189243 s, 55.4 MB/s 2024-04-25T10:17:08.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.238 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194414 s, 53.9 MB/s 2024-04-25T10:17:08.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.266 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238275 s, 44.0 MB/s 2024-04-25T10:17:08.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.295 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240417 s, 43.6 MB/s 2024-04-25T10:17:08.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257126 s, 40.8 MB/s 2024-04-25T10:17:08.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.351 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021808 s, 48.1 MB/s 2024-04-25T10:17:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.382 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251409 s, 41.7 MB/s 2024-04-25T10:17:08.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201742 s, 52.0 MB/s 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.434 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235481 s, 44.5 MB/s 2024-04-25T10:17:08.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219056 s, 47.9 MB/s 2024-04-25T10:17:08.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220412 s, 47.6 MB/s 2024-04-25T10:17:08.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.512 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224052 s, 46.8 MB/s 2024-04-25T10:17:08.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.538 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.538 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216519 s, 48.4 MB/s 2024-04-25T10:17:08.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.13 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.13/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:08.540 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:08.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.625 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.625 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223526 s, 46.9 MB/s 2024-04-25T10:17:08.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217022 s, 48.3 MB/s 2024-04-25T10:17:08.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.677 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.677 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232868 s, 45.0 MB/s 2024-04-25T10:17:08.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.703 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.703 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222452 s, 47.1 MB/s 2024-04-25T10:17:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.726 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019414 s, 54.0 MB/s 2024-04-25T10:17:08.750 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.751 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.751 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019978 s, 52.5 MB/s 2024-04-25T10:17:08.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.775 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203201 s, 51.6 MB/s 2024-04-25T10:17:08.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277234 s, 37.8 MB/s 2024-04-25T10:17:08.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203925 s, 51.4 MB/s 2024-04-25T10:17:08.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222161 s, 47.2 MB/s 2024-04-25T10:17:08.880 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.881 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0184749 s, 56.8 MB/s 2024-04-25T10:17:08.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182751 s, 57.4 MB/s 2024-04-25T10:17:08.926 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02083 s, 50.3 MB/s 2024-04-25T10:17:08.950 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.951 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.951 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019889 s, 52.7 MB/s 2024-04-25T10:17:08.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:08.978 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237719 s, 44.1 MB/s 2024-04-25T10:17:09.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.004 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215643 s, 48.6 MB/s 2024-04-25T10:17:09.028 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.028 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.028 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201137 s, 52.1 MB/s 2024-04-25T10:17:09.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.053 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208059 s, 50.4 MB/s 2024-04-25T10:17:09.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.080 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.080 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233423 s, 44.9 MB/s 2024-04-25T10:17:09.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.107 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.107 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237637 s, 44.1 MB/s 2024-04-25T10:17:09.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0280698 s, 37.4 MB/s 2024-04-25T10:17:09.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.167 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021219 s, 49.4 MB/s 2024-04-25T10:17:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.195 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245474 s, 42.7 MB/s 2024-04-25T10:17:09.223 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.224 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.224 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245895 s, 42.6 MB/s 2024-04-25T10:17:09.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186345 s, 56.3 MB/s 2024-04-25T10:17:09.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230633 s, 45.5 MB/s 2024-04-25T10:17:09.296 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.297 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201254 s, 52.1 MB/s 2024-04-25T10:17:09.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.321 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211974 s, 49.5 MB/s 2024-04-25T10:17:09.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.347 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.347 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222018 s, 47.2 MB/s 2024-04-25T10:17:09.372 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.373 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.373 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214735 s, 48.8 MB/s 2024-04-25T10:17:09.398 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.399 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.399 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220195 s, 47.6 MB/s 2024-04-25T10:17:09.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.431 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.431 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0267368 s, 39.2 MB/s 2024-04-25T10:17:09.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:09.433 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:09.433 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:09.433 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.14 2024-04-25T10:17:09.433 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:09.434 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.14/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:09.434 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:09.434 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:09.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.518 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.518 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02154 s, 48.7 MB/s 2024-04-25T10:17:09.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.547 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248346 s, 42.2 MB/s 2024-04-25T10:17:09.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.568 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.568 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0177314 s, 59.1 MB/s 2024-04-25T10:17:09.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024681 s, 42.5 MB/s 2024-04-25T10:17:09.625 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.625 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.625 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240234 s, 43.6 MB/s 2024-04-25T10:17:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224477 s, 46.7 MB/s 2024-04-25T10:17:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242601 s, 43.2 MB/s 2024-04-25T10:17:09.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228538 s, 45.9 MB/s 2024-04-25T10:17:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199013 s, 52.7 MB/s 2024-04-25T10:17:09.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.755 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212948 s, 49.2 MB/s 2024-04-25T10:17:09.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195115 s, 53.7 MB/s 2024-04-25T10:17:09.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.802 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199068 s, 52.7 MB/s 2024-04-25T10:17:09.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.827 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199737 s, 52.5 MB/s 2024-04-25T10:17:09.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.853 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.853 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231825 s, 45.2 MB/s 2024-04-25T10:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232006 s, 45.2 MB/s 2024-04-25T10:17:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199035 s, 52.7 MB/s 2024-04-25T10:17:09.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.929 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.929 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212695 s, 49.3 MB/s 2024-04-25T10:17:09.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.958 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024447 s, 42.9 MB/s 2024-04-25T10:17:09.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:09.981 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:09.981 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019698 s, 53.2 MB/s 2024-04-25T10:17:10.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.010 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.010 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245808 s, 42.7 MB/s 2024-04-25T10:17:10.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.037 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.037 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230236 s, 45.5 MB/s 2024-04-25T10:17:10.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.065 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.066 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245275 s, 42.8 MB/s 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.089 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194349 s, 54.0 MB/s 2024-04-25T10:17:10.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235203 s, 44.6 MB/s 2024-04-25T10:17:10.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.142 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227111 s, 46.2 MB/s 2024-04-25T10:17:10.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.169 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.169 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224753 s, 46.7 MB/s 2024-04-25T10:17:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.199 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262265 s, 40.0 MB/s 2024-04-25T10:17:10.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.226 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023328 s, 44.9 MB/s 2024-04-25T10:17:10.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.246 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168087 s, 62.4 MB/s 2024-04-25T10:17:10.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244959 s, 42.8 MB/s 2024-04-25T10:17:10.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.301 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.301 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214461 s, 48.9 MB/s 2024-04-25T10:17:10.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.327 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219773 s, 47.7 MB/s 2024-04-25T10:17:10.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.15 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.15/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:10.330 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.414 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0251925 s, 41.6 MB/s 2024-04-25T10:17:10.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.437 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.437 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194664 s, 53.9 MB/s 2024-04-25T10:17:10.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.464 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243575 s, 43.0 MB/s 2024-04-25T10:17:10.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.489 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.489 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207665 s, 50.5 MB/s 2024-04-25T10:17:10.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.517 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.517 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024428 s, 42.9 MB/s 2024-04-25T10:17:10.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.545 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217197 s, 48.3 MB/s 2024-04-25T10:17:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.570 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206389 s, 50.8 MB/s 2024-04-25T10:17:10.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220516 s, 47.6 MB/s 2024-04-25T10:17:10.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.624 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.624 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252204 s, 41.6 MB/s 2024-04-25T10:17:10.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.657 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.657 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284077 s, 36.9 MB/s 2024-04-25T10:17:10.682 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.683 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.683 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217673 s, 48.2 MB/s 2024-04-25T10:17:10.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.704 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.704 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185785 s, 56.4 MB/s 2024-04-25T10:17:10.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218996 s, 47.9 MB/s 2024-04-25T10:17:10.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.755 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.755 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213792 s, 49.0 MB/s 2024-04-25T10:17:10.779 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.780 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206805 s, 50.7 MB/s 2024-04-25T10:17:10.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241146 s, 43.5 MB/s 2024-04-25T10:17:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219067 s, 47.9 MB/s 2024-04-25T10:17:10.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.861 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231562 s, 45.3 MB/s 2024-04-25T10:17:10.885 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.886 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.886 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213787 s, 49.0 MB/s 2024-04-25T10:17:10.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.912 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.912 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216299 s, 48.5 MB/s 2024-04-25T10:17:10.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.940 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.940 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235987 s, 44.4 MB/s 2024-04-25T10:17:10.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:10.966 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:10.966 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215826 s, 48.6 MB/s 2024-04-25T10:17:11.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.001 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0312783 s, 33.5 MB/s 2024-04-25T10:17:11.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.026 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.026 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214981 s, 48.8 MB/s 2024-04-25T10:17:11.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.059 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.059 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289133 s, 36.3 MB/s 2024-04-25T10:17:11.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.085 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.085 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021874 s, 47.9 MB/s 2024-04-25T10:17:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.110 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212659 s, 49.3 MB/s 2024-04-25T10:17:11.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.137 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.137 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227891 s, 46.0 MB/s 2024-04-25T10:17:11.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.168 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.168 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266485 s, 39.3 MB/s 2024-04-25T10:17:11.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.192 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.192 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019633 s, 53.4 MB/s 2024-04-25T10:17:11.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.222 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.222 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260155 s, 40.3 MB/s 2024-04-25T10:17:11.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.247 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208525 s, 50.3 MB/s 2024-04-25T10:17:11.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.16 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.16/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:11.249 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:11.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191048 s, 54.9 MB/s 2024-04-25T10:17:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220066 s, 47.6 MB/s 2024-04-25T10:17:11.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.379 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023851 s, 44.0 MB/s 2024-04-25T10:17:11.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231446 s, 45.3 MB/s 2024-04-25T10:17:11.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.433 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233937 s, 44.8 MB/s 2024-04-25T10:17:11.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220299 s, 47.6 MB/s 2024-04-25T10:17:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.486 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229254 s, 45.7 MB/s 2024-04-25T10:17:11.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206831 s, 50.7 MB/s 2024-04-25T10:17:11.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215958 s, 48.6 MB/s 2024-04-25T10:17:11.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.560 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.560 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208285 s, 50.3 MB/s 2024-04-25T10:17:11.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.589 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249732 s, 42.0 MB/s 2024-04-25T10:17:11.615 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.615 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.615 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223978 s, 46.8 MB/s 2024-04-25T10:17:11.639 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.639 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.639 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204966 s, 51.2 MB/s 2024-04-25T10:17:11.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.665 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210254 s, 49.9 MB/s 2024-04-25T10:17:11.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.692 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.692 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226865 s, 46.2 MB/s 2024-04-25T10:17:11.717 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.717 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.717 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215619 s, 48.6 MB/s 2024-04-25T10:17:11.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.744 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214376 s, 48.9 MB/s 2024-04-25T10:17:11.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220912 s, 47.5 MB/s 2024-04-25T10:17:11.796 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.797 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.797 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227114 s, 46.2 MB/s 2024-04-25T10:17:11.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.821 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.821 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217784 s, 48.1 MB/s 2024-04-25T10:17:11.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241648 s, 43.4 MB/s 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219992 s, 47.7 MB/s 2024-04-25T10:17:11.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.902 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.902 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215901 s, 48.6 MB/s 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.927 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207786 s, 50.5 MB/s 2024-04-25T10:17:11.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.958 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.958 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0268785 s, 39.0 MB/s 2024-04-25T10:17:11.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:11.985 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:11.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230292 s, 45.5 MB/s 2024-04-25T10:17:12.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241419 s, 43.4 MB/s 2024-04-25T10:17:12.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.041 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221216 s, 47.4 MB/s 2024-04-25T10:17:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.066 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214867 s, 48.8 MB/s 2024-04-25T10:17:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247484 s, 42.4 MB/s 2024-04-25T10:17:12.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222783 s, 47.1 MB/s 2024-04-25T10:17:12.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.026909 s, 39.0 MB/s 2024-04-25T10:17:12.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.17 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.17/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:12.155 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:12.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.225 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197579 s, 53.1 MB/s 2024-04-25T10:17:12.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230271 s, 45.5 MB/s 2024-04-25T10:17:12.276 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.276 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.276 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214649 s, 48.9 MB/s 2024-04-25T10:17:12.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.302 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218216 s, 48.1 MB/s 2024-04-25T10:17:12.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.324 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.324 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195527 s, 53.6 MB/s 2024-04-25T10:17:12.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.348 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.348 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193251 s, 54.3 MB/s 2024-04-25T10:17:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023217 s, 45.2 MB/s 2024-04-25T10:17:12.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.403 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023959 s, 43.8 MB/s 2024-04-25T10:17:12.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.428 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.428 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206987 s, 50.7 MB/s 2024-04-25T10:17:12.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.458 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.458 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262597 s, 39.9 MB/s 2024-04-25T10:17:12.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.488 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261392 s, 40.1 MB/s 2024-04-25T10:17:12.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199773 s, 52.5 MB/s 2024-04-25T10:17:12.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208365 s, 50.3 MB/s 2024-04-25T10:17:12.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.563 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.563 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232744 s, 45.1 MB/s 2024-04-25T10:17:12.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.590 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233622 s, 44.9 MB/s 2024-04-25T10:17:12.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.617 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.617 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229274 s, 45.7 MB/s 2024-04-25T10:17:12.644 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.644 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.644 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234918 s, 44.6 MB/s 2024-04-25T10:17:12.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.673 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.673 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246076 s, 42.6 MB/s 2024-04-25T10:17:12.698 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.699 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214491 s, 48.9 MB/s 2024-04-25T10:17:12.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.722 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.722 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019892 s, 52.7 MB/s 2024-04-25T10:17:12.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.748 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222041 s, 47.2 MB/s 2024-04-25T10:17:12.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.780 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.780 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0282738 s, 37.1 MB/s 2024-04-25T10:17:12.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.807 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.807 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223465 s, 46.9 MB/s 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.831 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207767 s, 50.5 MB/s 2024-04-25T10:17:12.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.855 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.855 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202278 s, 51.8 MB/s 2024-04-25T10:17:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.879 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198303 s, 52.9 MB/s 2024-04-25T10:17:12.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.909 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.909 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262549 s, 39.9 MB/s 2024-04-25T10:17:12.931 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.932 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019113 s, 54.9 MB/s 2024-04-25T10:17:12.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.953 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.953 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187535 s, 55.9 MB/s 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:12.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:12.978 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200063 s, 52.4 MB/s 2024-04-25T10:17:13.004 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.005 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229554 s, 45.7 MB/s 2024-04-25T10:17:13.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.041 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0304519 s, 34.4 MB/s 2024-04-25T10:17:13.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:13.042 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:13.042 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:13.042 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.18 2024-04-25T10:17:13.042 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:13.043 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.18/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:13.043 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:13.043 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:13.122 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.123 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.123 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208223 s, 50.4 MB/s 2024-04-25T10:17:13.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.144 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182199 s, 57.6 MB/s 2024-04-25T10:17:13.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.175 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.175 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257502 s, 40.7 MB/s 2024-04-25T10:17:13.200 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.200 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.200 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215469 s, 48.7 MB/s 2024-04-25T10:17:13.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0272015 s, 38.5 MB/s 2024-04-25T10:17:13.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.257 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.257 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022629 s, 46.3 MB/s 2024-04-25T10:17:13.285 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.286 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245603 s, 42.7 MB/s 2024-04-25T10:17:13.313 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236878 s, 44.3 MB/s 2024-04-25T10:17:13.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.341 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.341 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233249 s, 45.0 MB/s 2024-04-25T10:17:13.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.367 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215916 s, 48.6 MB/s 2024-04-25T10:17:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0269746 s, 38.9 MB/s 2024-04-25T10:17:13.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.423 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.423 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216098 s, 48.5 MB/s 2024-04-25T10:17:13.449 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.450 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.450 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022097 s, 47.5 MB/s 2024-04-25T10:17:13.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.473 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.473 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212091 s, 49.4 MB/s 2024-04-25T10:17:13.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0259934 s, 40.3 MB/s 2024-04-25T10:17:13.529 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.530 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.530 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223597 s, 46.9 MB/s 2024-04-25T10:17:13.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.556 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.556 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221794 s, 47.3 MB/s 2024-04-25T10:17:13.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.581 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.582 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021408 s, 49.0 MB/s 2024-04-25T10:17:13.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.607 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.607 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221316 s, 47.4 MB/s 2024-04-25T10:17:13.633 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.633 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.633 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021802 s, 48.1 MB/s 2024-04-25T10:17:13.662 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.663 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.663 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257529 s, 40.7 MB/s 2024-04-25T10:17:13.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.688 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.688 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217171 s, 48.3 MB/s 2024-04-25T10:17:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197468 s, 53.1 MB/s 2024-04-25T10:17:13.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.740 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.740 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248881 s, 42.1 MB/s 2024-04-25T10:17:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.770 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261607 s, 40.1 MB/s 2024-04-25T10:17:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208743 s, 50.2 MB/s 2024-04-25T10:17:13.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.818 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.819 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203386 s, 51.6 MB/s 2024-04-25T10:17:13.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.842 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.842 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0196858 s, 53.3 MB/s 2024-04-25T10:17:13.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.868 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.868 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217731 s, 48.2 MB/s 2024-04-25T10:17:13.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257144 s, 40.8 MB/s 2024-04-25T10:17:13.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232772 s, 45.0 MB/s 2024-04-25T10:17:13.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:13.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:13.950 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021358 s, 49.1 MB/s 2024-04-25T10:17:13.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.19 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.19/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:13.952 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:14.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271325 s, 38.6 MB/s 2024-04-25T10:17:14.068 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.069 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221432 s, 47.4 MB/s 2024-04-25T10:17:14.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205842 s, 50.9 MB/s 2024-04-25T10:17:14.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.115 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.115 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192627 s, 54.4 MB/s 2024-04-25T10:17:14.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233111 s, 45.0 MB/s 2024-04-25T10:17:14.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252696 s, 41.5 MB/s 2024-04-25T10:17:14.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.196 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.196 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205298 s, 51.1 MB/s 2024-04-25T10:17:14.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.219 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0189655 s, 55.3 MB/s 2024-04-25T10:17:14.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219689 s, 47.7 MB/s 2024-04-25T10:17:14.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.273 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.273 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237352 s, 44.2 MB/s 2024-04-25T10:17:14.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.299 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.300 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022819 s, 46.0 MB/s 2024-04-25T10:17:14.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.327 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.327 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024262 s, 43.2 MB/s 2024-04-25T10:17:14.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.355 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.355 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239034 s, 43.9 MB/s 2024-04-25T10:17:14.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257235 s, 40.8 MB/s 2024-04-25T10:17:14.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.415 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255726 s, 41.0 MB/s 2024-04-25T10:17:14.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.439 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.439 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202319 s, 51.8 MB/s 2024-04-25T10:17:14.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.467 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238048 s, 44.0 MB/s 2024-04-25T10:17:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222796 s, 47.1 MB/s 2024-04-25T10:17:14.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.519 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216946 s, 48.3 MB/s 2024-04-25T10:17:14.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226166 s, 46.4 MB/s 2024-04-25T10:17:14.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.576 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.576 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0255303 s, 41.1 MB/s 2024-04-25T10:17:14.599 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.600 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197739 s, 53.0 MB/s 2024-04-25T10:17:14.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.629 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.629 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234604 s, 44.7 MB/s 2024-04-25T10:17:14.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191481 s, 54.8 MB/s 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.678 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220256 s, 47.6 MB/s 2024-04-25T10:17:14.705 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.706 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024551 s, 42.7 MB/s 2024-04-25T10:17:14.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.733 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.733 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238999 s, 43.9 MB/s 2024-04-25T10:17:14.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.759 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.759 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209857 s, 50.0 MB/s 2024-04-25T10:17:14.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.786 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.786 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229713 s, 45.6 MB/s 2024-04-25T10:17:14.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197812 s, 53.0 MB/s 2024-04-25T10:17:14.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.834 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.834 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206153 s, 50.9 MB/s 2024-04-25T10:17:14.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.859 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201137 s, 52.1 MB/s 2024-04-25T10:17:14.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:14.861 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:14.861 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:14.861 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.20 2024-04-25T10:17:14.861 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:14.861 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.20/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:14.861 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:14.862 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:14.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.949 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.949 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277693 s, 37.8 MB/s 2024-04-25T10:17:14.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:14.977 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:14.977 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234879 s, 44.6 MB/s 2024-04-25T10:17:15.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.005 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.005 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243106 s, 43.1 MB/s 2024-04-25T10:17:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.033 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250564 s, 41.8 MB/s 2024-04-25T10:17:15.057 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.057 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.057 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020095 s, 52.2 MB/s 2024-04-25T10:17:15.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.079 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.079 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182437 s, 57.5 MB/s 2024-04-25T10:17:15.106 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.106 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.106 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223743 s, 46.9 MB/s 2024-04-25T10:17:15.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.131 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.131 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198844 s, 52.7 MB/s 2024-04-25T10:17:15.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.158 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.158 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235565 s, 44.5 MB/s 2024-04-25T10:17:15.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.182 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.182 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198482 s, 52.8 MB/s 2024-04-25T10:17:15.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.208 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.208 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218966 s, 47.9 MB/s 2024-04-25T10:17:15.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.233 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.233 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213275 s, 49.2 MB/s 2024-04-25T10:17:15.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.262 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257575 s, 40.7 MB/s 2024-04-25T10:17:15.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.287 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.287 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208411 s, 50.3 MB/s 2024-04-25T10:17:15.316 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.316 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.316 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252106 s, 41.6 MB/s 2024-04-25T10:17:15.344 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.344 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.344 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231372 s, 45.3 MB/s 2024-04-25T10:17:15.366 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.367 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0199876 s, 52.5 MB/s 2024-04-25T10:17:15.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.395 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.395 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252789 s, 41.5 MB/s 2024-04-25T10:17:15.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.422 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.422 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220736 s, 47.5 MB/s 2024-04-25T10:17:15.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.447 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.447 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211368 s, 49.6 MB/s 2024-04-25T10:17:15.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.475 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.475 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235636 s, 44.5 MB/s 2024-04-25T10:17:15.503 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.504 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.504 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248435 s, 42.2 MB/s 2024-04-25T10:17:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.528 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202757 s, 51.7 MB/s 2024-04-25T10:17:15.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.559 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.559 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0276578 s, 37.9 MB/s 2024-04-25T10:17:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217631 s, 48.2 MB/s 2024-04-25T10:17:15.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.610 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.610 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197421 s, 53.1 MB/s 2024-04-25T10:17:15.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.635 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.635 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210892 s, 49.7 MB/s 2024-04-25T10:17:15.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.665 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.665 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0263784 s, 39.8 MB/s 2024-04-25T10:17:15.690 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.690 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.690 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203083 s, 51.6 MB/s 2024-04-25T10:17:15.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.714 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.714 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206821 s, 50.7 MB/s 2024-04-25T10:17:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0190493 s, 55.0 MB/s 2024-04-25T10:17:15.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.763 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.763 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215102 s, 48.7 MB/s 2024-04-25T10:17:15.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.21 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.21/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:15.766 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:15.849 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.849 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.849 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220986 s, 47.4 MB/s 2024-04-25T10:17:15.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.874 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220035 s, 47.7 MB/s 2024-04-25T10:17:15.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.898 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.898 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211229 s, 49.6 MB/s 2024-04-25T10:17:15.922 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.923 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208773 s, 50.2 MB/s 2024-04-25T10:17:15.946 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.947 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197929 s, 53.0 MB/s 2024-04-25T10:17:15.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:15.978 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:15.978 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0258046 s, 40.6 MB/s 2024-04-25T10:17:16.001 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.002 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.002 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197514 s, 53.1 MB/s 2024-04-25T10:17:16.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.032 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.032 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0264924 s, 39.6 MB/s 2024-04-25T10:17:16.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.060 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.061 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237709 s, 44.1 MB/s 2024-04-25T10:17:16.088 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.089 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.089 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244441 s, 42.9 MB/s 2024-04-25T10:17:16.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.121 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0274722 s, 38.2 MB/s 2024-04-25T10:17:16.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.149 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.149 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234854 s, 44.6 MB/s 2024-04-25T10:17:16.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0191558 s, 54.7 MB/s 2024-04-25T10:17:16.197 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.197 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.197 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203915 s, 51.4 MB/s 2024-04-25T10:17:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.225 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240408 s, 43.6 MB/s 2024-04-25T10:17:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.251 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221159 s, 47.4 MB/s 2024-04-25T10:17:16.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.275 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.275 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020684 s, 50.7 MB/s 2024-04-25T10:17:16.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.302 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226257 s, 46.3 MB/s 2024-04-25T10:17:16.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206899 s, 50.7 MB/s 2024-04-25T10:17:16.350 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.351 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206058 s, 50.9 MB/s 2024-04-25T10:17:16.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.380 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.380 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252391 s, 41.5 MB/s 2024-04-25T10:17:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227037 s, 46.2 MB/s 2024-04-25T10:17:16.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.433 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210936 s, 49.7 MB/s 2024-04-25T10:17:16.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226154 s, 46.4 MB/s 2024-04-25T10:17:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.484 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210322 s, 49.9 MB/s 2024-04-25T10:17:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.512 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.513 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246974 s, 42.5 MB/s 2024-04-25T10:17:16.535 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0193822 s, 54.1 MB/s 2024-04-25T10:17:16.564 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.565 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250618 s, 41.8 MB/s 2024-04-25T10:17:16.592 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.592 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.592 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233764 s, 44.9 MB/s 2024-04-25T10:17:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250179 s, 41.9 MB/s 2024-04-25T10:17:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227925 s, 46.0 MB/s 2024-04-25T10:17:16.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.672 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209829 s, 50.0 MB/s 2024-04-25T10:17:16.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:16.674 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:16.674 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:16.675 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.22 2024-04-25T10:17:16.675 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:16.675 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.22/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:16.675 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:16.675 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213852 s, 49.0 MB/s 2024-04-25T10:17:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234689 s, 44.7 MB/s 2024-04-25T10:17:16.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207398 s, 50.6 MB/s 2024-04-25T10:17:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186231 s, 56.3 MB/s 2024-04-25T10:17:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.858 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218261 s, 48.0 MB/s 2024-04-25T10:17:16.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.884 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.884 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221597 s, 47.3 MB/s 2024-04-25T10:17:16.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.910 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.910 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219468 s, 47.8 MB/s 2024-04-25T10:17:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021479 s, 48.8 MB/s 2024-04-25T10:17:16.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.957 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.957 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182675 s, 57.4 MB/s 2024-04-25T10:17:16.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:16.983 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:16.983 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217781 s, 48.1 MB/s 2024-04-25T10:17:17.011 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.011 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.012 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247456 s, 42.4 MB/s 2024-04-25T10:17:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243692 s, 43.0 MB/s 2024-04-25T10:17:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260303 s, 40.3 MB/s 2024-04-25T10:17:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213977 s, 49.0 MB/s 2024-04-25T10:17:17.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.126 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.126 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257229 s, 40.8 MB/s 2024-04-25T10:17:17.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.156 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.156 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262158 s, 40.0 MB/s 2024-04-25T10:17:17.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.178 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.178 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186308 s, 56.3 MB/s 2024-04-25T10:17:17.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.201 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.201 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204291 s, 51.3 MB/s 2024-04-25T10:17:17.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.226 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.226 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212612 s, 49.3 MB/s 2024-04-25T10:17:17.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.247 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168106 s, 62.4 MB/s 2024-04-25T10:17:17.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.272 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.272 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227139 s, 46.2 MB/s 2024-04-25T10:17:17.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.297 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.297 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213216 s, 49.2 MB/s 2024-04-25T10:17:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246619 s, 42.5 MB/s 2024-04-25T10:17:17.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.352 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.352 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210046 s, 49.9 MB/s 2024-04-25T10:17:17.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.378 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.378 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02155 s, 48.7 MB/s 2024-04-25T10:17:17.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.403 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207151 s, 50.6 MB/s 2024-04-25T10:17:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235008 s, 44.6 MB/s 2024-04-25T10:17:17.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.456 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.456 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225375 s, 46.5 MB/s 2024-04-25T10:17:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.482 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223252 s, 47.0 MB/s 2024-04-25T10:17:17.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.510 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.510 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245047 s, 42.8 MB/s 2024-04-25T10:17:17.537 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.537 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.537 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227909 s, 46.0 MB/s 2024-04-25T10:17:17.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.562 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.562 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207573 s, 50.5 MB/s 2024-04-25T10:17:17.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.23 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.23/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:17.564 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:17.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.651 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.651 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250555 s, 41.9 MB/s 2024-04-25T10:17:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261674 s, 40.1 MB/s 2024-04-25T10:17:17.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219254 s, 47.8 MB/s 2024-04-25T10:17:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0179063 s, 58.6 MB/s 2024-04-25T10:17:17.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.754 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.754 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021946 s, 47.8 MB/s 2024-04-25T10:17:17.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.781 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237996 s, 44.1 MB/s 2024-04-25T10:17:17.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.808 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224454 s, 46.7 MB/s 2024-04-25T10:17:17.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.836 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230471 s, 45.5 MB/s 2024-04-25T10:17:17.863 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.863 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.863 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217629 s, 48.2 MB/s 2024-04-25T10:17:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236057 s, 44.4 MB/s 2024-04-25T10:17:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.915 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202734 s, 51.7 MB/s 2024-04-25T10:17:17.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.941 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216888 s, 48.3 MB/s 2024-04-25T10:17:17.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.967 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.967 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218376 s, 48.0 MB/s 2024-04-25T10:17:17.995 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:17.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:17.996 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242157 s, 43.3 MB/s 2024-04-25T10:17:18.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.022 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229793 s, 45.6 MB/s 2024-04-25T10:17:18.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.044 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.044 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0186837 s, 56.1 MB/s 2024-04-25T10:17:18.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.067 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195164 s, 53.7 MB/s 2024-04-25T10:17:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216009 s, 48.5 MB/s 2024-04-25T10:17:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.120 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239536 s, 43.8 MB/s 2024-04-25T10:17:18.139 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.140 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.140 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0161072 s, 65.1 MB/s 2024-04-25T10:17:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.166 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222697 s, 47.1 MB/s 2024-04-25T10:17:18.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.191 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.191 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203978 s, 51.4 MB/s 2024-04-25T10:17:18.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.220 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.220 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024569 s, 42.7 MB/s 2024-04-25T10:17:18.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.245 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215713 s, 48.6 MB/s 2024-04-25T10:17:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247369 s, 42.4 MB/s 2024-04-25T10:17:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223175 s, 47.0 MB/s 2024-04-25T10:17:18.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.329 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.329 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0250908 s, 41.8 MB/s 2024-04-25T10:17:18.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.358 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.358 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0253977 s, 41.3 MB/s 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.381 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0187466 s, 55.9 MB/s 2024-04-25T10:17:18.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.405 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.405 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202619 s, 51.8 MB/s 2024-04-25T10:17:18.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.430 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.431 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021665 s, 48.4 MB/s 2024-04-25T10:17:18.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.459 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.459 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249195 s, 42.1 MB/s 2024-04-25T10:17:18.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:18.462 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:18.462 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:18.462 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.24 2024-04-25T10:17:18.463 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:18.463 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.24/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:18.463 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:18.463 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204911 s, 51.2 MB/s 2024-04-25T10:17:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022823 s, 45.9 MB/s 2024-04-25T10:17:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221326 s, 47.4 MB/s 2024-04-25T10:17:18.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.620 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197075 s, 53.2 MB/s 2024-04-25T10:17:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.646 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022111 s, 47.4 MB/s 2024-04-25T10:17:18.674 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.675 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247061 s, 42.4 MB/s 2024-04-25T10:17:18.699 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212606 s, 49.3 MB/s 2024-04-25T10:17:18.727 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243707 s, 43.0 MB/s 2024-04-25T10:17:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249227 s, 42.1 MB/s 2024-04-25T10:17:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221567 s, 47.3 MB/s 2024-04-25T10:17:18.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240422 s, 43.6 MB/s 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227586 s, 46.1 MB/s 2024-04-25T10:17:18.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.862 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.863 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218435 s, 48.0 MB/s 2024-04-25T10:17:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212318 s, 49.4 MB/s 2024-04-25T10:17:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233725 s, 44.9 MB/s 2024-04-25T10:17:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230874 s, 45.4 MB/s 2024-04-25T10:17:18.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02387 s, 43.9 MB/s 2024-04-25T10:17:18.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:18.996 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:18.996 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230611 s, 45.5 MB/s 2024-04-25T10:17:19.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.022 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.022 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222755 s, 47.1 MB/s 2024-04-25T10:17:19.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.050 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.050 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230577 s, 45.5 MB/s 2024-04-25T10:17:19.074 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.075 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.075 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208403 s, 50.3 MB/s 2024-04-25T10:17:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236415 s, 44.4 MB/s 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.130 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240422 s, 43.6 MB/s 2024-04-25T10:17:19.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.157 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.157 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231521 s, 45.3 MB/s 2024-04-25T10:17:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205243 s, 51.1 MB/s 2024-04-25T10:17:19.207 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.207 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.207 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212367 s, 49.4 MB/s 2024-04-25T10:17:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0270563 s, 38.8 MB/s 2024-04-25T10:17:19.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.267 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.267 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246151 s, 42.6 MB/s 2024-04-25T10:17:19.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.295 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.295 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237523 s, 44.1 MB/s 2024-04-25T10:17:19.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.323 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.323 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237209 s, 44.2 MB/s 2024-04-25T10:17:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244844 s, 42.8 MB/s 2024-04-25T10:17:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224824 s, 46.6 MB/s 2024-04-25T10:17:19.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:19.380 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:19.380 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:19.380 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.25 2024-04-25T10:17:19.380 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:19.380 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.25/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:19.381 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:19.381 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:19.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.465 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223612 s, 46.9 MB/s 2024-04-25T10:17:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260251 s, 40.3 MB/s 2024-04-25T10:17:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221733 s, 47.3 MB/s 2024-04-25T10:17:19.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.545 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021183 s, 49.5 MB/s 2024-04-25T10:17:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224104 s, 46.8 MB/s 2024-04-25T10:17:19.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202688 s, 51.7 MB/s 2024-04-25T10:17:19.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.621 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.621 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210103 s, 49.9 MB/s 2024-04-25T10:17:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218499 s, 48.0 MB/s 2024-04-25T10:17:19.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.670 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.671 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205843 s, 50.9 MB/s 2024-04-25T10:17:19.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.695 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.695 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206358 s, 50.8 MB/s 2024-04-25T10:17:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.721 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.721 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.021554 s, 48.6 MB/s 2024-04-25T10:17:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.747 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220413 s, 47.6 MB/s 2024-04-25T10:17:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200824 s, 52.2 MB/s 2024-04-25T10:17:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195041 s, 53.8 MB/s 2024-04-25T10:17:19.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.823 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.823 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248418 s, 42.2 MB/s 2024-04-25T10:17:19.848 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.849 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.849 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219246 s, 47.8 MB/s 2024-04-25T10:17:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210694 s, 49.8 MB/s 2024-04-25T10:17:19.904 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.905 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.905 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257244 s, 40.8 MB/s 2024-04-25T10:17:19.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.932 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.932 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234722 s, 44.7 MB/s 2024-04-25T10:17:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241867 s, 43.4 MB/s 2024-04-25T10:17:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204811 s, 51.2 MB/s 2024-04-25T10:17:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224613 s, 46.7 MB/s 2024-04-25T10:17:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256883 s, 40.8 MB/s 2024-04-25T10:17:20.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.066 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203135 s, 51.6 MB/s 2024-04-25T10:17:20.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.091 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.091 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210896 s, 49.7 MB/s 2024-04-25T10:17:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.116 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213881 s, 49.0 MB/s 2024-04-25T10:17:20.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.143 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022724 s, 46.1 MB/s 2024-04-25T10:17:20.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0182224 s, 57.5 MB/s 2024-04-25T10:17:20.189 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.190 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211068 s, 49.7 MB/s 2024-04-25T10:17:20.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.218 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.218 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022466 s, 46.7 MB/s 2024-04-25T10:17:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.244 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.245 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246058 s, 42.6 MB/s 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.271 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022677 s, 46.2 MB/s 2024-04-25T10:17:20.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:20.274 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:20.274 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:20.274 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.26 2024-04-25T10:17:20.274 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:20.274 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.26/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:20.274 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:20.275 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:20.362 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.363 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.363 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0248878 s, 42.1 MB/s 2024-04-25T10:17:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211566 s, 49.6 MB/s 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.415 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240145 s, 43.7 MB/s 2024-04-25T10:17:20.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.440 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.441 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213433 s, 49.1 MB/s 2024-04-25T10:17:20.466 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.466 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.466 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225805 s, 46.4 MB/s 2024-04-25T10:17:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200311 s, 52.3 MB/s 2024-04-25T10:17:20.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.513 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.513 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0197187 s, 53.2 MB/s 2024-04-25T10:17:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.539 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02117 s, 49.5 MB/s 2024-04-25T10:17:20.564 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.564 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.564 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204304 s, 51.3 MB/s 2024-04-25T10:17:20.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.588 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.588 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200954 s, 52.2 MB/s 2024-04-25T10:17:20.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.614 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.614 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227045 s, 46.2 MB/s 2024-04-25T10:17:20.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.640 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.640 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215842 s, 48.6 MB/s 2024-04-25T10:17:20.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.669 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.669 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025178 s, 41.6 MB/s 2024-04-25T10:17:20.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.697 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.697 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237834 s, 44.1 MB/s 2024-04-25T10:17:20.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221532 s, 47.3 MB/s 2024-04-25T10:17:20.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.747 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.748 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208943 s, 50.2 MB/s 2024-04-25T10:17:20.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.775 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.775 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237039 s, 44.2 MB/s 2024-04-25T10:17:20.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.802 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.802 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230702 s, 45.5 MB/s 2024-04-25T10:17:20.829 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.829 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.829 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0227805 s, 46.0 MB/s 2024-04-25T10:17:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234725 s, 44.7 MB/s 2024-04-25T10:17:20.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.881 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.881 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206824 s, 50.7 MB/s 2024-04-25T10:17:20.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.907 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.907 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211325 s, 49.6 MB/s 2024-04-25T10:17:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.934 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231272 s, 45.3 MB/s 2024-04-25T10:17:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.960 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.961 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224466 s, 46.7 MB/s 2024-04-25T10:17:20.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:20.987 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:20.987 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220436 s, 47.6 MB/s 2024-04-25T10:17:21.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.013 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.013 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221262 s, 47.4 MB/s 2024-04-25T10:17:21.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025139 s, 41.7 MB/s 2024-04-25T10:17:21.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.066 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.067 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194911 s, 53.8 MB/s 2024-04-25T10:17:21.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.093 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.093 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225279 s, 46.5 MB/s 2024-04-25T10:17:21.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.121 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.121 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022612 s, 46.4 MB/s 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.148 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223286 s, 47.0 MB/s 2024-04-25T10:17:21.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.172 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.172 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204138 s, 51.4 MB/s 2024-04-25T10:17:21.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.27 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.27/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:21.175 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:21.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0261607 s, 40.1 MB/s 2024-04-25T10:17:21.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.294 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0254058 s, 41.3 MB/s 2024-04-25T10:17:21.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.322 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.322 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242328 s, 43.3 MB/s 2024-04-25T10:17:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.349 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230608 s, 45.5 MB/s 2024-04-25T10:17:21.382 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.383 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.383 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298068 s, 35.2 MB/s 2024-04-25T10:17:21.407 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209573 s, 50.0 MB/s 2024-04-25T10:17:21.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.434 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216015 s, 48.5 MB/s 2024-04-25T10:17:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.461 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022832 s, 45.9 MB/s 2024-04-25T10:17:21.486 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.487 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222794 s, 47.1 MB/s 2024-04-25T10:17:21.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.514 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.514 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022279 s, 47.1 MB/s 2024-04-25T10:17:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.541 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236053 s, 44.4 MB/s 2024-04-25T10:17:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.566 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204606 s, 51.2 MB/s 2024-04-25T10:17:21.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.590 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.590 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203068 s, 51.6 MB/s 2024-04-25T10:17:21.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.618 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.618 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236694 s, 44.3 MB/s 2024-04-25T10:17:21.638 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.639 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.639 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0181087 s, 57.9 MB/s 2024-04-25T10:17:21.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.667 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.667 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244193 s, 42.9 MB/s 2024-04-25T10:17:21.691 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.691 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.691 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0205547 s, 51.0 MB/s 2024-04-25T10:17:21.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.719 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.719 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231035 s, 45.4 MB/s 2024-04-25T10:17:21.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.744 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.744 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208447 s, 50.3 MB/s 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.771 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022974 s, 45.6 MB/s 2024-04-25T10:17:21.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.799 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237316 s, 44.2 MB/s 2024-04-25T10:17:21.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.826 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.826 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023493 s, 44.6 MB/s 2024-04-25T10:17:21.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.850 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.850 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204728 s, 51.2 MB/s 2024-04-25T10:17:21.874 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.875 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.875 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0208904 s, 50.2 MB/s 2024-04-25T10:17:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200671 s, 52.3 MB/s 2024-04-25T10:17:21.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.919 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.919 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0173522 s, 60.4 MB/s 2024-04-25T10:17:21.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.941 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.942 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195613 s, 53.6 MB/s 2024-04-25T10:17:21.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.970 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246603 s, 42.5 MB/s 2024-04-25T10:17:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:21.990 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:21.991 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0172521 s, 60.8 MB/s 2024-04-25T10:17:22.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.015 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.015 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210894 s, 49.7 MB/s 2024-04-25T10:17:22.042 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.043 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.043 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230876 s, 45.4 MB/s 2024-04-25T10:17:22.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.069 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.069 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226092 s, 46.4 MB/s 2024-04-25T10:17:22.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.28 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.28/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:22.072 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.235 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257624 s, 40.7 MB/s 2024-04-25T10:17:22.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.259 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.259 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217279 s, 48.3 MB/s 2024-04-25T10:17:22.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.291 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.291 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277979 s, 37.7 MB/s 2024-04-25T10:17:22.316 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.316 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.316 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0207479 s, 50.5 MB/s 2024-04-25T10:17:22.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.342 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.023337 s, 44.9 MB/s 2024-04-25T10:17:22.367 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.368 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0220823 s, 47.5 MB/s 2024-04-25T10:17:22.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.402 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.402 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0297763 s, 35.2 MB/s 2024-04-25T10:17:22.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.429 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.429 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224228 s, 46.8 MB/s 2024-04-25T10:17:22.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.457 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.457 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240548 s, 43.6 MB/s 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.485 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240693 s, 43.6 MB/s 2024-04-25T10:17:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219775 s, 47.7 MB/s 2024-04-25T10:17:22.535 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.535 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.535 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0203848 s, 51.4 MB/s 2024-04-25T10:17:22.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.565 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.565 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256443 s, 40.9 MB/s 2024-04-25T10:17:22.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.595 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.595 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266705 s, 39.3 MB/s 2024-04-25T10:17:22.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.620 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.620 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0216991 s, 48.3 MB/s 2024-04-25T10:17:22.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.642 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.642 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.01959 s, 53.5 MB/s 2024-04-25T10:17:22.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.675 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.675 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.028534 s, 36.7 MB/s 2024-04-25T10:17:22.702 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.702 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.702 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225344 s, 46.5 MB/s 2024-04-25T10:17:22.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.730 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.730 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239947 s, 43.7 MB/s 2024-04-25T10:17:22.756 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.757 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.757 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222542 s, 47.1 MB/s 2024-04-25T10:17:22.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239166 s, 43.8 MB/s 2024-04-25T10:17:22.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.812 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.812 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235597 s, 44.5 MB/s 2024-04-25T10:17:22.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.840 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.840 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237605 s, 44.1 MB/s 2024-04-25T10:17:22.864 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.864 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.864 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0210558 s, 49.8 MB/s 2024-04-25T10:17:22.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.891 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.891 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234266 s, 44.8 MB/s 2024-04-25T10:17:22.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.915 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.915 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204069 s, 51.4 MB/s 2024-04-25T10:17:22.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.947 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.947 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266831 s, 39.3 MB/s 2024-04-25T10:17:22.970 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:22.971 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:22.971 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198499 s, 52.8 MB/s 2024-04-25T10:17:23.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.000 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241195 s, 43.5 MB/s 2024-04-25T10:17:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.025 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214292 s, 48.9 MB/s 2024-04-25T10:17:23.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.052 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.052 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229523 s, 45.7 MB/s 2024-04-25T10:17:23.081 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.081 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.081 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.02418 s, 43.4 MB/s 2024-04-25T10:17:23.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:23.083 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:23.084 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:23.084 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.29 2024-04-25T10:17:23.084 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:23.084 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.29/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:23.084 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:23.084 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:23.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.165 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.165 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221264 s, 47.4 MB/s 2024-04-25T10:17:23.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.193 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.193 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243852 s, 43.0 MB/s 2024-04-25T10:17:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.223 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0257637 s, 40.7 MB/s 2024-04-25T10:17:23.249 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.249 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.249 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0219883 s, 47.7 MB/s 2024-04-25T10:17:23.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.274 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.274 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0212764 s, 49.3 MB/s 2024-04-25T10:17:23.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.302 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.302 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237112 s, 44.2 MB/s 2024-04-25T10:17:23.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.330 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.330 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240016 s, 43.7 MB/s 2024-04-25T10:17:23.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.356 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.356 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0223988 s, 46.8 MB/s 2024-04-25T10:17:23.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.385 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.385 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0247612 s, 42.3 MB/s 2024-04-25T10:17:23.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.408 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.408 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019288 s, 54.4 MB/s 2024-04-25T10:17:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.433 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0206481 s, 50.8 MB/s 2024-04-25T10:17:23.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.460 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.460 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235713 s, 44.5 MB/s 2024-04-25T10:17:23.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.488 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.488 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243681 s, 43.0 MB/s 2024-04-25T10:17:23.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.516 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239147 s, 43.8 MB/s 2024-04-25T10:17:23.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.543 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.543 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233009 s, 45.0 MB/s 2024-04-25T10:17:23.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.574 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0266917 s, 39.3 MB/s 2024-04-25T10:17:23.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.603 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0246452 s, 42.5 MB/s 2024-04-25T10:17:23.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.624 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0168364 s, 62.3 MB/s 2024-04-25T10:17:23.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.650 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.650 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0229418 s, 45.7 MB/s 2024-04-25T10:17:23.678 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.679 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.679 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240755 s, 43.6 MB/s 2024-04-25T10:17:23.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.707 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.707 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0244543 s, 42.9 MB/s 2024-04-25T10:17:23.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.732 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.732 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0211097 s, 49.7 MB/s 2024-04-25T10:17:23.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.758 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.758 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221138 s, 47.4 MB/s 2024-04-25T10:17:23.784 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231271 s, 45.3 MB/s 2024-04-25T10:17:23.808 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.809 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.809 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198345 s, 52.9 MB/s 2024-04-25T10:17:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0204456 s, 51.3 MB/s 2024-04-25T10:17:23.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.860 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.860 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0224777 s, 46.6 MB/s 2024-04-25T10:17:23.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.889 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.889 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.024791 s, 42.3 MB/s 2024-04-25T10:17:23.917 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.917 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.918 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0238958 s, 43.9 MB/s 2024-04-25T10:17:23.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.943 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.943 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221994 s, 47.2 MB/s 2024-04-25T10:17:23.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:23.974 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:23.974 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0271517 s, 38.6 MB/s 2024-04-25T10:17:24.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.000 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.001 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215978 s, 48.6 MB/s 2024-04-25T10:17:24.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.30 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.30/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:24.003 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:24.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.101 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.101 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.033328 s, 31.5 MB/s 2024-04-25T10:17:24.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.124 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.124 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202821 s, 51.7 MB/s 2024-04-25T10:17:24.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0236459 s, 44.3 MB/s 2024-04-25T10:17:24.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234925 s, 44.6 MB/s 2024-04-25T10:17:24.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.205 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.205 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222451 s, 47.1 MB/s 2024-04-25T10:17:24.230 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.231 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.231 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221705 s, 47.3 MB/s 2024-04-25T10:17:24.262 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.263 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.263 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0277739 s, 37.8 MB/s 2024-04-25T10:17:24.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.289 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.289 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222742 s, 47.1 MB/s 2024-04-25T10:17:24.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.314 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.314 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0201933 s, 51.9 MB/s 2024-04-25T10:17:24.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.342 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.342 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243648 s, 43.0 MB/s 2024-04-25T10:17:24.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.376 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.376 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298845 s, 35.1 MB/s 2024-04-25T10:17:24.403 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.404 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.404 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0237253 s, 44.2 MB/s 2024-04-25T10:17:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.431 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0233696 s, 44.9 MB/s 2024-04-25T10:17:24.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.465 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.466 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0306865 s, 34.2 MB/s 2024-04-25T10:17:24.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.491 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.491 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0217207 s, 48.3 MB/s 2024-04-25T10:17:24.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.519 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.519 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234273 s, 44.8 MB/s 2024-04-25T10:17:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.549 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260608 s, 40.2 MB/s 2024-04-25T10:17:24.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.574 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.574 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0215312 s, 48.7 MB/s 2024-04-25T10:17:24.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.603 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.603 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0240513 s, 43.6 MB/s 2024-04-25T10:17:24.631 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.632 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.632 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0245617 s, 42.7 MB/s 2024-04-25T10:17:24.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.661 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.661 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0262869 s, 39.9 MB/s 2024-04-25T10:17:24.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.684 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.684 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194076 s, 54.0 MB/s 2024-04-25T10:17:24.710 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.711 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022431 s, 46.7 MB/s 2024-04-25T10:17:24.736 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.736 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.736 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218541 s, 48.0 MB/s 2024-04-25T10:17:24.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.760 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.760 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0198551 s, 52.8 MB/s 2024-04-25T10:17:24.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.785 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.785 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0209506 s, 50.0 MB/s 2024-04-25T10:17:24.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.810 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.810 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0202778 s, 51.7 MB/s 2024-04-25T10:17:24.836 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.837 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.837 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226209 s, 46.4 MB/s 2024-04-25T10:17:24.869 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.870 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.870 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0284062 s, 36.9 MB/s 2024-04-25T10:17:24.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.897 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.897 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235138 s, 44.6 MB/s 2024-04-25T10:17:24.923 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.924 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.924 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0222218 s, 47.2 MB/s 2024-04-25T10:17:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:24.952 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0243321 s, 43.1 MB/s 2024-04-25T10:17:24.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c ' 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p scrub_control_test_path/dir.31 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> for ((i = 0; i < 32; i++)); do 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/dev/urandom of=scrub_control_test_path/dir.31/filename.$i bs=1M conv=fdatasync count=1 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> done 2024-04-25T10:17:24.955 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:17:25.039 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.142 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.143 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214416 s, 48.9 MB/s 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242156 s, 43.3 MB/s 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0252702 s, 41.5 MB/s 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.144 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0221666 s, 47.3 MB/s 2024-04-25T10:17:25.151 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.152 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.152 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0249962 s, 41.9 MB/s 2024-04-25T10:17:25.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.179 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.179 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0226923 s, 46.2 MB/s 2024-04-25T10:17:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.204 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.020846 s, 50.3 MB/s 2024-04-25T10:17:25.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.238 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.238 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0298113 s, 35.2 MB/s 2024-04-25T10:17:25.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.264 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.264 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214742 s, 48.8 MB/s 2024-04-25T10:17:25.293 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.294 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.294 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0256 s, 41.0 MB/s 2024-04-25T10:17:25.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.321 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.321 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0230721 s, 45.4 MB/s 2024-04-25T10:17:25.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.351 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.351 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0260662 s, 40.2 MB/s 2024-04-25T10:17:25.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.379 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.379 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0239603 s, 43.8 MB/s 2024-04-25T10:17:25.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.406 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.406 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0225663 s, 46.5 MB/s 2024-04-25T10:17:25.433 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.434 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0235258 s, 44.6 MB/s 2024-04-25T10:17:25.466 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.466 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.467 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0289022 s, 36.3 MB/s 2024-04-25T10:17:25.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.493 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.493 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0228188 s, 46.0 MB/s 2024-04-25T10:17:25.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.516 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.516 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0192061 s, 54.6 MB/s 2024-04-25T10:17:25.544 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.545 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.545 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232386 s, 45.1 MB/s 2024-04-25T10:17:25.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.572 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.572 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0231994 s, 45.2 MB/s 2024-04-25T10:17:25.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.596 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.597 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0200594 s, 52.3 MB/s 2024-04-25T10:17:25.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.623 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.623 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0218378 s, 48.0 MB/s 2024-04-25T10:17:25.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.649 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.649 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.022005 s, 47.7 MB/s 2024-04-25T10:17:25.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.672 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.673 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0195324 s, 53.7 MB/s 2024-04-25T10:17:25.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.700 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.700 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0241996 s, 43.3 MB/s 2024-04-25T10:17:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.723 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0194889 s, 53.8 MB/s 2024-04-25T10:17:25.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.752 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.753 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.025473 s, 41.2 MB/s 2024-04-25T10:17:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.778 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214884 s, 48.8 MB/s 2024-04-25T10:17:25.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.798 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.798 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0174347 s, 60.1 MB/s 2024-04-25T10:17:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.825 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234768 s, 44.7 MB/s 2024-04-25T10:17:25.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.852 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.852 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0234911 s, 44.6 MB/s 2024-04-25T10:17:25.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-25T10:17:25.877 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-25T10:17:25.877 INFO:teuthology.orchestra.run.smithi165.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0213216 s, 49.2 MB/s 2024-04-25T10:17:25.880 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 1 2024-04-25T10:17:26.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.080+0000 7f1b2da00640 1 Processor -- start 2024-04-25T10:17:26.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.080+0000 7f1b2da00640 1 -- start start 2024-04-25T10:17:26.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.081+0000 7f1b2da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281446d0 0x7f1b280b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.081+0000 7f1b2da00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b28144d50 0x7f1b280b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b2da00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b280cccc0 con 0x7f1b281452a0 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b2da00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b280b2b20 con 0x7f1b28144d50 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b2da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2806d400 con 0x7f1b281446d0 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b1fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b28144d50 0x7f1b280b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b27400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1b281452a0 0x7f1b280cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:57114/0 (socket says 172.21.15.165:57114) 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b27400640 1 -- 172.21.15.165:0/1547847855 learned_addr learned my addr 172.21.15.165:0/1547847855 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b1fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b28144d50 0x7f1b280b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:34602/0 (socket says 172.21.15.165:34602) 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b26a00640 1 --2- 172.21.15.165:0/1547847855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281446d0 0x7f1b280b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b1fe00640 1 -- 172.21.15.165:0/1547847855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281446d0 msgr2=0x7f1b280b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.082+0000 7f1b1fe00640 1 --2- 172.21.15.165:0/1547847855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281446d0 0x7f1b280b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.083+0000 7f1b1fe00640 1 -- 172.21.15.165:0/1547847855 >> v1:172.21.15.165:6789/0 conn(0x7f1b281452a0 legacy=0x7f1b280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.083+0000 7f1b1fe00640 1 -- 172.21.15.165:0/1547847855 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280cc140 con 0x7f1b28144d50 2024-04-25T10:17:26.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.083+0000 7f1b26000640 1 -- 172.21.15.165:0/1547847855 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 106555096 0 0) 0x7f1b280cccc0 con 0x7f1b281452a0 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.084+0000 7f1b1fe00640 1 --2- 172.21.15.165:0/1547847855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280b4460 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f1b100097a0 tx=0x7f1b1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbe5ee3b79632438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.084+0000 7f1b26000640 1 -- 172.21.15.165:0/1547847855 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10037070 con 0x7f1b28144d50 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.084+0000 7f1b26000640 1 -- 172.21.15.165:0/1547847855 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b10032440 con 0x7f1b28144d50 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.084+0000 7f1b26000640 1 -- 172.21.15.165:0/1547847855 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10032810 con 0x7f1b28144d50 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.084+0000 7f1b2da00640 1 -- 172.21.15.165:0/1547847855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 msgr2=0x7f1b280b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.084+0000 7f1b2da00640 1 --2- 172.21.15.165:0/1547847855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280b4460 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f1b100097a0 tx=0x7f1b1002a180 comp rx=0 tx=0).stop 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.085+0000 7f1b2da00640 1 -- 172.21.15.165:0/1547847855 shutdown_connections 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.085+0000 7f1b2da00640 1 --2- 172.21.15.165:0/1547847855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280b4460 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.085+0000 7f1b2da00640 1 --2- 172.21.15.165:0/1547847855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281446d0 0x7f1b280b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.085+0000 7f1b2da00640 1 -- 172.21.15.165:0/1547847855 >> 172.21.15.165:0/1547847855 conn(0x7f1b2806abe0 msgr2=0x7f1b2806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:26.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.085+0000 7f1b2da00640 1 -- 172.21.15.165:0/1547847855 shutdown_connections 2024-04-25T10:17:26.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.085+0000 7f1b2da00640 1 -- 172.21.15.165:0/1547847855 wait complete. 2024-04-25T10:17:26.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.086+0000 7f1b2da00640 1 Processor -- start 2024-04-25T10:17:26.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.086+0000 7f1b2da00640 1 -- start start 2024-04-25T10:17:26.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.087+0000 7f1b2da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b281446d0 0x7f1b280b0d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.087+0000 7f1b2da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280cb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.087+0000 7f1b2da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281452a0 0x7f1b281366c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.087+0000 7f1b2da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280cc700 con 0x7f1b28144d50 2024-04-25T10:17:26.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.087+0000 7f1b2da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280b49a0 con 0x7f1b281446d0 2024-04-25T10:17:26.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.087+0000 7f1b2da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b280cc9e0 con 0x7f1b281452a0 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280cb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280cb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:34604/0 (socket says 172.21.15.165:34604) 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 -- 172.21.15.165:0/3452491346 learned_addr learned my addr 172.21.15.165:0/3452491346 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 -- 172.21.15.165:0/3452491346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281452a0 msgr2=0x7f1b281366c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281452a0 0x7f1b281366c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 -- 172.21.15.165:0/3452491346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b281446d0 msgr2=0x7f1b280b0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b281446d0 0x7f1b280b0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 -- 172.21.15.165:0/3452491346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b280ca400 con 0x7f1b28144d50 2024-04-25T10:17:26.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.088+0000 7f1b1fe00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280cb160 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f1b100097a0 tx=0x7f1b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.089+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10037050 con 0x7f1b28144d50 2024-04-25T10:17:26.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.089+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b10038040 con 0x7f1b28144d50 2024-04-25T10:17:26.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.089+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b281394e0 con 0x7f1b28144d50 2024-04-25T10:17:26.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.089+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b10004470 con 0x7f1b28144d50 2024-04-25T10:17:26.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.089+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b280cac40 con 0x7f1b28144d50 2024-04-25T10:17:26.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.090+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b28073a00 con 0x7f1b28144d50 2024-04-25T10:17:26.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.091+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b10030900 con 0x7f1b28144d50 2024-04-25T10:17:26.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.092+0000 7f1b1ea00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b0406b4d0 0x7f1b0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:26.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.092+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1b100ab120 con 0x7f1b28144d50 2024-04-25T10:17:26.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.092+0000 7f1b26a00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b0406b4d0 0x7f1b0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:26.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.093+0000 7f1b26a00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b0406b4d0 0x7f1b0406d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098f0 tx=0x7f1b18008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:26.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.097+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b281394e0 con 0x7f1b28144d50 2024-04-25T10:17:26.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:26.330+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"} v 0) v1 -- 0x7f1b280accb0 con 0x7f1b28144d50 2024-04-25T10:17:27.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.049+0000 7f1b1ea00640 1 -- 172.21.15.165:0/3452491346 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]=0 v102) v1 ==== 105+0+0 (secure 0 0 0) 0x7f1b280ca400 con 0x7f1b28144d50 2024-04-25T10:17:27.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b0406b4d0 msgr2=0x7f1b0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b0406b4d0 0x7f1b0406d990 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f1b180098f0 tx=0x7f1b18008040 comp rx=0 tx=0).stop 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 msgr2=0x7f1b280cb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280cb160 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f1b100097a0 tx=0x7f1b100046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 shutdown_connections 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b0406b4d0 0x7f1b0406d990 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b281452a0 0x7f1b281366c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b28144d50 0x7f1b280cb160 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 --2- 172.21.15.165:0/3452491346 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b281446d0 0x7f1b280b0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 >> 172.21.15.165:0/3452491346 conn(0x7f1b2806abe0 msgr2=0x7f1b28071170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 shutdown_connections 2024-04-25T10:17:27.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.054+0000 7f1b2da00640 1 -- 172.21.15.165:0/3452491346 wait complete. 2024-04-25T10:17:27.076 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-25T10:17:27.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.279+0000 7fbb34e00640 1 Processor -- start 2024-04-25T10:17:27.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.280+0000 7fbb34e00640 1 -- start start 2024-04-25T10:17:27.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.281+0000 7fbb34e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 0x7fbb300cf6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.281+0000 7fbb34e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbb300d0870 0x7fbb3006b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.281+0000 7fbb34e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb300617a0 con 0x7fbb3015dad0 2024-04-25T10:17:27.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.281+0000 7fbb34e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb30061a80 con 0x7fbb300d0870 2024-04-25T10:17:27.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.281+0000 7fbb34e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006ad80 con 0x7fbb3006f3d0 2024-04-25T10:17:27.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.282+0000 7fbb2f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbb3015dad0 0x7fbb3006ecc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41238/0 (socket says 172.21.15.165:41238) 2024-04-25T10:17:27.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.282+0000 7fbb2f400640 1 -- 172.21.15.165:0/2986970813 learned_addr learned my addr 172.21.15.165:0/2986970813 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.282+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/2986970813 >> v2:172.21.15.172:3300/0 conn(0x7fbb300d0870 0x7fbb3006b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.282+0000 7fbb2ea00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 0x7fbb300cf6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.282+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1027476353 0 0) 0x7fbb300617a0 con 0x7fbb3015dad0 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.282+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb14003520 con 0x7fbb3015dad0 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2233186825 0 0) 0x7fbb14003520 con 0x7fbb3015dad0 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 msgr2=0x7fbb300cf6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 0x7fbb300cf6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 >> v2:172.21.15.172:3300/0 conn(0x7fbb300d0870 msgr2=0x7fbb3006b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 --2- 172.21.15.165:0/2986970813 >> v2:172.21.15.172:3300/0 conn(0x7fbb300d0870 0x7fbb3006b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3006b060 con 0x7fbb3015dad0 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fbb20001670 con 0x7fbb3015dad0 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/2986970813 >> v2:172.21.15.172:3300/0 conn(0x7fbb300d0870 0x7fbb3006b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 >> v1:172.21.15.165:6789/0 conn(0x7fbb3015dad0 legacy=0x7fbb3006ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 0x7fbb140039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb14003f80 0x7fbb14004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb30061a80 con 0x7fbb14004d40 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3006ad80 con 0x7fbb14003f80 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1400b9c0 con 0x7fbb3006f3d0 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.283+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.284+0000 7fbb2ea00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb14003f80 0x7fbb14004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.284+0000 7fbb2fe00640 1 -- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 msgr2=0x7fbb140039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.284+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb3006f3d0 0x7fbb140039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.284+0000 7fbb2fe00640 1 -- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb14003f80 msgr2=0x7fbb14004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:27.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.284+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb14003f80 0x7fbb14004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.284+0000 7fbb2fe00640 1 -- 172.21.15.165:0/2986970813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb14003520 con 0x7fbb14004d40 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.285+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb140051c0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e380 tx=0x7fbb14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=587af4b8fec6c9d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.285+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb28009de0 con 0x7fbb14004d40 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.285+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb2802c620 con 0x7fbb14004d40 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.285+0000 7fbb2e000640 1 -- 172.21.15.165:0/2986970813 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2802c9d0 con 0x7fbb14004d40 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb34e00640 1 -- 172.21.15.165:0/2986970813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 msgr2=0x7fbb140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb34e00640 1 --2- 172.21.15.165:0/2986970813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb140051c0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7fbb2800e380 tx=0x7fbb14005830 comp rx=0 tx=0).stop 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb2fe00640 1 -- 172.21.15.165:0/2986970813 reap_dead start 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb34e00640 1 -- 172.21.15.165:0/2986970813 shutdown_connections 2024-04-25T10:17:27.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb34e00640 1 -- 172.21.15.165:0/2986970813 >> 172.21.15.165:0/2986970813 conn(0x7fbb300cfd50 msgr2=0x7fbb300d0160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:27.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb34e00640 1 -- 172.21.15.165:0/2986970813 shutdown_connections 2024-04-25T10:17:27.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.286+0000 7fbb34e00640 1 -- 172.21.15.165:0/2986970813 wait complete. 2024-04-25T10:17:27.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.288+0000 7fbb34e00640 1 Processor -- start 2024-04-25T10:17:27.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.288+0000 7fbb34e00640 1 -- start start 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.289+0000 7fbb34e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb301539c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb34e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb30160ac0 0x7fbb300b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb34e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb301492d0 0x7fbb30149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb34e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3015e030 con 0x7fbb14004d40 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb34e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3015deb0 con 0x7fbb301492d0 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb34e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb300cf6b0 con 0x7fbb30160ac0 2024-04-25T10:17:27.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb301539c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb301539c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35670/0 (socket says 172.21.15.165:35670) 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 -- 172.21.15.165:0/163611830 learned_addr learned my addr 172.21.15.165:0/163611830 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 -- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb30160ac0 msgr2=0x7fbb300b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2fe00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb301492d0 0x7fbb30149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb30160ac0 0x7fbb300b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 -- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb301492d0 msgr2=0x7fbb30149750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb301492d0 0x7fbb30149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:27.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.290+0000 7fbb2f400640 1 -- 172.21.15.165:0/163611830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb30153f00 con 0x7fbb14004d40 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.291+0000 7fbb2f400640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb301539c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c7a0 tx=0x7fbb2000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.291+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20018070 con 0x7fbb14004d40 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.291+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb20007500 con 0x7fbb14004d40 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.291+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb20004a20 con 0x7fbb14004d40 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.291+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3015fa20 con 0x7fbb14004d40 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.291+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3015ffe0 con 0x7fbb14004d40 2024-04-25T10:17:27.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.292+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb30073990 con 0x7fbb14004d40 2024-04-25T10:17:27.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.293+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbb20004bc0 con 0x7fbb14004d40 2024-04-25T10:17:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.294+0000 7fbb27400640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbb1006b500 0x7fbb1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.294+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbb20089990 con 0x7fbb14004d40 2024-04-25T10:17:27.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.294+0000 7fbb2ea00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbb1006b500 0x7fbb1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:27.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.295+0000 7fbb2ea00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbb1006b500 0x7fbb1006d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbb300b44f0 tx=0x7fbb1c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:27.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.299+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbb200585e0 con 0x7fbb14004d40 2024-04-25T10:17:27.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:27.536+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7fbb3015ca30 con 0x7fbb14004d40 2024-04-25T10:17:28.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.053+0000 7fbb27400640 1 -- 172.21.15.165:0/163611830 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v103) v1 ==== 121+0+0 (secure 0 0 0) 0x7fbb20057ee0 con 0x7fbb14004d40 2024-04-25T10:17:28.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbb1006b500 msgr2=0x7fbb1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbb1006b500 0x7fbb1006d9c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fbb300b44f0 tx=0x7fbb1c008040 comp rx=0 tx=0).stop 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 msgr2=0x7fbb301539c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb301539c0 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7fbb2000c7a0 tx=0x7fbb2000cc60 comp rx=0 tx=0).stop 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 shutdown_connections 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbb1006b500 0x7fbb1006d9c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbb301492d0 0x7fbb30149750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbb30160ac0 0x7fbb300b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 --2- 172.21.15.165:0/163611830 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbb14004d40 0x7fbb301539c0 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 >> 172.21.15.165:0/163611830 conn(0x7fbb300cfd50 msgr2=0x7fbb3006eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 shutdown_connections 2024-04-25T10:17:28.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.057+0000 7fbb34e00640 1 -- 172.21.15.165:0/163611830 wait complete. 2024-04-25T10:17:28.081 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.252+0000 7ff8c7200640 1 Processor -- start 2024-04-25T10:17:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.252+0000 7ff8c7200640 1 -- start start 2024-04-25T10:17:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.253+0000 7ff8c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0144440 0x7ff8c0070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.253+0000 7ff8c7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff8c0071530 0x7ff8c00cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8c015bf30 con 0x7ff8c00cd860 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8c015c860 con 0x7ff8c0071530 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c0071970 con 0x7ff8c0144440 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff8c0071530 0x7ff8c00cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff8c0071530 0x7ff8c00cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35686/0 (socket says 172.21.15.165:35686) 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 -- 172.21.15.165:0/414606939 learned_addr learned my addr 172.21.15.165:0/414606939 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5e00640 1 --2- 172.21.15.165:0/414606939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0144440 0x7ff8c0070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 -- 172.21.15.165:0/414606939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0144440 msgr2=0x7ff8c0070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 --2- 172.21.15.165:0/414606939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0144440 0x7ff8c0070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 -- 172.21.15.165:0/414606939 >> v1:172.21.15.172:6789/0 conn(0x7ff8c00cd860 legacy=0x7ff8c006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.254+0000 7ff8c5400640 1 -- 172.21.15.165:0/414606939 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c015c210 con 0x7ff8c0071530 2024-04-25T10:17:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.256+0000 7ff8c5400640 1 --2- 172.21.15.165:0/414606939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0071530 0x7ff8c00cd320 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009870 tx=0x7ff8b002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=745afe4b326fe3ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.256+0000 7ff8c4a00640 1 -- 172.21.15.165:0/414606939 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0037070 con 0x7ff8c0071530 2024-04-25T10:17:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.256+0000 7ff8c4a00640 1 -- 172.21.15.165:0/414606939 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8b0032440 con 0x7ff8c0071530 2024-04-25T10:17:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.256+0000 7ff8c4a00640 1 -- 172.21.15.165:0/414606939 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b0032810 con 0x7ff8c0071530 2024-04-25T10:17:28.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 -- 172.21.15.165:0/414606939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0071530 msgr2=0x7ff8c00cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 --2- 172.21.15.165:0/414606939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0071530 0x7ff8c00cd320 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7ff8b0009870 tx=0x7ff8b002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 -- 172.21.15.165:0/414606939 shutdown_connections 2024-04-25T10:17:28.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 --2- 172.21.15.165:0/414606939 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0071530 0x7ff8c00cd320 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 --2- 172.21.15.165:0/414606939 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0144440 0x7ff8c0070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 -- 172.21.15.165:0/414606939 >> 172.21.15.165:0/414606939 conn(0x7ff8c006abe0 msgr2=0x7ff8c006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 -- 172.21.15.165:0/414606939 shutdown_connections 2024-04-25T10:17:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.257+0000 7ff8c7200640 1 -- 172.21.15.165:0/414606939 wait complete. 2024-04-25T10:17:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.259+0000 7ff8c7200640 1 Processor -- start 2024-04-25T10:17:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.259+0000 7ff8c7200640 1 -- start start 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.260+0000 7ff8c7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0071530 0x7ff8c0139fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.260+0000 7ff8c7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c00cd860 0x7ff8c00c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.260+0000 7ff8c7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 0x7ff8c00c8940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00cdcd0 con 0x7ff8c0144440 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c015bc30 con 0x7ff8c00cd860 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8c00cde50 con 0x7ff8c0071530 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 0x7ff8c00c8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 0x7ff8c00c8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35700/0 (socket says 172.21.15.165:35700) 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 -- 172.21.15.165:0/2467280429 learned_addr learned my addr 172.21.15.165:0/2467280429 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 -- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0071530 msgr2=0x7ff8c0139fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0071530 0x7ff8c0139fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 -- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c00cd860 msgr2=0x7ff8c00c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c5400640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c00cd860 0x7ff8c00c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c00cd860 0x7ff8c00c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.261+0000 7ff8c6800640 1 -- 172.21.15.165:0/2467280429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8c00c78a0 con 0x7ff8c0144440 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.262+0000 7ff8c6800640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 0x7ff8c00c8940 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c780 tx=0x7ff8bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.262+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc004b60 con 0x7ff8c0144440 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.262+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc004d00 con 0x7ff8c0144440 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.262+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc00aa90 con 0x7ff8c0144440 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.262+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8c00c7b80 con 0x7ff8c0144440 2024-04-25T10:17:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.262+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8c00c8140 con 0x7ff8c0144440 2024-04-25T10:17:28.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.263+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c00b4610 con 0x7ff8c0144440 2024-04-25T10:17:28.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.264+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff8bc00ac30 con 0x7ff8c0144440 2024-04-25T10:17:28.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.265+0000 7ff8ba000640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:28.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.265+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff8bc08a660 con 0x7ff8c0144440 2024-04-25T10:17:28.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.265+0000 7ff8c5e00640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:28.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.266+0000 7ff8c5e00640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8a406b4d0 0x7ff8a406d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:28.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.270+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff8c00b4610 con 0x7ff8c0144440 2024-04-25T10:17:28.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.494+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8c006ec10 con 0x7ff8c0144440 2024-04-25T10:17:28.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.496+0000 7ff8ba000640 1 -- 172.21.15.165:0/2467280429 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4963 (secure 0 0 0) 0x7ff8bc088e20 con 0x7ff8c0144440 2024-04-25T10:17:28.499 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:28.499 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":104,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:17:28.059079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:17:28.499 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 104 2024-04-25T10:17:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.498+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8a406b4d0 msgr2=0x7ff8a406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.498+0000 7ff8c7200640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8a406b4d0 0x7ff8a406d990 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff8b4006fd0 tx=0x7ff8b4008040 comp rx=0 tx=0).stop 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.498+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 msgr2=0x7ff8c00c8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.498+0000 7ff8c7200640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 0x7ff8c00c8940 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c780 tx=0x7ff8bc00cc40 comp rx=0 tx=0).stop 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 shutdown_connections 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff8a406b4d0 0x7ff8a406d990 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff8c0144440 0x7ff8c00c8940 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff8c00cd860 0x7ff8c00c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 --2- 172.21.15.165:0/2467280429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff8c0071530 0x7ff8c0139fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 >> 172.21.15.165:0/2467280429 conn(0x7ff8c006abe0 msgr2=0x7ff8c0142c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 shutdown_connections 2024-04-25T10:17:28.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:28.499+0000 7ff8c7200640 1 -- 172.21.15.165:0/2467280429 wait complete. 2024-04-25T10:17:28.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 104, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-25T10:16:42.983841+0000', 'modified': '2024-04-25T10:17:28.059079+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9765}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9762': {'gid': 9762, 'name': 'a', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.165:6835/1838340482', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 1838340482}, {'type': 'v1', 'addr': '172.21.15.165:6835', 'nonce': 1838340482}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}, 'gid_9765': {'gid': 9765, 'name': 'c', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6837/2515540447', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6836', 'nonce': 2515540447}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2515540447}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:17:28.921 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:17:28.921 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:17:29.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.131+0000 7f87f7a00640 1 Processor -- start 2024-04-25T10:17:29.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.131+0000 7f87f7a00640 1 -- start start 2024-04-25T10:17:29.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.132+0000 7f87f7a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87f015b710 0x7f87f00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f7a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f87f00ac8b0 0x7f87f00acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f7a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87f006f0e0 con 0x7f87f006ec50 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f7a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f015e490 con 0x7f87f00ac8b0 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f7a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87f00cfd80 con 0x7f87f015b710 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f87f00ac8b0 0x7f87f00acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f87f00ac8b0 0x7f87f00acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35712/0 (socket says 172.21.15.165:35712) 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 -- 172.21.15.165:0/4128527485 learned_addr learned my addr 172.21.15.165:0/4128527485 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87efe00640 1 --1- 172.21.15.165:0/4128527485 >> v1:172.21.15.165:6789/0 conn(0x7f87f006ec50 0x7f87f015e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41240/0 (socket says 172.21.15.165:41240) 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5000640 1 --2- 172.21.15.165:0/4128527485 >> v2:172.21.15.172:3300/0 conn(0x7f87f015b710 0x7f87f00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 -- 172.21.15.165:0/4128527485 >> v2:172.21.15.172:3300/0 conn(0x7f87f015b710 msgr2=0x7f87f00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 --2- 172.21.15.165:0/4128527485 >> v2:172.21.15.172:3300/0 conn(0x7f87f015b710 0x7f87f00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 -- 172.21.15.165:0/4128527485 >> v1:172.21.15.165:6789/0 conn(0x7f87f006ec50 legacy=0x7f87f015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.133+0000 7f87f5a00640 1 -- 172.21.15.165:0/4128527485 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f00d0060 con 0x7f87f00ac8b0 2024-04-25T10:17:29.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.134+0000 7f87f5a00640 1 --2- 172.21.15.165:0/4128527485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f00ac8b0 0x7f87f00acd00 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f87f006b950 tx=0x7f87e802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c281e73132de3f2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.135+0000 7f87ef400640 1 -- 172.21.15.165:0/4128527485 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8036070 con 0x7f87f00ac8b0 2024-04-25T10:17:29.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.135+0000 7f87ef400640 1 -- 172.21.15.165:0/4128527485 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8031600 con 0x7f87f00ac8b0 2024-04-25T10:17:29.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.135+0000 7f87ef400640 1 -- 172.21.15.165:0/4128527485 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e80319d0 con 0x7f87f00ac8b0 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.135+0000 7f87f7a00640 1 -- 172.21.15.165:0/4128527485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f00ac8b0 msgr2=0x7f87f00acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.135+0000 7f87f7a00640 1 --2- 172.21.15.165:0/4128527485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f00ac8b0 0x7f87f00acd00 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f87f006b950 tx=0x7f87e802a730 comp rx=0 tx=0).stop 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.136+0000 7f87f7a00640 1 -- 172.21.15.165:0/4128527485 shutdown_connections 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.136+0000 7f87f7a00640 1 --2- 172.21.15.165:0/4128527485 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f00ac8b0 0x7f87f00acd00 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.136+0000 7f87f7a00640 1 --2- 172.21.15.165:0/4128527485 >> v2:172.21.15.172:3300/0 conn(0x7f87f015b710 0x7f87f00cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.136+0000 7f87f7a00640 1 -- 172.21.15.165:0/4128527485 >> 172.21.15.165:0/4128527485 conn(0x7f87f006abe0 msgr2=0x7f87f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.136+0000 7f87f7a00640 1 -- 172.21.15.165:0/4128527485 shutdown_connections 2024-04-25T10:17:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.136+0000 7f87f7a00640 1 -- 172.21.15.165:0/4128527485 wait complete. 2024-04-25T10:17:29.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.137+0000 7f87f7a00640 1 Processor -- start 2024-04-25T10:17:29.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 -- start start 2024-04-25T10:17:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 0x7f87f01449f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87f00ac8b0 0x7f87f0071440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87f015b710 0x7f87f0071980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f015bc70 con 0x7f87f006ec50 2024-04-25T10:17:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f015baf0 con 0x7f87f00ac8b0 2024-04-25T10:17:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.138+0000 7f87f7a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f015f030 con 0x7f87f015b710 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 0x7f87f01449f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 0x7f87f01449f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35716/0 (socket says 172.21.15.165:35716) 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 -- 172.21.15.165:0/1358480883 learned_addr learned my addr 172.21.15.165:0/1358480883 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 -- 172.21.15.165:0/1358480883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87f015b710 msgr2=0x7f87f0071980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87f015b710 0x7f87f0071980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 -- 172.21.15.165:0/1358480883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87f00ac8b0 msgr2=0x7f87f0071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87f00ac8b0 0x7f87f0071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 -- 172.21.15.165:0/1358480883 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f0144f30 con 0x7f87f006ec50 2024-04-25T10:17:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.139+0000 7f87f5000640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 0x7f87f01449f0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d5f0 tx=0x7f87e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.140+0000 7f87ecc00640 1 -- 172.21.15.165:0/1358480883 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e40125f0 con 0x7f87f006ec50 2024-04-25T10:17:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.140+0000 7f87ecc00640 1 -- 172.21.15.165:0/1358480883 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e4019070 con 0x7f87f006ec50 2024-04-25T10:17:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.140+0000 7f87ecc00640 1 -- 172.21.15.165:0/1358480883 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e40129b0 con 0x7f87f006ec50 2024-04-25T10:17:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.140+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f00affc0 con 0x7f87f006ec50 2024-04-25T10:17:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.140+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87f00b0580 con 0x7f87f006ec50 2024-04-25T10:17:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.142+0000 7f87ecc00640 1 -- 172.21.15.165:0/1358480883 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87e4012b50 con 0x7f87f006ec50 2024-04-25T10:17:29.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.143+0000 7f87ecc00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87d006b4d0 0x7f87d006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.143+0000 7f87ecc00640 1 -- 172.21.15.165:0/1358480883 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87e408a850 con 0x7f87f006ec50 2024-04-25T10:17:29.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.143+0000 7f87efe00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87d006b4d0 0x7f87d006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.143+0000 7f87fa869740 1 Processor -- start 2024-04-25T10:17:29.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 -- start start 2024-04-25T10:17:29.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87efe00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87d006b4d0 0x7f87d006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000f60 tx=0x7f87dc007a00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a26950 0x558479a4e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 0x558479a26d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a6f5c0 0x558479a25110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55847998a620 con 0x558479a4ecd0 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558479933ca0 con 0x558479a6f5c0 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.144+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5584799866c0 con 0x558479a26950 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 0x558479a26d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 0x558479a26d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35718/0 (socket says 172.21.15.165:35718) 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 -- 172.21.15.165:0/1241363027 learned_addr learned my addr 172.21.15.165:0/1241363027 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 -- 172.21.15.165:0/1241363027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a26950 msgr2=0x558479a4e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a26950 0x558479a4e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 -- 172.21.15.165:0/1241363027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a6f5c0 msgr2=0x558479a25110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a6f5c0 0x558479a25110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 -- 172.21.15.165:0/1241363027 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55847993af40 con 0x558479a4ecd0 2024-04-25T10:17:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.145+0000 7f87efe00640 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 0x558479a26d30 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f87dc001eb0 tx=0x7f87dc00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49fc2457697c9a48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87e1600640 1 -- 172.21.15.165:0/1241363027 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc016110 con 0x558479a4ecd0 2024-04-25T10:17:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87e1600640 1 -- 172.21.15.165:0/1241363027 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc005c80 con 0x558479a4ecd0 2024-04-25T10:17:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87e1600640 1 -- 172.21.15.165:0/1241363027 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0162b0 con 0x558479a4ecd0 2024-04-25T10:17:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 -- 172.21.15.165:0/1241363027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 msgr2=0x558479a26d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 0x558479a26d30 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f87dc001eb0 tx=0x7f87dc00f000 comp rx=0 tx=0).stop 2024-04-25T10:17:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 -- 172.21.15.165:0/1241363027 shutdown_connections 2024-04-25T10:17:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a6f5c0 0x558479a25110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a4ecd0 0x558479a26d30 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 --2- 172.21.15.165:0/1241363027 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a26950 0x558479a4e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 -- 172.21.15.165:0/1241363027 >> 172.21.15.165:0/1241363027 conn(0x558479a23b70 msgr2=0x558479a6c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.146+0000 7f87fa869740 1 -- 172.21.15.165:0/1241363027 shutdown_connections 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.147+0000 7f87fa869740 1 -- 172.21.15.165:0/1241363027 wait complete. 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.147+0000 7f87fa869740 1 -- start start 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.147+0000 7f87fa869740 1 Processor -- start 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4ecd0 0x558479a42e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4b430 0x558479a6ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558479a4e300 con 0x558479a6f5c0 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584799291f0 con 0x558479a4b430 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558479a4dcf0 con 0x558479a4ecd0 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.148+0000 7f87fa869740 10 client.0 _ll_register_callbacks cb 0x558479946ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35728/0 (socket says 172.21.15.165:35728) 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 -- 172.21.15.165:0/2623648236 learned_addr learned my addr 172.21.15.165:0/2623648236 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 -- 172.21.15.165:0/2623648236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4ecd0 msgr2=0x558479a42e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4ecd0 0x558479a42e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 -- 172.21.15.165:0/2623648236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4b430 msgr2=0x558479a6ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4b430 0x558479a6ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 -- 172.21.15.165:0/2623648236 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558479948c50 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87efe00640 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4c280 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x558479a22ee0 tx=0x7f87dc001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f83cbb1ed552c4ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.149+0000 7f87d6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87d6000640 1 -- 172.21.15.165:0/2623648236 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87dc016ab0 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87d6000640 1 -- 172.21.15.165:0/2623648236 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87dc016c50 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87dc005c80 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87d6000640 1 -- 172.21.15.165:0/2623648236 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87dc032050 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 10 client.9990 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55847969f2d0 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87d6000640 1 -- 172.21.15.165:0/2623648236 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87dc005c80 con 0x558479a6f5c0 2024-04-25T10:17:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87d6000640 1 -- 172.21.15.165:0/2623648236 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f87dc035020 con 0x558479a6f5c0 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 10 client.9990 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 10 client.9990 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 20 client.9990 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a448c0 0x558479a7cc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 4 client.9990 mds_command: new command op to 9765 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.150+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 --> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558479997b60 con 0x558479a448c0 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.151+0000 7f87f5000640 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a448c0 0x558479a7cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.151+0000 7f87f5000640 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a448c0 0x558479a7cc00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=622776969118f7ee server_cookie=bfdadfb475c0d9a9 in_seq=0 out_seq=0 2024-04-25T10:17:29.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.151+0000 7f87d6000640 10 client.9990 ms_handle_connect on v2:172.21.15.165:6836/2515540447 2024-04-25T10:17:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87d6000640 1 -- 172.21.15.165:0/2623648236 <== mds.0 v2:172.21.15.165:6836/2515540447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x558479997b60 con 0x558479a448c0 2024-04-25T10:17:29.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87d6000640 10 client.9990 handle_command_reply: tid=0 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 client.9990 shutdown 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 msgr2=0x558479a4c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4c280 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x558479a22ee0 tx=0x7f87dc001120 comp rx=0 tx=0).stop 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 shutdown_connections 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a448c0 0x558479a7cc00 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4b430 0x558479a6ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4c280 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87d6000640 0 client.9990 ms_handle_reset on v2:172.21.15.165:6836/2515540447 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 --2- 172.21.15.165:0/2623648236 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4ecd0 0x558479a42e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 >> 172.21.15.165:0/2623648236 conn(0x558479a26950 msgr2=0x558479a208b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 shutdown_connections 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.152+0000 7f87fa869740 1 -- 172.21.15.165:0/2623648236 wait complete. 2024-04-25T10:17:29.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.153+0000 7f87fa869740 20 client.9990 trim_cache size 0 max 16384 2024-04-25T10:17:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.169+0000 7f87fa869740 1 Processor -- start 2024-04-25T10:17:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 -- start start 2024-04-25T10:17:29.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a49e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 0x558479a6f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f0200360 con 0x558479a6f5c0 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87f00b32c0 con 0x558479a4ecd0 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.170+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87f01ff040 con 0x558479a4b430 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35734/0 (socket says 172.21.15.165:35734) 2024-04-25T10:17:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 -- 172.21.15.165:0/237803896 learned_addr learned my addr 172.21.15.165:0/237803896 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5000640 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a49e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 -- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 msgr2=0x558479a49e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a49e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 -- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 msgr2=0x558479a6f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 0x558479a6f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.171+0000 7f87f5a00640 1 -- 172.21.15.165:0/237803896 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584797696b0 con 0x558479a6f5c0 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.172+0000 7f87f5a00640 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4cc50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f87e802ad20 tx=0x7f87e8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=923b734bb13157dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.172+0000 7f87e0c00640 1 -- 172.21.15.165:0/237803896 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8036050 con 0x558479a6f5c0 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.172+0000 7f87e0c00640 1 -- 172.21.15.165:0/237803896 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8042070 con 0x558479a6f5c0 2024-04-25T10:17:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.172+0000 7f87e0c00640 1 -- 172.21.15.165:0/237803896 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8009510 con 0x558479a6f5c0 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.172+0000 7f87fa869740 1 -- 172.21.15.165:0/237803896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 msgr2=0x558479a4cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.172+0000 7f87fa869740 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4cc50 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7f87e802ad20 tx=0x7f87e8004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 -- 172.21.15.165:0/237803896 shutdown_connections 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479a4cc50 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 0x558479a6f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 --2- 172.21.15.165:0/237803896 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a49e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 -- 172.21.15.165:0/237803896 >> 172.21.15.165:0/237803896 conn(0x558479a26950 msgr2=0x558479a4a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 -- 172.21.15.165:0/237803896 shutdown_connections 2024-04-25T10:17:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 -- 172.21.15.165:0/237803896 wait complete. 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.173+0000 7f87fa869740 1 -- start start 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.174+0000 7f87fa869740 1 Processor -- start 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a4cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 0x558479a4d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479ad7030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x558479a44740 con 0x558479a6f5c0 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x558479a4e300 con 0x558479a4ecd0 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x558479a44d40 con 0x558479a4b430 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87fa869740 10 client.0 _ll_register_callbacks cb 0x7f87dc0073d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87f5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479ad7030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87f5a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479ad7030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35750/0 (socket says 172.21.15.165:35750) 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87f5a00640 1 -- 172.21.15.165:0/1933295685 learned_addr learned my addr 172.21.15.165:0/1933295685 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87f5a00640 1 -- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 msgr2=0x558479a4cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87f5000640 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a4cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.175+0000 7f87f5a00640 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a4cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87f5a00640 1 -- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 msgr2=0x558479a4d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87f5a00640 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 0x558479a4d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87f5a00640 1 -- 172.21.15.165:0/1933295685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558479a55770 con 0x558479a6f5c0 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87f5a00640 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479ad7030 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f87e802a770 tx=0x7f87e8039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968231a8210db87d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87d6000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87d6000640 1 -- 172.21.15.165:0/1933295685 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87e8009510 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f87e8042070 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87d6000640 1 -- 172.21.15.165:0/1933295685 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f87e8051410 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87d6000640 1 -- 172.21.15.165:0/1933295685 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f87e8036020 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87fa869740 10 client.9996 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.176+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x558479a55400 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87d6000640 1 -- 172.21.15.165:0/1933295685 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87e8042070 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87d6000640 1 -- 172.21.15.165:0/1933295685 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f87e805b080 con 0x558479a6f5c0 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87fa869740 10 client.9996 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87fa869740 10 client.9996 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87fa869740 20 client.9996 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87fa869740 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a7ab70 0x558479a7af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87fa869740 4 client.9996 mds_command: new command op to 9765 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 --> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558479991eb0 con 0x558479a7ab70 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.177+0000 7f87f5000640 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a7ab70 0x558479a7af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87f5000640 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a7ab70 0x558479a7af50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68abc59827c5998c server_cookie=5cb28f2c2ae28e0c in_seq=0 out_seq=0 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87d6000640 10 client.9996 ms_handle_connect on v2:172.21.15.165:6836/2515540447 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87d6000640 1 -- 172.21.15.165:0/1933295685 <== mds.0 v2:172.21.15.165:6836/2515540447 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x558479991eb0 con 0x558479a7ab70 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87d6000640 10 client.9996 handle_command_reply: tid=0 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 client.9996 shutdown 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 msgr2=0x558479ad7030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479ad7030 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f87e802a770 tx=0x7f87e8039000 comp rx=0 tx=0).stop 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 shutdown_connections 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x558479a6f5c0 0x558479ad7030 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.165:6836/2515540447,v1:172.21.15.165:6837/2515540447] conn(0x558479a7ab70 0x558479a7af50 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x558479a4ecd0 0x558479a4d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 --2- 172.21.15.165:0/1933295685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x558479a4b430 0x558479a4cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87d6000640 0 client.9996 ms_handle_reset on v2:172.21.15.165:6836/2515540447 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 >> 172.21.15.165:0/1933295685 conn(0x558479a26950 msgr2=0x558479a4a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 shutdown_connections 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 1 -- 172.21.15.165:0/1933295685 wait complete. 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.178+0000 7f87fa869740 20 client.9996 trim_cache size 0 max 16384 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:17:29.185 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 9765, 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 45.630478781999997, 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 104, 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 148, 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 148, 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 52.074174841000001 2024-04-25T10:17:29.186 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:17:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87d006b4d0 msgr2=0x7f87d006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87d006b4d0 0x7f87d006d990 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f87dc000f60 tx=0x7f87dc007a00 comp rx=0 tx=0).stop 2024-04-25T10:17:29.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 msgr2=0x7f87f01449f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 0x7f87f01449f0 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f87e400d5f0 tx=0x7f87e400dab0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 shutdown_connections 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f87d006b4d0 0x7f87d006d990 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87f015b710 0x7f87f0071980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87f00ac8b0 0x7f87f0071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 --2- 172.21.15.165:0/1358480883 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87f006ec50 0x7f87f01449f0 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 >> 172.21.15.165:0/1358480883 conn(0x7f87f006abe0 msgr2=0x7f87f015c120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 shutdown_connections 2024-04-25T10:17:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.179+0000 7f87f7a00640 1 -- 172.21.15.165:0/1358480883 wait complete. 2024-04-25T10:17:29.200 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub start abs_test_path recursive 2024-04-25T10:17:29.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.402+0000 7ffae9400640 1 Processor -- start 2024-04-25T10:17:29.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.402+0000 7ffae9400640 1 -- start start 2024-04-25T10:17:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.403+0000 7ffae9400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffae406e860 0x7ffae40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.403+0000 7ffae9400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae413e640 con 0x7ffae40b0730 2024-04-25T10:17:29.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.403+0000 7ffae9400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae406d390 con 0x7ffae413d9f0 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.403+0000 7ffae9400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae406d670 con 0x7ffae406e860 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.403+0000 7ffae3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ffae40b0730 0x7ffae406e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41244/0 (socket says 172.21.15.165:41244) 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.403+0000 7ffae3400640 1 -- 172.21.15.165:0/2469070087 learned_addr learned my addr 172.21.15.165:0/2469070087 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2173686431 0 0) 0x7ffae413e640 con 0x7ffae40b0730 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffac4003520 con 0x7ffae40b0730 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae8a00640 1 --2- 172.21.15.165:0/2469070087 >> v2:172.21.15.172:3300/0 conn(0x7ffae406e860 0x7ffae40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1759244688 0 0) 0x7ffac4003520 con 0x7ffae40b0730 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 >> v2:172.21.15.172:3300/0 conn(0x7ffae406e860 msgr2=0x7ffae40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 --2- 172.21.15.165:0/2469070087 >> v2:172.21.15.172:3300/0 conn(0x7ffae406e860 0x7ffae40b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 >> v1:172.21.15.172:6789/0 conn(0x7ffae413d9f0 legacy=0x7ffae413df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae406d950 con 0x7ffae40b0730 2024-04-25T10:17:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ffacc0019f0 con 0x7ffae40b0730 2024-04-25T10:17:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 >> v1:172.21.15.165:6789/0 conn(0x7ffae40b0730 legacy=0x7ffae406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac4003a00 0x7ffac4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffac4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac40053c0 0x7ffac400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae406d670 con 0x7ffac4004800 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffac400c280 con 0x7ffac40053c0 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.404+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffac400c480 con 0x7ffac4003a00 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae3400640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffac4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae3400640 1 -- 172.21.15.165:0/2469070087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac4003a00 msgr2=0x7ffac4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae3400640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffac4003a00 0x7ffac4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae3400640 1 -- 172.21.15.165:0/2469070087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac40053c0 msgr2=0x7ffac400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae3400640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffac40053c0 0x7ffac400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae3400640 1 -- 172.21.15.165:0/2469070087 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4003520 con 0x7ffac4004800 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.405+0000 7ffae8a00640 1 -- 172.21.15.165:0/2469070087 reap_dead start 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.406+0000 7ffae3400640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffac4004c80 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00a7a0 tx=0x7ffac4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679c5a26f555b6cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.406+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc00a3f0 con 0x7ffac4004800 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.406+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffacc02f070 con 0x7ffac4004800 2024-04-25T10:17:29.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.406+0000 7ffae2a00640 1 -- 172.21.15.165:0/2469070087 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc02bc60 con 0x7ffac4004800 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 -- 172.21.15.165:0/2469070087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 msgr2=0x7ffac4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffac4004c80 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00a7a0 tx=0x7ffac4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 -- 172.21.15.165:0/2469070087 shutdown_connections 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 --2- 172.21.15.165:0/2469070087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffac4004c80 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 -- 172.21.15.165:0/2469070087 >> 172.21.15.165:0/2469070087 conn(0x7ffae4061cc0 msgr2=0x7ffae406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 -- 172.21.15.165:0/2469070087 shutdown_connections 2024-04-25T10:17:29.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.407+0000 7ffae9400640 1 -- 172.21.15.165:0/2469070087 wait complete. 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.408+0000 7ffae9400640 1 Processor -- start 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 -- start start 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffae4138190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae4151f40 0x7ffae4153580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffae4153ac0 0x7ffae41ef3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b0d40 con 0x7ffac4004800 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b0bc0 con 0x7ffae4151f40 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.409+0000 7ffae9400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40b0ec0 con 0x7ffae4153ac0 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffae4138190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffae4138190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35776/0 (socket says 172.21.15.165:35776) 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 -- 172.21.15.165:0/211619648 learned_addr learned my addr 172.21.15.165:0/211619648 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 -- 172.21.15.165:0/211619648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffae4153ac0 msgr2=0x7ffae41ef3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 --2- 172.21.15.165:0/211619648 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffae4153ac0 0x7ffae41ef3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 -- 172.21.15.165:0/211619648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae4151f40 msgr2=0x7ffae4153580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 --2- 172.21.15.165:0/211619648 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffae4151f40 0x7ffae4153580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 -- 172.21.15.165:0/211619648 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae41524e0 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffae3e00640 1 --2- 172.21.15.165:0/211619648 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffac4004800 0x7ffae4138190 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7ffad400c480 tx=0x7ffad400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.410+0000 7ffadbe00640 1 -- 172.21.15.165:0/211619648 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4016030 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.411+0000 7ffadbe00640 1 -- 172.21.15.165:0/211619648 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad4007020 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.411+0000 7ffadbe00640 1 -- 172.21.15.165:0/211619648 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4007450 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.411+0000 7ffae9400640 1 -- 172.21.15.165:0/211619648 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffae41527c0 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.411+0000 7ffae9400640 1 -- 172.21.15.165:0/211619648 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae4152d80 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.413+0000 7ffadbe00640 1 -- 172.21.15.165:0/211619648 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffad4019070 con 0x7ffac4004800 2024-04-25T10:17:29.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.414+0000 7ffadbe00640 1 --2- 172.21.15.165:0/211619648 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffac006b4d0 0x7ffac006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.414+0000 7ffadbe00640 1 -- 172.21.15.165:0/211619648 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffad4089ff0 con 0x7ffac4004800 2024-04-25T10:17:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.415+0000 7ffae3400640 1 --2- 172.21.15.165:0/211619648 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffac006b4d0 0x7ffac006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.415+0000 7ffaed7dc740 1 Processor -- start 2024-04-25T10:17:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.415+0000 7ffaed7dc740 1 -- start start 2024-04-25T10:17:29.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.415+0000 7ffae3400640 1 --2- 172.21.15.165:0/211619648 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffac006b4d0 0x7ffac006d990 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7ffacc033000 tx=0x7ffacc007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a09ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a04760 0x557538a086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a09060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753896d880 con 0x557538a08c20 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538916d60 con 0x557538a09ae0 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557538969920 con 0x557538a04760 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffae8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a09060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffae8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a09060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35784/0 (socket says 172.21.15.165:35784) 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffae8a00640 1 -- 172.21.15.165:0/1652726021 learned_addr learned my addr 172.21.15.165:0/1652726021 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffae3e00640 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a09ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.416+0000 7ffae3400640 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a04760 0x557538a086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffae8a00640 1 -- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a04760 msgr2=0x557538a086e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffae8a00640 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a04760 0x557538a086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffae8a00640 1 -- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 msgr2=0x557538a09ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffae8a00640 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a09ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffae8a00640 1 -- 172.21.15.165:0/1652726021 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55753874e7e0 con 0x557538a08c20 2024-04-25T10:17:29.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffae8a00640 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a09060 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7ffae40b1460 tx=0x7ffadc0078b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9153dc606c057602 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffad3e00640 1 -- 172.21.15.165:0/1652726021 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc018070 con 0x557538a08c20 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffad3e00640 1 -- 172.21.15.165:0/1652726021 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc004240 con 0x557538a08c20 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.417+0000 7ffad3e00640 1 -- 172.21.15.165:0/1652726021 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc003610 con 0x557538a08c20 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 -- 172.21.15.165:0/1652726021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 msgr2=0x557538a09060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a09060 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7ffae40b1460 tx=0x7ffadc0078b0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 -- 172.21.15.165:0/1652726021 shutdown_connections 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a09060 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a04760 0x557538a086e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/1652726021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a09ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 -- 172.21.15.165:0/1652726021 >> 172.21.15.165:0/1652726021 conn(0x557538a06be0 msgr2=0x557538a4f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 -- 172.21.15.165:0/1652726021 shutdown_connections 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.418+0000 7ffaed7dc740 1 -- 172.21.15.165:0/1652726021 wait complete. 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.419+0000 7ffaed7dc740 1 -- start start 2024-04-25T10:17:29.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.419+0000 7ffaed7dc740 1 Processor -- start 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a08c20 0x557538a24f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 0x557538a24260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a247a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538a35670 con 0x557538a09ae0 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55753890c3c0 con 0x557538a08c20 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557538a34af0 con 0x557538a21ee0 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffaed7dc740 10 client.0 _ll_register_callbacks cb 0x557538918c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 0x557538a24260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 0x557538a24260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35792/0 (socket says 172.21.15.165:35792) 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 -- 172.21.15.165:0/185992728 learned_addr learned my addr 172.21.15.165:0/185992728 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae8a00640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a247a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3e00640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a08c20 0x557538a24f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 -- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 msgr2=0x557538a247a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a247a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 -- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a08c20 msgr2=0x557538a24f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a08c20 0x557538a24f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae3400640 1 -- 172.21.15.165:0/185992728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575387aef70 con 0x557538a09ae0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.420+0000 7ffae8a00640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffae3400640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 0x557538a24260 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7ffacc02b470 tx=0x7ffacc02d070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac2dd0a1373c5d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffad0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffad0c00640 1 -- 172.21.15.165:0/185992728 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffacc02be60 con 0x557538a09ae0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffad0c00640 1 -- 172.21.15.165:0/185992728 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffacc035950 con 0x557538a09ae0 2024-04-25T10:17:29.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffadc004240 con 0x557538a09ae0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffad0c00640 1 -- 172.21.15.165:0/185992728 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffacc035b50 con 0x557538a09ae0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffaed7dc740 10 client.10008 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5575386420d0 con 0x557538a09ae0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.421+0000 7ffad0c00640 1 -- 172.21.15.165:0/185992728 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc004240 con 0x557538a09ae0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffad0c00640 1 -- 172.21.15.165:0/185992728 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7ffacc056020 con 0x557538a09ae0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffaed7dc740 10 client.10008 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffaed7dc740 10 client.10008 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffaed7dc740 20 client.10008 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a63430 0x557538a63810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffaed7dc740 4 client.10008 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55753897adc0 con 0x557538a63430 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffae3e00640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a63430 0x557538a63810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffae3e00640 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a63430 0x557538a63810 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=1a994eb2a396b5ae server_cookie=6374ea15939658fa in_seq=0 out_seq=0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.422+0000 7ffad0c00640 10 client.10008 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffad0c00640 1 -- 172.21.15.165:0/185992728 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55753897adc0 con 0x557538a63430 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffad0c00640 10 client.10008 handle_command_reply: tid=0 2024-04-25T10:17:29.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 client.10008 shutdown 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 msgr2=0x557538a24260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 0x557538a24260 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7ffacc02b470 tx=0x7ffacc02d070 comp rx=0 tx=0).stop 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 shutdown_connections 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a63430 0x557538a63810 crc :-1 s=READY pgs=10 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a247a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a09ae0 0x557538a24260 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/185992728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a08c20 0x557538a24f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffad0c00640 0 client.10008 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.423+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 >> 172.21.15.165:0/185992728 conn(0x557538a04760 msgr2=0x557538a03e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.424+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 shutdown_connections 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.424+0000 7ffaed7dc740 1 -- 172.21.15.165:0/185992728 wait complete. 2024-04-25T10:17:29.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.424+0000 7ffaed7dc740 20 client.10008 trim_cache size 0 max 16384 2024-04-25T10:17:29.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.439+0000 7ffaed7dc740 1 Processor -- start 2024-04-25T10:17:29.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.439+0000 7ffaed7dc740 1 -- start start 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x5575389ff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a2b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41fc2b0 con 0x557538a08c20 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae40cd420 con 0x557538a09ae0 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae41fae90 con 0x557538a21ee0 2024-04-25T10:17:29.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x5575389ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x5575389ff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35796/0 (socket says 172.21.15.165:35796) 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.440+0000 7ffae3e00640 1 -- 172.21.15.165:0/2885152857 learned_addr learned my addr 172.21.15.165:0/2885152857 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffae3e00640 1 -- 172.21.15.165:0/2885152857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 msgr2=0x557538a2b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffae3e00640 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a2b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffae3e00640 1 -- 172.21.15.165:0/2885152857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 msgr2=0x557538a24d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffae3e00640 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffae3e00640 1 -- 172.21.15.165:0/2885152857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557538a45c70 con 0x557538a08c20 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffae3e00640 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x5575389ff840 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ffad40582e0 tx=0x7ffad4003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94d24d15223e6b44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffad3400640 1 -- 172.21.15.165:0/2885152857 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad40075f0 con 0x557538a08c20 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffad3400640 1 -- 172.21.15.165:0/2885152857 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad4070100 con 0x557538a08c20 2024-04-25T10:17:29.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.441+0000 7ffad3400640 1 -- 172.21.15.165:0/2885152857 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4054ca0 con 0x557538a08c20 2024-04-25T10:17:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 -- 172.21.15.165:0/2885152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 msgr2=0x5575389ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x5575389ff840 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ffad40582e0 tx=0x7ffad4003950 comp rx=0 tx=0).stop 2024-04-25T10:17:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 -- 172.21.15.165:0/2885152857 shutdown_connections 2024-04-25T10:17:29.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a21ee0 0x557538a2b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a09ae0 0x557538a24d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/2885152857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x5575389ff840 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 -- 172.21.15.165:0/2885152857 >> 172.21.15.165:0/2885152857 conn(0x557538a04760 msgr2=0x557538a5fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 -- 172.21.15.165:0/2885152857 shutdown_connections 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.442+0000 7ffaed7dc740 1 -- 172.21.15.165:0/2885152857 wait complete. 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.443+0000 7ffaed7dc740 1 -- start start 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.443+0000 7ffaed7dc740 1 Processor -- start 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a4e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a09ae0 0x557538a2b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a21ee0 0x557538ab6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538a20a30 con 0x557538a08c20 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557538a52b30 con 0x557538a21ee0 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557538a523e0 con 0x557538a09ae0 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffaed7dc740 10 client.0 _ll_register_callbacks cb 0x7ffac40036f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a4e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a4e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35812/0 (socket says 172.21.15.165:35812) 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 -- 172.21.15.165:0/874556302 learned_addr learned my addr 172.21.15.165:0/874556302 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 -- 172.21.15.165:0/874556302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a09ae0 msgr2=0x557538a2b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a09ae0 0x557538a2b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 -- 172.21.15.165:0/874556302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a21ee0 msgr2=0x557538ab6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a21ee0 0x557538ab6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.444+0000 7ffae3e00640 1 -- 172.21.15.165:0/874556302 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557538a2bd40 con 0x557538a08c20 2024-04-25T10:17:29.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffae3e00640 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a4e890 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ffad400ce40 tx=0x7ffad401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6450727529d4a3f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffad0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffad0c00640 1 -- 172.21.15.165:0/874556302 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad4054ca0 con 0x557538a08c20 2024-04-25T10:17:29.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffad0c00640 1 -- 172.21.15.165:0/874556302 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ffad4022ba0 con 0x557538a08c20 2024-04-25T10:17:29.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffad4070100 con 0x557538a08c20 2024-04-25T10:17:29.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffad0c00640 1 -- 172.21.15.165:0/874556302 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffad4054e40 con 0x557538a08c20 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffaed7dc740 10 client.10014 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x557538a2a790 con 0x557538a08c20 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffad0c00640 1 -- 172.21.15.165:0/874556302 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad4070100 con 0x557538a08c20 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.445+0000 7ffad0c00640 1 -- 172.21.15.165:0/874556302 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7ffad4023060 con 0x557538a08c20 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffaed7dc740 10 client.10014 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffaed7dc740 10 client.10014 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffaed7dc740 20 client.10014 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a037b0 0x557538a03b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffaed7dc740 4 client.10014 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}] 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "scrub start", "path": "abs_test_path", "scrubops": ["recursive"]}) v1 -- 0x557538a519f0 con 0x557538a037b0 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffae3400640 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a037b0 0x557538a03b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffae3400640 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a037b0 0x557538a03b90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=332b518af5406af7 server_cookie=bdb0f55962b68823 in_seq=0 out_seq=0 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.446+0000 7ffad0c00640 10 client.10014 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffad0c00640 1 -- 172.21.15.165:0/874556302 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x557538a519f0 con 0x557538a037b0 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffad0c00640 10 client.10014 handle_command_reply: tid=0 2024-04-25T10:17:29.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 client.10014 shutdown 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 msgr2=0x557538a4e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a4e890 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7ffad400ce40 tx=0x7ffad401d040 comp rx=0 tx=0).stop 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 shutdown_connections 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x557538a037b0 0x557538a03b90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557538a21ee0 0x557538ab6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557538a09ae0 0x557538a2b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 --2- 172.21.15.165:0/874556302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557538a08c20 0x557538a4e890 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffad0c00640 0 client.10014 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 >> 172.21.15.165:0/874556302 conn(0x557538a04760 msgr2=0x557538a24c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 shutdown_connections 2024-04-25T10:17:29.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 1 -- 172.21.15.165:0/874556302 wait complete. 2024-04-25T10:17:29.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.447+0000 7ffaed7dc740 20 client.10014 trim_cache size 0 max 16384 2024-04-25T10:17:29.455 INFO:teuthology.orchestra.run.smithi165.stderr:Error EINVAL: MDS is not active 2024-04-25T10:17:29.458 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:17:29.458 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub pause 2024-04-25T10:17:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.633+0000 7feedea00640 1 Processor -- start 2024-04-25T10:17:29.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.634+0000 7feedea00640 1 -- start start 2024-04-25T10:17:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.635+0000 7feedea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7feed80ced80 0x7feed80d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.635+0000 7feedea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feed8161710 0x7feed80d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.635+0000 7feedea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed815e000 con 0x7feed815dc20 2024-04-25T10:17:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.635+0000 7feedea00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80d1de0 con 0x7feed80ced80 2024-04-25T10:17:29.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.635+0000 7feedea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed8161c60 con 0x7feed8161710 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.635+0000 7feed7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7feed80ced80 0x7feed80d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feed7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7feed80ced80 0x7feed80d22e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35826/0 (socket says 172.21.15.165:35826) 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feed7400640 1 -- 172.21.15.165:0/886346922 learned_addr learned my addr 172.21.15.165:0/886346922 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feedca00640 1 --2- 172.21.15.165:0/886346922 >> v2:172.21.15.172:3300/0 conn(0x7feed8161710 0x7feed80d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feed7400640 1 -- 172.21.15.165:0/886346922 >> v2:172.21.15.172:3300/0 conn(0x7feed8161710 msgr2=0x7feed80d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feed7400640 1 --2- 172.21.15.165:0/886346922 >> v2:172.21.15.172:3300/0 conn(0x7feed8161710 0x7feed80d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feed7400640 1 -- 172.21.15.165:0/886346922 >> v1:172.21.15.172:6789/0 conn(0x7feed815dc20 legacy=0x7feed80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.636+0000 7feed7400640 1 -- 172.21.15.165:0/886346922 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8161f40 con 0x7feed80ced80 2024-04-25T10:17:29.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.637+0000 7feed7400640 1 --2- 172.21.15.165:0/886346922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed80d22e0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7feed80b6570 tx=0x7feec402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e67c394e3163619 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.638+0000 7feed6a00640 1 -- 172.21.15.165:0/886346922 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4037070 con 0x7feed80ced80 2024-04-25T10:17:29.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.638+0000 7feed6a00640 1 -- 172.21.15.165:0/886346922 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4032440 con 0x7feed80ced80 2024-04-25T10:17:29.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.638+0000 7feed6a00640 1 -- 172.21.15.165:0/886346922 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4032810 con 0x7feed80ced80 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.638+0000 7feedea00640 1 -- 172.21.15.165:0/886346922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 msgr2=0x7feed80d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.638+0000 7feedea00640 1 --2- 172.21.15.165:0/886346922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed80d22e0 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7feed80b6570 tx=0x7feec402a270 comp rx=0 tx=0).stop 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.639+0000 7feedea00640 1 -- 172.21.15.165:0/886346922 shutdown_connections 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.639+0000 7feedea00640 1 --2- 172.21.15.165:0/886346922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed80d22e0 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.639+0000 7feedea00640 1 --2- 172.21.15.165:0/886346922 >> v2:172.21.15.172:3300/0 conn(0x7feed8161710 0x7feed80d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.639+0000 7feedea00640 1 -- 172.21.15.165:0/886346922 >> 172.21.15.165:0/886346922 conn(0x7feed80d0420 msgr2=0x7feed80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.639+0000 7feedea00640 1 -- 172.21.15.165:0/886346922 shutdown_connections 2024-04-25T10:17:29.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.639+0000 7feedea00640 1 -- 172.21.15.165:0/886346922 wait complete. 2024-04-25T10:17:29.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.640+0000 7feedea00640 1 Processor -- start 2024-04-25T10:17:29.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.640+0000 7feedea00640 1 -- start start 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.641+0000 7feedea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed8148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.641+0000 7feedea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed815dc20 0x7feed80c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.641+0000 7feedea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feed8161710 0x7feed81587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.641+0000 7feedea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80b5e50 con 0x7feed80ced80 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.641+0000 7feedea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8162500 con 0x7feed815dc20 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.641+0000 7feedea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80b5fd0 con 0x7feed8161710 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed8148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed8148050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35832/0 (socket says 172.21.15.165:35832) 2024-04-25T10:17:29.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 -- 172.21.15.165:0/87518661 learned_addr learned my addr 172.21.15.165:0/87518661 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 -- 172.21.15.165:0/87518661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feed8161710 msgr2=0x7feed81587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 --2- 172.21.15.165:0/87518661 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feed8161710 0x7feed81587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 -- 172.21.15.165:0/87518661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed815dc20 msgr2=0x7feed80c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 --2- 172.21.15.165:0/87518661 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feed815dc20 0x7feed80c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 -- 172.21.15.165:0/87518661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed815a650 con 0x7feed80ced80 2024-04-25T10:17:29.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.642+0000 7feed7e00640 1 --2- 172.21.15.165:0/87518661 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feed80ced80 0x7feed8148050 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7feec800c3f0 tx=0x7feec800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.643+0000 7feecfe00640 1 -- 172.21.15.165:0/87518661 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec80052d0 con 0x7feed80ced80 2024-04-25T10:17:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.643+0000 7feecfe00640 1 -- 172.21.15.165:0/87518661 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec8005490 con 0x7feed80ced80 2024-04-25T10:17:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.643+0000 7feecfe00640 1 -- 172.21.15.165:0/87518661 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8007260 con 0x7feed80ced80 2024-04-25T10:17:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.643+0000 7feedea00640 1 -- 172.21.15.165:0/87518661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed815a930 con 0x7feed80ced80 2024-04-25T10:17:29.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.643+0000 7feedea00640 1 -- 172.21.15.165:0/87518661 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed815aef0 con 0x7feed80ced80 2024-04-25T10:17:29.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.645+0000 7feecfe00640 1 -- 172.21.15.165:0/87518661 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feec8007400 con 0x7feed80ced80 2024-04-25T10:17:29.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.646+0000 7feecfe00640 1 --2- 172.21.15.165:0/87518661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feeb806b4d0 0x7feeb806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.647+0000 7feecfe00640 1 -- 172.21.15.165:0/87518661 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feec8089880 con 0x7feed80ced80 2024-04-25T10:17:29.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.647+0000 7feed7400640 1 --2- 172.21.15.165:0/87518661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feeb806b4d0 0x7feeb806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.647+0000 7feee1986740 1 Processor -- start 2024-04-25T10:17:29.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.647+0000 7feee1986740 1 -- start start 2024-04-25T10:17:29.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.647+0000 7feed7400640 1 --2- 172.21.15.165:0/87518661 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feeb806b4d0 0x7feeb806d990 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7feec4034000 tx=0x7feec402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feee1986740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631de3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631de920 0x55f3631b3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feee1986740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3631176c0 con 0x55f3631fbf40 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3630c0d20 con 0x55f3631df060 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f363113760 con 0x55f3631de920 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feed7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feed7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35834/0 (socket says 172.21.15.165:35834) 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.648+0000 7feed7e00640 1 -- 172.21.15.165:0/2724572128 learned_addr learned my addr 172.21.15.165:0/2724572128 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feedca00640 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631de920 0x55f3631b3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7400640 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631de3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7e00640 1 -- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631de920 msgr2=0x55f3631b3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7e00640 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631de920 0x55f3631b3730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7e00640 1 -- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 msgr2=0x55f3631de3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7e00640 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631de3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7e00640 1 -- 172.21.15.165:0/2724572128 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f362ef6500 con 0x55f3631fbf40 2024-04-25T10:17:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7400640 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631de3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:29.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feed7e00640 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631ae8d0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7feec8087b50 tx=0x7feec800a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5364d784b570b796 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feec3e00640 1 -- 172.21.15.165:0/2724572128 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec8056160 con 0x55f3631fbf40 2024-04-25T10:17:29.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feec3e00640 1 -- 172.21.15.165:0/2724572128 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec80544d0 con 0x55f3631fbf40 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.649+0000 7feec3e00640 1 -- 172.21.15.165:0/2724572128 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec805e270 con 0x55f3631fbf40 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 -- 172.21.15.165:0/2724572128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 msgr2=0x55f3631ae8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631ae8d0 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7feec8087b50 tx=0x7feec800a000 comp rx=0 tx=0).stop 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 -- 172.21.15.165:0/2724572128 shutdown_connections 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631de920 0x55f3631b3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631de3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 --2- 172.21.15.165:0/2724572128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631ae8d0 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 -- 172.21.15.165:0/2724572128 >> 172.21.15.165:0/2724572128 conn(0x55f3631b0b40 msgr2=0x55f3631f93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 -- 172.21.15.165:0/2724572128 shutdown_connections 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.650+0000 7feee1986740 1 -- 172.21.15.165:0/2724572128 wait complete. 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.651+0000 7feee1986740 1 -- start start 2024-04-25T10:17:29.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.651+0000 7feee1986740 1 Processor -- start 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631f0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631fbf40 0x55f3631f1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3631b2cf0 con 0x55f3631e1a80 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3630d40d0 con 0x55f3631df060 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3631b20e0 con 0x55f3631fbf40 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feee1986740 10 client.0 _ll_register_callbacks cb 0x55f36310f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feedca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feedca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631f24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35840/0 (socket says 172.21.15.165:35840) 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.652+0000 7feedca00640 1 -- 172.21.15.165:0/2763741621 learned_addr learned my addr 172.21.15.165:0/2763741621 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feed7400640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631fbf40 0x55f3631f1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feed7e00640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631f0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feedca00640 1 -- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631fbf40 msgr2=0x55f3631f1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feedca00640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631fbf40 0x55f3631f1fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feedca00640 1 -- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 msgr2=0x55f3631f0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feedca00640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631f0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feedca00640 1 -- 172.21.15.165:0/2763741621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f362f56c90 con 0x55f3631e1a80 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feedca00640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631f24e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7feed0007a00 tx=0x7feed0007ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2d6958ae09092c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feec0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feec0c00640 1 -- 172.21.15.165:0/2763741621 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0018020 con 0x55f3631e1a80 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feec0c00640 1 -- 172.21.15.165:0/2763741621 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feed000fba0 con 0x55f3631e1a80 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.653+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feec80544d0 con 0x55f3631e1a80 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feec0c00640 1 -- 172.21.15.165:0/2763741621 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7feed0022050 con 0x55f3631e1a80 2024-04-25T10:17:29.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 10 client.10026 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55f362debe20 con 0x55f3631e1a80 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feec0c00640 1 -- 172.21.15.165:0/2763741621 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec80544d0 con 0x55f3631e1a80 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feec0c00640 1 -- 172.21.15.165:0/2763741621 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7feed0025020 con 0x55f3631e1a80 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 10 client.10026 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 10 client.10026 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 20 client.10026 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f3631e07a0 0x55f36320d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 4 client.10026 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f363124c00 con 0x55f3631e07a0 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.654+0000 7feed7e00640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f3631e07a0 0x55f36320d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.655+0000 7feed7e00640 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f3631e07a0 0x55f36320d660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=21cdb1dc8779e536 server_cookie=c537a00078f7dc6e in_seq=0 out_seq=0 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.655+0000 7feec0c00640 10 client.10026 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.655+0000 7feec0c00640 1 -- 172.21.15.165:0/2763741621 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55f363124c00 con 0x55f3631e07a0 2024-04-25T10:17:29.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.655+0000 7feec0c00640 10 client.10026 handle_command_reply: tid=0 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 client.10026 shutdown 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 msgr2=0x55f3631f24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631f24e0 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7feed0007a00 tx=0x7feed0007ec0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 shutdown_connections 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f3631e07a0 0x55f36320d660 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631f24e0 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631fbf40 0x55f3631f1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 --2- 172.21.15.165:0/2763741621 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631df060 0x55f3631f0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 >> 172.21.15.165:0/2763741621 conn(0x55f3631de920 msgr2=0x55f3631fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feec0c00640 0 client.10026 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 shutdown_connections 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 1 -- 172.21.15.165:0/2763741621 wait complete. 2024-04-25T10:17:29.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.656+0000 7feee1986740 20 client.10026 trim_cache size 0 max 16384 2024-04-25T10:17:29.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.673+0000 7feee1986740 1 Processor -- start 2024-04-25T10:17:29.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.673+0000 7feee1986740 1 -- start start 2024-04-25T10:17:29.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.674+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f3631b3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.674+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631e1a80 0x55f3631b3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.674+0000 7feee1986740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631f0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.674+0000 7feee1986740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed8200590 con 0x55f3631fbf40 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.674+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c9ac0 con 0x55f3631e1a80 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.674+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed81ff200 con 0x55f3631df060 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631f0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631f0d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35856/0 (socket says 172.21.15.165:35856) 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 -- 172.21.15.165:0/2050172218 learned_addr learned my addr 172.21.15.165:0/2050172218 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feed7e00640 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f3631b3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feed7400640 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631e1a80 0x55f3631b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 -- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 msgr2=0x55f3631b3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f3631b3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 -- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631e1a80 msgr2=0x55f3631b3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631e1a80 0x55f3631b3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 -- 172.21.15.165:0/2050172218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f3631e0ec0 con 0x55f3631fbf40 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.675+0000 7feedca00640 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631f0d80 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7feed0007f00 tx=0x7feed0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db486018ec1c03a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.676+0000 7feec3400640 1 -- 172.21.15.165:0/2050172218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0018040 con 0x55f3631fbf40 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.676+0000 7feec3400640 1 -- 172.21.15.165:0/2050172218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed0009110 con 0x55f3631fbf40 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.676+0000 7feec3400640 1 -- 172.21.15.165:0/2050172218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed000fbe0 con 0x55f3631fbf40 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.676+0000 7feee1986740 1 -- 172.21.15.165:0/2050172218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 msgr2=0x55f3631f0d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.676+0000 7feee1986740 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631f0d80 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7feed0007f00 tx=0x7feed0014000 comp rx=0 tx=0).stop 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 -- 172.21.15.165:0/2050172218 shutdown_connections 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631fbf40 0x55f3631f0d80 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631e1a80 0x55f3631b3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 --2- 172.21.15.165:0/2050172218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f3631b3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 -- 172.21.15.165:0/2050172218 >> 172.21.15.165:0/2050172218 conn(0x55f3631de920 msgr2=0x55f3631d9640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 -- 172.21.15.165:0/2050172218 shutdown_connections 2024-04-25T10:17:29.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 -- 172.21.15.165:0/2050172218 wait complete. 2024-04-25T10:17:29.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.677+0000 7feee1986740 1 -- start start 2024-04-25T10:17:29.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.678+0000 7feee1986740 1 Processor -- start 2024-04-25T10:17:29.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.678+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f363263e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.678+0000 7feee1986740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631d2740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feee1986740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631fbf40 0x55f3631d2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feee1986740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3631f4430 con 0x55f3631e1a80 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f3631dfad0 con 0x55f3631fbf40 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feee1986740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f3631e0bf0 con 0x55f3631df060 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feee1986740 10 client.0 _ll_register_callbacks cb 0x7feec8057c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631d2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631d2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35866/0 (socket says 172.21.15.165:35866) 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 -- 172.21.15.165:0/1725640487 learned_addr learned my addr 172.21.15.165:0/1725640487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 -- 172.21.15.165:0/1725640487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 msgr2=0x55f363263e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f363263e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 -- 172.21.15.165:0/1725640487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631fbf40 msgr2=0x55f3631d2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631fbf40 0x55f3631d2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.679+0000 7feed7400640 1 -- 172.21.15.165:0/1725640487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f3631dabe0 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feed7400640 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631d2740 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7feec4006da0 tx=0x7feec4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e2da86b5685134 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feec0c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feec0c00640 1 -- 172.21.15.165:0/1725640487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec404d070 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feec0c00640 1 -- 172.21.15.165:0/1725640487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7feec403d640 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feed0009110 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feec0c00640 1 -- 172.21.15.165:0/1725640487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7feec4037040 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 10 client.10032 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x55f3631f9fe0 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feec0c00640 1 -- 172.21.15.165:0/1725640487 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec4058070 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feec0c00640 1 -- 172.21.15.165:0/1725640487 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7feec4046040 con 0x55f3631e1a80 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 10 client.10032 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 10 client.10032 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 20 client.10032 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.680+0000 7feee1986740 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f36320c590 0x55f3631f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.681+0000 7feee1986740 4 client.10032 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "scrub pause"}] 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.681+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "scrub pause"}) v1 -- 0x55f36311ef50 con 0x55f36320c590 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.681+0000 7feed7e00640 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f36320c590 0x55f3631f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.681+0000 7feed7e00640 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f36320c590 0x55f3631f8f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=e80190a65d7c8219 server_cookie=982073dc14ad792e in_seq=0 out_seq=0 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.681+0000 7feec0c00640 10 client.10032 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feec0c00640 1 -- 172.21.15.165:0/1725640487 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x55f36311ef50 con 0x55f36320c590 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feec0c00640 10 client.10032 handle_command_reply: tid=0 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 client.10032 shutdown 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 msgr2=0x55f3631d2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631d2740 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7feec4006da0 tx=0x7feec4004730 comp rx=0 tx=0).stop 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 shutdown_connections 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x55f36320c590 0x55f3631f8f60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55f3631fbf40 0x55f3631d2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55f3631e1a80 0x55f3631d2740 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 --2- 172.21.15.165:0/1725640487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55f3631df060 0x55f363263e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feec0c00640 0 client.10032 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 >> 172.21.15.165:0/1725640487 conn(0x55f3631de920 msgr2=0x55f3631d3910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 shutdown_connections 2024-04-25T10:17:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 1 -- 172.21.15.165:0/1725640487 wait complete. 2024-04-25T10:17:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.682+0000 7feee1986740 20 client.10032 trim_cache size 0 max 16384 2024-04-25T10:17:29.690 INFO:teuthology.orchestra.run.smithi165.stderr:Error EINVAL: MDS is not active 2024-04-25T10:17:29.693 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:17:29.694 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub abort 2024-04-25T10:17:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.882+0000 7f8779a00640 1 Processor -- start 2024-04-25T10:17:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.882+0000 7f8779a00640 1 -- start start 2024-04-25T10:17:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.883+0000 7f8779a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f877415dd30 0x7f87740cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.883+0000 7f8779a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877406bff0 0x7f87740b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.883+0000 7f8779a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877415e110 con 0x7f87740b5970 2024-04-25T10:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.883+0000 7f8779a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877406d000 con 0x7f877415dd30 2024-04-25T10:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.883+0000 7f8779a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740b2fd0 con 0x7f877406bff0 2024-04-25T10:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f877415dd30 0x7f87740cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f877415dd30 0x7f87740cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35868/0 (socket says 172.21.15.165:35868) 2024-04-25T10:17:29.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 -- 172.21.15.165:0/887189554 learned_addr learned my addr 172.21.15.165:0/887189554 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773400640 1 --2- 172.21.15.165:0/887189554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877406bff0 0x7f87740b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 -- 172.21.15.165:0/887189554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877406bff0 msgr2=0x7f87740b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 --2- 172.21.15.165:0/887189554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877406bff0 0x7f87740b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 -- 172.21.15.165:0/887189554 >> v1:172.21.15.172:6789/0 conn(0x7f87740b5970 legacy=0x7f8774161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.884+0000 7f8773e00640 1 -- 172.21.15.165:0/887189554 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877406c480 con 0x7f877415dd30 2024-04-25T10:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.885+0000 7f8773e00640 1 --2- 172.21.15.165:0/887189554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415dd30 0x7f87740cebd0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f8764009870 tx=0x7f876402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28737eeacb555c13 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.885+0000 7f8772a00640 1 -- 172.21.15.165:0/887189554 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764037070 con 0x7f877415dd30 2024-04-25T10:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8772a00640 1 -- 172.21.15.165:0/887189554 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8764032440 con 0x7f877415dd30 2024-04-25T10:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8772a00640 1 -- 172.21.15.165:0/887189554 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764032810 con 0x7f877415dd30 2024-04-25T10:17:29.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 -- 172.21.15.165:0/887189554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415dd30 msgr2=0x7f87740cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 --2- 172.21.15.165:0/887189554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415dd30 0x7f87740cebd0 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f8764009870 tx=0x7f876402a210 comp rx=0 tx=0).stop 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 -- 172.21.15.165:0/887189554 shutdown_connections 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 --2- 172.21.15.165:0/887189554 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877406bff0 0x7f87740b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 --2- 172.21.15.165:0/887189554 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415dd30 0x7f87740cebd0 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 -- 172.21.15.165:0/887189554 >> 172.21.15.165:0/887189554 conn(0x7f87740d0610 msgr2=0x7f87740d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 -- 172.21.15.165:0/887189554 shutdown_connections 2024-04-25T10:17:29.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.886+0000 7f8779a00640 1 -- 172.21.15.165:0/887189554 wait complete. 2024-04-25T10:17:29.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.888+0000 7f8779a00640 1 Processor -- start 2024-04-25T10:17:29.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.888+0000 7f8779a00640 1 -- start start 2024-04-25T10:17:29.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87740b5970 0x7f8774157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877415dd30 0x7f877414de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415a770 0x7f87740b3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877406cd20 con 0x7f877415a770 2024-04-25T10:17:29.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877406ca40 con 0x7f87740b5970 2024-04-25T10:17:29.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87740cf110 con 0x7f877415dd30 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.889+0000 7f8779000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415a770 0x7f87740b3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415a770 0x7f87740b3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35874/0 (socket says 172.21.15.165:35874) 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 -- 172.21.15.165:0/2715720494 learned_addr learned my addr 172.21.15.165:0/2715720494 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8773e00640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87740b5970 0x7f8774157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8773400640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877415dd30 0x7f877414de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 -- 172.21.15.165:0/2715720494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877415dd30 msgr2=0x7f877414de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f877415dd30 0x7f877414de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 -- 172.21.15.165:0/2715720494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87740b5970 msgr2=0x7f8774157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87740b5970 0x7f8774157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 -- 172.21.15.165:0/2715720494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87741f6fa0 con 0x7f877415a770 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.890+0000 7f8779000640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f877415a770 0x7f87740b3c80 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f876c00c780 tx=0x7f876c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.891+0000 7f876be00640 1 -- 172.21.15.165:0/2715720494 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c017070 con 0x7f877415a770 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.891+0000 7f876be00640 1 -- 172.21.15.165:0/2715720494 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c003da0 con 0x7f877415a770 2024-04-25T10:17:29.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.891+0000 7f876be00640 1 -- 172.21.15.165:0/2715720494 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c0052b0 con 0x7f877415a770 2024-04-25T10:17:29.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.891+0000 7f8779a00640 1 -- 172.21.15.165:0/2715720494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87741f7280 con 0x7f877415a770 2024-04-25T10:17:29.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.891+0000 7f8779a00640 1 -- 172.21.15.165:0/2715720494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87741f7840 con 0x7f877415a770 2024-04-25T10:17:29.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.893+0000 7f876be00640 1 -- 172.21.15.165:0/2715720494 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f876c005450 con 0x7f877415a770 2024-04-25T10:17:29.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.894+0000 7f876be00640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f875406b4d0 0x7f875406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.894+0000 7f876be00640 1 -- 172.21.15.165:0/2715720494 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f876c089870 con 0x7f877415a770 2024-04-25T10:17:29.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.894+0000 7f8773e00640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f875406b4d0 0x7f875406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.895+0000 7f8773e00640 1 --2- 172.21.15.165:0/2715720494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f875406b4d0 0x7f875406d990 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f8764034000 tx=0x7f876402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.895+0000 7f877de26740 1 Processor -- start 2024-04-25T10:17:29.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.895+0000 7f877de26740 1 -- start start 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.896+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d94110 0x5598b2d46aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.896+0000 7f877de26740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 0x5598b2d765b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.896+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d76af0 0x5598b2d4b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.896+0000 7f877de26740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b2caf6c0 con 0x5598b2d77230 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.896+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b2c58d20 con 0x5598b2d76af0 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.896+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598b2cab760 con 0x5598b2d94110 2024-04-25T10:17:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 0x5598b2d765b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 0x5598b2d765b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35884/0 (socket says 172.21.15.165:35884) 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 -- 172.21.15.165:0/1419980690 learned_addr learned my addr 172.21.15.165:0/1419980690 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773e00640 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d94110 0x5598b2d46aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8779000640 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d76af0 0x5598b2d4b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 -- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d94110 msgr2=0x5598b2d46aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d94110 0x5598b2d46aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 -- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d76af0 msgr2=0x5598b2d4b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d76af0 0x5598b2d4b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773400640 1 -- 172.21.15.165:0/1419980690 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b2a8e500 con 0x5598b2d77230 2024-04-25T10:17:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.897+0000 7f8773e00640 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d94110 0x5598b2d46aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.898+0000 7f8773400640 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 0x5598b2d765b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f876000b930 tx=0x7f876000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6bf87f3763ef1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.898+0000 7f8768c00640 1 -- 172.21.15.165:0/1419980690 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760018020 con 0x5598b2d77230 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.898+0000 7f8768c00640 1 -- 172.21.15.165:0/1419980690 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876000fa80 con 0x5598b2d77230 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.898+0000 7f8768c00640 1 -- 172.21.15.165:0/1419980690 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876000fe50 con 0x5598b2d77230 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 -- 172.21.15.165:0/1419980690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 msgr2=0x5598b2d765b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 0x5598b2d765b0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f876000b930 tx=0x7f876000bdf0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 -- 172.21.15.165:0/1419980690 shutdown_connections 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d76af0 0x5598b2d4b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d77230 0x5598b2d765b0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 --2- 172.21.15.165:0/1419980690 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d94110 0x5598b2d46aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 -- 172.21.15.165:0/1419980690 >> 172.21.15.165:0/1419980690 conn(0x5598b2d48d10 msgr2=0x5598b2d915b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 -- 172.21.15.165:0/1419980690 shutdown_connections 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 -- 172.21.15.165:0/1419980690 wait complete. 2024-04-25T10:17:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.899+0000 7f877de26740 1 -- start start 2024-04-25T10:17:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.900+0000 7f877de26740 1 Processor -- start 2024-04-25T10:17:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d77720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d93140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d67300 0x5598b2d791d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b2d4aec0 con 0x5598b2d94110 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b2c6c0d0 con 0x5598b2d67300 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598b2d4a2b0 con 0x5598b2d77230 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f877de26740 10 client.0 _ll_register_callbacks cb 0x5598b2ca78b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d93140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d93140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35890/0 (socket says 172.21.15.165:35890) 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 -- 172.21.15.165:0/3425280933 learned_addr learned my addr 172.21.15.165:0/3425280933 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 -- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 msgr2=0x5598b2d77720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773e00640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d77720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d77720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8779000640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d67300 0x5598b2d791d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 -- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d67300 msgr2=0x5598b2d791d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d67300 0x5598b2d791d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.901+0000 7f8773400640 1 -- 172.21.15.165:0/3425280933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598b2aeec90 con 0x5598b2d94110 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f8773400640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d93140 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f8760000c00 tx=0x7f8760004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9329af05de9f0c3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f875d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f875d600640 1 -- 172.21.15.165:0/3425280933 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8760018050 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f875d600640 1 -- 172.21.15.165:0/3425280933 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f876000f880 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f876000fa80 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f875d600640 1 -- 172.21.15.165:0/3425280933 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8760003610 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f877de26740 10 client.10044 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.902+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5598b2983e20 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f875d600640 1 -- 172.21.15.165:0/3425280933 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876000fa80 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f875d600640 1 -- 172.21.15.165:0/3425280933 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f8760025020 con 0x5598b2d94110 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f877de26740 10 client.10044 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f877de26740 10 client.10044 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f877de26740 20 client.10044 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f877de26740 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8f730 0x5598b2d8fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f877de26740 4 client.10044 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598b2cbcc00 con 0x5598b2d8f730 2024-04-25T10:17:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.903+0000 7f8773e00640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8f730 0x5598b2d8fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.904+0000 7f8773e00640 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8f730 0x5598b2d8fb10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=38b0214c03f4416a server_cookie=50237010cc9870b7 in_seq=0 out_seq=0 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.904+0000 7f875d600640 10 client.10044 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f875d600640 1 -- 172.21.15.165:0/3425280933 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5598b2cbcc00 con 0x5598b2d8f730 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f875d600640 10 client.10044 handle_command_reply: tid=0 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 client.10044 shutdown 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 msgr2=0x5598b2d93140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d93140 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f8760000c00 tx=0x7f8760004730 comp rx=0 tx=0).stop 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 shutdown_connections 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8f730 0x5598b2d8fb10 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d67300 0x5598b2d791d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d93140 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 --2- 172.21.15.165:0/3425280933 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d77720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f875d600640 0 client.10044 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 >> 172.21.15.165:0/3425280933 conn(0x5598b2d76af0 msgr2=0x5598b2d915b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 shutdown_connections 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 1 -- 172.21.15.165:0/3425280933 wait complete. 2024-04-25T10:17:29.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.905+0000 7f877de26740 20 client.10044 trim_cache size 0 max 16384 2024-04-25T10:17:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.921+0000 7f877de26740 1 Processor -- start 2024-04-25T10:17:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.921+0000 7f877de26740 1 -- start start 2024-04-25T10:17:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d67300 0x5598b2d914b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d77230 0x5598b2d46c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f877de26740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d8fb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f877de26740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8774200280 con 0x5598b2d94110 2024-04-25T10:17:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87740cb880 con 0x5598b2d77230 2024-04-25T10:17:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87741f4690 con 0x5598b2d67300 2024-04-25T10:17:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.922+0000 7f8779000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d8fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d8fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35898/0 (socket says 172.21.15.165:35898) 2024-04-25T10:17:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 -- 172.21.15.165:0/4220981626 learned_addr learned my addr 172.21.15.165:0/4220981626 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 -- 172.21.15.165:0/4220981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d67300 msgr2=0x5598b2d914b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d67300 0x5598b2d914b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 -- 172.21.15.165:0/4220981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d77230 msgr2=0x5598b2d46c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d77230 0x5598b2d46c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 -- 172.21.15.165:0/4220981626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598b2d4c4b0 con 0x5598b2d94110 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f8779000640 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d8fb10 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f876c087fd0 tx=0x7f876c05e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=193da79c5897e448 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f875fe00640 1 -- 172.21.15.165:0/4220981626 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c060560 con 0x5598b2d94110 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f875fe00640 1 -- 172.21.15.165:0/4220981626 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c05a050 con 0x5598b2d94110 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.923+0000 7f875fe00640 1 -- 172.21.15.165:0/4220981626 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c056200 con 0x5598b2d94110 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 -- 172.21.15.165:0/4220981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 msgr2=0x5598b2d8fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d8fb10 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f876c087fd0 tx=0x7f876c05e040 comp rx=0 tx=0).stop 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 -- 172.21.15.165:0/4220981626 shutdown_connections 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d94110 0x5598b2d8fb10 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d77230 0x5598b2d46c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 --2- 172.21.15.165:0/4220981626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d67300 0x5598b2d914b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 -- 172.21.15.165:0/4220981626 >> 172.21.15.165:0/4220981626 conn(0x5598b2d76af0 msgr2=0x5598b2d923c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 -- 172.21.15.165:0/4220981626 shutdown_connections 2024-04-25T10:17:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.924+0000 7f877de26740 1 -- 172.21.15.165:0/4220981626 wait complete. 2024-04-25T10:17:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.925+0000 7f877de26740 1 -- start start 2024-04-25T10:17:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.925+0000 7f877de26740 1 Processor -- start 2024-04-25T10:17:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 0x5598b2d8e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d78ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d94110 0x5598b2d78ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b2d78930 con 0x5598b2d67300 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598b2d4aec0 con 0x5598b2d94110 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5598b2da5210 con 0x5598b2d77230 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f877de26740 10 client.0 _ll_register_callbacks cb 0x7f876000b320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.926+0000 7f8773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 0x5598b2d8e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 0x5598b2d8e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35904/0 (socket says 172.21.15.165:35904) 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 -- 172.21.15.165:0/2488859272 learned_addr learned my addr 172.21.15.165:0/2488859272 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 -- 172.21.15.165:0/2488859272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 msgr2=0x5598b2d78ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d78ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 -- 172.21.15.165:0/2488859272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d94110 msgr2=0x5598b2d78ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d94110 0x5598b2d78ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 -- 172.21.15.165:0/2488859272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598b2ddfd60 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f8773e00640 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 0x5598b2d8e2f0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f8764009870 tx=0x7f87640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=862d2692e3cc7305 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f875d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.927+0000 7f875d600640 1 -- 172.21.15.165:0/2488859272 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8764037040 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f875d600640 1 -- 172.21.15.165:0/2488859272 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8764057c70 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f876c05a050 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f875d600640 1 -- 172.21.15.165:0/2488859272 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7f876404d0f0 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 10 client.10050 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5598b2dfddf0 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f875d600640 1 -- 172.21.15.165:0/2488859272 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c05a050 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f875d600640 1 -- 172.21.15.165:0/2488859272 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7f876404d290 con 0x5598b2d67300 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 10 client.10050 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 10 client.10050 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 20 client.10050 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8b710 0x5598b2da3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 4 client.10050 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "scrub abort"}] 2024-04-25T10:17:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.928+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "scrub abort"}) v1 -- 0x5598b2d7a1c0 con 0x5598b2d8b710 2024-04-25T10:17:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f8773400640 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8b710 0x5598b2da3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f8773400640 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8b710 0x5598b2da3cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=1c837b400613bf66 server_cookie=b7795706209e031f in_seq=0 out_seq=0 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f875d600640 10 client.10050 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f875d600640 1 -- 172.21.15.165:0/2488859272 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x5598b2d7a1c0 con 0x5598b2d8b710 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f875d600640 10 client.10050 handle_command_reply: tid=0 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f877de26740 1 client.10050 shutdown 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 msgr2=0x5598b2d8e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.929+0000 7f877de26740 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 0x5598b2d8e2f0 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7f8764009870 tx=0x7f87640046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 shutdown_connections 2024-04-25T10:17:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5598b2d8b710 0x5598b2da3cd0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5598b2d94110 0x5598b2d78ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5598b2d77230 0x5598b2d78ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 --2- 172.21.15.165:0/2488859272 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5598b2d67300 0x5598b2d8e2f0 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f875d600640 0 client.10050 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 >> 172.21.15.165:0/2488859272 conn(0x5598b2d76af0 msgr2=0x5598b2d925d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 shutdown_connections 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 1 -- 172.21.15.165:0/2488859272 wait complete. 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:29.930+0000 7f877de26740 20 client.10050 trim_cache size 0 max 16384 2024-04-25T10:17:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:Error EINVAL: MDS is not active 2024-04-25T10:17:29.941 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:17:29.941 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a scrub resume 2024-04-25T10:17:30.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.122+0000 7fb66e400640 1 Processor -- start 2024-04-25T10:17:30.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.123+0000 7fb66e400640 1 -- start start 2024-04-25T10:17:30.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.124+0000 7fb66e400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb66815b3c0 0x7fb6680b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.124+0000 7fb66e400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6680b0b80 con 0x7fb66815b7a0 2024-04-25T10:17:30.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.124+0000 7fb66e400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6680cc800 con 0x7fb6680cc150 2024-04-25T10:17:30.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.124+0000 7fb66e400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6680cc5e0 con 0x7fb66815b3c0 2024-04-25T10:17:30.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.124+0000 7fb66d000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb66815b3c0 0x7fb6680b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.124+0000 7fb667e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb66815b7a0 0x7fb6680cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41256/0 (socket says 172.21.15.165:41256) 2024-04-25T10:17:30.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.125+0000 7fb667e00640 1 -- 172.21.15.165:0/4142706717 learned_addr learned my addr 172.21.15.165:0/4142706717 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:30.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.125+0000 7fb66d000640 1 -- 172.21.15.165:0/4142706717 >> v1:172.21.15.172:6789/0 conn(0x7fb6680cc150 legacy=0x7fb66815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.125+0000 7fb66d000640 1 -- 172.21.15.165:0/4142706717 >> v1:172.21.15.165:6789/0 conn(0x7fb66815b7a0 legacy=0x7fb6680cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.125+0000 7fb667400640 1 -- 172.21.15.165:0/4142706717 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2951614553 0 0) 0x7fb650001110 con 0x7fb66815b7a0 2024-04-25T10:17:30.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.125+0000 7fb66d000640 1 -- 172.21.15.165:0/4142706717 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6680ccae0 con 0x7fb66815b3c0 2024-04-25T10:17:30.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.126+0000 7fb66d000640 1 --2- 172.21.15.165:0/4142706717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b3c0 0x7fb6680b0f60 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097a0 tx=0x7fb65802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a1930d4cf5cbea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.126+0000 7fb667400640 1 -- 172.21.15.165:0/4142706717 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658037070 con 0x7fb66815b3c0 2024-04-25T10:17:30.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb667400640 1 -- 172.21.15.165:0/4142706717 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65802ae50 con 0x7fb66815b3c0 2024-04-25T10:17:30.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb667400640 1 -- 172.21.15.165:0/4142706717 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658032660 con 0x7fb66815b3c0 2024-04-25T10:17:30.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 -- 172.21.15.165:0/4142706717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b3c0 msgr2=0x7fb6680b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 --2- 172.21.15.165:0/4142706717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b3c0 0x7fb6680b0f60 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fb6580097a0 tx=0x7fb65802a140 comp rx=0 tx=0).stop 2024-04-25T10:17:30.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 -- 172.21.15.165:0/4142706717 shutdown_connections 2024-04-25T10:17:30.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 --2- 172.21.15.165:0/4142706717 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b3c0 0x7fb6680b0f60 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 -- 172.21.15.165:0/4142706717 >> 172.21.15.165:0/4142706717 conn(0x7fb668061cc0 msgr2=0x7fb66806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 -- 172.21.15.165:0/4142706717 shutdown_connections 2024-04-25T10:17:30.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.127+0000 7fb66e400640 1 -- 172.21.15.165:0/4142706717 wait complete. 2024-04-25T10:17:30.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.129+0000 7fb66e400640 1 Processor -- start 2024-04-25T10:17:30.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.129+0000 7fb66e400640 1 -- start start 2024-04-25T10:17:30.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.130+0000 7fb66e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6680cc150 0x7fb668155020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.130+0000 7fb66e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66815b3c0 0x7fb6681f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.130+0000 7fb66e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b7a0 0x7fb668157ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.130+0000 7fb66e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680ce1f0 con 0x7fb66815b7a0 2024-04-25T10:17:30.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.130+0000 7fb66e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680cd650 con 0x7fb66815b3c0 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.130+0000 7fb66e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6680ce370 con 0x7fb6680cc150 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b7a0 0x7fb668157ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b7a0 0x7fb668157ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35914/0 (socket says 172.21.15.165:35914) 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 -- 172.21.15.165:0/2318936221 learned_addr learned my addr 172.21.15.165:0/2318936221 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb667e00640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66815b3c0 0x7fb6681f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 -- 172.21.15.165:0/2318936221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6680cc150 msgr2=0x7fb668155020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb6680cc150 0x7fb668155020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 -- 172.21.15.165:0/2318936221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66815b3c0 msgr2=0x7fb6681f34b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66815b3c0 0x7fb6681f34b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb66da00640 1 -- 172.21.15.165:0/2318936221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6680c8fd0 con 0x7fb66815b7a0 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.131+0000 7fb667e00640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb66815b3c0 0x7fb6681f34b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:30.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.132+0000 7fb66da00640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb66815b7a0 0x7fb668157ac0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7fb66000c7b0 tx=0x7fb66000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.132+0000 7fb664c00640 1 -- 172.21.15.165:0/2318936221 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660004b60 con 0x7fb66815b7a0 2024-04-25T10:17:30.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.132+0000 7fb664c00640 1 -- 172.21.15.165:0/2318936221 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb660004d00 con 0x7fb66815b7a0 2024-04-25T10:17:30.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.132+0000 7fb664c00640 1 -- 172.21.15.165:0/2318936221 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb66000aa90 con 0x7fb66815b7a0 2024-04-25T10:17:30.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.132+0000 7fb66e400640 1 -- 172.21.15.165:0/2318936221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6680c92b0 con 0x7fb66815b7a0 2024-04-25T10:17:30.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.132+0000 7fb66e400640 1 -- 172.21.15.165:0/2318936221 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb668155560 con 0x7fb66815b7a0 2024-04-25T10:17:30.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.134+0000 7fb664c00640 1 -- 172.21.15.165:0/2318936221 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb66000ac30 con 0x7fb66815b7a0 2024-04-25T10:17:30.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.136+0000 7fb664c00640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb64406b4d0 0x7fb64406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.136+0000 7fb664c00640 1 -- 172.21.15.165:0/2318936221 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb66008a6d0 con 0x7fb66815b7a0 2024-04-25T10:17:30.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.136+0000 7fb66d000640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb64406b4d0 0x7fb64406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.136+0000 7fb6727a7740 1 Processor -- start 2024-04-25T10:17:30.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.137+0000 7fb6727a7740 1 -- start start 2024-04-25T10:17:30.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.137+0000 7fb66d000640 1 --2- 172.21.15.165:0/2318936221 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb64406b4d0 0x7fb64406d990 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fb658034000 tx=0x7fb65802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.137+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 0x5621e2276c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.137+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2277290 0x5621e2246a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.137+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2246fb0 0x5621e226ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb6727a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e21af870 con 0x5621e2276870 2024-04-25T10:17:30.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e2158d60 con 0x5621e2277290 2024-04-25T10:17:30.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e21ab910 con 0x5621e2246fb0 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 0x5621e2276c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 0x5621e2276c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35924/0 (socket says 172.21.15.165:35924) 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 -- 172.21.15.165:0/920217010 learned_addr learned my addr 172.21.15.165:0/920217010 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 -- 172.21.15.165:0/920217010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2246fb0 msgr2=0x5621e226ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2246fb0 0x5621e226ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 -- 172.21.15.165:0/920217010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2277290 msgr2=0x5621e2246a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2277290 0x5621e2246a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.138+0000 7fb66d000640 1 -- 172.21.15.165:0/920217010 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621e1f8e7d0 con 0x5621e2276870 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.139+0000 7fb66d000640 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 0x5621e2276c50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fb658035040 tx=0x7fb658008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc841ba558592106 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.139+0000 7fb65d600640 1 -- 172.21.15.165:0/920217010 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658037040 con 0x5621e2276870 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.139+0000 7fb65d600640 1 -- 172.21.15.165:0/920217010 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb658042070 con 0x5621e2276870 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.139+0000 7fb65d600640 1 -- 172.21.15.165:0/920217010 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65803d670 con 0x5621e2276870 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 -- 172.21.15.165:0/920217010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 msgr2=0x5621e2276c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 0x5621e2276c50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fb658035040 tx=0x7fb658008030 comp rx=0 tx=0).stop 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 -- 172.21.15.165:0/920217010 shutdown_connections 2024-04-25T10:17:30.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2246fb0 0x5621e226ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2277290 0x5621e2246a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 --2- 172.21.15.165:0/920217010 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2276870 0x5621e2276c50 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 -- 172.21.15.165:0/920217010 >> 172.21.15.165:0/920217010 conn(0x5621e2248ef0 msgr2=0x5621e2291680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 -- 172.21.15.165:0/920217010 shutdown_connections 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.140+0000 7fb6727a7740 1 -- 172.21.15.165:0/920217010 wait complete. 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.141+0000 7fb6727a7740 1 -- start start 2024-04-25T10:17:30.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.141+0000 7fb6727a7740 1 Processor -- start 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e227a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2266d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2299710 0x5621e226b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e2272fc0 con 0x5621e2277290 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e216c1c0 con 0x5621e2276870 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e2273260 con 0x5621e2299710 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb6727a7740 10 client.0 _ll_register_callbacks cb 0x5621e21aa8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb667e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2266d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb667e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2266d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35928/0 (socket says 172.21.15.165:35928) 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb667e00640 1 -- 172.21.15.165:0/1711046640 learned_addr learned my addr 172.21.15.165:0/1711046640 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb667e00640 1 -- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2299710 msgr2=0x5621e226b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb667e00640 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2299710 0x5621e226b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb667e00640 1 -- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 msgr2=0x5621e227a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.142+0000 7fb66d000640 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e227a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb667e00640 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e227a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb667e00640 1 -- 172.21.15.165:0/1711046640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621e1e840f0 con 0x5621e2277290 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb667e00640 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2266d90 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fb650001d70 tx=0x7fb65000d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e47d39a220b44b56 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb656000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb656000640 1 -- 172.21.15.165:0/1711046640 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6500070d0 con 0x5621e2277290 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb656000640 1 -- 172.21.15.165:0/1711046640 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb650013c10 con 0x5621e2277290 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb658042070 con 0x5621e2277290 2024-04-25T10:17:30.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb656000640 1 -- 172.21.15.165:0/1711046640 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb650013e60 con 0x5621e2277290 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb6727a7740 10 client.10065 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.143+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5621e1feef60 con 0x5621e2277290 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb656000640 1 -- 172.21.15.165:0/1711046640 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb658042070 con 0x5621e2277290 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb656000640 1 -- 172.21.15.165:0/1711046640 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fb650007270 con 0x5621e2277290 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb6727a7740 10 client.10065 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb6727a7740 10 client.10065 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb6727a7740 20 client.10065 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb6727a7740 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e22695c0 0x5621e2278cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb6727a7740 4 client.10065 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621e2185690 con 0x5621e22695c0 2024-04-25T10:17:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.144+0000 7fb66d000640 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e22695c0 0x5621e2278cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.145+0000 7fb66d000640 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e22695c0 0x5621e2278cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=cc1304e8390584e1 server_cookie=cf1dc7f89bf35c45 in_seq=0 out_seq=0 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.145+0000 7fb656000640 10 client.10065 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.145+0000 7fb656000640 1 -- 172.21.15.165:0/1711046640 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5621e2185690 con 0x5621e22695c0 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.145+0000 7fb656000640 10 client.10065 handle_command_reply: tid=0 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.145+0000 7fb6727a7740 1 client.10065 shutdown 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 msgr2=0x5621e2266d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2266d90 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fb650001d70 tx=0x7fb65000d520 comp rx=0 tx=0).stop 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 shutdown_connections 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e22695c0 0x5621e2278cb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2299710 0x5621e226b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2266d90 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 --2- 172.21.15.165:0/1711046640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e227a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 >> 172.21.15.165:0/1711046640 conn(0x5621e2246fb0 msgr2=0x5621e224a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb656000640 0 client.10065 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 shutdown_connections 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 1 -- 172.21.15.165:0/1711046640 wait complete. 2024-04-25T10:17:30.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.146+0000 7fb6727a7740 20 client.10065 trim_cache size 0 max 16384 2024-04-25T10:17:30.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.162+0000 7fb6727a7740 1 Processor -- start 2024-04-25T10:17:30.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.162+0000 7fb6727a7740 1 -- start start 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e224cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2277290 0x5621e226b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb6727a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6681ffca0 con 0x5621e2299710 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb668200300 con 0x5621e2276870 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6681f7690 con 0x5621e2277290 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb66da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb66da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35942/0 (socket says 172.21.15.165:35942) 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.163+0000 7fb66da00640 1 -- 172.21.15.165:0/947283097 learned_addr learned my addr 172.21.15.165:0/947283097 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66da00640 1 -- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2277290 msgr2=0x5621e226b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66da00640 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2277290 0x5621e226b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66da00640 1 -- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 msgr2=0x5621e224cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66d000640 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e224cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66da00640 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e224cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66da00640 1 -- 172.21.15.165:0/947283097 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621e1e5d1b0 con 0x5621e2299710 2024-04-25T10:17:30.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb66da00640 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 0x5621e226a5d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fb660060e90 tx=0x7fb660058bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c048705c2672e451 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb65cc00640 1 -- 172.21.15.165:0/947283097 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb660070030 con 0x5621e2299710 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb65cc00640 1 -- 172.21.15.165:0/947283097 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66005ce00 con 0x5621e2299710 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.164+0000 7fb65cc00640 1 -- 172.21.15.165:0/947283097 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6600613c0 con 0x5621e2299710 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 -- 172.21.15.165:0/947283097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 msgr2=0x5621e226a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 0x5621e226a5d0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fb660060e90 tx=0x7fb660058bb0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 -- 172.21.15.165:0/947283097 shutdown_connections 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2277290 0x5621e226b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 --2- 172.21.15.165:0/947283097 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2276870 0x5621e224cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 -- 172.21.15.165:0/947283097 >> 172.21.15.165:0/947283097 conn(0x5621e2246fb0 msgr2=0x5621e22758c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 -- 172.21.15.165:0/947283097 shutdown_connections 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.165+0000 7fb6727a7740 1 -- 172.21.15.165:0/947283097 wait complete. 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.166+0000 7fb6727a7740 1 -- start start 2024-04-25T10:17:30.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.166+0000 7fb6727a7740 1 Processor -- start 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2276870 0x5621e2278d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2274f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621e2269440 con 0x5621e2277290 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6681fefa0 con 0x5621e2299710 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621e2269a40 con 0x5621e2276870 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb6727a7740 10 client.0 _ll_register_callbacks cb 0x7fb658035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb667e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2274f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb667e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2274f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35950/0 (socket says 172.21.15.165:35950) 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.167+0000 7fb667e00640 1 -- 172.21.15.165:0/873887057 learned_addr learned my addr 172.21.15.165:0/873887057 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb66d000640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2276870 0x5621e2278d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb667e00640 1 -- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2276870 msgr2=0x5621e2278d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb667e00640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2276870 0x5621e2278d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb66da00640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb667e00640 1 -- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2299710 msgr2=0x5621e226a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb667e00640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb667e00640 1 -- 172.21.15.165:0/873887057 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621e2294f00 con 0x5621e2277290 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb66d000640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2276870 0x5621e2278d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb667e00640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2274f70 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fb6500077e0 tx=0x7fb650000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e24ee1a3c8764b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb656000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb656000640 1 -- 172.21.15.165:0/873887057 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb650024040 con 0x5621e2277290 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb656000640 1 -- 172.21.15.165:0/873887057 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb65002ac70 con 0x5621e2277290 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.168+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb66005ce00 con 0x5621e2277290 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb656000640 1 -- 172.21.15.165:0/873887057 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=104) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6500073d0 con 0x5621e2277290 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 10 client.10071 fetch_fsmap learned FSMap version 104 2024-04-25T10:17:30.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=104,monmap=2+}) v3 -- 0x5621e2275570 con 0x5621e2277290 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb656000640 1 -- 172.21.15.165:0/873887057 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb66005ce00 con 0x5621e2277290 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb656000640 1 -- 172.21.15.165:0/873887057 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 104) v1 ==== 3282+0+0 (secure 0 0 0) 0x7fb650031050 con 0x5621e2277290 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 10 client.10071 fetch_fsmap finished waiting for FSMap version 104 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 10 client.10071 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 20 client.10071 populate_metadata read hostname 'smithi165' 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e224b760 0x5621e224bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 4 client.10071 mds_command: new command op to 9762 tid=0 multi_id=0 [{"prefix": "scrub resume"}] 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 --> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] -- command(tid 0: {"prefix": "scrub resume"}) v1 -- 0x5621e21bcdb0 con 0x5621e224b760 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.169+0000 7fb66d000640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e224b760 0x5621e224bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb66d000640 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e224b760 0x5621e224bb40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.9762 client_cookie=cc4449cc7d845ca1 server_cookie=b552e21e53adbf80 in_seq=0 out_seq=0 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb656000640 10 client.10071 ms_handle_connect on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb656000640 1 -- 172.21.15.165:0/873887057 <== mds.9762 v2:172.21.15.165:6834/1838340482 1 ==== command_reply(tid 0: -22 MDS is not active) v1 ==== 25+0+0 (crc 0 0 0) 0x5621e21bcdb0 con 0x5621e224b760 2024-04-25T10:17:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb656000640 10 client.10071 handle_command_reply: tid=0 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 client.10071 shutdown 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 msgr2=0x5621e2274f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2274f70 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fb6500077e0 tx=0x7fb650000ed0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 shutdown_connections 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:6834/1838340482,v1:172.21.15.165:6835/1838340482] conn(0x5621e224b760 0x5621e224bb40 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5621e2299710 0x5621e226a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5621e2277290 0x5621e2274f70 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 --2- 172.21.15.165:0/873887057 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5621e2276870 0x5621e2278d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 >> 172.21.15.165:0/873887057 conn(0x5621e2246fb0 msgr2=0x5621e22675c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb656000640 0 client.10071 ms_handle_reset on v2:172.21.15.165:6834/1838340482 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.170+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 shutdown_connections 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.171+0000 7fb6727a7740 1 -- 172.21.15.165:0/873887057 wait complete. 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.171+0000 7fb6727a7740 20 client.10071 trim_cache size 0 max 16384 2024-04-25T10:17:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:Error EINVAL: MDS is not active 2024-04-25T10:17:30.182 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-25T10:17:30.182 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:30.183 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:30.278 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:30.386 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:30.386 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:17:30.386 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:30.444 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:17:30.444 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:30.500 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:30.615 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:30.621 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:17:30.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.705+0000 7f9a7c000640 1 Processor -- start 2024-04-25T10:17:30.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.705+0000 7f9a7c000640 1 -- start start 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.706+0000 7f9a7c000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9a7415da80 0x7f9a740b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.706+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.706+0000 7f9a7c000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a7406df50 0x7f9a7406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.706+0000 7f9a7c000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a740b4370 con 0x7f9a7415da80 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.706+0000 7f9a7c000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a7415de60 con 0x7f9a7406df50 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.706+0000 7f9a7c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740b3760 con 0x7f9a740ad5a0 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9a7415da80 0x7f9a740b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9a7415da80 0x7f9a740b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35956/0 (socket says 172.21.15.165:35956) 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 -- 172.21.15.165:0/2522081084 learned_addr learned my addr 172.21.15.165:0/2522081084 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a78c00640 1 --2- 172.21.15.165:0/2522081084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a7a000640 1 --2- 172.21.15.165:0/2522081084 >> v2:172.21.15.172:3300/0 conn(0x7f9a7406df50 0x7f9a7406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 -- 172.21.15.165:0/2522081084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 msgr2=0x7f9a7406da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 --2- 172.21.15.165:0/2522081084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 -- 172.21.15.165:0/2522081084 >> v2:172.21.15.172:3300/0 conn(0x7f9a7406df50 msgr2=0x7f9a7406bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 --2- 172.21.15.165:0/2522081084 >> v2:172.21.15.172:3300/0 conn(0x7f9a7406df50 0x7f9a7406bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a79600640 1 -- 172.21.15.165:0/2522081084 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a740b3a40 con 0x7f9a7415da80 2024-04-25T10:17:30.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.707+0000 7f9a7a000640 1 --2- 172.21.15.165:0/2522081084 >> v2:172.21.15.172:3300/0 conn(0x7f9a7406df50 0x7f9a7406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:30.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.708+0000 7f9a79600640 1 --2- 172.21.15.165:0/2522081084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a740b2e10 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f9a74061e60 tx=0x7f9a6802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a3b446969252626 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a73e00640 1 -- 172.21.15.165:0/2522081084 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68037070 con 0x7f9a7415da80 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a73e00640 1 -- 172.21.15.165:0/2522081084 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a68032440 con 0x7f9a7415da80 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a73e00640 1 -- 172.21.15.165:0/2522081084 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a68032810 con 0x7f9a7415da80 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 -- 172.21.15.165:0/2522081084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 msgr2=0x7f9a740b2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 --2- 172.21.15.165:0/2522081084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a740b2e10 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f9a74061e60 tx=0x7f9a6802a270 comp rx=0 tx=0).stop 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 -- 172.21.15.165:0/2522081084 shutdown_connections 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 --2- 172.21.15.165:0/2522081084 >> v2:172.21.15.172:3300/0 conn(0x7f9a7406df50 0x7f9a7406bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 --2- 172.21.15.165:0/2522081084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 --2- 172.21.15.165:0/2522081084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a740b2e10 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.709+0000 7f9a7c000640 1 -- 172.21.15.165:0/2522081084 >> 172.21.15.165:0/2522081084 conn(0x7f9a74071b00 msgr2=0x7f9a74071f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.710+0000 7f9a7c000640 1 -- 172.21.15.165:0/2522081084 shutdown_connections 2024-04-25T10:17:30.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.710+0000 7f9a7c000640 1 -- 172.21.15.165:0/2522081084 wait complete. 2024-04-25T10:17:30.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.711+0000 7f9a7c000640 1 Processor -- start 2024-04-25T10:17:30.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.711+0000 7f9a7c000640 1 -- start start 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.712+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a7406df50 0x7f9a74149b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.712+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.712+0000 7f9a7c000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a74146490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.712+0000 7f9a7c000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740b3350 con 0x7f9a7415da80 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.712+0000 7f9a7c000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740b4000 con 0x7f9a7406df50 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.712+0000 7f9a7c000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a740b34d0 con 0x7f9a740ad5a0 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a74146490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a74146490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35960/0 (socket says 172.21.15.165:35960) 2024-04-25T10:17:30.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 -- 172.21.15.165:0/1072333407 learned_addr learned my addr 172.21.15.165:0/1072333407 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a79600640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a7406df50 0x7f9a74149b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a78c00640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 -- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 msgr2=0x7f9a7406b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 -- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a7406df50 msgr2=0x7f9a74149b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a7406df50 0x7f9a74149b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 -- 172.21.15.165:0/1072333407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7414a0c0 con 0x7f9a7415da80 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.713+0000 7f9a7a000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a74146490 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00cef0 tx=0x7f9a6c00a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.714+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c01a070 con 0x7f9a7415da80 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.714+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c00d860 con 0x7f9a7415da80 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.714+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c00dc50 con 0x7f9a7415da80 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.714+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a74159c10 con 0x7f9a7415da80 2024-04-25T10:17:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.714+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a7415a4b0 con 0x7f9a7415da80 2024-04-25T10:17:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.715+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a74073990 con 0x7f9a7415da80 2024-04-25T10:17:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.716+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a6c01d070 con 0x7f9a7415da80 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.717+0000 7f9a71600640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5406b4d0 0x7f9a5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.717+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9a6c08ac10 con 0x7f9a7415da80 2024-04-25T10:17:30.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.717+0000 7f9a79600640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5406b4d0 0x7f9a5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:30.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.718+0000 7f9a79600640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5406b4d0 0x7f9a5406d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9a68034000 tx=0x7f9a6802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:30.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.722+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a7414a0c0 con 0x7f9a7415da80 2024-04-25T10:17:30.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.924+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a740729a0 con 0x7f9a7415da80 2024-04-25T10:17:30.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.926+0000 7f9a71600640 1 -- 172.21.15.165:0/1072333407 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f9a6c0592d0 con 0x7f9a7415da80 2024-04-25T10:17:30.931 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:30.931 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":148,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:44.518826+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.929+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5406b4d0 msgr2=0x7f9a5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.929+0000 7f9a7c000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5406b4d0 0x7f9a5406d990 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f9a68034000 tx=0x7f9a6802e000 comp rx=0 tx=0).stop 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.929+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 msgr2=0x7f9a74146490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.929+0000 7f9a7c000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a74146490 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00cef0 tx=0x7f9a6c00a710 comp rx=0 tx=0).stop 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 shutdown_connections 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5406b4d0 0x7f9a5406d990 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a7415da80 0x7f9a74146490 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a740ad5a0 0x7f9a7406b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 --2- 172.21.15.165:0/1072333407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a7406df50 0x7f9a74149b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 >> 172.21.15.165:0/1072333407 conn(0x7f9a74071b00 msgr2=0x7f9a7406f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 shutdown_connections 2024-04-25T10:17:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:30.930+0000 7f9a7c000640 1 -- 172.21.15.165:0/1072333407 wait complete. 2024-04-25T10:17:30.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:40.787562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:42.000970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:30.954 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:31.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.200+0000 7f6444000640 1 Processor -- start 2024-04-25T10:17:31.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.200+0000 7f6444000640 1 -- start start 2024-04-25T10:17:31.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6444000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f643c15dc20 0x7f643c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6444000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6444000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f643c15e000 con 0x7f643c0b9330 2024-04-25T10:17:31.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6444000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f643c0d1de0 con 0x7f643c15dc20 2024-04-25T10:17:31.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6444000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c161c60 con 0x7f643c161710 2024-04-25T10:17:31.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6440c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6441600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f643c15dc20 0x7f643c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.202+0000 7f6442000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f643c0b9330 0x7f643c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:42930/0 (socket says 172.21.15.165:42930) 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.203+0000 7f6442000640 1 -- 172.21.15.165:0/4099224999 learned_addr learned my addr 172.21.15.165:0/4099224999 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.203+0000 7f6440c00640 1 -- 172.21.15.165:0/4099224999 >> v2:172.21.15.172:3300/0 conn(0x7f643c15dc20 msgr2=0x7f643c0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.203+0000 7f6440c00640 1 --2- 172.21.15.165:0/4099224999 >> v2:172.21.15.172:3300/0 conn(0x7f643c15dc20 0x7f643c0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.203+0000 7f6440c00640 1 -- 172.21.15.165:0/4099224999 >> v1:172.21.15.172:6789/0 conn(0x7f643c0b9330 legacy=0x7f643c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.203+0000 7f6440c00640 1 -- 172.21.15.165:0/4099224999 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c161f40 con 0x7f643c161710 2024-04-25T10:17:31.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.205+0000 7f6440c00640 1 --2- 172.21.15.165:0/4099224999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c0cebb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f642c009d90 tx=0x7f642c02a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc1b73c08c2e58ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.205+0000 7f643be00640 1 -- 172.21.15.165:0/4099224999 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c007740 con 0x7f643c161710 2024-04-25T10:17:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.205+0000 7f643be00640 1 -- 172.21.15.165:0/4099224999 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c0078e0 con 0x7f643c161710 2024-04-25T10:17:31.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.205+0000 7f643be00640 1 -- 172.21.15.165:0/4099224999 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c007cb0 con 0x7f643c161710 2024-04-25T10:17:31.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 -- 172.21.15.165:0/4099224999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 msgr2=0x7f643c0cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 --2- 172.21.15.165:0/4099224999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c0cebb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f642c009d90 tx=0x7f642c02a660 comp rx=0 tx=0).stop 2024-04-25T10:17:31.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 -- 172.21.15.165:0/4099224999 shutdown_connections 2024-04-25T10:17:31.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 --2- 172.21.15.165:0/4099224999 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c0cebb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 --2- 172.21.15.165:0/4099224999 >> v2:172.21.15.172:3300/0 conn(0x7f643c15dc20 0x7f643c0ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 -- 172.21.15.165:0/4099224999 >> 172.21.15.165:0/4099224999 conn(0x7f643c0d0420 msgr2=0x7f643c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 -- 172.21.15.165:0/4099224999 shutdown_connections 2024-04-25T10:17:31.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.206+0000 7f6444000640 1 -- 172.21.15.165:0/4099224999 wait complete. 2024-04-25T10:17:31.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.208+0000 7f6444000640 1 Processor -- start 2024-04-25T10:17:31.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.208+0000 7f6444000640 1 -- start start 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6444000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 0x7f643c0cac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6444000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f643c15dc20 0x7f643c14c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6444000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c14bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6444000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0b5e50 con 0x7f643c0b9330 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6444000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643c162500 con 0x7f643c15dc20 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6444000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f643c0b5fd0 con 0x7f643c161710 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6441600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 0x7f643c0cac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6441600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 0x7f643c0cac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35964/0 (socket says 172.21.15.165:35964) 2024-04-25T10:17:31.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.209+0000 7f6441600640 1 -- 172.21.15.165:0/3881944374 learned_addr learned my addr 172.21.15.165:0/3881944374 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6442000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c14bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6440c00640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f643c15dc20 0x7f643c14c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6441600640 1 -- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 msgr2=0x7f643c14bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6441600640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c14bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6441600640 1 -- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f643c15dc20 msgr2=0x7f643c14c5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6441600640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f643c15dc20 0x7f643c14c5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6441600640 1 -- 172.21.15.165:0/3881944374 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643c14cb20 con 0x7f643c0b9330 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6441600640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 0x7f643c0cac20 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f643000e870 tx=0x7f643000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6430017070 con 0x7f643c0b9330 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f643000c940 con 0x7f643c0b9330 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f643000cd30 con 0x7f643c0b9330 2024-04-25T10:17:31.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.210+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643c14cdc0 con 0x7f643c0b9330 2024-04-25T10:17:31.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.211+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643c14d380 con 0x7f643c0b9330 2024-04-25T10:17:31.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.212+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643c073a00 con 0x7f643c0b9330 2024-04-25T10:17:31.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.213+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f643001a070 con 0x7f643c0b9330 2024-04-25T10:17:31.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.214+0000 7f6439600640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f642006b4d0 0x7f642006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.215+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f643008a6b0 con 0x7f643c0b9330 2024-04-25T10:17:31.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.215+0000 7f6440c00640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f642006b4d0 0x7f642006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.215+0000 7f6440c00640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f642006b4d0 0x7f642006d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f642c034000 tx=0x7f642c033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.219+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6430059300 con 0x7f643c0b9330 2024-04-25T10:17:31.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.415+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f643c0d16b0 con 0x7f643c0b9330 2024-04-25T10:17:31.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.416+0000 7f6439600640 1 -- 172.21.15.165:0/3881944374 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6430058c00 con 0x7f643c0b9330 2024-04-25T10:17:31.420 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.418+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f642006b4d0 msgr2=0x7f642006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.418+0000 7f6444000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f642006b4d0 0x7f642006d990 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f642c034000 tx=0x7f642c033040 comp rx=0 tx=0).stop 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.418+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 msgr2=0x7f643c0cac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.418+0000 7f6444000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 0x7f643c0cac20 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f643000e870 tx=0x7f643000ed30 comp rx=0 tx=0).stop 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 shutdown_connections 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f642006b4d0 0x7f642006d990 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f643c161710 0x7f643c14bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f643c15dc20 0x7f643c14c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 --2- 172.21.15.165:0/3881944374 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643c0b9330 0x7f643c0cac20 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 >> 172.21.15.165:0/3881944374 conn(0x7f643c0d0420 msgr2=0x7f643c15bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 shutdown_connections 2024-04-25T10:17:31.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.419+0000 7f6444000640 1 -- 172.21.15.165:0/3881944374 wait complete. 2024-04-25T10:17:31.441 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.649+0000 7f2f58000640 1 Processor -- start 2024-04-25T10:17:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.649+0000 7f2f58000640 1 -- start start 2024-04-25T10:17:31.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f58000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f501446d0 0x7f2f500b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f58000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2f50144d50 0x7f2f500b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f58000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f58000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f500b2b20 con 0x7f2f50144d50 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f58000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f5006d400 con 0x7f2f501446d0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f58000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500b49a0 con 0x7f2f501452a0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f54c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2f50144d50 0x7f2f500b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f54c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2f50144d50 0x7f2f500b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35980/0 (socket says 172.21.15.165:35980) 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f54c00640 1 -- 172.21.15.165:0/2585641660 learned_addr learned my addr 172.21.15.165:0/2585641660 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.651+0000 7f2f56000640 1 --2- 172.21.15.165:0/2585641660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.652+0000 7f2f55600640 1 --2- 172.21.15.165:0/2585641660 >> v2:172.21.15.172:3300/0 conn(0x7f2f501446d0 0x7f2f500b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.652+0000 7f2f54c00640 1 -- 172.21.15.165:0/2585641660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 msgr2=0x7f2f500cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.652+0000 7f2f54c00640 1 --2- 172.21.15.165:0/2585641660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.652+0000 7f2f54c00640 1 -- 172.21.15.165:0/2585641660 >> v2:172.21.15.172:3300/0 conn(0x7f2f501446d0 msgr2=0x7f2f500b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:31.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.652+0000 7f2f54c00640 1 --2- 172.21.15.165:0/2585641660 >> v2:172.21.15.172:3300/0 conn(0x7f2f501446d0 0x7f2f500b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.652+0000 7f2f54c00640 1 -- 172.21.15.165:0/2585641660 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500cccc0 con 0x7f2f50144d50 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.653+0000 7f2f54c00640 1 --2- 172.21.15.165:0/2585641660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f50144d50 0x7f2f500b4460 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f2f500cf560 tx=0x7f2f3c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d4ffe0f08582ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.653+0000 7f2f4fe00640 1 -- 172.21.15.165:0/2585641660 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c037070 con 0x7f2f50144d50 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.653+0000 7f2f4fe00640 1 -- 172.21.15.165:0/2585641660 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c032440 con 0x7f2f50144d50 2024-04-25T10:17:31.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.653+0000 7f2f4fe00640 1 -- 172.21.15.165:0/2585641660 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c032810 con 0x7f2f50144d50 2024-04-25T10:17:31.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 -- 172.21.15.165:0/2585641660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f50144d50 msgr2=0x7f2f500b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 --2- 172.21.15.165:0/2585641660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f50144d50 0x7f2f500b4460 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f2f500cf560 tx=0x7f2f3c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 -- 172.21.15.165:0/2585641660 shutdown_connections 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 --2- 172.21.15.165:0/2585641660 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 --2- 172.21.15.165:0/2585641660 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f50144d50 0x7f2f500b4460 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 --2- 172.21.15.165:0/2585641660 >> v2:172.21.15.172:3300/0 conn(0x7f2f501446d0 0x7f2f500b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 -- 172.21.15.165:0/2585641660 >> 172.21.15.165:0/2585641660 conn(0x7f2f5006abe0 msgr2=0x7f2f5006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 -- 172.21.15.165:0/2585641660 shutdown_connections 2024-04-25T10:17:31.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.654+0000 7f2f58000640 1 -- 172.21.15.165:0/2585641660 wait complete. 2024-04-25T10:17:31.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.656+0000 7f2f58000640 1 Processor -- start 2024-04-25T10:17:31.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.656+0000 7f2f58000640 1 -- start start 2024-04-25T10:17:31.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f58000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 0x7f2f500cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f58000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f50144d50 0x7f2f501f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f58000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f58000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cc700 con 0x7f2f501446d0 2024-04-25T10:17:31.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f58000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cc420 con 0x7f2f50144d50 2024-04-25T10:17:31.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f58000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cc9e0 con 0x7f2f501452a0 2024-04-25T10:17:31.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f55600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 0x7f2f500cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f55600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 0x7f2f500cef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35994/0 (socket says 172.21.15.165:35994) 2024-04-25T10:17:31.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.657+0000 7f2f55600640 1 -- 172.21.15.165:0/683725705 learned_addr learned my addr 172.21.15.165:0/683725705 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f54c00640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f50144d50 0x7f2f501f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f56000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f55600640 1 -- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 msgr2=0x7f2f500c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f55600640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f55600640 1 -- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f50144d50 msgr2=0x7f2f501f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f55600640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f50144d50 0x7f2f501f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f55600640 1 -- 172.21.15.165:0/683725705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500c4f00 con 0x7f2f501446d0 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f56000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f55600640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 0x7f2f500cef30 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400cb80 tx=0x7f2f4400cf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44015070 con 0x7f2f501446d0 2024-04-25T10:17:31.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44003b90 con 0x7f2f501446d0 2024-04-25T10:17:31.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4400d040 con 0x7f2f501446d0 2024-04-25T10:17:31.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.658+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500c51e0 con 0x7f2f501446d0 2024-04-25T10:17:31.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.659+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500c57a0 con 0x7f2f501446d0 2024-04-25T10:17:31.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.660+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f50073a00 con 0x7f2f501446d0 2024-04-25T10:17:31.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.661+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f4400a8b0 con 0x7f2f501446d0 2024-04-25T10:17:31.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.662+0000 7f2f4d600640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f3006b500 0x7f2f3006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:31.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.662+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f4408a350 con 0x7f2f501446d0 2024-04-25T10:17:31.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.662+0000 7f2f54c00640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f3006b500 0x7f2f3006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:31.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.663+0000 7f2f54c00640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f3006b500 0x7f2f3006d9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:31.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.666+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f44058fa0 con 0x7f2f501446d0 2024-04-25T10:17:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.863+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2f5006ea20 con 0x7f2f501446d0 2024-04-25T10:17:31.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.864+0000 7f2f4d600640 1 -- 172.21.15.165:0/683725705 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2f440588a0 con 0x7f2f501446d0 2024-04-25T10:17:31.868 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.866+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f3006b500 msgr2=0x7f2f3006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.866+0000 7f2f58000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f3006b500 0x7f2f3006d9c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c034000 tx=0x7f2f3c02e000 comp rx=0 tx=0).stop 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.866+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 msgr2=0x7f2f500cef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.866+0000 7f2f58000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 0x7f2f500cef30 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f2f4400cb80 tx=0x7f2f4400cf80 comp rx=0 tx=0).stop 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 shutdown_connections 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f3006b500 0x7f2f3006d9c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f501452a0 0x7f2f500c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f50144d50 0x7f2f501f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 --2- 172.21.15.165:0/683725705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f501446d0 0x7f2f500cef30 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 >> 172.21.15.165:0/683725705 conn(0x7f2f5006abe0 msgr2=0x7f2f50143840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:31.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 shutdown_connections 2024-04-25T10:17:31.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:31.867+0000 7f2f58000640 1 -- 172.21.15.165:0/683725705 wait complete. 2024-04-25T10:17:31.890 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:32.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.094+0000 7f7fc2200640 1 Processor -- start 2024-04-25T10:17:32.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.095+0000 7f7fc2200640 1 -- start start 2024-04-25T10:17:32.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fc2200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7fbc161890 0x7f7fbc0b6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fc2200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7fbc0b54a0 0x7f7fbc0b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fc2200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fbc0cf1d0 con 0x7f7fbc161240 2024-04-25T10:17:32.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fc2200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc0b7120 con 0x7f7fbc161890 2024-04-25T10:17:32.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fc2200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fbc0b5e30 con 0x7f7fbc0b54a0 2024-04-25T10:17:32.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fbaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7fbc161890 0x7f7fbc0b6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fbaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7fbc161890 0x7f7fbc0b6be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35998/0 (socket says 172.21.15.165:35998) 2024-04-25T10:17:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.096+0000 7f7fbaa00640 1 -- 172.21.15.165:0/1718351042 learned_addr learned my addr 172.21.15.165:0/1718351042 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.097+0000 7f7fbaa00640 1 -- 172.21.15.165:0/1718351042 >> v2:172.21.15.172:3300/0 conn(0x7f7fbc0b54a0 msgr2=0x7f7fbc0b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.097+0000 7f7fbaa00640 1 --2- 172.21.15.165:0/1718351042 >> v2:172.21.15.172:3300/0 conn(0x7f7fbc0b54a0 0x7f7fbc0b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.097+0000 7f7fbaa00640 1 -- 172.21.15.165:0/1718351042 >> v1:172.21.15.172:6789/0 conn(0x7f7fbc161240 legacy=0x7f7fbc0ceac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.097+0000 7f7fbaa00640 1 -- 172.21.15.165:0/1718351042 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc161620 con 0x7f7fbc161890 2024-04-25T10:17:32.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.098+0000 7f7fbaa00640 1 --2- 172.21.15.165:0/1718351042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0b6be0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc08ce10 tx=0x7f7fa802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9adfd2834e1e7a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.098+0000 7f7fba000640 1 -- 172.21.15.165:0/1718351042 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8037070 con 0x7f7fbc161890 2024-04-25T10:17:32.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.098+0000 7f7fba000640 1 -- 172.21.15.165:0/1718351042 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fa8032440 con 0x7f7fbc161890 2024-04-25T10:17:32.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.098+0000 7f7fba000640 1 -- 172.21.15.165:0/1718351042 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8032810 con 0x7f7fbc161890 2024-04-25T10:17:32.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 -- 172.21.15.165:0/1718351042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 msgr2=0x7f7fbc0b6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 --2- 172.21.15.165:0/1718351042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0b6be0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f7fbc08ce10 tx=0x7f7fa802a230 comp rx=0 tx=0).stop 2024-04-25T10:17:32.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 -- 172.21.15.165:0/1718351042 shutdown_connections 2024-04-25T10:17:32.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 --2- 172.21.15.165:0/1718351042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0b6be0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 --2- 172.21.15.165:0/1718351042 >> v2:172.21.15.172:3300/0 conn(0x7f7fbc0b54a0 0x7f7fbc0b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 -- 172.21.15.165:0/1718351042 >> 172.21.15.165:0/1718351042 conn(0x7f7fbc0d04a0 msgr2=0x7f7fbc0d08b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 -- 172.21.15.165:0/1718351042 shutdown_connections 2024-04-25T10:17:32.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.099+0000 7f7fc2200640 1 -- 172.21.15.165:0/1718351042 wait complete. 2024-04-25T10:17:32.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.101+0000 7f7fc2200640 1 Processor -- start 2024-04-25T10:17:32.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.101+0000 7f7fc2200640 1 -- start start 2024-04-25T10:17:32.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fc2200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fbc0b54a0 0x7f7fbc15cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fc2200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fbc161240 0x7f7fbc1f3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fc2200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fc2200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc1623c0 con 0x7f7fbc161890 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fc2200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc1620e0 con 0x7f7fbc161240 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fc2200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fbc1626a0 con 0x7f7fbc0b54a0 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36004/0 (socket says 172.21.15.165:36004) 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fbbe00640 1 -- 172.21.15.165:0/431534994 learned_addr learned my addr 172.21.15.165:0/431534994 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fbb400640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fbc0b54a0 0x7f7fbc15cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.102+0000 7f7fbaa00640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fbc161240 0x7f7fbc1f3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbbe00640 1 -- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fbc0b54a0 msgr2=0x7f7fbc15cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbbe00640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fbc0b54a0 0x7f7fbc15cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbbe00640 1 -- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fbc161240 msgr2=0x7f7fbc1f3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbbe00640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fbc161240 0x7f7fbc1f3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbbe00640 1 -- 172.21.15.165:0/431534994 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fbc15b470 con 0x7f7fbc161890 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbaa00640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fbc161240 0x7f7fbc1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fbbe00640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0c9c40 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400b970 tx=0x7f7fb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4019070 con 0x7f7fbc161890 2024-04-25T10:17:32.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb400cb60 con 0x7f7fbc161890 2024-04-25T10:17:32.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4014440 con 0x7f7fbc161890 2024-04-25T10:17:32.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fbc0cbd10 con 0x7f7fbc161890 2024-04-25T10:17:32.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.103+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fbc0cc2d0 con 0x7f7fbc161890 2024-04-25T10:17:32.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.105+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fbc073a00 con 0x7f7fbc161890 2024-04-25T10:17:32.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.105+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7fb401c070 con 0x7f7fbc161890 2024-04-25T10:17:32.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.106+0000 7f7fb3400640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f9c06b4d0 0x7f7f9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.106+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7fb408a150 con 0x7f7fbc161890 2024-04-25T10:17:32.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.107+0000 7f7fbb400640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f9c06b4d0 0x7f7f9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.107+0000 7f7fbb400640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f9c06b4d0 0x7f7f9c06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7fac007fd0 tx=0x7f7fac009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.111+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7fb4058da0 con 0x7f7fbc161890 2024-04-25T10:17:32.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.312+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7fbc0cc670 con 0x7f7fbc161890 2024-04-25T10:17:32.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.313+0000 7f7fb3400640 1 -- 172.21.15.165:0/431534994 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7fb40586a0 con 0x7f7fbc161890 2024-04-25T10:17:32.317 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:17:32.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.315+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f9c06b4d0 msgr2=0x7f7f9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.315+0000 7f7fc2200640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f9c06b4d0 0x7f7f9c06d990 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f7fac007fd0 tx=0x7f7fac009040 comp rx=0 tx=0).stop 2024-04-25T10:17:32.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.315+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 msgr2=0x7f7fbc0c9c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.315+0000 7f7fc2200640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0c9c40 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400b970 tx=0x7f7fb400be30 comp rx=0 tx=0).stop 2024-04-25T10:17:32.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 shutdown_connections 2024-04-25T10:17:32.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7f9c06b4d0 0x7f7f9c06d990 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7fbc161890 0x7f7fbc0c9c40 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7fbc161240 0x7f7fbc1f3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 --2- 172.21.15.165:0/431534994 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7fbc0b54a0 0x7f7fbc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 >> 172.21.15.165:0/431534994 conn(0x7f7fbc0d04a0 msgr2=0x7f7fbc0b34f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 shutdown_connections 2024-04-25T10:17:32.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.316+0000 7f7fc2200640 1 -- 172.21.15.165:0/431534994 wait complete. 2024-04-25T10:17:32.339 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:17:32.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.533+0000 7fa1ad200640 1 Processor -- start 2024-04-25T10:17:32.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.534+0000 7fa1ad200640 1 -- start start 2024-04-25T10:17:32.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.535+0000 7fa1ad200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa1a80d0d40 0x7fa1a8146830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.535+0000 7fa1ad200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a80b5490 con 0x7fa1a80b1140 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.535+0000 7fa1ad200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1a80b5730 con 0x7fa1a80b1fb0 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.535+0000 7fa1ad200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a815ea60 con 0x7fa1a80d0d40 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a7e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa1a80d0d40 0x7fa1a8146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa1a80b1140 0x7fa1a80b3690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41262/0 (socket says 172.21.15.165:41262) 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a7400640 1 -- 172.21.15.165:0/202230084 learned_addr learned my addr 172.21.15.165:0/202230084 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a7e00640 1 -- 172.21.15.165:0/202230084 >> v1:172.21.15.172:6789/0 conn(0x7fa1a80b1fb0 legacy=0x7fa1a80d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a7e00640 1 -- 172.21.15.165:0/202230084 >> v1:172.21.15.165:6789/0 conn(0x7fa1a80b1140 legacy=0x7fa1a80b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a7e00640 1 -- 172.21.15.165:0/202230084 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80b5a10 con 0x7fa1a80d0d40 2024-04-25T10:17:32.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.536+0000 7fa1a6000640 1 -- 172.21.15.165:0/202230084 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413911392 0 0) 0x7fa1a80b5490 con 0x7fa1a80b1140 2024-04-25T10:17:32.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.537+0000 7fa1a7e00640 1 --2- 172.21.15.165:0/202230084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a8146830 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009d90 tx=0x7fa1a002a580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2eee7c119aa5af server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1a6000640 1 -- 172.21.15.165:0/202230084 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0007740 con 0x7fa1a80d0d40 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1a6000640 1 -- 172.21.15.165:0/202230084 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a00078e0 con 0x7fa1a80d0d40 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1ad200640 1 -- 172.21.15.165:0/202230084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 msgr2=0x7fa1a8146830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1ad200640 1 --2- 172.21.15.165:0/202230084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a8146830 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fa1a0009d90 tx=0x7fa1a002a580 comp rx=0 tx=0).stop 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1ad200640 1 -- 172.21.15.165:0/202230084 shutdown_connections 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1ad200640 1 --2- 172.21.15.165:0/202230084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a8146830 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1ad200640 1 -- 172.21.15.165:0/202230084 >> 172.21.15.165:0/202230084 conn(0x7fa1a80728b0 msgr2=0x7fa1a8099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.538+0000 7fa1ad200640 1 -- 172.21.15.165:0/202230084 shutdown_connections 2024-04-25T10:17:32.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.539+0000 7fa1ad200640 1 -- 172.21.15.165:0/202230084 wait complete. 2024-04-25T10:17:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.540+0000 7fa1ad200640 1 Processor -- start 2024-04-25T10:17:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.540+0000 7fa1ad200640 1 -- start start 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.541+0000 7fa1ad200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1a80b1140 0x7fa1a809a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.541+0000 7fa1ad200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a80b1fb0 0x7fa1a814e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.541+0000 7fa1ad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a81598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.541+0000 7fa1ad200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a81601e0 con 0x7fa1a80d0d40 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.541+0000 7fa1ad200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a80b3da0 con 0x7fa1a80b1fb0 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.541+0000 7fa1ad200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a8160360 con 0x7fa1a80b1140 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a81598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a81598a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36030/0 (socket says 172.21.15.165:36030) 2024-04-25T10:17:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 -- 172.21.15.165:0/2051876487 learned_addr learned my addr 172.21.15.165:0/2051876487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 -- 172.21.15.165:0/2051876487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1a80b1140 msgr2=0x7fa1a809a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1a80b1140 0x7fa1a809a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 -- 172.21.15.165:0/2051876487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a80b1fb0 msgr2=0x7fa1a814e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a80b1fb0 0x7fa1a814e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 -- 172.21.15.165:0/2051876487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a8157d70 con 0x7fa1a80d0d40 2024-04-25T10:17:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.542+0000 7fa1a7e00640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a81598a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fa1a002ab70 tx=0x7fa1a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.543+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004450 con 0x7fa1a80d0d40 2024-04-25T10:17:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.543+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a00045f0 con 0x7fa1a80d0d40 2024-04-25T10:17:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.543+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0003e10 con 0x7fa1a80d0d40 2024-04-25T10:17:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.543+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a8158050 con 0x7fa1a80d0d40 2024-04-25T10:17:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.543+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a809acc0 con 0x7fa1a80d0d40 2024-04-25T10:17:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.544+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a8073990 con 0x7fa1a80d0d40 2024-04-25T10:17:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.545+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa1a0039040 con 0x7fa1a80d0d40 2024-04-25T10:17:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.546+0000 7fa19f400640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa18406b4d0 0x7fa18406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.546+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa1a00abca0 con 0x7fa1a80d0d40 2024-04-25T10:17:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.546+0000 7fa1a7400640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa18406b4d0 0x7fa18406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:32.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.547+0000 7fa1a7400640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa18406b4d0 0x7fa18406d990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa198000c00 tx=0x7fa1980079c0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:32.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.550+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa1a0040030 con 0x7fa1a80d0d40 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.762+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fa1a8160850 con 0x7fa1a80d0d40 2024-04-25T10:17:32.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.762+0000 7fa19f400640 1 -- 172.21.15.165:0/2051876487 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v148) v1 ==== 80+0+0 (secure 0 0 0) 0x7fa1a007a9b0 con 0x7fa1a80d0d40 2024-04-25T10:17:32.767 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:17:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa18406b4d0 msgr2=0x7fa18406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa18406b4d0 0x7fa18406d990 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fa198000c00 tx=0x7fa1980079c0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 msgr2=0x7fa1a81598a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a81598a0 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fa1a002ab70 tx=0x7fa1a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 shutdown_connections 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa18406b4d0 0x7fa18406d990 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1a80d0d40 0x7fa1a81598a0 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1a80b1fb0 0x7fa1a814e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 --2- 172.21.15.165:0/2051876487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1a80b1140 0x7fa1a809a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 >> 172.21.15.165:0/2051876487 conn(0x7fa1a80728b0 msgr2=0x7fa1a8099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 shutdown_connections 2024-04-25T10:17:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:32.766+0000 7fa1ad200640 1 -- 172.21.15.165:0/2051876487 wait complete. 2024-04-25T10:17:32.789 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:17:32.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:32.789 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:17:33.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:17:33.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:17:33.792 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:17:33.832 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:17:33.832 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:33.875 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:33.875 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:17:33.875 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:33.918 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:33.918 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:17:33.918 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:33.970 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:33.970 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.179+0000 7fdefb600640 1 Processor -- start 2024-04-25T10:17:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.179+0000 7fdefb600640 1 -- start start 2024-04-25T10:17:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdefb600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdef415dc00 0x7fdef40c6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdefb600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef40d16c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdefb600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdef415dfe0 con 0x7fdef40d1c00 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdefb600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdef40d2060 con 0x7fdef415dc00 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdefb600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40d00c0 con 0x7fdef40ceb20 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdef9600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fdef40d1c00 0x7fdef40ccc40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41266/0 (socket says 172.21.15.165:41266) 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.181+0000 7fdef9600640 1 -- 172.21.15.165:0/3506799384 learned_addr learned my addr 172.21.15.165:0/3506799384 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3e00640 1 --2- 172.21.15.165:0/3506799384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef40d16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef8c00640 1 --2- 172.21.15.165:0/3506799384 >> v2:172.21.15.172:3300/0 conn(0x7fdef415dc00 0x7fdef40c6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3400640 1 -- 172.21.15.165:0/3506799384 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 629913887 0 0) 0x7fdeec001110 con 0x7fdef40d1c00 2024-04-25T10:17:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3400640 1 -- 172.21.15.165:0/3506799384 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fded8003520 con 0x7fdef40d1c00 2024-04-25T10:17:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3e00640 1 -- 172.21.15.165:0/3506799384 >> v2:172.21.15.172:3300/0 conn(0x7fdef415dc00 msgr2=0x7fdef40c6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3e00640 1 --2- 172.21.15.165:0/3506799384 >> v2:172.21.15.172:3300/0 conn(0x7fdef415dc00 0x7fdef40c6df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3e00640 1 -- 172.21.15.165:0/3506799384 >> v1:172.21.15.165:6789/0 conn(0x7fdef40d1c00 legacy=0x7fdef40ccc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.182+0000 7fdef3e00640 1 -- 172.21.15.165:0/3506799384 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef40d03a0 con 0x7fdef40ceb20 2024-04-25T10:17:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.183+0000 7fdef3e00640 1 --2- 172.21.15.165:0/3506799384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef40d16c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0097a0 tx=0x7fdedc02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=58b9289b654eb10e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.184+0000 7fdef3400640 1 -- 172.21.15.165:0/3506799384 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc037070 con 0x7fdef40ceb20 2024-04-25T10:17:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.184+0000 7fdef3400640 1 -- 172.21.15.165:0/3506799384 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc032440 con 0x7fdef40ceb20 2024-04-25T10:17:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.184+0000 7fdef3400640 1 -- 172.21.15.165:0/3506799384 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0327f0 con 0x7fdef40ceb20 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 -- 172.21.15.165:0/3506799384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 msgr2=0x7fdef40d16c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 --2- 172.21.15.165:0/3506799384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef40d16c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fdedc0097a0 tx=0x7fdedc02a180 comp rx=0 tx=0).stop 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 -- 172.21.15.165:0/3506799384 shutdown_connections 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 --2- 172.21.15.165:0/3506799384 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef40d16c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 --2- 172.21.15.165:0/3506799384 >> v2:172.21.15.172:3300/0 conn(0x7fdef415dc00 0x7fdef40c6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 -- 172.21.15.165:0/3506799384 >> 172.21.15.165:0/3506799384 conn(0x7fdef406f5d0 msgr2=0x7fdef406f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 -- 172.21.15.165:0/3506799384 shutdown_connections 2024-04-25T10:17:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.185+0000 7fdefb600640 1 -- 172.21.15.165:0/3506799384 wait complete. 2024-04-25T10:17:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.187+0000 7fdefb600640 1 Processor -- start 2024-04-25T10:17:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.187+0000 7fdefb600640 1 -- start start 2024-04-25T10:17:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdefb600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef4145eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdefb600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 0x7fdef40ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdefb600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef415dc00 0x7fdef4142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdefb600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40cc130 con 0x7fdef40d1c00 2024-04-25T10:17:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdefb600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40cc720 con 0x7fdef415dc00 2024-04-25T10:17:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdefb600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdef40cfd90 con 0x7fdef40ceb20 2024-04-25T10:17:34.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdef3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 0x7fdef40ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdef3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 0x7fdef40ca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36042/0 (socket says 172.21.15.165:36042) 2024-04-25T10:17:34.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.188+0000 7fdef3e00640 1 -- 172.21.15.165:0/807070478 learned_addr learned my addr 172.21.15.165:0/807070478 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef8c00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef4145eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef3e00640 1 -- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 msgr2=0x7fdef4145eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef3e00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef4145eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef3e00640 1 -- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef415dc00 msgr2=0x7fdef4142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef3e00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef415dc00 0x7fdef4142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef3e00640 1 -- 172.21.15.165:0/807070478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef40cad80 con 0x7fdef40d1c00 2024-04-25T10:17:34.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef8c00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef4145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef3e00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 0x7fdef40ca5f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fdedc035040 tx=0x7fdedc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc037040 con 0x7fdef40d1c00 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdedc0044c0 con 0x7fdef40d1c00 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdedc0325f0 con 0x7fdef40d1c00 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef40cb060 con 0x7fdef40d1c00 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.189+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef40cb620 con 0x7fdef40d1c00 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.191+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdef4073990 con 0x7fdef40d1c00 2024-04-25T10:17:34.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.192+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdedc044020 con 0x7fdef40d1c00 2024-04-25T10:17:34.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.193+0000 7fdef0c00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.193+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdedc0aa5b0 con 0x7fdef40d1c00 2024-04-25T10:17:34.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.193+0000 7fdef8c00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.194+0000 7fdef8c00640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fded006b4d0 0x7fded006d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fdee4007440 tx=0x7fdee400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.197+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdedc0793b0 con 0x7fdef40d1c00 2024-04-25T10:17:34.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.435+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdef40cbd30 con 0x7fdef40d1c00 2024-04-25T10:17:34.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.436+0000 7fdef0c00640 1 -- 172.21.15.165:0/807070478 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fdedc078cb0 con 0x7fdef40d1c00 2024-04-25T10:17:34.441 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:34.441 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":104,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:17:28.059079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:17:34.441 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 104 2024-04-25T10:17:34.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fded006b4d0 msgr2=0x7fded006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fded006b4d0 0x7fded006d990 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fdee4007440 tx=0x7fdee400b040 comp rx=0 tx=0).stop 2024-04-25T10:17:34.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 msgr2=0x7fdef40ca5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 0x7fdef40ca5f0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fdedc035040 tx=0x7fdedc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 shutdown_connections 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fded006b4d0 0x7fded006d990 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdef415dc00 0x7fdef4142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdef40d1c00 0x7fdef40ca5f0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 --2- 172.21.15.165:0/807070478 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdef40ceb20 0x7fdef4145eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 >> 172.21.15.165:0/807070478 conn(0x7fdef406f5d0 msgr2=0x7fdef40b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 shutdown_connections 2024-04-25T10:17:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.439+0000 7fdefb600640 1 -- 172.21.15.165:0/807070478 wait complete. 2024-04-25T10:17:34.462 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.654+0000 7f9f4fe00640 1 Processor -- start 2024-04-25T10:17:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.654+0000 7f9f4fe00640 1 -- start start 2024-04-25T10:17:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9f50144440 0x7f9f50070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9f500cda30 0x7f9f5015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5015bf30 con 0x7f9f50071530 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4fe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f500719a0 con 0x7f9f50144440 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5015c860 con 0x7f9f500cda30 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9f50144440 0x7f9f50070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4e000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9f50071530 0x7f9f500cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41276/0 (socket says 172.21.15.165:41276) 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4e000640 1 -- 172.21.15.165:0/3947946139 learned_addr learned my addr 172.21.15.165:0/3947946139 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.656+0000 7f9f4f400640 1 --2- 172.21.15.165:0/3947946139 >> v2:172.21.15.172:3300/0 conn(0x7f9f500cda30 0x7f9f5015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.657+0000 7f9f4ea00640 1 -- 172.21.15.165:0/3947946139 >> v2:172.21.15.172:3300/0 conn(0x7f9f500cda30 msgr2=0x7f9f5015ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.657+0000 7f9f4ea00640 1 --2- 172.21.15.165:0/3947946139 >> v2:172.21.15.172:3300/0 conn(0x7f9f500cda30 0x7f9f5015ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.657+0000 7f9f4ea00640 1 -- 172.21.15.165:0/3947946139 >> v1:172.21.15.165:6789/0 conn(0x7f9f50071530 legacy=0x7f9f500cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.657+0000 7f9f4ea00640 1 -- 172.21.15.165:0/3947946139 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5015c210 con 0x7f9f50144440 2024-04-25T10:17:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.657+0000 7f9f4d600640 1 -- 172.21.15.165:0/3947946139 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463228422 0 0) 0x7f9f5015bf30 con 0x7f9f50071530 2024-04-25T10:17:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.658+0000 7f9f4ea00640 1 --2- 172.21.15.165:0/3947946139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50144440 0x7f9f50070ff0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9f500b2280 tx=0x7f9f4002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abe783b92978d8f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.658+0000 7f9f4d600640 1 -- 172.21.15.165:0/3947946139 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40037070 con 0x7f9f50144440 2024-04-25T10:17:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.658+0000 7f9f4d600640 1 -- 172.21.15.165:0/3947946139 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40032440 con 0x7f9f50144440 2024-04-25T10:17:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.658+0000 7f9f4d600640 1 -- 172.21.15.165:0/3947946139 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40032810 con 0x7f9f50144440 2024-04-25T10:17:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 -- 172.21.15.165:0/3947946139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50144440 msgr2=0x7f9f50070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/3947946139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50144440 0x7f9f50070ff0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f9f500b2280 tx=0x7f9f4002a230 comp rx=0 tx=0).stop 2024-04-25T10:17:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 -- 172.21.15.165:0/3947946139 shutdown_connections 2024-04-25T10:17:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/3947946139 >> v2:172.21.15.172:3300/0 conn(0x7f9f500cda30 0x7f9f5015ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/3947946139 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50144440 0x7f9f50070ff0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 -- 172.21.15.165:0/3947946139 >> 172.21.15.165:0/3947946139 conn(0x7f9f5006abe0 msgr2=0x7f9f5006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 -- 172.21.15.165:0/3947946139 shutdown_connections 2024-04-25T10:17:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.659+0000 7f9f4fe00640 1 -- 172.21.15.165:0/3947946139 wait complete. 2024-04-25T10:17:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.661+0000 7f9f4fe00640 1 Processor -- start 2024-04-25T10:17:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.661+0000 7f9f4fe00640 1 -- start start 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 0x7f9f501f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f500cda30 0x7f9f500c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f50144440 0x7f9f500a9850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015bc30 con 0x7f9f50071530 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f500cdea0 con 0x7f9f500cda30 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5015d230 con 0x7f9f50144440 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 0x7f9f501f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 0x7f9f501f7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36060/0 (socket says 172.21.15.165:36060) 2024-04-25T10:17:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 -- 172.21.15.165:0/2840884348 learned_addr learned my addr 172.21.15.165:0/2840884348 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 -- 172.21.15.165:0/2840884348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f50144440 msgr2=0x7f9f500a9850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f50144440 0x7f9f500a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 -- 172.21.15.165:0/2840884348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f500cda30 msgr2=0x7f9f500c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f500cda30 0x7f9f500c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.662+0000 7f9f4ea00640 1 -- 172.21.15.165:0/2840884348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f50132b70 con 0x7f9f50071530 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.663+0000 7f9f4ea00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 0x7f9f501f7080 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9f40000c00 tx=0x7f9f400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.663+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40037040 con 0x7f9f50071530 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.663+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f40042070 con 0x7f9f50071530 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.663+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f40030bd0 con 0x7f9f50071530 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.663+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f50132e50 con 0x7f9f50071530 2024-04-25T10:17:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.663+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f501336f0 con 0x7f9f50071530 2024-04-25T10:17:34.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.664+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f50073990 con 0x7f9f50071530 2024-04-25T10:17:34.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.665+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f40047460 con 0x7f9f50071530 2024-04-25T10:17:34.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.666+0000 7f9f47400640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f3006b4d0 0x7f9f3006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:34.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.666+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9f400aafd0 con 0x7f9f50071530 2024-04-25T10:17:34.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.667+0000 7f9f4e000640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f3006b4d0 0x7f9f3006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.667+0000 7f9f4e000640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f3006b4d0 0x7f9f3006d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007fd0 tx=0x7f9f3c007920 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:34.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.671+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f40032e20 con 0x7f9f50071530 2024-04-25T10:17:34.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.860+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f50133020 con 0x7f9f50071530 2024-04-25T10:17:34.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.862+0000 7f9f47400640 1 -- 172.21.15.165:0/2840884348 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f9f40079cb0 con 0x7f9f50071530 2024-04-25T10:17:34.867 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:34.867 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":148,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:44.518826+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f3006b4d0 msgr2=0x7f9f3006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f3006b4d0 0x7f9f3006d990 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c007fd0 tx=0x7f9f3c007920 comp rx=0 tx=0).stop 2024-04-25T10:17:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 msgr2=0x7f9f501f7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 0x7f9f501f7080 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7f9f40000c00 tx=0x7f9f400046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 shutdown_connections 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f3006b4d0 0x7f9f3006d990 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f50144440 0x7f9f500a9850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f500cda30 0x7f9f500c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 --2- 172.21.15.165:0/2840884348 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f50071530 0x7f9f501f7080 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.865+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 >> 172.21.15.165:0/2840884348 conn(0x7f9f5006abe0 msgr2=0x7f9f500cd320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.866+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 shutdown_connections 2024-04-25T10:17:34.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:34.866+0000 7f9f4fe00640 1 -- 172.21.15.165:0/2840884348 wait complete. 2024-04-25T10:17:34.887 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:16:40.787562+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '147', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:16:42.000970+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:34.888 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:35.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.070+0000 7f10af200640 1 Processor -- start 2024-04-25T10:17:35.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.070+0000 7f10af200640 1 -- start start 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10af200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f10b013d9f0 0x7f10b013df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10af200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10b006e860 0x7f10b00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10af200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10b006d390 con 0x7f10b00b0730 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10af200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b006d670 con 0x7f10b013d9f0 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10af200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10b006d950 con 0x7f10b006e860 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10ade00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f10b013d9f0 0x7f10b013df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10a5400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f10b00b0730 0x7f10b006e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41278/0 (socket says 172.21.15.165:41278) 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.071+0000 7f10a5400640 1 -- 172.21.15.165:0/3302088312 learned_addr learned my addr 172.21.15.165:0/3302088312 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ae800640 1 --2- 172.21.15.165:0/3302088312 >> v2:172.21.15.172:3300/0 conn(0x7f10b006e860 0x7f10b00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ade00640 1 -- 172.21.15.165:0/3302088312 >> v2:172.21.15.172:3300/0 conn(0x7f10b006e860 msgr2=0x7f10b00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ade00640 1 --2- 172.21.15.165:0/3302088312 >> v2:172.21.15.172:3300/0 conn(0x7f10b006e860 0x7f10b00b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ade00640 1 -- 172.21.15.165:0/3302088312 >> v1:172.21.15.165:6789/0 conn(0x7f10b00b0730 legacy=0x7f10b006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ade00640 1 -- 172.21.15.165:0/3302088312 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b006dc30 con 0x7f10b013d9f0 2024-04-25T10:17:35.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ad400640 1 -- 172.21.15.165:0/3302088312 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174931809 0 0) 0x7f10b006d390 con 0x7f10b00b0730 2024-04-25T10:17:35.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.072+0000 7f10ae800640 1 --2- 172.21.15.165:0/3302088312 >> v2:172.21.15.172:3300/0 conn(0x7f10b006e860 0x7f10b00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.073+0000 7f10ade00640 1 --2- 172.21.15.165:0/3302088312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b013d9f0 0x7f10b013df30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f10b00b2860 tx=0x7f10a002a080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e055d3ff444e7cf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.073+0000 7f10ad400640 1 -- 172.21.15.165:0/3302088312 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0037070 con 0x7f10b013d9f0 2024-04-25T10:17:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.073+0000 7f10ad400640 1 -- 172.21.15.165:0/3302088312 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a002add0 con 0x7f10b013d9f0 2024-04-25T10:17:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10ad400640 1 -- 172.21.15.165:0/3302088312 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a0032660 con 0x7f10b013d9f0 2024-04-25T10:17:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 -- 172.21.15.165:0/3302088312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b013d9f0 msgr2=0x7f10b013df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 --2- 172.21.15.165:0/3302088312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b013d9f0 0x7f10b013df30 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 crypto rx=0x7f10b00b2860 tx=0x7f10a002a080 comp rx=0 tx=0).stop 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 -- 172.21.15.165:0/3302088312 shutdown_connections 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 --2- 172.21.15.165:0/3302088312 >> v2:172.21.15.172:3300/0 conn(0x7f10b006e860 0x7f10b00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 --2- 172.21.15.165:0/3302088312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b013d9f0 0x7f10b013df30 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 -- 172.21.15.165:0/3302088312 >> 172.21.15.165:0/3302088312 conn(0x7f10b0061cc0 msgr2=0x7f10b006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 -- 172.21.15.165:0/3302088312 shutdown_connections 2024-04-25T10:17:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.074+0000 7f10af200640 1 -- 172.21.15.165:0/3302088312 wait complete. 2024-04-25T10:17:35.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.076+0000 7f10af200640 1 Processor -- start 2024-04-25T10:17:35.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.076+0000 7f10af200640 1 -- start start 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10af200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10b006e860 0x7f10b0139fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10af200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 0x7f10b013c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10af200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10b013d9f0 0x7f10b01547a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10af200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00ce880 con 0x7f10b00b0730 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10af200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10b013e680 con 0x7f10b013d9f0 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10af200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10b00cea00 con 0x7f10b006e860 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 0x7f10b013c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10a5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 0x7f10b013c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36074/0 (socket says 172.21.15.165:36074) 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.077+0000 7f10a5400640 1 -- 172.21.15.165:0/701666444 learned_addr learned my addr 172.21.15.165:0/701666444 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10ade00640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10b006e860 0x7f10b0139fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a5400640 1 -- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10b006e860 msgr2=0x7f10b0139fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a5400640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10b006e860 0x7f10b0139fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a5400640 1 -- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10b013d9f0 msgr2=0x7f10b01547a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10ae800640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10b013d9f0 0x7f10b01547a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a5400640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10b013d9f0 0x7f10b01547a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a5400640 1 -- 172.21.15.165:0/701666444 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10b013a520 con 0x7f10b00b0730 2024-04-25T10:17:35.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a5400640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 0x7f10b013c490 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f109800c500 tx=0x7f109800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098018070 con 0x7f10b00b0730 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1098007580 con 0x7f10b00b0730 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098004a20 con 0x7f10b00b0730 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.078+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10b013b110 con 0x7f10b00b0730 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.079+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10b013b6d0 con 0x7f10b00b0730 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.080+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10b0073990 con 0x7f10b00b0730 2024-04-25T10:17:35.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.081+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1098004bc0 con 0x7f10b00b0730 2024-04-25T10:17:35.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.081+0000 7f10a6a00640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108c06b4d0 0x7f108c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.082+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1098089650 con 0x7f10b00b0730 2024-04-25T10:17:35.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.082+0000 7f10ade00640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108c06b4d0 0x7f108c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.083+0000 7f10ade00640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108c06b4d0 0x7f108c06d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f10a0034000 tx=0x7f10a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.086+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1098058330 con 0x7f10b00b0730 2024-04-25T10:17:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.283+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10b006b0f0 con 0x7f10b00b0730 2024-04-25T10:17:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.283+0000 7f10a6a00640 1 -- 172.21.15.165:0/701666444 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1098057c30 con 0x7f10b00b0730 2024-04-25T10:17:35.288 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:17:35.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108c06b4d0 msgr2=0x7f108c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108c06b4d0 0x7f108c06d990 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f10a0034000 tx=0x7f10a002e000 comp rx=0 tx=0).stop 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 msgr2=0x7f10b013c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 0x7f10b013c490 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7f109800c500 tx=0x7f109800c9c0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 shutdown_connections 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108c06b4d0 0x7f108c06d990 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10b013d9f0 0x7f10b01547a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10b00b0730 0x7f10b013c490 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 --2- 172.21.15.165:0/701666444 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10b006e860 0x7f10b0139fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 >> 172.21.15.165:0/701666444 conn(0x7f10b0061cc0 msgr2=0x7f10b013f680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 shutdown_connections 2024-04-25T10:17:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.286+0000 7f10af200640 1 -- 172.21.15.165:0/701666444 wait complete. 2024-04-25T10:17:35.309 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:17:35.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.505+0000 7faf99800640 1 Processor -- start 2024-04-25T10:17:35.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.506+0000 7faf99800640 1 -- start start 2024-04-25T10:17:35.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf99800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 0x7faf940cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf99800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faf9406bff0 0x7faf940b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf99800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf9415e110 con 0x7faf940b5970 2024-04-25T10:17:35.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf99800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf9406d000 con 0x7faf9406bff0 2024-04-25T10:17:35.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf99800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf940b2fd0 con 0x7faf9415dd30 2024-04-25T10:17:35.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf98e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7faf940b5970 0x7faf94161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41286/0 (socket says 172.21.15.165:41286) 2024-04-25T10:17:35.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.507+0000 7faf98e00640 1 -- 172.21.15.165:0/2732440669 learned_addr learned my addr 172.21.15.165:0/2732440669 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:35.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf93e00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 0x7faf940cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1753725465 0 0) 0x7faf9415e110 con 0x7faf940b5970 2024-04-25T10:17:35.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf78003520 con 0x7faf940b5970 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf93400640 1 --2- 172.21.15.165:0/2732440669 >> v2:172.21.15.172:3300/0 conn(0x7faf9406bff0 0x7faf940b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4116898052 0 0) 0x7faf78003520 con 0x7faf940b5970 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 msgr2=0x7faf940cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 0x7faf940cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 >> v2:172.21.15.172:3300/0 conn(0x7faf9406bff0 msgr2=0x7faf940b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 --2- 172.21.15.165:0/2732440669 >> v2:172.21.15.172:3300/0 conn(0x7faf9406bff0 0x7faf940b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9406c480 con 0x7faf940b5970 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7faf8c0019f0 con 0x7faf940b5970 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 >> v1:172.21.15.165:6789/0 conn(0x7faf940b5970 legacy=0x7faf94161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf780039b0 0x7faf78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf78004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 0x7faf780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9406d000 con 0x7faf78004800 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf940b2fd0 con 0x7faf780039b0 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.508+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7800ba70 con 0x7faf9415dd30 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93400640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf78004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93e00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf780039b0 0x7faf78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf98e00640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 0x7faf780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93400640 1 -- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 msgr2=0x7faf780051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93400640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf9415dd30 0x7faf780051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93400640 1 -- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf780039b0 msgr2=0x7faf78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93400640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf780039b0 0x7faf78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.509+0000 7faf93400640 1 -- 172.21.15.165:0/2732440669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf78003520 con 0x7faf78004800 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.510+0000 7faf93400640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf78004c80 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00ba00 tx=0x7faf78005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8dbedba7a4219d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.510+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c030070 con 0x7faf78004800 2024-04-25T10:17:35.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.510+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c02adb0 con 0x7faf78004800 2024-04-25T10:17:35.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.510+0000 7faf92a00640 1 -- 172.21.15.165:0/2732440669 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c00b600 con 0x7faf78004800 2024-04-25T10:17:35.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf99800640 1 -- 172.21.15.165:0/2732440669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 msgr2=0x7faf78004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf99800640 1 --2- 172.21.15.165:0/2732440669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf78004c80 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7faf7c00ba00 tx=0x7faf78005830 comp rx=0 tx=0).stop 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf98e00640 1 -- 172.21.15.165:0/2732440669 reap_dead start 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf99800640 1 -- 172.21.15.165:0/2732440669 shutdown_connections 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf99800640 1 -- 172.21.15.165:0/2732440669 >> 172.21.15.165:0/2732440669 conn(0x7faf940d0610 msgr2=0x7faf940d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf99800640 1 -- 172.21.15.165:0/2732440669 shutdown_connections 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.511+0000 7faf99800640 1 -- 172.21.15.165:0/2732440669 wait complete. 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.513+0000 7faf99800640 1 Processor -- start 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.513+0000 7faf99800640 1 -- start start 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf99800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf94153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf99800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9415f260 0x7faf9415f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf99800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf94152740 0x7faf94152bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf99800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9406cd20 con 0x7faf78004800 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf99800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9406ca40 con 0x7faf9415f260 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf99800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf940d3170 con 0x7faf94152740 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf93e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf94153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf93e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf94153a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36082/0 (socket says 172.21.15.165:36082) 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.514+0000 7faf93e00640 1 -- 172.21.15.165:0/3746236504 learned_addr learned my addr 172.21.15.165:0/3746236504 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf98e00640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf94152740 0x7faf94152bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93400640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9415f260 0x7faf9415f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93e00640 1 -- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf94152740 msgr2=0x7faf94152bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93e00640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf94152740 0x7faf94152bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93e00640 1 -- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9415f260 msgr2=0x7faf9415f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93e00640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9415f260 0x7faf9415f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93e00640 1 -- 172.21.15.165:0/3746236504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf9415ff20 con 0x7faf78004800 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf93e00640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf94153a80 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7faf8400f580 tx=0x7faf8400fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84019020 con 0x7faf78004800 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf84014870 con 0x7faf78004800 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf94160200 con 0x7faf78004800 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.515+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf84014c60 con 0x7faf78004800 2024-04-25T10:17:35.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.516+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf941607c0 con 0x7faf78004800 2024-04-25T10:17:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.517+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf94073990 con 0x7faf78004800 2024-04-25T10:17:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.518+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf8401c020 con 0x7faf78004800 2024-04-25T10:17:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.519+0000 7faf8be00640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf7006b4d0 0x7faf7006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.519+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faf8408ab70 con 0x7faf78004800 2024-04-25T10:17:35.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.519+0000 7faf93400640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf7006b4d0 0x7faf7006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.520+0000 7faf93400640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf7006b4d0 0x7faf7006d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf780057e0 tx=0x7faf7c030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.524+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf84023020 con 0x7faf78004800 2024-04-25T10:17:35.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.719+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf940b4290 con 0x7faf78004800 2024-04-25T10:17:35.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.719+0000 7faf8be00640 1 -- 172.21.15.165:0/3746236504 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v148) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf840597d0 con 0x7faf78004800 2024-04-25T10:17:35.723 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:17:35.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf7006b4d0 msgr2=0x7faf7006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf7006b4d0 0x7faf7006d990 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7faf780057e0 tx=0x7faf7c030000 comp rx=0 tx=0).stop 2024-04-25T10:17:35.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 msgr2=0x7faf94153a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf94153a80 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7faf8400f580 tx=0x7faf8400fa40 comp rx=0 tx=0).stop 2024-04-25T10:17:35.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 shutdown_connections 2024-04-25T10:17:35.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf7006b4d0 0x7faf7006d990 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf94152740 0x7faf94152bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf9415f260 0x7faf9415f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 --2- 172.21.15.165:0/3746236504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf78004800 0x7faf94153a80 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 >> 172.21.15.165:0/3746236504 conn(0x7faf940d0610 msgr2=0x7faf940b25c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 shutdown_connections 2024-04-25T10:17:35.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.722+0000 7faf99800640 1 -- 172.21.15.165:0/3746236504 wait complete. 2024-04-25T10:17:35.744 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:17:35.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.945+0000 7f6382400640 1 Processor -- start 2024-04-25T10:17:35.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.946+0000 7f6382400640 1 -- start start 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.947+0000 7f6382400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c0cadb0 0x7f637c0cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.947+0000 7f6382400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f637c0ccfe0 0x7f637c0cd470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.947+0000 7f6382400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637c0afc80 con 0x7f637c15de80 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.947+0000 7f6382400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f637c0cb210 con 0x7f637c0ccfe0 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.947+0000 7f6382400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0ce4d0 con 0x7f637c0cadb0 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f637c15de80 0x7f637c0cefb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:41294/0 (socket says 172.21.15.165:41294) 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637b400640 1 -- 172.21.15.165:0/1783561409 learned_addr learned my addr 172.21.15.165:0/1783561409 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:35.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637be00640 1 --2- 172.21.15.165:0/1783561409 >> v2:172.21.15.165:3300/0 conn(0x7f637c0ccfe0 0x7f637c0cd470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637a000640 1 -- 172.21.15.165:0/1783561409 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1673848308 0 0) 0x7f637c0afc80 con 0x7f637c15de80 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637a000640 1 -- 172.21.15.165:0/1783561409 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6360003520 con 0x7f637c15de80 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637aa00640 1 --2- 172.21.15.165:0/1783561409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c0cadb0 0x7f637c0cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637be00640 1 -- 172.21.15.165:0/1783561409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c0cadb0 msgr2=0x7f637c0cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637be00640 1 --2- 172.21.15.165:0/1783561409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c0cadb0 0x7f637c0cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637be00640 1 -- 172.21.15.165:0/1783561409 >> v1:172.21.15.165:6789/0 conn(0x7f637c15de80 legacy=0x7f637c0cefb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637a000640 1 -- 172.21.15.165:0/1783561409 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046944971 0 0) 0x7f6360003520 con 0x7f637c15de80 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.948+0000 7f637be00640 1 -- 172.21.15.165:0/1783561409 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0ce7b0 con 0x7f637c0ccfe0 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.949+0000 7f637be00640 1 --2- 172.21.15.165:0/1783561409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c0cd470 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f63740097a0 tx=0x7f637402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cde22a5e3f35206 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f637a000640 1 -- 172.21.15.165:0/1783561409 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6374037070 con 0x7f637c0ccfe0 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f637a000640 1 -- 172.21.15.165:0/1783561409 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f637402aba0 con 0x7f637c0ccfe0 2024-04-25T10:17:35.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f637a000640 1 -- 172.21.15.165:0/1783561409 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c001670 con 0x7f637c0ccfe0 2024-04-25T10:17:35.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f6382400640 1 -- 172.21.15.165:0/1783561409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 msgr2=0x7f637c0cd470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f6382400640 1 --2- 172.21.15.165:0/1783561409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c0cd470 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7f63740097a0 tx=0x7f637402a110 comp rx=0 tx=0).stop 2024-04-25T10:17:35.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f6382400640 1 -- 172.21.15.165:0/1783561409 shutdown_connections 2024-04-25T10:17:35.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f6382400640 1 --2- 172.21.15.165:0/1783561409 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c0cd470 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f6382400640 1 --2- 172.21.15.165:0/1783561409 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c0cadb0 0x7f637c0cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.950+0000 7f6382400640 1 -- 172.21.15.165:0/1783561409 >> 172.21.15.165:0/1783561409 conn(0x7f637c0b2390 msgr2=0x7f637c0b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:35.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.951+0000 7f6382400640 1 -- 172.21.15.165:0/1783561409 shutdown_connections 2024-04-25T10:17:35.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.951+0000 7f6382400640 1 -- 172.21.15.165:0/1783561409 wait complete. 2024-04-25T10:17:35.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.952+0000 7f6382400640 1 Processor -- start 2024-04-25T10:17:35.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.952+0000 7f6382400640 1 -- start start 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.953+0000 7f6382400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637c0cadb0 0x7f637c0cc470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.953+0000 7f6382400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c14f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.953+0000 7f6382400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c15de80 0x7f637c14f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.953+0000 7f6382400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c15e260 con 0x7f637c0ccfe0 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.953+0000 7f6382400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0cfc00 con 0x7f637c0cadb0 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.953+0000 7f6382400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f637c0b2eb0 con 0x7f637c15de80 2024-04-25T10:17:35.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c14f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c14f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36100/0 (socket says 172.21.15.165:36100) 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 -- 172.21.15.165:0/3271901640 learned_addr learned my addr 172.21.15.165:0/3271901640 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637be00640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c15de80 0x7f637c14f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637b400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637c0cadb0 0x7f637c0cc470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 -- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c15de80 msgr2=0x7f637c14f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c15de80 0x7f637c14f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 -- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637c0cadb0 msgr2=0x7f637c0cc470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637c0cadb0 0x7f637c0cc470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637aa00640 1 -- 172.21.15.165:0/3271901640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637c0aecb0 con 0x7f637c0ccfe0 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637be00640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c15de80 0x7f637c14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:35.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.954+0000 7f637b400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637c0cadb0 0x7f637c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:35.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.955+0000 7f637aa00640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c14f350 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f6368007ef0 tx=0x7f636800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.955+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368014620 con 0x7f637c0ccfe0 2024-04-25T10:17:35.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.955+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6368019070 con 0x7f637c0ccfe0 2024-04-25T10:17:35.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.955+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f637c159030 con 0x7f637c0ccfe0 2024-04-25T10:17:35.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.955+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368014a00 con 0x7f637c0ccfe0 2024-04-25T10:17:35.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.955+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f637c0ae8f0 con 0x7f637c0ccfe0 2024-04-25T10:17:35.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.956+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637c073a00 con 0x7f637c0ccfe0 2024-04-25T10:17:35.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.957+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6368014ba0 con 0x7f637c0ccfe0 2024-04-25T10:17:35.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.958+0000 7f6373400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635c06b4d0 0x7f635c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:35.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.958+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f636808ace0 con 0x7f637c0ccfe0 2024-04-25T10:17:35.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.958+0000 7f637b400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635c06b4d0 0x7f635c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:35.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.959+0000 7f637b400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635c06b4d0 0x7f635c06d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f636c001380 tx=0x7f636c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:35.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:35.962+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63680599c0 con 0x7f637c0ccfe0 2024-04-25T10:17:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.147+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f637c15b7f0 con 0x7f637c0ccfe0 2024-04-25T10:17:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.148+0000 7f6373400640 1 -- 172.21.15.165:0/3271901640 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v148) v1 ==== 100+0+11 (secure 0 0 0) 0x7f63680592c0 con 0x7f637c0ccfe0 2024-04-25T10:17:36.152 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635c06b4d0 msgr2=0x7f635c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635c06b4d0 0x7f635c06d990 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f636c001380 tx=0x7f636c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 msgr2=0x7f637c14f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c14f350 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7f6368007ef0 tx=0x7f636800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 shutdown_connections 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f635c06b4d0 0x7f635c06d990 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f637c15de80 0x7f637c14f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f637c0ccfe0 0x7f637c14f350 unknown :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 --2- 172.21.15.165:0/3271901640 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f637c0cadb0 0x7f637c0cc470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 >> 172.21.15.165:0/3271901640 conn(0x7f637c0b2390 msgr2=0x7f637c0b2770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 shutdown_connections 2024-04-25T10:17:36.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.151+0000 7f6382400640 1 -- 172.21.15.165:0/3271901640 wait complete. 2024-04-25T10:17:36.172 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:36.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.369+0000 7fdf78400640 1 Processor -- start 2024-04-25T10:17:36.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.369+0000 7fdf78400640 1 -- start start 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.371+0000 7fdf78400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf7015dbe0 0x7fdf700d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.371+0000 7fdf78400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf700b92a0 0x7fdf700d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.371+0000 7fdf78400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf700b97a0 con 0x7fdf700b2cb0 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.371+0000 7fdf78400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf7015dfc0 con 0x7fdf700b92a0 2024-04-25T10:17:36.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.371+0000 7fdf78400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700d1a00 con 0x7fdf7015dbe0 2024-04-25T10:17:36.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.371+0000 7fdf75a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf7015dbe0 0x7fdf700d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.372+0000 7fdf76400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdf700b92a0 0x7fdf700d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.372+0000 7fdf75000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fdf700b2cb0 0x7fdf700b5400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:42948/0 (socket says 172.21.15.165:42948) 2024-04-25T10:17:36.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.372+0000 7fdf75000640 1 -- 172.21.15.165:0/3306514107 learned_addr learned my addr 172.21.15.165:0/3306514107 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.372+0000 7fdf6fe00640 1 -- 172.21.15.165:0/3306514107 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3723360278 0 0) 0x7fdf700b97a0 con 0x7fdf700b2cb0 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.373+0000 7fdf6fe00640 1 -- 172.21.15.165:0/3306514107 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf58003520 con 0x7fdf700b2cb0 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.373+0000 7fdf75a00640 1 -- 172.21.15.165:0/3306514107 >> v2:172.21.15.172:3300/0 conn(0x7fdf700b92a0 msgr2=0x7fdf700d0e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.373+0000 7fdf75a00640 1 --2- 172.21.15.165:0/3306514107 >> v2:172.21.15.172:3300/0 conn(0x7fdf700b92a0 0x7fdf700d0e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.373+0000 7fdf75a00640 1 -- 172.21.15.165:0/3306514107 >> v1:172.21.15.172:6789/0 conn(0x7fdf700b2cb0 legacy=0x7fdf700b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.373+0000 7fdf75a00640 1 -- 172.21.15.165:0/3306514107 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700d1c50 con 0x7fdf7015dbe0 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.374+0000 7fdf75a00640 1 --2- 172.21.15.165:0/3306514107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf7015dbe0 0x7fdf700d0910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf640097a0 tx=0x7fdf6402a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9668c2c215f3f7e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf6fe00640 1 -- 172.21.15.165:0/3306514107 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf64037070 con 0x7fdf7015dbe0 2024-04-25T10:17:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf6fe00640 1 -- 172.21.15.165:0/3306514107 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf64032440 con 0x7fdf7015dbe0 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf6fe00640 1 -- 172.21.15.165:0/3306514107 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf640327f0 con 0x7fdf7015dbe0 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 -- 172.21.15.165:0/3306514107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf7015dbe0 msgr2=0x7fdf700d0910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 --2- 172.21.15.165:0/3306514107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf7015dbe0 0x7fdf700d0910 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdf640097a0 tx=0x7fdf6402a180 comp rx=0 tx=0).stop 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 -- 172.21.15.165:0/3306514107 shutdown_connections 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 --2- 172.21.15.165:0/3306514107 >> v2:172.21.15.172:3300/0 conn(0x7fdf700b92a0 0x7fdf700d0e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 --2- 172.21.15.165:0/3306514107 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf7015dbe0 0x7fdf700d0910 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 -- 172.21.15.165:0/3306514107 >> 172.21.15.165:0/3306514107 conn(0x7fdf70061a20 msgr2=0x7fdf70061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 -- 172.21.15.165:0/3306514107 shutdown_connections 2024-04-25T10:17:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.375+0000 7fdf78400640 1 -- 172.21.15.165:0/3306514107 wait complete. 2024-04-25T10:17:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.377+0000 7fdf78400640 1 Processor -- start 2024-04-25T10:17:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.377+0000 7fdf78400640 1 -- start start 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf78400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf700b2cb0 0x7fdf701f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf78400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 0x7fdf701f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf78400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7015dbe0 0x7fdf700cb420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf78400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b6ab0 con 0x7fdf700b92a0 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf78400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700d2210 con 0x7fdf7015dbe0 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf78400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf700b6c30 con 0x7fdf700b2cb0 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf75000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 0x7fdf701f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf75000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 0x7fdf701f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:36108/0 (socket says 172.21.15.165:36108) 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.378+0000 7fdf75000640 1 -- 172.21.15.165:0/1826432766 learned_addr learned my addr 172.21.15.165:0/1826432766 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75a00640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf700b2cb0 0x7fdf701f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf76400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7015dbe0 0x7fdf700cb420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75000640 1 -- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf700b2cb0 msgr2=0x7fdf701f38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75000640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf700b2cb0 0x7fdf701f38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75000640 1 -- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7015dbe0 msgr2=0x7fdf700cb420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75000640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7015dbe0 0x7fdf700cb420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75000640 1 -- 172.21.15.165:0/1826432766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700cc2b0 con 0x7fdf700b92a0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.379+0000 7fdf75000640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 0x7fdf701f6800 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c390 tx=0x7fdf5c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.380+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c017070 con 0x7fdf700b92a0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.380+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5c005ce0 con 0x7fdf700b92a0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.380+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5c0072f0 con 0x7fdf700b92a0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.380+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf700cbfd0 con 0x7fdf700b92a0 2024-04-25T10:17:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.380+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf700b3620 con 0x7fdf700b92a0 2024-04-25T10:17:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.381+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf70073990 con 0x7fdf700b92a0 2024-04-25T10:17:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.382+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdf5c01a070 con 0x7fdf700b92a0 2024-04-25T10:17:36.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.383+0000 7fdf6d600640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf5006b500 0x7fdf5006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.383+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdf5c089770 con 0x7fdf700b92a0 2024-04-25T10:17:36.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.383+0000 7fdf75a00640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf5006b500 0x7fdf5006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.384+0000 7fdf75a00640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf5006b500 0x7fdf5006d9c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdf64034000 tx=0x7fdf6402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.391+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdf5c058450 con 0x7fdf700b92a0 2024-04-25T10:17:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.626+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdf700b7780 con 0x7fdf700b92a0 2024-04-25T10:17:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.627+0000 7fdf6d600640 1 -- 172.21.15.165:0/1826432766 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fdf5c057d50 con 0x7fdf700b92a0 2024-04-25T10:17:36.631 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:36.631 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":104,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:17:28.059079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:17:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 104 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf5006b500 msgr2=0x7fdf5006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf5006b500 0x7fdf5006d9c0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fdf64034000 tx=0x7fdf6402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 msgr2=0x7fdf701f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 0x7fdf701f6800 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 crypto rx=0x7fdf5c00c390 tx=0x7fdf5c00c850 comp rx=0 tx=0).stop 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 shutdown_connections 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdf5006b500 0x7fdf5006d9c0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdf7015dbe0 0x7fdf700cb420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdf700b92a0 0x7fdf701f6800 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 --2- 172.21.15.165:0/1826432766 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdf700b2cb0 0x7fdf701f38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.629+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 >> 172.21.15.165:0/1826432766 conn(0x7fdf70061a20 msgr2=0x7fdf70061e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.630+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 shutdown_connections 2024-04-25T10:17:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.630+0000 7fdf78400640 1 -- 172.21.15.165:0/1826432766 wait complete. 2024-04-25T10:17:36.651 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:36.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.832+0000 7fab27800640 1 Processor -- start 2024-04-25T10:17:36.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.833+0000 7fab27800640 1 -- start start 2024-04-25T10:17:36.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab27800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fab2015db70 0x7fab20162150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab27800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fab200b6cb0 0x7fab200b5520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab27800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab200b5a60 0x7fab2006c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab27800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab20162690 con 0x7fab2015db70 2024-04-25T10:17:36.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab27800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab200ceee0 con 0x7fab200b6cb0 2024-04-25T10:17:36.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab27800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200d1d10 con 0x7fab200b5a60 2024-04-25T10:17:36.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab24e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fab2015db70 0x7fab20162150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab24e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fab2015db70 0x7fab20162150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49314/0 (socket says 172.21.15.165:49314) 2024-04-25T10:17:36.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.834+0000 7fab24e00640 1 -- 172.21.15.165:0/2496392728 learned_addr learned my addr 172.21.15.165:0/2496392728 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:36.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.835+0000 7fab25800640 1 --2- 172.21.15.165:0/2496392728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab200b5a60 0x7fab2006c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.835+0000 7fab24e00640 1 -- 172.21.15.165:0/2496392728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab200b5a60 msgr2=0x7fab2006c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.835+0000 7fab24e00640 1 --2- 172.21.15.165:0/2496392728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab200b5a60 0x7fab2006c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.835+0000 7fab24e00640 1 -- 172.21.15.165:0/2496392728 >> v2:172.21.15.172:3300/0 conn(0x7fab200b6cb0 msgr2=0x7fab200b5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:36.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.835+0000 7fab24e00640 1 --2- 172.21.15.165:0/2496392728 >> v2:172.21.15.172:3300/0 conn(0x7fab200b6cb0 0x7fab200b5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.835+0000 7fab24e00640 1 -- 172.21.15.165:0/2496392728 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab200d1ff0 con 0x7fab2015db70 2024-04-25T10:17:36.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.836+0000 7fab24e00640 1 --2- 172.21.15.165:0/2496392728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab2015db70 0x7fab20162150 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fab2008c600 tx=0x7fab1002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9392fbbae08665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.836+0000 7fab1f400640 1 -- 172.21.15.165:0/2496392728 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10037070 con 0x7fab2015db70 2024-04-25T10:17:36.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.836+0000 7fab1f400640 1 -- 172.21.15.165:0/2496392728 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab10032440 con 0x7fab2015db70 2024-04-25T10:17:36.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.836+0000 7fab1f400640 1 -- 172.21.15.165:0/2496392728 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab10032810 con 0x7fab2015db70 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 -- 172.21.15.165:0/2496392728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab2015db70 msgr2=0x7fab20162150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 --2- 172.21.15.165:0/2496392728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab2015db70 0x7fab20162150 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7fab2008c600 tx=0x7fab1002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 -- 172.21.15.165:0/2496392728 shutdown_connections 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 --2- 172.21.15.165:0/2496392728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab200b5a60 0x7fab2006c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 --2- 172.21.15.165:0/2496392728 >> v2:172.21.15.172:3300/0 conn(0x7fab200b6cb0 0x7fab200b5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 --2- 172.21.15.165:0/2496392728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab2015db70 0x7fab20162150 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 -- 172.21.15.165:0/2496392728 >> 172.21.15.165:0/2496392728 conn(0x7fab200d0520 msgr2=0x7fab200d0930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:36.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 -- 172.21.15.165:0/2496392728 shutdown_connections 2024-04-25T10:17:36.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.837+0000 7fab27800640 1 -- 172.21.15.165:0/2496392728 wait complete. 2024-04-25T10:17:36.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.839+0000 7fab27800640 1 Processor -- start 2024-04-25T10:17:36.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.839+0000 7fab27800640 1 -- start start 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab27800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab200b5a60 0x7fab20151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab27800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 0x7fab201528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab27800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab2015db70 0x7fab20158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab27800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab200ce530 con 0x7fab200b6cb0 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab27800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2015df50 con 0x7fab200b5a60 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab27800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab200cf1c0 con 0x7fab2015db70 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab1fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 0x7fab201528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab1fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 0x7fab201528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49326/0 (socket says 172.21.15.165:49326) 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.840+0000 7fab1fe00640 1 -- 172.21.15.165:0/117182402 learned_addr learned my addr 172.21.15.165:0/117182402 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1fe00640 1 -- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab2015db70 msgr2=0x7fab20158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab24e00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab200b5a60 0x7fab20151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1fe00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab2015db70 0x7fab20158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1fe00640 1 -- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab200b5a60 msgr2=0x7fab20151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:36.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1fe00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab200b5a60 0x7fab20151ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:36.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1fe00640 1 -- 172.21.15.165:0/117182402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab201538e0 con 0x7fab200b6cb0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1fe00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 0x7fab201528d0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b450 tx=0x7fab0c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.841+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c015020 con 0x7fab200b6cb0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.842+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c011890 con 0x7fab200b6cb0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.842+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c011c80 con 0x7fab200b6cb0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.842+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab20153320 con 0x7fab200b6cb0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.842+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab201540e0 con 0x7fab200b6cb0 2024-04-25T10:17:36.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.843+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab201545d0 con 0x7fab200b6cb0 2024-04-25T10:17:36.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.844+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fab0c018020 con 0x7fab200b6cb0 2024-04-25T10:17:36.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.845+0000 7fab1cc00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab0006b4d0 0x7fab0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:36.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.845+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fab0c08a510 con 0x7fab200b6cb0 2024-04-25T10:17:36.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.845+0000 7fab24e00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab0006b4d0 0x7fab0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:36.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.846+0000 7fab24e00640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab0006b4d0 0x7fab0006d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fab10034000 tx=0x7fab1002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:36.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:36.849+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fab0c01f020 con 0x7fab200b6cb0 2024-04-25T10:17:37.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.042+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fab200d1820 con 0x7fab200b6cb0 2024-04-25T10:17:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.045+0000 7fab1cc00640 1 -- 172.21.15.165:0/117182402 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fab0c00d040 con 0x7fab200b6cb0 2024-04-25T10:17:37.050 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:37.050 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":148,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:44.518826+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab0006b4d0 msgr2=0x7fab0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab0006b4d0 0x7fab0006d990 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fab10034000 tx=0x7fab1002e000 comp rx=0 tx=0).stop 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 msgr2=0x7fab201528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 0x7fab201528d0 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00b450 tx=0x7fab0c00b910 comp rx=0 tx=0).stop 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 shutdown_connections 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fab0006b4d0 0x7fab0006d990 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fab2015db70 0x7fab20158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fab200b6cb0 0x7fab201528d0 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 --2- 172.21.15.165:0/117182402 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fab200b5a60 0x7fab20151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 >> 172.21.15.165:0/117182402 conn(0x7fab200d0520 msgr2=0x7fab200b70c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 shutdown_connections 2024-04-25T10:17:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.048+0000 7fab27800640 1 -- 172.21.15.165:0/117182402 wait complete. 2024-04-25T10:17:37.069 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:17:37.069 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:37.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.250+0000 7f990b200640 1 Processor -- start 2024-04-25T10:17:37.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.251+0000 7f990b200640 1 -- start start 2024-04-25T10:17:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.252+0000 7f990b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f990c071700 0x7f990c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.252+0000 7f990b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c15bf30 con 0x7f990c144440 2024-04-25T10:17:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.252+0000 7f990b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990c15c210 con 0x7f990c0cd860 2024-04-25T10:17:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.252+0000 7f990b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990c15c860 con 0x7f990c071700 2024-04-25T10:17:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.252+0000 7f9909e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f990c144440 0x7f990c070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58490/0 (socket says 172.21.15.165:58490) 2024-04-25T10:17:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.252+0000 7f9909e00640 1 -- 172.21.15.165:0/1198477367 learned_addr learned my addr 172.21.15.165:0/1198477367 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.253+0000 7f9903e00640 1 --2- 172.21.15.165:0/1198477367 >> v2:172.21.15.165:3300/0 conn(0x7f990c071700 0x7f990c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.253+0000 7f9909400640 1 -- 172.21.15.165:0/1198477367 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736378462 0 0) 0x7f990c15bf30 con 0x7f990c144440 2024-04-25T10:17:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.253+0000 7f9909400640 1 -- 172.21.15.165:0/1198477367 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f0003520 con 0x7f990c144440 2024-04-25T10:17:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.253+0000 7f9903e00640 1 -- 172.21.15.165:0/1198477367 >> v1:172.21.15.172:6789/0 conn(0x7f990c0cd860 legacy=0x7f990c15ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.253+0000 7f9903e00640 1 -- 172.21.15.165:0/1198477367 >> v1:172.21.15.165:6789/0 conn(0x7f990c144440 legacy=0x7f990c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.253+0000 7f9903e00640 1 -- 172.21.15.165:0/1198477367 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c15c4f0 con 0x7f990c071700 2024-04-25T10:17:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.254+0000 7f9903e00640 1 --2- 172.21.15.165:0/1198477367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c071700 0x7f990c0cd320 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f98f40097a0 tx=0x7f98f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54a5714ba0cdbfbf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.254+0000 7f9909400640 1 -- 172.21.15.165:0/1198477367 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4037070 con 0x7f990c071700 2024-04-25T10:17:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f9909400640 1 -- 172.21.15.165:0/1198477367 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f4032440 con 0x7f990c071700 2024-04-25T10:17:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f9909400640 1 -- 172.21.15.165:0/1198477367 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f4032810 con 0x7f990c071700 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 -- 172.21.15.165:0/1198477367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c071700 msgr2=0x7f990c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 --2- 172.21.15.165:0/1198477367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c071700 0x7f990c0cd320 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f98f40097a0 tx=0x7f98f402a270 comp rx=0 tx=0).stop 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 -- 172.21.15.165:0/1198477367 shutdown_connections 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 --2- 172.21.15.165:0/1198477367 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c071700 0x7f990c0cd320 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 -- 172.21.15.165:0/1198477367 >> 172.21.15.165:0/1198477367 conn(0x7f990c06abe0 msgr2=0x7f990c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 -- 172.21.15.165:0/1198477367 shutdown_connections 2024-04-25T10:17:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.255+0000 7f990b200640 1 -- 172.21.15.165:0/1198477367 wait complete. 2024-04-25T10:17:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.257+0000 7f990b200640 1 Processor -- start 2024-04-25T10:17:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.257+0000 7f990b200640 1 -- start start 2024-04-25T10:17:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.258+0000 7f990b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f990c071700 0x7f990c0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.258+0000 7f990b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c0cd860 0x7f990c0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.258+0000 7f990b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 0x7f990c15af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.258+0000 7f990b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0cde70 con 0x7f990c144440 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.258+0000 7f990b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990c0cdcf0 con 0x7f990c0cd860 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.258+0000 7f990b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990c15d400 con 0x7f990c071700 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 0x7f990c15af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 0x7f990c15af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49332/0 (socket says 172.21.15.165:49332) 2024-04-25T10:17:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 -- 172.21.15.165:0/3128917124 learned_addr learned my addr 172.21.15.165:0/3128917124 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f9909e00640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f990c071700 0x7f990c0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f9903e00640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c0cd860 0x7f990c0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 -- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f990c071700 msgr2=0x7f990c0cc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f990c071700 0x7f990c0cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 -- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c0cd860 msgr2=0x7f990c0c9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c0cd860 0x7f990c0c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 -- 172.21.15.165:0/3128917124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990c0c97b0 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f990a800640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 0x7f990c15af10 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f990400c7b0 tx=0x7f990400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.259+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904004b60 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.260+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9904004d00 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.260+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990400aa90 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.260+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990c0c9a50 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.260+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990c1e57a0 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.261+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990c073990 con 0x7f990c144440 2024-04-25T10:17:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.262+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f990400ac30 con 0x7f990c144440 2024-04-25T10:17:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.263+0000 7f9902000640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98e806b4d0 0x7f98e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.263+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f990408a8f0 con 0x7f990c144440 2024-04-25T10:17:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.263+0000 7f9909e00640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98e806b4d0 0x7f98e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.264+0000 7f9909e00640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98e806b4d0 0x7f98e806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f98fc000f60 tx=0x7f98fc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.268+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9904058f80 con 0x7f990c144440 2024-04-25T10:17:37.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.493+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f990c15b450 con 0x7f990c144440 2024-04-25T10:17:37.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.494+0000 7f9902000640 1 -- 172.21.15.165:0/3128917124 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f990405d020 con 0x7f990c144440 2024-04-25T10:17:37.498 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:37.499 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":104,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:17:28.059079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:17:37.499 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 104 2024-04-25T10:17:37.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98e806b4d0 msgr2=0x7f98e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98e806b4d0 0x7f98e806d990 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f98fc000f60 tx=0x7f98fc009040 comp rx=0 tx=0).stop 2024-04-25T10:17:37.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 msgr2=0x7f990c15af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 0x7f990c15af10 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f990400c7b0 tx=0x7f990400cc70 comp rx=0 tx=0).stop 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 shutdown_connections 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98e806b4d0 0x7f98e806d990 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f990c144440 0x7f990c15af10 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f990c0cd860 0x7f990c0c9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 --2- 172.21.15.165:0/3128917124 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f990c071700 0x7f990c0cc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 >> 172.21.15.165:0/3128917124 conn(0x7f990c06abe0 msgr2=0x7f990c140cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 shutdown_connections 2024-04-25T10:17:37.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.497+0000 7f990b200640 1 -- 172.21.15.165:0/3128917124 wait complete. 2024-04-25T10:17:37.519 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:37.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.709+0000 7fdc6be00640 1 Processor -- start 2024-04-25T10:17:37.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.710+0000 7fdc6be00640 1 -- start start 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.711+0000 7fdc6be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdc6c143e60 0x7fdc6c06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.711+0000 7fdc6be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c06e6a0 0x7fdc6c0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.711+0000 7fdc6be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fdc6c0cc840 0x7fdc6c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.711+0000 7fdc6be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c144240 con 0x7fdc6c143e60 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.711+0000 7fdc6be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c15e220 con 0x7fdc6c0cc840 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.711+0000 7fdc6be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c15e500 con 0x7fdc6c06e6a0 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdc6c143e60 0x7fdc6c06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdc6c143e60 0x7fdc6c06e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49340/0 (socket says 172.21.15.165:49340) 2024-04-25T10:17:37.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 -- 172.21.15.165:0/599778846 learned_addr learned my addr 172.21.15.165:0/599778846 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:37.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 -- 172.21.15.165:0/599778846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c06e6a0 msgr2=0x7fdc6c0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc62000640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c06e6a0 0x7fdc6c0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6b400640 1 --2- 172.21.15.165:0/599778846 >> v2:172.21.15.172:3300/0 conn(0x7fdc6c0cc840 0x7fdc6c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c06e6a0 0x7fdc6c0cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 -- 172.21.15.165:0/599778846 >> v2:172.21.15.172:3300/0 conn(0x7fdc6c0cc840 msgr2=0x7fdc6c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:37.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 --2- 172.21.15.165:0/599778846 >> v2:172.21.15.172:3300/0 conn(0x7fdc6c0cc840 0x7fdc6c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6aa00640 1 -- 172.21.15.165:0/599778846 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c15e7e0 con 0x7fdc6c143e60 2024-04-25T10:17:37.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc62000640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c06e6a0 0x7fdc6c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:37.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.712+0000 7fdc6b400640 1 --2- 172.21.15.165:0/599778846 >> v2:172.21.15.172:3300/0 conn(0x7fdc6c0cc840 0x7fdc6c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:37.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.713+0000 7fdc6aa00640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c06e160 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0b2740 tx=0x7fdc5c02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b933facf22dcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.714+0000 7fdc6a000640 1 -- 172.21.15.165:0/599778846 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00b2e0 con 0x7fdc6c143e60 2024-04-25T10:17:37.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.714+0000 7fdc6a000640 1 -- 172.21.15.165:0/599778846 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c009600 con 0x7fdc6c143e60 2024-04-25T10:17:37.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.714+0000 7fdc6a000640 1 -- 172.21.15.165:0/599778846 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c00b480 con 0x7fdc6c143e60 2024-04-25T10:17:37.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.714+0000 7fdc6be00640 1 -- 172.21.15.165:0/599778846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 msgr2=0x7fdc6c06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.714+0000 7fdc6be00640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c06e160 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c0b2740 tx=0x7fdc5c02c8a0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 -- 172.21.15.165:0/599778846 shutdown_connections 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 --2- 172.21.15.165:0/599778846 >> v2:172.21.15.172:3300/0 conn(0x7fdc6c0cc840 0x7fdc6c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c06e6a0 0x7fdc6c0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 --2- 172.21.15.165:0/599778846 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c06e160 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 -- 172.21.15.165:0/599778846 >> 172.21.15.165:0/599778846 conn(0x7fdc6c06abe0 msgr2=0x7fdc6c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 -- 172.21.15.165:0/599778846 shutdown_connections 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.715+0000 7fdc6be00640 1 -- 172.21.15.165:0/599778846 wait complete. 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.716+0000 7fdc6be00640 1 Processor -- start 2024-04-25T10:17:37.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.716+0000 7fdc6be00640 1 -- start start 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.717+0000 7fdc6be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc6c06e6a0 0x7fdc6c0ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.717+0000 7fdc6be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c0cc840 0x7fdc6c1effc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.717+0000 7fdc6be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c159b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.717+0000 7fdc6be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cce50 con 0x7fdc6c143e60 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.717+0000 7fdc6be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cccd0 con 0x7fdc6c06e6a0 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.717+0000 7fdc6be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c0b2d30 con 0x7fdc6c0cc840 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c159b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c159b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49344/0 (socket says 172.21.15.165:49344) 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 -- 172.21.15.165:0/1907734200 learned_addr learned my addr 172.21.15.165:0/1907734200 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:37.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc62000640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c0cc840 0x7fdc6c1effc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 -- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c0cc840 msgr2=0x7fdc6c1effc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c0cc840 0x7fdc6c1effc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6aa00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc6c06e6a0 0x7fdc6c0ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 -- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc6c06e6a0 msgr2=0x7fdc6c0ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc6c06e6a0 0x7fdc6c0ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 -- 172.21.15.165:0/1907734200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c1f0500 con 0x7fdc6c143e60 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc62000640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c0cc840 0x7fdc6c1effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.718+0000 7fdc6b400640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c159b60 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400c7f0 tx=0x7fdc6400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.719+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64015070 con 0x7fdc6c143e60 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.719+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc64007820 con 0x7fdc6c143e60 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.719+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c0ad4c0 con 0x7fdc6c143e60 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.719+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc64007c50 con 0x7fdc6c143e60 2024-04-25T10:17:37.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.719+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c0ada80 con 0x7fdc6c143e60 2024-04-25T10:17:37.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.720+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c073990 con 0x7fdc6c143e60 2024-04-25T10:17:37.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.721+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdc64018070 con 0x7fdc6c143e60 2024-04-25T10:17:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.722+0000 7fdc63400640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc4806b4d0 0x7fdc4806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.722+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdc6408a560 con 0x7fdc6c143e60 2024-04-25T10:17:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.722+0000 7fdc6aa00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc4806b4d0 0x7fdc4806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:37.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.723+0000 7fdc6aa00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc4806b4d0 0x7fdc4806d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c036000 tx=0x7fdc5c00b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:37.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.726+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdc6c1f0500 con 0x7fdc6c143e60 2024-04-25T10:17:37.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.961+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc6c06e480 con 0x7fdc6c143e60 2024-04-25T10:17:37.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.962+0000 7fdc63400640 1 -- 172.21.15.165:0/1907734200 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fdc640591b0 con 0x7fdc6c143e60 2024-04-25T10:17:37.966 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:37.967 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":104,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97}],"filesystems":[{"mdsmap":{"epoch":104,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-25T10:16:42.983841+0000","modified":"2024-04-25T10:17:28.059079+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":9765},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9762":{"gid":9762,"name":"a","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.165:6835/1838340482","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1838340482},{"type":"v1","addr":"172.21.15.165:6835","nonce":1838340482}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}},"gid_9765":{"gid":9765,"name":"c","rank":0,"incarnation":98,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/2515540447","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":2515540447},{"type":"v1","addr":"172.21.15.165:6837","nonce":2515540447}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2024-04-25T10:17:37.967 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 104 2024-04-25T10:17:37.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc4806b4d0 msgr2=0x7fdc4806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc4806b4d0 0x7fdc4806d990 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c036000 tx=0x7fdc5c00b000 comp rx=0 tx=0).stop 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 msgr2=0x7fdc6c159b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c159b60 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fdc6400c7f0 tx=0x7fdc6400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 shutdown_connections 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdc4806b4d0 0x7fdc4806d990 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdc6c143e60 0x7fdc6c159b60 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdc6c0cc840 0x7fdc6c1effc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 --2- 172.21.15.165:0/1907734200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdc6c06e6a0 0x7fdc6c0ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 >> 172.21.15.165:0/1907734200 conn(0x7fdc6c06abe0 msgr2=0x7fdc6c0cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 shutdown_connections 2024-04-25T10:17:37.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:37.965+0000 7fdc6be00640 1 -- 172.21.15.165:0/1907734200 wait complete. 2024-04-25T10:17:37.987 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:38.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.182+0000 7fc6b8a00640 1 Processor -- start 2024-04-25T10:17:38.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.182+0000 7fc6b8a00640 1 -- start start 2024-04-25T10:17:38.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6b40b93c0 0x7fc6b40cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b8a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc6b40d04f0 0x7fc6b40d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b8a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b4161df0 con 0x7fc6b415db90 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b8a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b40d1f50 con 0x7fc6b40b93c0 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b8a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b40d0940 con 0x7fc6b40d04f0 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc6b415db90 0x7fc6b41615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58506/0 (socket says 172.21.15.165:58506) 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b3400640 1 -- 172.21.15.165:0/373181603 learned_addr learned my addr 172.21.15.165:0/373181603 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b2a00640 1 --2- 172.21.15.165:0/373181603 >> v2:172.21.15.165:3300/0 conn(0x7fc6b40b93c0 0x7fc6b40cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b3e00640 1 --2- 172.21.15.165:0/373181603 >> v2:172.21.15.172:3300/0 conn(0x7fc6b40d04f0 0x7fc6b40d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b2000640 1 -- 172.21.15.165:0/373181603 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1223155103 0 0) 0x7fc6b4161df0 con 0x7fc6b415db90 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b2000640 1 -- 172.21.15.165:0/373181603 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc698003520 con 0x7fc6b415db90 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b2a00640 1 -- 172.21.15.165:0/373181603 >> v2:172.21.15.172:3300/0 conn(0x7fc6b40d04f0 msgr2=0x7fc6b40d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.184+0000 7fc6b2a00640 1 --2- 172.21.15.165:0/373181603 >> v2:172.21.15.172:3300/0 conn(0x7fc6b40d04f0 0x7fc6b40d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.185+0000 7fc6b2a00640 1 -- 172.21.15.165:0/373181603 >> v1:172.21.15.165:6789/0 conn(0x7fc6b415db90 legacy=0x7fc6b41615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.185+0000 7fc6b2a00640 1 -- 172.21.15.165:0/373181603 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b41620d0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.186+0000 7fc6b2a00640 1 --2- 172.21.15.165:0/373181603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b40cffb0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc6b408cc30 tx=0x7fc69c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b56186755625ee3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.186+0000 7fc6b2000640 1 -- 172.21.15.165:0/373181603 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c037070 con 0x7fc6b40b93c0 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.186+0000 7fc6b2000640 1 -- 172.21.15.165:0/373181603 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c032440 con 0x7fc6b40b93c0 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.186+0000 7fc6b2000640 1 -- 172.21.15.165:0/373181603 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c032810 con 0x7fc6b40b93c0 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.186+0000 7fc6b8a00640 1 -- 172.21.15.165:0/373181603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 msgr2=0x7fc6b40cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.186+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/373181603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b40cffb0 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fc6b408cc30 tx=0x7fc69c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.187+0000 7fc6b8a00640 1 -- 172.21.15.165:0/373181603 shutdown_connections 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.187+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/373181603 >> v2:172.21.15.172:3300/0 conn(0x7fc6b40d04f0 0x7fc6b40d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.187+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/373181603 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b40cffb0 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.187+0000 7fc6b8a00640 1 -- 172.21.15.165:0/373181603 >> 172.21.15.165:0/373181603 conn(0x7fc6b40b52b0 msgr2=0x7fc6b40b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.187+0000 7fc6b8a00640 1 -- 172.21.15.165:0/373181603 shutdown_connections 2024-04-25T10:17:38.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.187+0000 7fc6b8a00640 1 -- 172.21.15.165:0/373181603 wait complete. 2024-04-25T10:17:38.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.189+0000 7fc6b8a00640 1 Processor -- start 2024-04-25T10:17:38.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.189+0000 7fc6b8a00640 1 -- start start 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b4160210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6b40d04f0 0x7fc6b414d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6b415db90 0x7fc6b414dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40b68d0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40d1be0 con 0x7fc6b415db90 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b40b6a50 con 0x7fc6b40d04f0 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b4160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b4160210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49362/0 (socket says 172.21.15.165:49362) 2024-04-25T10:17:38.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 -- 172.21.15.165:0/1752391245 learned_addr learned my addr 172.21.15.165:0/1752391245 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:38.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 -- 172.21.15.165:0/1752391245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6b40d04f0 msgr2=0x7fc6b414d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6b40d04f0 0x7fc6b414d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 -- 172.21.15.165:0/1752391245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6b415db90 msgr2=0x7fc6b414dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6b415db90 0x7fc6b414dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.190+0000 7fc6b3400640 1 -- 172.21.15.165:0/1752391245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b40ca9a0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.191+0000 7fc6b3400640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b4160210 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c390 tx=0x7fc6a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.191+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4017070 con 0x7fc6b40b93c0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.191+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a4005ce0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.191+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b40cac80 con 0x7fc6b40b93c0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.191+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a40072f0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.191+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b40cb240 con 0x7fc6b40b93c0 2024-04-25T10:17:38.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.192+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b4073990 con 0x7fc6b40b93c0 2024-04-25T10:17:38.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.193+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc6a401a070 con 0x7fc6b40b93c0 2024-04-25T10:17:38.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.194+0000 7fc6ab400640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc69006b4d0 0x7fc69006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.194+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6a40897d0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.194+0000 7fc6b2a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc69006b4d0 0x7fc69006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.195+0000 7fc6b2a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc69006b4d0 0x7fc69006d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc69c034000 tx=0x7fc69c02c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.199+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc6a40584b0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.397+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6b40d14b0 con 0x7fc6b40b93c0 2024-04-25T10:17:38.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.400+0000 7fc6ab400640 1 -- 172.21.15.165:0/1752391245 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fc6a405ce20 con 0x7fc6b40b93c0 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:38.404 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":148,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:16:44.518826+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:16:40.787562+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"147","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_data","create_time":"2024-04-25T10:16:42.000970+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.402+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc69006b4d0 msgr2=0x7fc69006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.402+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc69006b4d0 0x7fc69006d990 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7fc69c034000 tx=0x7fc69c02c000 comp rx=0 tx=0).stop 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.402+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 msgr2=0x7fc6b4160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.402+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b4160210 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400c390 tx=0x7fc6a400c850 comp rx=0 tx=0).stop 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 shutdown_connections 2024-04-25T10:17:38.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc69006b4d0 0x7fc69006d990 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc6b415db90 0x7fc6b414dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc6b40d04f0 0x7fc6b414d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 --2- 172.21.15.165:0/1752391245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6b40b93c0 0x7fc6b4160210 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 >> 172.21.15.165:0/1752391245 conn(0x7fc6b40b52b0 msgr2=0x7fc6b40d26f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 shutdown_connections 2024-04-25T10:17:38.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.403+0000 7fc6b8a00640 1 -- 172.21.15.165:0/1752391245 wait complete. 2024-04-25T10:17:38.426 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:17:38.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.626+0000 7f1179200640 1 Processor -- start 2024-04-25T10:17:38.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.626+0000 7f1179200640 1 -- start start 2024-04-25T10:17:38.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.628+0000 7f1179200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 0x7f1174150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.628+0000 7f1179200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117415b5a0 con 0x7f11740cd4c0 2024-04-25T10:17:38.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.628+0000 7f1179200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f117415e6e0 con 0x7f117415b1c0 2024-04-25T10:17:38.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.628+0000 7f1179200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11740cc890 con 0x7f11740cd950 2024-04-25T10:17:38.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.628+0000 7f1171400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f11740cd4c0 0x7f117415dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58516/0 (socket says 172.21.15.165:58516) 2024-04-25T10:17:38.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.628+0000 7f1171400640 1 -- 172.21.15.165:0/1424736179 learned_addr learned my addr 172.21.15.165:0/1424736179 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:38.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1172800640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 0x7f1174150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203905785 0 0) 0x7f117415b5a0 con 0x7f11740cd4c0 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1158003520 con 0x7f11740cd4c0 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2379861492 0 0) 0x7f1158003520 con 0x7f11740cd4c0 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 msgr2=0x7f1174150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 0x7f1174150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 >> v1:172.21.15.172:6789/0 conn(0x7f117415b1c0 legacy=0x7f11740cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f117415e9c0 con 0x7f11740cd4c0 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1160001670 con 0x7f11740cd4c0 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 >> v1:172.21.15.165:6789/0 conn(0x7f11740cd4c0 legacy=0x7f117415dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 0x7f11580039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158003f60 0x7f1158004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158004d40 0x7f11580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11740cc890 con 0x7f1158004d40 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.629+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115800baa0 con 0x7f1158003f60 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115800deb0 con 0x7f11740cd950 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1172800640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158004d40 0x7f11580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1171e00640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 0x7f11580039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1171400640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158003f60 0x7f1158004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1172800640 1 -- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 msgr2=0x7f11580039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1172800640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11740cd950 0x7f11580039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1172800640 1 -- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158003f60 msgr2=0x7f1158004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1172800640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158003f60 0x7f1158004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1172800640 1 -- 172.21.15.165:0/1424736179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1158003520 con 0x7f1158004d40 2024-04-25T10:17:38.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.630+0000 7f1171400640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1158003f60 0x7f1158004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:38.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.631+0000 7f1172800640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158004d40 0x7f11580051c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f116c009f90 tx=0x7f1158005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43be0e47d6400340 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c009c10 con 0x7f1158004d40 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f116c036070 con 0x7f1158004d40 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1170a00640 1 -- 172.21.15.165:0/1424736179 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c031850 con 0x7f1158004d40 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1179200640 1 -- 172.21.15.165:0/1424736179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158004d40 msgr2=0x7f11580051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1179200640 1 --2- 172.21.15.165:0/1424736179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1158004d40 0x7f11580051c0 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f116c009f90 tx=0x7f1158005830 comp rx=0 tx=0).stop 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1172800640 1 -- 172.21.15.165:0/1424736179 reap_dead start 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1179200640 1 -- 172.21.15.165:0/1424736179 shutdown_connections 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1179200640 1 -- 172.21.15.165:0/1424736179 >> 172.21.15.165:0/1424736179 conn(0x7f117406abe0 msgr2=0x7f117406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1179200640 1 -- 172.21.15.165:0/1424736179 shutdown_connections 2024-04-25T10:17:38.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.632+0000 7f1179200640 1 -- 172.21.15.165:0/1424736179 wait complete. 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.634+0000 7f1179200640 1 Processor -- start 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.634+0000 7f1179200640 1 -- start start 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1179200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 0x7f117414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1179200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11740b09b0 0x7f11741540b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1179200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11741545f0 0x7f117415d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1179200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117415a670 con 0x7f1174142380 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1179200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f117415aca0 con 0x7f11740b09b0 2024-04-25T10:17:38.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1179200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f117415e370 con 0x7f11741545f0 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1171e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 0x7f117414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1171e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 0x7f117414c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49384/0 (socket says 172.21.15.165:49384) 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.635+0000 7f1171e00640 1 -- 172.21.15.165:0/4290116183 learned_addr learned my addr 172.21.15.165:0/4290116183 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1172800640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11741545f0 0x7f117415d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171400640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11740b09b0 0x7f11741540b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171e00640 1 -- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11741545f0 msgr2=0x7f117415d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171e00640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11741545f0 0x7f117415d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171e00640 1 -- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11740b09b0 msgr2=0x7f11741540b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171e00640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11740b09b0 0x7f11741540b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171e00640 1 -- 172.21.15.165:0/4290116183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11740b1170 con 0x7f1174142380 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171400640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11740b09b0 0x7f11741540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1171e00640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 0x7f117414c1d0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f116400e550 tx=0x7f116400ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11640052e0 con 0x7f1174142380 2024-04-25T10:17:38.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1164018070 con 0x7f1174142380 2024-04-25T10:17:38.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1164003810 con 0x7f1174142380 2024-04-25T10:17:38.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.636+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11740b1450 con 0x7f1174142380 2024-04-25T10:17:38.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.637+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11740b1a10 con 0x7f1174142380 2024-04-25T10:17:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.638+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1174073a00 con 0x7f1174142380 2024-04-25T10:17:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.639+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11640039b0 con 0x7f1174142380 2024-04-25T10:17:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.640+0000 7f116a000640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f115406b500 0x7f115406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.640+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f116408a0a0 con 0x7f1174142380 2024-04-25T10:17:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.640+0000 7f1171400640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f115406b500 0x7f115406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:38.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.641+0000 7f1171400640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f115406b500 0x7f115406d9c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1160001200 tx=0x7f1160009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:38.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.644+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1164058d00 con 0x7f1174142380 2024-04-25T10:17:38.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.879+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f11740ccb10 con 0x7f1174142380 2024-04-25T10:17:38.884 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:17:38.881+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:17:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.885+0000 7f116a000640 1 -- 172.21.15.165:0/4290116183 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1164058600 con 0x7f1174142380 2024-04-25T10:17:38.889 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:17:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f115406b500 msgr2=0x7f115406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f115406b500 0x7f115406d9c0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1160001200 tx=0x7f1160009040 comp rx=0 tx=0).stop 2024-04-25T10:17:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 msgr2=0x7f117414c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:38.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 0x7f117414c1d0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f116400e550 tx=0x7f116400ea10 comp rx=0 tx=0).stop 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 shutdown_connections 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f115406b500 0x7f115406d9c0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11741545f0 0x7f117415d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11740b09b0 0x7f11741540b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 --2- 172.21.15.165:0/4290116183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1174142380 0x7f117414c1d0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.888+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 >> 172.21.15.165:0/4290116183 conn(0x7f117406abe0 msgr2=0x7f11740ae620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.889+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 shutdown_connections 2024-04-25T10:17:38.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:38.889+0000 7f1179200640 1 -- 172.21.15.165:0/4290116183 wait complete. 2024-04-25T10:17:38.913 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:17:38.968 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:17:38.972 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:17:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.086+0000 7f681a800640 1 Processor -- start 2024-04-25T10:17:39.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.086+0000 7f681a800640 1 -- start start 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f681a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f68141446d0 0x7f68140b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f681a800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140cccc0 con 0x7f681406e8a0 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f681a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140d0460 con 0x7f6814144d50 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f681a800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68140b2b20 con 0x7f68141446d0 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f6813e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f681406e8a0 0x7f68140cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58540/0 (socket says 172.21.15.165:58540) 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f6813e00640 1 -- 172.21.15.165:0/4077085403 learned_addr learned my addr 172.21.15.165:0/4077085403 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:39.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.087+0000 7f6813400640 1 --2- 172.21.15.165:0/4077085403 >> v2:172.21.15.165:3300/0 conn(0x7f68141446d0 0x7f68140b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.088+0000 7f6812a00640 1 -- 172.21.15.165:0/4077085403 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4042081153 0 0) 0x7f68140cccc0 con 0x7f681406e8a0 2024-04-25T10:17:39.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.088+0000 7f6812a00640 1 -- 172.21.15.165:0/4077085403 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67f8003520 con 0x7f681406e8a0 2024-04-25T10:17:39.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.088+0000 7f6813400640 1 -- 172.21.15.165:0/4077085403 >> v1:172.21.15.172:6789/0 conn(0x7f6814144d50 legacy=0x7f68140cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:39.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.088+0000 7f6813400640 1 -- 172.21.15.165:0/4077085403 >> v1:172.21.15.165:6789/0 conn(0x7f681406e8a0 legacy=0x7f68140cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.088+0000 7f6813400640 1 -- 172.21.15.165:0/4077085403 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140cc140 con 0x7f68141446d0 2024-04-25T10:17:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.089+0000 7f6813400640 1 --2- 172.21.15.165:0/4077085403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68141446d0 0x7f68140b3f20 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f68040097a0 tx=0x7f680402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62c816e1718d6d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.089+0000 7f6812a00640 1 -- 172.21.15.165:0/4077085403 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804037070 con 0x7f68141446d0 2024-04-25T10:17:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.089+0000 7f6812a00640 1 -- 172.21.15.165:0/4077085403 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804032440 con 0x7f68141446d0 2024-04-25T10:17:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.089+0000 7f6812a00640 1 -- 172.21.15.165:0/4077085403 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804032810 con 0x7f68141446d0 2024-04-25T10:17:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 -- 172.21.15.165:0/4077085403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68141446d0 msgr2=0x7f68140b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 --2- 172.21.15.165:0/4077085403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68141446d0 0x7f68140b3f20 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f68040097a0 tx=0x7f680402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 -- 172.21.15.165:0/4077085403 shutdown_connections 2024-04-25T10:17:39.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 --2- 172.21.15.165:0/4077085403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68141446d0 0x7f68140b3f20 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 -- 172.21.15.165:0/4077085403 >> 172.21.15.165:0/4077085403 conn(0x7f681406abe0 msgr2=0x7f681406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 -- 172.21.15.165:0/4077085403 shutdown_connections 2024-04-25T10:17:39.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.090+0000 7f681a800640 1 -- 172.21.15.165:0/4077085403 wait complete. 2024-04-25T10:17:39.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.092+0000 7f681a800640 1 Processor -- start 2024-04-25T10:17:39.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.092+0000 7f681a800640 1 -- start start 2024-04-25T10:17:39.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f681a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 0x7f68140adb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f681a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68141446d0 0x7f681415a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f681a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814144d50 0x7f681415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f681a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68140cc700 con 0x7f681406e8a0 2024-04-25T10:17:39.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f681a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f681406d400 con 0x7f6814144d50 2024-04-25T10:17:39.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f681a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68140cc9e0 con 0x7f68141446d0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 0x7f68140adb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 0x7f68140adb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49432/0 (socket says 172.21.15.165:49432) 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 -- 172.21.15.165:0/2708502237 learned_addr learned my addr 172.21.15.165:0/2708502237 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 -- 172.21.15.165:0/2708502237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68141446d0 msgr2=0x7f681415a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68141446d0 0x7f681415a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 -- 172.21.15.165:0/2708502237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814144d50 msgr2=0x7f681415ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814144d50 0x7f681415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.093+0000 7f6813400640 1 -- 172.21.15.165:0/2708502237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68140ceb40 con 0x7f681406e8a0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.094+0000 7f6813400640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 0x7f68140adb70 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f6804000c00 tx=0x7f68040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.094+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804037040 con 0x7f681406e8a0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.094+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804042070 con 0x7f681406e8a0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.094+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804030940 con 0x7f681406e8a0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.094+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68140cee20 con 0x7f681406e8a0 2024-04-25T10:17:39.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.094+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68140c4b30 con 0x7f681406e8a0 2024-04-25T10:17:39.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.095+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68140c4d30 con 0x7f681406e8a0 2024-04-25T10:17:39.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.096+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6804030ae0 con 0x7f681406e8a0 2024-04-25T10:17:39.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.097+0000 7f680b400640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67f006b4d0 0x7f67f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:39.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.097+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f68040aad60 con 0x7f681406e8a0 2024-04-25T10:17:39.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.097+0000 7f680be00640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67f006b4d0 0x7f67f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:39.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.098+0000 7f680be00640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67f006b4d0 0x7f67f006d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:39.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.101+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f68140c4d30 con 0x7f681406e8a0 2024-04-25T10:17:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.325+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6814073a00 con 0x7f681406e8a0 2024-04-25T10:17:39.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.888+0000 7f680b400640 1 -- 172.21.15.165:0/2708502237 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6804045360 con 0x7f681406e8a0 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.892+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67f006b4d0 msgr2=0x7f67f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.892+0000 7f681a800640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67f006b4d0 0x7f67f006d990 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f67fc006fd0 tx=0x7f67fc006bc0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.892+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 msgr2=0x7f68140adb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.892+0000 7f681a800640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 0x7f68140adb70 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 crypto rx=0x7f6804000c00 tx=0x7f68040046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.892+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 shutdown_connections 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.892+0000 7f681a800640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67f006b4d0 0x7f67f006d990 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.893+0000 7f681a800640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6814144d50 0x7f681415ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.893+0000 7f681a800640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68141446d0 0x7f681415a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.893+0000 7f681a800640 1 --2- 172.21.15.165:0/2708502237 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f681406e8a0 0x7f68140adb70 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.893+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 >> 172.21.15.165:0/2708502237 conn(0x7f681406abe0 msgr2=0x7f6814143370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.893+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 shutdown_connections 2024-04-25T10:17:39.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:39.893+0000 7f681a800640 1 -- 172.21.15.165:0/2708502237 wait complete. 2024-04-25T10:17:39.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:17:40.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.105+0000 7f01c1600640 1 Processor -- start 2024-04-25T10:17:40.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.105+0000 7f01c1600640 1 -- start start 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.106+0000 7f01c1600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f01bc145140 0x7f01bc0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c1600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f01bc0ce950 0x7f01bc0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c1600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc0ced90 con 0x7f01bc0ce950 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c1600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01bc145520 con 0x7f01bc145140 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc071270 con 0x7f01bc0c5030 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f01bc0ce950 0x7f01bc0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f01bc0ce950 0x7f01bc0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49434/0 (socket says 172.21.15.165:49434) 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 -- 172.21.15.165:0/1513427314 learned_addr learned my addr 172.21.15.165:0/1513427314 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c0c00640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bbe00640 1 --2- 172.21.15.165:0/1513427314 >> v2:172.21.15.172:3300/0 conn(0x7f01bc145140 0x7f01bc0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 -- 172.21.15.165:0/1513427314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 msgr2=0x7f01bc06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:40.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 -- 172.21.15.165:0/1513427314 >> v2:172.21.15.172:3300/0 conn(0x7f01bc145140 msgr2=0x7f01bc0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:40.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 --2- 172.21.15.165:0/1513427314 >> v2:172.21.15.172:3300/0 conn(0x7f01bc145140 0x7f01bc0ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01bb400640 1 -- 172.21.15.165:0/1513427314 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc071550 con 0x7f01bc0ce950 2024-04-25T10:17:40.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.107+0000 7f01c0c00640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.108+0000 7f01bb400640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc0c4af0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f01bc06f690 tx=0x7f01a802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4e360fd4499527 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.109+0000 7f01baa00640 1 -- 172.21.15.165:0/1513427314 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037070 con 0x7f01bc0ce950 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.109+0000 7f01baa00640 1 -- 172.21.15.165:0/1513427314 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a8032440 con 0x7f01bc0ce950 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.109+0000 7f01baa00640 1 -- 172.21.15.165:0/1513427314 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8032810 con 0x7f01bc0ce950 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.109+0000 7f01c1600640 1 -- 172.21.15.165:0/1513427314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 msgr2=0x7f01bc0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.109+0000 7f01c1600640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc0c4af0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 crypto rx=0x7f01bc06f690 tx=0x7f01a802a140 comp rx=0 tx=0).stop 2024-04-25T10:17:40.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 -- 172.21.15.165:0/1513427314 shutdown_connections 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 --2- 172.21.15.165:0/1513427314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc0c4af0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 --2- 172.21.15.165:0/1513427314 >> v2:172.21.15.172:3300/0 conn(0x7f01bc145140 0x7f01bc0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 -- 172.21.15.165:0/1513427314 >> 172.21.15.165:0/1513427314 conn(0x7f01bc061cc0 msgr2=0x7f01bc06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 -- 172.21.15.165:0/1513427314 shutdown_connections 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.110+0000 7f01c1600640 1 -- 172.21.15.165:0/1513427314 wait complete. 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.111+0000 7f01c1600640 1 Processor -- start 2024-04-25T10:17:40.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.111+0000 7f01c1600640 1 -- start start 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01c1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc140130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01c1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc140670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01c1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01bc145140 0x7f01bc141f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01c1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0c5480 con 0x7f01bc0ce950 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01c1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc071b10 con 0x7f01bc145140 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01c1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01bc0c5600 con 0x7f01bc0c5030 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.112+0000 7f01bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc140670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc140670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49444/0 (socket says 172.21.15.165:49444) 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 -- 172.21.15.165:0/1412358126 learned_addr learned my addr 172.21.15.165:0/1412358126 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bbe00640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc140130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01c0c00640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01bc145140 0x7f01bc141f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 -- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 msgr2=0x7f01bc140130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc140130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 -- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01bc145140 msgr2=0x7f01bc141f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01bc145140 0x7f01bc141f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 -- 172.21.15.165:0/1412358126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01bc13db60 con 0x7f01bc0ce950 2024-04-25T10:17:40.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.113+0000 7f01bb400640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc140670 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f01a802a180 tx=0x7f01a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.114+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8037040 con 0x7f01bc0ce950 2024-04-25T10:17:40.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.114+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a8042070 con 0x7f01bc0ce950 2024-04-25T10:17:40.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.114+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01bc13de40 con 0x7f01bc0ce950 2024-04-25T10:17:40.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.114+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a8030bb0 con 0x7f01bc0ce950 2024-04-25T10:17:40.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.114+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01bc13e400 con 0x7f01bc0ce950 2024-04-25T10:17:40.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.115+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01bc073990 con 0x7f01bc0ce950 2024-04-25T10:17:40.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.116+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f01a8047460 con 0x7f01bc0ce950 2024-04-25T10:17:40.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.117+0000 7f01b3e00640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f019c06b500 0x7f019c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:40.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.117+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f01a80aaff0 con 0x7f01bc0ce950 2024-04-25T10:17:40.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.117+0000 7f01bbe00640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f019c06b500 0x7f019c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:40.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.118+0000 7f01bbe00640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f019c06b500 0x7f019c06d9c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006fd0 tx=0x7f01ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:40.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.121+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f01a8045360 con 0x7f01bc0ce950 2024-04-25T10:17:40.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:40.315+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01bc06c220 con 0x7f01bc0ce950 2024-04-25T10:17:41.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.242+0000 7f01b3e00640 1 -- 172.21.15.165:0/1412358126 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v150) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01a8079c80 con 0x7f01bc0ce950 2024-04-25T10:17:41.247 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:17:41.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.246+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f019c06b500 msgr2=0x7f019c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.246+0000 7f01c1600640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f019c06b500 0x7f019c06d9c0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f01ac006fd0 tx=0x7f01ac008040 comp rx=0 tx=0).stop 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.246+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 msgr2=0x7f01bc140670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.246+0000 7f01c1600640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc140670 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7f01a802a180 tx=0x7f01a80046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 shutdown_connections 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f019c06b500 0x7f019c06d9c0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f01bc145140 0x7f01bc141f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f01bc0ce950 0x7f01bc140670 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 --2- 172.21.15.165:0/1412358126 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f01bc0c5030 0x7f01bc140130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 >> 172.21.15.165:0/1412358126 conn(0x7f01bc061cc0 msgr2=0x7f01bc15beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.247+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 shutdown_connections 2024-04-25T10:17:41.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.248+0000 7f01c1600640 1 -- 172.21.15.165:0/1412358126 wait complete. 2024-04-25T10:17:41.274 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:17:41.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.470+0000 7fa785a00640 1 Processor -- start 2024-04-25T10:17:41.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.470+0000 7fa785a00640 1 -- start start 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa78006ebf0 0x7fa7800726c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa78006bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa78015e710 con 0x7fa78006c130 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa78006c560 con 0x7fa78006ebf0 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78015e9f0 con 0x7fa78015dfa0 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa78006c130 0x7fa780161590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58556/0 (socket says 172.21.15.165:58556) 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa785000640 1 -- 172.21.15.165:0/4218604263 learned_addr learned my addr 172.21.15.165:0/4218604263 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:41.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa77fe00640 1 --2- 172.21.15.165:0/4218604263 >> v2:172.21.15.165:3300/0 conn(0x7fa78006ebf0 0x7fa7800726c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa77f400640 1 --2- 172.21.15.165:0/4218604263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa78006bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa77ea00640 1 -- 172.21.15.165:0/4218604263 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 342248066 0 0) 0x7fa78015e710 con 0x7fa78006c130 2024-04-25T10:17:41.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa77ea00640 1 -- 172.21.15.165:0/4218604263 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa764003520 con 0x7fa78006c130 2024-04-25T10:17:41.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.472+0000 7fa77fe00640 1 -- 172.21.15.165:0/4218604263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 msgr2=0x7fa78006bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:41.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.473+0000 7fa77fe00640 1 --2- 172.21.15.165:0/4218604263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa78006bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.473+0000 7fa77fe00640 1 -- 172.21.15.165:0/4218604263 >> v1:172.21.15.165:6789/0 conn(0x7fa78006c130 legacy=0x7fa780161590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.473+0000 7fa77fe00640 1 -- 172.21.15.165:0/4218604263 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa78015ecd0 con 0x7fa78006ebf0 2024-04-25T10:17:41.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.474+0000 7fa77fe00640 1 --2- 172.21.15.165:0/4218604263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006ebf0 0x7fa7800726c0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097a0 tx=0x7fa77802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f48faa8c3619d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.474+0000 7fa77ea00640 1 -- 172.21.15.165:0/4218604263 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778037070 con 0x7fa78006ebf0 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.474+0000 7fa77ea00640 1 -- 172.21.15.165:0/4218604263 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778032440 con 0x7fa78006ebf0 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.474+0000 7fa77ea00640 1 -- 172.21.15.165:0/4218604263 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778032810 con 0x7fa78006ebf0 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 -- 172.21.15.165:0/4218604263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006ebf0 msgr2=0x7fa7800726c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 --2- 172.21.15.165:0/4218604263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006ebf0 0x7fa7800726c0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097a0 tx=0x7fa77802a180 comp rx=0 tx=0).stop 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 -- 172.21.15.165:0/4218604263 shutdown_connections 2024-04-25T10:17:41.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 --2- 172.21.15.165:0/4218604263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa78006bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 --2- 172.21.15.165:0/4218604263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006ebf0 0x7fa7800726c0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 -- 172.21.15.165:0/4218604263 >> 172.21.15.165:0/4218604263 conn(0x7fa780071af0 msgr2=0x7fa780071f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:41.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 -- 172.21.15.165:0/4218604263 shutdown_connections 2024-04-25T10:17:41.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.475+0000 7fa785a00640 1 -- 172.21.15.165:0/4218604263 wait complete. 2024-04-25T10:17:41.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.476+0000 7fa785a00640 1 Processor -- start 2024-04-25T10:17:41.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.477+0000 7fa785a00640 1 -- start start 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.477+0000 7fa785a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 0x7fa7801549d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa785a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa78006ebf0 0x7fa7801595f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa785a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa7800addd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa785a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78006efd0 con 0x7fa78006c130 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa785a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78015e400 con 0x7fa78006ebf0 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa785a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78006f150 con 0x7fa78015dfa0 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 0x7fa7801549d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 0x7fa7801549d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49462/0 (socket says 172.21.15.165:49462) 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 -- 172.21.15.165:0/194647337 learned_addr learned my addr 172.21.15.165:0/194647337 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 -- 172.21.15.165:0/194647337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 msgr2=0x7fa7800addd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa7800addd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 -- 172.21.15.165:0/194647337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa78006ebf0 msgr2=0x7fa7801595f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa78006ebf0 0x7fa7801595f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.478+0000 7fa77fe00640 1 -- 172.21.15.165:0/194647337 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7800ae740 con 0x7fa78006c130 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.479+0000 7fa77fe00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 0x7fa7801549d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098c0 tx=0x7fa7780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.479+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa778037040 con 0x7fa78006c130 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.479+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa778030500 con 0x7fa78006c130 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.479+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780308d0 con 0x7fa78006c130 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.479+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7800aea20 con 0x7fa78006c130 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.479+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7800cc160 con 0x7fa78006c130 2024-04-25T10:17:41.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.480+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa780073990 con 0x7fa78006c130 2024-04-25T10:17:41.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.481+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa778044020 con 0x7fa78006c130 2024-04-25T10:17:41.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.482+0000 7fa777e00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa75c06b4d0 0x7fa75c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:41.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.482+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5749+0+0 (secure 0 0 0) 0x7fa7780aa4f0 con 0x7fa78006c130 2024-04-25T10:17:41.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.482+0000 7fa77f400640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa75c06b4d0 0x7fa75c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:41.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.483+0000 7fa77f400640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa75c06b4d0 0x7fa75c06d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa7680070b0 tx=0x7fa768007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:41.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.486+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa778079330 con 0x7fa78006c130 2024-04-25T10:17:41.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:41.683+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa7801580c0 con 0x7fa78006c130 2024-04-25T10:17:42.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.255+0000 7fa777e00640 1 -- 172.21.15.165:0/194647337 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v151) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa778078c30 con 0x7fa78006c130 2024-04-25T10:17:42.259 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:17:42.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa75c06b4d0 msgr2=0x7fa75c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa75c06b4d0 0x7fa75c06d990 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fa7680070b0 tx=0x7fa768007040 comp rx=0 tx=0).stop 2024-04-25T10:17:42.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 msgr2=0x7fa7801549d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 0x7fa7801549d0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7fa7780098c0 tx=0x7fa7780046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 shutdown_connections 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa75c06b4d0 0x7fa75c06d990 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa78015dfa0 0x7fa7800addd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa78006ebf0 0x7fa7801595f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 --2- 172.21.15.165:0/194647337 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa78006c130 0x7fa7801549d0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.260+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 >> 172.21.15.165:0/194647337 conn(0x7fa780071af0 msgr2=0x7fa7800b3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.261+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 shutdown_connections 2024-04-25T10:17:42.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.261+0000 7fa785a00640 1 -- 172.21.15.165:0/194647337 wait complete. 2024-04-25T10:17:42.295 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:17:42.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.473+0000 7f967fe00640 1 Processor -- start 2024-04-25T10:17:42.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.473+0000 7f967fe00640 1 -- start start 2024-04-25T10:17:42.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9680145140 0x7f96800ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800c5650 con 0x7f96800ce950 2024-04-25T10:17:42.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96800ced90 con 0x7f9680145140 2024-04-25T10:17:42.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9680145520 con 0x7f96800c5200 2024-04-25T10:17:42.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:59714/0 (socket says 172.21.15.165:59714) 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967f400640 1 -- 172.21.15.165:0/2296585893 learned_addr learned my addr 172.21.15.165:0/2296585893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967ea00640 1 --2- 172.21.15.165:0/2296585893 >> v2:172.21.15.172:3300/0 conn(0x7f9680145140 0x7f96800ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967d600640 1 -- 172.21.15.165:0/2296585893 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1954404646 0 0) 0x7f96800c5650 con 0x7f96800ce950 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967f400640 1 -- 172.21.15.165:0/2296585893 >> v2:172.21.15.172:3300/0 conn(0x7f9680145140 msgr2=0x7f96800ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967f400640 1 --2- 172.21.15.165:0/2296585893 >> v2:172.21.15.172:3300/0 conn(0x7f9680145140 0x7f96800ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.475+0000 7f967f400640 1 -- 172.21.15.165:0/2296585893 >> v1:172.21.15.172:6789/0 conn(0x7f96800ce950 legacy=0x7f96800c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.476+0000 7f967f400640 1 -- 172.21.15.165:0/2296585893 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9680071270 con 0x7f96800c5200 2024-04-25T10:17:42.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.477+0000 7f967f400640 1 --2- 172.21.15.165:0/2296585893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015bfb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f967802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79eb5be22e0f3c47 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.477+0000 7f967d600640 1 -- 172.21.15.165:0/2296585893 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678036070 con 0x7f96800c5200 2024-04-25T10:17:42.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.477+0000 7f967d600640 1 -- 172.21.15.165:0/2296585893 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9678031600 con 0x7f96800c5200 2024-04-25T10:17:42.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.477+0000 7f967d600640 1 -- 172.21.15.165:0/2296585893 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780319d0 con 0x7f96800c5200 2024-04-25T10:17:42.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 -- 172.21.15.165:0/2296585893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 msgr2=0x7f968015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 --2- 172.21.15.165:0/2296585893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015bfb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f967802a680 comp rx=0 tx=0).stop 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 -- 172.21.15.165:0/2296585893 shutdown_connections 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 --2- 172.21.15.165:0/2296585893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015bfb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 --2- 172.21.15.165:0/2296585893 >> v2:172.21.15.172:3300/0 conn(0x7f9680145140 0x7f96800ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 -- 172.21.15.165:0/2296585893 >> 172.21.15.165:0/2296585893 conn(0x7f9680061cc0 msgr2=0x7f968006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 -- 172.21.15.165:0/2296585893 shutdown_connections 2024-04-25T10:17:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.478+0000 7f967fe00640 1 -- 172.21.15.165:0/2296585893 wait complete. 2024-04-25T10:17:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.480+0000 7f967fe00640 1 Processor -- start 2024-04-25T10:17:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.480+0000 7f967fe00640 1 -- start start 2024-04-25T10:17:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800ce950 0x7f96800ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 0x7f96800cd440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9680071b10 con 0x7f9680145140 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9680071830 con 0x7f96800ce950 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968015c4f0 con 0x7f96800c5200 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 0x7f96800cd440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 0x7f96800cd440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49468/0 (socket says 172.21.15.165:49468) 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967f400640 1 -- 172.21.15.165:0/632977832 learned_addr learned my addr 172.21.15.165:0/632977832 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.481+0000 7f967ea00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967e000640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800ce950 0x7f96800ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967f400640 1 -- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 msgr2=0x7f968015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967f400640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967f400640 1 -- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800ce950 msgr2=0x7f96800ce0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967f400640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800ce950 0x7f96800ce0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967f400640 1 -- 172.21.15.165:0/632977832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800cc3a0 con 0x7f9680145140 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967f400640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 0x7f96800cd440 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f9678004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678041020 con 0x7f9680145140 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9678037040 con 0x7f9680145140 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96800cc680 con 0x7f9680145140 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678036050 con 0x7f9680145140 2024-04-25T10:17:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.482+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96800ccc40 con 0x7f9680145140 2024-04-25T10:17:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.484+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9680073990 con 0x7f9680145140 2024-04-25T10:17:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.485+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9678007c50 con 0x7f9680145140 2024-04-25T10:17:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.486+0000 7f9677400640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f966006b4d0 0x7f966006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.486+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f96780aa7f0 con 0x7f9680145140 2024-04-25T10:17:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.486+0000 7f967ea00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f966006b4d0 0x7f966006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.487+0000 7f967ea00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f966006b4d0 0x7f966006d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9670009a30 tx=0x7f9670008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.490+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f967804b060 con 0x7f9680145140 2024-04-25T10:17:42.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.739+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f968015b950 con 0x7f9680145140 2024-04-25T10:17:42.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.744+0000 7f9677400640 1 -- 172.21.15.165:0/632977832 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f96780797d0 con 0x7f9680145140 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f966006b4d0 msgr2=0x7f966006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f966006b4d0 0x7f966006d990 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f9670009a30 tx=0x7f9670008040 comp rx=0 tx=0).stop 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 msgr2=0x7f96800cd440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 0x7f96800cd440 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f9678009e50 tx=0x7f9678004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 shutdown_connections 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f966006b4d0 0x7f966006d990 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9680145140 0x7f96800cd440 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f96800ce950 0x7f96800ce0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 --2- 172.21.15.165:0/632977832 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f96800c5200 0x7f968015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 >> 172.21.15.165:0/632977832 conn(0x7f9680061cc0 msgr2=0x7f968015d060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 shutdown_connections 2024-04-25T10:17:42.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.748+0000 7f967fe00640 1 -- 172.21.15.165:0/632977832 wait complete. 2024-04-25T10:17:42.769 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive' 2024-04-25T10:17:42.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.956+0000 7f62cbe00640 1 Processor -- start 2024-04-25T10:17:42.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.956+0000 7f62cbe00640 1 -- start start 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.957+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 0x7f62cc0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.957+0000 7f62cbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f62cc0b2270 0x7f62cc0ce3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62cbe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62cc15eda0 con 0x7f62cc099ab0 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62cbe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62cc0b1540 con 0x7f62cc0b2270 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62cbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc09a820 con 0x7f62cc15d9b0 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62cb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f62cc099ab0 0x7f62cc099f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58562/0 (socket says 172.21.15.165:58562) 2024-04-25T10:17:42.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62cb400640 1 -- 172.21.15.165:0/879585733 learned_addr learned my addr 172.21.15.165:0/879585733 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:42.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62caa00640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 0x7f62cc0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62ca000640 1 --2- 172.21.15.165:0/879585733 >> v2:172.21.15.172:3300/0 conn(0x7f62cc0b2270 0x7f62cc0ce3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.958+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382259884 0 0) 0x7f62cc15eda0 con 0x7f62cc099ab0 2024-04-25T10:17:42.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62ac003520 con 0x7f62cc099ab0 2024-04-25T10:17:42.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1363623159 0 0) 0x7f62ac003520 con 0x7f62cc099ab0 2024-04-25T10:17:42.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 msgr2=0x7f62cc0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 0x7f62cc0cde90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 >> v2:172.21.15.172:3300/0 conn(0x7f62cc0b2270 msgr2=0x7f62cc0ce3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 --2- 172.21.15.165:0/879585733 >> v2:172.21.15.172:3300/0 conn(0x7f62cc0b2270 0x7f62cc0ce3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0b1790 con 0x7f62cc099ab0 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f62c4001670 con 0x7f62cc099ab0 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 >> v1:172.21.15.165:6789/0 conn(0x7f62cc099ab0 legacy=0x7f62cc099f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62ac0039b0 0x7f62ac003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62ac004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 0x7f62ac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0b1540 con 0x7f62ac004800 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc09a820 con 0x7f62ac0039b0 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac00b9c0 con 0x7f62cc15d9b0 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.959+0000 7f62ca000640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62ac004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62caa00640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62ac0039b0 0x7f62ac003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62cb400640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 0x7f62ac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62ca000640 1 -- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 msgr2=0x7f62ac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62ca000640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc15d9b0 0x7f62ac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62ca000640 1 -- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62ac0039b0 msgr2=0x7f62ac003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:42.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62ca000640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62ac0039b0 0x7f62ac003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.960+0000 7f62ca000640 1 -- 172.21.15.165:0/879585733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac003520 con 0x7f62ac004800 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.961+0000 7f62ca000640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62ac004c80 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009fc0 tx=0x7f62ac005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79df1e1166e36ad0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b4009c10 con 0x7f62ac004800 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62b4036070 con 0x7f62ac004800 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62c9600640 1 -- 172.21.15.165:0/879585733 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62b402d070 con 0x7f62ac004800 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62cbe00640 1 -- 172.21.15.165:0/879585733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 msgr2=0x7f62ac004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62cbe00640 1 --2- 172.21.15.165:0/879585733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62ac004c80 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f62b4009fc0 tx=0x7f62ac005830 comp rx=0 tx=0).stop 2024-04-25T10:17:42.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62cb400640 1 -- 172.21.15.165:0/879585733 reap_dead start 2024-04-25T10:17:42.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62cbe00640 1 -- 172.21.15.165:0/879585733 shutdown_connections 2024-04-25T10:17:42.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.962+0000 7f62cbe00640 1 -- 172.21.15.165:0/879585733 >> 172.21.15.165:0/879585733 conn(0x7f62cc0d1610 msgr2=0x7f62cc0d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:42.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.963+0000 7f62cbe00640 1 -- 172.21.15.165:0/879585733 shutdown_connections 2024-04-25T10:17:42.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.963+0000 7f62cbe00640 1 -- 172.21.15.165:0/879585733 wait complete. 2024-04-25T10:17:42.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.965+0000 7f62cbe00640 1 Processor -- start 2024-04-25T10:17:42.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.965+0000 7f62cbe00640 1 -- start start 2024-04-25T10:17:42.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62cc14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc0c5b30 0x7f62cc15bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62cbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62cc0c5fb0 0x7f62cc15c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62cbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0cde90 con 0x7f62ac004800 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62cbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0b1d50 con 0x7f62cc0c5fb0 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62cbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62cc0ce010 con 0x7f62cc0c5b30 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62cc14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62cc14e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49494/0 (socket says 172.21.15.165:49494) 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.966+0000 7f62caa00640 1 -- 172.21.15.165:0/521114294 learned_addr learned my addr 172.21.15.165:0/521114294 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62ca000640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc0c5b30 0x7f62cc15bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62cb400640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62cc0c5fb0 0x7f62cc15c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62caa00640 1 -- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc0c5b30 msgr2=0x7f62cc15bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62caa00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc0c5b30 0x7f62cc15bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62caa00640 1 -- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62cc0c5fb0 msgr2=0x7f62cc15c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62caa00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62cc0c5fb0 0x7f62cc15c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62caa00640 1 -- 172.21.15.165:0/521114294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62cc0c6580 con 0x7f62ac004800 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62cb400640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62cc0c5fb0 0x7f62cc15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62caa00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62cc14e950 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011630 tx=0x7f62bc011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.967+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00f920 con 0x7f62ac004800 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.968+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62bc00fac0 con 0x7f62ac004800 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.968+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00fe70 con 0x7f62ac004800 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.968+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62cc0c6860 con 0x7f62ac004800 2024-04-25T10:17:42.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.968+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62cc0c6e20 con 0x7f62ac004800 2024-04-25T10:17:42.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.969+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62cc073990 con 0x7f62ac004800 2024-04-25T10:17:42.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.970+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f62bc015460 con 0x7f62ac004800 2024-04-25T10:17:42.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.971+0000 7f62c2a00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62a806b4d0 0x7f62a806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:42.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.971+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f62bc08aa50 con 0x7f62ac004800 2024-04-25T10:17:42.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.971+0000 7f62ca000640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62a806b4d0 0x7f62a806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:42.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.972+0000 7f62ca000640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62a806b4d0 0x7f62a806d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f62b4035000 tx=0x7f62b402f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:42.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:42.976+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f62bc019030 con 0x7f62ac004800 2024-04-25T10:17:43.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.230+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]} v 0) v1 -- 0x7f62cc0b1f90 con 0x7f62ac004800 2024-04-25T10:17:43.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.258+0000 7f62c2a00640 1 -- 172.21.15.165:0/521114294 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub_checks.TestScrubControls.test_scrub_when_mds_is_inactive"]}]=0 v720) v1 ==== 159+0+0 (secure 0 0 0) 0x7f62bc0599a0 con 0x7f62ac004800 2024-04-25T10:17:43.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.261+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62a806b4d0 msgr2=0x7f62a806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.261+0000 7f62cbe00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62a806b4d0 0x7f62a806d990 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f62b4035000 tx=0x7f62b402f000 comp rx=0 tx=0).stop 2024-04-25T10:17:43.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.261+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 msgr2=0x7f62cc14e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.261+0000 7f62cbe00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62cc14e950 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 crypto rx=0x7f62bc011630 tx=0x7f62bc011af0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 shutdown_connections 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f62a806b4d0 0x7f62a806d990 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f62cc0c5fb0 0x7f62cc15c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f62cc0c5b30 0x7f62cc15bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 --2- 172.21.15.165:0/521114294 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f62ac004800 0x7f62cc14e950 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 >> 172.21.15.165:0/521114294 conn(0x7f62cc0d1610 msgr2=0x7f62cc070b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 shutdown_connections 2024-04-25T10:17:43.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.262+0000 7f62cbe00640 1 -- 172.21.15.165:0/521114294 wait complete. 2024-04-25T10:17:43.284 INFO:tasks.cephfs_test_runner:test_scrub_when_mds_is_inactive (tasks.cephfs.test_scrub_checks.TestScrubControls) ... ok 2024-04-25T10:17:43.285 INFO:tasks.cephfs_test_runner:Starting test: test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:17:43.285 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:43.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.483+0000 7f924a800640 1 Processor -- start 2024-04-25T10:17:43.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.483+0000 7f924a800640 1 -- start start 2024-04-25T10:17:43.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f924a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9244164970 0x7f9244166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f924a800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9244144b30 con 0x7f9244144750 2024-04-25T10:17:43.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f924a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92440712d0 con 0x7f92440ce920 2024-04-25T10:17:43.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f924a800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92440715b0 con 0x7f9244164970 2024-04-25T10:17:43.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f9243400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9244144750 0x7f924406e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58568/0 (socket says 172.21.15.165:58568) 2024-04-25T10:17:43.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f9243400640 1 -- 172.21.15.165:0/3727157033 learned_addr learned my addr 172.21.15.165:0/3727157033 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2735752208 0 0) 0x7f9244144b30 con 0x7f9244144750 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.485+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9228003520 con 0x7f9244144750 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9243e00640 1 --2- 172.21.15.165:0/3727157033 >> v2:172.21.15.172:3300/0 conn(0x7f9244164970 0x7f9244166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229608874 0 0) 0x7f9228003520 con 0x7f9244144750 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 >> v2:172.21.15.172:3300/0 conn(0x7f9244164970 msgr2=0x7f9244166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 --2- 172.21.15.165:0/3727157033 >> v2:172.21.15.172:3300/0 conn(0x7f9244164970 0x7f9244166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 >> v1:172.21.15.172:6789/0 conn(0x7f92440ce920 legacy=0x7f92440cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244071890 con 0x7f9244144750 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9234001670 con 0x7f9244144750 2024-04-25T10:17:43.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9243e00640 1 --2- 172.21.15.165:0/3727157033 >> v2:172.21.15.172:3300/0 conn(0x7f9244164970 0x7f9244166d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 >> v1:172.21.15.165:6789/0 conn(0x7f9244144750 legacy=0x7f924406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92280039b0 0x7f9228003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9228004800 0x7f9228004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f922800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92440715b0 con 0x7f92280053c0 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f922800c260 con 0x7f92280039b0 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f922800c460 con 0x7f9228004800 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.486+0000 7f9243e00640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f922800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9243400640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92280039b0 0x7f9228003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9243e00640 1 -- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9228004800 msgr2=0x7f9228004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9242a00640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9228004800 0x7f9228004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9243e00640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9228004800 0x7f9228004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9243e00640 1 -- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92280039b0 msgr2=0x7f9228003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9243e00640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92280039b0 0x7f9228003db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9243e00640 1 -- 172.21.15.165:0/3727157033 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9228003520 con 0x7f92280053c0 2024-04-25T10:17:43.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.487+0000 7f9242a00640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9228004800 0x7f9228004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.488+0000 7f9243e00640 1 -- 172.21.15.165:0/3727157033 reap_dead start 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.488+0000 7f9243e00640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f922800b9c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f923c00aa80 tx=0x7f9228005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e99b0e3ddf21e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.488+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c037070 con 0x7f92280053c0 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.488+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923c02ac70 con 0x7f92280053c0 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.488+0000 7f9242000640 1 -- 172.21.15.165:0/3727157033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923c00a650 con 0x7f92280053c0 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 -- 172.21.15.165:0/3727157033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 msgr2=0x7f922800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f922800b9c0 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7f923c00aa80 tx=0x7f9228005830 comp rx=0 tx=0).stop 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 -- 172.21.15.165:0/3727157033 shutdown_connections 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 --2- 172.21.15.165:0/3727157033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f922800b9c0 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 -- 172.21.15.165:0/3727157033 >> 172.21.15.165:0/3727157033 conn(0x7f924406abe0 msgr2=0x7f924406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 -- 172.21.15.165:0/3727157033 shutdown_connections 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.489+0000 7f924a800640 1 -- 172.21.15.165:0/3727157033 wait complete. 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.491+0000 7f924a800640 1 Processor -- start 2024-04-25T10:17:43.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.491+0000 7f924a800640 1 -- start start 2024-04-25T10:17:43.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f924a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f92441f4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f924a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92441f4550 0x7f92441f6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f924a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92441f70b0 0x7f9244163990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f924a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92441455d0 con 0x7f92280053c0 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f924a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9244145450 con 0x7f92441f70b0 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f924a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924415c000 con 0x7f92441f4550 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f92441f4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f92441f4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49502/0 (socket says 172.21.15.165:49502) 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 -- 172.21.15.165:0/1078491713 learned_addr learned my addr 172.21.15.165:0/1078491713 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 -- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92441f4550 msgr2=0x7f92441f6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9242a00640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92441f4550 0x7f92441f6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92441f4550 0x7f92441f6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 -- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92441f70b0 msgr2=0x7f9244163990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92441f70b0 0x7f9244163990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.492+0000 7f9243400640 1 -- 172.21.15.165:0/1078491713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92441f75c0 con 0x7f92280053c0 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.493+0000 7f9243400640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f92441f4010 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f923400e730 tx=0x7f923400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.493+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400c6c0 con 0x7f92280053c0 2024-04-25T10:17:43.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.493+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9234018070 con 0x7f92280053c0 2024-04-25T10:17:43.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.493+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400caa0 con 0x7f92280053c0 2024-04-25T10:17:43.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.493+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92441f78a0 con 0x7f92280053c0 2024-04-25T10:17:43.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.493+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92441f3af0 con 0x7f92280053c0 2024-04-25T10:17:43.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.494+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92441f3cf0 con 0x7f92280053c0 2024-04-25T10:17:43.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.495+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f923400cc40 con 0x7f92280053c0 2024-04-25T10:17:43.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.497+0000 7f923b400640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922406b500 0x7f922406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.497+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9234089b10 con 0x7f92280053c0 2024-04-25T10:17:43.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.497+0000 7f9242a00640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922406b500 0x7f922406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.498+0000 7f9242a00640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922406b500 0x7f922406d9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f924413ee30 tx=0x7f9230009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.501+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92441f3cf0 con 0x7f92280053c0 2024-04-25T10:17:43.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.699+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92441f7a70 con 0x7f92280053c0 2024-04-25T10:17:43.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.701+0000 7f923b400640 1 -- 172.21.15.165:0/1078491713 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+13341 (secure 0 0 0) 0x7f9234022250 con 0x7f92280053c0 2024-04-25T10:17:43.704 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:43.704 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":151,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:42.244668+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/2515540447":"2024-04-26T10:17:38.882302+0000","172.21.15.165:6836/2515540447":"2024-04-26T10:17:38.882302+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922406b500 msgr2=0x7f922406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922406b500 0x7f922406d9c0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f924413ee30 tx=0x7f9230009040 comp rx=0 tx=0).stop 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 msgr2=0x7f92441f4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f92441f4010 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 crypto rx=0x7f923400e730 tx=0x7f923400ebf0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 shutdown_connections 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f922406b500 0x7f922406d9c0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92441f70b0 0x7f9244163990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92441f4550 0x7f92441f6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 --2- 172.21.15.165:0/1078491713 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92280053c0 0x7f92441f4010 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 >> 172.21.15.165:0/1078491713 conn(0x7f924406abe0 msgr2=0x7f924415b3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.704+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 shutdown_connections 2024-04-25T10:17:43.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.705+0000 7f924a800640 1 -- 172.21.15.165:0/1078491713 wait complete. 2024-04-25T10:17:43.726 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:43.727 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:43.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.913+0000 7fb038e00640 1 Processor -- start 2024-04-25T10:17:43.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.914+0000 7fb038e00640 1 -- start start 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb038e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb03415b7d0 0x7fb0340cf010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb038e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb038e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb03406cf50 con 0x7fb0340b5620 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb038e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0340ce530 con 0x7fb03415b7d0 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb038e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340ce810 con 0x7fb03406c9e0 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb031400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb0340b5620 0x7fb03406c2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58572/0 (socket says 172.21.15.165:58572) 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb031400640 1 -- 172.21.15.165:0/740939898 learned_addr learned my addr 172.21.15.165:0/740939898 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.915+0000 7fb031e00640 1 --2- 172.21.15.165:0/740939898 >> v2:172.21.15.165:3300/0 conn(0x7fb03415b7d0 0x7fb0340cf010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.916+0000 7fb032800640 1 --2- 172.21.15.165:0/740939898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.916+0000 7fb031e00640 1 -- 172.21.15.165:0/740939898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 msgr2=0x7fb0340b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.916+0000 7fb031e00640 1 --2- 172.21.15.165:0/740939898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.916+0000 7fb031e00640 1 -- 172.21.15.165:0/740939898 >> v1:172.21.15.165:6789/0 conn(0x7fb0340b5620 legacy=0x7fb03406c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.916+0000 7fb031e00640 1 -- 172.21.15.165:0/740939898 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340ceaf0 con 0x7fb03415b7d0 2024-04-25T10:17:43.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.917+0000 7fb031e00640 1 --2- 172.21.15.165:0/740939898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340cf010 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fb0240097a0 tx=0x7fb02402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74642c4be7a1719a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.917+0000 7fb030a00640 1 -- 172.21.15.165:0/740939898 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024037070 con 0x7fb03415b7d0 2024-04-25T10:17:43.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.917+0000 7fb030a00640 1 -- 172.21.15.165:0/740939898 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb024032440 con 0x7fb03415b7d0 2024-04-25T10:17:43.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.917+0000 7fb030a00640 1 -- 172.21.15.165:0/740939898 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb024032810 con 0x7fb03415b7d0 2024-04-25T10:17:43.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 -- 172.21.15.165:0/740939898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 msgr2=0x7fb0340cf010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 --2- 172.21.15.165:0/740939898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340cf010 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fb0240097a0 tx=0x7fb02402a180 comp rx=0 tx=0).stop 2024-04-25T10:17:43.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 -- 172.21.15.165:0/740939898 shutdown_connections 2024-04-25T10:17:43.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 --2- 172.21.15.165:0/740939898 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 --2- 172.21.15.165:0/740939898 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340cf010 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 -- 172.21.15.165:0/740939898 >> 172.21.15.165:0/740939898 conn(0x7fb0340d0800 msgr2=0x7fb0340d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:43.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 -- 172.21.15.165:0/740939898 shutdown_connections 2024-04-25T10:17:43.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.918+0000 7fb038e00640 1 -- 172.21.15.165:0/740939898 wait complete. 2024-04-25T10:17:43.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.920+0000 7fb038e00640 1 Processor -- start 2024-04-25T10:17:43.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.920+0000 7fb038e00640 1 -- start start 2024-04-25T10:17:43.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb038e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340d27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb038e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0340b5620 0x7fb0340c7b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb038e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340b4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb038e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340cf6d0 con 0x7fb03415b7d0 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb038e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340cf550 con 0x7fb0340b5620 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb038e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb03415bbb0 con 0x7fb03406c9e0 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb032800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340b4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb032800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340b4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49512/0 (socket says 172.21.15.165:49512) 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb032800640 1 -- 172.21.15.165:0/3387480326 learned_addr learned my addr 172.21.15.165:0/3387480326 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb031e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340d27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb031400640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0340b5620 0x7fb0340c7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb032800640 1 -- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 msgr2=0x7fb0340d27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb032800640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340d27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.921+0000 7fb032800640 1 -- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0340b5620 msgr2=0x7fb0340c7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb032800640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0340b5620 0x7fb0340c7b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:43.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb032800640 1 -- 172.21.15.165:0/3387480326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0341486a0 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb032800640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340b4550 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fb02c007d90 tx=0x7fb02c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c014420 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c019070 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c014820 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034148980 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.922+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb034148f40 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.924+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb034146240 con 0x7fb03415b7d0 2024-04-25T10:17:43.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.925+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb02c0149c0 con 0x7fb03415b7d0 2024-04-25T10:17:43.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.925+0000 7fb02a000640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb01006b4d0 0x7fb01006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:43.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.926+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb02c08a8f0 con 0x7fb03415b7d0 2024-04-25T10:17:43.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.926+0000 7fb031e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb01006b4d0 0x7fb01006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:43.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.927+0000 7fb031e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb01006b4d0 0x7fb01006d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb024034000 tx=0x7fb02402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:43.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:43.930+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb02c0598e0 con 0x7fb03415b7d0 2024-04-25T10:17:44.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.125+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb034073a00 con 0x7fb03415b7d0 2024-04-25T10:17:44.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.126+0000 7fb02a000640 1 -- 172.21.15.165:0/3387480326 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v151) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb02c0591e0 con 0x7fb03415b7d0 2024-04-25T10:17:44.129 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb01006b4d0 msgr2=0x7fb01006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb01006b4d0 0x7fb01006d990 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fb024034000 tx=0x7fb02402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 msgr2=0x7fb0340b4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340b4550 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fb02c007d90 tx=0x7fb02c00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 shutdown_connections 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb01006b4d0 0x7fb01006d990 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb03415b7d0 0x7fb0340b4550 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0340b5620 0x7fb0340c7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 --2- 172.21.15.165:0/3387480326 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb03406c9e0 0x7fb0340d27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 >> 172.21.15.165:0/3387480326 conn(0x7fb0340d0800 msgr2=0x7fb03415eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 shutdown_connections 2024-04-25T10:17:44.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.129+0000 7fb038e00640 1 -- 172.21.15.165:0/3387480326 wait complete. 2024-04-25T10:17:44.152 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:17:44.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.349+0000 7f265f200640 1 Processor -- start 2024-04-25T10:17:44.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.349+0000 7f265f200640 1 -- start start 2024-04-25T10:17:44.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.350+0000 7f265f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f266015e520 0x7f266015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2660070ff0 0x7f2660071440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26600ae590 con 0x7f266015f040 2024-04-25T10:17:44.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26600aee30 con 0x7f266015e520 2024-04-25T10:17:44.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2660071980 con 0x7f2660070ff0 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f266015e520 0x7f266015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f266015e520 0x7f266015e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49520/0 (socket says 172.21.15.165:49520) 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 -- 172.21.15.165:0/3703216908 learned_addr learned my addr 172.21.15.165:0/3703216908 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 -- 172.21.15.165:0/3703216908 >> v2:172.21.15.172:3300/0 conn(0x7f2660070ff0 msgr2=0x7f2660071440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 --2- 172.21.15.165:0/3703216908 >> v2:172.21.15.172:3300/0 conn(0x7f2660070ff0 0x7f2660071440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 -- 172.21.15.165:0/3703216908 >> v1:172.21.15.172:6789/0 conn(0x7f266015f040 legacy=0x7f26600adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:44.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.351+0000 7f265de00640 1 -- 172.21.15.165:0/3703216908 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26600ae870 con 0x7f266015e520 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.352+0000 7f265de00640 1 --2- 172.21.15.165:0/3703216908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f266015e900 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f266006f450 tx=0x7f265002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b876b2de0fda1893 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265ca00640 1 -- 172.21.15.165:0/3703216908 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650037070 con 0x7f266015e520 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265ca00640 1 -- 172.21.15.165:0/3703216908 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2650032440 con 0x7f266015e520 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265ca00640 1 -- 172.21.15.165:0/3703216908 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2650032810 con 0x7f266015e520 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265f200640 1 -- 172.21.15.165:0/3703216908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 msgr2=0x7f266015e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265f200640 1 --2- 172.21.15.165:0/3703216908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f266015e900 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7f266006f450 tx=0x7f265002a230 comp rx=0 tx=0).stop 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265f200640 1 -- 172.21.15.165:0/3703216908 shutdown_connections 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265f200640 1 --2- 172.21.15.165:0/3703216908 >> v2:172.21.15.172:3300/0 conn(0x7f2660070ff0 0x7f2660071440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265f200640 1 --2- 172.21.15.165:0/3703216908 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f266015e900 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.353+0000 7f265f200640 1 -- 172.21.15.165:0/3703216908 >> 172.21.15.165:0/3703216908 conn(0x7f266006abe0 msgr2=0x7f266006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:44.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.354+0000 7f265f200640 1 -- 172.21.15.165:0/3703216908 shutdown_connections 2024-04-25T10:17:44.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.354+0000 7f265f200640 1 -- 172.21.15.165:0/3703216908 wait complete. 2024-04-25T10:17:44.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.355+0000 7f265f200640 1 Processor -- start 2024-04-25T10:17:44.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.355+0000 7f265f200640 1 -- start start 2024-04-25T10:17:44.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2660070ff0 0x7f2660146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f2660151930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266015f040 0x7f2660151e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26600ae2a0 con 0x7f266015e520 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f266006d1f0 con 0x7f266015f040 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f266015f4d0 con 0x7f2660070ff0 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f2660151930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f2660151930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49536/0 (socket says 172.21.15.165:49536) 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.356+0000 7f265d400640 1 -- 172.21.15.165:0/1418667974 learned_addr learned my addr 172.21.15.165:0/1418667974 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265d400640 1 -- 172.21.15.165:0/1418667974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2660070ff0 msgr2=0x7f2660146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265d400640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2660070ff0 0x7f2660146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265d400640 1 -- 172.21.15.165:0/1418667974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266015f040 msgr2=0x7f2660151e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265d400640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266015f040 0x7f2660151e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:44.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265d400640 1 -- 172.21.15.165:0/1418667974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26601546e0 con 0x7f266015e520 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265d400640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f2660151930 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b3e0 tx=0x7f264c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c015020 con 0x7f266015e520 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f264c011850 con 0x7f266015e520 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f264c011c40 con 0x7f266015e520 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.357+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26601549c0 con 0x7f266015e520 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.358+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2660159f40 con 0x7f266015e520 2024-04-25T10:17:44.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.359+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2660073990 con 0x7f266015e520 2024-04-25T10:17:44.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.359+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f264c018020 con 0x7f266015e520 2024-04-25T10:17:44.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.360+0000 7f2656000640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f264006b4d0 0x7f264006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:44.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.360+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f264c089fb0 con 0x7f266015e520 2024-04-25T10:17:44.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.360+0000 7f265de00640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f264006b4d0 0x7f264006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:44.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.361+0000 7f265de00640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f264006b4d0 0x7f264006d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2650034000 tx=0x7f265002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:44.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.365+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f264c01f020 con 0x7f266015e520 2024-04-25T10:17:44.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:44.612+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f2660071dd0 con 0x7f266015e520 2024-04-25T10:17:45.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.272+0000 7f2656000640 1 -- 172.21.15.165:0/1418667974 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v722) v1 ==== 143+0+0 (secure 0 0 0) 0x7f264c0172c0 con 0x7f266015e520 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f264006b4d0 msgr2=0x7f264006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f264006b4d0 0x7f264006d990 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f2650034000 tx=0x7f265002c000 comp rx=0 tx=0).stop 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 msgr2=0x7f2660151930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f2660151930 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 crypto rx=0x7f264c00b3e0 tx=0x7f264c00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 shutdown_connections 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f264006b4d0 0x7f264006d990 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f266015f040 0x7f2660151e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f266015e520 0x7f2660151930 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 --2- 172.21.15.165:0/1418667974 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2660070ff0 0x7f2660146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 >> 172.21.15.165:0/1418667974 conn(0x7f266006abe0 msgr2=0x7f26600b0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.276+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 shutdown_connections 2024-04-25T10:17:45.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.277+0000 7f265f200640 1 -- 172.21.15.165:0/1418667974 wait complete. 2024-04-25T10:17:45.303 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:17:45.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.503+0000 7ff315e00640 1 Processor -- start 2024-04-25T10:17:45.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.503+0000 7ff315e00640 1 -- start start 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff315e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3100b5620 0x7ff3100b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff315e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff31006c2d0 0x7ff31006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff315e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff31006cc80 con 0x7ff31015b7d0 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff315e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff31006cf50 con 0x7ff31006c2d0 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff315e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3100ce530 con 0x7ff3100b5620 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff30f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff31006c2d0 0x7ff31006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff30f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff31006c2d0 0x7ff31006c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49538/0 (socket says 172.21.15.165:49538) 2024-04-25T10:17:45.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.505+0000 7ff30f400640 1 -- 172.21.15.165:0/2668203722 learned_addr learned my addr 172.21.15.165:0/2668203722 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:45.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.506+0000 7ff30e000640 1 --2- 172.21.15.165:0/2668203722 >> v2:172.21.15.172:3300/0 conn(0x7ff3100b5620 0x7ff3100b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.506+0000 7ff30f400640 1 -- 172.21.15.165:0/2668203722 >> v2:172.21.15.172:3300/0 conn(0x7ff3100b5620 msgr2=0x7ff3100b5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:45.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.506+0000 7ff30f400640 1 --2- 172.21.15.165:0/2668203722 >> v2:172.21.15.172:3300/0 conn(0x7ff3100b5620 0x7ff3100b5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.506+0000 7ff30f400640 1 -- 172.21.15.165:0/2668203722 >> v1:172.21.15.172:6789/0 conn(0x7ff31015b7d0 legacy=0x7ff3100cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:45.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.506+0000 7ff30f400640 1 -- 172.21.15.165:0/2668203722 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100ce810 con 0x7ff31006c2d0 2024-04-25T10:17:45.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.506+0000 7ff30e000640 1 --2- 172.21.15.165:0/2668203722 >> v2:172.21.15.172:3300/0 conn(0x7ff3100b5620 0x7ff3100b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.507+0000 7ff30f400640 1 --2- 172.21.15.165:0/2668203722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31006c2d0 0x7ff31006c740 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7ff3100ce180 tx=0x7ff30802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4995a0bcd6823e10 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.507+0000 7ff30d600640 1 -- 172.21.15.165:0/2668203722 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308036070 con 0x7ff31006c2d0 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.507+0000 7ff30d600640 1 -- 172.21.15.165:0/2668203722 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308031600 con 0x7ff31006c2d0 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.507+0000 7ff30d600640 1 -- 172.21.15.165:0/2668203722 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3080319d0 con 0x7ff31006c2d0 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 -- 172.21.15.165:0/2668203722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31006c2d0 msgr2=0x7ff31006c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 --2- 172.21.15.165:0/2668203722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31006c2d0 0x7ff31006c740 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 crypto rx=0x7ff3100ce180 tx=0x7ff30802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 -- 172.21.15.165:0/2668203722 shutdown_connections 2024-04-25T10:17:45.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 --2- 172.21.15.165:0/2668203722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31006c2d0 0x7ff31006c740 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 --2- 172.21.15.165:0/2668203722 >> v2:172.21.15.172:3300/0 conn(0x7ff3100b5620 0x7ff3100b5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 -- 172.21.15.165:0/2668203722 >> 172.21.15.165:0/2668203722 conn(0x7ff3100d0800 msgr2=0x7ff3100d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 -- 172.21.15.165:0/2668203722 shutdown_connections 2024-04-25T10:17:45.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.508+0000 7ff315e00640 1 -- 172.21.15.165:0/2668203722 wait complete. 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.510+0000 7ff315e00640 1 Processor -- start 2024-04-25T10:17:45.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.510+0000 7ff315e00640 1 -- start start 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff315e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff31006c2d0 0x7ff3100cbd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff315e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3100b5620 0x7ff3100aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff315e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 0x7ff3101f6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff315e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31015bbb0 con 0x7ff31015b7d0 2024-04-25T10:17:45.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff315e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff31006d230 con 0x7ff31006c2d0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff315e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff31015bd30 con 0x7ff3100b5620 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff30f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 0x7ff3101f6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff30f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 0x7ff3101f6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49540/0 (socket says 172.21.15.165:49540) 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.511+0000 7ff30f400640 1 -- 172.21.15.165:0/3816037432 learned_addr learned my addr 172.21.15.165:0/3816037432 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff30f400640 1 -- 172.21.15.165:0/3816037432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3100b5620 msgr2=0x7ff3100aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff30f400640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3100b5620 0x7ff3100aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff30f400640 1 -- 172.21.15.165:0/3816037432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff31006c2d0 msgr2=0x7ff3100cbd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff30f400640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff31006c2d0 0x7ff3100cbd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff30f400640 1 -- 172.21.15.165:0/3816037432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3100ab2b0 con 0x7ff31015b7d0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff30f400640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 0x7ff3101f6e60 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7ff308009f40 tx=0x7ff308004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308036050 con 0x7ff31015b7d0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff308042070 con 0x7ff31015b7d0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff308009510 con 0x7ff31015b7d0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.512+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3100ccf40 con 0x7ff31015b7d0 2024-04-25T10:17:45.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.513+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3100cd500 con 0x7ff31015b7d0 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.514+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff310073a00 con 0x7ff31015b7d0 2024-04-25T10:17:45.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.515+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff308007bd0 con 0x7ff31015b7d0 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.515+0000 7ff307400640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2ec06b4d0 0x7ff2ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:45.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.516+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff3080ab0d0 con 0x7ff31015b7d0 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.516+0000 7ff30ea00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2ec06b4d0 0x7ff2ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:45.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.517+0000 7ff30ea00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2ec06b4d0 0x7ff2ec06d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff300006fd0 tx=0x7ff300006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:45.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.520+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff30807a020 con 0x7ff31015b7d0 2024-04-25T10:17:45.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.723+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7ff31015b030 con 0x7ff31015b7d0 2024-04-25T10:17:45.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.729+0000 7ff307400640 1 -- 172.21.15.165:0/3816037432 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff308079920 con 0x7ff31015b7d0 2024-04-25T10:17:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2ec06b4d0 msgr2=0x7ff2ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2ec06b4d0 0x7ff2ec06d990 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff300006fd0 tx=0x7ff300006bc0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 msgr2=0x7ff3101f6e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 0x7ff3101f6e60 secure :-1 s=READY pgs=2210 cs=0 l=1 rev1=1 crypto rx=0x7ff308009f40 tx=0x7ff308004a40 comp rx=0 tx=0).stop 2024-04-25T10:17:45.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 shutdown_connections 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff2ec06b4d0 0x7ff2ec06d990 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff31015b7d0 0x7ff3101f6e60 unknown :-1 s=CLOSED pgs=2210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3100b5620 0x7ff3100aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 --2- 172.21.15.165:0/3816037432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff31006c2d0 0x7ff3100cbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 >> 172.21.15.165:0/3816037432 conn(0x7ff3100d0800 msgr2=0x7ff3100d2a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 shutdown_connections 2024-04-25T10:17:45.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:45.732+0000 7ff315e00640 1 -- 172.21.15.165:0/3816037432 wait complete. 2024-04-25T10:17:45.755 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:17:45.755 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:45.851 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:17:45.956 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:17:45.956 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:17:45.956 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:46.013 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:46.013 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:17:46.013 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:46.070 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:46.070 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:17:46.070 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:46.129 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:17:46.129 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:17:46.129 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:17:46.186 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:17:46.186 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:17:46.186 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:46.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.392+0000 7fc613200640 1 Processor -- start 2024-04-25T10:17:46.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.392+0000 7fc613200640 1 -- start start 2024-04-25T10:17:46.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc613200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 0x7fc61406e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc613200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc614144240 con 0x7fc6140cca10 2024-04-25T10:17:46.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc613200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc61415e220 con 0x7fc61406e6a0 2024-04-25T10:17:46.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc613200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61415e500 con 0x7fc614143e60 2024-04-25T10:17:46.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc612800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc6140cca10 0x7fc61415ab40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58576/0 (socket says 172.21.15.165:58576) 2024-04-25T10:17:46.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc612800640 1 -- 172.21.15.165:0/1293344179 learned_addr learned my addr 172.21.15.165:0/1293344179 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:46.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.394+0000 7fc611e00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 0x7fc61406e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 745045996 0 0) 0x7fc614144240 con 0x7fc6140cca10 2024-04-25T10:17:46.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5f8005730 con 0x7fc6140cca10 2024-04-25T10:17:46.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3140086645 0 0) 0x7fc5f8005730 con 0x7fc6140cca10 2024-04-25T10:17:46.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 msgr2=0x7fc61406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 0x7fc61406e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 >> v1:172.21.15.172:6789/0 conn(0x7fc61406e6a0 legacy=0x7fc6140cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61415e7e0 con 0x7fc6140cca10 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc60c0019f0 con 0x7fc6140cca10 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 >> v1:172.21.15.165:6789/0 conn(0x7fc6140cca10 legacy=0x7fc61415ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8005bf0 0x7fc5f8006010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 0x7fc5f8006a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f8006f50 0x7fc5f80073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc61415e500 con 0x7fc5f8005bf0 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f800ffd0 con 0x7fc5f8006f50 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.395+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f8010150 con 0x7fc614143e60 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611e00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8005bf0 0x7fc5f8006010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611400640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 0x7fc5f8006a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611e00640 1 -- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 msgr2=0x7fc5f8006a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611e00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc614143e60 0x7fc5f8006a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611e00640 1 -- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f8006f50 msgr2=0x7fc5f80073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc612800640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f8006f50 0x7fc5f80073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611e00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc5f8006f50 0x7fc5f80073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.396+0000 7fc611e00640 1 -- 172.21.15.165:0/1293344179 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f8005730 con 0x7fc5f8005bf0 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.397+0000 7fc611e00640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8005bf0 0x7fc5f8006010 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fc604009ac0 tx=0x7fc5f8007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f7ebe2315f099ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.397+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604036070 con 0x7fc5f8005bf0 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.397+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc604031400 con 0x7fc5f8005bf0 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.397+0000 7fc610a00640 1 -- 172.21.15.165:0/1293344179 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc604009690 con 0x7fc5f8005bf0 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc613200640 1 -- 172.21.15.165:0/1293344179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8005bf0 msgr2=0x7fc5f8006010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc613200640 1 --2- 172.21.15.165:0/1293344179 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc5f8005bf0 0x7fc5f8006010 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 crypto rx=0x7fc604009ac0 tx=0x7fc5f8007a40 comp rx=0 tx=0).stop 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc612800640 1 -- 172.21.15.165:0/1293344179 reap_dead start 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc613200640 1 -- 172.21.15.165:0/1293344179 shutdown_connections 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc613200640 1 -- 172.21.15.165:0/1293344179 >> 172.21.15.165:0/1293344179 conn(0x7fc61406abe0 msgr2=0x7fc61406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc613200640 1 -- 172.21.15.165:0/1293344179 shutdown_connections 2024-04-25T10:17:46.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.398+0000 7fc613200640 1 -- 172.21.15.165:0/1293344179 wait complete. 2024-04-25T10:17:46.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.400+0000 7fc613200640 1 Processor -- start 2024-04-25T10:17:46.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.400+0000 7fc613200640 1 -- start start 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc613200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5f8005bf0 0x7fc6140adc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc613200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 0x7fc6140a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc613200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc614159a70 0x7fc6140abc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc613200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140b2eb0 con 0x7fc614142240 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc613200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6140b2d30 con 0x7fc614159a70 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc613200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc61415b250 con 0x7fc5f8005bf0 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc611400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 0x7fc6140a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc611400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 0x7fc6140a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49558/0 (socket says 172.21.15.165:49558) 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.401+0000 7fc611400640 1 -- 172.21.15.165:0/557264574 learned_addr learned my addr 172.21.15.165:0/557264574 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc611400640 1 -- 172.21.15.165:0/557264574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5f8005bf0 msgr2=0x7fc6140adc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc611400640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5f8005bf0 0x7fc6140adc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc611400640 1 -- 172.21.15.165:0/557264574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc614159a70 msgr2=0x7fc6140abc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc611400640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc614159a70 0x7fc6140abc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc611400640 1 -- 172.21.15.165:0/557264574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6140a97b0 con 0x7fc614142240 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc611400640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 0x7fc6140a9270 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fc60000d5a0 tx=0x7fc60000da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc600003610 con 0x7fc614142240 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc600017070 con 0x7fc614142240 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6000039f0 con 0x7fc614142240 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6140ae560 con 0x7fc614142240 2024-04-25T10:17:46.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.402+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc614141760 con 0x7fc614142240 2024-04-25T10:17:46.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.404+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6141353f0 con 0x7fc614142240 2024-04-25T10:17:46.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.405+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc600003b90 con 0x7fc614142240 2024-04-25T10:17:46.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.406+0000 7fc60a000640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc5f406b4d0 0x7fc5f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.406+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc600089ee0 con 0x7fc614142240 2024-04-25T10:17:46.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.406+0000 7fc611e00640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc5f406b4d0 0x7fc5f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.407+0000 7fc611e00640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc5f406b4d0 0x7fc5f406d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc604033000 tx=0x7fc60402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.410+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc600058f80 con 0x7fc614142240 2024-04-25T10:17:46.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.644+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc61415da00 con 0x7fc614142240 2024-04-25T10:17:46.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.645+0000 7fc60a000640 1 -- 172.21.15.165:0/557264574 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3713 (secure 0 0 0) 0x7fc600058880 con 0x7fc614142240 2024-04-25T10:17:46.648 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:46.648 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5903,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/3332549799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3332549799},{"type":"v1","addr":"172.21.15.172:6833","nonce":3332549799}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":6568,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3049527733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":3049527733},{"type":"v1","addr":"172.21.15.172:6835","nonce":3049527733}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":97},{"gid":10191,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6835/3065822633","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3065822633},{"type":"v1","addr":"172.21.15.165:6835","nonce":3065822633}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106},{"gid":10194,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/474378996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6836","nonce":474378996},{"type":"v1","addr":"172.21.15.165:6837","nonce":474378996}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":106}],"filesystems":[]} 2024-04-25T10:17:46.649 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 106 2024-04-25T10:17:46.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.647+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc5f406b4d0 msgr2=0x7fc5f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.647+0000 7fc613200640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc5f406b4d0 0x7fc5f406d990 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7fc604033000 tx=0x7fc60402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:46.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.647+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 msgr2=0x7fc6140a9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.647+0000 7fc613200640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 0x7fc6140a9270 secure :-1 s=READY pgs=2213 cs=0 l=1 rev1=1 crypto rx=0x7fc60000d5a0 tx=0x7fc60000da60 comp rx=0 tx=0).stop 2024-04-25T10:17:46.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 shutdown_connections 2024-04-25T10:17:46.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc5f406b4d0 0x7fc5f406d990 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc614159a70 0x7fc6140abc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc614142240 0x7fc6140a9270 unknown :-1 s=CLOSED pgs=2213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 --2- 172.21.15.165:0/557264574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc5f8005bf0 0x7fc6140adc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 >> 172.21.15.165:0/557264574 conn(0x7fc61406abe0 msgr2=0x7fc61415bdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 shutdown_connections 2024-04-25T10:17:46.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.648+0000 7fc613200640 1 -- 172.21.15.165:0/557264574 wait complete. 2024-04-25T10:17:46.670 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:17:46.670 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:17:46.671 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:17:46.671 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:46.671 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:17:46.671 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:17:46.671 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:17:46.671 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:46.671 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:17:46.672 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:17:46.672 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:17:46.672 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:46.673 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:17:46.673 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:17:46.673 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:17:46.673 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:17:46.688 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:17:46.688 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:17:46.690 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:17:46.691 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:17:46.697 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:17:46.697 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:17:46.698 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:17:46.698 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:17:46.736 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:17:46.736 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:46.740 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:17:46.741 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:46.745 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:17:46.746 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:46.751 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:17:46.751 DEBUG:teuthology.parallel:result is None 2024-04-25T10:17:46.751 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:17:46.883 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:17:46.883 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:17:46.897 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:17:46.897 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:17:46.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.928+0000 7fb410600640 1 Processor -- start 2024-04-25T10:17:46.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.928+0000 7fb410600640 1 -- start start 2024-04-25T10:17:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.929+0000 7fb410600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb4080ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.929+0000 7fb410600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb408161710 0x7fb4080cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.929+0000 7fb410600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb40815e000 con 0x7fb4080b9330 2024-04-25T10:17:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.929+0000 7fb410600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4080d1de0 con 0x7fb408161710 2024-04-25T10:17:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.929+0000 7fb410600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb408161c60 con 0x7fb40815dc20 2024-04-25T10:17:46.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.929+0000 7fb40e600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb4080b9330 0x7fb40806bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50222/0 (socket says 172.21.15.165:50222) 2024-04-25T10:17:46.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40e600640 1 -- 172.21.15.165:0/1857214681 learned_addr learned my addr 172.21.15.165:0/1857214681 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:46.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40dc00640 1 --2- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb4080ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40d200640 1 --2- 172.21.15.165:0/1857214681 >> v2:172.21.15.172:3300/0 conn(0x7fb408161710 0x7fb4080cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb407e00640 1 -- 172.21.15.165:0/1857214681 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094586750 0 0) 0x7fb40815e000 con 0x7fb4080b9330 2024-04-25T10:17:46.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb407e00640 1 -- 172.21.15.165:0/1857214681 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3f0003520 con 0x7fb4080b9330 2024-04-25T10:17:46.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40d200640 1 -- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 msgr2=0x7fb4080ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:46.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40d200640 1 --2- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb4080ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40d200640 1 -- 172.21.15.165:0/1857214681 >> v1:172.21.15.165:6789/0 conn(0x7fb4080b9330 legacy=0x7fb40806bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.930+0000 7fb40d200640 1 -- 172.21.15.165:0/1857214681 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb408161f40 con 0x7fb408161710 2024-04-25T10:17:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.931+0000 7fb40d200640 1 --2- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb4080cebb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40097a0 tx=0x7fb3f402a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=560d4c3caddc9f80 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.931+0000 7fb407e00640 1 -- 172.21.15.165:0/1857214681 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4037070 con 0x7fb408161710 2024-04-25T10:17:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.931+0000 7fb407e00640 1 -- 172.21.15.165:0/1857214681 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f4032440 con 0x7fb408161710 2024-04-25T10:17:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.931+0000 7fb407e00640 1 -- 172.21.15.165:0/1857214681 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4032810 con 0x7fb408161710 2024-04-25T10:17:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 -- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 msgr2=0x7fb4080cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 --2- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb4080cebb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb3f40097a0 tx=0x7fb3f402a180 comp rx=0 tx=0).stop 2024-04-25T10:17:46.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 -- 172.21.15.165:0/1857214681 shutdown_connections 2024-04-25T10:17:46.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 --2- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb4080cebb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 --2- 172.21.15.165:0/1857214681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb4080ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 -- 172.21.15.165:0/1857214681 >> 172.21.15.165:0/1857214681 conn(0x7fb4080d0420 msgr2=0x7fb4080d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:46.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 -- 172.21.15.165:0/1857214681 shutdown_connections 2024-04-25T10:17:46.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.932+0000 7fb410600640 1 -- 172.21.15.165:0/1857214681 wait complete. 2024-04-25T10:17:46.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.934+0000 7fb410600640 1 Processor -- start 2024-04-25T10:17:46.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.934+0000 7fb410600640 1 -- start start 2024-04-25T10:17:46.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb410600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 0x7fb408158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb410600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb408154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb410600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb40815f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb410600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080b5e50 con 0x7fb4080b9330 2024-04-25T10:17:46.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb410600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb408162500 con 0x7fb408161710 2024-04-25T10:17:46.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb410600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080b5fd0 con 0x7fb40815dc20 2024-04-25T10:17:46.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb40dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 0x7fb408158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb40dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 0x7fb408158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45658/0 (socket says 172.21.15.165:45658) 2024-04-25T10:17:46.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.935+0000 7fb40dc00640 1 -- 172.21.15.165:0/237709940 learned_addr learned my addr 172.21.15.165:0/237709940 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:46.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40d200640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb408154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40e600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb40815f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40dc00640 1 -- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 msgr2=0x7fb408154ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40dc00640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb408154ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40dc00640 1 -- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 msgr2=0x7fb40815f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40dc00640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb40815f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40dc00640 1 -- 172.21.15.165:0/237709940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40815fbf0 con 0x7fb4080b9330 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb40dc00640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 0x7fb408158700 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00e870 tx=0x7fb3fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.936+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc017070 con 0x7fb4080b9330 2024-04-25T10:17:46.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.937+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3fc00c940 con 0x7fb4080b9330 2024-04-25T10:17:46.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.937+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb40815fe90 con 0x7fb4080b9330 2024-04-25T10:17:46.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.937+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc00cd30 con 0x7fb4080b9330 2024-04-25T10:17:46.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.937+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb408160450 con 0x7fb4080b9330 2024-04-25T10:17:46.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.938+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb408073a00 con 0x7fb4080b9330 2024-04-25T10:17:46.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.939+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3fc01a070 con 0x7fb4080b9330 2024-04-25T10:17:46.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.940+0000 7fb405600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3e806b4d0 0x7fb3e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:46.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.940+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb3fc089e10 con 0x7fb4080b9330 2024-04-25T10:17:46.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.940+0000 7fb40d200640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3e806b4d0 0x7fb3e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:46.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.941+0000 7fb40d200640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3e806b4d0 0x7fb3e806d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4034000 tx=0x7fb3f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:46.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:46.944+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3fc058840 con 0x7fb4080b9330 2024-04-25T10:17:47.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.152+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fb4080d16b0 con 0x7fb4080b9330 2024-04-25T10:17:47.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.302+0000 7fb405600640 1 -- 172.21.15.165:0/237709940 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v153) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb3fc05c8e0 con 0x7fb4080b9330 2024-04-25T10:17:47.306 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:17:47.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3e806b4d0 msgr2=0x7fb3e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3e806b4d0 0x7fb3e806d990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4034000 tx=0x7fb3f402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:47.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 msgr2=0x7fb408158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 0x7fb408158700 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00e870 tx=0x7fb3fc00ed30 comp rx=0 tx=0).stop 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 shutdown_connections 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3e806b4d0 0x7fb3e806d990 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb408161710 0x7fb40815f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb40815dc20 0x7fb408154ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 --2- 172.21.15.165:0/237709940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4080b9330 0x7fb408158700 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.307+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 >> 172.21.15.165:0/237709940 conn(0x7fb4080d0420 msgr2=0x7fb4080d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.308+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 shutdown_connections 2024-04-25T10:17:47.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.308+0000 7fb410600640 1 -- 172.21.15.165:0/237709940 wait complete. 2024-04-25T10:17:47.333 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.526+0000 7f3172000640 1 Processor -- start 2024-04-25T10:17:47.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.526+0000 7f3172000640 1 -- start start 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.527+0000 7f3172000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f316c144ae0 0x7f316c0ccb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.527+0000 7f3172000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f316c15bcf0 0x7f316c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.527+0000 7f3172000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f316c15c520 con 0x7f316c15fef0 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.527+0000 7f3172000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c15c800 con 0x7f316c144ae0 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.527+0000 7f3172000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f316c15cae0 con 0x7f316c15bcf0 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f316c144ae0 0x7f316c0ccb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f316c144ae0 0x7f316c0ccb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45670/0 (socket says 172.21.15.165:45670) 2024-04-25T10:17:47.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 -- 172.21.15.165:0/2017111594 learned_addr learned my addr 172.21.15.165:0/2017111594 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:47.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 -- 172.21.15.165:0/2017111594 >> v2:172.21.15.172:3300/0 conn(0x7f316c15bcf0 msgr2=0x7f316c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 --2- 172.21.15.165:0/2017111594 >> v2:172.21.15.172:3300/0 conn(0x7f316c15bcf0 0x7f316c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 -- 172.21.15.165:0/2017111594 >> v1:172.21.15.172:6789/0 conn(0x7f316c15fef0 legacy=0x7f316c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:47.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.528+0000 7f3170c00640 1 -- 172.21.15.165:0/2017111594 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c15c180 con 0x7f316c144ae0 2024-04-25T10:17:47.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.529+0000 7f3170c00640 1 --2- 172.21.15.165:0/2017111594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c0ccb00 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f316c0cf550 tx=0x7f315c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254093c2d04a362 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.529+0000 7f316be00640 1 -- 172.21.15.165:0/2017111594 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c037070 con 0x7f316c144ae0 2024-04-25T10:17:47.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.529+0000 7f316be00640 1 -- 172.21.15.165:0/2017111594 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c032440 con 0x7f316c144ae0 2024-04-25T10:17:47.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.529+0000 7f316be00640 1 -- 172.21.15.165:0/2017111594 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c032810 con 0x7f316c144ae0 2024-04-25T10:17:47.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 -- 172.21.15.165:0/2017111594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 msgr2=0x7f316c0ccb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 --2- 172.21.15.165:0/2017111594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c0ccb00 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f316c0cf550 tx=0x7f315c02a180 comp rx=0 tx=0).stop 2024-04-25T10:17:47.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 -- 172.21.15.165:0/2017111594 shutdown_connections 2024-04-25T10:17:47.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 --2- 172.21.15.165:0/2017111594 >> v2:172.21.15.172:3300/0 conn(0x7f316c15bcf0 0x7f316c15f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 --2- 172.21.15.165:0/2017111594 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c0ccb00 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 -- 172.21.15.165:0/2017111594 >> 172.21.15.165:0/2017111594 conn(0x7f316c061c30 msgr2=0x7f316c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 -- 172.21.15.165:0/2017111594 shutdown_connections 2024-04-25T10:17:47.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.530+0000 7f3172000640 1 -- 172.21.15.165:0/2017111594 wait complete. 2024-04-25T10:17:47.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.532+0000 7f3172000640 1 Processor -- start 2024-04-25T10:17:47.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.532+0000 7f3172000640 1 -- start start 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3172000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c13b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3172000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f316c15bcf0 0x7f316c13f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3172000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f316c15fef0 0x7f316c141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3172000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c0712c0 con 0x7f316c144ae0 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3172000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f316c071140 con 0x7f316c15bcf0 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3172000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f316c071440 con 0x7f316c15fef0 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3170c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c13b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3170c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c13b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45682/0 (socket says 172.21.15.165:45682) 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.533+0000 7f3170c00640 1 -- 172.21.15.165:0/3384538400 learned_addr learned my addr 172.21.15.165:0/3384538400 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:47.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3163e00640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f316c15bcf0 0x7f316c13f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3171600640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f316c15fef0 0x7f316c141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3170c00640 1 -- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f316c15fef0 msgr2=0x7f316c141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3170c00640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f316c15fef0 0x7f316c141a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3170c00640 1 -- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f316c15bcf0 msgr2=0x7f316c13f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3170c00640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f316c15bcf0 0x7f316c13f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3170c00640 1 -- 172.21.15.165:0/3384538400 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f316c0b04d0 con 0x7f316c144ae0 2024-04-25T10:17:47.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3170c00640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c13b020 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f315c02a1c0 tx=0x7f315c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.534+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c041070 con 0x7f316c144ae0 2024-04-25T10:17:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.535+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f315c03c890 con 0x7f316c144ae0 2024-04-25T10:17:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.535+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f315c037040 con 0x7f316c144ae0 2024-04-25T10:17:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.535+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f316c15a9f0 con 0x7f316c144ae0 2024-04-25T10:17:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.535+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f316c15afb0 con 0x7f316c144ae0 2024-04-25T10:17:47.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.536+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f316c073990 con 0x7f316c144ae0 2024-04-25T10:17:47.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.537+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f315c044070 con 0x7f316c144ae0 2024-04-25T10:17:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.538+0000 7f3169600640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f314806b4d0 0x7f314806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.538+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f315c0aae50 con 0x7f316c144ae0 2024-04-25T10:17:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.538+0000 7f3163e00640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f314806b4d0 0x7f314806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:47.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.539+0000 7f3163e00640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f314806b4d0 0x7f314806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f316c143ca0 tx=0x7f3154008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:47.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.542+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f315c079f10 con 0x7f316c144ae0 2024-04-25T10:17:47.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.788+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f316c15d7e0 con 0x7f316c144ae0 2024-04-25T10:17:47.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.789+0000 7f3169600640 1 -- 172.21.15.165:0/3384538400 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f315c079810 con 0x7f316c144ae0 2024-04-25T10:17:47.792 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:47.792 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:17:47.792 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:17:47.793 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:17:47.794 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:17:47.795 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.796 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.797 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.798 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.799 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:17:47.800 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:17:47.801 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:17:47.802 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:17:47.803 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:17:47.804 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:17:47.805 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:17:47.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f314806b4d0 msgr2=0x7f314806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f314806b4d0 0x7f314806d990 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f316c143ca0 tx=0x7f3154008040 comp rx=0 tx=0).stop 2024-04-25T10:17:47.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 msgr2=0x7f316c13b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:47.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c13b020 secure :-1 s=READY pgs=2220 cs=0 l=1 rev1=1 crypto rx=0x7f315c02a1c0 tx=0x7f315c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 shutdown_connections 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f314806b4d0 0x7f314806d990 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f316c15fef0 0x7f316c141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f316c15bcf0 0x7f316c13f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 --2- 172.21.15.165:0/3384538400 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f316c144ae0 0x7f316c13b020 unknown :-1 s=CLOSED pgs=2220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 >> 172.21.15.165:0/3384538400 conn(0x7f316c061c30 msgr2=0x7f316c15cd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 shutdown_connections 2024-04-25T10:17:47.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:47.792+0000 7f3172000640 1 -- 172.21.15.165:0/3384538400 wait complete. 2024-04-25T10:17:47.816 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:48.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.011+0000 7f0945c00640 1 Processor -- start 2024-04-25T10:17:48.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.012+0000 7f0945c00640 1 -- start start 2024-04-25T10:17:48.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.013+0000 7f0945c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f094015b8f0 0x7f09400c6c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.013+0000 7f0945c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f09400cf340 0x7f09400b33e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.013+0000 7f0945c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09400ccef0 con 0x7f09400b3920 2024-04-25T10:17:48.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.013+0000 7f0945c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09400d03c0 con 0x7f09400cf340 2024-04-25T10:17:48.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.013+0000 7f0945c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09400b3d90 con 0x7f094015b8f0 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f09400cf340 0x7f09400b33e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f09400cf340 0x7f09400b33e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45686/0 (socket says 172.21.15.165:45686) 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 -- 172.21.15.165:0/121313412 learned_addr learned my addr 172.21.15.165:0/121313412 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093ea00640 1 --2- 172.21.15.165:0/121313412 >> v2:172.21.15.172:3300/0 conn(0x7f094015b8f0 0x7f09400c6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 -- 172.21.15.165:0/121313412 >> v2:172.21.15.172:3300/0 conn(0x7f094015b8f0 msgr2=0x7f09400c6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 --2- 172.21.15.165:0/121313412 >> v2:172.21.15.172:3300/0 conn(0x7f094015b8f0 0x7f09400c6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 -- 172.21.15.165:0/121313412 >> v1:172.21.15.172:6789/0 conn(0x7f09400b3920 legacy=0x7f094006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093e000640 1 -- 172.21.15.165:0/121313412 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09400c7380 con 0x7f09400cf340 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.014+0000 7f093ea00640 1 --2- 172.21.15.165:0/121313412 >> v2:172.21.15.172:3300/0 conn(0x7f094015b8f0 0x7f09400c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.016+0000 7f093e000640 1 --2- 172.21.15.165:0/121313412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f09400b33e0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f09400d27d0 tx=0x7f092802a300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c13d8eb85c5eaa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.016+0000 7f093d600640 1 -- 172.21.15.165:0/121313412 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928037070 con 0x7f09400cf340 2024-04-25T10:17:48.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.016+0000 7f093d600640 1 -- 172.21.15.165:0/121313412 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928032640 con 0x7f09400cf340 2024-04-25T10:17:48.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.016+0000 7f093d600640 1 -- 172.21.15.165:0/121313412 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928032a10 con 0x7f09400cf340 2024-04-25T10:17:48.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 -- 172.21.15.165:0/121313412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 msgr2=0x7f09400b33e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 --2- 172.21.15.165:0/121313412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f09400b33e0 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f09400d27d0 tx=0x7f092802a300 comp rx=0 tx=0).stop 2024-04-25T10:17:48.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 -- 172.21.15.165:0/121313412 shutdown_connections 2024-04-25T10:17:48.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 --2- 172.21.15.165:0/121313412 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f09400b33e0 unknown :-1 s=CLOSED pgs=2221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 --2- 172.21.15.165:0/121313412 >> v2:172.21.15.172:3300/0 conn(0x7f094015b8f0 0x7f09400c6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 -- 172.21.15.165:0/121313412 >> 172.21.15.165:0/121313412 conn(0x7f09400d1800 msgr2=0x7f09400d1c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 -- 172.21.15.165:0/121313412 shutdown_connections 2024-04-25T10:17:48.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.017+0000 7f0945c00640 1 -- 172.21.15.165:0/121313412 wait complete. 2024-04-25T10:17:48.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.019+0000 7f0945c00640 1 Processor -- start 2024-04-25T10:17:48.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.019+0000 7f0945c00640 1 -- start start 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f0945c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09400b3920 0x7f094015c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f0945c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f094015de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f0945c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094015b8f0 0x7f094015e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f0945c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400c7f90 con 0x7f09400cf340 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f0945c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09400c7940 con 0x7f094015b8f0 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f0945c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09400c7c20 con 0x7f09400b3920 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f093e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f094015de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f093e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f094015de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45700/0 (socket says 172.21.15.165:45700) 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.020+0000 7f093e000640 1 -- 172.21.15.165:0/2014274431 learned_addr learned my addr 172.21.15.165:0/2014274431 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:48.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093ea00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09400b3920 0x7f094015c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093e000640 1 -- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09400b3920 msgr2=0x7f094015c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093e000640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09400b3920 0x7f094015c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093f400640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094015b8f0 0x7f094015e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093e000640 1 -- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094015b8f0 msgr2=0x7f094015e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093e000640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094015b8f0 0x7f094015e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093e000640 1 -- 172.21.15.165:0/2014274431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094015d050 con 0x7f09400cf340 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093f400640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094015b8f0 0x7f094015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f093e000640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f094015de30 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f09280097f0 tx=0x7f09280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928037040 con 0x7f09400cf340 2024-04-25T10:17:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0928004230 con 0x7f09400cf340 2024-04-25T10:17:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0928030770 con 0x7f09400cf340 2024-04-25T10:17:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.021+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094015d330 con 0x7f09400cf340 2024-04-25T10:17:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.022+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094015cd10 con 0x7f09400cf340 2024-04-25T10:17:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.023+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f094015d500 con 0x7f09400cf340 2024-04-25T10:17:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.024+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0928032500 con 0x7f09400cf340 2024-04-25T10:17:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.024+0000 7f0936a00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f091c06b500 0x7f091c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.025+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f09280aae00 con 0x7f09400cf340 2024-04-25T10:17:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.025+0000 7f093ea00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f091c06b500 0x7f091c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.026+0000 7f093ea00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f091c06b500 0x7f091c06d9c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f093000b840 tx=0x7f0930008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.029+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0928045e50 con 0x7f09400cf340 2024-04-25T10:17:48.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.222+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f094015d500 con 0x7f09400cf340 2024-04-25T10:17:48.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.223+0000 7f0936a00640 1 -- 172.21.15.165:0/2014274431 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+13210 (secure 0 0 0) 0x7f0928079f50 con 0x7f09400cf340 2024-04-25T10:17:48.227 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:48.227 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":155,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:47.321467+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":146,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f091c06b500 msgr2=0x7f091c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f091c06b500 0x7f091c06d9c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f093000b840 tx=0x7f0930008040 comp rx=0 tx=0).stop 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 msgr2=0x7f094015de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f094015de30 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f09280097f0 tx=0x7f09280046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 shutdown_connections 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f091c06b500 0x7f091c06d9c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f094015b8f0 0x7f094015e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f09400cf340 0x7f094015de30 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 --2- 172.21.15.165:0/2014274431 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f09400b3920 0x7f094015c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.226+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 >> 172.21.15.165:0/2014274431 conn(0x7f09400d1800 msgr2=0x7f0940158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.227+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 shutdown_connections 2024-04-25T10:17:48.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.227+0000 7f0945c00640 1 -- 172.21.15.165:0/2014274431 wait complete. 2024-04-25T10:17:48.248 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:17:48.248 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:17:48.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.442+0000 7f46a3200640 1 Processor -- start 2024-04-25T10:17:48.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.442+0000 7f46a3200640 1 -- start start 2024-04-25T10:17:48.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.443+0000 7f46a3200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f46a413d9f0 0x7f46a413df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.443+0000 7f46a3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a40b0730 0x7f46a40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f46a406e150 0x7f46a406e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a3200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a40b10d0 con 0x7f46a406e150 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a3200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a406d390 con 0x7f46a413d9f0 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a406d670 con 0x7f46a40b0730 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f46a406e150 0x7f46a406e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f46a406e150 0x7f46a406e5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45706/0 (socket says 172.21.15.165:45706) 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 -- 172.21.15.165:0/1382683148 learned_addr learned my addr 172.21.15.165:0/1382683148 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a1e00640 1 --2- 172.21.15.165:0/1382683148 >> v2:172.21.15.172:3300/0 conn(0x7f46a413d9f0 0x7f46a413df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a1400640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a40b0730 0x7f46a40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 -- 172.21.15.165:0/1382683148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a40b0730 msgr2=0x7f46a40b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a40b0730 0x7f46a40b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 -- 172.21.15.165:0/1382683148 >> v2:172.21.15.172:3300/0 conn(0x7f46a413d9f0 msgr2=0x7f46a413df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 --2- 172.21.15.165:0/1382683148 >> v2:172.21.15.172:3300/0 conn(0x7f46a413d9f0 0x7f46a413df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a2800640 1 -- 172.21.15.165:0/1382683148 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a406d950 con 0x7f46a406e150 2024-04-25T10:17:48.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.444+0000 7f46a1400640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a40b0730 0x7f46a40b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.445+0000 7f46a2800640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a406e150 0x7f46a406e5a0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f46a4072520 tx=0x7f469c02a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67238930ffa4b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a0a00640 1 -- 172.21.15.165:0/1382683148 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c036070 con 0x7f46a406e150 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a0a00640 1 -- 172.21.15.165:0/1382683148 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f469c031600 con 0x7f46a406e150 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a0a00640 1 -- 172.21.15.165:0/1382683148 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c0319d0 con 0x7f46a406e150 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 -- 172.21.15.165:0/1382683148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a406e150 msgr2=0x7f46a406e5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a406e150 0x7f46a406e5a0 secure :-1 s=READY pgs=2223 cs=0 l=1 rev1=1 crypto rx=0x7f46a4072520 tx=0x7f469c02a570 comp rx=0 tx=0).stop 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 -- 172.21.15.165:0/1382683148 shutdown_connections 2024-04-25T10:17:48.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a406e150 0x7f46a406e5a0 unknown :-1 s=CLOSED pgs=2223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 --2- 172.21.15.165:0/1382683148 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a40b0730 0x7f46a40b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 --2- 172.21.15.165:0/1382683148 >> v2:172.21.15.172:3300/0 conn(0x7f46a413d9f0 0x7f46a413df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.446+0000 7f46a3200640 1 -- 172.21.15.165:0/1382683148 >> 172.21.15.165:0/1382683148 conn(0x7f46a4061cc0 msgr2=0x7f46a406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.447+0000 7f46a3200640 1 -- 172.21.15.165:0/1382683148 shutdown_connections 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.447+0000 7f46a3200640 1 -- 172.21.15.165:0/1382683148 wait complete. 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.448+0000 7f46a3200640 1 Processor -- start 2024-04-25T10:17:48.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.448+0000 7f46a3200640 1 -- start start 2024-04-25T10:17:48.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.449+0000 7f46a3200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46a406e150 0x7f46a41f2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.449+0000 7f46a3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 0x7f46a4152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.449+0000 7f46a3200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a413d9f0 0x7f46a415b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.449+0000 7f46a3200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a413e470 con 0x7f46a40b0730 2024-04-25T10:17:48.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.449+0000 7f46a3200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a406eae0 con 0x7f46a406e150 2024-04-25T10:17:48.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.449+0000 7f46a3200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a413e5f0 con 0x7f46a413d9f0 2024-04-25T10:17:48.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 0x7f46a4152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 0x7f46a4152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45710/0 (socket says 172.21.15.165:45710) 2024-04-25T10:17:48.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 -- 172.21.15.165:0/9772945 learned_addr learned my addr 172.21.15.165:0/9772945 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:48.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a2800640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a413d9f0 0x7f46a415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1e00640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46a406e150 0x7f46a41f2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 -- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a413d9f0 msgr2=0x7f46a415b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a413d9f0 0x7f46a415b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 -- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46a406e150 msgr2=0x7f46a41f2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46a406e150 0x7f46a41f2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 -- 172.21.15.165:0/9772945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a41f2b40 con 0x7f46a40b0730 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.450+0000 7f46a1400640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 0x7f46a4152ca0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f469000c7c0 tx=0x7f469000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.451+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690015070 con 0x7f46a40b0730 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.451+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4690007a30 con 0x7f46a40b0730 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.451+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4690007e00 con 0x7f46a40b0730 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.451+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a41535f0 con 0x7f46a40b0730 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.451+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a413bb20 con 0x7f46a40b0730 2024-04-25T10:17:48.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.452+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a413b630 con 0x7f46a40b0730 2024-04-25T10:17:48.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.453+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4690018070 con 0x7f46a40b0730 2024-04-25T10:17:48.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.454+0000 7f469a000640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f468406b4d0 0x7f468406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.454+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f4690089f80 con 0x7f46a40b0730 2024-04-25T10:17:48.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.454+0000 7f46a1e00640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f468406b4d0 0x7f468406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.455+0000 7f46a1e00640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f468406b4d0 0x7f468406d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4694006fd0 tx=0x7f4694008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.459+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4690058f20 con 0x7f46a40b0730 2024-04-25T10:17:48.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.656+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f46a406efa0 con 0x7f46a40b0730 2024-04-25T10:17:48.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.657+0000 7f469a000640 1 -- 172.21.15.165:0/9772945 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4690058820 con 0x7f46a40b0730 2024-04-25T10:17:48.660 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:17:48.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f468406b4d0 msgr2=0x7f468406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f468406b4d0 0x7f468406d990 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f4694006fd0 tx=0x7f4694008040 comp rx=0 tx=0).stop 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 msgr2=0x7f46a4152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 0x7f46a4152ca0 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7f469000c7c0 tx=0x7f469000cc80 comp rx=0 tx=0).stop 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 shutdown_connections 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f468406b4d0 0x7f468406d990 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f46a413d9f0 0x7f46a415b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f46a40b0730 0x7f46a4152ca0 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 --2- 172.21.15.165:0/9772945 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f46a406e150 0x7f46a41f2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 >> 172.21.15.165:0/9772945 conn(0x7f46a4061cc0 msgr2=0x7f46a40ce9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.660+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 shutdown_connections 2024-04-25T10:17:48.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.661+0000 7f46a3200640 1 -- 172.21.15.165:0/9772945 wait complete. 2024-04-25T10:17:48.682 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:17:48.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.875+0000 7fcf51400640 1 Processor -- start 2024-04-25T10:17:48.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.875+0000 7fcf51400640 1 -- start start 2024-04-25T10:17:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf51400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcf4c0ccb00 0x7fcf4c15f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf51400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf4c06d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf51400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf4c15c520 con 0x7fcf4c144ae0 2024-04-25T10:17:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf51400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4c15c800 con 0x7fcf4c0ccb00 2024-04-25T10:17:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf51400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c15cae0 con 0x7fcf4c15fef0 2024-04-25T10:17:48.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf4be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcf4c144ae0 0x7fcf4c15bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50228/0 (socket says 172.21.15.165:50228) 2024-04-25T10:17:48.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.877+0000 7fcf4be00640 1 -- 172.21.15.165:0/740715053 learned_addr learned my addr 172.21.15.165:0/740715053 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:48.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf50a00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf4c06d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf43400640 1 --2- 172.21.15.165:0/740715053 >> v2:172.21.15.172:3300/0 conn(0x7fcf4c0ccb00 0x7fcf4c15f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3393432324 0 0) 0x7fcf4c15c520 con 0x7fcf4c144ae0 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf30003520 con 0x7fcf4c144ae0 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2007801576 0 0) 0x7fcf30003520 con 0x7fcf4c144ae0 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 msgr2=0x7fcf4c06d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf4c06d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 >> v2:172.21.15.172:3300/0 conn(0x7fcf4c0ccb00 msgr2=0x7fcf4c15f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 --2- 172.21.15.165:0/740715053 >> v2:172.21.15.172:3300/0 conn(0x7fcf4c0ccb00 0x7fcf4c15f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c06d7b0 con 0x7fcf4c144ae0 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.878+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fcf3c001670 con 0x7fcf4c144ae0 2024-04-25T10:17:48.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 >> v1:172.21.15.165:6789/0 conn(0x7fcf4c144ae0 legacy=0x7fcf4c15bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf300039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30003f30 0x7fcf30004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf30004d40 0x7fcf300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c15c800 con 0x7fcf30004d40 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c15cae0 con 0x7fcf30003f30 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3000b9c0 con 0x7fcf4c15fef0 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf50a00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf30004d40 0x7fcf300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf43400640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30003f30 0x7fcf30004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4be00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf300039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf50a00640 1 -- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 msgr2=0x7fcf300039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf50a00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf300039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf50a00640 1 -- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30003f30 msgr2=0x7fcf30004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:48.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf50a00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30003f30 0x7fcf30004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf50a00640 1 -- 172.21.15.165:0/740715053 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf30003520 con 0x7fcf30004d40 2024-04-25T10:17:48.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.879+0000 7fcf4be00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c15fef0 0x7fcf300039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:48.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf50a00640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf30004d40 0x7fcf300051c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009fc0 tx=0x7fcf30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6de2d40e561670a7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44009c10 con 0x7fcf30004d40 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf44036070 con 0x7fcf30004d40 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf4b400640 1 -- 172.21.15.165:0/740715053 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf44031850 con 0x7fcf30004d40 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf51400640 1 -- 172.21.15.165:0/740715053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf30004d40 msgr2=0x7fcf300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf51400640 1 --2- 172.21.15.165:0/740715053 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf30004d40 0x7fcf300051c0 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fcf44009fc0 tx=0x7fcf30005830 comp rx=0 tx=0).stop 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.881+0000 7fcf50a00640 1 -- 172.21.15.165:0/740715053 reap_dead start 2024-04-25T10:17:48.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.882+0000 7fcf51400640 1 -- 172.21.15.165:0/740715053 shutdown_connections 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.882+0000 7fcf51400640 1 -- 172.21.15.165:0/740715053 >> 172.21.15.165:0/740715053 conn(0x7fcf4c061c30 msgr2=0x7fcf4c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.882+0000 7fcf51400640 1 -- 172.21.15.165:0/740715053 shutdown_connections 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.882+0000 7fcf51400640 1 -- 172.21.15.165:0/740715053 wait complete. 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.883+0000 7fcf51400640 1 Processor -- start 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.883+0000 7fcf51400640 1 -- start start 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.884+0000 7fcf51400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30004d40 0x7fcf4c13dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.884+0000 7fcf51400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 0x7fcf4c0af720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.884+0000 7fcf51400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c1f47d0 0x7fcf4c1f4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.884+0000 7fcf51400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c06d3f0 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.884+0000 7fcf51400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c06d270 con 0x7fcf30004d40 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.884+0000 7fcf51400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4c06d570 con 0x7fcf4c1f47d0 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 0x7fcf4c0af720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 0x7fcf4c0af720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45736/0 (socket says 172.21.15.165:45736) 2024-04-25T10:17:48.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 -- 172.21.15.165:0/4220057855 learned_addr learned my addr 172.21.15.165:0/4220057855 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 -- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c1f47d0 msgr2=0x7fcf4c1f4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf50a00640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c1f47d0 0x7fcf4c1f4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c1f47d0 0x7fcf4c1f4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 -- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30004d40 msgr2=0x7fcf4c13dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf4be00640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30004d40 0x7fcf4c13dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30004d40 0x7fcf4c13dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 -- 172.21.15.165:0/4220057855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf4c0ad410 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.885+0000 7fcf43400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 0x7fcf4c0af720 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fcf34011630 tx=0x7fcf34011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.886+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400f920 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.886+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3400fac0 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.886+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3400fe70 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.886+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4c0ad6f0 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.886+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4c0adcb0 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.887+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf4c073990 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.888+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcf34015460 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.889+0000 7fcf48c00640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf2806b500 0x7fcf2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:48.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.889+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcf3408aa00 con 0x7fcf4c0af2a0 2024-04-25T10:17:48.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.889+0000 7fcf4be00640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf2806b500 0x7fcf2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:48.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.890+0000 7fcf4be00640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf2806b500 0x7fcf2806d9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0017e0 tx=0x7fcf3c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:48.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:48.893+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcf34019030 con 0x7fcf4c0af2a0 2024-04-25T10:17:49.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.088+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcf4c0b1590 con 0x7fcf4c0af2a0 2024-04-25T10:17:49.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.089+0000 7fcf48c00640 1 -- 172.21.15.165:0/4220057855 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v155) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcf340599a0 con 0x7fcf4c0af2a0 2024-04-25T10:17:49.092 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:49.092 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:17:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:17:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:17:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:17:49.093 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:17:49.093 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:17:49.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf2806b500 msgr2=0x7fcf2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf2806b500 0x7fcf2806d9c0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c0017e0 tx=0x7fcf3c009040 comp rx=0 tx=0).stop 2024-04-25T10:17:49.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 msgr2=0x7fcf4c0af720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 0x7fcf4c0af720 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fcf34011630 tx=0x7fcf34011af0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 shutdown_connections 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcf2806b500 0x7fcf2806d9c0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcf4c1f47d0 0x7fcf4c1f4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcf4c0af2a0 0x7fcf4c0af720 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 --2- 172.21.15.165:0/4220057855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcf30004d40 0x7fcf4c13dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 >> 172.21.15.165:0/4220057855 conn(0x7fcf4c061c30 msgr2=0x7fcf4c0b1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 shutdown_connections 2024-04-25T10:17:49.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.092+0000 7fcf51400640 1 -- 172.21.15.165:0/4220057855 wait complete. 2024-04-25T10:17:49.114 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:17:49.114 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:17:49.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.319+0000 7f58c5000640 1 Processor -- start 2024-04-25T10:17:49.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.319+0000 7f58c5000640 1 -- start start 2024-04-25T10:17:49.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.320+0000 7f58c5000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f58c0154350 0x7f58c006db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.320+0000 7f58c5000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c015ae90 con 0x7f58c0154730 2024-04-25T10:17:49.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58c5000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58c015b170 con 0x7f58c00b0e80 2024-04-25T10:17:49.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58c5000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58c015b450 con 0x7f58c0154350 2024-04-25T10:17:49.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58bea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f58c0154730 0x7f58c015a520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50238/0 (socket says 172.21.15.165:50238) 2024-04-25T10:17:49.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58bea00640 1 -- 172.21.15.165:0/3132848851 learned_addr learned my addr 172.21.15.165:0/3132848851 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58bf400640 1 --2- 172.21.15.165:0/3132848851 >> v2:172.21.15.172:3300/0 conn(0x7f58c0154350 0x7f58c006db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570822906 0 0) 0x7f58c015ae90 con 0x7f58c0154730 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.321+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58a4003520 con 0x7f58c0154730 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3985253612 0 0) 0x7f58a4003520 con 0x7f58c0154730 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 >> v2:172.21.15.172:3300/0 conn(0x7f58c0154350 msgr2=0x7f58c006db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 --2- 172.21.15.165:0/3132848851 >> v2:172.21.15.172:3300/0 conn(0x7f58c0154350 0x7f58c006db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 >> v1:172.21.15.172:6789/0 conn(0x7f58c00b0e80 legacy=0x7f58c00711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c006d320 con 0x7f58c0154730 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f58ac001670 con 0x7f58c0154730 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 >> v1:172.21.15.165:6789/0 conn(0x7f58c0154730 legacy=0x7f58c015a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58a4003a00 0x7f58a4003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58a40052e0 0x7f58a400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c015b450 con 0x7f58a4004800 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a400e3f0 con 0x7f58a40052e0 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58a400e5f0 con 0x7f58a4003a00 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.322+0000 7f58bea00640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bf400640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58a4003a00 0x7f58a4003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bfe00640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58a40052e0 0x7f58a400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bea00640 1 -- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58a4003a00 msgr2=0x7f58a4003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bea00640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58a4003a00 0x7f58a4003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bea00640 1 -- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58a40052e0 msgr2=0x7f58a400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:49.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bea00640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58a40052e0 0x7f58a400dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bfe00640 1 -- 172.21.15.165:0/3132848851 reap_dead start 2024-04-25T10:17:49.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.323+0000 7f58bea00640 1 -- 172.21.15.165:0/3132848851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a4003520 con 0x7f58a4004800 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.324+0000 7f58bea00640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58a4004c80 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f58c0071180 tx=0x7f58a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25115d0719a02258 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.324+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac007c20 con 0x7f58a4004800 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.324+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac005af0 con 0x7f58a4004800 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.324+0000 7f58be000640 1 -- 172.21.15.165:0/3132848851 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac007070 con 0x7f58a4004800 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 -- 172.21.15.165:0/3132848851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 msgr2=0x7f58a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58a4004c80 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f58c0071180 tx=0x7f58a4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 -- 172.21.15.165:0/3132848851 shutdown_connections 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 --2- 172.21.15.165:0/3132848851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58a4004c80 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 -- 172.21.15.165:0/3132848851 >> 172.21.15.165:0/3132848851 conn(0x7f58c0061cc0 msgr2=0x7f58c006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 -- 172.21.15.165:0/3132848851 shutdown_connections 2024-04-25T10:17:49.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.325+0000 7f58c5000640 1 -- 172.21.15.165:0/3132848851 wait complete. 2024-04-25T10:17:49.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.327+0000 7f58c5000640 1 Processor -- start 2024-04-25T10:17:49.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.327+0000 7f58c5000640 1 -- start start 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58c5000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58c0151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58c5000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58c00aaac0 0x7f58c015c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58c5000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58c00aaf40 0x7f58c00b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58c5000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c006dca0 con 0x7f58a4004800 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58c5000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58c006db20 con 0x7f58c00aaf40 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58c5000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58c006de20 con 0x7f58c00aaac0 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58c0151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58c0151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45758/0 (socket says 172.21.15.165:45758) 2024-04-25T10:17:49.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 -- 172.21.15.165:0/3061824971 learned_addr learned my addr 172.21.15.165:0/3061824971 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bfe00640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58c00aaf40 0x7f58c00b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 -- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58c00aaac0 msgr2=0x7f58c015c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58c00aaac0 0x7f58c015c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 -- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58c00aaf40 msgr2=0x7f58c00b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58c00aaf40 0x7f58c00b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.328+0000 7f58bf400640 1 -- 172.21.15.165:0/3061824971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58c015d0e0 con 0x7f58a4004800 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.329+0000 7f58bf400640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58c0151200 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f58b00107a0 tx=0x7f58b0010c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.329+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0019070 con 0x7f58a4004800 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.329+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b000ec60 con 0x7f58a4004800 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.329+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58c015d3c0 con 0x7f58a4004800 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.329+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b000b040 con 0x7f58a4004800 2024-04-25T10:17:49.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.329+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58c00aa5a0 con 0x7f58a4004800 2024-04-25T10:17:49.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.331+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58c0073990 con 0x7f58a4004800 2024-04-25T10:17:49.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.331+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f58b001c070 con 0x7f58a4004800 2024-04-25T10:17:49.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.332+0000 7f58b7400640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58a006b500 0x7f58a006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:49.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.332+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f58b008a2b0 con 0x7f58a4004800 2024-04-25T10:17:49.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.333+0000 7f58bea00640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58a006b500 0x7f58a006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:49.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.333+0000 7f58bea00640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58a006b500 0x7f58a006d9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f58ac032000 tx=0x7f58ac007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:49.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.337+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f58b0023030 con 0x7f58a4004800 2024-04-25T10:17:49.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:49.534+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f58c006e750 con 0x7f58a4004800 2024-04-25T10:17:50.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.332+0000 7f58b7400640 1 -- 172.21.15.165:0/3061824971 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v156) v1 ==== 138+0+0 (secure 0 0 0) 0x7f58b00592f0 con 0x7f58a4004800 2024-04-25T10:17:50.336 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:17:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58a006b500 msgr2=0x7f58a006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58a006b500 0x7f58a006d9c0 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f58ac032000 tx=0x7f58ac007000 comp rx=0 tx=0).stop 2024-04-25T10:17:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 msgr2=0x7f58c0151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58c0151200 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f58b00107a0 tx=0x7f58b0010c60 comp rx=0 tx=0).stop 2024-04-25T10:17:50.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 shutdown_connections 2024-04-25T10:17:50.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f58a006b500 0x7f58a006d9c0 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f58c00aaf40 0x7f58c00b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f58c00aaac0 0x7f58c015c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 --2- 172.21.15.165:0/3061824971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f58a4004800 0x7f58c0151200 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.337+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 >> 172.21.15.165:0/3061824971 conn(0x7f58c0061cc0 msgr2=0x7f58c00b3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.339+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 shutdown_connections 2024-04-25T10:17:50.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.339+0000 7f58c5000640 1 -- 172.21.15.165:0/3061824971 wait complete. 2024-04-25T10:17:50.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:17:50.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.560+0000 7f6775200640 1 Processor -- start 2024-04-25T10:17:50.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.560+0000 7f6775200640 1 -- start start 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.561+0000 7f6775200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6770145140 0x7f67700ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.561+0000 7f6775200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f67700c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f6775200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67700c5030 0x7f677006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f6775200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67700ced90 con 0x7f67700c5030 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f6775200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6770145520 con 0x7f6770145140 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f6775200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6770071270 con 0x7f67700ce950 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f676fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67700c5030 0x7f677006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f676fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67700c5030 0x7f677006d1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45772/0 (socket says 172.21.15.165:45772) 2024-04-25T10:17:50.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f676fe00640 1 -- 172.21.15.165:0/973493620 learned_addr learned my addr 172.21.15.165:0/973493620 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:50.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f676ea00640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f67700c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.562+0000 7f676f400640 1 --2- 172.21.15.165:0/973493620 >> v2:172.21.15.172:3300/0 conn(0x7f6770145140 0x7f67700ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.563+0000 7f676fe00640 1 -- 172.21.15.165:0/973493620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 msgr2=0x7f67700c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.563+0000 7f676fe00640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f67700c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.563+0000 7f676fe00640 1 -- 172.21.15.165:0/973493620 >> v2:172.21.15.172:3300/0 conn(0x7f6770145140 msgr2=0x7f67700ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:50.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.563+0000 7f676fe00640 1 --2- 172.21.15.165:0/973493620 >> v2:172.21.15.172:3300/0 conn(0x7f6770145140 0x7f67700ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.563+0000 7f676fe00640 1 -- 172.21.15.165:0/973493620 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6770071550 con 0x7f67700c5030 2024-04-25T10:17:50.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.563+0000 7f676ea00640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f67700c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:50.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.564+0000 7f676fe00640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677006d1f0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f677006f690 tx=0x7f676802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44790ba7bbf561c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.564+0000 7f676e000640 1 -- 172.21.15.165:0/973493620 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768037070 con 0x7f67700c5030 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.564+0000 7f676e000640 1 -- 172.21.15.165:0/973493620 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6768032440 con 0x7f67700c5030 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.564+0000 7f676e000640 1 -- 172.21.15.165:0/973493620 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6768032810 con 0x7f67700c5030 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 -- 172.21.15.165:0/973493620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 msgr2=0x7f677006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677006d1f0 secure :-1 s=READY pgs=2231 cs=0 l=1 rev1=1 crypto rx=0x7f677006f690 tx=0x7f676802a140 comp rx=0 tx=0).stop 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 -- 172.21.15.165:0/973493620 shutdown_connections 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677006d1f0 unknown :-1 s=CLOSED pgs=2231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 --2- 172.21.15.165:0/973493620 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f67700c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 --2- 172.21.15.165:0/973493620 >> v2:172.21.15.172:3300/0 conn(0x7f6770145140 0x7f67700ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 -- 172.21.15.165:0/973493620 >> 172.21.15.165:0/973493620 conn(0x7f6770061cc0 msgr2=0x7f677006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:50.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 -- 172.21.15.165:0/973493620 shutdown_connections 2024-04-25T10:17:50.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.565+0000 7f6775200640 1 -- 172.21.15.165:0/973493620 wait complete. 2024-04-25T10:17:50.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.567+0000 7f6775200640 1 Processor -- start 2024-04-25T10:17:50.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.567+0000 7f6775200640 1 -- start start 2024-04-25T10:17:50.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f6775200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f6775200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f677013bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f6775200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6770145140 0x7f677013c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f6775200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c54a0 con 0x7f67700c5030 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f6775200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6770071b10 con 0x7f6770145140 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f6775200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c5620 con 0x7f67700ce950 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677015ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45776/0 (socket says 172.21.15.165:45776) 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 -- 172.21.15.165:0/792783581 learned_addr learned my addr 172.21.15.165:0/792783581 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676ea00640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f677013bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 -- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 msgr2=0x7f677013bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f677013bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676fe00640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6770145140 0x7f677013c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 -- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6770145140 msgr2=0x7f677013c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6770145140 0x7f677013c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.568+0000 7f676f400640 1 -- 172.21.15.165:0/792783581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f677013f010 con 0x7f67700c5030 2024-04-25T10:17:50.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.569+0000 7f676f400640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677015ddf0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f6760007e70 tx=0x7f676000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.569+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760014620 con 0x7f67700c5030 2024-04-25T10:17:50.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.569+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6760019070 con 0x7f67700c5030 2024-04-25T10:17:50.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.569+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760014a00 con 0x7f67700c5030 2024-04-25T10:17:50.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.569+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67701408d0 con 0x7f67700c5030 2024-04-25T10:17:50.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.569+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6770140e90 con 0x7f67700c5030 2024-04-25T10:17:50.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.570+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770073990 con 0x7f67700c5030 2024-04-25T10:17:50.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.572+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6760014ba0 con 0x7f67700c5030 2024-04-25T10:17:50.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.573+0000 7f6767400640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f674c06b4d0 0x7f674c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:50.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.573+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f676008b950 con 0x7f67700c5030 2024-04-25T10:17:50.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.573+0000 7f676ea00640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f674c06b4d0 0x7f674c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:50.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.574+0000 7f676ea00640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f674c06b4d0 0x7f674c06d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6758006b10 tx=0x7f6758008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:50.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.577+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f676001c360 con 0x7f67700c5030 2024-04-25T10:17:50.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:50.772+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f677015bae0 con 0x7f67700c5030 2024-04-25T10:17:51.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.347+0000 7f6767400640 1 -- 172.21.15.165:0/792783581 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v157) v1 ==== 170+0+0 (secure 0 0 0) 0x7f676005a800 con 0x7f67700c5030 2024-04-25T10:17:51.351 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:17:51.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f674c06b4d0 msgr2=0x7f674c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:51.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f674c06b4d0 0x7f674c06d990 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6758006b10 tx=0x7f6758008040 comp rx=0 tx=0).stop 2024-04-25T10:17:51.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 msgr2=0x7f677015ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:51.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677015ddf0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f6760007e70 tx=0x7f676000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 shutdown_connections 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f674c06b4d0 0x7f674c06d990 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6770145140 0x7f677013c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67700ce950 0x7f677013bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 --2- 172.21.15.165:0/792783581 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67700c5030 0x7f677015ddf0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.351+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 >> 172.21.15.165:0/792783581 conn(0x7f6770061cc0 msgr2=0x7f67700cc200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.352+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 shutdown_connections 2024-04-25T10:17:51.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.352+0000 7f6775200640 1 -- 172.21.15.165:0/792783581 wait complete. 2024-04-25T10:17:51.387 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:17:51.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.559+0000 7fa4eb400640 1 Processor -- start 2024-04-25T10:17:51.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.560+0000 7fa4eb400640 1 -- start start 2024-04-25T10:17:51.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4eb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa4e415dc20 0x7fa4e40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:51.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4eb400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e406cda0 con 0x7fa4e4161710 2024-04-25T10:17:51.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4eb400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4e415e000 con 0x7fa4e40d32f0 2024-04-25T10:17:51.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4eb400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4e40d1de0 con 0x7fa4e415dc20 2024-04-25T10:17:51.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4e8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa4e415dc20 0x7fa4e40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:51.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4e8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa4e415dc20 0x7fa4e40ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45786/0 (socket says 172.21.15.165:45786) 2024-04-25T10:17:51.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.561+0000 7fa4e8a00640 1 -- 172.21.15.165:0/2564379413 learned_addr learned my addr 172.21.15.165:0/2564379413 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:51.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.562+0000 7fa4e8a00640 1 -- 172.21.15.165:0/2564379413 >> v1:172.21.15.172:6789/0 conn(0x7fa4e40d32f0 legacy=0x7fa4e406c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:51.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.562+0000 7fa4e8a00640 1 -- 172.21.15.165:0/2564379413 >> v1:172.21.15.165:6789/0 conn(0x7fa4e4161710 legacy=0x7fa4e406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:51.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.562+0000 7fa4e8a00640 1 -- 172.21.15.165:0/2564379413 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e4161c60 con 0x7fa4e415dc20 2024-04-25T10:17:51.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.563+0000 7fa4e8a00640 1 --2- 172.21.15.165:0/2564379413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40ce670 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009870 tx=0x7fa4d402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85f9048ba75107ca server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:51.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.563+0000 7fa4e3400640 1 -- 172.21.15.165:0/2564379413 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4037070 con 0x7fa4e415dc20 2024-04-25T10:17:51.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4e3400640 1 -- 172.21.15.165:0/2564379413 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4d4032440 con 0x7fa4e415dc20 2024-04-25T10:17:51.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4e3400640 1 -- 172.21.15.165:0/2564379413 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4d4032810 con 0x7fa4e415dc20 2024-04-25T10:17:51.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 -- 172.21.15.165:0/2564379413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 msgr2=0x7fa4e40ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:51.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 --2- 172.21.15.165:0/2564379413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40ce670 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4009870 tx=0x7fa4d402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 -- 172.21.15.165:0/2564379413 shutdown_connections 2024-04-25T10:17:51.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 --2- 172.21.15.165:0/2564379413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40ce670 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 -- 172.21.15.165:0/2564379413 >> 172.21.15.165:0/2564379413 conn(0x7fa4e40d0420 msgr2=0x7fa4e40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:51.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 -- 172.21.15.165:0/2564379413 shutdown_connections 2024-04-25T10:17:51.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.564+0000 7fa4eb400640 1 -- 172.21.15.165:0/2564379413 wait complete. 2024-04-25T10:17:51.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.567+0000 7fa4eb400640 1 Processor -- start 2024-04-25T10:17:51.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.567+0000 7fa4eb400640 1 -- start start 2024-04-25T10:17:51.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4eb400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e40d32f0 0x7fa4e415b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:51.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4eb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40cbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:51.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4eb400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4e4161710 0x7fa4e415bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:51.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4eb400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4162500 con 0x7fa4e415dc20 2024-04-25T10:17:51.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4eb400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e4162220 con 0x7fa4e40d32f0 2024-04-25T10:17:51.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4eb400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4e40b5e50 con 0x7fa4e4161710 2024-04-25T10:17:51.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40cbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:51.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4e3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40cbf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45788/0 (socket says 172.21.15.165:45788) 2024-04-25T10:17:51.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.568+0000 7fa4e3e00640 1 -- 172.21.15.165:0/73276358 learned_addr learned my addr 172.21.15.165:0/73276358 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:51.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e8a00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e40d32f0 0x7fa4e415b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:51.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e9400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4e4161710 0x7fa4e415bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e3e00640 1 -- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4e4161710 msgr2=0x7fa4e415bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e3e00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4e4161710 0x7fa4e415bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e3e00640 1 -- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e40d32f0 msgr2=0x7fa4e415b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e3e00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e40d32f0 0x7fa4e415b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e3e00640 1 -- 172.21.15.165:0/73276358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e415f5c0 con 0x7fa4e415dc20 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e3e00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40cbf20 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00c790 tx=0x7fa4cc00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc016020 con 0x7fa4e415dc20 2024-04-25T10:17:51.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4cc006d50 con 0x7fa4e415dc20 2024-04-25T10:17:51.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4e415f8a0 con 0x7fa4e415dc20 2024-04-25T10:17:51.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.569+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4cc003bb0 con 0x7fa4e415dc20 2024-04-25T10:17:51.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.570+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4e415fe60 con 0x7fa4e415dc20 2024-04-25T10:17:51.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.571+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4e4160340 con 0x7fa4e415dc20 2024-04-25T10:17:51.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.572+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa4cc00a850 con 0x7fa4e415dc20 2024-04-25T10:17:51.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.573+0000 7fa4e0c00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4c006b4d0 0x7fa4c006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:51.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.573+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fa4cc089fe0 con 0x7fa4e415dc20 2024-04-25T10:17:51.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.573+0000 7fa4e8a00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4c006b4d0 0x7fa4c006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:51.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.574+0000 7fa4e8a00640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4c006b4d0 0x7fa4c006d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4034000 tx=0x7fa4d402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:51.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.579+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa4e4160340 con 0x7fa4e415dc20 2024-04-25T10:17:51.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:51.813+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa4e40d16b0 con 0x7fa4e415dc20 2024-04-25T10:17:52.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.352+0000 7fa4e0c00640 1 -- 172.21.15.165:0/73276358 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v109) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa4cc020020 con 0x7fa4e415dc20 2024-04-25T10:17:52.355 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 23 and data pool 24 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4c006b4d0 msgr2=0x7fa4c006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4c006b4d0 0x7fa4c006d990 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fa4d4034000 tx=0x7fa4d402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 msgr2=0x7fa4e40cbf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40cbf20 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7fa4cc00c790 tx=0x7fa4cc00cc50 comp rx=0 tx=0).stop 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 shutdown_connections 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa4c006b4d0 0x7fa4c006d990 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.355+0000 7fa4eb400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa4e4161710 0x7fa4e415bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.356+0000 7fa4eb400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa4e415dc20 0x7fa4e40cbf20 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.356+0000 7fa4eb400640 1 --2- 172.21.15.165:0/73276358 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa4e40d32f0 0x7fa4e415b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.356+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 >> 172.21.15.165:0/73276358 conn(0x7fa4e40d0420 msgr2=0x7fa4e40d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:52.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.356+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 shutdown_connections 2024-04-25T10:17:52.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.356+0000 7fa4eb400640 1 -- 172.21.15.165:0/73276358 wait complete. 2024-04-25T10:17:52.405 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:52.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.562+0000 7fa87b200640 1 Processor -- start 2024-04-25T10:17:52.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.562+0000 7fa87b200640 1 -- start start 2024-04-25T10:17:52.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.563+0000 7fa87b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 0x7fa874070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.563+0000 7fa87b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87415bf30 con 0x7fa8740cda30 2024-04-25T10:17:52.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.563+0000 7fa87b200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa87415c210 con 0x7fa874071530 2024-04-25T10:17:52.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.563+0000 7fa87b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa87415c860 con 0x7fa874144440 2024-04-25T10:17:52.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa87a800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa8740cda30 0x7fa87415ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50248/0 (socket says 172.21.15.165:50248) 2024-04-25T10:17:52.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa87a800640 1 -- 172.21.15.165:0/364578613 learned_addr learned my addr 172.21.15.165:0/364578613 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:52.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3982537589 0 0) 0x7fa87415bf30 con 0x7fa8740cda30 2024-04-25T10:17:52.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa85c003520 con 0x7fa8740cda30 2024-04-25T10:17:52.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa879e00640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 0x7fa874070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4210115893 0 0) 0x7fa85c003520 con 0x7fa8740cda30 2024-04-25T10:17:52.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 msgr2=0x7fa874070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 0x7fa874070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 >> v1:172.21.15.172:6789/0 conn(0x7fa874071530 legacy=0x7fa8740cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87415c4f0 con 0x7fa8740cda30 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa8700019f0 con 0x7fa8740cda30 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 >> v1:172.21.15.165:6789/0 conn(0x7fa8740cda30 legacy=0x7fa87415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.564+0000 7fa878a00640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 0x7fa85c0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa878a00640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa85c004060 0x7fa85c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa878a00640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa85c004d40 0x7fa85c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa87415c860 con 0x7fa85c004d40 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c00bc90 con 0x7fa85c004060 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85c00be90 con 0x7fa874144440 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa87a800640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa85c004d40 0x7fa85c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa879e00640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 0x7fa85c0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa879400640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa85c004060 0x7fa85c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa87a800640 1 -- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 msgr2=0x7fa85c0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa87a800640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa874144440 0x7fa85c0039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa87a800640 1 -- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa85c004060 msgr2=0x7fa85c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa87a800640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa85c004060 0x7fa85c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.565+0000 7fa87a800640 1 -- 172.21.15.165:0/364578613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa85c003520 con 0x7fa85c004d40 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.566+0000 7fa87a800640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa85c004d40 0x7fa85c0051c0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa87000a790 tx=0x7fa85c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3a8914cb117f1ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.566+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000a3b0 con 0x7fa85c004d40 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.566+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa87002f070 con 0x7fa85c004d40 2024-04-25T10:17:52.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.566+0000 7fa878a00640 1 -- 172.21.15.165:0/364578613 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87002bc40 con 0x7fa85c004d40 2024-04-25T10:17:52.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87b200640 1 -- 172.21.15.165:0/364578613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa85c004d40 msgr2=0x7fa85c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87b200640 1 --2- 172.21.15.165:0/364578613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa85c004d40 0x7fa85c0051c0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fa87000a790 tx=0x7fa85c005830 comp rx=0 tx=0).stop 2024-04-25T10:17:52.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87a800640 1 -- 172.21.15.165:0/364578613 reap_dead start 2024-04-25T10:17:52.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87b200640 1 -- 172.21.15.165:0/364578613 shutdown_connections 2024-04-25T10:17:52.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87b200640 1 -- 172.21.15.165:0/364578613 >> 172.21.15.165:0/364578613 conn(0x7fa87406abe0 msgr2=0x7fa87406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:52.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87b200640 1 -- 172.21.15.165:0/364578613 shutdown_connections 2024-04-25T10:17:52.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.567+0000 7fa87b200640 1 -- 172.21.15.165:0/364578613 wait complete. 2024-04-25T10:17:52.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.569+0000 7fa87b200640 1 Processor -- start 2024-04-25T10:17:52.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.569+0000 7fa87b200640 1 -- start start 2024-04-25T10:17:52.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa85c004d40 0x7fa8741e9130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8741f6dc0 0x7fa8741f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 0x7fa8741e6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8740cdea0 con 0x7fa8741f7780 2024-04-25T10:17:52.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa874071970 con 0x7fa8741f6dc0 2024-04-25T10:17:52.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa874070ff0 con 0x7fa85c004d40 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 0x7fa8741e6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 0x7fa8741e6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45804/0 (socket says 172.21.15.165:45804) 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa87a800640 1 -- 172.21.15.165:0/1113285892 learned_addr learned my addr 172.21.15.165:0/1113285892 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.570+0000 7fa879e00640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa85c004d40 0x7fa8741e9130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa879400640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8741f6dc0 0x7fa8741f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87a800640 1 -- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa85c004d40 msgr2=0x7fa8741e9130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87a800640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa85c004d40 0x7fa8741e9130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87a800640 1 -- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8741f6dc0 msgr2=0x7fa8741f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87a800640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8741f6dc0 0x7fa8741f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87a800640 1 -- 172.21.15.165:0/1113285892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8741e4d10 con 0x7fa8741f7780 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa879400640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8741f6dc0 0x7fa8741f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87a800640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 0x7fa8741e6fe0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fa87000a8b0 tx=0x7fa87415c100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa870044070 con 0x7fa8741f7780 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8700019a0 con 0x7fa8741f7780 2024-04-25T10:17:52.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87000a3b0 con 0x7fa8741f7780 2024-04-25T10:17:52.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8741e6500 con 0x7fa8741f7780 2024-04-25T10:17:52.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.571+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8741e6ac0 con 0x7fa8741f7780 2024-04-25T10:17:52.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.572+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa87415b6a0 con 0x7fa8741f7780 2024-04-25T10:17:52.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.573+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa870047070 con 0x7fa8741f7780 2024-04-25T10:17:52.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.574+0000 7fa86e000640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa85806b500 0x7fa85806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.574+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa8700aa310 con 0x7fa8741f7780 2024-04-25T10:17:52.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.575+0000 7fa879e00640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa85806b500 0x7fa85806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.575+0000 7fa879e00640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa85806b500 0x7fa85806d9c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fd0 tx=0x7fa868008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:52.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.579+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa870078ac0 con 0x7fa8741f7780 2024-04-25T10:17:52.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.760+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa874073990 con 0x7fa8741f7780 2024-04-25T10:17:52.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.763+0000 7fa86e000640 1 -- 172.21.15.165:0/1113285892 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fa87007cb60 con 0x7fa8741f7780 2024-04-25T10:17:52.766 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:52.767 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":158,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:51.815296+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:52.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.765+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa85806b500 msgr2=0x7fa85806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.765+0000 7fa87b200640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa85806b500 0x7fa85806d9c0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fa868006fd0 tx=0x7fa868008040 comp rx=0 tx=0).stop 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.765+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 msgr2=0x7fa8741e6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.765+0000 7fa87b200640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 0x7fa8741e6fe0 secure :-1 s=READY pgs=2238 cs=0 l=1 rev1=1 crypto rx=0x7fa87000a8b0 tx=0x7fa87415c100 comp rx=0 tx=0).stop 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 shutdown_connections 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa85806b500 0x7fa85806d9c0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8741f7780 0x7fa8741e6fe0 unknown :-1 s=CLOSED pgs=2238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8741f6dc0 0x7fa8741f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 --2- 172.21.15.165:0/1113285892 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa85c004d40 0x7fa8741e9130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 >> 172.21.15.165:0/1113285892 conn(0x7fa87406abe0 msgr2=0x7fa8740c5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 shutdown_connections 2024-04-25T10:17:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.766+0000 7fa87b200640 1 -- 172.21.15.165:0/1113285892 wait complete. 2024-04-25T10:17:52.787 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:52.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.979+0000 7fe4d5600640 1 Processor -- start 2024-04-25T10:17:52.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.979+0000 7fe4d5600640 1 -- start start 2024-04-25T10:17:52.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.980+0000 7fe4d5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4d015b320 0x7fe4d0164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.980+0000 7fe4d5600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d0071ed0 con 0x7fe4d00cc6c0 2024-04-25T10:17:52.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.980+0000 7fe4d5600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4d015a840 con 0x7fe4d013d190 2024-04-25T10:17:52.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.980+0000 7fe4d5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4d013d570 con 0x7fe4d015b320 2024-04-25T10:17:52.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4c6000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe4d00cc6c0 0x7fe4d0071230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50264/0 (socket says 172.21.15.165:50264) 2024-04-25T10:17:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4c6000640 1 -- 172.21.15.165:0/4261702792 learned_addr learned my addr 172.21.15.165:0/4261702792 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4cf400640 1 --2- 172.21.15.165:0/4261702792 >> v2:172.21.15.172:3300/0 conn(0x7fe4d015b320 0x7fe4d0164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2896912280 0 0) 0x7fe4d0071ed0 con 0x7fe4d00cc6c0 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4b4003520 con 0x7fe4d00cc6c0 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1669995167 0 0) 0x7fe4b4003520 con 0x7fe4d00cc6c0 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 >> v2:172.21.15.172:3300/0 conn(0x7fe4d015b320 msgr2=0x7fe4d0164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 --2- 172.21.15.165:0/4261702792 >> v2:172.21.15.172:3300/0 conn(0x7fe4d015b320 0x7fe4d0164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 >> v1:172.21.15.172:6789/0 conn(0x7fe4d013d190 legacy=0x7fe4d00cbfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d015ab20 con 0x7fe4d00cc6c0 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe4b8001670 con 0x7fe4d00cc6c0 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 >> v1:172.21.15.165:6789/0 conn(0x7fe4d00cc6c0 legacy=0x7fe4d0071230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.981+0000 7fe4ce000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4b40039b0 0x7fe4b4003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4ce000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4ce000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4005210 0x7fe4b400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d013d570 con 0x7fe4b4004800 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b400c140 con 0x7fe4b4005210 2024-04-25T10:17:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4b400e550 con 0x7fe4b40039b0 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4c6000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4c6000640 1 -- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4b40039b0 msgr2=0x7fe4b4003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4cea00640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4b40039b0 0x7fe4b4003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4c6000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4b40039b0 0x7fe4b4003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4c6000640 1 -- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4005210 msgr2=0x7fe4b400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4c6000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4005210 0x7fe4b400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4c6000640 1 -- 172.21.15.165:0/4261702792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4003520 con 0x7fe4b4004800 2024-04-25T10:17:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.982+0000 7fe4cf400640 1 -- 172.21.15.165:0/4261702792 reap_dead start 2024-04-25T10:17:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.983+0000 7fe4c6000640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c210 tx=0x7fe4b4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc7a5253137f5529 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.983+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8007d10 con 0x7fe4b4004800 2024-04-25T10:17:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b80058d0 con 0x7fe4b4004800 2024-04-25T10:17:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4ce000640 1 -- 172.21.15.165:0/4261702792 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8005d20 con 0x7fe4b4004800 2024-04-25T10:17:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 -- 172.21.15.165:0/4261702792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b4004800 msgr2=0x7fe4b4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c210 tx=0x7fe4b4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 -- 172.21.15.165:0/4261702792 shutdown_connections 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 --2- 172.21.15.165:0/4261702792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4b4004800 0x7fe4b4004c80 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 -- 172.21.15.165:0/4261702792 >> 172.21.15.165:0/4261702792 conn(0x7fe4d006abe0 msgr2=0x7fe4d006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 -- 172.21.15.165:0/4261702792 shutdown_connections 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.984+0000 7fe4d5600640 1 -- 172.21.15.165:0/4261702792 wait complete. 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.986+0000 7fe4d5600640 1 Processor -- start 2024-04-25T10:17:52.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.986+0000 7fe4d5600640 1 -- start start 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4d5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4004800 0x7fe4d006ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4d5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 0x7fe4d01f7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4d5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4d01e3ba0 0x7fe4d0153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4d5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00cc130 con 0x7fe4d01e6b90 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4d5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00cbfb0 con 0x7fe4b4004800 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4d5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4d00cc2b0 con 0x7fe4d01e3ba0 2024-04-25T10:17:52.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.987+0000 7fe4c6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 0x7fe4d01f7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 0x7fe4d01f7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45818/0 (socket says 172.21.15.165:45818) 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 -- 172.21.15.165:0/2488820853 learned_addr learned my addr 172.21.15.165:0/2488820853 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4cea00640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4004800 0x7fe4d006ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4cf400640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4d01e3ba0 0x7fe4d0153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 -- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4d01e3ba0 msgr2=0x7fe4d0153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4d01e3ba0 0x7fe4d0153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 -- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4004800 msgr2=0x7fe4d006ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4004800 0x7fe4d006ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 -- 172.21.15.165:0/2488820853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4d0139cb0 con 0x7fe4d01e6b90 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4cf400640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4d01e3ba0 0x7fe4d0153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:52.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.988+0000 7fe4c6000640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 0x7fe4d01f7260 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c330 tx=0x7fe4d015aa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:52.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.989+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8041020 con 0x7fe4d01e6b90 2024-04-25T10:17:52.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.989+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b8000ed0 con 0x7fe4d01e6b90 2024-04-25T10:17:52.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.989+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b8038040 con 0x7fe4d01e6b90 2024-04-25T10:17:52.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.989+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4d0139f90 con 0x7fe4d01e6b90 2024-04-25T10:17:52.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.989+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4d01e7660 con 0x7fe4d01e6b90 2024-04-25T10:17:52.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.990+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4d00685e0 con 0x7fe4d01e6b90 2024-04-25T10:17:52.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.991+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4b8044020 con 0x7fe4d01e6b90 2024-04-25T10:17:52.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.992+0000 7fe4c7400640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4ac06b520 0x7fe4ac06d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:52.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.992+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4b80aa460 con 0x7fe4d01e6b90 2024-04-25T10:17:52.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.992+0000 7fe4cea00640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4ac06b520 0x7fe4ac06d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:52.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.993+0000 7fe4cea00640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4ac06b520 0x7fe4ac06d9e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0050820 tx=0x7fe4c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:52.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:52.996+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4d00685e0 con 0x7fe4d01e6b90 2024-04-25T10:17:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.180+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4d0073990 con 0x7fe4d01e6b90 2024-04-25T10:17:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.182+0000 7fe4c7400640 1 -- 172.21.15.165:0/2488820853 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16933 (secure 0 0 0) 0x7fe4b804b030 con 0x7fe4d01e6b90 2024-04-25T10:17:53.185 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:53.186 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":158,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:51.815296+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.184+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4ac06b520 msgr2=0x7fe4ac06d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4ac06b520 0x7fe4ac06d9e0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe4d0050820 tx=0x7fe4c0009040 comp rx=0 tx=0).stop 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 msgr2=0x7fe4d01f7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 0x7fe4d01f7260 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7fe4b800c330 tx=0x7fe4d015aa10 comp rx=0 tx=0).stop 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 shutdown_connections 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4ac06b520 0x7fe4ac06d9e0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4d01e3ba0 0x7fe4d0153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4d01e6b90 0x7fe4d01f7260 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 --2- 172.21.15.165:0/2488820853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4b4004800 0x7fe4d006ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 >> 172.21.15.165:0/2488820853 conn(0x7fe4d006abe0 msgr2=0x7fe4d01599f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:53.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 shutdown_connections 2024-04-25T10:17:53.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.185+0000 7fe4d5600640 1 -- 172.21.15.165:0/2488820853 wait complete. 2024-04-25T10:17:53.206 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:17:53.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.399+0000 7fb83b200640 1 Processor -- start 2024-04-25T10:17:53.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.399+0000 7fb83b200640 1 -- start start 2024-04-25T10:17:53.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.400+0000 7fb83b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8340ced80 0x7fb8340d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.400+0000 7fb83b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.400+0000 7fb83b200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb83415e000 con 0x7fb83415dc20 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.400+0000 7fb83b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8340d1de0 con 0x7fb8340ced80 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.400+0000 7fb83b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb834161c60 con 0x7fb834161710 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8340ced80 0x7fb8340d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb83415dc20 0x7fb8340ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50278/0 (socket says 172.21.15.165:50278) 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833e00640 1 -- 172.21.15.165:0/992172566 learned_addr learned my addr 172.21.15.165:0/992172566 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:53.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb839200640 1 --2- 172.21.15.165:0/992172566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833400640 1 -- 172.21.15.165:0/992172566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 msgr2=0x7fb8340d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:53.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833400640 1 --2- 172.21.15.165:0/992172566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340d2820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833400640 1 -- 172.21.15.165:0/992172566 >> v1:172.21.15.165:6789/0 conn(0x7fb83415dc20 legacy=0x7fb8340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb833400640 1 -- 172.21.15.165:0/992172566 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb834161f40 con 0x7fb8340ced80 2024-04-25T10:17:53.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.401+0000 7fb832a00640 1 -- 172.21.15.165:0/992172566 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907671350 0 0) 0x7fb83415e000 con 0x7fb83415dc20 2024-04-25T10:17:53.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.402+0000 7fb833400640 1 --2- 172.21.15.165:0/992172566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8340ced80 0x7fb8340d22e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fb8200097a0 tx=0x7fb82002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5294c9be057ec4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.402+0000 7fb832a00640 1 -- 172.21.15.165:0/992172566 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820037070 con 0x7fb8340ced80 2024-04-25T10:17:53.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.402+0000 7fb832a00640 1 -- 172.21.15.165:0/992172566 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820032440 con 0x7fb8340ced80 2024-04-25T10:17:53.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.402+0000 7fb832a00640 1 -- 172.21.15.165:0/992172566 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820032810 con 0x7fb8340ced80 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 -- 172.21.15.165:0/992172566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8340ced80 msgr2=0x7fb8340d22e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 --2- 172.21.15.165:0/992172566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8340ced80 0x7fb8340d22e0 secure :-1 s=READY pgs=2243 cs=0 l=1 rev1=1 crypto rx=0x7fb8200097a0 tx=0x7fb82002a180 comp rx=0 tx=0).stop 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 -- 172.21.15.165:0/992172566 shutdown_connections 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 --2- 172.21.15.165:0/992172566 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340d2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 --2- 172.21.15.165:0/992172566 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8340ced80 0x7fb8340d22e0 unknown :-1 s=CLOSED pgs=2243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 -- 172.21.15.165:0/992172566 >> 172.21.15.165:0/992172566 conn(0x7fb8340d0420 msgr2=0x7fb8340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 -- 172.21.15.165:0/992172566 shutdown_connections 2024-04-25T10:17:53.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.403+0000 7fb83b200640 1 -- 172.21.15.165:0/992172566 wait complete. 2024-04-25T10:17:53.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.405+0000 7fb83b200640 1 Processor -- start 2024-04-25T10:17:53.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.405+0000 7fb83b200640 1 -- start start 2024-04-25T10:17:53.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb83b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8340ced80 0x7fb834151e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb83b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 0x7fb8341e7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb83b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340c7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb83b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340b5e50 con 0x7fb83415dc20 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb83b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb834162500 con 0x7fb8340ced80 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb83b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8340b5fd0 con 0x7fb834161710 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb833400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 0x7fb8341e7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb833400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 0x7fb8341e7f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45828/0 (socket says 172.21.15.165:45828) 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.406+0000 7fb833400640 1 -- 172.21.15.165:0/3923513526 learned_addr learned my addr 172.21.15.165:0/3923513526 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb839200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340c7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833e00640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8340ced80 0x7fb834151e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833400640 1 -- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 msgr2=0x7fb8340c7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833400640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340c7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833400640 1 -- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8340ced80 msgr2=0x7fb834151e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833400640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8340ced80 0x7fb834151e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833400640 1 -- 172.21.15.165:0/3923513526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8341e8770 con 0x7fb83415dc20 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb833400640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 0x7fb8341e7f20 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fb8200098c0 tx=0x7fb8200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb820037040 con 0x7fb83415dc20 2024-04-25T10:17:53.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb820030500 con 0x7fb83415dc20 2024-04-25T10:17:53.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8200308d0 con 0x7fb83415dc20 2024-04-25T10:17:53.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.407+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8341e8a50 con 0x7fb83415dc20 2024-04-25T10:17:53.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.408+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8341e9010 con 0x7fb83415dc20 2024-04-25T10:17:53.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.409+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb834073a00 con 0x7fb83415dc20 2024-04-25T10:17:53.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.409+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb820044020 con 0x7fb83415dc20 2024-04-25T10:17:53.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.411+0000 7fb82be00640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb81406b4d0 0x7fb81406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:53.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.411+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8200aa710 con 0x7fb83415dc20 2024-04-25T10:17:53.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.411+0000 7fb833e00640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb81406b4d0 0x7fb81406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:53.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.412+0000 7fb833e00640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb81406b4d0 0x7fb81406d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb824000f60 tx=0x7fb824009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:53.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.415+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb8200792d0 con 0x7fb83415dc20 2024-04-25T10:17:53.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:53.644+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb8340d16b0 con 0x7fb83415dc20 2024-04-25T10:17:54.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.402+0000 7fb82be00640 1 -- 172.21.15.165:0/3923513526 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v112) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb820078bd0 con 0x7fb83415dc20 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb81406b4d0 msgr2=0x7fb81406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb81406b4d0 0x7fb81406d990 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fb824000f60 tx=0x7fb824009040 comp rx=0 tx=0).stop 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 msgr2=0x7fb8341e7f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 0x7fb8341e7f20 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7fb8200098c0 tx=0x7fb8200046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 shutdown_connections 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb81406b4d0 0x7fb81406d990 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb834161710 0x7fb8340c7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb83415dc20 0x7fb8341e7f20 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 --2- 172.21.15.165:0/3923513526 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8340ced80 0x7fb834151e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 >> 172.21.15.165:0/3923513526 conn(0x7fb8340d0420 msgr2=0x7fb8340b4800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 shutdown_connections 2024-04-25T10:17:54.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.406+0000 7fb83b200640 1 -- 172.21.15.165:0/3923513526 wait complete. 2024-04-25T10:17:54.431 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:17:54.431 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:17:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.632+0000 7f6ee3e00640 1 Processor -- start 2024-04-25T10:17:54.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.632+0000 7f6ee3e00640 1 -- start start 2024-04-25T10:17:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.634+0000 7f6ee3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 0x7f6edc071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.634+0000 7f6ee3e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc162ba0 con 0x7f6edc06ef10 2024-04-25T10:17:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.634+0000 7f6ee3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6edc162e80 con 0x7f6edc161fc0 2024-04-25T10:17:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.634+0000 7f6ee3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc165be0 con 0x7f6edc06ea90 2024-04-25T10:17:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.635+0000 7f6ee1e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6edc06ef10 0x7f6edc06d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50294/0 (socket says 172.21.15.165:50294) 2024-04-25T10:17:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.635+0000 7f6ee1e00640 1 -- 172.21.15.165:0/4071882471 learned_addr learned my addr 172.21.15.165:0/4071882471 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:54.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.635+0000 7f6ee0a00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 0x7f6edc071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.635+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689728361 0 0) 0x7f6edc162ba0 con 0x7f6edc06ef10 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.635+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ec4003520 con 0x7f6edc06ef10 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.635+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3215433114 0 0) 0x7f6ec4003520 con 0x7f6edc06ef10 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 msgr2=0x7f6edc071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 0x7f6edc071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 >> v1:172.21.15.172:6789/0 conn(0x7f6edc161fc0 legacy=0x7f6edc1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc163160 con 0x7f6edc06ef10 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6ed4001670 con 0x7f6edc06ef10 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 >> v1:172.21.15.165:6789/0 conn(0x7f6edc06ef10 legacy=0x7f6edc06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ec40039e0 0x7f6ec4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 0x7f6ec4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6ec40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec400baf0 con 0x7f6ec4004d40 2024-04-25T10:17:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec400bca0 con 0x7f6ec40039e0 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ec400bea0 con 0x7f6edc06ea90 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6ee1e00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6ec40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6ee1400640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ec40039e0 0x7f6ec4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6ee0a00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 0x7f6ec4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6ee1e00640 1 -- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 msgr2=0x7f6ec4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.636+0000 7f6ee1e00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc06ea90 0x7f6ec4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.637+0000 7f6ee1e00640 1 -- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ec40039e0 msgr2=0x7f6ec4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.637+0000 7f6ee1e00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6ec40039e0 0x7f6ec4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.637+0000 7f6ee1e00640 1 -- 172.21.15.165:0/4071882471 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ec4003520 con 0x7f6ec4004d40 2024-04-25T10:17:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6ee1e00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6ec40051c0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c210 tx=0x7f6ec4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c439f361604e56ce server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4007cf0 con 0x7f6ec4004d40 2024-04-25T10:17:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed4005af0 con 0x7f6ec4004d40 2024-04-25T10:17:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6edbe00640 1 -- 172.21.15.165:0/4071882471 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4007070 con 0x7f6ec4004d40 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6ee3e00640 1 -- 172.21.15.165:0/4071882471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 msgr2=0x7f6ec40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/4071882471 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6ec40051c0 secure :-1 s=READY pgs=2246 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c210 tx=0x7f6ec4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6ee1e00640 1 -- 172.21.15.165:0/4071882471 reap_dead start 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6ee3e00640 1 -- 172.21.15.165:0/4071882471 shutdown_connections 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.638+0000 7f6ee3e00640 1 -- 172.21.15.165:0/4071882471 >> 172.21.15.165:0/4071882471 conn(0x7f6edc06abe0 msgr2=0x7f6edc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.639+0000 7f6ee3e00640 1 -- 172.21.15.165:0/4071882471 shutdown_connections 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.639+0000 7f6ee3e00640 1 -- 172.21.15.165:0/4071882471 wait complete. 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.640+0000 7f6ee3e00640 1 Processor -- start 2024-04-25T10:17:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.640+0000 7f6ee3e00640 1 -- start start 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.641+0000 7f6ee3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6edc15ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.641+0000 7f6ee3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc15d5d0 0x7f6edc0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.641+0000 7f6ee3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6edc1549d0 0x7f6edc152630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.641+0000 7f6ee3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc06db60 con 0x7f6ec4004d40 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.641+0000 7f6ee3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc06d9e0 con 0x7f6edc1549d0 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.641+0000 7f6ee3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc071480 con 0x7f6edc15d5d0 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6edc15ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6edc15ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45852/0 (socket says 172.21.15.165:45852) 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 -- 172.21.15.165:0/2948493281 learned_addr learned my addr 172.21.15.165:0/2948493281 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 -- 172.21.15.165:0/2948493281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc15d5d0 msgr2=0x7f6edc0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc15d5d0 0x7f6edc0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 -- 172.21.15.165:0/2948493281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6edc1549d0 msgr2=0x7f6edc152630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6edc1549d0 0x7f6edc152630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 -- 172.21.15.165:0/2948493281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc0b0df0 con 0x7f6ec4004d40 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.642+0000 7f6ee1400640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6edc15ad70 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e550 tx=0x7f6ed000ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.643+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed00052e0 con 0x7f6ec4004d40 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.643+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed0018070 con 0x7f6ec4004d40 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.643+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc0b1070 con 0x7f6ec4004d40 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.643+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed0003810 con 0x7f6ec4004d40 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.643+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc0b1630 con 0x7f6ec4004d40 2024-04-25T10:17:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.644+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6edc073a00 con 0x7f6ec4004d40 2024-04-25T10:17:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.645+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6ed00039b0 con 0x7f6ec4004d40 2024-04-25T10:17:54.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.647+0000 7f6ed9600640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ec006b4d0 0x7f6ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:54.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.647+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6ed0089fd0 con 0x7f6ec4004d40 2024-04-25T10:17:54.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.647+0000 7f6ee0a00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ec006b4d0 0x7f6ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:54.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.648+0000 7f6ee0a00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ec006b4d0 0x7f6ec006d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0d03c0 tx=0x7f6ecc009750 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:54.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.652+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6edc0b0df0 con 0x7f6ec4004d40 2024-04-25T10:17:54.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:54.898+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6edc1ec110 con 0x7f6ec4004d40 2024-04-25T10:17:55.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.413+0000 7f6ed9600640 1 -- 172.21.15.165:0/2948493281 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v113) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6ed0058cb0 con 0x7f6ec4004d40 2024-04-25T10:17:55.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ec006b4d0 msgr2=0x7f6ec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ec006b4d0 0x7f6ec006d990 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0d03c0 tx=0x7f6ecc009750 comp rx=0 tx=0).stop 2024-04-25T10:17:55.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 msgr2=0x7f6edc15ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6edc15ad70 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f6ed000e550 tx=0x7f6ed000ea10 comp rx=0 tx=0).stop 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 shutdown_connections 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ec006b4d0 0x7f6ec006d990 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6edc1549d0 0x7f6edc152630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6edc15d5d0 0x7f6edc0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 --2- 172.21.15.165:0/2948493281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6ec4004d40 0x7f6edc15ad70 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 >> 172.21.15.165:0/2948493281 conn(0x7f6edc06abe0 msgr2=0x7f6edc0b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 shutdown_connections 2024-04-25T10:17:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.417+0000 7f6ee3e00640 1 -- 172.21.15.165:0/2948493281 wait complete. 2024-04-25T10:17:55.441 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:17:55.442 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:17:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.639+0000 7f0f4aa00640 1 Processor -- start 2024-04-25T10:17:55.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.639+0000 7f0f4aa00640 1 -- start start 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f4aa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0f44071700 0x7f0f440cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f4aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0f440cd860 0x7f0f4415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f4aa00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4415bf30 con 0x7f0f44144440 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f4aa00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4415c860 con 0x7f0f440cd860 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f4aa00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f4415c210 con 0x7f0f44071700 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f49600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0f44144440 0x7f0f44070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50308/0 (socket says 172.21.15.165:50308) 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f49600640 1 -- 172.21.15.165:0/869309579 learned_addr learned my addr 172.21.15.165:0/869309579 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f4a000640 1 --2- 172.21.15.165:0/869309579 >> v2:172.21.15.165:3300/0 conn(0x7f0f440cd860 0x7f0f4415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f43e00640 1 -- 172.21.15.165:0/869309579 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075054807 0 0) 0x7f0f4415bf30 con 0x7f0f44144440 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f48c00640 1 --2- 172.21.15.165:0/869309579 >> v2:172.21.15.172:3300/0 conn(0x7f0f44071700 0x7f0f440cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.641+0000 7f0f43e00640 1 -- 172.21.15.165:0/869309579 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f2c003520 con 0x7f0f44144440 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.642+0000 7f0f4a000640 1 -- 172.21.15.165:0/869309579 >> v2:172.21.15.172:3300/0 conn(0x7f0f44071700 msgr2=0x7f0f440cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.642+0000 7f0f4a000640 1 --2- 172.21.15.165:0/869309579 >> v2:172.21.15.172:3300/0 conn(0x7f0f44071700 0x7f0f440cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.642+0000 7f0f4a000640 1 -- 172.21.15.165:0/869309579 >> v1:172.21.15.165:6789/0 conn(0x7f0f44144440 legacy=0x7f0f44070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.642+0000 7f0f4a000640 1 -- 172.21.15.165:0/869309579 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4415c4f0 con 0x7f0f440cd860 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.643+0000 7f0f4a000640 1 --2- 172.21.15.165:0/869309579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f440cd860 0x7f0f4415ccf0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f0f4406b970 tx=0x7f0f3c02a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795eda719f353fb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.643+0000 7f0f43e00640 1 -- 172.21.15.165:0/869309579 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c007740 con 0x7f0f440cd860 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.643+0000 7f0f43e00640 1 -- 172.21.15.165:0/869309579 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c0078e0 con 0x7f0f440cd860 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.643+0000 7f0f43e00640 1 -- 172.21.15.165:0/869309579 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c007cb0 con 0x7f0f440cd860 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 -- 172.21.15.165:0/869309579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f440cd860 msgr2=0x7f0f4415ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/869309579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f440cd860 0x7f0f4415ccf0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7f0f4406b970 tx=0x7f0f3c02a710 comp rx=0 tx=0).stop 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 -- 172.21.15.165:0/869309579 shutdown_connections 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/869309579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f440cd860 0x7f0f4415ccf0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/869309579 >> v2:172.21.15.172:3300/0 conn(0x7f0f44071700 0x7f0f440cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 -- 172.21.15.165:0/869309579 >> 172.21.15.165:0/869309579 conn(0x7f0f4406abe0 msgr2=0x7f0f4406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 -- 172.21.15.165:0/869309579 shutdown_connections 2024-04-25T10:17:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.644+0000 7f0f4aa00640 1 -- 172.21.15.165:0/869309579 wait complete. 2024-04-25T10:17:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.646+0000 7f0f4aa00640 1 Processor -- start 2024-04-25T10:17:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.646+0000 7f0f4aa00640 1 -- start start 2024-04-25T10:17:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f4aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 0x7f0f4413c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f4aa00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f440cd860 0x7f0f44158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f4aa00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f44144440 0x7f0f4413a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f4aa00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440cde70 con 0x7f0f44071700 2024-04-25T10:17:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f4aa00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f440cdcf0 con 0x7f0f440cd860 2024-04-25T10:17:55.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f4aa00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415d230 con 0x7f0f44144440 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f49600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 0x7f0f4413c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f49600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 0x7f0f4413c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45874/0 (socket says 172.21.15.165:45874) 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.647+0000 7f0f49600640 1 -- 172.21.15.165:0/559865575 learned_addr learned my addr 172.21.15.165:0/559865575 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f48c00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f440cd860 0x7f0f44158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f49600640 1 -- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f44144440 msgr2=0x7f0f4413a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f49600640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f44144440 0x7f0f4413a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f49600640 1 -- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f440cd860 msgr2=0x7f0f44158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f49600640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f440cd860 0x7f0f44158870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f49600640 1 -- 172.21.15.165:0/559865575 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f44139460 con 0x7f0f44071700 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f49600640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 0x7f0f4413c010 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800c390 tx=0x7f0f3800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f38017070 con 0x7f0f44071700 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f38005ce0 con 0x7f0f44071700 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f380072f0 con 0x7f0f44071700 2024-04-25T10:17:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f44139740 con 0x7f0f44071700 2024-04-25T10:17:55.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.648+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f44139d00 con 0x7f0f44071700 2024-04-25T10:17:55.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.649+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f44073990 con 0x7f0f44071700 2024-04-25T10:17:55.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.650+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f3801a070 con 0x7f0f44071700 2024-04-25T10:17:55.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.651+0000 7f0f41600640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f2406b4d0 0x7f0f2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:55.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.651+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0f38089720 con 0x7f0f44071700 2024-04-25T10:17:55.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.652+0000 7f0f48c00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f2406b4d0 0x7f0f2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:55.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.652+0000 7f0f48c00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f2406b4d0 0x7f0f2406d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0f441f6830 tx=0x7f0f30008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:55.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.656+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f38058400 con 0x7f0f44071700 2024-04-25T10:17:55.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.894+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f441e9480 con 0x7f0f44071700 2024-04-25T10:17:55.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.895+0000 7f0f41600640 1 -- 172.21.15.165:0/559865575 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f0f38057d00 con 0x7f0f44071700 2024-04-25T10:17:55.898 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:51.816118+0000","modified":"2024-04-25T10:17:54.899844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6742":{"gid":6742,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/1799333237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1799333237},{"type":"v1","addr":"172.21.15.172:6834","nonce":1799333237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:17:55.899 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 113 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f2406b4d0 msgr2=0x7f0f2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f2406b4d0 0x7f0f2406d990 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f0f441f6830 tx=0x7f0f30008040 comp rx=0 tx=0).stop 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 msgr2=0x7f0f4413c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 0x7f0f4413c010 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7f0f3800c390 tx=0x7f0f3800c850 comp rx=0 tx=0).stop 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 shutdown_connections 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f2406b4d0 0x7f0f2406d990 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f44144440 0x7f0f4413a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f440cd860 0x7f0f44158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 --2- 172.21.15.165:0/559865575 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f44071700 0x7f0f4413c010 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:55.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 >> 172.21.15.165:0/559865575 conn(0x7f0f4406abe0 msgr2=0x7f0f440b3620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:55.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 shutdown_connections 2024-04-25T10:17:55.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:55.898+0000 7f0f4aa00640 1 -- 172.21.15.165:0/559865575 wait complete. 2024-04-25T10:17:55.919 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.108+0000 7f914ec00640 1 Processor -- start 2024-04-25T10:17:56.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.108+0000 7f914ec00640 1 -- start start 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.109+0000 7f914ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9148144440 0x7f9148070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9148071530 0x7f91480cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91480cd860 0x7f914806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91480719a0 con 0x7f9148071530 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f914815c860 con 0x7f9148144440 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f914815bf30 con 0x7f91480cd860 2024-04-25T10:17:56.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9148071530 0x7f91480cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ce00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9148071530 0x7f91480cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45890/0 (socket says 172.21.15.165:45890) 2024-04-25T10:17:56.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914ce00640 1 -- 172.21.15.165:0/968195633 learned_addr learned my addr 172.21.15.165:0/968195633 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:56.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914d800640 1 --2- 172.21.15.165:0/968195633 >> v2:172.21.15.172:3300/0 conn(0x7f9148144440 0x7f9148070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.110+0000 7f914e200640 1 --2- 172.21.15.165:0/968195633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91480cd860 0x7f914806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.111+0000 7f914ce00640 1 -- 172.21.15.165:0/968195633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91480cd860 msgr2=0x7f914806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.111+0000 7f914ce00640 1 --2- 172.21.15.165:0/968195633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91480cd860 0x7f914806d050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.111+0000 7f914ce00640 1 -- 172.21.15.165:0/968195633 >> v2:172.21.15.172:3300/0 conn(0x7f9148144440 msgr2=0x7f9148070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.111+0000 7f914ce00640 1 --2- 172.21.15.165:0/968195633 >> v2:172.21.15.172:3300/0 conn(0x7f9148144440 0x7f9148070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.111+0000 7f914ce00640 1 -- 172.21.15.165:0/968195633 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f914815c210 con 0x7f9148071530 2024-04-25T10:17:56.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.112+0000 7f914ce00640 1 --2- 172.21.15.165:0/968195633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9148071530 0x7f91480cd320 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f914806f470 tx=0x7f913402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a425e2b6d7dc82 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.112+0000 7f9147e00640 1 -- 172.21.15.165:0/968195633 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134037070 con 0x7f9148071530 2024-04-25T10:17:56.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.112+0000 7f9147e00640 1 -- 172.21.15.165:0/968195633 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9134032440 con 0x7f9148071530 2024-04-25T10:17:56.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.112+0000 7f9147e00640 1 -- 172.21.15.165:0/968195633 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134032810 con 0x7f9148071530 2024-04-25T10:17:56.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 -- 172.21.15.165:0/968195633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9148071530 msgr2=0x7f91480cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 --2- 172.21.15.165:0/968195633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9148071530 0x7f91480cd320 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7f914806f470 tx=0x7f913402a270 comp rx=0 tx=0).stop 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 -- 172.21.15.165:0/968195633 shutdown_connections 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 --2- 172.21.15.165:0/968195633 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91480cd860 0x7f914806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 --2- 172.21.15.165:0/968195633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9148071530 0x7f91480cd320 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 --2- 172.21.15.165:0/968195633 >> v2:172.21.15.172:3300/0 conn(0x7f9148144440 0x7f9148070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.113+0000 7f914ec00640 1 -- 172.21.15.165:0/968195633 >> 172.21.15.165:0/968195633 conn(0x7f914806abe0 msgr2=0x7f914806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.114+0000 7f914ec00640 1 -- 172.21.15.165:0/968195633 shutdown_connections 2024-04-25T10:17:56.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.114+0000 7f914ec00640 1 -- 172.21.15.165:0/968195633 wait complete. 2024-04-25T10:17:56.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.115+0000 7f914ec00640 1 Processor -- start 2024-04-25T10:17:56.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.116+0000 7f914ec00640 1 -- start start 2024-04-25T10:17:56.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.116+0000 7f914ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9148071530 0x7f914815adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 0x7f91480cb620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9148144440 0x7f91480cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480cdcd0 con 0x7f91480cd860 2024-04-25T10:17:56.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f914815bc30 con 0x7f9148071530 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91480cde50 con 0x7f9148144440 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 0x7f91480cb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 0x7f91480cb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45906/0 (socket says 172.21.15.165:45906) 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 -- 172.21.15.165:0/3574993183 learned_addr learned my addr 172.21.15.165:0/3574993183 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914d800640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9148071530 0x7f914815adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 -- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9148144440 msgr2=0x7f91480cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914e200640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9148144440 0x7f91480cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9148144440 0x7f91480cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 -- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9148071530 msgr2=0x7f914815adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9148071530 0x7f914815adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914ce00640 1 -- 172.21.15.165:0/3574993183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91481e49c0 con 0x7f91480cd860 2024-04-25T10:17:56.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.117+0000 7f914d800640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9148071530 0x7f914815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:56.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.118+0000 7f914ce00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 0x7f91480cb620 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f913402a2b0 tx=0x7f91340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.118+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134041020 con 0x7f91480cd860 2024-04-25T10:17:56.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.118+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91340306c0 con 0x7f91480cd860 2024-04-25T10:17:56.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.118+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9134037040 con 0x7f91480cd860 2024-04-25T10:17:56.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.118+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91481e4ca0 con 0x7f91480cd860 2024-04-25T10:17:56.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.118+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91481e5260 con 0x7f91480cd860 2024-04-25T10:17:56.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.119+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91480ce4b0 con 0x7f91480cd860 2024-04-25T10:17:56.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.120+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9134044020 con 0x7f91480cd860 2024-04-25T10:17:56.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.121+0000 7f9145600640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f912806b4d0 0x7f912806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.121+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f91340aabd0 con 0x7f91480cd860 2024-04-25T10:17:56.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.122+0000 7f914d800640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f912806b4d0 0x7f912806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.122+0000 7f914d800640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f912806b4d0 0x7f912806d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f913c009f50 tx=0x7f913c00b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.126+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91480ce4b0 con 0x7f91480cd860 2024-04-25T10:17:56.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.322+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9148073990 con 0x7f91480cd860 2024-04-25T10:17:56.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.325+0000 7f9145600640 1 -- 172.21.15.165:0/3574993183 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16920 (secure 0 0 0) 0x7f913404b030 con 0x7f91480cd860 2024-04-25T10:17:56.329 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:56.329 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":159,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:53.373638+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:17:56.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f912806b4d0 msgr2=0x7f912806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f912806b4d0 0x7f912806d990 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f913c009f50 tx=0x7f913c00b040 comp rx=0 tx=0).stop 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 msgr2=0x7f91480cb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 0x7f91480cb620 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7f913402a2b0 tx=0x7f91340046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 shutdown_connections 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f912806b4d0 0x7f912806d990 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9148144440 0x7f91480cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91480cd860 0x7f91480cb620 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 --2- 172.21.15.165:0/3574993183 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9148071530 0x7f914815adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.328+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 >> 172.21.15.165:0/3574993183 conn(0x7f914806abe0 msgr2=0x7f91480c8860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.329+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 shutdown_connections 2024-04-25T10:17:56.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.329+0000 7f914ec00640 1 -- 172.21.15.165:0/3574993183 wait complete. 2024-04-25T10:17:56.351 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:17:56.351 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:56.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.553+0000 7f7b4f200640 1 Processor -- start 2024-04-25T10:17:56.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.553+0000 7f7b4f200640 1 -- start start 2024-04-25T10:17:56.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.554+0000 7f7b4f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7b50144440 0x7f7b50070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7b500cda30 0x7f7b5015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b5015bf30 con 0x7f7b50071530 2024-04-25T10:17:56.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b500719a0 con 0x7f7b500cda30 2024-04-25T10:17:56.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b5015c860 con 0x7f7b50144440 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4e800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7b500cda30 0x7f7b5015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4d400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f7b50071530 0x7f7b500cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:50314/0 (socket says 172.21.15.165:50314) 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4d400640 1 -- 172.21.15.165:0/1667963684 learned_addr learned my addr 172.21.15.165:0/1667963684 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.555+0000 7f7b4de00640 1 --2- 172.21.15.165:0/1667963684 >> v2:172.21.15.172:3300/0 conn(0x7f7b50144440 0x7f7b50070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.556+0000 7f7b4e800640 1 -- 172.21.15.165:0/1667963684 >> v2:172.21.15.172:3300/0 conn(0x7f7b50144440 msgr2=0x7f7b50070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.556+0000 7f7b4e800640 1 --2- 172.21.15.165:0/1667963684 >> v2:172.21.15.172:3300/0 conn(0x7f7b50144440 0x7f7b50070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.556+0000 7f7b4e800640 1 -- 172.21.15.165:0/1667963684 >> v1:172.21.15.165:6789/0 conn(0x7f7b50071530 legacy=0x7f7b500cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.556+0000 7f7b4e800640 1 -- 172.21.15.165:0/1667963684 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5015c210 con 0x7f7b500cda30 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.556+0000 7f7b4ca00640 1 -- 172.21.15.165:0/1667963684 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3204751769 0 0) 0x7f7b5015bf30 con 0x7f7b50071530 2024-04-25T10:17:56.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.556+0000 7f7b4de00640 1 --2- 172.21.15.165:0/1667963684 >> v2:172.21.15.172:3300/0 conn(0x7f7b50144440 0x7f7b50070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:17:56.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.557+0000 7f7b4e800640 1 --2- 172.21.15.165:0/1667963684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b500cda30 0x7f7b5015ccf0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006f470 tx=0x7f7b4802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9b3252ad9d191b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.557+0000 7f7b4ca00640 1 -- 172.21.15.165:0/1667963684 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48037070 con 0x7f7b500cda30 2024-04-25T10:17:56.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.557+0000 7f7b4ca00640 1 -- 172.21.15.165:0/1667963684 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b48032440 con 0x7f7b500cda30 2024-04-25T10:17:56.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.557+0000 7f7b4ca00640 1 -- 172.21.15.165:0/1667963684 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48032810 con 0x7f7b500cda30 2024-04-25T10:17:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 -- 172.21.15.165:0/1667963684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b500cda30 msgr2=0x7f7b5015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1667963684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b500cda30 0x7f7b5015ccf0 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7f7b5006f470 tx=0x7f7b4802a230 comp rx=0 tx=0).stop 2024-04-25T10:17:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 -- 172.21.15.165:0/1667963684 shutdown_connections 2024-04-25T10:17:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1667963684 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b500cda30 0x7f7b5015ccf0 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1667963684 >> v2:172.21.15.172:3300/0 conn(0x7f7b50144440 0x7f7b50070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 -- 172.21.15.165:0/1667963684 >> 172.21.15.165:0/1667963684 conn(0x7f7b5006abe0 msgr2=0x7f7b5006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 -- 172.21.15.165:0/1667963684 shutdown_connections 2024-04-25T10:17:56.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.558+0000 7f7b4f200640 1 -- 172.21.15.165:0/1667963684 wait complete. 2024-04-25T10:17:56.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.560+0000 7f7b4f200640 1 Processor -- start 2024-04-25T10:17:56.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.560+0000 7f7b4f200640 1 -- start start 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b50071530 0x7f7b5015ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b500cda30 0x7f7b50137b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 0x7f7b501390e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5015bc30 con 0x7f7b50144440 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b500cdea0 con 0x7f7b50071530 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b5015d230 con 0x7f7b500cda30 2024-04-25T10:17:56.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 0x7f7b501390e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 0x7f7b501390e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45928/0 (socket says 172.21.15.165:45928) 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4e800640 1 -- 172.21.15.165:0/1126043263 learned_addr learned my addr 172.21.15.165:0/1126043263 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.561+0000 7f7b4d400640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b500cda30 0x7f7b50137b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4de00640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b50071530 0x7f7b5015ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4e800640 1 -- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b500cda30 msgr2=0x7f7b50137b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4e800640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b500cda30 0x7f7b50137b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4e800640 1 -- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b50071530 msgr2=0x7f7b5015ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4e800640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b50071530 0x7f7b5015ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4e800640 1 -- 172.21.15.165:0/1126043263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b5013bef0 con 0x7f7b50144440 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4e800640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 0x7f7b501390e0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f7b48035040 tx=0x7f7b480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48037050 con 0x7f7b50144440 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b48038040 con 0x7f7b50144440 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b48006e10 con 0x7f7b50144440 2024-04-25T10:17:56.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b5013c1d0 con 0x7f7b50144440 2024-04-25T10:17:56.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.562+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b500c9dc0 con 0x7f7b50144440 2024-04-25T10:17:56.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.564+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b5013c3a0 con 0x7f7b50144440 2024-04-25T10:17:56.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.564+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7b48030980 con 0x7f7b50144440 2024-04-25T10:17:56.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.565+0000 7f7b46000640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b2c06b4d0 0x7f7b2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:56.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.565+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7b480aaf00 con 0x7f7b50144440 2024-04-25T10:17:56.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.566+0000 7f7b4de00640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b2c06b4d0 0x7f7b2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:56.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.566+0000 7f7b4de00640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b2c06b4d0 0x7f7b2c06d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006fd0 tx=0x7f7b38008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:56.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.570+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7b48032e20 con 0x7f7b50144440 2024-04-25T10:17:56.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.762+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7b500c9ff0 con 0x7f7b2c06b4d0 2024-04-25T10:17:56.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.778+0000 7f7b46000640 1 -- 172.21.15.165:0/1126043263 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247518 (secure 0 0 0) 0x7f7b3800f860 con 0x7f7b2c06b4d0 2024-04-25T10:17:56.784 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:56.786 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.783+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b2c06b4d0 msgr2=0x7f7b2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.783+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b2c06b4d0 0x7f7b2c06d990 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006fd0 tx=0x7f7b38008040 comp rx=0 tx=0).stop 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 msgr2=0x7f7b501390e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 0x7f7b501390e0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7f7b48035040 tx=0x7f7b480046c0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 shutdown_connections 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7b2c06b4d0 0x7f7b2c06d990 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7b50144440 0x7f7b501390e0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7b500cda30 0x7f7b50137b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 --2- 172.21.15.165:0/1126043263 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7b50071530 0x7f7b5015ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 >> 172.21.15.165:0/1126043263 conn(0x7f7b5006abe0 msgr2=0x7f7b50140fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 shutdown_connections 2024-04-25T10:17:56.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:56.784+0000 7f7b4f200640 1 -- 172.21.15.165:0/1126043263 wait complete. 2024-04-25T10:17:56.806 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2024-04-25T10:17:56.453207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591040,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":167,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515776,"kb_used_data":16584,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754458944,"statfs":{"total":773094113280,"available":772565958656,"internally_reserved":0,"allocated":16982016,"data_stored":8586016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001308"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.368750+0000","last_change":"2024-04-25T10:17:52.368750+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.368750+0000","last_undegraded":"2024-04-25T10:17:52.368750+0000","last_fullsized":"2024-04-25T10:17:52.368750+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374180+0000","last_change":"2024-04-25T10:17:51.374180+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374180+0000","last_undegraded":"2024-04-25T10:17:51.374180+0000","last_fullsized":"2024-04-25T10:17:51.374180+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.389531+0000","last_change":"2024-04-25T10:17:52.377620+0000","last_active":"2024-04-25T10:17:52.389531+0000","last_peered":"2024-04-25T10:17:52.389531+0000","last_clean":"2024-04-25T10:17:52.389531+0000","last_became_active":"2024-04-25T10:17:52.377333+0000","last_became_peered":"2024-04-25T10:17:52.377333+0000","last_unstale":"2024-04-25T10:17:52.389531+0000","last_undegraded":"2024-04-25T10:17:52.389531+0000","last_fullsized":"2024-04-25T10:17:52.389531+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:18.918486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.367282+0000","last_change":"2024-04-25T10:17:52.367282+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.367282+0000","last_undegraded":"2024-04-25T10:17:52.367282+0000","last_fullsized":"2024-04-25T10:17:52.367282+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.367557+0000","last_change":"2024-04-25T10:17:51.367557+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.367557+0000","last_undegraded":"2024-04-25T10:17:51.367557+0000","last_fullsized":"2024-04-25T10:17:51.367557+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370850+0000","last_change":"2024-04-25T10:17:51.370850+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370850+0000","last_undegraded":"2024-04-25T10:17:51.370850+0000","last_fullsized":"2024-04-25T10:17:51.370850+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371561+0000","last_change":"2024-04-25T10:17:51.371561+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371561+0000","last_undegraded":"2024-04-25T10:17:51.371561+0000","last_fullsized":"2024-04-25T10:17:51.371561+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.369985+0000","last_change":"2024-04-25T10:17:51.369985+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.369985+0000","last_undegraded":"2024-04-25T10:17:51.369985+0000","last_fullsized":"2024-04-25T10:17:51.369985+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361953+0000","last_change":"2024-04-25T10:17:51.361953+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361953+0000","last_undegraded":"2024-04-25T10:17:51.361953+0000","last_fullsized":"2024-04-25T10:17:51.361953+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362115+0000","last_change":"2024-04-25T10:17:51.362115+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362115+0000","last_undegraded":"2024-04-25T10:17:51.362115+0000","last_fullsized":"2024-04-25T10:17:51.362115+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.373269+0000","last_change":"2024-04-25T10:17:51.373269+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.373269+0000","last_undegraded":"2024-04-25T10:17:51.373269+0000","last_fullsized":"2024-04-25T10:17:51.373269+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370995+0000","last_change":"2024-04-25T10:17:51.370995+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370995+0000","last_undegraded":"2024-04-25T10:17:51.370995+0000","last_fullsized":"2024-04-25T10:17:51.370995+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366391+0000","last_change":"2024-04-25T10:17:51.366391+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366391+0000","last_undegraded":"2024-04-25T10:17:51.366391+0000","last_fullsized":"2024-04-25T10:17:51.366391+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370391+0000","last_change":"2024-04-25T10:17:51.370391+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370391+0000","last_undegraded":"2024-04-25T10:17:51.370391+0000","last_fullsized":"2024-04-25T10:17:51.370391+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371819+0000","last_change":"2024-04-25T10:17:51.371819+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371819+0000","last_undegraded":"2024-04-25T10:17:51.371819+0000","last_fullsized":"2024-04-25T10:17:51.371819+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.385023+0000","last_change":"2024-04-25T10:17:52.385023+0000","last_active":"2024-04-25T10:17:52.385023+0000","last_peered":"2024-04-25T10:17:52.385023+0000","last_clean":"2024-04-25T10:17:52.385023+0000","last_became_active":"2024-04-25T10:17:52.384157+0000","last_became_peered":"2024-04-25T10:17:52.384157+0000","last_unstale":"2024-04-25T10:17:52.385023+0000","last_undegraded":"2024-04-25T10:17:52.385023+0000","last_fullsized":"2024-04-25T10:17:52.385023+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.242001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361325+0000","last_change":"2024-04-25T10:17:51.361325+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361325+0000","last_undegraded":"2024-04-25T10:17:51.361325+0000","last_fullsized":"2024-04-25T10:17:51.361325+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.362292+0000","last_change":"2024-04-25T10:17:52.362292+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.362292+0000","last_undegraded":"2024-04-25T10:17:52.362292+0000","last_fullsized":"2024-04-25T10:17:52.362292+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370262+0000","last_change":"2024-04-25T10:17:51.370262+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370262+0000","last_undegraded":"2024-04-25T10:17:51.370262+0000","last_fullsized":"2024-04-25T10:17:51.370262+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362679+0000","last_change":"2024-04-25T10:17:51.362679+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362679+0000","last_undegraded":"2024-04-25T10:17:51.362679+0000","last_fullsized":"2024-04-25T10:17:51.362679+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362767+0000","last_change":"2024-04-25T10:17:51.362767+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362767+0000","last_undegraded":"2024-04-25T10:17:51.362767+0000","last_fullsized":"2024-04-25T10:17:51.362767+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.368522+0000","last_change":"2024-04-25T10:17:51.368522+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.368522+0000","last_undegraded":"2024-04-25T10:17:51.368522+0000","last_fullsized":"2024-04-25T10:17:51.368522+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370747+0000","last_change":"2024-04-25T10:17:51.370747+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370747+0000","last_undegraded":"2024-04-25T10:17:51.370747+0000","last_fullsized":"2024-04-25T10:17:51.370747+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.385243+0000","last_change":"2024-04-25T10:17:52.385243+0000","last_active":"2024-04-25T10:17:52.385243+0000","last_peered":"2024-04-25T10:17:52.385243+0000","last_clean":"2024-04-25T10:17:52.385243+0000","last_became_active":"2024-04-25T10:17:52.384636+0000","last_became_peered":"2024-04-25T10:17:52.384636+0000","last_unstale":"2024-04-25T10:17:52.385243+0000","last_undegraded":"2024-04-25T10:17:52.385243+0000","last_fullsized":"2024-04-25T10:17:52.385243+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.007326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362563+0000","last_change":"2024-04-25T10:17:51.362563+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362563+0000","last_undegraded":"2024-04-25T10:17:51.362563+0000","last_fullsized":"2024-04-25T10:17:51.362563+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361021+0000","last_change":"2024-04-25T10:17:51.361021+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361021+0000","last_undegraded":"2024-04-25T10:17:51.361021+0000","last_fullsized":"2024-04-25T10:17:51.361021+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375238+0000","last_change":"2024-04-25T10:17:51.375238+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375238+0000","last_undegraded":"2024-04-25T10:17:51.375238+0000","last_fullsized":"2024-04-25T10:17:51.375238+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361848+0000","last_change":"2024-04-25T10:17:51.361848+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361848+0000","last_undegraded":"2024-04-25T10:17:51.361848+0000","last_fullsized":"2024-04-25T10:17:51.361848+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374653+0000","last_change":"2024-04-25T10:17:51.374653+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374653+0000","last_undegraded":"2024-04-25T10:17:51.374653+0000","last_fullsized":"2024-04-25T10:17:51.374653+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.363325+0000","last_change":"2024-04-25T10:17:52.363325+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.363325+0000","last_undegraded":"2024-04-25T10:17:52.363325+0000","last_fullsized":"2024-04-25T10:17:52.363325+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371347+0000","last_change":"2024-04-25T10:17:51.371347+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371347+0000","last_undegraded":"2024-04-25T10:17:51.371347+0000","last_fullsized":"2024-04-25T10:17:51.371347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.361408+0000","last_change":"2024-04-25T10:17:52.361408+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.361408+0000","last_undegraded":"2024-04-25T10:17:52.361408+0000","last_fullsized":"2024-04-25T10:17:52.361408+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374913+0000","last_change":"2024-04-25T10:17:51.374913+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374913+0000","last_undegraded":"2024-04-25T10:17:51.374913+0000","last_fullsized":"2024-04-25T10:17:51.374913+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.381190+0000","last_change":"2024-04-25T10:17:51.378796+0000","last_active":"2024-04-25T10:17:52.381190+0000","last_peered":"2024-04-25T10:17:52.381190+0000","last_clean":"2024-04-25T10:17:52.381190+0000","last_became_active":"2024-04-25T10:17:51.378504+0000","last_became_peered":"2024-04-25T10:17:51.378504+0000","last_unstale":"2024-04-25T10:17:52.381190+0000","last_undegraded":"2024-04-25T10:17:52.381190+0000","last_fullsized":"2024-04-25T10:17:52.381190+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:36.761276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360291+0000","last_change":"2024-04-25T10:17:51.360291+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360291+0000","last_undegraded":"2024-04-25T10:17:51.360291+0000","last_fullsized":"2024-04-25T10:17:51.360291+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385683+0000","last_change":"2024-04-25T10:17:51.383528+0000","last_active":"2024-04-25T10:17:51.385683+0000","last_peered":"2024-04-25T10:17:51.385683+0000","last_clean":"2024-04-25T10:17:51.385683+0000","last_became_active":"2024-04-25T10:17:51.382449+0000","last_became_peered":"2024-04-25T10:17:51.382449+0000","last_unstale":"2024-04-25T10:17:51.385683+0000","last_undegraded":"2024-04-25T10:17:51.385683+0000","last_fullsized":"2024-04-25T10:17:51.385683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:11.992831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359546+0000","last_change":"2024-04-25T10:17:51.359546+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359546+0000","last_undegraded":"2024-04-25T10:17:51.359546+0000","last_fullsized":"2024-04-25T10:17:51.359546+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.381144+0000","last_change":"2024-04-25T10:17:51.378958+0000","last_active":"2024-04-25T10:17:51.381144+0000","last_peered":"2024-04-25T10:17:51.381144+0000","last_clean":"2024-04-25T10:17:51.381144+0000","last_became_active":"2024-04-25T10:17:51.378655+0000","last_became_peered":"2024-04-25T10:17:51.378655+0000","last_unstale":"2024-04-25T10:17:51.381144+0000","last_undegraded":"2024-04-25T10:17:51.381144+0000","last_fullsized":"2024-04-25T10:17:51.381144+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:43.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370565+0000","last_change":"2024-04-25T10:17:51.370565+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370565+0000","last_undegraded":"2024-04-25T10:17:51.370565+0000","last_fullsized":"2024-04-25T10:17:51.370565+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.378523+0000","last_change":"2024-04-25T10:17:51.377735+0000","last_active":"2024-04-25T10:17:51.378523+0000","last_peered":"2024-04-25T10:17:51.378523+0000","last_clean":"2024-04-25T10:17:51.378523+0000","last_became_active":"2024-04-25T10:17:51.376589+0000","last_became_peered":"2024-04-25T10:17:51.376589+0000","last_unstale":"2024-04-25T10:17:51.378523+0000","last_undegraded":"2024-04-25T10:17:51.378523+0000","last_fullsized":"2024-04-25T10:17:51.378523+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:10.550511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.358852+0000","last_change":"2024-04-25T10:17:51.358852+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.358852+0000","last_undegraded":"2024-04-25T10:17:51.358852+0000","last_fullsized":"2024-04-25T10:17:51.358852+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380445+0000","last_change":"2024-04-25T10:17:51.379258+0000","last_active":"2024-04-25T10:17:51.380445+0000","last_peered":"2024-04-25T10:17:51.380445+0000","last_clean":"2024-04-25T10:17:51.380445+0000","last_became_active":"2024-04-25T10:17:51.378916+0000","last_became_peered":"2024-04-25T10:17:51.378916+0000","last_unstale":"2024-04-25T10:17:51.380445+0000","last_undegraded":"2024-04-25T10:17:51.380445+0000","last_fullsized":"2024-04-25T10:17:51.380445+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:28.372775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360084+0000","last_change":"2024-04-25T10:17:51.360084+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360084+0000","last_undegraded":"2024-04-25T10:17:51.360084+0000","last_fullsized":"2024-04-25T10:17:51.360084+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384738+0000","last_change":"2024-04-25T10:17:51.376585+0000","last_active":"2024-04-25T10:17:51.384738+0000","last_peered":"2024-04-25T10:17:51.384738+0000","last_clean":"2024-04-25T10:17:51.384738+0000","last_became_active":"2024-04-25T10:17:51.376266+0000","last_became_peered":"2024-04-25T10:17:51.376266+0000","last_unstale":"2024-04-25T10:17:51.384738+0000","last_undegraded":"2024-04-25T10:17:51.384738+0000","last_fullsized":"2024-04-25T10:17:51.384738+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:47.138590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360539+0000","last_change":"2024-04-25T10:17:51.360539+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360539+0000","last_undegraded":"2024-04-25T10:17:51.360539+0000","last_fullsized":"2024-04-25T10:17:51.360539+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384624+0000","last_change":"2024-04-25T10:17:51.383223+0000","last_active":"2024-04-25T10:17:51.384624+0000","last_peered":"2024-04-25T10:17:51.384624+0000","last_clean":"2024-04-25T10:17:51.384624+0000","last_became_active":"2024-04-25T10:17:51.382880+0000","last_became_peered":"2024-04-25T10:17:51.382880+0000","last_unstale":"2024-04-25T10:17:51.384624+0000","last_undegraded":"2024-04-25T10:17:51.384624+0000","last_fullsized":"2024-04-25T10:17:51.384624+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:38.440117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.368384+0000","last_change":"2024-04-25T10:17:52.368384+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.368384+0000","last_undegraded":"2024-04-25T10:17:52.368384+0000","last_fullsized":"2024-04-25T10:17:52.368384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384778+0000","last_change":"2024-04-25T10:17:51.382057+0000","last_active":"2024-04-25T10:17:51.384778+0000","last_peered":"2024-04-25T10:17:51.384778+0000","last_clean":"2024-04-25T10:17:51.384778+0000","last_became_active":"2024-04-25T10:17:51.381426+0000","last_became_peered":"2024-04-25T10:17:51.381426+0000","last_unstale":"2024-04-25T10:17:51.384778+0000","last_undegraded":"2024-04-25T10:17:51.384778+0000","last_fullsized":"2024-04-25T10:17:51.384778+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:19.839264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375547+0000","last_change":"2024-04-25T10:17:51.375547+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375547+0000","last_undegraded":"2024-04-25T10:17:51.375547+0000","last_fullsized":"2024-04-25T10:17:51.375547+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.367621+0000","last_change":"2024-04-25T10:17:51.365010+0000","last_active":"2024-04-25T10:17:51.367621+0000","last_peered":"2024-04-25T10:17:51.367621+0000","last_clean":"2024-04-25T10:17:51.367621+0000","last_became_active":"2024-04-25T10:17:51.364712+0000","last_became_peered":"2024-04-25T10:17:51.364712+0000","last_unstale":"2024-04-25T10:17:51.367621+0000","last_undegraded":"2024-04-25T10:17:51.367621+0000","last_fullsized":"2024-04-25T10:17:51.367621+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.697276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360010+0000","last_change":"2024-04-25T10:17:51.360010+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360010+0000","last_undegraded":"2024-04-25T10:17:51.360010+0000","last_fullsized":"2024-04-25T10:17:51.360010+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.376630+0000","last_change":"2024-04-25T10:17:51.382259+0000","last_active":"2024-04-25T10:17:52.376630+0000","last_peered":"2024-04-25T10:17:52.376630+0000","last_clean":"2024-04-25T10:17:52.376630+0000","last_became_active":"2024-04-25T10:17:51.381960+0000","last_became_peered":"2024-04-25T10:17:51.381960+0000","last_unstale":"2024-04-25T10:17:52.376630+0000","last_undegraded":"2024-04-25T10:17:52.376630+0000","last_fullsized":"2024-04-25T10:17:52.376630+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.670982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360814+0000","last_change":"2024-04-25T10:17:51.360814+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360814+0000","last_undegraded":"2024-04-25T10:17:51.360814+0000","last_fullsized":"2024-04-25T10:17:51.360814+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380759+0000","last_change":"2024-04-25T10:17:51.378190+0000","last_active":"2024-04-25T10:17:51.380759+0000","last_peered":"2024-04-25T10:17:51.380759+0000","last_clean":"2024-04-25T10:17:51.380759+0000","last_became_active":"2024-04-25T10:17:51.377878+0000","last_became_peered":"2024-04-25T10:17:51.377878+0000","last_unstale":"2024-04-25T10:17:51.380759+0000","last_undegraded":"2024-04-25T10:17:51.380759+0000","last_fullsized":"2024-04-25T10:17:51.380759+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:22.718962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.362642+0000","last_change":"2024-04-25T10:17:52.362642+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.362642+0000","last_undegraded":"2024-04-25T10:17:52.362642+0000","last_fullsized":"2024-04-25T10:17:52.362642+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385449+0000","last_change":"2024-04-25T10:17:51.377387+0000","last_active":"2024-04-25T10:17:51.385449+0000","last_peered":"2024-04-25T10:17:51.385449+0000","last_clean":"2024-04-25T10:17:51.385449+0000","last_became_active":"2024-04-25T10:17:51.376999+0000","last_became_peered":"2024-04-25T10:17:51.376999+0000","last_unstale":"2024-04-25T10:17:51.385449+0000","last_undegraded":"2024-04-25T10:17:51.385449+0000","last_fullsized":"2024-04-25T10:17:51.385449+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.697276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362120+0000","last_change":"2024-04-25T10:17:51.362120+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362120+0000","last_undegraded":"2024-04-25T10:17:51.362120+0000","last_fullsized":"2024-04-25T10:17:51.362120+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.376334+0000","last_change":"2024-04-25T10:17:51.374071+0000","last_active":"2024-04-25T10:17:51.376334+0000","last_peered":"2024-04-25T10:17:51.376334+0000","last_clean":"2024-04-25T10:17:51.376334+0000","last_became_active":"2024-04-25T10:17:51.371382+0000","last_became_peered":"2024-04-25T10:17:51.371382+0000","last_unstale":"2024-04-25T10:17:51.376334+0000","last_undegraded":"2024-04-25T10:17:51.376334+0000","last_fullsized":"2024-04-25T10:17:51.376334+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:19.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.369188+0000","last_change":"2024-04-25T10:17:51.369188+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.369188+0000","last_undegraded":"2024-04-25T10:17:51.369188+0000","last_fullsized":"2024-04-25T10:17:51.369188+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385844+0000","last_change":"2024-04-25T10:17:51.383883+0000","last_active":"2024-04-25T10:17:51.385844+0000","last_peered":"2024-04-25T10:17:51.385844+0000","last_clean":"2024-04-25T10:17:51.385844+0000","last_became_active":"2024-04-25T10:17:51.383352+0000","last_became_peered":"2024-04-25T10:17:51.383352+0000","last_unstale":"2024-04-25T10:17:51.385844+0000","last_undegraded":"2024-04-25T10:17:51.385844+0000","last_fullsized":"2024-04-25T10:17:51.385844+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:51.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.363481+0000","last_change":"2024-04-25T10:17:51.363481+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.363481+0000","last_undegraded":"2024-04-25T10:17:51.363481+0000","last_fullsized":"2024-04-25T10:17:51.363481+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384815+0000","last_change":"2024-04-25T10:17:51.382235+0000","last_active":"2024-04-25T10:17:51.384815+0000","last_peered":"2024-04-25T10:17:51.384815+0000","last_clean":"2024-04-25T10:17:51.384815+0000","last_became_active":"2024-04-25T10:17:51.381803+0000","last_became_peered":"2024-04-25T10:17:51.381803+0000","last_unstale":"2024-04-25T10:17:51.384815+0000","last_undegraded":"2024-04-25T10:17:51.384815+0000","last_fullsized":"2024-04-25T10:17:51.384815+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:43.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.365658+0000","last_change":"2024-04-25T10:17:51.365658+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.365658+0000","last_undegraded":"2024-04-25T10:17:51.365658+0000","last_fullsized":"2024-04-25T10:17:51.365658+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380988+0000","last_change":"2024-04-25T10:17:51.378215+0000","last_active":"2024-04-25T10:17:51.380988+0000","last_peered":"2024-04-25T10:17:51.380988+0000","last_clean":"2024-04-25T10:17:51.380988+0000","last_became_active":"2024-04-25T10:17:51.377905+0000","last_became_peered":"2024-04-25T10:17:51.377905+0000","last_unstale":"2024-04-25T10:17:51.380988+0000","last_undegraded":"2024-04-25T10:17:51.380988+0000","last_fullsized":"2024-04-25T10:17:51.380988+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:42.862391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.361848+0000","last_change":"2024-04-25T10:17:52.361848+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.361848+0000","last_undegraded":"2024-04-25T10:17:52.361848+0000","last_fullsized":"2024-04-25T10:17:52.361848+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.378602+0000","last_change":"2024-04-25T10:17:51.367493+0000","last_active":"2024-04-25T10:17:51.378602+0000","last_peered":"2024-04-25T10:17:51.378602+0000","last_clean":"2024-04-25T10:17:51.378602+0000","last_became_active":"2024-04-25T10:17:51.367097+0000","last_became_peered":"2024-04-25T10:17:51.367097+0000","last_unstale":"2024-04-25T10:17:51.378602+0000","last_undegraded":"2024-04-25T10:17:51.378602+0000","last_fullsized":"2024-04-25T10:17:51.378602+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:18.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.373721+0000","last_change":"2024-04-25T10:17:51.373721+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.373721+0000","last_undegraded":"2024-04-25T10:17:51.373721+0000","last_fullsized":"2024-04-25T10:17:51.373721+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385685+0000","last_change":"2024-04-25T10:17:51.383023+0000","last_active":"2024-04-25T10:17:51.385685+0000","last_peered":"2024-04-25T10:17:51.385685+0000","last_clean":"2024-04-25T10:17:51.385685+0000","last_became_active":"2024-04-25T10:17:51.382697+0000","last_became_peered":"2024-04-25T10:17:51.382697+0000","last_unstale":"2024-04-25T10:17:51.385685+0000","last_undegraded":"2024-04-25T10:17:51.385685+0000","last_fullsized":"2024-04-25T10:17:51.385685+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:44.662844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366804+0000","last_change":"2024-04-25T10:17:51.366804+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366804+0000","last_undegraded":"2024-04-25T10:17:51.366804+0000","last_fullsized":"2024-04-25T10:17:51.366804+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384889+0000","last_change":"2024-04-25T10:17:51.381541+0000","last_active":"2024-04-25T10:17:51.384889+0000","last_peered":"2024-04-25T10:17:51.384889+0000","last_clean":"2024-04-25T10:17:51.384889+0000","last_became_active":"2024-04-25T10:17:51.380331+0000","last_became_peered":"2024-04-25T10:17:51.380331+0000","last_unstale":"2024-04-25T10:17:51.384889+0000","last_undegraded":"2024-04-25T10:17:51.384889+0000","last_fullsized":"2024-04-25T10:17:51.384889+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:21.186217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359326+0000","last_change":"2024-04-25T10:17:51.359326+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359326+0000","last_undegraded":"2024-04-25T10:17:51.359326+0000","last_fullsized":"2024-04-25T10:17:51.359326+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.386734+0000","last_change":"2024-04-25T10:17:51.386166+0000","last_active":"2024-04-25T10:17:51.386734+0000","last_peered":"2024-04-25T10:17:51.386734+0000","last_clean":"2024-04-25T10:17:51.386734+0000","last_became_active":"2024-04-25T10:17:51.385874+0000","last_became_peered":"2024-04-25T10:17:51.385874+0000","last_unstale":"2024-04-25T10:17:51.386734+0000","last_undegraded":"2024-04-25T10:17:51.386734+0000","last_fullsized":"2024-04-25T10:17:51.386734+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:22.718962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361684+0000","last_change":"2024-04-25T10:17:51.361684+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361684+0000","last_undegraded":"2024-04-25T10:17:51.361684+0000","last_fullsized":"2024-04-25T10:17:51.361684+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.382580+0000","last_change":"2024-04-25T10:17:51.380152+0000","last_active":"2024-04-25T10:17:51.382580+0000","last_peered":"2024-04-25T10:17:51.382580+0000","last_clean":"2024-04-25T10:17:51.382580+0000","last_became_active":"2024-04-25T10:17:51.379860+0000","last_became_peered":"2024-04-25T10:17:51.379860+0000","last_unstale":"2024-04-25T10:17:51.382580+0000","last_undegraded":"2024-04-25T10:17:51.382580+0000","last_fullsized":"2024-04-25T10:17:51.382580+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:22.437023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.363696+0000","last_change":"2024-04-25T10:17:51.363696+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.363696+0000","last_undegraded":"2024-04-25T10:17:51.363696+0000","last_fullsized":"2024-04-25T10:17:51.363696+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.371607+0000","last_change":"2024-04-25T10:17:51.369127+0000","last_active":"2024-04-25T10:17:51.371607+0000","last_peered":"2024-04-25T10:17:51.371607+0000","last_clean":"2024-04-25T10:17:51.371607+0000","last_became_active":"2024-04-25T10:17:51.368836+0000","last_became_peered":"2024-04-25T10:17:51.368836+0000","last_unstale":"2024-04-25T10:17:51.371607+0000","last_undegraded":"2024-04-25T10:17:51.371607+0000","last_fullsized":"2024-04-25T10:17:51.371607+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:44.824247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375100+0000","last_change":"2024-04-25T10:17:51.375100+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375100+0000","last_undegraded":"2024-04-25T10:17:51.375100+0000","last_fullsized":"2024-04-25T10:17:51.375100+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.383804+0000","last_change":"2024-04-25T10:17:51.382409+0000","last_active":"2024-04-25T10:17:51.383804+0000","last_peered":"2024-04-25T10:17:51.383804+0000","last_clean":"2024-04-25T10:17:51.383804+0000","last_became_active":"2024-04-25T10:17:51.382048+0000","last_became_peered":"2024-04-25T10:17:51.382048+0000","last_unstale":"2024-04-25T10:17:51.383804+0000","last_undegraded":"2024-04-25T10:17:51.383804+0000","last_fullsized":"2024-04-25T10:17:51.383804+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:14.932827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.367562+0000","last_change":"2024-04-25T10:17:51.367562+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.367562+0000","last_undegraded":"2024-04-25T10:17:51.367562+0000","last_fullsized":"2024-04-25T10:17:51.367562+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385694+0000","last_change":"2024-04-25T10:17:51.382866+0000","last_active":"2024-04-25T10:17:51.385694+0000","last_peered":"2024-04-25T10:17:51.385694+0000","last_clean":"2024-04-25T10:17:51.385694+0000","last_became_active":"2024-04-25T10:17:51.382164+0000","last_became_peered":"2024-04-25T10:17:51.382164+0000","last_unstale":"2024-04-25T10:17:51.385694+0000","last_undegraded":"2024-04-25T10:17:51.385694+0000","last_fullsized":"2024-04-25T10:17:51.385694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366912+0000","last_change":"2024-04-25T10:17:51.366912+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366912+0000","last_undegraded":"2024-04-25T10:17:51.366912+0000","last_fullsized":"2024-04-25T10:17:51.366912+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.372980+0000","last_change":"2024-04-25T10:17:51.370374+0000","last_active":"2024-04-25T10:17:51.372980+0000","last_peered":"2024-04-25T10:17:51.372980+0000","last_clean":"2024-04-25T10:17:51.372980+0000","last_became_active":"2024-04-25T10:17:51.369266+0000","last_became_peered":"2024-04-25T10:17:51.369266+0000","last_unstale":"2024-04-25T10:17:51.372980+0000","last_undegraded":"2024-04-25T10:17:51.372980+0000","last_fullsized":"2024-04-25T10:17:51.372980+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:44.662844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359492+0000","last_change":"2024-04-25T10:17:51.359492+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359492+0000","last_undegraded":"2024-04-25T10:17:51.359492+0000","last_fullsized":"2024-04-25T10:17:51.359492+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380712+0000","last_change":"2024-04-25T10:17:51.379048+0000","last_active":"2024-04-25T10:17:51.380712+0000","last_peered":"2024-04-25T10:17:51.380712+0000","last_clean":"2024-04-25T10:17:51.380712+0000","last_became_active":"2024-04-25T10:17:51.378700+0000","last_became_peered":"2024-04-25T10:17:51.378700+0000","last_unstale":"2024-04-25T10:17:51.380712+0000","last_undegraded":"2024-04-25T10:17:51.380712+0000","last_fullsized":"2024-04-25T10:17:51.380712+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:48.614198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.364985+0000","last_change":"2024-04-25T10:17:51.364985+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.364985+0000","last_undegraded":"2024-04-25T10:17:51.364985+0000","last_fullsized":"2024-04-25T10:17:51.364985+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384589+0000","last_change":"2024-04-25T10:17:51.383501+0000","last_active":"2024-04-25T10:17:51.384589+0000","last_peered":"2024-04-25T10:17:51.384589+0000","last_clean":"2024-04-25T10:17:51.384589+0000","last_became_active":"2024-04-25T10:17:51.381248+0000","last_became_peered":"2024-04-25T10:17:51.381248+0000","last_unstale":"2024-04-25T10:17:51.384589+0000","last_undegraded":"2024-04-25T10:17:51.384589+0000","last_fullsized":"2024-04-25T10:17:51.384589+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:02.741516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.388545+0000","last_change":"2024-04-25T10:17:52.382153+0000","last_active":"2024-04-25T10:17:52.388545+0000","last_peered":"2024-04-25T10:17:52.388545+0000","last_clean":"2024-04-25T10:17:52.388545+0000","last_became_active":"2024-04-25T10:17:52.381688+0000","last_became_peered":"2024-04-25T10:17:52.381688+0000","last_unstale":"2024-04-25T10:17:52.388545+0000","last_undegraded":"2024-04-25T10:17:52.388545+0000","last_fullsized":"2024-04-25T10:17:52.388545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:53.783720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.376315+0000","last_change":"2024-04-25T10:17:51.384528+0000","last_active":"2024-04-25T10:17:52.376315+0000","last_peered":"2024-04-25T10:17:52.376315+0000","last_clean":"2024-04-25T10:17:52.376315+0000","last_became_active":"2024-04-25T10:17:51.384176+0000","last_became_peered":"2024-04-25T10:17:51.384176+0000","last_unstale":"2024-04-25T10:17:52.376315+0000","last_undegraded":"2024-04-25T10:17:52.376315+0000","last_fullsized":"2024-04-25T10:17:52.376315+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:48.896090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371188+0000","last_change":"2024-04-25T10:17:51.371188+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371188+0000","last_undegraded":"2024-04-25T10:17:51.371188+0000","last_fullsized":"2024-04-25T10:17:51.371188+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.368066+0000","last_change":"2024-04-25T10:17:51.366496+0000","last_active":"2024-04-25T10:17:51.368066+0000","last_peered":"2024-04-25T10:17:51.368066+0000","last_clean":"2024-04-25T10:17:51.368066+0000","last_became_active":"2024-04-25T10:17:51.364699+0000","last_became_peered":"2024-04-25T10:17:51.364699+0000","last_unstale":"2024-04-25T10:17:51.368066+0000","last_undegraded":"2024-04-25T10:17:51.368066+0000","last_fullsized":"2024-04-25T10:17:51.368066+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361029+0000","last_change":"2024-04-25T10:17:51.361029+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361029+0000","last_undegraded":"2024-04-25T10:17:51.361029+0000","last_fullsized":"2024-04-25T10:17:51.361029+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.381883+0000","last_change":"2024-04-25T10:17:51.379727+0000","last_active":"2024-04-25T10:17:51.381883+0000","last_peered":"2024-04-25T10:17:51.381883+0000","last_clean":"2024-04-25T10:17:51.381883+0000","last_became_active":"2024-04-25T10:17:51.379202+0000","last_became_peered":"2024-04-25T10:17:51.379202+0000","last_unstale":"2024-04-25T10:17:51.381883+0000","last_undegraded":"2024-04-25T10:17:51.381883+0000","last_fullsized":"2024-04-25T10:17:51.381883+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:56.752842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359514+0000","last_change":"2024-04-25T10:17:51.359514+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359514+0000","last_undegraded":"2024-04-25T10:17:51.359514+0000","last_fullsized":"2024-04-25T10:17:51.359514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385949+0000","last_change":"2024-04-25T10:17:51.384876+0000","last_active":"2024-04-25T10:17:51.385949+0000","last_peered":"2024-04-25T10:17:51.385949+0000","last_clean":"2024-04-25T10:17:51.385949+0000","last_became_active":"2024-04-25T10:17:51.384131+0000","last_became_peered":"2024-04-25T10:17:51.384131+0000","last_unstale":"2024-04-25T10:17:51.385949+0000","last_undegraded":"2024-04-25T10:17:51.385949+0000","last_fullsized":"2024-04-25T10:17:51.385949+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:19.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.368225+0000","last_change":"2024-04-25T10:17:51.368225+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.368225+0000","last_undegraded":"2024-04-25T10:17:51.368225+0000","last_fullsized":"2024-04-25T10:17:51.368225+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.373581+0000","last_change":"2024-04-25T10:17:51.371004+0000","last_active":"2024-04-25T10:17:51.373581+0000","last_peered":"2024-04-25T10:17:51.373581+0000","last_clean":"2024-04-25T10:17:51.373581+0000","last_became_active":"2024-04-25T10:17:51.370705+0000","last_became_peered":"2024-04-25T10:17:51.370705+0000","last_unstale":"2024-04-25T10:17:51.373581+0000","last_undegraded":"2024-04-25T10:17:51.373581+0000","last_fullsized":"2024-04-25T10:17:51.373581+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:11.992831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":431,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.376627+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:17:51.376627+0000","last_peered":"2024-04-25T10:17:51.376627+0000","last_clean":"2024-04-25T10:17:51.376627+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:17:51.376627+0000","last_undegraded":"2024-04-25T10:17:51.376627+0000","last_fullsized":"2024-04-25T10:17:51.376627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2568192,"data_stored":1516028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87192,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284648,"statfs":{"total":96636764160,"available":96547479552,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47580,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324260,"statfs":{"total":96636764160,"available":96588042240,"internally_reserved":0,"allocated":2568192,"data_stored":1516028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:56.811 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:57.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.046+0000 7fa109e00640 1 Processor -- start 2024-04-25T10:17:57.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.046+0000 7fa109e00640 1 -- start start 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa109e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa104144750 0x7fa10406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa109e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 0x7fa10406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa109e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa104144b30 con 0x7fa10406eb10 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa109e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040712d0 con 0x7fa104144750 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa109e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040715b0 con 0x7fa1040cf030 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa102a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa104144750 0x7fa10406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa102a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa104144750 0x7fa10406e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39696/0 (socket says 172.21.15.165:39696) 2024-04-25T10:17:57.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa102a00640 1 -- 172.21.15.165:0/1954056281 learned_addr learned my addr 172.21.15.165:0/1954056281 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa102a00640 1 -- 172.21.15.165:0/1954056281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 msgr2=0x7fa10406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.048+0000 7fa102a00640 1 --2- 172.21.15.165:0/1954056281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 0x7fa10406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.049+0000 7fa102a00640 1 -- 172.21.15.165:0/1954056281 >> v1:172.21.15.165:6789/0 conn(0x7fa10406eb10 legacy=0x7fa1040ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.049+0000 7fa102a00640 1 -- 172.21.15.165:0/1954056281 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa104071890 con 0x7fa104144750 2024-04-25T10:17:57.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.050+0000 7fa102a00640 1 --2- 172.21.15.165:0/1954056281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10406e5d0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009870 tx=0x7fa0f402a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4beaf351ac6e5e87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.050+0000 7fa102000640 1 -- 172.21.15.165:0/1954056281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4037070 con 0x7fa104144750 2024-04-25T10:17:57.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.050+0000 7fa102000640 1 -- 172.21.15.165:0/1954056281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f4032440 con 0x7fa104144750 2024-04-25T10:17:57.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.050+0000 7fa102000640 1 -- 172.21.15.165:0/1954056281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4032810 con 0x7fa104144750 2024-04-25T10:17:57.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 -- 172.21.15.165:0/1954056281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 msgr2=0x7fa10406e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 --2- 172.21.15.165:0/1954056281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10406e5d0 secure :-1 s=READY pgs=2257 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4009870 tx=0x7fa0f402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 -- 172.21.15.165:0/1954056281 shutdown_connections 2024-04-25T10:17:57.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 --2- 172.21.15.165:0/1954056281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 0x7fa10406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 --2- 172.21.15.165:0/1954056281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10406e5d0 unknown :-1 s=CLOSED pgs=2257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 -- 172.21.15.165:0/1954056281 >> 172.21.15.165:0/1954056281 conn(0x7fa10406abe0 msgr2=0x7fa10406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 -- 172.21.15.165:0/1954056281 shutdown_connections 2024-04-25T10:17:57.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.051+0000 7fa109e00640 1 -- 172.21.15.165:0/1954056281 wait complete. 2024-04-25T10:17:57.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.053+0000 7fa109e00640 1 Processor -- start 2024-04-25T10:17:57.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.053+0000 7fa109e00640 1 -- start start 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa109e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10406eb10 0x7fa10413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa109e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 0x7fa104142510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa109e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10415d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa109e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041455d0 con 0x7fa104144750 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa109e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104145450 con 0x7fa10406eb10 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa109e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10415c000 con 0x7fa1040cf030 2024-04-25T10:17:57.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10415d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10415d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39698/0 (socket says 172.21.15.165:39698) 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 -- 172.21.15.165:0/3275808971 learned_addr learned my addr 172.21.15.165:0/3275808971 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa102a00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10406eb10 0x7fa10413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 -- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 msgr2=0x7fa104142510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 0x7fa104142510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 -- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10406eb10 msgr2=0x7fa10413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10406eb10 0x7fa10413f500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.054+0000 7fa103400640 1 -- 172.21.15.165:0/3275808971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040a84a0 con 0x7fa104144750 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.055+0000 7fa103400640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10415d3d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b970 tx=0x7fa0fc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.055+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc019070 con 0x7fa104144750 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.055+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0fc00cb60 con 0x7fa104144750 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.055+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0fc014440 con 0x7fa104144750 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.055+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1040a8780 con 0x7fa104144750 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.055+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1040a8d40 con 0x7fa104144750 2024-04-25T10:17:57.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.056+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa104073990 con 0x7fa104144750 2024-04-25T10:17:57.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.057+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0fc01c070 con 0x7fa104144750 2024-04-25T10:17:57.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.058+0000 7fa0fb400640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.058+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa0fc08a0d0 con 0x7fa104144750 2024-04-25T10:17:57.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.058+0000 7fa102a00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.059+0000 7fa102a00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4034000 tx=0x7fa0f402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.062+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0fc058dc0 con 0x7fa104144750 2024-04-25T10:17:57.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.256+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa10406b160 con 0x7fa0e006b4d0 2024-04-25T10:17:57.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.274+0000 7fa0fb400640 1 -- 172.21.15.165:0/3275808971 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247518 (secure 0 0 0) 0x7fa10406b160 con 0x7fa0e006b4d0 2024-04-25T10:17:57.281 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:57.282 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:17:57.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 msgr2=0x7fa0e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4034000 tx=0x7fa0f402e000 comp rx=0 tx=0).stop 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 msgr2=0x7fa10415d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10415d3d0 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fa0fc00b970 tx=0x7fa0fc00be30 comp rx=0 tx=0).stop 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 shutdown_connections 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa0e006b4d0 0x7fa0e006d990 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa104144750 0x7fa10415d3d0 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa1040cf030 0x7fa104142510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 --2- 172.21.15.165:0/3275808971 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa10406eb10 0x7fa10413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 >> 172.21.15.165:0/3275808971 conn(0x7fa10406abe0 msgr2=0x7fa1040b1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 shutdown_connections 2024-04-25T10:17:57.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.280+0000 7fa109e00640 1 -- 172.21.15.165:0/3275808971 wait complete. 2024-04-25T10:17:57.303 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2024-04-25T10:17:56.453207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591040,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":167,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515776,"kb_used_data":16584,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754458944,"statfs":{"total":773094113280,"available":772565958656,"internally_reserved":0,"allocated":16982016,"data_stored":8586016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001308"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.368750+0000","last_change":"2024-04-25T10:17:52.368750+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.368750+0000","last_undegraded":"2024-04-25T10:17:52.368750+0000","last_fullsized":"2024-04-25T10:17:52.368750+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374180+0000","last_change":"2024-04-25T10:17:51.374180+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374180+0000","last_undegraded":"2024-04-25T10:17:51.374180+0000","last_fullsized":"2024-04-25T10:17:51.374180+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.389531+0000","last_change":"2024-04-25T10:17:52.377620+0000","last_active":"2024-04-25T10:17:52.389531+0000","last_peered":"2024-04-25T10:17:52.389531+0000","last_clean":"2024-04-25T10:17:52.389531+0000","last_became_active":"2024-04-25T10:17:52.377333+0000","last_became_peered":"2024-04-25T10:17:52.377333+0000","last_unstale":"2024-04-25T10:17:52.389531+0000","last_undegraded":"2024-04-25T10:17:52.389531+0000","last_fullsized":"2024-04-25T10:17:52.389531+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:18.918486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.367282+0000","last_change":"2024-04-25T10:17:52.367282+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.367282+0000","last_undegraded":"2024-04-25T10:17:52.367282+0000","last_fullsized":"2024-04-25T10:17:52.367282+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.367557+0000","last_change":"2024-04-25T10:17:51.367557+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.367557+0000","last_undegraded":"2024-04-25T10:17:51.367557+0000","last_fullsized":"2024-04-25T10:17:51.367557+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370850+0000","last_change":"2024-04-25T10:17:51.370850+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370850+0000","last_undegraded":"2024-04-25T10:17:51.370850+0000","last_fullsized":"2024-04-25T10:17:51.370850+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371561+0000","last_change":"2024-04-25T10:17:51.371561+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371561+0000","last_undegraded":"2024-04-25T10:17:51.371561+0000","last_fullsized":"2024-04-25T10:17:51.371561+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.369985+0000","last_change":"2024-04-25T10:17:51.369985+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.369985+0000","last_undegraded":"2024-04-25T10:17:51.369985+0000","last_fullsized":"2024-04-25T10:17:51.369985+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361953+0000","last_change":"2024-04-25T10:17:51.361953+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361953+0000","last_undegraded":"2024-04-25T10:17:51.361953+0000","last_fullsized":"2024-04-25T10:17:51.361953+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362115+0000","last_change":"2024-04-25T10:17:51.362115+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362115+0000","last_undegraded":"2024-04-25T10:17:51.362115+0000","last_fullsized":"2024-04-25T10:17:51.362115+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.373269+0000","last_change":"2024-04-25T10:17:51.373269+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.373269+0000","last_undegraded":"2024-04-25T10:17:51.373269+0000","last_fullsized":"2024-04-25T10:17:51.373269+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370995+0000","last_change":"2024-04-25T10:17:51.370995+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370995+0000","last_undegraded":"2024-04-25T10:17:51.370995+0000","last_fullsized":"2024-04-25T10:17:51.370995+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366391+0000","last_change":"2024-04-25T10:17:51.366391+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366391+0000","last_undegraded":"2024-04-25T10:17:51.366391+0000","last_fullsized":"2024-04-25T10:17:51.366391+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370391+0000","last_change":"2024-04-25T10:17:51.370391+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370391+0000","last_undegraded":"2024-04-25T10:17:51.370391+0000","last_fullsized":"2024-04-25T10:17:51.370391+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371819+0000","last_change":"2024-04-25T10:17:51.371819+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371819+0000","last_undegraded":"2024-04-25T10:17:51.371819+0000","last_fullsized":"2024-04-25T10:17:51.371819+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.385023+0000","last_change":"2024-04-25T10:17:52.385023+0000","last_active":"2024-04-25T10:17:52.385023+0000","last_peered":"2024-04-25T10:17:52.385023+0000","last_clean":"2024-04-25T10:17:52.385023+0000","last_became_active":"2024-04-25T10:17:52.384157+0000","last_became_peered":"2024-04-25T10:17:52.384157+0000","last_unstale":"2024-04-25T10:17:52.385023+0000","last_undegraded":"2024-04-25T10:17:52.385023+0000","last_fullsized":"2024-04-25T10:17:52.385023+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.242001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361325+0000","last_change":"2024-04-25T10:17:51.361325+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361325+0000","last_undegraded":"2024-04-25T10:17:51.361325+0000","last_fullsized":"2024-04-25T10:17:51.361325+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.362292+0000","last_change":"2024-04-25T10:17:52.362292+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.362292+0000","last_undegraded":"2024-04-25T10:17:52.362292+0000","last_fullsized":"2024-04-25T10:17:52.362292+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370262+0000","last_change":"2024-04-25T10:17:51.370262+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370262+0000","last_undegraded":"2024-04-25T10:17:51.370262+0000","last_fullsized":"2024-04-25T10:17:51.370262+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362679+0000","last_change":"2024-04-25T10:17:51.362679+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362679+0000","last_undegraded":"2024-04-25T10:17:51.362679+0000","last_fullsized":"2024-04-25T10:17:51.362679+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362767+0000","last_change":"2024-04-25T10:17:51.362767+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362767+0000","last_undegraded":"2024-04-25T10:17:51.362767+0000","last_fullsized":"2024-04-25T10:17:51.362767+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.368522+0000","last_change":"2024-04-25T10:17:51.368522+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.368522+0000","last_undegraded":"2024-04-25T10:17:51.368522+0000","last_fullsized":"2024-04-25T10:17:51.368522+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370747+0000","last_change":"2024-04-25T10:17:51.370747+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370747+0000","last_undegraded":"2024-04-25T10:17:51.370747+0000","last_fullsized":"2024-04-25T10:17:51.370747+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.385243+0000","last_change":"2024-04-25T10:17:52.385243+0000","last_active":"2024-04-25T10:17:52.385243+0000","last_peered":"2024-04-25T10:17:52.385243+0000","last_clean":"2024-04-25T10:17:52.385243+0000","last_became_active":"2024-04-25T10:17:52.384636+0000","last_became_peered":"2024-04-25T10:17:52.384636+0000","last_unstale":"2024-04-25T10:17:52.385243+0000","last_undegraded":"2024-04-25T10:17:52.385243+0000","last_fullsized":"2024-04-25T10:17:52.385243+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.007326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362563+0000","last_change":"2024-04-25T10:17:51.362563+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362563+0000","last_undegraded":"2024-04-25T10:17:51.362563+0000","last_fullsized":"2024-04-25T10:17:51.362563+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361021+0000","last_change":"2024-04-25T10:17:51.361021+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361021+0000","last_undegraded":"2024-04-25T10:17:51.361021+0000","last_fullsized":"2024-04-25T10:17:51.361021+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375238+0000","last_change":"2024-04-25T10:17:51.375238+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375238+0000","last_undegraded":"2024-04-25T10:17:51.375238+0000","last_fullsized":"2024-04-25T10:17:51.375238+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361848+0000","last_change":"2024-04-25T10:17:51.361848+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361848+0000","last_undegraded":"2024-04-25T10:17:51.361848+0000","last_fullsized":"2024-04-25T10:17:51.361848+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374653+0000","last_change":"2024-04-25T10:17:51.374653+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374653+0000","last_undegraded":"2024-04-25T10:17:51.374653+0000","last_fullsized":"2024-04-25T10:17:51.374653+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.363325+0000","last_change":"2024-04-25T10:17:52.363325+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.363325+0000","last_undegraded":"2024-04-25T10:17:52.363325+0000","last_fullsized":"2024-04-25T10:17:52.363325+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371347+0000","last_change":"2024-04-25T10:17:51.371347+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371347+0000","last_undegraded":"2024-04-25T10:17:51.371347+0000","last_fullsized":"2024-04-25T10:17:51.371347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.361408+0000","last_change":"2024-04-25T10:17:52.361408+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.361408+0000","last_undegraded":"2024-04-25T10:17:52.361408+0000","last_fullsized":"2024-04-25T10:17:52.361408+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374913+0000","last_change":"2024-04-25T10:17:51.374913+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374913+0000","last_undegraded":"2024-04-25T10:17:51.374913+0000","last_fullsized":"2024-04-25T10:17:51.374913+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.381190+0000","last_change":"2024-04-25T10:17:51.378796+0000","last_active":"2024-04-25T10:17:52.381190+0000","last_peered":"2024-04-25T10:17:52.381190+0000","last_clean":"2024-04-25T10:17:52.381190+0000","last_became_active":"2024-04-25T10:17:51.378504+0000","last_became_peered":"2024-04-25T10:17:51.378504+0000","last_unstale":"2024-04-25T10:17:52.381190+0000","last_undegraded":"2024-04-25T10:17:52.381190+0000","last_fullsized":"2024-04-25T10:17:52.381190+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:36.761276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360291+0000","last_change":"2024-04-25T10:17:51.360291+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360291+0000","last_undegraded":"2024-04-25T10:17:51.360291+0000","last_fullsized":"2024-04-25T10:17:51.360291+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385683+0000","last_change":"2024-04-25T10:17:51.383528+0000","last_active":"2024-04-25T10:17:51.385683+0000","last_peered":"2024-04-25T10:17:51.385683+0000","last_clean":"2024-04-25T10:17:51.385683+0000","last_became_active":"2024-04-25T10:17:51.382449+0000","last_became_peered":"2024-04-25T10:17:51.382449+0000","last_unstale":"2024-04-25T10:17:51.385683+0000","last_undegraded":"2024-04-25T10:17:51.385683+0000","last_fullsized":"2024-04-25T10:17:51.385683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:11.992831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359546+0000","last_change":"2024-04-25T10:17:51.359546+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359546+0000","last_undegraded":"2024-04-25T10:17:51.359546+0000","last_fullsized":"2024-04-25T10:17:51.359546+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.381144+0000","last_change":"2024-04-25T10:17:51.378958+0000","last_active":"2024-04-25T10:17:51.381144+0000","last_peered":"2024-04-25T10:17:51.381144+0000","last_clean":"2024-04-25T10:17:51.381144+0000","last_became_active":"2024-04-25T10:17:51.378655+0000","last_became_peered":"2024-04-25T10:17:51.378655+0000","last_unstale":"2024-04-25T10:17:51.381144+0000","last_undegraded":"2024-04-25T10:17:51.381144+0000","last_fullsized":"2024-04-25T10:17:51.381144+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:43.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370565+0000","last_change":"2024-04-25T10:17:51.370565+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370565+0000","last_undegraded":"2024-04-25T10:17:51.370565+0000","last_fullsized":"2024-04-25T10:17:51.370565+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.378523+0000","last_change":"2024-04-25T10:17:51.377735+0000","last_active":"2024-04-25T10:17:51.378523+0000","last_peered":"2024-04-25T10:17:51.378523+0000","last_clean":"2024-04-25T10:17:51.378523+0000","last_became_active":"2024-04-25T10:17:51.376589+0000","last_became_peered":"2024-04-25T10:17:51.376589+0000","last_unstale":"2024-04-25T10:17:51.378523+0000","last_undegraded":"2024-04-25T10:17:51.378523+0000","last_fullsized":"2024-04-25T10:17:51.378523+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:10.550511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.358852+0000","last_change":"2024-04-25T10:17:51.358852+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.358852+0000","last_undegraded":"2024-04-25T10:17:51.358852+0000","last_fullsized":"2024-04-25T10:17:51.358852+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380445+0000","last_change":"2024-04-25T10:17:51.379258+0000","last_active":"2024-04-25T10:17:51.380445+0000","last_peered":"2024-04-25T10:17:51.380445+0000","last_clean":"2024-04-25T10:17:51.380445+0000","last_became_active":"2024-04-25T10:17:51.378916+0000","last_became_peered":"2024-04-25T10:17:51.378916+0000","last_unstale":"2024-04-25T10:17:51.380445+0000","last_undegraded":"2024-04-25T10:17:51.380445+0000","last_fullsized":"2024-04-25T10:17:51.380445+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:28.372775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360084+0000","last_change":"2024-04-25T10:17:51.360084+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360084+0000","last_undegraded":"2024-04-25T10:17:51.360084+0000","last_fullsized":"2024-04-25T10:17:51.360084+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384738+0000","last_change":"2024-04-25T10:17:51.376585+0000","last_active":"2024-04-25T10:17:51.384738+0000","last_peered":"2024-04-25T10:17:51.384738+0000","last_clean":"2024-04-25T10:17:51.384738+0000","last_became_active":"2024-04-25T10:17:51.376266+0000","last_became_peered":"2024-04-25T10:17:51.376266+0000","last_unstale":"2024-04-25T10:17:51.384738+0000","last_undegraded":"2024-04-25T10:17:51.384738+0000","last_fullsized":"2024-04-25T10:17:51.384738+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:47.138590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360539+0000","last_change":"2024-04-25T10:17:51.360539+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360539+0000","last_undegraded":"2024-04-25T10:17:51.360539+0000","last_fullsized":"2024-04-25T10:17:51.360539+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384624+0000","last_change":"2024-04-25T10:17:51.383223+0000","last_active":"2024-04-25T10:17:51.384624+0000","last_peered":"2024-04-25T10:17:51.384624+0000","last_clean":"2024-04-25T10:17:51.384624+0000","last_became_active":"2024-04-25T10:17:51.382880+0000","last_became_peered":"2024-04-25T10:17:51.382880+0000","last_unstale":"2024-04-25T10:17:51.384624+0000","last_undegraded":"2024-04-25T10:17:51.384624+0000","last_fullsized":"2024-04-25T10:17:51.384624+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:38.440117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.368384+0000","last_change":"2024-04-25T10:17:52.368384+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.368384+0000","last_undegraded":"2024-04-25T10:17:52.368384+0000","last_fullsized":"2024-04-25T10:17:52.368384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384778+0000","last_change":"2024-04-25T10:17:51.382057+0000","last_active":"2024-04-25T10:17:51.384778+0000","last_peered":"2024-04-25T10:17:51.384778+0000","last_clean":"2024-04-25T10:17:51.384778+0000","last_became_active":"2024-04-25T10:17:51.381426+0000","last_became_peered":"2024-04-25T10:17:51.381426+0000","last_unstale":"2024-04-25T10:17:51.384778+0000","last_undegraded":"2024-04-25T10:17:51.384778+0000","last_fullsized":"2024-04-25T10:17:51.384778+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:19.839264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375547+0000","last_change":"2024-04-25T10:17:51.375547+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375547+0000","last_undegraded":"2024-04-25T10:17:51.375547+0000","last_fullsized":"2024-04-25T10:17:51.375547+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.367621+0000","last_change":"2024-04-25T10:17:51.365010+0000","last_active":"2024-04-25T10:17:51.367621+0000","last_peered":"2024-04-25T10:17:51.367621+0000","last_clean":"2024-04-25T10:17:51.367621+0000","last_became_active":"2024-04-25T10:17:51.364712+0000","last_became_peered":"2024-04-25T10:17:51.364712+0000","last_unstale":"2024-04-25T10:17:51.367621+0000","last_undegraded":"2024-04-25T10:17:51.367621+0000","last_fullsized":"2024-04-25T10:17:51.367621+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.697276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360010+0000","last_change":"2024-04-25T10:17:51.360010+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360010+0000","last_undegraded":"2024-04-25T10:17:51.360010+0000","last_fullsized":"2024-04-25T10:17:51.360010+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.376630+0000","last_change":"2024-04-25T10:17:51.382259+0000","last_active":"2024-04-25T10:17:52.376630+0000","last_peered":"2024-04-25T10:17:52.376630+0000","last_clean":"2024-04-25T10:17:52.376630+0000","last_became_active":"2024-04-25T10:17:51.381960+0000","last_became_peered":"2024-04-25T10:17:51.381960+0000","last_unstale":"2024-04-25T10:17:52.376630+0000","last_undegraded":"2024-04-25T10:17:52.376630+0000","last_fullsized":"2024-04-25T10:17:52.376630+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.670982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360814+0000","last_change":"2024-04-25T10:17:51.360814+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360814+0000","last_undegraded":"2024-04-25T10:17:51.360814+0000","last_fullsized":"2024-04-25T10:17:51.360814+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380759+0000","last_change":"2024-04-25T10:17:51.378190+0000","last_active":"2024-04-25T10:17:51.380759+0000","last_peered":"2024-04-25T10:17:51.380759+0000","last_clean":"2024-04-25T10:17:51.380759+0000","last_became_active":"2024-04-25T10:17:51.377878+0000","last_became_peered":"2024-04-25T10:17:51.377878+0000","last_unstale":"2024-04-25T10:17:51.380759+0000","last_undegraded":"2024-04-25T10:17:51.380759+0000","last_fullsized":"2024-04-25T10:17:51.380759+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:22.718962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.362642+0000","last_change":"2024-04-25T10:17:52.362642+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.362642+0000","last_undegraded":"2024-04-25T10:17:52.362642+0000","last_fullsized":"2024-04-25T10:17:52.362642+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385449+0000","last_change":"2024-04-25T10:17:51.377387+0000","last_active":"2024-04-25T10:17:51.385449+0000","last_peered":"2024-04-25T10:17:51.385449+0000","last_clean":"2024-04-25T10:17:51.385449+0000","last_became_active":"2024-04-25T10:17:51.376999+0000","last_became_peered":"2024-04-25T10:17:51.376999+0000","last_unstale":"2024-04-25T10:17:51.385449+0000","last_undegraded":"2024-04-25T10:17:51.385449+0000","last_fullsized":"2024-04-25T10:17:51.385449+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.697276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362120+0000","last_change":"2024-04-25T10:17:51.362120+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362120+0000","last_undegraded":"2024-04-25T10:17:51.362120+0000","last_fullsized":"2024-04-25T10:17:51.362120+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.376334+0000","last_change":"2024-04-25T10:17:51.374071+0000","last_active":"2024-04-25T10:17:51.376334+0000","last_peered":"2024-04-25T10:17:51.376334+0000","last_clean":"2024-04-25T10:17:51.376334+0000","last_became_active":"2024-04-25T10:17:51.371382+0000","last_became_peered":"2024-04-25T10:17:51.371382+0000","last_unstale":"2024-04-25T10:17:51.376334+0000","last_undegraded":"2024-04-25T10:17:51.376334+0000","last_fullsized":"2024-04-25T10:17:51.376334+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:19.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.369188+0000","last_change":"2024-04-25T10:17:51.369188+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.369188+0000","last_undegraded":"2024-04-25T10:17:51.369188+0000","last_fullsized":"2024-04-25T10:17:51.369188+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385844+0000","last_change":"2024-04-25T10:17:51.383883+0000","last_active":"2024-04-25T10:17:51.385844+0000","last_peered":"2024-04-25T10:17:51.385844+0000","last_clean":"2024-04-25T10:17:51.385844+0000","last_became_active":"2024-04-25T10:17:51.383352+0000","last_became_peered":"2024-04-25T10:17:51.383352+0000","last_unstale":"2024-04-25T10:17:51.385844+0000","last_undegraded":"2024-04-25T10:17:51.385844+0000","last_fullsized":"2024-04-25T10:17:51.385844+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:51.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.363481+0000","last_change":"2024-04-25T10:17:51.363481+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.363481+0000","last_undegraded":"2024-04-25T10:17:51.363481+0000","last_fullsized":"2024-04-25T10:17:51.363481+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384815+0000","last_change":"2024-04-25T10:17:51.382235+0000","last_active":"2024-04-25T10:17:51.384815+0000","last_peered":"2024-04-25T10:17:51.384815+0000","last_clean":"2024-04-25T10:17:51.384815+0000","last_became_active":"2024-04-25T10:17:51.381803+0000","last_became_peered":"2024-04-25T10:17:51.381803+0000","last_unstale":"2024-04-25T10:17:51.384815+0000","last_undegraded":"2024-04-25T10:17:51.384815+0000","last_fullsized":"2024-04-25T10:17:51.384815+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:43.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.365658+0000","last_change":"2024-04-25T10:17:51.365658+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.365658+0000","last_undegraded":"2024-04-25T10:17:51.365658+0000","last_fullsized":"2024-04-25T10:17:51.365658+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380988+0000","last_change":"2024-04-25T10:17:51.378215+0000","last_active":"2024-04-25T10:17:51.380988+0000","last_peered":"2024-04-25T10:17:51.380988+0000","last_clean":"2024-04-25T10:17:51.380988+0000","last_became_active":"2024-04-25T10:17:51.377905+0000","last_became_peered":"2024-04-25T10:17:51.377905+0000","last_unstale":"2024-04-25T10:17:51.380988+0000","last_undegraded":"2024-04-25T10:17:51.380988+0000","last_fullsized":"2024-04-25T10:17:51.380988+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:42.862391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.361848+0000","last_change":"2024-04-25T10:17:52.361848+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.361848+0000","last_undegraded":"2024-04-25T10:17:52.361848+0000","last_fullsized":"2024-04-25T10:17:52.361848+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.378602+0000","last_change":"2024-04-25T10:17:51.367493+0000","last_active":"2024-04-25T10:17:51.378602+0000","last_peered":"2024-04-25T10:17:51.378602+0000","last_clean":"2024-04-25T10:17:51.378602+0000","last_became_active":"2024-04-25T10:17:51.367097+0000","last_became_peered":"2024-04-25T10:17:51.367097+0000","last_unstale":"2024-04-25T10:17:51.378602+0000","last_undegraded":"2024-04-25T10:17:51.378602+0000","last_fullsized":"2024-04-25T10:17:51.378602+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:18.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.373721+0000","last_change":"2024-04-25T10:17:51.373721+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.373721+0000","last_undegraded":"2024-04-25T10:17:51.373721+0000","last_fullsized":"2024-04-25T10:17:51.373721+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385685+0000","last_change":"2024-04-25T10:17:51.383023+0000","last_active":"2024-04-25T10:17:51.385685+0000","last_peered":"2024-04-25T10:17:51.385685+0000","last_clean":"2024-04-25T10:17:51.385685+0000","last_became_active":"2024-04-25T10:17:51.382697+0000","last_became_peered":"2024-04-25T10:17:51.382697+0000","last_unstale":"2024-04-25T10:17:51.385685+0000","last_undegraded":"2024-04-25T10:17:51.385685+0000","last_fullsized":"2024-04-25T10:17:51.385685+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:44.662844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366804+0000","last_change":"2024-04-25T10:17:51.366804+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366804+0000","last_undegraded":"2024-04-25T10:17:51.366804+0000","last_fullsized":"2024-04-25T10:17:51.366804+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384889+0000","last_change":"2024-04-25T10:17:51.381541+0000","last_active":"2024-04-25T10:17:51.384889+0000","last_peered":"2024-04-25T10:17:51.384889+0000","last_clean":"2024-04-25T10:17:51.384889+0000","last_became_active":"2024-04-25T10:17:51.380331+0000","last_became_peered":"2024-04-25T10:17:51.380331+0000","last_unstale":"2024-04-25T10:17:51.384889+0000","last_undegraded":"2024-04-25T10:17:51.384889+0000","last_fullsized":"2024-04-25T10:17:51.384889+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:21.186217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359326+0000","last_change":"2024-04-25T10:17:51.359326+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359326+0000","last_undegraded":"2024-04-25T10:17:51.359326+0000","last_fullsized":"2024-04-25T10:17:51.359326+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.386734+0000","last_change":"2024-04-25T10:17:51.386166+0000","last_active":"2024-04-25T10:17:51.386734+0000","last_peered":"2024-04-25T10:17:51.386734+0000","last_clean":"2024-04-25T10:17:51.386734+0000","last_became_active":"2024-04-25T10:17:51.385874+0000","last_became_peered":"2024-04-25T10:17:51.385874+0000","last_unstale":"2024-04-25T10:17:51.386734+0000","last_undegraded":"2024-04-25T10:17:51.386734+0000","last_fullsized":"2024-04-25T10:17:51.386734+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:22.718962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361684+0000","last_change":"2024-04-25T10:17:51.361684+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361684+0000","last_undegraded":"2024-04-25T10:17:51.361684+0000","last_fullsized":"2024-04-25T10:17:51.361684+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.382580+0000","last_change":"2024-04-25T10:17:51.380152+0000","last_active":"2024-04-25T10:17:51.382580+0000","last_peered":"2024-04-25T10:17:51.382580+0000","last_clean":"2024-04-25T10:17:51.382580+0000","last_became_active":"2024-04-25T10:17:51.379860+0000","last_became_peered":"2024-04-25T10:17:51.379860+0000","last_unstale":"2024-04-25T10:17:51.382580+0000","last_undegraded":"2024-04-25T10:17:51.382580+0000","last_fullsized":"2024-04-25T10:17:51.382580+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:22.437023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.363696+0000","last_change":"2024-04-25T10:17:51.363696+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.363696+0000","last_undegraded":"2024-04-25T10:17:51.363696+0000","last_fullsized":"2024-04-25T10:17:51.363696+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.371607+0000","last_change":"2024-04-25T10:17:51.369127+0000","last_active":"2024-04-25T10:17:51.371607+0000","last_peered":"2024-04-25T10:17:51.371607+0000","last_clean":"2024-04-25T10:17:51.371607+0000","last_became_active":"2024-04-25T10:17:51.368836+0000","last_became_peered":"2024-04-25T10:17:51.368836+0000","last_unstale":"2024-04-25T10:17:51.371607+0000","last_undegraded":"2024-04-25T10:17:51.371607+0000","last_fullsized":"2024-04-25T10:17:51.371607+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:44.824247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375100+0000","last_change":"2024-04-25T10:17:51.375100+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375100+0000","last_undegraded":"2024-04-25T10:17:51.375100+0000","last_fullsized":"2024-04-25T10:17:51.375100+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.383804+0000","last_change":"2024-04-25T10:17:51.382409+0000","last_active":"2024-04-25T10:17:51.383804+0000","last_peered":"2024-04-25T10:17:51.383804+0000","last_clean":"2024-04-25T10:17:51.383804+0000","last_became_active":"2024-04-25T10:17:51.382048+0000","last_became_peered":"2024-04-25T10:17:51.382048+0000","last_unstale":"2024-04-25T10:17:51.383804+0000","last_undegraded":"2024-04-25T10:17:51.383804+0000","last_fullsized":"2024-04-25T10:17:51.383804+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:14.932827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.367562+0000","last_change":"2024-04-25T10:17:51.367562+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.367562+0000","last_undegraded":"2024-04-25T10:17:51.367562+0000","last_fullsized":"2024-04-25T10:17:51.367562+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385694+0000","last_change":"2024-04-25T10:17:51.382866+0000","last_active":"2024-04-25T10:17:51.385694+0000","last_peered":"2024-04-25T10:17:51.385694+0000","last_clean":"2024-04-25T10:17:51.385694+0000","last_became_active":"2024-04-25T10:17:51.382164+0000","last_became_peered":"2024-04-25T10:17:51.382164+0000","last_unstale":"2024-04-25T10:17:51.385694+0000","last_undegraded":"2024-04-25T10:17:51.385694+0000","last_fullsized":"2024-04-25T10:17:51.385694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366912+0000","last_change":"2024-04-25T10:17:51.366912+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366912+0000","last_undegraded":"2024-04-25T10:17:51.366912+0000","last_fullsized":"2024-04-25T10:17:51.366912+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.372980+0000","last_change":"2024-04-25T10:17:51.370374+0000","last_active":"2024-04-25T10:17:51.372980+0000","last_peered":"2024-04-25T10:17:51.372980+0000","last_clean":"2024-04-25T10:17:51.372980+0000","last_became_active":"2024-04-25T10:17:51.369266+0000","last_became_peered":"2024-04-25T10:17:51.369266+0000","last_unstale":"2024-04-25T10:17:51.372980+0000","last_undegraded":"2024-04-25T10:17:51.372980+0000","last_fullsized":"2024-04-25T10:17:51.372980+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:44.662844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359492+0000","last_change":"2024-04-25T10:17:51.359492+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359492+0000","last_undegraded":"2024-04-25T10:17:51.359492+0000","last_fullsized":"2024-04-25T10:17:51.359492+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380712+0000","last_change":"2024-04-25T10:17:51.379048+0000","last_active":"2024-04-25T10:17:51.380712+0000","last_peered":"2024-04-25T10:17:51.380712+0000","last_clean":"2024-04-25T10:17:51.380712+0000","last_became_active":"2024-04-25T10:17:51.378700+0000","last_became_peered":"2024-04-25T10:17:51.378700+0000","last_unstale":"2024-04-25T10:17:51.380712+0000","last_undegraded":"2024-04-25T10:17:51.380712+0000","last_fullsized":"2024-04-25T10:17:51.380712+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:48.614198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.364985+0000","last_change":"2024-04-25T10:17:51.364985+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.364985+0000","last_undegraded":"2024-04-25T10:17:51.364985+0000","last_fullsized":"2024-04-25T10:17:51.364985+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384589+0000","last_change":"2024-04-25T10:17:51.383501+0000","last_active":"2024-04-25T10:17:51.384589+0000","last_peered":"2024-04-25T10:17:51.384589+0000","last_clean":"2024-04-25T10:17:51.384589+0000","last_became_active":"2024-04-25T10:17:51.381248+0000","last_became_peered":"2024-04-25T10:17:51.381248+0000","last_unstale":"2024-04-25T10:17:51.384589+0000","last_undegraded":"2024-04-25T10:17:51.384589+0000","last_fullsized":"2024-04-25T10:17:51.384589+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:02.741516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.388545+0000","last_change":"2024-04-25T10:17:52.382153+0000","last_active":"2024-04-25T10:17:52.388545+0000","last_peered":"2024-04-25T10:17:52.388545+0000","last_clean":"2024-04-25T10:17:52.388545+0000","last_became_active":"2024-04-25T10:17:52.381688+0000","last_became_peered":"2024-04-25T10:17:52.381688+0000","last_unstale":"2024-04-25T10:17:52.388545+0000","last_undegraded":"2024-04-25T10:17:52.388545+0000","last_fullsized":"2024-04-25T10:17:52.388545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:53.783720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.376315+0000","last_change":"2024-04-25T10:17:51.384528+0000","last_active":"2024-04-25T10:17:52.376315+0000","last_peered":"2024-04-25T10:17:52.376315+0000","last_clean":"2024-04-25T10:17:52.376315+0000","last_became_active":"2024-04-25T10:17:51.384176+0000","last_became_peered":"2024-04-25T10:17:51.384176+0000","last_unstale":"2024-04-25T10:17:52.376315+0000","last_undegraded":"2024-04-25T10:17:52.376315+0000","last_fullsized":"2024-04-25T10:17:52.376315+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:48.896090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371188+0000","last_change":"2024-04-25T10:17:51.371188+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371188+0000","last_undegraded":"2024-04-25T10:17:51.371188+0000","last_fullsized":"2024-04-25T10:17:51.371188+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.368066+0000","last_change":"2024-04-25T10:17:51.366496+0000","last_active":"2024-04-25T10:17:51.368066+0000","last_peered":"2024-04-25T10:17:51.368066+0000","last_clean":"2024-04-25T10:17:51.368066+0000","last_became_active":"2024-04-25T10:17:51.364699+0000","last_became_peered":"2024-04-25T10:17:51.364699+0000","last_unstale":"2024-04-25T10:17:51.368066+0000","last_undegraded":"2024-04-25T10:17:51.368066+0000","last_fullsized":"2024-04-25T10:17:51.368066+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361029+0000","last_change":"2024-04-25T10:17:51.361029+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361029+0000","last_undegraded":"2024-04-25T10:17:51.361029+0000","last_fullsized":"2024-04-25T10:17:51.361029+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.381883+0000","last_change":"2024-04-25T10:17:51.379727+0000","last_active":"2024-04-25T10:17:51.381883+0000","last_peered":"2024-04-25T10:17:51.381883+0000","last_clean":"2024-04-25T10:17:51.381883+0000","last_became_active":"2024-04-25T10:17:51.379202+0000","last_became_peered":"2024-04-25T10:17:51.379202+0000","last_unstale":"2024-04-25T10:17:51.381883+0000","last_undegraded":"2024-04-25T10:17:51.381883+0000","last_fullsized":"2024-04-25T10:17:51.381883+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:56.752842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359514+0000","last_change":"2024-04-25T10:17:51.359514+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359514+0000","last_undegraded":"2024-04-25T10:17:51.359514+0000","last_fullsized":"2024-04-25T10:17:51.359514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385949+0000","last_change":"2024-04-25T10:17:51.384876+0000","last_active":"2024-04-25T10:17:51.385949+0000","last_peered":"2024-04-25T10:17:51.385949+0000","last_clean":"2024-04-25T10:17:51.385949+0000","last_became_active":"2024-04-25T10:17:51.384131+0000","last_became_peered":"2024-04-25T10:17:51.384131+0000","last_unstale":"2024-04-25T10:17:51.385949+0000","last_undegraded":"2024-04-25T10:17:51.385949+0000","last_fullsized":"2024-04-25T10:17:51.385949+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:19.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.368225+0000","last_change":"2024-04-25T10:17:51.368225+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.368225+0000","last_undegraded":"2024-04-25T10:17:51.368225+0000","last_fullsized":"2024-04-25T10:17:51.368225+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.373581+0000","last_change":"2024-04-25T10:17:51.371004+0000","last_active":"2024-04-25T10:17:51.373581+0000","last_peered":"2024-04-25T10:17:51.373581+0000","last_clean":"2024-04-25T10:17:51.373581+0000","last_became_active":"2024-04-25T10:17:51.370705+0000","last_became_peered":"2024-04-25T10:17:51.370705+0000","last_unstale":"2024-04-25T10:17:51.373581+0000","last_undegraded":"2024-04-25T10:17:51.373581+0000","last_fullsized":"2024-04-25T10:17:51.373581+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:11.992831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":431,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.376627+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:17:51.376627+0000","last_peered":"2024-04-25T10:17:51.376627+0000","last_clean":"2024-04-25T10:17:51.376627+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:17:51.376627+0000","last_undegraded":"2024-04-25T10:17:51.376627+0000","last_fullsized":"2024-04-25T10:17:51.376627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2568192,"data_stored":1516028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87192,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284648,"statfs":{"total":96636764160,"available":96547479552,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47580,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324260,"statfs":{"total":96636764160,"available":96588042240,"internally_reserved":0,"allocated":2568192,"data_stored":1516028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:57.307 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:57.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.545+0000 7fe9ebe00640 1 Processor -- start 2024-04-25T10:17:57.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.545+0000 7fe9ebe00640 1 -- start start 2024-04-25T10:17:57.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9ebe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe9ec0728a0 0x7fe9ec06bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9ebe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe9ec06c130 0x7fe9ec0b3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9ebe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9ec06c540 con 0x7fe9ec0addc0 2024-04-25T10:17:57.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9ebe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec15ed90 con 0x7fe9ec06c130 2024-04-25T10:17:57.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9ebe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9ec15e570 con 0x7fe9ec0728a0 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9eb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe9ec06c130 0x7fe9ec0b3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9eb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe9ec06c130 0x7fe9ec0b3560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39710/0 (socket says 172.21.15.165:39710) 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9eb400640 1 -- 172.21.15.165:0/3928173284 learned_addr learned my addr 172.21.15.165:0/3928173284 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.547+0000 7fe9eaa00640 1 --1- 172.21.15.165:0/3928173284 >> v1:172.21.15.165:6789/0 conn(0x7fe9ec0addc0 0x7fe9ec15de60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42194/0 (socket says 172.21.15.165:42194) 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.548+0000 7fe9ea000640 1 --2- 172.21.15.165:0/3928173284 >> v2:172.21.15.172:3300/0 conn(0x7fe9ec0728a0 0x7fe9ec06bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.548+0000 7fe9eb400640 1 -- 172.21.15.165:0/3928173284 >> v2:172.21.15.172:3300/0 conn(0x7fe9ec0728a0 msgr2=0x7fe9ec06bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:57.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.548+0000 7fe9eb400640 1 --2- 172.21.15.165:0/3928173284 >> v2:172.21.15.172:3300/0 conn(0x7fe9ec0728a0 0x7fe9ec06bbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.548+0000 7fe9eb400640 1 -- 172.21.15.165:0/3928173284 >> v1:172.21.15.165:6789/0 conn(0x7fe9ec0addc0 legacy=0x7fe9ec15de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.548+0000 7fe9eb400640 1 -- 172.21.15.165:0/3928173284 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec15e7d0 con 0x7fe9ec06c130 2024-04-25T10:17:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.549+0000 7fe9eb400640 1 --2- 172.21.15.165:0/3928173284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec06c130 0x7fe9ec0b3560 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec15ef10 tx=0x7fe9e402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6090b769f90e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.549+0000 7fe9e9600640 1 -- 172.21.15.165:0/3928173284 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4037070 con 0x7fe9ec06c130 2024-04-25T10:17:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.549+0000 7fe9e9600640 1 -- 172.21.15.165:0/3928173284 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4032440 con 0x7fe9ec06c130 2024-04-25T10:17:57.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.549+0000 7fe9e9600640 1 -- 172.21.15.165:0/3928173284 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4032810 con 0x7fe9ec06c130 2024-04-25T10:17:57.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 -- 172.21.15.165:0/3928173284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec06c130 msgr2=0x7fe9ec0b3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/3928173284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec06c130 0x7fe9ec0b3560 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec15ef10 tx=0x7fe9e402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 -- 172.21.15.165:0/3928173284 shutdown_connections 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/3928173284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec06c130 0x7fe9ec0b3560 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/3928173284 >> v2:172.21.15.172:3300/0 conn(0x7fe9ec0728a0 0x7fe9ec06bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 -- 172.21.15.165:0/3928173284 >> 172.21.15.165:0/3928173284 conn(0x7fe9ec0718d0 msgr2=0x7fe9ec071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 -- 172.21.15.165:0/3928173284 shutdown_connections 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.550+0000 7fe9ebe00640 1 -- 172.21.15.165:0/3928173284 wait complete. 2024-04-25T10:17:57.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.552+0000 7fe9ebe00640 1 Processor -- start 2024-04-25T10:17:57.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.552+0000 7fe9ebe00640 1 -- start start 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9ec06c130 0x7fe9ec148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 0x7fe9ec144830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ebe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ec0addc0 0x7fe9ec144d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ebe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b3040 con 0x7fe9ec0728a0 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ebe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0b2d60 con 0x7fe9ec0addc0 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ebe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9ec0ae1a0 con 0x7fe9ec06c130 2024-04-25T10:17:57.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ea000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 0x7fe9ec144830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ea000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 0x7fe9ec144830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39712/0 (socket says 172.21.15.165:39712) 2024-04-25T10:17:57.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.553+0000 7fe9ea000640 1 -- 172.21.15.165:0/1223049438 learned_addr learned my addr 172.21.15.165:0/1223049438 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9eaa00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9ec06c130 0x7fe9ec148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ea000640 1 -- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9ec06c130 msgr2=0x7fe9ec148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ea000640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9ec06c130 0x7fe9ec148660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9eb400640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ec0addc0 0x7fe9ec144d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ea000640 1 -- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ec0addc0 msgr2=0x7fe9ec144d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ea000640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ec0addc0 0x7fe9ec144d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ea000640 1 -- 172.21.15.165:0/1223049438 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9ec142940 con 0x7fe9ec0728a0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ea000640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 0x7fe9ec144830 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400d800 tx=0x7fe9d400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400b6c0 con 0x7fe9ec0728a0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9d4019070 con 0x7fe9ec0728a0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9d400ba80 con 0x7fe9ec0728a0 2024-04-25T10:17:57.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.554+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9ec142c20 con 0x7fe9ec0728a0 2024-04-25T10:17:57.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.555+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9ec1431e0 con 0x7fe9ec0728a0 2024-04-25T10:17:57.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.556+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9ec073990 con 0x7fe9ec0728a0 2024-04-25T10:17:57.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.557+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe9d400bc20 con 0x7fe9ec0728a0 2024-04-25T10:17:57.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.557+0000 7fe9e2a00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9c806b4d0 0x7fe9c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:57.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.558+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe9d408a850 con 0x7fe9ec0728a0 2024-04-25T10:17:57.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.558+0000 7fe9eaa00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9c806b4d0 0x7fe9c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:57.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.558+0000 7fe9eaa00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9c806b4d0 0x7fe9c806d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec156eb0 tx=0x7fe9dc009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:57.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.562+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9d4059530 con 0x7fe9ec0728a0 2024-04-25T10:17:57.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.752+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe9ec061c50 con 0x7fe9c806b4d0 2024-04-25T10:17:57.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.769+0000 7fe9e2a00640 1 -- 172.21.15.165:0/1223049438 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+247518 (secure 0 0 0) 0x7fe9ec061c50 con 0x7fe9c806b4d0 2024-04-25T10:17:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:57.777 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9c806b4d0 msgr2=0x7fe9c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9c806b4d0 0x7fe9c806d990 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec156eb0 tx=0x7fe9dc009040 comp rx=0 tx=0).stop 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 msgr2=0x7fe9ec144830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 0x7fe9ec144830 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fe9d400d800 tx=0x7fe9d400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 shutdown_connections 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe9c806b4d0 0x7fe9c806d990 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9ec0addc0 0x7fe9ec144d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe9ec0728a0 0x7fe9ec144830 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 --2- 172.21.15.165:0/1223049438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9ec06c130 0x7fe9ec148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:57.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 >> 172.21.15.165:0/1223049438 conn(0x7fe9ec0718d0 msgr2=0x7fe9ec074060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:57.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 shutdown_connections 2024-04-25T10:17:57.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:57.776+0000 7fe9ebe00640 1 -- 172.21.15.165:0/1223049438 wait complete. 2024-04-25T10:17:57.799 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":634,"stamp":"2024-04-25T10:17:56.453207+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591040,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":167,"num_write_kb":2418,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":124,"ondisk_log_size":124,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":45,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":515776,"kb_used_data":16584,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754458944,"statfs":{"total":773094113280,"available":772565958656,"internally_reserved":0,"allocated":16982016,"data_stored":8586016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":18,"apply_latency_ms":18,"commit_latency_ns":18000000,"apply_latency_ns":18000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001308"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.368750+0000","last_change":"2024-04-25T10:17:52.368750+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.368750+0000","last_undegraded":"2024-04-25T10:17:52.368750+0000","last_fullsized":"2024-04-25T10:17:52.368750+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":["4","6"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374180+0000","last_change":"2024-04-25T10:17:51.374180+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374180+0000","last_undegraded":"2024-04-25T10:17:51.374180+0000","last_fullsized":"2024-04-25T10:17:51.374180+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.389531+0000","last_change":"2024-04-25T10:17:52.377620+0000","last_active":"2024-04-25T10:17:52.389531+0000","last_peered":"2024-04-25T10:17:52.389531+0000","last_clean":"2024-04-25T10:17:52.389531+0000","last_became_active":"2024-04-25T10:17:52.377333+0000","last_became_peered":"2024-04-25T10:17:52.377333+0000","last_unstale":"2024-04-25T10:17:52.389531+0000","last_undegraded":"2024-04-25T10:17:52.389531+0000","last_fullsized":"2024-04-25T10:17:52.389531+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:18.918486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.367282+0000","last_change":"2024-04-25T10:17:52.367282+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.367282+0000","last_undegraded":"2024-04-25T10:17:52.367282+0000","last_fullsized":"2024-04-25T10:17:52.367282+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.367557+0000","last_change":"2024-04-25T10:17:51.367557+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.367557+0000","last_undegraded":"2024-04-25T10:17:51.367557+0000","last_fullsized":"2024-04-25T10:17:51.367557+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370850+0000","last_change":"2024-04-25T10:17:51.370850+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370850+0000","last_undegraded":"2024-04-25T10:17:51.370850+0000","last_fullsized":"2024-04-25T10:17:51.370850+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371561+0000","last_change":"2024-04-25T10:17:51.371561+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371561+0000","last_undegraded":"2024-04-25T10:17:51.371561+0000","last_fullsized":"2024-04-25T10:17:51.371561+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.369985+0000","last_change":"2024-04-25T10:17:51.369985+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.369985+0000","last_undegraded":"2024-04-25T10:17:51.369985+0000","last_fullsized":"2024-04-25T10:17:51.369985+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361953+0000","last_change":"2024-04-25T10:17:51.361953+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361953+0000","last_undegraded":"2024-04-25T10:17:51.361953+0000","last_fullsized":"2024-04-25T10:17:51.361953+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362115+0000","last_change":"2024-04-25T10:17:51.362115+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362115+0000","last_undegraded":"2024-04-25T10:17:51.362115+0000","last_fullsized":"2024-04-25T10:17:51.362115+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.373269+0000","last_change":"2024-04-25T10:17:51.373269+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.373269+0000","last_undegraded":"2024-04-25T10:17:51.373269+0000","last_fullsized":"2024-04-25T10:17:51.373269+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370995+0000","last_change":"2024-04-25T10:17:51.370995+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370995+0000","last_undegraded":"2024-04-25T10:17:51.370995+0000","last_fullsized":"2024-04-25T10:17:51.370995+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366391+0000","last_change":"2024-04-25T10:17:51.366391+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366391+0000","last_undegraded":"2024-04-25T10:17:51.366391+0000","last_fullsized":"2024-04-25T10:17:51.366391+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370391+0000","last_change":"2024-04-25T10:17:51.370391+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370391+0000","last_undegraded":"2024-04-25T10:17:51.370391+0000","last_fullsized":"2024-04-25T10:17:51.370391+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371819+0000","last_change":"2024-04-25T10:17:51.371819+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371819+0000","last_undegraded":"2024-04-25T10:17:51.371819+0000","last_fullsized":"2024-04-25T10:17:51.371819+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.385023+0000","last_change":"2024-04-25T10:17:52.385023+0000","last_active":"2024-04-25T10:17:52.385023+0000","last_peered":"2024-04-25T10:17:52.385023+0000","last_clean":"2024-04-25T10:17:52.385023+0000","last_became_active":"2024-04-25T10:17:52.384157+0000","last_became_peered":"2024-04-25T10:17:52.384157+0000","last_unstale":"2024-04-25T10:17:52.385023+0000","last_undegraded":"2024-04-25T10:17:52.385023+0000","last_fullsized":"2024-04-25T10:17:52.385023+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:25:16.242001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361325+0000","last_change":"2024-04-25T10:17:51.361325+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361325+0000","last_undegraded":"2024-04-25T10:17:51.361325+0000","last_fullsized":"2024-04-25T10:17:51.361325+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.362292+0000","last_change":"2024-04-25T10:17:52.362292+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.362292+0000","last_undegraded":"2024-04-25T10:17:52.362292+0000","last_fullsized":"2024-04-25T10:17:52.362292+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":["6","1"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370262+0000","last_change":"2024-04-25T10:17:51.370262+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370262+0000","last_undegraded":"2024-04-25T10:17:51.370262+0000","last_fullsized":"2024-04-25T10:17:51.370262+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362679+0000","last_change":"2024-04-25T10:17:51.362679+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362679+0000","last_undegraded":"2024-04-25T10:17:51.362679+0000","last_fullsized":"2024-04-25T10:17:51.362679+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362767+0000","last_change":"2024-04-25T10:17:51.362767+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362767+0000","last_undegraded":"2024-04-25T10:17:51.362767+0000","last_fullsized":"2024-04-25T10:17:51.362767+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.368522+0000","last_change":"2024-04-25T10:17:51.368522+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.368522+0000","last_undegraded":"2024-04-25T10:17:51.368522+0000","last_fullsized":"2024-04-25T10:17:51.368522+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370747+0000","last_change":"2024-04-25T10:17:51.370747+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370747+0000","last_undegraded":"2024-04-25T10:17:51.370747+0000","last_fullsized":"2024-04-25T10:17:51.370747+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.385243+0000","last_change":"2024-04-25T10:17:52.385243+0000","last_active":"2024-04-25T10:17:52.385243+0000","last_peered":"2024-04-25T10:17:52.385243+0000","last_clean":"2024-04-25T10:17:52.385243+0000","last_became_active":"2024-04-25T10:17:52.384636+0000","last_became_peered":"2024-04-25T10:17:52.384636+0000","last_unstale":"2024-04-25T10:17:52.385243+0000","last_undegraded":"2024-04-25T10:17:52.385243+0000","last_fullsized":"2024-04-25T10:17:52.385243+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:11:44.007326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362563+0000","last_change":"2024-04-25T10:17:51.362563+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362563+0000","last_undegraded":"2024-04-25T10:17:51.362563+0000","last_fullsized":"2024-04-25T10:17:51.362563+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361021+0000","last_change":"2024-04-25T10:17:51.361021+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361021+0000","last_undegraded":"2024-04-25T10:17:51.361021+0000","last_fullsized":"2024-04-25T10:17:51.361021+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375238+0000","last_change":"2024-04-25T10:17:51.375238+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375238+0000","last_undegraded":"2024-04-25T10:17:51.375238+0000","last_fullsized":"2024-04-25T10:17:51.375238+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361848+0000","last_change":"2024-04-25T10:17:51.361848+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361848+0000","last_undegraded":"2024-04-25T10:17:51.361848+0000","last_fullsized":"2024-04-25T10:17:51.361848+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374653+0000","last_change":"2024-04-25T10:17:51.374653+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374653+0000","last_undegraded":"2024-04-25T10:17:51.374653+0000","last_fullsized":"2024-04-25T10:17:51.374653+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.363325+0000","last_change":"2024-04-25T10:17:52.363325+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.363325+0000","last_undegraded":"2024-04-25T10:17:52.363325+0000","last_fullsized":"2024-04-25T10:17:52.363325+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":["6","0"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371347+0000","last_change":"2024-04-25T10:17:51.371347+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371347+0000","last_undegraded":"2024-04-25T10:17:51.371347+0000","last_fullsized":"2024-04-25T10:17:51.371347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.361408+0000","last_change":"2024-04-25T10:17:52.361408+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.361408+0000","last_undegraded":"2024-04-25T10:17:52.361408+0000","last_fullsized":"2024-04-25T10:17:52.361408+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.374913+0000","last_change":"2024-04-25T10:17:51.374913+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.374913+0000","last_undegraded":"2024-04-25T10:17:51.374913+0000","last_fullsized":"2024-04-25T10:17:51.374913+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.381190+0000","last_change":"2024-04-25T10:17:51.378796+0000","last_active":"2024-04-25T10:17:52.381190+0000","last_peered":"2024-04-25T10:17:52.381190+0000","last_clean":"2024-04-25T10:17:52.381190+0000","last_became_active":"2024-04-25T10:17:51.378504+0000","last_became_peered":"2024-04-25T10:17:51.378504+0000","last_unstale":"2024-04-25T10:17:52.381190+0000","last_undegraded":"2024-04-25T10:17:52.381190+0000","last_fullsized":"2024-04-25T10:17:52.381190+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:36.761276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360291+0000","last_change":"2024-04-25T10:17:51.360291+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360291+0000","last_undegraded":"2024-04-25T10:17:51.360291+0000","last_fullsized":"2024-04-25T10:17:51.360291+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385683+0000","last_change":"2024-04-25T10:17:51.383528+0000","last_active":"2024-04-25T10:17:51.385683+0000","last_peered":"2024-04-25T10:17:51.385683+0000","last_clean":"2024-04-25T10:17:51.385683+0000","last_became_active":"2024-04-25T10:17:51.382449+0000","last_became_peered":"2024-04-25T10:17:51.382449+0000","last_unstale":"2024-04-25T10:17:51.385683+0000","last_undegraded":"2024-04-25T10:17:51.385683+0000","last_fullsized":"2024-04-25T10:17:51.385683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:11.992831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359546+0000","last_change":"2024-04-25T10:17:51.359546+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359546+0000","last_undegraded":"2024-04-25T10:17:51.359546+0000","last_fullsized":"2024-04-25T10:17:51.359546+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.381144+0000","last_change":"2024-04-25T10:17:51.378958+0000","last_active":"2024-04-25T10:17:51.381144+0000","last_peered":"2024-04-25T10:17:51.381144+0000","last_clean":"2024-04-25T10:17:51.381144+0000","last_became_active":"2024-04-25T10:17:51.378655+0000","last_became_peered":"2024-04-25T10:17:51.378655+0000","last_unstale":"2024-04-25T10:17:51.381144+0000","last_undegraded":"2024-04-25T10:17:51.381144+0000","last_fullsized":"2024-04-25T10:17:51.381144+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:43.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.370565+0000","last_change":"2024-04-25T10:17:51.370565+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.370565+0000","last_undegraded":"2024-04-25T10:17:51.370565+0000","last_fullsized":"2024-04-25T10:17:51.370565+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.378523+0000","last_change":"2024-04-25T10:17:51.377735+0000","last_active":"2024-04-25T10:17:51.378523+0000","last_peered":"2024-04-25T10:17:51.378523+0000","last_clean":"2024-04-25T10:17:51.378523+0000","last_became_active":"2024-04-25T10:17:51.376589+0000","last_became_peered":"2024-04-25T10:17:51.376589+0000","last_unstale":"2024-04-25T10:17:51.378523+0000","last_undegraded":"2024-04-25T10:17:51.378523+0000","last_fullsized":"2024-04-25T10:17:51.378523+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:10.550511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.358852+0000","last_change":"2024-04-25T10:17:51.358852+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.358852+0000","last_undegraded":"2024-04-25T10:17:51.358852+0000","last_fullsized":"2024-04-25T10:17:51.358852+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380445+0000","last_change":"2024-04-25T10:17:51.379258+0000","last_active":"2024-04-25T10:17:51.380445+0000","last_peered":"2024-04-25T10:17:51.380445+0000","last_clean":"2024-04-25T10:17:51.380445+0000","last_became_active":"2024-04-25T10:17:51.378916+0000","last_became_peered":"2024-04-25T10:17:51.378916+0000","last_unstale":"2024-04-25T10:17:51.380445+0000","last_undegraded":"2024-04-25T10:17:51.380445+0000","last_fullsized":"2024-04-25T10:17:51.380445+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:40:28.372775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360084+0000","last_change":"2024-04-25T10:17:51.360084+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360084+0000","last_undegraded":"2024-04-25T10:17:51.360084+0000","last_fullsized":"2024-04-25T10:17:51.360084+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384738+0000","last_change":"2024-04-25T10:17:51.376585+0000","last_active":"2024-04-25T10:17:51.384738+0000","last_peered":"2024-04-25T10:17:51.384738+0000","last_clean":"2024-04-25T10:17:51.384738+0000","last_became_active":"2024-04-25T10:17:51.376266+0000","last_became_peered":"2024-04-25T10:17:51.376266+0000","last_unstale":"2024-04-25T10:17:51.384738+0000","last_undegraded":"2024-04-25T10:17:51.384738+0000","last_fullsized":"2024-04-25T10:17:51.384738+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:47.138590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360539+0000","last_change":"2024-04-25T10:17:51.360539+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360539+0000","last_undegraded":"2024-04-25T10:17:51.360539+0000","last_fullsized":"2024-04-25T10:17:51.360539+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384624+0000","last_change":"2024-04-25T10:17:51.383223+0000","last_active":"2024-04-25T10:17:51.384624+0000","last_peered":"2024-04-25T10:17:51.384624+0000","last_clean":"2024-04-25T10:17:51.384624+0000","last_became_active":"2024-04-25T10:17:51.382880+0000","last_became_peered":"2024-04-25T10:17:51.382880+0000","last_unstale":"2024-04-25T10:17:51.384624+0000","last_undegraded":"2024-04-25T10:17:51.384624+0000","last_fullsized":"2024-04-25T10:17:51.384624+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:27:38.440117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.368384+0000","last_change":"2024-04-25T10:17:52.368384+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.368384+0000","last_undegraded":"2024-04-25T10:17:52.368384+0000","last_fullsized":"2024-04-25T10:17:52.368384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":["4","5"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384778+0000","last_change":"2024-04-25T10:17:51.382057+0000","last_active":"2024-04-25T10:17:51.384778+0000","last_peered":"2024-04-25T10:17:51.384778+0000","last_clean":"2024-04-25T10:17:51.384778+0000","last_became_active":"2024-04-25T10:17:51.381426+0000","last_became_peered":"2024-04-25T10:17:51.381426+0000","last_unstale":"2024-04-25T10:17:51.384778+0000","last_undegraded":"2024-04-25T10:17:51.384778+0000","last_fullsized":"2024-04-25T10:17:51.384778+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:03:19.839264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375547+0000","last_change":"2024-04-25T10:17:51.375547+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375547+0000","last_undegraded":"2024-04-25T10:17:51.375547+0000","last_fullsized":"2024-04-25T10:17:51.375547+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.367621+0000","last_change":"2024-04-25T10:17:51.365010+0000","last_active":"2024-04-25T10:17:51.367621+0000","last_peered":"2024-04-25T10:17:51.367621+0000","last_clean":"2024-04-25T10:17:51.367621+0000","last_became_active":"2024-04-25T10:17:51.364712+0000","last_became_peered":"2024-04-25T10:17:51.364712+0000","last_unstale":"2024-04-25T10:17:51.367621+0000","last_undegraded":"2024-04-25T10:17:51.367621+0000","last_fullsized":"2024-04-25T10:17:51.367621+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.697276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360010+0000","last_change":"2024-04-25T10:17:51.360010+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360010+0000","last_undegraded":"2024-04-25T10:17:51.360010+0000","last_fullsized":"2024-04-25T10:17:51.360010+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.376630+0000","last_change":"2024-04-25T10:17:51.382259+0000","last_active":"2024-04-25T10:17:52.376630+0000","last_peered":"2024-04-25T10:17:52.376630+0000","last_clean":"2024-04-25T10:17:52.376630+0000","last_became_active":"2024-04-25T10:17:51.381960+0000","last_became_peered":"2024-04-25T10:17:51.381960+0000","last_unstale":"2024-04-25T10:17:52.376630+0000","last_undegraded":"2024-04-25T10:17:52.376630+0000","last_fullsized":"2024-04-25T10:17:52.376630+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.670982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.360814+0000","last_change":"2024-04-25T10:17:51.360814+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.360814+0000","last_undegraded":"2024-04-25T10:17:51.360814+0000","last_fullsized":"2024-04-25T10:17:51.360814+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380759+0000","last_change":"2024-04-25T10:17:51.378190+0000","last_active":"2024-04-25T10:17:51.380759+0000","last_peered":"2024-04-25T10:17:51.380759+0000","last_clean":"2024-04-25T10:17:51.380759+0000","last_became_active":"2024-04-25T10:17:51.377878+0000","last_became_peered":"2024-04-25T10:17:51.377878+0000","last_unstale":"2024-04-25T10:17:51.380759+0000","last_undegraded":"2024-04-25T10:17:51.380759+0000","last_fullsized":"2024-04-25T10:17:51.380759+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:22.718962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.362642+0000","last_change":"2024-04-25T10:17:52.362642+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.362642+0000","last_undegraded":"2024-04-25T10:17:52.362642+0000","last_fullsized":"2024-04-25T10:17:52.362642+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385449+0000","last_change":"2024-04-25T10:17:51.377387+0000","last_active":"2024-04-25T10:17:51.385449+0000","last_peered":"2024-04-25T10:17:51.385449+0000","last_clean":"2024-04-25T10:17:51.385449+0000","last_became_active":"2024-04-25T10:17:51.376999+0000","last_became_peered":"2024-04-25T10:17:51.376999+0000","last_unstale":"2024-04-25T10:17:51.385449+0000","last_undegraded":"2024-04-25T10:17:51.385449+0000","last_fullsized":"2024-04-25T10:17:51.385449+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:33.697276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.362120+0000","last_change":"2024-04-25T10:17:51.362120+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.362120+0000","last_undegraded":"2024-04-25T10:17:51.362120+0000","last_fullsized":"2024-04-25T10:17:51.362120+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.376334+0000","last_change":"2024-04-25T10:17:51.374071+0000","last_active":"2024-04-25T10:17:51.376334+0000","last_peered":"2024-04-25T10:17:51.376334+0000","last_clean":"2024-04-25T10:17:51.376334+0000","last_became_active":"2024-04-25T10:17:51.371382+0000","last_became_peered":"2024-04-25T10:17:51.371382+0000","last_unstale":"2024-04-25T10:17:51.376334+0000","last_undegraded":"2024-04-25T10:17:51.376334+0000","last_fullsized":"2024-04-25T10:17:51.376334+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:19.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.369188+0000","last_change":"2024-04-25T10:17:51.369188+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.369188+0000","last_undegraded":"2024-04-25T10:17:51.369188+0000","last_fullsized":"2024-04-25T10:17:51.369188+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385844+0000","last_change":"2024-04-25T10:17:51.383883+0000","last_active":"2024-04-25T10:17:51.385844+0000","last_peered":"2024-04-25T10:17:51.385844+0000","last_clean":"2024-04-25T10:17:51.385844+0000","last_became_active":"2024-04-25T10:17:51.383352+0000","last_became_peered":"2024-04-25T10:17:51.383352+0000","last_unstale":"2024-04-25T10:17:51.385844+0000","last_undegraded":"2024-04-25T10:17:51.385844+0000","last_fullsized":"2024-04-25T10:17:51.385844+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:51.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.363481+0000","last_change":"2024-04-25T10:17:51.363481+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.363481+0000","last_undegraded":"2024-04-25T10:17:51.363481+0000","last_fullsized":"2024-04-25T10:17:51.363481+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384815+0000","last_change":"2024-04-25T10:17:51.382235+0000","last_active":"2024-04-25T10:17:51.384815+0000","last_peered":"2024-04-25T10:17:51.384815+0000","last_clean":"2024-04-25T10:17:51.384815+0000","last_became_active":"2024-04-25T10:17:51.381803+0000","last_became_peered":"2024-04-25T10:17:51.381803+0000","last_unstale":"2024-04-25T10:17:51.384815+0000","last_undegraded":"2024-04-25T10:17:51.384815+0000","last_fullsized":"2024-04-25T10:17:51.384815+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:24:43.842417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.365658+0000","last_change":"2024-04-25T10:17:51.365658+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.365658+0000","last_undegraded":"2024-04-25T10:17:51.365658+0000","last_fullsized":"2024-04-25T10:17:51.365658+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380988+0000","last_change":"2024-04-25T10:17:51.378215+0000","last_active":"2024-04-25T10:17:51.380988+0000","last_peered":"2024-04-25T10:17:51.380988+0000","last_clean":"2024-04-25T10:17:51.380988+0000","last_became_active":"2024-04-25T10:17:51.377905+0000","last_became_peered":"2024-04-25T10:17:51.377905+0000","last_unstale":"2024-04-25T10:17:51.380988+0000","last_undegraded":"2024-04-25T10:17:51.380988+0000","last_fullsized":"2024-04-25T10:17:51.380988+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:42.862391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":3,"reported_epoch":158,"state":"creating+activating","last_fresh":"2024-04-25T10:17:52.361848+0000","last_change":"2024-04-25T10:17:52.361848+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:52.361848+0000","last_undegraded":"2024-04-25T10:17:52.361848+0000","last_fullsized":"2024-04-25T10:17:52.361848+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":["6","5"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.378602+0000","last_change":"2024-04-25T10:17:51.367493+0000","last_active":"2024-04-25T10:17:51.378602+0000","last_peered":"2024-04-25T10:17:51.378602+0000","last_clean":"2024-04-25T10:17:51.378602+0000","last_became_active":"2024-04-25T10:17:51.367097+0000","last_became_peered":"2024-04-25T10:17:51.367097+0000","last_unstale":"2024-04-25T10:17:51.378602+0000","last_undegraded":"2024-04-25T10:17:51.378602+0000","last_fullsized":"2024-04-25T10:17:51.378602+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:15:18.160313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.373721+0000","last_change":"2024-04-25T10:17:51.373721+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.373721+0000","last_undegraded":"2024-04-25T10:17:51.373721+0000","last_fullsized":"2024-04-25T10:17:51.373721+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385685+0000","last_change":"2024-04-25T10:17:51.383023+0000","last_active":"2024-04-25T10:17:51.385685+0000","last_peered":"2024-04-25T10:17:51.385685+0000","last_clean":"2024-04-25T10:17:51.385685+0000","last_became_active":"2024-04-25T10:17:51.382697+0000","last_became_peered":"2024-04-25T10:17:51.382697+0000","last_unstale":"2024-04-25T10:17:51.385685+0000","last_undegraded":"2024-04-25T10:17:51.385685+0000","last_fullsized":"2024-04-25T10:17:51.385685+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:44.662844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366804+0000","last_change":"2024-04-25T10:17:51.366804+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366804+0000","last_undegraded":"2024-04-25T10:17:51.366804+0000","last_fullsized":"2024-04-25T10:17:51.366804+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384889+0000","last_change":"2024-04-25T10:17:51.381541+0000","last_active":"2024-04-25T10:17:51.384889+0000","last_peered":"2024-04-25T10:17:51.384889+0000","last_clean":"2024-04-25T10:17:51.384889+0000","last_became_active":"2024-04-25T10:17:51.380331+0000","last_became_peered":"2024-04-25T10:17:51.380331+0000","last_unstale":"2024-04-25T10:17:51.384889+0000","last_undegraded":"2024-04-25T10:17:51.384889+0000","last_fullsized":"2024-04-25T10:17:51.384889+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:53:21.186217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359326+0000","last_change":"2024-04-25T10:17:51.359326+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359326+0000","last_undegraded":"2024-04-25T10:17:51.359326+0000","last_fullsized":"2024-04-25T10:17:51.359326+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.386734+0000","last_change":"2024-04-25T10:17:51.386166+0000","last_active":"2024-04-25T10:17:51.386734+0000","last_peered":"2024-04-25T10:17:51.386734+0000","last_clean":"2024-04-25T10:17:51.386734+0000","last_became_active":"2024-04-25T10:17:51.385874+0000","last_became_peered":"2024-04-25T10:17:51.385874+0000","last_unstale":"2024-04-25T10:17:51.386734+0000","last_undegraded":"2024-04-25T10:17:51.386734+0000","last_fullsized":"2024-04-25T10:17:51.386734+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:22.718962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361684+0000","last_change":"2024-04-25T10:17:51.361684+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361684+0000","last_undegraded":"2024-04-25T10:17:51.361684+0000","last_fullsized":"2024-04-25T10:17:51.361684+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.382580+0000","last_change":"2024-04-25T10:17:51.380152+0000","last_active":"2024-04-25T10:17:51.382580+0000","last_peered":"2024-04-25T10:17:51.382580+0000","last_clean":"2024-04-25T10:17:51.382580+0000","last_became_active":"2024-04-25T10:17:51.379860+0000","last_became_peered":"2024-04-25T10:17:51.379860+0000","last_unstale":"2024-04-25T10:17:51.382580+0000","last_undegraded":"2024-04-25T10:17:51.382580+0000","last_fullsized":"2024-04-25T10:17:51.382580+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:22.437023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.363696+0000","last_change":"2024-04-25T10:17:51.363696+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.363696+0000","last_undegraded":"2024-04-25T10:17:51.363696+0000","last_fullsized":"2024-04-25T10:17:51.363696+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.371607+0000","last_change":"2024-04-25T10:17:51.369127+0000","last_active":"2024-04-25T10:17:51.371607+0000","last_peered":"2024-04-25T10:17:51.371607+0000","last_clean":"2024-04-25T10:17:51.371607+0000","last_became_active":"2024-04-25T10:17:51.368836+0000","last_became_peered":"2024-04-25T10:17:51.368836+0000","last_unstale":"2024-04-25T10:17:51.371607+0000","last_undegraded":"2024-04-25T10:17:51.371607+0000","last_fullsized":"2024-04-25T10:17:51.371607+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:03:44.824247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.375100+0000","last_change":"2024-04-25T10:17:51.375100+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.375100+0000","last_undegraded":"2024-04-25T10:17:51.375100+0000","last_fullsized":"2024-04-25T10:17:51.375100+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.383804+0000","last_change":"2024-04-25T10:17:51.382409+0000","last_active":"2024-04-25T10:17:51.383804+0000","last_peered":"2024-04-25T10:17:51.383804+0000","last_clean":"2024-04-25T10:17:51.383804+0000","last_became_active":"2024-04-25T10:17:51.382048+0000","last_became_peered":"2024-04-25T10:17:51.382048+0000","last_unstale":"2024-04-25T10:17:51.383804+0000","last_undegraded":"2024-04-25T10:17:51.383804+0000","last_fullsized":"2024-04-25T10:17:51.383804+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:14.932827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.367562+0000","last_change":"2024-04-25T10:17:51.367562+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.367562+0000","last_undegraded":"2024-04-25T10:17:51.367562+0000","last_fullsized":"2024-04-25T10:17:51.367562+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385694+0000","last_change":"2024-04-25T10:17:51.382866+0000","last_active":"2024-04-25T10:17:51.385694+0000","last_peered":"2024-04-25T10:17:51.385694+0000","last_clean":"2024-04-25T10:17:51.385694+0000","last_became_active":"2024-04-25T10:17:51.382164+0000","last_became_peered":"2024-04-25T10:17:51.382164+0000","last_unstale":"2024-04-25T10:17:51.385694+0000","last_undegraded":"2024-04-25T10:17:51.385694+0000","last_fullsized":"2024-04-25T10:17:51.385694+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.366912+0000","last_change":"2024-04-25T10:17:51.366912+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.366912+0000","last_undegraded":"2024-04-25T10:17:51.366912+0000","last_fullsized":"2024-04-25T10:17:51.366912+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.372980+0000","last_change":"2024-04-25T10:17:51.370374+0000","last_active":"2024-04-25T10:17:51.372980+0000","last_peered":"2024-04-25T10:17:51.372980+0000","last_clean":"2024-04-25T10:17:51.372980+0000","last_became_active":"2024-04-25T10:17:51.369266+0000","last_became_peered":"2024-04-25T10:17:51.369266+0000","last_unstale":"2024-04-25T10:17:51.372980+0000","last_undegraded":"2024-04-25T10:17:51.372980+0000","last_fullsized":"2024-04-25T10:17:51.372980+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:44.662844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359492+0000","last_change":"2024-04-25T10:17:51.359492+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359492+0000","last_undegraded":"2024-04-25T10:17:51.359492+0000","last_fullsized":"2024-04-25T10:17:51.359492+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.380712+0000","last_change":"2024-04-25T10:17:51.379048+0000","last_active":"2024-04-25T10:17:51.380712+0000","last_peered":"2024-04-25T10:17:51.380712+0000","last_clean":"2024-04-25T10:17:51.380712+0000","last_became_active":"2024-04-25T10:17:51.378700+0000","last_became_peered":"2024-04-25T10:17:51.378700+0000","last_unstale":"2024-04-25T10:17:51.380712+0000","last_undegraded":"2024-04-25T10:17:51.380712+0000","last_fullsized":"2024-04-25T10:17:51.380712+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:48.614198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.364985+0000","last_change":"2024-04-25T10:17:51.364985+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.364985+0000","last_undegraded":"2024-04-25T10:17:51.364985+0000","last_fullsized":"2024-04-25T10:17:51.364985+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.384589+0000","last_change":"2024-04-25T10:17:51.383501+0000","last_active":"2024-04-25T10:17:51.384589+0000","last_peered":"2024-04-25T10:17:51.384589+0000","last_clean":"2024-04-25T10:17:51.384589+0000","last_became_active":"2024-04-25T10:17:51.381248+0000","last_became_peered":"2024-04-25T10:17:51.381248+0000","last_unstale":"2024-04-25T10:17:51.384589+0000","last_undegraded":"2024-04-25T10:17:51.384589+0000","last_fullsized":"2024-04-25T10:17:51.384589+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:25:02.741516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.388545+0000","last_change":"2024-04-25T10:17:52.382153+0000","last_active":"2024-04-25T10:17:52.388545+0000","last_peered":"2024-04-25T10:17:52.388545+0000","last_clean":"2024-04-25T10:17:52.388545+0000","last_became_active":"2024-04-25T10:17:52.381688+0000","last_became_peered":"2024-04-25T10:17:52.381688+0000","last_unstale":"2024-04-25T10:17:52.388545+0000","last_undegraded":"2024-04-25T10:17:52.388545+0000","last_fullsized":"2024-04-25T10:17:52.388545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:26:53.783720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2024-04-25T10:17:52.376315+0000","last_change":"2024-04-25T10:17:51.384528+0000","last_active":"2024-04-25T10:17:52.376315+0000","last_peered":"2024-04-25T10:17:52.376315+0000","last_clean":"2024-04-25T10:17:52.376315+0000","last_became_active":"2024-04-25T10:17:51.384176+0000","last_became_peered":"2024-04-25T10:17:51.384176+0000","last_unstale":"2024-04-25T10:17:52.376315+0000","last_undegraded":"2024-04-25T10:17:52.376315+0000","last_fullsized":"2024-04-25T10:17:52.376315+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:48.896090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.371188+0000","last_change":"2024-04-25T10:17:51.371188+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.371188+0000","last_undegraded":"2024-04-25T10:17:51.371188+0000","last_fullsized":"2024-04-25T10:17:51.371188+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.368066+0000","last_change":"2024-04-25T10:17:51.366496+0000","last_active":"2024-04-25T10:17:51.368066+0000","last_peered":"2024-04-25T10:17:51.368066+0000","last_clean":"2024-04-25T10:17:51.368066+0000","last_became_active":"2024-04-25T10:17:51.364699+0000","last_became_peered":"2024-04-25T10:17:51.364699+0000","last_unstale":"2024-04-25T10:17:51.368066+0000","last_undegraded":"2024-04-25T10:17:51.368066+0000","last_fullsized":"2024-04-25T10:17:51.368066+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:51.247936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.361029+0000","last_change":"2024-04-25T10:17:51.361029+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.361029+0000","last_undegraded":"2024-04-25T10:17:51.361029+0000","last_fullsized":"2024-04-25T10:17:51.361029+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.381883+0000","last_change":"2024-04-25T10:17:51.379727+0000","last_active":"2024-04-25T10:17:51.381883+0000","last_peered":"2024-04-25T10:17:51.381883+0000","last_clean":"2024-04-25T10:17:51.381883+0000","last_became_active":"2024-04-25T10:17:51.379202+0000","last_became_peered":"2024-04-25T10:17:51.379202+0000","last_unstale":"2024-04-25T10:17:51.381883+0000","last_undegraded":"2024-04-25T10:17:51.381883+0000","last_fullsized":"2024-04-25T10:17:51.381883+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:56.752842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.359514+0000","last_change":"2024-04-25T10:17:51.359514+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.359514+0000","last_undegraded":"2024-04-25T10:17:51.359514+0000","last_fullsized":"2024-04-25T10:17:51.359514+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.385949+0000","last_change":"2024-04-25T10:17:51.384876+0000","last_active":"2024-04-25T10:17:51.385949+0000","last_peered":"2024-04-25T10:17:51.385949+0000","last_clean":"2024-04-25T10:17:51.385949+0000","last_became_active":"2024-04-25T10:17:51.384131+0000","last_became_peered":"2024-04-25T10:17:51.384131+0000","last_unstale":"2024-04-25T10:17:51.385949+0000","last_undegraded":"2024-04-25T10:17:51.385949+0000","last_fullsized":"2024-04-25T10:17:51.385949+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:44:19.091592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":1,"reported_epoch":157,"state":"creating+peering","last_fresh":"2024-04-25T10:17:51.368225+0000","last_change":"2024-04-25T10:17:51.368225+0000","last_active":"2024-04-25T10:17:51.334045+0000","last_peered":"2024-04-25T10:17:51.334045+0000","last_clean":"2024-04-25T10:17:51.334045+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:17:51.368225+0000","last_undegraded":"2024-04-25T10:17:51.368225+0000","last_fullsized":"2024-04-25T10:17:51.368225+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.373581+0000","last_change":"2024-04-25T10:17:51.371004+0000","last_active":"2024-04-25T10:17:51.373581+0000","last_peered":"2024-04-25T10:17:51.373581+0000","last_clean":"2024-04-25T10:17:51.373581+0000","last_became_active":"2024-04-25T10:17:51.370705+0000","last_became_peered":"2024-04-25T10:17:51.370705+0000","last_unstale":"2024-04-25T10:17:51.373581+0000","last_undegraded":"2024-04-25T10:17:51.373581+0000","last_fullsized":"2024-04-25T10:17:51.373581+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:11.992831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":431,"reported_epoch":157,"state":"active+clean","last_fresh":"2024-04-25T10:17:51.376627+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:17:51.376627+0000","last_peered":"2024-04-25T10:17:51.376627+0000","last_clean":"2024-04-25T10:17:51.376627+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:17:51.376627+0000","last_undegraded":"2024-04-25T10:17:51.376627+0000","last_fullsized":"2024-04-25T10:17:51.376627+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575044,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60060,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311780,"statfs":{"total":96636764160,"available":96575262720,"internally_reserved":0,"allocated":2568192,"data_stored":1516028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575044,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55320,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316520,"statfs":{"total":96636764160,"available":96580116480,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575044,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87188,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284652,"statfs":{"total":96636764160,"available":96547483648,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575044,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87192,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284648,"statfs":{"total":96636764160,"available":96547479552,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575043,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59480,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312360,"statfs":{"total":96636764160,"available":96575856640,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575044,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59476,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312364,"statfs":{"total":96636764160,"available":96575860736,"internally_reserved":0,"allocated":1974272,"data_stored":925660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575044,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47580,"kb_used_data":2508,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324260,"statfs":{"total":96636764160,"available":96588042240,"internally_reserved":0,"allocated":2568192,"data_stored":1516028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3f is not active+clean 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.368750+0000', 'last_change': '2024-04-25T10:17:52.368750+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.368750+0000', 'last_undegraded': '2024-04-25T10:17:52.368750+0000', 'last_fullsized': '2024-04-25T10:17:52.368750+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': ['4', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3e is not active+clean 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.374180+0000', 'last_change': '2024-04-25T10:17:51.374180+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.374180+0000', 'last_undegraded': '2024-04-25T10:17:51.374180+0000', 'last_fullsized': '2024-04-25T10:17:51.374180+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3c is not active+clean 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.367282+0000', 'last_change': '2024-04-25T10:17:52.367282+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.367282+0000', 'last_undegraded': '2024-04-25T10:17:52.367282+0000', 'last_fullsized': '2024-04-25T10:17:52.367282+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3b is not active+clean 2024-04-25T10:17:57.803 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.367557+0000', 'last_change': '2024-04-25T10:17:51.367557+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.367557+0000', 'last_undegraded': '2024-04-25T10:17:51.367557+0000', 'last_fullsized': '2024-04-25T10:17:51.367557+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3a is not active+clean 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.370850+0000', 'last_change': '2024-04-25T10:17:51.370850+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.370850+0000', 'last_undegraded': '2024-04-25T10:17:51.370850+0000', 'last_fullsized': '2024-04-25T10:17:51.370850+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.39 is not active+clean 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.371561+0000', 'last_change': '2024-04-25T10:17:51.371561+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.371561+0000', 'last_undegraded': '2024-04-25T10:17:51.371561+0000', 'last_fullsized': '2024-04-25T10:17:51.371561+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.38 is not active+clean 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.369985+0000', 'last_change': '2024-04-25T10:17:51.369985+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.369985+0000', 'last_undegraded': '2024-04-25T10:17:51.369985+0000', 'last_fullsized': '2024-04-25T10:17:51.369985+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.37 is not active+clean 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.361953+0000', 'last_change': '2024-04-25T10:17:51.361953+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.361953+0000', 'last_undegraded': '2024-04-25T10:17:51.361953+0000', 'last_fullsized': '2024-04-25T10:17:51.361953+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.36 is not active+clean 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.362115+0000', 'last_change': '2024-04-25T10:17:51.362115+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.362115+0000', 'last_undegraded': '2024-04-25T10:17:51.362115+0000', 'last_fullsized': '2024-04-25T10:17:51.362115+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.804 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.35 is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.373269+0000', 'last_change': '2024-04-25T10:17:51.373269+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.373269+0000', 'last_undegraded': '2024-04-25T10:17:51.373269+0000', 'last_fullsized': '2024-04-25T10:17:51.373269+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.34 is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.370995+0000', 'last_change': '2024-04-25T10:17:51.370995+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.370995+0000', 'last_undegraded': '2024-04-25T10:17:51.370995+0000', 'last_fullsized': '2024-04-25T10:17:51.370995+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.33 is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.366391+0000', 'last_change': '2024-04-25T10:17:51.366391+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.366391+0000', 'last_undegraded': '2024-04-25T10:17:51.366391+0000', 'last_fullsized': '2024-04-25T10:17:51.366391+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.32 is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.370391+0000', 'last_change': '2024-04-25T10:17:51.370391+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.370391+0000', 'last_undegraded': '2024-04-25T10:17:51.370391+0000', 'last_fullsized': '2024-04-25T10:17:51.370391+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.31 is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.371819+0000', 'last_change': '2024-04-25T10:17:51.371819+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.371819+0000', 'last_undegraded': '2024-04-25T10:17:51.371819+0000', 'last_fullsized': '2024-04-25T10:17:51.371819+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2f is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.361325+0000', 'last_change': '2024-04-25T10:17:51.361325+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.361325+0000', 'last_undegraded': '2024-04-25T10:17:51.361325+0000', 'last_fullsized': '2024-04-25T10:17:51.361325+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2e is not active+clean 2024-04-25T10:17:57.805 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2e', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.362292+0000', 'last_change': '2024-04-25T10:17:52.362292+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.362292+0000', 'last_undegraded': '2024-04-25T10:17:52.362292+0000', 'last_fullsized': '2024-04-25T10:17:52.362292+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': ['6', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2d is not active+clean 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.370262+0000', 'last_change': '2024-04-25T10:17:51.370262+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.370262+0000', 'last_undegraded': '2024-04-25T10:17:51.370262+0000', 'last_fullsized': '2024-04-25T10:17:51.370262+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2c is not active+clean 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.362679+0000', 'last_change': '2024-04-25T10:17:51.362679+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.362679+0000', 'last_undegraded': '2024-04-25T10:17:51.362679+0000', 'last_fullsized': '2024-04-25T10:17:51.362679+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2b is not active+clean 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.362767+0000', 'last_change': '2024-04-25T10:17:51.362767+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.362767+0000', 'last_undegraded': '2024-04-25T10:17:51.362767+0000', 'last_fullsized': '2024-04-25T10:17:51.362767+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2a is not active+clean 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.368522+0000', 'last_change': '2024-04-25T10:17:51.368522+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.368522+0000', 'last_undegraded': '2024-04-25T10:17:51.368522+0000', 'last_fullsized': '2024-04-25T10:17:51.368522+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.29 is not active+clean 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.370747+0000', 'last_change': '2024-04-25T10:17:51.370747+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.370747+0000', 'last_undegraded': '2024-04-25T10:17:51.370747+0000', 'last_fullsized': '2024-04-25T10:17:51.370747+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:17:57.806 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.27 is not active+clean 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.362563+0000', 'last_change': '2024-04-25T10:17:51.362563+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.362563+0000', 'last_undegraded': '2024-04-25T10:17:51.362563+0000', 'last_fullsized': '2024-04-25T10:17:51.362563+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.26 is not active+clean 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.361021+0000', 'last_change': '2024-04-25T10:17:51.361021+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.361021+0000', 'last_undegraded': '2024-04-25T10:17:51.361021+0000', 'last_fullsized': '2024-04-25T10:17:51.361021+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.25 is not active+clean 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.375238+0000', 'last_change': '2024-04-25T10:17:51.375238+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.375238+0000', 'last_undegraded': '2024-04-25T10:17:51.375238+0000', 'last_fullsized': '2024-04-25T10:17:51.375238+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.24 is not active+clean 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.361848+0000', 'last_change': '2024-04-25T10:17:51.361848+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.361848+0000', 'last_undegraded': '2024-04-25T10:17:51.361848+0000', 'last_fullsized': '2024-04-25T10:17:51.361848+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.23 is not active+clean 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.374653+0000', 'last_change': '2024-04-25T10:17:51.374653+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.374653+0000', 'last_undegraded': '2024-04-25T10:17:51.374653+0000', 'last_fullsized': '2024-04-25T10:17:51.374653+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.22 is not active+clean 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.22', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.363325+0000', 'last_change': '2024-04-25T10:17:52.363325+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.363325+0000', 'last_undegraded': '2024-04-25T10:17:52.363325+0000', 'last_fullsized': '2024-04-25T10:17:52.363325+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': ['6', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:17:57.807 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.21 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.371347+0000', 'last_change': '2024-04-25T10:17:51.371347+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.371347+0000', 'last_undegraded': '2024-04-25T10:17:51.371347+0000', 'last_fullsized': '2024-04-25T10:17:51.371347+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.20 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.20', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.361408+0000', 'last_change': '2024-04-25T10:17:52.361408+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.361408+0000', 'last_undegraded': '2024-04-25T10:17:52.361408+0000', 'last_fullsized': '2024-04-25T10:17:52.361408+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.10 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.374913+0000', 'last_change': '2024-04-25T10:17:51.374913+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.374913+0000', 'last_undegraded': '2024-04-25T10:17:51.374913+0000', 'last_fullsized': '2024-04-25T10:17:51.374913+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.11 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.360291+0000', 'last_change': '2024-04-25T10:17:51.360291+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.360291+0000', 'last_undegraded': '2024-04-25T10:17:51.360291+0000', 'last_fullsized': '2024-04-25T10:17:51.360291+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.12 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.359546+0000', 'last_change': '2024-04-25T10:17:51.359546+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.359546+0000', 'last_undegraded': '2024-04-25T10:17:51.359546+0000', 'last_fullsized': '2024-04-25T10:17:51.359546+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.13 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.370565+0000', 'last_change': '2024-04-25T10:17:51.370565+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.370565+0000', 'last_undegraded': '2024-04-25T10:17:51.370565+0000', 'last_fullsized': '2024-04-25T10:17:51.370565+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.14 is not active+clean 2024-04-25T10:17:57.808 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.358852+0000', 'last_change': '2024-04-25T10:17:51.358852+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.358852+0000', 'last_undegraded': '2024-04-25T10:17:51.358852+0000', 'last_fullsized': '2024-04-25T10:17:51.358852+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.15 is not active+clean 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.360084+0000', 'last_change': '2024-04-25T10:17:51.360084+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.360084+0000', 'last_undegraded': '2024-04-25T10:17:51.360084+0000', 'last_fullsized': '2024-04-25T10:17:51.360084+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.16 is not active+clean 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.360539+0000', 'last_change': '2024-04-25T10:17:51.360539+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.360539+0000', 'last_undegraded': '2024-04-25T10:17:51.360539+0000', 'last_fullsized': '2024-04-25T10:17:51.360539+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.17 is not active+clean 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.17', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.368384+0000', 'last_change': '2024-04-25T10:17:52.368384+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.368384+0000', 'last_undegraded': '2024-04-25T10:17:52.368384+0000', 'last_fullsized': '2024-04-25T10:17:52.368384+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': ['4', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.18 is not active+clean 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.375547+0000', 'last_change': '2024-04-25T10:17:51.375547+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.375547+0000', 'last_undegraded': '2024-04-25T10:17:51.375547+0000', 'last_fullsized': '2024-04-25T10:17:51.375547+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1a is not active+clean 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.360010+0000', 'last_change': '2024-04-25T10:17:51.360010+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.360010+0000', 'last_undegraded': '2024-04-25T10:17:51.360010+0000', 'last_fullsized': '2024-04-25T10:17:51.360010+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1b is not active+clean 2024-04-25T10:17:57.809 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.360814+0000', 'last_change': '2024-04-25T10:17:51.360814+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.360814+0000', 'last_undegraded': '2024-04-25T10:17:51.360814+0000', 'last_fullsized': '2024-04-25T10:17:51.360814+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1c is not active+clean 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1c', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.362642+0000', 'last_change': '2024-04-25T10:17:52.362642+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.362642+0000', 'last_undegraded': '2024-04-25T10:17:52.362642+0000', 'last_fullsized': '2024-04-25T10:17:52.362642+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1d is not active+clean 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.362120+0000', 'last_change': '2024-04-25T10:17:51.362120+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.362120+0000', 'last_undegraded': '2024-04-25T10:17:51.362120+0000', 'last_fullsized': '2024-04-25T10:17:51.362120+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1e is not active+clean 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.369188+0000', 'last_change': '2024-04-25T10:17:51.369188+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.369188+0000', 'last_undegraded': '2024-04-25T10:17:51.369188+0000', 'last_fullsized': '2024-04-25T10:17:51.369188+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1f is not active+clean 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.363481+0000', 'last_change': '2024-04-25T10:17:51.363481+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.363481+0000', 'last_undegraded': '2024-04-25T10:17:51.363481+0000', 'last_fullsized': '2024-04-25T10:17:51.363481+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.0 is not active+clean 2024-04-25T10:17:57.810 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.365658+0000', 'last_change': '2024-04-25T10:17:51.365658+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.365658+0000', 'last_undegraded': '2024-04-25T10:17:51.365658+0000', 'last_fullsized': '2024-04-25T10:17:51.365658+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.1 is not active+clean 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.1', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 158, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:17:52.361848+0000', 'last_change': '2024-04-25T10:17:52.361848+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:52.361848+0000', 'last_undegraded': '2024-04-25T10:17:52.361848+0000', 'last_fullsized': '2024-04-25T10:17:52.361848+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': ['6', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.2 is not active+clean 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.373721+0000', 'last_change': '2024-04-25T10:17:51.373721+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.373721+0000', 'last_undegraded': '2024-04-25T10:17:51.373721+0000', 'last_fullsized': '2024-04-25T10:17:51.373721+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.3 is not active+clean 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.366804+0000', 'last_change': '2024-04-25T10:17:51.366804+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.366804+0000', 'last_undegraded': '2024-04-25T10:17:51.366804+0000', 'last_fullsized': '2024-04-25T10:17:51.366804+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.4 is not active+clean 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.359326+0000', 'last_change': '2024-04-25T10:17:51.359326+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.359326+0000', 'last_undegraded': '2024-04-25T10:17:51.359326+0000', 'last_fullsized': '2024-04-25T10:17:51.359326+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.5 is not active+clean 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.361684+0000', 'last_change': '2024-04-25T10:17:51.361684+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.361684+0000', 'last_undegraded': '2024-04-25T10:17:51.361684+0000', 'last_fullsized': '2024-04-25T10:17:51.361684+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.6 is not active+clean 2024-04-25T10:17:57.811 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.363696+0000', 'last_change': '2024-04-25T10:17:51.363696+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.363696+0000', 'last_undegraded': '2024-04-25T10:17:51.363696+0000', 'last_fullsized': '2024-04-25T10:17:51.363696+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.7 is not active+clean 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.375100+0000', 'last_change': '2024-04-25T10:17:51.375100+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.375100+0000', 'last_undegraded': '2024-04-25T10:17:51.375100+0000', 'last_fullsized': '2024-04-25T10:17:51.375100+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.8 is not active+clean 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.367562+0000', 'last_change': '2024-04-25T10:17:51.367562+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.367562+0000', 'last_undegraded': '2024-04-25T10:17:51.367562+0000', 'last_fullsized': '2024-04-25T10:17:51.367562+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.9 is not active+clean 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.366912+0000', 'last_change': '2024-04-25T10:17:51.366912+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.366912+0000', 'last_undegraded': '2024-04-25T10:17:51.366912+0000', 'last_fullsized': '2024-04-25T10:17:51.366912+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.a is not active+clean 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.359492+0000', 'last_change': '2024-04-25T10:17:51.359492+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.359492+0000', 'last_undegraded': '2024-04-25T10:17:51.359492+0000', 'last_fullsized': '2024-04-25T10:17:51.359492+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.b is not active+clean 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.364985+0000', 'last_change': '2024-04-25T10:17:51.364985+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.364985+0000', 'last_undegraded': '2024-04-25T10:17:51.364985+0000', 'last_fullsized': '2024-04-25T10:17:51.364985+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:17:57.812 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.d is not active+clean 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.371188+0000', 'last_change': '2024-04-25T10:17:51.371188+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.371188+0000', 'last_undegraded': '2024-04-25T10:17:51.371188+0000', 'last_fullsized': '2024-04-25T10:17:51.371188+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.e is not active+clean 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.361029+0000', 'last_change': '2024-04-25T10:17:51.361029+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.361029+0000', 'last_undegraded': '2024-04-25T10:17:51.361029+0000', 'last_fullsized': '2024-04-25T10:17:51.361029+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.f is not active+clean 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.359514+0000', 'last_change': '2024-04-25T10:17:51.359514+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.359514+0000', 'last_undegraded': '2024-04-25T10:17:51.359514+0000', 'last_fullsized': '2024-04-25T10:17:51.359514+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:PG 24.19 is not active+clean 2024-04-25T10:17:57.813 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '24.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 157, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:17:51.368225+0000', 'last_change': '2024-04-25T10:17:51.368225+0000', 'last_active': '2024-04-25T10:17:51.334045+0000', 'last_peered': '2024-04-25T10:17:51.334045+0000', 'last_clean': '2024-04-25T10:17:51.334045+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:17:51.368225+0000', 'last_undegraded': '2024-04-25T10:17:51.368225+0000', 'last_fullsized': '2024-04-25T10:17:51.368225+0000', 'mapping_epoch': 157, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 157, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'last_clean_scrub_stamp': '2024-04-25T10:17:51.334045+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:17:57.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:17:58.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.045+0000 7f1db1a00640 1 Processor -- start 2024-04-25T10:17:58.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.045+0000 7f1db1a00640 1 -- start start 2024-04-25T10:17:58.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.047+0000 7f1db1a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1dac154ee0 0x7f1dac0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.047+0000 7f1db1a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1dac0cfcf0 0x7f1dac0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.047+0000 7f1db1a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1dac071480 con 0x7f1dac0ce3c0 2024-04-25T10:17:58.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.047+0000 7f1db1a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac0ce830 con 0x7f1dac154ee0 2024-04-25T10:17:58.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.047+0000 7f1db1a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1dac072090 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.047+0000 7f1daaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1dac154ee0 0x7f1dac0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1daaa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1dac154ee0 0x7f1dac0cf7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39726/0 (socket says 172.21.15.165:39726) 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1daaa00640 1 -- 172.21.15.165:0/4044736287 learned_addr learned my addr 172.21.15.165:0/4044736287 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1da2000640 1 --2- 172.21.15.165:0/4044736287 >> v2:172.21.15.172:3300/0 conn(0x7f1dac0cfcf0 0x7f1dac0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1daaa00640 1 -- 172.21.15.165:0/4044736287 >> v2:172.21.15.172:3300/0 conn(0x7f1dac0cfcf0 msgr2=0x7f1dac0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1daaa00640 1 --2- 172.21.15.165:0/4044736287 >> v2:172.21.15.172:3300/0 conn(0x7f1dac0cfcf0 0x7f1dac0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1daaa00640 1 -- 172.21.15.165:0/4044736287 >> v1:172.21.15.172:6789/0 conn(0x7f1dac0ce3c0 legacy=0x7f1dac15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.048+0000 7f1daaa00640 1 -- 172.21.15.165:0/4044736287 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac071760 con 0x7f1dac154ee0 2024-04-25T10:17:58.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.049+0000 7f1daaa00640 1 --2- 172.21.15.165:0/4044736287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac154ee0 0x7f1dac0cf7b0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0cf560 tx=0x7f1d9c02c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35826e813674fed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.049+0000 7f1daa000640 1 -- 172.21.15.165:0/4044736287 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c0325a0 con 0x7f1dac154ee0 2024-04-25T10:17:58.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.049+0000 7f1daa000640 1 -- 172.21.15.165:0/4044736287 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9c038070 con 0x7f1dac154ee0 2024-04-25T10:17:58.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.049+0000 7f1daa000640 1 -- 172.21.15.165:0/4044736287 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9c032960 con 0x7f1dac154ee0 2024-04-25T10:17:58.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 -- 172.21.15.165:0/4044736287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac154ee0 msgr2=0x7f1dac0cf7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 --2- 172.21.15.165:0/4044736287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac154ee0 0x7f1dac0cf7b0 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7f1dac0cf560 tx=0x7f1d9c02c8e0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 -- 172.21.15.165:0/4044736287 shutdown_connections 2024-04-25T10:17:58.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 --2- 172.21.15.165:0/4044736287 >> v2:172.21.15.172:3300/0 conn(0x7f1dac0cfcf0 0x7f1dac0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 --2- 172.21.15.165:0/4044736287 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac154ee0 0x7f1dac0cf7b0 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 -- 172.21.15.165:0/4044736287 >> 172.21.15.165:0/4044736287 conn(0x7f1dac06abe0 msgr2=0x7f1dac06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 -- 172.21.15.165:0/4044736287 shutdown_connections 2024-04-25T10:17:58.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.050+0000 7f1db1a00640 1 -- 172.21.15.165:0/4044736287 wait complete. 2024-04-25T10:17:58.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.052+0000 7f1db1a00640 1 Processor -- start 2024-04-25T10:17:58.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.052+0000 7f1db1a00640 1 -- start start 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1db1a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1dac0ce3c0 0x7f1dac13ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1db1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 0x7f1dac142d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1db1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dac154ee0 0x7f1dac140cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1db1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac071d20 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1db1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac071230 con 0x7f1dac154ee0 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1db1a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dac1552c0 con 0x7f1dac0ce3c0 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1da2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 0x7f1dac142d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1da2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 0x7f1dac142d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39734/0 (socket says 172.21.15.165:39734) 2024-04-25T10:17:58.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.053+0000 7f1da2000640 1 -- 172.21.15.165:0/2144257176 learned_addr learned my addr 172.21.15.165:0/2144257176 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1da2000640 1 -- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1dac0ce3c0 msgr2=0x7f1dac13ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1daaa00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1dac0ce3c0 0x7f1dac13ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1dab400640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dac154ee0 0x7f1dac140cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1da2000640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1dac0ce3c0 0x7f1dac13ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1da2000640 1 -- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dac154ee0 msgr2=0x7f1dac140cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1da2000640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dac154ee0 0x7f1dac140cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1da2000640 1 -- 172.21.15.165:0/2144257176 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dac13fc20 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.054+0000 7f1da2000640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 0x7f1dac142d30 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400d830 tx=0x7f1d9400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.055+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400b660 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.055+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d94019070 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.055+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dac13ff00 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.055+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9400ba20 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.055+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dac1404c0 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.056+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dac073990 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.057+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d9401e460 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.058+0000 7f1da3400640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d8806b4d0 0x7f1d8806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.058+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1d9408aab0 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.058+0000 7f1daaa00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d8806b4d0 0x7f1d8806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.059+0000 7f1daaa00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d8806b4d0 0x7f1d8806d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c037000 tx=0x7f1d9c032000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.062+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d94059700 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.327 INFO:tasks.ceph.mds.d.smithi172.stderr:2024-04-25T10:17:58.324+0000 7fd701a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:17:58.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.326+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f1dac06d360 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.326+0000 7f1da3400640 1 -- 172.21.15.165:0/2144257176 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1438 (secure 0 0 0) 0x7f1d94059000 con 0x7f1dac0cfcf0 2024-04-25T10:17:58.329 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:58.330 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":979,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":159,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":52},{"state_name":"active+clean","count":37},{"state_name":"creating+activating","count":8}],"num_pgs":97,"num_pools":3,"num_objects":6,"data_bytes":591040,"bytes_used":528154624,"bytes_avail":772565958656,"bytes_total":773094113280,"inactive_pgs_ratio":0.6185566782951355,"write_bytes_sec":341,"read_op_per_sec":0,"write_op_per_sec":1},"fsmap":{"epoch":113,"id":11,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":11,"rank":0,"name":"d","status":"up:active","gid":6742}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":37,"modified":"2024-04-25T10:17:46.451001+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d8806b4d0 msgr2=0x7f1d8806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d8806b4d0 0x7f1d8806d990 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c037000 tx=0x7f1d9c032000 comp rx=0 tx=0).stop 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 msgr2=0x7f1dac142d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 0x7f1dac142d30 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7f1d9400d830 tx=0x7f1d9400dcf0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 shutdown_connections 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d8806b4d0 0x7f1d8806d990 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1dac154ee0 0x7f1dac140cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1dac0cfcf0 0x7f1dac142d30 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 --2- 172.21.15.165:0/2144257176 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1dac0ce3c0 0x7f1dac13ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 >> 172.21.15.165:0/2144257176 conn(0x7f1dac06abe0 msgr2=0x7f1dac153730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 shutdown_connections 2024-04-25T10:17:58.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.329+0000 7f1db1a00640 1 -- 172.21.15.165:0/2144257176 wait complete. 2024-04-25T10:17:58.349 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:17:58.349 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:17:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.544+0000 7f16f5200640 1 Processor -- start 2024-04-25T10:17:58.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.544+0000 7f16f5200640 1 -- start start 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16f5200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f16f00ce920 0x7f16f00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16f5200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f0144b30 con 0x7f16f006d290 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16f5200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f00712d0 con 0x7f16f0144750 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16f5200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f00715b0 con 0x7f16f00ce920 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16ee800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f16f006d290 0x7f16f006d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42208/0 (socket says 172.21.15.165:42208) 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16ee800640 1 -- 172.21.15.165:0/973236912 learned_addr learned my addr 172.21.15.165:0/973236912 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:17:58.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16ed400640 1 --2- 172.21.15.165:0/973236912 >> v2:172.21.15.172:3300/0 conn(0x7f16f00ce920 0x7f16f00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4091377626 0 0) 0x7f16f0144b30 con 0x7f16f006d290 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.546+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16d4003520 con 0x7f16f006d290 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107154282 0 0) 0x7f16d4003520 con 0x7f16f006d290 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 >> v2:172.21.15.172:3300/0 conn(0x7f16f00ce920 msgr2=0x7f16f00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 --2- 172.21.15.165:0/973236912 >> v2:172.21.15.172:3300/0 conn(0x7f16f00ce920 0x7f16f00cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 >> v1:172.21.15.172:6789/0 conn(0x7f16f0144750 legacy=0x7f16f006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f0071890 con 0x7f16f006d290 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f16e80019f0 con 0x7f16f006d290 2024-04-25T10:17:58.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 >> v1:172.21.15.165:6789/0 conn(0x7f16f006d290 legacy=0x7f16f006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40039b0 0x7f16d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f00715b0 con 0x7f16d40051c0 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d400c060 con 0x7f16d4004800 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d400c260 con 0x7f16d40039b0 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.547+0000 7f16ee800640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ede00640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40039b0 0x7f16d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ed400640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ee800640 1 -- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40039b0 msgr2=0x7f16d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ee800640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d40039b0 0x7f16d4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ee800640 1 -- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4004800 msgr2=0x7f16d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ee800640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ee800640 1 -- 172.21.15.165:0/973236912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d4003520 con 0x7f16d40051c0 2024-04-25T10:17:58.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.548+0000 7f16ed400640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16d4004800 0x7f16d4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:17:58.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16ee800640 1 -- 172.21.15.165:0/973236912 reap_dead start 2024-04-25T10:17:58.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16ee800640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f16e800a860 tx=0x7f16d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f188c592d0fbdc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800a480 con 0x7f16d40051c0 2024-04-25T10:17:58.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e802f070 con 0x7f16d40051c0 2024-04-25T10:17:58.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16eca00640 1 -- 172.21.15.165:0/973236912 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e802bca0 con 0x7f16d40051c0 2024-04-25T10:17:58.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16f5200640 1 -- 172.21.15.165:0/973236912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 msgr2=0x7f16d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.549+0000 7f16f5200640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f16e800a860 tx=0x7f16d4005830 comp rx=0 tx=0).stop 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.550+0000 7f16f5200640 1 -- 172.21.15.165:0/973236912 shutdown_connections 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.550+0000 7f16f5200640 1 --2- 172.21.15.165:0/973236912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16d400b9c0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.550+0000 7f16f5200640 1 -- 172.21.15.165:0/973236912 >> 172.21.15.165:0/973236912 conn(0x7f16f006abe0 msgr2=0x7f16f006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.550+0000 7f16f5200640 1 -- 172.21.15.165:0/973236912 shutdown_connections 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.550+0000 7f16f5200640 1 -- 172.21.15.165:0/973236912 wait complete. 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.551+0000 7f16f5200640 1 Processor -- start 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.551+0000 7f16f5200640 1 -- start start 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.552+0000 7f16f5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16f013f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.552+0000 7f16f5200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16f00a94f0 0x7f16f015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.552+0000 7f16f5200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16f015af90 0x7f16f015b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.552+0000 7f16f5200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f01455d0 con 0x7f16d40051c0 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.552+0000 7f16f5200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f0145450 con 0x7f16f015af90 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.552+0000 7f16f5200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f015c000 con 0x7f16f00a94f0 2024-04-25T10:17:58.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16f013f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16f013f330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39754/0 (socket says 172.21.15.165:39754) 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 -- 172.21.15.165:0/1997647845 learned_addr learned my addr 172.21.15.165:0/1997647845 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 -- 172.21.15.165:0/1997647845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16f00a94f0 msgr2=0x7f16f015e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16f00a94f0 0x7f16f015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 -- 172.21.15.165:0/1997647845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16f015af90 msgr2=0x7f16f015b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16f015af90 0x7f16f015b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 -- 172.21.15.165:0/1997647845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f00a9c90 con 0x7f16d40051c0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.553+0000 7f16ede00640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16f013f330 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f16e000e4d0 tx=0x7f16e000e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.554+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e00052e0 con 0x7f16d40051c0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.554+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e0018070 con 0x7f16d40051c0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.554+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e0003810 con 0x7f16d40051c0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.554+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16f00a9f70 con 0x7f16d40051c0 2024-04-25T10:17:58.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.554+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f012fec0 con 0x7f16d40051c0 2024-04-25T10:17:58.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.555+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f00aa490 con 0x7f16d40051c0 2024-04-25T10:17:58.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.556+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16e00039b0 con 0x7f16d40051c0 2024-04-25T10:17:58.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.557+0000 7f16e6000640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d006b500 0x7f16d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:17:58.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.557+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f16e008a060 con 0x7f16d40051c0 2024-04-25T10:17:58.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.558+0000 7f16ed400640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d006b500 0x7f16d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:17:58.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.558+0000 7f16ed400640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d006b500 0x7f16d006d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16f00af130 tx=0x7f16dc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:17:58.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.562+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16f00aa490 con 0x7f16d40051c0 2024-04-25T10:17:58.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.753+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16f00685e0 con 0x7f16d006b500 2024-04-25T10:17:58.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.771+0000 7f16e6000640 1 -- 172.21.15.165:0/1997647845 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f16f00685e0 con 0x7f16d006b500 2024-04-25T10:17:58.777 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:17:58.779 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d006b500 msgr2=0x7f16d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d006b500 0x7f16d006d9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f16f00af130 tx=0x7f16dc008040 comp rx=0 tx=0).stop 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 msgr2=0x7f16f013f330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16f013f330 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f16e000e4d0 tx=0x7f16e000e990 comp rx=0 tx=0).stop 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 shutdown_connections 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16d006b500 0x7f16d006d9c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16f015af90 0x7f16f015b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16f00a94f0 0x7f16f015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 --2- 172.21.15.165:0/1997647845 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d40051c0 0x7f16f013f330 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:17:58.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 >> 172.21.15.165:0/1997647845 conn(0x7f16f006abe0 msgr2=0x7f16f00ced90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:17:58.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 shutdown_connections 2024-04-25T10:17:58.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:17:58.778+0000 7f16f5200640 1 -- 172.21.15.165:0/1997647845 wait complete. 2024-04-25T10:17:58.800 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":635,"stamp":"2024-04-25T10:17:58.453771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":516044,"kb_used_data":16840,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754458676,"statfs":{"total":773094113280,"available":772565684224,"internally_reserved":0,"allocated":17244160,"data_stored":8701708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.101609"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396515+0000","last_change":"2024-04-25T10:17:52.392786+0000","last_active":"2024-04-25T10:17:53.396515+0000","last_peered":"2024-04-25T10:17:53.396515+0000","last_clean":"2024-04-25T10:17:53.396515+0000","last_became_active":"2024-04-25T10:17:52.391070+0000","last_became_peered":"2024-04-25T10:17:52.391070+0000","last_unstale":"2024-04-25T10:17:53.396515+0000","last_undegraded":"2024-04-25T10:17:53.396515+0000","last_fullsized":"2024-04-25T10:17:53.396515+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:36.131072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960367+0000","last_change":"2024-04-25T10:17:52.398361+0000","last_active":"2024-04-25T10:17:53.960367+0000","last_peered":"2024-04-25T10:17:53.960367+0000","last_clean":"2024-04-25T10:17:53.960367+0000","last_became_active":"2024-04-25T10:17:52.397707+0000","last_became_peered":"2024-04-25T10:17:52.397707+0000","last_unstale":"2024-04-25T10:17:53.960367+0000","last_undegraded":"2024-04-25T10:17:53.960367+0000","last_fullsized":"2024-04-25T10:17:53.960367+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:39.318816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396658+0000","last_change":"2024-04-25T10:17:52.377620+0000","last_active":"2024-04-25T10:17:53.396658+0000","last_peered":"2024-04-25T10:17:53.396658+0000","last_clean":"2024-04-25T10:17:53.396658+0000","last_became_active":"2024-04-25T10:17:52.377333+0000","last_became_peered":"2024-04-25T10:17:52.377333+0000","last_unstale":"2024-04-25T10:17:53.396658+0000","last_undegraded":"2024-04-25T10:17:53.396658+0000","last_fullsized":"2024-04-25T10:17:53.396658+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:09.616475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396157+0000","last_change":"2024-04-25T10:17:52.400656+0000","last_active":"2024-04-25T10:17:53.396157+0000","last_peered":"2024-04-25T10:17:53.396157+0000","last_clean":"2024-04-25T10:17:53.396157+0000","last_became_active":"2024-04-25T10:17:52.400364+0000","last_became_peered":"2024-04-25T10:17:52.400364+0000","last_unstale":"2024-04-25T10:17:53.396157+0000","last_undegraded":"2024-04-25T10:17:53.396157+0000","last_fullsized":"2024-04-25T10:17:53.396157+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:22.633919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392545+0000","last_change":"2024-04-25T10:17:52.394908+0000","last_active":"2024-04-25T10:17:53.392545+0000","last_peered":"2024-04-25T10:17:53.392545+0000","last_clean":"2024-04-25T10:17:53.392545+0000","last_became_active":"2024-04-25T10:17:52.394618+0000","last_became_peered":"2024-04-25T10:17:52.394618+0000","last_unstale":"2024-04-25T10:17:53.392545+0000","last_undegraded":"2024-04-25T10:17:53.392545+0000","last_fullsized":"2024-04-25T10:17:53.392545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:48.887720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959221+0000","last_change":"2024-04-25T10:17:52.375197+0000","last_active":"2024-04-25T10:17:53.959221+0000","last_peered":"2024-04-25T10:17:53.959221+0000","last_clean":"2024-04-25T10:17:53.959221+0000","last_became_active":"2024-04-25T10:17:52.374283+0000","last_became_peered":"2024-04-25T10:17:52.374283+0000","last_unstale":"2024-04-25T10:17:53.959221+0000","last_undegraded":"2024-04-25T10:17:53.959221+0000","last_fullsized":"2024-04-25T10:17:53.959221+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:49.631586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958623+0000","last_change":"2024-04-25T10:17:52.382788+0000","last_active":"2024-04-25T10:17:53.958623+0000","last_peered":"2024-04-25T10:17:53.958623+0000","last_clean":"2024-04-25T10:17:53.958623+0000","last_became_active":"2024-04-25T10:17:52.382509+0000","last_became_peered":"2024-04-25T10:17:52.382509+0000","last_unstale":"2024-04-25T10:17:53.958623+0000","last_undegraded":"2024-04-25T10:17:53.958623+0000","last_fullsized":"2024-04-25T10:17:53.958623+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:53.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958109+0000","last_change":"2024-04-25T10:17:52.382408+0000","last_active":"2024-04-25T10:17:53.958109+0000","last_peered":"2024-04-25T10:17:53.958109+0000","last_clean":"2024-04-25T10:17:53.958109+0000","last_became_active":"2024-04-25T10:17:52.378938+0000","last_became_peered":"2024-04-25T10:17:52.378938+0000","last_unstale":"2024-04-25T10:17:53.958109+0000","last_undegraded":"2024-04-25T10:17:53.958109+0000","last_fullsized":"2024-04-25T10:17:53.958109+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:10.407438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958663+0000","last_change":"2024-04-25T10:17:52.385684+0000","last_active":"2024-04-25T10:17:53.958663+0000","last_peered":"2024-04-25T10:17:53.958663+0000","last_clean":"2024-04-25T10:17:53.958663+0000","last_became_active":"2024-04-25T10:17:52.385384+0000","last_became_peered":"2024-04-25T10:17:52.385384+0000","last_unstale":"2024-04-25T10:17:53.958663+0000","last_undegraded":"2024-04-25T10:17:53.958663+0000","last_fullsized":"2024-04-25T10:17:53.958663+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:16.856105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959997+0000","last_change":"2024-04-25T10:17:52.380579+0000","last_active":"2024-04-25T10:17:53.959997+0000","last_peered":"2024-04-25T10:17:53.959997+0000","last_clean":"2024-04-25T10:17:53.959997+0000","last_became_active":"2024-04-25T10:17:52.380240+0000","last_became_peered":"2024-04-25T10:17:52.380240+0000","last_unstale":"2024-04-25T10:17:53.959997+0000","last_undegraded":"2024-04-25T10:17:53.959997+0000","last_fullsized":"2024-04-25T10:17:53.959997+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:45.680232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680800+0000","last_change":"2024-04-25T10:17:52.402123+0000","last_active":"2024-04-25T10:17:53.680800+0000","last_peered":"2024-04-25T10:17:53.680800+0000","last_clean":"2024-04-25T10:17:53.680800+0000","last_became_active":"2024-04-25T10:17:52.401791+0000","last_became_peered":"2024-04-25T10:17:52.401791+0000","last_unstale":"2024-04-25T10:17:53.680800+0000","last_undegraded":"2024-04-25T10:17:53.680800+0000","last_fullsized":"2024-04-25T10:17:53.680800+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:34.714664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959631+0000","last_change":"2024-04-25T10:17:52.376217+0000","last_active":"2024-04-25T10:17:53.959631+0000","last_peered":"2024-04-25T10:17:53.959631+0000","last_clean":"2024-04-25T10:17:53.959631+0000","last_became_active":"2024-04-25T10:17:52.375941+0000","last_became_peered":"2024-04-25T10:17:52.375941+0000","last_unstale":"2024-04-25T10:17:53.959631+0000","last_undegraded":"2024-04-25T10:17:53.959631+0000","last_fullsized":"2024-04-25T10:17:53.959631+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:21.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960777+0000","last_change":"2024-04-25T10:17:52.390871+0000","last_active":"2024-04-25T10:17:53.960777+0000","last_peered":"2024-04-25T10:17:53.960777+0000","last_clean":"2024-04-25T10:17:53.960777+0000","last_became_active":"2024-04-25T10:17:52.390483+0000","last_became_peered":"2024-04-25T10:17:52.390483+0000","last_unstale":"2024-04-25T10:17:53.960777+0000","last_undegraded":"2024-04-25T10:17:53.960777+0000","last_fullsized":"2024-04-25T10:17:53.960777+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:50.628598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391921+0000","last_change":"2024-04-25T10:17:52.390116+0000","last_active":"2024-04-25T10:17:53.391921+0000","last_peered":"2024-04-25T10:17:53.391921+0000","last_clean":"2024-04-25T10:17:53.391921+0000","last_became_active":"2024-04-25T10:17:52.389817+0000","last_became_peered":"2024-04-25T10:17:52.389817+0000","last_unstale":"2024-04-25T10:17:53.391921+0000","last_undegraded":"2024-04-25T10:17:53.391921+0000","last_fullsized":"2024-04-25T10:17:53.391921+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:03.643179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679747+0000","last_change":"2024-04-25T10:17:52.397051+0000","last_active":"2024-04-25T10:17:53.679747+0000","last_peered":"2024-04-25T10:17:53.679747+0000","last_clean":"2024-04-25T10:17:53.679747+0000","last_became_active":"2024-04-25T10:17:52.393600+0000","last_became_peered":"2024-04-25T10:17:52.393600+0000","last_unstale":"2024-04-25T10:17:53.679747+0000","last_undegraded":"2024-04-25T10:17:53.679747+0000","last_fullsized":"2024-04-25T10:17:53.679747+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:49.917637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396070+0000","last_change":"2024-04-25T10:17:52.385023+0000","last_active":"2024-04-25T10:17:53.396070+0000","last_peered":"2024-04-25T10:17:53.396070+0000","last_clean":"2024-04-25T10:17:53.396070+0000","last_became_active":"2024-04-25T10:17:52.384157+0000","last_became_peered":"2024-04-25T10:17:52.384157+0000","last_unstale":"2024-04-25T10:17:53.396070+0000","last_undegraded":"2024-04-25T10:17:53.396070+0000","last_fullsized":"2024-04-25T10:17:53.396070+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:08.834397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959105+0000","last_change":"2024-04-25T10:17:52.397097+0000","last_active":"2024-04-25T10:17:53.959105+0000","last_peered":"2024-04-25T10:17:53.959105+0000","last_clean":"2024-04-25T10:17:53.959105+0000","last_became_active":"2024-04-25T10:17:52.396832+0000","last_became_peered":"2024-04-25T10:17:52.396832+0000","last_unstale":"2024-04-25T10:17:53.959105+0000","last_undegraded":"2024-04-25T10:17:53.959105+0000","last_fullsized":"2024-04-25T10:17:53.959105+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:34.714664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.398174+0000","last_change":"2024-04-25T10:17:52.382934+0000","last_active":"2024-04-25T10:17:53.398174+0000","last_peered":"2024-04-25T10:17:53.398174+0000","last_clean":"2024-04-25T10:17:53.398174+0000","last_became_active":"2024-04-25T10:17:52.382648+0000","last_became_peered":"2024-04-25T10:17:52.382648+0000","last_unstale":"2024-04-25T10:17:53.398174+0000","last_undegraded":"2024-04-25T10:17:53.398174+0000","last_fullsized":"2024-04-25T10:17:53.398174+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:22.633919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680800+0000","last_change":"2024-04-25T10:17:52.395971+0000","last_active":"2024-04-25T10:17:53.680800+0000","last_peered":"2024-04-25T10:17:53.680800+0000","last_clean":"2024-04-25T10:17:53.680800+0000","last_became_active":"2024-04-25T10:17:52.395687+0000","last_became_peered":"2024-04-25T10:17:52.395687+0000","last_unstale":"2024-04-25T10:17:53.680800+0000","last_undegraded":"2024-04-25T10:17:53.680800+0000","last_fullsized":"2024-04-25T10:17:53.680800+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:52.265324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962442+0000","last_change":"2024-04-25T10:17:52.386081+0000","last_active":"2024-04-25T10:17:53.962442+0000","last_peered":"2024-04-25T10:17:53.962442+0000","last_clean":"2024-04-25T10:17:53.962442+0000","last_became_active":"2024-04-25T10:17:52.385803+0000","last_became_peered":"2024-04-25T10:17:52.385803+0000","last_unstale":"2024-04-25T10:17:53.962442+0000","last_undegraded":"2024-04-25T10:17:53.962442+0000","last_fullsized":"2024-04-25T10:17:53.962442+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:48.657580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960949+0000","last_change":"2024-04-25T10:17:52.381445+0000","last_active":"2024-04-25T10:17:53.960949+0000","last_peered":"2024-04-25T10:17:53.960949+0000","last_clean":"2024-04-25T10:17:53.960949+0000","last_became_active":"2024-04-25T10:17:52.381059+0000","last_became_peered":"2024-04-25T10:17:52.381059+0000","last_unstale":"2024-04-25T10:17:53.960949+0000","last_undegraded":"2024-04-25T10:17:53.960949+0000","last_fullsized":"2024-04-25T10:17:53.960949+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:29.040869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959691+0000","last_change":"2024-04-25T10:17:52.382845+0000","last_active":"2024-04-25T10:17:53.959691+0000","last_peered":"2024-04-25T10:17:53.959691+0000","last_clean":"2024-04-25T10:17:53.959691+0000","last_became_active":"2024-04-25T10:17:52.379000+0000","last_became_peered":"2024-04-25T10:17:52.379000+0000","last_unstale":"2024-04-25T10:17:53.959691+0000","last_undegraded":"2024-04-25T10:17:53.959691+0000","last_fullsized":"2024-04-25T10:17:53.959691+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:05.310910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391492+0000","last_change":"2024-04-25T10:17:52.379517+0000","last_active":"2024-04-25T10:17:53.391492+0000","last_peered":"2024-04-25T10:17:53.391492+0000","last_clean":"2024-04-25T10:17:53.391492+0000","last_became_active":"2024-04-25T10:17:52.379262+0000","last_became_peered":"2024-04-25T10:17:52.379262+0000","last_unstale":"2024-04-25T10:17:53.391492+0000","last_undegraded":"2024-04-25T10:17:53.391492+0000","last_fullsized":"2024-04-25T10:17:53.391492+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:39.027010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396411+0000","last_change":"2024-04-25T10:17:52.385243+0000","last_active":"2024-04-25T10:17:53.396411+0000","last_peered":"2024-04-25T10:17:53.396411+0000","last_clean":"2024-04-25T10:17:53.396411+0000","last_became_active":"2024-04-25T10:17:52.384636+0000","last_became_peered":"2024-04-25T10:17:52.384636+0000","last_unstale":"2024-04-25T10:17:53.396411+0000","last_undegraded":"2024-04-25T10:17:53.396411+0000","last_fullsized":"2024-04-25T10:17:53.396411+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:32.242017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961539+0000","last_change":"2024-04-25T10:17:52.398473+0000","last_active":"2024-04-25T10:17:53.961539+0000","last_peered":"2024-04-25T10:17:53.961539+0000","last_clean":"2024-04-25T10:17:53.961539+0000","last_became_active":"2024-04-25T10:17:52.397957+0000","last_became_peered":"2024-04-25T10:17:52.397957+0000","last_unstale":"2024-04-25T10:17:53.961539+0000","last_undegraded":"2024-04-25T10:17:53.961539+0000","last_fullsized":"2024-04-25T10:17:53.961539+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:21.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961737+0000","last_change":"2024-04-25T10:17:52.371633+0000","last_active":"2024-04-25T10:17:53.961737+0000","last_peered":"2024-04-25T10:17:53.961737+0000","last_clean":"2024-04-25T10:17:53.961737+0000","last_became_active":"2024-04-25T10:17:52.371270+0000","last_became_peered":"2024-04-25T10:17:52.371270+0000","last_unstale":"2024-04-25T10:17:53.961737+0000","last_undegraded":"2024-04-25T10:17:53.961737+0000","last_fullsized":"2024-04-25T10:17:53.961737+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:24.692043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959668+0000","last_change":"2024-04-25T10:17:52.381238+0000","last_active":"2024-04-25T10:17:53.959668+0000","last_peered":"2024-04-25T10:17:53.959668+0000","last_clean":"2024-04-25T10:17:53.959668+0000","last_became_active":"2024-04-25T10:17:52.379585+0000","last_became_peered":"2024-04-25T10:17:52.379585+0000","last_unstale":"2024-04-25T10:17:53.959668+0000","last_undegraded":"2024-04-25T10:17:53.959668+0000","last_fullsized":"2024-04-25T10:17:53.959668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:18.040469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961595+0000","last_change":"2024-04-25T10:17:52.382150+0000","last_active":"2024-04-25T10:17:53.961595+0000","last_peered":"2024-04-25T10:17:53.961595+0000","last_clean":"2024-04-25T10:17:53.961595+0000","last_became_active":"2024-04-25T10:17:52.381896+0000","last_became_peered":"2024-04-25T10:17:52.381896+0000","last_unstale":"2024-04-25T10:17:53.961595+0000","last_undegraded":"2024-04-25T10:17:53.961595+0000","last_fullsized":"2024-04-25T10:17:53.961595+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.277448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960114+0000","last_change":"2024-04-25T10:17:52.385743+0000","last_active":"2024-04-25T10:17:53.960114+0000","last_peered":"2024-04-25T10:17:53.960114+0000","last_clean":"2024-04-25T10:17:53.960114+0000","last_became_active":"2024-04-25T10:17:52.385354+0000","last_became_peered":"2024-04-25T10:17:52.385354+0000","last_unstale":"2024-04-25T10:17:53.960114+0000","last_undegraded":"2024-04-25T10:17:53.960114+0000","last_fullsized":"2024-04-25T10:17:53.960114+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:50.427063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.398316+0000","last_change":"2024-04-25T10:17:52.380488+0000","last_active":"2024-04-25T10:17:53.398316+0000","last_peered":"2024-04-25T10:17:53.398316+0000","last_clean":"2024-04-25T10:17:53.398316+0000","last_became_active":"2024-04-25T10:17:52.380194+0000","last_became_peered":"2024-04-25T10:17:52.380194+0000","last_unstale":"2024-04-25T10:17:53.398316+0000","last_undegraded":"2024-04-25T10:17:53.398316+0000","last_fullsized":"2024-04-25T10:17:53.398316+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:36.131072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679976+0000","last_change":"2024-04-25T10:17:52.389838+0000","last_active":"2024-04-25T10:17:53.679976+0000","last_peered":"2024-04-25T10:17:53.679976+0000","last_clean":"2024-04-25T10:17:53.679976+0000","last_became_active":"2024-04-25T10:17:52.389529+0000","last_became_peered":"2024-04-25T10:17:52.389529+0000","last_unstale":"2024-04-25T10:17:53.679976+0000","last_undegraded":"2024-04-25T10:17:53.679976+0000","last_fullsized":"2024-04-25T10:17:53.679976+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:16.856105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.397840+0000","last_change":"2024-04-25T10:17:52.385102+0000","last_active":"2024-04-25T10:17:53.397840+0000","last_peered":"2024-04-25T10:17:53.397840+0000","last_clean":"2024-04-25T10:17:53.397840+0000","last_became_active":"2024-04-25T10:17:52.384801+0000","last_became_peered":"2024-04-25T10:17:52.384801+0000","last_unstale":"2024-04-25T10:17:53.397840+0000","last_undegraded":"2024-04-25T10:17:53.397840+0000","last_fullsized":"2024-04-25T10:17:53.397840+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:30.844244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959378+0000","last_change":"2024-04-25T10:17:52.381030+0000","last_active":"2024-04-25T10:17:53.959378+0000","last_peered":"2024-04-25T10:17:53.959378+0000","last_clean":"2024-04-25T10:17:53.959378+0000","last_became_active":"2024-04-25T10:17:52.380511+0000","last_became_peered":"2024-04-25T10:17:52.380511+0000","last_unstale":"2024-04-25T10:17:53.959378+0000","last_undegraded":"2024-04-25T10:17:53.959378+0000","last_fullsized":"2024-04-25T10:17:53.959378+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:50.315964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396726+0000","last_change":"2024-04-25T10:17:51.378796+0000","last_active":"2024-04-25T10:17:53.396726+0000","last_peered":"2024-04-25T10:17:53.396726+0000","last_clean":"2024-04-25T10:17:53.396726+0000","last_became_active":"2024-04-25T10:17:51.378504+0000","last_became_peered":"2024-04-25T10:17:51.378504+0000","last_unstale":"2024-04-25T10:17:53.396726+0000","last_undegraded":"2024-04-25T10:17:53.396726+0000","last_fullsized":"2024-04-25T10:17:53.396726+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:36.761276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960078+0000","last_change":"2024-04-25T10:17:52.381496+0000","last_active":"2024-04-25T10:17:53.960078+0000","last_peered":"2024-04-25T10:17:53.960078+0000","last_clean":"2024-04-25T10:17:53.960078+0000","last_became_active":"2024-04-25T10:17:52.381214+0000","last_became_peered":"2024-04-25T10:17:52.381214+0000","last_unstale":"2024-04-25T10:17:53.960078+0000","last_undegraded":"2024-04-25T10:17:53.960078+0000","last_fullsized":"2024-04-25T10:17:53.960078+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:03.518830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391899+0000","last_change":"2024-04-25T10:17:51.383528+0000","last_active":"2024-04-25T10:17:53.391899+0000","last_peered":"2024-04-25T10:17:53.391899+0000","last_clean":"2024-04-25T10:17:53.391899+0000","last_became_active":"2024-04-25T10:17:51.382449+0000","last_became_peered":"2024-04-25T10:17:51.382449+0000","last_unstale":"2024-04-25T10:17:53.391899+0000","last_undegraded":"2024-04-25T10:17:53.391899+0000","last_fullsized":"2024-04-25T10:17:53.391899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:48.614198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391596+0000","last_change":"2024-04-25T10:17:52.393611+0000","last_active":"2024-04-25T10:17:53.391596+0000","last_peered":"2024-04-25T10:17:53.391596+0000","last_clean":"2024-04-25T10:17:53.391596+0000","last_became_active":"2024-04-25T10:17:52.393363+0000","last_became_peered":"2024-04-25T10:17:52.393363+0000","last_unstale":"2024-04-25T10:17:53.391596+0000","last_undegraded":"2024-04-25T10:17:53.391596+0000","last_fullsized":"2024-04-25T10:17:53.391596+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:38.803774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.681054+0000","last_change":"2024-04-25T10:17:51.378958+0000","last_active":"2024-04-25T10:17:53.681054+0000","last_peered":"2024-04-25T10:17:53.681054+0000","last_clean":"2024-04-25T10:17:53.681054+0000","last_became_active":"2024-04-25T10:17:51.378655+0000","last_became_peered":"2024-04-25T10:17:51.378655+0000","last_unstale":"2024-04-25T10:17:53.681054+0000","last_undegraded":"2024-04-25T10:17:53.681054+0000","last_fullsized":"2024-04-25T10:17:53.681054+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:50.024871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960446+0000","last_change":"2024-04-25T10:17:52.386786+0000","last_active":"2024-04-25T10:17:53.960446+0000","last_peered":"2024-04-25T10:17:53.960446+0000","last_clean":"2024-04-25T10:17:53.960446+0000","last_became_active":"2024-04-25T10:17:52.385813+0000","last_became_peered":"2024-04-25T10:17:52.385813+0000","last_unstale":"2024-04-25T10:17:53.960446+0000","last_undegraded":"2024-04-25T10:17:53.960446+0000","last_fullsized":"2024-04-25T10:17:53.960446+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:21.393109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962589+0000","last_change":"2024-04-25T10:17:51.377735+0000","last_active":"2024-04-25T10:17:53.962589+0000","last_peered":"2024-04-25T10:17:53.962589+0000","last_clean":"2024-04-25T10:17:53.962589+0000","last_became_active":"2024-04-25T10:17:51.376589+0000","last_became_peered":"2024-04-25T10:17:51.376589+0000","last_unstale":"2024-04-25T10:17:53.962589+0000","last_undegraded":"2024-04-25T10:17:53.962589+0000","last_fullsized":"2024-04-25T10:17:53.962589+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:04.624140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958972+0000","last_change":"2024-04-25T10:17:52.379579+0000","last_active":"2024-04-25T10:17:53.958972+0000","last_peered":"2024-04-25T10:17:53.958972+0000","last_clean":"2024-04-25T10:17:53.958972+0000","last_became_active":"2024-04-25T10:17:52.379299+0000","last_became_peered":"2024-04-25T10:17:52.379299+0000","last_unstale":"2024-04-25T10:17:53.958972+0000","last_undegraded":"2024-04-25T10:17:53.958972+0000","last_fullsized":"2024-04-25T10:17:53.958972+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:49.917637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959382+0000","last_change":"2024-04-25T10:17:51.379258+0000","last_active":"2024-04-25T10:17:53.959382+0000","last_peered":"2024-04-25T10:17:53.959382+0000","last_clean":"2024-04-25T10:17:53.959382+0000","last_became_active":"2024-04-25T10:17:51.378916+0000","last_became_peered":"2024-04-25T10:17:51.378916+0000","last_unstale":"2024-04-25T10:17:53.959382+0000","last_undegraded":"2024-04-25T10:17:53.959382+0000","last_fullsized":"2024-04-25T10:17:53.959382+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.703271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962002+0000","last_change":"2024-04-25T10:17:52.386364+0000","last_active":"2024-04-25T10:17:53.962002+0000","last_peered":"2024-04-25T10:17:53.962002+0000","last_clean":"2024-04-25T10:17:53.962002+0000","last_became_active":"2024-04-25T10:17:52.383228+0000","last_became_peered":"2024-04-25T10:17:52.383228+0000","last_unstale":"2024-04-25T10:17:53.962002+0000","last_undegraded":"2024-04-25T10:17:53.962002+0000","last_fullsized":"2024-04-25T10:17:53.962002+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.237783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391685+0000","last_change":"2024-04-25T10:17:51.376585+0000","last_active":"2024-04-25T10:17:53.391685+0000","last_peered":"2024-04-25T10:17:53.391685+0000","last_clean":"2024-04-25T10:17:53.391685+0000","last_became_active":"2024-04-25T10:17:51.376266+0000","last_became_peered":"2024-04-25T10:17:51.376266+0000","last_unstale":"2024-04-25T10:17:53.391685+0000","last_undegraded":"2024-04-25T10:17:53.391685+0000","last_fullsized":"2024-04-25T10:17:53.391685+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.293522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962029+0000","last_change":"2024-04-25T10:17:52.368799+0000","last_active":"2024-04-25T10:17:53.962029+0000","last_peered":"2024-04-25T10:17:53.962029+0000","last_clean":"2024-04-25T10:17:53.962029+0000","last_became_active":"2024-04-25T10:17:52.368494+0000","last_became_peered":"2024-04-25T10:17:52.368494+0000","last_unstale":"2024-04-25T10:17:53.962029+0000","last_undegraded":"2024-04-25T10:17:53.962029+0000","last_fullsized":"2024-04-25T10:17:53.962029+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:18.305204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958599+0000","last_change":"2024-04-25T10:17:51.383223+0000","last_active":"2024-04-25T10:17:53.958599+0000","last_peered":"2024-04-25T10:17:53.958599+0000","last_clean":"2024-04-25T10:17:53.958599+0000","last_became_active":"2024-04-25T10:17:51.382880+0000","last_became_peered":"2024-04-25T10:17:51.382880+0000","last_unstale":"2024-04-25T10:17:53.958599+0000","last_undegraded":"2024-04-25T10:17:53.958599+0000","last_fullsized":"2024-04-25T10:17:53.958599+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:22.503464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396783+0000","last_change":"2024-04-25T10:17:52.395475+0000","last_active":"2024-04-25T10:17:53.396783+0000","last_peered":"2024-04-25T10:17:53.396783+0000","last_clean":"2024-04-25T10:17:53.396783+0000","last_became_active":"2024-04-25T10:17:52.395115+0000","last_became_peered":"2024-04-25T10:17:52.395115+0000","last_unstale":"2024-04-25T10:17:53.396783+0000","last_undegraded":"2024-04-25T10:17:53.396783+0000","last_fullsized":"2024-04-25T10:17:53.396783+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:48.858300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958401+0000","last_change":"2024-04-25T10:17:51.382057+0000","last_active":"2024-04-25T10:17:53.958401+0000","last_peered":"2024-04-25T10:17:53.958401+0000","last_clean":"2024-04-25T10:17:53.958401+0000","last_became_active":"2024-04-25T10:17:51.381426+0000","last_became_peered":"2024-04-25T10:17:51.381426+0000","last_unstale":"2024-04-25T10:17:53.958401+0000","last_undegraded":"2024-04-25T10:17:53.958401+0000","last_fullsized":"2024-04-25T10:17:53.958401+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:51.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959178+0000","last_change":"2024-04-25T10:17:52.380780+0000","last_active":"2024-04-25T10:17:53.959178+0000","last_peered":"2024-04-25T10:17:53.959178+0000","last_clean":"2024-04-25T10:17:53.959178+0000","last_became_active":"2024-04-25T10:17:52.379987+0000","last_became_peered":"2024-04-25T10:17:52.379987+0000","last_unstale":"2024-04-25T10:17:53.959178+0000","last_undegraded":"2024-04-25T10:17:53.959178+0000","last_fullsized":"2024-04-25T10:17:53.959178+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:43.879780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961055+0000","last_change":"2024-04-25T10:17:51.365010+0000","last_active":"2024-04-25T10:17:53.961055+0000","last_peered":"2024-04-25T10:17:53.961055+0000","last_clean":"2024-04-25T10:17:53.961055+0000","last_became_active":"2024-04-25T10:17:51.364712+0000","last_became_peered":"2024-04-25T10:17:51.364712+0000","last_unstale":"2024-04-25T10:17:53.961055+0000","last_undegraded":"2024-04-25T10:17:53.961055+0000","last_fullsized":"2024-04-25T10:17:53.961055+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:49.298576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962218+0000","last_change":"2024-04-25T10:17:52.392583+0000","last_active":"2024-04-25T10:17:53.962218+0000","last_peered":"2024-04-25T10:17:53.962218+0000","last_clean":"2024-04-25T10:17:53.962218+0000","last_became_active":"2024-04-25T10:17:52.392225+0000","last_became_peered":"2024-04-25T10:17:52.392225+0000","last_unstale":"2024-04-25T10:17:53.962218+0000","last_undegraded":"2024-04-25T10:17:53.962218+0000","last_fullsized":"2024-04-25T10:17:53.962218+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:04.680743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":17,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396715+0000","last_change":"2024-04-25T10:17:51.382259+0000","last_active":"2024-04-25T10:17:53.396715+0000","last_peered":"2024-04-25T10:17:53.396715+0000","last_clean":"2024-04-25T10:17:53.396715+0000","last_became_active":"2024-04-25T10:17:51.381960+0000","last_became_peered":"2024-04-25T10:17:51.381960+0000","last_unstale":"2024-04-25T10:17:53.396715+0000","last_undegraded":"2024-04-25T10:17:53.396715+0000","last_fullsized":"2024-04-25T10:17:53.396715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.670982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962422+0000","last_change":"2024-04-25T10:17:52.389846+0000","last_active":"2024-04-25T10:17:53.962422+0000","last_peered":"2024-04-25T10:17:53.962422+0000","last_clean":"2024-04-25T10:17:53.962422+0000","last_became_active":"2024-04-25T10:17:52.388663+0000","last_became_peered":"2024-04-25T10:17:52.388663+0000","last_unstale":"2024-04-25T10:17:53.962422+0000","last_undegraded":"2024-04-25T10:17:53.962422+0000","last_fullsized":"2024-04-25T10:17:53.962422+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:19.429395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"158'6","reported_seq":20,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962055+0000","last_change":"2024-04-25T10:17:51.378190+0000","last_active":"2024-04-25T10:17:53.962055+0000","last_peered":"2024-04-25T10:17:53.962055+0000","last_clean":"2024-04-25T10:17:53.962055+0000","last_became_active":"2024-04-25T10:17:51.377878+0000","last_became_peered":"2024-04-25T10:17:51.377878+0000","last_unstale":"2024-04-25T10:17:53.962055+0000","last_undegraded":"2024-04-25T10:17:53.962055+0000","last_fullsized":"2024-04-25T10:17:53.962055+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.244741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.397806+0000","last_change":"2024-04-25T10:17:52.383868+0000","last_active":"2024-04-25T10:17:53.397806+0000","last_peered":"2024-04-25T10:17:53.397806+0000","last_clean":"2024-04-25T10:17:53.397806+0000","last_became_active":"2024-04-25T10:17:52.383558+0000","last_became_peered":"2024-04-25T10:17:52.383558+0000","last_unstale":"2024-04-25T10:17:53.397806+0000","last_undegraded":"2024-04-25T10:17:53.397806+0000","last_fullsized":"2024-04-25T10:17:53.397806+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:08.834397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392530+0000","last_change":"2024-04-25T10:17:51.377387+0000","last_active":"2024-04-25T10:17:53.392530+0000","last_peered":"2024-04-25T10:17:53.392530+0000","last_clean":"2024-04-25T10:17:53.392530+0000","last_became_active":"2024-04-25T10:17:51.376999+0000","last_became_peered":"2024-04-25T10:17:51.376999+0000","last_unstale":"2024-04-25T10:17:53.392530+0000","last_undegraded":"2024-04-25T10:17:53.392530+0000","last_fullsized":"2024-04-25T10:17:53.392530+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:49.298576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960335+0000","last_change":"2024-04-25T10:17:52.384209+0000","last_active":"2024-04-25T10:17:53.960335+0000","last_peered":"2024-04-25T10:17:53.960335+0000","last_clean":"2024-04-25T10:17:53.960335+0000","last_became_active":"2024-04-25T10:17:52.383948+0000","last_became_peered":"2024-04-25T10:17:52.383948+0000","last_unstale":"2024-04-25T10:17:53.960335+0000","last_undegraded":"2024-04-25T10:17:53.960335+0000","last_fullsized":"2024-04-25T10:17:53.960335+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:13.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960496+0000","last_change":"2024-04-25T10:17:51.374071+0000","last_active":"2024-04-25T10:17:53.960496+0000","last_peered":"2024-04-25T10:17:53.960496+0000","last_clean":"2024-04-25T10:17:53.960496+0000","last_became_active":"2024-04-25T10:17:51.371382+0000","last_became_peered":"2024-04-25T10:17:51.371382+0000","last_unstale":"2024-04-25T10:17:53.960496+0000","last_undegraded":"2024-04-25T10:17:53.960496+0000","last_fullsized":"2024-04-25T10:17:53.960496+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:06.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680288+0000","last_change":"2024-04-25T10:17:52.394834+0000","last_active":"2024-04-25T10:17:53.680288+0000","last_peered":"2024-04-25T10:17:53.680288+0000","last_clean":"2024-04-25T10:17:53.680288+0000","last_became_active":"2024-04-25T10:17:52.394550+0000","last_became_peered":"2024-04-25T10:17:52.394550+0000","last_unstale":"2024-04-25T10:17:53.680288+0000","last_undegraded":"2024-04-25T10:17:53.680288+0000","last_fullsized":"2024-04-25T10:17:53.680288+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:48.155978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680178+0000","last_change":"2024-04-25T10:17:51.383883+0000","last_active":"2024-04-25T10:17:53.680178+0000","last_peered":"2024-04-25T10:17:53.680178+0000","last_clean":"2024-04-25T10:17:53.680178+0000","last_became_active":"2024-04-25T10:17:51.383352+0000","last_became_peered":"2024-04-25T10:17:51.383352+0000","last_unstale":"2024-04-25T10:17:53.680178+0000","last_undegraded":"2024-04-25T10:17:53.680178+0000","last_fullsized":"2024-04-25T10:17:53.680178+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.703271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962006+0000","last_change":"2024-04-25T10:17:52.375739+0000","last_active":"2024-04-25T10:17:53.962006+0000","last_peered":"2024-04-25T10:17:53.962006+0000","last_clean":"2024-04-25T10:17:53.962006+0000","last_became_active":"2024-04-25T10:17:52.375434+0000","last_became_peered":"2024-04-25T10:17:52.375434+0000","last_unstale":"2024-04-25T10:17:53.962006+0000","last_undegraded":"2024-04-25T10:17:53.962006+0000","last_fullsized":"2024-04-25T10:17:53.962006+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:07.674007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"158'8","reported_seq":22,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958676+0000","last_change":"2024-04-25T10:17:51.382235+0000","last_active":"2024-04-25T10:17:53.958676+0000","last_peered":"2024-04-25T10:17:53.958676+0000","last_clean":"2024-04-25T10:17:53.958676+0000","last_became_active":"2024-04-25T10:17:51.381803+0000","last_became_peered":"2024-04-25T10:17:51.381803+0000","last_unstale":"2024-04-25T10:17:53.958676+0000","last_undegraded":"2024-04-25T10:17:53.958676+0000","last_fullsized":"2024-04-25T10:17:53.958676+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:05.311623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959127+0000","last_change":"2024-04-25T10:17:52.383371+0000","last_active":"2024-04-25T10:17:53.959127+0000","last_peered":"2024-04-25T10:17:53.959127+0000","last_clean":"2024-04-25T10:17:53.959127+0000","last_became_active":"2024-04-25T10:17:52.383109+0000","last_became_peered":"2024-04-25T10:17:52.383109+0000","last_unstale":"2024-04-25T10:17:53.959127+0000","last_undegraded":"2024-04-25T10:17:53.959127+0000","last_fullsized":"2024-04-25T10:17:53.959127+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:52.265324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961827+0000","last_change":"2024-04-25T10:17:51.378215+0000","last_active":"2024-04-25T10:17:53.961827+0000","last_peered":"2024-04-25T10:17:53.961827+0000","last_clean":"2024-04-25T10:17:53.961827+0000","last_became_active":"2024-04-25T10:17:51.377905+0000","last_became_peered":"2024-04-25T10:17:51.377905+0000","last_unstale":"2024-04-25T10:17:53.961827+0000","last_undegraded":"2024-04-25T10:17:53.961827+0000","last_fullsized":"2024-04-25T10:17:53.961827+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:20.734563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.398063+0000","last_change":"2024-04-25T10:17:52.395871+0000","last_active":"2024-04-25T10:17:53.398063+0000","last_peered":"2024-04-25T10:17:53.398063+0000","last_clean":"2024-04-25T10:17:53.398063+0000","last_became_active":"2024-04-25T10:17:52.394979+0000","last_became_peered":"2024-04-25T10:17:52.394979+0000","last_unstale":"2024-04-25T10:17:53.398063+0000","last_undegraded":"2024-04-25T10:17:53.398063+0000","last_fullsized":"2024-04-25T10:17:53.398063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:32.242017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960228+0000","last_change":"2024-04-25T10:17:51.367493+0000","last_active":"2024-04-25T10:17:53.960228+0000","last_peered":"2024-04-25T10:17:53.960228+0000","last_clean":"2024-04-25T10:17:53.960228+0000","last_became_active":"2024-04-25T10:17:51.367097+0000","last_became_peered":"2024-04-25T10:17:51.367097+0000","last_unstale":"2024-04-25T10:17:53.960228+0000","last_undegraded":"2024-04-25T10:17:53.960228+0000","last_fullsized":"2024-04-25T10:17:53.960228+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:35.759748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679894+0000","last_change":"2024-04-25T10:17:52.391148+0000","last_active":"2024-04-25T10:17:53.679894+0000","last_peered":"2024-04-25T10:17:53.679894+0000","last_clean":"2024-04-25T10:17:53.679894+0000","last_became_active":"2024-04-25T10:17:52.390826+0000","last_became_peered":"2024-04-25T10:17:52.390826+0000","last_unstale":"2024-04-25T10:17:53.679894+0000","last_undegraded":"2024-04-25T10:17:53.679894+0000","last_fullsized":"2024-04-25T10:17:53.679894+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:53.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392758+0000","last_change":"2024-04-25T10:17:51.383023+0000","last_active":"2024-04-25T10:17:53.392758+0000","last_peered":"2024-04-25T10:17:53.392758+0000","last_clean":"2024-04-25T10:17:53.392758+0000","last_became_active":"2024-04-25T10:17:51.382697+0000","last_became_peered":"2024-04-25T10:17:51.382697+0000","last_unstale":"2024-04-25T10:17:53.392758+0000","last_undegraded":"2024-04-25T10:17:53.392758+0000","last_fullsized":"2024-04-25T10:17:53.392758+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:17.023081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680211+0000","last_change":"2024-04-25T10:17:52.391324+0000","last_active":"2024-04-25T10:17:53.680211+0000","last_peered":"2024-04-25T10:17:53.680211+0000","last_clean":"2024-04-25T10:17:53.680211+0000","last_became_active":"2024-04-25T10:17:52.391014+0000","last_became_peered":"2024-04-25T10:17:52.391014+0000","last_unstale":"2024-04-25T10:17:53.680211+0000","last_undegraded":"2024-04-25T10:17:53.680211+0000","last_fullsized":"2024-04-25T10:17:53.680211+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.982915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959535+0000","last_change":"2024-04-25T10:17:51.381541+0000","last_active":"2024-04-25T10:17:53.959535+0000","last_peered":"2024-04-25T10:17:53.959535+0000","last_clean":"2024-04-25T10:17:53.959535+0000","last_became_active":"2024-04-25T10:17:51.380331+0000","last_became_peered":"2024-04-25T10:17:51.380331+0000","last_unstale":"2024-04-25T10:17:53.959535+0000","last_undegraded":"2024-04-25T10:17:53.959535+0000","last_fullsized":"2024-04-25T10:17:53.959535+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:22.437023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961887+0000","last_change":"2024-04-25T10:17:52.383748+0000","last_active":"2024-04-25T10:17:53.961887+0000","last_peered":"2024-04-25T10:17:53.961887+0000","last_clean":"2024-04-25T10:17:53.961887+0000","last_became_active":"2024-04-25T10:17:52.383479+0000","last_became_peered":"2024-04-25T10:17:52.383479+0000","last_unstale":"2024-04-25T10:17:53.961887+0000","last_undegraded":"2024-04-25T10:17:53.961887+0000","last_fullsized":"2024-04-25T10:17:53.961887+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.862802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392664+0000","last_change":"2024-04-25T10:17:51.386166+0000","last_active":"2024-04-25T10:17:53.392664+0000","last_peered":"2024-04-25T10:17:53.392664+0000","last_clean":"2024-04-25T10:17:53.392664+0000","last_became_active":"2024-04-25T10:17:51.385874+0000","last_became_peered":"2024-04-25T10:17:51.385874+0000","last_unstale":"2024-04-25T10:17:53.392664+0000","last_undegraded":"2024-04-25T10:17:53.392664+0000","last_fullsized":"2024-04-25T10:17:53.392664+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.244741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392511+0000","last_change":"2024-04-25T10:17:52.384487+0000","last_active":"2024-04-25T10:17:53.392511+0000","last_peered":"2024-04-25T10:17:53.392511+0000","last_clean":"2024-04-25T10:17:53.392511+0000","last_became_active":"2024-04-25T10:17:52.384223+0000","last_became_peered":"2024-04-25T10:17:52.384223+0000","last_unstale":"2024-04-25T10:17:53.392511+0000","last_undegraded":"2024-04-25T10:17:53.392511+0000","last_fullsized":"2024-04-25T10:17:53.392511+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:03.518830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679710+0000","last_change":"2024-04-25T10:17:51.380152+0000","last_active":"2024-04-25T10:17:53.679710+0000","last_peered":"2024-04-25T10:17:53.679710+0000","last_clean":"2024-04-25T10:17:53.679710+0000","last_became_active":"2024-04-25T10:17:51.379860+0000","last_became_peered":"2024-04-25T10:17:51.379860+0000","last_unstale":"2024-04-25T10:17:53.679710+0000","last_undegraded":"2024-04-25T10:17:53.679710+0000","last_fullsized":"2024-04-25T10:17:53.679710+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:05.311623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391913+0000","last_change":"2024-04-25T10:17:52.385210+0000","last_active":"2024-04-25T10:17:53.391913+0000","last_peered":"2024-04-25T10:17:53.391913+0000","last_clean":"2024-04-25T10:17:53.391913+0000","last_became_active":"2024-04-25T10:17:52.384316+0000","last_became_peered":"2024-04-25T10:17:52.384316+0000","last_unstale":"2024-04-25T10:17:53.391913+0000","last_undegraded":"2024-04-25T10:17:53.391913+0000","last_fullsized":"2024-04-25T10:17:53.391913+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:55.497122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959105+0000","last_change":"2024-04-25T10:17:51.369127+0000","last_active":"2024-04-25T10:17:53.959105+0000","last_peered":"2024-04-25T10:17:53.959105+0000","last_clean":"2024-04-25T10:17:53.959105+0000","last_became_active":"2024-04-25T10:17:51.368836+0000","last_became_peered":"2024-04-25T10:17:51.368836+0000","last_unstale":"2024-04-25T10:17:53.959105+0000","last_undegraded":"2024-04-25T10:17:53.959105+0000","last_fullsized":"2024-04-25T10:17:53.959105+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.528377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960175+0000","last_change":"2024-04-25T10:17:52.384144+0000","last_active":"2024-04-25T10:17:53.960175+0000","last_peered":"2024-04-25T10:17:53.960175+0000","last_clean":"2024-04-25T10:17:53.960175+0000","last_became_active":"2024-04-25T10:17:52.383856+0000","last_became_peered":"2024-04-25T10:17:52.383856+0000","last_unstale":"2024-04-25T10:17:53.960175+0000","last_undegraded":"2024-04-25T10:17:53.960175+0000","last_fullsized":"2024-04-25T10:17:53.960175+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:38.320219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959683+0000","last_change":"2024-04-25T10:17:51.382409+0000","last_active":"2024-04-25T10:17:53.959683+0000","last_peered":"2024-04-25T10:17:53.959683+0000","last_clean":"2024-04-25T10:17:53.959683+0000","last_became_active":"2024-04-25T10:17:51.382048+0000","last_became_peered":"2024-04-25T10:17:51.382048+0000","last_unstale":"2024-04-25T10:17:53.959683+0000","last_undegraded":"2024-04-25T10:17:53.959683+0000","last_fullsized":"2024-04-25T10:17:53.959683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:00.400583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679541+0000","last_change":"2024-04-25T10:17:52.388275+0000","last_active":"2024-04-25T10:17:53.679541+0000","last_peered":"2024-04-25T10:17:53.679541+0000","last_clean":"2024-04-25T10:17:53.679541+0000","last_became_active":"2024-04-25T10:17:52.387979+0000","last_became_peered":"2024-04-25T10:17:52.387979+0000","last_unstale":"2024-04-25T10:17:53.679541+0000","last_undegraded":"2024-04-25T10:17:53.679541+0000","last_fullsized":"2024-04-25T10:17:53.679541+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:10.407438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392183+0000","last_change":"2024-04-25T10:17:51.382866+0000","last_active":"2024-04-25T10:17:53.392183+0000","last_peered":"2024-04-25T10:17:53.392183+0000","last_clean":"2024-04-25T10:17:53.392183+0000","last_became_active":"2024-04-25T10:17:51.382164+0000","last_became_peered":"2024-04-25T10:17:51.382164+0000","last_unstale":"2024-04-25T10:17:53.392183+0000","last_undegraded":"2024-04-25T10:17:53.392183+0000","last_fullsized":"2024-04-25T10:17:53.392183+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:20.734563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959067+0000","last_change":"2024-04-25T10:17:52.390924+0000","last_active":"2024-04-25T10:17:53.959067+0000","last_peered":"2024-04-25T10:17:53.959067+0000","last_clean":"2024-04-25T10:17:53.959067+0000","last_became_active":"2024-04-25T10:17:52.390258+0000","last_became_peered":"2024-04-25T10:17:52.390258+0000","last_unstale":"2024-04-25T10:17:53.959067+0000","last_undegraded":"2024-04-25T10:17:53.959067+0000","last_fullsized":"2024-04-25T10:17:53.959067+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.982915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"159'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961335+0000","last_change":"2024-04-25T10:17:51.370374+0000","last_active":"2024-04-25T10:17:53.961335+0000","last_peered":"2024-04-25T10:17:53.961335+0000","last_clean":"2024-04-25T10:17:53.961335+0000","last_became_active":"2024-04-25T10:17:51.369266+0000","last_became_peered":"2024-04-25T10:17:51.369266+0000","last_unstale":"2024-04-25T10:17:53.961335+0000","last_undegraded":"2024-04-25T10:17:53.961335+0000","last_fullsized":"2024-04-25T10:17:53.961335+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.293522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960959+0000","last_change":"2024-04-25T10:17:52.372429+0000","last_active":"2024-04-25T10:17:53.960959+0000","last_peered":"2024-04-25T10:17:53.960959+0000","last_clean":"2024-04-25T10:17:53.960959+0000","last_became_active":"2024-04-25T10:17:52.366412+0000","last_became_peered":"2024-04-25T10:17:52.366412+0000","last_unstale":"2024-04-25T10:17:53.960959+0000","last_undegraded":"2024-04-25T10:17:53.960959+0000","last_fullsized":"2024-04-25T10:17:53.960959+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:39.199231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961493+0000","last_change":"2024-04-25T10:17:51.379048+0000","last_active":"2024-04-25T10:17:53.961493+0000","last_peered":"2024-04-25T10:17:53.961493+0000","last_clean":"2024-04-25T10:17:53.961493+0000","last_became_active":"2024-04-25T10:17:51.378700+0000","last_became_peered":"2024-04-25T10:17:51.378700+0000","last_unstale":"2024-04-25T10:17:53.961493+0000","last_undegraded":"2024-04-25T10:17:53.961493+0000","last_fullsized":"2024-04-25T10:17:53.961493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:37.302831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959073+0000","last_change":"2024-04-25T10:17:52.382677+0000","last_active":"2024-04-25T10:17:53.959073+0000","last_peered":"2024-04-25T10:17:53.959073+0000","last_clean":"2024-04-25T10:17:53.959073+0000","last_became_active":"2024-04-25T10:17:52.382396+0000","last_became_peered":"2024-04-25T10:17:52.382396+0000","last_unstale":"2024-04-25T10:17:53.959073+0000","last_undegraded":"2024-04-25T10:17:53.959073+0000","last_fullsized":"2024-04-25T10:17:53.959073+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:48.155978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962474+0000","last_change":"2024-04-25T10:17:51.383501+0000","last_active":"2024-04-25T10:17:53.962474+0000","last_peered":"2024-04-25T10:17:53.962474+0000","last_clean":"2024-04-25T10:17:53.962474+0000","last_became_active":"2024-04-25T10:17:51.381248+0000","last_became_peered":"2024-04-25T10:17:51.381248+0000","last_unstale":"2024-04-25T10:17:53.962474+0000","last_undegraded":"2024-04-25T10:17:53.962474+0000","last_fullsized":"2024-04-25T10:17:53.962474+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:49.392115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396180+0000","last_change":"2024-04-25T10:17:52.382153+0000","last_active":"2024-04-25T10:17:53.396180+0000","last_peered":"2024-04-25T10:17:53.396180+0000","last_clean":"2024-04-25T10:17:53.396180+0000","last_became_active":"2024-04-25T10:17:52.381688+0000","last_became_peered":"2024-04-25T10:17:52.381688+0000","last_unstale":"2024-04-25T10:17:53.396180+0000","last_undegraded":"2024-04-25T10:17:53.396180+0000","last_fullsized":"2024-04-25T10:17:53.396180+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:30.844244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396121+0000","last_change":"2024-04-25T10:17:51.384528+0000","last_active":"2024-04-25T10:17:53.396121+0000","last_peered":"2024-04-25T10:17:53.396121+0000","last_clean":"2024-04-25T10:17:53.396121+0000","last_became_active":"2024-04-25T10:17:51.384176+0000","last_became_peered":"2024-04-25T10:17:51.384176+0000","last_unstale":"2024-04-25T10:17:53.396121+0000","last_undegraded":"2024-04-25T10:17:53.396121+0000","last_fullsized":"2024-04-25T10:17:53.396121+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:48.896090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959995+0000","last_change":"2024-04-25T10:17:52.385687+0000","last_active":"2024-04-25T10:17:53.959995+0000","last_peered":"2024-04-25T10:17:53.959995+0000","last_clean":"2024-04-25T10:17:53.959995+0000","last_became_active":"2024-04-25T10:17:52.385381+0000","last_became_peered":"2024-04-25T10:17:52.385381+0000","last_unstale":"2024-04-25T10:17:53.959995+0000","last_undegraded":"2024-04-25T10:17:53.959995+0000","last_fullsized":"2024-04-25T10:17:53.959995+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:07.195022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961934+0000","last_change":"2024-04-25T10:17:51.366496+0000","last_active":"2024-04-25T10:17:53.961934+0000","last_peered":"2024-04-25T10:17:53.961934+0000","last_clean":"2024-04-25T10:17:53.961934+0000","last_became_active":"2024-04-25T10:17:51.364699+0000","last_became_peered":"2024-04-25T10:17:51.364699+0000","last_unstale":"2024-04-25T10:17:53.961934+0000","last_undegraded":"2024-04-25T10:17:53.961934+0000","last_fullsized":"2024-04-25T10:17:53.961934+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:49.409675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960587+0000","last_change":"2024-04-25T10:17:52.383196+0000","last_active":"2024-04-25T10:17:53.960587+0000","last_peered":"2024-04-25T10:17:53.960587+0000","last_clean":"2024-04-25T10:17:53.960587+0000","last_became_active":"2024-04-25T10:17:52.382938+0000","last_became_peered":"2024-04-25T10:17:52.382938+0000","last_unstale":"2024-04-25T10:17:53.960587+0000","last_undegraded":"2024-04-25T10:17:53.960587+0000","last_fullsized":"2024-04-25T10:17:53.960587+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:22.716552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680346+0000","last_change":"2024-04-25T10:17:51.379727+0000","last_active":"2024-04-25T10:17:53.680346+0000","last_peered":"2024-04-25T10:17:53.680346+0000","last_clean":"2024-04-25T10:17:53.680346+0000","last_became_active":"2024-04-25T10:17:51.379202+0000","last_became_peered":"2024-04-25T10:17:51.379202+0000","last_unstale":"2024-04-25T10:17:53.680346+0000","last_undegraded":"2024-04-25T10:17:53.680346+0000","last_fullsized":"2024-04-25T10:17:53.680346+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:23.325966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961582+0000","last_change":"2024-04-25T10:17:52.385481+0000","last_active":"2024-04-25T10:17:53.961582+0000","last_peered":"2024-04-25T10:17:53.961582+0000","last_clean":"2024-04-25T10:17:53.961582+0000","last_became_active":"2024-04-25T10:17:52.385221+0000","last_became_peered":"2024-04-25T10:17:52.385221+0000","last_unstale":"2024-04-25T10:17:53.961582+0000","last_undegraded":"2024-04-25T10:17:53.961582+0000","last_fullsized":"2024-04-25T10:17:53.961582+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:00.789628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391315+0000","last_change":"2024-04-25T10:17:51.384876+0000","last_active":"2024-04-25T10:17:53.391315+0000","last_peered":"2024-04-25T10:17:53.391315+0000","last_clean":"2024-04-25T10:17:53.391315+0000","last_became_active":"2024-04-25T10:17:51.384131+0000","last_became_peered":"2024-04-25T10:17:51.384131+0000","last_unstale":"2024-04-25T10:17:53.391315+0000","last_undegraded":"2024-04-25T10:17:53.391315+0000","last_fullsized":"2024-04-25T10:17:53.391315+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:42.862391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959304+0000","last_change":"2024-04-25T10:17:52.374453+0000","last_active":"2024-04-25T10:17:53.959304+0000","last_peered":"2024-04-25T10:17:53.959304+0000","last_clean":"2024-04-25T10:17:53.959304+0000","last_became_active":"2024-04-25T10:17:52.374110+0000","last_became_peered":"2024-04-25T10:17:52.374110+0000","last_unstale":"2024-04-25T10:17:53.959304+0000","last_undegraded":"2024-04-25T10:17:53.959304+0000","last_fullsized":"2024-04-25T10:17:53.959304+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:17.262129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961009+0000","last_change":"2024-04-25T10:17:51.371004+0000","last_active":"2024-04-25T10:17:53.961009+0000","last_peered":"2024-04-25T10:17:53.961009+0000","last_clean":"2024-04-25T10:17:53.961009+0000","last_became_active":"2024-04-25T10:17:51.370705+0000","last_became_peered":"2024-04-25T10:17:51.370705+0000","last_unstale":"2024-04-25T10:17:53.961009+0000","last_undegraded":"2024-04-25T10:17:53.961009+0000","last_fullsized":"2024-04-25T10:17:53.961009+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:17.023081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":436,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958769+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:17:53.958769+0000","last_peered":"2024-04-25T10:17:53.958769+0000","last_clean":"2024-04-25T10:17:53.958769+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:17:53.958769+0000","last_undegraded":"2024-04-25T10:17:53.958769+0000","last_fullsized":"2024-04-25T10:17:53.958769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60084,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311756,"statfs":{"total":96636764160,"available":96575238144,"internally_reserved":0,"allocated":2592768,"data_stored":1529887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59504,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312336,"statfs":{"total":96636764160,"available":96575832064,"internally_reserved":0,"allocated":1998848,"data_stored":939519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55352,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316488,"statfs":{"total":96636764160,"available":96580083712,"internally_reserved":0,"allocated":2007040,"data_stored":939643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284604,"statfs":{"total":96636764160,"available":96547434496,"internally_reserved":0,"allocated":2015232,"data_stored":940283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575045,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87236,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284604,"statfs":{"total":96636764160,"available":96547434496,"internally_reserved":0,"allocated":2019328,"data_stored":941735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59504,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312336,"statfs":{"total":96636764160,"available":96575832064,"internally_reserved":0,"allocated":1998848,"data_stored":939519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59520,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312320,"statfs":{"total":96636764160,"available":96575815680,"internally_reserved":0,"allocated":2011136,"data_stored":941131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47608,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324232,"statfs":{"total":96636764160,"available":96588013568,"internally_reserved":0,"allocated":2600960,"data_stored":1529991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:00.759 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:00.767 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:01.804 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:18:02.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.011+0000 7f2c5ca00640 1 Processor -- start 2024-04-25T10:18:02.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.011+0000 7f2c5ca00640 1 -- start start 2024-04-25T10:18:02.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c5ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2c5815df50 0x7f2c580cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c5ca00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c58071650 con 0x7f2c5815db70 2024-04-25T10:18:02.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c5ca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c580cf230 con 0x7f2c580cfc20 2024-04-25T10:18:02.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c5ca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c580cbd80 con 0x7f2c5815df50 2024-04-25T10:18:02.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c55e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2c5815db70 0x7f2c580cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42220/0 (socket says 172.21.15.165:42220) 2024-04-25T10:18:02.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c55e00640 1 -- 172.21.15.165:0/3161547506 learned_addr learned my addr 172.21.15.165:0/3161547506 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:02.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c56800640 1 --2- 172.21.15.165:0/3161547506 >> v2:172.21.15.172:3300/0 conn(0x7f2c5815df50 0x7f2c580cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769587677 0 0) 0x7f2c48001110 con 0x7f2c5815db70 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.013+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c3c003520 con 0x7f2c5815db70 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 769819775 0 0) 0x7f2c3c003520 con 0x7f2c5815db70 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 >> v2:172.21.15.172:3300/0 conn(0x7f2c5815df50 msgr2=0x7f2c580cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 --2- 172.21.15.165:0/3161547506 >> v2:172.21.15.172:3300/0 conn(0x7f2c5815df50 0x7f2c580cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 >> v1:172.21.15.172:6789/0 conn(0x7f2c580cfc20 legacy=0x7f2c5806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c58161b60 con 0x7f2c5815db70 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2c48001840 con 0x7f2c5815db70 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 >> v1:172.21.15.165:6789/0 conn(0x7f2c5815db70 legacy=0x7f2c580cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c3c003a00 0x7f2c3c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c004800 0x7f2c3c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c3c005210 0x7f2c3c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c580cbd80 con 0x7f2c3c005210 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c00c140 con 0x7f2c3c004800 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3c00c340 con 0x7f2c3c003a00 2024-04-25T10:18:02.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c56800640 1 --2- 172.21.15.165:0/3161547506 >> v2:172.21.15.172:3300/0 conn(0x7f2c5815df50 0x7f2c580cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.014+0000 7f2c56800640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c3c005210 0x7f2c3c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c55e00640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c3c003a00 0x7f2c3c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c4fe00640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c004800 0x7f2c3c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c56800640 1 -- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c3c003a00 msgr2=0x7f2c3c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c56800640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c3c003a00 0x7f2c3c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c56800640 1 -- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c004800 msgr2=0x7f2c3c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c56800640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c004800 0x7f2c3c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c56800640 1 -- 172.21.15.165:0/3161547506 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c003520 con 0x7f2c3c005210 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c55e00640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c3c003a00 0x7f2c3c003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.015+0000 7f2c4fe00640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c004800 0x7f2c3c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.016+0000 7f2c56800640 1 -- 172.21.15.165:0/3161547506 reap_dead start 2024-04-25T10:18:02.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.016+0000 7f2c56800640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c3c005210 0x7f2c3c00b9c0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f2c500073b0 tx=0x7f2c3c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362d5e42605a0198 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.016+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50009c70 con 0x7f2c3c005210 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.016+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5002c650 con 0x7f2c3c005210 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.016+0000 7f2c55400640 1 -- 172.21.15.165:0/3161547506 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5002ca20 con 0x7f2c3c005210 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 -- 172.21.15.165:0/3161547506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c3c005210 msgr2=0x7f2c3c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c3c005210 0x7f2c3c00b9c0 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7f2c500073b0 tx=0x7f2c3c005830 comp rx=0 tx=0).stop 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 -- 172.21.15.165:0/3161547506 shutdown_connections 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/3161547506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c3c005210 0x7f2c3c00b9c0 unknown :-1 s=CLOSED pgs=2268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 -- 172.21.15.165:0/3161547506 >> 172.21.15.165:0/3161547506 conn(0x7f2c5806abe0 msgr2=0x7f2c5806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 -- 172.21.15.165:0/3161547506 shutdown_connections 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.017+0000 7f2c5ca00640 1 -- 172.21.15.165:0/3161547506 wait complete. 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.019+0000 7f2c5ca00640 1 Processor -- start 2024-04-25T10:18:02.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.019+0000 7f2c5ca00640 1 -- start start 2024-04-25T10:18:02.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c5ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c005210 0x7f2c5814b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c5ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c5815e3c0 0x7f2c5815e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c5ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 0x7f2c58157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c5ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c58162120 con 0x7f2c58156f00 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c5ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c58161910 con 0x7f2c3c005210 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c5ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c58162640 con 0x7f2c5815e3c0 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 0x7f2c58157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 0x7f2c58157380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39778/0 (socket says 172.21.15.165:39778) 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.020+0000 7f2c56800640 1 -- 172.21.15.165:0/1324560381 learned_addr learned my addr 172.21.15.165:0/1324560381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c55e00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c005210 0x7f2c5814b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c56800640 1 -- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c5815e3c0 msgr2=0x7f2c5815e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c4fe00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c5815e3c0 0x7f2c5815e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c56800640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c5815e3c0 0x7f2c5815e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c56800640 1 -- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c005210 msgr2=0x7f2c5814b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c56800640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c005210 0x7f2c5814b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c56800640 1 -- 172.21.15.165:0/1324560381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5815ef30 con 0x7f2c58156f00 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c56800640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 0x7f2c58157380 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f2c5002b130 tx=0x7f2c50004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c50009c70 con 0x7f2c58156f00 2024-04-25T10:18:02.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c5002ce40 con 0x7f2c58156f00 2024-04-25T10:18:02.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c5815f210 con 0x7f2c58156f00 2024-04-25T10:18:02.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.021+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5002d9f0 con 0x7f2c58156f00 2024-04-25T10:18:02.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.022+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c5815f7d0 con 0x7f2c58156f00 2024-04-25T10:18:02.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.023+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c58073990 con 0x7f2c58156f00 2024-04-25T10:18:02.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.024+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2c5002db90 con 0x7f2c58156f00 2024-04-25T10:18:02.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.025+0000 7f2c4e000640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c3406b4d0 0x7f2c3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.025+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2c500aa7a0 con 0x7f2c58156f00 2024-04-25T10:18:02.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.025+0000 7f2c55e00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c3406b4d0 0x7f2c3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.026+0000 7f2c55e00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c3406b4d0 0x7f2c3406d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2c48001eb0 tx=0x7f2c48009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.029+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2c50032030 con 0x7f2c58156f00 2024-04-25T10:18:02.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.220+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2c580cbb60 con 0x7f2c3406b4d0 2024-04-25T10:18:02.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.230+0000 7f2c4e000640 1 -- 172.21.15.165:0/1324560381 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252480 (secure 0 0 0) 0x7f2c580cbb60 con 0x7f2c3406b4d0 2024-04-25T10:18:02.236 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:02.238 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:18:02.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c3406b4d0 msgr2=0x7f2c3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c3406b4d0 0x7f2c3406d990 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f2c48001eb0 tx=0x7f2c48009040 comp rx=0 tx=0).stop 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 msgr2=0x7f2c58157380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 0x7f2c58157380 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7f2c5002b130 tx=0x7f2c50004ab0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 shutdown_connections 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2c3406b4d0 0x7f2c3406d990 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2c58156f00 0x7f2c58157380 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2c5815e3c0 0x7f2c5815e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.235+0000 7f2c5ca00640 1 --2- 172.21.15.165:0/1324560381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2c3c005210 0x7f2c5814b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.236+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 >> 172.21.15.165:0/1324560381 conn(0x7f2c5806abe0 msgr2=0x7f2c5815cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.236+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 shutdown_connections 2024-04-25T10:18:02.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.236+0000 7f2c5ca00640 1 -- 172.21.15.165:0/1324560381 wait complete. 2024-04-25T10:18:02.258 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":636,"stamp":"2024-04-25T10:18:00.454892+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":516044,"kb_used_data":16840,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754458676,"statfs":{"total":773094113280,"available":772565684224,"internally_reserved":0,"allocated":17244160,"data_stored":8701708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.099159"},"pg_stats":[{"pgid":"24.3f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396515+0000","last_change":"2024-04-25T10:17:52.392786+0000","last_active":"2024-04-25T10:17:53.396515+0000","last_peered":"2024-04-25T10:17:53.396515+0000","last_clean":"2024-04-25T10:17:53.396515+0000","last_became_active":"2024-04-25T10:17:52.391070+0000","last_became_peered":"2024-04-25T10:17:52.391070+0000","last_unstale":"2024-04-25T10:17:53.396515+0000","last_undegraded":"2024-04-25T10:17:53.396515+0000","last_fullsized":"2024-04-25T10:17:53.396515+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:36.131072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960367+0000","last_change":"2024-04-25T10:17:52.398361+0000","last_active":"2024-04-25T10:17:53.960367+0000","last_peered":"2024-04-25T10:17:53.960367+0000","last_clean":"2024-04-25T10:17:53.960367+0000","last_became_active":"2024-04-25T10:17:52.397707+0000","last_became_peered":"2024-04-25T10:17:52.397707+0000","last_unstale":"2024-04-25T10:17:53.960367+0000","last_undegraded":"2024-04-25T10:17:53.960367+0000","last_fullsized":"2024-04-25T10:17:53.960367+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:56:39.318816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396658+0000","last_change":"2024-04-25T10:17:52.377620+0000","last_active":"2024-04-25T10:17:53.396658+0000","last_peered":"2024-04-25T10:17:53.396658+0000","last_clean":"2024-04-25T10:17:53.396658+0000","last_became_active":"2024-04-25T10:17:52.377333+0000","last_became_peered":"2024-04-25T10:17:52.377333+0000","last_unstale":"2024-04-25T10:17:53.396658+0000","last_undegraded":"2024-04-25T10:17:53.396658+0000","last_fullsized":"2024-04-25T10:17:53.396658+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:09.616475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396157+0000","last_change":"2024-04-25T10:17:52.400656+0000","last_active":"2024-04-25T10:17:53.396157+0000","last_peered":"2024-04-25T10:17:53.396157+0000","last_clean":"2024-04-25T10:17:53.396157+0000","last_became_active":"2024-04-25T10:17:52.400364+0000","last_became_peered":"2024-04-25T10:17:52.400364+0000","last_unstale":"2024-04-25T10:17:53.396157+0000","last_undegraded":"2024-04-25T10:17:53.396157+0000","last_fullsized":"2024-04-25T10:17:53.396157+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:22.633919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392545+0000","last_change":"2024-04-25T10:17:52.394908+0000","last_active":"2024-04-25T10:17:53.392545+0000","last_peered":"2024-04-25T10:17:53.392545+0000","last_clean":"2024-04-25T10:17:53.392545+0000","last_became_active":"2024-04-25T10:17:52.394618+0000","last_became_peered":"2024-04-25T10:17:52.394618+0000","last_unstale":"2024-04-25T10:17:53.392545+0000","last_undegraded":"2024-04-25T10:17:53.392545+0000","last_fullsized":"2024-04-25T10:17:53.392545+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:48.887720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959221+0000","last_change":"2024-04-25T10:17:52.375197+0000","last_active":"2024-04-25T10:17:53.959221+0000","last_peered":"2024-04-25T10:17:53.959221+0000","last_clean":"2024-04-25T10:17:53.959221+0000","last_became_active":"2024-04-25T10:17:52.374283+0000","last_became_peered":"2024-04-25T10:17:52.374283+0000","last_unstale":"2024-04-25T10:17:53.959221+0000","last_undegraded":"2024-04-25T10:17:53.959221+0000","last_fullsized":"2024-04-25T10:17:53.959221+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:49.631586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958623+0000","last_change":"2024-04-25T10:17:52.382788+0000","last_active":"2024-04-25T10:17:53.958623+0000","last_peered":"2024-04-25T10:17:53.958623+0000","last_clean":"2024-04-25T10:17:53.958623+0000","last_became_active":"2024-04-25T10:17:52.382509+0000","last_became_peered":"2024-04-25T10:17:52.382509+0000","last_unstale":"2024-04-25T10:17:53.958623+0000","last_undegraded":"2024-04-25T10:17:53.958623+0000","last_fullsized":"2024-04-25T10:17:53.958623+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:53.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958109+0000","last_change":"2024-04-25T10:17:52.382408+0000","last_active":"2024-04-25T10:17:53.958109+0000","last_peered":"2024-04-25T10:17:53.958109+0000","last_clean":"2024-04-25T10:17:53.958109+0000","last_became_active":"2024-04-25T10:17:52.378938+0000","last_became_peered":"2024-04-25T10:17:52.378938+0000","last_unstale":"2024-04-25T10:17:53.958109+0000","last_undegraded":"2024-04-25T10:17:53.958109+0000","last_fullsized":"2024-04-25T10:17:53.958109+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:10.407438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958663+0000","last_change":"2024-04-25T10:17:52.385684+0000","last_active":"2024-04-25T10:17:53.958663+0000","last_peered":"2024-04-25T10:17:53.958663+0000","last_clean":"2024-04-25T10:17:53.958663+0000","last_became_active":"2024-04-25T10:17:52.385384+0000","last_became_peered":"2024-04-25T10:17:52.385384+0000","last_unstale":"2024-04-25T10:17:53.958663+0000","last_undegraded":"2024-04-25T10:17:53.958663+0000","last_fullsized":"2024-04-25T10:17:53.958663+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:16.856105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959997+0000","last_change":"2024-04-25T10:17:52.380579+0000","last_active":"2024-04-25T10:17:53.959997+0000","last_peered":"2024-04-25T10:17:53.959997+0000","last_clean":"2024-04-25T10:17:53.959997+0000","last_became_active":"2024-04-25T10:17:52.380240+0000","last_became_peered":"2024-04-25T10:17:52.380240+0000","last_unstale":"2024-04-25T10:17:53.959997+0000","last_undegraded":"2024-04-25T10:17:53.959997+0000","last_fullsized":"2024-04-25T10:17:53.959997+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:51:45.680232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680800+0000","last_change":"2024-04-25T10:17:52.402123+0000","last_active":"2024-04-25T10:17:53.680800+0000","last_peered":"2024-04-25T10:17:53.680800+0000","last_clean":"2024-04-25T10:17:53.680800+0000","last_became_active":"2024-04-25T10:17:52.401791+0000","last_became_peered":"2024-04-25T10:17:52.401791+0000","last_unstale":"2024-04-25T10:17:53.680800+0000","last_undegraded":"2024-04-25T10:17:53.680800+0000","last_fullsized":"2024-04-25T10:17:53.680800+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:34.714664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959631+0000","last_change":"2024-04-25T10:17:52.376217+0000","last_active":"2024-04-25T10:17:53.959631+0000","last_peered":"2024-04-25T10:17:53.959631+0000","last_clean":"2024-04-25T10:17:53.959631+0000","last_became_active":"2024-04-25T10:17:52.375941+0000","last_became_peered":"2024-04-25T10:17:52.375941+0000","last_unstale":"2024-04-25T10:17:53.959631+0000","last_undegraded":"2024-04-25T10:17:53.959631+0000","last_fullsized":"2024-04-25T10:17:53.959631+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:21.751952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960777+0000","last_change":"2024-04-25T10:17:52.390871+0000","last_active":"2024-04-25T10:17:53.960777+0000","last_peered":"2024-04-25T10:17:53.960777+0000","last_clean":"2024-04-25T10:17:53.960777+0000","last_became_active":"2024-04-25T10:17:52.390483+0000","last_became_peered":"2024-04-25T10:17:52.390483+0000","last_unstale":"2024-04-25T10:17:53.960777+0000","last_undegraded":"2024-04-25T10:17:53.960777+0000","last_fullsized":"2024-04-25T10:17:53.960777+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:50.628598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391921+0000","last_change":"2024-04-25T10:17:52.390116+0000","last_active":"2024-04-25T10:17:53.391921+0000","last_peered":"2024-04-25T10:17:53.391921+0000","last_clean":"2024-04-25T10:17:53.391921+0000","last_became_active":"2024-04-25T10:17:52.389817+0000","last_became_peered":"2024-04-25T10:17:52.389817+0000","last_unstale":"2024-04-25T10:17:53.391921+0000","last_undegraded":"2024-04-25T10:17:53.391921+0000","last_fullsized":"2024-04-25T10:17:53.391921+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:03.643179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679747+0000","last_change":"2024-04-25T10:17:52.397051+0000","last_active":"2024-04-25T10:17:53.679747+0000","last_peered":"2024-04-25T10:17:53.679747+0000","last_clean":"2024-04-25T10:17:53.679747+0000","last_became_active":"2024-04-25T10:17:52.393600+0000","last_became_peered":"2024-04-25T10:17:52.393600+0000","last_unstale":"2024-04-25T10:17:53.679747+0000","last_undegraded":"2024-04-25T10:17:53.679747+0000","last_fullsized":"2024-04-25T10:17:53.679747+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:49.917637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.30","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396070+0000","last_change":"2024-04-25T10:17:52.385023+0000","last_active":"2024-04-25T10:17:53.396070+0000","last_peered":"2024-04-25T10:17:53.396070+0000","last_clean":"2024-04-25T10:17:53.396070+0000","last_became_active":"2024-04-25T10:17:52.384157+0000","last_became_peered":"2024-04-25T10:17:52.384157+0000","last_unstale":"2024-04-25T10:17:53.396070+0000","last_undegraded":"2024-04-25T10:17:53.396070+0000","last_fullsized":"2024-04-25T10:17:53.396070+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:08.834397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959105+0000","last_change":"2024-04-25T10:17:52.397097+0000","last_active":"2024-04-25T10:17:53.959105+0000","last_peered":"2024-04-25T10:17:53.959105+0000","last_clean":"2024-04-25T10:17:53.959105+0000","last_became_active":"2024-04-25T10:17:52.396832+0000","last_became_peered":"2024-04-25T10:17:52.396832+0000","last_unstale":"2024-04-25T10:17:53.959105+0000","last_undegraded":"2024-04-25T10:17:53.959105+0000","last_fullsized":"2024-04-25T10:17:53.959105+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:16:34.714664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.398174+0000","last_change":"2024-04-25T10:17:52.382934+0000","last_active":"2024-04-25T10:17:53.398174+0000","last_peered":"2024-04-25T10:17:53.398174+0000","last_clean":"2024-04-25T10:17:53.398174+0000","last_became_active":"2024-04-25T10:17:52.382648+0000","last_became_peered":"2024-04-25T10:17:52.382648+0000","last_unstale":"2024-04-25T10:17:53.398174+0000","last_undegraded":"2024-04-25T10:17:53.398174+0000","last_fullsized":"2024-04-25T10:17:53.398174+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:04:22.633919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680800+0000","last_change":"2024-04-25T10:17:52.395971+0000","last_active":"2024-04-25T10:17:53.680800+0000","last_peered":"2024-04-25T10:17:53.680800+0000","last_clean":"2024-04-25T10:17:53.680800+0000","last_became_active":"2024-04-25T10:17:52.395687+0000","last_became_peered":"2024-04-25T10:17:52.395687+0000","last_unstale":"2024-04-25T10:17:53.680800+0000","last_undegraded":"2024-04-25T10:17:53.680800+0000","last_fullsized":"2024-04-25T10:17:53.680800+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:52.265324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962442+0000","last_change":"2024-04-25T10:17:52.386081+0000","last_active":"2024-04-25T10:17:53.962442+0000","last_peered":"2024-04-25T10:17:53.962442+0000","last_clean":"2024-04-25T10:17:53.962442+0000","last_became_active":"2024-04-25T10:17:52.385803+0000","last_became_peered":"2024-04-25T10:17:52.385803+0000","last_unstale":"2024-04-25T10:17:53.962442+0000","last_undegraded":"2024-04-25T10:17:53.962442+0000","last_fullsized":"2024-04-25T10:17:53.962442+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:04:48.657580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960949+0000","last_change":"2024-04-25T10:17:52.381445+0000","last_active":"2024-04-25T10:17:53.960949+0000","last_peered":"2024-04-25T10:17:53.960949+0000","last_clean":"2024-04-25T10:17:53.960949+0000","last_became_active":"2024-04-25T10:17:52.381059+0000","last_became_peered":"2024-04-25T10:17:52.381059+0000","last_unstale":"2024-04-25T10:17:53.960949+0000","last_undegraded":"2024-04-25T10:17:53.960949+0000","last_fullsized":"2024-04-25T10:17:53.960949+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:47:29.040869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959691+0000","last_change":"2024-04-25T10:17:52.382845+0000","last_active":"2024-04-25T10:17:53.959691+0000","last_peered":"2024-04-25T10:17:53.959691+0000","last_clean":"2024-04-25T10:17:53.959691+0000","last_became_active":"2024-04-25T10:17:52.379000+0000","last_became_peered":"2024-04-25T10:17:52.379000+0000","last_unstale":"2024-04-25T10:17:53.959691+0000","last_undegraded":"2024-04-25T10:17:53.959691+0000","last_fullsized":"2024-04-25T10:17:53.959691+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:05.310910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391492+0000","last_change":"2024-04-25T10:17:52.379517+0000","last_active":"2024-04-25T10:17:53.391492+0000","last_peered":"2024-04-25T10:17:53.391492+0000","last_clean":"2024-04-25T10:17:53.391492+0000","last_became_active":"2024-04-25T10:17:52.379262+0000","last_became_peered":"2024-04-25T10:17:52.379262+0000","last_unstale":"2024-04-25T10:17:53.391492+0000","last_undegraded":"2024-04-25T10:17:53.391492+0000","last_fullsized":"2024-04-25T10:17:53.391492+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:14:39.027010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396411+0000","last_change":"2024-04-25T10:17:52.385243+0000","last_active":"2024-04-25T10:17:53.396411+0000","last_peered":"2024-04-25T10:17:53.396411+0000","last_clean":"2024-04-25T10:17:53.396411+0000","last_became_active":"2024-04-25T10:17:52.384636+0000","last_became_peered":"2024-04-25T10:17:52.384636+0000","last_unstale":"2024-04-25T10:17:53.396411+0000","last_undegraded":"2024-04-25T10:17:53.396411+0000","last_fullsized":"2024-04-25T10:17:53.396411+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:32.242017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961539+0000","last_change":"2024-04-25T10:17:52.398473+0000","last_active":"2024-04-25T10:17:53.961539+0000","last_peered":"2024-04-25T10:17:53.961539+0000","last_clean":"2024-04-25T10:17:53.961539+0000","last_became_active":"2024-04-25T10:17:52.397957+0000","last_became_peered":"2024-04-25T10:17:52.397957+0000","last_unstale":"2024-04-25T10:17:53.961539+0000","last_undegraded":"2024-04-25T10:17:53.961539+0000","last_fullsized":"2024-04-25T10:17:53.961539+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:07:21.300144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961737+0000","last_change":"2024-04-25T10:17:52.371633+0000","last_active":"2024-04-25T10:17:53.961737+0000","last_peered":"2024-04-25T10:17:53.961737+0000","last_clean":"2024-04-25T10:17:53.961737+0000","last_became_active":"2024-04-25T10:17:52.371270+0000","last_became_peered":"2024-04-25T10:17:52.371270+0000","last_unstale":"2024-04-25T10:17:53.961737+0000","last_undegraded":"2024-04-25T10:17:53.961737+0000","last_fullsized":"2024-04-25T10:17:53.961737+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:54:24.692043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959668+0000","last_change":"2024-04-25T10:17:52.381238+0000","last_active":"2024-04-25T10:17:53.959668+0000","last_peered":"2024-04-25T10:17:53.959668+0000","last_clean":"2024-04-25T10:17:53.959668+0000","last_became_active":"2024-04-25T10:17:52.379585+0000","last_became_peered":"2024-04-25T10:17:52.379585+0000","last_unstale":"2024-04-25T10:17:53.959668+0000","last_undegraded":"2024-04-25T10:17:53.959668+0000","last_fullsized":"2024-04-25T10:17:53.959668+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:18.040469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961595+0000","last_change":"2024-04-25T10:17:52.382150+0000","last_active":"2024-04-25T10:17:53.961595+0000","last_peered":"2024-04-25T10:17:53.961595+0000","last_clean":"2024-04-25T10:17:53.961595+0000","last_became_active":"2024-04-25T10:17:52.381896+0000","last_became_peered":"2024-04-25T10:17:52.381896+0000","last_unstale":"2024-04-25T10:17:53.961595+0000","last_undegraded":"2024-04-25T10:17:53.961595+0000","last_fullsized":"2024-04-25T10:17:53.961595+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:52:33.277448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960114+0000","last_change":"2024-04-25T10:17:52.385743+0000","last_active":"2024-04-25T10:17:53.960114+0000","last_peered":"2024-04-25T10:17:53.960114+0000","last_clean":"2024-04-25T10:17:53.960114+0000","last_became_active":"2024-04-25T10:17:52.385354+0000","last_became_peered":"2024-04-25T10:17:52.385354+0000","last_unstale":"2024-04-25T10:17:53.960114+0000","last_undegraded":"2024-04-25T10:17:53.960114+0000","last_fullsized":"2024-04-25T10:17:53.960114+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:50.427063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.398316+0000","last_change":"2024-04-25T10:17:52.380488+0000","last_active":"2024-04-25T10:17:53.398316+0000","last_peered":"2024-04-25T10:17:53.398316+0000","last_clean":"2024-04-25T10:17:53.398316+0000","last_became_active":"2024-04-25T10:17:52.380194+0000","last_became_peered":"2024-04-25T10:17:52.380194+0000","last_unstale":"2024-04-25T10:17:53.398316+0000","last_undegraded":"2024-04-25T10:17:53.398316+0000","last_fullsized":"2024-04-25T10:17:53.398316+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:36.131072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679976+0000","last_change":"2024-04-25T10:17:52.389838+0000","last_active":"2024-04-25T10:17:53.679976+0000","last_peered":"2024-04-25T10:17:53.679976+0000","last_clean":"2024-04-25T10:17:53.679976+0000","last_became_active":"2024-04-25T10:17:52.389529+0000","last_became_peered":"2024-04-25T10:17:52.389529+0000","last_unstale":"2024-04-25T10:17:53.679976+0000","last_undegraded":"2024-04-25T10:17:53.679976+0000","last_fullsized":"2024-04-25T10:17:53.679976+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:16.856105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.397840+0000","last_change":"2024-04-25T10:17:52.385102+0000","last_active":"2024-04-25T10:17:53.397840+0000","last_peered":"2024-04-25T10:17:53.397840+0000","last_clean":"2024-04-25T10:17:53.397840+0000","last_became_active":"2024-04-25T10:17:52.384801+0000","last_became_peered":"2024-04-25T10:17:52.384801+0000","last_unstale":"2024-04-25T10:17:53.397840+0000","last_undegraded":"2024-04-25T10:17:53.397840+0000","last_fullsized":"2024-04-25T10:17:53.397840+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:30.844244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959378+0000","last_change":"2024-04-25T10:17:52.381030+0000","last_active":"2024-04-25T10:17:53.959378+0000","last_peered":"2024-04-25T10:17:53.959378+0000","last_clean":"2024-04-25T10:17:53.959378+0000","last_became_active":"2024-04-25T10:17:52.380511+0000","last_became_peered":"2024-04-25T10:17:52.380511+0000","last_unstale":"2024-04-25T10:17:53.959378+0000","last_undegraded":"2024-04-25T10:17:53.959378+0000","last_fullsized":"2024-04-25T10:17:53.959378+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:50.315964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"158'4","reported_seq":18,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396726+0000","last_change":"2024-04-25T10:17:51.378796+0000","last_active":"2024-04-25T10:17:53.396726+0000","last_peered":"2024-04-25T10:17:53.396726+0000","last_clean":"2024-04-25T10:17:53.396726+0000","last_became_active":"2024-04-25T10:17:51.378504+0000","last_became_peered":"2024-04-25T10:17:51.378504+0000","last_unstale":"2024-04-25T10:17:53.396726+0000","last_undegraded":"2024-04-25T10:17:53.396726+0000","last_fullsized":"2024-04-25T10:17:53.396726+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:26:36.761276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960078+0000","last_change":"2024-04-25T10:17:52.381496+0000","last_active":"2024-04-25T10:17:53.960078+0000","last_peered":"2024-04-25T10:17:53.960078+0000","last_clean":"2024-04-25T10:17:53.960078+0000","last_became_active":"2024-04-25T10:17:52.381214+0000","last_became_peered":"2024-04-25T10:17:52.381214+0000","last_unstale":"2024-04-25T10:17:53.960078+0000","last_undegraded":"2024-04-25T10:17:53.960078+0000","last_fullsized":"2024-04-25T10:17:53.960078+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:03.518830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391899+0000","last_change":"2024-04-25T10:17:51.383528+0000","last_active":"2024-04-25T10:17:53.391899+0000","last_peered":"2024-04-25T10:17:53.391899+0000","last_clean":"2024-04-25T10:17:53.391899+0000","last_became_active":"2024-04-25T10:17:51.382449+0000","last_became_peered":"2024-04-25T10:17:51.382449+0000","last_unstale":"2024-04-25T10:17:53.391899+0000","last_undegraded":"2024-04-25T10:17:53.391899+0000","last_fullsized":"2024-04-25T10:17:53.391899+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:29:48.614198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391596+0000","last_change":"2024-04-25T10:17:52.393611+0000","last_active":"2024-04-25T10:17:53.391596+0000","last_peered":"2024-04-25T10:17:53.391596+0000","last_clean":"2024-04-25T10:17:53.391596+0000","last_became_active":"2024-04-25T10:17:52.393363+0000","last_became_peered":"2024-04-25T10:17:52.393363+0000","last_unstale":"2024-04-25T10:17:53.391596+0000","last_undegraded":"2024-04-25T10:17:53.391596+0000","last_fullsized":"2024-04-25T10:17:53.391596+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:38.803774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.681054+0000","last_change":"2024-04-25T10:17:51.378958+0000","last_active":"2024-04-25T10:17:53.681054+0000","last_peered":"2024-04-25T10:17:53.681054+0000","last_clean":"2024-04-25T10:17:53.681054+0000","last_became_active":"2024-04-25T10:17:51.378655+0000","last_became_peered":"2024-04-25T10:17:51.378655+0000","last_unstale":"2024-04-25T10:17:53.681054+0000","last_undegraded":"2024-04-25T10:17:53.681054+0000","last_fullsized":"2024-04-25T10:17:53.681054+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:35:50.024871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960446+0000","last_change":"2024-04-25T10:17:52.386786+0000","last_active":"2024-04-25T10:17:53.960446+0000","last_peered":"2024-04-25T10:17:53.960446+0000","last_clean":"2024-04-25T10:17:53.960446+0000","last_became_active":"2024-04-25T10:17:52.385813+0000","last_became_peered":"2024-04-25T10:17:52.385813+0000","last_unstale":"2024-04-25T10:17:53.960446+0000","last_undegraded":"2024-04-25T10:17:53.960446+0000","last_fullsized":"2024-04-25T10:17:53.960446+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:34:21.393109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962589+0000","last_change":"2024-04-25T10:17:51.377735+0000","last_active":"2024-04-25T10:17:53.962589+0000","last_peered":"2024-04-25T10:17:53.962589+0000","last_clean":"2024-04-25T10:17:53.962589+0000","last_became_active":"2024-04-25T10:17:51.376589+0000","last_became_peered":"2024-04-25T10:17:51.376589+0000","last_unstale":"2024-04-25T10:17:53.962589+0000","last_undegraded":"2024-04-25T10:17:53.962589+0000","last_fullsized":"2024-04-25T10:17:53.962589+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:55:04.624140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958972+0000","last_change":"2024-04-25T10:17:52.379579+0000","last_active":"2024-04-25T10:17:53.958972+0000","last_peered":"2024-04-25T10:17:53.958972+0000","last_clean":"2024-04-25T10:17:53.958972+0000","last_became_active":"2024-04-25T10:17:52.379299+0000","last_became_peered":"2024-04-25T10:17:52.379299+0000","last_unstale":"2024-04-25T10:17:53.958972+0000","last_undegraded":"2024-04-25T10:17:53.958972+0000","last_fullsized":"2024-04-25T10:17:53.958972+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:20:49.917637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959382+0000","last_change":"2024-04-25T10:17:51.379258+0000","last_active":"2024-04-25T10:17:53.959382+0000","last_peered":"2024-04-25T10:17:53.959382+0000","last_clean":"2024-04-25T10:17:53.959382+0000","last_became_active":"2024-04-25T10:17:51.378916+0000","last_became_peered":"2024-04-25T10:17:51.378916+0000","last_unstale":"2024-04-25T10:17:53.959382+0000","last_undegraded":"2024-04-25T10:17:53.959382+0000","last_fullsized":"2024-04-25T10:17:53.959382+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.703271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962002+0000","last_change":"2024-04-25T10:17:52.386364+0000","last_active":"2024-04-25T10:17:53.962002+0000","last_peered":"2024-04-25T10:17:53.962002+0000","last_clean":"2024-04-25T10:17:53.962002+0000","last_became_active":"2024-04-25T10:17:52.383228+0000","last_became_peered":"2024-04-25T10:17:52.383228+0000","last_unstale":"2024-04-25T10:17:53.962002+0000","last_undegraded":"2024-04-25T10:17:53.962002+0000","last_fullsized":"2024-04-25T10:17:53.962002+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:37:32.237783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391685+0000","last_change":"2024-04-25T10:17:51.376585+0000","last_active":"2024-04-25T10:17:53.391685+0000","last_peered":"2024-04-25T10:17:53.391685+0000","last_clean":"2024-04-25T10:17:53.391685+0000","last_became_active":"2024-04-25T10:17:51.376266+0000","last_became_peered":"2024-04-25T10:17:51.376266+0000","last_unstale":"2024-04-25T10:17:53.391685+0000","last_undegraded":"2024-04-25T10:17:53.391685+0000","last_fullsized":"2024-04-25T10:17:53.391685+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.293522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962029+0000","last_change":"2024-04-25T10:17:52.368799+0000","last_active":"2024-04-25T10:17:53.962029+0000","last_peered":"2024-04-25T10:17:53.962029+0000","last_clean":"2024-04-25T10:17:53.962029+0000","last_became_active":"2024-04-25T10:17:52.368494+0000","last_became_peered":"2024-04-25T10:17:52.368494+0000","last_unstale":"2024-04-25T10:17:53.962029+0000","last_undegraded":"2024-04-25T10:17:53.962029+0000","last_fullsized":"2024-04-25T10:17:53.962029+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:38:18.305204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958599+0000","last_change":"2024-04-25T10:17:51.383223+0000","last_active":"2024-04-25T10:17:53.958599+0000","last_peered":"2024-04-25T10:17:53.958599+0000","last_clean":"2024-04-25T10:17:53.958599+0000","last_became_active":"2024-04-25T10:17:51.382880+0000","last_became_peered":"2024-04-25T10:17:51.382880+0000","last_unstale":"2024-04-25T10:17:53.958599+0000","last_undegraded":"2024-04-25T10:17:53.958599+0000","last_fullsized":"2024-04-25T10:17:53.958599+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:22.503464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396783+0000","last_change":"2024-04-25T10:17:52.395475+0000","last_active":"2024-04-25T10:17:53.396783+0000","last_peered":"2024-04-25T10:17:53.396783+0000","last_clean":"2024-04-25T10:17:53.396783+0000","last_became_active":"2024-04-25T10:17:52.395115+0000","last_became_peered":"2024-04-25T10:17:52.395115+0000","last_unstale":"2024-04-25T10:17:53.396783+0000","last_undegraded":"2024-04-25T10:17:53.396783+0000","last_fullsized":"2024-04-25T10:17:53.396783+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:48.858300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958401+0000","last_change":"2024-04-25T10:17:51.382057+0000","last_active":"2024-04-25T10:17:53.958401+0000","last_peered":"2024-04-25T10:17:53.958401+0000","last_clean":"2024-04-25T10:17:53.958401+0000","last_became_active":"2024-04-25T10:17:51.381426+0000","last_became_peered":"2024-04-25T10:17:51.381426+0000","last_unstale":"2024-04-25T10:17:53.958401+0000","last_undegraded":"2024-04-25T10:17:53.958401+0000","last_fullsized":"2024-04-25T10:17:53.958401+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:29:51.095017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959178+0000","last_change":"2024-04-25T10:17:52.380780+0000","last_active":"2024-04-25T10:17:53.959178+0000","last_peered":"2024-04-25T10:17:53.959178+0000","last_clean":"2024-04-25T10:17:53.959178+0000","last_became_active":"2024-04-25T10:17:52.379987+0000","last_became_peered":"2024-04-25T10:17:52.379987+0000","last_unstale":"2024-04-25T10:17:53.959178+0000","last_undegraded":"2024-04-25T10:17:53.959178+0000","last_fullsized":"2024-04-25T10:17:53.959178+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:43.879780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961055+0000","last_change":"2024-04-25T10:17:51.365010+0000","last_active":"2024-04-25T10:17:53.961055+0000","last_peered":"2024-04-25T10:17:53.961055+0000","last_clean":"2024-04-25T10:17:53.961055+0000","last_became_active":"2024-04-25T10:17:51.364712+0000","last_became_peered":"2024-04-25T10:17:51.364712+0000","last_unstale":"2024-04-25T10:17:53.961055+0000","last_undegraded":"2024-04-25T10:17:53.961055+0000","last_fullsized":"2024-04-25T10:17:53.961055+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:49.298576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962218+0000","last_change":"2024-04-25T10:17:52.392583+0000","last_active":"2024-04-25T10:17:53.962218+0000","last_peered":"2024-04-25T10:17:53.962218+0000","last_clean":"2024-04-25T10:17:53.962218+0000","last_became_active":"2024-04-25T10:17:52.392225+0000","last_became_peered":"2024-04-25T10:17:52.392225+0000","last_unstale":"2024-04-25T10:17:53.962218+0000","last_undegraded":"2024-04-25T10:17:53.962218+0000","last_fullsized":"2024-04-25T10:17:53.962218+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:04.680743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"158'3","reported_seq":17,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396715+0000","last_change":"2024-04-25T10:17:51.382259+0000","last_active":"2024-04-25T10:17:53.396715+0000","last_peered":"2024-04-25T10:17:53.396715+0000","last_clean":"2024-04-25T10:17:53.396715+0000","last_became_active":"2024-04-25T10:17:51.381960+0000","last_became_peered":"2024-04-25T10:17:51.381960+0000","last_unstale":"2024-04-25T10:17:53.396715+0000","last_undegraded":"2024-04-25T10:17:53.396715+0000","last_fullsized":"2024-04-25T10:17:53.396715+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:52:24.670982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962422+0000","last_change":"2024-04-25T10:17:52.389846+0000","last_active":"2024-04-25T10:17:53.962422+0000","last_peered":"2024-04-25T10:17:53.962422+0000","last_clean":"2024-04-25T10:17:53.962422+0000","last_became_active":"2024-04-25T10:17:52.388663+0000","last_became_peered":"2024-04-25T10:17:52.388663+0000","last_unstale":"2024-04-25T10:17:53.962422+0000","last_undegraded":"2024-04-25T10:17:53.962422+0000","last_fullsized":"2024-04-25T10:17:53.962422+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:19.429395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"158'6","reported_seq":20,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962055+0000","last_change":"2024-04-25T10:17:51.378190+0000","last_active":"2024-04-25T10:17:53.962055+0000","last_peered":"2024-04-25T10:17:53.962055+0000","last_clean":"2024-04-25T10:17:53.962055+0000","last_became_active":"2024-04-25T10:17:51.377878+0000","last_became_peered":"2024-04-25T10:17:51.377878+0000","last_unstale":"2024-04-25T10:17:53.962055+0000","last_undegraded":"2024-04-25T10:17:53.962055+0000","last_fullsized":"2024-04-25T10:17:53.962055+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.244741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.397806+0000","last_change":"2024-04-25T10:17:52.383868+0000","last_active":"2024-04-25T10:17:53.397806+0000","last_peered":"2024-04-25T10:17:53.397806+0000","last_clean":"2024-04-25T10:17:53.397806+0000","last_became_active":"2024-04-25T10:17:52.383558+0000","last_became_peered":"2024-04-25T10:17:52.383558+0000","last_unstale":"2024-04-25T10:17:53.397806+0000","last_undegraded":"2024-04-25T10:17:53.397806+0000","last_fullsized":"2024-04-25T10:17:53.397806+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:42:08.834397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392530+0000","last_change":"2024-04-25T10:17:51.377387+0000","last_active":"2024-04-25T10:17:53.392530+0000","last_peered":"2024-04-25T10:17:53.392530+0000","last_clean":"2024-04-25T10:17:53.392530+0000","last_became_active":"2024-04-25T10:17:51.376999+0000","last_became_peered":"2024-04-25T10:17:51.376999+0000","last_unstale":"2024-04-25T10:17:53.392530+0000","last_undegraded":"2024-04-25T10:17:53.392530+0000","last_fullsized":"2024-04-25T10:17:53.392530+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:42:49.298576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960335+0000","last_change":"2024-04-25T10:17:52.384209+0000","last_active":"2024-04-25T10:17:53.960335+0000","last_peered":"2024-04-25T10:17:53.960335+0000","last_clean":"2024-04-25T10:17:53.960335+0000","last_became_active":"2024-04-25T10:17:52.383948+0000","last_became_peered":"2024-04-25T10:17:52.383948+0000","last_unstale":"2024-04-25T10:17:53.960335+0000","last_undegraded":"2024-04-25T10:17:53.960335+0000","last_fullsized":"2024-04-25T10:17:53.960335+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:14:13.010219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960496+0000","last_change":"2024-04-25T10:17:51.374071+0000","last_active":"2024-04-25T10:17:53.960496+0000","last_peered":"2024-04-25T10:17:53.960496+0000","last_clean":"2024-04-25T10:17:53.960496+0000","last_became_active":"2024-04-25T10:17:51.371382+0000","last_became_peered":"2024-04-25T10:17:51.371382+0000","last_unstale":"2024-04-25T10:17:53.960496+0000","last_undegraded":"2024-04-25T10:17:53.960496+0000","last_fullsized":"2024-04-25T10:17:53.960496+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:06.177634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680288+0000","last_change":"2024-04-25T10:17:52.394834+0000","last_active":"2024-04-25T10:17:53.680288+0000","last_peered":"2024-04-25T10:17:53.680288+0000","last_clean":"2024-04-25T10:17:53.680288+0000","last_became_active":"2024-04-25T10:17:52.394550+0000","last_became_peered":"2024-04-25T10:17:52.394550+0000","last_unstale":"2024-04-25T10:17:53.680288+0000","last_undegraded":"2024-04-25T10:17:53.680288+0000","last_fullsized":"2024-04-25T10:17:53.680288+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:48.155978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680178+0000","last_change":"2024-04-25T10:17:51.383883+0000","last_active":"2024-04-25T10:17:53.680178+0000","last_peered":"2024-04-25T10:17:53.680178+0000","last_clean":"2024-04-25T10:17:53.680178+0000","last_became_active":"2024-04-25T10:17:51.383352+0000","last_became_peered":"2024-04-25T10:17:51.383352+0000","last_unstale":"2024-04-25T10:17:53.680178+0000","last_undegraded":"2024-04-25T10:17:53.680178+0000","last_fullsized":"2024-04-25T10:17:53.680178+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:55:22.703271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962006+0000","last_change":"2024-04-25T10:17:52.375739+0000","last_active":"2024-04-25T10:17:53.962006+0000","last_peered":"2024-04-25T10:17:53.962006+0000","last_clean":"2024-04-25T10:17:53.962006+0000","last_became_active":"2024-04-25T10:17:52.375434+0000","last_became_peered":"2024-04-25T10:17:52.375434+0000","last_unstale":"2024-04-25T10:17:53.962006+0000","last_undegraded":"2024-04-25T10:17:53.962006+0000","last_fullsized":"2024-04-25T10:17:53.962006+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:38:07.674007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"158'8","reported_seq":22,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958676+0000","last_change":"2024-04-25T10:17:51.382235+0000","last_active":"2024-04-25T10:17:53.958676+0000","last_peered":"2024-04-25T10:17:53.958676+0000","last_clean":"2024-04-25T10:17:53.958676+0000","last_became_active":"2024-04-25T10:17:51.381803+0000","last_became_peered":"2024-04-25T10:17:51.381803+0000","last_unstale":"2024-04-25T10:17:53.958676+0000","last_undegraded":"2024-04-25T10:17:53.958676+0000","last_fullsized":"2024-04-25T10:17:53.958676+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:05.311623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959127+0000","last_change":"2024-04-25T10:17:52.383371+0000","last_active":"2024-04-25T10:17:53.959127+0000","last_peered":"2024-04-25T10:17:53.959127+0000","last_clean":"2024-04-25T10:17:53.959127+0000","last_became_active":"2024-04-25T10:17:52.383109+0000","last_became_peered":"2024-04-25T10:17:52.383109+0000","last_unstale":"2024-04-25T10:17:53.959127+0000","last_undegraded":"2024-04-25T10:17:53.959127+0000","last_fullsized":"2024-04-25T10:17:53.959127+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:52.265324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961827+0000","last_change":"2024-04-25T10:17:51.378215+0000","last_active":"2024-04-25T10:17:53.961827+0000","last_peered":"2024-04-25T10:17:53.961827+0000","last_clean":"2024-04-25T10:17:53.961827+0000","last_became_active":"2024-04-25T10:17:51.377905+0000","last_became_peered":"2024-04-25T10:17:51.377905+0000","last_unstale":"2024-04-25T10:17:53.961827+0000","last_undegraded":"2024-04-25T10:17:53.961827+0000","last_fullsized":"2024-04-25T10:17:53.961827+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:20.734563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.398063+0000","last_change":"2024-04-25T10:17:52.395871+0000","last_active":"2024-04-25T10:17:53.398063+0000","last_peered":"2024-04-25T10:17:53.398063+0000","last_clean":"2024-04-25T10:17:53.398063+0000","last_became_active":"2024-04-25T10:17:52.394979+0000","last_became_peered":"2024-04-25T10:17:52.394979+0000","last_unstale":"2024-04-25T10:17:53.398063+0000","last_undegraded":"2024-04-25T10:17:53.398063+0000","last_fullsized":"2024-04-25T10:17:53.398063+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:32.242017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960228+0000","last_change":"2024-04-25T10:17:51.367493+0000","last_active":"2024-04-25T10:17:53.960228+0000","last_peered":"2024-04-25T10:17:53.960228+0000","last_clean":"2024-04-25T10:17:53.960228+0000","last_became_active":"2024-04-25T10:17:51.367097+0000","last_became_peered":"2024-04-25T10:17:51.367097+0000","last_unstale":"2024-04-25T10:17:53.960228+0000","last_undegraded":"2024-04-25T10:17:53.960228+0000","last_fullsized":"2024-04-25T10:17:53.960228+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:14:35.759748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679894+0000","last_change":"2024-04-25T10:17:52.391148+0000","last_active":"2024-04-25T10:17:53.679894+0000","last_peered":"2024-04-25T10:17:53.679894+0000","last_clean":"2024-04-25T10:17:53.679894+0000","last_became_active":"2024-04-25T10:17:52.390826+0000","last_became_peered":"2024-04-25T10:17:52.390826+0000","last_unstale":"2024-04-25T10:17:53.679894+0000","last_undegraded":"2024-04-25T10:17:53.679894+0000","last_fullsized":"2024-04-25T10:17:53.679894+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:58:53.074943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"158'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392758+0000","last_change":"2024-04-25T10:17:51.383023+0000","last_active":"2024-04-25T10:17:53.392758+0000","last_peered":"2024-04-25T10:17:53.392758+0000","last_clean":"2024-04-25T10:17:53.392758+0000","last_became_active":"2024-04-25T10:17:51.382697+0000","last_became_peered":"2024-04-25T10:17:51.382697+0000","last_unstale":"2024-04-25T10:17:53.392758+0000","last_undegraded":"2024-04-25T10:17:53.392758+0000","last_fullsized":"2024-04-25T10:17:53.392758+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:17.023081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680211+0000","last_change":"2024-04-25T10:17:52.391324+0000","last_active":"2024-04-25T10:17:53.680211+0000","last_peered":"2024-04-25T10:17:53.680211+0000","last_clean":"2024-04-25T10:17:53.680211+0000","last_became_active":"2024-04-25T10:17:52.391014+0000","last_became_peered":"2024-04-25T10:17:52.391014+0000","last_unstale":"2024-04-25T10:17:53.680211+0000","last_undegraded":"2024-04-25T10:17:53.680211+0000","last_fullsized":"2024-04-25T10:17:53.680211+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.982915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959535+0000","last_change":"2024-04-25T10:17:51.381541+0000","last_active":"2024-04-25T10:17:53.959535+0000","last_peered":"2024-04-25T10:17:53.959535+0000","last_clean":"2024-04-25T10:17:53.959535+0000","last_became_active":"2024-04-25T10:17:51.380331+0000","last_became_peered":"2024-04-25T10:17:51.380331+0000","last_unstale":"2024-04-25T10:17:53.959535+0000","last_undegraded":"2024-04-25T10:17:53.959535+0000","last_fullsized":"2024-04-25T10:17:53.959535+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:22.437023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961887+0000","last_change":"2024-04-25T10:17:52.383748+0000","last_active":"2024-04-25T10:17:53.961887+0000","last_peered":"2024-04-25T10:17:53.961887+0000","last_clean":"2024-04-25T10:17:53.961887+0000","last_became_active":"2024-04-25T10:17:52.383479+0000","last_became_peered":"2024-04-25T10:17:52.383479+0000","last_unstale":"2024-04-25T10:17:53.961887+0000","last_undegraded":"2024-04-25T10:17:53.961887+0000","last_fullsized":"2024-04-25T10:17:53.961887+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:25.862802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392664+0000","last_change":"2024-04-25T10:17:51.386166+0000","last_active":"2024-04-25T10:17:53.392664+0000","last_peered":"2024-04-25T10:17:53.392664+0000","last_clean":"2024-04-25T10:17:53.392664+0000","last_became_active":"2024-04-25T10:17:51.385874+0000","last_became_peered":"2024-04-25T10:17:51.385874+0000","last_unstale":"2024-04-25T10:17:53.392664+0000","last_undegraded":"2024-04-25T10:17:53.392664+0000","last_fullsized":"2024-04-25T10:17:53.392664+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:16.244741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392511+0000","last_change":"2024-04-25T10:17:52.384487+0000","last_active":"2024-04-25T10:17:53.392511+0000","last_peered":"2024-04-25T10:17:53.392511+0000","last_clean":"2024-04-25T10:17:53.392511+0000","last_became_active":"2024-04-25T10:17:52.384223+0000","last_became_peered":"2024-04-25T10:17:52.384223+0000","last_unstale":"2024-04-25T10:17:53.392511+0000","last_undegraded":"2024-04-25T10:17:53.392511+0000","last_fullsized":"2024-04-25T10:17:53.392511+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:03.518830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679710+0000","last_change":"2024-04-25T10:17:51.380152+0000","last_active":"2024-04-25T10:17:53.679710+0000","last_peered":"2024-04-25T10:17:53.679710+0000","last_clean":"2024-04-25T10:17:53.679710+0000","last_became_active":"2024-04-25T10:17:51.379860+0000","last_became_peered":"2024-04-25T10:17:51.379860+0000","last_unstale":"2024-04-25T10:17:53.679710+0000","last_undegraded":"2024-04-25T10:17:53.679710+0000","last_fullsized":"2024-04-25T10:17:53.679710+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:39:05.311623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391913+0000","last_change":"2024-04-25T10:17:52.385210+0000","last_active":"2024-04-25T10:17:53.391913+0000","last_peered":"2024-04-25T10:17:53.391913+0000","last_clean":"2024-04-25T10:17:53.391913+0000","last_became_active":"2024-04-25T10:17:52.384316+0000","last_became_peered":"2024-04-25T10:17:52.384316+0000","last_unstale":"2024-04-25T10:17:53.391913+0000","last_undegraded":"2024-04-25T10:17:53.391913+0000","last_fullsized":"2024-04-25T10:17:53.391913+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:08:55.497122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959105+0000","last_change":"2024-04-25T10:17:51.369127+0000","last_active":"2024-04-25T10:17:53.959105+0000","last_peered":"2024-04-25T10:17:53.959105+0000","last_clean":"2024-04-25T10:17:53.959105+0000","last_became_active":"2024-04-25T10:17:51.368836+0000","last_became_peered":"2024-04-25T10:17:51.368836+0000","last_unstale":"2024-04-25T10:17:53.959105+0000","last_undegraded":"2024-04-25T10:17:53.959105+0000","last_fullsized":"2024-04-25T10:17:53.959105+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:02:38.528377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960175+0000","last_change":"2024-04-25T10:17:52.384144+0000","last_active":"2024-04-25T10:17:53.960175+0000","last_peered":"2024-04-25T10:17:53.960175+0000","last_clean":"2024-04-25T10:17:53.960175+0000","last_became_active":"2024-04-25T10:17:52.383856+0000","last_became_peered":"2024-04-25T10:17:52.383856+0000","last_unstale":"2024-04-25T10:17:53.960175+0000","last_undegraded":"2024-04-25T10:17:53.960175+0000","last_fullsized":"2024-04-25T10:17:53.960175+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:38.320219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959683+0000","last_change":"2024-04-25T10:17:51.382409+0000","last_active":"2024-04-25T10:17:53.959683+0000","last_peered":"2024-04-25T10:17:53.959683+0000","last_clean":"2024-04-25T10:17:53.959683+0000","last_became_active":"2024-04-25T10:17:51.382048+0000","last_became_peered":"2024-04-25T10:17:51.382048+0000","last_unstale":"2024-04-25T10:17:53.959683+0000","last_undegraded":"2024-04-25T10:17:53.959683+0000","last_fullsized":"2024-04-25T10:17:53.959683+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:44:00.400583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.679541+0000","last_change":"2024-04-25T10:17:52.388275+0000","last_active":"2024-04-25T10:17:53.679541+0000","last_peered":"2024-04-25T10:17:53.679541+0000","last_clean":"2024-04-25T10:17:53.679541+0000","last_became_active":"2024-04-25T10:17:52.387979+0000","last_became_peered":"2024-04-25T10:17:52.387979+0000","last_unstale":"2024-04-25T10:17:53.679541+0000","last_undegraded":"2024-04-25T10:17:53.679541+0000","last_fullsized":"2024-04-25T10:17:53.679541+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:42:10.407438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"158'2","reported_seq":16,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.392183+0000","last_change":"2024-04-25T10:17:51.382866+0000","last_active":"2024-04-25T10:17:53.392183+0000","last_peered":"2024-04-25T10:17:53.392183+0000","last_clean":"2024-04-25T10:17:53.392183+0000","last_became_active":"2024-04-25T10:17:51.382164+0000","last_became_peered":"2024-04-25T10:17:51.382164+0000","last_unstale":"2024-04-25T10:17:53.392183+0000","last_undegraded":"2024-04-25T10:17:53.392183+0000","last_fullsized":"2024-04-25T10:17:53.392183+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:02:20.734563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959067+0000","last_change":"2024-04-25T10:17:52.390924+0000","last_active":"2024-04-25T10:17:53.959067+0000","last_peered":"2024-04-25T10:17:53.959067+0000","last_clean":"2024-04-25T10:17:53.959067+0000","last_became_active":"2024-04-25T10:17:52.390258+0000","last_became_peered":"2024-04-25T10:17:52.390258+0000","last_unstale":"2024-04-25T10:17:53.959067+0000","last_undegraded":"2024-04-25T10:17:53.959067+0000","last_fullsized":"2024-04-25T10:17:53.959067+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:56:33.982915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"159'1","reported_seq":15,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961335+0000","last_change":"2024-04-25T10:17:51.370374+0000","last_active":"2024-04-25T10:17:53.961335+0000","last_peered":"2024-04-25T10:17:53.961335+0000","last_clean":"2024-04-25T10:17:53.961335+0000","last_became_active":"2024-04-25T10:17:51.369266+0000","last_became_peered":"2024-04-25T10:17:51.369266+0000","last_unstale":"2024-04-25T10:17:53.961335+0000","last_undegraded":"2024-04-25T10:17:53.961335+0000","last_fullsized":"2024-04-25T10:17:53.961335+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:04:04.293522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960959+0000","last_change":"2024-04-25T10:17:52.372429+0000","last_active":"2024-04-25T10:17:53.960959+0000","last_peered":"2024-04-25T10:17:53.960959+0000","last_clean":"2024-04-25T10:17:53.960959+0000","last_became_active":"2024-04-25T10:17:52.366412+0000","last_became_peered":"2024-04-25T10:17:52.366412+0000","last_unstale":"2024-04-25T10:17:53.960959+0000","last_undegraded":"2024-04-25T10:17:53.960959+0000","last_fullsized":"2024-04-25T10:17:53.960959+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:40:39.199231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961493+0000","last_change":"2024-04-25T10:17:51.379048+0000","last_active":"2024-04-25T10:17:53.961493+0000","last_peered":"2024-04-25T10:17:53.961493+0000","last_clean":"2024-04-25T10:17:53.961493+0000","last_became_active":"2024-04-25T10:17:51.378700+0000","last_became_peered":"2024-04-25T10:17:51.378700+0000","last_unstale":"2024-04-25T10:17:53.961493+0000","last_undegraded":"2024-04-25T10:17:53.961493+0000","last_fullsized":"2024-04-25T10:17:53.961493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:39:37.302831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959073+0000","last_change":"2024-04-25T10:17:52.382677+0000","last_active":"2024-04-25T10:17:53.959073+0000","last_peered":"2024-04-25T10:17:53.959073+0000","last_clean":"2024-04-25T10:17:53.959073+0000","last_became_active":"2024-04-25T10:17:52.382396+0000","last_became_peered":"2024-04-25T10:17:52.382396+0000","last_unstale":"2024-04-25T10:17:53.959073+0000","last_undegraded":"2024-04-25T10:17:53.959073+0000","last_fullsized":"2024-04-25T10:17:53.959073+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:35:48.155978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.962474+0000","last_change":"2024-04-25T10:17:51.383501+0000","last_active":"2024-04-25T10:17:53.962474+0000","last_peered":"2024-04-25T10:17:53.962474+0000","last_clean":"2024-04-25T10:17:53.962474+0000","last_became_active":"2024-04-25T10:17:51.381248+0000","last_became_peered":"2024-04-25T10:17:51.381248+0000","last_unstale":"2024-04-25T10:17:53.962474+0000","last_undegraded":"2024-04-25T10:17:53.962474+0000","last_fullsized":"2024-04-25T10:17:53.962474+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:49.392115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396180+0000","last_change":"2024-04-25T10:17:52.382153+0000","last_active":"2024-04-25T10:17:53.396180+0000","last_peered":"2024-04-25T10:17:53.396180+0000","last_clean":"2024-04-25T10:17:53.396180+0000","last_became_active":"2024-04-25T10:17:52.381688+0000","last_became_peered":"2024-04-25T10:17:52.381688+0000","last_unstale":"2024-04-25T10:17:53.396180+0000","last_undegraded":"2024-04-25T10:17:53.396180+0000","last_fullsized":"2024-04-25T10:17:53.396180+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:56:30.844244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.396121+0000","last_change":"2024-04-25T10:17:51.384528+0000","last_active":"2024-04-25T10:17:53.396121+0000","last_peered":"2024-04-25T10:17:53.396121+0000","last_clean":"2024-04-25T10:17:53.396121+0000","last_became_active":"2024-04-25T10:17:51.384176+0000","last_became_peered":"2024-04-25T10:17:51.384176+0000","last_unstale":"2024-04-25T10:17:53.396121+0000","last_undegraded":"2024-04-25T10:17:53.396121+0000","last_fullsized":"2024-04-25T10:17:53.396121+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:03:48.896090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959995+0000","last_change":"2024-04-25T10:17:52.385687+0000","last_active":"2024-04-25T10:17:53.959995+0000","last_peered":"2024-04-25T10:17:53.959995+0000","last_clean":"2024-04-25T10:17:53.959995+0000","last_became_active":"2024-04-25T10:17:52.385381+0000","last_became_peered":"2024-04-25T10:17:52.385381+0000","last_unstale":"2024-04-25T10:17:53.959995+0000","last_undegraded":"2024-04-25T10:17:53.959995+0000","last_fullsized":"2024-04-25T10:17:53.959995+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:59:07.195022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961934+0000","last_change":"2024-04-25T10:17:51.366496+0000","last_active":"2024-04-25T10:17:53.961934+0000","last_peered":"2024-04-25T10:17:53.961934+0000","last_clean":"2024-04-25T10:17:53.961934+0000","last_became_active":"2024-04-25T10:17:51.364699+0000","last_became_peered":"2024-04-25T10:17:51.364699+0000","last_unstale":"2024-04-25T10:17:53.961934+0000","last_undegraded":"2024-04-25T10:17:53.961934+0000","last_fullsized":"2024-04-25T10:17:53.961934+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:10:49.409675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.960587+0000","last_change":"2024-04-25T10:17:52.383196+0000","last_active":"2024-04-25T10:17:53.960587+0000","last_peered":"2024-04-25T10:17:53.960587+0000","last_clean":"2024-04-25T10:17:53.960587+0000","last_became_active":"2024-04-25T10:17:52.382938+0000","last_became_peered":"2024-04-25T10:17:52.382938+0000","last_unstale":"2024-04-25T10:17:53.960587+0000","last_undegraded":"2024-04-25T10:17:53.960587+0000","last_fullsized":"2024-04-25T10:17:53.960587+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:04:22.716552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.680346+0000","last_change":"2024-04-25T10:17:51.379727+0000","last_active":"2024-04-25T10:17:53.680346+0000","last_peered":"2024-04-25T10:17:53.680346+0000","last_clean":"2024-04-25T10:17:53.680346+0000","last_became_active":"2024-04-25T10:17:51.379202+0000","last_became_peered":"2024-04-25T10:17:51.379202+0000","last_unstale":"2024-04-25T10:17:53.680346+0000","last_undegraded":"2024-04-25T10:17:53.680346+0000","last_fullsized":"2024-04-25T10:17:53.680346+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:53:23.325966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961582+0000","last_change":"2024-04-25T10:17:52.385481+0000","last_active":"2024-04-25T10:17:53.961582+0000","last_peered":"2024-04-25T10:17:53.961582+0000","last_clean":"2024-04-25T10:17:53.961582+0000","last_became_active":"2024-04-25T10:17:52.385221+0000","last_became_peered":"2024-04-25T10:17:52.385221+0000","last_unstale":"2024-04-25T10:17:53.961582+0000","last_undegraded":"2024-04-25T10:17:53.961582+0000","last_fullsized":"2024-04-25T10:17:53.961582+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:00.789628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.391315+0000","last_change":"2024-04-25T10:17:51.384876+0000","last_active":"2024-04-25T10:17:53.391315+0000","last_peered":"2024-04-25T10:17:53.391315+0000","last_clean":"2024-04-25T10:17:53.391315+0000","last_became_active":"2024-04-25T10:17:51.384131+0000","last_became_peered":"2024-04-25T10:17:51.384131+0000","last_unstale":"2024-04-25T10:17:53.391315+0000","last_undegraded":"2024-04-25T10:17:53.391315+0000","last_fullsized":"2024-04-25T10:17:53.391315+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:42.862391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":12,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.959304+0000","last_change":"2024-04-25T10:17:52.374453+0000","last_active":"2024-04-25T10:17:53.959304+0000","last_peered":"2024-04-25T10:17:53.959304+0000","last_clean":"2024-04-25T10:17:53.959304+0000","last_became_active":"2024-04-25T10:17:52.374110+0000","last_became_peered":"2024-04-25T10:17:52.374110+0000","last_unstale":"2024-04-25T10:17:53.959304+0000","last_undegraded":"2024-04-25T10:17:53.959304+0000","last_fullsized":"2024-04-25T10:17:53.959304+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:51.334045+0000","last_clean_scrub_stamp":"2024-04-25T10:17:51.334045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:21:17.262129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":14,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.961009+0000","last_change":"2024-04-25T10:17:51.371004+0000","last_active":"2024-04-25T10:17:53.961009+0000","last_peered":"2024-04-25T10:17:53.961009+0000","last_clean":"2024-04-25T10:17:53.961009+0000","last_became_active":"2024-04-25T10:17:51.370705+0000","last_became_peered":"2024-04-25T10:17:51.370705+0000","last_unstale":"2024-04-25T10:17:53.961009+0000","last_undegraded":"2024-04-25T10:17:53.961009+0000","last_fullsized":"2024-04-25T10:17:53.961009+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:17:50.316657+0000","last_clean_scrub_stamp":"2024-04-25T10:17:50.316657+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:17.023081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":436,"reported_epoch":159,"state":"active+clean","last_fresh":"2024-04-25T10:17:53.958769+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:17:53.958769+0000","last_peered":"2024-04-25T10:17:53.958769+0000","last_clean":"2024-04-25T10:17:53.958769+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:17:53.958769+0000","last_undegraded":"2024-04-25T10:17:53.958769+0000","last_fullsized":"2024-04-25T10:17:53.958769+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":23,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575045,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60084,"kb_used_data":2532,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311756,"statfs":{"total":96636764160,"available":96575238144,"internally_reserved":0,"allocated":2592768,"data_stored":1529887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575045,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59504,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312336,"statfs":{"total":96636764160,"available":96575832064,"internally_reserved":0,"allocated":1998848,"data_stored":939519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575045,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55352,"kb_used_data":1960,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316488,"statfs":{"total":96636764160,"available":96580083712,"internally_reserved":0,"allocated":2007040,"data_stored":939643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575045,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87236,"kb_used_data":1968,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284604,"statfs":{"total":96636764160,"available":96547434496,"internally_reserved":0,"allocated":2015232,"data_stored":940283,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575045,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87236,"kb_used_data":1972,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284604,"statfs":{"total":96636764160,"available":96547434496,"internally_reserved":0,"allocated":2019328,"data_stored":941735,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575044,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59504,"kb_used_data":1952,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312336,"statfs":{"total":96636764160,"available":96575832064,"internally_reserved":0,"allocated":1998848,"data_stored":939519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575045,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59520,"kb_used_data":1964,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312320,"statfs":{"total":96636764160,"available":96575815680,"internally_reserved":0,"allocated":2011136,"data_stored":941131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575045,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47608,"kb_used_data":2540,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324232,"statfs":{"total":96636764160,"available":96588013568,"internally_reserved":0,"allocated":2600960,"data_stored":1529991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:18:02.262 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:18:02.262 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:18:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.513+0000 7f409dc00640 1 Processor -- start 2024-04-25T10:18:02.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.513+0000 7f409dc00640 1 -- start start 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.515+0000 7f409dc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4098154350 0x7f409806db20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.515+0000 7f409dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f40980b0e80 0x7f40980711d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.515+0000 7f409dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f409815ae90 con 0x7f4098154730 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.515+0000 7f409dc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409815b450 con 0x7f4098154350 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.515+0000 7f409dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f409815ac30 con 0x7f40980b0e80 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4098154350 0x7f409806db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4098154350 0x7f409806db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39784/0 (socket says 172.21.15.165:39784) 2024-04-25T10:18:02.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 -- 172.21.15.165:0/1662193395 learned_addr learned my addr 172.21.15.165:0/1662193395 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 -- 172.21.15.165:0/1662193395 >> v2:172.21.15.172:3300/0 conn(0x7f40980b0e80 msgr2=0x7f40980711d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f409d200640 1 --2- 172.21.15.165:0/1662193395 >> v2:172.21.15.172:3300/0 conn(0x7f40980b0e80 0x7f40980711d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 --2- 172.21.15.165:0/1662193395 >> v2:172.21.15.172:3300/0 conn(0x7f40980b0e80 0x7f40980711d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 -- 172.21.15.165:0/1662193395 >> v1:172.21.15.172:6789/0 conn(0x7f4098154730 legacy=0x7f409815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.516+0000 7f4097e00640 1 -- 172.21.15.165:0/1662193395 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409815b170 con 0x7f4098154350 2024-04-25T10:18:02.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.517+0000 7f4097e00640 1 --2- 172.21.15.165:0/1662193395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154350 0x7f409806db20 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f40980ced40 tx=0x7f408802a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ee398d60516711 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.519+0000 7f4097400640 1 -- 172.21.15.165:0/1662193395 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088037070 con 0x7f4098154350 2024-04-25T10:18:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.519+0000 7f4097400640 1 -- 172.21.15.165:0/1662193395 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408802ae10 con 0x7f4098154350 2024-04-25T10:18:02.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.519+0000 7f4097400640 1 -- 172.21.15.165:0/1662193395 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4088032660 con 0x7f4098154350 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 -- 172.21.15.165:0/1662193395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154350 msgr2=0x7f409806db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 --2- 172.21.15.165:0/1662193395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154350 0x7f409806db20 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f40980ced40 tx=0x7f408802a100 comp rx=0 tx=0).stop 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 -- 172.21.15.165:0/1662193395 shutdown_connections 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 --2- 172.21.15.165:0/1662193395 >> v2:172.21.15.172:3300/0 conn(0x7f40980b0e80 0x7f40980711d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 --2- 172.21.15.165:0/1662193395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154350 0x7f409806db20 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 -- 172.21.15.165:0/1662193395 >> 172.21.15.165:0/1662193395 conn(0x7f4098061cc0 msgr2=0x7f409806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 -- 172.21.15.165:0/1662193395 shutdown_connections 2024-04-25T10:18:02.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.520+0000 7f409dc00640 1 -- 172.21.15.165:0/1662193395 wait complete. 2024-04-25T10:18:02.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.522+0000 7f409dc00640 1 Processor -- start 2024-04-25T10:18:02.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.522+0000 7f409dc00640 1 -- start start 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40980b0e80 0x7f409814d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4098154350 0x7f40981562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 0x7f4098156810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409806e060 con 0x7f4098154730 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f409806d600 con 0x7f40980b0e80 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4098071710 con 0x7f4098154350 2024-04-25T10:18:02.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 0x7f4098156810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 0x7f4098156810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39800/0 (socket says 172.21.15.165:39800) 2024-04-25T10:18:02.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.523+0000 7f409d200640 1 -- 172.21.15.165:0/1052683340 learned_addr learned my addr 172.21.15.165:0/1052683340 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f4097e00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40980b0e80 0x7f409814d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f408fe00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4098154350 0x7f40981562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409d200640 1 -- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4098154350 msgr2=0x7f40981562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409d200640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4098154350 0x7f40981562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409d200640 1 -- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40980b0e80 msgr2=0x7f409814d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409d200640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40980b0e80 0x7f409814d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409d200640 1 -- 172.21.15.165:0/1052683340 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098157030 con 0x7f4098154730 2024-04-25T10:18:02.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f4097e00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40980b0e80 0x7f409814d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f408fe00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4098154350 0x7f40981562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409d200640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 0x7f4098156810 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f409000cd40 tx=0x7f409000a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090003b90 con 0x7f4098154730 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4090003d30 con 0x7f4098154730 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4090004240 con 0x7f4098154730 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.524+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40981572d0 con 0x7f4098154730 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.525+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4098157890 con 0x7f4098154730 2024-04-25T10:18:02.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.526+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4098073990 con 0x7f4098154730 2024-04-25T10:18:02.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.529+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f40900043e0 con 0x7f4098154730 2024-04-25T10:18:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.530+0000 7f4094c00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f407406b4d0 0x7f407406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.530+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f409008b1d0 con 0x7f4098154730 2024-04-25T10:18:02.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.531+0000 7f4097e00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f407406b4d0 0x7f407406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:02.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.531+0000 7f4097e00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f407406b4d0 0x7f407406d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4088034000 tx=0x7f408802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:02.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.532+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f409005a030 con 0x7f4098154730 2024-04-25T10:18:02.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.795+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f409806e750 con 0x7f4098154730 2024-04-25T10:18:02.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.799+0000 7f4094c00640 1 -- 172.21.15.165:0/1052683340 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) v1 ==== 190+0+147 (secure 0 0 0) 0x7f4090059930 con 0x7f4098154730 2024-04-25T10:18:02.802 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:18:02.802 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:18:02.803 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:18:02.803 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:18:02.803 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:18:02.803 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.801+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f407406b4d0 msgr2=0x7f407406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.801+0000 7f409dc00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f407406b4d0 0x7f407406d990 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f4088034000 tx=0x7f408802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.801+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 msgr2=0x7f4098156810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.801+0000 7f409dc00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 0x7f4098156810 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f409000cd40 tx=0x7f409000a660 comp rx=0 tx=0).stop 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 shutdown_connections 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f407406b4d0 0x7f407406d990 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4098154730 0x7f4098156810 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4098154350 0x7f40981562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 --2- 172.21.15.165:0/1052683340 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f40980b0e80 0x7f409814d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 >> 172.21.15.165:0/1052683340 conn(0x7f4098061cc0 msgr2=0x7f409814f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 shutdown_connections 2024-04-25T10:18:02.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:02.802+0000 7f409dc00640 1 -- 172.21.15.165:0/1052683340 wait complete. 2024-04-25T10:18:02.824 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:03.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.010+0000 7fd631c00640 1 Processor -- start 2024-04-25T10:18:03.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.010+0000 7fd631c00640 1 -- start start 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.011+0000 7fd631c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd62c144440 0x7fd62c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.011+0000 7fd631c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd62c071530 0x7fd62c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.011+0000 7fd631c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd62c15bf30 con 0x7fd62c0cd860 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.011+0000 7fd631c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c0719a0 con 0x7fd62c071530 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.011+0000 7fd631c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd62c15c860 con 0x7fd62c144440 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd623400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd62c071530 0x7fd62c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd631200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd62c0cd860 0x7fd62c06d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42236/0 (socket says 172.21.15.165:42236) 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd631200640 1 -- 172.21.15.165:0/2106498289 learned_addr learned my addr 172.21.15.165:0/2106498289 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:03.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd623400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd62c071530 0x7fd62c0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39810/0 (socket says 172.21.15.165:39810) 2024-04-25T10:18:03.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd623400640 1 -- 172.21.15.165:0/2106498289 >> v2:172.21.15.172:3300/0 conn(0x7fd62c144440 msgr2=0x7fd62c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd62be00640 1 --2- 172.21.15.165:0/2106498289 >> v2:172.21.15.172:3300/0 conn(0x7fd62c144440 0x7fd62c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd62b400640 1 -- 172.21.15.165:0/2106498289 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385396328 0 0) 0x7fd62c15bf30 con 0x7fd62c0cd860 2024-04-25T10:18:03.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd623400640 1 --2- 172.21.15.165:0/2106498289 >> v2:172.21.15.172:3300/0 conn(0x7fd62c144440 0x7fd62c070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd623400640 1 -- 172.21.15.165:0/2106498289 >> v1:172.21.15.165:6789/0 conn(0x7fd62c0cd860 legacy=0x7fd62c06d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.012+0000 7fd623400640 1 -- 172.21.15.165:0/2106498289 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c15c210 con 0x7fd62c071530 2024-04-25T10:18:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.013+0000 7fd623400640 1 --2- 172.21.15.165:0/2106498289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c0cd320 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fd62c06f470 tx=0x7fd61402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1139229d373bf40 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.013+0000 7fd62b400640 1 -- 172.21.15.165:0/2106498289 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614037070 con 0x7fd62c071530 2024-04-25T10:18:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.013+0000 7fd62b400640 1 -- 172.21.15.165:0/2106498289 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd614032440 con 0x7fd62c071530 2024-04-25T10:18:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.013+0000 7fd62b400640 1 -- 172.21.15.165:0/2106498289 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd614032810 con 0x7fd62c071530 2024-04-25T10:18:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 -- 172.21.15.165:0/2106498289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 msgr2=0x7fd62c0cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 --2- 172.21.15.165:0/2106498289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c0cd320 secure :-1 s=READY pgs=2273 cs=0 l=1 rev1=1 crypto rx=0x7fd62c06f470 tx=0x7fd61402a230 comp rx=0 tx=0).stop 2024-04-25T10:18:03.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 -- 172.21.15.165:0/2106498289 shutdown_connections 2024-04-25T10:18:03.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 --2- 172.21.15.165:0/2106498289 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c0cd320 unknown :-1 s=CLOSED pgs=2273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 --2- 172.21.15.165:0/2106498289 >> v2:172.21.15.172:3300/0 conn(0x7fd62c144440 0x7fd62c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 -- 172.21.15.165:0/2106498289 >> 172.21.15.165:0/2106498289 conn(0x7fd62c06abe0 msgr2=0x7fd62c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 -- 172.21.15.165:0/2106498289 shutdown_connections 2024-04-25T10:18:03.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.014+0000 7fd631c00640 1 -- 172.21.15.165:0/2106498289 wait complete. 2024-04-25T10:18:03.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.016+0000 7fd631c00640 1 Processor -- start 2024-04-25T10:18:03.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.016+0000 7fd631c00640 1 -- start start 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd631c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c1e46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd631c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd62c0cd860 0x7fd62c1e4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd631c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd62c144440 0x7fd62c12ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd631c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0cdcd0 con 0x7fd62c071530 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd631c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c15bc30 con 0x7fd62c0cd860 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd631c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c0cde50 con 0x7fd62c144440 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd62be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c1e46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd62be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c1e46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39820/0 (socket says 172.21.15.165:39820) 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.017+0000 7fd62be00640 1 -- 172.21.15.165:0/158686080 learned_addr learned my addr 172.21.15.165:0/158686080 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd623400640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd62c0cd860 0x7fd62c1e4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd631200640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd62c144440 0x7fd62c12ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd62be00640 1 -- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd62c144440 msgr2=0x7fd62c12ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd62be00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd62c144440 0x7fd62c12ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd62be00640 1 -- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd62c0cd860 msgr2=0x7fd62c1e4bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd62be00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd62c0cd860 0x7fd62c1e4bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd62be00640 1 -- 172.21.15.165:0/158686080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c12bc50 con 0x7fd62c071530 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd623400640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd62c0cd860 0x7fd62c1e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd62be00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c1e46b0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c820 tx=0x7fd61c00cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.018+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c015070 con 0x7fd62c071530 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.019+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd61c004240 con 0x7fd62c071530 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.019+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd61c00ce10 con 0x7fd62c071530 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.019+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c12bef0 con 0x7fd62c071530 2024-04-25T10:18:03.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.019+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c12c4b0 con 0x7fd62c071530 2024-04-25T10:18:03.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.020+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c073990 con 0x7fd62c071530 2024-04-25T10:18:03.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.021+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd61c007830 con 0x7fd62c071530 2024-04-25T10:18:03.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.022+0000 7fd628c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd60806b500 0x7fd60806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.022+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd61c08a2a0 con 0x7fd62c071530 2024-04-25T10:18:03.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.022+0000 7fd623400640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd60806b500 0x7fd60806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.023+0000 7fd623400640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd60806b500 0x7fd60806d9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd614034000 tx=0x7fd61402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.026+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd61c058ef0 con 0x7fd62c071530 2024-04-25T10:18:03.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.256+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd62c13d260 con 0x7fd62c071530 2024-04-25T10:18:03.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.257+0000 7fd628c00640 1 -- 172.21.15.165:0/158686080 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fd61c0587f0 con 0x7fd62c071530 2024-04-25T10:18:03.261 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:03.261 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:51.816118+0000","modified":"2024-04-25T10:17:54.899844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6742":{"gid":6742,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/1799333237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1799333237},{"type":"v1","addr":"172.21.15.172:6834","nonce":1799333237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:18:03.261 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 113 2024-04-25T10:18:03.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd60806b500 msgr2=0x7fd60806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd60806b500 0x7fd60806d9c0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fd614034000 tx=0x7fd61402e000 comp rx=0 tx=0).stop 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 msgr2=0x7fd62c1e46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c1e46b0 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7fd61c00c820 tx=0x7fd61c00cce0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 shutdown_connections 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd60806b500 0x7fd60806d9c0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd62c144440 0x7fd62c12ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd62c0cd860 0x7fd62c1e4bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 --2- 172.21.15.165:0/158686080 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd62c071530 0x7fd62c1e46b0 unknown :-1 s=CLOSED pgs=2274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 >> 172.21.15.165:0/158686080 conn(0x7fd62c06abe0 msgr2=0x7fd62c0c4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.260+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 shutdown_connections 2024-04-25T10:18:03.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.261+0000 7fd631c00640 1 -- 172.21.15.165:0/158686080 wait complete. 2024-04-25T10:18:03.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 113, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:17:51.816118+0000', 'modified': '2024-04-25T10:17:54.899844+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6742}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6742': {'gid': 6742, 'name': 'd', 'rank': 0, 'incarnation': 109, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.172:6834/1799333237', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.172:6832', 'nonce': 1799333237}, {'type': 'v1', 'addr': '172.21.15.172:6834', 'nonce': 1799333237}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:18:03.685 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:18:03.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-04-25T10:18:03.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.894+0000 7f3ee5600640 1 Processor -- start 2024-04-25T10:18:03.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.894+0000 7f3ee5600640 1 -- start start 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3ee0145140 0x7f3ee00ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3ee00c5200 0x7f3ee015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee5600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ee00c5650 con 0x7f3ee00ce950 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee5600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee00ced90 con 0x7f3ee00c5200 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ee0145520 con 0x7f3ee0145140 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee4c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3ee00c5200 0x7f3ee015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3edf400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3ee00ce950 0x7f3ee00c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42244/0 (socket says 172.21.15.165:42244) 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3ee4c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3ee00c5200 0x7f3ee015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39836/0 (socket says 172.21.15.165:39836) 2024-04-25T10:18:03.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3edf400640 1 -- 172.21.15.165:0/893484087 learned_addr learned my addr 172.21.15.165:0/893484087 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.896+0000 7f3edfe00640 1 --2- 172.21.15.165:0/893484087 >> v2:172.21.15.172:3300/0 conn(0x7f3ee0145140 0x7f3ee00ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.897+0000 7f3ee4c00640 1 -- 172.21.15.165:0/893484087 >> v2:172.21.15.172:3300/0 conn(0x7f3ee0145140 msgr2=0x7f3ee00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.897+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/893484087 >> v2:172.21.15.172:3300/0 conn(0x7f3ee0145140 0x7f3ee00ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.897+0000 7f3ee4c00640 1 -- 172.21.15.165:0/893484087 >> v1:172.21.15.165:6789/0 conn(0x7f3ee00ce950 legacy=0x7f3ee00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.897+0000 7f3ee4c00640 1 -- 172.21.15.165:0/893484087 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0071270 con 0x7f3ee00c5200 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.897+0000 7f3edea00640 1 -- 172.21.15.165:0/893484087 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059804691 0 0) 0x7f3ee00c5650 con 0x7f3ee00ce950 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.898+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/893484087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00c5200 0x7f3ee015bfb0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f3ee006f690 tx=0x7f3ed802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13da511a749c572d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.898+0000 7f3edea00640 1 -- 172.21.15.165:0/893484087 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8007740 con 0x7f3ee00c5200 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.898+0000 7f3edea00640 1 -- 172.21.15.165:0/893484087 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed80078e0 con 0x7f3ee00c5200 2024-04-25T10:18:03.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.898+0000 7f3edea00640 1 -- 172.21.15.165:0/893484087 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8007cb0 con 0x7f3ee00c5200 2024-04-25T10:18:03.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.898+0000 7f3ee5600640 1 -- 172.21.15.165:0/893484087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00c5200 msgr2=0x7f3ee015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.898+0000 7f3ee5600640 1 --2- 172.21.15.165:0/893484087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00c5200 0x7f3ee015bfb0 secure :-1 s=READY pgs=2276 cs=0 l=1 rev1=1 crypto rx=0x7f3ee006f690 tx=0x7f3ed802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.899+0000 7f3ee5600640 1 -- 172.21.15.165:0/893484087 shutdown_connections 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.899+0000 7f3ee5600640 1 --2- 172.21.15.165:0/893484087 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00c5200 0x7f3ee015bfb0 unknown :-1 s=CLOSED pgs=2276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.899+0000 7f3ee5600640 1 --2- 172.21.15.165:0/893484087 >> v2:172.21.15.172:3300/0 conn(0x7f3ee0145140 0x7f3ee00ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.899+0000 7f3ee5600640 1 -- 172.21.15.165:0/893484087 >> 172.21.15.165:0/893484087 conn(0x7f3ee0061cc0 msgr2=0x7f3ee006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.899+0000 7f3ee5600640 1 -- 172.21.15.165:0/893484087 shutdown_connections 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.899+0000 7f3ee5600640 1 -- 172.21.15.165:0/893484087 wait complete. 2024-04-25T10:18:03.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.900+0000 7f3ee5600640 1 Processor -- start 2024-04-25T10:18:03.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.901+0000 7f3ee5600640 1 -- start start 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee00c5200 0x7f3ee00cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 0x7f3ee00b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee0145140 0x7f3ee013f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0071b10 con 0x7f3ee00ce950 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee0071830 con 0x7f3ee0145140 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee015c4f0 con 0x7f3ee00c5200 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 0x7f3ee00b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 0x7f3ee00b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39850/0 (socket says 172.21.15.165:39850) 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 -- 172.21.15.165:0/214083218 learned_addr learned my addr 172.21.15.165:0/214083218 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edfe00640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee00c5200 0x7f3ee00cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee0145140 0x7f3ee013f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 -- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee00c5200 msgr2=0x7f3ee00cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee00c5200 0x7f3ee00cde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 -- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee0145140 msgr2=0x7f3ee013f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee0145140 0x7f3ee013f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.902+0000 7f3edf400640 1 -- 172.21.15.165:0/214083218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee0140a80 con 0x7f3ee00ce950 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.903+0000 7f3edf400640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 0x7f3ee00b18b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00c4d0 tx=0x7f3ecc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.903+0000 7f3ed7e00640 1 -- 172.21.15.165:0/214083218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc018070 con 0x7f3ee00ce950 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.903+0000 7f3ed7e00640 1 -- 172.21.15.165:0/214083218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc007500 con 0x7f3ee00ce950 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.903+0000 7f3ed7e00640 1 -- 172.21.15.165:0/214083218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004a20 con 0x7f3ee00ce950 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.903+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ee0140d60 con 0x7f3ee00ce950 2024-04-25T10:18:03.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.903+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ee0141840 con 0x7f3ee00ce950 2024-04-25T10:18:03.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.905+0000 7f3ed7e00640 1 -- 172.21.15.165:0/214083218 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3ecc01b070 con 0x7f3ee00ce950 2024-04-25T10:18:03.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.906+0000 7f3ed7e00640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ec006b4d0 0x7f3ec006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.906+0000 7f3ed7e00640 1 -- 172.21.15.165:0/214083218 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ecc089770 con 0x7f3ee00ce950 2024-04-25T10:18:03.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.907+0000 7f3edfe00640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ec006b4d0 0x7f3ec006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.907+0000 7f3ee99f3740 1 Processor -- start 2024-04-25T10:18:03.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.907+0000 7f3ee99f3740 1 -- start start 2024-04-25T10:18:03.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.907+0000 7f3edfe00640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ec006b4d0 0x7f3ec006d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006fd0 tx=0x7f3ed0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 0x563ab7a34970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a557a0 0x563ab7a0b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7970620 con 0x563ab7a0cb30 2024-04-25T10:18:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7919ca0 con 0x563ab7a557a0 2024-04-25T10:18:03.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab796c6c0 con 0x563ab7a34eb0 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 0x563ab7a34970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 0x563ab7a34970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39866/0 (socket says 172.21.15.165:39866) 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 -- 172.21.15.165:0/608088824 learned_addr learned my addr 172.21.15.165:0/608088824 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 -- 172.21.15.165:0/608088824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 msgr2=0x563ab7a0cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 -- 172.21.15.165:0/608088824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a557a0 msgr2=0x563ab7a0b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a557a0 0x563ab7a0b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.908+0000 7f3edfe00640 1 -- 172.21.15.165:0/608088824 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab7920f40 con 0x563ab7a0cb30 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.909+0000 7f3edfe00640 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 0x563ab7a34970 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000e810 tx=0x7f3ed0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce2f3bbe453ce2e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.909+0000 7f3ecbe00640 1 -- 172.21.15.165:0/608088824 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed0025070 con 0x563ab7a0cb30 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.909+0000 7f3ecbe00640 1 -- 172.21.15.165:0/608088824 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0015640 con 0x563ab7a0cb30 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.909+0000 7f3ecbe00640 1 -- 172.21.15.165:0/608088824 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed000f040 con 0x563ab7a0cb30 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 -- 172.21.15.165:0/608088824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 msgr2=0x563ab7a34970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 0x563ab7a34970 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7f3ed000e810 tx=0x7f3ed0011000 comp rx=0 tx=0).stop 2024-04-25T10:18:03.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 -- 172.21.15.165:0/608088824 shutdown_connections 2024-04-25T10:18:03.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a557a0 0x563ab7a0b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/608088824 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a0cb30 0x563ab7a34970 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 -- 172.21.15.165:0/608088824 >> 172.21.15.165:0/608088824 conn(0x563ab7a09d50 msgr2=0x563ab7a52730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 -- 172.21.15.165:0/608088824 shutdown_connections 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.910+0000 7f3ee99f3740 1 -- 172.21.15.165:0/608088824 wait complete. 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.911+0000 7f3ee99f3740 1 -- start start 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.911+0000 7f3ee99f3740 1 Processor -- start 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a250d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7a548b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7a344e0 con 0x563ab7a557a0 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab790f1f0 con 0x563ab7a34eb0 2024-04-25T10:18:03.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab7a33ed0 con 0x563ab7a3b150 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3ee99f3740 10 client.0 _ll_register_callbacks cb 0x563ab792cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a250d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a250d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39876/0 (socket says 172.21.15.165:39876) 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 -- 172.21.15.165:0/1988562674 learned_addr learned my addr 172.21.15.165:0/1988562674 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 -- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 msgr2=0x563ab7a548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7a548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 -- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 msgr2=0x563ab7a626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.912+0000 7f3edf400640 1 -- 172.21.15.165:0/1988562674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab792ec50 con 0x563ab7a557a0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3edf400640 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a250d0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc057c50 tx=0x7f3ecc057d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca241f1c6a5ff393 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ec8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ec8c00640 1 -- 172.21.15.165:0/1988562674 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc004dc0 con 0x563ab7a557a0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ec8c00640 1 -- 172.21.15.165:0/1988562674 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ecc057110 con 0x563ab7a557a0 2024-04-25T10:18:03.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ed0015640 con 0x563ab7a557a0 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ec8c00640 1 -- 172.21.15.165:0/1988562674 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ecc05e0c0 con 0x563ab7a557a0 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ee99f3740 10 client.10458 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x563ab76852d0 con 0x563ab7a557a0 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ec8c00640 1 -- 172.21.15.165:0/1988562674 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed0015640 con 0x563ab7a557a0 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ec8c00640 1 -- 172.21.15.165:0/1988562674 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ecc053a10 con 0x563ab7a557a0 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ee99f3740 10 client.10458 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ee99f3740 10 client.10458 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.913+0000 7f3ee99f3740 20 client.10458 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.914+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7a6e090 0x563ab7a70480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.914+0000 7f3ee99f3740 4 client.10458 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.914+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563ab797db60 con 0x563ab7a6e090 2024-04-25T10:18:03.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.914+0000 7f3edfe00640 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7a6e090 0x563ab7a70480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.915+0000 7f3edfe00640 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7a6e090 0x563ab7a70480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7660f70f56761aa7 server_cookie=2af4ab571e67bb82 in_seq=0 out_seq=0 2024-04-25T10:18:03.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.915+0000 7f3ec8c00640 10 client.10458 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ec8c00640 1 -- 172.21.15.165:0/1988562674 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563ab797db60 con 0x563ab7a6e090 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ec8c00640 10 client.10458 handle_command_reply: tid=0 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 client.10458 shutdown 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 msgr2=0x563ab7a250d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a250d0 secure :-1 s=READY pgs=2279 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc057c50 tx=0x7f3ecc057d80 comp rx=0 tx=0).stop 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 shutdown_connections 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7a6e090 0x563ab7a70480 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7a548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a250d0 unknown :-1 s=CLOSED pgs=2279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ec8c00640 0 client.10458 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1988562674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 >> 172.21.15.165:0/1988562674 conn(0x563ab7a0cb30 msgr2=0x563ab7a068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 shutdown_connections 2024-04-25T10:18:03.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1988562674 wait complete. 2024-04-25T10:18:03.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.918+0000 7f3ee99f3740 20 client.10458 trim_cache size 0 max 16384 2024-04-25T10:18:03.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.935+0000 7f3ee99f3740 1 Processor -- start 2024-04-25T10:18:03.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.935+0000 7f3ee99f3740 1 -- start start 2024-04-25T10:18:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a3b150 0x563ab7a2ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a2f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01ff270 con 0x563ab7a557a0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01e5f60 con 0x563ab7a3b150 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ee01feca0 con 0x563ab7a34eb0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a2f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a2f470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39878/0 (socket says 172.21.15.165:39878) 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.936+0000 7f3ee4c00640 1 -- 172.21.15.165:0/1034176980 learned_addr learned my addr 172.21.15.165:0/1034176980 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3edf400640 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a3b150 0x563ab7a2ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ee4c00640 1 -- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 msgr2=0x563ab7a0cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3edfe00640 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ee4c00640 1 -- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a3b150 msgr2=0x563ab7a2ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a3b150 0x563ab7a2ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ee4c00640 1 -- 172.21.15.165:0/1034176980 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563ab7a3abc0 con 0x563ab7a557a0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3edf400640 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a3b150 0x563ab7a2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a2f470 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009ae0 tx=0x7f3ed8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9b2cfbf8e24e90 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ecb400640 1 -- 172.21.15.165:0/1034176980 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8003c10 con 0x563ab7a557a0 2024-04-25T10:18:03.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ecb400640 1 -- 172.21.15.165:0/1034176980 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed8003db0 con 0x563ab7a557a0 2024-04-25T10:18:03.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.937+0000 7f3ecb400640 1 -- 172.21.15.165:0/1034176980 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8004260 con 0x563ab7a557a0 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1034176980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 msgr2=0x563ab7a2f470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a2f470 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009ae0 tx=0x7f3ed8004ab0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1034176980 shutdown_connections 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7a2f470 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a3b150 0x563ab7a2ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/1034176980 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a34eb0 0x563ab7a0cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1034176980 >> 172.21.15.165:0/1034176980 conn(0x563ab7a0cb30 msgr2=0x563ab7a39470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1034176980 shutdown_connections 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.938+0000 7f3ee99f3740 1 -- 172.21.15.165:0/1034176980 wait complete. 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.939+0000 7f3ee99f3740 1 -- start start 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.939+0000 7f3ee99f3740 1 Processor -- start 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a54230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7abc410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7abc950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7a3b5d0 con 0x563ab7a557a0 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563ab7a375f0 con 0x563ab7a34eb0 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563ab7a37fa0 con 0x563ab7a3b150 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee99f3740 10 client.0 _ll_register_callbacks cb 0x7f3ecc0583c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7abc950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7abc950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39892/0 (socket says 172.21.15.165:39892) 2024-04-25T10:18:03.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.940+0000 7f3ee4c00640 1 -- 172.21.15.165:0/3486511026 learned_addr learned my addr 172.21.15.165:0/3486511026 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3edf400640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7abc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3edfe00640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a54230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee4c00640 1 -- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 msgr2=0x563ab7abc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7abc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee4c00640 1 -- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 msgr2=0x563ab7a54230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a54230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee4c00640 1 -- 172.21.15.165:0/3486511026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563ab774f6b0 con 0x563ab7a557a0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee4c00640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7abc950 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009790 tx=0x7f3ed802f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be318a0a5409455 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ec8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ec8c00640 1 -- 172.21.15.165:0/3486511026 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ed8055060 con 0x563ab7a557a0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ec8c00640 1 -- 172.21.15.165:0/3486511026 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3ed8004790 con 0x563ab7a557a0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3ed8003db0 con 0x563ab7a557a0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ec8c00640 1 -- 172.21.15.165:0/3486511026 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ed805b050 con 0x563ab7a557a0 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee99f3740 10 client.10464 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:03.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.941+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x563ab7ac0d00 con 0x563ab7a557a0 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ec8c00640 1 -- 172.21.15.165:0/3486511026 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ed8003db0 con 0x563ab7a557a0 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ec8c00640 1 -- 172.21.15.165:0/3486511026 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3ed805e020 con 0x563ab7a557a0 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ee99f3740 10 client.10464 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ee99f3740 10 client.10464 resolve_mds: resolved name 'd' to daemon mds.d 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ee99f3740 20 client.10464 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7ac02a0 0x563ab7ac0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ee99f3740 4 client.10464 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563ab7977eb0 con 0x563ab7ac02a0 2024-04-25T10:18:03.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.942+0000 7f3edfe00640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7ac02a0 0x563ab7ac0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:03.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.943+0000 7f3edfe00640 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7ac02a0 0x563ab7ac0680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5821e070cfd0a65 server_cookie=a8e181e3b6a9b0ae in_seq=0 out_seq=0 2024-04-25T10:18:03.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.943+0000 7f3ec8c00640 10 client.10464 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ec8c00640 1 -- 172.21.15.165:0/3486511026 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x563ab7977eb0 con 0x563ab7ac02a0 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ec8c00640 10 client.10464 handle_command_reply: tid=0 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 client.10464 shutdown 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 msgr2=0x563ab7abc950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7abc950 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7f3ed8009790 tx=0x7f3ed802f000 comp rx=0 tx=0).stop 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 shutdown_connections 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563ab7a557a0 0x563ab7abc950 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563ab7ac02a0 0x563ab7ac0680 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563ab7a3b150 0x563ab7abc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 --2- 172.21.15.165:0/3486511026 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563ab7a34eb0 0x563ab7a54230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ec8c00640 0 client.10464 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.944+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 >> 172.21.15.165:0/3486511026 conn(0x563ab7a0cb30 msgr2=0x563ab7a2ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 shutdown_connections 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee99f3740 1 -- 172.21.15.165:0/3486511026 wait complete. 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee99f3740 20 client.10464 trim_cache size 0 max 16384 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 6742, 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 11.589096745000001, 2024-04-25T10:18:03.951 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 113, 2024-04-25T10:18:03.952 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 159, 2024-04-25T10:18:03.952 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 159, 2024-04-25T10:18:03.952 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 17.045945122999999 2024-04-25T10:18:03.952 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:18:03.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ec006b4d0 msgr2=0x7f3ec006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ec006b4d0 0x7f3ec006d990 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f3ed0006fd0 tx=0x7f3ed0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 msgr2=0x7f3ee00b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 0x7f3ee00b18b0 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f3ecc00c4d0 tx=0x7f3ecc00c990 comp rx=0 tx=0).stop 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 shutdown_connections 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ec006b4d0 0x7f3ec006d990 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3ee0145140 0x7f3ee013f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3ee00ce950 0x7f3ee00b18b0 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 --2- 172.21.15.165:0/214083218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3ee00c5200 0x7f3ee00cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 >> 172.21.15.165:0/214083218 conn(0x7f3ee0061cc0 msgr2=0x7f3ee00c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 shutdown_connections 2024-04-25T10:18:03.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:03.945+0000 7f3ee5600640 1 -- 172.21.15.165:0/214083218 wait complete. 2024-04-25T10:18:03.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:18:04.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.156+0000 7faad1000640 1 Processor -- start 2024-04-25T10:18:04.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.157+0000 7faad1000640 1 -- start start 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.158+0000 7faad1000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7faacc144750 0x7faacc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.158+0000 7faad1000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faacc06eb10 0x7faacc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.158+0000 7faad1000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc0cee60 0x7faacc15f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.158+0000 7faad1000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faacc144b30 con 0x7faacc06eb10 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.158+0000 7faad1000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faacc0712d0 con 0x7faacc144750 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.158+0000 7faad1000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc0715b0 con 0x7faacc0cee60 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faacc06eb10 0x7faacc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faacc06eb10 0x7faacc0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39898/0 (socket says 172.21.15.165:39898) 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 -- 172.21.15.165:0/1093831557 learned_addr learned my addr 172.21.15.165:0/1093831557 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:04.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faaca800640 1 --2- 172.21.15.165:0/1093831557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc0cee60 0x7faacc15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac9e00640 1 --2- 172.21.15.165:0/1093831557 >> v2:172.21.15.172:3300/0 conn(0x7faacc144750 0x7faacc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 -- 172.21.15.165:0/1093831557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc0cee60 msgr2=0x7faacc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 --2- 172.21.15.165:0/1093831557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc0cee60 0x7faacc15f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 -- 172.21.15.165:0/1093831557 >> v2:172.21.15.172:3300/0 conn(0x7faacc144750 msgr2=0x7faacc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 --2- 172.21.15.165:0/1093831557 >> v2:172.21.15.172:3300/0 conn(0x7faacc144750 0x7faacc06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.159+0000 7faac1400640 1 -- 172.21.15.165:0/1093831557 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc071890 con 0x7faacc06eb10 2024-04-25T10:18:04.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.160+0000 7faac1400640 1 --2- 172.21.15.165:0/1093831557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc06eb10 0x7faacc0ce920 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7faacc06f5d0 tx=0x7faab402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35ef460087fd3844 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faac9400640 1 -- 172.21.15.165:0/1093831557 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4037070 con 0x7faacc06eb10 2024-04-25T10:18:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faac9400640 1 -- 172.21.15.165:0/1093831557 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab4032440 con 0x7faacc06eb10 2024-04-25T10:18:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faac9400640 1 -- 172.21.15.165:0/1093831557 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4032810 con 0x7faacc06eb10 2024-04-25T10:18:04.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 -- 172.21.15.165:0/1093831557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc06eb10 msgr2=0x7faacc0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 --2- 172.21.15.165:0/1093831557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc06eb10 0x7faacc0ce920 secure :-1 s=READY pgs=2282 cs=0 l=1 rev1=1 crypto rx=0x7faacc06f5d0 tx=0x7faab402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 -- 172.21.15.165:0/1093831557 shutdown_connections 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 --2- 172.21.15.165:0/1093831557 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc0cee60 0x7faacc15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 --2- 172.21.15.165:0/1093831557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc06eb10 0x7faacc0ce920 unknown :-1 s=CLOSED pgs=2282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 --2- 172.21.15.165:0/1093831557 >> v2:172.21.15.172:3300/0 conn(0x7faacc144750 0x7faacc06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 -- 172.21.15.165:0/1093831557 >> 172.21.15.165:0/1093831557 conn(0x7faacc06abe0 msgr2=0x7faacc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 -- 172.21.15.165:0/1093831557 shutdown_connections 2024-04-25T10:18:04.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.161+0000 7faad1000640 1 -- 172.21.15.165:0/1093831557 wait complete. 2024-04-25T10:18:04.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.163+0000 7faad1000640 1 Processor -- start 2024-04-25T10:18:04.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.163+0000 7faad1000640 1 -- start start 2024-04-25T10:18:04.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faad1000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faacc06eb10 0x7faacc1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faad1000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 0x7faacc0b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faad1000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc144750 0x7faacc1f7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faad1000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc145450 con 0x7faacc0cee60 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faad1000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc0cf270 con 0x7faacc06eb10 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faad1000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faacc1455d0 con 0x7faacc144750 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faac1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 0x7faacc0b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faac1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 0x7faacc0b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39902/0 (socket says 172.21.15.165:39902) 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.164+0000 7faac1400640 1 -- 172.21.15.165:0/2692154129 learned_addr learned my addr 172.21.15.165:0/2692154129 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac9e00640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faacc06eb10 0x7faacc1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac1400640 1 -- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc144750 msgr2=0x7faacc1f7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac1400640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc144750 0x7faacc1f7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac1400640 1 -- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faacc06eb10 msgr2=0x7faacc1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac1400640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faacc06eb10 0x7faacc1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac1400640 1 -- 172.21.15.165:0/2692154129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faacc0b0470 con 0x7faacc0cee60 2024-04-25T10:18:04.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac9e00640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faacc06eb10 0x7faacc1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:04.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.165+0000 7faac1400640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 0x7faacc0b19d0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:04.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.166+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4041020 con 0x7faacc0cee60 2024-04-25T10:18:04.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.166+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab40306c0 con 0x7faacc0cee60 2024-04-25T10:18:04.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.166+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc0b0710 con 0x7faacc0cee60 2024-04-25T10:18:04.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.166+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab4037040 con 0x7faacc0cee60 2024-04-25T10:18:04.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.166+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc0b0cd0 con 0x7faacc0cee60 2024-04-25T10:18:04.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.167+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faacc073990 con 0x7faacc0cee60 2024-04-25T10:18:04.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.168+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faab4044020 con 0x7faacc0cee60 2024-04-25T10:18:04.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.169+0000 7faac2a00640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faaa806b4d0 0x7faaa806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:04.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.169+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faab40aabd0 con 0x7faacc0cee60 2024-04-25T10:18:04.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.170+0000 7faac9e00640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faaa806b4d0 0x7faaa806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:04.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.170+0000 7faac9e00640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faaa806b4d0 0x7faaa806d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faabc006fd0 tx=0x7faabc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:04.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.174+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faab404b030 con 0x7faacc0cee60 2024-04-25T10:18:04.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.416+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7faacc0d0510 con 0x7faacc0cee60 2024-04-25T10:18:04.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.417+0000 7faac2a00640 1 -- 172.21.15.165:0/2692154129 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v113) v1 ==== 53+0+73 (secure 0 0 0) 0x7faab40798b0 con 0x7faacc0cee60 2024-04-25T10:18:04.421 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:18:04.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faaa806b4d0 msgr2=0x7faaa806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faaa806b4d0 0x7faaa806d990 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7faabc006fd0 tx=0x7faabc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 msgr2=0x7faacc0b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 0x7faacc0b19d0 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7faab4000c00 tx=0x7faab40046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 shutdown_connections 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faaa806b4d0 0x7faaa806d990 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faacc144750 0x7faacc1f7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faacc0cee60 0x7faacc0b19d0 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 --2- 172.21.15.165:0/2692154129 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faacc06eb10 0x7faacc1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.420+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 >> 172.21.15.165:0/2692154129 conn(0x7faacc06abe0 msgr2=0x7faacc13e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.421+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 shutdown_connections 2024-04-25T10:18:04.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:04.421+0000 7faad1000640 1 -- 172.21.15.165:0/2692154129 wait complete. 2024-04-25T10:18:04.443 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:18:04.443 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:18:04.443 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:18:04.443 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:18:04.443 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:04.444 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:18:04.444 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:18:04.444 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:18:04.444 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:04.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:04.444 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:04.494 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41681sec preferred_lft 41681sec 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.495 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.496 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:18:04.497 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:18:04.497 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:18:04.497 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:18:04.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:04.498 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:18:04.577 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:18:04.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:04.581 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:18:04.627 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:18:04.627 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:18:04.628 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:18:04.628 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:18:04.628 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:18:04.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:04.629 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:18:04.629 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:18:04.629 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:18:04.629 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:18:04.629 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:18:04.629 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:18:04.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:04.827 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:18:04.877 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:18:04.878 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:04.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:04.878 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:18:04.958 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:04.958 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:05.012 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:18:05.012 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:18:05.013 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:05.063 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:18:05.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:05.064 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:05.193 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:05.193 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:05.193 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:18:05.212 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:05.213 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:05.213 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:18:05.213 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:18:05.213 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:05.238 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:18:05.239 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:18:05.242 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:18:05.242 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:18:05.242 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:18:05.242 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:18:05.242 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:18:05.246 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:18:05.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:05.246 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:05.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:05.319 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:18:05.392 INFO:teuthology.orchestra.run.smithi165.stdout:client6092 2024-04-25T10:18:05.392 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:18:05.392 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:05.448 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:18:05.448 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:18:05.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.654+0000 7f9391200640 1 Processor -- start 2024-04-25T10:18:05.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.654+0000 7f9391200640 1 -- start start 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9391200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f938c15b710 0x7f938c0cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9391200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 0x7f938c0accd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9391200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f938c06f0e0 con 0x7f938c06ec50 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9391200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f938c15e490 con 0x7f938c15b710 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9391200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0cfd80 con 0x7f938c0ac8b0 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9389400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f938c06ec50 0x7f938c15e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42250/0 (socket says 172.21.15.165:42250) 2024-04-25T10:18:05.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.656+0000 7f9389400640 1 -- 172.21.15.165:0/992827998 learned_addr learned my addr 172.21.15.165:0/992827998 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:05.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f938a800640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 0x7f938c0accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9389e00640 1 --2- 172.21.15.165:0/992827998 >> v2:172.21.15.172:3300/0 conn(0x7f938c15b710 0x7f938c0cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266954625 0 0) 0x7f938c06f0e0 con 0x7f938c06ec50 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9370003520 con 0x7f938c06ec50 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1391442985 0 0) 0x7f9370003520 con 0x7f938c06ec50 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 msgr2=0x7f938c0accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 0x7f938c0accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 >> v2:172.21.15.172:3300/0 conn(0x7f938c15b710 msgr2=0x7f938c0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 --2- 172.21.15.165:0/992827998 >> v2:172.21.15.172:3300/0 conn(0x7f938c15b710 0x7f938c0cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0d0060 con 0x7f938c06ec50 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9378001670 con 0x7f938c06ec50 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 >> v1:172.21.15.165:6789/0 conn(0x7f938c06ec50 legacy=0x7f938c15e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 0x7f93700039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9370003f30 0x7f9370004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370004d40 0x7f93700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c15e490 con 0x7f9370003f30 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c0cfd80 con 0x7f9370004d40 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.657+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937000b9c0 con 0x7f938c0ac8b0 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389400640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9370003f30 0x7f9370004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389e00640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 0x7f93700039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f938a800640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370004d40 0x7f93700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389400640 1 -- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 msgr2=0x7f93700039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389400640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c0ac8b0 0x7f93700039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389400640 1 -- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370004d40 msgr2=0x7f93700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389400640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370004d40 0x7f93700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.658+0000 7f9389400640 1 -- 172.21.15.165:0/992827998 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370003520 con 0x7f9370003f30 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.659+0000 7f9389400640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9370003f30 0x7f9370004800 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f937800c290 tx=0x7f9370005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85cc9b9ce0c86586 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.659+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378007d50 con 0x7f9370003f30 2024-04-25T10:18:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.659+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937802e070 con 0x7f9370003f30 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.659+0000 7f9388a00640 1 -- 172.21.15.165:0/992827998 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378005d70 con 0x7f9370003f30 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f9391200640 1 -- 172.21.15.165:0/992827998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9370003f30 msgr2=0x7f9370004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f9391200640 1 --2- 172.21.15.165:0/992827998 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9370003f30 0x7f9370004800 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f937800c290 tx=0x7f9370005830 comp rx=0 tx=0).stop 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f938a800640 1 -- 172.21.15.165:0/992827998 reap_dead start 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f9391200640 1 -- 172.21.15.165:0/992827998 shutdown_connections 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f9391200640 1 -- 172.21.15.165:0/992827998 >> 172.21.15.165:0/992827998 conn(0x7f938c06abe0 msgr2=0x7f938c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f9391200640 1 -- 172.21.15.165:0/992827998 shutdown_connections 2024-04-25T10:18:05.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.660+0000 7f9391200640 1 -- 172.21.15.165:0/992827998 wait complete. 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.662+0000 7f9391200640 1 Processor -- start 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.662+0000 7f9391200640 1 -- start start 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9391200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370003f30 0x7f938c14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9391200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 0x7f938c156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9391200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c1573c0 0x7f938c152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9391200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c15bc70 con 0x7f938c158350 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9391200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c15baf0 con 0x7f9370003f30 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9391200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c15f030 con 0x7f938c1573c0 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9389400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 0x7f938c156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9389400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 0x7f938c156e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39920/0 (socket says 172.21.15.165:39920) 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.663+0000 7f9389400640 1 -- 172.21.15.165:0/1721915014 learned_addr learned my addr 172.21.15.165:0/1721915014 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389e00640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370003f30 0x7f938c14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f938a800640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c1573c0 0x7f938c152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389400640 1 -- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c1573c0 msgr2=0x7f938c152f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389400640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c1573c0 0x7f938c152f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389400640 1 -- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370003f30 msgr2=0x7f938c14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389400640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370003f30 0x7f938c14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389400640 1 -- 172.21.15.165:0/1721915014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f938c0afe60 con 0x7f938c158350 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.664+0000 7f9389400640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 0x7f938c156e80 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f937800c3b0 tx=0x7f938c0d0230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.665+0000 7f9382000640 1 -- 172.21.15.165:0/1721915014 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378042070 con 0x7f938c158350 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.665+0000 7f9382000640 1 -- 172.21.15.165:0/1721915014 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9378007480 con 0x7f938c158350 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.665+0000 7f9382000640 1 -- 172.21.15.165:0/1721915014 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378001c80 con 0x7f938c158350 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.665+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f938c0b0140 con 0x7f938c158350 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.665+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f938c0b0700 con 0x7f938c158350 2024-04-25T10:18:05.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.667+0000 7f9382000640 1 -- 172.21.15.165:0/1721915014 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9378045070 con 0x7f938c158350 2024-04-25T10:18:05.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.668+0000 7f9382000640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f936c06b500 0x7f936c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.668+0000 7f9382000640 1 -- 172.21.15.165:0/1721915014 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f93780ab110 con 0x7f938c158350 2024-04-25T10:18:05.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.668+0000 7f9389e00640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f936c06b500 0x7f936c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.669+0000 7f9393f9b740 1 Processor -- start 2024-04-25T10:18:05.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.669+0000 7f9393f9b740 1 -- start start 2024-04-25T10:18:05.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.669+0000 7f9389e00640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f936c06b500 0x7f936c06d9c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f937c009a10 tx=0x7f937c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.669+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be62ec200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.669+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be6309c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be62c1b00 0x563be6307d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9393f9b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be6224d00 con 0x563be63096c0 2024-04-25T10:18:05.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be61cdf50 con 0x563be62c1b00 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be6220da0 con 0x563be62ebe20 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be6309c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be6309c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39932/0 (socket says 172.21.15.165:39932) 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 -- 172.21.15.165:0/2413344269 learned_addr learned my addr 172.21.15.165:0/2413344269 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389e00640 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be62ec200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 -- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 msgr2=0x563be62ec200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be62ec200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 -- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be62c1b00 msgr2=0x563be6307d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f938a800640 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be62c1b00 0x563be6307d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be62c1b00 0x563be6307d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.670+0000 7f9389400640 1 -- 172.21.15.165:0/2413344269 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563be61f1130 con 0x563be63096c0 2024-04-25T10:18:05.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.671+0000 7f9389400640 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be6309c10 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f9378075b50 tx=0x7f9378079160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=725166d0b328b7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.671+0000 7f9376a00640 1 -- 172.21.15.165:0/2413344269 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378077790 con 0x563be63096c0 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.671+0000 7f9376a00640 1 -- 172.21.15.165:0/2413344269 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937802e040 con 0x563be63096c0 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.671+0000 7f9376a00640 1 -- 172.21.15.165:0/2413344269 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9378075d60 con 0x563be63096c0 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.671+0000 7f9393f9b740 1 -- 172.21.15.165:0/2413344269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 msgr2=0x563be6309c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.671+0000 7f9393f9b740 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be6309c10 secure :-1 s=READY pgs=2287 cs=0 l=1 rev1=1 crypto rx=0x7f9378075b50 tx=0x7f9378079160 comp rx=0 tx=0).stop 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 -- 172.21.15.165:0/2413344269 shutdown_connections 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be62c1b00 0x563be6307d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be6309c10 unknown :-1 s=CLOSED pgs=2287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 --2- 172.21.15.165:0/2413344269 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be62ec200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 -- 172.21.15.165:0/2413344269 >> 172.21.15.165:0/2413344269 conn(0x563be62be430 msgr2=0x563be6306c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 -- 172.21.15.165:0/2413344269 shutdown_connections 2024-04-25T10:18:05.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 -- 172.21.15.165:0/2413344269 wait complete. 2024-04-25T10:18:05.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.672+0000 7f9393f9b740 1 -- start start 2024-04-25T10:18:05.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.673+0000 7f9393f9b740 1 Processor -- start 2024-04-25T10:18:05.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 0x563be62dd5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be63096c0 0x563be62ffb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be6308520 0x563be63089a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be61cdf50 con 0x563be62ebe20 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be61c3530 con 0x563be63096c0 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be6224d00 con 0x563be6308520 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9393f9b740 10 client.0 _ll_register_callbacks cb 0x563be621c430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 0x563be62dd5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 0x563be62dd5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39942/0 (socket says 172.21.15.165:39942) 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 -- 172.21.15.165:0/556587758 learned_addr learned my addr 172.21.15.165:0/556587758 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 -- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be6308520 msgr2=0x563be63089a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be6308520 0x563be63089a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 -- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be63096c0 msgr2=0x563be62ffb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389400640 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be63096c0 0x563be62ffb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be63096c0 0x563be62ffb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.674+0000 7f9389e00640 1 -- 172.21.15.165:0/556587758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563be61ebdd0 con 0x563be62ebe20 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f9389e00640 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 0x563be62dd5d0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f937c013f70 tx=0x7f937c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13c6bb83e9671618 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f936b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:05.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f936b400640 1 -- 172.21.15.165:0/556587758 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f937c027070 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f936b400640 1 -- 172.21.15.165:0/556587758 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f937c023820 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f937802e040 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f936b400640 1 -- 172.21.15.165:0/556587758 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f937c032050 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f9393f9b740 10 client.10482 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.675+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x563be5f393f0 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f936b400640 1 -- 172.21.15.165:0/556587758 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f937802e040 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f936b400640 1 -- 172.21.15.165:0/556587758 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f937c035020 con 0x563be62ebe20 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f9393f9b740 10 client.10482 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f9393f9b740 10 client.10482 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f9393f9b740 20 client.10482 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f9393f9b740 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62ef0d0 0x563be62ef560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f9393f9b740 4 client.10482 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.676+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563be6232240 con 0x563be62ef0d0 2024-04-25T10:18:05.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.677+0000 7f9389400640 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62ef0d0 0x563be62ef560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.678+0000 7f9389400640 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62ef0d0 0x563be62ef560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48a1a473b5e1f4fc server_cookie=46406a59450b9762 in_seq=0 out_seq=0 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.678+0000 7f936b400640 10 client.10482 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.679+0000 7f936b400640 1 -- 172.21.15.165:0/556587758 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563be6232240 con 0x563be62ef0d0 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.679+0000 7f936b400640 10 client.10482 handle_command_reply: tid=0 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.679+0000 7f9393f9b740 1 client.10482 shutdown 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 msgr2=0x563be62dd5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 0x563be62dd5d0 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f937c013f70 tx=0x7f937c004730 comp rx=0 tx=0).stop 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 shutdown_connections 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62ef0d0 0x563be62ef560 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be6308520 0x563be63089a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be63096c0 0x563be62ffb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 --2- 172.21.15.165:0/556587758 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be62ebe20 0x563be62dd5d0 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 >> 172.21.15.165:0/556587758 conn(0x563be62c1b00 msgr2=0x563be630aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f936b400640 0 client.10482 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:05.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 shutdown_connections 2024-04-25T10:18:05.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 1 -- 172.21.15.165:0/556587758 wait complete. 2024-04-25T10:18:05.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.680+0000 7f9393f9b740 20 client.10482 trim_cache size 0 max 16384 2024-04-25T10:18:05.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.693+0000 7f9393f9b740 1 Processor -- start 2024-04-25T10:18:05.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.693+0000 7f9393f9b740 1 -- start start 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63188e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6300500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9393f9b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c200c50 con 0x563be63096c0 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1ffe50 con 0x563be6308520 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1ff250 con 0x563be62ebe20 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fc5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39950/0 (socket says 172.21.15.165:39950) 2024-04-25T10:18:05.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 -- 172.21.15.165:0/886810424 learned_addr learned my addr 172.21.15.165:0/886810424 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:05.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 -- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 msgr2=0x563be63188e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9389e00640 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63188e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f9389400640 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6300500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63188e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 -- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 msgr2=0x563be6300500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6300500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.694+0000 7f938a800640 1 -- 172.21.15.165:0/886810424 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563be60037d0 con 0x563be63096c0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.695+0000 7f9389e00640 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.695+0000 7f938a800640 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fc5c0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f938c0ccf60 tx=0x7f9384003ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c254c0050ac56845 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.695+0000 7f9376000640 1 -- 172.21.15.165:0/886810424 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938401b020 con 0x563be63096c0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.695+0000 7f9376000640 1 -- 172.21.15.165:0/886810424 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384017470 con 0x563be63096c0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.695+0000 7f9376000640 1 -- 172.21.15.165:0/886810424 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384017840 con 0x563be63096c0 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 -- 172.21.15.165:0/886810424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 msgr2=0x563be62fc5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fc5c0 secure :-1 s=READY pgs=2289 cs=0 l=1 rev1=1 crypto rx=0x7f938c0ccf60 tx=0x7f9384003ed0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 -- 172.21.15.165:0/886810424 shutdown_connections 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fc5c0 unknown :-1 s=CLOSED pgs=2289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6300500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 --2- 172.21.15.165:0/886810424 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63188e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 -- 172.21.15.165:0/886810424 >> 172.21.15.165:0/886810424 conn(0x563be62c1b00 msgr2=0x563be6329650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 -- 172.21.15.165:0/886810424 shutdown_connections 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.696+0000 7f9393f9b740 1 -- 172.21.15.165:0/886810424 wait complete. 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.697+0000 7f9393f9b740 1 -- start start 2024-04-25T10:18:05.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.697+0000 7f9393f9b740 1 Processor -- start 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63552c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6307240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fcd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f938c1ff850 con 0x563be63096c0 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563be6352fa0 con 0x563be6308520 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563be6307bb0 con 0x563be62ebe20 2024-04-25T10:18:05.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9393f9b740 10 client.0 _ll_register_callbacks cb 0x563be62bc840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f938a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fcd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f938a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fcd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39966/0 (socket says 172.21.15.165:39966) 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f938a800640 1 -- 172.21.15.165:0/3524500381 learned_addr learned my addr 172.21.15.165:0/3524500381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9389400640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6307240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.698+0000 7f9389e00640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63552c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f938a800640 1 -- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 msgr2=0x563be63552c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f938a800640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63552c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f938a800640 1 -- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 msgr2=0x563be6307240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f938a800640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6307240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f938a800640 1 -- 172.21.15.165:0/3524500381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563be62fc5c0 con 0x563be63096c0 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f9389400640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6307240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f938a800640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fcd80 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f9384003f10 tx=0x7f938400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40877df17c9e0373 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f936b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f936b400640 1 -- 172.21.15.165:0/3524500381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9384004870 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f936b400640 1 -- 172.21.15.165:0/3524500381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f938402bc70 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.699+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9384017470 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f936b400640 1 -- 172.21.15.165:0/3524500381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9384021430 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 10 client.10488 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x563be62bbd30 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f936b400640 1 -- 172.21.15.165:0/3524500381 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9384017470 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f936b400640 1 -- 172.21.15.165:0/3524500381 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f9384032050 con 0x563be63096c0 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 10 client.10488 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 10 client.10488 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 20 client.10488 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62bb090 0x563be62ecae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 4 client.10488 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:18:05.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.700+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563be622c590 con 0x563be62bb090 2024-04-25T10:18:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.701+0000 7f9389e00640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62bb090 0x563be62ecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.701+0000 7f9389e00640 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62bb090 0x563be62ecae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d41ae6c213df6422 server_cookie=b03169d83774e25b in_seq=0 out_seq=0 2024-04-25T10:18:05.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.701+0000 7f936b400640 10 client.10488 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f936b400640 1 -- 172.21.15.165:0/3524500381 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563be622c590 con 0x563be62bb090 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f936b400640 10 client.10488 handle_command_reply: tid=0 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 client.10488 shutdown 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 msgr2=0x563be62fcd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fcd80 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f9384003f10 tx=0x7f938400b040 comp rx=0 tx=0).stop 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 shutdown_connections 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563be63096c0 0x563be62fcd80 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563be6308520 0x563be6307240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x563be62bb090 0x563be62ecae0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.702+0000 7f9393f9b740 1 --2- 172.21.15.165:0/3524500381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563be62ebe20 0x563be63552c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 >> 172.21.15.165:0/3524500381 conn(0x563be62c1b00 msgr2=0x563be6300620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f936b400640 0 client.10488 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 shutdown_connections 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9393f9b740 1 -- 172.21.15.165:0/3524500381 wait complete. 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9393f9b740 20 client.10488 trim_cache size 0 max 16384 2024-04-25T10:18:05.709 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f936c06b500 msgr2=0x7f936c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9391200640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f936c06b500 0x7f936c06d9c0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f937c009a10 tx=0x7f937c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 msgr2=0x7f938c156e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.703+0000 7f9391200640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 0x7f938c156e80 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f937800c3b0 tx=0x7f938c0d0230 comp rx=0 tx=0).stop 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 shutdown_connections 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f936c06b500 0x7f936c06d9c0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f938c1573c0 0x7f938c152f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f938c158350 0x7f938c156e80 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 --2- 172.21.15.165:0/1721915014 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9370003f30 0x7f938c14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 >> 172.21.15.165:0/1721915014 conn(0x7f938c06abe0 msgr2=0x7f938c15bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 shutdown_connections 2024-04-25T10:18:05.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.704+0000 7f9391200640 1 -- 172.21.15.165:0/1721915014 wait complete. 2024-04-25T10:18:05.724 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:18:05.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.905+0000 7f3026e00640 1 Processor -- start 2024-04-25T10:18:05.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.906+0000 7f3026e00640 1 -- start start 2024-04-25T10:18:05.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.907+0000 7f3026e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30200d06c0 0x7f3020168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.907+0000 7f3026e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30200d33f0 con 0x7f30200b93c0 2024-04-25T10:18:05.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.907+0000 7f3026e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3020161df0 con 0x7f302015db90 2024-04-25T10:18:05.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.907+0000 7f3026e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30200d1f50 con 0x7f30200d06c0 2024-04-25T10:18:05.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.907+0000 7f3025000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f30200b93c0 0x7f30200cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:42258/0 (socket says 172.21.15.165:42258) 2024-04-25T10:18:05.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.907+0000 7f3026400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30200d06c0 0x7f3020168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.908+0000 7f3025000640 1 -- 172.21.15.165:0/840943552 learned_addr learned my addr 172.21.15.165:0/840943552 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:05.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.908+0000 7f3026400640 1 -- 172.21.15.165:0/840943552 >> v1:172.21.15.172:6789/0 conn(0x7f302015db90 legacy=0x7f30201615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.908+0000 7f3026400640 1 -- 172.21.15.165:0/840943552 >> v1:172.21.15.165:6789/0 conn(0x7f30200b93c0 legacy=0x7f30200cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.908+0000 7f3026400640 1 -- 172.21.15.165:0/840943552 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30201620d0 con 0x7f30200d06c0 2024-04-25T10:18:05.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.908+0000 7f301fe00640 1 -- 172.21.15.165:0/840943552 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 327051587 0 0) 0x7f30200d33f0 con 0x7f30200b93c0 2024-04-25T10:18:05.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.909+0000 7f3026400640 1 --2- 172.21.15.165:0/840943552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200d06c0 0x7f3020168930 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f30180097a0 tx=0x7f301802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c597ccd7cbda7d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f301fe00640 1 -- 172.21.15.165:0/840943552 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018037070 con 0x7f30200d06c0 2024-04-25T10:18:05.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f301fe00640 1 -- 172.21.15.165:0/840943552 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3018032440 con 0x7f30200d06c0 2024-04-25T10:18:05.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f301fe00640 1 -- 172.21.15.165:0/840943552 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3018032810 con 0x7f30200d06c0 2024-04-25T10:18:05.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f3026e00640 1 -- 172.21.15.165:0/840943552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200d06c0 msgr2=0x7f3020168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f3026e00640 1 --2- 172.21.15.165:0/840943552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200d06c0 0x7f3020168930 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f30180097a0 tx=0x7f301802a230 comp rx=0 tx=0).stop 2024-04-25T10:18:05.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f3026e00640 1 -- 172.21.15.165:0/840943552 shutdown_connections 2024-04-25T10:18:05.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f3026e00640 1 --2- 172.21.15.165:0/840943552 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200d06c0 0x7f3020168930 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.910+0000 7f3026e00640 1 -- 172.21.15.165:0/840943552 >> 172.21.15.165:0/840943552 conn(0x7f30200b52b0 msgr2=0x7f30200b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:05.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.911+0000 7f3026e00640 1 -- 172.21.15.165:0/840943552 shutdown_connections 2024-04-25T10:18:05.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.911+0000 7f3026e00640 1 -- 172.21.15.165:0/840943552 wait complete. 2024-04-25T10:18:05.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.912+0000 7f3026e00640 1 Processor -- start 2024-04-25T10:18:05.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.912+0000 7f3026e00640 1 -- start start 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.913+0000 7f3026e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 0x7f30200ccb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.913+0000 7f3026e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30200d06c0 0x7f30201f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.913+0000 7f3026e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f302015db90 0x7f30201f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.913+0000 7f3026e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30200b68d0 con 0x7f30200b93c0 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.913+0000 7f3026e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30200d1be0 con 0x7f30200d06c0 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.913+0000 7f3026e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30200b6a50 con 0x7f302015db90 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 0x7f30200ccb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 0x7f30200ccb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39982/0 (socket says 172.21.15.165:39982) 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 -- 172.21.15.165:0/1043354161 learned_addr learned my addr 172.21.15.165:0/1043354161 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 -- 172.21.15.165:0/1043354161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f302015db90 msgr2=0x7f30201f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f302015db90 0x7f30201f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 -- 172.21.15.165:0/1043354161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30200d06c0 msgr2=0x7f30201f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30200d06c0 0x7f30201f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 -- 172.21.15.165:0/1043354161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30200ca9a0 con 0x7f30200b93c0 2024-04-25T10:18:05.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.914+0000 7f3025a00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 0x7f30200ccb30 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f301000c430 tx=0x7f301000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.915+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010017070 con 0x7f30200b93c0 2024-04-25T10:18:05.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.915+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30100052d0 con 0x7f30200b93c0 2024-04-25T10:18:05.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.915+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3010003830 con 0x7f30200b93c0 2024-04-25T10:18:05.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.915+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30200cac80 con 0x7f30200b93c0 2024-04-25T10:18:05.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.915+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30200cd070 con 0x7f30200b93c0 2024-04-25T10:18:05.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.916+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3020073990 con 0x7f30200b93c0 2024-04-25T10:18:05.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.917+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f301001a070 con 0x7f30200b93c0 2024-04-25T10:18:05.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.919+0000 7f301d600640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f300006b4d0 0x7f300006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:05.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.919+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f30100897b0 con 0x7f30200b93c0 2024-04-25T10:18:05.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.919+0000 7f3025000640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f300006b4d0 0x7f300006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:05.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.920+0000 7f3025000640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f300006b4d0 0x7f300006d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f300c000f80 tx=0x7f300c001670 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:05.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:05.923+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3010058400 con 0x7f30200b93c0 2024-04-25T10:18:06.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.158+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f30200d14b0 con 0x7f30200b93c0 2024-04-25T10:18:06.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.159+0000 7f301d600640 1 -- 172.21.15.165:0/1043354161 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3010057d00 con 0x7f30200b93c0 2024-04-25T10:18:06.162 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:06.162 INFO:teuthology.orchestra.run.smithi165.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:18:06.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f300006b4d0 msgr2=0x7f300006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f300006b4d0 0x7f300006d990 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f300c000f80 tx=0x7f300c001670 comp rx=0 tx=0).stop 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 msgr2=0x7f30200ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 0x7f30200ccb30 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f301000c430 tx=0x7f301000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 shutdown_connections 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f300006b4d0 0x7f300006d990 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f302015db90 0x7f30201f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30200d06c0 0x7f30201f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 --2- 172.21.15.165:0/1043354161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30200b93c0 0x7f30200ccb30 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 >> 172.21.15.165:0/1043354161 conn(0x7f30200b52b0 msgr2=0x7f30200c6b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.162+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 shutdown_connections 2024-04-25T10:18:06.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.163+0000 7f3026e00640 1 -- 172.21.15.165:0/1043354161 wait complete. 2024-04-25T10:18:06.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:06.184 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub start '~mdsdir' recursive 2024-04-25T10:18:06.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.383+0000 7f4968e00640 1 Processor -- start 2024-04-25T10:18:06.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.384+0000 7f4968e00640 1 -- start start 2024-04-25T10:18:06.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.385+0000 7f4968e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f496415dd30 0x7f49640cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.385+0000 7f4968e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49640b5b40 0x7f49640d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.385+0000 7f4968e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f496415e110 con 0x7f496406bff0 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.385+0000 7f4968e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f496406d000 con 0x7f49640b5b40 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.385+0000 7f4968e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640b2fd0 con 0x7f496415dd30 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49640b5b40 0x7f49640d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f49640b5b40 0x7f49640d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39998/0 (socket says 172.21.15.165:39998) 2024-04-25T10:18:06.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 -- 172.21.15.165:0/3943785063 learned_addr learned my addr 172.21.15.165:0/3943785063 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963400640 1 --2- 172.21.15.165:0/3943785063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f496415dd30 0x7f49640cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 -- 172.21.15.165:0/3943785063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f496415dd30 msgr2=0x7f49640cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 --2- 172.21.15.165:0/3943785063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f496415dd30 0x7f49640cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 -- 172.21.15.165:0/3943785063 >> v1:172.21.15.172:6789/0 conn(0x7f496406bff0 legacy=0x7f49640b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.386+0000 7f4963e00640 1 -- 172.21.15.165:0/3943785063 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f496406c480 con 0x7f49640b5b40 2024-04-25T10:18:06.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.387+0000 7f4963e00640 1 --2- 172.21.15.165:0/3943785063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49640b5b40 0x7f49640d2160 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f495c009870 tx=0x7f495c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c81485a45a0c4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.388+0000 7f4962000640 1 -- 172.21.15.165:0/3943785063 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c037070 con 0x7f49640b5b40 2024-04-25T10:18:06.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.388+0000 7f4962000640 1 -- 172.21.15.165:0/3943785063 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c032440 con 0x7f49640b5b40 2024-04-25T10:18:06.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.388+0000 7f4962000640 1 -- 172.21.15.165:0/3943785063 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c032810 con 0x7f49640b5b40 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.388+0000 7f4968e00640 1 -- 172.21.15.165:0/3943785063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49640b5b40 msgr2=0x7f49640d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.388+0000 7f4968e00640 1 --2- 172.21.15.165:0/3943785063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49640b5b40 0x7f49640d2160 secure :-1 s=READY pgs=2294 cs=0 l=1 rev1=1 crypto rx=0x7f495c009870 tx=0x7f495c02a270 comp rx=0 tx=0).stop 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.389+0000 7f4968e00640 1 -- 172.21.15.165:0/3943785063 shutdown_connections 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.389+0000 7f4968e00640 1 --2- 172.21.15.165:0/3943785063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49640b5b40 0x7f49640d2160 unknown :-1 s=CLOSED pgs=2294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.389+0000 7f4968e00640 1 --2- 172.21.15.165:0/3943785063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f496415dd30 0x7f49640cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.389+0000 7f4968e00640 1 -- 172.21.15.165:0/3943785063 >> 172.21.15.165:0/3943785063 conn(0x7f49640d0610 msgr2=0x7f49640d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.389+0000 7f4968e00640 1 -- 172.21.15.165:0/3943785063 shutdown_connections 2024-04-25T10:18:06.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.389+0000 7f4968e00640 1 -- 172.21.15.165:0/3943785063 wait complete. 2024-04-25T10:18:06.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.390+0000 7f4968e00640 1 Processor -- start 2024-04-25T10:18:06.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.391+0000 7f4968e00640 1 -- start start 2024-04-25T10:18:06.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.391+0000 7f4968e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49640b5b40 0x7f49641568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4968e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 0x7f4964153a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4968e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4964152c80 0x7f49641f6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4968e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496406cd20 con 0x7f496415dd30 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4968e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496406ca40 con 0x7f49640b5b40 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4968e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640cf110 con 0x7f4964152c80 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 0x7f4964153a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 0x7f4964153a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40012/0 (socket says 172.21.15.165:40012) 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 -- 172.21.15.165:0/3570068050 learned_addr learned my addr 172.21.15.165:0/3570068050 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 -- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4964152c80 msgr2=0x7f49641f6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4963e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4964152c80 0x7f49641f6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4963400640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49640b5b40 0x7f49641568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4964152c80 0x7f49641f6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 -- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49640b5b40 msgr2=0x7f49641568a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49640b5b40 0x7f49641568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4962a00640 1 -- 172.21.15.165:0/3570068050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640cb050 con 0x7f496415dd30 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.392+0000 7f4963e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4964152c80 0x7f49641f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.393+0000 7f4962a00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 0x7f4964153a40 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c790 tx=0x7f494c00cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.393+0000 7f495b400640 1 -- 172.21.15.165:0/3570068050 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c016020 con 0x7f496415dd30 2024-04-25T10:18:06.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.393+0000 7f495b400640 1 -- 172.21.15.165:0/3570068050 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c006d50 con 0x7f496415dd30 2024-04-25T10:18:06.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.393+0000 7f495b400640 1 -- 172.21.15.165:0/3570068050 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c003bb0 con 0x7f496415dd30 2024-04-25T10:18:06.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.393+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49640cb330 con 0x7f496415dd30 2024-04-25T10:18:06.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.393+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49641560c0 con 0x7f496415dd30 2024-04-25T10:18:06.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.395+0000 7f495b400640 1 -- 172.21.15.165:0/3570068050 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f494c00a850 con 0x7f496415dd30 2024-04-25T10:18:06.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.396+0000 7f495b400640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f494006b500 0x7f494006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.396+0000 7f495b400640 1 -- 172.21.15.165:0/3570068050 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f494c08a090 con 0x7f496415dd30 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.397+0000 7f4963400640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f494006b500 0x7f494006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.397+0000 7f496d208740 1 Processor -- start 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.397+0000 7f496d208740 1 -- start start 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.397+0000 7f4963400640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f494006b500 0x7f494006d9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999540 0x55e20f99cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f496d208740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f999920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f9b8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f496d208740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20f8d5ee0 con 0x55e20f99d010 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20f87ecf0 con 0x55e20f999e60 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20f8d1f80 con 0x55e20f999540 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f4962a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f999920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f4962a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f999920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40026/0 (socket says 172.21.15.165:40026) 2024-04-25T10:18:06.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.398+0000 7f4962a00640 1 -- 172.21.15.165:0/956790748 learned_addr learned my addr 172.21.15.165:0/956790748 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4962a00640 1 -- 172.21.15.165:0/956790748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999540 msgr2=0x55e20f99cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4962a00640 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999540 0x55e20f99cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4962a00640 1 -- 172.21.15.165:0/956790748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 msgr2=0x55e20f9b8f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4962a00640 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f9b8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4962a00640 1 -- 172.21.15.165:0/956790748 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20f6b6a20 con 0x55e20f99d010 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4962a00640 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f999920 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f494c088390 tx=0x7f494c011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4029bdfae8496252 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4953e00640 1 -- 172.21.15.165:0/956790748 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c086070 con 0x55e20f99d010 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4953e00640 1 -- 172.21.15.165:0/956790748 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c060d80 con 0x55e20f99d010 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.399+0000 7f4953e00640 1 -- 172.21.15.165:0/956790748 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c070070 con 0x55e20f99d010 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 -- 172.21.15.165:0/956790748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 msgr2=0x55e20f999920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f999920 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f494c088390 tx=0x7f494c011000 comp rx=0 tx=0).stop 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 -- 172.21.15.165:0/956790748 shutdown_connections 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f9b8f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f999920 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 --2- 172.21.15.165:0/956790748 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999540 0x55e20f99cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 -- 172.21.15.165:0/956790748 >> 172.21.15.165:0/956790748 conn(0x55e20f96f330 msgr2=0x55e20f970330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 -- 172.21.15.165:0/956790748 shutdown_connections 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.400+0000 7f496d208740 1 -- 172.21.15.165:0/956790748 wait complete. 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.401+0000 7f496d208740 1 -- start start 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.401+0000 7f496d208740 1 Processor -- start 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f995400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f9b0c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f96bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20f96dd70 con 0x55e20f99d010 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20f874240 con 0x55e20f999e60 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20f96d160 con 0x55e20f9b1140 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f496d208740 10 client.0 _ll_register_callbacks cb 0x55e20f5e9e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.402+0000 7f4962a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f9b0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f9b0c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40036/0 (socket says 172.21.15.165:40036) 2024-04-25T10:18:06.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 -- 172.21.15.165:0/1996401281 learned_addr learned my addr 172.21.15.165:0/1996401281 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4963e00640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f96bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4963400640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f995400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 -- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 msgr2=0x55e20f96bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f96bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 -- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 msgr2=0x55e20f995400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f995400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 -- 172.21.15.165:0/1996401281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e20f5aa310 con 0x55e20f99d010 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4962a00640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f9b0c00 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f494c00a430 tx=0x7f494c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d285bc564a0c1cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4950c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4950c00640 1 -- 172.21.15.165:0/1996401281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494c075060 con 0x55e20f99d010 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.403+0000 7f4950c00640 1 -- 172.21.15.165:0/1996401281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f494c00ab40 con 0x55e20f99d010 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f494c060d80 con 0x55e20f99d010 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f4950c00640 1 -- 172.21.15.165:0/1996401281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f494c07c050 con 0x55e20f99d010 2024-04-25T10:18:06.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 10 client.10509 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55e20f7171b0 con 0x55e20f99d010 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f4950c00640 1 -- 172.21.15.165:0/1996401281 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494c060d80 con 0x55e20f99d010 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f4950c00640 1 -- 172.21.15.165:0/1996401281 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f494c057ac0 con 0x55e20f99d010 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 10 client.10509 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 10 client.10509 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 20 client.10509 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f9717a0 0x55e20f971b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 4 client.10509 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.404+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e20f9340c0 con 0x55e20f9717a0 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.405+0000 7f4963400640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f9717a0 0x55e20f971b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.406+0000 7f4963400640 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f9717a0 0x55e20f971b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b611b1d92f227d52 server_cookie=be686f6efb38d396 in_seq=0 out_seq=0 2024-04-25T10:18:06.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.406+0000 7f4950c00640 10 client.10509 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f4950c00640 1 -- 172.21.15.165:0/1996401281 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e20f9340c0 con 0x55e20f9717a0 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f4950c00640 10 client.10509 handle_command_reply: tid=0 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 client.10509 shutdown 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 msgr2=0x55e20f9b0c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f9b0c00 secure :-1 s=READY pgs=2297 cs=0 l=1 rev1=1 crypto rx=0x7f494c00a430 tx=0x7f494c000f40 comp rx=0 tx=0).stop 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 shutdown_connections 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f9717a0 0x55e20f971b80 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f96bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f99d010 0x55e20f9b0c00 unknown :-1 s=CLOSED pgs=2297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 --2- 172.21.15.165:0/1996401281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f999e60 0x55e20f995400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f4950c00640 0 client.10509 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 >> 172.21.15.165:0/1996401281 conn(0x55e20f999540 msgr2=0x55e20f9b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 shutdown_connections 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 1 -- 172.21.15.165:0/1996401281 wait complete. 2024-04-25T10:18:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.408+0000 7f496d208740 20 client.10509 trim_cache size 0 max 16384 2024-04-25T10:18:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.425+0000 7f496d208740 1 Processor -- start 2024-04-25T10:18:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.425+0000 7f496d208740 1 -- start start 2024-04-25T10:18:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.426+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999e60 0x55e20f968950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.426+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20f9aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.426+0000 7f496d208740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 0x55e20f96e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.426+0000 7f496d208740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49642003b0 con 0x55e20f9b1140 2024-04-25T10:18:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.426+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49641ff510 con 0x55e20f99d010 2024-04-25T10:18:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.426+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49640ccda0 con 0x55e20f999e60 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 0x55e20f96e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 0x55e20f96e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40052/0 (socket says 172.21.15.165:40052) 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 -- 172.21.15.165:0/3970533032 learned_addr learned my addr 172.21.15.165:0/3970533032 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963400640 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999e60 0x55e20f968950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4962a00640 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20f9aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 -- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999e60 msgr2=0x55e20f968950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999e60 0x55e20f968950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 -- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 msgr2=0x55e20f9aeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20f9aeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 -- 172.21.15.165:0/3970533032 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e20f96c360 con 0x55e20f9b1140 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.427+0000 7f4963e00640 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 0x55e20f96e000 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f495c02a2b0 tx=0x7f495c02c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a94ad4e63db716 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f4953400640 1 -- 172.21.15.165:0/3970533032 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c037040 con 0x55e20f9b1140 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f4953400640 1 -- 172.21.15.165:0/3970533032 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c03e660 con 0x55e20f9b1140 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f4953400640 1 -- 172.21.15.165:0/3970533032 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495c03ea30 con 0x55e20f9b1140 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 -- 172.21.15.165:0/3970533032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 msgr2=0x55e20f96e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 0x55e20f96e000 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7f495c02a2b0 tx=0x7f495c02c000 comp rx=0 tx=0).stop 2024-04-25T10:18:06.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 -- 172.21.15.165:0/3970533032 shutdown_connections 2024-04-25T10:18:06.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f9b1140 0x55e20f96e000 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20f9aeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 --2- 172.21.15.165:0/3970533032 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f999e60 0x55e20f968950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.428+0000 7f496d208740 1 -- 172.21.15.165:0/3970533032 >> 172.21.15.165:0/3970533032 conn(0x55e20f999540 msgr2=0x55e20f973a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.429+0000 7f496d208740 1 -- 172.21.15.165:0/3970533032 shutdown_connections 2024-04-25T10:18:06.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.429+0000 7f496d208740 1 -- 172.21.15.165:0/3970533032 wait complete. 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.429+0000 7f496d208740 1 -- start start 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.429+0000 7f496d208740 1 Processor -- start 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.430+0000 7f496d208740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 0x55e20f96e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.430+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20fa146b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.430+0000 7f496d208740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f970100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.430+0000 7f496d208740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e20f9db4e0 con 0x55e20f999e60 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.430+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49641fef40 con 0x55e20f99d010 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.430+0000 7f496d208740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e20f9b4da0 con 0x55e20f9b1140 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f496d208740 10 client.0 _ll_register_callbacks cb 0x7f494c054990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:06.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 0x55e20f96e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 0x55e20f96e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40064/0 (socket says 172.21.15.165:40064) 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 -- 172.21.15.165:0/1963019912 learned_addr learned my addr 172.21.15.165:0/1963019912 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 -- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 msgr2=0x55e20f970100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f970100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 -- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 msgr2=0x55e20fa146b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20fa146b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.431+0000 7f4963400640 1 -- 172.21.15.165:0/1963019912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e20f583490 con 0x55e20f999e60 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4963400640 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 0x55e20f96e460 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x55e20f9a0740 tx=0x7f49540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5832f6d3958c50 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4950c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4950c00640 1 -- 172.21.15.165:0/1963019912 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4954032060 con 0x55e20f999e60 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4950c00640 1 -- 172.21.15.165:0/1963019912 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f495400ebc0 con 0x55e20f999e60 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f495c03e660 con 0x55e20f999e60 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4950c00640 1 -- 172.21.15.165:0/1963019912 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4954038050 con 0x55e20f999e60 2024-04-25T10:18:06.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 10 client.10515 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55e20f9983b0 con 0x55e20f999e60 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4950c00640 1 -- 172.21.15.165:0/1963019912 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495c03e660 con 0x55e20f999e60 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f4950c00640 1 -- 172.21.15.165:0/1963019912 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f495403b020 con 0x55e20f999e60 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 10 client.10515 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 10 client.10515 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 20 client.10515 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.432+0000 7f496d208740 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f970640 0x55e20f970ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.433+0000 7f496d208740 4 client.10515 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}] 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.433+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "scrub start", "path": "~mdsdir", "scrubops": ["recursive"]}) v1 -- 0x55e20f8e3420 con 0x55e20f970640 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.433+0000 7f4962a00640 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f970640 0x55e20f970ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.434+0000 7f4950c00640 10 client.10515 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.434+0000 7f4962a00640 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f970640 0x55e20f970ac0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=226508502475fdb4 server_cookie=ac7c4a7b760d7851 in_seq=0 out_seq=0 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f4950c00640 1 -- 172.21.15.165:0/1963019912 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55e20f8e3420 con 0x55e20f970640 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f4950c00640 10 client.10515 handle_command_reply: tid=0 2024-04-25T10:18:06.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 client.10515 shutdown 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 msgr2=0x55e20f96e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 0x55e20f96e460 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x55e20f9a0740 tx=0x7f49540046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 shutdown_connections 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55e20f970640 0x55e20f970ac0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e20f9b1140 0x55e20f970100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e20f99d010 0x55e20fa146b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 --2- 172.21.15.165:0/1963019912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e20f999e60 0x55e20f96e460 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f4950c00640 0 client.10515 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 >> 172.21.15.165:0/1963019912 conn(0x55e20f999540 msgr2=0x55e20f99c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 shutdown_connections 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 1 -- 172.21.15.165:0/1963019912 wait complete. 2024-04-25T10:18:06.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.436+0000 7f496d208740 20 client.10515 trim_cache size 0 max 16384 2024-04-25T10:18:06.445 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:18:06.445 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:18:06.445 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "f69e9618-73c3-47da-8381-504310848e7b", 2024-04-25T10:18:06.445 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:18:06.445 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f494006b500 msgr2=0x7f494006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f494006b500 0x7f494006d9c0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f4954006fd0 tx=0x7f4954008040 comp rx=0 tx=0).stop 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 msgr2=0x7f4964153a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 0x7f4964153a40 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f494c00c790 tx=0x7f494c00cc50 comp rx=0 tx=0).stop 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 shutdown_connections 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f494006b500 0x7f494006d9c0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4964152c80 0x7f49641f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f496415dd30 0x7f4964153a40 unknown :-1 s=CLOSED pgs=2295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 --2- 172.21.15.165:0/3570068050 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49640b5b40 0x7f49641568a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 >> 172.21.15.165:0/3570068050 conn(0x7f49640d0610 msgr2=0x7f49640741d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 shutdown_connections 2024-04-25T10:18:06.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.437+0000 7f4968e00640 1 -- 172.21.15.165:0/3570068050 wait complete. 2024-04-25T10:18:06.458 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:06.458 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:18:06.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.651+0000 7fb8ace00640 1 Processor -- start 2024-04-25T10:18:06.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.652+0000 7fb8ace00640 1 -- start start 2024-04-25T10:18:06.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8ace00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a81446d0 0x7fb8a80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8ace00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb8a8144d50 0x7fb8a80b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8ace00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8a81452a0 0x7fb8a80cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8ace00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a80b2b20 con 0x7fb8a81452a0 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8ace00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8a806d400 con 0x7fb8a8144d50 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8ace00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80b49a0 con 0x7fb8a81446d0 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8a6800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8a81452a0 0x7fb8a80cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8a6800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb8a81452a0 0x7fb8a80cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40066/0 (socket says 172.21.15.165:40066) 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.653+0000 7fb8a6800640 1 -- 172.21.15.165:0/823631857 learned_addr learned my addr 172.21.15.165:0/823631857 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/823631857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a81446d0 0x7fb8a80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb89fe00640 1 --2- 172.21.15.165:0/823631857 >> v2:172.21.15.172:3300/0 conn(0x7fb8a8144d50 0x7fb8a80b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb8a6800640 1 -- 172.21.15.165:0/823631857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a81446d0 msgr2=0x7fb8a80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb8a6800640 1 --2- 172.21.15.165:0/823631857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a81446d0 0x7fb8a80b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb8a6800640 1 -- 172.21.15.165:0/823631857 >> v2:172.21.15.172:3300/0 conn(0x7fb8a8144d50 msgr2=0x7fb8a80b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb8a6800640 1 --2- 172.21.15.165:0/823631857 >> v2:172.21.15.172:3300/0 conn(0x7fb8a8144d50 0x7fb8a80b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.654+0000 7fb8a6800640 1 -- 172.21.15.165:0/823631857 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80cccc0 con 0x7fb8a81452a0 2024-04-25T10:18:06.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.655+0000 7fb8a6800640 1 --2- 172.21.15.165:0/823631857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a80cf640 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80d07a0 tx=0x7fb8a002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86f4be3e401e636 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.655+0000 7fb8a5400640 1 -- 172.21.15.165:0/823631857 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0036070 con 0x7fb8a81452a0 2024-04-25T10:18:06.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.655+0000 7fb8a5400640 1 -- 172.21.15.165:0/823631857 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0031600 con 0x7fb8a81452a0 2024-04-25T10:18:06.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.655+0000 7fb8a5400640 1 -- 172.21.15.165:0/823631857 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00319d0 con 0x7fb8a81452a0 2024-04-25T10:18:06.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 -- 172.21.15.165:0/823631857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 msgr2=0x7fb8a80cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 --2- 172.21.15.165:0/823631857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a80cf640 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80d07a0 tx=0x7fb8a002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 -- 172.21.15.165:0/823631857 shutdown_connections 2024-04-25T10:18:06.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 --2- 172.21.15.165:0/823631857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a80cf640 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 --2- 172.21.15.165:0/823631857 >> v2:172.21.15.172:3300/0 conn(0x7fb8a8144d50 0x7fb8a80b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 --2- 172.21.15.165:0/823631857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a81446d0 0x7fb8a80b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 -- 172.21.15.165:0/823631857 >> 172.21.15.165:0/823631857 conn(0x7fb8a806abe0 msgr2=0x7fb8a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 -- 172.21.15.165:0/823631857 shutdown_connections 2024-04-25T10:18:06.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.656+0000 7fb8ace00640 1 -- 172.21.15.165:0/823631857 wait complete. 2024-04-25T10:18:06.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.658+0000 7fb8ace00640 1 Processor -- start 2024-04-25T10:18:06.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.658+0000 7fb8ace00640 1 -- start start 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8ace00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a81446d0 0x7fb8a80afe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8ace00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a8144d50 0x7fb8a80c4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8ace00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a81433c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8ace00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cc700 con 0x7fb8a81452a0 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8ace00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cc420 con 0x7fb8a81446d0 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8ace00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cc9e0 con 0x7fb8a8144d50 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a81433c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a81433c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40080/0 (socket says 172.21.15.165:40080) 2024-04-25T10:18:06.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.659+0000 7fb8a6800640 1 -- 172.21.15.165:0/1933920541 learned_addr learned my addr 172.21.15.165:0/1933920541 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a81446d0 0x7fb8a80afe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb89fe00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a8144d50 0x7fb8a80c4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a6800640 1 -- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a8144d50 msgr2=0x7fb8a80c4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a6800640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a8144d50 0x7fb8a80c4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a6800640 1 -- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a81446d0 msgr2=0x7fb8a80afe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a6800640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a81446d0 0x7fb8a80afe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a6800640 1 -- 172.21.15.165:0/1933920541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80cd740 con 0x7fb8a81452a0 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb89fe00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a8144d50 0x7fb8a80c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8a6800640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a81433c0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009f40 tx=0x7fb8a0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb89e000640 1 -- 172.21.15.165:0/1933920541 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0041020 con 0x7fb8a81452a0 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb89e000640 1 -- 172.21.15.165:0/1933920541 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0007780 con 0x7fb8a81452a0 2024-04-25T10:18:06.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb89e000640 1 -- 172.21.15.165:0/1933920541 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0036040 con 0x7fb8a81452a0 2024-04-25T10:18:06.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.660+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a80cda20 con 0x7fb8a81452a0 2024-04-25T10:18:06.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.661+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a80cdfe0 con 0x7fb8a81452a0 2024-04-25T10:18:06.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.663+0000 7fb89e000640 1 -- 172.21.15.165:0/1933920541 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb8a0044020 con 0x7fb8a81452a0 2024-04-25T10:18:06.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.664+0000 7fb89e000640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.664+0000 7fb89e000640 1 -- 172.21.15.165:0/1933920541 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8a00ab0f0 con 0x7fb8a81452a0 2024-04-25T10:18:06.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.664+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.664+0000 7fb8afc23740 1 Processor -- start 2024-04-25T10:18:06.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 -- start start 2024-04-25T10:18:06.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb898006fd0 tx=0x7fb898008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754027d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754025f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754050af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561753f89c70 con 0x561754050670 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561753f32cf0 con 0x561754025bb0 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.665+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561753f85d10 con 0x561754062280 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754050af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754050af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40086/0 (socket says 172.21.15.165:40086) 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 -- 172.21.15.165:0/159944086 learned_addr learned my addr 172.21.15.165:0/159944086 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 -- 172.21.15.165:0/159944086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 msgr2=0x561754025f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754025f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 -- 172.21.15.165:0/159944086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 msgr2=0x561754027d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754027d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 -- 172.21.15.165:0/159944086 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561753d687d0 con 0x561754050670 2024-04-25T10:18:06.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb8a6800640 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754050af0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0079690 tx=0x7fb8a0075ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c83d3e3b38b57ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.666+0000 7fb896a00640 1 -- 172.21.15.165:0/159944086 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a00a6020 con 0x561754050670 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb896a00640 1 -- 172.21.15.165:0/159944086 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0079700 con 0x561754050670 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb896a00640 1 -- 172.21.15.165:0/159944086 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a007d850 con 0x561754050670 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 -- 172.21.15.165:0/159944086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 msgr2=0x561754050af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754050af0 secure :-1 s=READY pgs=2302 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0079690 tx=0x7fb8a0075ad0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 -- 172.21.15.165:0/159944086 shutdown_connections 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754050af0 unknown :-1 s=CLOSED pgs=2302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754025f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 --2- 172.21.15.165:0/159944086 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754027d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 -- 172.21.15.165:0/159944086 >> 172.21.15.165:0/159944086 conn(0x561753feda30 msgr2=0x56175406b670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 -- 172.21.15.165:0/159944086 shutdown_connections 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.667+0000 7fb8afc23740 1 -- 172.21.15.165:0/159944086 wait complete. 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.668+0000 7fb8afc23740 1 -- start start 2024-04-25T10:18:06.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.668+0000 7fb8afc23740 1 Processor -- start 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 0x56175407b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754050670 0x561754066590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754052690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561753f32cf0 con 0x561754025bb0 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561753f283a0 con 0x561754050670 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561753f89c70 con 0x561754062280 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8afc23740 10 client.0 _ll_register_callbacks cb 0x561753f45b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.669+0000 7fb8a5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 0x56175407b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 0x56175407b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40096/0 (socket says 172.21.15.165:40096) 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 -- 172.21.15.165:0/2522859977 learned_addr learned my addr 172.21.15.165:0/2522859977 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a6800640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754052690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 -- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 msgr2=0x561754052690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb89fe00640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754050670 0x561754066590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754052690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 -- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754050670 msgr2=0x561754066590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754050670 0x561754066590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 -- 172.21.15.165:0/2522859977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561753c9e3f0 con 0x561754025bb0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 0x56175407b080 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fb898014490 tx=0x7fb898014890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc7dcef01d5e2f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.670+0000 7fb88b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb88b400640 1 -- 172.21.15.165:0/2522859977 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb898025060 con 0x561754025bb0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb88b400640 1 -- 172.21.15.165:0/2522859977 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb898015820 con 0x561754025bb0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb8a0079700 con 0x561754025bb0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb88b400640 1 -- 172.21.15.165:0/2522859977 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb89802b050 con 0x561754025bb0 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 10 client.10527 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:06.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x561753c5e0f0 con 0x561754025bb0 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb88b400640 1 -- 172.21.15.165:0/2522859977 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a0079700 con 0x561754025bb0 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb88b400640 1 -- 172.21.15.165:0/2522859977 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb89802e020 con 0x561754025bb0 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 10 client.10527 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 10 client.10527 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 20 client.10527 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x561754052f40 0x5617540679b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 4 client.10527 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.671+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561753f971b0 con 0x561754052f40 2024-04-25T10:18:06.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.672+0000 7fb89fe00640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x561754052f40 0x5617540679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.673+0000 7fb89fe00640 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x561754052f40 0x5617540679b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d6b0fef6a1be236 server_cookie=8ae20a8dd0bfcf67 in_seq=0 out_seq=0 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.673+0000 7fb88b400640 10 client.10527 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb88b400640 1 -- 172.21.15.165:0/2522859977 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561753f971b0 con 0x561754052f40 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb88b400640 10 client.10527 handle_command_reply: tid=0 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 client.10527 shutdown 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 msgr2=0x56175407b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 0x56175407b080 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7fb898014490 tx=0x7fb898014890 comp rx=0 tx=0).stop 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 shutdown_connections 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x561754052f40 0x5617540679b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754052690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754050670 0x561754066590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2522859977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754025bb0 0x56175407b080 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb88b400640 0 client.10527 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 >> 172.21.15.165:0/2522859977 conn(0x561753feda30 msgr2=0x56175406eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 shutdown_connections 2024-04-25T10:18:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 1 -- 172.21.15.165:0/2522859977 wait complete. 2024-04-25T10:18:06.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.675+0000 7fb8afc23740 20 client.10527 trim_cache size 0 max 16384 2024-04-25T10:18:06.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.692+0000 7fb8afc23740 1 Processor -- start 2024-04-25T10:18:06.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.692+0000 7fb8afc23740 1 -- start start 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x5617540674e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754067a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754068010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb8afc23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a82004e0 con 0x561754050670 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a81ff6b0 con 0x561754025bb0 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80cf2d0 con 0x561754062280 2024-04-25T10:18:06.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754067a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb89fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754067a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40110/0 (socket says 172.21.15.165:40110) 2024-04-25T10:18:06.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.693+0000 7fb89fe00640 1 -- 172.21.15.165:0/564419853 learned_addr learned my addr 172.21.15.165:0/564419853 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb89fe00640 1 -- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 msgr2=0x561754068010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb8a6800640 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb89fe00640 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754068010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x5617540674e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb89fe00640 1 -- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 msgr2=0x5617540674e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb89fe00640 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x5617540674e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb89fe00640 1 -- 172.21.15.165:0/564419853 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561754053960 con 0x561754050670 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb89fe00640 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754067a20 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fb890006fd0 tx=0x7fb890009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d9d09090b3a824 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb896000640 1 -- 172.21.15.165:0/564419853 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890004030 con 0x561754050670 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb896000640 1 -- 172.21.15.165:0/564419853 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89001a030 con 0x561754050670 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.694+0000 7fb896000640 1 -- 172.21.15.165:0/564419853 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb890004430 con 0x561754050670 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 -- 172.21.15.165:0/564419853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 msgr2=0x561754067a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754067a20 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7fb890006fd0 tx=0x7fb890009ea0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 -- 172.21.15.165:0/564419853 shutdown_connections 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754062280 0x561754068010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754050670 0x561754067a20 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 --2- 172.21.15.165:0/564419853 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x5617540674e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 -- 172.21.15.165:0/564419853 >> 172.21.15.165:0/564419853 conn(0x561753feda30 msgr2=0x561754026a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 -- 172.21.15.165:0/564419853 shutdown_connections 2024-04-25T10:18:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.695+0000 7fb8afc23740 1 -- 172.21.15.165:0/564419853 wait complete. 2024-04-25T10:18:06.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.696+0000 7fb8afc23740 1 -- start start 2024-04-25T10:18:06.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.696+0000 7fb8afc23740 1 Processor -- start 2024-04-25T10:18:06.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754061000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754050670 0x561754064b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 0x5617540650d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80ad5c0 con 0x561754062280 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561754054350 con 0x561754025bb0 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561754061a10 con 0x561754050670 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8afc23740 10 client.0 _ll_register_callbacks cb 0x7fb89800b680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 0x5617540650d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.697+0000 7fb8a6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 0x5617540650d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40126/0 (socket says 172.21.15.165:40126) 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 -- 172.21.15.165:0/2889882408 learned_addr learned my addr 172.21.15.165:0/2889882408 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754061000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb89fe00640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754050670 0x561754064b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 -- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754050670 msgr2=0x561754064b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754050670 0x561754064b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 -- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 msgr2=0x561754061000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754061000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 -- 172.21.15.165:0/2889882408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561754025630 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754061000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8a6800640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 0x5617540650d0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00818e0 tx=0x7fb8a0009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7522e7e638c3bc87 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb88b400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb88b400640 1 -- 172.21.15.165:0/2889882408 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a0098070 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb88b400640 1 -- 172.21.15.165:0/2889882408 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb8a0007b70 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.698+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb89001a030 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb88b400640 1 -- 172.21.15.165:0/2889882408 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb8a00a6040 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 10 client.10533 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x561754061750 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb88b400640 1 -- 172.21.15.165:0/2889882408 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb89001a030 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb88b400640 1 -- 172.21.15.165:0/2889882408 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fb8a0078b50 con 0x561754062280 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 10 client.10533 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 10 client.10533 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 20 client.10533 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:06.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x56175406f850 0x561754079d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:06.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 4 client.10533 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:18:06.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.699+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56175404e3f0 con 0x56175406f850 2024-04-25T10:18:06.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.700+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x56175406f850 0x561754079d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.701+0000 7fb8a5e00640 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x56175406f850 0x561754079d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=925266927652062 server_cookie=ef4c3385aa67fcaf in_seq=0 out_seq=0 2024-04-25T10:18:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.701+0000 7fb88b400640 10 client.10533 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb88b400640 1 -- 172.21.15.165:0/2889882408 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+257 (crc 0 0 0) 0x56175404e3f0 con 0x56175406f850 2024-04-25T10:18:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb88b400640 10 client.10533 handle_command_reply: tid=0 2024-04-25T10:18:06.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 client.10533 shutdown 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 msgr2=0x5617540650d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 0x5617540650d0 secure :-1 s=READY pgs=2305 cs=0 l=1 rev1=1 crypto rx=0x7fb8a00818e0 tx=0x7fb8a0009810 comp rx=0 tx=0).stop 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 shutdown_connections 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561754062280 0x5617540650d0 unknown :-1 s=CLOSED pgs=2305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x56175406f850 0x561754079d10 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561754050670 0x561754064b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 --2- 172.21.15.165:0/2889882408 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561754025bb0 0x561754061000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 >> 172.21.15.165:0/2889882408 conn(0x561753feda30 msgr2=0x56175401faa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb88b400640 0 client.10533 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 shutdown_connections 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 1 -- 172.21.15.165:0/2889882408 wait complete. 2024-04-25T10:18:06.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.702+0000 7fb8afc23740 20 client.10533 trim_cache size 0 max 16384 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: "f69e9618-73c3-47da-8381-504310848e7b": { 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "~mds0", 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "f69e9618-73c3-47da-8381-504310848e7b", 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive" 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:06.708 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:18:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb88406b4d0 msgr2=0x7fb88406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb88406b4d0 0x7fb88406d990 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fb898006fd0 tx=0x7fb898008040 comp rx=0 tx=0).stop 2024-04-25T10:18:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 msgr2=0x7fb8a81433c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a81433c0 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009f40 tx=0x7fb8a0004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 shutdown_connections 2024-04-25T10:18:06.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb88406b4d0 0x7fb88406d990 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb8a81452a0 0x7fb8a81433c0 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb8a8144d50 0x7fb8a80c4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 --2- 172.21.15.165:0/1933920541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb8a81446d0 0x7fb8a80afe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 >> 172.21.15.165:0/1933920541 conn(0x7fb8a806abe0 msgr2=0x7fb8a80af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 shutdown_connections 2024-04-25T10:18:06.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:06.703+0000 7fb8ace00640 1 -- 172.21.15.165:0/1933920541 wait complete. 2024-04-25T10:18:06.724 INFO:tasks.cephfs.filesystem:scrub status for tag:f69e9618-73c3-47da-8381-504310848e7b - {'path': '~mds0', 'tag': 'f69e9618-73c3-47da-8381-504310848e7b', 'options': 'recursive'} 2024-04-25T10:18:30.890 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:30.899 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:18:36.725 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:36.725 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.11:0 scrub status 2024-04-25T10:18:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.936+0000 7f14d6400640 1 Processor -- start 2024-04-25T10:18:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.936+0000 7f14d6400640 1 -- start start 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14d6400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f14d015b8a0 0x7f14d006abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14d6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d00af520 0x7f14d006b120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14d6400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f14d006b660 0x7f14d015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14d6400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00adf30 con 0x7f14d015b8a0 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14d6400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14d00ad970 con 0x7f14d006b660 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14d6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00adc50 con 0x7f14d00af520 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14cf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f14d015b8a0 0x7f14d006abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14cf400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f14d015b8a0 0x7f14d006abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47952/0 (socket says 172.21.15.165:47952) 2024-04-25T10:18:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.938+0000 7f14cf400640 1 -- 172.21.15.165:0/1429273446 learned_addr learned my addr 172.21.15.165:0/1429273446 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cea00640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d00af520 0x7f14d006b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cfe00640 1 --2- 172.21.15.165:0/1429273446 >> v2:172.21.15.172:3300/0 conn(0x7f14d006b660 0x7f14d015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cf400640 1 -- 172.21.15.165:0/1429273446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d00af520 msgr2=0x7f14d006b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cf400640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d00af520 0x7f14d006b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cf400640 1 -- 172.21.15.165:0/1429273446 >> v2:172.21.15.172:3300/0 conn(0x7f14d006b660 msgr2=0x7f14d015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cf400640 1 --2- 172.21.15.165:0/1429273446 >> v2:172.21.15.172:3300/0 conn(0x7f14d006b660 0x7f14d015f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cf400640 1 -- 172.21.15.165:0/1429273446 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d00b3840 con 0x7f14d015b8a0 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.939+0000 7f14cea00640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d00af520 0x7f14d006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.940+0000 7f14cf400640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d015b8a0 0x7f14d006abe0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009870 tx=0x7f14c002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ddd6661ed6d9ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.940+0000 7f14ce000640 1 -- 172.21.15.165:0/1429273446 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0037070 con 0x7f14d015b8a0 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.940+0000 7f14ce000640 1 -- 172.21.15.165:0/1429273446 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c0032440 con 0x7f14d015b8a0 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.940+0000 7f14ce000640 1 -- 172.21.15.165:0/1429273446 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0032810 con 0x7f14d015b8a0 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 -- 172.21.15.165:0/1429273446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d015b8a0 msgr2=0x7f14d006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d015b8a0 0x7f14d006abe0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7f14c0009870 tx=0x7f14c002a200 comp rx=0 tx=0).stop 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 -- 172.21.15.165:0/1429273446 shutdown_connections 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 --2- 172.21.15.165:0/1429273446 >> v2:172.21.15.172:3300/0 conn(0x7f14d006b660 0x7f14d015f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d00af520 0x7f14d006b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 --2- 172.21.15.165:0/1429273446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d015b8a0 0x7f14d006abe0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 -- 172.21.15.165:0/1429273446 >> 172.21.15.165:0/1429273446 conn(0x7f14d00cfec0 msgr2=0x7f14d00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 -- 172.21.15.165:0/1429273446 shutdown_connections 2024-04-25T10:18:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.941+0000 7f14d6400640 1 -- 172.21.15.165:0/1429273446 wait complete. 2024-04-25T10:18:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.943+0000 7f14d6400640 1 Processor -- start 2024-04-25T10:18:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.943+0000 7f14d6400640 1 -- start start 2024-04-25T10:18:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14d6400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14d006b660 0x7f14d0141100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14d6400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 0x7f14d015e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14d6400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d015b8a0 0x7f14d015e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14d6400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d015be00 con 0x7f14d00af520 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14d6400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d015bc80 con 0x7f14d006b660 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14d6400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d015fb10 con 0x7f14d015b8a0 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14cea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 0x7f14d015e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14cea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 0x7f14d015e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47968/0 (socket says 172.21.15.165:47968) 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.944+0000 7f14cea00640 1 -- 172.21.15.165:0/3012531854 learned_addr learned my addr 172.21.15.165:0/3012531854 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cf400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14d006b660 0x7f14d0141100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cea00640 1 -- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d015b8a0 msgr2=0x7f14d015e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d015b8a0 0x7f14d015e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cea00640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d015b8a0 0x7f14d015e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cea00640 1 -- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14d006b660 msgr2=0x7f14d0141100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cea00640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14d006b660 0x7f14d0141100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cea00640 1 -- 172.21.15.165:0/3012531854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d015d090 con 0x7f14d00af520 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14cea00640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 0x7f14d015e130 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00c7f0 tx=0x7f14bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.945+0000 7f14c7400640 1 -- 172.21.15.165:0/3012531854 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc015070 con 0x7f14d00af520 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.946+0000 7f14c7400640 1 -- 172.21.15.165:0/3012531854 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc007a30 con 0x7f14d00af520 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.946+0000 7f14c7400640 1 -- 172.21.15.165:0/3012531854 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc007e00 con 0x7f14d00af520 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.946+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14d015d370 con 0x7f14d00af520 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.946+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14d015d930 con 0x7f14d00af520 2024-04-25T10:18:36.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.948+0000 7f14c7400640 1 -- 172.21.15.165:0/3012531854 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f14bc018070 con 0x7f14d00af520 2024-04-25T10:18:36.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.949+0000 7f14c7400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f14b006b4d0 0x7f14b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.949+0000 7f14c7400640 1 -- 172.21.15.165:0/3012531854 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14bc08a350 con 0x7f14d00af520 2024-04-25T10:18:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.949+0000 7f14cf400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f14b006b4d0 0x7f14b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.949+0000 7f14d923b740 1 Processor -- start 2024-04-25T10:18:36.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.950+0000 7f14d923b740 1 -- start start 2024-04-25T10:18:36.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.950+0000 7f14cf400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f14b006b4d0 0x7f14b006d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f14c0034000 tx=0x7f14c002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.950+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb700fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb700df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14d923b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7038af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14d923b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb6f71c70 con 0x560cb7038670 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb6f1acf0 con 0x560cb704a280 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cb6f6dd10 con 0x560cb700dbb0 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7038af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7038af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47978/0 (socket says 172.21.15.165:47978) 2024-04-25T10:18:36.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 -- 172.21.15.165:0/3083583084 learned_addr learned my addr 172.21.15.165:0/3083583084 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:36.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 -- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 msgr2=0x560cb700fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cea00640 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb700df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cf400640 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb700fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb700fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 -- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 msgr2=0x560cb700df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb700df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.951+0000 7f14cfe00640 1 -- 172.21.15.165:0/3083583084 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb6d507d0 con 0x560cb7038670 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.952+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7038af0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f14c8008f30 tx=0x7f14c800d810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d62ae9bde09a44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.952+0000 7f14bbe00640 1 -- 172.21.15.165:0/3083583084 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8003da0 con 0x560cb7038670 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.952+0000 7f14bbe00640 1 -- 172.21.15.165:0/3083583084 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8004030 con 0x560cb7038670 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.952+0000 7f14bbe00640 1 -- 172.21.15.165:0/3083583084 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8004400 con 0x560cb7038670 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 -- 172.21.15.165:0/3083583084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 msgr2=0x560cb7038af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7038af0 secure :-1 s=READY pgs=2308 cs=0 l=1 rev1=1 crypto rx=0x7f14c8008f30 tx=0x7f14c800d810 comp rx=0 tx=0).stop 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 -- 172.21.15.165:0/3083583084 shutdown_connections 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7038af0 unknown :-1 s=CLOSED pgs=2308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb700df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 --2- 172.21.15.165:0/3083583084 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb700fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 -- 172.21.15.165:0/3083583084 >> 172.21.15.165:0/3083583084 conn(0x560cb6fd5a30 msgr2=0x560cb7053670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 -- 172.21.15.165:0/3083583084 shutdown_connections 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 -- 172.21.15.165:0/3083583084 wait complete. 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.953+0000 7f14d923b740 1 -- start start 2024-04-25T10:18:36.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.954+0000 7f14d923b740 1 Processor -- start 2024-04-25T10:18:36.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb7034e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7066d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb70672b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb6f103a0 con 0x560cb7038670 2024-04-25T10:18:36.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb70361c0 con 0x560cb704a280 2024-04-25T10:18:36.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cb704b820 con 0x560cb700dbb0 2024-04-25T10:18:36.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14d923b740 10 client.0 _ll_register_callbacks cb 0x560cb6f2db10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:36.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7066d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7066d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47992/0 (socket says 172.21.15.165:47992) 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 -- 172.21.15.165:0/836049028 learned_addr learned my addr 172.21.15.165:0/836049028 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cf400640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb7034e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cfe00640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb70672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 -- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 msgr2=0x560cb7034e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb7034e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 -- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 msgr2=0x560cb70672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.955+0000 7f14cea00640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb70672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14cea00640 1 -- 172.21.15.165:0/836049028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cb6c863f0 con 0x560cb7038670 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14cfe00640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb70672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14cea00640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7066d70 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f14bc004770 tx=0x7f14bc058a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=112250bc833c1f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14b8c00640 1 -- 172.21.15.165:0/836049028 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc01f040 con 0x560cb7038670 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14b8c00640 1 -- 172.21.15.165:0/836049028 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14bc057f80 con 0x560cb7038670 2024-04-25T10:18:36.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14c8004030 con 0x560cb7038670 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14b8c00640 1 -- 172.21.15.165:0/836049028 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14bc0701b0 con 0x560cb7038670 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14d923b740 10 client.10545 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.956+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x560cb6c460f0 con 0x560cb7038670 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14b8c00640 1 -- 172.21.15.165:0/836049028 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c8004030 con 0x560cb7038670 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14b8c00640 1 -- 172.21.15.165:0/836049028 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14bc088290 con 0x560cb7038670 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14d923b740 10 client.10545 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14d923b740 10 client.10545 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14d923b740 20 client.10545 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14d923b740 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb703a360 0x560cb703a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14d923b740 4 client.10545 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560cb6f7f1b0 con 0x560cb703a360 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.957+0000 7f14cf400640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb703a360 0x560cb703a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.958+0000 7f14cf400640 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb703a360 0x560cb703a740 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c5b65c95783e443 server_cookie=f7669d8b064b7b16 in_seq=0 out_seq=0 2024-04-25T10:18:36.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.958+0000 7f14b8c00640 10 client.10545 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.960+0000 7f14b8c00640 1 -- 172.21.15.165:0/836049028 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x560cb6f7f1b0 con 0x560cb703a360 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.960+0000 7f14b8c00640 10 client.10545 handle_command_reply: tid=0 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.960+0000 7f14d923b740 1 client.10545 shutdown 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.960+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 msgr2=0x560cb7066d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.960+0000 7f14d923b740 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7066d70 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f14bc004770 tx=0x7f14bc058a70 comp rx=0 tx=0).stop 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 shutdown_connections 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb70672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb7038670 0x560cb7066d70 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb703a360 0x560cb703a740 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 --2- 172.21.15.165:0/836049028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb700dbb0 0x560cb7034e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 >> 172.21.15.165:0/836049028 conn(0x560cb6fd5a30 msgr2=0x560cb7035890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14b8c00640 0 client.10545 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:36.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 shutdown_connections 2024-04-25T10:18:36.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 1 -- 172.21.15.165:0/836049028 wait complete. 2024-04-25T10:18:36.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.961+0000 7f14d923b740 20 client.10545 trim_cache size 0 max 16384 2024-04-25T10:18:36.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.977+0000 7f14d923b740 1 Processor -- start 2024-04-25T10:18:36.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.977+0000 7f14d923b740 1 -- start start 2024-04-25T10:18:36.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14d923b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 0x560cb704aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7007aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb703a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14d923b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d02004a0 con 0x560cb700dbb0 2024-04-25T10:18:36.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01ff6a0 con 0x560cb704a280 2024-04-25T10:18:36.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14d00cf100 con 0x560cb7038670 2024-04-25T10:18:36.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14cf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 0x560cb704aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14cf400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 0x560cb704aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48002/0 (socket says 172.21.15.165:48002) 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.978+0000 7f14cf400640 1 -- 172.21.15.165:0/1372569383 learned_addr learned my addr 172.21.15.165:0/1372569383 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14cf400640 1 -- 172.21.15.165:0/1372569383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 msgr2=0x560cb7007aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14cf400640 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7007aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14cf400640 1 -- 172.21.15.165:0/1372569383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 msgr2=0x560cb703a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14cf400640 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb703a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14cf400640 1 -- 172.21.15.165:0/1372569383 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560cb7053360 con 0x560cb700dbb0 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14cf400640 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 0x560cb704aa10 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f14c00090b0 tx=0x7f14c002c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69a19c99bbdb8577 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14bb400640 1 -- 172.21.15.165:0/1372569383 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0054070 con 0x560cb700dbb0 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14bb400640 1 -- 172.21.15.165:0/1372569383 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c004d3b0 con 0x560cb700dbb0 2024-04-25T10:18:36.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.979+0000 7f14bb400640 1 -- 172.21.15.165:0/1372569383 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c0037040 con 0x560cb700dbb0 2024-04-25T10:18:36.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 -- 172.21.15.165:0/1372569383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 msgr2=0x560cb704aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 0x560cb704aa10 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f14c00090b0 tx=0x7f14c002c000 comp rx=0 tx=0).stop 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 -- 172.21.15.165:0/1372569383 shutdown_connections 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb704a280 0x560cb703a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7007aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 --2- 172.21.15.165:0/1372569383 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb700dbb0 0x560cb704aa10 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 -- 172.21.15.165:0/1372569383 >> 172.21.15.165:0/1372569383 conn(0x560cb6fd5a30 msgr2=0x560cb700ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 -- 172.21.15.165:0/1372569383 shutdown_connections 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.980+0000 7f14d923b740 1 -- 172.21.15.165:0/1372569383 wait complete. 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.981+0000 7f14d923b740 1 -- start start 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.981+0000 7f14d923b740 1 Processor -- start 2024-04-25T10:18:36.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb700dbb0 0x560cb700c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7034c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 0x560cb7035190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x560cb70081b0 con 0x560cb704a280 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14d01ff0a0 con 0x560cb700dbb0 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x560cb70755e0 con 0x560cb7038670 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14d923b740 10 client.0 _ll_register_callbacks cb 0x7f14bc058780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 0x560cb7035190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 0x560cb7035190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48006/0 (socket says 172.21.15.165:48006) 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.982+0000 7f14cfe00640 1 -- 172.21.15.165:0/3409164035 learned_addr learned my addr 172.21.15.165:0/3409164035 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cf400640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb700dbb0 0x560cb700c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cea00640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7034c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cfe00640 1 -- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 msgr2=0x560cb7034c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7034c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cfe00640 1 -- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb700dbb0 msgr2=0x560cb700c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb700dbb0 0x560cb700c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cfe00640 1 -- 172.21.15.165:0/3409164035 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560cb7063510 con 0x560cb704a280 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14cfe00640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 0x560cb7035190 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f14c8008f00 tx=0x7f14c8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78875b201af789d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14b8c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14b8c00640 1 -- 172.21.15.165:0/3409164035 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14c8017b90 con 0x560cb704a280 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14b8c00640 1 -- 172.21.15.165:0/3409164035 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f14c8017d30 con 0x560cb704a280 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.983+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f14c004d3b0 con 0x560cb704a280 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14b8c00640 1 -- 172.21.15.165:0/3409164035 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f14c802b050 con 0x560cb704a280 2024-04-25T10:18:36.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 10 client.10551 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x560cb700d500 con 0x560cb704a280 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14b8c00640 1 -- 172.21.15.165:0/3409164035 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14c004d3b0 con 0x560cb704a280 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14b8c00640 1 -- 172.21.15.165:0/3409164035 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f14c802e020 con 0x560cb704a280 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 10 client.10551 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 10 client.10551 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 20 client.10551 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb70b5760 0x560cb702abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 4 client.10551 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.984+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560cb6f79500 con 0x560cb70b5760 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.985+0000 7f14cf400640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb70b5760 0x560cb702abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14cf400640 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb70b5760 0x560cb702abd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228b5962cb05e4b4 server_cookie=35141e73a366a1a1 in_seq=0 out_seq=0 2024-04-25T10:18:36.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14b8c00640 10 client.10551 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:36.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14b8c00640 1 -- 172.21.15.165:0/3409164035 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x560cb6f79500 con 0x560cb70b5760 2024-04-25T10:18:36.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14b8c00640 10 client.10551 handle_command_reply: tid=0 2024-04-25T10:18:36.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14d923b740 1 client.10551 shutdown 2024-04-25T10:18:36.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 msgr2=0x560cb7035190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14d923b740 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 0x560cb7035190 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f14c8008f00 tx=0x7f14c8004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 shutdown_connections 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14d923b740 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x560cb704a280 0x560cb7035190 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.986+0000 7f14d923b740 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x560cb70b5760 0x560cb702abd0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d923b740 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x560cb7038670 0x560cb7034c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d923b740 1 --2- 172.21.15.165:0/3409164035 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x560cb700dbb0 0x560cb700c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14b8c00640 0 client.10551 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 >> 172.21.15.165:0/3409164035 conn(0x560cb6fd5a30 msgr2=0x560cb7067090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 shutdown_connections 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d923b740 1 -- 172.21.15.165:0/3409164035 wait complete. 2024-04-25T10:18:36.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d923b740 20 client.10551 trim_cache size 0 max 16384 2024-04-25T10:18:36.997 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:18:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:18:36.997 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:18:36.997 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:18:36.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.987+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f14b006b4d0 msgr2=0x7f14b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f14b006b4d0 0x7f14b006d990 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f14c0034000 tx=0x7f14c002e000 comp rx=0 tx=0).stop 2024-04-25T10:18:36.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 msgr2=0x7f14d015e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 0x7f14d015e130 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7f14bc00c7f0 tx=0x7f14bc00cbf0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 shutdown_connections 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f14b006b4d0 0x7f14b006d990 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f14d015b8a0 0x7f14d015e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f14d00af520 0x7f14d015e130 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 --2- 172.21.15.165:0/3012531854 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f14d006b660 0x7f14d0141100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 >> 172.21.15.165:0/3012531854 conn(0x7f14d00cfec0 msgr2=0x7f14d006adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 shutdown_connections 2024-04-25T10:18:36.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:36.988+0000 7f14d6400640 1 -- 172.21.15.165:0/3012531854 wait complete. 2024-04-25T10:18:37.009 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:18:37.009 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:18:37.010 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.11:0 damage ls 2024-04-25T10:18:37.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.200+0000 7f63ade00640 1 Processor -- start 2024-04-25T10:18:37.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.200+0000 7f63ade00640 1 -- start start 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63ade00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f63a81446d0 0x7f63a80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63ade00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a80cccc0 con 0x7f63a8144d50 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63ade00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63a80d0460 con 0x7f63a806e8a0 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63ade00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63a80b2b20 con 0x7f63a81446d0 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63a6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f63a81446d0 0x7f63a80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63a6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f63a81446d0 0x7f63a80b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48008/0 (socket says 172.21.15.165:48008) 2024-04-25T10:18:37.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63a6000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f63a8144d50 0x7f63a80cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55212/0 (socket says 172.21.15.165:55212) 2024-04-25T10:18:37.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.202+0000 7f63a6a00640 1 -- 172.21.15.165:0/3900455979 learned_addr learned my addr 172.21.15.165:0/3900455979 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.203+0000 7f63a6a00640 1 -- 172.21.15.165:0/3900455979 >> v1:172.21.15.172:6789/0 conn(0x7f63a806e8a0 legacy=0x7f63a80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.203+0000 7f63a6a00640 1 -- 172.21.15.165:0/3900455979 >> v1:172.21.15.165:6789/0 conn(0x7f63a8144d50 legacy=0x7f63a80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.203+0000 7f63a6a00640 1 -- 172.21.15.165:0/3900455979 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80cc140 con 0x7f63a81446d0 2024-04-25T10:18:37.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.203+0000 7f63a5600640 1 -- 172.21.15.165:0/3900455979 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965017586 0 0) 0x7f63a80cccc0 con 0x7f63a8144d50 2024-04-25T10:18:37.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.204+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3900455979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a81446d0 0x7f63a80b3f20 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f6398009870 tx=0x7f639802a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=201fdf4c187b1438 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.204+0000 7f63a5600640 1 -- 172.21.15.165:0/3900455979 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398037070 con 0x7f63a81446d0 2024-04-25T10:18:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.204+0000 7f63a5600640 1 -- 172.21.15.165:0/3900455979 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398032440 con 0x7f63a81446d0 2024-04-25T10:18:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.204+0000 7f63a5600640 1 -- 172.21.15.165:0/3900455979 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398032810 con 0x7f63a81446d0 2024-04-25T10:18:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 -- 172.21.15.165:0/3900455979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a81446d0 msgr2=0x7f63a80b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 --2- 172.21.15.165:0/3900455979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a81446d0 0x7f63a80b3f20 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f6398009870 tx=0x7f639802a2a0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 -- 172.21.15.165:0/3900455979 shutdown_connections 2024-04-25T10:18:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 --2- 172.21.15.165:0/3900455979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a81446d0 0x7f63a80b3f20 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 -- 172.21.15.165:0/3900455979 >> 172.21.15.165:0/3900455979 conn(0x7f63a806abe0 msgr2=0x7f63a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 -- 172.21.15.165:0/3900455979 shutdown_connections 2024-04-25T10:18:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.205+0000 7f63ade00640 1 -- 172.21.15.165:0/3900455979 wait complete. 2024-04-25T10:18:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.207+0000 7f63ade00640 1 Processor -- start 2024-04-25T10:18:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.207+0000 7f63ade00640 1 -- start start 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63ade00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63a806e8a0 0x7f63a80cb080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63ade00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63a81446d0 0x7f63a80ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63ade00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 0x7f63a81429c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63ade00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80cc700 con 0x7f63a8144d50 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63ade00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a806d400 con 0x7f63a806e8a0 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63ade00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80cc9e0 con 0x7f63a81446d0 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 0x7f63a81429c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 0x7f63a81429c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48010/0 (socket says 172.21.15.165:48010) 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 -- 172.21.15.165:0/3119045867 learned_addr learned my addr 172.21.15.165:0/3119045867 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63a806e8a0 0x7f63a80cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a6000640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63a81446d0 0x7f63a80ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 -- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63a81446d0 msgr2=0x7f63a80ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63a81446d0 0x7f63a80ac3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 -- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63a806e8a0 msgr2=0x7f63a80cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63a806e8a0 0x7f63a80cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a7400640 1 -- 172.21.15.165:0/3119045867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a80a9750 con 0x7f63a8144d50 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a6000640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63a81446d0 0x7f63a80ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.208+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63a806e8a0 0x7f63a80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.209+0000 7f63a7400640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 0x7f63a81429c0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f63a000c780 tx=0x7f63a000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.209+0000 7f639f400640 1 -- 172.21.15.165:0/3119045867 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a0004b60 con 0x7f63a8144d50 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.209+0000 7f639f400640 1 -- 172.21.15.165:0/3119045867 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63a0004d00 con 0x7f63a8144d50 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.209+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63a80a9a30 con 0x7f63a8144d50 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.209+0000 7f639f400640 1 -- 172.21.15.165:0/3119045867 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63a000aa90 con 0x7f63a8144d50 2024-04-25T10:18:37.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.209+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63a80a9ff0 con 0x7f63a8144d50 2024-04-25T10:18:37.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.211+0000 7f639f400640 1 -- 172.21.15.165:0/3119045867 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63a000ac30 con 0x7f63a8144d50 2024-04-25T10:18:37.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.212+0000 7f639f400640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f638806b4d0 0x7f638806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.212+0000 7f639f400640 1 -- 172.21.15.165:0/3119045867 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f63a008a690 con 0x7f63a8144d50 2024-04-25T10:18:37.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.213+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f638806b4d0 0x7f638806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.213+0000 7f63b0bf7740 1 Processor -- start 2024-04-25T10:18:37.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.213+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f638806b4d0 0x7f638806d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6398034000 tx=0x7f639802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.213+0000 7f63b0bf7740 1 -- start start 2024-04-25T10:18:37.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c30200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 0x55c419c4dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c05b00 0x55c419c4bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c419b68d00 con 0x55c419c2fe20 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c419b11f50 con 0x55c419c05b00 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c419b64da0 con 0x55c419c4d6c0 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c30200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c30200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48022/0 (socket says 172.21.15.165:48022) 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.214+0000 7f63a6a00640 1 -- 172.21.15.165:0/1896939219 learned_addr learned my addr 172.21.15.165:0/1896939219 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f63a6a00640 1 -- 172.21.15.165:0/1896939219 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 msgr2=0x55c419c4dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f63a6a00640 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 0x55c419c4dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f63a6a00640 1 -- 172.21.15.165:0/1896939219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c05b00 msgr2=0x55c419c4bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f63a6a00640 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c05b00 0x55c419c4bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f63a6a00640 1 -- 172.21.15.165:0/1896939219 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c419b35130 con 0x55c419c2fe20 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f63a6a00640 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c30200 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f63980090b0 tx=0x7f639802c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df6e29a6752bb34d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f6393400640 1 -- 172.21.15.165:0/1896939219 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639804e070 con 0x55c419c2fe20 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f6393400640 1 -- 172.21.15.165:0/1896939219 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398049470 con 0x55c419c2fe20 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.215+0000 7f6393400640 1 -- 172.21.15.165:0/1896939219 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398037040 con 0x55c419c2fe20 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1896939219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 msgr2=0x55c419c30200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c30200 secure :-1 s=READY pgs=2315 cs=0 l=1 rev1=1 crypto rx=0x7f63980090b0 tx=0x7f639802c000 comp rx=0 tx=0).stop 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1896939219 shutdown_connections 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c05b00 0x55c419c4bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 0x55c419c4dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1896939219 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c30200 unknown :-1 s=CLOSED pgs=2315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1896939219 >> 172.21.15.165:0/1896939219 conn(0x55c419c02430 msgr2=0x55c419c4ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1896939219 shutdown_connections 2024-04-25T10:18:37.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.216+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1896939219 wait complete. 2024-04-25T10:18:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.217+0000 7f63b0bf7740 1 -- start start 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.217+0000 7f63b0bf7740 1 Processor -- start 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c5aca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 0x55c419c5e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c41ce0 0x55c419c5eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c419c068a0 con 0x55c419c2fe20 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c419b07530 con 0x55c419c41ce0 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c419c00360 con 0x55c419c4d6c0 2024-04-25T10:18:37.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63b0bf7740 10 client.0 _ll_register_callbacks cb 0x55c419b60430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.218+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c5aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c5aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48024/0 (socket says 172.21.15.165:48024) 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 -- 172.21.15.165:0/3151759623 learned_addr learned my addr 172.21.15.165:0/3151759623 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 -- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 msgr2=0x55c419c5e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 0x55c419c5e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 -- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c41ce0 msgr2=0x55c419c5eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c41ce0 0x55c419c5eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 -- 172.21.15.165:0/3151759623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c419b2fdd0 con 0x55c419c2fe20 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63a6a00640 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c5aca0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f6398000c00 tx=0x7f6398004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b77b17430694d99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f6387e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:37.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f6387e00640 1 -- 172.21.15.165:0/3151759623 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f639804e070 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f6387e00640 1 -- 172.21.15.165:0/3151759623 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6398053c50 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.219+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6398049470 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f6387e00640 1 -- 172.21.15.165:0/3151759623 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f639805a050 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 10 client.10566 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55c41987d3f0 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f6387e00640 1 -- 172.21.15.165:0/3151759623 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6398049470 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f6387e00640 1 -- 172.21.15.165:0/3151759623 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f639805d020 con 0x55c419c2fe20 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 10 client.10566 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 10 client.10566 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 20 client.10566 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:37.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c2d3d0 0x55c419c2d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 4 client.10566 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.220+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c419b76240 con 0x55c419c2d3d0 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.221+0000 7f63a6000640 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c2d3d0 0x55c419c2d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.222+0000 7f63a6000640 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c2d3d0 0x55c419c2d810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75a233d7816c03b3 server_cookie=1fe6d807d946ecb0 in_seq=0 out_seq=0 2024-04-25T10:18:37.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.222+0000 7f6387e00640 10 client.10566 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f6387e00640 1 -- 172.21.15.165:0/3151759623 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c419b76240 con 0x55c419c2d3d0 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f6387e00640 10 client.10566 handle_command_reply: tid=0 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 client.10566 shutdown 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 msgr2=0x55c419c5aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c5aca0 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7f6398000c00 tx=0x7f6398004730 comp rx=0 tx=0).stop 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 shutdown_connections 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c41ce0 0x55c419c5eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c2d3d0 0x55c419c2d810 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c4d6c0 0x55c419c5e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/3151759623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c5aca0 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f6387e00640 0 client.10566 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 >> 172.21.15.165:0/3151759623 conn(0x55c419c05b00 msgr2=0x55c419c4e1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 shutdown_connections 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 1 -- 172.21.15.165:0/3151759623 wait complete. 2024-04-25T10:18:37.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.224+0000 7f63b0bf7740 20 client.10566 trim_cache size 0 max 16384 2024-04-25T10:18:37.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.237+0000 7f63b0bf7740 1 Processor -- start 2024-04-25T10:18:37.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.237+0000 7f63b0bf7740 1 -- start start 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.238+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c2fe20 0x55c419c4b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.238+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 0x55c419c6ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.238+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.238+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a81fe0c0 con 0x55c419c41ce0 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.238+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a81fd2a0 con 0x55c419c4d6c0 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.238+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63a80ab8e0 con 0x55c419c2fe20 2024-04-25T10:18:37.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 0x55c419c6ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 0x55c419c6ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48030/0 (socket says 172.21.15.165:48030) 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 -- 172.21.15.165:0/801803572 learned_addr learned my addr 172.21.15.165:0/801803572 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6a00640 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c2fe20 0x55c419c4b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 -- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c2fe20 msgr2=0x55c419c4b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c2fe20 0x55c419c4b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 -- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 msgr2=0x55c419c4cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 -- 172.21.15.165:0/801803572 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4199477d0 con 0x55c419c41ce0 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f63a6000640 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 0x55c419c6ca00 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f6394001540 tx=0x7f6394007410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fbdee50fddb0aaf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.239+0000 7f6392a00640 1 -- 172.21.15.165:0/801803572 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6394001c00 con 0x55c419c41ce0 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f6392a00640 1 -- 172.21.15.165:0/801803572 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940146d0 con 0x55c419c41ce0 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f6392a00640 1 -- 172.21.15.165:0/801803572 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63940047e0 con 0x55c419c41ce0 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 -- 172.21.15.165:0/801803572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 msgr2=0x55c419c6ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 0x55c419c6ca00 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7f6394001540 tx=0x7f6394007410 comp rx=0 tx=0).stop 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 -- 172.21.15.165:0/801803572 shutdown_connections 2024-04-25T10:18:37.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c41ce0 0x55c419c6ca00 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/801803572 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c2fe20 0x55c419c4b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 -- 172.21.15.165:0/801803572 >> 172.21.15.165:0/801803572 conn(0x55c419c05b00 msgr2=0x55c419c2e890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 -- 172.21.15.165:0/801803572 shutdown_connections 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.240+0000 7f63b0bf7740 1 -- 172.21.15.165:0/801803572 wait complete. 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.241+0000 7f63b0bf7740 1 -- start start 2024-04-25T10:18:37.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.241+0000 7f63b0bf7740 1 Processor -- start 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c912c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c41ce0 0x55c419c327b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c419c49870 con 0x55c419c2fe20 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63a81fcca0 con 0x55c419c4d6c0 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c419c00ee0 con 0x55c419c41ce0 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63b0bf7740 10 client.0 _ll_register_callbacks cb 0x55c419c00840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:18:37.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.242+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c912c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c912c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48032/0 (socket says 172.21.15.165:48032) 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 -- 172.21.15.165:0/1722572450 learned_addr learned my addr 172.21.15.165:0/1722572450 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a7400640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6000640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c41ce0 0x55c419c327b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 -- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c41ce0 msgr2=0x55c419c327b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c41ce0 0x55c419c327b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 -- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 msgr2=0x55c419c4ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 -- 172.21.15.165:0/1722572450 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c419c6d0b0 con 0x55c419c2fe20 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a7400640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f63a6a00640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c912c0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f63980090b0 tx=0x7f6398008030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=956fa25452588ec1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.243+0000 7f6387e00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f6387e00640 1 -- 172.21.15.165:0/1722572450 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6398063060 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f6387e00640 1 -- 172.21.15.165:0/1722572450 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f639803e460 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63940146d0 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f6387e00640 1 -- 172.21.15.165:0/1722572450 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=113) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6398069050 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 10 client.10572 fetch_fsmap learned FSMap version 113 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=113,monmap=2+}) v3 -- 0x55c419c94600 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f6387e00640 1 -- 172.21.15.165:0/1722572450 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63940146d0 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f6387e00640 1 -- 172.21.15.165:0/1722572450 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 113) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f639806c020 con 0x55c419c2fe20 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 10 client.10572 fetch_fsmap finished waiting for FSMap version 113 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 10 client.10572 resolve_mds: resolved 11:0 to role '11:0' aka daemon mds.d 2024-04-25T10:18:37.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 20 client.10572 populate_metadata read hostname 'smithi165' 2024-04-25T10:18:37.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c4e720 0x55c419c4b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 4 client.10572 mds_command: new command op to 6742 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:18:37.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.244+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 --> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c419c43440 con 0x55c419c4e720 2024-04-25T10:18:37.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.245+0000 7f63a6000640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c4e720 0x55c419c4b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.246+0000 7f63a6000640 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c4e720 0x55c419c4b170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54cf103ff92cd017 server_cookie=24474eaf79e135be in_seq=0 out_seq=0 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.246+0000 7f6387e00640 10 client.10572 ms_handle_connect on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.246+0000 7f6387e00640 1 -- 172.21.15.165:0/1722572450 <== mds.0 v2:172.21.15.172:6832/1799333237 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c419c43440 con 0x55c419c4e720 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.246+0000 7f6387e00640 10 client.10572 handle_command_reply: tid=0 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.246+0000 7f63b0bf7740 1 client.10572 shutdown 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 msgr2=0x55c419c912c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c912c0 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f63980090b0 tx=0x7f6398008030 comp rx=0 tx=0).stop 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 shutdown_connections 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c419c4d6c0 0x55c419c4ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:6832/1799333237,v1:172.21.15.172:6834/1799333237] conn(0x55c419c4e720 0x55c419c4b170 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c419c41ce0 0x55c419c327b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 --2- 172.21.15.165:0/1722572450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c419c2fe20 0x55c419c912c0 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f6387e00640 0 client.10572 ms_handle_reset on v2:172.21.15.172:6832/1799333237 2024-04-25T10:18:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 >> 172.21.15.165:0/1722572450 conn(0x55c419c05b00 msgr2=0x55c419c2e840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 shutdown_connections 2024-04-25T10:18:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 1 -- 172.21.15.165:0/1722572450 wait complete. 2024-04-25T10:18:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.247+0000 7f63b0bf7740 20 client.10572 trim_cache size 0 max 16384 2024-04-25T10:18:37.257 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-25T10:18:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f638806b4d0 msgr2=0x7f638806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f638806b4d0 0x7f638806d990 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6398034000 tx=0x7f639802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 msgr2=0x7f63a81429c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 0x7f63a81429c0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f63a000c780 tx=0x7f63a000cc40 comp rx=0 tx=0).stop 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 shutdown_connections 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f638806b4d0 0x7f638806d990 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63a8144d50 0x7f63a81429c0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63a81446d0 0x7f63a80ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 --2- 172.21.15.165:0/3119045867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63a806e8a0 0x7f63a80cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 >> 172.21.15.165:0/3119045867 conn(0x7f63a806abe0 msgr2=0x7f63a80b1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 shutdown_connections 2024-04-25T10:18:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.248+0000 7f63ade00640 1 -- 172.21.15.165:0/3119045867 wait complete. 2024-04-25T10:18:37.269 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-25T10:18:37.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.465+0000 7f799d800640 1 Processor -- start 2024-04-25T10:18:37.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.466+0000 7f799d800640 1 -- start start 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.467+0000 7f799d800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f799815dd30 0x7f79980b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.467+0000 7f799d800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f79980b9770 0x7f79980d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.467+0000 7f799d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79980d1df0 con 0x7f79980d2e90 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.467+0000 7f799d800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79980b7a50 con 0x7f799815dd30 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.467+0000 7f799d800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79980d20d0 con 0x7f79980b9770 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.467+0000 7f7996a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f799815dd30 0x7f79980b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f799815dd30 0x7f79980b9230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48038/0 (socket says 172.21.15.165:48038) 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996a00640 1 -- 172.21.15.165:0/3441379299 learned_addr learned my addr 172.21.15.165:0/3441379299 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996000640 1 --2- 172.21.15.165:0/3441379299 >> v2:172.21.15.172:3300/0 conn(0x7f79980b9770 0x7f79980d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996a00640 1 -- 172.21.15.165:0/3441379299 >> v2:172.21.15.172:3300/0 conn(0x7f79980b9770 msgr2=0x7f79980d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996a00640 1 --2- 172.21.15.165:0/3441379299 >> v2:172.21.15.172:3300/0 conn(0x7f79980b9770 0x7f79980d03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996a00640 1 -- 172.21.15.165:0/3441379299 >> v1:172.21.15.172:6789/0 conn(0x7f79980d2e90 legacy=0x7f79980b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:37.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.468+0000 7f7996a00640 1 -- 172.21.15.165:0/3441379299 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79980d23b0 con 0x7f799815dd30 2024-04-25T10:18:37.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.469+0000 7f7996a00640 1 --2- 172.21.15.165:0/3441379299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f799815dd30 0x7f79980b9230 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f79980b6540 tx=0x7f798802a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=666b0c806258ce4c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f7995600640 1 -- 172.21.15.165:0/3441379299 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988037070 con 0x7f799815dd30 2024-04-25T10:18:37.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f7995600640 1 -- 172.21.15.165:0/3441379299 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798802ae20 con 0x7f799815dd30 2024-04-25T10:18:37.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f7995600640 1 -- 172.21.15.165:0/3441379299 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988032660 con 0x7f799815dd30 2024-04-25T10:18:37.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f799d800640 1 -- 172.21.15.165:0/3441379299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f799815dd30 msgr2=0x7f79980b9230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f799d800640 1 --2- 172.21.15.165:0/3441379299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f799815dd30 0x7f79980b9230 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f79980b6540 tx=0x7f798802a110 comp rx=0 tx=0).stop 2024-04-25T10:18:37.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f799d800640 1 -- 172.21.15.165:0/3441379299 shutdown_connections 2024-04-25T10:18:37.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f799d800640 1 --2- 172.21.15.165:0/3441379299 >> v2:172.21.15.172:3300/0 conn(0x7f79980b9770 0x7f79980d03c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f799d800640 1 --2- 172.21.15.165:0/3441379299 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f799815dd30 0x7f79980b9230 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.470+0000 7f799d800640 1 -- 172.21.15.165:0/3441379299 >> 172.21.15.165:0/3441379299 conn(0x7f7998061c30 msgr2=0x7f799806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.471+0000 7f799d800640 1 -- 172.21.15.165:0/3441379299 shutdown_connections 2024-04-25T10:18:37.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.471+0000 7f799d800640 1 -- 172.21.15.165:0/3441379299 wait complete. 2024-04-25T10:18:37.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.472+0000 7f799d800640 1 Processor -- start 2024-04-25T10:18:37.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.472+0000 7f799d800640 1 -- start start 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.473+0000 7f799d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 0x7f7998152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.473+0000 7f799d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79980d2e90 0x7f79981545c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.473+0000 7f799d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f799815dd30 0x7f7998154b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.473+0000 7f799d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79980d1a10 con 0x7f79980b9770 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.473+0000 7f799d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79980d2970 con 0x7f79980d2e90 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.473+0000 7f799d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79980d1b90 con 0x7f799815dd30 2024-04-25T10:18:37.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 0x7f7998152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 0x7f7998152480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48040/0 (socket says 172.21.15.165:48040) 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 -- 172.21.15.165:0/4281634227 learned_addr learned my addr 172.21.15.165:0/4281634227 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 -- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f799815dd30 msgr2=0x7f7998154b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f799815dd30 0x7f7998154b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 -- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79980d2e90 msgr2=0x7f79981545c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996000640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79980d2e90 0x7f79981545c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79980d2e90 0x7f79981545c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 -- 172.21.15.165:0/4281634227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7998153420 con 0x7f79980b9770 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996000640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79980d2e90 0x7f79981545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:37.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.474+0000 7f7996a00640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 0x7f7998152480 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f7988000c00 tx=0x7f79880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.475+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988041070 con 0x7f79980b9770 2024-04-25T10:18:37.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.475+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f798803c670 con 0x7f79980b9770 2024-04-25T10:18:37.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.475+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7988037040 con 0x7f79980b9770 2024-04-25T10:18:37.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.475+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79980cbed0 con 0x7f79980b9770 2024-04-25T10:18:37.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.475+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79980cc490 con 0x7f79980b9770 2024-04-25T10:18:37.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.476+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7998073990 con 0x7f79980b9770 2024-04-25T10:18:37.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.477+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7988044070 con 0x7f79980b9770 2024-04-25T10:18:37.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.478+0000 7f798ea00640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f797406b4d0 0x7f797406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:37.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.478+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f79880aac50 con 0x7f79980b9770 2024-04-25T10:18:37.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.478+0000 7f7996000640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f797406b4d0 0x7f797406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:37.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.479+0000 7f7996000640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f797406b4d0 0x7f797406d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7980006fd0 tx=0x7f7980008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:37.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.483+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7988079810 con 0x7f79980b9770 2024-04-25T10:18:37.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.718+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f79980d13d0 con 0x7f79980b9770 2024-04-25T10:18:37.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.718+0000 7f798ea00640 1 -- 172.21.15.165:0/4281634227 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f798802ae20 con 0x7f79980b9770 2024-04-25T10:18:37.722 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:37.723 INFO:teuthology.orchestra.run.smithi165.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-25T10:18:37.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f797406b4d0 msgr2=0x7f797406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f797406b4d0 0x7f797406d990 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7980006fd0 tx=0x7f7980008040 comp rx=0 tx=0).stop 2024-04-25T10:18:37.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 msgr2=0x7f7998152480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:37.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 0x7f7998152480 secure :-1 s=READY pgs=2320 cs=0 l=1 rev1=1 crypto rx=0x7f7988000c00 tx=0x7f79880046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 shutdown_connections 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f797406b4d0 0x7f797406d990 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f799815dd30 0x7f7998154b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f79980d2e90 0x7f79981545c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 --2- 172.21.15.165:0/4281634227 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f79980b9770 0x7f7998152480 unknown :-1 s=CLOSED pgs=2320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 >> 172.21.15.165:0/4281634227 conn(0x7f7998061c30 msgr2=0x7f79980d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 shutdown_connections 2024-04-25T10:18:37.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:37.721+0000 7f799d800640 1 -- 172.21.15.165:0/4281634227 wait complete. 2024-04-25T10:18:37.743 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:18:37.743 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:37.839 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:37.946 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:18:37.946 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:18:37.946 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:38.002 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:18:38.003 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:38.057 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:38.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.250+0000 7fa8c9a00640 1 Processor -- start 2024-04-25T10:18:38.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.250+0000 7fa8c9a00640 1 -- start start 2024-04-25T10:18:38.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.251+0000 7fa8c9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa8c4144750 0x7fa8c406e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c9a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c4144b30 con 0x7fa8c40cf030 2024-04-25T10:18:38.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c9a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40712d0 con 0x7fa8c406eb10 2024-04-25T10:18:38.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c9a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40715b0 con 0x7fa8c4144750 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa8c40cf030 0x7fa8c406d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55214/0 (socket says 172.21.15.165:55214) 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c3400640 1 -- 172.21.15.165:0/2666298284 learned_addr learned my addr 172.21.15.165:0/2666298284 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c2a00640 1 --2- 172.21.15.165:0/2666298284 >> v2:172.21.15.172:3300/0 conn(0x7fa8c4144750 0x7fa8c406e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.252+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1318966037 0 0) 0x7fa8c4144b30 con 0x7fa8c40cf030 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8a8003520 con 0x7fa8c40cf030 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3397774244 0 0) 0x7fa8a8003520 con 0x7fa8c40cf030 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 >> v2:172.21.15.172:3300/0 conn(0x7fa8c4144750 msgr2=0x7fa8c406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 --2- 172.21.15.165:0/2666298284 >> v2:172.21.15.172:3300/0 conn(0x7fa8c4144750 0x7fa8c406e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 >> v1:172.21.15.172:6789/0 conn(0x7fa8c406eb10 legacy=0x7fa8c40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c4071890 con 0x7fa8c40cf030 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa8bc001670 con 0x7fa8c40cf030 2024-04-25T10:18:38.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 >> v1:172.21.15.165:6789/0 conn(0x7fa8c40cf030 legacy=0x7fa8c406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a8003a00 0x7fa8a8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8004800 0x7fa8a8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a80051c0 0x7fa8a800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40715b0 con 0x7fa8a80051c0 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a800c0c0 con 0x7fa8a8004800 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8a800c2c0 con 0x7fa8a8003a00 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.253+0000 7fa8c3400640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a80051c0 0x7fa8a800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8c2a00640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a8003a00 0x7fa8a8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8ba000640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8004800 0x7fa8a8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8c3400640 1 -- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a8003a00 msgr2=0x7fa8a8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8c3400640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8a8003a00 0x7fa8a8003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8c3400640 1 -- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8004800 msgr2=0x7fa8a8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8c3400640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a8004800 0x7fa8a8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.254+0000 7fa8c3400640 1 -- 172.21.15.165:0/2666298284 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a8003520 con 0x7fa8a80051c0 2024-04-25T10:18:38.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.255+0000 7fa8c3400640 1 -- 172.21.15.165:0/2666298284 reap_dead start 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.255+0000 7fa8c3400640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a80051c0 0x7fa8a800b9c0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c110 tx=0x7fa8a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec98317bc5544505 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.255+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc007c40 con 0x7fa8a80051c0 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.255+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc005af0 con 0x7fa8a80051c0 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.255+0000 7fa8c2000640 1 -- 172.21.15.165:0/2666298284 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc007070 con 0x7fa8a80051c0 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2666298284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a80051c0 msgr2=0x7fa8a800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a80051c0 0x7fa8a800b9c0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc00c110 tx=0x7fa8a8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2666298284 shutdown_connections 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2666298284 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8a80051c0 0x7fa8a800b9c0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2666298284 >> 172.21.15.165:0/2666298284 conn(0x7fa8c406abe0 msgr2=0x7fa8c406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2666298284 shutdown_connections 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.256+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2666298284 wait complete. 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.258+0000 7fa8c9a00640 1 Processor -- start 2024-04-25T10:18:38.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.258+0000 7fa8c9a00640 1 -- start start 2024-04-25T10:18:38.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a80051c0 0x7fa8c41ef870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 0x7fa8c4139db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c415b130 0x7fa8c413a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c9a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c41455d0 con 0x7fa8c415acb0 2024-04-25T10:18:38.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4145450 con 0x7fa8a80051c0 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c415c000 con 0x7fa8c415b130 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 0x7fa8c4139db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 0x7fa8c4139db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48048/0 (socket says 172.21.15.165:48048) 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 -- 172.21.15.165:0/2884047106 learned_addr learned my addr 172.21.15.165:0/2884047106 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 -- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c415b130 msgr2=0x7fa8c413a2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c3400640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c415b130 0x7fa8c413a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c415b130 0x7fa8c413a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8c2a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a80051c0 0x7fa8c41ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 -- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a80051c0 msgr2=0x7fa8c41ef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a80051c0 0x7fa8c41ef870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.259+0000 7fa8ba000640 1 -- 172.21.15.165:0/2884047106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c41efdb0 con 0x7fa8c415acb0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.260+0000 7fa8ba000640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 0x7fa8c4139db0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00e440 tx=0x7fa8ac00e900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.260+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac019070 con 0x7fa8c415acb0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.260+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8ac0053a0 con 0x7fa8c415acb0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.260+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c4135040 con 0x7fa8c415acb0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.260+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ac0037f0 con 0x7fa8c415acb0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.260+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c41355e0 con 0x7fa8c415acb0 2024-04-25T10:18:38.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.261+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c4073990 con 0x7fa8c415acb0 2024-04-25T10:18:38.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.262+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa8ac01c070 con 0x7fa8c415acb0 2024-04-25T10:18:38.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.263+0000 7fa8bb400640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8a006b4d0 0x7fa8a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.263+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa8ac089850 con 0x7fa8c415acb0 2024-04-25T10:18:38.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.264+0000 7fa8c2a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8a006b4d0 0x7fa8a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.264+0000 7fa8c2a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8a006b4d0 0x7fa8a006d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa8c41f2d00 tx=0x7fa8b400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.268+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa8c41efdb0 con 0x7fa8c415acb0 2024-04-25T10:18:38.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.464+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8c40d0510 con 0x7fa8c415acb0 2024-04-25T10:18:38.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.467+0000 7fa8bb400640 1 -- 172.21.15.165:0/2884047106 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16920 (secure 0 0 0) 0x7fa8ac057ef0 con 0x7fa8c415acb0 2024-04-25T10:18:38.471 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:38.471 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":159,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:53.373638+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:38.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8a006b4d0 msgr2=0x7fa8a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8a006b4d0 0x7fa8a006d990 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fa8c41f2d00 tx=0x7fa8b400b040 comp rx=0 tx=0).stop 2024-04-25T10:18:38.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 msgr2=0x7fa8c4139db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 0x7fa8c4139db0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7fa8ac00e440 tx=0x7fa8ac00e900 comp rx=0 tx=0).stop 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 shutdown_connections 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa8a006b4d0 0x7fa8a006d990 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa8c415b130 0x7fa8c413a2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa8c415acb0 0x7fa8c4139db0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 --2- 172.21.15.165:0/2884047106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa8a80051c0 0x7fa8c41ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 >> 172.21.15.165:0/2884047106 conn(0x7fa8c406abe0 msgr2=0x7fa8c40b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 shutdown_connections 2024-04-25T10:18:38.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.470+0000 7fa8c9a00640 1 -- 172.21.15.165:0/2884047106 wait complete. 2024-04-25T10:18:38.492 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:49.587055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:50.826716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:38.493 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:38.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.679+0000 7f8877200640 1 Processor -- start 2024-04-25T10:18:38.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.680+0000 7f8877200640 1 -- start start 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8877200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f887015b8e0 0x7f88700cfd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8877200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f88701601e0 0x7f887015f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8877200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8870099ae0 con 0x7f887015bcc0 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8877200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8870099dc0 con 0x7f887015b8e0 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8877200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f887009a0a0 con 0x7f88701601e0 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8875e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f887015b8e0 0x7f88700cfd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8875e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f887015b8e0 0x7f88700cfd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48064/0 (socket says 172.21.15.165:48064) 2024-04-25T10:18:38.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.681+0000 7f8875e00640 1 -- 172.21.15.165:0/2391713893 learned_addr learned my addr 172.21.15.165:0/2391713893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:38.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.682+0000 7f8876800640 1 --2- 172.21.15.165:0/2391713893 >> v2:172.21.15.172:3300/0 conn(0x7f88701601e0 0x7f887015f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.682+0000 7f8875e00640 1 -- 172.21.15.165:0/2391713893 >> v2:172.21.15.172:3300/0 conn(0x7f88701601e0 msgr2=0x7f887015f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.682+0000 7f8875e00640 1 --2- 172.21.15.165:0/2391713893 >> v2:172.21.15.172:3300/0 conn(0x7f88701601e0 0x7f887015f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.682+0000 7f8875e00640 1 -- 172.21.15.165:0/2391713893 >> v1:172.21.15.172:6789/0 conn(0x7f887015bcc0 legacy=0x7f887009a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:38.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.682+0000 7f8875e00640 1 -- 172.21.15.165:0/2391713893 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f887009a380 con 0x7f887015b8e0 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.683+0000 7f8875e00640 1 --2- 172.21.15.165:0/2391713893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f887015b8e0 0x7f88700cfd40 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f887009b2e0 tx=0x7f886402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec9d308a554aac61 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.683+0000 7f8874a00640 1 -- 172.21.15.165:0/2391713893 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864037070 con 0x7f887015b8e0 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.683+0000 7f8874a00640 1 -- 172.21.15.165:0/2391713893 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8864032440 con 0x7f887015b8e0 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.683+0000 7f8874a00640 1 -- 172.21.15.165:0/2391713893 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8864032810 con 0x7f887015b8e0 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 -- 172.21.15.165:0/2391713893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f887015b8e0 msgr2=0x7f88700cfd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 --2- 172.21.15.165:0/2391713893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f887015b8e0 0x7f88700cfd40 secure :-1 s=READY pgs=2324 cs=0 l=1 rev1=1 crypto rx=0x7f887009b2e0 tx=0x7f886402a230 comp rx=0 tx=0).stop 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 -- 172.21.15.165:0/2391713893 shutdown_connections 2024-04-25T10:18:38.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 --2- 172.21.15.165:0/2391713893 >> v2:172.21.15.172:3300/0 conn(0x7f88701601e0 0x7f887015f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 --2- 172.21.15.165:0/2391713893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f887015b8e0 0x7f88700cfd40 unknown :-1 s=CLOSED pgs=2324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 -- 172.21.15.165:0/2391713893 >> 172.21.15.165:0/2391713893 conn(0x7f88700b3250 msgr2=0x7f88700b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 -- 172.21.15.165:0/2391713893 shutdown_connections 2024-04-25T10:18:38.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.684+0000 7f8877200640 1 -- 172.21.15.165:0/2391713893 wait complete. 2024-04-25T10:18:38.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.686+0000 7f8877200640 1 Processor -- start 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.686+0000 7f8877200640 1 -- start start 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8877200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f887015b8e0 0x7f8870154c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8877200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f887015bcc0 0x7f88700c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8877200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 0x7f8870157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8877200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700d0710 con 0x7f88701601e0 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8877200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88700d0590 con 0x7f887015bcc0 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8877200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f887015fb10 con 0x7f887015b8e0 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8876800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 0x7f8870157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8876800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 0x7f8870157ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48080/0 (socket says 172.21.15.165:48080) 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8876800640 1 -- 172.21.15.165:0/1951572676 learned_addr learned my addr 172.21.15.165:0/1951572676 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.687+0000 7f8875e00640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f887015b8e0 0x7f8870154c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8875400640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f887015bcc0 0x7f88700c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8876800640 1 -- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f887015b8e0 msgr2=0x7f8870154c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8876800640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f887015b8e0 0x7f8870154c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8876800640 1 -- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f887015bcc0 msgr2=0x7f88700c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8876800640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f887015bcc0 0x7f88700c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8876800640 1 -- 172.21.15.165:0/1951572676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88700c9130 con 0x7f88701601e0 2024-04-25T10:18:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8875400640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f887015bcc0 0x7f88700c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:38.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8876800640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 0x7f8870157ff0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f886c00cc60 tx=0x7f886c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c003990 con 0x7f88701601e0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f886c003b30 con 0x7f88701601e0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88700c9410 con 0x7f88701601e0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f886c004020 con 0x7f88701601e0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.688+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88700b44b0 con 0x7f88701601e0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.689+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88700b4010 con 0x7f88701601e0 2024-04-25T10:18:38.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.690+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f886c0041c0 con 0x7f88701601e0 2024-04-25T10:18:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.691+0000 7f886a000640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f885406b4d0 0x7f885406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.691+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f886c08a920 con 0x7f88701601e0 2024-04-25T10:18:38.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.692+0000 7f8875e00640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f885406b4d0 0x7f885406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:38.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.692+0000 7f8875e00640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f885406b4d0 0x7f885406d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8864034000 tx=0x7f886402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:38.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.695+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f886c014030 con 0x7f88701601e0 2024-04-25T10:18:38.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.891+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f88700cd330 con 0x7f88701601e0 2024-04-25T10:18:38.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.892+0000 7f886a000640 1 -- 172.21.15.165:0/1951572676 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7f886c059570 con 0x7f88701601e0 2024-04-25T10:18:38.896 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:18:38.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f885406b4d0 msgr2=0x7f885406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f885406b4d0 0x7f885406d990 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f8864034000 tx=0x7f886402e000 comp rx=0 tx=0).stop 2024-04-25T10:18:38.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 msgr2=0x7f8870157ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 0x7f8870157ff0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f886c00cc60 tx=0x7f886c00a5c0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 shutdown_connections 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f885406b4d0 0x7f885406d990 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f88701601e0 0x7f8870157ff0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f887015bcc0 0x7f88700c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 --2- 172.21.15.165:0/1951572676 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f887015b8e0 0x7f8870154c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 >> 172.21.15.165:0/1951572676 conn(0x7f88700b3250 msgr2=0x7f887009a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 shutdown_connections 2024-04-25T10:18:38.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:38.895+0000 7f8877200640 1 -- 172.21.15.165:0/1951572676 wait complete. 2024-04-25T10:18:38.917 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:39.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.110+0000 7f8863200640 1 Processor -- start 2024-04-25T10:18:39.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.110+0000 7f8863200640 1 -- start start 2024-04-25T10:18:39.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8863200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f88640cc290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8863200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f886409a6c0 0x7f88640c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8863200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88640cd430 con 0x7f886409ab20 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8863200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8864099be0 con 0x7f886409a6c0 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8863200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8864099ec0 con 0x7f886415b900 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8859400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f886409a6c0 0x7f88640c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8862800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f886409ab20 0x7f88640cf800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55228/0 (socket says 172.21.15.165:55228) 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.112+0000 7f8862800640 1 -- 172.21.15.165:0/1466844596 learned_addr learned my addr 172.21.15.165:0/1466844596 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.113+0000 7f8861e00640 1 --2- 172.21.15.165:0/1466844596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f88640cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.113+0000 7f8859400640 1 -- 172.21.15.165:0/1466844596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 msgr2=0x7f88640cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.113+0000 7f8859400640 1 --2- 172.21.15.165:0/1466844596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f88640cc290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.113+0000 7f8859400640 1 -- 172.21.15.165:0/1466844596 >> v1:172.21.15.165:6789/0 conn(0x7f886409ab20 legacy=0x7f88640cf800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.113+0000 7f8859400640 1 -- 172.21.15.165:0/1466844596 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f886409a1a0 con 0x7f886409a6c0 2024-04-25T10:18:39.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.113+0000 7f8861400640 1 -- 172.21.15.165:0/1466844596 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821676774 0 0) 0x7f88640cd430 con 0x7f886409ab20 2024-04-25T10:18:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.114+0000 7f8859400640 1 --2- 172.21.15.165:0/1466844596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409a6c0 0x7f88640c52d0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f884c0097a0 tx=0x7f884c02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=117cd0aeecde332d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.114+0000 7f8861400640 1 -- 172.21.15.165:0/1466844596 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c037070 con 0x7f886409a6c0 2024-04-25T10:18:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.114+0000 7f8861400640 1 -- 172.21.15.165:0/1466844596 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c032440 con 0x7f886409a6c0 2024-04-25T10:18:39.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.114+0000 7f8861400640 1 -- 172.21.15.165:0/1466844596 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c032810 con 0x7f886409a6c0 2024-04-25T10:18:39.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 -- 172.21.15.165:0/1466844596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409a6c0 msgr2=0x7f88640c52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 --2- 172.21.15.165:0/1466844596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409a6c0 0x7f88640c52d0 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7f884c0097a0 tx=0x7f884c02a180 comp rx=0 tx=0).stop 2024-04-25T10:18:39.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 -- 172.21.15.165:0/1466844596 shutdown_connections 2024-04-25T10:18:39.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 --2- 172.21.15.165:0/1466844596 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409a6c0 0x7f88640c52d0 unknown :-1 s=CLOSED pgs=2327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 --2- 172.21.15.165:0/1466844596 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f88640cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 -- 172.21.15.165:0/1466844596 >> 172.21.15.165:0/1466844596 conn(0x7f88640d1780 msgr2=0x7f88640d1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 -- 172.21.15.165:0/1466844596 shutdown_connections 2024-04-25T10:18:39.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.115+0000 7f8863200640 1 -- 172.21.15.165:0/1466844596 wait complete. 2024-04-25T10:18:39.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.117+0000 7f8863200640 1 Processor -- start 2024-04-25T10:18:39.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.117+0000 7f8863200640 1 -- start start 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8863200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f886409a6c0 0x7f886415ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8863200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 0x7f886409eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8863200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f8864070690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8863200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886415be60 con 0x7f886409ab20 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8863200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f886415bce0 con 0x7f886409a6c0 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8863200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88640c5810 con 0x7f886415b900 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 0x7f886409eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 0x7f886409eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48112/0 (socket says 172.21.15.165:48112) 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 -- 172.21.15.165:0/3857346792 learned_addr learned my addr 172.21.15.165:0/3857346792 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8862800640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f8864070690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 -- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 msgr2=0x7f8864070690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f8864070690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 -- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f886409a6c0 msgr2=0x7f886415ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f886409a6c0 0x7f886415ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.118+0000 7f8859400640 1 -- 172.21.15.165:0/3857346792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88641f7910 con 0x7f886409ab20 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.119+0000 7f8859400640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 0x7f886409eb40 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f884c035040 tx=0x7f884c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.119+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c037040 con 0x7f886409ab20 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.119+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f884c030770 con 0x7f886409ab20 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.119+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f884c030b80 con 0x7f886409ab20 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.119+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88641f7bf0 con 0x7f886409ab20 2024-04-25T10:18:39.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.119+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88641f81b0 con 0x7f886409ab20 2024-04-25T10:18:39.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.120+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f886409f650 con 0x7f886409ab20 2024-04-25T10:18:39.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.121+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f884c044020 con 0x7f886409ab20 2024-04-25T10:18:39.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.122+0000 7f885aa00640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f884006b4d0 0x7f884006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.122+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f884c0aa8e0 con 0x7f886409ab20 2024-04-25T10:18:39.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.122+0000 7f8861e00640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f884006b4d0 0x7f884006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.123+0000 7f8861e00640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f884006b4d0 0x7f884006d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8854006fd0 tx=0x7f8854008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.126+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f884c032e20 con 0x7f886409ab20 2024-04-25T10:18:39.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.320+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f88640b7700 con 0x7f886409ab20 2024-04-25T10:18:39.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.321+0000 7f885aa00640 1 -- 172.21.15.165:0/3857346792 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7f884c0795c0 con 0x7f886409ab20 2024-04-25T10:18:39.325 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:18:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f884006b4d0 msgr2=0x7f884006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f884006b4d0 0x7f884006d990 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f8854006fd0 tx=0x7f8854008040 comp rx=0 tx=0).stop 2024-04-25T10:18:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 msgr2=0x7f886409eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 0x7f886409eb40 secure :-1 s=READY pgs=2328 cs=0 l=1 rev1=1 crypto rx=0x7f884c035040 tx=0x7f884c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 shutdown_connections 2024-04-25T10:18:39.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f884006b4d0 0x7f884006d990 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f886415b900 0x7f8864070690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f886409ab20 0x7f886409eb40 unknown :-1 s=CLOSED pgs=2328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 --2- 172.21.15.165:0/3857346792 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f886409a6c0 0x7f886415ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 >> 172.21.15.165:0/3857346792 conn(0x7f88640d1780 msgr2=0x7f886406f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 shutdown_connections 2024-04-25T10:18:39.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.324+0000 7f8863200640 1 -- 172.21.15.165:0/3857346792 wait complete. 2024-04-25T10:18:39.346 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.534+0000 7f0377e00640 1 Processor -- start 2024-04-25T10:18:39.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.535+0000 7f0377e00640 1 -- start start 2024-04-25T10:18:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.536+0000 7f0377e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f037815fa30 0x7f0378162f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.536+0000 7f0377e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f037806abe0 0x7f037806b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0377e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0378165a80 0x7f0378165f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0377e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f037806b580 con 0x7f037815fa30 2024-04-25T10:18:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0377e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03780b5850 con 0x7f0378165a80 2024-04-25T10:18:39.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0377e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037815fe10 con 0x7f037806abe0 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f037815fa30 0x7f0378162f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f037815fa30 0x7f0378162f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48124/0 (socket says 172.21.15.165:48124) 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 -- 172.21.15.165:0/204654837 learned_addr learned my addr 172.21.15.165:0/204654837 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 -- 172.21.15.165:0/204654837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f037806abe0 msgr2=0x7f037806b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 --2- 172.21.15.165:0/204654837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f037806abe0 0x7f037806b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 -- 172.21.15.165:0/204654837 >> v2:172.21.15.172:3300/0 conn(0x7f0378165a80 msgr2=0x7f0378165f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 --2- 172.21.15.165:0/204654837 >> v2:172.21.15.172:3300/0 conn(0x7f0378165a80 0x7f0378165f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.537+0000 7f0376a00640 1 -- 172.21.15.165:0/204654837 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0378070430 con 0x7f037815fa30 2024-04-25T10:18:39.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.538+0000 7f0376a00640 1 --2- 172.21.15.165:0/204654837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f0378162f10 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f03780b6610 tx=0x7f036802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b399f4f9b32545 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0376000640 1 -- 172.21.15.165:0/204654837 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368037070 con 0x7f037815fa30 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0376000640 1 -- 172.21.15.165:0/204654837 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0368032440 con 0x7f037815fa30 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0376000640 1 -- 172.21.15.165:0/204654837 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0368032810 con 0x7f037815fa30 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 -- 172.21.15.165:0/204654837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 msgr2=0x7f0378162f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 --2- 172.21.15.165:0/204654837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f0378162f10 secure :-1 s=READY pgs=2329 cs=0 l=1 rev1=1 crypto rx=0x7f03780b6610 tx=0x7f036802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 -- 172.21.15.165:0/204654837 shutdown_connections 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 --2- 172.21.15.165:0/204654837 >> v2:172.21.15.172:3300/0 conn(0x7f0378165a80 0x7f0378165f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 --2- 172.21.15.165:0/204654837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f037806abe0 0x7f037806b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 --2- 172.21.15.165:0/204654837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f0378162f10 unknown :-1 s=CLOSED pgs=2329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 -- 172.21.15.165:0/204654837 >> 172.21.15.165:0/204654837 conn(0x7f03780b5a40 msgr2=0x7f03780b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 -- 172.21.15.165:0/204654837 shutdown_connections 2024-04-25T10:18:39.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.539+0000 7f0377e00640 1 -- 172.21.15.165:0/204654837 wait complete. 2024-04-25T10:18:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.541+0000 7f0377e00640 1 Processor -- start 2024-04-25T10:18:39.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.541+0000 7f0377e00640 1 -- start start 2024-04-25T10:18:39.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0377e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037806abe0 0x7f03781518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0377e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f037815e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0377e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0378165a80 0x7f037815e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0377e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03780b54e0 con 0x7f037815fa30 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0377e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03780b5200 con 0x7f037806abe0 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0377e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f037806fe00 con 0x7f0378165a80 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f036fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f037815e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f036fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f037815e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48132/0 (socket says 172.21.15.165:48132) 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f036fe00640 1 -- 172.21.15.165:0/3304862154 learned_addr learned my addr 172.21.15.165:0/3304862154 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.542+0000 7f0376a00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037806abe0 0x7f03781518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036fe00640 1 -- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0378165a80 msgr2=0x7f037815e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f0377400640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0378165a80 0x7f037815e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036fe00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0378165a80 0x7f037815e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036fe00640 1 -- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037806abe0 msgr2=0x7f03781518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036fe00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037806abe0 0x7f03781518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036fe00640 1 -- 172.21.15.165:0/3304862154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03781eb980 con 0x7f037815fa30 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036fe00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f037815e410 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f036000b410 tx=0x7f036000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360015020 con 0x7f037815fa30 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0360011850 con 0x7f037815fa30 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0360011c40 con 0x7f037815fa30 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03781ebc60 con 0x7f037815fa30 2024-04-25T10:18:39.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.543+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781ec220 con 0x7f037815fa30 2024-04-25T10:18:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.544+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0378073990 con 0x7f037815fa30 2024-04-25T10:18:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.546+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0360018020 con 0x7f037815fa30 2024-04-25T10:18:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.546+0000 7f036ea00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f035406b4d0 0x7f035406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.546+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f036008a2c0 con 0x7f037815fa30 2024-04-25T10:18:39.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.547+0000 7f0376a00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f035406b4d0 0x7f035406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.547+0000 7f0376a00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f035406b4d0 0x7f035406d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0368034000 tx=0x7f036802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.551+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f036001f020 con 0x7f037815fa30 2024-04-25T10:18:39.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.738+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f037806d330 con 0x7f037815fa30 2024-04-25T10:18:39.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.739+0000 7f036ea00640 1 -- 172.21.15.165:0/3304862154 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03600172c0 con 0x7f037815fa30 2024-04-25T10:18:39.743 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:18:39.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f035406b4d0 msgr2=0x7f035406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f035406b4d0 0x7f035406d990 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f0368034000 tx=0x7f036802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 msgr2=0x7f037815e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f037815e410 secure :-1 s=READY pgs=2330 cs=0 l=1 rev1=1 crypto rx=0x7f036000b410 tx=0x7f036000b8d0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 shutdown_connections 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f035406b4d0 0x7f035406d990 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0378165a80 0x7f037815e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f037815fa30 0x7f037815e410 unknown :-1 s=CLOSED pgs=2330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 --2- 172.21.15.165:0/3304862154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f037806abe0 0x7f03781518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.741+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 >> 172.21.15.165:0/3304862154 conn(0x7f03780b5a40 msgr2=0x7f03780adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.742+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 shutdown_connections 2024-04-25T10:18:39.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.742+0000 7f0377e00640 1 -- 172.21.15.165:0/3304862154 wait complete. 2024-04-25T10:18:39.763 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:18:39.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.952+0000 7fe505400640 1 Processor -- start 2024-04-25T10:18:39.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.952+0000 7fe505400640 1 -- start start 2024-04-25T10:18:39.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe505400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5000b0730 0x7fe5000b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe505400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe50006e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe505400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe50013e640 con 0x7fe50013d9f0 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe505400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe50006d390 con 0x7fe5000b0730 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe505400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe50006d670 con 0x7fe50006e150 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe4ff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5000b0730 0x7fe5000b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe4ff400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe5000b0730 0x7fe5000b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48138/0 (socket says 172.21.15.165:48138) 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe4ff400640 1 -- 172.21.15.165:0/2510684320 learned_addr learned my addr 172.21.15.165:0/2510684320 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.954+0000 7fe504a00640 1 --2- 172.21.15.165:0/2510684320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe50006e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.955+0000 7fe4ff400640 1 -- 172.21.15.165:0/2510684320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 msgr2=0x7fe50006e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.955+0000 7fe4ff400640 1 --2- 172.21.15.165:0/2510684320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe50006e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.955+0000 7fe4ff400640 1 -- 172.21.15.165:0/2510684320 >> v1:172.21.15.172:6789/0 conn(0x7fe50013d9f0 legacy=0x7fe50013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:39.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.955+0000 7fe4ff400640 1 -- 172.21.15.165:0/2510684320 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50006d950 con 0x7fe5000b0730 2024-04-25T10:18:39.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.956+0000 7fe4ff400640 1 --2- 172.21.15.165:0/2510684320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe5000b0bc0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009870 tx=0x7fe4ec02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be07b91ce2080fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.956+0000 7fe4fea00640 1 -- 172.21.15.165:0/2510684320 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec037070 con 0x7fe5000b0730 2024-04-25T10:18:39.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.956+0000 7fe4fea00640 1 -- 172.21.15.165:0/2510684320 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec02ae10 con 0x7fe5000b0730 2024-04-25T10:18:39.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.956+0000 7fe4fea00640 1 -- 172.21.15.165:0/2510684320 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec032660 con 0x7fe5000b0730 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.956+0000 7fe505400640 1 -- 172.21.15.165:0/2510684320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 msgr2=0x7fe5000b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.956+0000 7fe505400640 1 --2- 172.21.15.165:0/2510684320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe5000b0bc0 secure :-1 s=READY pgs=2331 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec009870 tx=0x7fe4ec02a100 comp rx=0 tx=0).stop 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.957+0000 7fe505400640 1 -- 172.21.15.165:0/2510684320 shutdown_connections 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.957+0000 7fe505400640 1 --2- 172.21.15.165:0/2510684320 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe50006e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.957+0000 7fe505400640 1 --2- 172.21.15.165:0/2510684320 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe5000b0bc0 unknown :-1 s=CLOSED pgs=2331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.957+0000 7fe505400640 1 -- 172.21.15.165:0/2510684320 >> 172.21.15.165:0/2510684320 conn(0x7fe500061cc0 msgr2=0x7fe50006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.957+0000 7fe505400640 1 -- 172.21.15.165:0/2510684320 shutdown_connections 2024-04-25T10:18:39.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.957+0000 7fe505400640 1 -- 172.21.15.165:0/2510684320 wait complete. 2024-04-25T10:18:39.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.958+0000 7fe505400640 1 Processor -- start 2024-04-25T10:18:39.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.958+0000 7fe505400640 1 -- start start 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.959+0000 7fe505400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe5001318d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.959+0000 7fe505400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe500157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.959+0000 7fe505400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50013d9f0 0x7fe5000cdba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.959+0000 7fe505400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000ce880 con 0x7fe5000b0730 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.959+0000 7fe505400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe50006eb00 con 0x7fe50013d9f0 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.959+0000 7fe505400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5000cea00 con 0x7fe50006e150 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe500157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe500157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48154/0 (socket says 172.21.15.165:48154) 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 -- 172.21.15.165:0/4268262659 learned_addr learned my addr 172.21.15.165:0/4268262659 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ffe00640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe5001318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe504a00640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50013d9f0 0x7fe5000cdba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 -- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 msgr2=0x7fe5001318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe5001318d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 -- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50013d9f0 msgr2=0x7fe5000cdba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:39.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50013d9f0 0x7fe5000cdba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:39.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 -- 172.21.15.165:0/4268262659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50015bac0 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.960+0000 7fe4ff400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe500157db0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.961+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec041070 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.961+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4ec03c670 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.961+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe500133a30 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.961+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4ec037040 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.961+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe500152690 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.962+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe500073990 con 0x7fe5000b0730 2024-04-25T10:18:39.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.963+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe4ec044070 con 0x7fe5000b0730 2024-04-25T10:18:39.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.964+0000 7fe4f7e00640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4e006b4d0 0x7fe4e006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:39.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.965+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe4ec0aad20 con 0x7fe5000b0730 2024-04-25T10:18:39.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.965+0000 7fe4ffe00640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4e006b4d0 0x7fe4e006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:39.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.965+0000 7fe4ffe00640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4e006b4d0 0x7fe4e006d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:39.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:39.969+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe50015bac0 con 0x7fe5000b0730 2024-04-25T10:18:40.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.160+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fe500074270 con 0x7fe5000b0730 2024-04-25T10:18:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.161+0000 7fe4f7e00640 1 -- 172.21.15.165:0/4268262659 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v159) v1 ==== 80+0+0 (secure 0 0 0) 0x7fe4ec079a90 con 0x7fe5000b0730 2024-04-25T10:18:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:18:40.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4e006b4d0 msgr2=0x7fe4e006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4e006b4d0 0x7fe4e006d990 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7fe4f0006fd0 tx=0x7fe4f0008040 comp rx=0 tx=0).stop 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 msgr2=0x7fe500157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe500157db0 secure :-1 s=READY pgs=2332 cs=0 l=1 rev1=1 crypto rx=0x7fe4ec000c00 tx=0x7fe4ec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 shutdown_connections 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe4e006b4d0 0x7fe4e006d990 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe50013d9f0 0x7fe5000cdba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5000b0730 0x7fe500157db0 unknown :-1 s=CLOSED pgs=2332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 --2- 172.21.15.165:0/4268262659 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe50006e150 0x7fe5001318d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.163+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 >> 172.21.15.165:0/4268262659 conn(0x7fe500061cc0 msgr2=0x7fe50015ac90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.164+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 shutdown_connections 2024-04-25T10:18:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.164+0000 7fe505400640 1 -- 172.21.15.165:0/4268262659 wait complete. 2024-04-25T10:18:40.185 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:18:40.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:40.185 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:18:40.272 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:18:40.272 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:18:40.272 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:18:40.320 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:18:40.320 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:40.378 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:40.378 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:18:40.378 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:40.433 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:40.433 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:18:40.433 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:40.490 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:40.490 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:40.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.691+0000 7f5777c00640 1 Processor -- start 2024-04-25T10:18:40.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.691+0000 7f5777c00640 1 -- start start 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5777c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 0x7f57700d0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5777c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f57700b92a0 0x7f57700d0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5777c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57700b97a0 con 0x7f57700b2cb0 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5777c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577015dfc0 con 0x7f57700b92a0 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5777c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700d1a00 con 0x7f577015dbe0 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f57700b92a0 0x7f57700d0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f57700b92a0 0x7f57700d0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48168/0 (socket says 172.21.15.165:48168) 2024-04-25T10:18:40.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 -- 172.21.15.165:0/2864852487 learned_addr learned my addr 172.21.15.165:0/2864852487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 -- 172.21.15.165:0/2864852487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 msgr2=0x7f57700d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 --2- 172.21.15.165:0/2864852487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 0x7f57700d0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 -- 172.21.15.165:0/2864852487 >> v1:172.21.15.172:6789/0 conn(0x7f57700b2cb0 legacy=0x7f57700b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:40.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.693+0000 7f5775c00640 1 -- 172.21.15.165:0/2864852487 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700d1c50 con 0x7f57700b92a0 2024-04-25T10:18:40.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.695+0000 7f5775c00640 1 --2- 172.21.15.165:0/2864852487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b92a0 0x7f57700d0e50 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f5768009870 tx=0x7f576802a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546f400355eba8b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.695+0000 7f576f400640 1 -- 172.21.15.165:0/2864852487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768037070 con 0x7f57700b92a0 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.695+0000 7f576f400640 1 -- 172.21.15.165:0/2864852487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5768032440 con 0x7f57700b92a0 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.695+0000 7f576f400640 1 -- 172.21.15.165:0/2864852487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5768032810 con 0x7f57700b92a0 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.695+0000 7f5777c00640 1 -- 172.21.15.165:0/2864852487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b92a0 msgr2=0x7f57700d0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.695+0000 7f5777c00640 1 --2- 172.21.15.165:0/2864852487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b92a0 0x7f57700d0e50 secure :-1 s=READY pgs=2333 cs=0 l=1 rev1=1 crypto rx=0x7f5768009870 tx=0x7f576802a200 comp rx=0 tx=0).stop 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.696+0000 7f5777c00640 1 -- 172.21.15.165:0/2864852487 shutdown_connections 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.696+0000 7f5777c00640 1 --2- 172.21.15.165:0/2864852487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b92a0 0x7f57700d0e50 unknown :-1 s=CLOSED pgs=2333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.696+0000 7f5777c00640 1 --2- 172.21.15.165:0/2864852487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 0x7f57700d0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.696+0000 7f5777c00640 1 -- 172.21.15.165:0/2864852487 >> 172.21.15.165:0/2864852487 conn(0x7f5770061a20 msgr2=0x7f5770061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.696+0000 7f5777c00640 1 -- 172.21.15.165:0/2864852487 shutdown_connections 2024-04-25T10:18:40.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.696+0000 7f5777c00640 1 -- 172.21.15.165:0/2864852487 wait complete. 2024-04-25T10:18:40.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.697+0000 7f5777c00640 1 Processor -- start 2024-04-25T10:18:40.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.698+0000 7f5777c00640 1 -- start start 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.698+0000 7f5777c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 0x7f57700cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.698+0000 7f5777c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57700b92a0 0x7f57700d2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5777c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 0x7f57700d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5777c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57700b6ab0 con 0x7f57700b2cb0 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5777c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57700d2210 con 0x7f57700b92a0 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5777c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57700b6c30 con 0x7f577015dbe0 2024-04-25T10:18:40.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 0x7f57700cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 0x7f57700cc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48178/0 (socket says 172.21.15.165:48178) 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 -- 172.21.15.165:0/984526667 learned_addr learned my addr 172.21.15.165:0/984526667 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 -- 172.21.15.165:0/984526667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 msgr2=0x7f57700d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 0x7f57700d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 -- 172.21.15.165:0/984526667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57700b92a0 msgr2=0x7f57700d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57700b92a0 0x7f57700d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 -- 172.21.15.165:0/984526667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57700ccb30 con 0x7f57700b2cb0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.699+0000 7f5775200640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 0x7f57700cc5f0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f576000ba50 tx=0x7f576000bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.700+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760019070 con 0x7f57700b2cb0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.700+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576000cc00 con 0x7f57700b2cb0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.700+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57700b3cf0 con 0x7f57700b2cb0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.700+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5760014440 con 0x7f57700b2cb0 2024-04-25T10:18:40.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.700+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57700b42b0 con 0x7f57700b2cb0 2024-04-25T10:18:40.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.701+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5770073990 con 0x7f57700b2cb0 2024-04-25T10:18:40.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.702+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f576001c070 con 0x7f57700b2cb0 2024-04-25T10:18:40.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.703+0000 7f576cc00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f574c06b4d0 0x7f574c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:40.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.703+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f576008a180 con 0x7f57700b2cb0 2024-04-25T10:18:40.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.703+0000 7f576fe00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f574c06b4d0 0x7f574c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:40.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.704+0000 7f576fe00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f574c06b4d0 0x7f574c06d990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f57700cd370 tx=0x7f5758007990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:40.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.707+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5760058e60 con 0x7f57700b2cb0 2024-04-25T10:18:40.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.941+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57700cdc10 con 0x7f57700b2cb0 2024-04-25T10:18:40.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.943+0000 7f576cc00640 1 -- 172.21.15.165:0/984526667 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f5760058760 con 0x7f57700b2cb0 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:51.816118+0000","modified":"2024-04-25T10:17:54.899844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6742":{"gid":6742,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/1799333237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1799333237},{"type":"v1","addr":"172.21.15.172:6834","nonce":1799333237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:18:40.947 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 113 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f574c06b4d0 msgr2=0x7f574c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f574c06b4d0 0x7f574c06d990 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f57700cd370 tx=0x7f5758007990 comp rx=0 tx=0).stop 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 msgr2=0x7f57700cc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 0x7f57700cc5f0 secure :-1 s=READY pgs=2334 cs=0 l=1 rev1=1 crypto rx=0x7f576000ba50 tx=0x7f576000bf10 comp rx=0 tx=0).stop 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 shutdown_connections 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f574c06b4d0 0x7f574c06d990 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f577015dbe0 0x7f57700d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f57700b92a0 0x7f57700d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 --2- 172.21.15.165:0/984526667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f57700b2cb0 0x7f57700cc5f0 unknown :-1 s=CLOSED pgs=2334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:40.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 >> 172.21.15.165:0/984526667 conn(0x7f5770061a20 msgr2=0x7f5770160820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:40.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 shutdown_connections 2024-04-25T10:18:40.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:40.946+0000 7f5777c00640 1 -- 172.21.15.165:0/984526667 wait complete. 2024-04-25T10:18:40.968 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.160+0000 7fafab200640 1 Processor -- start 2024-04-25T10:18:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.160+0000 7fafab200640 1 -- start start 2024-04-25T10:18:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.162+0000 7fafab200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fafac0b93c0 0x7fafac0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.162+0000 7fafab200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac161df0 con 0x7fafac0d04f0 2024-04-25T10:18:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.162+0000 7fafab200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafac1620d0 con 0x7fafac15db90 2024-04-25T10:18:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.162+0000 7fafab200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafac0d1f50 con 0x7fafac0b93c0 2024-04-25T10:18:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.162+0000 7fafaa800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fafac0d04f0 0x7fafac0d23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55242/0 (socket says 172.21.15.165:55242) 2024-04-25T10:18:41.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.162+0000 7fafaa800640 1 -- 172.21.15.165:0/2706719395 learned_addr learned my addr 172.21.15.165:0/2706719395 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa9400640 1 --2- 172.21.15.165:0/2706719395 >> v2:172.21.15.172:3300/0 conn(0x7fafac0b93c0 0x7fafac0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463662206 0 0) 0x7fafac161df0 con 0x7fafac0d04f0 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf90003520 con 0x7fafac0d04f0 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1748425480 0 0) 0x7faf90003520 con 0x7fafac0d04f0 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 >> v2:172.21.15.172:3300/0 conn(0x7fafac0b93c0 msgr2=0x7fafac0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 --2- 172.21.15.165:0/2706719395 >> v2:172.21.15.172:3300/0 conn(0x7fafac0b93c0 0x7fafac0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 >> v1:172.21.15.172:6789/0 conn(0x7fafac15db90 legacy=0x7fafac1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac1623b0 con 0x7fafac0d04f0 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fafa40019f0 con 0x7fafac0d04f0 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 >> v1:172.21.15.165:6789/0 conn(0x7fafac0d04f0 legacy=0x7fafac0d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90003a00 0x7faf90003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7faf90004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf900051c0 0x7faf9000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0d1f50 con 0x7faf90004800 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9000c0c0 con 0x7faf90003a00 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.163+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf9000c2c0 con 0x7faf900051c0 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9400640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7faf90004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9e00640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90003a00 0x7faf90003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9400640 1 -- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf900051c0 msgr2=0x7faf9000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9400640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faf900051c0 0x7faf9000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9400640 1 -- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90003a00 msgr2=0x7faf90003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9400640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faf90003a00 0x7faf90003e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafa9400640 1 -- 172.21.15.165:0/2706719395 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf90003520 con 0x7faf90004800 2024-04-25T10:18:41.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.164+0000 7fafaa800640 1 -- 172.21.15.165:0/2706719395 reap_dead start 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.165+0000 7fafa9400640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7faf90004c80 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7faf98009ac0 tx=0x7faf90005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ceef16cc29d0040 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.165+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98036070 con 0x7faf90004800 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf98031400 con 0x7faf90004800 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafa8a00640 1 -- 172.21.15.165:0/2706719395 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf98009690 con 0x7faf90004800 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 -- 172.21.15.165:0/2706719395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 msgr2=0x7faf90004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7faf90004c80 secure :-1 s=READY pgs=2336 cs=0 l=1 rev1=1 crypto rx=0x7faf98009ac0 tx=0x7faf90005830 comp rx=0 tx=0).stop 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 -- 172.21.15.165:0/2706719395 shutdown_connections 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 --2- 172.21.15.165:0/2706719395 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7faf90004c80 unknown :-1 s=CLOSED pgs=2336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 -- 172.21.15.165:0/2706719395 >> 172.21.15.165:0/2706719395 conn(0x7fafac0b52b0 msgr2=0x7fafac0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 -- 172.21.15.165:0/2706719395 shutdown_connections 2024-04-25T10:18:41.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.166+0000 7fafab200640 1 -- 172.21.15.165:0/2706719395 wait complete. 2024-04-25T10:18:41.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.168+0000 7fafab200640 1 Processor -- start 2024-04-25T10:18:41.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.168+0000 7fafab200640 1 -- start start 2024-04-25T10:18:41.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafab200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7fafac14d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafab200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac0cbe70 0x7fafac157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafab200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafac0cc2f0 0x7fafac157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafab200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0b68d0 con 0x7faf90004800 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafab200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0d0940 con 0x7fafac0cbe70 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafab200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac0b6a50 con 0x7fafac0cc2f0 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafa9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7fafac14d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafa9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7fafac14d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48208/0 (socket says 172.21.15.165:48208) 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.169+0000 7fafa9e00640 1 -- 172.21.15.165:0/84417090 learned_addr learned my addr 172.21.15.165:0/84417090 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa9e00640 1 -- 172.21.15.165:0/84417090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafac0cc2f0 msgr2=0x7fafac157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa9e00640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafac0cc2f0 0x7fafac157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa9e00640 1 -- 172.21.15.165:0/84417090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac0cbe70 msgr2=0x7fafac157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa9e00640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac0cbe70 0x7fafac157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa9e00640 1 -- 172.21.15.165:0/84417090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0cb560 con 0x7faf90004800 2024-04-25T10:18:41.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa9e00640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7fafac14d2f0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d400 tx=0x7faf9c00d8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c018070 con 0x7faf90004800 2024-04-25T10:18:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf9c003610 con 0x7faf90004800 2024-04-25T10:18:41.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafac14dc90 con 0x7faf90004800 2024-04-25T10:18:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf9c003a40 con 0x7faf90004800 2024-04-25T10:18:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.170+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafac14d830 con 0x7faf90004800 2024-04-25T10:18:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.171+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafac073990 con 0x7faf90004800 2024-04-25T10:18:41.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.172+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faf9c01b070 con 0x7faf90004800 2024-04-25T10:18:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.173+0000 7fafa2000640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf8c06b500 0x7faf8c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.173+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7faf9c089ba0 con 0x7faf90004800 2024-04-25T10:18:41.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.174+0000 7fafa9400640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf8c06b500 0x7faf8c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.174+0000 7fafa9400640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf8c06b500 0x7faf8c06d9c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7faf98033000 tx=0x7faf9802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.178+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faf9c0581a0 con 0x7faf90004800 2024-04-25T10:18:41.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.375+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafac0d14b0 con 0x7faf90004800 2024-04-25T10:18:41.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.378+0000 7fafa2000640 1 -- 172.21.15.165:0/84417090 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16920 (secure 0 0 0) 0x7faf9c05c240 con 0x7faf90004800 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:41.383 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":159,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:53.373638+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:41.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf8c06b500 msgr2=0x7faf8c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf8c06b500 0x7faf8c06d9c0 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7faf98033000 tx=0x7faf9802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 msgr2=0x7fafac14d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7fafac14d2f0 secure :-1 s=READY pgs=2337 cs=0 l=1 rev1=1 crypto rx=0x7faf9c00d400 tx=0x7faf9c00d8c0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 shutdown_connections 2024-04-25T10:18:41.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faf8c06b500 0x7faf8c06d9c0 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fafac0cc2f0 0x7fafac157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fafac0cbe70 0x7fafac157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 --2- 172.21.15.165:0/84417090 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faf90004800 0x7fafac14d2f0 unknown :-1 s=CLOSED pgs=2337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 >> 172.21.15.165:0/84417090 conn(0x7fafac0b52b0 msgr2=0x7fafac158720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 shutdown_connections 2024-04-25T10:18:41.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.381+0000 7fafab200640 1 -- 172.21.15.165:0/84417090 wait complete. 2024-04-25T10:18:41.404 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:17:49.587055+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:17:50.826716+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:41.405 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:41.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.593+0000 7fb551000640 1 Processor -- start 2024-04-25T10:18:41.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.593+0000 7fb551000640 1 -- start start 2024-04-25T10:18:41.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.595+0000 7fb551000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c145140 0x7fb54c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.595+0000 7fb551000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb54c0c5200 0x7fb54c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.595+0000 7fb551000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb54c0c5670 con 0x7fb54c0ce950 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.595+0000 7fb551000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb54c0ced90 con 0x7fb54c0c5200 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.595+0000 7fb551000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c145520 con 0x7fb54c145140 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb54c0c5200 0x7fb54c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb54c0c5200 0x7fb54c15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48222/0 (socket says 172.21.15.165:48222) 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 -- 172.21.15.165:0/231164082 learned_addr learned my addr 172.21.15.165:0/231164082 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54b400640 1 --2- 172.21.15.165:0/231164082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c145140 0x7fb54c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 -- 172.21.15.165:0/231164082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c145140 msgr2=0x7fb54c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 --2- 172.21.15.165:0/231164082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c145140 0x7fb54c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 -- 172.21.15.165:0/231164082 >> v1:172.21.15.172:6789/0 conn(0x7fb54c0ce950 legacy=0x7fb54c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.596+0000 7fb54be00640 1 -- 172.21.15.165:0/231164082 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c071270 con 0x7fb54c0c5200 2024-04-25T10:18:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.597+0000 7fb54be00640 1 --2- 172.21.15.165:0/231164082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0c5200 0x7fb54c15bfb0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb544009e50 tx=0x7fb54402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=632cb9970de7989f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb54aa00640 1 -- 172.21.15.165:0/231164082 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb544036070 con 0x7fb54c0c5200 2024-04-25T10:18:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb54aa00640 1 -- 172.21.15.165:0/231164082 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb544031600 con 0x7fb54c0c5200 2024-04-25T10:18:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb54aa00640 1 -- 172.21.15.165:0/231164082 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5440319d0 con 0x7fb54c0c5200 2024-04-25T10:18:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 -- 172.21.15.165:0/231164082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0c5200 msgr2=0x7fb54c15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 --2- 172.21.15.165:0/231164082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0c5200 0x7fb54c15bfb0 secure :-1 s=READY pgs=2338 cs=0 l=1 rev1=1 crypto rx=0x7fb544009e50 tx=0x7fb54402a630 comp rx=0 tx=0).stop 2024-04-25T10:18:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 -- 172.21.15.165:0/231164082 shutdown_connections 2024-04-25T10:18:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 --2- 172.21.15.165:0/231164082 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0c5200 0x7fb54c15bfb0 unknown :-1 s=CLOSED pgs=2338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 --2- 172.21.15.165:0/231164082 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c145140 0x7fb54c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 -- 172.21.15.165:0/231164082 >> 172.21.15.165:0/231164082 conn(0x7fb54c061cc0 msgr2=0x7fb54c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 -- 172.21.15.165:0/231164082 shutdown_connections 2024-04-25T10:18:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.598+0000 7fb551000640 1 -- 172.21.15.165:0/231164082 wait complete. 2024-04-25T10:18:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.600+0000 7fb551000640 1 Processor -- start 2024-04-25T10:18:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.600+0000 7fb551000640 1 -- start start 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.601+0000 7fb551000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0c5200 0x7fb54c15b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.601+0000 7fb551000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 0x7fb54c1e71e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.601+0000 7fb551000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c145140 0x7fb54c1e7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.601+0000 7fb551000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c071b10 con 0x7fb54c0ce950 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.601+0000 7fb551000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c071830 con 0x7fb54c145140 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.601+0000 7fb551000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb54c15c4f0 con 0x7fb54c0c5200 2024-04-25T10:18:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 0x7fb54c1e71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 0x7fb54c1e71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48238/0 (socket says 172.21.15.165:48238) 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 -- 172.21.15.165:0/2465995066 learned_addr learned my addr 172.21.15.165:0/2465995066 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb54b400640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0c5200 0x7fb54c15b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 -- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0c5200 msgr2=0x7fb54c15b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0c5200 0x7fb54c15b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 -- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c145140 msgr2=0x7fb54c1e7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb54be00640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c145140 0x7fb54c1e7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c145140 0x7fb54c1e7720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.602+0000 7fb543e00640 1 -- 172.21.15.165:0/2465995066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c13d2b0 con 0x7fb54c0ce950 2024-04-25T10:18:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.603+0000 7fb543e00640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 0x7fb54c1e71e0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fb53400c430 tx=0x7fb53400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.603+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534011790 con 0x7fb54c0ce950 2024-04-25T10:18:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.603+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb534017070 con 0x7fb54c0ce950 2024-04-25T10:18:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.603+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb534011bb0 con 0x7fb54c0ce950 2024-04-25T10:18:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.603+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54c140cc0 con 0x7fb54c0ce950 2024-04-25T10:18:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.603+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb54c141280 con 0x7fb54c0ce950 2024-04-25T10:18:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.604+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb54c141480 con 0x7fb54c0ce950 2024-04-25T10:18:41.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.605+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb53401c460 con 0x7fb54c0ce950 2024-04-25T10:18:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.606+0000 7fb543400640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.606+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fb53408a320 con 0x7fb54c0ce950 2024-04-25T10:18:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.606+0000 7fb54b400640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.607+0000 7fb54b400640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52806b4d0 0x7fb52806d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb53c006fd0 tx=0x7fb53c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:41.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.610+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb54c141480 con 0x7fb54c0ce950 2024-04-25T10:18:41.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.804+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb54c073990 con 0x7fb54c0ce950 2024-04-25T10:18:41.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.805+0000 7fb543400640 1 -- 172.21.15.165:0/2465995066 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb534058f70 con 0x7fb54c0ce950 2024-04-25T10:18:41.809 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.807+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52806b4d0 msgr2=0x7fb52806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.807+0000 7fb551000640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52806b4d0 0x7fb52806d990 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fb53c006fd0 tx=0x7fb53c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.807+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 msgr2=0x7fb54c1e71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.807+0000 7fb551000640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 0x7fb54c1e71e0 secure :-1 s=READY pgs=2339 cs=0 l=1 rev1=1 crypto rx=0x7fb53400c430 tx=0x7fb53400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 shutdown_connections 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb52806b4d0 0x7fb52806d990 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb54c145140 0x7fb54c1e7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb54c0ce950 0x7fb54c1e71e0 unknown :-1 s=CLOSED pgs=2339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 --2- 172.21.15.165:0/2465995066 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb54c0c5200 0x7fb54c15b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:41.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 >> 172.21.15.165:0/2465995066 conn(0x7fb54c061cc0 msgr2=0x7fb54c0c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:41.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 shutdown_connections 2024-04-25T10:18:41.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:41.808+0000 7fb551000640 1 -- 172.21.15.165:0/2465995066 wait complete. 2024-04-25T10:18:41.830 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:18:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.026+0000 7fbde3e00640 1 Processor -- start 2024-04-25T10:18:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.026+0000 7fbde3e00640 1 -- start start 2024-04-25T10:18:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.027+0000 7fbde3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbde413d9f0 0x7fbde413df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.027+0000 7fbde3e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde406d390 con 0x7fbde406e860 2024-04-25T10:18:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.027+0000 7fbde3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbde406d670 con 0x7fbde40b0730 2024-04-25T10:18:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.027+0000 7fbde3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbde406d950 con 0x7fbde413d9f0 2024-04-25T10:18:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.027+0000 7fbde3400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbde406e860 0x7fbde40ce880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55256/0 (socket says 172.21.15.165:55256) 2024-04-25T10:18:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.027+0000 7fbde3400640 1 -- 172.21.15.165:0/485758538 learned_addr learned my addr 172.21.15.165:0/485758538 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde2a00640 1 --2- 172.21.15.165:0/485758538 >> v2:172.21.15.172:3300/0 conn(0x7fbde413d9f0 0x7fbde413df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 635048001 0 0) 0x7fbde406d390 con 0x7fbde406e860 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc8003520 con 0x7fbde406e860 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2452688672 0 0) 0x7fbdc8003520 con 0x7fbde406e860 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 >> v2:172.21.15.172:3300/0 conn(0x7fbde413d9f0 msgr2=0x7fbde413df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 --2- 172.21.15.165:0/485758538 >> v2:172.21.15.172:3300/0 conn(0x7fbde413d9f0 0x7fbde413df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 >> v1:172.21.15.172:6789/0 conn(0x7fbde40b0730 legacy=0x7fbde406e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde406dc30 con 0x7fbde406e860 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fbddc0019f0 con 0x7fbde406e860 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 >> v1:172.21.15.165:6789/0 conn(0x7fbde406e860 legacy=0x7fbde40ce880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8003a00 0x7fbdc8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8004800 0x7fbdc8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc80052e0 0x7fbdc800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde406d950 con 0x7fbdc80052e0 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800c1e0 con 0x7fbdc8003a00 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.028+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800c3e0 con 0x7fbdc8004800 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde3400640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc80052e0 0x7fbdc800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde2a00640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8003a00 0x7fbdc8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde3400640 1 -- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8004800 msgr2=0x7fbdc8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde3400640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc8004800 0x7fbdc8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde3400640 1 -- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8003a00 msgr2=0x7fbdc8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde3400640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdc8003a00 0x7fbdc8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.029+0000 7fbde3400640 1 -- 172.21.15.165:0/485758538 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8003520 con 0x7fbdc80052e0 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.030+0000 7fbde3400640 1 -- 172.21.15.165:0/485758538 reap_dead start 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.030+0000 7fbde3400640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc80052e0 0x7fbdc800b9c0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00a370 tx=0x7fbdc8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d83abb8c8f062f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.030+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00a3b0 con 0x7fbdc80052e0 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.030+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc031070 con 0x7fbdc80052e0 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.030+0000 7fbde1600640 1 -- 172.21.15.165:0/485758538 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc02de10 con 0x7fbdc80052e0 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 -- 172.21.15.165:0/485758538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc80052e0 msgr2=0x7fbdc800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc80052e0 0x7fbdc800b9c0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00a370 tx=0x7fbdc8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 -- 172.21.15.165:0/485758538 shutdown_connections 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 --2- 172.21.15.165:0/485758538 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdc80052e0 0x7fbdc800b9c0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 -- 172.21.15.165:0/485758538 >> 172.21.15.165:0/485758538 conn(0x7fbde4061cc0 msgr2=0x7fbde406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 -- 172.21.15.165:0/485758538 shutdown_connections 2024-04-25T10:18:42.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.031+0000 7fbde3e00640 1 -- 172.21.15.165:0/485758538 wait complete. 2024-04-25T10:18:42.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.033+0000 7fbde3e00640 1 Processor -- start 2024-04-25T10:18:42.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.033+0000 7fbde3e00640 1 -- start start 2024-04-25T10:18:42.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc80052e0 0x7fbde41f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde412fae0 0x7fbde41554e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 0x7fbde4157940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b0f50 con 0x7fbde4155a20 2024-04-25T10:18:42.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40b0dd0 con 0x7fbde412fae0 2024-04-25T10:18:42.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbde40ca770 con 0x7fbdc80052e0 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 0x7fbde4157940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 0x7fbde4157940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48266/0 (socket says 172.21.15.165:48266) 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.034+0000 7fbde3400640 1 -- 172.21.15.165:0/2321099890 learned_addr learned my addr 172.21.15.165:0/2321099890 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde2000640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde412fae0 0x7fbde41554e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde2a00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc80052e0 0x7fbde41f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3400640 1 -- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc80052e0 msgr2=0x7fbde41f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3400640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc80052e0 0x7fbde41f3e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3400640 1 -- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde412fae0 msgr2=0x7fbde41554e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3400640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde412fae0 0x7fbde41554e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3400640 1 -- 172.21.15.165:0/2321099890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde40c9e40 con 0x7fbde4155a20 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde2000640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde412fae0 0x7fbde41554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3400640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 0x7fbde4157940 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00a760 tx=0x7fbde40ce7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc03b070 con 0x7fbde4155a20 2024-04-25T10:18:42.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbddc02ddc0 con 0x7fbde4155a20 2024-04-25T10:18:42.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbde40ca120 con 0x7fbde4155a20 2024-04-25T10:18:42.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.035+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbddc00a3b0 con 0x7fbde4155a20 2024-04-25T10:18:42.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.036+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbde412ff60 con 0x7fbde4155a20 2024-04-25T10:18:42.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.037+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbde4073990 con 0x7fbde4155a20 2024-04-25T10:18:42.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.038+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbddc036690 con 0x7fbde4155a20 2024-04-25T10:18:42.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.039+0000 7fbddb400640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdc406b4d0 0x7fbdc406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.039+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fbddc0aa4f0 con 0x7fbde4155a20 2024-04-25T10:18:42.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.039+0000 7fbde2a00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdc406b4d0 0x7fbdc406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.040+0000 7fbde2a00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdc406b4d0 0x7fbdc406d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbde4151e70 tx=0x7fbdd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.043+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbddc0791d0 con 0x7fbde4155a20 2024-04-25T10:18:42.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.237+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbde415ce10 con 0x7fbde4155a20 2024-04-25T10:18:42.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.237+0000 7fbddb400640 1 -- 172.21.15.165:0/2321099890 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v159) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbddc078ad0 con 0x7fbde4155a20 2024-04-25T10:18:42.241 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:18:42.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdc406b4d0 msgr2=0x7fbdc406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdc406b4d0 0x7fbdc406d990 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fbde4151e70 tx=0x7fbdd4008040 comp rx=0 tx=0).stop 2024-04-25T10:18:42.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 msgr2=0x7fbde4157940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 0x7fbde4157940 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fbddc00a760 tx=0x7fbde40ce7a0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 shutdown_connections 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdc406b4d0 0x7fbdc406d990 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbde4155a20 0x7fbde4157940 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbde412fae0 0x7fbde41554e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 --2- 172.21.15.165:0/2321099890 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdc80052e0 0x7fbde41f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 >> 172.21.15.165:0/2321099890 conn(0x7fbde4061cc0 msgr2=0x7fbde415c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 shutdown_connections 2024-04-25T10:18:42.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.240+0000 7fbde3e00640 1 -- 172.21.15.165:0/2321099890 wait complete. 2024-04-25T10:18:42.262 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:18:42.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.478+0000 7f2630600640 1 Processor -- start 2024-04-25T10:18:42.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.478+0000 7f2630600640 1 -- start start 2024-04-25T10:18:42.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.479+0000 7f2630600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26280b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.479+0000 7f2630600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2628144d50 0x7f26280b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.479+0000 7f2630600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26280cccc0 con 0x7f26281452a0 2024-04-25T10:18:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.479+0000 7f2630600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26280b2b20 con 0x7f2628144d50 2024-04-25T10:18:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.479+0000 7f2630600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262806d400 con 0x7f26281446d0 2024-04-25T10:18:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.480+0000 7f262e600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f26281452a0 0x7f26280cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55266/0 (socket says 172.21.15.165:55266) 2024-04-25T10:18:42.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.480+0000 7f262e600640 1 -- 172.21.15.165:0/3531104763 learned_addr learned my addr 172.21.15.165:0/3531104763 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.480+0000 7f262dc00640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26280b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.480+0000 7f262d200640 1 --2- 172.21.15.165:0/3531104763 >> v2:172.21.15.172:3300/0 conn(0x7f2628144d50 0x7f26280b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.480+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2215830792 0 0) 0x7f26280cccc0 con 0x7f26281452a0 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.480+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2610003520 con 0x7f26281452a0 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2592999323 0 0) 0x7f2610003520 con 0x7f26281452a0 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 msgr2=0x7f26280b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26280b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 >> v2:172.21.15.172:3300/0 conn(0x7f2628144d50 msgr2=0x7f26280b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 --2- 172.21.15.165:0/3531104763 >> v2:172.21.15.172:3300/0 conn(0x7f2628144d50 0x7f26280b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26280cc140 con 0x7f26281452a0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f26200019f0 con 0x7f26281452a0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 >> v1:172.21.15.165:6789/0 conn(0x7f26281452a0 legacy=0x7f26280cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26100039b0 0x7f2610003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2610004800 0x7f2610004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26100051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280b2b20 con 0x7f2610004800 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262806d400 con 0x7f26100039b0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f261000fdd0 con 0x7f26281446d0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.481+0000 7f262d200640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2610004800 0x7f2610004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262dc00640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26100039b0 0x7f2610003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262e600640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26100051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262d200640 1 -- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 msgr2=0x7f26100051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262d200640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26100051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262d200640 1 -- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26100039b0 msgr2=0x7f2610003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262d200640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f26100039b0 0x7f2610003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262d200640 1 -- 172.21.15.165:0/3531104763 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2610003520 con 0x7f2610004800 2024-04-25T10:18:42.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.482+0000 7f262e600640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f26281446d0 0x7f26100051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.483+0000 7f262d200640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2610004800 0x7f2610004c80 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f2614009ac0 tx=0x7f2610005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=410c78aaaf5b6a1b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.483+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614036070 con 0x7f2610004800 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.483+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614031400 con 0x7f2610004800 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.483+0000 7f2627e00640 1 -- 172.21.15.165:0/3531104763 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614009690 con 0x7f2610004800 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f2630600640 1 -- 172.21.15.165:0/3531104763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2610004800 msgr2=0x7f2610004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f2630600640 1 --2- 172.21.15.165:0/3531104763 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2610004800 0x7f2610004c80 secure :-1 s=READY pgs=2344 cs=0 l=1 rev1=1 crypto rx=0x7f2614009ac0 tx=0x7f2610005830 comp rx=0 tx=0).stop 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f262e600640 1 -- 172.21.15.165:0/3531104763 reap_dead start 2024-04-25T10:18:42.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f2630600640 1 -- 172.21.15.165:0/3531104763 shutdown_connections 2024-04-25T10:18:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f2630600640 1 -- 172.21.15.165:0/3531104763 >> 172.21.15.165:0/3531104763 conn(0x7f262806abe0 msgr2=0x7f262806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f2630600640 1 -- 172.21.15.165:0/3531104763 shutdown_connections 2024-04-25T10:18:42.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.484+0000 7f2630600640 1 -- 172.21.15.165:0/3531104763 wait complete. 2024-04-25T10:18:42.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.486+0000 7f2630600640 1 Processor -- start 2024-04-25T10:18:42.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.486+0000 7f2630600640 1 -- start start 2024-04-25T10:18:42.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f2630600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2610004800 0x7f262813c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f2630600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 0x7f26280a8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f2630600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f262813cf70 0x7f262813d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f2630600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280cc700 con 0x7f26280a84a0 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f2630600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26280b49a0 con 0x7f262813cf70 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f2630600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26280cc9e0 con 0x7f2610004800 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 0x7f26280a8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 0x7f26280a8920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48274/0 (socket says 172.21.15.165:48274) 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 -- 172.21.15.165:0/3901873886 learned_addr learned my addr 172.21.15.165:0/3901873886 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 -- 172.21.15.165:0/3901873886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2610004800 msgr2=0x7f262813c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2610004800 0x7f262813c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 -- 172.21.15.165:0/3901873886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f262813cf70 msgr2=0x7f262813d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f262813cf70 0x7f262813d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.487+0000 7f262d200640 1 -- 172.21.15.165:0/3901873886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262813db50 con 0x7f26280a84a0 2024-04-25T10:18:42.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.488+0000 7f262d200640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 0x7f26280a8920 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f2614009be0 tx=0x7f2614004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.488+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614040020 con 0x7f26280a84a0 2024-04-25T10:18:42.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.488+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2614003950 con 0x7f26280a84a0 2024-04-25T10:18:42.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.488+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2614036040 con 0x7f26280a84a0 2024-04-25T10:18:42.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.488+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262813de30 con 0x7f26280a84a0 2024-04-25T10:18:42.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.488+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262813e3f0 con 0x7f26280a84a0 2024-04-25T10:18:42.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.490+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2628073a00 con 0x7f26280a84a0 2024-04-25T10:18:42.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.491+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2614043020 con 0x7f26280a84a0 2024-04-25T10:18:42.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.492+0000 7f2625600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f260806b500 0x7f260806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.492+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f26140aa5c0 con 0x7f26280a84a0 2024-04-25T10:18:42.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.492+0000 7f262dc00640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f260806b500 0x7f260806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.493+0000 7f262dc00640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f260806b500 0x7f260806d9c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f26281440e0 tx=0x7f261c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.496+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f261404a070 con 0x7f26280a84a0 2024-04-25T10:18:42.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.691+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2628141c60 con 0x7f26280a84a0 2024-04-25T10:18:42.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.692+0000 7f2625600640 1 -- 172.21.15.165:0/3901873886 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v159) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2614079210 con 0x7f26280a84a0 2024-04-25T10:18:42.696 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.694+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f260806b500 msgr2=0x7f260806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.694+0000 7f2630600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f260806b500 0x7f260806d9c0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f26281440e0 tx=0x7f261c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.694+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 msgr2=0x7f26280a8920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.694+0000 7f2630600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 0x7f26280a8920 secure :-1 s=READY pgs=2345 cs=0 l=1 rev1=1 crypto rx=0x7f2614009be0 tx=0x7f2614004730 comp rx=0 tx=0).stop 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 shutdown_connections 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f260806b500 0x7f260806d9c0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f262813cf70 0x7f262813d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f26280a84a0 0x7f26280a8920 unknown :-1 s=CLOSED pgs=2345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 --2- 172.21.15.165:0/3901873886 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2610004800 0x7f262813c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 >> 172.21.15.165:0/3901873886 conn(0x7f262806abe0 msgr2=0x7f26280b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 shutdown_connections 2024-04-25T10:18:42.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.695+0000 7f2630600640 1 -- 172.21.15.165:0/3901873886 wait complete. 2024-04-25T10:18:42.718 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:42.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.919+0000 7f8d19400640 1 Processor -- start 2024-04-25T10:18:42.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.919+0000 7f8d19400640 1 -- start start 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.920+0000 7f8d19400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d141594f0 0x7f8d140b50f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.920+0000 7f8d19400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8d1415c3d0 0x7f8d140b5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.920+0000 7f8d19400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d140b5630 con 0x7f8d1415bf50 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.920+0000 7f8d19400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d141598d0 con 0x7f8d1415c3d0 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.920+0000 7f8d19400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140b4c60 con 0x7f8d141594f0 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8d1415c3d0 0x7f8d140b5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8d1415c3d0 0x7f8d140b5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48288/0 (socket says 172.21.15.165:48288) 2024-04-25T10:18:42.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d11e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8d1415bf50 0x7f8d140d0850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55272/0 (socket says 172.21.15.165:55272) 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 -- 172.21.15.165:0/2357960308 learned_addr learned my addr 172.21.15.165:0/2357960308 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d12800640 1 --2- 172.21.15.165:0/2357960308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d141594f0 0x7f8d140b50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 -- 172.21.15.165:0/2357960308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d141594f0 msgr2=0x7f8d140b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 --2- 172.21.15.165:0/2357960308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d141594f0 0x7f8d140b50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 -- 172.21.15.165:0/2357960308 >> v1:172.21.15.165:6789/0 conn(0x7f8d1415bf50 legacy=0x7f8d140d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d11400640 1 -- 172.21.15.165:0/2357960308 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2480702479 0 0) 0x7f8d140b5630 con 0x7f8d1415bf50 2024-04-25T10:18:42.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.921+0000 7f8d13200640 1 -- 172.21.15.165:0/2357960308 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d1415c930 con 0x7f8d1415c3d0 2024-04-25T10:18:42.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.922+0000 7f8d13200640 1 --2- 172.21.15.165:0/2357960308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1415c3d0 0x7f8d140b5a70 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009870 tx=0x7f8d0c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=398a213950eec571 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d11400640 1 -- 172.21.15.165:0/2357960308 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c037070 con 0x7f8d1415c3d0 2024-04-25T10:18:42.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d11400640 1 -- 172.21.15.165:0/2357960308 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d0c032440 con 0x7f8d1415c3d0 2024-04-25T10:18:42.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d11400640 1 -- 172.21.15.165:0/2357960308 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c032810 con 0x7f8d1415c3d0 2024-04-25T10:18:42.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 -- 172.21.15.165:0/2357960308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1415c3d0 msgr2=0x7f8d140b5a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 --2- 172.21.15.165:0/2357960308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1415c3d0 0x7f8d140b5a70 secure :-1 s=READY pgs=2347 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009870 tx=0x7f8d0c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 -- 172.21.15.165:0/2357960308 shutdown_connections 2024-04-25T10:18:42.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 --2- 172.21.15.165:0/2357960308 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d1415c3d0 0x7f8d140b5a70 unknown :-1 s=CLOSED pgs=2347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 --2- 172.21.15.165:0/2357960308 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d141594f0 0x7f8d140b50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 -- 172.21.15.165:0/2357960308 >> 172.21.15.165:0/2357960308 conn(0x7f8d1406f520 msgr2=0x7f8d1406f900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:42.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 -- 172.21.15.165:0/2357960308 shutdown_connections 2024-04-25T10:18:42.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.923+0000 7f8d19400640 1 -- 172.21.15.165:0/2357960308 wait complete. 2024-04-25T10:18:42.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.925+0000 7f8d19400640 1 Processor -- start 2024-04-25T10:18:42.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.925+0000 7f8d19400640 1 -- start start 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.926+0000 7f8d19400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 0x7f8d14157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.926+0000 7f8d19400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d1415bf50 0x7f8d141f3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.926+0000 7f8d19400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d1415c3d0 0x7f8d140c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.926+0000 7f8d19400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d14070e10 con 0x7f8d141594f0 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.926+0000 7f8d19400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140b48f0 con 0x7f8d1415bf50 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.926+0000 7f8d19400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d14070f90 con 0x7f8d1415c3d0 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 0x7f8d14157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 0x7f8d14157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48298/0 (socket says 172.21.15.165:48298) 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 -- 172.21.15.165:0/1537185670 learned_addr learned my addr 172.21.15.165:0/1537185670 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d13200640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d1415c3d0 0x7f8d140c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d11e00640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d1415bf50 0x7f8d141f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 -- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d1415c3d0 msgr2=0x7f8d140c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d1415c3d0 0x7f8d140c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 -- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d1415bf50 msgr2=0x7f8d141f3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d1415bf50 0x7f8d141f3770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 -- 172.21.15.165:0/1537185670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d140adc10 con 0x7f8d141594f0 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d13200640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d1415c3d0 0x7f8d140c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d11e00640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d1415bf50 0x7f8d141f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:42.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.927+0000 7f8d12800640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 0x7f8d14157890 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400d620 tx=0x7f8d0400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.928+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400b620 con 0x7f8d141594f0 2024-04-25T10:18:42.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.928+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d04019070 con 0x7f8d141594f0 2024-04-25T10:18:42.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.928+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400ba00 con 0x7f8d141594f0 2024-04-25T10:18:42.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.928+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d140adef0 con 0x7f8d141594f0 2024-04-25T10:18:42.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.928+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d140ae4b0 con 0x7f8d141594f0 2024-04-25T10:18:42.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.929+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d14073990 con 0x7f8d141594f0 2024-04-25T10:18:42.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.930+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8d0400bba0 con 0x7f8d141594f0 2024-04-25T10:18:42.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.931+0000 7f8d0aa00640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cf406b4d0 0x7f8cf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:42.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.931+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8d0408a8a0 con 0x7f8d141594f0 2024-04-25T10:18:42.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.931+0000 7f8d11e00640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cf406b4d0 0x7f8cf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:42.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.932+0000 7f8d11e00640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cf406b4d0 0x7f8cf406d990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8d141f6c90 tx=0x7f8d00009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:42.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:42.935+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8d040594f0 con 0x7f8d141594f0 2024-04-25T10:18:43.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.177+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8d1415ec50 con 0x7f8d141594f0 2024-04-25T10:18:43.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.178+0000 7f8d0aa00640 1 -- 172.21.15.165:0/1537185670 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f8d04058df0 con 0x7f8d141594f0 2024-04-25T10:18:43.182 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:43.182 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:51.816118+0000","modified":"2024-04-25T10:17:54.899844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6742":{"gid":6742,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/1799333237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1799333237},{"type":"v1","addr":"172.21.15.172:6834","nonce":1799333237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:18:43.182 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 113 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cf406b4d0 msgr2=0x7f8cf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cf406b4d0 0x7f8cf406d990 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f8d141f6c90 tx=0x7f8d00009040 comp rx=0 tx=0).stop 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 msgr2=0x7f8d14157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 0x7f8d14157890 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400d620 tx=0x7f8d0400dae0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 shutdown_connections 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8cf406b4d0 0x7f8cf406d990 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8d1415c3d0 0x7f8d140c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8d1415bf50 0x7f8d141f3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 --2- 172.21.15.165:0/1537185670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8d141594f0 0x7f8d14157890 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 >> 172.21.15.165:0/1537185670 conn(0x7f8d1406f520 msgr2=0x7f8d140b50f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 shutdown_connections 2024-04-25T10:18:43.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.181+0000 7f8d19400640 1 -- 172.21.15.165:0/1537185670 wait complete. 2024-04-25T10:18:43.203 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.395+0000 7fc65c800640 1 Processor -- start 2024-04-25T10:18:43.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.395+0000 7fc65c800640 1 -- start start 2024-04-25T10:18:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.396+0000 7fc65c800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6540ce920 0x7fc6540cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.396+0000 7fc65c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc65406d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.396+0000 7fc65c800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc654144b30 con 0x7fc654144750 2024-04-25T10:18:43.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.396+0000 7fc65c800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6540712d0 con 0x7fc6540ce920 2024-04-25T10:18:43.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.396+0000 7fc65c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6540715b0 con 0x7fc65406d290 2024-04-25T10:18:43.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6540ce920 0x7fc6540cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc6540ce920 0x7fc6540cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48314/0 (socket says 172.21.15.165:48314) 2024-04-25T10:18:43.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 -- 172.21.15.165:0/2448830831 learned_addr learned my addr 172.21.15.165:0/2448830831 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc65a800640 1 --2- 172.21.15.165:0/2448830831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc65406d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 -- 172.21.15.165:0/2448830831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 msgr2=0x7fc65406d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 --2- 172.21.15.165:0/2448830831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc65406d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 -- 172.21.15.165:0/2448830831 >> v1:172.21.15.172:6789/0 conn(0x7fc654144750 legacy=0x7fc65406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.397+0000 7fc659400640 1 -- 172.21.15.165:0/2448830831 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc654071890 con 0x7fc6540ce920 2024-04-25T10:18:43.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.398+0000 7fc659400640 1 --2- 172.21.15.165:0/2448830831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc6540cedb0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc64402a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e358b3d21bac2c0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc658a00640 1 -- 172.21.15.165:0/2448830831 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644037070 con 0x7fc6540ce920 2024-04-25T10:18:43.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc658a00640 1 -- 172.21.15.165:0/2448830831 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644032440 con 0x7fc6540ce920 2024-04-25T10:18:43.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc658a00640 1 -- 172.21.15.165:0/2448830831 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644032810 con 0x7fc6540ce920 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 -- 172.21.15.165:0/2448830831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 msgr2=0x7fc6540cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 --2- 172.21.15.165:0/2448830831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc6540cedb0 secure :-1 s=READY pgs=2349 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc64402a210 comp rx=0 tx=0).stop 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 -- 172.21.15.165:0/2448830831 shutdown_connections 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 --2- 172.21.15.165:0/2448830831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc65406d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 --2- 172.21.15.165:0/2448830831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc6540cedb0 unknown :-1 s=CLOSED pgs=2349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 -- 172.21.15.165:0/2448830831 >> 172.21.15.165:0/2448830831 conn(0x7fc65406abe0 msgr2=0x7fc65406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 -- 172.21.15.165:0/2448830831 shutdown_connections 2024-04-25T10:18:43.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.399+0000 7fc65c800640 1 -- 172.21.15.165:0/2448830831 wait complete. 2024-04-25T10:18:43.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.401+0000 7fc65c800640 1 Processor -- start 2024-04-25T10:18:43.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.401+0000 7fc65c800640 1 -- start start 2024-04-25T10:18:43.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc65c800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc6540afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc65c800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc65413fa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc65c800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc654144750 0x7fc654135dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc65c800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65406f020 con 0x7fc6540ce920 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc65c800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65406ece0 con 0x7fc654144750 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc65c800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc654145450 con 0x7fc65406d290 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc659400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc65413fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc659400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc65413fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48318/0 (socket says 172.21.15.165:48318) 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.402+0000 7fc659400640 1 -- 172.21.15.165:0/1127974451 learned_addr learned my addr 172.21.15.165:0/1127974451 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659e00640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc6540afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659400640 1 -- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 msgr2=0x7fc6540afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc65a800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc654144750 0x7fc654135dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659400640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc6540afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659400640 1 -- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc654144750 msgr2=0x7fc654135dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659400640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc654144750 0x7fc654135dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659400640 1 -- 172.21.15.165:0/1127974451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65415c000 con 0x7fc6540ce920 2024-04-25T10:18:43.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659e00640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc6540afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc659400640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc65413fa80 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc6440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644041020 con 0x7fc6540ce920 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644030970 con 0x7fc6540ce920 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc644037040 con 0x7fc6540ce920 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.403+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65415c2e0 con 0x7fc6540ce920 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.404+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65415c8a0 con 0x7fc6540ce920 2024-04-25T10:18:43.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.405+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc654073990 con 0x7fc6540ce920 2024-04-25T10:18:43.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.406+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc644044020 con 0x7fc6540ce920 2024-04-25T10:18:43.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.407+0000 7fc64e000640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.407+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc6440aae30 con 0x7fc6540ce920 2024-04-25T10:18:43.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.407+0000 7fc659e00640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.408+0000 7fc659e00640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc6541601a0 tx=0x7fc648009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.411+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc64404b030 con 0x7fc6540ce920 2024-04-25T10:18:43.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.606+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc6540cedf0 con 0x7fc6540ce920 2024-04-25T10:18:43.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.608+0000 7fc64e000640 1 -- 172.21.15.165:0/1127974451 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16920 (secure 0 0 0) 0x7fc644079a80 con 0x7fc6540ce920 2024-04-25T10:18:43.612 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:43.612 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":159,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:53.373638+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc63806b4d0 msgr2=0x7fc63806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc63806b4d0 0x7fc63806d990 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fc6541601a0 tx=0x7fc648009040 comp rx=0 tx=0).stop 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 msgr2=0x7fc65413fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc65413fa80 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fc644009870 tx=0x7fc6440046c0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 shutdown_connections 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc63806b4d0 0x7fc63806d990 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc654144750 0x7fc654135dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc6540ce920 0x7fc65413fa80 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 --2- 172.21.15.165:0/1127974451 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc65406d290 0x7fc6540afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 >> 172.21.15.165:0/1127974451 conn(0x7fc65406abe0 msgr2=0x7fc65406e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 shutdown_connections 2024-04-25T10:18:43.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.611+0000 7fc65c800640 1 -- 172.21.15.165:0/1127974451 wait complete. 2024-04-25T10:18:43.634 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:18:43.634 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.822+0000 7fa356c00640 1 Processor -- start 2024-04-25T10:18:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.823+0000 7fa356c00640 1 -- start start 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa356c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa3500cedf0 0x7fa3501616b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa356c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 0x7fa3500d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa356c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa35015e000 con 0x7fa35015dc20 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa356c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3500b6bb0 con 0x7fa3500cedf0 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa356c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500b6030 con 0x7fa3500b93a0 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa34fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa35015dc20 0x7fa3500ce6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55276/0 (socket says 172.21.15.165:55276) 2024-04-25T10:18:43.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.824+0000 7fa34fe00640 1 -- 172.21.15.165:0/3178076877 learned_addr learned my addr 172.21.15.165:0/3178076877 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34f400640 1 --2- 172.21.15.165:0/3178076877 >> v2:172.21.15.172:3300/0 conn(0x7fa3500cedf0 0x7fa3501616b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa354c00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 0x7fa3500d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601150725 0 0) 0x7fa338001110 con 0x7fa35015dc20 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa334003520 con 0x7fa35015dc20 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1905298440 0 0) 0x7fa334003520 con 0x7fa35015dc20 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 msgr2=0x7fa3500d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 0x7fa3500d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 >> v2:172.21.15.172:3300/0 conn(0x7fa3500cedf0 msgr2=0x7fa3501616b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 --2- 172.21.15.165:0/3178076877 >> v2:172.21.15.172:3300/0 conn(0x7fa3500cedf0 0x7fa3501616b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa350161de0 con 0x7fa35015dc20 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa338001840 con 0x7fa35015dc20 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 >> v1:172.21.15.165:6789/0 conn(0x7fa35015dc20 legacy=0x7fa3500ce6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 0x7fa3340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334003f80 0x7fa334004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa334004d40 0x7fa3340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500b6bb0 con 0x7fa334004d40 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500b6030 con 0x7fa334003f80 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.825+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa33400b9c0 con 0x7fa3500b93a0 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa354c00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa334004d40 0x7fa3340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa34f400640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334003f80 0x7fa334004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa34fe00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 0x7fa3340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa354c00640 1 -- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 msgr2=0x7fa3340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa354c00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa3500b93a0 0x7fa3340039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa354c00640 1 -- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334003f80 msgr2=0x7fa334004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa354c00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334003f80 0x7fa334004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa354c00640 1 -- 172.21.15.165:0/3178076877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334003520 con 0x7fa334004d40 2024-04-25T10:18:43.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.826+0000 7fa34f400640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334003f80 0x7fa334004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.827+0000 7fa354c00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa334004d40 0x7fa3340051c0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa348009af0 tx=0x7fa334005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b58de9e919ba9fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348036070 con 0x7fa334004d40 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa348031400 con 0x7fa334004d40 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa34ea00640 1 -- 172.21.15.165:0/3178076877 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa348009690 con 0x7fa334004d40 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa356c00640 1 -- 172.21.15.165:0/3178076877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa334004d40 msgr2=0x7fa3340051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa356c00640 1 --2- 172.21.15.165:0/3178076877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa334004d40 0x7fa3340051c0 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7fa348009af0 tx=0x7fa334005830 comp rx=0 tx=0).stop 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa354c00640 1 -- 172.21.15.165:0/3178076877 reap_dead start 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa356c00640 1 -- 172.21.15.165:0/3178076877 shutdown_connections 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa356c00640 1 -- 172.21.15.165:0/3178076877 >> 172.21.15.165:0/3178076877 conn(0x7fa3500d0490 msgr2=0x7fa3500d08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.828+0000 7fa356c00640 1 -- 172.21.15.165:0/3178076877 shutdown_connections 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.829+0000 7fa356c00640 1 -- 172.21.15.165:0/3178076877 wait complete. 2024-04-25T10:18:43.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.830+0000 7fa356c00640 1 Processor -- start 2024-04-25T10:18:43.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.830+0000 7fa356c00640 1 -- start start 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.831+0000 7fa356c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334004d40 0x7fa3501f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.831+0000 7fa356c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 0x7fa3500ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.831+0000 7fa356c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa35014b990 0x7fa35014bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.831+0000 7fa356c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500d2510 con 0x7fa3500caae0 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.831+0000 7fa356c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3501623a0 con 0x7fa334004d40 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.831+0000 7fa356c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3500d2690 con 0x7fa35014b990 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 0x7fa3500ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 0x7fa3500ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48332/0 (socket says 172.21.15.165:48332) 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 -- 172.21.15.165:0/155774278 learned_addr learned my addr 172.21.15.165:0/155774278 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34fe00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334004d40 0x7fa3501f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 -- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa35014b990 msgr2=0x7fa35014bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa35014b990 0x7fa35014bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 -- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334004d40 msgr2=0x7fa3501f7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334004d40 0x7fa3501f7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 -- 172.21.15.165:0/155774278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3500cb2f0 con 0x7fa3500caae0 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34fe00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334004d40 0x7fa3501f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.832+0000 7fa34f400640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 0x7fa3500ccba0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa340009fa0 tx=0x7fa340010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.833+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa34001a030 con 0x7fa3500caae0 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.833+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa340003990 con 0x7fa3500caae0 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.833+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3500cb5d0 con 0x7fa3500caae0 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.833+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa340003d60 con 0x7fa3500caae0 2024-04-25T10:18:43.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.833+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3500cbb90 con 0x7fa3500caae0 2024-04-25T10:18:43.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.834+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa350073990 con 0x7fa3500caae0 2024-04-25T10:18:43.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.835+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa34001d070 con 0x7fa3500caae0 2024-04-25T10:18:43.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.836+0000 7fa347e00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa32c06b4d0 0x7fa32c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:43.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.836+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa34008b290 con 0x7fa3500caae0 2024-04-25T10:18:43.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.836+0000 7fa34fe00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa32c06b4d0 0x7fa32c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:43.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.837+0000 7fa34fe00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa32c06b4d0 0x7fa32c06d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa3380019b0 tx=0x7fa338000f50 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:43.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:43.840+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa340059ee0 con 0x7fa3500caae0 2024-04-25T10:18:44.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.072+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3500d1720 con 0x7fa3500caae0 2024-04-25T10:18:44.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.073+0000 7fa347e00640 1 -- 172.21.15.165:0/155774278 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4960 (secure 0 0 0) 0x7fa3400597e0 con 0x7fa3500caae0 2024-04-25T10:18:44.077 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:44.077 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:51.816118+0000","modified":"2024-04-25T10:17:54.899844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6742":{"gid":6742,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/1799333237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1799333237},{"type":"v1","addr":"172.21.15.172:6834","nonce":1799333237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:18:44.077 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 113 2024-04-25T10:18:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa32c06b4d0 msgr2=0x7fa32c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa32c06b4d0 0x7fa32c06d990 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fa3380019b0 tx=0x7fa338000f50 comp rx=0 tx=0).stop 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 msgr2=0x7fa3500ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 0x7fa3500ccba0 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7fa340009fa0 tx=0x7fa340010590 comp rx=0 tx=0).stop 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 shutdown_connections 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa32c06b4d0 0x7fa32c06d990 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa35014b990 0x7fa35014bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa3500caae0 0x7fa3500ccba0 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 --2- 172.21.15.165:0/155774278 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa334004d40 0x7fa3501f7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 >> 172.21.15.165:0/155774278 conn(0x7fa3500d0490 msgr2=0x7fa3500b8580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 shutdown_connections 2024-04-25T10:18:44.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.076+0000 7fa356c00640 1 -- 172.21.15.165:0/155774278 wait complete. 2024-04-25T10:18:44.099 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:44.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.288+0000 7f19fb000640 1 Processor -- start 2024-04-25T10:18:44.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.288+0000 7f19fb000640 1 -- start start 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19fb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19fb000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f19f40d1020 0x7f19f40d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19fb000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40b97a0 con 0x7f19f415dbe0 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19fb000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f415dfc0 con 0x7f19f40d1020 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19fb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40d1a00 con 0x7f19f40b2cb0 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19f9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f19f40d1020 0x7f19f40d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19f9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f19f40d1020 0x7f19f40d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48340/0 (socket says 172.21.15.165:48340) 2024-04-25T10:18:44.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19f9000640 1 -- 172.21.15.165:0/3095773506 learned_addr learned my addr 172.21.15.165:0/3095773506 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:44.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.290+0000 7f19f3400640 1 --2- 172.21.15.165:0/3095773506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.291+0000 7f19f9000640 1 -- 172.21.15.165:0/3095773506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 msgr2=0x7f19f40d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.291+0000 7f19f9000640 1 --2- 172.21.15.165:0/3095773506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.291+0000 7f19f9000640 1 -- 172.21.15.165:0/3095773506 >> v1:172.21.15.172:6789/0 conn(0x7f19f415dbe0 legacy=0x7f19f40d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.291+0000 7f19f9000640 1 -- 172.21.15.165:0/3095773506 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40d1c50 con 0x7f19f40d1020 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.292+0000 7f19f9000640 1 --2- 172.21.15.165:0/3095773506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f40d2c70 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009870 tx=0x7f19ec02a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae42d9bc94c125ed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.292+0000 7f19f2a00640 1 -- 172.21.15.165:0/3095773506 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec037070 con 0x7f19f40d1020 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19f2a00640 1 -- 172.21.15.165:0/3095773506 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec032440 con 0x7f19f40d1020 2024-04-25T10:18:44.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19f2a00640 1 -- 172.21.15.165:0/3095773506 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec032810 con 0x7f19f40d1020 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 -- 172.21.15.165:0/3095773506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 msgr2=0x7f19f40d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 --2- 172.21.15.165:0/3095773506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f40d2c70 secure :-1 s=READY pgs=2354 cs=0 l=1 rev1=1 crypto rx=0x7f19ec009870 tx=0x7f19ec02a210 comp rx=0 tx=0).stop 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 -- 172.21.15.165:0/3095773506 shutdown_connections 2024-04-25T10:18:44.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 --2- 172.21.15.165:0/3095773506 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f40d2c70 unknown :-1 s=CLOSED pgs=2354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 --2- 172.21.15.165:0/3095773506 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 -- 172.21.15.165:0/3095773506 >> 172.21.15.165:0/3095773506 conn(0x7f19f4061a20 msgr2=0x7f19f4061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 -- 172.21.15.165:0/3095773506 shutdown_connections 2024-04-25T10:18:44.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.293+0000 7f19fb000640 1 -- 172.21.15.165:0/3095773506 wait complete. 2024-04-25T10:18:44.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.295+0000 7f19fb000640 1 Processor -- start 2024-04-25T10:18:44.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.295+0000 7f19fb000640 1 -- start start 2024-04-25T10:18:44.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19fb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40b58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19fb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f4153bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19fb000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f415dbe0 0x7f19f41f7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19fb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40b92a0 con 0x7f19f40d1020 2024-04-25T10:18:44.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19fb000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40d2210 con 0x7f19f415dbe0 2024-04-25T10:18:44.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19fb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40b9420 con 0x7f19f40b2cb0 2024-04-25T10:18:44.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19f3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f4153bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19f3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f4153bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48354/0 (socket says 172.21.15.165:48354) 2024-04-25T10:18:44.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.296+0000 7f19f3400640 1 -- 172.21.15.165:0/2003101857 learned_addr learned my addr 172.21.15.165:0/2003101857 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:44.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19f3400640 1 -- 172.21.15.165:0/2003101857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 msgr2=0x7f19f40b58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19f3400640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40b58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19f3400640 1 -- 172.21.15.165:0/2003101857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f415dbe0 msgr2=0x7f19f41f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19f3400640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f415dbe0 0x7f19f41f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19f3400640 1 -- 172.21.15.165:0/2003101857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f40cbf90 con 0x7f19f40d1020 2024-04-25T10:18:44.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19f3400640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f4153bf0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d800 tx=0x7f19dc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00b6c0 con 0x7f19f40d1020 2024-04-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19dc019070 con 0x7f19f40d1020 2024-04-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.297+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40cb9d0 con 0x7f19f40d1020 2024-04-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.298+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19dc00ba80 con 0x7f19f40d1020 2024-04-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.298+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40b31c0 con 0x7f19f40d1020 2024-04-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.299+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f4073990 con 0x7f19f40d1020 2024-04-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.300+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f19dc00bc20 con 0x7f19f40d1020 2024-04-25T10:18:44.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.301+0000 7f19ebe00640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19d006b500 0x7f19d006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.301+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f19dc08a760 con 0x7f19f40d1020 2024-04-25T10:18:44.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.301+0000 7f19f3e00640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19d006b500 0x7f19d006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.302+0000 7f19f3e00640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19d006b500 0x7f19d006d9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ab90 tx=0x7f19e4009230 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.305+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f19dc01c360 con 0x7f19f40d1020 2024-04-25T10:18:44.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.539+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f19f415b790 con 0x7f19f40d1020 2024-04-25T10:18:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.540+0000 7f19ebe00640 1 -- 172.21.15.165:0/2003101857 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 113 v113) v1 ==== 95+0+4960 (secure 0 0 0) 0x7f19dc0594e0 con 0x7f19f40d1020 2024-04-25T10:18:44.544 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:44.544 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":113,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":113,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:17:51.816118+0000","modified":"2024-04-25T10:17:54.899844+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":6742},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6742":{"gid":6742,"name":"d","rank":0,"incarnation":109,"state":"up:active","state_seq":3,"addr":"172.21.15.172:6834/1799333237","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1799333237},{"type":"v1","addr":"172.21.15.172:6834","nonce":1799333237}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2024-04-25T10:18:44.544 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 113 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.542+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19d006b500 msgr2=0x7f19d006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.542+0000 7f19fb000640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19d006b500 0x7f19d006d9c0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f19e400ab90 tx=0x7f19e4009230 comp rx=0 tx=0).stop 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.542+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 msgr2=0x7f19f4153bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.542+0000 7f19fb000640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f4153bf0 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f19dc00d800 tx=0x7f19dc00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 shutdown_connections 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f19d006b500 0x7f19d006d9c0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f19f415dbe0 0x7f19f41f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f19f40d1020 0x7f19f4153bf0 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 --2- 172.21.15.165:0/2003101857 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f19f40b2cb0 0x7f19f40b58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 >> 172.21.15.165:0/2003101857 conn(0x7f19f4061a20 msgr2=0x7f19f40b8120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 shutdown_connections 2024-04-25T10:18:44.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.543+0000 7f19fb000640 1 -- 172.21.15.165:0/2003101857 wait complete. 2024-04-25T10:18:44.566 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:44.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.761+0000 7f0d9f200640 1 Processor -- start 2024-04-25T10:18:44.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.761+0000 7f0d9f200640 1 -- start start 2024-04-25T10:18:44.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.762+0000 7f0d9f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0da015db90 0x7f0da00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.762+0000 7f0d9f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 0x7f0da00cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.762+0000 7f0d9f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da00d33f0 con 0x7f0da00d04f0 2024-04-25T10:18:44.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da00d0980 con 0x7f0da015db90 2024-04-25T10:18:44.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00d1f50 con 0x7f0da01615a0 2024-04-25T10:18:44.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9e800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0da00d04f0 0x7f0da0061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55292/0 (socket says 172.21.15.165:55292) 2024-04-25T10:18:44.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9e800640 1 -- 172.21.15.165:0/2952772429 learned_addr learned my addr 172.21.15.165:0/2952772429 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9de00640 1 --2- 172.21.15.165:0/2952772429 >> v2:172.21.15.172:3300/0 conn(0x7f0da015db90 0x7f0da00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9d400640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 0x7f0da00cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 963788273 0 0) 0x7f0da00d33f0 con 0x7f0da00d04f0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.763+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d80003520 con 0x7f0da00d04f0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1018819848 0 0) 0x7f0d80003520 con 0x7f0da00d04f0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 msgr2=0x7f0da00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 0x7f0da00cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 >> v2:172.21.15.172:3300/0 conn(0x7f0da015db90 msgr2=0x7f0da00b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 --2- 172.21.15.165:0/2952772429 >> v2:172.21.15.172:3300/0 conn(0x7f0da015db90 0x7f0da00b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0161df0 con 0x7f0da00d04f0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f0d98001670 con 0x7f0da00d04f0 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 >> v1:172.21.15.165:6789/0 conn(0x7f0da00d04f0 legacy=0x7f0da0061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d800039b0 0x7f0d80003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 0x7f0d80004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00d0980 con 0x7f0d80004d40 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00d1f50 con 0x7f0d800039b0 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8000b9c0 con 0x7f0da01615a0 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.764+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.765+0000 7f0d9de00640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d800039b0 0x7f0d80003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.765+0000 7f0d9e800640 1 -- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 msgr2=0x7f0d80004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.765+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0da01615a0 0x7f0d80004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.765+0000 7f0d9e800640 1 -- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d800039b0 msgr2=0x7f0d80003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.765+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0d800039b0 0x7f0d80003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.765+0000 7f0d9e800640 1 -- 172.21.15.165:0/2952772429 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d80003520 con 0x7f0d80004d40 2024-04-25T10:18:44.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.766+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c210 tx=0x7f0d80005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d1edef3b73c18c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.766+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98007d50 con 0x7f0d80004d40 2024-04-25T10:18:44.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.766+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98005af0 con 0x7f0d80004d40 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.766+0000 7f0d9ca00640 1 -- 172.21.15.165:0/2952772429 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98007070 con 0x7f0d80004d40 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.766+0000 7f0d9f200640 1 -- 172.21.15.165:0/2952772429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d80004d40 msgr2=0x7f0d800051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.766+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2952772429 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0d80004d40 0x7f0d800051c0 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f0d9800c210 tx=0x7f0d80005830 comp rx=0 tx=0).stop 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.767+0000 7f0d9e800640 1 -- 172.21.15.165:0/2952772429 reap_dead start 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.767+0000 7f0d9f200640 1 -- 172.21.15.165:0/2952772429 shutdown_connections 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.767+0000 7f0d9f200640 1 -- 172.21.15.165:0/2952772429 >> 172.21.15.165:0/2952772429 conn(0x7f0da00b52b0 msgr2=0x7f0da00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.767+0000 7f0d9f200640 1 -- 172.21.15.165:0/2952772429 shutdown_connections 2024-04-25T10:18:44.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.767+0000 7f0d9f200640 1 -- 172.21.15.165:0/2952772429 wait complete. 2024-04-25T10:18:44.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.768+0000 7f0d9f200640 1 Processor -- start 2024-04-25T10:18:44.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.769+0000 7f0d9f200640 1 -- start start 2024-04-25T10:18:44.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d80004d40 0x7f0da015f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0154d80 0x7f0da01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 0x7f0da00c87e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da00d1be0 con 0x7f0da014abb0 2024-04-25T10:18:44.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da01623b0 con 0x7f0da0154d80 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da0161a20 con 0x7f0d80004d40 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 0x7f0da00c87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 0x7f0da00c87e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48372/0 (socket says 172.21.15.165:48372) 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 -- 172.21.15.165:0/2376939862 learned_addr learned my addr 172.21.15.165:0/2376939862 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9de00640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d80004d40 0x7f0da015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 -- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d80004d40 msgr2=0x7f0da015f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9d400640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0154d80 0x7f0da01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d80004d40 0x7f0da015f220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 -- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0154d80 msgr2=0x7f0da01608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0154d80 0x7f0da01608a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9e800640 1 -- 172.21.15.165:0/2376939862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da0160de0 con 0x7f0da014abb0 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.770+0000 7f0d9d400640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0154d80 0x7f0da01608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.771+0000 7f0d9e800640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 0x7f0da00c87e0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f0d98034040 tx=0x7f0d98001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.771+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d98007480 con 0x7f0da014abb0 2024-04-25T10:18:44.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.771+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98044070 con 0x7f0da014abb0 2024-04-25T10:18:44.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.771+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9803e480 con 0x7f0da014abb0 2024-04-25T10:18:44.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.771+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da015f800 con 0x7f0da014abb0 2024-04-25T10:18:44.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.771+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da015fdc0 con 0x7f0da014abb0 2024-04-25T10:18:44.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.772+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da0073990 con 0x7f0da014abb0 2024-04-25T10:18:44.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.773+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d9803e620 con 0x7f0da014abb0 2024-04-25T10:18:44.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.774+0000 7f0d96000640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:44.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.774+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d980aaef0 con 0x7f0da014abb0 2024-04-25T10:18:44.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.775+0000 7f0d9de00640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:44.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.775+0000 7f0d9de00640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0da00c9570 tx=0x7f0d90008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:44.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.779+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0d98079bd0 con 0x7f0da014abb0 2024-04-25T10:18:44.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.979+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0da01588f0 con 0x7f0da014abb0 2024-04-25T10:18:44.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.981+0000 7f0d96000640 1 -- 172.21.15.165:0/2376939862 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16920 (secure 0 0 0) 0x7f0d980794d0 con 0x7f0da014abb0 2024-04-25T10:18:44.985 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:44.986 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":159,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:17:53.373638+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:17:49.587055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_data","create_time":"2024-04-25T10:17:50.826716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.984+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d7c06b4d0 msgr2=0x7f0d7c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.984+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f0da00c9570 tx=0x7f0d90008040 comp rx=0 tx=0).stop 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.984+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 msgr2=0x7f0da00c87e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.984+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 0x7f0da00c87e0 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f0d98034040 tx=0x7f0d98001a40 comp rx=0 tx=0).stop 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 shutdown_connections 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d7c06b4d0 0x7f0d7c06d990 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0da014abb0 0x7f0da00c87e0 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0da0154d80 0x7f0da01608a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 --2- 172.21.15.165:0/2376939862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0d80004d40 0x7f0da015f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:44.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 >> 172.21.15.165:0/2376939862 conn(0x7f0da00b52b0 msgr2=0x7f0da00b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:44.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 shutdown_connections 2024-04-25T10:18:44.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:44.985+0000 7f0d9f200640 1 -- 172.21.15.165:0/2376939862 wait complete. 2024-04-25T10:18:45.008 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:18:45.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.205+0000 7f5e4d800640 1 Processor -- start 2024-04-25T10:18:45.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.206+0000 7f5e4d800640 1 -- start start 2024-04-25T10:18:45.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.207+0000 7f5e4d800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5e4815b5a0 0x7f5e480b5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.207+0000 7f5e4d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e480cea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.207+0000 7f5e4d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e480cf320 con 0x7f5e4815b980 2024-04-25T10:18:45.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.207+0000 7f5e4d800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e480cf040 con 0x7f5e4815b5a0 2024-04-25T10:18:45.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.207+0000 7f5e4d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e48161970 con 0x7f5e48162450 2024-04-25T10:18:45.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5e4815b5a0 0x7f5e480b5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5e4815b5a0 0x7f5e480b5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48376/0 (socket says 172.21.15.165:48376) 2024-04-25T10:18:45.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 -- 172.21.15.165:0/952763487 learned_addr learned my addr 172.21.15.165:0/952763487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:45.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 -- 172.21.15.165:0/952763487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 msgr2=0x7f5e480cea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:45.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 --2- 172.21.15.165:0/952763487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e480cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 -- 172.21.15.165:0/952763487 >> v1:172.21.15.165:6789/0 conn(0x7f5e4815b980 legacy=0x7f5e480ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.208+0000 7f5e46a00640 1 -- 172.21.15.165:0/952763487 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e48161c50 con 0x7f5e4815b5a0 2024-04-25T10:18:45.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.209+0000 7f5e46a00640 1 --2- 172.21.15.165:0/952763487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b5a0 0x7f5e480b5530 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e3802a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f8b34e41a5002a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e45600640 1 -- 172.21.15.165:0/952763487 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38037070 con 0x7f5e4815b5a0 2024-04-25T10:18:45.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e45600640 1 -- 172.21.15.165:0/952763487 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e38032440 con 0x7f5e4815b5a0 2024-04-25T10:18:45.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e45600640 1 -- 172.21.15.165:0/952763487 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e38032810 con 0x7f5e4815b5a0 2024-04-25T10:18:45.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 -- 172.21.15.165:0/952763487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b5a0 msgr2=0x7f5e480b5530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 --2- 172.21.15.165:0/952763487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b5a0 0x7f5e480b5530 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f5e38009870 tx=0x7f5e3802a250 comp rx=0 tx=0).stop 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 -- 172.21.15.165:0/952763487 shutdown_connections 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 --2- 172.21.15.165:0/952763487 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e480cea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 --2- 172.21.15.165:0/952763487 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b5a0 0x7f5e480b5530 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 -- 172.21.15.165:0/952763487 >> 172.21.15.165:0/952763487 conn(0x7f5e480d0530 msgr2=0x7f5e480d0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 -- 172.21.15.165:0/952763487 shutdown_connections 2024-04-25T10:18:45.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.210+0000 7f5e4d800640 1 -- 172.21.15.165:0/952763487 wait complete. 2024-04-25T10:18:45.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.212+0000 7f5e4d800640 1 Processor -- start 2024-04-25T10:18:45.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.212+0000 7f5e4d800640 1 -- start start 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e4d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4815b5a0 0x7f5e480cab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e4d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 0x7f5e480cb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e4d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e4815eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e4d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480b5bf0 con 0x7f5e4815b980 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e4d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480b5a70 con 0x7f5e4815b5a0 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e4d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e480b5d70 con 0x7f5e48162450 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.213+0000 7f5e46000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 0x7f5e480cb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 0x7f5e480cb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48384/0 (socket says 172.21.15.165:48384) 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 -- 172.21.15.165:0/126889894 learned_addr learned my addr 172.21.15.165:0/126889894 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46a00640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4815b5a0 0x7f5e480cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 -- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 msgr2=0x7f5e4815eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e47400640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e4815eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e4815eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 -- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4815b5a0 msgr2=0x7f5e480cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4815b5a0 0x7f5e480cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 -- 172.21.15.165:0/126889894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e480c9900 con 0x7f5e4815b980 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46a00640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4815b5a0 0x7f5e480cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.214+0000 7f5e46000640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 0x7f5e480cb0c0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000c400 tx=0x7f5e3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.215+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e300052d0 con 0x7f5e4815b980 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.215+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e30005490 con 0x7f5e4815b980 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.215+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e480cb6d0 con 0x7f5e4815b980 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.215+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e30007260 con 0x7f5e4815b980 2024-04-25T10:18:45.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.215+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e480cbc90 con 0x7f5e4815b980 2024-04-25T10:18:45.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.216+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e48073a00 con 0x7f5e4815b980 2024-04-25T10:18:45.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.217+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e30007400 con 0x7f5e4815b980 2024-04-25T10:18:45.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.218+0000 7f5e3ea00640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.218+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e300898f0 con 0x7f5e4815b980 2024-04-25T10:18:45.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.218+0000 7f5e46a00640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.219+0000 7f5e46a00640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5e38034000 tx=0x7f5e3802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.223+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e480cf320 con 0x7f5e4815b980 2024-04-25T10:18:45.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.459+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5e480d17c0 con 0x7f5e4815b980 2024-04-25T10:18:45.464 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:18:45.460+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:18:45.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.464+0000 7f5e3ea00640 1 -- 172.21.15.165:0/126889894 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v114) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5e300585c0 con 0x7f5e4815b980 2024-04-25T10:18:45.468 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:18:45.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.467+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2406b4d0 msgr2=0x7f5e2406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.467+0000 7f5e4d800640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2406b4d0 0x7f5e2406d990 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f5e38034000 tx=0x7f5e3802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:45.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.467+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 msgr2=0x7f5e480cb0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.467+0000 7f5e4d800640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 0x7f5e480cb0c0 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000c400 tx=0x7f5e3000c8c0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 shutdown_connections 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e2406b4d0 0x7f5e2406d990 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e48162450 0x7f5e4815eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e4815b980 0x7f5e480cb0c0 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 --2- 172.21.15.165:0/126889894 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e4815b5a0 0x7f5e480cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 >> 172.21.15.165:0/126889894 conn(0x7f5e480d0530 msgr2=0x7f5e480d2740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 shutdown_connections 2024-04-25T10:18:45.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.468+0000 7f5e4d800640 1 -- 172.21.15.165:0/126889894 wait complete. 2024-04-25T10:18:45.490 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:18:45.500 INFO:tasks.ceph.mds.d.smithi172.stderr: -3567> 2024-04-25T10:17:58.324+0000 7fd701a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:18:45.590 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:18:45.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.680+0000 7ff551e00640 1 Processor -- start 2024-04-25T10:18:45.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.680+0000 7ff551e00640 1 -- start start 2024-04-25T10:18:45.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.682+0000 7ff551e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 0x7ff54c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.682+0000 7ff551e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c145520 con 0x7ff54c0c5030 2024-04-25T10:18:45.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.682+0000 7ff551e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c071270 con 0x7ff54c145140 2024-04-25T10:18:45.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.682+0000 7ff551e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c071550 con 0x7ff54c0ceb20 2024-04-25T10:18:45.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.682+0000 7ff551400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff54c0c5030 0x7ff54c15bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55308/0 (socket says 172.21.15.165:55308) 2024-04-25T10:18:45.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.682+0000 7ff551400640 1 -- 172.21.15.165:0/4273420274 learned_addr learned my addr 172.21.15.165:0/4273420274 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:45.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff543e00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 0x7ff54c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4095859891 0 0) 0x7ff54c145520 con 0x7ff54c0c5030 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff530003520 con 0x7ff54c0c5030 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1476371933 0 0) 0x7ff530003520 con 0x7ff54c0c5030 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 msgr2=0x7ff54c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 0x7ff54c0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 >> v1:172.21.15.172:6789/0 conn(0x7ff54c145140 legacy=0x7ff54c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c071830 con 0x7ff54c0c5030 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.683+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff5440019f0 con 0x7ff54c0c5030 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 >> v1:172.21.15.165:6789/0 conn(0x7ff54c0c5030 legacy=0x7ff54c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5300039e0 0x7ff530003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 0x7ff530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c071550 con 0x7ff530004d40 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53000baf0 con 0x7ff5300039e0 2024-04-25T10:18:45.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff53000bcf0 con 0x7ff54c0ceb20 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff551400640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff550a00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5300039e0 0x7ff530003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff543e00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 0x7ff530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff551400640 1 -- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 msgr2=0x7ff530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff551400640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff54c0ceb20 0x7ff530004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff551400640 1 -- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5300039e0 msgr2=0x7ff530003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff551400640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff5300039e0 0x7ff530003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.684+0000 7ff551400640 1 -- 172.21.15.165:0/4273420274 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff530003520 con 0x7ff530004d40 2024-04-25T10:18:45.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.685+0000 7ff551400640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7ff54400a840 tx=0x7ff530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f95d154cb6f70cf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400a460 con 0x7ff530004d40 2024-04-25T10:18:45.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54402f070 con 0x7ff530004d40 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff54be00640 1 -- 172.21.15.165:0/4273420274 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54402bcb0 con 0x7ff530004d40 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff551e00640 1 -- 172.21.15.165:0/4273420274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff530004d40 msgr2=0x7ff5300051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff551e00640 1 --2- 172.21.15.165:0/4273420274 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff530004d40 0x7ff5300051c0 secure :-1 s=READY pgs=2365 cs=0 l=1 rev1=1 crypto rx=0x7ff54400a840 tx=0x7ff530005830 comp rx=0 tx=0).stop 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff551400640 1 -- 172.21.15.165:0/4273420274 reap_dead start 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff551e00640 1 -- 172.21.15.165:0/4273420274 shutdown_connections 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.686+0000 7ff551e00640 1 -- 172.21.15.165:0/4273420274 >> 172.21.15.165:0/4273420274 conn(0x7ff54c061cc0 msgr2=0x7ff54c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.687+0000 7ff551e00640 1 -- 172.21.15.165:0/4273420274 shutdown_connections 2024-04-25T10:18:45.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.687+0000 7ff551e00640 1 -- 172.21.15.165:0/4273420274 wait complete. 2024-04-25T10:18:45.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.688+0000 7ff551e00640 1 Processor -- start 2024-04-25T10:18:45.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.688+0000 7ff551e00640 1 -- start start 2024-04-25T10:18:45.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.689+0000 7ff551e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff530004d40 0x7ff54c15b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.689+0000 7ff551e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 0x7ff54c0ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.689+0000 7ff551e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54c0aa960 0x7ff54c0aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.689+0000 7ff551e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0c5640 con 0x7ff54c0ad180 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.689+0000 7ff551e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0c54c0 con 0x7ff54c0aa960 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.689+0000 7ff551e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0c4af0 con 0x7ff530004d40 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 0x7ff54c0ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 0x7ff54c0ad600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48408/0 (socket says 172.21.15.165:48408) 2024-04-25T10:18:45.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 -- 172.21.15.165:0/3648357043 learned_addr learned my addr 172.21.15.165:0/3648357043 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:45.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff550a00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff530004d40 0x7ff54c15b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 -- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff530004d40 msgr2=0x7ff54c15b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff551400640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54c0aa960 0x7ff54c0aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff530004d40 0x7ff54c15b5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 -- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54c0aa960 msgr2=0x7ff54c0aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54c0aa960 0x7ff54c0aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 -- 172.21.15.165:0/3648357043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c0ab6e0 con 0x7ff54c0ad180 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff551400640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54c0aa960 0x7ff54c0aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.690+0000 7ff543e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 0x7ff54c0ad600 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7ff534010860 tx=0x7ff534010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.691+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53400f040 con 0x7ff54c0ad180 2024-04-25T10:18:45.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.691+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53400eb60 con 0x7ff54c0ad180 2024-04-25T10:18:45.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.691+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53401c400 con 0x7ff54c0ad180 2024-04-25T10:18:45.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.691+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c0ab9c0 con 0x7ff54c0ad180 2024-04-25T10:18:45.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.691+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c0abf80 con 0x7ff54c0ad180 2024-04-25T10:18:45.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.692+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c073990 con 0x7ff54c0ad180 2024-04-25T10:18:45.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.693+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff53401c5a0 con 0x7ff54c0ad180 2024-04-25T10:18:45.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.694+0000 7ff549600640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff52806b4d0 0x7ff52806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:45.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.694+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6187+0+0 (secure 0 0 0) 0x7ff53408ab20 con 0x7ff54c0ad180 2024-04-25T10:18:45.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.694+0000 7ff550a00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff52806b4d0 0x7ff52806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:45.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.695+0000 7ff550a00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff52806b4d0 0x7ff52806d990 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001590 tx=0x7ff53c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:45.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.698+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff534018030 con 0x7ff54c0ad180 2024-04-25T10:18:45.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:45.931+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff54c06ed00 con 0x7ff54c0ad180 2024-04-25T10:18:46.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.467+0000 7ff549600640 1 -- 172.21.15.165:0/3648357043 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v115) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff534059720 con 0x7ff54c0ad180 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff52806b4d0 msgr2=0x7ff52806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff52806b4d0 0x7ff52806d990 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7ff53c001590 tx=0x7ff53c009040 comp rx=0 tx=0).stop 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 msgr2=0x7ff54c0ad600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 0x7ff54c0ad600 secure :-1 s=READY pgs=2366 cs=0 l=1 rev1=1 crypto rx=0x7ff534010860 tx=0x7ff534010d20 comp rx=0 tx=0).stop 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 shutdown_connections 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff52806b4d0 0x7ff52806d990 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff54c0aa960 0x7ff54c0aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff54c0ad180 0x7ff54c0ad600 unknown :-1 s=CLOSED pgs=2366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 --2- 172.21.15.165:0/3648357043 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff530004d40 0x7ff54c15b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.471+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 >> 172.21.15.165:0/3648357043 conn(0x7ff54c061cc0 msgr2=0x7ff54c0c57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.472+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 shutdown_connections 2024-04-25T10:18:46.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.472+0000 7ff551e00640 1 -- 172.21.15.165:0/3648357043 wait complete. 2024-04-25T10:18:46.497 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:18:46.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.695+0000 7f4c09000640 1 Processor -- start 2024-04-25T10:18:46.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.695+0000 7f4c09000640 1 -- start start 2024-04-25T10:18:46.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c09000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c0415b7a0 0x7f4c040b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c09000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040b0b80 con 0x7f4c040cc150 2024-04-25T10:18:46.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c09000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c040cc800 con 0x7f4c0415b3c0 2024-04-25T10:18:46.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c09000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c040cc5e0 con 0x7f4c0415b7a0 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c03e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4c040cc150 0x7f4c0415e010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:55316/0 (socket says 172.21.15.165:55316) 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c03e00640 1 -- 172.21.15.165:0/1809366547 learned_addr learned my addr 172.21.15.165:0/1809366547 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4bfbe00640 1 --2- 172.21.15.165:0/1809366547 >> v2:172.21.15.172:3300/0 conn(0x7f4c0415b7a0 0x7f4c040b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2699643894 0 0) 0x7f4bfc001490 con 0x7f4c040cc150 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be4003520 con 0x7f4c040cc150 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3715256933 0 0) 0x7f4be4003520 con 0x7f4c040cc150 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 >> v2:172.21.15.172:3300/0 conn(0x7f4c0415b7a0 msgr2=0x7f4c040b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 --2- 172.21.15.165:0/1809366547 >> v2:172.21.15.172:3300/0 conn(0x7f4c0415b7a0 0x7f4c040b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 >> v1:172.21.15.172:6789/0 conn(0x7f4c0415b3c0 legacy=0x7f4c040cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.698+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c040ccae0 con 0x7f4c040cc150 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4bfc001bc0 con 0x7f4c040cc150 2024-04-25T10:18:46.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 >> v1:172.21.15.165:6789/0 conn(0x7f4c040cc150 legacy=0x7f4c0415e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be40039b0 0x7f4be4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4be4004800 0x7f4be4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40051c0 0x7f4be400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040cc5e0 con 0x7f4be40039b0 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400c160 con 0x7f4be40051c0 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4be400c360 con 0x7f4be4004800 2024-04-25T10:18:46.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c03400640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be40039b0 0x7f4be4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.699+0000 7f4c03e00640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40051c0 0x7f4be400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4bfbe00640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4be4004800 0x7f4be4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03400640 1 -- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4be4004800 msgr2=0x7f4be4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03400640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4be4004800 0x7f4be4004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03400640 1 -- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40051c0 msgr2=0x7f4be400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03400640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40051c0 0x7f4be400b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03400640 1 -- 172.21.15.165:0/1809366547 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4be4003520 con 0x7f4be40039b0 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03e00640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40051c0 0x7f4be400b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:46.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.700+0000 7f4c03e00640 1 -- 172.21.15.165:0/1809366547 reap_dead start 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.701+0000 7f4c03400640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be40039b0 0x7f4be4003dd0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4001750 tx=0x7f4be4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0318841a70e9f89 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.701+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf4001370 con 0x7f4be40039b0 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.701+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf40052d0 con 0x7f4be40039b0 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.701+0000 7f4c02a00640 1 -- 172.21.15.165:0/1809366547 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf402abc0 con 0x7f4be40039b0 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 -- 172.21.15.165:0/1809366547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be40039b0 msgr2=0x7f4be4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be40039b0 0x7f4be4003dd0 secure :-1 s=READY pgs=2368 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4001750 tx=0x7f4be4005830 comp rx=0 tx=0).stop 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 -- 172.21.15.165:0/1809366547 shutdown_connections 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 --2- 172.21.15.165:0/1809366547 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4be40039b0 0x7f4be4003dd0 unknown :-1 s=CLOSED pgs=2368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 -- 172.21.15.165:0/1809366547 >> 172.21.15.165:0/1809366547 conn(0x7f4c04061cc0 msgr2=0x7f4c0406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 -- 172.21.15.165:0/1809366547 shutdown_connections 2024-04-25T10:18:46.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.702+0000 7f4c09000640 1 -- 172.21.15.165:0/1809366547 wait complete. 2024-04-25T10:18:46.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.704+0000 7f4c09000640 1 Processor -- start 2024-04-25T10:18:46.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.704+0000 7f4c09000640 1 -- start start 2024-04-25T10:18:46.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c09000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40039b0 0x7f4c0415e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c09000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c0414c080 0x7f4c04148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c09000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 0x7f4c041f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c09000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040b0f60 con 0x7f4c04148cd0 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c09000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040cd650 con 0x7f4be40039b0 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c09000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040b10e0 con 0x7f4c0414c080 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 0x7f4c041f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c03e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 0x7f4c041f7390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:48424/0 (socket says 172.21.15.165:48424) 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c03e00640 1 -- 172.21.15.165:0/999938529 learned_addr learned my addr 172.21.15.165:0/999938529 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4bfbe00640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c0414c080 0x7f4c04148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.705+0000 7f4c03400640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40039b0 0x7f4c0415e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03e00640 1 -- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c0414c080 msgr2=0x7f4c04148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03e00640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c0414c080 0x7f4c04148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03e00640 1 -- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40039b0 msgr2=0x7f4c0415e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03e00640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40039b0 0x7f4c0415e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03e00640 1 -- 172.21.15.165:0/999938529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c041f7fa0 con 0x7f4c04148cd0 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03400640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40039b0 0x7f4c0415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c03e00640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 0x7f4c041f7390 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f4c04161560 tx=0x7f4bfc00b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc001ba0 con 0x7f4c04148cd0 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bfc00bb20 con 0x7f4c04148cd0 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0414b5a0 con 0x7f4c04148cd0 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bfc007070 con 0x7f4c04148cd0 2024-04-25T10:18:46.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.706+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0415e650 con 0x7f4c04148cd0 2024-04-25T10:18:46.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.707+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c04073990 con 0x7f4c04148cd0 2024-04-25T10:18:46.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.708+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4bfc01b020 con 0x7f4c04148cd0 2024-04-25T10:18:46.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.709+0000 7f4bfb400640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4be006b500 0x7f4be006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:46.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.709+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f4bfc08a550 con 0x7f4c04148cd0 2024-04-25T10:18:46.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.710+0000 7f4c03400640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4be006b500 0x7f4be006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:46.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.710+0000 7f4c03400640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4be006b500 0x7f4be006d9c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4032000 tx=0x7f4bf402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:46.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.714+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4c040b0b80 con 0x7f4c04148cd0 2024-04-25T10:18:46.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:46.910+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4c04072360 con 0x7f4c04148cd0 2024-04-25T10:18:47.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.478+0000 7f4bfb400640 1 -- 172.21.15.165:0/999938529 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v161) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4bfc0591e0 con 0x7f4c04148cd0 2024-04-25T10:18:47.482 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:18:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4be006b500 msgr2=0x7f4be006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4be006b500 0x7f4be006d9c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4032000 tx=0x7f4bf402e000 comp rx=0 tx=0).stop 2024-04-25T10:18:47.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 msgr2=0x7f4c041f7390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 0x7f4c041f7390 secure :-1 s=READY pgs=2369 cs=0 l=1 rev1=1 crypto rx=0x7f4c04161560 tx=0x7f4bfc00b4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 shutdown_connections 2024-04-25T10:18:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4be006b500 0x7f4be006d9c0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c04148cd0 0x7f4c041f7390 unknown :-1 s=CLOSED pgs=2369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c0414c080 0x7f4c04148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 --2- 172.21.15.165:0/999938529 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4be40039b0 0x7f4c0415e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.483+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 >> 172.21.15.165:0/999938529 conn(0x7f4c04061cc0 msgr2=0x7f4c040cb0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:47.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.485+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 shutdown_connections 2024-04-25T10:18:47.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.486+0000 7f4c09000640 1 -- 172.21.15.165:0/999938529 wait complete. 2024-04-25T10:18:47.511 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:18:47.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.702+0000 7f5a19e00640 1 Processor -- start 2024-04-25T10:18:47.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.702+0000 7f5a19e00640 1 -- start start 2024-04-25T10:18:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.703+0000 7f5a19e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a141452a0 0x7f5a1406dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.703+0000 7f5a19e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140cccc0 con 0x7f5a141446d0 2024-04-25T10:18:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.703+0000 7f5a19e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a140cc140 con 0x7f5a14144d50 2024-04-25T10:18:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.703+0000 7f5a19e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a140b2b20 con 0x7f5a141452a0 2024-04-25T10:18:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.703+0000 7f5a12a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5a141446d0 0x7f5a140b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44204/0 (socket says 172.21.15.165:44204) 2024-04-25T10:18:47.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.703+0000 7f5a12a00640 1 -- 172.21.15.165:0/3159377662 learned_addr learned my addr 172.21.15.165:0/3159377662 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:47.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a13400640 1 --2- 172.21.15.165:0/3159377662 >> v2:172.21.15.172:3300/0 conn(0x7f5a141452a0 0x7f5a1406dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828922976 0 0) 0x7f5a140cccc0 con 0x7f5a141446d0 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f8003520 con 0x7f5a141446d0 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038301201 0 0) 0x7f59f8003520 con 0x7f5a141446d0 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 >> v2:172.21.15.172:3300/0 conn(0x7f5a141452a0 msgr2=0x7f5a1406dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 --2- 172.21.15.165:0/3159377662 >> v2:172.21.15.172:3300/0 conn(0x7f5a141452a0 0x7f5a1406dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 >> v1:172.21.15.172:6789/0 conn(0x7f5a14144d50 legacy=0x7f5a1406d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140cc420 con 0x7f5a141446d0 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.704+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5a04001670 con 0x7f5a141446d0 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 >> v1:172.21.15.165:6789/0 conn(0x7f5a141446d0 legacy=0x7f5a140b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59f80039b0 0x7f59f8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f8004800 0x7f59f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80051c0 0x7f59f800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140b2b20 con 0x7f59f80039b0 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59f800c060 con 0x7f59f8004800 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59f800c260 con 0x7f59f80051c0 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12a00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59f80039b0 0x7f59f8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a13400640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80051c0 0x7f59f800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12a00640 1 -- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80051c0 msgr2=0x7f59f800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a0be00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f8004800 0x7f59f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12a00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80051c0 0x7f59f800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12a00640 1 -- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f8004800 msgr2=0x7f59f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12a00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f59f8004800 0x7f59f8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a12a00640 1 -- 172.21.15.165:0/3159377662 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f8003520 con 0x7f59f80039b0 2024-04-25T10:18:47.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.705+0000 7f5a13400640 1 -- 172.21.15.165:0/3159377662 reap_dead start 2024-04-25T10:18:47.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.706+0000 7f5a12a00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59f80039b0 0x7f59f8003dd0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c240 tx=0x7f59f8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a21a91e4747e7f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04007cf0 con 0x7f59f80039b0 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a04005af0 con 0x7f59f80039b0 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a12000640 1 -- 172.21.15.165:0/3159377662 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a04007070 con 0x7f59f80039b0 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a19e00640 1 -- 172.21.15.165:0/3159377662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59f80039b0 msgr2=0x7f59f8003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a19e00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59f80039b0 0x7f59f8003dd0 secure :-1 s=READY pgs=2371 cs=0 l=1 rev1=1 crypto rx=0x7f5a0400c240 tx=0x7f59f8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a19e00640 1 -- 172.21.15.165:0/3159377662 shutdown_connections 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a19e00640 1 --2- 172.21.15.165:0/3159377662 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f59f80039b0 0x7f59f8003dd0 unknown :-1 s=CLOSED pgs=2371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.707+0000 7f5a19e00640 1 -- 172.21.15.165:0/3159377662 >> 172.21.15.165:0/3159377662 conn(0x7f5a1406abe0 msgr2=0x7f5a1406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.708+0000 7f5a19e00640 1 -- 172.21.15.165:0/3159377662 shutdown_connections 2024-04-25T10:18:47.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.708+0000 7f5a19e00640 1 -- 172.21.15.165:0/3159377662 wait complete. 2024-04-25T10:18:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.709+0000 7f5a19e00640 1 Processor -- start 2024-04-25T10:18:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.709+0000 7f5a19e00640 1 -- start start 2024-04-25T10:18:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.710+0000 7f5a19e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80039b0 0x7f5a141eeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.710+0000 7f5a19e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 0x7f5a140ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.710+0000 7f5a19e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140ab4b0 0x7f5a140adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.710+0000 7f5a19e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a140cc9e0 con 0x7f5a140ae100 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.710+0000 7f5a19e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1406d400 con 0x7f5a140ab4b0 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.710+0000 7f5a19e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a14144ab0 con 0x7f59f80039b0 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 0x7f5a140ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 0x7f5a140ad560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41738/0 (socket says 172.21.15.165:41738) 2024-04-25T10:18:47.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 -- 172.21.15.165:0/2596048960 learned_addr learned my addr 172.21.15.165:0/2596048960 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a12a00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80039b0 0x7f5a141eeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a13400640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140ab4b0 0x7f5a140adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 -- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80039b0 msgr2=0x7f5a141eeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80039b0 0x7f5a141eeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 -- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140ab4b0 msgr2=0x7f5a140adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:47.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140ab4b0 0x7f5a140adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 -- 172.21.15.165:0/2596048960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a140ab9b0 con 0x7f5a140ae100 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.711+0000 7f5a0be00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 0x7f5a140ad560 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d4c0 tx=0x7f59fc00d980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.712+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc005480 con 0x7f5a140ae100 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.712+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc017070 con 0x7f5a140ae100 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.712+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc003830 con 0x7f5a140ae100 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.712+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a140abc90 con 0x7f5a140ae100 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.712+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a140ac250 con 0x7f5a140ae100 2024-04-25T10:18:47.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.713+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a14073a00 con 0x7f5a140ae100 2024-04-25T10:18:47.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.714+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f59fc003a90 con 0x7f5a140ae100 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.716+0000 7f5a0b400640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.716+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f59fc08a140 con 0x7f5a140ae100 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.716+0000 7f5a12a00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:47.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.716+0000 7f5a12a00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59f006b4d0 0x7f59f006d990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5a04032000 tx=0x7f5a04009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:47.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.719+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f59fc058e60 con 0x7f5a140ae100 2024-04-25T10:18:47.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:47.931+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5a1406ea00 con 0x7f5a140ae100 2024-04-25T10:18:48.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.483+0000 7f5a0b400640 1 -- 172.21.15.165:0/2596048960 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v162) v1 ==== 169+0+0 (secure 0 0 0) 0x7f59fc058760 con 0x7f5a140ae100 2024-04-25T10:18:48.486 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:18:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59f006b4d0 msgr2=0x7f59f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59f006b4d0 0x7f59f006d990 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f5a04032000 tx=0x7f5a04009030 comp rx=0 tx=0).stop 2024-04-25T10:18:48.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 msgr2=0x7f5a140ad560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 0x7f5a140ad560 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d4c0 tx=0x7f59fc00d980 comp rx=0 tx=0).stop 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 shutdown_connections 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f59f006b4d0 0x7f59f006d990 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140ab4b0 0x7f5a140adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a140ae100 0x7f5a140ad560 secure :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7f59fc00d4c0 tx=0x7f59fc00d980 comp rx=0 tx=0).stop 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.486+0000 7f5a19e00640 1 --2- 172.21.15.165:0/2596048960 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f59f80039b0 0x7f5a141eeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.487+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 >> 172.21.15.165:0/2596048960 conn(0x7f5a1406abe0 msgr2=0x7f5a1406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:48.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.487+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 shutdown_connections 2024-04-25T10:18:48.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.487+0000 7f5a19e00640 1 -- 172.21.15.165:0/2596048960 wait complete. 2024-04-25T10:18:48.515 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:18:48.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.703+0000 7fb7b0600640 1 Processor -- start 2024-04-25T10:18:48.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.703+0000 7fb7b0600640 1 -- start start 2024-04-25T10:18:48.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7b0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 0x7fb7a80d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7b0600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a815e000 con 0x7fb7a8161710 2024-04-25T10:18:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7b0600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a80d1de0 con 0x7fb7a815dc20 2024-04-25T10:18:48.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7b0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8161c60 con 0x7fb7a80ced80 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7ae600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb7a8161710 0x7fb7a80d2820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44220/0 (socket says 172.21.15.165:44220) 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7ae600640 1 -- 172.21.15.165:0/4170591917 learned_addr learned my addr 172.21.15.165:0/4170591917 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.705+0000 7fb7ad200640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 0x7fb7a80d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690740336 0 0) 0x7fb7a815e000 con 0x7fb7a8161710 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb790003520 con 0x7fb7a8161710 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1285245411 0 0) 0x7fb790003520 con 0x7fb7a8161710 2024-04-25T10:18:48.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 msgr2=0x7fb7a80d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 0x7fb7a80d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 >> v1:172.21.15.172:6789/0 conn(0x7fb7a815dc20 legacy=0x7fb7a80ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a8161f40 con 0x7fb7a8161710 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb7a00019f0 con 0x7fb7a8161710 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 >> v1:172.21.15.165:6789/0 conn(0x7fb7a8161710 legacy=0x7fb7a80d2820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7900039e0 0x7fb790003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb790004800 0x7fb790004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 0x7fb7900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8161c60 con 0x7fb7900039e0 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb79000baa0 con 0x7fb790004800 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb79000bca0 con 0x7fb7a80ced80 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.706+0000 7fb7adc00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7900039e0 0x7fb790003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7ae600640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 0x7fb7900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7ad200640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb790004800 0x7fb790004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7adc00640 1 -- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 msgr2=0x7fb7900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:48.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7adc00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a80ced80 0x7fb7900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7adc00640 1 -- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb790004800 msgr2=0x7fb790004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:48.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7adc00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb790004800 0x7fb790004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7adc00640 1 -- 172.21.15.165:0/4170591917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790003520 con 0x7fb7900039e0 2024-04-25T10:18:48.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.707+0000 7fb7ad200640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb790004800 0x7fb790004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.708+0000 7fb7adc00640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7900039e0 0x7fb790003e00 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fb79c001720 tx=0x7fb790005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3ba08f986989f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.708+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c001370 con 0x7fb7900039e0 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.708+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c0052d0 con 0x7fb7900039e0 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.708+0000 7fb7a7e00640 1 -- 172.21.15.165:0/4170591917 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c02ab90 con 0x7fb7900039e0 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7b0600640 1 -- 172.21.15.165:0/4170591917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7900039e0 msgr2=0x7fb790003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7b0600640 1 --2- 172.21.15.165:0/4170591917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7900039e0 0x7fb790003e00 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7fb79c001720 tx=0x7fb790005830 comp rx=0 tx=0).stop 2024-04-25T10:18:48.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7ae600640 1 -- 172.21.15.165:0/4170591917 reap_dead start 2024-04-25T10:18:48.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7b0600640 1 -- 172.21.15.165:0/4170591917 shutdown_connections 2024-04-25T10:18:48.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7b0600640 1 -- 172.21.15.165:0/4170591917 >> 172.21.15.165:0/4170591917 conn(0x7fb7a80d0420 msgr2=0x7fb7a80d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:48.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7b0600640 1 -- 172.21.15.165:0/4170591917 shutdown_connections 2024-04-25T10:18:48.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.709+0000 7fb7b0600640 1 -- 172.21.15.165:0/4170591917 wait complete. 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.711+0000 7fb7b0600640 1 Processor -- start 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.711+0000 7fb7b0600640 1 -- start start 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7b0600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7900039e0 0x7fb7a8160680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7b0600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 0x7fb7a815a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7b0600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a815a7f0 0x7fb7a8157740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7b0600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b5e50 con 0x7fb7a80c9db0 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7b0600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a8162500 con 0x7fb7900039e0 2024-04-25T10:18:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7b0600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a80b5fd0 con 0x7fb7a815a7f0 2024-04-25T10:18:48.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7ad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 0x7fb7a815a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7ad200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 0x7fb7a815a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41760/0 (socket says 172.21.15.165:41760) 2024-04-25T10:18:48.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.712+0000 7fb7ad200640 1 -- 172.21.15.165:0/2456600385 learned_addr learned my addr 172.21.15.165:0/2456600385 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7adc00640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7900039e0 0x7fb7a8160680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ae600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a815a7f0 0x7fb7a8157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ad200640 1 -- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a815a7f0 msgr2=0x7fb7a8157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ad200640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a815a7f0 0x7fb7a8157740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ad200640 1 -- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7900039e0 msgr2=0x7fb7a8160680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ad200640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7900039e0 0x7fb7a8160680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ad200640 1 -- 172.21.15.165:0/2456600385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a815f5e0 con 0x7fb7a80c9db0 2024-04-25T10:18:48.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7ad200640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 0x7fb7a815a2b0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fb794010890 tx=0x7fb794010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79400f040 con 0x7fb7a80c9db0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79400eb00 con 0x7fb7a80c9db0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a815f8c0 con 0x7fb7a80c9db0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.713+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79401c400 con 0x7fb7a80c9db0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.714+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a815fe80 con 0x7fb7a80c9db0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.715+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a8073a00 con 0x7fb7a80c9db0 2024-04-25T10:18:48.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.716+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb79401c5a0 con 0x7fb7a80c9db0 2024-04-25T10:18:48.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.717+0000 7fb7a5600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb78806b500 0x7fb78806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:48.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.717+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fb79408a5f0 con 0x7fb7a80c9db0 2024-04-25T10:18:48.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.717+0000 7fb7adc00640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb78806b500 0x7fb78806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:48.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.718+0000 7fb7adc00640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb78806b500 0x7fb78806d9c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb79c032000 tx=0x7fb79c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:48.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.721+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb794018030 con 0x7fb7a80c9db0 2024-04-25T10:18:48.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.918+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb7a8160bc0 con 0x7fb7a80c9db0 2024-04-25T10:18:48.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.926+0000 7fb7a5600640 1 -- 172.21.15.165:0/2456600385 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb794059540 con 0x7fb7a80c9db0 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb78806b500 msgr2=0x7fb78806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb78806b500 0x7fb78806d9c0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fb79c032000 tx=0x7fb79c02e000 comp rx=0 tx=0).stop 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 msgr2=0x7fb7a815a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 0x7fb7a815a2b0 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7fb794010890 tx=0x7fb794010d50 comp rx=0 tx=0).stop 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 shutdown_connections 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb78806b500 0x7fb78806d9c0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb7a815a7f0 0x7fb7a8157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb7a80c9db0 0x7fb7a815a2b0 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 --2- 172.21.15.165:0/2456600385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb7900039e0 0x7fb7a8160680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 >> 172.21.15.165:0/2456600385 conn(0x7fb7a80d0420 msgr2=0x7fb7a80b7440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 shutdown_connections 2024-04-25T10:18:48.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:48.929+0000 7fb7b0600640 1 -- 172.21.15.165:0/2456600385 wait complete. 2024-04-25T10:18:48.950 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace' 2024-04-25T10:18:49.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.142+0000 7f33a7800640 1 Processor -- start 2024-04-25T10:18:49.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.142+0000 7f33a7800640 1 -- start start 2024-04-25T10:18:49.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f33a7800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f33a00ce590 0x7f33a0165900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f33a7800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a0071480 con 0x7f33a00cfec0 2024-04-25T10:18:49.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f33a7800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a0071760 con 0x7f33a0154ee0 2024-04-25T10:18:49.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f33a7800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33a0072090 con 0x7f33a00ce590 2024-04-25T10:18:49.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f3397e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f33a00cfec0 0x7f33a00cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44222/0 (socket says 172.21.15.165:44222) 2024-04-25T10:18:49.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f3397e00640 1 -- 172.21.15.165:0/3431477300 learned_addr learned my addr 172.21.15.165:0/3431477300 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:49.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f33a5800640 1 --2- 172.21.15.165:0/3431477300 >> v2:172.21.15.172:3300/0 conn(0x7f33a00ce590 0x7f33a0165900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.144+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3081263319 0 0) 0x7f33a0071480 con 0x7f33a00cfec0 2024-04-25T10:18:49.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3384003520 con 0x7f33a00cfec0 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1479844316 0 0) 0x7f3384003520 con 0x7f33a00cfec0 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 >> v2:172.21.15.172:3300/0 conn(0x7f33a00ce590 msgr2=0x7f33a0165900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 --2- 172.21.15.165:0/3431477300 >> v2:172.21.15.172:3300/0 conn(0x7f33a00ce590 0x7f33a0165900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 >> v1:172.21.15.172:6789/0 conn(0x7f33a0154ee0 legacy=0x7f33a00cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a0071a40 con 0x7f33a00cfec0 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3388001670 con 0x7f33a00cfec0 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 >> v1:172.21.15.165:6789/0 conn(0x7f33a00cfec0 legacy=0x7f33a00cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33840039b0 0x7f3384003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3384004800 0x7f3384004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f338400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338400e240 con 0x7f33840051c0 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338400e3f0 con 0x7f33840039b0 2024-04-25T10:18:49.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338400e5f0 con 0x7f3384004800 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.145+0000 7f33a5800640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f338400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f33a4e00640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33840039b0 0x7f3384003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f3397e00640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3384004800 0x7f3384004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f33a5800640 1 -- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3384004800 msgr2=0x7f3384004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f33a5800640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3384004800 0x7f3384004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f33a5800640 1 -- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33840039b0 msgr2=0x7f3384003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f33a5800640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33840039b0 0x7f3384003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.146+0000 7f33a5800640 1 -- 172.21.15.165:0/3431477300 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3384003520 con 0x7f33840051c0 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.147+0000 7f33a5800640 1 -- 172.21.15.165:0/3431477300 reap_dead start 2024-04-25T10:18:49.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.147+0000 7f33a5800640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f338400dbd0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f3398009f90 tx=0x7f3384005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ac78023b4762dd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.147+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398035020 con 0x7f33840051c0 2024-04-25T10:18:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.147+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398003b90 con 0x7f33840051c0 2024-04-25T10:18:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.147+0000 7f339fe00640 1 -- 172.21.15.165:0/3431477300 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398009c10 con 0x7f33840051c0 2024-04-25T10:18:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 -- 172.21.15.165:0/3431477300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 msgr2=0x7f338400dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f338400dbd0 secure :-1 s=READY pgs=2377 cs=0 l=1 rev1=1 crypto rx=0x7f3398009f90 tx=0x7f3384005830 comp rx=0 tx=0).stop 2024-04-25T10:18:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 -- 172.21.15.165:0/3431477300 shutdown_connections 2024-04-25T10:18:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 --2- 172.21.15.165:0/3431477300 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f338400dbd0 unknown :-1 s=CLOSED pgs=2377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 -- 172.21.15.165:0/3431477300 >> 172.21.15.165:0/3431477300 conn(0x7f33a006abe0 msgr2=0x7f33a006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 -- 172.21.15.165:0/3431477300 shutdown_connections 2024-04-25T10:18:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.148+0000 7f33a7800640 1 -- 172.21.15.165:0/3431477300 wait complete. 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.150+0000 7f33a7800640 1 Processor -- start 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.150+0000 7f33a7800640 1 -- start start 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a7800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f33a01f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a7800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33a013fc80 0x7f33a01eea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a7800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33a01401d0 0x7f33a0152b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a7800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a01552c0 con 0x7f33840051c0 2024-04-25T10:18:49.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a7800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0071230 con 0x7f33a01401d0 2024-04-25T10:18:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a7800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a0155440 con 0x7f33a013fc80 2024-04-25T10:18:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f33a01f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f33a01f32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41782/0 (socket says 172.21.15.165:41782) 2024-04-25T10:18:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.151+0000 7f33a4e00640 1 -- 172.21.15.165:0/1153977966 learned_addr learned my addr 172.21.15.165:0/1153977966 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a5800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33a01401d0 0x7f33a0152b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f3397e00640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33a013fc80 0x7f33a01eea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a4e00640 1 -- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33a013fc80 msgr2=0x7f33a01eea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a4e00640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33a013fc80 0x7f33a01eea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a4e00640 1 -- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33a01401d0 msgr2=0x7f33a0152b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a4e00640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33a01401d0 0x7f33a0152b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a4e00640 1 -- 172.21.15.165:0/1153977966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a01f2850 con 0x7f33840051c0 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a4e00640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f33a01f32f0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f339000e500 tx=0x7f339000e9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33900052e0 con 0x7f33840051c0 2024-04-25T10:18:49.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3390018070 con 0x7f33840051c0 2024-04-25T10:18:49.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3390003810 con 0x7f33840051c0 2024-04-25T10:18:49.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.152+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a014dff0 con 0x7f33840051c0 2024-04-25T10:18:49.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.153+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a014e2d0 con 0x7f33840051c0 2024-04-25T10:18:49.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.154+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a0073990 con 0x7f33840051c0 2024-04-25T10:18:49.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.155+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33900039b0 con 0x7f33840051c0 2024-04-25T10:18:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.156+0000 7f339d600640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f337c06b4d0 0x7f337c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.156+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3390089dc0 con 0x7f33840051c0 2024-04-25T10:18:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.156+0000 7f3397e00640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f337c06b4d0 0x7f337c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.157+0000 7f3397e00640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f337c06b4d0 0x7f337c06d990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3388001eb0 tx=0x7f3388000ef0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.160+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3390058d20 con 0x7f33840051c0 2024-04-25T10:18:49.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.433+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]} v 0) v1 -- 0x7f33a006d360 con 0x7f33840051c0 2024-04-25T10:18:49.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.486+0000 7f339d600640 1 -- 172.21.15.165:0/1153977966 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_mdsdir_scrub_backtrace"]}]=0 v770) v1 ==== 140+0+0 (secure 0 0 0) 0x7f3390058620 con 0x7f33840051c0 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f337c06b4d0 msgr2=0x7f337c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f337c06b4d0 0x7f337c06d990 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3388001eb0 tx=0x7f3388000ef0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 msgr2=0x7f33a01f32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f33a01f32f0 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f339000e500 tx=0x7f339000e9c0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 shutdown_connections 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f337c06b4d0 0x7f337c06d990 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33a01401d0 0x7f33a0152b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33a013fc80 0x7f33a01eea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 --2- 172.21.15.165:0/1153977966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33840051c0 0x7f33a01f32f0 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.489+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 >> 172.21.15.165:0/1153977966 conn(0x7f33a006abe0 msgr2=0x7f33a015c0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.490+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 shutdown_connections 2024-04-25T10:18:49.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.490+0000 7f33a7800640 1 -- 172.21.15.165:0/1153977966 wait complete. 2024-04-25T10:18:49.512 INFO:tasks.cephfs_test_runner:test_mdsdir_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:18:49.512 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:18:49.512 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.704+0000 7f11c9a00640 1 Processor -- start 2024-04-25T10:18:49.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.704+0000 7f11c9a00640 1 -- start start 2024-04-25T10:18:49.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.705+0000 7f11c9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 0x7f11c40cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.705+0000 7f11c9a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c415bf30 con 0x7f11c4071700 2024-04-25T10:18:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11c9a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11c415c210 con 0x7f11c4144440 2024-04-25T10:18:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11c9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415c860 con 0x7f11c415dd00 2024-04-25T10:18:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11bb400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f11c4071700 0x7f11c40cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44224/0 (socket says 172.21.15.165:44224) 2024-04-25T10:18:49.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11bb400640 1 -- 172.21.15.165:0/3301881536 learned_addr learned my addr 172.21.15.165:0/3301881536 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11c9000640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 0x7f11c40cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 244869798 0 0) 0x7f11c415bf30 con 0x7f11c4071700 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.706+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11a8003520 con 0x7f11c4071700 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 768715225 0 0) 0x7f11a8003520 con 0x7f11c4071700 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 msgr2=0x7f11c40cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 0x7f11c40cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 >> v1:172.21.15.172:6789/0 conn(0x7f11c4144440 legacy=0x7f11c4070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c415c4f0 con 0x7f11c4071700 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f11ac001670 con 0x7f11c4071700 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 >> v1:172.21.15.165:6789/0 conn(0x7f11c4071700 legacy=0x7f11c40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 0x7f11a80039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8003fb0 0x7f11a8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c415c860 con 0x7f11a8003fb0 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11a800bb90 con 0x7f11a8004d40 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11a800bd90 con 0x7f11c415dd00 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.707+0000 7f11bb400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8003fb0 0x7f11a8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11c3e00640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 0x7f11a80039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11c9000640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11bb400640 1 -- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 msgr2=0x7f11a80039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11bb400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11c415dd00 0x7f11a80039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11bb400640 1 -- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a8004d40 msgr2=0x7f11a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11bb400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11a8004d40 0x7f11a80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.708+0000 7f11bb400640 1 -- 172.21.15.165:0/3301881536 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a8003520 con 0x7f11a8003fb0 2024-04-25T10:18:49.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.709+0000 7f11bb400640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8003fb0 0x7f11a8004800 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c110 tx=0x7f11a8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a511d161d54e2131 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.709+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac007c40 con 0x7f11a8003fb0 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.709+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11ac005af0 con 0x7f11a8003fb0 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.709+0000 7f11c3400640 1 -- 172.21.15.165:0/3301881536 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11ac007070 con 0x7f11a8003fb0 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9a00640 1 -- 172.21.15.165:0/3301881536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8003fb0 msgr2=0x7f11a8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3301881536 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11a8003fb0 0x7f11a8004800 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7f11ac00c110 tx=0x7f11a8005830 comp rx=0 tx=0).stop 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9000640 1 -- 172.21.15.165:0/3301881536 reap_dead start 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9a00640 1 -- 172.21.15.165:0/3301881536 shutdown_connections 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9a00640 1 -- 172.21.15.165:0/3301881536 >> 172.21.15.165:0/3301881536 conn(0x7f11c406abe0 msgr2=0x7f11c406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9a00640 1 -- 172.21.15.165:0/3301881536 shutdown_connections 2024-04-25T10:18:49.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.710+0000 7f11c9a00640 1 -- 172.21.15.165:0/3301881536 wait complete. 2024-04-25T10:18:49.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.712+0000 7f11c9a00640 1 Processor -- start 2024-04-25T10:18:49.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.712+0000 7f11c9a00640 1 -- start start 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a8003fb0 0x7f11c415af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c40c6150 0x7f11c40c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 0x7f11c413b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40cdbb0 con 0x7f11c413af30 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40cda30 con 0x7f11c40c6150 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11c40cdd30 con 0x7f11a8003fb0 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 0x7f11c413b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 0x7f11c413b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41808/0 (socket says 172.21.15.165:41808) 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 -- 172.21.15.165:0/3274979937 learned_addr learned my addr 172.21.15.165:0/3274979937 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c3e00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a8003fb0 0x7f11c415af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 -- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a8003fb0 msgr2=0x7f11c415af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a8003fb0 0x7f11c415af80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 -- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c40c6150 msgr2=0x7f11c40c80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c40c6150 0x7f11c40c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.713+0000 7f11c9000640 1 -- 172.21.15.165:0/3274979937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11c40c65e0 con 0x7f11c413af30 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.714+0000 7f11c9000640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 0x7f11c413b3e0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f11c40cef20 tx=0x7f11bc00f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.714+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc01c070 con 0x7f11c413af30 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.714+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11bc017400 con 0x7f11c413af30 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.714+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11bc0177f0 con 0x7f11c413af30 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.714+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11c40c68c0 con 0x7f11c413af30 2024-04-25T10:18:49.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.714+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c40c6e80 con 0x7f11c413af30 2024-04-25T10:18:49.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.715+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11c4073990 con 0x7f11c413af30 2024-04-25T10:18:49.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.716+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11bc01f070 con 0x7f11c413af30 2024-04-25T10:18:49.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.717+0000 7f11c0c00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11a006b4d0 0x7f11a006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:49.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.717+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f11bc08a1b0 con 0x7f11c413af30 2024-04-25T10:18:49.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.717+0000 7f11c3e00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11a006b4d0 0x7f11a006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:49.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.718+0000 7f11c3e00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11a006b4d0 0x7f11a006d990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f11b4007fd0 tx=0x7f11b4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:49.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.722+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11bc012070 con 0x7f11c413af30 2024-04-25T10:18:49.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.918+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11c4140480 con 0x7f11c413af30 2024-04-25T10:18:49.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.919+0000 7f11c0c00640 1 -- 172.21.15.165:0/3274979937 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v162) v1 ==== 74+0+13341 (secure 0 0 0) 0x7f11bc058ab0 con 0x7f11c413af30 2024-04-25T10:18:49.922 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:49.922 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":162,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:18:48.479492+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.172:6834/1799333237":"2024-04-26T10:18:45.462093+0000","172.21.15.172:6832/1799333237":"2024-04-26T10:18:45.462093+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.921+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11a006b4d0 msgr2=0x7f11a006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11a006b4d0 0x7f11a006d990 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f11b4007fd0 tx=0x7f11b4009040 comp rx=0 tx=0).stop 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 msgr2=0x7f11c413b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 0x7f11c413b3e0 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7f11c40cef20 tx=0x7f11bc00f4a0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 shutdown_connections 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f11a006b4d0 0x7f11a006d990 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11c413af30 0x7f11c413b3e0 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11c40c6150 0x7f11c40c80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 --2- 172.21.15.165:0/3274979937 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11a8003fb0 0x7f11c415af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 >> 172.21.15.165:0/3274979937 conn(0x7f11c406abe0 msgr2=0x7f11c413f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 shutdown_connections 2024-04-25T10:18:49.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:49.922+0000 7f11c9a00640 1 -- 172.21.15.165:0/3274979937 wait complete. 2024-04-25T10:18:49.944 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:49.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:50.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.147+0000 7ff3f1600640 1 Processor -- start 2024-04-25T10:18:50.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.148+0000 7ff3f1600640 1 -- start start 2024-04-25T10:18:50.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.149+0000 7ff3f1600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff3ec144d50 0x7ff3ec06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.149+0000 7ff3f1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 0x7ff3ec06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.149+0000 7ff3f1600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3ec0cccc0 con 0x7ff3ec1446d0 2024-04-25T10:18:50.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.149+0000 7ff3f1600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3ec0b2b20 con 0x7ff3ec144d50 2024-04-25T10:18:50.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.149+0000 7ff3f1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec06d400 con 0x7ff3ec1452a0 2024-04-25T10:18:50.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.150+0000 7ff3eaa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff3ec1446d0 0x7ff3ec0b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44230/0 (socket says 172.21.15.165:44230) 2024-04-25T10:18:50.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.150+0000 7ff3eaa00640 1 -- 172.21.15.165:0/2591585468 learned_addr learned my addr 172.21.15.165:0/2591585468 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:50.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.150+0000 7ff3eb400640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 0x7ff3ec06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.150+0000 7ff3e3e00640 1 --2- 172.21.15.165:0/2591585468 >> v2:172.21.15.172:3300/0 conn(0x7ff3ec144d50 0x7ff3ec06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.150+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2635467495 0 0) 0x7ff3ec0cccc0 con 0x7ff3ec1446d0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.150+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3d0003520 con 0x7ff3ec1446d0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 954382100 0 0) 0x7ff3d0003520 con 0x7ff3ec1446d0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 msgr2=0x7ff3ec06ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 0x7ff3ec06ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 >> v2:172.21.15.172:3300/0 conn(0x7ff3ec144d50 msgr2=0x7ff3ec06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 --2- 172.21.15.165:0/2591585468 >> v2:172.21.15.172:3300/0 conn(0x7ff3ec144d50 0x7ff3ec06d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec0cc140 con 0x7ff3ec1446d0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff3dc001670 con 0x7ff3ec1446d0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 >> v1:172.21.15.165:6789/0 conn(0x7ff3ec1446d0 legacy=0x7ff3ec0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3d0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3d0004800 0x7ff3d0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 0x7ff3d00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0b2b20 con 0x7ff3d00039b0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec06d400 con 0x7ff3d0004800 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d000b9c0 con 0x7ff3ec1452a0 2024-04-25T10:18:50.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.151+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3d0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.152+0000 7ff3eaa00640 1 -- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 msgr2=0x7ff3d00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.152+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec1452a0 0x7ff3d00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.152+0000 7ff3eaa00640 1 -- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3d0004800 msgr2=0x7ff3d0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.152+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3d0004800 0x7ff3d0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.152+0000 7ff3eaa00640 1 -- 172.21.15.165:0/2591585468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0003520 con 0x7ff3d00039b0 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3d0003dd0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c190 tx=0x7ff3d0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ad6ada2dcea605d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007c60 con 0x7ff3d00039b0 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc005af0 con 0x7ff3d00039b0 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3ea000640 1 -- 172.21.15.165:0/2591585468 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007070 con 0x7ff3d00039b0 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3f1600640 1 -- 172.21.15.165:0/2591585468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 msgr2=0x7ff3d0003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3f1600640 1 --2- 172.21.15.165:0/2591585468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3d0003dd0 secure :-1 s=READY pgs=2383 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc00c190 tx=0x7ff3d0005830 comp rx=0 tx=0).stop 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.153+0000 7ff3eb400640 1 -- 172.21.15.165:0/2591585468 reap_dead start 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.154+0000 7ff3f1600640 1 -- 172.21.15.165:0/2591585468 shutdown_connections 2024-04-25T10:18:50.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.154+0000 7ff3f1600640 1 -- 172.21.15.165:0/2591585468 >> 172.21.15.165:0/2591585468 conn(0x7ff3ec06abe0 msgr2=0x7ff3ec06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.154+0000 7ff3f1600640 1 -- 172.21.15.165:0/2591585468 shutdown_connections 2024-04-25T10:18:50.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.154+0000 7ff3f1600640 1 -- 172.21.15.165:0/2591585468 wait complete. 2024-04-25T10:18:50.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.156+0000 7ff3f1600640 1 Processor -- start 2024-04-25T10:18:50.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.156+0000 7ff3f1600640 1 -- start start 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.157+0000 7ff3f1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3ec142c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.157+0000 7ff3f1600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec0aff50 0x7ff3ec0cad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3f1600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0cb240 0x7ff3ec1f35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3f1600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cc9e0 con 0x7ff3d00039b0 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3f1600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec0cc700 con 0x7ff3ec0cb240 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3f1600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ec144ab0 con 0x7ff3ec0aff50 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3ec142c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3ec142c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41834/0 (socket says 172.21.15.165:41834) 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 -- 172.21.15.165:0/1233757321 learned_addr learned my addr 172.21.15.165:0/1233757321 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:50.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 -- 172.21.15.165:0/1233757321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec0aff50 msgr2=0x7ff3ec0cad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec0aff50 0x7ff3ec0cad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 -- 172.21.15.165:0/1233757321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0cb240 msgr2=0x7ff3ec1f35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0cb240 0x7ff3ec1f35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.158+0000 7ff3eaa00640 1 -- 172.21.15.165:0/1233757321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ec1f45a0 con 0x7ff3d00039b0 2024-04-25T10:18:50.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.159+0000 7ff3eaa00640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3ec142c00 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc02ad60 tx=0x7ff3ec06d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.159+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc041020 con 0x7ff3d00039b0 2024-04-25T10:18:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.159+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3dc001430 con 0x7ff3d00039b0 2024-04-25T10:18:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.159+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3dc007070 con 0x7ff3d00039b0 2024-04-25T10:18:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.159+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3ec0cb6c0 con 0x7ff3d00039b0 2024-04-25T10:18:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.159+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ec0af470 con 0x7ff3d00039b0 2024-04-25T10:18:50.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.160+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ec073a00 con 0x7ff3d00039b0 2024-04-25T10:18:50.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.161+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff3dc044020 con 0x7ff3d00039b0 2024-04-25T10:18:50.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.162+0000 7ff3e2a00640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3c806b4d0 0x7ff3c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.163+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7ff3dc0aa2e0 con 0x7ff3d00039b0 2024-04-25T10:18:50.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.163+0000 7ff3e3e00640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3c806b4d0 0x7ff3c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.163+0000 7ff3e3e00640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3c806b4d0 0x7ff3c806d990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4006e80 tx=0x7ff3d4006dd0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.167+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff3dc079360 con 0x7ff3d00039b0 2024-04-25T10:18:50.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.362+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff3ec06ec20 con 0x7ff3d00039b0 2024-04-25T10:18:50.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.363+0000 7ff3e2a00640 1 -- 172.21.15.165:0/1233757321 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v162) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3dc078c60 con 0x7ff3d00039b0 2024-04-25T10:18:50.367 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3c806b4d0 msgr2=0x7ff3c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3c806b4d0 0x7ff3c806d990 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4006e80 tx=0x7ff3d4006dd0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 msgr2=0x7ff3ec142c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3ec142c00 secure :-1 s=READY pgs=2384 cs=0 l=1 rev1=1 crypto rx=0x7ff3dc02ad60 tx=0x7ff3ec06d370 comp rx=0 tx=0).stop 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 shutdown_connections 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff3c806b4d0 0x7ff3c806d990 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff3ec0cb240 0x7ff3ec1f35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff3ec0aff50 0x7ff3ec0cad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 --2- 172.21.15.165:0/1233757321 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff3d00039b0 0x7ff3ec142c00 unknown :-1 s=CLOSED pgs=2384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 >> 172.21.15.165:0/1233757321 conn(0x7ff3ec06abe0 msgr2=0x7ff3ec0cd2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 shutdown_connections 2024-04-25T10:18:50.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.366+0000 7ff3f1600640 1 -- 172.21.15.165:0/1233757321 wait complete. 2024-04-25T10:18:50.388 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:18:50.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.587+0000 7fd4d7200640 1 Processor -- start 2024-04-25T10:18:50.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.587+0000 7fd4d7200640 1 -- start start 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d8071700 0x7fd4d80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd4d80cd860 0x7fd4d815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d7200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4d815bf30 con 0x7fd4d8144440 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d815c860 con 0x7fd4d80cd860 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d815c210 con 0x7fd4d8071700 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d5e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd4d8144440 0x7fd4d8070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44232/0 (socket says 172.21.15.165:44232) 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d5e00640 1 -- 172.21.15.165:0/393417093 learned_addr learned my addr 172.21.15.165:0/393417093 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:50.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d6800640 1 --2- 172.21.15.165:0/393417093 >> v2:172.21.15.165:3300/0 conn(0x7fd4d80cd860 0x7fd4d815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.589+0000 7fd4d5400640 1 --2- 172.21.15.165:0/393417093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d8071700 0x7fd4d80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.590+0000 7fd4d6800640 1 -- 172.21.15.165:0/393417093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d8071700 msgr2=0x7fd4d80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:50.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.590+0000 7fd4d6800640 1 --2- 172.21.15.165:0/393417093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d8071700 0x7fd4d80cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.590+0000 7fd4d6800640 1 -- 172.21.15.165:0/393417093 >> v1:172.21.15.165:6789/0 conn(0x7fd4d8144440 legacy=0x7fd4d8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.590+0000 7fd4d6800640 1 -- 172.21.15.165:0/393417093 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d815c4f0 con 0x7fd4d80cd860 2024-04-25T10:18:50.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.590+0000 7fd4d4a00640 1 -- 172.21.15.165:0/393417093 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1458161316 0 0) 0x7fd4d815bf30 con 0x7fd4d8144440 2024-04-25T10:18:50.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.591+0000 7fd4d6800640 1 --2- 172.21.15.165:0/393417093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d80cd860 0x7fd4d815ccf0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009d90 tx=0x7fd4d002a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3feb762bf7be46 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.591+0000 7fd4d4a00640 1 -- 172.21.15.165:0/393417093 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0007740 con 0x7fd4d80cd860 2024-04-25T10:18:50.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.591+0000 7fd4d4a00640 1 -- 172.21.15.165:0/393417093 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d00078e0 con 0x7fd4d80cd860 2024-04-25T10:18:50.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.591+0000 7fd4d4a00640 1 -- 172.21.15.165:0/393417093 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0007cb0 con 0x7fd4d80cd860 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 -- 172.21.15.165:0/393417093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d80cd860 msgr2=0x7fd4d815ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 --2- 172.21.15.165:0/393417093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d80cd860 0x7fd4d815ccf0 secure :-1 s=READY pgs=2386 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009d90 tx=0x7fd4d002a6f0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 -- 172.21.15.165:0/393417093 shutdown_connections 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 --2- 172.21.15.165:0/393417093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d80cd860 0x7fd4d815ccf0 unknown :-1 s=CLOSED pgs=2386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 --2- 172.21.15.165:0/393417093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d8071700 0x7fd4d80cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 -- 172.21.15.165:0/393417093 >> 172.21.15.165:0/393417093 conn(0x7fd4d806abe0 msgr2=0x7fd4d806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 -- 172.21.15.165:0/393417093 shutdown_connections 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.592+0000 7fd4d7200640 1 -- 172.21.15.165:0/393417093 wait complete. 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.594+0000 7fd4d7200640 1 Processor -- start 2024-04-25T10:18:50.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.594+0000 7fd4d7200640 1 -- start start 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d8071700 0x7fd4d80a9b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d80cd860 0x7fd4d80cb890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 0x7fd4d81f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d80cde70 con 0x7fd4d8144440 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d80cdcf0 con 0x7fd4d8071700 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d815d230 con 0x7fd4d80cd860 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 0x7fd4d81f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 0x7fd4d81f36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41838/0 (socket says 172.21.15.165:41838) 2024-04-25T10:18:50.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d6800640 1 -- 172.21.15.165:0/548830728 learned_addr learned my addr 172.21.15.165:0/548830728 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:50.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d5400640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d80cd860 0x7fd4d80cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.595+0000 7fd4d5e00640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d8071700 0x7fd4d80a9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d6800640 1 -- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d80cd860 msgr2=0x7fd4d80cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d6800640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d80cd860 0x7fd4d80cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d6800640 1 -- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d8071700 msgr2=0x7fd4d80a9b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d6800640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d8071700 0x7fd4d80a9b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d6800640 1 -- 172.21.15.165:0/548830728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d80cc0c0 con 0x7fd4d8144440 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d6800640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 0x7fd4d81f36a0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009eb0 tx=0x7fd4d0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d0040070 con 0x7fd4d8144440 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d0031bc0 con 0x7fd4d8144440 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.596+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d80cc3a0 con 0x7fd4d8144440 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.597+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d003c420 con 0x7fd4d8144440 2024-04-25T10:18:50.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.597+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d80ccc40 con 0x7fd4d8144440 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.598+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d80cc570 con 0x7fd4d8144440 2024-04-25T10:18:50.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.599+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd4d0039040 con 0x7fd4d8144440 2024-04-25T10:18:50.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.600+0000 7fd4ce000640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4b806b4d0 0x7fd4b806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:50.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.600+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fd4d00ab550 con 0x7fd4d8144440 2024-04-25T10:18:50.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.600+0000 7fd4d5e00640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4b806b4d0 0x7fd4b806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:50.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.601+0000 7fd4d5e00640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4b806b4d0 0x7fd4b806d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000f60 tx=0x7fd4c8007c00 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:50.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.605+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd4d007a530 con 0x7fd4d8144440 2024-04-25T10:18:50.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:50.873+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7fd4d8073990 con 0x7fd4d8144440 2024-04-25T10:18:51.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.500+0000 7fd4ce000640 1 -- 172.21.15.165:0/548830728 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v772) v1 ==== 136+0+0 (secure 0 0 0) 0x7fd4d0079e30 con 0x7fd4d8144440 2024-04-25T10:18:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4b806b4d0 msgr2=0x7fd4b806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4b806b4d0 0x7fd4b806d990 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8000f60 tx=0x7fd4c8007c00 comp rx=0 tx=0).stop 2024-04-25T10:18:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 msgr2=0x7fd4d81f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 0x7fd4d81f36a0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fd4d0009eb0 tx=0x7fd4d0004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 shutdown_connections 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd4b806b4d0 0x7fd4b806d990 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd4d8144440 0x7fd4d81f36a0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd4d80cd860 0x7fd4d80cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 --2- 172.21.15.165:0/548830728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd4d8071700 0x7fd4d80a9b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 >> 172.21.15.165:0/548830728 conn(0x7fd4d806abe0 msgr2=0x7fd4d80c5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 shutdown_connections 2024-04-25T10:18:51.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.503+0000 7fd4d7200640 1 -- 172.21.15.165:0/548830728 wait complete. 2024-04-25T10:18:51.526 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:18:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.729+0000 7fc778c00640 1 Processor -- start 2024-04-25T10:18:51.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.729+0000 7fc778c00640 1 -- start start 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc778c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc77415dd30 0x7fc7740cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc778c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc77406bff0 0x7fc7740b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc778c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77415e110 con 0x7fc7740b5970 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc778c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77406d000 con 0x7fc77406bff0 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc778c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740b2fd0 con 0x7fc77415dd30 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc76aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc77406bff0 0x7fc7740b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc76aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc77406bff0 0x7fc7740b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41854/0 (socket says 172.21.15.165:41854) 2024-04-25T10:18:51.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.731+0000 7fc76aa00640 1 -- 172.21.15.165:0/2129714438 learned_addr learned my addr 172.21.15.165:0/2129714438 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:51.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc773400640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc77415dd30 0x7fc7740cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc76aa00640 1 -- 172.21.15.165:0/2129714438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc77415dd30 msgr2=0x7fc7740cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc76aa00640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc77415dd30 0x7fc7740cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc76aa00640 1 -- 172.21.15.165:0/2129714438 >> v1:172.21.15.165:6789/0 conn(0x7fc7740b5970 legacy=0x7fc774161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc772a00640 1 -- 172.21.15.165:0/2129714438 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147997451 0 0) 0x7fc77415e110 con 0x7fc7740b5970 2024-04-25T10:18:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc76aa00640 1 -- 172.21.15.165:0/2129714438 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77406c480 con 0x7fc77406bff0 2024-04-25T10:18:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.732+0000 7fc773400640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc77415dd30 0x7fc7740cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:51.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.733+0000 7fc76aa00640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77406bff0 0x7fc7740b5430 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89200bb7096e2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.733+0000 7fc772a00640 1 -- 172.21.15.165:0/2129714438 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c037070 con 0x7fc77406bff0 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.733+0000 7fc772a00640 1 -- 172.21.15.165:0/2129714438 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc75c032440 con 0x7fc77406bff0 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.733+0000 7fc772a00640 1 -- 172.21.15.165:0/2129714438 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc75c032810 con 0x7fc77406bff0 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 -- 172.21.15.165:0/2129714438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77406bff0 msgr2=0x7fc7740b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77406bff0 0x7fc7740b5430 secure :-1 s=READY pgs=2389 cs=0 l=1 rev1=1 crypto rx=0x7fc75c009870 tx=0x7fc75c02a270 comp rx=0 tx=0).stop 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 -- 172.21.15.165:0/2129714438 shutdown_connections 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77406bff0 0x7fc7740b5430 unknown :-1 s=CLOSED pgs=2389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 --2- 172.21.15.165:0/2129714438 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc77415dd30 0x7fc7740cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 -- 172.21.15.165:0/2129714438 >> 172.21.15.165:0/2129714438 conn(0x7fc7740d0610 msgr2=0x7fc7740d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 -- 172.21.15.165:0/2129714438 shutdown_connections 2024-04-25T10:18:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.734+0000 7fc778c00640 1 -- 172.21.15.165:0/2129714438 wait complete. 2024-04-25T10:18:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.736+0000 7fc778c00640 1 Processor -- start 2024-04-25T10:18:51.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.736+0000 7fc778c00640 1 -- start start 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc778c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7740b5970 0x7fc7741f3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc778c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77415dd30 0x7fc7740b4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc778c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 0x7fc774159910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc778c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77406cd20 con 0x7fc7740aad10 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc778c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77406ca40 con 0x7fc77415dd30 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc778c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7740cf110 con 0x7fc7740b5970 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 0x7fc774159910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 0x7fc774159910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41862/0 (socket says 172.21.15.165:41862) 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc773e00640 1 -- 172.21.15.165:0/1234313869 learned_addr learned my addr 172.21.15.165:0/1234313869 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc76aa00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77415dd30 0x7fc7740b4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc773e00640 1 -- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7740b5970 msgr2=0x7fc7741f3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.737+0000 7fc773400640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7740b5970 0x7fc7741f3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc773e00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7740b5970 0x7fc7741f3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc773e00640 1 -- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77415dd30 msgr2=0x7fc7740b4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc773e00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77415dd30 0x7fc7740b4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc773e00640 1 -- 172.21.15.165:0/1234313869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7740cb900 con 0x7fc7740aad10 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc76aa00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77415dd30 0x7fc7740b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc773400640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7740b5970 0x7fc7741f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc773e00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 0x7fc774159910 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c760 tx=0x7fc76c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c003da0 con 0x7fc7740aad10 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c005070 con 0x7fc7740aad10 2024-04-25T10:18:51.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005460 con 0x7fc7740aad10 2024-04-25T10:18:51.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7740cbbe0 con 0x7fc7740aad10 2024-04-25T10:18:51.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.738+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7740cc1a0 con 0x7fc7740aad10 2024-04-25T10:18:51.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.740+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc774073990 con 0x7fc7740aad10 2024-04-25T10:18:51.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.740+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc76c005600 con 0x7fc7740aad10 2024-04-25T10:18:51.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.741+0000 7fc76be00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc75006b4d0 0x7fc75006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:51.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.741+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fc76c089fe0 con 0x7fc7740aad10 2024-04-25T10:18:51.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.742+0000 7fc773400640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc75006b4d0 0x7fc75006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:51.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.742+0000 7fc773400640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc75006b4d0 0x7fc75006d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc764006fd0 tx=0x7fc764008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:51.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.746+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc76c059000 con 0x7fc7740aad10 2024-04-25T10:18:51.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.939+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc7740b55a0 con 0x7fc7740aad10 2024-04-25T10:18:51.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.945+0000 7fc76be00640 1 -- 172.21.15.165:0/1234313869 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc76c058900 con 0x7fc7740aad10 2024-04-25T10:18:51.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc75006b4d0 msgr2=0x7fc75006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc75006b4d0 0x7fc75006d990 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fc764006fd0 tx=0x7fc764008040 comp rx=0 tx=0).stop 2024-04-25T10:18:51.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 msgr2=0x7fc774159910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:51.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 0x7fc774159910 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c760 tx=0x7fc76c00cc20 comp rx=0 tx=0).stop 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 shutdown_connections 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc75006b4d0 0x7fc75006d990 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7740aad10 0x7fc774159910 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77415dd30 0x7fc7740b4890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 --2- 172.21.15.165:0/1234313869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7740b5970 0x7fc7741f3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 >> 172.21.15.165:0/1234313869 conn(0x7fc7740d0610 msgr2=0x7fc7740b6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 shutdown_connections 2024-04-25T10:18:51.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:51.948+0000 7fc778c00640 1 -- 172.21.15.165:0/1234313869 wait complete. 2024-04-25T10:18:51.970 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:18:51.970 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:52.078 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:18:52.184 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:18:52.184 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:18:52.184 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:52.241 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:52.241 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:18:52.241 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:52.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:52.298 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:18:52.298 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:52.358 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:18:52.358 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:18:52.359 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:18:52.420 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:18:52.420 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:18:52.420 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:18:52.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.628+0000 7f73ff200640 1 Processor -- start 2024-04-25T10:18:52.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.628+0000 7f73ff200640 1 -- start start 2024-04-25T10:18:52.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.629+0000 7f73ff200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7400143e60 0x7f740006e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.629+0000 7f73ff200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f740006e6a0 0x7f74000cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.629+0000 7f73ff200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f740015bb50 con 0x7f74000cc840 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.629+0000 7f73ff200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7400144240 con 0x7f7400143e60 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.629+0000 7f73ff200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f740015e220 con 0x7f740006e6a0 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7400143e60 0x7f740006e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f7400143e60 0x7f740006e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41870/0 (socket says 172.21.15.165:41870) 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 -- 172.21.15.165:0/1502623361 learned_addr learned my addr 172.21.15.165:0/1502623361 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:52.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 -- 172.21.15.165:0/1502623361 >> v2:172.21.15.172:3300/0 conn(0x7f740006e6a0 msgr2=0x7f74000cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:52.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 --2- 172.21.15.165:0/1502623361 >> v2:172.21.15.172:3300/0 conn(0x7f740006e6a0 0x7f74000cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 -- 172.21.15.165:0/1502623361 >> v1:172.21.15.172:6789/0 conn(0x7f74000cc840 legacy=0x7f7400070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:52.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.630+0000 7f73fde00640 1 -- 172.21.15.165:0/1502623361 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740015e500 con 0x7f7400143e60 2024-04-25T10:18:52.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.631+0000 7f73fde00640 1 --2- 172.21.15.165:0/1502623361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7400143e60 0x7f740006e160 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f740006b970 tx=0x7f73f002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e7b0beec6d2d36 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.632+0000 7f73fd400640 1 -- 172.21.15.165:0/1502623361 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0037070 con 0x7f7400143e60 2024-04-25T10:18:52.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.632+0000 7f73fd400640 1 -- 172.21.15.165:0/1502623361 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f0032440 con 0x7f7400143e60 2024-04-25T10:18:52.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.632+0000 7f73fd400640 1 -- 172.21.15.165:0/1502623361 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0032810 con 0x7f7400143e60 2024-04-25T10:18:52.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 -- 172.21.15.165:0/1502623361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7400143e60 msgr2=0x7f740006e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 --2- 172.21.15.165:0/1502623361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7400143e60 0x7f740006e160 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7f740006b970 tx=0x7f73f002a230 comp rx=0 tx=0).stop 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 -- 172.21.15.165:0/1502623361 shutdown_connections 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 --2- 172.21.15.165:0/1502623361 >> v2:172.21.15.172:3300/0 conn(0x7f740006e6a0 0x7f74000cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 --2- 172.21.15.165:0/1502623361 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7400143e60 0x7f740006e160 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 -- 172.21.15.165:0/1502623361 >> 172.21.15.165:0/1502623361 conn(0x7f740006abe0 msgr2=0x7f740006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 -- 172.21.15.165:0/1502623361 shutdown_connections 2024-04-25T10:18:52.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.633+0000 7f73ff200640 1 -- 172.21.15.165:0/1502623361 wait complete. 2024-04-25T10:18:52.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.635+0000 7f73ff200640 1 Processor -- start 2024-04-25T10:18:52.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.635+0000 7f73ff200640 1 -- start start 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.636+0000 7f73ff200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f740006e6a0 0x7f740015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.636+0000 7f73ff200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 0x7f740015d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.636+0000 7f73ff200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7400143e60 0x7f740006d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.636+0000 7f73ff200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74000cccd0 con 0x7f74000cc840 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.636+0000 7f73ff200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f740015eac0 con 0x7f7400143e60 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.636+0000 7f73ff200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74000cce50 con 0x7f740006e6a0 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 0x7f740015d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 0x7f740015d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41886/0 (socket says 172.21.15.165:41886) 2024-04-25T10:18:52.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 -- 172.21.15.165:0/676410280 learned_addr learned my addr 172.21.15.165:0/676410280 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 -- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f740006e6a0 msgr2=0x7f740015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f740006e6a0 0x7f740015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 -- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7400143e60 msgr2=0x7f740006d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73fe800640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7400143e60 0x7f740006d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7400143e60 0x7f740006d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 -- 172.21.15.165:0/676410280 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f740015c580 con 0x7f74000cc840 2024-04-25T10:18:52.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.637+0000 7f73f5400640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 0x7f740015d620 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b4a0 tx=0x7f73e800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.638+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8014070 con 0x7f74000cc840 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.638+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e8010610 con 0x7f74000cc840 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.638+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e8010a00 con 0x7f74000cc840 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.638+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f740015c860 con 0x7f74000cc840 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.638+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f740015ce20 con 0x7f74000cc840 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.639+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7400073990 con 0x7f74000cc840 2024-04-25T10:18:52.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.640+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73e8017070 con 0x7f74000cc840 2024-04-25T10:18:52.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.641+0000 7f73f6a00640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73dc06b500 0x7f73dc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.641+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f73e8089ab0 con 0x7f74000cc840 2024-04-25T10:18:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.641+0000 7f73fde00640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73dc06b500 0x7f73dc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:52.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.642+0000 7f73fde00640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73dc06b500 0x7f73dc06d9c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f73f0034000 tx=0x7f73f002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:52.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.645+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73e8058aa0 con 0x7f74000cc840 2024-04-25T10:18:52.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.878+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74000b1ee0 con 0x7f74000cc840 2024-04-25T10:18:52.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.879+0000 7f73f6a00640 1 -- 172.21.15.165:0/676410280 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 115 v115) v1 ==== 95+0+3716 (secure 0 0 0) 0x7f73e80583a0 con 0x7f74000cc840 2024-04-25T10:18:52.882 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:52.882 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":115,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6047,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/2014738290","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2014738290},{"type":"v1","addr":"172.21.15.165:6837","nonce":2014738290}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6056,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/4013458675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":4013458675},{"type":"v1","addr":"172.21.15.172:6835","nonce":4013458675}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":108},{"gid":6727,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3135619372","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3135619372},{"type":"v1","addr":"172.21.15.165:6836","nonce":3135619372}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":107},{"gid":6877,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3611209664","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3611209664},{"type":"v1","addr":"172.21.15.172:6834","nonce":3611209664}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":115}],"filesystems":[]} 2024-04-25T10:18:52.882 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 115 2024-04-25T10:18:52.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73dc06b500 msgr2=0x7f73dc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73dc06b500 0x7f73dc06d9c0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f73f0034000 tx=0x7f73f002c000 comp rx=0 tx=0).stop 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 msgr2=0x7f740015d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 0x7f740015d620 secure :-1 s=READY pgs=2392 cs=0 l=1 rev1=1 crypto rx=0x7f73e800b4a0 tx=0x7f73e800b960 comp rx=0 tx=0).stop 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 shutdown_connections 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73dc06b500 0x7f73dc06d9c0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7400143e60 0x7f740006d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74000cc840 0x7f740015d620 unknown :-1 s=CLOSED pgs=2392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 --2- 172.21.15.165:0/676410280 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f740006e6a0 0x7f740015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.881+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 >> 172.21.15.165:0/676410280 conn(0x7f740006abe0 msgr2=0x7f74000b2d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.882+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 shutdown_connections 2024-04-25T10:18:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:52.882+0000 7f73ff200640 1 -- 172.21.15.165:0/676410280 wait complete. 2024-04-25T10:18:52.904 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:18:52.904 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:18:52.905 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:18:52.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:52.905 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:18:52.905 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:18:52.905 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:18:52.905 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:52.905 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:18:52.905 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:18:52.906 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:18:52.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:52.906 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:18:52.906 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:18:52.906 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:18:52.906 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:18:52.923 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:18:52.923 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:18:52.924 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:18:52.925 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:18:52.926 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:18:52.926 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:18:52.927 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:18:52.927 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:18:52.973 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:18:52.974 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:52.975 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:18:52.975 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:52.979 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:18:52.980 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:52.982 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:18:52.982 DEBUG:teuthology.parallel:result is None 2024-04-25T10:18:52.982 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:18:53.120 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:18:53.121 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:18:53.132 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:18:53.133 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:18:53.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.159+0000 7fbdd3600640 1 Processor -- start 2024-04-25T10:18:53.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.160+0000 7fbdd3600640 1 -- start start 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdd3600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fbdcc06ea90 0x7fbdcc071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdd3600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc162ba0 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdd3600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdcc162e80 con 0x7fbdcc161fc0 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdd3600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdcc165be0 con 0x7fbdcc06ea90 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdd1600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fbdcc06d2d0 0x7fbdcc166070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44284/0 (socket says 172.21.15.165:44284) 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdd1600640 1 -- 172.21.15.165:0/2727336241 learned_addr learned my addr 172.21.15.165:0/2727336241 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.161+0000 7fbdcbe00640 1 --2- 172.21.15.165:0/2727336241 >> v2:172.21.15.172:3300/0 conn(0x7fbdcc06ea90 0x7fbdcc071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 712552829 0 0) 0x7fbdcc162e80 con 0x7fbdcc161fc0 2024-04-25T10:18:53.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdb4003520 con 0x7fbdcc161fc0 2024-04-25T10:18:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148977748 0 0) 0x7fbdcc162ba0 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdcc162e80 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcbe00640 1 -- 172.21.15.165:0/2727336241 >> v1:172.21.15.172:6789/0 conn(0x7fbdcc161fc0 legacy=0x7fbdcc1623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcbe00640 1 -- 172.21.15.165:0/2727336241 >> v1:172.21.15.165:6789/0 conn(0x7fbdcc06d2d0 legacy=0x7fbdcc166070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.162+0000 7fbdcbe00640 1 -- 172.21.15.165:0/2727336241 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc163160 con 0x7fbdcc06ea90 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.163+0000 7fbdcbe00640 1 --2- 172.21.15.165:0/2727336241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc071480 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097a0 tx=0x7fbdb802a1a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2d53fa3d1381609 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8037070 con 0x7fbdcc06ea90 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8032440 con 0x7fbdcc06ea90 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdcb400640 1 -- 172.21.15.165:0/2727336241 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8032810 con 0x7fbdcc06ea90 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 -- 172.21.15.165:0/2727336241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 msgr2=0x7fbdcc071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2727336241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc071480 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fbdb80097a0 tx=0x7fbdb802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 -- 172.21.15.165:0/2727336241 shutdown_connections 2024-04-25T10:18:53.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2727336241 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc071480 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 -- 172.21.15.165:0/2727336241 >> 172.21.15.165:0/2727336241 conn(0x7fbdcc06abe0 msgr2=0x7fbdcc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 -- 172.21.15.165:0/2727336241 shutdown_connections 2024-04-25T10:18:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.164+0000 7fbdd3600640 1 -- 172.21.15.165:0/2727336241 wait complete. 2024-04-25T10:18:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.166+0000 7fbdd3600640 1 Processor -- start 2024-04-25T10:18:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.166+0000 7fbdd3600640 1 -- start start 2024-04-25T10:18:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd3600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 0x7fbdcc14fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd3600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc14c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdcc161fc0 0x7fbdcc15f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd3600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc071b40 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd3600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc0719c0 con 0x7fbdcc06ea90 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdcc06ef40 con 0x7fbdcc161fc0 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 0x7fbdcc14fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.167+0000 7fbdd0c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 0x7fbdcc14fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41912/0 (socket says 172.21.15.165:41912) 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 -- 172.21.15.165:0/2201837826 learned_addr learned my addr 172.21.15.165:0/2201837826 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdcbe00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc14c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd1600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdcc161fc0 0x7fbdcc15f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 -- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdcc161fc0 msgr2=0x7fbdcc15f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdcc161fc0 0x7fbdcc15f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 -- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 msgr2=0x7fbdcc14c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:53.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc14c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 -- 172.21.15.165:0/2201837826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdcc0aff70 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd1600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdcc161fc0 0x7fbdcc15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.168+0000 7fbdd0c00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 0x7fbdcc14fa40 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c4d0 tx=0x7fbdc000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.169+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc00052d0 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.169+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdc0005470 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.169+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdcc0b0250 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.169+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdc0012870 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.169+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdcc0b0810 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.170+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdcc073a00 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.171+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fbdc0012a10 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.172+0000 7fbdc8c00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdac06b4d0 0x7fbdac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.172+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fbdc0089c70 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.173+0000 7fbdcbe00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdac06b4d0 0x7fbdac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.173+0000 7fbdcbe00640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdac06b4d0 0x7fbdac06d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8034000 tx=0x7fbdb802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:53.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.176+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fbdc0058d10 con 0x7fbdcc06d2d0 2024-04-25T10:18:53.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:53.382+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbdcc0ae4f0 con 0x7fbdcc06d2d0 2024-04-25T10:18:54.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.539+0000 7fbdc8c00640 1 -- 172.21.15.165:0/2201837826 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v167) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbdc0058610 con 0x7fbdcc06d2d0 2024-04-25T10:18:54.543 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdac06b4d0 msgr2=0x7fbdac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdac06b4d0 0x7fbdac06d990 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fbdb8034000 tx=0x7fbdb802e000 comp rx=0 tx=0).stop 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 msgr2=0x7fbdcc14fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 0x7fbdcc14fa40 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7fbdc000c4d0 tx=0x7fbdc000c990 comp rx=0 tx=0).stop 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 shutdown_connections 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fbdac06b4d0 0x7fbdac06d990 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fbdcc161fc0 0x7fbdcc15f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fbdcc06ea90 0x7fbdcc14c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 --2- 172.21.15.165:0/2201837826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fbdcc06d2d0 0x7fbdcc14fa40 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 >> 172.21.15.165:0/2201837826 conn(0x7fbdcc06abe0 msgr2=0x7fbdcc164d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.544+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 shutdown_connections 2024-04-25T10:18:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.545+0000 7fbdd3600640 1 -- 172.21.15.165:0/2201837826 wait complete. 2024-04-25T10:18:54.570 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:18:54.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.762+0000 7f75ffc00640 1 Processor -- start 2024-04-25T10:18:54.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.762+0000 7f75ffc00640 1 -- start start 2024-04-25T10:18:54.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75ffc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f75f80cc930 0x7f75f806d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75ffc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f815c150 con 0x7f75f8144d40 2024-04-25T10:18:54.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75ffc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75f80cd4b0 con 0x7f75f80ce950 2024-04-25T10:18:54.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75ffc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75f806d850 con 0x7f75f80cc930 2024-04-25T10:18:54.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75fd200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f75f8144d40 0x7f75f815c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44294/0 (socket says 172.21.15.165:44294) 2024-04-25T10:18:54.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75fd200640 1 -- 172.21.15.165:0/197205369 learned_addr learned my addr 172.21.15.165:0/197205369 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:54.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75f7e00640 1 --2- 172.21.15.165:0/197205369 >> v2:172.21.15.172:3300/0 conn(0x7f75f80cc930 0x7f75f806d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.764+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170339764 0 0) 0x7f75f815c150 con 0x7f75f8144d40 2024-04-25T10:18:54.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75e0003520 con 0x7f75f8144d40 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2163822764 0 0) 0x7f75e0003520 con 0x7f75f8144d40 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 >> v2:172.21.15.172:3300/0 conn(0x7f75f80cc930 msgr2=0x7f75f806d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 --2- 172.21.15.165:0/197205369 >> v2:172.21.15.172:3300/0 conn(0x7f75f80cc930 0x7f75f806d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 >> v1:172.21.15.172:6789/0 conn(0x7f75f80ce950 legacy=0x7f75f80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80ccef0 con 0x7f75f8144d40 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f75ec001670 con 0x7f75f8144d40 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 >> v1:172.21.15.165:6789/0 conn(0x7f75f8144d40 legacy=0x7f75f815c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00039b0 0x7f75e0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e000c180 con 0x7f75e00052e0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e000c330 con 0x7f75e0004800 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75e000c530 con 0x7f75e00039b0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.765+0000 7f75fdc00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75fd200640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00039b0 0x7f75e0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75f7e00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75fdc00640 1 -- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00039b0 msgr2=0x7f75e0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75fdc00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00039b0 0x7f75e0003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75fdc00640 1 -- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e0004800 msgr2=0x7f75e0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75fdc00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75e0004800 0x7f75e0004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.766+0000 7f75fdc00640 1 -- 172.21.15.165:0/197205369 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0003520 con 0x7f75e00052e0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.767+0000 7f75fdc00640 1 -- 172.21.15.165:0/197205369 reap_dead start 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.767+0000 7f75fdc00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f75f0001e20 tx=0x7f75e0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27789bcba096d71c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.767+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0001a70 con 0x7f75e00052e0 2024-04-25T10:18:54.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.767+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75f002ae40 con 0x7f75e00052e0 2024-04-25T10:18:54.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.767+0000 7f75f7400640 1 -- 172.21.15.165:0/197205369 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f0003bb0 con 0x7f75e00052e0 2024-04-25T10:18:54.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 -- 172.21.15.165:0/197205369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75e00052e0 msgr2=0x7f75e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f75f0001e20 tx=0x7f75e0005830 comp rx=0 tx=0).stop 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 -- 172.21.15.165:0/197205369 shutdown_connections 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 --2- 172.21.15.165:0/197205369 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75e00052e0 0x7f75e000b9c0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 -- 172.21.15.165:0/197205369 >> 172.21.15.165:0/197205369 conn(0x7f75f806abe0 msgr2=0x7f75f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 -- 172.21.15.165:0/197205369 shutdown_connections 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.768+0000 7f75ffc00640 1 -- 172.21.15.165:0/197205369 wait complete. 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.770+0000 7f75ffc00640 1 Processor -- start 2024-04-25T10:18:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.770+0000 7f75ffc00640 1 -- start start 2024-04-25T10:18:54.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75ffc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00052e0 0x7f75f80b0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75ffc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 0x7f75f80b3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75ffc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f8139f20 0x7f75f81f3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75ffc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f806d310 con 0x7f75f80afd70 2024-04-25T10:18:54.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75ffc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75f815c3f0 con 0x7f75f8139f20 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75ffc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75f806d490 con 0x7f75e00052e0 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 0x7f75f80b3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75f7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 0x7f75f80b3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41928/0 (socket says 172.21.15.165:41928) 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.771+0000 7f75f7e00640 1 -- 172.21.15.165:0/3171059347 learned_addr learned my addr 172.21.15.165:0/3171059347 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75fd200640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00052e0 0x7f75f80b0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75fdc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f8139f20 0x7f75f81f3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f7e00640 1 -- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00052e0 msgr2=0x7f75f80b0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f7e00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00052e0 0x7f75f80b0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f7e00640 1 -- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f8139f20 msgr2=0x7f75f81f3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f7e00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f8139f20 0x7f75f81f3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f7e00640 1 -- 172.21.15.165:0/3171059347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f80b2760 con 0x7f75f80afd70 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75fdc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f8139f20 0x7f75f81f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f7e00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 0x7f75f80b3ae0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f75e4010860 tx=0x7f75e4010d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e400f040 con 0x7f75f80afd70 2024-04-25T10:18:54.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e400eb60 con 0x7f75f80afd70 2024-04-25T10:18:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75f80b2a40 con 0x7f75f80afd70 2024-04-25T10:18:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e401c400 con 0x7f75f80afd70 2024-04-25T10:18:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.772+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75f80b3000 con 0x7f75f80afd70 2024-04-25T10:18:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.774+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75f8073a00 con 0x7f75f80afd70 2024-04-25T10:18:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.775+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f75e401c5a0 con 0x7f75f80afd70 2024-04-25T10:18:54.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.776+0000 7f75f4c00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:54.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.776+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f75e408a7a0 con 0x7f75f80afd70 2024-04-25T10:18:54.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.776+0000 7f75fd200640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:54.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.777+0000 7f75fd200640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75d806b4d0 0x7f75d806d990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f75ec000ff0 tx=0x7f75ec009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:54.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:54.781+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f75e4018030 con 0x7f75f80afd70 2024-04-25T10:18:55.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.046+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f75f815d5e0 con 0x7f75f80afd70 2024-04-25T10:18:55.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.047+0000 7f75f4c00640 1 -- 172.21.15.165:0/3171059347 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v23) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f75e4059740 con 0x7f75f80afd70 2024-04-25T10:18:55.050 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:55.050 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:18:55.051 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.052 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.053 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.054 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.055 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.056 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.057 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:18:55.058 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:18:55.059 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:18:55.060 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:18:55.061 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:18:55.062 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:18:55.063 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75d806b4d0 msgr2=0x7f75d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75d806b4d0 0x7f75d806d990 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f75ec000ff0 tx=0x7f75ec009040 comp rx=0 tx=0).stop 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 msgr2=0x7f75f80b3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 0x7f75f80b3ae0 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f75e4010860 tx=0x7f75e4010d20 comp rx=0 tx=0).stop 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 shutdown_connections 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f75d806b4d0 0x7f75d806d990 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f75f8139f20 0x7f75f81f3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f75f80afd70 0x7f75f80b3ae0 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 --2- 172.21.15.165:0/3171059347 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f75e00052e0 0x7f75f80b0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 >> 172.21.15.165:0/3171059347 conn(0x7f75f806abe0 msgr2=0x7f75f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 shutdown_connections 2024-04-25T10:18:55.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.050+0000 7f75ffc00640 1 -- 172.21.15.165:0/3171059347 wait complete. 2024-04-25T10:18:55.072 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:55.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.275+0000 7f172e000640 1 Processor -- start 2024-04-25T10:18:55.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.275+0000 7f172e000640 1 -- start start 2024-04-25T10:18:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.277+0000 7f172e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1728144750 0x7f172806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.277+0000 7f172e000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1728144b30 con 0x7f17280cf030 2024-04-25T10:18:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.277+0000 7f172e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17280712d0 con 0x7f172806eb10 2024-04-25T10:18:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.277+0000 7f172e000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17280715b0 con 0x7f1728144750 2024-04-25T10:18:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.277+0000 7f1727e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f17280cf030 0x7f172806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44304/0 (socket says 172.21.15.165:44304) 2024-04-25T10:18:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.277+0000 7f1727e00640 1 -- 172.21.15.165:0/35462108 learned_addr learned my addr 172.21.15.165:0/35462108 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2795479209 0 0) 0x7f1728144b30 con 0x7f17280cf030 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1727400640 1 --2- 172.21.15.165:0/35462108 >> v2:172.21.15.172:3300/0 conn(0x7f1728144750 0x7f172806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f170c003520 con 0x7f17280cf030 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3293736534 0 0) 0x7f170c003520 con 0x7f17280cf030 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 >> v2:172.21.15.172:3300/0 conn(0x7f1728144750 msgr2=0x7f172806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 --2- 172.21.15.165:0/35462108 >> v2:172.21.15.172:3300/0 conn(0x7f1728144750 0x7f172806e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 >> v1:172.21.15.172:6789/0 conn(0x7f172806eb10 legacy=0x7f17280ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1728071890 con 0x7f17280cf030 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f17200019f0 con 0x7f17280cf030 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 >> v1:172.21.15.165:6789/0 conn(0x7f17280cf030 legacy=0x7f172806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f170c004800 0x7f170c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c0051c0 0x7f170c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17280715b0 con 0x7f170c004800 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f170c00c110 con 0x7f170c0039b0 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.278+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f170c00c310 con 0x7f170c0051c0 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1726a00640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f170c004800 0x7f170c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1727400640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1726a00640 1 -- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c0051c0 msgr2=0x7f170c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1726a00640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c0051c0 0x7f170c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1726a00640 1 -- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170c0039b0 msgr2=0x7f170c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1726a00640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f170c0039b0 0x7f170c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1726a00640 1 -- 172.21.15.165:0/35462108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f170c003520 con 0x7f170c004800 2024-04-25T10:18:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.279+0000 7f1727e00640 1 -- 172.21.15.165:0/35462108 reap_dead start 2024-04-25T10:18:55.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.280+0000 7f1726a00640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f170c004800 0x7f170c004c80 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f1714001750 tx=0x7f170c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943db94e6fad1fcb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.280+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1714001370 con 0x7f170c004800 2024-04-25T10:18:55.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.280+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17140052d0 con 0x7f170c004800 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.280+0000 7f1726000640 1 -- 172.21.15.165:0/35462108 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171402abc0 con 0x7f170c004800 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 -- 172.21.15.165:0/35462108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f170c004800 msgr2=0x7f170c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f170c004800 0x7f170c004c80 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f1714001750 tx=0x7f170c005830 comp rx=0 tx=0).stop 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 -- 172.21.15.165:0/35462108 shutdown_connections 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 --2- 172.21.15.165:0/35462108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f170c004800 0x7f170c004c80 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 -- 172.21.15.165:0/35462108 >> 172.21.15.165:0/35462108 conn(0x7f172806abe0 msgr2=0x7f172806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 -- 172.21.15.165:0/35462108 shutdown_connections 2024-04-25T10:18:55.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.281+0000 7f172e000640 1 -- 172.21.15.165:0/35462108 wait complete. 2024-04-25T10:18:55.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.283+0000 7f172e000640 1 Processor -- start 2024-04-25T10:18:55.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.283+0000 7f172e000640 1 -- start start 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.284+0000 7f172e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c004800 0x7f17281f4770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.284+0000 7f172e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17280b11d0 0x7f17280b1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.284+0000 7f172e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 0x7f1728130ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.284+0000 7f172e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17281455d0 con 0x7f1728130650 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.284+0000 7f172e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1728145450 con 0x7f17280b11d0 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.284+0000 7f172e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172815c000 con 0x7f170c004800 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 0x7f1728130ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 0x7f1728130ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41960/0 (socket says 172.21.15.165:41960) 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 -- 172.21.15.165:0/2211574067 learned_addr learned my addr 172.21.15.165:0/2211574067 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:55.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1726a00640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17280b11d0 0x7f17280b1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727400640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c004800 0x7f17281f4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 -- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c004800 msgr2=0x7f17281f4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c004800 0x7f17281f4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 -- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17280b11d0 msgr2=0x7f17280b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17280b11d0 0x7f17280b1650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 -- 172.21.15.165:0/2211574067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f172815e000 con 0x7f1728130650 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727400640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c004800 0x7f17281f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.285+0000 7f1727e00640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 0x7f1728130ad0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f172000c770 tx=0x7f172000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.286+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1720004d60 con 0x7f1728130650 2024-04-25T10:18:55.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.286+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f172000a5e0 con 0x7f1728130650 2024-04-25T10:18:55.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.286+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f172000a9b0 con 0x7f1728130650 2024-04-25T10:18:55.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.286+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172815e2e0 con 0x7f1728130650 2024-04-25T10:18:55.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.286+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172815e8a0 con 0x7f1728130650 2024-04-25T10:18:55.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.287+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17280b1b90 con 0x7f1728130650 2024-04-25T10:18:55.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.288+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1720007070 con 0x7f1728130650 2024-04-25T10:18:55.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.289+0000 7f171f400640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f170806b4d0 0x7f170806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.289+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f1720089b30 con 0x7f1728130650 2024-04-25T10:18:55.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.289+0000 7f1727400640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f170806b4d0 0x7f170806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.290+0000 7f1727400640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f170806b4d0 0x7f170806d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f17281316c0 tx=0x7f1718008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.294+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1720058b00 con 0x7f1728130650 2024-04-25T10:18:55.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.493+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f17280af030 con 0x7f1728130650 2024-04-25T10:18:55.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.496+0000 7f171f400640 1 -- 172.21.15.165:0/2211574067 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) v1 ==== 74+0+13210 (secure 0 0 0) 0x7f17280af030 con 0x7f1728130650 2024-04-25T10:18:55.499 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:55.499 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":167,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:18:54.536468+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:18:55.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f170806b4d0 msgr2=0x7f170806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f170806b4d0 0x7f170806d990 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f17281316c0 tx=0x7f1718008040 comp rx=0 tx=0).stop 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 msgr2=0x7f1728130ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 0x7f1728130ad0 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f172000c770 tx=0x7f172000cc30 comp rx=0 tx=0).stop 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 shutdown_connections 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f170806b4d0 0x7f170806d990 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1728130650 0x7f1728130ad0 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.498+0000 7f172e000640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f17280b11d0 0x7f17280b1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.499+0000 7f172e000640 1 --2- 172.21.15.165:0/2211574067 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f170c004800 0x7f17281f4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.499+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 >> 172.21.15.165:0/2211574067 conn(0x7f172806abe0 msgr2=0x7f17281439c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.499+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 shutdown_connections 2024-04-25T10:18:55.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.499+0000 7f172e000640 1 -- 172.21.15.165:0/2211574067 wait complete. 2024-04-25T10:18:55.520 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:18:55.521 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:18:55.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.724+0000 7f5b33e00640 1 Processor -- start 2024-04-25T10:18:55.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.724+0000 7f5b33e00640 1 -- start start 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b33e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 0x7f5b340c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b33e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5b340c5030 0x7f5b3415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b33e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b34145520 con 0x7f5b34145140 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b33e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b34071270 con 0x7f5b340c5030 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b33e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34071550 con 0x7f5b340ceb20 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b32a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5b34145140 0x7f5b340ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:44310/0 (socket says 172.21.15.165:44310) 2024-04-25T10:18:55.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.726+0000 7f5b32a00640 1 -- 172.21.15.165:0/1489629206 learned_addr learned my addr 172.21.15.165:0/1489629206 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131453513 0 0) 0x7f5b34145520 con 0x7f5b34145140 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b33400640 1 --2- 172.21.15.165:0/1489629206 >> v2:172.21.15.172:3300/0 conn(0x7f5b340c5030 0x7f5b3415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b18003520 con 0x7f5b34145140 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b32000640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 0x7f5b340c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3530726473 0 0) 0x7f5b18003520 con 0x7f5b34145140 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 msgr2=0x7f5b340c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 0x7f5b340c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 >> v2:172.21.15.172:3300/0 conn(0x7f5b340c5030 msgr2=0x7f5b3415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 --2- 172.21.15.165:0/1489629206 >> v2:172.21.15.172:3300/0 conn(0x7f5b340c5030 0x7f5b3415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b34071830 con 0x7f5b34145140 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5b240019f0 con 0x7f5b34145140 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 >> v1:172.21.15.165:6789/0 conn(0x7f5b34145140 legacy=0x7f5b340ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b180039b0 0x7f5b18003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 0x7f5b18004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b18004d40 0x7f5b180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34071270 con 0x7f5b180039b0 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34071550 con 0x7f5b18004d40 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1800bb20 con 0x7f5b340ceb20 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b33400640 1 --2- 172.21.15.165:0/1489629206 >> v2:172.21.15.172:3300/0 conn(0x7f5b340c5030 0x7f5b3415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.727+0000 7f5b32a00640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b180039b0 0x7f5b18003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.728+0000 7f5b32a00640 1 -- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 msgr2=0x7f5b18004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.728+0000 7f5b32a00640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b340ceb20 0x7f5b18004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.728+0000 7f5b32a00640 1 -- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b18004d40 msgr2=0x7f5b180051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:55.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.728+0000 7f5b33400640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b18004d40 0x7f5b180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.728+0000 7f5b32a00640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b18004d40 0x7f5b180051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.728+0000 7f5b32a00640 1 -- 172.21.15.165:0/1489629206 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b18003520 con 0x7f5b180039b0 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.729+0000 7f5b32a00640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b180039b0 0x7f5b18003dd0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400a880 tx=0x7f5b18005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a61f46500611361 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.729+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2400a4a0 con 0x7f5b180039b0 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.729+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2402f070 con 0x7f5b180039b0 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.729+0000 7f5b31600640 1 -- 172.21.15.165:0/1489629206 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2402bd00 con 0x7f5b180039b0 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33e00640 1 -- 172.21.15.165:0/1489629206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b180039b0 msgr2=0x7f5b18003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33e00640 1 --2- 172.21.15.165:0/1489629206 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b180039b0 0x7f5b18003dd0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f5b2400a880 tx=0x7f5b18005830 comp rx=0 tx=0).stop 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33400640 1 -- 172.21.15.165:0/1489629206 reap_dead start 2024-04-25T10:18:55.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33e00640 1 -- 172.21.15.165:0/1489629206 shutdown_connections 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33e00640 1 -- 172.21.15.165:0/1489629206 >> 172.21.15.165:0/1489629206 conn(0x7f5b34061cc0 msgr2=0x7f5b3406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33e00640 1 -- 172.21.15.165:0/1489629206 shutdown_connections 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.730+0000 7f5b33e00640 1 -- 172.21.15.165:0/1489629206 wait complete. 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.732+0000 7f5b33e00640 1 Processor -- start 2024-04-25T10:18:55.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.732+0000 7f5b33e00640 1 -- start start 2024-04-25T10:18:55.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b180039b0 0x7f5b3413f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b3415eb90 0x7f5b34131d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 0x7f5b34132710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340c5640 con 0x7f5b34132290 2024-04-25T10:18:55.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340c54c0 con 0x7f5b3415eb90 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b340c4af0 con 0x7f5b180039b0 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 0x7f5b34132710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 0x7f5b34132710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41982/0 (socket says 172.21.15.165:41982) 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.733+0000 7f5b33400640 1 -- 172.21.15.165:0/752292779 learned_addr learned my addr 172.21.15.165:0/752292779 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b32a00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b180039b0 0x7f5b3413f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b32000640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b3415eb90 0x7f5b34131d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33400640 1 -- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b180039b0 msgr2=0x7f5b3413f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33400640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b180039b0 0x7f5b3413f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33400640 1 -- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b3415eb90 msgr2=0x7f5b34131d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33400640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b3415eb90 0x7f5b34131d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33400640 1 -- 172.21.15.165:0/752292779 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340a84a0 con 0x7f5b34132290 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b32000640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b3415eb90 0x7f5b34131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33400640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 0x7f5b34132710 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f5b3406bb90 tx=0x7f5b2c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00fea0 con 0x7f5b34132290 2024-04-25T10:18:55.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c019070 con 0x7f5b34132290 2024-04-25T10:18:55.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3415f010 con 0x7f5b34132290 2024-04-25T10:18:55.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.734+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c004430 con 0x7f5b34132290 2024-04-25T10:18:55.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.735+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b340a8a20 con 0x7f5b34132290 2024-04-25T10:18:55.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.736+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b34073990 con 0x7f5b34132290 2024-04-25T10:18:55.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.737+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b2c012070 con 0x7f5b34132290 2024-04-25T10:18:55.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.737+0000 7f5b2b400640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b1006b500 0x7f5b1006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:55.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.738+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5b2c089a50 con 0x7f5b34132290 2024-04-25T10:18:55.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.738+0000 7f5b32a00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b1006b500 0x7f5b1006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:55.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.739+0000 7f5b32a00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b1006b500 0x7f5b1006d9c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5b24033000 tx=0x7f5b24007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:55.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.742+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5b2c058a80 con 0x7f5b34132290 2024-04-25T10:18:55.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.937+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5b3406af60 con 0x7f5b34132290 2024-04-25T10:18:55.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.938+0000 7f5b2b400640 1 -- 172.21.15.165:0/752292779 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v167) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5b2c058380 con 0x7f5b34132290 2024-04-25T10:18:55.941 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:18:55.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b1006b500 msgr2=0x7f5b1006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b1006b500 0x7f5b1006d9c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f5b24033000 tx=0x7f5b24007000 comp rx=0 tx=0).stop 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 msgr2=0x7f5b34132710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 0x7f5b34132710 secure :-1 s=READY pgs=2408 cs=0 l=1 rev1=1 crypto rx=0x7f5b3406bb90 tx=0x7f5b2c00f670 comp rx=0 tx=0).stop 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 shutdown_connections 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b1006b500 0x7f5b1006d9c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5b34132290 0x7f5b34132710 unknown :-1 s=CLOSED pgs=2408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b3415eb90 0x7f5b34131d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 --2- 172.21.15.165:0/752292779 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b180039b0 0x7f5b3413f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.940+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 >> 172.21.15.165:0/752292779 conn(0x7f5b34061cc0 msgr2=0x7f5b341443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.941+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 shutdown_connections 2024-04-25T10:18:55.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:55.941+0000 7f5b33e00640 1 -- 172.21.15.165:0/752292779 wait complete. 2024-04-25T10:18:55.964 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:18:56.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.166+0000 7fe45f400640 1 Processor -- start 2024-04-25T10:18:56.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.166+0000 7fe45f400640 1 -- start start 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe45815db10 0x7fe458162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe4580b3220 0x7fe4580b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45f400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe4580cf3f0 0x7fe4580cf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45f400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4580b3bc0 con 0x7fe4580b3220 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45f400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe458072280 con 0x7fe4580cf3f0 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe458162770 con 0x7fe45815db10 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe457e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe4580b3220 0x7fe4580b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe457e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe4580b3220 0x7fe4580b3680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41986/0 (socket says 172.21.15.165:41986) 2024-04-25T10:18:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe457e00640 1 -- 172.21.15.165:0/675248867 learned_addr learned my addr 172.21.15.165:0/675248867 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45ca00640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe45815db10 0x7fe458162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.168+0000 7fe45d400640 1 --2- 172.21.15.165:0/675248867 >> v2:172.21.15.172:3300/0 conn(0x7fe4580cf3f0 0x7fe4580cf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.169+0000 7fe457e00640 1 -- 172.21.15.165:0/675248867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe45815db10 msgr2=0x7fe458162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:56.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.169+0000 7fe457e00640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe45815db10 0x7fe458162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.169+0000 7fe457e00640 1 -- 172.21.15.165:0/675248867 >> v2:172.21.15.172:3300/0 conn(0x7fe4580cf3f0 msgr2=0x7fe4580cf880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.169+0000 7fe457e00640 1 --2- 172.21.15.165:0/675248867 >> v2:172.21.15.172:3300/0 conn(0x7fe4580cf3f0 0x7fe4580cf880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.169+0000 7fe457e00640 1 -- 172.21.15.165:0/675248867 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe458071670 con 0x7fe4580b3220 2024-04-25T10:18:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.169+0000 7fe45ca00640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe45815db10 0x7fe458162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:18:56.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.170+0000 7fe457e00640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4580b3220 0x7fe4580b3680 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fe4580b2e80 tx=0x7fe44402a1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd178ece76e66677 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.170+0000 7fe457400640 1 -- 172.21.15.165:0/675248867 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444037070 con 0x7fe4580b3220 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.170+0000 7fe457400640 1 -- 172.21.15.165:0/675248867 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe444032440 con 0x7fe4580b3220 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.170+0000 7fe457400640 1 -- 172.21.15.165:0/675248867 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe444032810 con 0x7fe4580b3220 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 -- 172.21.15.165:0/675248867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4580b3220 msgr2=0x7fe4580b3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4580b3220 0x7fe4580b3680 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7fe4580b2e80 tx=0x7fe44402a1d0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 -- 172.21.15.165:0/675248867 shutdown_connections 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 --2- 172.21.15.165:0/675248867 >> v2:172.21.15.172:3300/0 conn(0x7fe4580cf3f0 0x7fe4580cf880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe4580b3220 0x7fe4580b3680 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 --2- 172.21.15.165:0/675248867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe45815db10 0x7fe458162230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 -- 172.21.15.165:0/675248867 >> 172.21.15.165:0/675248867 conn(0x7fe45806abe0 msgr2=0x7fe45806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 -- 172.21.15.165:0/675248867 shutdown_connections 2024-04-25T10:18:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.171+0000 7fe45f400640 1 -- 172.21.15.165:0/675248867 wait complete. 2024-04-25T10:18:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.173+0000 7fe45f400640 1 Processor -- start 2024-04-25T10:18:56.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.173+0000 7fe45f400640 1 -- start start 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4580b3220 0x7fe4580ada40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4580cf3f0 0x7fe4581f2db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 0x7fe458157e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45f400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458071f10 con 0x7fe45815db10 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe458071c30 con 0x7fe4580b3220 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4580af0e0 con 0x7fe4580cf3f0 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 0x7fe458157e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 0x7fe458157e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41998/0 (socket says 172.21.15.165:41998) 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.174+0000 7fe45d400640 1 -- 172.21.15.165:0/1908032687 learned_addr learned my addr 172.21.15.165:0/1908032687 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45ca00640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4580b3220 0x7fe4580ada40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe457e00640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4580cf3f0 0x7fe4581f2db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45d400640 1 -- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4580cf3f0 msgr2=0x7fe4581f2db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45d400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4580cf3f0 0x7fe4581f2db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45d400640 1 -- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4580b3220 msgr2=0x7fe4580ada40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45d400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4580b3220 0x7fe4580ada40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45d400640 1 -- 172.21.15.165:0/1908032687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4580ae3b0 con 0x7fe45815db10 2024-04-25T10:18:56.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45d400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 0x7fe458157e70 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fe450007e60 tx=0x7fe45000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450014620 con 0x7fe45815db10 2024-04-25T10:18:56.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe450019070 con 0x7fe45815db10 2024-04-25T10:18:56.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe450014a20 con 0x7fe45815db10 2024-04-25T10:18:56.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.175+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4580ae690 con 0x7fe45815db10 2024-04-25T10:18:56.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.176+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4580cc4c0 con 0x7fe45815db10 2024-04-25T10:18:56.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.177+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4580712b0 con 0x7fe45815db10 2024-04-25T10:18:56.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.178+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe450014bc0 con 0x7fe45815db10 2024-04-25T10:18:56.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.179+0000 7fe454c00640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe43806b4d0 0x7fe43806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.179+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fe45008a9a0 con 0x7fe45815db10 2024-04-25T10:18:56.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.179+0000 7fe45ca00640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe43806b4d0 0x7fe43806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.180+0000 7fe45ca00640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe43806b4d0 0x7fe43806d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe44c006fd0 tx=0x7fe44c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.183+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe4580712b0 con 0x7fe45815db10 2024-04-25T10:18:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.382+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe4580cbfb0 con 0x7fe45815db10 2024-04-25T10:18:56.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.383+0000 7fe454c00640 1 -- 172.21.15.165:0/1908032687 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v167) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe450059940 con 0x7fe45815db10 2024-04-25T10:18:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:18:56.386 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:18:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:18:56.386 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:18:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:18:56.387 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:18:56.387 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.385+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe43806b4d0 msgr2=0x7fe43806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.385+0000 7fe45f400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe43806b4d0 0x7fe43806d990 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fe44c006fd0 tx=0x7fe44c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.385+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 msgr2=0x7fe458157e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.385+0000 7fe45f400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 0x7fe458157e70 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7fe450007e60 tx=0x7fe45000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 shutdown_connections 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe43806b4d0 0x7fe43806d990 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe45815db10 0x7fe458157e70 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe4580cf3f0 0x7fe4581f2db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 --2- 172.21.15.165:0/1908032687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe4580b3220 0x7fe4580ada40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 >> 172.21.15.165:0/1908032687 conn(0x7fe45806abe0 msgr2=0x7fe4580af380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 shutdown_connections 2024-04-25T10:18:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.386+0000 7fe45f400640 1 -- 172.21.15.165:0/1908032687 wait complete. 2024-04-25T10:18:56.408 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:18:56.408 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:18:56.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.607+0000 7fc939000640 1 Processor -- start 2024-04-25T10:18:56.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.608+0000 7fc939000640 1 -- start start 2024-04-25T10:18:56.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.609+0000 7fc939000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9340ced80 0x7fc9340d22e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.609+0000 7fc939000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc9340d2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.609+0000 7fc939000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc93415e000 con 0x7fc93415dc20 2024-04-25T10:18:56.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.609+0000 7fc939000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9340d1de0 con 0x7fc9340ced80 2024-04-25T10:18:56.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.609+0000 7fc939000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc934161c60 con 0x7fc934161710 2024-04-25T10:18:56.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.610+0000 7fc932800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc9340d2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.610+0000 7fc92be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc9340ced80 0x7fc9340d22e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.610+0000 7fc932800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc9340d2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:33058/0 (socket says 172.21.15.165:33058) 2024-04-25T10:18:56.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.610+0000 7fc931e00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc93415dc20 0x7fc9340ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:46572/0 (socket says 172.21.15.165:46572) 2024-04-25T10:18:56.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.610+0000 7fc932800640 1 -- 172.21.15.165:0/2267745617 learned_addr learned my addr 172.21.15.165:0/2267745617 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:56.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.611+0000 7fc932800640 1 -- 172.21.15.165:0/2267745617 >> v2:172.21.15.172:3300/0 conn(0x7fc9340ced80 msgr2=0x7fc9340d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:56.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.611+0000 7fc932800640 1 --2- 172.21.15.165:0/2267745617 >> v2:172.21.15.172:3300/0 conn(0x7fc9340ced80 0x7fc9340d22e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.611+0000 7fc932800640 1 -- 172.21.15.165:0/2267745617 >> v1:172.21.15.172:6789/0 conn(0x7fc93415dc20 legacy=0x7fc9340ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.611+0000 7fc932800640 1 -- 172.21.15.165:0/2267745617 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc934161f40 con 0x7fc934161710 2024-04-25T10:18:56.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.612+0000 7fc932800640 1 --2- 172.21.15.165:0/2267745617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc9340d2820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009e50 tx=0x7fc92c02a710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5dd95adada53bb2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.612+0000 7fc931400640 1 -- 172.21.15.165:0/2267745617 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c036070 con 0x7fc934161710 2024-04-25T10:18:56.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc931400640 1 -- 172.21.15.165:0/2267745617 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92c031600 con 0x7fc934161710 2024-04-25T10:18:56.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc931400640 1 -- 172.21.15.165:0/2267745617 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92c0319d0 con 0x7fc934161710 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 -- 172.21.15.165:0/2267745617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 msgr2=0x7fc9340d2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 --2- 172.21.15.165:0/2267745617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc9340d2820 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc92c009e50 tx=0x7fc92c02a710 comp rx=0 tx=0).stop 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 -- 172.21.15.165:0/2267745617 shutdown_connections 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 --2- 172.21.15.165:0/2267745617 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc9340d2820 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 --2- 172.21.15.165:0/2267745617 >> v2:172.21.15.172:3300/0 conn(0x7fc9340ced80 0x7fc9340d22e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 -- 172.21.15.165:0/2267745617 >> 172.21.15.165:0/2267745617 conn(0x7fc9340d0420 msgr2=0x7fc9340d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 -- 172.21.15.165:0/2267745617 shutdown_connections 2024-04-25T10:18:56.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.613+0000 7fc939000640 1 -- 172.21.15.165:0/2267745617 wait complete. 2024-04-25T10:18:56.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.615+0000 7fc939000640 1 Processor -- start 2024-04-25T10:18:56.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.615+0000 7fc939000640 1 -- start start 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc939000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 0x7fc93415ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc939000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc93415dc20 0x7fc934152da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc939000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc93415f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc939000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9340b5e50 con 0x7fc9340ced80 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc939000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc934162500 con 0x7fc93415dc20 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc939000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9340b5fd0 con 0x7fc934161710 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.616+0000 7fc931e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 0x7fc93415ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 0x7fc93415ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42000/0 (socket says 172.21.15.165:42000) 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 -- 172.21.15.165:0/2792635336 learned_addr learned my addr 172.21.15.165:0/2792635336 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc92be00640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc93415dc20 0x7fc934152da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc932800640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc93415f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 -- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 msgr2=0x7fc93415f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc93415f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 -- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc93415dc20 msgr2=0x7fc934152da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc93415dc20 0x7fc934152da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 -- 172.21.15.165:0/2792635336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9341602d0 con 0x7fc9340ced80 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc932800640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc93415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:56.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.617+0000 7fc931e00640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 0x7fc93415ee70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c3c0 tx=0x7fc92400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.618+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9240074e0 con 0x7fc9340ced80 2024-04-25T10:18:56.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.618+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc924007680 con 0x7fc9340ced80 2024-04-25T10:18:56.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.618+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924004a00 con 0x7fc9340ced80 2024-04-25T10:18:56.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.618+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc93415e390 con 0x7fc9340ced80 2024-04-25T10:18:56.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.618+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc93415e950 con 0x7fc9340ced80 2024-04-25T10:18:56.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.619+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc934073a00 con 0x7fc9340ced80 2024-04-25T10:18:56.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.620+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc92401a020 con 0x7fc9340ced80 2024-04-25T10:18:56.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.621+0000 7fc92a000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc91006b500 0x7fc91006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:56.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.621+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fc924089770 con 0x7fc9340ced80 2024-04-25T10:18:56.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.621+0000 7fc92be00640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc91006b500 0x7fc91006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:56.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.622+0000 7fc92be00640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc91006b500 0x7fc91006d9c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc93415d5b0 tx=0x7fc91c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:56.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.628+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc9240587a0 con 0x7fc9340ced80 2024-04-25T10:18:56.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:56.822+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fc9340d16b0 con 0x7fc9340ced80 2024-04-25T10:18:57.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.568+0000 7fc92a000640 1 -- 172.21.15.165:0/2792635336 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v168) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc9240580a0 con 0x7fc9340ced80 2024-04-25T10:18:57.571 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc91006b500 msgr2=0x7fc91006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc91006b500 0x7fc91006d9c0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fc93415d5b0 tx=0x7fc91c008040 comp rx=0 tx=0).stop 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 msgr2=0x7fc93415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 0x7fc93415ee70 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fc92400c3c0 tx=0x7fc92400c880 comp rx=0 tx=0).stop 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 shutdown_connections 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc91006b500 0x7fc91006d9c0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc934161710 0x7fc93415f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc93415dc20 0x7fc934152da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 --2- 172.21.15.165:0/2792635336 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc9340ced80 0x7fc93415ee70 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 >> 172.21.15.165:0/2792635336 conn(0x7fc9340d0420 msgr2=0x7fc93415c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:57.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.571+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 shutdown_connections 2024-04-25T10:18:57.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.572+0000 7fc939000640 1 -- 172.21.15.165:0/2792635336 wait complete. 2024-04-25T10:18:57.602 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:18:57.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.799+0000 7f3a18c00640 1 Processor -- start 2024-04-25T10:18:57.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.799+0000 7f3a18c00640 1 -- start start 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a18c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3a140cc4f0 0x7f3a140cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a18c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a140cdb20 0x7f3a140cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a18c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3a1415b270 0x7f3a1415b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a18c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a140ce4c0 con 0x7f3a140cc4f0 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a18c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a14071270 con 0x7f3a1415b270 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a18c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14072280 con 0x7f3a140cdb20 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a11e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3a140cc4f0 0x7f3a140cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a11e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3a140cc4f0 0x7f3a140cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40754/0 (socket says 172.21.15.165:40754) 2024-04-25T10:18:57.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.801+0000 7f3a11e00640 1 -- 172.21.15.165:0/1870485403 learned_addr learned my addr 172.21.15.165:0/1870485403 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:57.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a12800640 1 --2- 172.21.15.165:0/1870485403 >> v2:172.21.15.172:3300/0 conn(0x7f3a1415b270 0x7f3a1415b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a09400640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a140cdb20 0x7f3a140cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a11e00640 1 -- 172.21.15.165:0/1870485403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a140cdb20 msgr2=0x7f3a140cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a11e00640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a140cdb20 0x7f3a140cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a11e00640 1 -- 172.21.15.165:0/1870485403 >> v2:172.21.15.172:3300/0 conn(0x7f3a1415b270 msgr2=0x7f3a1415b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a11e00640 1 --2- 172.21.15.165:0/1870485403 >> v2:172.21.15.172:3300/0 conn(0x7f3a1415b270 0x7f3a1415b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a11e00640 1 -- 172.21.15.165:0/1870485403 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a140714c0 con 0x7f3a140cc4f0 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a09400640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a140cdb20 0x7f3a140cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.802+0000 7f3a12800640 1 --2- 172.21.15.165:0/1870485403 >> v2:172.21.15.172:3300/0 conn(0x7f3a1415b270 0x7f3a1415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.803+0000 7f3a11e00640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a140cc8d0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f3a14158aa0 tx=0x7f3a0402a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869f811b092e7870 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.803+0000 7f3a11400640 1 -- 172.21.15.165:0/1870485403 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04036070 con 0x7f3a140cc4f0 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.803+0000 7f3a11400640 1 -- 172.21.15.165:0/1870485403 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a04031600 con 0x7f3a140cc4f0 2024-04-25T10:18:57.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.803+0000 7f3a11400640 1 -- 172.21.15.165:0/1870485403 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a040319d0 con 0x7f3a140cc4f0 2024-04-25T10:18:57.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 -- 172.21.15.165:0/1870485403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 msgr2=0x7f3a140cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a140cc8d0 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7f3a14158aa0 tx=0x7f3a0402a6a0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 -- 172.21.15.165:0/1870485403 shutdown_connections 2024-04-25T10:18:57.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 --2- 172.21.15.165:0/1870485403 >> v2:172.21.15.172:3300/0 conn(0x7f3a1415b270 0x7f3a1415b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a140cdb20 0x7f3a140cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 --2- 172.21.15.165:0/1870485403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a140cc8d0 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 -- 172.21.15.165:0/1870485403 >> 172.21.15.165:0/1870485403 conn(0x7f3a1406abe0 msgr2=0x7f3a1406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 -- 172.21.15.165:0/1870485403 shutdown_connections 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.804+0000 7f3a18c00640 1 -- 172.21.15.165:0/1870485403 wait complete. 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.806+0000 7f3a18c00640 1 Processor -- start 2024-04-25T10:18:57.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.806+0000 7f3a18c00640 1 -- start start 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a18c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a1414cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a18c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a140cdb20 0x7f3a14159f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a18c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a1415b270 0x7f3a141f7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a18c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14071d60 con 0x7f3a140cc4f0 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a18c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14071a80 con 0x7f3a140cdb20 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a18c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a1415bbc0 con 0x7f3a1415b270 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a1414cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a1414cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40766/0 (socket says 172.21.15.165:40766) 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 -- 172.21.15.165:0/3673407256 learned_addr learned my addr 172.21.15.165:0/3673407256 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 -- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a1415b270 msgr2=0x7f3a141f7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a09400640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a140cdb20 0x7f3a14159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a12800640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a1415b270 0x7f3a141f7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a1415b270 0x7f3a141f7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 -- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a140cdb20 msgr2=0x7f3a14159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a140cdb20 0x7f3a14159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a11e00640 1 -- 172.21.15.165:0/3673407256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a14156610 con 0x7f3a140cc4f0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.807+0000 7f3a09400640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a140cdb20 0x7f3a14159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.808+0000 7f3a11e00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a1414cfe0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009f40 tx=0x7f3a04004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.808+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0403f020 con 0x7f3a140cc4f0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.808+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a04037040 con 0x7f3a140cc4f0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.808+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a04036070 con 0x7f3a140cc4f0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.808+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a141568f0 con 0x7f3a140cc4f0 2024-04-25T10:18:57.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.808+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1414d520 con 0x7f3a140cc4f0 2024-04-25T10:18:57.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.809+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14073a00 con 0x7f3a140cc4f0 2024-04-25T10:18:57.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.810+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3a04042020 con 0x7f3a140cc4f0 2024-04-25T10:18:57.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.811+0000 7f3a0aa00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f39f006b4d0 0x7f39f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:57.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.811+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f3a040aa740 con 0x7f3a140cc4f0 2024-04-25T10:18:57.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.812+0000 7f3a09400640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f39f006b4d0 0x7f39f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:57.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.812+0000 7f3a09400640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f39f006b4d0 0x7f39f006d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3a141512b0 tx=0x7f39fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:57.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:57.817+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3a04049030 con 0x7f3a140cc4f0 2024-04-25T10:18:58.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.011+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3a14151850 con 0x7f3a140cc4f0 2024-04-25T10:18:58.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.585+0000 7f3a0aa00640 1 -- 172.21.15.165:0/3673407256 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v169) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3a04079680 con 0x7f3a140cc4f0 2024-04-25T10:18:58.588 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:18:58.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f39f006b4d0 msgr2=0x7f39f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f39f006b4d0 0x7f39f006d990 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f3a141512b0 tx=0x7f39fc008040 comp rx=0 tx=0).stop 2024-04-25T10:18:58.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 msgr2=0x7f3a1414cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a1414cfe0 secure :-1 s=READY pgs=2413 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009f40 tx=0x7f3a04004a40 comp rx=0 tx=0).stop 2024-04-25T10:18:58.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 shutdown_connections 2024-04-25T10:18:58.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f39f006b4d0 0x7f39f006d990 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3a1415b270 0x7f3a141f7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3a140cdb20 0x7f3a14159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 --2- 172.21.15.165:0/3673407256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3a140cc4f0 0x7f3a1414cfe0 unknown :-1 s=CLOSED pgs=2413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 >> 172.21.15.165:0/3673407256 conn(0x7f3a1406abe0 msgr2=0x7f3a140b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:58.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 shutdown_connections 2024-04-25T10:18:58.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.589+0000 7f3a18c00640 1 -- 172.21.15.165:0/3673407256 wait complete. 2024-04-25T10:18:58.623 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:18:58.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.798+0000 7f71f5600640 1 Processor -- start 2024-04-25T10:18:58.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.799+0000 7f71f5600640 1 -- start start 2024-04-25T10:18:58.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71f5600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f71f015b710 0x7f71f00cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71f5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f006ec50 0x7f71f015e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71f5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71f015ee60 0x7f71f00ac8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71f5600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f015e490 con 0x7f71f015ee60 2024-04-25T10:18:58.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71f5600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f00cfd80 con 0x7f71f015b710 2024-04-25T10:18:58.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71f5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00d0060 con 0x7f71f006ec50 2024-04-25T10:18:58.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.800+0000 7f71ef400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71f015ee60 0x7f71f00ac8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f71f015ee60 0x7f71f00ac8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40776/0 (socket says 172.21.15.165:40776) 2024-04-25T10:18:58.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 -- 172.21.15.165:0/4214225624 learned_addr learned my addr 172.21.15.165:0/4214225624 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:58.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71e6000640 1 --2- 172.21.15.165:0/4214225624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f006ec50 0x7f71f015e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71eea00640 1 --2- 172.21.15.165:0/4214225624 >> v2:172.21.15.172:3300/0 conn(0x7f71f015b710 0x7f71f00cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 -- 172.21.15.165:0/4214225624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f006ec50 msgr2=0x7f71f015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:58.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 --2- 172.21.15.165:0/4214225624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f006ec50 0x7f71f015e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 -- 172.21.15.165:0/4214225624 >> v2:172.21.15.172:3300/0 conn(0x7f71f015b710 msgr2=0x7f71f00cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:58.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 --2- 172.21.15.165:0/4214225624 >> v2:172.21.15.172:3300/0 conn(0x7f71f015b710 0x7f71f00cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.801+0000 7f71ef400640 1 -- 172.21.15.165:0/4214225624 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00d0340 con 0x7f71f015ee60 2024-04-25T10:18:58.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.802+0000 7f71ef400640 1 --2- 172.21.15.165:0/4214225624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f015ee60 0x7f71f00ac8b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f71f006b950 tx=0x7f71e802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f957103d207dfc2d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:58.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.803+0000 7f71ee000640 1 -- 172.21.15.165:0/4214225624 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8036070 con 0x7f71f015ee60 2024-04-25T10:18:58.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.803+0000 7f71ee000640 1 -- 172.21.15.165:0/4214225624 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e8031600 con 0x7f71f015ee60 2024-04-25T10:18:58.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.803+0000 7f71ee000640 1 -- 172.21.15.165:0/4214225624 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e80319d0 con 0x7f71f015ee60 2024-04-25T10:18:58.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.803+0000 7f71f5600640 1 -- 172.21.15.165:0/4214225624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f015ee60 msgr2=0x7f71f00ac8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.803+0000 7f71f5600640 1 --2- 172.21.15.165:0/4214225624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f015ee60 0x7f71f00ac8b0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f71f006b950 tx=0x7f71e802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 -- 172.21.15.165:0/4214225624 shutdown_connections 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 --2- 172.21.15.165:0/4214225624 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f015ee60 0x7f71f00ac8b0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 --2- 172.21.15.165:0/4214225624 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f006ec50 0x7f71f015e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 --2- 172.21.15.165:0/4214225624 >> v2:172.21.15.172:3300/0 conn(0x7f71f015b710 0x7f71f00cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 -- 172.21.15.165:0/4214225624 >> 172.21.15.165:0/4214225624 conn(0x7f71f006abe0 msgr2=0x7f71f006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 -- 172.21.15.165:0/4214225624 shutdown_connections 2024-04-25T10:18:58.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.804+0000 7f71f5600640 1 -- 172.21.15.165:0/4214225624 wait complete. 2024-04-25T10:18:58.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.805+0000 7f71f5600640 1 Processor -- start 2024-04-25T10:18:58.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.806+0000 7f71f5600640 1 -- start start 2024-04-25T10:18:58.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71f5600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 0x7f71f014e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71f5600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015b710 0x7f71f01f6760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71f5600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f015ee60 0x7f71f00c50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71f5600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f015bc70 con 0x7f71f006ec50 2024-04-25T10:18:58.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71f5600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f015baf0 con 0x7f71f015ee60 2024-04-25T10:18:58.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71f5600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00acdf0 con 0x7f71f015b710 2024-04-25T10:18:58.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 0x7f71f014e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 0x7f71f014e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40778/0 (socket says 172.21.15.165:40778) 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 -- 172.21.15.165:0/195432006 learned_addr learned my addr 172.21.15.165:0/195432006 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 -- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015b710 msgr2=0x7f71f01f6760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71e6000640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015b710 0x7f71f01f6760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015b710 0x7f71f01f6760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 -- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f015ee60 msgr2=0x7f71f00c50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f015ee60 0x7f71f00c50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.807+0000 7f71eea00640 1 -- 172.21.15.165:0/195432006 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f01f5700 con 0x7f71f006ec50 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.808+0000 7f71eea00640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 0x7f71f014e270 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f71e000d800 tx=0x7f71e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:58.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.808+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000b6c0 con 0x7f71f006ec50 2024-04-25T10:18:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.808+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e0019070 con 0x7f71f006ec50 2024-04-25T10:18:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.808+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000ba80 con 0x7f71f006ec50 2024-04-25T10:18:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.808+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f01f59a0 con 0x7f71f006ec50 2024-04-25T10:18:58.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.808+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f01f5f60 con 0x7f71f006ec50 2024-04-25T10:18:58.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.810+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f0073a00 con 0x7f71f006ec50 2024-04-25T10:18:58.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.811+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f71e000bc20 con 0x7f71f006ec50 2024-04-25T10:18:58.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.811+0000 7f71e7400640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71cc06b500 0x7f71cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:58.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.811+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f71e008a7e0 con 0x7f71f006ec50 2024-04-25T10:18:58.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.812+0000 7f71e6000640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71cc06b500 0x7f71cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:58.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.812+0000 7f71e6000640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71cc06b500 0x7f71cc06d9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f71f0158010 tx=0x7f71d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:58.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:58.816+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f71e0059530 con 0x7f71f006ec50 2024-04-25T10:18:59.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.041+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f71f00b3c80 con 0x7f71f006ec50 2024-04-25T10:18:59.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.589+0000 7f71e7400640 1 -- 172.21.15.165:0/195432006 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v117) v1 ==== 174+0+0 (secure 0 0 0) 0x7f71e0058e30 con 0x7f71f006ec50 2024-04-25T10:18:59.592 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 25 and data pool 26 2024-04-25T10:18:59.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.594+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71cc06b500 msgr2=0x7f71cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.594+0000 7f71f5600640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71cc06b500 0x7f71cc06d9c0 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f71f0158010 tx=0x7f71d8008040 comp rx=0 tx=0).stop 2024-04-25T10:18:59.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.594+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 msgr2=0x7f71f014e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.594+0000 7f71f5600640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 0x7f71f014e270 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7f71e000d800 tx=0x7f71e000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.595+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 shutdown_connections 2024-04-25T10:18:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.595+0000 7f71f5600640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f71cc06b500 0x7f71cc06d9c0 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.595+0000 7f71f5600640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f71f015ee60 0x7f71f00c50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.595+0000 7f71f5600640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f71f015b710 0x7f71f01f6760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.595+0000 7f71f5600640 1 --2- 172.21.15.165:0/195432006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f71f006ec50 0x7f71f014e270 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.595+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 >> 172.21.15.165:0/195432006 conn(0x7f71f006abe0 msgr2=0x7f71f015c7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.596+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 shutdown_connections 2024-04-25T10:18:59.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.597+0000 7f71f5600640 1 -- 172.21.15.165:0/195432006 wait complete. 2024-04-25T10:18:59.636 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:18:59.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.803+0000 7efe7dc00640 1 Processor -- start 2024-04-25T10:18:59.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.803+0000 7efe7dc00640 1 -- start start 2024-04-25T10:18:59.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.804+0000 7efe7dc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efe7815dd00 0x7efe780cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.804+0000 7efe7dc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7815bf30 con 0x7efe78144440 2024-04-25T10:18:59.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.804+0000 7efe7dc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7815c210 con 0x7efe78071700 2024-04-25T10:18:59.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.804+0000 7efe7dc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe7815c860 con 0x7efe7815dd00 2024-04-25T10:18:59.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe77e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efe78144440 0x7efe78070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54066/0 (socket says 172.21.15.165:54066) 2024-04-25T10:18:59.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe77e00640 1 -- 172.21.15.165:0/2580223571 learned_addr learned my addr 172.21.15.165:0/2580223571 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe7d200640 1 --2- 172.21.15.165:0/2580223571 >> v2:172.21.15.165:3300/0 conn(0x7efe7815dd00 0x7efe780cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe76a00640 1 -- 172.21.15.165:0/2580223571 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 812191373 0 0) 0x7efe7815bf30 con 0x7efe78144440 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe76a00640 1 -- 172.21.15.165:0/2580223571 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe5c003520 con 0x7efe78144440 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe7d200640 1 -- 172.21.15.165:0/2580223571 >> v1:172.21.15.172:6789/0 conn(0x7efe78071700 legacy=0x7efe780cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe7d200640 1 -- 172.21.15.165:0/2580223571 >> v1:172.21.15.165:6789/0 conn(0x7efe78144440 legacy=0x7efe78070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.805+0000 7efe7d200640 1 -- 172.21.15.165:0/2580223571 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe7815c4f0 con 0x7efe7815dd00 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.806+0000 7efe7d200640 1 --2- 172.21.15.165:0/2580223571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe7815dd00 0x7efe780cda30 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7efe7815c9e0 tx=0x7efe7002a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb0dfcd2da7ed85 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.806+0000 7efe76a00640 1 -- 172.21.15.165:0/2580223571 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70007740 con 0x7efe7815dd00 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.806+0000 7efe76a00640 1 -- 172.21.15.165:0/2580223571 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe700078e0 con 0x7efe7815dd00 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.806+0000 7efe76a00640 1 -- 172.21.15.165:0/2580223571 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70007cb0 con 0x7efe7815dd00 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 -- 172.21.15.165:0/2580223571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe7815dd00 msgr2=0x7efe780cda30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2580223571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe7815dd00 0x7efe780cda30 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7efe7815c9e0 tx=0x7efe7002a730 comp rx=0 tx=0).stop 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 -- 172.21.15.165:0/2580223571 shutdown_connections 2024-04-25T10:18:59.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2580223571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe7815dd00 0x7efe780cda30 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 -- 172.21.15.165:0/2580223571 >> 172.21.15.165:0/2580223571 conn(0x7efe7806abe0 msgr2=0x7efe7806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:18:59.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 -- 172.21.15.165:0/2580223571 shutdown_connections 2024-04-25T10:18:59.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.807+0000 7efe7dc00640 1 -- 172.21.15.165:0/2580223571 wait complete. 2024-04-25T10:18:59.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.809+0000 7efe7dc00640 1 Processor -- start 2024-04-25T10:18:59.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.809+0000 7efe7dc00640 1 -- start start 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe7dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe78071700 0x7efe78165020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe7dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 0x7efe78163e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe7dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe7815dd00 0x7efe7813ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe7dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe78166400 con 0x7efe78144440 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe7dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe78166280 con 0x7efe7815dd00 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe7dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe78166580 con 0x7efe78071700 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 0x7efe78163e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 0x7efe78163e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40800/0 (socket says 172.21.15.165:40800) 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 -- 172.21.15.165:0/2409905479 learned_addr learned my addr 172.21.15.165:0/2409905479 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 -- 172.21.15.165:0/2409905479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe78071700 msgr2=0x7efe78165020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe78071700 0x7efe78165020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 -- 172.21.15.165:0/2409905479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe7815dd00 msgr2=0x7efe7813ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe7815dd00 0x7efe7813ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.810+0000 7efe77400640 1 -- 172.21.15.165:0/2409905479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe78165600 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.811+0000 7efe77400640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 0x7efe78163e60 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c430 tx=0x7efe6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.811+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64011790 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.811+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe64017070 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.811+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64011bb0 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.811+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe781658e0 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.811+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe78163940 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.812+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe78163b40 con 0x7efe78144440 2024-04-25T10:18:59.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.813+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efe6401c460 con 0x7efe78144440 2024-04-25T10:18:59.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.814+0000 7efe6fe00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe5806b4d0 0x7efe5806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:18:59.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.814+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efe6408a310 con 0x7efe78144440 2024-04-25T10:18:59.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.814+0000 7efe77e00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe5806b4d0 0x7efe5806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:18:59.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.815+0000 7efe77e00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe5806b4d0 0x7efe5806d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7efe68000f60 tx=0x7efe680078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:18:59.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.818+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efe78163b40 con 0x7efe78144440 2024-04-25T10:19:00.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.998+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe78073990 con 0x7efe78144440 2024-04-25T10:19:00.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:18:59.999+0000 7efe6fe00640 1 -- 172.21.15.165:0/2409905479 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16937 (secure 0 0 0) 0x7efe64058f60 con 0x7efe78144440 2024-04-25T10:19:00.003 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:00.003 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":170,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:18:59.042615+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe5806b4d0 msgr2=0x7efe5806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe5806b4d0 0x7efe5806d990 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7efe68000f60 tx=0x7efe680078f0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 msgr2=0x7efe78163e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 0x7efe78163e60 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7efe6400c430 tx=0x7efe6400c8f0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 shutdown_connections 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efe5806b4d0 0x7efe5806d990 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efe7815dd00 0x7efe7813ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efe78144440 0x7efe78163e60 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 --2- 172.21.15.165:0/2409905479 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efe78071700 0x7efe78165020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 >> 172.21.15.165:0/2409905479 conn(0x7efe7806abe0 msgr2=0x7efe7806d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 shutdown_connections 2024-04-25T10:19:00.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.002+0000 7efe7dc00640 1 -- 172.21.15.165:0/2409905479 wait complete. 2024-04-25T10:19:00.023 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:00.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.213+0000 7fc265200640 1 Processor -- start 2024-04-25T10:19:00.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.213+0000 7fc265200640 1 -- start start 2024-04-25T10:19:00.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.215+0000 7fc265200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc260144440 0x7fc260070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.215+0000 7fc265200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 0x7fc26015ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.215+0000 7fc265200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc26015bf30 con 0x7fc260071530 2024-04-25T10:19:00.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.215+0000 7fc265200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2600719a0 con 0x7fc260144440 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.215+0000 7fc265200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26015c860 con 0x7fc2600cda30 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.215+0000 7fc25fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 0x7fc26015ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 0x7fc26015ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:41220/0 (socket says 172.21.15.165:41220) 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25ea00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc260071530 0x7fc2600cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:34840/0 (socket says 172.21.15.165:34840) 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25fe00640 1 -- 172.21.15.165:0/799841221 learned_addr learned my addr 172.21.15.165:0/799841221 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25f400640 1 --2- 172.21.15.165:0/799841221 >> v2:172.21.15.172:3300/0 conn(0x7fc260144440 0x7fc260070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25fe00640 1 -- 172.21.15.165:0/799841221 >> v2:172.21.15.172:3300/0 conn(0x7fc260144440 msgr2=0x7fc260070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25fe00640 1 --2- 172.21.15.165:0/799841221 >> v2:172.21.15.172:3300/0 conn(0x7fc260144440 0x7fc260070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25fe00640 1 -- 172.21.15.165:0/799841221 >> v1:172.21.15.172:6789/0 conn(0x7fc260071530 legacy=0x7fc2600cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.216+0000 7fc25fe00640 1 -- 172.21.15.165:0/799841221 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc26015c210 con 0x7fc2600cda30 2024-04-25T10:19:00.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.218+0000 7fc25fe00640 1 --2- 172.21.15.165:0/799841221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 0x7fc26015ccf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc258009e50 tx=0x7fc25802a7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e42c4e972109b8d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.219+0000 7fc25e000640 1 -- 172.21.15.165:0/799841221 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258036070 con 0x7fc2600cda30 2024-04-25T10:19:00.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.219+0000 7fc25e000640 1 -- 172.21.15.165:0/799841221 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258031600 con 0x7fc2600cda30 2024-04-25T10:19:00.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.219+0000 7fc25e000640 1 -- 172.21.15.165:0/799841221 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2580319d0 con 0x7fc2600cda30 2024-04-25T10:19:00.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 -- 172.21.15.165:0/799841221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 msgr2=0x7fc26015ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 --2- 172.21.15.165:0/799841221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 0x7fc26015ccf0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc258009e50 tx=0x7fc25802a7a0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 -- 172.21.15.165:0/799841221 shutdown_connections 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 --2- 172.21.15.165:0/799841221 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc2600cda30 0x7fc26015ccf0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 --2- 172.21.15.165:0/799841221 >> v2:172.21.15.172:3300/0 conn(0x7fc260144440 0x7fc260070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 -- 172.21.15.165:0/799841221 >> 172.21.15.165:0/799841221 conn(0x7fc26006abe0 msgr2=0x7fc26006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 -- 172.21.15.165:0/799841221 shutdown_connections 2024-04-25T10:19:00.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.220+0000 7fc265200640 1 -- 172.21.15.165:0/799841221 wait complete. 2024-04-25T10:19:00.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.222+0000 7fc265200640 1 Processor -- start 2024-04-25T10:19:00.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.222+0000 7fc265200640 1 -- start start 2024-04-25T10:19:00.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.223+0000 7fc265200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc260071530 0x7fc2600a9d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc265200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2600cda30 0x7fc26015aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc265200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 0x7fc26015afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc265200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26015bc30 con 0x7fc260144440 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc265200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2600cdea0 con 0x7fc2600cda30 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc265200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc26015d230 con 0x7fc260071530 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 0x7fc26015afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 0x7fc26015afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40816/0 (socket says 172.21.15.165:40816) 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 -- 172.21.15.165:0/3742996260 learned_addr learned my addr 172.21.15.165:0/3742996260 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25f400640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc260071530 0x7fc2600a9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25ea00640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2600cda30 0x7fc26015aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 -- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc260071530 msgr2=0x7fc2600a9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc260071530 0x7fc2600a9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 -- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2600cda30 msgr2=0x7fc26015aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2600cda30 0x7fc26015aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.224+0000 7fc25fe00640 1 -- 172.21.15.165:0/3742996260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2600a85f0 con 0x7fc260144440 2024-04-25T10:19:00.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.225+0000 7fc25fe00640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 0x7fc26015afb0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc258034040 tx=0x7fc258004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.225+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258036050 con 0x7fc260144440 2024-04-25T10:19:00.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.225+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc258042070 con 0x7fc260144440 2024-04-25T10:19:00.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.225+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2600a8890 con 0x7fc260144440 2024-04-25T10:19:00.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.225+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc258009510 con 0x7fc260144440 2024-04-25T10:19:00.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.225+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2600a8e50 con 0x7fc260144440 2024-04-25T10:19:00.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.227+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2600a9340 con 0x7fc260144440 2024-04-25T10:19:00.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.228+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc258007bf0 con 0x7fc260144440 2024-04-25T10:19:00.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.229+0000 7fc257400640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc24006b4d0 0x7fc24006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.229+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc2580ab160 con 0x7fc260144440 2024-04-25T10:19:00.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.230+0000 7fc25f400640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc24006b4d0 0x7fc24006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.230+0000 7fc25f400640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc24006b4d0 0x7fc24006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b1840 tx=0x7fc250008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.234+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc258079e40 con 0x7fc260144440 2024-04-25T10:19:00.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.432+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc260073990 con 0x7fc260144440 2024-04-25T10:19:00.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.435+0000 7fc257400640 1 -- 172.21.15.165:0/3742996260 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16937 (secure 0 0 0) 0x7fc258079740 con 0x7fc260144440 2024-04-25T10:19:00.438 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:00.439 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":170,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:18:59.042615+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:00.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc24006b4d0 msgr2=0x7fc24006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc24006b4d0 0x7fc24006d990 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7fc2600b1840 tx=0x7fc250008040 comp rx=0 tx=0).stop 2024-04-25T10:19:00.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 msgr2=0x7fc26015afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 0x7fc26015afb0 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7fc258034040 tx=0x7fc258004a40 comp rx=0 tx=0).stop 2024-04-25T10:19:00.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 shutdown_connections 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc24006b4d0 0x7fc24006d990 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc260144440 0x7fc26015afb0 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2600cda30 0x7fc26015aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 --2- 172.21.15.165:0/3742996260 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc260071530 0x7fc2600a9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 >> 172.21.15.165:0/3742996260 conn(0x7fc26006abe0 msgr2=0x7fc26013fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 shutdown_connections 2024-04-25T10:19:00.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.438+0000 7fc265200640 1 -- 172.21.15.165:0/3742996260 wait complete. 2024-04-25T10:19:00.459 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:19:00.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.655+0000 7f8c66200640 1 Processor -- start 2024-04-25T10:19:00.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.655+0000 7f8c66200640 1 -- start start 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c66200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 0x7f8c6015c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c66200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8c600b4820 0x7f8c6006f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c66200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c6006e410 con 0x7f8c6015b950 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c66200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c6006e6b0 con 0x7f8c600b4820 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c66200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6006e990 con 0x7f8c601586d0 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c56a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8c6015b950 0x7f8c600b4110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54070/0 (socket says 172.21.15.165:54070) 2024-04-25T10:19:00.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.657+0000 7f8c56a00640 1 -- 172.21.15.165:0/1011840787 learned_addr learned my addr 172.21.15.165:0/1011840787 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:00.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5f400640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 0x7f8c6015c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1011840787 >> v2:172.21.15.172:3300/0 conn(0x7f8c600b4820 0x7f8c6006f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 359150962 0 0) 0x7f8c6006e410 con 0x7f8c6015b950 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c44003520 con 0x7f8c6015b950 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 312895128 0 0) 0x7f8c44003520 con 0x7f8c6015b950 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 msgr2=0x7f8c6015c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 0x7f8c6015c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 >> v2:172.21.15.172:3300/0 conn(0x7f8c600b4820 msgr2=0x7f8c6006f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 --2- 172.21.15.165:0/1011840787 >> v2:172.21.15.172:3300/0 conn(0x7f8c600b4820 0x7f8c6006f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c6006ec70 con 0x7f8c6015b950 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8c48001670 con 0x7f8c6015b950 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 >> v1:172.21.15.165:6789/0 conn(0x7f8c6015b950 legacy=0x7f8c600b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c440039b0 0x7f8c44003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 0x7f8c44004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c44004d40 0x7f8c440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6006e6b0 con 0x7f8c44004d40 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6006e990 con 0x7f8c440039b0 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.658+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4400dbd0 con 0x7f8c601586d0 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c44004d40 0x7f8c440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 msgr2=0x7f8c44004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c601586d0 0x7f8c44004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5f400640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c440039b0 0x7f8c44003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c440039b0 msgr2=0x7f8c44003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c440039b0 0x7f8c44003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.659+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1011840787 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c44003520 con 0x7f8c44004d40 2024-04-25T10:19:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.660+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c44004d40 0x7f8c440051c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009fc0 tx=0x7f8c44005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c43e283688a29c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.660+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58009c10 con 0x7f8c44004d40 2024-04-25T10:19:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.660+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58036070 con 0x7f8c44004d40 2024-04-25T10:19:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.660+0000 7f8c5ea00640 1 -- 172.21.15.165:0/1011840787 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c5802d070 con 0x7f8c44004d40 2024-04-25T10:19:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c66200640 1 -- 172.21.15.165:0/1011840787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c44004d40 msgr2=0x7f8c440051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c66200640 1 --2- 172.21.15.165:0/1011840787 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c44004d40 0x7f8c440051c0 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f8c58009fc0 tx=0x7f8c44005830 comp rx=0 tx=0).stop 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1011840787 reap_dead start 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c66200640 1 -- 172.21.15.165:0/1011840787 shutdown_connections 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c66200640 1 -- 172.21.15.165:0/1011840787 >> 172.21.15.165:0/1011840787 conn(0x7f8c600b5af0 msgr2=0x7f8c600b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c66200640 1 -- 172.21.15.165:0/1011840787 shutdown_connections 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.661+0000 7f8c66200640 1 -- 172.21.15.165:0/1011840787 wait complete. 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.663+0000 7f8c66200640 1 Processor -- start 2024-04-25T10:19:00.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.663+0000 7f8c66200640 1 -- start start 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c66200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c44004d40 0x7f8c601ef870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c66200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c60154da0 0x7f8c60155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c66200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 0x7f8c6015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c66200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6015bdb0 con 0x7f8c60159d70 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c66200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c6006f230 con 0x7f8c44004d40 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c66200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c600b3c60 con 0x7f8c60154da0 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 0x7f8c6015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c5fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 0x7f8c6015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40828/0 (socket says 172.21.15.165:40828) 2024-04-25T10:19:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.664+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1201629733 learned_addr learned my addr 172.21.15.165:0/1201629733 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c56a00640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c60154da0 0x7f8c60155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5f400640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c44004d40 0x7f8c601ef870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c60154da0 msgr2=0x7f8c60155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c60154da0 0x7f8c60155220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c44004d40 msgr2=0x7f8c601ef870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c44004d40 0x7f8c601ef870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5fe00640 1 -- 172.21.15.165:0/1201629733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c601f3590 con 0x7f8c60159d70 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5f400640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c44004d40 0x7f8c601ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c5fe00640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 0x7f8c6015a1f0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f8c58033040 tx=0x7f8c58009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58009c10 con 0x7f8c60159d70 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58003bb0 con 0x7f8c60159d70 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.665+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58031e30 con 0x7f8c60159d70 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.666+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c601f3800 con 0x7f8c60159d70 2024-04-25T10:19:00.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.666+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c601efdb0 con 0x7f8c60159d70 2024-04-25T10:19:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.667+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c60073a00 con 0x7f8c60159d70 2024-04-25T10:19:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.668+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c58005d10 con 0x7f8c60159d70 2024-04-25T10:19:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.668+0000 7f8c57e00640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:00.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.669+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8c580ab850 con 0x7f8c60159d70 2024-04-25T10:19:00.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.669+0000 7f8c5f400640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:00.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.669+0000 7f8c5f400640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8c600afe00 tx=0x7f8c50008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.673+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c5807a4a0 con 0x7f8c60159d70 2024-04-25T10:19:00.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:00.889+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8c60072f60 con 0x7f8c60159d70 2024-04-25T10:19:01.028 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:01.036 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:01.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.639+0000 7f8c57e00640 1 -- 172.21.15.165:0/1201629733 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v120) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8c58079da0 con 0x7f8c60159d70 2024-04-25T10:19:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c3c06b4d0 msgr2=0x7f8c3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f8c600afe00 tx=0x7f8c50008040 comp rx=0 tx=0).stop 2024-04-25T10:19:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 msgr2=0x7f8c6015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 0x7f8c6015a1f0 secure :-1 s=READY pgs=2422 cs=0 l=1 rev1=1 crypto rx=0x7f8c58033040 tx=0x7f8c58009750 comp rx=0 tx=0).stop 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 shutdown_connections 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8c3c06b4d0 0x7f8c3c06d990 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c60159d70 0x7f8c6015a1f0 unknown :-1 s=CLOSED pgs=2422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c60154da0 0x7f8c60155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 --2- 172.21.15.165:0/1201629733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c44004d40 0x7f8c601ef870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.642+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 >> 172.21.15.165:0/1201629733 conn(0x7f8c600b5af0 msgr2=0x7f8c600b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.643+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 shutdown_connections 2024-04-25T10:19:01.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.643+0000 7f8c66200640 1 -- 172.21.15.165:0/1201629733 wait complete. 2024-04-25T10:19:01.667 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:19:01.667 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:19:01.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.868+0000 7f5ec5e00640 1 Processor -- start 2024-04-25T10:19:01.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.868+0000 7f5ec5e00640 1 -- start start 2024-04-25T10:19:01.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ec5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ec00cc4f0 0x7f5ec00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ec5e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5ec015b980 0x7f5ec00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ec5e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec0072280 con 0x7f5ec00cdb20 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ec5e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec0071270 con 0x7f5ec00cc4f0 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ec5e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec015bd80 con 0x7f5ec015b980 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ebea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ec00cc4f0 0x7f5ec00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ebea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5ec00cc4f0 0x7f5ec00cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40840/0 (socket says 172.21.15.165:40840) 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.870+0000 7f5ebea00640 1 -- 172.21.15.165:0/2041632276 learned_addr learned my addr 172.21.15.165:0/2041632276 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebf400640 1 --2- 172.21.15.165:0/2041632276 >> v2:172.21.15.172:3300/0 conn(0x7f5ec015b980 0x7f5ec00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebea00640 1 -- 172.21.15.165:0/2041632276 >> v2:172.21.15.172:3300/0 conn(0x7f5ec015b980 msgr2=0x7f5ec00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebea00640 1 --2- 172.21.15.165:0/2041632276 >> v2:172.21.15.172:3300/0 conn(0x7f5ec015b980 0x7f5ec00cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebea00640 1 -- 172.21.15.165:0/2041632276 >> v1:172.21.15.165:6789/0 conn(0x7f5ec00cdb20 legacy=0x7f5ec015b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebe000640 1 -- 172.21.15.165:0/2041632276 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2291340433 0 0) 0x7f5ec0072280 con 0x7f5ec00cdb20 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebea00640 1 -- 172.21.15.165:0/2041632276 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00714c0 con 0x7f5ec00cc4f0 2024-04-25T10:19:01.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.871+0000 7f5ebf400640 1 --2- 172.21.15.165:0/2041632276 >> v2:172.21.15.172:3300/0 conn(0x7f5ec015b980 0x7f5ec00cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.872+0000 7f5ebea00640 1 --2- 172.21.15.165:0/2041632276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cc4f0 0x7f5ec00cc8d0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00b4c00 tx=0x7f5eb002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=967a5f1f9e0f0380 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.872+0000 7f5ebe000640 1 -- 172.21.15.165:0/2041632276 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0037070 con 0x7f5ec00cc4f0 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.872+0000 7f5ebe000640 1 -- 172.21.15.165:0/2041632276 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb0032440 con 0x7f5ec00cc4f0 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.872+0000 7f5ebe000640 1 -- 172.21.15.165:0/2041632276 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb0032810 con 0x7f5ec00cc4f0 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 -- 172.21.15.165:0/2041632276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cc4f0 msgr2=0x7f5ec00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/2041632276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cc4f0 0x7f5ec00cc8d0 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f5ec00b4c00 tx=0x7f5eb002a270 comp rx=0 tx=0).stop 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 -- 172.21.15.165:0/2041632276 shutdown_connections 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/2041632276 >> v2:172.21.15.172:3300/0 conn(0x7f5ec015b980 0x7f5ec00cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/2041632276 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cc4f0 0x7f5ec00cc8d0 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 -- 172.21.15.165:0/2041632276 >> 172.21.15.165:0/2041632276 conn(0x7f5ec006abe0 msgr2=0x7f5ec006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 -- 172.21.15.165:0/2041632276 shutdown_connections 2024-04-25T10:19:01.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.873+0000 7f5ec5e00640 1 -- 172.21.15.165:0/2041632276 wait complete. 2024-04-25T10:19:01.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.875+0000 7f5ec5e00640 1 Processor -- start 2024-04-25T10:19:01.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.875+0000 7f5ec5e00640 1 -- start start 2024-04-25T10:19:01.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ec5e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec00cc4f0 0x7f5ec01f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ec5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 0x7f5ec00af910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ec5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ec015b980 0x7f5ec00afe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ec5e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0071d60 con 0x7f5ec00cdb20 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ec5e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec0071a80 con 0x7f5ec00cc4f0 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ec5e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00ae500 con 0x7f5ec015b980 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 0x7f5ec00af910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 0x7f5ec00af910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40846/0 (socket says 172.21.15.165:40846) 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 -- 172.21.15.165:0/953360417 learned_addr learned my addr 172.21.15.165:0/953360417 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5ebf400640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ec015b980 0x7f5ec00afe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 -- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ec015b980 msgr2=0x7f5ec00afe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ec015b980 0x7f5ec00afe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 -- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec00cc4f0 msgr2=0x7f5ec01f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec00cc4f0 0x7f5ec01f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.876+0000 7f5eb6000640 1 -- 172.21.15.165:0/953360417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec01f4740 con 0x7f5ec00cdb20 2024-04-25T10:19:01.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.877+0000 7f5eb6000640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 0x7f5ec00af910 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c4b0 tx=0x7f5ea800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.877+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8017070 con 0x7f5ec00cdb20 2024-04-25T10:19:01.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.877+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8005480 con 0x7f5ec00cdb20 2024-04-25T10:19:01.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.877+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8003830 con 0x7f5ec00cdb20 2024-04-25T10:19:01.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.877+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec00b0480 con 0x7f5ec00cdb20 2024-04-25T10:19:01.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.877+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec00b0a40 con 0x7f5ec00cdb20 2024-04-25T10:19:01.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.878+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec0073a00 con 0x7f5ec00cdb20 2024-04-25T10:19:01.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.879+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ea801a070 con 0x7f5ec00cdb20 2024-04-25T10:19:01.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.881+0000 7f5eb7e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e9c06b4d0 0x7f5e9c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:01.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.881+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5ea80897a0 con 0x7f5ec00cdb20 2024-04-25T10:19:01.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.881+0000 7f5ebea00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e9c06b4d0 0x7f5e9c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:01.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.882+0000 7f5ebea00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e9c06b4d0 0x7f5e9c06d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0034000 tx=0x7f5eb002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:01.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:01.885+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ea80583f0 con 0x7f5ec00cdb20 2024-04-25T10:19:02.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.101+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5ec006d3a0 con 0x7f5ec00cdb20 2024-04-25T10:19:02.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.650+0000 7f5eb7e00640 1 -- 172.21.15.165:0/953360417 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v121) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5ea8057cf0 con 0x7f5ec00cdb20 2024-04-25T10:19:02.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e9c06b4d0 msgr2=0x7f5e9c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e9c06b4d0 0x7f5e9c06d990 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f5eb0034000 tx=0x7f5eb002c000 comp rx=0 tx=0).stop 2024-04-25T10:19:02.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 msgr2=0x7f5ec00af910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 0x7f5ec00af910 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c4b0 tx=0x7f5ea800c970 comp rx=0 tx=0).stop 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 shutdown_connections 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e9c06b4d0 0x7f5e9c06d990 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5ec015b980 0x7f5ec00afe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5ec00cdb20 0x7f5ec00af910 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 --2- 172.21.15.165:0/953360417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5ec00cc4f0 0x7f5ec01f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 >> 172.21.15.165:0/953360417 conn(0x7f5ec006abe0 msgr2=0x7f5ec015b1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 shutdown_connections 2024-04-25T10:19:02.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.654+0000 7f5ec5e00640 1 -- 172.21.15.165:0/953360417 wait complete. 2024-04-25T10:19:02.678 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:19:02.679 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.881+0000 7f18acc00640 1 Processor -- start 2024-04-25T10:19:02.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.881+0000 7f18acc00640 1 -- start start 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.882+0000 7f18acc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8144440 0x7f18a8070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.882+0000 7f18acc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f18a8071530 0x7f18a80cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.882+0000 7f18acc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f18a80cd860 0x7f18a806d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.882+0000 7f18acc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a815c860 con 0x7f18a8071530 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.882+0000 7f18acc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a8071970 con 0x7f18a80cd860 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.882+0000 7f18acc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a815bf30 con 0x7f18a8144440 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f18a8071530 0x7f18a80cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f18a8071530 0x7f18a80cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40862/0 (socket says 172.21.15.165:40862) 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 -- 172.21.15.165:0/360502821 learned_addr learned my addr 172.21.15.165:0/360502821 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a7400640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8144440 0x7f18a8070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 -- 172.21.15.165:0/360502821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8144440 msgr2=0x7f18a8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8144440 0x7f18a8070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 -- 172.21.15.165:0/360502821 >> v2:172.21.15.172:3300/0 conn(0x7f18a80cd860 msgr2=0x7f18a806d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a7e00640 1 --2- 172.21.15.165:0/360502821 >> v2:172.21.15.172:3300/0 conn(0x7f18a80cd860 0x7f18a806d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 --2- 172.21.15.165:0/360502821 >> v2:172.21.15.172:3300/0 conn(0x7f18a80cd860 0x7f18a806d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a6a00640 1 -- 172.21.15.165:0/360502821 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a815c210 con 0x7f18a8071530 2024-04-25T10:19:02.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.883+0000 7f18a7400640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8144440 0x7f18a8070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.884+0000 7f18a6a00640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a8071530 0x7f18a80cd320 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f18a8145360 tx=0x7f189002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea53520b5f155c52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18a6000640 1 -- 172.21.15.165:0/360502821 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890037070 con 0x7f18a8071530 2024-04-25T10:19:02.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18a6000640 1 -- 172.21.15.165:0/360502821 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1890032440 con 0x7f18a8071530 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18a6000640 1 -- 172.21.15.165:0/360502821 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890032810 con 0x7f18a8071530 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 -- 172.21.15.165:0/360502821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a8071530 msgr2=0x7f18a80cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a8071530 0x7f18a80cd320 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f18a8145360 tx=0x7f189002a270 comp rx=0 tx=0).stop 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 -- 172.21.15.165:0/360502821 shutdown_connections 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 --2- 172.21.15.165:0/360502821 >> v2:172.21.15.172:3300/0 conn(0x7f18a80cd860 0x7f18a806d050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a8071530 0x7f18a80cd320 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 --2- 172.21.15.165:0/360502821 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8144440 0x7f18a8070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.885+0000 7f18acc00640 1 -- 172.21.15.165:0/360502821 >> 172.21.15.165:0/360502821 conn(0x7f18a806abe0 msgr2=0x7f18a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.886+0000 7f18acc00640 1 -- 172.21.15.165:0/360502821 shutdown_connections 2024-04-25T10:19:02.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.886+0000 7f18acc00640 1 -- 172.21.15.165:0/360502821 wait complete. 2024-04-25T10:19:02.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.888+0000 7f18acc00640 1 Processor -- start 2024-04-25T10:19:02.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.888+0000 7f18acc00640 1 -- start start 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18acc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8071530 0x7f18a81e1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18acc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 0x7f18a81e25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18acc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18a8144440 0x7f18a80cad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18acc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cdcd0 con 0x7f18a80cd860 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18acc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a815bc30 con 0x7f18a8144440 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18acc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a80cde50 con 0x7f18a8071530 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 0x7f18a81e25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 0x7f18a81e25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40874/0 (socket says 172.21.15.165:40874) 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18a6a00640 1 -- 172.21.15.165:0/3608398439 learned_addr learned my addr 172.21.15.165:0/3608398439 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.889+0000 7f18a7400640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8071530 0x7f18a81e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a6a00640 1 -- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8071530 msgr2=0x7f18a81e1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a6a00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8071530 0x7f18a81e1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a6a00640 1 -- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18a8144440 msgr2=0x7f18a80cad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a7e00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18a8144440 0x7f18a80cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a6a00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18a8144440 0x7f18a80cad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a6a00640 1 -- 172.21.15.165:0/3608398439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a80c8340 con 0x7f18a80cd860 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a7e00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18a8144440 0x7f18a80cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18a6a00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 0x7f18a81e25f0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f189002a2b0 tx=0x7f18900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890037040 con 0x7f18a80cd860 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1890042070 con 0x7f18a80cd860 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a80c8620 con 0x7f18a80cd860 2024-04-25T10:19:02.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890030bb0 con 0x7f18a80cd860 2024-04-25T10:19:02.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.890+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a80c8be0 con 0x7f18a80cd860 2024-04-25T10:19:02.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.892+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a80ce7d0 con 0x7f18a80cd860 2024-04-25T10:19:02.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.893+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1890047460 con 0x7f18a80cd860 2024-04-25T10:19:02.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.894+0000 7f189f400640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f188406b4d0 0x7f188406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:02.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.894+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f18900ab120 con 0x7f18a80cd860 2024-04-25T10:19:02.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.894+0000 7f18a7400640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f188406b4d0 0x7f188406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:02.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.895+0000 7f18a7400640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f188406b4d0 0x7f188406d990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f189800a950 tx=0x7f1898008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:02.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:02.898+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f18a80ce7d0 con 0x7f18a80cd860 2024-04-25T10:19:03.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.128+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18a8073990 con 0x7f18a80cd860 2024-04-25T10:19:03.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.129+0000 7f189f400640 1 -- 172.21.15.165:0/3608398439 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f1890079d70 con 0x7f18a80cd860 2024-04-25T10:19:03.132 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:03.132 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:03.132 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:03.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.131+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f188406b4d0 msgr2=0x7f188406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.131+0000 7f18acc00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f188406b4d0 0x7f188406d990 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f189800a950 tx=0x7f1898008040 comp rx=0 tx=0).stop 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.131+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 msgr2=0x7f18a81e25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.131+0000 7f18acc00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 0x7f18a81e25f0 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f189002a2b0 tx=0x7f18900046c0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 shutdown_connections 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f188406b4d0 0x7f188406d990 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f18a8144440 0x7f18a80cad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f18a80cd860 0x7f18a81e25f0 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 --2- 172.21.15.165:0/3608398439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f18a8071530 0x7f18a81e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 >> 172.21.15.165:0/3608398439 conn(0x7f18a806abe0 msgr2=0x7f18a80b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 shutdown_connections 2024-04-25T10:19:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.132+0000 7f18acc00640 1 -- 172.21.15.165:0/3608398439 wait complete. 2024-04-25T10:19:03.154 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:03.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.342+0000 7f999f000640 1 Processor -- start 2024-04-25T10:19:03.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.342+0000 7f999f000640 1 -- start start 2024-04-25T10:19:03.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f999f000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f999815db70 0x7f99980cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f999f000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f99980cfc20 0x7f999806d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f999f000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9998071650 con 0x7f999815df50 2024-04-25T10:19:03.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f999f000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99980cbd80 con 0x7f999815db70 2024-04-25T10:19:03.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f999f000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9998161910 con 0x7f99980cfc20 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f9997e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f999815db70 0x7f99980cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f998f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f999815df50 0x7f99980cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54088/0 (socket says 172.21.15.165:54088) 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f998f400640 1 -- 172.21.15.165:0/2468269465 learned_addr learned my addr 172.21.15.165:0/2468269465 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f9997e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f999815db70 0x7f99980cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40884/0 (socket says 172.21.15.165:40884) 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.344+0000 7f999d000640 1 --2- 172.21.15.165:0/2468269465 >> v2:172.21.15.172:3300/0 conn(0x7f99980cfc20 0x7f999806d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.345+0000 7f9997e00640 1 -- 172.21.15.165:0/2468269465 >> v2:172.21.15.172:3300/0 conn(0x7f99980cfc20 msgr2=0x7f999806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.345+0000 7f9997e00640 1 --2- 172.21.15.165:0/2468269465 >> v2:172.21.15.172:3300/0 conn(0x7f99980cfc20 0x7f999806d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.345+0000 7f9997e00640 1 -- 172.21.15.165:0/2468269465 >> v1:172.21.15.165:6789/0 conn(0x7f999815df50 legacy=0x7f99980cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.345+0000 7f9997e00640 1 -- 172.21.15.165:0/2468269465 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99980cf230 con 0x7f999815db70 2024-04-25T10:19:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.345+0000 7f9997400640 1 -- 172.21.15.165:0/2468269465 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820508294 0 0) 0x7f9980001110 con 0x7f999815df50 2024-04-25T10:19:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.346+0000 7f9997e00640 1 --2- 172.21.15.165:0/2468269465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815db70 0x7f99980cb840 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f99980b4d10 tx=0x7f998802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ee6a5a655a7476 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.346+0000 7f9997400640 1 -- 172.21.15.165:0/2468269465 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988037070 con 0x7f999815db70 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.346+0000 7f9997400640 1 -- 172.21.15.165:0/2468269465 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9988032440 con 0x7f999815db70 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.346+0000 7f9997400640 1 -- 172.21.15.165:0/2468269465 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9988032810 con 0x7f999815db70 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 -- 172.21.15.165:0/2468269465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815db70 msgr2=0x7f99980cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 --2- 172.21.15.165:0/2468269465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815db70 0x7f99980cb840 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7f99980b4d10 tx=0x7f998802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 -- 172.21.15.165:0/2468269465 shutdown_connections 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 --2- 172.21.15.165:0/2468269465 >> v2:172.21.15.172:3300/0 conn(0x7f99980cfc20 0x7f999806d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 --2- 172.21.15.165:0/2468269465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815db70 0x7f99980cb840 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 -- 172.21.15.165:0/2468269465 >> 172.21.15.165:0/2468269465 conn(0x7f999806abe0 msgr2=0x7f999806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 -- 172.21.15.165:0/2468269465 shutdown_connections 2024-04-25T10:19:03.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.347+0000 7f999f000640 1 -- 172.21.15.165:0/2468269465 wait complete. 2024-04-25T10:19:03.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.349+0000 7f999f000640 1 Processor -- start 2024-04-25T10:19:03.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.349+0000 7f999f000640 1 -- start start 2024-04-25T10:19:03.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999f000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99980cfc20 0x7f999813fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999f000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f999815db70 0x7f99981e9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999f000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 0x7f9998140290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999f000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998162120 con 0x7f999815df50 2024-04-25T10:19:03.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999f000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9998161e40 con 0x7f99980cfc20 2024-04-25T10:19:03.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999f000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9998162640 con 0x7f999815db70 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 0x7f9998140290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 0x7f9998140290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40890/0 (socket says 172.21.15.165:40890) 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 -- 172.21.15.165:0/2359378702 learned_addr learned my addr 172.21.15.165:0/2359378702 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f998f400640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f999815db70 0x7f99981e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f9997e00640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99980cfc20 0x7f999813fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 -- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f999815db70 msgr2=0x7f99981e9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f999815db70 0x7f99981e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 -- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99980cfc20 msgr2=0x7f999813fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99980cfc20 0x7f999813fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.350+0000 7f999d000640 1 -- 172.21.15.165:0/2359378702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99981e8680 con 0x7f999815df50 2024-04-25T10:19:03.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.351+0000 7f999d000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 0x7f9998140290 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f9990007e60 tx=0x7f999000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.351+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990014420 con 0x7f999815df50 2024-04-25T10:19:03.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.351+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990019070 con 0x7f999815df50 2024-04-25T10:19:03.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.351+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990014820 con 0x7f999815df50 2024-04-25T10:19:03.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.351+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99981e8960 con 0x7f999815df50 2024-04-25T10:19:03.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.351+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99981e8f20 con 0x7f999815df50 2024-04-25T10:19:03.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.352+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9998073990 con 0x7f999815df50 2024-04-25T10:19:03.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.353+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99900149c0 con 0x7f999815df50 2024-04-25T10:19:03.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.354+0000 7f9994c00640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f997406b500 0x7f997406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.354+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f999008ac60 con 0x7f999815df50 2024-04-25T10:19:03.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.354+0000 7f9997e00640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f997406b500 0x7f997406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.355+0000 7f9997e00640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f997406b500 0x7f997406d9c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9988034000 tx=0x7f998802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.358+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99900598b0 con 0x7f999815df50 2024-04-25T10:19:03.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.558+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99980cbb40 con 0x7f999815df50 2024-04-25T10:19:03.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.560+0000 7f9994c00640 1 -- 172.21.15.165:0/2359378702 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16924 (secure 0 0 0) 0x7f99900591b0 con 0x7f999815df50 2024-04-25T10:19:03.563 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:03.563 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":171,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:00.610081+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:03.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.562+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f997406b500 msgr2=0x7f997406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.562+0000 7f999f000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f997406b500 0x7f997406d9c0 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f9988034000 tx=0x7f998802e000 comp rx=0 tx=0).stop 2024-04-25T10:19:03.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.562+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 msgr2=0x7f9998140290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.562+0000 7f999f000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 0x7f9998140290 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7f9990007e60 tx=0x7f999000e4b0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 shutdown_connections 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f997406b500 0x7f997406d9c0 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f999815df50 0x7f9998140290 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f999815db70 0x7f99981e9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 --2- 172.21.15.165:0/2359378702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99980cfc20 0x7f999813fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 >> 172.21.15.165:0/2359378702 conn(0x7f999806abe0 msgr2=0x7f999815e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 shutdown_connections 2024-04-25T10:19:03.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.563+0000 7f999f000640 1 -- 172.21.15.165:0/2359378702 wait complete. 2024-04-25T10:19:03.584 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:19:03.585 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:03.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.782+0000 7fc896200640 1 Processor -- start 2024-04-25T10:19:03.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.783+0000 7fc896200640 1 -- start start 2024-04-25T10:19:03.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.784+0000 7fc896200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc89006e430 0x7fc89006e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.784+0000 7fc896200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8900b3e10 con 0x7fc89015ec20 2024-04-25T10:19:03.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.784+0000 7fc896200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8900b3590 con 0x7fc89006abe0 2024-04-25T10:19:03.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.784+0000 7fc896200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8900b3850 con 0x7fc89006e430 2024-04-25T10:19:03.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.785+0000 7fc88ea00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc89015ec20 0x7fc8900b5270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54098/0 (socket says 172.21.15.165:54098) 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.785+0000 7fc88ea00640 1 -- 172.21.15.165:0/16408868 learned_addr learned my addr 172.21.15.165:0/16408868 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.785+0000 7fc88fe00640 1 --2- 172.21.15.165:0/16408868 >> v2:172.21.15.172:3300/0 conn(0x7fc89006e430 0x7fc89006e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.785+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654024219 0 0) 0x7fc878001110 con 0x7fc89015ec20 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.785+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc874003520 con 0x7fc89015ec20 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3890046129 0 0) 0x7fc874003520 con 0x7fc89015ec20 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 >> v2:172.21.15.172:3300/0 conn(0x7fc89006e430 msgr2=0x7fc89006e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 --2- 172.21.15.165:0/16408868 >> v2:172.21.15.172:3300/0 conn(0x7fc89006e430 0x7fc89006e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 >> v1:172.21.15.172:6789/0 conn(0x7fc89006abe0 legacy=0x7fc89006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8900b3b30 con 0x7fc89015ec20 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc878001840 con 0x7fc89015ec20 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 >> v1:172.21.15.165:6789/0 conn(0x7fc89015ec20 legacy=0x7fc8900b5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8740039b0 0x7fc874003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc874004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8740051c0 0x7fc87400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8900b3850 con 0x7fc874004800 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc87400c060 con 0x7fc8740051c0 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87400c260 con 0x7fc8740039b0 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88ea00640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc874004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.786+0000 7fc88f400640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8740039b0 0x7fc874003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88ea00640 1 -- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8740039b0 msgr2=0x7fc874003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88ea00640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8740039b0 0x7fc874003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88ea00640 1 -- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8740051c0 msgr2=0x7fc87400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88fe00640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8740051c0 0x7fc87400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88ea00640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc8740051c0 0x7fc87400b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88ea00640 1 -- 172.21.15.165:0/16408868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874003520 con 0x7fc874004800 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88fe00640 1 -- 172.21.15.165:0/16408868 reap_dead start 2024-04-25T10:19:03.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.787+0000 7fc88f400640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8740039b0 0x7fc874003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:03.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.788+0000 7fc88ea00640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc874004c80 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c4c0 tx=0x7fc874005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109731b0aa220130 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.788+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87800c0e0 con 0x7fc874004800 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.788+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc878034070 con 0x7fc874004800 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.788+0000 7fc88e000640 1 -- 172.21.15.165:0/16408868 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc878005d70 con 0x7fc874004800 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.788+0000 7fc896200640 1 -- 172.21.15.165:0/16408868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 msgr2=0x7fc874004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.788+0000 7fc896200640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc874004c80 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c4c0 tx=0x7fc874005830 comp rx=0 tx=0).stop 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.789+0000 7fc896200640 1 -- 172.21.15.165:0/16408868 shutdown_connections 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.789+0000 7fc896200640 1 --2- 172.21.15.165:0/16408868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc874004c80 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.789+0000 7fc896200640 1 -- 172.21.15.165:0/16408868 >> 172.21.15.165:0/16408868 conn(0x7fc8900b5af0 msgr2=0x7fc8900b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.789+0000 7fc896200640 1 -- 172.21.15.165:0/16408868 shutdown_connections 2024-04-25T10:19:03.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.789+0000 7fc896200640 1 -- 172.21.15.165:0/16408868 wait complete. 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.790+0000 7fc896200640 1 Processor -- start 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.791+0000 7fc896200640 1 -- start start 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.791+0000 7fc896200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc890157650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc896200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc89009e0f0 0x7fc8901f71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc896200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc890145d60 0x7fc8901461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc896200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89015f0b0 con 0x7fc874004800 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc896200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89015e700 con 0x7fc89009e0f0 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc896200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89015f230 con 0x7fc890145d60 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc890157650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc890157650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40912/0 (socket says 172.21.15.165:40912) 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 -- 172.21.15.165:0/2030979604 learned_addr learned my addr 172.21.15.165:0/2030979604 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:03.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88fe00640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc890145d60 0x7fc8901461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88ea00640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc89009e0f0 0x7fc8901f71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 -- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc890145d60 msgr2=0x7fc8901461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc890145d60 0x7fc8901461e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 -- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc89009e0f0 msgr2=0x7fc8901f71f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc89009e0f0 0x7fc8901f71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.792+0000 7fc88f400640 1 -- 172.21.15.165:0/2030979604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc890157c30 con 0x7fc874004800 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.793+0000 7fc88f400640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc890157650 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fc88800d800 tx=0x7fc88800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.793+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888003da0 con 0x7fc874004800 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.793+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888017070 con 0x7fc874004800 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.793+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88800b8f0 con 0x7fc874004800 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.793+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc890157f10 con 0x7fc874004800 2024-04-25T10:19:03.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.793+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8901584d0 con 0x7fc874004800 2024-04-25T10:19:03.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.794+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89006cc80 con 0x7fc874004800 2024-04-25T10:19:03.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.795+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc88800ba90 con 0x7fc874004800 2024-04-25T10:19:03.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.796+0000 7fc887400640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc86c06b500 0x7fc86c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:03.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.796+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc88808a550 con 0x7fc874004800 2024-04-25T10:19:03.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.796+0000 7fc88ea00640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc86c06b500 0x7fc86c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:03.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.797+0000 7fc88ea00640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc86c06b500 0x7fc86c06d9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c490 tx=0x7fc87802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:03.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.801+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc8880591a0 con 0x7fc874004800 2024-04-25T10:19:03.970 INFO:tasks.ceph.mds.a.smithi165.stderr:2024-04-25T10:19:03.967+0000 7fc301c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:19:03.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:03.995+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc890073990 con 0x7fc86c06b500 2024-04-25T10:19:04.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.013+0000 7fc887400640 1 -- 172.21.15.165:0/2030979604 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252489 (secure 0 0 0) 0x7fc890073990 con 0x7fc86c06b500 2024-04-25T10:19:04.019 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:04.020 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:19:04.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc86c06b500 msgr2=0x7fc86c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc86c06b500 0x7fc86c06d9c0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fc87800c490 tx=0x7fc87802e000 comp rx=0 tx=0).stop 2024-04-25T10:19:04.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 msgr2=0x7fc890157650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc890157650 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7fc88800d800 tx=0x7fc88800dcc0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 shutdown_connections 2024-04-25T10:19:04.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc86c06b500 0x7fc86c06d9c0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc890145d60 0x7fc8901461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.018+0000 7fc896200640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc89009e0f0 0x7fc8901f71f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.019+0000 7fc896200640 1 --2- 172.21.15.165:0/2030979604 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc874004800 0x7fc890157650 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.019+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 >> 172.21.15.165:0/2030979604 conn(0x7fc8900b5af0 msgr2=0x7fc8900b0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.019+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 shutdown_connections 2024-04-25T10:19:04.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.019+0000 7fc896200640 1 -- 172.21.15.165:0/2030979604 wait complete. 2024-04-25T10:19:04.040 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":679,"stamp":"2024-04-25T10:19:02.476929+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517268,"kb_used_data":18056,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754457452,"statfs":{"total":773094113280,"available":772564430848,"internally_reserved":0,"allocated":18489344,"data_stored":9287460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.904362"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218068+0000","last_change":"2024-04-25T10:18:59.628251+0000","last_active":"2024-04-25T10:19:01.218068+0000","last_peered":"2024-04-25T10:19:01.218068+0000","last_clean":"2024-04-25T10:19:01.218068+0000","last_became_active":"2024-04-25T10:18:59.627522+0000","last_became_peered":"2024-04-25T10:18:59.627522+0000","last_unstale":"2024-04-25T10:19:01.218068+0000","last_undegraded":"2024-04-25T10:19:01.218068+0000","last_fullsized":"2024-04-25T10:19:01.218068+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:42.780434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218134+0000","last_change":"2024-04-25T10:18:59.625093+0000","last_active":"2024-04-25T10:19:01.218134+0000","last_peered":"2024-04-25T10:19:01.218134+0000","last_clean":"2024-04-25T10:19:01.218134+0000","last_became_active":"2024-04-25T10:18:59.624810+0000","last_became_peered":"2024-04-25T10:18:59.624810+0000","last_unstale":"2024-04-25T10:19:01.218134+0000","last_undegraded":"2024-04-25T10:19:01.218134+0000","last_fullsized":"2024-04-25T10:19:01.218134+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:21.633460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633700+0000","last_change":"2024-04-25T10:18:59.617331+0000","last_active":"2024-04-25T10:19:00.633700+0000","last_peered":"2024-04-25T10:19:00.633700+0000","last_clean":"2024-04-25T10:19:00.633700+0000","last_became_active":"2024-04-25T10:18:59.616984+0000","last_became_peered":"2024-04-25T10:18:59.616984+0000","last_unstale":"2024-04-25T10:19:00.633700+0000","last_undegraded":"2024-04-25T10:19:00.633700+0000","last_fullsized":"2024-04-25T10:19:00.633700+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:20.245838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215700+0000","last_change":"2024-04-25T10:18:59.612826+0000","last_active":"2024-04-25T10:19:01.215700+0000","last_peered":"2024-04-25T10:19:01.215700+0000","last_clean":"2024-04-25T10:19:01.215700+0000","last_became_active":"2024-04-25T10:18:59.612522+0000","last_became_peered":"2024-04-25T10:18:59.612522+0000","last_unstale":"2024-04-25T10:19:01.215700+0000","last_undegraded":"2024-04-25T10:19:01.215700+0000","last_fullsized":"2024-04-25T10:19:01.215700+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:17.891340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626207+0000","last_change":"2024-04-25T10:18:59.632895+0000","last_active":"2024-04-25T10:19:00.626207+0000","last_peered":"2024-04-25T10:19:00.626207+0000","last_clean":"2024-04-25T10:19:00.626207+0000","last_became_active":"2024-04-25T10:18:59.632597+0000","last_became_peered":"2024-04-25T10:18:59.632597+0000","last_unstale":"2024-04-25T10:19:00.626207+0000","last_undegraded":"2024-04-25T10:19:00.626207+0000","last_fullsized":"2024-04-25T10:19:00.626207+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:13.168086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626249+0000","last_change":"2024-04-25T10:18:59.612739+0000","last_active":"2024-04-25T10:19:00.626249+0000","last_peered":"2024-04-25T10:19:00.626249+0000","last_clean":"2024-04-25T10:19:00.626249+0000","last_became_active":"2024-04-25T10:18:59.612431+0000","last_became_peered":"2024-04-25T10:18:59.612431+0000","last_unstale":"2024-04-25T10:19:00.626249+0000","last_undegraded":"2024-04-25T10:19:00.626249+0000","last_fullsized":"2024-04-25T10:19:00.626249+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:56.354084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942128+0000","last_change":"2024-04-25T10:18:59.623749+0000","last_active":"2024-04-25T10:19:00.942128+0000","last_peered":"2024-04-25T10:19:00.942128+0000","last_clean":"2024-04-25T10:19:00.942128+0000","last_became_active":"2024-04-25T10:18:59.623413+0000","last_became_peered":"2024-04-25T10:18:59.623413+0000","last_unstale":"2024-04-25T10:19:00.942128+0000","last_undegraded":"2024-04-25T10:19:00.942128+0000","last_fullsized":"2024-04-25T10:19:00.942128+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:36.865218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217952+0000","last_change":"2024-04-25T10:18:59.640283+0000","last_active":"2024-04-25T10:19:01.217952+0000","last_peered":"2024-04-25T10:19:01.217952+0000","last_clean":"2024-04-25T10:19:01.217952+0000","last_became_active":"2024-04-25T10:18:59.639971+0000","last_became_peered":"2024-04-25T10:18:59.639971+0000","last_unstale":"2024-04-25T10:19:01.217952+0000","last_undegraded":"2024-04-25T10:19:01.217952+0000","last_fullsized":"2024-04-25T10:19:01.217952+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:09.103209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218328+0000","last_change":"2024-04-25T10:18:59.621167+0000","last_active":"2024-04-25T10:19:01.218328+0000","last_peered":"2024-04-25T10:19:01.218328+0000","last_clean":"2024-04-25T10:19:01.218328+0000","last_became_active":"2024-04-25T10:18:59.620854+0000","last_became_peered":"2024-04-25T10:18:59.620854+0000","last_unstale":"2024-04-25T10:19:01.218328+0000","last_undegraded":"2024-04-25T10:19:01.218328+0000","last_fullsized":"2024-04-25T10:19:01.218328+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.473398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627250+0000","last_change":"2024-04-25T10:18:59.626866+0000","last_active":"2024-04-25T10:19:00.627250+0000","last_peered":"2024-04-25T10:19:00.627250+0000","last_clean":"2024-04-25T10:19:00.627250+0000","last_became_active":"2024-04-25T10:18:59.626541+0000","last_became_peered":"2024-04-25T10:18:59.626541+0000","last_unstale":"2024-04-25T10:19:00.627250+0000","last_undegraded":"2024-04-25T10:19:00.627250+0000","last_fullsized":"2024-04-25T10:19:00.627250+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:14.974811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216990+0000","last_change":"2024-04-25T10:18:59.624601+0000","last_active":"2024-04-25T10:19:01.216990+0000","last_peered":"2024-04-25T10:19:01.216990+0000","last_clean":"2024-04-25T10:19:01.216990+0000","last_became_active":"2024-04-25T10:18:59.624293+0000","last_became_peered":"2024-04-25T10:18:59.624293+0000","last_unstale":"2024-04-25T10:19:01.216990+0000","last_undegraded":"2024-04-25T10:19:01.216990+0000","last_fullsized":"2024-04-25T10:19:01.216990+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:33.592556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215976+0000","last_change":"2024-04-25T10:18:59.623644+0000","last_active":"2024-04-25T10:19:01.215976+0000","last_peered":"2024-04-25T10:19:01.215976+0000","last_clean":"2024-04-25T10:19:01.215976+0000","last_became_active":"2024-04-25T10:18:59.623353+0000","last_became_peered":"2024-04-25T10:18:59.623353+0000","last_unstale":"2024-04-25T10:19:01.215976+0000","last_undegraded":"2024-04-25T10:19:01.215976+0000","last_fullsized":"2024-04-25T10:19:01.215976+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:32.500708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631639+0000","last_change":"2024-04-25T10:18:59.618958+0000","last_active":"2024-04-25T10:19:00.631639+0000","last_peered":"2024-04-25T10:19:00.631639+0000","last_clean":"2024-04-25T10:19:00.631639+0000","last_became_active":"2024-04-25T10:18:59.616212+0000","last_became_peered":"2024-04-25T10:18:59.616212+0000","last_unstale":"2024-04-25T10:19:00.631639+0000","last_undegraded":"2024-04-25T10:19:00.631639+0000","last_fullsized":"2024-04-25T10:19:00.631639+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:45.555838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942292+0000","last_change":"2024-04-25T10:18:59.619440+0000","last_active":"2024-04-25T10:19:00.942292+0000","last_peered":"2024-04-25T10:19:00.942292+0000","last_clean":"2024-04-25T10:19:00.942292+0000","last_became_active":"2024-04-25T10:18:59.618697+0000","last_became_peered":"2024-04-25T10:18:59.618697+0000","last_unstale":"2024-04-25T10:19:00.942292+0000","last_undegraded":"2024-04-25T10:19:00.942292+0000","last_fullsized":"2024-04-25T10:19:00.942292+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:36.625781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217239+0000","last_change":"2024-04-25T10:18:59.622081+0000","last_active":"2024-04-25T10:19:01.217239+0000","last_peered":"2024-04-25T10:19:01.217239+0000","last_clean":"2024-04-25T10:19:01.217239+0000","last_became_active":"2024-04-25T10:18:59.621807+0000","last_became_peered":"2024-04-25T10:18:59.621807+0000","last_unstale":"2024-04-25T10:19:01.217239+0000","last_undegraded":"2024-04-25T10:19:01.217239+0000","last_fullsized":"2024-04-25T10:19:01.217239+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:52.481359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626145+0000","last_change":"2024-04-25T10:18:59.637931+0000","last_active":"2024-04-25T10:19:00.626145+0000","last_peered":"2024-04-25T10:19:00.626145+0000","last_clean":"2024-04-25T10:19:00.626145+0000","last_became_active":"2024-04-25T10:18:59.636825+0000","last_became_peered":"2024-04-25T10:18:59.636825+0000","last_unstale":"2024-04-25T10:19:00.626145+0000","last_undegraded":"2024-04-25T10:19:00.626145+0000","last_fullsized":"2024-04-25T10:19:00.626145+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.660713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627485+0000","last_change":"2024-04-25T10:18:59.626920+0000","last_active":"2024-04-25T10:19:00.627485+0000","last_peered":"2024-04-25T10:19:00.627485+0000","last_clean":"2024-04-25T10:19:00.627485+0000","last_became_active":"2024-04-25T10:18:59.626296+0000","last_became_peered":"2024-04-25T10:18:59.626296+0000","last_unstale":"2024-04-25T10:19:00.627485+0000","last_undegraded":"2024-04-25T10:19:00.627485+0000","last_fullsized":"2024-04-25T10:19:00.627485+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:00.761607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633113+0000","last_change":"2024-04-25T10:18:59.628149+0000","last_active":"2024-04-25T10:19:00.633113+0000","last_peered":"2024-04-25T10:19:00.633113+0000","last_clean":"2024-04-25T10:19:00.633113+0000","last_became_active":"2024-04-25T10:18:59.627561+0000","last_became_peered":"2024-04-25T10:18:59.627561+0000","last_unstale":"2024-04-25T10:19:00.633113+0000","last_undegraded":"2024-04-25T10:19:00.633113+0000","last_fullsized":"2024-04-25T10:19:00.633113+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:52.919451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.632726+0000","last_change":"2024-04-25T10:18:59.618401+0000","last_active":"2024-04-25T10:19:00.632726+0000","last_peered":"2024-04-25T10:19:00.632726+0000","last_clean":"2024-04-25T10:19:00.632726+0000","last_became_active":"2024-04-25T10:18:59.617451+0000","last_became_peered":"2024-04-25T10:18:59.617451+0000","last_unstale":"2024-04-25T10:19:00.632726+0000","last_undegraded":"2024-04-25T10:19:00.632726+0000","last_fullsized":"2024-04-25T10:19:00.632726+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:28.628728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626259+0000","last_change":"2024-04-25T10:18:59.629192+0000","last_active":"2024-04-25T10:19:00.626259+0000","last_peered":"2024-04-25T10:19:00.626259+0000","last_clean":"2024-04-25T10:19:00.626259+0000","last_became_active":"2024-04-25T10:18:59.628926+0000","last_became_peered":"2024-04-25T10:18:59.628926+0000","last_unstale":"2024-04-25T10:19:00.626259+0000","last_undegraded":"2024-04-25T10:19:00.626259+0000","last_fullsized":"2024-04-25T10:19:00.626259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:56.524045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217965+0000","last_change":"2024-04-25T10:18:59.617258+0000","last_active":"2024-04-25T10:19:01.217965+0000","last_peered":"2024-04-25T10:19:01.217965+0000","last_clean":"2024-04-25T10:19:01.217965+0000","last_became_active":"2024-04-25T10:18:59.616981+0000","last_became_peered":"2024-04-25T10:18:59.616981+0000","last_unstale":"2024-04-25T10:19:01.217965+0000","last_undegraded":"2024-04-25T10:19:01.217965+0000","last_fullsized":"2024-04-25T10:19:01.217965+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.083207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217235+0000","last_change":"2024-04-25T10:18:59.623143+0000","last_active":"2024-04-25T10:19:01.217235+0000","last_peered":"2024-04-25T10:19:01.217235+0000","last_clean":"2024-04-25T10:19:01.217235+0000","last_became_active":"2024-04-25T10:18:59.621725+0000","last_became_peered":"2024-04-25T10:18:59.621725+0000","last_unstale":"2024-04-25T10:19:01.217235+0000","last_undegraded":"2024-04-25T10:19:01.217235+0000","last_fullsized":"2024-04-25T10:19:01.217235+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:39.097033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216313+0000","last_change":"2024-04-25T10:18:59.628513+0000","last_active":"2024-04-25T10:19:01.216313+0000","last_peered":"2024-04-25T10:19:01.216313+0000","last_clean":"2024-04-25T10:19:01.216313+0000","last_became_active":"2024-04-25T10:18:59.627330+0000","last_became_peered":"2024-04-25T10:18:59.627330+0000","last_unstale":"2024-04-25T10:19:01.216313+0000","last_undegraded":"2024-04-25T10:19:01.216313+0000","last_fullsized":"2024-04-25T10:19:01.216313+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.083207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216714+0000","last_change":"2024-04-25T10:18:59.619247+0000","last_active":"2024-04-25T10:19:01.216714+0000","last_peered":"2024-04-25T10:19:01.216714+0000","last_clean":"2024-04-25T10:19:01.216714+0000","last_became_active":"2024-04-25T10:18:59.618966+0000","last_became_peered":"2024-04-25T10:18:59.618966+0000","last_unstale":"2024-04-25T10:19:01.216714+0000","last_undegraded":"2024-04-25T10:19:01.216714+0000","last_fullsized":"2024-04-25T10:19:01.216714+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:38.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942872+0000","last_change":"2024-04-25T10:18:59.620906+0000","last_active":"2024-04-25T10:19:00.942872+0000","last_peered":"2024-04-25T10:19:00.942872+0000","last_clean":"2024-04-25T10:19:00.942872+0000","last_became_active":"2024-04-25T10:18:59.620518+0000","last_became_peered":"2024-04-25T10:18:59.620518+0000","last_unstale":"2024-04-25T10:19:00.942872+0000","last_undegraded":"2024-04-25T10:19:00.942872+0000","last_fullsized":"2024-04-25T10:19:00.942872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:05.005848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216314+0000","last_change":"2024-04-25T10:18:59.637999+0000","last_active":"2024-04-25T10:19:01.216314+0000","last_peered":"2024-04-25T10:19:01.216314+0000","last_clean":"2024-04-25T10:19:01.216314+0000","last_became_active":"2024-04-25T10:18:59.637370+0000","last_became_peered":"2024-04-25T10:18:59.637370+0000","last_unstale":"2024-04-25T10:19:01.216314+0000","last_undegraded":"2024-04-25T10:19:01.216314+0000","last_fullsized":"2024-04-25T10:19:01.216314+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.916361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217670+0000","last_change":"2024-04-25T10:18:59.637919+0000","last_active":"2024-04-25T10:19:01.217670+0000","last_peered":"2024-04-25T10:19:01.217670+0000","last_clean":"2024-04-25T10:19:01.217670+0000","last_became_active":"2024-04-25T10:18:59.637588+0000","last_became_peered":"2024-04-25T10:18:59.637588+0000","last_unstale":"2024-04-25T10:19:01.217670+0000","last_undegraded":"2024-04-25T10:19:01.217670+0000","last_fullsized":"2024-04-25T10:19:01.217670+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:20.432975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626883+0000","last_change":"2024-04-25T10:18:59.632740+0000","last_active":"2024-04-25T10:19:00.626883+0000","last_peered":"2024-04-25T10:19:00.626883+0000","last_clean":"2024-04-25T10:19:00.626883+0000","last_became_active":"2024-04-25T10:18:59.632413+0000","last_became_peered":"2024-04-25T10:18:59.632413+0000","last_unstale":"2024-04-25T10:19:00.626883+0000","last_undegraded":"2024-04-25T10:19:00.626883+0000","last_fullsized":"2024-04-25T10:19:00.626883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:22.550434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216374+0000","last_change":"2024-04-25T10:18:59.633756+0000","last_active":"2024-04-25T10:19:01.216374+0000","last_peered":"2024-04-25T10:19:01.216374+0000","last_clean":"2024-04-25T10:19:01.216374+0000","last_became_active":"2024-04-25T10:18:59.633446+0000","last_became_peered":"2024-04-25T10:18:59.633446+0000","last_unstale":"2024-04-25T10:19:01.216374+0000","last_undegraded":"2024-04-25T10:19:01.216374+0000","last_fullsized":"2024-04-25T10:19:01.216374+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:41.444345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218358+0000","last_change":"2024-04-25T10:18:59.624003+0000","last_active":"2024-04-25T10:19:01.218358+0000","last_peered":"2024-04-25T10:19:01.218358+0000","last_clean":"2024-04-25T10:19:01.218358+0000","last_became_active":"2024-04-25T10:18:59.623674+0000","last_became_peered":"2024-04-25T10:18:59.623674+0000","last_unstale":"2024-04-25T10:19:01.218358+0000","last_undegraded":"2024-04-25T10:19:01.218358+0000","last_fullsized":"2024-04-25T10:19:01.218358+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:59.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942223+0000","last_change":"2024-04-25T10:18:59.618835+0000","last_active":"2024-04-25T10:19:00.942223+0000","last_peered":"2024-04-25T10:19:00.942223+0000","last_clean":"2024-04-25T10:19:00.942223+0000","last_became_active":"2024-04-25T10:18:59.618525+0000","last_became_peered":"2024-04-25T10:18:59.618525+0000","last_unstale":"2024-04-25T10:19:00.942223+0000","last_undegraded":"2024-04-25T10:19:00.942223+0000","last_fullsized":"2024-04-25T10:19:00.942223+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:26.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626053+0000","last_change":"2024-04-25T10:18:59.629650+0000","last_active":"2024-04-25T10:19:00.626053+0000","last_peered":"2024-04-25T10:19:00.626053+0000","last_clean":"2024-04-25T10:19:00.626053+0000","last_became_active":"2024-04-25T10:18:59.625699+0000","last_became_peered":"2024-04-25T10:18:59.625699+0000","last_unstale":"2024-04-25T10:19:00.626053+0000","last_undegraded":"2024-04-25T10:19:00.626053+0000","last_fullsized":"2024-04-25T10:19:00.626053+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.957316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217380+0000","last_change":"2024-04-25T10:18:59.638243+0000","last_active":"2024-04-25T10:19:01.217380+0000","last_peered":"2024-04-25T10:19:01.217380+0000","last_clean":"2024-04-25T10:19:01.217380+0000","last_became_active":"2024-04-25T10:18:59.636046+0000","last_became_peered":"2024-04-25T10:18:59.636046+0000","last_unstale":"2024-04-25T10:19:01.217380+0000","last_undegraded":"2024-04-25T10:19:01.217380+0000","last_fullsized":"2024-04-25T10:19:01.217380+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:21.633460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"170'4","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216781+0000","last_change":"2024-04-25T10:18:58.626021+0000","last_active":"2024-04-25T10:19:01.216781+0000","last_peered":"2024-04-25T10:19:01.216781+0000","last_clean":"2024-04-25T10:19:01.216781+0000","last_became_active":"2024-04-25T10:18:58.623471+0000","last_became_peered":"2024-04-25T10:18:58.623471+0000","last_unstale":"2024-04-25T10:19:01.216781+0000","last_undegraded":"2024-04-25T10:19:01.216781+0000","last_fullsized":"2024-04-25T10:19:01.216781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:41.765484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942765+0000","last_change":"2024-04-25T10:18:59.627188+0000","last_active":"2024-04-25T10:19:00.942765+0000","last_peered":"2024-04-25T10:19:00.942765+0000","last_clean":"2024-04-25T10:19:00.942765+0000","last_became_active":"2024-04-25T10:18:59.625966+0000","last_became_peered":"2024-04-25T10:18:59.625966+0000","last_unstale":"2024-04-25T10:19:00.942765+0000","last_undegraded":"2024-04-25T10:19:00.942765+0000","last_fullsized":"2024-04-25T10:19:00.942765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:52.095423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627493+0000","last_change":"2024-04-25T10:18:58.620716+0000","last_active":"2024-04-25T10:19:00.627493+0000","last_peered":"2024-04-25T10:19:00.627493+0000","last_clean":"2024-04-25T10:19:00.627493+0000","last_became_active":"2024-04-25T10:18:58.620384+0000","last_became_peered":"2024-04-25T10:18:58.620384+0000","last_unstale":"2024-04-25T10:19:00.627493+0000","last_undegraded":"2024-04-25T10:19:00.627493+0000","last_fullsized":"2024-04-25T10:19:00.627493+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:46.877771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216714+0000","last_change":"2024-04-25T10:18:59.638215+0000","last_active":"2024-04-25T10:19:01.216714+0000","last_peered":"2024-04-25T10:19:01.216714+0000","last_clean":"2024-04-25T10:19:01.216714+0000","last_became_active":"2024-04-25T10:18:59.637734+0000","last_became_peered":"2024-04-25T10:18:59.637734+0000","last_unstale":"2024-04-25T10:19:01.216714+0000","last_undegraded":"2024-04-25T10:19:01.216714+0000","last_fullsized":"2024-04-25T10:19:01.216714+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:27.444607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217469+0000","last_change":"2024-04-25T10:18:58.622801+0000","last_active":"2024-04-25T10:19:01.217469+0000","last_peered":"2024-04-25T10:19:01.217469+0000","last_clean":"2024-04-25T10:19:01.217469+0000","last_became_active":"2024-04-25T10:18:58.621805+0000","last_became_peered":"2024-04-25T10:18:58.621805+0000","last_unstale":"2024-04-25T10:19:01.217469+0000","last_undegraded":"2024-04-25T10:19:01.217469+0000","last_fullsized":"2024-04-25T10:19:01.217469+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:55.106290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633762+0000","last_change":"2024-04-25T10:18:59.633505+0000","last_active":"2024-04-25T10:19:00.633762+0000","last_peered":"2024-04-25T10:19:00.633762+0000","last_clean":"2024-04-25T10:19:00.633762+0000","last_became_active":"2024-04-25T10:18:59.633215+0000","last_became_peered":"2024-04-25T10:18:59.633215+0000","last_unstale":"2024-04-25T10:19:00.633762+0000","last_undegraded":"2024-04-25T10:19:00.633762+0000","last_fullsized":"2024-04-25T10:19:00.633762+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.344598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.632881+0000","last_change":"2024-04-25T10:18:58.616032+0000","last_active":"2024-04-25T10:19:00.632881+0000","last_peered":"2024-04-25T10:19:00.632881+0000","last_clean":"2024-04-25T10:19:00.632881+0000","last_became_active":"2024-04-25T10:18:58.615698+0000","last_became_peered":"2024-04-25T10:18:58.615698+0000","last_unstale":"2024-04-25T10:19:00.632881+0000","last_undegraded":"2024-04-25T10:19:00.632881+0000","last_fullsized":"2024-04-25T10:19:00.632881+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:40.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.630587+0000","last_change":"2024-04-25T10:18:59.615359+0000","last_active":"2024-04-25T10:19:00.630587+0000","last_peered":"2024-04-25T10:19:00.630587+0000","last_clean":"2024-04-25T10:19:00.630587+0000","last_became_active":"2024-04-25T10:18:59.615061+0000","last_became_peered":"2024-04-25T10:18:59.615061+0000","last_unstale":"2024-04-25T10:19:00.630587+0000","last_undegraded":"2024-04-25T10:19:00.630587+0000","last_fullsized":"2024-04-25T10:19:00.630587+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:12.546528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626695+0000","last_change":"2024-04-25T10:18:58.619517+0000","last_active":"2024-04-25T10:19:00.626695+0000","last_peered":"2024-04-25T10:19:00.626695+0000","last_clean":"2024-04-25T10:19:00.626695+0000","last_became_active":"2024-04-25T10:18:58.619252+0000","last_became_peered":"2024-04-25T10:18:58.619252+0000","last_unstale":"2024-04-25T10:19:00.626695+0000","last_undegraded":"2024-04-25T10:19:00.626695+0000","last_fullsized":"2024-04-25T10:19:00.626695+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:13.470612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217251+0000","last_change":"2024-04-25T10:18:59.613882+0000","last_active":"2024-04-25T10:19:01.217251+0000","last_peered":"2024-04-25T10:19:01.217251+0000","last_clean":"2024-04-25T10:19:01.217251+0000","last_became_active":"2024-04-25T10:18:59.613080+0000","last_became_peered":"2024-04-25T10:18:59.613080+0000","last_unstale":"2024-04-25T10:19:01.217251+0000","last_undegraded":"2024-04-25T10:19:01.217251+0000","last_fullsized":"2024-04-25T10:19:01.217251+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:54.795683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216223+0000","last_change":"2024-04-25T10:18:58.619967+0000","last_active":"2024-04-25T10:19:01.216223+0000","last_peered":"2024-04-25T10:19:01.216223+0000","last_clean":"2024-04-25T10:19:01.216223+0000","last_became_active":"2024-04-25T10:18:58.619696+0000","last_became_peered":"2024-04-25T10:18:58.619696+0000","last_unstale":"2024-04-25T10:19:01.216223+0000","last_undegraded":"2024-04-25T10:19:01.216223+0000","last_fullsized":"2024-04-25T10:19:01.216223+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:04.934349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218269+0000","last_change":"2024-04-25T10:18:59.624128+0000","last_active":"2024-04-25T10:19:01.218269+0000","last_peered":"2024-04-25T10:19:01.218269+0000","last_clean":"2024-04-25T10:19:01.218269+0000","last_became_active":"2024-04-25T10:18:59.623842+0000","last_became_peered":"2024-04-25T10:18:59.623842+0000","last_unstale":"2024-04-25T10:19:01.218269+0000","last_undegraded":"2024-04-25T10:19:01.218269+0000","last_fullsized":"2024-04-25T10:19:01.218269+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:20.432975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.630806+0000","last_change":"2024-04-25T10:18:58.619968+0000","last_active":"2024-04-25T10:19:00.630806+0000","last_peered":"2024-04-25T10:19:00.630806+0000","last_clean":"2024-04-25T10:19:00.630806+0000","last_became_active":"2024-04-25T10:18:58.619279+0000","last_became_peered":"2024-04-25T10:18:58.619279+0000","last_unstale":"2024-04-25T10:19:00.630806+0000","last_undegraded":"2024-04-25T10:19:00.630806+0000","last_fullsized":"2024-04-25T10:19:00.630806+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:40.933234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217696+0000","last_change":"2024-04-25T10:18:59.639191+0000","last_active":"2024-04-25T10:19:01.217696+0000","last_peered":"2024-04-25T10:19:01.217696+0000","last_clean":"2024-04-25T10:19:01.217696+0000","last_became_active":"2024-04-25T10:18:59.638849+0000","last_became_peered":"2024-04-25T10:18:59.638849+0000","last_unstale":"2024-04-25T10:19:01.217696+0000","last_undegraded":"2024-04-25T10:19:01.217696+0000","last_fullsized":"2024-04-25T10:19:01.217696+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.473398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627420+0000","last_change":"2024-04-25T10:18:58.616113+0000","last_active":"2024-04-25T10:19:00.627420+0000","last_peered":"2024-04-25T10:19:00.627420+0000","last_clean":"2024-04-25T10:19:00.627420+0000","last_became_active":"2024-04-25T10:18:58.614173+0000","last_became_peered":"2024-04-25T10:18:58.614173+0000","last_unstale":"2024-04-25T10:19:00.627420+0000","last_undegraded":"2024-04-25T10:19:00.627420+0000","last_fullsized":"2024-04-25T10:19:00.627420+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:56.460787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627060+0000","last_change":"2024-04-25T10:18:59.618999+0000","last_active":"2024-04-25T10:19:00.627060+0000","last_peered":"2024-04-25T10:19:00.627060+0000","last_clean":"2024-04-25T10:19:00.627060+0000","last_became_active":"2024-04-25T10:18:59.615816+0000","last_became_peered":"2024-04-25T10:18:59.615816+0000","last_unstale":"2024-04-25T10:19:00.627060+0000","last_undegraded":"2024-04-25T10:19:00.627060+0000","last_fullsized":"2024-04-25T10:19:00.627060+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:29.284669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218434+0000","last_change":"2024-04-25T10:18:58.623351+0000","last_active":"2024-04-25T10:19:01.218434+0000","last_peered":"2024-04-25T10:19:01.218434+0000","last_clean":"2024-04-25T10:19:01.218434+0000","last_became_active":"2024-04-25T10:18:58.623081+0000","last_became_peered":"2024-04-25T10:18:58.623081+0000","last_unstale":"2024-04-25T10:19:01.218434+0000","last_undegraded":"2024-04-25T10:19:01.218434+0000","last_fullsized":"2024-04-25T10:19:01.218434+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:55.106290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633592+0000","last_change":"2024-04-25T10:18:59.627885+0000","last_active":"2024-04-25T10:19:00.633592+0000","last_peered":"2024-04-25T10:19:00.633592+0000","last_clean":"2024-04-25T10:19:00.633592+0000","last_became_active":"2024-04-25T10:18:59.626069+0000","last_became_peered":"2024-04-25T10:18:59.626069+0000","last_unstale":"2024-04-25T10:19:00.633592+0000","last_undegraded":"2024-04-25T10:19:00.633592+0000","last_fullsized":"2024-04-25T10:19:00.633592+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:46.262628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"170'3","reported_seq":17,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216238+0000","last_change":"2024-04-25T10:18:58.625180+0000","last_active":"2024-04-25T10:19:01.216238+0000","last_peered":"2024-04-25T10:19:01.216238+0000","last_clean":"2024-04-25T10:19:01.216238+0000","last_became_active":"2024-04-25T10:18:58.624162+0000","last_became_peered":"2024-04-25T10:18:58.624162+0000","last_unstale":"2024-04-25T10:19:01.216238+0000","last_undegraded":"2024-04-25T10:19:01.216238+0000","last_fullsized":"2024-04-25T10:19:01.216238+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:08.086161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218344+0000","last_change":"2024-04-25T10:18:59.633886+0000","last_active":"2024-04-25T10:19:01.218344+0000","last_peered":"2024-04-25T10:19:01.218344+0000","last_clean":"2024-04-25T10:19:01.218344+0000","last_became_active":"2024-04-25T10:18:59.633490+0000","last_became_peered":"2024-04-25T10:18:59.633490+0000","last_unstale":"2024-04-25T10:19:01.218344+0000","last_undegraded":"2024-04-25T10:19:01.218344+0000","last_fullsized":"2024-04-25T10:19:01.218344+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:09.103209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"170'6","reported_seq":20,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633183+0000","last_change":"2024-04-25T10:18:58.610641+0000","last_active":"2024-04-25T10:19:00.633183+0000","last_peered":"2024-04-25T10:19:00.633183+0000","last_clean":"2024-04-25T10:19:00.633183+0000","last_became_active":"2024-04-25T10:18:58.610329+0000","last_became_peered":"2024-04-25T10:18:58.610329+0000","last_unstale":"2024-04-25T10:19:00.633183+0000","last_undegraded":"2024-04-25T10:19:00.633183+0000","last_fullsized":"2024-04-25T10:19:00.633183+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:54.374547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942044+0000","last_change":"2024-04-25T10:18:59.619818+0000","last_active":"2024-04-25T10:19:00.942044+0000","last_peered":"2024-04-25T10:19:00.942044+0000","last_clean":"2024-04-25T10:19:00.942044+0000","last_became_active":"2024-04-25T10:18:59.619298+0000","last_became_peered":"2024-04-25T10:18:59.619298+0000","last_unstale":"2024-04-25T10:19:00.942044+0000","last_undegraded":"2024-04-25T10:19:00.942044+0000","last_fullsized":"2024-04-25T10:19:00.942044+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.093918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631905+0000","last_change":"2024-04-25T10:18:58.611475+0000","last_active":"2024-04-25T10:19:00.631905+0000","last_peered":"2024-04-25T10:19:00.631905+0000","last_clean":"2024-04-25T10:19:00.631905+0000","last_became_active":"2024-04-25T10:18:58.609518+0000","last_became_peered":"2024-04-25T10:18:58.609518+0000","last_unstale":"2024-04-25T10:19:00.631905+0000","last_undegraded":"2024-04-25T10:19:00.631905+0000","last_fullsized":"2024-04-25T10:19:00.631905+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:58.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626116+0000","last_change":"2024-04-25T10:18:59.621531+0000","last_active":"2024-04-25T10:19:00.626116+0000","last_peered":"2024-04-25T10:19:00.626116+0000","last_clean":"2024-04-25T10:19:00.626116+0000","last_became_active":"2024-04-25T10:18:59.621237+0000","last_became_peered":"2024-04-25T10:18:59.621237+0000","last_unstale":"2024-04-25T10:19:00.626116+0000","last_undegraded":"2024-04-25T10:19:00.626116+0000","last_fullsized":"2024-04-25T10:19:00.626116+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.290287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217968+0000","last_change":"2024-04-25T10:18:58.626832+0000","last_active":"2024-04-25T10:19:01.217968+0000","last_peered":"2024-04-25T10:19:01.217968+0000","last_clean":"2024-04-25T10:19:01.217968+0000","last_became_active":"2024-04-25T10:18:58.626318+0000","last_became_peered":"2024-04-25T10:18:58.626318+0000","last_unstale":"2024-04-25T10:19:01.217968+0000","last_undegraded":"2024-04-25T10:19:01.217968+0000","last_fullsized":"2024-04-25T10:19:01.217968+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:09.737400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631158+0000","last_change":"2024-04-25T10:18:59.617305+0000","last_active":"2024-04-25T10:19:00.631158+0000","last_peered":"2024-04-25T10:19:00.631158+0000","last_clean":"2024-04-25T10:19:00.631158+0000","last_became_active":"2024-04-25T10:18:59.611475+0000","last_became_peered":"2024-04-25T10:18:59.611475+0000","last_unstale":"2024-04-25T10:19:00.631158+0000","last_undegraded":"2024-04-25T10:19:00.631158+0000","last_fullsized":"2024-04-25T10:19:00.631158+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:57.662681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215361+0000","last_change":"2024-04-25T10:18:58.615178+0000","last_active":"2024-04-25T10:19:01.215361+0000","last_peered":"2024-04-25T10:19:01.215361+0000","last_clean":"2024-04-25T10:19:01.215361+0000","last_became_active":"2024-04-25T10:18:58.614831+0000","last_became_peered":"2024-04-25T10:18:58.614831+0000","last_unstale":"2024-04-25T10:19:01.215361+0000","last_undegraded":"2024-04-25T10:19:01.215361+0000","last_fullsized":"2024-04-25T10:19:01.215361+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.639816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631818+0000","last_change":"2024-04-25T10:18:59.615739+0000","last_active":"2024-04-25T10:19:00.631818+0000","last_peered":"2024-04-25T10:19:00.631818+0000","last_clean":"2024-04-25T10:19:00.631818+0000","last_became_active":"2024-04-25T10:18:59.611610+0000","last_became_peered":"2024-04-25T10:18:59.611610+0000","last_unstale":"2024-04-25T10:19:00.631818+0000","last_undegraded":"2024-04-25T10:19:00.631818+0000","last_fullsized":"2024-04-25T10:19:00.631818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:46.554434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"170'8","reported_seq":22,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217687+0000","last_change":"2024-04-25T10:18:58.612843+0000","last_active":"2024-04-25T10:19:01.217687+0000","last_peered":"2024-04-25T10:19:01.217687+0000","last_clean":"2024-04-25T10:19:01.217687+0000","last_became_active":"2024-04-25T10:18:58.612070+0000","last_became_peered":"2024-04-25T10:18:58.612070+0000","last_unstale":"2024-04-25T10:19:01.217687+0000","last_undegraded":"2024-04-25T10:19:01.217687+0000","last_fullsized":"2024-04-25T10:19:01.217687+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:01.715692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942535+0000","last_change":"2024-04-25T10:18:59.615440+0000","last_active":"2024-04-25T10:19:00.942535+0000","last_peered":"2024-04-25T10:19:00.942535+0000","last_clean":"2024-04-25T10:19:00.942535+0000","last_became_active":"2024-04-25T10:18:59.613327+0000","last_became_peered":"2024-04-25T10:18:59.613327+0000","last_unstale":"2024-04-25T10:19:00.942535+0000","last_undegraded":"2024-04-25T10:19:00.942535+0000","last_fullsized":"2024-04-25T10:19:00.942535+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:29.439224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217296+0000","last_change":"2024-04-25T10:18:58.614115+0000","last_active":"2024-04-25T10:19:01.217296+0000","last_peered":"2024-04-25T10:19:01.217296+0000","last_clean":"2024-04-25T10:19:01.217296+0000","last_became_active":"2024-04-25T10:18:58.613796+0000","last_became_peered":"2024-04-25T10:18:58.613796+0000","last_unstale":"2024-04-25T10:19:01.217296+0000","last_undegraded":"2024-04-25T10:19:01.217296+0000","last_fullsized":"2024-04-25T10:19:01.217296+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:56.847168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626654+0000","last_change":"2024-04-25T10:18:59.626767+0000","last_active":"2024-04-25T10:19:00.626654+0000","last_peered":"2024-04-25T10:19:00.626654+0000","last_clean":"2024-04-25T10:19:00.626654+0000","last_became_active":"2024-04-25T10:18:59.625904+0000","last_became_peered":"2024-04-25T10:18:59.625904+0000","last_unstale":"2024-04-25T10:19:00.626654+0000","last_undegraded":"2024-04-25T10:19:00.626654+0000","last_fullsized":"2024-04-25T10:19:00.626654+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:06.887610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216395+0000","last_change":"2024-04-25T10:18:58.611238+0000","last_active":"2024-04-25T10:19:01.216395+0000","last_peered":"2024-04-25T10:19:01.216395+0000","last_clean":"2024-04-25T10:19:01.216395+0000","last_became_active":"2024-04-25T10:18:58.610890+0000","last_became_peered":"2024-04-25T10:18:58.610890+0000","last_unstale":"2024-04-25T10:19:01.216395+0000","last_undegraded":"2024-04-25T10:19:01.216395+0000","last_fullsized":"2024-04-25T10:19:01.216395+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:23.579024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217126+0000","last_change":"2024-04-25T10:18:59.638059+0000","last_active":"2024-04-25T10:19:01.217126+0000","last_peered":"2024-04-25T10:19:01.217126+0000","last_clean":"2024-04-25T10:19:01.217126+0000","last_became_active":"2024-04-25T10:18:59.637570+0000","last_became_peered":"2024-04-25T10:18:59.637570+0000","last_unstale":"2024-04-25T10:19:01.217126+0000","last_undegraded":"2024-04-25T10:19:01.217126+0000","last_fullsized":"2024-04-25T10:19:01.217126+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:10.219279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633941+0000","last_change":"2024-04-25T10:18:58.614048+0000","last_active":"2024-04-25T10:19:00.633941+0000","last_peered":"2024-04-25T10:19:00.633941+0000","last_clean":"2024-04-25T10:19:00.633941+0000","last_became_active":"2024-04-25T10:18:58.613722+0000","last_became_peered":"2024-04-25T10:18:58.613722+0000","last_unstale":"2024-04-25T10:19:00.633941+0000","last_undegraded":"2024-04-25T10:19:00.633941+0000","last_fullsized":"2024-04-25T10:19:00.633941+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:23.074674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218559+0000","last_change":"2024-04-25T10:18:59.614937+0000","last_active":"2024-04-25T10:19:01.218559+0000","last_peered":"2024-04-25T10:19:01.218559+0000","last_clean":"2024-04-25T10:19:01.218559+0000","last_became_active":"2024-04-25T10:18:59.614114+0000","last_became_peered":"2024-04-25T10:18:59.614114+0000","last_unstale":"2024-04-25T10:19:01.218559+0000","last_undegraded":"2024-04-25T10:19:01.218559+0000","last_fullsized":"2024-04-25T10:19:01.218559+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.782532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216427+0000","last_change":"2024-04-25T10:18:58.620382+0000","last_active":"2024-04-25T10:19:01.216427+0000","last_peered":"2024-04-25T10:19:01.216427+0000","last_clean":"2024-04-25T10:19:01.216427+0000","last_became_active":"2024-04-25T10:18:58.620000+0000","last_became_peered":"2024-04-25T10:18:58.620000+0000","last_unstale":"2024-04-25T10:19:01.216427+0000","last_undegraded":"2024-04-25T10:19:01.216427+0000","last_fullsized":"2024-04-25T10:19:01.216427+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.861749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216714+0000","last_change":"2024-04-25T10:18:59.614613+0000","last_active":"2024-04-25T10:19:01.216714+0000","last_peered":"2024-04-25T10:19:01.216714+0000","last_clean":"2024-04-25T10:19:01.216714+0000","last_became_active":"2024-04-25T10:18:59.613647+0000","last_became_peered":"2024-04-25T10:18:59.613647+0000","last_unstale":"2024-04-25T10:19:01.216714+0000","last_undegraded":"2024-04-25T10:19:01.216714+0000","last_fullsized":"2024-04-25T10:19:01.216714+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:25.120169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.632877+0000","last_change":"2024-04-25T10:18:58.612263+0000","last_active":"2024-04-25T10:19:00.632877+0000","last_peered":"2024-04-25T10:19:00.632877+0000","last_clean":"2024-04-25T10:19:00.632877+0000","last_became_active":"2024-04-25T10:18:58.611873+0000","last_became_peered":"2024-04-25T10:18:58.611873+0000","last_unstale":"2024-04-25T10:19:00.632877+0000","last_undegraded":"2024-04-25T10:19:00.632877+0000","last_fullsized":"2024-04-25T10:19:00.632877+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:51.898802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215476+0000","last_change":"2024-04-25T10:18:59.617732+0000","last_active":"2024-04-25T10:19:01.215476+0000","last_peered":"2024-04-25T10:19:01.215476+0000","last_clean":"2024-04-25T10:19:01.215476+0000","last_became_active":"2024-04-25T10:18:59.616509+0000","last_became_peered":"2024-04-25T10:18:59.616509+0000","last_unstale":"2024-04-25T10:19:01.215476+0000","last_undegraded":"2024-04-25T10:19:01.215476+0000","last_fullsized":"2024-04-25T10:19:01.215476+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:30.769481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215690+0000","last_change":"2024-04-25T10:18:58.625481+0000","last_active":"2024-04-25T10:19:01.215690+0000","last_peered":"2024-04-25T10:19:01.215690+0000","last_clean":"2024-04-25T10:19:01.215690+0000","last_became_active":"2024-04-25T10:18:58.624192+0000","last_became_peered":"2024-04-25T10:18:58.624192+0000","last_unstale":"2024-04-25T10:19:01.215690+0000","last_undegraded":"2024-04-25T10:19:01.215690+0000","last_fullsized":"2024-04-25T10:19:01.215690+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:13.147991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216109+0000","last_change":"2024-04-25T10:18:59.617419+0000","last_active":"2024-04-25T10:19:01.216109+0000","last_peered":"2024-04-25T10:19:01.216109+0000","last_clean":"2024-04-25T10:19:01.216109+0000","last_became_active":"2024-04-25T10:18:59.617164+0000","last_became_peered":"2024-04-25T10:18:59.617164+0000","last_unstale":"2024-04-25T10:19:01.216109+0000","last_undegraded":"2024-04-25T10:19:01.216109+0000","last_fullsized":"2024-04-25T10:19:01.216109+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.419097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942484+0000","last_change":"2024-04-25T10:18:58.627404+0000","last_active":"2024-04-25T10:19:00.942484+0000","last_peered":"2024-04-25T10:19:00.942484+0000","last_clean":"2024-04-25T10:19:00.942484+0000","last_became_active":"2024-04-25T10:18:58.627136+0000","last_became_peered":"2024-04-25T10:18:58.627136+0000","last_unstale":"2024-04-25T10:19:00.942484+0000","last_undegraded":"2024-04-25T10:19:00.942484+0000","last_fullsized":"2024-04-25T10:19:00.942484+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:00.002290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626502+0000","last_change":"2024-04-25T10:18:59.620448+0000","last_active":"2024-04-25T10:19:00.626502+0000","last_peered":"2024-04-25T10:19:00.626502+0000","last_clean":"2024-04-25T10:19:00.626502+0000","last_became_active":"2024-04-25T10:18:59.617616+0000","last_became_peered":"2024-04-25T10:18:59.617616+0000","last_unstale":"2024-04-25T10:19:00.626502+0000","last_undegraded":"2024-04-25T10:19:00.626502+0000","last_fullsized":"2024-04-25T10:19:00.626502+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:45.071800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631632+0000","last_change":"2024-04-25T10:18:58.619681+0000","last_active":"2024-04-25T10:19:00.631632+0000","last_peered":"2024-04-25T10:19:00.631632+0000","last_clean":"2024-04-25T10:19:00.631632+0000","last_became_active":"2024-04-25T10:18:58.619392+0000","last_became_peered":"2024-04-25T10:18:58.619392+0000","last_unstale":"2024-04-25T10:19:00.631632+0000","last_undegraded":"2024-04-25T10:19:00.631632+0000","last_fullsized":"2024-04-25T10:19:00.631632+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.293513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942708+0000","last_change":"2024-04-25T10:18:59.626653+0000","last_active":"2024-04-25T10:19:00.942708+0000","last_peered":"2024-04-25T10:19:00.942708+0000","last_clean":"2024-04-25T10:19:00.942708+0000","last_became_active":"2024-04-25T10:18:59.626396+0000","last_became_peered":"2024-04-25T10:18:59.626396+0000","last_unstale":"2024-04-25T10:19:00.942708+0000","last_undegraded":"2024-04-25T10:19:00.942708+0000","last_fullsized":"2024-04-25T10:19:00.942708+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:28.092270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216868+0000","last_change":"2024-04-25T10:18:58.620759+0000","last_active":"2024-04-25T10:19:01.216868+0000","last_peered":"2024-04-25T10:19:01.216868+0000","last_clean":"2024-04-25T10:19:01.216868+0000","last_became_active":"2024-04-25T10:18:58.620470+0000","last_became_peered":"2024-04-25T10:18:58.620470+0000","last_unstale":"2024-04-25T10:19:01.216868+0000","last_undegraded":"2024-04-25T10:19:01.216868+0000","last_fullsized":"2024-04-25T10:19:01.216868+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:35.763788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218442+0000","last_change":"2024-04-25T10:18:59.616376+0000","last_active":"2024-04-25T10:19:01.218442+0000","last_peered":"2024-04-25T10:19:01.218442+0000","last_clean":"2024-04-25T10:19:01.218442+0000","last_became_active":"2024-04-25T10:18:59.615624+0000","last_became_peered":"2024-04-25T10:18:59.615624+0000","last_unstale":"2024-04-25T10:19:01.218442+0000","last_undegraded":"2024-04-25T10:19:01.218442+0000","last_fullsized":"2024-04-25T10:19:01.218442+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:24.596072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"171'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.606420+0000","last_change":"2024-04-25T10:18:58.628702+0000","last_active":"2024-04-25T10:19:01.606420+0000","last_peered":"2024-04-25T10:19:01.606420+0000","last_clean":"2024-04-25T10:19:01.606420+0000","last_became_active":"2024-04-25T10:18:58.627907+0000","last_became_peered":"2024-04-25T10:18:58.627907+0000","last_unstale":"2024-04-25T10:19:01.606420+0000","last_undegraded":"2024-04-25T10:19:01.606420+0000","last_fullsized":"2024-04-25T10:19:01.606420+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:43.997233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217299+0000","last_change":"2024-04-25T10:18:59.638772+0000","last_active":"2024-04-25T10:19:01.217299+0000","last_peered":"2024-04-25T10:19:01.217299+0000","last_clean":"2024-04-25T10:19:01.217299+0000","last_became_active":"2024-04-25T10:18:59.637280+0000","last_became_peered":"2024-04-25T10:18:59.637280+0000","last_unstale":"2024-04-25T10:19:01.217299+0000","last_undegraded":"2024-04-25T10:19:01.217299+0000","last_fullsized":"2024-04-25T10:19:01.217299+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:42.780434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626873+0000","last_change":"2024-04-25T10:18:58.614526+0000","last_active":"2024-04-25T10:19:00.626873+0000","last_peered":"2024-04-25T10:19:00.626873+0000","last_clean":"2024-04-25T10:19:00.626873+0000","last_became_active":"2024-04-25T10:18:58.614216+0000","last_became_peered":"2024-04-25T10:18:58.614216+0000","last_unstale":"2024-04-25T10:19:00.626873+0000","last_undegraded":"2024-04-25T10:19:00.626873+0000","last_fullsized":"2024-04-25T10:19:00.626873+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.114090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.943169+0000","last_change":"2024-04-25T10:18:59.634193+0000","last_active":"2024-04-25T10:19:00.943169+0000","last_peered":"2024-04-25T10:19:00.943169+0000","last_clean":"2024-04-25T10:19:00.943169+0000","last_became_active":"2024-04-25T10:18:59.633634+0000","last_became_peered":"2024-04-25T10:18:59.633634+0000","last_unstale":"2024-04-25T10:19:00.943169+0000","last_undegraded":"2024-04-25T10:19:00.943169+0000","last_fullsized":"2024-04-25T10:19:00.943169+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:30.690029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626743+0000","last_change":"2024-04-25T10:18:58.625627+0000","last_active":"2024-04-25T10:19:00.626743+0000","last_peered":"2024-04-25T10:19:00.626743+0000","last_clean":"2024-04-25T10:19:00.626743+0000","last_became_active":"2024-04-25T10:18:58.624480+0000","last_became_peered":"2024-04-25T10:18:58.624480+0000","last_unstale":"2024-04-25T10:19:00.626743+0000","last_undegraded":"2024-04-25T10:19:00.626743+0000","last_fullsized":"2024-04-25T10:19:00.626743+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:47.999369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631142+0000","last_change":"2024-04-25T10:18:59.614265+0000","last_active":"2024-04-25T10:19:00.631142+0000","last_peered":"2024-04-25T10:19:00.631142+0000","last_clean":"2024-04-25T10:19:00.631142+0000","last_became_active":"2024-04-25T10:18:59.609382+0000","last_became_peered":"2024-04-25T10:18:59.609382+0000","last_unstale":"2024-04-25T10:19:00.631142+0000","last_undegraded":"2024-04-25T10:19:00.631142+0000","last_fullsized":"2024-04-25T10:19:00.631142+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:59.500943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216915+0000","last_change":"2024-04-25T10:18:58.622104+0000","last_active":"2024-04-25T10:19:01.216915+0000","last_peered":"2024-04-25T10:19:01.216915+0000","last_clean":"2024-04-25T10:19:01.216915+0000","last_became_active":"2024-04-25T10:18:58.621629+0000","last_became_peered":"2024-04-25T10:18:58.621629+0000","last_unstale":"2024-04-25T10:19:01.216915+0000","last_undegraded":"2024-04-25T10:19:01.216915+0000","last_fullsized":"2024-04-25T10:19:01.216915+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:25.121221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631053+0000","last_change":"2024-04-25T10:18:59.619693+0000","last_active":"2024-04-25T10:19:00.631053+0000","last_peered":"2024-04-25T10:19:00.631053+0000","last_clean":"2024-04-25T10:19:00.631053+0000","last_became_active":"2024-04-25T10:18:59.618985+0000","last_became_peered":"2024-04-25T10:18:59.618985+0000","last_unstale":"2024-04-25T10:19:00.631053+0000","last_undegraded":"2024-04-25T10:19:00.631053+0000","last_fullsized":"2024-04-25T10:19:00.631053+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.430640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216031+0000","last_change":"2024-04-25T10:18:58.625155+0000","last_active":"2024-04-25T10:19:01.216031+0000","last_peered":"2024-04-25T10:19:01.216031+0000","last_clean":"2024-04-25T10:19:01.216031+0000","last_became_active":"2024-04-25T10:18:58.624868+0000","last_became_peered":"2024-04-25T10:18:58.624868+0000","last_unstale":"2024-04-25T10:19:01.216031+0000","last_undegraded":"2024-04-25T10:19:01.216031+0000","last_fullsized":"2024-04-25T10:19:01.216031+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:58.829842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217665+0000","last_change":"2024-04-25T10:18:59.615266+0000","last_active":"2024-04-25T10:19:01.217665+0000","last_peered":"2024-04-25T10:19:01.217665+0000","last_clean":"2024-04-25T10:19:01.217665+0000","last_became_active":"2024-04-25T10:18:59.614605+0000","last_became_peered":"2024-04-25T10:18:59.614605+0000","last_unstale":"2024-04-25T10:19:01.217665+0000","last_undegraded":"2024-04-25T10:19:01.217665+0000","last_fullsized":"2024-04-25T10:19:01.217665+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942604+0000","last_change":"2024-04-25T10:18:58.618811+0000","last_active":"2024-04-25T10:19:00.942604+0000","last_peered":"2024-04-25T10:19:00.942604+0000","last_clean":"2024-04-25T10:19:00.942604+0000","last_became_active":"2024-04-25T10:18:58.618330+0000","last_became_peered":"2024-04-25T10:18:58.618330+0000","last_unstale":"2024-04-25T10:19:00.942604+0000","last_undegraded":"2024-04-25T10:19:00.942604+0000","last_fullsized":"2024-04-25T10:19:00.942604+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:22.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216601+0000","last_change":"2024-04-25T10:18:59.632020+0000","last_active":"2024-04-25T10:19:01.216601+0000","last_peered":"2024-04-25T10:19:01.216601+0000","last_clean":"2024-04-25T10:19:01.216601+0000","last_became_active":"2024-04-25T10:18:59.629949+0000","last_became_peered":"2024-04-25T10:18:59.629949+0000","last_unstale":"2024-04-25T10:19:01.216601+0000","last_undegraded":"2024-04-25T10:19:01.216601+0000","last_fullsized":"2024-04-25T10:19:01.216601+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631400+0000","last_change":"2024-04-25T10:18:58.623872+0000","last_active":"2024-04-25T10:19:00.631400+0000","last_peered":"2024-04-25T10:19:00.631400+0000","last_clean":"2024-04-25T10:19:00.631400+0000","last_became_active":"2024-04-25T10:18:58.623534+0000","last_became_peered":"2024-04-25T10:18:58.623534+0000","last_unstale":"2024-04-25T10:19:00.631400+0000","last_undegraded":"2024-04-25T10:19:00.631400+0000","last_fullsized":"2024-04-25T10:19:00.631400+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:16.626008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633245+0000","last_change":"2024-04-25T10:18:59.615280+0000","last_active":"2024-04-25T10:19:00.633245+0000","last_peered":"2024-04-25T10:19:00.633245+0000","last_clean":"2024-04-25T10:19:00.633245+0000","last_became_active":"2024-04-25T10:18:59.614890+0000","last_became_peered":"2024-04-25T10:18:59.614890+0000","last_unstale":"2024-04-25T10:19:00.633245+0000","last_undegraded":"2024-04-25T10:19:00.633245+0000","last_fullsized":"2024-04-25T10:19:00.633245+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:52.915850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215355+0000","last_change":"2024-04-25T10:18:58.607179+0000","last_active":"2024-04-25T10:19:01.215355+0000","last_peered":"2024-04-25T10:19:01.215355+0000","last_clean":"2024-04-25T10:19:01.215355+0000","last_became_active":"2024-04-25T10:18:58.606920+0000","last_became_peered":"2024-04-25T10:18:58.606920+0000","last_unstale":"2024-04-25T10:19:01.215355+0000","last_undegraded":"2024-04-25T10:19:01.215355+0000","last_fullsized":"2024-04-25T10:19:01.215355+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:29.456350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":457,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218023+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:19:01.218023+0000","last_peered":"2024-04-25T10:19:01.218023+0000","last_clean":"2024-04-25T10:19:01.218023+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:19:01.218023+0000","last_undegraded":"2024-04-25T10:19:01.218023+0000","last_fullsized":"2024-04-25T10:19:01.218023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575058,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60240,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311600,"statfs":{"total":96636764160,"available":96575078400,"internally_reserved":0,"allocated":2748416,"data_stored":1603106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575058,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2158592,"data_stored":1012808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575058,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55516,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316324,"statfs":{"total":96636764160,"available":96579915776,"internally_reserved":0,"allocated":2170880,"data_stored":1014372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575058,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87380,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284460,"statfs":{"total":96636764160,"available":96547287040,"internally_reserved":0,"allocated":2170880,"data_stored":1014384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575058,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87380,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284460,"statfs":{"total":96636764160,"available":96547287040,"internally_reserved":0,"allocated":2162688,"data_stored":1012898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575057,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2158592,"data_stored":1013320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2154496,"data_stored":1012738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575058,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47772,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324068,"statfs":{"total":96636764160,"available":96587845632,"internally_reserved":0,"allocated":2764800,"data_stored":1603834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:04.045 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:19:04.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.282+0000 7f73fca00640 1 Processor -- start 2024-04-25T10:19:04.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.282+0000 7f73fca00640 1 -- start start 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73fca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f73f815dd00 0x7f73f80cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73fca00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f815bf30 con 0x7f73f8144440 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73fca00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73f815c210 con 0x7f73f8071700 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73fca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73f815c860 con 0x7f73f815dd00 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73f6800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f73f8144440 0x7f73f8070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54104/0 (socket says 172.21.15.165:54104) 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73f6800640 1 -- 172.21.15.165:0/467983847 learned_addr learned my addr 172.21.15.165:0/467983847 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73f7200640 1 --2- 172.21.15.165:0/467983847 >> v2:172.21.15.172:3300/0 conn(0x7f73f815dd00 0x7f73f80cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587242441 0 0) 0x7f73f815bf30 con 0x7f73f8144440 2024-04-25T10:19:04.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.284+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f73d8003520 con 0x7f73f8144440 2024-04-25T10:19:04.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1490018886 0 0) 0x7f73d8003520 con 0x7f73f8144440 2024-04-25T10:19:04.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 >> v2:172.21.15.172:3300/0 conn(0x7f73f815dd00 msgr2=0x7f73f80cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 --2- 172.21.15.165:0/467983847 >> v2:172.21.15.172:3300/0 conn(0x7f73f815dd00 0x7f73f80cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 >> v1:172.21.15.172:6789/0 conn(0x7f73f8071700 legacy=0x7f73f80cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f815c4f0 con 0x7f73f8144440 2024-04-25T10:19:04.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f73e8001670 con 0x7f73f8144440 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 >> v1:172.21.15.165:6789/0 conn(0x7f73f8144440 legacy=0x7f73f8070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73d8003a00 0x7f73d8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73d8004800 0x7f73d8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d80051c0 0x7f73d800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f815c860 con 0x7f73d8004800 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73d800c0d0 con 0x7f73d80051c0 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73d800c2d0 con 0x7f73d8003a00 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.285+0000 7f73ede00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73d8004800 0x7f73d8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73f6800640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73d8003a00 0x7f73d8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73ede00640 1 -- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73d8003a00 msgr2=0x7f73d8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73ede00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73d8003a00 0x7f73d8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73ede00640 1 -- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d80051c0 msgr2=0x7f73d800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73f7200640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d80051c0 0x7f73d800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73ede00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d80051c0 0x7f73d800b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73ede00640 1 -- 172.21.15.165:0/467983847 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d8003520 con 0x7f73d8004800 2024-04-25T10:19:04.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73f7200640 1 -- 172.21.15.165:0/467983847 reap_dead start 2024-04-25T10:19:04.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.286+0000 7f73f6800640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73d8003a00 0x7f73d8003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:04.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.287+0000 7f73ede00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73d8004800 0x7f73d8004c80 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f73e0001750 tx=0x7f73d8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8d3a0003838543 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.287+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0001370 con 0x7f73d8004800 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.287+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e00052d0 con 0x7f73d8004800 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.287+0000 7f73f5e00640 1 -- 172.21.15.165:0/467983847 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e002ab50 con 0x7f73d8004800 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.287+0000 7f73fca00640 1 -- 172.21.15.165:0/467983847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73d8004800 msgr2=0x7f73d8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.288+0000 7f73fca00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73d8004800 0x7f73d8004c80 secure :-1 s=READY pgs=2435 cs=0 l=1 rev1=1 crypto rx=0x7f73e0001750 tx=0x7f73d8005830 comp rx=0 tx=0).stop 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.288+0000 7f73fca00640 1 -- 172.21.15.165:0/467983847 shutdown_connections 2024-04-25T10:19:04.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.288+0000 7f73fca00640 1 --2- 172.21.15.165:0/467983847 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73d8004800 0x7f73d8004c80 unknown :-1 s=CLOSED pgs=2435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.288+0000 7f73fca00640 1 -- 172.21.15.165:0/467983847 >> 172.21.15.165:0/467983847 conn(0x7f73f806abe0 msgr2=0x7f73f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.288+0000 7f73fca00640 1 -- 172.21.15.165:0/467983847 shutdown_connections 2024-04-25T10:19:04.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.288+0000 7f73fca00640 1 -- 172.21.15.165:0/467983847 wait complete. 2024-04-25T10:19:04.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.289+0000 7f73fca00640 1 Processor -- start 2024-04-25T10:19:04.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.290+0000 7f73fca00640 1 -- start start 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.290+0000 7f73fca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d8004800 0x7f73f814c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73fca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73f81e3100 0x7f73f814d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73fca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 0x7f73f813e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73fca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8071170 con 0x7f73f814e3b0 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73fca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73f8070ff0 con 0x7f73d8004800 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73fca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73f80712f0 con 0x7f73f81e3100 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 0x7f73f813e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 0x7f73f813e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40928/0 (socket says 172.21.15.165:40928) 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 -- 172.21.15.165:0/1490478722 learned_addr learned my addr 172.21.15.165:0/1490478722 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:04.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73ede00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73f81e3100 0x7f73f814d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 -- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73f81e3100 msgr2=0x7f73f814d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73f81e3100 0x7f73f814d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 -- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d8004800 msgr2=0x7f73f814c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f6800640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d8004800 0x7f73f814c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d8004800 0x7f73f814c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.291+0000 7f73f7200640 1 -- 172.21.15.165:0/1490478722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f814c720 con 0x7f73f814e3b0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.292+0000 7f73f7200640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 0x7f73f813e0c0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f73f815e190 tx=0x7f73f000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.292+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0016620 con 0x7f73f814e3b0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.292+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73f001b070 con 0x7f73f814e3b0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.292+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73f814ca00 con 0x7f73f814e3b0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.292+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73f0016a20 con 0x7f73f814e3b0 2024-04-25T10:19:04.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.292+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73f814cfc0 con 0x7f73f814e3b0 2024-04-25T10:19:04.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.293+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73f8073990 con 0x7f73f814e3b0 2024-04-25T10:19:04.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.294+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f73f0016bc0 con 0x7f73f814e3b0 2024-04-25T10:19:04.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.295+0000 7f73ef400640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73d406b4d0 0x7f73d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.295+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f73f008b2f0 con 0x7f73f814e3b0 2024-04-25T10:19:04.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.296+0000 7f73f6800640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73d406b4d0 0x7f73d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.297+0000 7f73f6800640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73d406b4d0 0x7f73d406d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f73e80017e0 tx=0x7f73e8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.300+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f73f0059fd0 con 0x7f73f814e3b0 2024-04-25T10:19:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.491+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73f8141de0 con 0x7f73d406b4d0 2024-04-25T10:19:04.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.510+0000 7f73ef400640 1 -- 172.21.15.165:0/1490478722 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252489 (secure 0 0 0) 0x7f73f8141de0 con 0x7f73d406b4d0 2024-04-25T10:19:04.516 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:04.517 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:19:04.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73d406b4d0 msgr2=0x7f73d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73d406b4d0 0x7f73d406d990 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f73e80017e0 tx=0x7f73e8009040 comp rx=0 tx=0).stop 2024-04-25T10:19:04.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 msgr2=0x7f73f813e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 0x7f73f813e0c0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7f73f815e190 tx=0x7f73f000f520 comp rx=0 tx=0).stop 2024-04-25T10:19:04.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 shutdown_connections 2024-04-25T10:19:04.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f73d406b4d0 0x7f73d406d990 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f73f814e3b0 0x7f73f813e0c0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f73f81e3100 0x7f73f814d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 --2- 172.21.15.165:0/1490478722 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f73d8004800 0x7f73f814c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 >> 172.21.15.165:0/1490478722 conn(0x7f73f806abe0 msgr2=0x7f73f81425e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 shutdown_connections 2024-04-25T10:19:04.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.515+0000 7f73fca00640 1 -- 172.21.15.165:0/1490478722 wait complete. 2024-04-25T10:19:04.537 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":680,"stamp":"2024-04-25T10:19:04.477744+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":168,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":517268,"kb_used_data":18056,"kb_used_omap":12,"kb_used_meta":499059,"kb_avail":754457452,"statfs":{"total":773094113280,"available":772564430848,"internally_reserved":0,"allocated":18489344,"data_stored":9287460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":511036978},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001888"},"pg_stats":[{"pgid":"26.3f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218068+0000","last_change":"2024-04-25T10:18:59.628251+0000","last_active":"2024-04-25T10:19:01.218068+0000","last_peered":"2024-04-25T10:19:01.218068+0000","last_clean":"2024-04-25T10:19:01.218068+0000","last_became_active":"2024-04-25T10:18:59.627522+0000","last_became_peered":"2024-04-25T10:18:59.627522+0000","last_unstale":"2024-04-25T10:19:01.218068+0000","last_undegraded":"2024-04-25T10:19:01.218068+0000","last_fullsized":"2024-04-25T10:19:01.218068+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:42.780434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218134+0000","last_change":"2024-04-25T10:18:59.625093+0000","last_active":"2024-04-25T10:19:01.218134+0000","last_peered":"2024-04-25T10:19:01.218134+0000","last_clean":"2024-04-25T10:19:01.218134+0000","last_became_active":"2024-04-25T10:18:59.624810+0000","last_became_peered":"2024-04-25T10:18:59.624810+0000","last_unstale":"2024-04-25T10:19:01.218134+0000","last_undegraded":"2024-04-25T10:19:01.218134+0000","last_fullsized":"2024-04-25T10:19:01.218134+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:21.633460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633700+0000","last_change":"2024-04-25T10:18:59.617331+0000","last_active":"2024-04-25T10:19:00.633700+0000","last_peered":"2024-04-25T10:19:00.633700+0000","last_clean":"2024-04-25T10:19:00.633700+0000","last_became_active":"2024-04-25T10:18:59.616984+0000","last_became_peered":"2024-04-25T10:18:59.616984+0000","last_unstale":"2024-04-25T10:19:00.633700+0000","last_undegraded":"2024-04-25T10:19:00.633700+0000","last_fullsized":"2024-04-25T10:19:00.633700+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:15:20.245838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215700+0000","last_change":"2024-04-25T10:18:59.612826+0000","last_active":"2024-04-25T10:19:01.215700+0000","last_peered":"2024-04-25T10:19:01.215700+0000","last_clean":"2024-04-25T10:19:01.215700+0000","last_became_active":"2024-04-25T10:18:59.612522+0000","last_became_peered":"2024-04-25T10:18:59.612522+0000","last_unstale":"2024-04-25T10:19:01.215700+0000","last_undegraded":"2024-04-25T10:19:01.215700+0000","last_fullsized":"2024-04-25T10:19:01.215700+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:17.891340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626207+0000","last_change":"2024-04-25T10:18:59.632895+0000","last_active":"2024-04-25T10:19:00.626207+0000","last_peered":"2024-04-25T10:19:00.626207+0000","last_clean":"2024-04-25T10:19:00.626207+0000","last_became_active":"2024-04-25T10:18:59.632597+0000","last_became_peered":"2024-04-25T10:18:59.632597+0000","last_unstale":"2024-04-25T10:19:00.626207+0000","last_undegraded":"2024-04-25T10:19:00.626207+0000","last_fullsized":"2024-04-25T10:19:00.626207+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:17:13.168086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626249+0000","last_change":"2024-04-25T10:18:59.612739+0000","last_active":"2024-04-25T10:19:00.626249+0000","last_peered":"2024-04-25T10:19:00.626249+0000","last_clean":"2024-04-25T10:19:00.626249+0000","last_became_active":"2024-04-25T10:18:59.612431+0000","last_became_peered":"2024-04-25T10:18:59.612431+0000","last_unstale":"2024-04-25T10:19:00.626249+0000","last_undegraded":"2024-04-25T10:19:00.626249+0000","last_fullsized":"2024-04-25T10:19:00.626249+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:56.354084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942128+0000","last_change":"2024-04-25T10:18:59.623749+0000","last_active":"2024-04-25T10:19:00.942128+0000","last_peered":"2024-04-25T10:19:00.942128+0000","last_clean":"2024-04-25T10:19:00.942128+0000","last_became_active":"2024-04-25T10:18:59.623413+0000","last_became_peered":"2024-04-25T10:18:59.623413+0000","last_unstale":"2024-04-25T10:19:00.942128+0000","last_undegraded":"2024-04-25T10:19:00.942128+0000","last_fullsized":"2024-04-25T10:19:00.942128+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:36.865218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217952+0000","last_change":"2024-04-25T10:18:59.640283+0000","last_active":"2024-04-25T10:19:01.217952+0000","last_peered":"2024-04-25T10:19:01.217952+0000","last_clean":"2024-04-25T10:19:01.217952+0000","last_became_active":"2024-04-25T10:18:59.639971+0000","last_became_peered":"2024-04-25T10:18:59.639971+0000","last_unstale":"2024-04-25T10:19:01.217952+0000","last_undegraded":"2024-04-25T10:19:01.217952+0000","last_fullsized":"2024-04-25T10:19:01.217952+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:09.103209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218328+0000","last_change":"2024-04-25T10:18:59.621167+0000","last_active":"2024-04-25T10:19:01.218328+0000","last_peered":"2024-04-25T10:19:01.218328+0000","last_clean":"2024-04-25T10:19:01.218328+0000","last_became_active":"2024-04-25T10:18:59.620854+0000","last_became_peered":"2024-04-25T10:18:59.620854+0000","last_unstale":"2024-04-25T10:19:01.218328+0000","last_undegraded":"2024-04-25T10:19:01.218328+0000","last_fullsized":"2024-04-25T10:19:01.218328+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.473398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627250+0000","last_change":"2024-04-25T10:18:59.626866+0000","last_active":"2024-04-25T10:19:00.627250+0000","last_peered":"2024-04-25T10:19:00.627250+0000","last_clean":"2024-04-25T10:19:00.627250+0000","last_became_active":"2024-04-25T10:18:59.626541+0000","last_became_peered":"2024-04-25T10:18:59.626541+0000","last_unstale":"2024-04-25T10:19:00.627250+0000","last_undegraded":"2024-04-25T10:19:00.627250+0000","last_fullsized":"2024-04-25T10:19:00.627250+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:06:14.974811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216990+0000","last_change":"2024-04-25T10:18:59.624601+0000","last_active":"2024-04-25T10:19:01.216990+0000","last_peered":"2024-04-25T10:19:01.216990+0000","last_clean":"2024-04-25T10:19:01.216990+0000","last_became_active":"2024-04-25T10:18:59.624293+0000","last_became_peered":"2024-04-25T10:18:59.624293+0000","last_unstale":"2024-04-25T10:19:01.216990+0000","last_undegraded":"2024-04-25T10:19:01.216990+0000","last_fullsized":"2024-04-25T10:19:01.216990+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:03:33.592556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215976+0000","last_change":"2024-04-25T10:18:59.623644+0000","last_active":"2024-04-25T10:19:01.215976+0000","last_peered":"2024-04-25T10:19:01.215976+0000","last_clean":"2024-04-25T10:19:01.215976+0000","last_became_active":"2024-04-25T10:18:59.623353+0000","last_became_peered":"2024-04-25T10:18:59.623353+0000","last_unstale":"2024-04-25T10:19:01.215976+0000","last_undegraded":"2024-04-25T10:19:01.215976+0000","last_fullsized":"2024-04-25T10:19:01.215976+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:33:32.500708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631639+0000","last_change":"2024-04-25T10:18:59.618958+0000","last_active":"2024-04-25T10:19:00.631639+0000","last_peered":"2024-04-25T10:19:00.631639+0000","last_clean":"2024-04-25T10:19:00.631639+0000","last_became_active":"2024-04-25T10:18:59.616212+0000","last_became_peered":"2024-04-25T10:18:59.616212+0000","last_unstale":"2024-04-25T10:19:00.631639+0000","last_undegraded":"2024-04-25T10:19:00.631639+0000","last_fullsized":"2024-04-25T10:19:00.631639+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:40:45.555838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942292+0000","last_change":"2024-04-25T10:18:59.619440+0000","last_active":"2024-04-25T10:19:00.942292+0000","last_peered":"2024-04-25T10:19:00.942292+0000","last_clean":"2024-04-25T10:19:00.942292+0000","last_became_active":"2024-04-25T10:18:59.618697+0000","last_became_peered":"2024-04-25T10:18:59.618697+0000","last_unstale":"2024-04-25T10:19:00.942292+0000","last_undegraded":"2024-04-25T10:19:00.942292+0000","last_fullsized":"2024-04-25T10:19:00.942292+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:41:36.625781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217239+0000","last_change":"2024-04-25T10:18:59.622081+0000","last_active":"2024-04-25T10:19:01.217239+0000","last_peered":"2024-04-25T10:19:01.217239+0000","last_clean":"2024-04-25T10:19:01.217239+0000","last_became_active":"2024-04-25T10:18:59.621807+0000","last_became_peered":"2024-04-25T10:18:59.621807+0000","last_unstale":"2024-04-25T10:19:01.217239+0000","last_undegraded":"2024-04-25T10:19:01.217239+0000","last_fullsized":"2024-04-25T10:19:01.217239+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:55:52.481359+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626145+0000","last_change":"2024-04-25T10:18:59.637931+0000","last_active":"2024-04-25T10:19:00.626145+0000","last_peered":"2024-04-25T10:19:00.626145+0000","last_clean":"2024-04-25T10:19:00.626145+0000","last_became_active":"2024-04-25T10:18:59.636825+0000","last_became_peered":"2024-04-25T10:18:59.636825+0000","last_unstale":"2024-04-25T10:19:00.626145+0000","last_undegraded":"2024-04-25T10:19:00.626145+0000","last_fullsized":"2024-04-25T10:19:00.626145+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:48:34.660713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627485+0000","last_change":"2024-04-25T10:18:59.626920+0000","last_active":"2024-04-25T10:19:00.627485+0000","last_peered":"2024-04-25T10:19:00.627485+0000","last_clean":"2024-04-25T10:19:00.627485+0000","last_became_active":"2024-04-25T10:18:59.626296+0000","last_became_peered":"2024-04-25T10:18:59.626296+0000","last_unstale":"2024-04-25T10:19:00.627485+0000","last_undegraded":"2024-04-25T10:19:00.627485+0000","last_fullsized":"2024-04-25T10:19:00.627485+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:33:00.761607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633113+0000","last_change":"2024-04-25T10:18:59.628149+0000","last_active":"2024-04-25T10:19:00.633113+0000","last_peered":"2024-04-25T10:19:00.633113+0000","last_clean":"2024-04-25T10:19:00.633113+0000","last_became_active":"2024-04-25T10:18:59.627561+0000","last_became_peered":"2024-04-25T10:18:59.627561+0000","last_unstale":"2024-04-25T10:19:00.633113+0000","last_undegraded":"2024-04-25T10:19:00.633113+0000","last_fullsized":"2024-04-25T10:19:00.633113+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:50:52.919451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.632726+0000","last_change":"2024-04-25T10:18:59.618401+0000","last_active":"2024-04-25T10:19:00.632726+0000","last_peered":"2024-04-25T10:19:00.632726+0000","last_clean":"2024-04-25T10:19:00.632726+0000","last_became_active":"2024-04-25T10:18:59.617451+0000","last_became_peered":"2024-04-25T10:18:59.617451+0000","last_unstale":"2024-04-25T10:19:00.632726+0000","last_undegraded":"2024-04-25T10:19:00.632726+0000","last_fullsized":"2024-04-25T10:19:00.632726+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:35:28.628728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626259+0000","last_change":"2024-04-25T10:18:59.629192+0000","last_active":"2024-04-25T10:19:00.626259+0000","last_peered":"2024-04-25T10:19:00.626259+0000","last_clean":"2024-04-25T10:19:00.626259+0000","last_became_active":"2024-04-25T10:18:59.628926+0000","last_became_peered":"2024-04-25T10:18:59.628926+0000","last_unstale":"2024-04-25T10:19:00.626259+0000","last_undegraded":"2024-04-25T10:19:00.626259+0000","last_fullsized":"2024-04-25T10:19:00.626259+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:56.524045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217965+0000","last_change":"2024-04-25T10:18:59.617258+0000","last_active":"2024-04-25T10:19:01.217965+0000","last_peered":"2024-04-25T10:19:01.217965+0000","last_clean":"2024-04-25T10:19:01.217965+0000","last_became_active":"2024-04-25T10:18:59.616981+0000","last_became_peered":"2024-04-25T10:18:59.616981+0000","last_unstale":"2024-04-25T10:19:01.217965+0000","last_undegraded":"2024-04-25T10:19:01.217965+0000","last_fullsized":"2024-04-25T10:19:01.217965+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.083207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217235+0000","last_change":"2024-04-25T10:18:59.623143+0000","last_active":"2024-04-25T10:19:01.217235+0000","last_peered":"2024-04-25T10:19:01.217235+0000","last_clean":"2024-04-25T10:19:01.217235+0000","last_became_active":"2024-04-25T10:18:59.621725+0000","last_became_peered":"2024-04-25T10:18:59.621725+0000","last_unstale":"2024-04-25T10:19:01.217235+0000","last_undegraded":"2024-04-25T10:19:01.217235+0000","last_fullsized":"2024-04-25T10:19:01.217235+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:39.097033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216313+0000","last_change":"2024-04-25T10:18:59.628513+0000","last_active":"2024-04-25T10:19:01.216313+0000","last_peered":"2024-04-25T10:19:01.216313+0000","last_clean":"2024-04-25T10:19:01.216313+0000","last_became_active":"2024-04-25T10:18:59.627330+0000","last_became_peered":"2024-04-25T10:18:59.627330+0000","last_unstale":"2024-04-25T10:19:01.216313+0000","last_undegraded":"2024-04-25T10:19:01.216313+0000","last_fullsized":"2024-04-25T10:19:01.216313+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:08:26.083207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216714+0000","last_change":"2024-04-25T10:18:59.619247+0000","last_active":"2024-04-25T10:19:01.216714+0000","last_peered":"2024-04-25T10:19:01.216714+0000","last_clean":"2024-04-25T10:19:01.216714+0000","last_became_active":"2024-04-25T10:18:59.618966+0000","last_became_peered":"2024-04-25T10:18:59.618966+0000","last_unstale":"2024-04-25T10:19:01.216714+0000","last_undegraded":"2024-04-25T10:19:01.216714+0000","last_fullsized":"2024-04-25T10:19:01.216714+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:45:38.934726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942872+0000","last_change":"2024-04-25T10:18:59.620906+0000","last_active":"2024-04-25T10:19:00.942872+0000","last_peered":"2024-04-25T10:19:00.942872+0000","last_clean":"2024-04-25T10:19:00.942872+0000","last_became_active":"2024-04-25T10:18:59.620518+0000","last_became_peered":"2024-04-25T10:18:59.620518+0000","last_unstale":"2024-04-25T10:19:00.942872+0000","last_undegraded":"2024-04-25T10:19:00.942872+0000","last_fullsized":"2024-04-25T10:19:00.942872+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:12:05.005848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216314+0000","last_change":"2024-04-25T10:18:59.637999+0000","last_active":"2024-04-25T10:19:01.216314+0000","last_peered":"2024-04-25T10:19:01.216314+0000","last_clean":"2024-04-25T10:19:01.216314+0000","last_became_active":"2024-04-25T10:18:59.637370+0000","last_became_peered":"2024-04-25T10:18:59.637370+0000","last_unstale":"2024-04-25T10:19:01.216314+0000","last_undegraded":"2024-04-25T10:19:01.216314+0000","last_fullsized":"2024-04-25T10:19:01.216314+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:11.916361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217670+0000","last_change":"2024-04-25T10:18:59.637919+0000","last_active":"2024-04-25T10:19:01.217670+0000","last_peered":"2024-04-25T10:19:01.217670+0000","last_clean":"2024-04-25T10:19:01.217670+0000","last_became_active":"2024-04-25T10:18:59.637588+0000","last_became_peered":"2024-04-25T10:18:59.637588+0000","last_unstale":"2024-04-25T10:19:01.217670+0000","last_undegraded":"2024-04-25T10:19:01.217670+0000","last_fullsized":"2024-04-25T10:19:01.217670+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:20.432975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626883+0000","last_change":"2024-04-25T10:18:59.632740+0000","last_active":"2024-04-25T10:19:00.626883+0000","last_peered":"2024-04-25T10:19:00.626883+0000","last_clean":"2024-04-25T10:19:00.626883+0000","last_became_active":"2024-04-25T10:18:59.632413+0000","last_became_peered":"2024-04-25T10:18:59.632413+0000","last_unstale":"2024-04-25T10:19:00.626883+0000","last_undegraded":"2024-04-25T10:19:00.626883+0000","last_fullsized":"2024-04-25T10:19:00.626883+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:22:22.550434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216374+0000","last_change":"2024-04-25T10:18:59.633756+0000","last_active":"2024-04-25T10:19:01.216374+0000","last_peered":"2024-04-25T10:19:01.216374+0000","last_clean":"2024-04-25T10:19:01.216374+0000","last_became_active":"2024-04-25T10:18:59.633446+0000","last_became_peered":"2024-04-25T10:18:59.633446+0000","last_unstale":"2024-04-25T10:19:01.216374+0000","last_undegraded":"2024-04-25T10:19:01.216374+0000","last_fullsized":"2024-04-25T10:19:01.216374+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:22:41.444345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218358+0000","last_change":"2024-04-25T10:18:59.624003+0000","last_active":"2024-04-25T10:19:01.218358+0000","last_peered":"2024-04-25T10:19:01.218358+0000","last_clean":"2024-04-25T10:19:01.218358+0000","last_became_active":"2024-04-25T10:18:59.623674+0000","last_became_peered":"2024-04-25T10:18:59.623674+0000","last_unstale":"2024-04-25T10:19:01.218358+0000","last_undegraded":"2024-04-25T10:19:01.218358+0000","last_fullsized":"2024-04-25T10:19:01.218358+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:59.846891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942223+0000","last_change":"2024-04-25T10:18:59.618835+0000","last_active":"2024-04-25T10:19:00.942223+0000","last_peered":"2024-04-25T10:19:00.942223+0000","last_clean":"2024-04-25T10:19:00.942223+0000","last_became_active":"2024-04-25T10:18:59.618525+0000","last_became_peered":"2024-04-25T10:18:59.618525+0000","last_unstale":"2024-04-25T10:19:00.942223+0000","last_undegraded":"2024-04-25T10:19:00.942223+0000","last_fullsized":"2024-04-25T10:19:00.942223+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:16:26.413319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626053+0000","last_change":"2024-04-25T10:18:59.629650+0000","last_active":"2024-04-25T10:19:00.626053+0000","last_peered":"2024-04-25T10:19:00.626053+0000","last_clean":"2024-04-25T10:19:00.626053+0000","last_became_active":"2024-04-25T10:18:59.625699+0000","last_became_peered":"2024-04-25T10:18:59.625699+0000","last_unstale":"2024-04-25T10:19:00.626053+0000","last_undegraded":"2024-04-25T10:19:00.626053+0000","last_fullsized":"2024-04-25T10:19:00.626053+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:30:28.957316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217380+0000","last_change":"2024-04-25T10:18:59.638243+0000","last_active":"2024-04-25T10:19:01.217380+0000","last_peered":"2024-04-25T10:19:01.217380+0000","last_clean":"2024-04-25T10:19:01.217380+0000","last_became_active":"2024-04-25T10:18:59.636046+0000","last_became_peered":"2024-04-25T10:18:59.636046+0000","last_unstale":"2024-04-25T10:19:01.217380+0000","last_undegraded":"2024-04-25T10:19:01.217380+0000","last_fullsized":"2024-04-25T10:19:01.217380+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:05:21.633460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"170'4","reported_seq":18,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216781+0000","last_change":"2024-04-25T10:18:58.626021+0000","last_active":"2024-04-25T10:19:01.216781+0000","last_peered":"2024-04-25T10:19:01.216781+0000","last_clean":"2024-04-25T10:19:01.216781+0000","last_became_active":"2024-04-25T10:18:58.623471+0000","last_became_peered":"2024-04-25T10:18:58.623471+0000","last_unstale":"2024-04-25T10:19:01.216781+0000","last_undegraded":"2024-04-25T10:19:01.216781+0000","last_fullsized":"2024-04-25T10:19:01.216781+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:41.765484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942765+0000","last_change":"2024-04-25T10:18:59.627188+0000","last_active":"2024-04-25T10:19:00.942765+0000","last_peered":"2024-04-25T10:19:00.942765+0000","last_clean":"2024-04-25T10:19:00.942765+0000","last_became_active":"2024-04-25T10:18:59.625966+0000","last_became_peered":"2024-04-25T10:18:59.625966+0000","last_unstale":"2024-04-25T10:19:00.942765+0000","last_undegraded":"2024-04-25T10:19:00.942765+0000","last_fullsized":"2024-04-25T10:19:00.942765+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:25:52.095423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627493+0000","last_change":"2024-04-25T10:18:58.620716+0000","last_active":"2024-04-25T10:19:00.627493+0000","last_peered":"2024-04-25T10:19:00.627493+0000","last_clean":"2024-04-25T10:19:00.627493+0000","last_became_active":"2024-04-25T10:18:58.620384+0000","last_became_peered":"2024-04-25T10:18:58.620384+0000","last_unstale":"2024-04-25T10:19:00.627493+0000","last_undegraded":"2024-04-25T10:19:00.627493+0000","last_fullsized":"2024-04-25T10:19:00.627493+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:39:46.877771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216714+0000","last_change":"2024-04-25T10:18:59.638215+0000","last_active":"2024-04-25T10:19:01.216714+0000","last_peered":"2024-04-25T10:19:01.216714+0000","last_clean":"2024-04-25T10:19:01.216714+0000","last_became_active":"2024-04-25T10:18:59.637734+0000","last_became_peered":"2024-04-25T10:18:59.637734+0000","last_unstale":"2024-04-25T10:19:01.216714+0000","last_undegraded":"2024-04-25T10:19:01.216714+0000","last_fullsized":"2024-04-25T10:19:01.216714+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:40:27.444607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217469+0000","last_change":"2024-04-25T10:18:58.622801+0000","last_active":"2024-04-25T10:19:01.217469+0000","last_peered":"2024-04-25T10:19:01.217469+0000","last_clean":"2024-04-25T10:19:01.217469+0000","last_became_active":"2024-04-25T10:18:58.621805+0000","last_became_peered":"2024-04-25T10:18:58.621805+0000","last_unstale":"2024-04-25T10:19:01.217469+0000","last_undegraded":"2024-04-25T10:19:01.217469+0000","last_fullsized":"2024-04-25T10:19:01.217469+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:55.106290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633762+0000","last_change":"2024-04-25T10:18:59.633505+0000","last_active":"2024-04-25T10:19:00.633762+0000","last_peered":"2024-04-25T10:19:00.633762+0000","last_clean":"2024-04-25T10:19:00.633762+0000","last_became_active":"2024-04-25T10:18:59.633215+0000","last_became_peered":"2024-04-25T10:18:59.633215+0000","last_unstale":"2024-04-25T10:19:00.633762+0000","last_undegraded":"2024-04-25T10:19:00.633762+0000","last_fullsized":"2024-04-25T10:19:00.633762+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:45:27.344598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.632881+0000","last_change":"2024-04-25T10:18:58.616032+0000","last_active":"2024-04-25T10:19:00.632881+0000","last_peered":"2024-04-25T10:19:00.632881+0000","last_clean":"2024-04-25T10:19:00.632881+0000","last_became_active":"2024-04-25T10:18:58.615698+0000","last_became_peered":"2024-04-25T10:18:58.615698+0000","last_unstale":"2024-04-25T10:19:00.632881+0000","last_undegraded":"2024-04-25T10:19:00.632881+0000","last_fullsized":"2024-04-25T10:19:00.632881+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:57:40.201484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.630587+0000","last_change":"2024-04-25T10:18:59.615359+0000","last_active":"2024-04-25T10:19:00.630587+0000","last_peered":"2024-04-25T10:19:00.630587+0000","last_clean":"2024-04-25T10:19:00.630587+0000","last_became_active":"2024-04-25T10:18:59.615061+0000","last_became_peered":"2024-04-25T10:18:59.615061+0000","last_unstale":"2024-04-25T10:19:00.630587+0000","last_undegraded":"2024-04-25T10:19:00.630587+0000","last_fullsized":"2024-04-25T10:19:00.630587+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:05:12.546528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626695+0000","last_change":"2024-04-25T10:18:58.619517+0000","last_active":"2024-04-25T10:19:00.626695+0000","last_peered":"2024-04-25T10:19:00.626695+0000","last_clean":"2024-04-25T10:19:00.626695+0000","last_became_active":"2024-04-25T10:18:58.619252+0000","last_became_peered":"2024-04-25T10:18:58.619252+0000","last_unstale":"2024-04-25T10:19:00.626695+0000","last_undegraded":"2024-04-25T10:19:00.626695+0000","last_fullsized":"2024-04-25T10:19:00.626695+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:59:13.470612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217251+0000","last_change":"2024-04-25T10:18:59.613882+0000","last_active":"2024-04-25T10:19:01.217251+0000","last_peered":"2024-04-25T10:19:01.217251+0000","last_clean":"2024-04-25T10:19:01.217251+0000","last_became_active":"2024-04-25T10:18:59.613080+0000","last_became_peered":"2024-04-25T10:18:59.613080+0000","last_unstale":"2024-04-25T10:19:01.217251+0000","last_undegraded":"2024-04-25T10:19:01.217251+0000","last_fullsized":"2024-04-25T10:19:01.217251+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:26:54.795683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216223+0000","last_change":"2024-04-25T10:18:58.619967+0000","last_active":"2024-04-25T10:19:01.216223+0000","last_peered":"2024-04-25T10:19:01.216223+0000","last_clean":"2024-04-25T10:19:01.216223+0000","last_became_active":"2024-04-25T10:18:58.619696+0000","last_became_peered":"2024-04-25T10:18:58.619696+0000","last_unstale":"2024-04-25T10:19:01.216223+0000","last_undegraded":"2024-04-25T10:19:01.216223+0000","last_fullsized":"2024-04-25T10:19:01.216223+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:05:04.934349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218269+0000","last_change":"2024-04-25T10:18:59.624128+0000","last_active":"2024-04-25T10:19:01.218269+0000","last_peered":"2024-04-25T10:19:01.218269+0000","last_clean":"2024-04-25T10:19:01.218269+0000","last_became_active":"2024-04-25T10:18:59.623842+0000","last_became_peered":"2024-04-25T10:18:59.623842+0000","last_unstale":"2024-04-25T10:19:01.218269+0000","last_undegraded":"2024-04-25T10:19:01.218269+0000","last_fullsized":"2024-04-25T10:19:01.218269+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:40:20.432975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.630806+0000","last_change":"2024-04-25T10:18:58.619968+0000","last_active":"2024-04-25T10:19:00.630806+0000","last_peered":"2024-04-25T10:19:00.630806+0000","last_clean":"2024-04-25T10:19:00.630806+0000","last_became_active":"2024-04-25T10:18:58.619279+0000","last_became_peered":"2024-04-25T10:18:58.619279+0000","last_unstale":"2024-04-25T10:19:00.630806+0000","last_undegraded":"2024-04-25T10:19:00.630806+0000","last_fullsized":"2024-04-25T10:19:00.630806+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:17:40.933234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217696+0000","last_change":"2024-04-25T10:18:59.639191+0000","last_active":"2024-04-25T10:19:01.217696+0000","last_peered":"2024-04-25T10:19:01.217696+0000","last_clean":"2024-04-25T10:19:01.217696+0000","last_became_active":"2024-04-25T10:18:59.638849+0000","last_became_peered":"2024-04-25T10:18:59.638849+0000","last_unstale":"2024-04-25T10:19:01.217696+0000","last_undegraded":"2024-04-25T10:19:01.217696+0000","last_fullsized":"2024-04-25T10:19:01.217696+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:30.473398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627420+0000","last_change":"2024-04-25T10:18:58.616113+0000","last_active":"2024-04-25T10:19:00.627420+0000","last_peered":"2024-04-25T10:19:00.627420+0000","last_clean":"2024-04-25T10:19:00.627420+0000","last_became_active":"2024-04-25T10:18:58.614173+0000","last_became_peered":"2024-04-25T10:18:58.614173+0000","last_unstale":"2024-04-25T10:19:00.627420+0000","last_undegraded":"2024-04-25T10:19:00.627420+0000","last_fullsized":"2024-04-25T10:19:00.627420+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:56.460787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.627060+0000","last_change":"2024-04-25T10:18:59.618999+0000","last_active":"2024-04-25T10:19:00.627060+0000","last_peered":"2024-04-25T10:19:00.627060+0000","last_clean":"2024-04-25T10:19:00.627060+0000","last_became_active":"2024-04-25T10:18:59.615816+0000","last_became_peered":"2024-04-25T10:18:59.615816+0000","last_unstale":"2024-04-25T10:19:00.627060+0000","last_undegraded":"2024-04-25T10:19:00.627060+0000","last_fullsized":"2024-04-25T10:19:00.627060+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:29.284669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218434+0000","last_change":"2024-04-25T10:18:58.623351+0000","last_active":"2024-04-25T10:19:01.218434+0000","last_peered":"2024-04-25T10:19:01.218434+0000","last_clean":"2024-04-25T10:19:01.218434+0000","last_became_active":"2024-04-25T10:18:58.623081+0000","last_became_peered":"2024-04-25T10:18:58.623081+0000","last_unstale":"2024-04-25T10:19:01.218434+0000","last_undegraded":"2024-04-25T10:19:01.218434+0000","last_fullsized":"2024-04-25T10:19:01.218434+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:33:55.106290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633592+0000","last_change":"2024-04-25T10:18:59.627885+0000","last_active":"2024-04-25T10:19:00.633592+0000","last_peered":"2024-04-25T10:19:00.633592+0000","last_clean":"2024-04-25T10:19:00.633592+0000","last_became_active":"2024-04-25T10:18:59.626069+0000","last_became_peered":"2024-04-25T10:18:59.626069+0000","last_unstale":"2024-04-25T10:19:00.633592+0000","last_undegraded":"2024-04-25T10:19:00.633592+0000","last_fullsized":"2024-04-25T10:19:00.633592+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:15:46.262628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"170'3","reported_seq":17,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216238+0000","last_change":"2024-04-25T10:18:58.625180+0000","last_active":"2024-04-25T10:19:01.216238+0000","last_peered":"2024-04-25T10:19:01.216238+0000","last_clean":"2024-04-25T10:19:01.216238+0000","last_became_active":"2024-04-25T10:18:58.624162+0000","last_became_peered":"2024-04-25T10:18:58.624162+0000","last_unstale":"2024-04-25T10:19:01.216238+0000","last_undegraded":"2024-04-25T10:19:01.216238+0000","last_fullsized":"2024-04-25T10:19:01.216238+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:08.086161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218344+0000","last_change":"2024-04-25T10:18:59.633886+0000","last_active":"2024-04-25T10:19:01.218344+0000","last_peered":"2024-04-25T10:19:01.218344+0000","last_clean":"2024-04-25T10:19:01.218344+0000","last_became_active":"2024-04-25T10:18:59.633490+0000","last_became_peered":"2024-04-25T10:18:59.633490+0000","last_unstale":"2024-04-25T10:19:01.218344+0000","last_undegraded":"2024-04-25T10:19:01.218344+0000","last_fullsized":"2024-04-25T10:19:01.218344+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:54:09.103209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"170'6","reported_seq":20,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633183+0000","last_change":"2024-04-25T10:18:58.610641+0000","last_active":"2024-04-25T10:19:00.633183+0000","last_peered":"2024-04-25T10:19:00.633183+0000","last_clean":"2024-04-25T10:19:00.633183+0000","last_became_active":"2024-04-25T10:18:58.610329+0000","last_became_peered":"2024-04-25T10:18:58.610329+0000","last_unstale":"2024-04-25T10:19:00.633183+0000","last_undegraded":"2024-04-25T10:19:00.633183+0000","last_fullsized":"2024-04-25T10:19:00.633183+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:36:54.374547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942044+0000","last_change":"2024-04-25T10:18:59.619818+0000","last_active":"2024-04-25T10:19:00.942044+0000","last_peered":"2024-04-25T10:19:00.942044+0000","last_clean":"2024-04-25T10:19:00.942044+0000","last_became_active":"2024-04-25T10:18:59.619298+0000","last_became_peered":"2024-04-25T10:18:59.619298+0000","last_unstale":"2024-04-25T10:19:00.942044+0000","last_undegraded":"2024-04-25T10:19:00.942044+0000","last_fullsized":"2024-04-25T10:19:00.942044+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:56.093918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631905+0000","last_change":"2024-04-25T10:18:58.611475+0000","last_active":"2024-04-25T10:19:00.631905+0000","last_peered":"2024-04-25T10:19:00.631905+0000","last_clean":"2024-04-25T10:19:00.631905+0000","last_became_active":"2024-04-25T10:18:58.609518+0000","last_became_peered":"2024-04-25T10:18:58.609518+0000","last_unstale":"2024-04-25T10:19:00.631905+0000","last_undegraded":"2024-04-25T10:19:00.631905+0000","last_fullsized":"2024-04-25T10:19:00.631905+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:58.483894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626116+0000","last_change":"2024-04-25T10:18:59.621531+0000","last_active":"2024-04-25T10:19:00.626116+0000","last_peered":"2024-04-25T10:19:00.626116+0000","last_clean":"2024-04-25T10:19:00.626116+0000","last_became_active":"2024-04-25T10:18:59.621237+0000","last_became_peered":"2024-04-25T10:18:59.621237+0000","last_unstale":"2024-04-25T10:19:00.626116+0000","last_undegraded":"2024-04-25T10:19:00.626116+0000","last_fullsized":"2024-04-25T10:19:00.626116+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:54:33.290287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217968+0000","last_change":"2024-04-25T10:18:58.626832+0000","last_active":"2024-04-25T10:19:01.217968+0000","last_peered":"2024-04-25T10:19:01.217968+0000","last_clean":"2024-04-25T10:19:01.217968+0000","last_became_active":"2024-04-25T10:18:58.626318+0000","last_became_peered":"2024-04-25T10:18:58.626318+0000","last_unstale":"2024-04-25T10:19:01.217968+0000","last_undegraded":"2024-04-25T10:19:01.217968+0000","last_fullsized":"2024-04-25T10:19:01.217968+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:17:09.737400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631158+0000","last_change":"2024-04-25T10:18:59.617305+0000","last_active":"2024-04-25T10:19:00.631158+0000","last_peered":"2024-04-25T10:19:00.631158+0000","last_clean":"2024-04-25T10:19:00.631158+0000","last_became_active":"2024-04-25T10:18:59.611475+0000","last_became_peered":"2024-04-25T10:18:59.611475+0000","last_unstale":"2024-04-25T10:19:00.631158+0000","last_undegraded":"2024-04-25T10:19:00.631158+0000","last_fullsized":"2024-04-25T10:19:00.631158+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:11:57.662681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215361+0000","last_change":"2024-04-25T10:18:58.615178+0000","last_active":"2024-04-25T10:19:01.215361+0000","last_peered":"2024-04-25T10:19:01.215361+0000","last_clean":"2024-04-25T10:19:01.215361+0000","last_became_active":"2024-04-25T10:18:58.614831+0000","last_became_peered":"2024-04-25T10:18:58.614831+0000","last_unstale":"2024-04-25T10:19:01.215361+0000","last_undegraded":"2024-04-25T10:19:01.215361+0000","last_fullsized":"2024-04-25T10:19:01.215361+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:09:05.639816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631818+0000","last_change":"2024-04-25T10:18:59.615739+0000","last_active":"2024-04-25T10:19:00.631818+0000","last_peered":"2024-04-25T10:19:00.631818+0000","last_clean":"2024-04-25T10:19:00.631818+0000","last_became_active":"2024-04-25T10:18:59.611610+0000","last_became_peered":"2024-04-25T10:18:59.611610+0000","last_unstale":"2024-04-25T10:19:00.631818+0000","last_undegraded":"2024-04-25T10:19:00.631818+0000","last_fullsized":"2024-04-25T10:19:00.631818+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:57:46.554434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"170'8","reported_seq":22,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217687+0000","last_change":"2024-04-25T10:18:58.612843+0000","last_active":"2024-04-25T10:19:01.217687+0000","last_peered":"2024-04-25T10:19:01.217687+0000","last_clean":"2024-04-25T10:19:01.217687+0000","last_became_active":"2024-04-25T10:18:58.612070+0000","last_became_peered":"2024-04-25T10:18:58.612070+0000","last_unstale":"2024-04-25T10:19:01.217687+0000","last_undegraded":"2024-04-25T10:19:01.217687+0000","last_fullsized":"2024-04-25T10:19:01.217687+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:10:01.715692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942535+0000","last_change":"2024-04-25T10:18:59.615440+0000","last_active":"2024-04-25T10:19:00.942535+0000","last_peered":"2024-04-25T10:19:00.942535+0000","last_clean":"2024-04-25T10:19:00.942535+0000","last_became_active":"2024-04-25T10:18:59.613327+0000","last_became_peered":"2024-04-25T10:18:59.613327+0000","last_unstale":"2024-04-25T10:19:00.942535+0000","last_undegraded":"2024-04-25T10:19:00.942535+0000","last_fullsized":"2024-04-25T10:19:00.942535+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:54:29.439224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217296+0000","last_change":"2024-04-25T10:18:58.614115+0000","last_active":"2024-04-25T10:19:01.217296+0000","last_peered":"2024-04-25T10:19:01.217296+0000","last_clean":"2024-04-25T10:19:01.217296+0000","last_became_active":"2024-04-25T10:18:58.613796+0000","last_became_peered":"2024-04-25T10:18:58.613796+0000","last_unstale":"2024-04-25T10:19:01.217296+0000","last_undegraded":"2024-04-25T10:19:01.217296+0000","last_fullsized":"2024-04-25T10:19:01.217296+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:56.847168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626654+0000","last_change":"2024-04-25T10:18:59.626767+0000","last_active":"2024-04-25T10:19:00.626654+0000","last_peered":"2024-04-25T10:19:00.626654+0000","last_clean":"2024-04-25T10:19:00.626654+0000","last_became_active":"2024-04-25T10:18:59.625904+0000","last_became_peered":"2024-04-25T10:18:59.625904+0000","last_unstale":"2024-04-25T10:19:00.626654+0000","last_undegraded":"2024-04-25T10:19:00.626654+0000","last_fullsized":"2024-04-25T10:19:00.626654+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:16:06.887610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216395+0000","last_change":"2024-04-25T10:18:58.611238+0000","last_active":"2024-04-25T10:19:01.216395+0000","last_peered":"2024-04-25T10:19:01.216395+0000","last_clean":"2024-04-25T10:19:01.216395+0000","last_became_active":"2024-04-25T10:18:58.610890+0000","last_became_peered":"2024-04-25T10:18:58.610890+0000","last_unstale":"2024-04-25T10:19:01.216395+0000","last_undegraded":"2024-04-25T10:19:01.216395+0000","last_fullsized":"2024-04-25T10:19:01.216395+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:23.579024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217126+0000","last_change":"2024-04-25T10:18:59.638059+0000","last_active":"2024-04-25T10:19:01.217126+0000","last_peered":"2024-04-25T10:19:01.217126+0000","last_clean":"2024-04-25T10:19:01.217126+0000","last_became_active":"2024-04-25T10:18:59.637570+0000","last_became_peered":"2024-04-25T10:18:59.637570+0000","last_unstale":"2024-04-25T10:19:01.217126+0000","last_undegraded":"2024-04-25T10:19:01.217126+0000","last_fullsized":"2024-04-25T10:19:01.217126+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:49:10.219279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"170'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633941+0000","last_change":"2024-04-25T10:18:58.614048+0000","last_active":"2024-04-25T10:19:00.633941+0000","last_peered":"2024-04-25T10:19:00.633941+0000","last_clean":"2024-04-25T10:19:00.633941+0000","last_became_active":"2024-04-25T10:18:58.613722+0000","last_became_peered":"2024-04-25T10:18:58.613722+0000","last_unstale":"2024-04-25T10:19:00.633941+0000","last_undegraded":"2024-04-25T10:19:00.633941+0000","last_fullsized":"2024-04-25T10:19:00.633941+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:23.074674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218559+0000","last_change":"2024-04-25T10:18:59.614937+0000","last_active":"2024-04-25T10:19:01.218559+0000","last_peered":"2024-04-25T10:19:01.218559+0000","last_clean":"2024-04-25T10:19:01.218559+0000","last_became_active":"2024-04-25T10:18:59.614114+0000","last_became_peered":"2024-04-25T10:18:59.614114+0000","last_unstale":"2024-04-25T10:19:01.218559+0000","last_undegraded":"2024-04-25T10:19:01.218559+0000","last_fullsized":"2024-04-25T10:19:01.218559+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:42.782532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216427+0000","last_change":"2024-04-25T10:18:58.620382+0000","last_active":"2024-04-25T10:19:01.216427+0000","last_peered":"2024-04-25T10:19:01.216427+0000","last_clean":"2024-04-25T10:19:01.216427+0000","last_became_active":"2024-04-25T10:18:58.620000+0000","last_became_peered":"2024-04-25T10:18:58.620000+0000","last_unstale":"2024-04-25T10:19:01.216427+0000","last_undegraded":"2024-04-25T10:19:01.216427+0000","last_fullsized":"2024-04-25T10:19:01.216427+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:09.861749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216714+0000","last_change":"2024-04-25T10:18:59.614613+0000","last_active":"2024-04-25T10:19:01.216714+0000","last_peered":"2024-04-25T10:19:01.216714+0000","last_clean":"2024-04-25T10:19:01.216714+0000","last_became_active":"2024-04-25T10:18:59.613647+0000","last_became_peered":"2024-04-25T10:18:59.613647+0000","last_unstale":"2024-04-25T10:19:01.216714+0000","last_undegraded":"2024-04-25T10:19:01.216714+0000","last_fullsized":"2024-04-25T10:19:01.216714+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:25.120169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.632877+0000","last_change":"2024-04-25T10:18:58.612263+0000","last_active":"2024-04-25T10:19:00.632877+0000","last_peered":"2024-04-25T10:19:00.632877+0000","last_clean":"2024-04-25T10:19:00.632877+0000","last_became_active":"2024-04-25T10:18:58.611873+0000","last_became_peered":"2024-04-25T10:18:58.611873+0000","last_unstale":"2024-04-25T10:19:00.632877+0000","last_undegraded":"2024-04-25T10:19:00.632877+0000","last_fullsized":"2024-04-25T10:19:00.632877+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:51.898802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215476+0000","last_change":"2024-04-25T10:18:59.617732+0000","last_active":"2024-04-25T10:19:01.215476+0000","last_peered":"2024-04-25T10:19:01.215476+0000","last_clean":"2024-04-25T10:19:01.215476+0000","last_became_active":"2024-04-25T10:18:59.616509+0000","last_became_peered":"2024-04-25T10:18:59.616509+0000","last_unstale":"2024-04-25T10:19:01.215476+0000","last_undegraded":"2024-04-25T10:19:01.215476+0000","last_fullsized":"2024-04-25T10:19:01.215476+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:43:30.769481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215690+0000","last_change":"2024-04-25T10:18:58.625481+0000","last_active":"2024-04-25T10:19:01.215690+0000","last_peered":"2024-04-25T10:19:01.215690+0000","last_clean":"2024-04-25T10:19:01.215690+0000","last_became_active":"2024-04-25T10:18:58.624192+0000","last_became_peered":"2024-04-25T10:18:58.624192+0000","last_unstale":"2024-04-25T10:19:01.215690+0000","last_undegraded":"2024-04-25T10:19:01.215690+0000","last_fullsized":"2024-04-25T10:19:01.215690+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:13.147991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216109+0000","last_change":"2024-04-25T10:18:59.617419+0000","last_active":"2024-04-25T10:19:01.216109+0000","last_peered":"2024-04-25T10:19:01.216109+0000","last_clean":"2024-04-25T10:19:01.216109+0000","last_became_active":"2024-04-25T10:18:59.617164+0000","last_became_peered":"2024-04-25T10:18:59.617164+0000","last_unstale":"2024-04-25T10:19:01.216109+0000","last_undegraded":"2024-04-25T10:19:01.216109+0000","last_fullsized":"2024-04-25T10:19:01.216109+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:42.419097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942484+0000","last_change":"2024-04-25T10:18:58.627404+0000","last_active":"2024-04-25T10:19:00.942484+0000","last_peered":"2024-04-25T10:19:00.942484+0000","last_clean":"2024-04-25T10:19:00.942484+0000","last_became_active":"2024-04-25T10:18:58.627136+0000","last_became_peered":"2024-04-25T10:18:58.627136+0000","last_unstale":"2024-04-25T10:19:00.942484+0000","last_undegraded":"2024-04-25T10:19:00.942484+0000","last_fullsized":"2024-04-25T10:19:00.942484+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:28:00.002290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626502+0000","last_change":"2024-04-25T10:18:59.620448+0000","last_active":"2024-04-25T10:19:00.626502+0000","last_peered":"2024-04-25T10:19:00.626502+0000","last_clean":"2024-04-25T10:19:00.626502+0000","last_became_active":"2024-04-25T10:18:59.617616+0000","last_became_peered":"2024-04-25T10:18:59.617616+0000","last_unstale":"2024-04-25T10:19:00.626502+0000","last_undegraded":"2024-04-25T10:19:00.626502+0000","last_fullsized":"2024-04-25T10:19:00.626502+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:00:45.071800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631632+0000","last_change":"2024-04-25T10:18:58.619681+0000","last_active":"2024-04-25T10:19:00.631632+0000","last_peered":"2024-04-25T10:19:00.631632+0000","last_clean":"2024-04-25T10:19:00.631632+0000","last_became_active":"2024-04-25T10:18:58.619392+0000","last_became_peered":"2024-04-25T10:18:58.619392+0000","last_unstale":"2024-04-25T10:19:00.631632+0000","last_undegraded":"2024-04-25T10:19:00.631632+0000","last_fullsized":"2024-04-25T10:19:00.631632+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:59:59.293513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942708+0000","last_change":"2024-04-25T10:18:59.626653+0000","last_active":"2024-04-25T10:19:00.942708+0000","last_peered":"2024-04-25T10:19:00.942708+0000","last_clean":"2024-04-25T10:19:00.942708+0000","last_became_active":"2024-04-25T10:18:59.626396+0000","last_became_peered":"2024-04-25T10:18:59.626396+0000","last_unstale":"2024-04-25T10:19:00.942708+0000","last_undegraded":"2024-04-25T10:19:00.942708+0000","last_fullsized":"2024-04-25T10:19:00.942708+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:04:28.092270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"170'2","reported_seq":16,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216868+0000","last_change":"2024-04-25T10:18:58.620759+0000","last_active":"2024-04-25T10:19:01.216868+0000","last_peered":"2024-04-25T10:19:01.216868+0000","last_clean":"2024-04-25T10:19:01.216868+0000","last_became_active":"2024-04-25T10:18:58.620470+0000","last_became_peered":"2024-04-25T10:18:58.620470+0000","last_unstale":"2024-04-25T10:19:01.216868+0000","last_undegraded":"2024-04-25T10:19:01.216868+0000","last_fullsized":"2024-04-25T10:19:01.216868+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:35.763788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218442+0000","last_change":"2024-04-25T10:18:59.616376+0000","last_active":"2024-04-25T10:19:01.218442+0000","last_peered":"2024-04-25T10:19:01.218442+0000","last_clean":"2024-04-25T10:19:01.218442+0000","last_became_active":"2024-04-25T10:18:59.615624+0000","last_became_peered":"2024-04-25T10:18:59.615624+0000","last_unstale":"2024-04-25T10:19:01.218442+0000","last_undegraded":"2024-04-25T10:19:01.218442+0000","last_fullsized":"2024-04-25T10:19:01.218442+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:31:24.596072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"171'1","reported_seq":15,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.606420+0000","last_change":"2024-04-25T10:18:58.628702+0000","last_active":"2024-04-25T10:19:01.606420+0000","last_peered":"2024-04-25T10:19:01.606420+0000","last_clean":"2024-04-25T10:19:01.606420+0000","last_became_active":"2024-04-25T10:18:58.627907+0000","last_became_peered":"2024-04-25T10:18:58.627907+0000","last_unstale":"2024-04-25T10:19:01.606420+0000","last_undegraded":"2024-04-25T10:19:01.606420+0000","last_fullsized":"2024-04-25T10:19:01.606420+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:43.997233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217299+0000","last_change":"2024-04-25T10:18:59.638772+0000","last_active":"2024-04-25T10:19:01.217299+0000","last_peered":"2024-04-25T10:19:01.217299+0000","last_clean":"2024-04-25T10:19:01.217299+0000","last_became_active":"2024-04-25T10:18:59.637280+0000","last_became_peered":"2024-04-25T10:18:59.637280+0000","last_unstale":"2024-04-25T10:19:01.217299+0000","last_undegraded":"2024-04-25T10:19:01.217299+0000","last_fullsized":"2024-04-25T10:19:01.217299+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:05:42.780434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626873+0000","last_change":"2024-04-25T10:18:58.614526+0000","last_active":"2024-04-25T10:19:00.626873+0000","last_peered":"2024-04-25T10:19:00.626873+0000","last_clean":"2024-04-25T10:19:00.626873+0000","last_became_active":"2024-04-25T10:18:58.614216+0000","last_became_peered":"2024-04-25T10:18:58.614216+0000","last_unstale":"2024-04-25T10:19:00.626873+0000","last_undegraded":"2024-04-25T10:19:00.626873+0000","last_fullsized":"2024-04-25T10:19:00.626873+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:00:43.114090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.943169+0000","last_change":"2024-04-25T10:18:59.634193+0000","last_active":"2024-04-25T10:19:00.943169+0000","last_peered":"2024-04-25T10:19:00.943169+0000","last_clean":"2024-04-25T10:19:00.943169+0000","last_became_active":"2024-04-25T10:18:59.633634+0000","last_became_peered":"2024-04-25T10:18:59.633634+0000","last_unstale":"2024-04-25T10:19:00.943169+0000","last_undegraded":"2024-04-25T10:19:00.943169+0000","last_fullsized":"2024-04-25T10:19:00.943169+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:51:30.690029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.626743+0000","last_change":"2024-04-25T10:18:58.625627+0000","last_active":"2024-04-25T10:19:00.626743+0000","last_peered":"2024-04-25T10:19:00.626743+0000","last_clean":"2024-04-25T10:19:00.626743+0000","last_became_active":"2024-04-25T10:18:58.624480+0000","last_became_peered":"2024-04-25T10:18:58.624480+0000","last_unstale":"2024-04-25T10:19:00.626743+0000","last_undegraded":"2024-04-25T10:19:00.626743+0000","last_fullsized":"2024-04-25T10:19:00.626743+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:46:47.999369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631142+0000","last_change":"2024-04-25T10:18:59.614265+0000","last_active":"2024-04-25T10:19:00.631142+0000","last_peered":"2024-04-25T10:19:00.631142+0000","last_clean":"2024-04-25T10:19:00.631142+0000","last_became_active":"2024-04-25T10:18:59.609382+0000","last_became_peered":"2024-04-25T10:18:59.609382+0000","last_unstale":"2024-04-25T10:19:00.631142+0000","last_undegraded":"2024-04-25T10:19:00.631142+0000","last_fullsized":"2024-04-25T10:19:00.631142+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:59.500943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216915+0000","last_change":"2024-04-25T10:18:58.622104+0000","last_active":"2024-04-25T10:19:01.216915+0000","last_peered":"2024-04-25T10:19:01.216915+0000","last_clean":"2024-04-25T10:19:01.216915+0000","last_became_active":"2024-04-25T10:18:58.621629+0000","last_became_peered":"2024-04-25T10:18:58.621629+0000","last_unstale":"2024-04-25T10:19:01.216915+0000","last_undegraded":"2024-04-25T10:19:01.216915+0000","last_fullsized":"2024-04-25T10:19:01.216915+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:35:25.121221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631053+0000","last_change":"2024-04-25T10:18:59.619693+0000","last_active":"2024-04-25T10:19:00.631053+0000","last_peered":"2024-04-25T10:19:00.631053+0000","last_clean":"2024-04-25T10:19:00.631053+0000","last_became_active":"2024-04-25T10:18:59.618985+0000","last_became_peered":"2024-04-25T10:18:59.618985+0000","last_unstale":"2024-04-25T10:19:00.631053+0000","last_undegraded":"2024-04-25T10:19:00.631053+0000","last_fullsized":"2024-04-25T10:19:00.631053+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:00:14.430640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216031+0000","last_change":"2024-04-25T10:18:58.625155+0000","last_active":"2024-04-25T10:19:01.216031+0000","last_peered":"2024-04-25T10:19:01.216031+0000","last_clean":"2024-04-25T10:19:01.216031+0000","last_became_active":"2024-04-25T10:18:58.624868+0000","last_became_peered":"2024-04-25T10:18:58.624868+0000","last_unstale":"2024-04-25T10:19:01.216031+0000","last_undegraded":"2024-04-25T10:19:01.216031+0000","last_fullsized":"2024-04-25T10:19:01.216031+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:36:58.829842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.217665+0000","last_change":"2024-04-25T10:18:59.615266+0000","last_active":"2024-04-25T10:19:01.217665+0000","last_peered":"2024-04-25T10:19:01.217665+0000","last_clean":"2024-04-25T10:19:01.217665+0000","last_became_active":"2024-04-25T10:18:59.614605+0000","last_became_peered":"2024-04-25T10:18:59.614605+0000","last_unstale":"2024-04-25T10:19:01.217665+0000","last_undegraded":"2024-04-25T10:19:01.217665+0000","last_fullsized":"2024-04-25T10:19:01.217665+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.942604+0000","last_change":"2024-04-25T10:18:58.618811+0000","last_active":"2024-04-25T10:19:00.942604+0000","last_peered":"2024-04-25T10:19:00.942604+0000","last_clean":"2024-04-25T10:19:00.942604+0000","last_became_active":"2024-04-25T10:18:58.618330+0000","last_became_peered":"2024-04-25T10:18:58.618330+0000","last_unstale":"2024-04-25T10:19:00.942604+0000","last_undegraded":"2024-04-25T10:19:00.942604+0000","last_fullsized":"2024-04-25T10:19:00.942604+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:26:22.460570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.216601+0000","last_change":"2024-04-25T10:18:59.632020+0000","last_active":"2024-04-25T10:19:01.216601+0000","last_peered":"2024-04-25T10:19:01.216601+0000","last_clean":"2024-04-25T10:19:01.216601+0000","last_became_active":"2024-04-25T10:18:59.629949+0000","last_became_peered":"2024-04-25T10:18:59.629949+0000","last_unstale":"2024-04-25T10:19:01.216601+0000","last_undegraded":"2024-04-25T10:19:01.216601+0000","last_fullsized":"2024-04-25T10:19:01.216601+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:48:51.574396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.631400+0000","last_change":"2024-04-25T10:18:58.623872+0000","last_active":"2024-04-25T10:19:00.631400+0000","last_peered":"2024-04-25T10:19:00.631400+0000","last_clean":"2024-04-25T10:19:00.631400+0000","last_became_active":"2024-04-25T10:18:58.623534+0000","last_became_peered":"2024-04-25T10:18:58.623534+0000","last_unstale":"2024-04-25T10:19:00.631400+0000","last_undegraded":"2024-04-25T10:19:00.631400+0000","last_fullsized":"2024-04-25T10:19:00.631400+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:43:16.626008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":12,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:00.633245+0000","last_change":"2024-04-25T10:18:59.615280+0000","last_active":"2024-04-25T10:19:00.633245+0000","last_peered":"2024-04-25T10:19:00.633245+0000","last_clean":"2024-04-25T10:19:00.633245+0000","last_became_active":"2024-04-25T10:18:59.614890+0000","last_became_peered":"2024-04-25T10:18:59.614890+0000","last_unstale":"2024-04-25T10:19:00.633245+0000","last_undegraded":"2024-04-25T10:19:00.633245+0000","last_fullsized":"2024-04-25T10:19:00.633245+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":169,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:58.569663+0000","last_clean_scrub_stamp":"2024-04-25T10:18:58.569663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:52.915850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.215355+0000","last_change":"2024-04-25T10:18:58.607179+0000","last_active":"2024-04-25T10:19:01.215355+0000","last_peered":"2024-04-25T10:19:01.215355+0000","last_clean":"2024-04-25T10:19:01.215355+0000","last_became_active":"2024-04-25T10:18:58.606920+0000","last_became_peered":"2024-04-25T10:18:58.606920+0000","last_unstale":"2024-04-25T10:19:01.215355+0000","last_undegraded":"2024-04-25T10:19:01.215355+0000","last_fullsized":"2024-04-25T10:19:01.215355+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:18:57.552615+0000","last_clean_scrub_stamp":"2024-04-25T10:18:57.552615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:02:29.456350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":457,"reported_epoch":171,"state":"active+clean","last_fresh":"2024-04-25T10:19:01.218023+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:19:01.218023+0000","last_peered":"2024-04-25T10:19:01.218023+0000","last_clean":"2024-04-25T10:19:01.218023+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:19:01.218023+0000","last_undegraded":"2024-04-25T10:19:01.218023+0000","last_fullsized":"2024-04-25T10:19:01.218023+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575058,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60240,"kb_used_data":2684,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311600,"statfs":{"total":96636764160,"available":96575078400,"internally_reserved":0,"allocated":2748416,"data_stored":1603106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575058,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2158592,"data_stored":1012808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575058,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55516,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316324,"statfs":{"total":96636764160,"available":96579915776,"internally_reserved":0,"allocated":2170880,"data_stored":1014372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575058,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87380,"kb_used_data":2120,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284460,"statfs":{"total":96636764160,"available":96547287040,"internally_reserved":0,"allocated":2170880,"data_stored":1014384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575058,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87380,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284460,"statfs":{"total":96636764160,"available":96547287040,"internally_reserved":0,"allocated":2162688,"data_stored":1012898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575057,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59664,"kb_used_data":2108,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312176,"statfs":{"total":96636764160,"available":96575668224,"internally_reserved":0,"allocated":2158592,"data_stored":1013320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575058,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59652,"kb_used_data":2104,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312188,"statfs":{"total":96636764160,"available":96575680512,"internally_reserved":0,"allocated":2154496,"data_stored":1012738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575058,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47772,"kb_used_data":2700,"kb_used_omap":1,"kb_used_meta":45054,"kb_avail":94324068,"statfs":{"total":96636764160,"available":96587845632,"internally_reserved":0,"allocated":2764800,"data_stored":1603834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":46135749},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:19:04.541 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:19:04.541 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:19:04.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.793+0000 7f10b3600640 1 Processor -- start 2024-04-25T10:19:04.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.793+0000 7f10b3600640 1 -- start start 2024-04-25T10:19:04.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.794+0000 7f10b3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f10ac0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.794+0000 7f10b3600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f10ac0d32f0 0x7f10ac0cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.794+0000 7f10b3600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10ac15e000 con 0x7f10ac161710 2024-04-25T10:19:04.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.794+0000 7f10b3600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10ac0d1de0 con 0x7f10ac0d32f0 2024-04-25T10:19:04.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.794+0000 7f10b3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac161c60 con 0x7f10ac15dc20 2024-04-25T10:19:04.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.795+0000 7f10abe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f10ac161710 0x7f10ac06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54106/0 (socket says 172.21.15.165:54106) 2024-04-25T10:19:04.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.795+0000 7f10abe00640 1 -- 172.21.15.165:0/384183810 learned_addr learned my addr 172.21.15.165:0/384183810 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:04.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.795+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2790900620 0 0) 0x7f10ac15e000 con 0x7f10ac161710 2024-04-25T10:19:04.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.795+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1090003520 con 0x7f10ac161710 2024-04-25T10:19:04.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.795+0000 7f10b0c00640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f10ac0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.795+0000 7f10b1600640 1 --2- 172.21.15.165:0/384183810 >> v2:172.21.15.172:3300/0 conn(0x7f10ac0d32f0 0x7f10ac0cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1081041094 0 0) 0x7f1090003520 con 0x7f10ac161710 2024-04-25T10:19:04.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 msgr2=0x7f10ac0ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f10ac0ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 >> v2:172.21.15.172:3300/0 conn(0x7f10ac0d32f0 msgr2=0x7f10ac0cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 --2- 172.21.15.165:0/384183810 >> v2:172.21.15.172:3300/0 conn(0x7f10ac0d32f0 0x7f10ac0cebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac161f40 con 0x7f10ac161710 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1094001670 con 0x7f10ac161710 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 >> v1:172.21.15.165:6789/0 conn(0x7f10ac161710 legacy=0x7f10ac06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10900039b0 0x7f1090003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f1090004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1090004d40 0x7f10900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0d1de0 con 0x7f1090004d40 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac161c60 con 0x7f10900039b0 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109000baf0 con 0x7f10ac15dc20 2024-04-25T10:19:04.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10b1600640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1090004d40 0x7f10900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.796+0000 7f10b0c00640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10900039b0 0x7f1090003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10abe00640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f1090004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10b1600640 1 -- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 msgr2=0x7f1090004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10b1600640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f1090004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10b1600640 1 -- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10900039b0 msgr2=0x7f1090003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10b1600640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f10900039b0 0x7f1090003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10b1600640 1 -- 172.21.15.165:0/384183810 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1090003520 con 0x7f1090004d40 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.797+0000 7f10abe00640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac15dc20 0x7f1090004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:04.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.798+0000 7f10b1600640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1090004d40 0x7f10900051c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f10a400ba10 tx=0x7f1090005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef18f6fb639dde97 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.798+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4030070 con 0x7f1090004d40 2024-04-25T10:19:04.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.798+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a402ae00 con 0x7f1090004d40 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.798+0000 7f10ab400640 1 -- 172.21.15.165:0/384183810 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400b610 con 0x7f1090004d40 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.798+0000 7f10b3600640 1 -- 172.21.15.165:0/384183810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1090004d40 msgr2=0x7f10900051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.798+0000 7f10b3600640 1 --2- 172.21.15.165:0/384183810 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1090004d40 0x7f10900051c0 secure :-1 s=READY pgs=2438 cs=0 l=1 rev1=1 crypto rx=0x7f10a400ba10 tx=0x7f1090005830 comp rx=0 tx=0).stop 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.799+0000 7f10b1600640 1 -- 172.21.15.165:0/384183810 reap_dead start 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.799+0000 7f10b3600640 1 -- 172.21.15.165:0/384183810 shutdown_connections 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.799+0000 7f10b3600640 1 -- 172.21.15.165:0/384183810 >> 172.21.15.165:0/384183810 conn(0x7f10ac0d0420 msgr2=0x7f10ac0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.799+0000 7f10b3600640 1 -- 172.21.15.165:0/384183810 shutdown_connections 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.799+0000 7f10b3600640 1 -- 172.21.15.165:0/384183810 wait complete. 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.800+0000 7f10b3600640 1 Processor -- start 2024-04-25T10:19:04.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.801+0000 7f10b3600640 1 -- start start 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.801+0000 7f10b3600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1090004d40 0x7f10ac0caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b3600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac0c9c00 0x7f10ac150da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b3600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 0x7f10ac0b8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b3600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b5e50 con 0x7f10ac0b86f0 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b3600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac162500 con 0x7f1090004d40 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b3600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10ac0b5fd0 con 0x7f10ac0c9c00 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 0x7f10ac0b8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 0x7f10ac0b8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40952/0 (socket says 172.21.15.165:40952) 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 -- 172.21.15.165:0/4254406069 learned_addr learned my addr 172.21.15.165:0/4254406069 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 -- 172.21.15.165:0/4254406069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac0c9c00 msgr2=0x7f10ac150da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac0c9c00 0x7f10ac150da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 -- 172.21.15.165:0/4254406069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1090004d40 msgr2=0x7f10ac0caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1090004d40 0x7f10ac0caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.802+0000 7f10b1600640 1 -- 172.21.15.165:0/4254406069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10ac0cb350 con 0x7f10ac0b86f0 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.803+0000 7f10b1600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 0x7f10ac0b8b70 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f10a402a8f0 tx=0x7f10a4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.803+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a4042030 con 0x7f10ac0b86f0 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.803+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10a402ac00 con 0x7f10ac0b86f0 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.803+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10ac0cb630 con 0x7f10ac0b86f0 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.803+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10a400b610 con 0x7f10ac0b86f0 2024-04-25T10:19:04.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.803+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10ac0cbbf0 con 0x7f10ac0b86f0 2024-04-25T10:19:04.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.804+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10ac073a00 con 0x7f10ac0b86f0 2024-04-25T10:19:04.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.805+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f10a403d650 con 0x7f10ac0b86f0 2024-04-25T10:19:04.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.806+0000 7f10a8c00640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b500 0x7f108806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:04.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.806+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f10a40a9f60 con 0x7f10ac0b86f0 2024-04-25T10:19:04.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.806+0000 7f10b0c00640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b500 0x7f108806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:04.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.807+0000 7f10b0c00640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b500 0x7f108806d9c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f109c0042d0 tx=0x7f109c003990 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:04.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:04.810+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f10a404b020 con 0x7f10ac0b86f0 2024-04-25T10:19:05.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.064+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f10ac0d16b0 con 0x7f10ac0b86f0 2024-04-25T10:19:05.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.068+0000 7f10a8c00640 1 -- 172.21.15.165:0/4254406069 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v24) v1 ==== 190+0+147 (secure 0 0 0) 0x7f10a4078c40 con 0x7f10ac0b86f0 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:19:05.072 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:19:05.073 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:19:05.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b500 msgr2=0x7f108806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b500 0x7f108806d9c0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f109c0042d0 tx=0x7f109c003990 comp rx=0 tx=0).stop 2024-04-25T10:19:05.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 msgr2=0x7f10ac0b8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 0x7f10ac0b8b70 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7f10a402a8f0 tx=0x7f10a4004730 comp rx=0 tx=0).stop 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 shutdown_connections 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f108806b500 0x7f108806d9c0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f10ac0b86f0 0x7f10ac0b8b70 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f10ac0c9c00 0x7f10ac150da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 --2- 172.21.15.165:0/4254406069 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1090004d40 0x7f10ac0caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 >> 172.21.15.165:0/4254406069 conn(0x7f10ac0d0420 msgr2=0x7f10ac15cef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 shutdown_connections 2024-04-25T10:19:05.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.072+0000 7f10b3600640 1 -- 172.21.15.165:0/4254406069 wait complete. 2024-04-25T10:19:05.092 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:05.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.284+0000 7f9b42e00640 1 Processor -- start 2024-04-25T10:19:05.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.284+0000 7f9b42e00640 1 -- start start 2024-04-25T10:19:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9b3c0d04b0 0x7f9b3c15e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9b3c0b1ae0 0x7f9b3c0d0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b3c0b1f50 con 0x7f9b3c0b34c0 2024-04-25T10:19:05.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c15b990 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b5130 con 0x7f9b3c0b1ae0 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b41a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9b3c0d04b0 0x7f9b3c15e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9b3c0b34c0 0x7f9b3c06abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54112/0 (socket says 172.21.15.165:54112) 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b42400640 1 -- 172.21.15.165:0/1877217223 learned_addr learned my addr 172.21.15.165:0/1877217223 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b41000640 1 --2- 172.21.15.165:0/1877217223 >> v2:172.21.15.172:3300/0 conn(0x7f9b3c0b1ae0 0x7f9b3c0d0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b41a00640 1 -- 172.21.15.165:0/1877217223 >> v2:172.21.15.172:3300/0 conn(0x7f9b3c0b1ae0 msgr2=0x7f9b3c0d0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.286+0000 7f9b41a00640 1 --2- 172.21.15.165:0/1877217223 >> v2:172.21.15.172:3300/0 conn(0x7f9b3c0b1ae0 0x7f9b3c0d0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.287+0000 7f9b41a00640 1 -- 172.21.15.165:0/1877217223 >> v1:172.21.15.165:6789/0 conn(0x7f9b3c0b34c0 legacy=0x7f9b3c06abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.287+0000 7f9b41a00640 1 -- 172.21.15.165:0/1877217223 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c0d0dd0 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.287+0000 7f9b3be00640 1 -- 172.21.15.165:0/1877217223 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1729054289 0 0) 0x7f9b34001490 con 0x7f9b3c0b34c0 2024-04-25T10:19:05.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.287+0000 7f9b41000640 1 --2- 172.21.15.165:0/1877217223 >> v2:172.21.15.172:3300/0 conn(0x7f9b3c0b1ae0 0x7f9b3c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.288+0000 7f9b41a00640 1 --2- 172.21.15.165:0/1877217223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c15e9b0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c073120 tx=0x7f9b3002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e98de2c521967a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.288+0000 7f9b3be00640 1 -- 172.21.15.165:0/1877217223 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30037070 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.288+0000 7f9b3be00640 1 -- 172.21.15.165:0/1877217223 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b30032440 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.288+0000 7f9b3be00640 1 -- 172.21.15.165:0/1877217223 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b30032810 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 -- 172.21.15.165:0/1877217223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 msgr2=0x7f9b3c15e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 --2- 172.21.15.165:0/1877217223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c15e9b0 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f9b3c073120 tx=0x7f9b3002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 -- 172.21.15.165:0/1877217223 shutdown_connections 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 --2- 172.21.15.165:0/1877217223 >> v2:172.21.15.172:3300/0 conn(0x7f9b3c0b1ae0 0x7f9b3c0d0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 --2- 172.21.15.165:0/1877217223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c15e9b0 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 -- 172.21.15.165:0/1877217223 >> 172.21.15.165:0/1877217223 conn(0x7f9b3c0b60c0 msgr2=0x7f9b3c099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 -- 172.21.15.165:0/1877217223 shutdown_connections 2024-04-25T10:19:05.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.289+0000 7f9b42e00640 1 -- 172.21.15.165:0/1877217223 wait complete. 2024-04-25T10:19:05.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.290+0000 7f9b42e00640 1 Processor -- start 2024-04-25T10:19:05.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 -- start start 2024-04-25T10:19:05.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b3c0b1ae0 0x7f9b3c14d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3c0b34c0 0x7f9b3c09cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c09d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b5390 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0af660 con 0x7f9b3c0b34c0 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.291+0000 7f9b42e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b3c0b5670 con 0x7f9b3c0b1ae0 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c09d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c09d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40964/0 (socket says 172.21.15.165:40964) 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 -- 172.21.15.165:0/2750713323 learned_addr learned my addr 172.21.15.165:0/2750713323 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 -- 172.21.15.165:0/2750713323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b3c0b1ae0 msgr2=0x7f9b3c14d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b3c0b1ae0 0x7f9b3c14d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 -- 172.21.15.165:0/2750713323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3c0b34c0 msgr2=0x7f9b3c09cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3c0b34c0 0x7f9b3c09cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 -- 172.21.15.165:0/2750713323 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b3c09b450 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.292+0000 7f9b42400640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c09d2f0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400cb90 tx=0x7f9b3400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.293+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b3400ace0 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.293+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b34012400 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.293+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b340127f0 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.293+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b3c09b730 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.293+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b3c09bcf0 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.294+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b3c073990 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.295+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b34012990 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.296+0000 7f9b39600640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b1c06b4d0 0x7f9b1c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:05.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.296+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9b3408aa70 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.296+0000 7f9b41a00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b1c06b4d0 0x7f9b1c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:05.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.297+0000 7f9b41a00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b1c06b4d0 0x7f9b1c06d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9b30034000 tx=0x7f9b3002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:05.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.300+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b3c0b1f50 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.530+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9b3c0d0a30 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.531+0000 7f9b39600640 1 -- 172.21.15.165:0/2750713323 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f9b34059750 con 0x7f9b3c0d04b0 2024-04-25T10:19:05.534 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:05.535 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:05.535 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:05.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b1c06b4d0 msgr2=0x7f9b1c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b1c06b4d0 0x7f9b1c06d990 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f9b30034000 tx=0x7f9b3002c000 comp rx=0 tx=0).stop 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 msgr2=0x7f9b3c09d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c09d2f0 secure :-1 s=READY pgs=2442 cs=0 l=1 rev1=1 crypto rx=0x7f9b3400cb90 tx=0x7f9b3400a430 comp rx=0 tx=0).stop 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 shutdown_connections 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b1c06b4d0 0x7f9b1c06d990 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b3c0d04b0 0x7f9b3c09d2f0 unknown :-1 s=CLOSED pgs=2442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b3c0b34c0 0x7f9b3c09cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 --2- 172.21.15.165:0/2750713323 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b3c0b1ae0 0x7f9b3c14d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:05.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 >> 172.21.15.165:0/2750713323 conn(0x7f9b3c0b60c0 msgr2=0x7f9b3c0b0cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:05.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 shutdown_connections 2024-04-25T10:19:05.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:05.534+0000 7f9b42e00640 1 -- 172.21.15.165:0/2750713323 wait complete. 2024-04-25T10:19:05.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 121, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:18:59.043438+0000', 'modified': '2024-04-25T10:19:02.102707+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10767}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10767': {'gid': 10767, 'name': 'a', 'rank': 0, 'incarnation': 117, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6836/1299523073', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6834', 'nonce': 1299523073}, {'type': 'v1', 'addr': '172.21.15.165:6836', 'nonce': 1299523073}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:19:05.957 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:19:05.957 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-04-25T10:19:06.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.167+0000 7f98dd000640 1 Processor -- start 2024-04-25T10:19:06.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.167+0000 7f98dd000640 1 -- start start 2024-04-25T10:19:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.169+0000 7f98dd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98d80b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.169+0000 7f98dd000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d80b0b80 con 0x7f98d815b3c0 2024-04-25T10:19:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.169+0000 7f98dd000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d80cc800 con 0x7f98d815b7a0 2024-04-25T10:19:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.169+0000 7f98dd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80cd650 con 0x7f98d815e720 2024-04-25T10:19:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.169+0000 7f98d7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f98d815b3c0 0x7f98d80cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54120/0 (socket says 172.21.15.165:54120) 2024-04-25T10:19:06.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.169+0000 7f98d7400640 1 -- 172.21.15.165:0/4279347666 learned_addr learned my addr 172.21.15.165:0/4279347666 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597246245 0 0) 0x7f98c8001110 con 0x7f98d815b3c0 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98bc003520 con 0x7f98d815b3c0 2024-04-25T10:19:06.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d7e00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98d80b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 553573121 0 0) 0x7f98bc003520 con 0x7f98d815b3c0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 msgr2=0x7f98d80b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98d80b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 >> v1:172.21.15.172:6789/0 conn(0x7f98d815b7a0 legacy=0x7f98d815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80ccae0 con 0x7f98d815b3c0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f98c8001840 con 0x7f98d815b3c0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 >> v1:172.21.15.165:6789/0 conn(0x7f98d815b3c0 legacy=0x7f98d80cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98bc0039e0 0x7f98bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.170+0000 7f98d6a00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d6a00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc004d40 0x7f98bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80cd650 con 0x7f98bc0039e0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc00bbd0 con 0x7f98bc004d40 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98bc00bdd0 con 0x7f98d815e720 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7400640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98bc0039e0 0x7f98bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7e00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc004d40 0x7f98bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98cfe00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7400640 1 -- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 msgr2=0x7f98bc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7400640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98bc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7400640 1 -- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc004d40 msgr2=0x7f98bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7400640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc004d40 0x7f98bc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7400640 1 -- 172.21.15.165:0/4279347666 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98bc003520 con 0x7f98bc0039e0 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98d7e00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98bc004d40 0x7f98bc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:06.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.171+0000 7f98cfe00640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98d815e720 0x7f98bc004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:06.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.173+0000 7f98d7400640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98bc0039e0 0x7f98bc003e00 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c320 tx=0x7f98bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bef536512db03920 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.173+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8007e10 con 0x7f98bc0039e0 2024-04-25T10:19:06.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.173+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8005ad0 con 0x7f98bc0039e0 2024-04-25T10:19:06.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.173+0000 7f98d6a00640 1 -- 172.21.15.165:0/4279347666 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8007070 con 0x7f98bc0039e0 2024-04-25T10:19:06.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98dd000640 1 -- 172.21.15.165:0/4279347666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98bc0039e0 msgr2=0x7f98bc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98dd000640 1 --2- 172.21.15.165:0/4279347666 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98bc0039e0 0x7f98bc003e00 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c320 tx=0x7f98bc005830 comp rx=0 tx=0).stop 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98d7e00640 1 -- 172.21.15.165:0/4279347666 reap_dead start 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98dd000640 1 -- 172.21.15.165:0/4279347666 shutdown_connections 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98dd000640 1 -- 172.21.15.165:0/4279347666 >> 172.21.15.165:0/4279347666 conn(0x7f98d8061cc0 msgr2=0x7f98d806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98dd000640 1 -- 172.21.15.165:0/4279347666 shutdown_connections 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.174+0000 7f98dd000640 1 -- 172.21.15.165:0/4279347666 wait complete. 2024-04-25T10:19:06.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.176+0000 7f98dd000640 1 Processor -- start 2024-04-25T10:19:06.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.176+0000 7f98dd000640 1 -- start start 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98dd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98bc0039e0 0x7f98d8152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98dd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8155bb0 0x7f98d81600a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98dd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 0x7f98d8152540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98dd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80ce370 con 0x7f98d81520c0 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98dd000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80ce1f0 con 0x7f98d8155bb0 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98dd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d80b0f60 con 0x7f98bc0039e0 2024-04-25T10:19:06.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.177+0000 7f98d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 0x7f98d8152540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 0x7f98d8152540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40996/0 (socket says 172.21.15.165:40996) 2024-04-25T10:19:06.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 -- 172.21.15.165:0/3886870862 learned_addr learned my addr 172.21.15.165:0/3886870862 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7400640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98bc0039e0 0x7f98d8152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 -- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98bc0039e0 msgr2=0x7f98d8152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98bc0039e0 0x7f98d8152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 -- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8155bb0 msgr2=0x7f98d81600a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8155bb0 0x7f98d81600a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 -- 172.21.15.165:0/3886870862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d81538c0 con 0x7f98d81520c0 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98d7e00640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 0x7f98d8152540 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f98d80cba40 tx=0x7f98d0009d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.178+0000 7f98cf400640 1 -- 172.21.15.165:0/3886870862 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0019070 con 0x7f98d81520c0 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.179+0000 7f98cf400640 1 -- 172.21.15.165:0/3886870862 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98d0007d10 con 0x7f98d81520c0 2024-04-25T10:19:06.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.179+0000 7f98cf400640 1 -- 172.21.15.165:0/3886870862 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0014640 con 0x7f98d81520c0 2024-04-25T10:19:06.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.179+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d81531f0 con 0x7f98d81520c0 2024-04-25T10:19:06.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.179+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d81f5320 con 0x7f98d81520c0 2024-04-25T10:19:06.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.181+0000 7f98cf400640 1 -- 172.21.15.165:0/3886870862 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f98d001c070 con 0x7f98d81520c0 2024-04-25T10:19:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.182+0000 7f98cf400640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98b406b4d0 0x7f98b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.182+0000 7f98cf400640 1 -- 172.21.15.165:0/3886870862 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98d0089e70 con 0x7f98d81520c0 2024-04-25T10:19:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.182+0000 7f98e140a740 1 Processor -- start 2024-04-25T10:19:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.183+0000 7f98d7400640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98b406b4d0 0x7f98b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.183+0000 7f98e140a740 1 -- start start 2024-04-25T10:19:06.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.183+0000 7f98d7400640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98b406b4d0 0x7f98b406d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f98c8033000 tx=0x7f98c802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.183+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a49b30 0x55a5e9a71970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98e140a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a49f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9a482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98e140a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e99ad620 con 0x55a5e9a71eb0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9956ca0 con 0x55a5e9a927a0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5e99a96c0 con 0x55a5e9a49b30 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a49f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a49f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41004/0 (socket says 172.21.15.165:41004) 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 -- 172.21.15.165:0/2195459413 learned_addr learned my addr 172.21.15.165:0/2195459413 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 -- 172.21.15.165:0/2195459413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a49b30 msgr2=0x55a5e9a71970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a49b30 0x55a5e9a71970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 -- 172.21.15.165:0/2195459413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 msgr2=0x55a5e9a482f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9a482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 -- 172.21.15.165:0/2195459413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e995df40 con 0x55a5e9a71eb0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.184+0000 7f98cfe00640 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a49f10 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f98c0001730 tx=0x7f98c0001bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd651ba4fc6342c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98c7e00640 1 -- 172.21.15.165:0/2195459413 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000db40 con 0x55a5e9a71eb0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98c7e00640 1 -- 172.21.15.165:0/2195459413 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000dd00 con 0x55a5e9a71eb0 2024-04-25T10:19:06.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98c7e00640 1 -- 172.21.15.165:0/2195459413 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007270 con 0x55a5e9a71eb0 2024-04-25T10:19:06.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 -- 172.21.15.165:0/2195459413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 msgr2=0x55a5e9a49f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a49f10 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f98c0001730 tx=0x7f98c0001bf0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 -- 172.21.15.165:0/2195459413 shutdown_connections 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9a482f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a49f10 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 --2- 172.21.15.165:0/2195459413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a49b30 0x55a5e9a71970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.185+0000 7f98e140a740 1 -- 172.21.15.165:0/2195459413 >> 172.21.15.165:0/2195459413 conn(0x55a5e9a46d50 msgr2=0x55a5e9a8f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.186+0000 7f98e140a740 1 -- 172.21.15.165:0/2195459413 shutdown_connections 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.186+0000 7f98e140a740 1 -- 172.21.15.165:0/2195459413 wait complete. 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.186+0000 7f98e140a740 1 -- start start 2024-04-25T10:19:06.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.186+0000 7f98e140a740 1 Processor -- start 2024-04-25T10:19:06.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.187+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a71eb0 0x55a5e9a673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.187+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9aa3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.187+0000 7f98e140a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 0x55a5e9a629c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.187+0000 7f98e140a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9a714e0 con 0x55a5e9a62540 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.187+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e994c1f0 con 0x55a5e9a927a0 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9a70ed0 con 0x55a5e9a71eb0 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98e140a740 10 client.0 _ll_register_callbacks cb 0x55a5e9969ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 0x55a5e9a629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 0x55a5e9a629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41020/0 (socket says 172.21.15.165:41020) 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 -- 172.21.15.165:0/1272065354 learned_addr learned my addr 172.21.15.165:0/1272065354 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 -- 172.21.15.165:0/1272065354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a71eb0 msgr2=0x55a5e9a673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a71eb0 0x55a5e9a673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 -- 172.21.15.165:0/1272065354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 msgr2=0x55a5e9aa3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9aa3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 -- 172.21.15.165:0/1272065354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e996bc50 con 0x55a5e9a62540 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98d7e00640 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 0x55a5e9a629c0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f98d0002230 tx=0x7f98d0058cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4435e655cc73a1c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.188+0000 7f98c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:06.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98c4c00640 1 -- 172.21.15.165:0/1272065354 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d0086060 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98c4c00640 1 -- 172.21.15.165:0/1272065354 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98d00588e0 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98c000dd00 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98c4c00640 1 -- 172.21.15.165:0/1272065354 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98d0075050 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 10 client.10911 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55a5e96c22d0 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98c4c00640 1 -- 172.21.15.165:0/1272065354 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000dd00 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98c4c00640 1 -- 172.21.15.165:0/1272065354 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98d0060700 con 0x55a5e9a62540 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 10 client.10911 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 10 client.10911 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 20 client.10911 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6e210 0x55a5e9a6e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 4 client.10911 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.189+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5e99bab60 con 0x55a5e9a6e210 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.190+0000 7f98d7400640 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6e210 0x55a5e9a6e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.190+0000 7f98d7400640 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6e210 0x55a5e9a6e5f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d5c4eda5108b8c75 server_cookie=43c0eda1cf5fa147 in_seq=0 out_seq=0 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.190+0000 7f98c4c00640 10 client.10911 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:06.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98c4c00640 1 -- 172.21.15.165:0/1272065354 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a5e99bab60 con 0x55a5e9a6e210 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98c4c00640 10 client.10911 handle_command_reply: tid=0 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 client.10911 shutdown 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 msgr2=0x55a5e9a629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 0x55a5e9a629c0 secure :-1 s=READY pgs=2447 cs=0 l=1 rev1=1 crypto rx=0x7f98d0002230 tx=0x7f98d0058cc0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 shutdown_connections 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6e210 0x55a5e9a6e5f0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a62540 0x55a5e9a629c0 unknown :-1 s=CLOSED pgs=2447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9aa3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 --2- 172.21.15.165:0/1272065354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a71eb0 0x55a5e9a673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 >> 172.21.15.165:0/1272065354 conn(0x55a5e9a49b30 msgr2=0x55a5e9a49f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98c4c00640 0 client.10911 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 shutdown_connections 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.191+0000 7f98e140a740 1 -- 172.21.15.165:0/1272065354 wait complete. 2024-04-25T10:19:06.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.192+0000 7f98e140a740 20 client.10911 trim_cache size 0 max 16384 2024-04-25T10:19:06.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.208+0000 7f98e140a740 1 Processor -- start 2024-04-25T10:19:06.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.208+0000 7f98e140a740 1 -- start start 2024-04-25T10:19:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.209+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a62540 0x55a5e9a403f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.209+0000 7f98e140a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a6e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.209+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9a6eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.209+0000 7f98e140a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d81ffe50 con 0x55a5e9a71eb0 2024-04-25T10:19:06.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.209+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d81f6e50 con 0x55a5e9a927a0 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.209+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d81ff850 con 0x55a5e9a62540 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a6e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a6e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41034/0 (socket says 172.21.15.165:41034) 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 -- 172.21.15.165:0/801165470 learned_addr learned my addr 172.21.15.165:0/801165470 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 -- 172.21.15.165:0/801165470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a62540 msgr2=0x55a5e9a403f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a62540 0x55a5e9a403f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 -- 172.21.15.165:0/801165470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 msgr2=0x55a5e9a6eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9a6eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 -- 172.21.15.165:0/801165470 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5e9a6c150 con 0x55a5e9a71eb0 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.210+0000 7f98cfe00640 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a6e5f0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f98c0007000 tx=0x7f98c0010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54077e189fb2595f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98c7400640 1 -- 172.21.15.165:0/801165470 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000d940 con 0x55a5e9a71eb0 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98c7400640 1 -- 172.21.15.165:0/801165470 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000db20 con 0x55a5e9a71eb0 2024-04-25T10:19:06.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98c7400640 1 -- 172.21.15.165:0/801165470 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007030 con 0x55a5e9a71eb0 2024-04-25T10:19:06.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 -- 172.21.15.165:0/801165470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 msgr2=0x55a5e9a6e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a6e5f0 secure :-1 s=READY pgs=2448 cs=0 l=1 rev1=1 crypto rx=0x7f98c0007000 tx=0x7f98c0010000 comp rx=0 tx=0).stop 2024-04-25T10:19:06.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 -- 172.21.15.165:0/801165470 shutdown_connections 2024-04-25T10:19:06.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a927a0 0x55a5e9a6eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9a6e5f0 unknown :-1 s=CLOSED pgs=2448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 --2- 172.21.15.165:0/801165470 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a62540 0x55a5e9a403f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 -- 172.21.15.165:0/801165470 >> 172.21.15.165:0/801165470 conn(0x55a5e9a49b30 msgr2=0x55a5e9a49f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 -- 172.21.15.165:0/801165470 shutdown_connections 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.211+0000 7f98e140a740 1 -- 172.21.15.165:0/801165470 wait complete. 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.212+0000 7f98e140a740 1 -- start start 2024-04-25T10:19:06.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.212+0000 7f98e140a740 1 Processor -- start 2024-04-25T10:19:06.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a62540 0x55a5e9a603f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9afa130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a927a0 0x55a5e9a91270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9a6f230 con 0x55a5e9a71eb0 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9a68370 con 0x55a5e9a62540 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a5e9a69100 con 0x55a5e9a927a0 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.213+0000 7f98e140a740 10 client.0 _ll_register_callbacks cb 0x7f98d0054720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9afa130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9afa130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41042/0 (socket says 172.21.15.165:41042) 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 -- 172.21.15.165:0/2867491774 learned_addr learned my addr 172.21.15.165:0/2867491774 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 -- 172.21.15.165:0/2867491774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a927a0 msgr2=0x55a5e9a91270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a927a0 0x55a5e9a91270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 -- 172.21.15.165:0/2867491774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a62540 msgr2=0x55a5e9a603f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a62540 0x55a5e9a603f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 -- 172.21.15.165:0/2867491774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a5e978c6b0 con 0x55a5e9a71eb0 2024-04-25T10:19:06.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98cfe00640 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9afa130 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f98c000d520 tx=0x7f98c0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b442393e7e724da8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.214+0000 7f98c4c00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98c4c00640 1 -- 172.21.15.165:0/2867491774 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0007240 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98c4c00640 1 -- 172.21.15.165:0/2867491774 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f98c00073e0 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98c000db20 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98c4c00640 1 -- 172.21.15.165:0/2867491774 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98c00075e0 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 10 client.10917 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55a5e9a91cd0 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98c4c00640 1 -- 172.21.15.165:0/2867491774 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c000db20 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98c4c00640 1 -- 172.21.15.165:0/2867491774 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f98c00067d0 con 0x55a5e9a71eb0 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 10 client.10917 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 10 client.10917 resolve_mds: resolved name 'a' to daemon mds.a 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 20 client.10917 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6f400 0x55a5e9a6f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 4 client.10917 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.215+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a5e9a474b0 con 0x55a5e9a6f400 2024-04-25T10:19:06.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.216+0000 7f98d7400640 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6f400 0x55a5e9a6f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.216+0000 7f98d7400640 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6f400 0x55a5e9a6f7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a81f9152c09e78 server_cookie=b145b51ee9d7fafd in_seq=0 out_seq=0 2024-04-25T10:19:06.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.216+0000 7f98c4c00640 10 client.10917 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.216+0000 7f98c4c00640 1 -- 172.21.15.165:0/2867491774 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55a5e9a474b0 con 0x55a5e9a6f400 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.216+0000 7f98c4c00640 10 client.10917 handle_command_reply: tid=0 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.216+0000 7f98e140a740 1 client.10917 shutdown 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 msgr2=0x55a5e9afa130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9afa130 secure :-1 s=READY pgs=2449 cs=0 l=1 rev1=1 crypto rx=0x7f98c000d520 tx=0x7f98c0000bc0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 shutdown_connections 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55a5e9a6f400 0x55a5e9a6f7e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a5e9a927a0 0x55a5e9a91270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a5e9a71eb0 0x55a5e9afa130 unknown :-1 s=CLOSED pgs=2449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98c4c00640 0 client.10917 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:06.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 --2- 172.21.15.165:0/2867491774 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a5e9a62540 0x55a5e9a603f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 >> 172.21.15.165:0/2867491774 conn(0x55a5e9a49b30 msgr2=0x55a5e9a6e5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 shutdown_connections 2024-04-25T10:19:06.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 1 -- 172.21.15.165:0/2867491774 wait complete. 2024-04-25T10:19:06.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.217+0000 7f98e140a740 20 client.10917 trim_cache size 0 max 16384 2024-04-25T10:19:06.223 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 10767, 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.6261207280000001, 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 121, 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 171, 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 171, 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 13.094631896999999 2024-04-25T10:19:06.224 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:06.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98b406b4d0 msgr2=0x7f98b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98b406b4d0 0x7f98b406d990 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f98c8033000 tx=0x7f98c802e000 comp rx=0 tx=0).stop 2024-04-25T10:19:06.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 msgr2=0x7f98d8152540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 0x7f98d8152540 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f98d80cba40 tx=0x7f98d0009d50 comp rx=0 tx=0).stop 2024-04-25T10:19:06.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 shutdown_connections 2024-04-25T10:19:06.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f98b406b4d0 0x7f98b406d990 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f98d81520c0 0x7f98d8152540 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f98d8155bb0 0x7f98d81600a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 --2- 172.21.15.165:0/3886870862 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f98bc0039e0 0x7f98d8152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 >> 172.21.15.165:0/3886870862 conn(0x7f98d8061cc0 msgr2=0x7f98d815a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 shutdown_connections 2024-04-25T10:19:06.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.218+0000 7f98dd000640 1 -- 172.21.15.165:0/3886870862 wait complete. 2024-04-25T10:19:06.237 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:19:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.413+0000 7f5e3a800640 1 Processor -- start 2024-04-25T10:19:06.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.413+0000 7f5e3a800640 1 -- start start 2024-04-25T10:19:06.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.415+0000 7f5e3a800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e34144440 0x7f5e34070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.415+0000 7f5e3a800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3415bf30 con 0x7f5e340cda30 2024-04-25T10:19:06.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.415+0000 7f5e3a800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3415c210 con 0x7f5e34071530 2024-04-25T10:19:06.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.415+0000 7f5e3a800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e340719a0 con 0x7f5e34144440 2024-04-25T10:19:06.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e39e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5e340cda30 0x7f5e3415ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:54124/0 (socket says 172.21.15.165:54124) 2024-04-25T10:19:06.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e39e00640 1 -- 172.21.15.165:0/3393429848 learned_addr learned my addr 172.21.15.165:0/3393429848 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e39400640 1 --2- 172.21.15.165:0/3393429848 >> v2:172.21.15.172:3300/0 conn(0x7f5e34144440 0x7f5e34070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1754317150 0 0) 0x7f5e3415bf30 con 0x7f5e340cda30 2024-04-25T10:19:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5e1c003520 con 0x7f5e340cda30 2024-04-25T10:19:06.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2516779467 0 0) 0x7f5e1c003520 con 0x7f5e340cda30 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 >> v2:172.21.15.172:3300/0 conn(0x7f5e34144440 msgr2=0x7f5e34070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3393429848 >> v2:172.21.15.172:3300/0 conn(0x7f5e34144440 0x7f5e34070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 >> v1:172.21.15.172:6789/0 conn(0x7f5e34071530 legacy=0x7f5e340cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3415c4f0 con 0x7f5e340cda30 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.416+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5e2c0019f0 con 0x7f5e340cda30 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 >> v1:172.21.15.165:6789/0 conn(0x7f5e340cda30 legacy=0x7f5e3415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e1c0039b0 0x7f5e1c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e1c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e1c0051c0 0x7f5e1c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340719a0 con 0x7f5e1c004800 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c00c070 con 0x7f5e1c0051c0 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e1c00c270 con 0x7f5e1c0039b0 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e38a00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e1c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e38a00640 1 -- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e1c0039b0 msgr2=0x7f5e1c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e38a00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e1c0039b0 0x7f5e1c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e39e00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e1c0051c0 0x7f5e1c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e38a00640 1 -- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e1c0051c0 msgr2=0x7f5e1c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:06.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e38a00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e1c0051c0 0x7f5e1c00b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e38a00640 1 -- 172.21.15.165:0/3393429848 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e1c003520 con 0x7f5e1c004800 2024-04-25T10:19:06.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.417+0000 7f5e39e00640 1 -- 172.21.15.165:0/3393429848 reap_dead start 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.418+0000 7f5e38a00640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e1c004c80 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f5e20001750 tx=0x7f5e1c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da20e74263dd50f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20001370 con 0x7f5e1c004800 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e200052d0 con 0x7f5e1c004800 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e33e00640 1 -- 172.21.15.165:0/3393429848 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2002abc0 con 0x7f5e1c004800 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e3a800640 1 -- 172.21.15.165:0/3393429848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 msgr2=0x7f5e1c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e3a800640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e1c004c80 secure :-1 s=READY pgs=2451 cs=0 l=1 rev1=1 crypto rx=0x7f5e20001750 tx=0x7f5e1c005830 comp rx=0 tx=0).stop 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e3a800640 1 -- 172.21.15.165:0/3393429848 shutdown_connections 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e3a800640 1 --2- 172.21.15.165:0/3393429848 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e1c004c80 unknown :-1 s=CLOSED pgs=2451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.419+0000 7f5e3a800640 1 -- 172.21.15.165:0/3393429848 >> 172.21.15.165:0/3393429848 conn(0x7f5e3406abe0 msgr2=0x7f5e3406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.420+0000 7f5e3a800640 1 -- 172.21.15.165:0/3393429848 shutdown_connections 2024-04-25T10:19:06.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.420+0000 7f5e3a800640 1 -- 172.21.15.165:0/3393429848 wait complete. 2024-04-25T10:19:06.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.421+0000 7f5e3a800640 1 Processor -- start 2024-04-25T10:19:06.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.421+0000 7f5e3a800640 1 -- start start 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.422+0000 7f5e3a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e341e5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.422+0000 7f5e3a800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e341e7100 0x7f5e340cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.422+0000 7f5e3a800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3415ac30 0x7f5e3415b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.422+0000 7f5e3a800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340cdea0 con 0x7f5e1c004800 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.422+0000 7f5e3a800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3415c860 con 0x7f5e3415ac30 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.422+0000 7f5e3a800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34070ff0 con 0x7f5e341e7100 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e341e5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e341e5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41070/0 (socket says 172.21.15.165:41070) 2024-04-25T10:19:06.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 -- 172.21.15.165:0/1287287718 learned_addr learned my addr 172.21.15.165:0/1287287718 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e38a00640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e341e7100 0x7f5e340cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 -- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e341e7100 msgr2=0x7f5e340cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e341e7100 0x7f5e340cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 -- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3415ac30 msgr2=0x7f5e3415b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3415ac30 0x7f5e3415b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 -- 172.21.15.165:0/1287287718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e341e6060 con 0x7f5e1c004800 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.423+0000 7f5e39400640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e341e5460 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e770 tx=0x7f5e2800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.424+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28017070 con 0x7f5e1c004800 2024-04-25T10:19:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.424+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2800c730 con 0x7f5e1c004800 2024-04-25T10:19:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.424+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2800cb20 con 0x7f5e1c004800 2024-04-25T10:19:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.424+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e341e6340 con 0x7f5e1c004800 2024-04-25T10:19:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.424+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e341e6900 con 0x7f5e1c004800 2024-04-25T10:19:06.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.425+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e34073990 con 0x7f5e1c004800 2024-04-25T10:19:06.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.427+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e2801a070 con 0x7f5e1c004800 2024-04-25T10:19:06.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.427+0000 7f5e31600640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:06.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.427+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e2808a850 con 0x7f5e1c004800 2024-04-25T10:19:06.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.428+0000 7f5e38a00640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:06.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.428+0000 7f5e38a00640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5e20032000 tx=0x7f5e2002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:06.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.431+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e280594a0 con 0x7f5e1c004800 2024-04-25T10:19:06.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.671+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5e3406b0f0 con 0x7f5e1c004800 2024-04-25T10:19:06.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.673+0000 7f5e31600640 1 -- 172.21.15.165:0/1287287718 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v121) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5e28058da0 con 0x7f5e1c004800 2024-04-25T10:19:06.676 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 msgr2=0x7f5e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f5e20032000 tx=0x7f5e2002e000 comp rx=0 tx=0).stop 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 msgr2=0x7f5e341e5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e341e5460 secure :-1 s=READY pgs=2452 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800e770 tx=0x7f5e2800ec30 comp rx=0 tx=0).stop 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 shutdown_connections 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e3415ac30 0x7f5e3415b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e341e7100 0x7f5e340cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 --2- 172.21.15.165:0/1287287718 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e1c004800 0x7f5e341e5460 unknown :-1 s=CLOSED pgs=2452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:06.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.676+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 >> 172.21.15.165:0/1287287718 conn(0x7f5e3406abe0 msgr2=0x7f5e340c4450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:06.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.677+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 shutdown_connections 2024-04-25T10:19:06.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:06.677+0000 7f5e3a800640 1 -- 172.21.15.165:0/1287287718 wait complete. 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:19:06.699 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:19:06.700 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:06.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:06.700 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:19:06.747 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:19:06.747 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:06.748 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41619sec preferred_lft 41619sec 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:19:06.749 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.750 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:19:06.750 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:19:06.750 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:19:06.750 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:19:06.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:06.751 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:19:06.824 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:19:06.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:06.828 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:19:06.874 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:19:06.875 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:19:06.875 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:19:06.875 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:19:06.875 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:19:06.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:06.875 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:19:06.875 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:19:06.875 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:19:06.875 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:19:06.875 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:19:06.876 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:19:07.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.067 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:19:07.114 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:19:07.115 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:19:07.115 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:07.115 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.115 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:19:07.240 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:19:07.240 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.295 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:19:07.295 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:19:07.295 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.344 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:19:07.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.344 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:07.474 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:07.474 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:19:07.474 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:19:07.495 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:07.495 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:19:07.495 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:19:07.495 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:19:07.495 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:19:07.520 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:19:07.522 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:19:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:19:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:19:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:19:07.522 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:19:07.525 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:19:07.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.525 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:07.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.598 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:19:07.657 INFO:teuthology.orchestra.run.smithi165.stdout:client10926 2024-04-25T10:19:07.658 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:07.658 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:07.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.707 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2024-04-25T10:19:07.814 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:07.814 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:07.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:07.868 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:19:08.038 INFO:teuthology.orchestra.run.smithi165.stderr:6+0 records in 2024-04-25T10:19:08.038 INFO:teuthology.orchestra.run.smithi165.stderr:6+0 records out 2024-04-25T10:19:08.039 INFO:teuthology.orchestra.run.smithi165.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.117193 s, 53.7 MB/s 2024-04-25T10:19:08.041 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:19:08.046 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.174 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.175 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.176 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on skipped message = False 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-25T10:19:08.177 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-25T10:19:08.178 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-25T10:19:08.179 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 20 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 20 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 10 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.180 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-25T10:19:08.181 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:19:08.182 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: mon op complaint time = 120 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: client mount timeout = 600 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: debug client = 20 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = "" 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-04-25T10:19:08.183 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.d] 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout:[mds] 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds = 20 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds balancer = 20 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: mds cache max size = 1000 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug frag = True 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug scatterstat = True 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: mds log max segments = 1 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: mds op complaint time = 180 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: mds verify scatter = False 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:19:08.184 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.185 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.191 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.192 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.193 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-25T10:19:08.194 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.195 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-25T10:19:08.196 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-25T10:19:08.197 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.198 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:19:08.199 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-25T10:19:08.200 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: mds cache max size = 1000 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = True 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: mds log max segments = 1 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = False 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.201 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.225 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:19:08.230 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:19:08.361 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-25T10:19:08.361 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-25T10:19:08.361 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-25T10:19:08.362 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-25T10:19:08.363 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:19:08.364 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.365 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:19:08.366 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on skipped message = False 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.367 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-25T10:19:08.368 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-04-25T10:19:08.368 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:19:08.368 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.368 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-25T10:19:08.368 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-04-25T10:19:08.369 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.370 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-25T10:19:08.371 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 20 2024-04-25T10:19:08.372 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 20 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 10 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:19:08.373 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.374 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:19:08.375 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: mon op complaint time = 120 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:19:08.376 INFO:teuthology.orchestra.run.smithi165.stdout: client mount timeout = 600 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: debug client = 20 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = "" 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.d] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout:[mds] 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds = 20 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds balancer = 20 2024-04-25T10:19:08.377 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: mds cache max size = 1000 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug frag = True 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug scatterstat = False 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: mds log max segments = 1 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: mds op complaint time = 180 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: mds verify scatter = False 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.378 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-25T10:19:08.379 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:19:08.380 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-25T10:19:08.381 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.382 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-25T10:19:08.383 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.384 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-25T10:19:08.385 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:19:08.386 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-25T10:19:08.387 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: mds cache max size = 1000 2024-04-25T10:19:08.388 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = False 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: mds log max segments = 1 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = False 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:19:08.389 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> import json 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> import sys 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> except OSError as e: 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> sys.exit(e.errno) 2024-04-25T10:19:08.398 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:19:08.399 DEBUG:teuthology.orchestra.run.smithi165:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:19:08.399 DEBUG:teuthology.orchestra.run.smithi165:> print(json.dumps( 2024-04-25T10:19:08.399 DEBUG:teuthology.orchestra.run.smithi165:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:19:08.399 DEBUG:teuthology.orchestra.run.smithi165:> indent=2)) 2024-04-25T10:19:08.399 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:19:08.535 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:08.535 INFO:teuthology.orchestra.run.smithi165.stdout: "st_mode": 33188, 2024-04-25T10:19:08.535 INFO:teuthology.orchestra.run.smithi165.stdout: "st_ino": 1099511627777, 2024-04-25T10:19:08.535 INFO:teuthology.orchestra.run.smithi165.stdout: "st_dev": 40, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_nlink": 1, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_uid": 1000, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_gid": 1316, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_size": 6291456, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_atime": 1714040347.9174836, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_mtime": 1714040347.9644828, 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout: "st_ctime": 1714040347.9644828 2024-04-25T10:19:08.536 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:08.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:19:08.549 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:08.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.792+0000 7fc27ac00640 1 Processor -- start 2024-04-25T10:19:08.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.792+0000 7fc27ac00640 1 -- start start 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27ac00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc27406e870 0x7fc2740cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27ac00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc2740cc840 0x7fc27415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27ac00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc274144240 con 0x7fc274143e60 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27ac00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27415e220 con 0x7fc2740cc840 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27ac00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc27415e500 con 0x7fc27406e870 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc2740cc840 0x7fc27415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc2740cc840 0x7fc27415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49280/0 (socket says 172.21.15.165:49280) 2024-04-25T10:19:08.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 -- 172.21.15.165:0/408016633 learned_addr learned my addr 172.21.15.165:0/408016633 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:08.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 -- 172.21.15.165:0/408016633 >> v2:172.21.15.172:3300/0 conn(0x7fc27406e870 msgr2=0x7fc2740cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:08.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 --2- 172.21.15.165:0/408016633 >> v2:172.21.15.172:3300/0 conn(0x7fc27406e870 0x7fc2740cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 -- 172.21.15.165:0/408016633 >> v1:172.21.15.172:6789/0 conn(0x7fc274143e60 legacy=0x7fc27406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:08.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.794+0000 7fc27a200640 1 -- 172.21.15.165:0/408016633 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc27415e7e0 con 0x7fc2740cc840 2024-04-25T10:19:08.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.795+0000 7fc27a200640 1 --2- 172.21.15.165:0/408016633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2740cc840 0x7fc27415ab40 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fc27406b970 tx=0x7fc26c02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb08e30afcc40567 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:08.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.796+0000 7fc273e00640 1 -- 172.21.15.165:0/408016633 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c037070 con 0x7fc2740cc840 2024-04-25T10:19:08.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.796+0000 7fc273e00640 1 -- 172.21.15.165:0/408016633 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc26c032440 con 0x7fc2740cc840 2024-04-25T10:19:08.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.796+0000 7fc273e00640 1 -- 172.21.15.165:0/408016633 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26c032810 con 0x7fc2740cc840 2024-04-25T10:19:08.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.796+0000 7fc27ac00640 1 -- 172.21.15.165:0/408016633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2740cc840 msgr2=0x7fc27415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.796+0000 7fc27ac00640 1 --2- 172.21.15.165:0/408016633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2740cc840 0x7fc27415ab40 secure :-1 s=READY pgs=2454 cs=0 l=1 rev1=1 crypto rx=0x7fc27406b970 tx=0x7fc26c02a230 comp rx=0 tx=0).stop 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.797+0000 7fc27ac00640 1 -- 172.21.15.165:0/408016633 shutdown_connections 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.797+0000 7fc27ac00640 1 --2- 172.21.15.165:0/408016633 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc2740cc840 0x7fc27415ab40 unknown :-1 s=CLOSED pgs=2454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.797+0000 7fc27ac00640 1 --2- 172.21.15.165:0/408016633 >> v2:172.21.15.172:3300/0 conn(0x7fc27406e870 0x7fc2740cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.797+0000 7fc27ac00640 1 -- 172.21.15.165:0/408016633 >> 172.21.15.165:0/408016633 conn(0x7fc27406abe0 msgr2=0x7fc27406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.797+0000 7fc27ac00640 1 -- 172.21.15.165:0/408016633 shutdown_connections 2024-04-25T10:19:08.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.797+0000 7fc27ac00640 1 -- 172.21.15.165:0/408016633 wait complete. 2024-04-25T10:19:08.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.799+0000 7fc27ac00640 1 Processor -- start 2024-04-25T10:19:08.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.799+0000 7fc27ac00640 1 -- start start 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc27ac00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 0x7fc274159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc27ac00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2740cc840 0x7fc2741410b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc27ac00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc274143e60 0x7fc2740ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc27ac00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b2eb0 con 0x7fc27406e870 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc27ac00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740b2d30 con 0x7fc2740cc840 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc27ac00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2740ccca0 con 0x7fc274143e60 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc279800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 0x7fc274159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc279800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 0x7fc274159e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49284/0 (socket says 172.21.15.165:49284) 2024-04-25T10:19:08.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.800+0000 7fc279800640 1 -- 172.21.15.165:0/1895893929 learned_addr learned my addr 172.21.15.165:0/1895893929 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:08.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc278e00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2740cc840 0x7fc2741410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc279800640 1 -- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc274143e60 msgr2=0x7fc2740ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc279800640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc274143e60 0x7fc2740ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc279800640 1 -- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2740cc840 msgr2=0x7fc2741410b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc279800640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2740cc840 0x7fc2741410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc279800640 1 -- 172.21.15.165:0/1895893929 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2741340a0 con 0x7fc27406e870 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc279800640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 0x7fc274159e20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b410 tx=0x7fc26400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264015020 con 0x7fc27406e870 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc264011830 con 0x7fc27406e870 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc264011c20 con 0x7fc27406e870 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.801+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27413fd70 con 0x7fc27406e870 2024-04-25T10:19:08.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.802+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2741402f0 con 0x7fc27406e870 2024-04-25T10:19:08.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.803+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc274073990 con 0x7fc27406e870 2024-04-25T10:19:08.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.804+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc264018020 con 0x7fc27406e870 2024-04-25T10:19:08.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.805+0000 7fc271600640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc25806b4d0 0x7fc25806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:08.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.806+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc26408a350 con 0x7fc27406e870 2024-04-25T10:19:08.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.806+0000 7fc278e00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc25806b4d0 0x7fc25806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:08.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.807+0000 7fc278e00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc25806b4d0 0x7fc25806d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc2741f8250 tx=0x7fc2680072f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:08.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:08.809+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc26401f020 con 0x7fc27406e870 2024-04-25T10:19:09.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.043+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2740afea0 con 0x7fc27406e870 2024-04-25T10:19:09.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.044+0000 7fc271600640 1 -- 172.21.15.165:0/1895893929 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fc2741340a0 con 0x7fc27406e870 2024-04-25T10:19:09.047 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:09.047 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:09.047 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc25806b4d0 msgr2=0x7fc25806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc25806b4d0 0x7fc25806d990 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc2741f8250 tx=0x7fc2680072f0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 msgr2=0x7fc274159e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 0x7fc274159e20 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7fc26400b410 tx=0x7fc26400b8d0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 shutdown_connections 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc25806b4d0 0x7fc25806d990 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc274143e60 0x7fc2740ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc2740cc840 0x7fc2741410b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 --2- 172.21.15.165:0/1895893929 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc27406e870 0x7fc274159e20 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 >> 172.21.15.165:0/1895893929 conn(0x7fc27406abe0 msgr2=0x7fc2740b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 shutdown_connections 2024-04-25T10:19:09.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.047+0000 7fc27ac00640 1 -- 172.21.15.165:0/1895893929 wait complete. 2024-04-25T10:19:09.069 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:19:09.302 INFO:teuthology.orchestra.run.smithi165.stdout:{"message":"","return_code":0} 2024-04-25T10:19:09.302 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:19:09.303 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent '' 2024-04-25T10:19:09.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.403+0000 7f0d293cbf40 1 Processor -- start 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.403+0000 7f0d293cbf40 1 -- start start 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d293cbf40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5d85050 0x556ae5db3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d293cbf40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 0x556ae5e56b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d293cbf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5e42d50 0x556ae5d833a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d293cbf40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5dce680 con 0x556ae5daf340 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d293cbf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5d2cdd0 con 0x556ae5e42d50 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d293cbf40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ae5d859a0 con 0x556ae5d85050 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 0x556ae5e56b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 0x556ae5e56b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49308/0 (socket says 172.21.15.165:49308) 2024-04-25T10:19:09.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 -- 172.21.15.165:0/1882705992 learned_addr learned my addr 172.21.15.165:0/1882705992 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d26600640 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5d85050 0x556ae5db3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 -- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5d85050 msgr2=0x556ae5db3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5d85050 0x556ae5db3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 -- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5e42d50 msgr2=0x556ae5d833a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5e42d50 0x556ae5d833a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.404+0000 7f0d25c00640 1 -- 172.21.15.165:0/1882705992 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae5e42270 con 0x556ae5daf340 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.405+0000 7f0d25c00640 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 0x556ae5e56b90 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009600 tx=0x7f0d14009ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3110290f73ebc60 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.405+0000 7f0d1fe00640 1 -- 172.21.15.165:0/1882705992 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14017070 con 0x556ae5daf340 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.405+0000 7f0d1fe00640 1 -- 172.21.15.165:0/1882705992 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d140066b0 con 0x556ae5daf340 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.405+0000 7f0d1fe00640 1 -- 172.21.15.165:0/1882705992 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d14006a60 con 0x556ae5daf340 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 -- 172.21.15.165:0/1882705992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 msgr2=0x556ae5e56b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 0x556ae5e56b90 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f0d14009600 tx=0x7f0d14009ac0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 -- 172.21.15.165:0/1882705992 shutdown_connections 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5e42d50 0x556ae5d833a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5daf340 0x556ae5e56b90 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/1882705992 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5d85050 0x556ae5db3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 -- 172.21.15.165:0/1882705992 >> 172.21.15.165:0/1882705992 conn(0x556ae5db1c60 msgr2=0x556ae5d82c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 -- 172.21.15.165:0/1882705992 shutdown_connections 2024-04-25T10:19:09.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 -- 172.21.15.165:0/1882705992 wait complete. 2024-04-25T10:19:09.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.406+0000 7f0d293cbf40 1 Processor -- start 2024-04-25T10:19:09.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 -- start start 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5d85050 0x556ae5e55500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5daf340 0x556ae5e3b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5e42d50 0x556ae5e3dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5db0e80 con 0x556ae5d85050 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ae5e42020 con 0x556ae5daf340 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.407+0000 7f0d293cbf40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x556ae5e570d0 con 0x556ae5e42d50 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5d85050 0x556ae5e55500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5d85050 0x556ae5e55500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49318/0 (socket says 172.21.15.165:49318) 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 -- 172.21.15.165:0/798909247 learned_addr learned my addr 172.21.15.165:0/798909247 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d27000640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5e42d50 0x556ae5e3dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d25c00640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5daf340 0x556ae5e3b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 -- 172.21.15.165:0/798909247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5e42d50 msgr2=0x556ae5e3dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x556ae5e42d50 0x556ae5e3dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 -- 172.21.15.165:0/798909247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5daf340 msgr2=0x556ae5e3b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x556ae5daf340 0x556ae5e3b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.408+0000 7f0d26600640 1 -- 172.21.15.165:0/798909247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ae5e419f0 con 0x556ae5d85050 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.409+0000 7f0d26600640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5d85050 0x556ae5e55500 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800cef0 tx=0x7f0d1800a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.409+0000 7f0d1ea00640 1 -- 172.21.15.165:0/798909247 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d18008040 con 0x556ae5d85050 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.409+0000 7f0d1ea00640 1 -- 172.21.15.165:0/798909247 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d18014020 con 0x556ae5d85050 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.409+0000 7f0d1ea00640 1 -- 172.21.15.165:0/798909247 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1800f400 con 0x556ae5d85050 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.409+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x556ae5e3a980 con 0x556ae5d85050 2024-04-25T10:19:09.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.409+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ae5e3af40 con 0x556ae5d85050 2024-04-25T10:19:09.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.411+0000 7f0d1ea00640 1 -- 172.21.15.165:0/798909247 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0d1800f5a0 con 0x556ae5d85050 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.412+0000 7f0d1ea00640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d08069ee0 0x7f0d0806c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.412+0000 7f0d1ea00640 1 -- 172.21.15.165:0/798909247 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0d18088e50 con 0x556ae5d85050 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.412+0000 7f0d25c00640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d08069ee0 0x7f0d0806c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.412+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x556ae5e55ad0 0x556ae5e3cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.412+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 --> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [setxattr parent in=6b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e171) v9 -- 0x556ae5e3be90 con 0x556ae5e55ad0 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.413+0000 7f0d27000640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x556ae5e55ad0 0x556ae5e3cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.413+0000 7f0d25c00640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d08069ee0 0x7f0d0806c3a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x556ae5d858a0 tx=0x7f0d1400f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.413+0000 7f0d27000640 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x556ae5e55ad0 0x556ae5e3cad0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d27000640 1 -- 172.21.15.165:0/798909247 <== osd.2 v2:172.21.15.165:6824/3063902068 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr] v171'4 uv4 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f0d20050520 con 0x556ae5e55ad0 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x556ae5e55ad0 msgr2=0x556ae5e3cad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x556ae5e55ad0 0x556ae5e3cad0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d08069ee0 msgr2=0x7f0d0806c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0d08069ee0 0x7f0d0806c3a0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x556ae5d858a0 tx=0x7f0d1400f000 comp rx=0 tx=0).stop 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5d85050 msgr2=0x556ae5e55500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 --2- 172.21.15.165:0/798909247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x556ae5d85050 0x556ae5e55500 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f0d1800cef0 tx=0x7f0d1800a5d0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d27000640 1 -- 172.21.15.165:0/798909247 reap_dead start 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 shutdown_connections 2024-04-25T10:19:09.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 >> 172.21.15.165:0/798909247 conn(0x556ae5db1c60 msgr2=0x556ae5db5540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 shutdown_connections 2024-04-25T10:19:09.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.416+0000 7f0d293cbf40 1 -- 172.21.15.165:0/798909247 wait complete. 2024-04-25T10:19:09.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:09.427 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub start / recursive,repair 2024-04-25T10:19:09.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.621+0000 7fd065600640 1 Processor -- start 2024-04-25T10:19:09.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.621+0000 7fd065600640 1 -- start start 2024-04-25T10:19:09.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.623+0000 7fd065600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd0600cd620 0x7fd0600cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.623+0000 7fd065600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd060070680 con 0x7fd06015f3b0 2024-04-25T10:19:09.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.623+0000 7fd065600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd060070940 con 0x7fd06015b6c0 2024-04-25T10:19:09.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.623+0000 7fd065600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0600ce250 con 0x7fd0600cd620 2024-04-25T10:19:09.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.623+0000 7fd057400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd06015f3b0 0x7fd0600ccf10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:33434/0 (socket says 172.21.15.165:33434) 2024-04-25T10:19:09.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.623+0000 7fd057400640 1 -- 172.21.15.165:0/1378968339 learned_addr learned my addr 172.21.15.165:0/1378968339 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd064c00640 1 --2- 172.21.15.165:0/1378968339 >> v2:172.21.15.172:3300/0 conn(0x7fd0600cd620 0x7fd0600cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 818336206 0 0) 0x7fd060070680 con 0x7fd06015f3b0 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd044003520 con 0x7fd06015f3b0 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3044250502 0 0) 0x7fd044003520 con 0x7fd06015f3b0 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 >> v2:172.21.15.172:3300/0 conn(0x7fd0600cd620 msgr2=0x7fd0600cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 --2- 172.21.15.165:0/1378968339 >> v2:172.21.15.172:3300/0 conn(0x7fd0600cd620 0x7fd0600cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 >> v1:172.21.15.172:6789/0 conn(0x7fd06015b6c0 legacy=0x7fd0600cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06006ac80 con 0x7fd06015f3b0 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fd048001670 con 0x7fd06015f3b0 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 >> v1:172.21.15.165:6789/0 conn(0x7fd06015f3b0 legacy=0x7fd0600ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044003a00 0x7fd044003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd044004800 0x7fd044004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0440051c0 0x7fd04400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0600ce250 con 0x7fd044003a00 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd04400c0c0 con 0x7fd0440051c0 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.624+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd04400c2c0 con 0x7fd044004800 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044003a00 0x7fd044003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd057400640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd044004800 0x7fd044004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd05fe00640 1 -- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd044004800 msgr2=0x7fd044004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd044004800 0x7fd044004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd05fe00640 1 -- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0440051c0 msgr2=0x7fd04400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd0440051c0 0x7fd04400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd05fe00640 1 -- 172.21.15.165:0/1378968339 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd044003520 con 0x7fd044003a00 2024-04-25T10:19:09.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.625+0000 7fd064c00640 1 -- 172.21.15.165:0/1378968339 reap_dead start 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.626+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044003a00 0x7fd044003e20 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fd050001750 tx=0x7fd044005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da265eb831933b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.626+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050001370 con 0x7fd044003a00 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.626+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0500052d0 con 0x7fd044003a00 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.626+0000 7fd05f400640 1 -- 172.21.15.165:0/1378968339 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd05002abe0 con 0x7fd044003a00 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 -- 172.21.15.165:0/1378968339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044003a00 msgr2=0x7fd044003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044003a00 0x7fd044003e20 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fd050001750 tx=0x7fd044005830 comp rx=0 tx=0).stop 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 -- 172.21.15.165:0/1378968339 shutdown_connections 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 --2- 172.21.15.165:0/1378968339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd044003a00 0x7fd044003e20 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 -- 172.21.15.165:0/1378968339 >> 172.21.15.165:0/1378968339 conn(0x7fd06006e3b0 msgr2=0x7fd06006e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 -- 172.21.15.165:0/1378968339 shutdown_connections 2024-04-25T10:19:09.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.627+0000 7fd065600640 1 -- 172.21.15.165:0/1378968339 wait complete. 2024-04-25T10:19:09.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.629+0000 7fd065600640 1 Processor -- start 2024-04-25T10:19:09.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.629+0000 7fd065600640 1 -- start start 2024-04-25T10:19:09.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd065600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044003a00 0x7fd06015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd065600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 0x7fd0600b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd065600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd060157d10 0x7fd060158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd065600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd060070c20 con 0x7fd0600b1070 2024-04-25T10:19:09.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd065600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06006b480 con 0x7fd044003a00 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd065600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06015baa0 con 0x7fd060157d10 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd057400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 0x7fd0600b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd057400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 0x7fd0600b14f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49340/0 (socket says 172.21.15.165:49340) 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.630+0000 7fd057400640 1 -- 172.21.15.165:0/2333434656 learned_addr learned my addr 172.21.15.165:0/2333434656 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd064c00640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd060157d10 0x7fd060158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd05fe00640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044003a00 0x7fd06015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd057400640 1 -- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd060157d10 msgr2=0x7fd060158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd057400640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd060157d10 0x7fd060158190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd057400640 1 -- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044003a00 msgr2=0x7fd06015e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd057400640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044003a00 0x7fd06015e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd057400640 1 -- 172.21.15.165:0/2333434656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060157830 con 0x7fd0600b1070 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd05fe00640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044003a00 0x7fd06015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd057400640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 0x7fd0600b14f0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fd04800d770 tx=0x7fd04800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd05cc00640 1 -- 172.21.15.165:0/2333434656 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048019070 con 0x7fd0600b1070 2024-04-25T10:19:09.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd05cc00640 1 -- 172.21.15.165:0/2333434656 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0480074e0 con 0x7fd0600b1070 2024-04-25T10:19:09.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd05cc00640 1 -- 172.21.15.165:0/2333434656 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd048004a20 con 0x7fd0600b1070 2024-04-25T10:19:09.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.631+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0600b1d30 con 0x7fd0600b1070 2024-04-25T10:19:09.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.632+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0600b22f0 con 0x7fd0600b1070 2024-04-25T10:19:09.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.634+0000 7fd05cc00640 1 -- 172.21.15.165:0/2333434656 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd048004bc0 con 0x7fd0600b1070 2024-04-25T10:19:09.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.635+0000 7fd05cc00640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd03c06b500 0x7fd03c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.635+0000 7fd05cc00640 1 -- 172.21.15.165:0/2333434656 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd048089c30 con 0x7fd0600b1070 2024-04-25T10:19:09.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.635+0000 7fd05fe00640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd03c06b500 0x7fd03c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.635+0000 7fd069a19740 1 Processor -- start 2024-04-25T10:19:09.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.636+0000 7fd069a19740 1 -- start start 2024-04-25T10:19:09.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.636+0000 7fd05fe00640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd03c06b500 0x7fd03c06d9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd050032000 tx=0x7fd05002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.636+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7364370 0x55c9b7364750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.636+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b7382160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.636+0000 7fd069a19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 0x55c9b73802d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd069a19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b729c980 con 0x55c9b733a050 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b7245de0 con 0x55c9b7381c10 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9b7298a20 con 0x55c9b7364370 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 0x55c9b73802d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 0x55c9b73802d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49350/0 (socket says 172.21.15.165:49350) 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 -- 172.21.15.165:0/1665762063 learned_addr learned my addr 172.21.15.165:0/1665762063 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7364370 0x55c9b7364750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd057400640 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b7382160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 -- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7364370 msgr2=0x55c9b7364750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7364370 0x55c9b7364750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 -- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 msgr2=0x55c9b7382160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b7382160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.637+0000 7fd064c00640 1 -- 172.21.15.165:0/1665762063 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b7263e30 con 0x55c9b733a050 2024-04-25T10:19:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.638+0000 7fd064c00640 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 0x55c9b73802d0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fd06006ff40 tx=0x7fd058009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d01da5855485da server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.638+0000 7fd04fe00640 1 -- 172.21.15.165:0/1665762063 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580153f0 con 0x55c9b733a050 2024-04-25T10:19:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.638+0000 7fd04fe00640 1 -- 172.21.15.165:0/1665762063 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05801a070 con 0x55c9b733a050 2024-04-25T10:19:09.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.638+0000 7fd04fe00640 1 -- 172.21.15.165:0/1665762063 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0580157b0 con 0x55c9b733a050 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 -- 172.21.15.165:0/1665762063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 msgr2=0x55c9b73802d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 0x55c9b73802d0 secure :-1 s=READY pgs=2463 cs=0 l=1 rev1=1 crypto rx=0x7fd06006ff40 tx=0x7fd058009750 comp rx=0 tx=0).stop 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 -- 172.21.15.165:0/1665762063 shutdown_connections 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b733a050 0x55c9b73802d0 unknown :-1 s=CLOSED pgs=2463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b7382160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 --2- 172.21.15.165:0/1665762063 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7364370 0x55c9b7364750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 -- 172.21.15.165:0/1665762063 >> 172.21.15.165:0/1665762063 conn(0x55c9b7336980 msgr2=0x55c9b737f180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 -- 172.21.15.165:0/1665762063 shutdown_connections 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.639+0000 7fd069a19740 1 -- 172.21.15.165:0/1665762063 wait complete. 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.640+0000 7fd069a19740 1 -- start start 2024-04-25T10:19:09.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.640+0000 7fd069a19740 1 Processor -- start 2024-04-25T10:19:09.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b7376230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b738f680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b733adf0 con 0x55c9b7364370 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b723b3c0 con 0x55c9b7381c10 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9b73348b0 con 0x55c9b73782f0 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd069a19740 10 client.0 _ll_register_callbacks cb 0x55c9b6fb0c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b7376230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b7376230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49364/0 (socket says 172.21.15.165:49364) 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.641+0000 7fd05fe00640 1 -- 172.21.15.165:0/3821976099 learned_addr learned my addr 172.21.15.165:0/3821976099 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd064c00640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd057400640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b738f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd05fe00640 1 -- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 msgr2=0x55c9b738fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd05fe00640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd05fe00640 1 -- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 msgr2=0x55c9b738f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd05fe00640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b738f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd05fe00640 1 -- 172.21.15.165:0/3821976099 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9b70797e0 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd05fe00640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b7376230 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fd05002be00 tx=0x7fd05002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8024ad3fcf77aee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd04cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd04cc00640 1 -- 172.21.15.165:0/3821976099 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0500074d0 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd04cc00640 1 -- 172.21.15.165:0/3821976099 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0500035f0 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd05801a070 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd04cc00640 1 -- 172.21.15.165:0/3821976099 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd050007670 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd069a19740 10 client.10956 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.642+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55c9b70d9f70 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd04cc00640 1 -- 172.21.15.165:0/3821976099 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05801a070 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd04cc00640 1 -- 172.21.15.165:0/3821976099 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd050058080 con 0x55c9b7364370 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd069a19740 10 client.10956 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd069a19740 10 client.10956 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-25T10:19:09.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd069a19740 20 client.10956 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd069a19740 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b7366a60 0x55c9b7366ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd069a19740 4 client.10956 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9b72a9ec0 con 0x55c9b7366a60 2024-04-25T10:19:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.643+0000 7fd057400640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b7366a60 0x55c9b7366ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.644+0000 7fd057400640 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b7366a60 0x55c9b7366ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=473ae45739dfd5cf server_cookie=764b9e2be3cdfe4e in_seq=0 out_seq=0 2024-04-25T10:19:09.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.644+0000 7fd04cc00640 10 client.10956 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.644+0000 7fd04cc00640 1 -- 172.21.15.165:0/3821976099 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55c9b72a9ec0 con 0x55c9b7366a60 2024-04-25T10:19:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.644+0000 7fd04cc00640 10 client.10956 handle_command_reply: tid=0 2024-04-25T10:19:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.644+0000 7fd069a19740 1 client.10956 shutdown 2024-04-25T10:19:09.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 msgr2=0x55c9b7376230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b7376230 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fd05002be00 tx=0x7fd05002c070 comp rx=0 tx=0).stop 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 shutdown_connections 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b7366a60 0x55c9b7366ee0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7381c10 0x55c9b738f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 --2- 172.21.15.165:0/3821976099 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b7376230 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 >> 172.21.15.165:0/3821976099 conn(0x55c9b733a050 msgr2=0x55c9b7382f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd04cc00640 0 client.10956 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 shutdown_connections 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 1 -- 172.21.15.165:0/3821976099 wait complete. 2024-04-25T10:19:09.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.645+0000 7fd069a19740 20 client.10956 trim_cache size 0 max 16384 2024-04-25T10:19:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.662+0000 7fd069a19740 1 Processor -- start 2024-04-25T10:19:09.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.662+0000 7fd069a19740 1 -- start start 2024-04-25T10:19:09.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.662+0000 7fd069a19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b73e1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.662+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b73782f0 0x55c9b733a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7381c10 0x55c9b733a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd069a19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd060200a80 con 0x55c9b7364370 2024-04-25T10:19:09.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601ffbc0 con 0x55c9b73782f0 2024-04-25T10:19:09.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601491e0 con 0x55c9b7381c10 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b73e1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd05fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b73e1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49378/0 (socket says 172.21.15.165:49378) 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd05fe00640 1 -- 172.21.15.165:0/1324838623 learned_addr learned my addr 172.21.15.165:0/1324838623 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd064c00640 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7381c10 0x55c9b733a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd057400640 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b73782f0 0x55c9b733a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.663+0000 7fd05fe00640 1 -- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7381c10 msgr2=0x55c9b733a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7381c10 0x55c9b733a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd05fe00640 1 -- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b73782f0 msgr2=0x55c9b733a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b73782f0 0x55c9b733a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd05fe00640 1 -- 172.21.15.165:0/1324838623 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9b6f71170 con 0x55c9b7364370 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd057400640 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b73782f0 0x55c9b733a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd05fe00640 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b73e1650 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fd05002bfd0 tx=0x7fd05003a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e6879dcc090b852 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd04f400640 1 -- 172.21.15.165:0/1324838623 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050006ba0 con 0x55c9b7364370 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd04f400640 1 -- 172.21.15.165:0/1324838623 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05004d070 con 0x55c9b7364370 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.664+0000 7fd04f400640 1 -- 172.21.15.165:0/1324838623 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd050003980 con 0x55c9b7364370 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 -- 172.21.15.165:0/1324838623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 msgr2=0x55c9b73e1650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b73e1650 secure :-1 s=READY pgs=2465 cs=0 l=1 rev1=1 crypto rx=0x7fd05002bfd0 tx=0x7fd05003a000 comp rx=0 tx=0).stop 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 -- 172.21.15.165:0/1324838623 shutdown_connections 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b7381c10 0x55c9b733a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b73782f0 0x55c9b733a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 --2- 172.21.15.165:0/1324838623 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7364370 0x55c9b73e1650 unknown :-1 s=CLOSED pgs=2465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 -- 172.21.15.165:0/1324838623 >> 172.21.15.165:0/1324838623 conn(0x55c9b733a050 msgr2=0x55c9b73787d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 -- 172.21.15.165:0/1324838623 shutdown_connections 2024-04-25T10:19:09.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.665+0000 7fd069a19740 1 -- 172.21.15.165:0/1324838623 wait complete. 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.666+0000 7fd069a19740 1 -- start start 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.666+0000 7fd069a19740 1 Processor -- start 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7364370 0x55c9b7377890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 0x55c9b7375440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9b732fe40 con 0x55c9b7381c10 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0601ff640 con 0x55c9b7364370 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c9b737a930 con 0x55c9b73782f0 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd069a19740 10 client.0 _ll_register_callbacks cb 0x55c9b7334d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 0x55c9b7375440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 0x55c9b7375440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49388/0 (socket says 172.21.15.165:49388) 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 -- 172.21.15.165:0/2789800095 learned_addr learned my addr 172.21.15.165:0/2789800095 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 -- 172.21.15.165:0/2789800095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 msgr2=0x55c9b738bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 -- 172.21.15.165:0/2789800095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7364370 msgr2=0x55c9b7377890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7364370 0x55c9b7377890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.667+0000 7fd064c00640 1 -- 172.21.15.165:0/2789800095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9b73e4d90 con 0x55c9b7381c10 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd064c00640 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 0x55c9b7375440 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fd058016000 tx=0x7fd058004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84153119067b7ee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd04cc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd04cc00640 1 -- 172.21.15.165:0/2789800095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058003e40 con 0x55c9b7381c10 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd04cc00640 1 -- 172.21.15.165:0/2789800095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd058004250 con 0x55c9b7381c10 2024-04-25T10:19:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd05004d070 con 0x55c9b7381c10 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd04cc00640 1 -- 172.21.15.165:0/2789800095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd05802f050 con 0x55c9b7381c10 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd069a19740 10 client.10962 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.668+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55c9b738c680 con 0x55c9b7381c10 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd04cc00640 1 -- 172.21.15.165:0/2789800095 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05004d070 con 0x55c9b7381c10 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd04cc00640 1 -- 172.21.15.165:0/2789800095 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fd058032020 con 0x55c9b7381c10 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd069a19740 10 client.10962 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd069a19740 10 client.10962 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd069a19740 20 client.10962 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd069a19740 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b733a9d0 0x55c9b7337e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd069a19740 4 client.10962 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x55c9b72a4210 con 0x55c9b733a9d0 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd05fe00640 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b733a9d0 0x55c9b7337e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd04cc00640 10 client.10962 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.669+0000 7fd05fe00640 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b733a9d0 0x55c9b7337e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2ff64204e4cbf2 server_cookie=f10049e0cfe74762 in_seq=0 out_seq=0 2024-04-25T10:19:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.670+0000 7fd04cc00640 1 -- 172.21.15.165:0/2789800095 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c9b72a4210 con 0x55c9b733a9d0 2024-04-25T10:19:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.670+0000 7fd04cc00640 10 client.10962 handle_command_reply: tid=0 2024-04-25T10:19:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.670+0000 7fd069a19740 1 client.10962 shutdown 2024-04-25T10:19:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.670+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 msgr2=0x55c9b7375440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.670+0000 7fd069a19740 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 0x55c9b7375440 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7fd058016000 tx=0x7fd058004ab0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.676 INFO:tasks.ceph.mds.a.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fc2fcc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:19:07.958482+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={10926=0-12582912@1} caps={10926=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=10926 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55648960b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 shutdown_connections 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55c9b733a9d0 0x55c9b7337e20 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55c9b7381c10 0x55c9b7375440 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55c9b73782f0 0x55c9b738bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 --2- 172.21.15.165:0/2789800095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55c9b7364370 0x55c9b7377890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 >> 172.21.15.165:0/2789800095 conn(0x55c9b733a050 msgr2=0x55c9b73a1050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd04cc00640 0 client.10962 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 shutdown_connections 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 1 -- 172.21.15.165:0/2789800095 wait complete. 2024-04-25T10:19:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.671+0000 7fd069a19740 20 client.10962 trim_cache size 0 max 16384 2024-04-25T10:19:09.678 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:09.678 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:19:09.678 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "117de01f-6402-4bd3-a9c7-1bb28fce9c60", 2024-04-25T10:19:09.678 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:19:09.678 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd03c06b500 msgr2=0x7fd03c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd03c06b500 0x7fd03c06d9c0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7fd050032000 tx=0x7fd05002e000 comp rx=0 tx=0).stop 2024-04-25T10:19:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 msgr2=0x7fd0600b14f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 0x7fd0600b14f0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fd04800d770 tx=0x7fd04800dc30 comp rx=0 tx=0).stop 2024-04-25T10:19:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 shutdown_connections 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd03c06b500 0x7fd03c06d9c0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd060157d10 0x7fd060158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0600b1070 0x7fd0600b14f0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 --2- 172.21.15.165:0/2333434656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd044003a00 0x7fd06015e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 >> 172.21.15.165:0/2333434656 conn(0x7fd06006e3b0 msgr2=0x7fd0600b3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 shutdown_connections 2024-04-25T10:19:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.672+0000 7fd065600640 1 -- 172.21.15.165:0/2333434656 wait complete. 2024-04-25T10:19:09.693 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:09.693 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:19:09.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.883+0000 7f6733e00640 1 Processor -- start 2024-04-25T10:19:09.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.884+0000 7f6733e00640 1 -- start start 2024-04-25T10:19:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.885+0000 7f6733e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f673415b3c0 0x7f67340b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.885+0000 7f6733e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f673415b7a0 0x7f67340cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.885+0000 7f6733e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67340cdae0 0x7f67340cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.885+0000 7f6733e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67340cd650 con 0x7f673415b3c0 2024-04-25T10:19:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.885+0000 7f6733e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67340b0b80 con 0x7f673415b7a0 2024-04-25T10:19:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.885+0000 7f6733e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67340cc800 con 0x7f67340cdae0 2024-04-25T10:19:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f673415b3c0 0x7f67340b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f673415b3c0 0x7f67340b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49404/0 (socket says 172.21.15.165:49404) 2024-04-25T10:19:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 -- 172.21.15.165:0/1753800455 learned_addr learned my addr 172.21.15.165:0/1753800455 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6733400640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67340cdae0 0x7f67340cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f672be00640 1 --2- 172.21.15.165:0/1753800455 >> v2:172.21.15.172:3300/0 conn(0x7f673415b7a0 0x7f67340cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 -- 172.21.15.165:0/1753800455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67340cdae0 msgr2=0x7f67340cdf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67340cdae0 0x7f67340cdf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 -- 172.21.15.165:0/1753800455 >> v2:172.21.15.172:3300/0 conn(0x7f673415b7a0 msgr2=0x7f67340cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 --2- 172.21.15.165:0/1753800455 >> v2:172.21.15.172:3300/0 conn(0x7f673415b7a0 0x7f67340cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6732a00640 1 -- 172.21.15.165:0/1753800455 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67340ccae0 con 0x7f673415b3c0 2024-04-25T10:19:09.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.886+0000 7f6733400640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67340cdae0 0x7f67340cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.888+0000 7f6732a00640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340b0f60 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f673415f180 tx=0x7f672402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69621b1a2a3bd7db server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.888+0000 7f6732000640 1 -- 172.21.15.165:0/1753800455 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724037070 con 0x7f673415b3c0 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.888+0000 7f6732000640 1 -- 172.21.15.165:0/1753800455 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6724032440 con 0x7f673415b3c0 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.888+0000 7f6732000640 1 -- 172.21.15.165:0/1753800455 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724032810 con 0x7f673415b3c0 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 -- 172.21.15.165:0/1753800455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 msgr2=0x7f67340b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340b0f60 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7f673415f180 tx=0x7f672402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 -- 172.21.15.165:0/1753800455 shutdown_connections 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67340cdae0 0x7f67340cdf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 --2- 172.21.15.165:0/1753800455 >> v2:172.21.15.172:3300/0 conn(0x7f673415b7a0 0x7f67340cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 --2- 172.21.15.165:0/1753800455 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340b0f60 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 -- 172.21.15.165:0/1753800455 >> 172.21.15.165:0/1753800455 conn(0x7f6734061cc0 msgr2=0x7f673406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 -- 172.21.15.165:0/1753800455 shutdown_connections 2024-04-25T10:19:09.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.889+0000 7f6733e00640 1 -- 172.21.15.165:0/1753800455 wait complete. 2024-04-25T10:19:09.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.891+0000 7f6733e00640 1 Processor -- start 2024-04-25T10:19:09.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.891+0000 7f6733e00640 1 -- start start 2024-04-25T10:19:09.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f6733e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67340cdae0 0x7f67341f4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f6733e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340aae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f6733e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673415b7a0 0x7f67340aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f6733e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673415e190 con 0x7f673415b3c0 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f6733e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673415e010 con 0x7f67340cdae0 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f6733e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673415e310 con 0x7f673415b7a0 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340aae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340aae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49418/0 (socket says 172.21.15.165:49418) 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.892+0000 7f672be00640 1 -- 172.21.15.165:0/3289925712 learned_addr learned my addr 172.21.15.165:0/3289925712 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672be00640 1 -- 172.21.15.165:0/3289925712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673415b7a0 msgr2=0x7f67340aff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672be00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673415b7a0 0x7f67340aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672be00640 1 -- 172.21.15.165:0/3289925712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67340cdae0 msgr2=0x7f67341f4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672be00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67340cdae0 0x7f67341f4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672be00640 1 -- 172.21.15.165:0/3289925712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67340a85b0 con 0x7f673415b3c0 2024-04-25T10:19:09.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672be00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340aae10 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c7f0 tx=0x7f671c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672aa00640 1 -- 172.21.15.165:0/3289925712 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c015070 con 0x7f673415b3c0 2024-04-25T10:19:09.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672aa00640 1 -- 172.21.15.165:0/3289925712 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c007820 con 0x7f673415b3c0 2024-04-25T10:19:09.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f672aa00640 1 -- 172.21.15.165:0/3289925712 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c007bd0 con 0x7f673415b3c0 2024-04-25T10:19:09.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67340a8890 con 0x7f673415b3c0 2024-04-25T10:19:09.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.893+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67340a8e50 con 0x7f673415b3c0 2024-04-25T10:19:09.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.896+0000 7f672aa00640 1 -- 172.21.15.165:0/3289925712 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f671c018070 con 0x7f673415b3c0 2024-04-25T10:19:09.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.896+0000 7f672aa00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f671006b4d0 0x7f671006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.897+0000 7f672aa00640 1 -- 172.21.15.165:0/3289925712 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f671c08a120 con 0x7f673415b3c0 2024-04-25T10:19:09.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.897+0000 7f6732a00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f671006b4d0 0x7f671006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.897+0000 7f673c625740 1 Processor -- start 2024-04-25T10:19:09.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.897+0000 7f673c625740 1 -- start start 2024-04-25T10:19:09.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f6732a00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f671006b4d0 0x7f671006d990 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6724034000 tx=0x7f672402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b7ded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073b7c150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f673c625740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f673c625740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561073adfc70 con 0x561073ba6830 2024-04-25T10:19:09.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561073a88cf0 con 0x561073b7bd70 2024-04-25T10:19:09.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.898+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561073adbd10 con 0x561073bb8440 2024-04-25T10:19:09.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba6cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49428/0 (socket says 172.21.15.165:49428) 2024-04-25T10:19:09.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 -- 172.21.15.165:0/2495845702 learned_addr learned my addr 172.21.15.165:0/2495845702 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f672be00640 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073b7c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6732a00640 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b7ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 -- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 msgr2=0x561073b7c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073b7c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 -- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 msgr2=0x561073b7ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b7ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.899+0000 7f6733400640 1 -- 172.21.15.165:0/2495845702 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5610738be7d0 con 0x561073ba6830 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.900+0000 7f6733400640 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba6cb0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f672c007ef0 tx=0x7f672c00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe630e6439036da0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.900+0000 7f6723400640 1 -- 172.21.15.165:0/2495845702 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c003b90 con 0x561073ba6830 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.900+0000 7f6723400640 1 -- 172.21.15.165:0/2495845702 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c003d30 con 0x561073ba6830 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.900+0000 7f6723400640 1 -- 172.21.15.165:0/2495845702 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c00d880 con 0x561073ba6830 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.900+0000 7f673c625740 1 -- 172.21.15.165:0/2495845702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 msgr2=0x561073ba6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.900+0000 7f673c625740 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba6cb0 secure :-1 s=READY pgs=2469 cs=0 l=1 rev1=1 crypto rx=0x7f672c007ef0 tx=0x7f672c00c790 comp rx=0 tx=0).stop 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 -- 172.21.15.165:0/2495845702 shutdown_connections 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba6cb0 unknown :-1 s=CLOSED pgs=2469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073b7c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 --2- 172.21.15.165:0/2495845702 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b7ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 -- 172.21.15.165:0/2495845702 >> 172.21.15.165:0/2495845702 conn(0x561073b43a30 msgr2=0x561073bc1830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 -- 172.21.15.165:0/2495845702 shutdown_connections 2024-04-25T10:19:09.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 -- 172.21.15.165:0/2495845702 wait complete. 2024-04-25T10:19:09.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.901+0000 7f673c625740 1 -- start start 2024-04-25T10:19:09.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.902+0000 7f673c625740 1 Processor -- start 2024-04-25T10:19:09.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073b7bd70 0x561073ba6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba57d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073bb8440 0x561073ba5d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561073a7e3a0 con 0x561073ba6830 2024-04-25T10:19:09.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x561073ba4380 con 0x561073bb8440 2024-04-25T10:19:09.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561073bba480 con 0x561073b7bd70 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f673c625740 10 client.0 _ll_register_callbacks cb 0x561073a9bb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba57d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49434/0 (socket says 172.21.15.165:49434) 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 -- 172.21.15.165:0/998521341 learned_addr learned my addr 172.21.15.165:0/998521341 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f6732a00640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073b7bd70 0x561073ba6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f6733400640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073bb8440 0x561073ba5d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 -- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073b7bd70 msgr2=0x561073ba6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073b7bd70 0x561073ba6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 -- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073bb8440 msgr2=0x561073ba5d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073bb8440 0x561073ba5d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.903+0000 7f672be00640 1 -- 172.21.15.165:0/998521341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5610737f43f0 con 0x561073ba6830 2024-04-25T10:19:09.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f672be00640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba57d0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f671c006b10 tx=0x7f671c05c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6094c50c8bd8066d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f671be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f671be00640 1 -- 172.21.15.165:0/998521341 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c01f040 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f671be00640 1 -- 172.21.15.165:0/998521341 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f671c057b50 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f672c003d30 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f671be00640 1 -- 172.21.15.165:0/998521341 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f671c088120 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f673c625740 10 client.10974 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x5610737b40f0 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f671be00640 1 -- 172.21.15.165:0/998521341 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c003d30 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.904+0000 7f671be00640 1 -- 172.21.15.165:0/998521341 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f671c022060 con 0x561073ba6830 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f673c625740 10 client.10974 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:09.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f673c625740 10 client.10974 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f673c625740 20 client.10974 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f673c625740 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073b93330 0x561073bd5240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f673c625740 4 client.10974 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561073aed1b0 con 0x561073b93330 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.905+0000 7f6732a00640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073b93330 0x561073bd5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f6732a00640 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073b93330 0x561073bd5240 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df606e08ea31df38 server_cookie=34c50daa16f4aa91 in_seq=0 out_seq=0 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f671be00640 10 client.10974 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f671be00640 1 -- 172.21.15.165:0/998521341 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x561073aed1b0 con 0x561073b93330 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f671be00640 10 client.10974 handle_command_reply: tid=0 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f673c625740 1 client.10974 shutdown 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 msgr2=0x561073ba57d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f673c625740 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba57d0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f671c006b10 tx=0x7f671c05c850 comp rx=0 tx=0).stop 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 shutdown_connections 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.906+0000 7f673c625740 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073b93330 0x561073bd5240 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073bb8440 0x561073ba5d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073ba57d0 unknown :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 1 --2- 172.21.15.165:0/998521341 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073b7bd70 0x561073ba6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f671be00640 0 client.10974 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 >> 172.21.15.165:0/998521341 conn(0x561073b43a30 msgr2=0x561073bc4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 shutdown_connections 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 1 -- 172.21.15.165:0/998521341 wait complete. 2024-04-25T10:19:09.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.907+0000 7f673c625740 20 client.10974 trim_cache size 0 max 16384 2024-04-25T10:19:09.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.923+0000 7f673c625740 1 Processor -- start 2024-04-25T10:19:09.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.923+0000 7f673c625740 1 -- start start 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073baa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073ba6830 0x561073be46d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f673c625740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 0x561073bc1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f673c625740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6734200360 con 0x561073bb8440 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67341ff4c0 con 0x561073b7bd70 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67341f7590 con 0x561073ba6830 2024-04-25T10:19:09.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f6733400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 0x561073bc1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.924+0000 7f6733400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 0x561073bc1fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49440/0 (socket says 172.21.15.165:49440) 2024-04-25T10:19:09.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 -- 172.21.15.165:0/3040172191 learned_addr learned my addr 172.21.15.165:0/3040172191 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f672be00640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073ba6830 0x561073be46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6732a00640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073baa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 -- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073ba6830 msgr2=0x561073be46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073ba6830 0x561073be46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 -- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 msgr2=0x561073baa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073baa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 -- 172.21.15.165:0/3040172191 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561073b72940 con 0x561073bb8440 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f672be00640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073ba6830 0x561073be46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6732a00640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073baa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6733400640 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 0x561073bc1fe0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f672c00cc90 tx=0x7f672c00f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f4dc5ebb8c5171c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.925+0000 7f6722a00640 1 -- 172.21.15.165:0/3040172191 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c023070 con 0x561073bb8440 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f6722a00640 1 -- 172.21.15.165:0/3040172191 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c007a80 con 0x561073bb8440 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f6722a00640 1 -- 172.21.15.165:0/3040172191 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c003bb0 con 0x561073bb8440 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 -- 172.21.15.165:0/3040172191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 msgr2=0x561073bc1fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 0x561073bc1fe0 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f672c00cc90 tx=0x7f672c00f000 comp rx=0 tx=0).stop 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 -- 172.21.15.165:0/3040172191 shutdown_connections 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073bb8440 0x561073bc1fe0 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073ba6830 0x561073be46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 --2- 172.21.15.165:0/3040172191 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073baa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 -- 172.21.15.165:0/3040172191 >> 172.21.15.165:0/3040172191 conn(0x561073b43a30 msgr2=0x561073b72230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 -- 172.21.15.165:0/3040172191 shutdown_connections 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.926+0000 7f673c625740 1 -- 172.21.15.165:0/3040172191 wait complete. 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.927+0000 7f673c625740 1 -- start start 2024-04-25T10:19:09.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.927+0000 7f673c625740 1 Processor -- start 2024-04-25T10:19:09.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b89130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073c21840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073bcff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x561073b78f90 con 0x561073ba6830 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67341feef0 con 0x561073b7bd70 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x561073a9cfb0 con 0x561073bb8440 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f673c625740 10 client.0 _ll_register_callbacks cb 0x7f671c0584d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073c21840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.928+0000 7f672be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073c21840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:49442/0 (socket says 172.21.15.165:49442) 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 -- 172.21.15.165:0/3483725504 learned_addr learned my addr 172.21.15.165:0/3483725504 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f6732a00640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b89130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f6733400640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073bcff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 -- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 msgr2=0x561073bcff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073bcff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 -- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 msgr2=0x561073b89130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b89130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 -- 172.21.15.165:0/3483725504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561073b7b4c0 con 0x561073ba6830 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f6733400640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073bcff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f672be00640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073c21840 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f671c007410 tx=0x7f671c006aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8870df8733bf78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f671be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.929+0000 7f671be00640 1 -- 172.21.15.165:0/3483725504 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c01f070 con 0x561073ba6830 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f671be00640 1 -- 172.21.15.165:0/3483725504 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f671c0577d0 con 0x561073ba6830 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f672c007a80 con 0x561073ba6830 2024-04-25T10:19:09.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f671be00640 1 -- 172.21.15.165:0/3483725504 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f671c059a30 con 0x561073ba6830 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 10 client.10980 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x561073bd0780 con 0x561073ba6830 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f671be00640 1 -- 172.21.15.165:0/3483725504 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f672c007a80 con 0x561073ba6830 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f671be00640 1 -- 172.21.15.165:0/3483725504 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f671c059bd0 con 0x561073ba6830 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 10 client.10980 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 10 client.10980 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 20 client.10980 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073ba2350 0x561073bb7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 4 client.10980 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.930+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561073ae7500 con 0x561073ba2350 2024-04-25T10:19:09.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f6732a00640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073ba2350 0x561073bb7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f6732a00640 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073ba2350 0x561073bb7b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7798de5bcc27fe0 server_cookie=8eae5c2af7930354 in_seq=0 out_seq=0 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f671be00640 10 client.10980 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f671be00640 1 -- 172.21.15.165:0/3483725504 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x561073ae7500 con 0x561073ba2350 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f671be00640 10 client.10980 handle_command_reply: tid=0 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f673c625740 1 client.10980 shutdown 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 msgr2=0x561073c21840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.931+0000 7f673c625740 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073c21840 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f671c007410 tx=0x7f671c006aa0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 shutdown_connections 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x561073ba2350 0x561073bb7b60 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x561073bb8440 0x561073bcff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x561073ba6830 0x561073c21840 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 --2- 172.21.15.165:0/3483725504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x561073b7bd70 0x561073b89130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f671be00640 0 client.10980 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 >> 172.21.15.165:0/3483725504 conn(0x561073b43a30 msgr2=0x561073bdb6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 shutdown_connections 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 1 -- 172.21.15.165:0/3483725504 wait complete. 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f673c625740 20 client.10980 trim_cache size 0 max 16384 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:19:09.940 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout: "117de01f-6402-4bd3-a9c7-1bb28fce9c60": { 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/", 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "117de01f-6402-4bd3-a9c7-1bb28fce9c60", 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive,repair" 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:09.941 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:09.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f671006b4d0 msgr2=0x7f671006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f6733e00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f671006b4d0 0x7f671006d990 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f6724034000 tx=0x7f672402c000 comp rx=0 tx=0).stop 2024-04-25T10:19:09.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 msgr2=0x7f67340aae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:09.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.932+0000 7f6733e00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340aae10 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7f671c00c7f0 tx=0x7f671c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 shutdown_connections 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f671006b4d0 0x7f671006d990 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f673415b7a0 0x7f67340aff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f673415b3c0 0x7f67340aae10 unknown :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 --2- 172.21.15.165:0/3289925712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67340cdae0 0x7f67341f4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 >> 172.21.15.165:0/3289925712 conn(0x7f6734061cc0 msgr2=0x7f673415d310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 shutdown_connections 2024-04-25T10:19:09.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:09.933+0000 7f6733e00640 1 -- 172.21.15.165:0/3289925712 wait complete. 2024-04-25T10:19:09.952 INFO:tasks.cephfs.filesystem:scrub status for tag:117de01f-6402-4bd3-a9c7-1bb28fce9c60 - {'path': '/', 'tag': '117de01f-6402-4bd3-a9c7-1bb28fce9c60', 'options': 'recursive,repair'} 2024-04-25T10:19:31.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:31.167 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:19:39.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:39.953 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.12:0 scrub status 2024-04-25T10:19:40.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.157+0000 7f4c72c00640 1 Processor -- start 2024-04-25T10:19:40.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.157+0000 7f4c72c00640 1 -- start start 2024-04-25T10:19:40.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 0x7f4c6c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4c6c0ce950 0x7f4c6c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6c145520 con 0x7f4c6c0c5030 2024-04-25T10:19:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6c0ced90 con 0x7f4c6c0ce950 2024-04-25T10:19:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c071270 con 0x7f4c6c145140 2024-04-25T10:19:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f4c6c0c5030 0x7f4c6c06d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58488/0 (socket says 172.21.15.165:58488) 2024-04-25T10:19:40.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c72200640 1 -- 172.21.15.165:0/144669256 learned_addr learned my addr 172.21.15.165:0/144669256 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c70e00640 1 --2- 172.21.15.165:0/144669256 >> v2:172.21.15.172:3300/0 conn(0x7f4c6c0ce950 0x7f4c6c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.159+0000 7f4c71800640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 0x7f4c6c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234527873 0 0) 0x7f4c6c145520 con 0x7f4c6c0c5030 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c54003520 con 0x7f4c6c0c5030 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1149289417 0 0) 0x7f4c54003520 con 0x7f4c6c0c5030 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 msgr2=0x7f4c6c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 0x7f4c6c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 >> v2:172.21.15.172:3300/0 conn(0x7f4c6c0ce950 msgr2=0x7f4c6c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 --2- 172.21.15.165:0/144669256 >> v2:172.21.15.172:3300/0 conn(0x7f4c6c0ce950 0x7f4c6c0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c071550 con 0x7f4c6c0c5030 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4c640019f0 con 0x7f4c6c0c5030 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 >> v1:172.21.15.165:6789/0 conn(0x7f4c6c0c5030 legacy=0x7f4c6c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c540039b0 0x7f4c54003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 0x7f4c54004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0ced90 con 0x7f4c54004d40 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c071270 con 0x7f4c540039b0 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5400b9c0 con 0x7f4c6c145140 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.160+0000 7f4c72200640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c70e00640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 0x7f4c54004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c71800640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c540039b0 0x7f4c54003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c72200640 1 -- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 msgr2=0x7f4c54004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c72200640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c145140 0x7f4c54004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c72200640 1 -- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c540039b0 msgr2=0x7f4c54003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c72200640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c540039b0 0x7f4c54003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.161+0000 7f4c72200640 1 -- 172.21.15.165:0/144669256 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54003520 con 0x7f4c54004d40 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.162+0000 7f4c72200640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400a880 tx=0x7f4c54005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb3570b2f62d077 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.162+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6400a4a0 con 0x7f4c54004d40 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.162+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c6402f070 con 0x7f4c54004d40 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.162+0000 7f4c6be00640 1 -- 172.21.15.165:0/144669256 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c6402bd00 con 0x7f4c54004d40 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72c00640 1 -- 172.21.15.165:0/144669256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c54004d40 msgr2=0x7f4c540051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72c00640 1 --2- 172.21.15.165:0/144669256 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c54004d40 0x7f4c540051c0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7f4c6400a880 tx=0x7f4c54005830 comp rx=0 tx=0).stop 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72200640 1 -- 172.21.15.165:0/144669256 reap_dead start 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72c00640 1 -- 172.21.15.165:0/144669256 shutdown_connections 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72c00640 1 -- 172.21.15.165:0/144669256 >> 172.21.15.165:0/144669256 conn(0x7f4c6c061cc0 msgr2=0x7f4c6c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72c00640 1 -- 172.21.15.165:0/144669256 shutdown_connections 2024-04-25T10:19:40.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.163+0000 7f4c72c00640 1 -- 172.21.15.165:0/144669256 wait complete. 2024-04-25T10:19:40.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.165+0000 7f4c72c00640 1 Processor -- start 2024-04-25T10:19:40.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.165+0000 7f4c72c00640 1 -- start start 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.166+0000 7f4c72c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c54004d40 0x7f4c6c0cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.166+0000 7f4c72c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 0x7f4c6c0cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.166+0000 7f4c72c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c0a9e90 0x7f4c6c0aa310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.166+0000 7f4c72c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0c54a0 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.166+0000 7f4c72c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c071b10 con 0x7f4c54004d40 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.166+0000 7f4c72c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0c5620 con 0x7f4c6c0a9e90 2024-04-25T10:19:40.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 0x7f4c6c0cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 0x7f4c6c0cad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35536/0 (socket says 172.21.15.165:35536) 2024-04-25T10:19:40.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 -- 172.21.15.165:0/505823719 learned_addr learned my addr 172.21.15.165:0/505823719 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c71800640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c54004d40 0x7f4c6c0cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 -- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c0a9e90 msgr2=0x7f4c6c0aa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c72200640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c0a9e90 0x7f4c6c0aa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c0a9e90 0x7f4c6c0aa310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 -- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c54004d40 msgr2=0x7f4c6c0cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c54004d40 0x7f4c6c0cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 -- 172.21.15.165:0/505823719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6c0a9100 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c71800640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c54004d40 0x7f4c6c0cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.167+0000 7f4c70e00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 0x7f4c6c0cad20 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f4c58011730 tx=0x7f4c58011bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.168+0000 7f4c69600640 1 -- 172.21.15.165:0/505823719 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5800f9c0 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.168+0000 7f4c69600640 1 -- 172.21.15.165:0/505823719 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5800fb60 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.168+0000 7f4c69600640 1 -- 172.21.15.165:0/505823719 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58015460 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.168+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6c0a93e0 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.168+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6c1f46c0 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.170+0000 7f4c69600640 1 -- 172.21.15.165:0/505823719 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4c58015600 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.172+0000 7f4c69600640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.172+0000 7f4c69600640 1 -- 172.21.15.165:0/505823719 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c5808ae50 con 0x7f4c6c1f0b60 2024-04-25T10:19:40.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.172+0000 7f4c71800640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.172+0000 7f4c77187740 1 Processor -- start 2024-04-25T10:19:40.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.173+0000 7f4c77187740 1 -- start start 2024-04-25T10:19:40.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.173+0000 7f4c71800640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009a00 tx=0x7f4c60008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.173+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba0993df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c77187740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba0995c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba0995cec0 0x55ba09914dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c77187740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09879c70 con 0x55ba09917f10 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09822cf0 con 0x55ba0993db60 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba09875d10 con 0x55ba0995cec0 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba0995c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba0995c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35544/0 (socket says 172.21.15.165:35544) 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 -- 172.21.15.165:0/1251981381 learned_addr learned my addr 172.21.15.165:0/1251981381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c71800640 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba0993df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 -- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba0995cec0 msgr2=0x55ba09914dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba0995cec0 0x55ba09914dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 -- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 msgr2=0x55ba0993df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba0993df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.174+0000 7f4c70e00640 1 -- 172.21.15.165:0/1251981381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba096587d0 con 0x55ba09917f10 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.175+0000 7f4c70e00640 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba0995c980 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f4c58061580 tx=0x7f4c58055770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8bdcee967c20646 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.175+0000 7f4c5e000640 1 -- 172.21.15.165:0/1251981381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580576a0 con 0x55ba09917f10 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.175+0000 7f4c5e000640 1 -- 172.21.15.165:0/1251981381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5805d160 con 0x55ba09917f10 2024-04-25T10:19:40.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.175+0000 7f4c5e000640 1 -- 172.21.15.165:0/1251981381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c580587f0 con 0x55ba09917f10 2024-04-25T10:19:40.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.175+0000 7f4c77187740 1 -- 172.21.15.165:0/1251981381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 msgr2=0x55ba0995c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.175+0000 7f4c77187740 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba0995c980 secure :-1 s=READY pgs=2476 cs=0 l=1 rev1=1 crypto rx=0x7f4c58061580 tx=0x7f4c58055770 comp rx=0 tx=0).stop 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 -- 172.21.15.165:0/1251981381 shutdown_connections 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba0995cec0 0x55ba09914dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba0995c980 unknown :-1 s=CLOSED pgs=2476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 --2- 172.21.15.165:0/1251981381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba0993df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 -- 172.21.15.165:0/1251981381 >> 172.21.15.165:0/1251981381 conn(0x55ba098dda30 msgr2=0x55ba0995b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 -- 172.21.15.165:0/1251981381 shutdown_connections 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 -- 172.21.15.165:0/1251981381 wait complete. 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.176+0000 7f4c77187740 1 -- start start 2024-04-25T10:19:40.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.177+0000 7f4c77187740 1 Processor -- start 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba09917f10 0x55ba09924f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 0x55ba099363b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 0x55ba09914d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09912370 con 0x55ba0993db60 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba098183a0 con 0x55ba09917f10 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba09916ea0 con 0x55ba09935cf0 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c77187740 10 client.0 _ll_register_callbacks cb 0x55ba09835b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:40.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 0x55ba099363b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 0x55ba099363b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35552/0 (socket says 172.21.15.165:35552) 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 -- 172.21.15.165:0/3137536130 learned_addr learned my addr 172.21.15.165:0/3137536130 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c71800640 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba09917f10 0x55ba09924f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 -- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 msgr2=0x55ba09914d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 0x55ba09914d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 -- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba09917f10 msgr2=0x55ba09924f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba09917f10 0x55ba09924f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.178+0000 7f4c70e00640 1 -- 172.21.15.165:0/3137536130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba0958e3f0 con 0x55ba0993db60 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c70e00640 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 0x55ba099363b0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f4c5801c000 tx=0x7f4c5801f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef0377195cf6412d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c52a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c52a00640 1 -- 172.21.15.165:0/3137536130 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58087030 con 0x55ba0993db60 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c52a00640 1 -- 172.21.15.165:0/3137536130 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c580587f0 con 0x55ba0993db60 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c5805d160 con 0x55ba0993db60 2024-04-25T10:19:40.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c52a00640 1 -- 172.21.15.165:0/3137536130 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c58058a10 con 0x55ba0993db60 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c77187740 10 client.10992 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55ba0954e0f0 con 0x55ba0993db60 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c52a00640 1 -- 172.21.15.165:0/3137536130 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5805d160 con 0x55ba0993db60 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c52a00640 1 -- 172.21.15.165:0/3137536130 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4c580550f0 con 0x55ba0993db60 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c77187740 10 client.10992 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c77187740 10 client.10992 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.179+0000 7f4c77187740 20 client.10992 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.180+0000 7f4c77187740 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09973000 0x55ba09973470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.180+0000 7f4c77187740 4 client.10992 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.180+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba098871b0 con 0x55ba09973000 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.180+0000 7f4c71800640 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09973000 0x55ba09973470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.180+0000 7f4c71800640 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09973000 0x55ba09973470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1811441ccad8e1b9 server_cookie=98c8647d355bad0a in_seq=0 out_seq=0 2024-04-25T10:19:40.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.180+0000 7f4c52a00640 10 client.10992 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c52a00640 1 -- 172.21.15.165:0/3137536130 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ba098871b0 con 0x55ba09973000 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c52a00640 10 client.10992 handle_command_reply: tid=0 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 client.10992 shutdown 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 msgr2=0x55ba099363b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 0x55ba099363b0 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f4c5801c000 tx=0x7f4c5801f070 comp rx=0 tx=0).stop 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 shutdown_connections 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09973000 0x55ba09973470 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 0x55ba09914d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba0993db60 0x55ba099363b0 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 --2- 172.21.15.165:0/3137536130 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba09917f10 0x55ba09924f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c52a00640 0 client.10992 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.181+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 >> 172.21.15.165:0/3137536130 conn(0x55ba098dda30 msgr2=0x55ba0993b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.182+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 shutdown_connections 2024-04-25T10:19:40.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.182+0000 7f4c77187740 1 -- 172.21.15.165:0/3137536130 wait complete. 2024-04-25T10:19:40.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.182+0000 7f4c77187740 20 client.10992 trim_cache size 0 max 16384 2024-04-25T10:19:40.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.198+0000 7f4c77187740 1 Processor -- start 2024-04-25T10:19:40.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.198+0000 7f4c77187740 1 -- start start 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09917f10 0x55ba099827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c77187740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 0x55ba09952010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09921590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c77187740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c1fc570 con 0x55ba09935cf0 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c1fb840 con 0x55ba0993db60 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0cd7a0 con 0x55ba09917f10 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.199+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 0x55ba09952010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 0x55ba09952010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35568/0 (socket says 172.21.15.165:35568) 2024-04-25T10:19:40.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 -- 172.21.15.165:0/157040812 learned_addr learned my addr 172.21.15.165:0/157040812 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c71800640 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09917f10 0x55ba099827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 -- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09917f10 msgr2=0x55ba099827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09917f10 0x55ba099827a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 -- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 msgr2=0x55ba09921590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09921590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 -- 172.21.15.165:0/157040812 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba09944ce0 con 0x55ba09935cf0 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c70e00640 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 0x55ba09952010 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f4c580590a0 tx=0x7f4c58083000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913504f39ade6960 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.200+0000 7f4c5d600640 1 -- 172.21.15.165:0/157040812 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5805d160 con 0x55ba09935cf0 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c5d600640 1 -- 172.21.15.165:0/157040812 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58073030 con 0x55ba09935cf0 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c5d600640 1 -- 172.21.15.165:0/157040812 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c58058460 con 0x55ba09935cf0 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 -- 172.21.15.165:0/157040812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 msgr2=0x55ba09952010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 0x55ba09952010 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f4c580590a0 tx=0x7f4c58083000 comp rx=0 tx=0).stop 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 -- 172.21.15.165:0/157040812 shutdown_connections 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09921590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09935cf0 0x55ba09952010 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 --2- 172.21.15.165:0/157040812 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09917f10 0x55ba099827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 -- 172.21.15.165:0/157040812 >> 172.21.15.165:0/157040812 conn(0x55ba098dda30 msgr2=0x55ba0990c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 -- 172.21.15.165:0/157040812 shutdown_connections 2024-04-25T10:19:40.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.201+0000 7f4c77187740 1 -- 172.21.15.165:0/157040812 wait complete. 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.202+0000 7f4c77187740 1 -- start start 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.202+0000 7f4c77187740 1 Processor -- start 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba099205a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 0x55ba09979440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09979980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6c0cdda0 con 0x55ba09917f10 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba09916b90 con 0x55ba0993db60 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ba09923510 con 0x55ba09935cf0 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c77187740 10 client.0 _ll_register_callbacks cb 0x7f4c58059b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c71800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba099205a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c71800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba099205a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35580/0 (socket says 172.21.15.165:35580) 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.203+0000 7f4c71800640 1 -- 172.21.15.165:0/3898536897 learned_addr learned my addr 172.21.15.165:0/3898536897 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c71800640 1 -- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 msgr2=0x55ba09979440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c71800640 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 0x55ba09979440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c72200640 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09979980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c71800640 1 -- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 msgr2=0x55ba09979980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c71800640 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09979980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c71800640 1 -- 172.21.15.165:0/3898536897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba0993b960 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c71800640 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba099205a0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f4c60012b80 tx=0x7f4c60004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e440c946a1bcfe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c52a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c52a00640 1 -- 172.21.15.165:0/3898536897 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c60024cc0 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c52a00640 1 -- 172.21.15.165:0/3898536897 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f4c600291d0 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.204+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c58073030 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c52a00640 1 -- 172.21.15.165:0/3898536897 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=121) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c60039050 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 10 client.10998 fetch_fsmap learned FSMap version 121 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=121,monmap=2+}) v3 -- 0x55ba09923310 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c52a00640 1 -- 172.21.15.165:0/3898536897 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c58073030 con 0x55ba09917f10 2024-04-25T10:19:40.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c52a00640 1 -- 172.21.15.165:0/3898536897 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 121) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f4c6003c020 con 0x55ba09917f10 2024-04-25T10:19:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 10 client.10998 fetch_fsmap finished waiting for FSMap version 121 2024-04-25T10:19:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 10 client.10998 resolve_mds: resolved 12:0 to role '12:0' aka daemon mds.a 2024-04-25T10:19:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 20 client.10998 populate_metadata read hostname 'smithi165' 2024-04-25T10:19:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09939920 0x55ba09939d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 4 client.10998 mds_command: new command op to 10767 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:19:40.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 --> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba09943a40 con 0x55ba09939920 2024-04-25T10:19:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.205+0000 7f4c70e00640 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09939920 0x55ba09939d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c70e00640 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09939920 0x55ba09939d40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f904af6d6146f34 server_cookie=910c6585c58b4b10 in_seq=0 out_seq=0 2024-04-25T10:19:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c52a00640 10 client.10998 ms_handle_connect on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c52a00640 1 -- 172.21.15.165:0/3898536897 <== mds.0 v2:172.21.15.165:6834/1299523073 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ba09943a40 con 0x55ba09939920 2024-04-25T10:19:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c52a00640 10 client.10998 handle_command_reply: tid=0 2024-04-25T10:19:40.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 client.10998 shutdown 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 msgr2=0x55ba099205a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba099205a0 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7f4c60012b80 tx=0x7f4c60004730 comp rx=0 tx=0).stop 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 shutdown_connections 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:6834/1299523073,v1:172.21.15.165:6836/1299523073] conn(0x55ba09939920 0x55ba09939d40 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ba0993db60 0x55ba09979980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c52a00640 0 client.10998 ms_handle_reset on v2:172.21.15.165:6834/1299523073 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ba09935cf0 0x55ba09979440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 --2- 172.21.15.165:0/3898536897 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ba09917f10 0x55ba099205a0 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.206+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 >> 172.21.15.165:0/3898536897 conn(0x55ba098dda30 msgr2=0x55ba099515b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 shutdown_connections 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c77187740 1 -- 172.21.15.165:0/3898536897 wait complete. 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c77187740 20 client.10998 trim_cache size 0 max 16384 2024-04-25T10:19:40.213 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:40.214 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:19:40.214 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:19:40.214 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c4c06b4d0 msgr2=0x7f4c4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c72c00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009a00 tx=0x7f4c60008040 comp rx=0 tx=0).stop 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 msgr2=0x7f4c6c0cad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.207+0000 7f4c72c00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 0x7f4c6c0cad20 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7f4c58011730 tx=0x7f4c58011bf0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 shutdown_connections 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4c4c06b4d0 0x7f4c4c06d990 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4c6c0a9e90 0x7f4c6c0aa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4c6c1f0b60 0x7f4c6c0cad20 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 --2- 172.21.15.165:0/505823719 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4c54004d40 0x7f4c6c0cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 >> 172.21.15.165:0/505823719 conn(0x7f4c6c061cc0 msgr2=0x7f4c6c15dd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 shutdown_connections 2024-04-25T10:19:40.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.208+0000 7f4c72c00640 1 -- 172.21.15.165:0/505823719 wait complete. 2024-04-25T10:19:40.229 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:19:40.229 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:19:40.229 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:19:40.229 DEBUG:teuthology.orchestra.run.smithi165:> import stat 2024-04-25T10:19:40.229 DEBUG:teuthology.orchestra.run.smithi165:> import json 2024-04-25T10:19:40.229 DEBUG:teuthology.orchestra.run.smithi165:> import sys 2024-04-25T10:19:40.229 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> except OSError as e: 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> sys.exit(e.errno) 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> print(json.dumps( 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> dict([(a, getattr(s, a)) for a in attrs]), 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> indent=2)) 2024-04-25T10:19:40.230 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:19:40.371 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:40.371 INFO:teuthology.orchestra.run.smithi165.stdout: "st_mode": 33188, 2024-04-25T10:19:40.371 INFO:teuthology.orchestra.run.smithi165.stdout: "st_ino": 1099511627777, 2024-04-25T10:19:40.371 INFO:teuthology.orchestra.run.smithi165.stdout: "st_dev": 40, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_nlink": 1, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_uid": 1000, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_gid": 1316, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_size": 6291456, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_atime": 1714040347.9174836, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_mtime": 1714040347.9644828, 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout: "st_ctime": 1714040347.9644828 2024-04-25T10:19:40.372 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:40.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:19:40.385 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:40.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.622+0000 7f498b200640 1 Processor -- start 2024-04-25T10:19:40.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.622+0000 7f498b200640 1 -- start start 2024-04-25T10:19:40.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.623+0000 7f498b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c144440 0x7f498c070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.623+0000 7f498b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f498c071530 0x7f498c0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.623+0000 7f498b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f498c0cd860 0x7f498c06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.623+0000 7f498b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c15c860 con 0x7f498c0cd860 2024-04-25T10:19:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.623+0000 7f498b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c071970 con 0x7f498c071530 2024-04-25T10:19:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.623+0000 7f498b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15bf30 con 0x7f498c144440 2024-04-25T10:19:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f498c0cd860 0x7f498c06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f498c0cd860 0x7f498c06d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35596/0 (socket says 172.21.15.165:35596) 2024-04-25T10:19:40.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 -- 172.21.15.165:0/1575607728 learned_addr learned my addr 172.21.15.165:0/1575607728 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f4983e00640 1 --2- 172.21.15.165:0/1575607728 >> v2:172.21.15.172:3300/0 conn(0x7f498c071530 0x7f498c0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f4989e00640 1 --2- 172.21.15.165:0/1575607728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c144440 0x7f498c070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 -- 172.21.15.165:0/1575607728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c144440 msgr2=0x7f498c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 --2- 172.21.15.165:0/1575607728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c144440 0x7f498c070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 -- 172.21.15.165:0/1575607728 >> v2:172.21.15.172:3300/0 conn(0x7f498c071530 msgr2=0x7f498c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 --2- 172.21.15.165:0/1575607728 >> v2:172.21.15.172:3300/0 conn(0x7f498c071530 0x7f498c0cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.624+0000 7f498a800640 1 -- 172.21.15.165:0/1575607728 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c15c210 con 0x7f498c0cd860 2024-04-25T10:19:40.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.626+0000 7f498a800640 1 --2- 172.21.15.165:0/1575607728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c0cd860 0x7f498c06d050 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f498c072a00 tx=0x7f498402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4a4a466f7ef0ff server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.626+0000 7f4989400640 1 -- 172.21.15.165:0/1575607728 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4984036070 con 0x7f498c0cd860 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.626+0000 7f4989400640 1 -- 172.21.15.165:0/1575607728 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4984031600 con 0x7f498c0cd860 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.626+0000 7f4989400640 1 -- 172.21.15.165:0/1575607728 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49840319d0 con 0x7f498c0cd860 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 -- 172.21.15.165:0/1575607728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c0cd860 msgr2=0x7f498c06d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 --2- 172.21.15.165:0/1575607728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c0cd860 0x7f498c06d050 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f498c072a00 tx=0x7f498402a730 comp rx=0 tx=0).stop 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 -- 172.21.15.165:0/1575607728 shutdown_connections 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 --2- 172.21.15.165:0/1575607728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c0cd860 0x7f498c06d050 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 --2- 172.21.15.165:0/1575607728 >> v2:172.21.15.172:3300/0 conn(0x7f498c071530 0x7f498c0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 --2- 172.21.15.165:0/1575607728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c144440 0x7f498c070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 -- 172.21.15.165:0/1575607728 >> 172.21.15.165:0/1575607728 conn(0x7f498c06abe0 msgr2=0x7f498c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 -- 172.21.15.165:0/1575607728 shutdown_connections 2024-04-25T10:19:40.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.627+0000 7f498b200640 1 -- 172.21.15.165:0/1575607728 wait complete. 2024-04-25T10:19:40.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.629+0000 7f498b200640 1 Processor -- start 2024-04-25T10:19:40.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.629+0000 7f498b200640 1 -- start start 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f498b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 0x7f498c15b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f498b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c0cd860 0x7f498c0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f498b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c144440 0x7f498c0cb220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f498b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0cdcd0 con 0x7f498c071530 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f498b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15bc30 con 0x7f498c144440 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f498b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0cde50 con 0x7f498c0cd860 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f4989e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 0x7f498c15b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f4989e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 0x7f498c15b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35612/0 (socket says 172.21.15.165:35612) 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.630+0000 7f4989e00640 1 -- 172.21.15.165:0/3606026683 learned_addr learned my addr 172.21.15.165:0/3606026683 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:40.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4983e00640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c0cd860 0x7f498c0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4989e00640 1 -- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c0cd860 msgr2=0x7f498c0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f498a800640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c144440 0x7f498c0cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4989e00640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c0cd860 0x7f498c0cace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4989e00640 1 -- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c144440 msgr2=0x7f498c0cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4989e00640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c144440 0x7f498c0cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4989e00640 1 -- 172.21.15.165:0/3606026683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c139540 con 0x7f498c071530 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4989e00640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 0x7f498c15b010 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d5f0 tx=0x7f497c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c0125f0 con 0x7f498c071530 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c019070 con 0x7f498c071530 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c139820 con 0x7f498c071530 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c0129d0 con 0x7f498c071530 2024-04-25T10:19:40.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.631+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c139de0 con 0x7f498c071530 2024-04-25T10:19:40.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.633+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c073990 con 0x7f498c071530 2024-04-25T10:19:40.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.634+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f497c012b70 con 0x7f498c071530 2024-04-25T10:19:40.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.634+0000 7f4982000640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f496806b4d0 0x7f496806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:40.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.635+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f497c08a820 con 0x7f498c071530 2024-04-25T10:19:40.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.635+0000 7f4983e00640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f496806b4d0 0x7f496806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:40.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.636+0000 7f4983e00640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f496806b4d0 0x7f496806d990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f498c0b0fb0 tx=0x7f4974008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:40.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.639+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f497c059470 con 0x7f498c071530 2024-04-25T10:19:40.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.878+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f498c0c8fd0 con 0x7f498c071530 2024-04-25T10:19:40.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.879+0000 7f4982000640 1 -- 172.21.15.165:0/3606026683 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f497c058d70 con 0x7f498c071530 2024-04-25T10:19:40.883 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:40.883 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:40.884 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f496806b4d0 msgr2=0x7f496806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f496806b4d0 0x7f496806d990 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f498c0b0fb0 tx=0x7f4974008040 comp rx=0 tx=0).stop 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 msgr2=0x7f498c15b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 0x7f498c15b010 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7f497c00d5f0 tx=0x7f497c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 shutdown_connections 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f496806b4d0 0x7f496806d990 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f498c144440 0x7f498c0cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f498c0cd860 0x7f498c0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 --2- 172.21.15.165:0/3606026683 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f498c071530 0x7f498c15b010 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:40.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 >> 172.21.15.165:0/3606026683 conn(0x7f498c06abe0 msgr2=0x7f498c0b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:40.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 shutdown_connections 2024-04-25T10:19:40.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:40.882+0000 7f498b200640 1 -- 172.21.15.165:0/3606026683 wait complete. 2024-04-25T10:19:40.906 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json flush journal 2024-04-25T10:19:41.121 INFO:teuthology.orchestra.run.smithi165.stdout:{"message":"","return_code":0} 2024-04-25T10:19:41.122 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:19:41.122 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2024-04-25T10:19:41.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.215+0000 7f0329758f40 1 Processor -- start 2024-04-25T10:19:41.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.215+0000 7f0329758f40 1 -- start start 2024-04-25T10:19:41.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0329758f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 0x562adfc36be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0329758f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562adfbab2a0 0x562adfc39790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0329758f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 0x562adfc39cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0329758f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x562adfb5f840 con 0x7f030c0022a0 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0329758f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x562adfb08dd0 con 0x562adfbab2a0 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0329758f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x562adfb5fae0 con 0x562adfbab6c0 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0326a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 0x562adfc36be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0326a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 0x562adfc36be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35636/0 (socket says 172.21.15.165:35636) 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.216+0000 7f0326a00640 1 -- 172.21.15.165:0/4018827023 learned_addr learned my addr 172.21.15.165:0/4018827023 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0326a00640 1 -- 172.21.15.165:0/4018827023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 msgr2=0x562adfc39cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0326a00640 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 0x562adfc39cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0326a00640 1 -- 172.21.15.165:0/4018827023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562adfbab2a0 msgr2=0x562adfc39790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0326a00640 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562adfbab2a0 0x562adfc39790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0326a00640 1 -- 172.21.15.165:0/4018827023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562adfc37120 con 0x7f030c0022a0 2024-04-25T10:19:41.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0326a00640 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 0x562adfc36be0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f0318025bb0 tx=0x7f031804f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0b74687fe15517b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0324c00640 1 -- 172.21.15.165:0/4018827023 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031800a320 con 0x7f030c0022a0 2024-04-25T10:19:41.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0324c00640 1 -- 172.21.15.165:0/4018827023 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031805e070 con 0x7f030c0022a0 2024-04-25T10:19:41.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.217+0000 7f0324c00640 1 -- 172.21.15.165:0/4018827023 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318058900 con 0x7f030c0022a0 2024-04-25T10:19:41.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 -- 172.21.15.165:0/4018827023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 msgr2=0x562adfc36be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 0x562adfc36be0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f0318025bb0 tx=0x7f031804f000 comp rx=0 tx=0).stop 2024-04-25T10:19:41.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 -- 172.21.15.165:0/4018827023 shutdown_connections 2024-04-25T10:19:41.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 0x562adfc39cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x562adfbab2a0 0x562adfc39790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 --2- 172.21.15.165:0/4018827023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f030c0022a0 0x562adfc36be0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 -- 172.21.15.165:0/4018827023 >> 172.21.15.165:0/4018827023 conn(0x562adfb90e60 msgr2=0x562adfb941b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 -- 172.21.15.165:0/4018827023 shutdown_connections 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 -- 172.21.15.165:0/4018827023 wait complete. 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.218+0000 7f0329758f40 1 Processor -- start 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 -- start start 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562adfbab2a0 0x562adfc37c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 0x562adfc319f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f030c0022a0 0x562adfc35ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x562adfc2eff0 con 0x562adfbab2a0 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x562adfc2e670 con 0x7f030c0022a0 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.219+0000 7f0329758f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x562adfbac1a0 con 0x562adfbab6c0 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562adfbab2a0 0x562adfc37c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562adfbab2a0 0x562adfc37c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35652/0 (socket says 172.21.15.165:35652) 2024-04-25T10:19:41.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 -- 172.21.15.165:0/1058188338 learned_addr learned my addr 172.21.15.165:0/1058188338 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:41.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 -- 172.21.15.165:0/1058188338 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 msgr2=0x562adfc319f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x562adfbab6c0 0x562adfc319f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 -- 172.21.15.165:0/1058188338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f030c0022a0 msgr2=0x562adfc35ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f030c0022a0 0x562adfc35ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 -- 172.21.15.165:0/1058188338 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562adfc32060 con 0x562adfbab2a0 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.220+0000 7f0326a00640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562adfbab2a0 0x562adfc37c90 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f0318064040 tx=0x7f03180033a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.221+0000 7f031f400640 1 -- 172.21.15.165:0/1058188338 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318071070 con 0x562adfbab2a0 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.221+0000 7f031f400640 1 -- 172.21.15.165:0/1058188338 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0318058b80 con 0x562adfbab2a0 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.221+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562adfc2f650 con 0x562adfbab2a0 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.221+0000 7f031f400640 1 -- 172.21.15.165:0/1058188338 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318062460 con 0x562adfbab2a0 2024-04-25T10:19:41.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.221+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562adfc29e30 con 0x562adfbab2a0 2024-04-25T10:19:41.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.223+0000 7f031f400640 1 -- 172.21.15.165:0/1058188338 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0318074070 con 0x562adfbab2a0 2024-04-25T10:19:41.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.223+0000 7f031f400640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0304069eb0 0x7f030406c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.224+0000 7f031f400640 1 -- 172.21.15.165:0/1058188338 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03180ce6e0 con 0x562adfbab2a0 2024-04-25T10:19:41.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.224+0000 7f0326000640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0304069eb0 0x7f030406c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.224+0000 7f0329758f40 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x562adfb92140 0x562adfb925d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.224+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 --> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] -- osd_op(unknown.0.0:1 26.7 26:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e171) v9 -- 0x562adfb92b10 con 0x562adfb92140 2024-04-25T10:19:41.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.224+0000 7f0327400640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x562adfb92140 0x562adfb925d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.224+0000 7f0326000640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0304069eb0 0x7f030406c370 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f0310001b80 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0327400640 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x562adfb92140 0x562adfb925d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0327400640 1 -- 172.21.15.165:0/1058188338 <== osd.2 v2:172.21.15.165:6824/3063902068 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7f0320009410 con 0x562adfb92140 2024-04-25T10:19:41.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x562adfb92140 msgr2=0x562adfb925d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x562adfb92140 0x562adfb925d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0304069eb0 msgr2=0x7f030406c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0304069eb0 0x7f030406c370 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f0310000c00 tx=0x7f0310001b80 comp rx=0 tx=0).stop 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562adfbab2a0 msgr2=0x562adfc37c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 --2- 172.21.15.165:0/1058188338 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x562adfbab2a0 0x562adfc37c90 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f0318064040 tx=0x7f03180033a0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0327400640 1 -- 172.21.15.165:0/1058188338 reap_dead start 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 shutdown_connections 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 >> 172.21.15.165:0/1058188338 conn(0x562adfb90e60 msgr2=0x562adfb5ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 shutdown_connections 2024-04-25T10:19:41.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.225+0000 7f0329758f40 1 -- 172.21.15.165:0/1058188338 wait complete. 2024-04-25T10:19:41.233 INFO:teuthology.orchestra.run.smithi165.stdout:Ysixmegssubdir  2024-04-25T10:19:41.234 DEBUG:teuthology.orchestra.run.smithi165:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2024-04-25T10:19:41.370 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:41.370 INFO:teuthology.orchestra.run.smithi165.stdout: "ino": 1099511627777, 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "ancestors": [ 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "dirino": 1099511627776, 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "dname": "sixmegs", 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "version": 8 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "dirino": 1, 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "dname": "subdir", 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "version": 11 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: ], 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "pool": 26, 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: "old_pools": [] 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:41.371 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:41.386 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:19:41.386 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:41.492 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:41.601 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:19:41.602 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:41.602 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:41.659 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:41.659 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:41.714 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:41.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.915+0000 7fca58800640 1 Processor -- start 2024-04-25T10:19:41.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.915+0000 7fca58800640 1 -- start start 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca58800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca50165680 0x7fca500aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca58800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 0x7fca500b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca58800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca500cf330 con 0x7fca500cc6c0 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca58800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca50072340 con 0x7fca50165680 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca58800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca500b2b90 con 0x7fca500cec70 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca56800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fca500cc6c0 0x7fca500cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58508/0 (socket says 172.21.15.165:58508) 2024-04-25T10:19:41.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.917+0000 7fca56800640 1 -- 172.21.15.165:0/87122157 learned_addr learned my addr 172.21.15.165:0/87122157 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca55e00640 1 --2- 172.21.15.165:0/87122157 >> v2:172.21.15.172:3300/0 conn(0x7fca50165680 0x7fca500aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca55400640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 0x7fca500b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4040606098 0 0) 0x7fca500cf330 con 0x7fca500cc6c0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca38003520 con 0x7fca500cc6c0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 142089094 0 0) 0x7fca38003520 con 0x7fca500cc6c0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 msgr2=0x7fca500b2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 0x7fca500b2650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 >> v2:172.21.15.172:3300/0 conn(0x7fca50165680 msgr2=0x7fca500aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 --2- 172.21.15.165:0/87122157 >> v2:172.21.15.172:3300/0 conn(0x7fca50165680 0x7fca500aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca50071d80 con 0x7fca500cc6c0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fca4c0019f0 con 0x7fca500cc6c0 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 >> v1:172.21.15.165:6789/0 conn(0x7fca500cc6c0 legacy=0x7fca500cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 0x7fca380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38003f30 0x7fca38004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca38004d40 0x7fca380051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50072340 con 0x7fca38004d40 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca500b2b90 con 0x7fca38003f30 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.918+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca3800ba10 con 0x7fca500cec70 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca56800640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca38004d40 0x7fca380051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca55e00640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 0x7fca380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca56800640 1 -- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 msgr2=0x7fca380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca56800640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca500cec70 0x7fca380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca56800640 1 -- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38003f30 msgr2=0x7fca38004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca55400640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38003f30 0x7fca38004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca56800640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38003f30 0x7fca38004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.919+0000 7fca56800640 1 -- 172.21.15.165:0/87122157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca38003520 con 0x7fca38004d40 2024-04-25T10:19:41.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.920+0000 7fca56800640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca38004d40 0x7fca380051c0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00a8b0 tx=0x7fca38005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c84394e47a8c35 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c00a4a0 con 0x7fca38004d40 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c02f070 con 0x7fca38004d40 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca54a00640 1 -- 172.21.15.165:0/87122157 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c02bcf0 con 0x7fca38004d40 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca58800640 1 -- 172.21.15.165:0/87122157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca38004d40 msgr2=0x7fca380051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca58800640 1 --2- 172.21.15.165:0/87122157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca38004d40 0x7fca380051c0 secure :-1 s=READY pgs=2487 cs=0 l=1 rev1=1 crypto rx=0x7fca4c00a8b0 tx=0x7fca38005830 comp rx=0 tx=0).stop 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca56800640 1 -- 172.21.15.165:0/87122157 reap_dead start 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca58800640 1 -- 172.21.15.165:0/87122157 shutdown_connections 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca58800640 1 -- 172.21.15.165:0/87122157 >> 172.21.15.165:0/87122157 conn(0x7fca5006abe0 msgr2=0x7fca5006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca58800640 1 -- 172.21.15.165:0/87122157 shutdown_connections 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.921+0000 7fca58800640 1 -- 172.21.15.165:0/87122157 wait complete. 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.923+0000 7fca58800640 1 Processor -- start 2024-04-25T10:19:41.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.923+0000 7fca58800640 1 -- start start 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.924+0000 7fca58800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38004d40 0x7fca50157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.924+0000 7fca58800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca501f7ae0 0x7fca501f7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.924+0000 7fca58800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 0x7fca500cf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.924+0000 7fca58800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca500cf0d0 con 0x7fca500cf550 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.924+0000 7fca58800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca50071670 con 0x7fca38004d40 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.924+0000 7fca58800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca5006ea30 con 0x7fca501f7ae0 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 0x7fca500cf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 0x7fca500cf9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35670/0 (socket says 172.21.15.165:35670) 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 -- 172.21.15.165:0/793521072 learned_addr learned my addr 172.21.15.165:0/793521072 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:41.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca55e00640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38004d40 0x7fca50157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca55400640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca501f7ae0 0x7fca501f7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 -- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca501f7ae0 msgr2=0x7fca501f7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca501f7ae0 0x7fca501f7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 -- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38004d40 msgr2=0x7fca50157ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38004d40 0x7fca50157ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 -- 172.21.15.165:0/793521072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca501f8610 con 0x7fca500cf550 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca55400640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca501f7ae0 0x7fca501f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:41.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.925+0000 7fca56800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 0x7fca500cf9d0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fca4c02b580 tx=0x7fca5006e5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.926+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c001ff0 con 0x7fca500cf550 2024-04-25T10:19:41.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.926+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca4c02b890 con 0x7fca500cf550 2024-04-25T10:19:41.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.926+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca4c02bc40 con 0x7fca500cf550 2024-04-25T10:19:41.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.926+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca501f88f0 con 0x7fca500cf550 2024-04-25T10:19:41.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.926+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca5015a6a0 con 0x7fca500cf550 2024-04-25T10:19:41.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.927+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca5015a1b0 con 0x7fca500cf550 2024-04-25T10:19:41.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.928+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca4c034470 con 0x7fca500cf550 2024-04-25T10:19:41.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.929+0000 7fca4a000640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca3406b4d0 0x7fca3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:41.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.930+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca4c0aa4a0 con 0x7fca500cf550 2024-04-25T10:19:41.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.930+0000 7fca55e00640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca3406b4d0 0x7fca3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:41.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.930+0000 7fca55e00640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca3406b4d0 0x7fca3406d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fca5015f960 tx=0x7fca44008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:41.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:41.934+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca4c0790f0 con 0x7fca500cf550 2024-04-25T10:19:42.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.129+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fca50073990 con 0x7fca500cf550 2024-04-25T10:19:42.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.130+0000 7fca4a000640 1 -- 172.21.15.165:0/793521072 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16924 (secure 0 0 0) 0x7fca4c0789f0 con 0x7fca500cf550 2024-04-25T10:19:42.135 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:42.135 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":171,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:00.610081+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca3406b4d0 msgr2=0x7fca3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca3406b4d0 0x7fca3406d990 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fca5015f960 tx=0x7fca44008040 comp rx=0 tx=0).stop 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 msgr2=0x7fca500cf9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 0x7fca500cf9d0 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fca4c02b580 tx=0x7fca5006e5d0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 shutdown_connections 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca3406b4d0 0x7fca3406d990 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca500cf550 0x7fca500cf9d0 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca501f7ae0 0x7fca501f7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 --2- 172.21.15.165:0/793521072 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca38004d40 0x7fca50157ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 >> 172.21.15.165:0/793521072 conn(0x7fca5006abe0 msgr2=0x7fca50164890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 shutdown_connections 2024-04-25T10:19:42.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.133+0000 7fca58800640 1 -- 172.21.15.165:0/793521072 wait complete. 2024-04-25T10:19:42.157 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:18:56.874641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:58.066411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:42.157 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:42.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.353+0000 7ff1db400640 1 Processor -- start 2024-04-25T10:19:42.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.353+0000 7ff1db400640 1 -- start start 2024-04-25T10:19:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.354+0000 7ff1db400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff1d40cc4f0 0x7ff1d40cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.354+0000 7ff1db400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff1d40cdb20 0x7ff1d40cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.354+0000 7ff1db400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d4072280 con 0x7ff1d415b270 2024-04-25T10:19:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.354+0000 7ff1db400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d4071270 con 0x7ff1d40cc4f0 2024-04-25T10:19:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.354+0000 7ff1db400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d415bd80 con 0x7ff1d40cdb20 2024-04-25T10:19:42.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff1d40cc4f0 0x7ff1d40cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff1d40cc4f0 0x7ff1d40cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35686/0 (socket says 172.21.15.165:35686) 2024-04-25T10:19:42.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 -- 172.21.15.165:0/2961523380 learned_addr learned my addr 172.21.15.165:0/2961523380 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:42.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 -- 172.21.15.165:0/2961523380 >> v2:172.21.15.172:3300/0 conn(0x7ff1d40cdb20 msgr2=0x7ff1d40cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 --2- 172.21.15.165:0/2961523380 >> v2:172.21.15.172:3300/0 conn(0x7ff1d40cdb20 0x7ff1d40cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 -- 172.21.15.165:0/2961523380 >> v1:172.21.15.172:6789/0 conn(0x7ff1d415b270 legacy=0x7ff1d415b670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.355+0000 7ff1d8a00640 1 -- 172.21.15.165:0/2961523380 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d40714c0 con 0x7ff1d40cc4f0 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.356+0000 7ff1d8a00640 1 --2- 172.21.15.165:0/2961523380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d40cc8d0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4158aa0 tx=0x7ff1c402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edeb6d7c3f17e006 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.356+0000 7ff1d3e00640 1 -- 172.21.15.165:0/2961523380 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4037070 con 0x7ff1d40cc4f0 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.356+0000 7ff1d3e00640 1 -- 172.21.15.165:0/2961523380 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c4032440 con 0x7ff1d40cc4f0 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1d3e00640 1 -- 172.21.15.165:0/2961523380 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4032810 con 0x7ff1d40cc4f0 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 -- 172.21.15.165:0/2961523380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d40cc4f0 msgr2=0x7ff1d40cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 --2- 172.21.15.165:0/2961523380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d40cc8d0 secure :-1 s=READY pgs=2489 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4158aa0 tx=0x7ff1c402a230 comp rx=0 tx=0).stop 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 -- 172.21.15.165:0/2961523380 shutdown_connections 2024-04-25T10:19:42.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 --2- 172.21.15.165:0/2961523380 >> v2:172.21.15.172:3300/0 conn(0x7ff1d40cdb20 0x7ff1d40cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 --2- 172.21.15.165:0/2961523380 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d40cc8d0 unknown :-1 s=CLOSED pgs=2489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 -- 172.21.15.165:0/2961523380 >> 172.21.15.165:0/2961523380 conn(0x7ff1d406abe0 msgr2=0x7ff1d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 -- 172.21.15.165:0/2961523380 shutdown_connections 2024-04-25T10:19:42.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.357+0000 7ff1db400640 1 -- 172.21.15.165:0/2961523380 wait complete. 2024-04-25T10:19:42.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.359+0000 7ff1db400640 1 Processor -- start 2024-04-25T10:19:42.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.359+0000 7ff1db400640 1 -- start start 2024-04-25T10:19:42.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1db400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d414d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1db400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1d40cdb20 0x7ff1d41f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1db400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 0x7ff1d414c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1db400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4071d60 con 0x7ff1d415b270 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1db400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4071a80 con 0x7ff1d40cc4f0 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1db400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40ae500 con 0x7ff1d40cdb20 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 0x7ff1d414c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1d9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 0x7ff1d414c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35692/0 (socket says 172.21.15.165:35692) 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.360+0000 7ff1d9400640 1 -- 172.21.15.165:0/904875049 learned_addr learned my addr 172.21.15.165:0/904875049 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d8a00640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d414d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1cbe00640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1d40cdb20 0x7ff1d41f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d9400640 1 -- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1d40cdb20 msgr2=0x7ff1d41f45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d9400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1d40cdb20 0x7ff1d41f45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d9400640 1 -- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1d40cc4f0 msgr2=0x7ff1d414d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d9400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d414d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d9400640 1 -- 172.21.15.165:0/904875049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d41f7250 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d9400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 0x7ff1d414c920 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c7b0 tx=0x7ff1cc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc017070 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc003da0 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.361+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0052b0 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.362+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d41f7530 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.362+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d41f7af0 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.363+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d40ce960 con 0x7ff1d415b270 2024-04-25T10:19:42.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.364+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff1cc005450 con 0x7ff1d415b270 2024-04-25T10:19:42.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.365+0000 7ff1d1600640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff1b006b4d0 0x7ff1b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.365+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1cc0898e0 con 0x7ff1d415b270 2024-04-25T10:19:42.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.365+0000 7ff1d8a00640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff1b006b4d0 0x7ff1b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.366+0000 7ff1d8a00640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff1b006b4d0 0x7ff1b006d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4034000 tx=0x7ff1c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.369+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff1d40ce960 con 0x7ff1d415b270 2024-04-25T10:19:42.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.564+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff1d4073a00 con 0x7ff1d415b270 2024-04-25T10:19:42.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.565+0000 7ff1d1600640 1 -- 172.21.15.165:0/904875049 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff1cc0585c0 con 0x7ff1d415b270 2024-04-25T10:19:42.568 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff1b006b4d0 msgr2=0x7ff1b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff1b006b4d0 0x7ff1b006d990 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4034000 tx=0x7ff1c402e000 comp rx=0 tx=0).stop 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 msgr2=0x7ff1d414c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 0x7ff1d414c920 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc00c7b0 tx=0x7ff1cc00cc70 comp rx=0 tx=0).stop 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 shutdown_connections 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff1b006b4d0 0x7ff1b006d990 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff1d415b270 0x7ff1d414c920 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff1d40cdb20 0x7ff1d41f45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 --2- 172.21.15.165:0/904875049 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1d40cc4f0 0x7ff1d414d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 >> 172.21.15.165:0/904875049 conn(0x7ff1d406abe0 msgr2=0x7ff1d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 shutdown_connections 2024-04-25T10:19:42.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.567+0000 7ff1db400640 1 -- 172.21.15.165:0/904875049 wait complete. 2024-04-25T10:19:42.589 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:19:42.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.791+0000 7fd06d200640 1 Processor -- start 2024-04-25T10:19:42.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.792+0000 7fd06d200640 1 -- start start 2024-04-25T10:19:42.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd06d200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd06815dd30 0x7fd0680cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd06d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd06806bff0 0x7fd0680b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd06d200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd0680b5970 0x7fd068161820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd06d200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06806d000 con 0x7fd06815dd30 2024-04-25T10:19:42.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd06d200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06815e110 con 0x7fd0680b5970 2024-04-25T10:19:42.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd06d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680b2fd0 con 0x7fd06806bff0 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd067400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd06815dd30 0x7fd0680cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd067400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd06815dd30 0x7fd0680cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35696/0 (socket says 172.21.15.165:35696) 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.793+0000 7fd067400640 1 -- 172.21.15.165:0/3279435096 learned_addr learned my addr 172.21.15.165:0/3279435096 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.794+0000 7fd067400640 1 -- 172.21.15.165:0/3279435096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd06806bff0 msgr2=0x7fd0680b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.794+0000 7fd067400640 1 --2- 172.21.15.165:0/3279435096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd06806bff0 0x7fd0680b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.794+0000 7fd067400640 1 -- 172.21.15.165:0/3279435096 >> v2:172.21.15.172:3300/0 conn(0x7fd0680b5970 msgr2=0x7fd068161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.794+0000 7fd067400640 1 --2- 172.21.15.165:0/3279435096 >> v2:172.21.15.172:3300/0 conn(0x7fd0680b5970 0x7fd068161820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.794+0000 7fd067400640 1 -- 172.21.15.165:0/3279435096 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06806c480 con 0x7fd06815dd30 2024-04-25T10:19:42.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.795+0000 7fd067400640 1 --2- 172.21.15.165:0/3279435096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06815dd30 0x7fd0680cebd0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b6540 tx=0x7fd05802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c26e64aab20dea server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.799+0000 7fd066000640 1 -- 172.21.15.165:0/3279435096 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058037070 con 0x7fd06815dd30 2024-04-25T10:19:42.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.799+0000 7fd066000640 1 -- 172.21.15.165:0/3279435096 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd058032440 con 0x7fd06815dd30 2024-04-25T10:19:42.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.799+0000 7fd066000640 1 -- 172.21.15.165:0/3279435096 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058032810 con 0x7fd06815dd30 2024-04-25T10:19:42.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.799+0000 7fd06d200640 1 -- 172.21.15.165:0/3279435096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06815dd30 msgr2=0x7fd0680cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.799+0000 7fd06d200640 1 --2- 172.21.15.165:0/3279435096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06815dd30 0x7fd0680cebd0 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b6540 tx=0x7fd05802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 -- 172.21.15.165:0/3279435096 shutdown_connections 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 --2- 172.21.15.165:0/3279435096 >> v2:172.21.15.172:3300/0 conn(0x7fd0680b5970 0x7fd068161820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 --2- 172.21.15.165:0/3279435096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd06806bff0 0x7fd0680b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 --2- 172.21.15.165:0/3279435096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd06815dd30 0x7fd0680cebd0 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 -- 172.21.15.165:0/3279435096 >> 172.21.15.165:0/3279435096 conn(0x7fd0680d0610 msgr2=0x7fd0680d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 -- 172.21.15.165:0/3279435096 shutdown_connections 2024-04-25T10:19:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.800+0000 7fd06d200640 1 -- 172.21.15.165:0/3279435096 wait complete. 2024-04-25T10:19:42.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.801+0000 7fd06d200640 1 Processor -- start 2024-04-25T10:19:42.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.802+0000 7fd06d200640 1 -- start start 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.802+0000 7fd06d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 0x7fd06814def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd06d200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd06815dd30 0x7fd068160920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd06d200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0680b8280 0x7fd0680b8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd06d200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06806cd20 con 0x7fd0680b5970 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd06d200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06806ca40 con 0x7fd06815dd30 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd06d200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0680cf110 con 0x7fd0680b8280 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 0x7fd06814def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 0x7fd06814def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35708/0 (socket says 172.21.15.165:35708) 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 -- 172.21.15.165:0/2236695425 learned_addr learned my addr 172.21.15.165:0/2236695425 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 -- 172.21.15.165:0/2236695425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0680b8280 msgr2=0x7fd0680b8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0680b8280 0x7fd0680b8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 -- 172.21.15.165:0/2236695425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd06815dd30 msgr2=0x7fd068160920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:42.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd06815dd30 0x7fd068160920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:42.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 -- 172.21.15.165:0/2236695425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06815f880 con 0x7fd0680b5970 2024-04-25T10:19:42.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.803+0000 7fd067400640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 0x7fd06814def0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd0580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.804+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058041070 con 0x7fd0680b5970 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.804+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd05803c870 con 0x7fd0680b5970 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.804+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd058037040 con 0x7fd0680b5970 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.804+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06815fb60 con 0x7fd0680b5970 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.804+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd068160120 con 0x7fd0680b5970 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.805+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd068073990 con 0x7fd0680b5970 2024-04-25T10:19:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.806+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd058044070 con 0x7fd0680b5970 2024-04-25T10:19:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.807+0000 7fd05f400640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04806b4d0 0x7fd04806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.807+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd0580ab010 con 0x7fd0680b5970 2024-04-25T10:19:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.807+0000 7fd066a00640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04806b4d0 0x7fd04806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.808+0000 7fd066a00640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04806b4d0 0x7fd04806d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b91d0 tx=0x7fd054006c90 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:42.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:42.811+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd058079cf0 con 0x7fd0680b5970 2024-04-25T10:19:43.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.005+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd0680d16c0 con 0x7fd0680b5970 2024-04-25T10:19:43.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.006+0000 7fd05f400640 1 -- 172.21.15.165:0/2236695425 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd0580795f0 con 0x7fd0680b5970 2024-04-25T10:19:43.010 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04806b4d0 msgr2=0x7fd04806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04806b4d0 0x7fd04806d990 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fd0680b91d0 tx=0x7fd054006c90 comp rx=0 tx=0).stop 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 msgr2=0x7fd06814def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 0x7fd06814def0 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fd058000c00 tx=0x7fd0580046c0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 shutdown_connections 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd04806b4d0 0x7fd04806d990 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd0680b8280 0x7fd0680b8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd06815dd30 0x7fd068160920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 --2- 172.21.15.165:0/2236695425 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd0680b5970 0x7fd06814def0 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 >> 172.21.15.165:0/2236695425 conn(0x7fd0680d0610 msgr2=0x7fd0680b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 shutdown_connections 2024-04-25T10:19:43.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.009+0000 7fd06d200640 1 -- 172.21.15.165:0/2236695425 wait complete. 2024-04-25T10:19:43.032 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:19:43.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.234+0000 7f16acc00640 1 Processor -- start 2024-04-25T10:19:43.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.235+0000 7f16acc00640 1 -- start start 2024-04-25T10:19:43.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.236+0000 7f16acc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f16a8161fc0 0x7f16a81623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.236+0000 7f16acc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a806d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.236+0000 7f16acc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a81628e0 con 0x7f16a806ea90 2024-04-25T10:19:43.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.236+0000 7f16acc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a8071b90 con 0x7f16a8161fc0 2024-04-25T10:19:43.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.236+0000 7f16acc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a806dc80 con 0x7f16a806d2d0 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f16a8161fc0 0x7f16a81623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f16a8161fc0 0x7f16a81623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35712/0 (socket says 172.21.15.165:35712) 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 -- 172.21.15.165:0/2400545006 learned_addr learned my addr 172.21.15.165:0/2400545006 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a6800640 1 --2- 172.21.15.165:0/2400545006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a806d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 -- 172.21.15.165:0/2400545006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 msgr2=0x7f16a806d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 --2- 172.21.15.165:0/2400545006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a806d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 -- 172.21.15.165:0/2400545006 >> v1:172.21.15.172:6789/0 conn(0x7f16a806ea90 legacy=0x7f16a8071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.237+0000 7f16a5e00640 1 -- 172.21.15.165:0/2400545006 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a8162ba0 con 0x7f16a8161fc0 2024-04-25T10:19:43.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.238+0000 7f16a5e00640 1 --2- 172.21.15.165:0/2400545006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a8161fc0 0x7f16a81623a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f16a806ba40 tx=0x7f169802a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f91c3317f04fde99 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.238+0000 7f16a5400640 1 -- 172.21.15.165:0/2400545006 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698037070 con 0x7f16a8161fc0 2024-04-25T10:19:43.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.238+0000 7f16a5400640 1 -- 172.21.15.165:0/2400545006 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169802ae50 con 0x7f16a8161fc0 2024-04-25T10:19:43.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.238+0000 7f16a5400640 1 -- 172.21.15.165:0/2400545006 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1698032660 con 0x7f16a8161fc0 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 -- 172.21.15.165:0/2400545006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a8161fc0 msgr2=0x7f16a81623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 --2- 172.21.15.165:0/2400545006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a8161fc0 0x7f16a81623a0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f16a806ba40 tx=0x7f169802a140 comp rx=0 tx=0).stop 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 -- 172.21.15.165:0/2400545006 shutdown_connections 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 --2- 172.21.15.165:0/2400545006 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a806d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 --2- 172.21.15.165:0/2400545006 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a8161fc0 0x7f16a81623a0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 -- 172.21.15.165:0/2400545006 >> 172.21.15.165:0/2400545006 conn(0x7f16a806abe0 msgr2=0x7f16a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 -- 172.21.15.165:0/2400545006 shutdown_connections 2024-04-25T10:19:43.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.239+0000 7f16acc00640 1 -- 172.21.15.165:0/2400545006 wait complete. 2024-04-25T10:19:43.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.241+0000 7f16acc00640 1 Processor -- start 2024-04-25T10:19:43.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.241+0000 7f16acc00640 1 -- start start 2024-04-25T10:19:43.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f16acc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a8154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f16acc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 0x7f16a81f7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f16acc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8161fc0 0x7f16a80b3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f16acc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8163160 con 0x7f16a806ea90 2024-04-25T10:19:43.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f16acc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a8163440 con 0x7f16a8161fc0 2024-04-25T10:19:43.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f16acc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a806ef40 con 0x7f16a806d2d0 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f169fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 0x7f16a81f7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f169fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 0x7f16a81f7da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35716/0 (socket says 172.21.15.165:35716) 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.242+0000 7f169fe00640 1 -- 172.21.15.165:0/2938247569 learned_addr learned my addr 172.21.15.165:0/2938247569 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f16a5e00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a8154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169fe00640 1 -- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 msgr2=0x7f16a8154e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169fe00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a8154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169fe00640 1 -- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8161fc0 msgr2=0x7f16a80b3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f16a6800640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8161fc0 0x7f16a80b3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169fe00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8161fc0 0x7f16a80b3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169fe00640 1 -- 172.21.15.165:0/2938247569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a81f8840 con 0x7f16a806ea90 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f16a6800640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8161fc0 0x7f16a80b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169fe00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 0x7f16a81f7da0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f169000c430 tx=0x7f169000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.243+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690011790 con 0x7f16a806ea90 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.244+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1690017070 con 0x7f16a806ea90 2024-04-25T10:19:43.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.244+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1690011bb0 con 0x7f16a806ea90 2024-04-25T10:19:43.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.244+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a81f8b20 con 0x7f16a806ea90 2024-04-25T10:19:43.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.244+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a81553c0 con 0x7f16a806ea90 2024-04-25T10:19:43.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.245+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a81547b0 con 0x7f16a806ea90 2024-04-25T10:19:43.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.246+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f169001c460 con 0x7f16a806ea90 2024-04-25T10:19:43.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.247+0000 7f169e000640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f168406b4d0 0x7f168406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.247+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f169008a290 con 0x7f16a806ea90 2024-04-25T10:19:43.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.247+0000 7f16a5e00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f168406b4d0 0x7f168406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.248+0000 7f16a5e00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f168406b4d0 0x7f168406d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1698034000 tx=0x7f169802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.251+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1690058f70 con 0x7f16a806ea90 2024-04-25T10:19:43.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.445+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f16a8073a00 con 0x7f16a806ea90 2024-04-25T10:19:43.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.446+0000 7f169e000640 1 -- 172.21.15.165:0/2938247569 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1690058870 con 0x7f16a806ea90 2024-04-25T10:19:43.450 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:19:43.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.448+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f168406b4d0 msgr2=0x7f168406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f168406b4d0 0x7f168406d990 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f1698034000 tx=0x7f169802e000 comp rx=0 tx=0).stop 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 msgr2=0x7f16a81f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 0x7f16a81f7da0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f169000c430 tx=0x7f169000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 shutdown_connections 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f168406b4d0 0x7f168406d990 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16a8161fc0 0x7f16a80b3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16a806ea90 0x7f16a81f7da0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 --2- 172.21.15.165:0/2938247569 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16a806d2d0 0x7f16a8154e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 >> 172.21.15.165:0/2938247569 conn(0x7f16a806abe0 msgr2=0x7f16a80b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 shutdown_connections 2024-04-25T10:19:43.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.449+0000 7f16acc00640 1 -- 172.21.15.165:0/2938247569 wait complete. 2024-04-25T10:19:43.471 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:19:43.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.669+0000 7f2f4ce00640 1 Processor -- start 2024-04-25T10:19:43.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.669+0000 7f2f4ce00640 1 -- start start 2024-04-25T10:19:43.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f4ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 0x7f2f4806b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f4ce00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2f480b4040 0x7f2f480b44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f4ce00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f480adf30 con 0x7f2f4815b8a0 2024-04-25T10:19:43.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f4ce00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f480ad970 con 0x7f2f480b4040 2024-04-25T10:19:43.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f4ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480adc50 con 0x7f2f480af520 2024-04-25T10:19:43.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f45e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2f4815b8a0 0x7f2f4806abe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58518/0 (socket says 172.21.15.165:58518) 2024-04-25T10:19:43.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f45e00640 1 -- 172.21.15.165:0/3941354687 learned_addr learned my addr 172.21.15.165:0/3941354687 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.671+0000 7f2f45400640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 0x7f2f4806b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f46800640 1 --2- 172.21.15.165:0/3941354687 >> v2:172.21.15.172:3300/0 conn(0x7f2f480b4040 0x7f2f480b44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220176677 0 0) 0x7f2f38001110 con 0x7f2f4815b8a0 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f2c003520 con 0x7f2f4815b8a0 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 405403176 0 0) 0x7f2f2c003520 con 0x7f2f4815b8a0 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 msgr2=0x7f2f4806b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 0x7f2f4806b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 >> v2:172.21.15.172:3300/0 conn(0x7f2f480b4040 msgr2=0x7f2f480b44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 --2- 172.21.15.165:0/3941354687 >> v2:172.21.15.172:3300/0 conn(0x7f2f480b4040 0x7f2f480b44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480b3840 con 0x7f2f4815b8a0 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2f38001840 con 0x7f2f4815b8a0 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 >> v1:172.21.15.165:6789/0 conn(0x7f2f4815b8a0 legacy=0x7f2f4806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f2c0039b0 0x7f2f2c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 0x7f2f2c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f2c004d40 0x7f2f2c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480ad970 con 0x7f2f2c0039b0 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f480adc50 con 0x7f2f2c004d40 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.672+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c00b9c0 con 0x7f2f480af520 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45e00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f2c0039b0 0x7f2f2c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45400640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 0x7f2f2c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f46800640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f2c004d40 0x7f2f2c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45e00640 1 -- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 msgr2=0x7f2f2c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45e00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f480af520 0x7f2f2c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45e00640 1 -- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f2c004d40 msgr2=0x7f2f2c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:43.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45e00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f2c004d40 0x7f2f2c0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.673+0000 7f2f45e00640 1 -- 172.21.15.165:0/3941354687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c003520 con 0x7f2f2c0039b0 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.674+0000 7f2f45e00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f2c0039b0 0x7f2f2c003dd0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c4c0 tx=0x7f2f2c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649a673897756f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.674+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3800c0e0 con 0x7f2f2c0039b0 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.674+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f38034070 con 0x7f2f2c0039b0 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.674+0000 7f2f44a00640 1 -- 172.21.15.165:0/3941354687 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f38005d30 con 0x7f2f2c0039b0 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f4ce00640 1 -- 172.21.15.165:0/3941354687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f2c0039b0 msgr2=0x7f2f2c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/3941354687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f2c0039b0 0x7f2f2c003dd0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f2f3800c4c0 tx=0x7f2f2c005830 comp rx=0 tx=0).stop 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f46800640 1 -- 172.21.15.165:0/3941354687 reap_dead start 2024-04-25T10:19:43.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f4ce00640 1 -- 172.21.15.165:0/3941354687 shutdown_connections 2024-04-25T10:19:43.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f4ce00640 1 -- 172.21.15.165:0/3941354687 >> 172.21.15.165:0/3941354687 conn(0x7f2f480cfec0 msgr2=0x7f2f480d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f4ce00640 1 -- 172.21.15.165:0/3941354687 shutdown_connections 2024-04-25T10:19:43.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.675+0000 7f2f4ce00640 1 -- 172.21.15.165:0/3941354687 wait complete. 2024-04-25T10:19:43.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.677+0000 7f2f4ce00640 1 Processor -- start 2024-04-25T10:19:43.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.677+0000 7f2f4ce00640 1 -- start start 2024-04-25T10:19:43.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f4ce00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f2c0039b0 0x7f2f481f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f4ce00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f48152330 0x7f2f48154df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f4ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 0x7f2f481484c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f4ce00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815be00 con 0x7f2f480afb70 2024-04-25T10:19:43.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f4ce00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4815bc80 con 0x7f2f48152330 2024-04-25T10:19:43.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f4ce00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4806b2f0 con 0x7f2f2c0039b0 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f46800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 0x7f2f481484c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f46800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 0x7f2f481484c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35724/0 (socket says 172.21.15.165:35724) 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.678+0000 7f2f46800640 1 -- 172.21.15.165:0/1764070860 learned_addr learned my addr 172.21.15.165:0/1764070860 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f45e00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f2c0039b0 0x7f2f481f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f45400640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f48152330 0x7f2f48154df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f46800640 1 -- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f2c0039b0 msgr2=0x7f2f481f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f46800640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f2c0039b0 0x7f2f481f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f46800640 1 -- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f48152330 msgr2=0x7f2f48154df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f46800640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f48152330 0x7f2f48154df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f46800640 1 -- 172.21.15.165:0/1764070860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f48151d50 con 0x7f2f480afb70 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f46800640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 0x7f2f481484c0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f2f480d0de0 tx=0x7f2f4000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40016620 con 0x7f2f480afb70 2024-04-25T10:19:43.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f4001b070 con 0x7f2f480afb70 2024-04-25T10:19:43.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f40016a20 con 0x7f2f480afb70 2024-04-25T10:19:43.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.679+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f48152830 con 0x7f2f480afb70 2024-04-25T10:19:43.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.680+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f48152df0 con 0x7f2f480afb70 2024-04-25T10:19:43.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.681+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f48073a00 con 0x7f2f480afb70 2024-04-25T10:19:43.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.682+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2f40005d10 con 0x7f2f480afb70 2024-04-25T10:19:43.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.683+0000 7f2f3e000640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b500 0x7f2f2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:43.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.683+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f2f4008ad00 con 0x7f2f480afb70 2024-04-25T10:19:43.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.683+0000 7f2f45e00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b500 0x7f2f2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:43.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.684+0000 7f2f45e00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b500 0x7f2f2806d9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2f38033000 tx=0x7f2f3802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:43.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.688+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2f48151d50 con 0x7f2f480afb70 2024-04-25T10:19:43.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.901+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2f4815f340 con 0x7f2f480afb70 2024-04-25T10:19:43.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.902+0000 7f2f3e000640 1 -- 172.21.15.165:0/1764070860 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v171) v1 ==== 80+0+0 (secure 0 0 0) 0x7f2f400599e0 con 0x7f2f480afb70 2024-04-25T10:19:43.906 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:19:43.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b500 msgr2=0x7f2f2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b500 0x7f2f2806d9c0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2f38033000 tx=0x7f2f3802e000 comp rx=0 tx=0).stop 2024-04-25T10:19:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 msgr2=0x7f2f481484c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 0x7f2f481484c0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f2f480d0de0 tx=0x7f2f4000f520 comp rx=0 tx=0).stop 2024-04-25T10:19:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 shutdown_connections 2024-04-25T10:19:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2f2806b500 0x7f2f2806d9c0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2f480afb70 0x7f2f481484c0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2f48152330 0x7f2f48154df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 --2- 172.21.15.165:0/1764070860 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2f2c0039b0 0x7f2f481f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 >> 172.21.15.165:0/1764070860 conn(0x7f2f480cfec0 msgr2=0x7f2f480d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 shutdown_connections 2024-04-25T10:19:43.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:43.905+0000 7f2f4ce00640 1 -- 172.21.15.165:0/1764070860 wait complete. 2024-04-25T10:19:43.928 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:19:43.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:43.928 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:19:44.039 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:19:44.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:19:44.039 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:19:44.102 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:19:44.102 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:44.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:44.143 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:44.144 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:44.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:44.182 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:19:44.182 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:44.234 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:44.234 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:44.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.423+0000 7fc71e000640 1 Processor -- start 2024-04-25T10:19:44.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.423+0000 7fc71e000640 1 -- start start 2024-04-25T10:19:44.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.425+0000 7fc71e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc71806c1d0 0x7fc718168930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.425+0000 7fc71e000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7180adcf0 con 0x7fc718071ac0 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.425+0000 7fc71e000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc71815ec60 con 0x7fc718154ed0 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.425+0000 7fc71e000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc71815ef40 con 0x7fc71806c1d0 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.425+0000 7fc716a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc718071ac0 0x7fc7180ad5e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58526/0 (socket says 172.21.15.165:58526) 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.425+0000 7fc716a00640 1 -- 172.21.15.165:0/305069674 learned_addr learned my addr 172.21.15.165:0/305069674 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc717e00640 1 --2- 172.21.15.165:0/305069674 >> v2:172.21.15.165:3300/0 conn(0x7fc71806c1d0 0x7fc718168930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc716000640 1 -- 172.21.15.165:0/305069674 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715415180 0 0) 0x7fc7180adcf0 con 0x7fc718071ac0 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc716000640 1 -- 172.21.15.165:0/305069674 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc6fc003520 con 0x7fc718071ac0 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc717e00640 1 -- 172.21.15.165:0/305069674 >> v1:172.21.15.172:6789/0 conn(0x7fc718154ed0 legacy=0x7fc7180aced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc717e00640 1 -- 172.21.15.165:0/305069674 >> v1:172.21.15.165:6789/0 conn(0x7fc718071ac0 legacy=0x7fc7180ad5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc716000640 1 -- 172.21.15.165:0/305069674 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3578749871 0 0) 0x7fc71815ec60 con 0x7fc718154ed0 2024-04-25T10:19:44.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.426+0000 7fc717e00640 1 -- 172.21.15.165:0/305069674 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71806db90 con 0x7fc71806c1d0 2024-04-25T10:19:44.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.428+0000 7fc717e00640 1 --2- 172.21.15.165:0/305069674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc71806c1d0 0x7fc718168930 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fc7100097a0 tx=0x7fc71002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2f6f4a97ad5f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:44.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.428+0000 7fc716000640 1 -- 172.21.15.165:0/305069674 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710037070 con 0x7fc71806c1d0 2024-04-25T10:19:44.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.428+0000 7fc716000640 1 -- 172.21.15.165:0/305069674 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc710032440 con 0x7fc71806c1d0 2024-04-25T10:19:44.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.428+0000 7fc716000640 1 -- 172.21.15.165:0/305069674 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc710032810 con 0x7fc71806c1d0 2024-04-25T10:19:44.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 -- 172.21.15.165:0/305069674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc71806c1d0 msgr2=0x7fc718168930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 --2- 172.21.15.165:0/305069674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc71806c1d0 0x7fc718168930 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7fc7100097a0 tx=0x7fc71002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 -- 172.21.15.165:0/305069674 shutdown_connections 2024-04-25T10:19:44.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 --2- 172.21.15.165:0/305069674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc71806c1d0 0x7fc718168930 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 -- 172.21.15.165:0/305069674 >> 172.21.15.165:0/305069674 conn(0x7fc7180af670 msgr2=0x7fc7180b1bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:44.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 -- 172.21.15.165:0/305069674 shutdown_connections 2024-04-25T10:19:44.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.429+0000 7fc71e000640 1 -- 172.21.15.165:0/305069674 wait complete. 2024-04-25T10:19:44.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.431+0000 7fc71e000640 1 Processor -- start 2024-04-25T10:19:44.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.431+0000 7fc71e000640 1 -- start start 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc71e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 0x7fc7181489c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc71e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc718154ed0 0x7fc7180cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc71e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7180cc7d0 0x7fc71815f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc71e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc718168ff0 con 0x7fc718071ac0 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc71e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc718168e70 con 0x7fc718154ed0 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc71e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc718169170 con 0x7fc7180cc7d0 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc717400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 0x7fc7181489c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc717400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 0x7fc7181489c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35744/0 (socket says 172.21.15.165:35744) 2024-04-25T10:19:44.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc717400640 1 -- 172.21.15.165:0/1629189616 learned_addr learned my addr 172.21.15.165:0/1629189616 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:44.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.432+0000 7fc716a00640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc718154ed0 0x7fc7180cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc717400640 1 -- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7180cc7d0 msgr2=0x7fc71815f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc717400640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7180cc7d0 0x7fc71815f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc717400640 1 -- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc718154ed0 msgr2=0x7fc7180cc290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc717400640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc718154ed0 0x7fc7180cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc717400640 1 -- 172.21.15.165:0/1629189616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71814b4a0 con 0x7fc718071ac0 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc717400640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 0x7fc7181489c0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c820 tx=0x7fc70800cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708017070 con 0x7fc718071ac0 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc708003da0 con 0x7fc718071ac0 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71814a6e0 con 0x7fc718071ac0 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc708005290 con 0x7fc718071ac0 2024-04-25T10:19:44.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.433+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71814aca0 con 0x7fc718071ac0 2024-04-25T10:19:44.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.434+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7180b2b10 con 0x7fc718071ac0 2024-04-25T10:19:44.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.435+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc70801a070 con 0x7fc718071ac0 2024-04-25T10:19:44.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.436+0000 7fc70f400640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6f806b520 0x7fc6f806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.436+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc708089a10 con 0x7fc718071ac0 2024-04-25T10:19:44.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.436+0000 7fc716a00640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6f806b520 0x7fc6f806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.437+0000 7fc716a00640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6f806b520 0x7fc6f806d9e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc71814d060 tx=0x7fc704009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:44.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.441+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc708058770 con 0x7fc718071ac0 2024-04-25T10:19:44.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.678+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7180b5140 con 0x7fc718071ac0 2024-04-25T10:19:44.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.679+0000 7fc70f400640 1 -- 172.21.15.165:0/1629189616 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fc708058070 con 0x7fc718071ac0 2024-04-25T10:19:44.683 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:44.683 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:44.683 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6f806b520 msgr2=0x7fc6f806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6f806b520 0x7fc6f806d9e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7fc71814d060 tx=0x7fc704009040 comp rx=0 tx=0).stop 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 msgr2=0x7fc7181489c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 0x7fc7181489c0 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7fc70800c820 tx=0x7fc70800cce0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 shutdown_connections 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc6f806b520 0x7fc6f806d9e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7180cc7d0 0x7fc71815f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc718154ed0 0x7fc7180cc290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 --2- 172.21.15.165:0/1629189616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc718071ac0 0x7fc7181489c0 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 >> 172.21.15.165:0/1629189616 conn(0x7fc71806c1d0 msgr2=0x7fc718160cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:44.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 shutdown_connections 2024-04-25T10:19:44.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.682+0000 7fc71e000640 1 -- 172.21.15.165:0/1629189616 wait complete. 2024-04-25T10:19:44.704 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:44.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.900+0000 7f6801400640 1 Processor -- start 2024-04-25T10:19:44.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.900+0000 7f6801400640 1 -- start start 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f6801400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f67fc144750 0x7f67fc06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f6801400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67fc06eb10 0x7f67fc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f6801400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67fc144b30 con 0x7f67fc0cee60 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f6801400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc0712d0 con 0x7f67fc06eb10 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f6801400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67fc0715b0 con 0x7f67fc144750 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f67f9e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f67fc06eb10 0x7f67fc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f67fb200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f67fc0cee60 0x7f67fc15f610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58540/0 (socket says 172.21.15.165:58540) 2024-04-25T10:19:44.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f67fb200640 1 -- 172.21.15.165:0/1325163340 learned_addr learned my addr 172.21.15.165:0/1325163340 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.902+0000 7f67fa800640 1 --2- 172.21.15.165:0/1325163340 >> v2:172.21.15.172:3300/0 conn(0x7f67fc144750 0x7f67fc06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.903+0000 7f67f9e00640 1 -- 172.21.15.165:0/1325163340 >> v2:172.21.15.172:3300/0 conn(0x7f67fc144750 msgr2=0x7f67fc06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.903+0000 7f67f9e00640 1 --2- 172.21.15.165:0/1325163340 >> v2:172.21.15.172:3300/0 conn(0x7f67fc144750 0x7f67fc06e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.903+0000 7f67f9e00640 1 -- 172.21.15.165:0/1325163340 >> v1:172.21.15.165:6789/0 conn(0x7f67fc0cee60 legacy=0x7f67fc15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.903+0000 7f67f9e00640 1 -- 172.21.15.165:0/1325163340 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc071890 con 0x7f67fc06eb10 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.903+0000 7f67f9400640 1 -- 172.21.15.165:0/1325163340 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789344601 0 0) 0x7f67fc144b30 con 0x7f67fc0cee60 2024-04-25T10:19:44.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.904+0000 7f67f9e00640 1 --2- 172.21.15.165:0/1325163340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc0ce920 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f67fc06f5d0 tx=0x7f67e802a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d8fcaa7c05a45d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.904+0000 7f67f9400640 1 -- 172.21.15.165:0/1325163340 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8037070 con 0x7f67fc06eb10 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.904+0000 7f67f9400640 1 -- 172.21.15.165:0/1325163340 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67e8032440 con 0x7f67fc06eb10 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.904+0000 7f67f9400640 1 -- 172.21.15.165:0/1325163340 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e8032810 con 0x7f67fc06eb10 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 -- 172.21.15.165:0/1325163340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 msgr2=0x7f67fc0ce920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 --2- 172.21.15.165:0/1325163340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc0ce920 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f67fc06f5d0 tx=0x7f67e802a1a0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 -- 172.21.15.165:0/1325163340 shutdown_connections 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 --2- 172.21.15.165:0/1325163340 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc0ce920 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 --2- 172.21.15.165:0/1325163340 >> v2:172.21.15.172:3300/0 conn(0x7f67fc144750 0x7f67fc06e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 -- 172.21.15.165:0/1325163340 >> 172.21.15.165:0/1325163340 conn(0x7f67fc06abe0 msgr2=0x7f67fc06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 -- 172.21.15.165:0/1325163340 shutdown_connections 2024-04-25T10:19:44.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.905+0000 7f6801400640 1 -- 172.21.15.165:0/1325163340 wait complete. 2024-04-25T10:19:44.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.907+0000 7f6801400640 1 Processor -- start 2024-04-25T10:19:44.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.907+0000 7f6801400640 1 -- start start 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f6801400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc1e5f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f6801400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67fc0cee60 0x7f67fc159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f6801400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67fc144750 0x7f67fc1f4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f6801400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc145450 con 0x7f67fc06eb10 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f6801400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc06f020 con 0x7f67fc0cee60 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f6801400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67fc1455d0 con 0x7f67fc144750 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc1e5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc1e5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35760/0 (socket says 172.21.15.165:35760) 2024-04-25T10:19:44.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 -- 172.21.15.165:0/1987910885 learned_addr learned my addr 172.21.15.165:0/1987910885 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:44.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 -- 172.21.15.165:0/1987910885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67fc144750 msgr2=0x7f67fc1f4020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67fc144750 0x7f67fc1f4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 -- 172.21.15.165:0/1987910885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67fc0cee60 msgr2=0x7f67fc159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:44.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67fc0cee60 0x7f67fc159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:44.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.908+0000 7f67fa800640 1 -- 172.21.15.165:0/1987910885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67fc13e000 con 0x7f67fc06eb10 2024-04-25T10:19:44.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.909+0000 7f67fa800640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc1e5f90 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00d800 tx=0x7f67ec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:44.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.909+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec00b6c0 con 0x7f67fc06eb10 2024-04-25T10:19:44.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.909+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67ec019070 con 0x7f67fc06eb10 2024-04-25T10:19:44.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.909+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67ec00ba80 con 0x7f67fc06eb10 2024-04-25T10:19:44.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.909+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67fc1f4c00 con 0x7f67fc06eb10 2024-04-25T10:19:44.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.909+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67fc1e5790 con 0x7f67fc06eb10 2024-04-25T10:19:44.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.910+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67fc073990 con 0x7f67fc06eb10 2024-04-25T10:19:44.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.911+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f67ec00bc20 con 0x7f67fc06eb10 2024-04-25T10:19:44.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.912+0000 7f67f2a00640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67dc06b4d0 0x7f67dc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:44.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.912+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f67ec08a850 con 0x7f67fc06eb10 2024-04-25T10:19:44.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.912+0000 7f67f9e00640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67dc06b4d0 0x7f67dc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:44.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.913+0000 7f67f9e00640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67dc06b4d0 0x7f67dc06d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f67e8034000 tx=0x7f67e802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:44.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:44.916+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f67ec059530 con 0x7f67fc06eb10 2024-04-25T10:19:45.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.111+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67fc141b00 con 0x7f67fc06eb10 2024-04-25T10:19:45.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.112+0000 7f67f2a00640 1 -- 172.21.15.165:0/1987910885 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16924 (secure 0 0 0) 0x7f67ec058e30 con 0x7f67fc06eb10 2024-04-25T10:19:45.116 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:45.117 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":171,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:00.610081+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.115+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67dc06b4d0 msgr2=0x7f67dc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.115+0000 7f6801400640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67dc06b4d0 0x7f67dc06d990 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f67e8034000 tx=0x7f67e802c000 comp rx=0 tx=0).stop 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.115+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 msgr2=0x7f67fc1e5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.115+0000 7f6801400640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc1e5f90 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f67ec00d800 tx=0x7f67ec00dcc0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 shutdown_connections 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f67dc06b4d0 0x7f67dc06d990 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f67fc144750 0x7f67fc1f4020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f67fc0cee60 0x7f67fc159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 --2- 172.21.15.165:0/1987910885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f67fc06eb10 0x7f67fc1e5f90 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 >> 172.21.15.165:0/1987910885 conn(0x7f67fc06abe0 msgr2=0x7f67fc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:45.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 shutdown_connections 2024-04-25T10:19:45.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.116+0000 7f6801400640 1 -- 172.21.15.165:0/1987910885 wait complete. 2024-04-25T10:19:45.141 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:18:56.874641+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:18:58.066411+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '171', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:45.141 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.344+0000 7f3fe9200640 1 Processor -- start 2024-04-25T10:19:45.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.345+0000 7f3fe9200640 1 -- start start 2024-04-25T10:19:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.346+0000 7f3fe9200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3fe4161fc0 0x7f3fe41623a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.346+0000 7f3fe9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe4071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.346+0000 7f3fe9200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fe41628e0 con 0x7f3fe406ef10 2024-04-25T10:19:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.346+0000 7f3fe9200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe4165be0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.346+0000 7f3fe9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe4162ba0 con 0x7f3fe406ea90 2024-04-25T10:19:45.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3fe4161fc0 0x7f3fe41623a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe2800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3fe406ef10 0x7f3fe406d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58546/0 (socket says 172.21.15.165:58546) 2024-04-25T10:19:45.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe2800640 1 -- 172.21.15.165:0/1348527245 learned_addr learned my addr 172.21.15.165:0/1348527245 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3fe4161fc0 0x7f3fe41623a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35766/0 (socket says 172.21.15.165:35766) 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fdbe00640 1 --2- 172.21.15.165:0/1348527245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe4071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1e00640 1 -- 172.21.15.165:0/1348527245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 msgr2=0x7f3fe4071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1e00640 1 --2- 172.21.15.165:0/1348527245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe4071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1e00640 1 -- 172.21.15.165:0/1348527245 >> v1:172.21.15.165:6789/0 conn(0x7f3fe406ef10 legacy=0x7f3fe406d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1e00640 1 -- 172.21.15.165:0/1348527245 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4162e80 con 0x7f3fe4161fc0 2024-04-25T10:19:45.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.347+0000 7f3fe1400640 1 -- 172.21.15.165:0/1348527245 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930386987 0 0) 0x7f3fe41628e0 con 0x7f3fe406ef10 2024-04-25T10:19:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.348+0000 7f3fe1e00640 1 --2- 172.21.15.165:0/1348527245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe41623a0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097a0 tx=0x7f3fd402a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b95a981af00f3f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:45.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe1400640 1 -- 172.21.15.165:0/1348527245 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4037070 con 0x7f3fe4161fc0 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe1400640 1 -- 172.21.15.165:0/1348527245 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd402ae50 con 0x7f3fe4161fc0 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe1400640 1 -- 172.21.15.165:0/1348527245 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4032660 con 0x7f3fe4161fc0 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe9200640 1 -- 172.21.15.165:0/1348527245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 msgr2=0x7f3fe41623a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1348527245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe41623a0 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7f3fd40097a0 tx=0x7f3fd402a100 comp rx=0 tx=0).stop 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe9200640 1 -- 172.21.15.165:0/1348527245 shutdown_connections 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1348527245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe4071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1348527245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe41623a0 unknown :-1 s=CLOSED pgs=2505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.349+0000 7f3fe9200640 1 -- 172.21.15.165:0/1348527245 >> 172.21.15.165:0/1348527245 conn(0x7f3fe406abe0 msgr2=0x7f3fe406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.350+0000 7f3fe9200640 1 -- 172.21.15.165:0/1348527245 shutdown_connections 2024-04-25T10:19:45.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.350+0000 7f3fe9200640 1 -- 172.21.15.165:0/1348527245 wait complete. 2024-04-25T10:19:45.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.351+0000 7f3fe9200640 1 Processor -- start 2024-04-25T10:19:45.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.352+0000 7f3fe9200640 1 -- start start 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.352+0000 7f3fe9200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe415f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.352+0000 7f3fe9200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fe406ef10 0x7f3fe40b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe4148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe9200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe40719c0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe9200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe4163440 con 0x7f3fe406ef10 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe9200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe4071b40 con 0x7f3fe406ea90 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe4148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe4148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35774/0 (socket says 172.21.15.165:35774) 2024-04-25T10:19:45.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 -- 172.21.15.165:0/1961454828 learned_addr learned my addr 172.21.15.165:0/1961454828 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fdbe00640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fe406ef10 0x7f3fe40b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe1e00640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe415f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 -- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 msgr2=0x7f3fe415f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe415f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 -- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fe406ef10 msgr2=0x7f3fe40b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fe406ef10 0x7f3fe40b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.353+0000 7f3fe2800640 1 -- 172.21.15.165:0/1961454828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe4148e90 con 0x7f3fe4161fc0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.354+0000 7f3fe2800640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe4148950 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc001560 tx=0x7f3fdc00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.354+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc018070 con 0x7f3fe4161fc0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.354+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fdc00a640 con 0x7f3fe4161fc0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.354+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fdc00aa10 con 0x7f3fe4161fc0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.354+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe415e810 con 0x7f3fe4161fc0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.354+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe415edd0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.355+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe4073a00 con 0x7f3fe4161fc0 2024-04-25T10:19:45.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.356+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3fdc00abb0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.357+0000 7f3fda000640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.357+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3fdc0899e0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.357+0000 7f3fe1e00640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.358+0000 7f3fe1e00640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3fc006b4d0 0x7f3fc006d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4034000 tx=0x7f3fd402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:45.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.361+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3fe4148e90 con 0x7f3fe4161fc0 2024-04-25T10:19:45.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.556+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3fe40b07b0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.557+0000 7f3fda000640 1 -- 172.21.15.165:0/1961454828 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v171) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3fdc0586c0 con 0x7f3fe4161fc0 2024-04-25T10:19:45.561 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:19:45.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3fc006b4d0 msgr2=0x7f3fc006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3fc006b4d0 0x7f3fc006d990 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4034000 tx=0x7f3fd402e000 comp rx=0 tx=0).stop 2024-04-25T10:19:45.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 msgr2=0x7f3fe4148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe4148950 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7f3fdc001560 tx=0x7f3fdc00cc70 comp rx=0 tx=0).stop 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 shutdown_connections 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3fc006b4d0 0x7f3fc006d990 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3fe4161fc0 0x7f3fe4148950 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3fe406ef10 0x7f3fe40b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 --2- 172.21.15.165:0/1961454828 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3fe406ea90 0x7f3fe415f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.560+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 >> 172.21.15.165:0/1961454828 conn(0x7f3fe406abe0 msgr2=0x7f3fe406d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.561+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 shutdown_connections 2024-04-25T10:19:45.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.561+0000 7f3fe9200640 1 -- 172.21.15.165:0/1961454828 wait complete. 2024-04-25T10:19:45.583 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:19:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.776+0000 7fed26e00640 1 Processor -- start 2024-04-25T10:19:45.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.776+0000 7fed26e00640 1 -- start start 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed26e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fed200ccf30 0x7fed20162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed26e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200cdcd0 con 0x7fed20163100 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed26e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed200cd430 con 0x7fed2015f950 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed26e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed2006f100 con 0x7fed200ccf30 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed25000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fed200ccf30 0x7fed20162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed25000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fed200ccf30 0x7fed20162bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35784/0 (socket says 172.21.15.165:35784) 2024-04-25T10:19:45.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.778+0000 7fed25000640 1 -- 172.21.15.165:0/1605176250 learned_addr learned my addr 172.21.15.165:0/1605176250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.779+0000 7fed25000640 1 -- 172.21.15.165:0/1605176250 >> v1:172.21.15.172:6789/0 conn(0x7fed2015f950 legacy=0x7fed2006e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.779+0000 7fed25000640 1 -- 172.21.15.165:0/1605176250 >> v1:172.21.15.165:6789/0 conn(0x7fed20163100 legacy=0x7fed20165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.779+0000 7fed25000640 1 -- 172.21.15.165:0/1605176250 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed200cd710 con 0x7fed200ccf30 2024-04-25T10:19:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.780+0000 7fed25000640 1 --2- 172.21.15.165:0/1605176250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed200ccf30 0x7fed20162bc0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0098a0 tx=0x7fed0c02a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3486e8f2b834686c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:45.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.780+0000 7fed1fe00640 1 -- 172.21.15.165:0/1605176250 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c037070 con 0x7fed200ccf30 2024-04-25T10:19:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.780+0000 7fed1fe00640 1 -- 172.21.15.165:0/1605176250 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed0c032440 con 0x7fed200ccf30 2024-04-25T10:19:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.780+0000 7fed1fe00640 1 -- 172.21.15.165:0/1605176250 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed0c032810 con 0x7fed200ccf30 2024-04-25T10:19:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 -- 172.21.15.165:0/1605176250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed200ccf30 msgr2=0x7fed20162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 --2- 172.21.15.165:0/1605176250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed200ccf30 0x7fed20162bc0 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fed0c0098a0 tx=0x7fed0c02a2a0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 -- 172.21.15.165:0/1605176250 shutdown_connections 2024-04-25T10:19:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 --2- 172.21.15.165:0/1605176250 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed200ccf30 0x7fed20162bc0 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 -- 172.21.15.165:0/1605176250 >> 172.21.15.165:0/1605176250 conn(0x7fed20061cc0 msgr2=0x7fed2006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 -- 172.21.15.165:0/1605176250 shutdown_connections 2024-04-25T10:19:45.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.781+0000 7fed26e00640 1 -- 172.21.15.165:0/1605176250 wait complete. 2024-04-25T10:19:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.782+0000 7fed26e00640 1 Processor -- start 2024-04-25T10:19:45.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 -- start start 2024-04-25T10:19:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed200ccf30 0x7fed201560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed2015f950 0x7fed20155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 0x7fed20157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed201626a0 con 0x7fed2013fff0 2024-04-25T10:19:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed20162440 con 0x7fed200ccf30 2024-04-25T10:19:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.783+0000 7fed26e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2015fd30 con 0x7fed2015f950 2024-04-25T10:19:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 0x7fed20157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 0x7fed20157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35794/0 (socket says 172.21.15.165:35794) 2024-04-25T10:19:45.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 -- 172.21.15.165:0/2683080381 learned_addr learned my addr 172.21.15.165:0/2683080381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed25a00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed200ccf30 0x7fed201560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 -- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed2015f950 msgr2=0x7fed20155770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed25000640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed2015f950 0x7fed20155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed2015f950 0x7fed20155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 -- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed200ccf30 msgr2=0x7fed201560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed200ccf30 0x7fed201560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed26400640 1 -- 172.21.15.165:0/2683080381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed2015dba0 con 0x7fed2013fff0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.784+0000 7fed25a00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed200ccf30 0x7fed201560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.785+0000 7fed26400640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 0x7fed20157b10 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c730 tx=0x7fed1800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.785+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18003b70 con 0x7fed2013fff0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.785+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed18003d10 con 0x7fed2013fff0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.785+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed18005310 con 0x7fed2013fff0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.785+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed2015de80 con 0x7fed2013fff0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.785+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed2015e440 con 0x7fed2013fff0 2024-04-25T10:19:45.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.786+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed20073990 con 0x7fed2013fff0 2024-04-25T10:19:45.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.787+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fed180056c0 con 0x7fed2013fff0 2024-04-25T10:19:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.788+0000 7fed1d600640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed0006b4d0 0x7fed0006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.788+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fed1808a480 con 0x7fed2013fff0 2024-04-25T10:19:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.788+0000 7fed25a00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed0006b4d0 0x7fed0006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:45.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.789+0000 7fed25a00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed0006b4d0 0x7fed0006d990 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fed14000f60 tx=0x7fed14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:45.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.793+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fed180590d0 con 0x7fed2013fff0 2024-04-25T10:19:45.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.991+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed200b1290 con 0x7fed2013fff0 2024-04-25T10:19:45.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.992+0000 7fed1d600640 1 -- 172.21.15.165:0/2683080381 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v171) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed180589d0 con 0x7fed2013fff0 2024-04-25T10:19:45.996 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.994+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed0006b4d0 msgr2=0x7fed0006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.994+0000 7fed26e00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed0006b4d0 0x7fed0006d990 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fed14000f60 tx=0x7fed14009040 comp rx=0 tx=0).stop 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.994+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 msgr2=0x7fed20157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.994+0000 7fed26e00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 0x7fed20157b10 secure :-1 s=READY pgs=2509 cs=0 l=1 rev1=1 crypto rx=0x7fed1800c730 tx=0x7fed1800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 shutdown_connections 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fed0006b4d0 0x7fed0006d990 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fed2013fff0 0x7fed20157b10 unknown :-1 s=CLOSED pgs=2509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fed2015f950 0x7fed20155770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 --2- 172.21.15.165:0/2683080381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fed200ccf30 0x7fed201560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 >> 172.21.15.165:0/2683080381 conn(0x7fed20061cc0 msgr2=0x7fed200cc190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 shutdown_connections 2024-04-25T10:19:45.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:45.995+0000 7fed26e00640 1 -- 172.21.15.165:0/2683080381 wait complete. 2024-04-25T10:19:46.018 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:19:46.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.222+0000 7ff12ea00640 1 Processor -- start 2024-04-25T10:19:46.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.223+0000 7ff12ea00640 1 -- start start 2024-04-25T10:19:46.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12ea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff12806bff0 0x7ff1280b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12ea00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff12815e110 con 0x7ff1280b5970 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1280b2fd0 con 0x7ff12815dd30 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12ea00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff12806d000 con 0x7ff12806bff0 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12cc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff12806bff0 0x7ff1280b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12cc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff12806bff0 0x7ff1280b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35808/0 (socket says 172.21.15.165:35808) 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12cc00640 1 -- 172.21.15.165:0/647627529 learned_addr learned my addr 172.21.15.165:0/647627529 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.224+0000 7ff12e000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff1280b5970 0x7ff1280d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58576/0 (socket says 172.21.15.165:58576) 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.225+0000 7ff12cc00640 1 -- 172.21.15.165:0/647627529 >> v1:172.21.15.172:6789/0 conn(0x7ff12815dd30 legacy=0x7ff1280cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.225+0000 7ff12cc00640 1 -- 172.21.15.165:0/647627529 >> v1:172.21.15.165:6789/0 conn(0x7ff1280b5970 legacy=0x7ff1280d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.225+0000 7ff12cc00640 1 -- 172.21.15.165:0/647627529 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12806c480 con 0x7ff12806bff0 2024-04-25T10:19:46.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.225+0000 7ff127e00640 1 -- 172.21.15.165:0/647627529 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3583688629 0 0) 0x7ff12815e110 con 0x7ff1280b5970 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.226+0000 7ff12cc00640 1 --2- 172.21.15.165:0/647627529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12806bff0 0x7ff1280b5430 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ff118009e50 tx=0x7ff11802a760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab790ce0029715d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.226+0000 7ff127e00640 1 -- 172.21.15.165:0/647627529 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118036070 con 0x7ff12806bff0 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.226+0000 7ff127e00640 1 -- 172.21.15.165:0/647627529 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118031600 con 0x7ff12806bff0 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.226+0000 7ff127e00640 1 -- 172.21.15.165:0/647627529 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180319d0 con 0x7ff12806bff0 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 -- 172.21.15.165:0/647627529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12806bff0 msgr2=0x7ff1280b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 --2- 172.21.15.165:0/647627529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12806bff0 0x7ff1280b5430 secure :-1 s=READY pgs=2511 cs=0 l=1 rev1=1 crypto rx=0x7ff118009e50 tx=0x7ff11802a760 comp rx=0 tx=0).stop 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 -- 172.21.15.165:0/647627529 shutdown_connections 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 --2- 172.21.15.165:0/647627529 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12806bff0 0x7ff1280b5430 unknown :-1 s=CLOSED pgs=2511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 -- 172.21.15.165:0/647627529 >> 172.21.15.165:0/647627529 conn(0x7ff1280d0610 msgr2=0x7ff1280d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 -- 172.21.15.165:0/647627529 shutdown_connections 2024-04-25T10:19:46.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.227+0000 7ff12ea00640 1 -- 172.21.15.165:0/647627529 wait complete. 2024-04-25T10:19:46.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.229+0000 7ff12ea00640 1 Processor -- start 2024-04-25T10:19:46.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.229+0000 7ff12ea00640 1 -- start start 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1280b5970 0x7ff1280cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 0x7ff1280cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff12815b630 0x7ff12815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12806cd20 con 0x7ff12815dd30 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff12806ca40 con 0x7ff1280b5970 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1280d2870 con 0x7ff12815b630 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 0x7ff1280cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 0x7ff1280cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35822/0 (socket says 172.21.15.165:35822) 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 -- 172.21.15.165:0/812475511 learned_addr learned my addr 172.21.15.165:0/812475511 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12d600640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1280b5970 0x7ff1280cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12e000640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff12815b630 0x7ff12815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 -- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff12815b630 msgr2=0x7ff12815ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff12815b630 0x7ff12815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 -- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1280b5970 msgr2=0x7ff1280cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1280b5970 0x7ff1280cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.230+0000 7ff12cc00640 1 -- 172.21.15.165:0/812475511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12815be80 con 0x7ff12815dd30 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff12d600640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1280b5970 0x7ff1280cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff12cc00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 0x7ff1280cbad0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7ff118009e50 tx=0x7ff118004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118041020 con 0x7ff12815dd30 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118007820 con 0x7ff12815dd30 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118036040 con 0x7ff12815dd30 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1280cc190 con 0x7ff12815dd30 2024-04-25T10:19:46.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.231+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1280cc750 con 0x7ff12815dd30 2024-04-25T10:19:46.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.232+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff128073990 con 0x7ff12815dd30 2024-04-25T10:19:46.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.233+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff118044020 con 0x7ff12815dd30 2024-04-25T10:19:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.234+0000 7ff125600640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff10c06b4d0 0x7ff10c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.234+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff1180ab0c0 con 0x7ff12815dd30 2024-04-25T10:19:46.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.234+0000 7ff12d600640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff10c06b4d0 0x7ff10c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.235+0000 7ff12d600640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff10c06b4d0 0x7ff10c06d990 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00a830 tx=0x7ff11c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.238+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff11804b030 con 0x7ff12815dd30 2024-04-25T10:19:46.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.435+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1280b8b10 con 0x7ff12815dd30 2024-04-25T10:19:46.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.436+0000 7ff125600640 1 -- 172.21.15.165:0/812475511 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v171) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff12815be80 con 0x7ff12815dd30 2024-04-25T10:19:46.440 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff10c06b4d0 msgr2=0x7ff10c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff10c06b4d0 0x7ff10c06d990 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7ff11c00a830 tx=0x7ff11c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 msgr2=0x7ff1280cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 0x7ff1280cbad0 secure :-1 s=READY pgs=2512 cs=0 l=1 rev1=1 crypto rx=0x7ff118009e50 tx=0x7ff118004a40 comp rx=0 tx=0).stop 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 shutdown_connections 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff10c06b4d0 0x7ff10c06d990 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff12815b630 0x7ff12815ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff12815dd30 0x7ff1280cbad0 unknown :-1 s=CLOSED pgs=2512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 --2- 172.21.15.165:0/812475511 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff1280b5970 0x7ff1280cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 >> 172.21.15.165:0/812475511 conn(0x7ff1280d0610 msgr2=0x7ff1280b6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 shutdown_connections 2024-04-25T10:19:46.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.439+0000 7ff12ea00640 1 -- 172.21.15.165:0/812475511 wait complete. 2024-04-25T10:19:46.460 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:46.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.652+0000 7fadff200640 1 Processor -- start 2024-04-25T10:19:46.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.652+0000 7fadff200640 1 -- start start 2024-04-25T10:19:46.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.653+0000 7fadff200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fae00071700 0x7fae00071b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.653+0000 7fadff200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae001511b0 con 0x7fae0015e440 2024-04-25T10:19:46.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.653+0000 7fadff200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae0015a230 con 0x7fae00150d00 2024-04-25T10:19:46.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.653+0000 7fadff200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae0015ae30 con 0x7fae00071700 2024-04-25T10:19:46.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfde00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fae0015e440 0x7fae000ae8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:58578/0 (socket says 172.21.15.165:58578) 2024-04-25T10:19:46.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfde00640 1 -- 172.21.15.165:0/1043294417 learned_addr learned my addr 172.21.15.165:0/1043294417 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:46.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090187382 0 0) 0x7fae001511b0 con 0x7fae0015e440 2024-04-25T10:19:46.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade4005730 con 0x7fae0015e440 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfe800640 1 --2- 172.21.15.165:0/1043294417 >> v2:172.21.15.172:3300/0 conn(0x7fae00071700 0x7fae00071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075463125 0 0) 0x7fade4005730 con 0x7fae0015e440 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 >> v2:172.21.15.172:3300/0 conn(0x7fae00071700 msgr2=0x7fae00071b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.654+0000 7fadfd400640 1 --2- 172.21.15.165:0/1043294417 >> v2:172.21.15.172:3300/0 conn(0x7fae00071700 0x7fae00071b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 >> v1:172.21.15.172:6789/0 conn(0x7fae00150d00 legacy=0x7fae00070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae0015a500 con 0x7fae0015e440 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fadf0001670 con 0x7fae0015e440 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 >> v1:172.21.15.165:6789/0 conn(0x7fae0015e440 legacy=0x7fae000ae8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fade4005fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade4006a10 0x7fade4006e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fade40073d0 0x7fade400dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015ae30 con 0x7fade4005bc0 2024-04-25T10:19:46.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade400e300 con 0x7fade4006a10 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade400e500 con 0x7fade40073d0 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfde00640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fade4005fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfe800640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fade40073d0 0x7fade400dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadf5400640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade4006a10 0x7fade4006e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfde00640 1 -- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fade40073d0 msgr2=0x7fade400dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfde00640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fade40073d0 0x7fade400dbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfde00640 1 -- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade4006a10 msgr2=0x7fade4006e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfde00640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade4006a10 0x7fade4006e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.655+0000 7fadfde00640 1 -- 172.21.15.165:0/1043294417 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4005730 con 0x7fade4005bc0 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.656+0000 7fadf5400640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fade4006a10 0x7fade4006e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:46.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.656+0000 7fadfe800640 1 -- 172.21.15.165:0/1043294417 reap_dead start 2024-04-25T10:19:46.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.656+0000 7fadfde00640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fade4005fc0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fadf0007fd0 tx=0x7fade4007a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41d6d976f9640b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.657+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007d50 con 0x7fade4005bc0 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.657+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0005af0 con 0x7fade4005bc0 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.657+0000 7fadfd400640 1 -- 172.21.15.165:0/1043294417 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0007070 con 0x7fade4005bc0 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.657+0000 7fadff200640 1 -- 172.21.15.165:0/1043294417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 msgr2=0x7fade4005fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.657+0000 7fadff200640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fade4005fc0 secure :-1 s=READY pgs=2514 cs=0 l=1 rev1=1 crypto rx=0x7fadf0007fd0 tx=0x7fade4007a40 comp rx=0 tx=0).stop 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.658+0000 7fadff200640 1 -- 172.21.15.165:0/1043294417 shutdown_connections 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.658+0000 7fadff200640 1 --2- 172.21.15.165:0/1043294417 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fade4005fc0 unknown :-1 s=CLOSED pgs=2514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.658+0000 7fadff200640 1 -- 172.21.15.165:0/1043294417 >> 172.21.15.165:0/1043294417 conn(0x7fae0006abe0 msgr2=0x7fae0006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.658+0000 7fadff200640 1 -- 172.21.15.165:0/1043294417 shutdown_connections 2024-04-25T10:19:46.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.658+0000 7fadff200640 1 -- 172.21.15.165:0/1043294417 wait complete. 2024-04-25T10:19:46.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.659+0000 7fadff200640 1 Processor -- start 2024-04-25T10:19:46.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.660+0000 7fadff200640 1 -- start start 2024-04-25T10:19:46.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.660+0000 7fadff200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fae000afa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.660+0000 7fadff200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae0015c580 0x7fae00151910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.660+0000 7fadff200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0015ca00 0x7fae001457e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadff200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae00070ff0 con 0x7fade4005bc0 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadff200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0015aac0 con 0x7fae0015ca00 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadff200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae00071170 con 0x7fae0015c580 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fae000afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fae000afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:35842/0 (socket says 172.21.15.165:35842) 2024-04-25T10:19:46.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 -- 172.21.15.165:0/1171476232 learned_addr learned my addr 172.21.15.165:0/1171476232 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:46.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadf5400640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae0015c580 0x7fae00151910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfe800640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0015ca00 0x7fae001457e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 -- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae0015c580 msgr2=0x7fae00151910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae0015c580 0x7fae00151910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 -- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0015ca00 msgr2=0x7fae001457e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0015ca00 0x7fae001457e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadfde00640 1 -- 172.21.15.165:0/1171476232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae00152390 con 0x7fade4005bc0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.661+0000 7fadf5400640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae0015c580 0x7fae00151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.662+0000 7fadfde00640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fae000afa90 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009000 tx=0x7fadf0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.662+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf0037040 con 0x7fade4005bc0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.662+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadf0001430 con 0x7fade4005bc0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.662+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae00152670 con 0x7fade4005bc0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.662+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadf00047e0 con 0x7fade4005bc0 2024-04-25T10:19:46.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.662+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae000affd0 con 0x7fade4005bc0 2024-04-25T10:19:46.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.663+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae000af080 con 0x7fade4005bc0 2024-04-25T10:19:46.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.664+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadf0043020 con 0x7fade4005bc0 2024-04-25T10:19:46.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.665+0000 7fadf6a00640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faddc06b4d0 0x7faddc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:46.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.665+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fadf00a9a20 con 0x7fade4005bc0 2024-04-25T10:19:46.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.666+0000 7fadf5400640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faddc06b4d0 0x7faddc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:46.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.666+0000 7fadf5400640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faddc06b4d0 0x7faddc06d990 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fade8007fd0 tx=0x7fade8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:46.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.669+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadf0044370 con 0x7fade4005bc0 2024-04-25T10:19:46.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.897+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae00073990 con 0x7fade4005bc0 2024-04-25T10:19:46.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.898+0000 7fadf6a00640 1 -- 172.21.15.165:0/1171476232 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fadf0078700 con 0x7fade4005bc0 2024-04-25T10:19:46.902 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:46.902 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:46.902 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:46.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faddc06b4d0 msgr2=0x7faddc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faddc06b4d0 0x7faddc06d990 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7fade8007fd0 tx=0x7fade8009040 comp rx=0 tx=0).stop 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 msgr2=0x7fae000afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fae000afa90 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fadf0009000 tx=0x7fadf0001a40 comp rx=0 tx=0).stop 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 shutdown_connections 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faddc06b4d0 0x7faddc06d990 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fae0015ca00 0x7fae001457e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fae0015c580 0x7fae00151910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 --2- 172.21.15.165:0/1171476232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fade4005bc0 0x7fae000afa90 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 >> 172.21.15.165:0/1171476232 conn(0x7fae0006abe0 msgr2=0x7fae000ae750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 shutdown_connections 2024-04-25T10:19:46.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:46.901+0000 7fadff200640 1 -- 172.21.15.165:0/1171476232 wait complete. 2024-04-25T10:19:46.924 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:47.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.115+0000 7f959a200640 1 Processor -- start 2024-04-25T10:19:47.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.115+0000 7f959a200640 1 -- start start 2024-04-25T10:19:47.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.116+0000 7f959a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 0x7f95940cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.116+0000 7f959a200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959415bf30 con 0x7f95940cd860 2024-04-25T10:19:47.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f959a200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f959415c210 con 0x7f9594144440 2024-04-25T10:19:47.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f959a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f959415c860 con 0x7f9594071700 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f9599800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f95940cd860 0x7f959415ccf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60588/0 (socket says 172.21.15.165:60588) 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f9599800640 1 -- 172.21.15.165:0/424511096 learned_addr learned my addr 172.21.15.165:0/424511096 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f958be00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 0x7f95940cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1784916595 0 0) 0x7f959415bf30 con 0x7f95940cd860 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.117+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9578003520 con 0x7f95940cd860 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1046411029 0 0) 0x7f9578003520 con 0x7f95940cd860 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 msgr2=0x7f95940cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 0x7f95940cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 >> v1:172.21.15.172:6789/0 conn(0x7f9594144440 legacy=0x7f9594070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959415c4f0 con 0x7f95940cd860 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f958c0019f0 con 0x7f95940cd860 2024-04-25T10:19:47.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 >> v1:172.21.15.165:6789/0 conn(0x7f95940cd860 legacy=0x7f959415ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780039e0 0x7f9578003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 0x7f9578004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f95780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f959415c860 con 0x7f9578004d40 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f957800bb00 con 0x7f95780039e0 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f957800bd00 con 0x7f9594071700 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.118+0000 7f9599800640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f95780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f958be00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 0x7f9578004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9598e00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780039e0 0x7f9578003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9599800640 1 -- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 msgr2=0x7f9578004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9599800640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9594071700 0x7f9578004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9599800640 1 -- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780039e0 msgr2=0x7f9578003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9599800640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780039e0 0x7f9578003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9599800640 1 -- 172.21.15.165:0/424511096 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578003520 con 0x7f9578004d40 2024-04-25T10:19:47.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.119+0000 7f9598e00640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f95780039e0 0x7f9578003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.120+0000 7f9599800640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f95780051c0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f958c00a830 tx=0x7f9578005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604dd454aa2848f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.120+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c00a480 con 0x7f9578004d40 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.120+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f958c02f070 con 0x7f9578004d40 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.120+0000 7f9593e00640 1 -- 172.21.15.165:0/424511096 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f958c02bcb0 con 0x7f9578004d40 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f959a200640 1 -- 172.21.15.165:0/424511096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 msgr2=0x7f95780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f959a200640 1 --2- 172.21.15.165:0/424511096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f95780051c0 secure :-1 s=READY pgs=2517 cs=0 l=1 rev1=1 crypto rx=0x7f958c00a830 tx=0x7f9578005830 comp rx=0 tx=0).stop 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f9599800640 1 -- 172.21.15.165:0/424511096 reap_dead start 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f959a200640 1 -- 172.21.15.165:0/424511096 shutdown_connections 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f959a200640 1 -- 172.21.15.165:0/424511096 >> 172.21.15.165:0/424511096 conn(0x7f959406abe0 msgr2=0x7f959406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f959a200640 1 -- 172.21.15.165:0/424511096 shutdown_connections 2024-04-25T10:19:47.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.121+0000 7f959a200640 1 -- 172.21.15.165:0/424511096 wait complete. 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.123+0000 7f959a200640 1 Processor -- start 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.123+0000 7f959a200640 1 -- start start 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f959a200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f959413bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f959a200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f959413e630 0x7f95941eeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f959a200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f959413d3e0 0x7f959413d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f959a200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940cde70 con 0x7f9578004d40 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f959a200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95940cdcf0 con 0x7f959413e630 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f959a200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95940cd320 con 0x7f959413d3e0 2024-04-25T10:19:47.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f959413bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f959413bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39870/0 (socket says 172.21.15.165:39870) 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 -- 172.21.15.165:0/2007499386 learned_addr learned my addr 172.21.15.165:0/2007499386 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 -- 172.21.15.165:0/2007499386 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f959413d3e0 msgr2=0x7f959413d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f959413d3e0 0x7f959413d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 -- 172.21.15.165:0/2007499386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f959413e630 msgr2=0x7f95941eeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f959413e630 0x7f95941eeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.124+0000 7f9598e00640 1 -- 172.21.15.165:0/2007499386 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959413c340 con 0x7f9578004d40 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.125+0000 7f9598e00640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f959413bcd0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f958400d7d0 tx=0x7f958400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.125+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584005070 con 0x7f9578004d40 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.125+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9584005230 con 0x7f9578004d40 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.125+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9584007030 con 0x7f9578004d40 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.125+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f959413c620 con 0x7f9578004d40 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.125+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f959413cbe0 con 0x7f9578004d40 2024-04-25T10:19:47.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.126+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9594073990 con 0x7f9578004d40 2024-04-25T10:19:47.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.127+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f958401a030 con 0x7f9578004d40 2024-04-25T10:19:47.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.128+0000 7f9591600640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f957006b500 0x7f957006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.128+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f9584089a60 con 0x7f9578004d40 2024-04-25T10:19:47.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.128+0000 7f958be00640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f957006b500 0x7f957006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.129+0000 7f958be00640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f957006b500 0x7f957006d9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f959413bc60 tx=0x7f957c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.132+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9584021020 con 0x7f9578004d40 2024-04-25T10:19:47.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.330+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9594142ac0 con 0x7f9578004d40 2024-04-25T10:19:47.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.331+0000 7f9591600640 1 -- 172.21.15.165:0/2007499386 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v171) v1 ==== 74+0+16924 (secure 0 0 0) 0x7f9584058740 con 0x7f9578004d40 2024-04-25T10:19:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:47.335 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":171,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:00.610081+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f957006b500 msgr2=0x7f957006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f957006b500 0x7f957006d9c0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f959413bc60 tx=0x7f957c008040 comp rx=0 tx=0).stop 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 msgr2=0x7f959413bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f959413bcd0 secure :-1 s=READY pgs=2518 cs=0 l=1 rev1=1 crypto rx=0x7f958400d7d0 tx=0x7f958400dc90 comp rx=0 tx=0).stop 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 shutdown_connections 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f957006b500 0x7f957006d9c0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f959413d3e0 0x7f959413d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f959413e630 0x7f95941eeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 --2- 172.21.15.165:0/2007499386 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9578004d40 0x7f959413bcd0 unknown :-1 s=CLOSED pgs=2518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 >> 172.21.15.165:0/2007499386 conn(0x7f959406abe0 msgr2=0x7f95940b3c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 shutdown_connections 2024-04-25T10:19:47.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.334+0000 7f959a200640 1 -- 172.21.15.165:0/2007499386 wait complete. 2024-04-25T10:19:47.356 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:19:47.356 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:47.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.553+0000 7f74b0800640 1 Processor -- start 2024-04-25T10:19:47.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.553+0000 7f74b0800640 1 -- start start 2024-04-25T10:19:47.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74b0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 0x7f74a80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74b0800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a8071650 con 0x7f74a815df50 2024-04-25T10:19:47.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74b0800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74a80cf230 con 0x7f74a815db70 2024-04-25T10:19:47.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74b0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80cbd80 con 0x7f74a80cfc20 2024-04-25T10:19:47.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74ae800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f74a815df50 0x7f74a806d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60592/0 (socket says 172.21.15.165:60592) 2024-04-25T10:19:47.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74ae800640 1 -- 172.21.15.165:0/4217101819 learned_addr learned my addr 172.21.15.165:0/4217101819 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74ad400640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 0x7f74a80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.555+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3034949999 0 0) 0x7f74a4001490 con 0x7f74a815df50 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7490003520 con 0x7f74a815df50 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 143774016 0 0) 0x7f7490003520 con 0x7f74a815df50 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 msgr2=0x7f74a80cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 0x7f74a80cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 >> v1:172.21.15.172:6789/0 conn(0x7f74a815db70 legacy=0x7f74a80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a8161b60 con 0x7f74a815df50 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f74a4001bc0 con 0x7f74a815df50 2024-04-25T10:19:47.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 >> v1:172.21.15.165:6789/0 conn(0x7f74a815df50 legacy=0x7f74a806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f7490003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7490004800 0x7f7490004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 0x7f74900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a80cbd80 con 0x7f74900039e0 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f749000bad0 con 0x7f7490004800 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f749000bcd0 con 0x7f74a80cfc20 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.556+0000 7f74ade00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f7490003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ad400640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7490004800 0x7f7490004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ae800640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 0x7f74900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ade00640 1 -- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 msgr2=0x7f74900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ade00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a80cfc20 0x7f74900051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ade00640 1 -- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7490004800 msgr2=0x7f7490004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:47.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ade00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7490004800 0x7f7490004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.557+0000 7f74ade00640 1 -- 172.21.15.165:0/4217101819 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7490003520 con 0x7f74900039e0 2024-04-25T10:19:47.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.558+0000 7f74ade00640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f7490003e00 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f749c001720 tx=0x7f7490005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55cd6bcaeb8b684a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.558+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c001370 con 0x7f74900039e0 2024-04-25T10:19:47.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.558+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c0052d0 con 0x7f74900039e0 2024-04-25T10:19:47.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.558+0000 7f74aca00640 1 -- 172.21.15.165:0/4217101819 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c02ab90 con 0x7f74900039e0 2024-04-25T10:19:47.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74b0800640 1 -- 172.21.15.165:0/4217101819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 msgr2=0x7f7490003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74b0800640 1 --2- 172.21.15.165:0/4217101819 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f7490003e00 secure :-1 s=READY pgs=2520 cs=0 l=1 rev1=1 crypto rx=0x7f749c001720 tx=0x7f7490005830 comp rx=0 tx=0).stop 2024-04-25T10:19:47.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74ae800640 1 -- 172.21.15.165:0/4217101819 reap_dead start 2024-04-25T10:19:47.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74b0800640 1 -- 172.21.15.165:0/4217101819 shutdown_connections 2024-04-25T10:19:47.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74b0800640 1 -- 172.21.15.165:0/4217101819 >> 172.21.15.165:0/4217101819 conn(0x7f74a806abe0 msgr2=0x7f74a806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74b0800640 1 -- 172.21.15.165:0/4217101819 shutdown_connections 2024-04-25T10:19:47.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.559+0000 7f74b0800640 1 -- 172.21.15.165:0/4217101819 wait complete. 2024-04-25T10:19:47.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.561+0000 7f74b0800640 1 Processor -- start 2024-04-25T10:19:47.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.561+0000 7f74b0800640 1 -- start start 2024-04-25T10:19:47.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74b0800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f74a81f7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74b0800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a8152190 0x7f74a8151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74b0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a8160600 0x7f74a8160a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74b0800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8162120 con 0x7f74900039e0 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74b0800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8161910 con 0x7f74a8152190 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74b0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74a8162640 con 0x7f74a8160600 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f74a81f7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f74a81f7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39892/0 (socket says 172.21.15.165:39892) 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 -- 172.21.15.165:0/2153829712 learned_addr learned my addr 172.21.15.165:0/2153829712 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ad400640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a8152190 0x7f74a8151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 -- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a8160600 msgr2=0x7f74a8160a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a8160600 0x7f74a8160a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 -- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a8152190 msgr2=0x7f74a8151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.562+0000 7f74ade00640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a8152190 0x7f74a8151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74ade00640 1 -- 172.21.15.165:0/2153829712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74a81f7a10 con 0x7f74900039e0 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74ad400640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a8152190 0x7f74a8151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74ade00640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f74a81f7430 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f749c001840 tx=0x7f749c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c037040 con 0x7f74900039e0 2024-04-25T10:19:47.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749c005480 con 0x7f74900039e0 2024-04-25T10:19:47.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749c007470 con 0x7f74900039e0 2024-04-25T10:19:47.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74a81f7cf0 con 0x7f74900039e0 2024-04-25T10:19:47.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.563+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74a81f82b0 con 0x7f74900039e0 2024-04-25T10:19:47.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.565+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74a8073990 con 0x7f74900039e0 2024-04-25T10:19:47.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.566+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f749c044020 con 0x7f74900039e0 2024-04-25T10:19:47.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.566+0000 7f74a2000640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f748c06b500 0x7f748c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:47.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.567+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f749c0a9ff0 con 0x7f74900039e0 2024-04-25T10:19:47.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.567+0000 7f74ad400640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f748c06b500 0x7f748c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:47.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.567+0000 7f74ad400640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f748c06b500 0x7f748c06d9c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f74a80aeb30 tx=0x7f7498008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:47.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.571+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f749c078d70 con 0x7f74900039e0 2024-04-25T10:19:47.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.808+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f74a80aeb60 con 0x7f74900039e0 2024-04-25T10:19:47.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.810+0000 7f74a2000640 1 -- 172.21.15.165:0/2153829712 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f749c078670 con 0x7f74900039e0 2024-04-25T10:19:47.814 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:47.814 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:47.814 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f748c06b500 msgr2=0x7f748c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f748c06b500 0x7f748c06d9c0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f74a80aeb30 tx=0x7f7498008040 comp rx=0 tx=0).stop 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 msgr2=0x7f74a81f7430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f74a81f7430 secure :-1 s=READY pgs=2521 cs=0 l=1 rev1=1 crypto rx=0x7f749c001840 tx=0x7f749c001120 comp rx=0 tx=0).stop 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 shutdown_connections 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f748c06b500 0x7f748c06d9c0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f74a8160600 0x7f74a8160a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f74a8152190 0x7f74a8151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 --2- 172.21.15.165:0/2153829712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f74900039e0 0x7f74a81f7430 unknown :-1 s=CLOSED pgs=2521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:47.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 >> 172.21.15.165:0/2153829712 conn(0x7f74a806abe0 msgr2=0x7f74a80add80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:47.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.812+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 shutdown_connections 2024-04-25T10:19:47.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:47.813+0000 7f74b0800640 1 -- 172.21.15.165:0/2153829712 wait complete. 2024-04-25T10:19:47.836 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.021+0000 7fb497200640 1 Processor -- start 2024-04-25T10:19:48.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.021+0000 7fb497200640 1 -- start start 2024-04-25T10:19:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb497200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb490144750 0x7fb49006e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb497200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb490144b30 con 0x7fb4900cf030 2024-04-25T10:19:48.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb497200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4900712d0 con 0x7fb49006eb10 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb497200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4900715b0 con 0x7fb490144750 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb48fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb490144750 0x7fb49006e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb495200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb4900cf030 0x7fb49006d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60608/0 (socket says 172.21.15.165:60608) 2024-04-25T10:19:48.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.023+0000 7fb495200640 1 -- 172.21.15.165:0/1253028403 learned_addr learned my addr 172.21.15.165:0/1253028403 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.024+0000 7fb48fe00640 1 -- 172.21.15.165:0/1253028403 >> v1:172.21.15.172:6789/0 conn(0x7fb49006eb10 legacy=0x7fb4900ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.024+0000 7fb48fe00640 1 -- 172.21.15.165:0/1253028403 >> v1:172.21.15.165:6789/0 conn(0x7fb4900cf030 legacy=0x7fb49006d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.024+0000 7fb48fe00640 1 -- 172.21.15.165:0/1253028403 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490071890 con 0x7fb490144750 2024-04-25T10:19:48.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.024+0000 7fb48f400640 1 -- 172.21.15.165:0/1253028403 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266547421 0 0) 0x7fb490144b30 con 0x7fb4900cf030 2024-04-25T10:19:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.026+0000 7fb48fe00640 1 --2- 172.21.15.165:0/1253028403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb49006e5d0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fb4800097a0 tx=0x7fb48002a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fd958626d323d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.026+0000 7fb48f400640 1 -- 172.21.15.165:0/1253028403 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480037070 con 0x7fb490144750 2024-04-25T10:19:48.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.026+0000 7fb48f400640 1 -- 172.21.15.165:0/1253028403 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb480032440 con 0x7fb490144750 2024-04-25T10:19:48.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.026+0000 7fb48f400640 1 -- 172.21.15.165:0/1253028403 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb480032810 con 0x7fb490144750 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 -- 172.21.15.165:0/1253028403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 msgr2=0x7fb49006e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 --2- 172.21.15.165:0/1253028403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb49006e5d0 secure :-1 s=READY pgs=2523 cs=0 l=1 rev1=1 crypto rx=0x7fb4800097a0 tx=0x7fb48002a230 comp rx=0 tx=0).stop 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 -- 172.21.15.165:0/1253028403 shutdown_connections 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 --2- 172.21.15.165:0/1253028403 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb49006e5d0 unknown :-1 s=CLOSED pgs=2523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 -- 172.21.15.165:0/1253028403 >> 172.21.15.165:0/1253028403 conn(0x7fb49006abe0 msgr2=0x7fb49006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 -- 172.21.15.165:0/1253028403 shutdown_connections 2024-04-25T10:19:48.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.027+0000 7fb497200640 1 -- 172.21.15.165:0/1253028403 wait complete. 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.029+0000 7fb497200640 1 Processor -- start 2024-04-25T10:19:48.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.029+0000 7fb497200640 1 -- start start 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb497200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb49006eb10 0x7fb4900a84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb497200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4900cf030 0x7fb49015b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb497200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb490143070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb497200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4901455d0 con 0x7fb490144750 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb497200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb490145450 con 0x7fb49006eb10 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb497200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb49015c000 con 0x7fb4900cf030 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb495200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb490143070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb495200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb490143070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39908/0 (socket says 172.21.15.165:39908) 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb495200640 1 -- 172.21.15.165:0/2744357058 learned_addr learned my addr 172.21.15.165:0/2744357058 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb48fe00640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb49006eb10 0x7fb4900a84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.030+0000 7fb487400640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4900cf030 0x7fb49015b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb495200640 1 -- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4900cf030 msgr2=0x7fb49015b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb495200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4900cf030 0x7fb49015b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb495200640 1 -- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb49006eb10 msgr2=0x7fb4900a84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb495200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb49006eb10 0x7fb4900a84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb495200640 1 -- 172.21.15.165:0/2744357058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb490142020 con 0x7fb490144750 2024-04-25T10:19:48.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb48fe00640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb49006eb10 0x7fb4900a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb495200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb490143070 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c7d0 tx=0x7fb48800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb488018070 con 0x7fb490144750 2024-04-25T10:19:48.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb48800a5e0 con 0x7fb490144750 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4901422b0 con 0x7fb490144750 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb48800aa10 con 0x7fb490144750 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.031+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb490142b50 con 0x7fb490144750 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.032+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb490142d50 con 0x7fb490144750 2024-04-25T10:19:48.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.033+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb48801b070 con 0x7fb490144750 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.034+0000 7fb48cc00640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb46c06b4d0 0x7fb46c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.034+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6188+0+0 (secure 0 0 0) 0x7fb488089a30 con 0x7fb490144750 2024-04-25T10:19:48.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.035+0000 7fb48fe00640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb46c06b4d0 0x7fb46c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.035+0000 7fb48fe00640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb46c06b4d0 0x7fb46c06d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb480034000 tx=0x7fb48002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.039+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb490142d50 con 0x7fb490144750 2024-04-25T10:19:48.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.270+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb490073990 con 0x7fb490144750 2024-04-25T10:19:48.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.270+0000 7fb48cc00640 1 -- 172.21.15.165:0/2744357058 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fb488058630 con 0x7fb490144750 2024-04-25T10:19:48.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:48.275 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":121,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116}],"filesystems":[{"mdsmap":{"epoch":121,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:18:59.043438+0000","modified":"2024-04-25T10:19:02.102707+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":10767},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10767":{"gid":10767,"name":"a","rank":0,"incarnation":117,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6836/1299523073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":1299523073},{"type":"v1","addr":"172.21.15.165:6836","nonce":1299523073}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2024-04-25T10:19:48.275 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 121 2024-04-25T10:19:48.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.273+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb46c06b4d0 msgr2=0x7fb46c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.273+0000 7fb497200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb46c06b4d0 0x7fb46c06d990 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb480034000 tx=0x7fb48002e000 comp rx=0 tx=0).stop 2024-04-25T10:19:48.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.273+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 msgr2=0x7fb490143070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.273+0000 7fb497200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb490143070 secure :-1 s=READY pgs=2524 cs=0 l=1 rev1=1 crypto rx=0x7fb48800c7d0 tx=0x7fb48800cc90 comp rx=0 tx=0).stop 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 shutdown_connections 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb46c06b4d0 0x7fb46c06d990 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb490144750 0x7fb490143070 unknown :-1 s=CLOSED pgs=2524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4900cf030 0x7fb49015b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 --2- 172.21.15.165:0/2744357058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb49006eb10 0x7fb4900a84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 >> 172.21.15.165:0/2744357058 conn(0x7fb49006abe0 msgr2=0x7fb4900af2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 shutdown_connections 2024-04-25T10:19:48.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.274+0000 7fb497200640 1 -- 172.21.15.165:0/2744357058 wait complete. 2024-04-25T10:19:48.297 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.493+0000 7f1d81800640 1 Processor -- start 2024-04-25T10:19:48.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.493+0000 7f1d81800640 1 -- start start 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d81800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1d7c144750 0x7f1d7c06e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d81800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c06eb10 0x7f1d7c0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d81800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1d7c0cee60 0x7f1d7c15f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d81800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c144b30 con 0x7f1d7c144750 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d81800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d7c0712d0 con 0x7f1d7c0cee60 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d81800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c0715b0 con 0x7f1d7c06eb10 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d7aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1d7c144750 0x7f1d7c06e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d7aa00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1d7c144750 0x7f1d7c06e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39922/0 (socket says 172.21.15.165:39922) 2024-04-25T10:19:48.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.495+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3461623587 learned_addr learned my addr 172.21.15.165:0/3461623587 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3461623587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c06eb10 msgr2=0x7f1d7c0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7b400640 1 --2- 172.21.15.165:0/3461623587 >> v2:172.21.15.172:3300/0 conn(0x7f1d7c0cee60 0x7f1d7c15f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7aa00640 1 --2- 172.21.15.165:0/3461623587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c06eb10 0x7f1d7c0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3461623587 >> v2:172.21.15.172:3300/0 conn(0x7f1d7c0cee60 msgr2=0x7f1d7c15f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7aa00640 1 --2- 172.21.15.165:0/3461623587 >> v2:172.21.15.172:3300/0 conn(0x7f1d7c0cee60 0x7f1d7c15f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3461623587 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c071890 con 0x7f1d7c144750 2024-04-25T10:19:48.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.496+0000 7f1d7b400640 1 --2- 172.21.15.165:0/3461623587 >> v2:172.21.15.172:3300/0 conn(0x7f1d7c0cee60 0x7f1d7c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.497+0000 7f1d7aa00640 1 --2- 172.21.15.165:0/3461623587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c144750 0x7f1d7c06e5d0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0b4aa0 tx=0x7f1d6c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72b19056043178c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.497+0000 7f1d79600640 1 -- 172.21.15.165:0/3461623587 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c037070 con 0x7f1d7c144750 2024-04-25T10:19:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d79600640 1 -- 172.21.15.165:0/3461623587 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c032440 con 0x7f1d7c144750 2024-04-25T10:19:48.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d79600640 1 -- 172.21.15.165:0/3461623587 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c032810 con 0x7f1d7c144750 2024-04-25T10:19:48.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 -- 172.21.15.165:0/3461623587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c144750 msgr2=0x7f1d7c06e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 --2- 172.21.15.165:0/3461623587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c144750 0x7f1d7c06e5d0 secure :-1 s=READY pgs=2525 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0b4aa0 tx=0x7f1d6c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 -- 172.21.15.165:0/3461623587 shutdown_connections 2024-04-25T10:19:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 --2- 172.21.15.165:0/3461623587 >> v2:172.21.15.172:3300/0 conn(0x7f1d7c0cee60 0x7f1d7c15f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 --2- 172.21.15.165:0/3461623587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c06eb10 0x7f1d7c0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 --2- 172.21.15.165:0/3461623587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c144750 0x7f1d7c06e5d0 unknown :-1 s=CLOSED pgs=2525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 -- 172.21.15.165:0/3461623587 >> 172.21.15.165:0/3461623587 conn(0x7f1d7c06abe0 msgr2=0x7f1d7c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 -- 172.21.15.165:0/3461623587 shutdown_connections 2024-04-25T10:19:48.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.498+0000 7f1d81800640 1 -- 172.21.15.165:0/3461623587 wait complete. 2024-04-25T10:19:48.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.500+0000 7f1d81800640 1 Processor -- start 2024-04-25T10:19:48.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.500+0000 7f1d81800640 1 -- start start 2024-04-25T10:19:48.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d81800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 0x7f1d7c13a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d81800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c0cee60 0x7f1d7c13c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d81800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c144750 0x7f1d7c13c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d81800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c145450 con 0x7f1d7c06eb10 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d81800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c06f020 con 0x7f1d7c144750 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d81800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d7c1455d0 con 0x7f1d7c0cee60 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.501+0000 7f1d7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 0x7f1d7c13a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 0x7f1d7c13a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39928/0 (socket says 172.21.15.165:39928) 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3615023466 learned_addr learned my addr 172.21.15.165:0/3615023466 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7a000640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c0cee60 0x7f1d7c13c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c0cee60 msgr2=0x7f1d7c13c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7b400640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c144750 0x7f1d7c13c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c0cee60 0x7f1d7c13c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c144750 msgr2=0x7f1d7c13c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c144750 0x7f1d7c13c630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 -- 172.21.15.165:0/3615023466 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c1f7550 con 0x7f1d7c06eb10 2024-04-25T10:19:48.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7a000640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c0cee60 0x7f1d7c13c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.502+0000 7f1d7aa00640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 0x7f1d7c13a340 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c02a220 tx=0x7f1d6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.503+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c037040 con 0x7f1d7c06eb10 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.503+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d6c042070 con 0x7f1d7c06eb10 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.503+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d6c030bb0 con 0x7f1d7c06eb10 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.503+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d7c1f77f0 con 0x7f1d7c06eb10 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.503+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d7c1f7db0 con 0x7f1d7c06eb10 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.504+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d7c073990 con 0x7f1d7c06eb10 2024-04-25T10:19:48.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.505+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1d6c047460 con 0x7f1d7c06eb10 2024-04-25T10:19:48.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.506+0000 7f1d72a00640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d5c06b4d0 0x7f1d5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.506+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f1d6c0ab080 con 0x7f1d7c06eb10 2024-04-25T10:19:48.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.506+0000 7f1d7a000640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d5c06b4d0 0x7f1d5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.507+0000 7f1d7a000640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d5c06b4d0 0x7f1d5c06d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c13b310 tx=0x7f1d68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.510+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1d6c045360 con 0x7f1d7c06eb10 2024-04-25T10:19:48.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.705+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d7c0d0510 con 0x7f1d7c06eb10 2024-04-25T10:19:48.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.707+0000 7f1d72a00640 1 -- 172.21.15.165:0/3615023466 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+17064 (secure 0 0 0) 0x7f1d6c079d10 con 0x7f1d7c06eb10 2024-04-25T10:19:48.711 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:48.711 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":172,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:47.829585+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:18:56.874641+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_data","create_time":"2024-04-25T10:18:58.066411+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"171","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"26.5","mappings":[{"from":1,"to":0}]},{"pgid":"26.8","mappings":[{"from":1,"to":0}]},{"pgid":"26.27","mappings":[{"from":3,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.710+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d5c06b4d0 msgr2=0x7f1d5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.710+0000 7f1d81800640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d5c06b4d0 0x7f1d5c06d990 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c13b310 tx=0x7f1d68008040 comp rx=0 tx=0).stop 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.710+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 msgr2=0x7f1d7c13a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.710+0000 7f1d81800640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 0x7f1d7c13a340 secure :-1 s=READY pgs=2526 cs=0 l=1 rev1=1 crypto rx=0x7f1d6c02a220 tx=0x7f1d6c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 shutdown_connections 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1d5c06b4d0 0x7f1d5c06d990 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1d7c144750 0x7f1d7c13c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1d7c0cee60 0x7f1d7c13c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 --2- 172.21.15.165:0/3615023466 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1d7c06eb10 0x7f1d7c13a340 unknown :-1 s=CLOSED pgs=2526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 >> 172.21.15.165:0/3615023466 conn(0x7f1d7c06abe0 msgr2=0x7f1d7c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 shutdown_connections 2024-04-25T10:19:48.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.711+0000 7f1d81800640 1 -- 172.21.15.165:0/3615023466 wait complete. 2024-04-25T10:19:48.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:19:48.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.940+0000 7f1b94a00640 1 Processor -- start 2024-04-25T10:19:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.940+0000 7f1b94a00640 1 -- start start 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b94a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1b900cd8a0 0x7f1b9015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b94a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9015e6e0 con 0x7f1b900cd4c0 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b94a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b9015e9c0 con 0x7f1b90150b90 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b94a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b9015a9c0 con 0x7f1b900cd8a0 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b8de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1b900cd4c0 0x7f1b9015dc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60616/0 (socket says 172.21.15.165:60616) 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b8de00640 1 -- 172.21.15.165:0/3616467263 learned_addr learned my addr 172.21.15.165:0/3616467263 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.941+0000 7f1b87e00640 1 --2- 172.21.15.165:0/3616467263 >> v2:172.21.15.165:3300/0 conn(0x7f1b900cd8a0 0x7f1b9015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.942+0000 7f1b87e00640 1 -- 172.21.15.165:0/3616467263 >> v1:172.21.15.172:6789/0 conn(0x7f1b90150b90 legacy=0x7f1b90071270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.942+0000 7f1b87e00640 1 -- 172.21.15.165:0/3616467263 >> v1:172.21.15.165:6789/0 conn(0x7f1b900cd4c0 legacy=0x7f1b9015dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.942+0000 7f1b87e00640 1 -- 172.21.15.165:0/3616467263 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b9015aca0 con 0x7f1b900cd8a0 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.942+0000 7f1b8d400640 1 -- 172.21.15.165:0/3616467263 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3269781286 0 0) 0x7f1b9015e6e0 con 0x7f1b900cd4c0 2024-04-25T10:19:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.943+0000 7f1b87e00640 1 --2- 172.21.15.165:0/3616467263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015b1c0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800a8c0 tx=0x7f1b7802b300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc4c1ed081c8fbd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.943+0000 7f1b8d400640 1 -- 172.21.15.165:0/3616467263 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78038070 con 0x7f1b900cd8a0 2024-04-25T10:19:48.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.943+0000 7f1b8d400640 1 -- 172.21.15.165:0/3616467263 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78033640 con 0x7f1b900cd8a0 2024-04-25T10:19:48.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.943+0000 7f1b8d400640 1 -- 172.21.15.165:0/3616467263 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78033a10 con 0x7f1b900cd8a0 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 -- 172.21.15.165:0/3616467263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 msgr2=0x7f1b9015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 --2- 172.21.15.165:0/3616467263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015b1c0 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f1b7800a8c0 tx=0x7f1b7802b300 comp rx=0 tx=0).stop 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 -- 172.21.15.165:0/3616467263 shutdown_connections 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 --2- 172.21.15.165:0/3616467263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015b1c0 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 -- 172.21.15.165:0/3616467263 >> 172.21.15.165:0/3616467263 conn(0x7f1b9006abe0 msgr2=0x7f1b9006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 -- 172.21.15.165:0/3616467263 shutdown_connections 2024-04-25T10:19:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.944+0000 7f1b94a00640 1 -- 172.21.15.165:0/3616467263 wait complete. 2024-04-25T10:19:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.946+0000 7f1b94a00640 1 Processor -- start 2024-04-25T10:19:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.946+0000 7f1b94a00640 1 -- start start 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b94a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cd4c0 0x7f1b900ad290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b94a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b94a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b90150b90 0x7f1b9006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b94a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9006eca0 con 0x7f1b900cd8a0 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b94a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9015e370 con 0x7f1b900cd4c0 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b94a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9006ee20 con 0x7f1b90150b90 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39944/0 (socket says 172.21.15.165:39944) 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b87e00640 1 -- 172.21.15.165:0/2480872805 learned_addr learned my addr 172.21.15.165:0/2480872805 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.947+0000 7f1b8de00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cd4c0 0x7f1b900ad290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b87e00640 1 -- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b90150b90 msgr2=0x7f1b9006d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b87e00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b90150b90 0x7f1b9006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b87e00640 1 -- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cd4c0 msgr2=0x7f1b900ad290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b87e00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cd4c0 0x7f1b900ad290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b87e00640 1 -- 172.21.15.165:0/2480872805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900ad950 con 0x7f1b900cd8a0 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b8de00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cd4c0 0x7f1b900ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b87e00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015cdf0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f1b78036040 tx=0x7f1b780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b78038040 con 0x7f1b900cd8a0 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b78043070 con 0x7f1b900cd8a0 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b7803e890 con 0x7f1b900cd8a0 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b900af310 con 0x7f1b900cd8a0 2024-04-25T10:19:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.948+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b900af8d0 con 0x7f1b900cd8a0 2024-04-25T10:19:48.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.950+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b90073a00 con 0x7f1b900cd8a0 2024-04-25T10:19:48.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.950+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1b7803ea30 con 0x7f1b900cd8a0 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.951+0000 7f1b86000640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b6c06b4d0 0x7f1b6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.951+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 6188+0+0 (secure 0 0 0) 0x7f1b780aaff0 con 0x7f1b900cd8a0 2024-04-25T10:19:48.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.952+0000 7f1b8de00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b6c06b4d0 0x7f1b6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:48.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.952+0000 7f1b8de00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b6c06b4d0 0x7f1b6c06d990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f60 tx=0x7f1b80009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:48.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:48.956+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1b78031e20 con 0x7f1b900cd8a0 2024-04-25T10:19:49.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.191+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f1b90154660 con 0x7f1b900cd8a0 2024-04-25T10:19:49.196 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:19:49.193+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:19:49.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.196+0000 7f1b86000640 1 -- 172.21.15.165:0/2480872805 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v122) v1 ==== 161+0+0 (secure 0 0 0) 0x7f1b78079d30 con 0x7f1b900cd8a0 2024-04-25T10:19:49.200 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:19:49.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b6c06b4d0 msgr2=0x7f1b6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b6c06b4d0 0x7f1b6c06d990 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f1b80000f60 tx=0x7f1b80009040 comp rx=0 tx=0).stop 2024-04-25T10:19:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 msgr2=0x7f1b9015cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015cdf0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f1b78036040 tx=0x7f1b780046c0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 shutdown_connections 2024-04-25T10:19:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1b6c06b4d0 0x7f1b6c06d990 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1b90150b90 0x7f1b9006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1b900cd8a0 0x7f1b9015cdf0 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 --2- 172.21.15.165:0/2480872805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1b900cd4c0 0x7f1b900ad290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 >> 172.21.15.165:0/2480872805 conn(0x7f1b9006abe0 msgr2=0x7f1b900b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.200+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 shutdown_connections 2024-04-25T10:19:49.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.201+0000 7f1b94a00640 1 -- 172.21.15.165:0/2480872805 wait complete. 2024-04-25T10:19:49.222 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:19:49.236 INFO:tasks.ceph.mds.a.smithi165.stderr: -6352> 2024-04-25T10:19:03.967+0000 7fc301c00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:19:49.248 INFO:tasks.ceph.mds.a.smithi165.stderr: -3427> 2024-04-25T10:19:09.671+0000 7fc2fcc00640 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v6 ap=1 DIRTYPARENT s=5242880 n(v0 rc2024-04-25T10:19:07.958482+0000 b5242880 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={10926=0-12582912@1} caps={10926=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@4},l=10926 | ptrwaiter=0 request=0 lock=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x55648960b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(26)0x10000000001:[<0x10000000000/sixmegs v6>,<0x1/subdir v8>]//[]","error_str":"failed to decode on-disk backtrace (0 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2024-04-25T10:19:49.304 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:19:49.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.407+0000 7f16dd000640 1 Processor -- start 2024-04-25T10:19:49.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.407+0000 7f16dd000640 1 -- start start 2024-04-25T10:19:49.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.408+0000 7f16dd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 0x7f16d80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.408+0000 7f16dd000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80cccc0 con 0x7f16d8144d50 2024-04-25T10:19:49.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.408+0000 7f16dd000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16d80d0460 con 0x7f16d806e8a0 2024-04-25T10:19:49.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.408+0000 7f16dd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b2b20 con 0x7f16d81446d0 2024-04-25T10:19:49.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.409+0000 7f16cfe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f16d8144d50 0x7f16d80cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60644/0 (socket says 172.21.15.165:60644) 2024-04-25T10:19:49.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.409+0000 7f16cfe00640 1 -- 172.21.15.165:0/1471690800 learned_addr learned my addr 172.21.15.165:0/1471690800 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:49.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.409+0000 7f16d5e00640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 0x7f16d80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:49.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.409+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672463770 0 0) 0x7f16d80cccc0 con 0x7f16d8144d50 2024-04-25T10:19:49.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.409+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f16bc003520 con 0x7f16d8144d50 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3686617673 0 0) 0x7f16bc003520 con 0x7f16d8144d50 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 msgr2=0x7f16d80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 0x7f16d80b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 >> v1:172.21.15.172:6789/0 conn(0x7f16d806e8a0 legacy=0x7f16d80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d80cc140 con 0x7f16d8144d50 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f16c0001670 con 0x7f16d8144d50 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 >> v1:172.21.15.165:6789/0 conn(0x7f16d8144d50 legacy=0x7f16d80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 0x7f16bc0039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16bc003f60 0x7f16bc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc004d40 0x7f16bc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80b2b20 con 0x7f16bc003f60 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc00baa0 con 0x7f16bc004d40 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16bc00bca0 con 0x7f16d81446d0 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16cfe00640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16bc003f60 0x7f16bc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:49.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16d5e00640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 0x7f16bc0039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:49.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16cfe00640 1 -- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 msgr2=0x7f16bc0039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:49.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16cfe00640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81446d0 0x7f16bc0039e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16cfe00640 1 -- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc004d40 msgr2=0x7f16bc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:49.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.410+0000 7f16cfe00640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc004d40 0x7f16bc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.411+0000 7f16cfe00640 1 -- 172.21.15.165:0/1471690800 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16bc003520 con 0x7f16bc003f60 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16cfe00640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16bc003f60 0x7f16bc004800 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f16c000c210 tx=0x7f16bc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94484be7dbc00cb5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0007d30 con 0x7f16bc003f60 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c0005ad0 con 0x7f16bc003f60 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16d5400640 1 -- 172.21.15.165:0/1471690800 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c0007070 con 0x7f16bc003f60 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16dd000640 1 -- 172.21.15.165:0/1471690800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16bc003f60 msgr2=0x7f16bc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16dd000640 1 --2- 172.21.15.165:0/1471690800 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16bc003f60 0x7f16bc004800 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7f16c000c210 tx=0x7f16bc005830 comp rx=0 tx=0).stop 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.412+0000 7f16d6800640 1 -- 172.21.15.165:0/1471690800 reap_dead start 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.413+0000 7f16dd000640 1 -- 172.21.15.165:0/1471690800 shutdown_connections 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.413+0000 7f16dd000640 1 -- 172.21.15.165:0/1471690800 >> 172.21.15.165:0/1471690800 conn(0x7f16d806abe0 msgr2=0x7f16d806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.413+0000 7f16dd000640 1 -- 172.21.15.165:0/1471690800 shutdown_connections 2024-04-25T10:19:49.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.413+0000 7f16dd000640 1 -- 172.21.15.165:0/1471690800 wait complete. 2024-04-25T10:19:49.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.415+0000 7f16dd000640 1 Processor -- start 2024-04-25T10:19:49.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.415+0000 7f16dd000640 1 -- start start 2024-04-25T10:19:49.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16dd000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc003f60 0x7f16d8150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16dd000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81390e0 0x7f16d81f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16dd000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 0x7f16d81526f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16dd000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80cc700 con 0x7f16d8152270 2024-04-25T10:19:49.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16dd000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16d806d400 con 0x7f16bc003f60 2024-04-25T10:19:49.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16dd000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16d80cc9e0 con 0x7f16d81390e0 2024-04-25T10:19:49.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 0x7f16d81526f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:49.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 0x7f16d81526f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39972/0 (socket says 172.21.15.165:39972) 2024-04-25T10:19:49.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 -- 172.21.15.165:0/1656539632 learned_addr learned my addr 172.21.15.165:0/1656539632 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 -- 172.21.15.165:0/1656539632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81390e0 msgr2=0x7f16d81f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81390e0 0x7f16d81f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 -- 172.21.15.165:0/1656539632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc003f60 msgr2=0x7f16d8150710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc003f60 0x7f16d8150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.416+0000 7f16d6800640 1 -- 172.21.15.165:0/1656539632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16d8150e10 con 0x7f16d8152270 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.417+0000 7f16d6800640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 0x7f16d81526f0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f16d806f710 tx=0x7f16d000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.417+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d0017070 con 0x7f16d8152270 2024-04-25T10:19:49.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.417+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16d0005d60 con 0x7f16d8152270 2024-04-25T10:19:49.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.417+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16d000a880 con 0x7f16d8152270 2024-04-25T10:19:49.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.417+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16d8153040 con 0x7f16d8152270 2024-04-25T10:19:49.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.417+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16d8153600 con 0x7f16d8152270 2024-04-25T10:19:49.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.418+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16d8073a00 con 0x7f16d8152270 2024-04-25T10:19:49.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.419+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f16d000aa20 con 0x7f16d8152270 2024-04-25T10:19:49.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.420+0000 7f16ce000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16b406b500 0x7f16b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:49.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.420+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f16d00897b0 con 0x7f16d8152270 2024-04-25T10:19:49.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.421+0000 7f16d5e00640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16b406b500 0x7f16b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:49.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.421+0000 7f16d5e00640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16b406b500 0x7f16b406d9c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009950 tx=0x7f16c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:49.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.425+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f16d8150e10 con 0x7f16d8152270 2024-04-25T10:19:49.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:49.649+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f16d8143ef0 con 0x7f16d8152270 2024-04-25T10:19:50.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.200+0000 7f16ce000640 1 -- 172.21.15.165:0/1656539632 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v123) v1 ==== 104+0+0 (secure 0 0 0) 0x7f16d0087e20 con 0x7f16d8152270 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16b406b500 msgr2=0x7f16b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16b406b500 0x7f16b406d9c0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f16c8009950 tx=0x7f16c8008040 comp rx=0 tx=0).stop 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 msgr2=0x7f16d81526f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 0x7f16d81526f0 secure :-1 s=READY pgs=2535 cs=0 l=1 rev1=1 crypto rx=0x7f16d806f710 tx=0x7f16d000cb90 comp rx=0 tx=0).stop 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 shutdown_connections 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f16b406b500 0x7f16b406d9c0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f16d8152270 0x7f16d81526f0 unknown :-1 s=CLOSED pgs=2535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f16d81390e0 0x7f16d81f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 --2- 172.21.15.165:0/1656539632 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f16bc003f60 0x7f16d8150710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 >> 172.21.15.165:0/1656539632 conn(0x7f16d806abe0 msgr2=0x7f16d80cd2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:50.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 shutdown_connections 2024-04-25T10:19:50.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.204+0000 7f16dd000640 1 -- 172.21.15.165:0/1656539632 wait complete. 2024-04-25T10:19:50.229 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:19:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.475+0000 7f87b6600640 1 Processor -- start 2024-04-25T10:19:50.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.475+0000 7f87b6600640 1 -- start start 2024-04-25T10:19:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.476+0000 7f87b6600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f87b00ceb20 0x7f87b00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.476+0000 7f87b6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 0x7f87b015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.476+0000 7f87b6600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87b0145520 con 0x7f87b0145140 2024-04-25T10:19:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.476+0000 7f87b6600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87b0071270 con 0x7f87b00ceb20 2024-04-25T10:19:50.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.476+0000 7f87b6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0071550 con 0x7f87b00c5030 2024-04-25T10:19:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.477+0000 7f87b5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 0x7f87b015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.477+0000 7f87b5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 0x7f87b015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:47512/0 (socket says 172.21.15.165:47512) 2024-04-25T10:19:50.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.477+0000 7f87b5c00640 1 -- 172.21.15.165:0/4100448265 learned_addr learned my addr 172.21.15.165:0/4100448265 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:50.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.477+0000 7f87a7e00640 1 --2- 172.21.15.165:0/4100448265 >> v2:172.21.15.172:3300/0 conn(0x7f87b00ceb20 0x7f87b00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:50.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.478+0000 7f87b5c00640 1 -- 172.21.15.165:0/4100448265 >> v2:172.21.15.172:3300/0 conn(0x7f87b00ceb20 msgr2=0x7f87b00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:50.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.478+0000 7f87b5c00640 1 --2- 172.21.15.165:0/4100448265 >> v2:172.21.15.172:3300/0 conn(0x7f87b00ceb20 0x7f87b00c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.478+0000 7f87b5c00640 1 -- 172.21.15.165:0/4100448265 >> v1:172.21.15.172:6789/0 conn(0x7f87b0145140 legacy=0x7f87b00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:50.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.478+0000 7f87b5c00640 1 -- 172.21.15.165:0/4100448265 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b0071830 con 0x7f87b00c5030 2024-04-25T10:19:50.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.479+0000 7f87b5c00640 1 --2- 172.21.15.165:0/4100448265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 0x7f87b015bfb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009e50 tx=0x7f87a802a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e25f9db05d59a78 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:50.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.480+0000 7f87afe00640 1 -- 172.21.15.165:0/4100448265 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a8036070 con 0x7f87b00c5030 2024-04-25T10:19:50.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.480+0000 7f87afe00640 1 -- 172.21.15.165:0/4100448265 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a8031600 con 0x7f87b00c5030 2024-04-25T10:19:50.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.480+0000 7f87afe00640 1 -- 172.21.15.165:0/4100448265 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a80319d0 con 0x7f87b00c5030 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 -- 172.21.15.165:0/4100448265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 msgr2=0x7f87b015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 --2- 172.21.15.165:0/4100448265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 0x7f87b015bfb0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f87a8009e50 tx=0x7f87a802a680 comp rx=0 tx=0).stop 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 -- 172.21.15.165:0/4100448265 shutdown_connections 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 --2- 172.21.15.165:0/4100448265 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b00c5030 0x7f87b015bfb0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 --2- 172.21.15.165:0/4100448265 >> v2:172.21.15.172:3300/0 conn(0x7f87b00ceb20 0x7f87b00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 -- 172.21.15.165:0/4100448265 >> 172.21.15.165:0/4100448265 conn(0x7f87b0061cc0 msgr2=0x7f87b006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 -- 172.21.15.165:0/4100448265 shutdown_connections 2024-04-25T10:19:50.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.481+0000 7f87b6600640 1 -- 172.21.15.165:0/4100448265 wait complete. 2024-04-25T10:19:50.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.483+0000 7f87b6600640 1 Processor -- start 2024-04-25T10:19:50.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.483+0000 7f87b6600640 1 -- start start 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 0x7f87b01f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87b00ceb20 0x7f87b01e8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b0145140 0x7f87b01e8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00c5620 con 0x7f87b00c5030 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b00c54a0 con 0x7f87b00ceb20 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87b015c4f0 con 0x7f87b0145140 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.484+0000 7f87b5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 0x7f87b01f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:50.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 0x7f87b01f67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39978/0 (socket says 172.21.15.165:39978) 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 -- 172.21.15.165:0/4001485244 learned_addr learned my addr 172.21.15.165:0/4001485244 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87a7e00640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87b00ceb20 0x7f87b01e8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5c00640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b0145140 0x7f87b01e8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 -- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b0145140 msgr2=0x7f87b01e8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b0145140 0x7f87b01e8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 -- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87b00ceb20 msgr2=0x7f87b01e8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87b00ceb20 0x7f87b01e8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 -- 172.21.15.165:0/4001485244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87b01e4bd0 con 0x7f87b00c5030 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87a7e00640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87b00ceb20 0x7f87b01e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:50.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5c00640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b0145140 0x7f87b01e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:50.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.485+0000 7f87b5200640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 0x7f87b01f67e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c430 tx=0x7f87a000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:50.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.486+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0017070 con 0x7f87b00c5030 2024-04-25T10:19:50.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.486+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87a00052d0 con 0x7f87b00c5030 2024-04-25T10:19:50.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.486+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87a0003830 con 0x7f87b00c5030 2024-04-25T10:19:50.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.486+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b01e4eb0 con 0x7f87b00c5030 2024-04-25T10:19:50.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.486+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b01e5470 con 0x7f87b00c5030 2024-04-25T10:19:50.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.487+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b015bfb0 con 0x7f87b00c5030 2024-04-25T10:19:50.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.488+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f87a001a070 con 0x7f87b00c5030 2024-04-25T10:19:50.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.489+0000 7f87ad600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f878c06b4d0 0x7f878c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:50.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.489+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6274+0+0 (secure 0 0 0) 0x7f87a00897d0 con 0x7f87b00c5030 2024-04-25T10:19:50.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.489+0000 7f87a7e00640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f878c06b4d0 0x7f878c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:50.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.490+0000 7f87a7e00640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f878c06b4d0 0x7f878c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f87b015df60 tx=0x7f8798008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:50.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.493+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f87b015bfb0 con 0x7f87b00c5030 2024-04-25T10:19:50.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:50.686+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f87b00ce900 con 0x7f87b00c5030 2024-04-25T10:19:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.210+0000 7f87ad600640 1 -- 172.21.15.165:0/4001485244 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f87b00ce900 con 0x7f87b00c5030 2024-04-25T10:19:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:19:51.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f878c06b4d0 msgr2=0x7f878c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:51.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f878c06b4d0 0x7f878c06d990 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f87b015df60 tx=0x7f8798008040 comp rx=0 tx=0).stop 2024-04-25T10:19:51.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 msgr2=0x7f87b01f67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:51.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 0x7f87b01f67e0 secure :-1 s=READY pgs=2536 cs=0 l=1 rev1=1 crypto rx=0x7f87a000c430 tx=0x7f87a000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 shutdown_connections 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f878c06b4d0 0x7f878c06d990 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f87b0145140 0x7f87b01e8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f87b00ceb20 0x7f87b01e8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 --2- 172.21.15.165:0/4001485244 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f87b00c5030 0x7f87b01f67e0 unknown :-1 s=CLOSED pgs=2536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.214+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 >> 172.21.15.165:0/4001485244 conn(0x7f87b0061cc0 msgr2=0x7f87b006eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.215+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 shutdown_connections 2024-04-25T10:19:51.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.215+0000 7f87b6600640 1 -- 172.21.15.165:0/4001485244 wait complete. 2024-04-25T10:19:51.242 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:19:51.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.439+0000 7f77ae800640 1 Processor -- start 2024-04-25T10:19:51.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.439+0000 7f77ae800640 1 -- start start 2024-04-25T10:19:51.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77ae800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f77a80cd4c0 0x7f77a80cd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77ae800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a815b1c0 0x7f77a815dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77ae800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77a815e1a0 0x7f77a8150b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77ae800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a815b620 con 0x7f77a80cd4c0 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77ae800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77a815e6e0 con 0x7f77a815e1a0 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77ae800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a815e9c0 con 0x7f77a815b1c0 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77a7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f77a80cd4c0 0x7f77a80cd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77a7400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f77a80cd4c0 0x7f77a80cd8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39994/0 (socket says 172.21.15.165:39994) 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.441+0000 7f77a7400640 1 -- 172.21.15.165:0/1473118837 learned_addr learned my addr 172.21.15.165:0/1473118837 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a6a00640 1 --2- 172.21.15.165:0/1473118837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a815b1c0 0x7f77a815dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7e00640 1 --2- 172.21.15.165:0/1473118837 >> v2:172.21.15.172:3300/0 conn(0x7f77a815e1a0 0x7f77a8150b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7400640 1 -- 172.21.15.165:0/1473118837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a815b1c0 msgr2=0x7f77a815dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7400640 1 --2- 172.21.15.165:0/1473118837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a815b1c0 0x7f77a815dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7400640 1 -- 172.21.15.165:0/1473118837 >> v2:172.21.15.172:3300/0 conn(0x7f77a815e1a0 msgr2=0x7f77a8150b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7400640 1 --2- 172.21.15.165:0/1473118837 >> v2:172.21.15.172:3300/0 conn(0x7f77a815e1a0 0x7f77a8150b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7400640 1 -- 172.21.15.165:0/1473118837 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a815a9c0 con 0x7f77a80cd4c0 2024-04-25T10:19:51.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.442+0000 7f77a7e00640 1 --2- 172.21.15.165:0/1473118837 >> v2:172.21.15.172:3300/0 conn(0x7f77a815e1a0 0x7f77a8150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:51.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.443+0000 7f77a7400640 1 --2- 172.21.15.165:0/1473118837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a80cd4c0 0x7f77a80cd8a0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f77a80af3a0 tx=0x7f779802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0b57f0bd4b9563 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:51.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.443+0000 7f77a6000640 1 -- 172.21.15.165:0/1473118837 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798037070 con 0x7f77a80cd4c0 2024-04-25T10:19:51.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.443+0000 7f77a6000640 1 -- 172.21.15.165:0/1473118837 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7798032440 con 0x7f77a80cd4c0 2024-04-25T10:19:51.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.443+0000 7f77a6000640 1 -- 172.21.15.165:0/1473118837 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7798032810 con 0x7f77a80cd4c0 2024-04-25T10:19:51.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 -- 172.21.15.165:0/1473118837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a80cd4c0 msgr2=0x7f77a80cd8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:51.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 --2- 172.21.15.165:0/1473118837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a80cd4c0 0x7f77a80cd8a0 secure :-1 s=READY pgs=2537 cs=0 l=1 rev1=1 crypto rx=0x7f77a80af3a0 tx=0x7f779802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 -- 172.21.15.165:0/1473118837 shutdown_connections 2024-04-25T10:19:51.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 --2- 172.21.15.165:0/1473118837 >> v2:172.21.15.172:3300/0 conn(0x7f77a815e1a0 0x7f77a8150b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 --2- 172.21.15.165:0/1473118837 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a815b1c0 0x7f77a815dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 --2- 172.21.15.165:0/1473118837 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a80cd4c0 0x7f77a80cd8a0 unknown :-1 s=CLOSED pgs=2537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 -- 172.21.15.165:0/1473118837 >> 172.21.15.165:0/1473118837 conn(0x7f77a806abe0 msgr2=0x7f77a806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:51.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 -- 172.21.15.165:0/1473118837 shutdown_connections 2024-04-25T10:19:51.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.444+0000 7f77ae800640 1 -- 172.21.15.165:0/1473118837 wait complete. 2024-04-25T10:19:51.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.446+0000 7f77ae800640 1 Processor -- start 2024-04-25T10:19:51.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.446+0000 7f77ae800640 1 -- start start 2024-04-25T10:19:51.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.447+0000 7f77ae800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a80cd4c0 0x7f77a815d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.447+0000 7f77ae800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 0x7f77a8159ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.447+0000 7f77ae800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a815e1a0 0x7f77a8149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.447+0000 7f77ae800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806eca0 con 0x7f77a815b1c0 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.447+0000 7f77ae800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a815a670 con 0x7f77a815e1a0 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.447+0000 7f77ae800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a806ee20 con 0x7f77a80cd4c0 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 0x7f77a8159ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 0x7f77a8159ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39998/0 (socket says 172.21.15.165:39998) 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 -- 172.21.15.165:0/1995711749 learned_addr learned my addr 172.21.15.165:0/1995711749 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a7400640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a80cd4c0 0x7f77a815d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 -- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a80cd4c0 msgr2=0x7f77a815d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a80cd4c0 0x7f77a815d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 -- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a815e1a0 msgr2=0x7f77a8149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a7e00640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a815e1a0 0x7f77a8149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a815e1a0 0x7f77a8149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:51.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a6a00640 1 -- 172.21.15.165:0/1995711749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a814a0d0 con 0x7f77a815b1c0 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.448+0000 7f77a7e00640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a815e1a0 0x7f77a8149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.449+0000 7f77a6a00640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 0x7f77a8159ab0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f779000d7d0 tx=0x7f779000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.449+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790018030 con 0x7f77a815b1c0 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.449+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7790004020 con 0x7f77a815b1c0 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.449+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77a80af5f0 con 0x7f77a815b1c0 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.449+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7790004230 con 0x7f77a815b1c0 2024-04-25T10:19:51.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.449+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77a80afbb0 con 0x7f77a815b1c0 2024-04-25T10:19:51.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.450+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77a8073a00 con 0x7f77a815b1c0 2024-04-25T10:19:51.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.451+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f779001b070 con 0x7f77a815b1c0 2024-04-25T10:19:51.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.452+0000 7f779f400640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f778406b4d0 0x7f778406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:51.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.452+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5836+0+0 (secure 0 0 0) 0x7f779008a440 con 0x7f77a815b1c0 2024-04-25T10:19:51.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.452+0000 7f77a7400640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f778406b4d0 0x7f778406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:51.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.453+0000 7f77a7400640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f778406b4d0 0x7f778406d990 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7798034000 tx=0x7f779802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:51.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.457+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7790059240 con 0x7f77a815b1c0 2024-04-25T10:19:51.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:51.650+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f77a80ccb10 con 0x7f77a815b1c0 2024-04-25T10:19:52.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.214+0000 7f779f400640 1 -- 172.21.15.165:0/1995711749 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7790058b40 con 0x7f77a815b1c0 2024-04-25T10:19:52.218 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:19:52.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.218+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f778406b4d0 msgr2=0x7f778406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.218+0000 7f77ae800640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f778406b4d0 0x7f778406d990 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f7798034000 tx=0x7f779802e000 comp rx=0 tx=0).stop 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.218+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 msgr2=0x7f77a8159ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.218+0000 7f77ae800640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 0x7f77a8159ab0 secure :-1 s=READY pgs=2538 cs=0 l=1 rev1=1 crypto rx=0x7f779000d7d0 tx=0x7f779000dc90 comp rx=0 tx=0).stop 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 shutdown_connections 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f778406b4d0 0x7f778406d990 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77a815e1a0 0x7f77a8149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77a815b1c0 0x7f77a8159ab0 unknown :-1 s=CLOSED pgs=2538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 --2- 172.21.15.165:0/1995711749 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77a80cd4c0 0x7f77a815d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 >> 172.21.15.165:0/1995711749 conn(0x7f77a806abe0 msgr2=0x7f77a80b3120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:52.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.219+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 shutdown_connections 2024-04-25T10:19:52.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.222+0000 7f77ae800640 1 -- 172.21.15.165:0/1995711749 wait complete. 2024-04-25T10:19:52.249 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:19:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.439+0000 7f9f95000640 1 Processor -- start 2024-04-25T10:19:52.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.439+0000 7f9f95000640 1 -- start start 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f95000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f90145140 0x7f9f900ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f95000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9f900c5200 0x7f9f9015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f95000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f900c5670 con 0x7f9f900ce950 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f95000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f900ced90 con 0x7f9f900c5200 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f95000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90145520 con 0x7f9f90145140 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9f900c5200 0x7f9f9015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9f900c5200 0x7f9f9015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40012/0 (socket says 172.21.15.165:40012) 2024-04-25T10:19:52.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 -- 172.21.15.165:0/1291246878 learned_addr learned my addr 172.21.15.165:0/1291246878 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8f400640 1 --2- 172.21.15.165:0/1291246878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f90145140 0x7f9f900ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 -- 172.21.15.165:0/1291246878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f90145140 msgr2=0x7f9f900ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 --2- 172.21.15.165:0/1291246878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f90145140 0x7f9f900ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 -- 172.21.15.165:0/1291246878 >> v1:172.21.15.172:6789/0 conn(0x7f9f900ce950 legacy=0x7f9f900c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:52.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.441+0000 7f9f8fe00640 1 -- 172.21.15.165:0/1291246878 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f90071270 con 0x7f9f900c5200 2024-04-25T10:19:52.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.442+0000 7f9f8fe00640 1 --2- 172.21.15.165:0/1291246878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900c5200 0x7f9f9015bfb0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009e50 tx=0x7f9f8802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b60a929a0815f66f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:52.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.442+0000 7f9f8e000640 1 -- 172.21.15.165:0/1291246878 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f88036070 con 0x7f9f900c5200 2024-04-25T10:19:52.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.442+0000 7f9f8e000640 1 -- 172.21.15.165:0/1291246878 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f88031600 con 0x7f9f900c5200 2024-04-25T10:19:52.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.442+0000 7f9f8e000640 1 -- 172.21.15.165:0/1291246878 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f880319d0 con 0x7f9f900c5200 2024-04-25T10:19:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 -- 172.21.15.165:0/1291246878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900c5200 msgr2=0x7f9f9015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 --2- 172.21.15.165:0/1291246878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900c5200 0x7f9f9015bfb0 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f9f88009e50 tx=0x7f9f8802a640 comp rx=0 tx=0).stop 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 -- 172.21.15.165:0/1291246878 shutdown_connections 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 --2- 172.21.15.165:0/1291246878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900c5200 0x7f9f9015bfb0 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 --2- 172.21.15.165:0/1291246878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f90145140 0x7f9f900ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 -- 172.21.15.165:0/1291246878 >> 172.21.15.165:0/1291246878 conn(0x7f9f90061cc0 msgr2=0x7f9f9006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 -- 172.21.15.165:0/1291246878 shutdown_connections 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.443+0000 7f9f95000640 1 -- 172.21.15.165:0/1291246878 wait complete. 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.445+0000 7f9f95000640 1 Processor -- start 2024-04-25T10:19:52.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.445+0000 7f9f95000640 1 -- start start 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f95000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f900c5200 0x7f9f900ad650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f95000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 0x7f9f9015b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f95000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f90145140 0x7f9f9015d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f95000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90071b10 con 0x7f9f900ce950 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f95000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f90071830 con 0x7f9f90145140 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f95000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f9015c4f0 con 0x7f9f900c5200 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f8ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 0x7f9f9015b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f8ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 0x7f9f9015b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40028/0 (socket says 172.21.15.165:40028) 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.446+0000 7f9f8ea00640 1 -- 172.21.15.165:0/2783509500 learned_addr learned my addr 172.21.15.165:0/2783509500 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:52.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8f400640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f900c5200 0x7f9f900ad650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8fe00640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f90145140 0x7f9f9015d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8ea00640 1 -- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f900c5200 msgr2=0x7f9f900ad650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8ea00640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f900c5200 0x7f9f900ad650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8ea00640 1 -- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f90145140 msgr2=0x7f9f9015d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8ea00640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f90145140 0x7f9f9015d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8ea00640 1 -- 172.21.15.165:0/2783509500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9015b820 con 0x7f9f900ce950 2024-04-25T10:19:52.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.447+0000 7f9f8ea00640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 0x7f9f9015b2e0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c430 tx=0x7f9f7800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:52.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.448+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78011790 con 0x7f9f900ce950 2024-04-25T10:19:52.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.448+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f78017070 con 0x7f9f900ce950 2024-04-25T10:19:52.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.448+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f78011bb0 con 0x7f9f900ce950 2024-04-25T10:19:52.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.448+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f900ac8b0 con 0x7f9f900ce950 2024-04-25T10:19:52.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.448+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f900acb70 con 0x7f9f900ce950 2024-04-25T10:19:52.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.449+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f90073990 con 0x7f9f900ce950 2024-04-25T10:19:52.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.450+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9f7801c460 con 0x7f9f900ce950 2024-04-25T10:19:52.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.451+0000 7f9f87400640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f6c06b500 0x7f9f6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.451+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9f78089f30 con 0x7f9f900ce950 2024-04-25T10:19:52.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.451+0000 7f9f8f400640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f6c06b500 0x7f9f6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.452+0000 7f9f8f400640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f6c06b500 0x7f9f6c06d9c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:52.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.455+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9f78058f10 con 0x7f9f900ce950 2024-04-25T10:19:52.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.668+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9f900b3fc0 con 0x7f9f900ce950 2024-04-25T10:19:52.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.675+0000 7f9f87400640 1 -- 172.21.15.165:0/2783509500 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v31) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9f78058810 con 0x7f9f900ce950 2024-04-25T10:19:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f6c06b500 msgr2=0x7f9f6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f6c06b500 0x7f9f6c06d9c0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f9f80006fd0 tx=0x7f9f80008040 comp rx=0 tx=0).stop 2024-04-25T10:19:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 msgr2=0x7f9f9015b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 0x7f9f9015b2e0 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f9f7800c430 tx=0x7f9f7800c8f0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 shutdown_connections 2024-04-25T10:19:52.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9f6c06b500 0x7f9f6c06d9c0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9f90145140 0x7f9f9015d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9f900ce950 0x7f9f9015b2e0 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 --2- 172.21.15.165:0/2783509500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9f900c5200 0x7f9f900ad650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 >> 172.21.15.165:0/2783509500 conn(0x7f9f90061cc0 msgr2=0x7f9f900c4af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 shutdown_connections 2024-04-25T10:19:52.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.679+0000 7f9f95000640 1 -- 172.21.15.165:0/2783509500 wait complete. 2024-04-25T10:19:52.700 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace' 2024-04-25T10:19:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.882+0000 7fadbf200640 1 Processor -- start 2024-04-25T10:19:52.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.882+0000 7fadbf200640 1 -- start start 2024-04-25T10:19:52.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbf200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fadc015b6b0 0x7fadc00b3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbf200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fadc015ecc0 0x7fadc0070fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbf200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadc0071910 con 0x7fadc00ae520 2024-04-25T10:19:52.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbf200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc00b2470 con 0x7fadc015ecc0 2024-04-25T10:19:52.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbf200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadc00b2750 con 0x7fadc015b6b0 2024-04-25T10:19:52.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbd400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fadc015ecc0 0x7fadc0070fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbd400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fadc015ecc0 0x7fadc0070fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40042/0 (socket says 172.21.15.165:40042) 2024-04-25T10:19:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.884+0000 7fadbd400640 1 -- 172.21.15.165:0/2304123706 learned_addr learned my addr 172.21.15.165:0/2304123706 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.885+0000 7fadbd400640 1 -- 172.21.15.165:0/2304123706 >> v2:172.21.15.172:3300/0 conn(0x7fadc015b6b0 msgr2=0x7fadc00b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.885+0000 7fadbd400640 1 --2- 172.21.15.165:0/2304123706 >> v2:172.21.15.172:3300/0 conn(0x7fadc015b6b0 0x7fadc00b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.885+0000 7fadbd400640 1 -- 172.21.15.165:0/2304123706 >> v1:172.21.15.172:6789/0 conn(0x7fadc00ae520 legacy=0x7fadc00acb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.885+0000 7fadbd400640 1 -- 172.21.15.165:0/2304123706 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00b2a30 con 0x7fadc015ecc0 2024-04-25T10:19:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.886+0000 7fadbd400640 1 --2- 172.21.15.165:0/2304123706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc015ecc0 0x7fadc0070fa0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fadc0073c60 tx=0x7fadac02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620e8a0948d82e6d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.886+0000 7fadbca00640 1 -- 172.21.15.165:0/2304123706 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac037070 con 0x7fadc015ecc0 2024-04-25T10:19:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.886+0000 7fadbca00640 1 -- 172.21.15.165:0/2304123706 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadac032440 con 0x7fadc015ecc0 2024-04-25T10:19:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.886+0000 7fadbca00640 1 -- 172.21.15.165:0/2304123706 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadac032810 con 0x7fadc015ecc0 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 -- 172.21.15.165:0/2304123706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc015ecc0 msgr2=0x7fadc0070fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 --2- 172.21.15.165:0/2304123706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc015ecc0 0x7fadc0070fa0 secure :-1 s=READY pgs=2541 cs=0 l=1 rev1=1 crypto rx=0x7fadc0073c60 tx=0x7fadac02a230 comp rx=0 tx=0).stop 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 -- 172.21.15.165:0/2304123706 shutdown_connections 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 --2- 172.21.15.165:0/2304123706 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc015ecc0 0x7fadc0070fa0 unknown :-1 s=CLOSED pgs=2541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 --2- 172.21.15.165:0/2304123706 >> v2:172.21.15.172:3300/0 conn(0x7fadc015b6b0 0x7fadc00b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 -- 172.21.15.165:0/2304123706 >> 172.21.15.165:0/2304123706 conn(0x7fadc00b1170 msgr2=0x7fadc00b1550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 -- 172.21.15.165:0/2304123706 shutdown_connections 2024-04-25T10:19:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 -- 172.21.15.165:0/2304123706 wait complete. 2024-04-25T10:19:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.887+0000 7fadbf200640 1 Processor -- start 2024-04-25T10:19:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.889+0000 7fadbf200640 1 -- start start 2024-04-25T10:19:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbf200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 0x7fadc006d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbf200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadc015b6b0 0x7fadc006d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbf200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadc015ecc0 0x7fadc00cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbf200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc0071660 con 0x7fadc00ae520 2024-04-25T10:19:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbf200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00714e0 con 0x7fadc015b6b0 2024-04-25T10:19:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbf200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadc00ae9d0 con 0x7fadc015ecc0 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 0x7fadc006d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 0x7fadc006d000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40046/0 (socket says 172.21.15.165:40046) 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 -- 172.21.15.165:0/3636314290 learned_addr learned my addr 172.21.15.165:0/3636314290 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbd400640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadc015b6b0 0x7fadc006d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 -- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadc015ecc0 msgr2=0x7fadc00cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadc015ecc0 0x7fadc00cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 -- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadc015b6b0 msgr2=0x7fadc006d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadc015b6b0 0x7fadc006d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.890+0000 7fadbde00640 1 -- 172.21.15.165:0/3636314290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc00cdc30 con 0x7fadc00ae520 2024-04-25T10:19:52.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.891+0000 7fadbde00640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 0x7fadc006d000 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fadb000b4d0 tx=0x7fadb000b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:52.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.891+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0014070 con 0x7fadc00ae520 2024-04-25T10:19:52.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.891+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0010870 con 0x7fadc00ae520 2024-04-25T10:19:52.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.891+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadc0156d70 con 0x7fadc00ae520 2024-04-25T10:19:52.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.891+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0010c60 con 0x7fadc00ae520 2024-04-25T10:19:52.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.891+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadc015d7a0 con 0x7fadc00ae520 2024-04-25T10:19:52.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.892+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadc015d9e0 con 0x7fadc00ae520 2024-04-25T10:19:52.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.893+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fadb0017070 con 0x7fadc00ae520 2024-04-25T10:19:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.893+0000 7fadb6000640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fada006b4d0 0x7fada006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.893+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fadb0089ec0 con 0x7fadc00ae520 2024-04-25T10:19:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.893+0000 7fadbd400640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fada006b4d0 0x7fada006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.894+0000 7fadbd400640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fada006b4d0 0x7fada006d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fadac034000 tx=0x7fadac02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:52.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:52.898+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fadb0058ea0 con 0x7fadc00ae520 2024-04-25T10:19:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.168+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]} v 0) v1 -- 0x7fadc0073990 con 0x7fadc00ae520 2024-04-25T10:19:53.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.234+0000 7fadb6000640 1 -- 172.21.15.165:0/3636314290 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace"]}]=0 v817) v1 ==== 133+0+0 (secure 0 0 0) 0x7fadb00587a0 con 0x7fadc00ae520 2024-04-25T10:19:53.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.237+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fada006b4d0 msgr2=0x7fada006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.237+0000 7fadbf200640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fada006b4d0 0x7fada006d990 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fadac034000 tx=0x7fadac02e000 comp rx=0 tx=0).stop 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 msgr2=0x7fadc006d000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 0x7fadc006d000 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7fadb000b4d0 tx=0x7fadb000b990 comp rx=0 tx=0).stop 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 shutdown_connections 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fada006b4d0 0x7fada006d990 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadc015ecc0 0x7fadc00cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadc015b6b0 0x7fadc006d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 --2- 172.21.15.165:0/3636314290 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadc00ae520 0x7fadc006d000 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 >> 172.21.15.165:0/3636314290 conn(0x7fadc00b1170 msgr2=0x7fadc00ad3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 shutdown_connections 2024-04-25T10:19:53.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.238+0000 7fadbf200640 1 -- 172.21.15.165:0/3636314290 wait complete. 2024-04-25T10:19:53.260 INFO:tasks.cephfs_test_runner:test_scrub_backtrace (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:19:53.260 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:19:53.260 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:53.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.456+0000 7f0f2ec00640 1 Processor -- start 2024-04-25T10:19:53.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.456+0000 7f0f2ec00640 1 -- start start 2024-04-25T10:19:53.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.458+0000 7f0f2ec00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0f280ce920 0x7f0f280cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.458+0000 7f0f2ec00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f28144b30 con 0x7f0f2806d290 2024-04-25T10:19:53.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.458+0000 7f0f2ec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f280712d0 con 0x7f0f28144750 2024-04-25T10:19:53.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.458+0000 7f0f2ec00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f280715b0 con 0x7f0f280ce920 2024-04-25T10:19:53.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.458+0000 7f0f2cc00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0f2806d290 0x7f0f2806d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60660/0 (socket says 172.21.15.165:60660) 2024-04-25T10:19:53.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.458+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3667485214 learned_addr learned my addr 172.21.15.165:0/3667485214 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:53.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f27400640 1 --2- 172.21.15.165:0/3667485214 >> v2:172.21.15.172:3300/0 conn(0x7f0f280ce920 0x7f0f280cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020000733 0 0) 0x7f0f28144b30 con 0x7f0f2806d290 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f0c003520 con 0x7f0f2806d290 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 723241989 0 0) 0x7f0f0c003520 con 0x7f0f2806d290 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 >> v2:172.21.15.172:3300/0 conn(0x7f0f280ce920 msgr2=0x7f0f280cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 --2- 172.21.15.165:0/3667485214 >> v2:172.21.15.172:3300/0 conn(0x7f0f280ce920 0x7f0f280cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 >> v1:172.21.15.172:6789/0 conn(0x7f0f28144750 legacy=0x7f0f2806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28071890 con 0x7f0f2806d290 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f0f200019f0 con 0x7f0f2806d290 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 >> v1:172.21.15.165:6789/0 conn(0x7f0f2806d290 legacy=0x7f0f2806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f0c0039b0 0x7f0f0c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c004800 0x7f0f0c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f0c0051c0 0x7f0f0c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280715b0 con 0x7f0f0c0051c0 2024-04-25T10:19:53.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c060 con 0x7f0f0c0039b0 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.459+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f0c00c260 con 0x7f0f0c004800 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f0c0051c0 0x7f0f0c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f27e00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f0c0039b0 0x7f0f0c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f27400640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c004800 0x7f0f0c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c004800 msgr2=0x7f0f0c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c004800 0x7f0f0c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f0c0039b0 msgr2=0x7f0f0c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f0c0039b0 0x7f0f0c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.460+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3667485214 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f0c003520 con 0x7f0f0c0051c0 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.461+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3667485214 reap_dead start 2024-04-25T10:19:53.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.461+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f0c0051c0 0x7f0f0c00b9c0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000a830 tx=0x7f0f0c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8467112903c952e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:53.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.461+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2000a480 con 0x7f0f0c0051c0 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.461+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2002f070 con 0x7f0f0c0051c0 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.461+0000 7f0f26a00640 1 -- 172.21.15.165:0/3667485214 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2002bcb0 con 0x7f0f0c0051c0 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3667485214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f0c0051c0 msgr2=0x7f0f0c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f0c0051c0 0x7f0f0c00b9c0 secure :-1 s=READY pgs=2544 cs=0 l=1 rev1=1 crypto rx=0x7f0f2000a830 tx=0x7f0f0c005830 comp rx=0 tx=0).stop 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3667485214 shutdown_connections 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3667485214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f0c0051c0 0x7f0f0c00b9c0 unknown :-1 s=CLOSED pgs=2544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3667485214 >> 172.21.15.165:0/3667485214 conn(0x7f0f2806abe0 msgr2=0x7f0f2806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3667485214 shutdown_connections 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.462+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3667485214 wait complete. 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.464+0000 7f0f2ec00640 1 Processor -- start 2024-04-25T10:19:53.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.464+0000 7f0f2ec00640 1 -- start start 2024-04-25T10:19:53.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c0051c0 0x7f0f2815b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f280b3690 0x7f0f280b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 0x7f0f281f4820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f281455d0 con 0x7f0f281f43a0 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28145450 con 0x7f0f280b3690 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2815c000 con 0x7f0f0c0051c0 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 0x7f0f281f4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 0x7f0f281f4820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40064/0 (socket says 172.21.15.165:40064) 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.465+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3537386110 learned_addr learned my addr 172.21.15.165:0/3537386110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f27e00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c0051c0 0x7f0f2815b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f27400640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f280b3690 0x7f0f280b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c0051c0 msgr2=0x7f0f2815b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c0051c0 0x7f0f2815b330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f280b3690 msgr2=0x7f0f280b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f280b3690 0x7f0f280b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f2cc00640 1 -- 172.21.15.165:0/3537386110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f280ad630 con 0x7f0f281f43a0 2024-04-25T10:19:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f2cc00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 0x7f0f281f4820 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000f80 tx=0x7f0f20001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:53.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.466+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2002b850 con 0x7f0f281f43a0 2024-04-25T10:19:53.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.467+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f2002b9f0 con 0x7f0f281f43a0 2024-04-25T10:19:53.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.467+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f2002bdc0 con 0x7f0f281f43a0 2024-04-25T10:19:53.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.467+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f280ad910 con 0x7f0f281f43a0 2024-04-25T10:19:53.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.467+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f280aded0 con 0x7f0f281f43a0 2024-04-25T10:19:53.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.468+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f281f4090 con 0x7f0f281f43a0 2024-04-25T10:19:53.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.469+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f20034470 con 0x7f0f281f43a0 2024-04-25T10:19:53.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.470+0000 7f0f1fe00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b500 0x7f0f0806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.470+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f0f200a9d10 con 0x7f0f281f43a0 2024-04-25T10:19:53.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.470+0000 7f0f27e00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b500 0x7f0f0806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.471+0000 7f0f27e00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b500 0x7f0f0806d9c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0f18000c00 tx=0x7f0f18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:53.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.474+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f200af020 con 0x7f0f281f43a0 2024-04-25T10:19:53.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.671+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0f28073990 con 0x7f0f281f43a0 2024-04-25T10:19:53.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.673+0000 7f0f1fe00640 1 -- 172.21.15.165:0/3537386110 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v176) v1 ==== 74+0+13341 (secure 0 0 0) 0x7f0f20047030 con 0x7f0f281f43a0 2024-04-25T10:19:53.677 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:53.677 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":176,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:52.211702+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6836/1299523073":"2024-04-26T10:19:49.194284+0000","172.21.15.165:6834/1299523073":"2024-04-26T10:19:49.194284+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:53.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.676+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b500 msgr2=0x7f0f0806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.676+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b500 0x7f0f0806d9c0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f0f18000c00 tx=0x7f0f18009040 comp rx=0 tx=0).stop 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.676+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 msgr2=0x7f0f281f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.676+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 0x7f0f281f4820 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7f0f20000f80 tx=0x7f0f20001360 comp rx=0 tx=0).stop 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 shutdown_connections 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b500 0x7f0f0806d9c0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f281f43a0 0x7f0f281f4820 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f280b3690 0x7f0f280b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 --2- 172.21.15.165:0/3537386110 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f0c0051c0 0x7f0f2815b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 >> 172.21.15.165:0/3537386110 conn(0x7f0f2806abe0 msgr2=0x7f0f2815c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 shutdown_connections 2024-04-25T10:19:53.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.677+0000 7f0f2ec00640 1 -- 172.21.15.165:0/3537386110 wait complete. 2024-04-25T10:19:53.698 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:53.698 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:53.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.892+0000 7f65bb000640 1 Processor -- start 2024-04-25T10:19:53.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.892+0000 7f65bb000640 1 -- start start 2024-04-25T10:19:53.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.893+0000 7f65bb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 0x7f65b413df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.893+0000 7f65bb000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f65b40b0730 0x7f65b40b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.893+0000 7f65bb000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65b406d390 con 0x7f65b406e150 2024-04-25T10:19:53.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.893+0000 7f65bb000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65b40b10d0 con 0x7f65b40b0730 2024-04-25T10:19:53.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.893+0000 7f65bb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b406d670 con 0x7f65b413d9f0 2024-04-25T10:19:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65ba600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f65b406e150 0x7f65b406e5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60662/0 (socket says 172.21.15.165:60662) 2024-04-25T10:19:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65ba600640 1 -- 172.21.15.165:0/1435823599 learned_addr learned my addr 172.21.15.165:0/1435823599 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b9c00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 0x7f65b413df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1973912651 0 0) 0x7f65b406d390 con 0x7f65b406e150 2024-04-25T10:19:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f659c003520 con 0x7f65b406e150 2024-04-25T10:19:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b9200640 1 --2- 172.21.15.165:0/1435823599 >> v2:172.21.15.172:3300/0 conn(0x7f65b40b0730 0x7f65b40b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1979265675 0 0) 0x7f659c003520 con 0x7f65b406e150 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 msgr2=0x7f65b413df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 0x7f65b413df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 >> v2:172.21.15.172:3300/0 conn(0x7f65b40b0730 msgr2=0x7f65b40b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 --2- 172.21.15.165:0/1435823599 >> v2:172.21.15.172:3300/0 conn(0x7f65b40b0730 0x7f65b40b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b406d950 con 0x7f65b406e150 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.894+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f65ac0019f0 con 0x7f65b406e150 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 >> v1:172.21.15.165:6789/0 conn(0x7f65b406e150 legacy=0x7f65b406e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f659c0039b0 0x7f659c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f659c004800 0x7f659c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 0x7f659c005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b406d670 con 0x7f659c0039b0 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659c00bb50 con 0x7f659c004800 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659c00bd20 con 0x7f65b413d9f0 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b9c00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f659c0039b0 0x7f659c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65ba600640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 0x7f659c005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b9c00640 1 -- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 msgr2=0x7f659c005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b9c00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f65b413d9f0 0x7f659c005270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b9c00640 1 -- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f659c004800 msgr2=0x7f659c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:53.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b9c00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f659c004800 0x7f659c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.895+0000 7f65b9c00640 1 -- 172.21.15.165:0/1435823599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f659c003520 con 0x7f659c0039b0 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.896+0000 7f65b9c00640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f659c0039b0 0x7f659c003dd0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f65a800ba30 tx=0x7f659c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d894327cf0affdc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.897+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a8030070 con 0x7f659c0039b0 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.897+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a802ade0 con 0x7f659c0039b0 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.897+0000 7f65b3e00640 1 -- 172.21.15.165:0/1435823599 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a800b600 con 0x7f659c0039b0 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.897+0000 7f65bb000640 1 -- 172.21.15.165:0/1435823599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f659c0039b0 msgr2=0x7f659c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.897+0000 7f65bb000640 1 --2- 172.21.15.165:0/1435823599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f659c0039b0 0x7f659c003dd0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7f65a800ba30 tx=0x7f659c005830 comp rx=0 tx=0).stop 2024-04-25T10:19:53.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.897+0000 7f65ba600640 1 -- 172.21.15.165:0/1435823599 reap_dead start 2024-04-25T10:19:53.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.898+0000 7f65bb000640 1 -- 172.21.15.165:0/1435823599 shutdown_connections 2024-04-25T10:19:53.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.898+0000 7f65bb000640 1 -- 172.21.15.165:0/1435823599 >> 172.21.15.165:0/1435823599 conn(0x7f65b4061cc0 msgr2=0x7f65b406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:53.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.898+0000 7f65bb000640 1 -- 172.21.15.165:0/1435823599 shutdown_connections 2024-04-25T10:19:53.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.898+0000 7f65bb000640 1 -- 172.21.15.165:0/1435823599 wait complete. 2024-04-25T10:19:53.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.899+0000 7f65bb000640 1 Processor -- start 2024-04-25T10:19:53.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.900+0000 7f65bb000640 1 -- start start 2024-04-25T10:19:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.900+0000 7f65bb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f659c0039b0 0x7f65b4159d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.900+0000 7f65bb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 0x7f65b4138290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.900+0000 7f65bb000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b413ebf0 0x7f65b413f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.900+0000 7f65bb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b413e0b0 con 0x7f65b413cc00 2024-04-25T10:19:53.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65bb000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65b413df30 con 0x7f65b413ebf0 2024-04-25T10:19:53.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65bb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65b413e230 con 0x7f659c0039b0 2024-04-25T10:19:53.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 0x7f65b4138290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 0x7f65b4138290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40084/0 (socket says 172.21.15.165:40084) 2024-04-25T10:19:53.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 -- 172.21.15.165:0/820385712 learned_addr learned my addr 172.21.15.165:0/820385712 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9c00640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f659c0039b0 0x7f65b4159d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 -- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f659c0039b0 msgr2=0x7f65b4159d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65ba600640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b413ebf0 0x7f65b413f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f659c0039b0 0x7f65b4159d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 -- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b413ebf0 msgr2=0x7f65b413f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b413ebf0 0x7f65b413f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.901+0000 7f65b9200640 1 -- 172.21.15.165:0/820385712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65b4166600 con 0x7f65b413cc00 2024-04-25T10:19:53.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.902+0000 7f65b9200640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 0x7f65b4138290 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f65a000d830 tx=0x7f65a000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.902+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000b660 con 0x7f65b413cc00 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.902+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a0019070 con 0x7f65b413cc00 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.902+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a000ba40 con 0x7f65b413cc00 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.902+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65b41668e0 con 0x7f65b413cc00 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.902+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65b41387d0 con 0x7f65b413cc00 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.903+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65b4073990 con 0x7f65b413cc00 2024-04-25T10:19:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.904+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f65a001e460 con 0x7f65b413cc00 2024-04-25T10:19:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.905+0000 7f65b1600640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f659406b4d0 0x7f659406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.905+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f65a008a660 con 0x7f65b413cc00 2024-04-25T10:19:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.905+0000 7f65b9c00640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f659406b4d0 0x7f659406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.906+0000 7f65b9c00640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f659406b4d0 0x7f659406d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f65a800ba00 tx=0x7f65a8030000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:53.910+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f65a00595b0 con 0x7f65b413cc00 2024-04-25T10:19:54.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.095+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f65b41587b0 con 0x7f65b413cc00 2024-04-25T10:19:54.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.096+0000 7f65b1600640 1 -- 172.21.15.165:0/820385712 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v176) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65a0058eb0 con 0x7f65b413cc00 2024-04-25T10:19:54.099 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f659406b4d0 msgr2=0x7f659406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f659406b4d0 0x7f659406d990 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f65a800ba00 tx=0x7f65a8030000 comp rx=0 tx=0).stop 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 msgr2=0x7f65b4138290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 0x7f65b4138290 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7f65a000d830 tx=0x7f65a000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 shutdown_connections 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f659406b4d0 0x7f659406d990 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f65b413ebf0 0x7f65b413f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f65b413cc00 0x7f65b4138290 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 --2- 172.21.15.165:0/820385712 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f659c0039b0 0x7f65b4159d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 >> 172.21.15.165:0/820385712 conn(0x7f65b4061cc0 msgr2=0x7f65b4158a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:54.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 shutdown_connections 2024-04-25T10:19:54.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.098+0000 7f65bb000640 1 -- 172.21.15.165:0/820385712 wait complete. 2024-04-25T10:19:54.118 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:19:54.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.310+0000 7f643b400640 1 Processor -- start 2024-04-25T10:19:54.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.310+0000 7f643b400640 1 -- start start 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f643b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64340ce920 0x7f64340ced80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f643b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f643406d290 0x7f643406d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f643b400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6434144b30 con 0x7f6434144750 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f643b400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64340712d0 con 0x7f643406d290 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f643b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340715b0 con 0x7f64340ce920 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f6439400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f643406d290 0x7f643406d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f6439400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f643406d290 0x7f643406d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40094/0 (socket says 172.21.15.165:40094) 2024-04-25T10:19:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.312+0000 7f6439400640 1 -- 172.21.15.165:0/727455543 learned_addr learned my addr 172.21.15.165:0/727455543 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:54.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.313+0000 7f6433e00640 1 --2- 172.21.15.165:0/727455543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64340ce920 0x7f64340ced80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:54.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.313+0000 7f6439400640 1 -- 172.21.15.165:0/727455543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64340ce920 msgr2=0x7f64340ced80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:54.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.313+0000 7f6439400640 1 --2- 172.21.15.165:0/727455543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64340ce920 0x7f64340ced80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.313+0000 7f6439400640 1 -- 172.21.15.165:0/727455543 >> v1:172.21.15.172:6789/0 conn(0x7f6434144750 legacy=0x7f643406e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:54.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.313+0000 7f6439400640 1 -- 172.21.15.165:0/727455543 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6434071890 con 0x7f643406d290 2024-04-25T10:19:54.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.314+0000 7f6439400640 1 --2- 172.21.15.165:0/727455543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f643406d720 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f642c009870 tx=0x7f642c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f4a7671464e1a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:54.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.314+0000 7f6433400640 1 -- 172.21.15.165:0/727455543 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c037070 con 0x7f643406d290 2024-04-25T10:19:54.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.314+0000 7f6433400640 1 -- 172.21.15.165:0/727455543 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642c032440 con 0x7f643406d290 2024-04-25T10:19:54.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.314+0000 7f6433400640 1 -- 172.21.15.165:0/727455543 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f642c032810 con 0x7f643406d290 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 -- 172.21.15.165:0/727455543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 msgr2=0x7f643406d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 --2- 172.21.15.165:0/727455543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f643406d720 secure :-1 s=READY pgs=2549 cs=0 l=1 rev1=1 crypto rx=0x7f642c009870 tx=0x7f642c02a270 comp rx=0 tx=0).stop 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 -- 172.21.15.165:0/727455543 shutdown_connections 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 --2- 172.21.15.165:0/727455543 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f643406d720 unknown :-1 s=CLOSED pgs=2549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 --2- 172.21.15.165:0/727455543 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f64340ce920 0x7f64340ced80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 -- 172.21.15.165:0/727455543 >> 172.21.15.165:0/727455543 conn(0x7f643406abe0 msgr2=0x7f643406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 -- 172.21.15.165:0/727455543 shutdown_connections 2024-04-25T10:19:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.315+0000 7f643b400640 1 -- 172.21.15.165:0/727455543 wait complete. 2024-04-25T10:19:54.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.317+0000 7f643b400640 1 Processor -- start 2024-04-25T10:19:54.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.318+0000 7f643b400640 1 -- start start 2024-04-25T10:19:54.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f643b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f64341592e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:54.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f643b400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64340ce920 0x7f6434138860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:54.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f643b400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6434144750 0x7f643413f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:54.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f643b400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643406f020 con 0x7f643406d290 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f643b400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643406ece0 con 0x7f64340ce920 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f643b400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6434145450 con 0x7f6434144750 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6438a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f64341592e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6438a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f64341592e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40110/0 (socket says 172.21.15.165:40110) 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6438a00640 1 -- 172.21.15.165:0/3472476811 learned_addr learned my addr 172.21.15.165:0/3472476811 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6439400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6434144750 0x7f643413f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6438a00640 1 -- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6434144750 msgr2=0x7f643413f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6433e00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64340ce920 0x7f6434138860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.319+0000 7f6438a00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6434144750 0x7f643413f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6438a00640 1 -- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64340ce920 msgr2=0x7f6434138860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6438a00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64340ce920 0x7f6434138860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6438a00640 1 -- 172.21.15.165:0/3472476811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64341390b0 con 0x7f643406d290 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6439400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6434144750 0x7f643413f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6438a00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f64341592e0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f642400c790 tx=0x7f642400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:54.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424016020 con 0x7f643406d290 2024-04-25T10:19:54.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6424006d50 con 0x7f643406d290 2024-04-25T10:19:54.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6434139390 con 0x7f643406d290 2024-04-25T10:19:54.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6424003bb0 con 0x7f643406d290 2024-04-25T10:19:54.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.320+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64341f18c0 con 0x7f643406d290 2024-04-25T10:19:54.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.322+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64341f1ac0 con 0x7f643406d290 2024-04-25T10:19:54.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.322+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f642400a850 con 0x7f643406d290 2024-04-25T10:19:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.323+0000 7f6430c00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f641406b4d0 0x7f641406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.324+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f6424089da0 con 0x7f643406d290 2024-04-25T10:19:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.324+0000 7f6433e00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f641406b4d0 0x7f641406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.324+0000 7f6433e00640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f641406b4d0 0x7f641406d990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6434160300 tx=0x7f6420008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.328+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6424020020 con 0x7f643406d290 2024-04-25T10:19:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:54.598+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f6434073990 con 0x7f643406d290 2024-04-25T10:19:55.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.240+0000 7f6430c00640 1 -- 172.21.15.165:0/3472476811 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v819) v1 ==== 150+0+0 (secure 0 0 0) 0x7f6424019310 con 0x7f643406d290 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f641406b4d0 msgr2=0x7f641406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f641406b4d0 0x7f641406d990 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f6434160300 tx=0x7f6420008040 comp rx=0 tx=0).stop 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 msgr2=0x7f64341592e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f64341592e0 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7f642400c790 tx=0x7f642400cc50 comp rx=0 tx=0).stop 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 shutdown_connections 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f641406b4d0 0x7f641406d990 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6434144750 0x7f643413f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f64340ce920 0x7f6434138860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 --2- 172.21.15.165:0/3472476811 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f643406d290 0x7f64341592e0 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 >> 172.21.15.165:0/3472476811 conn(0x7f643406abe0 msgr2=0x7f6434140250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.244+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 shutdown_connections 2024-04-25T10:19:55.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.245+0000 7f643b400640 1 -- 172.21.15.165:0/3472476811 wait complete. 2024-04-25T10:19:55.268 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:19:55.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.463+0000 7f9b73200640 1 Processor -- start 2024-04-25T10:19:55.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.463+0000 7f9b73200640 1 -- start start 2024-04-25T10:19:55.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b73200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 0x7f9b740ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b73200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9b740c5200 0x7f9b7415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b73200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b740c5670 con 0x7f9b740ce950 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b73200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b740ced90 con 0x7f9b740c5200 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b73200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b74145520 con 0x7f9b74145140 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b6be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9b740ce950 0x7f9b740c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60664/0 (socket says 172.21.15.165:60664) 2024-04-25T10:19:55.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b6be00640 1 -- 172.21.15.165:0/901665778 learned_addr learned my addr 172.21.15.165:0/901665778 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:55.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b71e00640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 0x7f9b740ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3929959737 0 0) 0x7f9b740c5670 con 0x7f9b740ce950 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b72800640 1 --2- 172.21.15.165:0/901665778 >> v2:172.21.15.172:3300/0 conn(0x7f9b740c5200 0x7f9b7415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.465+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b58003520 con 0x7f9b740ce950 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3228736950 0 0) 0x7f9b58003520 con 0x7f9b740ce950 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 msgr2=0x7f9b740ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 0x7f9b740ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 >> v2:172.21.15.172:3300/0 conn(0x7f9b740c5200 msgr2=0x7f9b7415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 --2- 172.21.15.165:0/901665778 >> v2:172.21.15.172:3300/0 conn(0x7f9b740c5200 0x7f9b7415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b74071270 con 0x7f9b740ce950 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9b5c001670 con 0x7f9b740ce950 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 >> v1:172.21.15.165:6789/0 conn(0x7f9b740ce950 legacy=0x7f9b740c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 0x7f9b580039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b58004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b58004d40 0x7f9b580051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740ced90 con 0x7f9b58003f30 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b74145520 con 0x7f9b58004d40 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5800bac0 con 0x7f9b74145140 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.466+0000 7f9b6be00640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b58004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.467+0000 7f9b6be00640 1 -- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 msgr2=0x7f9b580039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.467+0000 7f9b6be00640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b74145140 0x7f9b580039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.467+0000 7f9b6be00640 1 -- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b58004d40 msgr2=0x7f9b580051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.467+0000 7f9b6be00640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b58004d40 0x7f9b580051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.467+0000 7f9b6be00640 1 -- 172.21.15.165:0/901665778 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b58003520 con 0x7f9b58003f30 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b6be00640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b58004800 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c210 tx=0x7f9b58005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=244e5b181744457a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c007d50 con 0x7f9b58003f30 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5c005ad0 con 0x7f9b58003f30 2024-04-25T10:19:55.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b71400640 1 -- 172.21.15.165:0/901665778 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5c007070 con 0x7f9b58003f30 2024-04-25T10:19:55.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b73200640 1 -- 172.21.15.165:0/901665778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 msgr2=0x7f9b58004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b73200640 1 --2- 172.21.15.165:0/901665778 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b58004800 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c00c210 tx=0x7f9b58005830 comp rx=0 tx=0).stop 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.468+0000 7f9b72800640 1 -- 172.21.15.165:0/901665778 reap_dead start 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.469+0000 7f9b73200640 1 -- 172.21.15.165:0/901665778 shutdown_connections 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.469+0000 7f9b73200640 1 -- 172.21.15.165:0/901665778 >> 172.21.15.165:0/901665778 conn(0x7f9b74061cc0 msgr2=0x7f9b7406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.469+0000 7f9b73200640 1 -- 172.21.15.165:0/901665778 shutdown_connections 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.469+0000 7f9b73200640 1 -- 172.21.15.165:0/901665778 wait complete. 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.470+0000 7f9b73200640 1 Processor -- start 2024-04-25T10:19:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.471+0000 7f9b73200640 1 -- start start 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.471+0000 7f9b73200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b7415ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.471+0000 7f9b73200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b7415d910 0x7f9b7415dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b73200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b741e4090 0x7f9b741e4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b73200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b74071b10 con 0x7f9b58003f30 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b73200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b74071830 con 0x7f9b741e4090 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b73200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b740ce410 con 0x7f9b7415d910 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b7415ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b7415ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40124/0 (socket says 172.21.15.165:40124) 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 -- 172.21.15.165:0/1879168698 learned_addr learned my addr 172.21.15.165:0/1879168698 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b6be00640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b7415d910 0x7f9b7415dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:55.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 -- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b7415d910 msgr2=0x7f9b7415dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b7415d910 0x7f9b7415dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 -- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b741e4090 msgr2=0x7f9b741e4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b741e4090 0x7f9b741e4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.472+0000 7f9b71e00640 1 -- 172.21.15.165:0/1879168698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b7415e3d0 con 0x7f9b58003f30 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.473+0000 7f9b71e00640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b7415ae40 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400e7c0 tx=0x7f9b6400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.473+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b64017070 con 0x7f9b58003f30 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.473+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b6400c940 con 0x7f9b58003f30 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.473+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b6400cd10 con 0x7f9b58003f30 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.473+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b7415e6b0 con 0x7f9b58003f30 2024-04-25T10:19:55.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.473+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b7413ef40 con 0x7f9b58003f30 2024-04-25T10:19:55.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.474+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b7413ea50 con 0x7f9b58003f30 2024-04-25T10:19:55.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.475+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9b6401a070 con 0x7f9b58003f30 2024-04-25T10:19:55.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.476+0000 7f9b6a000640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b5006b4d0 0x7f9b5006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:55.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.476+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f9b6408a2b0 con 0x7f9b58003f30 2024-04-25T10:19:55.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.476+0000 7f9b6be00640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b5006b4d0 0x7f9b5006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:55.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.477+0000 7f9b6be00640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b5006b4d0 0x7f9b5006d990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c033000 tx=0x7f9b5c007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:55.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.480+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9b64059200 con 0x7f9b58003f30 2024-04-25T10:19:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.677+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9b74073990 con 0x7f9b58003f30 2024-04-25T10:19:55.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.684+0000 7f9b6a000640 1 -- 172.21.15.165:0/1879168698 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9b64058b00 con 0x7f9b58003f30 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b5006b4d0 msgr2=0x7f9b5006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b5006b4d0 0x7f9b5006d990 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c033000 tx=0x7f9b5c007000 comp rx=0 tx=0).stop 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 msgr2=0x7f9b7415ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b7415ae40 secure :-1 s=READY pgs=2553 cs=0 l=1 rev1=1 crypto rx=0x7f9b6400e7c0 tx=0x7f9b6400ec80 comp rx=0 tx=0).stop 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 shutdown_connections 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9b5006b4d0 0x7f9b5006d990 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9b741e4090 0x7f9b741e4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9b7415d910 0x7f9b7415dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 --2- 172.21.15.165:0/1879168698 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9b58003f30 0x7f9b7415ae40 unknown :-1 s=CLOSED pgs=2553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 >> 172.21.15.165:0/1879168698 conn(0x7f9b74061cc0 msgr2=0x7f9b741446a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 shutdown_connections 2024-04-25T10:19:55.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:55.687+0000 7f9b73200640 1 -- 172.21.15.165:0/1879168698 wait complete. 2024-04-25T10:19:55.709 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:19:55.709 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:55.802 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:19:55.919 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:19:55.919 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:55.919 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:55.977 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:55.977 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:19:55.977 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:56.032 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:56.032 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:19:56.032 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:56.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:19:56.090 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:19:56.090 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:19:56.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:19:56.147 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:19:56.147 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:19:56.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.349+0000 7f53b5c00640 1 Processor -- start 2024-04-25T10:19:56.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.350+0000 7f53b5c00640 1 -- start start 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53b5c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53b5c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f53b00d06c0 0x7f53b00d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53b5c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53b0161df0 con 0x7f53b01615a0 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53b5c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b00d1f50 con 0x7f53b015db90 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53b5c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53b01620d0 con 0x7f53b00d06c0 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53afe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53afe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40128/0 (socket says 172.21.15.165:40128) 2024-04-25T10:19:56.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53afe00640 1 -- 172.21.15.165:0/2075326007 learned_addr learned my addr 172.21.15.165:0/2075326007 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.351+0000 7f53af400640 1 --1- 172.21.15.165:0/2075326007 >> v1:172.21.15.165:6789/0 conn(0x7f53b01615a0 0x7f53b00cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60680/0 (socket says 172.21.15.165:60680) 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.352+0000 7f53afe00640 1 -- 172.21.15.165:0/2075326007 >> v2:172.21.15.172:3300/0 conn(0x7f53b00d06c0 msgr2=0x7f53b00d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.352+0000 7f53afe00640 1 --2- 172.21.15.165:0/2075326007 >> v2:172.21.15.172:3300/0 conn(0x7f53b00d06c0 0x7f53b00d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.352+0000 7f53afe00640 1 -- 172.21.15.165:0/2075326007 >> v1:172.21.15.165:6789/0 conn(0x7f53b01615a0 legacy=0x7f53b00cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.352+0000 7f53afe00640 1 -- 172.21.15.165:0/2075326007 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b01623b0 con 0x7f53b015db90 2024-04-25T10:19:56.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.353+0000 7f53afe00640 1 --2- 172.21.15.165:0/2075326007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00b93c0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f53b00d1400 tx=0x7f53a002a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=962ce225e0b53692 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.353+0000 7f53aea00640 1 -- 172.21.15.165:0/2075326007 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0037070 con 0x7f53b015db90 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.353+0000 7f53aea00640 1 -- 172.21.15.165:0/2075326007 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53a0032440 con 0x7f53b015db90 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.353+0000 7f53aea00640 1 -- 172.21.15.165:0/2075326007 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53a0032810 con 0x7f53b015db90 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 -- 172.21.15.165:0/2075326007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 msgr2=0x7f53b00b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 --2- 172.21.15.165:0/2075326007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00b93c0 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7f53b00d1400 tx=0x7f53a002a2a0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 -- 172.21.15.165:0/2075326007 shutdown_connections 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 --2- 172.21.15.165:0/2075326007 >> v2:172.21.15.172:3300/0 conn(0x7f53b00d06c0 0x7f53b00d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 --2- 172.21.15.165:0/2075326007 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00b93c0 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 -- 172.21.15.165:0/2075326007 >> 172.21.15.165:0/2075326007 conn(0x7f53b00b52b0 msgr2=0x7f53b00b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 -- 172.21.15.165:0/2075326007 shutdown_connections 2024-04-25T10:19:56.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.354+0000 7f53b5c00640 1 -- 172.21.15.165:0/2075326007 wait complete. 2024-04-25T10:19:56.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.356+0000 7f53b5c00640 1 Processor -- start 2024-04-25T10:19:56.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.356+0000 7f53b5c00640 1 -- start start 2024-04-25T10:19:56.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00d06c0 0x7f53b00cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00ccae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b01615a0 0x7f53b00cd020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0161ba0 con 0x7f53b015db90 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53b0161a20 con 0x7f53b01615a0 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53b00b68d0 con 0x7f53b00d06c0 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53af400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00ccae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53af400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00ccae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40130/0 (socket says 172.21.15.165:40130) 2024-04-25T10:19:56.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53af400640 1 -- 172.21.15.165:0/1910679616 learned_addr learned my addr 172.21.15.165:0/1910679616 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:56.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.357+0000 7f53b5200640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b01615a0 0x7f53b00cd020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53afe00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00d06c0 0x7f53b00cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53af400640 1 -- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00d06c0 msgr2=0x7f53b00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53af400640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00d06c0 0x7f53b00cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53af400640 1 -- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b01615a0 msgr2=0x7f53b00cd020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53af400640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b01615a0 0x7f53b00cd020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53af400640 1 -- 172.21.15.165:0/1910679616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b01f49c0 con 0x7f53b015db90 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53b5200640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b01615a0 0x7f53b00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53af400640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00ccae0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f539800c3f0 tx=0x7f539800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53980074e0 con 0x7f53b015db90 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5398007680 con 0x7f53b015db90 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5398004a00 con 0x7f53b015db90 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53b01f4f20 con 0x7f53b015db90 2024-04-25T10:19:56.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.358+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53b00cc750 con 0x7f53b015db90 2024-04-25T10:19:56.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.360+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53b0073990 con 0x7f53b015db90 2024-04-25T10:19:56.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.361+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f539801a020 con 0x7f53b015db90 2024-04-25T10:19:56.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.361+0000 7f53a7e00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.362+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f5398089730 con 0x7f53b015db90 2024-04-25T10:19:56.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.362+0000 7f53afe00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.363+0000 7f53afe00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f53a0034000 tx=0x7f53a002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:56.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.366+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f53b01f49c0 con 0x7f53b015db90 2024-04-25T10:19:56.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.603+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53b00cab20 con 0x7f53b015db90 2024-04-25T10:19:56.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.604+0000 7f53a7e00640 1 -- 172.21.15.165:0/1910679616 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 123 v123) v1 ==== 95+0+3711 (secure 0 0 0) 0x7f53980587b0 con 0x7f53b015db90 2024-04-25T10:19:56.607 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:56.607 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":123,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6185,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3222481692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3222481692},{"type":"v1","addr":"172.21.15.165:6837","nonce":3222481692}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6191,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/3993442403","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":3993442403},{"type":"v1","addr":"172.21.15.172:6834","nonce":3993442403}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":6928,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/3542613985","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":3542613985},{"type":"v1","addr":"172.21.15.172:6835","nonce":3542613985}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":116},{"gid":11139,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/23473807","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":23473807},{"type":"v1","addr":"172.21.15.165:6836","nonce":23473807}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":123}],"filesystems":[]} 2024-04-25T10:19:56.607 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 123 2024-04-25T10:19:56.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 msgr2=0x7f538c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f53a0034000 tx=0x7f53a002e000 comp rx=0 tx=0).stop 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 msgr2=0x7f53b00ccae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00ccae0 secure :-1 s=READY pgs=2556 cs=0 l=1 rev1=1 crypto rx=0x7f539800c3f0 tx=0x7f539800c8b0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 shutdown_connections 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f538c06b4d0 0x7f538c06d990 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f53b01615a0 0x7f53b00cd020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f53b015db90 0x7f53b00ccae0 unknown :-1 s=CLOSED pgs=2556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 --2- 172.21.15.165:0/1910679616 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f53b00d06c0 0x7f53b00cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 >> 172.21.15.165:0/1910679616 conn(0x7f53b00b52b0 msgr2=0x7f53b00b6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 shutdown_connections 2024-04-25T10:19:56.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.607+0000 7f53b5c00640 1 -- 172.21.15.165:0/1910679616 wait complete. 2024-04-25T10:19:56.628 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:19:56.628 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:19:56.628 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:19:56.629 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:56.629 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:19:56.629 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:19:56.629 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:19:56.629 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:56.629 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:19:56.629 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:19:56.630 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:19:56.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:56.630 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:19:56.630 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:19:56.630 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:19:56.630 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:19:56.646 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:19:56.646 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:19:56.649 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:19:56.649 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:19:56.655 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:19:56.655 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:19:56.656 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:19:56.656 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:19:56.694 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:19:56.695 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:56.699 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:19:56.699 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:56.704 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:19:56.704 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:56.710 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:19:56.710 DEBUG:teuthology.parallel:result is None 2024-04-25T10:19:56.710 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:19:56.845 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:19:56.846 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:19:56.857 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:19:56.867 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:19:56.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.891+0000 7fece8200640 1 Processor -- start 2024-04-25T10:19:56.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.892+0000 7fece8200640 1 -- start start 2024-04-25T10:19:56.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece8200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fece00b5620 0x7fece00b5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece006c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece8200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece006cc80 con 0x7fece015b7d0 2024-04-25T10:19:56.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece8200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece006cf50 con 0x7fece00b5620 2024-04-25T10:19:56.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece00ce530 con 0x7fece006c2d0 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece5800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fece015b7d0 0x7fece00cf010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56410/0 (socket says 172.21.15.165:56410) 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece5800640 1 -- 172.21.15.165:0/56334339 learned_addr learned my addr 172.21.15.165:0/56334339 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece4e00640 1 --2- 172.21.15.165:0/56334339 >> v2:172.21.15.165:3300/0 conn(0x7fece00b5620 0x7fece00b5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.893+0000 7fece6200640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece006c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.894+0000 7fece4e00640 1 -- 172.21.15.165:0/56334339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 msgr2=0x7fece006c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.894+0000 7fece4e00640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece006c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.894+0000 7fece4e00640 1 -- 172.21.15.165:0/56334339 >> v1:172.21.15.165:6789/0 conn(0x7fece015b7d0 legacy=0x7fece00cf010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.894+0000 7fece4e00640 1 -- 172.21.15.165:0/56334339 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece00ce810 con 0x7fece00b5620 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.894+0000 7fecdfe00640 1 -- 172.21.15.165:0/56334339 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796129653 0 0) 0x7fece006cc80 con 0x7fece015b7d0 2024-04-25T10:19:56.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.894+0000 7fece6200640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:56.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.895+0000 7fece4e00640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b5620 0x7fece00b5ab0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7feccc0097a0 tx=0x7feccc02a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cad295c064aeed server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.895+0000 7fecdfe00640 1 -- 172.21.15.165:0/56334339 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc037070 con 0x7fece00b5620 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.895+0000 7fecdfe00640 1 -- 172.21.15.165:0/56334339 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feccc032440 con 0x7fece00b5620 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.895+0000 7fecdfe00640 1 -- 172.21.15.165:0/56334339 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feccc032810 con 0x7fece00b5620 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 -- 172.21.15.165:0/56334339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b5620 msgr2=0x7fece00b5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b5620 0x7fece00b5ab0 secure :-1 s=READY pgs=2562 cs=0 l=1 rev1=1 crypto rx=0x7feccc0097a0 tx=0x7feccc02a180 comp rx=0 tx=0).stop 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 -- 172.21.15.165:0/56334339 shutdown_connections 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece006c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 --2- 172.21.15.165:0/56334339 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece00b5620 0x7fece00b5ab0 unknown :-1 s=CLOSED pgs=2562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 -- 172.21.15.165:0/56334339 >> 172.21.15.165:0/56334339 conn(0x7fece00d0800 msgr2=0x7fece00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 -- 172.21.15.165:0/56334339 shutdown_connections 2024-04-25T10:19:56.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.896+0000 7fece8200640 1 -- 172.21.15.165:0/56334339 wait complete. 2024-04-25T10:19:56.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.898+0000 7fece8200640 1 Processor -- start 2024-04-25T10:19:56.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.898+0000 7fece8200640 1 -- start start 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece8200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece014f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece8200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b5620 0x7fece00b4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece8200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 0x7fece0160840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece8200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece015bbb0 con 0x7fece015b7d0 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece8200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece006d230 con 0x7fece00b5620 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece8200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece015bd30 con 0x7fece006c2d0 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 0x7fece0160840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece6200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 0x7fece0160840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41756/0 (socket says 172.21.15.165:41756) 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece6200640 1 -- 172.21.15.165:0/3372174947 learned_addr learned my addr 172.21.15.165:0/3372174947 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece5800640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece014f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.899+0000 7fece4e00640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b5620 0x7fece00b4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece6200640 1 -- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 msgr2=0x7fece014f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece6200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece014f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece6200640 1 -- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b5620 msgr2=0x7fece00b4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece6200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b5620 0x7fece00b4240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece6200640 1 -- 172.21.15.165:0/3372174947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece015f7a0 con 0x7fece015b7d0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece4e00640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b5620 0x7fece00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece6200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 0x7fece0160840 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fecd8007de0 tx=0x7fecd800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd801a070 con 0x7fece015b7d0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd8003990 con 0x7fece015b7d0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd8003d80 con 0x7fece015b7d0 2024-04-25T10:19:56.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece015fa80 con 0x7fece015b7d0 2024-04-25T10:19:56.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.900+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece0160040 con 0x7fece015b7d0 2024-04-25T10:19:56.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.902+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece0073a00 con 0x7fece015b7d0 2024-04-25T10:19:56.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.903+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fecd801d070 con 0x7fece015b7d0 2024-04-25T10:19:56.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.904+0000 7fecdd600640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecc006b500 0x7fecc006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:56.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.904+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fecd808a3c0 con 0x7fece015b7d0 2024-04-25T10:19:56.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.904+0000 7fece5800640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecc006b500 0x7fecc006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:56.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.905+0000 7fece5800640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecc006b500 0x7fecc006d9c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007fd0 tx=0x7fecd4007de0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:56.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:56.908+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fecd8010070 con 0x7fece015b7d0 2024-04-25T10:19:57.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.114+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fece00cf470 con 0x7fece015b7d0 2024-04-25T10:19:57.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.266+0000 7fecdd600640 1 -- 172.21.15.165:0/3372174947 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v178) v1 ==== 97+0+0 (secure 0 0 0) 0x7fecd8058cc0 con 0x7fece015b7d0 2024-04-25T10:19:57.269 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:19:57.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.269+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecc006b500 msgr2=0x7fecc006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.269+0000 7fece8200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecc006b500 0x7fecc006d9c0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007fd0 tx=0x7fecd4007de0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.269+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 msgr2=0x7fece0160840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.269+0000 7fece8200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 0x7fece0160840 secure :-1 s=READY pgs=2563 cs=0 l=1 rev1=1 crypto rx=0x7fecd8007de0 tx=0x7fecd800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.269+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 shutdown_connections 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.269+0000 7fece8200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fecc006b500 0x7fecc006d9c0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.270+0000 7fece8200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fece015b7d0 0x7fece0160840 unknown :-1 s=CLOSED pgs=2563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.270+0000 7fece8200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fece00b5620 0x7fece00b4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.270+0000 7fece8200640 1 --2- 172.21.15.165:0/3372174947 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fece006c2d0 0x7fece014f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.270+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 >> 172.21.15.165:0/3372174947 conn(0x7fece00d0800 msgr2=0x7fece01e88e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.270+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 shutdown_connections 2024-04-25T10:19:57.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.270+0000 7fece8200640 1 -- 172.21.15.165:0/3372174947 wait complete. 2024-04-25T10:19:57.294 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:19:57.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.484+0000 7f5a2f200640 1 Processor -- start 2024-04-25T10:19:57.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.484+0000 7f5a2f200640 1 -- start start 2024-04-25T10:19:57.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5a300ccb00 0x7f5a3015f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a3015fef0 0x7f5a3006d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2f200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3015c520 con 0x7f5a30144ae0 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3015c800 con 0x7f5a300ccb00 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015cae0 con 0x7f5a3015fef0 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5a30144ae0 0x7f5a3015bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56424/0 (socket says 172.21.15.165:56424) 2024-04-25T10:19:57.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2de00640 1 -- 172.21.15.165:0/2629344270 learned_addr learned my addr 172.21.15.165:0/2629344270 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a25400640 1 --2- 172.21.15.165:0/2629344270 >> v2:172.21.15.172:3300/0 conn(0x7f5a300ccb00 0x7f5a3015f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.486+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3687622916 0 0) 0x7f5a3015c520 con 0x7f5a30144ae0 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a14003520 con 0x7f5a30144ae0 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2e800640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a3015fef0 0x7f5a3006d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1304587615 0 0) 0x7f5a14003520 con 0x7f5a30144ae0 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a3015fef0 msgr2=0x7f5a3006d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a3015fef0 0x7f5a3006d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 >> v2:172.21.15.172:3300/0 conn(0x7f5a300ccb00 msgr2=0x7f5a3015f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:57.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 --2- 172.21.15.165:0/2629344270 >> v2:172.21.15.172:3300/0 conn(0x7f5a300ccb00 0x7f5a3015f9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3006d7b0 con 0x7f5a30144ae0 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5a20001670 con 0x7f5a30144ae0 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 >> v1:172.21.15.165:6789/0 conn(0x7f5a30144ae0 legacy=0x7f5a3015bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140039b0 0x7f5a14003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a14004800 0x7f5a14004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a140051c0 0x7f5a1400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015c800 con 0x7f5a14004800 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3015cae0 con 0x7f5a140039b0 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a1400c030 con 0x7f5a140051c0 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a25400640 1 --2- 172.21.15.165:0/2629344270 >> v2:172.21.15.172:3300/0 conn(0x7f5a300ccb00 0x7f5a3015f9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a2e800640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a3015fef0 0x7f5a3006d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.487+0000 7f5a25400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a14004800 0x7f5a14004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.488+0000 7f5a2de00640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140039b0 0x7f5a14003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.488+0000 7f5a25400640 1 -- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a140051c0 msgr2=0x7f5a1400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:57.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.488+0000 7f5a25400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a140051c0 0x7f5a1400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.488+0000 7f5a25400640 1 -- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140039b0 msgr2=0x7f5a14003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:57.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.488+0000 7f5a25400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a140039b0 0x7f5a14003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.488+0000 7f5a25400640 1 -- 172.21.15.165:0/2629344270 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a14003520 con 0x7f5a14004800 2024-04-25T10:19:57.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.489+0000 7f5a25400640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a14004800 0x7f5a14004c80 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a14005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=565ef5ae0f2c6d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:57.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.489+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a18037070 con 0x7f5a14004800 2024-04-25T10:19:57.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.489+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a1802ad20 con 0x7f5a14004800 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.489+0000 7f5a2d400640 1 -- 172.21.15.165:0/2629344270 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a1800f040 con 0x7f5a14004800 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2f200640 1 -- 172.21.15.165:0/2629344270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a14004800 msgr2=0x7f5a14004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2629344270 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a14004800 0x7f5a14004c80 secure :-1 s=READY pgs=2565 cs=0 l=1 rev1=1 crypto rx=0x7f5a180098a0 tx=0x7f5a14005830 comp rx=0 tx=0).stop 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2e800640 1 -- 172.21.15.165:0/2629344270 reap_dead start 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2f200640 1 -- 172.21.15.165:0/2629344270 shutdown_connections 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2f200640 1 -- 172.21.15.165:0/2629344270 >> 172.21.15.165:0/2629344270 conn(0x7f5a30061c30 msgr2=0x7f5a3006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2f200640 1 -- 172.21.15.165:0/2629344270 shutdown_connections 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.490+0000 7f5a2f200640 1 -- 172.21.15.165:0/2629344270 wait complete. 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.492+0000 7f5a2f200640 1 Processor -- start 2024-04-25T10:19:57.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.492+0000 7f5a2f200640 1 -- start start 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a14004800 0x7f5a301f3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a300cd150 0x7f5a301f6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 0x7f5a301f6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3006d3f0 con 0x7f5a300cd590 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3006d270 con 0x7f5a300cd150 2024-04-25T10:19:57.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3006d570 con 0x7f5a14004800 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 0x7f5a301f6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 0x7f5a301f6ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41772/0 (socket says 172.21.15.165:41772) 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.493+0000 7f5a2e800640 1 -- 172.21.15.165:0/2063591365 learned_addr learned my addr 172.21.15.165:0/2063591365 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a25400640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a300cd150 0x7f5a301f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2de00640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a14004800 0x7f5a301f3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2e800640 1 -- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a14004800 msgr2=0x7f5a301f3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2e800640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a14004800 0x7f5a301f3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2e800640 1 -- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a300cd150 msgr2=0x7f5a301f6580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2e800640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a300cd150 0x7f5a301f6580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2e800640 1 -- 172.21.15.165:0/2063591365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a300cb3a0 con 0x7f5a300cd590 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a25400640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a300cd150 0x7f5a301f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a2e800640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 0x7f5a301f6ac0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f5a3006eff0 tx=0x7f5a2800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2801a070 con 0x7f5a300cd590 2024-04-25T10:19:57.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.494+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a28003990 con 0x7f5a300cd590 2024-04-25T10:19:57.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.495+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a301f70d0 con 0x7f5a300cd590 2024-04-25T10:19:57.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.495+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a28003d80 con 0x7f5a300cd590 2024-04-25T10:19:57.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.495+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a301f32c0 con 0x7f5a300cd590 2024-04-25T10:19:57.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.496+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a30073990 con 0x7f5a300cd590 2024-04-25T10:19:57.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.497+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5a2801d070 con 0x7f5a300cd590 2024-04-25T10:19:57.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.498+0000 7f5a26a00640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0c06b500 0x7f5a0c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.498+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5a28089e60 con 0x7f5a300cd590 2024-04-25T10:19:57.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.498+0000 7f5a2de00640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0c06b500 0x7f5a0c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.499+0000 7f5a2de00640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0c06b500 0x7f5a0c06d9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5a20001790 tx=0x7f5a20000f10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:57.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.502+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5a28010070 con 0x7f5a300cd590 2024-04-25T10:19:57.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.767+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5a3006c220 con 0x7f5a300cd590 2024-04-25T10:19:57.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.768+0000 7f5a26a00640 1 -- 172.21.15.165:0/2063591365 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v24) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5a28058e90 con 0x7f5a300cd590 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.771 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:19:57.772 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.773 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.774 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:19:57.775 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.776 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.777 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:19:57.778 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:19:57.779 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:19:57.780 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.781 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:19:57.782 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:19:57.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.770+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0c06b500 msgr2=0x7f5a0c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.770+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0c06b500 0x7f5a0c06d9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5a20001790 tx=0x7f5a20000f10 comp rx=0 tx=0).stop 2024-04-25T10:19:57.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.770+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 msgr2=0x7f5a301f6ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.770+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 0x7f5a301f6ac0 secure :-1 s=READY pgs=2566 cs=0 l=1 rev1=1 crypto rx=0x7f5a3006eff0 tx=0x7f5a2800d4a0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 shutdown_connections 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5a0c06b500 0x7f5a0c06d9c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5a300cd590 0x7f5a301f6ac0 unknown :-1 s=CLOSED pgs=2566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5a300cd150 0x7f5a301f6580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 --2- 172.21.15.165:0/2063591365 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5a14004800 0x7f5a301f3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 >> 172.21.15.165:0/2063591365 conn(0x7f5a30061c30 msgr2=0x7f5a300aff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 shutdown_connections 2024-04-25T10:19:57.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.771+0000 7f5a2f200640 1 -- 172.21.15.165:0/2063591365 wait complete. 2024-04-25T10:19:57.791 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:19:57.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.985+0000 7fb09be00640 1 Processor -- start 2024-04-25T10:19:57.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.985+0000 7fb09be00640 1 -- start start 2024-04-25T10:19:57.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb09be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb09415db70 0x7fb0940cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb09be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb0940cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb09be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb0940cec70 0x7fb0940cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb09be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0940cbd80 con 0x7fb0940cec70 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb09be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb094161910 con 0x7fb09415db70 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb09be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094071650 con 0x7fb09415df50 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb099e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb0940cec70 0x7fb0940cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb099e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb0940cec70 0x7fb0940cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41778/0 (socket says 172.21.15.165:41778) 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.987+0000 7fb099e00640 1 -- 172.21.15.165:0/3989544127 learned_addr learned my addr 172.21.15.165:0/3989544127 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb099400640 1 --2- 172.21.15.165:0/3989544127 >> v2:172.21.15.172:3300/0 conn(0x7fb09415db70 0x7fb0940cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb098a00640 1 --2- 172.21.15.165:0/3989544127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb0940cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb099e00640 1 -- 172.21.15.165:0/3989544127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 msgr2=0x7fb0940cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb099e00640 1 --2- 172.21.15.165:0/3989544127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb0940cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb099e00640 1 -- 172.21.15.165:0/3989544127 >> v2:172.21.15.172:3300/0 conn(0x7fb09415db70 msgr2=0x7fb0940cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:57.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb099e00640 1 --2- 172.21.15.165:0/3989544127 >> v2:172.21.15.172:3300/0 conn(0x7fb09415db70 0x7fb0940cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.988+0000 7fb099e00640 1 -- 172.21.15.165:0/3989544127 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0940cf230 con 0x7fb0940cec70 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.989+0000 7fb099e00640 1 --2- 172.21.15.165:0/3989544127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0940cec70 0x7fb0940cfa50 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fb0940cd3d0 tx=0x7fb08c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee901dc8b50bd5f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.989+0000 7fb093e00640 1 -- 172.21.15.165:0/3989544127 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c036070 con 0x7fb0940cec70 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.989+0000 7fb093e00640 1 -- 172.21.15.165:0/3989544127 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08c031600 con 0x7fb0940cec70 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.989+0000 7fb093e00640 1 -- 172.21.15.165:0/3989544127 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0319d0 con 0x7fb0940cec70 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 -- 172.21.15.165:0/3989544127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0940cec70 msgr2=0x7fb0940cfa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 --2- 172.21.15.165:0/3989544127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0940cec70 0x7fb0940cfa50 secure :-1 s=READY pgs=2567 cs=0 l=1 rev1=1 crypto rx=0x7fb0940cd3d0 tx=0x7fb08c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 -- 172.21.15.165:0/3989544127 shutdown_connections 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 --2- 172.21.15.165:0/3989544127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb0940cec70 0x7fb0940cfa50 unknown :-1 s=CLOSED pgs=2567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 --2- 172.21.15.165:0/3989544127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb0940cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 --2- 172.21.15.165:0/3989544127 >> v2:172.21.15.172:3300/0 conn(0x7fb09415db70 0x7fb0940cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 -- 172.21.15.165:0/3989544127 >> 172.21.15.165:0/3989544127 conn(0x7fb09406abe0 msgr2=0x7fb09406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 -- 172.21.15.165:0/3989544127 shutdown_connections 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.990+0000 7fb09be00640 1 -- 172.21.15.165:0/3989544127 wait complete. 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.992+0000 7fb09be00640 1 Processor -- start 2024-04-25T10:19:57.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.992+0000 7fb09be00640 1 -- start start 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb09be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0940cec70 0x7fb094159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb09be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 0x7fb094160550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb09be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb09414e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb09be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094162120 con 0x7fb09415db70 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb09be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb094161e40 con 0x7fb0940cec70 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb09be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb094162640 con 0x7fb09415df50 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb098a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 0x7fb094160550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb098a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 0x7fb094160550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41780/0 (socket says 172.21.15.165:41780) 2024-04-25T10:19:57.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.993+0000 7fb098a00640 1 -- 172.21.15.165:0/1010855497 learned_addr learned my addr 172.21.15.165:0/1010855497 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb099400640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0940cec70 0x7fb094159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb099e00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb09414e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb098a00640 1 -- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 msgr2=0x7fb09414e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb098a00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb09414e480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb098a00640 1 -- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0940cec70 msgr2=0x7fb094159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb098a00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0940cec70 0x7fb094159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb098a00640 1 -- 172.21.15.165:0/1010855497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09414f400 con 0x7fb09415db70 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb099400640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0940cec70 0x7fb094159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb098a00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 0x7fb094160550 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d5f0 tx=0x7fb08000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800125f0 con 0x7fb09415db70 2024-04-25T10:19:57.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb080019070 con 0x7fb09415db70 2024-04-25T10:19:57.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0800129d0 con 0x7fb09415db70 2024-04-25T10:19:57.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.994+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb09414f6e0 con 0x7fb09415db70 2024-04-25T10:19:57.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.995+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb094157c80 con 0x7fb09415db70 2024-04-25T10:19:57.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.996+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb09414f180 con 0x7fb09415db70 2024-04-25T10:19:57.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.997+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb080012b70 con 0x7fb09415db70 2024-04-25T10:19:58.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.998+0000 7fb091600640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb07406b4d0 0x7fb07406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.998+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fb08008a4b0 con 0x7fb09415db70 2024-04-25T10:19:58.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.998+0000 7fb099400640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb07406b4d0 0x7fb07406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:57.999+0000 7fb099400640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb07406b4d0 0x7fb07406d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb088006fd0 tx=0x7fb088008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.002+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb09414f180 con 0x7fb09415db70 2024-04-25T10:19:58.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.200+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb094073990 con 0x7fb09415db70 2024-04-25T10:19:58.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.202+0000 7fb091600640 1 -- 172.21.15.165:0/1010855497 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+13210 (secure 0 0 0) 0x7fb080059450 con 0x7fb09415db70 2024-04-25T10:19:58.205 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:58.205 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":180,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:19:57.278632+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:19:58.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb07406b4d0 msgr2=0x7fb07406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb07406b4d0 0x7fb07406d990 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fb088006fd0 tx=0x7fb088008040 comp rx=0 tx=0).stop 2024-04-25T10:19:58.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 msgr2=0x7fb094160550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 0x7fb094160550 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7fb08000d5f0 tx=0x7fb08000dab0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 shutdown_connections 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb07406b4d0 0x7fb07406d990 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb09415df50 0x7fb09414e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb09415db70 0x7fb094160550 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 --2- 172.21.15.165:0/1010855497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb0940cec70 0x7fb094159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.204+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 >> 172.21.15.165:0/1010855497 conn(0x7fb09406abe0 msgr2=0x7fb09406d290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.205+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 shutdown_connections 2024-04-25T10:19:58.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.205+0000 7fb09be00640 1 -- 172.21.15.165:0/1010855497 wait complete. 2024-04-25T10:19:58.227 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:19:58.227 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:19:58.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.415+0000 7f8ac7000640 1 Processor -- start 2024-04-25T10:19:58.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.415+0000 7f8ac7000640 1 -- start start 2024-04-25T10:19:58.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.417+0000 7f8ac7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 0x7f8ac006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.417+0000 7f8ac7000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac00cfd80 con 0x7f8ac00ac8b0 2024-04-25T10:19:58.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.417+0000 7f8ac7000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac00d0060 con 0x7f8ac015b710 2024-04-25T10:19:58.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.417+0000 7f8ac7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac006f190 con 0x7f8ac00cf640 2024-04-25T10:19:58.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.417+0000 7f8ac5000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8ac00ac8b0 0x7f8ac0071bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56436/0 (socket says 172.21.15.165:56436) 2024-04-25T10:19:58.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.417+0000 7f8ac5000640 1 -- 172.21.15.165:0/1445386574 learned_addr learned my addr 172.21.15.165:0/1445386574 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:58.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8ab7400640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 0x7f8ac006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2608701125 0 0) 0x7f8ac00cfd80 con 0x7f8ac00ac8b0 2024-04-25T10:19:58.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa4003520 con 0x7f8ac00ac8b0 2024-04-25T10:19:58.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3436161220 0 0) 0x7f8aa4003520 con 0x7f8ac00ac8b0 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 msgr2=0x7f8ac006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 0x7f8ac006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 >> v1:172.21.15.172:6789/0 conn(0x7f8ac015b710 legacy=0x7f8ac015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00d0340 con 0x7f8ac00ac8b0 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8ab80019f0 con 0x7f8ac00ac8b0 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 >> v1:172.21.15.165:6789/0 conn(0x7f8ac00ac8b0 legacy=0x7f8ac0071bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa40039e0 0x7f8aa4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4004800 0x7f8aa4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 0x7f8aa40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac006f190 con 0x7f8aa40039e0 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa400bad0 con 0x7f8aa4004800 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.418+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa400bcd0 con 0x7f8ac00cf640 2024-04-25T10:19:58.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.419+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa40039e0 0x7f8aa4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.419+0000 7f8abfe00640 1 -- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 msgr2=0x7f8aa40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.419+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac00cf640 0x7f8aa40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.419+0000 7f8abfe00640 1 -- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4004800 msgr2=0x7f8aa4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.419+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa4004800 0x7f8aa4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.419+0000 7f8abfe00640 1 -- 172.21.15.165:0/1445386574 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa4003520 con 0x7f8aa40039e0 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.420+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa40039e0 0x7f8aa4003e00 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0001720 tx=0x7f8aa4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=752d73132107bcee server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.420+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab0001370 con 0x7f8aa40039e0 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.420+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab00052d0 con 0x7f8aa40039e0 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.420+0000 7f8abf400640 1 -- 172.21.15.165:0/1445386574 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab002ab90 con 0x7f8aa40039e0 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac7000640 1 -- 172.21.15.165:0/1445386574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa40039e0 msgr2=0x7f8aa4003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1445386574 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8aa40039e0 0x7f8aa4003e00 secure :-1 s=READY pgs=2570 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0001720 tx=0x7f8aa4005830 comp rx=0 tx=0).stop 2024-04-25T10:19:58.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac5000640 1 -- 172.21.15.165:0/1445386574 reap_dead start 2024-04-25T10:19:58.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac7000640 1 -- 172.21.15.165:0/1445386574 shutdown_connections 2024-04-25T10:19:58.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac7000640 1 -- 172.21.15.165:0/1445386574 >> 172.21.15.165:0/1445386574 conn(0x7f8ac006abe0 msgr2=0x7f8ac006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac7000640 1 -- 172.21.15.165:0/1445386574 shutdown_connections 2024-04-25T10:19:58.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.421+0000 7f8ac7000640 1 -- 172.21.15.165:0/1445386574 wait complete. 2024-04-25T10:19:58.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.423+0000 7f8ac7000640 1 Processor -- start 2024-04-25T10:19:58.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.423+0000 7f8ac7000640 1 -- start start 2024-04-25T10:19:58.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ac7000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa40039e0 0x7f8ac00c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ac7000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 0x7f8ac014b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ac7000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac014bcd0 0x7f8ac00b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ac7000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac015baf0 con 0x7f8ac00c5050 2024-04-25T10:19:58.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ac7000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac015e490 con 0x7f8aa40039e0 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ac7000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac015bc70 con 0x7f8ac014bcd0 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 0x7f8ac014b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ab7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 0x7f8ac014b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41790/0 (socket says 172.21.15.165:41790) 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.424+0000 7f8ab7400640 1 -- 172.21.15.165:0/1309031741 learned_addr learned my addr 172.21.15.165:0/1309031741 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa40039e0 0x7f8ac00c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ac5000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac014bcd0 0x7f8ac00b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ab7400640 1 -- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac014bcd0 msgr2=0x7f8ac00b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ab7400640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac014bcd0 0x7f8ac00b1950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ab7400640 1 -- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa40039e0 msgr2=0x7f8ac00c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ab7400640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa40039e0 0x7f8ac00c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ab7400640 1 -- 172.21.15.165:0/1309031741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00c5530 con 0x7f8ac00c5050 2024-04-25T10:19:58.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ab7400640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 0x7f8ac014b180 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800e7d0 tx=0x7f8aa800ec90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa8017070 con 0x7f8ac00c5050 2024-04-25T10:19:58.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa800c940 con 0x7f8ac00c5050 2024-04-25T10:19:58.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa800cd10 con 0x7f8ac00c5050 2024-04-25T10:19:58.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.425+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac00b0e70 con 0x7f8ac00c5050 2024-04-25T10:19:58.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.426+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac00c4c10 con 0x7f8ac00c5050 2024-04-25T10:19:58.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.427+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac015d510 con 0x7f8ac00c5050 2024-04-25T10:19:58.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.428+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8aa801a070 con 0x7f8ac00c5050 2024-04-25T10:19:58.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.429+0000 7f8abcc00640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.429+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8aa808a220 con 0x7f8ac00c5050 2024-04-25T10:19:58.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.429+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.430+0000 7f8abfe00640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0032000 tx=0x7f8ab002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.433+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8ac015d510 con 0x7f8ac00c5050 2024-04-25T10:19:58.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.631+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8ac006b210 con 0x7f8ac00c5050 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.632+0000 7f8abcc00640 1 -- 172.21.15.165:0/1309031741 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v180) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8aa80591c0 con 0x7f8ac00c5050 2024-04-25T10:19:58.635 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:19:58.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9c06b500 msgr2=0x7f8a9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0032000 tx=0x7f8ab002e000 comp rx=0 tx=0).stop 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 msgr2=0x7f8ac014b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 0x7f8ac014b180 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f8aa800e7d0 tx=0x7f8aa800ec90 comp rx=0 tx=0).stop 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 shutdown_connections 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a9c06b500 0x7f8a9c06d9c0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ac014bcd0 0x7f8ac00b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ac00c5050 0x7f8ac014b180 unknown :-1 s=CLOSED pgs=2571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 --2- 172.21.15.165:0/1309031741 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8aa40039e0 0x7f8ac00c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.635+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 >> 172.21.15.165:0/1309031741 conn(0x7f8ac006abe0 msgr2=0x7f8ac00acd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.636+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 shutdown_connections 2024-04-25T10:19:58.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.636+0000 7f8ac7000640 1 -- 172.21.15.165:0/1309031741 wait complete. 2024-04-25T10:19:58.656 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:19:58.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.843+0000 7f5f78800640 1 Processor -- start 2024-04-25T10:19:58.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.843+0000 7f5f78800640 1 -- start start 2024-04-25T10:19:58.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f78800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5f70144d40 0x7f5f7006d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f78800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5f700ce950 0x7f5f700cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f78800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7015c150 con 0x7f5f700cc930 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f78800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7006d850 con 0x7f5f70144d40 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f78800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f700cd4b0 con 0x7f5f700ce950 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f5f70144d40 0x7f5f7006d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f5f700cc930 0x7f5f7015c910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56452/0 (socket says 172.21.15.165:56452) 2024-04-25T10:19:58.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75400640 1 -- 172.21.15.165:0/2948647341 learned_addr learned my addr 172.21.15.165:0/2948647341 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f76800640 1 --2- 172.21.15.165:0/2948647341 >> v2:172.21.15.172:3300/0 conn(0x7f5f700ce950 0x7f5f700cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75e00640 1 -- 172.21.15.165:0/2948647341 >> v2:172.21.15.172:3300/0 conn(0x7f5f700ce950 msgr2=0x7f5f700cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75e00640 1 --2- 172.21.15.165:0/2948647341 >> v2:172.21.15.172:3300/0 conn(0x7f5f700ce950 0x7f5f700cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75e00640 1 -- 172.21.15.165:0/2948647341 >> v1:172.21.15.165:6789/0 conn(0x7f5f700cc930 legacy=0x7f5f7015c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f75e00640 1 -- 172.21.15.165:0/2948647341 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700ccef0 con 0x7f5f70144d40 2024-04-25T10:19:58.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.845+0000 7f5f74a00640 1 -- 172.21.15.165:0/2948647341 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4164382624 0 0) 0x7f5f7015c150 con 0x7f5f700cc930 2024-04-25T10:19:58.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.847+0000 7f5f75e00640 1 --2- 172.21.15.165:0/2948647341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f70144d40 0x7f5f7006d310 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f5f700ccdf0 tx=0x7f5f6402a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed7df8961cc68b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.847+0000 7f5f74a00640 1 -- 172.21.15.165:0/2948647341 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f64037070 con 0x7f5f70144d40 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.847+0000 7f5f74a00640 1 -- 172.21.15.165:0/2948647341 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f64032440 con 0x7f5f70144d40 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.847+0000 7f5f74a00640 1 -- 172.21.15.165:0/2948647341 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6402ae60 con 0x7f5f70144d40 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.847+0000 7f5f78800640 1 -- 172.21.15.165:0/2948647341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f70144d40 msgr2=0x7f5f7006d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.847+0000 7f5f78800640 1 --2- 172.21.15.165:0/2948647341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f70144d40 0x7f5f7006d310 secure :-1 s=READY pgs=2573 cs=0 l=1 rev1=1 crypto rx=0x7f5f700ccdf0 tx=0x7f5f6402a110 comp rx=0 tx=0).stop 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.848+0000 7f5f78800640 1 -- 172.21.15.165:0/2948647341 shutdown_connections 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.848+0000 7f5f78800640 1 --2- 172.21.15.165:0/2948647341 >> v2:172.21.15.172:3300/0 conn(0x7f5f700ce950 0x7f5f700cedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.848+0000 7f5f78800640 1 --2- 172.21.15.165:0/2948647341 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f70144d40 0x7f5f7006d310 unknown :-1 s=CLOSED pgs=2573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.848+0000 7f5f78800640 1 -- 172.21.15.165:0/2948647341 >> 172.21.15.165:0/2948647341 conn(0x7f5f7006abe0 msgr2=0x7f5f7006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.848+0000 7f5f78800640 1 -- 172.21.15.165:0/2948647341 shutdown_connections 2024-04-25T10:19:58.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.848+0000 7f5f78800640 1 -- 172.21.15.165:0/2948647341 wait complete. 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.849+0000 7f5f78800640 1 Processor -- start 2024-04-25T10:19:58.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 -- start start 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f700cc930 0x7f5f701f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 0x7f5f700c6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f70144d40 0x7f5f700c6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f700cf2f0 con 0x7f5f700ce950 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7015c3f0 con 0x7f5f700cc930 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.850+0000 7f5f78800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7006da60 con 0x7f5f70144d40 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 0x7f5f700c6150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 0x7f5f700c6150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41808/0 (socket says 172.21.15.165:41808) 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 -- 172.21.15.165:0/4013683537 learned_addr learned my addr 172.21.15.165:0/4013683537 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:58.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 -- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f70144d40 msgr2=0x7f5f700c6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f70144d40 0x7f5f700c6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75e00640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f700cc930 0x7f5f701f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 -- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f700cc930 msgr2=0x7f5f701f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f700cc930 0x7f5f701f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.851+0000 7f5f75400640 1 -- 172.21.15.165:0/4013683537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f700ac0e0 con 0x7f5f700ce950 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.852+0000 7f5f75400640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 0x7f5f700c6150 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c4d0 tx=0x7f5f6000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.852+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f60017070 con 0x7f5f700ce950 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.852+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f60005d00 con 0x7f5f700ce950 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.852+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f600072b0 con 0x7f5f700ce950 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.852+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f700ac3c0 con 0x7f5f700ce950 2024-04-25T10:19:58.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.852+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f700ac980 con 0x7f5f700ce950 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.853+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f70073a00 con 0x7f5f700ce950 2024-04-25T10:19:58.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.854+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5f6001a070 con 0x7f5f700ce950 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.856+0000 7f5f6a000640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f5406b4d0 0x7f5f5406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.856+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f5f60089490 con 0x7f5f700ce950 2024-04-25T10:19:58.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.856+0000 7f5f75e00640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f5406b4d0 0x7f5f5406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:58.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.857+0000 7f5f75e00640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f5406b4d0 0x7f5f5406d990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5f64034000 tx=0x7f5f6402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:58.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:58.860+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5f60058430 con 0x7f5f700ce950 2024-04-25T10:19:59.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.055+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5f70067e60 con 0x7f5f700ce950 2024-04-25T10:19:59.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.056+0000 7f5f6a000640 1 -- 172.21.15.165:0/4013683537 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v180) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5f60057d30 con 0x7f5f700ce950 2024-04-25T10:19:59.058 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:19:59.059 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:19:59.059 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:19:59.059 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:19:59.059 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:19:59.059 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:19:59.059 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:19:59.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.058+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f5406b4d0 msgr2=0x7f5f5406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.058+0000 7f5f78800640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f5406b4d0 0x7f5f5406d990 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5f64034000 tx=0x7f5f6402e000 comp rx=0 tx=0).stop 2024-04-25T10:19:59.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.058+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 msgr2=0x7f5f700c6150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.058+0000 7f5f78800640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 0x7f5f700c6150 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f5f6000c4d0 tx=0x7f5f6000c990 comp rx=0 tx=0).stop 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 shutdown_connections 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5f5406b4d0 0x7f5f5406d990 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5f70144d40 0x7f5f700c6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5f700ce950 0x7f5f700c6150 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 --2- 172.21.15.165:0/4013683537 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5f700cc930 0x7f5f701f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 >> 172.21.15.165:0/4013683537 conn(0x7f5f7006abe0 msgr2=0x7f5f7006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 shutdown_connections 2024-04-25T10:19:59.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.059+0000 7f5f78800640 1 -- 172.21.15.165:0/4013683537 wait complete. 2024-04-25T10:19:59.081 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:19:59.081 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:19:59.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.284+0000 7f30ec400640 1 Processor -- start 2024-04-25T10:19:59.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.285+0000 7f30ec400640 1 -- start start 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30ec400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30ec400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30e4161710 0x7f30e40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30ec400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e415e000 con 0x7f30e40b9330 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30ec400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40d1de0 con 0x7f30e4161710 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30ec400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4161c60 con 0x7f30e415dc20 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30e9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30e4161710 0x7f30e40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30e9000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f30e4161710 0x7f30e40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41820/0 (socket says 172.21.15.165:41820) 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.286+0000 7f30e9000640 1 -- 172.21.15.165:0/3795917008 learned_addr learned my addr 172.21.15.165:0/3795917008 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.287+0000 7f30e9a00640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.287+0000 7f30e9000640 1 -- 172.21.15.165:0/3795917008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 msgr2=0x7f30e40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.287+0000 7f30e9000640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.287+0000 7f30e9000640 1 -- 172.21.15.165:0/3795917008 >> v1:172.21.15.172:6789/0 conn(0x7f30e40b9330 legacy=0x7f30e406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.287+0000 7f30e9000640 1 -- 172.21.15.165:0/3795917008 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e4161f40 con 0x7f30e4161710 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.287+0000 7f30e9a00640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.288+0000 7f30e9000640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e40cebb0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009870 tx=0x7f30d002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84d37a569cc1082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.288+0000 7f30e3e00640 1 -- 172.21.15.165:0/3795917008 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0037070 con 0x7f30e4161710 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.288+0000 7f30e3e00640 1 -- 172.21.15.165:0/3795917008 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0032440 con 0x7f30e4161710 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.288+0000 7f30e3e00640 1 -- 172.21.15.165:0/3795917008 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0032810 con 0x7f30e4161710 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 -- 172.21.15.165:0/3795917008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 msgr2=0x7f30e40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e40cebb0 secure :-1 s=READY pgs=2575 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009870 tx=0x7f30d002a210 comp rx=0 tx=0).stop 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 -- 172.21.15.165:0/3795917008 shutdown_connections 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e40cebb0 unknown :-1 s=CLOSED pgs=2575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 --2- 172.21.15.165:0/3795917008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 -- 172.21.15.165:0/3795917008 >> 172.21.15.165:0/3795917008 conn(0x7f30e40d0420 msgr2=0x7f30e40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:19:59.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 -- 172.21.15.165:0/3795917008 shutdown_connections 2024-04-25T10:19:59.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.289+0000 7f30ec400640 1 -- 172.21.15.165:0/3795917008 wait complete. 2024-04-25T10:19:59.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.291+0000 7f30ec400640 1 Processor -- start 2024-04-25T10:19:59.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.291+0000 7f30ec400640 1 -- start start 2024-04-25T10:19:59.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.292+0000 7f30ec400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40b9330 0x7f30e40caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.292+0000 7f30ec400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e4159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.292+0000 7f30ec400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e4159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.292+0000 7f30ec400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40b5e50 con 0x7f30e4161710 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.292+0000 7f30ec400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e4162500 con 0x7f30e40b9330 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.292+0000 7f30ec400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40b5fd0 con 0x7f30e415dc20 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e4159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e4159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41822/0 (socket says 172.21.15.165:41822) 2024-04-25T10:19:59.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 -- 172.21.15.165:0/1644460023 learned_addr learned my addr 172.21.15.165:0/1644460023 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:19:59.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30e9000640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e4159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30e9a00640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40b9330 0x7f30e40caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 -- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 msgr2=0x7f30e4159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e4159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 -- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40b9330 msgr2=0x7f30e40caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40b9330 0x7f30e40caeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30ea400640 1 -- 172.21.15.165:0/1644460023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e41e9da0 con 0x7f30e4161710 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.293+0000 7f30e9000640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.294+0000 7f30ea400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e4159fb0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00b9e0 tx=0x7f30dc00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.294+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc019070 con 0x7f30e4161710 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.294+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30dc00caa0 con 0x7f30e4161710 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.294+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc014440 con 0x7f30e4161710 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.294+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40cb9c0 con 0x7f30e4161710 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.294+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e415d140 con 0x7f30e4161710 2024-04-25T10:19:59.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.295+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e4073a00 con 0x7f30e4161710 2024-04-25T10:19:59.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.296+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f30dc01c070 con 0x7f30e4161710 2024-04-25T10:19:59.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.297+0000 7f30e1600640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30c406b4d0 0x7f30c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:19:59.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.297+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f30dc089cf0 con 0x7f30e4161710 2024-04-25T10:19:59.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.297+0000 7f30e9a00640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30c406b4d0 0x7f30c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:19:59.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.298+0000 7f30e9a00640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30c406b4d0 0x7f30c406d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f30d800b840 tx=0x7f30d8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:19:59.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.302+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f30dc08e070 con 0x7f30e4161710 2024-04-25T10:19:59.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:19:59.496+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f30e40b8eb0 con 0x7f30e4161710 2024-04-25T10:20:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.297+0000 7f30e1600640 1 -- 172.21.15.165:0/1644460023 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v181) v1 ==== 138+0+0 (secure 0 0 0) 0x7f30dc058d30 con 0x7f30e4161710 2024-04-25T10:20:00.300 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30c406b4d0 msgr2=0x7f30c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30c406b4d0 0x7f30c406d990 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f30d800b840 tx=0x7f30d8008040 comp rx=0 tx=0).stop 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 msgr2=0x7f30e4159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e4159fb0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7f30dc00b9e0 tx=0x7f30dc00bea0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 shutdown_connections 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f30c406b4d0 0x7f30c406d990 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f30e4161710 0x7f30e4159fb0 unknown :-1 s=CLOSED pgs=2576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f30e415dc20 0x7f30e4159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 --2- 172.21.15.165:0/1644460023 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f30e40b9330 0x7f30e40caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 >> 172.21.15.165:0/1644460023 conn(0x7f30e40d0420 msgr2=0x7f30e40d2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 shutdown_connections 2024-04-25T10:20:00.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.301+0000 7f30ec400640 1 -- 172.21.15.165:0/1644460023 wait complete. 2024-04-25T10:20:00.328 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:20:00.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.520+0000 7f6e8ca00640 1 Processor -- start 2024-04-25T10:20:00.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.521+0000 7f6e8ca00640 1 -- start start 2024-04-25T10:20:00.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e8ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6e88099cb0 0x7f6e8806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e880d1120 0x7f6e880cf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e8ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e8809a090 0x7f6e8815d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e8ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8806c4b0 con 0x7f6e8809a090 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e8ca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e880cfe30 con 0x7f6e88099cb0 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e880d0080 con 0x7f6e880d1120 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.522+0000 7f6e87e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e8809a090 0x7f6e8815d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6e8809a090 0x7f6e8815d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41838/0 (socket says 172.21.15.165:41838) 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 -- 172.21.15.165:0/1016903835 learned_addr learned my addr 172.21.15.165:0/1016903835 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e7ea00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e880d1120 0x7f6e880cf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87400640 1 --2- 172.21.15.165:0/1016903835 >> v2:172.21.15.172:3300/0 conn(0x7f6e88099cb0 0x7f6e8806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 -- 172.21.15.165:0/1016903835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e880d1120 msgr2=0x7f6e880cf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e880d1120 0x7f6e880cf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 -- 172.21.15.165:0/1016903835 >> v2:172.21.15.172:3300/0 conn(0x7f6e88099cb0 msgr2=0x7f6e8806bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:00.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 --2- 172.21.15.165:0/1016903835 >> v2:172.21.15.172:3300/0 conn(0x7f6e88099cb0 0x7f6e8806bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e87e00640 1 -- 172.21.15.165:0/1016903835 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880d0360 con 0x7f6e8809a090 2024-04-25T10:20:00.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.523+0000 7f6e7ea00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e880d1120 0x7f6e880cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.524+0000 7f6e87e00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8809a090 0x7f6e8815d700 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f6e880730d0 tx=0x7f6e8002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a46c8f2d14acbd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.524+0000 7f6e86a00640 1 -- 172.21.15.165:0/1016903835 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80037070 con 0x7f6e8809a090 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e86a00640 1 -- 172.21.15.165:0/1016903835 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80032440 con 0x7f6e8809a090 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e86a00640 1 -- 172.21.15.165:0/1016903835 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80032810 con 0x7f6e8809a090 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 -- 172.21.15.165:0/1016903835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8809a090 msgr2=0x7f6e8815d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8809a090 0x7f6e8815d700 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7f6e880730d0 tx=0x7f6e8002a270 comp rx=0 tx=0).stop 2024-04-25T10:20:00.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 -- 172.21.15.165:0/1016903835 shutdown_connections 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e8809a090 0x7f6e8815d700 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/1016903835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e880d1120 0x7f6e880cf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/1016903835 >> v2:172.21.15.172:3300/0 conn(0x7f6e88099cb0 0x7f6e8806bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 -- 172.21.15.165:0/1016903835 >> 172.21.15.165:0/1016903835 conn(0x7f6e880b3250 msgr2=0x7f6e880b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 -- 172.21.15.165:0/1016903835 shutdown_connections 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.525+0000 7f6e8ca00640 1 -- 172.21.15.165:0/1016903835 wait complete. 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.527+0000 7f6e8ca00640 1 Processor -- start 2024-04-25T10:20:00.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.527+0000 7f6e8ca00640 1 -- start start 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e88099cb0 0x7f6e88159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8809a090 0x7f6e880c9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e8ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 0x7f6e8815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e880d0c00 con 0x7f6e880d1120 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e880d0920 con 0x7f6e8809a090 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e8ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8815dc40 con 0x7f6e88099cb0 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.528+0000 7f6e87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 0x7f6e8815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 0x7f6e8815a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41844/0 (socket says 172.21.15.165:41844) 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 -- 172.21.15.165:0/281179880 learned_addr learned my addr 172.21.15.165:0/281179880 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87400640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e88099cb0 0x7f6e88159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e7ea00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8809a090 0x7f6e880c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 -- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e88099cb0 msgr2=0x7f6e88159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e88099cb0 0x7f6e88159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 -- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8809a090 msgr2=0x7f6e880c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8809a090 0x7f6e880c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 -- 172.21.15.165:0/281179880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880c8a80 con 0x7f6e880d1120 2024-04-25T10:20:00.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.529+0000 7f6e87e00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 0x7f6e8815a1e0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f6e800090b0 tx=0x7f6e800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.530+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80037040 con 0x7f6e880d1120 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.530+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e80042070 con 0x7f6e880d1120 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.530+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e80030920 con 0x7f6e880d1120 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.530+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e880c8d60 con 0x7f6e880d1120 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.530+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e880c9d80 con 0x7f6e880d1120 2024-04-25T10:20:00.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.531+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e880b4010 con 0x7f6e880d1120 2024-04-25T10:20:00.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.532+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6e80030ac0 con 0x7f6e880d1120 2024-04-25T10:20:00.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.533+0000 7f6e7fe00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e6406b4d0 0x7f6e6406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:00.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.533+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f6e800aaa20 con 0x7f6e880d1120 2024-04-25T10:20:00.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.533+0000 7f6e87400640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e6406b4d0 0x7f6e6406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:00.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.534+0000 7f6e87400640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e6406b4d0 0x7f6e6406d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6e78004410 tx=0x7f6e780039f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:00.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.538+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6e80079330 con 0x7f6e880d1120 2024-04-25T10:20:00.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:00.729+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f6e880b7600 con 0x7f6e880d1120 2024-04-25T10:20:01.280 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:01.289 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:01.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.301+0000 7f6e7fe00640 1 -- 172.21.15.165:0/281179880 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v182) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6e8007d3d0 con 0x7f6e880d1120 2024-04-25T10:20:01.304 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.304+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e6406b4d0 msgr2=0x7f6e6406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.304+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e6406b4d0 0x7f6e6406d990 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6e78004410 tx=0x7f6e780039f0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 msgr2=0x7f6e8815a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 0x7f6e8815a1e0 secure :-1 s=READY pgs=2578 cs=0 l=1 rev1=1 crypto rx=0x7f6e800090b0 tx=0x7f6e800046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 shutdown_connections 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6e6406b4d0 0x7f6e6406d990 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6e880d1120 0x7f6e8815a1e0 unknown :-1 s=CLOSED pgs=2578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6e8809a090 0x7f6e880c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 --2- 172.21.15.165:0/281179880 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6e88099cb0 0x7f6e88159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 >> 172.21.15.165:0/281179880 conn(0x7f6e880b3250 msgr2=0x7f6e8815f130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 shutdown_connections 2024-04-25T10:20:01.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.305+0000 7f6e8ca00640 1 -- 172.21.15.165:0/281179880 wait complete. 2024-04-25T10:20:01.349 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:20:01.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.560+0000 7fd6b0e00640 1 Processor -- start 2024-04-25T10:20:01.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.560+0000 7fd6b0e00640 1 -- start start 2024-04-25T10:20:01.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.561+0000 7fd6b0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac144d50 0x7fd6ac06d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.561+0000 7fd6b0e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd6ac1452a0 0x7fd6ac06ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.561+0000 7fd6b0e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ac0cccc0 con 0x7fd6ac1446d0 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.561+0000 7fd6b0e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6ac0b2b20 con 0x7fd6ac1452a0 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.561+0000 7fd6b0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac06d400 con 0x7fd6ac144d50 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd6ac1452a0 0x7fd6ac06ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd6ac1452a0 0x7fd6ac06ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41858/0 (socket says 172.21.15.165:41858) 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 -- 172.21.15.165:0/1086091263 learned_addr learned my addr 172.21.15.165:0/1086091263 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6a9400640 1 --2- 172.21.15.165:0/1086091263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac144d50 0x7fd6ac06d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 -- 172.21.15.165:0/1086091263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac144d50 msgr2=0x7fd6ac06d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 --2- 172.21.15.165:0/1086091263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac144d50 0x7fd6ac06d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 -- 172.21.15.165:0/1086091263 >> v1:172.21.15.172:6789/0 conn(0x7fd6ac1446d0 legacy=0x7fd6ac0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:01.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.562+0000 7fd6aa800640 1 -- 172.21.15.165:0/1086091263 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0cc140 con 0x7fd6ac1452a0 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.563+0000 7fd6aa800640 1 --2- 172.21.15.165:0/1086091263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac06ddd0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009e50 tx=0x7fd6a402a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bced62b91626451b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.563+0000 7fd6a8a00640 1 -- 172.21.15.165:0/1086091263 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4036070 con 0x7fd6ac1452a0 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.563+0000 7fd6a8a00640 1 -- 172.21.15.165:0/1086091263 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a4031600 con 0x7fd6ac1452a0 2024-04-25T10:20:01.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.563+0000 7fd6a8a00640 1 -- 172.21.15.165:0/1086091263 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a40319d0 con 0x7fd6ac1452a0 2024-04-25T10:20:01.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 -- 172.21.15.165:0/1086091263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 msgr2=0x7fd6ac06ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:01.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/1086091263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac06ddd0 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4009e50 tx=0x7fd6a402a6c0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 -- 172.21.15.165:0/1086091263 shutdown_connections 2024-04-25T10:20:01.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/1086091263 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac06ddd0 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/1086091263 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac144d50 0x7fd6ac06d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 -- 172.21.15.165:0/1086091263 >> 172.21.15.165:0/1086091263 conn(0x7fd6ac06abe0 msgr2=0x7fd6ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:01.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 -- 172.21.15.165:0/1086091263 shutdown_connections 2024-04-25T10:20:01.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.564+0000 7fd6b0e00640 1 -- 172.21.15.165:0/1086091263 wait complete. 2024-04-25T10:20:01.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.566+0000 7fd6b0e00640 1 Processor -- start 2024-04-25T10:20:01.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.566+0000 7fd6b0e00640 1 -- start start 2024-04-25T10:20:01.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6b0e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac1446d0 0x7fd6ac0cd2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:01.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6b0e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6ac144d50 0x7fd6ac1f3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:01.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6b0e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac1f40c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:01.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6b0e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0cc9e0 con 0x7fd6ac1452a0 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6b0e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac0cc700 con 0x7fd6ac144d50 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6b0e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ac144ab0 con 0x7fd6ac1446d0 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac1f40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac1f40c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41872/0 (socket says 172.21.15.165:41872) 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 -- 172.21.15.165:0/3007909540 learned_addr learned my addr 172.21.15.165:0/3007909540 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6a9e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac1446d0 0x7fd6ac0cd2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 -- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac1446d0 msgr2=0x7fd6ac0cd2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac1446d0 0x7fd6ac0cd2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 -- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6ac144d50 msgr2=0x7fd6ac1f3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6ac144d50 0x7fd6ac1f3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.567+0000 7fd6aa800640 1 -- 172.21.15.165:0/3007909540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ac0cdbf0 con 0x7fd6ac1452a0 2024-04-25T10:20:01.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.568+0000 7fd6aa800640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac1f40c0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4034040 tx=0x7fd6a4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:01.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.568+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4036050 con 0x7fd6ac1452a0 2024-04-25T10:20:01.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.568+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6a4042070 con 0x7fd6ac1452a0 2024-04-25T10:20:01.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.568+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6a4009510 con 0x7fd6ac1452a0 2024-04-25T10:20:01.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.568+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ac0cdeb0 con 0x7fd6ac1452a0 2024-04-25T10:20:01.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.568+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ac0ce470 con 0x7fd6ac1452a0 2024-04-25T10:20:01.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.569+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ac073a00 con 0x7fd6ac1452a0 2024-04-25T10:20:01.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.570+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6a4007bd0 con 0x7fd6ac1452a0 2024-04-25T10:20:01.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.571+0000 7fd6a2000640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd68c06b4d0 0x7fd68c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:01.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.571+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6001+0+0 (secure 0 0 0) 0x7fd6a40ab320 con 0x7fd6ac1452a0 2024-04-25T10:20:01.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.572+0000 7fd6a9e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd68c06b4d0 0x7fd68c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:01.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.572+0000 7fd6a9e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd68c06b4d0 0x7fd68c06d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003d90 tx=0x7fd69c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:01.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.577+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6a4079a10 con 0x7fd6ac1452a0 2024-04-25T10:20:01.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:01.815+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd6ac0affc0 con 0x7fd6ac1452a0 2024-04-25T10:20:02.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.349+0000 7fd6a2000640 1 -- 172.21.15.165:0/3007909540 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 27 and data pool 28 v126) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd6a407dab0 con 0x7fd6ac1452a0 2024-04-25T10:20:02.353 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 27 and data pool 28 2024-04-25T10:20:02.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.362+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd68c06b4d0 msgr2=0x7fd68c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.362+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd68c06b4d0 0x7fd68c06d990 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fd69c003d90 tx=0x7fd69c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:02.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.362+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 msgr2=0x7fd6ac1f40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.362+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac1f40c0 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7fd6a4034040 tx=0x7fd6a4004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.363+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 shutdown_connections 2024-04-25T10:20:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.363+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd68c06b4d0 0x7fd68c06d990 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.363+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6ac1452a0 0x7fd6ac1f40c0 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.363+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6ac144d50 0x7fd6ac1f3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.363+0000 7fd6b0e00640 1 --2- 172.21.15.165:0/3007909540 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6ac1446d0 0x7fd6ac0cd2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.363+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 >> 172.21.15.165:0/3007909540 conn(0x7fd6ac06abe0 msgr2=0x7fd6ac071450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:02.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.365+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 shutdown_connections 2024-04-25T10:20:02.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.366+0000 7fd6b0e00640 1 -- 172.21.15.165:0/3007909540 wait complete. 2024-04-25T10:20:02.397 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:02.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.567+0000 7fc414c00640 1 Processor -- start 2024-04-25T10:20:02.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.567+0000 7fc414c00640 1 -- start start 2024-04-25T10:20:02.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc414c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc41015bd30 0x7fc4100cf900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc414c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc4100d1b00 0x7fc4100b4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc414c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4100d0350 con 0x7fc4100cfe40 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc414c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100d0630 con 0x7fc4100d1b00 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc414c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100d0910 con 0x7fc41015bd30 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc40d400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc4100cfe40 0x7fc4100d13f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56456/0 (socket says 172.21.15.165:56456) 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc40d400640 1 -- 172.21.15.165:0/2421678979 learned_addr learned my addr 172.21.15.165:0/2421678979 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:02.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc40e800640 1 --2- 172.21.15.165:0/2421678979 >> v2:172.21.15.165:3300/0 conn(0x7fc4100d1b00 0x7fc4100b4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.569+0000 7fc40de00640 1 --2- 172.21.15.165:0/2421678979 >> v2:172.21.15.172:3300/0 conn(0x7fc41015bd30 0x7fc4100cf900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.570+0000 7fc40ca00640 1 -- 172.21.15.165:0/2421678979 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2692259860 0 0) 0x7fc4100d0350 con 0x7fc4100cfe40 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.570+0000 7fc40ca00640 1 -- 172.21.15.165:0/2421678979 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3f4003520 con 0x7fc4100cfe40 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.570+0000 7fc40e800640 1 -- 172.21.15.165:0/2421678979 >> v2:172.21.15.172:3300/0 conn(0x7fc41015bd30 msgr2=0x7fc4100cf900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.570+0000 7fc40e800640 1 --2- 172.21.15.165:0/2421678979 >> v2:172.21.15.172:3300/0 conn(0x7fc41015bd30 0x7fc4100cf900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.570+0000 7fc40e800640 1 -- 172.21.15.165:0/2421678979 >> v1:172.21.15.165:6789/0 conn(0x7fc4100cfe40 legacy=0x7fc4100d13f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.570+0000 7fc40e800640 1 -- 172.21.15.165:0/2421678979 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100d0bf0 con 0x7fc4100d1b00 2024-04-25T10:20:02.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.571+0000 7fc40e800640 1 --2- 172.21.15.165:0/2421678979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4100d1b00 0x7fc4100b4a30 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc4100702b0 tx=0x7fc40802a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dfc447fe22f14c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:02.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.571+0000 7fc40ca00640 1 -- 172.21.15.165:0/2421678979 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408007740 con 0x7fc4100d1b00 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.571+0000 7fc40ca00640 1 -- 172.21.15.165:0/2421678979 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4080078e0 con 0x7fc4100d1b00 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.571+0000 7fc40ca00640 1 -- 172.21.15.165:0/2421678979 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408007cb0 con 0x7fc4100d1b00 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 -- 172.21.15.165:0/2421678979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4100d1b00 msgr2=0x7fc4100b4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 --2- 172.21.15.165:0/2421678979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4100d1b00 0x7fc4100b4a30 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7fc4100702b0 tx=0x7fc40802a710 comp rx=0 tx=0).stop 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 -- 172.21.15.165:0/2421678979 shutdown_connections 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 --2- 172.21.15.165:0/2421678979 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc4100d1b00 0x7fc4100b4a30 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 --2- 172.21.15.165:0/2421678979 >> v2:172.21.15.172:3300/0 conn(0x7fc41015bd30 0x7fc4100cf900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 -- 172.21.15.165:0/2421678979 >> 172.21.15.165:0/2421678979 conn(0x7fc41006f590 msgr2=0x7fc41006f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 -- 172.21.15.165:0/2421678979 shutdown_connections 2024-04-25T10:20:02.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.572+0000 7fc414c00640 1 -- 172.21.15.165:0/2421678979 wait complete. 2024-04-25T10:20:02.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.574+0000 7fc414c00640 1 Processor -- start 2024-04-25T10:20:02.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.574+0000 7fc414c00640 1 -- start start 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc414c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4100cfe40 0x7fc4100caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc414c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4100d1b00 0x7fc4100c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc414c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 0x7fc4100c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc414c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100b73b0 con 0x7fc41015bd30 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc414c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41015c110 con 0x7fc4100cfe40 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc414c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100b7530 con 0x7fc4100d1b00 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 0x7fc4100c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:02.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 0x7fc4100c8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41882/0 (socket says 172.21.15.165:41882) 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 -- 172.21.15.165:0/1292268893 learned_addr learned my addr 172.21.15.165:0/1292268893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40de00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4100cfe40 0x7fc4100caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40d400640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4100d1b00 0x7fc4100c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 -- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4100d1b00 msgr2=0x7fc4100c8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4100d1b00 0x7fc4100c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 -- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4100cfe40 msgr2=0x7fc4100caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.575+0000 7fc40e800640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4100cfe40 0x7fc4100caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc40e800640 1 -- 172.21.15.165:0/1292268893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100c9510 con 0x7fc41015bd30 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc40e800640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 0x7fc4100c8e90 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc40802ad00 tx=0x7fc408004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408040070 con 0x7fc41015bd30 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc408031bc0 con 0x7fc41015bd30 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408009450 con 0x7fc41015bd30 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4100c97f0 con 0x7fc41015bd30 2024-04-25T10:20:02.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.576+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4100c9db0 con 0x7fc41015bd30 2024-04-25T10:20:02.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.578+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410073990 con 0x7fc41015bd30 2024-04-25T10:20:02.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.578+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc408039040 con 0x7fc41015bd30 2024-04-25T10:20:02.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.579+0000 7fc406000640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3f006b4d0 0x7fc3f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:02.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.579+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc4080abb70 con 0x7fc41015bd30 2024-04-25T10:20:02.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.580+0000 7fc40de00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3f006b4d0 0x7fc3f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:02.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.580+0000 7fc40de00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3f006b4d0 0x7fc3f006d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:02.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.583+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc40807a970 con 0x7fc41015bd30 2024-04-25T10:20:02.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.775+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4100b42c0 con 0x7fc41015bd30 2024-04-25T10:20:02.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.777+0000 7fc406000640 1 -- 172.21.15.165:0/1292268893 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17001 (secure 0 0 0) 0x7fc40807a270 con 0x7fc41015bd30 2024-04-25T10:20:02.780 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:02.780 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":183,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:01.816619+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:02.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3f006b4d0 msgr2=0x7fc3f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3f006b4d0 0x7fc3f006d990 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fc400006fd0 tx=0x7fc400008040 comp rx=0 tx=0).stop 2024-04-25T10:20:02.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 msgr2=0x7fc4100c8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:02.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 0x7fc4100c8e90 secure :-1 s=READY pgs=2583 cs=0 l=1 rev1=1 crypto rx=0x7fc40802ad00 tx=0x7fc408004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:02.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 shutdown_connections 2024-04-25T10:20:02.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc3f006b4d0 0x7fc3f006d990 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc41015bd30 0x7fc4100c8e90 unknown :-1 s=CLOSED pgs=2583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc4100d1b00 0x7fc4100c8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 --2- 172.21.15.165:0/1292268893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc4100cfe40 0x7fc4100caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:02.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 >> 172.21.15.165:0/1292268893 conn(0x7fc41006f590 msgr2=0x7fc4100b3430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:02.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 shutdown_connections 2024-04-25T10:20:02.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.779+0000 7fc414c00640 1 -- 172.21.15.165:0/1292268893 wait complete. 2024-04-25T10:20:02.800 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:02.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.996+0000 7f7e06c00640 1 Processor -- start 2024-04-25T10:20:02.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.996+0000 7f7e06c00640 1 -- start start 2024-04-25T10:20:03.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.998+0000 7f7e06c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e000b93c0 0x7f7e000cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.998+0000 7f7e06c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7e000d04f0 0x7f7e000d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.998+0000 7f7e06c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e00161df0 con 0x7f7e0015db90 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.998+0000 7f7e06c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e000d0980 con 0x7f7e000d04f0 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.998+0000 7f7e06c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000d1f50 con 0x7f7e000b93c0 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.999+0000 7f7e04e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e000b93c0 0x7f7e000cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.999+0000 7f7e06200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f7e000d04f0 0x7f7e000d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.999+0000 7f7e05800640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f7e0015db90 0x7f7e001615a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:59940/0 (socket says 172.21.15.165:59940) 2024-04-25T10:20:03.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:02.999+0000 7f7e05800640 1 -- 172.21.15.165:0/756040868 learned_addr learned my addr 172.21.15.165:0/756040868 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:03.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.000+0000 7f7e04e00640 1 -- 172.21.15.165:0/756040868 >> v2:172.21.15.172:3300/0 conn(0x7f7e000d04f0 msgr2=0x7f7e000d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:03.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.000+0000 7f7e04e00640 1 --2- 172.21.15.165:0/756040868 >> v2:172.21.15.172:3300/0 conn(0x7f7e000d04f0 0x7f7e000d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.000+0000 7f7e04e00640 1 -- 172.21.15.165:0/756040868 >> v1:172.21.15.172:6789/0 conn(0x7f7e0015db90 legacy=0x7f7e001615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.000+0000 7f7e04e00640 1 -- 172.21.15.165:0/756040868 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e001620d0 con 0x7f7e000b93c0 2024-04-25T10:20:03.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.000+0000 7f7e06200640 1 --2- 172.21.15.165:0/756040868 >> v2:172.21.15.172:3300/0 conn(0x7f7e000d04f0 0x7f7e000d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:03.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.001+0000 7f7e04e00640 1 --2- 172.21.15.165:0/756040868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e000b93c0 0x7f7e000cffb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0097a0 tx=0x7f7dec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba74eac9b9f408d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:03.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.002+0000 7f7dffe00640 1 -- 172.21.15.165:0/756040868 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec037070 con 0x7f7e000b93c0 2024-04-25T10:20:03.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.002+0000 7f7dffe00640 1 -- 172.21.15.165:0/756040868 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec032440 con 0x7f7e000b93c0 2024-04-25T10:20:03.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.002+0000 7f7dffe00640 1 -- 172.21.15.165:0/756040868 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec032810 con 0x7f7e000b93c0 2024-04-25T10:20:03.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.002+0000 7f7e06c00640 1 -- 172.21.15.165:0/756040868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e000b93c0 msgr2=0x7f7e000cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.002+0000 7f7e06c00640 1 --2- 172.21.15.165:0/756040868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e000b93c0 0x7f7e000cffb0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7dec0097a0 tx=0x7f7dec02a180 comp rx=0 tx=0).stop 2024-04-25T10:20:03.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.003+0000 7f7e06c00640 1 -- 172.21.15.165:0/756040868 shutdown_connections 2024-04-25T10:20:03.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.003+0000 7f7e06c00640 1 --2- 172.21.15.165:0/756040868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e000b93c0 0x7f7e000cffb0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.003+0000 7f7e06c00640 1 --2- 172.21.15.165:0/756040868 >> v2:172.21.15.172:3300/0 conn(0x7f7e000d04f0 0x7f7e000d23e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.003+0000 7f7e06c00640 1 -- 172.21.15.165:0/756040868 >> 172.21.15.165:0/756040868 conn(0x7f7e000b52b0 msgr2=0x7f7e000b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:03.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.003+0000 7f7e06c00640 1 -- 172.21.15.165:0/756040868 shutdown_connections 2024-04-25T10:20:03.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.003+0000 7f7e06c00640 1 -- 172.21.15.165:0/756040868 wait complete. 2024-04-25T10:20:03.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.004+0000 7f7e06c00640 1 Processor -- start 2024-04-25T10:20:03.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.004+0000 7f7e06c00640 1 -- start start 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.005+0000 7f7e06c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e000b93c0 0x7f7e001e7400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.005+0000 7f7e06c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 0x7f7e001608f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.005+0000 7f7e06c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e0015db90 0x7f7e0015e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.005+0000 7f7e06c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000b68d0 con 0x7f7e000d04f0 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.005+0000 7f7e06c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000d1be0 con 0x7f7e000b93c0 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.005+0000 7f7e06c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e000b6a50 con 0x7f7e0015db90 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 0x7f7e001608f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 0x7f7e001608f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41890/0 (socket says 172.21.15.165:41890) 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 -- 172.21.15.165:0/1479797494 learned_addr learned my addr 172.21.15.165:0/1479797494 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e05800640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e000b93c0 0x7f7e001e7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 -- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e0015db90 msgr2=0x7f7e0015e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e0015db90 0x7f7e0015e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 -- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e000b93c0 msgr2=0x7f7e001e7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e000b93c0 0x7f7e001e7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e04e00640 1 -- 172.21.15.165:0/1479797494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e001e7c50 con 0x7f7e000d04f0 2024-04-25T10:20:03.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.006+0000 7f7e05800640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e000b93c0 0x7f7e001e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:03.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.007+0000 7f7e04e00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 0x7f7e001608f0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7dec035040 tx=0x7f7dec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:03.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.007+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec037040 con 0x7f7e000d04f0 2024-04-25T10:20:03.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.007+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dec03b4f0 con 0x7f7e000d04f0 2024-04-25T10:20:03.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.007+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dec03b8c0 con 0x7f7e000d04f0 2024-04-25T10:20:03.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.007+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e001e7f30 con 0x7f7e000d04f0 2024-04-25T10:20:03.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.007+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e001e84f0 con 0x7f7e000d04f0 2024-04-25T10:20:03.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.008+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e00073990 con 0x7f7e000d04f0 2024-04-25T10:20:03.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.009+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f7dec044020 con 0x7f7e000d04f0 2024-04-25T10:20:03.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.010+0000 7f7dfd600640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7de006b4d0 0x7f7de006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.010+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f7dec0aa520 con 0x7f7e000d04f0 2024-04-25T10:20:03.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.010+0000 7f7e05800640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7de006b4d0 0x7f7de006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.011+0000 7f7e05800640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7de006b4d0 0x7f7de006d990 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7e001583a0 tx=0x7f7df0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:03.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.015+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f7dec0792a0 con 0x7f7e000d04f0 2024-04-25T10:20:03.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.211+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e00159150 con 0x7f7e000d04f0 2024-04-25T10:20:03.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.215+0000 7f7dfd600640 1 -- 172.21.15.165:0/1479797494 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17001 (secure 0 0 0) 0x7f7dec078ba0 con 0x7f7e000d04f0 2024-04-25T10:20:03.217 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:03.218 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":183,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:01.816619+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7de006b4d0 msgr2=0x7f7de006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7de006b4d0 0x7f7de006d990 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f7e001583a0 tx=0x7f7df0009040 comp rx=0 tx=0).stop 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 msgr2=0x7f7e001608f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 0x7f7e001608f0 secure :-1 s=READY pgs=2584 cs=0 l=1 rev1=1 crypto rx=0x7f7dec035040 tx=0x7f7dec0046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 shutdown_connections 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f7de006b4d0 0x7f7de006d990 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f7e0015db90 0x7f7e0015e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f7e000d04f0 0x7f7e001608f0 unknown :-1 s=CLOSED pgs=2584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 --2- 172.21.15.165:0/1479797494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f7e000b93c0 0x7f7e001e7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.217+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 >> 172.21.15.165:0/1479797494 conn(0x7f7e000b52b0 msgr2=0x7f7e000d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.218+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 shutdown_connections 2024-04-25T10:20:03.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.218+0000 7f7e06c00640 1 -- 172.21.15.165:0/1479797494 wait complete. 2024-04-25T10:20:03.240 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:20:03.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.449+0000 7fca4e800640 1 Processor -- start 2024-04-25T10:20:03.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.449+0000 7fca4e800640 1 -- start start 2024-04-25T10:20:03.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.451+0000 7fca4e800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 0x7fca4806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.451+0000 7fca4e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca4806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.451+0000 7fca4e800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca480cccc0 con 0x7fca481446d0 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.451+0000 7fca4e800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca480b2b20 con 0x7fca48144d50 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.451+0000 7fca4e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4806d400 con 0x7fca481452a0 2024-04-25T10:20:03.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.452+0000 7fca3ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 0x7fca4806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.452+0000 7fca47e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca4806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.452+0000 7fca3ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 0x7fca4806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:58432/0 (socket says 172.21.15.165:58432) 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.452+0000 7fca47e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca4806ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:48048/0 (socket says 172.21.15.165:48048) 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.452+0000 7fca3ea00640 1 -- 172.21.15.165:0/228750966 learned_addr learned my addr 172.21.15.165:0/228750966 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.453+0000 7fca47e00640 1 -- 172.21.15.165:0/228750966 >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 msgr2=0x7fca4806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.453+0000 7fca47e00640 1 --2- 172.21.15.165:0/228750966 >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 0x7fca4806d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.453+0000 7fca47e00640 1 -- 172.21.15.165:0/228750966 >> v1:172.21.15.172:6789/0 conn(0x7fca481446d0 legacy=0x7fca480b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.453+0000 7fca47e00640 1 -- 172.21.15.165:0/228750966 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca480cc140 con 0x7fca481452a0 2024-04-25T10:20:03.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.453+0000 7fca3ea00640 1 --2- 172.21.15.165:0/228750966 >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 0x7fca4806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:03.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.454+0000 7fca47e00640 1 --2- 172.21.15.165:0/228750966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca4806ddd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fca40009d90 tx=0x7fca4002a660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7360738c35040ab5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:03.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.454+0000 7fca46a00640 1 -- 172.21.15.165:0/228750966 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40007740 con 0x7fca481452a0 2024-04-25T10:20:03.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.454+0000 7fca46a00640 1 -- 172.21.15.165:0/228750966 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca400078e0 con 0x7fca481452a0 2024-04-25T10:20:03.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca46a00640 1 -- 172.21.15.165:0/228750966 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca40007cb0 con 0x7fca481452a0 2024-04-25T10:20:03.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 -- 172.21.15.165:0/228750966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 msgr2=0x7fca4806ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 --2- 172.21.15.165:0/228750966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca4806ddd0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fca40009d90 tx=0x7fca4002a660 comp rx=0 tx=0).stop 2024-04-25T10:20:03.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 -- 172.21.15.165:0/228750966 shutdown_connections 2024-04-25T10:20:03.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 --2- 172.21.15.165:0/228750966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca4806ddd0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 --2- 172.21.15.165:0/228750966 >> v2:172.21.15.172:3300/0 conn(0x7fca48144d50 0x7fca4806d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 -- 172.21.15.165:0/228750966 >> 172.21.15.165:0/228750966 conn(0x7fca4806abe0 msgr2=0x7fca4806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:03.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 -- 172.21.15.165:0/228750966 shutdown_connections 2024-04-25T10:20:03.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.455+0000 7fca4e800640 1 -- 172.21.15.165:0/228750966 wait complete. 2024-04-25T10:20:03.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.457+0000 7fca4e800640 1 Processor -- start 2024-04-25T10:20:03.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.458+0000 7fca4e800640 1 -- start start 2024-04-25T10:20:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca4e800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca481446d0 0x7fca480ced30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca4e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 0x7fca481504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca4e800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca481e5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca4e800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480cc9e0 con 0x7fca48144d50 2024-04-25T10:20:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca4e800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca480cc700 con 0x7fca481446d0 2024-04-25T10:20:03.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca4e800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca48144ab0 con 0x7fca481452a0 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 0x7fca481504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 0x7fca481504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41904/0 (socket says 172.21.15.165:41904) 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 -- 172.21.15.165:0/3840518062 learned_addr learned my addr 172.21.15.165:0/3840518062 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca47400640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca481446d0 0x7fca480ced30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 -- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 msgr2=0x7fca481e5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca481e5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 -- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca481446d0 msgr2=0x7fca480ced30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca481446d0 0x7fca480ced30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca3ea00640 1 -- 172.21.15.165:0/3840518062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca4813e110 con 0x7fca48144d50 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.459+0000 7fca47400640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca481446d0 0x7fca480ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:03.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.460+0000 7fca3ea00640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 0x7fca481504a0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fca3000e9b0 tx=0x7fca3000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:03.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.460+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca30017070 con 0x7fca48144d50 2024-04-25T10:20:03.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.460+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca3000c9a0 con 0x7fca48144d50 2024-04-25T10:20:03.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.460+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca3000cd50 con 0x7fca48144d50 2024-04-25T10:20:03.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.460+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca4813d070 con 0x7fca48144d50 2024-04-25T10:20:03.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.460+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4813d630 con 0x7fca48144d50 2024-04-25T10:20:03.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.462+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca48073a00 con 0x7fca48144d50 2024-04-25T10:20:03.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.462+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fca3001a070 con 0x7fca48144d50 2024-04-25T10:20:03.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.463+0000 7fca3fe00640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:03.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.463+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fca3008a180 con 0x7fca48144d50 2024-04-25T10:20:03.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.464+0000 7fca47400640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:03.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.464+0000 7fca47400640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca2406b500 0x7fca2406d9c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fca380015e0 tx=0x7fca38009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:03.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.469+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fca30058f00 con 0x7fca48144d50 2024-04-25T10:20:03.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:03.709+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fca480b0cd0 con 0x7fca48144d50 2024-04-25T10:20:04.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.357+0000 7fca3fe00640 1 -- 172.21.15.165:0/3840518062 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v129) v1 ==== 118+0+0 (secure 0 0 0) 0x7fca30058800 con 0x7fca48144d50 2024-04-25T10:20:04.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca2406b500 msgr2=0x7fca2406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca2406b500 0x7fca2406d9c0 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fca380015e0 tx=0x7fca38009040 comp rx=0 tx=0).stop 2024-04-25T10:20:04.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 msgr2=0x7fca481504a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 0x7fca481504a0 secure :-1 s=READY pgs=2585 cs=0 l=1 rev1=1 crypto rx=0x7fca3000e9b0 tx=0x7fca3000ee70 comp rx=0 tx=0).stop 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 shutdown_connections 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fca2406b500 0x7fca2406d9c0 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fca481452a0 0x7fca481e5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fca48144d50 0x7fca481504a0 unknown :-1 s=CLOSED pgs=2585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 --2- 172.21.15.165:0/3840518062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fca481446d0 0x7fca480ced30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 >> 172.21.15.165:0/3840518062 conn(0x7fca4806abe0 msgr2=0x7fca480af340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:04.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 shutdown_connections 2024-04-25T10:20:04.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.361+0000 7fca4e800640 1 -- 172.21.15.165:0/3840518062 wait complete. 2024-04-25T10:20:04.384 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:20:04.385 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:20:04.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.581+0000 7fe70d800640 1 Processor -- start 2024-04-25T10:20:04.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.581+0000 7fe70d800640 1 -- start start 2024-04-25T10:20:04.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.582+0000 7fe70d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe7080b4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe70d800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080d1d50 con 0x7fe708158230 2024-04-25T10:20:04.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe70d800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7080d2020 con 0x7fe70806c7e0 2024-04-25T10:20:04.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe70d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe708158610 con 0x7fe70806c330 2024-04-25T10:20:04.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe706a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe708158230 0x7fe7080d1640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56466/0 (socket says 172.21.15.165:56466) 2024-04-25T10:20:04.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe706a00640 1 -- 172.21.15.165:0/2434692512 learned_addr learned my addr 172.21.15.165:0/2434692512 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:04.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 753896220 0 0) 0x7fe7080d1d50 con 0x7fe708158230 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ec003520 con 0x7fe708158230 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.583+0000 7fe6ffe00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe7080b4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911717002 0 0) 0x7fe6ec003520 con 0x7fe708158230 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 msgr2=0x7fe7080b4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe7080b4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 >> v1:172.21.15.172:6789/0 conn(0x7fe70806c7e0 legacy=0x7fe70806f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7080d2300 con 0x7fe708158230 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe6f8001670 con 0x7fe708158230 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 >> v1:172.21.15.165:6789/0 conn(0x7fe708158230 legacy=0x7fe7080d1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe6ec003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe6ec004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec004d40 0x7fe6ec0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe708158610 con 0x7fe6ec0039e0 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00bbd0 con 0x7fe6ec004d40 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6ec00bdd0 con 0x7fe70806c330 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe706a00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe6ec003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.584+0000 7fe6ffe00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe6ec004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.585+0000 7fe706a00640 1 -- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 msgr2=0x7fe6ec004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.585+0000 7fe706a00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe6ec004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.585+0000 7fe706a00640 1 -- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec004d40 msgr2=0x7fe6ec0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.585+0000 7fe706a00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe6ec004d40 0x7fe6ec0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.585+0000 7fe706a00640 1 -- 172.21.15.165:0/2434692512 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ec003520 con 0x7fe6ec0039e0 2024-04-25T10:20:04.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.585+0000 7fe6ffe00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe70806c330 0x7fe6ec004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:04.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.586+0000 7fe706a00640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe6ec003e00 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c190 tx=0x7fe6ec005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b49d8138cecca8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:04.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.586+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8007c60 con 0x7fe6ec0039e0 2024-04-25T10:20:04.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.586+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8005ad0 con 0x7fe6ec0039e0 2024-04-25T10:20:04.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.586+0000 7fe706000640 1 -- 172.21.15.165:0/2434692512 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8007070 con 0x7fe6ec0039e0 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe70d800640 1 -- 172.21.15.165:0/2434692512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 msgr2=0x7fe6ec003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe70d800640 1 --2- 172.21.15.165:0/2434692512 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe6ec003e00 secure :-1 s=READY pgs=2587 cs=0 l=1 rev1=1 crypto rx=0x7fe6f800c190 tx=0x7fe6ec005830 comp rx=0 tx=0).stop 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe707400640 1 -- 172.21.15.165:0/2434692512 reap_dead start 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe70d800640 1 -- 172.21.15.165:0/2434692512 shutdown_connections 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe70d800640 1 -- 172.21.15.165:0/2434692512 >> 172.21.15.165:0/2434692512 conn(0x7fe708061600 msgr2=0x7fe7080619e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe70d800640 1 -- 172.21.15.165:0/2434692512 shutdown_connections 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.587+0000 7fe70d800640 1 -- 172.21.15.165:0/2434692512 wait complete. 2024-04-25T10:20:04.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.589+0000 7fe70d800640 1 Processor -- start 2024-04-25T10:20:04.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.589+0000 7fe70d800640 1 -- start start 2024-04-25T10:20:04.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe70d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe7081e5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe70d800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7080b3b80 0x7fe7081e43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe70d800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7081e48f0 0x7fe70813ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe70d800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7080d25e0 con 0x7fe6ec0039e0 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe70d800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe70806bea0 con 0x7fe7080b3b80 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe70d800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe70815e320 con 0x7fe7081e48f0 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe7081e5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe7081e5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41916/0 (socket says 172.21.15.165:41916) 2024-04-25T10:20:04.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 -- 172.21.15.165:0/3944850155 learned_addr learned my addr 172.21.15.165:0/3944850155 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:04.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe707400640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7081e48f0 0x7fe70813ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 -- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7081e48f0 msgr2=0x7fe70813ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe6ffe00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7080b3b80 0x7fe7081e43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7081e48f0 0x7fe70813ecd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 -- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7080b3b80 msgr2=0x7fe7081e43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7080b3b80 0x7fe7081e43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.590+0000 7fe706a00640 1 -- 172.21.15.165:0/3944850155 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe70815b290 con 0x7fe6ec0039e0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.591+0000 7fe706a00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe7081e5580 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802ad60 tx=0x7fe6f8001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.591+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f8043070 con 0x7fe6ec0039e0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.591+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f8038040 con 0x7fe6ec0039e0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.591+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80072b0 con 0x7fe6ec0039e0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.591+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70815b570 con 0x7fe6ec0039e0 2024-04-25T10:20:04.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.591+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe70814ffa0 con 0x7fe6ec0039e0 2024-04-25T10:20:04.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.592+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe708073a00 con 0x7fe6ec0039e0 2024-04-25T10:20:04.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.593+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe6f8007450 con 0x7fe6ec0039e0 2024-04-25T10:20:04.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.594+0000 7fe6fea00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe6e406b4d0 0x7fe6e406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:04.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.595+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe6f80a9ff0 con 0x7fe6ec0039e0 2024-04-25T10:20:04.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.595+0000 7fe6ffe00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe6e406b4d0 0x7fe6e406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:04.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.595+0000 7fe6ffe00640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe6e406b4d0 0x7fe6e406d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe7081e6620 tx=0x7fe6f0008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:04.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.599+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe6f8005aa0 con 0x7fe6ec0039e0 2024-04-25T10:20:04.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:04.839+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe70815beb0 con 0x7fe6ec0039e0 2024-04-25T10:20:05.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.370+0000 7fe6fea00640 1 -- 172.21.15.165:0/3944850155 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v130) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe6f8078650 con 0x7fe6ec0039e0 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe6e406b4d0 msgr2=0x7fe6e406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe6e406b4d0 0x7fe6e406d990 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fe7081e6620 tx=0x7fe6f0008040 comp rx=0 tx=0).stop 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 msgr2=0x7fe7081e5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe7081e5580 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7fe6f802ad60 tx=0x7fe6f8001970 comp rx=0 tx=0).stop 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 shutdown_connections 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe6e406b4d0 0x7fe6e406d990 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe7081e48f0 0x7fe70813ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe7080b3b80 0x7fe7081e43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 --2- 172.21.15.165:0/3944850155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe6ec0039e0 0x7fe7081e5580 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 >> 172.21.15.165:0/3944850155 conn(0x7fe708061600 msgr2=0x7fe7080ce630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:05.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 shutdown_connections 2024-04-25T10:20:05.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.376+0000 7fe70d800640 1 -- 172.21.15.165:0/3944850155 wait complete. 2024-04-25T10:20:05.400 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:20:05.400 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:05.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.590+0000 7f9083600640 1 Processor -- start 2024-04-25T10:20:05.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.590+0000 7f9083600640 1 -- start start 2024-04-25T10:20:05.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.592+0000 7f9083600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f907c154ee0 0x7f907c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.592+0000 7f9083600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 0x7f907c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.592+0000 7f9083600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f907c071480 con 0x7f907c0ce3c0 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.592+0000 7f9083600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f907c0ce830 con 0x7f907c154ee0 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.592+0000 7f9083600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c072090 con 0x7f907c0cfcf0 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.593+0000 7f907be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 0x7f907c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.593+0000 7f907be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 0x7f907c0cde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:48096/0 (socket says 172.21.15.165:48096) 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.593+0000 7f907be00640 1 -- 172.21.15.165:0/4013559288 learned_addr learned my addr 172.21.15.165:0/4013559288 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:05.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.593+0000 7f9080c00640 1 --2- 172.21.15.165:0/4013559288 >> v2:172.21.15.172:3300/0 conn(0x7f907c154ee0 0x7f907c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.594+0000 7f907be00640 1 -- 172.21.15.165:0/4013559288 >> v2:172.21.15.172:3300/0 conn(0x7f907c154ee0 msgr2=0x7f907c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.594+0000 7f907be00640 1 --2- 172.21.15.165:0/4013559288 >> v2:172.21.15.172:3300/0 conn(0x7f907c154ee0 0x7f907c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.594+0000 7f907be00640 1 -- 172.21.15.165:0/4013559288 >> v1:172.21.15.172:6789/0 conn(0x7f907c0ce3c0 legacy=0x7f907c15e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.594+0000 7f907be00640 1 -- 172.21.15.165:0/4013559288 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c071760 con 0x7f907c0cfcf0 2024-04-25T10:20:05.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.595+0000 7f907be00640 1 --2- 172.21.15.165:0/4013559288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 0x7f907c0cde80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9064009e50 tx=0x7f906402c920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51e9860d2d1ac160 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:05.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f907b400640 1 -- 172.21.15.165:0/4013559288 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064032610 con 0x7f907c0cfcf0 2024-04-25T10:20:05.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f907b400640 1 -- 172.21.15.165:0/4013559288 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9064037070 con 0x7f907c0cfcf0 2024-04-25T10:20:05.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f907b400640 1 -- 172.21.15.165:0/4013559288 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90640329d0 con 0x7f907c0cfcf0 2024-04-25T10:20:05.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f9083600640 1 -- 172.21.15.165:0/4013559288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 msgr2=0x7f907c0cde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f9083600640 1 --2- 172.21.15.165:0/4013559288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 0x7f907c0cde80 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9064009e50 tx=0x7f906402c920 comp rx=0 tx=0).stop 2024-04-25T10:20:05.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f9083600640 1 -- 172.21.15.165:0/4013559288 shutdown_connections 2024-04-25T10:20:05.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f9083600640 1 --2- 172.21.15.165:0/4013559288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0cfcf0 0x7f907c0cde80 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f9083600640 1 --2- 172.21.15.165:0/4013559288 >> v2:172.21.15.172:3300/0 conn(0x7f907c154ee0 0x7f907c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.596+0000 7f9083600640 1 -- 172.21.15.165:0/4013559288 >> 172.21.15.165:0/4013559288 conn(0x7f907c06abe0 msgr2=0x7f907c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:05.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.597+0000 7f9083600640 1 -- 172.21.15.165:0/4013559288 shutdown_connections 2024-04-25T10:20:05.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.597+0000 7f9083600640 1 -- 172.21.15.165:0/4013559288 wait complete. 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.598+0000 7f9083600640 1 Processor -- start 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.598+0000 7f9083600640 1 -- start start 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.599+0000 7f9083600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0ce3c0 0x7f907c14f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.599+0000 7f9083600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 0x7f907c1f4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.599+0000 7f9083600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f907c154ee0 0x7f907c1599e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.599+0000 7f9083600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c071230 con 0x7f907c0cfcf0 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.599+0000 7f9083600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f907c071d20 con 0x7f907c154ee0 2024-04-25T10:20:05.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.599+0000 7f9083600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f907c1552c0 con 0x7f907c0ce3c0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 0x7f907c1f4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 0x7f907c1f4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41918/0 (socket says 172.21.15.165:41918) 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 -- 172.21.15.165:0/863406406 learned_addr learned my addr 172.21.15.165:0/863406406 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f9080c00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0ce3c0 0x7f907c14f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f9081600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f907c154ee0 0x7f907c1599e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 -- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0ce3c0 msgr2=0x7f907c14f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0ce3c0 0x7f907c14f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 -- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f907c154ee0 msgr2=0x7f907c1599e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f907c154ee0 0x7f907c1599e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 -- 172.21.15.165:0/863406406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c1436f0 con 0x7f907c0cfcf0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.600+0000 7f907be00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 0x7f907c1f4ce0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f9064009f70 tx=0x7f9064004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.601+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064041070 con 0x7f907c0cfcf0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.601+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f906403b620 con 0x7f907c0cfcf0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.601+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f907c142e50 con 0x7f907c0cfcf0 2024-04-25T10:20:05.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.601+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9064009510 con 0x7f907c0cfcf0 2024-04-25T10:20:05.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.601+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c143410 con 0x7f907c0cfcf0 2024-04-25T10:20:05.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.602+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f907c073990 con 0x7f907c0cfcf0 2024-04-25T10:20:05.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.603+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9064005d10 con 0x7f907c0cfcf0 2024-04-25T10:20:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.604+0000 7f9078c00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f905806b4d0 0x7f905806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.604+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f90640aac60 con 0x7f907c0cfcf0 2024-04-25T10:20:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.604+0000 7f9080c00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f905806b4d0 0x7f905806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:05.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.605+0000 7f9080c00640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f905806b4d0 0x7f905806d990 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f907c1543a0 tx=0x7f906c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:05.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.609+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f906404b030 con 0x7f907c0cfcf0 2024-04-25T10:20:05.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.830+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f907c06d360 con 0x7f907c0cfcf0 2024-04-25T10:20:05.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.831+0000 7f9078c00640 1 -- 172.21.15.165:0/863406406 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4964 (secure 0 0 0) 0x7f906402f070 con 0x7f907c0cfcf0 2024-04-25T10:20:05.834 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:05.834 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:01.817104+0000","modified":"2024-04-25T10:20:04.840723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11211":{"gid":11211,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1159947775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1159947775},{"type":"v1","addr":"172.21.15.165:6837","nonce":1159947775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:20:05.835 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 130 2024-04-25T10:20:05.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f905806b4d0 msgr2=0x7f905806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f905806b4d0 0x7f905806d990 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f907c1543a0 tx=0x7f906c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:05.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 msgr2=0x7f907c1f4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:05.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 0x7f907c1f4ce0 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f9064009f70 tx=0x7f9064004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 shutdown_connections 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f905806b4d0 0x7f905806d990 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f907c154ee0 0x7f907c1599e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f907c0cfcf0 0x7f907c1f4ce0 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 --2- 172.21.15.165:0/863406406 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f907c0ce3c0 0x7f907c14f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.834+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 >> 172.21.15.165:0/863406406 conn(0x7f907c06abe0 msgr2=0x7f907c0b2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.835+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 shutdown_connections 2024-04-25T10:20:05.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:05.835+0000 7f9083600640 1 -- 172.21.15.165:0/863406406 wait complete. 2024-04-25T10:20:05.856 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:06.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.046+0000 7f8ad8a00640 1 Processor -- start 2024-04-25T10:20:06.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.046+0000 7f8ad8a00640 1 -- start start 2024-04-25T10:20:06.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.047+0000 7f8ad8a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8ad4144440 0x7f8ad4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.047+0000 7f8ad8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8ad40cda30 0x7f8ad415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad8a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad415bf30 con 0x7f8ad4071530 2024-04-25T10:20:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad8a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad40719a0 con 0x7f8ad40cda30 2024-04-25T10:20:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad8a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad415c860 con 0x7f8ad4144440 2024-04-25T10:20:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad3200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8ad40cda30 0x7f8ad415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ac9e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8ad4071530 0x7f8ad40cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56476/0 (socket says 172.21.15.165:56476) 2024-04-25T10:20:06.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ac9e00640 1 -- 172.21.15.165:0/1807381900 learned_addr learned my addr 172.21.15.165:0/1807381900 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad2800640 1 --2- 172.21.15.165:0/1807381900 >> v2:172.21.15.172:3300/0 conn(0x7f8ad4144440 0x7f8ad4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad3200640 1 -- 172.21.15.165:0/1807381900 >> v2:172.21.15.172:3300/0 conn(0x7f8ad4144440 msgr2=0x7f8ad4070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad3200640 1 --2- 172.21.15.165:0/1807381900 >> v2:172.21.15.172:3300/0 conn(0x7f8ad4144440 0x7f8ad4070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad3200640 1 -- 172.21.15.165:0/1807381900 >> v1:172.21.15.165:6789/0 conn(0x7f8ad4071530 legacy=0x7f8ad40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad3200640 1 -- 172.21.15.165:0/1807381900 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad415c210 con 0x7f8ad40cda30 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad2800640 1 --2- 172.21.15.165:0/1807381900 >> v2:172.21.15.172:3300/0 conn(0x7f8ad4144440 0x7f8ad4070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:06.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.048+0000 7f8ad1e00640 1 -- 172.21.15.165:0/1807381900 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199669046 0 0) 0x7f8ad415bf30 con 0x7f8ad4071530 2024-04-25T10:20:06.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.050+0000 7f8ad3200640 1 --2- 172.21.15.165:0/1807381900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad40cda30 0x7f8ad415ccf0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f8ad406f470 tx=0x7f8acc02a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=701a0275b6c891c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.050+0000 7f8ad1e00640 1 -- 172.21.15.165:0/1807381900 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc037070 con 0x7f8ad40cda30 2024-04-25T10:20:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.050+0000 7f8ad1e00640 1 -- 172.21.15.165:0/1807381900 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc032440 con 0x7f8ad40cda30 2024-04-25T10:20:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.050+0000 7f8ad1e00640 1 -- 172.21.15.165:0/1807381900 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc032810 con 0x7f8ad40cda30 2024-04-25T10:20:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.050+0000 7f8ad8a00640 1 -- 172.21.15.165:0/1807381900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad40cda30 msgr2=0x7f8ad415ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.050+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/1807381900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad40cda30 0x7f8ad415ccf0 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f8ad406f470 tx=0x7f8acc02a230 comp rx=0 tx=0).stop 2024-04-25T10:20:06.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.051+0000 7f8ad8a00640 1 -- 172.21.15.165:0/1807381900 shutdown_connections 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.051+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/1807381900 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad40cda30 0x7f8ad415ccf0 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.051+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/1807381900 >> v2:172.21.15.172:3300/0 conn(0x7f8ad4144440 0x7f8ad4070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.051+0000 7f8ad8a00640 1 -- 172.21.15.165:0/1807381900 >> 172.21.15.165:0/1807381900 conn(0x7f8ad406abe0 msgr2=0x7f8ad406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.051+0000 7f8ad8a00640 1 -- 172.21.15.165:0/1807381900 shutdown_connections 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.051+0000 7f8ad8a00640 1 -- 172.21.15.165:0/1807381900 wait complete. 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.052+0000 7f8ad8a00640 1 Processor -- start 2024-04-25T10:20:06.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.052+0000 7f8ad8a00640 1 -- start start 2024-04-25T10:20:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.053+0000 7f8ad8a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ad4071530 0x7f8ad41f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.053+0000 7f8ad8a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad40cda30 0x7f8ad40cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.053+0000 7f8ad8a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 0x7f8ad40cba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.053+0000 7f8ad8a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad415bc30 con 0x7f8ad4144440 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.053+0000 7f8ad8a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40cdea0 con 0x7f8ad40cda30 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.053+0000 7f8ad8a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad415d230 con 0x7f8ad4071530 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 0x7f8ad40cba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 0x7f8ad40cba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41938/0 (socket says 172.21.15.165:41938) 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 -- 172.21.15.165:0/693461354 learned_addr learned my addr 172.21.15.165:0/693461354 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:06.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ac9e00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad40cda30 0x7f8ad40cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 -- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ad4071530 msgr2=0x7f8ad41f3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ad4071530 0x7f8ad41f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 -- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad40cda30 msgr2=0x7f8ad40cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad40cda30 0x7f8ad40cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 -- 172.21.15.165:0/693461354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad41f3770 con 0x7f8ad4144440 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.054+0000 7f8ad3200640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 0x7f8ad40cba30 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009890 tx=0x7f8acc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.055+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc037040 con 0x7f8ad4144440 2024-04-25T10:20:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.055+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8acc042070 con 0x7f8ad4144440 2024-04-25T10:20:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.055+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8acc030b30 con 0x7f8ad4144440 2024-04-25T10:20:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.055+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad40c8ff0 con 0x7f8ad4144440 2024-04-25T10:20:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.055+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad40c95b0 con 0x7f8ad4144440 2024-04-25T10:20:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.056+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad40c9cc0 con 0x7f8ad4144440 2024-04-25T10:20:06.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.057+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8acc030cd0 con 0x7f8ad4144440 2024-04-25T10:20:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.058+0000 7f8acb400640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ab006b4d0 0x7f8ab006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.058+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8acc0ab080 con 0x7f8ad4144440 2024-04-25T10:20:06.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.058+0000 7f8ad2800640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ab006b4d0 0x7f8ab006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.059+0000 7f8ad2800640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ab006b4d0 0x7f8ab006d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4008e00 tx=0x7f8ac4008330 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.062+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8acc045360 con 0x7f8ad4144440 2024-04-25T10:20:06.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.260+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ad41e2d80 con 0x7f8ad4144440 2024-04-25T10:20:06.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.263+0000 7f8acb400640 1 -- 172.21.15.165:0/693461354 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f8acc079cd0 con 0x7f8ad4144440 2024-04-25T10:20:06.266 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:06.266 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":184,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:03.322848+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:06.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.265+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ab006b4d0 msgr2=0x7f8ab006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.265+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ab006b4d0 0x7f8ab006d990 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4008e00 tx=0x7f8ac4008330 comp rx=0 tx=0).stop 2024-04-25T10:20:06.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.265+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 msgr2=0x7f8ad40cba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.265+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 0x7f8ad40cba30 secure :-1 s=READY pgs=2592 cs=0 l=1 rev1=1 crypto rx=0x7f8acc009890 tx=0x7f8acc0046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 shutdown_connections 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8ab006b4d0 0x7f8ab006d990 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8ad4144440 0x7f8ad40cba30 unknown :-1 s=CLOSED pgs=2592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8ad40cda30 0x7f8ad40cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 --2- 172.21.15.165:0/693461354 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8ad4071530 0x7f8ad41f3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 >> 172.21.15.165:0/693461354 conn(0x7f8ad406abe0 msgr2=0x7f8ad41e24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:06.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 shutdown_connections 2024-04-25T10:20:06.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.266+0000 7f8ad8a00640 1 -- 172.21.15.165:0/693461354 wait complete. 2024-04-25T10:20:06.288 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:20:06.288 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:06.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.474+0000 7efd56200640 1 Processor -- start 2024-04-25T10:20:06.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.475+0000 7efd56200640 1 -- start start 2024-04-25T10:20:06.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd56200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd500b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd56200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd501615a0 0x7efd500cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd56200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd500d33f0 con 0x7efd500d04f0 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd56200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd500d0980 con 0x7efd501615a0 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd56200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500d1f50 con 0x7efd5015db90 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd47e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd501615a0 0x7efd500cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd47e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efd501615a0 0x7efd500cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41944/0 (socket says 172.21.15.165:41944) 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.476+0000 7efd47e00640 1 -- 172.21.15.165:0/2245130586 learned_addr learned my addr 172.21.15.165:0/2245130586 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.477+0000 7efd54e00640 1 --2- 172.21.15.165:0/2245130586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd500b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.477+0000 7efd47e00640 1 -- 172.21.15.165:0/2245130586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 msgr2=0x7efd500b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.477+0000 7efd47e00640 1 --2- 172.21.15.165:0/2245130586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd500b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.477+0000 7efd47e00640 1 -- 172.21.15.165:0/2245130586 >> v1:172.21.15.172:6789/0 conn(0x7efd500d04f0 legacy=0x7efd50061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.477+0000 7efd47e00640 1 -- 172.21.15.165:0/2245130586 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd50161df0 con 0x7efd501615a0 2024-04-25T10:20:06.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.478+0000 7efd47e00640 1 --2- 172.21.15.165:0/2245130586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd501615a0 0x7efd500cffb0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7efd38009870 tx=0x7efd3802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d3641efb56e4596 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.478+0000 7efd4fe00640 1 -- 172.21.15.165:0/2245130586 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38037070 con 0x7efd501615a0 2024-04-25T10:20:06.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.478+0000 7efd4fe00640 1 -- 172.21.15.165:0/2245130586 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38032440 con 0x7efd501615a0 2024-04-25T10:20:06.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.478+0000 7efd4fe00640 1 -- 172.21.15.165:0/2245130586 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd38032810 con 0x7efd501615a0 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 -- 172.21.15.165:0/2245130586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd501615a0 msgr2=0x7efd500cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 --2- 172.21.15.165:0/2245130586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd501615a0 0x7efd500cffb0 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7efd38009870 tx=0x7efd3802a210 comp rx=0 tx=0).stop 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 -- 172.21.15.165:0/2245130586 shutdown_connections 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 --2- 172.21.15.165:0/2245130586 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd501615a0 0x7efd500cffb0 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 --2- 172.21.15.165:0/2245130586 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd500b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 -- 172.21.15.165:0/2245130586 >> 172.21.15.165:0/2245130586 conn(0x7efd500b52b0 msgr2=0x7efd500b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 -- 172.21.15.165:0/2245130586 shutdown_connections 2024-04-25T10:20:06.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.479+0000 7efd56200640 1 -- 172.21.15.165:0/2245130586 wait complete. 2024-04-25T10:20:06.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.481+0000 7efd56200640 1 Processor -- start 2024-04-25T10:20:06.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.481+0000 7efd56200640 1 -- start start 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.482+0000 7efd56200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 0x7efd500ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.482+0000 7efd56200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd50156f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.482+0000 7efd56200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd501615a0 0x7efd50157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.482+0000 7efd56200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500d1be0 con 0x7efd500d04f0 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.482+0000 7efd56200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd501623b0 con 0x7efd501615a0 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.482+0000 7efd56200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd50161a20 con 0x7efd5015db90 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 0x7efd500ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 0x7efd500ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:41958/0 (socket says 172.21.15.165:41958) 2024-04-25T10:20:06.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 -- 172.21.15.165:0/3302052009 learned_addr learned my addr 172.21.15.165:0/3302052009 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 -- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 msgr2=0x7efd50156f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd50156f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd55800640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd501615a0 0x7efd50157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 -- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd501615a0 msgr2=0x7efd50157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd501615a0 0x7efd50157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd54e00640 1 -- 172.21.15.165:0/3302052009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd5015e9b0 con 0x7efd500d04f0 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.483+0000 7efd55800640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd501615a0 0x7efd50157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:06.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.484+0000 7efd54e00640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 0x7efd500ca870 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7efd4000d800 tx=0x7efd4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.484+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000b6c0 con 0x7efd500d04f0 2024-04-25T10:20:06.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.484+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40019070 con 0x7efd500d04f0 2024-04-25T10:20:06.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.484+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4000ba80 con 0x7efd500d04f0 2024-04-25T10:20:06.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.484+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd500cb0c0 con 0x7efd500d04f0 2024-04-25T10:20:06.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.484+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd500cb680 con 0x7efd500d04f0 2024-04-25T10:20:06.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.485+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd500cb8c0 con 0x7efd500d04f0 2024-04-25T10:20:06.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.486+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efd4000bc20 con 0x7efd500d04f0 2024-04-25T10:20:06.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.488+0000 7efd4d600640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd2c06b4d0 0x7efd2c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.488+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7efd4008a8e0 con 0x7efd500d04f0 2024-04-25T10:20:06.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.488+0000 7efd47e00640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd2c06b4d0 0x7efd2c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.488+0000 7efd47e00640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd2c06b4d0 0x7efd2c06d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efd38034000 tx=0x7efd3802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.492+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efd40059530 con 0x7efd500d04f0 2024-04-25T10:20:06.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.682+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efd50073990 con 0x7efd2c06b4d0 2024-04-25T10:20:06.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.691+0000 7efd4d600640 1 -- 172.21.15.165:0/3302052009 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248840 (secure 0 0 0) 0x7efd50073990 con 0x7efd2c06b4d0 2024-04-25T10:20:06.697 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:06.699 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:06.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd2c06b4d0 msgr2=0x7efd2c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd2c06b4d0 0x7efd2c06d990 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7efd38034000 tx=0x7efd3802c000 comp rx=0 tx=0).stop 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 msgr2=0x7efd500ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 0x7efd500ca870 secure :-1 s=READY pgs=2594 cs=0 l=1 rev1=1 crypto rx=0x7efd4000d800 tx=0x7efd4000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 shutdown_connections 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efd2c06b4d0 0x7efd2c06d990 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efd501615a0 0x7efd50157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efd5015db90 0x7efd50156f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 --2- 172.21.15.165:0/3302052009 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efd500d04f0 0x7efd500ca870 unknown :-1 s=CLOSED pgs=2594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 >> 172.21.15.165:0/3302052009 conn(0x7efd500b52b0 msgr2=0x7efd500d3780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 shutdown_connections 2024-04-25T10:20:06.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.696+0000 7efd56200640 1 -- 172.21.15.165:0/3302052009 wait complete. 2024-04-25T10:20:06.716 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":724,"stamp":"2024-04-25T10:20:06.498880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536284,"kb_used_data":19112,"kb_used_omap":12,"kb_used_meta":517043,"kb_avail":754438436,"statfs":{"total":773094113280,"available":772544958464,"internally_reserved":0,"allocated":19570688,"data_stored":9816696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":529452594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001990"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.345532+0000","last_change":"2024-04-25T10:20:02.345532+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.345532+0000","last_undegraded":"2024-04-25T10:20:02.345532+0000","last_fullsized":"2024-04-25T10:20:02.345532+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.319974+0000","last_change":"2024-04-25T10:20:01.319974+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.319974+0000","last_undegraded":"2024-04-25T10:20:01.319974+0000","last_fullsized":"2024-04-25T10:20:01.319974+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318808+0000","last_change":"2024-04-25T10:20:01.318808+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318808+0000","last_undegraded":"2024-04-25T10:20:01.318808+0000","last_fullsized":"2024-04-25T10:20:01.318808+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.325908+0000","last_change":"2024-04-25T10:20:01.325908+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.325908+0000","last_undegraded":"2024-04-25T10:20:01.325908+0000","last_fullsized":"2024-04-25T10:20:01.325908+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328410+0000","last_change":"2024-04-25T10:20:02.328410+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328410+0000","last_undegraded":"2024-04-25T10:20:02.328410+0000","last_fullsized":"2024-04-25T10:20:02.328410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.342264+0000","last_change":"2024-04-25T10:20:02.342264+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.342264+0000","last_undegraded":"2024-04-25T10:20:02.342264+0000","last_fullsized":"2024-04-25T10:20:02.342264+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328472+0000","last_change":"2024-04-25T10:20:02.328472+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328472+0000","last_undegraded":"2024-04-25T10:20:02.328472+0000","last_fullsized":"2024-04-25T10:20:02.328472+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.366027+0000","last_change":"2024-04-25T10:20:02.363273+0000","last_active":"2024-04-25T10:20:02.366027+0000","last_peered":"2024-04-25T10:20:02.366027+0000","last_clean":"2024-04-25T10:20:02.366027+0000","last_became_active":"2024-04-25T10:20:02.361340+0000","last_became_peered":"2024-04-25T10:20:02.361340+0000","last_unstale":"2024-04-25T10:20:02.366027+0000","last_undegraded":"2024-04-25T10:20:02.366027+0000","last_fullsized":"2024-04-25T10:20:02.366027+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:02.575100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.325436+0000","last_change":"2024-04-25T10:20:01.325436+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.325436+0000","last_undegraded":"2024-04-25T10:20:01.325436+0000","last_fullsized":"2024-04-25T10:20:01.325436+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.365799+0000","last_change":"2024-04-25T10:20:02.362899+0000","last_active":"2024-04-25T10:20:02.365799+0000","last_peered":"2024-04-25T10:20:02.365799+0000","last_clean":"2024-04-25T10:20:02.365799+0000","last_became_active":"2024-04-25T10:20:02.362609+0000","last_became_peered":"2024-04-25T10:20:02.362609+0000","last_unstale":"2024-04-25T10:20:02.365799+0000","last_undegraded":"2024-04-25T10:20:02.365799+0000","last_fullsized":"2024-04-25T10:20:02.365799+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:12.947488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318399+0000","last_change":"2024-04-25T10:20:01.318399+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318399+0000","last_undegraded":"2024-04-25T10:20:01.318399+0000","last_fullsized":"2024-04-25T10:20:01.318399+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328908+0000","last_change":"2024-04-25T10:20:02.328908+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328908+0000","last_undegraded":"2024-04-25T10:20:02.328908+0000","last_fullsized":"2024-04-25T10:20:02.328908+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324780+0000","last_change":"2024-04-25T10:20:01.324780+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324780+0000","last_undegraded":"2024-04-25T10:20:01.324780+0000","last_fullsized":"2024-04-25T10:20:01.324780+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352544+0000","last_change":"2024-04-25T10:20:02.350419+0000","last_active":"2024-04-25T10:20:02.352544+0000","last_peered":"2024-04-25T10:20:02.352544+0000","last_clean":"2024-04-25T10:20:02.352544+0000","last_became_active":"2024-04-25T10:20:02.350117+0000","last_became_peered":"2024-04-25T10:20:02.350117+0000","last_unstale":"2024-04-25T10:20:02.352544+0000","last_undegraded":"2024-04-25T10:20:02.352544+0000","last_fullsized":"2024-04-25T10:20:02.352544+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.826629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360697+0000","last_change":"2024-04-25T10:20:02.358059+0000","last_active":"2024-04-25T10:20:02.360697+0000","last_peered":"2024-04-25T10:20:02.360697+0000","last_clean":"2024-04-25T10:20:02.360697+0000","last_became_active":"2024-04-25T10:20:02.357465+0000","last_became_peered":"2024-04-25T10:20:02.357465+0000","last_unstale":"2024-04-25T10:20:02.360697+0000","last_undegraded":"2024-04-25T10:20:02.360697+0000","last_fullsized":"2024-04-25T10:20:02.360697+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:54.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315510+0000","last_change":"2024-04-25T10:20:01.315510+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315510+0000","last_undegraded":"2024-04-25T10:20:01.315510+0000","last_fullsized":"2024-04-25T10:20:01.315510+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324993+0000","last_change":"2024-04-25T10:20:01.324993+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324993+0000","last_undegraded":"2024-04-25T10:20:01.324993+0000","last_fullsized":"2024-04-25T10:20:01.324993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.321276+0000","last_change":"2024-04-25T10:20:01.321276+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.321276+0000","last_undegraded":"2024-04-25T10:20:01.321276+0000","last_fullsized":"2024-04-25T10:20:01.321276+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.355356+0000","last_change":"2024-04-25T10:20:02.353518+0000","last_active":"2024-04-25T10:20:02.355356+0000","last_peered":"2024-04-25T10:20:02.355356+0000","last_clean":"2024-04-25T10:20:02.355356+0000","last_became_active":"2024-04-25T10:20:02.352415+0000","last_became_peered":"2024-04-25T10:20:02.352415+0000","last_unstale":"2024-04-25T10:20:02.355356+0000","last_undegraded":"2024-04-25T10:20:02.355356+0000","last_fullsized":"2024-04-25T10:20:02.355356+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.644570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352156+0000","last_change":"2024-04-25T10:20:02.349464+0000","last_active":"2024-04-25T10:20:02.352156+0000","last_peered":"2024-04-25T10:20:02.352156+0000","last_clean":"2024-04-25T10:20:02.352156+0000","last_became_active":"2024-04-25T10:20:02.349133+0000","last_became_peered":"2024-04-25T10:20:02.349133+0000","last_unstale":"2024-04-25T10:20:02.352156+0000","last_undegraded":"2024-04-25T10:20:02.352156+0000","last_fullsized":"2024-04-25T10:20:02.352156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:31.263971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.361393+0000","last_change":"2024-04-25T10:20:02.356526+0000","last_active":"2024-04-25T10:20:02.361393+0000","last_peered":"2024-04-25T10:20:02.361393+0000","last_clean":"2024-04-25T10:20:02.361393+0000","last_became_active":"2024-04-25T10:20:02.355119+0000","last_became_peered":"2024-04-25T10:20:02.355119+0000","last_unstale":"2024-04-25T10:20:02.361393+0000","last_undegraded":"2024-04-25T10:20:02.361393+0000","last_fullsized":"2024-04-25T10:20:02.361393+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:44.172554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352518+0000","last_change":"2024-04-25T10:20:02.350034+0000","last_active":"2024-04-25T10:20:02.352518+0000","last_peered":"2024-04-25T10:20:02.352518+0000","last_clean":"2024-04-25T10:20:02.352518+0000","last_became_active":"2024-04-25T10:20:02.349692+0000","last_became_peered":"2024-04-25T10:20:02.349692+0000","last_unstale":"2024-04-25T10:20:02.352518+0000","last_undegraded":"2024-04-25T10:20:02.352518+0000","last_fullsized":"2024-04-25T10:20:02.352518+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:43.241274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315170+0000","last_change":"2024-04-25T10:20:01.315170+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315170+0000","last_undegraded":"2024-04-25T10:20:01.315170+0000","last_fullsized":"2024-04-25T10:20:01.315170+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328373+0000","last_change":"2024-04-25T10:20:02.328373+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328373+0000","last_undegraded":"2024-04-25T10:20:02.328373+0000","last_fullsized":"2024-04-25T10:20:02.328373+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324520+0000","last_change":"2024-04-25T10:20:01.324520+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324520+0000","last_undegraded":"2024-04-25T10:20:01.324520+0000","last_fullsized":"2024-04-25T10:20:01.324520+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.353985+0000","last_change":"2024-04-25T10:20:02.353985+0000","last_active":"2024-04-25T10:20:02.353985+0000","last_peered":"2024-04-25T10:20:02.353985+0000","last_clean":"2024-04-25T10:20:02.353985+0000","last_became_active":"2024-04-25T10:20:02.353555+0000","last_became_peered":"2024-04-25T10:20:02.353555+0000","last_unstale":"2024-04-25T10:20:02.353985+0000","last_undegraded":"2024-04-25T10:20:02.353985+0000","last_fullsized":"2024-04-25T10:20:02.353985+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:02.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317986+0000","last_change":"2024-04-25T10:20:01.317986+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317986+0000","last_undegraded":"2024-04-25T10:20:01.317986+0000","last_fullsized":"2024-04-25T10:20:01.317986+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328773+0000","last_change":"2024-04-25T10:20:02.328773+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328773+0000","last_undegraded":"2024-04-25T10:20:02.328773+0000","last_fullsized":"2024-04-25T10:20:02.328773+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.319285+0000","last_change":"2024-04-25T10:20:01.319285+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.319285+0000","last_undegraded":"2024-04-25T10:20:01.319285+0000","last_fullsized":"2024-04-25T10:20:01.319285+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360903+0000","last_change":"2024-04-25T10:20:02.358637+0000","last_active":"2024-04-25T10:20:02.360903+0000","last_peered":"2024-04-25T10:20:02.360903+0000","last_clean":"2024-04-25T10:20:02.360903+0000","last_became_active":"2024-04-25T10:20:02.355276+0000","last_became_peered":"2024-04-25T10:20:02.355276+0000","last_unstale":"2024-04-25T10:20:02.360903+0000","last_undegraded":"2024-04-25T10:20:02.360903+0000","last_fullsized":"2024-04-25T10:20:02.360903+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:28.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.341785+0000","last_change":"2024-04-25T10:20:02.341785+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.341785+0000","last_undegraded":"2024-04-25T10:20:02.341785+0000","last_fullsized":"2024-04-25T10:20:02.341785+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.343799+0000","last_change":"2024-04-25T10:20:02.343799+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.343799+0000","last_undegraded":"2024-04-25T10:20:02.343799+0000","last_fullsized":"2024-04-25T10:20:02.343799+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315053+0000","last_change":"2024-04-25T10:20:01.315053+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315053+0000","last_undegraded":"2024-04-25T10:20:01.315053+0000","last_fullsized":"2024-04-25T10:20:01.315053+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.380366+0000","last_change":"2024-04-25T10:20:01.332782+0000","last_active":"2024-04-25T10:20:02.380366+0000","last_peered":"2024-04-25T10:20:02.380366+0000","last_clean":"2024-04-25T10:20:02.380366+0000","last_became_active":"2024-04-25T10:20:01.332481+0000","last_became_peered":"2024-04-25T10:20:01.332481+0000","last_unstale":"2024-04-25T10:20:02.380366+0000","last_undegraded":"2024-04-25T10:20:02.380366+0000","last_fullsized":"2024-04-25T10:20:02.380366+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:54.192098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344143+0000","last_change":"2024-04-25T10:20:02.344143+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344143+0000","last_undegraded":"2024-04-25T10:20:02.344143+0000","last_fullsized":"2024-04-25T10:20:02.344143+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345106+0000","last_change":"2024-04-25T10:20:01.335869+0000","last_active":"2024-04-25T10:20:02.345106+0000","last_peered":"2024-04-25T10:20:02.345106+0000","last_clean":"2024-04-25T10:20:02.345106+0000","last_became_active":"2024-04-25T10:20:01.335113+0000","last_became_peered":"2024-04-25T10:20:01.335113+0000","last_unstale":"2024-04-25T10:20:02.345106+0000","last_undegraded":"2024-04-25T10:20:02.345106+0000","last_fullsized":"2024-04-25T10:20:02.345106+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:59.188576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.351161+0000","last_change":"2024-04-25T10:20:02.343477+0000","last_active":"2024-04-25T10:20:02.351161+0000","last_peered":"2024-04-25T10:20:02.351161+0000","last_clean":"2024-04-25T10:20:02.351161+0000","last_became_active":"2024-04-25T10:20:02.342712+0000","last_became_peered":"2024-04-25T10:20:02.342712+0000","last_unstale":"2024-04-25T10:20:02.351161+0000","last_undegraded":"2024-04-25T10:20:02.351161+0000","last_fullsized":"2024-04-25T10:20:02.351161+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:00.592425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337187+0000","last_change":"2024-04-25T10:20:01.334447+0000","last_active":"2024-04-25T10:20:01.337187+0000","last_peered":"2024-04-25T10:20:01.337187+0000","last_clean":"2024-04-25T10:20:01.337187+0000","last_became_active":"2024-04-25T10:20:01.334157+0000","last_became_peered":"2024-04-25T10:20:01.334157+0000","last_unstale":"2024-04-25T10:20:01.337187+0000","last_undegraded":"2024-04-25T10:20:01.337187+0000","last_fullsized":"2024-04-25T10:20:01.337187+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.385948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.329080+0000","last_change":"2024-04-25T10:20:02.329080+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.329080+0000","last_undegraded":"2024-04-25T10:20:02.329080+0000","last_fullsized":"2024-04-25T10:20:02.329080+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.330180+0000","last_change":"2024-04-25T10:20:01.327488+0000","last_active":"2024-04-25T10:20:01.330180+0000","last_peered":"2024-04-25T10:20:01.330180+0000","last_clean":"2024-04-25T10:20:01.330180+0000","last_became_active":"2024-04-25T10:20:01.327214+0000","last_became_peered":"2024-04-25T10:20:01.327214+0000","last_unstale":"2024-04-25T10:20:01.330180+0000","last_undegraded":"2024-04-25T10:20:01.330180+0000","last_fullsized":"2024-04-25T10:20:01.330180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:08.837078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.361541+0000","last_change":"2024-04-25T10:20:02.360025+0000","last_active":"2024-04-25T10:20:02.361541+0000","last_peered":"2024-04-25T10:20:02.361541+0000","last_clean":"2024-04-25T10:20:02.361541+0000","last_became_active":"2024-04-25T10:20:02.359709+0000","last_became_peered":"2024-04-25T10:20:02.359709+0000","last_unstale":"2024-04-25T10:20:02.361541+0000","last_undegraded":"2024-04-25T10:20:02.361541+0000","last_fullsized":"2024-04-25T10:20:02.361541+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:45.508642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.343642+0000","last_change":"2024-04-25T10:20:01.341558+0000","last_active":"2024-04-25T10:20:01.343642+0000","last_peered":"2024-04-25T10:20:01.343642+0000","last_clean":"2024-04-25T10:20:01.343642+0000","last_became_active":"2024-04-25T10:20:01.340467+0000","last_became_peered":"2024-04-25T10:20:01.340467+0000","last_unstale":"2024-04-25T10:20:01.343642+0000","last_undegraded":"2024-04-25T10:20:01.343642+0000","last_fullsized":"2024-04-25T10:20:01.343642+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:28.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352047+0000","last_change":"2024-04-25T10:20:02.345657+0000","last_active":"2024-04-25T10:20:02.352047+0000","last_peered":"2024-04-25T10:20:02.352047+0000","last_clean":"2024-04-25T10:20:02.352047+0000","last_became_active":"2024-04-25T10:20:02.345353+0000","last_became_peered":"2024-04-25T10:20:02.345353+0000","last_unstale":"2024-04-25T10:20:02.352047+0000","last_undegraded":"2024-04-25T10:20:02.352047+0000","last_fullsized":"2024-04-25T10:20:02.352047+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.343333+0000","last_change":"2024-04-25T10:20:01.342761+0000","last_active":"2024-04-25T10:20:02.343333+0000","last_peered":"2024-04-25T10:20:02.343333+0000","last_clean":"2024-04-25T10:20:02.343333+0000","last_became_active":"2024-04-25T10:20:01.342145+0000","last_became_peered":"2024-04-25T10:20:01.342145+0000","last_unstale":"2024-04-25T10:20:02.343333+0000","last_undegraded":"2024-04-25T10:20:02.343333+0000","last_fullsized":"2024-04-25T10:20:02.343333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315024+0000","last_change":"2024-04-25T10:20:01.315024+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315024+0000","last_undegraded":"2024-04-25T10:20:01.315024+0000","last_fullsized":"2024-04-25T10:20:01.315024+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.339952+0000","last_change":"2024-04-25T10:20:01.338974+0000","last_active":"2024-04-25T10:20:01.339952+0000","last_peered":"2024-04-25T10:20:01.339952+0000","last_clean":"2024-04-25T10:20:01.339952+0000","last_became_active":"2024-04-25T10:20:01.338631+0000","last_became_peered":"2024-04-25T10:20:01.338631+0000","last_unstale":"2024-04-25T10:20:01.339952+0000","last_undegraded":"2024-04-25T10:20:01.339952+0000","last_fullsized":"2024-04-25T10:20:01.339952+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:08.857929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360631+0000","last_change":"2024-04-25T10:20:02.357818+0000","last_active":"2024-04-25T10:20:02.360631+0000","last_peered":"2024-04-25T10:20:02.360631+0000","last_clean":"2024-04-25T10:20:02.360631+0000","last_became_active":"2024-04-25T10:20:02.357502+0000","last_became_peered":"2024-04-25T10:20:02.357502+0000","last_unstale":"2024-04-25T10:20:02.360631+0000","last_undegraded":"2024-04-25T10:20:02.360631+0000","last_fullsized":"2024-04-25T10:20:02.360631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:57.523892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.330720+0000","last_change":"2024-04-25T10:20:01.332708+0000","last_active":"2024-04-25T10:20:02.330720+0000","last_peered":"2024-04-25T10:20:02.330720+0000","last_clean":"2024-04-25T10:20:02.330720+0000","last_became_active":"2024-04-25T10:20:01.332230+0000","last_became_peered":"2024-04-25T10:20:01.332230+0000","last_unstale":"2024-04-25T10:20:02.330720+0000","last_undegraded":"2024-04-25T10:20:02.330720+0000","last_fullsized":"2024-04-25T10:20:02.330720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:32.682708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.322584+0000","last_change":"2024-04-25T10:20:01.322584+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.322584+0000","last_undegraded":"2024-04-25T10:20:01.322584+0000","last_fullsized":"2024-04-25T10:20:01.322584+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.346371+0000","last_change":"2024-04-25T10:20:01.345530+0000","last_active":"2024-04-25T10:20:01.346371+0000","last_peered":"2024-04-25T10:20:01.346371+0000","last_clean":"2024-04-25T10:20:01.346371+0000","last_became_active":"2024-04-25T10:20:01.345190+0000","last_became_peered":"2024-04-25T10:20:01.345190+0000","last_unstale":"2024-04-25T10:20:01.346371+0000","last_undegraded":"2024-04-25T10:20:01.346371+0000","last_fullsized":"2024-04-25T10:20:01.346371+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:08.857929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315747+0000","last_change":"2024-04-25T10:20:01.315747+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315747+0000","last_undegraded":"2024-04-25T10:20:01.315747+0000","last_fullsized":"2024-04-25T10:20:01.315747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.331668+0000","last_change":"2024-04-25T10:20:01.329959+0000","last_active":"2024-04-25T10:20:01.331668+0000","last_peered":"2024-04-25T10:20:01.331668+0000","last_clean":"2024-04-25T10:20:01.331668+0000","last_became_active":"2024-04-25T10:20:01.329609+0000","last_became_peered":"2024-04-25T10:20:01.329609+0000","last_unstale":"2024-04-25T10:20:01.331668+0000","last_undegraded":"2024-04-25T10:20:01.331668+0000","last_fullsized":"2024-04-25T10:20:01.331668+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:45.592237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.329575+0000","last_change":"2024-04-25T10:20:02.329575+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.329575+0000","last_undegraded":"2024-04-25T10:20:02.329575+0000","last_fullsized":"2024-04-25T10:20:02.329575+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.344171+0000","last_change":"2024-04-25T10:20:01.343264+0000","last_active":"2024-04-25T10:20:01.344171+0000","last_peered":"2024-04-25T10:20:01.344171+0000","last_clean":"2024-04-25T10:20:01.344171+0000","last_became_active":"2024-04-25T10:20:01.342715+0000","last_became_peered":"2024-04-25T10:20:01.342715+0000","last_unstale":"2024-04-25T10:20:01.344171+0000","last_undegraded":"2024-04-25T10:20:01.344171+0000","last_fullsized":"2024-04-25T10:20:01.344171+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:41.996295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318832+0000","last_change":"2024-04-25T10:20:01.318832+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318832+0000","last_undegraded":"2024-04-25T10:20:01.318832+0000","last_fullsized":"2024-04-25T10:20:01.318832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.338100+0000","last_change":"2024-04-25T10:20:01.337329+0000","last_active":"2024-04-25T10:20:01.338100+0000","last_peered":"2024-04-25T10:20:01.338100+0000","last_clean":"2024-04-25T10:20:01.338100+0000","last_became_active":"2024-04-25T10:20:01.336415+0000","last_became_peered":"2024-04-25T10:20:01.336415+0000","last_unstale":"2024-04-25T10:20:01.338100+0000","last_undegraded":"2024-04-25T10:20:01.338100+0000","last_fullsized":"2024-04-25T10:20:01.338100+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.782792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317486+0000","last_change":"2024-04-25T10:20:01.317486+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317486+0000","last_undegraded":"2024-04-25T10:20:01.317486+0000","last_fullsized":"2024-04-25T10:20:01.317486+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.331224+0000","last_change":"2024-04-25T10:20:01.330029+0000","last_active":"2024-04-25T10:20:01.331224+0000","last_peered":"2024-04-25T10:20:01.331224+0000","last_clean":"2024-04-25T10:20:01.331224+0000","last_became_active":"2024-04-25T10:20:01.328434+0000","last_became_peered":"2024-04-25T10:20:01.328434+0000","last_unstale":"2024-04-25T10:20:01.331224+0000","last_undegraded":"2024-04-25T10:20:01.331224+0000","last_fullsized":"2024-04-25T10:20:01.331224+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:41.996295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318997+0000","last_change":"2024-04-25T10:20:01.318997+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318997+0000","last_undegraded":"2024-04-25T10:20:01.318997+0000","last_fullsized":"2024-04-25T10:20:01.318997+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.336749+0000","last_change":"2024-04-25T10:20:01.332999+0000","last_active":"2024-04-25T10:20:01.336749+0000","last_peered":"2024-04-25T10:20:01.336749+0000","last_clean":"2024-04-25T10:20:01.336749+0000","last_became_active":"2024-04-25T10:20:01.332513+0000","last_became_peered":"2024-04-25T10:20:01.332513+0000","last_unstale":"2024-04-25T10:20:01.336749+0000","last_undegraded":"2024-04-25T10:20:01.336749+0000","last_fullsized":"2024-04-25T10:20:01.336749+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:28.158755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.332203+0000","last_change":"2024-04-25T10:20:01.332203+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.332203+0000","last_undegraded":"2024-04-25T10:20:01.332203+0000","last_fullsized":"2024-04-25T10:20:01.332203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.327921+0000","last_change":"2024-04-25T10:20:01.338593+0000","last_active":"2024-04-25T10:20:02.327921+0000","last_peered":"2024-04-25T10:20:02.327921+0000","last_clean":"2024-04-25T10:20:02.327921+0000","last_became_active":"2024-04-25T10:20:01.338279+0000","last_became_peered":"2024-04-25T10:20:01.338279+0000","last_unstale":"2024-04-25T10:20:02.327921+0000","last_undegraded":"2024-04-25T10:20:02.327921+0000","last_fullsized":"2024-04-25T10:20:02.327921+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.211682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.366048+0000","last_change":"2024-04-25T10:20:02.363884+0000","last_active":"2024-04-25T10:20:02.366048+0000","last_peered":"2024-04-25T10:20:02.366048+0000","last_clean":"2024-04-25T10:20:02.366048+0000","last_became_active":"2024-04-25T10:20:02.363593+0000","last_became_peered":"2024-04-25T10:20:02.363593+0000","last_unstale":"2024-04-25T10:20:02.366048+0000","last_undegraded":"2024-04-25T10:20:02.366048+0000","last_fullsized":"2024-04-25T10:20:02.366048+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:28.866478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345829+0000","last_change":"2024-04-25T10:20:01.341364+0000","last_active":"2024-04-25T10:20:02.345829+0000","last_peered":"2024-04-25T10:20:02.345829+0000","last_clean":"2024-04-25T10:20:02.345829+0000","last_became_active":"2024-04-25T10:20:01.340654+0000","last_became_peered":"2024-04-25T10:20:01.340654+0000","last_unstale":"2024-04-25T10:20:02.345829+0000","last_undegraded":"2024-04-25T10:20:02.345829+0000","last_fullsized":"2024-04-25T10:20:02.345829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:34.211448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344027+0000","last_change":"2024-04-25T10:20:02.344027+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344027+0000","last_undegraded":"2024-04-25T10:20:02.344027+0000","last_fullsized":"2024-04-25T10:20:02.344027+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345295+0000","last_change":"2024-04-25T10:20:01.343272+0000","last_active":"2024-04-25T10:20:02.345295+0000","last_peered":"2024-04-25T10:20:02.345295+0000","last_clean":"2024-04-25T10:20:02.345295+0000","last_became_active":"2024-04-25T10:20:01.342389+0000","last_became_peered":"2024-04-25T10:20:01.342389+0000","last_unstale":"2024-04-25T10:20:02.345295+0000","last_undegraded":"2024-04-25T10:20:02.345295+0000","last_fullsized":"2024-04-25T10:20:02.345295+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:32.480221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.343886+0000","last_change":"2024-04-25T10:20:02.343886+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.343886+0000","last_undegraded":"2024-04-25T10:20:02.343886+0000","last_fullsized":"2024-04-25T10:20:02.343886+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.341155+0000","last_change":"2024-04-25T10:20:01.340613+0000","last_active":"2024-04-25T10:20:01.341155+0000","last_peered":"2024-04-25T10:20:01.341155+0000","last_clean":"2024-04-25T10:20:01.341155+0000","last_became_active":"2024-04-25T10:20:01.340250+0000","last_became_peered":"2024-04-25T10:20:01.340250+0000","last_unstale":"2024-04-25T10:20:01.341155+0000","last_undegraded":"2024-04-25T10:20:01.341155+0000","last_fullsized":"2024-04-25T10:20:01.341155+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:17.164219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.351303+0000","last_change":"2024-04-25T10:20:02.346899+0000","last_active":"2024-04-25T10:20:02.351303+0000","last_peered":"2024-04-25T10:20:02.351303+0000","last_clean":"2024-04-25T10:20:02.351303+0000","last_became_active":"2024-04-25T10:20:02.346568+0000","last_became_peered":"2024-04-25T10:20:02.346568+0000","last_unstale":"2024-04-25T10:20:02.351303+0000","last_undegraded":"2024-04-25T10:20:02.351303+0000","last_fullsized":"2024-04-25T10:20:02.351303+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.329544+0000","last_change":"2024-04-25T10:20:01.343143+0000","last_active":"2024-04-25T10:20:02.329544+0000","last_peered":"2024-04-25T10:20:02.329544+0000","last_clean":"2024-04-25T10:20:02.329544+0000","last_became_active":"2024-04-25T10:20:01.342912+0000","last_became_peered":"2024-04-25T10:20:01.342912+0000","last_unstale":"2024-04-25T10:20:02.329544+0000","last_undegraded":"2024-04-25T10:20:02.329544+0000","last_fullsized":"2024-04-25T10:20:02.329544+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.956577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318305+0000","last_change":"2024-04-25T10:20:01.318305+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318305+0000","last_undegraded":"2024-04-25T10:20:01.318305+0000","last_fullsized":"2024-04-25T10:20:01.318305+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.330106+0000","last_change":"2024-04-25T10:20:01.333143+0000","last_active":"2024-04-25T10:20:02.330106+0000","last_peered":"2024-04-25T10:20:02.330106+0000","last_clean":"2024-04-25T10:20:02.330106+0000","last_became_active":"2024-04-25T10:20:01.332865+0000","last_became_peered":"2024-04-25T10:20:01.332865+0000","last_unstale":"2024-04-25T10:20:02.330106+0000","last_undegraded":"2024-04-25T10:20:02.330106+0000","last_fullsized":"2024-04-25T10:20:02.330106+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:29.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.329524+0000","last_change":"2024-04-25T10:20:01.329524+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.329524+0000","last_undegraded":"2024-04-25T10:20:01.329524+0000","last_fullsized":"2024-04-25T10:20:01.329524+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.336032+0000","last_change":"2024-04-25T10:20:01.332184+0000","last_active":"2024-04-25T10:20:01.336032+0000","last_peered":"2024-04-25T10:20:01.336032+0000","last_clean":"2024-04-25T10:20:01.336032+0000","last_became_active":"2024-04-25T10:20:01.331558+0000","last_became_peered":"2024-04-25T10:20:01.331558+0000","last_unstale":"2024-04-25T10:20:01.336032+0000","last_undegraded":"2024-04-25T10:20:01.336032+0000","last_fullsized":"2024-04-25T10:20:01.336032+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:19.209857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.316990+0000","last_change":"2024-04-25T10:20:01.316990+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.316990+0000","last_undegraded":"2024-04-25T10:20:01.316990+0000","last_fullsized":"2024-04-25T10:20:01.316990+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.334000+0000","last_change":"2024-04-25T10:20:01.333236+0000","last_active":"2024-04-25T10:20:02.334000+0000","last_peered":"2024-04-25T10:20:02.334000+0000","last_clean":"2024-04-25T10:20:02.334000+0000","last_became_active":"2024-04-25T10:20:01.332681+0000","last_became_peered":"2024-04-25T10:20:01.332681+0000","last_unstale":"2024-04-25T10:20:02.334000+0000","last_undegraded":"2024-04-25T10:20:02.334000+0000","last_fullsized":"2024-04-25T10:20:02.334000+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:26.830908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.348938+0000","last_change":"2024-04-25T10:20:02.341693+0000","last_active":"2024-04-25T10:20:02.348938+0000","last_peered":"2024-04-25T10:20:02.348938+0000","last_clean":"2024-04-25T10:20:02.348938+0000","last_became_active":"2024-04-25T10:20:02.341396+0000","last_became_peered":"2024-04-25T10:20:02.341396+0000","last_unstale":"2024-04-25T10:20:02.348938+0000","last_undegraded":"2024-04-25T10:20:02.348938+0000","last_fullsized":"2024-04-25T10:20:02.348938+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:32.680379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345393+0000","last_change":"2024-04-25T10:20:01.334912+0000","last_active":"2024-04-25T10:20:02.345393+0000","last_peered":"2024-04-25T10:20:02.345393+0000","last_clean":"2024-04-25T10:20:02.345393+0000","last_became_active":"2024-04-25T10:20:01.334608+0000","last_became_peered":"2024-04-25T10:20:01.334608+0000","last_unstale":"2024-04-25T10:20:02.345393+0000","last_undegraded":"2024-04-25T10:20:02.345393+0000","last_fullsized":"2024-04-25T10:20:02.345393+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:19.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.363405+0000","last_change":"2024-04-25T10:20:02.356760+0000","last_active":"2024-04-25T10:20:02.363405+0000","last_peered":"2024-04-25T10:20:02.363405+0000","last_clean":"2024-04-25T10:20:02.363405+0000","last_became_active":"2024-04-25T10:20:02.356400+0000","last_became_peered":"2024-04-25T10:20:02.356400+0000","last_unstale":"2024-04-25T10:20:02.363405+0000","last_undegraded":"2024-04-25T10:20:02.363405+0000","last_fullsized":"2024-04-25T10:20:02.363405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:30.172815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337601+0000","last_change":"2024-04-25T10:20:01.335981+0000","last_active":"2024-04-25T10:20:01.337601+0000","last_peered":"2024-04-25T10:20:01.337601+0000","last_clean":"2024-04-25T10:20:01.337601+0000","last_became_active":"2024-04-25T10:20:01.335703+0000","last_became_peered":"2024-04-25T10:20:01.335703+0000","last_unstale":"2024-04-25T10:20:01.337601+0000","last_undegraded":"2024-04-25T10:20:01.337601+0000","last_fullsized":"2024-04-25T10:20:01.337601+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:29.441671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317980+0000","last_change":"2024-04-25T10:20:01.317980+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317980+0000","last_undegraded":"2024-04-25T10:20:01.317980+0000","last_fullsized":"2024-04-25T10:20:01.317980+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.336304+0000","last_change":"2024-04-25T10:20:01.332051+0000","last_active":"2024-04-25T10:20:02.336304+0000","last_peered":"2024-04-25T10:20:02.336304+0000","last_clean":"2024-04-25T10:20:02.336304+0000","last_became_active":"2024-04-25T10:20:01.331777+0000","last_became_peered":"2024-04-25T10:20:01.331777+0000","last_unstale":"2024-04-25T10:20:02.336304+0000","last_undegraded":"2024-04-25T10:20:02.336304+0000","last_fullsized":"2024-04-25T10:20:02.336304+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.465188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315284+0000","last_change":"2024-04-25T10:20:01.315284+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315284+0000","last_undegraded":"2024-04-25T10:20:01.315284+0000","last_fullsized":"2024-04-25T10:20:01.315284+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.334793+0000","last_change":"2024-04-25T10:20:01.323209+0000","last_active":"2024-04-25T10:20:02.334793+0000","last_peered":"2024-04-25T10:20:02.334793+0000","last_clean":"2024-04-25T10:20:02.334793+0000","last_became_active":"2024-04-25T10:20:01.319553+0000","last_became_peered":"2024-04-25T10:20:01.319553+0000","last_unstale":"2024-04-25T10:20:02.334793+0000","last_undegraded":"2024-04-25T10:20:02.334793+0000","last_fullsized":"2024-04-25T10:20:02.334793+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:40.645466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344286+0000","last_change":"2024-04-25T10:20:02.344286+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344286+0000","last_undegraded":"2024-04-25T10:20:02.344286+0000","last_fullsized":"2024-04-25T10:20:02.344286+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.346509+0000","last_change":"2024-04-25T10:20:01.336445+0000","last_active":"2024-04-25T10:20:02.346509+0000","last_peered":"2024-04-25T10:20:02.346509+0000","last_clean":"2024-04-25T10:20:02.346509+0000","last_became_active":"2024-04-25T10:20:01.336136+0000","last_became_peered":"2024-04-25T10:20:01.336136+0000","last_unstale":"2024-04-25T10:20:02.346509+0000","last_undegraded":"2024-04-25T10:20:02.346509+0000","last_fullsized":"2024-04-25T10:20:02.346509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.879308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.365379+0000","last_change":"2024-04-25T10:20:02.361100+0000","last_active":"2024-04-25T10:20:02.365379+0000","last_peered":"2024-04-25T10:20:02.365379+0000","last_clean":"2024-04-25T10:20:02.365379+0000","last_became_active":"2024-04-25T10:20:02.360829+0000","last_became_peered":"2024-04-25T10:20:02.360829+0000","last_unstale":"2024-04-25T10:20:02.365379+0000","last_undegraded":"2024-04-25T10:20:02.365379+0000","last_fullsized":"2024-04-25T10:20:02.365379+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:36.320764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.335687+0000","last_change":"2024-04-25T10:20:01.329676+0000","last_active":"2024-04-25T10:20:02.335687+0000","last_peered":"2024-04-25T10:20:02.335687+0000","last_clean":"2024-04-25T10:20:02.335687+0000","last_became_active":"2024-04-25T10:20:01.329366+0000","last_became_peered":"2024-04-25T10:20:01.329366+0000","last_unstale":"2024-04-25T10:20:02.335687+0000","last_undegraded":"2024-04-25T10:20:02.335687+0000","last_fullsized":"2024-04-25T10:20:02.335687+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:57.834078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.329095+0000","last_change":"2024-04-25T10:20:01.329095+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.329095+0000","last_undegraded":"2024-04-25T10:20:01.329095+0000","last_fullsized":"2024-04-25T10:20:01.329095+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337190+0000","last_change":"2024-04-25T10:20:01.334388+0000","last_active":"2024-04-25T10:20:01.337190+0000","last_peered":"2024-04-25T10:20:01.337190+0000","last_clean":"2024-04-25T10:20:01.337190+0000","last_became_active":"2024-04-25T10:20:01.333900+0000","last_became_peered":"2024-04-25T10:20:01.333900+0000","last_unstale":"2024-04-25T10:20:01.337190+0000","last_undegraded":"2024-04-25T10:20:01.337190+0000","last_fullsized":"2024-04-25T10:20:01.337190+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.991496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.327831+0000","last_change":"2024-04-25T10:20:01.327831+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.327831+0000","last_undegraded":"2024-04-25T10:20:01.327831+0000","last_fullsized":"2024-04-25T10:20:01.327831+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.346482+0000","last_change":"2024-04-25T10:20:01.335371+0000","last_active":"2024-04-25T10:20:02.346482+0000","last_peered":"2024-04-25T10:20:02.346482+0000","last_clean":"2024-04-25T10:20:02.346482+0000","last_became_active":"2024-04-25T10:20:01.335092+0000","last_became_peered":"2024-04-25T10:20:01.335092+0000","last_unstale":"2024-04-25T10:20:02.346482+0000","last_undegraded":"2024-04-25T10:20:02.346482+0000","last_fullsized":"2024-04-25T10:20:02.346482+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:44.129836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318525+0000","last_change":"2024-04-25T10:20:01.318525+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318525+0000","last_undegraded":"2024-04-25T10:20:01.318525+0000","last_fullsized":"2024-04-25T10:20:01.318525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.328352+0000","last_change":"2024-04-25T10:20:01.339862+0000","last_active":"2024-04-25T10:20:02.328352+0000","last_peered":"2024-04-25T10:20:02.328352+0000","last_clean":"2024-04-25T10:20:02.328352+0000","last_became_active":"2024-04-25T10:20:01.339331+0000","last_became_peered":"2024-04-25T10:20:01.339331+0000","last_unstale":"2024-04-25T10:20:02.328352+0000","last_undegraded":"2024-04-25T10:20:02.328352+0000","last_fullsized":"2024-04-25T10:20:02.328352+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:54.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344495+0000","last_change":"2024-04-25T10:20:02.344495+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344495+0000","last_undegraded":"2024-04-25T10:20:02.344495+0000","last_fullsized":"2024-04-25T10:20:02.344495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.330229+0000","last_change":"2024-04-25T10:20:01.329000+0000","last_active":"2024-04-25T10:20:01.330229+0000","last_peered":"2024-04-25T10:20:01.330229+0000","last_clean":"2024-04-25T10:20:01.330229+0000","last_became_active":"2024-04-25T10:20:01.328744+0000","last_became_peered":"2024-04-25T10:20:01.328744+0000","last_unstale":"2024-04-25T10:20:01.330229+0000","last_undegraded":"2024-04-25T10:20:01.330229+0000","last_fullsized":"2024-04-25T10:20:01.330229+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:31.060432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":480,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.344004+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:02.344004+0000","last_peered":"2024-04-25T10:20:02.344004+0000","last_clean":"2024-04-25T10:20:02.344004+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:02.344004+0000","last_undegraded":"2024-04-25T10:20:02.344004+0000","last_fullsized":"2024-04-25T10:20:02.344004+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575070,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60376,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311464,"statfs":{"total":96636764160,"available":96574939136,"internally_reserved":0,"allocated":2891776,"data_stored":1669863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59796,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312044,"statfs":{"total":96636764160,"available":96575533056,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87508,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284332,"statfs":{"total":96636764160,"available":96547155968,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575070,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87504,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284336,"statfs":{"total":96636764160,"available":96547160064,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575070,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65880,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305960,"statfs":{"total":96636764160,"available":96569303040,"internally_reserved":0,"allocated":2891776,"data_stored":1669863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:06.720 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.956+0000 7f9117800640 1 Processor -- start 2024-04-25T10:20:06.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.956+0000 7f9117800640 1 -- start start 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9117800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9110154ee0 0x7f91100cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9117800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 0x7f91100cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9117800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9110071480 con 0x7f91100ce3c0 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9117800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91100ce830 con 0x7f9110154ee0 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9117800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9110072090 con 0x7f91100cfcf0 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9115800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f91100ce3c0 0x7f911015e570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49174/0 (socket says 172.21.15.165:49174) 2024-04-25T10:20:06.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.958+0000 7f9115800640 1 -- 172.21.15.165:0/591533728 learned_addr learned my addr 172.21.15.165:0/591533728 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f9114e00640 1 --2- 172.21.15.165:0/591533728 >> v2:172.21.15.172:3300/0 conn(0x7f9110154ee0 0x7f91100cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910fe00640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 0x7f91100cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1714765009 0 0) 0x7f9110071480 con 0x7f91100ce3c0 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90f4003520 con 0x7f91100ce3c0 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2534934524 0 0) 0x7f90f4003520 con 0x7f91100ce3c0 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 msgr2=0x7f91100cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 0x7f91100cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 >> v2:172.21.15.172:3300/0 conn(0x7f9110154ee0 msgr2=0x7f91100cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 --2- 172.21.15.165:0/591533728 >> v2:172.21.15.172:3300/0 conn(0x7f9110154ee0 0x7f91100cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110071760 con 0x7f91100ce3c0 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9108001670 con 0x7f91100ce3c0 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 >> v1:172.21.15.165:6789/0 conn(0x7f91100ce3c0 legacy=0x7f911015e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 0x7f90f40039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f4003f30 0x7f90f4004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f90f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91100ce830 con 0x7f90f4004d40 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f400dbd0 con 0x7f90f4003f30 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.959+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f400dd50 con 0x7f91100cfcf0 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9115800640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f90f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f910fe00640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f4003f30 0x7f90f4004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9114e00640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 0x7f90f40039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9115800640 1 -- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 msgr2=0x7f90f40039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9115800640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91100cfcf0 0x7f90f40039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9115800640 1 -- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f4003f30 msgr2=0x7f90f4004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:06.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9115800640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f90f4003f30 0x7f90f4004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.960+0000 7f9115800640 1 -- 172.21.15.165:0/591533728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4003520 con 0x7f90f4004d40 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.961+0000 7f9115800640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f90f40051c0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f910800c290 tx=0x7f90f4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ce652ae96eec96 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.961+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108007d50 con 0x7f90f4004d40 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.961+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910802e070 con 0x7f90f4004d40 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.961+0000 7f910f400640 1 -- 172.21.15.165:0/591533728 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9108005d70 con 0x7f90f4004d40 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9117800640 1 -- 172.21.15.165:0/591533728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 msgr2=0x7f90f40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9117800640 1 --2- 172.21.15.165:0/591533728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f90f40051c0 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7f910800c290 tx=0x7f90f4005830 comp rx=0 tx=0).stop 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9115800640 1 -- 172.21.15.165:0/591533728 reap_dead start 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9117800640 1 -- 172.21.15.165:0/591533728 shutdown_connections 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9117800640 1 -- 172.21.15.165:0/591533728 >> 172.21.15.165:0/591533728 conn(0x7f911006abe0 msgr2=0x7f911006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9117800640 1 -- 172.21.15.165:0/591533728 shutdown_connections 2024-04-25T10:20:06.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.962+0000 7f9117800640 1 -- 172.21.15.165:0/591533728 wait complete. 2024-04-25T10:20:06.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.964+0000 7f9117800640 1 Processor -- start 2024-04-25T10:20:06.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.964+0000 7f9117800640 1 -- start start 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9117800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f91101f3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9117800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f911014d820 0x7f9110155dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9117800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110156300 0x7f911015c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9117800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110071230 con 0x7f90f4004d40 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9117800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9110071d20 con 0x7f911014d820 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9117800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91101552c0 con 0x7f9110156300 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9114e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f91101f3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9114e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f91101f3800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47764/0 (socket says 172.21.15.165:47764) 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.965+0000 7f9114e00640 1 -- 172.21.15.165:0/1338866950 learned_addr learned my addr 172.21.15.165:0/1338866950 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f910fe00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f911014d820 0x7f9110155dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9115800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110156300 0x7f911015c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9114e00640 1 -- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110156300 msgr2=0x7f911015c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9114e00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110156300 0x7f911015c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9114e00640 1 -- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f911014d820 msgr2=0x7f9110155dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9114e00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f911014d820 0x7f9110155dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9114e00640 1 -- 172.21.15.165:0/1338866950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911014e2e0 con 0x7f90f4004d40 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9115800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110156300 0x7f911015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9114e00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f91101f3800 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f9100007f90 tx=0x7f9100011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910001a020 con 0x7f90f4004d40 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9100011e90 con 0x7f90f4004d40 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91000168c0 con 0x7f90f4004d40 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.966+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9110156930 con 0x7f90f4004d40 2024-04-25T10:20:06.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.967+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91101f3000 con 0x7f90f4004d40 2024-04-25T10:20:06.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.968+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9110073990 con 0x7f90f4004d40 2024-04-25T10:20:06.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.968+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9100005d10 con 0x7f90f4004d40 2024-04-25T10:20:06.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.969+0000 7f910cc00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:06.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.969+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f910008b300 con 0x7f90f4004d40 2024-04-25T10:20:06.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.970+0000 7f910fe00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:06.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.970+0000 7f910fe00640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f911014e900 tx=0x7f90fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:06.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:06.973+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9100024030 con 0x7f90f4004d40 2024-04-25T10:20:07.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.164+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f911006b3b0 con 0x7f90f006b4d0 2024-04-25T10:20:07.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.175+0000 7f910cc00640 1 -- 172.21.15.165:0/1338866950 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248840 (secure 0 0 0) 0x7f911006b3b0 con 0x7f90f006b4d0 2024-04-25T10:20:07.180 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:07.183 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 msgr2=0x7f90f006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f911014e900 tx=0x7f90fc008040 comp rx=0 tx=0).stop 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 msgr2=0x7f91101f3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f91101f3800 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f9100007f90 tx=0x7f9100011660 comp rx=0 tx=0).stop 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 shutdown_connections 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f90f006b4d0 0x7f90f006d990 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9110156300 0x7f911015c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f911014d820 0x7f9110155dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 --2- 172.21.15.165:0/1338866950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f90f4004d40 0x7f91101f3800 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 >> 172.21.15.165:0/1338866950 conn(0x7f911006abe0 msgr2=0x7f91100b0190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 shutdown_connections 2024-04-25T10:20:07.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.182+0000 7f9117800640 1 -- 172.21.15.165:0/1338866950 wait complete. 2024-04-25T10:20:07.204 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":724,"stamp":"2024-04-25T10:20:06.498880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536284,"kb_used_data":19112,"kb_used_omap":12,"kb_used_meta":517043,"kb_avail":754438436,"statfs":{"total":773094113280,"available":772544958464,"internally_reserved":0,"allocated":19570688,"data_stored":9816696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":529452594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001990"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.345532+0000","last_change":"2024-04-25T10:20:02.345532+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.345532+0000","last_undegraded":"2024-04-25T10:20:02.345532+0000","last_fullsized":"2024-04-25T10:20:02.345532+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.319974+0000","last_change":"2024-04-25T10:20:01.319974+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.319974+0000","last_undegraded":"2024-04-25T10:20:01.319974+0000","last_fullsized":"2024-04-25T10:20:01.319974+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318808+0000","last_change":"2024-04-25T10:20:01.318808+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318808+0000","last_undegraded":"2024-04-25T10:20:01.318808+0000","last_fullsized":"2024-04-25T10:20:01.318808+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.325908+0000","last_change":"2024-04-25T10:20:01.325908+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.325908+0000","last_undegraded":"2024-04-25T10:20:01.325908+0000","last_fullsized":"2024-04-25T10:20:01.325908+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328410+0000","last_change":"2024-04-25T10:20:02.328410+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328410+0000","last_undegraded":"2024-04-25T10:20:02.328410+0000","last_fullsized":"2024-04-25T10:20:02.328410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.342264+0000","last_change":"2024-04-25T10:20:02.342264+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.342264+0000","last_undegraded":"2024-04-25T10:20:02.342264+0000","last_fullsized":"2024-04-25T10:20:02.342264+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328472+0000","last_change":"2024-04-25T10:20:02.328472+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328472+0000","last_undegraded":"2024-04-25T10:20:02.328472+0000","last_fullsized":"2024-04-25T10:20:02.328472+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.366027+0000","last_change":"2024-04-25T10:20:02.363273+0000","last_active":"2024-04-25T10:20:02.366027+0000","last_peered":"2024-04-25T10:20:02.366027+0000","last_clean":"2024-04-25T10:20:02.366027+0000","last_became_active":"2024-04-25T10:20:02.361340+0000","last_became_peered":"2024-04-25T10:20:02.361340+0000","last_unstale":"2024-04-25T10:20:02.366027+0000","last_undegraded":"2024-04-25T10:20:02.366027+0000","last_fullsized":"2024-04-25T10:20:02.366027+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:02.575100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.325436+0000","last_change":"2024-04-25T10:20:01.325436+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.325436+0000","last_undegraded":"2024-04-25T10:20:01.325436+0000","last_fullsized":"2024-04-25T10:20:01.325436+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.365799+0000","last_change":"2024-04-25T10:20:02.362899+0000","last_active":"2024-04-25T10:20:02.365799+0000","last_peered":"2024-04-25T10:20:02.365799+0000","last_clean":"2024-04-25T10:20:02.365799+0000","last_became_active":"2024-04-25T10:20:02.362609+0000","last_became_peered":"2024-04-25T10:20:02.362609+0000","last_unstale":"2024-04-25T10:20:02.365799+0000","last_undegraded":"2024-04-25T10:20:02.365799+0000","last_fullsized":"2024-04-25T10:20:02.365799+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:12.947488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318399+0000","last_change":"2024-04-25T10:20:01.318399+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318399+0000","last_undegraded":"2024-04-25T10:20:01.318399+0000","last_fullsized":"2024-04-25T10:20:01.318399+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328908+0000","last_change":"2024-04-25T10:20:02.328908+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328908+0000","last_undegraded":"2024-04-25T10:20:02.328908+0000","last_fullsized":"2024-04-25T10:20:02.328908+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324780+0000","last_change":"2024-04-25T10:20:01.324780+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324780+0000","last_undegraded":"2024-04-25T10:20:01.324780+0000","last_fullsized":"2024-04-25T10:20:01.324780+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352544+0000","last_change":"2024-04-25T10:20:02.350419+0000","last_active":"2024-04-25T10:20:02.352544+0000","last_peered":"2024-04-25T10:20:02.352544+0000","last_clean":"2024-04-25T10:20:02.352544+0000","last_became_active":"2024-04-25T10:20:02.350117+0000","last_became_peered":"2024-04-25T10:20:02.350117+0000","last_unstale":"2024-04-25T10:20:02.352544+0000","last_undegraded":"2024-04-25T10:20:02.352544+0000","last_fullsized":"2024-04-25T10:20:02.352544+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.826629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360697+0000","last_change":"2024-04-25T10:20:02.358059+0000","last_active":"2024-04-25T10:20:02.360697+0000","last_peered":"2024-04-25T10:20:02.360697+0000","last_clean":"2024-04-25T10:20:02.360697+0000","last_became_active":"2024-04-25T10:20:02.357465+0000","last_became_peered":"2024-04-25T10:20:02.357465+0000","last_unstale":"2024-04-25T10:20:02.360697+0000","last_undegraded":"2024-04-25T10:20:02.360697+0000","last_fullsized":"2024-04-25T10:20:02.360697+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:54.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315510+0000","last_change":"2024-04-25T10:20:01.315510+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315510+0000","last_undegraded":"2024-04-25T10:20:01.315510+0000","last_fullsized":"2024-04-25T10:20:01.315510+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324993+0000","last_change":"2024-04-25T10:20:01.324993+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324993+0000","last_undegraded":"2024-04-25T10:20:01.324993+0000","last_fullsized":"2024-04-25T10:20:01.324993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.321276+0000","last_change":"2024-04-25T10:20:01.321276+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.321276+0000","last_undegraded":"2024-04-25T10:20:01.321276+0000","last_fullsized":"2024-04-25T10:20:01.321276+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.355356+0000","last_change":"2024-04-25T10:20:02.353518+0000","last_active":"2024-04-25T10:20:02.355356+0000","last_peered":"2024-04-25T10:20:02.355356+0000","last_clean":"2024-04-25T10:20:02.355356+0000","last_became_active":"2024-04-25T10:20:02.352415+0000","last_became_peered":"2024-04-25T10:20:02.352415+0000","last_unstale":"2024-04-25T10:20:02.355356+0000","last_undegraded":"2024-04-25T10:20:02.355356+0000","last_fullsized":"2024-04-25T10:20:02.355356+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.644570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352156+0000","last_change":"2024-04-25T10:20:02.349464+0000","last_active":"2024-04-25T10:20:02.352156+0000","last_peered":"2024-04-25T10:20:02.352156+0000","last_clean":"2024-04-25T10:20:02.352156+0000","last_became_active":"2024-04-25T10:20:02.349133+0000","last_became_peered":"2024-04-25T10:20:02.349133+0000","last_unstale":"2024-04-25T10:20:02.352156+0000","last_undegraded":"2024-04-25T10:20:02.352156+0000","last_fullsized":"2024-04-25T10:20:02.352156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:31.263971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.361393+0000","last_change":"2024-04-25T10:20:02.356526+0000","last_active":"2024-04-25T10:20:02.361393+0000","last_peered":"2024-04-25T10:20:02.361393+0000","last_clean":"2024-04-25T10:20:02.361393+0000","last_became_active":"2024-04-25T10:20:02.355119+0000","last_became_peered":"2024-04-25T10:20:02.355119+0000","last_unstale":"2024-04-25T10:20:02.361393+0000","last_undegraded":"2024-04-25T10:20:02.361393+0000","last_fullsized":"2024-04-25T10:20:02.361393+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:44.172554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352518+0000","last_change":"2024-04-25T10:20:02.350034+0000","last_active":"2024-04-25T10:20:02.352518+0000","last_peered":"2024-04-25T10:20:02.352518+0000","last_clean":"2024-04-25T10:20:02.352518+0000","last_became_active":"2024-04-25T10:20:02.349692+0000","last_became_peered":"2024-04-25T10:20:02.349692+0000","last_unstale":"2024-04-25T10:20:02.352518+0000","last_undegraded":"2024-04-25T10:20:02.352518+0000","last_fullsized":"2024-04-25T10:20:02.352518+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:43.241274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315170+0000","last_change":"2024-04-25T10:20:01.315170+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315170+0000","last_undegraded":"2024-04-25T10:20:01.315170+0000","last_fullsized":"2024-04-25T10:20:01.315170+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328373+0000","last_change":"2024-04-25T10:20:02.328373+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328373+0000","last_undegraded":"2024-04-25T10:20:02.328373+0000","last_fullsized":"2024-04-25T10:20:02.328373+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324520+0000","last_change":"2024-04-25T10:20:01.324520+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324520+0000","last_undegraded":"2024-04-25T10:20:01.324520+0000","last_fullsized":"2024-04-25T10:20:01.324520+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.353985+0000","last_change":"2024-04-25T10:20:02.353985+0000","last_active":"2024-04-25T10:20:02.353985+0000","last_peered":"2024-04-25T10:20:02.353985+0000","last_clean":"2024-04-25T10:20:02.353985+0000","last_became_active":"2024-04-25T10:20:02.353555+0000","last_became_peered":"2024-04-25T10:20:02.353555+0000","last_unstale":"2024-04-25T10:20:02.353985+0000","last_undegraded":"2024-04-25T10:20:02.353985+0000","last_fullsized":"2024-04-25T10:20:02.353985+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:02.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317986+0000","last_change":"2024-04-25T10:20:01.317986+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317986+0000","last_undegraded":"2024-04-25T10:20:01.317986+0000","last_fullsized":"2024-04-25T10:20:01.317986+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328773+0000","last_change":"2024-04-25T10:20:02.328773+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328773+0000","last_undegraded":"2024-04-25T10:20:02.328773+0000","last_fullsized":"2024-04-25T10:20:02.328773+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.319285+0000","last_change":"2024-04-25T10:20:01.319285+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.319285+0000","last_undegraded":"2024-04-25T10:20:01.319285+0000","last_fullsized":"2024-04-25T10:20:01.319285+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360903+0000","last_change":"2024-04-25T10:20:02.358637+0000","last_active":"2024-04-25T10:20:02.360903+0000","last_peered":"2024-04-25T10:20:02.360903+0000","last_clean":"2024-04-25T10:20:02.360903+0000","last_became_active":"2024-04-25T10:20:02.355276+0000","last_became_peered":"2024-04-25T10:20:02.355276+0000","last_unstale":"2024-04-25T10:20:02.360903+0000","last_undegraded":"2024-04-25T10:20:02.360903+0000","last_fullsized":"2024-04-25T10:20:02.360903+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:28.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.341785+0000","last_change":"2024-04-25T10:20:02.341785+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.341785+0000","last_undegraded":"2024-04-25T10:20:02.341785+0000","last_fullsized":"2024-04-25T10:20:02.341785+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.343799+0000","last_change":"2024-04-25T10:20:02.343799+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.343799+0000","last_undegraded":"2024-04-25T10:20:02.343799+0000","last_fullsized":"2024-04-25T10:20:02.343799+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315053+0000","last_change":"2024-04-25T10:20:01.315053+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315053+0000","last_undegraded":"2024-04-25T10:20:01.315053+0000","last_fullsized":"2024-04-25T10:20:01.315053+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.380366+0000","last_change":"2024-04-25T10:20:01.332782+0000","last_active":"2024-04-25T10:20:02.380366+0000","last_peered":"2024-04-25T10:20:02.380366+0000","last_clean":"2024-04-25T10:20:02.380366+0000","last_became_active":"2024-04-25T10:20:01.332481+0000","last_became_peered":"2024-04-25T10:20:01.332481+0000","last_unstale":"2024-04-25T10:20:02.380366+0000","last_undegraded":"2024-04-25T10:20:02.380366+0000","last_fullsized":"2024-04-25T10:20:02.380366+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:54.192098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344143+0000","last_change":"2024-04-25T10:20:02.344143+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344143+0000","last_undegraded":"2024-04-25T10:20:02.344143+0000","last_fullsized":"2024-04-25T10:20:02.344143+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345106+0000","last_change":"2024-04-25T10:20:01.335869+0000","last_active":"2024-04-25T10:20:02.345106+0000","last_peered":"2024-04-25T10:20:02.345106+0000","last_clean":"2024-04-25T10:20:02.345106+0000","last_became_active":"2024-04-25T10:20:01.335113+0000","last_became_peered":"2024-04-25T10:20:01.335113+0000","last_unstale":"2024-04-25T10:20:02.345106+0000","last_undegraded":"2024-04-25T10:20:02.345106+0000","last_fullsized":"2024-04-25T10:20:02.345106+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:59.188576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.351161+0000","last_change":"2024-04-25T10:20:02.343477+0000","last_active":"2024-04-25T10:20:02.351161+0000","last_peered":"2024-04-25T10:20:02.351161+0000","last_clean":"2024-04-25T10:20:02.351161+0000","last_became_active":"2024-04-25T10:20:02.342712+0000","last_became_peered":"2024-04-25T10:20:02.342712+0000","last_unstale":"2024-04-25T10:20:02.351161+0000","last_undegraded":"2024-04-25T10:20:02.351161+0000","last_fullsized":"2024-04-25T10:20:02.351161+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:00.592425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337187+0000","last_change":"2024-04-25T10:20:01.334447+0000","last_active":"2024-04-25T10:20:01.337187+0000","last_peered":"2024-04-25T10:20:01.337187+0000","last_clean":"2024-04-25T10:20:01.337187+0000","last_became_active":"2024-04-25T10:20:01.334157+0000","last_became_peered":"2024-04-25T10:20:01.334157+0000","last_unstale":"2024-04-25T10:20:01.337187+0000","last_undegraded":"2024-04-25T10:20:01.337187+0000","last_fullsized":"2024-04-25T10:20:01.337187+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.385948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.329080+0000","last_change":"2024-04-25T10:20:02.329080+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.329080+0000","last_undegraded":"2024-04-25T10:20:02.329080+0000","last_fullsized":"2024-04-25T10:20:02.329080+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.330180+0000","last_change":"2024-04-25T10:20:01.327488+0000","last_active":"2024-04-25T10:20:01.330180+0000","last_peered":"2024-04-25T10:20:01.330180+0000","last_clean":"2024-04-25T10:20:01.330180+0000","last_became_active":"2024-04-25T10:20:01.327214+0000","last_became_peered":"2024-04-25T10:20:01.327214+0000","last_unstale":"2024-04-25T10:20:01.330180+0000","last_undegraded":"2024-04-25T10:20:01.330180+0000","last_fullsized":"2024-04-25T10:20:01.330180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:08.837078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.361541+0000","last_change":"2024-04-25T10:20:02.360025+0000","last_active":"2024-04-25T10:20:02.361541+0000","last_peered":"2024-04-25T10:20:02.361541+0000","last_clean":"2024-04-25T10:20:02.361541+0000","last_became_active":"2024-04-25T10:20:02.359709+0000","last_became_peered":"2024-04-25T10:20:02.359709+0000","last_unstale":"2024-04-25T10:20:02.361541+0000","last_undegraded":"2024-04-25T10:20:02.361541+0000","last_fullsized":"2024-04-25T10:20:02.361541+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:45.508642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.343642+0000","last_change":"2024-04-25T10:20:01.341558+0000","last_active":"2024-04-25T10:20:01.343642+0000","last_peered":"2024-04-25T10:20:01.343642+0000","last_clean":"2024-04-25T10:20:01.343642+0000","last_became_active":"2024-04-25T10:20:01.340467+0000","last_became_peered":"2024-04-25T10:20:01.340467+0000","last_unstale":"2024-04-25T10:20:01.343642+0000","last_undegraded":"2024-04-25T10:20:01.343642+0000","last_fullsized":"2024-04-25T10:20:01.343642+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:28.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352047+0000","last_change":"2024-04-25T10:20:02.345657+0000","last_active":"2024-04-25T10:20:02.352047+0000","last_peered":"2024-04-25T10:20:02.352047+0000","last_clean":"2024-04-25T10:20:02.352047+0000","last_became_active":"2024-04-25T10:20:02.345353+0000","last_became_peered":"2024-04-25T10:20:02.345353+0000","last_unstale":"2024-04-25T10:20:02.352047+0000","last_undegraded":"2024-04-25T10:20:02.352047+0000","last_fullsized":"2024-04-25T10:20:02.352047+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.343333+0000","last_change":"2024-04-25T10:20:01.342761+0000","last_active":"2024-04-25T10:20:02.343333+0000","last_peered":"2024-04-25T10:20:02.343333+0000","last_clean":"2024-04-25T10:20:02.343333+0000","last_became_active":"2024-04-25T10:20:01.342145+0000","last_became_peered":"2024-04-25T10:20:01.342145+0000","last_unstale":"2024-04-25T10:20:02.343333+0000","last_undegraded":"2024-04-25T10:20:02.343333+0000","last_fullsized":"2024-04-25T10:20:02.343333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315024+0000","last_change":"2024-04-25T10:20:01.315024+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315024+0000","last_undegraded":"2024-04-25T10:20:01.315024+0000","last_fullsized":"2024-04-25T10:20:01.315024+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.339952+0000","last_change":"2024-04-25T10:20:01.338974+0000","last_active":"2024-04-25T10:20:01.339952+0000","last_peered":"2024-04-25T10:20:01.339952+0000","last_clean":"2024-04-25T10:20:01.339952+0000","last_became_active":"2024-04-25T10:20:01.338631+0000","last_became_peered":"2024-04-25T10:20:01.338631+0000","last_unstale":"2024-04-25T10:20:01.339952+0000","last_undegraded":"2024-04-25T10:20:01.339952+0000","last_fullsized":"2024-04-25T10:20:01.339952+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:08.857929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360631+0000","last_change":"2024-04-25T10:20:02.357818+0000","last_active":"2024-04-25T10:20:02.360631+0000","last_peered":"2024-04-25T10:20:02.360631+0000","last_clean":"2024-04-25T10:20:02.360631+0000","last_became_active":"2024-04-25T10:20:02.357502+0000","last_became_peered":"2024-04-25T10:20:02.357502+0000","last_unstale":"2024-04-25T10:20:02.360631+0000","last_undegraded":"2024-04-25T10:20:02.360631+0000","last_fullsized":"2024-04-25T10:20:02.360631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:57.523892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.330720+0000","last_change":"2024-04-25T10:20:01.332708+0000","last_active":"2024-04-25T10:20:02.330720+0000","last_peered":"2024-04-25T10:20:02.330720+0000","last_clean":"2024-04-25T10:20:02.330720+0000","last_became_active":"2024-04-25T10:20:01.332230+0000","last_became_peered":"2024-04-25T10:20:01.332230+0000","last_unstale":"2024-04-25T10:20:02.330720+0000","last_undegraded":"2024-04-25T10:20:02.330720+0000","last_fullsized":"2024-04-25T10:20:02.330720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:32.682708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.322584+0000","last_change":"2024-04-25T10:20:01.322584+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.322584+0000","last_undegraded":"2024-04-25T10:20:01.322584+0000","last_fullsized":"2024-04-25T10:20:01.322584+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.346371+0000","last_change":"2024-04-25T10:20:01.345530+0000","last_active":"2024-04-25T10:20:01.346371+0000","last_peered":"2024-04-25T10:20:01.346371+0000","last_clean":"2024-04-25T10:20:01.346371+0000","last_became_active":"2024-04-25T10:20:01.345190+0000","last_became_peered":"2024-04-25T10:20:01.345190+0000","last_unstale":"2024-04-25T10:20:01.346371+0000","last_undegraded":"2024-04-25T10:20:01.346371+0000","last_fullsized":"2024-04-25T10:20:01.346371+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:08.857929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315747+0000","last_change":"2024-04-25T10:20:01.315747+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315747+0000","last_undegraded":"2024-04-25T10:20:01.315747+0000","last_fullsized":"2024-04-25T10:20:01.315747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.331668+0000","last_change":"2024-04-25T10:20:01.329959+0000","last_active":"2024-04-25T10:20:01.331668+0000","last_peered":"2024-04-25T10:20:01.331668+0000","last_clean":"2024-04-25T10:20:01.331668+0000","last_became_active":"2024-04-25T10:20:01.329609+0000","last_became_peered":"2024-04-25T10:20:01.329609+0000","last_unstale":"2024-04-25T10:20:01.331668+0000","last_undegraded":"2024-04-25T10:20:01.331668+0000","last_fullsized":"2024-04-25T10:20:01.331668+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:45.592237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.329575+0000","last_change":"2024-04-25T10:20:02.329575+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.329575+0000","last_undegraded":"2024-04-25T10:20:02.329575+0000","last_fullsized":"2024-04-25T10:20:02.329575+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.344171+0000","last_change":"2024-04-25T10:20:01.343264+0000","last_active":"2024-04-25T10:20:01.344171+0000","last_peered":"2024-04-25T10:20:01.344171+0000","last_clean":"2024-04-25T10:20:01.344171+0000","last_became_active":"2024-04-25T10:20:01.342715+0000","last_became_peered":"2024-04-25T10:20:01.342715+0000","last_unstale":"2024-04-25T10:20:01.344171+0000","last_undegraded":"2024-04-25T10:20:01.344171+0000","last_fullsized":"2024-04-25T10:20:01.344171+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:41.996295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318832+0000","last_change":"2024-04-25T10:20:01.318832+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318832+0000","last_undegraded":"2024-04-25T10:20:01.318832+0000","last_fullsized":"2024-04-25T10:20:01.318832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.338100+0000","last_change":"2024-04-25T10:20:01.337329+0000","last_active":"2024-04-25T10:20:01.338100+0000","last_peered":"2024-04-25T10:20:01.338100+0000","last_clean":"2024-04-25T10:20:01.338100+0000","last_became_active":"2024-04-25T10:20:01.336415+0000","last_became_peered":"2024-04-25T10:20:01.336415+0000","last_unstale":"2024-04-25T10:20:01.338100+0000","last_undegraded":"2024-04-25T10:20:01.338100+0000","last_fullsized":"2024-04-25T10:20:01.338100+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.782792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317486+0000","last_change":"2024-04-25T10:20:01.317486+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317486+0000","last_undegraded":"2024-04-25T10:20:01.317486+0000","last_fullsized":"2024-04-25T10:20:01.317486+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.331224+0000","last_change":"2024-04-25T10:20:01.330029+0000","last_active":"2024-04-25T10:20:01.331224+0000","last_peered":"2024-04-25T10:20:01.331224+0000","last_clean":"2024-04-25T10:20:01.331224+0000","last_became_active":"2024-04-25T10:20:01.328434+0000","last_became_peered":"2024-04-25T10:20:01.328434+0000","last_unstale":"2024-04-25T10:20:01.331224+0000","last_undegraded":"2024-04-25T10:20:01.331224+0000","last_fullsized":"2024-04-25T10:20:01.331224+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:41.996295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318997+0000","last_change":"2024-04-25T10:20:01.318997+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318997+0000","last_undegraded":"2024-04-25T10:20:01.318997+0000","last_fullsized":"2024-04-25T10:20:01.318997+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.336749+0000","last_change":"2024-04-25T10:20:01.332999+0000","last_active":"2024-04-25T10:20:01.336749+0000","last_peered":"2024-04-25T10:20:01.336749+0000","last_clean":"2024-04-25T10:20:01.336749+0000","last_became_active":"2024-04-25T10:20:01.332513+0000","last_became_peered":"2024-04-25T10:20:01.332513+0000","last_unstale":"2024-04-25T10:20:01.336749+0000","last_undegraded":"2024-04-25T10:20:01.336749+0000","last_fullsized":"2024-04-25T10:20:01.336749+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:28.158755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.332203+0000","last_change":"2024-04-25T10:20:01.332203+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.332203+0000","last_undegraded":"2024-04-25T10:20:01.332203+0000","last_fullsized":"2024-04-25T10:20:01.332203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.327921+0000","last_change":"2024-04-25T10:20:01.338593+0000","last_active":"2024-04-25T10:20:02.327921+0000","last_peered":"2024-04-25T10:20:02.327921+0000","last_clean":"2024-04-25T10:20:02.327921+0000","last_became_active":"2024-04-25T10:20:01.338279+0000","last_became_peered":"2024-04-25T10:20:01.338279+0000","last_unstale":"2024-04-25T10:20:02.327921+0000","last_undegraded":"2024-04-25T10:20:02.327921+0000","last_fullsized":"2024-04-25T10:20:02.327921+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.211682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.366048+0000","last_change":"2024-04-25T10:20:02.363884+0000","last_active":"2024-04-25T10:20:02.366048+0000","last_peered":"2024-04-25T10:20:02.366048+0000","last_clean":"2024-04-25T10:20:02.366048+0000","last_became_active":"2024-04-25T10:20:02.363593+0000","last_became_peered":"2024-04-25T10:20:02.363593+0000","last_unstale":"2024-04-25T10:20:02.366048+0000","last_undegraded":"2024-04-25T10:20:02.366048+0000","last_fullsized":"2024-04-25T10:20:02.366048+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:28.866478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345829+0000","last_change":"2024-04-25T10:20:01.341364+0000","last_active":"2024-04-25T10:20:02.345829+0000","last_peered":"2024-04-25T10:20:02.345829+0000","last_clean":"2024-04-25T10:20:02.345829+0000","last_became_active":"2024-04-25T10:20:01.340654+0000","last_became_peered":"2024-04-25T10:20:01.340654+0000","last_unstale":"2024-04-25T10:20:02.345829+0000","last_undegraded":"2024-04-25T10:20:02.345829+0000","last_fullsized":"2024-04-25T10:20:02.345829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:34.211448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344027+0000","last_change":"2024-04-25T10:20:02.344027+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344027+0000","last_undegraded":"2024-04-25T10:20:02.344027+0000","last_fullsized":"2024-04-25T10:20:02.344027+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345295+0000","last_change":"2024-04-25T10:20:01.343272+0000","last_active":"2024-04-25T10:20:02.345295+0000","last_peered":"2024-04-25T10:20:02.345295+0000","last_clean":"2024-04-25T10:20:02.345295+0000","last_became_active":"2024-04-25T10:20:01.342389+0000","last_became_peered":"2024-04-25T10:20:01.342389+0000","last_unstale":"2024-04-25T10:20:02.345295+0000","last_undegraded":"2024-04-25T10:20:02.345295+0000","last_fullsized":"2024-04-25T10:20:02.345295+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:32.480221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.343886+0000","last_change":"2024-04-25T10:20:02.343886+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.343886+0000","last_undegraded":"2024-04-25T10:20:02.343886+0000","last_fullsized":"2024-04-25T10:20:02.343886+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.341155+0000","last_change":"2024-04-25T10:20:01.340613+0000","last_active":"2024-04-25T10:20:01.341155+0000","last_peered":"2024-04-25T10:20:01.341155+0000","last_clean":"2024-04-25T10:20:01.341155+0000","last_became_active":"2024-04-25T10:20:01.340250+0000","last_became_peered":"2024-04-25T10:20:01.340250+0000","last_unstale":"2024-04-25T10:20:01.341155+0000","last_undegraded":"2024-04-25T10:20:01.341155+0000","last_fullsized":"2024-04-25T10:20:01.341155+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:17.164219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.351303+0000","last_change":"2024-04-25T10:20:02.346899+0000","last_active":"2024-04-25T10:20:02.351303+0000","last_peered":"2024-04-25T10:20:02.351303+0000","last_clean":"2024-04-25T10:20:02.351303+0000","last_became_active":"2024-04-25T10:20:02.346568+0000","last_became_peered":"2024-04-25T10:20:02.346568+0000","last_unstale":"2024-04-25T10:20:02.351303+0000","last_undegraded":"2024-04-25T10:20:02.351303+0000","last_fullsized":"2024-04-25T10:20:02.351303+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.329544+0000","last_change":"2024-04-25T10:20:01.343143+0000","last_active":"2024-04-25T10:20:02.329544+0000","last_peered":"2024-04-25T10:20:02.329544+0000","last_clean":"2024-04-25T10:20:02.329544+0000","last_became_active":"2024-04-25T10:20:01.342912+0000","last_became_peered":"2024-04-25T10:20:01.342912+0000","last_unstale":"2024-04-25T10:20:02.329544+0000","last_undegraded":"2024-04-25T10:20:02.329544+0000","last_fullsized":"2024-04-25T10:20:02.329544+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.956577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318305+0000","last_change":"2024-04-25T10:20:01.318305+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318305+0000","last_undegraded":"2024-04-25T10:20:01.318305+0000","last_fullsized":"2024-04-25T10:20:01.318305+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.330106+0000","last_change":"2024-04-25T10:20:01.333143+0000","last_active":"2024-04-25T10:20:02.330106+0000","last_peered":"2024-04-25T10:20:02.330106+0000","last_clean":"2024-04-25T10:20:02.330106+0000","last_became_active":"2024-04-25T10:20:01.332865+0000","last_became_peered":"2024-04-25T10:20:01.332865+0000","last_unstale":"2024-04-25T10:20:02.330106+0000","last_undegraded":"2024-04-25T10:20:02.330106+0000","last_fullsized":"2024-04-25T10:20:02.330106+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:29.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.329524+0000","last_change":"2024-04-25T10:20:01.329524+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.329524+0000","last_undegraded":"2024-04-25T10:20:01.329524+0000","last_fullsized":"2024-04-25T10:20:01.329524+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.336032+0000","last_change":"2024-04-25T10:20:01.332184+0000","last_active":"2024-04-25T10:20:01.336032+0000","last_peered":"2024-04-25T10:20:01.336032+0000","last_clean":"2024-04-25T10:20:01.336032+0000","last_became_active":"2024-04-25T10:20:01.331558+0000","last_became_peered":"2024-04-25T10:20:01.331558+0000","last_unstale":"2024-04-25T10:20:01.336032+0000","last_undegraded":"2024-04-25T10:20:01.336032+0000","last_fullsized":"2024-04-25T10:20:01.336032+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:19.209857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.316990+0000","last_change":"2024-04-25T10:20:01.316990+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.316990+0000","last_undegraded":"2024-04-25T10:20:01.316990+0000","last_fullsized":"2024-04-25T10:20:01.316990+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.334000+0000","last_change":"2024-04-25T10:20:01.333236+0000","last_active":"2024-04-25T10:20:02.334000+0000","last_peered":"2024-04-25T10:20:02.334000+0000","last_clean":"2024-04-25T10:20:02.334000+0000","last_became_active":"2024-04-25T10:20:01.332681+0000","last_became_peered":"2024-04-25T10:20:01.332681+0000","last_unstale":"2024-04-25T10:20:02.334000+0000","last_undegraded":"2024-04-25T10:20:02.334000+0000","last_fullsized":"2024-04-25T10:20:02.334000+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:26.830908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.348938+0000","last_change":"2024-04-25T10:20:02.341693+0000","last_active":"2024-04-25T10:20:02.348938+0000","last_peered":"2024-04-25T10:20:02.348938+0000","last_clean":"2024-04-25T10:20:02.348938+0000","last_became_active":"2024-04-25T10:20:02.341396+0000","last_became_peered":"2024-04-25T10:20:02.341396+0000","last_unstale":"2024-04-25T10:20:02.348938+0000","last_undegraded":"2024-04-25T10:20:02.348938+0000","last_fullsized":"2024-04-25T10:20:02.348938+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:32.680379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345393+0000","last_change":"2024-04-25T10:20:01.334912+0000","last_active":"2024-04-25T10:20:02.345393+0000","last_peered":"2024-04-25T10:20:02.345393+0000","last_clean":"2024-04-25T10:20:02.345393+0000","last_became_active":"2024-04-25T10:20:01.334608+0000","last_became_peered":"2024-04-25T10:20:01.334608+0000","last_unstale":"2024-04-25T10:20:02.345393+0000","last_undegraded":"2024-04-25T10:20:02.345393+0000","last_fullsized":"2024-04-25T10:20:02.345393+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:19.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.363405+0000","last_change":"2024-04-25T10:20:02.356760+0000","last_active":"2024-04-25T10:20:02.363405+0000","last_peered":"2024-04-25T10:20:02.363405+0000","last_clean":"2024-04-25T10:20:02.363405+0000","last_became_active":"2024-04-25T10:20:02.356400+0000","last_became_peered":"2024-04-25T10:20:02.356400+0000","last_unstale":"2024-04-25T10:20:02.363405+0000","last_undegraded":"2024-04-25T10:20:02.363405+0000","last_fullsized":"2024-04-25T10:20:02.363405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:30.172815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337601+0000","last_change":"2024-04-25T10:20:01.335981+0000","last_active":"2024-04-25T10:20:01.337601+0000","last_peered":"2024-04-25T10:20:01.337601+0000","last_clean":"2024-04-25T10:20:01.337601+0000","last_became_active":"2024-04-25T10:20:01.335703+0000","last_became_peered":"2024-04-25T10:20:01.335703+0000","last_unstale":"2024-04-25T10:20:01.337601+0000","last_undegraded":"2024-04-25T10:20:01.337601+0000","last_fullsized":"2024-04-25T10:20:01.337601+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:29.441671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317980+0000","last_change":"2024-04-25T10:20:01.317980+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317980+0000","last_undegraded":"2024-04-25T10:20:01.317980+0000","last_fullsized":"2024-04-25T10:20:01.317980+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.336304+0000","last_change":"2024-04-25T10:20:01.332051+0000","last_active":"2024-04-25T10:20:02.336304+0000","last_peered":"2024-04-25T10:20:02.336304+0000","last_clean":"2024-04-25T10:20:02.336304+0000","last_became_active":"2024-04-25T10:20:01.331777+0000","last_became_peered":"2024-04-25T10:20:01.331777+0000","last_unstale":"2024-04-25T10:20:02.336304+0000","last_undegraded":"2024-04-25T10:20:02.336304+0000","last_fullsized":"2024-04-25T10:20:02.336304+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.465188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315284+0000","last_change":"2024-04-25T10:20:01.315284+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315284+0000","last_undegraded":"2024-04-25T10:20:01.315284+0000","last_fullsized":"2024-04-25T10:20:01.315284+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.334793+0000","last_change":"2024-04-25T10:20:01.323209+0000","last_active":"2024-04-25T10:20:02.334793+0000","last_peered":"2024-04-25T10:20:02.334793+0000","last_clean":"2024-04-25T10:20:02.334793+0000","last_became_active":"2024-04-25T10:20:01.319553+0000","last_became_peered":"2024-04-25T10:20:01.319553+0000","last_unstale":"2024-04-25T10:20:02.334793+0000","last_undegraded":"2024-04-25T10:20:02.334793+0000","last_fullsized":"2024-04-25T10:20:02.334793+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:40.645466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344286+0000","last_change":"2024-04-25T10:20:02.344286+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344286+0000","last_undegraded":"2024-04-25T10:20:02.344286+0000","last_fullsized":"2024-04-25T10:20:02.344286+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.346509+0000","last_change":"2024-04-25T10:20:01.336445+0000","last_active":"2024-04-25T10:20:02.346509+0000","last_peered":"2024-04-25T10:20:02.346509+0000","last_clean":"2024-04-25T10:20:02.346509+0000","last_became_active":"2024-04-25T10:20:01.336136+0000","last_became_peered":"2024-04-25T10:20:01.336136+0000","last_unstale":"2024-04-25T10:20:02.346509+0000","last_undegraded":"2024-04-25T10:20:02.346509+0000","last_fullsized":"2024-04-25T10:20:02.346509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.879308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.365379+0000","last_change":"2024-04-25T10:20:02.361100+0000","last_active":"2024-04-25T10:20:02.365379+0000","last_peered":"2024-04-25T10:20:02.365379+0000","last_clean":"2024-04-25T10:20:02.365379+0000","last_became_active":"2024-04-25T10:20:02.360829+0000","last_became_peered":"2024-04-25T10:20:02.360829+0000","last_unstale":"2024-04-25T10:20:02.365379+0000","last_undegraded":"2024-04-25T10:20:02.365379+0000","last_fullsized":"2024-04-25T10:20:02.365379+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:36.320764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.335687+0000","last_change":"2024-04-25T10:20:01.329676+0000","last_active":"2024-04-25T10:20:02.335687+0000","last_peered":"2024-04-25T10:20:02.335687+0000","last_clean":"2024-04-25T10:20:02.335687+0000","last_became_active":"2024-04-25T10:20:01.329366+0000","last_became_peered":"2024-04-25T10:20:01.329366+0000","last_unstale":"2024-04-25T10:20:02.335687+0000","last_undegraded":"2024-04-25T10:20:02.335687+0000","last_fullsized":"2024-04-25T10:20:02.335687+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:57.834078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.329095+0000","last_change":"2024-04-25T10:20:01.329095+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.329095+0000","last_undegraded":"2024-04-25T10:20:01.329095+0000","last_fullsized":"2024-04-25T10:20:01.329095+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337190+0000","last_change":"2024-04-25T10:20:01.334388+0000","last_active":"2024-04-25T10:20:01.337190+0000","last_peered":"2024-04-25T10:20:01.337190+0000","last_clean":"2024-04-25T10:20:01.337190+0000","last_became_active":"2024-04-25T10:20:01.333900+0000","last_became_peered":"2024-04-25T10:20:01.333900+0000","last_unstale":"2024-04-25T10:20:01.337190+0000","last_undegraded":"2024-04-25T10:20:01.337190+0000","last_fullsized":"2024-04-25T10:20:01.337190+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.991496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.327831+0000","last_change":"2024-04-25T10:20:01.327831+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.327831+0000","last_undegraded":"2024-04-25T10:20:01.327831+0000","last_fullsized":"2024-04-25T10:20:01.327831+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.346482+0000","last_change":"2024-04-25T10:20:01.335371+0000","last_active":"2024-04-25T10:20:02.346482+0000","last_peered":"2024-04-25T10:20:02.346482+0000","last_clean":"2024-04-25T10:20:02.346482+0000","last_became_active":"2024-04-25T10:20:01.335092+0000","last_became_peered":"2024-04-25T10:20:01.335092+0000","last_unstale":"2024-04-25T10:20:02.346482+0000","last_undegraded":"2024-04-25T10:20:02.346482+0000","last_fullsized":"2024-04-25T10:20:02.346482+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:44.129836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318525+0000","last_change":"2024-04-25T10:20:01.318525+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318525+0000","last_undegraded":"2024-04-25T10:20:01.318525+0000","last_fullsized":"2024-04-25T10:20:01.318525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.328352+0000","last_change":"2024-04-25T10:20:01.339862+0000","last_active":"2024-04-25T10:20:02.328352+0000","last_peered":"2024-04-25T10:20:02.328352+0000","last_clean":"2024-04-25T10:20:02.328352+0000","last_became_active":"2024-04-25T10:20:01.339331+0000","last_became_peered":"2024-04-25T10:20:01.339331+0000","last_unstale":"2024-04-25T10:20:02.328352+0000","last_undegraded":"2024-04-25T10:20:02.328352+0000","last_fullsized":"2024-04-25T10:20:02.328352+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:54.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344495+0000","last_change":"2024-04-25T10:20:02.344495+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344495+0000","last_undegraded":"2024-04-25T10:20:02.344495+0000","last_fullsized":"2024-04-25T10:20:02.344495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.330229+0000","last_change":"2024-04-25T10:20:01.329000+0000","last_active":"2024-04-25T10:20:01.330229+0000","last_peered":"2024-04-25T10:20:01.330229+0000","last_clean":"2024-04-25T10:20:01.330229+0000","last_became_active":"2024-04-25T10:20:01.328744+0000","last_became_peered":"2024-04-25T10:20:01.328744+0000","last_unstale":"2024-04-25T10:20:01.330229+0000","last_undegraded":"2024-04-25T10:20:01.330229+0000","last_fullsized":"2024-04-25T10:20:01.330229+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:31.060432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":480,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.344004+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:02.344004+0000","last_peered":"2024-04-25T10:20:02.344004+0000","last_clean":"2024-04-25T10:20:02.344004+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:02.344004+0000","last_undegraded":"2024-04-25T10:20:02.344004+0000","last_fullsized":"2024-04-25T10:20:02.344004+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575070,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60376,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311464,"statfs":{"total":96636764160,"available":96574939136,"internally_reserved":0,"allocated":2891776,"data_stored":1669863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59796,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312044,"statfs":{"total":96636764160,"available":96575533056,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87508,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284332,"statfs":{"total":96636764160,"available":96547155968,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575070,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87504,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284336,"statfs":{"total":96636764160,"available":96547160064,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575070,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65880,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305960,"statfs":{"total":96636764160,"available":96569303040,"internally_reserved":0,"allocated":2891776,"data_stored":1669863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:07.208 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.453+0000 7f8052200640 1 Processor -- start 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.453+0000 7f8052200640 1 -- start start 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.454+0000 7f8052200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfec0 0x7f804c0cde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.454+0000 7f8052200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f804c0ce3c0 0x7f804c06d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.454+0000 7f8052200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804c071480 con 0x7f804c154ee0 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.454+0000 7f8052200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c0ce830 con 0x7f804c0ce3c0 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.454+0000 7f8052200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804c072090 con 0x7f804c0cfec0 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f804c0ce3c0 0x7f804c06d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f804c0ce3c0 0x7f804c06d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47776/0 (socket says 172.21.15.165:47776) 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 -- 172.21.15.165:0/665320301 learned_addr learned my addr 172.21.15.165:0/665320301 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f8042a00640 1 --2- 172.21.15.165:0/665320301 >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfec0 0x7f804c0cde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 -- 172.21.15.165:0/665320301 >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfec0 msgr2=0x7f804c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 --2- 172.21.15.165:0/665320301 >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfec0 0x7f804c0cde80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 -- 172.21.15.165:0/665320301 >> v1:172.21.15.172:6789/0 conn(0x7f804c154ee0 legacy=0x7f804c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f804be00640 1 -- 172.21.15.165:0/665320301 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c071760 con 0x7f804c0ce3c0 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.455+0000 7f8042a00640 1 --2- 172.21.15.165:0/665320301 >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfec0 0x7f804c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.456+0000 7f804be00640 1 --2- 172.21.15.165:0/665320301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0ce3c0 0x7f804c06d6f0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f804c0cf560 tx=0x7f804402c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=161c31e351ae2e98 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.456+0000 7f804aa00640 1 -- 172.21.15.165:0/665320301 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804400b2d0 con 0x7f804c0ce3c0 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f804aa00640 1 -- 172.21.15.165:0/665320301 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f804400b470 con 0x7f804c0ce3c0 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f804aa00640 1 -- 172.21.15.165:0/665320301 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8044009820 con 0x7f804c0ce3c0 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 -- 172.21.15.165:0/665320301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0ce3c0 msgr2=0x7f804c06d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 --2- 172.21.15.165:0/665320301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0ce3c0 0x7f804c06d6f0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f804c0cf560 tx=0x7f804402c8f0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 -- 172.21.15.165:0/665320301 shutdown_connections 2024-04-25T10:20:07.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 --2- 172.21.15.165:0/665320301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0ce3c0 0x7f804c06d6f0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 --2- 172.21.15.165:0/665320301 >> v2:172.21.15.172:3300/0 conn(0x7f804c0cfec0 0x7f804c0cde80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 -- 172.21.15.165:0/665320301 >> 172.21.15.165:0/665320301 conn(0x7f804c06abe0 msgr2=0x7f804c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:07.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 -- 172.21.15.165:0/665320301 shutdown_connections 2024-04-25T10:20:07.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.457+0000 7f8052200640 1 -- 172.21.15.165:0/665320301 wait complete. 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.459+0000 7f8052200640 1 Processor -- start 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.459+0000 7f8052200640 1 -- start start 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8052200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0ce3c0 0x7f804c153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8052200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 0x7f804c159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8052200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f804c154ee0 0x7f804c159570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8052200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c071d20 con 0x7f804c0cfec0 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8052200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f804c071230 con 0x7f804c0ce3c0 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8052200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f804c1552c0 con 0x7f804c154ee0 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8042a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 0x7f804c159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8042a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 0x7f804c159030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47780/0 (socket says 172.21.15.165:47780) 2024-04-25T10:20:07.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.460+0000 7f8042a00640 1 -- 172.21.15.165:0/3911699407 learned_addr learned my addr 172.21.15.165:0/3911699407 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f804be00640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f804c154ee0 0x7f804c159570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f804b400640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0ce3c0 0x7f804c153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8042a00640 1 -- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f804c154ee0 msgr2=0x7f804c159570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8042a00640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f804c154ee0 0x7f804c159570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8042a00640 1 -- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0ce3c0 msgr2=0x7f804c153b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8042a00640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0ce3c0 0x7f804c153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8042a00640 1 -- 172.21.15.165:0/3911699407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f804c0ae2e0 con 0x7f804c0cfec0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8042a00640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 0x7f804c159030 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f803400c820 tx=0x7f803400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034015070 con 0x7f804c0cfec0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034007a30 con 0x7f804c0cfec0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.461+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f804c159ba0 con 0x7f804c0cfec0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.462+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034007e00 con 0x7f804c0cfec0 2024-04-25T10:20:07.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.462+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f804c0a8550 con 0x7f804c0cfec0 2024-04-25T10:20:07.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.463+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f804c073990 con 0x7f804c0cfec0 2024-04-25T10:20:07.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.464+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8034018070 con 0x7f804c0cfec0 2024-04-25T10:20:07.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.464+0000 7f8043e00640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802806b4d0 0x7f802806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.465+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f803408a400 con 0x7f804c0cfec0 2024-04-25T10:20:07.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.465+0000 7f804b400640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802806b4d0 0x7f802806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.465+0000 7f804b400640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802806b4d0 0x7f802806d990 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f803c006fd0 tx=0x7f803c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:07.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.469+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8034059050 con 0x7f804c0cfec0 2024-04-25T10:20:07.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.654+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f804c0cde60 con 0x7f802806b4d0 2024-04-25T10:20:07.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.673+0000 7f8043e00640 1 -- 172.21.15.165:0/3911699407 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+248840 (secure 0 0 0) 0x7f804c0cde60 con 0x7f802806b4d0 2024-04-25T10:20:07.679 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:07.680 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802806b4d0 msgr2=0x7f802806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802806b4d0 0x7f802806d990 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f803c006fd0 tx=0x7f803c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 msgr2=0x7f804c159030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 0x7f804c159030 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f803400c820 tx=0x7f803400cc20 comp rx=0 tx=0).stop 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 shutdown_connections 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f802806b4d0 0x7f802806d990 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f804c154ee0 0x7f804c159570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f804c0cfec0 0x7f804c159030 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 --2- 172.21.15.165:0/3911699407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f804c0ce3c0 0x7f804c153b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 >> 172.21.15.165:0/3911699407 conn(0x7f804c06abe0 msgr2=0x7f804c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:07.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 shutdown_connections 2024-04-25T10:20:07.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.678+0000 7f8052200640 1 -- 172.21.15.165:0/3911699407 wait complete. 2024-04-25T10:20:07.699 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":724,"stamp":"2024-04-25T10:20:06.498880+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590950,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":163,"num_write_kb":2417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":121,"ondisk_log_size":121,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":52,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536284,"kb_used_data":19112,"kb_used_omap":12,"kb_used_meta":517043,"kb_avail":754438436,"statfs":{"total":773094113280,"available":772544958464,"internally_reserved":0,"allocated":19570688,"data_stored":9816696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":529452594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001990"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.345532+0000","last_change":"2024-04-25T10:20:02.345532+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.345532+0000","last_undegraded":"2024-04-25T10:20:02.345532+0000","last_fullsized":"2024-04-25T10:20:02.345532+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.319974+0000","last_change":"2024-04-25T10:20:01.319974+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.319974+0000","last_undegraded":"2024-04-25T10:20:01.319974+0000","last_fullsized":"2024-04-25T10:20:01.319974+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318808+0000","last_change":"2024-04-25T10:20:01.318808+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318808+0000","last_undegraded":"2024-04-25T10:20:01.318808+0000","last_fullsized":"2024-04-25T10:20:01.318808+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.325908+0000","last_change":"2024-04-25T10:20:01.325908+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.325908+0000","last_undegraded":"2024-04-25T10:20:01.325908+0000","last_fullsized":"2024-04-25T10:20:01.325908+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328410+0000","last_change":"2024-04-25T10:20:02.328410+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328410+0000","last_undegraded":"2024-04-25T10:20:02.328410+0000","last_fullsized":"2024-04-25T10:20:02.328410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.342264+0000","last_change":"2024-04-25T10:20:02.342264+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.342264+0000","last_undegraded":"2024-04-25T10:20:02.342264+0000","last_fullsized":"2024-04-25T10:20:02.342264+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":["7","1"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328472+0000","last_change":"2024-04-25T10:20:02.328472+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328472+0000","last_undegraded":"2024-04-25T10:20:02.328472+0000","last_fullsized":"2024-04-25T10:20:02.328472+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.366027+0000","last_change":"2024-04-25T10:20:02.363273+0000","last_active":"2024-04-25T10:20:02.366027+0000","last_peered":"2024-04-25T10:20:02.366027+0000","last_clean":"2024-04-25T10:20:02.366027+0000","last_became_active":"2024-04-25T10:20:02.361340+0000","last_became_peered":"2024-04-25T10:20:02.361340+0000","last_unstale":"2024-04-25T10:20:02.366027+0000","last_undegraded":"2024-04-25T10:20:02.366027+0000","last_fullsized":"2024-04-25T10:20:02.366027+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:38:02.575100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.325436+0000","last_change":"2024-04-25T10:20:01.325436+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.325436+0000","last_undegraded":"2024-04-25T10:20:01.325436+0000","last_fullsized":"2024-04-25T10:20:01.325436+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.365799+0000","last_change":"2024-04-25T10:20:02.362899+0000","last_active":"2024-04-25T10:20:02.365799+0000","last_peered":"2024-04-25T10:20:02.365799+0000","last_clean":"2024-04-25T10:20:02.365799+0000","last_became_active":"2024-04-25T10:20:02.362609+0000","last_became_peered":"2024-04-25T10:20:02.362609+0000","last_unstale":"2024-04-25T10:20:02.365799+0000","last_undegraded":"2024-04-25T10:20:02.365799+0000","last_fullsized":"2024-04-25T10:20:02.365799+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:12.947488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318399+0000","last_change":"2024-04-25T10:20:01.318399+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318399+0000","last_undegraded":"2024-04-25T10:20:01.318399+0000","last_fullsized":"2024-04-25T10:20:01.318399+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328908+0000","last_change":"2024-04-25T10:20:02.328908+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328908+0000","last_undegraded":"2024-04-25T10:20:02.328908+0000","last_fullsized":"2024-04-25T10:20:02.328908+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324780+0000","last_change":"2024-04-25T10:20:01.324780+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324780+0000","last_undegraded":"2024-04-25T10:20:01.324780+0000","last_fullsized":"2024-04-25T10:20:01.324780+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352544+0000","last_change":"2024-04-25T10:20:02.350419+0000","last_active":"2024-04-25T10:20:02.352544+0000","last_peered":"2024-04-25T10:20:02.352544+0000","last_clean":"2024-04-25T10:20:02.352544+0000","last_became_active":"2024-04-25T10:20:02.350117+0000","last_became_peered":"2024-04-25T10:20:02.350117+0000","last_unstale":"2024-04-25T10:20:02.352544+0000","last_undegraded":"2024-04-25T10:20:02.352544+0000","last_fullsized":"2024-04-25T10:20:02.352544+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:35.826629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360697+0000","last_change":"2024-04-25T10:20:02.358059+0000","last_active":"2024-04-25T10:20:02.360697+0000","last_peered":"2024-04-25T10:20:02.360697+0000","last_clean":"2024-04-25T10:20:02.360697+0000","last_became_active":"2024-04-25T10:20:02.357465+0000","last_became_peered":"2024-04-25T10:20:02.357465+0000","last_unstale":"2024-04-25T10:20:02.360697+0000","last_undegraded":"2024-04-25T10:20:02.360697+0000","last_fullsized":"2024-04-25T10:20:02.360697+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:54.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315510+0000","last_change":"2024-04-25T10:20:01.315510+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315510+0000","last_undegraded":"2024-04-25T10:20:01.315510+0000","last_fullsized":"2024-04-25T10:20:01.315510+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324993+0000","last_change":"2024-04-25T10:20:01.324993+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324993+0000","last_undegraded":"2024-04-25T10:20:01.324993+0000","last_fullsized":"2024-04-25T10:20:01.324993+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.321276+0000","last_change":"2024-04-25T10:20:01.321276+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.321276+0000","last_undegraded":"2024-04-25T10:20:01.321276+0000","last_fullsized":"2024-04-25T10:20:01.321276+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.355356+0000","last_change":"2024-04-25T10:20:02.353518+0000","last_active":"2024-04-25T10:20:02.355356+0000","last_peered":"2024-04-25T10:20:02.355356+0000","last_clean":"2024-04-25T10:20:02.355356+0000","last_became_active":"2024-04-25T10:20:02.352415+0000","last_became_peered":"2024-04-25T10:20:02.352415+0000","last_unstale":"2024-04-25T10:20:02.355356+0000","last_undegraded":"2024-04-25T10:20:02.355356+0000","last_fullsized":"2024-04-25T10:20:02.355356+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:08:14.644570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352156+0000","last_change":"2024-04-25T10:20:02.349464+0000","last_active":"2024-04-25T10:20:02.352156+0000","last_peered":"2024-04-25T10:20:02.352156+0000","last_clean":"2024-04-25T10:20:02.352156+0000","last_became_active":"2024-04-25T10:20:02.349133+0000","last_became_peered":"2024-04-25T10:20:02.349133+0000","last_unstale":"2024-04-25T10:20:02.352156+0000","last_undegraded":"2024-04-25T10:20:02.352156+0000","last_fullsized":"2024-04-25T10:20:02.352156+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:09:31.263971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.361393+0000","last_change":"2024-04-25T10:20:02.356526+0000","last_active":"2024-04-25T10:20:02.361393+0000","last_peered":"2024-04-25T10:20:02.361393+0000","last_clean":"2024-04-25T10:20:02.361393+0000","last_became_active":"2024-04-25T10:20:02.355119+0000","last_became_peered":"2024-04-25T10:20:02.355119+0000","last_unstale":"2024-04-25T10:20:02.361393+0000","last_undegraded":"2024-04-25T10:20:02.361393+0000","last_fullsized":"2024-04-25T10:20:02.361393+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:44.172554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352518+0000","last_change":"2024-04-25T10:20:02.350034+0000","last_active":"2024-04-25T10:20:02.352518+0000","last_peered":"2024-04-25T10:20:02.352518+0000","last_clean":"2024-04-25T10:20:02.352518+0000","last_became_active":"2024-04-25T10:20:02.349692+0000","last_became_peered":"2024-04-25T10:20:02.349692+0000","last_unstale":"2024-04-25T10:20:02.352518+0000","last_undegraded":"2024-04-25T10:20:02.352518+0000","last_fullsized":"2024-04-25T10:20:02.352518+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:54:43.241274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315170+0000","last_change":"2024-04-25T10:20:01.315170+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315170+0000","last_undegraded":"2024-04-25T10:20:01.315170+0000","last_fullsized":"2024-04-25T10:20:01.315170+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328373+0000","last_change":"2024-04-25T10:20:02.328373+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328373+0000","last_undegraded":"2024-04-25T10:20:02.328373+0000","last_fullsized":"2024-04-25T10:20:02.328373+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.324520+0000","last_change":"2024-04-25T10:20:01.324520+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.324520+0000","last_undegraded":"2024-04-25T10:20:01.324520+0000","last_fullsized":"2024-04-25T10:20:01.324520+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":8,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.353985+0000","last_change":"2024-04-25T10:20:02.353985+0000","last_active":"2024-04-25T10:20:02.353985+0000","last_peered":"2024-04-25T10:20:02.353985+0000","last_clean":"2024-04-25T10:20:02.353985+0000","last_became_active":"2024-04-25T10:20:02.353555+0000","last_became_peered":"2024-04-25T10:20:02.353555+0000","last_unstale":"2024-04-25T10:20:02.353985+0000","last_undegraded":"2024-04-25T10:20:02.353985+0000","last_fullsized":"2024-04-25T10:20:02.353985+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:02.797523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317986+0000","last_change":"2024-04-25T10:20:01.317986+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317986+0000","last_undegraded":"2024-04-25T10:20:01.317986+0000","last_fullsized":"2024-04-25T10:20:01.317986+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.328773+0000","last_change":"2024-04-25T10:20:02.328773+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.328773+0000","last_undegraded":"2024-04-25T10:20:02.328773+0000","last_fullsized":"2024-04-25T10:20:02.328773+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":["2","7"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.319285+0000","last_change":"2024-04-25T10:20:01.319285+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.319285+0000","last_undegraded":"2024-04-25T10:20:01.319285+0000","last_fullsized":"2024-04-25T10:20:01.319285+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360903+0000","last_change":"2024-04-25T10:20:02.358637+0000","last_active":"2024-04-25T10:20:02.360903+0000","last_peered":"2024-04-25T10:20:02.360903+0000","last_clean":"2024-04-25T10:20:02.360903+0000","last_became_active":"2024-04-25T10:20:02.355276+0000","last_became_peered":"2024-04-25T10:20:02.355276+0000","last_unstale":"2024-04-25T10:20:02.360903+0000","last_undegraded":"2024-04-25T10:20:02.360903+0000","last_fullsized":"2024-04-25T10:20:02.360903+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:28.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.341785+0000","last_change":"2024-04-25T10:20:02.341785+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.341785+0000","last_undegraded":"2024-04-25T10:20:02.341785+0000","last_fullsized":"2024-04-25T10:20:02.341785+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.343799+0000","last_change":"2024-04-25T10:20:02.343799+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.343799+0000","last_undegraded":"2024-04-25T10:20:02.343799+0000","last_fullsized":"2024-04-25T10:20:02.343799+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315053+0000","last_change":"2024-04-25T10:20:01.315053+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315053+0000","last_undegraded":"2024-04-25T10:20:01.315053+0000","last_fullsized":"2024-04-25T10:20:01.315053+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"183'4","reported_seq":16,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.380366+0000","last_change":"2024-04-25T10:20:01.332782+0000","last_active":"2024-04-25T10:20:02.380366+0000","last_peered":"2024-04-25T10:20:02.380366+0000","last_clean":"2024-04-25T10:20:02.380366+0000","last_became_active":"2024-04-25T10:20:01.332481+0000","last_became_peered":"2024-04-25T10:20:01.332481+0000","last_unstale":"2024-04-25T10:20:02.380366+0000","last_undegraded":"2024-04-25T10:20:02.380366+0000","last_fullsized":"2024-04-25T10:20:02.380366+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:54.192098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344143+0000","last_change":"2024-04-25T10:20:02.344143+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344143+0000","last_undegraded":"2024-04-25T10:20:02.344143+0000","last_fullsized":"2024-04-25T10:20:02.344143+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":["5","4"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345106+0000","last_change":"2024-04-25T10:20:01.335869+0000","last_active":"2024-04-25T10:20:02.345106+0000","last_peered":"2024-04-25T10:20:02.345106+0000","last_clean":"2024-04-25T10:20:02.345106+0000","last_became_active":"2024-04-25T10:20:01.335113+0000","last_became_peered":"2024-04-25T10:20:01.335113+0000","last_unstale":"2024-04-25T10:20:02.345106+0000","last_undegraded":"2024-04-25T10:20:02.345106+0000","last_fullsized":"2024-04-25T10:20:02.345106+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:59.188576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.351161+0000","last_change":"2024-04-25T10:20:02.343477+0000","last_active":"2024-04-25T10:20:02.351161+0000","last_peered":"2024-04-25T10:20:02.351161+0000","last_clean":"2024-04-25T10:20:02.351161+0000","last_became_active":"2024-04-25T10:20:02.342712+0000","last_became_peered":"2024-04-25T10:20:02.342712+0000","last_unstale":"2024-04-25T10:20:02.351161+0000","last_undegraded":"2024-04-25T10:20:02.351161+0000","last_fullsized":"2024-04-25T10:20:02.351161+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:16:00.592425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337187+0000","last_change":"2024-04-25T10:20:01.334447+0000","last_active":"2024-04-25T10:20:01.337187+0000","last_peered":"2024-04-25T10:20:01.337187+0000","last_clean":"2024-04-25T10:20:01.337187+0000","last_became_active":"2024-04-25T10:20:01.334157+0000","last_became_peered":"2024-04-25T10:20:01.334157+0000","last_unstale":"2024-04-25T10:20:01.337187+0000","last_undegraded":"2024-04-25T10:20:01.337187+0000","last_fullsized":"2024-04-25T10:20:01.337187+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.385948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.329080+0000","last_change":"2024-04-25T10:20:02.329080+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.329080+0000","last_undegraded":"2024-04-25T10:20:02.329080+0000","last_fullsized":"2024-04-25T10:20:02.329080+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.330180+0000","last_change":"2024-04-25T10:20:01.327488+0000","last_active":"2024-04-25T10:20:01.330180+0000","last_peered":"2024-04-25T10:20:01.330180+0000","last_clean":"2024-04-25T10:20:01.330180+0000","last_became_active":"2024-04-25T10:20:01.327214+0000","last_became_peered":"2024-04-25T10:20:01.327214+0000","last_unstale":"2024-04-25T10:20:01.330180+0000","last_undegraded":"2024-04-25T10:20:01.330180+0000","last_fullsized":"2024-04-25T10:20:01.330180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:08.837078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.361541+0000","last_change":"2024-04-25T10:20:02.360025+0000","last_active":"2024-04-25T10:20:02.361541+0000","last_peered":"2024-04-25T10:20:02.361541+0000","last_clean":"2024-04-25T10:20:02.361541+0000","last_became_active":"2024-04-25T10:20:02.359709+0000","last_became_peered":"2024-04-25T10:20:02.359709+0000","last_unstale":"2024-04-25T10:20:02.361541+0000","last_undegraded":"2024-04-25T10:20:02.361541+0000","last_fullsized":"2024-04-25T10:20:02.361541+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:45.508642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.343642+0000","last_change":"2024-04-25T10:20:01.341558+0000","last_active":"2024-04-25T10:20:01.343642+0000","last_peered":"2024-04-25T10:20:01.343642+0000","last_clean":"2024-04-25T10:20:01.343642+0000","last_became_active":"2024-04-25T10:20:01.340467+0000","last_became_peered":"2024-04-25T10:20:01.340467+0000","last_unstale":"2024-04-25T10:20:01.343642+0000","last_undegraded":"2024-04-25T10:20:01.343642+0000","last_fullsized":"2024-04-25T10:20:01.343642+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:28.330559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.352047+0000","last_change":"2024-04-25T10:20:02.345657+0000","last_active":"2024-04-25T10:20:02.352047+0000","last_peered":"2024-04-25T10:20:02.352047+0000","last_clean":"2024-04-25T10:20:02.352047+0000","last_became_active":"2024-04-25T10:20:02.345353+0000","last_became_peered":"2024-04-25T10:20:02.345353+0000","last_unstale":"2024-04-25T10:20:02.352047+0000","last_undegraded":"2024-04-25T10:20:02.352047+0000","last_fullsized":"2024-04-25T10:20:02.352047+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.343333+0000","last_change":"2024-04-25T10:20:01.342761+0000","last_active":"2024-04-25T10:20:02.343333+0000","last_peered":"2024-04-25T10:20:02.343333+0000","last_clean":"2024-04-25T10:20:02.343333+0000","last_became_active":"2024-04-25T10:20:01.342145+0000","last_became_peered":"2024-04-25T10:20:01.342145+0000","last_unstale":"2024-04-25T10:20:02.343333+0000","last_undegraded":"2024-04-25T10:20:02.343333+0000","last_fullsized":"2024-04-25T10:20:02.343333+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315024+0000","last_change":"2024-04-25T10:20:01.315024+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315024+0000","last_undegraded":"2024-04-25T10:20:01.315024+0000","last_fullsized":"2024-04-25T10:20:01.315024+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.339952+0000","last_change":"2024-04-25T10:20:01.338974+0000","last_active":"2024-04-25T10:20:01.339952+0000","last_peered":"2024-04-25T10:20:01.339952+0000","last_clean":"2024-04-25T10:20:01.339952+0000","last_became_active":"2024-04-25T10:20:01.338631+0000","last_became_peered":"2024-04-25T10:20:01.338631+0000","last_unstale":"2024-04-25T10:20:01.339952+0000","last_undegraded":"2024-04-25T10:20:01.339952+0000","last_fullsized":"2024-04-25T10:20:01.339952+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:08.857929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.360631+0000","last_change":"2024-04-25T10:20:02.357818+0000","last_active":"2024-04-25T10:20:02.360631+0000","last_peered":"2024-04-25T10:20:02.360631+0000","last_clean":"2024-04-25T10:20:02.360631+0000","last_became_active":"2024-04-25T10:20:02.357502+0000","last_became_peered":"2024-04-25T10:20:02.357502+0000","last_unstale":"2024-04-25T10:20:02.360631+0000","last_undegraded":"2024-04-25T10:20:02.360631+0000","last_fullsized":"2024-04-25T10:20:02.360631+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:57.523892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.330720+0000","last_change":"2024-04-25T10:20:01.332708+0000","last_active":"2024-04-25T10:20:02.330720+0000","last_peered":"2024-04-25T10:20:02.330720+0000","last_clean":"2024-04-25T10:20:02.330720+0000","last_became_active":"2024-04-25T10:20:01.332230+0000","last_became_peered":"2024-04-25T10:20:01.332230+0000","last_unstale":"2024-04-25T10:20:02.330720+0000","last_undegraded":"2024-04-25T10:20:02.330720+0000","last_fullsized":"2024-04-25T10:20:02.330720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:32.682708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.322584+0000","last_change":"2024-04-25T10:20:01.322584+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.322584+0000","last_undegraded":"2024-04-25T10:20:01.322584+0000","last_fullsized":"2024-04-25T10:20:01.322584+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.346371+0000","last_change":"2024-04-25T10:20:01.345530+0000","last_active":"2024-04-25T10:20:01.346371+0000","last_peered":"2024-04-25T10:20:01.346371+0000","last_clean":"2024-04-25T10:20:01.346371+0000","last_became_active":"2024-04-25T10:20:01.345190+0000","last_became_peered":"2024-04-25T10:20:01.345190+0000","last_unstale":"2024-04-25T10:20:01.346371+0000","last_undegraded":"2024-04-25T10:20:01.346371+0000","last_fullsized":"2024-04-25T10:20:01.346371+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:08.857929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315747+0000","last_change":"2024-04-25T10:20:01.315747+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315747+0000","last_undegraded":"2024-04-25T10:20:01.315747+0000","last_fullsized":"2024-04-25T10:20:01.315747+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.331668+0000","last_change":"2024-04-25T10:20:01.329959+0000","last_active":"2024-04-25T10:20:01.331668+0000","last_peered":"2024-04-25T10:20:01.331668+0000","last_clean":"2024-04-25T10:20:01.331668+0000","last_became_active":"2024-04-25T10:20:01.329609+0000","last_became_peered":"2024-04-25T10:20:01.329609+0000","last_unstale":"2024-04-25T10:20:01.331668+0000","last_undegraded":"2024-04-25T10:20:01.331668+0000","last_fullsized":"2024-04-25T10:20:01.331668+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:45.592237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.329575+0000","last_change":"2024-04-25T10:20:02.329575+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.329575+0000","last_undegraded":"2024-04-25T10:20:02.329575+0000","last_fullsized":"2024-04-25T10:20:02.329575+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":["2","6"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.344171+0000","last_change":"2024-04-25T10:20:01.343264+0000","last_active":"2024-04-25T10:20:01.344171+0000","last_peered":"2024-04-25T10:20:01.344171+0000","last_clean":"2024-04-25T10:20:01.344171+0000","last_became_active":"2024-04-25T10:20:01.342715+0000","last_became_peered":"2024-04-25T10:20:01.342715+0000","last_unstale":"2024-04-25T10:20:01.344171+0000","last_undegraded":"2024-04-25T10:20:01.344171+0000","last_fullsized":"2024-04-25T10:20:01.344171+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:41.996295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318832+0000","last_change":"2024-04-25T10:20:01.318832+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318832+0000","last_undegraded":"2024-04-25T10:20:01.318832+0000","last_fullsized":"2024-04-25T10:20:01.318832+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.338100+0000","last_change":"2024-04-25T10:20:01.337329+0000","last_active":"2024-04-25T10:20:01.338100+0000","last_peered":"2024-04-25T10:20:01.338100+0000","last_clean":"2024-04-25T10:20:01.338100+0000","last_became_active":"2024-04-25T10:20:01.336415+0000","last_became_peered":"2024-04-25T10:20:01.336415+0000","last_unstale":"2024-04-25T10:20:01.338100+0000","last_undegraded":"2024-04-25T10:20:01.338100+0000","last_fullsized":"2024-04-25T10:20:01.338100+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:50.782792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317486+0000","last_change":"2024-04-25T10:20:01.317486+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317486+0000","last_undegraded":"2024-04-25T10:20:01.317486+0000","last_fullsized":"2024-04-25T10:20:01.317486+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.331224+0000","last_change":"2024-04-25T10:20:01.330029+0000","last_active":"2024-04-25T10:20:01.331224+0000","last_peered":"2024-04-25T10:20:01.331224+0000","last_clean":"2024-04-25T10:20:01.331224+0000","last_became_active":"2024-04-25T10:20:01.328434+0000","last_became_peered":"2024-04-25T10:20:01.328434+0000","last_unstale":"2024-04-25T10:20:01.331224+0000","last_undegraded":"2024-04-25T10:20:01.331224+0000","last_fullsized":"2024-04-25T10:20:01.331224+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:25:41.996295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318997+0000","last_change":"2024-04-25T10:20:01.318997+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318997+0000","last_undegraded":"2024-04-25T10:20:01.318997+0000","last_fullsized":"2024-04-25T10:20:01.318997+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.336749+0000","last_change":"2024-04-25T10:20:01.332999+0000","last_active":"2024-04-25T10:20:01.336749+0000","last_peered":"2024-04-25T10:20:01.336749+0000","last_clean":"2024-04-25T10:20:01.336749+0000","last_became_active":"2024-04-25T10:20:01.332513+0000","last_became_peered":"2024-04-25T10:20:01.332513+0000","last_unstale":"2024-04-25T10:20:01.336749+0000","last_undegraded":"2024-04-25T10:20:01.336749+0000","last_fullsized":"2024-04-25T10:20:01.336749+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:28.158755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.332203+0000","last_change":"2024-04-25T10:20:01.332203+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.332203+0000","last_undegraded":"2024-04-25T10:20:01.332203+0000","last_fullsized":"2024-04-25T10:20:01.332203+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.327921+0000","last_change":"2024-04-25T10:20:01.338593+0000","last_active":"2024-04-25T10:20:02.327921+0000","last_peered":"2024-04-25T10:20:02.327921+0000","last_clean":"2024-04-25T10:20:02.327921+0000","last_became_active":"2024-04-25T10:20:01.338279+0000","last_became_peered":"2024-04-25T10:20:01.338279+0000","last_unstale":"2024-04-25T10:20:02.327921+0000","last_undegraded":"2024-04-25T10:20:02.327921+0000","last_fullsized":"2024-04-25T10:20:02.327921+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.211682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.366048+0000","last_change":"2024-04-25T10:20:02.363884+0000","last_active":"2024-04-25T10:20:02.366048+0000","last_peered":"2024-04-25T10:20:02.366048+0000","last_clean":"2024-04-25T10:20:02.366048+0000","last_became_active":"2024-04-25T10:20:02.363593+0000","last_became_peered":"2024-04-25T10:20:02.363593+0000","last_unstale":"2024-04-25T10:20:02.366048+0000","last_undegraded":"2024-04-25T10:20:02.366048+0000","last_fullsized":"2024-04-25T10:20:02.366048+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:36:28.866478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345829+0000","last_change":"2024-04-25T10:20:01.341364+0000","last_active":"2024-04-25T10:20:02.345829+0000","last_peered":"2024-04-25T10:20:02.345829+0000","last_clean":"2024-04-25T10:20:02.345829+0000","last_became_active":"2024-04-25T10:20:01.340654+0000","last_became_peered":"2024-04-25T10:20:01.340654+0000","last_unstale":"2024-04-25T10:20:02.345829+0000","last_undegraded":"2024-04-25T10:20:02.345829+0000","last_fullsized":"2024-04-25T10:20:02.345829+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:34.211448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344027+0000","last_change":"2024-04-25T10:20:02.344027+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344027+0000","last_undegraded":"2024-04-25T10:20:02.344027+0000","last_fullsized":"2024-04-25T10:20:02.344027+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345295+0000","last_change":"2024-04-25T10:20:01.343272+0000","last_active":"2024-04-25T10:20:02.345295+0000","last_peered":"2024-04-25T10:20:02.345295+0000","last_clean":"2024-04-25T10:20:02.345295+0000","last_became_active":"2024-04-25T10:20:01.342389+0000","last_became_peered":"2024-04-25T10:20:01.342389+0000","last_unstale":"2024-04-25T10:20:02.345295+0000","last_undegraded":"2024-04-25T10:20:02.345295+0000","last_fullsized":"2024-04-25T10:20:02.345295+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:32.480221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.343886+0000","last_change":"2024-04-25T10:20:02.343886+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.343886+0000","last_undegraded":"2024-04-25T10:20:02.343886+0000","last_fullsized":"2024-04-25T10:20:02.343886+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.341155+0000","last_change":"2024-04-25T10:20:01.340613+0000","last_active":"2024-04-25T10:20:01.341155+0000","last_peered":"2024-04-25T10:20:01.341155+0000","last_clean":"2024-04-25T10:20:01.341155+0000","last_became_active":"2024-04-25T10:20:01.340250+0000","last_became_peered":"2024-04-25T10:20:01.340250+0000","last_unstale":"2024-04-25T10:20:01.341155+0000","last_undegraded":"2024-04-25T10:20:01.341155+0000","last_fullsized":"2024-04-25T10:20:01.341155+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:17.164219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.351303+0000","last_change":"2024-04-25T10:20:02.346899+0000","last_active":"2024-04-25T10:20:02.351303+0000","last_peered":"2024-04-25T10:20:02.351303+0000","last_clean":"2024-04-25T10:20:02.351303+0000","last_became_active":"2024-04-25T10:20:02.346568+0000","last_became_peered":"2024-04-25T10:20:02.346568+0000","last_unstale":"2024-04-25T10:20:02.351303+0000","last_undegraded":"2024-04-25T10:20:02.351303+0000","last_fullsized":"2024-04-25T10:20:02.351303+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:49:39.004696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.329544+0000","last_change":"2024-04-25T10:20:01.343143+0000","last_active":"2024-04-25T10:20:02.329544+0000","last_peered":"2024-04-25T10:20:02.329544+0000","last_clean":"2024-04-25T10:20:02.329544+0000","last_became_active":"2024-04-25T10:20:01.342912+0000","last_became_peered":"2024-04-25T10:20:01.342912+0000","last_unstale":"2024-04-25T10:20:02.329544+0000","last_undegraded":"2024-04-25T10:20:02.329544+0000","last_fullsized":"2024-04-25T10:20:02.329544+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.956577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318305+0000","last_change":"2024-04-25T10:20:01.318305+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318305+0000","last_undegraded":"2024-04-25T10:20:01.318305+0000","last_fullsized":"2024-04-25T10:20:01.318305+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.330106+0000","last_change":"2024-04-25T10:20:01.333143+0000","last_active":"2024-04-25T10:20:02.330106+0000","last_peered":"2024-04-25T10:20:02.330106+0000","last_clean":"2024-04-25T10:20:02.330106+0000","last_became_active":"2024-04-25T10:20:01.332865+0000","last_became_peered":"2024-04-25T10:20:01.332865+0000","last_unstale":"2024-04-25T10:20:02.330106+0000","last_undegraded":"2024-04-25T10:20:02.330106+0000","last_fullsized":"2024-04-25T10:20:02.330106+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:29.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.329524+0000","last_change":"2024-04-25T10:20:01.329524+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.329524+0000","last_undegraded":"2024-04-25T10:20:01.329524+0000","last_fullsized":"2024-04-25T10:20:01.329524+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.336032+0000","last_change":"2024-04-25T10:20:01.332184+0000","last_active":"2024-04-25T10:20:01.336032+0000","last_peered":"2024-04-25T10:20:01.336032+0000","last_clean":"2024-04-25T10:20:01.336032+0000","last_became_active":"2024-04-25T10:20:01.331558+0000","last_became_peered":"2024-04-25T10:20:01.331558+0000","last_unstale":"2024-04-25T10:20:01.336032+0000","last_undegraded":"2024-04-25T10:20:01.336032+0000","last_fullsized":"2024-04-25T10:20:01.336032+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:19:19.209857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.316990+0000","last_change":"2024-04-25T10:20:01.316990+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.316990+0000","last_undegraded":"2024-04-25T10:20:01.316990+0000","last_fullsized":"2024-04-25T10:20:01.316990+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.334000+0000","last_change":"2024-04-25T10:20:01.333236+0000","last_active":"2024-04-25T10:20:02.334000+0000","last_peered":"2024-04-25T10:20:02.334000+0000","last_clean":"2024-04-25T10:20:02.334000+0000","last_became_active":"2024-04-25T10:20:01.332681+0000","last_became_peered":"2024-04-25T10:20:01.332681+0000","last_unstale":"2024-04-25T10:20:02.334000+0000","last_undegraded":"2024-04-25T10:20:02.334000+0000","last_fullsized":"2024-04-25T10:20:02.334000+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:26.830908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.348938+0000","last_change":"2024-04-25T10:20:02.341693+0000","last_active":"2024-04-25T10:20:02.348938+0000","last_peered":"2024-04-25T10:20:02.348938+0000","last_clean":"2024-04-25T10:20:02.348938+0000","last_became_active":"2024-04-25T10:20:02.341396+0000","last_became_peered":"2024-04-25T10:20:02.341396+0000","last_unstale":"2024-04-25T10:20:02.348938+0000","last_undegraded":"2024-04-25T10:20:02.348938+0000","last_fullsized":"2024-04-25T10:20:02.348938+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:06:32.680379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.345393+0000","last_change":"2024-04-25T10:20:01.334912+0000","last_active":"2024-04-25T10:20:02.345393+0000","last_peered":"2024-04-25T10:20:02.345393+0000","last_clean":"2024-04-25T10:20:02.345393+0000","last_became_active":"2024-04-25T10:20:01.334608+0000","last_became_peered":"2024-04-25T10:20:01.334608+0000","last_unstale":"2024-04-25T10:20:02.345393+0000","last_undegraded":"2024-04-25T10:20:02.345393+0000","last_fullsized":"2024-04-25T10:20:02.345393+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:19.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.363405+0000","last_change":"2024-04-25T10:20:02.356760+0000","last_active":"2024-04-25T10:20:02.363405+0000","last_peered":"2024-04-25T10:20:02.363405+0000","last_clean":"2024-04-25T10:20:02.363405+0000","last_became_active":"2024-04-25T10:20:02.356400+0000","last_became_peered":"2024-04-25T10:20:02.356400+0000","last_unstale":"2024-04-25T10:20:02.363405+0000","last_undegraded":"2024-04-25T10:20:02.363405+0000","last_fullsized":"2024-04-25T10:20:02.363405+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:41:30.172815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337601+0000","last_change":"2024-04-25T10:20:01.335981+0000","last_active":"2024-04-25T10:20:01.337601+0000","last_peered":"2024-04-25T10:20:01.337601+0000","last_clean":"2024-04-25T10:20:01.337601+0000","last_became_active":"2024-04-25T10:20:01.335703+0000","last_became_peered":"2024-04-25T10:20:01.335703+0000","last_unstale":"2024-04-25T10:20:01.337601+0000","last_undegraded":"2024-04-25T10:20:01.337601+0000","last_fullsized":"2024-04-25T10:20:01.337601+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:29.441671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.317980+0000","last_change":"2024-04-25T10:20:01.317980+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.317980+0000","last_undegraded":"2024-04-25T10:20:01.317980+0000","last_fullsized":"2024-04-25T10:20:01.317980+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.336304+0000","last_change":"2024-04-25T10:20:01.332051+0000","last_active":"2024-04-25T10:20:02.336304+0000","last_peered":"2024-04-25T10:20:02.336304+0000","last_clean":"2024-04-25T10:20:02.336304+0000","last_became_active":"2024-04-25T10:20:01.331777+0000","last_became_peered":"2024-04-25T10:20:01.331777+0000","last_unstale":"2024-04-25T10:20:02.336304+0000","last_undegraded":"2024-04-25T10:20:02.336304+0000","last_fullsized":"2024-04-25T10:20:02.336304+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.465188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.315284+0000","last_change":"2024-04-25T10:20:01.315284+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.315284+0000","last_undegraded":"2024-04-25T10:20:01.315284+0000","last_fullsized":"2024-04-25T10:20:01.315284+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.334793+0000","last_change":"2024-04-25T10:20:01.323209+0000","last_active":"2024-04-25T10:20:02.334793+0000","last_peered":"2024-04-25T10:20:02.334793+0000","last_clean":"2024-04-25T10:20:02.334793+0000","last_became_active":"2024-04-25T10:20:01.319553+0000","last_became_peered":"2024-04-25T10:20:01.319553+0000","last_unstale":"2024-04-25T10:20:02.334793+0000","last_undegraded":"2024-04-25T10:20:02.334793+0000","last_fullsized":"2024-04-25T10:20:02.334793+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:40.645466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344286+0000","last_change":"2024-04-25T10:20:02.344286+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344286+0000","last_undegraded":"2024-04-25T10:20:02.344286+0000","last_fullsized":"2024-04-25T10:20:02.344286+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.346509+0000","last_change":"2024-04-25T10:20:01.336445+0000","last_active":"2024-04-25T10:20:02.346509+0000","last_peered":"2024-04-25T10:20:02.346509+0000","last_clean":"2024-04-25T10:20:02.346509+0000","last_became_active":"2024-04-25T10:20:01.336136+0000","last_became_peered":"2024-04-25T10:20:01.336136+0000","last_unstale":"2024-04-25T10:20:02.346509+0000","last_undegraded":"2024-04-25T10:20:02.346509+0000","last_fullsized":"2024-04-25T10:20:02.346509+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.879308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":9,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.365379+0000","last_change":"2024-04-25T10:20:02.361100+0000","last_active":"2024-04-25T10:20:02.365379+0000","last_peered":"2024-04-25T10:20:02.365379+0000","last_clean":"2024-04-25T10:20:02.365379+0000","last_became_active":"2024-04-25T10:20:02.360829+0000","last_became_peered":"2024-04-25T10:20:02.360829+0000","last_unstale":"2024-04-25T10:20:02.365379+0000","last_undegraded":"2024-04-25T10:20:02.365379+0000","last_fullsized":"2024-04-25T10:20:02.365379+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:04:36.320764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.335687+0000","last_change":"2024-04-25T10:20:01.329676+0000","last_active":"2024-04-25T10:20:02.335687+0000","last_peered":"2024-04-25T10:20:02.335687+0000","last_clean":"2024-04-25T10:20:02.335687+0000","last_became_active":"2024-04-25T10:20:01.329366+0000","last_became_peered":"2024-04-25T10:20:01.329366+0000","last_unstale":"2024-04-25T10:20:02.335687+0000","last_undegraded":"2024-04-25T10:20:02.335687+0000","last_fullsized":"2024-04-25T10:20:02.335687+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:57.834078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.329095+0000","last_change":"2024-04-25T10:20:01.329095+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.329095+0000","last_undegraded":"2024-04-25T10:20:01.329095+0000","last_fullsized":"2024-04-25T10:20:01.329095+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.337190+0000","last_change":"2024-04-25T10:20:01.334388+0000","last_active":"2024-04-25T10:20:01.337190+0000","last_peered":"2024-04-25T10:20:01.337190+0000","last_clean":"2024-04-25T10:20:01.337190+0000","last_became_active":"2024-04-25T10:20:01.333900+0000","last_became_peered":"2024-04-25T10:20:01.333900+0000","last_unstale":"2024-04-25T10:20:01.337190+0000","last_undegraded":"2024-04-25T10:20:01.337190+0000","last_fullsized":"2024-04-25T10:20:01.337190+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:04.991496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.327831+0000","last_change":"2024-04-25T10:20:01.327831+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.327831+0000","last_undegraded":"2024-04-25T10:20:01.327831+0000","last_fullsized":"2024-04-25T10:20:01.327831+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.346482+0000","last_change":"2024-04-25T10:20:01.335371+0000","last_active":"2024-04-25T10:20:02.346482+0000","last_peered":"2024-04-25T10:20:02.346482+0000","last_clean":"2024-04-25T10:20:02.346482+0000","last_became_active":"2024-04-25T10:20:01.335092+0000","last_became_peered":"2024-04-25T10:20:01.335092+0000","last_unstale":"2024-04-25T10:20:02.346482+0000","last_undegraded":"2024-04-25T10:20:02.346482+0000","last_fullsized":"2024-04-25T10:20:02.346482+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:44.129836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":1,"reported_epoch":182,"state":"creating+peering","last_fresh":"2024-04-25T10:20:01.318525+0000","last_change":"2024-04-25T10:20:01.318525+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:01.318525+0000","last_undegraded":"2024-04-25T10:20:01.318525+0000","last_fullsized":"2024-04-25T10:20:01.318525+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":12,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.328352+0000","last_change":"2024-04-25T10:20:01.339862+0000","last_active":"2024-04-25T10:20:02.328352+0000","last_peered":"2024-04-25T10:20:02.328352+0000","last_clean":"2024-04-25T10:20:02.328352+0000","last_became_active":"2024-04-25T10:20:01.339331+0000","last_became_peered":"2024-04-25T10:20:01.339331+0000","last_unstale":"2024-04-25T10:20:02.328352+0000","last_undegraded":"2024-04-25T10:20:02.328352+0000","last_fullsized":"2024-04-25T10:20:02.328352+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:54.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":3,"reported_epoch":183,"state":"creating+activating","last_fresh":"2024-04-25T10:20:02.344495+0000","last_change":"2024-04-25T10:20:02.344495+0000","last_active":"2024-04-25T10:20:01.297872+0000","last_peered":"2024-04-25T10:20:01.297872+0000","last_clean":"2024-04-25T10:20:01.297872+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-04-25T10:20:02.344495+0000","last_undegraded":"2024-04-25T10:20:02.344495+0000","last_fullsized":"2024-04-25T10:20:02.344495+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":["5","6"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":9,"reported_epoch":182,"state":"active+clean","last_fresh":"2024-04-25T10:20:01.330229+0000","last_change":"2024-04-25T10:20:01.329000+0000","last_active":"2024-04-25T10:20:01.330229+0000","last_peered":"2024-04-25T10:20:01.330229+0000","last_clean":"2024-04-25T10:20:01.330229+0000","last_became_active":"2024-04-25T10:20:01.328744+0000","last_became_peered":"2024-04-25T10:20:01.328744+0000","last_unstale":"2024-04-25T10:20:01.330229+0000","last_undegraded":"2024-04-25T10:20:01.330229+0000","last_fullsized":"2024-04-25T10:20:01.330229+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:31.060432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":480,"reported_epoch":183,"state":"active+clean","last_fresh":"2024-04-25T10:20:02.344004+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:02.344004+0000","last_peered":"2024-04-25T10:20:02.344004+0000","last_clean":"2024-04-25T10:20:02.344004+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:02.344004+0000","last_undegraded":"2024-04-25T10:20:02.344004+0000","last_fullsized":"2024-04-25T10:20:02.344004+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575070,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60376,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311464,"statfs":{"total":96636764160,"available":96574939136,"internally_reserved":0,"allocated":2891776,"data_stored":1669863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59796,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312044,"statfs":{"total":96636764160,"available":96575533056,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55636,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316204,"statfs":{"total":96636764160,"available":96579792896,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575070,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87508,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284332,"statfs":{"total":96636764160,"available":96547155968,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575070,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87504,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284336,"statfs":{"total":96636764160,"available":96547160064,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575069,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575070,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59792,"kb_used_data":2244,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312048,"statfs":{"total":96636764160,"available":96575537152,"internally_reserved":0,"allocated":2297856,"data_stored":1079495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575070,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65880,"kb_used_data":2824,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305960,"statfs":{"total":96636764160,"available":96569303040,"internally_reserved":0,"allocated":2891776,"data_stored":1669863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:07.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3f is not active+clean 2024-04-25T10:20:07.703 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3f', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.345532+0000', 'last_change': '2024-04-25T10:20:02.345532+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.345532+0000', 'last_undegraded': '2024-04-25T10:20:02.345532+0000', 'last_fullsized': '2024-04-25T10:20:02.345532+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:20:07.703 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3e is not active+clean 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.319974+0000', 'last_change': '2024-04-25T10:20:01.319974+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.319974+0000', 'last_undegraded': '2024-04-25T10:20:01.319974+0000', 'last_fullsized': '2024-04-25T10:20:01.319974+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3d is not active+clean 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.318808+0000', 'last_change': '2024-04-25T10:20:01.318808+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.318808+0000', 'last_undegraded': '2024-04-25T10:20:01.318808+0000', 'last_fullsized': '2024-04-25T10:20:01.318808+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3c is not active+clean 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.325908+0000', 'last_change': '2024-04-25T10:20:01.325908+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.325908+0000', 'last_undegraded': '2024-04-25T10:20:01.325908+0000', 'last_fullsized': '2024-04-25T10:20:01.325908+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3b is not active+clean 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.328410+0000', 'last_change': '2024-04-25T10:20:02.328410+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.328410+0000', 'last_undegraded': '2024-04-25T10:20:02.328410+0000', 'last_fullsized': '2024-04-25T10:20:02.328410+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3a is not active+clean 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.342264+0000', 'last_change': '2024-04-25T10:20:02.342264+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.342264+0000', 'last_undegraded': '2024-04-25T10:20:02.342264+0000', 'last_fullsized': '2024-04-25T10:20:02.342264+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': ['7', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.39 is not active+clean 2024-04-25T10:20:07.704 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.39', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.328472+0000', 'last_change': '2024-04-25T10:20:02.328472+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.328472+0000', 'last_undegraded': '2024-04-25T10:20:02.328472+0000', 'last_fullsized': '2024-04-25T10:20:02.328472+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.37 is not active+clean 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.325436+0000', 'last_change': '2024-04-25T10:20:01.325436+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.325436+0000', 'last_undegraded': '2024-04-25T10:20:01.325436+0000', 'last_fullsized': '2024-04-25T10:20:01.325436+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.35 is not active+clean 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.318399+0000', 'last_change': '2024-04-25T10:20:01.318399+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.318399+0000', 'last_undegraded': '2024-04-25T10:20:01.318399+0000', 'last_fullsized': '2024-04-25T10:20:01.318399+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.34 is not active+clean 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.34', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.328908+0000', 'last_change': '2024-04-25T10:20:02.328908+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.328908+0000', 'last_undegraded': '2024-04-25T10:20:02.328908+0000', 'last_fullsized': '2024-04-25T10:20:02.328908+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.33 is not active+clean 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.324780+0000', 'last_change': '2024-04-25T10:20:01.324780+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.324780+0000', 'last_undegraded': '2024-04-25T10:20:01.324780+0000', 'last_fullsized': '2024-04-25T10:20:01.324780+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.30 is not active+clean 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.315510+0000', 'last_change': '2024-04-25T10:20:01.315510+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.315510+0000', 'last_undegraded': '2024-04-25T10:20:01.315510+0000', 'last_fullsized': '2024-04-25T10:20:01.315510+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.2f is not active+clean 2024-04-25T10:20:07.705 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.324993+0000', 'last_change': '2024-04-25T10:20:01.324993+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.324993+0000', 'last_undegraded': '2024-04-25T10:20:01.324993+0000', 'last_fullsized': '2024-04-25T10:20:01.324993+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.2e is not active+clean 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.321276+0000', 'last_change': '2024-04-25T10:20:01.321276+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.321276+0000', 'last_undegraded': '2024-04-25T10:20:01.321276+0000', 'last_fullsized': '2024-04-25T10:20:01.321276+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.29 is not active+clean 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.315170+0000', 'last_change': '2024-04-25T10:20:01.315170+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.315170+0000', 'last_undegraded': '2024-04-25T10:20:01.315170+0000', 'last_fullsized': '2024-04-25T10:20:01.315170+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.28 is not active+clean 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.28', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.328373+0000', 'last_change': '2024-04-25T10:20:02.328373+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.328373+0000', 'last_undegraded': '2024-04-25T10:20:02.328373+0000', 'last_fullsized': '2024-04-25T10:20:02.328373+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.27 is not active+clean 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.324520+0000', 'last_change': '2024-04-25T10:20:01.324520+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.324520+0000', 'last_undegraded': '2024-04-25T10:20:01.324520+0000', 'last_fullsized': '2024-04-25T10:20:01.324520+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.25 is not active+clean 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.317986+0000', 'last_change': '2024-04-25T10:20:01.317986+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.317986+0000', 'last_undegraded': '2024-04-25T10:20:01.317986+0000', 'last_fullsized': '2024-04-25T10:20:01.317986+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.24 is not active+clean 2024-04-25T10:20:07.706 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.24', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.328773+0000', 'last_change': '2024-04-25T10:20:02.328773+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.328773+0000', 'last_undegraded': '2024-04-25T10:20:02.328773+0000', 'last_fullsized': '2024-04-25T10:20:02.328773+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': ['2', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.23 is not active+clean 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.319285+0000', 'last_change': '2024-04-25T10:20:01.319285+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.319285+0000', 'last_undegraded': '2024-04-25T10:20:01.319285+0000', 'last_fullsized': '2024-04-25T10:20:01.319285+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.21 is not active+clean 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.21', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.341785+0000', 'last_change': '2024-04-25T10:20:02.341785+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.341785+0000', 'last_undegraded': '2024-04-25T10:20:02.341785+0000', 'last_fullsized': '2024-04-25T10:20:02.341785+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': ['7', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.20 is not active+clean 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.20', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.343799+0000', 'last_change': '2024-04-25T10:20:02.343799+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.343799+0000', 'last_undegraded': '2024-04-25T10:20:02.343799+0000', 'last_fullsized': '2024-04-25T10:20:02.343799+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': ['7', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.18 is not active+clean 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.315053+0000', 'last_change': '2024-04-25T10:20:01.315053+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.315053+0000', 'last_undegraded': '2024-04-25T10:20:01.315053+0000', 'last_fullsized': '2024-04-25T10:20:01.315053+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.19 is not active+clean 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.19', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.344143+0000', 'last_change': '2024-04-25T10:20:02.344143+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.344143+0000', 'last_undegraded': '2024-04-25T10:20:02.344143+0000', 'last_fullsized': '2024-04-25T10:20:02.344143+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': ['5', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.1b is not active+clean 2024-04-25T10:20:07.707 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.1b', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.329080+0000', 'last_change': '2024-04-25T10:20:02.329080+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.329080+0000', 'last_undegraded': '2024-04-25T10:20:02.329080+0000', 'last_fullsized': '2024-04-25T10:20:02.329080+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.1f is not active+clean 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.315024+0000', 'last_change': '2024-04-25T10:20:01.315024+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.315024+0000', 'last_undegraded': '2024-04-25T10:20:01.315024+0000', 'last_fullsized': '2024-04-25T10:20:01.315024+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.11 is not active+clean 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.322584+0000', 'last_change': '2024-04-25T10:20:01.322584+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.322584+0000', 'last_undegraded': '2024-04-25T10:20:01.322584+0000', 'last_fullsized': '2024-04-25T10:20:01.322584+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.12 is not active+clean 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.315747+0000', 'last_change': '2024-04-25T10:20:01.315747+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.315747+0000', 'last_undegraded': '2024-04-25T10:20:01.315747+0000', 'last_fullsized': '2024-04-25T10:20:01.315747+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.13 is not active+clean 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.13', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.329575+0000', 'last_change': '2024-04-25T10:20:02.329575+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.329575+0000', 'last_undegraded': '2024-04-25T10:20:02.329575+0000', 'last_fullsized': '2024-04-25T10:20:02.329575+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': ['2', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.14 is not active+clean 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.318832+0000', 'last_change': '2024-04-25T10:20:01.318832+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.318832+0000', 'last_undegraded': '2024-04-25T10:20:01.318832+0000', 'last_fullsized': '2024-04-25T10:20:01.318832+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.15 is not active+clean 2024-04-25T10:20:07.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.317486+0000', 'last_change': '2024-04-25T10:20:01.317486+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.317486+0000', 'last_undegraded': '2024-04-25T10:20:01.317486+0000', 'last_fullsized': '2024-04-25T10:20:01.317486+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.16 is not active+clean 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.318997+0000', 'last_change': '2024-04-25T10:20:01.318997+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.318997+0000', 'last_undegraded': '2024-04-25T10:20:01.318997+0000', 'last_fullsized': '2024-04-25T10:20:01.318997+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.17 is not active+clean 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.332203+0000', 'last_change': '2024-04-25T10:20:01.332203+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.332203+0000', 'last_undegraded': '2024-04-25T10:20:01.332203+0000', 'last_fullsized': '2024-04-25T10:20:01.332203+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.9 is not active+clean 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.9', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.344027+0000', 'last_change': '2024-04-25T10:20:02.344027+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.344027+0000', 'last_undegraded': '2024-04-25T10:20:02.344027+0000', 'last_fullsized': '2024-04-25T10:20:02.344027+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.a is not active+clean 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.a', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.343886+0000', 'last_change': '2024-04-25T10:20:02.343886+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.343886+0000', 'last_undegraded': '2024-04-25T10:20:02.343886+0000', 'last_fullsized': '2024-04-25T10:20:02.343886+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.c is not active+clean 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.318305+0000', 'last_change': '2024-04-25T10:20:01.318305+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.318305+0000', 'last_undegraded': '2024-04-25T10:20:01.318305+0000', 'last_fullsized': '2024-04-25T10:20:01.318305+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.d is not active+clean 2024-04-25T10:20:07.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.329524+0000', 'last_change': '2024-04-25T10:20:01.329524+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.329524+0000', 'last_undegraded': '2024-04-25T10:20:01.329524+0000', 'last_fullsized': '2024-04-25T10:20:01.329524+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.e is not active+clean 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.316990+0000', 'last_change': '2024-04-25T10:20:01.316990+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.316990+0000', 'last_undegraded': '2024-04-25T10:20:01.316990+0000', 'last_fullsized': '2024-04-25T10:20:01.316990+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.1 is not active+clean 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.317980+0000', 'last_change': '2024-04-25T10:20:01.317980+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.317980+0000', 'last_undegraded': '2024-04-25T10:20:01.317980+0000', 'last_fullsized': '2024-04-25T10:20:01.317980+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.2 is not active+clean 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.315284+0000', 'last_change': '2024-04-25T10:20:01.315284+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.315284+0000', 'last_undegraded': '2024-04-25T10:20:01.315284+0000', 'last_fullsized': '2024-04-25T10:20:01.315284+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.3 is not active+clean 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.3', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.344286+0000', 'last_change': '2024-04-25T10:20:02.344286+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.344286+0000', 'last_undegraded': '2024-04-25T10:20:02.344286+0000', 'last_fullsized': '2024-04-25T10:20:02.344286+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.5 is not active+clean 2024-04-25T10:20:07.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.329095+0000', 'last_change': '2024-04-25T10:20:01.329095+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.329095+0000', 'last_undegraded': '2024-04-25T10:20:01.329095+0000', 'last_fullsized': '2024-04-25T10:20:01.329095+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.6 is not active+clean 2024-04-25T10:20:07.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.327831+0000', 'last_change': '2024-04-25T10:20:01.327831+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.327831+0000', 'last_undegraded': '2024-04-25T10:20:01.327831+0000', 'last_fullsized': '2024-04-25T10:20:01.327831+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-04-25T10:20:07.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.7 is not active+clean 2024-04-25T10:20:07.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 182, 'state': 'creating+peering', 'last_fresh': '2024-04-25T10:20:01.318525+0000', 'last_change': '2024-04-25T10:20:01.318525+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:01.318525+0000', 'last_undegraded': '2024-04-25T10:20:01.318525+0000', 'last_fullsized': '2024-04-25T10:20:01.318525+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-04-25T10:20:07.711 INFO:tasks.cephfs.filesystem.ceph_manager:PG 28.1d is not active+clean 2024-04-25T10:20:07.711 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '28.1d', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 183, 'state': 'creating+activating', 'last_fresh': '2024-04-25T10:20:02.344495+0000', 'last_change': '2024-04-25T10:20:02.344495+0000', 'last_active': '2024-04-25T10:20:01.297872+0000', 'last_peered': '2024-04-25T10:20:01.297872+0000', 'last_clean': '2024-04-25T10:20:01.297872+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-04-25T10:20:02.344495+0000', 'last_undegraded': '2024-04-25T10:20:02.344495+0000', 'last_fullsized': '2024-04-25T10:20:02.344495+0000', 'mapping_epoch': 182, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 182, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'last_clean_scrub_stamp': '2024-04-25T10:20:01.297872+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': ['5', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-04-25T10:20:07.712 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-25T10:20:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.934+0000 7fadbbe00640 1 Processor -- start 2024-04-25T10:20:07.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.934+0000 7fadbbe00640 1 -- start start 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadbbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fadb415db70 0x7fadb40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadbbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fadb415df50 0x7fadb40cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb40cec70 0x7fadb40cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadbbe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb40cbd80 con 0x7fadb415df50 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadbbe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadb4161910 con 0x7fadb415db70 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadbbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4071650 con 0x7fadb40cec70 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadb8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fadb415df50 0x7fadb40cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadb8a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fadb415df50 0x7fadb40cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47794/0 (socket says 172.21.15.165:47794) 2024-04-25T10:20:07.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.936+0000 7fadb8a00640 1 -- 172.21.15.165:0/1922617670 learned_addr learned my addr 172.21.15.165:0/1922617670 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb8a00640 1 -- 172.21.15.165:0/1922617670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb40cec70 msgr2=0x7fadb40cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb9400640 1 --2- 172.21.15.165:0/1922617670 >> v2:172.21.15.172:3300/0 conn(0x7fadb415db70 0x7fadb40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb9e00640 1 --2- 172.21.15.165:0/1922617670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb40cec70 0x7fadb40cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb8a00640 1 --2- 172.21.15.165:0/1922617670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb40cec70 0x7fadb40cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb8a00640 1 -- 172.21.15.165:0/1922617670 >> v2:172.21.15.172:3300/0 conn(0x7fadb415db70 msgr2=0x7fadb40cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb8a00640 1 --2- 172.21.15.165:0/1922617670 >> v2:172.21.15.172:3300/0 conn(0x7fadb415db70 0x7fadb40cb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.937+0000 7fadb8a00640 1 -- 172.21.15.165:0/1922617670 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb40cf230 con 0x7fadb415df50 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.938+0000 7fadb8a00640 1 --2- 172.21.15.165:0/1922617670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb415df50 0x7fadb40cf510 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7fadb40cd3d0 tx=0x7fada002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e34498ffcc66eb server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.938+0000 7fadb3e00640 1 -- 172.21.15.165:0/1922617670 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0037070 con 0x7fadb415df50 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.938+0000 7fadb3e00640 1 -- 172.21.15.165:0/1922617670 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada0032440 con 0x7fadb415df50 2024-04-25T10:20:07.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.938+0000 7fadb3e00640 1 -- 172.21.15.165:0/1922617670 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada0032810 con 0x7fadb415df50 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 -- 172.21.15.165:0/1922617670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb415df50 msgr2=0x7fadb40cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 --2- 172.21.15.165:0/1922617670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb415df50 0x7fadb40cf510 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7fadb40cd3d0 tx=0x7fada002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 -- 172.21.15.165:0/1922617670 shutdown_connections 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 --2- 172.21.15.165:0/1922617670 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb40cec70 0x7fadb40cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 --2- 172.21.15.165:0/1922617670 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb415df50 0x7fadb40cf510 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 --2- 172.21.15.165:0/1922617670 >> v2:172.21.15.172:3300/0 conn(0x7fadb415db70 0x7fadb40cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 -- 172.21.15.165:0/1922617670 >> 172.21.15.165:0/1922617670 conn(0x7fadb406abe0 msgr2=0x7fadb406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 -- 172.21.15.165:0/1922617670 shutdown_connections 2024-04-25T10:20:07.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.939+0000 7fadbbe00640 1 -- 172.21.15.165:0/1922617670 wait complete. 2024-04-25T10:20:07.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.941+0000 7fadbbe00640 1 Processor -- start 2024-04-25T10:20:07.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.941+0000 7fadbbe00640 1 -- start start 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 0x7fadb41542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb415db70 0x7fadb40b1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadbbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadb415df50 0x7fadb40ada30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadbbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4162120 con 0x7fadb40cec70 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadbbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4161e40 con 0x7fadb415df50 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadbbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadb4162640 con 0x7fadb415db70 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 0x7fadb41542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 0x7fadb41542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47796/0 (socket says 172.21.15.165:47796) 2024-04-25T10:20:07.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 -- 172.21.15.165:0/401537134 learned_addr learned my addr 172.21.15.165:0/401537134 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:07.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 -- 172.21.15.165:0/401537134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb415db70 msgr2=0x7fadb40b1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb415db70 0x7fadb40b1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 -- 172.21.15.165:0/401537134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadb415df50 msgr2=0x7fadb40ada30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:07.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadb415df50 0x7fadb40ada30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.942+0000 7fadb9400640 1 -- 172.21.15.165:0/401537134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb4160800 con 0x7fadb40cec70 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.943+0000 7fadb9400640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 0x7fadb41542e0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fada800ce10 tx=0x7fada800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.943+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8003b90 con 0x7fadb40cec70 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.943+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada8003d30 con 0x7fadb40cec70 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.943+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadb4160520 con 0x7fadb40cec70 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.943+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8004240 con 0x7fadb40cec70 2024-04-25T10:20:07.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.943+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadb40af900 con 0x7fadb40cec70 2024-04-25T10:20:07.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.944+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadb4073990 con 0x7fadb40cec70 2024-04-25T10:20:07.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.945+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fada80043e0 con 0x7fadb40cec70 2024-04-25T10:20:07.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.946+0000 7fadb1600640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad9406b500 0x7fad9406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:07.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.946+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fada808aee0 con 0x7fadb40cec70 2024-04-25T10:20:07.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.946+0000 7fadb8a00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad9406b500 0x7fad9406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:07.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.947+0000 7fadb8a00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad9406b500 0x7fad9406d9c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:07.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:07.951+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fada801d340 con 0x7fadb40cec70 2024-04-25T10:20:08.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.196+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fadb4158f30 con 0x7fadb40cec70 2024-04-25T10:20:08.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.197+0000 7fadb1600640 1 -- 172.21.15.165:0/401537134 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1313 (secure 0 0 0) 0x7fada8059b30 con 0x7fadb40cec70 2024-04-25T10:20:08.200 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:08.200 INFO:teuthology.orchestra.run.smithi165.stdout:{"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1109,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":184,"num_osds":8,"num_up_osds":8,"osd_up_since":1714039311,"num_in_osds":8,"osd_in_since":1714039302,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":52},{"state_name":"creating+peering","count":29},{"state_name":"creating+activating","count":16}],"num_pgs":97,"num_pools":3,"num_objects":4,"data_bytes":590950,"bytes_used":549154816,"bytes_avail":772544958464,"bytes_total":773094113280,"inactive_pgs_ratio":0.46391752362251282,"write_bytes_sec":170,"read_op_per_sec":0,"write_op_per_sec":0},"fsmap":{"epoch":130,"id":13,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":13,"rank":0,"name":"c","status":"up:active","gid":11211}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":40,"modified":"2024-04-25T10:19:54.495080+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-04-25T10:20:08.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad9406b500 msgr2=0x7fad9406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad9406b500 0x7fad9406d9c0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7fada0034000 tx=0x7fada002c000 comp rx=0 tx=0).stop 2024-04-25T10:20:08.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 msgr2=0x7fadb41542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 0x7fadb41542e0 secure :-1 s=READY pgs=2601 cs=0 l=1 rev1=1 crypto rx=0x7fada800ce10 tx=0x7fada800a660 comp rx=0 tx=0).stop 2024-04-25T10:20:08.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 shutdown_connections 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fad9406b500 0x7fad9406d9c0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fadb415df50 0x7fadb40ada30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fadb415db70 0x7fadb40b1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 --2- 172.21.15.165:0/401537134 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fadb40cec70 0x7fadb41542e0 unknown :-1 s=CLOSED pgs=2601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 >> 172.21.15.165:0/401537134 conn(0x7fadb406abe0 msgr2=0x7fadb41596f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 shutdown_connections 2024-04-25T10:20:08.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.200+0000 7fadbbe00640 1 -- 172.21.15.165:0/401537134 wait complete. 2024-04-25T10:20:08.220 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2024-04-25T10:20:08.220 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:08.272 INFO:tasks.ceph.mds.c.smithi165.stderr:2024-04-25T10:20:08.269+0000 7f93a5800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:20:08.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.411+0000 7fd6d9a00640 1 Processor -- start 2024-04-25T10:20:08.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.411+0000 7fd6d9a00640 1 -- start start 2024-04-25T10:20:08.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d9a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6d4154ee0 0x7fd6d40cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:08.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d9a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd6d40ce590 0x7fd6d406d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:08.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d9a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d4071480 con 0x7fd6d40cfcf0 2024-04-25T10:20:08.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d9a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d4072090 con 0x7fd6d40ce590 2024-04-25T10:20:08.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d9a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d4071230 con 0x7fd6d4154ee0 2024-04-25T10:20:08.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd6d40ce590 0x7fd6d406d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:08.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd6d40ce590 0x7fd6d406d6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47804/0 (socket says 172.21.15.165:47804) 2024-04-25T10:20:08.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 -- 172.21.15.165:0/3948207033 learned_addr learned my addr 172.21.15.165:0/3948207033 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:08.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d2a00640 1 --2- 172.21.15.165:0/3948207033 >> v2:172.21.15.172:3300/0 conn(0x7fd6d4154ee0 0x7fd6d40cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:08.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 -- 172.21.15.165:0/3948207033 >> v2:172.21.15.172:3300/0 conn(0x7fd6d4154ee0 msgr2=0x7fd6d40cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:08.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 --2- 172.21.15.165:0/3948207033 >> v2:172.21.15.172:3300/0 conn(0x7fd6d4154ee0 0x7fd6d40cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 -- 172.21.15.165:0/3948207033 >> v1:172.21.15.165:6789/0 conn(0x7fd6d40cfcf0 legacy=0x7fd6d40cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.413+0000 7fd6d3400640 1 -- 172.21.15.165:0/3948207033 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d4071760 con 0x7fd6d40ce590 2024-04-25T10:20:08.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.415+0000 7fd6d3400640 1 --2- 172.21.15.165:0/3948207033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d406d6f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40d02e0 tx=0x7fd6c402c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1834967e6836968c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:08.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.415+0000 7fd6d1600640 1 -- 172.21.15.165:0/3948207033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4038070 con 0x7fd6d40ce590 2024-04-25T10:20:08.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.415+0000 7fd6d1600640 1 -- 172.21.15.165:0/3948207033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c4031710 con 0x7fd6d40ce590 2024-04-25T10:20:08.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.415+0000 7fd6d1600640 1 -- 172.21.15.165:0/3948207033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c4031ae0 con 0x7fd6d40ce590 2024-04-25T10:20:08.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 -- 172.21.15.165:0/3948207033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 msgr2=0x7fd6d406d6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/3948207033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d406d6f0 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7fd6d40d02e0 tx=0x7fd6c402c480 comp rx=0 tx=0).stop 2024-04-25T10:20:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 -- 172.21.15.165:0/3948207033 shutdown_connections 2024-04-25T10:20:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/3948207033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d406d6f0 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/3948207033 >> v2:172.21.15.172:3300/0 conn(0x7fd6d4154ee0 0x7fd6d40cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 -- 172.21.15.165:0/3948207033 >> 172.21.15.165:0/3948207033 conn(0x7fd6d406abe0 msgr2=0x7fd6d406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 -- 172.21.15.165:0/3948207033 shutdown_connections 2024-04-25T10:20:08.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.416+0000 7fd6d9a00640 1 -- 172.21.15.165:0/3948207033 wait complete. 2024-04-25T10:20:08.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.418+0000 7fd6d9a00640 1 Processor -- start 2024-04-25T10:20:08.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.418+0000 7fd6d9a00640 1 -- start start 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d9a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d40cca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d9a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6d40cfcf0 0x7fd6d41eda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d9a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4154ee0 0x7fd6d415d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d9a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d41552c0 con 0x7fd6d40ce590 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d9a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4071d20 con 0x7fd6d4154ee0 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d9a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d4155440 con 0x7fd6d40cfcf0 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d40cca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d40cca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47816/0 (socket says 172.21.15.165:47816) 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 -- 172.21.15.165:0/2567096200 learned_addr learned my addr 172.21.15.165:0/2567096200 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d3400640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4154ee0 0x7fd6d415d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:08.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 -- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6d40cfcf0 msgr2=0x7fd6d41eda70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6d40cfcf0 0x7fd6d41eda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 -- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4154ee0 msgr2=0x7fd6d415d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4154ee0 0x7fd6d415d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.419+0000 7fd6d2a00640 1 -- 172.21.15.165:0/2567096200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d41f71f0 con 0x7fd6d40ce590 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.420+0000 7fd6d2a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d40cca70 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc007e70 tx=0x7fd6cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.420+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc014620 con 0x7fd6d40ce590 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.420+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6cc019070 con 0x7fd6d40ce590 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.420+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6cc014a00 con 0x7fd6d40ce590 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.420+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d41f74d0 con 0x7fd6d40ce590 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.420+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d41f7a90 con 0x7fd6d40ce590 2024-04-25T10:20:08.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.421+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d40cede0 con 0x7fd6d40ce590 2024-04-25T10:20:08.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.422+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd6cc014ba0 con 0x7fd6d40ce590 2024-04-25T10:20:08.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.423+0000 7fd6caa00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:08.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.423+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd6cc08ac00 con 0x7fd6d40ce590 2024-04-25T10:20:08.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.424+0000 7fd6d2000640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:08.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.424+0000 7fd6d2000640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6b006b4d0 0x7fd6b006d990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc000f60 tx=0x7fd6bc0079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:08.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.428+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd6d40cede0 con 0x7fd6d40ce590 2024-04-25T10:20:08.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.604+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd6d4073990 con 0x7fd6b006b4d0 2024-04-25T10:20:08.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.623+0000 7fd6caa00640 1 -- 172.21.15.165:0/2567096200 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252493 (secure 0 0 0) 0x7fd6d4073990 con 0x7fd6b006b4d0 2024-04-25T10:20:08.629 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:08.630 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6b006b4d0 msgr2=0x7fd6b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6b006b4d0 0x7fd6b006d990 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc000f60 tx=0x7fd6bc0079d0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 msgr2=0x7fd6d40cca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d40cca70 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7fd6cc007e70 tx=0x7fd6cc00e4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 shutdown_connections 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd6b006b4d0 0x7fd6b006d990 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd6d4154ee0 0x7fd6d415d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6d40cfcf0 0x7fd6d41eda70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 --2- 172.21.15.165:0/2567096200 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd6d40ce590 0x7fd6d40cca70 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:08.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.629+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 >> 172.21.15.165:0/2567096200 conn(0x7fd6d406abe0 msgr2=0x7fd6d40b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:08.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.630+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 shutdown_connections 2024-04-25T10:20:08.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:08.630+0000 7fd6d9a00640 1 -- 172.21.15.165:0/2567096200 wait complete. 2024-04-25T10:20:08.651 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":725,"stamp":"2024-04-25T10:20:08.499261+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536556,"kb_used_data":19368,"kb_used_omap":12,"kb_used_meta":517043,"kb_avail":754438164,"statfs":{"total":773094113280,"available":772544679936,"internally_reserved":0,"allocated":19832832,"data_stored":9932388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":529452594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.192943"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413484+0000","last_change":"2024-04-25T10:20:02.358152+0000","last_active":"2024-04-25T10:20:03.413484+0000","last_peered":"2024-04-25T10:20:03.413484+0000","last_clean":"2024-04-25T10:20:03.413484+0000","last_became_active":"2024-04-25T10:20:02.357325+0000","last_became_peered":"2024-04-25T10:20:02.357325+0000","last_unstale":"2024-04-25T10:20:03.413484+0000","last_undegraded":"2024-04-25T10:20:03.413484+0000","last_fullsized":"2024-04-25T10:20:03.413484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:09.615819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329479+0000","last_change":"2024-04-25T10:20:02.361672+0000","last_active":"2024-04-25T10:20:03.329479+0000","last_peered":"2024-04-25T10:20:03.329479+0000","last_clean":"2024-04-25T10:20:03.329479+0000","last_became_active":"2024-04-25T10:20:02.361315+0000","last_became_peered":"2024-04-25T10:20:02.361315+0000","last_unstale":"2024-04-25T10:20:03.329479+0000","last_undegraded":"2024-04-25T10:20:03.329479+0000","last_fullsized":"2024-04-25T10:20:03.329479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:38.899350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372016+0000","last_change":"2024-04-25T10:20:02.371729+0000","last_active":"2024-04-25T10:20:03.372016+0000","last_peered":"2024-04-25T10:20:03.372016+0000","last_clean":"2024-04-25T10:20:03.372016+0000","last_became_active":"2024-04-25T10:20:02.371437+0000","last_became_peered":"2024-04-25T10:20:02.371437+0000","last_unstale":"2024-04-25T10:20:03.372016+0000","last_undegraded":"2024-04-25T10:20:03.372016+0000","last_fullsized":"2024-04-25T10:20:03.372016+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:14.409869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393346+0000","last_change":"2024-04-25T10:20:02.358692+0000","last_active":"2024-04-25T10:20:03.393346+0000","last_peered":"2024-04-25T10:20:03.393346+0000","last_clean":"2024-04-25T10:20:03.393346+0000","last_became_active":"2024-04-25T10:20:02.358409+0000","last_became_peered":"2024-04-25T10:20:02.358409+0000","last_unstale":"2024-04-25T10:20:03.393346+0000","last_undegraded":"2024-04-25T10:20:03.393346+0000","last_fullsized":"2024-04-25T10:20:03.393346+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:29.176224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692013+0000","last_change":"2024-04-25T10:20:02.340677+0000","last_active":"2024-04-25T10:20:03.692013+0000","last_peered":"2024-04-25T10:20:03.692013+0000","last_clean":"2024-04-25T10:20:03.692013+0000","last_became_active":"2024-04-25T10:20:02.339272+0000","last_became_peered":"2024-04-25T10:20:02.339272+0000","last_unstale":"2024-04-25T10:20:03.692013+0000","last_undegraded":"2024-04-25T10:20:03.692013+0000","last_fullsized":"2024-04-25T10:20:03.692013+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:48.284046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392723+0000","last_change":"2024-04-25T10:20:02.359749+0000","last_active":"2024-04-25T10:20:03.392723+0000","last_peered":"2024-04-25T10:20:03.392723+0000","last_clean":"2024-04-25T10:20:03.392723+0000","last_became_active":"2024-04-25T10:20:02.359441+0000","last_became_peered":"2024-04-25T10:20:02.359441+0000","last_unstale":"2024-04-25T10:20:03.392723+0000","last_undegraded":"2024-04-25T10:20:03.392723+0000","last_fullsized":"2024-04-25T10:20:03.392723+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:09.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.691731+0000","last_change":"2024-04-25T10:20:02.341866+0000","last_active":"2024-04-25T10:20:03.691731+0000","last_peered":"2024-04-25T10:20:03.691731+0000","last_clean":"2024-04-25T10:20:03.691731+0000","last_became_active":"2024-04-25T10:20:02.340905+0000","last_became_peered":"2024-04-25T10:20:02.340905+0000","last_unstale":"2024-04-25T10:20:03.691731+0000","last_undegraded":"2024-04-25T10:20:03.691731+0000","last_fullsized":"2024-04-25T10:20:03.691731+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:00.390890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.333608+0000","last_change":"2024-04-25T10:20:02.363273+0000","last_active":"2024-04-25T10:20:03.333608+0000","last_peered":"2024-04-25T10:20:03.333608+0000","last_clean":"2024-04-25T10:20:03.333608+0000","last_became_active":"2024-04-25T10:20:02.361340+0000","last_became_peered":"2024-04-25T10:20:02.361340+0000","last_unstale":"2024-04-25T10:20:03.333608+0000","last_undegraded":"2024-04-25T10:20:03.333608+0000","last_fullsized":"2024-04-25T10:20:03.333608+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:28.503164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588151+0000","last_change":"2024-04-25T10:20:02.367909+0000","last_active":"2024-04-25T10:20:03.588151+0000","last_peered":"2024-04-25T10:20:03.588151+0000","last_clean":"2024-04-25T10:20:03.588151+0000","last_became_active":"2024-04-25T10:20:02.367241+0000","last_became_peered":"2024-04-25T10:20:02.367241+0000","last_unstale":"2024-04-25T10:20:03.588151+0000","last_undegraded":"2024-04-25T10:20:03.588151+0000","last_fullsized":"2024-04-25T10:20:03.588151+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:40.310175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335988+0000","last_change":"2024-04-25T10:20:02.362899+0000","last_active":"2024-04-25T10:20:03.335988+0000","last_peered":"2024-04-25T10:20:03.335988+0000","last_clean":"2024-04-25T10:20:03.335988+0000","last_became_active":"2024-04-25T10:20:02.362609+0000","last_became_peered":"2024-04-25T10:20:02.362609+0000","last_unstale":"2024-04-25T10:20:03.335988+0000","last_undegraded":"2024-04-25T10:20:03.335988+0000","last_fullsized":"2024-04-25T10:20:03.335988+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.361669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371888+0000","last_change":"2024-04-25T10:20:02.368036+0000","last_active":"2024-04-25T10:20:03.371888+0000","last_peered":"2024-04-25T10:20:03.371888+0000","last_clean":"2024-04-25T10:20:03.371888+0000","last_became_active":"2024-04-25T10:20:02.367531+0000","last_became_peered":"2024-04-25T10:20:02.367531+0000","last_unstale":"2024-04-25T10:20:03.371888+0000","last_undegraded":"2024-04-25T10:20:03.371888+0000","last_fullsized":"2024-04-25T10:20:03.371888+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:19.407813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692218+0000","last_change":"2024-04-25T10:20:02.357465+0000","last_active":"2024-04-25T10:20:03.692218+0000","last_peered":"2024-04-25T10:20:03.692218+0000","last_clean":"2024-04-25T10:20:03.692218+0000","last_became_active":"2024-04-25T10:20:02.357177+0000","last_became_peered":"2024-04-25T10:20:02.357177+0000","last_unstale":"2024-04-25T10:20:03.692218+0000","last_undegraded":"2024-04-25T10:20:03.692218+0000","last_fullsized":"2024-04-25T10:20:03.692218+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:31.356936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393042+0000","last_change":"2024-04-25T10:20:02.367756+0000","last_active":"2024-04-25T10:20:03.393042+0000","last_peered":"2024-04-25T10:20:03.393042+0000","last_clean":"2024-04-25T10:20:03.393042+0000","last_became_active":"2024-04-25T10:20:02.367208+0000","last_became_peered":"2024-04-25T10:20:02.367208+0000","last_unstale":"2024-04-25T10:20:03.393042+0000","last_undegraded":"2024-04-25T10:20:03.393042+0000","last_fullsized":"2024-04-25T10:20:03.393042+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:46.609706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335021+0000","last_change":"2024-04-25T10:20:02.350419+0000","last_active":"2024-04-25T10:20:03.335021+0000","last_peered":"2024-04-25T10:20:03.335021+0000","last_clean":"2024-04-25T10:20:03.335021+0000","last_became_active":"2024-04-25T10:20:02.350117+0000","last_became_peered":"2024-04-25T10:20:02.350117+0000","last_unstale":"2024-04-25T10:20:03.335021+0000","last_undegraded":"2024-04-25T10:20:03.335021+0000","last_fullsized":"2024-04-25T10:20:03.335021+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.209567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334551+0000","last_change":"2024-04-25T10:20:02.358059+0000","last_active":"2024-04-25T10:20:03.334551+0000","last_peered":"2024-04-25T10:20:03.334551+0000","last_clean":"2024-04-25T10:20:03.334551+0000","last_became_active":"2024-04-25T10:20:02.357465+0000","last_became_peered":"2024-04-25T10:20:02.357465+0000","last_unstale":"2024-04-25T10:20:03.334551+0000","last_undegraded":"2024-04-25T10:20:03.334551+0000","last_fullsized":"2024-04-25T10:20:03.334551+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:27.324281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371544+0000","last_change":"2024-04-25T10:20:02.362462+0000","last_active":"2024-04-25T10:20:03.371544+0000","last_peered":"2024-04-25T10:20:03.371544+0000","last_clean":"2024-04-25T10:20:03.371544+0000","last_became_active":"2024-04-25T10:20:02.362143+0000","last_became_peered":"2024-04-25T10:20:02.362143+0000","last_unstale":"2024-04-25T10:20:03.371544+0000","last_undegraded":"2024-04-25T10:20:03.371544+0000","last_fullsized":"2024-04-25T10:20:03.371544+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:47.608253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588306+0000","last_change":"2024-04-25T10:20:02.364732+0000","last_active":"2024-04-25T10:20:03.588306+0000","last_peered":"2024-04-25T10:20:03.588306+0000","last_clean":"2024-04-25T10:20:03.588306+0000","last_became_active":"2024-04-25T10:20:02.363520+0000","last_became_peered":"2024-04-25T10:20:02.363520+0000","last_unstale":"2024-04-25T10:20:03.588306+0000","last_undegraded":"2024-04-25T10:20:03.588306+0000","last_fullsized":"2024-04-25T10:20:03.588306+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:18.802796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588197+0000","last_change":"2024-04-25T10:20:02.370945+0000","last_active":"2024-04-25T10:20:03.588197+0000","last_peered":"2024-04-25T10:20:03.588197+0000","last_clean":"2024-04-25T10:20:03.588197+0000","last_became_active":"2024-04-25T10:20:02.370652+0000","last_became_peered":"2024-04-25T10:20:02.370652+0000","last_unstale":"2024-04-25T10:20:03.588197+0000","last_undegraded":"2024-04-25T10:20:03.588197+0000","last_fullsized":"2024-04-25T10:20:03.588197+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:02.464323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336410+0000","last_change":"2024-04-25T10:20:02.353518+0000","last_active":"2024-04-25T10:20:03.336410+0000","last_peered":"2024-04-25T10:20:03.336410+0000","last_clean":"2024-04-25T10:20:03.336410+0000","last_became_active":"2024-04-25T10:20:02.352415+0000","last_became_peered":"2024-04-25T10:20:02.352415+0000","last_unstale":"2024-04-25T10:20:03.336410+0000","last_undegraded":"2024-04-25T10:20:03.336410+0000","last_fullsized":"2024-04-25T10:20:03.336410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.161184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336751+0000","last_change":"2024-04-25T10:20:02.349464+0000","last_active":"2024-04-25T10:20:03.336751+0000","last_peered":"2024-04-25T10:20:03.336751+0000","last_clean":"2024-04-25T10:20:03.336751+0000","last_became_active":"2024-04-25T10:20:02.349133+0000","last_became_peered":"2024-04-25T10:20:02.349133+0000","last_unstale":"2024-04-25T10:20:03.336751+0000","last_undegraded":"2024-04-25T10:20:03.336751+0000","last_fullsized":"2024-04-25T10:20:03.336751+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:33.201607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335623+0000","last_change":"2024-04-25T10:20:02.356526+0000","last_active":"2024-04-25T10:20:03.335623+0000","last_peered":"2024-04-25T10:20:03.335623+0000","last_clean":"2024-04-25T10:20:03.335623+0000","last_became_active":"2024-04-25T10:20:02.355119+0000","last_became_peered":"2024-04-25T10:20:02.355119+0000","last_unstale":"2024-04-25T10:20:03.335623+0000","last_undegraded":"2024-04-25T10:20:03.335623+0000","last_fullsized":"2024-04-25T10:20:03.335623+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:54.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335139+0000","last_change":"2024-04-25T10:20:02.350034+0000","last_active":"2024-04-25T10:20:03.335139+0000","last_peered":"2024-04-25T10:20:03.335139+0000","last_clean":"2024-04-25T10:20:03.335139+0000","last_became_active":"2024-04-25T10:20:02.349692+0000","last_became_peered":"2024-04-25T10:20:02.349692+0000","last_unstale":"2024-04-25T10:20:03.335139+0000","last_undegraded":"2024-04-25T10:20:03.335139+0000","last_fullsized":"2024-04-25T10:20:03.335139+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328678+0000","last_change":"2024-04-25T10:20:02.367895+0000","last_active":"2024-04-25T10:20:03.328678+0000","last_peered":"2024-04-25T10:20:03.328678+0000","last_clean":"2024-04-25T10:20:03.328678+0000","last_became_active":"2024-04-25T10:20:02.366828+0000","last_became_peered":"2024-04-25T10:20:02.366828+0000","last_unstale":"2024-04-25T10:20:03.328678+0000","last_undegraded":"2024-04-25T10:20:03.328678+0000","last_fullsized":"2024-04-25T10:20:03.328678+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:53.422192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692026+0000","last_change":"2024-04-25T10:20:02.359741+0000","last_active":"2024-04-25T10:20:03.692026+0000","last_peered":"2024-04-25T10:20:03.692026+0000","last_clean":"2024-04-25T10:20:03.692026+0000","last_became_active":"2024-04-25T10:20:02.359423+0000","last_became_peered":"2024-04-25T10:20:02.359423+0000","last_unstale":"2024-04-25T10:20:03.692026+0000","last_undegraded":"2024-04-25T10:20:03.692026+0000","last_fullsized":"2024-04-25T10:20:03.692026+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:49.282643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588616+0000","last_change":"2024-04-25T10:20:02.364981+0000","last_active":"2024-04-25T10:20:03.588616+0000","last_peered":"2024-04-25T10:20:03.588616+0000","last_clean":"2024-04-25T10:20:03.588616+0000","last_became_active":"2024-04-25T10:20:02.364434+0000","last_became_peered":"2024-04-25T10:20:02.364434+0000","last_unstale":"2024-04-25T10:20:03.588616+0000","last_undegraded":"2024-04-25T10:20:03.588616+0000","last_fullsized":"2024-04-25T10:20:03.588616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.020977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413181+0000","last_change":"2024-04-25T10:20:02.353985+0000","last_active":"2024-04-25T10:20:03.413181+0000","last_peered":"2024-04-25T10:20:03.413181+0000","last_clean":"2024-04-25T10:20:03.413181+0000","last_became_active":"2024-04-25T10:20:02.353555+0000","last_became_peered":"2024-04-25T10:20:02.353555+0000","last_unstale":"2024-04-25T10:20:03.413181+0000","last_undegraded":"2024-04-25T10:20:03.413181+0000","last_fullsized":"2024-04-25T10:20:03.413181+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:47.800009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372065+0000","last_change":"2024-04-25T10:20:02.367801+0000","last_active":"2024-04-25T10:20:03.372065+0000","last_peered":"2024-04-25T10:20:03.372065+0000","last_clean":"2024-04-25T10:20:03.372065+0000","last_became_active":"2024-04-25T10:20:02.367167+0000","last_became_peered":"2024-04-25T10:20:02.367167+0000","last_unstale":"2024-04-25T10:20:03.372065+0000","last_undegraded":"2024-04-25T10:20:03.372065+0000","last_fullsized":"2024-04-25T10:20:03.372065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.169728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692509+0000","last_change":"2024-04-25T10:20:02.353780+0000","last_active":"2024-04-25T10:20:03.692509+0000","last_peered":"2024-04-25T10:20:03.692509+0000","last_clean":"2024-04-25T10:20:03.692509+0000","last_became_active":"2024-04-25T10:20:02.353480+0000","last_became_peered":"2024-04-25T10:20:02.353480+0000","last_unstale":"2024-04-25T10:20:03.692509+0000","last_undegraded":"2024-04-25T10:20:03.692509+0000","last_fullsized":"2024-04-25T10:20:03.692509+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:48.767601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587683+0000","last_change":"2024-04-25T10:20:02.362222+0000","last_active":"2024-04-25T10:20:03.587683+0000","last_peered":"2024-04-25T10:20:03.587683+0000","last_clean":"2024-04-25T10:20:03.587683+0000","last_became_active":"2024-04-25T10:20:02.361935+0000","last_became_peered":"2024-04-25T10:20:02.361935+0000","last_unstale":"2024-04-25T10:20:03.587683+0000","last_undegraded":"2024-04-25T10:20:03.587683+0000","last_fullsized":"2024-04-25T10:20:03.587683+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:16.523314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.333550+0000","last_change":"2024-04-25T10:20:02.358637+0000","last_active":"2024-04-25T10:20:03.333550+0000","last_peered":"2024-04-25T10:20:03.333550+0000","last_clean":"2024-04-25T10:20:03.333550+0000","last_became_active":"2024-04-25T10:20:02.355276+0000","last_became_peered":"2024-04-25T10:20:02.355276+0000","last_unstale":"2024-04-25T10:20:03.333550+0000","last_undegraded":"2024-04-25T10:20:03.333550+0000","last_fullsized":"2024-04-25T10:20:03.333550+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.161184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392968+0000","last_change":"2024-04-25T10:20:02.366085+0000","last_active":"2024-04-25T10:20:03.392968+0000","last_peered":"2024-04-25T10:20:03.392968+0000","last_clean":"2024-04-25T10:20:03.392968+0000","last_became_active":"2024-04-25T10:20:02.365739+0000","last_became_peered":"2024-04-25T10:20:02.365739+0000","last_unstale":"2024-04-25T10:20:03.392968+0000","last_undegraded":"2024-04-25T10:20:03.392968+0000","last_fullsized":"2024-04-25T10:20:03.392968+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:32.077900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392708+0000","last_change":"2024-04-25T10:20:02.354880+0000","last_active":"2024-04-25T10:20:03.392708+0000","last_peered":"2024-04-25T10:20:03.392708+0000","last_clean":"2024-04-25T10:20:03.392708+0000","last_became_active":"2024-04-25T10:20:02.354592+0000","last_became_peered":"2024-04-25T10:20:02.354592+0000","last_unstale":"2024-04-25T10:20:03.392708+0000","last_undegraded":"2024-04-25T10:20:03.392708+0000","last_fullsized":"2024-04-25T10:20:03.392708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:14.409869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328727+0000","last_change":"2024-04-25T10:20:02.372978+0000","last_active":"2024-04-25T10:20:03.328727+0000","last_peered":"2024-04-25T10:20:03.328727+0000","last_clean":"2024-04-25T10:20:03.328727+0000","last_became_active":"2024-04-25T10:20:02.368461+0000","last_became_peered":"2024-04-25T10:20:02.368461+0000","last_unstale":"2024-04-25T10:20:03.328727+0000","last_undegraded":"2024-04-25T10:20:03.328727+0000","last_fullsized":"2024-04-25T10:20:03.328727+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.020977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"183'4","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334928+0000","last_change":"2024-04-25T10:20:01.332782+0000","last_active":"2024-04-25T10:20:03.334928+0000","last_peered":"2024-04-25T10:20:03.334928+0000","last_clean":"2024-04-25T10:20:03.334928+0000","last_became_active":"2024-04-25T10:20:01.332481+0000","last_became_peered":"2024-04-25T10:20:01.332481+0000","last_unstale":"2024-04-25T10:20:03.334928+0000","last_undegraded":"2024-04-25T10:20:03.334928+0000","last_fullsized":"2024-04-25T10:20:03.334928+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:54.192098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412404+0000","last_change":"2024-04-25T10:20:02.362911+0000","last_active":"2024-04-25T10:20:03.412404+0000","last_peered":"2024-04-25T10:20:03.412404+0000","last_clean":"2024-04-25T10:20:03.412404+0000","last_became_active":"2024-04-25T10:20:02.361867+0000","last_became_peered":"2024-04-25T10:20:02.361867+0000","last_unstale":"2024-04-25T10:20:03.412404+0000","last_undegraded":"2024-04-25T10:20:03.412404+0000","last_fullsized":"2024-04-25T10:20:03.412404+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:36.018496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413313+0000","last_change":"2024-04-25T10:20:01.335869+0000","last_active":"2024-04-25T10:20:03.413313+0000","last_peered":"2024-04-25T10:20:03.413313+0000","last_clean":"2024-04-25T10:20:03.413313+0000","last_became_active":"2024-04-25T10:20:01.335113+0000","last_became_peered":"2024-04-25T10:20:01.335113+0000","last_unstale":"2024-04-25T10:20:03.413313+0000","last_undegraded":"2024-04-25T10:20:03.413313+0000","last_fullsized":"2024-04-25T10:20:03.413313+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:59.188576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335420+0000","last_change":"2024-04-25T10:20:02.343477+0000","last_active":"2024-04-25T10:20:03.335420+0000","last_peered":"2024-04-25T10:20:03.335420+0000","last_clean":"2024-04-25T10:20:03.335420+0000","last_became_active":"2024-04-25T10:20:02.342712+0000","last_became_peered":"2024-04-25T10:20:02.342712+0000","last_unstale":"2024-04-25T10:20:03.335420+0000","last_undegraded":"2024-04-25T10:20:03.335420+0000","last_fullsized":"2024-04-25T10:20:03.335420+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:57.523892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329123+0000","last_change":"2024-04-25T10:20:01.334447+0000","last_active":"2024-04-25T10:20:03.329123+0000","last_peered":"2024-04-25T10:20:03.329123+0000","last_clean":"2024-04-25T10:20:03.329123+0000","last_became_active":"2024-04-25T10:20:01.334157+0000","last_became_peered":"2024-04-25T10:20:01.334157+0000","last_unstale":"2024-04-25T10:20:03.329123+0000","last_undegraded":"2024-04-25T10:20:03.329123+0000","last_fullsized":"2024-04-25T10:20:03.329123+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.603090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692441+0000","last_change":"2024-04-25T10:20:02.360457+0000","last_active":"2024-04-25T10:20:03.692441+0000","last_peered":"2024-04-25T10:20:03.692441+0000","last_clean":"2024-04-25T10:20:03.692441+0000","last_became_active":"2024-04-25T10:20:02.360136+0000","last_became_peered":"2024-04-25T10:20:02.360136+0000","last_unstale":"2024-04-25T10:20:03.692441+0000","last_undegraded":"2024-04-25T10:20:03.692441+0000","last_fullsized":"2024-04-25T10:20:03.692441+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:55.647660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588180+0000","last_change":"2024-04-25T10:20:01.327488+0000","last_active":"2024-04-25T10:20:03.588180+0000","last_peered":"2024-04-25T10:20:03.588180+0000","last_clean":"2024-04-25T10:20:03.588180+0000","last_became_active":"2024-04-25T10:20:01.327214+0000","last_became_peered":"2024-04-25T10:20:01.327214+0000","last_unstale":"2024-04-25T10:20:03.588180+0000","last_undegraded":"2024-04-25T10:20:03.588180+0000","last_fullsized":"2024-04-25T10:20:03.588180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:23.238260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334014+0000","last_change":"2024-04-25T10:20:02.360025+0000","last_active":"2024-04-25T10:20:03.334014+0000","last_peered":"2024-04-25T10:20:03.334014+0000","last_clean":"2024-04-25T10:20:03.334014+0000","last_became_active":"2024-04-25T10:20:02.359709+0000","last_became_peered":"2024-04-25T10:20:02.359709+0000","last_unstale":"2024-04-25T10:20:03.334014+0000","last_undegraded":"2024-04-25T10:20:03.334014+0000","last_fullsized":"2024-04-25T10:20:03.334014+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:33.201607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393119+0000","last_change":"2024-04-25T10:20:01.341558+0000","last_active":"2024-04-25T10:20:03.393119+0000","last_peered":"2024-04-25T10:20:03.393119+0000","last_clean":"2024-04-25T10:20:03.393119+0000","last_became_active":"2024-04-25T10:20:01.340467+0000","last_became_peered":"2024-04-25T10:20:01.340467+0000","last_unstale":"2024-04-25T10:20:03.393119+0000","last_undegraded":"2024-04-25T10:20:03.393119+0000","last_fullsized":"2024-04-25T10:20:03.393119+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:14.878826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334529+0000","last_change":"2024-04-25T10:20:02.345657+0000","last_active":"2024-04-25T10:20:03.334529+0000","last_peered":"2024-04-25T10:20:03.334529+0000","last_clean":"2024-04-25T10:20:03.334529+0000","last_became_active":"2024-04-25T10:20:02.345353+0000","last_became_peered":"2024-04-25T10:20:02.345353+0000","last_unstale":"2024-04-25T10:20:03.334529+0000","last_undegraded":"2024-04-25T10:20:03.334529+0000","last_fullsized":"2024-04-25T10:20:03.334529+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:11.831418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392504+0000","last_change":"2024-04-25T10:20:01.342761+0000","last_active":"2024-04-25T10:20:03.392504+0000","last_peered":"2024-04-25T10:20:03.392504+0000","last_clean":"2024-04-25T10:20:03.392504+0000","last_became_active":"2024-04-25T10:20:01.342145+0000","last_became_peered":"2024-04-25T10:20:01.342145+0000","last_unstale":"2024-04-25T10:20:03.392504+0000","last_undegraded":"2024-04-25T10:20:03.392504+0000","last_fullsized":"2024-04-25T10:20:03.392504+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588718+0000","last_change":"2024-04-25T10:20:02.362124+0000","last_active":"2024-04-25T10:20:03.588718+0000","last_peered":"2024-04-25T10:20:03.588718+0000","last_clean":"2024-04-25T10:20:03.588718+0000","last_became_active":"2024-04-25T10:20:02.357833+0000","last_became_peered":"2024-04-25T10:20:02.357833+0000","last_unstale":"2024-04-25T10:20:03.588718+0000","last_undegraded":"2024-04-25T10:20:03.588718+0000","last_fullsized":"2024-04-25T10:20:03.588718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.734006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371516+0000","last_change":"2024-04-25T10:20:01.338974+0000","last_active":"2024-04-25T10:20:03.371516+0000","last_peered":"2024-04-25T10:20:03.371516+0000","last_clean":"2024-04-25T10:20:03.371516+0000","last_became_active":"2024-04-25T10:20:01.338631+0000","last_became_peered":"2024-04-25T10:20:01.338631+0000","last_unstale":"2024-04-25T10:20:03.371516+0000","last_undegraded":"2024-04-25T10:20:03.371516+0000","last_fullsized":"2024-04-25T10:20:03.371516+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:30.668056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.333363+0000","last_change":"2024-04-25T10:20:02.357818+0000","last_active":"2024-04-25T10:20:03.333363+0000","last_peered":"2024-04-25T10:20:03.333363+0000","last_clean":"2024-04-25T10:20:03.333363+0000","last_became_active":"2024-04-25T10:20:02.357502+0000","last_became_peered":"2024-04-25T10:20:02.357502+0000","last_unstale":"2024-04-25T10:20:03.333363+0000","last_undegraded":"2024-04-25T10:20:03.333363+0000","last_fullsized":"2024-04-25T10:20:03.333363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.361669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692952+0000","last_change":"2024-04-25T10:20:01.332708+0000","last_active":"2024-04-25T10:20:03.692952+0000","last_peered":"2024-04-25T10:20:03.692952+0000","last_clean":"2024-04-25T10:20:03.692952+0000","last_became_active":"2024-04-25T10:20:01.332230+0000","last_became_peered":"2024-04-25T10:20:01.332230+0000","last_unstale":"2024-04-25T10:20:03.692952+0000","last_undegraded":"2024-04-25T10:20:03.692952+0000","last_fullsized":"2024-04-25T10:20:03.692952+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:32.682708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371614+0000","last_change":"2024-04-25T10:20:02.369394+0000","last_active":"2024-04-25T10:20:03.371614+0000","last_peered":"2024-04-25T10:20:03.371614+0000","last_clean":"2024-04-25T10:20:03.371614+0000","last_became_active":"2024-04-25T10:20:02.369122+0000","last_became_peered":"2024-04-25T10:20:02.369122+0000","last_unstale":"2024-04-25T10:20:03.371614+0000","last_undegraded":"2024-04-25T10:20:03.371614+0000","last_fullsized":"2024-04-25T10:20:03.371614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:25.084733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393814+0000","last_change":"2024-04-25T10:20:01.345530+0000","last_active":"2024-04-25T10:20:03.393814+0000","last_peered":"2024-04-25T10:20:03.393814+0000","last_clean":"2024-04-25T10:20:03.393814+0000","last_became_active":"2024-04-25T10:20:01.345190+0000","last_became_peered":"2024-04-25T10:20:01.345190+0000","last_unstale":"2024-04-25T10:20:03.393814+0000","last_undegraded":"2024-04-25T10:20:03.393814+0000","last_fullsized":"2024-04-25T10:20:03.393814+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:58.064824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587751+0000","last_change":"2024-04-25T10:20:02.366882+0000","last_active":"2024-04-25T10:20:03.587751+0000","last_peered":"2024-04-25T10:20:03.587751+0000","last_clean":"2024-04-25T10:20:03.587751+0000","last_became_active":"2024-04-25T10:20:02.366604+0000","last_became_peered":"2024-04-25T10:20:02.366604+0000","last_unstale":"2024-04-25T10:20:03.587751+0000","last_undegraded":"2024-04-25T10:20:03.587751+0000","last_fullsized":"2024-04-25T10:20:03.587751+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:46.930941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"183'3","reported_seq":17,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588094+0000","last_change":"2024-04-25T10:20:01.329959+0000","last_active":"2024-04-25T10:20:03.588094+0000","last_peered":"2024-04-25T10:20:03.588094+0000","last_clean":"2024-04-25T10:20:03.588094+0000","last_became_active":"2024-04-25T10:20:01.329609+0000","last_became_peered":"2024-04-25T10:20:01.329609+0000","last_unstale":"2024-04-25T10:20:03.588094+0000","last_undegraded":"2024-04-25T10:20:03.588094+0000","last_fullsized":"2024-04-25T10:20:03.588094+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.385948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692471+0000","last_change":"2024-04-25T10:20:02.352991+0000","last_active":"2024-04-25T10:20:03.692471+0000","last_peered":"2024-04-25T10:20:03.692471+0000","last_clean":"2024-04-25T10:20:03.692471+0000","last_became_active":"2024-04-25T10:20:02.352639+0000","last_became_peered":"2024-04-25T10:20:02.352639+0000","last_unstale":"2024-04-25T10:20:03.692471+0000","last_undegraded":"2024-04-25T10:20:03.692471+0000","last_fullsized":"2024-04-25T10:20:03.692471+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:48.990837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"183'6","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393131+0000","last_change":"2024-04-25T10:20:01.343264+0000","last_active":"2024-04-25T10:20:03.393131+0000","last_peered":"2024-04-25T10:20:03.393131+0000","last_clean":"2024-04-25T10:20:03.393131+0000","last_became_active":"2024-04-25T10:20:01.342715+0000","last_became_peered":"2024-04-25T10:20:01.342715+0000","last_unstale":"2024-04-25T10:20:03.393131+0000","last_undegraded":"2024-04-25T10:20:03.393131+0000","last_fullsized":"2024-04-25T10:20:03.393131+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.371453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329451+0000","last_change":"2024-04-25T10:20:02.371420+0000","last_active":"2024-04-25T10:20:03.329451+0000","last_peered":"2024-04-25T10:20:03.329451+0000","last_clean":"2024-04-25T10:20:03.329451+0000","last_became_active":"2024-04-25T10:20:02.371140+0000","last_became_peered":"2024-04-25T10:20:02.371140+0000","last_unstale":"2024-04-25T10:20:03.329451+0000","last_undegraded":"2024-04-25T10:20:03.329451+0000","last_fullsized":"2024-04-25T10:20:03.329451+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.734006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328978+0000","last_change":"2024-04-25T10:20:01.337329+0000","last_active":"2024-04-25T10:20:03.328978+0000","last_peered":"2024-04-25T10:20:03.328978+0000","last_clean":"2024-04-25T10:20:03.328978+0000","last_became_active":"2024-04-25T10:20:01.336415+0000","last_became_peered":"2024-04-25T10:20:01.336415+0000","last_unstale":"2024-04-25T10:20:03.328978+0000","last_undegraded":"2024-04-25T10:20:03.328978+0000","last_fullsized":"2024-04-25T10:20:03.328978+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:41.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372221+0000","last_change":"2024-04-25T10:20:02.375970+0000","last_active":"2024-04-25T10:20:03.372221+0000","last_peered":"2024-04-25T10:20:03.372221+0000","last_clean":"2024-04-25T10:20:03.372221+0000","last_became_active":"2024-04-25T10:20:02.375664+0000","last_became_peered":"2024-04-25T10:20:02.375664+0000","last_unstale":"2024-04-25T10:20:03.372221+0000","last_undegraded":"2024-04-25T10:20:03.372221+0000","last_fullsized":"2024-04-25T10:20:03.372221+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:32.077900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371487+0000","last_change":"2024-04-25T10:20:01.330029+0000","last_active":"2024-04-25T10:20:03.371487+0000","last_peered":"2024-04-25T10:20:03.371487+0000","last_clean":"2024-04-25T10:20:03.371487+0000","last_became_active":"2024-04-25T10:20:01.328434+0000","last_became_peered":"2024-04-25T10:20:01.328434+0000","last_unstale":"2024-04-25T10:20:03.371487+0000","last_undegraded":"2024-04-25T10:20:03.371487+0000","last_fullsized":"2024-04-25T10:20:03.371487+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588359+0000","last_change":"2024-04-25T10:20:02.371094+0000","last_active":"2024-04-25T10:20:03.588359+0000","last_peered":"2024-04-25T10:20:03.588359+0000","last_clean":"2024-04-25T10:20:03.588359+0000","last_became_active":"2024-04-25T10:20:02.370802+0000","last_became_peered":"2024-04-25T10:20:02.370802+0000","last_unstale":"2024-04-25T10:20:03.588359+0000","last_undegraded":"2024-04-25T10:20:03.588359+0000","last_fullsized":"2024-04-25T10:20:03.588359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:53.422192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587410+0000","last_change":"2024-04-25T10:20:01.332999+0000","last_active":"2024-04-25T10:20:03.587410+0000","last_peered":"2024-04-25T10:20:03.587410+0000","last_clean":"2024-04-25T10:20:03.587410+0000","last_became_active":"2024-04-25T10:20:01.332513+0000","last_became_peered":"2024-04-25T10:20:01.332513+0000","last_unstale":"2024-04-25T10:20:03.587410+0000","last_undegraded":"2024-04-25T10:20:03.587410+0000","last_fullsized":"2024-04-25T10:20:03.587410+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:36.561856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393685+0000","last_change":"2024-04-25T10:20:02.364808+0000","last_active":"2024-04-25T10:20:03.393685+0000","last_peered":"2024-04-25T10:20:03.393685+0000","last_clean":"2024-04-25T10:20:03.393685+0000","last_became_active":"2024-04-25T10:20:02.364082+0000","last_became_peered":"2024-04-25T10:20:02.364082+0000","last_unstale":"2024-04-25T10:20:03.393685+0000","last_undegraded":"2024-04-25T10:20:03.393685+0000","last_fullsized":"2024-04-25T10:20:03.393685+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:49.775129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"183'8","reported_seq":22,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.691543+0000","last_change":"2024-04-25T10:20:01.338593+0000","last_active":"2024-04-25T10:20:03.691543+0000","last_peered":"2024-04-25T10:20:03.691543+0000","last_clean":"2024-04-25T10:20:03.691543+0000","last_became_active":"2024-04-25T10:20:01.338279+0000","last_became_peered":"2024-04-25T10:20:01.338279+0000","last_unstale":"2024-04-25T10:20:03.691543+0000","last_undegraded":"2024-04-25T10:20:03.691543+0000","last_fullsized":"2024-04-25T10:20:03.691543+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.211682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334004+0000","last_change":"2024-04-25T10:20:02.363884+0000","last_active":"2024-04-25T10:20:03.334004+0000","last_peered":"2024-04-25T10:20:03.334004+0000","last_clean":"2024-04-25T10:20:03.334004+0000","last_became_active":"2024-04-25T10:20:02.363593+0000","last_became_peered":"2024-04-25T10:20:02.363593+0000","last_unstale":"2024-04-25T10:20:03.334004+0000","last_undegraded":"2024-04-25T10:20:03.334004+0000","last_fullsized":"2024-04-25T10:20:03.334004+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:41.662935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413459+0000","last_change":"2024-04-25T10:20:01.341364+0000","last_active":"2024-04-25T10:20:03.413459+0000","last_peered":"2024-04-25T10:20:03.413459+0000","last_clean":"2024-04-25T10:20:03.413459+0000","last_became_active":"2024-04-25T10:20:01.340654+0000","last_became_peered":"2024-04-25T10:20:01.340654+0000","last_unstale":"2024-04-25T10:20:03.413459+0000","last_undegraded":"2024-04-25T10:20:03.413459+0000","last_fullsized":"2024-04-25T10:20:03.413459+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:34.211448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413145+0000","last_change":"2024-04-25T10:20:02.363357+0000","last_active":"2024-04-25T10:20:03.413145+0000","last_peered":"2024-04-25T10:20:03.413145+0000","last_clean":"2024-04-25T10:20:03.413145+0000","last_became_active":"2024-04-25T10:20:02.361449+0000","last_became_peered":"2024-04-25T10:20:02.361449+0000","last_unstale":"2024-04-25T10:20:03.413145+0000","last_undegraded":"2024-04-25T10:20:03.413145+0000","last_fullsized":"2024-04-25T10:20:03.413145+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:59.252254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413440+0000","last_change":"2024-04-25T10:20:01.343272+0000","last_active":"2024-04-25T10:20:03.413440+0000","last_peered":"2024-04-25T10:20:03.413440+0000","last_clean":"2024-04-25T10:20:03.413440+0000","last_became_active":"2024-04-25T10:20:01.342389+0000","last_became_peered":"2024-04-25T10:20:01.342389+0000","last_unstale":"2024-04-25T10:20:03.413440+0000","last_undegraded":"2024-04-25T10:20:03.413440+0000","last_fullsized":"2024-04-25T10:20:03.413440+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:32.480221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412982+0000","last_change":"2024-04-25T10:20:02.359471+0000","last_active":"2024-04-25T10:20:03.412982+0000","last_peered":"2024-04-25T10:20:03.412982+0000","last_clean":"2024-04-25T10:20:03.412982+0000","last_became_active":"2024-04-25T10:20:02.356111+0000","last_became_peered":"2024-04-25T10:20:02.356111+0000","last_unstale":"2024-04-25T10:20:03.412982+0000","last_undegraded":"2024-04-25T10:20:03.412982+0000","last_fullsized":"2024-04-25T10:20:03.412982+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:59.082293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329198+0000","last_change":"2024-04-25T10:20:01.340613+0000","last_active":"2024-04-25T10:20:03.329198+0000","last_peered":"2024-04-25T10:20:03.329198+0000","last_clean":"2024-04-25T10:20:03.329198+0000","last_became_active":"2024-04-25T10:20:01.340250+0000","last_became_peered":"2024-04-25T10:20:01.340250+0000","last_unstale":"2024-04-25T10:20:03.329198+0000","last_undegraded":"2024-04-25T10:20:03.329198+0000","last_fullsized":"2024-04-25T10:20:03.329198+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:15.505845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335698+0000","last_change":"2024-04-25T10:20:02.346899+0000","last_active":"2024-04-25T10:20:03.335698+0000","last_peered":"2024-04-25T10:20:03.335698+0000","last_clean":"2024-04-25T10:20:03.335698+0000","last_became_active":"2024-04-25T10:20:02.346568+0000","last_became_peered":"2024-04-25T10:20:02.346568+0000","last_unstale":"2024-04-25T10:20:03.335698+0000","last_undegraded":"2024-04-25T10:20:03.335698+0000","last_fullsized":"2024-04-25T10:20:03.335698+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:28.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692474+0000","last_change":"2024-04-25T10:20:01.343143+0000","last_active":"2024-04-25T10:20:03.692474+0000","last_peered":"2024-04-25T10:20:03.692474+0000","last_clean":"2024-04-25T10:20:03.692474+0000","last_became_active":"2024-04-25T10:20:01.342912+0000","last_became_peered":"2024-04-25T10:20:01.342912+0000","last_unstale":"2024-04-25T10:20:03.692474+0000","last_undegraded":"2024-04-25T10:20:03.692474+0000","last_fullsized":"2024-04-25T10:20:03.692474+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.956577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329256+0000","last_change":"2024-04-25T10:20:02.364467+0000","last_active":"2024-04-25T10:20:03.329256+0000","last_peered":"2024-04-25T10:20:03.329256+0000","last_clean":"2024-04-25T10:20:03.329256+0000","last_became_active":"2024-04-25T10:20:02.359243+0000","last_became_peered":"2024-04-25T10:20:02.359243+0000","last_unstale":"2024-04-25T10:20:03.329256+0000","last_undegraded":"2024-04-25T10:20:03.329256+0000","last_fullsized":"2024-04-25T10:20:03.329256+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:49.582825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692675+0000","last_change":"2024-04-25T10:20:01.333143+0000","last_active":"2024-04-25T10:20:03.692675+0000","last_peered":"2024-04-25T10:20:03.692675+0000","last_clean":"2024-04-25T10:20:03.692675+0000","last_became_active":"2024-04-25T10:20:01.332865+0000","last_became_peered":"2024-04-25T10:20:01.332865+0000","last_unstale":"2024-04-25T10:20:03.692675+0000","last_undegraded":"2024-04-25T10:20:03.692675+0000","last_fullsized":"2024-04-25T10:20:03.692675+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:29.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393546+0000","last_change":"2024-04-25T10:20:02.366377+0000","last_active":"2024-04-25T10:20:03.393546+0000","last_peered":"2024-04-25T10:20:03.393546+0000","last_clean":"2024-04-25T10:20:03.393546+0000","last_became_active":"2024-04-25T10:20:02.366065+0000","last_became_peered":"2024-04-25T10:20:02.366065+0000","last_unstale":"2024-04-25T10:20:03.393546+0000","last_undegraded":"2024-04-25T10:20:03.393546+0000","last_fullsized":"2024-04-25T10:20:03.393546+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:06.008965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328772+0000","last_change":"2024-04-25T10:20:01.332184+0000","last_active":"2024-04-25T10:20:03.328772+0000","last_peered":"2024-04-25T10:20:03.328772+0000","last_clean":"2024-04-25T10:20:03.328772+0000","last_became_active":"2024-04-25T10:20:01.331558+0000","last_became_peered":"2024-04-25T10:20:01.331558+0000","last_unstale":"2024-04-25T10:20:03.328772+0000","last_undegraded":"2024-04-25T10:20:03.328772+0000","last_fullsized":"2024-04-25T10:20:03.328772+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.100718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588637+0000","last_change":"2024-04-25T10:20:02.370318+0000","last_active":"2024-04-25T10:20:03.588637+0000","last_peered":"2024-04-25T10:20:03.588637+0000","last_clean":"2024-04-25T10:20:03.588637+0000","last_became_active":"2024-04-25T10:20:02.358118+0000","last_became_peered":"2024-04-25T10:20:02.358118+0000","last_unstale":"2024-04-25T10:20:03.588637+0000","last_undegraded":"2024-04-25T10:20:03.588637+0000","last_fullsized":"2024-04-25T10:20:03.588637+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:49.582825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334864+0000","last_change":"2024-04-25T10:20:01.333236+0000","last_active":"2024-04-25T10:20:03.334864+0000","last_peered":"2024-04-25T10:20:03.334864+0000","last_clean":"2024-04-25T10:20:03.334864+0000","last_became_active":"2024-04-25T10:20:01.332681+0000","last_became_peered":"2024-04-25T10:20:01.332681+0000","last_unstale":"2024-04-25T10:20:03.334864+0000","last_undegraded":"2024-04-25T10:20:03.334864+0000","last_fullsized":"2024-04-25T10:20:03.334864+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:26.830908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336827+0000","last_change":"2024-04-25T10:20:02.341693+0000","last_active":"2024-04-25T10:20:03.336827+0000","last_peered":"2024-04-25T10:20:03.336827+0000","last_clean":"2024-04-25T10:20:03.336827+0000","last_became_active":"2024-04-25T10:20:02.341396+0000","last_became_peered":"2024-04-25T10:20:02.341396+0000","last_unstale":"2024-04-25T10:20:03.336827+0000","last_undegraded":"2024-04-25T10:20:03.336827+0000","last_fullsized":"2024-04-25T10:20:03.336827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:11.831418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412516+0000","last_change":"2024-04-25T10:20:01.334912+0000","last_active":"2024-04-25T10:20:03.412516+0000","last_peered":"2024-04-25T10:20:03.412516+0000","last_clean":"2024-04-25T10:20:03.412516+0000","last_became_active":"2024-04-25T10:20:01.334608+0000","last_became_peered":"2024-04-25T10:20:01.334608+0000","last_unstale":"2024-04-25T10:20:03.412516+0000","last_undegraded":"2024-04-25T10:20:03.412516+0000","last_fullsized":"2024-04-25T10:20:03.412516+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:19.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334006+0000","last_change":"2024-04-25T10:20:02.356760+0000","last_active":"2024-04-25T10:20:03.334006+0000","last_peered":"2024-04-25T10:20:03.334006+0000","last_clean":"2024-04-25T10:20:03.334006+0000","last_became_active":"2024-04-25T10:20:02.356400+0000","last_became_peered":"2024-04-25T10:20:02.356400+0000","last_unstale":"2024-04-25T10:20:03.334006+0000","last_undegraded":"2024-04-25T10:20:03.334006+0000","last_fullsized":"2024-04-25T10:20:03.334006+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:27.848377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588720+0000","last_change":"2024-04-25T10:20:01.335981+0000","last_active":"2024-04-25T10:20:03.588720+0000","last_peered":"2024-04-25T10:20:03.588720+0000","last_clean":"2024-04-25T10:20:03.588720+0000","last_became_active":"2024-04-25T10:20:01.335703+0000","last_became_peered":"2024-04-25T10:20:01.335703+0000","last_unstale":"2024-04-25T10:20:03.588720+0000","last_undegraded":"2024-04-25T10:20:03.588720+0000","last_fullsized":"2024-04-25T10:20:03.588720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:18.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371392+0000","last_change":"2024-04-25T10:20:02.367475+0000","last_active":"2024-04-25T10:20:03.371392+0000","last_peered":"2024-04-25T10:20:03.371392+0000","last_clean":"2024-04-25T10:20:03.371392+0000","last_became_active":"2024-04-25T10:20:02.367142+0000","last_became_peered":"2024-04-25T10:20:02.367142+0000","last_unstale":"2024-04-25T10:20:03.371392+0000","last_undegraded":"2024-04-25T10:20:03.371392+0000","last_fullsized":"2024-04-25T10:20:03.371392+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:03.489816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"184'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336125+0000","last_change":"2024-04-25T10:20:01.332051+0000","last_active":"2024-04-25T10:20:03.336125+0000","last_peered":"2024-04-25T10:20:03.336125+0000","last_clean":"2024-04-25T10:20:03.336125+0000","last_became_active":"2024-04-25T10:20:01.331777+0000","last_became_peered":"2024-04-25T10:20:01.331777+0000","last_unstale":"2024-04-25T10:20:03.336125+0000","last_undegraded":"2024-04-25T10:20:03.336125+0000","last_fullsized":"2024-04-25T10:20:03.336125+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.465188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587653+0000","last_change":"2024-04-25T10:20:02.372312+0000","last_active":"2024-04-25T10:20:03.587653+0000","last_peered":"2024-04-25T10:20:03.587653+0000","last_clean":"2024-04-25T10:20:03.587653+0000","last_became_active":"2024-04-25T10:20:02.372038+0000","last_became_peered":"2024-04-25T10:20:02.372038+0000","last_unstale":"2024-04-25T10:20:03.587653+0000","last_undegraded":"2024-04-25T10:20:03.587653+0000","last_fullsized":"2024-04-25T10:20:03.587653+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:44.354381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334535+0000","last_change":"2024-04-25T10:20:01.323209+0000","last_active":"2024-04-25T10:20:03.334535+0000","last_peered":"2024-04-25T10:20:03.334535+0000","last_clean":"2024-04-25T10:20:03.334535+0000","last_became_active":"2024-04-25T10:20:01.319553+0000","last_became_peered":"2024-04-25T10:20:01.319553+0000","last_unstale":"2024-04-25T10:20:03.334535+0000","last_undegraded":"2024-04-25T10:20:03.334535+0000","last_fullsized":"2024-04-25T10:20:03.334535+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:40.645466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412928+0000","last_change":"2024-04-25T10:20:02.360953+0000","last_active":"2024-04-25T10:20:03.412928+0000","last_peered":"2024-04-25T10:20:03.412928+0000","last_clean":"2024-04-25T10:20:03.412928+0000","last_became_active":"2024-04-25T10:20:02.358576+0000","last_became_peered":"2024-04-25T10:20:02.358576+0000","last_unstale":"2024-04-25T10:20:03.412928+0000","last_undegraded":"2024-04-25T10:20:03.412928+0000","last_fullsized":"2024-04-25T10:20:03.412928+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:15.896295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412868+0000","last_change":"2024-04-25T10:20:01.336445+0000","last_active":"2024-04-25T10:20:03.412868+0000","last_peered":"2024-04-25T10:20:03.412868+0000","last_clean":"2024-04-25T10:20:03.412868+0000","last_became_active":"2024-04-25T10:20:01.336136+0000","last_became_peered":"2024-04-25T10:20:01.336136+0000","last_unstale":"2024-04-25T10:20:03.412868+0000","last_undegraded":"2024-04-25T10:20:03.412868+0000","last_fullsized":"2024-04-25T10:20:03.412868+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.879308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335837+0000","last_change":"2024-04-25T10:20:02.361100+0000","last_active":"2024-04-25T10:20:03.335837+0000","last_peered":"2024-04-25T10:20:03.335837+0000","last_clean":"2024-04-25T10:20:03.335837+0000","last_became_active":"2024-04-25T10:20:02.360829+0000","last_became_peered":"2024-04-25T10:20:02.360829+0000","last_unstale":"2024-04-25T10:20:03.335837+0000","last_undegraded":"2024-04-25T10:20:03.335837+0000","last_fullsized":"2024-04-25T10:20:03.335837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:45.508642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335916+0000","last_change":"2024-04-25T10:20:01.329676+0000","last_active":"2024-04-25T10:20:03.335916+0000","last_peered":"2024-04-25T10:20:03.335916+0000","last_clean":"2024-04-25T10:20:03.335916+0000","last_became_active":"2024-04-25T10:20:01.329366+0000","last_became_peered":"2024-04-25T10:20:01.329366+0000","last_unstale":"2024-04-25T10:20:03.335916+0000","last_undegraded":"2024-04-25T10:20:03.335916+0000","last_fullsized":"2024-04-25T10:20:03.335916+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:57.834078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393632+0000","last_change":"2024-04-25T10:20:02.369691+0000","last_active":"2024-04-25T10:20:03.393632+0000","last_peered":"2024-04-25T10:20:03.393632+0000","last_clean":"2024-04-25T10:20:03.393632+0000","last_became_active":"2024-04-25T10:20:02.369391+0000","last_became_peered":"2024-04-25T10:20:02.369391+0000","last_unstale":"2024-04-25T10:20:03.393632+0000","last_undegraded":"2024-04-25T10:20:03.393632+0000","last_fullsized":"2024-04-25T10:20:03.393632+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:30.459139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588323+0000","last_change":"2024-04-25T10:20:01.334388+0000","last_active":"2024-04-25T10:20:03.588323+0000","last_peered":"2024-04-25T10:20:03.588323+0000","last_clean":"2024-04-25T10:20:03.588323+0000","last_became_active":"2024-04-25T10:20:01.333900+0000","last_became_peered":"2024-04-25T10:20:01.333900+0000","last_unstale":"2024-04-25T10:20:03.588323+0000","last_undegraded":"2024-04-25T10:20:03.588323+0000","last_fullsized":"2024-04-25T10:20:03.588323+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:06.491126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393760+0000","last_change":"2024-04-25T10:20:02.368901+0000","last_active":"2024-04-25T10:20:03.393760+0000","last_peered":"2024-04-25T10:20:03.393760+0000","last_clean":"2024-04-25T10:20:03.393760+0000","last_became_active":"2024-04-25T10:20:02.368081+0000","last_became_peered":"2024-04-25T10:20:02.368081+0000","last_unstale":"2024-04-25T10:20:03.393760+0000","last_undegraded":"2024-04-25T10:20:03.393760+0000","last_fullsized":"2024-04-25T10:20:03.393760+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:07.508595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413183+0000","last_change":"2024-04-25T10:20:01.335371+0000","last_active":"2024-04-25T10:20:03.413183+0000","last_peered":"2024-04-25T10:20:03.413183+0000","last_clean":"2024-04-25T10:20:03.413183+0000","last_became_active":"2024-04-25T10:20:01.335092+0000","last_became_peered":"2024-04-25T10:20:01.335092+0000","last_unstale":"2024-04-25T10:20:03.413183+0000","last_undegraded":"2024-04-25T10:20:03.413183+0000","last_fullsized":"2024-04-25T10:20:03.413183+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:44.129836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372130+0000","last_change":"2024-04-25T10:20:02.366426+0000","last_active":"2024-04-25T10:20:03.372130+0000","last_peered":"2024-04-25T10:20:03.372130+0000","last_clean":"2024-04-25T10:20:03.372130+0000","last_became_active":"2024-04-25T10:20:02.366147+0000","last_became_peered":"2024-04-25T10:20:02.366147+0000","last_unstale":"2024-04-25T10:20:03.372130+0000","last_undegraded":"2024-04-25T10:20:03.372130+0000","last_fullsized":"2024-04-25T10:20:03.372130+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:09.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.691914+0000","last_change":"2024-04-25T10:20:01.339862+0000","last_active":"2024-04-25T10:20:03.691914+0000","last_peered":"2024-04-25T10:20:03.691914+0000","last_clean":"2024-04-25T10:20:03.691914+0000","last_became_active":"2024-04-25T10:20:01.339331+0000","last_became_peered":"2024-04-25T10:20:01.339331+0000","last_unstale":"2024-04-25T10:20:03.691914+0000","last_undegraded":"2024-04-25T10:20:03.691914+0000","last_fullsized":"2024-04-25T10:20:03.691914+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:54.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412804+0000","last_change":"2024-04-25T10:20:02.360346+0000","last_active":"2024-04-25T10:20:03.412804+0000","last_peered":"2024-04-25T10:20:03.412804+0000","last_clean":"2024-04-25T10:20:03.412804+0000","last_became_active":"2024-04-25T10:20:02.353684+0000","last_became_peered":"2024-04-25T10:20:02.353684+0000","last_unstale":"2024-04-25T10:20:03.412804+0000","last_undegraded":"2024-04-25T10:20:03.412804+0000","last_fullsized":"2024-04-25T10:20:03.412804+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:37.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588049+0000","last_change":"2024-04-25T10:20:01.329000+0000","last_active":"2024-04-25T10:20:03.588049+0000","last_peered":"2024-04-25T10:20:03.588049+0000","last_clean":"2024-04-25T10:20:03.588049+0000","last_became_active":"2024-04-25T10:20:01.328744+0000","last_became_peered":"2024-04-25T10:20:01.328744+0000","last_unstale":"2024-04-25T10:20:03.588049+0000","last_undegraded":"2024-04-25T10:20:03.588049+0000","last_fullsized":"2024-04-25T10:20:03.588049+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:48.757660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":482,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392644+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:03.392644+0000","last_peered":"2024-04-25T10:20:03.392644+0000","last_clean":"2024-04-25T10:20:03.392644+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:03.392644+0000","last_undegraded":"2024-04-25T10:20:03.392644+0000","last_fullsized":"2024-04-25T10:20:03.392644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575071,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":2932736,"data_stored":1685404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575071,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59836,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312004,"statfs":{"total":96636764160,"available":96575492096,"internally_reserved":0,"allocated":2338816,"data_stored":1095548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575071,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2330624,"data_stored":1093410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575071,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87532,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284308,"statfs":{"total":96636764160,"available":96547131392,"internally_reserved":0,"allocated":2326528,"data_stored":1093936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575071,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87548,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284292,"statfs":{"total":96636764160,"available":96547115008,"internally_reserved":0,"allocated":2334720,"data_stored":1093480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59828,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312012,"statfs":{"total":96636764160,"available":96575500288,"internally_reserved":0,"allocated":2326528,"data_stored":1093444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2322432,"data_stored":1093354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575071,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65904,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305936,"statfs":{"total":96636764160,"available":96569278464,"internally_reserved":0,"allocated":2920448,"data_stored":1683812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:11.656 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:11.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.856+0000 7fdfca000640 1 Processor -- start 2024-04-25T10:20:11.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.856+0000 7fdfca000640 1 -- start start 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfca000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc4165680 0x7fdfc40aecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfca000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fdfc40cec70 0x7fdfc40b2650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfca000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfc40cf330 con 0x7fdfc40cc6c0 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfca000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfc4072340 con 0x7fdfc40cec70 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfca000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40b2b90 con 0x7fdfc4165680 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfc3e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fdfc40cc6c0 0x7fdfc40cf580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49190/0 (socket says 172.21.15.165:49190) 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfc3e00640 1 -- 172.21.15.165:0/1864876681 learned_addr learned my addr 172.21.15.165:0/1864876681 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.858+0000 7fdfbbe00640 1 --2- 172.21.15.165:0/1864876681 >> v2:172.21.15.165:3300/0 conn(0x7fdfc40cec70 0x7fdfc40b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfc3400640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc4165680 0x7fdfc40aecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfc2a00640 1 -- 172.21.15.165:0/1864876681 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2317807937 0 0) 0x7fdfc40cf330 con 0x7fdfc40cc6c0 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfc2a00640 1 -- 172.21.15.165:0/1864876681 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdfa8003520 con 0x7fdfc40cc6c0 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfbbe00640 1 -- 172.21.15.165:0/1864876681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc4165680 msgr2=0x7fdfc40aecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:11.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfbbe00640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc4165680 0x7fdfc40aecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfbbe00640 1 -- 172.21.15.165:0/1864876681 >> v1:172.21.15.165:6789/0 conn(0x7fdfc40cc6c0 legacy=0x7fdfc40cf580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:11.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfbbe00640 1 -- 172.21.15.165:0/1864876681 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4071d80 con 0x7fdfc40cec70 2024-04-25T10:20:11.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.859+0000 7fdfc3400640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc4165680 0x7fdfc40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.860+0000 7fdfbbe00640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b2650 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0097a0 tx=0x7fdfac02a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd3135037e6da86 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfc2a00640 1 -- 172.21.15.165:0/1864876681 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac037070 con 0x7fdfc40cec70 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfc2a00640 1 -- 172.21.15.165:0/1864876681 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac02ae50 con 0x7fdfc40cec70 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfc2a00640 1 -- 172.21.15.165:0/1864876681 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac032640 con 0x7fdfc40cec70 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 -- 172.21.15.165:0/1864876681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 msgr2=0x7fdfc40b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b2650 secure :-1 s=READY pgs=2606 cs=0 l=1 rev1=1 crypto rx=0x7fdfac0097a0 tx=0x7fdfac02a100 comp rx=0 tx=0).stop 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 -- 172.21.15.165:0/1864876681 shutdown_connections 2024-04-25T10:20:11.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b2650 unknown :-1 s=CLOSED pgs=2606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 --2- 172.21.15.165:0/1864876681 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc4165680 0x7fdfc40aecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 -- 172.21.15.165:0/1864876681 >> 172.21.15.165:0/1864876681 conn(0x7fdfc406abe0 msgr2=0x7fdfc406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.861+0000 7fdfca000640 1 -- 172.21.15.165:0/1864876681 shutdown_connections 2024-04-25T10:20:11.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.862+0000 7fdfca000640 1 -- 172.21.15.165:0/1864876681 wait complete. 2024-04-25T10:20:11.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.863+0000 7fdfca000640 1 Processor -- start 2024-04-25T10:20:11.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.863+0000 7fdfca000640 1 -- start start 2024-04-25T10:20:11.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfca000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40cc6c0 0x7fdfc40b1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:11.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfca000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:11.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfca000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc4165680 0x7fdfc40aa520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:11.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfca000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc40cf0d0 con 0x7fdfc40cec70 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfca000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc4071670 con 0x7fdfc4165680 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfca000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfc406ea30 con 0x7fdfc40cc6c0 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.864+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b1d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47834/0 (socket says 172.21.15.165:47834) 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 -- 172.21.15.165:0/143729726 learned_addr learned my addr 172.21.15.165:0/143729726 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfc3400640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40cc6c0 0x7fdfc40b1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 -- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40cc6c0 msgr2=0x7fdfc40b1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40cc6c0 0x7fdfc40b1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 -- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc4165680 msgr2=0x7fdfc40aa520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc4165680 0x7fdfc40aa520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:11.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 -- 172.21.15.165:0/143729726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc40ad120 con 0x7fdfc40cec70 2024-04-25T10:20:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.865+0000 7fdfbbe00640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b1d50 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02a140 tx=0x7fdfac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.866+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac041070 con 0x7fdfc40cec70 2024-04-25T10:20:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.866+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfac03c670 con 0x7fdfc40cec70 2024-04-25T10:20:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.866+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfac037040 con 0x7fdfc40cec70 2024-04-25T10:20:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.866+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfc40ad400 con 0x7fdfc40cec70 2024-04-25T10:20:11.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.866+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfc40acdf0 con 0x7fdfc40cec70 2024-04-25T10:20:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.867+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfc4073990 con 0x7fdfc40cec70 2024-04-25T10:20:11.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.868+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fdfac044070 con 0x7fdfc40cec70 2024-04-25T10:20:11.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.869+0000 7fdfbb400640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa006b500 0x7fdfa006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:11.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.869+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fdfac0aaa30 con 0x7fdfc40cec70 2024-04-25T10:20:11.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.869+0000 7fdfc3400640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa006b500 0x7fdfa006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:11.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.870+0000 7fdfc3400640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa006b500 0x7fdfa006d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400d8b0 tx=0x7fdfb4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:11.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:11.874+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fdfac079710 con 0x7fdfc40cec70 2024-04-25T10:20:12.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.066+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdfc40ad920 con 0x7fdfa006b500 2024-04-25T10:20:12.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.084+0000 7fdfbb400640 1 -- 172.21.15.165:0/143729726 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252493 (secure 0 0 0) 0x7fdfb400d4c0 con 0x7fdfa006b500 2024-04-25T10:20:12.089 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:12.092 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa006b500 msgr2=0x7fdfa006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa006b500 0x7fdfa006d9c0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fdfb400d8b0 tx=0x7fdfb4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 msgr2=0x7fdfc40b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b1d50 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7fdfac02a140 tx=0x7fdfac0046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 shutdown_connections 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fdfa006b500 0x7fdfa006d9c0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fdfc4165680 0x7fdfc40aa520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fdfc40cec70 0x7fdfc40b1d50 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 --2- 172.21.15.165:0/143729726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fdfc40cc6c0 0x7fdfc40b1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 >> 172.21.15.165:0/143729726 conn(0x7fdfc406abe0 msgr2=0x7fdfc40b3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 shutdown_connections 2024-04-25T10:20:12.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.091+0000 7fdfca000640 1 -- 172.21.15.165:0/143729726 wait complete. 2024-04-25T10:20:12.114 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":726,"stamp":"2024-04-25T10:20:10.500330+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":536556,"kb_used_data":19368,"kb_used_omap":12,"kb_used_meta":517043,"kb_avail":754438164,"statfs":{"total":773094113280,"available":772544679936,"internally_reserved":0,"allocated":19832832,"data_stored":9932388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":529452594},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.174465"},"pg_stats":[{"pgid":"28.3f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413484+0000","last_change":"2024-04-25T10:20:02.358152+0000","last_active":"2024-04-25T10:20:03.413484+0000","last_peered":"2024-04-25T10:20:03.413484+0000","last_clean":"2024-04-25T10:20:03.413484+0000","last_became_active":"2024-04-25T10:20:02.357325+0000","last_became_peered":"2024-04-25T10:20:02.357325+0000","last_unstale":"2024-04-25T10:20:03.413484+0000","last_undegraded":"2024-04-25T10:20:03.413484+0000","last_fullsized":"2024-04-25T10:20:03.413484+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:17:09.615819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.3e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329479+0000","last_change":"2024-04-25T10:20:02.361672+0000","last_active":"2024-04-25T10:20:03.329479+0000","last_peered":"2024-04-25T10:20:03.329479+0000","last_clean":"2024-04-25T10:20:03.329479+0000","last_became_active":"2024-04-25T10:20:02.361315+0000","last_became_peered":"2024-04-25T10:20:02.361315+0000","last_unstale":"2024-04-25T10:20:03.329479+0000","last_undegraded":"2024-04-25T10:20:03.329479+0000","last_fullsized":"2024-04-25T10:20:03.329479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:38.899350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.3d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372016+0000","last_change":"2024-04-25T10:20:02.371729+0000","last_active":"2024-04-25T10:20:03.372016+0000","last_peered":"2024-04-25T10:20:03.372016+0000","last_clean":"2024-04-25T10:20:03.372016+0000","last_became_active":"2024-04-25T10:20:02.371437+0000","last_became_peered":"2024-04-25T10:20:02.371437+0000","last_unstale":"2024-04-25T10:20:03.372016+0000","last_undegraded":"2024-04-25T10:20:03.372016+0000","last_fullsized":"2024-04-25T10:20:03.372016+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:14.409869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.3c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393346+0000","last_change":"2024-04-25T10:20:02.358692+0000","last_active":"2024-04-25T10:20:03.393346+0000","last_peered":"2024-04-25T10:20:03.393346+0000","last_clean":"2024-04-25T10:20:03.393346+0000","last_became_active":"2024-04-25T10:20:02.358409+0000","last_became_peered":"2024-04-25T10:20:02.358409+0000","last_unstale":"2024-04-25T10:20:03.393346+0000","last_undegraded":"2024-04-25T10:20:03.393346+0000","last_fullsized":"2024-04-25T10:20:03.393346+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:29.176224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.3b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692013+0000","last_change":"2024-04-25T10:20:02.340677+0000","last_active":"2024-04-25T10:20:03.692013+0000","last_peered":"2024-04-25T10:20:03.692013+0000","last_clean":"2024-04-25T10:20:03.692013+0000","last_became_active":"2024-04-25T10:20:02.339272+0000","last_became_peered":"2024-04-25T10:20:02.339272+0000","last_unstale":"2024-04-25T10:20:03.692013+0000","last_undegraded":"2024-04-25T10:20:03.692013+0000","last_fullsized":"2024-04-25T10:20:03.692013+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:48.284046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.3a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392723+0000","last_change":"2024-04-25T10:20:02.359749+0000","last_active":"2024-04-25T10:20:03.392723+0000","last_peered":"2024-04-25T10:20:03.392723+0000","last_clean":"2024-04-25T10:20:03.392723+0000","last_became_active":"2024-04-25T10:20:02.359441+0000","last_became_peered":"2024-04-25T10:20:02.359441+0000","last_unstale":"2024-04-25T10:20:03.392723+0000","last_undegraded":"2024-04-25T10:20:03.392723+0000","last_fullsized":"2024-04-25T10:20:03.392723+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:09.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.39","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.691731+0000","last_change":"2024-04-25T10:20:02.341866+0000","last_active":"2024-04-25T10:20:03.691731+0000","last_peered":"2024-04-25T10:20:03.691731+0000","last_clean":"2024-04-25T10:20:03.691731+0000","last_became_active":"2024-04-25T10:20:02.340905+0000","last_became_peered":"2024-04-25T10:20:02.340905+0000","last_unstale":"2024-04-25T10:20:03.691731+0000","last_undegraded":"2024-04-25T10:20:03.691731+0000","last_fullsized":"2024-04-25T10:20:03.691731+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:13:00.390890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.38","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.333608+0000","last_change":"2024-04-25T10:20:02.363273+0000","last_active":"2024-04-25T10:20:03.333608+0000","last_peered":"2024-04-25T10:20:03.333608+0000","last_clean":"2024-04-25T10:20:03.333608+0000","last_became_active":"2024-04-25T10:20:02.361340+0000","last_became_peered":"2024-04-25T10:20:02.361340+0000","last_unstale":"2024-04-25T10:20:03.333608+0000","last_undegraded":"2024-04-25T10:20:03.333608+0000","last_fullsized":"2024-04-25T10:20:03.333608+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:41:28.503164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.37","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588151+0000","last_change":"2024-04-25T10:20:02.367909+0000","last_active":"2024-04-25T10:20:03.588151+0000","last_peered":"2024-04-25T10:20:03.588151+0000","last_clean":"2024-04-25T10:20:03.588151+0000","last_became_active":"2024-04-25T10:20:02.367241+0000","last_became_peered":"2024-04-25T10:20:02.367241+0000","last_unstale":"2024-04-25T10:20:03.588151+0000","last_undegraded":"2024-04-25T10:20:03.588151+0000","last_fullsized":"2024-04-25T10:20:03.588151+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:28:40.310175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.36","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335988+0000","last_change":"2024-04-25T10:20:02.362899+0000","last_active":"2024-04-25T10:20:03.335988+0000","last_peered":"2024-04-25T10:20:03.335988+0000","last_clean":"2024-04-25T10:20:03.335988+0000","last_became_active":"2024-04-25T10:20:02.362609+0000","last_became_peered":"2024-04-25T10:20:02.362609+0000","last_unstale":"2024-04-25T10:20:03.335988+0000","last_undegraded":"2024-04-25T10:20:03.335988+0000","last_fullsized":"2024-04-25T10:20:03.335988+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.361669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.35","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371888+0000","last_change":"2024-04-25T10:20:02.368036+0000","last_active":"2024-04-25T10:20:03.371888+0000","last_peered":"2024-04-25T10:20:03.371888+0000","last_clean":"2024-04-25T10:20:03.371888+0000","last_became_active":"2024-04-25T10:20:02.367531+0000","last_became_peered":"2024-04-25T10:20:02.367531+0000","last_unstale":"2024-04-25T10:20:03.371888+0000","last_undegraded":"2024-04-25T10:20:03.371888+0000","last_fullsized":"2024-04-25T10:20:03.371888+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T22:14:19.407813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.34","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692218+0000","last_change":"2024-04-25T10:20:02.357465+0000","last_active":"2024-04-25T10:20:03.692218+0000","last_peered":"2024-04-25T10:20:03.692218+0000","last_clean":"2024-04-25T10:20:03.692218+0000","last_became_active":"2024-04-25T10:20:02.357177+0000","last_became_peered":"2024-04-25T10:20:02.357177+0000","last_unstale":"2024-04-25T10:20:03.692218+0000","last_undegraded":"2024-04-25T10:20:03.692218+0000","last_fullsized":"2024-04-25T10:20:03.692218+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:36:31.356936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.33","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393042+0000","last_change":"2024-04-25T10:20:02.367756+0000","last_active":"2024-04-25T10:20:03.393042+0000","last_peered":"2024-04-25T10:20:03.393042+0000","last_clean":"2024-04-25T10:20:03.393042+0000","last_became_active":"2024-04-25T10:20:02.367208+0000","last_became_peered":"2024-04-25T10:20:02.367208+0000","last_unstale":"2024-04-25T10:20:03.393042+0000","last_undegraded":"2024-04-25T10:20:03.393042+0000","last_fullsized":"2024-04-25T10:20:03.393042+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:46.609706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.32","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335021+0000","last_change":"2024-04-25T10:20:02.350419+0000","last_active":"2024-04-25T10:20:03.335021+0000","last_peered":"2024-04-25T10:20:03.335021+0000","last_clean":"2024-04-25T10:20:03.335021+0000","last_became_active":"2024-04-25T10:20:02.350117+0000","last_became_peered":"2024-04-25T10:20:02.350117+0000","last_unstale":"2024-04-25T10:20:03.335021+0000","last_undegraded":"2024-04-25T10:20:03.335021+0000","last_fullsized":"2024-04-25T10:20:03.335021+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:55.209567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.31","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334551+0000","last_change":"2024-04-25T10:20:02.358059+0000","last_active":"2024-04-25T10:20:03.334551+0000","last_peered":"2024-04-25T10:20:03.334551+0000","last_clean":"2024-04-25T10:20:03.334551+0000","last_became_active":"2024-04-25T10:20:02.357465+0000","last_became_peered":"2024-04-25T10:20:02.357465+0000","last_unstale":"2024-04-25T10:20:03.334551+0000","last_undegraded":"2024-04-25T10:20:03.334551+0000","last_fullsized":"2024-04-25T10:20:03.334551+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:32:27.324281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.30","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371544+0000","last_change":"2024-04-25T10:20:02.362462+0000","last_active":"2024-04-25T10:20:03.371544+0000","last_peered":"2024-04-25T10:20:03.371544+0000","last_clean":"2024-04-25T10:20:03.371544+0000","last_became_active":"2024-04-25T10:20:02.362143+0000","last_became_peered":"2024-04-25T10:20:02.362143+0000","last_unstale":"2024-04-25T10:20:03.371544+0000","last_undegraded":"2024-04-25T10:20:03.371544+0000","last_fullsized":"2024-04-25T10:20:03.371544+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:59:47.608253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.2f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588306+0000","last_change":"2024-04-25T10:20:02.364732+0000","last_active":"2024-04-25T10:20:03.588306+0000","last_peered":"2024-04-25T10:20:03.588306+0000","last_clean":"2024-04-25T10:20:03.588306+0000","last_became_active":"2024-04-25T10:20:02.363520+0000","last_became_peered":"2024-04-25T10:20:02.363520+0000","last_unstale":"2024-04-25T10:20:03.588306+0000","last_undegraded":"2024-04-25T10:20:03.588306+0000","last_fullsized":"2024-04-25T10:20:03.588306+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:18.802796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588197+0000","last_change":"2024-04-25T10:20:02.370945+0000","last_active":"2024-04-25T10:20:03.588197+0000","last_peered":"2024-04-25T10:20:03.588197+0000","last_clean":"2024-04-25T10:20:03.588197+0000","last_became_active":"2024-04-25T10:20:02.370652+0000","last_became_peered":"2024-04-25T10:20:02.370652+0000","last_unstale":"2024-04-25T10:20:03.588197+0000","last_undegraded":"2024-04-25T10:20:03.588197+0000","last_fullsized":"2024-04-25T10:20:03.588197+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:02.464323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.2d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336410+0000","last_change":"2024-04-25T10:20:02.353518+0000","last_active":"2024-04-25T10:20:03.336410+0000","last_peered":"2024-04-25T10:20:03.336410+0000","last_clean":"2024-04-25T10:20:03.336410+0000","last_became_active":"2024-04-25T10:20:02.352415+0000","last_became_peered":"2024-04-25T10:20:02.352415+0000","last_unstale":"2024-04-25T10:20:03.336410+0000","last_undegraded":"2024-04-25T10:20:03.336410+0000","last_fullsized":"2024-04-25T10:20:03.336410+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.161184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336751+0000","last_change":"2024-04-25T10:20:02.349464+0000","last_active":"2024-04-25T10:20:03.336751+0000","last_peered":"2024-04-25T10:20:03.336751+0000","last_clean":"2024-04-25T10:20:03.336751+0000","last_became_active":"2024-04-25T10:20:02.349133+0000","last_became_peered":"2024-04-25T10:20:02.349133+0000","last_unstale":"2024-04-25T10:20:03.336751+0000","last_undegraded":"2024-04-25T10:20:03.336751+0000","last_fullsized":"2024-04-25T10:20:03.336751+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:33.201607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335623+0000","last_change":"2024-04-25T10:20:02.356526+0000","last_active":"2024-04-25T10:20:03.335623+0000","last_peered":"2024-04-25T10:20:03.335623+0000","last_clean":"2024-04-25T10:20:03.335623+0000","last_became_active":"2024-04-25T10:20:02.355119+0000","last_became_peered":"2024-04-25T10:20:02.355119+0000","last_unstale":"2024-04-25T10:20:03.335623+0000","last_undegraded":"2024-04-25T10:20:03.335623+0000","last_fullsized":"2024-04-25T10:20:03.335623+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:49:54.302605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335139+0000","last_change":"2024-04-25T10:20:02.350034+0000","last_active":"2024-04-25T10:20:03.335139+0000","last_peered":"2024-04-25T10:20:03.335139+0000","last_clean":"2024-04-25T10:20:03.335139+0000","last_became_active":"2024-04-25T10:20:02.349692+0000","last_became_peered":"2024-04-25T10:20:02.349692+0000","last_unstale":"2024-04-25T10:20:03.335139+0000","last_undegraded":"2024-04-25T10:20:03.335139+0000","last_fullsized":"2024-04-25T10:20:03.335139+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:41.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.29","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328678+0000","last_change":"2024-04-25T10:20:02.367895+0000","last_active":"2024-04-25T10:20:03.328678+0000","last_peered":"2024-04-25T10:20:03.328678+0000","last_clean":"2024-04-25T10:20:03.328678+0000","last_became_active":"2024-04-25T10:20:02.366828+0000","last_became_peered":"2024-04-25T10:20:02.366828+0000","last_unstale":"2024-04-25T10:20:03.328678+0000","last_undegraded":"2024-04-25T10:20:03.328678+0000","last_fullsized":"2024-04-25T10:20:03.328678+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:53.422192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.28","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692026+0000","last_change":"2024-04-25T10:20:02.359741+0000","last_active":"2024-04-25T10:20:03.692026+0000","last_peered":"2024-04-25T10:20:03.692026+0000","last_clean":"2024-04-25T10:20:03.692026+0000","last_became_active":"2024-04-25T10:20:02.359423+0000","last_became_peered":"2024-04-25T10:20:02.359423+0000","last_unstale":"2024-04-25T10:20:03.692026+0000","last_undegraded":"2024-04-25T10:20:03.692026+0000","last_fullsized":"2024-04-25T10:20:03.692026+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:58:49.282643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.27","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588616+0000","last_change":"2024-04-25T10:20:02.364981+0000","last_active":"2024-04-25T10:20:03.588616+0000","last_peered":"2024-04-25T10:20:03.588616+0000","last_clean":"2024-04-25T10:20:03.588616+0000","last_became_active":"2024-04-25T10:20:02.364434+0000","last_became_peered":"2024-04-25T10:20:02.364434+0000","last_unstale":"2024-04-25T10:20:03.588616+0000","last_undegraded":"2024-04-25T10:20:03.588616+0000","last_fullsized":"2024-04-25T10:20:03.588616+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.020977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.26","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413181+0000","last_change":"2024-04-25T10:20:02.353985+0000","last_active":"2024-04-25T10:20:03.413181+0000","last_peered":"2024-04-25T10:20:03.413181+0000","last_clean":"2024-04-25T10:20:03.413181+0000","last_became_active":"2024-04-25T10:20:02.353555+0000","last_became_peered":"2024-04-25T10:20:02.353555+0000","last_unstale":"2024-04-25T10:20:03.413181+0000","last_undegraded":"2024-04-25T10:20:03.413181+0000","last_fullsized":"2024-04-25T10:20:03.413181+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:47.800009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.25","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372065+0000","last_change":"2024-04-25T10:20:02.367801+0000","last_active":"2024-04-25T10:20:03.372065+0000","last_peered":"2024-04-25T10:20:03.372065+0000","last_clean":"2024-04-25T10:20:03.372065+0000","last_became_active":"2024-04-25T10:20:02.367167+0000","last_became_peered":"2024-04-25T10:20:02.367167+0000","last_unstale":"2024-04-25T10:20:03.372065+0000","last_undegraded":"2024-04-25T10:20:03.372065+0000","last_fullsized":"2024-04-25T10:20:03.372065+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:33.169728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.24","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692509+0000","last_change":"2024-04-25T10:20:02.353780+0000","last_active":"2024-04-25T10:20:03.692509+0000","last_peered":"2024-04-25T10:20:03.692509+0000","last_clean":"2024-04-25T10:20:03.692509+0000","last_became_active":"2024-04-25T10:20:02.353480+0000","last_became_peered":"2024-04-25T10:20:02.353480+0000","last_unstale":"2024-04-25T10:20:03.692509+0000","last_undegraded":"2024-04-25T10:20:03.692509+0000","last_fullsized":"2024-04-25T10:20:03.692509+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:08:48.767601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.23","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587683+0000","last_change":"2024-04-25T10:20:02.362222+0000","last_active":"2024-04-25T10:20:03.587683+0000","last_peered":"2024-04-25T10:20:03.587683+0000","last_clean":"2024-04-25T10:20:03.587683+0000","last_became_active":"2024-04-25T10:20:02.361935+0000","last_became_peered":"2024-04-25T10:20:02.361935+0000","last_unstale":"2024-04-25T10:20:03.587683+0000","last_undegraded":"2024-04-25T10:20:03.587683+0000","last_fullsized":"2024-04-25T10:20:03.587683+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:16.523314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.22","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.333550+0000","last_change":"2024-04-25T10:20:02.358637+0000","last_active":"2024-04-25T10:20:03.333550+0000","last_peered":"2024-04-25T10:20:03.333550+0000","last_clean":"2024-04-25T10:20:03.333550+0000","last_became_active":"2024-04-25T10:20:02.355276+0000","last_became_peered":"2024-04-25T10:20:02.355276+0000","last_unstale":"2024-04-25T10:20:03.333550+0000","last_undegraded":"2024-04-25T10:20:03.333550+0000","last_fullsized":"2024-04-25T10:20:03.333550+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:23.161184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.21","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392968+0000","last_change":"2024-04-25T10:20:02.366085+0000","last_active":"2024-04-25T10:20:03.392968+0000","last_peered":"2024-04-25T10:20:03.392968+0000","last_clean":"2024-04-25T10:20:03.392968+0000","last_became_active":"2024-04-25T10:20:02.365739+0000","last_became_peered":"2024-04-25T10:20:02.365739+0000","last_unstale":"2024-04-25T10:20:03.392968+0000","last_undegraded":"2024-04-25T10:20:03.392968+0000","last_fullsized":"2024-04-25T10:20:03.392968+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:32.077900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.20","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392708+0000","last_change":"2024-04-25T10:20:02.354880+0000","last_active":"2024-04-25T10:20:03.392708+0000","last_peered":"2024-04-25T10:20:03.392708+0000","last_clean":"2024-04-25T10:20:03.392708+0000","last_became_active":"2024-04-25T10:20:02.354592+0000","last_became_peered":"2024-04-25T10:20:02.354592+0000","last_unstale":"2024-04-25T10:20:03.392708+0000","last_undegraded":"2024-04-25T10:20:03.392708+0000","last_fullsized":"2024-04-25T10:20:03.392708+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:14.409869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328727+0000","last_change":"2024-04-25T10:20:02.372978+0000","last_active":"2024-04-25T10:20:03.328727+0000","last_peered":"2024-04-25T10:20:03.328727+0000","last_clean":"2024-04-25T10:20:03.328727+0000","last_became_active":"2024-04-25T10:20:02.368461+0000","last_became_peered":"2024-04-25T10:20:02.368461+0000","last_unstale":"2024-04-25T10:20:03.328727+0000","last_undegraded":"2024-04-25T10:20:03.328727+0000","last_fullsized":"2024-04-25T10:20:03.328727+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:11.020977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1f","version":"183'4","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334928+0000","last_change":"2024-04-25T10:20:01.332782+0000","last_active":"2024-04-25T10:20:03.334928+0000","last_peered":"2024-04-25T10:20:03.334928+0000","last_clean":"2024-04-25T10:20:03.334928+0000","last_became_active":"2024-04-25T10:20:01.332481+0000","last_became_peered":"2024-04-25T10:20:01.332481+0000","last_unstale":"2024-04-25T10:20:03.334928+0000","last_undegraded":"2024-04-25T10:20:03.334928+0000","last_fullsized":"2024-04-25T10:20:03.334928+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:56:54.192098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412404+0000","last_change":"2024-04-25T10:20:02.362911+0000","last_active":"2024-04-25T10:20:03.412404+0000","last_peered":"2024-04-25T10:20:03.412404+0000","last_clean":"2024-04-25T10:20:03.412404+0000","last_became_active":"2024-04-25T10:20:02.361867+0000","last_became_peered":"2024-04-25T10:20:02.361867+0000","last_unstale":"2024-04-25T10:20:03.412404+0000","last_undegraded":"2024-04-25T10:20:03.412404+0000","last_fullsized":"2024-04-25T10:20:03.412404+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:36.018496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413313+0000","last_change":"2024-04-25T10:20:01.335869+0000","last_active":"2024-04-25T10:20:03.413313+0000","last_peered":"2024-04-25T10:20:03.413313+0000","last_clean":"2024-04-25T10:20:03.413313+0000","last_became_active":"2024-04-25T10:20:01.335113+0000","last_became_peered":"2024-04-25T10:20:01.335113+0000","last_unstale":"2024-04-25T10:20:03.413313+0000","last_undegraded":"2024-04-25T10:20:03.413313+0000","last_fullsized":"2024-04-25T10:20:03.413313+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:59.188576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335420+0000","last_change":"2024-04-25T10:20:02.343477+0000","last_active":"2024-04-25T10:20:03.335420+0000","last_peered":"2024-04-25T10:20:03.335420+0000","last_clean":"2024-04-25T10:20:03.335420+0000","last_became_active":"2024-04-25T10:20:02.342712+0000","last_became_peered":"2024-04-25T10:20:02.342712+0000","last_unstale":"2024-04-25T10:20:03.335420+0000","last_undegraded":"2024-04-25T10:20:03.335420+0000","last_fullsized":"2024-04-25T10:20:03.335420+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:27:57.523892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1d","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329123+0000","last_change":"2024-04-25T10:20:01.334447+0000","last_active":"2024-04-25T10:20:03.329123+0000","last_peered":"2024-04-25T10:20:03.329123+0000","last_clean":"2024-04-25T10:20:03.329123+0000","last_became_active":"2024-04-25T10:20:01.334157+0000","last_became_peered":"2024-04-25T10:20:01.334157+0000","last_unstale":"2024-04-25T10:20:03.329123+0000","last_undegraded":"2024-04-25T10:20:03.329123+0000","last_fullsized":"2024-04-25T10:20:03.329123+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:52:59.603090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.1b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692441+0000","last_change":"2024-04-25T10:20:02.360457+0000","last_active":"2024-04-25T10:20:03.692441+0000","last_peered":"2024-04-25T10:20:03.692441+0000","last_clean":"2024-04-25T10:20:03.692441+0000","last_became_active":"2024-04-25T10:20:02.360136+0000","last_became_peered":"2024-04-25T10:20:02.360136+0000","last_unstale":"2024-04-25T10:20:03.692441+0000","last_undegraded":"2024-04-25T10:20:03.692441+0000","last_fullsized":"2024-04-25T10:20:03.692441+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:51:55.647660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.1c","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588180+0000","last_change":"2024-04-25T10:20:01.327488+0000","last_active":"2024-04-25T10:20:03.588180+0000","last_peered":"2024-04-25T10:20:03.588180+0000","last_clean":"2024-04-25T10:20:03.588180+0000","last_became_active":"2024-04-25T10:20:01.327214+0000","last_became_peered":"2024-04-25T10:20:01.327214+0000","last_unstale":"2024-04-25T10:20:03.588180+0000","last_undegraded":"2024-04-25T10:20:03.588180+0000","last_fullsized":"2024-04-25T10:20:03.588180+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:23.238260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334014+0000","last_change":"2024-04-25T10:20:02.360025+0000","last_active":"2024-04-25T10:20:03.334014+0000","last_peered":"2024-04-25T10:20:03.334014+0000","last_clean":"2024-04-25T10:20:03.334014+0000","last_became_active":"2024-04-25T10:20:02.359709+0000","last_became_peered":"2024-04-25T10:20:02.359709+0000","last_unstale":"2024-04-25T10:20:03.334014+0000","last_undegraded":"2024-04-25T10:20:03.334014+0000","last_fullsized":"2024-04-25T10:20:03.334014+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:03:33.201607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1b","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393119+0000","last_change":"2024-04-25T10:20:01.341558+0000","last_active":"2024-04-25T10:20:03.393119+0000","last_peered":"2024-04-25T10:20:03.393119+0000","last_clean":"2024-04-25T10:20:03.393119+0000","last_became_active":"2024-04-25T10:20:01.340467+0000","last_became_peered":"2024-04-25T10:20:01.340467+0000","last_unstale":"2024-04-25T10:20:03.393119+0000","last_undegraded":"2024-04-25T10:20:03.393119+0000","last_fullsized":"2024-04-25T10:20:03.393119+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:14.878826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334529+0000","last_change":"2024-04-25T10:20:02.345657+0000","last_active":"2024-04-25T10:20:03.334529+0000","last_peered":"2024-04-25T10:20:03.334529+0000","last_clean":"2024-04-25T10:20:03.334529+0000","last_became_active":"2024-04-25T10:20:02.345353+0000","last_became_peered":"2024-04-25T10:20:02.345353+0000","last_unstale":"2024-04-25T10:20:03.334529+0000","last_undegraded":"2024-04-25T10:20:03.334529+0000","last_fullsized":"2024-04-25T10:20:03.334529+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:11.831418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392504+0000","last_change":"2024-04-25T10:20:01.342761+0000","last_active":"2024-04-25T10:20:03.392504+0000","last_peered":"2024-04-25T10:20:03.392504+0000","last_clean":"2024-04-25T10:20:03.392504+0000","last_became_active":"2024-04-25T10:20:01.342145+0000","last_became_peered":"2024-04-25T10:20:01.342145+0000","last_unstale":"2024-04-25T10:20:03.392504+0000","last_undegraded":"2024-04-25T10:20:03.392504+0000","last_fullsized":"2024-04-25T10:20:03.392504+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588718+0000","last_change":"2024-04-25T10:20:02.362124+0000","last_active":"2024-04-25T10:20:03.588718+0000","last_peered":"2024-04-25T10:20:03.588718+0000","last_clean":"2024-04-25T10:20:03.588718+0000","last_became_active":"2024-04-25T10:20:02.357833+0000","last_became_peered":"2024-04-25T10:20:02.357833+0000","last_unstale":"2024-04-25T10:20:03.588718+0000","last_undegraded":"2024-04-25T10:20:03.588718+0000","last_fullsized":"2024-04-25T10:20:03.588718+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.734006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371516+0000","last_change":"2024-04-25T10:20:01.338974+0000","last_active":"2024-04-25T10:20:03.371516+0000","last_peered":"2024-04-25T10:20:03.371516+0000","last_clean":"2024-04-25T10:20:03.371516+0000","last_became_active":"2024-04-25T10:20:01.338631+0000","last_became_peered":"2024-04-25T10:20:01.338631+0000","last_unstale":"2024-04-25T10:20:03.371516+0000","last_undegraded":"2024-04-25T10:20:03.371516+0000","last_fullsized":"2024-04-25T10:20:03.371516+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:31:30.668056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.10","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.333363+0000","last_change":"2024-04-25T10:20:02.357818+0000","last_active":"2024-04-25T10:20:03.333363+0000","last_peered":"2024-04-25T10:20:03.333363+0000","last_clean":"2024-04-25T10:20:03.333363+0000","last_became_active":"2024-04-25T10:20:02.357502+0000","last_became_peered":"2024-04-25T10:20:02.357502+0000","last_unstale":"2024-04-25T10:20:03.333363+0000","last_undegraded":"2024-04-25T10:20:03.333363+0000","last_fullsized":"2024-04-25T10:20:03.333363+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:06:24.361669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692952+0000","last_change":"2024-04-25T10:20:01.332708+0000","last_active":"2024-04-25T10:20:03.692952+0000","last_peered":"2024-04-25T10:20:03.692952+0000","last_clean":"2024-04-25T10:20:03.692952+0000","last_became_active":"2024-04-25T10:20:01.332230+0000","last_became_peered":"2024-04-25T10:20:01.332230+0000","last_unstale":"2024-04-25T10:20:03.692952+0000","last_undegraded":"2024-04-25T10:20:03.692952+0000","last_fullsized":"2024-04-25T10:20:03.692952+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:16:32.682708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371614+0000","last_change":"2024-04-25T10:20:02.369394+0000","last_active":"2024-04-25T10:20:03.371614+0000","last_peered":"2024-04-25T10:20:03.371614+0000","last_clean":"2024-04-25T10:20:03.371614+0000","last_became_active":"2024-04-25T10:20:02.369122+0000","last_became_peered":"2024-04-25T10:20:02.369122+0000","last_unstale":"2024-04-25T10:20:03.371614+0000","last_undegraded":"2024-04-25T10:20:03.371614+0000","last_fullsized":"2024-04-25T10:20:03.371614+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:53:25.084733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393814+0000","last_change":"2024-04-25T10:20:01.345530+0000","last_active":"2024-04-25T10:20:03.393814+0000","last_peered":"2024-04-25T10:20:03.393814+0000","last_clean":"2024-04-25T10:20:03.393814+0000","last_became_active":"2024-04-25T10:20:01.345190+0000","last_became_peered":"2024-04-25T10:20:01.345190+0000","last_unstale":"2024-04-25T10:20:03.393814+0000","last_undegraded":"2024-04-25T10:20:03.393814+0000","last_fullsized":"2024-04-25T10:20:03.393814+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:58.064824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587751+0000","last_change":"2024-04-25T10:20:02.366882+0000","last_active":"2024-04-25T10:20:03.587751+0000","last_peered":"2024-04-25T10:20:03.587751+0000","last_clean":"2024-04-25T10:20:03.587751+0000","last_became_active":"2024-04-25T10:20:02.366604+0000","last_became_peered":"2024-04-25T10:20:02.366604+0000","last_unstale":"2024-04-25T10:20:03.587751+0000","last_undegraded":"2024-04-25T10:20:03.587751+0000","last_fullsized":"2024-04-25T10:20:03.587751+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:32:46.930941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.15","version":"183'3","reported_seq":17,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588094+0000","last_change":"2024-04-25T10:20:01.329959+0000","last_active":"2024-04-25T10:20:03.588094+0000","last_peered":"2024-04-25T10:20:03.588094+0000","last_clean":"2024-04-25T10:20:03.588094+0000","last_became_active":"2024-04-25T10:20:01.329609+0000","last_became_peered":"2024-04-25T10:20:01.329609+0000","last_unstale":"2024-04-25T10:20:03.588094+0000","last_undegraded":"2024-04-25T10:20:03.588094+0000","last_fullsized":"2024-04-25T10:20:03.588094+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:23:42.385948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692471+0000","last_change":"2024-04-25T10:20:02.352991+0000","last_active":"2024-04-25T10:20:03.692471+0000","last_peered":"2024-04-25T10:20:03.692471+0000","last_clean":"2024-04-25T10:20:03.692471+0000","last_became_active":"2024-04-25T10:20:02.352639+0000","last_became_peered":"2024-04-25T10:20:02.352639+0000","last_unstale":"2024-04-25T10:20:03.692471+0000","last_undegraded":"2024-04-25T10:20:03.692471+0000","last_fullsized":"2024-04-25T10:20:03.692471+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:16:48.990837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.14","version":"183'6","reported_seq":20,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393131+0000","last_change":"2024-04-25T10:20:01.343264+0000","last_active":"2024-04-25T10:20:03.393131+0000","last_peered":"2024-04-25T10:20:03.393131+0000","last_clean":"2024-04-25T10:20:03.393131+0000","last_became_active":"2024-04-25T10:20:01.342715+0000","last_became_peered":"2024-04-25T10:20:01.342715+0000","last_unstale":"2024-04-25T10:20:03.393131+0000","last_undegraded":"2024-04-25T10:20:03.393131+0000","last_fullsized":"2024-04-25T10:20:03.393131+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:36.371453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329451+0000","last_change":"2024-04-25T10:20:02.371420+0000","last_active":"2024-04-25T10:20:03.329451+0000","last_peered":"2024-04-25T10:20:03.329451+0000","last_clean":"2024-04-25T10:20:03.329451+0000","last_became_active":"2024-04-25T10:20:02.371140+0000","last_became_peered":"2024-04-25T10:20:02.371140+0000","last_unstale":"2024-04-25T10:20:03.329451+0000","last_undegraded":"2024-04-25T10:20:03.329451+0000","last_fullsized":"2024-04-25T10:20:03.329451+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:41:38.734006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328978+0000","last_change":"2024-04-25T10:20:01.337329+0000","last_active":"2024-04-25T10:20:03.328978+0000","last_peered":"2024-04-25T10:20:03.328978+0000","last_clean":"2024-04-25T10:20:03.328978+0000","last_became_active":"2024-04-25T10:20:01.336415+0000","last_became_peered":"2024-04-25T10:20:01.336415+0000","last_unstale":"2024-04-25T10:20:03.328978+0000","last_undegraded":"2024-04-25T10:20:03.328978+0000","last_fullsized":"2024-04-25T10:20:03.328978+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:13:41.144989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.15","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372221+0000","last_change":"2024-04-25T10:20:02.375970+0000","last_active":"2024-04-25T10:20:03.372221+0000","last_peered":"2024-04-25T10:20:03.372221+0000","last_clean":"2024-04-25T10:20:03.372221+0000","last_became_active":"2024-04-25T10:20:02.375664+0000","last_became_peered":"2024-04-25T10:20:02.375664+0000","last_unstale":"2024-04-25T10:20:03.372221+0000","last_undegraded":"2024-04-25T10:20:03.372221+0000","last_fullsized":"2024-04-25T10:20:03.372221+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:11:32.077900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371487+0000","last_change":"2024-04-25T10:20:01.330029+0000","last_active":"2024-04-25T10:20:03.371487+0000","last_peered":"2024-04-25T10:20:03.371487+0000","last_clean":"2024-04-25T10:20:03.371487+0000","last_became_active":"2024-04-25T10:20:01.328434+0000","last_became_peered":"2024-04-25T10:20:01.328434+0000","last_unstale":"2024-04-25T10:20:03.371487+0000","last_undegraded":"2024-04-25T10:20:03.371487+0000","last_fullsized":"2024-04-25T10:20:03.371487+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:55:35.001027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588359+0000","last_change":"2024-04-25T10:20:02.371094+0000","last_active":"2024-04-25T10:20:03.588359+0000","last_peered":"2024-04-25T10:20:03.588359+0000","last_clean":"2024-04-25T10:20:03.588359+0000","last_became_active":"2024-04-25T10:20:02.370802+0000","last_became_peered":"2024-04-25T10:20:02.370802+0000","last_unstale":"2024-04-25T10:20:03.588359+0000","last_undegraded":"2024-04-25T10:20:03.588359+0000","last_fullsized":"2024-04-25T10:20:03.588359+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:22:53.422192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587410+0000","last_change":"2024-04-25T10:20:01.332999+0000","last_active":"2024-04-25T10:20:03.587410+0000","last_peered":"2024-04-25T10:20:03.587410+0000","last_clean":"2024-04-25T10:20:03.587410+0000","last_became_active":"2024-04-25T10:20:01.332513+0000","last_became_peered":"2024-04-25T10:20:01.332513+0000","last_unstale":"2024-04-25T10:20:03.587410+0000","last_undegraded":"2024-04-25T10:20:03.587410+0000","last_fullsized":"2024-04-25T10:20:03.587410+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:07:36.561856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393685+0000","last_change":"2024-04-25T10:20:02.364808+0000","last_active":"2024-04-25T10:20:03.393685+0000","last_peered":"2024-04-25T10:20:03.393685+0000","last_clean":"2024-04-25T10:20:03.393685+0000","last_became_active":"2024-04-25T10:20:02.364082+0000","last_became_peered":"2024-04-25T10:20:02.364082+0000","last_unstale":"2024-04-25T10:20:03.393685+0000","last_undegraded":"2024-04-25T10:20:03.393685+0000","last_fullsized":"2024-04-25T10:20:03.393685+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:49.775129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.10","version":"183'8","reported_seq":22,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.691543+0000","last_change":"2024-04-25T10:20:01.338593+0000","last_active":"2024-04-25T10:20:03.691543+0000","last_peered":"2024-04-25T10:20:03.691543+0000","last_clean":"2024-04-25T10:20:03.691543+0000","last_became_active":"2024-04-25T10:20:01.338279+0000","last_became_peered":"2024-04-25T10:20:01.338279+0000","last_unstale":"2024-04-25T10:20:03.691543+0000","last_undegraded":"2024-04-25T10:20:03.691543+0000","last_fullsized":"2024-04-25T10:20:03.691543+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:49:01.211682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334004+0000","last_change":"2024-04-25T10:20:02.363884+0000","last_active":"2024-04-25T10:20:03.334004+0000","last_peered":"2024-04-25T10:20:03.334004+0000","last_clean":"2024-04-25T10:20:03.334004+0000","last_became_active":"2024-04-25T10:20:02.363593+0000","last_became_peered":"2024-04-25T10:20:02.363593+0000","last_unstale":"2024-04-25T10:20:03.334004+0000","last_undegraded":"2024-04-25T10:20:03.334004+0000","last_fullsized":"2024-04-25T10:20:03.334004+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:41.662935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.f","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413459+0000","last_change":"2024-04-25T10:20:01.341364+0000","last_active":"2024-04-25T10:20:03.413459+0000","last_peered":"2024-04-25T10:20:03.413459+0000","last_clean":"2024-04-25T10:20:03.413459+0000","last_became_active":"2024-04-25T10:20:01.340654+0000","last_became_peered":"2024-04-25T10:20:01.340654+0000","last_unstale":"2024-04-25T10:20:03.413459+0000","last_undegraded":"2024-04-25T10:20:03.413459+0000","last_fullsized":"2024-04-25T10:20:03.413459+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:34:34.211448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413145+0000","last_change":"2024-04-25T10:20:02.363357+0000","last_active":"2024-04-25T10:20:03.413145+0000","last_peered":"2024-04-25T10:20:03.413145+0000","last_clean":"2024-04-25T10:20:03.413145+0000","last_became_active":"2024-04-25T10:20:02.361449+0000","last_became_peered":"2024-04-25T10:20:02.361449+0000","last_unstale":"2024-04-25T10:20:03.413145+0000","last_undegraded":"2024-04-25T10:20:03.413145+0000","last_fullsized":"2024-04-25T10:20:03.413145+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:10:59.252254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413440+0000","last_change":"2024-04-25T10:20:01.343272+0000","last_active":"2024-04-25T10:20:03.413440+0000","last_peered":"2024-04-25T10:20:03.413440+0000","last_clean":"2024-04-25T10:20:03.413440+0000","last_became_active":"2024-04-25T10:20:01.342389+0000","last_became_peered":"2024-04-25T10:20:01.342389+0000","last_unstale":"2024-04-25T10:20:03.413440+0000","last_undegraded":"2024-04-25T10:20:03.413440+0000","last_fullsized":"2024-04-25T10:20:03.413440+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:44:32.480221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412982+0000","last_change":"2024-04-25T10:20:02.359471+0000","last_active":"2024-04-25T10:20:03.412982+0000","last_peered":"2024-04-25T10:20:03.412982+0000","last_clean":"2024-04-25T10:20:03.412982+0000","last_became_active":"2024-04-25T10:20:02.356111+0000","last_became_peered":"2024-04-25T10:20:02.356111+0000","last_unstale":"2024-04-25T10:20:03.412982+0000","last_undegraded":"2024-04-25T10:20:03.412982+0000","last_fullsized":"2024-04-25T10:20:03.412982+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:41:59.082293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.d","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329198+0000","last_change":"2024-04-25T10:20:01.340613+0000","last_active":"2024-04-25T10:20:03.329198+0000","last_peered":"2024-04-25T10:20:03.329198+0000","last_clean":"2024-04-25T10:20:03.329198+0000","last_became_active":"2024-04-25T10:20:01.340250+0000","last_became_peered":"2024-04-25T10:20:01.340250+0000","last_unstale":"2024-04-25T10:20:03.329198+0000","last_undegraded":"2024-04-25T10:20:03.329198+0000","last_fullsized":"2024-04-25T10:20:03.329198+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:15.505845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.b","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335698+0000","last_change":"2024-04-25T10:20:02.346899+0000","last_active":"2024-04-25T10:20:03.335698+0000","last_peered":"2024-04-25T10:20:03.335698+0000","last_clean":"2024-04-25T10:20:03.335698+0000","last_became_active":"2024-04-25T10:20:02.346568+0000","last_became_peered":"2024-04-25T10:20:02.346568+0000","last_unstale":"2024-04-25T10:20:03.335698+0000","last_undegraded":"2024-04-25T10:20:03.335698+0000","last_fullsized":"2024-04-25T10:20:03.335698+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:09:28.811416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692474+0000","last_change":"2024-04-25T10:20:01.343143+0000","last_active":"2024-04-25T10:20:03.692474+0000","last_peered":"2024-04-25T10:20:03.692474+0000","last_clean":"2024-04-25T10:20:03.692474+0000","last_became_active":"2024-04-25T10:20:01.342912+0000","last_became_peered":"2024-04-25T10:20:01.342912+0000","last_unstale":"2024-04-25T10:20:03.692474+0000","last_undegraded":"2024-04-25T10:20:03.692474+0000","last_fullsized":"2024-04-25T10:20:03.692474+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:16:21.956577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.329256+0000","last_change":"2024-04-25T10:20:02.364467+0000","last_active":"2024-04-25T10:20:03.329256+0000","last_peered":"2024-04-25T10:20:03.329256+0000","last_clean":"2024-04-25T10:20:03.329256+0000","last_became_active":"2024-04-25T10:20:02.359243+0000","last_became_peered":"2024-04-25T10:20:02.359243+0000","last_unstale":"2024-04-25T10:20:03.329256+0000","last_undegraded":"2024-04-25T10:20:03.329256+0000","last_fullsized":"2024-04-25T10:20:03.329256+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:49.582825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.b","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.692675+0000","last_change":"2024-04-25T10:20:01.333143+0000","last_active":"2024-04-25T10:20:03.692675+0000","last_peered":"2024-04-25T10:20:03.692675+0000","last_clean":"2024-04-25T10:20:03.692675+0000","last_became_active":"2024-04-25T10:20:01.332865+0000","last_became_peered":"2024-04-25T10:20:01.332865+0000","last_unstale":"2024-04-25T10:20:03.692675+0000","last_undegraded":"2024-04-25T10:20:03.692675+0000","last_fullsized":"2024-04-25T10:20:03.692675+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:46:29.055338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393546+0000","last_change":"2024-04-25T10:20:02.366377+0000","last_active":"2024-04-25T10:20:03.393546+0000","last_peered":"2024-04-25T10:20:03.393546+0000","last_clean":"2024-04-25T10:20:03.393546+0000","last_became_active":"2024-04-25T10:20:02.366065+0000","last_became_peered":"2024-04-25T10:20:02.366065+0000","last_unstale":"2024-04-25T10:20:03.393546+0000","last_undegraded":"2024-04-25T10:20:03.393546+0000","last_fullsized":"2024-04-25T10:20:03.393546+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:06.008965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.328772+0000","last_change":"2024-04-25T10:20:01.332184+0000","last_active":"2024-04-25T10:20:03.328772+0000","last_peered":"2024-04-25T10:20:03.328772+0000","last_clean":"2024-04-25T10:20:03.328772+0000","last_became_active":"2024-04-25T10:20:01.331558+0000","last_became_peered":"2024-04-25T10:20:01.331558+0000","last_unstale":"2024-04-25T10:20:03.328772+0000","last_undegraded":"2024-04-25T10:20:03.328772+0000","last_fullsized":"2024-04-25T10:20:03.328772+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:07:59.100718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588637+0000","last_change":"2024-04-25T10:20:02.370318+0000","last_active":"2024-04-25T10:20:03.588637+0000","last_peered":"2024-04-25T10:20:03.588637+0000","last_clean":"2024-04-25T10:20:03.588637+0000","last_became_active":"2024-04-25T10:20:02.358118+0000","last_became_peered":"2024-04-25T10:20:02.358118+0000","last_unstale":"2024-04-25T10:20:03.588637+0000","last_undegraded":"2024-04-25T10:20:03.588637+0000","last_fullsized":"2024-04-25T10:20:03.588637+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:45:49.582825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334864+0000","last_change":"2024-04-25T10:20:01.333236+0000","last_active":"2024-04-25T10:20:03.334864+0000","last_peered":"2024-04-25T10:20:03.334864+0000","last_clean":"2024-04-25T10:20:03.334864+0000","last_became_active":"2024-04-25T10:20:01.332681+0000","last_became_peered":"2024-04-25T10:20:01.332681+0000","last_unstale":"2024-04-25T10:20:03.334864+0000","last_undegraded":"2024-04-25T10:20:03.334864+0000","last_fullsized":"2024-04-25T10:20:03.334864+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:26.830908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.f","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336827+0000","last_change":"2024-04-25T10:20:02.341693+0000","last_active":"2024-04-25T10:20:03.336827+0000","last_peered":"2024-04-25T10:20:03.336827+0000","last_clean":"2024-04-25T10:20:03.336827+0000","last_became_active":"2024-04-25T10:20:02.341396+0000","last_became_peered":"2024-04-25T10:20:02.341396+0000","last_unstale":"2024-04-25T10:20:03.336827+0000","last_undegraded":"2024-04-25T10:20:03.336827+0000","last_fullsized":"2024-04-25T10:20:03.336827+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:55:11.831418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412516+0000","last_change":"2024-04-25T10:20:01.334912+0000","last_active":"2024-04-25T10:20:03.412516+0000","last_peered":"2024-04-25T10:20:03.412516+0000","last_clean":"2024-04-25T10:20:03.412516+0000","last_became_active":"2024-04-25T10:20:01.334608+0000","last_became_peered":"2024-04-25T10:20:01.334608+0000","last_unstale":"2024-04-25T10:20:03.412516+0000","last_undegraded":"2024-04-25T10:20:03.412516+0000","last_fullsized":"2024-04-25T10:20:03.412516+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:09:19.602080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334006+0000","last_change":"2024-04-25T10:20:02.356760+0000","last_active":"2024-04-25T10:20:03.334006+0000","last_peered":"2024-04-25T10:20:03.334006+0000","last_clean":"2024-04-25T10:20:03.334006+0000","last_became_active":"2024-04-25T10:20:02.356400+0000","last_became_peered":"2024-04-25T10:20:02.356400+0000","last_unstale":"2024-04-25T10:20:03.334006+0000","last_undegraded":"2024-04-25T10:20:03.334006+0000","last_fullsized":"2024-04-25T10:20:03.334006+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:01:27.848377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.7","version":"183'2","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588720+0000","last_change":"2024-04-25T10:20:01.335981+0000","last_active":"2024-04-25T10:20:03.588720+0000","last_peered":"2024-04-25T10:20:03.588720+0000","last_clean":"2024-04-25T10:20:03.588720+0000","last_became_active":"2024-04-25T10:20:01.335703+0000","last_became_peered":"2024-04-25T10:20:01.335703+0000","last_unstale":"2024-04-25T10:20:03.588720+0000","last_undegraded":"2024-04-25T10:20:03.588720+0000","last_fullsized":"2024-04-25T10:20:03.588720+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:13:18.277747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.371392+0000","last_change":"2024-04-25T10:20:02.367475+0000","last_active":"2024-04-25T10:20:03.371392+0000","last_peered":"2024-04-25T10:20:03.371392+0000","last_clean":"2024-04-25T10:20:03.371392+0000","last_became_active":"2024-04-25T10:20:02.367142+0000","last_became_peered":"2024-04-25T10:20:02.367142+0000","last_unstale":"2024-04-25T10:20:03.371392+0000","last_undegraded":"2024-04-25T10:20:03.371392+0000","last_fullsized":"2024-04-25T10:20:03.371392+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:34:03.489816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.6","version":"184'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.336125+0000","last_change":"2024-04-25T10:20:01.332051+0000","last_active":"2024-04-25T10:20:03.336125+0000","last_peered":"2024-04-25T10:20:03.336125+0000","last_clean":"2024-04-25T10:20:03.336125+0000","last_became_active":"2024-04-25T10:20:01.331777+0000","last_became_peered":"2024-04-25T10:20:01.331777+0000","last_unstale":"2024-04-25T10:20:03.336125+0000","last_undegraded":"2024-04-25T10:20:03.336125+0000","last_fullsized":"2024-04-25T10:20:03.336125+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:18:12.465188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.587653+0000","last_change":"2024-04-25T10:20:02.372312+0000","last_active":"2024-04-25T10:20:03.587653+0000","last_peered":"2024-04-25T10:20:03.587653+0000","last_clean":"2024-04-25T10:20:03.587653+0000","last_became_active":"2024-04-25T10:20:02.372038+0000","last_became_peered":"2024-04-25T10:20:02.372038+0000","last_unstale":"2024-04-25T10:20:03.587653+0000","last_undegraded":"2024-04-25T10:20:03.587653+0000","last_fullsized":"2024-04-25T10:20:03.587653+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:27:44.354381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.334535+0000","last_change":"2024-04-25T10:20:01.323209+0000","last_active":"2024-04-25T10:20:03.334535+0000","last_peered":"2024-04-25T10:20:03.334535+0000","last_clean":"2024-04-25T10:20:03.334535+0000","last_became_active":"2024-04-25T10:20:01.319553+0000","last_became_peered":"2024-04-25T10:20:01.319553+0000","last_unstale":"2024-04-25T10:20:03.334535+0000","last_undegraded":"2024-04-25T10:20:03.334535+0000","last_fullsized":"2024-04-25T10:20:03.334535+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:46:40.645466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412928+0000","last_change":"2024-04-25T10:20:02.360953+0000","last_active":"2024-04-25T10:20:03.412928+0000","last_peered":"2024-04-25T10:20:03.412928+0000","last_clean":"2024-04-25T10:20:03.412928+0000","last_became_active":"2024-04-25T10:20:02.358576+0000","last_became_peered":"2024-04-25T10:20:02.358576+0000","last_unstale":"2024-04-25T10:20:03.412928+0000","last_undegraded":"2024-04-25T10:20:03.412928+0000","last_fullsized":"2024-04-25T10:20:03.412928+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:15.896295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412868+0000","last_change":"2024-04-25T10:20:01.336445+0000","last_active":"2024-04-25T10:20:03.412868+0000","last_peered":"2024-04-25T10:20:03.412868+0000","last_clean":"2024-04-25T10:20:03.412868+0000","last_became_active":"2024-04-25T10:20:01.336136+0000","last_became_peered":"2024-04-25T10:20:01.336136+0000","last_unstale":"2024-04-25T10:20:03.412868+0000","last_undegraded":"2024-04-25T10:20:03.412868+0000","last_fullsized":"2024-04-25T10:20:03.412868+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:27:20.879308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335837+0000","last_change":"2024-04-25T10:20:02.361100+0000","last_active":"2024-04-25T10:20:03.335837+0000","last_peered":"2024-04-25T10:20:03.335837+0000","last_clean":"2024-04-25T10:20:03.335837+0000","last_became_active":"2024-04-25T10:20:02.360829+0000","last_became_peered":"2024-04-25T10:20:02.360829+0000","last_unstale":"2024-04-25T10:20:03.335837+0000","last_undegraded":"2024-04-25T10:20:03.335837+0000","last_fullsized":"2024-04-25T10:20:03.335837+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:06:45.508642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.335916+0000","last_change":"2024-04-25T10:20:01.329676+0000","last_active":"2024-04-25T10:20:03.335916+0000","last_peered":"2024-04-25T10:20:03.335916+0000","last_clean":"2024-04-25T10:20:03.335916+0000","last_became_active":"2024-04-25T10:20:01.329366+0000","last_became_peered":"2024-04-25T10:20:01.329366+0000","last_unstale":"2024-04-25T10:20:03.335916+0000","last_undegraded":"2024-04-25T10:20:03.335916+0000","last_fullsized":"2024-04-25T10:20:03.335916+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:34:57.834078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393632+0000","last_change":"2024-04-25T10:20:02.369691+0000","last_active":"2024-04-25T10:20:03.393632+0000","last_peered":"2024-04-25T10:20:03.393632+0000","last_clean":"2024-04-25T10:20:03.393632+0000","last_became_active":"2024-04-25T10:20:02.369391+0000","last_became_peered":"2024-04-25T10:20:02.369391+0000","last_unstale":"2024-04-25T10:20:03.393632+0000","last_undegraded":"2024-04-25T10:20:03.393632+0000","last_fullsized":"2024-04-25T10:20:03.393632+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:33:30.459139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588323+0000","last_change":"2024-04-25T10:20:01.334388+0000","last_active":"2024-04-25T10:20:03.588323+0000","last_peered":"2024-04-25T10:20:03.588323+0000","last_clean":"2024-04-25T10:20:03.588323+0000","last_became_active":"2024-04-25T10:20:01.333900+0000","last_became_peered":"2024-04-25T10:20:01.333900+0000","last_unstale":"2024-04-25T10:20:03.588323+0000","last_undegraded":"2024-04-25T10:20:03.588323+0000","last_fullsized":"2024-04-25T10:20:03.588323+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:06.491126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.393760+0000","last_change":"2024-04-25T10:20:02.368901+0000","last_active":"2024-04-25T10:20:03.393760+0000","last_peered":"2024-04-25T10:20:03.393760+0000","last_clean":"2024-04-25T10:20:03.393760+0000","last_became_active":"2024-04-25T10:20:02.368081+0000","last_became_peered":"2024-04-25T10:20:02.368081+0000","last_unstale":"2024-04-25T10:20:03.393760+0000","last_undegraded":"2024-04-25T10:20:03.393760+0000","last_fullsized":"2024-04-25T10:20:03.393760+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:07.508595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.413183+0000","last_change":"2024-04-25T10:20:01.335371+0000","last_active":"2024-04-25T10:20:03.413183+0000","last_peered":"2024-04-25T10:20:03.413183+0000","last_clean":"2024-04-25T10:20:03.413183+0000","last_became_active":"2024-04-25T10:20:01.335092+0000","last_became_peered":"2024-04-25T10:20:01.335092+0000","last_unstale":"2024-04-25T10:20:03.413183+0000","last_undegraded":"2024-04-25T10:20:03.413183+0000","last_fullsized":"2024-04-25T10:20:03.413183+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:14:44.129836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.7","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.372130+0000","last_change":"2024-04-25T10:20:02.366426+0000","last_active":"2024-04-25T10:20:03.372130+0000","last_peered":"2024-04-25T10:20:03.372130+0000","last_clean":"2024-04-25T10:20:03.372130+0000","last_became_active":"2024-04-25T10:20:02.366147+0000","last_became_peered":"2024-04-25T10:20:02.366147+0000","last_unstale":"2024-04-25T10:20:03.372130+0000","last_undegraded":"2024-04-25T10:20:03.372130+0000","last_fullsized":"2024-04-25T10:20:03.372130+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:09.854547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.691914+0000","last_change":"2024-04-25T10:20:01.339862+0000","last_active":"2024-04-25T10:20:03.691914+0000","last_peered":"2024-04-25T10:20:03.691914+0000","last_clean":"2024-04-25T10:20:03.691914+0000","last_became_active":"2024-04-25T10:20:01.339331+0000","last_became_peered":"2024-04-25T10:20:01.339331+0000","last_unstale":"2024-04-25T10:20:03.691914+0000","last_undegraded":"2024-04-25T10:20:03.691914+0000","last_fullsized":"2024-04-25T10:20:03.691914+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:54.626590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1d","version":"0'0","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.412804+0000","last_change":"2024-04-25T10:20:02.360346+0000","last_active":"2024-04-25T10:20:03.412804+0000","last_peered":"2024-04-25T10:20:03.412804+0000","last_clean":"2024-04-25T10:20:03.412804+0000","last_became_active":"2024-04-25T10:20:02.353684+0000","last_became_peered":"2024-04-25T10:20:02.353684+0000","last_unstale":"2024-04-25T10:20:03.412804+0000","last_undegraded":"2024-04-25T10:20:03.412804+0000","last_fullsized":"2024-04-25T10:20:03.412804+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:01.297872+0000","last_clean_scrub_stamp":"2024-04-25T10:20:01.297872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:49:37.388922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1a","version":"183'1","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.588049+0000","last_change":"2024-04-25T10:20:01.329000+0000","last_active":"2024-04-25T10:20:03.588049+0000","last_peered":"2024-04-25T10:20:03.588049+0000","last_clean":"2024-04-25T10:20:03.588049+0000","last_became_active":"2024-04-25T10:20:01.328744+0000","last_became_peered":"2024-04-25T10:20:01.328744+0000","last_unstale":"2024-04-25T10:20:03.588049+0000","last_undegraded":"2024-04-25T10:20:03.588049+0000","last_fullsized":"2024-04-25T10:20:03.588049+0000","mapping_epoch":181,"log_start":"0'0","ondisk_log_start":"0'0","created":181,"last_epoch_clean":182,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:00.280403+0000","last_clean_scrub_stamp":"2024-04-25T10:20:00.280403+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:38:48.757660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":482,"reported_epoch":184,"state":"active+clean","last_fresh":"2024-04-25T10:20:03.392644+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:03.392644+0000","last_peered":"2024-04-25T10:20:03.392644+0000","last_clean":"2024-04-25T10:20:03.392644+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:03.392644+0000","last_undegraded":"2024-04-25T10:20:03.392644+0000","last_fullsized":"2024-04-25T10:20:03.392644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":28,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":27,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575071,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60416,"kb_used_data":2864,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311424,"statfs":{"total":96636764160,"available":96574898176,"internally_reserved":0,"allocated":2932736,"data_stored":1685404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":58915275},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575071,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59836,"kb_used_data":2284,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312004,"statfs":{"total":96636764160,"available":96575492096,"internally_reserved":0,"allocated":2338816,"data_stored":1095548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575071,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55668,"kb_used_data":2276,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316172,"statfs":{"total":96636764160,"available":96579760128,"internally_reserved":0,"allocated":2330624,"data_stored":1093410,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575071,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87532,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284308,"statfs":{"total":96636764160,"available":96547131392,"internally_reserved":0,"allocated":2326528,"data_stored":1093936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575071,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87548,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284292,"statfs":{"total":96636764160,"available":96547115008,"internally_reserved":0,"allocated":2334720,"data_stored":1093480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59828,"kb_used_data":2272,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312012,"statfs":{"total":96636764160,"available":96575500288,"internally_reserved":0,"allocated":2326528,"data_stored":1093444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575071,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59824,"kb_used_data":2268,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94312016,"statfs":{"total":96636764160,"available":96575504384,"internally_reserved":0,"allocated":2322432,"data_stored":1093354,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575071,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":65904,"kb_used_data":2852,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305936,"statfs":{"total":96636764160,"available":96569278464,"internally_reserved":0,"allocated":2920448,"data_stored":1683812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:12.117 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:20:12.117 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:20:12.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.367+0000 7f76d7600640 1 Processor -- start 2024-04-25T10:20:12.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.367+0000 7f76d7600640 1 -- start start 2024-04-25T10:20:12.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d7600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f76d00cf640 0x7f76d006ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d7600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f76d00ac8b0 0x7f76d00acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d7600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76d00cfad0 con 0x7f76d015b710 2024-04-25T10:20:12.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d7600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d006f190 con 0x7f76d00ac8b0 2024-04-25T10:20:12.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d7600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76d015e490 con 0x7f76d00cf640 2024-04-25T10:20:12.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d5600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f76d00ac8b0 0x7f76d00acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d4c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f76d015b710 0x7f76d015e920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49196/0 (socket says 172.21.15.165:49196) 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.369+0000 7f76d4c00640 1 -- 172.21.15.165:0/1880381643 learned_addr learned my addr 172.21.15.165:0/1880381643 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.370+0000 7f76cfe00640 1 --2- 172.21.15.165:0/1880381643 >> v2:172.21.15.172:3300/0 conn(0x7f76d00cf640 0x7f76d006ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.370+0000 7f76d5600640 1 -- 172.21.15.165:0/1880381643 >> v2:172.21.15.172:3300/0 conn(0x7f76d00cf640 msgr2=0x7f76d006ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.370+0000 7f76d5600640 1 --2- 172.21.15.165:0/1880381643 >> v2:172.21.15.172:3300/0 conn(0x7f76d00cf640 0x7f76d006ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.370+0000 7f76d5600640 1 -- 172.21.15.165:0/1880381643 >> v1:172.21.15.165:6789/0 conn(0x7f76d015b710 legacy=0x7f76d015e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.370+0000 7f76d5600640 1 -- 172.21.15.165:0/1880381643 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d00cfd80 con 0x7f76d00ac8b0 2024-04-25T10:20:12.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.371+0000 7f76d5600640 1 --2- 172.21.15.165:0/1880381643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00acd00 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f76d006ba10 tx=0x7f76c802a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41c1fc4ec630643d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.371+0000 7f76cf400640 1 -- 172.21.15.165:0/1880381643 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8007740 con 0x7f76d00ac8b0 2024-04-25T10:20:12.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.371+0000 7f76cf400640 1 -- 172.21.15.165:0/1880381643 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c80078e0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.371+0000 7f76cf400640 1 -- 172.21.15.165:0/1880381643 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c8007cb0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 -- 172.21.15.165:0/1880381643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 msgr2=0x7f76d00acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 --2- 172.21.15.165:0/1880381643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00acd00 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f76d006ba10 tx=0x7f76c802a680 comp rx=0 tx=0).stop 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 -- 172.21.15.165:0/1880381643 shutdown_connections 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 --2- 172.21.15.165:0/1880381643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00acd00 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 --2- 172.21.15.165:0/1880381643 >> v2:172.21.15.172:3300/0 conn(0x7f76d00cf640 0x7f76d006ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 -- 172.21.15.165:0/1880381643 >> 172.21.15.165:0/1880381643 conn(0x7f76d006abe0 msgr2=0x7f76d006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 -- 172.21.15.165:0/1880381643 shutdown_connections 2024-04-25T10:20:12.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.372+0000 7f76d7600640 1 -- 172.21.15.165:0/1880381643 wait complete. 2024-04-25T10:20:12.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.374+0000 7f76d7600640 1 Processor -- start 2024-04-25T10:20:12.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.374+0000 7f76d7600640 1 -- start start 2024-04-25T10:20:12.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d7600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00abee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d7600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76d00cf640 0x7f76d0159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d7600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76d015b710 0x7f76d01500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d7600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d015baf0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d7600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76d00d0340 con 0x7f76d015b710 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d7600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d015bc70 con 0x7f76d00cf640 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00abee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d4c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00abee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47850/0 (socket says 172.21.15.165:47850) 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.375+0000 7f76d4c00640 1 -- 172.21.15.165:0/2708521703 learned_addr learned my addr 172.21.15.165:0/2708521703 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76cfe00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76d00cf640 0x7f76d0159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d5600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76d015b710 0x7f76d01500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d4c00640 1 -- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76d00cf640 msgr2=0x7f76d0159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d4c00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76d00cf640 0x7f76d0159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d4c00640 1 -- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76d015b710 msgr2=0x7f76d01500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d4c00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76d015b710 0x7f76d01500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d4c00640 1 -- 172.21.15.165:0/2708521703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d00aa9a0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d5600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76d015b710 0x7f76d01500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:12.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d4c00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00abee0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c3c0 tx=0x7f76c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c4017070 con 0x7f76d00ac8b0 2024-04-25T10:20:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76c4005ce0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76c40072f0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.376+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76d00aac40 con 0x7f76d00ac8b0 2024-04-25T10:20:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.377+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76d00ab200 con 0x7f76d00ac8b0 2024-04-25T10:20:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.378+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76d0073a00 con 0x7f76d00ac8b0 2024-04-25T10:20:12.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.379+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f76c401a070 con 0x7f76d00ac8b0 2024-04-25T10:20:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.379+0000 7f76ccc00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76b006b4d0 0x7f76b006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.380+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f76c4089790 con 0x7f76d00ac8b0 2024-04-25T10:20:12.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.380+0000 7f76cfe00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76b006b4d0 0x7f76b006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.380+0000 7f76cfe00640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76b006b4d0 0x7f76b006d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76d0161540 tx=0x7f76bc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.384+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f76c40583e0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.650+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f76d00ae4f0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.655+0000 7f76ccc00640 1 -- 172.21.15.165:0/2708521703 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v25) v1 ==== 190+0+147 (secure 0 0 0) 0x7f76c4057ce0 con 0x7f76d00ac8b0 2024-04-25T10:20:12.658 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:20:12.658 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:20:12.658 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:20:12.659 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:20:12.659 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:12.659 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76b006b4d0 msgr2=0x7f76b006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76b006b4d0 0x7f76b006d990 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f76d0161540 tx=0x7f76bc008040 comp rx=0 tx=0).stop 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 msgr2=0x7f76d00abee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00abee0 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f76c400c3c0 tx=0x7f76c400c880 comp rx=0 tx=0).stop 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 shutdown_connections 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f76b006b4d0 0x7f76b006d990 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f76d015b710 0x7f76d01500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f76d00cf640 0x7f76d0159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 --2- 172.21.15.165:0/2708521703 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f76d00ac8b0 0x7f76d00abee0 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 >> 172.21.15.165:0/2708521703 conn(0x7f76d006abe0 msgr2=0x7f76d015bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 shutdown_connections 2024-04-25T10:20:12.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.658+0000 7f76d7600640 1 -- 172.21.15.165:0/2708521703 wait complete. 2024-04-25T10:20:12.680 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:12.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.865+0000 7fa170200640 1 Processor -- start 2024-04-25T10:20:12.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.865+0000 7fa170200640 1 -- start start 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.866+0000 7fa170200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa16806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.866+0000 7fa170200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa16806eb10 0x7fa1680ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.866+0000 7fa170200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa1680cee60 0x7fa16815f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.866+0000 7fa170200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa168144b30 con 0x7fa1680cee60 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.866+0000 7fa170200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1680712d0 con 0x7fa16806eb10 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.866+0000 7fa170200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1680715b0 con 0x7fa168144750 2024-04-25T10:20:12.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa1680cee60 0x7fa16815f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa1680cee60 0x7fa16815f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47862/0 (socket says 172.21.15.165:47862) 2024-04-25T10:20:12.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 -- 172.21.15.165:0/3920724071 learned_addr learned my addr 172.21.15.165:0/3920724071 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16d800640 1 --2- 172.21.15.165:0/3920724071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa16806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 -- 172.21.15.165:0/3920724071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 msgr2=0x7fa16806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 --2- 172.21.15.165:0/3920724071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa16806e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 -- 172.21.15.165:0/3920724071 >> v2:172.21.15.172:3300/0 conn(0x7fa16806eb10 msgr2=0x7fa1680ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16ce00640 1 --2- 172.21.15.165:0/3920724071 >> v2:172.21.15.172:3300/0 conn(0x7fa16806eb10 0x7fa1680ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 --2- 172.21.15.165:0/3920724071 >> v2:172.21.15.172:3300/0 conn(0x7fa16806eb10 0x7fa1680ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16e200640 1 -- 172.21.15.165:0/3920724071 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168071890 con 0x7fa1680cee60 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.867+0000 7fa16ce00640 1 --2- 172.21.15.165:0/3920724071 >> v2:172.21.15.172:3300/0 conn(0x7fa16806eb10 0x7fa1680ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:12.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.868+0000 7fa16e200640 1 --2- 172.21.15.165:0/3920724071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1680cee60 0x7fa16815f610 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fa1680cd3d0 tx=0x7fa16002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b26588f2867071c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa167e00640 1 -- 172.21.15.165:0/3920724071 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa160036070 con 0x7fa1680cee60 2024-04-25T10:20:12.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa167e00640 1 -- 172.21.15.165:0/3920724071 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa160031600 con 0x7fa1680cee60 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa167e00640 1 -- 172.21.15.165:0/3920724071 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1600319d0 con 0x7fa1680cee60 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 -- 172.21.15.165:0/3920724071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1680cee60 msgr2=0x7fa16815f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 --2- 172.21.15.165:0/3920724071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1680cee60 0x7fa16815f610 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7fa1680cd3d0 tx=0x7fa16002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 -- 172.21.15.165:0/3920724071 shutdown_connections 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 --2- 172.21.15.165:0/3920724071 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1680cee60 0x7fa16815f610 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 --2- 172.21.15.165:0/3920724071 >> v2:172.21.15.172:3300/0 conn(0x7fa16806eb10 0x7fa1680ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 --2- 172.21.15.165:0/3920724071 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa16806e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 -- 172.21.15.165:0/3920724071 >> 172.21.15.165:0/3920724071 conn(0x7fa16806abe0 msgr2=0x7fa16806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 -- 172.21.15.165:0/3920724071 shutdown_connections 2024-04-25T10:20:12.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.869+0000 7fa170200640 1 -- 172.21.15.165:0/3920724071 wait complete. 2024-04-25T10:20:12.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.871+0000 7fa170200640 1 Processor -- start 2024-04-25T10:20:12.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.871+0000 7fa170200640 1 -- start start 2024-04-25T10:20:12.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa170200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 0x7fa168159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa170200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1680cee60 0x7fa1680ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa170200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa1680ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa170200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa168145450 con 0x7fa16806eb10 2024-04-25T10:20:12.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa170200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16806f020 con 0x7fa1680cee60 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa170200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1681455d0 con 0x7fa168144750 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa16d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 0x7fa168159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa16d800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 0x7fa168159a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47864/0 (socket says 172.21.15.165:47864) 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.872+0000 7fa16d800640 1 -- 172.21.15.165:0/4044476305 learned_addr learned my addr 172.21.15.165:0/4044476305 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16e200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa1680ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16d800640 1 -- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 msgr2=0x7fa1680ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16d800640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa1680ad800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16d800640 1 -- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1680cee60 msgr2=0x7fa1680ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16d800640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1680cee60 0x7fa1680ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16d800640 1 -- 172.21.15.165:0/4044476305 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1680ade80 con 0x7fa16806eb10 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa16d800640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 0x7fa168159a00 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c7f0 tx=0x7fa15c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c015070 con 0x7fa16806eb10 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15c004240 con 0x7fa16806eb10 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15c007620 con 0x7fa16806eb10 2024-04-25T10:20:12.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.873+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1680ae160 con 0x7fa16806eb10 2024-04-25T10:20:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.874+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1680b2b80 con 0x7fa16806eb10 2024-04-25T10:20:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.875+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1680b2fb0 con 0x7fa16806eb10 2024-04-25T10:20:12.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.876+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa15c0079d0 con 0x7fa16806eb10 2024-04-25T10:20:12.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.877+0000 7fa165600640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa14806b500 0x7fa14806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:12.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.877+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fa15c08a4e0 con 0x7fa16806eb10 2024-04-25T10:20:12.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.877+0000 7fa16ce00640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa14806b500 0x7fa14806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:12.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.878+0000 7fa16ce00640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa14806b500 0x7fa14806d9c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa154006b10 tx=0x7fa154008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:12.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:12.881+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa15c059130 con 0x7fa16806eb10 2024-04-25T10:20:13.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.112+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa168073990 con 0x7fa16806eb10 2024-04-25T10:20:13.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.113+0000 7fa165600640 1 -- 172.21.15.165:0/4044476305 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4964 (secure 0 0 0) 0x7fa15c058a30 con 0x7fa16806eb10 2024-04-25T10:20:13.116 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:13.116 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:01.817104+0000","modified":"2024-04-25T10:20:04.840723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11211":{"gid":11211,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1159947775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1159947775},{"type":"v1","addr":"172.21.15.165:6837","nonce":1159947775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:20:13.117 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 130 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa14806b500 msgr2=0x7fa14806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa14806b500 0x7fa14806d9c0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa154006b10 tx=0x7fa154008040 comp rx=0 tx=0).stop 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 msgr2=0x7fa168159a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 0x7fa168159a00 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7fa15c00c7f0 tx=0x7fa15c00ccb0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 shutdown_connections 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa14806b500 0x7fa14806d9c0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa168144750 0x7fa1680ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1680cee60 0x7fa1680ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 --2- 172.21.15.165:0/4044476305 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa16806eb10 0x7fa168159a00 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 >> 172.21.15.165:0/4044476305 conn(0x7fa16806abe0 msgr2=0x7fa168143800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 shutdown_connections 2024-04-25T10:20:13.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.116+0000 7fa170200640 1 -- 172.21.15.165:0/4044476305 wait complete. 2024-04-25T10:20:13.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 130, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:01.817104+0000', 'modified': '2024-04-25T10:20:04.840723+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11211}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11211': {'gid': 11211, 'name': 'c', 'rank': 0, 'incarnation': 126, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6837/1159947775', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 1159947775}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 1159947775}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [28], 'metadata_pool': 27, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:13.539 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:20:13.539 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:20:13.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.746+0000 7f045ea00640 1 Processor -- start 2024-04-25T10:20:13.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.746+0000 7f045ea00640 1 -- start start 2024-04-25T10:20:13.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.748+0000 7f045ea00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0458145140 0x7f04580ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.748+0000 7f045ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 0x7f04580c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.748+0000 7f045ea00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0458145520 con 0x7f04580c5030 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.748+0000 7f045ea00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04580ced90 con 0x7f0458145140 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.748+0000 7f045ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0458071270 con 0x7f04580ce950 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.749+0000 7f045d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0458145140 0x7f04580ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.749+0000 7f045cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 0x7f04580c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.749+0000 7f045d600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0458145140 0x7f04580ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:39338/0 (socket says 172.21.15.165:39338) 2024-04-25T10:20:13.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.749+0000 7f045d600640 1 -- 172.21.15.165:0/2127195482 learned_addr learned my addr 172.21.15.165:0/2127195482 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:13.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.749+0000 7f045cc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 0x7f04580c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:38900/0 (socket says 172.21.15.165:38900) 2024-04-25T10:20:13.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.750+0000 7f045cc00640 1 -- 172.21.15.165:0/2127195482 >> v2:172.21.15.172:3300/0 conn(0x7f0458145140 msgr2=0x7f04580ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.750+0000 7f045cc00640 1 --2- 172.21.15.165:0/2127195482 >> v2:172.21.15.172:3300/0 conn(0x7f0458145140 0x7f04580ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.750+0000 7f045cc00640 1 -- 172.21.15.165:0/2127195482 >> v1:172.21.15.172:6789/0 conn(0x7f04580c5030 legacy=0x7f045806d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.750+0000 7f045cc00640 1 -- 172.21.15.165:0/2127195482 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458071550 con 0x7f04580ce950 2024-04-25T10:20:13.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.751+0000 7f045cc00640 1 --2- 172.21.15.165:0/2127195482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 0x7f04580c4af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f04440097a0 tx=0x7f044402a0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e865781499caf3f1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.752+0000 7f0457e00640 1 -- 172.21.15.165:0/2127195482 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444037070 con 0x7f04580ce950 2024-04-25T10:20:13.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.752+0000 7f0457e00640 1 -- 172.21.15.165:0/2127195482 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044402ae40 con 0x7f04580ce950 2024-04-25T10:20:13.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.752+0000 7f0457e00640 1 -- 172.21.15.165:0/2127195482 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444032660 con 0x7f04580ce950 2024-04-25T10:20:13.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 -- 172.21.15.165:0/2127195482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 msgr2=0x7f04580c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 --2- 172.21.15.165:0/2127195482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 0x7f04580c4af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f04440097a0 tx=0x7f044402a0f0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 -- 172.21.15.165:0/2127195482 shutdown_connections 2024-04-25T10:20:13.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 --2- 172.21.15.165:0/2127195482 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580ce950 0x7f04580c4af0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 --2- 172.21.15.165:0/2127195482 >> v2:172.21.15.172:3300/0 conn(0x7f0458145140 0x7f04580ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 -- 172.21.15.165:0/2127195482 >> 172.21.15.165:0/2127195482 conn(0x7f0458061cc0 msgr2=0x7f045806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 -- 172.21.15.165:0/2127195482 shutdown_connections 2024-04-25T10:20:13.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.753+0000 7f045ea00640 1 -- 172.21.15.165:0/2127195482 wait complete. 2024-04-25T10:20:13.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.754+0000 7f045ea00640 1 Processor -- start 2024-04-25T10:20:13.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.755+0000 7f045ea00640 1 -- start start 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045ea00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580c5030 0x7f04580a9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 0x7f045815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045ea00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0458145140 0x7f04580cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045ea00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580c54a0 con 0x7f04580ce950 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045ea00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0458071b10 con 0x7f0458145140 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045ea00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04580c5620 con 0x7f04580c5030 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 0x7f045815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 0x7f045815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47874/0 (socket says 172.21.15.165:47874) 2024-04-25T10:20:13.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 -- 172.21.15.165:0/3546391168 learned_addr learned my addr 172.21.15.165:0/3546391168 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045d600640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580c5030 0x7f04580a9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045e000640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0458145140 0x7f04580cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 -- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580c5030 msgr2=0x7f04580a9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580c5030 0x7f04580a9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 -- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0458145140 msgr2=0x7f04580cdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0458145140 0x7f04580cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.756+0000 7f045cc00640 1 -- 172.21.15.165:0/3546391168 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0458132fe0 con 0x7f04580ce950 2024-04-25T10:20:13.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.757+0000 7f045cc00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 0x7f045815b2c0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f0444035040 tx=0x7f04440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.757+0000 7f0455600640 1 -- 172.21.15.165:0/3546391168 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444041020 con 0x7f04580ce950 2024-04-25T10:20:13.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.757+0000 7f0455600640 1 -- 172.21.15.165:0/3546391168 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044402ac40 con 0x7f04580ce950 2024-04-25T10:20:13.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.757+0000 7f0455600640 1 -- 172.21.15.165:0/3546391168 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444037040 con 0x7f04580ce950 2024-04-25T10:20:13.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.757+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04581332c0 con 0x7f04580ce950 2024-04-25T10:20:13.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.757+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0458133880 con 0x7f04580ce950 2024-04-25T10:20:13.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.759+0000 7f0455600640 1 -- 172.21.15.165:0/3546391168 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0444044020 con 0x7f04580ce950 2024-04-25T10:20:13.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.760+0000 7f0455600640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.760+0000 7f0455600640 1 -- 172.21.15.165:0/3546391168 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04440aa960 con 0x7f04580ce950 2024-04-25T10:20:13.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.761+0000 7f045d600640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.761+0000 7f0462e2a740 1 Processor -- start 2024-04-25T10:20:13.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.761+0000 7f0462e2a740 1 -- start start 2024-04-25T10:20:13.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.761+0000 7f045d600640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f04580b2660 tx=0x7f044c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 0x55ce12332970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1230af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce123092f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f0462e2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1226e620 con 0x55ce1230ab30 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce12217ca0 con 0x55ce12332eb0 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce1226a6c0 con 0x55ce123537a0 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f045d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 0x55ce12332970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f045d600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 0x55ce12332970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47884/0 (socket says 172.21.15.165:47884) 2024-04-25T10:20:13.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.762+0000 7f045d600640 1 -- 172.21.15.165:0/191242751 learned_addr learned my addr 172.21.15.165:0/191242751 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045e000640 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce123092f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045cc00640 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1230af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045d600640 1 -- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 msgr2=0x55ce123092f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045d600640 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce123092f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045d600640 1 -- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 msgr2=0x55ce1230af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045d600640 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1230af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045d600640 1 -- 172.21.15.165:0/191242751 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce1221ef40 con 0x55ce1230ab30 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f045d600640 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 0x55ce12332970 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f044c010a20 tx=0x7f044c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1eaa1d5bec0d9d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f0449c00640 1 -- 172.21.15.165:0/191242751 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c01c050 con 0x55ce1230ab30 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f0449c00640 1 -- 172.21.15.165:0/191242751 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c026070 con 0x55ce1230ab30 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.763+0000 7f0449c00640 1 -- 172.21.15.165:0/191242751 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00f040 con 0x55ce1230ab30 2024-04-25T10:20:13.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 -- 172.21.15.165:0/191242751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 msgr2=0x55ce12332970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 0x55ce12332970 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f044c010a20 tx=0x7f044c013000 comp rx=0 tx=0).stop 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 -- 172.21.15.165:0/191242751 shutdown_connections 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce123092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1230af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 --2- 172.21.15.165:0/191242751 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce1230ab30 0x55ce12332970 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 -- 172.21.15.165:0/191242751 >> 172.21.15.165:0/191242751 conn(0x55ce12307d50 msgr2=0x55ce12350730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 -- 172.21.15.165:0/191242751 shutdown_connections 2024-04-25T10:20:13.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.764+0000 7f0462e2a740 1 -- 172.21.15.165:0/191242751 wait complete. 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.765+0000 7f0462e2a740 1 -- start start 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.765+0000 7f0462e2a740 1 Processor -- start 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1232e280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 0x55ce123383c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12335290 0x55ce12335710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce123324e0 con 0x55ce123537a0 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce1220d1f0 con 0x55ce12332eb0 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce12331ed0 con 0x55ce12335290 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f0462e2a740 10 client.0 _ll_register_callbacks cb 0x55ce1222aad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 0x55ce123383c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 0x55ce123383c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47892/0 (socket says 172.21.15.165:47892) 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.766+0000 7f045cc00640 1 -- 172.21.15.165:0/3573306115 learned_addr learned my addr 172.21.15.165:0/3573306115 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045e000640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12335290 0x55ce12335710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045cc00640 1 -- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12335290 msgr2=0x55ce12335710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045d600640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1232e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045cc00640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12335290 0x55ce12335710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045cc00640 1 -- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 msgr2=0x55ce1232e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045cc00640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1232e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045cc00640 1 -- 172.21.15.165:0/3573306115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce1222cc50 con 0x55ce123537a0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f045cc00640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 0x55ce123383c0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f044402a6e0 tx=0x7f04440795b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce26e7eaffefaee1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f043e000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f043e000640 1 -- 172.21.15.165:0/3573306115 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444090020 con 0x55ce123537a0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f043e000640 1 -- 172.21.15.165:0/3573306115 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f04440a3e00 con 0x55ce123537a0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.767+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f044c026070 con 0x55ce123537a0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f043e000640 1 -- 172.21.15.165:0/3573306115 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04440815c0 con 0x55ce123537a0 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 10 client.11376 fetch_fsmap learned FSMap version 130 2024-04-25T10:20:13.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x55ce11f832d0 con 0x55ce123537a0 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f043e000640 1 -- 172.21.15.165:0/3573306115 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c026070 con 0x55ce123537a0 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f043e000640 1 -- 172.21.15.165:0/3573306115 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 130) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0444079110 con 0x55ce123537a0 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 10 client.11376 fetch_fsmap finished waiting for FSMap version 130 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 10 client.11376 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 20 client.11376 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12334040 0x55ce12363f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 4 client.11376 mds_command: new command op to 11211 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 --> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce1227bb60 con 0x55ce12334040 2024-04-25T10:20:13.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.768+0000 7f045d600640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12334040 0x55ce12363f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.769+0000 7f045d600640 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12334040 0x55ce12363f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b63f845f2f911c server_cookie=c9eeb1e28409b298 in_seq=0 out_seq=0 2024-04-25T10:20:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.769+0000 7f043e000640 10 client.11376 ms_handle_connect on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:13.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f043e000640 1 -- 172.21.15.165:0/3573306115 <== mds.0 v2:172.21.15.165:6835/1159947775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ce1227bb60 con 0x55ce12334040 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f043e000640 10 client.11376 handle_command_reply: tid=0 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 client.11376 shutdown 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 msgr2=0x55ce123383c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 0x55ce123383c0 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f044402a6e0 tx=0x7f04440795b0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 shutdown_connections 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12334040 0x55ce12363f50 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12335290 0x55ce12335710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce123537a0 0x55ce123383c0 unknown :-1 s=CLOSED pgs=2615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f043e000640 0 client.11376 ms_handle_reset on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 --2- 172.21.15.165:0/3573306115 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce1232e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 >> 172.21.15.165:0/3573306115 conn(0x55ce1230ab30 msgr2=0x55ce123048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 shutdown_connections 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 1 -- 172.21.15.165:0/3573306115 wait complete. 2024-04-25T10:20:13.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.770+0000 7f0462e2a740 20 client.11376 trim_cache size 0 max 16384 2024-04-25T10:20:13.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.787+0000 7f0462e2a740 1 Processor -- start 2024-04-25T10:20:13.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.787+0000 7f0462e2a740 1 -- start start 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce12369f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f0462e2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04581ff520 con 0x55ce12335290 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04580a9100 con 0x55ce12332eb0 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04581fef20 con 0x55ce123537a0 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47898/0 (socket says 172.21.15.165:47898) 2024-04-25T10:20:13.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.788+0000 7f045cc00640 1 -- 172.21.15.165:0/652053705 learned_addr learned my addr 172.21.15.165:0/652053705 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:13.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f045cc00640 1 -- 172.21.15.165:0/652053705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 msgr2=0x55ce12369f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f045cc00640 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce12369f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f045cc00640 1 -- 172.21.15.165:0/652053705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 msgr2=0x55ce123048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f045cc00640 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f045cc00640 1 -- 172.21.15.165:0/652053705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce1204d6b0 con 0x55ce12335290 2024-04-25T10:20:13.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f045cc00640 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f0444006b10 tx=0x7f044403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cab2069bb0eb03c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f0449200640 1 -- 172.21.15.165:0/652053705 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444090040 con 0x55ce12335290 2024-04-25T10:20:13.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f0449200640 1 -- 172.21.15.165:0/652053705 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0444077250 con 0x55ce12335290 2024-04-25T10:20:13.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.789+0000 7f0449200640 1 -- 172.21.15.165:0/652053705 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04440789f0 con 0x55ce12335290 2024-04-25T10:20:13.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 -- 172.21.15.165:0/652053705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 msgr2=0x55ce12304df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7f0444006b10 tx=0x7f044403f000 comp rx=0 tx=0).stop 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 -- 172.21.15.165:0/652053705 shutdown_connections 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce123537a0 0x55ce12369f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 --2- 172.21.15.165:0/652053705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 -- 172.21.15.165:0/652053705 >> 172.21.15.165:0/652053705 conn(0x55ce1230ab30 msgr2=0x55ce1232e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 -- 172.21.15.165:0/652053705 shutdown_connections 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.790+0000 7f0462e2a740 1 -- 172.21.15.165:0/652053705 wait complete. 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.791+0000 7f0462e2a740 1 -- start start 2024-04-25T10:20:13.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.791+0000 7f0462e2a740 1 Processor -- start 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce123537a0 0x55ce12354190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce12333ec0 con 0x55ce12335290 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce12337c20 con 0x55ce123537a0 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ce12330d00 con 0x55ce12332eb0 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f0462e2a740 10 client.0 _ll_register_callbacks cb 0x7f0444009400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f045cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47910/0 (socket says 172.21.15.165:47910) 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.792+0000 7f045cc00640 1 -- 172.21.15.165:0/1214945093 learned_addr learned my addr 172.21.15.165:0/1214945093 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045d600640 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045cc00640 1 -- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12332eb0 msgr2=0x55ce123048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045cc00640 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045cc00640 1 -- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce123537a0 msgr2=0x55ce12354190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045cc00640 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce123537a0 0x55ce12354190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045cc00640 1 -- 172.21.15.165:0/1214945093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ce12370610 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f045cc00640 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f0444079070 tx=0x7f04440a8b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f49b691cbbba066d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f043e000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f043e000640 1 -- 172.21.15.165:0/1214945093 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444090040 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f043e000640 1 -- 172.21.15.165:0/1214945093 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0444078300 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0444077250 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f043e000640 1 -- 172.21.15.165:0/1214945093 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0444078500 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f0462e2a740 10 client.11382 fetch_fsmap learned FSMap version 130 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.793+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x55ce12370610 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f043e000640 1 -- 172.21.15.165:0/1214945093 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0444077250 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f043e000640 1 -- 172.21.15.165:0/1214945093 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 130) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f04440a2050 con 0x55ce12335290 2024-04-25T10:20:13.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f0462e2a740 10 client.11382 fetch_fsmap finished waiting for FSMap version 130 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f0462e2a740 10 client.11382 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f0462e2a740 20 client.11382 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f0462e2a740 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12350980 0x55ce123546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f0462e2a740 4 client.11382 mds_command: new command op to 11211 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 --> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ce123084b0 con 0x55ce12350980 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.794+0000 7f045d600640 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12350980 0x55ce123546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f045d600640 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12350980 0x55ce123546d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34f99181859a0d92 server_cookie=e347916c521e087c in_seq=0 out_seq=0 2024-04-25T10:20:13.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f043e000640 10 client.11382 ms_handle_connect on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f043e000640 1 -- 172.21.15.165:0/1214945093 <== mds.0 v2:172.21.15.165:6835/1159947775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x55ce123084b0 con 0x55ce12350980 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f043e000640 10 client.11382 handle_command_reply: tid=0 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 client.11382 shutdown 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 msgr2=0x55ce12304df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7f0444079070 tx=0x7f04440a8b30 comp rx=0 tx=0).stop 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 shutdown_connections 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ce12350980 0x55ce123546d0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ce123537a0 0x55ce12354190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ce12335290 0x55ce12304df0 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 --2- 172.21.15.165:0/1214945093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ce12332eb0 0x55ce123048b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 >> 172.21.15.165:0/1214945093 conn(0x55ce1230ab30 msgr2=0x55ce123092f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f043e000640 0 client.11382 ms_handle_reset on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 shutdown_connections 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 1 -- 172.21.15.165:0/1214945093 wait complete. 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.795+0000 7f0462e2a740 20 client.11382 trim_cache size 0 max 16384 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 11211, 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:20:13.801 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 11.445425865000001, 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 130, 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 184, 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 184, 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 16.945432525000001 2024-04-25T10:20:13.802 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f043806b4d0 msgr2=0x7f043806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f043806b4d0 0x7f043806d990 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f04580b2660 tx=0x7f044c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 msgr2=0x7f045815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 0x7f045815b2c0 secure :-1 s=READY pgs=2613 cs=0 l=1 rev1=1 crypto rx=0x7f0444035040 tx=0x7f04440046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 shutdown_connections 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f043806b4d0 0x7f043806d990 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0458145140 0x7f04580cdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f04580ce950 0x7f045815b2c0 unknown :-1 s=CLOSED pgs=2613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 --2- 172.21.15.165:0/3546391168 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f04580c5030 0x7f04580a9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 >> 172.21.15.165:0/3546391168 conn(0x7f0458061cc0 msgr2=0x7f04580cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 shutdown_connections 2024-04-25T10:20:13.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.796+0000 7f045ea00640 1 -- 172.21.15.165:0/3546391168 wait complete. 2024-04-25T10:20:13.815 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:20:14.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:13.999+0000 7f139a600640 1 Processor -- start 2024-04-25T10:20:14.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.000+0000 7f139a600640 1 -- start start 2024-04-25T10:20:14.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f139a600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f13940cc8e0 0x7f13941625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f139a600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13940cce50 con 0x7f139415dc90 2024-04-25T10:20:14.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f139a600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1394161af0 con 0x7f13940d1810 2024-04-25T10:20:14.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f139a600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1394161dd0 con 0x7f13940cc8e0 2024-04-25T10:20:14.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f1393400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f139415dc90 0x7f139415ee50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49208/0 (socket says 172.21.15.165:49208) 2024-04-25T10:20:14.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f1393400640 1 -- 172.21.15.165:0/1427624288 learned_addr learned my addr 172.21.15.165:0/1427624288 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3645110234 0 0) 0x7f13940cce50 con 0x7f139415dc90 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f1392a00640 1 --2- 172.21.15.165:0/1427624288 >> v2:172.21.15.172:3300/0 conn(0x7f13940cc8e0 0x7f13941625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.001+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1378003520 con 0x7f139415dc90 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3200199467 0 0) 0x7f1378003520 con 0x7f139415dc90 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 >> v2:172.21.15.172:3300/0 conn(0x7f13940cc8e0 msgr2=0x7f13941625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 --2- 172.21.15.165:0/1427624288 >> v2:172.21.15.172:3300/0 conn(0x7f13940cc8e0 0x7f13941625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 >> v1:172.21.15.172:6789/0 conn(0x7f13940d1810 legacy=0x7f13940cfde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13941620b0 con 0x7f139415dc90 2024-04-25T10:20:14.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f1384001670 con 0x7f139415dc90 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 >> v1:172.21.15.165:6789/0 conn(0x7f139415dc90 legacy=0x7f139415ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378003a00 0x7f1378003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1378004800 0x7f1378004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1378005290 0x7f137800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394161dd0 con 0x7f1378005290 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f137800c1e0 con 0x7f1378003a00 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f137800c3e0 con 0x7f1378004800 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.002+0000 7f1393e00640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1378005290 0x7f137800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393400640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378003a00 0x7f1378003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393e00640 1 -- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1378004800 msgr2=0x7f1378004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393e00640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1378004800 0x7f1378004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393e00640 1 -- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378003a00 msgr2=0x7f1378003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393e00640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378003a00 0x7f1378003e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393e00640 1 -- 172.21.15.165:0/1427624288 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1378003520 con 0x7f1378005290 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.003+0000 7f1393e00640 1 -- 172.21.15.165:0/1427624288 reap_dead start 2024-04-25T10:20:14.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.004+0000 7f1393e00640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1378005290 0x7f137800b9c0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f138c003db0 tx=0x7f1378005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbf30d2bb4128a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.004+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c003980 con 0x7f1378005290 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.004+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c007240 con 0x7f1378005290 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.004+0000 7f1392000640 1 -- 172.21.15.165:0/1427624288 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c007630 con 0x7f1378005290 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.004+0000 7f139a600640 1 -- 172.21.15.165:0/1427624288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1378005290 msgr2=0x7f137800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.004+0000 7f139a600640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1378005290 0x7f137800b9c0 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f138c003db0 tx=0x7f1378005830 comp rx=0 tx=0).stop 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.005+0000 7f139a600640 1 -- 172.21.15.165:0/1427624288 shutdown_connections 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.005+0000 7f139a600640 1 --2- 172.21.15.165:0/1427624288 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1378005290 0x7f137800b9c0 unknown :-1 s=CLOSED pgs=2619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.005+0000 7f139a600640 1 -- 172.21.15.165:0/1427624288 >> 172.21.15.165:0/1427624288 conn(0x7f139406f590 msgr2=0x7f139406f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.005+0000 7f139a600640 1 -- 172.21.15.165:0/1427624288 shutdown_connections 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.005+0000 7f139a600640 1 -- 172.21.15.165:0/1427624288 wait complete. 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.006+0000 7f139a600640 1 Processor -- start 2024-04-25T10:20:14.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.006+0000 7f139a600640 1 -- start start 2024-04-25T10:20:14.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.007+0000 7f139a600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378005290 0x7f13940cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.007+0000 7f139a600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13940cb2c0 0x7f13941f7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.007+0000 7f139a600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 0x7f13940ce2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.007+0000 7f139a600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139415e1f0 con 0x7f13941f7c80 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.007+0000 7f139a600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f139415e070 con 0x7f1378005290 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.007+0000 7f139a600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13941625d0 con 0x7f13940cb2c0 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 0x7f13940ce2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 0x7f13940ce2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47922/0 (socket says 172.21.15.165:47922) 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 -- 172.21.15.165:0/1805773782 learned_addr learned my addr 172.21.15.165:0/1805773782 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1392a00640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13940cb2c0 0x7f13941f7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 -- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13940cb2c0 msgr2=0x7f13941f7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13940cb2c0 0x7f13941f7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 -- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378005290 msgr2=0x7f13940cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378005290 0x7f13940cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 -- 172.21.15.165:0/1805773782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13941f3ba0 con 0x7f13941f7c80 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.008+0000 7f1393e00640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 0x7f13940ce2b0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f138c003ed0 tx=0x7f138c0036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.009+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c041070 con 0x7f13941f7c80 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.009+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138c0067d0 con 0x7f13941f7c80 2024-04-25T10:20:14.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.009+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138c006b80 con 0x7f13941f7c80 2024-04-25T10:20:14.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.009+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941f3e80 con 0x7f13941f7c80 2024-04-25T10:20:14.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.009+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13940cb820 con 0x7f13941f7c80 2024-04-25T10:20:14.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.010+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1394073990 con 0x7f13941f7c80 2024-04-25T10:20:14.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.011+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f138c044070 con 0x7f13941f7c80 2024-04-25T10:20:14.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.012+0000 7f138b400640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f137006b4d0 0x7f137006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:14.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.012+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f138c0a96d0 con 0x7f13941f7c80 2024-04-25T10:20:14.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.012+0000 7f1393400640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f137006b4d0 0x7f137006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:14.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.013+0000 7f1393400640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f137006b4d0 0x7f137006d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f139415ffe0 tx=0x7f1384009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:14.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.016+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f138c077cd0 con 0x7f13941f7c80 2024-04-25T10:20:14.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.250+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1394070330 con 0x7f13941f7c80 2024-04-25T10:20:14.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.251+0000 7f138b400640 1 -- 172.21.15.165:0/1805773782 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v130) v1 ==== 53+0+73 (secure 0 0 0) 0x7f138c07bd70 con 0x7f13941f7c80 2024-04-25T10:20:14.254 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f137006b4d0 msgr2=0x7f137006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f137006b4d0 0x7f137006d990 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f139415ffe0 tx=0x7f1384009040 comp rx=0 tx=0).stop 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 msgr2=0x7f13940ce2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 0x7f13940ce2b0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f138c003ed0 tx=0x7f138c0036f0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 shutdown_connections 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f137006b4d0 0x7f137006d990 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13941f7c80 0x7f13940ce2b0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13940cb2c0 0x7f13941f7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 --2- 172.21.15.165:0/1805773782 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1378005290 0x7f13940cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 >> 172.21.15.165:0/1805773782 conn(0x7f139406f590 msgr2=0x7f1394160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 shutdown_connections 2024-04-25T10:20:14.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:14.254+0000 7f139a600640 1 -- 172.21.15.165:0/1805773782 wait complete. 2024-04-25T10:20:14.275 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:20:14.275 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:20:14.276 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:14.276 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.276 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:20:14.324 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:14.325 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41551sec preferred_lft 41551sec 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:20:14.326 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:20:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.327 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:20:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:20:14.327 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:14.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.328 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:20:14.402 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:20:14.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.407 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:20:14.455 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:20:14.456 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:20:14.456 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:20:14.456 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:20:14.456 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:20:14.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.456 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:20:14.456 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:20:14.456 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:20:14.456 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:20:14.457 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:20:14.457 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:20:14.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.598 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:20:14.648 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:20:14.648 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:20:14.649 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:14.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.649 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:20:14.728 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:14.728 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:14.781 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:14.781 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:14.781 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:14.830 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:20:14.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:14.830 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:14.949 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:14.950 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:14.950 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:20:14.973 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:14.973 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:14.973 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:20:14.973 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:20:14.973 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:20:15.001 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:15.001 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:15.001 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:20:15.002 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:20:15.002 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:20:15.002 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:20:15.002 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:20:15.002 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:20:15.004 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:20:15.005 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:20:15.005 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:20:15.005 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:20:15.005 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:20:15.009 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:20:15.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:15.009 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:15.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:15.080 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:20:15.144 INFO:teuthology.orchestra.run.smithi165.stdout:client7162 2024-04-25T10:20:15.145 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:15.145 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:15.193 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:15.193 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:15.291 DEBUG:teuthology.orchestra.run.smithi165:> stdin-killer -- python3 -c ' 2024-04-25T10:20:15.291 DEBUG:teuthology.orchestra.run.smithi165:> import os 2024-04-25T10:20:15.291 DEBUG:teuthology.orchestra.run.smithi165:> import uuid 2024-04-25T10:20:15.291 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:20:15.291 DEBUG:teuthology.orchestra.run.smithi165:> n = 1000 2024-04-25T10:20:15.291 DEBUG:teuthology.orchestra.run.smithi165:> create_hard_links = False 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> if 0 > 0: 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> create_hard_links = True 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> path = "/home/ubuntu/cephtest/mnt.0/scrub-new-files/file" 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> dpath = os.path.dirname(path) 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> fnameprefix = os.path.basename(path) 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> os.makedirs(dpath, exist_ok=True) 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> try: 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> dirfd = os.open(dpath, os.O_DIRECTORY) 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> for i in range(n): 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> fpath = os.path.join(dpath, f"{fnameprefix}_{i}") 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> with open(fpath, '"'"'w'"'"') as f: 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> f.write(f"{i}") 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:20:15.292 DEBUG:teuthology.orchestra.run.smithi165:> f.flush() 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(f.fileno()) 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> os.unlink(fpath) 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(dirfd) 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> if create_hard_links: 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> for j in range(0): 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> os.system(f"ln {fpath} {dpath}/{fnameprefix}_{i}_{uuid.uuid4()}") 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> if False: 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> os.fsync(dirfd) 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> finally: 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> os.close(dirfd) 2024-04-25T10:20:15.293 DEBUG:teuthology.orchestra.run.smithi165:> ' 2024-04-25T10:20:16.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-25T10:20:16.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:16.287 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.13:0 scrub status 2024-04-25T10:20:16.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.523+0000 7f3460a00640 1 Processor -- start 2024-04-25T10:20:16.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.524+0000 7f3460a00640 1 -- start start 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.525+0000 7f3460a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f345c15dc20 0x7f345c0ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.525+0000 7f3460a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c06cda0 con 0x7f345c161710 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.525+0000 7f3460a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c15e000 con 0x7f345c0d32f0 2024-04-25T10:20:16.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.525+0000 7f3460a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0d1de0 con 0x7f345c15dc20 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.525+0000 7f3459e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f345c15dc20 0x7f345c0ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.526+0000 7f3451400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f345c161710 0x7f345c06bf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:49214/0 (socket says 172.21.15.165:49214) 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.526+0000 7f3459e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f345c15dc20 0x7f345c0ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47928/0 (socket says 172.21.15.165:47928) 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.526+0000 7f3451400640 1 -- 172.21.15.165:0/419254427 learned_addr learned my addr 172.21.15.165:0/419254427 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.526+0000 7f3459e00640 1 -- 172.21.15.165:0/419254427 >> v1:172.21.15.172:6789/0 conn(0x7f345c0d32f0 legacy=0x7f345c06c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.526+0000 7f3459e00640 1 -- 172.21.15.165:0/419254427 >> v1:172.21.15.165:6789/0 conn(0x7f345c161710 legacy=0x7f345c06bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.526+0000 7f3459e00640 1 -- 172.21.15.165:0/419254427 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c161c60 con 0x7f345c15dc20 2024-04-25T10:20:16.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.527+0000 7f3459e00640 1 --2- 172.21.15.165:0/419254427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c15dc20 0x7f345c0ce670 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d90 tx=0x7f344c02a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703f4295d15dbcbe server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.527+0000 7f3459400640 1 -- 172.21.15.165:0/419254427 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c007740 con 0x7f345c15dc20 2024-04-25T10:20:16.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3459400640 1 -- 172.21.15.165:0/419254427 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c0078e0 con 0x7f345c15dc20 2024-04-25T10:20:16.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3459400640 1 -- 172.21.15.165:0/419254427 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c007cb0 con 0x7f345c15dc20 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3460a00640 1 -- 172.21.15.165:0/419254427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c15dc20 msgr2=0x7f345c0ce670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3460a00640 1 --2- 172.21.15.165:0/419254427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c15dc20 0x7f345c0ce670 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f344c009d90 tx=0x7f344c02a6a0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3460a00640 1 -- 172.21.15.165:0/419254427 shutdown_connections 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3460a00640 1 --2- 172.21.15.165:0/419254427 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c15dc20 0x7f345c0ce670 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3460a00640 1 -- 172.21.15.165:0/419254427 >> 172.21.15.165:0/419254427 conn(0x7f345c0d0420 msgr2=0x7f345c0d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.528+0000 7f3460a00640 1 -- 172.21.15.165:0/419254427 shutdown_connections 2024-04-25T10:20:16.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.529+0000 7f3460a00640 1 -- 172.21.15.165:0/419254427 wait complete. 2024-04-25T10:20:16.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.530+0000 7f3460a00640 1 Processor -- start 2024-04-25T10:20:16.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.530+0000 7f3460a00640 1 -- start start 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.531+0000 7f3460a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 0x7f345c14cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.531+0000 7f3460a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f345c15dc20 0x7f345c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.531+0000 7f3460a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f345c161710 0x7f345c160120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.531+0000 7f3460a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c162500 con 0x7f345c0d32f0 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.531+0000 7f3460a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c162220 con 0x7f345c161710 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.531+0000 7f3460a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0b5e50 con 0x7f345c15dc20 2024-04-25T10:20:16.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 0x7f345c14cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 0x7f345c14cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47930/0 (socket says 172.21.15.165:47930) 2024-04-25T10:20:16.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 -- 172.21.15.165:0/1897985561 learned_addr learned my addr 172.21.15.165:0/1897985561 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3451400640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f345c15dc20 0x7f345c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 -- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f345c15dc20 msgr2=0x7f345c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f345c15dc20 0x7f345c15fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f345a800640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f345c161710 0x7f345c160120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 -- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f345c161710 msgr2=0x7f345c160120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f345c161710 0x7f345c160120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f3459e00640 1 -- 172.21.15.165:0/1897985561 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0cba80 con 0x7f345c0d32f0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.532+0000 7f345a800640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f345c161710 0x7f345c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.533+0000 7f3459e00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 0x7f345c14cba0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f344c009eb0 tx=0x7f344c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.533+0000 7f3452a00640 1 -- 172.21.15.165:0/1897985561 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c03f070 con 0x7f345c0d32f0 2024-04-25T10:20:16.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.533+0000 7f3452a00640 1 -- 172.21.15.165:0/1897985561 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f344c031d00 con 0x7f345c0d32f0 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.533+0000 7f3452a00640 1 -- 172.21.15.165:0/1897985561 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c044660 con 0x7f345c0d32f0 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.533+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c0cbd60 con 0x7f345c0d32f0 2024-04-25T10:20:16.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.533+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c0cc320 con 0x7f345c0d32f0 2024-04-25T10:20:16.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.537+0000 7f3452a00640 1 -- 172.21.15.165:0/1897985561 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f344c03b410 con 0x7f345c0d32f0 2024-04-25T10:20:16.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.537+0000 7f34637f7740 1 Processor -- start 2024-04-25T10:20:16.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.538+0000 7f34637f7740 1 -- start start 2024-04-25T10:20:16.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.538+0000 7f3452a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f343806b4d0 0x7f343806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.538+0000 7f3452a00640 1 -- 172.21.15.165:0/1897985561 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f344c0abdc0 con 0x7f345c0d32f0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2d2d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f343806b4d0 0x7f343806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2d0f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f34637f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe2fbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f34637f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe234c70 con 0x55ffbe2fb670 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe1ddcf0 con 0x55ffbe2d0bb0 2024-04-25T10:20:16.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe230d10 con 0x55ffbe30d280 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe2fbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe2fbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47944/0 (socket says 172.21.15.165:47944) 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 -- 172.21.15.165:0/1165067037 learned_addr learned my addr 172.21.15.165:0/1165067037 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 -- 172.21.15.165:0/1165067037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 msgr2=0x55ffbe2d0f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2d0f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 -- 172.21.15.165:0/1165067037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 msgr2=0x55ffbe2d2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.539+0000 7f345a800640 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2d2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.540+0000 7f345a800640 1 -- 172.21.15.165:0/1165067037 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbe0137d0 con 0x55ffbe2fb670 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.540+0000 7f345a800640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f343806b4d0 0x7f343806d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3454015530 tx=0x7f345400ad10 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.540+0000 7f345a800640 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe2fbaf0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f34540176e0 tx=0x7f3454018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41186a81cf90574d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.540+0000 7f344a600640 1 -- 172.21.15.165:0/1165067037 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345401e070 con 0x55ffbe2fb670 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.540+0000 7f344a600640 1 -- 172.21.15.165:0/1165067037 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34540167d0 con 0x55ffbe2fb670 2024-04-25T10:20:16.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.540+0000 7f344a600640 1 -- 172.21.15.165:0/1165067037 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400f040 con 0x55ffbe2fb670 2024-04-25T10:20:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 -- 172.21.15.165:0/1165067037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 msgr2=0x55ffbe2fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe2fbaf0 secure :-1 s=READY pgs=2624 cs=0 l=1 rev1=1 crypto rx=0x7f34540176e0 tx=0x7f3454018000 comp rx=0 tx=0).stop 2024-04-25T10:20:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 -- 172.21.15.165:0/1165067037 shutdown_connections 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe2fbaf0 unknown :-1 s=CLOSED pgs=2624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2d0f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 --2- 172.21.15.165:0/1165067037 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2d2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 -- 172.21.15.165:0/1165067037 >> 172.21.15.165:0/1165067037 conn(0x55ffbe298a30 msgr2=0x55ffbe316670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 -- 172.21.15.165:0/1165067037 shutdown_connections 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.541+0000 7f34637f7740 1 -- 172.21.15.165:0/1165067037 wait complete. 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.542+0000 7f34637f7740 1 -- start start 2024-04-25T10:20:16.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.542+0000 7f34637f7740 1 Processor -- start 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe319970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe319eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe1d33a0 con 0x55ffbe2fb670 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe2f91c0 con 0x55ffbe2d0bb0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe326540 con 0x55ffbe30d280 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.543+0000 7f34637f7740 10 client.0 _ll_register_callbacks cb 0x55ffbe1f0b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe319970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe319970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47956/0 (socket says 172.21.15.165:47956) 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 -- 172.21.15.165:0/82968805 learned_addr learned my addr 172.21.15.165:0/82968805 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 -- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 msgr2=0x55ffbe319eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f345a800640 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe319eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe319eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 -- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 msgr2=0x55ffbe30e0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 -- 172.21.15.165:0/82968805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbdf493f0 con 0x55ffbe2fb670 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f3451400640 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe319970 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f3444000f60 tx=0x7f344400d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=693a05559b81ff65 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.544+0000 7f343ea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f343ea00640 1 -- 172.21.15.165:0/82968805 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444018060 con 0x55ffbe2fb670 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f343ea00640 1 -- 172.21.15.165:0/82968805 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f34440058b0 con 0x55ffbe2fb670 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f34540167d0 con 0x55ffbe2fb670 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f343ea00640 1 -- 172.21.15.165:0/82968805 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f344401e050 con 0x55ffbe2fb670 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f34637f7740 10 client.11403 fetch_fsmap learned FSMap version 130 2024-04-25T10:20:16.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x55ffbdf090f0 con 0x55ffbe2fb670 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.545+0000 7f343ea00640 1 -- 172.21.15.165:0/82968805 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34540167d0 con 0x55ffbe2fb670 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f343ea00640 1 -- 172.21.15.165:0/82968805 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 130) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3444021020 con 0x55ffbe2fb670 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f34637f7740 10 client.11403 fetch_fsmap finished waiting for FSMap version 130 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f34637f7740 10 client.11403 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f34637f7740 20 client.11403 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f34637f7740 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe318400 0x55ffbe318820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f34637f7740 4 client.11403 mds_command: new command op to 11211 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 --> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffbe2421b0 con 0x55ffbe318400 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.546+0000 7f3459e00640 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe318400 0x55ffbe318820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.547+0000 7f3459e00640 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe318400 0x55ffbe318820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1b94a7730ae357b server_cookie=d4f6c187d4617e3 in_seq=0 out_seq=0 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.547+0000 7f343ea00640 10 client.11403 ms_handle_connect on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f343ea00640 1 -- 172.21.15.165:0/82968805 <== mds.0 v2:172.21.15.165:6835/1159947775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55ffbe2421b0 con 0x55ffbe318400 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f343ea00640 10 client.11403 handle_command_reply: tid=0 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 client.11403 shutdown 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 msgr2=0x55ffbe319970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe319970 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f3444000f60 tx=0x7f344400d520 comp rx=0 tx=0).stop 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 shutdown_connections 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe318400 0x55ffbe318820 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe319eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe319970 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 --2- 172.21.15.165:0/82968805 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 >> 172.21.15.165:0/82968805 conn(0x55ffbe298a30 msgr2=0x55ffbe2cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f343ea00640 0 client.11403 ms_handle_reset on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.548+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 shutdown_connections 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.549+0000 7f34637f7740 1 -- 172.21.15.165:0/82968805 wait complete. 2024-04-25T10:20:16.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.549+0000 7f34637f7740 20 client.11403 trim_cache size 0 max 16384 2024-04-25T10:20:16.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.582+0000 7f34637f7740 1 Processor -- start 2024-04-25T10:20:16.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.582+0000 7f34637f7740 1 -- start start 2024-04-25T10:20:16.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f34637f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe30ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2fdac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f34637f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c2006b0 con 0x55ffbe2fb670 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1ff890 con 0x55ffbe2d0bb0 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1f4e80 con 0x55ffbe30d280 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f3451400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe30ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f3451400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe30ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47958/0 (socket says 172.21.15.165:47958) 2024-04-25T10:20:16.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f3451400640 1 -- 172.21.15.165:0/3889721459 learned_addr learned my addr 172.21.15.165:0/3889721459 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.584+0000 7f3459e00640 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f345a800640 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2fdac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3451400640 1 -- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 msgr2=0x55ffbe2fdac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3451400640 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2fdac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3451400640 1 -- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 msgr2=0x55ffbe30f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3451400640 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3451400640 1 -- 172.21.15.165:0/3889721459 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffbe2cc4a0 con 0x55ffbe2fb670 2024-04-25T10:20:16.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3459e00640 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3451400640 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe30ff90 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f3444005cc0 tx=0x7f344400b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0e8aae572b83d9e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3449c00640 1 -- 172.21.15.165:0/3889721459 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3444005920 con 0x55ffbe2fb670 2024-04-25T10:20:16.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3449c00640 1 -- 172.21.15.165:0/3889721459 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444026070 con 0x55ffbe2fb670 2024-04-25T10:20:16.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.585+0000 7f3449c00640 1 -- 172.21.15.165:0/3889721459 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34440072d0 con 0x55ffbe2fb670 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 -- 172.21.15.165:0/3889721459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 msgr2=0x55ffbe30ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe30ff90 secure :-1 s=READY pgs=2626 cs=0 l=1 rev1=1 crypto rx=0x7f3444005cc0 tx=0x7f344400b000 comp rx=0 tx=0).stop 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 -- 172.21.15.165:0/3889721459 shutdown_connections 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe30d280 0x55ffbe2fdac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe2fb670 0x55ffbe30ff90 unknown :-1 s=CLOSED pgs=2626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 --2- 172.21.15.165:0/3889721459 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe30f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 -- 172.21.15.165:0/3889721459 >> 172.21.15.165:0/3889721459 conn(0x55ffbe298a30 msgr2=0x55ffbe328940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.586+0000 7f34637f7740 1 -- 172.21.15.165:0/3889721459 shutdown_connections 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.587+0000 7f34637f7740 1 -- 172.21.15.165:0/3889721459 wait complete. 2024-04-25T10:20:16.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.588+0000 7f34637f7740 1 -- start start 2024-04-25T10:20:16.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.588+0000 7f34637f7740 1 Processor -- start 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.589+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2cfe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.589+0000 7f34637f7740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe2fb670 0x55ffbe32a200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.589+0000 7f34637f7740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 0x55ffbe3187e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.589+0000 7f34637f7740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c1ff290 con 0x55ffbe30d280 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.589+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe30db00 con 0x55ffbe2d0bb0 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.589+0000 7f34637f7740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ffbe30fb00 con 0x55ffbe2fb670 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f34637f7740 10 client.0 _ll_register_callbacks cb 0x7f3444007c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 0x55ffbe3187e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 0x55ffbe3187e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47968/0 (socket says 172.21.15.165:47968) 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 -- 172.21.15.165:0/2548944073 learned_addr learned my addr 172.21.15.165:0/2548944073 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f3459e00640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2cfe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f3451400640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe2fb670 0x55ffbe32a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 -- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe2fb670 msgr2=0x55ffbe32a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe2fb670 0x55ffbe32a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 -- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 msgr2=0x55ffbe2cfe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2cfe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.590+0000 7f345a800640 1 -- 172.21.15.165:0/2548944073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ffbe30c130 con 0x55ffbe30d280 2024-04-25T10:20:16.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f345a800640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 0x55ffbe3187e0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f3454000c00 tx=0x7f3454004680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cab048a6ce7092 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f343ea00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:16.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f343ea00640 1 -- 172.21.15.165:0/2548944073 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400f050 con 0x55ffbe30d280 2024-04-25T10:20:16.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f343ea00640 1 -- 172.21.15.165:0/2548944073 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3454027040 con 0x55ffbe30d280 2024-04-25T10:20:16.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3444026070 con 0x55ffbe30d280 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f343ea00640 1 -- 172.21.15.165:0/2548944073 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3454036050 con 0x55ffbe30d280 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f34637f7740 10 client.11409 fetch_fsmap learned FSMap version 130 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.591+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x55ffbe3811e0 con 0x55ffbe30d280 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f343ea00640 1 -- 172.21.15.165:0/2548944073 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3444026070 con 0x55ffbe30d280 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f343ea00640 1 -- 172.21.15.165:0/2548944073 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 130) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f3454039020 con 0x55ffbe30d280 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f34637f7740 10 client.11409 fetch_fsmap finished waiting for FSMap version 130 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f34637f7740 10 client.11409 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f34637f7740 20 client.11409 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f34637f7740 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe2d2630 0x55ffbe318d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f34637f7740 4 client.11409 mds_command: new command op to 11211 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 --> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ffbe23c500 con 0x55ffbe2d2630 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.592+0000 7f3459e00640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe2d2630 0x55ffbe318d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.593+0000 7f343ea00640 10 client.11409 ms_handle_connect on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.593+0000 7f3459e00640 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe2d2630 0x55ffbe318d20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f3a5c2c442c9be server_cookie=247b209a8d4724e1 in_seq=0 out_seq=0 2024-04-25T10:20:16.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f343ea00640 1 -- 172.21.15.165:0/2548944073 <== mds.0 v2:172.21.15.165:6835/1159947775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ffbe23c500 con 0x55ffbe2d2630 2024-04-25T10:20:16.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f343ea00640 10 client.11409 handle_command_reply: tid=0 2024-04-25T10:20:16.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 client.11409 shutdown 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 msgr2=0x55ffbe3187e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 0x55ffbe3187e0 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f3454000c00 tx=0x7f3454004680 comp rx=0 tx=0).stop 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 shutdown_connections 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55ffbe2d2630 0x55ffbe318d20 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55ffbe30d280 0x55ffbe3187e0 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55ffbe2fb670 0x55ffbe32a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 --2- 172.21.15.165:0/2548944073 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55ffbe2d0bb0 0x55ffbe2cfe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 >> 172.21.15.165:0/2548944073 conn(0x55ffbe298a30 msgr2=0x55ffbe2fadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f343ea00640 0 client.11409 ms_handle_reset on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 shutdown_connections 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 1 -- 172.21.15.165:0/2548944073 wait complete. 2024-04-25T10:20:16.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.594+0000 7f34637f7740 20 client.11409 trim_cache size 0 max 16384 2024-04-25T10:20:16.599 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:20:16.599 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:20:16.599 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:16.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f343806b4d0 msgr2=0x7f343806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f343806b4d0 0x7f343806d990 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7f3454015530 tx=0x7f345400ad10 comp rx=0 tx=0).stop 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 msgr2=0x7f345c14cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 0x7f345c14cba0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f344c009eb0 tx=0x7f344c004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 shutdown_connections 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f343806b4d0 0x7f343806d990 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f345c161710 0x7f345c160120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f345c15dc20 0x7f345c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 --2- 172.21.15.165:0/1897985561 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f345c0d32f0 0x7f345c14cba0 unknown :-1 s=CLOSED pgs=2623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 >> 172.21.15.165:0/1897985561 conn(0x7f345c0d0420 msgr2=0x7f345c0b8480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 shutdown_connections 2024-04-25T10:20:16.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.595+0000 7f3460a00640 1 -- 172.21.15.165:0/1897985561 wait complete. 2024-04-25T10:20:16.614 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:20:16.614 INFO:teuthology.orchestra.run:Running command with timeout 120 2024-04-25T10:20:16.614 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.13:0 damage ls 2024-04-25T10:20:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.787+0000 7fc87f200640 1 Processor -- start 2024-04-25T10:20:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.787+0000 7fc87f200640 1 -- start start 2024-04-25T10:20:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc87f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc87f200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc878144d50 0x7fc8780b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc87f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8780cccc0 con 0x7fc8781452a0 2024-04-25T10:20:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc87f200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8780b2b20 con 0x7fc878144d50 2024-04-25T10:20:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc87f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc87806d400 con 0x7fc8781446d0 2024-04-25T10:20:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc877e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.790+0000 7fc877e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:39046/0 (socket says 172.21.15.165:39046) 2024-04-25T10:20:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc87d200640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fc8781452a0 0x7fc8780cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:58420/0 (socket says 172.21.15.165:58420) 2024-04-25T10:20:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.790+0000 7fc87d200640 1 -- 172.21.15.165:0/3194896951 learned_addr learned my addr 172.21.15.165:0/3194896951 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.789+0000 7fc86fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc878144d50 0x7fc8780b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.791+0000 7fc877e00640 1 -- 172.21.15.165:0/3194896951 >> v2:172.21.15.172:3300/0 conn(0x7fc878144d50 msgr2=0x7fc8780b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.791+0000 7fc877e00640 1 --2- 172.21.15.165:0/3194896951 >> v2:172.21.15.172:3300/0 conn(0x7fc878144d50 0x7fc8780b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.790+0000 7fc877400640 1 -- 172.21.15.165:0/3194896951 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156629440 0 0) 0x7fc8780cccc0 con 0x7fc8781452a0 2024-04-25T10:20:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.791+0000 7fc877e00640 1 -- 172.21.15.165:0/3194896951 >> v1:172.21.15.172:6789/0 conn(0x7fc8781452a0 legacy=0x7fc8780cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.791+0000 7fc877e00640 1 -- 172.21.15.165:0/3194896951 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8780cc140 con 0x7fc8781446d0 2024-04-25T10:20:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.791+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3194896951 >> v2:172.21.15.172:3300/0 conn(0x7fc878144d50 0x7fc8780b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:16.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.792+0000 7fc877e00640 1 --2- 172.21.15.165:0/3194896951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780b3f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8680097a0 tx=0x7fc86802a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b802d026a169e44 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc877400640 1 -- 172.21.15.165:0/3194896951 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868037070 con 0x7fc8781446d0 2024-04-25T10:20:16.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc877400640 1 -- 172.21.15.165:0/3194896951 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc868032440 con 0x7fc8781446d0 2024-04-25T10:20:16.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc877400640 1 -- 172.21.15.165:0/3194896951 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868032810 con 0x7fc8781446d0 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc87f200640 1 -- 172.21.15.165:0/3194896951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 msgr2=0x7fc8780b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc87f200640 1 --2- 172.21.15.165:0/3194896951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780b3f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc8680097a0 tx=0x7fc86802a180 comp rx=0 tx=0).stop 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc87f200640 1 -- 172.21.15.165:0/3194896951 shutdown_connections 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc87f200640 1 --2- 172.21.15.165:0/3194896951 >> v2:172.21.15.172:3300/0 conn(0x7fc878144d50 0x7fc8780b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc87f200640 1 --2- 172.21.15.165:0/3194896951 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780b3f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.793+0000 7fc87f200640 1 -- 172.21.15.165:0/3194896951 >> 172.21.15.165:0/3194896951 conn(0x7fc87806abe0 msgr2=0x7fc87806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.794+0000 7fc87f200640 1 -- 172.21.15.165:0/3194896951 shutdown_connections 2024-04-25T10:20:16.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.794+0000 7fc87f200640 1 -- 172.21.15.165:0/3194896951 wait complete. 2024-04-25T10:20:16.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.795+0000 7fc87f200640 1 Processor -- start 2024-04-25T10:20:16.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.795+0000 7fc87f200640 1 -- start start 2024-04-25T10:20:16.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.796+0000 7fc87f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780a9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.796+0000 7fc87f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc878144d50 0x7fc8780713e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.796+0000 7fc87f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 0x7fc87815acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.796+0000 7fc87f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780cc700 con 0x7fc8781452a0 2024-04-25T10:20:16.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.796+0000 7fc87f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780b49a0 con 0x7fc878144d50 2024-04-25T10:20:16.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.796+0000 7fc87f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780cc9e0 con 0x7fc8781446d0 2024-04-25T10:20:16.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 0x7fc87815acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 0x7fc87815acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47978/0 (socket says 172.21.15.165:47978) 2024-04-25T10:20:16.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 -- 172.21.15.165:0/1088027655 learned_addr learned my addr 172.21.15.165:0/1088027655 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc877e00640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780a9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc86fe00640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc878144d50 0x7fc8780713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 -- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 msgr2=0x7fc8780a9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780a9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 -- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc878144d50 msgr2=0x7fc8780713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc878144d50 0x7fc8780713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc87d200640 1 -- 172.21.15.165:0/1088027655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc87813d2e0 con 0x7fc8781452a0 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.797+0000 7fc877e00640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.798+0000 7fc87d200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 0x7fc87815acc0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c700 tx=0x7fc87000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.798+0000 7fc874c00640 1 -- 172.21.15.165:0/1088027655 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870018070 con 0x7fc8781452a0 2024-04-25T10:20:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.798+0000 7fc874c00640 1 -- 172.21.15.165:0/1088027655 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000ce10 con 0x7fc8781452a0 2024-04-25T10:20:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.798+0000 7fc874c00640 1 -- 172.21.15.165:0/1088027655 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87000a860 con 0x7fc8781452a0 2024-04-25T10:20:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.798+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87813c760 con 0x7fc8781452a0 2024-04-25T10:20:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.798+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87813cd20 con 0x7fc8781452a0 2024-04-25T10:20:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.800+0000 7fc874c00640 1 -- 172.21.15.165:0/1088027655 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc87000aa00 con 0x7fc8781452a0 2024-04-25T10:20:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.801+0000 7fc874c00640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85406b4d0 0x7fc85406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.802+0000 7fc874c00640 1 -- 172.21.15.165:0/1088027655 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc870089980 con 0x7fc8781452a0 2024-04-25T10:20:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.802+0000 7fc877e00640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85406b4d0 0x7fc85406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.802+0000 7fc882175740 1 Processor -- start 2024-04-25T10:20:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.802+0000 7fc882175740 1 -- start start 2024-04-25T10:20:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.802+0000 7fc877e00640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85406b4d0 0x7fc85406d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc868034000 tx=0x7fc86802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.803+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc641e8200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.803+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc64205c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.803+0000 7fc882175740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 0x55dc64203d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.803+0000 7fc882175740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc64120d00 con 0x55dc641bdb00 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.803+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc640c9f50 con 0x55dc641e7e20 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.803+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc6411cda0 con 0x55dc642056c0 2024-04-25T10:20:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 0x55dc64203d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 0x55dc64203d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:47994/0 (socket says 172.21.15.165:47994) 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 -- 172.21.15.165:0/592667891 learned_addr learned my addr 172.21.15.165:0/592667891 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc86fe00640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc64205c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc877e00640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc641e8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 -- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 msgr2=0x55dc64205c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc64205c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 -- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 msgr2=0x55dc641e8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc641e8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 -- 172.21.15.165:0/592667891 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc640ed130 con 0x55dc641bdb00 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc877e00640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc641e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc86fe00640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc64205c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.804+0000 7fc87d200640 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 0x55dc64203d80 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fc870057ea0 tx=0x7fc870057db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46244bc35943fde9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc86cc00640 1 -- 172.21.15.165:0/592667891 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870087d90 con 0x55dc641bdb00 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc86cc00640 1 -- 172.21.15.165:0/592667891 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000acf0 con 0x55dc641bdb00 2024-04-25T10:20:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc86cc00640 1 -- 172.21.15.165:0/592667891 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc87005e070 con 0x55dc641bdb00 2024-04-25T10:20:16.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 -- 172.21.15.165:0/592667891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 msgr2=0x55dc64203d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 0x55dc64203d80 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7fc870057ea0 tx=0x7fc870057db0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 -- 172.21.15.165:0/592667891 shutdown_connections 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bdb00 0x55dc64203d80 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc64205c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 --2- 172.21.15.165:0/592667891 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc641e8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 -- 172.21.15.165:0/592667891 >> 172.21.15.165:0/592667891 conn(0x55dc641ba430 msgr2=0x55dc64202c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 -- 172.21.15.165:0/592667891 shutdown_connections 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.805+0000 7fc882175740 1 -- 172.21.15.165:0/592667891 wait complete. 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.806+0000 7fc882175740 1 -- start start 2024-04-25T10:20:16.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.806+0000 7fc882175740 1 Processor -- start 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 0x55dc641e4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc641e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641eb8b0 0x55dc64216e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc641be8a0 con 0x55dc641e7e20 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc640bf530 con 0x55dc642056c0 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc641b8360 con 0x55dc641eb8b0 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.807+0000 7fc882175740 10 client.0 _ll_register_callbacks cb 0x55dc64118430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 0x55dc641e4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 0x55dc641e4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58736/0 (socket says 172.21.15.165:58736) 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 -- 172.21.15.165:0/3053823314 learned_addr learned my addr 172.21.15.165:0/3053823314 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc87d200640 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641eb8b0 0x55dc64216e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 -- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641eb8b0 msgr2=0x55dc64216e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641eb8b0 0x55dc64216e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 -- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 msgr2=0x55dc641e8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc641e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 -- 172.21.15.165:0/3053823314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc640e7dd0 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc877e00640 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 0x55dc641e4620 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fc86802a770 tx=0x7fc868004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1c15639d1432b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.808+0000 7fc865600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc865600640 1 -- 172.21.15.165:0/3053823314 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc868037040 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc865600640 1 -- 172.21.15.165:0/3053823314 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc86803caf0 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc87000acf0 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc865600640 1 -- 172.21.15.165:0/3053823314 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc868054050 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc882175740 10 client.11418 fetch_fsmap learned FSMap version 130 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x55dc63e353f0 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc865600640 1 -- 172.21.15.165:0/3053823314 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc87000acf0 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc865600640 1 -- 172.21.15.165:0/3053823314 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 130) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc868057020 con 0x55dc641e7e20 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc882175740 10 client.11418 fetch_fsmap finished waiting for FSMap version 130 2024-04-25T10:20:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.809+0000 7fc882175740 10 client.11418 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc882175740 20 client.11418 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc882175740 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc641fc450 0x55dc641f9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc882175740 4 client.11418 mds_command: new command op to 11211 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 --> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dc6412e240 con 0x55dc641fc450 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc641fc450 0x55dc641f9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc641fc450 0x55dc641f9ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cda1940f1f7bdcc0 server_cookie=aa08cc34270c5ea2 in_seq=0 out_seq=0 2024-04-25T10:20:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.810+0000 7fc865600640 10 client.11418 ms_handle_connect on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc865600640 1 -- 172.21.15.165:0/3053823314 <== mds.0 v2:172.21.15.165:6835/1159947775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55dc6412e240 con 0x55dc641fc450 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc865600640 10 client.11418 handle_command_reply: tid=0 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 client.11418 shutdown 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 msgr2=0x55dc641e4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 0x55dc641e4620 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7fc86802a770 tx=0x7fc868004730 comp rx=0 tx=0).stop 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 shutdown_connections 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc641fc450 0x55dc641f9ce0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641eb8b0 0x55dc64216e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc641e8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 --2- 172.21.15.165:0/3053823314 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641e7e20 0x55dc641e4620 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 >> 172.21.15.165:0/3053823314 conn(0x55dc641bdb00 msgr2=0x55dc64206a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.811+0000 7fc865600640 0 client.11418 ms_handle_reset on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.812+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 shutdown_connections 2024-04-25T10:20:16.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.812+0000 7fc882175740 1 -- 172.21.15.165:0/3053823314 wait complete. 2024-04-25T10:20:16.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.812+0000 7fc882175740 20 client.11418 trim_cache size 0 max 16384 2024-04-25T10:20:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.825+0000 7fc882175740 1 Processor -- start 2024-04-25T10:20:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.825+0000 7fc882175740 1 -- start start 2024-04-25T10:20:16.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc642046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc882175740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 0x55dc6426c790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc641bdf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc882175740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc878200350 con 0x55dc641eb8b0 2024-04-25T10:20:16.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780ce610 con 0x55dc641e7e20 2024-04-25T10:20:16.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8781f5be0 con 0x55dc642056c0 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 0x55dc6426c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc86fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 0x55dc6426c790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58750/0 (socket says 172.21.15.165:58750) 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc86fe00640 1 -- 172.21.15.165:0/3795879838 learned_addr learned my addr 172.21.15.165:0/3795879838 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.826+0000 7fc87d200640 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc641bdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc877e00640 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc642046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc86fe00640 1 -- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 msgr2=0x55dc641bdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc641bdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc86fe00640 1 -- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 msgr2=0x55dc642046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc642046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc86fe00640 1 -- 172.21.15.165:0/3795879838 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dc63eff7d0 con 0x55dc641eb8b0 2024-04-25T10:20:16.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc87d200640 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc641bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 0x55dc6426c790 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fc8600090e0 tx=0x7fc860004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d3130f3dc4c6f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc867e00640 1 -- 172.21.15.165:0/3795879838 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860009e00 con 0x55dc641eb8b0 2024-04-25T10:20:16.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc867e00640 1 -- 172.21.15.165:0/3795879838 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86001e030 con 0x55dc641eb8b0 2024-04-25T10:20:16.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.827+0000 7fc867e00640 1 -- 172.21.15.165:0/3795879838 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc860003c80 con 0x55dc641eb8b0 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 -- 172.21.15.165:0/3795879838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 msgr2=0x55dc6426c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 0x55dc6426c790 secure :-1 s=READY pgs=2631 cs=0 l=1 rev1=1 crypto rx=0x7fc8600090e0 tx=0x7fc860004410 comp rx=0 tx=0).stop 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 -- 172.21.15.165:0/3795879838 shutdown_connections 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc642056c0 0x55dc641bdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641eb8b0 0x55dc6426c790 unknown :-1 s=CLOSED pgs=2631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 --2- 172.21.15.165:0/3795879838 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc641e7e20 0x55dc642046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 -- 172.21.15.165:0/3795879838 >> 172.21.15.165:0/3795879838 conn(0x55dc641bdb00 msgr2=0x55dc641fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 -- 172.21.15.165:0/3795879838 shutdown_connections 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.828+0000 7fc882175740 1 -- 172.21.15.165:0/3795879838 wait complete. 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.829+0000 7fc882175740 1 -- start start 2024-04-25T10:20:16.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.829+0000 7fc882175740 1 Processor -- start 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641e7e20 0x55dc64221890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc64225110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 0x55dc6426acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dc641fd120 con 0x55dc641bef40 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8780a94c0 con 0x55dc642056c0 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dc641b74e0 con 0x55dc641e7e20 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc882175740 10 client.0 _ll_register_callbacks cb 0x55dc641b8840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc87d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 0x55dc6426acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc87d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 0x55dc6426acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58766/0 (socket says 172.21.15.165:58766) 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc87d200640 1 -- 172.21.15.165:0/3734054378 learned_addr learned my addr 172.21.15.165:0/3734054378 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc877e00640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641e7e20 0x55dc64221890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.830+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc64225110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc87d200640 1 -- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641e7e20 msgr2=0x55dc64221890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc87d200640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641e7e20 0x55dc64221890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc87d200640 1 -- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 msgr2=0x55dc64225110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc87d200640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc64225110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc87d200640 1 -- 172.21.15.165:0/3734054378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dc641f9ae0 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc877e00640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641e7e20 0x55dc64221890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc86fe00640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc64225110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc87d200640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 0x55dc6426acf0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fc870060140 tx=0x7fc870001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16daa6117e2f4ea7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc865600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc865600640 1 -- 172.21.15.165:0/3734054378 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc870057aa0 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc865600640 1 -- 172.21.15.165:0/3734054378 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc87005e070 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc86001e030 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc865600640 1 -- 172.21.15.165:0/3734054378 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=130) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc87005e270 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc882175740 10 client.11424 fetch_fsmap learned FSMap version 130 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.831+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=130,monmap=2+}) v3 -- 0x55dc64221e30 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc865600640 1 -- 172.21.15.165:0/3734054378 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86001e030 con 0x55dc641bef40 2024-04-25T10:20:16.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc865600640 1 -- 172.21.15.165:0/3734054378 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 130) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc8700573b0 con 0x55dc641bef40 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc882175740 10 client.11424 fetch_fsmap finished waiting for FSMap version 130 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc882175740 10 client.11424 resolve_mds: resolved 13:0 to role '13:0' aka daemon mds.c 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc882175740 20 client.11424 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc882175740 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc64225ec0 0x55dc6426b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc882175740 4 client.11424 mds_command: new command op to 11211 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 --> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dc64128590 con 0x55dc64225ec0 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.832+0000 7fc877e00640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc64225ec0 0x55dc6426b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc877e00640 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc64225ec0 0x55dc6426b230 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc5eab1d64f05d70 server_cookie=872d52cec47bf164 in_seq=0 out_seq=0 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc865600640 10 client.11424 ms_handle_connect on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc865600640 1 -- 172.21.15.165:0/3734054378 <== mds.0 v2:172.21.15.165:6835/1159947775 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dc64128590 con 0x55dc64225ec0 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc865600640 10 client.11424 handle_command_reply: tid=0 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 client.11424 shutdown 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 msgr2=0x55dc6426acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 0x55dc6426acf0 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7fc870060140 tx=0x7fc870001250 comp rx=0 tx=0).stop 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 shutdown_connections 2024-04-25T10:20:16.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:6835/1159947775,v1:172.21.15.165:6837/1159947775] conn(0x55dc64225ec0 0x55dc6426b230 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55dc641bef40 0x55dc6426acf0 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55dc642056c0 0x55dc64225110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 --2- 172.21.15.165:0/3734054378 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55dc641e7e20 0x55dc64221890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc865600640 0 client.11424 ms_handle_reset on v2:172.21.15.165:6835/1159947775 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 >> 172.21.15.165:0/3734054378 conn(0x55dc641bdb00 msgr2=0x55dc641f9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 shutdown_connections 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 1 -- 172.21.15.165:0/3734054378 wait complete. 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.833+0000 7fc882175740 20 client.11424 trim_cache size 0 max 16384 2024-04-25T10:20:16.839 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85406b4d0 msgr2=0x7fc85406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85406b4d0 0x7fc85406d990 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc868034000 tx=0x7fc86802e000 comp rx=0 tx=0).stop 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 msgr2=0x7fc87815acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 0x7fc87815acc0 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7fc87000c700 tx=0x7fc87000cbc0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 shutdown_connections 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc85406b4d0 0x7fc85406d990 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc8781452a0 0x7fc87815acc0 unknown :-1 s=CLOSED pgs=2628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc878144d50 0x7fc8780713e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 --2- 172.21.15.165:0/1088027655 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc8781446d0 0x7fc8780a9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:16.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 >> 172.21.15.165:0/1088027655 conn(0x7fc87806abe0 msgr2=0x7fc8780b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:16.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 shutdown_connections 2024-04-25T10:20:16.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:16.834+0000 7fc87f200640 1 -- 172.21.15.165:0/1088027655 wait complete. 2024-04-25T10:20:16.854 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:20:16.854 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:16.966 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:17.081 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:20:17.082 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:17.082 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:17.140 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:17.140 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:17.190 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:17.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.442+0000 7f3f92800640 1 Processor -- start 2024-04-25T10:20:17.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.442+0000 7f3f92800640 1 -- start start 2024-04-25T10:20:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f92800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f92800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f8c1615a0 0x7f3f8c0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f92800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3f8c0d04f0 0x7f3f8c061600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f92800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8c0d0980 con 0x7f3f8c1615a0 2024-04-25T10:20:17.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f92800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f8c0d1f50 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f92800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c161df0 con 0x7f3f8c15db90 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f90a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f8c1615a0 0x7f3f8c0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f90a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3f8c1615a0 0x7f3f8c0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58772/0 (socket says 172.21.15.165:58772) 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f90a00640 1 -- 172.21.15.165:0/2883716739 learned_addr learned my addr 172.21.15.165:0/2883716739 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f91e00640 1 --2- 172.21.15.165:0/2883716739 >> v2:172.21.15.172:3300/0 conn(0x7f3f8c0d04f0 0x7f3f8c061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.444+0000 7f3f91400640 1 --2- 172.21.15.165:0/2883716739 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.445+0000 7f3f90a00640 1 -- 172.21.15.165:0/2883716739 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 msgr2=0x7f3f8c0b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.445+0000 7f3f90a00640 1 --2- 172.21.15.165:0/2883716739 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c0b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.445+0000 7f3f90a00640 1 -- 172.21.15.165:0/2883716739 >> v2:172.21.15.172:3300/0 conn(0x7f3f8c0d04f0 msgr2=0x7f3f8c061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.445+0000 7f3f90a00640 1 --2- 172.21.15.165:0/2883716739 >> v2:172.21.15.172:3300/0 conn(0x7f3f8c0d04f0 0x7f3f8c061600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.445+0000 7f3f90a00640 1 -- 172.21.15.165:0/2883716739 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c1620d0 con 0x7f3f8c1615a0 2024-04-25T10:20:17.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.445+0000 7f3f91e00640 1 --2- 172.21.15.165:0/2883716739 >> v2:172.21.15.172:3300/0 conn(0x7f3f8c0d04f0 0x7f3f8c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:17.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.446+0000 7f3f90a00640 1 --2- 172.21.15.165:0/2883716739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0cffb0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c08bcb0 tx=0x7f3f7802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbcbb455062c6d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.446+0000 7f3f8be00640 1 -- 172.21.15.165:0/2883716739 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78037070 con 0x7f3f8c1615a0 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.446+0000 7f3f8be00640 1 -- 172.21.15.165:0/2883716739 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f78032440 con 0x7f3f8c1615a0 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.446+0000 7f3f8be00640 1 -- 172.21.15.165:0/2883716739 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78032810 con 0x7f3f8c1615a0 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.446+0000 7f3f92800640 1 -- 172.21.15.165:0/2883716739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c1615a0 msgr2=0x7f3f8c0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 --2- 172.21.15.165:0/2883716739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0cffb0 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c08bcb0 tx=0x7f3f7802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 -- 172.21.15.165:0/2883716739 shutdown_connections 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 --2- 172.21.15.165:0/2883716739 >> v2:172.21.15.172:3300/0 conn(0x7f3f8c0d04f0 0x7f3f8c061600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 --2- 172.21.15.165:0/2883716739 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0cffb0 unknown :-1 s=CLOSED pgs=2633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 --2- 172.21.15.165:0/2883716739 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c0b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 -- 172.21.15.165:0/2883716739 >> 172.21.15.165:0/2883716739 conn(0x7f3f8c0b52b0 msgr2=0x7f3f8c0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 -- 172.21.15.165:0/2883716739 shutdown_connections 2024-04-25T10:20:17.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.447+0000 7f3f92800640 1 -- 172.21.15.165:0/2883716739 wait complete. 2024-04-25T10:20:17.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.448+0000 7f3f92800640 1 Processor -- start 2024-04-25T10:20:17.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.449+0000 7f3f92800640 1 -- start start 2024-04-25T10:20:17.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.449+0000 7f3f92800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 0x7f3f8c15cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.449+0000 7f3f92800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c157bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f92800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f92800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c161a20 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f92800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c0d1be0 con 0x7f3f8c1615a0 2024-04-25T10:20:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f92800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f8c161ba0 con 0x7f3f8c15db90 2024-04-25T10:20:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 0x7f3f8c15cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 0x7f3f8c15cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58782/0 (socket says 172.21.15.165:58782) 2024-04-25T10:20:17.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 -- 172.21.15.165:0/1388268643 learned_addr learned my addr 172.21.15.165:0/1388268643 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 -- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 msgr2=0x7f3f8c157bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c157bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 -- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c1615a0 msgr2=0x7f3f8c0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91e00640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91400640 1 -- 172.21.15.165:0/1388268643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c15ac10 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.450+0000 7f3f91e00640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.451+0000 7f3f91400640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 0x7f3f8c15cba0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000d800 tx=0x7f3f8000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.451+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000b6c0 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.451+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f80019070 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.451+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8000ba80 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.451+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8c15a930 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.451+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c0cacf0 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.452+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f8c073990 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.453+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3f8000bc20 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.455+0000 7f3f89600640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f6c06b500 0x7f3f6c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.455+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3f8008a8f0 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.455+0000 7f3f90a00640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f6c06b500 0x7f3f6c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.456+0000 7f3f90a00640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f6c06b500 0x7f3f6c06d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3f78034000 tx=0x7f3f7802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.460+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3f8001c360 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.660+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f8c0d14b0 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.661+0000 7f3f89600640 1 -- 172.21.15.165:0/1388268643 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f3f80059540 con 0x7f3f8c0d04f0 2024-04-25T10:20:17.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:17.664 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":184,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:03.322848+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f6c06b500 msgr2=0x7f3f6c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f6c06b500 0x7f3f6c06d9c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f3f78034000 tx=0x7f3f7802c000 comp rx=0 tx=0).stop 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 msgr2=0x7f3f8c15cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 0x7f3f8c15cba0 secure :-1 s=READY pgs=2634 cs=0 l=1 rev1=1 crypto rx=0x7f3f8000d800 tx=0x7f3f8000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 shutdown_connections 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3f6c06b500 0x7f3f6c06d9c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3f8c1615a0 0x7f3f8c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3f8c15db90 0x7f3f8c157bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 --2- 172.21.15.165:0/1388268643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3f8c0d04f0 0x7f3f8c15cba0 unknown :-1 s=CLOSED pgs=2634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 >> 172.21.15.165:0/1388268643 conn(0x7f3f8c0b52b0 msgr2=0x7f3f8c0b6890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:17.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 shutdown_connections 2024-04-25T10:20:17.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.664+0000 7f3f92800640 1 -- 172.21.15.165:0/1388268643 wait complete. 2024-04-25T10:20:17.685 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:19:59.548746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:00.783201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:17.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:17.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.882+0000 7f33d7200640 1 Processor -- start 2024-04-25T10:20:17.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.882+0000 7f33d7200640 1 -- start start 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.883+0000 7f33d7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d813df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.883+0000 7f33d7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33d806e860 0x7f33d80b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.883+0000 7f33d7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d806d390 con 0x7f33d80b0730 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.883+0000 7f33d7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d80b10d0 con 0x7f33d806e860 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.883+0000 7f33d7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d806d670 con 0x7f33d813d9f0 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33d806e860 0x7f33d80b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f33d806e860 0x7f33d80b0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58798/0 (socket says 172.21.15.165:58798) 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 -- 172.21.15.165:0/3397874344 learned_addr learned my addr 172.21.15.165:0/3397874344 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d5e00640 1 --2- 172.21.15.165:0/3397874344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d813df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 -- 172.21.15.165:0/3397874344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 msgr2=0x7f33d813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 --2- 172.21.15.165:0/3397874344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d813df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 -- 172.21.15.165:0/3397874344 >> v1:172.21.15.172:6789/0 conn(0x7f33d80b0730 legacy=0x7f33d806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:17.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.884+0000 7f33d6800640 1 -- 172.21.15.165:0/3397874344 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d806d950 con 0x7f33d806e860 2024-04-25T10:20:17.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.885+0000 7f33d6800640 1 --2- 172.21.15.165:0/3397874344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80b0b90 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009870 tx=0x7f33d002a100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba0f6741aa7a8d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d4a00640 1 -- 172.21.15.165:0/3397874344 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0037070 con 0x7f33d806e860 2024-04-25T10:20:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d4a00640 1 -- 172.21.15.165:0/3397874344 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d002ae10 con 0x7f33d806e860 2024-04-25T10:20:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d4a00640 1 -- 172.21.15.165:0/3397874344 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d0032660 con 0x7f33d806e860 2024-04-25T10:20:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d7200640 1 -- 172.21.15.165:0/3397874344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 msgr2=0x7f33d80b0b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d7200640 1 --2- 172.21.15.165:0/3397874344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80b0b90 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7f33d0009870 tx=0x7f33d002a100 comp rx=0 tx=0).stop 2024-04-25T10:20:17.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d7200640 1 -- 172.21.15.165:0/3397874344 shutdown_connections 2024-04-25T10:20:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d7200640 1 --2- 172.21.15.165:0/3397874344 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80b0b90 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d7200640 1 --2- 172.21.15.165:0/3397874344 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d813df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.886+0000 7f33d7200640 1 -- 172.21.15.165:0/3397874344 >> 172.21.15.165:0/3397874344 conn(0x7f33d8061cc0 msgr2=0x7f33d806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.887+0000 7f33d7200640 1 -- 172.21.15.165:0/3397874344 shutdown_connections 2024-04-25T10:20:17.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.887+0000 7f33d7200640 1 -- 172.21.15.165:0/3397874344 wait complete. 2024-04-25T10:20:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.888+0000 7f33d7200640 1 Processor -- start 2024-04-25T10:20:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.888+0000 7f33d7200640 1 -- start start 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.889+0000 7f33d7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.889+0000 7f33d7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d80b0730 0x7f33d812f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.889+0000 7f33d7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d8130720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.889+0000 7f33d7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d813e5f0 con 0x7f33d806e860 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.889+0000 7f33d7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d813e470 con 0x7f33d80b0730 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.889+0000 7f33d7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d80ce880 con 0x7f33d813d9f0 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80cbbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58804/0 (socket says 172.21.15.165:58804) 2024-04-25T10:20:17.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 -- 172.21.15.165:0/2024487034 learned_addr learned my addr 172.21.15.165:0/2024487034 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d6800640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d8130720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5400640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d80b0730 0x7f33d812f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 -- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 msgr2=0x7f33d8130720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d8130720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 -- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d80b0730 msgr2=0x7f33d812f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d80b0730 0x7f33d812f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 -- 172.21.15.165:0/2024487034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8132dc0 con 0x7f33d806e860 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d6800640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d8130720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.890+0000 7f33d5e00640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80cbbd0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c7f0 tx=0x7f33c800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.891+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8015070 con 0x7f33d806e860 2024-04-25T10:20:17.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.891+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c8004220 con 0x7f33d806e860 2024-04-25T10:20:17.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.891+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d81330a0 con 0x7f33d806e860 2024-04-25T10:20:17.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.891+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c8007620 con 0x7f33d806e860 2024-04-25T10:20:17.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.891+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d812df60 con 0x7f33d806e860 2024-04-25T10:20:17.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.892+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d812e160 con 0x7f33d806e860 2024-04-25T10:20:17.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.893+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f33c80079d0 con 0x7f33d806e860 2024-04-25T10:20:17.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.894+0000 7f33ce000640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:17.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.894+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f33c808a880 con 0x7f33d806e860 2024-04-25T10:20:17.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.894+0000 7f33d5400640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:17.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.895+0000 7f33d5400640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f33c0007fd0 tx=0x7f33c0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:17.898+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f33d812e160 con 0x7f33d806e860 2024-04-25T10:20:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.098+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33d8073990 con 0x7f33d806e860 2024-04-25T10:20:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.098+0000 7f33ce000640 1 -- 172.21.15.165:0/2024487034 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v184) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33c8059440 con 0x7f33d806e860 2024-04-25T10:20:18.101 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 msgr2=0x7f33b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7f33c0007fd0 tx=0x7f33c0009040 comp rx=0 tx=0).stop 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 msgr2=0x7f33d80cbbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80cbbd0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7f33c800c7f0 tx=0x7f33c800ccb0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 shutdown_connections 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f33b406b4d0 0x7f33b406d990 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f33d813d9f0 0x7f33d8130720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f33d80b0730 0x7f33d812f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 --2- 172.21.15.165:0/2024487034 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f33d806e860 0x7f33d80cbbd0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 >> 172.21.15.165:0/2024487034 conn(0x7f33d8061cc0 msgr2=0x7f33d80b28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.101+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 shutdown_connections 2024-04-25T10:20:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.102+0000 7f33d7200640 1 -- 172.21.15.165:0/2024487034 wait complete. 2024-04-25T10:20:18.121 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:18.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.323+0000 7f68ada00640 1 Processor -- start 2024-04-25T10:20:18.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.323+0000 7f68ada00640 1 -- start start 2024-04-25T10:20:18.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.325+0000 7f68ada00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f68a815dd30 0x7f68a80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.325+0000 7f68ada00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a815e110 con 0x7f68a806bff0 2024-04-25T10:20:18.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.325+0000 7f68ada00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68a80b2fd0 con 0x7f68a80b5b40 2024-04-25T10:20:18.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.325+0000 7f68ada00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68a806d000 con 0x7f68a815dd30 2024-04-25T10:20:18.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.325+0000 7f68a7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f68a806bff0 0x7f68a80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52338/0 (socket says 172.21.15.165:52338) 2024-04-25T10:20:18.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.325+0000 7f68a7400640 1 -- 172.21.15.165:0/2790531095 learned_addr learned my addr 172.21.15.165:0/2790531095 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:18.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879799740 0 0) 0x7f68a815e110 con 0x7f68a806bff0 2024-04-25T10:20:18.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f688c003520 con 0x7f68a806bff0 2024-04-25T10:20:18.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a7e00640 1 --2- 172.21.15.165:0/2790531095 >> v2:172.21.15.172:3300/0 conn(0x7f68a815dd30 0x7f68a80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2243872176 0 0) 0x7f688c003520 con 0x7f68a806bff0 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 >> v2:172.21.15.172:3300/0 conn(0x7f68a815dd30 msgr2=0x7f68a80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 --2- 172.21.15.165:0/2790531095 >> v2:172.21.15.172:3300/0 conn(0x7f68a815dd30 0x7f68a80cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 >> v1:172.21.15.172:6789/0 conn(0x7f68a80b5b40 legacy=0x7f68a80d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a806c480 con 0x7f68a806bff0 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.326+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f6894001670 con 0x7f68a806bff0 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 >> v1:172.21.15.165:6789/0 conn(0x7f68a806bff0 legacy=0x7f68a80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f688c0039b0 0x7f688c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004800 0x7f688c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0051c0 0x7f688c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a806d000 con 0x7f688c0039b0 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c110 con 0x7f688c004800 2024-04-25T10:20:18.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f688c00c310 con 0x7f688c0051c0 2024-04-25T10:20:18.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7e00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f688c0039b0 0x7f688c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7400640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004800 0x7f688c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7e00640 1 -- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0051c0 msgr2=0x7f688c00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7e00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0051c0 0x7f688c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7e00640 1 -- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004800 msgr2=0x7f688c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7e00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004800 0x7f688c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7e00640 1 -- 172.21.15.165:0/2790531095 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f688c003520 con 0x7f688c0039b0 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68ad000640 1 -- 172.21.15.165:0/2790531095 reap_dead start 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.327+0000 7f68a7400640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f688c004800 0x7f688c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.329+0000 7f68a7e00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f688c0039b0 0x7f688c003dd0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f689800ba30 tx=0x7f688c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=976d93f35864c5c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.329+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6898030070 con 0x7f688c0039b0 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.329+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689802ae10 con 0x7f688c0039b0 2024-04-25T10:20:18.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.329+0000 7f68a6a00640 1 -- 172.21.15.165:0/2790531095 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f689800b600 con 0x7f688c0039b0 2024-04-25T10:20:18.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 -- 172.21.15.165:0/2790531095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f688c0039b0 msgr2=0x7f688c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f688c0039b0 0x7f688c003dd0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f689800ba30 tx=0x7f688c005830 comp rx=0 tx=0).stop 2024-04-25T10:20:18.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 -- 172.21.15.165:0/2790531095 shutdown_connections 2024-04-25T10:20:18.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 --2- 172.21.15.165:0/2790531095 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f688c0039b0 0x7f688c003dd0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 -- 172.21.15.165:0/2790531095 >> 172.21.15.165:0/2790531095 conn(0x7f68a80d0610 msgr2=0x7f68a80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:18.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 -- 172.21.15.165:0/2790531095 shutdown_connections 2024-04-25T10:20:18.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.330+0000 7f68ada00640 1 -- 172.21.15.165:0/2790531095 wait complete. 2024-04-25T10:20:18.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.332+0000 7f68ada00640 1 Processor -- start 2024-04-25T10:20:18.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.332+0000 7f68ada00640 1 -- start start 2024-04-25T10:20:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68ada00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0039b0 0x7f68a81507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68ada00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 0x7f68a81f6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68ada00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68a81f71e0 0x7f68a80cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68ada00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a806cd20 con 0x7f68a81606e0 2024-04-25T10:20:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68ada00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68a806ca40 con 0x7f68a81f71e0 2024-04-25T10:20:18.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68ada00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68a80b5430 con 0x7f688c0039b0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68a7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 0x7f68a81f6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68a7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 0x7f68a81f6ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58818/0 (socket says 172.21.15.165:58818) 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.333+0000 7f68a7400640 1 -- 172.21.15.165:0/1871810622 learned_addr learned my addr 172.21.15.165:0/1871810622 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68a7400640 1 -- 172.21.15.165:0/1871810622 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0039b0 msgr2=0x7f68a81507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68a7400640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0039b0 0x7f68a81507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68a7400640 1 -- 172.21.15.165:0/1871810622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68a81f71e0 msgr2=0x7f68a80cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68a7400640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68a81f71e0 0x7f68a80cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68a7400640 1 -- 172.21.15.165:0/1871810622 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a8150ce0 con 0x7f68a81606e0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68a7400640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 0x7f68a81f6ca0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f689400d7a0 tx=0x7f689400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6894019070 con 0x7f68a81606e0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f689400de50 con 0x7f68a81606e0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68a8153ce0 con 0x7f68a81606e0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.334+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68940047e0 con 0x7f68a81606e0 2024-04-25T10:20:18.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.335+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68a815fee0 con 0x7f68a81606e0 2024-04-25T10:20:18.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.336+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68a81f7600 con 0x7f68a81606e0 2024-04-25T10:20:18.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.337+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f689401c070 con 0x7f68a81606e0 2024-04-25T10:20:18.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.338+0000 7f689fe00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f688806b500 0x7f688806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.338+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6894089e20 con 0x7f68a81606e0 2024-04-25T10:20:18.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.338+0000 7f68a7e00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f688806b500 0x7f688806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.339+0000 7f68a7e00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f688806b500 0x7f688806d9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f689800ba00 tx=0x7f689802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.349+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6894058a70 con 0x7f68a81606e0 2024-04-25T10:20:18.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.550+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68a8073990 con 0x7f68a81606e0 2024-04-25T10:20:18.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.551+0000 7f689fe00640 1 -- 172.21.15.165:0/1871810622 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v184) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6894058370 con 0x7f68a81606e0 2024-04-25T10:20:18.554 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f688806b500 msgr2=0x7f688806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f688806b500 0x7f688806d9c0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f689800ba00 tx=0x7f689802c000 comp rx=0 tx=0).stop 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 msgr2=0x7f68a81f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 0x7f68a81f6ca0 secure :-1 s=READY pgs=2639 cs=0 l=1 rev1=1 crypto rx=0x7f689400d7a0 tx=0x7f689400dc60 comp rx=0 tx=0).stop 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 shutdown_connections 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f688806b500 0x7f688806d9c0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68a81f71e0 0x7f68a80cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68a81606e0 0x7f68a81f6ca0 unknown :-1 s=CLOSED pgs=2639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 --2- 172.21.15.165:0/1871810622 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f688c0039b0 0x7f68a81507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 >> 172.21.15.165:0/1871810622 conn(0x7f68a80d0610 msgr2=0x7f68a80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:18.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 shutdown_connections 2024-04-25T10:20:18.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.554+0000 7f68ada00640 1 -- 172.21.15.165:0/1871810622 wait complete. 2024-04-25T10:20:18.576 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.779+0000 7fc48b200640 1 Processor -- start 2024-04-25T10:20:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.779+0000 7fc48b200640 1 -- start start 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.780+0000 7fc48b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.780+0000 7fc48b200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc48c0ce950 0x7fc48c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.780+0000 7fc48b200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fc48c0c5030 0x7fc48c06d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.780+0000 7fc48b200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48c0ced90 con 0x7fc48c0ce950 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.780+0000 7fc48b200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48c145520 con 0x7fc48c0c5030 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.780+0000 7fc48b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c071270 con 0x7fc48c145140 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc48c0ce950 0x7fc48c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc48c0ce950 0x7fc48c0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58820/0 (socket says 172.21.15.165:58820) 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 -- 172.21.15.165:0/3847989807 learned_addr learned my addr 172.21.15.165:0/3847989807 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc489e00640 1 --2- 172.21.15.165:0/3847989807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc48a800640 1 --2- 172.21.15.165:0/3847989807 >> v2:172.21.15.172:3300/0 conn(0x7fc48c0c5030 0x7fc48c06d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 -- 172.21.15.165:0/3847989807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 msgr2=0x7fc48c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 --2- 172.21.15.165:0/3847989807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 -- 172.21.15.165:0/3847989807 >> v2:172.21.15.172:3300/0 conn(0x7fc48c0c5030 msgr2=0x7fc48c06d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.781+0000 7fc481400640 1 --2- 172.21.15.165:0/3847989807 >> v2:172.21.15.172:3300/0 conn(0x7fc48c0c5030 0x7fc48c06d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.782+0000 7fc481400640 1 -- 172.21.15.165:0/3847989807 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c071550 con 0x7fc48c0ce950 2024-04-25T10:20:18.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.783+0000 7fc481400640 1 --2- 172.21.15.165:0/3847989807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0ce950 0x7fc48c0c4af0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0b2860 tx=0x7fc47402a140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1e2406ed285a4b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.783+0000 7fc489400640 1 -- 172.21.15.165:0/3847989807 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474037070 con 0x7fc48c0ce950 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.783+0000 7fc489400640 1 -- 172.21.15.165:0/3847989807 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc474032440 con 0x7fc48c0ce950 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.783+0000 7fc489400640 1 -- 172.21.15.165:0/3847989807 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474032810 con 0x7fc48c0ce950 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 -- 172.21.15.165:0/3847989807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0ce950 msgr2=0x7fc48c0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 --2- 172.21.15.165:0/3847989807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0ce950 0x7fc48c0c4af0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7fc48c0b2860 tx=0x7fc47402a140 comp rx=0 tx=0).stop 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 -- 172.21.15.165:0/3847989807 shutdown_connections 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 --2- 172.21.15.165:0/3847989807 >> v2:172.21.15.172:3300/0 conn(0x7fc48c0c5030 0x7fc48c06d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 --2- 172.21.15.165:0/3847989807 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0ce950 0x7fc48c0c4af0 unknown :-1 s=CLOSED pgs=2640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 --2- 172.21.15.165:0/3847989807 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 -- 172.21.15.165:0/3847989807 >> 172.21.15.165:0/3847989807 conn(0x7fc48c061cc0 msgr2=0x7fc48c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 -- 172.21.15.165:0/3847989807 shutdown_connections 2024-04-25T10:20:18.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.784+0000 7fc48b200640 1 -- 172.21.15.165:0/3847989807 wait complete. 2024-04-25T10:20:18.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.786+0000 7fc48b200640 1 Processor -- start 2024-04-25T10:20:18.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.786+0000 7fc48b200640 1 -- start start 2024-04-25T10:20:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.787+0000 7fc48b200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 0x7fc48c15b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.787+0000 7fc48b200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc48c0ce950 0x7fc48c0cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.787+0000 7fc48b200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.787+0000 7fc48b200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c0c54a0 con 0x7fc48c0c5030 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.787+0000 7fc48b200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c071b10 con 0x7fc48c0ce950 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.787+0000 7fc48b200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48c0c5620 con 0x7fc48c145140 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 0x7fc48c15b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 0x7fc48c15b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58824/0 (socket says 172.21.15.165:58824) 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 -- 172.21.15.165:0/3456976626 learned_addr learned my addr 172.21.15.165:0/3456976626 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 -- 172.21.15.165:0/3456976626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 msgr2=0x7fc48c0ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 -- 172.21.15.165:0/3456976626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc48c0ce950 msgr2=0x7fc48c0cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc48c0ce950 0x7fc48c0cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 -- 172.21.15.165:0/3456976626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc48c0a84a0 con 0x7fc48c0c5030 2024-04-25T10:20:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.788+0000 7fc489e00640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 0x7fc48c15b6b0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00d5f0 tx=0x7fc47c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.789+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0125f0 con 0x7fc48c0c5030 2024-04-25T10:20:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.789+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47c019070 con 0x7fc48c0c5030 2024-04-25T10:20:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.789+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48c15bbf0 con 0x7fc48c0c5030 2024-04-25T10:20:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.789+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47c0129d0 con 0x7fc48c0c5030 2024-04-25T10:20:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.789+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc48c0a8a20 con 0x7fc48c0c5030 2024-04-25T10:20:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.790+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48c073990 con 0x7fc48c0c5030 2024-04-25T10:20:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.791+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc47c012b70 con 0x7fc48c0c5030 2024-04-25T10:20:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.792+0000 7fc482a00640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc46806b4d0 0x7fc46806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.792+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fc47c08a840 con 0x7fc48c0c5030 2024-04-25T10:20:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.792+0000 7fc481400640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc46806b4d0 0x7fc46806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:18.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.793+0000 7fc481400640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc46806b4d0 0x7fc46806d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc474034000 tx=0x7fc47402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:18.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.798+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fc47c059490 con 0x7fc48c0c5030 2024-04-25T10:20:18.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.994+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc48c0b1ed0 con 0x7fc48c0c5030 2024-04-25T10:20:18.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.995+0000 7fc482a00640 1 -- 172.21.15.165:0/3456976626 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v184) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc47c058d90 con 0x7fc48c0c5030 2024-04-25T10:20:18.998 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.997+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc46806b4d0 msgr2=0x7fc46806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.997+0000 7fc48b200640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc46806b4d0 0x7fc46806d990 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc474034000 tx=0x7fc47402c000 comp rx=0 tx=0).stop 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.997+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 msgr2=0x7fc48c15b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.997+0000 7fc48b200640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 0x7fc48c15b6b0 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fc47c00d5f0 tx=0x7fc47c00dab0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 shutdown_connections 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc46806b4d0 0x7fc46806d990 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc48c145140 0x7fc48c0ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc48c0ce950 0x7fc48c0cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 --2- 172.21.15.165:0/3456976626 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc48c0c5030 0x7fc48c15b6b0 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 >> 172.21.15.165:0/3456976626 conn(0x7fc48c061cc0 msgr2=0x7fc48c0b3d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 shutdown_connections 2024-04-25T10:20:19.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:18.998+0000 7fc48b200640 1 -- 172.21.15.165:0/3456976626 wait complete. 2024-04-25T10:20:19.019 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:20:19.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.226+0000 7f851fe00640 1 Processor -- start 2024-04-25T10:20:19.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.226+0000 7f851fe00640 1 -- start start 2024-04-25T10:20:19.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.228+0000 7f851fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 0x7f85200b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.228+0000 7f851fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852013e640 con 0x7f85200b0730 2024-04-25T10:20:19.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.228+0000 7f851fe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f852006d390 con 0x7f852013d9f0 2024-04-25T10:20:19.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.228+0000 7f851fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f852006d670 con 0x7f852006e860 2024-04-25T10:20:19.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.228+0000 7f8517e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f85200b0730 0x7f852006e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52352/0 (socket says 172.21.15.165:52352) 2024-04-25T10:20:19.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.228+0000 7f8517e00640 1 -- 172.21.15.165:0/675580912 learned_addr learned my addr 172.21.15.165:0/675580912 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851f400640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 0x7f85200b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873013772 0 0) 0x7f852013e640 con 0x7f85200b0730 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8504003520 con 0x7f85200b0730 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2036774813 0 0) 0x7f8504003520 con 0x7f85200b0730 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 msgr2=0x7f85200b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 0x7f85200b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 >> v1:172.21.15.172:6789/0 conn(0x7f852013d9f0 legacy=0x7f852013df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852006d950 con 0x7f85200b0730 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8508001670 con 0x7f85200b0730 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 >> v1:172.21.15.165:6789/0 conn(0x7f85200b0730 legacy=0x7f852006e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85040039e0 0x7f8504003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 0x7f850400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f852006d670 con 0x7f85040039e0 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8504005540 con 0x7f8504004800 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850400e110 con 0x7f852006e860 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.229+0000 7f851ea00640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85040039e0 0x7f8504003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.230+0000 7f8517e00640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.230+0000 7f851ea00640 1 -- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 msgr2=0x7f850400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.230+0000 7f851ea00640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f852006e860 0x7f850400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.230+0000 7f851ea00640 1 -- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8504004800 msgr2=0x7f8504004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.230+0000 7f851ea00640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8504004800 0x7f8504004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.230+0000 7f851ea00640 1 -- 172.21.15.165:0/675580912 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8504003520 con 0x7f85040039e0 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.231+0000 7f851ea00640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85040039e0 0x7f8504003e00 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f8510001750 tx=0x7f8504005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96360f6895855354 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.231+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8510001370 con 0x7f85040039e0 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.231+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85100052d0 con 0x7f85040039e0 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.231+0000 7f851e000640 1 -- 172.21.15.165:0/675580912 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f851002abe0 con 0x7f85040039e0 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851fe00640 1 -- 172.21.15.165:0/675580912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85040039e0 msgr2=0x7f8504003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851fe00640 1 --2- 172.21.15.165:0/675580912 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85040039e0 0x7f8504003e00 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f8510001750 tx=0x7f8504005830 comp rx=0 tx=0).stop 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851f400640 1 -- 172.21.15.165:0/675580912 reap_dead start 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851fe00640 1 -- 172.21.15.165:0/675580912 shutdown_connections 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851fe00640 1 -- 172.21.15.165:0/675580912 >> 172.21.15.165:0/675580912 conn(0x7f8520061cc0 msgr2=0x7f852006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851fe00640 1 -- 172.21.15.165:0/675580912 shutdown_connections 2024-04-25T10:20:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.232+0000 7f851fe00640 1 -- 172.21.15.165:0/675580912 wait complete. 2024-04-25T10:20:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.234+0000 7f851fe00640 1 Processor -- start 2024-04-25T10:20:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.234+0000 7f851fe00640 1 -- start start 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.234+0000 7f851fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85040039e0 0x7f85200cc780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 0x7f8520139b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85201f3ba0 0x7f852013a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b0d40 con 0x7f85201f3720 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b0bc0 con 0x7f85040039e0 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85200b0ec0 con 0x7f85201f3ba0 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 0x7f8520139b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 0x7f8520139b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58844/0 (socket says 172.21.15.165:58844) 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 -- 172.21.15.165:0/870295967 learned_addr learned my addr 172.21.15.165:0/870295967 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851f400640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85201f3ba0 0x7f852013a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 -- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85201f3ba0 msgr2=0x7f852013a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85201f3ba0 0x7f852013a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 -- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85040039e0 msgr2=0x7f85200cc780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f851ea00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85040039e0 0x7f85200cc780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85040039e0 0x7f85200cc780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.235+0000 7f8517e00640 1 -- 172.21.15.165:0/870295967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85200cccc0 con 0x7f85201f3720 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.236+0000 7f8517e00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 0x7f8520139b40 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f850800d7a0 tx=0x7f850800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.236+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508019070 con 0x7f85201f3720 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.236+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850800de50 con 0x7f85201f3720 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.236+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85080047e0 con 0x7f85201f3720 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.236+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85200ccf60 con 0x7f85201f3720 2024-04-25T10:20:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.236+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85200cd520 con 0x7f85201f3720 2024-04-25T10:20:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.237+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8520073990 con 0x7f85201f3720 2024-04-25T10:20:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.238+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8508004980 con 0x7f85201f3720 2024-04-25T10:20:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.240+0000 7f8516a00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84fc06b4d0 0x7f84fc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.240+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f8508089d40 con 0x7f85201f3720 2024-04-25T10:20:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.240+0000 7f851ea00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84fc06b4d0 0x7f84fc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.241+0000 7f851ea00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84fc06b4d0 0x7f84fc06d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8510032000 tx=0x7f851002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.244+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8508004c70 con 0x7f85201f3720 2024-04-25T10:20:19.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.454+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f852015b880 con 0x7f85201f3720 2024-04-25T10:20:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.454+0000 7f8516a00640 1 -- 172.21.15.165:0/870295967 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v184) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8508058a20 con 0x7f85201f3720 2024-04-25T10:20:19.457 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:20:19.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.457+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84fc06b4d0 msgr2=0x7f84fc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.457+0000 7f851fe00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84fc06b4d0 0x7f84fc06d990 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f8510032000 tx=0x7f851002e000 comp rx=0 tx=0).stop 2024-04-25T10:20:19.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.457+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 msgr2=0x7f8520139b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.457+0000 7f851fe00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 0x7f8520139b40 secure :-1 s=READY pgs=2644 cs=0 l=1 rev1=1 crypto rx=0x7f850800d7a0 tx=0x7f850800dc60 comp rx=0 tx=0).stop 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 shutdown_connections 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f84fc06b4d0 0x7f84fc06d990 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85201f3ba0 0x7f852013a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85201f3720 0x7f8520139b40 unknown :-1 s=CLOSED pgs=2644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 --2- 172.21.15.165:0/870295967 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85040039e0 0x7f85200cc780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 >> 172.21.15.165:0/870295967 conn(0x7f8520061cc0 msgr2=0x7f85200b2fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 shutdown_connections 2024-04-25T10:20:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:19.458+0000 7f851fe00640 1 -- 172.21.15.165:0/870295967 wait complete. 2024-04-25T10:20:19.479 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:20:19.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:19.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:19.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:20:19.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:19.982 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:20:20.021 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:20:20.021 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:20.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:20.078 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:20.078 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:20.126 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:20.126 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:20:20.127 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:20.182 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:20.183 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.380+0000 7f92ed000640 1 Processor -- start 2024-04-25T10:20:20.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.380+0000 7f92ed000640 1 -- start start 2024-04-25T10:20:20.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.382+0000 7f92ed000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 0x7f92e815ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.382+0000 7f92ed000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92e806b2f0 0x7f92e80b3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.382+0000 7f92ed000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e81604c0 con 0x7f92e80afa00 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.382+0000 7f92ed000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e80aef60 con 0x7f92e806b2f0 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.382+0000 7f92ed000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80b5590 con 0x7f92e8145310 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 0x7f92e815ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e6800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f92e806b2f0 0x7f92e80b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e5e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 0x7f92e815ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:56926/0 (socket says 172.21.15.165:56926) 2024-04-25T10:20:20.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e5e00640 1 -- 172.21.15.165:0/1742877607 learned_addr learned my addr 172.21.15.165:0/1742877607 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e5e00640 1 -- 172.21.15.165:0/1742877607 >> v2:172.21.15.172:3300/0 conn(0x7f92e806b2f0 msgr2=0x7f92e80b3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e5e00640 1 --2- 172.21.15.165:0/1742877607 >> v2:172.21.15.172:3300/0 conn(0x7f92e806b2f0 0x7f92e80b3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.383+0000 7f92e5e00640 1 -- 172.21.15.165:0/1742877607 >> v1:172.21.15.172:6789/0 conn(0x7f92e80afa00 legacy=0x7f92e806abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.384+0000 7f92e5e00640 1 -- 172.21.15.165:0/1742877607 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80af200 con 0x7f92e8145310 2024-04-25T10:20:20.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.385+0000 7f92e5e00640 1 --2- 172.21.15.165:0/1742877607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 0x7f92e815ff80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009870 tx=0x7f92d802a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87dedca6515710e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.385+0000 7f92e5400640 1 -- 172.21.15.165:0/1742877607 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8037070 con 0x7f92e8145310 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92e5400640 1 -- 172.21.15.165:0/1742877607 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d8032440 con 0x7f92e8145310 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92e5400640 1 -- 172.21.15.165:0/1742877607 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8032810 con 0x7f92e8145310 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 -- 172.21.15.165:0/1742877607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 msgr2=0x7f92e815ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 --2- 172.21.15.165:0/1742877607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 0x7f92e815ff80 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009870 tx=0x7f92d802a270 comp rx=0 tx=0).stop 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 -- 172.21.15.165:0/1742877607 shutdown_connections 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 --2- 172.21.15.165:0/1742877607 >> v2:172.21.15.172:3300/0 conn(0x7f92e806b2f0 0x7f92e80b3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 --2- 172.21.15.165:0/1742877607 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e8145310 0x7f92e815ff80 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 -- 172.21.15.165:0/1742877607 >> 172.21.15.165:0/1742877607 conn(0x7f92e80b5af0 msgr2=0x7f92e80b5f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 -- 172.21.15.165:0/1742877607 shutdown_connections 2024-04-25T10:20:20.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.386+0000 7f92ed000640 1 -- 172.21.15.165:0/1742877607 wait complete. 2024-04-25T10:20:20.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.388+0000 7f92ed000640 1 Processor -- start 2024-04-25T10:20:20.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.388+0000 7f92ed000640 1 -- start start 2024-04-25T10:20:20.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92ed000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 0x7f92e815b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92ed000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e80afa00 0x7f92e80ce830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92ed000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8145310 0x7f92e80ced70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92ed000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80ad7e0 con 0x7f92e806b2f0 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92ed000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80b5870 con 0x7f92e8145310 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92ed000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80b51f0 con 0x7f92e80afa00 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92e5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 0x7f92e815b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92e5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 0x7f92e815b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58856/0 (socket says 172.21.15.165:58856) 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.389+0000 7f92e5e00640 1 -- 172.21.15.165:0/3161278381 learned_addr learned my addr 172.21.15.165:0/3161278381 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92dfe00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e80afa00 0x7f92e80ce830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e6800640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8145310 0x7f92e80ced70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e5e00640 1 -- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e80afa00 msgr2=0x7f92e80ce830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e5e00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e80afa00 0x7f92e80ce830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e5e00640 1 -- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8145310 msgr2=0x7f92e80ced70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e5e00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8145310 0x7f92e80ced70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e5e00640 1 -- 172.21.15.165:0/3161278381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e815cbe0 con 0x7f92e806b2f0 2024-04-25T10:20:20.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92dfe00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e80afa00 0x7f92e80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:20.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e6800640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8145310 0x7f92e80ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:20.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.390+0000 7f92e5e00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 0x7f92e815b600 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f92d802a2b0 tx=0x7f92d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.391+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8041030 con 0x7f92e806b2f0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.391+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d8030440 con 0x7f92e806b2f0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.391+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8037040 con 0x7f92e806b2f0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.391+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e815ce80 con 0x7f92e806b2f0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.391+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e815d440 con 0x7f92e806b2f0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.392+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e8073990 con 0x7f92e806b2f0 2024-04-25T10:20:20.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.393+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f92d8044070 con 0x7f92e806b2f0 2024-04-25T10:20:20.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.394+0000 7f92de000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92c406b550 0x7f92c406da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.394+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f92d80aaa40 con 0x7f92e806b2f0 2024-04-25T10:20:20.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.394+0000 7f92dfe00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92c406b550 0x7f92c406da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.395+0000 7f92dfe00640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92c406b550 0x7f92c406da10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f92e81440a0 tx=0x7f92d0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.399+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f92d8079830 con 0x7f92e806b2f0 2024-04-25T10:20:20.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.631+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f92e806b710 con 0x7f92e806b2f0 2024-04-25T10:20:20.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.632+0000 7f92de000640 1 -- 172.21.15.165:0/3161278381 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4964 (secure 0 0 0) 0x7f92d8079130 con 0x7f92e806b2f0 2024-04-25T10:20:20.635 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:20.635 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:01.817104+0000","modified":"2024-04-25T10:20:04.840723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11211":{"gid":11211,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1159947775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1159947775},{"type":"v1","addr":"172.21.15.165:6837","nonce":1159947775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:20:20.635 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 130 2024-04-25T10:20:20.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.634+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92c406b550 msgr2=0x7f92c406da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.634+0000 7f92ed000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92c406b550 0x7f92c406da10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f92e81440a0 tx=0x7f92d0009040 comp rx=0 tx=0).stop 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.634+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 msgr2=0x7f92e815b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.634+0000 7f92ed000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 0x7f92e815b600 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f92d802a2b0 tx=0x7f92d80046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 shutdown_connections 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f92c406b550 0x7f92c406da10 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f92e8145310 0x7f92e80ced70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f92e80afa00 0x7f92e80ce830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 --2- 172.21.15.165:0/3161278381 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f92e806b2f0 0x7f92e815b600 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 >> 172.21.15.165:0/3161278381 conn(0x7f92e80b5af0 msgr2=0x7f92e80b3ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 shutdown_connections 2024-04-25T10:20:20.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.635+0000 7f92ed000640 1 -- 172.21.15.165:0/3161278381 wait complete. 2024-04-25T10:20:20.655 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:20.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.844+0000 7f2021a00640 1 Processor -- start 2024-04-25T10:20:20.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.844+0000 7f2021a00640 1 -- start start 2024-04-25T10:20:20.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f2021a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f201c0b3800 0x7f201c06e580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f2021a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201c0b3420 con 0x7f201c15c100 2024-04-25T10:20:20.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f2021a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f201c0b2810 con 0x7f201c06eac0 2024-04-25T10:20:20.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f2021a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f201c0b2af0 con 0x7f201c0b3800 2024-04-25T10:20:20.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f201be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f201c15c100 0x7f201c071740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52354/0 (socket says 172.21.15.165:52354) 2024-04-25T10:20:20.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f201be00640 1 -- 172.21.15.165:0/203965306 learned_addr learned my addr 172.21.15.165:0/203965306 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:20.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f201b400640 1 --2- 172.21.15.165:0/203965306 >> v2:172.21.15.172:3300/0 conn(0x7f201c0b3800 0x7f201c06e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2932788468 0 0) 0x7f201c0b3420 con 0x7f201c15c100 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.846+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2000003520 con 0x7f201c15c100 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 456994459 0 0) 0x7f2000003520 con 0x7f201c15c100 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 >> v2:172.21.15.172:3300/0 conn(0x7f201c0b3800 msgr2=0x7f201c06e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 --2- 172.21.15.165:0/203965306 >> v2:172.21.15.172:3300/0 conn(0x7f201c0b3800 0x7f201c06e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 >> v1:172.21.15.172:6789/0 conn(0x7f201c06eac0 legacy=0x7f201c06d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c0b2dd0 con 0x7f201c15c100 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f200c001670 con 0x7f201c15c100 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 >> v1:172.21.15.165:6789/0 conn(0x7f201c15c100 legacy=0x7f201c071740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000039b0 0x7f2000003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2000004800 0x7f2000004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20000051c0 0x7f200000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c0b2af0 con 0x7f20000051c0 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f200000c0f0 con 0x7f2000004800 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f200000c2f0 con 0x7f20000039b0 2024-04-25T10:20:20.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f2021000640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20000051c0 0x7f200000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.847+0000 7f201b400640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2000004800 0x7f2000004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f201be00640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000039b0 0x7f2000003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f2021000640 1 -- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000039b0 msgr2=0x7f2000003db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f2021000640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000039b0 0x7f2000003db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f2021000640 1 -- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2000004800 msgr2=0x7f2000004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f2021000640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2000004800 0x7f2000004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f2021000640 1 -- 172.21.15.165:0/203965306 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2000003520 con 0x7f20000051c0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.848+0000 7f2021000640 1 -- 172.21.15.165:0/203965306 reap_dead start 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.849+0000 7f2021000640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20000051c0 0x7f200000b9c0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f2014004480 tx=0x7f2000005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f572c238aca44be server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.849+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014004080 con 0x7f20000051c0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.849+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f201402ad80 con 0x7f20000051c0 2024-04-25T10:20:20.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.849+0000 7f201aa00640 1 -- 172.21.15.165:0/203965306 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2014006d90 con 0x7f20000051c0 2024-04-25T10:20:20.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.849+0000 7f2021a00640 1 -- 172.21.15.165:0/203965306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20000051c0 msgr2=0x7f200000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.849+0000 7f2021a00640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20000051c0 0x7f200000b9c0 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f2014004480 tx=0x7f2000005830 comp rx=0 tx=0).stop 2024-04-25T10:20:20.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.850+0000 7f2021a00640 1 -- 172.21.15.165:0/203965306 shutdown_connections 2024-04-25T10:20:20.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.850+0000 7f2021a00640 1 --2- 172.21.15.165:0/203965306 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f20000051c0 0x7f200000b9c0 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.850+0000 7f2021a00640 1 -- 172.21.15.165:0/203965306 >> 172.21.15.165:0/203965306 conn(0x7f201c06abe0 msgr2=0x7f201c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:20.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.850+0000 7f2021a00640 1 -- 172.21.15.165:0/203965306 shutdown_connections 2024-04-25T10:20:20.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.850+0000 7f2021a00640 1 -- 172.21.15.165:0/203965306 wait complete. 2024-04-25T10:20:20.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.851+0000 7f2021a00640 1 Processor -- start 2024-04-25T10:20:20.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.851+0000 7f2021a00640 1 -- start start 2024-04-25T10:20:20.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.852+0000 7f2021a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000051c0 0x7f201c1f6df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.852+0000 7f2021a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 0x7f201c156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.852+0000 7f2021a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f201c0ce130 0x7f201c1569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.852+0000 7f2021a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c0711b0 con 0x7f201c0cdcb0 2024-04-25T10:20:20.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.852+0000 7f2021a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201c0b24e0 con 0x7f201c0ce130 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.852+0000 7f2021a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f201c071330 con 0x7f20000051c0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 0x7f201c156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 0x7f201c156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58874/0 (socket says 172.21.15.165:58874) 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 -- 172.21.15.165:0/1052100733 learned_addr learned my addr 172.21.15.165:0/1052100733 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 -- 172.21.15.165:0/1052100733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000051c0 msgr2=0x7f201c1f6df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000051c0 0x7f201c1f6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 -- 172.21.15.165:0/1052100733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f201c0ce130 msgr2=0x7f201c1569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f201c0ce130 0x7f201c1569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 -- 172.21.15.165:0/1052100733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f201c14edc0 con 0x7f201c0cdcb0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f201b400640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 0x7f201c156480 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f200800f550 tx=0x7f200800fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008013810 con 0x7f201c0cdcb0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.853+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f200801a070 con 0x7f201c0cdcb0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.854+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2008013bb0 con 0x7f201c0cdcb0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.854+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f201c14f0a0 con 0x7f201c0cdcb0 2024-04-25T10:20:20.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.854+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f201c14f660 con 0x7f201c0cdcb0 2024-04-25T10:20:20.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.855+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f201c073990 con 0x7f201c0cdcb0 2024-04-25T10:20:20.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.856+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2008012070 con 0x7f201c0cdcb0 2024-04-25T10:20:20.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.856+0000 7f2013e00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1ffc06b4d0 0x7f1ffc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:20.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.857+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f200808a3e0 con 0x7f201c0cdcb0 2024-04-25T10:20:20.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.857+0000 7f201be00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1ffc06b4d0 0x7f1ffc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:20.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.857+0000 7f201be00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1ffc06b4d0 0x7f1ffc06d990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f200c0017e0 tx=0x7f200c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:20.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:20.861+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f200801c310 con 0x7f201c0cdcb0 2024-04-25T10:20:21.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.043+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f201c166370 con 0x7f201c0cdcb0 2024-04-25T10:20:21.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.044+0000 7f2013e00640 1 -- 172.21.15.165:0/1052100733 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f2008059030 con 0x7f201c0cdcb0 2024-04-25T10:20:21.047 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:21.047 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":184,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:03.322848+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:21.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1ffc06b4d0 msgr2=0x7f1ffc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1ffc06b4d0 0x7f1ffc06d990 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f200c0017e0 tx=0x7f200c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 msgr2=0x7f201c156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 0x7f201c156480 secure :-1 s=READY pgs=2648 cs=0 l=1 rev1=1 crypto rx=0x7f200800f550 tx=0x7f200800fa10 comp rx=0 tx=0).stop 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 shutdown_connections 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1ffc06b4d0 0x7f1ffc06d990 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f201c0ce130 0x7f201c1569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f201c0cdcb0 0x7f201c156480 unknown :-1 s=CLOSED pgs=2648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 --2- 172.21.15.165:0/1052100733 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f20000051c0 0x7f201c1f6df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 >> 172.21.15.165:0/1052100733 conn(0x7f201c06abe0 msgr2=0x7f201c164b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 shutdown_connections 2024-04-25T10:20:21.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.048+0000 7f2021a00640 1 -- 172.21.15.165:0/1052100733 wait complete. 2024-04-25T10:20:21.069 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:19:59.548746+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '183', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:00.783201+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:21.070 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:21.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.264+0000 7f13f7200640 1 Processor -- start 2024-04-25T10:20:21.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.264+0000 7f13f7200640 1 -- start start 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.265+0000 7f13f7200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f13f806e860 0x7f13f80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f7200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f813e640 con 0x7f13f80b0730 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f806d390 con 0x7f13f813d9f0 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f7200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f806d670 con 0x7f13f806e860 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f6800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f13f806e860 0x7f13f80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13ed400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f13f80b0730 0x7f13f806e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52360/0 (socket says 172.21.15.165:52360) 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13ed400640 1 -- 172.21.15.165:0/1146846214 learned_addr learned my addr 172.21.15.165:0/1146846214 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f6800640 1 -- 172.21.15.165:0/1146846214 >> v1:172.21.15.172:6789/0 conn(0x7f13f813d9f0 legacy=0x7f13f813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f6800640 1 -- 172.21.15.165:0/1146846214 >> v1:172.21.15.165:6789/0 conn(0x7f13f80b0730 legacy=0x7f13f806e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.266+0000 7f13f6800640 1 -- 172.21.15.165:0/1146846214 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f806d950 con 0x7f13f806e860 2024-04-25T10:20:21.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.267+0000 7f13f6800640 1 --2- 172.21.15.165:0/1146846214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f80b0bc0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009d90 tx=0x7f13f002c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=740f2396cb984df1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.267+0000 7f13f5400640 1 -- 172.21.15.165:0/1146846214 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007740 con 0x7f13f806e860 2024-04-25T10:20:21.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f5400640 1 -- 172.21.15.165:0/1146846214 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f00078e0 con 0x7f13f806e860 2024-04-25T10:20:21.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f5400640 1 -- 172.21.15.165:0/1146846214 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007cb0 con 0x7f13f806e860 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 -- 172.21.15.165:0/1146846214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 msgr2=0x7f13f80b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 --2- 172.21.15.165:0/1146846214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f80b0bc0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009d90 tx=0x7f13f002c790 comp rx=0 tx=0).stop 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 -- 172.21.15.165:0/1146846214 shutdown_connections 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 --2- 172.21.15.165:0/1146846214 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f80b0bc0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 -- 172.21.15.165:0/1146846214 >> 172.21.15.165:0/1146846214 conn(0x7f13f8061cc0 msgr2=0x7f13f806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 -- 172.21.15.165:0/1146846214 shutdown_connections 2024-04-25T10:20:21.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.268+0000 7f13f7200640 1 -- 172.21.15.165:0/1146846214 wait complete. 2024-04-25T10:20:21.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.270+0000 7f13f7200640 1 Processor -- start 2024-04-25T10:20:21.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.270+0000 7f13f7200640 1 -- start start 2024-04-25T10:20:21.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f8151440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f80b0730 0x7f13f815d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f813d9f0 0x7f13f815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80cea00 con 0x7f13f806e860 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80ce880 con 0x7f13f80b0730 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f80ceb80 con 0x7f13f813d9f0 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f8151440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f8151440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58892/0 (socket says 172.21.15.165:58892) 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 -- 172.21.15.165:0/3181649077 learned_addr learned my addr 172.21.15.165:0/3181649077 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 -- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f813d9f0 msgr2=0x7f13f815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13ed400640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f80b0730 0x7f13f815d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f813d9f0 0x7f13f815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 -- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f80b0730 msgr2=0x7f13f815d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f80b0730 0x7f13f815d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.271+0000 7f13f5e00640 1 -- 172.21.15.165:0/3181649077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f8139600 con 0x7f13f806e860 2024-04-25T10:20:21.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.272+0000 7f13f5e00640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f8151440 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b410 tx=0x7f13e800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.272+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8015020 con 0x7f13f806e860 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.272+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8011830 con 0x7f13f806e860 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.272+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8011c20 con 0x7f13f806e860 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.272+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f80cdbd0 con 0x7f13f806e860 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.272+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f8139b00 con 0x7f13f806e860 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.273+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f81397d0 con 0x7f13f806e860 2024-04-25T10:20:21.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.274+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f13e8018020 con 0x7f13f806e860 2024-04-25T10:20:21.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.275+0000 7f13eea00640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13d406b4d0 0x7f13d406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.275+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f13e808a350 con 0x7f13f806e860 2024-04-25T10:20:21.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.276+0000 7f13ed400640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13d406b4d0 0x7f13d406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.276+0000 7f13ed400640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13d406b4d0 0x7f13d406d990 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f13f80cd9c0 tx=0x7f13e0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.280+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f13e801f020 con 0x7f13f806e860 2024-04-25T10:20:21.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.459+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f13f8073990 con 0x7f13f806e860 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.460+0000 7f13eea00640 1 -- 172.21.15.165:0/3181649077 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v184) v1 ==== 93+0+10 (secure 0 0 0) 0x7f13e80172c0 con 0x7f13f806e860 2024-04-25T10:20:21.463 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:20:21.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.462+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13d406b4d0 msgr2=0x7f13d406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.462+0000 7f13f7200640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13d406b4d0 0x7f13d406d990 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f13f80cd9c0 tx=0x7f13e0009040 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.462+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 msgr2=0x7f13f8151440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f8151440 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f13e800b410 tx=0x7f13e800b8d0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 shutdown_connections 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f13d406b4d0 0x7f13d406d990 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13f813d9f0 0x7f13f815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f13f80b0730 0x7f13f815d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 --2- 172.21.15.165:0/3181649077 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13f806e860 0x7f13f8151440 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 >> 172.21.15.165:0/3181649077 conn(0x7f13f8061cc0 msgr2=0x7f13f813c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 shutdown_connections 2024-04-25T10:20:21.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.463+0000 7f13f7200640 1 -- 172.21.15.165:0/3181649077 wait complete. 2024-04-25T10:20:21.483 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:21.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.673+0000 7fd5ba200640 1 Processor -- start 2024-04-25T10:20:21.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.673+0000 7fd5ba200640 1 -- start start 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5ba200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd5b415dd30 0x7fd5b40cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5ba200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd5b406bff0 0x7fd5b40b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5ba200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5b415e110 con 0x7fd5b40b5970 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5ba200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b406d000 con 0x7fd5b406bff0 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5ba200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5b40b2fd0 con 0x7fd5b415dd30 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5b3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd5b406bff0 0x7fd5b40b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5b9800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fd5b40b5970 0x7fd5b4161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52370/0 (socket says 172.21.15.165:52370) 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.675+0000 7fd5b9800640 1 -- 172.21.15.165:0/1815100446 learned_addr learned my addr 172.21.15.165:0/1815100446 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:21.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b8e00640 1 --2- 172.21.15.165:0/1815100446 >> v2:172.21.15.172:3300/0 conn(0x7fd5b415dd30 0x7fd5b40cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b3400640 1 -- 172.21.15.165:0/1815100446 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2033223588 0 0) 0x7fd5b415e110 con 0x7fd5b40b5970 2024-04-25T10:20:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b3400640 1 -- 172.21.15.165:0/1815100446 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd598003520 con 0x7fd5b40b5970 2024-04-25T10:20:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b3e00640 1 -- 172.21.15.165:0/1815100446 >> v2:172.21.15.172:3300/0 conn(0x7fd5b415dd30 msgr2=0x7fd5b40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b3e00640 1 --2- 172.21.15.165:0/1815100446 >> v2:172.21.15.172:3300/0 conn(0x7fd5b415dd30 0x7fd5b40cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b3e00640 1 -- 172.21.15.165:0/1815100446 >> v1:172.21.15.165:6789/0 conn(0x7fd5b40b5970 legacy=0x7fd5b4161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.676+0000 7fd5b3e00640 1 -- 172.21.15.165:0/1815100446 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b406c480 con 0x7fd5b406bff0 2024-04-25T10:20:21.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.677+0000 7fd5b3e00640 1 --2- 172.21.15.165:0/1815100446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b406bff0 0x7fd5b40b5430 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7fd5b408bfe0 tx=0x7fd59c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=776dfdd04eaed285 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5b3400640 1 -- 172.21.15.165:0/1815100446 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c037070 con 0x7fd5b406bff0 2024-04-25T10:20:21.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5b3400640 1 -- 172.21.15.165:0/1815100446 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd59c032440 con 0x7fd5b406bff0 2024-04-25T10:20:21.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5b3400640 1 -- 172.21.15.165:0/1815100446 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd59c032810 con 0x7fd5b406bff0 2024-04-25T10:20:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 -- 172.21.15.165:0/1815100446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b406bff0 msgr2=0x7fd5b40b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 --2- 172.21.15.165:0/1815100446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b406bff0 0x7fd5b40b5430 secure :-1 s=READY pgs=2653 cs=0 l=1 rev1=1 crypto rx=0x7fd5b408bfe0 tx=0x7fd59c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 -- 172.21.15.165:0/1815100446 shutdown_connections 2024-04-25T10:20:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 --2- 172.21.15.165:0/1815100446 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b406bff0 0x7fd5b40b5430 unknown :-1 s=CLOSED pgs=2653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 --2- 172.21.15.165:0/1815100446 >> v2:172.21.15.172:3300/0 conn(0x7fd5b415dd30 0x7fd5b40cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 -- 172.21.15.165:0/1815100446 >> 172.21.15.165:0/1815100446 conn(0x7fd5b40d0610 msgr2=0x7fd5b40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 -- 172.21.15.165:0/1815100446 shutdown_connections 2024-04-25T10:20:21.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.678+0000 7fd5ba200640 1 -- 172.21.15.165:0/1815100446 wait complete. 2024-04-25T10:20:21.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.681+0000 7fd5ba200640 1 Processor -- start 2024-04-25T10:20:21.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.681+0000 7fd5ba200640 1 -- start start 2024-04-25T10:20:21.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5ba200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5b40b5970 0x7fd5b40b78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5ba200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b415dd30 0x7fd5b41597f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5ba200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 0x7fd5b40b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5ba200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b406cd20 con 0x7fd5b4159d30 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5ba200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b406ca40 con 0x7fd5b415dd30 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5ba200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b40cf110 con 0x7fd5b40b5970 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5b9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 0x7fd5b40b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5b9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 0x7fd5b40b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58906/0 (socket says 172.21.15.165:58906) 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5b9800640 1 -- 172.21.15.165:0/3731690936 learned_addr learned my addr 172.21.15.165:0/3731690936 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.682+0000 7fd5b3e00640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b415dd30 0x7fd5b41597f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b8e00640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5b40b5970 0x7fd5b40b78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b9800640 1 -- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5b40b5970 msgr2=0x7fd5b40b78a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b9800640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5b40b5970 0x7fd5b40b78a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b9800640 1 -- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b415dd30 msgr2=0x7fd5b41597f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b9800640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b415dd30 0x7fd5b41597f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b9800640 1 -- 172.21.15.165:0/3731690936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5b41f3be0 con 0x7fd5b4159d30 2024-04-25T10:20:21.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b9800640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 0x7fd5b40b1610 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00cb40 tx=0x7fd5ac00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac017070 con 0x7fd5b4159d30 2024-04-25T10:20:21.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5ac00aca0 con 0x7fd5b4159d30 2024-04-25T10:20:21.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5ac012620 con 0x7fd5b4159d30 2024-04-25T10:20:21.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b41f3ec0 con 0x7fd5b4159d30 2024-04-25T10:20:21.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.683+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b41f4480 con 0x7fd5b4159d30 2024-04-25T10:20:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.685+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b4073990 con 0x7fd5b4159d30 2024-04-25T10:20:21.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.686+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd5ac01a070 con 0x7fd5b4159d30 2024-04-25T10:20:21.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.687+0000 7fd5b0c00640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59006b4d0 0x7fd59006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:21.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.687+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd5ac08a2c0 con 0x7fd5b4159d30 2024-04-25T10:20:21.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.687+0000 7fd5b8e00640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59006b4d0 0x7fd59006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:21.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.687+0000 7fd5b8e00640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59006b4d0 0x7fd59006d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4006fd0 tx=0x7fd5a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:21.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.691+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd5ac058f10 con 0x7fd5b4159d30 2024-04-25T10:20:21.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.885+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd5b4067df0 con 0x7fd5b4159d30 2024-04-25T10:20:21.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.886+0000 7fd5b0c00640 1 -- 172.21.15.165:0/3731690936 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v184) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd5ac058810 con 0x7fd5b4159d30 2024-04-25T10:20:21.888 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:20:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.888+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59006b4d0 msgr2=0x7fd59006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.888+0000 7fd5ba200640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59006b4d0 0x7fd59006d990 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd5a4006fd0 tx=0x7fd5a4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:21.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.888+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 msgr2=0x7fd5b40b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.888+0000 7fd5ba200640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 0x7fd5b40b1610 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac00cb40 tx=0x7fd5ac00a060 comp rx=0 tx=0).stop 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 shutdown_connections 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd59006b4d0 0x7fd59006d990 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd5b4159d30 0x7fd5b40b1610 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd5b415dd30 0x7fd5b41597f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 --2- 172.21.15.165:0/3731690936 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd5b40b5970 0x7fd5b40b78a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 >> 172.21.15.165:0/3731690936 conn(0x7fd5b40d0610 msgr2=0x7fd5b40740d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 shutdown_connections 2024-04-25T10:20:21.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:21.889+0000 7fd5ba200640 1 -- 172.21.15.165:0/3731690936 wait complete. 2024-04-25T10:20:21.909 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:22.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.102+0000 7f81ee000640 1 Processor -- start 2024-04-25T10:20:22.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.102+0000 7f81ee000640 1 -- start start 2024-04-25T10:20:22.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81ee000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f81e80ccf30 0x7f81e8162bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81ee000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e80cdcd0 con 0x7f81e8163100 2024-04-25T10:20:22.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81ee000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81e80cd430 con 0x7f81e815f950 2024-04-25T10:20:22.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81ee000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81e806f100 con 0x7f81e80ccf30 2024-04-25T10:20:22.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81ed600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f81e8163100 0x7f81e8165a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52380/0 (socket says 172.21.15.165:52380) 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81dfe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f81e80ccf30 0x7f81e8162bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.104+0000 7f81ed600640 1 -- 172.21.15.165:0/139987432 learned_addr learned my addr 172.21.15.165:0/139987432 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.105+0000 7f81e7e00640 1 -- 172.21.15.165:0/139987432 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375316392 0 0) 0x7f81e80cdcd0 con 0x7f81e8163100 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.105+0000 7f81e7e00640 1 -- 172.21.15.165:0/139987432 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81cc003520 con 0x7f81e8163100 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.105+0000 7f81dfe00640 1 -- 172.21.15.165:0/139987432 >> v1:172.21.15.172:6789/0 conn(0x7f81e815f950 legacy=0x7f81e806e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.105+0000 7f81dfe00640 1 -- 172.21.15.165:0/139987432 >> v1:172.21.15.165:6789/0 conn(0x7f81e8163100 legacy=0x7f81e8165a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.105+0000 7f81dfe00640 1 -- 172.21.15.165:0/139987432 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e80cd710 con 0x7f81e80ccf30 2024-04-25T10:20:22.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.106+0000 7f81dfe00640 1 --2- 172.21.15.165:0/139987432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e80ccf30 0x7f81e8162bc0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f81d00097d0 tx=0x7f81d002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6894f8be765b1616 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81e7e00640 1 -- 172.21.15.165:0/139987432 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0037070 con 0x7f81e80ccf30 2024-04-25T10:20:22.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81e7e00640 1 -- 172.21.15.165:0/139987432 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0032440 con 0x7f81e80ccf30 2024-04-25T10:20:22.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81e7e00640 1 -- 172.21.15.165:0/139987432 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0032810 con 0x7f81e80ccf30 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 -- 172.21.15.165:0/139987432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e80ccf30 msgr2=0x7f81e8162bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 --2- 172.21.15.165:0/139987432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e80ccf30 0x7f81e8162bc0 secure :-1 s=READY pgs=2656 cs=0 l=1 rev1=1 crypto rx=0x7f81d00097d0 tx=0x7f81d002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 -- 172.21.15.165:0/139987432 shutdown_connections 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 --2- 172.21.15.165:0/139987432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e80ccf30 0x7f81e8162bc0 unknown :-1 s=CLOSED pgs=2656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 -- 172.21.15.165:0/139987432 >> 172.21.15.165:0/139987432 conn(0x7f81e8061cc0 msgr2=0x7f81e806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 -- 172.21.15.165:0/139987432 shutdown_connections 2024-04-25T10:20:22.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.107+0000 7f81ee000640 1 -- 172.21.15.165:0/139987432 wait complete. 2024-04-25T10:20:22.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.109+0000 7f81ee000640 1 Processor -- start 2024-04-25T10:20:22.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.109+0000 7f81ee000640 1 -- start start 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ee000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e80ccf30 0x7f81e8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ee000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81e815f950 0x7f81e8147280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ee000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 0x7f81e8157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ee000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e81626a0 con 0x7f81e8153300 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ee000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e8162440 con 0x7f81e80ccf30 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ee000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81e815fd30 con 0x7f81e815f950 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.110+0000 7f81ed600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 0x7f81e8157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 0x7f81e8157d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58924/0 (socket says 172.21.15.165:58924) 2024-04-25T10:20:22.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 -- 172.21.15.165:0/1941346656 learned_addr learned my addr 172.21.15.165:0/1941346656 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 -- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81e815f950 msgr2=0x7f81e8147280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81dfe00640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81e815f950 0x7f81e8147280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ecc00640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e80ccf30 0x7f81e8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81e815f950 0x7f81e8147280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 -- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e80ccf30 msgr2=0x7f81e8155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e80ccf30 0x7f81e8155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 -- 172.21.15.165:0/1941346656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e815deb0 con 0x7f81e8153300 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81ed600640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 0x7f81e8157d00 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f81e000caa0 tx=0x7f81e000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.111+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0017070 con 0x7f81e8153300 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.112+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e000aba0 con 0x7f81e8153300 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.112+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e0012400 con 0x7f81e8153300 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.112+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e81566d0 con 0x7f81e8153300 2024-04-25T10:20:22.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.112+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e8155c40 con 0x7f81e8153300 2024-04-25T10:20:22.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.113+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e815e080 con 0x7f81e8153300 2024-04-25T10:20:22.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.114+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81e001a070 con 0x7f81e8153300 2024-04-25T10:20:22.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.115+0000 7f81e5600640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.115+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f81e008a190 con 0x7f81e8153300 2024-04-25T10:20:22.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.115+0000 7f81ecc00640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.116+0000 7f81ecc00640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81c406b4d0 0x7f81c406d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f81e815f190 tx=0x7f81d8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.120+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81e0058de0 con 0x7f81e8153300 2024-04-25T10:20:22.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.297+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f81e80b1a50 con 0x7f81e8153300 2024-04-25T10:20:22.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.299+0000 7f81e5600640 1 -- 172.21.15.165:0/1941346656 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v184) v1 ==== 100+0+11 (secure 0 0 0) 0x7f81e00586e0 con 0x7f81e8153300 2024-04-25T10:20:22.301 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81c406b4d0 msgr2=0x7f81c406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81c406b4d0 0x7f81c406d990 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f81e815f190 tx=0x7f81d8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 msgr2=0x7f81e8157d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 0x7f81e8157d00 secure :-1 s=READY pgs=2657 cs=0 l=1 rev1=1 crypto rx=0x7f81e000caa0 tx=0x7f81e000cf60 comp rx=0 tx=0).stop 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 shutdown_connections 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81c406b4d0 0x7f81c406d990 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81e8153300 0x7f81e8157d00 unknown :-1 s=CLOSED pgs=2657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81e815f950 0x7f81e8147280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 --2- 172.21.15.165:0/1941346656 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81e80ccf30 0x7f81e8155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 >> 172.21.15.165:0/1941346656 conn(0x7f81e8061cc0 msgr2=0x7f81e80b33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 shutdown_connections 2024-04-25T10:20:22.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.301+0000 7f81ee000640 1 -- 172.21.15.165:0/1941346656 wait complete. 2024-04-25T10:20:22.320 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:22.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.503+0000 7fcc9ec00640 1 Processor -- start 2024-04-25T10:20:22.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.504+0000 7fcc9ec00640 1 -- start start 2024-04-25T10:20:22.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.505+0000 7fcc9ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e900 0x7fcc980adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.505+0000 7fcc9ec00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc980ae0d0 0x7fcc98070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.505+0000 7fcc9ec00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc980ae590 con 0x7fcc9815e520 2024-04-25T10:20:22.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.505+0000 7fcc9ec00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc980aee30 con 0x7fcc980ae0d0 2024-04-25T10:20:22.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.505+0000 7fcc9ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980ae870 con 0x7fcc9815e900 2024-04-25T10:20:22.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc980ae0d0 0x7fcc98070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcc980ae0d0 0x7fcc98070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58940/0 (socket says 172.21.15.165:58940) 2024-04-25T10:20:22.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 -- 172.21.15.165:0/3166528218 learned_addr learned my addr 172.21.15.165:0/3166528218 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:22.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 -- 172.21.15.165:0/3166528218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e900 msgr2=0x7fcc980adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9ce00640 1 --2- 172.21.15.165:0/3166528218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e900 0x7fcc980adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 --2- 172.21.15.165:0/3166528218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e900 0x7fcc980adb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 -- 172.21.15.165:0/3166528218 >> v1:172.21.15.172:6789/0 conn(0x7fcc9815e520 legacy=0x7fcc9815f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.506+0000 7fcc9e200640 1 -- 172.21.15.165:0/3166528218 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc980aeb50 con 0x7fcc980ae0d0 2024-04-25T10:20:22.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.507+0000 7fcc9e200640 1 --2- 172.21.15.165:0/3166528218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc980ae0d0 0x7fcc98070ff0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009870 tx=0x7fcc9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74f17491dc742867 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.507+0000 7fcc97e00640 1 -- 172.21.15.165:0/3166528218 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90037070 con 0x7fcc980ae0d0 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.507+0000 7fcc97e00640 1 -- 172.21.15.165:0/3166528218 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc90032440 con 0x7fcc980ae0d0 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.507+0000 7fcc97e00640 1 -- 172.21.15.165:0/3166528218 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc90032810 con 0x7fcc980ae0d0 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 -- 172.21.15.165:0/3166528218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc980ae0d0 msgr2=0x7fcc98070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/3166528218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc980ae0d0 0x7fcc98070ff0 secure :-1 s=READY pgs=2658 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009870 tx=0x7fcc9002a210 comp rx=0 tx=0).stop 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 -- 172.21.15.165:0/3166528218 shutdown_connections 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/3166528218 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc980ae0d0 0x7fcc98070ff0 unknown :-1 s=CLOSED pgs=2658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/3166528218 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc9815e900 0x7fcc980adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 -- 172.21.15.165:0/3166528218 >> 172.21.15.165:0/3166528218 conn(0x7fcc9806abe0 msgr2=0x7fcc9806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 -- 172.21.15.165:0/3166528218 shutdown_connections 2024-04-25T10:20:22.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.508+0000 7fcc9ec00640 1 -- 172.21.15.165:0/3166528218 wait complete. 2024-04-25T10:20:22.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.510+0000 7fcc9ec00640 1 Processor -- start 2024-04-25T10:20:22.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.510+0000 7fcc9ec00640 1 -- start start 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ec00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc980ae0d0 0x7fcc98159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ec00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 0x7fcc9815ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ec00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815e900 0x7fcc980cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ec00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc98071530 con 0x7fcc9815e520 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ec00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc9815ed60 con 0x7fcc9815e900 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ec00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc980716b0 con 0x7fcc980ae0d0 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 0x7fcc9815ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 0x7fcc9815ad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58952/0 (socket says 172.21.15.165:58952) 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.511+0000 7fcc9ce00640 1 -- 172.21.15.165:0/193049923 learned_addr learned my addr 172.21.15.165:0/193049923 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:22.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9d800640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc980ae0d0 0x7fcc98159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9e200640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815e900 0x7fcc980cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ce00640 1 -- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc980ae0d0 msgr2=0x7fcc98159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ce00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc980ae0d0 0x7fcc98159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ce00640 1 -- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815e900 msgr2=0x7fcc980cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ce00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815e900 0x7fcc980cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ce00640 1 -- 172.21.15.165:0/193049923 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc98159c60 con 0x7fcc9815e520 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ce00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 0x7fcc9815ad00 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c7f0 tx=0x7fcc8400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84015070 con 0x7fcc9815e520 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc84004220 con 0x7fcc9815e520 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc98159f40 con 0x7fcc9815e520 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.512+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc84007620 con 0x7fcc9815e520 2024-04-25T10:20:22.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.513+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc9815a500 con 0x7fcc9815e520 2024-04-25T10:20:22.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.514+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc98073990 con 0x7fcc9815e520 2024-04-25T10:20:22.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.514+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc840079d0 con 0x7fcc9815e520 2024-04-25T10:20:22.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.516+0000 7fcc95600640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.516+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fcc8408a7f0 con 0x7fcc9815e520 2024-04-25T10:20:22.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.516+0000 7fcc9d800640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.517+0000 7fcc9d800640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c007fd0 tx=0x7fcc8c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.520+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcc84059440 con 0x7fcc9815e520 2024-04-25T10:20:22.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.755+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc980cf110 con 0x7fcc9815e520 2024-04-25T10:20:22.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.755+0000 7fcc95600640 1 -- 172.21.15.165:0/193049923 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4964 (secure 0 0 0) 0x7fcc84058d40 con 0x7fcc9815e520 2024-04-25T10:20:22.759 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:22.759 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:01.817104+0000","modified":"2024-04-25T10:20:04.840723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11211":{"gid":11211,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1159947775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1159947775},{"type":"v1","addr":"172.21.15.165:6837","nonce":1159947775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:20:22.759 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 130 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 msgr2=0x7fcc7806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c007fd0 tx=0x7fcc8c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 msgr2=0x7fcc9815ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 0x7fcc9815ad00 secure :-1 s=READY pgs=2659 cs=0 l=1 rev1=1 crypto rx=0x7fcc8400c7f0 tx=0x7fcc8400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 shutdown_connections 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcc7806b4d0 0x7fcc7806d990 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc9815e900 0x7fcc980cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcc9815e520 0x7fcc9815ad00 unknown :-1 s=CLOSED pgs=2659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 --2- 172.21.15.165:0/193049923 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc980ae0d0 0x7fcc98159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 >> 172.21.15.165:0/193049923 conn(0x7fcc9806abe0 msgr2=0x7fcc9815f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 shutdown_connections 2024-04-25T10:20:22.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.758+0000 7fcc9ec00640 1 -- 172.21.15.165:0/193049923 wait complete. 2024-04-25T10:20:22.781 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:22.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.983+0000 7fda2ca00640 1 Processor -- start 2024-04-25T10:20:22.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.983+0000 7fda2ca00640 1 -- start start 2024-04-25T10:20:22.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda2ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda2815db90 0x7fda280b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda2ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda281615a0 0x7fda280cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda2ca00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda280d33f0 con 0x7fda280d04f0 2024-04-25T10:20:22.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda2ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda280d0980 con 0x7fda2815db90 2024-04-25T10:20:22.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda2ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda280d1f50 con 0x7fda281615a0 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda26800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fda2815db90 0x7fda280b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda27200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fda280d04f0 0x7fda28061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52382/0 (socket says 172.21.15.165:52382) 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.985+0000 7fda27200640 1 -- 172.21.15.165:0/3758140033 learned_addr learned my addr 172.21.15.165:0/3758140033 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.986+0000 7fda25e00640 1 --2- 172.21.15.165:0/3758140033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda281615a0 0x7fda280cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.986+0000 7fda26800640 1 -- 172.21.15.165:0/3758140033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda281615a0 msgr2=0x7fda280cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.986+0000 7fda26800640 1 --2- 172.21.15.165:0/3758140033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda281615a0 0x7fda280cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.986+0000 7fda26800640 1 -- 172.21.15.165:0/3758140033 >> v1:172.21.15.165:6789/0 conn(0x7fda280d04f0 legacy=0x7fda28061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.986+0000 7fda26800640 1 -- 172.21.15.165:0/3758140033 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda28161df0 con 0x7fda2815db90 2024-04-25T10:20:22.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.986+0000 7fda25400640 1 -- 172.21.15.165:0/3758140033 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 548899302 0 0) 0x7fda280d33f0 con 0x7fda280d04f0 2024-04-25T10:20:22.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.987+0000 7fda26800640 1 --2- 172.21.15.165:0/3758140033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2815db90 0x7fda280b93c0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7fda100097a0 tx=0x7fda1002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b5e2d8d7a90923 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.987+0000 7fda25400640 1 -- 172.21.15.165:0/3758140033 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10037070 con 0x7fda2815db90 2024-04-25T10:20:22.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.987+0000 7fda25400640 1 -- 172.21.15.165:0/3758140033 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda10032440 con 0x7fda2815db90 2024-04-25T10:20:22.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.987+0000 7fda25400640 1 -- 172.21.15.165:0/3758140033 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda10032810 con 0x7fda2815db90 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 -- 172.21.15.165:0/3758140033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2815db90 msgr2=0x7fda280b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 --2- 172.21.15.165:0/3758140033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2815db90 0x7fda280b93c0 secure :-1 s=READY pgs=2661 cs=0 l=1 rev1=1 crypto rx=0x7fda100097a0 tx=0x7fda1002a180 comp rx=0 tx=0).stop 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 -- 172.21.15.165:0/3758140033 shutdown_connections 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 --2- 172.21.15.165:0/3758140033 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda281615a0 0x7fda280cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 --2- 172.21.15.165:0/3758140033 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda2815db90 0x7fda280b93c0 unknown :-1 s=CLOSED pgs=2661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 -- 172.21.15.165:0/3758140033 >> 172.21.15.165:0/3758140033 conn(0x7fda280b52b0 msgr2=0x7fda280b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 -- 172.21.15.165:0/3758140033 shutdown_connections 2024-04-25T10:20:22.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.988+0000 7fda2ca00640 1 -- 172.21.15.165:0/3758140033 wait complete. 2024-04-25T10:20:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.990+0000 7fda2ca00640 1 Processor -- start 2024-04-25T10:20:22.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.990+0000 7fda2ca00640 1 -- start start 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda2ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda280d04f0 0x7fda281f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda2ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2815db90 0x7fda28155170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda2ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 0x7fda280b8020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda2ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda280d1be0 con 0x7fda281615a0 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda2ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda281623b0 con 0x7fda280d04f0 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda2ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda28161a20 con 0x7fda2815db90 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda27200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 0x7fda280b8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda27200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 0x7fda280b8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:58974/0 (socket says 172.21.15.165:58974) 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.991+0000 7fda27200640 1 -- 172.21.15.165:0/2624658468 learned_addr learned my addr 172.21.15.165:0/2624658468 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:22.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda27200640 1 -- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2815db90 msgr2=0x7fda28155170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda27200640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2815db90 0x7fda28155170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda27200640 1 -- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda280d04f0 msgr2=0x7fda281f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda26800640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda280d04f0 0x7fda281f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda27200640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda280d04f0 0x7fda281f2da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda27200640 1 -- 172.21.15.165:0/2624658468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda281556b0 con 0x7fda281615a0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda27200640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 0x7fda280b8020 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fda2000c3c0 tx=0x7fda2000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda200074e0 con 0x7fda281615a0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda200047e0 con 0x7fda281615a0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda20004c50 con 0x7fda281615a0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.992+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda281540d0 con 0x7fda281615a0 2024-04-25T10:20:22.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.993+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda28154690 con 0x7fda281615a0 2024-04-25T10:20:22.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.994+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda28073990 con 0x7fda281615a0 2024-04-25T10:20:22.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.995+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fda20011070 con 0x7fda281615a0 2024-04-25T10:20:22.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.996+0000 7fda1ea00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0406b4d0 0x7fda0406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:22.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.996+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fda20089f70 con 0x7fda281615a0 2024-04-25T10:20:22.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.996+0000 7fda26800640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0406b4d0 0x7fda0406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:22.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:22.997+0000 7fda26800640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0406b4d0 0x7fda0406d990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fda10034000 tx=0x7fda1002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.000+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fda2001a2b0 con 0x7fda281615a0 2024-04-25T10:20:23.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.179+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda280b7300 con 0x7fda281615a0 2024-04-25T10:20:23.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.181+0000 7fda1ea00640 1 -- 172.21.15.165:0/2624658468 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16988 (secure 0 0 0) 0x7fda20058bc0 con 0x7fda281615a0 2024-04-25T10:20:23.184 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:23.185 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":184,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:03.322848+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0406b4d0 msgr2=0x7fda0406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0406b4d0 0x7fda0406d990 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fda10034000 tx=0x7fda1002e000 comp rx=0 tx=0).stop 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 msgr2=0x7fda280b8020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 0x7fda280b8020 secure :-1 s=READY pgs=2662 cs=0 l=1 rev1=1 crypto rx=0x7fda2000c3c0 tx=0x7fda2000c880 comp rx=0 tx=0).stop 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 shutdown_connections 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fda0406b4d0 0x7fda0406d990 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fda281615a0 0x7fda280b8020 unknown :-1 s=CLOSED pgs=2662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fda2815db90 0x7fda28155170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 --2- 172.21.15.165:0/2624658468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fda280d04f0 0x7fda281f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 >> 172.21.15.165:0/2624658468 conn(0x7fda280b52b0 msgr2=0x7fda28158360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 shutdown_connections 2024-04-25T10:20:23.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.184+0000 7fda2ca00640 1 -- 172.21.15.165:0/2624658468 wait complete. 2024-04-25T10:20:23.205 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:20:23.205 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.389+0000 7fde4be00640 1 Processor -- start 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.389+0000 7fde4be00640 1 -- start start 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.391+0000 7fde4be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fde4c15dd00 0x7fde4c0cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.391+0000 7fde4be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c15bf30 con 0x7fde4c144440 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.391+0000 7fde4be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c15c210 con 0x7fde4c071700 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.391+0000 7fde4be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c15c860 con 0x7fde4c15dd00 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.391+0000 7fde4aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fde4c144440 0x7fde4c070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52398/0 (socket says 172.21.15.165:52398) 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.391+0000 7fde4aa00640 1 -- 172.21.15.165:0/2082426152 learned_addr learned my addr 172.21.15.165:0/2082426152 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:23.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4b400640 1 --2- 172.21.15.165:0/2082426152 >> v2:172.21.15.172:3300/0 conn(0x7fde4c15dd00 0x7fde4c0cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193220074 0 0) 0x7fde4c15bf30 con 0x7fde4c144440 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde30003520 con 0x7fde4c144440 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2896214187 0 0) 0x7fde30003520 con 0x7fde4c144440 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 >> v2:172.21.15.172:3300/0 conn(0x7fde4c15dd00 msgr2=0x7fde4c0cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 --2- 172.21.15.165:0/2082426152 >> v2:172.21.15.172:3300/0 conn(0x7fde4c15dd00 0x7fde4c0cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 >> v1:172.21.15.172:6789/0 conn(0x7fde4c071700 legacy=0x7fde4c0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c15c4f0 con 0x7fde4c144440 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fde3c001670 con 0x7fde4c144440 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 >> v1:172.21.15.165:6789/0 conn(0x7fde4c144440 legacy=0x7fde4c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde300039b0 0x7fde30003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde30004800 0x7fde30004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde3000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15c860 con 0x7fde30005210 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000c120 con 0x7fde300039b0 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.392+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3000c320 con 0x7fde30004800 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4b400640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde3000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde43e00640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde30004800 0x7fde30004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4b400640 1 -- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde30004800 msgr2=0x7fde30004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4b400640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde30004800 0x7fde30004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4b400640 1 -- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde300039b0 msgr2=0x7fde30003df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4aa00640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde300039b0 0x7fde30003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4b400640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde300039b0 0x7fde30003df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.393+0000 7fde4b400640 1 -- 172.21.15.165:0/2082426152 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde30003520 con 0x7fde30005210 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.394+0000 7fde4b400640 1 -- 172.21.15.165:0/2082426152 reap_dead start 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.394+0000 7fde4b400640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde3000b9c0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fde44007330 tx=0x7fde30005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36103d5af018080f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.394+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde44009be0 con 0x7fde30005210 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.394+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde44036600 con 0x7fde30005210 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.394+0000 7fde4a000640 1 -- 172.21.15.165:0/2082426152 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440369d0 con 0x7fde30005210 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 -- 172.21.15.165:0/2082426152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 msgr2=0x7fde3000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde3000b9c0 secure :-1 s=READY pgs=2664 cs=0 l=1 rev1=1 crypto rx=0x7fde44007330 tx=0x7fde30005830 comp rx=0 tx=0).stop 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 -- 172.21.15.165:0/2082426152 shutdown_connections 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 --2- 172.21.15.165:0/2082426152 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde3000b9c0 unknown :-1 s=CLOSED pgs=2664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 -- 172.21.15.165:0/2082426152 >> 172.21.15.165:0/2082426152 conn(0x7fde4c06abe0 msgr2=0x7fde4c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 -- 172.21.15.165:0/2082426152 shutdown_connections 2024-04-25T10:20:23.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.395+0000 7fde4be00640 1 -- 172.21.15.165:0/2082426152 wait complete. 2024-04-25T10:20:23.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.397+0000 7fde4be00640 1 Processor -- start 2024-04-25T10:20:23.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.397+0000 7fde4be00640 1 -- start start 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde4c13a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde4c0cb2d0 0x7fde4c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4c13cef0 0x7fde4c13d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c071170 con 0x7fde30005210 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c070ff0 con 0x7fde4c0cb2d0 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0712f0 con 0x7fde4c13cef0 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde4c13a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde4c13a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59000/0 (socket says 172.21.15.165:59000) 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 -- 172.21.15.165:0/3836558649 learned_addr learned my addr 172.21.15.165:0/3836558649 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 -- 172.21.15.165:0/3836558649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4c13cef0 msgr2=0x7fde4c13d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4c13cef0 0x7fde4c13d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 -- 172.21.15.165:0/3836558649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde4c0cb2d0 msgr2=0x7fde4c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde4c0cb2d0 0x7fde4c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.398+0000 7fde4aa00640 1 -- 172.21.15.165:0/3836558649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c13a6e0 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.399+0000 7fde4aa00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde4c13a1a0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fde3c000ff0 tx=0x7fde3c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.399+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c019070 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.399+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c0072f0 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.399+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c0047e0 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.399+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c0c8a90 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.399+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c0c9050 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.400+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c073990 con 0x7fde30005210 2024-04-25T10:20:23.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.401+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fde3c01c070 con 0x7fde30005210 2024-04-25T10:20:23.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.402+0000 7fde42a00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde2806b500 0x7fde2806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.402+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fde3c089c10 con 0x7fde30005210 2024-04-25T10:20:23.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.402+0000 7fde43e00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde2806b500 0x7fde2806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.403+0000 7fde43e00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde2806b500 0x7fde2806d9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fde4c1408d0 tx=0x7fde34009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.406+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fde3c058210 con 0x7fde30005210 2024-04-25T10:20:23.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.633+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde4c143080 con 0x7fde30005210 2024-04-25T10:20:23.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.634+0000 7fde42a00640 1 -- 172.21.15.165:0/3836558649 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4964 (secure 0 0 0) 0x7fde3c05c2b0 con 0x7fde30005210 2024-04-25T10:20:23.637 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:23.637 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:01.817104+0000","modified":"2024-04-25T10:20:04.840723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11211":{"gid":11211,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1159947775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1159947775},{"type":"v1","addr":"172.21.15.165:6837","nonce":1159947775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:20:23.637 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 130 2024-04-25T10:20:23.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.636+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde2806b500 msgr2=0x7fde2806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.636+0000 7fde4be00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde2806b500 0x7fde2806d9c0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7fde4c1408d0 tx=0x7fde34009040 comp rx=0 tx=0).stop 2024-04-25T10:20:23.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.636+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 msgr2=0x7fde4c13a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.636+0000 7fde4be00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde4c13a1a0 secure :-1 s=READY pgs=2665 cs=0 l=1 rev1=1 crypto rx=0x7fde3c000ff0 tx=0x7fde3c00dc00 comp rx=0 tx=0).stop 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 shutdown_connections 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fde2806b500 0x7fde2806d9c0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fde4c13cef0 0x7fde4c13d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fde4c0cb2d0 0x7fde4c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 --2- 172.21.15.165:0/3836558649 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fde30005210 0x7fde4c13a1a0 unknown :-1 s=CLOSED pgs=2665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 >> 172.21.15.165:0/3836558649 conn(0x7fde4c06abe0 msgr2=0x7fde4c144820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 shutdown_connections 2024-04-25T10:20:23.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.637+0000 7fde4be00640 1 -- 172.21.15.165:0/3836558649 wait complete. 2024-04-25T10:20:23.657 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.828+0000 7fd8ba800640 1 Processor -- start 2024-04-25T10:20:23.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.828+0000 7fd8ba800640 1 -- start start 2024-04-25T10:20:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.830+0000 7fd8ba800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8b415db70 0x7fd8b40cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.830+0000 7fd8ba800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd8b40cfc20 0x7fd8b406d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8ba800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b4071650 con 0x7fd8b415df50 2024-04-25T10:20:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8ba800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b40cbd80 con 0x7fd8b40cfc20 2024-04-25T10:20:23.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8ba800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b4161910 con 0x7fd8b415db70 2024-04-25T10:20:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd8b40cfc20 0x7fd8b406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd8b40cfc20 0x7fd8b406d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59016/0 (socket says 172.21.15.165:59016) 2024-04-25T10:20:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3e00640 1 -- 172.21.15.165:0/353408922 learned_addr learned my addr 172.21.15.165:0/353408922 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:23.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3400640 1 --2- 172.21.15.165:0/353408922 >> v2:172.21.15.172:3300/0 conn(0x7fd8b415db70 0x7fd8b40cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3e00640 1 -- 172.21.15.165:0/353408922 >> v2:172.21.15.172:3300/0 conn(0x7fd8b415db70 msgr2=0x7fd8b40cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:23.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3e00640 1 --2- 172.21.15.165:0/353408922 >> v2:172.21.15.172:3300/0 conn(0x7fd8b415db70 0x7fd8b40cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.831+0000 7fd8b3e00640 1 -- 172.21.15.165:0/353408922 >> v1:172.21.15.172:6789/0 conn(0x7fd8b415df50 legacy=0x7fd8b40cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:23.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.832+0000 7fd8b3e00640 1 -- 172.21.15.165:0/353408922 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b40cf230 con 0x7fd8b40cfc20 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.832+0000 7fd8b3e00640 1 --2- 172.21.15.165:0/353408922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b406d290 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40cd3d0 tx=0x7fd8ac02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e656a388eadfb70 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8b2000640 1 -- 172.21.15.165:0/353408922 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac036070 con 0x7fd8b40cfc20 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8b2000640 1 -- 172.21.15.165:0/353408922 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac031600 con 0x7fd8b40cfc20 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8b2000640 1 -- 172.21.15.165:0/353408922 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0319d0 con 0x7fd8b40cfc20 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8ba800640 1 -- 172.21.15.165:0/353408922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 msgr2=0x7fd8b406d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8ba800640 1 --2- 172.21.15.165:0/353408922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b406d290 secure :-1 s=READY pgs=2666 cs=0 l=1 rev1=1 crypto rx=0x7fd8b40cd3d0 tx=0x7fd8ac02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8ba800640 1 -- 172.21.15.165:0/353408922 shutdown_connections 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8ba800640 1 --2- 172.21.15.165:0/353408922 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b406d290 unknown :-1 s=CLOSED pgs=2666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8ba800640 1 --2- 172.21.15.165:0/353408922 >> v2:172.21.15.172:3300/0 conn(0x7fd8b415db70 0x7fd8b40cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.833+0000 7fd8ba800640 1 -- 172.21.15.165:0/353408922 >> 172.21.15.165:0/353408922 conn(0x7fd8b406abe0 msgr2=0x7fd8b406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:23.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.834+0000 7fd8ba800640 1 -- 172.21.15.165:0/353408922 shutdown_connections 2024-04-25T10:20:23.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.834+0000 7fd8ba800640 1 -- 172.21.15.165:0/353408922 wait complete. 2024-04-25T10:20:23.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.835+0000 7fd8ba800640 1 Processor -- start 2024-04-25T10:20:23.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.835+0000 7fd8ba800640 1 -- start start 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8ba800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b4147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8ba800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8b415db70 0x7fd8b41f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8ba800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8b415df50 0x7fd8b4159ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8ba800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4162120 con 0x7fd8b40cfc20 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8ba800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4161e40 con 0x7fd8b415df50 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8ba800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b4162640 con 0x7fd8b415db70 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.836+0000 7fd8b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b4147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b4147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59032/0 (socket says 172.21.15.165:59032) 2024-04-25T10:20:23.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 -- 172.21.15.165:0/3953039439 learned_addr learned my addr 172.21.15.165:0/3953039439 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 -- 172.21.15.165:0/3953039439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8b415db70 msgr2=0x7fd8b41f3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8b415db70 0x7fd8b41f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 -- 172.21.15.165:0/3953039439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8b415df50 msgr2=0x7fd8b4159ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8b415df50 0x7fd8b4159ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 -- 172.21.15.165:0/3953039439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b4147570 con 0x7fd8b40cfc20 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8b3400640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b4147030 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400d800 tx=0x7fd8a400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400b6c0 con 0x7fd8b40cfc20 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a4019070 con 0x7fd8b40cfc20 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a400ba80 con 0x7fd8b40cfc20 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.837+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8b415f6e0 con 0x7fd8b40cfc20 2024-04-25T10:20:23.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.838+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b415ff80 con 0x7fd8b40cfc20 2024-04-25T10:20:23.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.839+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b4073990 con 0x7fd8b40cfc20 2024-04-25T10:20:23.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.840+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd8a400bc20 con 0x7fd8b40cfc20 2024-04-25T10:20:23.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.841+0000 7fd8ab400640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd89406b4d0 0x7fd89406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:23.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.841+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fd8a408a8e0 con 0x7fd8b40cfc20 2024-04-25T10:20:23.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.841+0000 7fd8b2a00640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd89406b4d0 0x7fd89406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:23.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.842+0000 7fd8b2a00640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd89406b4d0 0x7fd89406d990 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd8b41f6fb0 tx=0x7fd8a00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:23.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:23.845+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8a4059530 con 0x7fd8b40cfc20 2024-04-25T10:20:24.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.079+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd8b406b4c0 con 0x7fd8b40cfc20 2024-04-25T10:20:24.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.080+0000 7fd8ab400640 1 -- 172.21.15.165:0/3953039439 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+4964 (secure 0 0 0) 0x7fd8a4058e30 con 0x7fd8b40cfc20 2024-04-25T10:20:24.083 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:24.083 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":130,"default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124}],"filesystems":[{"mdsmap":{"epoch":130,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:01.817104+0000","modified":"2024-04-25T10:20:04.840723+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11211},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11211":{"gid":11211,"name":"c","rank":0,"incarnation":126,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/1159947775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1159947775},{"type":"v1","addr":"172.21.15.165:6837","nonce":1159947775}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[28],"metadata_pool":27,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2024-04-25T10:20:24.083 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 130 2024-04-25T10:20:24.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd89406b4d0 msgr2=0x7fd89406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd89406b4d0 0x7fd89406d990 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fd8b41f6fb0 tx=0x7fd8a00079b0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 msgr2=0x7fd8b4147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b4147030 secure :-1 s=READY pgs=2667 cs=0 l=1 rev1=1 crypto rx=0x7fd8a400d800 tx=0x7fd8a400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 shutdown_connections 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd89406b4d0 0x7fd89406d990 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8b415df50 0x7fd8b4159ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd8b415db70 0x7fd8b41f3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 --2- 172.21.15.165:0/3953039439 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8b40cfc20 0x7fd8b4147030 unknown :-1 s=CLOSED pgs=2667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 >> 172.21.15.165:0/3953039439 conn(0x7fd8b406abe0 msgr2=0x7fd8b415b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 shutdown_connections 2024-04-25T10:20:24.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.083+0000 7fd8ba800640 1 -- 172.21.15.165:0/3953039439 wait complete. 2024-04-25T10:20:24.105 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:24.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.305+0000 7f03e0a00640 1 Processor -- start 2024-04-25T10:20:24.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.305+0000 7f03e0a00640 1 -- start start 2024-04-25T10:20:24.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03e0a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f03dc0b93c0 0x7f03dc0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03e0a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc161df0 con 0x7f03dc0d04f0 2024-04-25T10:20:24.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03e0a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03dc1620d0 con 0x7f03dc15db90 2024-04-25T10:20:24.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03e0a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03dc0d1f50 con 0x7f03dc0b93c0 2024-04-25T10:20:24.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03d1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f03dc0b93c0 0x7f03dc0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03db200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f03dc0d04f0 0x7f03dc0d23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52412/0 (socket says 172.21.15.165:52412) 2024-04-25T10:20:24.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.307+0000 7f03db200640 1 -- 172.21.15.165:0/1144675885 learned_addr learned my addr 172.21.15.165:0/1144675885 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:24.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.308+0000 7f03d1e00640 1 -- 172.21.15.165:0/1144675885 >> v1:172.21.15.172:6789/0 conn(0x7f03dc15db90 legacy=0x7f03dc1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:24.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.308+0000 7f03d1e00640 1 -- 172.21.15.165:0/1144675885 >> v1:172.21.15.165:6789/0 conn(0x7f03dc0d04f0 legacy=0x7f03dc0d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.308+0000 7f03d1e00640 1 -- 172.21.15.165:0/1144675885 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc1623b0 con 0x7f03dc0b93c0 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.309+0000 7f03d1e00640 1 --2- 172.21.15.165:0/1144675885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc0b93c0 0x7f03dc0cffb0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f03c40097a0 tx=0x7f03c402a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdab5b03231e3b52 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.309+0000 7f03d9e00640 1 -- 172.21.15.165:0/1144675885 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4037070 con 0x7f03dc0b93c0 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.309+0000 7f03d9e00640 1 -- 172.21.15.165:0/1144675885 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c402ad60 con 0x7f03dc0b93c0 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.309+0000 7f03d9e00640 1 -- 172.21.15.165:0/1144675885 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c4032660 con 0x7f03dc0b93c0 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.309+0000 7f03e0a00640 1 -- 172.21.15.165:0/1144675885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc0b93c0 msgr2=0x7f03dc0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.309+0000 7f03e0a00640 1 --2- 172.21.15.165:0/1144675885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc0b93c0 0x7f03dc0cffb0 secure :-1 s=READY pgs=2669 cs=0 l=1 rev1=1 crypto rx=0x7f03c40097a0 tx=0x7f03c402a0d0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.310+0000 7f03e0a00640 1 -- 172.21.15.165:0/1144675885 shutdown_connections 2024-04-25T10:20:24.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.310+0000 7f03e0a00640 1 --2- 172.21.15.165:0/1144675885 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc0b93c0 0x7f03dc0cffb0 unknown :-1 s=CLOSED pgs=2669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.310+0000 7f03e0a00640 1 -- 172.21.15.165:0/1144675885 >> 172.21.15.165:0/1144675885 conn(0x7f03dc0b52b0 msgr2=0x7f03dc0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.310+0000 7f03e0a00640 1 -- 172.21.15.165:0/1144675885 shutdown_connections 2024-04-25T10:20:24.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.310+0000 7f03e0a00640 1 -- 172.21.15.165:0/1144675885 wait complete. 2024-04-25T10:20:24.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.311+0000 7f03e0a00640 1 Processor -- start 2024-04-25T10:20:24.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.311+0000 7f03e0a00640 1 -- start start 2024-04-25T10:20:24.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.312+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03dc0b93c0 0x7f03dc15ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.312+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc0d04f0 0x7f03dc15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.312+0000 7f03e0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 0x7f03dc14ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.312+0000 7f03e0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0b68d0 con 0x7f03dc15db90 2024-04-25T10:20:24.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.312+0000 7f03e0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0d0940 con 0x7f03dc0d04f0 2024-04-25T10:20:24.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.312+0000 7f03e0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03dc0b6a50 con 0x7f03dc0b93c0 2024-04-25T10:20:24.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 0x7f03dc14ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 0x7f03dc14ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59060/0 (socket says 172.21.15.165:59060) 2024-04-25T10:20:24.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 -- 172.21.15.165:0/774549834 learned_addr learned my addr 172.21.15.165:0/774549834 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03d1e00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc0d04f0 0x7f03dc15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 -- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03dc0b93c0 msgr2=0x7f03dc15ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03da800640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03dc0b93c0 0x7f03dc15ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03dc0b93c0 0x7f03dc15ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 -- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc0d04f0 msgr2=0x7f03dc15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc0d04f0 0x7f03dc15df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 -- 172.21.15.165:0/774549834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc15e6e0 con 0x7f03dc15db90 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.313+0000 7f03db200640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 0x7f03dc14ec80 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f03d400c890 tx=0x7f03d400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.314+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d4018070 con 0x7f03dc15db90 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.314+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03d400a640 con 0x7f03dc15db90 2024-04-25T10:20:24.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.314+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03d400aa70 con 0x7f03dc15db90 2024-04-25T10:20:24.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.314+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03dc15e9c0 con 0x7f03dc15db90 2024-04-25T10:20:24.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.314+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03dc15ef80 con 0x7f03dc15db90 2024-04-25T10:20:24.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.315+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03dc073990 con 0x7f03dc15db90 2024-04-25T10:20:24.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.316+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f03d401b070 con 0x7f03dc15db90 2024-04-25T10:20:24.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.317+0000 7f03d3400640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f03b806b500 0x7f03b806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.317+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f03d4089b20 con 0x7f03dc15db90 2024-04-25T10:20:24.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.317+0000 7f03da800640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f03b806b500 0x7f03b806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.318+0000 7f03da800640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f03b806b500 0x7f03b806d9c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00a750 tx=0x7f03cc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.321+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f03d4058770 con 0x7f03dc15db90 2024-04-25T10:20:24.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.519+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03dc15d340 con 0x7f03dc15db90 2024-04-25T10:20:24.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.521+0000 7f03d3400640 1 -- 172.21.15.165:0/774549834 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16988 (secure 0 0 0) 0x7f03d4058070 con 0x7f03dc15db90 2024-04-25T10:20:24.524 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:24.524 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":184,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:03.322848+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:19:59.548746+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"183","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:00.783201+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.523+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f03b806b500 msgr2=0x7f03b806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.523+0000 7f03e0a00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f03b806b500 0x7f03b806d9c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f03cc00a750 tx=0x7f03cc008040 comp rx=0 tx=0).stop 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.523+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 msgr2=0x7f03dc14ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.523+0000 7f03e0a00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 0x7f03dc14ec80 secure :-1 s=READY pgs=2670 cs=0 l=1 rev1=1 crypto rx=0x7f03d400c890 tx=0x7f03d400cd50 comp rx=0 tx=0).stop 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 shutdown_connections 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f03b806b500 0x7f03b806d9c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f03dc15db90 0x7f03dc14ec80 unknown :-1 s=CLOSED pgs=2670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f03dc0d04f0 0x7f03dc15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 --2- 172.21.15.165:0/774549834 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f03dc0b93c0 0x7f03dc15ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 >> 172.21.15.165:0/774549834 conn(0x7f03dc0b52b0 msgr2=0x7f03dc0b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 shutdown_connections 2024-04-25T10:20:24.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.524+0000 7f03e0a00640 1 -- 172.21.15.165:0/774549834 wait complete. 2024-04-25T10:20:24.546 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:20:24.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.742+0000 7fe539c00640 1 Processor -- start 2024-04-25T10:20:24.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.742+0000 7fe539c00640 1 -- start start 2024-04-25T10:20:24.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.743+0000 7fe539c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe5340cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.743+0000 7fe539c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe53415b270 0x7fe53415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.743+0000 7fe539c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe534072280 con 0x7fe5340cc4f0 2024-04-25T10:20:24.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe539c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe534071270 con 0x7fe53415b270 2024-04-25T10:20:24.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe539c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5340714c0 con 0x7fe5340ce230 2024-04-25T10:20:24.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe533400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe53415b270 0x7fe53415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe52a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe5340cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe533400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe53415b270 0x7fe53415b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:44250/0 (socket says 172.21.15.165:44250) 2024-04-25T10:20:24.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe532a00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7fe5340cc4f0 0x7fe5340cdb20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:60318/0 (socket says 172.21.15.165:60318) 2024-04-25T10:20:24.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.744+0000 7fe533400640 1 -- 172.21.15.165:0/2416475668 learned_addr learned my addr 172.21.15.165:0/2416475668 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:24.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.745+0000 7fe533400640 1 -- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 msgr2=0x7fe5340cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:24.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.745+0000 7fe533400640 1 --2- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe5340cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.745+0000 7fe533400640 1 -- 172.21.15.165:0/2416475668 >> v1:172.21.15.172:6789/0 conn(0x7fe5340cc4f0 legacy=0x7fe5340cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.745+0000 7fe533400640 1 -- 172.21.15.165:0/2416475668 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340717a0 con 0x7fe53415b270 2024-04-25T10:20:24.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.745+0000 7fe52a000640 1 --2- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe5340cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:24.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.747+0000 7fe533400640 1 --2- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe53415b6c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d150 tx=0x7fe52c02c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d186215233e1caf server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.747+0000 7fe532000640 1 -- 172.21.15.165:0/2416475668 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c009600 con 0x7fe53415b270 2024-04-25T10:20:24.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.747+0000 7fe532000640 1 -- 172.21.15.165:0/2416475668 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe52c037070 con 0x7fe53415b270 2024-04-25T10:20:24.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.747+0000 7fe532000640 1 -- 172.21.15.165:0/2416475668 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe52c0099c0 con 0x7fe53415b270 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 -- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 msgr2=0x7fe53415b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 --2- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe53415b6c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fe52c00d150 tx=0x7fe52c02c920 comp rx=0 tx=0).stop 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 -- 172.21.15.165:0/2416475668 shutdown_connections 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 --2- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe5340cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 --2- 172.21.15.165:0/2416475668 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe53415b6c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 -- 172.21.15.165:0/2416475668 >> 172.21.15.165:0/2416475668 conn(0x7fe53406abe0 msgr2=0x7fe53406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 -- 172.21.15.165:0/2416475668 shutdown_connections 2024-04-25T10:20:24.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.748+0000 7fe539c00640 1 -- 172.21.15.165:0/2416475668 wait complete. 2024-04-25T10:20:24.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.751+0000 7fe539c00640 1 Processor -- start 2024-04-25T10:20:24.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.751+0000 7fe539c00640 1 -- start start 2024-04-25T10:20:24.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.752+0000 7fe539c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 0x7fe53413fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.752+0000 7fe539c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe534141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.752+0000 7fe539c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe5341599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.752+0000 7fe539c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415bc00 con 0x7fe5340cc4f0 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.752+0000 7fe539c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe534071d60 con 0x7fe53415b270 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.752+0000 7fe539c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53415bd80 con 0x7fe5340ce230 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 0x7fe53413fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 0x7fe53413fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59062/0 (socket says 172.21.15.165:59062) 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 -- 172.21.15.165:0/2971987674 learned_addr learned my addr 172.21.15.165:0/2971987674 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe52a000640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe534141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe533400640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe5341599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 -- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 msgr2=0x7fe534141ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe534141ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 -- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 msgr2=0x7fe5341599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:24.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe5341599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:24.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 -- 172.21.15.165:0/2971987674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5341404a0 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.753+0000 7fe532a00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 0x7fe53413fda0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fe52400c3f0 tx=0x7fe52400c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.754+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5240074e0 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.754+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe524007680 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.754+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe524004a00 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.754+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe534140780 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.754+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe534140d40 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.755+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe534073a00 con 0x7fe5340cc4f0 2024-04-25T10:20:24.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.756+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe52401a020 con 0x7fe5340cc4f0 2024-04-25T10:20:24.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.757+0000 7fe52b400640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe51006b500 0x7fe51006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:24.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.757+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6101+0+0 (secure 0 0 0) 0x7fe524089b60 con 0x7fe5340cc4f0 2024-04-25T10:20:24.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.757+0000 7fe52a000640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe51006b500 0x7fe51006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:24.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.758+0000 7fe52a000640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe51006b500 0x7fe51006d9c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009560 tx=0x7fe51c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:24.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:24.761+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe5240587b0 con 0x7fe5340cc4f0 2024-04-25T10:20:25.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.001+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe5340b3f30 con 0x7fe5340cc4f0 2024-04-25T10:20:25.004 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:20:25.002+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:20:25.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.006+0000 7fe52b400640 1 -- 172.21.15.165:0/2971987674 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v131) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe5240580b0 con 0x7fe5340cc4f0 2024-04-25T10:20:25.008 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:20:25.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.009+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe51006b500 msgr2=0x7fe51006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.009+0000 7fe539c00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe51006b500 0x7fe51006d9c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009560 tx=0x7fe51c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:25.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.009+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 msgr2=0x7fe53413fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.009+0000 7fe539c00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 0x7fe53413fda0 secure :-1 s=READY pgs=2671 cs=0 l=1 rev1=1 crypto rx=0x7fe52400c3f0 tx=0x7fe52400c8b0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 shutdown_connections 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe51006b500 0x7fe51006d9c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe53415b270 0x7fe5341599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe5340ce230 0x7fe534141ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 --2- 172.21.15.165:0/2971987674 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe5340cc4f0 0x7fe53413fda0 unknown :-1 s=CLOSED pgs=2671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 >> 172.21.15.165:0/2971987674 conn(0x7fe53406abe0 msgr2=0x7fe5340cdb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 shutdown_connections 2024-04-25T10:20:25.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.010+0000 7fe539c00640 1 -- 172.21.15.165:0/2971987674 wait complete. 2024-04-25T10:20:25.035 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:20:25.092 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:20:25.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.213+0000 7f8bd7200640 1 Processor -- start 2024-04-25T10:20:25.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.213+0000 7f8bd7200640 1 -- start start 2024-04-25T10:20:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.214+0000 7f8bd7200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 0x7f8bd00c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.214+0000 7f8bd7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.214+0000 7f8bd7200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bd0145520 con 0x7f8bd0145140 2024-04-25T10:20:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.214+0000 7f8bd7200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bd0071270 con 0x7f8bd00ceb20 2024-04-25T10:20:25.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.214+0000 7f8bd7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd0071550 con 0x7f8bd00c5030 2024-04-25T10:20:25.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.215+0000 7f8bd5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 0x7f8bd00c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.215+0000 7f8bd6800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.215+0000 7f8bd6800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd015bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:57120/0 (socket says 172.21.15.165:57120) 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.215+0000 7f8bd5400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 0x7f8bd00c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:44270/0 (socket says 172.21.15.165:44270) 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.215+0000 7f8bd6800640 1 -- 172.21.15.165:0/1091200432 learned_addr learned my addr 172.21.15.165:0/1091200432 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.216+0000 7f8bd6800640 1 -- 172.21.15.165:0/1091200432 >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 msgr2=0x7f8bd00c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.216+0000 7f8bd6800640 1 --2- 172.21.15.165:0/1091200432 >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 0x7f8bd00c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.216+0000 7f8bd6800640 1 -- 172.21.15.165:0/1091200432 >> v1:172.21.15.172:6789/0 conn(0x7f8bd0145140 legacy=0x7f8bd00ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.216+0000 7f8bd6800640 1 -- 172.21.15.165:0/1091200432 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd0071830 con 0x7f8bd00c5030 2024-04-25T10:20:25.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.216+0000 7f8bd5400640 1 --2- 172.21.15.165:0/1091200432 >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 0x7f8bd00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:25.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.217+0000 7f8bd6800640 1 --2- 172.21.15.165:0/1091200432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd015bfb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009e50 tx=0x7f8bcc02a680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be67d4a52bd6f8d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd4a00640 1 -- 172.21.15.165:0/1091200432 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc036070 con 0x7f8bd00c5030 2024-04-25T10:20:25.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd4a00640 1 -- 172.21.15.165:0/1091200432 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc031600 con 0x7f8bd00c5030 2024-04-25T10:20:25.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd4a00640 1 -- 172.21.15.165:0/1091200432 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0319d0 con 0x7f8bd00c5030 2024-04-25T10:20:25.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd7200640 1 -- 172.21.15.165:0/1091200432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 msgr2=0x7f8bd015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1091200432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd015bfb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009e50 tx=0x7f8bcc02a680 comp rx=0 tx=0).stop 2024-04-25T10:20:25.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd7200640 1 -- 172.21.15.165:0/1091200432 shutdown_connections 2024-04-25T10:20:25.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1091200432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd015bfb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1091200432 >> v2:172.21.15.172:3300/0 conn(0x7f8bd00ceb20 0x7f8bd00c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.218+0000 7f8bd7200640 1 -- 172.21.15.165:0/1091200432 >> 172.21.15.165:0/1091200432 conn(0x7f8bd0061cc0 msgr2=0x7f8bd006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:25.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.219+0000 7f8bd7200640 1 -- 172.21.15.165:0/1091200432 shutdown_connections 2024-04-25T10:20:25.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.219+0000 7f8bd7200640 1 -- 172.21.15.165:0/1091200432 wait complete. 2024-04-25T10:20:25.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.220+0000 7f8bd7200640 1 Processor -- start 2024-04-25T10:20:25.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.221+0000 7f8bd7200640 1 -- start start 2024-04-25T10:20:25.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd7200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd0143480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd7200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd00ceb20 0x7f8bd00cb270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd7200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 0x7f8bd0140710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd7200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00c5620 con 0x7f8bd0145140 2024-04-25T10:20:25.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd7200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd00c54a0 con 0x7f8bd00ceb20 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd7200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd015c4f0 con 0x7f8bd00c5030 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 0x7f8bd0140710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 0x7f8bd0140710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59086/0 (socket says 172.21.15.165:59086) 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd6800640 1 -- 172.21.15.165:0/1051212147 learned_addr learned my addr 172.21.15.165:0/1051212147 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd5400640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd00ceb20 0x7f8bd00cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.222+0000 7f8bd5e00640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd0143480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd6800640 1 -- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 msgr2=0x7f8bd0143480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd6800640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd0143480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd6800640 1 -- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd00ceb20 msgr2=0x7f8bd00cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd6800640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd00ceb20 0x7f8bd00cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd6800640 1 -- 172.21.15.165:0/1051212147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd01f5060 con 0x7f8bd0145140 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd5400640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd00ceb20 0x7f8bd00cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:25.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd5e00640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd0143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:25.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd6800640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 0x7f8bd0140710 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc034040 tx=0x7f8bcc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc036050 con 0x7f8bd0145140 2024-04-25T10:20:25.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bcc042070 con 0x7f8bd0145140 2024-04-25T10:20:25.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc009510 con 0x7f8bd0145140 2024-04-25T10:20:25.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd01f5340 con 0x7f8bd0145140 2024-04-25T10:20:25.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.223+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd01f5900 con 0x7f8bd0145140 2024-04-25T10:20:25.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.225+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bd0073990 con 0x7f8bd0145140 2024-04-25T10:20:25.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.225+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8bcc007bf0 con 0x7f8bd0145140 2024-04-25T10:20:25.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.226+0000 7f8bca000640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bb406b4d0 0x7f8bb406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:25.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.226+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f8bcc0ab330 con 0x7f8bd0145140 2024-04-25T10:20:25.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.227+0000 7f8bd5e00640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bb406b4d0 0x7f8bb406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:25.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.227+0000 7f8bd5e00640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bb406b4d0 0x7f8bb406d990 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40015e0 tx=0x7f8bc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:25.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.231+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8bcc045360 con 0x7f8bd0145140 2024-04-25T10:20:25.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:25.476+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8bd00cf220 con 0x7f8bd0145140 2024-04-25T10:20:26.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.009+0000 7f8bca000640 1 -- 172.21.15.165:0/1051212147 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v132) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8bcc079f30 con 0x7f8bd0145140 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bb406b4d0 msgr2=0x7f8bb406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bb406b4d0 0x7f8bb406d990 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f8bc40015e0 tx=0x7f8bc4009040 comp rx=0 tx=0).stop 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 msgr2=0x7f8bd0140710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 0x7f8bd0140710 secure :-1 s=READY pgs=2674 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc034040 tx=0x7f8bcc004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 shutdown_connections 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bb406b4d0 0x7f8bb406d990 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8bd0145140 0x7f8bd0140710 unknown :-1 s=CLOSED pgs=2674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8bd00ceb20 0x7f8bd00cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 --2- 172.21.15.165:0/1051212147 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8bd00c5030 0x7f8bd0143480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 >> 172.21.15.165:0/1051212147 conn(0x7f8bd0061cc0 msgr2=0x7f8bd015db20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 shutdown_connections 2024-04-25T10:20:26.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.013+0000 7f8bd7200640 1 -- 172.21.15.165:0/1051212147 wait complete. 2024-04-25T10:20:26.037 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:20:26.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.241+0000 7f8210400640 1 Processor -- start 2024-04-25T10:20:26.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.241+0000 7f8210400640 1 -- start start 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f8210400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f82081446d0 0x7f82080b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f8210400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 0x7f82080b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f8210400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f82080cccc0 con 0x7f82081452a0 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f8210400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82080b2b20 con 0x7f82081446d0 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f8210400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820806d400 con 0x7f8208144d50 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f820e400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f82081452a0 0x7f82080cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52424/0 (socket says 172.21.15.165:52424) 2024-04-25T10:20:26.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.242+0000 7f820e400640 1 -- 172.21.15.165:0/1148004877 learned_addr learned my addr 172.21.15.165:0/1148004877 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:26.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f820da00640 1 --2- 172.21.15.165:0/1148004877 >> v2:172.21.15.172:3300/0 conn(0x7f82081446d0 0x7f82080b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f820d000640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 0x7f82080b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1185733085 0 0) 0x7f82080cccc0 con 0x7f82081452a0 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f0003520 con 0x7f82081452a0 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 748471230 0 0) 0x7f81f0003520 con 0x7f82081452a0 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 msgr2=0x7f82080b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 0x7f82080b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 >> v2:172.21.15.172:3300/0 conn(0x7f82081446d0 msgr2=0x7f82080b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 --2- 172.21.15.165:0/1148004877 >> v2:172.21.15.172:3300/0 conn(0x7f82081446d0 0x7f82080b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080cc140 con 0x7f82081452a0 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f82000019f0 con 0x7f82081452a0 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.243+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 >> v1:172.21.15.165:6789/0 conn(0x7f82081452a0 legacy=0x7f82080cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f8207e00640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f00039b0 0x7f81f0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f8207e00640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f0004800 0x7f81f0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f8207e00640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 0x7f81f00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080b2b20 con 0x7f81f0004800 2024-04-25T10:20:26.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820806d400 con 0x7f81f00039b0 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f000b9c0 con 0x7f8208144d50 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820d000640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f0004800 0x7f81f0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820da00640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f00039b0 0x7f81f0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820e400640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 0x7f81f00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820d000640 1 -- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 msgr2=0x7f81f00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820d000640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8208144d50 0x7f81f00051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820d000640 1 -- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f00039b0 msgr2=0x7f81f0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820d000640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f81f00039b0 0x7f81f0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.244+0000 7f820d000640 1 -- 172.21.15.165:0/1148004877 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f0003520 con 0x7f81f0004800 2024-04-25T10:20:26.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f820d000640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f0004800 0x7f81f0004c80 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f81f800ba30 tx=0x7f81f0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3960ed0f336309 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8030070 con 0x7f81f0004800 2024-04-25T10:20:26.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f802ade0 con 0x7f81f0004800 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f8207e00640 1 -- 172.21.15.165:0/1148004877 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f800b600 con 0x7f81f0004800 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f8210400640 1 -- 172.21.15.165:0/1148004877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f0004800 msgr2=0x7f81f0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f8210400640 1 --2- 172.21.15.165:0/1148004877 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f81f0004800 0x7f81f0004c80 secure :-1 s=READY pgs=2676 cs=0 l=1 rev1=1 crypto rx=0x7f81f800ba30 tx=0x7f81f0005830 comp rx=0 tx=0).stop 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.246+0000 7f820e400640 1 -- 172.21.15.165:0/1148004877 reap_dead start 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.247+0000 7f8210400640 1 -- 172.21.15.165:0/1148004877 shutdown_connections 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.247+0000 7f8210400640 1 -- 172.21.15.165:0/1148004877 >> 172.21.15.165:0/1148004877 conn(0x7f820806abe0 msgr2=0x7f820806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.247+0000 7f8210400640 1 -- 172.21.15.165:0/1148004877 shutdown_connections 2024-04-25T10:20:26.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.247+0000 7f8210400640 1 -- 172.21.15.165:0/1148004877 wait complete. 2024-04-25T10:20:26.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.248+0000 7f8210400640 1 Processor -- start 2024-04-25T10:20:26.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.249+0000 7f8210400640 1 -- start start 2024-04-25T10:20:26.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.249+0000 7f8210400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f0004800 0x7f82081f3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.249+0000 7f8210400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 0x7f820813bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.249+0000 7f8210400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8208141b10 0x7f8208141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f8210400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080cc9e0 con 0x7f82081390b0 2024-04-25T10:20:26.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f8210400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82080cc700 con 0x7f8208141b10 2024-04-25T10:20:26.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f8210400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8208144ab0 con 0x7f81f0004800 2024-04-25T10:20:26.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 0x7f820813bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 0x7f820813bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:59090/0 (socket says 172.21.15.165:59090) 2024-04-25T10:20:26.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 -- 172.21.15.165:0/834469154 learned_addr learned my addr 172.21.15.165:0/834469154 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:26.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 -- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f0004800 msgr2=0x7f82081f3870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820da00640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f0004800 0x7f82081f3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820e400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8208141b10 0x7f8208141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f0004800 0x7f82081f3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 -- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8208141b10 msgr2=0x7f8208141f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:26.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8208141b10 0x7f8208141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.250+0000 7f820d000640 1 -- 172.21.15.165:0/834469154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82080af600 con 0x7f82081390b0 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.251+0000 7f820d000640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 0x7f820813bdb0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f81f802a8d0 tx=0x7f81f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.251+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f8043020 con 0x7f82081390b0 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.251+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f803f680 con 0x7f82081390b0 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.251+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f800b600 con 0x7f82081390b0 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.251+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82080af8a0 con 0x7f82081390b0 2024-04-25T10:20:26.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.251+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8208139620 con 0x7f82081390b0 2024-04-25T10:20:26.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.252+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8208073a00 con 0x7f82081390b0 2024-04-25T10:20:26.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.253+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f81f8046020 con 0x7f82081390b0 2024-04-25T10:20:26.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.254+0000 7f8205600640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81ec06b4d0 0x7f81ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:26.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.254+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 6187+0+0 (secure 0 0 0) 0x7f81f80aab80 con 0x7f82081390b0 2024-04-25T10:20:26.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.254+0000 7f820da00640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81ec06b4d0 0x7f81ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:26.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.255+0000 7f820da00640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81ec06b4d0 0x7f81ec06d990 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0099d0 tx=0x7f81fc008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:26.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.258+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f81f804d070 con 0x7f82081390b0 2024-04-25T10:20:26.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:26.461+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f82080afc40 con 0x7f82081390b0 2024-04-25T10:20:27.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.019+0000 7f8205600640 1 -- 172.21.15.165:0/834469154 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v186) v1 ==== 181+0+0 (secure 0 0 0) 0x7f81f8079810 con 0x7f82081390b0 2024-04-25T10:20:27.023 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81ec06b4d0 msgr2=0x7f81ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81ec06b4d0 0x7f81ec06d990 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0099d0 tx=0x7f81fc008040 comp rx=0 tx=0).stop 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 msgr2=0x7f820813bdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 0x7f820813bdb0 secure :-1 s=READY pgs=2677 cs=0 l=1 rev1=1 crypto rx=0x7f81f802a8d0 tx=0x7f81f80046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 shutdown_connections 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f81ec06b4d0 0x7f81ec06d990 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8208141b10 0x7f8208141f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82081390b0 0x7f820813bdb0 unknown :-1 s=CLOSED pgs=2677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 --2- 172.21.15.165:0/834469154 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f81f0004800 0x7f82081f3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.023+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 >> 172.21.15.165:0/834469154 conn(0x7f820806abe0 msgr2=0x7f82080ac940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.024+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 shutdown_connections 2024-04-25T10:20:27.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.024+0000 7f8210400640 1 -- 172.21.15.165:0/834469154 wait complete. 2024-04-25T10:20:27.050 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:20:27.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.254+0000 7f603be00640 1 Processor -- start 2024-04-25T10:20:27.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.255+0000 7f603be00640 1 -- start start 2024-04-25T10:20:27.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.256+0000 7f603be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f603c145140 0x7f603c0ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.256+0000 7f603be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c0c5650 con 0x7f603c0ce950 2024-04-25T10:20:27.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.256+0000 7f603be00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c145520 con 0x7f603c0c5200 2024-04-25T10:20:27.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.256+0000 7f603be00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c0ced90 con 0x7f603c145140 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f603a000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f603c0ce950 0x7f603c0c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51880/0 (socket says 172.21.15.165:51880) 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f603a000640 1 -- 172.21.15.165:0/1144510128 learned_addr learned my addr 172.21.15.165:0/1144510128 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f603aa00640 1 --2- 172.21.15.165:0/1144510128 >> v2:172.21.15.165:3300/0 conn(0x7f603c145140 0x7f603c0ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f6039600640 1 -- 172.21.15.165:0/1144510128 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 438173287 0 0) 0x7f603c0c5650 con 0x7f603c0ce950 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f6039600640 1 -- 172.21.15.165:0/1144510128 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6020003520 con 0x7f603c0ce950 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f603aa00640 1 -- 172.21.15.165:0/1144510128 >> v1:172.21.15.172:6789/0 conn(0x7f603c0c5200 legacy=0x7f603c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f603aa00640 1 -- 172.21.15.165:0/1144510128 >> v1:172.21.15.165:6789/0 conn(0x7f603c0ce950 legacy=0x7f603c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.257+0000 7f603aa00640 1 -- 172.21.15.165:0/1144510128 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c071270 con 0x7f603c145140 2024-04-25T10:20:27.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.259+0000 7f603aa00640 1 --2- 172.21.15.165:0/1144510128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0ce410 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f602c009d90 tx=0x7f602c02a610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2906261e017089f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.259+0000 7f6039600640 1 -- 172.21.15.165:0/1144510128 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c031600 con 0x7f603c145140 2024-04-25T10:20:27.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.259+0000 7f6039600640 1 -- 172.21.15.165:0/1144510128 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c0317a0 con 0x7f603c145140 2024-04-25T10:20:27.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.259+0000 7f6039600640 1 -- 172.21.15.165:0/1144510128 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c031b70 con 0x7f603c145140 2024-04-25T10:20:27.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 -- 172.21.15.165:0/1144510128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 msgr2=0x7f603c0ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 --2- 172.21.15.165:0/1144510128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0ce410 secure :-1 s=READY pgs=2679 cs=0 l=1 rev1=1 crypto rx=0x7f602c009d90 tx=0x7f602c02a610 comp rx=0 tx=0).stop 2024-04-25T10:20:27.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 -- 172.21.15.165:0/1144510128 shutdown_connections 2024-04-25T10:20:27.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 --2- 172.21.15.165:0/1144510128 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0ce410 unknown :-1 s=CLOSED pgs=2679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 -- 172.21.15.165:0/1144510128 >> 172.21.15.165:0/1144510128 conn(0x7f603c061cc0 msgr2=0x7f603c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:27.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 -- 172.21.15.165:0/1144510128 shutdown_connections 2024-04-25T10:20:27.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.260+0000 7f603be00640 1 -- 172.21.15.165:0/1144510128 wait complete. 2024-04-25T10:20:27.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.261+0000 7f603be00640 1 Processor -- start 2024-04-25T10:20:27.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.262+0000 7f603be00640 1 -- start start 2024-04-25T10:20:27.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f603c0c5200 0x7f603c1332b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f603c0ce950 0x7f603c1e4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c071b10 con 0x7f603c145140 2024-04-25T10:20:27.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c071830 con 0x7f603c0c5200 2024-04-25T10:20:27.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c15c6c0 con 0x7f603c0ce950 2024-04-25T10:20:27.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0cae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39910/0 (socket says 172.21.15.165:39910) 2024-04-25T10:20:27.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603b400640 1 -- 172.21.15.165:0/2625236106 learned_addr learned my addr 172.21.15.165:0/2625236106 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:27.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603aa00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f603c0c5200 0x7f603c1332b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603a000640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f603c0ce950 0x7f603c1e4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603b400640 1 -- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f603c0ce950 msgr2=0x7f603c1e4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.263+0000 7f603b400640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f603c0ce950 0x7f603c1e4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603b400640 1 -- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f603c0c5200 msgr2=0x7f603c1332b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603b400640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f603c0c5200 0x7f603c1332b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603b400640 1 -- 172.21.15.165:0/2625236106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c13fce0 con 0x7f603c145140 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603a000640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f603c0ce950 0x7f603c1e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603b400640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0cae90 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f603400c440 tx=0x7f603400c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034011790 con 0x7f603c145140 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6034017070 con 0x7f603c145140 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6034011b70 con 0x7f603c145140 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f603c13ff60 con 0x7f603c145140 2024-04-25T10:20:27.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.264+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c140520 con 0x7f603c145140 2024-04-25T10:20:27.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.265+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c073990 con 0x7f603c145140 2024-04-25T10:20:27.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.267+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6034011d10 con 0x7f603c145140 2024-04-25T10:20:27.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.268+0000 7f6032a00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f601806b4d0 0x7f601806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:27.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.268+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5749+0+0 (secure 0 0 0) 0x7f603408a0d0 con 0x7f603c145140 2024-04-25T10:20:27.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.268+0000 7f603aa00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f601806b4d0 0x7f601806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:27.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.269+0000 7f603aa00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f601806b4d0 0x7f601806d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f602c034000 tx=0x7f602c033040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:27.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.272+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f603c13fce0 con 0x7f603c145140 2024-04-25T10:20:27.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:27.468+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f603c0c4b90 con 0x7f603c145140 2024-04-25T10:20:28.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.023+0000 7f6032a00640 1 -- 172.21.15.165:0/2625236106 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v187) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6034058e80 con 0x7f603c145140 2024-04-25T10:20:28.026 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:20:28.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.028+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f601806b4d0 msgr2=0x7f601806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.028+0000 7f603be00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f601806b4d0 0x7f601806d990 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f602c034000 tx=0x7f602c033040 comp rx=0 tx=0).stop 2024-04-25T10:20:28.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.028+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 msgr2=0x7f603c0cae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.028+0000 7f603be00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0cae90 secure :-1 s=READY pgs=2680 cs=0 l=1 rev1=1 crypto rx=0x7f603400c440 tx=0x7f603400c900 comp rx=0 tx=0).stop 2024-04-25T10:20:28.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.029+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 shutdown_connections 2024-04-25T10:20:28.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.029+0000 7f603be00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f601806b4d0 0x7f601806d990 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.029+0000 7f603be00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f603c145140 0x7f603c0cae90 unknown :-1 s=CLOSED pgs=2680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.029+0000 7f603be00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f603c0ce950 0x7f603c1e4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.029+0000 7f603be00640 1 --2- 172.21.15.165:0/2625236106 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f603c0c5200 0x7f603c1332b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.029+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 >> 172.21.15.165:0/2625236106 conn(0x7f603c061cc0 msgr2=0x7f603c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:28.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.031+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 shutdown_connections 2024-04-25T10:20:28.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.032+0000 7f603be00640 1 -- 172.21.15.165:0/2625236106 wait complete. 2024-04-25T10:20:28.061 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:20:28.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.248+0000 7f49ecc00640 1 Processor -- start 2024-04-25T10:20:28.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.248+0000 7f49ecc00640 1 -- start start 2024-04-25T10:20:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49ecc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f49e806bff0 0x7f49e80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49ecc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 0x7f49e80d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49ecc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49e815e110 con 0x7f49e815dd30 2024-04-25T10:20:28.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49ecc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49e806d000 con 0x7f49e806bff0 2024-04-25T10:20:28.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49ecc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80b2fd0 con 0x7f49e80b5970 2024-04-25T10:20:28.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49e7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f49e815dd30 0x7f49e80cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51882/0 (socket says 172.21.15.165:51882) 2024-04-25T10:20:28.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.250+0000 7f49e7400640 1 -- 172.21.15.165:0/1463696843 learned_addr learned my addr 172.21.15.165:0/1463696843 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6a00640 1 --2- 172.21.15.165:0/1463696843 >> v2:172.21.15.172:3300/0 conn(0x7f49e806bff0 0x7f49e80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e7e00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 0x7f49e80d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547150517 0 0) 0x7f49e815e110 con 0x7f49e815dd30 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49cc003520 con 0x7f49e815dd30 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 156246614 0 0) 0x7f49cc003520 con 0x7f49e815dd30 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 msgr2=0x7f49e80d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 0x7f49e80d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 >> v2:172.21.15.172:3300/0 conn(0x7f49e806bff0 msgr2=0x7f49e80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 --2- 172.21.15.165:0/1463696843 >> v2:172.21.15.172:3300/0 conn(0x7f49e806bff0 0x7f49e80b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e806c480 con 0x7f49e815dd30 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f49d8001670 con 0x7f49e815dd30 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 >> v1:172.21.15.165:6789/0 conn(0x7f49e815dd30 legacy=0x7f49e80cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 0x7f49cc0039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc003f30 0x7f49cc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.251+0000 7f49e6000640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e806d000 con 0x7f49cc004d40 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80b2fd0 con 0x7f49cc003f30 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc00b9c0 con 0x7f49e80b5970 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7e00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7400640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 0x7f49cc0039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e6a00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc003f30 0x7f49cc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7e00640 1 -- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 msgr2=0x7f49cc0039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7e00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e80b5970 0x7f49cc0039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7e00640 1 -- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc003f30 msgr2=0x7f49cc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7e00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49cc003f30 0x7f49cc004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.252+0000 7f49e7e00640 1 -- 172.21.15.165:0/1463696843 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc003520 con 0x7f49cc004d40 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.253+0000 7f49e7e00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009fc0 tx=0x7f49cc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cefaebae3d0c48 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0009c10 con 0x7f49cc004d40 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49e0036070 con 0x7f49cc004d40 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49e6000640 1 -- 172.21.15.165:0/1463696843 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e0031670 con 0x7f49cc004d40 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49ecc00640 1 -- 172.21.15.165:0/1463696843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 msgr2=0x7f49cc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49ecc00640 1 --2- 172.21.15.165:0/1463696843 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49cc0051c0 secure :-1 s=READY pgs=2682 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009fc0 tx=0x7f49cc005830 comp rx=0 tx=0).stop 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49e7e00640 1 -- 172.21.15.165:0/1463696843 reap_dead start 2024-04-25T10:20:28.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49ecc00640 1 -- 172.21.15.165:0/1463696843 shutdown_connections 2024-04-25T10:20:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49ecc00640 1 -- 172.21.15.165:0/1463696843 >> 172.21.15.165:0/1463696843 conn(0x7f49e80d0610 msgr2=0x7f49e80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49ecc00640 1 -- 172.21.15.165:0/1463696843 shutdown_connections 2024-04-25T10:20:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.254+0000 7f49ecc00640 1 -- 172.21.15.165:0/1463696843 wait complete. 2024-04-25T10:20:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.256+0000 7f49ecc00640 1 Processor -- start 2024-04-25T10:20:28.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.256+0000 7f49ecc00640 1 -- start start 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.257+0000 7f49ecc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49e80cc640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.257+0000 7f49ecc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e8155be0 0x7f49e80caf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.257+0000 7f49ecc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49e80cd870 0x7f49e8154ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.257+0000 7f49ecc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e806cd20 con 0x7f49cc004d40 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.257+0000 7f49ecc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e806ca40 con 0x7f49e80cd870 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.257+0000 7f49ecc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49e80d2160 con 0x7f49e8155be0 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49e80cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49e80cc640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39930/0 (socket says 172.21.15.165:39930) 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 -- 172.21.15.165:0/2351096771 learned_addr learned my addr 172.21.15.165:0/2351096771 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 -- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e8155be0 msgr2=0x7f49e80caf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7e00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49e80cd870 0x7f49e8154ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e6a00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e8155be0 0x7f49e80caf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e8155be0 0x7f49e80caf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 -- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49e80cd870 msgr2=0x7f49e8154ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49e80cd870 0x7f49e8154ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 -- 172.21.15.165:0/2351096771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80cb5a0 con 0x7f49cc004d40 2024-04-25T10:20:28.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.258+0000 7f49e7400640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49e80cc640 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e8c0 tx=0x7f49d800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.259+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800c870 con 0x7f49cc004d40 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.259+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d8018070 con 0x7f49cc004d40 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.259+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d800cc50 con 0x7f49cc004d40 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.259+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e80cb880 con 0x7f49cc004d40 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.259+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e80cbe40 con 0x7f49cc004d40 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.260+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49e8073990 con 0x7f49cc004d40 2024-04-25T10:20:28.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.261+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f49d801d460 con 0x7f49cc004d40 2024-04-25T10:20:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.262+0000 7f49df400640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f49c806b4d0 0x7f49c806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.263+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f49d8089ff0 con 0x7f49cc004d40 2024-04-25T10:20:28.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.263+0000 7f49e6a00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f49c806b4d0 0x7f49c806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.263+0000 7f49e6a00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f49c806b4d0 0x7f49c806d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f49e8159970 tx=0x7f49d4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.267+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f49d8058ff0 con 0x7f49cc004d40 2024-04-25T10:20:28.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.466+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f49e80d0bc0 con 0x7f49cc004d40 2024-04-25T10:20:28.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.472+0000 7f49df400640 1 -- 172.21.15.165:0/2351096771 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7f49d80588f0 con 0x7f49cc004d40 2024-04-25T10:20:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f49c806b4d0 msgr2=0x7f49c806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f49c806b4d0 0x7f49c806d990 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f49e8159970 tx=0x7f49d4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 msgr2=0x7f49e80cc640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49e80cc640 secure :-1 s=READY pgs=2683 cs=0 l=1 rev1=1 crypto rx=0x7f49d800e8c0 tx=0x7f49d800ed80 comp rx=0 tx=0).stop 2024-04-25T10:20:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 shutdown_connections 2024-04-25T10:20:28.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f49c806b4d0 0x7f49c806d990 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f49e80cd870 0x7f49e8154ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f49e8155be0 0x7f49e80caf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 --2- 172.21.15.165:0/2351096771 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f49cc004d40 0x7f49e80cc640 unknown :-1 s=CLOSED pgs=2683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 >> 172.21.15.165:0/2351096771 conn(0x7f49e80d0610 msgr2=0x7f49e80d38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:28.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 shutdown_connections 2024-04-25T10:20:28.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.475+0000 7f49ecc00640 1 -- 172.21.15.165:0/2351096771 wait complete. 2024-04-25T10:20:28.498 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files' 2024-04-25T10:20:28.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.707+0000 7f1c0a000640 1 Processor -- start 2024-04-25T10:20:28.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.707+0000 7f1c0a000640 1 -- start start 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.708+0000 7f1c0a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c040cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.708+0000 7f1c0a000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1c0415df50 0x7f1c040cf510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.708+0000 7f1c0a000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1c040cec70 0x7f1c040cfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.708+0000 7f1c0a000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c040cbd80 con 0x7f1c0415df50 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.708+0000 7f1c0a000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c04161910 con 0x7f1c040cec70 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.708+0000 7f1c0a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04071650 con 0x7f1c0415db70 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1c0415df50 0x7f1c040cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1c0415df50 0x7f1c040cf510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39932/0 (socket says 172.21.15.165:39932) 2024-04-25T10:20:28.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 -- 172.21.15.165:0/19946967 learned_addr learned my addr 172.21.15.165:0/19946967 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:28.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c03400640 1 --2- 172.21.15.165:0/19946967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c040cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c03e00640 1 --2- 172.21.15.165:0/19946967 >> v2:172.21.15.172:3300/0 conn(0x7f1c040cec70 0x7f1c040cfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 -- 172.21.15.165:0/19946967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 msgr2=0x7f1c040cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 --2- 172.21.15.165:0/19946967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c040cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 -- 172.21.15.165:0/19946967 >> v2:172.21.15.172:3300/0 conn(0x7f1c040cec70 msgr2=0x7f1c040cfa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:28.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 --2- 172.21.15.165:0/19946967 >> v2:172.21.15.172:3300/0 conn(0x7f1c040cec70 0x7f1c040cfa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.709+0000 7f1c02a00640 1 -- 172.21.15.165:0/19946967 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040cf230 con 0x7f1c0415df50 2024-04-25T10:20:28.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.710+0000 7f1c02a00640 1 --2- 172.21.15.165:0/19946967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0415df50 0x7f1c040cf510 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f1c040af270 tx=0x7f1bf002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26d5ff65bb8d6ef9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c02000640 1 -- 172.21.15.165:0/19946967 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0037070 con 0x7f1c0415df50 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c02000640 1 -- 172.21.15.165:0/19946967 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf0032440 con 0x7f1c0415df50 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c02000640 1 -- 172.21.15.165:0/19946967 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf0032810 con 0x7f1c0415df50 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 -- 172.21.15.165:0/19946967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0415df50 msgr2=0x7f1c040cf510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 --2- 172.21.15.165:0/19946967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0415df50 0x7f1c040cf510 secure :-1 s=READY pgs=2684 cs=0 l=1 rev1=1 crypto rx=0x7f1c040af270 tx=0x7f1bf002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 -- 172.21.15.165:0/19946967 shutdown_connections 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 --2- 172.21.15.165:0/19946967 >> v2:172.21.15.172:3300/0 conn(0x7f1c040cec70 0x7f1c040cfa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 --2- 172.21.15.165:0/19946967 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c0415df50 0x7f1c040cf510 unknown :-1 s=CLOSED pgs=2684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 --2- 172.21.15.165:0/19946967 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c040cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 -- 172.21.15.165:0/19946967 >> 172.21.15.165:0/19946967 conn(0x7f1c0406abe0 msgr2=0x7f1c0406aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:28.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 -- 172.21.15.165:0/19946967 shutdown_connections 2024-04-25T10:20:28.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.711+0000 7f1c0a000640 1 -- 172.21.15.165:0/19946967 wait complete. 2024-04-25T10:20:28.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.713+0000 7f1c0a000640 1 Processor -- start 2024-04-25T10:20:28.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.713+0000 7f1c0a000640 1 -- start start 2024-04-25T10:20:28.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c0a000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 0x7f1c04144de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c0a000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c04152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c0a000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0415df50 0x7f1c041528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c0a000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04162120 con 0x7f1c040cec70 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c0a000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04161e40 con 0x7f1c0415df50 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c0a000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04162640 con 0x7f1c0415db70 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.714+0000 7f1c03400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 0x7f1c04144de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 0x7f1c04144de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39940/0 (socket says 172.21.15.165:39940) 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 -- 172.21.15.165:0/1870019042 learned_addr learned my addr 172.21.15.165:0/1870019042 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c02a00640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c04152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 -- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 msgr2=0x7f1c04152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c04152390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03e00640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0415df50 0x7f1c041528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 -- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0415df50 msgr2=0x7f1c041528d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0415df50 0x7f1c041528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 -- 172.21.15.165:0/1870019042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c0415f130 con 0x7f1c040cec70 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.715+0000 7f1c03400640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 0x7f1c04144de0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400d800 tx=0x7f1bf400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.716+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400b6c0 con 0x7f1c040cec70 2024-04-25T10:20:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.716+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1bf4019070 con 0x7f1c040cec70 2024-04-25T10:20:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.716+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bf400ba80 con 0x7f1c040cec70 2024-04-25T10:20:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.716+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0415f410 con 0x7f1c040cec70 2024-04-25T10:20:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.716+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0415f9d0 con 0x7f1c040cec70 2024-04-25T10:20:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.717+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c0415a6f0 con 0x7f1c040cec70 2024-04-25T10:20:28.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.718+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1bf400bc20 con 0x7f1c040cec70 2024-04-25T10:20:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.719+0000 7f1bfb400640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:28.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.719+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f1bf408a570 con 0x7f1c040cec70 2024-04-25T10:20:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.719+0000 7f1c02a00640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:28.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.720+0000 7f1c02a00640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1be406b4d0 0x7f1be406d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0034000 tx=0x7f1bf002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:28.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.723+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c0415a6f0 con 0x7f1c040cec70 2024-04-25T10:20:28.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:28.992+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]} v 0) v1 -- 0x7f1c041577e0 con 0x7f1c040cec70 2024-04-25T10:20:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.040+0000 7f1bfb400640 1 -- 172.21.15.165:0/1870019042 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_backtrace_for_new_files"]}]=0 v848) v1 ==== 147+0+0 (secure 0 0 0) 0x7f1bf40594c0 con 0x7f1c040cec70 2024-04-25T10:20:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.043+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1be406b4d0 msgr2=0x7f1be406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.043+0000 7f1c0a000640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1be406b4d0 0x7f1be406d990 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f1bf0034000 tx=0x7f1bf002e000 comp rx=0 tx=0).stop 2024-04-25T10:20:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.043+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 msgr2=0x7f1c04144de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.043+0000 7f1c0a000640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 0x7f1c04144de0 secure :-1 s=READY pgs=2685 cs=0 l=1 rev1=1 crypto rx=0x7f1bf400d800 tx=0x7f1bf400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.043+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 shutdown_connections 2024-04-25T10:20:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.043+0000 7f1c0a000640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1be406b4d0 0x7f1be406d990 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.044+0000 7f1c0a000640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c0415df50 0x7f1c041528d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.044+0000 7f1c0a000640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c0415db70 0x7f1c04152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.044+0000 7f1c0a000640 1 --2- 172.21.15.165:0/1870019042 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c040cec70 0x7f1c04144de0 unknown :-1 s=CLOSED pgs=2685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.044+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 >> 172.21.15.165:0/1870019042 conn(0x7f1c0406abe0 msgr2=0x7f1c040ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.044+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 shutdown_connections 2024-04-25T10:20:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.044+0000 7f1c0a000640 1 -- 172.21.15.165:0/1870019042 wait complete. 2024-04-25T10:20:29.065 INFO:tasks.cephfs_test_runner:test_scrub_backtrace_for_new_files (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:20:29.065 INFO:tasks.cephfs_test_runner:Starting test: test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) 2024-04-25T10:20:29.065 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:29.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.260+0000 7faade400640 1 Processor -- start 2024-04-25T10:20:29.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.260+0000 7faade400640 1 -- start start 2024-04-25T10:20:29.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faade400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7faad815df50 0x7faad80cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faade400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad8071650 con 0x7faad815db70 2024-04-25T10:20:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faade400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faad80cf230 con 0x7faad80cfc20 2024-04-25T10:20:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faade400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faad80cbd80 con 0x7faad815df50 2024-04-25T10:20:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faad7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7faad815db70 0x7faad80cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51888/0 (socket says 172.21.15.165:51888) 2024-04-25T10:20:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faad7400640 1 -- 172.21.15.165:0/1327124927 learned_addr learned my addr 172.21.15.165:0/1327124927 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.262+0000 7faad7e00640 1 --2- 172.21.15.165:0/1327124927 >> v2:172.21.15.165:3300/0 conn(0x7faad815df50 0x7faad80cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.263+0000 7faad6000640 1 -- 172.21.15.165:0/1327124927 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3627540594 0 0) 0x7faac8001110 con 0x7faad815db70 2024-04-25T10:20:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.263+0000 7faad6000640 1 -- 172.21.15.165:0/1327124927 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faabc003520 con 0x7faad815db70 2024-04-25T10:20:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.263+0000 7faad7e00640 1 -- 172.21.15.165:0/1327124927 >> v1:172.21.15.172:6789/0 conn(0x7faad80cfc20 legacy=0x7faad806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.263+0000 7faad7e00640 1 -- 172.21.15.165:0/1327124927 >> v1:172.21.15.165:6789/0 conn(0x7faad815db70 legacy=0x7faad80cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.263+0000 7faad7e00640 1 -- 172.21.15.165:0/1327124927 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad8161b60 con 0x7faad815df50 2024-04-25T10:20:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.264+0000 7faad7e00640 1 --2- 172.21.15.165:0/1327124927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad815df50 0x7faad80cb840 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7faad0009d90 tx=0x7faad002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85034813210f0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.264+0000 7faad6000640 1 -- 172.21.15.165:0/1327124927 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0007740 con 0x7faad815df50 2024-04-25T10:20:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.264+0000 7faad6000640 1 -- 172.21.15.165:0/1327124927 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faad00078e0 con 0x7faad815df50 2024-04-25T10:20:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.264+0000 7faad6000640 1 -- 172.21.15.165:0/1327124927 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad0007cb0 con 0x7faad815df50 2024-04-25T10:20:29.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 -- 172.21.15.165:0/1327124927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad815df50 msgr2=0x7faad80cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 --2- 172.21.15.165:0/1327124927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad815df50 0x7faad80cb840 secure :-1 s=READY pgs=2687 cs=0 l=1 rev1=1 crypto rx=0x7faad0009d90 tx=0x7faad002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 -- 172.21.15.165:0/1327124927 shutdown_connections 2024-04-25T10:20:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 --2- 172.21.15.165:0/1327124927 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad815df50 0x7faad80cb840 unknown :-1 s=CLOSED pgs=2687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 -- 172.21.15.165:0/1327124927 >> 172.21.15.165:0/1327124927 conn(0x7faad806abe0 msgr2=0x7faad806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 -- 172.21.15.165:0/1327124927 shutdown_connections 2024-04-25T10:20:29.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.265+0000 7faade400640 1 -- 172.21.15.165:0/1327124927 wait complete. 2024-04-25T10:20:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.267+0000 7faade400640 1 Processor -- start 2024-04-25T10:20:29.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.267+0000 7faade400640 1 -- start start 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faade400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 0x7faad81512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faade400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad815db70 0x7faad81f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faade400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad815df50 0x7faad8151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faade400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8162120 con 0x7faad80cfc20 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faade400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faad8161910 con 0x7faad815db70 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faade400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faad8162640 con 0x7faad815df50 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 0x7faad81512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 0x7faad81512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39962/0 (socket says 172.21.15.165:39962) 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 -- 172.21.15.165:0/2750643826 learned_addr learned my addr 172.21.15.165:0/2750643826 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 -- 172.21.15.165:0/2750643826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad815df50 msgr2=0x7faad8151af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad815df50 0x7faad8151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 -- 172.21.15.165:0/2750643826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad815db70 msgr2=0x7faad81f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad815db70 0x7faad81f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.268+0000 7faad7400640 1 -- 172.21.15.165:0/2750643826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad8152620 con 0x7faad80cfc20 2024-04-25T10:20:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.269+0000 7faad7400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 0x7faad81512d0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7faac800c760 tx=0x7faac800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.269+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8017070 con 0x7faad80cfc20 2024-04-25T10:20:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.269+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faac8005ad0 con 0x7faad80cfc20 2024-04-25T10:20:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.269+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faad8152900 con 0x7faad80cfc20 2024-04-25T10:20:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.269+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faac8007e20 con 0x7faad80cfc20 2024-04-25T10:20:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.269+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faad81f3cc0 con 0x7faad80cfc20 2024-04-25T10:20:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.270+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faad8073990 con 0x7faad80cfc20 2024-04-25T10:20:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.271+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7faac801a070 con 0x7faad80cfc20 2024-04-25T10:20:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.272+0000 7faacf400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab806b500 0x7faab806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.272+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7faac8089360 con 0x7faad80cfc20 2024-04-25T10:20:29.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.272+0000 7faad6a00640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab806b500 0x7faab806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.273+0000 7faad6a00640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab806b500 0x7faab806d9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faac4006fd0 tx=0x7faac4006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.277+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7faad8071650 con 0x7faad80cfc20 2024-04-25T10:20:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.475+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faad80cedd0 con 0x7faad80cfc20 2024-04-25T10:20:29.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.477+0000 7faacf400640 1 -- 172.21.15.165:0/2750643826 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v187) v1 ==== 74+0+13341 (secure 0 0 0) 0x7faac8087e20 con 0x7faad80cfc20 2024-04-25T10:20:29.480 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:29.480 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":187,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:28.020093+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/1159947775":"2024-04-26T10:20:25.002556+0000","172.21.15.165:6835/1159947775":"2024-04-26T10:20:25.002556+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab806b500 msgr2=0x7faab806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab806b500 0x7faab806d9c0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7faac4006fd0 tx=0x7faac4006bc0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 msgr2=0x7faad81512d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 0x7faad81512d0 secure :-1 s=READY pgs=2688 cs=0 l=1 rev1=1 crypto rx=0x7faac800c760 tx=0x7faac800cc20 comp rx=0 tx=0).stop 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 shutdown_connections 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7faab806b500 0x7faab806d9c0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7faad815df50 0x7faad8151af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7faad815db70 0x7faad81f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 --2- 172.21.15.165:0/2750643826 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7faad80cfc20 0x7faad81512d0 unknown :-1 s=CLOSED pgs=2688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 >> 172.21.15.165:0/2750643826 conn(0x7faad806abe0 msgr2=0x7faad8158070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 shutdown_connections 2024-04-25T10:20:29.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.480+0000 7faade400640 1 -- 172.21.15.165:0/2750643826 wait complete. 2024-04-25T10:20:29.502 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:29.502 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.704+0000 7f4364a00640 1 Processor -- start 2024-04-25T10:20:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.704+0000 7f4364a00640 1 -- start start 2024-04-25T10:20:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.706+0000 7f4364a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f4360161710 0x7f43600d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.706+0000 7f4364a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f436015e000 con 0x7f436015dc20 2024-04-25T10:20:29.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.706+0000 7f4364a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43600d1de0 con 0x7f43600ced80 2024-04-25T10:20:29.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.706+0000 7f4364a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4360161c60 con 0x7f4360161710 2024-04-25T10:20:29.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.706+0000 7f435de00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f436015dc20 0x7f43600ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51904/0 (socket says 172.21.15.165:51904) 2024-04-25T10:20:29.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.706+0000 7f435de00640 1 -- 172.21.15.165:0/1896783893 learned_addr learned my addr 172.21.15.165:0/1896783893 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435e800640 1 --2- 172.21.15.165:0/1896783893 >> v2:172.21.15.172:3300/0 conn(0x7f4360161710 0x7f43600d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964265318 0 0) 0x7f436015e000 con 0x7f436015dc20 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4344003520 con 0x7f436015dc20 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2158907138 0 0) 0x7f4344003520 con 0x7f436015dc20 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 >> v2:172.21.15.172:3300/0 conn(0x7f4360161710 msgr2=0x7f43600d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 --2- 172.21.15.165:0/1896783893 >> v2:172.21.15.172:3300/0 conn(0x7f4360161710 0x7f43600d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 >> v1:172.21.15.172:6789/0 conn(0x7f43600ced80 legacy=0x7f43600d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4360161f40 con 0x7f436015dc20 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f4348001670 con 0x7f436015dc20 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 >> v1:172.21.15.165:6789/0 conn(0x7f436015dc20 legacy=0x7f43600ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4344003a00 0x7f4344003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4344005210 0x7f434400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4360161c60 con 0x7f4344003a00 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434400c160 con 0x7f4344004800 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.707+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f434400c360 con 0x7f4344005210 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435de00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4344003a00 0x7f4344003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435d400640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435de00640 1 -- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4344005210 msgr2=0x7f434400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435de00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4344005210 0x7f434400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435de00640 1 -- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344004800 msgr2=0x7f4344004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435de00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344004800 0x7f4344004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435de00640 1 -- 172.21.15.165:0/1896783893 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344003520 con 0x7f4344003a00 2024-04-25T10:20:29.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.708+0000 7f435e800640 1 -- 172.21.15.165:0/1896783893 reap_dead start 2024-04-25T10:20:29.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.709+0000 7f435de00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4344003a00 0x7f4344003e20 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f434800c110 tx=0x7f4344005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1796280ba7e61151 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348007c40 con 0x7f4344003a00 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4348005af0 con 0x7f4344003a00 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f435ca00640 1 -- 172.21.15.165:0/1896783893 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4348007070 con 0x7f4344003a00 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 -- 172.21.15.165:0/1896783893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4344003a00 msgr2=0x7f4344003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4344003a00 0x7f4344003e20 secure :-1 s=READY pgs=2690 cs=0 l=1 rev1=1 crypto rx=0x7f434800c110 tx=0x7f4344005830 comp rx=0 tx=0).stop 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 -- 172.21.15.165:0/1896783893 shutdown_connections 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 --2- 172.21.15.165:0/1896783893 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4344003a00 0x7f4344003e20 unknown :-1 s=CLOSED pgs=2690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 -- 172.21.15.165:0/1896783893 >> 172.21.15.165:0/1896783893 conn(0x7f43600d0420 msgr2=0x7f43600d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 -- 172.21.15.165:0/1896783893 shutdown_connections 2024-04-25T10:20:29.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.710+0000 7f4364a00640 1 -- 172.21.15.165:0/1896783893 wait complete. 2024-04-25T10:20:29.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.712+0000 7f4364a00640 1 Processor -- start 2024-04-25T10:20:29.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.712+0000 7f4364a00640 1 -- start start 2024-04-25T10:20:29.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f4364a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344003a00 0x7f4360160b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f4364a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436015c5e0 0x7f43600caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f4364a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 0x7f43600cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f4364a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b5e50 con 0x7f43600cb480 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f4364a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4360162500 con 0x7f4344003a00 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f4364a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43600b5fd0 con 0x7f436015c5e0 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.713+0000 7f435e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 0x7f43600cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 0x7f43600cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39970/0 (socket says 172.21.15.165:39970) 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 -- 172.21.15.165:0/1958803413 learned_addr learned my addr 172.21.15.165:0/1958803413 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435d400640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436015c5e0 0x7f43600caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435de00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344003a00 0x7f4360160b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 -- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436015c5e0 msgr2=0x7f43600caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436015c5e0 0x7f43600caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 -- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344003a00 msgr2=0x7f4360160b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344003a00 0x7f4360160b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 -- 172.21.15.165:0/1958803413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43601f5030 con 0x7f43600cb480 2024-04-25T10:20:29.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.714+0000 7f435e800640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 0x7f43600cc880 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f43600b3290 tx=0x7f435800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.715+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435800ed90 con 0x7f43600cb480 2024-04-25T10:20:29.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.715+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358019070 con 0x7f43600cb480 2024-04-25T10:20:29.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.715+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358014660 con 0x7f43600cb480 2024-04-25T10:20:29.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.715+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436015ec20 con 0x7f43600cb480 2024-04-25T10:20:29.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.715+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436015f1e0 con 0x7f43600cb480 2024-04-25T10:20:29.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.716+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4360073a00 con 0x7f43600cb480 2024-04-25T10:20:29.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.717+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4358005d10 con 0x7f43600cb480 2024-04-25T10:20:29.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.718+0000 7f4356000640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f433c06b500 0x7f433c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:29.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.718+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f435808a360 con 0x7f43600cb480 2024-04-25T10:20:29.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.718+0000 7f435de00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f433c06b500 0x7f433c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:29.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.719+0000 7f435de00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f433c06b500 0x7f433c06d9c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4348032000 tx=0x7f4348009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.723+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4358023020 con 0x7f43600cb480 2024-04-25T10:20:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.917+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f43600cad20 con 0x7f43600cb480 2024-04-25T10:20:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.918+0000 7f4356000640 1 -- 172.21.15.165:0/1958803413 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v187) v1 ==== 93+0+10 (secure 0 0 0) 0x7f43580592b0 con 0x7f43600cb480 2024-04-25T10:20:29.921 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f433c06b500 msgr2=0x7f433c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f433c06b500 0x7f433c06d9c0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4348032000 tx=0x7f4348009030 comp rx=0 tx=0).stop 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 msgr2=0x7f43600cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 0x7f43600cc880 secure :-1 s=READY pgs=2691 cs=0 l=1 rev1=1 crypto rx=0x7f43600b3290 tx=0x7f435800e4b0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 shutdown_connections 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f433c06b500 0x7f433c06d9c0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f43600cb480 0x7f43600cc880 unknown :-1 s=CLOSED pgs=2691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f436015c5e0 0x7f43600caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 --2- 172.21.15.165:0/1958803413 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4344003a00 0x7f4360160b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 >> 172.21.15.165:0/1958803413 conn(0x7f43600d0420 msgr2=0x7f43600d4210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 shutdown_connections 2024-04-25T10:20:29.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:29.921+0000 7f4364a00640 1 -- 172.21.15.165:0/1958803413 wait complete. 2024-04-25T10:20:29.943 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:20:30.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.143+0000 7f3c31000640 1 Processor -- start 2024-04-25T10:20:30.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.143+0000 7f3c31000640 1 -- start start 2024-04-25T10:20:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.145+0000 7f3c31000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3c2c154ee0 0x7f3c2c0cf7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.145+0000 7f3c31000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2c071480 con 0x7f3c2c0cfcf0 2024-04-25T10:20:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.145+0000 7f3c31000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c2c071760 con 0x7f3c2c0ce590 2024-04-25T10:20:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.145+0000 7f3c31000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c2c072090 con 0x7f3c2c154ee0 2024-04-25T10:20:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.145+0000 7f3c21400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3c2c0cfcf0 0x7f3c2c0cde80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51908/0 (socket says 172.21.15.165:51908) 2024-04-25T10:20:30.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.145+0000 7f3c21400640 1 -- 172.21.15.165:0/3959844357 learned_addr learned my addr 172.21.15.165:0/3959844357 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29e00640 1 --2- 172.21.15.165:0/3959844357 >> v2:172.21.15.172:3300/0 conn(0x7f3c2c154ee0 0x7f3c2c0cf7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867638102 0 0) 0x7f3c2c071480 con 0x7f3c2c0cfcf0 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c10003520 con 0x7f3c2c0cfcf0 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2689419631 0 0) 0x7f3c10003520 con 0x7f3c2c0cfcf0 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 >> v2:172.21.15.172:3300/0 conn(0x7f3c2c154ee0 msgr2=0x7f3c2c0cf7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 --2- 172.21.15.165:0/3959844357 >> v2:172.21.15.172:3300/0 conn(0x7f3c2c154ee0 0x7f3c2c0cf7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 >> v1:172.21.15.172:6789/0 conn(0x7f3c2c0ce590 legacy=0x7f3c2c06d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c071a40 con 0x7f3c2c0cfcf0 2024-04-25T10:20:30.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3c14001670 con 0x7f3c2c0cfcf0 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29e00640 1 --2- 172.21.15.165:0/3959844357 >> v2:172.21.15.172:3300/0 conn(0x7f3c2c154ee0 0x7f3c2c0cf7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 >> v1:172.21.15.165:6789/0 conn(0x7f3c2c0cfcf0 legacy=0x7f3c2c0cde80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c10003a00 0x7f3c10003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.146+0000 7f3c29400640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10004800 0x7f3c10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c29400640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c1000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000c130 con 0x7f3c10005210 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000c2e0 con 0x7f3c10004800 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1000c4e0 con 0x7f3c10003a00 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c2a800640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c1000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c21400640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10004800 0x7f3c10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c2a800640 1 -- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c10003a00 msgr2=0x7f3c10003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c2a800640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c10003a00 0x7f3c10003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c2a800640 1 -- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10004800 msgr2=0x7f3c10004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c2a800640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c10004800 0x7f3c10004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.147+0000 7f3c2a800640 1 -- 172.21.15.165:0/3959844357 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c10003520 con 0x7f3c10005210 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.148+0000 7f3c2a800640 1 -- 172.21.15.165:0/3959844357 reap_dead start 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.148+0000 7f3c2a800640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c1000b9c0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f3c24001e50 tx=0x7f3c10005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72c69568cc3830 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.148+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24001a70 con 0x7f3c10005210 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.148+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c24003970 con 0x7f3c10005210 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.148+0000 7f3c29400640 1 -- 172.21.15.165:0/3959844357 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c24003d60 con 0x7f3c10005210 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 -- 172.21.15.165:0/3959844357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 msgr2=0x7f3c1000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c1000b9c0 secure :-1 s=READY pgs=2693 cs=0 l=1 rev1=1 crypto rx=0x7f3c24001e50 tx=0x7f3c10005830 comp rx=0 tx=0).stop 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 -- 172.21.15.165:0/3959844357 shutdown_connections 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 --2- 172.21.15.165:0/3959844357 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c1000b9c0 unknown :-1 s=CLOSED pgs=2693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 -- 172.21.15.165:0/3959844357 >> 172.21.15.165:0/3959844357 conn(0x7f3c2c06abe0 msgr2=0x7f3c2c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 -- 172.21.15.165:0/3959844357 shutdown_connections 2024-04-25T10:20:30.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.149+0000 7f3c31000640 1 -- 172.21.15.165:0/3959844357 wait complete. 2024-04-25T10:20:30.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.151+0000 7f3c31000640 1 Processor -- start 2024-04-25T10:20:30.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.151+0000 7f3c31000640 1 -- start start 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c31000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c2c0afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c31000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c2c159a90 0x7f3c2c1f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c31000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c2c15c870 0x7f3c2c15cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c31000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c1552c0 con 0x7f3c10005210 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c31000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c071230 con 0x7f3c2c159a90 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c31000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2c155440 con 0x7f3c2c15c870 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c29e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c2c0afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c29e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c2c0afcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39982/0 (socket says 172.21.15.165:39982) 2024-04-25T10:20:30.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.152+0000 7f3c29e00640 1 -- 172.21.15.165:0/361478635 learned_addr learned my addr 172.21.15.165:0/361478635 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:30.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c29e00640 1 -- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c2c15c870 msgr2=0x7f3c2c15cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c29e00640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c2c15c870 0x7f3c2c15cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c29e00640 1 -- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c2c159a90 msgr2=0x7f3c2c1f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c21400640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c2c159a90 0x7f3c2c1f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c29e00640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c2c159a90 0x7f3c2c1f6710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c29e00640 1 -- 172.21.15.165:0/361478635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c2c1f6c50 con 0x7f3c10005210 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c29e00640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c2c0afcd0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00e600 tx=0x7f3c1c00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c00f040 con 0x7f3c10005210 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1c004230 con 0x7f3c10005210 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.153+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c2c0b0360 con 0x7f3c10005210 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.154+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1c003610 con 0x7f3c10005210 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.154+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c2c0b0920 con 0x7f3c10005210 2024-04-25T10:20:30.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.155+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c2c15a820 con 0x7f3c10005210 2024-04-25T10:20:30.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.156+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3c1c0037b0 con 0x7f3c10005210 2024-04-25T10:20:30.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.157+0000 7f3c22a00640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c0806b4d0 0x7f3c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:30.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.157+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f3c1c089d50 con 0x7f3c10005210 2024-04-25T10:20:30.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.157+0000 7f3c21400640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c0806b4d0 0x7f3c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:30.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.157+0000 7f3c21400640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c0806b4d0 0x7f3c0806d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f3c14001200 tx=0x7f3c14009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:30.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.160+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3c2c15a820 con 0x7f3c10005210 2024-04-25T10:20:30.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:30.426+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f3c2c073990 con 0x7f3c10005210 2024-04-25T10:20:31.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.046+0000 7f3c22a00640 1 -- 172.21.15.165:0/361478635 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v850) v1 ==== 136+0+0 (secure 0 0 0) 0x7f3c1c017020 con 0x7f3c10005210 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c0806b4d0 msgr2=0x7f3c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c0806b4d0 0x7f3c0806d990 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f3c14001200 tx=0x7f3c14009040 comp rx=0 tx=0).stop 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 msgr2=0x7f3c2c0afcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c2c0afcd0 secure :-1 s=READY pgs=2694 cs=0 l=1 rev1=1 crypto rx=0x7f3c1c00e600 tx=0x7f3c1c00eac0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 shutdown_connections 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3c0806b4d0 0x7f3c0806d990 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3c2c15c870 0x7f3c2c15cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3c2c159a90 0x7f3c2c1f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 --2- 172.21.15.165:0/361478635 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3c10005210 0x7f3c2c0afcd0 unknown :-1 s=CLOSED pgs=2694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 >> 172.21.15.165:0/361478635 conn(0x7f3c2c06abe0 msgr2=0x7f3c2c0b3010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 shutdown_connections 2024-04-25T10:20:31.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.050+0000 7f3c31000640 1 -- 172.21.15.165:0/361478635 wait complete. 2024-04-25T10:20:31.074 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-04-25T10:20:31.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.275+0000 7f6101a00640 1 Processor -- start 2024-04-25T10:20:31.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.276+0000 7f6101a00640 1 -- start start 2024-04-25T10:20:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f6101a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60fc0b1140 0x7f60fc0b3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f6101a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0d0d40 0x7f60fc146830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f6101a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60fc0b5490 con 0x7f60fc0b1fb0 2024-04-25T10:20:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f6101a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60fc15ea60 con 0x7f60fc0b1140 2024-04-25T10:20:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f6101a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b5730 con 0x7f60fc0d0d40 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f6101000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0d0d40 0x7f60fc146830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.278+0000 7f60fbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60fc0b1140 0x7f60fc0b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f60fbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f60fc0b1140 0x7f60fc0b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:56892/0 (socket says 172.21.15.165:56892) 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f6101000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0d0d40 0x7f60fc146830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:55652/0 (socket says 172.21.15.165:55652) 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f60fbe00640 1 -- 172.21.15.165:0/1792927867 learned_addr learned my addr 172.21.15.165:0/1792927867 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f60fbe00640 1 -- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0d0d40 msgr2=0x7f60fc146830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f60fbe00640 1 --2- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0d0d40 0x7f60fc146830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f60fbe00640 1 -- 172.21.15.165:0/1792927867 >> v1:172.21.15.172:6789/0 conn(0x7f60fc0b1fb0 legacy=0x7f60fc0d0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.279+0000 7f60fbe00640 1 -- 172.21.15.165:0/1792927867 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc0b5a10 con 0x7f60fc0b1140 2024-04-25T10:20:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.280+0000 7f60fbe00640 1 --2- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0b1140 0x7f60fc0b3690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009e50 tx=0x7f60ec02a5f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1791b4ebdfb36a0c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.281+0000 7f60faa00640 1 -- 172.21.15.165:0/1792927867 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec036070 con 0x7f60fc0b1140 2024-04-25T10:20:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.281+0000 7f60faa00640 1 -- 172.21.15.165:0/1792927867 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec031600 con 0x7f60fc0b1140 2024-04-25T10:20:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.281+0000 7f60faa00640 1 -- 172.21.15.165:0/1792927867 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec0319d0 con 0x7f60fc0b1140 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 -- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0b1140 msgr2=0x7f60fc0b3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 --2- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0b1140 0x7f60fc0b3690 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f60ec009e50 tx=0x7f60ec02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 -- 172.21.15.165:0/1792927867 shutdown_connections 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 --2- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0d0d40 0x7f60fc146830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 --2- 172.21.15.165:0/1792927867 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0b1140 0x7f60fc0b3690 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 -- 172.21.15.165:0/1792927867 >> 172.21.15.165:0/1792927867 conn(0x7f60fc0728b0 msgr2=0x7f60fc099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 -- 172.21.15.165:0/1792927867 shutdown_connections 2024-04-25T10:20:31.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.282+0000 7f6101a00640 1 -- 172.21.15.165:0/1792927867 wait complete. 2024-04-25T10:20:31.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.284+0000 7f6101a00640 1 Processor -- start 2024-04-25T10:20:31.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.285+0000 7f6101a00640 1 -- start start 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 0x7f60fc1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0b1fb0 0x7f60fc0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0d0d40 0x7f60fc150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b3d50 con 0x7f60fc0b1140 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b3bd0 con 0x7f60fc0d0d40 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60fc0b3ed0 con 0x7f60fc0b1fb0 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f60fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 0x7f60fc1f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f60fbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 0x7f60fc1f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39984/0 (socket says 172.21.15.165:39984) 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f60fbe00640 1 -- 172.21.15.165:0/130857917 learned_addr learned my addr 172.21.15.165:0/130857917 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:31.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f6101000640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0d0d40 0x7f60fc150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.286+0000 7f60fb400640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0b1fb0 0x7f60fc0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fbe00640 1 -- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0b1fb0 msgr2=0x7f60fc0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fbe00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0b1fb0 0x7f60fc0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fbe00640 1 -- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0d0d40 msgr2=0x7f60fc150c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fbe00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0d0d40 0x7f60fc150c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fbe00640 1 -- 172.21.15.165:0/130857917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60fc1f6a40 con 0x7f60fc0b1140 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fb400640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0b1fb0 0x7f60fc0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f6101000640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0d0d40 0x7f60fc150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60fbe00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 0x7f60fc1f5230 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f80 tx=0x7f60ec004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec036050 con 0x7f60fc0b1140 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ec042070 con 0x7f60fc0b1140 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ec007c70 con 0x7f60fc0b1140 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60fc1f59a0 con 0x7f60fc0b1140 2024-04-25T10:20:31.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.287+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60fc1f5f60 con 0x7f60fc0b1140 2024-04-25T10:20:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.288+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60fc073990 con 0x7f60fc0b1140 2024-04-25T10:20:31.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.289+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f60ec047460 con 0x7f60fc0b1140 2024-04-25T10:20:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.290+0000 7f60f3e00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60d806b4d0 0x7f60d806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.290+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f60ec0aafa0 con 0x7f60fc0b1140 2024-04-25T10:20:31.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.290+0000 7f60fb400640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60d806b4d0 0x7f60d806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:31.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.291+0000 7f60fb400640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60d806b4d0 0x7f60d806d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f60fc15d580 tx=0x7f60e4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:31.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.295+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f60ec079f00 con 0x7f60fc0b1140 2024-04-25T10:20:31.434 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:31.442 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:20:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.490+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f60fc099ea0 con 0x7f60fc0b1140 2024-04-25T10:20:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.494+0000 7f60f3e00640 1 -- 172.21.15.165:0/130857917 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f60ec079800 con 0x7f60fc0b1140 2024-04-25T10:20:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60d806b4d0 msgr2=0x7f60d806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60d806b4d0 0x7f60d806d990 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f60fc15d580 tx=0x7f60e4009040 comp rx=0 tx=0).stop 2024-04-25T10:20:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 msgr2=0x7f60fc1f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 0x7f60fc1f5230 secure :-1 s=READY pgs=2695 cs=0 l=1 rev1=1 crypto rx=0x7f60ec000f80 tx=0x7f60ec004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 shutdown_connections 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f60d806b4d0 0x7f60d806d990 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60fc0d0d40 0x7f60fc150c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f60fc0b1fb0 0x7f60fc0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 --2- 172.21.15.165:0/130857917 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f60fc0b1140 0x7f60fc1f5230 unknown :-1 s=CLOSED pgs=2695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 >> 172.21.15.165:0/130857917 conn(0x7f60fc0728b0 msgr2=0x7f60fc0af610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 shutdown_connections 2024-04-25T10:20:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:31.497+0000 7f6101a00640 1 -- 172.21.15.165:0/130857917 wait complete. 2024-04-25T10:20:31.518 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:20:31.519 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:31.619 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:20:31.722 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:20:31.723 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:31.723 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.784 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:31.784 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:20:31.785 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.841 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:31.841 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:31.841 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:20:31.898 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:20:31.898 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:31.957 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:20:31.958 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-04-25T10:20:31.958 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:32.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.167+0000 7fcbc0000640 1 Processor -- start 2024-04-25T10:20:32.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.167+0000 7fcbc0000640 1 -- start start 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbc0000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 0x7fcbb80b9230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbc0000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcbb80b9770 0x7fcbb80d03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbc0000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb80d1df0 con 0x7fcbb80d2e90 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbc0000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb80b7a50 con 0x7fcbb80b9770 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbc0000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80d20d0 con 0x7fcbb815dd30 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbbe000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcbb80d2e90 0x7fcbb80b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51918/0 (socket says 172.21.15.165:51918) 2024-04-25T10:20:32.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.169+0000 7fcbbe000640 1 -- 172.21.15.165:0/1297579277 learned_addr learned my addr 172.21.15.165:0/1297579277 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbbcc00640 1 --2- 172.21.15.165:0/1297579277 >> v2:172.21.15.172:3300/0 conn(0x7fcbb80b9770 0x7fcbb80d03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbbd600640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 0x7fcbb80b9230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823124948 0 0) 0x7fcbb80d1df0 con 0x7fcbb80d2e90 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb9c003520 con 0x7fcbb80d2e90 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644534364 0 0) 0x7fcb9c003520 con 0x7fcbb80d2e90 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 msgr2=0x7fcbb80b9230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 0x7fcbb80b9230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 >> v2:172.21.15.172:3300/0 conn(0x7fcbb80b9770 msgr2=0x7fcbb80d03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 --2- 172.21.15.165:0/1297579277 >> v2:172.21.15.172:3300/0 conn(0x7fcbb80b9770 0x7fcbb80d03c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80d23b0 con 0x7fcbb80d2e90 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fcbb00019f0 con 0x7fcbb80d2e90 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 >> v1:172.21.15.165:6789/0 conn(0x7fcbb80d2e90 legacy=0x7fcbb80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.170+0000 7fcbb7e00640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb9c0039b0 0x7fcb9c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbb7e00640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 0x7fcb9c004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbb7e00640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c004d40 0x7fcb9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80b7a50 con 0x7fcb9c0039b0 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80d20d0 con 0x7fcb9c004d40 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9c00b9f0 con 0x7fcbb815dd30 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbd600640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb9c0039b0 0x7fcb9c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbcc00640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 0x7fcb9c004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbe000640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c004d40 0x7fcb9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbd600640 1 -- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 msgr2=0x7fcb9c004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbd600640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815dd30 0x7fcb9c004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbd600640 1 -- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c004d40 msgr2=0x7fcb9c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbd600640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c004d40 0x7fcb9c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbd600640 1 -- 172.21.15.165:0/1297579277 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c003520 con 0x7fcb9c0039b0 2024-04-25T10:20:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.171+0000 7fcbbe000640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c004d40 0x7fcb9c0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.172+0000 7fcbbd600640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb9c0039b0 0x7fcb9c003dd0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ac0 tx=0x7fcb9c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23894a5606ba269c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8036070 con 0x7fcb9c0039b0 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba8031400 con 0x7fcb9c0039b0 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbb7e00640 1 -- 172.21.15.165:0/1297579277 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8009690 con 0x7fcb9c0039b0 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbc0000640 1 -- 172.21.15.165:0/1297579277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb9c0039b0 msgr2=0x7fcb9c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbc0000640 1 --2- 172.21.15.165:0/1297579277 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcb9c0039b0 0x7fcb9c003dd0 secure :-1 s=READY pgs=2697 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009ac0 tx=0x7fcb9c005830 comp rx=0 tx=0).stop 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbbe000640 1 -- 172.21.15.165:0/1297579277 reap_dead start 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbc0000640 1 -- 172.21.15.165:0/1297579277 shutdown_connections 2024-04-25T10:20:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbc0000640 1 -- 172.21.15.165:0/1297579277 >> 172.21.15.165:0/1297579277 conn(0x7fcbb8061c30 msgr2=0x7fcbb806c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbc0000640 1 -- 172.21.15.165:0/1297579277 shutdown_connections 2024-04-25T10:20:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.173+0000 7fcbc0000640 1 -- 172.21.15.165:0/1297579277 wait complete. 2024-04-25T10:20:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.175+0000 7fcbc0000640 1 Processor -- start 2024-04-25T10:20:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.175+0000 7fcbc0000640 1 -- start start 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbc0000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c0039b0 0x7fcbb80addc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbc0000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 0x7fcbb80b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbc0000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815e370 0x7fcbb815e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbc0000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80d1a10 con 0x7fcbb815ba80 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbc0000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80d2970 con 0x7fcb9c0039b0 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbc0000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb80d1b90 con 0x7fcbb815e370 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.176+0000 7fcbbcc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 0x7fcbb80b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 0x7fcbb80b02a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40000/0 (socket says 172.21.15.165:40000) 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 -- 172.21.15.165:0/2394723541 learned_addr learned my addr 172.21.15.165:0/2394723541 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbd600640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c0039b0 0x7fcbb80addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbe000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815e370 0x7fcbb815e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 -- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815e370 msgr2=0x7fcbb815e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815e370 0x7fcbb815e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 -- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c0039b0 msgr2=0x7fcbb80addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c0039b0 0x7fcbb80addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 -- 172.21.15.165:0/2394723541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb80ae450 con 0x7fcbb815ba80 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbd600640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c0039b0 0x7fcbb80addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.177+0000 7fcbbcc00640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 0x7fcbb80b02a0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fcba40108d0 tx=0x7fcba4010d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.178+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba400f040 con 0x7fcbb815ba80 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.178+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba400eb60 con 0x7fcbb815ba80 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.178+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba401c400 con 0x7fcbb815ba80 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.178+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb80ae730 con 0x7fcbb815ba80 2024-04-25T10:20:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.178+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb80aecf0 con 0x7fcbb815ba80 2024-04-25T10:20:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.179+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb8073990 con 0x7fcbb815ba80 2024-04-25T10:20:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.180+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcba401c5a0 con 0x7fcbb815ba80 2024-04-25T10:20:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.181+0000 7fcbb5600640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb9806b4d0 0x7fcb9806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.181+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7fcba408a850 con 0x7fcbb815ba80 2024-04-25T10:20:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.182+0000 7fcbbd600640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb9806b4d0 0x7fcb9806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.182+0000 7fcbbd600640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb9806b4d0 0x7fcb9806d990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcba8033000 tx=0x7fcba8008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.186+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcba4018030 con 0x7fcbb815ba80 2024-04-25T10:20:32.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.423+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcbb80b6520 con 0x7fcbb815ba80 2024-04-25T10:20:32.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.424+0000 7fcbb5600640 1 -- 172.21.15.165:0/2394723541 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 132 v132) v1 ==== 95+0+3718 (secure 0 0 0) 0x7fcba4059150 con 0x7fcbb815ba80 2024-04-25T10:20:32.427 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:32.428 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":132,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6833/2345681563","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":2345681563},{"type":"v1","addr":"172.21.15.172:6833","nonce":2345681563}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":6326,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1853715425","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6834","nonce":1853715425},{"type":"v1","addr":"172.21.15.172:6835","nonce":1853715425}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":125},{"gid":11205,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/4032276279","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":4032276279},{"type":"v1","addr":"172.21.15.165:6836","nonce":4032276279}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":124},{"gid":11535,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/1665333785","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":1665333785},{"type":"v1","addr":"172.21.15.165:6837","nonce":1665333785}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":132}],"filesystems":[]} 2024-04-25T10:20:32.428 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 132 2024-04-25T10:20:32.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb9806b4d0 msgr2=0x7fcb9806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb9806b4d0 0x7fcb9806d990 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fcba8033000 tx=0x7fcba8008030 comp rx=0 tx=0).stop 2024-04-25T10:20:32.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 msgr2=0x7fcbb80b02a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 0x7fcbb80b02a0 secure :-1 s=READY pgs=2698 cs=0 l=1 rev1=1 crypto rx=0x7fcba40108d0 tx=0x7fcba4010d90 comp rx=0 tx=0).stop 2024-04-25T10:20:32.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 shutdown_connections 2024-04-25T10:20:32.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcb9806b4d0 0x7fcb9806d990 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcbb815e370 0x7fcbb815e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbb815ba80 0x7fcbb80b02a0 unknown :-1 s=CLOSED pgs=2698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 --2- 172.21.15.165:0/2394723541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcb9c0039b0 0x7fcbb80addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.426+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 >> 172.21.15.165:0/2394723541 conn(0x7fcbb8061c30 msgr2=0x7fcbb80d4360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.427+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 shutdown_connections 2024-04-25T10:20:32.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.427+0000 7fcbc0000640 1 -- 172.21.15.165:0/2394723541 wait complete. 2024-04-25T10:20:32.449 INFO:tasks.ceph.mds.a:Restarting daemon 2024-04-25T10:20:32.449 INFO:tasks.ceph.mds.a:Stopping old one... 2024-04-25T10:20:32.450 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:20:32.450 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:32.450 INFO:tasks.ceph.mds.c:Restarting daemon 2024-04-25T10:20:32.451 INFO:tasks.ceph.mds.c:Stopping old one... 2024-04-25T10:20:32.451 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:20:32.451 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:32.452 INFO:tasks.ceph.mds.b:Restarting daemon 2024-04-25T10:20:32.452 INFO:tasks.ceph.mds.b:Stopping old one... 2024-04-25T10:20:32.452 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:20:32.452 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:32.453 INFO:tasks.ceph.mds.d:Restarting daemon 2024-04-25T10:20:32.453 INFO:tasks.ceph.mds.d:Stopping old one... 2024-04-25T10:20:32.453 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:20:32.454 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:20:32.473 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:20:32.473 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-04-25T10:20:32.474 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:20:32.475 DEBUG:teuthology.orchestra.run.smithi172:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-04-25T10:20:32.476 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:20:32.477 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-04-25T10:20:32.478 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:20:32.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-04-25T10:20:32.520 INFO:tasks.ceph.mds.b:Started 2024-04-25T10:20:32.520 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:32.522 INFO:tasks.ceph.mds.a:Started 2024-04-25T10:20:32.523 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:32.525 INFO:tasks.ceph.mds.d:Started 2024-04-25T10:20:32.525 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:32.526 INFO:tasks.ceph.mds.c:Started 2024-04-25T10:20:32.527 DEBUG:teuthology.parallel:result is None 2024-04-25T10:20:32.527 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-04-25T10:20:32.667 INFO:tasks.ceph.mds.a.smithi165.stdout:starting mds.a at 2024-04-25T10:20:32.667 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:20:32.674 INFO:tasks.ceph.mds.b.smithi172.stdout:starting mds.b at 2024-04-25T10:20:32.675 INFO:tasks.ceph.mds.d.smithi172.stdout:starting mds.d at 2024-04-25T10:20:32.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.702+0000 7f8c25400640 1 Processor -- start 2024-04-25T10:20:32.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.702+0000 7f8c25400640 1 -- start start 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.703+0000 7f8c25400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c200b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c25400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8c20144d50 0x7f8c200b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c25400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c200cccc0 con 0x7f8c201452a0 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c25400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c200b2b20 con 0x7f8c20144d50 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c25400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2006d400 con 0x7f8c201446d0 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c15e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8c20144d50 0x7f8c200b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c1f200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8c201452a0 0x7f8c200cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51936/0 (socket says 172.21.15.165:51936) 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c1f200640 1 -- 172.21.15.165:0/2327032103 learned_addr learned my addr 172.21.15.165:0/2327032103 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c1e800640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c200b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c15e00640 1 -- 172.21.15.165:0/2327032103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 msgr2=0x7f8c200b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c15e00640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c200b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c15e00640 1 -- 172.21.15.165:0/2327032103 >> v1:172.21.15.165:6789/0 conn(0x7f8c201452a0 legacy=0x7f8c200cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c15e00640 1 -- 172.21.15.165:0/2327032103 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c200cc140 con 0x7f8c20144d50 2024-04-25T10:20:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c1de00640 1 -- 172.21.15.165:0/2327032103 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529143523 0 0) 0x7f8c200cccc0 con 0x7f8c201452a0 2024-04-25T10:20:32.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.704+0000 7f8c1e800640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c200b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.705+0000 7f8c15e00640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c200b4460 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c0802a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36893c5bfded1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c1de00640 1 -- 172.21.15.165:0/2327032103 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08037070 con 0x7f8c20144d50 2024-04-25T10:20:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c1de00640 1 -- 172.21.15.165:0/2327032103 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08032440 con 0x7f8c20144d50 2024-04-25T10:20:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c1de00640 1 -- 172.21.15.165:0/2327032103 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08032810 con 0x7f8c20144d50 2024-04-25T10:20:32.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 -- 172.21.15.165:0/2327032103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 msgr2=0x7f8c200b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c200b4460 secure :-1 s=READY pgs=2705 cs=0 l=1 rev1=1 crypto rx=0x7f8c080097a0 tx=0x7f8c0802a180 comp rx=0 tx=0).stop 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 -- 172.21.15.165:0/2327032103 shutdown_connections 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c200b4460 unknown :-1 s=CLOSED pgs=2705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 --2- 172.21.15.165:0/2327032103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c200b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 -- 172.21.15.165:0/2327032103 >> 172.21.15.165:0/2327032103 conn(0x7f8c2006abe0 msgr2=0x7f8c2006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 -- 172.21.15.165:0/2327032103 shutdown_connections 2024-04-25T10:20:32.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.706+0000 7f8c25400640 1 -- 172.21.15.165:0/2327032103 wait complete. 2024-04-25T10:20:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.708+0000 7f8c25400640 1 Processor -- start 2024-04-25T10:20:32.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.708+0000 7f8c25400640 1 -- start start 2024-04-25T10:20:32.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c25400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c2013e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c25400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c2014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c25400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c201452a0 0x7f8c200ad900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c25400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200cc700 con 0x7f8c20144d50 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c25400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200b49a0 con 0x7f8c201452a0 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c25400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c200cc9e0 con 0x7f8c201446d0 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c15e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c2014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c15e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c2014ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40048/0 (socket says 172.21.15.165:40048) 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.709+0000 7f8c15e00640 1 -- 172.21.15.165:0/2935666494 learned_addr learned my addr 172.21.15.165:0/2935666494 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c1e800640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c2013e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c1f200640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c201452a0 0x7f8c200ad900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c15e00640 1 -- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 msgr2=0x7f8c2013e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c15e00640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c2013e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c15e00640 1 -- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c201452a0 msgr2=0x7f8c200ad900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c15e00640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c201452a0 0x7f8c200ad900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c15e00640 1 -- 172.21.15.165:0/2935666494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c2013e660 con 0x7f8c20144d50 2024-04-25T10:20:32.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c1f200640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c201452a0 0x7f8c200ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c15e00640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c2014ffd0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f8c08035040 tx=0x7f8c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08037040 con 0x7f8c20144d50 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c08030770 con 0x7f8c20144d50 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c08030b80 con 0x7f8c20144d50 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.710+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c200cdf10 con 0x7f8c20144d50 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.711+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c201e0d00 con 0x7f8c20144d50 2024-04-25T10:20:32.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.712+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c20073a00 con 0x7f8c20144d50 2024-04-25T10:20:32.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.713+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8c08044020 con 0x7f8c20144d50 2024-04-25T10:20:32.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.714+0000 7f8c17400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bfc06b4d0 0x7f8bfc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:32.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.714+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5335+0+0 (secure 0 0 0) 0x7f8c080aa520 con 0x7f8c20144d50 2024-04-25T10:20:32.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.714+0000 7f8c1e800640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bfc06b4d0 0x7f8bfc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:32.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.715+0000 7f8c1e800640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bfc06b4d0 0x7f8bfc06d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8c201f6e70 tx=0x7f8c10009890 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.718+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8c08032e20 con 0x7f8c20144d50 2024-04-25T10:20:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:32.929+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f8c2006efd0 con 0x7f8c20144d50 2024-04-25T10:20:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.074+0000 7f8c17400640 1 -- 172.21.15.165:0/2935666494 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v189) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8c08079510 con 0x7f8c20144d50 2024-04-25T10:20:33.077 INFO:teuthology.orchestra.run.smithi165.stderr: removed all blocklist entries 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bfc06b4d0 msgr2=0x7f8bfc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bfc06b4d0 0x7f8bfc06d990 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f8c201f6e70 tx=0x7f8c10009890 comp rx=0 tx=0).stop 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 msgr2=0x7f8c2014ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c2014ffd0 secure :-1 s=READY pgs=2706 cs=0 l=1 rev1=1 crypto rx=0x7f8c08035040 tx=0x7f8c080046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 shutdown_connections 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8bfc06b4d0 0x7f8bfc06d990 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8c201452a0 0x7f8c200ad900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8c20144d50 0x7f8c2014ffd0 unknown :-1 s=CLOSED pgs=2706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 --2- 172.21.15.165:0/2935666494 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8c201446d0 0x7f8c2013e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 >> 172.21.15.165:0/2935666494 conn(0x7f8c2006abe0 msgr2=0x7f8c201408b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 shutdown_connections 2024-04-25T10:20:33.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.079+0000 7f8c25400640 1 -- 172.21.15.165:0/2935666494 wait complete. 2024-04-25T10:20:33.105 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-04-25T10:20:33.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.291+0000 7f22a9800640 1 Processor -- start 2024-04-25T10:20:33.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.291+0000 7f22a9800640 1 -- start start 2024-04-25T10:20:33.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22a4162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a9800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a4071670 con 0x7f22a415db10 2024-04-25T10:20:33.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a9800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a4071950 con 0x7f22a40ad250 2024-04-25T10:20:33.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4072280 con 0x7f22a40b3930 2024-04-25T10:20:33.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a2a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f22a415db10 0x7f22a40b3220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51948/0 (socket says 172.21.15.165:51948) 2024-04-25T10:20:33.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a2a00640 1 -- 172.21.15.165:0/2921851247 learned_addr learned my addr 172.21.15.165:0/2921851247 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a2000640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22a4162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.293+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192266854 0 0) 0x7f22a4071670 con 0x7f22a415db10 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2288003520 con 0x7f22a415db10 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2528166501 0 0) 0x7f2288003520 con 0x7f22a415db10 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 msgr2=0x7f22a4162230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22a4162230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 >> v1:172.21.15.172:6789/0 conn(0x7f22a40ad250 legacy=0x7f22a40ad6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4071c30 con 0x7f22a415db10 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2294001670 con 0x7f22a415db10 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 >> v1:172.21.15.165:6789/0 conn(0x7f22a415db10 legacy=0x7f22a40b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22880039e0 0x7f2288003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288004800 0x7f2288004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22880051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4072280 con 0x7f22880039e0 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228800bac0 con 0x7f2288004800 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.294+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228800bcc0 con 0x7f22a40b3930 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2a00640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22880039e0 0x7f2288003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2000640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288004800 0x7f2288004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2a00640 1 -- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 msgr2=0x7f22880051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a3400640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22880051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2a00640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22880051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2a00640 1 -- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288004800 msgr2=0x7f2288004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2a00640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288004800 0x7f2288004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2a00640 1 -- 172.21.15.165:0/2921851247 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288003520 con 0x7f22880039e0 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a3400640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a40b3930 0x7f22880051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:33.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.295+0000 7f22a2000640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2288004800 0x7f2288004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:33.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.296+0000 7f22a2a00640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22880039e0 0x7f2288003e20 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f229400c0e0 tx=0x7f2288005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31120f446d40d2b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.296+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294007c20 con 0x7f22880039e0 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.296+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2294005af0 con 0x7f22880039e0 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.296+0000 7f22a1600640 1 -- 172.21.15.165:0/2921851247 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2294007070 con 0x7f22880039e0 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a9800640 1 -- 172.21.15.165:0/2921851247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22880039e0 msgr2=0x7f2288003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a9800640 1 --2- 172.21.15.165:0/2921851247 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22880039e0 0x7f2288003e20 secure :-1 s=READY pgs=2708 cs=0 l=1 rev1=1 crypto rx=0x7f229400c0e0 tx=0x7f2288005830 comp rx=0 tx=0).stop 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a3400640 1 -- 172.21.15.165:0/2921851247 reap_dead start 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a9800640 1 -- 172.21.15.165:0/2921851247 shutdown_connections 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a9800640 1 -- 172.21.15.165:0/2921851247 >> 172.21.15.165:0/2921851247 conn(0x7f22a406abe0 msgr2=0x7f22a406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a9800640 1 -- 172.21.15.165:0/2921851247 shutdown_connections 2024-04-25T10:20:33.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.297+0000 7f22a9800640 1 -- 172.21.15.165:0/2921851247 wait complete. 2024-04-25T10:20:33.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.299+0000 7f22a9800640 1 Processor -- start 2024-04-25T10:20:33.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.299+0000 7f22a9800640 1 -- start start 2024-04-25T10:20:33.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a9800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22880039e0 0x7f22a4156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 0x7f22a414a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a9800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a41f6f10 0x7f22a41f7390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a9800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40712b0 con 0x7f22a40cc140 2024-04-25T10:20:33.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a9800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40af0e0 con 0x7f22880039e0 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a9800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a4071430 con 0x7f22a41f6f10 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 0x7f22a414a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 0x7f22a414a250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40056/0 (socket says 172.21.15.165:40056) 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a2000640 1 -- 172.21.15.165:0/1826546587 learned_addr learned my addr 172.21.15.165:0/1826546587 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a2a00640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22880039e0 0x7f22a4156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.300+0000 7f22a3400640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a41f6f10 0x7f22a41f7390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2000640 1 -- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a41f6f10 msgr2=0x7f22a41f7390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2000640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a41f6f10 0x7f22a41f7390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2000640 1 -- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22880039e0 msgr2=0x7f22a4156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2000640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22880039e0 0x7f22a4156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2000640 1 -- 172.21.15.165:0/1826546587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40cc5c0 con 0x7f22a40cc140 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a3400640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a41f6f10 0x7f22a41f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2a00640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22880039e0 0x7f22a4156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:33.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a2000640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 0x7f22a414a250 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f22900108b0 tx=0x7f2290010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229000f040 con 0x7f22a40cc140 2024-04-25T10:20:33.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229000eb60 con 0x7f22a40cc140 2024-04-25T10:20:33.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229001c400 con 0x7f22a40cc140 2024-04-25T10:20:33.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a4156dc0 con 0x7f22a40cc140 2024-04-25T10:20:33.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.301+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a4154250 con 0x7f22a40cc140 2024-04-25T10:20:33.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.303+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a4073a00 con 0x7f22a40cc140 2024-04-25T10:20:33.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.303+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f229001c5a0 con 0x7f22a40cc140 2024-04-25T10:20:33.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.304+0000 7f229aa00640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f228406b500 0x7f228406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.305+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f229008a680 con 0x7f22a40cc140 2024-04-25T10:20:33.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.305+0000 7f22a2a00640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f228406b500 0x7f228406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.305+0000 7f22a2a00640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f228406b500 0x7f228406d9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2294032000 tx=0x7f2294009030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.309+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2290018030 con 0x7f22a40cc140 2024-04-25T10:20:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.578+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f22a40cbe80 con 0x7f22a40cc140 2024-04-25T10:20:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.579+0000 7f229aa00640 1 -- 172.21.15.165:0/1826546587 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v25) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2290059620 con 0x7f22a40cc140 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "auth_dump": [ 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.a", 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmByMBNxAAzBpGt+EwcqZywECXrB2qcQ==", 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.582 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.b", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmK4JaGBAAZu6FAL+d+d03cgrU5FrN2g==", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.583 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.c", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmC8xYChAA18kRnNOlta2XryjCqahQ3w==", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mds.d", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmf3J6IxAAG2rf6PeUf+ZYjBXo0ciBKA==", 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.584 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.0", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDgKSpmIautJRAA5JK+YKWUPnCFV8YpP6AYNQ==", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.585 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.1", 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDkKSpmOyMPAxAAjeLsMrw8a3QeX7DhktLsbw==", 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.2", 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDnKSpmnSYuDhAAlx+Xh32R06qo31BMSkvU3g==", 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.586 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.3", 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDqKSpmKsDsGBAAh4JFbn1ymZoRRfc87WuLUA==", 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.587 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.4", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDwKSpmJ3zCNRAAOcbz4YP+uWvyCfI5O7JBnQ==", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.5", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD0KSpm5HKMBRAAZyGVp4Tvbw1U/WwD745h2g==", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.588 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.6", 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD3KSpmTzghERAAKLUabELDelMaWd5S27ADhA==", 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "osd.7", 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQD6KSpmDWtPKxAAoAyC5HmcGEW7hmUcV3hEjw==", 2024-04-25T10:20:33.589 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow profile osd", 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile osd", 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.0", 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw==", 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow r", 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.590 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.1", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmbEQHBxAAY7N+1widlZDNlcuRUv67fw==", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.2", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDcKSpm0qdgOBAAe/aqx35+83g51LB6TFUeeg==", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.591 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.3", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDdKSpmYIKGDxAAuh3tBQhRkhYiqg0f3dGiNQ==", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow r", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow rw", 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow rwx" 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.592 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "client.admin", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDaKSpmK/D2NBAAlVYkydxZC99bQds5asowOw==", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "mgr": "allow *", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow *", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.x", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmo6uhLBAA0GNnj29IDqQmQwEW7cgnOA==", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:20:33.593 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: }, 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "entity": "mgr.y", 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "key": "AQDbKSpmzMqfJBAA1bAf4mUclcK/8VevxTbVlw==", 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "caps": { 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "mds": "allow *", 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "mon": "allow profile mgr", 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: "osd": "allow *" 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:33.594 INFO:teuthology.orchestra.run.smithi165.stdout: ] 2024-04-25T10:20:33.595 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f228406b500 msgr2=0x7f228406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f228406b500 0x7f228406d9c0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2294032000 tx=0x7f2294009030 comp rx=0 tx=0).stop 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 msgr2=0x7f22a414a250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 0x7f22a414a250 secure :-1 s=READY pgs=2709 cs=0 l=1 rev1=1 crypto rx=0x7f22900108b0 tx=0x7f2290010d70 comp rx=0 tx=0).stop 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 shutdown_connections 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f228406b500 0x7f228406d9c0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f22a41f6f10 0x7f22a41f7390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f22a40cc140 0x7f22a414a250 unknown :-1 s=CLOSED pgs=2709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 --2- 172.21.15.165:0/1826546587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f22880039e0 0x7f22a4156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 >> 172.21.15.165:0/1826546587 conn(0x7f22a406abe0 msgr2=0x7f22a415f440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 shutdown_connections 2024-04-25T10:20:33.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.582+0000 7f22a9800640 1 -- 172.21.15.165:0/1826546587 wait complete. 2024-04-25T10:20:33.603 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:33.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.802+0000 7fa184600640 1 Processor -- start 2024-04-25T10:20:33.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.803+0000 7fa184600640 1 -- start start 2024-04-25T10:20:33.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.804+0000 7fa184600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa17c0ce920 0x7fa17c0cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.804+0000 7fa184600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c144b30 con 0x7fa17c144750 2024-04-25T10:20:33.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.804+0000 7fa184600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa17c0712d0 con 0x7fa17c06d290 2024-04-25T10:20:33.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.804+0000 7fa184600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa17c0715b0 con 0x7fa17c0ce920 2024-04-25T10:20:33.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.804+0000 7fa181c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fa17c144750 0x7fa17c06e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51956/0 (socket says 172.21.15.165:51956) 2024-04-25T10:20:33.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.804+0000 7fa181c00640 1 -- 172.21.15.165:0/617442928 learned_addr learned my addr 172.21.15.165:0/617442928 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa181200640 1 --2- 172.21.15.165:0/617442928 >> v2:172.21.15.172:3300/0 conn(0x7fa17c0ce920 0x7fa17c0cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300356068 0 0) 0x7fa17c144b30 con 0x7fa17c144750 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa164003520 con 0x7fa17c144750 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1256179040 0 0) 0x7fa164003520 con 0x7fa17c144750 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 >> v2:172.21.15.172:3300/0 conn(0x7fa17c0ce920 msgr2=0x7fa17c0cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 --2- 172.21.15.165:0/617442928 >> v2:172.21.15.172:3300/0 conn(0x7fa17c0ce920 0x7fa17c0cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 >> v1:172.21.15.172:6789/0 conn(0x7fa17c06d290 legacy=0x7fa17c06d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c071890 con 0x7fa17c144750 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fa170001670 con 0x7fa17c144750 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 >> v1:172.21.15.165:6789/0 conn(0x7fa17c144750 legacy=0x7fa17c06e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640039b0 0x7fa164003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa164004800 0x7fa164004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1640051c0 0x7fa16400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c0715b0 con 0x7fa1640051c0 2024-04-25T10:20:33.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400c060 con 0x7fa1640039b0 2024-04-25T10:20:33.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.805+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa16400c260 con 0x7fa164004800 2024-04-25T10:20:33.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa182600640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1640051c0 0x7fa16400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa181200640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa164004800 0x7fa164004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa182600640 1 -- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa164004800 msgr2=0x7fa164004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa181c00640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640039b0 0x7fa164003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa182600640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa164004800 0x7fa164004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa182600640 1 -- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640039b0 msgr2=0x7fa164003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:33.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa182600640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640039b0 0x7fa164003dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.806+0000 7fa182600640 1 -- 172.21.15.165:0/617442928 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa164003520 con 0x7fa1640051c0 2024-04-25T10:20:33.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.807+0000 7fa182600640 1 -- 172.21.15.165:0/617442928 reap_dead start 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.807+0000 7fa182600640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1640051c0 0x7fa16400b9c0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fa174001e50 tx=0x7fa164005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c9dc6dca26008e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.807+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174001a70 con 0x7fa1640051c0 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.807+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174003970 con 0x7fa1640051c0 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.807+0000 7fa17be00640 1 -- 172.21.15.165:0/617442928 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174003d60 con 0x7fa1640051c0 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 -- 172.21.15.165:0/617442928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1640051c0 msgr2=0x7fa16400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1640051c0 0x7fa16400b9c0 secure :-1 s=READY pgs=2711 cs=0 l=1 rev1=1 crypto rx=0x7fa174001e50 tx=0x7fa164005830 comp rx=0 tx=0).stop 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 -- 172.21.15.165:0/617442928 shutdown_connections 2024-04-25T10:20:33.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 --2- 172.21.15.165:0/617442928 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa1640051c0 0x7fa16400b9c0 unknown :-1 s=CLOSED pgs=2711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 -- 172.21.15.165:0/617442928 >> 172.21.15.165:0/617442928 conn(0x7fa17c06abe0 msgr2=0x7fa17c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 -- 172.21.15.165:0/617442928 shutdown_connections 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.808+0000 7fa184600640 1 -- 172.21.15.165:0/617442928 wait complete. 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.810+0000 7fa184600640 1 Processor -- start 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.810+0000 7fa184600640 1 -- start start 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa184600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640051c0 0x7fa17c0adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa184600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa17c0b2ae0 0x7fa17c1572f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa184600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 0x7fa17c13d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa184600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1455d0 con 0x7fa17c13d0f0 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa184600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c145450 con 0x7fa1640051c0 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa184600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c15c000 con 0x7fa17c0b2ae0 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa182600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 0x7fa17c13d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa182600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 0x7fa17c13d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40086/0 (socket says 172.21.15.165:40086) 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.811+0000 7fa182600640 1 -- 172.21.15.165:0/80045432 learned_addr learned my addr 172.21.15.165:0/80045432 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:33.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa181c00640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640051c0 0x7fa17c0adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa182600640 1 -- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa17c0b2ae0 msgr2=0x7fa17c1572f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa182600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa17c0b2ae0 0x7fa17c1572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa182600640 1 -- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640051c0 msgr2=0x7fa17c0adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa182600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640051c0 0x7fa17c0adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa182600640 1 -- 172.21.15.165:0/80045432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa17c0b3050 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa182600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 0x7fa17c13d570 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fa174005040 tx=0x7fa1740014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174007db0 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.812+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa174001250 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.813+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c0b3330 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.813+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa174006b50 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.813+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c0b38f0 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.814+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c13dab0 con 0x7fa17c13d0f0 2024-04-25T10:20:33.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.815+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa174006cf0 con 0x7fa17c13d0f0 2024-04-25T10:20:33.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.816+0000 7fa179600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa15c06b500 0x7fa15c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:33.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.816+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fa1740a96e0 con 0x7fa17c13d0f0 2024-04-25T10:20:33.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.816+0000 7fa181c00640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa15c06b500 0x7fa15c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:33.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.817+0000 7fa181c00640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa15c06b500 0x7fa15c06d9c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa170001920 tx=0x7fa170009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:33.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:33.820+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa174078720 con 0x7fa17c13d0f0 2024-04-25T10:20:34.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.018+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa17c0b3cd0 con 0x7fa17c13d0f0 2024-04-25T10:20:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.019+0000 7fa179600640 1 -- 172.21.15.165:0/80045432 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+13210 (secure 0 0 0) 0x7fa174078020 con 0x7fa17c13d0f0 2024-04-25T10:20:34.022 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:34.023 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":191,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:33.086786+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":28,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":182,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa15c06b500 msgr2=0x7fa15c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa15c06b500 0x7fa15c06d9c0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa170001920 tx=0x7fa170009040 comp rx=0 tx=0).stop 2024-04-25T10:20:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 msgr2=0x7fa17c13d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 0x7fa17c13d570 secure :-1 s=READY pgs=2712 cs=0 l=1 rev1=1 crypto rx=0x7fa174005040 tx=0x7fa1740014a0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 shutdown_connections 2024-04-25T10:20:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa15c06b500 0x7fa15c06d9c0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa17c13d0f0 0x7fa17c13d570 unknown :-1 s=CLOSED pgs=2712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa17c0b2ae0 0x7fa17c1572f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 --2- 172.21.15.165:0/80045432 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa1640051c0 0x7fa17c0adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 >> 172.21.15.165:0/80045432 conn(0x7fa17c06abe0 msgr2=0x7fa17c06e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 shutdown_connections 2024-04-25T10:20:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.022+0000 7fa184600640 1 -- 172.21.15.165:0/80045432 wait complete. 2024-04-25T10:20:34.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:34.045 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.240+0000 7f86a4e00640 1 Processor -- start 2024-04-25T10:20:34.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.240+0000 7f86a4e00640 1 -- start start 2024-04-25T10:20:34.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.241+0000 7f86a4e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f86a015dd30 0x7f86a00cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.241+0000 7f86a4e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f86a006bff0 0x7f86a00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.241+0000 7f86a4e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86a015e110 con 0x7f86a00b5970 2024-04-25T10:20:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.241+0000 7f86a4e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a006d000 con 0x7f86a006bff0 2024-04-25T10:20:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.241+0000 7f86a4e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86a00b2fd0 con 0x7f86a015dd30 2024-04-25T10:20:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f86a006bff0 0x7f86a00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f86a006bff0 0x7f86a00b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40090/0 (socket says 172.21.15.165:40090) 2024-04-25T10:20:34.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 -- 172.21.15.165:0/4208498642 learned_addr learned my addr 172.21.15.165:0/4208498642 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:34.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f869f400640 1 --2- 172.21.15.165:0/4208498642 >> v2:172.21.15.172:3300/0 conn(0x7f86a015dd30 0x7f86a00cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 -- 172.21.15.165:0/4208498642 >> v2:172.21.15.172:3300/0 conn(0x7f86a015dd30 msgr2=0x7f86a00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 --2- 172.21.15.165:0/4208498642 >> v2:172.21.15.172:3300/0 conn(0x7f86a015dd30 0x7f86a00cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 -- 172.21.15.165:0/4208498642 >> v1:172.21.15.172:6789/0 conn(0x7f86a00b5970 legacy=0x7f86a0161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f8697e00640 1 -- 172.21.15.165:0/4208498642 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a006c480 con 0x7f86a006bff0 2024-04-25T10:20:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.242+0000 7f869f400640 1 --2- 172.21.15.165:0/4208498642 >> v2:172.21.15.172:3300/0 conn(0x7f86a015dd30 0x7f86a00cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.243+0000 7f8697e00640 1 --2- 172.21.15.165:0/4208498642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a006bff0 0x7f86a00b5430 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f86a008bfe0 tx=0x7f868802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7086f55b96bf9cc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f869ea00640 1 -- 172.21.15.165:0/4208498642 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688037070 con 0x7f86a006bff0 2024-04-25T10:20:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f869ea00640 1 -- 172.21.15.165:0/4208498642 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8688032440 con 0x7f86a006bff0 2024-04-25T10:20:34.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f869ea00640 1 -- 172.21.15.165:0/4208498642 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688032810 con 0x7f86a006bff0 2024-04-25T10:20:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f86a4e00640 1 -- 172.21.15.165:0/4208498642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a006bff0 msgr2=0x7f86a00b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f86a4e00640 1 --2- 172.21.15.165:0/4208498642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a006bff0 0x7f86a00b5430 secure :-1 s=READY pgs=2713 cs=0 l=1 rev1=1 crypto rx=0x7f86a008bfe0 tx=0x7f868802a230 comp rx=0 tx=0).stop 2024-04-25T10:20:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f86a4e00640 1 -- 172.21.15.165:0/4208498642 shutdown_connections 2024-04-25T10:20:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f86a4e00640 1 --2- 172.21.15.165:0/4208498642 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a006bff0 0x7f86a00b5430 unknown :-1 s=CLOSED pgs=2713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f86a4e00640 1 --2- 172.21.15.165:0/4208498642 >> v2:172.21.15.172:3300/0 conn(0x7f86a015dd30 0x7f86a00cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.244+0000 7f86a4e00640 1 -- 172.21.15.165:0/4208498642 >> 172.21.15.165:0/4208498642 conn(0x7f86a00d0610 msgr2=0x7f86a00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.245+0000 7f86a4e00640 1 -- 172.21.15.165:0/4208498642 shutdown_connections 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.245+0000 7f86a4e00640 1 -- 172.21.15.165:0/4208498642 wait complete. 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.246+0000 7f86a4e00640 1 Processor -- start 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.246+0000 7f86a4e00640 1 -- start start 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.247+0000 7f86a4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 0x7f86a0160b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.247+0000 7f86a4e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86a015dd30 0x7f86a01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.247+0000 7f86a4e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86a0159b00 0x7f86a00aa400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.247+0000 7f86a4e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a006cd20 con 0x7f86a00b5970 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.247+0000 7f86a4e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86a006ca40 con 0x7f86a015dd30 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.247+0000 7f86a4e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86a00cf110 con 0x7f86a0159b00 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 0x7f86a0160b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 0x7f86a0160b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40102/0 (socket says 172.21.15.165:40102) 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 -- 172.21.15.165:0/121402711 learned_addr learned my addr 172.21.15.165:0/121402711 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:34.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 -- 172.21.15.165:0/121402711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86a0159b00 msgr2=0x7f86a00aa400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86a0159b00 0x7f86a00aa400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 -- 172.21.15.165:0/121402711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86a015dd30 msgr2=0x7f86a01595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86a015dd30 0x7f86a01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 -- 172.21.15.165:0/121402711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a015eed0 con 0x7f86a00b5970 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.248+0000 7f869f400640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 0x7f86a0160b30 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f869000c820 tx=0x7f869000cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.249+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690015070 con 0x7f86a00b5970 2024-04-25T10:20:34.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.249+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86900079d0 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.249+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8690007da0 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.249+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86a015f1b0 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.249+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86a015f770 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.250+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86a0073990 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.251+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8690018070 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.252+0000 7f8697400640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f867c06b4d0 0x7f867c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.252+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f8690089fc0 con 0x7f86a00b5970 2024-04-25T10:20:34.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.252+0000 7f8697e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f867c06b4d0 0x7f867c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.253+0000 7f8697e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f867c06b4d0 0x7f867c06d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8688034000 tx=0x7f868802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.257+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8690058f60 con 0x7f86a00b5970 2024-04-25T10:20:34.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.438+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f86a00d16c0 con 0x7f86a00b5970 2024-04-25T10:20:34.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.439+0000 7f8697400640 1 -- 172.21.15.165:0/121402711 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v191) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8690058860 con 0x7f86a00b5970 2024-04-25T10:20:34.442 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:20:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.441+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f867c06b4d0 msgr2=0x7f867c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.441+0000 7f86a4e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f867c06b4d0 0x7f867c06d990 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f8688034000 tx=0x7f868802c000 comp rx=0 tx=0).stop 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 msgr2=0x7f86a0160b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 0x7f86a0160b30 secure :-1 s=READY pgs=2714 cs=0 l=1 rev1=1 crypto rx=0x7f869000c820 tx=0x7f869000cce0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 shutdown_connections 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f867c06b4d0 0x7f867c06d990 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f86a0159b00 0x7f86a00aa400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f86a015dd30 0x7f86a01595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 --2- 172.21.15.165:0/121402711 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f86a00b5970 0x7f86a0160b30 unknown :-1 s=CLOSED pgs=2714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 >> 172.21.15.165:0/121402711 conn(0x7f86a00d0610 msgr2=0x7f86a00b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 shutdown_connections 2024-04-25T10:20:34.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.442+0000 7f86a4e00640 1 -- 172.21.15.165:0/121402711 wait complete. 2024-04-25T10:20:34.462 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-25T10:20:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.659+0000 7fcddfe00640 1 Processor -- start 2024-04-25T10:20:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.659+0000 7fcddfe00640 1 -- start start 2024-04-25T10:20:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00ceb80 0x7fcde00cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcddfe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcde00b96e0 0x7fcde006c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcddfe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcde00d2730 con 0x7fcde00d1530 2024-04-25T10:20:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcddfe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcde00d2a10 con 0x7fcde00b96e0 2024-04-25T10:20:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcddfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00d2cf0 con 0x7fcde00ceb80 2024-04-25T10:20:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcde00b96e0 0x7fcde006c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fcde00b96e0 0x7fcde006c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40104/0 (socket says 172.21.15.165:40104) 2024-04-25T10:20:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 -- 172.21.15.165:0/831369640 learned_addr learned my addr 172.21.15.165:0/831369640 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcddea00640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00ceb80 0x7fcde00cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 -- 172.21.15.165:0/831369640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00ceb80 msgr2=0x7fcde00cef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00ceb80 0x7fcde00cef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 -- 172.21.15.165:0/831369640 >> v1:172.21.15.172:6789/0 conn(0x7fcde00d1530 legacy=0x7fcde006c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.661+0000 7fcdde000640 1 -- 172.21.15.165:0/831369640 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00d2fd0 con 0x7fcde00b96e0 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.662+0000 7fcddea00640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00ceb80 0x7fcde00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.662+0000 7fcdde000640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde006c110 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009e50 tx=0x7fcdc802a640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=795cefeda3a5e988 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.663+0000 7fcddd600640 1 -- 172.21.15.165:0/831369640 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc8036070 con 0x7fcde00b96e0 2024-04-25T10:20:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.663+0000 7fcddd600640 1 -- 172.21.15.165:0/831369640 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc8031600 con 0x7fcde00b96e0 2024-04-25T10:20:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.663+0000 7fcddd600640 1 -- 172.21.15.165:0/831369640 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc80319d0 con 0x7fcde00b96e0 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.663+0000 7fcddfe00640 1 -- 172.21.15.165:0/831369640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 msgr2=0x7fcde006c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.663+0000 7fcddfe00640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde006c110 secure :-1 s=READY pgs=2715 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8009e50 tx=0x7fcdc802a640 comp rx=0 tx=0).stop 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.664+0000 7fcddfe00640 1 -- 172.21.15.165:0/831369640 shutdown_connections 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.664+0000 7fcddfe00640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde006c110 unknown :-1 s=CLOSED pgs=2715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.664+0000 7fcddfe00640 1 --2- 172.21.15.165:0/831369640 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00ceb80 0x7fcde00cef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.664+0000 7fcddfe00640 1 -- 172.21.15.165:0/831369640 >> 172.21.15.165:0/831369640 conn(0x7fcde00d0470 msgr2=0x7fcde00d0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.664+0000 7fcddfe00640 1 -- 172.21.15.165:0/831369640 shutdown_connections 2024-04-25T10:20:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.664+0000 7fcddfe00640 1 -- 172.21.15.165:0/831369640 wait complete. 2024-04-25T10:20:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.665+0000 7fcddfe00640 1 Processor -- start 2024-04-25T10:20:34.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.666+0000 7fcddfe00640 1 -- start start 2024-04-25T10:20:34.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.666+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde01f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.666+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde00ceb80 0x7fcde01e6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddfe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00d1530 0x7fcde01e6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddfe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00cf4a0 con 0x7fcde00b96e0 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddfe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde00d2420 con 0x7fcde00ceb80 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddfe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde014c6e0 con 0x7fcde00d1530 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde01f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde01f3210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40108/0 (socket says 172.21.15.165:40108) 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 -- 172.21.15.165:0/656303611 learned_addr learned my addr 172.21.15.165:0/656303611 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 -- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00d1530 msgr2=0x7fcde01e6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcdde000640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde00ceb80 0x7fcde01e6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddf400640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00d1530 0x7fcde01e6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00d1530 0x7fcde01e6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 -- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde00ceb80 msgr2=0x7fcde01e6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde00ceb80 0x7fcde01e6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddea00640 1 -- 172.21.15.165:0/656303611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00ca1d0 con 0x7fcde00b96e0 2024-04-25T10:20:34.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.667+0000 7fcddf400640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00d1530 0x7fcde01e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.668+0000 7fcddea00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde01f3210 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000c7f0 tx=0x7fcdd000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.668+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0015070 con 0x7fcde00b96e0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.668+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd0007a30 con 0x7fcde00b96e0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.668+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcde01e4250 con 0x7fcde00b96e0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.668+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd0007e00 con 0x7fcde00b96e0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.668+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde01f3750 con 0x7fcde00b96e0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.669+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcde0073990 con 0x7fcde00b96e0 2024-04-25T10:20:34.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.670+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcdd0018070 con 0x7fcde00b96e0 2024-04-25T10:20:34.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.671+0000 7fcdd7400640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:34.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.671+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7fcdd0089f60 con 0x7fcde00b96e0 2024-04-25T10:20:34.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.671+0000 7fcdde000640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:34.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.672+0000 7fcdde000640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8033000 tx=0x7fcdc8009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:34.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.675+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fcdd0059030 con 0x7fcde00b96e0 2024-04-25T10:20:34.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.856+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fcde00ce370 con 0x7fcde00b96e0 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.857+0000 7fcdd7400640 1 -- 172.21.15.165:0/656303611 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v191) v1 ==== 84+0+65 (secure 0 0 0) 0x7fcdd0058930 con 0x7fcde00b96e0 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi165.stdout:[ 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi165.stdout: { 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolnum": 1, 2024-04-25T10:20:34.860 INFO:teuthology.orchestra.run.smithi165.stdout: "poolname": ".mgr" 2024-04-25T10:20:34.861 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:34.861 INFO:teuthology.orchestra.run.smithi165.stdout:] 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdbc06b4d0 msgr2=0x7fcdbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fcdc8033000 tx=0x7fcdc8009000 comp rx=0 tx=0).stop 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 msgr2=0x7fcde01f3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde01f3210 secure :-1 s=READY pgs=2716 cs=0 l=1 rev1=1 crypto rx=0x7fcdd000c7f0 tx=0x7fcdd000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 shutdown_connections 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcdbc06b4d0 0x7fcdbc06d990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcde00d1530 0x7fcde01e6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcde00ceb80 0x7fcde01e6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 --2- 172.21.15.165:0/656303611 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcde00b96e0 0x7fcde01f3210 unknown :-1 s=CLOSED pgs=2716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:34.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 >> 172.21.15.165:0/656303611 conn(0x7fcde00d0470 msgr2=0x7fcde00b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:34.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 shutdown_connections 2024-04-25T10:20:34.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:34.860+0000 7fcddfe00640 1 -- 172.21.15.165:0/656303611 wait complete. 2024-04-25T10:20:34.882 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-25T10:20:34.882 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-25T10:20:35.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.083+0000 7f2b17000640 1 Processor -- start 2024-04-25T10:20:35.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.084+0000 7f2b17000640 1 -- start start 2024-04-25T10:20:35.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b17000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 0x7f2b100cc230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b17000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b100cb190 con 0x7f2b100ccdd0 2024-04-25T10:20:35.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b17000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b100cb470 con 0x7f2b100d00e0 2024-04-25T10:20:35.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b17000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100cb750 con 0x7f2b1015bb20 2024-04-25T10:20:35.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b07e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2b100ccdd0 0x7f2b100cf9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:51960/0 (socket says 172.21.15.165:51960) 2024-04-25T10:20:35.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b07e00640 1 -- 172.21.15.165:0/1883538958 learned_addr learned my addr 172.21.15.165:0/1883538958 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b0fe00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 0x7f2b100cc230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.086+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416595241 0 0) 0x7f2b100cb190 con 0x7f2b100ccdd0 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2af4003520 con 0x7f2b100ccdd0 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 28227639 0 0) 0x7f2af4003520 con 0x7f2b100ccdd0 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 msgr2=0x7f2b100cc230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 0x7f2b100cc230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 >> v1:172.21.15.172:6789/0 conn(0x7f2b100d00e0 legacy=0x7f2b100ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b100cba30 con 0x7f2b100ccdd0 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f2af8001670 con 0x7f2b100ccdd0 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 >> v1:172.21.15.165:6789/0 conn(0x7f2b100ccdd0 legacy=0x7f2b100cf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2af40039e0 0x7f2af4003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2af4004800 0x7f2af4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 0x7f2af4005270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100cb750 con 0x7f2af4004800 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af400bb50 con 0x7f2af40039e0 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af400bd50 con 0x7f2b1015bb20 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.087+0000 7f2b07e00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2af4004800 0x7f2af4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b15000640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 0x7f2af4005270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b0fe00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2af40039e0 0x7f2af4003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b07e00640 1 -- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 msgr2=0x7f2af4005270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b07e00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2b1015bb20 0x7f2af4005270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b07e00640 1 -- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2af40039e0 msgr2=0x7f2af4003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b07e00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2af40039e0 0x7f2af4003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b07e00640 1 -- 172.21.15.165:0/1883538958 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4003520 con 0x7f2af4004800 2024-04-25T10:20:35.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.088+0000 7f2b0fe00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2af40039e0 0x7f2af4003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.089+0000 7f2b07e00640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2af4004800 0x7f2af4004c80 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c110 tx=0x7f2af4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5deb0e8e6e6687a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.089+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8007c40 con 0x7f2af4004800 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.089+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2af8005af0 con 0x7f2af4004800 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.089+0000 7f2b0f400640 1 -- 172.21.15.165:0/1883538958 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2af8007070 con 0x7f2af4004800 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b17000640 1 -- 172.21.15.165:0/1883538958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2af4004800 msgr2=0x7f2af4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b17000640 1 --2- 172.21.15.165:0/1883538958 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2af4004800 0x7f2af4004c80 secure :-1 s=READY pgs=2718 cs=0 l=1 rev1=1 crypto rx=0x7f2af800c110 tx=0x7f2af4005830 comp rx=0 tx=0).stop 2024-04-25T10:20:35.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b15000640 1 -- 172.21.15.165:0/1883538958 reap_dead start 2024-04-25T10:20:35.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b17000640 1 -- 172.21.15.165:0/1883538958 shutdown_connections 2024-04-25T10:20:35.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b17000640 1 -- 172.21.15.165:0/1883538958 >> 172.21.15.165:0/1883538958 conn(0x7f2b100b2390 msgr2=0x7f2b100b27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:35.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b17000640 1 -- 172.21.15.165:0/1883538958 shutdown_connections 2024-04-25T10:20:35.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.090+0000 7f2b17000640 1 -- 172.21.15.165:0/1883538958 wait complete. 2024-04-25T10:20:35.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.092+0000 7f2b17000640 1 Processor -- start 2024-04-25T10:20:35.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.092+0000 7f2b17000640 1 -- start start 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b17000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2af4004800 0x7f2b100c8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b17000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b100ca600 0x7f2b1014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b17000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 0x7f2b10159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b17000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1015c080 con 0x7f2b100c62c0 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b17000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1015bf00 con 0x7f2b100ca600 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b17000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b100cadb0 con 0x7f2af4004800 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 0x7f2b10159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 0x7f2b10159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40122/0 (socket says 172.21.15.165:40122) 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 -- 172.21.15.165:0/4160212496 learned_addr learned my addr 172.21.15.165:0/4160212496 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b07e00640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b100ca600 0x7f2b1014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b0fe00640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2af4004800 0x7f2b100c8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 -- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2af4004800 msgr2=0x7f2b100c8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2af4004800 0x7f2b100c8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 -- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b100ca600 msgr2=0x7f2b1014d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b100ca600 0x7f2b1014d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b15000640 1 -- 172.21.15.165:0/4160212496 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1014d820 con 0x7f2b100c62c0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.093+0000 7f2b07e00640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b100ca600 0x7f2b1014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.094+0000 7f2b15000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 0x7f2b10159ae0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f2b1015f6a0 tx=0x7f2b0800ea00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.094+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b08003b80 con 0x7f2b100c62c0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.094+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b08018070 con 0x7f2b100c62c0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.094+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b080050d0 con 0x7f2b100c62c0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.094+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b100c9560 con 0x7f2b100c62c0 2024-04-25T10:20:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.094+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b100c9b20 con 0x7f2b100c62c0 2024-04-25T10:20:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.095+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b10073a00 con 0x7f2b100c62c0 2024-04-25T10:20:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.096+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2b08005270 con 0x7f2b100c62c0 2024-04-25T10:20:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.097+0000 7f2b0cc00640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2aec06b500 0x7f2aec06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.097+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5249+0+0 (secure 0 0 0) 0x7f2b08089d30 con 0x7f2b100c62c0 2024-04-25T10:20:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.097+0000 7f2b0fe00640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2aec06b500 0x7f2aec06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.098+0000 7f2b0fe00640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2aec06b500 0x7f2aec06d9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2b000099b0 tx=0x7f2b00008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:35.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.101+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2b1014d820 con 0x7f2b100c62c0 2024-04-25T10:20:35.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:35.302+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2b100c5850 con 0x7f2b100c62c0 2024-04-25T10:20:36.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.101+0000 7f2b0cc00640 1 -- 172.21.15.165:0/4160212496 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v192) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2b08058d60 con 0x7f2b100c62c0 2024-04-25T10:20:36.104 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' created 2024-04-25T10:20:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2aec06b500 msgr2=0x7f2aec06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2aec06b500 0x7f2aec06d9c0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f2b000099b0 tx=0x7f2b00008040 comp rx=0 tx=0).stop 2024-04-25T10:20:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 msgr2=0x7f2b10159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 0x7f2b10159ae0 secure :-1 s=READY pgs=2719 cs=0 l=1 rev1=1 crypto rx=0x7f2b1015f6a0 tx=0x7f2b0800ea00 comp rx=0 tx=0).stop 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 shutdown_connections 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2aec06b500 0x7f2aec06d9c0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2b100c62c0 0x7f2b10159ae0 unknown :-1 s=CLOSED pgs=2719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2b100ca600 0x7f2b1014d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 --2- 172.21.15.165:0/4160212496 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2af4004800 0x7f2b100c8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 >> 172.21.15.165:0/4160212496 conn(0x7f2b100b2390 msgr2=0x7f2b1006ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 shutdown_connections 2024-04-25T10:20:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.105+0000 7f2b17000640 1 -- 172.21.15.165:0/4160212496 wait complete. 2024-04-25T10:20:36.134 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-04-25T10:20:36.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.335+0000 7f1c62c00640 1 Processor -- start 2024-04-25T10:20:36.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.335+0000 7f1c62c00640 1 -- start start 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c62c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c62c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1c5c15b7a0 0x7f1c5c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c62c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c0b0b80 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c62c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c0cd650 con 0x7f1c5c15b7a0 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c62c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0cc800 con 0x7f1c5c15b3c0 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c60e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1c5c15b7a0 0x7f1c5c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c60e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1c5c15b7a0 0x7f1c5c0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40126/0 (socket says 172.21.15.165:40126) 2024-04-25T10:20:36.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.337+0000 7f1c60e00640 1 -- 172.21.15.165:0/843259223 learned_addr learned my addr 172.21.15.165:0/843259223 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:36.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.338+0000 7f1c61800640 1 --2- 172.21.15.165:0/843259223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.338+0000 7f1c60e00640 1 -- 172.21.15.165:0/843259223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 msgr2=0x7f1c5c0b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:36.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.338+0000 7f1c60e00640 1 --2- 172.21.15.165:0/843259223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.338+0000 7f1c60e00640 1 -- 172.21.15.165:0/843259223 >> v1:172.21.15.172:6789/0 conn(0x7f1c5c0cdae0 legacy=0x7f1c5c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:36.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.338+0000 7f1c60e00640 1 -- 172.21.15.165:0/843259223 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c0ccae0 con 0x7f1c5c15b7a0 2024-04-25T10:20:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.339+0000 7f1c60e00640 1 --2- 172.21.15.165:0/843259223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c15b7a0 0x7f1c5c0cc150 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009870 tx=0x7f1c4802a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f37ee69cd2b4cc49 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.339+0000 7f1c5be00640 1 -- 172.21.15.165:0/843259223 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48037070 con 0x7f1c5c15b7a0 2024-04-25T10:20:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.339+0000 7f1c5be00640 1 -- 172.21.15.165:0/843259223 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c48032440 con 0x7f1c5c15b7a0 2024-04-25T10:20:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.339+0000 7f1c5be00640 1 -- 172.21.15.165:0/843259223 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c48032810 con 0x7f1c5c15b7a0 2024-04-25T10:20:36.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 -- 172.21.15.165:0/843259223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c15b7a0 msgr2=0x7f1c5c0cc150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 --2- 172.21.15.165:0/843259223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c15b7a0 0x7f1c5c0cc150 secure :-1 s=READY pgs=2720 cs=0 l=1 rev1=1 crypto rx=0x7f1c48009870 tx=0x7f1c4802a210 comp rx=0 tx=0).stop 2024-04-25T10:20:36.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 -- 172.21.15.165:0/843259223 shutdown_connections 2024-04-25T10:20:36.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 --2- 172.21.15.165:0/843259223 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c15b7a0 0x7f1c5c0cc150 unknown :-1 s=CLOSED pgs=2720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 --2- 172.21.15.165:0/843259223 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c0b0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 -- 172.21.15.165:0/843259223 >> 172.21.15.165:0/843259223 conn(0x7f1c5c061cc0 msgr2=0x7f1c5c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.340+0000 7f1c62c00640 1 -- 172.21.15.165:0/843259223 shutdown_connections 2024-04-25T10:20:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.341+0000 7f1c62c00640 1 -- 172.21.15.165:0/843259223 wait complete. 2024-04-25T10:20:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.342+0000 7f1c62c00640 1 Processor -- start 2024-04-25T10:20:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.343+0000 7f1c62c00640 1 -- start start 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c62c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 0x7f1c5c1f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c62c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c62c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c5c15b7a0 0x7f1c5c158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c62c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15e8a0 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c62c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c15e720 con 0x7f1c5c15b7a0 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c62c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c0cdf40 con 0x7f1c5c15b3c0 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 0x7f1c5c1f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 0x7f1c5c1f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:40138/0 (socket says 172.21.15.165:40138) 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 -- 172.21.15.165:0/810970851 learned_addr learned my addr 172.21.15.165:0/810970851 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c60e00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 -- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 msgr2=0x7f1c5c157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 -- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c5c15b7a0 msgr2=0x7f1c5c158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:36.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c5c15b7a0 0x7f1c5c158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.344+0000 7f1c61800640 1 -- 172.21.15.165:0/810970851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c1471d0 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.345+0000 7f1c61800640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 0x7f1c5c1f48c0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c7f0 tx=0x7f1c5000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.345+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50015070 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.345+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c50004240 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.345+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c1474b0 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.345+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c50007620 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.345+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c147a70 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.346+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c073990 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.348+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c500079d0 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.349+0000 7f1c59600640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c3c06b4d0 0x7f1c3c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.349+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5619+0+0 (secure 0 0 0) 0x7f1c5008a5d0 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.349+0000 7f1c60e00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c3c06b4d0 0x7f1c3c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:36.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.350+0000 7f1c60e00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c3c06b4d0 0x7f1c3c06d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1c48034000 tx=0x7f1c4802e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:36.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.354+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c50059490 con 0x7f1c5c0cdae0 2024-04-25T10:20:36.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:36.550+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1c5c0b0750 con 0x7f1c5c0cdae0 2024-04-25T10:20:37.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.120+0000 7f1c59600640 1 -- 172.21.15.165:0/810970851 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v193) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1c50058d90 con 0x7f1c5c0cdae0 2024-04-25T10:20:37.123 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' created 2024-04-25T10:20:37.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c3c06b4d0 msgr2=0x7f1c3c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c3c06b4d0 0x7f1c3c06d990 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f1c48034000 tx=0x7f1c4802e000 comp rx=0 tx=0).stop 2024-04-25T10:20:37.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 msgr2=0x7f1c5c1f48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 0x7f1c5c1f48c0 secure :-1 s=READY pgs=2721 cs=0 l=1 rev1=1 crypto rx=0x7f1c5000c7f0 tx=0x7f1c5000ccb0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 shutdown_connections 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c3c06b4d0 0x7f1c3c06d990 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1c5c15b7a0 0x7f1c5c158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1c5c15b3c0 0x7f1c5c157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 --2- 172.21.15.165:0/810970851 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1c5c0cdae0 0x7f1c5c1f48c0 unknown :-1 s=CLOSED pgs=2721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 >> 172.21.15.165:0/810970851 conn(0x7f1c5c061cc0 msgr2=0x7f1c5c160340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 shutdown_connections 2024-04-25T10:20:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.124+0000 7f1c62c00640 1 -- 172.21.15.165:0/810970851 wait complete. 2024-04-25T10:20:37.160 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-25T10:20:37.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.337+0000 7f77da400640 1 Processor -- start 2024-04-25T10:20:37.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.337+0000 7f77da400640 1 -- start start 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.338+0000 7f77da400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f77d41446d0 0x7f77d40b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.338+0000 7f77da400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.338+0000 7f77da400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f77d41452a0 0x7f77d40cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.338+0000 7f77da400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d40b2b20 con 0x7f77d41446d0 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.338+0000 7f77da400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77d406d400 con 0x7f77d41452a0 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.338+0000 7f77da400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40b49a0 con 0x7f77d4144d50 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f77d41446d0 0x7f77d40b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f77d41446d0 0x7f77d40b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39594/0 (socket says 172.21.15.165:39594) 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 -- 172.21.15.165:0/127007901 learned_addr learned my addr 172.21.15.165:0/127007901 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:37.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3e00640 1 --2- 172.21.15.165:0/127007901 >> v2:172.21.15.172:3300/0 conn(0x7f77d41452a0 0x7f77d40cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 -- 172.21.15.165:0/127007901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 msgr2=0x7f77d40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 --2- 172.21.15.165:0/127007901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 -- 172.21.15.165:0/127007901 >> v2:172.21.15.172:3300/0 conn(0x7f77d41452a0 msgr2=0x7f77d40cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:37.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 --2- 172.21.15.165:0/127007901 >> v2:172.21.15.172:3300/0 conn(0x7f77d41452a0 0x7f77d40cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3400640 1 -- 172.21.15.165:0/127007901 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d40cccc0 con 0x7f77d41446d0 2024-04-25T10:20:37.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.339+0000 7f77d3e00640 1 --2- 172.21.15.165:0/127007901 >> v2:172.21.15.172:3300/0 conn(0x7f77d41452a0 0x7f77d40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:37.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.340+0000 7f77d3400640 1 --2- 172.21.15.165:0/127007901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41446d0 0x7f77d40b3f20 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f77d4145880 tx=0x7f77c402a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1034b84a298b75aa server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.341+0000 7f77d2000640 1 -- 172.21.15.165:0/127007901 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4037070 con 0x7f77d41446d0 2024-04-25T10:20:37.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.341+0000 7f77d2000640 1 -- 172.21.15.165:0/127007901 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77c4032440 con 0x7f77d41446d0 2024-04-25T10:20:37.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.341+0000 7f77d2000640 1 -- 172.21.15.165:0/127007901 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77c4032810 con 0x7f77d41446d0 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.341+0000 7f77da400640 1 -- 172.21.15.165:0/127007901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41446d0 msgr2=0x7f77d40b3f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.341+0000 7f77da400640 1 --2- 172.21.15.165:0/127007901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41446d0 0x7f77d40b3f20 secure :-1 s=READY pgs=2722 cs=0 l=1 rev1=1 crypto rx=0x7f77d4145880 tx=0x7f77c402a1e0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 -- 172.21.15.165:0/127007901 shutdown_connections 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 --2- 172.21.15.165:0/127007901 >> v2:172.21.15.172:3300/0 conn(0x7f77d41452a0 0x7f77d40cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 --2- 172.21.15.165:0/127007901 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 --2- 172.21.15.165:0/127007901 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41446d0 0x7f77d40b3f20 unknown :-1 s=CLOSED pgs=2722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 -- 172.21.15.165:0/127007901 >> 172.21.15.165:0/127007901 conn(0x7f77d406abe0 msgr2=0x7f77d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 -- 172.21.15.165:0/127007901 shutdown_connections 2024-04-25T10:20:37.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.342+0000 7f77da400640 1 -- 172.21.15.165:0/127007901 wait complete. 2024-04-25T10:20:37.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.343+0000 7f77da400640 1 Processor -- start 2024-04-25T10:20:37.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.344+0000 7f77da400640 1 -- start start 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77da400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77d41446d0 0x7f77d4139030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77da400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d4141b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77da400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 0x7f77d4142050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77da400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40cc700 con 0x7f77d41452a0 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77da400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40cc420 con 0x7f77d41446d0 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77da400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77d40cc9e0 con 0x7f77d4144d50 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 0x7f77d4142050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 0x7f77d4142050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39610/0 (socket says 172.21.15.165:39610) 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 -- 172.21.15.165:0/3055413869 learned_addr learned my addr 172.21.15.165:0/3055413869 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77d41446d0 0x7f77d4139030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 -- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 msgr2=0x7f77d4141b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d2a00640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d4141b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d4141b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 -- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77d41446d0 msgr2=0x7f77d4139030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:37.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77d41446d0 0x7f77d4139030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:37.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.345+0000 7f77d3e00640 1 -- 172.21.15.165:0/3055413869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d413bf10 con 0x7f77d41452a0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.346+0000 7f77d3e00640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 0x7f77d4142050 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cd50 tx=0x7f77cc00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.346+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00f680 con 0x7f77d41452a0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.346+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77cc00f840 con 0x7f77d41452a0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.346+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77d40b0090 con 0x7f77d41452a0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.346+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77cc00fc30 con 0x7f77d41452a0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.346+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77d40b0650 con 0x7f77d41452a0 2024-04-25T10:20:37.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.347+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77d40b0850 con 0x7f77d41452a0 2024-04-25T10:20:37.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.349+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f77cc0213f0 con 0x7f77d41452a0 2024-04-25T10:20:37.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.349+0000 7f77cb400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f77b406b4d0 0x7f77b406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:37.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.350+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6001+0+0 (secure 0 0 0) 0x7f77cc08b0f0 con 0x7f77d41452a0 2024-04-25T10:20:37.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.350+0000 7f77d3400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f77b406b4d0 0x7f77b406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:37.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.351+0000 7f77d3400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f77b406b4d0 0x7f77b406d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f77c4034000 tx=0x7f77c402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:37.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.355+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f77d40b0850 con 0x7f77d41452a0 2024-04-25T10:20:37.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:37.585+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f77d4073a00 con 0x7f77d41452a0 2024-04-25T10:20:38.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.133+0000 7f77cb400640 1 -- 172.21.15.165:0/3055413869 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 29 and data pool 30 v135) v1 ==== 174+0+0 (secure 0 0 0) 0x7f77cc014030 con 0x7f77d41452a0 2024-04-25T10:20:38.139 INFO:teuthology.orchestra.run.smithi165.stderr:new fs with metadata pool 29 and data pool 30 2024-04-25T10:20:38.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.144+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f77b406b4d0 msgr2=0x7f77b406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.144+0000 7f77da400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f77b406b4d0 0x7f77b406d990 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f77c4034000 tx=0x7f77c402e000 comp rx=0 tx=0).stop 2024-04-25T10:20:38.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.144+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 msgr2=0x7f77d4142050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.144+0000 7f77da400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 0x7f77d4142050 secure :-1 s=READY pgs=2723 cs=0 l=1 rev1=1 crypto rx=0x7f77cc00cd50 tx=0x7f77cc00a650 comp rx=0 tx=0).stop 2024-04-25T10:20:38.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.145+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 shutdown_connections 2024-04-25T10:20:38.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.145+0000 7f77da400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f77b406b4d0 0x7f77b406d990 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.145+0000 7f77da400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f77d41452a0 0x7f77d4142050 unknown :-1 s=CLOSED pgs=2723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.145+0000 7f77da400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f77d4144d50 0x7f77d4141b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.145+0000 7f77da400640 1 --2- 172.21.15.165:0/3055413869 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f77d41446d0 0x7f77d4139030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.145+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 >> 172.21.15.165:0/3055413869 conn(0x7f77d406abe0 msgr2=0x7f77d40ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.147+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 shutdown_connections 2024-04-25T10:20:38.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.147+0000 7f77da400640 1 -- 172.21.15.165:0/3055413869 wait complete. 2024-04-25T10:20:38.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:38.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.348+0000 7f915e200640 1 Processor -- start 2024-04-25T10:20:38.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.348+0000 7f915e200640 1 -- start start 2024-04-25T10:20:38.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.349+0000 7f915e200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91580ce920 0x7f91580cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f915e200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9158144b30 con 0x7f915806d290 2024-04-25T10:20:38.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f915e200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580712d0 con 0x7f9158144750 2024-04-25T10:20:38.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f915e200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91580715b0 con 0x7f91580ce920 2024-04-25T10:20:38.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f9156a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91580ce920 0x7f91580cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f9156a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f91580ce920 0x7f91580cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39624/0 (socket says 172.21.15.165:39624) 2024-04-25T10:20:38.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f9156a00640 1 -- 172.21.15.165:0/1931066076 learned_addr learned my addr 172.21.15.165:0/1931066076 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:38.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f9156a00640 1 -- 172.21.15.165:0/1931066076 >> v1:172.21.15.172:6789/0 conn(0x7f9158144750 legacy=0x7f915806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:38.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f9156a00640 1 -- 172.21.15.165:0/1931066076 >> v1:172.21.15.165:6789/0 conn(0x7f915806d290 legacy=0x7f915806d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.350+0000 7f9156a00640 1 -- 172.21.15.165:0/1931066076 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9158071890 con 0x7f91580ce920 2024-04-25T10:20:38.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.352+0000 7f9156a00640 1 --2- 172.21.15.165:0/1931066076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91580ce920 0x7f91580cedb0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f9144009870 tx=0x7f914402a2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a6b33c65e932d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.352+0000 7f9156000640 1 -- 172.21.15.165:0/1931066076 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144037070 con 0x7f91580ce920 2024-04-25T10:20:38.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.352+0000 7f9156000640 1 -- 172.21.15.165:0/1931066076 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9144032440 con 0x7f91580ce920 2024-04-25T10:20:38.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.352+0000 7f9156000640 1 -- 172.21.15.165:0/1931066076 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9144032810 con 0x7f91580ce920 2024-04-25T10:20:38.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 -- 172.21.15.165:0/1931066076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91580ce920 msgr2=0x7f91580cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 --2- 172.21.15.165:0/1931066076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91580ce920 0x7f91580cedb0 secure :-1 s=READY pgs=2725 cs=0 l=1 rev1=1 crypto rx=0x7f9144009870 tx=0x7f914402a2a0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 -- 172.21.15.165:0/1931066076 shutdown_connections 2024-04-25T10:20:38.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 --2- 172.21.15.165:0/1931066076 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f91580ce920 0x7f91580cedb0 unknown :-1 s=CLOSED pgs=2725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 -- 172.21.15.165:0/1931066076 >> 172.21.15.165:0/1931066076 conn(0x7f915806abe0 msgr2=0x7f915806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 -- 172.21.15.165:0/1931066076 shutdown_connections 2024-04-25T10:20:38.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.353+0000 7f915e200640 1 -- 172.21.15.165:0/1931066076 wait complete. 2024-04-25T10:20:38.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.355+0000 7f915e200640 1 Processor -- start 2024-04-25T10:20:38.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.355+0000 7f915e200640 1 -- start start 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f915e200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f915806d290 0x7f91580b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f915e200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91580ce920 0x7f91581e5410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f915e200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 0x7f9158157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f915e200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915806f020 con 0x7f9158144750 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f915e200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915806ece0 con 0x7f915806d290 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f915e200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9158145450 con 0x7f91580ce920 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 0x7f9158157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 0x7f9158157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39636/0 (socket says 172.21.15.165:39636) 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 -- 172.21.15.165:0/2628992382 learned_addr learned my addr 172.21.15.165:0/2628992382 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157400640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f915806d290 0x7f91580b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9156a00640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91580ce920 0x7f91581e5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 -- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91580ce920 msgr2=0x7f91581e5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91580ce920 0x7f91581e5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 -- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f915806d290 msgr2=0x7f91580b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f915806d290 0x7f91580b0f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.356+0000 7f9157e00640 1 -- 172.21.15.165:0/2628992382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91581e6e20 con 0x7f9158144750 2024-04-25T10:20:38.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.357+0000 7f9157e00640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 0x7f9158157110 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f915000c6b0 tx=0x7f915000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.357+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150003b70 con 0x7f9158144750 2024-04-25T10:20:38.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.357+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9150003d30 con 0x7f9158144750 2024-04-25T10:20:38.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.357+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9150005310 con 0x7f9158144750 2024-04-25T10:20:38.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.357+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91581e5de0 con 0x7f9158144750 2024-04-25T10:20:38.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.357+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91581e6620 con 0x7f9158144750 2024-04-25T10:20:38.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.358+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9158073990 con 0x7f9158144750 2024-04-25T10:20:38.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.359+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f91500054b0 con 0x7f9158144750 2024-04-25T10:20:38.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.360+0000 7f914f400640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f913806b4d0 0x7f913806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.360+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f915008a1f0 con 0x7f9158144750 2024-04-25T10:20:38.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.361+0000 7f9157400640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f913806b4d0 0x7f913806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.361+0000 7f9157400640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f913806b4d0 0x7f913806d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9148003d90 tx=0x7f9148009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.370+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f91581e5de0 con 0x7f9158144750 2024-04-25T10:20:38.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.572+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91580d0510 con 0x7f9158144750 2024-04-25T10:20:38.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.575+0000 7f914f400640 1 -- 172.21.15.165:0/2628992382 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f9150058ee0 con 0x7f9158144750 2024-04-25T10:20:38.578 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:38.578 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":194,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:37.587253+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:38.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f913806b4d0 msgr2=0x7f913806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f913806b4d0 0x7f913806d990 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f9148003d90 tx=0x7f9148009040 comp rx=0 tx=0).stop 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 msgr2=0x7f9158157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 0x7f9158157110 secure :-1 s=READY pgs=2726 cs=0 l=1 rev1=1 crypto rx=0x7f915000c6b0 tx=0x7f915000cb70 comp rx=0 tx=0).stop 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 shutdown_connections 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f913806b4d0 0x7f913806d990 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9158144750 0x7f9158157110 unknown :-1 s=CLOSED pgs=2726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f91580ce920 0x7f91581e5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 --2- 172.21.15.165:0/2628992382 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f915806d290 0x7f91580b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 >> 172.21.15.165:0/2628992382 conn(0x7f915806abe0 msgr2=0x7f915806e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 shutdown_connections 2024-04-25T10:20:38.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.578+0000 7f915e200640 1 -- 172.21.15.165:0/2628992382 wait complete. 2024-04-25T10:20:38.599 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:38.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.791+0000 7f8274000640 1 Processor -- start 2024-04-25T10:20:38.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.791+0000 7f8274000640 1 -- start start 2024-04-25T10:20:38.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8274000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8274000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f826c0cfc20 0x7f826c06d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8274000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f826c071650 con 0x7f826c15df50 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8274000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f826c0cbd80 con 0x7f826c0cfc20 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8274000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c161910 con 0x7f826c15db70 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8272000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f826c0cfc20 0x7f826c06d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8270c00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f826c15df50 0x7f826c0cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36246/0 (socket says 172.21.15.165:36246) 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.793+0000 7f8270c00640 1 -- 172.21.15.165:0/1202397793 learned_addr learned my addr 172.21.15.165:0/1202397793 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.794+0000 7f8271600640 1 --2- 172.21.15.165:0/1202397793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.794+0000 7f8272000640 1 -- 172.21.15.165:0/1202397793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 msgr2=0x7f826c0cb840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.794+0000 7f8272000640 1 --2- 172.21.15.165:0/1202397793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0cb840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.794+0000 7f8272000640 1 -- 172.21.15.165:0/1202397793 >> v1:172.21.15.165:6789/0 conn(0x7f826c15df50 legacy=0x7f826c0cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.794+0000 7f8272000640 1 -- 172.21.15.165:0/1202397793 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c0cf230 con 0x7f826c0cfc20 2024-04-25T10:20:38.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.794+0000 7f826be00640 1 -- 172.21.15.165:0/1202397793 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3725437462 0 0) 0x7f8258001110 con 0x7f826c15df50 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.795+0000 7f8272000640 1 --2- 172.21.15.165:0/1202397793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c06d290 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f8264009d90 tx=0x7f826402a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edfebb40258a5f2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.795+0000 7f826be00640 1 -- 172.21.15.165:0/1202397793 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264007740 con 0x7f826c0cfc20 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.795+0000 7f826be00640 1 -- 172.21.15.165:0/1202397793 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82640078e0 con 0x7f826c0cfc20 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.795+0000 7f826be00640 1 -- 172.21.15.165:0/1202397793 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8264007cb0 con 0x7f826c0cfc20 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 -- 172.21.15.165:0/1202397793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 msgr2=0x7f826c06d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 --2- 172.21.15.165:0/1202397793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c06d290 secure :-1 s=READY pgs=2728 cs=0 l=1 rev1=1 crypto rx=0x7f8264009d90 tx=0x7f826402a660 comp rx=0 tx=0).stop 2024-04-25T10:20:38.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 -- 172.21.15.165:0/1202397793 shutdown_connections 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 --2- 172.21.15.165:0/1202397793 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c06d290 unknown :-1 s=CLOSED pgs=2728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 --2- 172.21.15.165:0/1202397793 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0cb840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 -- 172.21.15.165:0/1202397793 >> 172.21.15.165:0/1202397793 conn(0x7f826c06abe0 msgr2=0x7f826c06aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 -- 172.21.15.165:0/1202397793 shutdown_connections 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.796+0000 7f8274000640 1 -- 172.21.15.165:0/1202397793 wait complete. 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.798+0000 7f8274000640 1 Processor -- start 2024-04-25T10:20:38.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.798+0000 7f8274000640 1 -- start start 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.799+0000 7f8274000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c154900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.799+0000 7f8274000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8274000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f826c15df50 0x7f826c1518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8274000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c162120 con 0x7f826c0cfc20 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8274000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c161e40 con 0x7f826c15df50 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8274000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f826c162640 con 0x7f826c15db70 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c154900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c154900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39650/0 (socket says 172.21.15.165:39650) 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 -- 172.21.15.165:0/3166901279 learned_addr learned my addr 172.21.15.165:0/3166901279 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 -- 172.21.15.165:0/3166901279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 msgr2=0x7f826c0af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 -- 172.21.15.165:0/3166901279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f826c15df50 msgr2=0x7f826c1518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f826c15df50 0x7f826c1518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 -- 172.21.15.165:0/3166901279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f826c15f700 con 0x7f826c0cfc20 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.800+0000 7f8271600640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c154900 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f826000d800 tx=0x7f826000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.801+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000b6c0 con 0x7f826c0cfc20 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.801+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8260019070 con 0x7f826c0cfc20 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.801+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826000ba80 con 0x7f826c0cfc20 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.801+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c152590 con 0x7f826c0cfc20 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.801+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c160fb0 con 0x7f826c0cfc20 2024-04-25T10:20:38.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.802+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c073990 con 0x7f826c0cfc20 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.803+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f826000bc20 con 0x7f826c0cfc20 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.804+0000 7f8269600640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f824c06b4d0 0x7f824c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.804+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f826008a850 con 0x7f826c0cfc20 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.805+0000 7f8270c00640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f824c06b4d0 0x7f824c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:38.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.805+0000 7f8270c00640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f824c06b4d0 0x7f824c06d990 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8258007fd0 tx=0x7f8258007960 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:38.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:38.808+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f8260059530 con 0x7f826c0cfc20 2024-04-25T10:20:39.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.002+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f826c159080 con 0x7f826c0cfc20 2024-04-25T10:20:39.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.004+0000 7f8269600640 1 -- 172.21.15.165:0/3166901279 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v194) v1 ==== 74+0+16933 (secure 0 0 0) 0x7f8260058e30 con 0x7f826c0cfc20 2024-04-25T10:20:39.007 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:39.008 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":194,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:37.587253+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:39.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f824c06b4d0 msgr2=0x7f824c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f824c06b4d0 0x7f824c06d990 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f8258007fd0 tx=0x7f8258007960 comp rx=0 tx=0).stop 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 msgr2=0x7f826c154900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c154900 secure :-1 s=READY pgs=2729 cs=0 l=1 rev1=1 crypto rx=0x7f826000d800 tx=0x7f826000dcc0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 shutdown_connections 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f824c06b4d0 0x7f824c06d990 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f826c15df50 0x7f826c1518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f826c15db70 0x7f826c0af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.007+0000 7f8274000640 1 --2- 172.21.15.165:0/3166901279 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f826c0cfc20 0x7f826c154900 unknown :-1 s=CLOSED pgs=2729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.008+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 >> 172.21.15.165:0/3166901279 conn(0x7f826c06abe0 msgr2=0x7f826c0b4570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.008+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 shutdown_connections 2024-04-25T10:20:39.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.008+0000 7f8274000640 1 -- 172.21.15.165:0/3166901279 wait complete. 2024-04-25T10:20:39.029 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-25T10:20:39.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.232+0000 7f521e400640 1 Processor -- start 2024-04-25T10:20:39.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.232+0000 7f521e400640 1 -- start start 2024-04-25T10:20:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.233+0000 7f521e400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f521815b3c0 0x7f52180b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.233+0000 7f521e400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52180b0b80 con 0x7f521815b7a0 2024-04-25T10:20:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.233+0000 7f521e400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52180cc800 con 0x7f52180cc150 2024-04-25T10:20:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.233+0000 7f521e400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52180cc5e0 con 0x7f521815b3c0 2024-04-25T10:20:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.234+0000 7f520fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f521815b7a0 0x7f52180cdae0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36260/0 (socket says 172.21.15.165:36260) 2024-04-25T10:20:39.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.234+0000 7f520fe00640 1 -- 172.21.15.165:0/3314416903 learned_addr learned my addr 172.21.15.165:0/3314416903 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:39.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.234+0000 7f521d000640 1 --2- 172.21.15.165:0/3314416903 >> v2:172.21.15.172:3300/0 conn(0x7f521815b3c0 0x7f52180b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.234+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2844868072 0 0) 0x7f5200001110 con 0x7f521815b7a0 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51fc003520 con 0x7f521815b7a0 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1594343084 0 0) 0x7f51fc003520 con 0x7f521815b7a0 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 >> v2:172.21.15.172:3300/0 conn(0x7f521815b3c0 msgr2=0x7f52180b0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 --2- 172.21.15.165:0/3314416903 >> v2:172.21.15.172:3300/0 conn(0x7f521815b3c0 0x7f52180b0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 >> v1:172.21.15.172:6789/0 conn(0x7f52180cc150 legacy=0x7f521815e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52180ccae0 con 0x7f521815b7a0 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f5200001840 con 0x7f521815b7a0 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 >> v1:172.21.15.165:6789/0 conn(0x7f521815b7a0 legacy=0x7f52180cdae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51fc003a00 0x7f51fc003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51fc004800 0x7f51fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc005210 0x7f51fc00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52180cc5e0 con 0x7f51fc003a00 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc00c160 con 0x7f51fc005210 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.235+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc00c360 con 0x7f51fc004800 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521d000640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51fc003a00 0x7f51fc003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521d000640 1 -- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51fc004800 msgr2=0x7f51fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521d000640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51fc004800 0x7f51fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521da00640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc005210 0x7f51fc00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521d000640 1 -- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc005210 msgr2=0x7f51fc00b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521d000640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc005210 0x7f51fc00b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521d000640 1 -- 172.21.15.165:0/3314416903 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51fc003520 con 0x7f51fc003a00 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521da00640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc005210 0x7f51fc00b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:39.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.236+0000 7f521da00640 1 -- 172.21.15.165:0/3314416903 reap_dead start 2024-04-25T10:20:39.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.237+0000 7f521d000640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51fc003a00 0x7f51fc003e20 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f5208009a90 tx=0x7f51fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e9d7c1f5036ad7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.237+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208036070 con 0x7f51fc003a00 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.237+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5208031400 con 0x7f51fc003a00 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.237+0000 7f5217e00640 1 -- 172.21.15.165:0/3314416903 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5208009690 con 0x7f51fc003a00 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 -- 172.21.15.165:0/3314416903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51fc003a00 msgr2=0x7f51fc003e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51fc003a00 0x7f51fc003e20 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f5208009a90 tx=0x7f51fc005830 comp rx=0 tx=0).stop 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 -- 172.21.15.165:0/3314416903 shutdown_connections 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 --2- 172.21.15.165:0/3314416903 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f51fc003a00 0x7f51fc003e20 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 -- 172.21.15.165:0/3314416903 >> 172.21.15.165:0/3314416903 conn(0x7f5218061cc0 msgr2=0x7f521806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 -- 172.21.15.165:0/3314416903 shutdown_connections 2024-04-25T10:20:39.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.238+0000 7f521e400640 1 -- 172.21.15.165:0/3314416903 wait complete. 2024-04-25T10:20:39.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.240+0000 7f521e400640 1 Processor -- start 2024-04-25T10:20:39.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.240+0000 7f521e400640 1 -- start start 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f521e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc003a00 0x7f52181e6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f521e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 0x7f52180caf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f521e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f521814ca00 0x7f521814ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f521e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52180b0f60 con 0x7f5218149510 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f521e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52180cd650 con 0x7f51fc003a00 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f521e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52180b10e0 con 0x7f521814ca00 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f520fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 0x7f52180caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f520fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 0x7f52180caf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39668/0 (socket says 172.21.15.165:39668) 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.241+0000 7f520fe00640 1 -- 172.21.15.165:0/3416092309 learned_addr learned my addr 172.21.15.165:0/3416092309 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f521d000640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc003a00 0x7f52181e6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f521da00640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f521814ca00 0x7f521814ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f520fe00640 1 -- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f521814ca00 msgr2=0x7f521814ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f520fe00640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f521814ca00 0x7f521814ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f520fe00640 1 -- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc003a00 msgr2=0x7f52181e6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f520fe00640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc003a00 0x7f52181e6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f520fe00640 1 -- 172.21.15.165:0/3416092309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5218149a10 con 0x7f5218149510 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f521da00640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f521814ca00 0x7f521814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f521d000640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc003a00 0x7f52181e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f520fe00640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 0x7f52180caf90 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f520000c730 tx=0x7f520000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5200018070 con 0x7f5218149510 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52000072f0 con 0x7f5218149510 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52000047e0 con 0x7f5218149510 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.242+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5218149cf0 con 0x7f5218149510 2024-04-25T10:20:39.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.243+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521814a2b0 con 0x7f5218149510 2024-04-25T10:20:39.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.244+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5218073990 con 0x7f5218149510 2024-04-25T10:20:39.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.245+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5200004b90 con 0x7f5218149510 2024-04-25T10:20:39.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.246+0000 7f5215600640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:39.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.246+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5200089c20 con 0x7f5218149510 2024-04-25T10:20:39.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.246+0000 7f521d000640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:39.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.247+0000 7f521d000640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51f406b4d0 0x7f51f406d990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5208033000 tx=0x7f5208008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:39.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.250+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f52180b0b80 con 0x7f5218149510 2024-04-25T10:20:39.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:39.467+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f521806b4f0 con 0x7f5218149510 2024-04-25T10:20:40.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.147+0000 7f5215600640 1 -- 172.21.15.165:0/3416092309 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v138) v1 ==== 118+0+0 (secure 0 0 0) 0x7f52000587e0 con 0x7f5218149510 2024-04-25T10:20:40.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51f406b4d0 msgr2=0x7f51f406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51f406b4d0 0x7f51f406d990 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f5208033000 tx=0x7f5208008030 comp rx=0 tx=0).stop 2024-04-25T10:20:40.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 msgr2=0x7f52180caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 0x7f52180caf90 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f520000c730 tx=0x7f520000cbf0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 shutdown_connections 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f51f406b4d0 0x7f51f406d990 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f521814ca00 0x7f521814ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5218149510 0x7f52180caf90 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 --2- 172.21.15.165:0/3416092309 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51fc003a00 0x7f52181e6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.151+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 >> 172.21.15.165:0/3416092309 conn(0x7f5218061cc0 msgr2=0x7f521815ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.152+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 shutdown_connections 2024-04-25T10:20:40.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.152+0000 7f521e400640 1 -- 172.21.15.165:0/3416092309 wait complete. 2024-04-25T10:20:40.175 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-04-25T10:20:40.175 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-04-25T10:20:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.375+0000 7f6855c00640 1 Processor -- start 2024-04-25T10:20:40.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.376+0000 7f6855c00640 1 -- start start 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.377+0000 7f6855c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f685015dd30 0x7f68500cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.377+0000 7f6855c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f685006bff0 0x7f68500b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.377+0000 7f6855c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685015e110 con 0x7f68500b5970 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.377+0000 7f6855c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685006d000 con 0x7f685015dd30 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.377+0000 7f6855c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b2fd0 con 0x7f685006bff0 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.378+0000 7f6855200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f68500b5970 0x7f6850161820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36274/0 (socket says 172.21.15.165:36274) 2024-04-25T10:20:40.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.378+0000 7f6855200640 1 -- 172.21.15.165:0/702850070 learned_addr learned my addr 172.21.15.165:0/702850070 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:40.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.378+0000 7f684fe00640 1 --2- 172.21.15.165:0/702850070 >> v2:172.21.15.172:3300/0 conn(0x7f685015dd30 0x7f68500cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.378+0000 7f6847400640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f685006bff0 0x7f68500b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.378+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182670826 0 0) 0x7f685015e110 con 0x7f68500b5970 2024-04-25T10:20:40.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.378+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6834003520 con 0x7f68500b5970 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3569757120 0 0) 0x7f6834003520 con 0x7f68500b5970 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f685006bff0 msgr2=0x7f68500b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f685006bff0 0x7f68500b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 >> v2:172.21.15.172:3300/0 conn(0x7f685015dd30 msgr2=0x7f68500cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 --2- 172.21.15.165:0/702850070 >> v2:172.21.15.172:3300/0 conn(0x7f685015dd30 0x7f68500cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685006c480 con 0x7f68500b5970 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f68480019f0 con 0x7f68500b5970 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 >> v1:172.21.15.165:6789/0 conn(0x7f68500b5970 legacy=0x7f6850161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68340039b0 0x7f6834003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6834004800 0x7f6834004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6834005180 0x7f683400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685006d000 con 0x7f68340039b0 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68500b2fd0 con 0x7f6834005180 2024-04-25T10:20:40.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6834004600 con 0x7f6834004800 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684fe00640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68340039b0 0x7f6834003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684fe00640 1 -- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6834004800 msgr2=0x7f6834004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f6847400640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6834004800 0x7f6834004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684fe00640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6834004800 0x7f6834004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.379+0000 7f684fe00640 1 -- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6834005180 msgr2=0x7f683400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.380+0000 7f684fe00640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6834005180 0x7f683400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.380+0000 7f684fe00640 1 -- 172.21.15.165:0/702850070 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6834003520 con 0x7f68340039b0 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.380+0000 7f684fe00640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68340039b0 0x7f6834003dd0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f6840009ac0 tx=0x7f6834005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b768d0c1a749620 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.381+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840036070 con 0x7f68340039b0 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.381+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6840031400 con 0x7f68340039b0 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.381+0000 7f684f400640 1 -- 172.21.15.165:0/702850070 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6840009690 con 0x7f68340039b0 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.381+0000 7f6855c00640 1 -- 172.21.15.165:0/702850070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68340039b0 msgr2=0x7f6834003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.381+0000 7f6855c00640 1 --2- 172.21.15.165:0/702850070 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68340039b0 0x7f6834003dd0 secure :-1 s=READY pgs=2734 cs=0 l=1 rev1=1 crypto rx=0x7f6840009ac0 tx=0x7f6834005830 comp rx=0 tx=0).stop 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.381+0000 7f6855200640 1 -- 172.21.15.165:0/702850070 reap_dead start 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.382+0000 7f6855c00640 1 -- 172.21.15.165:0/702850070 shutdown_connections 2024-04-25T10:20:40.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.382+0000 7f6855c00640 1 -- 172.21.15.165:0/702850070 >> 172.21.15.165:0/702850070 conn(0x7f68500d0610 msgr2=0x7f68500d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:40.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.382+0000 7f6855c00640 1 -- 172.21.15.165:0/702850070 shutdown_connections 2024-04-25T10:20:40.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.382+0000 7f6855c00640 1 -- 172.21.15.165:0/702850070 wait complete. 2024-04-25T10:20:40.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.383+0000 7f6855c00640 1 Processor -- start 2024-04-25T10:20:40.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.383+0000 7f6855c00640 1 -- start start 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.384+0000 7f6855c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68340039b0 0x7f685015afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.384+0000 7f6855c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68500ab1d0 0x7f68500b7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.384+0000 7f6855c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 0x7f68500b8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.384+0000 7f6855c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685006cd20 con 0x7f68500ab650 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.384+0000 7f6855c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685006ca40 con 0x7f68500ab1d0 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.384+0000 7f6855c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68500cf110 con 0x7f68340039b0 2024-04-25T10:20:40.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 0x7f68500b8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 0x7f68500b8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39680/0 (socket says 172.21.15.165:39680) 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 -- 172.21.15.165:0/3906541301 learned_addr learned my addr 172.21.15.165:0/3906541301 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 -- 172.21.15.165:0/3906541301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68340039b0 msgr2=0x7f685015afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68340039b0 0x7f685015afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 -- 172.21.15.165:0/3906541301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68500ab1d0 msgr2=0x7f68500b7bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68500ab1d0 0x7f68500b7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 -- 172.21.15.165:0/3906541301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68500caa80 con 0x7f68500ab650 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855200640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 0x7f68500b8130 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f6848001560 tx=0x7f684800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684800a610 con 0x7f68500ab650 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684800a7b0 con 0x7f68500ab650 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.385+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68500abbe0 con 0x7f68500ab650 2024-04-25T10:20:40.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.386+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684800aba0 con 0x7f68500ab650 2024-04-25T10:20:40.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.386+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68500cb020 con 0x7f68500ab650 2024-04-25T10:20:40.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.387+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685006bff0 con 0x7f68500ab650 2024-04-25T10:20:40.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.388+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6848011070 con 0x7f68500ab650 2024-04-25T10:20:40.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.389+0000 7f684cc00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682c06b500 0x7f682c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:40.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.389+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f6848089dd0 con 0x7f68500ab650 2024-04-25T10:20:40.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.389+0000 7f684fe00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682c06b500 0x7f682c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:40.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.389+0000 7f684fe00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682c06b500 0x7f682c06d9c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6840033000 tx=0x7f684002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:40.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.393+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f685006bff0 con 0x7f68500ab650 2024-04-25T10:20:40.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:40.632+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f68500d16c0 con 0x7f68500ab650 2024-04-25T10:20:41.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.161+0000 7f684cc00640 1 -- 172.21.15.165:0/3906541301 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v139) v1 ==== 122+0+0 (secure 0 0 0) 0x7f684801a060 con 0x7f68500ab650 2024-04-25T10:20:41.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.164+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682c06b500 msgr2=0x7f682c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.164+0000 7f6855c00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682c06b500 0x7f682c06d9c0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6840033000 tx=0x7f684002e000 comp rx=0 tx=0).stop 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.164+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 msgr2=0x7f68500b8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.164+0000 7f6855c00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 0x7f68500b8130 secure :-1 s=READY pgs=2735 cs=0 l=1 rev1=1 crypto rx=0x7f6848001560 tx=0x7f684800ccd0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 shutdown_connections 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f682c06b500 0x7f682c06d9c0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f68500ab650 0x7f68500b8130 unknown :-1 s=CLOSED pgs=2735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f68500ab1d0 0x7f68500b7bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 --2- 172.21.15.165:0/3906541301 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f68340039b0 0x7f685015afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 >> 172.21.15.165:0/3906541301 conn(0x7f68500d0610 msgr2=0x7f685015ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 shutdown_connections 2024-04-25T10:20:41.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.165+0000 7f6855c00640 1 -- 172.21.15.165:0/3906541301 wait complete. 2024-04-25T10:20:41.189 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-04-25T10:20:41.190 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:41.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.387+0000 7f0b6d400640 1 Processor -- start 2024-04-25T10:20:41.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.387+0000 7f0b6d400640 1 -- start start 2024-04-25T10:20:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b6d400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0b68144440 0x7f0b68070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b6d400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f0b680cda30 0x7f0b6815ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b6d400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b6815bf30 con 0x7f0b68071530 2024-04-25T10:20:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b6d400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b680719a0 con 0x7f0b68144440 2024-04-25T10:20:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b6d400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b6815c860 con 0x7f0b680cda30 2024-04-25T10:20:41.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b67e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0b68144440 0x7f0b68070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b67e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0b68144440 0x7f0b68070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39690/0 (socket says 172.21.15.165:39690) 2024-04-25T10:20:41.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.388+0000 7f0b67e00640 1 -- 172.21.15.165:0/4188422116 learned_addr learned my addr 172.21.15.165:0/4188422116 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:41.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.389+0000 7f0b67e00640 1 -- 172.21.15.165:0/4188422116 >> v2:172.21.15.172:3300/0 conn(0x7f0b680cda30 msgr2=0x7f0b6815ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:41.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.389+0000 7f0b67e00640 1 --2- 172.21.15.165:0/4188422116 >> v2:172.21.15.172:3300/0 conn(0x7f0b680cda30 0x7f0b6815ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.389+0000 7f0b67e00640 1 -- 172.21.15.165:0/4188422116 >> v1:172.21.15.165:6789/0 conn(0x7f0b68071530 legacy=0x7f0b680cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.389+0000 7f0b67e00640 1 -- 172.21.15.165:0/4188422116 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6815c210 con 0x7f0b68144440 2024-04-25T10:20:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.390+0000 7f0b67e00640 1 --2- 172.21.15.165:0/4188422116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68144440 0x7f0b68070ff0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f0b680b2280 tx=0x7f0b5802a310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d0f37e5b018d1a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.390+0000 7f0b67400640 1 -- 172.21.15.165:0/4188422116 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58037070 con 0x7f0b68144440 2024-04-25T10:20:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.390+0000 7f0b67400640 1 -- 172.21.15.165:0/4188422116 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58032640 con 0x7f0b68144440 2024-04-25T10:20:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.390+0000 7f0b67400640 1 -- 172.21.15.165:0/4188422116 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5802ae60 con 0x7f0b68144440 2024-04-25T10:20:41.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 -- 172.21.15.165:0/4188422116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68144440 msgr2=0x7f0b68070ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 --2- 172.21.15.165:0/4188422116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68144440 0x7f0b68070ff0 secure :-1 s=READY pgs=2737 cs=0 l=1 rev1=1 crypto rx=0x7f0b680b2280 tx=0x7f0b5802a310 comp rx=0 tx=0).stop 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 -- 172.21.15.165:0/4188422116 shutdown_connections 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 --2- 172.21.15.165:0/4188422116 >> v2:172.21.15.172:3300/0 conn(0x7f0b680cda30 0x7f0b6815ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 --2- 172.21.15.165:0/4188422116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68144440 0x7f0b68070ff0 unknown :-1 s=CLOSED pgs=2737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 -- 172.21.15.165:0/4188422116 >> 172.21.15.165:0/4188422116 conn(0x7f0b6806abe0 msgr2=0x7f0b6806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 -- 172.21.15.165:0/4188422116 shutdown_connections 2024-04-25T10:20:41.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.391+0000 7f0b6d400640 1 -- 172.21.15.165:0/4188422116 wait complete. 2024-04-25T10:20:41.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.393+0000 7f0b6d400640 1 Processor -- start 2024-04-25T10:20:41.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.393+0000 7f0b6d400640 1 -- start start 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6d400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 0x7f0b680a89f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6d400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b680cda30 0x7f0b680cada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6d400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68144440 0x7f0b680c8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6d400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6815bc30 con 0x7f0b68071530 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6d400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b680cdea0 con 0x7f0b68144440 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6d400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6815d230 con 0x7f0b680cda30 2024-04-25T10:20:41.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 0x7f0b680a89f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 0x7f0b680a89f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39702/0 (socket says 172.21.15.165:39702) 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 -- 172.21.15.165:0/3194357669 learned_addr learned my addr 172.21.15.165:0/3194357669 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 -- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b680cda30 msgr2=0x7f0b680cada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b680cda30 0x7f0b680cada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 -- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68144440 msgr2=0x7f0b680c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b6ca00640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68144440 0x7f0b680c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68144440 0x7f0b680c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.394+0000 7f0b67e00640 1 -- 172.21.15.165:0/3194357669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b680cb2e0 con 0x7f0b68071530 2024-04-25T10:20:41.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.395+0000 7f0b67e00640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 0x7f0b680a89f0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f0b58000c00 tx=0x7f0b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.395+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58041020 con 0x7f0b68071530 2024-04-25T10:20:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.395+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b58030970 con 0x7f0b68071530 2024-04-25T10:20:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.395+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b58037040 con 0x7f0b68071530 2024-04-25T10:20:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.395+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b681542d0 con 0x7f0b68071530 2024-04-25T10:20:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.395+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b680cb860 con 0x7f0b68071530 2024-04-25T10:20:41.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.396+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6813f870 con 0x7f0b68071530 2024-04-25T10:20:41.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.397+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0b58044020 con 0x7f0b68071530 2024-04-25T10:20:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.398+0000 7f0b64c00640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b4406b4d0 0x7f0b4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.398+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f0b580aabd0 con 0x7f0b68071530 2024-04-25T10:20:41.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.399+0000 7f0b5f400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b4406b4d0 0x7f0b4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.399+0000 7f0b5f400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b4406b4d0 0x7f0b4406d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000f60 tx=0x7f0b500079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.403+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0b6813f870 con 0x7f0b68071530 2024-04-25T10:20:41.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.635+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b680683e0 con 0x7f0b68071530 2024-04-25T10:20:41.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.637+0000 7f0b64c00640 1 -- 172.21.15.165:0/3194357669 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f0b680683e0 con 0x7f0b68071530 2024-04-25T10:20:41.640 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:41.640 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":139,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:40.633611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11610":{"gid":11610,"name":"c","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/3988146762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3988146762},{"type":"v1","addr":"172.21.15.165:6837","nonce":3988146762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:20:41.640 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 139 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b4406b4d0 msgr2=0x7f0b4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b4406b4d0 0x7f0b4406d990 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f0b50000f60 tx=0x7f0b500079d0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 msgr2=0x7f0b680a89f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 0x7f0b680a89f0 secure :-1 s=READY pgs=2738 cs=0 l=1 rev1=1 crypto rx=0x7f0b58000c00 tx=0x7f0b580046c0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 shutdown_connections 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0b4406b4d0 0x7f0b4406d990 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0b68144440 0x7f0b680c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0b680cda30 0x7f0b680cada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 --2- 172.21.15.165:0/3194357669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0b68071530 0x7f0b680a89f0 unknown :-1 s=CLOSED pgs=2738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 >> 172.21.15.165:0/3194357669 conn(0x7f0b6806abe0 msgr2=0x7f0b680c4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 shutdown_connections 2024-04-25T10:20:41.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.640+0000 7f0b6d400640 1 -- 172.21.15.165:0/3194357669 wait complete. 2024-04-25T10:20:41.662 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:41.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.857+0000 7f83db000640 1 Processor -- start 2024-04-25T10:20:41.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.857+0000 7f83db000640 1 -- start start 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83db000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f83d4071700 0x7f83d40cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83db000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f83d40cd860 0x7f83d415ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83db000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83d415bf30 con 0x7f83d4144440 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83db000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d415c860 con 0x7f83d40cd860 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83db000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83d415c210 con 0x7f83d4071700 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83da600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f83d40cd860 0x7f83d415ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83da600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f83d40cd860 0x7f83d415ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39712/0 (socket says 172.21.15.165:39712) 2024-04-25T10:20:41.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83da600640 1 -- 172.21.15.165:0/2324378571 learned_addr learned my addr 172.21.15.165:0/2324378571 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:41.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.859+0000 7f83d9200640 1 --2- 172.21.15.165:0/2324378571 >> v2:172.21.15.172:3300/0 conn(0x7f83d4071700 0x7f83d40cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.860+0000 7f83da600640 1 -- 172.21.15.165:0/2324378571 >> v2:172.21.15.172:3300/0 conn(0x7f83d4071700 msgr2=0x7f83d40cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:41.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.860+0000 7f83da600640 1 --2- 172.21.15.165:0/2324378571 >> v2:172.21.15.172:3300/0 conn(0x7f83d4071700 0x7f83d40cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.860+0000 7f83da600640 1 -- 172.21.15.165:0/2324378571 >> v1:172.21.15.172:6789/0 conn(0x7f83d4144440 legacy=0x7f83d4070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:41.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.860+0000 7f83da600640 1 -- 172.21.15.165:0/2324378571 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d415c4f0 con 0x7f83d40cd860 2024-04-25T10:20:41.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.861+0000 7f83da600640 1 --2- 172.21.15.165:0/2324378571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d40cd860 0x7f83d415ccf0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f83d406b970 tx=0x7f83cc02a780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf0ea8a25ac1ea8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.861+0000 7f83d3e00640 1 -- 172.21.15.165:0/2324378571 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc036070 con 0x7f83d40cd860 2024-04-25T10:20:41.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.861+0000 7f83d3e00640 1 -- 172.21.15.165:0/2324378571 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc031600 con 0x7f83d40cd860 2024-04-25T10:20:41.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.861+0000 7f83d3e00640 1 -- 172.21.15.165:0/2324378571 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc0319d0 con 0x7f83d40cd860 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 -- 172.21.15.165:0/2324378571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d40cd860 msgr2=0x7f83d415ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 --2- 172.21.15.165:0/2324378571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d40cd860 0x7f83d415ccf0 secure :-1 s=READY pgs=2739 cs=0 l=1 rev1=1 crypto rx=0x7f83d406b970 tx=0x7f83cc02a780 comp rx=0 tx=0).stop 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 -- 172.21.15.165:0/2324378571 shutdown_connections 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 --2- 172.21.15.165:0/2324378571 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d40cd860 0x7f83d415ccf0 unknown :-1 s=CLOSED pgs=2739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 --2- 172.21.15.165:0/2324378571 >> v2:172.21.15.172:3300/0 conn(0x7f83d4071700 0x7f83d40cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 -- 172.21.15.165:0/2324378571 >> 172.21.15.165:0/2324378571 conn(0x7f83d406abe0 msgr2=0x7f83d406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 -- 172.21.15.165:0/2324378571 shutdown_connections 2024-04-25T10:20:41.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.862+0000 7f83db000640 1 -- 172.21.15.165:0/2324378571 wait complete. 2024-04-25T10:20:41.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.864+0000 7f83db000640 1 Processor -- start 2024-04-25T10:20:41.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.864+0000 7f83db000640 1 -- start start 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83db000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83d4071700 0x7f83d415b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83db000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83d40cd860 0x7f83d4131850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83db000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 0x7f83d40a8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83db000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cde70 con 0x7f83d4144440 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83db000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83d40cdcf0 con 0x7f83d40cd860 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83db000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83d415d230 con 0x7f83d4071700 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83da600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 0x7f83d40a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83da600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 0x7f83d40a8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39714/0 (socket says 172.21.15.165:39714) 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.865+0000 7f83da600640 1 -- 172.21.15.165:0/3530556021 learned_addr learned my addr 172.21.15.165:0/3530556021 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:41.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83d9c00640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83d4071700 0x7f83d415b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83d9200640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83d40cd860 0x7f83d4131850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83da600640 1 -- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83d4071700 msgr2=0x7f83d415b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83da600640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83d4071700 0x7f83d415b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83da600640 1 -- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83d40cd860 msgr2=0x7f83d4131850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83da600640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83d40cd860 0x7f83d4131850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83da600640 1 -- 172.21.15.165:0/3530556021 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83d4132230 con 0x7f83d4144440 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83d9200640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83d40cd860 0x7f83d4131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83da600640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 0x7f83d40a8550 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f83cc02ad70 tx=0x7f83cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.866+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc036050 con 0x7f83d4144440 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.867+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83cc042070 con 0x7f83d4144440 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.867+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83d4132510 con 0x7f83d4144440 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.867+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83cc009510 con 0x7f83d4144440 2024-04-25T10:20:41.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.867+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83d40cae40 con 0x7f83d4144440 2024-04-25T10:20:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.868+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83d4073990 con 0x7f83d4144440 2024-04-25T10:20:41.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.869+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f83cc007bd0 con 0x7f83d4144440 2024-04-25T10:20:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.870+0000 7f83d1600640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83b406b500 0x7f83b406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.870+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83cc0ab290 con 0x7f83d4144440 2024-04-25T10:20:41.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.870+0000 7f83d9c00640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83b406b500 0x7f83b406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:41.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.871+0000 7f83d9c00640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83b406b500 0x7f83b406d9c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006fd0 tx=0x7f83c8008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:41.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:41.874+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83cc03a360 con 0x7f83d4144440 2024-04-25T10:20:42.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.072+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83d406ec10 con 0x7f83d4144440 2024-04-25T10:20:42.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.074+0000 7f83d1600640 1 -- 172.21.15.165:0/3530556021 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16920 (secure 0 0 0) 0x7f83cc079ee0 con 0x7f83d4144440 2024-04-25T10:20:42.077 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:42.077 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":195,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:39.134241+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83b406b500 msgr2=0x7f83b406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83b406b500 0x7f83b406d9c0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f83c8006fd0 tx=0x7f83c8008040 comp rx=0 tx=0).stop 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 msgr2=0x7f83d40a8550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 0x7f83d40a8550 secure :-1 s=READY pgs=2740 cs=0 l=1 rev1=1 crypto rx=0x7f83cc02ad70 tx=0x7f83cc004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 shutdown_connections 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f83b406b500 0x7f83b406d9c0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f83d4144440 0x7f83d40a8550 unknown :-1 s=CLOSED pgs=2740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83d40cd860 0x7f83d4131850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 --2- 172.21.15.165:0/3530556021 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f83d4071700 0x7f83d415b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 >> 172.21.15.165:0/3530556021 conn(0x7f83d406abe0 msgr2=0x7f83d413fdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 shutdown_connections 2024-04-25T10:20:42.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.078+0000 7f83db000640 1 -- 172.21.15.165:0/3530556021 wait complete. 2024-04-25T10:20:42.100 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-25T10:20:42.100 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:42.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.300+0000 7f994fe00640 1 Processor -- start 2024-04-25T10:20:42.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.300+0000 7f994fe00640 1 -- start start 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.301+0000 7f994fe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9950144440 0x7f9950070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.301+0000 7f994fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 0x7f99500cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.301+0000 7f994fe00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f995015bf30 con 0x7f99500cd860 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.301+0000 7f994fe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f995015c860 con 0x7f9950144440 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.301+0000 7f994fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9950071970 con 0x7f9950071530 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.302+0000 7f994f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f99500cd860 0x7f995006d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36304/0 (socket says 172.21.15.165:36304) 2024-04-25T10:20:42.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.302+0000 7f994f400640 1 -- 172.21.15.165:0/2001456248 learned_addr learned my addr 172.21.15.165:0/2001456248 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.302+0000 7f994e000640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 0x7f99500cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.302+0000 7f994ea00640 1 --2- 172.21.15.165:0/2001456248 >> v2:172.21.15.172:3300/0 conn(0x7f9950144440 0x7f9950070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.302+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2333532254 0 0) 0x7f995015bf30 con 0x7f99500cd860 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.302+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9934003520 con 0x7f99500cd860 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2963753316 0 0) 0x7f9934003520 con 0x7f99500cd860 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 msgr2=0x7f99500cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 0x7f99500cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 >> v2:172.21.15.172:3300/0 conn(0x7f9950144440 msgr2=0x7f9950070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 --2- 172.21.15.165:0/2001456248 >> v2:172.21.15.172:3300/0 conn(0x7f9950144440 0x7f9950070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f995015c210 con 0x7f99500cd860 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f99480019f0 con 0x7f99500cd860 2024-04-25T10:20:42.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 >> v1:172.21.15.165:6789/0 conn(0x7f99500cd860 legacy=0x7f995006d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 0x7f99340039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9934004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9934004d40 0x7f99340051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015c860 con 0x7f9934003f80 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9950071970 con 0x7f9934004d40 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993400ba40 con 0x7f9950071530 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.303+0000 7f994e000640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9934004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.304+0000 7f994ea00640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 0x7f99340039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.304+0000 7f994e000640 1 -- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 msgr2=0x7f99340039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.304+0000 7f994e000640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9950071530 0x7f99340039b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.304+0000 7f994e000640 1 -- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9934004d40 msgr2=0x7f99340051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.304+0000 7f994e000640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9934004d40 0x7f99340051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.304+0000 7f994e000640 1 -- 172.21.15.165:0/2001456248 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9934003520 con 0x7f9934003f80 2024-04-25T10:20:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.305+0000 7f994e000640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9934004800 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f9938009ac0 tx=0x7f9934005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af7a2dc43ebf47d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.305+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938036070 con 0x7f9934003f80 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.305+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9938031400 con 0x7f9934003f80 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.305+0000 7f994d600640 1 -- 172.21.15.165:0/2001456248 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938009690 con 0x7f9934003f80 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994fe00640 1 -- 172.21.15.165:0/2001456248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 msgr2=0x7f9934004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994fe00640 1 --2- 172.21.15.165:0/2001456248 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9934004800 secure :-1 s=READY pgs=2742 cs=0 l=1 rev1=1 crypto rx=0x7f9938009ac0 tx=0x7f9934005830 comp rx=0 tx=0).stop 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994f400640 1 -- 172.21.15.165:0/2001456248 reap_dead start 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994fe00640 1 -- 172.21.15.165:0/2001456248 shutdown_connections 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994fe00640 1 -- 172.21.15.165:0/2001456248 >> 172.21.15.165:0/2001456248 conn(0x7f995006abe0 msgr2=0x7f995006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994fe00640 1 -- 172.21.15.165:0/2001456248 shutdown_connections 2024-04-25T10:20:42.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.306+0000 7f994fe00640 1 -- 172.21.15.165:0/2001456248 wait complete. 2024-04-25T10:20:42.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.308+0000 7f994fe00640 1 Processor -- start 2024-04-25T10:20:42.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.308+0000 7f994fe00640 1 -- start start 2024-04-25T10:20:42.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9950153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994fe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9950137440 0x7f995013a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994fe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99500c46c0 0x7f99501f2a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994fe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99500cdcd0 con 0x7f9934003f80 2024-04-25T10:20:42.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994fe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f995015bc30 con 0x7f9950137440 2024-04-25T10:20:42.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994fe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99500cde50 con 0x7f99500c46c0 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9950153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994ea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9950153050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39740/0 (socket says 172.21.15.165:39740) 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994ea00640 1 -- 172.21.15.165:0/2805543804 learned_addr learned my addr 172.21.15.165:0/2805543804 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994e000640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9950137440 0x7f995013a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994ea00640 1 -- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99500c46c0 msgr2=0x7f99501f2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994ea00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99500c46c0 0x7f99501f2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.309+0000 7f994ea00640 1 -- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9950137440 msgr2=0x7f995013a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f994ea00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9950137440 0x7f995013a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f994ea00640 1 -- 172.21.15.165:0/2805543804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9950137c50 con 0x7f9934003f80 2024-04-25T10:20:42.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f994ea00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9950153050 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f99400108e0 tx=0x7f9940010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994000f040 con 0x7f9934003f80 2024-04-25T10:20:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994000eb60 con 0x7f9934003f80 2024-04-25T10:20:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9950139350 con 0x7f9934003f80 2024-04-25T10:20:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994001c400 con 0x7f9934003f80 2024-04-25T10:20:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.310+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9950139910 con 0x7f9934003f80 2024-04-25T10:20:42.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.311+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9950073990 con 0x7f9934003f80 2024-04-25T10:20:42.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.313+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f994001c5a0 con 0x7f9934003f80 2024-04-25T10:20:42.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.313+0000 7f9947400640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f992c06b500 0x7f992c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.314+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f994008ab90 con 0x7f9934003f80 2024-04-25T10:20:42.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.314+0000 7f994e000640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f992c06b500 0x7f992c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.314+0000 7f994e000640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f992c06b500 0x7f992c06d9c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9938033000 tx=0x7f9938008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.318+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9950137c50 con 0x7f9934003f80 2024-04-25T10:20:42.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.517+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f995006ec50 con 0x7f992c06b500 2024-04-25T10:20:42.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.525+0000 7f9947400640 1 -- 172.21.15.165:0/2805543804 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7f995006ec50 con 0x7f992c06b500 2024-04-25T10:20:42.531 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:42.533 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f992c06b500 msgr2=0x7f992c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f992c06b500 0x7f992c06d9c0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9938033000 tx=0x7f9938008030 comp rx=0 tx=0).stop 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 msgr2=0x7f9950153050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9950153050 secure :-1 s=READY pgs=2743 cs=0 l=1 rev1=1 crypto rx=0x7f99400108e0 tx=0x7f9940010da0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 shutdown_connections 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f992c06b500 0x7f992c06d9c0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99500c46c0 0x7f99501f2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9950137440 0x7f995013a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 --2- 172.21.15.165:0/2805543804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9934003f80 0x7f9950153050 unknown :-1 s=CLOSED pgs=2743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 >> 172.21.15.165:0/2805543804 conn(0x7f995006abe0 msgr2=0x7f9950140500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 shutdown_connections 2024-04-25T10:20:42.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.533+0000 7f994fe00640 1 -- 172.21.15.165:0/2805543804 wait complete. 2024-04-25T10:20:42.554 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":753,"stamp":"2024-04-25T10:20:42.510198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":541820,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":521203,"kb_avail":754432900,"statfs":{"total":773094113280,"available":772539289600,"internally_reserved":0,"allocated":20979712,"data_stored":10472484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":533712434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002105"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655375+0000","last_change":"2024-04-25T10:20:38.158137+0000","last_active":"2024-04-25T10:20:40.655375+0000","last_peered":"2024-04-25T10:20:40.655375+0000","last_clean":"2024-04-25T10:20:40.655375+0000","last_became_active":"2024-04-25T10:20:38.157081+0000","last_became_peered":"2024-04-25T10:20:38.157081+0000","last_unstale":"2024-04-25T10:20:40.655375+0000","last_undegraded":"2024-04-25T10:20:40.655375+0000","last_fullsized":"2024-04-25T10:20:40.655375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:06.657804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149774+0000","last_change":"2024-04-25T10:20:38.152055+0000","last_active":"2024-04-25T10:20:39.149774+0000","last_peered":"2024-04-25T10:20:39.149774+0000","last_clean":"2024-04-25T10:20:39.149774+0000","last_became_active":"2024-04-25T10:20:38.151754+0000","last_became_peered":"2024-04-25T10:20:38.151754+0000","last_unstale":"2024-04-25T10:20:39.149774+0000","last_undegraded":"2024-04-25T10:20:39.149774+0000","last_fullsized":"2024-04-25T10:20:39.149774+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:54.309372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654579+0000","last_change":"2024-04-25T10:20:38.158759+0000","last_active":"2024-04-25T10:20:40.654579+0000","last_peered":"2024-04-25T10:20:40.654579+0000","last_clean":"2024-04-25T10:20:40.654579+0000","last_became_active":"2024-04-25T10:20:38.158237+0000","last_became_peered":"2024-04-25T10:20:38.158237+0000","last_unstale":"2024-04-25T10:20:40.654579+0000","last_undegraded":"2024-04-25T10:20:40.654579+0000","last_fullsized":"2024-04-25T10:20:40.654579+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:30.430305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152828+0000","last_change":"2024-04-25T10:20:38.150529+0000","last_active":"2024-04-25T10:20:39.152828+0000","last_peered":"2024-04-25T10:20:39.152828+0000","last_clean":"2024-04-25T10:20:39.152828+0000","last_became_active":"2024-04-25T10:20:38.150218+0000","last_became_peered":"2024-04-25T10:20:38.150218+0000","last_unstale":"2024-04-25T10:20:39.152828+0000","last_undegraded":"2024-04-25T10:20:39.152828+0000","last_fullsized":"2024-04-25T10:20:39.152828+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:07.490219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656271+0000","last_change":"2024-04-25T10:20:38.153633+0000","last_active":"2024-04-25T10:20:40.656271+0000","last_peered":"2024-04-25T10:20:40.656271+0000","last_clean":"2024-04-25T10:20:40.656271+0000","last_became_active":"2024-04-25T10:20:38.153300+0000","last_became_peered":"2024-04-25T10:20:38.153300+0000","last_unstale":"2024-04-25T10:20:40.656271+0000","last_undegraded":"2024-04-25T10:20:40.656271+0000","last_fullsized":"2024-04-25T10:20:40.656271+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:53.408618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653896+0000","last_change":"2024-04-25T10:20:38.148674+0000","last_active":"2024-04-25T10:20:40.653896+0000","last_peered":"2024-04-25T10:20:40.653896+0000","last_clean":"2024-04-25T10:20:40.653896+0000","last_became_active":"2024-04-25T10:20:38.148398+0000","last_became_peered":"2024-04-25T10:20:38.148398+0000","last_unstale":"2024-04-25T10:20:40.653896+0000","last_undegraded":"2024-04-25T10:20:40.653896+0000","last_fullsized":"2024-04-25T10:20:40.653896+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:20.159076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654166+0000","last_change":"2024-04-25T10:20:38.153566+0000","last_active":"2024-04-25T10:20:40.654166+0000","last_peered":"2024-04-25T10:20:40.654166+0000","last_clean":"2024-04-25T10:20:40.654166+0000","last_became_active":"2024-04-25T10:20:38.153304+0000","last_became_peered":"2024-04-25T10:20:38.153304+0000","last_unstale":"2024-04-25T10:20:40.654166+0000","last_undegraded":"2024-04-25T10:20:40.654166+0000","last_fullsized":"2024-04-25T10:20:40.654166+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:24.946068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653131+0000","last_change":"2024-04-25T10:20:38.143360+0000","last_active":"2024-04-25T10:20:40.653131+0000","last_peered":"2024-04-25T10:20:40.653131+0000","last_clean":"2024-04-25T10:20:40.653131+0000","last_became_active":"2024-04-25T10:20:38.143041+0000","last_became_peered":"2024-04-25T10:20:38.143041+0000","last_unstale":"2024-04-25T10:20:40.653131+0000","last_undegraded":"2024-04-25T10:20:40.653131+0000","last_fullsized":"2024-04-25T10:20:40.653131+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.986382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653646+0000","last_change":"2024-04-25T10:20:38.143759+0000","last_active":"2024-04-25T10:20:40.653646+0000","last_peered":"2024-04-25T10:20:40.653646+0000","last_clean":"2024-04-25T10:20:40.653646+0000","last_became_active":"2024-04-25T10:20:38.141717+0000","last_became_peered":"2024-04-25T10:20:38.141717+0000","last_unstale":"2024-04-25T10:20:40.653646+0000","last_undegraded":"2024-04-25T10:20:40.653646+0000","last_fullsized":"2024-04-25T10:20:40.653646+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:04.616110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654589+0000","last_change":"2024-04-25T10:20:38.155488+0000","last_active":"2024-04-25T10:20:40.654589+0000","last_peered":"2024-04-25T10:20:40.654589+0000","last_clean":"2024-04-25T10:20:40.654589+0000","last_became_active":"2024-04-25T10:20:38.154003+0000","last_became_peered":"2024-04-25T10:20:38.154003+0000","last_unstale":"2024-04-25T10:20:40.654589+0000","last_undegraded":"2024-04-25T10:20:40.654589+0000","last_fullsized":"2024-04-25T10:20:40.654589+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:06.126252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153610+0000","last_change":"2024-04-25T10:20:38.143779+0000","last_active":"2024-04-25T10:20:39.153610+0000","last_peered":"2024-04-25T10:20:39.153610+0000","last_clean":"2024-04-25T10:20:39.153610+0000","last_became_active":"2024-04-25T10:20:38.143421+0000","last_became_peered":"2024-04-25T10:20:38.143421+0000","last_unstale":"2024-04-25T10:20:39.153610+0000","last_undegraded":"2024-04-25T10:20:39.153610+0000","last_fullsized":"2024-04-25T10:20:39.153610+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:11.823190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153620+0000","last_change":"2024-04-25T10:20:38.146347+0000","last_active":"2024-04-25T10:20:39.153620+0000","last_peered":"2024-04-25T10:20:39.153620+0000","last_clean":"2024-04-25T10:20:39.153620+0000","last_became_active":"2024-04-25T10:20:38.143874+0000","last_became_peered":"2024-04-25T10:20:38.143874+0000","last_unstale":"2024-04-25T10:20:39.153620+0000","last_undegraded":"2024-04-25T10:20:39.153620+0000","last_fullsized":"2024-04-25T10:20:39.153620+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:04.980918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654188+0000","last_change":"2024-04-25T10:20:38.148082+0000","last_active":"2024-04-25T10:20:40.654188+0000","last_peered":"2024-04-25T10:20:40.654188+0000","last_clean":"2024-04-25T10:20:40.654188+0000","last_became_active":"2024-04-25T10:20:38.147759+0000","last_became_peered":"2024-04-25T10:20:38.147759+0000","last_unstale":"2024-04-25T10:20:40.654188+0000","last_undegraded":"2024-04-25T10:20:40.654188+0000","last_fullsized":"2024-04-25T10:20:40.654188+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:17.967152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154425+0000","last_change":"2024-04-25T10:20:38.149538+0000","last_active":"2024-04-25T10:20:39.154425+0000","last_peered":"2024-04-25T10:20:39.154425+0000","last_clean":"2024-04-25T10:20:39.154425+0000","last_became_active":"2024-04-25T10:20:38.149208+0000","last_became_peered":"2024-04-25T10:20:38.149208+0000","last_unstale":"2024-04-25T10:20:39.154425+0000","last_undegraded":"2024-04-25T10:20:39.154425+0000","last_fullsized":"2024-04-25T10:20:39.154425+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:06.263834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157117+0000","last_change":"2024-04-25T10:20:38.138879+0000","last_active":"2024-04-25T10:20:39.157117+0000","last_peered":"2024-04-25T10:20:39.157117+0000","last_clean":"2024-04-25T10:20:39.157117+0000","last_became_active":"2024-04-25T10:20:38.138557+0000","last_became_peered":"2024-04-25T10:20:38.138557+0000","last_unstale":"2024-04-25T10:20:39.157117+0000","last_undegraded":"2024-04-25T10:20:39.157117+0000","last_fullsized":"2024-04-25T10:20:39.157117+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.825672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653364+0000","last_change":"2024-04-25T10:20:38.148469+0000","last_active":"2024-04-25T10:20:40.653364+0000","last_peered":"2024-04-25T10:20:40.653364+0000","last_clean":"2024-04-25T10:20:40.653364+0000","last_became_active":"2024-04-25T10:20:38.148027+0000","last_became_peered":"2024-04-25T10:20:38.148027+0000","last_unstale":"2024-04-25T10:20:40.653364+0000","last_undegraded":"2024-04-25T10:20:40.653364+0000","last_fullsized":"2024-04-25T10:20:40.653364+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:05.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154411+0000","last_change":"2024-04-25T10:20:38.155341+0000","last_active":"2024-04-25T10:20:39.154411+0000","last_peered":"2024-04-25T10:20:39.154411+0000","last_clean":"2024-04-25T10:20:39.154411+0000","last_became_active":"2024-04-25T10:20:38.154987+0000","last_became_peered":"2024-04-25T10:20:38.154987+0000","last_unstale":"2024-04-25T10:20:39.154411+0000","last_undegraded":"2024-04-25T10:20:39.154411+0000","last_fullsized":"2024-04-25T10:20:39.154411+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:25.579823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654536+0000","last_change":"2024-04-25T10:20:38.152992+0000","last_active":"2024-04-25T10:20:40.654536+0000","last_peered":"2024-04-25T10:20:40.654536+0000","last_clean":"2024-04-25T10:20:40.654536+0000","last_became_active":"2024-04-25T10:20:38.151638+0000","last_became_peered":"2024-04-25T10:20:38.151638+0000","last_unstale":"2024-04-25T10:20:40.654536+0000","last_undegraded":"2024-04-25T10:20:40.654536+0000","last_fullsized":"2024-04-25T10:20:40.654536+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:58.413715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655755+0000","last_change":"2024-04-25T10:20:38.158884+0000","last_active":"2024-04-25T10:20:40.655755+0000","last_peered":"2024-04-25T10:20:40.655755+0000","last_clean":"2024-04-25T10:20:40.655755+0000","last_became_active":"2024-04-25T10:20:38.158240+0000","last_became_peered":"2024-04-25T10:20:38.158240+0000","last_unstale":"2024-04-25T10:20:40.655755+0000","last_undegraded":"2024-04-25T10:20:40.655755+0000","last_fullsized":"2024-04-25T10:20:40.655755+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:39.462733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654657+0000","last_change":"2024-04-25T10:20:38.138958+0000","last_active":"2024-04-25T10:20:40.654657+0000","last_peered":"2024-04-25T10:20:40.654657+0000","last_clean":"2024-04-25T10:20:40.654657+0000","last_became_active":"2024-04-25T10:20:38.138641+0000","last_became_peered":"2024-04-25T10:20:38.138641+0000","last_unstale":"2024-04-25T10:20:40.654657+0000","last_undegraded":"2024-04-25T10:20:40.654657+0000","last_fullsized":"2024-04-25T10:20:40.654657+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:52.328008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653928+0000","last_change":"2024-04-25T10:20:38.157853+0000","last_active":"2024-04-25T10:20:40.653928+0000","last_peered":"2024-04-25T10:20:40.653928+0000","last_clean":"2024-04-25T10:20:40.653928+0000","last_became_active":"2024-04-25T10:20:38.157538+0000","last_became_peered":"2024-04-25T10:20:38.157538+0000","last_unstale":"2024-04-25T10:20:40.653928+0000","last_undegraded":"2024-04-25T10:20:40.653928+0000","last_fullsized":"2024-04-25T10:20:40.653928+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.739340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.156934+0000","last_change":"2024-04-25T10:20:38.133820+0000","last_active":"2024-04-25T10:20:39.156934+0000","last_peered":"2024-04-25T10:20:39.156934+0000","last_clean":"2024-04-25T10:20:39.156934+0000","last_became_active":"2024-04-25T10:20:38.133119+0000","last_became_peered":"2024-04-25T10:20:38.133119+0000","last_unstale":"2024-04-25T10:20:39.156934+0000","last_undegraded":"2024-04-25T10:20:39.156934+0000","last_fullsized":"2024-04-25T10:20:39.156934+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:29.226886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653192+0000","last_change":"2024-04-25T10:20:38.145153+0000","last_active":"2024-04-25T10:20:40.653192+0000","last_peered":"2024-04-25T10:20:40.653192+0000","last_clean":"2024-04-25T10:20:40.653192+0000","last_became_active":"2024-04-25T10:20:38.144605+0000","last_became_peered":"2024-04-25T10:20:38.144605+0000","last_unstale":"2024-04-25T10:20:40.653192+0000","last_undegraded":"2024-04-25T10:20:40.653192+0000","last_fullsized":"2024-04-25T10:20:40.653192+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:17.629936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654904+0000","last_change":"2024-04-25T10:20:38.157333+0000","last_active":"2024-04-25T10:20:40.654904+0000","last_peered":"2024-04-25T10:20:40.654904+0000","last_clean":"2024-04-25T10:20:40.654904+0000","last_became_active":"2024-04-25T10:20:38.156480+0000","last_became_peered":"2024-04-25T10:20:38.156480+0000","last_unstale":"2024-04-25T10:20:40.654904+0000","last_undegraded":"2024-04-25T10:20:40.654904+0000","last_fullsized":"2024-04-25T10:20:40.654904+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:36.309315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655865+0000","last_change":"2024-04-25T10:20:38.160513+0000","last_active":"2024-04-25T10:20:40.655865+0000","last_peered":"2024-04-25T10:20:40.655865+0000","last_clean":"2024-04-25T10:20:40.655865+0000","last_became_active":"2024-04-25T10:20:38.160210+0000","last_became_peered":"2024-04-25T10:20:38.160210+0000","last_unstale":"2024-04-25T10:20:40.655865+0000","last_undegraded":"2024-04-25T10:20:40.655865+0000","last_fullsized":"2024-04-25T10:20:40.655865+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:03.777294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654644+0000","last_change":"2024-04-25T10:20:38.152751+0000","last_active":"2024-04-25T10:20:40.654644+0000","last_peered":"2024-04-25T10:20:40.654644+0000","last_clean":"2024-04-25T10:20:40.654644+0000","last_became_active":"2024-04-25T10:20:38.151952+0000","last_became_peered":"2024-04-25T10:20:38.151952+0000","last_unstale":"2024-04-25T10:20:40.654644+0000","last_undegraded":"2024-04-25T10:20:40.654644+0000","last_fullsized":"2024-04-25T10:20:40.654644+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:14.719747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154944+0000","last_change":"2024-04-25T10:20:38.149351+0000","last_active":"2024-04-25T10:20:39.154944+0000","last_peered":"2024-04-25T10:20:39.154944+0000","last_clean":"2024-04-25T10:20:39.154944+0000","last_became_active":"2024-04-25T10:20:38.149036+0000","last_became_peered":"2024-04-25T10:20:38.149036+0000","last_unstale":"2024-04-25T10:20:39.154944+0000","last_undegraded":"2024-04-25T10:20:39.154944+0000","last_fullsized":"2024-04-25T10:20:39.154944+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:51.700989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153989+0000","last_change":"2024-04-25T10:20:38.149611+0000","last_active":"2024-04-25T10:20:39.153989+0000","last_peered":"2024-04-25T10:20:39.153989+0000","last_clean":"2024-04-25T10:20:39.153989+0000","last_became_active":"2024-04-25T10:20:38.149329+0000","last_became_peered":"2024-04-25T10:20:38.149329+0000","last_unstale":"2024-04-25T10:20:39.153989+0000","last_undegraded":"2024-04-25T10:20:39.153989+0000","last_fullsized":"2024-04-25T10:20:39.153989+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:41.813659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149214+0000","last_change":"2024-04-25T10:20:38.148450+0000","last_active":"2024-04-25T10:20:39.149214+0000","last_peered":"2024-04-25T10:20:39.149214+0000","last_clean":"2024-04-25T10:20:39.149214+0000","last_became_active":"2024-04-25T10:20:38.148180+0000","last_became_peered":"2024-04-25T10:20:38.148180+0000","last_unstale":"2024-04-25T10:20:39.149214+0000","last_undegraded":"2024-04-25T10:20:39.149214+0000","last_fullsized":"2024-04-25T10:20:39.149214+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:54.726129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653782+0000","last_change":"2024-04-25T10:20:38.133548+0000","last_active":"2024-04-25T10:20:40.653782+0000","last_peered":"2024-04-25T10:20:40.653782+0000","last_clean":"2024-04-25T10:20:40.653782+0000","last_became_active":"2024-04-25T10:20:38.133130+0000","last_became_peered":"2024-04-25T10:20:38.133130+0000","last_unstale":"2024-04-25T10:20:40.653782+0000","last_undegraded":"2024-04-25T10:20:40.653782+0000","last_fullsized":"2024-04-25T10:20:40.653782+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.922881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653031+0000","last_change":"2024-04-25T10:20:38.144924+0000","last_active":"2024-04-25T10:20:40.653031+0000","last_peered":"2024-04-25T10:20:40.653031+0000","last_clean":"2024-04-25T10:20:40.653031+0000","last_became_active":"2024-04-25T10:20:38.144284+0000","last_became_peered":"2024-04-25T10:20:38.144284+0000","last_unstale":"2024-04-25T10:20:40.653031+0000","last_undegraded":"2024-04-25T10:20:40.653031+0000","last_fullsized":"2024-04-25T10:20:40.653031+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:31.014261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654627+0000","last_change":"2024-04-25T10:20:38.146389+0000","last_active":"2024-04-25T10:20:40.654627+0000","last_peered":"2024-04-25T10:20:40.654627+0000","last_clean":"2024-04-25T10:20:40.654627+0000","last_became_active":"2024-04-25T10:20:38.146072+0000","last_became_peered":"2024-04-25T10:20:38.146072+0000","last_unstale":"2024-04-25T10:20:40.654627+0000","last_undegraded":"2024-04-25T10:20:40.654627+0000","last_fullsized":"2024-04-25T10:20:40.654627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:36.425253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654694+0000","last_change":"2024-04-25T10:20:38.151572+0000","last_active":"2024-04-25T10:20:40.654694+0000","last_peered":"2024-04-25T10:20:40.654694+0000","last_clean":"2024-04-25T10:20:40.654694+0000","last_became_active":"2024-04-25T10:20:38.151293+0000","last_became_peered":"2024-04-25T10:20:38.151293+0000","last_unstale":"2024-04-25T10:20:40.654694+0000","last_undegraded":"2024-04-25T10:20:40.654694+0000","last_fullsized":"2024-04-25T10:20:40.654694+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:43.505763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"194'4","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653631+0000","last_change":"2024-04-25T10:20:37.148775+0000","last_active":"2024-04-25T10:20:40.653631+0000","last_peered":"2024-04-25T10:20:40.653631+0000","last_clean":"2024-04-25T10:20:40.653631+0000","last_became_active":"2024-04-25T10:20:37.148473+0000","last_became_peered":"2024-04-25T10:20:37.148473+0000","last_unstale":"2024-04-25T10:20:40.653631+0000","last_undegraded":"2024-04-25T10:20:40.653631+0000","last_fullsized":"2024-04-25T10:20:40.653631+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:47.734938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654886+0000","last_change":"2024-04-25T10:20:38.151096+0000","last_active":"2024-04-25T10:20:40.654886+0000","last_peered":"2024-04-25T10:20:40.654886+0000","last_clean":"2024-04-25T10:20:40.654886+0000","last_became_active":"2024-04-25T10:20:38.149126+0000","last_became_peered":"2024-04-25T10:20:38.149126+0000","last_unstale":"2024-04-25T10:20:40.654886+0000","last_undegraded":"2024-04-25T10:20:40.654886+0000","last_fullsized":"2024-04-25T10:20:40.654886+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:36.595475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153482+0000","last_change":"2024-04-25T10:20:37.153758+0000","last_active":"2024-04-25T10:20:39.153482+0000","last_peered":"2024-04-25T10:20:39.153482+0000","last_clean":"2024-04-25T10:20:39.153482+0000","last_became_active":"2024-04-25T10:20:37.153417+0000","last_became_peered":"2024-04-25T10:20:37.153417+0000","last_unstale":"2024-04-25T10:20:39.153482+0000","last_undegraded":"2024-04-25T10:20:39.153482+0000","last_fullsized":"2024-04-25T10:20:39.153482+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:33.869742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153495+0000","last_change":"2024-04-25T10:20:38.146111+0000","last_active":"2024-04-25T10:20:39.153495+0000","last_peered":"2024-04-25T10:20:39.153495+0000","last_clean":"2024-04-25T10:20:39.153495+0000","last_became_active":"2024-04-25T10:20:38.145742+0000","last_became_peered":"2024-04-25T10:20:38.145742+0000","last_unstale":"2024-04-25T10:20:39.153495+0000","last_undegraded":"2024-04-25T10:20:39.153495+0000","last_fullsized":"2024-04-25T10:20:39.153495+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:22.414400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654300+0000","last_change":"2024-04-25T10:20:37.167453+0000","last_active":"2024-04-25T10:20:40.654300+0000","last_peered":"2024-04-25T10:20:40.654300+0000","last_clean":"2024-04-25T10:20:40.654300+0000","last_became_active":"2024-04-25T10:20:37.167139+0000","last_became_peered":"2024-04-25T10:20:37.167139+0000","last_unstale":"2024-04-25T10:20:40.654300+0000","last_undegraded":"2024-04-25T10:20:40.654300+0000","last_fullsized":"2024-04-25T10:20:40.654300+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:12.366774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653727+0000","last_change":"2024-04-25T10:20:38.151185+0000","last_active":"2024-04-25T10:20:40.653727+0000","last_peered":"2024-04-25T10:20:40.653727+0000","last_clean":"2024-04-25T10:20:40.653727+0000","last_became_active":"2024-04-25T10:20:38.150898+0000","last_became_peered":"2024-04-25T10:20:38.150898+0000","last_unstale":"2024-04-25T10:20:40.653727+0000","last_undegraded":"2024-04-25T10:20:40.653727+0000","last_fullsized":"2024-04-25T10:20:40.653727+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:41.633955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157115+0000","last_change":"2024-04-25T10:20:37.163407+0000","last_active":"2024-04-25T10:20:39.157115+0000","last_peered":"2024-04-25T10:20:39.157115+0000","last_clean":"2024-04-25T10:20:39.157115+0000","last_became_active":"2024-04-25T10:20:37.163155+0000","last_became_peered":"2024-04-25T10:20:37.163155+0000","last_unstale":"2024-04-25T10:20:39.157115+0000","last_undegraded":"2024-04-25T10:20:39.157115+0000","last_fullsized":"2024-04-25T10:20:39.157115+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:52.969137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152762+0000","last_change":"2024-04-25T10:20:38.151729+0000","last_active":"2024-04-25T10:20:39.152762+0000","last_peered":"2024-04-25T10:20:39.152762+0000","last_clean":"2024-04-25T10:20:39.152762+0000","last_became_active":"2024-04-25T10:20:38.151419+0000","last_became_peered":"2024-04-25T10:20:38.151419+0000","last_unstale":"2024-04-25T10:20:39.152762+0000","last_undegraded":"2024-04-25T10:20:39.152762+0000","last_fullsized":"2024-04-25T10:20:39.152762+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:45.659241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655164+0000","last_change":"2024-04-25T10:20:37.165683+0000","last_active":"2024-04-25T10:20:40.655164+0000","last_peered":"2024-04-25T10:20:40.655164+0000","last_clean":"2024-04-25T10:20:40.655164+0000","last_became_active":"2024-04-25T10:20:37.165369+0000","last_became_peered":"2024-04-25T10:20:37.165369+0000","last_unstale":"2024-04-25T10:20:40.655164+0000","last_undegraded":"2024-04-25T10:20:40.655164+0000","last_fullsized":"2024-04-25T10:20:40.655164+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:19.141831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157133+0000","last_change":"2024-04-25T10:20:38.132511+0000","last_active":"2024-04-25T10:20:39.157133+0000","last_peered":"2024-04-25T10:20:39.157133+0000","last_clean":"2024-04-25T10:20:39.157133+0000","last_became_active":"2024-04-25T10:20:38.131839+0000","last_became_peered":"2024-04-25T10:20:38.131839+0000","last_unstale":"2024-04-25T10:20:39.157133+0000","last_undegraded":"2024-04-25T10:20:39.157133+0000","last_fullsized":"2024-04-25T10:20:39.157133+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:25.387519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.151523+0000","last_change":"2024-04-25T10:20:37.163893+0000","last_active":"2024-04-25T10:20:39.151523+0000","last_peered":"2024-04-25T10:20:39.151523+0000","last_clean":"2024-04-25T10:20:39.151523+0000","last_became_active":"2024-04-25T10:20:37.163536+0000","last_became_peered":"2024-04-25T10:20:37.163536+0000","last_unstale":"2024-04-25T10:20:39.151523+0000","last_undegraded":"2024-04-25T10:20:39.151523+0000","last_fullsized":"2024-04-25T10:20:39.151523+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:37.584972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653891+0000","last_change":"2024-04-25T10:20:38.145293+0000","last_active":"2024-04-25T10:20:40.653891+0000","last_peered":"2024-04-25T10:20:40.653891+0000","last_clean":"2024-04-25T10:20:40.653891+0000","last_became_active":"2024-04-25T10:20:38.145032+0000","last_became_peered":"2024-04-25T10:20:38.145032+0000","last_unstale":"2024-04-25T10:20:40.653891+0000","last_undegraded":"2024-04-25T10:20:40.653891+0000","last_fullsized":"2024-04-25T10:20:40.653891+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:21.313336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653971+0000","last_change":"2024-04-25T10:20:37.144223+0000","last_active":"2024-04-25T10:20:40.653971+0000","last_peered":"2024-04-25T10:20:40.653971+0000","last_clean":"2024-04-25T10:20:40.653971+0000","last_became_active":"2024-04-25T10:20:37.143930+0000","last_became_peered":"2024-04-25T10:20:37.143930+0000","last_unstale":"2024-04-25T10:20:40.653971+0000","last_undegraded":"2024-04-25T10:20:40.653971+0000","last_fullsized":"2024-04-25T10:20:40.653971+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:12.372716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157484+0000","last_change":"2024-04-25T10:20:38.149612+0000","last_active":"2024-04-25T10:20:39.157484+0000","last_peered":"2024-04-25T10:20:39.157484+0000","last_clean":"2024-04-25T10:20:39.157484+0000","last_became_active":"2024-04-25T10:20:38.149198+0000","last_became_peered":"2024-04-25T10:20:38.149198+0000","last_unstale":"2024-04-25T10:20:39.157484+0000","last_undegraded":"2024-04-25T10:20:39.157484+0000","last_fullsized":"2024-04-25T10:20:39.157484+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:14.704044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656402+0000","last_change":"2024-04-25T10:20:37.159290+0000","last_active":"2024-04-25T10:20:40.656402+0000","last_peered":"2024-04-25T10:20:40.656402+0000","last_clean":"2024-04-25T10:20:40.656402+0000","last_became_active":"2024-04-25T10:20:37.158979+0000","last_became_peered":"2024-04-25T10:20:37.158979+0000","last_unstale":"2024-04-25T10:20:40.656402+0000","last_undegraded":"2024-04-25T10:20:40.656402+0000","last_fullsized":"2024-04-25T10:20:40.656402+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:53.590246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157736+0000","last_change":"2024-04-25T10:20:38.148862+0000","last_active":"2024-04-25T10:20:39.157736+0000","last_peered":"2024-04-25T10:20:39.157736+0000","last_clean":"2024-04-25T10:20:39.157736+0000","last_became_active":"2024-04-25T10:20:38.148546+0000","last_became_peered":"2024-04-25T10:20:38.148546+0000","last_unstale":"2024-04-25T10:20:39.157736+0000","last_undegraded":"2024-04-25T10:20:39.157736+0000","last_fullsized":"2024-04-25T10:20:39.157736+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:30.098612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153802+0000","last_change":"2024-04-25T10:20:37.164109+0000","last_active":"2024-04-25T10:20:39.153802+0000","last_peered":"2024-04-25T10:20:39.153802+0000","last_clean":"2024-04-25T10:20:39.153802+0000","last_became_active":"2024-04-25T10:20:37.163776+0000","last_became_peered":"2024-04-25T10:20:37.163776+0000","last_unstale":"2024-04-25T10:20:39.153802+0000","last_undegraded":"2024-04-25T10:20:39.153802+0000","last_fullsized":"2024-04-25T10:20:39.153802+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:04.135477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654778+0000","last_change":"2024-04-25T10:20:38.153214+0000","last_active":"2024-04-25T10:20:40.654778+0000","last_peered":"2024-04-25T10:20:40.654778+0000","last_clean":"2024-04-25T10:20:40.654778+0000","last_became_active":"2024-04-25T10:20:38.152479+0000","last_became_peered":"2024-04-25T10:20:38.152479+0000","last_unstale":"2024-04-25T10:20:40.654778+0000","last_undegraded":"2024-04-25T10:20:40.654778+0000","last_fullsized":"2024-04-25T10:20:40.654778+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.955232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"194'3","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656326+0000","last_change":"2024-04-25T10:20:37.157112+0000","last_active":"2024-04-25T10:20:40.656326+0000","last_peered":"2024-04-25T10:20:40.656326+0000","last_clean":"2024-04-25T10:20:40.656326+0000","last_became_active":"2024-04-25T10:20:37.156824+0000","last_became_peered":"2024-04-25T10:20:37.156824+0000","last_unstale":"2024-04-25T10:20:40.656326+0000","last_undegraded":"2024-04-25T10:20:40.656326+0000","last_fullsized":"2024-04-25T10:20:40.656326+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.251772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153852+0000","last_change":"2024-04-25T10:20:38.157865+0000","last_active":"2024-04-25T10:20:39.153852+0000","last_peered":"2024-04-25T10:20:39.153852+0000","last_clean":"2024-04-25T10:20:39.153852+0000","last_became_active":"2024-04-25T10:20:38.157219+0000","last_became_peered":"2024-04-25T10:20:38.157219+0000","last_unstale":"2024-04-25T10:20:39.153852+0000","last_undegraded":"2024-04-25T10:20:39.153852+0000","last_fullsized":"2024-04-25T10:20:39.153852+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:13.193616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"194'6","reported_seq":20,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655039+0000","last_change":"2024-04-25T10:20:37.165749+0000","last_active":"2024-04-25T10:20:40.655039+0000","last_peered":"2024-04-25T10:20:40.655039+0000","last_clean":"2024-04-25T10:20:40.655039+0000","last_became_active":"2024-04-25T10:20:37.165461+0000","last_became_peered":"2024-04-25T10:20:37.165461+0000","last_unstale":"2024-04-25T10:20:40.655039+0000","last_undegraded":"2024-04-25T10:20:40.655039+0000","last_fullsized":"2024-04-25T10:20:40.655039+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:10.346221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149228+0000","last_change":"2024-04-25T10:20:38.143202+0000","last_active":"2024-04-25T10:20:39.149228+0000","last_peered":"2024-04-25T10:20:39.149228+0000","last_clean":"2024-04-25T10:20:39.149228+0000","last_became_active":"2024-04-25T10:20:38.142743+0000","last_became_peered":"2024-04-25T10:20:38.142743+0000","last_unstale":"2024-04-25T10:20:39.149228+0000","last_undegraded":"2024-04-25T10:20:39.149228+0000","last_fullsized":"2024-04-25T10:20:39.149228+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:53.142942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154867+0000","last_change":"2024-04-25T10:20:37.157617+0000","last_active":"2024-04-25T10:20:39.154867+0000","last_peered":"2024-04-25T10:20:39.154867+0000","last_clean":"2024-04-25T10:20:39.154867+0000","last_became_active":"2024-04-25T10:20:37.156570+0000","last_became_peered":"2024-04-25T10:20:37.156570+0000","last_unstale":"2024-04-25T10:20:39.154867+0000","last_undegraded":"2024-04-25T10:20:39.154867+0000","last_fullsized":"2024-04-25T10:20:39.154867+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:04.135477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153064+0000","last_change":"2024-04-25T10:20:38.132416+0000","last_active":"2024-04-25T10:20:39.153064+0000","last_peered":"2024-04-25T10:20:39.153064+0000","last_clean":"2024-04-25T10:20:39.153064+0000","last_became_active":"2024-04-25T10:20:38.131237+0000","last_became_peered":"2024-04-25T10:20:38.131237+0000","last_unstale":"2024-04-25T10:20:39.153064+0000","last_undegraded":"2024-04-25T10:20:39.153064+0000","last_fullsized":"2024-04-25T10:20:39.153064+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:07.882595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.148987+0000","last_change":"2024-04-25T10:20:37.147339+0000","last_active":"2024-04-25T10:20:39.148987+0000","last_peered":"2024-04-25T10:20:39.148987+0000","last_clean":"2024-04-25T10:20:39.148987+0000","last_became_active":"2024-04-25T10:20:37.146325+0000","last_became_peered":"2024-04-25T10:20:37.146325+0000","last_unstale":"2024-04-25T10:20:39.148987+0000","last_undegraded":"2024-04-25T10:20:39.148987+0000","last_fullsized":"2024-04-25T10:20:39.148987+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:49.160760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654268+0000","last_change":"2024-04-25T10:20:38.156176+0000","last_active":"2024-04-25T10:20:40.654268+0000","last_peered":"2024-04-25T10:20:40.654268+0000","last_clean":"2024-04-25T10:20:40.654268+0000","last_became_active":"2024-04-25T10:20:38.154594+0000","last_became_peered":"2024-04-25T10:20:38.154594+0000","last_unstale":"2024-04-25T10:20:40.654268+0000","last_undegraded":"2024-04-25T10:20:40.654268+0000","last_fullsized":"2024-04-25T10:20:40.654268+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:39.451524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653212+0000","last_change":"2024-04-25T10:20:37.159768+0000","last_active":"2024-04-25T10:20:40.653212+0000","last_peered":"2024-04-25T10:20:40.653212+0000","last_clean":"2024-04-25T10:20:40.653212+0000","last_became_active":"2024-04-25T10:20:37.159369+0000","last_became_peered":"2024-04-25T10:20:37.159369+0000","last_unstale":"2024-04-25T10:20:40.653212+0000","last_undegraded":"2024-04-25T10:20:40.653212+0000","last_fullsized":"2024-04-25T10:20:40.653212+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:59.043179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157283+0000","last_change":"2024-04-25T10:20:38.140775+0000","last_active":"2024-04-25T10:20:39.157283+0000","last_peered":"2024-04-25T10:20:39.157283+0000","last_clean":"2024-04-25T10:20:39.157283+0000","last_became_active":"2024-04-25T10:20:38.140036+0000","last_became_peered":"2024-04-25T10:20:38.140036+0000","last_unstale":"2024-04-25T10:20:39.157283+0000","last_undegraded":"2024-04-25T10:20:39.157283+0000","last_fullsized":"2024-04-25T10:20:39.157283+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:14.538700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"194'8","reported_seq":22,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152084+0000","last_change":"2024-04-25T10:20:37.162113+0000","last_active":"2024-04-25T10:20:39.152084+0000","last_peered":"2024-04-25T10:20:39.152084+0000","last_clean":"2024-04-25T10:20:39.152084+0000","last_became_active":"2024-04-25T10:20:37.153138+0000","last_became_peered":"2024-04-25T10:20:37.153138+0000","last_unstale":"2024-04-25T10:20:39.152084+0000","last_undegraded":"2024-04-25T10:20:39.152084+0000","last_fullsized":"2024-04-25T10:20:39.152084+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:44.662847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149413+0000","last_change":"2024-04-25T10:20:38.149496+0000","last_active":"2024-04-25T10:20:39.149413+0000","last_peered":"2024-04-25T10:20:39.149413+0000","last_clean":"2024-04-25T10:20:39.149413+0000","last_became_active":"2024-04-25T10:20:38.149219+0000","last_became_peered":"2024-04-25T10:20:38.149219+0000","last_unstale":"2024-04-25T10:20:39.149413+0000","last_undegraded":"2024-04-25T10:20:39.149413+0000","last_fullsized":"2024-04-25T10:20:39.149413+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:26.953418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654647+0000","last_change":"2024-04-25T10:20:37.161281+0000","last_active":"2024-04-25T10:20:40.654647+0000","last_peered":"2024-04-25T10:20:40.654647+0000","last_clean":"2024-04-25T10:20:40.654647+0000","last_became_active":"2024-04-25T10:20:37.160968+0000","last_became_peered":"2024-04-25T10:20:37.160968+0000","last_unstale":"2024-04-25T10:20:40.654647+0000","last_undegraded":"2024-04-25T10:20:40.654647+0000","last_fullsized":"2024-04-25T10:20:40.654647+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:11.320807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656019+0000","last_change":"2024-04-25T10:20:38.154550+0000","last_active":"2024-04-25T10:20:40.656019+0000","last_peered":"2024-04-25T10:20:40.656019+0000","last_clean":"2024-04-25T10:20:40.656019+0000","last_became_active":"2024-04-25T10:20:38.153854+0000","last_became_peered":"2024-04-25T10:20:38.153854+0000","last_unstale":"2024-04-25T10:20:40.656019+0000","last_undegraded":"2024-04-25T10:20:40.656019+0000","last_fullsized":"2024-04-25T10:20:40.656019+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653297+0000","last_change":"2024-04-25T10:20:37.143337+0000","last_active":"2024-04-25T10:20:40.653297+0000","last_peered":"2024-04-25T10:20:40.653297+0000","last_clean":"2024-04-25T10:20:40.653297+0000","last_became_active":"2024-04-25T10:20:37.143047+0000","last_became_peered":"2024-04-25T10:20:37.143047+0000","last_unstale":"2024-04-25T10:20:40.653297+0000","last_undegraded":"2024-04-25T10:20:40.653297+0000","last_fullsized":"2024-04-25T10:20:40.653297+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:11.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654893+0000","last_change":"2024-04-25T10:20:38.146179+0000","last_active":"2024-04-25T10:20:40.654893+0000","last_peered":"2024-04-25T10:20:40.654893+0000","last_clean":"2024-04-25T10:20:40.654893+0000","last_became_active":"2024-04-25T10:20:38.145328+0000","last_became_peered":"2024-04-25T10:20:38.145328+0000","last_unstale":"2024-04-25T10:20:40.654893+0000","last_undegraded":"2024-04-25T10:20:40.654893+0000","last_fullsized":"2024-04-25T10:20:40.654893+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:47.083549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152825+0000","last_change":"2024-04-25T10:20:37.165828+0000","last_active":"2024-04-25T10:20:39.152825+0000","last_peered":"2024-04-25T10:20:39.152825+0000","last_clean":"2024-04-25T10:20:39.152825+0000","last_became_active":"2024-04-25T10:20:37.164409+0000","last_became_peered":"2024-04-25T10:20:37.164409+0000","last_unstale":"2024-04-25T10:20:39.152825+0000","last_undegraded":"2024-04-25T10:20:39.152825+0000","last_fullsized":"2024-04-25T10:20:39.152825+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.176371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154824+0000","last_change":"2024-04-25T10:20:38.147646+0000","last_active":"2024-04-25T10:20:39.154824+0000","last_peered":"2024-04-25T10:20:39.154824+0000","last_clean":"2024-04-25T10:20:39.154824+0000","last_became_active":"2024-04-25T10:20:38.147359+0000","last_became_peered":"2024-04-25T10:20:38.147359+0000","last_unstale":"2024-04-25T10:20:39.154824+0000","last_undegraded":"2024-04-25T10:20:39.154824+0000","last_fullsized":"2024-04-25T10:20:39.154824+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:43.313289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154221+0000","last_change":"2024-04-25T10:20:37.165159+0000","last_active":"2024-04-25T10:20:39.154221+0000","last_peered":"2024-04-25T10:20:39.154221+0000","last_clean":"2024-04-25T10:20:39.154221+0000","last_became_active":"2024-04-25T10:20:37.164847+0000","last_became_peered":"2024-04-25T10:20:37.164847+0000","last_unstale":"2024-04-25T10:20:39.154221+0000","last_undegraded":"2024-04-25T10:20:39.154221+0000","last_fullsized":"2024-04-25T10:20:39.154221+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:22.587459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.151830+0000","last_change":"2024-04-25T10:20:38.155691+0000","last_active":"2024-04-25T10:20:39.151830+0000","last_peered":"2024-04-25T10:20:39.151830+0000","last_clean":"2024-04-25T10:20:39.151830+0000","last_became_active":"2024-04-25T10:20:38.155153+0000","last_became_peered":"2024-04-25T10:20:38.155153+0000","last_unstale":"2024-04-25T10:20:39.151830+0000","last_undegraded":"2024-04-25T10:20:39.151830+0000","last_fullsized":"2024-04-25T10:20:39.151830+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:50.214563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.155138+0000","last_change":"2024-04-25T10:20:37.157357+0000","last_active":"2024-04-25T10:20:39.155138+0000","last_peered":"2024-04-25T10:20:39.155138+0000","last_clean":"2024-04-25T10:20:39.155138+0000","last_became_active":"2024-04-25T10:20:37.157001+0000","last_became_peered":"2024-04-25T10:20:37.157001+0000","last_unstale":"2024-04-25T10:20:39.155138+0000","last_undegraded":"2024-04-25T10:20:39.155138+0000","last_fullsized":"2024-04-25T10:20:39.155138+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:10.246954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655796+0000","last_change":"2024-04-25T10:20:38.159995+0000","last_active":"2024-04-25T10:20:40.655796+0000","last_peered":"2024-04-25T10:20:40.655796+0000","last_clean":"2024-04-25T10:20:40.655796+0000","last_became_active":"2024-04-25T10:20:38.159685+0000","last_became_peered":"2024-04-25T10:20:38.159685+0000","last_unstale":"2024-04-25T10:20:40.655796+0000","last_undegraded":"2024-04-25T10:20:40.655796+0000","last_fullsized":"2024-04-25T10:20:40.655796+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:10.570193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157105+0000","last_change":"2024-04-25T10:20:37.161882+0000","last_active":"2024-04-25T10:20:39.157105+0000","last_peered":"2024-04-25T10:20:39.157105+0000","last_clean":"2024-04-25T10:20:39.157105+0000","last_became_active":"2024-04-25T10:20:37.161518+0000","last_became_peered":"2024-04-25T10:20:37.161518+0000","last_unstale":"2024-04-25T10:20:39.157105+0000","last_undegraded":"2024-04-25T10:20:39.157105+0000","last_fullsized":"2024-04-25T10:20:39.157105+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:04.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149071+0000","last_change":"2024-04-25T10:20:38.142476+0000","last_active":"2024-04-25T10:20:39.149071+0000","last_peered":"2024-04-25T10:20:39.149071+0000","last_clean":"2024-04-25T10:20:39.149071+0000","last_became_active":"2024-04-25T10:20:38.142197+0000","last_became_peered":"2024-04-25T10:20:38.142197+0000","last_unstale":"2024-04-25T10:20:39.149071+0000","last_undegraded":"2024-04-25T10:20:39.149071+0000","last_fullsized":"2024-04-25T10:20:39.149071+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:41.320349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.156955+0000","last_change":"2024-04-25T10:20:37.152463+0000","last_active":"2024-04-25T10:20:39.156955+0000","last_peered":"2024-04-25T10:20:39.156955+0000","last_clean":"2024-04-25T10:20:39.156955+0000","last_became_active":"2024-04-25T10:20:37.152172+0000","last_became_peered":"2024-04-25T10:20:37.152172+0000","last_unstale":"2024-04-25T10:20:39.156955+0000","last_undegraded":"2024-04-25T10:20:39.156955+0000","last_fullsized":"2024-04-25T10:20:39.156955+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:26.587710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654329+0000","last_change":"2024-04-25T10:20:38.143093+0000","last_active":"2024-04-25T10:20:40.654329+0000","last_peered":"2024-04-25T10:20:40.654329+0000","last_clean":"2024-04-25T10:20:40.654329+0000","last_became_active":"2024-04-25T10:20:38.141843+0000","last_became_peered":"2024-04-25T10:20:38.141843+0000","last_unstale":"2024-04-25T10:20:40.654329+0000","last_undegraded":"2024-04-25T10:20:40.654329+0000","last_fullsized":"2024-04-25T10:20:40.654329+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:10.072815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.652915+0000","last_change":"2024-04-25T10:20:37.142126+0000","last_active":"2024-04-25T10:20:40.652915+0000","last_peered":"2024-04-25T10:20:40.652915+0000","last_clean":"2024-04-25T10:20:40.652915+0000","last_became_active":"2024-04-25T10:20:37.141836+0000","last_became_peered":"2024-04-25T10:20:37.141836+0000","last_unstale":"2024-04-25T10:20:40.652915+0000","last_undegraded":"2024-04-25T10:20:40.652915+0000","last_fullsized":"2024-04-25T10:20:40.652915+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:04.960265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149610+0000","last_change":"2024-04-25T10:20:38.148889+0000","last_active":"2024-04-25T10:20:39.149610+0000","last_peered":"2024-04-25T10:20:39.149610+0000","last_clean":"2024-04-25T10:20:39.149610+0000","last_became_active":"2024-04-25T10:20:38.148600+0000","last_became_peered":"2024-04-25T10:20:38.148600+0000","last_unstale":"2024-04-25T10:20:39.149610+0000","last_undegraded":"2024-04-25T10:20:39.149610+0000","last_fullsized":"2024-04-25T10:20:39.149610+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:33.738452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.155202+0000","last_change":"2024-04-25T10:20:37.157262+0000","last_active":"2024-04-25T10:20:39.155202+0000","last_peered":"2024-04-25T10:20:39.155202+0000","last_clean":"2024-04-25T10:20:39.155202+0000","last_became_active":"2024-04-25T10:20:37.156914+0000","last_became_peered":"2024-04-25T10:20:37.156914+0000","last_unstale":"2024-04-25T10:20:39.155202+0000","last_undegraded":"2024-04-25T10:20:39.155202+0000","last_fullsized":"2024-04-25T10:20:39.155202+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:50.683744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656354+0000","last_change":"2024-04-25T10:20:38.159775+0000","last_active":"2024-04-25T10:20:40.656354+0000","last_peered":"2024-04-25T10:20:40.656354+0000","last_clean":"2024-04-25T10:20:40.656354+0000","last_became_active":"2024-04-25T10:20:38.159387+0000","last_became_peered":"2024-04-25T10:20:38.159387+0000","last_unstale":"2024-04-25T10:20:40.656354+0000","last_undegraded":"2024-04-25T10:20:40.656354+0000","last_fullsized":"2024-04-25T10:20:40.656354+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:31.284436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"195'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157694+0000","last_change":"2024-04-25T10:20:37.155823+0000","last_active":"2024-04-25T10:20:39.157694+0000","last_peered":"2024-04-25T10:20:39.157694+0000","last_clean":"2024-04-25T10:20:39.157694+0000","last_became_active":"2024-04-25T10:20:37.155544+0000","last_became_peered":"2024-04-25T10:20:37.155544+0000","last_unstale":"2024-04-25T10:20:39.157694+0000","last_undegraded":"2024-04-25T10:20:39.157694+0000","last_fullsized":"2024-04-25T10:20:39.157694+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:16.949907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655503+0000","last_change":"2024-04-25T10:20:38.134548+0000","last_active":"2024-04-25T10:20:40.655503+0000","last_peered":"2024-04-25T10:20:40.655503+0000","last_clean":"2024-04-25T10:20:40.655503+0000","last_became_active":"2024-04-25T10:20:38.134188+0000","last_became_peered":"2024-04-25T10:20:38.134188+0000","last_unstale":"2024-04-25T10:20:40.655503+0000","last_undegraded":"2024-04-25T10:20:40.655503+0000","last_fullsized":"2024-04-25T10:20:40.655503+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.735635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157763+0000","last_change":"2024-04-25T10:20:37.144342+0000","last_active":"2024-04-25T10:20:39.157763+0000","last_peered":"2024-04-25T10:20:39.157763+0000","last_clean":"2024-04-25T10:20:39.157763+0000","last_became_active":"2024-04-25T10:20:37.143984+0000","last_became_peered":"2024-04-25T10:20:37.143984+0000","last_unstale":"2024-04-25T10:20:39.157763+0000","last_undegraded":"2024-04-25T10:20:39.157763+0000","last_fullsized":"2024-04-25T10:20:39.157763+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:34.905637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653486+0000","last_change":"2024-04-25T10:20:38.148345+0000","last_active":"2024-04-25T10:20:40.653486+0000","last_peered":"2024-04-25T10:20:40.653486+0000","last_clean":"2024-04-25T10:20:40.653486+0000","last_became_active":"2024-04-25T10:20:38.143713+0000","last_became_peered":"2024-04-25T10:20:38.143713+0000","last_unstale":"2024-04-25T10:20:40.653486+0000","last_undegraded":"2024-04-25T10:20:40.653486+0000","last_fullsized":"2024-04-25T10:20:40.653486+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:33.328586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154514+0000","last_change":"2024-04-25T10:20:37.149136+0000","last_active":"2024-04-25T10:20:39.154514+0000","last_peered":"2024-04-25T10:20:39.154514+0000","last_clean":"2024-04-25T10:20:39.154514+0000","last_became_active":"2024-04-25T10:20:37.148826+0000","last_became_peered":"2024-04-25T10:20:37.148826+0000","last_unstale":"2024-04-25T10:20:39.154514+0000","last_undegraded":"2024-04-25T10:20:39.154514+0000","last_fullsized":"2024-04-25T10:20:39.154514+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653945+0000","last_change":"2024-04-25T10:20:38.144929+0000","last_active":"2024-04-25T10:20:40.653945+0000","last_peered":"2024-04-25T10:20:40.653945+0000","last_clean":"2024-04-25T10:20:40.653945+0000","last_became_active":"2024-04-25T10:20:38.144584+0000","last_became_peered":"2024-04-25T10:20:38.144584+0000","last_unstale":"2024-04-25T10:20:40.653945+0000","last_undegraded":"2024-04-25T10:20:40.653945+0000","last_fullsized":"2024-04-25T10:20:40.653945+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:07.712669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.155199+0000","last_change":"2024-04-25T10:20:37.156319+0000","last_active":"2024-04-25T10:20:39.155199+0000","last_peered":"2024-04-25T10:20:39.155199+0000","last_clean":"2024-04-25T10:20:39.155199+0000","last_became_active":"2024-04-25T10:20:37.151537+0000","last_became_peered":"2024-04-25T10:20:37.151537+0000","last_unstale":"2024-04-25T10:20:39.155199+0000","last_undegraded":"2024-04-25T10:20:39.155199+0000","last_fullsized":"2024-04-25T10:20:39.155199+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:10.805946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653996+0000","last_change":"2024-04-25T10:20:38.145092+0000","last_active":"2024-04-25T10:20:40.653996+0000","last_peered":"2024-04-25T10:20:40.653996+0000","last_clean":"2024-04-25T10:20:40.653996+0000","last_became_active":"2024-04-25T10:20:38.144740+0000","last_became_peered":"2024-04-25T10:20:38.144740+0000","last_unstale":"2024-04-25T10:20:40.653996+0000","last_undegraded":"2024-04-25T10:20:40.653996+0000","last_fullsized":"2024-04-25T10:20:40.653996+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:09.896266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152697+0000","last_change":"2024-04-25T10:20:37.155995+0000","last_active":"2024-04-25T10:20:39.152697+0000","last_peered":"2024-04-25T10:20:39.152697+0000","last_clean":"2024-04-25T10:20:39.152697+0000","last_became_active":"2024-04-25T10:20:37.151152+0000","last_became_peered":"2024-04-25T10:20:37.151152+0000","last_unstale":"2024-04-25T10:20:39.152697+0000","last_undegraded":"2024-04-25T10:20:39.152697+0000","last_fullsized":"2024-04-25T10:20:39.152697+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:29.881912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653492+0000","last_change":"2024-04-25T10:20:38.151550+0000","last_active":"2024-04-25T10:20:40.653492+0000","last_peered":"2024-04-25T10:20:40.653492+0000","last_clean":"2024-04-25T10:20:40.653492+0000","last_became_active":"2024-04-25T10:20:38.150406+0000","last_became_peered":"2024-04-25T10:20:38.150406+0000","last_unstale":"2024-04-25T10:20:40.653492+0000","last_undegraded":"2024-04-25T10:20:40.653492+0000","last_fullsized":"2024-04-25T10:20:40.653492+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:30.107299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655833+0000","last_change":"2024-04-25T10:20:37.160036+0000","last_active":"2024-04-25T10:20:40.655833+0000","last_peered":"2024-04-25T10:20:40.655833+0000","last_clean":"2024-04-25T10:20:40.655833+0000","last_became_active":"2024-04-25T10:20:37.159629+0000","last_became_peered":"2024-04-25T10:20:37.159629+0000","last_unstale":"2024-04-25T10:20:40.655833+0000","last_undegraded":"2024-04-25T10:20:40.655833+0000","last_fullsized":"2024-04-25T10:20:40.655833+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:21.718390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154987+0000","last_change":"2024-04-25T10:20:38.142897+0000","last_active":"2024-04-25T10:20:39.154987+0000","last_peered":"2024-04-25T10:20:39.154987+0000","last_clean":"2024-04-25T10:20:39.154987+0000","last_became_active":"2024-04-25T10:20:38.142589+0000","last_became_peered":"2024-04-25T10:20:38.142589+0000","last_unstale":"2024-04-25T10:20:39.154987+0000","last_undegraded":"2024-04-25T10:20:39.154987+0000","last_fullsized":"2024-04-25T10:20:39.154987+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:00.060423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152402+0000","last_change":"2024-04-25T10:20:37.153623+0000","last_active":"2024-04-25T10:20:39.152402+0000","last_peered":"2024-04-25T10:20:39.152402+0000","last_clean":"2024-04-25T10:20:39.152402+0000","last_became_active":"2024-04-25T10:20:37.152749+0000","last_became_peered":"2024-04-25T10:20:37.152749+0000","last_unstale":"2024-04-25T10:20:39.152402+0000","last_undegraded":"2024-04-25T10:20:39.152402+0000","last_fullsized":"2024-04-25T10:20:39.152402+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:17.801213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654633+0000","last_change":"2024-04-25T10:20:38.149400+0000","last_active":"2024-04-25T10:20:40.654633+0000","last_peered":"2024-04-25T10:20:40.654633+0000","last_clean":"2024-04-25T10:20:40.654633+0000","last_became_active":"2024-04-25T10:20:38.149106+0000","last_became_peered":"2024-04-25T10:20:38.149106+0000","last_unstale":"2024-04-25T10:20:40.654633+0000","last_undegraded":"2024-04-25T10:20:40.654633+0000","last_fullsized":"2024-04-25T10:20:40.654633+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:00.308101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655342+0000","last_change":"2024-04-25T10:20:37.155732+0000","last_active":"2024-04-25T10:20:40.655342+0000","last_peered":"2024-04-25T10:20:40.655342+0000","last_clean":"2024-04-25T10:20:40.655342+0000","last_became_active":"2024-04-25T10:20:37.154402+0000","last_became_peered":"2024-04-25T10:20:37.154402+0000","last_unstale":"2024-04-25T10:20:40.655342+0000","last_undegraded":"2024-04-25T10:20:40.655342+0000","last_fullsized":"2024-04-25T10:20:40.655342+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:15.097625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":501,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655175+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:40.655175+0000","last_peered":"2024-04-25T10:20:40.655175+0000","last_clean":"2024-04-25T10:20:40.655175+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:40.655175+0000","last_undegraded":"2024-04-25T10:20:40.655175+0000","last_fullsized":"2024-04-25T10:20:40.655175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64708,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307132,"statfs":{"total":96636764160,"available":96570503168,"internally_reserved":0,"allocated":3067904,"data_stored":1751906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59984,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311856,"statfs":{"total":96636764160,"available":96575340544,"internally_reserved":0,"allocated":2490368,"data_stored":1162604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55816,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316024,"statfs":{"total":96636764160,"available":96579608576,"internally_reserved":0,"allocated":2482176,"data_stored":1162548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575078,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87672,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284168,"statfs":{"total":96636764160,"available":96546988032,"internally_reserved":0,"allocated":2469888,"data_stored":1160956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575078,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87672,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284168,"statfs":{"total":96636764160,"available":96546988032,"internally_reserved":0,"allocated":2469888,"data_stored":1160888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575077,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1161482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2465792,"data_stored":1160866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66040,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305800,"statfs":{"total":96636764160,"available":96569139200,"internally_reserved":0,"allocated":3059712,"data_stored":1751234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:42.558 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:20:42.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.796+0000 7f1ecbe00640 1 Processor -- start 2024-04-25T10:20:42.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.796+0000 7f1ecbe00640 1 -- start start 2024-04-25T10:20:42.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.798+0000 7f1ecbe00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ecc15db90 0x7f1ecc0b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.798+0000 7f1ecbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc1615a0 0x7f1ecc0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.798+0000 7f1ecbe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ecc0d33f0 con 0x7f1ecc0d04f0 2024-04-25T10:20:42.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.798+0000 7f1ecbe00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ecc0d0980 con 0x7f1ecc15db90 2024-04-25T10:20:42.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.798+0000 7f1ecbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc0d1f50 con 0x7f1ecc1615a0 2024-04-25T10:20:42.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.799+0000 7f1ecaa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ecc15db90 0x7f1ecc0b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.799+0000 7f1eca000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc1615a0 0x7f1ecc0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.799+0000 7f1ecb400640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f1ecc0d04f0 0x7f1ecc061600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:44618/0 (socket says 172.21.15.165:44618) 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.799+0000 7f1ecb400640 1 -- 172.21.15.165:0/4191905717 learned_addr learned my addr 172.21.15.165:0/4191905717 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.799+0000 7f1ecaa00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f1ecc15db90 0x7f1ecc0b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:54952/0 (socket says 172.21.15.165:54952) 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.800+0000 7f1ecaa00640 1 -- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc1615a0 msgr2=0x7f1ecc0cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.800+0000 7f1ecaa00640 1 --2- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc1615a0 0x7f1ecc0cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.800+0000 7f1ecaa00640 1 -- 172.21.15.165:0/4191905717 >> v1:172.21.15.172:6789/0 conn(0x7f1ecc0d04f0 legacy=0x7f1ecc061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.800+0000 7f1ecaa00640 1 -- 172.21.15.165:0/4191905717 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc161df0 con 0x7f1ecc15db90 2024-04-25T10:20:42.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.801+0000 7f1ecaa00640 1 --2- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc0b93c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097a0 tx=0x7f1ebc02a180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3d504505ccbeaf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.802+0000 7f1ec9600640 1 -- 172.21.15.165:0/4191905717 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc037070 con 0x7f1ecc15db90 2024-04-25T10:20:42.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.802+0000 7f1ec9600640 1 -- 172.21.15.165:0/4191905717 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ebc032440 con 0x7f1ecc15db90 2024-04-25T10:20:42.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.802+0000 7f1ec9600640 1 -- 172.21.15.165:0/4191905717 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc032810 con 0x7f1ecc15db90 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 -- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 msgr2=0x7f1ecc0b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc0b93c0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc0097a0 tx=0x7f1ebc02a180 comp rx=0 tx=0).stop 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 -- 172.21.15.165:0/4191905717 shutdown_connections 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc1615a0 0x7f1ecc0cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/4191905717 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc0b93c0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 -- 172.21.15.165:0/4191905717 >> 172.21.15.165:0/4191905717 conn(0x7f1ecc0b52b0 msgr2=0x7f1ecc0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 -- 172.21.15.165:0/4191905717 shutdown_connections 2024-04-25T10:20:42.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.803+0000 7f1ecbe00640 1 -- 172.21.15.165:0/4191905717 wait complete. 2024-04-25T10:20:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.805+0000 7f1ecbe00640 1 Processor -- start 2024-04-25T10:20:42.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.805+0000 7f1ecbe00640 1 -- start start 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc0d04f0 0x7f1ecc14e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecbe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 0x7f1ecc061600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecbe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc0d1be0 con 0x7f1ecc1615a0 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecbe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc1623b0 con 0x7f1ecc15db90 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecbe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc161a20 con 0x7f1ecc0d04f0 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 0x7f1ecc061600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecb400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 0x7f1ecc061600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39752/0 (socket says 172.21.15.165:39752) 2024-04-25T10:20:42.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.806+0000 7f1ecb400640 1 -- 172.21.15.165:0/1274162672 learned_addr learned my addr 172.21.15.165:0/1274162672 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:42.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1eca000640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecaa00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc0d04f0 0x7f1ecc14e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecb400640 1 -- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc0d04f0 msgr2=0x7f1ecc14e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecb400640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc0d04f0 0x7f1ecc14e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecb400640 1 -- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 msgr2=0x7f1ecc15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecb400640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc15df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecb400640 1 -- 172.21.15.165:0/1274162672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc0c7cf0 con 0x7f1ecc1615a0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ecb400640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 0x7f1ecc061600 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400cad0 tx=0x7f1ec400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.807+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4017070 con 0x7f1ecc1615a0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.808+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec400ab70 con 0x7f1ecc1615a0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.808+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ecc160890 con 0x7f1ecc1615a0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.808+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4012400 con 0x7f1ecc1615a0 2024-04-25T10:20:42.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.808+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ecc0c7f90 con 0x7f1ecc1615a0 2024-04-25T10:20:42.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.809+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ecc073990 con 0x7f1ecc1615a0 2024-04-25T10:20:42.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.810+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1ec401a070 con 0x7f1ecc1615a0 2024-04-25T10:20:42.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.811+0000 7f1ec2a00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eac06b4d0 0x7f1eac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:42.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.811+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f1ec408a0b0 con 0x7f1ecc1615a0 2024-04-25T10:20:42.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.811+0000 7f1ecaa00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eac06b4d0 0x7f1eac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:42.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.812+0000 7f1ecaa00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eac06b4d0 0x7f1eac06d990 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:42.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:42.815+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1ec4058d00 con 0x7f1ecc1615a0 2024-04-25T10:20:43.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.007+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ecc0c6320 con 0x7f1eac06b4d0 2024-04-25T10:20:43.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.019+0000 7f1ec2a00640 1 -- 172.21.15.165:0/1274162672 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+252488 (secure 0 0 0) 0x7f1ecc0c6320 con 0x7f1eac06b4d0 2024-04-25T10:20:43.025 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:43.026 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:20:43.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eac06b4d0 msgr2=0x7f1eac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eac06b4d0 0x7f1eac06d990 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc034000 tx=0x7f1ebc02e000 comp rx=0 tx=0).stop 2024-04-25T10:20:43.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 msgr2=0x7f1ecc061600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 0x7f1ecc061600 secure :-1 s=READY pgs=2744 cs=0 l=1 rev1=1 crypto rx=0x7f1ec400cad0 tx=0x7f1ec400ced0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 shutdown_connections 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1eac06b4d0 0x7f1eac06d990 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ecc1615a0 0x7f1ecc061600 unknown :-1 s=CLOSED pgs=2744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ecc15db90 0x7f1ecc15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 --2- 172.21.15.165:0/1274162672 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ecc0d04f0 0x7f1ecc14e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 >> 172.21.15.165:0/1274162672 conn(0x7f1ecc0b52b0 msgr2=0x7f1ecc1e7400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 shutdown_connections 2024-04-25T10:20:43.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.024+0000 7f1ecbe00640 1 -- 172.21.15.165:0/1274162672 wait complete. 2024-04-25T10:20:43.047 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":753,"stamp":"2024-04-25T10:20:42.510198+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":160,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":541820,"kb_used_data":20488,"kb_used_omap":12,"kb_used_meta":521203,"kb_avail":754432900,"statfs":{"total":773094113280,"available":772539289600,"internally_reserved":0,"allocated":20979712,"data_stored":10472484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":533712434},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002105"},"pg_stats":[{"pgid":"30.3f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655375+0000","last_change":"2024-04-25T10:20:38.158137+0000","last_active":"2024-04-25T10:20:40.655375+0000","last_peered":"2024-04-25T10:20:40.655375+0000","last_clean":"2024-04-25T10:20:40.655375+0000","last_became_active":"2024-04-25T10:20:38.157081+0000","last_became_peered":"2024-04-25T10:20:38.157081+0000","last_unstale":"2024-04-25T10:20:40.655375+0000","last_undegraded":"2024-04-25T10:20:40.655375+0000","last_fullsized":"2024-04-25T10:20:40.655375+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:46:06.657804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149774+0000","last_change":"2024-04-25T10:20:38.152055+0000","last_active":"2024-04-25T10:20:39.149774+0000","last_peered":"2024-04-25T10:20:39.149774+0000","last_clean":"2024-04-25T10:20:39.149774+0000","last_became_active":"2024-04-25T10:20:38.151754+0000","last_became_peered":"2024-04-25T10:20:38.151754+0000","last_unstale":"2024-04-25T10:20:39.149774+0000","last_undegraded":"2024-04-25T10:20:39.149774+0000","last_fullsized":"2024-04-25T10:20:39.149774+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:41:54.309372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.3d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654579+0000","last_change":"2024-04-25T10:20:38.158759+0000","last_active":"2024-04-25T10:20:40.654579+0000","last_peered":"2024-04-25T10:20:40.654579+0000","last_clean":"2024-04-25T10:20:40.654579+0000","last_became_active":"2024-04-25T10:20:38.158237+0000","last_became_peered":"2024-04-25T10:20:38.158237+0000","last_unstale":"2024-04-25T10:20:40.654579+0000","last_undegraded":"2024-04-25T10:20:40.654579+0000","last_fullsized":"2024-04-25T10:20:40.654579+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:53:30.430305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.3c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152828+0000","last_change":"2024-04-25T10:20:38.150529+0000","last_active":"2024-04-25T10:20:39.152828+0000","last_peered":"2024-04-25T10:20:39.152828+0000","last_clean":"2024-04-25T10:20:39.152828+0000","last_became_active":"2024-04-25T10:20:38.150218+0000","last_became_peered":"2024-04-25T10:20:38.150218+0000","last_unstale":"2024-04-25T10:20:39.152828+0000","last_undegraded":"2024-04-25T10:20:39.152828+0000","last_fullsized":"2024-04-25T10:20:39.152828+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:07.490219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.3b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656271+0000","last_change":"2024-04-25T10:20:38.153633+0000","last_active":"2024-04-25T10:20:40.656271+0000","last_peered":"2024-04-25T10:20:40.656271+0000","last_clean":"2024-04-25T10:20:40.656271+0000","last_became_active":"2024-04-25T10:20:38.153300+0000","last_became_peered":"2024-04-25T10:20:38.153300+0000","last_unstale":"2024-04-25T10:20:40.656271+0000","last_undegraded":"2024-04-25T10:20:40.656271+0000","last_fullsized":"2024-04-25T10:20:40.656271+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:48:53.408618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653896+0000","last_change":"2024-04-25T10:20:38.148674+0000","last_active":"2024-04-25T10:20:40.653896+0000","last_peered":"2024-04-25T10:20:40.653896+0000","last_clean":"2024-04-25T10:20:40.653896+0000","last_became_active":"2024-04-25T10:20:38.148398+0000","last_became_peered":"2024-04-25T10:20:38.148398+0000","last_unstale":"2024-04-25T10:20:40.653896+0000","last_undegraded":"2024-04-25T10:20:40.653896+0000","last_fullsized":"2024-04-25T10:20:40.653896+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:20.159076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.39","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654166+0000","last_change":"2024-04-25T10:20:38.153566+0000","last_active":"2024-04-25T10:20:40.654166+0000","last_peered":"2024-04-25T10:20:40.654166+0000","last_clean":"2024-04-25T10:20:40.654166+0000","last_became_active":"2024-04-25T10:20:38.153304+0000","last_became_peered":"2024-04-25T10:20:38.153304+0000","last_unstale":"2024-04-25T10:20:40.654166+0000","last_undegraded":"2024-04-25T10:20:40.654166+0000","last_fullsized":"2024-04-25T10:20:40.654166+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:01:24.946068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.38","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653131+0000","last_change":"2024-04-25T10:20:38.143360+0000","last_active":"2024-04-25T10:20:40.653131+0000","last_peered":"2024-04-25T10:20:40.653131+0000","last_clean":"2024-04-25T10:20:40.653131+0000","last_became_active":"2024-04-25T10:20:38.143041+0000","last_became_peered":"2024-04-25T10:20:38.143041+0000","last_unstale":"2024-04-25T10:20:40.653131+0000","last_undegraded":"2024-04-25T10:20:40.653131+0000","last_fullsized":"2024-04-25T10:20:40.653131+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:53.986382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.37","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653646+0000","last_change":"2024-04-25T10:20:38.143759+0000","last_active":"2024-04-25T10:20:40.653646+0000","last_peered":"2024-04-25T10:20:40.653646+0000","last_clean":"2024-04-25T10:20:40.653646+0000","last_became_active":"2024-04-25T10:20:38.141717+0000","last_became_peered":"2024-04-25T10:20:38.141717+0000","last_unstale":"2024-04-25T10:20:40.653646+0000","last_undegraded":"2024-04-25T10:20:40.653646+0000","last_fullsized":"2024-04-25T10:20:40.653646+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:10:04.616110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.36","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654589+0000","last_change":"2024-04-25T10:20:38.155488+0000","last_active":"2024-04-25T10:20:40.654589+0000","last_peered":"2024-04-25T10:20:40.654589+0000","last_clean":"2024-04-25T10:20:40.654589+0000","last_became_active":"2024-04-25T10:20:38.154003+0000","last_became_peered":"2024-04-25T10:20:38.154003+0000","last_unstale":"2024-04-25T10:20:40.654589+0000","last_undegraded":"2024-04-25T10:20:40.654589+0000","last_fullsized":"2024-04-25T10:20:40.654589+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:32:06.126252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.35","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153610+0000","last_change":"2024-04-25T10:20:38.143779+0000","last_active":"2024-04-25T10:20:39.153610+0000","last_peered":"2024-04-25T10:20:39.153610+0000","last_clean":"2024-04-25T10:20:39.153610+0000","last_became_active":"2024-04-25T10:20:38.143421+0000","last_became_peered":"2024-04-25T10:20:38.143421+0000","last_unstale":"2024-04-25T10:20:39.153610+0000","last_undegraded":"2024-04-25T10:20:39.153610+0000","last_fullsized":"2024-04-25T10:20:39.153610+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:11.823190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.34","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153620+0000","last_change":"2024-04-25T10:20:38.146347+0000","last_active":"2024-04-25T10:20:39.153620+0000","last_peered":"2024-04-25T10:20:39.153620+0000","last_clean":"2024-04-25T10:20:39.153620+0000","last_became_active":"2024-04-25T10:20:38.143874+0000","last_became_peered":"2024-04-25T10:20:38.143874+0000","last_unstale":"2024-04-25T10:20:39.153620+0000","last_undegraded":"2024-04-25T10:20:39.153620+0000","last_fullsized":"2024-04-25T10:20:39.153620+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:32:04.980918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.33","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654188+0000","last_change":"2024-04-25T10:20:38.148082+0000","last_active":"2024-04-25T10:20:40.654188+0000","last_peered":"2024-04-25T10:20:40.654188+0000","last_clean":"2024-04-25T10:20:40.654188+0000","last_became_active":"2024-04-25T10:20:38.147759+0000","last_became_peered":"2024-04-25T10:20:38.147759+0000","last_unstale":"2024-04-25T10:20:40.654188+0000","last_undegraded":"2024-04-25T10:20:40.654188+0000","last_fullsized":"2024-04-25T10:20:40.654188+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:17.967152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.32","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154425+0000","last_change":"2024-04-25T10:20:38.149538+0000","last_active":"2024-04-25T10:20:39.154425+0000","last_peered":"2024-04-25T10:20:39.154425+0000","last_clean":"2024-04-25T10:20:39.154425+0000","last_became_active":"2024-04-25T10:20:38.149208+0000","last_became_peered":"2024-04-25T10:20:38.149208+0000","last_unstale":"2024-04-25T10:20:39.154425+0000","last_undegraded":"2024-04-25T10:20:39.154425+0000","last_fullsized":"2024-04-25T10:20:39.154425+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:34:06.263834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.31","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157117+0000","last_change":"2024-04-25T10:20:38.138879+0000","last_active":"2024-04-25T10:20:39.157117+0000","last_peered":"2024-04-25T10:20:39.157117+0000","last_clean":"2024-04-25T10:20:39.157117+0000","last_became_active":"2024-04-25T10:20:38.138557+0000","last_became_peered":"2024-04-25T10:20:38.138557+0000","last_unstale":"2024-04-25T10:20:39.157117+0000","last_undegraded":"2024-04-25T10:20:39.157117+0000","last_fullsized":"2024-04-25T10:20:39.157117+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:44:46.825672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.30","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653364+0000","last_change":"2024-04-25T10:20:38.148469+0000","last_active":"2024-04-25T10:20:40.653364+0000","last_peered":"2024-04-25T10:20:40.653364+0000","last_clean":"2024-04-25T10:20:40.653364+0000","last_became_active":"2024-04-25T10:20:38.148027+0000","last_became_peered":"2024-04-25T10:20:38.148027+0000","last_unstale":"2024-04-25T10:20:40.653364+0000","last_undegraded":"2024-04-25T10:20:40.653364+0000","last_fullsized":"2024-04-25T10:20:40.653364+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:05.710268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154411+0000","last_change":"2024-04-25T10:20:38.155341+0000","last_active":"2024-04-25T10:20:39.154411+0000","last_peered":"2024-04-25T10:20:39.154411+0000","last_clean":"2024-04-25T10:20:39.154411+0000","last_became_active":"2024-04-25T10:20:38.154987+0000","last_became_peered":"2024-04-25T10:20:38.154987+0000","last_unstale":"2024-04-25T10:20:39.154411+0000","last_undegraded":"2024-04-25T10:20:39.154411+0000","last_fullsized":"2024-04-25T10:20:39.154411+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:25.579823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.2e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654536+0000","last_change":"2024-04-25T10:20:38.152992+0000","last_active":"2024-04-25T10:20:40.654536+0000","last_peered":"2024-04-25T10:20:40.654536+0000","last_clean":"2024-04-25T10:20:40.654536+0000","last_became_active":"2024-04-25T10:20:38.151638+0000","last_became_peered":"2024-04-25T10:20:38.151638+0000","last_unstale":"2024-04-25T10:20:40.654536+0000","last_undegraded":"2024-04-25T10:20:40.654536+0000","last_fullsized":"2024-04-25T10:20:40.654536+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:59:58.413715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655755+0000","last_change":"2024-04-25T10:20:38.158884+0000","last_active":"2024-04-25T10:20:40.655755+0000","last_peered":"2024-04-25T10:20:40.655755+0000","last_clean":"2024-04-25T10:20:40.655755+0000","last_became_active":"2024-04-25T10:20:38.158240+0000","last_became_peered":"2024-04-25T10:20:38.158240+0000","last_unstale":"2024-04-25T10:20:40.655755+0000","last_undegraded":"2024-04-25T10:20:40.655755+0000","last_fullsized":"2024-04-25T10:20:40.655755+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:02:39.462733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.2c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654657+0000","last_change":"2024-04-25T10:20:38.138958+0000","last_active":"2024-04-25T10:20:40.654657+0000","last_peered":"2024-04-25T10:20:40.654657+0000","last_clean":"2024-04-25T10:20:40.654657+0000","last_became_active":"2024-04-25T10:20:38.138641+0000","last_became_peered":"2024-04-25T10:20:38.138641+0000","last_unstale":"2024-04-25T10:20:40.654657+0000","last_undegraded":"2024-04-25T10:20:40.654657+0000","last_fullsized":"2024-04-25T10:20:40.654657+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:55:52.328008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.2b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653928+0000","last_change":"2024-04-25T10:20:38.157853+0000","last_active":"2024-04-25T10:20:40.653928+0000","last_peered":"2024-04-25T10:20:40.653928+0000","last_clean":"2024-04-25T10:20:40.653928+0000","last_became_active":"2024-04-25T10:20:38.157538+0000","last_became_peered":"2024-04-25T10:20:38.157538+0000","last_unstale":"2024-04-25T10:20:40.653928+0000","last_undegraded":"2024-04-25T10:20:40.653928+0000","last_fullsized":"2024-04-25T10:20:40.653928+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:52:25.739340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.2a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.156934+0000","last_change":"2024-04-25T10:20:38.133820+0000","last_active":"2024-04-25T10:20:39.156934+0000","last_peered":"2024-04-25T10:20:39.156934+0000","last_clean":"2024-04-25T10:20:39.156934+0000","last_became_active":"2024-04-25T10:20:38.133119+0000","last_became_peered":"2024-04-25T10:20:38.133119+0000","last_unstale":"2024-04-25T10:20:39.156934+0000","last_undegraded":"2024-04-25T10:20:39.156934+0000","last_fullsized":"2024-04-25T10:20:39.156934+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:23:29.226886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.29","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653192+0000","last_change":"2024-04-25T10:20:38.145153+0000","last_active":"2024-04-25T10:20:40.653192+0000","last_peered":"2024-04-25T10:20:40.653192+0000","last_clean":"2024-04-25T10:20:40.653192+0000","last_became_active":"2024-04-25T10:20:38.144605+0000","last_became_peered":"2024-04-25T10:20:38.144605+0000","last_unstale":"2024-04-25T10:20:40.653192+0000","last_undegraded":"2024-04-25T10:20:40.653192+0000","last_fullsized":"2024-04-25T10:20:40.653192+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:17.629936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.28","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654904+0000","last_change":"2024-04-25T10:20:38.157333+0000","last_active":"2024-04-25T10:20:40.654904+0000","last_peered":"2024-04-25T10:20:40.654904+0000","last_clean":"2024-04-25T10:20:40.654904+0000","last_became_active":"2024-04-25T10:20:38.156480+0000","last_became_peered":"2024-04-25T10:20:38.156480+0000","last_unstale":"2024-04-25T10:20:40.654904+0000","last_undegraded":"2024-04-25T10:20:40.654904+0000","last_fullsized":"2024-04-25T10:20:40.654904+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:29:36.309315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.27","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655865+0000","last_change":"2024-04-25T10:20:38.160513+0000","last_active":"2024-04-25T10:20:40.655865+0000","last_peered":"2024-04-25T10:20:40.655865+0000","last_clean":"2024-04-25T10:20:40.655865+0000","last_became_active":"2024-04-25T10:20:38.160210+0000","last_became_peered":"2024-04-25T10:20:38.160210+0000","last_unstale":"2024-04-25T10:20:40.655865+0000","last_undegraded":"2024-04-25T10:20:40.655865+0000","last_fullsized":"2024-04-25T10:20:40.655865+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:40:03.777294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.26","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654644+0000","last_change":"2024-04-25T10:20:38.152751+0000","last_active":"2024-04-25T10:20:40.654644+0000","last_peered":"2024-04-25T10:20:40.654644+0000","last_clean":"2024-04-25T10:20:40.654644+0000","last_became_active":"2024-04-25T10:20:38.151952+0000","last_became_peered":"2024-04-25T10:20:38.151952+0000","last_unstale":"2024-04-25T10:20:40.654644+0000","last_undegraded":"2024-04-25T10:20:40.654644+0000","last_fullsized":"2024-04-25T10:20:40.654644+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:28:14.719747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.25","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154944+0000","last_change":"2024-04-25T10:20:38.149351+0000","last_active":"2024-04-25T10:20:39.154944+0000","last_peered":"2024-04-25T10:20:39.154944+0000","last_clean":"2024-04-25T10:20:39.154944+0000","last_became_active":"2024-04-25T10:20:38.149036+0000","last_became_peered":"2024-04-25T10:20:38.149036+0000","last_unstale":"2024-04-25T10:20:39.154944+0000","last_undegraded":"2024-04-25T10:20:39.154944+0000","last_fullsized":"2024-04-25T10:20:39.154944+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:51.700989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.24","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153989+0000","last_change":"2024-04-25T10:20:38.149611+0000","last_active":"2024-04-25T10:20:39.153989+0000","last_peered":"2024-04-25T10:20:39.153989+0000","last_clean":"2024-04-25T10:20:39.153989+0000","last_became_active":"2024-04-25T10:20:38.149329+0000","last_became_peered":"2024-04-25T10:20:38.149329+0000","last_unstale":"2024-04-25T10:20:39.153989+0000","last_undegraded":"2024-04-25T10:20:39.153989+0000","last_fullsized":"2024-04-25T10:20:39.153989+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:26:41.813659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.23","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149214+0000","last_change":"2024-04-25T10:20:38.148450+0000","last_active":"2024-04-25T10:20:39.149214+0000","last_peered":"2024-04-25T10:20:39.149214+0000","last_clean":"2024-04-25T10:20:39.149214+0000","last_became_active":"2024-04-25T10:20:38.148180+0000","last_became_peered":"2024-04-25T10:20:38.148180+0000","last_unstale":"2024-04-25T10:20:39.149214+0000","last_undegraded":"2024-04-25T10:20:39.149214+0000","last_fullsized":"2024-04-25T10:20:39.149214+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:38:54.726129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.22","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653782+0000","last_change":"2024-04-25T10:20:38.133548+0000","last_active":"2024-04-25T10:20:40.653782+0000","last_peered":"2024-04-25T10:20:40.653782+0000","last_clean":"2024-04-25T10:20:40.653782+0000","last_became_active":"2024-04-25T10:20:38.133130+0000","last_became_peered":"2024-04-25T10:20:38.133130+0000","last_unstale":"2024-04-25T10:20:40.653782+0000","last_undegraded":"2024-04-25T10:20:40.653782+0000","last_fullsized":"2024-04-25T10:20:40.653782+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:35.922881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.21","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653031+0000","last_change":"2024-04-25T10:20:38.144924+0000","last_active":"2024-04-25T10:20:40.653031+0000","last_peered":"2024-04-25T10:20:40.653031+0000","last_clean":"2024-04-25T10:20:40.653031+0000","last_became_active":"2024-04-25T10:20:38.144284+0000","last_became_peered":"2024-04-25T10:20:38.144284+0000","last_unstale":"2024-04-25T10:20:40.653031+0000","last_undegraded":"2024-04-25T10:20:40.653031+0000","last_fullsized":"2024-04-25T10:20:40.653031+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:31.014261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.20","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654627+0000","last_change":"2024-04-25T10:20:38.146389+0000","last_active":"2024-04-25T10:20:40.654627+0000","last_peered":"2024-04-25T10:20:40.654627+0000","last_clean":"2024-04-25T10:20:40.654627+0000","last_became_active":"2024-04-25T10:20:38.146072+0000","last_became_peered":"2024-04-25T10:20:38.146072+0000","last_unstale":"2024-04-25T10:20:40.654627+0000","last_undegraded":"2024-04-25T10:20:40.654627+0000","last_fullsized":"2024-04-25T10:20:40.654627+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:53:36.425253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654694+0000","last_change":"2024-04-25T10:20:38.151572+0000","last_active":"2024-04-25T10:20:40.654694+0000","last_peered":"2024-04-25T10:20:40.654694+0000","last_clean":"2024-04-25T10:20:40.654694+0000","last_became_active":"2024-04-25T10:20:38.151293+0000","last_became_peered":"2024-04-25T10:20:38.151293+0000","last_unstale":"2024-04-25T10:20:40.654694+0000","last_undegraded":"2024-04-25T10:20:40.654694+0000","last_fullsized":"2024-04-25T10:20:40.654694+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:57:43.505763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"194'4","reported_seq":18,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653631+0000","last_change":"2024-04-25T10:20:37.148775+0000","last_active":"2024-04-25T10:20:40.653631+0000","last_peered":"2024-04-25T10:20:40.653631+0000","last_clean":"2024-04-25T10:20:40.653631+0000","last_became_active":"2024-04-25T10:20:37.148473+0000","last_became_peered":"2024-04-25T10:20:37.148473+0000","last_unstale":"2024-04-25T10:20:40.653631+0000","last_undegraded":"2024-04-25T10:20:40.653631+0000","last_fullsized":"2024-04-25T10:20:40.653631+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:50:47.734938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.1d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654886+0000","last_change":"2024-04-25T10:20:38.151096+0000","last_active":"2024-04-25T10:20:40.654886+0000","last_peered":"2024-04-25T10:20:40.654886+0000","last_clean":"2024-04-25T10:20:40.654886+0000","last_became_active":"2024-04-25T10:20:38.149126+0000","last_became_peered":"2024-04-25T10:20:38.149126+0000","last_unstale":"2024-04-25T10:20:40.654886+0000","last_undegraded":"2024-04-25T10:20:40.654886+0000","last_fullsized":"2024-04-25T10:20:40.654886+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:25:36.595475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153482+0000","last_change":"2024-04-25T10:20:37.153758+0000","last_active":"2024-04-25T10:20:39.153482+0000","last_peered":"2024-04-25T10:20:39.153482+0000","last_clean":"2024-04-25T10:20:39.153482+0000","last_became_active":"2024-04-25T10:20:37.153417+0000","last_became_peered":"2024-04-25T10:20:37.153417+0000","last_unstale":"2024-04-25T10:20:39.153482+0000","last_undegraded":"2024-04-25T10:20:39.153482+0000","last_fullsized":"2024-04-25T10:20:39.153482+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:42:33.869742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153495+0000","last_change":"2024-04-25T10:20:38.146111+0000","last_active":"2024-04-25T10:20:39.153495+0000","last_peered":"2024-04-25T10:20:39.153495+0000","last_clean":"2024-04-25T10:20:39.153495+0000","last_became_active":"2024-04-25T10:20:38.145742+0000","last_became_peered":"2024-04-25T10:20:38.145742+0000","last_unstale":"2024-04-25T10:20:39.153495+0000","last_undegraded":"2024-04-25T10:20:39.153495+0000","last_fullsized":"2024-04-25T10:20:39.153495+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:39:22.414400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654300+0000","last_change":"2024-04-25T10:20:37.167453+0000","last_active":"2024-04-25T10:20:40.654300+0000","last_peered":"2024-04-25T10:20:40.654300+0000","last_clean":"2024-04-25T10:20:40.654300+0000","last_became_active":"2024-04-25T10:20:37.167139+0000","last_became_peered":"2024-04-25T10:20:37.167139+0000","last_unstale":"2024-04-25T10:20:40.654300+0000","last_undegraded":"2024-04-25T10:20:40.654300+0000","last_fullsized":"2024-04-25T10:20:40.654300+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:08:12.366774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653727+0000","last_change":"2024-04-25T10:20:38.151185+0000","last_active":"2024-04-25T10:20:40.653727+0000","last_peered":"2024-04-25T10:20:40.653727+0000","last_clean":"2024-04-25T10:20:40.653727+0000","last_became_active":"2024-04-25T10:20:38.150898+0000","last_became_peered":"2024-04-25T10:20:38.150898+0000","last_unstale":"2024-04-25T10:20:40.653727+0000","last_undegraded":"2024-04-25T10:20:40.653727+0000","last_fullsized":"2024-04-25T10:20:40.653727+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:41:41.633955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157115+0000","last_change":"2024-04-25T10:20:37.163407+0000","last_active":"2024-04-25T10:20:39.157115+0000","last_peered":"2024-04-25T10:20:39.157115+0000","last_clean":"2024-04-25T10:20:39.157115+0000","last_became_active":"2024-04-25T10:20:37.163155+0000","last_became_peered":"2024-04-25T10:20:37.163155+0000","last_unstale":"2024-04-25T10:20:39.157115+0000","last_undegraded":"2024-04-25T10:20:39.157115+0000","last_fullsized":"2024-04-25T10:20:39.157115+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:10:52.969137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152762+0000","last_change":"2024-04-25T10:20:38.151729+0000","last_active":"2024-04-25T10:20:39.152762+0000","last_peered":"2024-04-25T10:20:39.152762+0000","last_clean":"2024-04-25T10:20:39.152762+0000","last_became_active":"2024-04-25T10:20:38.151419+0000","last_became_peered":"2024-04-25T10:20:38.151419+0000","last_unstale":"2024-04-25T10:20:39.152762+0000","last_undegraded":"2024-04-25T10:20:39.152762+0000","last_fullsized":"2024-04-25T10:20:39.152762+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:42:45.659241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655164+0000","last_change":"2024-04-25T10:20:37.165683+0000","last_active":"2024-04-25T10:20:40.655164+0000","last_peered":"2024-04-25T10:20:40.655164+0000","last_clean":"2024-04-25T10:20:40.655164+0000","last_became_active":"2024-04-25T10:20:37.165369+0000","last_became_peered":"2024-04-25T10:20:37.165369+0000","last_unstale":"2024-04-25T10:20:40.655164+0000","last_undegraded":"2024-04-25T10:20:40.655164+0000","last_fullsized":"2024-04-25T10:20:40.655164+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:28:19.141831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157133+0000","last_change":"2024-04-25T10:20:38.132511+0000","last_active":"2024-04-25T10:20:39.157133+0000","last_peered":"2024-04-25T10:20:39.157133+0000","last_clean":"2024-04-25T10:20:39.157133+0000","last_became_active":"2024-04-25T10:20:38.131839+0000","last_became_peered":"2024-04-25T10:20:38.131839+0000","last_unstale":"2024-04-25T10:20:39.157133+0000","last_undegraded":"2024-04-25T10:20:39.157133+0000","last_fullsized":"2024-04-25T10:20:39.157133+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:46:25.387519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.151523+0000","last_change":"2024-04-25T10:20:37.163893+0000","last_active":"2024-04-25T10:20:39.151523+0000","last_peered":"2024-04-25T10:20:39.151523+0000","last_clean":"2024-04-25T10:20:39.151523+0000","last_became_active":"2024-04-25T10:20:37.163536+0000","last_became_peered":"2024-04-25T10:20:37.163536+0000","last_unstale":"2024-04-25T10:20:39.151523+0000","last_undegraded":"2024-04-25T10:20:39.151523+0000","last_fullsized":"2024-04-25T10:20:39.151523+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:51:37.584972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653891+0000","last_change":"2024-04-25T10:20:38.145293+0000","last_active":"2024-04-25T10:20:40.653891+0000","last_peered":"2024-04-25T10:20:40.653891+0000","last_clean":"2024-04-25T10:20:40.653891+0000","last_became_active":"2024-04-25T10:20:38.145032+0000","last_became_peered":"2024-04-25T10:20:38.145032+0000","last_unstale":"2024-04-25T10:20:40.653891+0000","last_undegraded":"2024-04-25T10:20:40.653891+0000","last_fullsized":"2024-04-25T10:20:40.653891+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:21.313336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653971+0000","last_change":"2024-04-25T10:20:37.144223+0000","last_active":"2024-04-25T10:20:40.653971+0000","last_peered":"2024-04-25T10:20:40.653971+0000","last_clean":"2024-04-25T10:20:40.653971+0000","last_became_active":"2024-04-25T10:20:37.143930+0000","last_became_peered":"2024-04-25T10:20:37.143930+0000","last_unstale":"2024-04-25T10:20:40.653971+0000","last_undegraded":"2024-04-25T10:20:40.653971+0000","last_fullsized":"2024-04-25T10:20:40.653971+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:49:12.372716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.14","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157484+0000","last_change":"2024-04-25T10:20:38.149612+0000","last_active":"2024-04-25T10:20:39.157484+0000","last_peered":"2024-04-25T10:20:39.157484+0000","last_clean":"2024-04-25T10:20:39.157484+0000","last_became_active":"2024-04-25T10:20:38.149198+0000","last_became_peered":"2024-04-25T10:20:38.149198+0000","last_unstale":"2024-04-25T10:20:39.157484+0000","last_undegraded":"2024-04-25T10:20:39.157484+0000","last_fullsized":"2024-04-25T10:20:39.157484+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:14.704044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656402+0000","last_change":"2024-04-25T10:20:37.159290+0000","last_active":"2024-04-25T10:20:40.656402+0000","last_peered":"2024-04-25T10:20:40.656402+0000","last_clean":"2024-04-25T10:20:40.656402+0000","last_became_active":"2024-04-25T10:20:37.158979+0000","last_became_peered":"2024-04-25T10:20:37.158979+0000","last_unstale":"2024-04-25T10:20:40.656402+0000","last_undegraded":"2024-04-25T10:20:40.656402+0000","last_fullsized":"2024-04-25T10:20:40.656402+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:54:53.590246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.15","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157736+0000","last_change":"2024-04-25T10:20:38.148862+0000","last_active":"2024-04-25T10:20:39.157736+0000","last_peered":"2024-04-25T10:20:39.157736+0000","last_clean":"2024-04-25T10:20:39.157736+0000","last_became_active":"2024-04-25T10:20:38.148546+0000","last_became_peered":"2024-04-25T10:20:38.148546+0000","last_unstale":"2024-04-25T10:20:39.157736+0000","last_undegraded":"2024-04-25T10:20:39.157736+0000","last_fullsized":"2024-04-25T10:20:39.157736+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:52:30.098612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153802+0000","last_change":"2024-04-25T10:20:37.164109+0000","last_active":"2024-04-25T10:20:39.153802+0000","last_peered":"2024-04-25T10:20:39.153802+0000","last_clean":"2024-04-25T10:20:39.153802+0000","last_became_active":"2024-04-25T10:20:37.163776+0000","last_became_peered":"2024-04-25T10:20:37.163776+0000","last_unstale":"2024-04-25T10:20:39.153802+0000","last_undegraded":"2024-04-25T10:20:39.153802+0000","last_fullsized":"2024-04-25T10:20:39.153802+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:04.135477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654778+0000","last_change":"2024-04-25T10:20:38.153214+0000","last_active":"2024-04-25T10:20:40.654778+0000","last_peered":"2024-04-25T10:20:40.654778+0000","last_clean":"2024-04-25T10:20:40.654778+0000","last_became_active":"2024-04-25T10:20:38.152479+0000","last_became_peered":"2024-04-25T10:20:38.152479+0000","last_unstale":"2024-04-25T10:20:40.654778+0000","last_undegraded":"2024-04-25T10:20:40.654778+0000","last_fullsized":"2024-04-25T10:20:40.654778+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:55:49.955232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"194'3","reported_seq":17,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656326+0000","last_change":"2024-04-25T10:20:37.157112+0000","last_active":"2024-04-25T10:20:40.656326+0000","last_peered":"2024-04-25T10:20:40.656326+0000","last_clean":"2024-04-25T10:20:40.656326+0000","last_became_active":"2024-04-25T10:20:37.156824+0000","last_became_peered":"2024-04-25T10:20:37.156824+0000","last_unstale":"2024-04-25T10:20:40.656326+0000","last_undegraded":"2024-04-25T10:20:40.656326+0000","last_fullsized":"2024-04-25T10:20:40.656326+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:22:37.251772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153852+0000","last_change":"2024-04-25T10:20:38.157865+0000","last_active":"2024-04-25T10:20:39.153852+0000","last_peered":"2024-04-25T10:20:39.153852+0000","last_clean":"2024-04-25T10:20:39.153852+0000","last_became_active":"2024-04-25T10:20:38.157219+0000","last_became_peered":"2024-04-25T10:20:38.157219+0000","last_unstale":"2024-04-25T10:20:39.153852+0000","last_undegraded":"2024-04-25T10:20:39.153852+0000","last_fullsized":"2024-04-25T10:20:39.153852+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:13.193616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"194'6","reported_seq":20,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655039+0000","last_change":"2024-04-25T10:20:37.165749+0000","last_active":"2024-04-25T10:20:40.655039+0000","last_peered":"2024-04-25T10:20:40.655039+0000","last_clean":"2024-04-25T10:20:40.655039+0000","last_became_active":"2024-04-25T10:20:37.165461+0000","last_became_peered":"2024-04-25T10:20:37.165461+0000","last_unstale":"2024-04-25T10:20:40.655039+0000","last_undegraded":"2024-04-25T10:20:40.655039+0000","last_fullsized":"2024-04-25T10:20:40.655039+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:10:10.346221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.10","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149228+0000","last_change":"2024-04-25T10:20:38.143202+0000","last_active":"2024-04-25T10:20:39.149228+0000","last_peered":"2024-04-25T10:20:39.149228+0000","last_clean":"2024-04-25T10:20:39.149228+0000","last_became_active":"2024-04-25T10:20:38.142743+0000","last_became_peered":"2024-04-25T10:20:38.142743+0000","last_unstale":"2024-04-25T10:20:39.149228+0000","last_undegraded":"2024-04-25T10:20:39.149228+0000","last_fullsized":"2024-04-25T10:20:39.149228+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:39:53.142942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154867+0000","last_change":"2024-04-25T10:20:37.157617+0000","last_active":"2024-04-25T10:20:39.154867+0000","last_peered":"2024-04-25T10:20:39.154867+0000","last_clean":"2024-04-25T10:20:39.154867+0000","last_became_active":"2024-04-25T10:20:37.156570+0000","last_became_peered":"2024-04-25T10:20:37.156570+0000","last_unstale":"2024-04-25T10:20:39.154867+0000","last_undegraded":"2024-04-25T10:20:39.154867+0000","last_fullsized":"2024-04-25T10:20:39.154867+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:33:04.135477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.153064+0000","last_change":"2024-04-25T10:20:38.132416+0000","last_active":"2024-04-25T10:20:39.153064+0000","last_peered":"2024-04-25T10:20:39.153064+0000","last_clean":"2024-04-25T10:20:39.153064+0000","last_became_active":"2024-04-25T10:20:38.131237+0000","last_became_peered":"2024-04-25T10:20:38.131237+0000","last_unstale":"2024-04-25T10:20:39.153064+0000","last_undegraded":"2024-04-25T10:20:39.153064+0000","last_fullsized":"2024-04-25T10:20:39.153064+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:07.882595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.148987+0000","last_change":"2024-04-25T10:20:37.147339+0000","last_active":"2024-04-25T10:20:39.148987+0000","last_peered":"2024-04-25T10:20:39.148987+0000","last_clean":"2024-04-25T10:20:39.148987+0000","last_became_active":"2024-04-25T10:20:37.146325+0000","last_became_peered":"2024-04-25T10:20:37.146325+0000","last_unstale":"2024-04-25T10:20:39.148987+0000","last_undegraded":"2024-04-25T10:20:39.148987+0000","last_fullsized":"2024-04-25T10:20:39.148987+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:24:49.160760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654268+0000","last_change":"2024-04-25T10:20:38.156176+0000","last_active":"2024-04-25T10:20:40.654268+0000","last_peered":"2024-04-25T10:20:40.654268+0000","last_clean":"2024-04-25T10:20:40.654268+0000","last_became_active":"2024-04-25T10:20:38.154594+0000","last_became_peered":"2024-04-25T10:20:38.154594+0000","last_unstale":"2024-04-25T10:20:40.654268+0000","last_undegraded":"2024-04-25T10:20:40.654268+0000","last_fullsized":"2024-04-25T10:20:40.654268+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:12:39.451524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653212+0000","last_change":"2024-04-25T10:20:37.159768+0000","last_active":"2024-04-25T10:20:40.653212+0000","last_peered":"2024-04-25T10:20:40.653212+0000","last_clean":"2024-04-25T10:20:40.653212+0000","last_became_active":"2024-04-25T10:20:37.159369+0000","last_became_peered":"2024-04-25T10:20:37.159369+0000","last_unstale":"2024-04-25T10:20:40.653212+0000","last_undegraded":"2024-04-25T10:20:40.653212+0000","last_fullsized":"2024-04-25T10:20:40.653212+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:10:59.043179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.13","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157283+0000","last_change":"2024-04-25T10:20:38.140775+0000","last_active":"2024-04-25T10:20:39.157283+0000","last_peered":"2024-04-25T10:20:39.157283+0000","last_clean":"2024-04-25T10:20:39.157283+0000","last_became_active":"2024-04-25T10:20:38.140036+0000","last_became_peered":"2024-04-25T10:20:38.140036+0000","last_unstale":"2024-04-25T10:20:39.157283+0000","last_undegraded":"2024-04-25T10:20:39.157283+0000","last_fullsized":"2024-04-25T10:20:39.157283+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:42:14.538700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"194'8","reported_seq":22,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152084+0000","last_change":"2024-04-25T10:20:37.162113+0000","last_active":"2024-04-25T10:20:39.152084+0000","last_peered":"2024-04-25T10:20:39.152084+0000","last_clean":"2024-04-25T10:20:39.152084+0000","last_became_active":"2024-04-25T10:20:37.153138+0000","last_became_peered":"2024-04-25T10:20:37.153138+0000","last_unstale":"2024-04-25T10:20:39.152084+0000","last_undegraded":"2024-04-25T10:20:39.152084+0000","last_fullsized":"2024-04-25T10:20:39.152084+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:00:44.662847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149413+0000","last_change":"2024-04-25T10:20:38.149496+0000","last_active":"2024-04-25T10:20:39.149413+0000","last_peered":"2024-04-25T10:20:39.149413+0000","last_clean":"2024-04-25T10:20:39.149413+0000","last_became_active":"2024-04-25T10:20:38.149219+0000","last_became_peered":"2024-04-25T10:20:38.149219+0000","last_unstale":"2024-04-25T10:20:39.149413+0000","last_undegraded":"2024-04-25T10:20:39.149413+0000","last_fullsized":"2024-04-25T10:20:39.149413+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:20:26.953418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654647+0000","last_change":"2024-04-25T10:20:37.161281+0000","last_active":"2024-04-25T10:20:40.654647+0000","last_peered":"2024-04-25T10:20:40.654647+0000","last_clean":"2024-04-25T10:20:40.654647+0000","last_became_active":"2024-04-25T10:20:37.160968+0000","last_became_peered":"2024-04-25T10:20:37.160968+0000","last_unstale":"2024-04-25T10:20:40.654647+0000","last_undegraded":"2024-04-25T10:20:40.654647+0000","last_fullsized":"2024-04-25T10:20:40.654647+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:48:11.320807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.d","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656019+0000","last_change":"2024-04-25T10:20:38.154550+0000","last_active":"2024-04-25T10:20:40.656019+0000","last_peered":"2024-04-25T10:20:40.656019+0000","last_clean":"2024-04-25T10:20:40.656019+0000","last_became_active":"2024-04-25T10:20:38.153854+0000","last_became_peered":"2024-04-25T10:20:38.153854+0000","last_unstale":"2024-04-25T10:20:40.656019+0000","last_undegraded":"2024-04-25T10:20:40.656019+0000","last_fullsized":"2024-04-25T10:20:40.656019+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:55:57.460057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653297+0000","last_change":"2024-04-25T10:20:37.143337+0000","last_active":"2024-04-25T10:20:40.653297+0000","last_peered":"2024-04-25T10:20:40.653297+0000","last_clean":"2024-04-25T10:20:40.653297+0000","last_became_active":"2024-04-25T10:20:37.143047+0000","last_became_peered":"2024-04-25T10:20:37.143047+0000","last_unstale":"2024-04-25T10:20:40.653297+0000","last_undegraded":"2024-04-25T10:20:40.653297+0000","last_fullsized":"2024-04-25T10:20:40.653297+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:05:11.108214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.e","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654893+0000","last_change":"2024-04-25T10:20:38.146179+0000","last_active":"2024-04-25T10:20:40.654893+0000","last_peered":"2024-04-25T10:20:40.654893+0000","last_clean":"2024-04-25T10:20:40.654893+0000","last_became_active":"2024-04-25T10:20:38.145328+0000","last_became_peered":"2024-04-25T10:20:38.145328+0000","last_unstale":"2024-04-25T10:20:40.654893+0000","last_undegraded":"2024-04-25T10:20:40.654893+0000","last_fullsized":"2024-04-25T10:20:40.654893+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T10:52:47.083549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"194'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152825+0000","last_change":"2024-04-25T10:20:37.165828+0000","last_active":"2024-04-25T10:20:39.152825+0000","last_peered":"2024-04-25T10:20:39.152825+0000","last_clean":"2024-04-25T10:20:39.152825+0000","last_became_active":"2024-04-25T10:20:37.164409+0000","last_became_peered":"2024-04-25T10:20:37.164409+0000","last_unstale":"2024-04-25T10:20:39.152825+0000","last_undegraded":"2024-04-25T10:20:39.152825+0000","last_fullsized":"2024-04-25T10:20:39.152825+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:50:12.176371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154824+0000","last_change":"2024-04-25T10:20:38.147646+0000","last_active":"2024-04-25T10:20:39.154824+0000","last_peered":"2024-04-25T10:20:39.154824+0000","last_clean":"2024-04-25T10:20:39.154824+0000","last_became_active":"2024-04-25T10:20:38.147359+0000","last_became_peered":"2024-04-25T10:20:38.147359+0000","last_unstale":"2024-04-25T10:20:39.154824+0000","last_undegraded":"2024-04-25T10:20:39.154824+0000","last_fullsized":"2024-04-25T10:20:39.154824+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:57:43.313289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154221+0000","last_change":"2024-04-25T10:20:37.165159+0000","last_active":"2024-04-25T10:20:39.154221+0000","last_peered":"2024-04-25T10:20:39.154221+0000","last_clean":"2024-04-25T10:20:39.154221+0000","last_became_active":"2024-04-25T10:20:37.164847+0000","last_became_peered":"2024-04-25T10:20:37.164847+0000","last_unstale":"2024-04-25T10:20:39.154221+0000","last_undegraded":"2024-04-25T10:20:39.154221+0000","last_fullsized":"2024-04-25T10:20:39.154221+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:02:22.587459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.151830+0000","last_change":"2024-04-25T10:20:38.155691+0000","last_active":"2024-04-25T10:20:39.151830+0000","last_peered":"2024-04-25T10:20:39.151830+0000","last_clean":"2024-04-25T10:20:39.151830+0000","last_became_active":"2024-04-25T10:20:38.155153+0000","last_became_peered":"2024-04-25T10:20:38.155153+0000","last_unstale":"2024-04-25T10:20:39.151830+0000","last_undegraded":"2024-04-25T10:20:39.151830+0000","last_fullsized":"2024-04-25T10:20:39.151830+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:14:50.214563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.155138+0000","last_change":"2024-04-25T10:20:37.157357+0000","last_active":"2024-04-25T10:20:39.155138+0000","last_peered":"2024-04-25T10:20:39.155138+0000","last_clean":"2024-04-25T10:20:39.155138+0000","last_became_active":"2024-04-25T10:20:37.157001+0000","last_became_peered":"2024-04-25T10:20:37.157001+0000","last_unstale":"2024-04-25T10:20:39.155138+0000","last_undegraded":"2024-04-25T10:20:39.155138+0000","last_fullsized":"2024-04-25T10:20:39.155138+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:24:10.246954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655796+0000","last_change":"2024-04-25T10:20:38.159995+0000","last_active":"2024-04-25T10:20:40.655796+0000","last_peered":"2024-04-25T10:20:40.655796+0000","last_clean":"2024-04-25T10:20:40.655796+0000","last_became_active":"2024-04-25T10:20:38.159685+0000","last_became_peered":"2024-04-25T10:20:38.159685+0000","last_unstale":"2024-04-25T10:20:40.655796+0000","last_undegraded":"2024-04-25T10:20:40.655796+0000","last_fullsized":"2024-04-25T10:20:40.655796+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:19:10.570193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157105+0000","last_change":"2024-04-25T10:20:37.161882+0000","last_active":"2024-04-25T10:20:39.157105+0000","last_peered":"2024-04-25T10:20:39.157105+0000","last_clean":"2024-04-25T10:20:39.157105+0000","last_became_active":"2024-04-25T10:20:37.161518+0000","last_became_peered":"2024-04-25T10:20:37.161518+0000","last_unstale":"2024-04-25T10:20:39.157105+0000","last_undegraded":"2024-04-25T10:20:39.157105+0000","last_fullsized":"2024-04-25T10:20:39.157105+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:06:04.693023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149071+0000","last_change":"2024-04-25T10:20:38.142476+0000","last_active":"2024-04-25T10:20:39.149071+0000","last_peered":"2024-04-25T10:20:39.149071+0000","last_clean":"2024-04-25T10:20:39.149071+0000","last_became_active":"2024-04-25T10:20:38.142197+0000","last_became_peered":"2024-04-25T10:20:38.142197+0000","last_unstale":"2024-04-25T10:20:39.149071+0000","last_undegraded":"2024-04-25T10:20:39.149071+0000","last_fullsized":"2024-04-25T10:20:39.149071+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:07:41.320349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.156955+0000","last_change":"2024-04-25T10:20:37.152463+0000","last_active":"2024-04-25T10:20:39.156955+0000","last_peered":"2024-04-25T10:20:39.156955+0000","last_clean":"2024-04-25T10:20:39.156955+0000","last_became_active":"2024-04-25T10:20:37.152172+0000","last_became_peered":"2024-04-25T10:20:37.152172+0000","last_unstale":"2024-04-25T10:20:39.156955+0000","last_undegraded":"2024-04-25T10:20:39.156955+0000","last_fullsized":"2024-04-25T10:20:39.156955+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:11:26.587710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.b","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654329+0000","last_change":"2024-04-25T10:20:38.143093+0000","last_active":"2024-04-25T10:20:40.654329+0000","last_peered":"2024-04-25T10:20:40.654329+0000","last_clean":"2024-04-25T10:20:40.654329+0000","last_became_active":"2024-04-25T10:20:38.141843+0000","last_became_peered":"2024-04-25T10:20:38.141843+0000","last_unstale":"2024-04-25T10:20:40.654329+0000","last_undegraded":"2024-04-25T10:20:40.654329+0000","last_fullsized":"2024-04-25T10:20:40.654329+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:18:10.072815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.652915+0000","last_change":"2024-04-25T10:20:37.142126+0000","last_active":"2024-04-25T10:20:40.652915+0000","last_peered":"2024-04-25T10:20:40.652915+0000","last_clean":"2024-04-25T10:20:40.652915+0000","last_became_active":"2024-04-25T10:20:37.141836+0000","last_became_peered":"2024-04-25T10:20:37.141836+0000","last_unstale":"2024-04-25T10:20:40.652915+0000","last_undegraded":"2024-04-25T10:20:40.652915+0000","last_fullsized":"2024-04-25T10:20:40.652915+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T17:42:04.960265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.149610+0000","last_change":"2024-04-25T10:20:38.148889+0000","last_active":"2024-04-25T10:20:39.149610+0000","last_peered":"2024-04-25T10:20:39.149610+0000","last_clean":"2024-04-25T10:20:39.149610+0000","last_became_active":"2024-04-25T10:20:38.148600+0000","last_became_peered":"2024-04-25T10:20:38.148600+0000","last_unstale":"2024-04-25T10:20:39.149610+0000","last_undegraded":"2024-04-25T10:20:39.149610+0000","last_fullsized":"2024-04-25T10:20:39.149610+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:47:33.738452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.155202+0000","last_change":"2024-04-25T10:20:37.157262+0000","last_active":"2024-04-25T10:20:39.155202+0000","last_peered":"2024-04-25T10:20:39.155202+0000","last_clean":"2024-04-25T10:20:39.155202+0000","last_became_active":"2024-04-25T10:20:37.156914+0000","last_became_peered":"2024-04-25T10:20:37.156914+0000","last_unstale":"2024-04-25T10:20:39.155202+0000","last_undegraded":"2024-04-25T10:20:39.155202+0000","last_fullsized":"2024-04-25T10:20:39.155202+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:18:50.683744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.656354+0000","last_change":"2024-04-25T10:20:38.159775+0000","last_active":"2024-04-25T10:20:40.656354+0000","last_peered":"2024-04-25T10:20:40.656354+0000","last_clean":"2024-04-25T10:20:40.656354+0000","last_became_active":"2024-04-25T10:20:38.159387+0000","last_became_peered":"2024-04-25T10:20:38.159387+0000","last_unstale":"2024-04-25T10:20:40.656354+0000","last_undegraded":"2024-04-25T10:20:40.656354+0000","last_fullsized":"2024-04-25T10:20:40.656354+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T21:45:31.284436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"195'1","reported_seq":15,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157694+0000","last_change":"2024-04-25T10:20:37.155823+0000","last_active":"2024-04-25T10:20:39.157694+0000","last_peered":"2024-04-25T10:20:39.157694+0000","last_clean":"2024-04-25T10:20:39.157694+0000","last_became_active":"2024-04-25T10:20:37.155544+0000","last_became_peered":"2024-04-25T10:20:37.155544+0000","last_unstale":"2024-04-25T10:20:39.157694+0000","last_undegraded":"2024-04-25T10:20:39.157694+0000","last_fullsized":"2024-04-25T10:20:39.157694+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:14:16.949907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655503+0000","last_change":"2024-04-25T10:20:38.134548+0000","last_active":"2024-04-25T10:20:40.655503+0000","last_peered":"2024-04-25T10:20:40.655503+0000","last_clean":"2024-04-25T10:20:40.655503+0000","last_became_active":"2024-04-25T10:20:38.134188+0000","last_became_peered":"2024-04-25T10:20:38.134188+0000","last_unstale":"2024-04-25T10:20:40.655503+0000","last_undegraded":"2024-04-25T10:20:40.655503+0000","last_fullsized":"2024-04-25T10:20:40.655503+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:22.735635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.157763+0000","last_change":"2024-04-25T10:20:37.144342+0000","last_active":"2024-04-25T10:20:39.157763+0000","last_peered":"2024-04-25T10:20:39.157763+0000","last_clean":"2024-04-25T10:20:39.157763+0000","last_became_active":"2024-04-25T10:20:37.143984+0000","last_became_peered":"2024-04-25T10:20:37.143984+0000","last_unstale":"2024-04-25T10:20:39.157763+0000","last_undegraded":"2024-04-25T10:20:39.157763+0000","last_fullsized":"2024-04-25T10:20:39.157763+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T11:08:34.905637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.7","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653486+0000","last_change":"2024-04-25T10:20:38.148345+0000","last_active":"2024-04-25T10:20:40.653486+0000","last_peered":"2024-04-25T10:20:40.653486+0000","last_clean":"2024-04-25T10:20:40.653486+0000","last_became_active":"2024-04-25T10:20:38.143713+0000","last_became_peered":"2024-04-25T10:20:38.143713+0000","last_unstale":"2024-04-25T10:20:40.653486+0000","last_undegraded":"2024-04-25T10:20:40.653486+0000","last_fullsized":"2024-04-25T10:20:40.653486+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T12:28:33.328586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154514+0000","last_change":"2024-04-25T10:20:37.149136+0000","last_active":"2024-04-25T10:20:39.154514+0000","last_peered":"2024-04-25T10:20:39.154514+0000","last_clean":"2024-04-25T10:20:39.154514+0000","last_became_active":"2024-04-25T10:20:37.148826+0000","last_became_peered":"2024-04-25T10:20:37.148826+0000","last_unstale":"2024-04-25T10:20:39.154514+0000","last_undegraded":"2024-04-25T10:20:39.154514+0000","last_fullsized":"2024-04-25T10:20:39.154514+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T20:11:34.039703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653945+0000","last_change":"2024-04-25T10:20:38.144929+0000","last_active":"2024-04-25T10:20:40.653945+0000","last_peered":"2024-04-25T10:20:40.653945+0000","last_clean":"2024-04-25T10:20:40.653945+0000","last_became_active":"2024-04-25T10:20:38.144584+0000","last_became_peered":"2024-04-25T10:20:38.144584+0000","last_unstale":"2024-04-25T10:20:40.653945+0000","last_undegraded":"2024-04-25T10:20:40.653945+0000","last_fullsized":"2024-04-25T10:20:40.653945+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:36:07.712669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.155199+0000","last_change":"2024-04-25T10:20:37.156319+0000","last_active":"2024-04-25T10:20:39.155199+0000","last_peered":"2024-04-25T10:20:39.155199+0000","last_clean":"2024-04-25T10:20:39.155199+0000","last_became_active":"2024-04-25T10:20:37.151537+0000","last_became_peered":"2024-04-25T10:20:37.151537+0000","last_unstale":"2024-04-25T10:20:39.155199+0000","last_undegraded":"2024-04-25T10:20:39.155199+0000","last_fullsized":"2024-04-25T10:20:39.155199+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:56:10.805946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653996+0000","last_change":"2024-04-25T10:20:38.145092+0000","last_active":"2024-04-25T10:20:40.653996+0000","last_peered":"2024-04-25T10:20:40.653996+0000","last_clean":"2024-04-25T10:20:40.653996+0000","last_became_active":"2024-04-25T10:20:38.144740+0000","last_became_peered":"2024-04-25T10:20:38.144740+0000","last_unstale":"2024-04-25T10:20:40.653996+0000","last_undegraded":"2024-04-25T10:20:40.653996+0000","last_fullsized":"2024-04-25T10:20:40.653996+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:47:09.896266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152697+0000","last_change":"2024-04-25T10:20:37.155995+0000","last_active":"2024-04-25T10:20:39.152697+0000","last_peered":"2024-04-25T10:20:39.152697+0000","last_clean":"2024-04-25T10:20:39.152697+0000","last_became_active":"2024-04-25T10:20:37.151152+0000","last_became_peered":"2024-04-25T10:20:37.151152+0000","last_unstale":"2024-04-25T10:20:39.152697+0000","last_undegraded":"2024-04-25T10:20:39.152697+0000","last_fullsized":"2024-04-25T10:20:39.152697+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T18:57:29.881912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.653492+0000","last_change":"2024-04-25T10:20:38.151550+0000","last_active":"2024-04-25T10:20:40.653492+0000","last_peered":"2024-04-25T10:20:40.653492+0000","last_clean":"2024-04-25T10:20:40.653492+0000","last_became_active":"2024-04-25T10:20:38.150406+0000","last_became_peered":"2024-04-25T10:20:38.150406+0000","last_unstale":"2024-04-25T10:20:40.653492+0000","last_undegraded":"2024-04-25T10:20:40.653492+0000","last_fullsized":"2024-04-25T10:20:40.653492+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T14:50:30.107299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655833+0000","last_change":"2024-04-25T10:20:37.160036+0000","last_active":"2024-04-25T10:20:40.655833+0000","last_peered":"2024-04-25T10:20:40.655833+0000","last_clean":"2024-04-25T10:20:40.655833+0000","last_became_active":"2024-04-25T10:20:37.159629+0000","last_became_peered":"2024-04-25T10:20:37.159629+0000","last_unstale":"2024-04-25T10:20:40.655833+0000","last_undegraded":"2024-04-25T10:20:40.655833+0000","last_fullsized":"2024-04-25T10:20:40.655833+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:33:21.718390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.154987+0000","last_change":"2024-04-25T10:20:38.142897+0000","last_active":"2024-04-25T10:20:39.154987+0000","last_peered":"2024-04-25T10:20:39.154987+0000","last_clean":"2024-04-25T10:20:39.154987+0000","last_became_active":"2024-04-25T10:20:38.142589+0000","last_became_peered":"2024-04-25T10:20:38.142589+0000","last_unstale":"2024-04-25T10:20:39.154987+0000","last_undegraded":"2024-04-25T10:20:39.154987+0000","last_fullsized":"2024-04-25T10:20:39.154987+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:11:00.060423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:39.152402+0000","last_change":"2024-04-25T10:20:37.153623+0000","last_active":"2024-04-25T10:20:39.152402+0000","last_peered":"2024-04-25T10:20:39.152402+0000","last_clean":"2024-04-25T10:20:39.152402+0000","last_became_active":"2024-04-25T10:20:37.152749+0000","last_became_peered":"2024-04-25T10:20:37.152749+0000","last_unstale":"2024-04-25T10:20:39.152402+0000","last_undegraded":"2024-04-25T10:20:39.152402+0000","last_fullsized":"2024-04-25T10:20:39.152402+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:26:17.801213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.1f","version":"0'0","reported_seq":12,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.654633+0000","last_change":"2024-04-25T10:20:38.149400+0000","last_active":"2024-04-25T10:20:40.654633+0000","last_peered":"2024-04-25T10:20:40.654633+0000","last_clean":"2024-04-25T10:20:40.654633+0000","last_became_active":"2024-04-25T10:20:38.149106+0000","last_became_peered":"2024-04-25T10:20:38.149106+0000","last_unstale":"2024-04-25T10:20:40.654633+0000","last_undegraded":"2024-04-25T10:20:40.654633+0000","last_fullsized":"2024-04-25T10:20:40.654633+0000","mapping_epoch":193,"log_start":"0'0","ondisk_log_start":"0'0","created":193,"last_epoch_clean":194,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:37.102566+0000","last_clean_scrub_stamp":"2024-04-25T10:20:37.102566+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T19:57:00.308101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"194'2","reported_seq":16,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655342+0000","last_change":"2024-04-25T10:20:37.155732+0000","last_active":"2024-04-25T10:20:40.655342+0000","last_peered":"2024-04-25T10:20:40.655342+0000","last_clean":"2024-04-25T10:20:40.655342+0000","last_became_active":"2024-04-25T10:20:37.154402+0000","last_became_peered":"2024-04-25T10:20:37.154402+0000","last_unstale":"2024-04-25T10:20:40.655342+0000","last_undegraded":"2024-04-25T10:20:40.655342+0000","last_fullsized":"2024-04-25T10:20:40.655342+0000","mapping_epoch":192,"log_start":"0'0","ondisk_log_start":"0'0","created":192,"last_epoch_clean":193,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:20:36.085322+0000","last_clean_scrub_stamp":"2024-04-25T10:20:36.085322+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T16:29:15.097625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":501,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-25T10:20:40.655175+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:20:40.655175+0000","last_peered":"2024-04-25T10:20:40.655175+0000","last_clean":"2024-04-25T10:20:40.655175+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:20:40.655175+0000","last_undegraded":"2024-04-25T10:20:40.655175+0000","last_fullsized":"2024-04-25T10:20:40.655175+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":30,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":29,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575078,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64708,"kb_used_data":2996,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307132,"statfs":{"total":96636764160,"available":96570503168,"internally_reserved":0,"allocated":3067904,"data_stored":1751906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575078,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59984,"kb_used_data":2432,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311856,"statfs":{"total":96636764160,"available":96575340544,"internally_reserved":0,"allocated":2490368,"data_stored":1162604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575078,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55816,"kb_used_data":2424,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94316024,"statfs":{"total":96636764160,"available":96579608576,"internally_reserved":0,"allocated":2482176,"data_stored":1162548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575078,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87672,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284168,"statfs":{"total":96636764160,"available":96546988032,"internally_reserved":0,"allocated":2469888,"data_stored":1160956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575078,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":87672,"kb_used_data":2412,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284168,"statfs":{"total":96636764160,"available":96546988032,"internally_reserved":0,"allocated":2469888,"data_stored":1160888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575077,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2416,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2473984,"data_stored":1161482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59964,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311876,"statfs":{"total":96636764160,"available":96575361024,"internally_reserved":0,"allocated":2465792,"data_stored":1160866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575078,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66040,"kb_used_data":2988,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305800,"statfs":{"total":96636764160,"available":96569139200,"internally_reserved":0,"allocated":3059712,"data_stored":1751234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:20:43.051 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-25T10:20:43.051 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-04-25T10:20:43.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.296+0000 7f976be00640 1 Processor -- start 2024-04-25T10:20:43.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.296+0000 7f976be00640 1 -- start start 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f976be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f976c13d9f0 0x7f976c13df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f976be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 0x7f976c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f976be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f976c06d390 con 0x7f976c0b0730 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f976be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f976c06d670 con 0x7f976c13d9f0 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f976be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c06d950 con 0x7f976c06e860 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f9763e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f976c0b0730 0x7f976c06e150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36312/0 (socket says 172.21.15.165:36312) 2024-04-25T10:20:43.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.298+0000 7f9763e00640 1 -- 172.21.15.165:0/2164599205 learned_addr learned my addr 172.21.15.165:0/2164599205 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:43.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800096327 0 0) 0x7f976c06d390 con 0x7f976c0b0730 2024-04-25T10:20:43.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9750003520 con 0x7f976c0b0730 2024-04-25T10:20:43.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976aa00640 1 --2- 172.21.15.165:0/2164599205 >> v2:172.21.15.172:3300/0 conn(0x7f976c13d9f0 0x7f976c13df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976b400640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 0x7f976c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2514640939 0 0) 0x7f9750003520 con 0x7f976c0b0730 2024-04-25T10:20:43.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 msgr2=0x7f976c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 0x7f976c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 >> v2:172.21.15.172:3300/0 conn(0x7f976c13d9f0 msgr2=0x7f976c13df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 --2- 172.21.15.165:0/2164599205 >> v2:172.21.15.172:3300/0 conn(0x7f976c13d9f0 0x7f976c13df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c06dc30 con 0x7f976c0b0730 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f9754001670 con 0x7f976c0b0730 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 >> v1:172.21.15.165:6789/0 conn(0x7f976c0b0730 legacy=0x7f976c06e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.299+0000 7f976a000640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 0x7f97500039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976a000640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f9750004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976a000640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9750004d40 0x7f97500051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c06d670 con 0x7f9750003fe0 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c06d950 con 0x7f9750004d40 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f975000de40 con 0x7f976c06e860 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f9763e00640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f9750004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976b400640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9750004d40 0x7f97500051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f9763e00640 1 -- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 msgr2=0x7f97500039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f9763e00640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c06e860 0x7f97500039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f9763e00640 1 -- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9750004d40 msgr2=0x7f97500051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f9763e00640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9750004d40 0x7f97500051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f9763e00640 1 -- 172.21.15.165:0/2164599205 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750003520 con 0x7f9750003fe0 2024-04-25T10:20:43.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.300+0000 7f976b400640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9750004d40 0x7f97500051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.301+0000 7f9763e00640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f9750004800 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f975400c210 tx=0x7f9750005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ff0293f00bd557 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754007d50 con 0x7f9750003fe0 2024-04-25T10:20:43.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9754005af0 con 0x7f9750003fe0 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976a000640 1 -- 172.21.15.165:0/2164599205 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9754007070 con 0x7f9750003fe0 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976be00640 1 -- 172.21.15.165:0/2164599205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 msgr2=0x7f9750004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976be00640 1 --2- 172.21.15.165:0/2164599205 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f9750004800 secure :-1 s=READY pgs=2746 cs=0 l=1 rev1=1 crypto rx=0x7f975400c210 tx=0x7f9750005830 comp rx=0 tx=0).stop 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976b400640 1 -- 172.21.15.165:0/2164599205 reap_dead start 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976be00640 1 -- 172.21.15.165:0/2164599205 shutdown_connections 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976be00640 1 -- 172.21.15.165:0/2164599205 >> 172.21.15.165:0/2164599205 conn(0x7f976c061cc0 msgr2=0x7f976c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976be00640 1 -- 172.21.15.165:0/2164599205 shutdown_connections 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.302+0000 7f976be00640 1 -- 172.21.15.165:0/2164599205 wait complete. 2024-04-25T10:20:43.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.304+0000 7f976be00640 1 Processor -- start 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.304+0000 7f976be00640 1 -- start start 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f976c0cea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c15d7c0 0x7f976c138000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f976c13bde0 0x7f976c13c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0b0bc0 con 0x7f9750003fe0 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976c13e680 con 0x7f976c13bde0 2024-04-25T10:20:43.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f976c0b0d40 con 0x7f976c15d7c0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f976c0cea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976aa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f976c0cea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39772/0 (socket says 172.21.15.165:39772) 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.305+0000 7f976aa00640 1 -- 172.21.15.165:0/3018243687 learned_addr learned my addr 172.21.15.165:0/3018243687 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.306+0000 7f976aa00640 1 -- 172.21.15.165:0/3018243687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c15d7c0 msgr2=0x7f976c138000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.306+0000 7f976aa00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c15d7c0 0x7f976c138000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.306+0000 7f976aa00640 1 -- 172.21.15.165:0/3018243687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f976c13bde0 msgr2=0x7f976c13c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.306+0000 7f976aa00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f976c13bde0 0x7f976c13c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.306+0000 7f976aa00640 1 -- 172.21.15.165:0/3018243687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976c0cc780 con 0x7f9750003fe0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.306+0000 7f976aa00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f976c0cea20 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d860 tx=0x7f975c00dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.307+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00de50 con 0x7f9750003fe0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.307+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975c019070 con 0x7f9750003fe0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.307+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975c00bb10 con 0x7f9750003fe0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.307+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976c0cca60 con 0x7f9750003fe0 2024-04-25T10:20:43.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.307+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976c0cd020 con 0x7f9750003fe0 2024-04-25T10:20:43.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.308+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f976c073990 con 0x7f9750003fe0 2024-04-25T10:20:43.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.309+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f975c00bcb0 con 0x7f9750003fe0 2024-04-25T10:20:43.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.310+0000 7f9762a00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f974806b4d0 0x7f974806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.310+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f975c08a7c0 con 0x7f9750003fe0 2024-04-25T10:20:43.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.311+0000 7f9763e00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f974806b4d0 0x7f974806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.311+0000 7f9763e00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f974806b4d0 0x7f974806d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9754033000 tx=0x7f9754007000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.315+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f975c059540 con 0x7f9750003fe0 2024-04-25T10:20:43.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.583+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f976c1536b0 con 0x7f9750003fe0 2024-04-25T10:20:43.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.588+0000 7f9762a00640 1 -- 172.21.15.165:0/3018243687 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v26) v1 ==== 190+0+147 (secure 0 0 0) 0x7f975c058e40 con 0x7f9750003fe0 2024-04-25T10:20:43.591 INFO:teuthology.orchestra.run.smithi165.stdout:[client.0] 2024-04-25T10:20:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: key = AQDcKSpmAMwHLxAAyBIbc5vr4WhWYS9a6L7SYw== 2024-04-25T10:20:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: caps mds = "allow" 2024-04-25T10:20:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: caps mon = "allow r" 2024-04-25T10:20:43.591 INFO:teuthology.orchestra.run.smithi165.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-04-25T10:20:43.591 INFO:teuthology.orchestra.run.smithi165.stderr:updated caps for client.0 2024-04-25T10:20:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f974806b4d0 msgr2=0x7f974806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f974806b4d0 0x7f974806d990 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f9754033000 tx=0x7f9754007000 comp rx=0 tx=0).stop 2024-04-25T10:20:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 msgr2=0x7f976c0cea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f976c0cea20 secure :-1 s=READY pgs=2747 cs=0 l=1 rev1=1 crypto rx=0x7f975c00d860 tx=0x7f975c00dd20 comp rx=0 tx=0).stop 2024-04-25T10:20:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 shutdown_connections 2024-04-25T10:20:43.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f974806b4d0 0x7f974806d990 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f976c13bde0 0x7f976c13c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f976c15d7c0 0x7f976c138000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 --2- 172.21.15.165:0/3018243687 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9750003fe0 0x7f976c0cea20 unknown :-1 s=CLOSED pgs=2747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.591+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 >> 172.21.15.165:0/3018243687 conn(0x7f976c061cc0 msgr2=0x7f976c071a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.592+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 shutdown_connections 2024-04-25T10:20:43.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.592+0000 7f976be00640 1 -- 172.21.15.165:0/3018243687 wait complete. 2024-04-25T10:20:43.614 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:43.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.811+0000 7f3d96200640 1 Processor -- start 2024-04-25T10:20:43.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.812+0000 7f3d96200640 1 -- start start 2024-04-25T10:20:43.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.813+0000 7f3d96200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3d9015db90 0x7f3d900b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.813+0000 7f3d96200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3d900d06c0 0x7f3d900d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.813+0000 7f3d96200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d90161df0 con 0x7f3d901615a0 2024-04-25T10:20:43.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.813+0000 7f3d96200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d900d1f50 con 0x7f3d900d06c0 2024-04-25T10:20:43.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.813+0000 7f3d96200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d901620d0 con 0x7f3d9015db90 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3d900d06c0 0x7f3d900d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f3d900d06c0 0x7f3d900d23e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39780/0 (socket says 172.21.15.165:39780) 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 -- 172.21.15.165:0/914360724 learned_addr learned my addr 172.21.15.165:0/914360724 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d94e00640 1 --2- 172.21.15.165:0/914360724 >> v2:172.21.15.172:3300/0 conn(0x7f3d9015db90 0x7f3d900b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 -- 172.21.15.165:0/914360724 >> v2:172.21.15.172:3300/0 conn(0x7f3d9015db90 msgr2=0x7f3d900b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 --2- 172.21.15.165:0/914360724 >> v2:172.21.15.172:3300/0 conn(0x7f3d9015db90 0x7f3d900b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 -- 172.21.15.165:0/914360724 >> v1:172.21.15.172:6789/0 conn(0x7f3d901615a0 legacy=0x7f3d900cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:43.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.814+0000 7f3d95800640 1 -- 172.21.15.165:0/914360724 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d901623b0 con 0x7f3d900d06c0 2024-04-25T10:20:43.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.815+0000 7f3d95800640 1 --2- 172.21.15.165:0/914360724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d900d06c0 0x7f3d900d23e0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f3d9006ceb0 tx=0x7f3d8802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e7ec4eecf6cf3ac server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.815+0000 7f3d8fe00640 1 -- 172.21.15.165:0/914360724 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d88036070 con 0x7f3d900d06c0 2024-04-25T10:20:43.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d8fe00640 1 -- 172.21.15.165:0/914360724 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d88031600 con 0x7f3d900d06c0 2024-04-25T10:20:43.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d8fe00640 1 -- 172.21.15.165:0/914360724 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d880319d0 con 0x7f3d900d06c0 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 -- 172.21.15.165:0/914360724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d900d06c0 msgr2=0x7f3d900d23e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 --2- 172.21.15.165:0/914360724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d900d06c0 0x7f3d900d23e0 secure :-1 s=READY pgs=2748 cs=0 l=1 rev1=1 crypto rx=0x7f3d9006ceb0 tx=0x7f3d8802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 -- 172.21.15.165:0/914360724 shutdown_connections 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 --2- 172.21.15.165:0/914360724 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d900d06c0 0x7f3d900d23e0 unknown :-1 s=CLOSED pgs=2748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 --2- 172.21.15.165:0/914360724 >> v2:172.21.15.172:3300/0 conn(0x7f3d9015db90 0x7f3d900b93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 -- 172.21.15.165:0/914360724 >> 172.21.15.165:0/914360724 conn(0x7f3d900b52b0 msgr2=0x7f3d900b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:43.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 -- 172.21.15.165:0/914360724 shutdown_connections 2024-04-25T10:20:43.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.816+0000 7f3d96200640 1 -- 172.21.15.165:0/914360724 wait complete. 2024-04-25T10:20:43.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.818+0000 7f3d96200640 1 Processor -- start 2024-04-25T10:20:43.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.818+0000 7f3d96200640 1 -- start start 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d96200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d900d06c0 0x7f3d9014ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d96200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 0x7f3d901604a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d96200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d901615a0 0x7f3d901609e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d96200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90161ba0 con 0x7f3d9015db90 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d96200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d90161a20 con 0x7f3d900d06c0 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d96200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d900b68d0 con 0x7f3d901615a0 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.819+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 0x7f3d901604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 0x7f3d901604a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39786/0 (socket says 172.21.15.165:39786) 2024-04-25T10:20:43.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 -- 172.21.15.165:0/2467126523 learned_addr learned my addr 172.21.15.165:0/2467126523 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:43.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d94e00640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d900d06c0 0x7f3d9014ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d95800640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d901615a0 0x7f3d901609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 -- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d901615a0 msgr2=0x7f3d901609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d901615a0 0x7f3d901609e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 -- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d900d06c0 msgr2=0x7f3d9014ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d900d06c0 0x7f3d9014ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 -- 172.21.15.165:0/2467126523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d9014f130 con 0x7f3d9015db90 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.820+0000 7f3d87e00640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 0x7f3d901604a0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800b3f0 tx=0x7f3d7800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.821+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78015020 con 0x7f3d9015db90 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.821+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d78011830 con 0x7f3d9015db90 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.821+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d78011c20 con 0x7f3d9015db90 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.821+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d9015ec60 con 0x7f3d9015db90 2024-04-25T10:20:43.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.821+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d9015f220 con 0x7f3d9015db90 2024-04-25T10:20:43.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.822+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d9015f8e0 con 0x7f3d9015db90 2024-04-25T10:20:43.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.823+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3d78018020 con 0x7f3d9015db90 2024-04-25T10:20:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.824+0000 7f3d8d600640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.824+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f3d78089fa0 con 0x7f3d9015db90 2024-04-25T10:20:43.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.824+0000 7f3d94e00640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:43.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.825+0000 7f3d94e00640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3d80006fd0 tx=0x7f3d80008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:43.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:43.828+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3d9015f8e0 con 0x7f3d9015db90 2024-04-25T10:20:44.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.055+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d90073990 con 0x7f3d9015db90 2024-04-25T10:20:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.056+0000 7f3d8d600640 1 -- 172.21.15.165:0/2467126523 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4963 (secure 0 0 0) 0x7f3d7801f030 con 0x7f3d9015db90 2024-04-25T10:20:44.059 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:44.059 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":139,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:40.633611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11610":{"gid":11610,"name":"c","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/3988146762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3988146762},{"type":"v1","addr":"172.21.15.165:6837","nonce":3988146762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:20:44.059 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 139 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.058+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d6c06b4d0 msgr2=0x7f3d6c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.058+0000 7f3d96200640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3d80006fd0 tx=0x7f3d80008040 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.058+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 msgr2=0x7f3d901604a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.058+0000 7f3d96200640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 0x7f3d901604a0 secure :-1 s=READY pgs=2749 cs=0 l=1 rev1=1 crypto rx=0x7f3d7800b3f0 tx=0x7f3d7800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 shutdown_connections 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3d6c06b4d0 0x7f3d6c06d990 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3d901615a0 0x7f3d901609e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3d9015db90 0x7f3d901604a0 unknown :-1 s=CLOSED pgs=2749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 --2- 172.21.15.165:0/2467126523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3d900d06c0 0x7f3d9014ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 >> 172.21.15.165:0/2467126523 conn(0x7f3d900b52b0 msgr2=0x7f3d900c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 shutdown_connections 2024-04-25T10:20:44.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.059+0000 7f3d96200640 1 -- 172.21.15.165:0/2467126523 wait complete. 2024-04-25T10:20:44.083 INFO:tasks.ceph.mds.c.smithi165.stderr:2024-04-25T10:20:44.081+0000 7f0a39600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:20:44.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 139, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:37.587736+0000', 'modified': '2024-04-25T10:20:40.633611+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11610}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11610': {'gid': 11610, 'name': 'c', 'rank': 0, 'incarnation': 135, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.165:6837/3988146762', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 3988146762}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 3988146762}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:44.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:20:44.484 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:20:44.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.691+0000 7f97fd200640 1 Processor -- start 2024-04-25T10:20:44.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.691+0000 7f97fd200640 1 -- start start 2024-04-25T10:20:44.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.693+0000 7f97fd200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 0x7f97f80b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.693+0000 7f97fd200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80cccc0 con 0x7f97f806e8a0 2024-04-25T10:20:44.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.693+0000 7f97fd200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97f80d0460 con 0x7f97f8144d50 2024-04-25T10:20:44.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.693+0000 7f97fd200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80b2b20 con 0x7f97f81446d0 2024-04-25T10:20:44.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.693+0000 7f97f6800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f97f806e8a0 0x7f97f80cfd50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36316/0 (socket says 172.21.15.165:36316) 2024-04-25T10:20:44.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.693+0000 7f97f6800640 1 -- 172.21.15.165:0/3870033463 learned_addr learned my addr 172.21.15.165:0/3870033463 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 0x7f97f80b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3923294354 0 0) 0x7f97f80cccc0 con 0x7f97f806e8a0 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97dc003520 con 0x7f97f806e8a0 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2830578027 0 0) 0x7f97dc003520 con 0x7f97f806e8a0 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 msgr2=0x7f97f80b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 0x7f97f80b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 >> v1:172.21.15.172:6789/0 conn(0x7f97f8144d50 legacy=0x7f97f80cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f80cc140 con 0x7f97f806e8a0 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f97f0001670 con 0x7f97f806e8a0 2024-04-25T10:20:44.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 >> v1:172.21.15.165:6789/0 conn(0x7f97f806e8a0 legacy=0x7f97f80cfd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97dc0039e0 0x7f97dc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 0x7f97dc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80b2b20 con 0x7f97dc0039e0 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00baa0 con 0x7f97dc004800 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.694+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97dc00bca0 con 0x7f97f81446d0 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97dc0039e0 0x7f97dc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5400640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5e00640 1 -- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 msgr2=0x7f97dc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97f81446d0 0x7f97dc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5e00640 1 -- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97dc004800 msgr2=0x7f97dc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97dc004800 0x7f97dc004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.695+0000 7f97f5e00640 1 -- 172.21.15.165:0/3870033463 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97dc003520 con 0x7f97dc0039e0 2024-04-25T10:20:44.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.696+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97dc0039e0 0x7f97dc003e00 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009f90 tx=0x7f97dc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f325e6c8ae111b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.696+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8009c10 con 0x7f97dc0039e0 2024-04-25T10:20:44.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.696+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e8036070 con 0x7f97dc0039e0 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.696+0000 7f97f4a00640 1 -- 172.21.15.165:0/3870033463 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8031850 con 0x7f97dc0039e0 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97fd200640 1 -- 172.21.15.165:0/3870033463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97dc0039e0 msgr2=0x7f97dc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97fd200640 1 --2- 172.21.15.165:0/3870033463 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97dc0039e0 0x7f97dc003e00 secure :-1 s=READY pgs=2751 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009f90 tx=0x7f97dc005830 comp rx=0 tx=0).stop 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97f6800640 1 -- 172.21.15.165:0/3870033463 reap_dead start 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97fd200640 1 -- 172.21.15.165:0/3870033463 shutdown_connections 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97fd200640 1 -- 172.21.15.165:0/3870033463 >> 172.21.15.165:0/3870033463 conn(0x7f97f806abe0 msgr2=0x7f97f806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97fd200640 1 -- 172.21.15.165:0/3870033463 shutdown_connections 2024-04-25T10:20:44.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.697+0000 7f97fd200640 1 -- 172.21.15.165:0/3870033463 wait complete. 2024-04-25T10:20:44.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.699+0000 7f97fd200640 1 Processor -- start 2024-04-25T10:20:44.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.699+0000 7f97fd200640 1 -- start start 2024-04-25T10:20:44.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97fd200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97dc0039e0 0x7f97f813c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97fd200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 0x7f97f813eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97fd200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97f80b1610 0x7f97f813f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97fd200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80cc700 con 0x7f97f80b1190 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97fd200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f806d400 con 0x7f97f80b1610 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97fd200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f80cc9e0 con 0x7f97dc0039e0 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 0x7f97f813eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 0x7f97f813eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39808/0 (socket says 172.21.15.165:39808) 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.700+0000 7f97f5400640 1 -- 172.21.15.165:0/3636235103 learned_addr learned my addr 172.21.15.165:0/3636235103 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97dc0039e0 0x7f97f813c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f6800640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97f80b1610 0x7f97f813f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5400640 1 -- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97dc0039e0 msgr2=0x7f97f813c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5400640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97dc0039e0 0x7f97f813c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5400640 1 -- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97f80b1610 msgr2=0x7f97f813f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5400640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97f80b1610 0x7f97f813f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5400640 1 -- 172.21.15.165:0/3636235103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f813e490 con 0x7f97f80b1190 2024-04-25T10:20:44.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97f5400640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 0x7f97f813eef0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f97e000d540 tx=0x7f97e000da00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97ee000640 1 -- 172.21.15.165:0/3636235103 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0005480 con 0x7f97f80b1190 2024-04-25T10:20:44.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97ee000640 1 -- 172.21.15.165:0/3636235103 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0017070 con 0x7f97f80b1190 2024-04-25T10:20:44.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97ee000640 1 -- 172.21.15.165:0/3636235103 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0003810 con 0x7f97f80b1190 2024-04-25T10:20:44.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.701+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97f813e770 con 0x7f97f80b1190 2024-04-25T10:20:44.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.702+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97f80af4e0 con 0x7f97f80b1190 2024-04-25T10:20:44.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.704+0000 7f97ee000640 1 -- 172.21.15.165:0/3636235103 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f97e00039b0 con 0x7f97f80b1190 2024-04-25T10:20:44.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.705+0000 7f97ee000640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97d406b500 0x7f97d406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.705+0000 7f97ee000640 1 -- 172.21.15.165:0/3636235103 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97e008a1c0 con 0x7f97f80b1190 2024-04-25T10:20:44.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.706+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97d406b500 0x7f97d406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.706+0000 7f980002d740 1 Processor -- start 2024-04-25T10:20:44.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.706+0000 7f980002d740 1 -- start start 2024-04-25T10:20:44.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.706+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97d406b500 0x7f97d406d9c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f97e8035000 tx=0x7f97e802f000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f980002d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 0x5644a34e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34bbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a34ba110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f980002d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a341f620 con 0x5644a34bb950 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a33c8ca0 con 0x5644a34e3cd0 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644a341b6c0 con 0x5644a35045c0 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f97f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 0x5644a34e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f97f5e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 0x5644a34e3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39814/0 (socket says 172.21.15.165:39814) 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.707+0000 7f97f5e00640 1 -- 172.21.15.165:0/3865217533 learned_addr learned my addr 172.21.15.165:0/3865217533 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5400640 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34bbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f6800640 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a34ba110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5e00640 1 -- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 msgr2=0x5644a34ba110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a34ba110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5e00640 1 -- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 msgr2=0x5644a34bbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34bbd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5e00640 1 -- 172.21.15.165:0/3865217533 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644a33cff40 con 0x5644a34bb950 2024-04-25T10:20:44.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97f5e00640 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 0x5644a34e3790 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f97e802aeb0 tx=0x7f97e802d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=640eca3e7dc62072 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.708+0000 7f97e6a00640 1 -- 172.21.15.165:0/3865217533 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e8003b90 con 0x5644a34bb950 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f97e6a00640 1 -- 172.21.15.165:0/3865217533 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e8036070 con 0x5644a34bb950 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f97e6a00640 1 -- 172.21.15.165:0/3865217533 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e803ce10 con 0x5644a34bb950 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 -- 172.21.15.165:0/3865217533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 msgr2=0x5644a34e3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 0x5644a34e3790 secure :-1 s=READY pgs=2753 cs=0 l=1 rev1=1 crypto rx=0x7f97e802aeb0 tx=0x7f97e802d000 comp rx=0 tx=0).stop 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 -- 172.21.15.165:0/3865217533 shutdown_connections 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a34ba110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34bbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 --2- 172.21.15.165:0/3865217533 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a34bb950 0x5644a34e3790 unknown :-1 s=CLOSED pgs=2753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 -- 172.21.15.165:0/3865217533 >> 172.21.15.165:0/3865217533 conn(0x5644a34b8b70 msgr2=0x5644a3501550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 -- 172.21.15.165:0/3865217533 shutdown_connections 2024-04-25T10:20:44.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.709+0000 7f980002d740 1 -- 172.21.15.165:0/3865217533 wait complete. 2024-04-25T10:20:44.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.710+0000 7f980002d740 1 -- start start 2024-04-25T10:20:44.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.710+0000 7f980002d740 1 Processor -- start 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34df0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a35117e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 0x5644a34e1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a341f620 con 0x5644a3511d20 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a33be1f0 con 0x5644a34e3cd0 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644a34e3300 con 0x5644a35045c0 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.711+0000 7f980002d740 10 client.0 _ll_register_callbacks cb 0x5644a33dbad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 0x5644a34e1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 0x5644a34e1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39822/0 (socket says 172.21.15.165:39822) 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 -- 172.21.15.165:0/615621985 learned_addr learned my addr 172.21.15.165:0/615621985 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f5400640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a35117e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f5e00640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34df0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 -- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 msgr2=0x5644a35117e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a35117e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 -- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 msgr2=0x5644a34df0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34df0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 -- 172.21.15.165:0/615621985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644a33ddc50 con 0x5644a3511d20 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97f6800640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 0x5644a34e1c80 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f97f80cc090 tx=0x7f97f000e5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e974b8ec366655e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97db400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97db400640 1 -- 172.21.15.165:0/615621985 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0014480 con 0x5644a3511d20 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.712+0000 7f97db400640 1 -- 172.21.15.165:0/615621985 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97f0014620 con 0x5644a3511d20 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97e8036070 con 0x5644a3511d20 2024-04-25T10:20:44.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f97db400640 1 -- 172.21.15.165:0/615621985 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97f001e050 con 0x5644a3511d20 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 10 client.11745 fetch_fsmap learned FSMap version 139 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x5644a31342d0 con 0x5644a3511d20 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f97db400640 1 -- 172.21.15.165:0/615621985 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e8036070 con 0x5644a3511d20 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f97db400640 1 -- 172.21.15.165:0/615621985 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 139) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97f0021020 con 0x5644a3511d20 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 10 client.11745 fetch_fsmap finished waiting for FSMap version 139 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 10 client.11745 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 20 client.11745 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a34e6de0 0x5644a34e4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 4 client.11745 mds_command: new command op to 11610 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.713+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 --> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644a342cb60 con 0x5644a34e6de0 2024-04-25T10:20:44.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.714+0000 7f97f5e00640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a34e6de0 0x5644a34e4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.714+0000 7f97f5e00640 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a34e6de0 0x5644a34e4660 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11d6e4909f145738 server_cookie=11e0d2093677a597 in_seq=0 out_seq=0 2024-04-25T10:20:44.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.714+0000 7f97db400640 10 client.11745 ms_handle_connect on v2:172.21.15.165:6835/3988146762 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.715+0000 7f97db400640 1 -- 172.21.15.165:0/615621985 <== mds.0 v2:172.21.15.165:6835/3988146762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5644a342cb60 con 0x5644a34e6de0 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.715+0000 7f97db400640 10 client.11745 handle_command_reply: tid=0 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.715+0000 7f980002d740 1 client.11745 shutdown 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.715+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 msgr2=0x5644a34e1c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.715+0000 7f980002d740 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 0x5644a34e1c80 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7f97f80cc090 tx=0x7f97f000e5f0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 shutdown_connections 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a34e6de0 0x5644a34e4660 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a3511d20 0x5644a34e1c80 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a35045c0 0x5644a35117e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 --2- 172.21.15.165:0/615621985 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a34e3cd0 0x5644a34df0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f97db400640 0 client.11745 ms_handle_reset on v2:172.21.15.165:6835/3988146762 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 >> 172.21.15.165:0/615621985 conn(0x5644a34bb950 msgr2=0x5644a34ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 shutdown_connections 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 1 -- 172.21.15.165:0/615621985 wait complete. 2024-04-25T10:20:44.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.716+0000 7f980002d740 20 client.11745 trim_cache size 0 max 16384 2024-04-25T10:20:44.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.733+0000 7f980002d740 1 Processor -- start 2024-04-25T10:20:44.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.733+0000 7f980002d740 1 -- start start 2024-04-25T10:20:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3514510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f980002d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a34e4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a34e4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f980002d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81ffeb0 con 0x5644a35045c0 2024-04-25T10:20:44.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81ff300 con 0x5644a3511d20 2024-04-25T10:20:44.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97f81ff8b0 con 0x5644a34e3cd0 2024-04-25T10:20:44.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a34e4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a34e4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39824/0 (socket says 172.21.15.165:39824) 2024-04-25T10:20:44.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 -- 172.21.15.165:0/1081046975 learned_addr learned my addr 172.21.15.165:0/1081046975 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 -- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 msgr2=0x5644a3514510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5e00640 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3514510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3514510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f6800640 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a34e4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 -- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 msgr2=0x5644a34e4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a34e4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.734+0000 7f97f5400640 1 -- 172.21.15.165:0/1081046975 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644a31fe6b0 con 0x5644a35045c0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.735+0000 7f97f5400640 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a34e4630 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f97e0054b70 tx=0x7f97e0058310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1eee90750545c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.735+0000 7f97e6000640 1 -- 172.21.15.165:0/1081046975 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0058e10 con 0x5644a35045c0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.735+0000 7f97e6000640 1 -- 172.21.15.165:0/1081046975 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0058710 con 0x5644a35045c0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.735+0000 7f97e6000640 1 -- 172.21.15.165:0/1081046975 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e005c860 con 0x5644a35045c0 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.735+0000 7f980002d740 1 -- 172.21.15.165:0/1081046975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 msgr2=0x5644a34e4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.735+0000 7f980002d740 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a34e4630 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f97e0054b70 tx=0x7f97e0058310 comp rx=0 tx=0).stop 2024-04-25T10:20:44.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 -- 172.21.15.165:0/1081046975 shutdown_connections 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a34e4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a34e4630 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 --2- 172.21.15.165:0/1081046975 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3514510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 -- 172.21.15.165:0/1081046975 >> 172.21.15.165:0/1081046975 conn(0x5644a34bb950 msgr2=0x5644a34b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 -- 172.21.15.165:0/1081046975 shutdown_connections 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 -- 172.21.15.165:0/1081046975 wait complete. 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 -- start start 2024-04-25T10:20:44.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.736+0000 7f980002d740 1 Processor -- start 2024-04-25T10:20:44.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3565d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a3503b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a356bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a34e6c60 con 0x5644a35045c0 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644a34e6820 con 0x5644a3511d20 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5644a34e7260 con 0x5644a34e3cd0 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.737+0000 7f980002d740 10 client.0 _ll_register_callbacks cb 0x7f97f0001970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a3503b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a3503b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39830/0 (socket says 172.21.15.165:39830) 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 -- 172.21.15.165:0/2308146399 learned_addr learned my addr 172.21.15.165:0/2308146399 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f6800640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a356bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 -- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 msgr2=0x5644a3565d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5e00640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3565d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3565d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 -- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 msgr2=0x5644a356bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a356bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 -- 172.21.15.165:0/2308146399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644a34f78a0 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97f5400640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a3503b60 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f97e00608f0 tx=0x7f97e0000f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eddfe6a8d5c4edfd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.738+0000 7f97db400640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f97db400640 1 -- 172.21.15.165:0/2308146399 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97e0060e20 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f97db400640 1 -- 172.21.15.165:0/2308146399 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f97e0003ca0 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f97e0058710 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f97db400640 1 -- 172.21.15.165:0/2308146399 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=139) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97e0077300 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 10 client.11751 fetch_fsmap learned FSMap version 139 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=139,monmap=2+}) v3 -- 0x5644a356c680 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f97db400640 1 -- 172.21.15.165:0/2308146399 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97e0058710 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f97db400640 1 -- 172.21.15.165:0/2308146399 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 139) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f97e0057b60 con 0x5644a35045c0 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 10 client.11751 fetch_fsmap finished waiting for FSMap version 139 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 10 client.11751 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 20 client.11751 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a3504c30 0x5644a34e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 4 client.11751 mds_command: new command op to 11610 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.739+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 --> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5644a3426eb0 con 0x5644a3504c30 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f97f5e00640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a3504c30 0x5644a34e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f97f5e00640 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a3504c30 0x5644a34e0ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91daefa331ec0f45 server_cookie=f6fd311df5565c28 in_seq=0 out_seq=0 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f97db400640 10 client.11751 ms_handle_connect on v2:172.21.15.165:6835/3988146762 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f97db400640 1 -- 172.21.15.165:0/2308146399 <== mds.0 v2:172.21.15.165:6835/3988146762 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x5644a3426eb0 con 0x5644a3504c30 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f97db400640 10 client.11751 handle_command_reply: tid=0 2024-04-25T10:20:44.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f980002d740 1 client.11751 shutdown 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 msgr2=0x5644a3503b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f980002d740 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a3503b60 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7f97e00608f0 tx=0x7f97e0000f70 comp rx=0 tx=0).stop 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 shutdown_connections 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f980002d740 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:6835/3988146762,v1:172.21.15.165:6837/3988146762] conn(0x5644a3504c30 0x5644a34e0ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.740+0000 7f980002d740 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5644a3511d20 0x5644a356bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f980002d740 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5644a35045c0 0x5644a3503b60 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f980002d740 1 --2- 172.21.15.165:0/2308146399 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5644a34e3cd0 0x5644a3565d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f97db400640 0 client.11751 ms_handle_reset on v2:172.21.15.165:6835/3988146762 2024-04-25T10:20:44.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 >> 172.21.15.165:0/2308146399 conn(0x5644a34bb950 msgr2=0x5644a34ba110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 shutdown_connections 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f980002d740 1 -- 172.21.15.165:0/2308146399 wait complete. 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f980002d740 20 client.11751 trim_cache size 0 max 16384 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 11610, 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:20:44.748 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 6.6033821369999997, 2024-04-25T10:20:44.749 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 139, 2024-04-25T10:20:44.749 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 195, 2024-04-25T10:20:44.749 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 195, 2024-04-25T10:20:44.749 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 12.072426368 2024-04-25T10:20:44.749 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97d406b500 msgr2=0x7f97d406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f97fd200640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97d406b500 0x7f97d406d9c0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f97e8035000 tx=0x7f97e802f000 comp rx=0 tx=0).stop 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 msgr2=0x7f97f813eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.741+0000 7f97fd200640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 0x7f97f813eef0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f97e000d540 tx=0x7f97e000da00 comp rx=0 tx=0).stop 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 shutdown_connections 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f97d406b500 0x7f97d406d9c0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f97f80b1610 0x7f97f813f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f97f80b1190 0x7f97f813eef0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 --2- 172.21.15.165:0/3636235103 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f97dc0039e0 0x7f97f813c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 >> 172.21.15.165:0/3636235103 conn(0x7f97f806abe0 msgr2=0x7f97f815ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 shutdown_connections 2024-04-25T10:20:44.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.742+0000 7f97fd200640 1 -- 172.21.15.165:0/3636235103 wait complete. 2024-04-25T10:20:44.761 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-04-25T10:20:44.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.937+0000 7f5e33e00640 1 Processor -- start 2024-04-25T10:20:44.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.938+0000 7f5e33e00640 1 -- start start 2024-04-25T10:20:44.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.939+0000 7f5e33e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 0x7f5e340b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.939+0000 7f5e33e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e341615a0 0x7f5e340cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.939+0000 7f5e33e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e340d33f0 con 0x7f5e340d04f0 2024-04-25T10:20:44.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.939+0000 7f5e33e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e340d0980 con 0x7f5e341615a0 2024-04-25T10:20:44.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.939+0000 7f5e33e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340d1f50 con 0x7f5e3415db90 2024-04-25T10:20:44.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 0x7f5e340b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5e341615a0 0x7f5e340cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 0x7f5e340b93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:37516/0 (socket says 172.21.15.165:37516) 2024-04-25T10:20:44.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 -- 172.21.15.165:0/2113906291 learned_addr learned my addr 172.21.15.165:0/2113906291 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 -- 172.21.15.165:0/2113906291 >> v2:172.21.15.172:3300/0 conn(0x7f5e341615a0 msgr2=0x7f5e340cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:44.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 --2- 172.21.15.165:0/2113906291 >> v2:172.21.15.172:3300/0 conn(0x7f5e341615a0 0x7f5e340cffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 -- 172.21.15.165:0/2113906291 >> v1:172.21.15.172:6789/0 conn(0x7f5e340d04f0 legacy=0x7f5e34061600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.940+0000 7f5e32a00640 1 -- 172.21.15.165:0/2113906291 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34161df0 con 0x7f5e3415db90 2024-04-25T10:20:44.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.942+0000 7f5e32a00640 1 --2- 172.21.15.165:0/2113906291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 0x7f5e340b93c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009870 tx=0x7f5e2402a270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13782081995bff8e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.942+0000 7f5e31600640 1 -- 172.21.15.165:0/2113906291 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24037070 con 0x7f5e3415db90 2024-04-25T10:20:44.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.942+0000 7f5e31600640 1 -- 172.21.15.165:0/2113906291 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e24032440 con 0x7f5e3415db90 2024-04-25T10:20:44.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.942+0000 7f5e31600640 1 -- 172.21.15.165:0/2113906291 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e24032810 con 0x7f5e3415db90 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 -- 172.21.15.165:0/2113906291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 msgr2=0x7f5e340b93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 --2- 172.21.15.165:0/2113906291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 0x7f5e340b93c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009870 tx=0x7f5e2402a270 comp rx=0 tx=0).stop 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 -- 172.21.15.165:0/2113906291 shutdown_connections 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 --2- 172.21.15.165:0/2113906291 >> v2:172.21.15.172:3300/0 conn(0x7f5e341615a0 0x7f5e340cffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 --2- 172.21.15.165:0/2113906291 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e3415db90 0x7f5e340b93c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 -- 172.21.15.165:0/2113906291 >> 172.21.15.165:0/2113906291 conn(0x7f5e340b52b0 msgr2=0x7f5e340b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 -- 172.21.15.165:0/2113906291 shutdown_connections 2024-04-25T10:20:44.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.943+0000 7f5e33e00640 1 -- 172.21.15.165:0/2113906291 wait complete. 2024-04-25T10:20:44.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.945+0000 7f5e33e00640 1 Processor -- start 2024-04-25T10:20:44.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.946+0000 7f5e33e00640 1 -- start start 2024-04-25T10:20:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e33e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e340d04f0 0x7f5e340caa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e33e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 0x7f5e3414ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e33e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e341615a0 0x7f5e340b7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e33e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e340d1be0 con 0x7f5e3415db90 2024-04-25T10:20:44.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e33e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e341623b0 con 0x7f5e341615a0 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e33e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e34161a20 con 0x7f5e340d04f0 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 0x7f5e3414ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e32000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 0x7f5e3414ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:39832/0 (socket says 172.21.15.165:39832) 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e32000640 1 -- 172.21.15.165:0/3269947924 learned_addr learned my addr 172.21.15.165:0/3269947924 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.947+0000 7f5e32a00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e340d04f0 0x7f5e340caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e33400640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e341615a0 0x7f5e340b7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e32000640 1 -- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e340d04f0 msgr2=0x7f5e340caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e32000640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e340d04f0 0x7f5e340caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e32000640 1 -- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e341615a0 msgr2=0x7f5e340b7770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e32000640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e341615a0 0x7f5e340b7770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:44.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e32000640 1 -- 172.21.15.165:0/3269947924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e340b7cb0 con 0x7f5e3415db90 2024-04-25T10:20:44.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e32000640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 0x7f5e3414ec80 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e870 tx=0x7f5e2000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.948+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20017070 con 0x7f5e3415db90 2024-04-25T10:20:44.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.949+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2000c940 con 0x7f5e3415db90 2024-04-25T10:20:44.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.949+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2000cd30 con 0x7f5e3415db90 2024-04-25T10:20:44.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.949+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e340cb110 con 0x7f5e3415db90 2024-04-25T10:20:44.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.949+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e34150970 con 0x7f5e3415db90 2024-04-25T10:20:44.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.950+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e34073990 con 0x7f5e3415db90 2024-04-25T10:20:44.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.951+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5e2001a070 con 0x7f5e3415db90 2024-04-25T10:20:44.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.952+0000 7f5e2aa00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:44.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.952+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f5e2008a1a0 con 0x7f5e3415db90 2024-04-25T10:20:44.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.952+0000 7f5e32a00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:44.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.953+0000 7f5e32a00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5e24034000 tx=0x7f5e2402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:44.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:44.956+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5e20058840 con 0x7f5e3415db90 2024-04-25T10:20:45.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.193+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f5e340d14b0 con 0x7f5e3415db90 2024-04-25T10:20:45.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.193+0000 7f5e2aa00640 1 -- 172.21.15.165:0/3269947924 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v139) v1 ==== 53+0+73 (secure 0 0 0) 0x7f5e2005c8e0 con 0x7f5e3415db90 2024-04-25T10:20:45.197 INFO:teuthology.orchestra.run.smithi165.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 msgr2=0x7f5e1406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5e24034000 tx=0x7f5e2402e000 comp rx=0 tx=0).stop 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 msgr2=0x7f5e3414ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 0x7f5e3414ec80 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7f5e2000e870 tx=0x7f5e2000ed30 comp rx=0 tx=0).stop 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 shutdown_connections 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5e1406b4d0 0x7f5e1406d990 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5e341615a0 0x7f5e340b7770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5e3415db90 0x7f5e3414ec80 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 --2- 172.21.15.165:0/3269947924 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5e340d04f0 0x7f5e340caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:45.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.196+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 >> 172.21.15.165:0/3269947924 conn(0x7f5e340b52b0 msgr2=0x7f5e340b8680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:45.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.197+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 shutdown_connections 2024-04-25T10:20:45.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:45.197+0000 7f5e33e00640 1 -- 172.21.15.165:0/3269947924 wait complete. 2024-04-25T10:20:45.219 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-25T10:20:45.350 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi165.front.sepia.ceph.com 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-25T10:20:45.351 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:45.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.352 DEBUG:teuthology.orchestra.run.smithi165:> ip addr 2024-04-25T10:20:45.411 INFO:teuthology.orchestra.run.smithi165.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-25T10:20:45.412 INFO:teuthology.orchestra.run.smithi165.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-25T10:20:45.412 INFO:teuthology.orchestra.run.smithi165.stdout: inet 127.0.0.1/8 scope host lo 2024-04-25T10:20:45.412 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.412 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 ::1/128 scope host 2024-04-25T10:20:45.412 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.412 INFO:teuthology.orchestra.run.smithi165.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:94 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f0 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:d9:3a:95 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout: altname enp7s0f1 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-25T10:20:45.413 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:62 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 0c:c4:7a:88:7f:63 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: inet 172.21.15.165/20 brd 172.21.15.255 scope global dynamic enp3s0f1 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft 41520sec preferred_lft 41520sec 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::ec4:7aff:fe88:7f63/64 scope link 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-04-25T10:20:45.414 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::b485:aaff:fe9c:21ad/64 scope link 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether 9e:2a:86:75:48:66 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::9c2a:86ff:fe75:4866/64 scope link 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: link/ether ce:82:64:cb:c4:9e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: inet6 fe80::cc82:64ff:fecb:c49e/64 scope link 2024-04-25T10:20:45.415 INFO:teuthology.orchestra.run.smithi165.stdout: valid_lft forever preferred_lft forever 2024-04-25T10:20:45.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.416 DEBUG:teuthology.orchestra.run.smithi165:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-25T10:20:45.488 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-25T10:20:45.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.492 DEBUG:teuthology.orchestra.run.smithi165:> route 2024-04-25T10:20:45.544 INFO:teuthology.orchestra.run.smithi165.stdout:Kernel IP routing table 2024-04-25T10:20:45.544 INFO:teuthology.orchestra.run.smithi165.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-25T10:20:45.544 INFO:teuthology.orchestra.run.smithi165.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-25T10:20:45.544 INFO:teuthology.orchestra.run.smithi165.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-25T10:20:45.544 INFO:teuthology.orchestra.run.smithi165.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-25T10:20:45.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.545 DEBUG:teuthology.orchestra.run.smithi165:> (cd / && exec bash -c ' 2024-04-25T10:20:45.545 DEBUG:teuthology.orchestra.run.smithi165:> set -e 2024-04-25T10:20:45.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-25T10:20:45.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-25T10:20:45.545 DEBUG:teuthology.orchestra.run.smithi165:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-25T10:20:45.545 DEBUG:teuthology.orchestra.run.smithi165:> ') 2024-04-25T10:20:45.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.691 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:20:45.741 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:20:45.741 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:20:45.742 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:45.742 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.742 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link set brx.0 up 2024-04-25T10:20:45.822 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:45.823 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:45.876 INFO:teuthology.orchestra.run.smithi165.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-25T10:20:45.877 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-25T10:20:45.877 DEBUG:teuthology.orchestra.run.smithi165:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:45.927 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-25T10:20:45.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:45.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.055 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.055 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:46.055 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:20:46.077 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.077 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:46.077 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:20:46.077 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:20:46.077 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:invalid new device string format 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:3300,172.21.15.172:3300,172.21.15.172:3301" 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: resolved to: "172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790" 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.165:6789,172.21.15.172:6789,172.21.15.172:6790:/ 2024-04-25T10:20:46.104 INFO:teuthology.orchestra.run.smithi165.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=55d22d87-b25b-4ccd-bc45-d797a973b3c0" will pass to kernel 2024-04-25T10:20:46.106 INFO:teuthology.orchestra.run.smithi165.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-25T10:20:46.106 INFO:teuthology.orchestra.run.smithi165.stdout: You just mounted a file system that supports labels which does not 2024-04-25T10:20:46.106 INFO:teuthology.orchestra.run.smithi165.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-25T10:20:46.106 INFO:teuthology.orchestra.run.smithi165.stdout: applications will generate AVC messages and not be allowed access to 2024-04-25T10:20:46.106 INFO:teuthology.orchestra.run.smithi165.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-25T10:20:46.109 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-25T10:20:46.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.110 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:46.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.180 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-25T10:20:46.240 INFO:teuthology.orchestra.run.smithi165.stdout:client6494 2024-04-25T10:20:46.241 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:46.241 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:46.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.289 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent) 2024-04-25T10:20:46.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.401 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent/child) 2024-04-25T10:20:46.457 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:46.457 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:46.507 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.507 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/parentfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:20:46.672 INFO:teuthology.orchestra.run.smithi165.stderr:6+0 records in 2024-04-25T10:20:46.672 INFO:teuthology.orchestra.run.smithi165.stderr:6+0 records out 2024-04-25T10:20:46.673 INFO:teuthology.orchestra.run.smithi165.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.110303 s, 57.0 MB/s 2024-04-25T10:20:46.674 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:46.674 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:46.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:46.731 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=parent/child/childfile bs=1M conv=fdatasync count=6 seek=0) 2024-04-25T10:20:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:6+0 records in 2024-04-25T10:20:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:6+0 records out 2024-04-25T10:20:46.932 INFO:teuthology.orchestra.run.smithi165.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.149435 s, 42.1 MB/s 2024-04-25T10:20:46.935 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:20:46.940 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.060 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-25T10:20:47.061 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:20:47.062 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.063 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on skipped message = False 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-25T10:20:47.064 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:20:47.065 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 20 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 20 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.066 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 10 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.067 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.068 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: mon op complaint time = 120 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-25T10:20:47.069 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: client mount timeout = 600 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: debug client = 20 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = "" 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-04-25T10:20:47.070 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.d] 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout:[mds] 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds = 20 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds balancer = 20 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout: mds cache max size = 1000 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug frag = True 2024-04-25T10:20:47.071 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug scatterstat = False 2024-04-25T10:20:47.072 INFO:teuthology.orchestra.run.smithi165.stdout: mds log max segments = 1 2024-04-25T10:20:47.072 INFO:teuthology.orchestra.run.smithi165.stdout: mds op complaint time = 180 2024-04-25T10:20:47.072 INFO:teuthology.orchestra.run.smithi165.stdout: mds verify scatter = False 2024-04-25T10:20:47.072 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:20:47.072 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.072 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-25T10:20:47.089 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:20:47.090 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-25T10:20:47.091 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-25T10:20:47.092 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.093 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-25T10:20:47.094 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.095 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:20:47.096 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:20:47.097 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-25T10:20:47.098 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: mds cache max size = 1000 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = False 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: mds log max segments = 1 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = False 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.099 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.124 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:20:47.129 DEBUG:teuthology.orchestra.run.smithi172:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-25T10:20:47.261 INFO:teuthology.orchestra.run.smithi172.stdout:[global] 2024-04-25T10:20:47.262 INFO:teuthology.orchestra.run.smithi172.stdout: chdir = "" 2024-04-25T10:20:47.262 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:20:47.262 INFO:teuthology.orchestra.run.smithi172.stdout: auth supported = cephx 2024-04-25T10:20:47.262 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.262 INFO:teuthology.orchestra.run.smithi172.stdout: filestore xattr use omap = true 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: mon clock drift allowed = 1.000 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: osd crush chooseleaf type = 0 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: auth debug = true 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on old message = true 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on bug = true 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: mon pg warn max object skew = 0 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:20:47.263 INFO:teuthology.orchestra.run.smithi172.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default size = 2 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow primary affinity = true 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd allow pg remap = true 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on no sortbitwise = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon warn on too few osds = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: mon_allow_pool_size_one = true 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.264 INFO:teuthology.orchestra.run.smithi172.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: osd default data pool replay window = 5 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: mon allow pool delete = true 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: mon cluster log file level = debug 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: debug asserts on shutdown = true 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: mon health detail to clog = false 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: ms die on skipped message = False 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout:[osd] 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: osd journal size = 100 2024-04-25T10:20:47.265 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub load threshold = 5.0 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd scrub max interval = 600 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd recover clone overlap = true 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd recovery max chunk = 1048576 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug shutdown = true 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug op order = true 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug verify stray on activate = true 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug trim objects = true 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.266 INFO:teuthology.orchestra.run.smithi172.stdout: osd open classes on start = true 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug pg log writeout = true 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: osd map max advance = 10 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: journal zero on create = true 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: filestore apply finisher threads = 3 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: bdev debug aio = true 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: osd debug misdirected ops = true 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore block size = 96636764160 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: bluestore fsck on mount = True 2024-04-25T10:20:47.267 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluefs = 20 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug bluestore = 20 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug osd = 20 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug rocksdb = 10 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd full ratio = 0.9 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: osd objectstore = bluestore 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout:[mgr] 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug mgr = 20 2024-04-25T10:20:47.268 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mgr/telemetry/nag = false 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout:[mon] 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: debug mon = 20 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: debug paxos = 20 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: debug auth = 20 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon data avail warn = 5 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon mgr mkfs grace = 240 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd reporter subtree level = osd 2024-04-25T10:20:47.269 INFO:teuthology.orchestra.run.smithi172.stdout: mon osd prime pg temp = true 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: # 1m isn't quite enough 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.270 INFO:teuthology.orchestra.run.smithi172.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: mon op complaint time = 120 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout:[client] 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: rgw cache enabled = true 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable ops log = true 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: rgw enable usage log = true 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: client mount timeout = 600 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: debug client = 20 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: pid file = "" 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.271 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.a] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.b] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mon.c] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.a] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.c] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.b] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mds.d] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout:[mds] 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds = 20 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: debug mds balancer = 20 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: debug ms = 1 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: mds cache max size = 1000 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug frag = True 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: mds debug scatterstat = False 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: mds log max segments = 1 2024-04-25T10:20:47.272 INFO:teuthology.orchestra.run.smithi172.stdout: mds op complaint time = 180 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi172.stdout: mds verify scatter = False 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi172.stdout: osd op complaint time = 180 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi172.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi172.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-25T10:20:47.273 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.274 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-25T10:20:47.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = "172.21.15.165,172.21.15.172,[v2:172.21.15.172:3301,v1:172.21.15.172:6790]" 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on skipped message = False 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 55d22d87-b25b-4ccd-bc45-d797a973b3c0 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-25T10:20:47.276 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-25T10:20:47.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-25T10:20:47.278 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 20 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 20 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 10 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:20:47.279 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-25T10:20:47.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-25T10:20:47.281 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: mon op complaint time = 120 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: client mount timeout = 600 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: debug client = 20 2024-04-25T10:20:47.282 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = "" 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.b] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.c] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.a] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.c] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.b] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mds.d] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout:[mds] 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds = 20 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: debug mds balancer = 20 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: mds cache max size = 1000 2024-04-25T10:20:47.283 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug frag = True 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: mds debug scatterstat = False 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: mds log max segments = 1 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: mds op complaint time = 180 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: mds verify scatter = False 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: osd op complaint time = 180 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: rados mon op timeout = 900 2024-04-25T10:20:47.284 INFO:teuthology.orchestra.run.smithi165.stdout: rados osd op timeout = 900 2024-04-25T10:20:47.297 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:47.297 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:47.348 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:20:47.348 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:20:47.445 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:20:47.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.637+0000 7f85b3e00640 1 Processor -- start 2024-04-25T10:20:47.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.637+0000 7f85b3e00640 1 -- start start 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.638+0000 7f85b3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f85b4144440 0x7f85b4070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.638+0000 7f85b3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85b4071530 0x7f85b40cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.638+0000 7f85b3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b415bf30 con 0x7f85b40cd860 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.638+0000 7f85b3e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b40719a0 con 0x7f85b4071530 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.638+0000 7f85b3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b415c860 con 0x7f85b4144440 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85b4071530 0x7f85b40cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85b4071530 0x7f85b40cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52094/0 (socket says 172.21.15.165:52094) 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 -- 172.21.15.165:0/2450718415 learned_addr learned my addr 172.21.15.165:0/2450718415 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2a00640 1 --2- 172.21.15.165:0/2450718415 >> v2:172.21.15.172:3300/0 conn(0x7f85b4144440 0x7f85b4070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 -- 172.21.15.165:0/2450718415 >> v2:172.21.15.172:3300/0 conn(0x7f85b4144440 msgr2=0x7f85b4070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:47.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 --2- 172.21.15.165:0/2450718415 >> v2:172.21.15.172:3300/0 conn(0x7f85b4144440 0x7f85b4070ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 -- 172.21.15.165:0/2450718415 >> v1:172.21.15.172:6789/0 conn(0x7f85b40cd860 legacy=0x7f85b406d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:47.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.639+0000 7f85b2000640 1 -- 172.21.15.165:0/2450718415 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b415c210 con 0x7f85b4071530 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.640+0000 7f85b2000640 1 --2- 172.21.15.165:0/2450718415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4071530 0x7f85b40cd320 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f85b406f470 tx=0x7f85a002a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834eb12ab96a4804 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b1600640 1 -- 172.21.15.165:0/2450718415 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0037070 con 0x7f85b4071530 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b1600640 1 -- 172.21.15.165:0/2450718415 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a0032440 con 0x7f85b4071530 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b1600640 1 -- 172.21.15.165:0/2450718415 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a0032810 con 0x7f85b4071530 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b3e00640 1 -- 172.21.15.165:0/2450718415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4071530 msgr2=0x7f85b40cd320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b3e00640 1 --2- 172.21.15.165:0/2450718415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4071530 0x7f85b40cd320 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f85b406f470 tx=0x7f85a002a2c0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b3e00640 1 -- 172.21.15.165:0/2450718415 shutdown_connections 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b3e00640 1 --2- 172.21.15.165:0/2450718415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4071530 0x7f85b40cd320 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b3e00640 1 --2- 172.21.15.165:0/2450718415 >> v2:172.21.15.172:3300/0 conn(0x7f85b4144440 0x7f85b4070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.641+0000 7f85b3e00640 1 -- 172.21.15.165:0/2450718415 >> 172.21.15.165:0/2450718415 conn(0x7f85b406abe0 msgr2=0x7f85b406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:47.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.642+0000 7f85b3e00640 1 -- 172.21.15.165:0/2450718415 shutdown_connections 2024-04-25T10:20:47.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.642+0000 7f85b3e00640 1 -- 172.21.15.165:0/2450718415 wait complete. 2024-04-25T10:20:47.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.643+0000 7f85b3e00640 1 Processor -- start 2024-04-25T10:20:47.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.643+0000 7f85b3e00640 1 -- start start 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85b4071530 0x7f85b40cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85b40cd860 0x7f85b413d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 0x7f85b413a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40cdcd0 con 0x7f85b4144440 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b415bc30 con 0x7f85b40cd860 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b40cde50 con 0x7f85b4071530 2024-04-25T10:20:47.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.644+0000 7f85b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 0x7f85b413a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 0x7f85b413a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52096/0 (socket says 172.21.15.165:52096) 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 -- 172.21.15.165:0/3528683926 learned_addr learned my addr 172.21.15.165:0/3528683926 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b2a00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85b4071530 0x7f85b40cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b2000640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85b40cd860 0x7f85b413d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 -- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85b4071530 msgr2=0x7f85b40cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85b4071530 0x7f85b40cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 -- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85b40cd860 msgr2=0x7f85b413d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85b40cd860 0x7f85b413d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 -- 172.21.15.165:0/3528683926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b40c7270 con 0x7f85b4144440 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85b3400640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 0x7f85b413a930 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c7b0 tx=0x7f85ac00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.645+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac017070 con 0x7f85b4144440 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.646+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac003da0 con 0x7f85b4144440 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.646+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0052b0 con 0x7f85b4144440 2024-04-25T10:20:47.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.646+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b40c7550 con 0x7f85b4144440 2024-04-25T10:20:47.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.646+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b40c7b10 con 0x7f85b4144440 2024-04-25T10:20:47.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.647+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b4073990 con 0x7f85b4144440 2024-04-25T10:20:47.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.648+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f85ac005450 con 0x7f85b4144440 2024-04-25T10:20:47.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.649+0000 7f85aaa00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f859406b4d0 0x7f859406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:47.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.649+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f85ac089a20 con 0x7f85b4144440 2024-04-25T10:20:47.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.649+0000 7f85b2a00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f859406b4d0 0x7f859406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:47.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.650+0000 7f85b2a00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f859406b4d0 0x7f859406d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f85a4006fd0 tx=0x7f85a4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:47.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.653+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85ac058670 con 0x7f85b4144440 2024-04-25T10:20:47.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.850+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85b406bff0 con 0x7f85b4144440 2024-04-25T10:20:47.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.852+0000 7f85aaa00640 1 -- 172.21.15.165:0/3528683926 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16920 (secure 0 0 0) 0x7f85ac057f70 con 0x7f85b4144440 2024-04-25T10:20:47.856 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:47.856 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":195,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:39.134241+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:20:47.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f859406b4d0 msgr2=0x7f859406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f859406b4d0 0x7f859406d990 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f85a4006fd0 tx=0x7f85a4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:47.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 msgr2=0x7f85b413a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:47.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 0x7f85b413a930 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f85ac00c7b0 tx=0x7f85ac00cc70 comp rx=0 tx=0).stop 2024-04-25T10:20:47.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 shutdown_connections 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f859406b4d0 0x7f859406d990 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85b4144440 0x7f85b413a930 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85b40cd860 0x7f85b413d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 --2- 172.21.15.165:0/3528683926 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85b4071530 0x7f85b40cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 >> 172.21.15.165:0/3528683926 conn(0x7f85b406abe0 msgr2=0x7f85b40c4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.856+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 shutdown_connections 2024-04-25T10:20:47.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:47.857+0000 7f85b3e00640 1 -- 172.21.15.165:0/3528683926 wait complete. 2024-04-25T10:20:47.880 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:20:35.357863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:36.605736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:20:47.881 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:20:48.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.078+0000 7f9460a00640 1 Processor -- start 2024-04-25T10:20:48.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.078+0000 7f9460a00640 1 -- start start 2024-04-25T10:20:48.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f9460a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f945c15b980 0x7f945c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f9460a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c072280 con 0x7f945c0ce230 2024-04-25T10:20:48.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f9460a00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c0714c0 con 0x7f945c0cc4f0 2024-04-25T10:20:48.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f9460a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945c071270 con 0x7f945c15b980 2024-04-25T10:20:48.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f9453e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f945c0ce230 0x7f945c15b270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52312/0 (socket says 172.21.15.165:52312) 2024-04-25T10:20:48.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f9453e00640 1 -- 172.21.15.165:0/3475269866 learned_addr learned my addr 172.21.15.165:0/3475269866 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:48.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.080+0000 7f945a800640 1 --2- 172.21.15.165:0/3475269866 >> v2:172.21.15.165:3300/0 conn(0x7f945c15b980 0x7f945c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.081+0000 7f9459400640 1 -- 172.21.15.165:0/3475269866 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 836465152 0 0) 0x7f945c072280 con 0x7f945c0ce230 2024-04-25T10:20:48.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.081+0000 7f9459400640 1 -- 172.21.15.165:0/3475269866 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9440005730 con 0x7f945c0ce230 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.081+0000 7f945a800640 1 -- 172.21.15.165:0/3475269866 >> v1:172.21.15.172:6789/0 conn(0x7f945c0cc4f0 legacy=0x7f945c0cdb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.081+0000 7f945a800640 1 -- 172.21.15.165:0/3475269866 >> v1:172.21.15.165:6789/0 conn(0x7f945c0ce230 legacy=0x7f945c15b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.081+0000 7f945a800640 1 -- 172.21.15.165:0/3475269866 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c0717a0 con 0x7f945c15b980 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.082+0000 7f945a800640 1 --2- 172.21.15.165:0/3475269866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c0cc8d0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f9454009d90 tx=0x7f945402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34720e2f063adfae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.082+0000 7f9459400640 1 -- 172.21.15.165:0/3475269866 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454007740 con 0x7f945c15b980 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.082+0000 7f9459400640 1 -- 172.21.15.165:0/3475269866 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94540078e0 con 0x7f945c15b980 2024-04-25T10:20:48.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.082+0000 7f9459400640 1 -- 172.21.15.165:0/3475269866 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454007cb0 con 0x7f945c15b980 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 -- 172.21.15.165:0/3475269866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 msgr2=0x7f945c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 --2- 172.21.15.165:0/3475269866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c0cc8d0 secure :-1 s=READY pgs=2761 cs=0 l=1 rev1=1 crypto rx=0x7f9454009d90 tx=0x7f945402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 -- 172.21.15.165:0/3475269866 shutdown_connections 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 --2- 172.21.15.165:0/3475269866 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c0cc8d0 unknown :-1 s=CLOSED pgs=2761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 -- 172.21.15.165:0/3475269866 >> 172.21.15.165:0/3475269866 conn(0x7f945c06abe0 msgr2=0x7f945c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 -- 172.21.15.165:0/3475269866 shutdown_connections 2024-04-25T10:20:48.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.083+0000 7f9460a00640 1 -- 172.21.15.165:0/3475269866 wait complete. 2024-04-25T10:20:48.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.084+0000 7f9460a00640 1 Processor -- start 2024-04-25T10:20:48.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.085+0000 7f9460a00640 1 -- start start 2024-04-25T10:20:48.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9460a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f945c0cc4f0 0x7f945c153020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9460a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0ce230 0x7f945c13c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9460a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c15cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9460a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c06e8a0 con 0x7f945c15b980 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9460a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c071d60 con 0x7f945c0ce230 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9460a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c0ae500 con 0x7f945c0cc4f0 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c15cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c15cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52112/0 (socket says 172.21.15.165:52112) 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 -- 172.21.15.165:0/1174252613 learned_addr learned my addr 172.21.15.165:0/1174252613 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9453e00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0ce230 0x7f945c13c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f9459e00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f945c0cc4f0 0x7f945c153020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 -- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f945c0cc4f0 msgr2=0x7f945c153020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f945c0cc4f0 0x7f945c153020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 -- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0ce230 msgr2=0x7f945c13c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0ce230 0x7f945c13c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.086+0000 7f945a800640 1 -- 172.21.15.165:0/1174252613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c153870 con 0x7f945c15b980 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.087+0000 7f945a800640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c15cb10 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f9454009d90 tx=0x7f9454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.087+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454040070 con 0x7f945c15b980 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.087+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454031bc0 con 0x7f945c15b980 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.087+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454009450 con 0x7f945c15b980 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.087+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f945c153b50 con 0x7f945c15b980 2024-04-25T10:20:48.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.087+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c0acb80 con 0x7f945c15b980 2024-04-25T10:20:48.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.089+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c073a00 con 0x7f945c15b980 2024-04-25T10:20:48.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.089+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9454039040 con 0x7f945c15b980 2024-04-25T10:20:48.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.091+0000 7f9452000640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943806b500 0x7f943806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.091+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f94540ab480 con 0x7f945c15b980 2024-04-25T10:20:48.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.091+0000 7f9459e00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943806b500 0x7f943806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.092+0000 7f9459e00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943806b500 0x7f943806d9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f60 tx=0x7f944c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.095+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9454079b10 con 0x7f945c15b980 2024-04-25T10:20:48.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.296+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f945c0ce6a0 con 0x7f945c15b980 2024-04-25T10:20:48.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.297+0000 7f9452000640 1 -- 172.21.15.165:0/1174252613 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v196) v1 ==== 93+0+10 (secure 0 0 0) 0x7f945407dbb0 con 0x7f945c15b980 2024-04-25T10:20:48.301 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943806b500 msgr2=0x7f943806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943806b500 0x7f943806d9c0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f944c000f60 tx=0x7f944c009040 comp rx=0 tx=0).stop 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 msgr2=0x7f945c15cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c15cb10 secure :-1 s=READY pgs=2762 cs=0 l=1 rev1=1 crypto rx=0x7f9454009d90 tx=0x7f9454004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 shutdown_connections 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f943806b500 0x7f943806d9c0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f945c15b980 0x7f945c15cb10 unknown :-1 s=CLOSED pgs=2762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f945c0ce230 0x7f945c13c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 --2- 172.21.15.165:0/1174252613 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f945c0cc4f0 0x7f945c153020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 >> 172.21.15.165:0/1174252613 conn(0x7f945c06abe0 msgr2=0x7f945c0cc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 shutdown_connections 2024-04-25T10:20:48.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.300+0000 7f9460a00640 1 -- 172.21.15.165:0/1174252613 wait complete. 2024-04-25T10:20:48.323 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:20:48.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.525+0000 7f63f0200640 1 Processor -- start 2024-04-25T10:20:48.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.525+0000 7f63f0200640 1 -- start start 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63f0200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f63e80cf640 0x7f63e806ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63f0200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f63e80ac8b0 0x7f63e80acd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63f0200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e80cfad0 con 0x7f63e815b710 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63f0200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e806f190 con 0x7f63e80cf640 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63f0200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e815e490 con 0x7f63e80ac8b0 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63ece00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f63e80cf640 0x7f63e806ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63ece00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f63e80cf640 0x7f63e806ec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52126/0 (socket says 172.21.15.165:52126) 2024-04-25T10:20:48.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63ece00640 1 -- 172.21.15.165:0/4193430669 learned_addr learned my addr 172.21.15.165:0/4193430669 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:48.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63ece00640 1 -- 172.21.15.165:0/4193430669 >> v2:172.21.15.172:3300/0 conn(0x7f63e80ac8b0 msgr2=0x7f63e80acd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:48.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.527+0000 7f63ee200640 1 --2- 172.21.15.165:0/4193430669 >> v2:172.21.15.172:3300/0 conn(0x7f63e80ac8b0 0x7f63e80acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.528+0000 7f63ece00640 1 --2- 172.21.15.165:0/4193430669 >> v2:172.21.15.172:3300/0 conn(0x7f63e80ac8b0 0x7f63e80acd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.528+0000 7f63ece00640 1 -- 172.21.15.165:0/4193430669 >> v1:172.21.15.172:6789/0 conn(0x7f63e815b710 legacy=0x7f63e815e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:48.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.528+0000 7f63ece00640 1 -- 172.21.15.165:0/4193430669 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e80cfd80 con 0x7f63e80cf640 2024-04-25T10:20:48.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.528+0000 7f63ee200640 1 --2- 172.21.15.165:0/4193430669 >> v2:172.21.15.172:3300/0 conn(0x7f63e80ac8b0 0x7f63e80acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:48.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.529+0000 7f63ece00640 1 --2- 172.21.15.165:0/4193430669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80cf640 0x7f63e806ec50 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f63e80cd2e0 tx=0x7f63d402a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10af0fa8e79e549e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.529+0000 7f63e7e00640 1 -- 172.21.15.165:0/4193430669 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4037070 con 0x7f63e80cf640 2024-04-25T10:20:48.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.529+0000 7f63e7e00640 1 -- 172.21.15.165:0/4193430669 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d4032440 con 0x7f63e80cf640 2024-04-25T10:20:48.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.529+0000 7f63e7e00640 1 -- 172.21.15.165:0/4193430669 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d4032810 con 0x7f63e80cf640 2024-04-25T10:20:48.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.529+0000 7f63f0200640 1 -- 172.21.15.165:0/4193430669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80cf640 msgr2=0x7f63e806ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.529+0000 7f63f0200640 1 --2- 172.21.15.165:0/4193430669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80cf640 0x7f63e806ec50 secure :-1 s=READY pgs=2763 cs=0 l=1 rev1=1 crypto rx=0x7f63e80cd2e0 tx=0x7f63d402a230 comp rx=0 tx=0).stop 2024-04-25T10:20:48.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.530+0000 7f63f0200640 1 -- 172.21.15.165:0/4193430669 shutdown_connections 2024-04-25T10:20:48.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.530+0000 7f63f0200640 1 --2- 172.21.15.165:0/4193430669 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80cf640 0x7f63e806ec50 unknown :-1 s=CLOSED pgs=2763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.530+0000 7f63f0200640 1 --2- 172.21.15.165:0/4193430669 >> v2:172.21.15.172:3300/0 conn(0x7f63e80ac8b0 0x7f63e80acd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.530+0000 7f63f0200640 1 -- 172.21.15.165:0/4193430669 >> 172.21.15.165:0/4193430669 conn(0x7f63e806abe0 msgr2=0x7f63e806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.530+0000 7f63f0200640 1 -- 172.21.15.165:0/4193430669 shutdown_connections 2024-04-25T10:20:48.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.530+0000 7f63f0200640 1 -- 172.21.15.165:0/4193430669 wait complete. 2024-04-25T10:20:48.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.531+0000 7f63f0200640 1 Processor -- start 2024-04-25T10:20:48.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.531+0000 7f63f0200640 1 -- start start 2024-04-25T10:20:48.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.532+0000 7f63f0200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 0x7f63e80b1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.532+0000 7f63f0200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63e80cf640 0x7f63e80b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.532+0000 7f63f0200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e815b710 0x7f63e815d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.532+0000 7f63f0200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e815baf0 con 0x7f63e80ac8b0 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.532+0000 7f63f0200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e80d0340 con 0x7f63e815b710 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.532+0000 7f63f0200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e815bc70 con 0x7f63e80cf640 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 0x7f63e80b1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 0x7f63e80b1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52142/0 (socket says 172.21.15.165:52142) 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 -- 172.21.15.165:0/3190007016 learned_addr learned my addr 172.21.15.165:0/3190007016 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 -- 172.21.15.165:0/3190007016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63e80cf640 msgr2=0x7f63e80b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63e80cf640 0x7f63e80b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 -- 172.21.15.165:0/3190007016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e815b710 msgr2=0x7f63e815d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e815b710 0x7f63e815d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 -- 172.21.15.165:0/3190007016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e80b1ff0 con 0x7f63e80ac8b0 2024-04-25T10:20:48.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.533+0000 7f63ed800640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 0x7f63e80b1a10 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00b3e0 tx=0x7f63dc00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.534+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc015020 con 0x7f63e80ac8b0 2024-04-25T10:20:48.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.534+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc011850 con 0x7f63e80ac8b0 2024-04-25T10:20:48.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.534+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc011c40 con 0x7f63e80ac8b0 2024-04-25T10:20:48.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.534+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e80b22d0 con 0x7f63e80ac8b0 2024-04-25T10:20:48.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.534+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e80b2890 con 0x7f63e80ac8b0 2024-04-25T10:20:48.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.535+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e8073a00 con 0x7f63e80ac8b0 2024-04-25T10:20:48.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.536+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f63dc018020 con 0x7f63e80ac8b0 2024-04-25T10:20:48.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.537+0000 7f63e5600640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f63c806b500 0x7f63c806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.537+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f63dc089e70 con 0x7f63e80ac8b0 2024-04-25T10:20:48.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.537+0000 7f63ece00640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f63c806b500 0x7f63c806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.538+0000 7f63ece00640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f63c806b500 0x7f63c806d9c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f63d4034000 tx=0x7f63d402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.541+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f63dc01f070 con 0x7f63e80ac8b0 2024-04-25T10:20:48.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.721+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f63e806e180 con 0x7f63e80ac8b0 2024-04-25T10:20:48.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.722+0000 7f63e5600640 1 -- 172.21.15.165:0/3190007016 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v196) v1 ==== 104+0+11 (secure 0 0 0) 0x7f63dc018310 con 0x7f63e80ac8b0 2024-04-25T10:20:48.727 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:20:48.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f63c806b500 msgr2=0x7f63c806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f63c806b500 0x7f63c806d9c0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f63d4034000 tx=0x7f63d402c000 comp rx=0 tx=0).stop 2024-04-25T10:20:48.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 msgr2=0x7f63e80b1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 0x7f63e80b1a10 secure :-1 s=READY pgs=2764 cs=0 l=1 rev1=1 crypto rx=0x7f63dc00b3e0 tx=0x7f63dc00b8a0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 shutdown_connections 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f63c806b500 0x7f63c806d9c0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f63e815b710 0x7f63e815d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f63e80cf640 0x7f63e80b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 --2- 172.21.15.165:0/3190007016 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f63e80ac8b0 0x7f63e80b1a10 unknown :-1 s=CLOSED pgs=2764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.725+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 >> 172.21.15.165:0/3190007016 conn(0x7f63e806abe0 msgr2=0x7f63e806d2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.726+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 shutdown_connections 2024-04-25T10:20:48.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.726+0000 7f63f0200640 1 -- 172.21.15.165:0/3190007016 wait complete. 2024-04-25T10:20:48.746 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:20:48.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.938+0000 7f9bf3e00640 1 Processor -- start 2024-04-25T10:20:48.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.939+0000 7f9bf3e00640 1 -- start start 2024-04-25T10:20:48.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.940+0000 7f9bf3e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9bf415dd30 0x7f9bf40cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.940+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf40d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.940+0000 7f9bf3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bf415e110 con 0x7f9bf406bff0 2024-04-25T10:20:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.940+0000 7f9bf3e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bf406d000 con 0x7f9bf415dd30 2024-04-25T10:20:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.940+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40b2fd0 con 0x7f9bf40b5b40 2024-04-25T10:20:48.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.941+0000 7f9bf3400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf40d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.941+0000 7f9bf2a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9bf415dd30 0x7f9bf40cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.941+0000 7f9bf2000640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f9bf406bff0 0x7f9bf40b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:44296/0 (socket says 172.21.15.165:44296) 2024-04-25T10:20:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.941+0000 7f9bf3400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf40d2160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3301/0 says I am v2:172.21.15.165:48322/0 (socket says 172.21.15.165:48322) 2024-04-25T10:20:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.941+0000 7f9bf2000640 1 -- 172.21.15.165:0/3469476619 learned_addr learned my addr 172.21.15.165:0/3469476619 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.942+0000 7f9bf3400640 1 -- 172.21.15.165:0/3469476619 >> v2:172.21.15.172:3300/0 conn(0x7f9bf415dd30 msgr2=0x7f9bf40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:48.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.942+0000 7f9bf3400640 1 --2- 172.21.15.165:0/3469476619 >> v2:172.21.15.172:3300/0 conn(0x7f9bf415dd30 0x7f9bf40cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.942+0000 7f9bf3400640 1 -- 172.21.15.165:0/3469476619 >> v1:172.21.15.172:6789/0 conn(0x7f9bf406bff0 legacy=0x7f9bf40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.942+0000 7f9bf3400640 1 -- 172.21.15.165:0/3469476619 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf406c480 con 0x7f9bf40b5b40 2024-04-25T10:20:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.942+0000 7f9bf2a00640 1 --2- 172.21.15.165:0/3469476619 >> v2:172.21.15.172:3300/0 conn(0x7f9bf415dd30 0x7f9bf40cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:20:48.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.943+0000 7f9bf3400640 1 --2- 172.21.15.165:0/3469476619 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf40d2160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0097a0 tx=0x7f9bec02a180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da3827c1d097b7c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.944+0000 7f9bf1600640 1 -- 172.21.15.165:0/3469476619 <== mon.2 v2:172.21.15.172:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec037070 con 0x7f9bf40b5b40 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.944+0000 7f9bf1600640 1 -- 172.21.15.165:0/3469476619 <== mon.2 v2:172.21.15.172:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bec032440 con 0x7f9bf40b5b40 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.944+0000 7f9bf1600640 1 -- 172.21.15.165:0/3469476619 <== mon.2 v2:172.21.15.172:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bec032810 con 0x7f9bf40b5b40 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.944+0000 7f9bf3e00640 1 -- 172.21.15.165:0/3469476619 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 msgr2=0x7f9bf40d2160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.944+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/3469476619 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf40d2160 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f9bec0097a0 tx=0x7f9bec02a180 comp rx=0 tx=0).stop 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.945+0000 7f9bf3e00640 1 -- 172.21.15.165:0/3469476619 shutdown_connections 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.945+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/3469476619 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf40d2160 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.945+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/3469476619 >> v2:172.21.15.172:3300/0 conn(0x7f9bf415dd30 0x7f9bf40cebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.945+0000 7f9bf3e00640 1 -- 172.21.15.165:0/3469476619 >> 172.21.15.165:0/3469476619 conn(0x7f9bf40d0610 msgr2=0x7f9bf40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.945+0000 7f9bf3e00640 1 -- 172.21.15.165:0/3469476619 shutdown_connections 2024-04-25T10:20:48.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.945+0000 7f9bf3e00640 1 -- 172.21.15.165:0/3469476619 wait complete. 2024-04-25T10:20:48.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.946+0000 7f9bf3e00640 1 Processor -- start 2024-04-25T10:20:48.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.946+0000 7f9bf3e00640 1 -- start start 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.947+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf4152ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.947+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 0x7f9bf415eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.947+0000 7f9bf3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bf40cd4c0 0x7f9bf4152550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.947+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf406cd20 con 0x7f9bf415dd30 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.947+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf406ca40 con 0x7f9bf40cd4c0 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.947+0000 7f9bf3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf40cf110 con 0x7f9bf40b5b40 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 0x7f9bf415eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 0x7f9bf415eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52144/0 (socket says 172.21.15.165:52144) 2024-04-25T10:20:48.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 -- 172.21.15.165:0/2081244579 learned_addr learned my addr 172.21.15.165:0/2081244579 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2a00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf4152ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf3400640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bf40cd4c0 0x7f9bf4152550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 -- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 msgr2=0x7f9bf4152ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf4152ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 -- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bf40cd4c0 msgr2=0x7f9bf4152550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bf40cd4c0 0x7f9bf4152550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 -- 172.21.15.165:0/2081244579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bf4160fe0 con 0x7f9bf415dd30 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.948+0000 7f9bf2000640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 0x7f9bf415eca0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00cb10 tx=0x7f9bdc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.949+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc017070 con 0x7f9bf415dd30 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.949+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc00ac70 con 0x7f9bf415dd30 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.949+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc012620 con 0x7f9bf415dd30 2024-04-25T10:20:48.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.949+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf4153400 con 0x7f9bf415dd30 2024-04-25T10:20:48.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.949+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf415f460 con 0x7f9bf415dd30 2024-04-25T10:20:48.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.950+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf4073990 con 0x7f9bf415dd30 2024-04-25T10:20:48.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.951+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9bdc01a070 con 0x7f9bf415dd30 2024-04-25T10:20:48.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.952+0000 7f9beaa00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bd006b4d0 0x7f9bd006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:48.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.952+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f9bdc08a1d0 con 0x7f9bf415dd30 2024-04-25T10:20:48.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.952+0000 7f9bf2a00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bd006b4d0 0x7f9bd006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:48.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.953+0000 7f9bf2a00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bd006b4d0 0x7f9bd006d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009950 tx=0x7f9be4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:48.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:48.956+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9bdc058e40 con 0x7f9bf415dd30 2024-04-25T10:20:49.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.148+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9bf415d350 con 0x7f9bf415dd30 2024-04-25T10:20:49.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.149+0000 7f9beaa00640 1 -- 172.21.15.165:0/2081244579 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v196) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9bdc058740 con 0x7f9bf415dd30 2024-04-25T10:20:49.153 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:20:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bd006b4d0 msgr2=0x7f9bd006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bd006b4d0 0x7f9bd006d990 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f9be4009950 tx=0x7f9be4008040 comp rx=0 tx=0).stop 2024-04-25T10:20:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 msgr2=0x7f9bf415eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 0x7f9bf415eca0 secure :-1 s=READY pgs=2765 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00cb10 tx=0x7f9bdc00a060 comp rx=0 tx=0).stop 2024-04-25T10:20:49.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 shutdown_connections 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9bd006b4d0 0x7f9bd006d990 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9bf40cd4c0 0x7f9bf4152550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9bf415dd30 0x7f9bf415eca0 unknown :-1 s=CLOSED pgs=2765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 --2- 172.21.15.165:0/2081244579 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9bf40b5b40 0x7f9bf4152ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.151+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 >> 172.21.15.165:0/2081244579 conn(0x7f9bf40d0610 msgr2=0x7f9bf40b5420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.152+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 shutdown_connections 2024-04-25T10:20:49.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.152+0000 7f9bf3e00640 1 -- 172.21.15.165:0/2081244579 wait complete. 2024-04-25T10:20:49.175 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-25T10:20:49.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.363+0000 7f6d88c00640 1 Processor -- start 2024-04-25T10:20:49.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.364+0000 7f6d88c00640 1 -- start start 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d88c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6d8406ea90 0x7f6d84071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d88c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406d2d0 0x7f6d8406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d88c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d84162ba0 con 0x7f6d84161fc0 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d88c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d8406dc80 con 0x7f6d8406ea90 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d88c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84165be0 con 0x7f6d8406d2d0 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d81e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6d84161fc0 0x7f6d841623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52322/0 (socket says 172.21.15.165:52322) 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.365+0000 7f6d81e00640 1 -- 172.21.15.165:0/1616910422 learned_addr learned my addr 172.21.15.165:0/1616910422 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:49.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d81400640 1 --2- 172.21.15.165:0/1616910422 >> v2:172.21.15.165:3300/0 conn(0x7f6d8406ea90 0x7f6d84071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:49.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d80a00640 1 -- 172.21.15.165:0/1616910422 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559162340 0 0) 0x7f6d84162ba0 con 0x7f6d84161fc0 2024-04-25T10:20:49.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d82800640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406d2d0 0x7f6d8406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:49.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d80a00640 1 -- 172.21.15.165:0/1616910422 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d64003520 con 0x7f6d84161fc0 2024-04-25T10:20:49.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d81400640 1 -- 172.21.15.165:0/1616910422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406d2d0 msgr2=0x7f6d8406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:49.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d81400640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406d2d0 0x7f6d8406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d81400640 1 -- 172.21.15.165:0/1616910422 >> v1:172.21.15.165:6789/0 conn(0x7f6d84161fc0 legacy=0x7f6d841623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d81400640 1 -- 172.21.15.165:0/1616910422 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d84162e80 con 0x7f6d8406ea90 2024-04-25T10:20:49.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.366+0000 7f6d82800640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406d2d0 0x7f6d8406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.367+0000 7f6d81400640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406ea90 0x7f6d84071480 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009dc0 tx=0x7f6d6c02a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed45f75b63d11878 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d80a00640 1 -- 172.21.15.165:0/1616910422 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c007740 con 0x7f6d8406ea90 2024-04-25T10:20:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d80a00640 1 -- 172.21.15.165:0/1616910422 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d6c0078e0 con 0x7f6d8406ea90 2024-04-25T10:20:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d80a00640 1 -- 172.21.15.165:0/1616910422 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d6c007cb0 con 0x7f6d8406ea90 2024-04-25T10:20:49.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 -- 172.21.15.165:0/1616910422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406ea90 msgr2=0x7f6d84071480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406ea90 0x7f6d84071480 secure :-1 s=READY pgs=2767 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c009dc0 tx=0x7f6d6c02a5f0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 -- 172.21.15.165:0/1616910422 shutdown_connections 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406d2d0 0x7f6d8406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1616910422 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406ea90 0x7f6d84071480 unknown :-1 s=CLOSED pgs=2767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 -- 172.21.15.165:0/1616910422 >> 172.21.15.165:0/1616910422 conn(0x7f6d8406abe0 msgr2=0x7f6d8406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 -- 172.21.15.165:0/1616910422 shutdown_connections 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.368+0000 7f6d88c00640 1 -- 172.21.15.165:0/1616910422 wait complete. 2024-04-25T10:20:49.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.370+0000 7f6d88c00640 1 Processor -- start 2024-04-25T10:20:49.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.370+0000 7f6d88c00640 1 -- start start 2024-04-25T10:20:49.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d88c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 0x7f6d8414d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:49.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d88c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406ea90 0x7f6d840b1cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:49.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d88c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d84161fc0 0x7f6d840aeee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d88c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d840719c0 con 0x7f6d8406d2d0 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d88c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84163440 con 0x7f6d84161fc0 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d88c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d84071b40 con 0x7f6d8406ea90 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d81e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 0x7f6d8414d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d81e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 0x7f6d8414d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52162/0 (socket says 172.21.15.165:52162) 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.371+0000 7f6d81e00640 1 -- 172.21.15.165:0/1712280275 learned_addr learned my addr 172.21.15.165:0/1712280275 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d81e00640 1 -- 172.21.15.165:0/1712280275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406ea90 msgr2=0x7f6d840b1cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d81e00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406ea90 0x7f6d840b1cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d81e00640 1 -- 172.21.15.165:0/1712280275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d84161fc0 msgr2=0x7f6d840aeee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d81e00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d84161fc0 0x7f6d840aeee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d81e00640 1 -- 172.21.15.165:0/1712280275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d840b2690 con 0x7f6d8406d2d0 2024-04-25T10:20:49.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d81e00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 0x7f6d8414d9a0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400c390 tx=0x7f6d7400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:49.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74017070 con 0x7f6d8406d2d0 2024-04-25T10:20:49.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d74005ce0 con 0x7f6d8406d2d0 2024-04-25T10:20:49.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d740072f0 con 0x7f6d8406d2d0 2024-04-25T10:20:49.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.372+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d840b2970 con 0x7f6d8406d2d0 2024-04-25T10:20:49.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.373+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d840b2f30 con 0x7f6d8406d2d0 2024-04-25T10:20:49.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.374+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d84073a00 con 0x7f6d8406d2d0 2024-04-25T10:20:49.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.375+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6d7401a070 con 0x7f6d8406d2d0 2024-04-25T10:20:49.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.376+0000 7f6d7a000640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6d6006b4d0 0x7f6d6006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:49.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.376+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f6d740898b0 con 0x7f6d8406d2d0 2024-04-25T10:20:49.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.376+0000 7f6d81400640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6d6006b4d0 0x7f6d6006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:49.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.377+0000 7f6d81400640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6d6006b4d0 0x7f6d6006d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c034000 tx=0x7f6d6c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:49.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.381+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6d74058490 con 0x7f6d8406d2d0 2024-04-25T10:20:49.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.588+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6d84072170 con 0x7f6d8406d2d0 2024-04-25T10:20:49.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.589+0000 7f6d7a000640 1 -- 172.21.15.165:0/1712280275 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v197) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6d74057d90 con 0x7f6d8406d2d0 2024-04-25T10:20:49.592 INFO:teuthology.orchestra.run.smithi165.stderr:listed 0 entries 2024-04-25T10:20:49.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6d6006b4d0 msgr2=0x7f6d6006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6d6006b4d0 0x7f6d6006d990 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f6d6c034000 tx=0x7f6d6c02e000 comp rx=0 tx=0).stop 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 msgr2=0x7f6d8414d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 0x7f6d8414d9a0 secure :-1 s=READY pgs=2768 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400c390 tx=0x7f6d7400c850 comp rx=0 tx=0).stop 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 shutdown_connections 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6d6006b4d0 0x7f6d6006d990 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6d84161fc0 0x7f6d840aeee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6d8406ea90 0x7f6d840b1cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 --2- 172.21.15.165:0/1712280275 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6d8406d2d0 0x7f6d8414d9a0 unknown :-1 s=CLOSED pgs=2768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.591+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 >> 172.21.15.165:0/1712280275 conn(0x7f6d8406abe0 msgr2=0x7f6d840b0b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.592+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 shutdown_connections 2024-04-25T10:20:49.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:49.592+0000 7f6d88c00640 1 -- 172.21.15.165:0/1712280275 wait complete. 2024-04-25T10:20:49.615 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-25T10:20:49.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:49.615 DEBUG:teuthology.orchestra.run.smithi165:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-25T10:20:49.750 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:20:49.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:49.754 DEBUG:teuthology.orchestra.run.smithi165:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-25T10:20:49.798 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:50.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.012+0000 7fa0c0a00640 1 Processor -- start 2024-04-25T10:20:50.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.012+0000 7fa0c0a00640 1 -- start start 2024-04-25T10:20:50.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0c0a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fa0bc144440 0x7fa0bc070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0c0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc0cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0c0a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa0bc0cd860 0x7fa0bc06d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0c0a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc15c860 con 0x7fa0bc0cd860 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0c0a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc071970 con 0x7fa0bc144440 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0c0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15bf30 con 0x7fa0bc071530 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0bb200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa0bc0cd860 0x7fa0bc06d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0bb200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fa0bc0cd860 0x7fa0bc06d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52166/0 (socket says 172.21.15.165:52166) 2024-04-25T10:20:50.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.014+0000 7fa0bb200640 1 -- 172.21.15.165:0/3474807125 learned_addr learned my addr 172.21.15.165:0/3474807125 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0bb200640 1 -- 172.21.15.165:0/3474807125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 msgr2=0x7fa0bc0cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0ba800640 1 --2- 172.21.15.165:0/3474807125 >> v2:172.21.15.172:3300/0 conn(0x7fa0bc144440 0x7fa0bc070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0b3e00640 1 --2- 172.21.15.165:0/3474807125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc0cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0bb200640 1 --2- 172.21.15.165:0/3474807125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc0cd320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0bb200640 1 -- 172.21.15.165:0/3474807125 >> v2:172.21.15.172:3300/0 conn(0x7fa0bc144440 msgr2=0x7fa0bc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0bb200640 1 --2- 172.21.15.165:0/3474807125 >> v2:172.21.15.172:3300/0 conn(0x7fa0bc144440 0x7fa0bc070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0bb200640 1 -- 172.21.15.165:0/3474807125 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc15c210 con 0x7fa0bc0cd860 2024-04-25T10:20:50.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.015+0000 7fa0ba800640 1 --2- 172.21.15.165:0/3474807125 >> v2:172.21.15.172:3300/0 conn(0x7fa0bc144440 0x7fa0bc070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.016+0000 7fa0bb200640 1 --2- 172.21.15.165:0/3474807125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc0cd860 0x7fa0bc06d050 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc06f470 tx=0x7fa0b402a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e1a6cbb194875c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.016+0000 7fa0b9e00640 1 -- 172.21.15.165:0/3474807125 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4036070 con 0x7fa0bc0cd860 2024-04-25T10:20:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.016+0000 7fa0b9e00640 1 -- 172.21.15.165:0/3474807125 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4031600 con 0x7fa0bc0cd860 2024-04-25T10:20:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.016+0000 7fa0b9e00640 1 -- 172.21.15.165:0/3474807125 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b40319d0 con 0x7fa0bc0cd860 2024-04-25T10:20:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 -- 172.21.15.165:0/3474807125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc0cd860 msgr2=0x7fa0bc06d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/3474807125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc0cd860 0x7fa0bc06d050 secure :-1 s=READY pgs=2769 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc06f470 tx=0x7fa0b402a730 comp rx=0 tx=0).stop 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 -- 172.21.15.165:0/3474807125 shutdown_connections 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/3474807125 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc0cd860 0x7fa0bc06d050 unknown :-1 s=CLOSED pgs=2769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/3474807125 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc0cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/3474807125 >> v2:172.21.15.172:3300/0 conn(0x7fa0bc144440 0x7fa0bc070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 -- 172.21.15.165:0/3474807125 >> 172.21.15.165:0/3474807125 conn(0x7fa0bc06abe0 msgr2=0x7fa0bc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 -- 172.21.15.165:0/3474807125 shutdown_connections 2024-04-25T10:20:50.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.017+0000 7fa0c0a00640 1 -- 172.21.15.165:0/3474807125 wait complete. 2024-04-25T10:20:50.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.019+0000 7fa0c0a00640 1 Processor -- start 2024-04-25T10:20:50.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.019+0000 7fa0c0a00640 1 -- start start 2024-04-25T10:20:50.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.020+0000 7fa0c0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc13c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0c0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0bc0cd860 0x7fa0bc0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0c0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 0x7fa0bc0aee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0c0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0cdcd0 con 0x7fa0bc144440 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0c0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15bc30 con 0x7fa0bc0cd860 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0c0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0cde50 con 0x7fa0bc071530 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 0x7fa0bc0aee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 0x7fa0bc0aee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52176/0 (socket says 172.21.15.165:52176) 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 -- 172.21.15.165:0/1969548025 learned_addr learned my addr 172.21.15.165:0/1969548025 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 -- 172.21.15.165:0/1969548025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 msgr2=0x7fa0bc13c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc13c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 -- 172.21.15.165:0/1969548025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0bc0cd860 msgr2=0x7fa0bc0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0bc0cd860 0x7fa0bc0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.021+0000 7fa0bb200640 1 -- 172.21.15.165:0/1969548025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0ac4d0 con 0x7fa0bc144440 2024-04-25T10:20:50.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.022+0000 7fa0bb200640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 0x7fa0bc0aee50 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009f40 tx=0x7fa0b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.022+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4036050 con 0x7fa0bc144440 2024-04-25T10:20:50.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.022+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b4042070 con 0x7fa0bc144440 2024-04-25T10:20:50.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.022+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4009510 con 0x7fa0bc144440 2024-04-25T10:20:50.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.022+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc137f80 con 0x7fa0bc144440 2024-04-25T10:20:50.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.022+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc138540 con 0x7fa0bc144440 2024-04-25T10:20:50.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.024+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc073990 con 0x7fa0bc144440 2024-04-25T10:20:50.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.024+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fa0b4007c70 con 0x7fa0bc144440 2024-04-25T10:20:50.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.025+0000 7fa0b2a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa09806b4d0 0x7fa09806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.025+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6217+0+0 (secure 0 0 0) 0x7fa0b40ab410 con 0x7fa0bc144440 2024-04-25T10:20:50.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.026+0000 7fa0ba800640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa09806b4d0 0x7fa09806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.026+0000 7fa0ba800640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa09806b4d0 0x7fa09806d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.029+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fa0bc0ac4d0 con 0x7fa0bc144440 2024-04-25T10:20:50.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.255+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0bc138c30 con 0x7fa0bc144440 2024-04-25T10:20:50.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.257+0000 7fa0b2a00640 1 -- 172.21.15.165:0/1969548025 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 139 v139) v1 ==== 95+0+4963 (secure 0 0 0) 0x7fa0b4045360 con 0x7fa0bc144440 2024-04-25T10:20:50.260 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:50.261 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":139,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":139,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:40.633611+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11610},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11610":{"gid":11610,"name":"c","rank":0,"incarnation":135,"state":"up:active","state_seq":3,"addr":"172.21.15.165:6837/3988146762","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3988146762},{"type":"v1","addr":"172.21.15.165:6837","nonce":3988146762}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:20:50.261 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 139 2024-04-25T10:20:50.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa09806b4d0 msgr2=0x7fa09806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa09806b4d0 0x7fa09806d990 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac006fd0 tx=0x7fa0ac008040 comp rx=0 tx=0).stop 2024-04-25T10:20:50.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 msgr2=0x7fa0bc0aee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 0x7fa0bc0aee50 secure :-1 s=READY pgs=2770 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009f40 tx=0x7fa0b4004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:50.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 shutdown_connections 2024-04-25T10:20:50.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fa09806b4d0 0x7fa09806d990 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fa0bc144440 0x7fa0bc0aee50 unknown :-1 s=CLOSED pgs=2770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fa0bc0cd860 0x7fa0bc0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 --2- 172.21.15.165:0/1969548025 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fa0bc071530 0x7fa0bc13c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 >> 172.21.15.165:0/1969548025 conn(0x7fa0bc06abe0 msgr2=0x7fa0bc142ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:50.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.259+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 shutdown_connections 2024-04-25T10:20:50.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.260+0000 7fa0c0a00640 1 -- 172.21.15.165:0/1969548025 wait complete. 2024-04-25T10:20:50.280 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json flush journal 2024-04-25T10:20:50.520 INFO:teuthology.orchestra.run.smithi165.stdout:{"message":"","return_code":0} 2024-04-25T10:20:50.521 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2024-04-25T10:20:50.521 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:20:50.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.713+0000 7f34ef200640 1 Processor -- start 2024-04-25T10:20:50.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.713+0000 7f34ef200640 1 -- start start 2024-04-25T10:20:50.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.714+0000 7f34ef200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f34f006bff0 0x7f34f00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.714+0000 7f34ef200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f015e110 con 0x7f34f00b5970 2024-04-25T10:20:50.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.714+0000 7f34ef200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34f00b2fd0 con 0x7f34f015dd30 2024-04-25T10:20:50.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.714+0000 7f34ef200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34f006d000 con 0x7f34f006bff0 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ee800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f34f00b5970 0x7f34f00d2160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52330/0 (socket says 172.21.15.165:52330) 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ee800640 1 -- 172.21.15.165:0/2260348127 learned_addr learned my addr 172.21.15.165:0/2260348127 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34e5400640 1 --2- 172.21.15.165:0/2260348127 >> v2:172.21.15.172:3300/0 conn(0x7f34f006bff0 0x7f34f00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3498413793 0 0) 0x7f34f015e110 con 0x7f34f00b5970 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d4003520 con 0x7f34f00b5970 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2023345790 0 0) 0x7f34d4003520 con 0x7f34f00b5970 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 >> v2:172.21.15.172:3300/0 conn(0x7f34f006bff0 msgr2=0x7f34f00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 --2- 172.21.15.165:0/2260348127 >> v2:172.21.15.172:3300/0 conn(0x7f34f006bff0 0x7f34f00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 >> v1:172.21.15.172:6789/0 conn(0x7f34f015dd30 legacy=0x7f34f00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f006c480 con 0x7f34f00b5970 2024-04-25T10:20:50.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.715+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f34e80019f0 con 0x7f34f00b5970 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 >> v1:172.21.15.165:6789/0 conn(0x7f34f00b5970 legacy=0x7f34f00d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d40039b0 0x7f34d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d4004800 0x7f34d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d40051c0 0x7f34d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f006d000 con 0x7f34d40051c0 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d400c060 con 0x7f34d4004800 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d400c260 con 0x7f34d40039b0 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ee800640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d40051c0 0x7f34d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34e5400640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d4004800 0x7f34d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ee800640 1 -- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d40039b0 msgr2=0x7f34d4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ee800640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34d40039b0 0x7f34d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ee800640 1 -- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d4004800 msgr2=0x7f34d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ee800640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d4004800 0x7f34d4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.716+0000 7f34ee800640 1 -- 172.21.15.165:0/2260348127 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4003520 con 0x7f34d40051c0 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.717+0000 7f34ee800640 1 -- 172.21.15.165:0/2260348127 reap_dead start 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.717+0000 7f34ee800640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d40051c0 0x7f34d400b9c0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f34e800a850 tx=0x7f34d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74d0a248875e531 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e800a4a0 con 0x7f34d40051c0 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e802f070 con 0x7f34d40051c0 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ed400640 1 -- 172.21.15.165:0/2260348127 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e802bce0 con 0x7f34d40051c0 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 -- 172.21.15.165:0/2260348127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d40051c0 msgr2=0x7f34d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d40051c0 0x7f34d400b9c0 secure :-1 s=READY pgs=2772 cs=0 l=1 rev1=1 crypto rx=0x7f34e800a850 tx=0x7f34d4005830 comp rx=0 tx=0).stop 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 -- 172.21.15.165:0/2260348127 shutdown_connections 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 --2- 172.21.15.165:0/2260348127 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34d40051c0 0x7f34d400b9c0 unknown :-1 s=CLOSED pgs=2772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 -- 172.21.15.165:0/2260348127 >> 172.21.15.165:0/2260348127 conn(0x7f34f00d0610 msgr2=0x7f34f00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 -- 172.21.15.165:0/2260348127 shutdown_connections 2024-04-25T10:20:50.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.718+0000 7f34ef200640 1 -- 172.21.15.165:0/2260348127 wait complete. 2024-04-25T10:20:50.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.720+0000 7f34ef200640 1 Processor -- start 2024-04-25T10:20:50.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.720+0000 7f34ef200640 1 -- start start 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34ef200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d40051c0 0x7f34f01529f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34ef200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 0x7f34f0154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34ef200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34f0157890 0x7f34f0160bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34ef200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f006cd20 con 0x7f34f0159510 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34ef200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f006ca40 con 0x7f34d40051c0 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34ef200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34f00b5430 con 0x7f34f0157890 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.721+0000 7f34e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 0x7f34f0154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 0x7f34f0154090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52204/0 (socket says 172.21.15.165:52204) 2024-04-25T10:20:50.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 -- 172.21.15.165:0/2901511143 learned_addr learned my addr 172.21.15.165:0/2901511143 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:50.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 -- 172.21.15.165:0/2901511143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34f0157890 msgr2=0x7f34f0160bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34f0157890 0x7f34f0160bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 -- 172.21.15.165:0/2901511143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d40051c0 msgr2=0x7f34f01529f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d40051c0 0x7f34f01529f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 -- 172.21.15.165:0/2901511143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f0159aa0 con 0x7f34f0159510 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e5400640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 0x7f34f0154090 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f34d80115a0 tx=0x7f34d8011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.722+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d8015930 con 0x7f34f0159510 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.723+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d8015ad0 con 0x7f34f0159510 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.723+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f0159d80 con 0x7f34f0159510 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.723+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d801d400 con 0x7f34f0159510 2024-04-25T10:20:50.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.723+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f0153890 con 0x7f34f0159510 2024-04-25T10:20:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.724+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f0073990 con 0x7f34f0159510 2024-04-25T10:20:50.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.725+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f34d8005990 con 0x7f34f0159510 2024-04-25T10:20:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.726+0000 7f34e6a00640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34cc06b500 0x7f34cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.726+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6217+0+0 (secure 0 0 0) 0x7f34d808b170 con 0x7f34f0159510 2024-04-25T10:20:50.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.726+0000 7f34ede00640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34cc06b500 0x7f34cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:50.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.727+0000 7f34ede00640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34cc06b500 0x7f34cc06d9c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f34e00015b0 tx=0x7f34e00079b0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:50.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.730+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f34d8022060 con 0x7f34f0159510 2024-04-25T10:20:50.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.947+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f34f00cf100 con 0x7f34f0159510 2024-04-25T10:20:50.952 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:20:50.949+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:20:50.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.953+0000 7f34e6a00640 1 -- 172.21.15.165:0/2901511143 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v140) v1 ==== 161+0+0 (secure 0 0 0) 0x7f34d8019030 con 0x7f34f0159510 2024-04-25T10:20:50.956 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:20:50.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.955+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34cc06b500 msgr2=0x7f34cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.955+0000 7f34ef200640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34cc06b500 0x7f34cc06d9c0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f34e00015b0 tx=0x7f34e00079b0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 msgr2=0x7f34f0154090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:50.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 0x7f34f0154090 secure :-1 s=READY pgs=2773 cs=0 l=1 rev1=1 crypto rx=0x7f34d80115a0 tx=0x7f34d8011a60 comp rx=0 tx=0).stop 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 shutdown_connections 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f34cc06b500 0x7f34cc06d9c0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f34f0157890 0x7f34f0160bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f34f0159510 0x7f34f0154090 unknown :-1 s=CLOSED pgs=2773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 --2- 172.21.15.165:0/2901511143 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f34d40051c0 0x7f34f01529f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 >> 172.21.15.165:0/2901511143 conn(0x7f34f00d0610 msgr2=0x7f34f00b6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:50.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.956+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 shutdown_connections 2024-04-25T10:20:50.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:50.957+0000 7f34ef200640 1 -- 172.21.15.165:0/2901511143 wait complete. 2024-04-25T10:20:50.969 INFO:tasks.ceph.mds.c.smithi165.stderr: -5412> 2024-04-25T10:20:44.081+0000 7f0a39600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-25T10:20:50.977 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getomapval 10000000000.00000000 parentfile_head - 2024-04-25T10:20:51.021 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:20:51.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.061+0000 7f5babc83f40 1 Processor -- start 2024-04-25T10:20:51.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.061+0000 7f5babc83f40 1 -- start start 2024-04-25T10:20:51.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.061+0000 7f5babc83f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b900022a0 0x563bcfe2dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.061+0000 7f5babc83f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bcfe15ad0 0x563bcfeb4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.061+0000 7f5babc83f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 0x563bcfe2d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5babc83f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfd8add0 con 0x563bcfebd0b0 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5babc83f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfd8ed30 con 0x563bcfe15ad0 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5babc83f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bcfde1b40 con 0x7f5b900022a0 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 0x563bcfe2d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 0x563bcfe2d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52252/0 (socket says 172.21.15.165:52252) 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 -- 172.21.15.165:0/3656349302 learned_addr learned my addr 172.21.15.165:0/3656349302 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 -- 172.21.15.165:0/3656349302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b900022a0 msgr2=0x563bcfe2dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b900022a0 0x563bcfe2dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 -- 172.21.15.165:0/3656349302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bcfe15ad0 msgr2=0x563bcfeb4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bcfe15ad0 0x563bcfeb4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 -- 172.21.15.165:0/3656349302 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bcfeb4c20 con 0x563bcfebd0b0 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.062+0000 7f5ba9800640 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 0x563bcfe2d060 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4002460 tx=0x7f5ba4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8054f1ce104c9eba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5ba2a00640 1 -- 172.21.15.165:0/3656349302 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40124c0 con 0x563bcfebd0b0 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5ba2a00640 1 -- 172.21.15.165:0/3656349302 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ba4018070 con 0x563bcfebd0b0 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5ba2a00640 1 -- 172.21.15.165:0/3656349302 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ba40128c0 con 0x563bcfebd0b0 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 -- 172.21.15.165:0/3656349302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 msgr2=0x563bcfe2d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 0x563bcfe2d060 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7f5ba4002460 tx=0x7f5ba4005d80 comp rx=0 tx=0).stop 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 -- 172.21.15.165:0/3656349302 shutdown_connections 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfebd0b0 0x563bcfe2d060 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563bcfe15ad0 0x563bcfeb4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 --2- 172.21.15.165:0/3656349302 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5b900022a0 0x563bcfe2dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 -- 172.21.15.165:0/3656349302 >> 172.21.15.165:0/3656349302 conn(0x563bcfe12e60 msgr2=0x563bcfe13240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 -- 172.21.15.165:0/3656349302 shutdown_connections 2024-04-25T10:20:51.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.063+0000 7f5babc83f40 1 -- 172.21.15.165:0/3656349302 wait complete. 2024-04-25T10:20:51.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.064+0000 7f5babc83f40 1 Processor -- start 2024-04-25T10:20:51.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.064+0000 7f5babc83f40 1 -- start start 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5babc83f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfe15ad0 0x563bcfe14140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5babc83f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563bcfebd0b0 0x563bcfe2d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5babc83f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b900022a0 0x563bcfe9d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5babc83f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfebc8b0 con 0x563bcfe15ad0 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5babc83f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bcfeb4960 con 0x7f5b900022a0 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5babc83f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563bcfebcb90 con 0x563bcfebd0b0 2024-04-25T10:20:51.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5ba8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfe15ad0 0x563bcfe14140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5ba8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfe15ad0 0x563bcfe14140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52258/0 (socket says 172.21.15.165:52258) 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.065+0000 7f5ba8e00640 1 -- 172.21.15.165:0/953160180 learned_addr learned my addr 172.21.15.165:0/953160180 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba8e00640 1 -- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563bcfebd0b0 msgr2=0x563bcfe2d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba8e00640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563bcfebd0b0 0x563bcfe2d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba8e00640 1 -- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b900022a0 msgr2=0x563bcfe9d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba8e00640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5b900022a0 0x563bcfe9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba8e00640 1 -- 172.21.15.165:0/953160180 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bcfebe160 con 0x563bcfe15ad0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba8e00640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfe15ad0 0x563bcfe14140 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0257a0 tx=0x7f5b9c054cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba1600640 1 -- 172.21.15.165:0/953160180 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c064020 con 0x563bcfe15ad0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba1600640 1 -- 172.21.15.165:0/953160180 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b9c05a040 con 0x563bcfe15ad0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563bcfe10200 con 0x563bcfe15ad0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.066+0000 7f5ba1600640 1 -- 172.21.15.165:0/953160180 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9c059070 con 0x563bcfe15ad0 2024-04-25T10:20:51.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.067+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bcfeb8b10 con 0x563bcfe15ad0 2024-04-25T10:20:51.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.069+0000 7f5ba1600640 1 -- 172.21.15.165:0/953160180 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5b9c067020 con 0x563bcfe15ad0 2024-04-25T10:20:51.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.069+0000 7f5ba1600640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b88069eb0 0x7f5b8806c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.069+0000 7f5ba1600640 1 -- 172.21.15.165:0/953160180 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5b9c0ccb90 con 0x563bcfe15ad0 2024-04-25T10:20:51.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.070+0000 7f5ba3e00640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b88069eb0 0x7f5b8806c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.070+0000 7f5babc83f40 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x563bcfe2d790 0x563bcfeb7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.070+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=23b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e198) v9 -- 0x563bcfeb81e0 con 0x563bcfe2d790 2024-04-25T10:20:51.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.070+0000 7f5ba3e00640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b88069eb0 0x7f5b8806c370 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f5b94000f60 tx=0x7f5b94001490 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.070+0000 7f5ba9800640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x563bcfe2d790 0x563bcfeb7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.071+0000 7f5ba9800640 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x563bcfe2d790 0x563bcfeb7ca0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.072+0000 7f5ba9800640 1 -- 172.21.15.165:0/953160180 <== osd.6 v2:172.21.15.172:6808/4001543510 1 ==== osd_op_reply(1 10000000000.00000000 [omap-get-vals-by-keys out=596b] v0'0 uv10 ondisk = 0) v8 ==== 164+0+596 (crc 0 0 0) 0x7f5ba40052a0 con 0x563bcfe2d790 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:Writing to - 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.072+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x563bcfe2d790 msgr2=0x563bcfeb7ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.072+0000 7f5babc83f40 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x563bcfe2d790 0x563bcfeb7ca0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.072+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b88069eb0 msgr2=0x7f5b8806c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.072+0000 7f5babc83f40 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5b88069eb0 0x7f5b8806c370 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f5b94000f60 tx=0x7f5b94001490 comp rx=0 tx=0).stop 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfe15ad0 msgr2=0x563bcfe14140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5babc83f40 1 --2- 172.21.15.165:0/953160180 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563bcfe15ad0 0x563bcfe14140 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7f5b9c0257a0 tx=0x7f5b9c054cf0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5ba9800640 1 -- 172.21.15.165:0/953160180 reap_dead start 2024-04-25T10:20:51.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 shutdown_connections 2024-04-25T10:20:51.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 >> 172.21.15.165:0/953160180 conn(0x563bcfe12e60 msgr2=0x563bcfe2df40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 shutdown_connections 2024-04-25T10:20:51.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.073+0000 7f5babc83f40 1 -- 172.21.15.165:0/953160180 wait complete. 2024-04-25T10:20:51.081 INFO:teuthology.orchestra.run.smithi165.stdout:i* �~.*f�$���$@@`��������~.*f�$~.*f��A!(8`~.*f�$8`~.*f�$��������~.*f��A!����`security.selinux%unconfined_u:object_r:unlabeled_t:s0�������� 2024-04-25T10:20:51.081 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata setomapval 10000000000.00000000 shadow_head 2024-04-25T10:20:51.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.206+0000 7f43c7925f40 1 Processor -- start 2024-04-25T10:20:51.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.206+0000 7f43c7925f40 1 -- start start 2024-04-25T10:20:51.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.206+0000 7f43c7925f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b126f0 0x559df8b94670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.206+0000 7f43c7925f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8ba31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.206+0000 7f43c7925f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b17bb0 0x559df8ba3720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c7925f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df8ac8840 con 0x559df8b177d0 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c7925f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df8a71dd0 con 0x559df8b126f0 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c7925f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x559df8ac8ae0 con 0x559df8b17bb0 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8ba31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8ba31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52272/0 (socket says 172.21.15.165:52272) 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 -- 172.21.15.165:0/2672862085 learned_addr learned my addr 172.21.15.165:0/2672862085 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 -- 172.21.15.165:0/2672862085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b17bb0 msgr2=0x559df8ba3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b17bb0 0x559df8ba3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 -- 172.21.15.165:0/2672862085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b126f0 msgr2=0x559df8b94670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b126f0 0x559df8b94670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 -- 172.21.15.165:0/2672862085 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559df8afca40 con 0x559df8b177d0 2024-04-25T10:20:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.207+0000 7f43c3800640 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8ba31e0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f43ac057040 tx=0x7f43ac04e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3749f3a6530f5da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c2400640 1 -- 172.21.15.165:0/2672862085 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac065070 con 0x559df8b177d0 2024-04-25T10:20:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c2400640 1 -- 172.21.15.165:0/2672862085 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac0549f0 con 0x559df8b177d0 2024-04-25T10:20:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c2400640 1 -- 172.21.15.165:0/2672862085 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac059040 con 0x559df8b177d0 2024-04-25T10:20:51.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 -- 172.21.15.165:0/2672862085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 msgr2=0x559df8ba31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8ba31e0 secure :-1 s=READY pgs=2783 cs=0 l=1 rev1=1 crypto rx=0x7f43ac057040 tx=0x7f43ac04e000 comp rx=0 tx=0).stop 2024-04-25T10:20:51.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 -- 172.21.15.165:0/2672862085 shutdown_connections 2024-04-25T10:20:51.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b17bb0 0x559df8ba3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8ba31e0 unknown :-1 s=CLOSED pgs=2783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2672862085 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b126f0 0x559df8b94670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 -- 172.21.15.165:0/2672862085 >> 172.21.15.165:0/2672862085 conn(0x559df8af9e60 msgr2=0x559df8b148b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 -- 172.21.15.165:0/2672862085 shutdown_connections 2024-04-25T10:20:51.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.208+0000 7f43c7925f40 1 -- 172.21.15.165:0/2672862085 wait complete. 2024-04-25T10:20:51.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.209+0000 7f43c7925f40 1 Processor -- start 2024-04-25T10:20:51.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.209+0000 7f43c7925f40 1 -- start start 2024-04-25T10:20:51.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c7925f40 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b126f0 0x559df8b14aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c7925f40 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8b9a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c7925f40 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b17bb0 0x559df8c31ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c7925f40 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df8afcd20 con 0x559df8b177d0 2024-04-25T10:20:51.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c7925f40 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x559df8b9c100 con 0x559df8b17bb0 2024-04-25T10:20:51.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c7925f40 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x559df8afd000 con 0x559df8b126f0 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c3800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8b9a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c3800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8b9a320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52288/0 (socket says 172.21.15.165:52288) 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.210+0000 7f43c3800640 1 -- 172.21.15.165:0/2975728064 learned_addr learned my addr 172.21.15.165:0/2975728064 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c4200640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b126f0 0x559df8b14aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c4c00640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b17bb0 0x559df8c31ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c3800640 1 -- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b126f0 msgr2=0x559df8b14aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c3800640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b126f0 0x559df8b14aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c3800640 1 -- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b17bb0 msgr2=0x559df8c31ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c3800640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x559df8b17bb0 0x559df8c31ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c3800640 1 -- 172.21.15.165:0/2975728064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559df8b98710 con 0x559df8b177d0 2024-04-25T10:20:51.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c4200640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x559df8b126f0 0x559df8b14aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c3800640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8b9a320 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f43ac063040 tx=0x7f43ac002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c1000640 1 -- 172.21.15.165:0/2975728064 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac06d070 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c1000640 1 -- 172.21.15.165:0/2975728064 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43ac060420 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c1000640 1 -- 172.21.15.165:0/2975728064 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43ac065040 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.211+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559df8b989f0 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.212+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559df8c316d0 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.214+0000 7f43c1000640 1 -- 172.21.15.165:0/2975728064 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f43ac070070 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.214+0000 7f43c1000640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f43a0069eb0 0x7f43a006c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.215+0000 7f43c1000640 1 -- 172.21.15.165:0/2975728064 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f43ac0ce2c0 con 0x559df8b177d0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.215+0000 7f43c4200640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f43a0069eb0 0x7f43a006c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.215+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x559df8b14fe0 0x559df8c3bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.215+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- osd_op(unknown.0.0:1 29.10 29:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=592b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e198) v9 -- 0x559df8c3c220 con 0x559df8b14fe0 2024-04-25T10:20:51.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.215+0000 7f43c4200640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f43a0069eb0 0x7f43a006c370 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005fd0 tx=0x7f43b4007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.216+0000 7f43c4c00640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x559df8b14fe0 0x559df8c3bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.217+0000 7f43c4c00640 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x559df8b14fe0 0x559df8c3bce0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c4c00640 1 -- 172.21.15.165:0/2975728064 <== osd.6 v2:172.21.15.172:6808/4001543510 1 ==== osd_op_reply(1 10000000000.00000000 [omap-set-vals] v198'11 uv11 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f43bc00e360 con 0x559df8b14fe0 2024-04-25T10:20:51.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x559df8b14fe0 msgr2=0x559df8c3bce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x559df8b14fe0 0x559df8c3bce0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f43a0069eb0 msgr2=0x7f43a006c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f43a0069eb0 0x7f43a006c370 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f43b4005fd0 tx=0x7f43b4007040 comp rx=0 tx=0).stop 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 msgr2=0x559df8b9a320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 --2- 172.21.15.165:0/2975728064 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x559df8b177d0 0x559df8b9a320 secure :-1 s=READY pgs=2784 cs=0 l=1 rev1=1 crypto rx=0x7f43ac063040 tx=0x7f43ac002fb0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c4c00640 1 -- 172.21.15.165:0/2975728064 reap_dead start 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 shutdown_connections 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 >> 172.21.15.165:0/2975728064 conn(0x559df8af9e60 msgr2=0x559df8ac7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 shutdown_connections 2024-04-25T10:20:51.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.221+0000 7f43c7925f40 1 -- 172.21.15.165:0/2975728064 wait complete. 2024-04-25T10:20:51.230 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds mds_hack_allow_loading_invalid_metadata True 2024-04-25T10:20:51.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.465+0000 7f993ee00640 1 Processor -- start 2024-04-25T10:20:51.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.465+0000 7f993ee00640 1 -- start start 2024-04-25T10:20:51.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993ee00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99380b0730 0x7f99380b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993ee00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993813e640 con 0x7f993806e150 2024-04-25T10:20:51.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993ee00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f993806d390 con 0x7f993813d9f0 2024-04-25T10:20:51.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993ee00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f993806d670 con 0x7f99380b0730 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993d000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f99380b0730 0x7f99380b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993e400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f993806e150 0x7f993806e580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52362/0 (socket says 172.21.15.165:52362) 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.467+0000 7f993e400640 1 -- 172.21.15.165:0/1545421199 learned_addr learned my addr 172.21.15.165:0/1545421199 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.468+0000 7f993d000640 1 -- 172.21.15.165:0/1545421199 >> v1:172.21.15.172:6789/0 conn(0x7f993813d9f0 legacy=0x7f993813df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.468+0000 7f993d000640 1 -- 172.21.15.165:0/1545421199 >> v1:172.21.15.165:6789/0 conn(0x7f993806e150 legacy=0x7f993806e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.468+0000 7f993d000640 1 -- 172.21.15.165:0/1545421199 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993806d950 con 0x7f99380b0730 2024-04-25T10:20:51.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.468+0000 7f9937e00640 1 -- 172.21.15.165:0/1545421199 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3439355893 0 0) 0x7f993813e640 con 0x7f993806e150 2024-04-25T10:20:51.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.469+0000 7f993d000640 1 --2- 172.21.15.165:0/1545421199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99380b0730 0x7f99380b0bc0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f99240097a0 tx=0x7f992402a0b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4525be3a51376b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.469+0000 7f9937e00640 1 -- 172.21.15.165:0/1545421199 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924037070 con 0x7f99380b0730 2024-04-25T10:20:51.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.469+0000 7f9937e00640 1 -- 172.21.15.165:0/1545421199 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992402adc0 con 0x7f99380b0730 2024-04-25T10:20:51.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.469+0000 7f9937e00640 1 -- 172.21.15.165:0/1545421199 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9924032660 con 0x7f99380b0730 2024-04-25T10:20:51.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 -- 172.21.15.165:0/1545421199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99380b0730 msgr2=0x7f99380b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 --2- 172.21.15.165:0/1545421199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99380b0730 0x7f99380b0bc0 secure :-1 s=READY pgs=2786 cs=0 l=1 rev1=1 crypto rx=0x7f99240097a0 tx=0x7f992402a0b0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 -- 172.21.15.165:0/1545421199 shutdown_connections 2024-04-25T10:20:51.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 --2- 172.21.15.165:0/1545421199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99380b0730 0x7f99380b0bc0 unknown :-1 s=CLOSED pgs=2786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 -- 172.21.15.165:0/1545421199 >> 172.21.15.165:0/1545421199 conn(0x7f9938061cc0 msgr2=0x7f993806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 -- 172.21.15.165:0/1545421199 shutdown_connections 2024-04-25T10:20:51.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.470+0000 7f993ee00640 1 -- 172.21.15.165:0/1545421199 wait complete. 2024-04-25T10:20:51.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.471+0000 7f993ee00640 1 Processor -- start 2024-04-25T10:20:51.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.472+0000 7f993ee00640 1 -- start start 2024-04-25T10:20:51.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.472+0000 7f993ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f993806e150 0x7f9938158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.472+0000 7f993ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99380b0730 0x7f99381f3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 0x7f9938156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380cea00 con 0x7f993813d9f0 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99380ce880 con 0x7f993806e150 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99380ceb80 con 0x7f99380b0730 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 0x7f9938156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 0x7f9938156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52302/0 (socket says 172.21.15.165:52302) 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 -- 172.21.15.165:0/2666797262 learned_addr learned my addr 172.21.15.165:0/2666797262 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993da00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f993806e150 0x7f9938158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993d000640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99380b0730 0x7f99381f3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 -- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99380b0730 msgr2=0x7f99381f3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99380b0730 0x7f99381f3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 -- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f993806e150 msgr2=0x7f9938158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f993806e150 0x7f9938158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.473+0000 7f993e400640 1 -- 172.21.15.165:0/2666797262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9938154d10 con 0x7f993813d9f0 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.474+0000 7f993e400640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 0x7f9938156530 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f993000c7d0 tx=0x7f993000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.474+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9930018070 con 0x7f993813d9f0 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.474+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f993000a640 con 0x7f993813d9f0 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.474+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f993000aa70 con 0x7f993813d9f0 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.474+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99381570f0 con 0x7f993813d9f0 2024-04-25T10:20:51.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.474+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99381576b0 con 0x7f993813d9f0 2024-04-25T10:20:51.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.475+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9938073990 con 0x7f993813d9f0 2024-04-25T10:20:51.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.476+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f993000ac10 con 0x7f993813d9f0 2024-04-25T10:20:51.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.477+0000 7f9935600640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f991806b4d0 0x7f991806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.477+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f9930089b30 con 0x7f993813d9f0 2024-04-25T10:20:51.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.477+0000 7f993da00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f991806b4d0 0x7f991806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.478+0000 7f993da00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f991806b4d0 0x7f991806d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.482+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f99300586c0 con 0x7f993813d9f0 2024-04-25T10:20:51.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.661+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}] v 0) v1 -- 0x7f99381f4250 con 0x7f993813d9f0 2024-04-25T10:20:51.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.669+0000 7f9935600640 1 -- 172.21.15.165:0/2666797262 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_hack_allow_loading_invalid_metadata}]=0 v35)=0 v35) v1 ==== 140+0+0 (secure 0 0 0) 0x7f9930057fc0 con 0x7f993813d9f0 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f991806b4d0 msgr2=0x7f991806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f991806b4d0 0x7f991806d990 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 msgr2=0x7f9938156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 0x7f9938156530 secure :-1 s=READY pgs=2787 cs=0 l=1 rev1=1 crypto rx=0x7f993000c7d0 tx=0x7f993000cc90 comp rx=0 tx=0).stop 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 shutdown_connections 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f991806b4d0 0x7f991806d990 secure :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f992c006fd0 tx=0x7f992c008040 comp rx=0 tx=0).stop 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f993813d9f0 0x7f9938156530 unknown :-1 s=CLOSED pgs=2787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f99380b0730 0x7f99381f3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 --2- 172.21.15.165:0/2666797262 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f993806e150 0x7f9938158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 >> 172.21.15.165:0/2666797262 conn(0x7f9938061cc0 msgr2=0x7f99380b0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 shutdown_connections 2024-04-25T10:20:51.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.672+0000 7f993ee00640 1 -- 172.21.15.165:0/2666797262 wait complete. 2024-04-25T10:20:51.693 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs joinable true 2024-04-25T10:20:51.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.889+0000 7efde6c00640 1 Processor -- start 2024-04-25T10:20:51.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.889+0000 7efde6c00640 1 -- start start 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efde6c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7efde01452a0 0x7efde006dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efde6c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde00cccc0 con 0x7efde0144d50 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efde6c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde00cc140 con 0x7efde01446d0 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efde6c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde00b2b20 con 0x7efde01452a0 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efdd7400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7efde0144d50 0x7efde006d890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52372/0 (socket says 172.21.15.165:52372) 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efdd7400640 1 -- 172.21.15.165:0/2951094557 learned_addr learned my addr 172.21.15.165:0/2951094557 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:51.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.891+0000 7efde4c00640 1 --2- 172.21.15.165:0/2951094557 >> v2:172.21.15.165:3300/0 conn(0x7efde01452a0 0x7efde006dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.892+0000 7efddf400640 1 -- 172.21.15.165:0/2951094557 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2622808631 0 0) 0x7efde00cccc0 con 0x7efde0144d50 2024-04-25T10:20:51.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.892+0000 7efddf400640 1 -- 172.21.15.165:0/2951094557 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdc4003520 con 0x7efde0144d50 2024-04-25T10:20:51.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.892+0000 7efde4c00640 1 -- 172.21.15.165:0/2951094557 >> v1:172.21.15.172:6789/0 conn(0x7efde01446d0 legacy=0x7efde00b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:51.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.892+0000 7efde4c00640 1 -- 172.21.15.165:0/2951094557 >> v1:172.21.15.165:6789/0 conn(0x7efde0144d50 legacy=0x7efde006d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.892+0000 7efde4c00640 1 -- 172.21.15.165:0/2951094557 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde00cc420 con 0x7efde01452a0 2024-04-25T10:20:51.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.893+0000 7efde4c00640 1 --2- 172.21.15.165:0/2951094557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01452a0 0x7efde006dfa0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009d90 tx=0x7efdd802a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604c982207ff0ada server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.893+0000 7efddf400640 1 -- 172.21.15.165:0/2951094557 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8007740 con 0x7efde01452a0 2024-04-25T10:20:51.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.893+0000 7efddf400640 1 -- 172.21.15.165:0/2951094557 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd80078e0 con 0x7efde01452a0 2024-04-25T10:20:51.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.893+0000 7efddf400640 1 -- 172.21.15.165:0/2951094557 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd8007cb0 con 0x7efde01452a0 2024-04-25T10:20:51.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 -- 172.21.15.165:0/2951094557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01452a0 msgr2=0x7efde006dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 --2- 172.21.15.165:0/2951094557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01452a0 0x7efde006dfa0 secure :-1 s=READY pgs=2789 cs=0 l=1 rev1=1 crypto rx=0x7efdd8009d90 tx=0x7efdd802a6a0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 -- 172.21.15.165:0/2951094557 shutdown_connections 2024-04-25T10:20:51.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 --2- 172.21.15.165:0/2951094557 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01452a0 0x7efde006dfa0 unknown :-1 s=CLOSED pgs=2789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 -- 172.21.15.165:0/2951094557 >> 172.21.15.165:0/2951094557 conn(0x7efde006abe0 msgr2=0x7efde006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:51.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 -- 172.21.15.165:0/2951094557 shutdown_connections 2024-04-25T10:20:51.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.894+0000 7efde6c00640 1 -- 172.21.15.165:0/2951094557 wait complete. 2024-04-25T10:20:51.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.896+0000 7efde6c00640 1 Processor -- start 2024-04-25T10:20:51.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.896+0000 7efde6c00640 1 -- start start 2024-04-25T10:20:51.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde6c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 0x7efde00cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde6c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde0144d50 0x7efde01f3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde6c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efde01452a0 0x7efde01f41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde6c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde00cc9e0 con 0x7efde01446d0 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde6c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde006d400 con 0x7efde0144d50 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde6c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde0144ab0 con 0x7efde01452a0 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efddfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 0x7efde00cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efddfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 0x7efde00cde90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52322/0 (socket says 172.21.15.165:52322) 2024-04-25T10:20:51.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efddfe00640 1 -- 172.21.15.165:0/2745291407 learned_addr learned my addr 172.21.15.165:0/2745291407 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:51.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efddfe00640 1 -- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efde01452a0 msgr2=0x7efde01f41b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efde4c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efde01452a0 0x7efde01f41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efdd7400640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde0144d50 0x7efde01f3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efddfe00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efde01452a0 0x7efde01f41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.897+0000 7efddfe00640 1 -- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde0144d50 msgr2=0x7efde01f3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efddfe00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde0144d50 0x7efde01f3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efddfe00640 1 -- 172.21.15.165:0/2745291407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde0140e60 con 0x7efde01446d0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efdd7400640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde0144d50 0x7efde01f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efddfe00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 0x7efde00cde90 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b3f0 tx=0x7efdd000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0015020 con 0x7efde01446d0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd0011830 con 0x7efde01446d0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd0011c20 con 0x7efde01446d0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde01408a0 con 0x7efde01446d0 2024-04-25T10:20:51.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.898+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde00ac950 con 0x7efde01446d0 2024-04-25T10:20:51.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.900+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde0073a00 con 0x7efde01446d0 2024-04-25T10:20:51.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.901+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7efdd0018020 con 0x7efde01446d0 2024-04-25T10:20:51.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.902+0000 7efddcc00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdbc06b4d0 0x7efdbc06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:51.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.902+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7efdd0089fe0 con 0x7efde01446d0 2024-04-25T10:20:51.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.902+0000 7efdd7400640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdbc06b4d0 0x7efdbc06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:51.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.903+0000 7efdd7400640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdbc06b4d0 0x7efdbc06d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efde00ae3c0 tx=0x7efdc8009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:51.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:51.906+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7efdd001f030 con 0x7efde01446d0 2024-04-25T10:20:52.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.142+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7efde01431c0 con 0x7efde01446d0 2024-04-25T10:20:52.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.960+0000 7efddcc00640 1 -- 172.21.15.165:0/2745291407 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]=0 cephfs marked joinable; MDS may join as newly active. v142) v1 ==== 162+0+0 (secure 0 0 0) 0x7efdd00172c0 con 0x7efde01446d0 2024-04-25T10:20:52.964 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked joinable; MDS may join as newly active. 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdbc06b4d0 msgr2=0x7efdbc06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdbc06b4d0 0x7efdbc06d990 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7efde00ae3c0 tx=0x7efdc8009040 comp rx=0 tx=0).stop 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 msgr2=0x7efde00cde90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 0x7efde00cde90 secure :-1 s=READY pgs=2790 cs=0 l=1 rev1=1 crypto rx=0x7efdd000b3f0 tx=0x7efdd000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 shutdown_connections 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7efdbc06b4d0 0x7efdbc06d990 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7efde01452a0 0x7efde01f41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7efde0144d50 0x7efde01f3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 --2- 172.21.15.165:0/2745291407 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7efde01446d0 0x7efde00cde90 unknown :-1 s=CLOSED pgs=2790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 >> 172.21.15.165:0/2745291407 conn(0x7efde006abe0 msgr2=0x7efde01436e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 shutdown_connections 2024-04-25T10:20:52.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:52.964+0000 7efde6c00640 1 -- 172.21.15.165:0/2745291407 wait complete. 2024-04-25T10:20:52.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.167+0000 7fe957e00640 1 Processor -- start 2024-04-25T10:20:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.167+0000 7fe957e00640 1 -- start start 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe957e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 0x7fe9580cc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe957e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe95806b230 0x7fe9580d0b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe957e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95806fa10 con 0x7fe95806ae50 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe957e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9580702b0 con 0x7fe95806b230 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe957e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806f7c0 con 0x7fe9580cd8a0 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe956a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe95806ae50 0x7fe958161dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52376/0 (socket says 172.21.15.165:52376) 2024-04-25T10:20:53.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.169+0000 7fe956a00640 1 -- 172.21.15.165:0/1576283211 learned_addr learned my addr 172.21.15.165:0/1576283211 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe957400640 1 --2- 172.21.15.165:0/1576283211 >> v2:172.21.15.172:3300/0 conn(0x7fe95806b230 0x7fe9580d0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe956000640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 0x7fe9580cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 677295580 0 0) 0x7fe95806fa10 con 0x7fe95806ae50 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe93c003520 con 0x7fe95806ae50 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2594369524 0 0) 0x7fe93c003520 con 0x7fe95806ae50 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 msgr2=0x7fe9580cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 0x7fe9580cc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 >> v2:172.21.15.172:3300/0 conn(0x7fe95806b230 msgr2=0x7fe9580d0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 --2- 172.21.15.165:0/1576283211 >> v2:172.21.15.172:3300/0 conn(0x7fe95806b230 0x7fe9580d0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95806fcf0 con 0x7fe95806ae50 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe948001670 con 0x7fe95806ae50 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.170+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 >> v1:172.21.15.165:6789/0 conn(0x7fe95806ae50 legacy=0x7fe958161dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe955600640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c0039b0 0x7fe93c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe955600640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe93c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe955600640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 0x7fe93c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95806f7c0 con 0x7fe93c004800 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c00b9c0 con 0x7fe93c0039b0 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe93c00bb90 con 0x7fe9580cd8a0 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe956000640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe93c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe956000640 1 -- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 msgr2=0x7fe93c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe956000640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580cd8a0 0x7fe93c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe956000640 1 -- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c0039b0 msgr2=0x7fe93c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe956000640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe93c0039b0 0x7fe93c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.171+0000 7fe956000640 1 -- 172.21.15.165:0/1576283211 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe93c003520 con 0x7fe93c004800 2024-04-25T10:20:53.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.172+0000 7fe956000640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe93c004c80 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7fe944009ac0 tx=0x7fe93c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7fbfedea24799f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944036070 con 0x7fe93c004800 2024-04-25T10:20:53.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe944031400 con 0x7fe93c004800 2024-04-25T10:20:53.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe955600640 1 -- 172.21.15.165:0/1576283211 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944009690 con 0x7fe93c004800 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957e00640 1 -- 172.21.15.165:0/1576283211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 msgr2=0x7fe93c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957e00640 1 --2- 172.21.15.165:0/1576283211 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe93c004c80 secure :-1 s=READY pgs=2792 cs=0 l=1 rev1=1 crypto rx=0x7fe944009ac0 tx=0x7fe93c005830 comp rx=0 tx=0).stop 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957400640 1 -- 172.21.15.165:0/1576283211 reap_dead start 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957e00640 1 -- 172.21.15.165:0/1576283211 shutdown_connections 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957e00640 1 -- 172.21.15.165:0/1576283211 >> 172.21.15.165:0/1576283211 conn(0x7fe958072660 msgr2=0x7fe958099a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957e00640 1 -- 172.21.15.165:0/1576283211 shutdown_connections 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.173+0000 7fe957e00640 1 -- 172.21.15.165:0/1576283211 wait complete. 2024-04-25T10:20:53.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.175+0000 7fe957e00640 1 Processor -- start 2024-04-25T10:20:53.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.175+0000 7fe957e00640 1 -- start start 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.176+0000 7fe957e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe9580c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.176+0000 7fe957e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580c7f60 0x7fe958152450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.176+0000 7fe957e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9580c9950 0x7fe9580c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.176+0000 7fe957e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9581624e0 con 0x7fe93c004800 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.176+0000 7fe957e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe958161940 con 0x7fe9580c9950 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.176+0000 7fe957e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe958162660 con 0x7fe9580c7f60 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe9580c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe9580c59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52348/0 (socket says 172.21.15.165:52348) 2024-04-25T10:20:53.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 -- 172.21.15.165:0/15173916 learned_addr learned my addr 172.21.15.165:0/15173916 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:53.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 -- 172.21.15.165:0/15173916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580c7f60 msgr2=0x7fe958152450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580c7f60 0x7fe958152450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 -- 172.21.15.165:0/15173916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9580c9950 msgr2=0x7fe9580c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9580c9950 0x7fe9580c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 -- 172.21.15.165:0/15173916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9580c60f0 con 0x7fe93c004800 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe956a00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe9580c59e0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c730 tx=0x7fe94800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.177+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948018070 con 0x7fe93c004800 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.178+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9480074e0 con 0x7fe93c004800 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.178+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9580c63d0 con 0x7fe93c004800 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.178+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948004a20 con 0x7fe93c004800 2024-04-25T10:20:53.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.178+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9580ca310 con 0x7fe93c004800 2024-04-25T10:20:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.179+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe958073990 con 0x7fe93c004800 2024-04-25T10:20:53.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.180+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe94801b070 con 0x7fe93c004800 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.181+0000 7fe94f400640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:53.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.181+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe9480899a0 con 0x7fe93c004800 2024-04-25T10:20:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.181+0000 7fe956000640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:53.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.182+0000 7fe956000640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe944033000 tx=0x7fe94402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:53.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.185+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe9480585c0 con 0x7fe93c004800 2024-04-25T10:20:53.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.418+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe9580ce2a0 con 0x7fe93c004800 2024-04-25T10:20:53.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.419+0000 7fe94f400640 1 -- 172.21.15.165:0/15173916 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 143 v143) v1 ==== 95+0+4965 (secure 0 0 0) 0x7fe948057ec0 con 0x7fe93c004800 2024-04-25T10:20:53.423 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:53.423 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":143,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":143,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:52.967185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:replay","state_seq":1,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:20:53.423 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 143 2024-04-25T10:20:53.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.421+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe93806b4d0 msgr2=0x7fe93806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe93806b4d0 0x7fe93806d990 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fe944033000 tx=0x7fe94402e000 comp rx=0 tx=0).stop 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 msgr2=0x7fe9580c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe9580c59e0 secure :-1 s=READY pgs=2793 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c730 tx=0x7fe94800cbf0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 shutdown_connections 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe93806b4d0 0x7fe93806d990 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe9580c9950 0x7fe9580c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe9580c7f60 0x7fe958152450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 --2- 172.21.15.165:0/15173916 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe93c004800 0x7fe9580c59e0 unknown :-1 s=CLOSED pgs=2793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 >> 172.21.15.165:0/15173916 conn(0x7fe958072660 msgr2=0x7fe9580adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 shutdown_connections 2024-04-25T10:20:53.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:53.422+0000 7fe957e00640 1 -- 172.21.15.165:0/15173916 wait complete. 2024-04-25T10:20:53.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 143, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:37.587736+0000', 'modified': '2024-04-25T10:20:52.967185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11808': {'gid': 11808, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.165:6837/2156983703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 2156983703}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2156983703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:53.845 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11808:up:replay 2024-04-25T10:20:54.846 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:55.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.054+0000 7f2376c00640 1 Processor -- start 2024-04-25T10:20:55.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.055+0000 7f2376c00640 1 -- start start 2024-04-25T10:20:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.056+0000 7f2376c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23700b2cb0 0x7f23700d2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.056+0000 7f2376c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f23700d1020 0x7f23700d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.056+0000 7f2376c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23700b97a0 con 0x7f237015dbe0 2024-04-25T10:20:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.056+0000 7f2376c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237015dfc0 con 0x7f23700d1020 2024-04-25T10:20:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.056+0000 7f2376c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23700d1a00 con 0x7f23700b2cb0 2024-04-25T10:20:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.057+0000 7f236fe00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f237015dbe0 0x7f23700d0910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:44344/0 (socket says 172.21.15.165:44344) 2024-04-25T10:20:55.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.057+0000 7f236fe00640 1 -- 172.21.15.165:0/2797216136 learned_addr learned my addr 172.21.15.165:0/2797216136 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.057+0000 7f2374c00640 1 --2- 172.21.15.165:0/2797216136 >> v2:172.21.15.172:3300/0 conn(0x7f23700d1020 0x7f23700d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.057+0000 7f236f400640 1 --2- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23700b2cb0 0x7f23700d2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.058+0000 7f236ea00640 1 -- 172.21.15.165:0/2797216136 <== mon.1 v1:172.21.15.172:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996565317 0 0) 0x7f23700b97a0 con 0x7f237015dbe0 2024-04-25T10:20:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.058+0000 7f236ea00640 1 -- 172.21.15.165:0/2797216136 --> v1:172.21.15.172:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2350003520 con 0x7f237015dbe0 2024-04-25T10:20:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.058+0000 7f2374c00640 1 -- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23700b2cb0 msgr2=0x7f23700d2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.058+0000 7f2374c00640 1 --2- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23700b2cb0 0x7f23700d2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.058+0000 7f2374c00640 1 -- 172.21.15.165:0/2797216136 >> v1:172.21.15.172:6789/0 conn(0x7f237015dbe0 legacy=0x7f23700d0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:55.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.058+0000 7f2374c00640 1 -- 172.21.15.165:0/2797216136 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700d1c50 con 0x7f23700d1020 2024-04-25T10:20:55.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.059+0000 7f2374c00640 1 --2- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700d1020 0x7f23700d2c70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2368009d90 tx=0x7f236802a660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f4e52abe0f2adf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f236ea00640 1 -- 172.21.15.165:0/2797216136 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368007740 con 0x7f23700d1020 2024-04-25T10:20:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f236ea00640 1 -- 172.21.15.165:0/2797216136 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23680078e0 con 0x7f23700d1020 2024-04-25T10:20:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f236ea00640 1 -- 172.21.15.165:0/2797216136 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2368007cb0 con 0x7f23700d1020 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f2376c00640 1 -- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700d1020 msgr2=0x7f23700d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f2376c00640 1 --2- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700d1020 0x7f23700d2c70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f2368009d90 tx=0x7f236802a660 comp rx=0 tx=0).stop 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f2376c00640 1 -- 172.21.15.165:0/2797216136 shutdown_connections 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f2376c00640 1 --2- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f23700b2cb0 0x7f23700d2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f2376c00640 1 --2- 172.21.15.165:0/2797216136 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700d1020 0x7f23700d2c70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.060+0000 7f2376c00640 1 -- 172.21.15.165:0/2797216136 >> 172.21.15.165:0/2797216136 conn(0x7f2370061a20 msgr2=0x7f2370061e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.061+0000 7f2376c00640 1 -- 172.21.15.165:0/2797216136 shutdown_connections 2024-04-25T10:20:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.061+0000 7f2376c00640 1 -- 172.21.15.165:0/2797216136 wait complete. 2024-04-25T10:20:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.062+0000 7f2376c00640 1 Processor -- start 2024-04-25T10:20:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.063+0000 7f2376c00640 1 -- start start 2024-04-25T10:20:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.063+0000 7f2376c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700b2cb0 0x7f237014f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f2376c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 0x7f2370156960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f2376c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237015dbe0 0x7f237015ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f2376c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23700b92a0 con 0x7f23700d1020 2024-04-25T10:20:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f2376c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23700d2210 con 0x7f23700b2cb0 2024-04-25T10:20:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f2376c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23700b9420 con 0x7f237015dbe0 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 0x7f2370156960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 0x7f2370156960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:52362/0 (socket says 172.21.15.165:52362) 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 -- 172.21.15.165:0/334853161 learned_addr learned my addr 172.21.15.165:0/334853161 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236fe00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700b2cb0 0x7f237014f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f2374c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237015dbe0 0x7f237015ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 -- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237015dbe0 msgr2=0x7f237015ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237015dbe0 0x7f237015ff60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 -- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700b2cb0 msgr2=0x7f237014f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700b2cb0 0x7f237014f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236f400640 1 -- 172.21.15.165:0/334853161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23700cabd0 con 0x7f23700d1020 2024-04-25T10:20:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.064+0000 7f236fe00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700b2cb0 0x7f237014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.065+0000 7f236f400640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 0x7f2370156960 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f235800e9e0 tx=0x7f235800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.065+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2358017070 con 0x7f23700d1020 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.065+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f235800c9d0 con 0x7f23700d1020 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.065+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235800cdc0 con 0x7f23700d1020 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.065+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23700cae90 con 0x7f23700d1020 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.065+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23700cd650 con 0x7f23700d1020 2024-04-25T10:20:55.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.066+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2370073990 con 0x7f23700d1020 2024-04-25T10:20:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.067+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f235801a070 con 0x7f23700d1020 2024-04-25T10:20:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.068+0000 7f2367e00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.068+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f235808a450 con 0x7f23700d1020 2024-04-25T10:20:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.068+0000 7f236fe00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.069+0000 7f236fe00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f234c06b500 0x7f234c06d9c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f23600011e0 tx=0x7f2360009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.073+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2358059070 con 0x7f23700d1020 2024-04-25T10:20:55.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.301+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23700ca2e0 con 0x7f23700d1020 2024-04-25T10:20:55.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.302+0000 7f2367e00640 1 -- 172.21.15.165:0/334853161 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 145 v145) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f2358058970 con 0x7f23700d1020 2024-04-25T10:20:55.306 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:55.306 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":145,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":145,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:53.979899+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:rejoin","state_seq":3,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:20:55.307 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 145 2024-04-25T10:20:55.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f234c06b500 msgr2=0x7f234c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:55.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f234c06b500 0x7f234c06d9c0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f23600011e0 tx=0x7f2360009040 comp rx=0 tx=0).stop 2024-04-25T10:20:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 msgr2=0x7f2370156960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 0x7f2370156960 secure :-1 s=READY pgs=2794 cs=0 l=1 rev1=1 crypto rx=0x7f235800e9e0 tx=0x7f235800eea0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 shutdown_connections 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f234c06b500 0x7f234c06d9c0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f237015dbe0 0x7f237015ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f23700d1020 0x7f2370156960 unknown :-1 s=CLOSED pgs=2794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 --2- 172.21.15.165:0/334853161 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f23700b2cb0 0x7f237014f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.305+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 >> 172.21.15.165:0/334853161 conn(0x7f2370061a20 msgr2=0x7f23700b8b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.306+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 shutdown_connections 2024-04-25T10:20:55.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:55.306+0000 7f2376c00640 1 -- 172.21.15.165:0/334853161 wait complete. 2024-04-25T10:20:55.730 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 145, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:37.587736+0000', 'modified': '2024-04-25T10:20:53.979899+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11808': {'gid': 11808, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:rejoin', 'state_seq': 3, 'addr': '172.21.15.165:6837/2156983703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 2156983703}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2156983703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:55.731 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_11808:up:rejoin 2024-04-25T10:20:56.731 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:20:56.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 Processor -- start 2024-04-25T10:20:56.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 -- start start 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f78cc143e60 0x7f78cc06e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc06e6a0 0x7f78cc0cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f78cc0cc840 0x7f78cc070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc144240 con 0x7f78cc143e60 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78cc15e220 con 0x7f78cc0cc840 2024-04-25T10:20:56.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.942+0000 7f78d0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc15e500 con 0x7f78cc06e6a0 2024-04-25T10:20:56.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f78cc143e60 0x7f78cc06e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f78cc143e60 0x7f78cc06e160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53912/0 (socket says 172.21.15.165:53912) 2024-04-25T10:20:56.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 -- 172.21.15.165:0/1095498088 learned_addr learned my addr 172.21.15.165:0/1095498088 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78caa00640 1 --2- 172.21.15.165:0/1095498088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc06e6a0 0x7f78cc0cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cbe00640 1 --2- 172.21.15.165:0/1095498088 >> v2:172.21.15.172:3300/0 conn(0x7f78cc0cc840 0x7f78cc070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 -- 172.21.15.165:0/1095498088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc06e6a0 msgr2=0x7f78cc0cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 --2- 172.21.15.165:0/1095498088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc06e6a0 0x7f78cc0cc300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 -- 172.21.15.165:0/1095498088 >> v2:172.21.15.172:3300/0 conn(0x7f78cc0cc840 msgr2=0x7f78cc070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 --2- 172.21.15.165:0/1095498088 >> v2:172.21.15.172:3300/0 conn(0x7f78cc0cc840 0x7f78cc070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cb400640 1 -- 172.21.15.165:0/1095498088 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc15e7e0 con 0x7f78cc143e60 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.943+0000 7f78cbe00640 1 --2- 172.21.15.165:0/1095498088 >> v2:172.21.15.172:3300/0 conn(0x7f78cc0cc840 0x7f78cc070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.944+0000 7f78cb400640 1 --2- 172.21.15.165:0/1095498088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc143e60 0x7f78cc06e160 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0b2740 tx=0x7f78bc02c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2f48efcffe02499 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.944+0000 7f78ca000640 1 -- 172.21.15.165:0/1095498088 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00b2e0 con 0x7f78cc143e60 2024-04-25T10:20:56.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78ca000640 1 -- 172.21.15.165:0/1095498088 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78bc009600 con 0x7f78cc143e60 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78ca000640 1 -- 172.21.15.165:0/1095498088 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78bc00b480 con 0x7f78cc143e60 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 -- 172.21.15.165:0/1095498088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc143e60 msgr2=0x7f78cc06e160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1095498088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc143e60 0x7f78cc06e160 secure :-1 s=READY pgs=2795 cs=0 l=1 rev1=1 crypto rx=0x7f78cc0b2740 tx=0x7f78bc02c8a0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 -- 172.21.15.165:0/1095498088 shutdown_connections 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1095498088 >> v2:172.21.15.172:3300/0 conn(0x7f78cc0cc840 0x7f78cc070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1095498088 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc06e6a0 0x7f78cc0cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1095498088 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc143e60 0x7f78cc06e160 unknown :-1 s=CLOSED pgs=2795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 -- 172.21.15.165:0/1095498088 >> 172.21.15.165:0/1095498088 conn(0x7f78cc06abe0 msgr2=0x7f78cc06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 -- 172.21.15.165:0/1095498088 shutdown_connections 2024-04-25T10:20:56.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.945+0000 7f78d0a00640 1 -- 172.21.15.165:0/1095498088 wait complete. 2024-04-25T10:20:56.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.947+0000 7f78d0a00640 1 Processor -- start 2024-04-25T10:20:56.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.947+0000 7f78d0a00640 1 -- start start 2024-04-25T10:20:56.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78d0a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78cc06e6a0 0x7f78cc13fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78d0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 0x7f78cc13eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78d0a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc143e60 0x7f78cc0ac030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78d0a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0cce50 con 0x7f78cc0cc840 2024-04-25T10:20:56.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78d0a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0cccd0 con 0x7f78cc06e6a0 2024-04-25T10:20:56.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78d0a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78cc0b2d30 con 0x7f78cc143e60 2024-04-25T10:20:56.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.948+0000 7f78caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 0x7f78cc13eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 0x7f78cc13eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53924/0 (socket says 172.21.15.165:53924) 2024-04-25T10:20:56.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 -- 172.21.15.165:0/1381298396 learned_addr learned my addr 172.21.15.165:0/1381298396 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78cb400640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78cc06e6a0 0x7f78cc13fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78cbe00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc143e60 0x7f78cc0ac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 -- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc143e60 msgr2=0x7f78cc0ac030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc143e60 0x7f78cc0ac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 -- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78cc06e6a0 msgr2=0x7f78cc13fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78cc06e6a0 0x7f78cc13fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 -- 172.21.15.165:0/1381298396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78cc0a9320 con 0x7f78cc0cc840 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78caa00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 0x7f78cc13eca0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c5a0 tx=0x7f78b800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.949+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8015070 con 0x7f78cc0cc840 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.950+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78b8004220 con 0x7f78cc0cc840 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.950+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003610 con 0x7f78cc0cc840 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.950+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78cc0ae0c0 con 0x7f78cc0cc840 2024-04-25T10:20:56.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.950+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78cc0a9870 con 0x7f78cc0cc840 2024-04-25T10:20:56.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.951+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78cc0683e0 con 0x7f78cc0cc840 2024-04-25T10:20:56.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.952+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f78b80037b0 con 0x7f78cc0cc840 2024-04-25T10:20:56.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.953+0000 7f78c3400640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78ac06b4d0 0x7f78ac06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:56.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.953+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f78b808a1c0 con 0x7f78cc0cc840 2024-04-25T10:20:56.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.953+0000 7f78cb400640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78ac06b4d0 0x7f78ac06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:56.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.954+0000 7f78cb400640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78ac06b4d0 0x7f78ac06d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f78bc036000 tx=0x7f78bc00b000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:56.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:56.957+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f78cc0683e0 con 0x7f78cc0cc840 2024-04-25T10:20:57.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.192+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78cc0a9a40 con 0x7f78cc0cc840 2024-04-25T10:20:57.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.193+0000 7f78c3400640 1 -- 172.21.15.165:0/1381298396 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f78b8058de0 con 0x7f78cc0cc840 2024-04-25T10:20:57.197 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:20:57.197 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:54.991856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:20:57.197 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 146 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78ac06b4d0 msgr2=0x7f78ac06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78ac06b4d0 0x7f78ac06d990 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f78bc036000 tx=0x7f78bc00b000 comp rx=0 tx=0).stop 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 msgr2=0x7f78cc13eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 0x7f78cc13eca0 secure :-1 s=READY pgs=2796 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c5a0 tx=0x7f78b800ca60 comp rx=0 tx=0).stop 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 shutdown_connections 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f78ac06b4d0 0x7f78ac06d990 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f78cc143e60 0x7f78cc0ac030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f78cc0cc840 0x7f78cc13eca0 unknown :-1 s=CLOSED pgs=2796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 --2- 172.21.15.165:0/1381298396 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f78cc06e6a0 0x7f78cc13fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 >> 172.21.15.165:0/1381298396 conn(0x7f78cc06abe0 msgr2=0x7f78cc070ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 shutdown_connections 2024-04-25T10:20:57.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.196+0000 7f78d0a00640 1 -- 172.21.15.165:0/1381298396 wait complete. 2024-04-25T10:20:57.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 146, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:37.587736+0000', 'modified': '2024-04-25T10:20:54.991856+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11808': {'gid': 11808, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.165:6837/2156983703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 2156983703}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2156983703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:20:57.621 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:20:57.622 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:20:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.828+0000 7f8ddb000640 1 Processor -- start 2024-04-25T10:20:57.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.828+0000 7f8ddb000640 1 -- start start 2024-04-25T10:20:57.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8ddb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 0x7f8dd40cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8ddb000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd415e110 con 0x7f8dd406bff0 2024-04-25T10:20:57.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8ddb000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd40b2fd0 con 0x7f8dd40b5b40 2024-04-25T10:20:57.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8ddb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd406d000 con 0x7f8dd415dd30 2024-04-25T10:20:57.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd9200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8dd406bff0 0x7f8dd40b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60550/0 (socket says 172.21.15.165:60550) 2024-04-25T10:20:57.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd9200640 1 -- 172.21.15.165:0/4148544855 learned_addr learned my addr 172.21.15.165:0/4148544855 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 0x7f8dd40cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3008034471 0 0) 0x7f8dd415e110 con 0x7f8dd406bff0 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8dbc003520 con 0x7f8dd406bff0 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3916906010 0 0) 0x7f8dbc003520 con 0x7f8dd406bff0 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 msgr2=0x7f8dd40cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 0x7f8dd40cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 >> v1:172.21.15.172:6789/0 conn(0x7f8dd40b5b40 legacy=0x7f8dd40d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd406c480 con 0x7f8dd406bff0 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8dc40019f0 con 0x7f8dd406bff0 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 >> v1:172.21.15.165:6789/0 conn(0x7f8dd406bff0 legacy=0x7f8dd40b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dbc0039e0 0x7f8dbc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 0x7f8dbc004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dbc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd406d000 con 0x7f8dbc004d40 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc00bb50 con 0x7f8dbc0039e0 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.830+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc00bd20 con 0x7f8dd415dd30 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dda600640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dbc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dbc0039e0 0x7f8dbc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dd9200640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 0x7f8dbc004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dda600640 1 -- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 msgr2=0x7f8dbc004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dda600640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd415dd30 0x7f8dbc004800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dda600640 1 -- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dbc0039e0 msgr2=0x7f8dbc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dda600640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dbc0039e0 0x7f8dbc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dda600640 1 -- 172.21.15.165:0/4148544855 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc003520 con 0x7f8dbc004d40 2024-04-25T10:20:57.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.831+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dbc0039e0 0x7f8dbc003e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.832+0000 7f8dda600640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dbc0051c0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc001780 tx=0x7f8dbc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d54b4f873182b21 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.832+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc001370 con 0x7f8dbc004d40 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.832+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dcc0052d0 con 0x7f8dbc004d40 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.832+0000 7f8dd3e00640 1 -- 172.21.15.165:0/4148544855 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dcc02ac00 con 0x7f8dbc004d40 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8ddb000640 1 -- 172.21.15.165:0/4148544855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 msgr2=0x7f8dbc0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8ddb000640 1 --2- 172.21.15.165:0/4148544855 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dbc0051c0 secure :-1 s=READY pgs=2798 cs=0 l=1 rev1=1 crypto rx=0x7f8dcc001780 tx=0x7f8dbc005830 comp rx=0 tx=0).stop 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8dda600640 1 -- 172.21.15.165:0/4148544855 reap_dead start 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8ddb000640 1 -- 172.21.15.165:0/4148544855 shutdown_connections 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8ddb000640 1 -- 172.21.15.165:0/4148544855 >> 172.21.15.165:0/4148544855 conn(0x7f8dd40d0610 msgr2=0x7f8dd40d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8ddb000640 1 -- 172.21.15.165:0/4148544855 shutdown_connections 2024-04-25T10:20:57.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.833+0000 7f8ddb000640 1 -- 172.21.15.165:0/4148544855 wait complete. 2024-04-25T10:20:57.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.834+0000 7f8ddb000640 1 Processor -- start 2024-04-25T10:20:57.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 -- start start 2024-04-25T10:20:57.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dd40cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd4160760 0x7f8dd414e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4160bc0 0x7f8dd414ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd406cd20 con 0x7f8dbc004d40 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd406ca40 con 0x7f8dd4160bc0 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.835+0000 7f8ddb000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd40cebd0 con 0x7f8dd4160760 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dd40cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dd40cd0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53940/0 (socket says 172.21.15.165:53940) 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 -- 172.21.15.165:0/1151065062 learned_addr learned my addr 172.21.15.165:0/1151065062 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9200640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd4160760 0x7f8dd414e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 -- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd4160760 msgr2=0x7f8dd414e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd4160760 0x7f8dd414e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dda600640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4160bc0 0x7f8dd414ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 -- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4160bc0 msgr2=0x7f8dd414ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4160bc0 0x7f8dd414ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.836+0000 7f8dd9c00640 1 -- 172.21.15.165:0/1151065062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd40cd620 con 0x7f8dbc004d40 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.837+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dd40cd0e0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e7e0 tx=0x7f8dc800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.837+0000 7f8dd1600640 1 -- 172.21.15.165:0/1151065062 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8017070 con 0x7f8dbc004d40 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.837+0000 7f8dd1600640 1 -- 172.21.15.165:0/1151065062 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc800c6d0 con 0x7f8dbc004d40 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.837+0000 7f8dd1600640 1 -- 172.21.15.165:0/1151065062 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc800f040 con 0x7f8dbc004d40 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.837+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dd41f35f0 con 0x7f8dbc004d40 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.837+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd41f3b90 con 0x7f8dbc004d40 2024-04-25T10:20:57.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.839+0000 7f8dd1600640 1 -- 172.21.15.165:0/1151065062 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8dc801a070 con 0x7f8dbc004d40 2024-04-25T10:20:57.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.840+0000 7f8dd1600640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8db806b4d0 0x7f8db806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.840+0000 7f8dd1600640 1 -- 172.21.15.165:0/1151065062 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8dc8089fe0 con 0x7f8dbc004d40 2024-04-25T10:20:57.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.840+0000 7f8dd9200640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8db806b4d0 0x7f8db806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.841+0000 7f8ddf481740 1 Processor -- start 2024-04-25T10:20:57.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.841+0000 7f8dd9200640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8db806b4d0 0x7f8db806d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4001580 tx=0x7f8dc4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.841+0000 7f8ddf481740 1 -- start start 2024-04-25T10:20:57.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e544fb30 0x5604e5477970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e544ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e544e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8ddf481740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e53b3620 con 0x5604e5477eb0 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e535cca0 con 0x5604e544fb30 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604e53af6c0 con 0x5604e54987a0 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e544ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e544ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53954/0 (socket says 172.21.15.165:53954) 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.842+0000 7f8dd9200640 1 -- 172.21.15.165:0/2697765434 learned_addr learned my addr 172.21.15.165:0/2697765434 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dd9200640 1 -- 172.21.15.165:0/2697765434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 msgr2=0x5604e544e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dd9200640 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e544e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dd9200640 1 -- 172.21.15.165:0/2697765434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e544fb30 msgr2=0x5604e5477970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dd9200640 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e544fb30 0x5604e5477970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dd9200640 1 -- 172.21.15.165:0/2697765434 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e5363f40 con 0x5604e5477eb0 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dd9200640 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e544ff10 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4005e30 tx=0x7f8dc4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8750fb9f8e2db0f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dc2000640 1 -- 172.21.15.165:0/2697765434 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4005e60 con 0x5604e5477eb0 2024-04-25T10:20:57.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dc2000640 1 -- 172.21.15.165:0/2697765434 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4013890 con 0x5604e5477eb0 2024-04-25T10:20:57.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.843+0000 7f8dc2000640 1 -- 172.21.15.165:0/2697765434 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4013c80 con 0x5604e5477eb0 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 -- 172.21.15.165:0/2697765434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 msgr2=0x5604e544ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e544ff10 secure :-1 s=READY pgs=2800 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4005e30 tx=0x7f8dc4011000 comp rx=0 tx=0).stop 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 -- 172.21.15.165:0/2697765434 shutdown_connections 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e544e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e544ff10 unknown :-1 s=CLOSED pgs=2800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 --2- 172.21.15.165:0/2697765434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e544fb30 0x5604e5477970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 -- 172.21.15.165:0/2697765434 >> 172.21.15.165:0/2697765434 conn(0x5604e544cd50 msgr2=0x5604e5495730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 -- 172.21.15.165:0/2697765434 shutdown_connections 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.844+0000 7f8ddf481740 1 -- 172.21.15.165:0/2697765434 wait complete. 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.845+0000 7f8ddf481740 1 -- start start 2024-04-25T10:20:57.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.845+0000 7f8ddf481740 1 Processor -- start 2024-04-25T10:20:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e54a9a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e5497cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 0x5604e546bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e54774e0 con 0x5604e5477eb0 2024-04-25T10:20:57.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e53521f0 con 0x5604e546b690 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604e5476ed0 con 0x5604e54987a0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8ddf481740 10 client.0 _ll_register_callbacks cb 0x5604e536fad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e54a9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e54a9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53966/0 (socket says 172.21.15.165:53966) 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3596941468 learned_addr learned my addr 172.21.15.165:0/3596941468 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3596941468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 msgr2=0x5604e5497cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e5497cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3596941468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 msgr2=0x5604e546bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 0x5604e546bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.846+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3596941468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604e5371c50 con 0x5604e5477eb0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e54a9a60 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8054930 tx=0x7f8dc8058d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d14cba3ac393fe5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8db6a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8db6a00640 1 -- 172.21.15.165:0/3596941468 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8021070 con 0x5604e5477eb0 2024-04-25T10:20:57.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8db6a00640 1 -- 172.21.15.165:0/3596941468 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8dc805c370 con 0x5604e5477eb0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dc4013890 con 0x5604e5477eb0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8db6a00640 1 -- 172.21.15.165:0/3596941468 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dc8088050 con 0x5604e5477eb0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 10 client.11874 fetch_fsmap learned FSMap version 146 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5604e50c82d0 con 0x5604e5477eb0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8db6a00640 1 -- 172.21.15.165:0/3596941468 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4013890 con 0x5604e5477eb0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8db6a00640 1 -- 172.21.15.165:0/3596941468 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8dc8012050 con 0x5604e5477eb0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 10 client.11874 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 10 client.11874 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 20 client.11874 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e5473b40 0x5604e5473f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.847+0000 7f8ddf481740 4 client.11874 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.848+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604e53c0b60 con 0x5604e5473b40 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.848+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e5473b40 0x5604e5473f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.848+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e5473b40 0x5604e5473f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=447aba6598dd1eae server_cookie=eaa2b60f963651cb in_seq=0 out_seq=0 2024-04-25T10:20:57.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.848+0000 7f8db6a00640 10 client.11874 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:57.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8db6a00640 1 -- 172.21.15.165:0/3596941468 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5604e53c0b60 con 0x5604e5473b40 2024-04-25T10:20:57.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8db6a00640 10 client.11874 handle_command_reply: tid=0 2024-04-25T10:20:57.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 client.11874 shutdown 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 msgr2=0x5604e54a9a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e54a9a60 secure :-1 s=READY pgs=2801 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8054930 tx=0x7f8dc8058d60 comp rx=0 tx=0).stop 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 shutdown_connections 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e5473b40 0x5604e5473f20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 0x5604e546bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e5497cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8db6a00640 0 client.11874 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3596941468 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e54a9a60 unknown :-1 s=CLOSED pgs=2801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.849+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 >> 172.21.15.165:0/3596941468 conn(0x5604e544fb30 msgr2=0x5604e54498b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.850+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 shutdown_connections 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.850+0000 7f8ddf481740 1 -- 172.21.15.165:0/3596941468 wait complete. 2024-04-25T10:20:57.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.850+0000 7f8ddf481740 20 client.11874 trim_cache size 0 max 16384 2024-04-25T10:20:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.867+0000 7f8ddf481740 1 Processor -- start 2024-04-25T10:20:57.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.867+0000 7f8ddf481740 1 -- start start 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e547e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e54498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8ddf481740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41ffe40 con 0x5604e5477eb0 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41ff240 con 0x5604e546b690 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd41ff840 con 0x5604e54987a0 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.868+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e547e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e547e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53970/0 (socket says 172.21.15.165:53970) 2024-04-25T10:20:57.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 -- 172.21.15.165:0/3289243583 learned_addr learned my addr 172.21.15.165:0/3289243583 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 -- 172.21.15.165:0/3289243583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 msgr2=0x5604e54498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e54498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 -- 172.21.15.165:0/3289243583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 msgr2=0x5604e547e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 -- 172.21.15.165:0/3289243583 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5604e51926b0 con 0x5604e5477eb0 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e547e5b0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009010 tx=0x7f8dc4016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4209476d563cb6d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dc1600640 1 -- 172.21.15.165:0/3289243583 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4032070 con 0x5604e5477eb0 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dc1600640 1 -- 172.21.15.165:0/3289243583 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4004d60 con 0x5604e5477eb0 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8dc1600640 1 -- 172.21.15.165:0/3289243583 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc402c530 con 0x5604e5477eb0 2024-04-25T10:20:57.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 -- 172.21.15.165:0/3289243583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 msgr2=0x5604e547e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e547e5b0 secure :-1 s=READY pgs=2802 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4009010 tx=0x7f8dc4016040 comp rx=0 tx=0).stop 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 -- 172.21.15.165:0/3289243583 shutdown_connections 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54987a0 0x5604e54498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e5477eb0 0x5604e547e5b0 unknown :-1 s=CLOSED pgs=2802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3289243583 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 -- 172.21.15.165:0/3289243583 >> 172.21.15.165:0/3289243583 conn(0x5604e544fb30 msgr2=0x5604e5476d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 -- 172.21.15.165:0/3289243583 shutdown_connections 2024-04-25T10:20:57.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.870+0000 7f8ddf481740 1 -- 172.21.15.165:0/3289243583 wait complete. 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.871+0000 7f8ddf481740 1 -- start start 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.871+0000 7f8ddf481740 1 Processor -- start 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e54987a0 0x5604e547e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54fdf70 0x5604e55040e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e5474ad0 con 0x5604e546b690 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604e546f880 con 0x5604e54987a0 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5604e546cee0 con 0x5604e54fdf70 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8ddf481740 10 client.0 _ll_register_callbacks cb 0x7f8dc80547c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8dd9c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:53976/0 (socket says 172.21.15.165:53976) 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.872+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3637862450 learned_addr learned my addr 172.21.15.165:0/3637862450 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:57.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3637862450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54fdf70 msgr2=0x5604e55040e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54fdf70 0x5604e55040e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3637862450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e54987a0 msgr2=0x5604e547e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e54987a0 0x5604e547e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8dd9c00640 1 -- 172.21.15.165:0/3637862450 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5604e544efb0 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8dd9c00640 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 0x5604e547e070 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f8dc805c5e0 tx=0x7f8dc8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7370d9a49dffd5b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8db6a00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8db6a00640 1 -- 172.21.15.165:0/3637862450 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc8057e80 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8db6a00640 1 -- 172.21.15.165:0/3637862450 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f8dc8054270 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8dc4004d60 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8db6a00640 1 -- 172.21.15.165:0/3637862450 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8dc8054470 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.873+0000 7f8ddf481740 10 client.11880 fetch_fsmap learned FSMap version 146 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5604e54b8760 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8db6a00640 1 -- 172.21.15.165:0/3637862450 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4004d60 con 0x5604e546b690 2024-04-25T10:20:57.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8db6a00640 1 -- 172.21.15.165:0/3637862450 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f8dc8012040 con 0x5604e546b690 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 10 client.11880 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 10 client.11880 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 20 client.11880 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e546a680 0x5604e546aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 4 client.11880 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5604e53baeb0 con 0x5604e546a680 2024-04-25T10:20:57.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.874+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e546a680 0x5604e546aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:57.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8dd9200640 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e546a680 0x5604e546aa60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=363433896176fd25 server_cookie=b0902dd15122355c in_seq=0 out_seq=0 2024-04-25T10:20:57.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8db6a00640 10 client.11880 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:57.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8db6a00640 1 -- 172.21.15.165:0/3637862450 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5604e53baeb0 con 0x5604e546a680 2024-04-25T10:20:57.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8db6a00640 10 client.11880 handle_command_reply: tid=0 2024-04-25T10:20:57.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 client.11880 shutdown 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 msgr2=0x5604e547e070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 0x5604e547e070 secure :-1 s=READY pgs=2803 cs=0 l=1 rev1=1 crypto rx=0x7f8dc805c5e0 tx=0x7f8dc8000bc0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 shutdown_connections 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5604e546a680 0x5604e546aa60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5604e54fdf70 0x5604e55040e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5604e54987a0 0x5604e547e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 --2- 172.21.15.165:0/3637862450 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5604e546b690 0x5604e547e070 unknown :-1 s=CLOSED pgs=2803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8db6a00640 0 client.11880 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 >> 172.21.15.165:0/3637862450 conn(0x5604e544fb30 msgr2=0x5604e5495530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 shutdown_connections 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.875+0000 7f8ddf481740 1 -- 172.21.15.165:0/3637862450 wait complete. 2024-04-25T10:20:57.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddf481740 20 client.11880 trim_cache size 0 max 16384 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 11808, 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 4.9023162190000003, 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 146, 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 198, 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 198, 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 6.853979517 2024-04-25T10:20:57.883 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:57.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8db806b4d0 msgr2=0x7f8db806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8db806b4d0 0x7f8db806d990 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4001580 tx=0x7f8dc4009040 comp rx=0 tx=0).stop 2024-04-25T10:20:57.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 msgr2=0x7f8dd40cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:57.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dd40cd0e0 secure :-1 s=READY pgs=2799 cs=0 l=1 rev1=1 crypto rx=0x7f8dc800e7e0 tx=0x7f8dc800eca0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 shutdown_connections 2024-04-25T10:20:57.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8db806b4d0 0x7f8db806d990 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8dd4160bc0 0x7f8dd414ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8dd4160760 0x7f8dd414e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 --2- 172.21.15.165:0/1151065062 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8dbc004d40 0x7f8dd40cd0e0 unknown :-1 s=CLOSED pgs=2799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 >> 172.21.15.165:0/1151065062 conn(0x7f8dd40d0610 msgr2=0x7f8dd40b2540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 shutdown_connections 2024-04-25T10:20:57.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:57.876+0000 7f8ddb000640 1 -- 172.21.15.165:0/1151065062 wait complete. 2024-04-25T10:20:57.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:57.897 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:20:58.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.088+0000 7fc79fc00640 1 Processor -- start 2024-04-25T10:20:58.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.088+0000 7fc79fc00640 1 -- start start 2024-04-25T10:20:58.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 0x7fc79806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79fc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980cccc0 con 0x7fc7981446d0 2024-04-25T10:20:58.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79fc00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7980cc140 con 0x7fc798144d50 2024-04-25T10:20:58.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b2b20 con 0x7fc7981452a0 2024-04-25T10:20:58.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79d200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fc7981446d0 0x7fc7980b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60552/0 (socket says 172.21.15.165:60552) 2024-04-25T10:20:58.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79d200640 1 -- 172.21.15.165:0/2431382157 learned_addr learned my addr 172.21.15.165:0/2431382157 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc79dc00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 0x7fc79806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536703777 0 0) 0x7fc7980cccc0 con 0x7fc7981446d0 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc77c003520 con 0x7fc7981446d0 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2981157600 0 0) 0x7fc77c003520 con 0x7fc7981446d0 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 msgr2=0x7fc79806dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 0x7fc79806dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 >> v1:172.21.15.172:6789/0 conn(0x7fc798144d50 legacy=0x7fc79806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7980cc420 con 0x7fc7981446d0 2024-04-25T10:20:58.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fc788001670 con 0x7fc7981446d0 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 >> v1:172.21.15.165:6789/0 conn(0x7fc7981446d0 legacy=0x7fc7980b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc77c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77c004800 0x7fc77c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 0x7fc77c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980b2b20 con 0x7fc77c0039e0 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c00baa0 con 0x7fc77c004800 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.090+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c00bca0 con 0x7fc7981452a0 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc79d200640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc77c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc78fe00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77c004800 0x7fc77c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc79d200640 1 -- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 msgr2=0x7fc77c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc79d200640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7981452a0 0x7fc77c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc79d200640 1 -- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77c004800 msgr2=0x7fc77c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc79d200640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc77c004800 0x7fc77c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.091+0000 7fc79d200640 1 -- 172.21.15.165:0/2431382157 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c003520 con 0x7fc77c0039e0 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.093+0000 7fc79d200640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc77c003e00 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c210 tx=0x7fc77c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f283f25c1f2ced7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.093+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788007cf0 con 0x7fc77c0039e0 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.093+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788005af0 con 0x7fc77c0039e0 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.093+0000 7fc797e00640 1 -- 172.21.15.165:0/2431382157 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788007070 con 0x7fc77c0039e0 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79fc00640 1 -- 172.21.15.165:0/2431382157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 msgr2=0x7fc77c003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79fc00640 1 --2- 172.21.15.165:0/2431382157 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc77c003e00 secure :-1 s=READY pgs=2805 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c210 tx=0x7fc77c005830 comp rx=0 tx=0).stop 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79dc00640 1 -- 172.21.15.165:0/2431382157 reap_dead start 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79fc00640 1 -- 172.21.15.165:0/2431382157 shutdown_connections 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79fc00640 1 -- 172.21.15.165:0/2431382157 >> 172.21.15.165:0/2431382157 conn(0x7fc79806abe0 msgr2=0x7fc79806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79fc00640 1 -- 172.21.15.165:0/2431382157 shutdown_connections 2024-04-25T10:20:58.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.094+0000 7fc79fc00640 1 -- 172.21.15.165:0/2431382157 wait complete. 2024-04-25T10:20:58.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.097+0000 7fc79fc00640 1 Processor -- start 2024-04-25T10:20:58.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.097+0000 7fc79fc00640 1 -- start start 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79fc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc7981f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980add20 0x7fc7980b11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79fc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc798158470 0x7fc798157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79fc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980cc9e0 con 0x7fc77c0039e0 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79fc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc79806d400 con 0x7fc798158470 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc798144ab0 con 0x7fc7980add20 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc7981f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc7981f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54000/0 (socket says 172.21.15.165:54000) 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79d200640 1 -- 172.21.15.165:0/3978050028 learned_addr learned my addr 172.21.15.165:0/3978050028 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc78fe00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980add20 0x7fc7980b11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.098+0000 7fc79dc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc798158470 0x7fc798157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79d200640 1 -- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980add20 msgr2=0x7fc7980b11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79d200640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980add20 0x7fc7980b11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79d200640 1 -- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc798158470 msgr2=0x7fc798157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79d200640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc798158470 0x7fc798157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79d200640 1 -- 172.21.15.165:0/3978050028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7981f1e90 con 0x7fc77c0039e0 2024-04-25T10:20:58.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79d200640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc7981f1840 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c330 tx=0x7fc79806d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc795600640 1 -- 172.21.15.165:0/3978050028 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788040020 con 0x7fc77c0039e0 2024-04-25T10:20:58.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc795600640 1 -- 172.21.15.165:0/3978050028 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc788001430 con 0x7fc77c0039e0 2024-04-25T10:20:58.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7981f2170 con 0x7fc77c0039e0 2024-04-25T10:20:58.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc795600640 1 -- 172.21.15.165:0/3978050028 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788037040 con 0x7fc77c0039e0 2024-04-25T10:20:58.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.099+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7981f2730 con 0x7fc77c0039e0 2024-04-25T10:20:58.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.101+0000 7fc795600640 1 -- 172.21.15.165:0/3978050028 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc788043020 con 0x7fc77c0039e0 2024-04-25T10:20:58.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.102+0000 7fc795600640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc77406b4d0 0x7fc77406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.102+0000 7fc795600640 1 -- 172.21.15.165:0/3978050028 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc7880aa280 con 0x7fc77c0039e0 2024-04-25T10:20:58.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.102+0000 7fc78fe00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc77406b4d0 0x7fc77406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.102+0000 7fc7a2a03740 1 Processor -- start 2024-04-25T10:20:58.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 -- start start 2024-04-25T10:20:58.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc78fe00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc77406b4d0 0x7fc77406d990 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc7980af360 tx=0x7fc780009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da622990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6403a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 0x5566da63e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566da556980 con 0x5566da5f8290 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566da4ffde0 con 0x5566da6225b0 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566da552a20 con 0x5566da63fe50 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 0x5566da63e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 0x5566da63e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54012/0 (socket says 172.21.15.165:54012) 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 -- 172.21.15.165:0/3640690515 learned_addr learned my addr 172.21.15.165:0/3640690515 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79d200640 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da622990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc78fe00640 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6403a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 -- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 msgr2=0x5566da6403a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6403a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 -- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 msgr2=0x5566da622990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da622990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79dc00640 1 -- 172.21.15.165:0/3640690515 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566da51de30 con 0x5566da5f8290 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.104+0000 7fc79d200640 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da622990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:58.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.105+0000 7fc79dc00640 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 0x5566da63e510 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7fc7980cd290 tx=0x7fc79000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3292aac62bc96668 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.105+0000 7fc78d600640 1 -- 172.21.15.165:0/3640690515 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790015600 con 0x5566da5f8290 2024-04-25T10:20:58.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.105+0000 7fc78d600640 1 -- 172.21.15.165:0/3640690515 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79001a070 con 0x5566da5f8290 2024-04-25T10:20:58.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.105+0000 7fc78d600640 1 -- 172.21.15.165:0/3640690515 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc790015a00 con 0x5566da5f8290 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3640690515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 msgr2=0x5566da63e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 0x5566da63e510 secure :-1 s=READY pgs=2807 cs=0 l=1 rev1=1 crypto rx=0x7fc7980cd290 tx=0x7fc79000b040 comp rx=0 tx=0).stop 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3640690515 shutdown_connections 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da5f8290 0x5566da63e510 unknown :-1 s=CLOSED pgs=2807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6403a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3640690515 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da622990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3640690515 >> 172.21.15.165:0/3640690515 conn(0x5566da5f4bc0 msgr2=0x5566da63d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3640690515 shutdown_connections 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.106+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3640690515 wait complete. 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.107+0000 7fc7a2a03740 1 -- start start 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.107+0000 7fc7a2a03740 1 Processor -- start 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da6225b0 0x5566da61ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 0x5566da634c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da6351a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566da5f9030 con 0x5566da63fe50 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566da4f53c0 con 0x5566da6389c0 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566da5f2af0 con 0x5566da6225b0 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc7a2a03740 10 client.0 _ll_register_callbacks cb 0x5566da26ac60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 0x5566da634c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 0x5566da634c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54026/0 (socket says 172.21.15.165:54026) 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 -- 172.21.15.165:0/1141844093 learned_addr learned my addr 172.21.15.165:0/1141844093 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc79d200640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da6225b0 0x5566da61ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 -- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da6225b0 msgr2=0x5566da61ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc79dc00640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da6351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da6225b0 0x5566da61ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 -- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 msgr2=0x5566da6351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da6351a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.108+0000 7fc78fe00640 1 -- 172.21.15.165:0/1141844093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566da3337e0 con 0x5566da63fe50 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc78fe00640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 0x5566da634c60 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fc780003ba0 tx=0x7fc780001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9738ceaec831b23f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc786000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:58.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc786000640 1 -- 172.21.15.165:0/1141844093 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780019070 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc786000640 1 -- 172.21.15.165:0/1141844093 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc780010040 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc79001a070 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc786000640 1 -- 172.21.15.165:0/1141844093 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc78000fdc0 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc7a2a03740 10 client.11892 fetch_fsmap learned FSMap version 146 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.109+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5566da393f70 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc786000640 1 -- 172.21.15.165:0/1141844093 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc79001a070 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc786000640 1 -- 172.21.15.165:0/1141844093 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc78001f070 con 0x5566da63fe50 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc7a2a03740 10 client.11892 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc7a2a03740 10 client.11892 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc7a2a03740 20 client.11892 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da624960 0x5566da65b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc7a2a03740 4 client.11892 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566da563ec0 con 0x5566da624960 2024-04-25T10:20:58.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.111+0000 7fc79d200640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da624960 0x5566da65b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.112+0000 7fc79d200640 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da624960 0x5566da65b460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20df3ce3bcb0a947 server_cookie=a20a4ae41b4f054f in_seq=0 out_seq=0 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.112+0000 7fc786000640 10 client.11892 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.112+0000 7fc786000640 1 -- 172.21.15.165:0/1141844093 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5566da563ec0 con 0x5566da624960 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.112+0000 7fc786000640 10 client.11892 handle_command_reply: tid=0 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.112+0000 7fc7a2a03740 1 client.11892 shutdown 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 msgr2=0x5566da634c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 0x5566da634c60 secure :-1 s=READY pgs=2808 cs=0 l=1 rev1=1 crypto rx=0x7fc780003ba0 tx=0x7fc780001120 comp rx=0 tx=0).stop 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 shutdown_connections 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da624960 0x5566da65b460 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da6351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da63fe50 0x5566da634c60 unknown :-1 s=CLOSED pgs=2808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/1141844093 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da6225b0 0x5566da61ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc786000640 0 client.11892 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 >> 172.21.15.165:0/1141844093 conn(0x5566da5f8290 msgr2=0x5566da640940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 shutdown_connections 2024-04-25T10:20:58.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 1 -- 172.21.15.165:0/1141844093 wait complete. 2024-04-25T10:20:58.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.113+0000 7fc7a2a03740 20 client.11892 trim_cache size 0 max 16384 2024-04-25T10:20:58.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.129+0000 7fc7a2a03740 1 Processor -- start 2024-04-25T10:20:58.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.129+0000 7fc7a2a03740 1 -- start start 2024-04-25T10:20:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.129+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 0x5566da6259b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.129+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da63ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da63b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7981fcbe0 con 0x5566da6225b0 2024-04-25T10:20:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980cf330 con 0x5566da6389c0 2024-04-25T10:20:58.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980ce730 con 0x5566da63fe50 2024-04-25T10:20:58.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 0x5566da6259b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 0x5566da6259b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54034/0 (socket says 172.21.15.165:54034) 2024-04-25T10:20:58.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 -- 172.21.15.165:0/3273350728 learned_addr learned my addr 172.21.15.165:0/3273350728 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 -- 172.21.15.165:0/3273350728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 msgr2=0x5566da63b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da63b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 -- 172.21.15.165:0/3273350728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 msgr2=0x5566da63ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da63ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 -- 172.21.15.165:0/3273350728 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566da22b170 con 0x5566da6225b0 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc79d200640 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 0x5566da6259b0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fc78802adb0 tx=0x7fc7880a8040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7525c90232ca22a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc78cc00640 1 -- 172.21.15.165:0/3273350728 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788078710 con 0x5566da6225b0 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc78cc00640 1 -- 172.21.15.165:0/3273350728 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78807c860 con 0x5566da6225b0 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc78cc00640 1 -- 172.21.15.165:0/3273350728 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc788080e20 con 0x5566da6225b0 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3273350728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 msgr2=0x5566da6259b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 0x5566da6259b0 secure :-1 s=READY pgs=2809 cs=0 l=1 rev1=1 crypto rx=0x7fc78802adb0 tx=0x7fc7880a8040 comp rx=0 tx=0).stop 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3273350728 shutdown_connections 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da63b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6389c0 0x5566da63ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/3273350728 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6225b0 0x5566da6259b0 unknown :-1 s=CLOSED pgs=2809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3273350728 >> 172.21.15.165:0/3273350728 conn(0x5566da5f8290 msgr2=0x5566da63d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3273350728 shutdown_connections 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.131+0000 7fc7a2a03740 1 -- 172.21.15.165:0/3273350728 wait complete. 2024-04-25T10:20:58.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.133+0000 7fc7a2a03740 1 -- start start 2024-04-25T10:20:58.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.133+0000 7fc7a2a03740 1 Processor -- start 2024-04-25T10:20:58.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da63b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 0x5566da6a7a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6a7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566da641750 con 0x5566da6389c0 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7980ced30 con 0x5566da6225b0 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5566da63db60 con 0x5566da63fe50 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.134+0000 7fc7a2a03740 10 client.0 _ll_register_callbacks cb 0x5566da5f2fd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 0x5566da6a7a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 0x5566da6a7a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54044/0 (socket says 172.21.15.165:54044) 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 -- 172.21.15.165:0/854916285 learned_addr learned my addr 172.21.15.165:0/854916285 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 -- 172.21.15.165:0/854916285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 msgr2=0x5566da6a7f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6a7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 -- 172.21.15.165:0/854916285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 msgr2=0x5566da63b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da63b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 -- 172.21.15.165:0/854916285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566da63b7a0 con 0x5566da6389c0 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc78fe00640 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 0x5566da6a7a50 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fc78000f940 tx=0x7fc780005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3b102c7a59a3d63 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc786000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc786000640 1 -- 172.21.15.165:0/854916285 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc780039060 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc786000640 1 -- 172.21.15.165:0/854916285 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fc780034450 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc78807c860 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc786000640 1 -- 172.21.15.165:0/854916285 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc78002b4a0 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 10 client.11898 fetch_fsmap learned FSMap version 146 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5566da6ac930 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc786000640 1 -- 172.21.15.165:0/854916285 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc78807c860 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc786000640 1 -- 172.21.15.165:0/854916285 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fc780034670 con 0x5566da6389c0 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 10 client.11898 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 10 client.11898 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 20 client.11898 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da63ba50 0x5566da640630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 4 client.11898 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x5566da624000 con 0x5566da63ba50 2024-04-25T10:20:58.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.135+0000 7fc79d200640 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da63ba50 0x5566da640630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.137+0000 7fc79d200640 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da63ba50 0x5566da640630 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b2e32377f0d4813 server_cookie=4b559c5c15c39a77 in_seq=0 out_seq=0 2024-04-25T10:20:58.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.137+0000 7fc786000640 10 client.11898 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc786000640 1 -- 172.21.15.165:0/854916285 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5566da624000 con 0x5566da63ba50 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc786000640 10 client.11898 handle_command_reply: tid=0 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 client.11898 shutdown 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 msgr2=0x5566da6a7a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 0x5566da6a7a50 secure :-1 s=READY pgs=2810 cs=0 l=1 rev1=1 crypto rx=0x7fc78000f940 tx=0x7fc780005000 comp rx=0 tx=0).stop 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 shutdown_connections 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5566da63ba50 0x5566da640630 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc786000640 0 client.11898 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5566da63fe50 0x5566da6a7f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5566da6389c0 0x5566da6a7a50 unknown :-1 s=CLOSED pgs=2810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 --2- 172.21.15.165:0/854916285 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5566da6225b0 0x5566da63b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 >> 172.21.15.165:0/854916285 conn(0x5566da5f8290 msgr2=0x5566da61ed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 shutdown_connections 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 1 -- 172.21.15.165:0/854916285 wait complete. 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc7a2a03740 20 client.11898 trim_cache size 0 max 16384 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:20:58.144 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "a0688ba2-17c8-46ab-9648-4e77b8a5abfb", 2024-04-25T10:20:58.145 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:20:58.145 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:58.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc77406b4d0 msgr2=0x7fc77406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc77406b4d0 0x7fc77406d990 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc7980af360 tx=0x7fc780009040 comp rx=0 tx=0).stop 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 msgr2=0x7fc7981f1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc7981f1840 secure :-1 s=READY pgs=2806 cs=0 l=1 rev1=1 crypto rx=0x7fc78800c330 tx=0x7fc79806d370 comp rx=0 tx=0).stop 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 shutdown_connections 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc77406b4d0 0x7fc77406d990 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc798158470 0x7fc798157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7980add20 0x7fc7980b11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 --2- 172.21.15.165:0/3978050028 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc77c0039e0 0x7fc7981f1840 unknown :-1 s=CLOSED pgs=2806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 >> 172.21.15.165:0/3978050028 conn(0x7fc79806abe0 msgr2=0x7fc79806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 shutdown_connections 2024-04-25T10:20:58.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.138+0000 7fc79fc00640 1 -- 172.21.15.165:0/3978050028 wait complete. 2024-04-25T10:20:58.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:20:58.160 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:20:58.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.356+0000 7f0692600640 1 Processor -- start 2024-04-25T10:20:58.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.356+0000 7f0692600640 1 -- start start 2024-04-25T10:20:58.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0692600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0692600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f068c0c5030 0x7f068c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0692600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f068c145520 con 0x7f068c145140 2024-04-25T10:20:58.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0692600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f068c071270 con 0x7f068c0c5030 2024-04-25T10:20:58.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0692600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c071550 con 0x7f068c0ceb20 2024-04-25T10:20:58.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0691c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f068c0c5030 0x7f068c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0691c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f068c0c5030 0x7f068c15bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54046/0 (socket says 172.21.15.165:54046) 2024-04-25T10:20:58.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f0691c00640 1 -- 172.21.15.165:0/3819641788 learned_addr learned my addr 172.21.15.165:0/3819641788 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.358+0000 7f068be00640 1 --2- 172.21.15.165:0/3819641788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.359+0000 7f0691c00640 1 -- 172.21.15.165:0/3819641788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 msgr2=0x7f068c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.359+0000 7f0691c00640 1 --2- 172.21.15.165:0/3819641788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.359+0000 7f0691c00640 1 -- 172.21.15.165:0/3819641788 >> v1:172.21.15.172:6789/0 conn(0x7f068c145140 legacy=0x7f068c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.359+0000 7f0691c00640 1 -- 172.21.15.165:0/3819641788 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c071830 con 0x7f068c0c5030 2024-04-25T10:20:58.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.360+0000 7f0691c00640 1 --2- 172.21.15.165:0/3819641788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c0c5030 0x7f068c15bfb0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f0684009e50 tx=0x7f068402a630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a71b3a91bcda1b28 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.360+0000 7f068b400640 1 -- 172.21.15.165:0/3819641788 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684036070 con 0x7f068c0c5030 2024-04-25T10:20:58.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.360+0000 7f068b400640 1 -- 172.21.15.165:0/3819641788 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0684031600 con 0x7f068c0c5030 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.360+0000 7f068b400640 1 -- 172.21.15.165:0/3819641788 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06840319d0 con 0x7f068c0c5030 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 -- 172.21.15.165:0/3819641788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c0c5030 msgr2=0x7f068c15bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 --2- 172.21.15.165:0/3819641788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c0c5030 0x7f068c15bfb0 secure :-1 s=READY pgs=2811 cs=0 l=1 rev1=1 crypto rx=0x7f0684009e50 tx=0x7f068402a630 comp rx=0 tx=0).stop 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 -- 172.21.15.165:0/3819641788 shutdown_connections 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 --2- 172.21.15.165:0/3819641788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c0c5030 0x7f068c15bfb0 unknown :-1 s=CLOSED pgs=2811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 --2- 172.21.15.165:0/3819641788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 -- 172.21.15.165:0/3819641788 >> 172.21.15.165:0/3819641788 conn(0x7f068c061cc0 msgr2=0x7f068c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 -- 172.21.15.165:0/3819641788 shutdown_connections 2024-04-25T10:20:58.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.361+0000 7f0692600640 1 -- 172.21.15.165:0/3819641788 wait complete. 2024-04-25T10:20:58.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.363+0000 7f0692600640 1 Processor -- start 2024-04-25T10:20:58.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.363+0000 7f0692600640 1 -- start start 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0692600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f068c0c5030 0x7f068c0cdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0692600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0cae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0692600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 0x7f068c0cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0692600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0c5640 con 0x7f068c145140 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0692600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c0c54c0 con 0x7f068c0c5030 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0692600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c15c4f0 con 0x7f068c0ceb20 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0691c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 0x7f068c0cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0691c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 0x7f068c0cb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54058/0 (socket says 172.21.15.165:54058) 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0691c00640 1 -- 172.21.15.165:0/1817866868 learned_addr learned my addr 172.21.15.165:0/1817866868 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.364+0000 7f0691200640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f068c0c5030 0x7f068c0cdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0691c00640 1 -- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 msgr2=0x7f068c0cae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f068be00640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0cae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0691c00640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0cae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0691c00640 1 -- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f068c0c5030 msgr2=0x7f068c0cdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0691c00640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f068c0c5030 0x7f068c0cdb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0691c00640 1 -- 172.21.15.165:0/1817866868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c0b1b30 con 0x7f068c145140 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f068be00640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0691c00640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 0x7f068c0cb3d0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f0684009e50 tx=0x7f0684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0688c00640 1 -- 172.21.15.165:0/1817866868 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684036050 con 0x7f068c145140 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0688c00640 1 -- 172.21.15.165:0/1817866868 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0684042070 con 0x7f068c145140 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0688c00640 1 -- 172.21.15.165:0/1817866868 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0684009510 con 0x7f068c145140 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.365+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f068c1f3350 con 0x7f068c145140 2024-04-25T10:20:58.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.366+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f068c15cfc0 con 0x7f068c145140 2024-04-25T10:20:58.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.368+0000 7f0688c00640 1 -- 172.21.15.165:0/1817866868 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0684007bd0 con 0x7f068c145140 2024-04-25T10:20:58.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.369+0000 7f0688c00640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f066c06b4d0 0x7f066c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.369+0000 7f0688c00640 1 -- 172.21.15.165:0/1817866868 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f06840ab430 con 0x7f068c145140 2024-04-25T10:20:58.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.369+0000 7f0691200640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f066c06b4d0 0x7f066c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.370+0000 7f0691200640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f066c06b4d0 0x7f066c06d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0680006fd0 tx=0x7f0680008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.370+0000 7f06969cc740 1 Processor -- start 2024-04-25T10:20:58.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.370+0000 7f06969cc740 1 -- start start 2024-04-25T10:20:58.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e1278d70 0x55a7e127aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f06969cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 0x55a7e1279150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f06969cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e11dcc70 con 0x55a7e12b5440 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e1185cf0 con 0x55a7e12a3830 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e11d8d10 con 0x55a7e1278d70 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f068be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 0x55a7e1279150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f068be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 0x55a7e1279150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54060/0 (socket says 172.21.15.165:54060) 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.371+0000 7f068be00640 1 -- 172.21.15.165:0/821362835 learned_addr learned my addr 172.21.15.165:0/821362835 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f068be00640 1 -- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e1278d70 msgr2=0x55a7e127aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f068be00640 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e1278d70 0x55a7e127aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f0691c00640 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f068be00640 1 -- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 msgr2=0x55a7e12a3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f068be00640 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f068be00640 1 -- 172.21.15.165:0/821362835 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7e0fbb7d0 con 0x55a7e12b5440 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f068be00640 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 0x55a7e1279150 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f0678009a20 tx=0x7f0678009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8b68eba2fd8583 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f067d600640 1 -- 172.21.15.165:0/821362835 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678017030 con 0x55a7e12b5440 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f067d600640 1 -- 172.21.15.165:0/821362835 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678004020 con 0x55a7e12b5440 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.372+0000 7f067d600640 1 -- 172.21.15.165:0/821362835 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800f040 con 0x55a7e12b5440 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 -- 172.21.15.165:0/821362835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 msgr2=0x55a7e1279150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 0x55a7e1279150 secure :-1 s=READY pgs=2813 cs=0 l=1 rev1=1 crypto rx=0x7f0678009a20 tx=0x7f0678009ee0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 -- 172.21.15.165:0/821362835 shutdown_connections 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12b5440 0x55a7e1279150 unknown :-1 s=CLOSED pgs=2813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 --2- 172.21.15.165:0/821362835 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e1278d70 0x55a7e127aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 -- 172.21.15.165:0/821362835 >> 172.21.15.165:0/821362835 conn(0x55a7e1240a30 msgr2=0x55a7e12be830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 -- 172.21.15.165:0/821362835 shutdown_connections 2024-04-25T10:20:58.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.373+0000 7f06969cc740 1 -- 172.21.15.165:0/821362835 wait complete. 2024-04-25T10:20:58.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.374+0000 7f06969cc740 1 -- start start 2024-04-25T10:20:58.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.374+0000 7f06969cc740 1 Processor -- start 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e128c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a4d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 0x55a7e12a52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e117b3a0 con 0x55a7e1278d70 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e12a1380 con 0x55a7e12a3830 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e12a78b0 con 0x55a7e12b5440 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f06969cc740 10 client.0 _ll_register_callbacks cb 0x55a7e1198b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.375+0000 7f0691200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e128c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e128c060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54070/0 (socket says 172.21.15.165:54070) 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 -- 172.21.15.165:0/49709293 learned_addr learned my addr 172.21.15.165:0/49709293 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 -- 172.21.15.165:0/49709293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 msgr2=0x55a7e12a52c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 0x55a7e12a52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 -- 172.21.15.165:0/49709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 msgr2=0x55a7e12a4d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 -- 172.21.15.165:0/49709293 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7e0ef13f0 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0691200640 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e128c060 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f068000b610 tx=0x7f0680014800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f894806fd8aa417e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0672000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0672000640 1 -- 172.21.15.165:0/49709293 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680004020 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.376+0000 7f0672000640 1 -- 172.21.15.165:0/49709293 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f06800041c0 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0678004020 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f0672000640 1 -- 172.21.15.165:0/49709293 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06800043c0 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 10 client.11910 fetch_fsmap learned FSMap version 146 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55a7e0eb10f0 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f0672000640 1 -- 172.21.15.165:0/49709293 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678004020 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f0672000640 1 -- 172.21.15.165:0/49709293 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f068002c080 con 0x55a7e1278d70 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 10 client.11910 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 10 client.11910 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 20 client.11910 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:58.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12d8950 0x55a7e12dad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 4 client.11910 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.377+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a7e11ea1b0 con 0x55a7e12d8950 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.378+0000 7f068be00640 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12d8950 0x55a7e12dad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.378+0000 7f068be00640 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12d8950 0x55a7e12dad40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bdff1e90788835f server_cookie=e0bf030e35cac4b7 in_seq=0 out_seq=0 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.378+0000 7f0672000640 10 client.11910 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f0672000640 1 -- 172.21.15.165:0/49709293 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55a7e11ea1b0 con 0x55a7e12d8950 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f0672000640 10 client.11910 handle_command_reply: tid=0 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 client.11910 shutdown 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 msgr2=0x55a7e128c060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e128c060 secure :-1 s=READY pgs=2814 cs=0 l=1 rev1=1 crypto rx=0x7f068000b610 tx=0x7f0680014800 comp rx=0 tx=0).stop 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 shutdown_connections 2024-04-25T10:20:58.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12d8950 0x55a7e12dad40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 0x55a7e12a52c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12a3830 0x55a7e12a4d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 --2- 172.21.15.165:0/49709293 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e128c060 unknown :-1 s=CLOSED pgs=2814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f0672000640 0 client.11910 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 >> 172.21.15.165:0/49709293 conn(0x55a7e1240a30 msgr2=0x55a7e129e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 shutdown_connections 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.379+0000 7f06969cc740 1 -- 172.21.15.165:0/49709293 wait complete. 2024-04-25T10:20:58.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.380+0000 7f06969cc740 20 client.11910 trim_cache size 0 max 16384 2024-04-25T10:20:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.396+0000 7f06969cc740 1 Processor -- start 2024-04-25T10:20:58.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.396+0000 7f06969cc740 1 -- start start 2024-04-25T10:20:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e1278d70 0x55a7e1299d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f06969cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 0x55a7e12df770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 0x55a7e1273460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f06969cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c2005f0 con 0x55a7e12a3830 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c1ff770 con 0x55a7e1278d70 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f068c1feb70 con 0x55a7e12b5440 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 0x55a7e12df770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 0x55a7e12df770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54080/0 (socket says 172.21.15.165:54080) 2024-04-25T10:20:58.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 -- 172.21.15.165:0/3955969118 learned_addr learned my addr 172.21.15.165:0/3955969118 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 -- 172.21.15.165:0/3955969118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 msgr2=0x55a7e1273460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 0x55a7e1273460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 -- 172.21.15.165:0/3955969118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e1278d70 msgr2=0x55a7e1299d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e1278d70 0x55a7e1299d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.397+0000 7f068be00640 1 -- 172.21.15.165:0/3955969118 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a7e12c2270 con 0x55a7e12a3830 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.398+0000 7f068be00640 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 0x55a7e12df770 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f0678016040 tx=0x7f0678013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2323e0420421406c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.398+0000 7f067cc00640 1 -- 172.21.15.165:0/3955969118 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0678023d00 con 0x55a7e12a3830 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.398+0000 7f067cc00640 1 -- 172.21.15.165:0/3955969118 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678009300 con 0x55a7e12a3830 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.398+0000 7f067cc00640 1 -- 172.21.15.165:0/3955969118 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f067800ea70 con 0x55a7e12a3830 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 -- 172.21.15.165:0/3955969118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 msgr2=0x55a7e12df770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 0x55a7e12df770 secure :-1 s=READY pgs=2815 cs=0 l=1 rev1=1 crypto rx=0x7f0678016040 tx=0x7f0678013000 comp rx=0 tx=0).stop 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 -- 172.21.15.165:0/3955969118 shutdown_connections 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12b5440 0x55a7e1273460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e12a3830 0x55a7e12df770 unknown :-1 s=CLOSED pgs=2815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 --2- 172.21.15.165:0/3955969118 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e1278d70 0x55a7e1299d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 -- 172.21.15.165:0/3955969118 >> 172.21.15.165:0/3955969118 conn(0x55a7e1240a30 msgr2=0x55a7e12a6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 -- 172.21.15.165:0/3955969118 shutdown_connections 2024-04-25T10:20:58.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 -- 172.21.15.165:0/3955969118 wait complete. 2024-04-25T10:20:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.399+0000 7f06969cc740 1 -- start start 2024-04-25T10:20:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.400+0000 7f06969cc740 1 Processor -- start 2024-04-25T10:20:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e12b4b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12a3830 0x55a7e13301c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12b5440 0x55a7e12df080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a7e12a3e30 con 0x55a7e1278d70 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f068c1ff170 con 0x55a7e12b5440 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a7e12d49d0 con 0x55a7e12a3830 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f06969cc740 10 client.0 _ll_register_callbacks cb 0x7f0680006ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e12b4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e12b4b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:54090/0 (socket says 172.21.15.165:54090) 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 -- 172.21.15.165:0/1954867820 learned_addr learned my addr 172.21.15.165:0/1954867820 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691c00640 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12b5440 0x55a7e12df080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 -- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12a3830 msgr2=0x55a7e13301c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12a3830 0x55a7e13301c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 -- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12b5440 msgr2=0x55a7e12df080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12b5440 0x55a7e12df080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.401+0000 7f0691200640 1 -- 172.21.15.165:0/1954867820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a7e129e1b0 con 0x55a7e1278d70 2024-04-25T10:20:58.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0691200640 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e12b4b80 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f068000b1b0 tx=0x7f0680012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6b7a973acd0d4ae server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0672000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0672000640 1 -- 172.21.15.165:0/1954867820 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0680030070 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0672000640 1 -- 172.21.15.165:0/1954867820 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f068000d5b0 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0678009300 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0672000640 1 -- 172.21.15.165:0/1954867820 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f068003c050 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f06969cc740 10 client.11916 fetch_fsmap learned FSMap version 146 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55a7e12d47d0 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0672000640 1 -- 172.21.15.165:0/1954867820 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0678009300 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f0672000640 1 -- 172.21.15.165:0/1954867820 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f068003f020 con 0x55a7e1278d70 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f06969cc740 10 client.11916 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.402+0000 7f06969cc740 10 client.11916 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f06969cc740 20 client.11916 populate_metadata read hostname 'smithi165' 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f06969cc740 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12df610 0x55a7e12989c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f06969cc740 4 client.11916 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:20:58.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a7e11e4500 con 0x55a7e12df610 2024-04-25T10:20:58.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f068be00640 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12df610 0x55a7e12989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f068be00640 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12df610 0x55a7e12989c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8871b6c5a40278 server_cookie=4bcf86b11bed671 in_seq=0 out_seq=0 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.403+0000 7f0672000640 10 client.11916 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f0672000640 1 -- 172.21.15.165:0/1954867820 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55a7e11e4500 con 0x55a7e12df610 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f0672000640 10 client.11916 handle_command_reply: tid=0 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 client.11916 shutdown 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 msgr2=0x55a7e12b4b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e12b4b80 secure :-1 s=READY pgs=2816 cs=0 l=1 rev1=1 crypto rx=0x7f068000b1b0 tx=0x7f0680012000 comp rx=0 tx=0).stop 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 shutdown_connections 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55a7e12df610 0x55a7e12989c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55a7e12b5440 0x55a7e12df080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55a7e12a3830 0x55a7e13301c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 --2- 172.21.15.165:0/1954867820 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55a7e1278d70 0x55a7e12b4b80 unknown :-1 s=CLOSED pgs=2816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f0672000640 0 client.11916 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 >> 172.21.15.165:0/1954867820 conn(0x55a7e1240a30 msgr2=0x55a7e1299be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 shutdown_connections 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 1 -- 172.21.15.165:0/1954867820 wait complete. 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.404+0000 7f06969cc740 20 client.11916 trim_cache size 0 max 16384 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:20:58.410 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout: "a0688ba2-17c8-46ab-9648-4e77b8a5abfb": { 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/", 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "a0688ba2-17c8-46ab-9648-4e77b8a5abfb", 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive,repair" 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:20:58.411 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f066c06b4d0 msgr2=0x7f066c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f066c06b4d0 0x7f066c06d990 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f0680006fd0 tx=0x7f0680008040 comp rx=0 tx=0).stop 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 msgr2=0x7f068c0cb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 0x7f068c0cb3d0 secure :-1 s=READY pgs=2812 cs=0 l=1 rev1=1 crypto rx=0x7f0684009e50 tx=0x7f0684004a40 comp rx=0 tx=0).stop 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 shutdown_connections 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f066c06b4d0 0x7f066c06d990 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f068c145140 0x7f068c0cb3d0 unknown :-1 s=CLOSED pgs=2812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f068c0ceb20 0x7f068c0cae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 --2- 172.21.15.165:0/1817866868 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f068c0c5030 0x7f068c0cdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 >> 172.21.15.165:0/1817866868 conn(0x7f068c061cc0 msgr2=0x7f068c0ce410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 shutdown_connections 2024-04-25T10:20:58.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:20:58.405+0000 7f0692600640 1 -- 172.21.15.165:0/1817866868 wait complete. 2024-04-25T10:20:58.424 INFO:tasks.cephfs.filesystem:scrub status for tag:a0688ba2-17c8-46ab-9648-4e77b8a5abfb - {'path': '/', 'tag': 'a0688ba2-17c8-46ab-9648-4e77b8a5abfb', 'options': 'recursive,repair'} 2024-04-25T10:21:01.822 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:01.831 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:28.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:28.426 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:21:28.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.637+0000 7f706ae00640 1 Processor -- start 2024-04-25T10:21:28.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.637+0000 7f706ae00640 1 -- start start 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f706ae00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f70640cc4f0 0x7f70640cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f706ae00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f70640cdb20 0x7f70640cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f706ae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f706415b270 0x7f706415b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f706ae00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7064071270 con 0x7f70640cdb20 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f706ae00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7064072280 con 0x7f70640cc4f0 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f706ae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70640714c0 con 0x7f706415b270 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.639+0000 7f7063400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f70640cdb20 0x7f70640cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f70640cdb20 0x7f70640cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50470/0 (socket says 172.21.15.165:50470) 2024-04-25T10:21:28.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 -- 172.21.15.165:0/4198037539 learned_addr learned my addr 172.21.15.165:0/4198037539 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063e00640 1 --2- 172.21.15.165:0/4198037539 >> v2:172.21.15.172:3300/0 conn(0x7f70640cc4f0 0x7f70640cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 -- 172.21.15.165:0/4198037539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f706415b270 msgr2=0x7f706415b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 --2- 172.21.15.165:0/4198037539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f706415b270 0x7f706415b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 -- 172.21.15.165:0/4198037539 >> v2:172.21.15.172:3300/0 conn(0x7f70640cc4f0 msgr2=0x7f70640cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 --2- 172.21.15.165:0/4198037539 >> v2:172.21.15.172:3300/0 conn(0x7f70640cc4f0 0x7f70640cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.640+0000 7f7063400640 1 -- 172.21.15.165:0/4198037539 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640717a0 con 0x7f70640cdb20 2024-04-25T10:21:28.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.641+0000 7f7063400640 1 --2- 172.21.15.165:0/4198037539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cdb20 0x7f70640cdfb0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f70640cd290 tx=0x7f705002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc52a8c6bc456ce3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.641+0000 7f7062a00640 1 -- 172.21.15.165:0/4198037539 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050037070 con 0x7f70640cdb20 2024-04-25T10:21:28.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.641+0000 7f7062a00640 1 -- 172.21.15.165:0/4198037539 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050032440 con 0x7f70640cdb20 2024-04-25T10:21:28.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.641+0000 7f7062a00640 1 -- 172.21.15.165:0/4198037539 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050032810 con 0x7f70640cdb20 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 -- 172.21.15.165:0/4198037539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cdb20 msgr2=0x7f70640cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 --2- 172.21.15.165:0/4198037539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cdb20 0x7f70640cdfb0 secure :-1 s=READY pgs=2817 cs=0 l=1 rev1=1 crypto rx=0x7f70640cd290 tx=0x7f705002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 -- 172.21.15.165:0/4198037539 shutdown_connections 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 --2- 172.21.15.165:0/4198037539 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f706415b270 0x7f706415b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 --2- 172.21.15.165:0/4198037539 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cdb20 0x7f70640cdfb0 unknown :-1 s=CLOSED pgs=2817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 --2- 172.21.15.165:0/4198037539 >> v2:172.21.15.172:3300/0 conn(0x7f70640cc4f0 0x7f70640cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 -- 172.21.15.165:0/4198037539 >> 172.21.15.165:0/4198037539 conn(0x7f706406abe0 msgr2=0x7f706406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 -- 172.21.15.165:0/4198037539 shutdown_connections 2024-04-25T10:21:28.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.642+0000 7f706ae00640 1 -- 172.21.15.165:0/4198037539 wait complete. 2024-04-25T10:21:28.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.644+0000 7f706ae00640 1 Processor -- start 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.644+0000 7f706ae00640 1 -- start start 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f706ae00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 0x7f7064137fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f706ae00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70640cdb20 0x7f7064167310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f706ae00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f706415b270 0x7f7064167850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f706ae00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706415bc00 con 0x7f70640cc4f0 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f706ae00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064071d60 con 0x7f706415b270 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f706ae00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f706415bd80 con 0x7f70640cdb20 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f7063e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 0x7f7064137fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f7063e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 0x7f7064137fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50482/0 (socket says 172.21.15.165:50482) 2024-04-25T10:21:28.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.645+0000 7f7063e00640 1 -- 172.21.15.165:0/4195543352 learned_addr learned my addr 172.21.15.165:0/4195543352 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7068e00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f706415b270 0x7f7064167850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7063e00640 1 -- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70640cdb20 msgr2=0x7f7064167310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7063e00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70640cdb20 0x7f7064167310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7063e00640 1 -- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f706415b270 msgr2=0x7f7064167850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7063e00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f706415b270 0x7f7064167850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7063e00640 1 -- 172.21.15.165:0/4195543352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70640c96f0 con 0x7f70640cc4f0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f7063e00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 0x7f7064137fc0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f705400c7f0 tx=0x7f705400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f705be00640 1 -- 172.21.15.165:0/4195543352 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054015070 con 0x7f70640cc4f0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f705be00640 1 -- 172.21.15.165:0/4195543352 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054004220 con 0x7f70640cc4f0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f705be00640 1 -- 172.21.15.165:0/4195543352 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054007620 con 0x7f70640cc4f0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70640c99d0 con 0x7f70640cc4f0 2024-04-25T10:21:28.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.646+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70640c9f90 con 0x7f70640cc4f0 2024-04-25T10:21:28.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.648+0000 7f705be00640 1 -- 172.21.15.165:0/4195543352 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f70540079d0 con 0x7f70640cc4f0 2024-04-25T10:21:28.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.649+0000 7f705be00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f704406b500 0x7f704406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.650+0000 7f705be00640 1 -- 172.21.15.165:0/4195543352 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f705408a570 con 0x7f70640cc4f0 2024-04-25T10:21:28.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.650+0000 7f7063400640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f704406b500 0x7f704406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.650+0000 7f706dc04740 1 Processor -- start 2024-04-25T10:21:28.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 -- start start 2024-04-25T10:21:28.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f7063400640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f704406b500 0x7f704406d9c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7050034000 tx=0x7f705002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8adbb0 0x5634de8afd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de8adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8d8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634de811c70 con 0x5634de8d8670 2024-04-25T10:21:28.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634de7bacf0 con 0x5634de8ea280 2024-04-25T10:21:28.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.651+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634de80dd10 con 0x5634de8adbb0 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8d8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8d8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50486/0 (socket says 172.21.15.165:50486) 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 -- 172.21.15.165:0/282770058 learned_addr learned my addr 172.21.15.165:0/282770058 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7063400640 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de8adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7063e00640 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8adbb0 0x5634de8afd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 -- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8adbb0 msgr2=0x5634de8afd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8adbb0 0x5634de8afd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 -- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 msgr2=0x5634de8adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de8adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.652+0000 7f7068e00640 1 -- 172.21.15.165:0/282770058 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634de5f07d0 con 0x5634de8d8670 2024-04-25T10:21:28.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.653+0000 7f7068e00640 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8d8af0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f705c007e40 tx=0x7f705c00c580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f179ba3f17a47ba server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.653+0000 7f7058c00640 1 -- 172.21.15.165:0/282770058 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c015020 con 0x5634de8d8670 2024-04-25T10:21:28.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.653+0000 7f7058c00640 1 -- 172.21.15.165:0/282770058 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c011650 con 0x5634de8d8670 2024-04-25T10:21:28.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.653+0000 7f7058c00640 1 -- 172.21.15.165:0/282770058 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c00a070 con 0x5634de8d8670 2024-04-25T10:21:28.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 -- 172.21.15.165:0/282770058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 msgr2=0x5634de8d8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8d8af0 secure :-1 s=READY pgs=2819 cs=0 l=1 rev1=1 crypto rx=0x7f705c007e40 tx=0x7f705c00c580 comp rx=0 tx=0).stop 2024-04-25T10:21:28.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 -- 172.21.15.165:0/282770058 shutdown_connections 2024-04-25T10:21:28.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8d8af0 unknown :-1 s=CLOSED pgs=2819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de8adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 --2- 172.21.15.165:0/282770058 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8adbb0 0x5634de8afd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 -- 172.21.15.165:0/282770058 >> 172.21.15.165:0/282770058 conn(0x5634de875a30 msgr2=0x5634de8f3670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 -- 172.21.15.165:0/282770058 shutdown_connections 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.654+0000 7f706dc04740 1 -- 172.21.15.165:0/282770058 wait complete. 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.655+0000 7f706dc04740 1 -- start start 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.655+0000 7f706dc04740 1 Processor -- start 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8adbb0 0x5634de8ecfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8edb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8ee0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634de7b03a0 con 0x5634de8d8670 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634de8d61c0 con 0x5634de8adbb0 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634de8da110 con 0x5634de8ea280 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f706dc04740 10 client.0 _ll_register_callbacks cb 0x5634de7cdb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f7063400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8edb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f7063400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8edb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50488/0 (socket says 172.21.15.165:50488) 2024-04-25T10:21:28.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f7063400640 1 -- 172.21.15.165:0/2570394814 learned_addr learned my addr 172.21.15.165:0/2570394814 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.656+0000 7f7063e00640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8adbb0 0x5634de8ecfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7063400640 1 -- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 msgr2=0x5634de8ee0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7068e00640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8ee0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7063400640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8ee0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7063400640 1 -- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8adbb0 msgr2=0x5634de8ecfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7063400640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8adbb0 0x5634de8ecfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7063400640 1 -- 172.21.15.165:0/2570394814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634de5263f0 con 0x5634de8d8670 2024-04-25T10:21:28.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f7063400640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8edb80 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f7050035040 tx=0x7f705002c070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=564ee744d3288813 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f704d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:28.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f704d600640 1 -- 172.21.15.165:0/2570394814 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705004d060 con 0x5634de8d8670 2024-04-25T10:21:28.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f704d600640 1 -- 172.21.15.165:0/2570394814 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f705003daf0 con 0x5634de8d8670 2024-04-25T10:21:28.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.657+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f705c011650 con 0x5634de8d8670 2024-04-25T10:21:28.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f704d600640 1 -- 172.21.15.165:0/2570394814 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7050037040 con 0x5634de8d8670 2024-04-25T10:21:28.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 10 client.11928 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:28.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5634de4e60f0 con 0x5634de8d8670 2024-04-25T10:21:28.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f704d600640 1 -- 172.21.15.165:0/2570394814 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c011650 con 0x5634de8d8670 2024-04-25T10:21:28.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f704d600640 1 -- 172.21.15.165:0/2570394814 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7050056020 con 0x5634de8d8670 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 10 client.11928 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 10 client.11928 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 20 client.11928 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de8c39c0 0x5634de8c53c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 4 client.11928 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634de81f1b0 con 0x5634de8c39c0 2024-04-25T10:21:28.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.658+0000 7f7063e00640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de8c39c0 0x5634de8c53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.659+0000 7f7063e00640 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de8c39c0 0x5634de8c53c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32d957e56ace904c server_cookie=e383d3fa3997efc6 in_seq=0 out_seq=0 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.659+0000 7f704d600640 10 client.11928 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.659+0000 7f704d600640 1 -- 172.21.15.165:0/2570394814 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5634de81f1b0 con 0x5634de8c39c0 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.659+0000 7f704d600640 10 client.11928 handle_command_reply: tid=0 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 client.11928 shutdown 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 msgr2=0x5634de8edb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8edb80 secure :-1 s=READY pgs=2820 cs=0 l=1 rev1=1 crypto rx=0x7f7050035040 tx=0x7f705002c070 comp rx=0 tx=0).stop 2024-04-25T10:21:28.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 shutdown_connections 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de8c39c0 0x5634de8c53c0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8ee0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8d8670 0x5634de8edb80 unknown :-1 s=CLOSED pgs=2820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 --2- 172.21.15.165:0/2570394814 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8adbb0 0x5634de8ecfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f704d600640 0 client.11928 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 >> 172.21.15.165:0/2570394814 conn(0x5634de875a30 msgr2=0x5634de8d5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 shutdown_connections 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 1 -- 172.21.15.165:0/2570394814 wait complete. 2024-04-25T10:21:28.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.660+0000 7f706dc04740 20 client.11928 trim_cache size 0 max 16384 2024-04-25T10:21:28.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.677+0000 7f706dc04740 1 Processor -- start 2024-04-25T10:21:28.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.677+0000 7f706dc04740 1 -- start start 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.678+0000 7f706dc04740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8eee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.678+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8d8670 0x5634de90cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.678+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de90d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.678+0000 7f706dc04740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064200770 con 0x5634de8adbb0 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.678+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641ff910 con 0x5634de8ea280 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.678+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70641471d0 con 0x5634de8d8670 2024-04-25T10:21:28.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8eee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8eee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50490/0 (socket says 172.21.15.165:50490) 2024-04-25T10:21:28.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 -- 172.21.15.165:0/669317501 learned_addr learned my addr 172.21.15.165:0/669317501 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 -- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8d8670 msgr2=0x5634de90cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7068e00640 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de90d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8d8670 0x5634de90cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 -- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 msgr2=0x5634de90d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de90d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.679+0000 7f7063e00640 1 -- 172.21.15.165:0/669317501 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634de8a9e90 con 0x5634de8adbb0 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.680+0000 7f7063e00640 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8eee20 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f7054000c00 tx=0x7f7054088680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f3039653150f014 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.680+0000 7f704fe00640 1 -- 172.21.15.165:0/669317501 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705401e040 con 0x5634de8adbb0 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.680+0000 7f704fe00640 1 -- 172.21.15.165:0/669317501 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054056c70 con 0x5634de8adbb0 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.680+0000 7f704fe00640 1 -- 172.21.15.165:0/669317501 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054007cc0 con 0x5634de8adbb0 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 -- 172.21.15.165:0/669317501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 msgr2=0x5634de8eee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8eee20 secure :-1 s=READY pgs=2821 cs=0 l=1 rev1=1 crypto rx=0x7f7054000c00 tx=0x7f7054088680 comp rx=0 tx=0).stop 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 -- 172.21.15.165:0/669317501 shutdown_connections 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8ea280 0x5634de90d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8d8670 0x5634de90cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 --2- 172.21.15.165:0/669317501 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8eee20 unknown :-1 s=CLOSED pgs=2821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 -- 172.21.15.165:0/669317501 >> 172.21.15.165:0/669317501 conn(0x5634de875a30 msgr2=0x5634de8f3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 -- 172.21.15.165:0/669317501 shutdown_connections 2024-04-25T10:21:28.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.681+0000 7f706dc04740 1 -- 172.21.15.165:0/669317501 wait complete. 2024-04-25T10:21:28.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.682+0000 7f706dc04740 1 -- start start 2024-04-25T10:21:28.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.682+0000 7f706dc04740 1 Processor -- start 2024-04-25T10:21:28.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8d8670 0x5634de8edde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641ff310 con 0x5634de8adbb0 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634de8d53f0 con 0x5634de8d8670 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5634de8f44e0 con 0x5634de8ea280 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f706dc04740 10 client.0 _ll_register_callbacks cb 0x7f7050035070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50502/0 (socket says 172.21.15.165:50502) 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 -- 172.21.15.165:0/1036523434 learned_addr learned my addr 172.21.15.165:0/1036523434 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063400640 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8d8670 0x5634de8edde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 -- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 msgr2=0x5634de8f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 -- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8d8670 msgr2=0x5634de8edde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8d8670 0x5634de8edde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.683+0000 7f7063e00640 1 -- 172.21.15.165:0/1036523434 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634de8ee320 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f7063e00640 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8c2d50 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f7054058670 tx=0x7f705401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8311d27381da41e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f704d600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f704d600640 1 -- 172.21.15.165:0/1036523434 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7054076050 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f704d600640 1 -- 172.21.15.165:0/1036523434 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f7054054fe0 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7054056c70 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f704d600640 1 -- 172.21.15.165:0/1036523434 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f705401e070 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f706dc04740 10 client.11934 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5634de902180 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.684+0000 7f704d600640 1 -- 172.21.15.165:0/1036523434 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7054056c70 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f704d600640 1 -- 172.21.15.165:0/1036523434 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f7054086020 con 0x5634de8adbb0 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f706dc04740 10 client.11934 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f706dc04740 10 client.11934 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:28.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f706dc04740 20 client.11934 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:28.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f706dc04740 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de901c50 0x5634de8f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f706dc04740 4 client.11934 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:28.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5634de819500 con 0x5634de901c50 2024-04-25T10:21:28.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f7063400640 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de901c50 0x5634de8f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f7063400640 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de901c50 0x5634de8f72b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e13dc5f8b82d1c82 server_cookie=b4bc36430e2ebef2 in_seq=0 out_seq=0 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.685+0000 7f704d600640 10 client.11934 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f704d600640 1 -- 172.21.15.165:0/1036523434 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5634de819500 con 0x5634de901c50 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f704d600640 10 client.11934 handle_command_reply: tid=0 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 client.11934 shutdown 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 msgr2=0x5634de8c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8c2d50 secure :-1 s=READY pgs=2822 cs=0 l=1 rev1=1 crypto rx=0x7f7054058670 tx=0x7f705401b040 comp rx=0 tx=0).stop 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 shutdown_connections 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5634de901c50 0x5634de8f72b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5634de8ea280 0x5634de8f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5634de8d8670 0x5634de8edde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f704d600640 0 client.11934 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 --2- 172.21.15.165:0/1036523434 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5634de8adbb0 0x5634de8c2d50 unknown :-1 s=CLOSED pgs=2822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 >> 172.21.15.165:0/1036523434 conn(0x5634de875a30 msgr2=0x5634de8ee610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 shutdown_connections 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 1 -- 172.21.15.165:0/1036523434 wait complete. 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.686+0000 7f706dc04740 20 client.11934 trim_cache size 0 max 16384 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:21:28.693 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:21:28.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f704406b500 msgr2=0x7f704406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f704406b500 0x7f704406d9c0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f7050034000 tx=0x7f705002c000 comp rx=0 tx=0).stop 2024-04-25T10:21:28.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 msgr2=0x7f7064137fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 0x7f7064137fc0 secure :-1 s=READY pgs=2818 cs=0 l=1 rev1=1 crypto rx=0x7f705400c7f0 tx=0x7f705400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 shutdown_connections 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f704406b500 0x7f704406d9c0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f706415b270 0x7f7064167850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f70640cdb20 0x7f7064167310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 --2- 172.21.15.165:0/4195543352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f70640cc4f0 0x7f7064137fc0 unknown :-1 s=CLOSED pgs=2818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 >> 172.21.15.165:0/4195543352 conn(0x7f706406abe0 msgr2=0x7f70640b1ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 shutdown_connections 2024-04-25T10:21:28.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.687+0000 7f706ae00640 1 -- 172.21.15.165:0/4195543352 wait complete. 2024-04-25T10:21:28.706 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:21:28.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:28.707 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub start / recursive,repair 2024-04-25T10:21:28.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.898+0000 7f0781e00640 1 Processor -- start 2024-04-25T10:21:28.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.898+0000 7f0781e00640 1 -- start start 2024-04-25T10:21:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.899+0000 7f0781e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f077c0ceb20 0x7f077c0c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.899+0000 7f0781e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0c5030 0x7f077c15bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.899+0000 7f0781e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077c145520 con 0x7f077c145140 2024-04-25T10:21:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.899+0000 7f0781e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f077c071270 con 0x7f077c0ceb20 2024-04-25T10:21:28.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.899+0000 7f0781e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c071550 con 0x7f077c0c5030 2024-04-25T10:21:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.900+0000 7f0773e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f077c0ceb20 0x7f077c0c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.900+0000 7f0781400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0c5030 0x7f077c15bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.900+0000 7f0780a00640 1 --1- >> v1:172.21.15.172:6789/0 conn(0x7f077c145140 0x7f077c0ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.172:6789/0 says I am v1:172.21.15.165:58092/0 (socket says 172.21.15.165:58092) 2024-04-25T10:21:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.900+0000 7f0780a00640 1 -- 172.21.15.165:0/3167773599 learned_addr learned my addr 172.21.15.165:0/3167773599 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:21:28.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.900+0000 7f0773e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f077c0ceb20 0x7f077c0c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:41402/0 (socket says 172.21.15.165:41402) 2024-04-25T10:21:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.901+0000 7f0773e00640 1 -- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0c5030 msgr2=0x7f077c15bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.901+0000 7f0773e00640 1 --2- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0c5030 0x7f077c15bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.901+0000 7f0773e00640 1 -- 172.21.15.165:0/3167773599 >> v1:172.21.15.172:6789/0 conn(0x7f077c145140 legacy=0x7f077c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.901+0000 7f0773e00640 1 -- 172.21.15.165:0/3167773599 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c071830 con 0x7f077c0ceb20 2024-04-25T10:21:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.901+0000 7f0781400640 1 --2- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0c5030 0x7f077c15bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:21:28.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.902+0000 7f0773e00640 1 --2- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0ceb20 0x7f077c0c4af0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f07640097a0 tx=0x7f076402a100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dbb4c6a103e93f64 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f077be00640 1 -- 172.21.15.165:0/3167773599 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764037070 con 0x7f077c0ceb20 2024-04-25T10:21:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f077be00640 1 -- 172.21.15.165:0/3167773599 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076402ae50 con 0x7f077c0ceb20 2024-04-25T10:21:28.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f077be00640 1 -- 172.21.15.165:0/3167773599 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764032660 con 0x7f077c0ceb20 2024-04-25T10:21:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f0781e00640 1 -- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0ceb20 msgr2=0x7f077c0c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f0781e00640 1 --2- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0ceb20 0x7f077c0c4af0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f07640097a0 tx=0x7f076402a100 comp rx=0 tx=0).stop 2024-04-25T10:21:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f0781e00640 1 -- 172.21.15.165:0/3167773599 shutdown_connections 2024-04-25T10:21:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f0781e00640 1 --2- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0c5030 0x7f077c15bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f0781e00640 1 --2- 172.21.15.165:0/3167773599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0ceb20 0x7f077c0c4af0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.903+0000 7f0781e00640 1 -- 172.21.15.165:0/3167773599 >> 172.21.15.165:0/3167773599 conn(0x7f077c061cc0 msgr2=0x7f077c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.904+0000 7f0781e00640 1 -- 172.21.15.165:0/3167773599 shutdown_connections 2024-04-25T10:21:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.904+0000 7f0781e00640 1 -- 172.21.15.165:0/3167773599 wait complete. 2024-04-25T10:21:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.905+0000 7f0781e00640 1 Processor -- start 2024-04-25T10:21:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.905+0000 7f0781e00640 1 -- start start 2024-04-25T10:21:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.906+0000 7f0781e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0c5030 0x7f077c1e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.906+0000 7f0781e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0ceb20 0x7f077c131fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.906+0000 7f0781e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 0x7f077c1e4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.906+0000 7f0781e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0c5620 con 0x7f077c145140 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.906+0000 7f0781e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0c54a0 con 0x7f077c0c5030 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.906+0000 7f0781e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c15c4f0 con 0x7f077c0ceb20 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 0x7f077c1e4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 0x7f077c1e4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50516/0 (socket says 172.21.15.165:50516) 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 -- 172.21.15.165:0/1388346074 learned_addr learned my addr 172.21.15.165:0/1388346074 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0773e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0ceb20 0x7f077c131fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0780a00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0c5030 0x7f077c1e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 -- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0ceb20 msgr2=0x7f077c131fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0ceb20 0x7f077c131fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 -- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0c5030 msgr2=0x7f077c1e3f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0c5030 0x7f077c1e3f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 -- 172.21.15.165:0/1388346074 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f077c1f2410 con 0x7f077c145140 2024-04-25T10:21:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0780a00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0c5030 0x7f077c1e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.907+0000 7f0781400640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 0x7f077c1e4a80 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f0774009f00 tx=0x7f077400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.908+0000 7f0779600640 1 -- 172.21.15.165:0/1388346074 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774018030 con 0x7f077c145140 2024-04-25T10:21:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.908+0000 7f0779600640 1 -- 172.21.15.165:0/1388346074 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f077400ce50 con 0x7f077c145140 2024-04-25T10:21:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.908+0000 7f0779600640 1 -- 172.21.15.165:0/1388346074 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774003bb0 con 0x7f077c145140 2024-04-25T10:21:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.908+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077c1f26f0 con 0x7f077c145140 2024-04-25T10:21:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.908+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f077c1f2cb0 con 0x7f077c145140 2024-04-25T10:21:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.910+0000 7f0779600640 1 -- 172.21.15.165:0/1388346074 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f077401b070 con 0x7f077c145140 2024-04-25T10:21:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.912+0000 7f0779600640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f075806b4d0 0x7f075806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.912+0000 7f0779600640 1 -- 172.21.15.165:0/1388346074 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f077408af70 con 0x7f077c145140 2024-04-25T10:21:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.912+0000 7f0780a00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f075806b4d0 0x7f075806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.912+0000 7f07861f5740 1 Processor -- start 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 -- start start 2024-04-25T10:21:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f0780a00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f075806b4d0 0x7f075806d990 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f076c000f60 tx=0x7f076c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 0x557aa3f9b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f71050 0x557aa3fb72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa3ed3980 con 0x557aa3fb8c10 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa3e7cde0 con 0x557aa3f9b370 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.913+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aa3ecfa20 con 0x557aa3f71050 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50518/0 (socket says 172.21.15.165:50518) 2024-04-25T10:21:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 -- 172.21.15.165:0/1480718003 learned_addr learned my addr 172.21.15.165:0/1480718003 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 -- 172.21.15.165:0/1480718003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f71050 msgr2=0x557aa3fb72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f71050 0x557aa3fb72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 -- 172.21.15.165:0/1480718003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 msgr2=0x557aa3f9b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 0x557aa3f9b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 -- 172.21.15.165:0/1480718003 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa3e9ae30 con 0x557aa3fb8c10 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.914+0000 7f0773e00640 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb9160 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f0764035040 tx=0x7f076402c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9663dd209307f2b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.915+0000 7f0771200640 1 -- 172.21.15.165:0/1480718003 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764037040 con 0x557aa3fb8c10 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.915+0000 7f0771200640 1 -- 172.21.15.165:0/1480718003 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076403e720 con 0x557aa3fb8c10 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.915+0000 7f0771200640 1 -- 172.21.15.165:0/1480718003 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076403eaf0 con 0x557aa3fb8c10 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.915+0000 7f07861f5740 1 -- 172.21.15.165:0/1480718003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 msgr2=0x557aa3fb9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.915+0000 7f07861f5740 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb9160 secure :-1 s=READY pgs=2824 cs=0 l=1 rev1=1 crypto rx=0x7f0764035040 tx=0x7f076402c000 comp rx=0 tx=0).stop 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 -- 172.21.15.165:0/1480718003 shutdown_connections 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f71050 0x557aa3fb72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb9160 unknown :-1 s=CLOSED pgs=2824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 --2- 172.21.15.165:0/1480718003 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 0x557aa3f9b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 -- 172.21.15.165:0/1480718003 >> 172.21.15.165:0/1480718003 conn(0x557aa3f6d980 msgr2=0x557aa3fb6180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 -- 172.21.15.165:0/1480718003 shutdown_connections 2024-04-25T10:21:28.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 -- 172.21.15.165:0/1480718003 wait complete. 2024-04-25T10:21:28.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.916+0000 7f07861f5740 1 -- start start 2024-04-25T10:21:28.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.917+0000 7f07861f5740 1 Processor -- start 2024-04-25T10:21:28.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f9b370 0x557aa3f9dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3f6a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f87980 0x557aa3f87e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa3f71df0 con 0x557aa3fb8c10 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa3e723c0 con 0x557aa3f87980 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aa3f6b8b0 con 0x557aa3f9b370 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f07861f5740 10 client.0 _ll_register_callbacks cb 0x557aa3be7c60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3f6a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3f6a9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50522/0 (socket says 172.21.15.165:50522) 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 -- 172.21.15.165:0/3060470514 learned_addr learned my addr 172.21.15.165:0/3060470514 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 -- 172.21.15.165:0/3060470514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f9b370 msgr2=0x557aa3f9dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f9b370 0x557aa3f9dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 -- 172.21.15.165:0/3060470514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f87980 msgr2=0x557aa3f87e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f87980 0x557aa3f87e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.918+0000 7f0773e00640 1 -- 172.21.15.165:0/3060470514 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557aa3cb07e0 con 0x557aa3fb8c10 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0773e00640 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3f6a9a0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f0764004440 tx=0x7f0764004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1fa069ee3c6748 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0769600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:28.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0769600640 1 -- 172.21.15.165:0/3060470514 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764037070 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0769600640 1 -- 172.21.15.165:0/3060470514 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f0764047cf0 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f076403e720 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0769600640 1 -- 172.21.15.165:0/3060470514 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f076404e050 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f07861f5740 10 client.11943 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x557aa3d10f70 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0769600640 1 -- 172.21.15.165:0/3060470514 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076403e720 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f0769600640 1 -- 172.21.15.165:0/3060470514 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f0764051020 con 0x557aa3fb8c10 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f07861f5740 10 client.11943 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.919+0000 7f07861f5740 10 client.11943 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f07861f5740 20 client.11943 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f07861f5740 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa3fca390 0x557aa3fca770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f07861f5740 4 client.11943 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557aa3ee0ec0 con 0x557aa3fca390 2024-04-25T10:21:28.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f0780a00640 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa3fca390 0x557aa3fca770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f0780a00640 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa3fca390 0x557aa3fca770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e06106fc82a70d3 server_cookie=dd5338030fbb6f07 in_seq=0 out_seq=0 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.920+0000 7f0769600640 10 client.11943 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f0769600640 1 -- 172.21.15.165:0/3060470514 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x557aa3ee0ec0 con 0x557aa3fca390 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f0769600640 10 client.11943 handle_command_reply: tid=0 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 client.11943 shutdown 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 msgr2=0x557aa3f6a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3f6a9a0 secure :-1 s=READY pgs=2825 cs=0 l=1 rev1=1 crypto rx=0x7f0764004440 tx=0x7f0764004730 comp rx=0 tx=0).stop 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 shutdown_connections 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa3fca390 0x557aa3fca770 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f87980 0x557aa3f87e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f0769600640 0 client.11943 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3f6a9a0 unknown :-1 s=CLOSED pgs=2825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 --2- 172.21.15.165:0/3060470514 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f9b370 0x557aa3f9dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.921+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 >> 172.21.15.165:0/3060470514 conn(0x557aa3f71050 msgr2=0x557aa3fb9700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.922+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 shutdown_connections 2024-04-25T10:21:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.922+0000 7f07861f5740 1 -- 172.21.15.165:0/3060470514 wait complete. 2024-04-25T10:21:28.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.922+0000 7f07861f5740 20 client.11943 trim_cache size 0 max 16384 2024-04-25T10:21:28.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.938+0000 7f07861f5740 1 Processor -- start 2024-04-25T10:21:28.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.938+0000 7f07861f5740 1 -- start start 2024-04-25T10:21:28.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 0x557aa3fb5150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f07861f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f07861f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1fc180 con 0x557aa3fb8c10 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0cdb30 con 0x557aa3f9b370 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077c1f4480 con 0x557aa3f87980 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f0781400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f0781400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50524/0 (socket says 172.21.15.165:50524) 2024-04-25T10:21:28.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.939+0000 7f0781400640 1 -- 172.21.15.165:0/3671518001 learned_addr learned my addr 172.21.15.165:0/3671518001 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f0781400640 1 -- 172.21.15.165:0/3671518001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 msgr2=0x557aa3fb6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f0781400640 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f0781400640 1 -- 172.21.15.165:0/3671518001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 msgr2=0x557aa3fb5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f0781400640 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 0x557aa3fb5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f0781400640 1 -- 172.21.15.165:0/3671518001 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557aa3f9bac0 con 0x557aa3fb8c10 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f0781400640 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb5690 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f0774059390 tx=0x7f0774059300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92f995583395e4a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f076be00640 1 -- 172.21.15.165:0/3671518001 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f077405d550 con 0x557aa3fb8c10 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f076be00640 1 -- 172.21.15.165:0/3671518001 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774061b10 con 0x557aa3fb8c10 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.940+0000 7f076be00640 1 -- 172.21.15.165:0/3671518001 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0774055a70 con 0x557aa3fb8c10 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 -- 172.21.15.165:0/3671518001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 msgr2=0x557aa3fb5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb5690 secure :-1 s=READY pgs=2826 cs=0 l=1 rev1=1 crypto rx=0x7f0774059390 tx=0x7f0774059300 comp rx=0 tx=0).stop 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 -- 172.21.15.165:0/3671518001 shutdown_connections 2024-04-25T10:21:28.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3fb8c10 0x557aa3fb5690 unknown :-1 s=CLOSED pgs=2826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3f9b370 0x557aa3fb5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 --2- 172.21.15.165:0/3671518001 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 -- 172.21.15.165:0/3671518001 >> 172.21.15.165:0/3671518001 conn(0x557aa3f71050 msgr2=0x557aa3f66920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 -- 172.21.15.165:0/3671518001 shutdown_connections 2024-04-25T10:21:28.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.941+0000 7f07861f5740 1 -- 172.21.15.165:0/3671518001 wait complete. 2024-04-25T10:21:28.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.942+0000 7f07861f5740 1 -- start start 2024-04-25T10:21:28.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.942+0000 7f07861f5740 1 Processor -- start 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 0x557aa3fac030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3fb8c10 0x557aa3fac570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077c0cd530 con 0x557aa3f9b370 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x557aa3f8df50 con 0x557aa3fb8c10 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x557aa3fb6b30 con 0x557aa3f87980 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f07861f5740 10 client.0 _ll_register_callbacks cb 0x557aa3f6bd90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f0773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 0x557aa3fac030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f0773e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 0x557aa3fac030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50532/0 (socket says 172.21.15.165:50532) 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f0773e00640 1 -- 172.21.15.165:0/49833351 learned_addr learned my addr 172.21.15.165:0/49833351 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f0780a00640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.943+0000 7f0781400640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3fb8c10 0x557aa3fac570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0773e00640 1 -- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 msgr2=0x557aa3fb6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0773e00640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0773e00640 1 -- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3fb8c10 msgr2=0x557aa3fac570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0773e00640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3fb8c10 0x557aa3fac570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0773e00640 1 -- 172.21.15.165:0/49833351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557aa3ba8170 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0773e00640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 0x557aa3fac030 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f076402a6f0 tx=0x7f07640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8603471a6a5878d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0769600640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0769600640 1 -- 172.21.15.165:0/49833351 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764037040 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f0769600640 1 -- 172.21.15.165:0/49833351 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f0764003a20 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.944+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0774061b10 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f0769600640 1 -- 172.21.15.165:0/49833351 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f076405c050 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 10 client.11949 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x557aa3fb7b40 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f0769600640 1 -- 172.21.15.165:0/49833351 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0774061b10 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f0769600640 1 -- 172.21.15.165:0/49833351 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f076405f020 con 0x557aa3f9b370 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 10 client.11949 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:28.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 10 client.11949 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:28.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 20 client.11949 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:28.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa402c920 0x557aa3f96960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:28.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 4 client.11949 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}] 2024-04-25T10:21:28.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive", "repair"]}) v1 -- 0x557aa3f94260 con 0x557aa402c920 2024-04-25T10:21:28.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.945+0000 7f0780a00640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa402c920 0x557aa3f96960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:28.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.946+0000 7f0780a00640 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa402c920 0x557aa3f96960 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd4aa6e10afcc31b server_cookie=ea7fb70e239ad6b8 in_seq=0 out_seq=0 2024-04-25T10:21:28.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.946+0000 7f0769600640 10 client.11949 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.946+0000 7f0769600640 1 -- 172.21.15.165:0/49833351 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557aa3f94260 con 0x557aa402c920 2024-04-25T10:21:28.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.946+0000 7f0769600640 10 client.11949 handle_command_reply: tid=0 2024-04-25T10:21:28.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.946+0000 7f07861f5740 1 client.11949 shutdown 2024-04-25T10:21:28.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 msgr2=0x557aa3fac030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 0x557aa3fac030 secure :-1 s=READY pgs=2827 cs=0 l=1 rev1=1 crypto rx=0x7f076402a6f0 tx=0x7f07640046c0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 shutdown_connections 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x557aa402c920 0x557aa3f96960 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x557aa3fb8c10 0x557aa3fac570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x557aa3f9b370 0x557aa3fac030 unknown :-1 s=CLOSED pgs=2827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 --2- 172.21.15.165:0/49833351 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x557aa3f87980 0x557aa3fb6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f0769600640 0 client.11949 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 >> 172.21.15.165:0/49833351 conn(0x557aa3f71050 msgr2=0x557aa3f66920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 shutdown_connections 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 1 -- 172.21.15.165:0/49833351 wait complete. 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.947+0000 7f07861f5740 20 client.11949 trim_cache size 0 max 16384 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stdout: "return_code": 0, 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stdout: "scrub_tag": "19d7b494-8b1e-4f71-a4d1-ff236cb0bb8f", 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stdout: "mode": "asynchronous" 2024-04-25T10:21:28.953 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:21:28.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f075806b4d0 msgr2=0x7f075806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f075806b4d0 0x7f075806d990 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f076c000f60 tx=0x7f076c009040 comp rx=0 tx=0).stop 2024-04-25T10:21:28.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 msgr2=0x7f077c1e4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 0x7f077c1e4a80 secure :-1 s=READY pgs=2823 cs=0 l=1 rev1=1 crypto rx=0x7f0774009f00 tx=0x7f077400c490 comp rx=0 tx=0).stop 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 shutdown_connections 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f075806b4d0 0x7f075806d990 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f077c145140 0x7f077c1e4a80 unknown :-1 s=CLOSED pgs=2823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f077c0ceb20 0x7f077c131fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 --2- 172.21.15.165:0/1388346074 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f077c0c5030 0x7f077c1e3f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 >> 172.21.15.165:0/1388346074 conn(0x7f077c061cc0 msgr2=0x7f077c15d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 shutdown_connections 2024-04-25T10:21:28.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:28.948+0000 7f0781e00640 1 -- 172.21.15.165:0/1388346074 wait complete. 2024-04-25T10:21:28.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:28.968 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:21:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.169+0000 7f136b800640 1 Processor -- start 2024-04-25T10:21:29.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.169+0000 7f136b800640 1 -- start start 2024-04-25T10:21:29.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f136b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641446d0 0x7f13640b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f136b800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1364144d50 0x7f13640b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f136b800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f13641452a0 0x7f13640cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f136b800640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13640b2b20 con 0x7f1364144d50 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f136b800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f136406d400 con 0x7f13641452a0 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f136b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13640b49a0 con 0x7f13641446d0 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f135be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1364144d50 0x7f13640b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f135be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1364144d50 0x7f13640b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50546/0 (socket says 172.21.15.165:50546) 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f135be00640 1 -- 172.21.15.165:0/2999466977 learned_addr learned my addr 172.21.15.165:0/2999466977 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f1369800640 1 --2- 172.21.15.165:0/2999466977 >> v2:172.21.15.172:3300/0 conn(0x7f13641452a0 0x7f13640cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.171+0000 7f1368e00640 1 --2- 172.21.15.165:0/2999466977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641446d0 0x7f13640b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.172+0000 7f135be00640 1 -- 172.21.15.165:0/2999466977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641446d0 msgr2=0x7f13640b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.172+0000 7f135be00640 1 --2- 172.21.15.165:0/2999466977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641446d0 0x7f13640b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.172+0000 7f135be00640 1 -- 172.21.15.165:0/2999466977 >> v2:172.21.15.172:3300/0 conn(0x7f13641452a0 msgr2=0x7f13640cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.172+0000 7f135be00640 1 --2- 172.21.15.165:0/2999466977 >> v2:172.21.15.172:3300/0 conn(0x7f13641452a0 0x7f13640cf640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.172+0000 7f135be00640 1 -- 172.21.15.165:0/2999466977 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13640cccc0 con 0x7f1364144d50 2024-04-25T10:21:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.173+0000 7f135be00640 1 --2- 172.21.15.165:0/2999466977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1364144d50 0x7f13640b4460 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f1364145880 tx=0x7f134c02a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9203b5111abc6b3b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.173+0000 7f1363e00640 1 -- 172.21.15.165:0/2999466977 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c037070 con 0x7f1364144d50 2024-04-25T10:21:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.173+0000 7f1363e00640 1 -- 172.21.15.165:0/2999466977 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134c032440 con 0x7f1364144d50 2024-04-25T10:21:29.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.173+0000 7f1363e00640 1 -- 172.21.15.165:0/2999466977 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c032810 con 0x7f1364144d50 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 -- 172.21.15.165:0/2999466977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1364144d50 msgr2=0x7f13640b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 --2- 172.21.15.165:0/2999466977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1364144d50 0x7f13640b4460 secure :-1 s=READY pgs=2828 cs=0 l=1 rev1=1 crypto rx=0x7f1364145880 tx=0x7f134c02a1e0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 -- 172.21.15.165:0/2999466977 shutdown_connections 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 --2- 172.21.15.165:0/2999466977 >> v2:172.21.15.172:3300/0 conn(0x7f13641452a0 0x7f13640cf640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 --2- 172.21.15.165:0/2999466977 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1364144d50 0x7f13640b4460 unknown :-1 s=CLOSED pgs=2828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 --2- 172.21.15.165:0/2999466977 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641446d0 0x7f13640b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 -- 172.21.15.165:0/2999466977 >> 172.21.15.165:0/2999466977 conn(0x7f136406abe0 msgr2=0x7f136406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 -- 172.21.15.165:0/2999466977 shutdown_connections 2024-04-25T10:21:29.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.174+0000 7f136b800640 1 -- 172.21.15.165:0/2999466977 wait complete. 2024-04-25T10:21:29.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.175+0000 7f136b800640 1 Processor -- start 2024-04-25T10:21:29.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.176+0000 7f136b800640 1 -- start start 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f136b800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 0x7f136406da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f136b800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1364144d50 0x7f13641f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f136b800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641452a0 0x7f13641f7060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f136b800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13640cc700 con 0x7f13641446d0 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f136b800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13640cc420 con 0x7f1364144d50 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f136b800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13640cc9e0 con 0x7f13641452a0 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 0x7f136406da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 0x7f136406da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50562/0 (socket says 172.21.15.165:50562) 2024-04-25T10:21:29.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 -- 172.21.15.165:0/2176247253 learned_addr learned my addr 172.21.15.165:0/2176247253 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f135be00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1364144d50 0x7f13641f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1369800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641452a0 0x7f13641f7060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 -- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641452a0 msgr2=0x7f13641f7060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641452a0 0x7f13641f7060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 -- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1364144d50 msgr2=0x7f13641f6b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1364144d50 0x7f13641f6b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f1368e00640 1 -- 172.21.15.165:0/2176247253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13641f75a0 con 0x7f13641446d0 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.177+0000 7f135be00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1364144d50 0x7f13641f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:29.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.178+0000 7f1368e00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 0x7f136406da00 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f135400d800 tx=0x7f135400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.178+0000 7f1361600640 1 -- 172.21.15.165:0/2176247253 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354018020 con 0x7f13641446d0 2024-04-25T10:21:29.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.178+0000 7f1361600640 1 -- 172.21.15.165:0/2176247253 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1354009640 con 0x7f13641446d0 2024-04-25T10:21:29.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.178+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13641f7870 con 0x7f13641446d0 2024-04-25T10:21:29.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.178+0000 7f1361600640 1 -- 172.21.15.165:0/2176247253 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1354009a10 con 0x7f13641446d0 2024-04-25T10:21:29.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.178+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13641f37d0 con 0x7f13641446d0 2024-04-25T10:21:29.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.181+0000 7f1361600640 1 -- 172.21.15.165:0/2176247253 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f135401b020 con 0x7f13641446d0 2024-04-25T10:21:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.181+0000 7f1361600640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f134006b4d0 0x7f134006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.182+0000 7f1361600640 1 -- 172.21.15.165:0/2176247253 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f135408a650 con 0x7f13641446d0 2024-04-25T10:21:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.182+0000 7f135be00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f134006b4d0 0x7f134006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.182+0000 7f136e622740 1 Processor -- start 2024-04-25T10:21:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.182+0000 7f136e622740 1 -- start start 2024-04-25T10:21:29.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f135be00640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f134006b4d0 0x7f134006d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f134c034000 tx=0x7f134c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524af0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524aeef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f136e622740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b19af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f136e622740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e524a52c70 con 0x55e524b19670 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5249fbcf0 con 0x55e524b2b280 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.183+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e524a4ed10 con 0x55e524aeebb0 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b19af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b19af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50578/0 (socket says 172.21.15.165:50578) 2024-04-25T10:21:29.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 -- 172.21.15.165:0/1645704643 learned_addr learned my addr 172.21.15.165:0/1645704643 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f135be00640 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524aeef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 -- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 msgr2=0x55e524af0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524af0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 -- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 msgr2=0x55e524aeef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524aeef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 -- 172.21.15.165:0/1645704643 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e5248317d0 con 0x55e524b19670 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1369800640 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b19af0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f135c009f70 tx=0x7f135c0077c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3165673b2b23be05 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.184+0000 7f1359600640 1 -- 172.21.15.165:0/1645704643 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c013780 con 0x55e524b19670 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f1359600640 1 -- 172.21.15.165:0/1645704643 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c019070 con 0x55e524b19670 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f1359600640 1 -- 172.21.15.165:0/1645704643 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c013b60 con 0x55e524b19670 2024-04-25T10:21:29.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 -- 172.21.15.165:0/1645704643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 msgr2=0x55e524b19af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b19af0 secure :-1 s=READY pgs=2830 cs=0 l=1 rev1=1 crypto rx=0x7f135c009f70 tx=0x7f135c0077c0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 -- 172.21.15.165:0/1645704643 shutdown_connections 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b19af0 unknown :-1 s=CLOSED pgs=2830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524aeef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 --2- 172.21.15.165:0/1645704643 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524af0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 -- 172.21.15.165:0/1645704643 >> 172.21.15.165:0/1645704643 conn(0x55e524ab6a30 msgr2=0x55e524b34670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 -- 172.21.15.165:0/1645704643 shutdown_connections 2024-04-25T10:21:29.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.185+0000 7f136e622740 1 -- 172.21.15.165:0/1645704643 wait complete. 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.186+0000 7f136e622740 1 -- start start 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.186+0000 7f136e622740 1 Processor -- start 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 0x55e524b2c210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524b19670 0x55e524b2ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524b1bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e5249f13a0 con 0x55e524aeebb0 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e524b171c0 con 0x55e524b2b280 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e524b2c770 con 0x55e524b19670 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f136e622740 10 client.0 _ll_register_callbacks cb 0x55e524a0eb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.187+0000 7f1368e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 0x55e524b2c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 0x55e524b2c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50586/0 (socket says 172.21.15.165:50586) 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 -- 172.21.15.165:0/1075173108 learned_addr learned my addr 172.21.15.165:0/1075173108 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:29.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 -- 172.21.15.165:0/1075173108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524b19670 msgr2=0x55e524b2ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524b19670 0x55e524b2ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 -- 172.21.15.165:0/1075173108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 msgr2=0x55e524b1bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524b1bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 -- 172.21.15.165:0/1075173108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e5247673f0 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1368e00640 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 0x55e524b2c210 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f1354058870 tx=0x7f1354058cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6be724027878552 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1352000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1352000640 1 -- 172.21.15.165:0/1075173108 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13540551e0 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f1352000640 1 -- 172.21.15.165:0/1075173108 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f1354060df0 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.188+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f135c019070 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f1352000640 1 -- 172.21.15.165:0/1075173108 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1354055380 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 10 client.11961 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55e5247270f0 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f1352000640 1 -- 172.21.15.165:0/1075173108 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c019070 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f1352000640 1 -- 172.21.15.165:0/1075173108 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f1354013050 con 0x55e524aeebb0 2024-04-25T10:21:29.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 10 client.11961 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 10 client.11961 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 20 client.11961 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b04ab0 0x55e524b06400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 4 client.11961 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e524a601b0 con 0x55e524b04ab0 2024-04-25T10:21:29.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.189+0000 7f135be00640 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b04ab0 0x55e524b06400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.190+0000 7f135be00640 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b04ab0 0x55e524b06400 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ca12a955f6766c server_cookie=75641215242fb9ba in_seq=0 out_seq=0 2024-04-25T10:21:29.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.190+0000 7f1352000640 10 client.11961 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:29.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f1352000640 1 -- 172.21.15.165:0/1075173108 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x55e524a601b0 con 0x55e524b04ab0 2024-04-25T10:21:29.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f1352000640 10 client.11961 handle_command_reply: tid=0 2024-04-25T10:21:29.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 client.11961 shutdown 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 msgr2=0x55e524b2c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 0x55e524b2c210 secure :-1 s=READY pgs=2831 cs=0 l=1 rev1=1 crypto rx=0x7f1354058870 tx=0x7f1354058cb0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 shutdown_connections 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b04ab0 0x55e524b06400 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524b1bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f1352000640 0 client.11961 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524b19670 0x55e524b2ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 --2- 172.21.15.165:0/1075173108 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524aeebb0 0x55e524b2c210 unknown :-1 s=CLOSED pgs=2831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 >> 172.21.15.165:0/1075173108 conn(0x55e524ab6a30 msgr2=0x55e524b16a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 shutdown_connections 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 1 -- 172.21.15.165:0/1075173108 wait complete. 2024-04-25T10:21:29.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.191+0000 7f136e622740 20 client.11961 trim_cache size 0 max 16384 2024-04-25T10:21:29.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.208+0000 7f136e622740 1 Processor -- start 2024-04-25T10:21:29.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.208+0000 7f136e622740 1 -- start start 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.209+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524b1ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.209+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b19670 0x55e524b2b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.209+0000 7f136e622740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 0x55e524b16c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.209+0000 7f136e622740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1364200510 con 0x55e524b2b280 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.209+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13641ff710 con 0x55e524b19670 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.209+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13640c56b0 con 0x55e524aeebb0 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 0x55e524b16c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 0x55e524b16c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50594/0 (socket says 172.21.15.165:50594) 2024-04-25T10:21:29.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 -- 172.21.15.165:0/2426059606 learned_addr learned my addr 172.21.15.165:0/2426059606 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 -- 172.21.15.165:0/2426059606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 msgr2=0x55e524b1ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524b1ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 -- 172.21.15.165:0/2426059606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b19670 msgr2=0x55e524b2b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b19670 0x55e524b2b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 -- 172.21.15.165:0/2426059606 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e524b06a80 con 0x55e524b2b280 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.210+0000 7f1369800640 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 0x55e524b16c10 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f135c00b010 tx=0x7f135c012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b75a929e59384695 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f1358c00640 1 -- 172.21.15.165:0/2426059606 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c003b90 con 0x55e524b2b280 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f1358c00640 1 -- 172.21.15.165:0/2426059606 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c019070 con 0x55e524b2b280 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f1358c00640 1 -- 172.21.15.165:0/2426059606 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c004040 con 0x55e524b2b280 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 -- 172.21.15.165:0/2426059606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 msgr2=0x55e524b16c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 0x55e524b16c10 secure :-1 s=READY pgs=2832 cs=0 l=1 rev1=1 crypto rx=0x7f135c00b010 tx=0x7f135c012000 comp rx=0 tx=0).stop 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 -- 172.21.15.165:0/2426059606 shutdown_connections 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b2b280 0x55e524b16c10 unknown :-1 s=CLOSED pgs=2832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b19670 0x55e524b2b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 --2- 172.21.15.165:0/2426059606 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524b1ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 -- 172.21.15.165:0/2426059606 >> 172.21.15.165:0/2426059606 conn(0x55e524ab6a30 msgr2=0x55e524b1b330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.211+0000 7f136e622740 1 -- 172.21.15.165:0/2426059606 shutdown_connections 2024-04-25T10:21:29.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.212+0000 7f136e622740 1 -- 172.21.15.165:0/2426059606 wait complete. 2024-04-25T10:21:29.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.212+0000 7f136e622740 1 -- start start 2024-04-25T10:21:29.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.212+0000 7f136e622740 1 Processor -- start 2024-04-25T10:21:29.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524b15c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b46c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524b47160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e524ae46d0 con 0x55e524b19670 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13641ff110 con 0x55e524b2b280 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e524ae9c00 con 0x55e524aeebb0 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.213+0000 7f136e622740 10 client.0 _ll_register_callbacks cb 0x7f1354058940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b46c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b46c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:50606/0 (socket says 172.21.15.165:50606) 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 -- 172.21.15.165:0/3650576497 learned_addr learned my addr 172.21.15.165:0/3650576497 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 -- 172.21.15.165:0/3650576497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 msgr2=0x55e524b15c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524b15c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 -- 172.21.15.165:0/3650576497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 msgr2=0x55e524b47160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524b47160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 -- 172.21.15.165:0/3650576497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e524b03850 con 0x55e524b19670 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f135be00640 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b46c20 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f134c009960 tx=0x7f134c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f9e23260fb9665 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.214+0000 7f1352000640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f1352000640 1 -- 172.21.15.165:0/3650576497 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134c037040 con 0x55e524b19670 2024-04-25T10:21:29.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f1352000640 1 -- 172.21.15.165:0/3650576497 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f134c04b270 con 0x55e524b19670 2024-04-25T10:21:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f135c019070 con 0x55e524b19670 2024-04-25T10:21:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f1352000640 1 -- 172.21.15.165:0/3650576497 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7f134c05a050 con 0x55e524b19670 2024-04-25T10:21:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 10 client.11967 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:29.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x55e524ba0000 con 0x55e524b19670 2024-04-25T10:21:29.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f1352000640 1 -- 172.21.15.165:0/3650576497 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f135c019070 con 0x55e524b19670 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f1352000640 1 -- 172.21.15.165:0/3650576497 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7f134c05d020 con 0x55e524b19670 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 10 client.11967 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 10 client.11967 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 20 client.11967 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b4db00 0x55e524b4df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 4 client.11967 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.215+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55e524a5a500 con 0x55e524b4db00 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.216+0000 7f1368e00640 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b4db00 0x55e524b4df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.216+0000 7f1368e00640 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b4db00 0x55e524b4df20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba3e0078102c56ea server_cookie=557b211237856521 in_seq=0 out_seq=0 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.216+0000 7f1352000640 10 client.11967 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.216+0000 7f1352000640 1 -- 172.21.15.165:0/3650576497 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x55e524a5a500 con 0x55e524b4db00 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.216+0000 7f1352000640 10 client.11967 handle_command_reply: tid=0 2024-04-25T10:21:29.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.216+0000 7f136e622740 1 client.11967 shutdown 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 msgr2=0x55e524b46c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b46c20 secure :-1 s=READY pgs=2833 cs=0 l=1 rev1=1 crypto rx=0x7f134c009960 tx=0x7f134c0046c0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 shutdown_connections 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x55e524b4db00 0x55e524b4df20 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x55e524b2b280 0x55e524b47160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x55e524b19670 0x55e524b46c20 unknown :-1 s=CLOSED pgs=2833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 --2- 172.21.15.165:0/3650576497 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x55e524aeebb0 0x55e524b15c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f1352000640 0 client.11967 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 >> 172.21.15.165:0/3650576497 conn(0x55e524ab6a30 msgr2=0x55e524b2b980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 shutdown_connections 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 1 -- 172.21.15.165:0/3650576497 wait complete. 2024-04-25T10:21:29.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.217+0000 7f136e622740 20 client.11967 trim_cache size 0 max 16384 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": { 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: "19d7b494-8b1e-4f71-a4d1-ff236cb0bb8f": { 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: "path": "/", 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: "tag": "19d7b494-8b1e-4f71-a4d1-ff236cb0bb8f", 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: "options": "recursive,repair" 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout: } 2024-04-25T10:21:29.223 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:21:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f134006b4d0 msgr2=0x7f134006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f134006b4d0 0x7f134006d990 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f134c034000 tx=0x7f134c02e000 comp rx=0 tx=0).stop 2024-04-25T10:21:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 msgr2=0x7f136406da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 0x7f136406da00 secure :-1 s=READY pgs=2829 cs=0 l=1 rev1=1 crypto rx=0x7f135400d800 tx=0x7f135400dcc0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 shutdown_connections 2024-04-25T10:21:29.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f134006b4d0 0x7f134006d990 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f13641452a0 0x7f13641f7060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1364144d50 0x7f13641f6b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 --2- 172.21.15.165:0/2176247253 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f13641446d0 0x7f136406da00 unknown :-1 s=CLOSED pgs=2829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 >> 172.21.15.165:0/2176247253 conn(0x7f136406abe0 msgr2=0x7f13640b1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 shutdown_connections 2024-04-25T10:21:29.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:29.218+0000 7f136b800640 1 -- 172.21.15.165:0/2176247253 wait complete. 2024-04-25T10:21:29.242 INFO:tasks.cephfs.filesystem:scrub status for tag:19d7b494-8b1e-4f71-a4d1-ff236cb0bb8f - {'path': '/', 'tag': '19d7b494-8b1e-4f71-a4d1-ff236cb0bb8f', 'options': 'recursive,repair'} 2024-04-25T10:21:31.949 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:31.957 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:21:59.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:21:59.244 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.14:0 scrub status 2024-04-25T10:21:59.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.456+0000 7ffb6dc00640 1 Processor -- start 2024-04-25T10:21:59.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.457+0000 7ffb6dc00640 1 -- start start 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.458+0000 7ffb6dc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ffb68144d50 0x7ffb6806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.458+0000 7ffb6dc00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ffb681452a0 0x7ffb6806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.458+0000 7ffb6dc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb680cccc0 con 0x7ffb681446d0 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.458+0000 7ffb6dc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb680b2b20 con 0x7ffb68144d50 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.458+0000 7ffb6dc00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb6806d400 con 0x7ffb681452a0 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.458+0000 7ffb66a00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ffb681446d0 0x7ffb680b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36428/0 (socket says 172.21.15.165:36428) 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb66a00640 1 -- 172.21.15.165:0/1031949507 learned_addr learned my addr 172.21.15.165:0/1031949507 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb5e000640 1 --2- 172.21.15.165:0/1031949507 >> v2:172.21.15.165:3300/0 conn(0x7ffb68144d50 0x7ffb6806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb67400640 1 --2- 172.21.15.165:0/1031949507 >> v2:172.21.15.172:3300/0 conn(0x7ffb681452a0 0x7ffb6806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb66000640 1 -- 172.21.15.165:0/1031949507 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693619929 0 0) 0x7ffb680cccc0 con 0x7ffb681446d0 2024-04-25T10:21:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb66000640 1 -- 172.21.15.165:0/1031949507 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb4c003520 con 0x7ffb681446d0 2024-04-25T10:21:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb5e000640 1 -- 172.21.15.165:0/1031949507 >> v2:172.21.15.172:3300/0 conn(0x7ffb681452a0 msgr2=0x7ffb6806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb5e000640 1 --2- 172.21.15.165:0/1031949507 >> v2:172.21.15.172:3300/0 conn(0x7ffb681452a0 0x7ffb6806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb5e000640 1 -- 172.21.15.165:0/1031949507 >> v1:172.21.15.165:6789/0 conn(0x7ffb681446d0 legacy=0x7ffb680b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb5e000640 1 -- 172.21.15.165:0/1031949507 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680cc140 con 0x7ffb68144d50 2024-04-25T10:21:59.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.459+0000 7ffb67400640 1 --2- 172.21.15.165:0/1031949507 >> v2:172.21.15.172:3300/0 conn(0x7ffb681452a0 0x7ffb6806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:21:59.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.460+0000 7ffb5e000640 1 --2- 172.21.15.165:0/1031949507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb6806d890 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7ffb6815bfe0 tx=0x7ffb5002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5531ff813668095 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb66000640 1 -- 172.21.15.165:0/1031949507 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50037070 con 0x7ffb68144d50 2024-04-25T10:21:59.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb66000640 1 -- 172.21.15.165:0/1031949507 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb50032440 con 0x7ffb68144d50 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb66000640 1 -- 172.21.15.165:0/1031949507 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb500327f0 con 0x7ffb68144d50 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 -- 172.21.15.165:0/1031949507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 msgr2=0x7ffb6806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/1031949507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb6806d890 secure :-1 s=READY pgs=2835 cs=0 l=1 rev1=1 crypto rx=0x7ffb6815bfe0 tx=0x7ffb5002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 -- 172.21.15.165:0/1031949507 shutdown_connections 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/1031949507 >> v2:172.21.15.172:3300/0 conn(0x7ffb681452a0 0x7ffb6806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/1031949507 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb6806d890 unknown :-1 s=CLOSED pgs=2835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 -- 172.21.15.165:0/1031949507 >> 172.21.15.165:0/1031949507 conn(0x7ffb6806abe0 msgr2=0x7ffb6806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 -- 172.21.15.165:0/1031949507 shutdown_connections 2024-04-25T10:21:59.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.461+0000 7ffb6dc00640 1 -- 172.21.15.165:0/1031949507 wait complete. 2024-04-25T10:21:59.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.463+0000 7ffb6dc00640 1 Processor -- start 2024-04-25T10:21:59.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.463+0000 7ffb6dc00640 1 -- start start 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.464+0000 7ffb6dc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb681446d0 0x7ffb68155050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.464+0000 7ffb6dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb681f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.464+0000 7ffb6dc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb681452a0 0x7ffb680b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.464+0000 7ffb6dc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680cc9e0 con 0x7ffb68144d50 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.464+0000 7ffb6dc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680cc700 con 0x7ffb681452a0 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.464+0000 7ffb6dc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb68144ab0 con 0x7ffb681446d0 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb681f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb681f5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42356/0 (socket says 172.21.15.165:42356) 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 -- 172.21.15.165:0/364401078 learned_addr learned my addr 172.21.15.165:0/364401078 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 -- 172.21.15.165:0/364401078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb681446d0 msgr2=0x7ffb68155050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb681446d0 0x7ffb68155050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 -- 172.21.15.165:0/364401078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb681452a0 msgr2=0x7ffb680b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb681452a0 0x7ffb680b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 -- 172.21.15.165:0/364401078 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb680ab920 con 0x7ffb68144d50 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.465+0000 7ffb5e000640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb681f5590 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7ffb50000c00 tx=0x7ffb500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.466+0000 7ffb5f400640 1 -- 172.21.15.165:0/364401078 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50037040 con 0x7ffb68144d50 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.466+0000 7ffb5f400640 1 -- 172.21.15.165:0/364401078 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb500304c0 con 0x7ffb68144d50 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.466+0000 7ffb5f400640 1 -- 172.21.15.165:0/364401078 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50030890 con 0x7ffb68144d50 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.466+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb680adbf0 con 0x7ffb68144d50 2024-04-25T10:21:59.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.466+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb680ae1b0 con 0x7ffb68144d50 2024-04-25T10:21:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.468+0000 7ffb5f400640 1 -- 172.21.15.165:0/364401078 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ffb50044020 con 0x7ffb68144d50 2024-04-25T10:21:59.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.469+0000 7ffb5f400640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb4406b4d0 0x7ffb4406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.469+0000 7ffb5f400640 1 -- 172.21.15.165:0/364401078 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffb500aa860 con 0x7ffb68144d50 2024-04-25T10:21:59.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.469+0000 7ffb66a00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb4406b4d0 0x7ffb4406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.470+0000 7ffb70b6a740 1 Processor -- start 2024-04-25T10:21:59.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.470+0000 7ffb66a00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb4406b4d0 0x7ffb4406d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ffb58000f60 tx=0x7ffb580078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.470+0000 7ffb70b6a740 1 -- start start 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 0x563e5665c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56696690 0x563e5665a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e56684f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e565b9c70 con 0x563e56659fc0 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e56562cf0 con 0x563e56696690 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e565b5d10 con 0x563e56684a80 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb66a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 0x563e5665c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb66a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 0x563e5665c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42362/0 (socket says 172.21.15.165:42362) 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.471+0000 7ffb66a00640 1 -- 172.21.15.165:0/3357513440 learned_addr learned my addr 172.21.15.165:0/3357513440 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:59.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb5e000640 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56696690 0x563e5665a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb67400640 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e56684f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb66a00640 1 -- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 msgr2=0x563e56684f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb66a00640 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e56684f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb66a00640 1 -- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56696690 msgr2=0x563e5665a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb66a00640 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56696690 0x563e5665a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb66a00640 1 -- 172.21.15.165:0/3357513440 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e563987d0 con 0x563e56659fc0 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb66a00640 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 0x563e5665c120 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005d70 tx=0x7ffb5800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8c0d66c557aefb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.472+0000 7ffb57400640 1 -- 172.21.15.165:0/3357513440 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58005e60 con 0x563e56659fc0 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb57400640 1 -- 172.21.15.165:0/3357513440 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58007560 con 0x563e56659fc0 2024-04-25T10:21:59.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb57400640 1 -- 172.21.15.165:0/3357513440 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb58016b50 con 0x563e56659fc0 2024-04-25T10:21:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 -- 172.21.15.165:0/3357513440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 msgr2=0x563e5665c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 0x563e5665c120 secure :-1 s=READY pgs=2837 cs=0 l=1 rev1=1 crypto rx=0x7ffb58005d70 tx=0x7ffb5800f000 comp rx=0 tx=0).stop 2024-04-25T10:21:59.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 -- 172.21.15.165:0/3357513440 shutdown_connections 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e56684f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56696690 0x563e5665a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/3357513440 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56659fc0 0x563e5665c120 unknown :-1 s=CLOSED pgs=2837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 -- 172.21.15.165:0/3357513440 >> 172.21.15.165:0/3357513440 conn(0x563e5661da30 msgr2=0x563e5669fa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 -- 172.21.15.165:0/3357513440 shutdown_connections 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.473+0000 7ffb70b6a740 1 -- 172.21.15.165:0/3357513440 wait complete. 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.474+0000 7ffb70b6a740 1 -- start start 2024-04-25T10:21:59.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.474+0000 7ffb70b6a740 1 Processor -- start 2024-04-25T10:21:59.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.475+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5669a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.475+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e5669a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566a2b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e565583a0 con 0x563e56696690 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e566825d0 con 0x563e56659fc0 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e56672260 con 0x563e56684a80 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb70b6a740 10 client.0 _ll_register_callbacks cb 0x563e56575b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566a2b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566a2b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42378/0 (socket says 172.21.15.165:42378) 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 -- 172.21.15.165:0/2826339966 learned_addr learned my addr 172.21.15.165:0/2826339966 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb66a00640 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5669a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 -- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 msgr2=0x563e5669a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e5669a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 -- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 msgr2=0x563e5669a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5669a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.476+0000 7ffb67400640 1 -- 172.21.15.165:0/2826339966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e562ce3f0 con 0x563e56696690 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb67400640 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566a2b80 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000ea90 tx=0x7ffb6000ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7d87ef8cdc9016 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb4be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:59.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb4be00640 1 -- 172.21.15.165:0/2826339966 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6001b060 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb4be00640 1 -- 172.21.15.165:0/2826339966 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffb60016430 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb58007560 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb4be00640 1 -- 172.21.15.165:0/2826339966 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb60021050 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb70b6a740 10 client.11982 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x563e5628e0f0 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.477+0000 7ffb4be00640 1 -- 172.21.15.165:0/2826339966 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb58007560 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb4be00640 1 -- 172.21.15.165:0/2826339966 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb60024020 con 0x563e56696690 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb70b6a740 10 client.11982 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb70b6a740 10 client.11982 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb70b6a740 20 client.11982 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e566724b0 0x563e56672900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb70b6a740 4 client.11982 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e565c71b0 con 0x563e566724b0 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.478+0000 7ffb66a00640 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e566724b0 0x563e56672900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb66a00640 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e566724b0 0x563e56672900 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=506c2a9f991549b2 server_cookie=45d752a5df89e293 in_seq=0 out_seq=0 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb4be00640 10 client.11982 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb4be00640 1 -- 172.21.15.165:0/2826339966 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x563e565c71b0 con 0x563e566724b0 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb4be00640 10 client.11982 handle_command_reply: tid=0 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb70b6a740 1 client.11982 shutdown 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 msgr2=0x563e566a2b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.479+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566a2b80 secure :-1 s=READY pgs=2838 cs=0 l=1 rev1=1 crypto rx=0x7ffb6000ea90 tx=0x7ffb6000ef50 comp rx=0 tx=0).stop 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 shutdown_connections 2024-04-25T10:21:59.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e566724b0 0x563e56672900 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566a2b80 unknown :-1 s=CLOSED pgs=2838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e5669a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/2826339966 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5669a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb4be00640 0 client.11982 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 >> 172.21.15.165:0/2826339966 conn(0x563e5661da30 msgr2=0x563e56681d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 shutdown_connections 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 1 -- 172.21.15.165:0/2826339966 wait complete. 2024-04-25T10:21:59.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.480+0000 7ffb70b6a740 20 client.11982 trim_cache size 0 max 16384 2024-04-25T10:21:59.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.496+0000 7ffb70b6a740 1 Processor -- start 2024-04-25T10:21:59.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.496+0000 7ffb70b6a740 1 -- start start 2024-04-25T10:21:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.497+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5665afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.497+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 0x563e56688570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.497+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56696690 0x563e566852d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.497+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb681fc110 con 0x563e56684a80 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.497+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680ceaf0 con 0x563e56659fc0 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.497+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680acf40 con 0x563e56696690 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 0x563e56688570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 0x563e56688570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42384/0 (socket says 172.21.15.165:42384) 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 -- 172.21.15.165:0/805979155 learned_addr learned my addr 172.21.15.165:0/805979155 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 -- 172.21.15.165:0/805979155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56696690 msgr2=0x563e566852d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56696690 0x563e566852d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 -- 172.21.15.165:0/805979155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 msgr2=0x563e5665afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5665afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 -- 172.21.15.165:0/805979155 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e5669b410 con 0x563e56684a80 2024-04-25T10:21:59.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.498+0000 7ffb5e000640 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 0x563e56688570 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7ffb500a8a40 tx=0x7ffb500788f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=869699d2f841c597 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb56a00640 1 -- 172.21.15.165:0/805979155 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50078cf0 con 0x563e56684a80 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb56a00640 1 -- 172.21.15.165:0/805979155 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5007ce40 con 0x563e56684a80 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb56a00640 1 -- 172.21.15.165:0/805979155 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb50081400 con 0x563e56684a80 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 -- 172.21.15.165:0/805979155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 msgr2=0x563e56688570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 0x563e56688570 secure :-1 s=READY pgs=2839 cs=0 l=1 rev1=1 crypto rx=0x7ffb500a8a40 tx=0x7ffb500788f0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 -- 172.21.15.165:0/805979155 shutdown_connections 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56696690 0x563e566852d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56684a80 0x563e56688570 unknown :-1 s=CLOSED pgs=2839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/805979155 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e5665afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 -- 172.21.15.165:0/805979155 >> 172.21.15.165:0/805979155 conn(0x563e5661da30 msgr2=0x563e566b2620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 -- 172.21.15.165:0/805979155 shutdown_connections 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.499+0000 7ffb70b6a740 1 -- 172.21.15.165:0/805979155 wait complete. 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.500+0000 7ffb70b6a740 1 -- start start 2024-04-25T10:21:59.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.500+0000 7ffb70b6a740 1 Processor -- start 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e56703720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e566852a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e56655460 con 0x563e56696690 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb680ce4f0 con 0x563e56659fc0 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x563e56659bd0 con 0x563e56684a80 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.501+0000 7ffb70b6a740 10 client.0 _ll_register_callbacks cb 0x7ffb60009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:21:59.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566857e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42400/0 (socket says 172.21.15.165:42400) 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 -- 172.21.15.165:0/1256374532 learned_addr learned my addr 172.21.15.165:0/1256374532 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 -- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 msgr2=0x563e566852a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e566852a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 -- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 msgr2=0x563e56703720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb66a00640 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e56703720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e56703720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 -- 172.21.15.165:0/1256374532 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e566957d0 con 0x563e56696690 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb67400640 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566857e0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009530 tx=0x7ffb60005d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d87f7623ce55a54 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.502+0000 7ffb4be00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb4be00640 1 -- 172.21.15.165:0/1256374532 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb6000c850 con 0x563e56696690 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb4be00640 1 -- 172.21.15.165:0/1256374532 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ffb6000cd90 con 0x563e56696690 2024-04-25T10:21:59.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb5007ce40 con 0x563e56696690 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb4be00640 1 -- 172.21.15.165:0/1256374532 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb60033050 con 0x563e56696690 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 10 client.11988 fetch_fsmap learned FSMap version 146 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x563e566a26c0 con 0x563e56696690 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb4be00640 1 -- 172.21.15.165:0/1256374532 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb5007ce40 con 0x563e56696690 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb4be00640 1 -- 172.21.15.165:0/1256374532 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7ffb60036020 con 0x563e56696690 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 10 client.11988 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 10 client.11988 resolve_mds: resolved 14:0 to role '14:0' aka daemon mds.c 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 20 client.11988 populate_metadata read hostname 'smithi165' 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e56701e80 0x563e56670f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 4 client.11988 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-25T10:21:59.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.503+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563e565c1500 con 0x563e56701e80 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb66a00640 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e56701e80 0x563e56670f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb66a00640 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e56701e80 0x563e56670f00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3294bd8cbc26aa2 server_cookie=b98fd01bbb1347a0 in_seq=0 out_seq=0 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb4be00640 10 client.11988 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb4be00640 1 -- 172.21.15.165:0/1256374532 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563e565c1500 con 0x563e56701e80 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb4be00640 10 client.11988 handle_command_reply: tid=0 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb70b6a740 1 client.11988 shutdown 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.504+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 msgr2=0x563e566857e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566857e0 secure :-1 s=READY pgs=2840 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009530 tx=0x7ffb60005d10 comp rx=0 tx=0).stop 2024-04-25T10:21:59.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 shutdown_connections 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x563e56701e80 0x563e56670f00 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x563e56696690 0x563e566857e0 unknown :-1 s=CLOSED pgs=2840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x563e56684a80 0x563e566852a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 --2- 172.21.15.165:0/1256374532 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x563e56659fc0 0x563e56703720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb4be00640 0 client.11988 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 >> 172.21.15.165:0/1256374532 conn(0x563e5661da30 msgr2=0x563e56681090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 shutdown_connections 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 1 -- 172.21.15.165:0/1256374532 wait complete. 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.505+0000 7ffb70b6a740 20 client.11988 trim_cache size 0 max 16384 2024-04-25T10:21:59.513 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:21:59.514 INFO:teuthology.orchestra.run.smithi165.stdout: "status": "no active scrubs running", 2024-04-25T10:21:59.514 INFO:teuthology.orchestra.run.smithi165.stdout: "scrubs": {} 2024-04-25T10:21:59.514 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb4406b4d0 msgr2=0x7ffb4406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb4406b4d0 0x7ffb4406d990 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7ffb58000f60 tx=0x7ffb580078f0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 msgr2=0x7ffb681f5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb681f5590 secure :-1 s=READY pgs=2836 cs=0 l=1 rev1=1 crypto rx=0x7ffb50000c00 tx=0x7ffb500046c0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 shutdown_connections 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ffb4406b4d0 0x7ffb4406d990 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ffb681452a0 0x7ffb680b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ffb68144d50 0x7ffb681f5590 unknown :-1 s=CLOSED pgs=2836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 --2- 172.21.15.165:0/364401078 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ffb681446d0 0x7ffb68155050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 >> 172.21.15.165:0/364401078 conn(0x7ffb6806abe0 msgr2=0x7ffb680b21b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 shutdown_connections 2024-04-25T10:21:59.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:21:59.506+0000 7ffb6dc00640 1 -- 172.21.15.165:0/364401078 wait complete. 2024-04-25T10:21:59.527 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-25T10:21:59.930 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:00.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.140+0000 7fb326e00640 1 Processor -- start 2024-04-25T10:22:00.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.140+0000 7fb326e00640 1 -- start start 2024-04-25T10:22:00.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.141+0000 7fb326e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32015e520 0x7fb32015e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.141+0000 7fb326e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fb320070ff0 0x7fb320071410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.141+0000 7fb326e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb320071950 con 0x7fb32015f040 2024-04-25T10:22:00.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.141+0000 7fb326e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3200aee30 con 0x7fb320070ff0 2024-04-25T10:22:00.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.141+0000 7fb326e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200ae590 con 0x7fb32015e520 2024-04-25T10:22:00.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb325000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb32015f040 0x7fb3200adb90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36432/0 (socket says 172.21.15.165:36432) 2024-04-25T10:22:00.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb325000640 1 -- 172.21.15.165:0/945870012 learned_addr learned my addr 172.21.15.165:0/945870012 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:00.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb326400640 1 --2- 172.21.15.165:0/945870012 >> v2:172.21.15.165:3300/0 conn(0x7fb320070ff0 0x7fb320071410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb325a00640 1 --2- 172.21.15.165:0/945870012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32015e520 0x7fb32015e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb31fe00640 1 -- 172.21.15.165:0/945870012 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2565930175 0 0) 0x7fb320071950 con 0x7fb32015f040 2024-04-25T10:22:00.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb31fe00640 1 -- 172.21.15.165:0/945870012 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb308003520 con 0x7fb32015f040 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb326400640 1 -- 172.21.15.165:0/945870012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32015e520 msgr2=0x7fb32015e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb326400640 1 --2- 172.21.15.165:0/945870012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32015e520 0x7fb32015e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.142+0000 7fb326400640 1 -- 172.21.15.165:0/945870012 >> v1:172.21.15.165:6789/0 conn(0x7fb32015f040 legacy=0x7fb3200adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.143+0000 7fb326400640 1 -- 172.21.15.165:0/945870012 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3200ae870 con 0x7fb320070ff0 2024-04-25T10:22:00.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.144+0000 7fb326400640 1 --2- 172.21.15.165:0/945870012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb320070ff0 0x7fb320071410 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fb318009d90 tx=0x7fb31802a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26283ebe1fbe075c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.144+0000 7fb31fe00640 1 -- 172.21.15.165:0/945870012 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318007740 con 0x7fb320070ff0 2024-04-25T10:22:00.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.144+0000 7fb31fe00640 1 -- 172.21.15.165:0/945870012 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3180078e0 con 0x7fb320070ff0 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.144+0000 7fb31fe00640 1 -- 172.21.15.165:0/945870012 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb318007cb0 con 0x7fb320070ff0 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.144+0000 7fb326e00640 1 -- 172.21.15.165:0/945870012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb320070ff0 msgr2=0x7fb320071410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.144+0000 7fb326e00640 1 --2- 172.21.15.165:0/945870012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb320070ff0 0x7fb320071410 secure :-1 s=READY pgs=2842 cs=0 l=1 rev1=1 crypto rx=0x7fb318009d90 tx=0x7fb31802a660 comp rx=0 tx=0).stop 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.145+0000 7fb326e00640 1 -- 172.21.15.165:0/945870012 shutdown_connections 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.145+0000 7fb326e00640 1 --2- 172.21.15.165:0/945870012 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb320070ff0 0x7fb320071410 unknown :-1 s=CLOSED pgs=2842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.145+0000 7fb326e00640 1 --2- 172.21.15.165:0/945870012 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb32015e520 0x7fb32015e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.145+0000 7fb326e00640 1 -- 172.21.15.165:0/945870012 >> 172.21.15.165:0/945870012 conn(0x7fb32006abe0 msgr2=0x7fb32006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.145+0000 7fb326e00640 1 -- 172.21.15.165:0/945870012 shutdown_connections 2024-04-25T10:22:00.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.145+0000 7fb326e00640 1 -- 172.21.15.165:0/945870012 wait complete. 2024-04-25T10:22:00.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.146+0000 7fb326e00640 1 Processor -- start 2024-04-25T10:22:00.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.147+0000 7fb326e00640 1 -- start start 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.147+0000 7fb326e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb320070ff0 0x7fb3201f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb326e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 0x7fb32015a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb326e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32015f040 0x7fb32013ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb326e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3200ae2a0 con 0x7fb32015e520 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb326e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb32006d1f0 con 0x7fb32015f040 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb326e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb32015f4a0 con 0x7fb320070ff0 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 0x7fb32015a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 0x7fb32015a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42422/0 (socket says 172.21.15.165:42422) 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 -- 172.21.15.165:0/1322477823 learned_addr learned my addr 172.21.15.165:0/1322477823 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb326400640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32015f040 0x7fb32013ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 -- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb320070ff0 msgr2=0x7fb3201f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb320070ff0 0x7fb3201f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 -- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32015f040 msgr2=0x7fb32013ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32015f040 0x7fb32013ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.148+0000 7fb325000640 1 -- 172.21.15.165:0/1322477823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb32015a730 con 0x7fb32015e520 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.149+0000 7fb325000640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 0x7fb32015a1f0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fb31000c430 tx=0x7fb31000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.149+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb310017070 con 0x7fb32015e520 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.149+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb310005ce0 con 0x7fb32015e520 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.149+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3100072f0 con 0x7fb32015e520 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.149+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3201f7060 con 0x7fb32015e520 2024-04-25T10:22:00.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.149+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3201f7620 con 0x7fb32015e520 2024-04-25T10:22:00.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.150+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb320073990 con 0x7fb32015e520 2024-04-25T10:22:00.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.151+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb31001a070 con 0x7fb32015e520 2024-04-25T10:22:00.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.152+0000 7fb31d600640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30406b4d0 0x7fb30406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.153+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fb310089940 con 0x7fb32015e520 2024-04-25T10:22:00.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.153+0000 7fb325a00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30406b4d0 0x7fb30406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.153+0000 7fb325a00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30406b4d0 0x7fb30406d990 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb314006a90 tx=0x7fb314006a20 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.157+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb310058560 con 0x7fb32015e520 2024-04-25T10:22:00.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.387+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb32015ec80 con 0x7fb32015e520 2024-04-25T10:22:00.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.388+0000 7fb31d600640 1 -- 172.21.15.165:0/1322477823 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4965 (secure 0 0 0) 0x7fb310057e60 con 0x7fb32015e520 2024-04-25T10:22:00.391 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:00.392 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:54.991856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:22:00.392 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 146 2024-04-25T10:22:00.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.390+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30406b4d0 msgr2=0x7fb30406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.390+0000 7fb326e00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30406b4d0 0x7fb30406d990 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fb314006a90 tx=0x7fb314006a20 comp rx=0 tx=0).stop 2024-04-25T10:22:00.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.390+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 msgr2=0x7fb32015a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.390+0000 7fb326e00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 0x7fb32015a1f0 secure :-1 s=READY pgs=2843 cs=0 l=1 rev1=1 crypto rx=0x7fb31000c430 tx=0x7fb31000c8f0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 shutdown_connections 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb30406b4d0 0x7fb30406d990 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb32015f040 0x7fb32013ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb32015e520 0x7fb32015a1f0 unknown :-1 s=CLOSED pgs=2843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 --2- 172.21.15.165:0/1322477823 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb320070ff0 0x7fb3201f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 >> 172.21.15.165:0/1322477823 conn(0x7fb32006abe0 msgr2=0x7fb32006d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 shutdown_connections 2024-04-25T10:22:00.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.391+0000 7fb326e00640 1 -- 172.21.15.165:0/1322477823 wait complete. 2024-04-25T10:22:00.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 146, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-25T10:20:37.587736+0000', 'modified': '2024-04-25T10:20:54.991856+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 198, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11808}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11808': {'gid': 11808, 'name': 'c', 'rank': 0, 'incarnation': 143, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.165:6837/2156983703', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.165:6835', 'nonce': 2156983703}, {'type': 'v1', 'addr': '172.21.15.165:6837', 'nonce': 2156983703}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [30], 'metadata_pool': 29, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2024-04-25T10:22:00.413 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-25T10:22:00.413 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-04-25T10:22:00.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.620+0000 7fe1f4e00640 1 Processor -- start 2024-04-25T10:22:00.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.620+0000 7fe1f4e00640 1 -- start start 2024-04-25T10:22:00.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1f4e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe1f00b5b40 0x7fe1f0168b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1f4e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f015e110 con 0x7fe1f015dd30 2024-04-25T10:22:00.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1f4e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00b2fd0 con 0x7fe1f006bff0 2024-04-25T10:22:00.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1f4e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f006d000 con 0x7fe1f00b5b40 2024-04-25T10:22:00.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1ef400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe1f015dd30 0x7fe1f00cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36436/0 (socket says 172.21.15.165:36436) 2024-04-25T10:22:00.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1ef400640 1 -- 172.21.15.165:0/3867875504 learned_addr learned my addr 172.21.15.165:0/3867875504 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1efe00640 1 --2- 172.21.15.165:0/3867875504 >> v2:172.21.15.172:3300/0 conn(0x7fe1f00b5b40 0x7fe1f0168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.622+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340428060 0 0) 0x7fe1f015e110 con 0x7fe1f015dd30 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1d4003520 con 0x7fe1f015dd30 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3358324217 0 0) 0x7fe1d4003520 con 0x7fe1f015dd30 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 >> v2:172.21.15.172:3300/0 conn(0x7fe1f00b5b40 msgr2=0x7fe1f0168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 --2- 172.21.15.165:0/3867875504 >> v2:172.21.15.172:3300/0 conn(0x7fe1f00b5b40 0x7fe1f0168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 >> v1:172.21.15.172:6789/0 conn(0x7fe1f006bff0 legacy=0x7fe1f00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f006c480 con 0x7fe1f015dd30 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe1e0001670 con 0x7fe1f015dd30 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 >> v1:172.21.15.165:6789/0 conn(0x7fe1f015dd30 legacy=0x7fe1f00cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1d4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1d4004800 0x7fe1d4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d40051c0 0x7fe1d400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f006d000 con 0x7fe1d40039b0 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400c110 con 0x7fe1d40051c0 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1d400c310 con 0x7fe1d4004800 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.623+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1d4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.624+0000 7fe1ef400640 1 -- 172.21.15.165:0/3867875504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1d4004800 msgr2=0x7fe1d4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.624+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1d4004800 0x7fe1d4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.624+0000 7fe1ef400640 1 -- 172.21.15.165:0/3867875504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d40051c0 msgr2=0x7fe1d400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.624+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1d40051c0 0x7fe1d400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.624+0000 7fe1ef400640 1 -- 172.21.15.165:0/3867875504 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1d4003520 con 0x7fe1d40039b0 2024-04-25T10:22:00.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.624+0000 7fe1efe00640 1 -- 172.21.15.165:0/3867875504 reap_dead start 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.625+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1d4003dd0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c240 tx=0x7fe1d4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b49dcea8f660fe4f server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.625+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007d10 con 0x7fe1d40039b0 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.625+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0005ad0 con 0x7fe1d40039b0 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.625+0000 7fe1eea00640 1 -- 172.21.15.165:0/3867875504 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007070 con 0x7fe1d40039b0 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3867875504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 msgr2=0x7fe1d4003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1d4003dd0 secure :-1 s=READY pgs=2845 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000c240 tx=0x7fe1d4005830 comp rx=0 tx=0).stop 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3867875504 shutdown_connections 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3867875504 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1d4003dd0 unknown :-1 s=CLOSED pgs=2845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3867875504 >> 172.21.15.165:0/3867875504 conn(0x7fe1f00d0610 msgr2=0x7fe1f00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3867875504 shutdown_connections 2024-04-25T10:22:00.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.626+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3867875504 wait complete. 2024-04-25T10:22:00.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.627+0000 7fe1f4e00640 1 Processor -- start 2024-04-25T10:22:00.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.628+0000 7fe1f4e00640 1 -- start start 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.628+0000 7fe1f4e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1f01f3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1f4e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1f0160ac0 0x7fe1f01f2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1f4e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1f0155e00 0x7fe1f01562b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1f4e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f006cd20 con 0x7fe1d40039b0 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1f4e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f006ca40 con 0x7fe1f0155e00 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1f4e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00b5430 con 0x7fe1f0160ac0 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1f01f3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1f01f3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42440/0 (socket says 172.21.15.165:42440) 2024-04-25T10:22:00.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 -- 172.21.15.165:0/3981742436 learned_addr learned my addr 172.21.15.165:0/3981742436 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 -- 172.21.15.165:0/3981742436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1f0160ac0 msgr2=0x7fe1f01f2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1f0160ac0 0x7fe1f01f2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 -- 172.21.15.165:0/3981742436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1f0155e00 msgr2=0x7fe1f01562b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1f0155e00 0x7fe1f01562b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.629+0000 7fe1ef400640 1 -- 172.21.15.165:0/3981742436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f01f4440 con 0x7fe1d40039b0 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.630+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1f01f3f00 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0034040 tx=0x7fe1f00b62b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.630+0000 7fe1e7e00640 1 -- 172.21.15.165:0/3981742436 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0041070 con 0x7fe1d40039b0 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.630+0000 7fe1e7e00640 1 -- 172.21.15.165:0/3981742436 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0000ed0 con 0x7fe1d40039b0 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.630+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f0156d80 con 0x7fe1d40039b0 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.630+0000 7fe1e7e00640 1 -- 172.21.15.165:0/3981742436 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0007d10 con 0x7fe1d40039b0 2024-04-25T10:22:00.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.630+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f015fd00 con 0x7fe1d40039b0 2024-04-25T10:22:00.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.632+0000 7fe1e7e00640 1 -- 172.21.15.165:0/3981742436 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe1e0044070 con 0x7fe1d40039b0 2024-04-25T10:22:00.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.633+0000 7fe1e7e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe1cc06b500 0x7fe1cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.633+0000 7fe1e7e00640 1 -- 172.21.15.165:0/3981742436 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe1e00aad80 con 0x7fe1d40039b0 2024-04-25T10:22:00.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.634+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe1cc06b500 0x7fe1cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.634+0000 7fe1f9236740 1 Processor -- start 2024-04-25T10:22:00.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.634+0000 7fe1f9236740 1 -- start start 2024-04-25T10:22:00.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.634+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe1cc06b500 0x7fe1cc06d9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00cb900 tx=0x7fe1d80079d0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 0x5561d8f64970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f64eb0 0x5561d8f3cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f3b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1f9236740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d8ea0620 con 0x5561d8f3cb30 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d8e49ca0 con 0x5561d8f857a0 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d8e9c6c0 con 0x5561d8f64eb0 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 0x5561d8f64970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 0x5561d8f64970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42446/0 (socket says 172.21.15.165:42446) 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.635+0000 7fe1ef400640 1 -- 172.21.15.165:0/1797919475 learned_addr learned my addr 172.21.15.165:0/1797919475 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:00.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1ef400640 1 -- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f64eb0 msgr2=0x5561d8f3cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f64eb0 0x5561d8f3cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1efe00640 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f3b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1ef400640 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f64eb0 0x5561d8f3cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1ef400640 1 -- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 msgr2=0x5561d8f3b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1ef400640 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f3b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1ef400640 1 -- 172.21.15.165:0/1797919475 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d8e50f40 con 0x5561d8f3cb30 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f64eb0 0x5561d8f3cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1ef400640 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 0x5561d8f64970 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0081590 tx=0x7fe1e0004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158bba473ab6c6ec server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1dfe00640 1 -- 172.21.15.165:0/1797919475 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e007d470 con 0x5561d8f3cb30 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1dfe00640 1 -- 172.21.15.165:0/1797919475 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0081ac0 con 0x5561d8f3cb30 2024-04-25T10:22:00.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.636+0000 7fe1dfe00640 1 -- 172.21.15.165:0/1797919475 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0077620 con 0x5561d8f3cb30 2024-04-25T10:22:00.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 -- 172.21.15.165:0/1797919475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 msgr2=0x5561d8f64970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 0x5561d8f64970 secure :-1 s=READY pgs=2847 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0081590 tx=0x7fe1e0004e20 comp rx=0 tx=0).stop 2024-04-25T10:22:00.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 -- 172.21.15.165:0/1797919475 shutdown_connections 2024-04-25T10:22:00.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f3b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f64eb0 0x5561d8f3cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1797919475 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f3cb30 0x5561d8f64970 unknown :-1 s=CLOSED pgs=2847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 -- 172.21.15.165:0/1797919475 >> 172.21.15.165:0/1797919475 conn(0x5561d8f39d50 msgr2=0x5561d8f82730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 -- 172.21.15.165:0/1797919475 shutdown_connections 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.637+0000 7fe1f9236740 1 -- 172.21.15.165:0/1797919475 wait complete. 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.638+0000 7fe1f9236740 1 -- start start 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.638+0000 7fe1f9236740 1 Processor -- start 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8f68360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f55040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f84740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d8e9c6c0 con 0x5561d8f64eb0 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d8e3f1f0 con 0x5561d8f857a0 2024-04-25T10:22:00.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d8ea0620 con 0x5561d8f842c0 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.639+0000 7fe1f9236740 10 client.0 _ll_register_callbacks cb 0x5561d8e5cad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8f68360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8f68360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42456/0 (socket says 172.21.15.165:42456) 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 -- 172.21.15.165:0/3456644397 learned_addr learned my addr 172.21.15.165:0/3456644397 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1efe00640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f84740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f55040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 -- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 msgr2=0x5561d8f84740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f84740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 -- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 msgr2=0x5561d8f55040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f55040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 -- 172.21.15.165:0/3456644397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561d8e5ec50 con 0x5561d8f64eb0 2024-04-25T10:22:00.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1efe00640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f84740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1ef400640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8f68360 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fe1e002ae20 tx=0x7fe1e007c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9432905b367ef082 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.640+0000 7fe1dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1dcc00640 1 -- 172.21.15.165:0/3456644397 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0086030 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1dcc00640 1 -- 172.21.15.165:0/3456644397 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe1e007d150 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1e0081ac0 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1dcc00640 1 -- 172.21.15.165:0/3456644397 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1e008d020 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1f9236740 10 client.12009 fetch_fsmap learned FSMap version 146 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5561d8bb52d0 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1dcc00640 1 -- 172.21.15.165:0/3456644397 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0081ac0 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1dcc00640 1 -- 172.21.15.165:0/3456644397 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe1e004e060 con 0x5561d8f64eb0 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.641+0000 7fe1f9236740 10 client.12009 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1f9236740 10 client.12009 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1f9236740 20 client.12009 populate_metadata read hostname 'smithi165' 2024-04-25T10:22:00.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1f9236740 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f669a0 0x5561d8f9fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1f9236740 4 client.12009 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-25T10:22:00.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5561d8eadb60 con 0x5561d8f669a0 2024-04-25T10:22:00.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f669a0 0x5561d8f9fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f669a0 0x5561d8f9fcd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f74bdbd74d88bd9 server_cookie=7dd30d64dede5781 in_seq=0 out_seq=0 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.642+0000 7fe1dcc00640 10 client.12009 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1dcc00640 1 -- 172.21.15.165:0/3456644397 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+19073 (crc 0 0 0) 0x5561d8eadb60 con 0x5561d8f669a0 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1dcc00640 10 client.12009 handle_command_reply: tid=0 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1f9236740 1 client.12009 shutdown 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 msgr2=0x5561d8f68360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1f9236740 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8f68360 secure :-1 s=READY pgs=2848 cs=0 l=1 rev1=1 crypto rx=0x7fe1e002ae20 tx=0x7fe1e007c040 comp rx=0 tx=0).stop 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 shutdown_connections 2024-04-25T10:22:00.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.643+0000 7fe1f9236740 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f669a0 0x5561d8f9fcd0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f84740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f55040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1dcc00640 0 client.12009 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 1 --2- 172.21.15.165:0/3456644397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8f68360 unknown :-1 s=CLOSED pgs=2848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 >> 172.21.15.165:0/3456644397 conn(0x5561d8f3cb30 msgr2=0x5561d8f82530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 shutdown_connections 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 1 -- 172.21.15.165:0/3456644397 wait complete. 2024-04-25T10:22:00.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.644+0000 7fe1f9236740 20 client.12009 trim_cache size 0 max 16384 2024-04-25T10:22:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.661+0000 7fe1f9236740 1 Processor -- start 2024-04-25T10:22:00.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.661+0000 7fe1f9236740 1 -- start start 2024-04-25T10:22:00.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f64eb0 0x5561d8f63bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f81400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 0x5561d8f81940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1f9236740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01fff50 con 0x5561d8f857a0 2024-04-25T10:22:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01f76d0 con 0x5561d8f64eb0 2024-04-25T10:22:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01ff950 con 0x5561d8f842c0 2024-04-25T10:22:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 0x5561d8f81940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 0x5561d8f81940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42468/0 (socket says 172.21.15.165:42468) 2024-04-25T10:22:00.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1efe00640 1 -- 172.21.15.165:0/1812058770 learned_addr learned my addr 172.21.15.165:0/1812058770 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1ef400640 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f64eb0 0x5561d8f63bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.662+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f81400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1efe00640 1 -- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 msgr2=0x5561d8f81400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1efe00640 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f81400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1efe00640 1 -- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f64eb0 msgr2=0x5561d8f63bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1efe00640 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f64eb0 0x5561d8f63bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1efe00640 1 -- 172.21.15.165:0/1812058770 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561d8c7f6b0 con 0x5561d8f857a0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1efe00640 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 0x5561d8f81940 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00b9aa0 tx=0x7fe1e8008630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c66ff10ea4bccb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1df400640 1 -- 172.21.15.165:0/1812058770 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8003990 con 0x5561d8f857a0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1df400640 1 -- 172.21.15.165:0/1812058770 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e8003b70 con 0x5561d8f857a0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.663+0000 7fe1df400640 1 -- 172.21.15.165:0/1812058770 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8004020 con 0x5561d8f857a0 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 -- 172.21.15.165:0/1812058770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 msgr2=0x5561d8f81940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 0x5561d8f81940 secure :-1 s=READY pgs=2849 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00b9aa0 tx=0x7fe1e8008630 comp rx=0 tx=0).stop 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 -- 172.21.15.165:0/1812058770 shutdown_connections 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f857a0 0x5561d8f81940 unknown :-1 s=CLOSED pgs=2849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f81400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 --2- 172.21.15.165:0/1812058770 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f64eb0 0x5561d8f63bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 -- 172.21.15.165:0/1812058770 >> 172.21.15.165:0/1812058770 conn(0x5561d8f3cb30 msgr2=0x5561d8f3dbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 -- 172.21.15.165:0/1812058770 shutdown_connections 2024-04-25T10:22:00.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.664+0000 7fe1f9236740 1 -- 172.21.15.165:0/1812058770 wait complete. 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.665+0000 7fe1f9236740 1 -- start start 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.665+0000 7fe1f9236740 1 Processor -- start 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8fec270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f639b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f63ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d8bb52d0 con 0x5561d8f64eb0 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561d8e49ca0 con 0x5561d8f857a0 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561d8f93140 con 0x5561d8f842c0 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1f9236740 10 client.0 _ll_register_callbacks cb 0x7fe1e0078d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-25T10:22:00.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8fec270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1ef400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8fec270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42480/0 (socket says 172.21.15.165:42480) 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.666+0000 7fe1ef400640 1 -- 172.21.15.165:0/972699199 learned_addr learned my addr 172.21.15.165:0/972699199 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f639b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1ef400640 1 -- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 msgr2=0x5561d8f639b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1ef400640 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f639b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1ef400640 1 -- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 msgr2=0x5561d8f63ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1ef400640 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f63ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1ef400640 1 -- 172.21.15.165:0/972699199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561d8f554d0 con 0x5561d8f64eb0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1ef400640 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8fec270 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0001970 tx=0x7fe1e00791a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1eaf75dea9e2b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1dcc00640 10 client.0 ms_handle_connect on v2:172.21.15.165:3300/0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1dcc00640 1 -- 172.21.15.165:0/972699199 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0086070 con 0x5561d8f64eb0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1dcc00640 1 -- 172.21.15.165:0/972699199 <== mon.0 v2:172.21.15.165:3300/0 2 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe1e0075070 con 0x5561d8f64eb0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1e8003b70 con 0x5561d8f64eb0 2024-04-25T10:22:00.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1dcc00640 1 -- 172.21.15.165:0/972699199 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_get_version_reply(handle=1 version=146) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1e0081ac0 con 0x5561d8f64eb0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1f9236740 10 client.12015 fetch_fsmap learned FSMap version 146 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.667+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,fsmap=146,monmap=2+}) v3 -- 0x5561d8f554d0 con 0x5561d8f64eb0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1dcc00640 1 -- 172.21.15.165:0/972699199 <== mon.0 v2:172.21.15.165:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e8003b70 con 0x5561d8f64eb0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1dcc00640 1 -- 172.21.15.165:0/972699199 <== mon.0 v2:172.21.15.165:3300/0 5 ==== fsmap(e 146) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fe1e0075270 con 0x5561d8f64eb0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1f9236740 10 client.12015 fetch_fsmap finished waiting for FSMap version 146 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1f9236740 10 client.12015 resolve_mds: resolved name 'c' to daemon mds.c 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1f9236740 20 client.12015 populate_metadata read hostname 'smithi165' 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1f9236740 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f3b630 0x5561d8f3ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1f9236740 4 client.12015 mds_command: new command op to 11808 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 --> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5561d8f3a4b0 con 0x5561d8f3b630 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.668+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f3b630 0x5561d8f3ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1e6a00640 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f3b630 0x5561d8f3ba10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61e95576b430046e server_cookie=38f73e6e7c6b552c in_seq=0 out_seq=0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1dcc00640 10 client.12015 ms_handle_connect on v2:172.21.15.165:6835/2156983703 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1dcc00640 1 -- 172.21.15.165:0/972699199 <== mds.0 v2:172.21.15.165:6835/2156983703 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+335 (crc 0 0 0) 0x5561d8f3a4b0 con 0x5561d8f3b630 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1dcc00640 10 client.12015 handle_command_reply: tid=0 2024-04-25T10:22:00.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 client.12015 shutdown 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 msgr2=0x5561d8fec270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8fec270 secure :-1 s=READY pgs=2850 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0001970 tx=0x7fe1e00791a0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 shutdown_connections 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:6835/2156983703,v1:172.21.15.165:6837/2156983703] conn(0x5561d8f3b630 0x5561d8f3ba10 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x5561d8f857a0 0x5561d8f63ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x5561d8f842c0 0x5561d8f639b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 --2- 172.21.15.165:0/972699199 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x5561d8f64eb0 0x5561d8fec270 unknown :-1 s=CLOSED pgs=2850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1dcc00640 0 client.12015 ms_handle_reset on v2:172.21.15.165:6835/2156983703 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.669+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 >> 172.21.15.165:0/972699199 conn(0x5561d8f3cb30 msgr2=0x5561d8f67fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 shutdown_connections 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f9236740 1 -- 172.21.15.165:0/972699199 wait complete. 2024-04-25T10:22:00.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f9236740 20 client.12015 trim_cache size 0 max 16384 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "cluster_fsid": "55d22d87-b25b-4ccd-bc45-d797a973b3c0", 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "whoami": 0, 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "id": 11808, 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "want_state": "up:active", 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "state": "up:active", 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "fs_name": "cephfs", 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "rank_uptime": 67.696325665000003, 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "mdsmap_epoch": 146, 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch": 198, 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "osdmap_epoch_barrier": 198, 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout: "uptime": 69.647990269000005 2024-04-25T10:22:00.677 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:00.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe1cc06b500 msgr2=0x7fe1cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe1cc06b500 0x7fe1cc06d9c0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7fe1f00cb900 tx=0x7fe1d80079d0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 msgr2=0x7fe1f01f3f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1f01f3f00 secure :-1 s=READY pgs=2846 cs=0 l=1 rev1=1 crypto rx=0x7fe1e0034040 tx=0x7fe1f00b62b0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 shutdown_connections 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe1cc06b500 0x7fe1cc06d9c0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe1f0155e00 0x7fe1f01562b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe1f0160ac0 0x7fe1f01f2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 --2- 172.21.15.165:0/3981742436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1d40039b0 0x7fe1f01f3f00 unknown :-1 s=CLOSED pgs=2846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.670+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 >> 172.21.15.165:0/3981742436 conn(0x7fe1f00d0610 msgr2=0x7fe1f00b6da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.671+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 shutdown_connections 2024-04-25T10:22:00.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:00.671+0000 7fe1f4e00640 1 -- 172.21.15.165:0/3981742436 wait complete. 2024-04-25T10:22:00.691 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2024-04-25T10:22:00.691 DEBUG:teuthology.orchestra.run.smithi165:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:00.783 DEBUG:teuthology.orchestra.run.smithi172:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-04-25T10:22:00.890 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2024-04-25T10:22:00.890 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:22:00.890 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:00.948 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:00.948 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:22:00.948 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:01.004 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:01.004 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:22:01.004 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:01.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:01.063 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:22:01.063 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:01.122 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:01.122 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:01.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.328+0000 7ff046c00640 1 Processor -- start 2024-04-25T10:22:01.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.328+0000 7ff046c00640 1 -- start start 2024-04-25T10:22:01.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.329+0000 7ff046c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff04015dd00 0x7ff0400cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.329+0000 7ff046c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04015bf30 con 0x7ff040071700 2024-04-25T10:22:01.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.329+0000 7ff046c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff04015c210 con 0x7ff040144440 2024-04-25T10:22:01.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.329+0000 7ff046c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff04015c860 con 0x7ff04015dd00 2024-04-25T10:22:01.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.330+0000 7ff044e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff040071700 0x7ff0400cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36438/0 (socket says 172.21.15.165:36438) 2024-04-25T10:22:01.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.330+0000 7ff044e00640 1 -- 172.21.15.165:0/1253977831 learned_addr learned my addr 172.21.15.165:0/1253977831 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.330+0000 7ff046200640 1 --2- 172.21.15.165:0/1253977831 >> v2:172.21.15.172:3300/0 conn(0x7ff04015dd00 0x7ff0400cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.330+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 974678144 0 0) 0x7ff04015bf30 con 0x7ff040071700 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.330+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff028003520 con 0x7ff040071700 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4016602332 0 0) 0x7ff028003520 con 0x7ff040071700 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 >> v2:172.21.15.172:3300/0 conn(0x7ff04015dd00 msgr2=0x7ff0400cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 --2- 172.21.15.165:0/1253977831 >> v2:172.21.15.172:3300/0 conn(0x7ff04015dd00 0x7ff0400cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 >> v1:172.21.15.172:6789/0 conn(0x7ff040144440 legacy=0x7ff040070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04015c4f0 con 0x7ff040071700 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7ff02c001670 con 0x7ff040071700 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 >> v1:172.21.15.165:6789/0 conn(0x7ff040071700 legacy=0x7ff0400cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0280039b0 0x7ff028003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff028004800 0x7ff028004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280051c0 0x7ff02800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff04015c860 con 0x7ff0280039b0 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff02800c070 con 0x7ff028004800 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff02800c270 con 0x7ff0280051c0 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff045800640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0280039b0 0x7ff028003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff045800640 1 -- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280051c0 msgr2=0x7ff02800b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:01.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.331+0000 7ff045800640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280051c0 0x7ff02800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.332+0000 7ff044e00640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff028004800 0x7ff028004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.332+0000 7ff045800640 1 -- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff028004800 msgr2=0x7ff028004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:01.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.332+0000 7ff045800640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff028004800 0x7ff028004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.332+0000 7ff045800640 1 -- 172.21.15.165:0/1253977831 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff028003520 con 0x7ff0280039b0 2024-04-25T10:22:01.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.332+0000 7ff046200640 1 -- 172.21.15.165:0/1253977831 reap_dead start 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.333+0000 7ff045800640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0280039b0 0x7ff028003df0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7ff034001750 tx=0x7ff028005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88c6a23bae71af8a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.333+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff034001370 con 0x7ff0280039b0 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.333+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0340052d0 con 0x7ff0280039b0 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.333+0000 7ff03fe00640 1 -- 172.21.15.165:0/1253977831 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03402abc0 con 0x7ff0280039b0 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.333+0000 7ff046c00640 1 -- 172.21.15.165:0/1253977831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0280039b0 msgr2=0x7ff028003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.333+0000 7ff046c00640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0280039b0 0x7ff028003df0 secure :-1 s=READY pgs=2852 cs=0 l=1 rev1=1 crypto rx=0x7ff034001750 tx=0x7ff028005830 comp rx=0 tx=0).stop 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.334+0000 7ff046c00640 1 -- 172.21.15.165:0/1253977831 shutdown_connections 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.334+0000 7ff046c00640 1 --2- 172.21.15.165:0/1253977831 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0280039b0 0x7ff028003df0 unknown :-1 s=CLOSED pgs=2852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.334+0000 7ff046c00640 1 -- 172.21.15.165:0/1253977831 >> 172.21.15.165:0/1253977831 conn(0x7ff04006abe0 msgr2=0x7ff04006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:01.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.334+0000 7ff046c00640 1 -- 172.21.15.165:0/1253977831 shutdown_connections 2024-04-25T10:22:01.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.334+0000 7ff046c00640 1 -- 172.21.15.165:0/1253977831 wait complete. 2024-04-25T10:22:01.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.335+0000 7ff046c00640 1 Processor -- start 2024-04-25T10:22:01.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.336+0000 7ff046c00640 1 -- start start 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.336+0000 7ff046c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280039b0 0x7ff04015aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.336+0000 7ff046c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04013a870 0x7ff04015d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 0x7ff0400c7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040071170 con 0x7ff0400c73d0 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff040070ff0 con 0x7ff04013a870 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0400712f0 con 0x7ff0280039b0 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 0x7ff0400c7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 0x7ff0400c7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42494/0 (socket says 172.21.15.165:42494) 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 -- 172.21.15.165:0/728496665 learned_addr learned my addr 172.21.15.165:0/728496665 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff044e00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04013a870 0x7ff04015d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 -- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280039b0 msgr2=0x7ff04015aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280039b0 0x7ff04015aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 -- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04013a870 msgr2=0x7ff04015d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04013a870 0x7ff04015d3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff046200640 1 -- 172.21.15.165:0/728496665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04013ad50 con 0x7ff0400c73d0 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.337+0000 7ff044e00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04013a870 0x7ff04015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.338+0000 7ff046200640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 0x7ff0400c7880 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7ff04015e190 tx=0x7ff03800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.338+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff03800bd10 con 0x7ff0400c73d0 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.338+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff03801a070 con 0x7ff0400c73d0 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.338+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff038014640 con 0x7ff0400c73d0 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.338+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff04013b030 con 0x7ff0400c73d0 2024-04-25T10:22:01.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.338+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff04013b5f0 con 0x7ff0400c73d0 2024-04-25T10:22:01.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.339+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff04013b920 con 0x7ff0400c73d0 2024-04-25T10:22:01.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.340+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff038005d10 con 0x7ff0400c73d0 2024-04-25T10:22:01.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.341+0000 7ff03d600640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff02006b500 0x7ff02006d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.341+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff03808a7b0 con 0x7ff0400c73d0 2024-04-25T10:22:01.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.341+0000 7ff045800640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff02006b500 0x7ff02006d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.342+0000 7ff045800640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff02006b500 0x7ff02006d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff034032000 tx=0x7ff03402c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:01.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.346+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff038024020 con 0x7ff0400c73d0 2024-04-25T10:22:01.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.576+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff040073990 con 0x7ff0400c73d0 2024-04-25T10:22:01.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.577+0000 7ff03d600640 1 -- 172.21.15.165:0/728496665 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4965 (secure 0 0 0) 0x7ff0380592b0 con 0x7ff0400c73d0 2024-04-25T10:22:01.581 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:01.581 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:54.991856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:22:01.581 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 146 2024-04-25T10:22:01.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.580+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff02006b500 msgr2=0x7ff02006d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.580+0000 7ff046c00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff02006b500 0x7ff02006d9c0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7ff034032000 tx=0x7ff03402c000 comp rx=0 tx=0).stop 2024-04-25T10:22:01.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.580+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 msgr2=0x7ff0400c7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.580+0000 7ff046c00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 0x7ff0400c7880 secure :-1 s=READY pgs=2853 cs=0 l=1 rev1=1 crypto rx=0x7ff04015e190 tx=0x7ff03800b480 comp rx=0 tx=0).stop 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 shutdown_connections 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff02006b500 0x7ff02006d9c0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff0400c73d0 0x7ff0400c7880 unknown :-1 s=CLOSED pgs=2853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff04013a870 0x7ff04015d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 --2- 172.21.15.165:0/728496665 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff0280039b0 0x7ff04015aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 >> 172.21.15.165:0/728496665 conn(0x7ff04006abe0 msgr2=0x7ff040144820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 shutdown_connections 2024-04-25T10:22:01.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.581+0000 7ff046c00640 1 -- 172.21.15.165:0/728496665 wait complete. 2024-04-25T10:22:01.604 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:01.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.789+0000 7f5551800640 1 Processor -- start 2024-04-25T10:22:01.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.789+0000 7f5551800640 1 -- start start 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.790+0000 7f5551800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f554c0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.790+0000 7f5551800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c15dee0 con 0x7f554c06c450 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.790+0000 7f5551800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f554c15e160 con 0x7f554c06ba90 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.790+0000 7f5551800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c15e440 con 0x7f554c06bf20 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554aa00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f554c06c450 0x7f554c06c830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36450/0 (socket says 172.21.15.165:36450) 2024-04-25T10:22:01.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554aa00640 1 -- 172.21.15.165:0/3465297609 learned_addr learned my addr 172.21.15.165:0/3465297609 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960026106 0 0) 0x7f553c001490 con 0x7f554c06c450 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5530003520 con 0x7f554c06c450 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554b400640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f554c0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2927490820 0 0) 0x7f5530003520 con 0x7f554c06c450 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 msgr2=0x7f554c0b4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f554c0b4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 >> v1:172.21.15.172:6789/0 conn(0x7f554c06ba90 legacy=0x7f554c0b4850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c15e720 con 0x7f554c06c450 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f553c001bc0 con 0x7f554c06c450 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.791+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 >> v1:172.21.15.165:6789/0 conn(0x7f554c06c450 legacy=0x7f554c06c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554a000640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f55300039e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554a000640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004060 0x7f5530004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554a000640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5530004e60 0x7f553000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c15e440 con 0x7f5530004e60 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5530005680 con 0x7f5530004060 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f553000bf00 con 0x7f554c06bf20 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554b400640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5530004e60 0x7f553000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554b400640 1 -- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 msgr2=0x7f55300039e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554aa00640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f55300039e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f5542000640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004060 0x7f5530004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554b400640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f55300039e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554b400640 1 -- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004060 msgr2=0x7f5530004800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554b400640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004060 0x7f5530004800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554b400640 1 -- 172.21.15.165:0/3465297609 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5530003520 con 0x7f5530004e60 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f554aa00640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c06bf20 0x7f55300039e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.792+0000 7f5542000640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004060 0x7f5530004800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f554b400640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5530004e60 0x7f553000b9c0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f554400ba40 tx=0x7f5530005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccd29724f83b0248 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5544030070 con 0x7f5530004e60 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f554402ade0 con 0x7f5530004e60 2024-04-25T10:22:01.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f554a000640 1 -- 172.21.15.165:0/3465297609 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400b610 con 0x7f5530004e60 2024-04-25T10:22:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f5551800640 1 -- 172.21.15.165:0/3465297609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5530004e60 msgr2=0x7f553000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f5551800640 1 --2- 172.21.15.165:0/3465297609 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f5530004e60 0x7f553000b9c0 secure :-1 s=READY pgs=2855 cs=0 l=1 rev1=1 crypto rx=0x7f554400ba40 tx=0x7f5530005830 comp rx=0 tx=0).stop 2024-04-25T10:22:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.794+0000 7f554b400640 1 -- 172.21.15.165:0/3465297609 reap_dead start 2024-04-25T10:22:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.795+0000 7f5551800640 1 -- 172.21.15.165:0/3465297609 shutdown_connections 2024-04-25T10:22:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.795+0000 7f5551800640 1 -- 172.21.15.165:0/3465297609 >> 172.21.15.165:0/3465297609 conn(0x7f554c0b3450 msgr2=0x7f554c0b3830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:01.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.795+0000 7f5551800640 1 -- 172.21.15.165:0/3465297609 shutdown_connections 2024-04-25T10:22:01.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.795+0000 7f5551800640 1 -- 172.21.15.165:0/3465297609 wait complete. 2024-04-25T10:22:01.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.796+0000 7f5551800640 1 Processor -- start 2024-04-25T10:22:01.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.796+0000 7f5551800640 1 -- start start 2024-04-25T10:22:01.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.797+0000 7f5551800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004e60 0x7f554c147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.797+0000 7f5551800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c148580 0x7f554c0cd950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.797+0000 7f5551800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 0x7f554c0caad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.797+0000 7f5551800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0b10b0 con 0x7f554c148a00 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.797+0000 7f5551800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f554c0b0f30 con 0x7f5530004e60 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.797+0000 7f5551800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f554c071950 con 0x7f554c148580 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 0x7f554c0caad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 0x7f554c0caad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42514/0 (socket says 172.21.15.165:42514) 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 -- 172.21.15.165:0/2181936685 learned_addr learned my addr 172.21.15.165:0/2181936685 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 -- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c148580 msgr2=0x7f554c0cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c148580 0x7f554c0cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554aa00640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004e60 0x7f554c147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 -- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004e60 msgr2=0x7f554c147e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004e60 0x7f554c147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.798+0000 7f554b400640 1 -- 172.21.15.165:0/2181936685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554c0b13b0 con 0x7f554c148a00 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.799+0000 7f554b400640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 0x7f554c0caad0 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f554402a8d0 tx=0x7f55440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:01.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.799+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554400b610 con 0x7f554c148a00 2024-04-25T10:22:01.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.799+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5544045070 con 0x7f554c148a00 2024-04-25T10:22:01.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.799+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f554403f6b0 con 0x7f554c148a00 2024-04-25T10:22:01.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.799+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f554c0b1670 con 0x7f554c148a00 2024-04-25T10:22:01.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.799+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f554c0b1c30 con 0x7f554c148a00 2024-04-25T10:22:01.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.800+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f554c06de80 con 0x7f554c148a00 2024-04-25T10:22:01.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.801+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f554403f850 con 0x7f554c148a00 2024-04-25T10:22:01.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.802+0000 7f5543400640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f552806b4d0 0x7f552806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:01.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.802+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f55440aa960 con 0x7f554c148a00 2024-04-25T10:22:01.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.802+0000 7f554aa00640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f552806b4d0 0x7f552806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:01.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.803+0000 7f554aa00640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f552806b4d0 0x7f552806d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f553c007c40 tx=0x7f553c009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:01.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.806+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f55440794f0 con 0x7f554c148a00 2024-04-25T10:22:01.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.987+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f554c073990 con 0x7f554c148a00 2024-04-25T10:22:01.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.989+0000 7f5543400640 1 -- 172.21.15.165:0/2181936685 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17247 (secure 0 0 0) 0x7f5544078df0 con 0x7f554c148a00 2024-04-25T10:22:01.993 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:01.993 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":198,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:50.950400+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/3988146762":"2024-04-26T10:20:50.950335+0000","172.21.15.165:6835/3988146762":"2024-04-26T10:20:50.950335+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f552806b4d0 msgr2=0x7f552806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f552806b4d0 0x7f552806d990 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f553c007c40 tx=0x7f553c009040 comp rx=0 tx=0).stop 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 msgr2=0x7f554c0caad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 0x7f554c0caad0 secure :-1 s=READY pgs=2856 cs=0 l=1 rev1=1 crypto rx=0x7f554402a8d0 tx=0x7f55440046c0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 shutdown_connections 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f552806b4d0 0x7f552806d990 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f554c148a00 0x7f554c0caad0 unknown :-1 s=CLOSED pgs=2856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f554c148580 0x7f554c0cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 --2- 172.21.15.165:0/2181936685 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5530004e60 0x7f554c147e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:01.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.991+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 >> 172.21.15.165:0/2181936685 conn(0x7f554c0b3450 msgr2=0x7f554c15ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:01.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.992+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 shutdown_connections 2024-04-25T10:22:01.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:01.992+0000 7f5551800640 1 -- 172.21.15.165:0/2181936685 wait complete. 2024-04-25T10:22:02.012 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-25T10:01:52.195367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-25T10:20:35.357863+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '194', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'cephfs_data', 'create_time': '2024-04-25T10:20:36.605736+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-25T10:22:02.013 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-25T10:22:02.099 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:02.106 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:02.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.200+0000 7f848c400640 1 Processor -- start 2024-04-25T10:22:02.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.201+0000 7f848c400640 1 -- start start 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.202+0000 7f848c400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f84840b38d0 0x7f84840b3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.202+0000 7f848c400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848415e020 con 0x7f848406bd20 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.202+0000 7f848c400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f848406ac80 con 0x7f848406e360 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.202+0000 7f848c400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f848406b890 con 0x7f84840b38d0 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.202+0000 7f8489000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f84840b38d0 0x7f84840b3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.203+0000 7f8489000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f84840b38d0 0x7f84840b3d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42516/0 (socket says 172.21.15.165:42516) 2024-04-25T10:22:02.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.203+0000 7f8489000640 1 -- 172.21.15.165:0/3292814960 learned_addr learned my addr 172.21.15.165:0/3292814960 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:02.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.203+0000 7f8489000640 1 -- 172.21.15.165:0/3292814960 >> v1:172.21.15.172:6789/0 conn(0x7f848406e360 legacy=0x7f848406e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.203+0000 7f8489000640 1 -- 172.21.15.165:0/3292814960 >> v1:172.21.15.165:6789/0 conn(0x7f848406bd20 legacy=0x7f84840acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.203+0000 7f8489000640 1 -- 172.21.15.165:0/3292814960 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848406af60 con 0x7f84840b38d0 2024-04-25T10:22:02.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.204+0000 7f8489000640 1 --2- 172.21.15.165:0/3292814960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840b3d10 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f8470009870 tx=0x7f847002a3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc5c8210094f3b26 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:02.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.204+0000 7f8483e00640 1 -- 172.21.15.165:0/3292814960 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470037070 con 0x7f84840b38d0 2024-04-25T10:22:02.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.204+0000 7f8483e00640 1 -- 172.21.15.165:0/3292814960 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470032640 con 0x7f84840b38d0 2024-04-25T10:22:02.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.204+0000 7f8483e00640 1 -- 172.21.15.165:0/3292814960 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8470032a10 con 0x7f84840b38d0 2024-04-25T10:22:02.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 -- 172.21.15.165:0/3292814960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 msgr2=0x7f84840b3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 --2- 172.21.15.165:0/3292814960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840b3d10 secure :-1 s=READY pgs=2858 cs=0 l=1 rev1=1 crypto rx=0x7f8470009870 tx=0x7f847002a3c0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 -- 172.21.15.165:0/3292814960 shutdown_connections 2024-04-25T10:22:02.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 --2- 172.21.15.165:0/3292814960 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840b3d10 unknown :-1 s=CLOSED pgs=2858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 -- 172.21.15.165:0/3292814960 >> 172.21.15.165:0/3292814960 conn(0x7f84840cfec0 msgr2=0x7f84840d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:02.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 -- 172.21.15.165:0/3292814960 shutdown_connections 2024-04-25T10:22:02.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.205+0000 7f848c400640 1 -- 172.21.15.165:0/3292814960 wait complete. 2024-04-25T10:22:02.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.207+0000 7f848c400640 1 Processor -- start 2024-04-25T10:22:02.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.207+0000 7f848c400640 1 -- start start 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.207+0000 7f848c400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f848406bd20 0x7f84841f40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848c400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f848406e360 0x7f84840ae650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848c400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840af4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848c400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848406ee50 con 0x7f84840b38d0 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848c400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848406b520 con 0x7f848406bd20 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848c400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84840b2fd0 con 0x7f848406e360 2024-04-25T10:22:02.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840af4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840af4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42532/0 (socket says 172.21.15.165:42532) 2024-04-25T10:22:02.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 -- 172.21.15.165:0/1577504705 learned_addr learned my addr 172.21.15.165:0/1577504705 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f8489a00640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f848406bd20 0x7f84841f40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 -- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f848406e360 msgr2=0x7f84840ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f848406e360 0x7f84840ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 -- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f848406bd20 msgr2=0x7f84841f40e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f848406bd20 0x7f84841f40e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f848a400640 1 -- 172.21.15.165:0/1577504705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84840ad5f0 con 0x7f84840b38d0 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.208+0000 7f8489a00640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f848406bd20 0x7f84841f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.209+0000 7f848a400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840af4a0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c6b0 tx=0x7f847c00cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.209+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c003b70 con 0x7f84840b38d0 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.209+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c003d30 con 0x7f84840b38d0 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.209+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c005310 con 0x7f84840b38d0 2024-04-25T10:22:02.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.209+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84840ad890 con 0x7f84840b38d0 2024-04-25T10:22:02.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.209+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84840ade50 con 0x7f84840b38d0 2024-04-25T10:22:02.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.210+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8484073a00 con 0x7f84840b38d0 2024-04-25T10:22:02.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.211+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f847c0054b0 con 0x7f84840b38d0 2024-04-25T10:22:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.212+0000 7f8481600640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.212+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f847c08a390 con 0x7f84840b38d0 2024-04-25T10:22:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.213+0000 7f8489a00640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.213+0000 7f8489a00640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f846406b4d0 0x7f846406d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8478000f60 tx=0x7f8478009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:02.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.217+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f847c058f20 con 0x7f84840b38d0 2024-04-25T10:22:02.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.414+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f84840ad3d0 con 0x7f84840b38d0 2024-04-25T10:22:02.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.414+0000 7f8481600640 1 -- 172.21.15.165:0/1577504705 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7f847c058820 con 0x7f84840b38d0 2024-04-25T10:22:02.418 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 1 2024-04-25T10:22:02.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.417+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f846406b4d0 msgr2=0x7f846406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.417+0000 7f848c400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f846406b4d0 0x7f846406d990 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f8478000f60 tx=0x7f8478009040 comp rx=0 tx=0).stop 2024-04-25T10:22:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.417+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 msgr2=0x7f84840af4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.417+0000 7f848c400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840af4a0 secure :-1 s=READY pgs=2859 cs=0 l=1 rev1=1 crypto rx=0x7f847c00c6b0 tx=0x7f847c00cb70 comp rx=0 tx=0).stop 2024-04-25T10:22:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.417+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 shutdown_connections 2024-04-25T10:22:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.417+0000 7f848c400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f846406b4d0 0x7f846406d990 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.418+0000 7f848c400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f84840b38d0 0x7f84840af4a0 unknown :-1 s=CLOSED pgs=2859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.418+0000 7f848c400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f848406e360 0x7f84840ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.418+0000 7f848c400640 1 --2- 172.21.15.165:0/1577504705 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f848406bd20 0x7f84841f40e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.418+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 >> 172.21.15.165:0/1577504705 conn(0x7f84840cfec0 msgr2=0x7f84840cec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.418+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 shutdown_connections 2024-04-25T10:22:02.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.418+0000 7f848c400640 1 -- 172.21.15.165:0/1577504705 wait complete. 2024-04-25T10:22:02.440 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-25T10:22:02.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.688+0000 7fe127e00640 1 Processor -- start 2024-04-25T10:22:02.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.688+0000 7fe127e00640 1 -- start start 2024-04-25T10:22:02.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe127e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe128145140 0x7fe1280ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe127e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1280c5650 con 0x7fe1280c5200 2024-04-25T10:22:02.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe127e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe128145520 con 0x7fe1280ce950 2024-04-25T10:22:02.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe127e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1280ced90 con 0x7fe128145140 2024-04-25T10:22:02.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe127400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fe1280c5200 0x7fe12815bfb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36458/0 (socket says 172.21.15.165:36458) 2024-04-25T10:22:02.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe127400640 1 -- 172.21.15.165:0/996105878 learned_addr learned my addr 172.21.15.165:0/996105878 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.689+0000 7fe126a00640 1 --2- 172.21.15.165:0/996105878 >> v2:172.21.15.172:3300/0 conn(0x7fe128145140 0x7fe1280ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 196857266 0 0) 0x7fe1280c5650 con 0x7fe1280c5200 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe10c003520 con 0x7fe1280c5200 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3636283737 0 0) 0x7fe10c003520 con 0x7fe1280c5200 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 >> v2:172.21.15.172:3300/0 conn(0x7fe128145140 msgr2=0x7fe1280ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 --2- 172.21.15.165:0/996105878 >> v2:172.21.15.172:3300/0 conn(0x7fe128145140 0x7fe1280ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 >> v1:172.21.15.172:6789/0 conn(0x7fe1280ce950 legacy=0x7fe1280c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128071270 con 0x7fe1280c5200 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fe1200019f0 con 0x7fe1280c5200 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 >> v1:172.21.15.165:6789/0 conn(0x7fe1280c5200 legacy=0x7fe12815bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe10c003a00 0x7fe10c003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c004800 0x7fe10c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe10c005290 0x7fe10c00b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280ced90 con 0x7fe10c005290 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c00c1e0 con 0x7fe10c003a00 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe10c00c3e0 con 0x7fe10c004800 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.690+0000 7fe127400640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe10c005290 0x7fe10c00b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe126a00640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe10c003a00 0x7fe10c003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe127400640 1 -- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c004800 msgr2=0x7fe10c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe126000640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c004800 0x7fe10c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe127400640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c004800 0x7fe10c004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe127400640 1 -- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe10c003a00 msgr2=0x7fe10c003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe127400640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe10c003a00 0x7fe10c003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.691+0000 7fe127400640 1 -- 172.21.15.165:0/996105878 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe10c003520 con 0x7fe10c005290 2024-04-25T10:22:02.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.692+0000 7fe127400640 1 -- 172.21.15.165:0/996105878 reap_dead start 2024-04-25T10:22:02.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.692+0000 7fe127400640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe10c005290 0x7fe10c00b9c0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7fe12000a790 tx=0x7fe10c005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b7bdb4ad328861 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.692+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12000a3b0 con 0x7fe10c005290 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.692+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe12002f070 con 0x7fe10c005290 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.692+0000 7fe125600640 1 -- 172.21.15.165:0/996105878 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12002bbe0 con 0x7fe10c005290 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 -- 172.21.15.165:0/996105878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe10c005290 msgr2=0x7fe10c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe10c005290 0x7fe10c00b9c0 secure :-1 s=READY pgs=2861 cs=0 l=1 rev1=1 crypto rx=0x7fe12000a790 tx=0x7fe10c005830 comp rx=0 tx=0).stop 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 -- 172.21.15.165:0/996105878 shutdown_connections 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 --2- 172.21.15.165:0/996105878 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe10c005290 0x7fe10c00b9c0 unknown :-1 s=CLOSED pgs=2861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 -- 172.21.15.165:0/996105878 >> 172.21.15.165:0/996105878 conn(0x7fe128061cc0 msgr2=0x7fe12806abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 -- 172.21.15.165:0/996105878 shutdown_connections 2024-04-25T10:22:02.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.693+0000 7fe127e00640 1 -- 172.21.15.165:0/996105878 wait complete. 2024-04-25T10:22:02.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.695+0000 7fe127e00640 1 Processor -- start 2024-04-25T10:22:02.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.695+0000 7fe127e00640 1 -- start start 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c005290 0x7fe12813a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe12813eca0 0x7fe128140c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 0x7fe1281f1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe128071b10 con 0x7fe1281401d0 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe128071830 con 0x7fe12813eca0 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1280ce410 con 0x7fe10c005290 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 0x7fe1281f1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 0x7fe1281f1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42538/0 (socket says 172.21.15.165:42538) 2024-04-25T10:22:02.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.696+0000 7fe127400640 1 -- 172.21.15.165:0/2809084610 learned_addr learned my addr 172.21.15.165:0/2809084610 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:02.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe126a00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c005290 0x7fe12813a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe126000640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe12813eca0 0x7fe128140c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe127400640 1 -- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c005290 msgr2=0x7fe12813a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe127400640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c005290 0x7fe12813a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe127400640 1 -- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe12813eca0 msgr2=0x7fe128140c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe127400640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe12813eca0 0x7fe128140c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe127400640 1 -- 172.21.15.165:0/2809084610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe12813f130 con 0x7fe1281401d0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.697+0000 7fe127400640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 0x7fe1281f1850 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fe120000f80 tx=0x7fe1281456f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.698+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120001f00 con 0x7fe1281401d0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.698+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1200019e0 con 0x7fe1281401d0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.698+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe120034670 con 0x7fe1281401d0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.698+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12813f410 con 0x7fe1281401d0 2024-04-25T10:22:02.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.698+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12813f9d0 con 0x7fe1281401d0 2024-04-25T10:22:02.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.699+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe128073990 con 0x7fe1281401d0 2024-04-25T10:22:02.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.700+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe120034810 con 0x7fe1281401d0 2024-04-25T10:22:02.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.701+0000 7fe11ea00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe10806b4d0 0x7fe10806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:02.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.701+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7fe1200a9d10 con 0x7fe1281401d0 2024-04-25T10:22:02.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.701+0000 7fe126a00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe10806b4d0 0x7fe10806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:02.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.702+0000 7fe126a00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe10806b4d0 0x7fe10806d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe118006fd0 tx=0x7fe118008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:02.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.705+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe1200af020 con 0x7fe1281401d0 2024-04-25T10:22:02.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.902+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe1280b3a60 con 0x7fe1281401d0 2024-04-25T10:22:02.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.903+0000 7fe11ea00640 1 -- 172.21.15.165:0/2809084610 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe1200788a0 con 0x7fe1281401d0 2024-04-25T10:22:02.907 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 32 2024-04-25T10:22:02.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe10806b4d0 msgr2=0x7fe10806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe10806b4d0 0x7fe10806d990 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe118006fd0 tx=0x7fe118008040 comp rx=0 tx=0).stop 2024-04-25T10:22:02.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 msgr2=0x7fe1281f1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 0x7fe1281f1850 secure :-1 s=READY pgs=2862 cs=0 l=1 rev1=1 crypto rx=0x7fe120000f80 tx=0x7fe1281456f0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 shutdown_connections 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe10806b4d0 0x7fe10806d990 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe1281401d0 0x7fe1281f1850 unknown :-1 s=CLOSED pgs=2862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe12813eca0 0x7fe128140c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 --2- 172.21.15.165:0/2809084610 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe10c005290 0x7fe12813a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 >> 172.21.15.165:0/2809084610 conn(0x7fe128061cc0 msgr2=0x7fe1280b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 shutdown_connections 2024-04-25T10:22:02.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:02.906+0000 7fe127e00640 1 -- 172.21.15.165:0/2809084610 wait complete. 2024-04-25T10:22:02.927 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-25T10:22:03.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.125+0000 7f6c32600640 1 Processor -- start 2024-04-25T10:22:03.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.125+0000 7f6c32600640 1 -- start start 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c32600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6c2c15b3c0 0x7f6c2c0b0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c32600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f6c2c15b7a0 0x7f6c2c0cc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c32600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c2c0b0b80 con 0x7f6c2c0cdae0 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c32600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0cd650 con 0x7f6c2c15b3c0 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c32600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c2c0cc800 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6c2c15b3c0 0x7f6c2c0b0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6c2c15b3c0 0x7f6c2c0b0f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42548/0 (socket says 172.21.15.165:42548) 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 -- 172.21.15.165:0/2858762804 learned_addr learned my addr 172.21.15.165:0/2858762804 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c2be00640 1 --2- 172.21.15.165:0/2858762804 >> v2:172.21.15.172:3300/0 conn(0x7f6c2c15b7a0 0x7f6c2c0cc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 -- 172.21.15.165:0/2858762804 >> v2:172.21.15.172:3300/0 conn(0x7f6c2c15b7a0 msgr2=0x7f6c2c0cc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 --2- 172.21.15.165:0/2858762804 >> v2:172.21.15.172:3300/0 conn(0x7f6c2c15b7a0 0x7f6c2c0cc150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 -- 172.21.15.165:0/2858762804 >> v1:172.21.15.172:6789/0 conn(0x7f6c2c0cdae0 legacy=0x7f6c2c15e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:03.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.127+0000 7f6c31200640 1 -- 172.21.15.165:0/2858762804 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c0ccae0 con 0x7f6c2c15b3c0 2024-04-25T10:22:03.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.128+0000 7f6c31200640 1 --2- 172.21.15.165:0/2858762804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0b0f60 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c15f180 tx=0x7f6c1c02a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3f63b5385b8454 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:03.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.129+0000 7f6c2b400640 1 -- 172.21.15.165:0/2858762804 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c037070 con 0x7f6c2c15b3c0 2024-04-25T10:22:03.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.129+0000 7f6c2b400640 1 -- 172.21.15.165:0/2858762804 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c032440 con 0x7f6c2c15b3c0 2024-04-25T10:22:03.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.129+0000 7f6c2b400640 1 -- 172.21.15.165:0/2858762804 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c032810 con 0x7f6c2c15b3c0 2024-04-25T10:22:03.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.129+0000 7f6c32600640 1 -- 172.21.15.165:0/2858762804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b3c0 msgr2=0x7f6c2c0b0f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.129+0000 7f6c32600640 1 --2- 172.21.15.165:0/2858762804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0b0f60 secure :-1 s=READY pgs=2863 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c15f180 tx=0x7f6c1c02a1a0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.130+0000 7f6c32600640 1 -- 172.21.15.165:0/2858762804 shutdown_connections 2024-04-25T10:22:03.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.130+0000 7f6c32600640 1 --2- 172.21.15.165:0/2858762804 >> v2:172.21.15.172:3300/0 conn(0x7f6c2c15b7a0 0x7f6c2c0cc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.130+0000 7f6c32600640 1 --2- 172.21.15.165:0/2858762804 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0b0f60 unknown :-1 s=CLOSED pgs=2863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.130+0000 7f6c32600640 1 -- 172.21.15.165:0/2858762804 >> 172.21.15.165:0/2858762804 conn(0x7f6c2c061cc0 msgr2=0x7f6c2c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:03.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.130+0000 7f6c32600640 1 -- 172.21.15.165:0/2858762804 shutdown_connections 2024-04-25T10:22:03.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.130+0000 7f6c32600640 1 -- 172.21.15.165:0/2858762804 wait complete. 2024-04-25T10:22:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.132+0000 7f6c32600640 1 Processor -- start 2024-04-25T10:22:03.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.132+0000 7f6c32600640 1 -- start start 2024-04-25T10:22:03.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.133+0000 7f6c32600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c2c0cdae0 0x7f6c2c1f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.133+0000 7f6c32600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.133+0000 7f6c32600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 0x7f6c2c151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.133+0000 7f6c32600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c15e8a0 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.133+0000 7f6c32600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c15e720 con 0x7f6c2c15b3c0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.133+0000 7f6c32600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2c0cdf70 con 0x7f6c2c0cdae0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 0x7f6c2c151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 0x7f6c2c151100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42550/0 (socket says 172.21.15.165:42550) 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 -- 172.21.15.165:0/2709086688 learned_addr learned my addr 172.21.15.165:0/2709086688 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31200640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c2c0cdae0 0x7f6c2c1f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c2be00640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 -- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c2c0cdae0 msgr2=0x7f6c2c1f70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c2c0cdae0 0x7f6c2c1f70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 -- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c2c15b3c0 msgr2=0x7f6c2c0ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c31c00640 1 -- 172.21.15.165:0/2709086688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c1f8040 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.134+0000 7f6c2be00640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:03.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.135+0000 7f6c31c00640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 0x7f6c2c151100 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c780 tx=0x7f6c2400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.135+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c24017070 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.135+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c24003da0 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.135+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c1f77a0 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.135+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c240052b0 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.135+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2c1f7d60 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.136+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2c073990 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.137+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6c2401a070 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.138+0000 7f6c28c00640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c0806b4d0 0x7f6c0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.138+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6c24089af0 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.138+0000 7f6c31200640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c0806b4d0 0x7f6c0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.139+0000 7f6c31200640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c0806b4d0 0x7f6c0806d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c034000 tx=0x7f6c1c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:03.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.142+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6c24058710 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.340+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6c2c0aa9f0 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.341+0000 7f6c28c00640 1 -- 172.21.15.165:0/2709086688 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v198) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6c24058010 con 0x7f6c2c15b7a0 2024-04-25T10:22:03.345 INFO:teuthology.orchestra.run.smithi165.stdout:pg_num: 64 2024-04-25T10:22:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.343+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c0806b4d0 msgr2=0x7f6c0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.343+0000 7f6c32600640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c0806b4d0 0x7f6c0806d990 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c034000 tx=0x7f6c1c02e000 comp rx=0 tx=0).stop 2024-04-25T10:22:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.343+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 msgr2=0x7f6c2c151100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.343+0000 7f6c32600640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 0x7f6c2c151100 secure :-1 s=READY pgs=2864 cs=0 l=1 rev1=1 crypto rx=0x7f6c2400c780 tx=0x7f6c2400cc40 comp rx=0 tx=0).stop 2024-04-25T10:22:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 shutdown_connections 2024-04-25T10:22:03.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6c0806b4d0 0x7f6c0806d990 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6c2c15b7a0 0x7f6c2c151100 unknown :-1 s=CLOSED pgs=2864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6c2c15b3c0 0x7f6c2c0ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 --2- 172.21.15.165:0/2709086688 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6c2c0cdae0 0x7f6c2c1f70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 >> 172.21.15.165:0/2709086688 conn(0x7f6c2c061cc0 msgr2=0x7f6c2c0711d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 shutdown_connections 2024-04-25T10:22:03.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.344+0000 7f6c32600640 1 -- 172.21.15.165:0/2709086688 wait complete. 2024-04-25T10:22:03.366 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:03.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.563+0000 7f6feb200640 1 Processor -- start 2024-04-25T10:22:03.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.563+0000 7f6feb200640 1 -- start start 2024-04-25T10:22:03.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6feb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe406e870 0x7f6fe40cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6feb200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6fe40cc840 0x7f6fe415ab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6feb200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fe4144240 con 0x7f6fe4143e60 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6feb200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fe415e220 con 0x7f6fe40cc840 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6feb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe415e500 con 0x7f6fe406e870 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6fe40cc840 0x7f6fe415ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6fe40cc840 0x7f6fe415ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42562/0 (socket says 172.21.15.165:42562) 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 -- 172.21.15.165:0/2084640829 learned_addr learned my addr 172.21.15.165:0/2084640829 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fe9400640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe406e870 0x7f6fe40cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 -- 172.21.15.165:0/2084640829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe406e870 msgr2=0x7f6fe40cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe406e870 0x7f6fe40cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 -- 172.21.15.165:0/2084640829 >> v1:172.21.15.172:6789/0 conn(0x7f6fe4143e60 legacy=0x7f6fe406e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fea800640 1 -- 172.21.15.165:0/2084640829 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe415e7e0 con 0x7f6fe40cc840 2024-04-25T10:22:03.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.564+0000 7f6fe9400640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe406e870 0x7f6fe40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:03.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.566+0000 7f6fea800640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe40cc840 0x7f6fe415ab40 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009e50 tx=0x7f6fe002a6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a5e90c56382cd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:03.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.566+0000 7f6fe8a00640 1 -- 172.21.15.165:0/2084640829 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0036070 con 0x7f6fe40cc840 2024-04-25T10:22:03.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.567+0000 7f6fe8a00640 1 -- 172.21.15.165:0/2084640829 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe0031600 con 0x7f6fe40cc840 2024-04-25T10:22:03.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.567+0000 7f6fe8a00640 1 -- 172.21.15.165:0/2084640829 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe00319d0 con 0x7f6fe40cc840 2024-04-25T10:22:03.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.567+0000 7f6feb200640 1 -- 172.21.15.165:0/2084640829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe40cc840 msgr2=0x7f6fe415ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.567+0000 7f6feb200640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe40cc840 0x7f6fe415ab40 secure :-1 s=READY pgs=2865 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009e50 tx=0x7f6fe002a6c0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.568+0000 7f6feb200640 1 -- 172.21.15.165:0/2084640829 shutdown_connections 2024-04-25T10:22:03.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.568+0000 7f6feb200640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe40cc840 0x7f6fe415ab40 unknown :-1 s=CLOSED pgs=2865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.568+0000 7f6feb200640 1 --2- 172.21.15.165:0/2084640829 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe406e870 0x7f6fe40cc300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.568+0000 7f6feb200640 1 -- 172.21.15.165:0/2084640829 >> 172.21.15.165:0/2084640829 conn(0x7f6fe406abe0 msgr2=0x7f6fe406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:03.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.568+0000 7f6feb200640 1 -- 172.21.15.165:0/2084640829 shutdown_connections 2024-04-25T10:22:03.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.568+0000 7f6feb200640 1 -- 172.21.15.165:0/2084640829 wait complete. 2024-04-25T10:22:03.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.569+0000 7f6feb200640 1 Processor -- start 2024-04-25T10:22:03.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.569+0000 7f6feb200640 1 -- start start 2024-04-25T10:22:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.570+0000 7f6feb200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 0x7f6fe41f3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.570+0000 7f6feb200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe40cc840 0x7f6fe41f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.570+0000 7f6feb200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe4143e60 0x7f6fe41636c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.570+0000 7f6feb200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b2d30 con 0x7f6fe406e870 2024-04-25T10:22:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.570+0000 7f6feb200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40cccf0 con 0x7f6fe40cc840 2024-04-25T10:22:03.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.570+0000 7f6feb200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fe40b2eb0 con 0x7f6fe4143e60 2024-04-25T10:22:03.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 0x7f6fe41f3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 0x7f6fe41f3ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42576/0 (socket says 172.21.15.165:42576) 2024-04-25T10:22:03.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 -- 172.21.15.165:0/2127405114 learned_addr learned my addr 172.21.15.165:0/2127405114 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9400640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe40cc840 0x7f6fe41f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 -- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe4143e60 msgr2=0x7f6fe41636c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe4143e60 0x7f6fe41636c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 -- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe40cc840 msgr2=0x7f6fe41f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe40cc840 0x7f6fe41f6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 -- 172.21.15.165:0/2127405114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe413d5d0 con 0x7f6fe406e870 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.571+0000 7f6fe9e00640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 0x7f6fe41f3ae0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b3f0 tx=0x7f6fd800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.572+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8015020 con 0x7f6fe406e870 2024-04-25T10:22:03.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.572+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fd8011830 con 0x7f6fe406e870 2024-04-25T10:22:03.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.572+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fd8011c20 con 0x7f6fe406e870 2024-04-25T10:22:03.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.572+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fe413d8b0 con 0x7f6fe406e870 2024-04-25T10:22:03.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.572+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fe413de70 con 0x7f6fe406e870 2024-04-25T10:22:03.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.573+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fe4073990 con 0x7f6fe406e870 2024-04-25T10:22:03.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.574+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fd8018020 con 0x7f6fe406e870 2024-04-25T10:22:03.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.575+0000 7f6fde000640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6fc806b4d0 0x7f6fc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:03.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.575+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6fd808a130 con 0x7f6fe406e870 2024-04-25T10:22:03.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.575+0000 7f6fe9400640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6fc806b4d0 0x7f6fc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:03.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.576+0000 7f6fe9400640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6fc806b4d0 0x7f6fc806d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40cbc50 tx=0x7f6fd4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:03.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.580+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fd801f030 con 0x7f6fe406e870 2024-04-25T10:22:03.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.812+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fe40b2520 con 0x7f6fe406e870 2024-04-25T10:22:03.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.813+0000 7f6fde000640 1 -- 172.21.15.165:0/2127405114 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f6fd8058cc0 con 0x7f6fe406e870 2024-04-25T10:22:03.815 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:03.815 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:54.991856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:22:03.815 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 146 2024-04-25T10:22:03.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6fc806b4d0 msgr2=0x7f6fc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6fc806b4d0 0x7f6fc806d990 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f6fe40cbc50 tx=0x7f6fd4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:03.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 msgr2=0x7f6fe41f3ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:03.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 0x7f6fe41f3ae0 secure :-1 s=READY pgs=2866 cs=0 l=1 rev1=1 crypto rx=0x7f6fd800b3f0 tx=0x7f6fd800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 shutdown_connections 2024-04-25T10:22:03.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6fc806b4d0 0x7f6fc806d990 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fe4143e60 0x7f6fe41636c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fe40cc840 0x7f6fe41f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 --2- 172.21.15.165:0/2127405114 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fe406e870 0x7f6fe41f3ae0 unknown :-1 s=CLOSED pgs=2866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 >> 172.21.15.165:0/2127405114 conn(0x7f6fe406abe0 msgr2=0x7f6fe406d050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.815+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 shutdown_connections 2024-04-25T10:22:03.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:03.816+0000 7f6feb200640 1 -- 172.21.15.165:0/2127405114 wait complete. 2024-04-25T10:22:03.837 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:04.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.036+0000 7f85a8e00640 1 Processor -- start 2024-04-25T10:22:04.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.036+0000 7f85a8e00640 1 -- start start 2024-04-25T10:22:04.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f85a8e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40addc0 0x7f85a415de60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f85a8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85a40728a0 0x7f85a406bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f85a8e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a415e7d0 con 0x7f85a406c130 2024-04-25T10:22:04.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f85a8e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a406c5a0 con 0x7f85a40728a0 2024-04-25T10:22:04.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f85a8e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a415ed90 con 0x7f85a40addc0 2024-04-25T10:22:04.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f859be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85a40728a0 0x7f85a406bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f859be00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f85a40728a0 0x7f85a406bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42578/0 (socket says 172.21.15.165:42578) 2024-04-25T10:22:04.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.038+0000 7f859be00640 1 -- 172.21.15.165:0/1581640558 learned_addr learned my addr 172.21.15.165:0/1581640558 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.039+0000 7f85a3400640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40addc0 0x7f85a415de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.039+0000 7f859be00640 1 -- 172.21.15.165:0/1581640558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40addc0 msgr2=0x7f85a415de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.039+0000 7f859be00640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40addc0 0x7f85a415de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.039+0000 7f859be00640 1 -- 172.21.15.165:0/1581640558 >> v1:172.21.15.165:6789/0 conn(0x7f85a406c130 legacy=0x7f85a4161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.039+0000 7f859be00640 1 -- 172.21.15.165:0/1581640558 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a415eab0 con 0x7f85a40728a0 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.039+0000 7f85a3400640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40addc0 0x7f85a415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:04.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.040+0000 7f859be00640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a40728a0 0x7f85a406bbf0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f858c009870 tx=0x7f858c02a1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9aec6c59c3f92e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.040+0000 7f85a2a00640 1 -- 172.21.15.165:0/1581640558 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c037070 con 0x7f85a40728a0 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.040+0000 7f85a2a00640 1 -- 172.21.15.165:0/1581640558 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858c032440 con 0x7f85a40728a0 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.040+0000 7f85a2a00640 1 -- 172.21.15.165:0/1581640558 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858c032810 con 0x7f85a40728a0 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 -- 172.21.15.165:0/1581640558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a40728a0 msgr2=0x7f85a406bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a40728a0 0x7f85a406bbf0 secure :-1 s=READY pgs=2868 cs=0 l=1 rev1=1 crypto rx=0x7f858c009870 tx=0x7f858c02a1c0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 -- 172.21.15.165:0/1581640558 shutdown_connections 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a40728a0 0x7f85a406bbf0 unknown :-1 s=CLOSED pgs=2868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 --2- 172.21.15.165:0/1581640558 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40addc0 0x7f85a415de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 -- 172.21.15.165:0/1581640558 >> 172.21.15.165:0/1581640558 conn(0x7f85a40718d0 msgr2=0x7f85a4071ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 -- 172.21.15.165:0/1581640558 shutdown_connections 2024-04-25T10:22:04.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.041+0000 7f85a8e00640 1 -- 172.21.15.165:0/1581640558 wait complete. 2024-04-25T10:22:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.043+0000 7f85a8e00640 1 Processor -- start 2024-04-25T10:22:04.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.043+0000 7f85a8e00640 1 -- start start 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.044+0000 7f85a8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 0x7f85a4149740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.044+0000 7f85a8e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40728a0 0x7f85a414e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.044+0000 7f85a8e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a40addc0 0x7f85a414c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.044+0000 7f85a8e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40ae1a0 con 0x7f85a406c130 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.044+0000 7f85a8e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40b3040 con 0x7f85a40addc0 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.044+0000 7f85a8e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a415e3a0 con 0x7f85a40728a0 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 0x7f85a4149740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 0x7f85a4149740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42592/0 (socket says 172.21.15.165:42592) 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 -- 172.21.15.165:0/2389426888 learned_addr learned my addr 172.21.15.165:0/2389426888 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f859be00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40728a0 0x7f85a414e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a40addc0 0x7f85a414c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 -- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40728a0 msgr2=0x7f85a414e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40728a0 0x7f85a414e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 -- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a40addc0 msgr2=0x7f85a414c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a40addc0 0x7f85a414c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.045+0000 7f85a3400640 1 -- 172.21.15.165:0/2389426888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a40af720 con 0x7f85a406c130 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.046+0000 7f85a3400640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 0x7f85a4149740 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f859400c7f0 tx=0x7f859400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.046+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594015070 con 0x7f85a406c130 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.046+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8594007820 con 0x7f85a406c130 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.046+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a40afa00 con 0x7f85a406c130 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.046+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8594007bd0 con 0x7f85a406c130 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.046+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a40affc0 con 0x7f85a406c130 2024-04-25T10:22:04.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.047+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a4073990 con 0x7f85a406c130 2024-04-25T10:22:04.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.049+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8594018070 con 0x7f85a406c130 2024-04-25T10:22:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.049+0000 7f859b400640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.050+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f859408a660 con 0x7f85a406c130 2024-04-25T10:22:04.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.050+0000 7f859be00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.051+0000 7f859be00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f858c034000 tx=0x7f858c02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.054+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f85940591f0 con 0x7f85a406c130 2024-04-25T10:22:04.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.251+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f85a4061c50 con 0x7f85a406c130 2024-04-25T10:22:04.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.254+0000 7f859b400640 1 -- 172.21.15.165:0/2389426888 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17247 (secure 0 0 0) 0x7f8594058af0 con 0x7f85a406c130 2024-04-25T10:22:04.257 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:04.257 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":198,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:50.950400+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/3988146762":"2024-04-26T10:20:50.950335+0000","172.21.15.165:6835/3988146762":"2024-04-26T10:20:50.950335+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f858006b4d0 msgr2=0x7f858006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f858006b4d0 0x7f858006d990 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f858c034000 tx=0x7f858c02e000 comp rx=0 tx=0).stop 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 msgr2=0x7f85a4149740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 0x7f85a4149740 secure :-1 s=READY pgs=2869 cs=0 l=1 rev1=1 crypto rx=0x7f859400c7f0 tx=0x7f859400ccb0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 shutdown_connections 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f858006b4d0 0x7f858006d990 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f85a40addc0 0x7f85a414c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f85a40728a0 0x7f85a414e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 --2- 172.21.15.165:0/2389426888 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f85a406c130 0x7f85a4149740 unknown :-1 s=CLOSED pgs=2869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 >> 172.21.15.165:0/2389426888 conn(0x7f85a40718d0 msgr2=0x7f85a41613e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:04.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 shutdown_connections 2024-04-25T10:22:04.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.257+0000 7f85a8e00640 1 -- 172.21.15.165:0/2389426888 wait complete. 2024-04-25T10:22:04.279 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:22:04.279 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:04.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.482+0000 7ff60be00640 1 Processor -- start 2024-04-25T10:22:04.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.482+0000 7ff60be00640 1 -- start start 2024-04-25T10:22:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60be00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7ff60c15e520 0x7ff60c15e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c15f040 0x7ff60c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60be00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff60c0ae590 con 0x7ff60c0ae0d0 2024-04-25T10:22:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60be00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff60c0aee30 con 0x7ff60c15e520 2024-04-25T10:22:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c0ae870 con 0x7ff60c15f040 2024-04-25T10:22:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60b400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7ff60c0ae0d0 0x7ff60c070ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36466/0 (socket says 172.21.15.165:36466) 2024-04-25T10:22:04.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60b400640 1 -- 172.21.15.165:0/1165189276 learned_addr learned my addr 172.21.15.165:0/1165189276 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60a000640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c15f040 0x7ff60c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.484+0000 7ff60aa00640 1 --2- 172.21.15.165:0/1165189276 >> v2:172.21.15.172:3300/0 conn(0x7ff60c15e520 0x7ff60c15e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff609600640 1 -- 172.21.15.165:0/1165189276 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2838887452 0 0) 0x7ff60c0ae590 con 0x7ff60c0ae0d0 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff609600640 1 -- 172.21.15.165:0/1165189276 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0003520 con 0x7ff60c0ae0d0 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff609600640 1 -- 172.21.15.165:0/1165189276 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1870114423 0 0) 0x7ff5f0003520 con 0x7ff60c0ae0d0 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff60aa00640 1 -- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c15f040 msgr2=0x7ff60c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff60aa00640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c15f040 0x7ff60c0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff60aa00640 1 -- 172.21.15.165:0/1165189276 >> v1:172.21.15.165:6789/0 conn(0x7ff60c0ae0d0 legacy=0x7ff60c070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff60aa00640 1 -- 172.21.15.165:0/1165189276 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c0aeb50 con 0x7ff60c15e520 2024-04-25T10:22:04.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.485+0000 7ff60a000640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c15f040 0x7ff60c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-25T10:22:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.486+0000 7ff60aa00640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c15e900 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097a0 tx=0x7ff5fc02a110 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bc0a7b62382de00c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.487+0000 7ff609600640 1 -- 172.21.15.165:0/1165189276 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc037070 con 0x7ff60c15e520 2024-04-25T10:22:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.487+0000 7ff609600640 1 -- 172.21.15.165:0/1165189276 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc02aba0 con 0x7ff60c15e520 2024-04-25T10:22:04.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.487+0000 7ff609600640 1 -- 172.21.15.165:0/1165189276 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040019f0 con 0x7ff60c15e520 2024-04-25T10:22:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 -- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 msgr2=0x7ff60c15e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c15e900 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc0097a0 tx=0x7ff5fc02a110 comp rx=0 tx=0).stop 2024-04-25T10:22:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 -- 172.21.15.165:0/1165189276 shutdown_connections 2024-04-25T10:22:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c15f040 0x7ff60c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 --2- 172.21.15.165:0/1165189276 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c15e900 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 -- 172.21.15.165:0/1165189276 >> 172.21.15.165:0/1165189276 conn(0x7ff60c06abe0 msgr2=0x7ff60c06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 -- 172.21.15.165:0/1165189276 shutdown_connections 2024-04-25T10:22:04.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.488+0000 7ff60be00640 1 -- 172.21.15.165:0/1165189276 wait complete. 2024-04-25T10:22:04.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.490+0000 7ff60be00640 1 Processor -- start 2024-04-25T10:22:04.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.490+0000 7ff60be00640 1 -- start start 2024-04-25T10:22:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60be00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c0ae0d0 0x7ff60c148eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60be00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c0ce6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60be00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 0x7ff60c0b32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60be00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15f620 con 0x7ff60c15f040 2024-04-25T10:22:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60be00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c15f4a0 con 0x7ff60c15e520 2024-04-25T10:22:04.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60be00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff60c071700 con 0x7ff60c0ae0d0 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 0x7ff60c0b32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 0x7ff60c0b32c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42594/0 (socket says 172.21.15.165:42594) 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60b400640 1 -- 172.21.15.165:0/3338418333 learned_addr learned my addr 172.21.15.165:0/3338418333 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60aa00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c0ae0d0 0x7ff60c148eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.492+0000 7ff60a000640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c0ce6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60b400640 1 -- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c0ae0d0 msgr2=0x7ff60c148eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60b400640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c0ae0d0 0x7ff60c148eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60b400640 1 -- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 msgr2=0x7ff60c0ce6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60b400640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c0ce6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60b400640 1 -- 172.21.15.165:0/3338418333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff60c1f3de0 con 0x7ff60c15f040 2024-04-25T10:22:04.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60b400640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 0x7ff60c0b32c0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ff604001b60 tx=0x7ff60400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff604017070 con 0x7ff60c15f040 2024-04-25T10:22:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff604012400 con 0x7ff60c15f040 2024-04-25T10:22:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6040127b0 con 0x7ff60c15f040 2024-04-25T10:22:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.493+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff60c1f4080 con 0x7ff60c15f040 2024-04-25T10:22:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.494+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff60c0cbcd0 con 0x7ff60c15f040 2024-04-25T10:22:04.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.495+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff60c1f45a0 con 0x7ff60c15f040 2024-04-25T10:22:04.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.496+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff60401a070 con 0x7ff60c15f040 2024-04-25T10:22:04.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.497+0000 7ff603400640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5ec06b4d0 0x7ff5ec06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.497+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7ff60408a2f0 con 0x7ff60c15f040 2024-04-25T10:22:04.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.497+0000 7ff60aa00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5ec06b4d0 0x7ff5ec06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.498+0000 7ff60aa00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5ec06b4d0 0x7ff5ec06d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc034000 tx=0x7ff5fc02e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.501+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7ff604058830 con 0x7ff60c15f040 2024-04-25T10:22:04.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.740+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff60c073990 con 0x7ff60c15f040 2024-04-25T10:22:04.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.741+0000 7ff603400640 1 -- 172.21.15.165:0/3338418333 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4965 (secure 0 0 0) 0x7ff60405c8d0 con 0x7ff60c15f040 2024-04-25T10:22:04.744 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:04.744 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:54.991856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:22:04.744 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 146 2024-04-25T10:22:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5ec06b4d0 msgr2=0x7ff5ec06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5ec06b4d0 0x7ff5ec06d990 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc034000 tx=0x7ff5fc02e000 comp rx=0 tx=0).stop 2024-04-25T10:22:04.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 msgr2=0x7ff60c0b32c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 0x7ff60c0b32c0 secure :-1 s=READY pgs=2871 cs=0 l=1 rev1=1 crypto rx=0x7ff604001b60 tx=0x7ff60400a430 comp rx=0 tx=0).stop 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 shutdown_connections 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff5ec06b4d0 0x7ff5ec06d990 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff60c15f040 0x7ff60c0b32c0 unknown :-1 s=CLOSED pgs=2871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff60c15e520 0x7ff60c0ce6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 --2- 172.21.15.165:0/3338418333 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff60c0ae0d0 0x7ff60c148eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 >> 172.21.15.165:0/3338418333 conn(0x7ff60c06abe0 msgr2=0x7ff60c0af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 shutdown_connections 2024-04-25T10:22:04.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.744+0000 7ff60be00640 1 -- 172.21.15.165:0/3338418333 wait complete. 2024-04-25T10:22:04.766 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.964+0000 7f600da00640 1 Processor -- start 2024-04-25T10:22:04.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.964+0000 7f600da00640 1 -- start start 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f600da00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6008144750 0x7f600806e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f600da00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f600806eb10 0x7f60080ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f600da00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6008144b30 con 0x7f60080cee60 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f600da00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080712d0 con 0x7f6008144750 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f600da00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080715b0 con 0x7f600806eb10 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f6006a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6008144750 0x7f600806e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f6006a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6008144750 0x7f600806e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42596/0 (socket says 172.21.15.165:42596) 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.966+0000 7f6006a00640 1 -- 172.21.15.165:0/1144057436 learned_addr learned my addr 172.21.15.165:0/1144057436 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:04.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.967+0000 7f6006000640 1 --2- 172.21.15.165:0/1144057436 >> v2:172.21.15.172:3300/0 conn(0x7f600806eb10 0x7f60080ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.967+0000 7f6006a00640 1 -- 172.21.15.165:0/1144057436 >> v2:172.21.15.172:3300/0 conn(0x7f600806eb10 msgr2=0x7f60080ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.967+0000 7f6006a00640 1 --2- 172.21.15.165:0/1144057436 >> v2:172.21.15.172:3300/0 conn(0x7f600806eb10 0x7f60080ce920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.967+0000 7f6006a00640 1 -- 172.21.15.165:0/1144057436 >> v1:172.21.15.172:6789/0 conn(0x7f60080cee60 legacy=0x7f600815f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.967+0000 7f6006a00640 1 -- 172.21.15.165:0/1144057436 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6008071890 con 0x7f6008144750 2024-04-25T10:22:04.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.968+0000 7f6006a00640 1 --2- 172.21.15.165:0/1144057436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6008144750 0x7f600806e5d0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f60081606b0 tx=0x7f5ff802a230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb1508afd7c3d7d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.969+0000 7f6005600640 1 -- 172.21.15.165:0/1144057436 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037070 con 0x7f6008144750 2024-04-25T10:22:04.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.969+0000 7f6005600640 1 -- 172.21.15.165:0/1144057436 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff8032440 con 0x7f6008144750 2024-04-25T10:22:04.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.969+0000 7f6005600640 1 -- 172.21.15.165:0/1144057436 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8032810 con 0x7f6008144750 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 -- 172.21.15.165:0/1144057436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6008144750 msgr2=0x7f600806e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 --2- 172.21.15.165:0/1144057436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6008144750 0x7f600806e5d0 secure :-1 s=READY pgs=2872 cs=0 l=1 rev1=1 crypto rx=0x7f60081606b0 tx=0x7f5ff802a230 comp rx=0 tx=0).stop 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 -- 172.21.15.165:0/1144057436 shutdown_connections 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 --2- 172.21.15.165:0/1144057436 >> v2:172.21.15.172:3300/0 conn(0x7f600806eb10 0x7f60080ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 --2- 172.21.15.165:0/1144057436 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6008144750 0x7f600806e5d0 unknown :-1 s=CLOSED pgs=2872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 -- 172.21.15.165:0/1144057436 >> 172.21.15.165:0/1144057436 conn(0x7f600806abe0 msgr2=0x7f600806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 -- 172.21.15.165:0/1144057436 shutdown_connections 2024-04-25T10:22:04.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.970+0000 7f600da00640 1 -- 172.21.15.165:0/1144057436 wait complete. 2024-04-25T10:22:04.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.972+0000 7f600da00640 1 Processor -- start 2024-04-25T10:22:04.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.972+0000 7f600da00640 1 -- start start 2024-04-25T10:22:04.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.973+0000 7f600da00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 0x7f600813ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.973+0000 7f600da00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cee60 0x7f600814d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.973+0000 7f600da00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6008144750 0x7f600813b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.973+0000 7f600da00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6008145450 con 0x7f600806eb10 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.973+0000 7f600da00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600806f020 con 0x7f60080cee60 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.973+0000 7f600da00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60081455d0 con 0x7f6008144750 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 0x7f600813ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 0x7f600813ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42608/0 (socket says 172.21.15.165:42608) 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 -- 172.21.15.165:0/3732231465 learned_addr learned my addr 172.21.15.165:0/3732231465 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 -- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6008144750 msgr2=0x7f600813b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6008144750 0x7f600813b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 -- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cee60 msgr2=0x7f600814d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006000640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cee60 0x7f600814d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cee60 0x7f600814d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:04.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 -- 172.21.15.165:0/3732231465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600813b740 con 0x7f600806eb10 2024-04-25T10:22:04.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.974+0000 7f6006a00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 0x7f600813ccb0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f5ff802a270 tx=0x7f5ff80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.975+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8041020 con 0x7f600806eb10 2024-04-25T10:22:04.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.975+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff80306c0 con 0x7f600806eb10 2024-04-25T10:22:04.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.975+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600813ba10 con 0x7f600806eb10 2024-04-25T10:22:04.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.975+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff8037040 con 0x7f600806eb10 2024-04-25T10:22:04.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.975+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600813bfd0 con 0x7f600806eb10 2024-04-25T10:22:04.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.976+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6008073990 con 0x7f600806eb10 2024-04-25T10:22:04.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.977+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f5ff8044020 con 0x7f600806eb10 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.978+0000 7f5ffea00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe806b520 0x7f5fe806d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.978+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f5ff80aac80 con 0x7f600806eb10 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.979+0000 7f6006000640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe806b520 0x7f5fe806d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:04.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.979+0000 7f6006000640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe806b520 0x7f5fe806d9e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4006fd0 tx=0x7f5ff4008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:04.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:04.983+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f5ff804b030 con 0x7f600806eb10 2024-04-25T10:22:05.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.219+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f60081e92a0 con 0x7f600806eb10 2024-04-25T10:22:05.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.220+0000 7f5ffea00640 1 -- 172.21.15.165:0/3732231465 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 146 v146) v1 ==== 95+0+4965 (secure 0 0 0) 0x7f5ff8079890 con 0x7f600806eb10 2024-04-25T10:22:05.222 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:05.223 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":146,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134}],"filesystems":[{"mdsmap":{"epoch":146,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-25T10:20:37.587736+0000","modified":"2024-04-25T10:20:54.991856+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":198,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":11808},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11808":{"gid":11808,"name":"c","rank":0,"incarnation":143,"state":"up:active","state_seq":4,"addr":"172.21.15.165:6837/2156983703","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":2156983703},{"type":"v1","addr":"172.21.15.165:6837","nonce":2156983703}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[30],"metadata_pool":29,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2024-04-25T10:22:05.223 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 146 2024-04-25T10:22:05.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.222+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe806b520 msgr2=0x7f5fe806d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.222+0000 7f600da00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe806b520 0x7f5fe806d9e0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f5ff4006fd0 tx=0x7f5ff4008040 comp rx=0 tx=0).stop 2024-04-25T10:22:05.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.222+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 msgr2=0x7f600813ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.222+0000 7f600da00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 0x7f600813ccb0 secure :-1 s=READY pgs=2873 cs=0 l=1 rev1=1 crypto rx=0x7f5ff802a270 tx=0x7f5ff80046c0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 shutdown_connections 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f5fe806b520 0x7f5fe806d9e0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6008144750 0x7f600813b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f60080cee60 0x7f600814d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 --2- 172.21.15.165:0/3732231465 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f600806eb10 0x7f600813ccb0 unknown :-1 s=CLOSED pgs=2873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 >> 172.21.15.165:0/3732231465 conn(0x7f600806abe0 msgr2=0x7f60080b1520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 shutdown_connections 2024-04-25T10:22:05.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.223+0000 7f600da00640 1 -- 172.21.15.165:0/3732231465 wait complete. 2024-04-25T10:22:05.245 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:05.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.442+0000 7f274e400640 1 Processor -- start 2024-04-25T10:22:05.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.443+0000 7f274e400640 1 -- start start 2024-04-25T10:22:05.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f274e400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f274815df50 0x7f27480cb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f274e400640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2748071650 con 0x7f274815db70 2024-04-25T10:22:05.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f274e400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27480cf230 con 0x7f27480cfc20 2024-04-25T10:22:05.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f274e400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27480cbd80 con 0x7f274815df50 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f2747400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f274815db70 0x7f27480cf510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36474/0 (socket says 172.21.15.165:36474) 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f2747400640 1 -- 172.21.15.165:0/3582675155 learned_addr learned my addr 172.21.15.165:0/3582675155 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.444+0000 7f2747e00640 1 --2- 172.21.15.165:0/3582675155 >> v2:172.21.15.165:3300/0 conn(0x7f274815df50 0x7f27480cb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.445+0000 7f2746a00640 1 -- 172.21.15.165:0/3582675155 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793824260 0 0) 0x7f2738001110 con 0x7f274815db70 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.445+0000 7f2746a00640 1 -- 172.21.15.165:0/3582675155 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f272c003520 con 0x7f274815db70 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.445+0000 7f2747e00640 1 -- 172.21.15.165:0/3582675155 >> v1:172.21.15.172:6789/0 conn(0x7f27480cfc20 legacy=0x7f274806d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.445+0000 7f2747e00640 1 -- 172.21.15.165:0/3582675155 >> v1:172.21.15.165:6789/0 conn(0x7f274815db70 legacy=0x7f27480cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.445+0000 7f2747e00640 1 -- 172.21.15.165:0/3582675155 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748161b60 con 0x7f274815df50 2024-04-25T10:22:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.446+0000 7f2747e00640 1 --2- 172.21.15.165:0/3582675155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f27480cb840 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f2740009d90 tx=0x7f274002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946567ef4981fd7e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.446+0000 7f2746a00640 1 -- 172.21.15.165:0/3582675155 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740007740 con 0x7f274815df50 2024-04-25T10:22:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.446+0000 7f2746a00640 1 -- 172.21.15.165:0/3582675155 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27400078e0 con 0x7f274815df50 2024-04-25T10:22:05.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.446+0000 7f2746a00640 1 -- 172.21.15.165:0/3582675155 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740007cb0 con 0x7f274815df50 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 -- 172.21.15.165:0/3582675155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 msgr2=0x7f27480cb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 --2- 172.21.15.165:0/3582675155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f27480cb840 secure :-1 s=READY pgs=2875 cs=0 l=1 rev1=1 crypto rx=0x7f2740009d90 tx=0x7f274002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 -- 172.21.15.165:0/3582675155 shutdown_connections 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 --2- 172.21.15.165:0/3582675155 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f27480cb840 unknown :-1 s=CLOSED pgs=2875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 -- 172.21.15.165:0/3582675155 >> 172.21.15.165:0/3582675155 conn(0x7f274806abe0 msgr2=0x7f274806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 -- 172.21.15.165:0/3582675155 shutdown_connections 2024-04-25T10:22:05.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.447+0000 7f274e400640 1 -- 172.21.15.165:0/3582675155 wait complete. 2024-04-25T10:22:05.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.449+0000 7f274e400640 1 Processor -- start 2024-04-25T10:22:05.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.449+0000 7f274e400640 1 -- start start 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f274e400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27480cfc20 0x7f27481512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f274e400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f274815db70 0x7f27481f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f274e400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f2748151af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f274e400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748162120 con 0x7f274815df50 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f274e400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2748161910 con 0x7f274815db70 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f274e400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2748162640 con 0x7f27480cfc20 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f2747e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f2748151af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f2747e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f2748151af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42616/0 (socket says 172.21.15.165:42616) 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f2747e00640 1 -- 172.21.15.165:0/3274221839 learned_addr learned my addr 172.21.15.165:0/3274221839 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.450+0000 7f273ea00640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f274815db70 0x7f27481f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27480cfc20 0x7f27481512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747e00640 1 -- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27480cfc20 msgr2=0x7f27481512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747e00640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27480cfc20 0x7f27481512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747e00640 1 -- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f274815db70 msgr2=0x7f27481f41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747e00640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f274815db70 0x7f27481f41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747e00640 1 -- 172.21.15.165:0/3274221839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2748152620 con 0x7f274815df50 2024-04-25T10:22:05.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f273ea00640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f274815db70 0x7f27481f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f2747e00640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f2748151af0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f2740009eb0 tx=0x7f2740004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740040070 con 0x7f274815df50 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2740031bc0 con 0x7f274815df50 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740009450 con 0x7f274815df50 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2748152900 con 0x7f274815df50 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.451+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27481f3cc0 con 0x7f274815df50 2024-04-25T10:22:05.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.452+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2748073990 con 0x7f274815df50 2024-04-25T10:22:05.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.453+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2740039040 con 0x7f274815df50 2024-04-25T10:22:05.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.454+0000 7f273fe00640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f272406b500 0x7f272406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.454+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f27400ab580 con 0x7f274815df50 2024-04-25T10:22:05.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.454+0000 7f2747400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f272406b500 0x7f272406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.455+0000 7f2747400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f272406b500 0x7f272406d9c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f27380012b0 tx=0x7f2738009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:05.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.458+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f2740079b50 con 0x7f274815df50 2024-04-25T10:22:05.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.656+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27480cedd0 con 0x7f274815df50 2024-04-25T10:22:05.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.658+0000 7f273fe00640 1 -- 172.21.15.165:0/3274221839 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17247 (secure 0 0 0) 0x7f27400a8370 con 0x7f274815df50 2024-04-25T10:22:05.661 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:05.661 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":198,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:20:50.950400+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_metadata","create_time":"2024-04-25T10:20:35.357863+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"194","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"cephfs_data","create_time":"2024-04-25T10:20:36.605736+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"29.b","mappings":[{"from":6,"to":0}]},{"pgid":"29.f","mappings":[{"from":6,"to":0}]},{"pgid":"29.19","mappings":[{"from":6,"to":0}]},{"pgid":"29.1b","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6837/3988146762":"2024-04-26T10:20:50.950335+0000","172.21.15.165:6835/3988146762":"2024-04-26T10:20:50.950335+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f272406b500 msgr2=0x7f272406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f272406b500 0x7f272406d9c0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f27380012b0 tx=0x7f2738009040 comp rx=0 tx=0).stop 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 msgr2=0x7f2748151af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f2748151af0 secure :-1 s=READY pgs=2876 cs=0 l=1 rev1=1 crypto rx=0x7f2740009eb0 tx=0x7f2740004a40 comp rx=0 tx=0).stop 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 shutdown_connections 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f272406b500 0x7f272406d9c0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f274815df50 0x7f2748151af0 unknown :-1 s=CLOSED pgs=2876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f274815db70 0x7f27481f41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 --2- 172.21.15.165:0/3274221839 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f27480cfc20 0x7f27481512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 >> 172.21.15.165:0/3274221839 conn(0x7f274806abe0 msgr2=0x7f2748158070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 shutdown_connections 2024-04-25T10:22:05.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.661+0000 7f274e400640 1 -- 172.21.15.165:0/3274221839 wait complete. 2024-04-25T10:22:05.683 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-25T10:22:05.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.877+0000 7f6cc9400640 1 Processor -- start 2024-04-25T10:22:05.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.878+0000 7f6cc9400640 1 -- start start 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc9400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc415dc20 0x7f6cc40ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc9400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6cc4161710 0x7f6cc40cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc9400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc415e000 con 0x7f6cc40b9330 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc9400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cc40d1de0 con 0x7f6cc4161710 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc9400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4161c60 con 0x7f6cc415dc20 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6cc4161710 0x7f6cc40cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6cc4161710 0x7f6cc40cebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42630/0 (socket says 172.21.15.165:42630) 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.879+0000 7f6cc1e00640 1 -- 172.21.15.165:0/2367113730 learned_addr learned my addr 172.21.15.165:0/2367113730 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:05.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.880+0000 7f6cc2800640 1 --2- 172.21.15.165:0/2367113730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc415dc20 0x7f6cc40ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.880+0000 7f6cc1e00640 1 -- 172.21.15.165:0/2367113730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc415dc20 msgr2=0x7f6cc40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.880+0000 7f6cc1e00640 1 --2- 172.21.15.165:0/2367113730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc415dc20 0x7f6cc40ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.880+0000 7f6cc1e00640 1 -- 172.21.15.165:0/2367113730 >> v1:172.21.15.172:6789/0 conn(0x7f6cc40b9330 legacy=0x7f6cc406bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.880+0000 7f6cc1e00640 1 -- 172.21.15.165:0/2367113730 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc4161f40 con 0x7f6cc4161710 2024-04-25T10:22:05.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.881+0000 7f6cc1e00640 1 --2- 172.21.15.165:0/2367113730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc4161710 0x7f6cc40cebb0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009870 tx=0x7f6cb002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64010bd5715d5b0a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.882+0000 7f6cc1400640 1 -- 172.21.15.165:0/2367113730 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0037070 con 0x7f6cc4161710 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.882+0000 7f6cc1400640 1 -- 172.21.15.165:0/2367113730 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0032440 con 0x7f6cc4161710 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.882+0000 7f6cc1400640 1 -- 172.21.15.165:0/2367113730 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0032810 con 0x7f6cc4161710 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.882+0000 7f6cc9400640 1 -- 172.21.15.165:0/2367113730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc4161710 msgr2=0x7f6cc40cebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.882+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2367113730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc4161710 0x7f6cc40cebb0 secure :-1 s=READY pgs=2877 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0009870 tx=0x7f6cb002a210 comp rx=0 tx=0).stop 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.883+0000 7f6cc9400640 1 -- 172.21.15.165:0/2367113730 shutdown_connections 2024-04-25T10:22:05.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.883+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2367113730 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc4161710 0x7f6cc40cebb0 unknown :-1 s=CLOSED pgs=2877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.883+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2367113730 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc415dc20 0x7f6cc40ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.883+0000 7f6cc9400640 1 -- 172.21.15.165:0/2367113730 >> 172.21.15.165:0/2367113730 conn(0x7f6cc40d0420 msgr2=0x7f6cc40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:05.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.883+0000 7f6cc9400640 1 -- 172.21.15.165:0/2367113730 shutdown_connections 2024-04-25T10:22:05.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.883+0000 7f6cc9400640 1 -- 172.21.15.165:0/2367113730 wait complete. 2024-04-25T10:22:05.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.884+0000 7f6cc9400640 1 Processor -- start 2024-04-25T10:22:05.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 -- start start 2024-04-25T10:22:05.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 0x7f6cc40b7a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cc415dc20 0x7f6cc40cd080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc4161710 0x7f6cc40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40b5e50 con 0x7f6cc40b9330 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc4162500 con 0x7f6cc415dc20 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.885+0000 7f6cc9400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc40b5fd0 con 0x7f6cc4161710 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 0x7f6cc40b7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 0x7f6cc40b7a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42632/0 (socket says 172.21.15.165:42632) 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 -- 172.21.15.165:0/2352149486 learned_addr learned my addr 172.21.15.165:0/2352149486 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 -- 172.21.15.165:0/2352149486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc4161710 msgr2=0x7f6cc40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc4161710 0x7f6cc40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 -- 172.21.15.165:0/2352149486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cc415dc20 msgr2=0x7f6cc40cd080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cc415dc20 0x7f6cc40cd080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 -- 172.21.15.165:0/2352149486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40cdb00 con 0x7f6cc40b9330 2024-04-25T10:22:05.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.886+0000 7f6cc2800640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 0x7f6cc40b7a70 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d5f0 tx=0x7f6cb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:05.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.887+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40125f0 con 0x7f6cc40b9330 2024-04-25T10:22:05.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.887+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb4019070 con 0x7f6cc40b9330 2024-04-25T10:22:05.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.887+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb40129d0 con 0x7f6cc40b9330 2024-04-25T10:22:05.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.887+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc40cbb60 con 0x7f6cc40b9330 2024-04-25T10:22:05.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.887+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc40cc120 con 0x7f6cc40b9330 2024-04-25T10:22:05.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.888+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc4073a00 con 0x7f6cc40b9330 2024-04-25T10:22:05.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.889+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6cb4012b70 con 0x7f6cc40b9330 2024-04-25T10:22:05.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.890+0000 7f6cbaa00640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ca406b4d0 0x7f6ca406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:05.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.890+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6303+0+0 (secure 0 0 0) 0x7f6cb408a8e0 con 0x7f6cc40b9330 2024-04-25T10:22:05.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.890+0000 7f6cc1e00640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ca406b4d0 0x7f6ca406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:05.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.891+0000 7f6cc1e00640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ca406b4d0 0x7f6ca406d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0034000 tx=0x7f6cb002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:05.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:05.894+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6cb4059470 con 0x7f6cc40b9330 2024-04-25T10:22:06.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.128+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f6cc406b010 con 0x7f6cc40b9330 2024-04-25T10:22:06.132 INFO:tasks.ceph.mon.a.smithi165.stderr:2024-04-25T10:22:06.129+0000 7f5481a00640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-25T10:22:06.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.134+0000 7f6cbaa00640 1 -- 172.21.15.165:0/2352149486 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v147) v1 ==== 161+0+0 (secure 0 0 0) 0x7f6cb4058d70 con 0x7f6cc40b9330 2024-04-25T10:22:06.136 INFO:teuthology.orchestra.run.smithi165.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-25T10:22:06.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.137+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ca406b4d0 msgr2=0x7f6ca406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:06.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.137+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ca406b4d0 0x7f6ca406d990 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f6cb0034000 tx=0x7f6cb002c000 comp rx=0 tx=0).stop 2024-04-25T10:22:06.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.137+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 msgr2=0x7f6cc40b7a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:06.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.137+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 0x7f6cc40b7a70 secure :-1 s=READY pgs=2878 cs=0 l=1 rev1=1 crypto rx=0x7f6cb400d5f0 tx=0x7f6cb400dab0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 shutdown_connections 2024-04-25T10:22:06.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6ca406b4d0 0x7f6ca406d990 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6cc4161710 0x7f6cc40cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6cc415dc20 0x7f6cc40cd080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 --2- 172.21.15.165:0/2352149486 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6cc40b9330 0x7f6cc40b7a70 unknown :-1 s=CLOSED pgs=2878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 >> 172.21.15.165:0/2352149486 conn(0x7f6cc40d0420 msgr2=0x7f6cc40b4aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:06.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.138+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 shutdown_connections 2024-04-25T10:22:06.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.139+0000 7f6cc9400640 1 -- 172.21.15.165:0/2352149486 wait complete. 2024-04-25T10:22:06.164 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-25T10:22:06.222 INFO:tasks.ceph.mds.c.smithi165.stdout:starting mds.c at 2024-04-25T10:22:06.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.346+0000 7f1739000640 1 Processor -- start 2024-04-25T10:22:06.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.346+0000 7f1739000640 1 -- start start 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1739000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f173415b710 0x7f17340cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1739000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173406ec50 0x7f173415e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1739000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17340cfd80 con 0x7f173415ee60 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1739000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f173415e490 con 0x7f173415b710 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1739000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17340d0060 con 0x7f173406ec50 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1731e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f173415b710 0x7f17340cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1732800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f173415ee60 0x7f17340ac8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:36476/0 (socket says 172.21.15.165:36476) 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.348+0000 7f1732800640 1 -- 172.21.15.165:0/3356870664 learned_addr learned my addr 172.21.15.165:0/3356870664 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:06.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1729400640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173406ec50 0x7f173415e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:06.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1731400640 1 -- 172.21.15.165:0/3356870664 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1173556311 0 0) 0x7f17340cfd80 con 0x7f173415ee60 2024-04-25T10:22:06.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1731400640 1 -- 172.21.15.165:0/3356870664 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1718003520 con 0x7f173415ee60 2024-04-25T10:22:06.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1731e00640 1 -- 172.21.15.165:0/3356870664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173406ec50 msgr2=0x7f173415e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:06.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1731e00640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173406ec50 0x7f173415e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1731e00640 1 -- 172.21.15.165:0/3356870664 >> v1:172.21.15.165:6789/0 conn(0x7f173415ee60 legacy=0x7f17340ac8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:06.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1731e00640 1 -- 172.21.15.165:0/3356870664 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340d0340 con 0x7f173415b710 2024-04-25T10:22:06.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.349+0000 7f1729400640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173406ec50 0x7f173415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:06.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.350+0000 7f1731e00640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173415b710 0x7f17340cf640 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f17240097a0 tx=0x7f172402a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f10e003bdb167d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1731400640 1 -- 172.21.15.165:0/3356870664 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724037070 con 0x7f173415b710 2024-04-25T10:22:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1731400640 1 -- 172.21.15.165:0/3356870664 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724032440 con 0x7f173415b710 2024-04-25T10:22:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1731400640 1 -- 172.21.15.165:0/3356870664 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17240327f0 con 0x7f173415b710 2024-04-25T10:22:06.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1739000640 1 -- 172.21.15.165:0/3356870664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173415b710 msgr2=0x7f17340cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1739000640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173415b710 0x7f17340cf640 secure :-1 s=READY pgs=2881 cs=0 l=1 rev1=1 crypto rx=0x7f17240097a0 tx=0x7f172402a180 comp rx=0 tx=0).stop 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1739000640 1 -- 172.21.15.165:0/3356870664 shutdown_connections 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1739000640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173406ec50 0x7f173415e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1739000640 1 --2- 172.21.15.165:0/3356870664 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173415b710 0x7f17340cf640 unknown :-1 s=CLOSED pgs=2881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.351+0000 7f1739000640 1 -- 172.21.15.165:0/3356870664 >> 172.21.15.165:0/3356870664 conn(0x7f173406abe0 msgr2=0x7f173406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.352+0000 7f1739000640 1 -- 172.21.15.165:0/3356870664 shutdown_connections 2024-04-25T10:22:06.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.352+0000 7f1739000640 1 -- 172.21.15.165:0/3356870664 wait complete. 2024-04-25T10:22:06.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.354+0000 7f1739000640 1 Processor -- start 2024-04-25T10:22:06.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.354+0000 7f1739000640 1 -- start start 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1739000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 0x7f1734158df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1739000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173415b710 0x7f17341f31a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1739000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f173415ee60 0x7f17341f36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1739000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173415bc70 con 0x7f173406ec50 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1739000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f173415baf0 con 0x7f173415ee60 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1739000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17340acfc0 con 0x7f173415b710 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1731e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 0x7f1734158df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:06.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1731e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 0x7f1734158df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:42650/0 (socket says 172.21.15.165:42650) 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.355+0000 7f1731e00640 1 -- 172.21.15.165:0/1885845385 learned_addr learned my addr 172.21.15.165:0/1885845385 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1731e00640 1 -- 172.21.15.165:0/1885845385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173415b710 msgr2=0x7f17341f31a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1731e00640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173415b710 0x7f17341f31a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1731e00640 1 -- 172.21.15.165:0/1885845385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f173415ee60 msgr2=0x7f17341f36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1731e00640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f173415ee60 0x7f17341f36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1731e00640 1 -- 172.21.15.165:0/1885845385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1734159760 con 0x7f173406ec50 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1731e00640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 0x7f1734158df0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f172402a1c0 tx=0x7f17240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724041020 con 0x7f173406ec50 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1724030700 con 0x7f173406ec50 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1724037040 con 0x7f173406ec50 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.356+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1734159a40 con 0x7f173406ec50 2024-04-25T10:22:06.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.357+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f173415a000 con 0x7f173406ec50 2024-04-25T10:22:06.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.358+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1734073a00 con 0x7f173406ec50 2024-04-25T10:22:06.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.359+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1724044020 con 0x7f173406ec50 2024-04-25T10:22:06.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.360+0000 7f172aa00640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f171006b4d0 0x7f171006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:06.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.360+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f17240aada0 con 0x7f173406ec50 2024-04-25T10:22:06.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.360+0000 7f1729400640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f171006b4d0 0x7f171006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:06.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.361+0000 7f1729400640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f171006b4d0 0x7f171006d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f17340cb960 tx=0x7f171c0094a0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:06.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.364+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f172404b030 con 0x7f173406ec50 2024-04-25T10:22:06.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:06.596+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1734161350 con 0x7f173406ec50 2024-04-25T10:22:07.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.137+0000 7f172aa00640 1 -- 172.21.15.165:0/1885845385 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v148) v1 ==== 104+0+0 (secure 0 0 0) 0x7f17240798d0 con 0x7f173406ec50 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.141+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f171006b4d0 msgr2=0x7f171006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.141+0000 7f1739000640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f171006b4d0 0x7f171006d990 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f17340cb960 tx=0x7f171c0094a0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.141+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 msgr2=0x7f1734158df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.141+0000 7f1739000640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 0x7f1734158df0 secure :-1 s=READY pgs=2882 cs=0 l=1 rev1=1 crypto rx=0x7f172402a1c0 tx=0x7f17240046c0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 shutdown_connections 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f171006b4d0 0x7f171006d990 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f173415ee60 0x7f17341f36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f173415b710 0x7f17341f31a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 --2- 172.21.15.165:0/1885845385 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f173406ec50 0x7f1734158df0 unknown :-1 s=CLOSED pgs=2882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 >> 172.21.15.165:0/1885845385 conn(0x7f173406abe0 msgr2=0x7f173415a8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:07.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 shutdown_connections 2024-04-25T10:22:07.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.142+0000 7f1739000640 1 -- 172.21.15.165:0/1885845385 wait complete. 2024-04-25T10:22:07.167 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-25T10:22:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.366+0000 7f4fac200640 1 Processor -- start 2024-04-25T10:22:07.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.367+0000 7f4fac200640 1 -- start start 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.368+0000 7f4fac200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4fa4144d50 0x7f4fa406d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.368+0000 7f4fac200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa406ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.368+0000 7f4fac200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fa40cccc0 con 0x7f4fa41446d0 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.368+0000 7f4fac200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fa40b2b20 con 0x7f4fa4144d50 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.368+0000 7f4fac200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa406d400 con 0x7f4fa41452a0 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.368+0000 7f4fa8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4fa4144d50 0x7f4fa406d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4fa8e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f4fa4144d50 0x7f4fa406d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33504/0 (socket says 172.21.15.165:33504) 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4fa8e00640 1 -- 172.21.15.165:0/4071969652 learned_addr learned my addr 172.21.15.165:0/4071969652 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4faa200640 1 --2- 172.21.15.165:0/4071969652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa406ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:07.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4fa8e00640 1 -- 172.21.15.165:0/4071969652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 msgr2=0x7f4fa406ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4fa8e00640 1 --2- 172.21.15.165:0/4071969652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa406ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4fa8e00640 1 -- 172.21.15.165:0/4071969652 >> v1:172.21.15.172:6789/0 conn(0x7f4fa41446d0 legacy=0x7f4fa40b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:07.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.369+0000 7f4fa8e00640 1 -- 172.21.15.165:0/4071969652 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa40cc140 con 0x7f4fa4144d50 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.370+0000 7f4fa8e00640 1 --2- 172.21.15.165:0/4071969652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa406d890 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f9002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da33f7c1aad15464 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.370+0000 7f4fa3e00640 1 -- 172.21.15.165:0/4071969652 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90037070 con 0x7f4fa4144d50 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.370+0000 7f4fa3e00640 1 -- 172.21.15.165:0/4071969652 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90032440 con 0x7f4fa4144d50 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.370+0000 7f4fa3e00640 1 -- 172.21.15.165:0/4071969652 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90032810 con 0x7f4fa4144d50 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.371+0000 7f4fac200640 1 -- 172.21.15.165:0/4071969652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 msgr2=0x7f4fa406d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.371+0000 7f4fac200640 1 --2- 172.21.15.165:0/4071969652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa406d890 secure :-1 s=READY pgs=2883 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f9002a210 comp rx=0 tx=0).stop 2024-04-25T10:22:07.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.371+0000 7f4fac200640 1 -- 172.21.15.165:0/4071969652 shutdown_connections 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.371+0000 7f4fac200640 1 --2- 172.21.15.165:0/4071969652 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa406ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.371+0000 7f4fac200640 1 --2- 172.21.15.165:0/4071969652 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa406d890 unknown :-1 s=CLOSED pgs=2883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.371+0000 7f4fac200640 1 -- 172.21.15.165:0/4071969652 >> 172.21.15.165:0/4071969652 conn(0x7f4fa406abe0 msgr2=0x7f4fa406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.372+0000 7f4fac200640 1 -- 172.21.15.165:0/4071969652 shutdown_connections 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.372+0000 7f4fac200640 1 -- 172.21.15.165:0/4071969652 wait complete. 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.373+0000 7f4fac200640 1 Processor -- start 2024-04-25T10:22:07.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.373+0000 7f4fac200640 1 -- start start 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.374+0000 7f4fac200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fa41446d0 0x7f4fa40c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.374+0000 7f4fac200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa4153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.374+0000 7f4fac200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa41f3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.374+0000 7f4fac200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40cc9e0 con 0x7f4fa4144d50 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.374+0000 7f4fac200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa40cc700 con 0x7f4fa41446d0 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.374+0000 7f4fac200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fa4144ab0 con 0x7f4fa41452a0 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa4153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa4153b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33514/0 (socket says 172.21.15.165:33514) 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 -- 172.21.15.165:0/3702112281 learned_addr learned my addr 172.21.15.165:0/3702112281 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 -- 172.21.15.165:0/3702112281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 msgr2=0x7f4fa41f3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:07.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa41f3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 -- 172.21.15.165:0/3702112281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fa41446d0 msgr2=0x7f4fa40c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fa41446d0 0x7f4fa40c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 -- 172.21.15.165:0/3702112281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa4154050 con 0x7f4fa4144d50 2024-04-25T10:22:07.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.375+0000 7f4fa8e00640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa4153b10 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.376+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90041020 con 0x7f4fa4144d50 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.376+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f90030970 con 0x7f4fa4144d50 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.376+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f90037040 con 0x7f4fa4144d50 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.376+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fa41542d0 con 0x7f4fa4144d50 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.376+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fa4154890 con 0x7f4fa4144d50 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.377+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fa40cd2e0 con 0x7f4fa4144d50 2024-04-25T10:22:07.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.378+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4f90044020 con 0x7f4fa4144d50 2024-04-25T10:22:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.379+0000 7f4fa1600640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4f8406b500 0x7f4f8406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.379+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6389+0+0 (secure 0 0 0) 0x7f4f900aab90 con 0x7f4fa4144d50 2024-04-25T10:22:07.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.379+0000 7f4fa9800640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4f8406b500 0x7f4f8406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:07.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.380+0000 7f4fa9800640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4f8406b500 0x7f4f8406d9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800ab90 tx=0x7f4f98009230 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:07.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.383+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4f9004b070 con 0x7f4fa4144d50 2024-04-25T10:22:07.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:07.581+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4fa4073a00 con 0x7f4fa4144d50 2024-04-25T10:22:08.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.433+0000 7f4fa1600640 1 -- 172.21.15.165:0/3702112281 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v200) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4fa4154050 con 0x7f4fa4144d50 2024-04-25T10:22:08.436 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_metadata' removed 2024-04-25T10:22:08.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.438+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4f8406b500 msgr2=0x7f4f8406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:08.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.438+0000 7f4fac200640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4f8406b500 0x7f4f8406d9c0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f4f9800ab90 tx=0x7f4f98009230 comp rx=0 tx=0).stop 2024-04-25T10:22:08.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.438+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 msgr2=0x7f4fa4153b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:08.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.438+0000 7f4fac200640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa4153b10 secure :-1 s=READY pgs=2884 cs=0 l=1 rev1=1 crypto rx=0x7f4f90009870 tx=0x7f4f900046c0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 shutdown_connections 2024-04-25T10:22:08.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4f8406b500 0x7f4f8406d9c0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f4fa41452a0 0x7f4fa41f3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f4fa4144d50 0x7f4fa4153b10 unknown :-1 s=CLOSED pgs=2884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 --2- 172.21.15.165:0/3702112281 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f4fa41446d0 0x7f4fa40c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 >> 172.21.15.165:0/3702112281 conn(0x7f4fa406abe0 msgr2=0x7f4fa406d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:08.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.439+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 shutdown_connections 2024-04-25T10:22:08.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.440+0000 7f4fac200640 1 -- 172.21.15.165:0/3702112281 wait complete. 2024-04-25T10:22:08.466 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-25T10:22:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.668+0000 7f3bc7e00640 1 Processor -- start 2024-04-25T10:22:08.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.669+0000 7f3bc7e00640 1 -- start start 2024-04-25T10:22:08.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.670+0000 7f3bc7e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3bc815dd30 0x7f3bc80cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:08.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.670+0000 7f3bc7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc806bff0 0x7f3bc80b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:08.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.670+0000 7f3bc7e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc815e110 con 0x7f3bc80b5970 2024-04-25T10:22:08.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.670+0000 7f3bc7e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc806d000 con 0x7f3bc815dd30 2024-04-25T10:22:08.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.670+0000 7f3bc7e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80b2fd0 con 0x7f3bc806bff0 2024-04-25T10:22:08.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.671+0000 7f3bc6a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3bc815dd30 0x7f3bc80cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:08.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.671+0000 7f3bc6000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc806bff0 0x7f3bc80b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:08.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.671+0000 7f3bc6a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3bc815dd30 0x7f3bc80cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:50372/0 (socket says 172.21.15.165:50372) 2024-04-25T10:22:08.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.671+0000 7f3bc6a00640 1 -- 172.21.15.165:0/3373797469 learned_addr learned my addr 172.21.15.165:0/3373797469 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:08.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.672+0000 7f3bc6a00640 1 -- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc806bff0 msgr2=0x7f3bc80b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:08.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.672+0000 7f3bc6a00640 1 --2- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc806bff0 0x7f3bc80b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.672+0000 7f3bc6a00640 1 -- 172.21.15.165:0/3373797469 >> v1:172.21.15.172:6789/0 conn(0x7f3bc80b5970 legacy=0x7f3bc8161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:08.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.672+0000 7f3bc6a00640 1 -- 172.21.15.165:0/3373797469 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc806c480 con 0x7f3bc815dd30 2024-04-25T10:22:08.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.673+0000 7f3bc6a00640 1 --2- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815dd30 0x7f3bc80cebd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009870 tx=0x7f3bb002a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3328d14cace8a5b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:08.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.674+0000 7f3bc5600640 1 -- 172.21.15.165:0/3373797469 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0037070 con 0x7f3bc815dd30 2024-04-25T10:22:08.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.674+0000 7f3bc5600640 1 -- 172.21.15.165:0/3373797469 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb0032440 con 0x7f3bc815dd30 2024-04-25T10:22:08.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.674+0000 7f3bc5600640 1 -- 172.21.15.165:0/3373797469 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0032810 con 0x7f3bc815dd30 2024-04-25T10:22:08.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815dd30 msgr2=0x7f3bc80cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:08.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815dd30 0x7f3bc80cebd0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0009870 tx=0x7f3bb002a270 comp rx=0 tx=0).stop 2024-04-25T10:22:08.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3373797469 shutdown_connections 2024-04-25T10:22:08.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc806bff0 0x7f3bc80b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/3373797469 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815dd30 0x7f3bc80cebd0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3373797469 >> 172.21.15.165:0/3373797469 conn(0x7f3bc80d0610 msgr2=0x7f3bc80d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:08.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3373797469 shutdown_connections 2024-04-25T10:22:08.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.675+0000 7f3bc7e00640 1 -- 172.21.15.165:0/3373797469 wait complete. 2024-04-25T10:22:08.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.677+0000 7f3bc7e00640 1 Processor -- start 2024-04-25T10:22:08.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.677+0000 7f3bc7e00640 1 -- start start 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b5970 0x7f3bc8143d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 0x7f3bc8152b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815e2e0 0x7f3bc815e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc806cd20 con 0x7f3bc815dd30 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc806ca40 con 0x7f3bc815e2e0 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc80cf110 con 0x7f3bc80b5970 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 0x7f3bc8152b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 0x7f3bc8152b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33528/0 (socket says 172.21.15.165:33528) 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 -- 172.21.15.165:0/2781625639 learned_addr learned my addr 172.21.15.165:0/2781625639 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6a00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b5970 0x7f3bc8143d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc7400640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815e2e0 0x7f3bc815e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 -- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b5970 msgr2=0x7f3bc8143d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:08.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b5970 0x7f3bc8143d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 -- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815e2e0 msgr2=0x7f3bc815e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815e2e0 0x7f3bc815e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.679+0000 7f3bc6000640 1 -- 172.21.15.165:0/2781625639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc8144240 con 0x7f3bc815dd30 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bc7400640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815e2e0 0x7f3bc815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bc6000640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 0x7f3bc8152b70 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800efc0 tx=0x7f3bb800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8018020 con 0x7f3bc815dd30 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8014480 con 0x7f3bc815dd30 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc8161390 con 0x7f3bc815dd30 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8014850 con 0x7f3bc815dd30 2024-04-25T10:22:08.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.680+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc80cbdb0 con 0x7f3bc815dd30 2024-04-25T10:22:08.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.682+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc8073990 con 0x7f3bc815dd30 2024-04-25T10:22:08.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.682+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3bb8005d10 con 0x7f3bc815dd30 2024-04-25T10:22:08.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.683+0000 7f3bbea00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:08.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.684+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5835+0+0 (secure 0 0 0) 0x7f3bb808b180 con 0x7f3bc815dd30 2024-04-25T10:22:08.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.684+0000 7f3bc6a00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:08.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.685+0000 7f3bc6a00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0034000 tx=0x7f3bb002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:08.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.688+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3bb8022030 con 0x7f3bc815dd30 2024-04-25T10:22:08.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:08.898+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3bc81530b0 con 0x7f3bc815dd30 2024-04-25T10:22:09.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.444+0000 7f3bbea00640 1 -- 172.21.15.165:0/2781625639 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v201) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3bb8010040 con 0x7f3bc815dd30 2024-04-25T10:22:09.447 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'cephfs_data' removed 2024-04-25T10:22:09.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.449+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 msgr2=0x7f3ba406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.449+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0034000 tx=0x7f3bb002e000 comp rx=0 tx=0).stop 2024-04-25T10:22:09.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.449+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 msgr2=0x7f3bc8152b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.449+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 0x7f3bc8152b70 secure :-1 s=READY pgs=2885 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800efc0 tx=0x7f3bb800c490 comp rx=0 tx=0).stop 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 shutdown_connections 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3ba406b4d0 0x7f3ba406d990 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bc815e2e0 0x7f3bc815e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bc815dd30 0x7f3bc8152b70 unknown :-1 s=CLOSED pgs=2885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 --2- 172.21.15.165:0/2781625639 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bc80b5970 0x7f3bc8143d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 >> 172.21.15.165:0/2781625639 conn(0x7f3bc80d0610 msgr2=0x7f3bc815cfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 shutdown_connections 2024-04-25T10:22:09.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.450+0000 7f3bc7e00640 1 -- 172.21.15.165:0/2781625639 wait complete. 2024-04-25T10:22:09.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds mds_hack_allow_loading_invalid_metadata 2024-04-25T10:22:09.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.668+0000 7f510f400640 1 Processor -- start 2024-04-25T10:22:09.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.668+0000 7f510f400640 1 -- start start 2024-04-25T10:22:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.669+0000 7f510f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51081446d0 0x7f51080b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f510806e8a0 0x7f51080b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510f400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51080cccc0 con 0x7f5108144d50 2024-04-25T10:22:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510f400640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51080b2b20 con 0x7f510806e8a0 2024-04-25T10:22:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510806d400 con 0x7f51081446d0 2024-04-25T10:22:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510d400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f510806e8a0 0x7f51080b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510d400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f510806e8a0 0x7f51080b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33544/0 (socket says 172.21.15.165:33544) 2024-04-25T10:22:09.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510d400640 1 -- 172.21.15.165:0/3013532799 learned_addr learned my addr 172.21.15.165:0/3013532799 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.670+0000 7f510ca00640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51081446d0 0x7f51080b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.671+0000 7f510d400640 1 -- 172.21.15.165:0/3013532799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51081446d0 msgr2=0x7f51080b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.671+0000 7f510d400640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51081446d0 0x7f51080b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.671+0000 7f510d400640 1 -- 172.21.15.165:0/3013532799 >> v1:172.21.15.172:6789/0 conn(0x7f5108144d50 legacy=0x7f51080cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.671+0000 7f510d400640 1 -- 172.21.15.165:0/3013532799 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080cc140 con 0x7f510806e8a0 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.671+0000 7f510ca00640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51081446d0 0x7f51080b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:09.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.672+0000 7f510d400640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080b4460 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f5100009e50 tx=0x7f510002a6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25b0719c2f25df0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.672+0000 7f5107400640 1 -- 172.21.15.165:0/3013532799 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5100036070 con 0x7f510806e8a0 2024-04-25T10:22:09.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.672+0000 7f5107400640 1 -- 172.21.15.165:0/3013532799 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5100031600 con 0x7f510806e8a0 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.672+0000 7f5107400640 1 -- 172.21.15.165:0/3013532799 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51000319d0 con 0x7f510806e8a0 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 -- 172.21.15.165:0/3013532799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 msgr2=0x7f51080b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080b4460 secure :-1 s=READY pgs=2886 cs=0 l=1 rev1=1 crypto rx=0x7f5100009e50 tx=0x7f510002a6d0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 -- 172.21.15.165:0/3013532799 shutdown_connections 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080b4460 unknown :-1 s=CLOSED pgs=2886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 --2- 172.21.15.165:0/3013532799 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f51081446d0 0x7f51080b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 -- 172.21.15.165:0/3013532799 >> 172.21.15.165:0/3013532799 conn(0x7f510806abe0 msgr2=0x7f510806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 -- 172.21.15.165:0/3013532799 shutdown_connections 2024-04-25T10:22:09.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.673+0000 7f510f400640 1 -- 172.21.15.165:0/3013532799 wait complete. 2024-04-25T10:22:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.675+0000 7f510f400640 1 Processor -- start 2024-04-25T10:22:09.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.675+0000 7f510f400640 1 -- start start 2024-04-25T10:22:09.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510f400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51081446d0 0x7f51081ed400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5108144d50 0x7f510813eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510f400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080cc700 con 0x7f510806e8a0 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510f400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51080b49a0 con 0x7f51081446d0 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510f400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080cc9e0 con 0x7f5108144d50 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080c46d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33560/0 (socket says 172.21.15.165:33560) 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 -- 172.21.15.165:0/524969667 learned_addr learned my addr 172.21.15.165:0/524969667 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 -- 172.21.15.165:0/524969667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5108144d50 msgr2=0x7f510813eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5108144d50 0x7f510813eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 -- 172.21.15.165:0/524969667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51081446d0 msgr2=0x7f51081ed400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51081446d0 0x7f51081ed400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.676+0000 7f510ca00640 1 -- 172.21.15.165:0/524969667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51080c4c10 con 0x7f510806e8a0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.677+0000 7f510ca00640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080c46d0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c820 tx=0x7f50fc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.677+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc015070 con 0x7f510806e8a0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.677+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc007a30 con 0x7f510806e8a0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.677+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc007de0 con 0x7f510806e8a0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.677+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51080cdcf0 con 0x7f510806e8a0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.677+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51080ce2b0 con 0x7f510806e8a0 2024-04-25T10:22:09.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.678+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5108073a00 con 0x7f510806e8a0 2024-04-25T10:22:09.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.679+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f50fc018070 con 0x7f510806e8a0 2024-04-25T10:22:09.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.680+0000 7f5104c00640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f50e806b4d0 0x7f50e806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:09.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.680+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f50fc08a080 con 0x7f510806e8a0 2024-04-25T10:22:09.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.680+0000 7f5107e00640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f50e806b4d0 0x7f50e806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:09.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.681+0000 7f5107e00640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f50e806b4d0 0x7f50e806d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f4006bc0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:09.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.685+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f50fc059010 con 0x7f510806e8a0 2024-04-25T10:22:09.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.878+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"} v 0) v1 -- 0x7f5108071310 con 0x7f510806e8a0 2024-04-25T10:22:09.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.883+0000 7f5104c00640 1 -- 172.21.15.165:0/524969667 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "mds_hack_allow_loading_invalid_metadata"}]=0 v36) v1 ==== 122+0+0 (secure 0 0 0) 0x7f50fc058910 con 0x7f510806e8a0 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.885+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f50e806b4d0 msgr2=0x7f50e806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.885+0000 7f510f400640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f50e806b4d0 0x7f50e806d990 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006fd0 tx=0x7f50f4006bc0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.885+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 msgr2=0x7f51080c46d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.885+0000 7f510f400640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080c46d0 secure :-1 s=READY pgs=2887 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00c820 tx=0x7f50fc00cc20 comp rx=0 tx=0).stop 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 shutdown_connections 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f50e806b4d0 0x7f50e806d990 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f5108144d50 0x7f510813eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f51081446d0 0x7f51081ed400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 --2- 172.21.15.165:0/524969667 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f510806e8a0 0x7f51080c46d0 unknown :-1 s=CLOSED pgs=2887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 >> 172.21.15.165:0/524969667 conn(0x7f510806abe0 msgr2=0x7f510806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 shutdown_connections 2024-04-25T10:22:09.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:09.886+0000 7f510f400640 1 -- 172.21.15.165:0/524969667 wait complete. 2024-04-25T10:22:09.907 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2024-04-25T10:22:10.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.093+0000 7feb41e00640 1 Processor -- start 2024-04-25T10:22:10.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.094+0000 7feb41e00640 1 -- start start 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.095+0000 7feb41e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7feb3c15cfc0 0x7feb3c0c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.095+0000 7feb41e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c145520 con 0x7feb3c0ceb20 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.095+0000 7feb41e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c071270 con 0x7feb3c145140 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.095+0000 7feb41e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb3c071550 con 0x7feb3c15cfc0 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7feb3c0ceb20 0x7feb3c0c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60964/0 (socket says 172.21.15.165:60964) 2024-04-25T10:22:10.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3be00640 1 -- 172.21.15.165:0/3156254599 learned_addr learned my addr 172.21.15.165:0/3156254599 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb41400640 1 --2- 172.21.15.165:0/3156254599 >> v2:172.21.15.172:3300/0 conn(0x7feb3c15cfc0 0x7feb3c0c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2242846675 0 0) 0x7feb3c145520 con 0x7feb3c0ceb20 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb24003520 con 0x7feb3c0ceb20 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3974830195 0 0) 0x7feb24003520 con 0x7feb3c0ceb20 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 >> v2:172.21.15.172:3300/0 conn(0x7feb3c15cfc0 msgr2=0x7feb3c0c5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 --2- 172.21.15.165:0/3156254599 >> v2:172.21.15.172:3300/0 conn(0x7feb3c15cfc0 0x7feb3c0c5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 >> v1:172.21.15.172:6789/0 conn(0x7feb3c145140 legacy=0x7feb3c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c071830 con 0x7feb3c0ceb20 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.096+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7feb28001670 con 0x7feb3c0ceb20 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 >> v1:172.21.15.165:6789/0 conn(0x7feb3c0ceb20 legacy=0x7feb3c0c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb24003db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb24004800 0x7feb24004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb240051c0 0x7feb2400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c071550 con 0x7feb240039b0 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400c0f0 con 0x7feb240051c0 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400c2f0 con 0x7feb24004800 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb40a00640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb24003db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb40a00640 1 -- 172.21.15.165:0/3156254599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb24004800 msgr2=0x7feb24004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb40a00640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb24004800 0x7feb24004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb40a00640 1 -- 172.21.15.165:0/3156254599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb240051c0 msgr2=0x7feb2400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb40a00640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb240051c0 0x7feb2400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb40a00640 1 -- 172.21.15.165:0/3156254599 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24003520 con 0x7feb240039b0 2024-04-25T10:22:10.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.097+0000 7feb41400640 1 -- 172.21.15.165:0/3156254599 reap_dead start 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.098+0000 7feb40a00640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb24003db0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7feb30003db0 tx=0x7feb24005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff785e60714282c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30037070 con 0x7feb240039b0 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3002a9f0 con 0x7feb240039b0 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb3b400640 1 -- 172.21.15.165:0/3156254599 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30003980 con 0x7feb240039b0 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb41e00640 1 -- 172.21.15.165:0/3156254599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 msgr2=0x7feb24003db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb41e00640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb24003db0 secure :-1 s=READY pgs=2889 cs=0 l=1 rev1=1 crypto rx=0x7feb30003db0 tx=0x7feb24005830 comp rx=0 tx=0).stop 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb41e00640 1 -- 172.21.15.165:0/3156254599 shutdown_connections 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb41e00640 1 --2- 172.21.15.165:0/3156254599 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb24003db0 unknown :-1 s=CLOSED pgs=2889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.099+0000 7feb41e00640 1 -- 172.21.15.165:0/3156254599 >> 172.21.15.165:0/3156254599 conn(0x7feb3c061cc0 msgr2=0x7feb3c06abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.100+0000 7feb41e00640 1 -- 172.21.15.165:0/3156254599 shutdown_connections 2024-04-25T10:22:10.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.100+0000 7feb41e00640 1 -- 172.21.15.165:0/3156254599 wait complete. 2024-04-25T10:22:10.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.101+0000 7feb41e00640 1 Processor -- start 2024-04-25T10:22:10.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.101+0000 7feb41e00640 1 -- start start 2024-04-25T10:22:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.102+0000 7feb41e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb3c0b0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.102+0000 7feb41e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c13de60 0x7feb3c142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.102+0000 7feb41e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c13e2c0 0x7feb3c143100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.102+0000 7feb41e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0ce590 con 0x7feb240039b0 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.102+0000 7feb41e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0ce410 con 0x7feb3c13de60 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.102+0000 7feb41e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0ce710 con 0x7feb3c13e2c0 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb3c0b0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb3c0b0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33580/0 (socket says 172.21.15.165:33580) 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 -- 172.21.15.165:0/2543756902 learned_addr learned my addr 172.21.15.165:0/2543756902 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 -- 172.21.15.165:0/2543756902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c13e2c0 msgr2=0x7feb3c143100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c13e2c0 0x7feb3c143100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 -- 172.21.15.165:0/2543756902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c13de60 msgr2=0x7feb3c142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c13de60 0x7feb3c142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 -- 172.21.15.165:0/2543756902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c13ee50 con 0x7feb240039b0 2024-04-25T10:22:10.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.103+0000 7feb40a00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb3c0b0c50 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7feb30003db0 tx=0x7feb300036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.104+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30037050 con 0x7feb240039b0 2024-04-25T10:22:10.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.104+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30038040 con 0x7feb240039b0 2024-04-25T10:22:10.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.104+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30007040 con 0x7feb240039b0 2024-04-25T10:22:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.104+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c13e890 con 0x7feb240039b0 2024-04-25T10:22:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.104+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c14dd60 con 0x7feb240039b0 2024-04-25T10:22:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.105+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c073990 con 0x7feb240039b0 2024-04-25T10:22:10.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.106+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7feb30045020 con 0x7feb240039b0 2024-04-25T10:22:10.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.107+0000 7feb38c00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1c06b500 0x7feb1c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.107+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7feb300a9cc0 con 0x7feb240039b0 2024-04-25T10:22:10.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.108+0000 7feb3be00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1c06b500 0x7feb1c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.108+0000 7feb3be00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1c06b500 0x7feb1c06d9c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7feb3c140120 tx=0x7feb28001560 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.112+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7feb30078cf0 con 0x7feb240039b0 2024-04-25T10:22:10.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.312+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7feb3c1444f0 con 0x7feb240039b0 2024-04-25T10:22:10.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.320+0000 7feb38c00640 1 -- 172.21.15.165:0/2543756902 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7feb300785f0 con 0x7feb240039b0 2024-04-25T10:22:10.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1c06b500 msgr2=0x7feb1c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1c06b500 0x7feb1c06d9c0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7feb3c140120 tx=0x7feb28001560 comp rx=0 tx=0).stop 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 msgr2=0x7feb3c0b0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb3c0b0c50 secure :-1 s=READY pgs=2890 cs=0 l=1 rev1=1 crypto rx=0x7feb30003db0 tx=0x7feb300036f0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 shutdown_connections 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7feb1c06b500 0x7feb1c06d9c0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7feb3c13e2c0 0x7feb3c143100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7feb3c13de60 0x7feb3c142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 --2- 172.21.15.165:0/2543756902 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7feb240039b0 0x7feb3c0b0c50 unknown :-1 s=CLOSED pgs=2890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.323+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 >> 172.21.15.165:0/2543756902 conn(0x7feb3c061cc0 msgr2=0x7feb3c15de00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.324+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 shutdown_connections 2024-04-25T10:22:10.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.324+0000 7feb41e00640 1 -- 172.21.15.165:0/2543756902 wait complete. 2024-04-25T10:22:10.345 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode' 2024-04-25T10:22:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.547+0000 7f9d5ee00640 1 Processor -- start 2024-04-25T10:22:10.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.547+0000 7f9d5ee00640 1 -- start start 2024-04-25T10:22:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d5ee00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9d5815db10 0x7f9d58162230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d5ee00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9d580b3220 0x7f9d580b3680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d5ee00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d58071670 con 0x7f9d580cf3f0 2024-04-25T10:22:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d5ee00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d580b3bc0 con 0x7f9d5815db10 2024-04-25T10:22:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d5ee00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d58072280 con 0x7f9d580b3220 2024-04-25T10:22:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d57e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9d5815db10 0x7f9d58162230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d57e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9d5815db10 0x7f9d58162230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33594/0 (socket says 172.21.15.165:33594) 2024-04-25T10:22:10.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.549+0000 7f9d57e00640 1 -- 172.21.15.165:0/2566814492 learned_addr learned my addr 172.21.15.165:0/2566814492 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.550+0000 7f9d57400640 1 --2- 172.21.15.165:0/2566814492 >> v2:172.21.15.172:3300/0 conn(0x7f9d580b3220 0x7f9d580b3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.550+0000 7f9d57e00640 1 -- 172.21.15.165:0/2566814492 >> v2:172.21.15.172:3300/0 conn(0x7f9d580b3220 msgr2=0x7f9d580b3680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.550+0000 7f9d57e00640 1 --2- 172.21.15.165:0/2566814492 >> v2:172.21.15.172:3300/0 conn(0x7f9d580b3220 0x7f9d580b3680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.550+0000 7f9d57e00640 1 -- 172.21.15.165:0/2566814492 >> v1:172.21.15.172:6789/0 conn(0x7f9d580cf3f0 legacy=0x7f9d580cf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.550+0000 7f9d57e00640 1 -- 172.21.15.165:0/2566814492 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d58071950 con 0x7f9d5815db10 2024-04-25T10:22:10.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.550+0000 7f9d57400640 1 --2- 172.21.15.165:0/2566814492 >> v2:172.21.15.172:3300/0 conn(0x7f9d580b3220 0x7f9d580b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.551+0000 7f9d57e00640 1 --2- 172.21.15.165:0/2566814492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d58162230 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f9d580aee70 tx=0x7f9d4802a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7182b331317b9bd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.551+0000 7f9d56a00640 1 -- 172.21.15.165:0/2566814492 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48036070 con 0x7f9d5815db10 2024-04-25T10:22:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.551+0000 7f9d56a00640 1 -- 172.21.15.165:0/2566814492 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d48031600 con 0x7f9d5815db10 2024-04-25T10:22:10.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.551+0000 7f9d56a00640 1 -- 172.21.15.165:0/2566814492 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d480319d0 con 0x7f9d5815db10 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 -- 172.21.15.165:0/2566814492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 msgr2=0x7f9d58162230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/2566814492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d58162230 secure :-1 s=READY pgs=2891 cs=0 l=1 rev1=1 crypto rx=0x7f9d580aee70 tx=0x7f9d4802a6f0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 -- 172.21.15.165:0/2566814492 shutdown_connections 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/2566814492 >> v2:172.21.15.172:3300/0 conn(0x7f9d580b3220 0x7f9d580b3680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/2566814492 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d58162230 unknown :-1 s=CLOSED pgs=2891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 -- 172.21.15.165:0/2566814492 >> 172.21.15.165:0/2566814492 conn(0x7f9d5806abe0 msgr2=0x7f9d5806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 -- 172.21.15.165:0/2566814492 shutdown_connections 2024-04-25T10:22:10.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.552+0000 7f9d5ee00640 1 -- 172.21.15.165:0/2566814492 wait complete. 2024-04-25T10:22:10.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.554+0000 7f9d5ee00640 1 Processor -- start 2024-04-25T10:22:10.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.554+0000 7f9d5ee00640 1 -- start start 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ee00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d580b3220 0x7f9d580ce6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ee00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d580cf3f0 0x7f9d5815b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ee00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d580cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ee00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58071f10 con 0x7f9d5815db10 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ee00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d58162770 con 0x7f9d580cf3f0 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ee00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d580af0e0 con 0x7f9d580b3220 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.555+0000 7f9d5ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d580cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d580cd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33596/0 (socket says 172.21.15.165:33596) 2024-04-25T10:22:10.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 -- 172.21.15.165:0/4146537510 learned_addr learned my addr 172.21.15.165:0/4146537510 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d57400640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d580cf3f0 0x7f9d5815b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d57e00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d580b3220 0x7f9d580ce6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 -- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d580b3220 msgr2=0x7f9d580ce6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d580b3220 0x7f9d580ce6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 -- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d580cf3f0 msgr2=0x7f9d5815b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d580cf3f0 0x7f9d5815b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 -- 172.21.15.165:0/4146537510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5815a270 con 0x7f9d5815db10 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.556+0000 7f9d5ce00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d580cd230 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c3c0 tx=0x7f9d5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.557+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50017070 con 0x7f9d5815db10 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.557+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d500035f0 con 0x7f9d5815db10 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.557+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5815a550 con 0x7f9d5815db10 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.557+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d500039e0 con 0x7f9d5815db10 2024-04-25T10:22:10.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.557+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5815ab10 con 0x7f9d5815db10 2024-04-25T10:22:10.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.558+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d58073a00 con 0x7f9d5815db10 2024-04-25T10:22:10.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.559+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9d5001a070 con 0x7f9d5815db10 2024-04-25T10:22:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.560+0000 7f9d4fe00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d3806b4d0 0x7f9d3806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.560+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f9d50089520 con 0x7f9d5815db10 2024-04-25T10:22:10.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.560+0000 7f9d57e00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d3806b4d0 0x7f9d3806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:10.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.561+0000 7f9d57e00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d3806b4d0 0x7f9d3806d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9d48033000 tx=0x7f9d48009000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:10.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.564+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9d50058420 con 0x7f9d5815db10 2024-04-25T10:22:10.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:10.843+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]} v 0) v1 -- 0x7f9d5806ebf0 con 0x7f9d5815db10 2024-04-25T10:22:11.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.460+0000 7f9d4fe00640 1 -- 172.21.15.165:0/4146537510 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_scrub.TestScrub.test_scrub_dup_inode"]}]=0 v918) v1 ==== 133+0+0 (secure 0 0 0) 0x7f9d50057d20 con 0x7f9d5815db10 2024-04-25T10:22:11.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d3806b4d0 msgr2=0x7f9d3806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d3806b4d0 0x7f9d3806d990 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f9d48033000 tx=0x7f9d48009000 comp rx=0 tx=0).stop 2024-04-25T10:22:11.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 msgr2=0x7f9d580cd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:11.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d580cd230 secure :-1 s=READY pgs=2892 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c3c0 tx=0x7f9d5000c880 comp rx=0 tx=0).stop 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 shutdown_connections 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9d3806b4d0 0x7f9d3806d990 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9d5815db10 0x7f9d580cd230 unknown :-1 s=CLOSED pgs=2892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9d580cf3f0 0x7f9d5815b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 --2- 172.21.15.165:0/4146537510 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9d580b3220 0x7f9d580ce6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 >> 172.21.15.165:0/4146537510 conn(0x7f9d5806abe0 msgr2=0x7f9d580b0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 shutdown_connections 2024-04-25T10:22:11.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:11.464+0000 7f9d5ee00640 1 -- 172.21.15.165:0/4146537510 wait complete. 2024-04-25T10:22:11.488 INFO:tasks.cephfs_test_runner:test_scrub_dup_inode (tasks.cephfs.test_scrub.TestScrub) ... ok 2024-04-25T10:22:11.488 INFO:tasks.cephfs_test_runner: 2024-04-25T10:22:11.488 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-04-25T10:22:11.488 INFO:tasks.cephfs_test_runner:Ran 13 tests in 1150.939s 2024-04-25T10:22:11.488 INFO:tasks.cephfs_test_runner: 2024-04-25T10:22:11.488 INFO:tasks.cephfs_test_runner:OK 2024-04-25T10:22:11.489 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-04-25T10:22:11.497 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-25T10:22:11.506 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-25T10:22:11.506 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:22:11.506 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:11.559 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-25T10:22:11.560 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.611 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-04-25T10:22:11.612 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:22:11.612 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.669 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:11.669 DEBUG:teuthology.orchestra.run.smithi172:> set -ex 2024-04-25T10:22:11.669 DEBUG:teuthology.orchestra.run.smithi172:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-25T10:22:11.722 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052733,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249000k,nr_inodes=812250,mode=700,uid=1000,gid=1316 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498004k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-04-25T10:22:11.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.722 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:22:11.779 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-25T10:22:11.779 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-04-25T10:22:11.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.781 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-25T10:22:11.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-04-25T10:22:11.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:11.942 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link delete ceph-brx 2024-04-25T10:22:12.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.069 DEBUG:teuthology.orchestra.run.smithi165:> ip netns list 2024-04-25T10:22:12.111 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.112 DEBUG:teuthology.orchestra.run.smithi165:> sudo ip link delete ceph-brx 2024-04-25T10:22:12.231 INFO:teuthology.orchestra.run.smithi165.stderr:Cannot find device "ceph-brx" 2024-04-25T10:22:12.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:22:12.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.235 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-25T10:22:12.292 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-25T10:22:12.293 INFO:teuthology.orchestra.run.smithi172.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-04-25T10:22:12.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.294 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-25T10:22:12.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.381 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-04-25T10:22:12.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.462 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-25T10:22:12.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.583 DEBUG:teuthology.orchestra.run.smithi172:> ip netns list 2024-04-25T10:22:12.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-25T10:22:12.624 DEBUG:teuthology.orchestra.run.smithi172:> sudo ip link delete ceph-brx 2024-04-25T10:22:12.743 INFO:teuthology.orchestra.run.smithi172.stderr:Cannot find device "ceph-brx" 2024-04-25T10:22:12.746 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:22:12.747 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-25T10:22:12.756 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-25T10:22:12.756 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:12.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.959+0000 7f8317e00640 1 Processor -- start 2024-04-25T10:22:12.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.959+0000 7f8317e00640 1 -- start start 2024-04-25T10:22:12.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.960+0000 7f8317e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f83180cda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.960+0000 7f8317e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831815bf30 con 0x7f8318071700 2024-04-25T10:22:12.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.960+0000 7f8317e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f831815c210 con 0x7f8318144440 2024-04-25T10:22:12.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.960+0000 7f8317e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f831815c860 con 0x7f831815dd00 2024-04-25T10:22:12.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.961+0000 7f830fe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8318071700 0x7f83180cd320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60978/0 (socket says 172.21.15.165:60978) 2024-04-25T10:22:12.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.961+0000 7f830fe00640 1 -- 172.21.15.165:0/3662619865 learned_addr learned my addr 172.21.15.165:0/3662619865 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:12.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.961+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966987035 0 0) 0x7f831815bf30 con 0x7f8318071700 2024-04-25T10:22:12.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.961+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82fc003520 con 0x7f8318071700 2024-04-25T10:22:12.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.961+0000 7f8317400640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f83180cda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2079699787 0 0) 0x7f82fc003520 con 0x7f8318071700 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 msgr2=0x7f83180cda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f83180cda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 >> v1:172.21.15.172:6789/0 conn(0x7f8318144440 legacy=0x7f8318070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f831815c4f0 con 0x7f8318071700 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f8300001670 con 0x7f8318071700 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8317400640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f83180cda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 >> v1:172.21.15.165:6789/0 conn(0x7f8318071700 legacy=0x7f83180cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc0039e0 0x7f82fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc004800 0x7f82fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f82fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f831815c860 con 0x7f82fc004800 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc00bb80 con 0x7f82fc0039e0 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82fc00bd80 con 0x7f831815dd00 2024-04-25T10:22:12.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f830fe00640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc004800 0x7f82fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.962+0000 7f8316a00640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc0039e0 0x7f82fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.963+0000 7f8317400640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f82fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.963+0000 7f830fe00640 1 -- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 msgr2=0x7f82fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.963+0000 7f830fe00640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f831815dd00 0x7f82fc0051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.963+0000 7f830fe00640 1 -- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc0039e0 msgr2=0x7f82fc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.963+0000 7f830fe00640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f82fc0039e0 0x7f82fc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.963+0000 7f830fe00640 1 -- 172.21.15.165:0/3662619865 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82fc003520 con 0x7f82fc004800 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.964+0000 7f830fe00640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc004800 0x7f82fc004c80 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f830000c240 tx=0x7f82fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dc5e505ff66f7fc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.964+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300007d10 con 0x7f82fc004800 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.964+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8300005ad0 con 0x7f82fc004800 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.964+0000 7f8316000640 1 -- 172.21.15.165:0/3662619865 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300007070 con 0x7f82fc004800 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317e00640 1 -- 172.21.15.165:0/3662619865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc004800 msgr2=0x7f82fc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317e00640 1 --2- 172.21.15.165:0/3662619865 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f82fc004800 0x7f82fc004c80 secure :-1 s=READY pgs=2894 cs=0 l=1 rev1=1 crypto rx=0x7f830000c240 tx=0x7f82fc005830 comp rx=0 tx=0).stop 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317400640 1 -- 172.21.15.165:0/3662619865 reap_dead start 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317e00640 1 -- 172.21.15.165:0/3662619865 shutdown_connections 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317e00640 1 -- 172.21.15.165:0/3662619865 >> 172.21.15.165:0/3662619865 conn(0x7f831806abe0 msgr2=0x7f831806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317e00640 1 -- 172.21.15.165:0/3662619865 shutdown_connections 2024-04-25T10:22:12.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.965+0000 7f8317e00640 1 -- 172.21.15.165:0/3662619865 wait complete. 2024-04-25T10:22:12.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.967+0000 7f8317e00640 1 Processor -- start 2024-04-25T10:22:12.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.967+0000 7f8317e00640 1 -- start start 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc004800 0x7f831815ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83180b25e0 0x7f83180b2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 0x7f831815b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83180cdbb0 con 0x7f831815abf0 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83180cda30 con 0x7f83180b25e0 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83180cdd30 con 0x7f82fc004800 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 0x7f831815b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 0x7f831815b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33600/0 (socket says 172.21.15.165:33600) 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.968+0000 7f8317400640 1 -- 172.21.15.165:0/1797818055 learned_addr learned my addr 172.21.15.165:0/1797818055 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8316a00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc004800 0x7f831815ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f830fe00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83180b25e0 0x7f83180b2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317400640 1 -- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc004800 msgr2=0x7f831815ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317400640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc004800 0x7f831815ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317400640 1 -- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83180b25e0 msgr2=0x7f83180b2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317400640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83180b25e0 0x7f83180b2a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317400640 1 -- 172.21.15.165:0/1797818055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180b2fa0 con 0x7f831815abf0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317400640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 0x7f831815b030 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f83180cef20 tx=0x7f8310004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310019070 con 0x7f831815abf0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8310007d30 con 0x7f831815abf0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8310014640 con 0x7f831815abf0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83180b3230 con 0x7f831815abf0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.969+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8318132970 con 0x7f831815abf0 2024-04-25T10:22:12.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.971+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8318073990 con 0x7f831815abf0 2024-04-25T10:22:12.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.972+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f831001c070 con 0x7f831815abf0 2024-04-25T10:22:12.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.973+0000 7f830ea00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82f406b500 0x7f82f406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:12.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.973+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8310089b60 con 0x7f831815abf0 2024-04-25T10:22:12.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.973+0000 7f8316a00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82f406b500 0x7f82f406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:12.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.974+0000 7f8316a00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82f406b500 0x7f82f406d9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8308007fd0 tx=0x7f8308009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:12.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:12.977+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f83100584b0 con 0x7f831815abf0 2024-04-25T10:22:13.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.168+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8318141cb0 con 0x7f82f406b500 2024-04-25T10:22:13.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.170+0000 7f830ea00640 1 -- 172.21.15.165:0/1797818055 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13525 (secure 0 0 0) 0x7f8318141cb0 con 0x7f82f406b500 2024-04-25T10:22:13.173 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:13.173 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:22:13.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.173+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82f406b500 msgr2=0x7f82f406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.173+0000 7f8317e00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82f406b500 0x7f82f406d9c0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f8308007fd0 tx=0x7f8308009040 comp rx=0 tx=0).stop 2024-04-25T10:22:13.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.173+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 msgr2=0x7f831815b030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.173+0000 7f8317e00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 0x7f831815b030 secure :-1 s=READY pgs=2895 cs=0 l=1 rev1=1 crypto rx=0x7f83180cef20 tx=0x7f8310004760 comp rx=0 tx=0).stop 2024-04-25T10:22:13.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.173+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 shutdown_connections 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.173+0000 7f8317e00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f82f406b500 0x7f82f406d9c0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.174+0000 7f8317e00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f831815abf0 0x7f831815b030 unknown :-1 s=CLOSED pgs=2895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.174+0000 7f8317e00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f83180b25e0 0x7f83180b2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.174+0000 7f8317e00640 1 --2- 172.21.15.165:0/1797818055 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f82fc004800 0x7f831815ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.174+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 >> 172.21.15.165:0/1797818055 conn(0x7f831806abe0 msgr2=0x7f83180b17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.174+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 shutdown_connections 2024-04-25T10:22:13.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.174+0000 7f8317e00640 1 -- 172.21.15.165:0/1797818055 wait complete. 2024-04-25T10:22:13.195 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":804,"stamp":"2024-04-25T10:22:12.544175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":546556,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":525363,"kb_avail":754428164,"statfs":{"total":773094113280,"available":772534439936,"internally_reserved":0,"allocated":21569536,"data_stored":10778368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":537972274},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624444,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-22,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406045"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":514,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:22:10.780373+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:22:10.780373+0000","last_peered":"2024-04-25T10:22:10.780373+0000","last_clean":"2024-04-25T10:22:10.780373+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:22:10.780373+0000","last_undegraded":"2024-04-25T10:22:10.780373+0000","last_fullsized":"2024-04-25T10:22:10.780373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575096,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64780,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307060,"statfs":{"total":96636764160,"available":96570429440,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60040,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575096,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55880,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575096,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87748,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284092,"statfs":{"total":96636764160,"available":96546910208,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87752,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284088,"statfs":{"total":96636764160,"available":96546906112,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575095,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64200,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307640,"statfs":{"total":96636764160,"available":96571023360,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575096,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60036,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575096,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66120,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305720,"statfs":{"total":96636764160,"available":96569057280,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:13.195 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:13.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.393+0000 7f3bf0800640 1 Processor -- start 2024-04-25T10:22:13.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.393+0000 7f3bf0800640 1 -- start start 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.394+0000 7f3bf0800640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f3be8164970 0x7f3be8166d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.394+0000 7f3bf0800640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be8144b30 con 0x7f3be8144750 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.394+0000 7f3bf0800640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3be80712d0 con 0x7f3be80ce920 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.394+0000 7f3bf0800640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3be80715b0 con 0x7f3be8164970 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3bede00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f3be8144750 0x7f3be806e5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:60992/0 (socket says 172.21.15.165:60992) 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3bede00640 1 -- 172.21.15.165:0/3399360243 learned_addr learned my addr 172.21.15.165:0/3399360243 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:13.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3bee800640 1 --2- 172.21.15.165:0/3399360243 >> v2:172.21.15.172:3300/0 conn(0x7f3be8164970 0x7f3be8166d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985722933 0 0) 0x7f3be8144b30 con 0x7f3be8144750 2024-04-25T10:22:13.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd0003520 con 0x7f3be8144750 2024-04-25T10:22:13.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3193962668 0 0) 0x7f3bd0003520 con 0x7f3be8144750 2024-04-25T10:22:13.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 >> v2:172.21.15.172:3300/0 conn(0x7f3be8164970 msgr2=0x7f3be8166d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3beca00640 1 --2- 172.21.15.165:0/3399360243 >> v2:172.21.15.172:3300/0 conn(0x7f3be8164970 0x7f3be8166d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.395+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 >> v1:172.21.15.172:6789/0 conn(0x7f3be80ce920 legacy=0x7f3be80cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8071890 con 0x7f3be8144750 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f3bd4001670 con 0x7f3be8144750 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 >> v1:172.21.15.165:6789/0 conn(0x7f3be8144750 legacy=0x7f3be806e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd00039b0 0x7f3bd0003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be80715b0 con 0x7f3bd0004800 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd000c160 con 0x7f3bd00051c0 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bd000c360 con 0x7f3bd00039b0 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bed400640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bede00640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd00039b0 0x7f3bd0003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bee800640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bed400640 1 -- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd00039b0 msgr2=0x7f3bd0003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bed400640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd00039b0 0x7f3bd0003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bed400640 1 -- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bd00051c0 msgr2=0x7f3bd000b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.396+0000 7f3bed400640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3bd00051c0 0x7f3bd000b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.397+0000 7f3bed400640 1 -- 172.21.15.165:0/3399360243 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd0003520 con 0x7f3bd0004800 2024-04-25T10:22:13.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.397+0000 7f3bee800640 1 -- 172.21.15.165:0/3399360243 reap_dead start 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3bed400640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc001750 tx=0x7f3bd0005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c427f0fe23e0fd1e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc001370 con 0x7f3bd0004800 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc0052d0 con 0x7f3bd0004800 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3beca00640 1 -- 172.21.15.165:0/3399360243 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc02abc0 con 0x7f3bd0004800 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3bf0800640 1 -- 172.21.15.165:0/3399360243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bd0004800 msgr2=0x7f3bd0004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3bf0800640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 secure :-1 s=READY pgs=2897 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc001750 tx=0x7f3bd0005830 comp rx=0 tx=0).stop 2024-04-25T10:22:13.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.398+0000 7f3bf0800640 1 -- 172.21.15.165:0/3399360243 shutdown_connections 2024-04-25T10:22:13.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.399+0000 7f3bf0800640 1 --2- 172.21.15.165:0/3399360243 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3bd0004800 0x7f3bd0004c80 unknown :-1 s=CLOSED pgs=2897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.399+0000 7f3bf0800640 1 -- 172.21.15.165:0/3399360243 >> 172.21.15.165:0/3399360243 conn(0x7f3be806abe0 msgr2=0x7f3be806aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.399+0000 7f3bf0800640 1 -- 172.21.15.165:0/3399360243 shutdown_connections 2024-04-25T10:22:13.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.399+0000 7f3bf0800640 1 -- 172.21.15.165:0/3399360243 wait complete. 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.400+0000 7f3bf0800640 1 Processor -- start 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.401+0000 7f3bf0800640 1 -- start start 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.401+0000 7f3bf0800640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd0004800 0x7f3be813c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bf0800640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3be81426c0 0x7f3be81f7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bf0800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 0x7f3be81f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bf0800640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be81455d0 con 0x7f3be8142b40 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bf0800640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be8145450 con 0x7f3be81426c0 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bf0800640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3be815c000 con 0x7f3bd0004800 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 0x7f3be81f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 0x7f3be81f7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33628/0 (socket says 172.21.15.165:33628) 2024-04-25T10:22:13.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 -- 172.21.15.165:0/338853172 learned_addr learned my addr 172.21.15.165:0/338853172 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bed400640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3be81426c0 0x7f3be81f7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 -- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd0004800 msgr2=0x7f3be813c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd0004800 0x7f3be813c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 -- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3be81426c0 msgr2=0x7f3be81f7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3be81426c0 0x7f3be81f7250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.402+0000 7f3bee800640 1 -- 172.21.15.165:0/338853172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be815a3b0 con 0x7f3be8142b40 2024-04-25T10:22:13.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.403+0000 7f3bee800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 0x7f3be81f7790 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f3be80b4aa0 tx=0x7f3be40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.403+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4011db0 con 0x7f3be8142b40 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.403+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be401b070 con 0x7f3be8142b40 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.403+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be4003990 con 0x7f3be8142b40 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.403+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be815a690 con 0x7f3be8142b40 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.403+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be81f7eb0 con 0x7f3be8142b40 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.404+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be8073990 con 0x7f3be8142b40 2024-04-25T10:22:13.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.405+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3be4003b30 con 0x7f3be8142b40 2024-04-25T10:22:13.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.406+0000 7f3be2000640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3bc806b4d0 0x7f3bc806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.406+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f3be408aa50 con 0x7f3be8142b40 2024-04-25T10:22:13.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.406+0000 7f3bede00640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3bc806b4d0 0x7f3bc806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.407+0000 7f3bede00640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3bc806b4d0 0x7f3bc806d990 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4001200 tx=0x7f3bd4001d70 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.410+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3be4003e20 con 0x7f3be8142b40 2024-04-25T10:22:13.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.605+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3be80d0510 con 0x7f3bc806b4d0 2024-04-25T10:22:13.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.606+0000 7f3be2000640 1 -- 172.21.15.165:0/338853172 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13525 (secure 0 0 0) 0x7f3be80d0510 con 0x7f3bc806b4d0 2024-04-25T10:22:13.609 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:13.609 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3bc806b4d0 msgr2=0x7f3bc806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3bc806b4d0 0x7f3bc806d990 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4001200 tx=0x7f3bd4001d70 comp rx=0 tx=0).stop 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 msgr2=0x7f3be81f7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 0x7f3be81f7790 secure :-1 s=READY pgs=2898 cs=0 l=1 rev1=1 crypto rx=0x7f3be80b4aa0 tx=0x7f3be40115a0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 shutdown_connections 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f3bc806b4d0 0x7f3bc806d990 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f3be8142b40 0x7f3be81f7790 unknown :-1 s=CLOSED pgs=2898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f3be81426c0 0x7f3be81f7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 --2- 172.21.15.165:0/338853172 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3bd0004800 0x7f3be813c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.609+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 >> 172.21.15.165:0/338853172 conn(0x7f3be806abe0 msgr2=0x7f3be806e7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.610+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 shutdown_connections 2024-04-25T10:22:13.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.610+0000 7f3bf0800640 1 -- 172.21.15.165:0/338853172 wait complete. 2024-04-25T10:22:13.630 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":804,"stamp":"2024-04-25T10:22:12.544175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":546556,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":525363,"kb_avail":754428164,"statfs":{"total":773094113280,"available":772534439936,"internally_reserved":0,"allocated":21569536,"data_stored":10778368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":537972274},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624444,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-22,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406045"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":514,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:22:10.780373+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:22:10.780373+0000","last_peered":"2024-04-25T10:22:10.780373+0000","last_clean":"2024-04-25T10:22:10.780373+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:22:10.780373+0000","last_undegraded":"2024-04-25T10:22:10.780373+0000","last_fullsized":"2024-04-25T10:22:10.780373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575096,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64780,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307060,"statfs":{"total":96636764160,"available":96570429440,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60040,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575096,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55880,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575096,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87748,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284092,"statfs":{"total":96636764160,"available":96546910208,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87752,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284088,"statfs":{"total":96636764160,"available":96546906112,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575095,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64200,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307640,"statfs":{"total":96636764160,"available":96571023360,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575096,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60036,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575096,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66120,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305720,"statfs":{"total":96636764160,"available":96569057280,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:13.630 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-25T10:22:13.630 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:13.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.828+0000 7fb4c6600640 1 Processor -- start 2024-04-25T10:22:13.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.828+0000 7fb4c6600640 1 -- start start 2024-04-25T10:22:13.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.829+0000 7fb4c6600640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb4c00ce920 0x7fb4c00cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4c6600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c0144b30 con 0x7fb4c006d290 2024-04-25T10:22:13.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4c6600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4c00712d0 con 0x7fb4c0144750 2024-04-25T10:22:13.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4c6600640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4c00715b0 con 0x7fb4c00ce920 2024-04-25T10:22:13.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4bfe00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb4c006d290 0x7fb4c006d6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:32768/0 (socket says 172.21.15.165:32768) 2024-04-25T10:22:13.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4bfe00640 1 -- 172.21.15.165:0/3989530587 learned_addr learned my addr 172.21.15.165:0/3989530587 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:13.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/3989530587 >> v2:172.21.15.172:3300/0 conn(0x7fb4c00ce920 0x7fb4c00cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1159800395 0 0) 0x7fb4c0144b30 con 0x7fb4c006d290 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.830+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4a4003520 con 0x7fb4c006d290 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3490939495 0 0) 0x7fb4a4003520 con 0x7fb4c006d290 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 >> v2:172.21.15.172:3300/0 conn(0x7fb4c00ce920 msgr2=0x7fb4c00cedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 --2- 172.21.15.165:0/3989530587 >> v2:172.21.15.172:3300/0 conn(0x7fb4c00ce920 0x7fb4c00cedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 >> v1:172.21.15.172:6789/0 conn(0x7fb4c0144750 legacy=0x7fb4c006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c0071890 con 0x7fb4c006d290 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb4b80019f0 con 0x7fb4c006d290 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 >> v1:172.21.15.165:6789/0 conn(0x7fb4c006d290 legacy=0x7fb4c006d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40039b0 0x7fb4a4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4004800 0x7fb4a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a40051c0 0x7fb4a400b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c00715b0 con 0x7fb4a4004800 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a400c060 con 0x7fb4a40039b0 2024-04-25T10:22:13.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a400c260 con 0x7fb4a40051c0 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.831+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4004800 0x7fb4a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4bf400640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40039b0 0x7fb4a4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4b7e00640 1 -- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a40051c0 msgr2=0x7fb4a400b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a40051c0 0x7fb4a400b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4b7e00640 1 -- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40039b0 msgr2=0x7fb4a4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40039b0 0x7fb4a4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4b7e00640 1 -- 172.21.15.165:0/3989530587 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a4003520 con 0x7fb4a4004800 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4bfe00640 1 -- 172.21.15.165:0/3989530587 reap_dead start 2024-04-25T10:22:13.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.832+0000 7fb4bf400640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4a40039b0 0x7fb4a4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.833+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4004800 0x7fb4a4004c80 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009ac0 tx=0x7fb4a4005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d16db843af1b84c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.833+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8036070 con 0x7fb4a4004800 2024-04-25T10:22:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.833+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a8031400 con 0x7fb4a4004800 2024-04-25T10:22:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.833+0000 7fb4bea00640 1 -- 172.21.15.165:0/3989530587 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8009690 con 0x7fb4a4004800 2024-04-25T10:22:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 -- 172.21.15.165:0/3989530587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4004800 msgr2=0x7fb4a4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4004800 0x7fb4a4004c80 secure :-1 s=READY pgs=2900 cs=0 l=1 rev1=1 crypto rx=0x7fb4a8009ac0 tx=0x7fb4a4005830 comp rx=0 tx=0).stop 2024-04-25T10:22:13.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 -- 172.21.15.165:0/3989530587 shutdown_connections 2024-04-25T10:22:13.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 --2- 172.21.15.165:0/3989530587 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4a4004800 0x7fb4a4004c80 unknown :-1 s=CLOSED pgs=2900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 -- 172.21.15.165:0/3989530587 >> 172.21.15.165:0/3989530587 conn(0x7fb4c006abe0 msgr2=0x7fb4c006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:13.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 -- 172.21.15.165:0/3989530587 shutdown_connections 2024-04-25T10:22:13.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.834+0000 7fb4c6600640 1 -- 172.21.15.165:0/3989530587 wait complete. 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.835+0000 7fb4c6600640 1 Processor -- start 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.836+0000 7fb4c6600640 1 -- start start 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.836+0000 7fb4c6600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a4004800 0x7fb4c01f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4c6600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c00add20 0x7fb4c014af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4c6600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 0x7fb4c013f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4c6600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c01455d0 con 0x7fb4c013f0b0 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4c6600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c0145450 con 0x7fb4c00add20 2024-04-25T10:22:13.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4c6600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c015c000 con 0x7fb4a4004800 2024-04-25T10:22:13.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 0x7fb4c013f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 0x7fb4c013f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33652/0 (socket says 172.21.15.165:33652) 2024-04-25T10:22:13.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 -- 172.21.15.165:0/612927266 learned_addr learned my addr 172.21.15.165:0/612927266 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c00add20 0x7fb4c014af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bf400640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a4004800 0x7fb4c01f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 -- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a4004800 msgr2=0x7fb4c01f20a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a4004800 0x7fb4c01f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 -- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c00add20 msgr2=0x7fb4c014af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c00add20 0x7fb4c014af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4bfe00640 1 -- 172.21.15.165:0/612927266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4c01f5040 con 0x7fb4c013f0b0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.837+0000 7fb4b7e00640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c00add20 0x7fb4c014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.838+0000 7fb4bfe00640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 0x7fb4c013f530 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7fb4c006ba60 tx=0x7fb4b800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.838+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800a610 con 0x7fb4c013f0b0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.838+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4b800a7b0 con 0x7fb4c013f0b0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.838+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4b800ab80 con 0x7fb4c013f0b0 2024-04-25T10:22:13.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.838+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4c01f5320 con 0x7fb4c013f0b0 2024-04-25T10:22:13.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.838+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4c01f58e0 con 0x7fb4c013f0b0 2024-04-25T10:22:13.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.839+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4c0073990 con 0x7fb4c013f0b0 2024-04-25T10:22:13.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.840+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb4b800ad20 con 0x7fb4c013f0b0 2024-04-25T10:22:13.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.841+0000 7fb4b7400640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb49c06b500 0x7fb49c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:13.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.841+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb4b8089670 con 0x7fb4c013f0b0 2024-04-25T10:22:13.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.841+0000 7fb4bf400640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb49c06b500 0x7fb49c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:13.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.842+0000 7fb4bf400640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb49c06b500 0x7fb49c06d9c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0000c00 tx=0x7fb4b0009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:13.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:13.845+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb4b801a030 con 0x7fb4c013f0b0 2024-04-25T10:22:14.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.037+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb4c006bc00 con 0x7fb49c06b500 2024-04-25T10:22:14.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.038+0000 7fb4b7400640 1 -- 172.21.15.165:0/612927266 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13525 (secure 0 0 0) 0x7fb4c006bc00 con 0x7fb49c06b500 2024-04-25T10:22:14.041 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:14.041 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:22:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb49c06b500 msgr2=0x7fb49c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb49c06b500 0x7fb49c06d9c0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fb4b0000c00 tx=0x7fb4b0009040 comp rx=0 tx=0).stop 2024-04-25T10:22:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 msgr2=0x7fb4c013f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 0x7fb4c013f530 secure :-1 s=READY pgs=2901 cs=0 l=1 rev1=1 crypto rx=0x7fb4c006ba60 tx=0x7fb4b800cc20 comp rx=0 tx=0).stop 2024-04-25T10:22:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 shutdown_connections 2024-04-25T10:22:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb49c06b500 0x7fb49c06d9c0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb4c013f0b0 0x7fb4c013f530 unknown :-1 s=CLOSED pgs=2901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb4c00add20 0x7fb4c014af90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 --2- 172.21.15.165:0/612927266 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb4a4004800 0x7fb4c01f20a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.041+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 >> 172.21.15.165:0/612927266 conn(0x7fb4c006abe0 msgr2=0x7fb4c00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.042+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 shutdown_connections 2024-04-25T10:22:14.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.042+0000 7fb4c6600640 1 -- 172.21.15.165:0/612927266 wait complete. 2024-04-25T10:22:14.062 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":804,"stamp":"2024-04-25T10:22:12.544175+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":84,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":546556,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":525363,"kb_avail":754428164,"statfs":{"total":773094113280,"available":772534439936,"internally_reserved":0,"allocated":21569536,"data_stored":10778368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":537972274},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12624444,"num_objects":-28,"num_object_clones":0,"num_object_copies":-56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-28,"num_whiteouts":0,"num_read":-99,"num_read_kb":-105,"num_write":-96,"num_write_kb":-12361,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":-22,"num_keys_recovered":0,"num_objects_omap":-16,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.406045"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":514,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:22:10.780373+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:22:10.780373+0000","last_peered":"2024-04-25T10:22:10.780373+0000","last_clean":"2024-04-25T10:22:10.780373+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:22:10.780373+0000","last_undegraded":"2024-04-25T10:22:10.780373+0000","last_fullsized":"2024-04-25T10:22:10.780373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575096,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64780,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307060,"statfs":{"total":96636764160,"available":96570429440,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60040,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575096,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55880,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575096,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87748,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284092,"statfs":{"total":96636764160,"available":96546910208,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87752,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284088,"statfs":{"total":96636764160,"available":96546906112,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575095,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64200,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307640,"statfs":{"total":96636764160,"available":96571023360,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575096,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60036,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575096,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66120,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305720,"statfs":{"total":96636764160,"available":96569057280,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:14.062 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-25T10:22:14.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.251+0000 7f9a09c00640 1 Processor -- start 2024-04-25T10:22:14.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.251+0000 7f9a09c00640 1 -- start start 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a09c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 0x7f9a040b93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a09c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a040d06c0 0x7f9a040d23e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a09c00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a04161df0 con 0x7f9a041615a0 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a09c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040d1f50 con 0x7f9a040d06c0 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a09c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041620d0 con 0x7f9a0415db90 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a03400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f9a041615a0 0x7f9a040cffb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:32770/0 (socket says 172.21.15.165:32770) 2024-04-25T10:22:14.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a03400640 1 -- 172.21.15.165:0/3170305312 learned_addr learned my addr 172.21.15.165:0/3170305312 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a09200640 1 --2- 172.21.15.165:0/3170305312 >> v2:172.21.15.172:3300/0 conn(0x7f9a040d06c0 0x7f9a040d23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.253+0000 7f9a03e00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 0x7f9a040b93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1474013578 0 0) 0x7f9a04161df0 con 0x7f9a041615a0 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99e8003520 con 0x7f9a041615a0 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2370021563 0 0) 0x7f99e8003520 con 0x7f9a041615a0 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 msgr2=0x7f9a040b93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 0x7f9a040b93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 >> v2:172.21.15.172:3300/0 conn(0x7f9a040d06c0 msgr2=0x7f9a040d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 --2- 172.21.15.165:0/3170305312 >> v2:172.21.15.172:3300/0 conn(0x7f9a040d06c0 0x7f9a040d23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a041623b0 con 0x7f9a041615a0 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7f99f0001670 con 0x7f9a041615a0 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 >> v1:172.21.15.165:6789/0 conn(0x7f9a041615a0 legacy=0x7f9a040cffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e80039b0 0x7f99e8003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99e8004800 0x7f99e8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 0x7f99e80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040d1f50 con 0x7f99e8004800 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a041620d0 con 0x7f99e80039b0 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.254+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e800b9c0 con 0x7f9a0415db90 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03400640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99e8004800 0x7f99e8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03e00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e80039b0 0x7f99e8003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a09200640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 0x7f99e80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03400640 1 -- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 msgr2=0x7f99e80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03400640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a0415db90 0x7f99e80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03400640 1 -- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e80039b0 msgr2=0x7f99e8003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03400640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e80039b0 0x7f99e8003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.255+0000 7f9a03400640 1 -- 172.21.15.165:0/3170305312 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e8003520 con 0x7f99e8004800 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.256+0000 7f9a03400640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99e8004800 0x7f99e8004c80 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c210 tx=0x7f99e8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af313c56d61eb70b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.256+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007d50 con 0x7f99e8004800 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.256+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0005af0 con 0x7f99e8004800 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.256+0000 7f9a02a00640 1 -- 172.21.15.165:0/3170305312 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0007070 con 0x7f99e8004800 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09c00640 1 -- 172.21.15.165:0/3170305312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99e8004800 msgr2=0x7f99e8004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3170305312 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f99e8004800 0x7f99e8004c80 secure :-1 s=READY pgs=2903 cs=0 l=1 rev1=1 crypto rx=0x7f99f000c210 tx=0x7f99e8005830 comp rx=0 tx=0).stop 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09200640 1 -- 172.21.15.165:0/3170305312 reap_dead start 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09c00640 1 -- 172.21.15.165:0/3170305312 shutdown_connections 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09c00640 1 -- 172.21.15.165:0/3170305312 >> 172.21.15.165:0/3170305312 conn(0x7f9a040b52b0 msgr2=0x7f9a040b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09c00640 1 -- 172.21.15.165:0/3170305312 shutdown_connections 2024-04-25T10:22:14.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.257+0000 7f9a09c00640 1 -- 172.21.15.165:0/3170305312 wait complete. 2024-04-25T10:22:14.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.259+0000 7f9a09c00640 1 Processor -- start 2024-04-25T10:22:14.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.259+0000 7f9a09c00640 1 -- start start 2024-04-25T10:22:14.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a09c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e8004800 0x7f9a0415f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a09c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 0x7f9a04157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a09c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040cb080 0x7f9a040cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a09c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04161ba0 con 0x7f9a040ca870 2024-04-25T10:22:14.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a09c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04161a20 con 0x7f99e8004800 2024-04-25T10:22:14.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a09c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040b68d0 con 0x7f9a040cb080 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a03400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 0x7f9a04157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a03400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 0x7f9a04157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33660/0 (socket says 172.21.15.165:33660) 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.260+0000 7f9a03400640 1 -- 172.21.15.165:0/3619675522 learned_addr learned my addr 172.21.15.165:0/3619675522 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a09200640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040cb080 0x7f9a040cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03e00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e8004800 0x7f9a0415f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03400640 1 -- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040cb080 msgr2=0x7f9a040cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03400640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040cb080 0x7f9a040cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03400640 1 -- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e8004800 msgr2=0x7f9a0415f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03400640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e8004800 0x7f9a0415f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03400640 1 -- 172.21.15.165:0/3619675522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0415f630 con 0x7f9a040ca870 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03e00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e8004800 0x7f9a0415f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a09200640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040cb080 0x7f9a040cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.261+0000 7f9a03400640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 0x7f9a04157230 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034040 tx=0x7f99f0001a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.262+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0042070 con 0x7f9a040ca870 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.262+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f0007480 con 0x7f9a040ca870 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.262+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f0001c90 con 0x7f9a040ca870 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.262+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a04157770 con 0x7f9a040ca870 2024-04-25T10:22:14.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.262+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a0415fbb0 con 0x7f9a040ca870 2024-04-25T10:22:14.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.263+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a040b95b0 con 0x7f9a040ca870 2024-04-25T10:22:14.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.264+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f99f0045070 con 0x7f9a040ca870 2024-04-25T10:22:14.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.265+0000 7f99fbe00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:14.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.265+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f99f00aaca0 con 0x7f9a040ca870 2024-04-25T10:22:14.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.265+0000 7f9a03e00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:14.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.266+0000 7f9a03e00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e406b500 0x7f99e406d9c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9a04160cc0 tx=0x7f99f400b040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:14.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.270+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a040b95b0 con 0x7f9a040ca870 2024-04-25T10:22:14.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.466+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9a04073990 con 0x7f9a040ca870 2024-04-25T10:22:14.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.467+0000 7f99fbe00640 1 -- 172.21.15.165:0/3619675522 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+13473 (secure 0 0 0) 0x7f99f0005ac0 con 0x7f9a040ca870 2024-04-25T10:22:14.469 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:14.470 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":201,"fsid":"55d22d87-b25b-4ccd-bc45-d797a973b3c0","created":"2024-04-25T10:01:39.001717+0000","modified":"2024-04-25T10:22:09.434588+0000","last_up_change":"2024-04-25T10:01:51.534106+0000","last_in_change":"2024-04-25T10:01:42.431260+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":30,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-25T10:01:52.195367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"1aedd4b5-8325-4232-9d73-f008e988fa3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6801","nonce":4162285384}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6803","nonce":4162285384}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6807","nonce":4162285384}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":4162285384},{"type":"v1","addr":"172.21.15.165:6805","nonce":4162285384}]},"public_addr":"172.21.15.165:6801/4162285384","cluster_addr":"172.21.15.165:6803/4162285384","heartbeat_back_addr":"172.21.15.165:6807/4162285384","heartbeat_front_addr":"172.21.15.165:6805/4162285384","state":["exists","up"]},{"osd":1,"uuid":"e91c7f9f-4fa9-468f-b6e2-afd7c09b8886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6809","nonce":4150887630}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6811","nonce":4150887630}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6815","nonce":4150887630}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":4150887630},{"type":"v1","addr":"172.21.15.165:6813","nonce":4150887630}]},"public_addr":"172.21.15.165:6809/4150887630","cluster_addr":"172.21.15.165:6811/4150887630","heartbeat_back_addr":"172.21.15.165:6815/4150887630","heartbeat_front_addr":"172.21.15.165:6813/4150887630","state":["exists","up"]},{"osd":2,"uuid":"ff3dc377-50c0-4312-a25f-e4148ffbc34f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6825","nonce":3063902068}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6826","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6827","nonce":3063902068}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6830","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6831","nonce":3063902068}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6828","nonce":3063902068},{"type":"v1","addr":"172.21.15.165:6829","nonce":3063902068}]},"public_addr":"172.21.15.165:6825/3063902068","cluster_addr":"172.21.15.165:6827/3063902068","heartbeat_back_addr":"172.21.15.165:6831/3063902068","heartbeat_front_addr":"172.21.15.165:6829/3063902068","state":["exists","up"]},{"osd":3,"uuid":"f8d3e5bf-8745-4d91-9248-9e088389dc1a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6817","nonce":995850359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6819","nonce":995850359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6823","nonce":995850359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":995850359},{"type":"v1","addr":"172.21.15.165:6821","nonce":995850359}]},"public_addr":"172.21.15.165:6817/995850359","cluster_addr":"172.21.15.165:6819/995850359","heartbeat_back_addr":"172.21.15.165:6823/995850359","heartbeat_front_addr":"172.21.15.165:6821/995850359","state":["exists","up"]},{"osd":4,"uuid":"8fc4af7f-9417-4389-8c94-5a457e3ab6d4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":196,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6817","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6819","nonce":928908638}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6821","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6823","nonce":928908638}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6829","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6831","nonce":928908638}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6825","nonce":928908638},{"type":"v1","addr":"172.21.15.172:6827","nonce":928908638}]},"public_addr":"172.21.15.172:6819/928908638","cluster_addr":"172.21.15.172:6823/928908638","heartbeat_back_addr":"172.21.15.172:6831/928908638","heartbeat_front_addr":"172.21.15.172:6827/928908638","state":["exists","up"]},{"osd":5,"uuid":"9c66e050-67b9-48ca-b29b-4f3620e3cd10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6800","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6801","nonce":4199573342}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6802","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6803","nonce":4199573342}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6806","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6807","nonce":4199573342}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6804","nonce":4199573342},{"type":"v1","addr":"172.21.15.172:6805","nonce":4199573342}]},"public_addr":"172.21.15.172:6801/4199573342","cluster_addr":"172.21.15.172:6803/4199573342","heartbeat_back_addr":"172.21.15.172:6807/4199573342","heartbeat_front_addr":"172.21.15.172:6805/4199573342","state":["exists","up"]},{"osd":6,"uuid":"640aa33e-e6c9-4b58-8101-5bd88c38fbc8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6808","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6809","nonce":4001543510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6810","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6811","nonce":4001543510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6814","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6815","nonce":4001543510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6812","nonce":4001543510},{"type":"v1","addr":"172.21.15.172:6813","nonce":4001543510}]},"public_addr":"172.21.15.172:6809/4001543510","cluster_addr":"172.21.15.172:6811/4001543510","heartbeat_back_addr":"172.21.15.172:6815/4001543510","heartbeat_front_addr":"172.21.15.172:6813/4001543510","state":["exists","up"]},{"osd":7,"uuid":"d1f4680d-75ad-4400-822c-95b0e6898501","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":193,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6816","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6818","nonce":3337160074}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6820","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6822","nonce":3337160074}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6828","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6830","nonce":3337160074}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6824","nonce":3337160074},{"type":"v1","addr":"172.21.15.172:6826","nonce":3337160074}]},"public_addr":"172.21.15.172:6818/3337160074","cluster_addr":"172.21.15.172:6822/3337160074","heartbeat_back_addr":"172.21.15.172:6830/3337160074","heartbeat_front_addr":"172.21.15.172:6826/3337160074","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.765553+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.777985+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.630175+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.564466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.881107+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.803951+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.595092+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-25T10:01:47.640307+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.165:6835/2156983703":"2024-04-26T10:22:06.130048+0000","172.21.15.165:6837/2156983703":"2024-04-26T10:22:06.130048+0000","172.21.15.165:6837/3988146762":"2024-04-26T10:20:50.950335+0000","172.21.15.165:6835/3988146762":"2024-04-26T10:20:50.950335+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.469+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e406b500 msgr2=0x7f99e406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.469+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e406b500 0x7f99e406d9c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f9a04160cc0 tx=0x7f99f400b040 comp rx=0 tx=0).stop 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.469+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 msgr2=0x7f9a04157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.469+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 0x7f9a04157230 secure :-1 s=READY pgs=2904 cs=0 l=1 rev1=1 crypto rx=0x7f99f0034040 tx=0x7f99f0001a40 comp rx=0 tx=0).stop 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 shutdown_connections 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f99e406b500 0x7f99e406d9c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a040cb080 0x7f9a040cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a040ca870 0x7f9a04157230 unknown :-1 s=CLOSED pgs=2904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 --2- 172.21.15.165:0/3619675522 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f99e8004800 0x7f9a0415f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:14.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 >> 172.21.15.165:0/3619675522 conn(0x7f9a040b52b0 msgr2=0x7f9a040d2d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:14.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 shutdown_connections 2024-04-25T10:22:14.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:14.470+0000 7f9a09c00640 1 -- 172.21.15.165:0/3619675522 wait complete. 2024-04-25T10:22:15.492 INFO:tasks.ceph:Scrubbing osd.0 2024-04-25T10:22:15.492 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:15.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.703+0000 7fb322200640 1 Processor -- start 2024-04-25T10:22:15.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.704+0000 7fb322200640 1 -- start start 2024-04-25T10:22:15.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.705+0000 7fb322200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 0x7fb31c0cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.705+0000 7fb322200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31c15e110 con 0x7fb31c06bff0 2024-04-25T10:22:15.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.705+0000 7fb322200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31c0b2fd0 con 0x7fb31c0b5b40 2024-04-25T10:22:15.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.705+0000 7fb322200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c06d000 con 0x7fb31c15dd30 2024-04-25T10:22:15.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.705+0000 7fb31be00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb31c06bff0 0x7fb31c0b5430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:32776/0 (socket says 172.21.15.165:32776) 2024-04-25T10:22:15.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31be00640 1 -- 172.21.15.165:0/3784863500 learned_addr learned my addr 172.21.15.165:0/3784863500 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb320e00640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 0x7fb31c0cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288127787 0 0) 0x7fb31c15e110 con 0x7fb31c06bff0 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc003520 con 0x7fb31c06bff0 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 72783672 0 0) 0x7fb2fc003520 con 0x7fb31c06bff0 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 msgr2=0x7fb31c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 0x7fb31c0cebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 >> v1:172.21.15.172:6789/0 conn(0x7fb31c0b5b40 legacy=0x7fb31c0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c06c480 con 0x7fb31c06bff0 2024-04-25T10:22:15.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb3040019f0 con 0x7fb31c06bff0 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.706+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 >> v1:172.21.15.165:6789/0 conn(0x7fb31c06bff0 legacy=0x7fb31c0b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31b400640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2fc0039e0 0x7fb2fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31b400640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc004800 0x7fb2fc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31b400640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 0x7fb2fc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c06d000 con 0x7fb2fc0039e0 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00ba70 con 0x7fb2fc004800 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00bc70 con 0x7fb31c15dd30 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb320e00640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2fc0039e0 0x7fb2fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb31be00640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc004800 0x7fb2fc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb320e00640 1 -- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 msgr2=0x7fb2fc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb320e00640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c15dd30 0x7fb2fc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb320e00640 1 -- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc004800 msgr2=0x7fb2fc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb320e00640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc004800 0x7fb2fc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.707+0000 7fb320e00640 1 -- 172.21.15.165:0/3784863500 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc003520 con 0x7fb2fc0039e0 2024-04-25T10:22:15.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.708+0000 7fb320e00640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2fc0039e0 0x7fb2fc003e00 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009a90 tx=0x7fb2fc005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7a6a94d3eea6e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c036070 con 0x7fb2fc0039e0 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c031400 con 0x7fb2fc0039e0 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb31b400640 1 -- 172.21.15.165:0/3784863500 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c009690 con 0x7fb2fc0039e0 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb322200640 1 -- 172.21.15.165:0/3784863500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2fc0039e0 msgr2=0x7fb2fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb322200640 1 --2- 172.21.15.165:0/3784863500 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb2fc0039e0 0x7fb2fc003e00 secure :-1 s=READY pgs=2906 cs=0 l=1 rev1=1 crypto rx=0x7fb30c009a90 tx=0x7fb2fc005830 comp rx=0 tx=0).stop 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb321800640 1 -- 172.21.15.165:0/3784863500 reap_dead start 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb322200640 1 -- 172.21.15.165:0/3784863500 shutdown_connections 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb322200640 1 -- 172.21.15.165:0/3784863500 >> 172.21.15.165:0/3784863500 conn(0x7fb31c0d0610 msgr2=0x7fb31c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb322200640 1 -- 172.21.15.165:0/3784863500 shutdown_connections 2024-04-25T10:22:15.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.709+0000 7fb322200640 1 -- 172.21.15.165:0/3784863500 wait complete. 2024-04-25T10:22:15.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.711+0000 7fb322200640 1 Processor -- start 2024-04-25T10:22:15.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.711+0000 7fb322200640 1 -- start start 2024-04-25T10:22:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.712+0000 7fb322200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc0039e0 0x7fb31c1f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.712+0000 7fb322200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c0caaf0 0x7fb31c0caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.712+0000 7fb322200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31c15faa0 0x7fb31c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.712+0000 7fb322200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c06cd20 con 0x7fb31c15faa0 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.712+0000 7fb322200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c06ca40 con 0x7fb2fc0039e0 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.712+0000 7fb322200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb31c0cebd0 con 0x7fb31c0caaf0 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31c15faa0 0x7fb31c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31c15faa0 0x7fb31c15ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33670/0 (socket says 172.21.15.165:33670) 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 -- 172.21.15.165:0/102308940 learned_addr learned my addr 172.21.15.165:0/102308940 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb320e00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc0039e0 0x7fb31c1f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb31be00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c0caaf0 0x7fb31c0caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 -- 172.21.15.165:0/102308940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c0caaf0 msgr2=0x7fb31c0caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb31c0caaf0 0x7fb31c0caf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 -- 172.21.15.165:0/102308940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc0039e0 msgr2=0x7fb31c1f39e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb2fc0039e0 0x7fb31c1f39e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 -- 172.21.15.165:0/102308940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb31c0cd650 con 0x7fb31c15faa0 2024-04-25T10:22:15.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.713+0000 7fb321800640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31c15faa0 0x7fb31c15ff20 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c390 tx=0x7fb31400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.714+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314007280 con 0x7fb31c15faa0 2024-04-25T10:22:15.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.714+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb314007420 con 0x7fb31c15faa0 2024-04-25T10:22:15.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.714+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb314004020 con 0x7fb31c15faa0 2024-04-25T10:22:15.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.714+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb31c0cd930 con 0x7fb31c15faa0 2024-04-25T10:22:15.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.714+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb31c15f580 con 0x7fb31c15faa0 2024-04-25T10:22:15.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.715+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb31c15f370 con 0x7fb31c15faa0 2024-04-25T10:22:15.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.716+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb314011070 con 0x7fb31c15faa0 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.717+0000 7fb318c00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.717+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb3140890c0 con 0x7fb31c15faa0 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.717+0000 7fb320e00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.717+0000 7fb318c00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fb2f8071a10 0x7fb2f8073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.717+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 --> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb2f8074520 con 0x7fb2f8071a10 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.717+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb314090050 con 0x7fb31c15faa0 2024-04-25T10:22:15.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.718+0000 7fb31be00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fb2f8071a10 0x7fb2f8073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.718+0000 7fb320e00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb30c033000 tx=0x7fb30c008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.718+0000 7fb31be00640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fb2f8071a10 0x7fb2f8073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.719+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== osd.0 v2:172.21.15.165:6800/4162285384 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb2f8074520 con 0x7fb2f8071a10 2024-04-25T10:22:15.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.733+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 --> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb31c073990 con 0x7fb2f8071a10 2024-04-25T10:22:15.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb318c00640 1 -- 172.21.15.165:0/102308940 <== osd.0 v2:172.21.15.165:6800/4162285384 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fb31c073990 con 0x7fb2f8071a10 2024-04-25T10:22:15.736 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:15.736 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:15.736 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fb2f8071a10 msgr2=0x7fb2f8073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6800/4162285384,v1:172.21.15.165:6801/4162285384] conn(0x7fb2f8071a10 0x7fb2f8073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 msgr2=0x7fb2f806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb2f806b4d0 0x7fb2f806d990 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fb30c033000 tx=0x7fb30c008030 comp rx=0 tx=0).stop 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31c15faa0 msgr2=0x7fb31c15ff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 --2- 172.21.15.165:0/102308940 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb31c15faa0 0x7fb31c15ff20 secure :-1 s=READY pgs=2907 cs=0 l=1 rev1=1 crypto rx=0x7fb31400c390 tx=0x7fb31400c850 comp rx=0 tx=0).stop 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb321800640 1 -- 172.21.15.165:0/102308940 reap_dead start 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 shutdown_connections 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.734+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 >> 172.21.15.165:0/102308940 conn(0x7fb31c0d0610 msgr2=0x7fb31c0b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.735+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 shutdown_connections 2024-04-25T10:22:15.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.735+0000 7fb322200640 1 -- 172.21.15.165:0/102308940 wait complete. 2024-04-25T10:22:15.755 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.950+0000 7fb3e9000640 1 Processor -- start 2024-04-25T10:22:15.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.951+0000 7fb3e9000640 1 -- start start 2024-04-25T10:22:15.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.952+0000 7fb3e9000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fb3e4161710 0x7fb3e40d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.952+0000 7fb3e9000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e415e000 con 0x7fb3e415dc20 2024-04-25T10:22:15.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.952+0000 7fb3e9000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3e40d1de0 con 0x7fb3e40ced80 2024-04-25T10:22:15.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.952+0000 7fb3e9000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3e4161c60 con 0x7fb3e4161710 2024-04-25T10:22:15.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.952+0000 7fb3e1e00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fb3e415dc20 0x7fb3e40ce670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:32778/0 (socket says 172.21.15.165:32778) 2024-04-25T10:22:15.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.952+0000 7fb3e1e00640 1 -- 172.21.15.165:0/3308222416 learned_addr learned my addr 172.21.15.165:0/3308222416 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:15.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e2800640 1 --2- 172.21.15.165:0/3308222416 >> v2:172.21.15.172:3300/0 conn(0x7fb3e4161710 0x7fb3e40d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 562854631 0 0) 0x7fb3e415e000 con 0x7fb3e415dc20 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3c8003520 con 0x7fb3e415dc20 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105735701 0 0) 0x7fb3c8003520 con 0x7fb3e415dc20 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 >> v2:172.21.15.172:3300/0 conn(0x7fb3e4161710 msgr2=0x7fb3e40d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 --2- 172.21.15.165:0/3308222416 >> v2:172.21.15.172:3300/0 conn(0x7fb3e4161710 0x7fb3e40d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 >> v1:172.21.15.172:6789/0 conn(0x7fb3e40ced80 legacy=0x7fb3e40d22e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4161f40 con 0x7fb3e415dc20 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fb3d4001670 con 0x7fb3e415dc20 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 >> v1:172.21.15.165:6789/0 conn(0x7fb3e415dc20 legacy=0x7fb3e40ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c8003a00 0x7fb3c8003e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c8004800 0x7fb3c8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c80051c0 0x7fb3c800b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4161c60 con 0x7fb3c80051c0 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c800c0c0 con 0x7fb3c8003a00 2024-04-25T10:22:15.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.953+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3c800c2c0 con 0x7fb3c8004800 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e2800640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c80051c0 0x7fb3c800b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e1e00640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c8003a00 0x7fb3c8003e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e1400640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c8004800 0x7fb3c8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e2800640 1 -- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c8004800 msgr2=0x7fb3c8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e2800640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c8004800 0x7fb3c8004c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e2800640 1 -- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c8003a00 msgr2=0x7fb3c8003e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e2800640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3c8003a00 0x7fb3c8003e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.954+0000 7fb3e2800640 1 -- 172.21.15.165:0/3308222416 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8003520 con 0x7fb3c80051c0 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.955+0000 7fb3e2800640 1 -- 172.21.15.165:0/3308222416 reap_dead start 2024-04-25T10:22:15.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.955+0000 7fb3e2800640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c80051c0 0x7fb3c800b9c0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009fc0 tx=0x7fb3c8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a33b97f6af6ba9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc035020 con 0x7fb3c80051c0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3dc003b90 con 0x7fb3c80051c0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e0a00640 1 -- 172.21.15.165:0/3308222416 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc009c10 con 0x7fb3c80051c0 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 -- 172.21.15.165:0/3308222416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c80051c0 msgr2=0x7fb3c800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c80051c0 0x7fb3c800b9c0 secure :-1 s=READY pgs=2909 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc009fc0 tx=0x7fb3c8005830 comp rx=0 tx=0).stop 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 -- 172.21.15.165:0/3308222416 shutdown_connections 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 --2- 172.21.15.165:0/3308222416 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3c80051c0 0x7fb3c800b9c0 unknown :-1 s=CLOSED pgs=2909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 -- 172.21.15.165:0/3308222416 >> 172.21.15.165:0/3308222416 conn(0x7fb3e40d0420 msgr2=0x7fb3e40d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 -- 172.21.15.165:0/3308222416 shutdown_connections 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.956+0000 7fb3e9000640 1 -- 172.21.15.165:0/3308222416 wait complete. 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.958+0000 7fb3e9000640 1 Processor -- start 2024-04-25T10:22:15.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.958+0000 7fb3e9000640 1 -- start start 2024-04-25T10:22:15.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e9000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80051c0 0x7fb3e40c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e9000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 0x7fb3e4156300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e9000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e41f36b0 0x7fb3e41f3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e9000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40b5e50 con 0x7fb3e41604f0 2024-04-25T10:22:15.963 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e9000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e4162500 con 0x7fb3e41f36b0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e9000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e40b5fd0 con 0x7fb3c80051c0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 0x7fb3e4156300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e1400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 0x7fb3e4156300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33700/0 (socket says 172.21.15.165:33700) 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.959+0000 7fb3e1400640 1 -- 172.21.15.165:0/2674205541 learned_addr learned my addr 172.21.15.165:0/2674205541 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e2800640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e41f36b0 0x7fb3e41f3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1e00640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80051c0 0x7fb3e40c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1400640 1 -- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80051c0 msgr2=0x7fb3e40c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1400640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80051c0 0x7fb3e40c96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1400640 1 -- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e41f36b0 msgr2=0x7fb3e41f3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1400640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e41f36b0 0x7fb3e41f3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1400640 1 -- 172.21.15.165:0/2674205541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3e4156c20 con 0x7fb3e41604f0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1e00640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80051c0 0x7fb3e40c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.960+0000 7fb3e1400640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 0x7fb3e4156300 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d4f0 tx=0x7fb3d000d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.961+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0005480 con 0x7fb3e41604f0 2024-04-25T10:22:15.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.961+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0017070 con 0x7fb3e41604f0 2024-04-25T10:22:15.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.961+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e4156f00 con 0x7fb3e41604f0 2024-04-25T10:22:15.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.961+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0003810 con 0x7fb3e41604f0 2024-04-25T10:22:15.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.961+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e41574c0 con 0x7fb3e41604f0 2024-04-25T10:22:15.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.962+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3e4073a00 con 0x7fb3e41604f0 2024-04-25T10:22:15.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.963+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fb3d00039b0 con 0x7fb3e41604f0 2024-04-25T10:22:15.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.964+0000 7fb3da000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3c406b500 0x7fb3c406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:15.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.965+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fb3d0089fc0 con 0x7fb3e41604f0 2024-04-25T10:22:15.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.965+0000 7fb3e1e00640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3c406b500 0x7fb3c406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:15.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.965+0000 7fb3e1e00640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3c406b500 0x7fb3c406d9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40017d0 tx=0x7fb3d4009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:15.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:15.969+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fb3d0058e30 con 0x7fb3e41604f0 2024-04-25T10:22:16.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.162+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3e406cec0 con 0x7fb3c406b500 2024-04-25T10:22:16.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.163+0000 7fb3da000640 1 -- 172.21.15.165:0/2674205541 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fb3e406cec0 con 0x7fb3c406b500 2024-04-25T10:22:16.165 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 0 to deep-scrub 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3c406b500 msgr2=0x7fb3c406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3c406b500 0x7fb3c406d9c0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40017d0 tx=0x7fb3d4009040 comp rx=0 tx=0).stop 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 msgr2=0x7fb3e4156300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 0x7fb3e4156300 secure :-1 s=READY pgs=2910 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000d4f0 tx=0x7fb3d000d9b0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 shutdown_connections 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fb3c406b500 0x7fb3c406d9c0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fb3e41f36b0 0x7fb3e41f3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fb3e41604f0 0x7fb3e4156300 unknown :-1 s=CLOSED pgs=2910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 --2- 172.21.15.165:0/2674205541 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fb3c80051c0 0x7fb3e40c96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.165+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 >> 172.21.15.165:0/2674205541 conn(0x7fb3e40d0420 msgr2=0x7fb3e40b8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.166+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 shutdown_connections 2024-04-25T10:22:16.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.166+0000 7fb3e9000640 1 -- 172.21.15.165:0/2674205541 wait complete. 2024-04-25T10:22:16.186 INFO:tasks.ceph:Scrubbing osd.1 2024-04-25T10:22:16.186 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:16.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.392+0000 7fcc1b000640 1 Processor -- start 2024-04-25T10:22:16.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.393+0000 7fcc1b000640 1 -- start start 2024-04-25T10:22:16.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fcc1415dbd0 0x7fcc14161620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 0x7fcc140b53d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1b000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc14161e70 con 0x7fcc140b5910 2024-04-25T10:22:16.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc140d1ce0 con 0x7fcc1415dbd0 2024-04-25T10:22:16.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14162150 con 0x7fcc140b95c0 2024-04-25T10:22:16.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1a600640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7fcc140b5910 0x7fcc1406bf70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:32782/0 (socket says 172.21.15.165:32782) 2024-04-25T10:22:16.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.394+0000 7fcc1a600640 1 -- 172.21.15.165:0/1308974008 learned_addr learned my addr 172.21.15.165:0/1308974008 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc19200640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 0x7fcc140b53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc19c00640 1 --2- 172.21.15.165:0/1308974008 >> v2:172.21.15.172:3300/0 conn(0x7fcc1415dbd0 0x7fcc14161620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 268910644 0 0) 0x7fcc14161e70 con 0x7fcc140b5910 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf8003520 con 0x7fcc140b5910 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 <== mon.0 v1:172.21.15.165:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3701300912 0 0) 0x7fcbf8003520 con 0x7fcc140b5910 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 msgr2=0x7fcc140b53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 0x7fcc140b53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 >> v2:172.21.15.172:3300/0 conn(0x7fcc1415dbd0 msgr2=0x7fcc14161620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 --2- 172.21.15.165:0/1308974008 >> v2:172.21.15.172:3300/0 conn(0x7fcc1415dbd0 0x7fcc14161620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 --> v1:172.21.15.165:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc14162430 con 0x7fcc140b5910 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 <== mon.0 v1:172.21.15.165:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 515418789 0 0) 0x7fcc0c0019f0 con 0x7fcc140b5910 2024-04-25T10:22:16.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 >> v1:172.21.15.165:6789/0 conn(0x7fcc140b5910 legacy=0x7fcc1406bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 0x7fcbf80039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcbf8004800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.395+0000 7fcc13e00640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8004d40 0x7fcbf80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140d1ce0 con 0x7fcbf8003f30 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14162150 con 0x7fcbf8004d40 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf800b9c0 con 0x7fcc140b95c0 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc19200640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcbf8004800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc1a600640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8004d40 0x7fcbf80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc19200640 1 -- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 msgr2=0x7fcbf80039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc19200640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc140b95c0 0x7fcbf80039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc19200640 1 -- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8004d40 msgr2=0x7fcbf80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc19200640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcbf8004d40 0x7fcbf80051c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.396+0000 7fcc19200640 1 -- 172.21.15.165:0/1308974008 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf8003520 con 0x7fcbf8003f30 2024-04-25T10:22:16.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.397+0000 7fcc19200640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcbf8004800 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009ac0 tx=0x7fcbf8005830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94b637d3bb40035e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00036070 con 0x7fcbf8003f30 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00031400 con 0x7fcbf8003f30 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc13e00640 1 -- 172.21.15.165:0/1308974008 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00009690 con 0x7fcbf8003f30 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1b000640 1 -- 172.21.15.165:0/1308974008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 msgr2=0x7fcbf8004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1b000640 1 --2- 172.21.15.165:0/1308974008 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcbf8004800 secure :-1 s=READY pgs=2912 cs=0 l=1 rev1=1 crypto rx=0x7fcc00009ac0 tx=0x7fcbf8005830 comp rx=0 tx=0).stop 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1a600640 1 -- 172.21.15.165:0/1308974008 reap_dead start 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1b000640 1 -- 172.21.15.165:0/1308974008 shutdown_connections 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1b000640 1 -- 172.21.15.165:0/1308974008 >> 172.21.15.165:0/1308974008 conn(0x7fcc140d03d0 msgr2=0x7fcc140d07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1b000640 1 -- 172.21.15.165:0/1308974008 shutdown_connections 2024-04-25T10:22:16.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.398+0000 7fcc1b000640 1 -- 172.21.15.165:0/1308974008 wait complete. 2024-04-25T10:22:16.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.400+0000 7fcc1b000640 1 Processor -- start 2024-04-25T10:22:16.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.400+0000 7fcc1b000640 1 -- start start 2024-04-25T10:22:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.401+0000 7fcc1b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcc140cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.401+0000 7fcc1b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc14159070 0x7fcc140b7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.401+0000 7fcc1b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1414b960 0x7fcc1414be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.401+0000 7fcc1b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140d3180 con 0x7fcbf8003f30 2024-04-25T10:22:16.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.401+0000 7fcc1b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc14161b60 con 0x7fcc14159070 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.401+0000 7fcc1b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc140d3300 con 0x7fcc1414b960 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcc140cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcc140cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33728/0 (socket says 172.21.15.165:33728) 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 -- 172.21.15.165:0/3802177245 learned_addr learned my addr 172.21.15.165:0/3802177245 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19200640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc14159070 0x7fcc140b7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 -- 172.21.15.165:0/3802177245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1414b960 msgr2=0x7fcc1414be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fcc1414b960 0x7fcc1414be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 -- 172.21.15.165:0/3802177245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc14159070 msgr2=0x7fcc140b7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fcc14159070 0x7fcc140b7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 -- 172.21.15.165:0/3802177245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc140b8650 con 0x7fcbf8003f30 2024-04-25T10:22:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.402+0000 7fcc19c00640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcc140cd590 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400e860 tx=0x7fcc0400ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.403+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc04017070 con 0x7fcbf8003f30 2024-04-25T10:22:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.403+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0400c940 con 0x7fcbf8003f30 2024-04-25T10:22:16.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.403+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc1415f060 con 0x7fcbf8003f30 2024-04-25T10:22:16.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.403+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0400cd10 con 0x7fcbf8003f30 2024-04-25T10:22:16.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.403+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc1415f620 con 0x7fcbf8003f30 2024-04-25T10:22:16.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.404+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcc140c7080 con 0x7fcbf8003f30 2024-04-25T10:22:16.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.405+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fcc0401a070 con 0x7fcbf8003f30 2024-04-25T10:22:16.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.406+0000 7fcc11600640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbf406b4d0 0x7fcbf406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.406+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fcc0408a0f0 con 0x7fcbf8003f30 2024-04-25T10:22:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.406+0000 7fcc19200640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbf406b4d0 0x7fcbf406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.406+0000 7fcc11600640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7fcbf4071a10 0x7fcbf4073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.406+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 --> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcc04020e50 con 0x7fcbf4071a10 2024-04-25T10:22:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.406+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc04091050 con 0x7fcbf8003f30 2024-04-25T10:22:16.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.407+0000 7fcc1a600640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7fcbf4071a10 0x7fcbf4073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.407+0000 7fcc19200640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbf406b4d0 0x7fcbf406d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcc00033000 tx=0x7fcc00008030 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.407+0000 7fcc1a600640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7fcbf4071a10 0x7fcbf4073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.408+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== osd.1 v2:172.21.15.165:6808/4150887630 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fcc04020e50 con 0x7fcbf4071a10 2024-04-25T10:22:16.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 --> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fcc140d1520 con 0x7fcbf4071a10 2024-04-25T10:22:16.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc11600640 1 -- 172.21.15.165:0/3802177245 <== osd.1 v2:172.21.15.165:6808/4150887630 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fcc140d1520 con 0x7fcbf4071a10 2024-04-25T10:22:16.425 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:16.426 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:16.426 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7fcbf4071a10 msgr2=0x7fcbf4073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6808/4150887630,v1:172.21.15.165:6809/4150887630] conn(0x7fcbf4071a10 0x7fcbf4073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbf406b4d0 msgr2=0x7fcbf406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fcbf406b4d0 0x7fcbf406d990 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fcc00033000 tx=0x7fcc00008030 comp rx=0 tx=0).stop 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 msgr2=0x7fcc140cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1b000640 1 --2- 172.21.15.165:0/3802177245 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fcbf8003f30 0x7fcc140cd590 secure :-1 s=READY pgs=2913 cs=0 l=1 rev1=1 crypto rx=0x7fcc0400e860 tx=0x7fcc0400ed20 comp rx=0 tx=0).stop 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.423+0000 7fcc1a600640 1 -- 172.21.15.165:0/3802177245 reap_dead start 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.424+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 shutdown_connections 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.424+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 >> 172.21.15.165:0/3802177245 conn(0x7fcc140d03d0 msgr2=0x7fcc140b4ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.424+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 shutdown_connections 2024-04-25T10:22:16.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.424+0000 7fcc1b000640 1 -- 172.21.15.165:0/3802177245 wait complete. 2024-04-25T10:22:16.442 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-25T10:22:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.634+0000 7fe04b000640 1 Processor -- start 2024-04-25T10:22:16.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.635+0000 7fe04b000640 1 -- start start 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.636+0000 7fe04b000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0440ceb20 0x7fe0440c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.636+0000 7fe04b000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe0440c5030 0x7fe04415bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.636+0000 7fe04b000640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe044145520 con 0x7fe044145140 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.636+0000 7fe04b000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe044071270 con 0x7fe0440ceb20 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.636+0000 7fe04b000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe044071550 con 0x7fe0440c5030 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0440ceb20 0x7fe0440c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0440ceb20 0x7fe0440c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33732/0 (socket says 172.21.15.165:33732) 2024-04-25T10:22:16.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 -- 172.21.15.165:0/1662414405 learned_addr learned my addr 172.21.15.165:0/1662414405 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:16.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe04a600640 1 --2- 172.21.15.165:0/1662414405 >> v2:172.21.15.172:3300/0 conn(0x7fe0440c5030 0x7fe04415bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 -- 172.21.15.165:0/1662414405 >> v2:172.21.15.172:3300/0 conn(0x7fe0440c5030 msgr2=0x7fe04415bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 --2- 172.21.15.165:0/1662414405 >> v2:172.21.15.172:3300/0 conn(0x7fe0440c5030 0x7fe04415bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 -- 172.21.15.165:0/1662414405 >> v1:172.21.15.172:6789/0 conn(0x7fe044145140 legacy=0x7fe0440ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe049200640 1 -- 172.21.15.165:0/1662414405 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044071830 con 0x7fe0440ceb20 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.637+0000 7fe04a600640 1 --2- 172.21.15.165:0/1662414405 >> v2:172.21.15.172:3300/0 conn(0x7fe0440c5030 0x7fe04415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.638+0000 7fe049200640 1 --2- 172.21.15.165:0/1662414405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440ceb20 0x7fe0440c4af0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fe04406bb90 tx=0x7fe03002a1a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa594ec49123253b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.639+0000 7fe043e00640 1 -- 172.21.15.165:0/1662414405 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030037070 con 0x7fe0440ceb20 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.639+0000 7fe043e00640 1 -- 172.21.15.165:0/1662414405 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030032440 con 0x7fe0440ceb20 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.639+0000 7fe043e00640 1 -- 172.21.15.165:0/1662414405 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030032810 con 0x7fe0440ceb20 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.639+0000 7fe04b000640 1 -- 172.21.15.165:0/1662414405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440ceb20 msgr2=0x7fe0440c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.639+0000 7fe04b000640 1 --2- 172.21.15.165:0/1662414405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440ceb20 0x7fe0440c4af0 secure :-1 s=READY pgs=2914 cs=0 l=1 rev1=1 crypto rx=0x7fe04406bb90 tx=0x7fe03002a1a0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.640+0000 7fe04b000640 1 -- 172.21.15.165:0/1662414405 shutdown_connections 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.640+0000 7fe04b000640 1 --2- 172.21.15.165:0/1662414405 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440ceb20 0x7fe0440c4af0 unknown :-1 s=CLOSED pgs=2914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.640+0000 7fe04b000640 1 --2- 172.21.15.165:0/1662414405 >> v2:172.21.15.172:3300/0 conn(0x7fe0440c5030 0x7fe04415bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.640+0000 7fe04b000640 1 -- 172.21.15.165:0/1662414405 >> 172.21.15.165:0/1662414405 conn(0x7fe044061cc0 msgr2=0x7fe04406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.640+0000 7fe04b000640 1 -- 172.21.15.165:0/1662414405 shutdown_connections 2024-04-25T10:22:16.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.640+0000 7fe04b000640 1 -- 172.21.15.165:0/1662414405 wait complete. 2024-04-25T10:22:16.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.642+0000 7fe04b000640 1 Processor -- start 2024-04-25T10:22:16.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.642+0000 7fe04b000640 1 -- start start 2024-04-25T10:22:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe04b000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 0x7fe04415b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe04b000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0440ceb20 0x7fe0440aed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe04b000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe044145140 0x7fe0440b0970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe04b000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440c5620 con 0x7fe0440c5030 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe04b000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0440c54a0 con 0x7fe044145140 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe04b000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04415c4f0 con 0x7fe0440ceb20 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe049c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 0x7fe04415b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe049c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 0x7fe04415b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:33734/0 (socket says 172.21.15.165:33734) 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.643+0000 7fe049c00640 1 -- 172.21.15.165:0/620236808 learned_addr learned my addr 172.21.15.165:0/620236808 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe04a600640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe044145140 0x7fe0440b0970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe049c00640 1 -- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0440ceb20 msgr2=0x7fe0440aed40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe049c00640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0440ceb20 0x7fe0440aed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe049c00640 1 -- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe044145140 msgr2=0x7fe0440b0970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe049c00640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe044145140 0x7fe0440b0970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe049c00640 1 -- 172.21.15.165:0/620236808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0441381c0 con 0x7fe0440c5030 2024-04-25T10:22:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe04a600640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe044145140 0x7fe0440b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:16.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe049c00640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 0x7fe04415b450 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b3e0 tx=0x7fe03800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.644+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038015020 con 0x7fe0440c5030 2024-04-25T10:22:16.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.645+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe038011850 con 0x7fe0440c5030 2024-04-25T10:22:16.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.645+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe038011c40 con 0x7fe0440c5030 2024-04-25T10:22:16.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.645+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe044138460 con 0x7fe0440c5030 2024-04-25T10:22:16.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.645+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe044138a20 con 0x7fe0440c5030 2024-04-25T10:22:16.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.646+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe044138e50 con 0x7fe0440c5030 2024-04-25T10:22:16.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.647+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe038018020 con 0x7fe0440c5030 2024-04-25T10:22:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.648+0000 7fe041600640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe02406b4d0 0x7fe02406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.648+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe038089fe0 con 0x7fe0440c5030 2024-04-25T10:22:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.648+0000 7fe049200640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe02406b4d0 0x7fe02406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:16.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.649+0000 7fe049200640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe02406b4d0 0x7fe02406d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe030034000 tx=0x7fe03002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:16.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.653+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe03801f020 con 0x7fe0440c5030 2024-04-25T10:22:16.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.846+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe044073990 con 0x7fe02406b4d0 2024-04-25T10:22:16.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.846+0000 7fe041600640 1 -- 172.21.15.165:0/620236808 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe044073990 con 0x7fe02406b4d0 2024-04-25T10:22:16.849 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 1 to deep-scrub 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe02406b4d0 msgr2=0x7fe02406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe02406b4d0 0x7fe02406d990 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fe030034000 tx=0x7fe03002c000 comp rx=0 tx=0).stop 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 msgr2=0x7fe04415b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 0x7fe04415b450 secure :-1 s=READY pgs=2915 cs=0 l=1 rev1=1 crypto rx=0x7fe03800b3e0 tx=0x7fe03800b8a0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 shutdown_connections 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe02406b4d0 0x7fe02406d990 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe044145140 0x7fe0440b0970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0440ceb20 0x7fe0440aed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 --2- 172.21.15.165:0/620236808 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0440c5030 0x7fe04415b450 unknown :-1 s=CLOSED pgs=2915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 >> 172.21.15.165:0/620236808 conn(0x7fe044061cc0 msgr2=0x7fe0440b3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:16.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 shutdown_connections 2024-04-25T10:22:16.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:16.849+0000 7fe04b000640 1 -- 172.21.15.165:0/620236808 wait complete. 2024-04-25T10:22:16.870 INFO:tasks.ceph:Scrubbing osd.2 2024-04-25T10:22:16.870 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.059+0000 7f2a83200640 1 Processor -- start 2024-04-25T10:22:17.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.059+0000 7f2a83200640 1 -- start start 2024-04-25T10:22:17.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a83200640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a84145140 0x7f2a840ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a83200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2a840ce950 0x7f2a840c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a83200640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a84145520 con 0x7f2a840c5030 2024-04-25T10:22:17.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a83200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a840ced90 con 0x7f2a840ce950 2024-04-25T10:22:17.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a83200640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a84071270 con 0x7f2a84145140 2024-04-25T10:22:17.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a81400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2a840ce950 0x7f2a840c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a82800640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2a840c5030 0x7f2a8406d1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52336/0 (socket says 172.21.15.165:52336) 2024-04-25T10:22:17.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a82800640 1 -- 172.21.15.165:0/3888506738 learned_addr learned my addr 172.21.15.165:0/3888506738 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.061+0000 7f2a81e00640 1 --2- 172.21.15.165:0/3888506738 >> v2:172.21.15.172:3300/0 conn(0x7f2a84145140 0x7f2a840ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.062+0000 7f2a81400640 1 -- 172.21.15.165:0/3888506738 >> v2:172.21.15.172:3300/0 conn(0x7f2a84145140 msgr2=0x7f2a840ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.062+0000 7f2a81400640 1 --2- 172.21.15.165:0/3888506738 >> v2:172.21.15.172:3300/0 conn(0x7f2a84145140 0x7f2a840ce410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.062+0000 7f2a81400640 1 -- 172.21.15.165:0/3888506738 >> v1:172.21.15.165:6789/0 conn(0x7f2a840c5030 legacy=0x7f2a8406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.062+0000 7f2a80a00640 1 -- 172.21.15.165:0/3888506738 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3094785245 0 0) 0x7f2a84145520 con 0x7f2a840c5030 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.062+0000 7f2a81400640 1 -- 172.21.15.165:0/3888506738 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a84071550 con 0x7f2a840ce950 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.062+0000 7f2a81e00640 1 --2- 172.21.15.165:0/3888506738 >> v2:172.21.15.172:3300/0 conn(0x7f2a84145140 0x7f2a840ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.063+0000 7f2a81400640 1 --2- 172.21.15.165:0/3888506738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a840ce950 0x7f2a840c4af0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f2a8406f690 tx=0x7f2a7002a110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1659555a2eec67 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.063+0000 7f2a80a00640 1 -- 172.21.15.165:0/3888506738 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a70037070 con 0x7f2a840ce950 2024-04-25T10:22:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.063+0000 7f2a80a00640 1 -- 172.21.15.165:0/3888506738 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a70032440 con 0x7f2a840ce950 2024-04-25T10:22:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.063+0000 7f2a80a00640 1 -- 172.21.15.165:0/3888506738 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7002ae60 con 0x7f2a840ce950 2024-04-25T10:22:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.063+0000 7f2a83200640 1 -- 172.21.15.165:0/3888506738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a840ce950 msgr2=0x7f2a840c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.063+0000 7f2a83200640 1 --2- 172.21.15.165:0/3888506738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a840ce950 0x7f2a840c4af0 secure :-1 s=READY pgs=2917 cs=0 l=1 rev1=1 crypto rx=0x7f2a8406f690 tx=0x7f2a7002a110 comp rx=0 tx=0).stop 2024-04-25T10:22:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.064+0000 7f2a83200640 1 -- 172.21.15.165:0/3888506738 shutdown_connections 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.064+0000 7f2a83200640 1 --2- 172.21.15.165:0/3888506738 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a840ce950 0x7f2a840c4af0 unknown :-1 s=CLOSED pgs=2917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.064+0000 7f2a83200640 1 --2- 172.21.15.165:0/3888506738 >> v2:172.21.15.172:3300/0 conn(0x7f2a84145140 0x7f2a840ce410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.064+0000 7f2a83200640 1 -- 172.21.15.165:0/3888506738 >> 172.21.15.165:0/3888506738 conn(0x7f2a84061cc0 msgr2=0x7f2a8406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.064+0000 7f2a83200640 1 -- 172.21.15.165:0/3888506738 shutdown_connections 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.064+0000 7f2a83200640 1 -- 172.21.15.165:0/3888506738 wait complete. 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.065+0000 7f2a83200640 1 Processor -- start 2024-04-25T10:22:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.065+0000 7f2a83200640 1 -- start start 2024-04-25T10:22:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.066+0000 7f2a83200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a840c5030 0x7f2a840ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.066+0000 7f2a83200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a840ce950 0x7f2a841f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.066+0000 7f2a83200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a84145140 0x7f2a840aa170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.066+0000 7f2a83200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840c5480 con 0x7f2a84145140 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.066+0000 7f2a83200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a84071b10 con 0x7f2a840ce950 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.066+0000 7f2a83200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a840c5600 con 0x7f2a840c5030 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a84145140 0x7f2a840aa170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a84145140 0x7f2a840aa170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51562/0 (socket says 172.21.15.165:51562) 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 -- 172.21.15.165:0/3391317081 learned_addr learned my addr 172.21.15.165:0/3391317081 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 -- 172.21.15.165:0/3391317081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a840c5030 msgr2=0x7f2a840ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a840c5030 0x7f2a840ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a81400640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a840ce950 0x7f2a841f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 -- 172.21.15.165:0/3391317081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a840ce950 msgr2=0x7f2a841f6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a840ce950 0x7f2a841f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 -- 172.21.15.165:0/3391317081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a840aaa00 con 0x7f2a84145140 2024-04-25T10:22:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a82800640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a84145140 0x7f2a840aa170 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c450 tx=0x7f2a7c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.067+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c018070 con 0x7f2a84145140 2024-04-25T10:22:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.068+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a7c007500 con 0x7f2a84145140 2024-04-25T10:22:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.068+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a7c004a20 con 0x7f2a84145140 2024-04-25T10:22:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.068+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a840aace0 con 0x7f2a84145140 2024-04-25T10:22:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.068+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a840ab2a0 con 0x7f2a84145140 2024-04-25T10:22:17.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.069+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a840b46e0 con 0x7f2a84145140 2024-04-25T10:22:17.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.070+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a7c01b070 con 0x7f2a84145140 2024-04-25T10:22:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a7a000640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a6406b500 0x7f2a6406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2a7c0898b0 con 0x7f2a84145140 2024-04-25T10:22:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a81e00640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a6406b500 0x7f2a6406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a7a000640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f2a64071a40 0x7f2a64073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 --> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a64074550 con 0x7f2a64071a40 2024-04-25T10:22:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a7c089d10 con 0x7f2a84145140 2024-04-25T10:22:17.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.071+0000 7f2a81400640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f2a64071a40 0x7f2a64073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.072+0000 7f2a81e00640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a6406b500 0x7f2a6406d9c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2a74008e00 tx=0x7f2a74008330 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.072+0000 7f2a81400640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f2a64071a40 0x7f2a64073e80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.073+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== osd.2 v2:172.21.15.165:6824/3063902068 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2a64074550 con 0x7f2a64071a40 2024-04-25T10:22:17.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.087+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 --> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a8406ef40 con 0x7f2a64071a40 2024-04-25T10:22:17.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a7a000640 1 -- 172.21.15.165:0/3391317081 <== osd.2 v2:172.21.15.165:6824/3063902068 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2a8406ef40 con 0x7f2a64071a40 2024-04-25T10:22:17.090 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:17.090 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:17.090 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:17.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f2a64071a40 msgr2=0x7f2a64073e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a83200640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6824/3063902068,v1:172.21.15.165:6825/3063902068] conn(0x7f2a64071a40 0x7f2a64073e80 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a6406b500 msgr2=0x7f2a6406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a83200640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a6406b500 0x7f2a6406d9c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f2a74008e00 tx=0x7f2a74008330 comp rx=0 tx=0).stop 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a84145140 msgr2=0x7f2a840aa170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.088+0000 7f2a83200640 1 --2- 172.21.15.165:0/3391317081 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a84145140 0x7f2a840aa170 secure :-1 s=READY pgs=2918 cs=0 l=1 rev1=1 crypto rx=0x7f2a7c00c450 tx=0x7f2a7c00c910 comp rx=0 tx=0).stop 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.089+0000 7f2a82800640 1 -- 172.21.15.165:0/3391317081 reap_dead start 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.089+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 shutdown_connections 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.089+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 >> 172.21.15.165:0/3391317081 conn(0x7f2a84061cc0 msgr2=0x7f2a84144390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.089+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 shutdown_connections 2024-04-25T10:22:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.089+0000 7f2a83200640 1 -- 172.21.15.165:0/3391317081 wait complete. 2024-04-25T10:22:17.108 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-25T10:22:17.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.300+0000 7f1ca3e00640 1 Processor -- start 2024-04-25T10:22:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.300+0000 7f1ca3e00640 1 -- start start 2024-04-25T10:22:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca3e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f1ca415cfc0 0x7f1ca40c5200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca3e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca4145520 con 0x7f1ca40ceb20 2024-04-25T10:22:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca3e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca4071270 con 0x7f1ca4145140 2024-04-25T10:22:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca3e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ca4071550 con 0x7f1ca415cfc0 2024-04-25T10:22:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca2000640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f1ca40ceb20 0x7f1ca40c4af0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52338/0 (socket says 172.21.15.165:52338) 2024-04-25T10:22:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca2000640 1 -- 172.21.15.165:0/826609156 learned_addr learned my addr 172.21.15.165:0/826609156 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.302+0000 7f1ca3400640 1 --2- 172.21.15.165:0/826609156 >> v2:172.21.15.165:3300/0 conn(0x7f1ca415cfc0 0x7f1ca40c5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.303+0000 7f1ca1600640 1 -- 172.21.15.165:0/826609156 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2597943773 0 0) 0x7f1ca4145520 con 0x7f1ca40ceb20 2024-04-25T10:22:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.303+0000 7f1ca1600640 1 -- 172.21.15.165:0/826609156 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c88003520 con 0x7f1ca40ceb20 2024-04-25T10:22:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.303+0000 7f1ca3400640 1 -- 172.21.15.165:0/826609156 >> v1:172.21.15.172:6789/0 conn(0x7f1ca4145140 legacy=0x7f1ca40ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.303+0000 7f1ca3400640 1 -- 172.21.15.165:0/826609156 >> v1:172.21.15.165:6789/0 conn(0x7f1ca40ceb20 legacy=0x7f1ca40c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.303+0000 7f1ca3400640 1 -- 172.21.15.165:0/826609156 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4071830 con 0x7f1ca415cfc0 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.304+0000 7f1ca3400640 1 --2- 172.21.15.165:0/826609156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca40c5200 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009d90 tx=0x7f1c9c02a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7c5b4fc0ac6b2a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.304+0000 7f1ca1600640 1 -- 172.21.15.165:0/826609156 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007740 con 0x7f1ca415cfc0 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.304+0000 7f1ca1600640 1 -- 172.21.15.165:0/826609156 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c0078e0 con 0x7f1ca415cfc0 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.304+0000 7f1ca1600640 1 -- 172.21.15.165:0/826609156 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c007cb0 con 0x7f1ca415cfc0 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 -- 172.21.15.165:0/826609156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 msgr2=0x7f1ca40c5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/826609156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca40c5200 secure :-1 s=READY pgs=2920 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009d90 tx=0x7f1c9c02a660 comp rx=0 tx=0).stop 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 -- 172.21.15.165:0/826609156 shutdown_connections 2024-04-25T10:22:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/826609156 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca40c5200 unknown :-1 s=CLOSED pgs=2920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 -- 172.21.15.165:0/826609156 >> 172.21.15.165:0/826609156 conn(0x7f1ca4061cc0 msgr2=0x7f1ca406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 -- 172.21.15.165:0/826609156 shutdown_connections 2024-04-25T10:22:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.305+0000 7f1ca3e00640 1 -- 172.21.15.165:0/826609156 wait complete. 2024-04-25T10:22:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.306+0000 7f1ca3e00640 1 Processor -- start 2024-04-25T10:22:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 -- start start 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca40ceb20 0x7f1ca41363c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ca4145140 0x7f1ca40cb100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca415a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca41665b0 con 0x7f1ca415cfc0 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4166430 con 0x7f1ca40ceb20 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.307+0000 7f1ca3e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca4166730 con 0x7f1ca4145140 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca415a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca415a240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51574/0 (socket says 172.21.15.165:51574) 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 -- 172.21.15.165:0/4065485397 learned_addr learned my addr 172.21.15.165:0/4065485397 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 -- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ca4145140 msgr2=0x7f1ca40cb100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca2a00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca40ceb20 0x7f1ca41363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ca4145140 0x7f1ca40cb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 -- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca40ceb20 msgr2=0x7f1ca41363c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca40ceb20 0x7f1ca41363c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca3400640 1 -- 172.21.15.165:0/4065485397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca4134d30 con 0x7f1ca415cfc0 2024-04-25T10:22:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.308+0000 7f1ca2a00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca40ceb20 0x7f1ca41363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.309+0000 7f1ca3400640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca415a240 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009eb0 tx=0x7f1c9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.309+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c03f070 con 0x7f1ca415cfc0 2024-04-25T10:22:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.309+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9c031bc0 con 0x7f1ca415cfc0 2024-04-25T10:22:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.309+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9c009450 con 0x7f1ca415cfc0 2024-04-25T10:22:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.309+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca4135010 con 0x7f1ca415cfc0 2024-04-25T10:22:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.309+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca41355d0 con 0x7f1ca415cfc0 2024-04-25T10:22:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.310+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca4073990 con 0x7f1ca415cfc0 2024-04-25T10:22:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.311+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f1c9c03b410 con 0x7f1ca415cfc0 2024-04-25T10:22:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.312+0000 7f1c9aa00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c8406b4d0 0x7f1c8406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.312+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f1c9c0ab960 con 0x7f1ca415cfc0 2024-04-25T10:22:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.312+0000 7f1ca2a00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c8406b4d0 0x7f1c8406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.313+0000 7f1ca2a00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c8406b4d0 0x7f1c8406d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1ca413d4a0 tx=0x7f1c94009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.316+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f1c9c07a860 con 0x7f1ca415cfc0 2024-04-25T10:22:17.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.514+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ca406ad10 con 0x7f1c8406b4d0 2024-04-25T10:22:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.515+0000 7f1c9aa00640 1 -- 172.21.15.165:0/4065485397 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1ca406ad10 con 0x7f1c8406b4d0 2024-04-25T10:22:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 2 to deep-scrub 2024-04-25T10:22:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.517+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c8406b4d0 msgr2=0x7f1c8406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.517+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c8406b4d0 0x7f1c8406d990 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f1ca413d4a0 tx=0x7f1c94009040 comp rx=0 tx=0).stop 2024-04-25T10:22:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.517+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 msgr2=0x7f1ca415a240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.517+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca415a240 secure :-1 s=READY pgs=2921 cs=0 l=1 rev1=1 crypto rx=0x7f1c9c009eb0 tx=0x7f1c9c004a40 comp rx=0 tx=0).stop 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 shutdown_connections 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f1c8406b4d0 0x7f1c8406d990 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f1ca415cfc0 0x7f1ca415a240 unknown :-1 s=CLOSED pgs=2921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f1ca4145140 0x7f1ca40cb100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 --2- 172.21.15.165:0/4065485397 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f1ca40ceb20 0x7f1ca41363c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 >> 172.21.15.165:0/4065485397 conn(0x7f1ca4061cc0 msgr2=0x7f1ca40b1240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 shutdown_connections 2024-04-25T10:22:17.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.518+0000 7f1ca3e00640 1 -- 172.21.15.165:0/4065485397 wait complete. 2024-04-25T10:22:17.539 INFO:tasks.ceph:Scrubbing osd.3 2024-04-25T10:22:17.539 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:17.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.740+0000 7f2a75000640 1 Processor -- start 2024-04-25T10:22:17.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.740+0000 7f2a75000640 1 -- start start 2024-04-25T10:22:17.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a75000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f2a700ceb20 0x7f2a700c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a75000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f2a700c5030 0x7f2a7015bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a75000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a70145520 con 0x7f2a70145140 2024-04-25T10:22:17.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a75000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a70071270 con 0x7f2a700c5030 2024-04-25T10:22:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a75000640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a70071550 con 0x7f2a700ceb20 2024-04-25T10:22:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a6f400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f2a70145140 0x7f2a700ce410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52352/0 (socket says 172.21.15.165:52352) 2024-04-25T10:22:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a6f400640 1 -- 172.21.15.165:0/2857136496 learned_addr learned my addr 172.21.15.165:0/2857136496 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.742+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2857136496 >> v2:172.21.15.165:3300/0 conn(0x7f2a700c5030 0x7f2a7015bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a67e00640 1 --2- 172.21.15.165:0/2857136496 >> v2:172.21.15.172:3300/0 conn(0x7f2a700ceb20 0x7f2a700c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2857136496 >> v2:172.21.15.172:3300/0 conn(0x7f2a700ceb20 msgr2=0x7f2a700c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:17.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2857136496 >> v2:172.21.15.172:3300/0 conn(0x7f2a700ceb20 0x7f2a700c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2857136496 >> v1:172.21.15.165:6789/0 conn(0x7f2a70145140 legacy=0x7f2a700ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2857136496 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a70071830 con 0x7f2a700c5030 2024-04-25T10:22:17.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a67e00640 1 --2- 172.21.15.165:0/2857136496 >> v2:172.21.15.172:3300/0 conn(0x7f2a700ceb20 0x7f2a700c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:17.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.743+0000 7f2a6ea00640 1 -- 172.21.15.165:0/2857136496 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2576135143 0 0) 0x7f2a70145520 con 0x7f2a70145140 2024-04-25T10:22:17.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.744+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2857136496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700c5030 0x7f2a7015bfb0 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f2a700b0c00 tx=0x7f2a6802a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce9087a5ab344ab6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.744+0000 7f2a6ea00640 1 -- 172.21.15.165:0/2857136496 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68007740 con 0x7f2a700c5030 2024-04-25T10:22:17.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.744+0000 7f2a6ea00640 1 -- 172.21.15.165:0/2857136496 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a680078e0 con 0x7f2a700c5030 2024-04-25T10:22:17.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.744+0000 7f2a6ea00640 1 -- 172.21.15.165:0/2857136496 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68007cb0 con 0x7f2a700c5030 2024-04-25T10:22:17.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 -- 172.21.15.165:0/2857136496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700c5030 msgr2=0x7f2a7015bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 --2- 172.21.15.165:0/2857136496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700c5030 0x7f2a7015bfb0 secure :-1 s=READY pgs=2923 cs=0 l=1 rev1=1 crypto rx=0x7f2a700b0c00 tx=0x7f2a6802a5f0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 -- 172.21.15.165:0/2857136496 shutdown_connections 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 --2- 172.21.15.165:0/2857136496 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a700c5030 0x7f2a7015bfb0 unknown :-1 s=CLOSED pgs=2923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 --2- 172.21.15.165:0/2857136496 >> v2:172.21.15.172:3300/0 conn(0x7f2a700ceb20 0x7f2a700c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 -- 172.21.15.165:0/2857136496 >> 172.21.15.165:0/2857136496 conn(0x7f2a70061cc0 msgr2=0x7f2a7006abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 -- 172.21.15.165:0/2857136496 shutdown_connections 2024-04-25T10:22:17.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.745+0000 7f2a75000640 1 -- 172.21.15.165:0/2857136496 wait complete. 2024-04-25T10:22:17.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.747+0000 7f2a75000640 1 Processor -- start 2024-04-25T10:22:17.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.747+0000 7f2a75000640 1 -- start start 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a75000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a700c5030 0x7f2a700b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a75000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700ceb20 0x7f2a7013fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a75000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a70145140 0x7f2a70141820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a75000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700c5620 con 0x7f2a70145140 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a75000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a700c54a0 con 0x7f2a700c5030 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a75000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a7015c4f0 con 0x7f2a700ceb20 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a70145140 0x7f2a70141820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a6fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a70145140 0x7f2a70141820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51592/0 (socket says 172.21.15.165:51592) 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2799480568 learned_addr learned my addr 172.21.15.165:0/2799480568 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:17.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.748+0000 7f2a6f400640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a700c5030 0x7f2a700b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2799480568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700ceb20 msgr2=0x7f2a7013fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f2a700ceb20 0x7f2a7013fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2799480568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a700c5030 msgr2=0x7f2a700b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a700c5030 0x7f2a700b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2799480568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a700a9190 con 0x7f2a70145140 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6f400640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f2a700c5030 0x7f2a700b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:17.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a6fe00640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a70145140 0x7f2a70141820 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802abe0 tx=0x7f2a68004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68040070 con 0x7f2a70145140 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a68031c40 con 0x7f2a70145140 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a68003990 con 0x7f2a70145140 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a700a9470 con 0x7f2a70145140 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.749+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a701e5e70 con 0x7f2a70145140 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.750+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2a70073990 con 0x7f2a70145140 2024-04-25T10:22:17.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.751+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f2a68039040 con 0x7f2a70145140 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.752+0000 7f2a67400640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.752+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f2a680ab4b0 con 0x7f2a70145140 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.753+0000 7f2a6f400640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.753+0000 7f2a67400640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f2a4c071a10 0x7f2a4c073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.753+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 --> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2a68032e70 con 0x7f2a4c071a10 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.753+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a680b2050 con 0x7f2a70145140 2024-04-25T10:22:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.753+0000 7f2a67e00640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f2a4c071a10 0x7f2a4c073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.753+0000 7f2a6f400640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2a60000f60 tx=0x7f2a60009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.754+0000 7f2a67e00640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f2a4c071a10 0x7f2a4c073e50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.754+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== osd.3 v2:172.21.15.165:6816/995850359 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f2a68032e70 con 0x7f2a4c071a10 2024-04-25T10:22:17.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.769+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 --> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f2a7006b010 con 0x7f2a4c071a10 2024-04-25T10:22:17.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.769+0000 7f2a67400640 1 -- 172.21.15.165:0/2799480568 <== osd.3 v2:172.21.15.165:6816/995850359 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f2a7006b010 con 0x7f2a4c071a10 2024-04-25T10:22:17.772 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:17.772 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:17.772 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f2a4c071a10 msgr2=0x7f2a4c073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6816/995850359,v1:172.21.15.165:6817/995850359] conn(0x7f2a4c071a10 0x7f2a4c073e50 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 msgr2=0x7f2a4c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f2a4c06b4d0 0x7f2a4c06d990 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f2a60000f60 tx=0x7f2a60009040 comp rx=0 tx=0).stop 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a70145140 msgr2=0x7f2a70141820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 --2- 172.21.15.165:0/2799480568 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f2a70145140 0x7f2a70141820 secure :-1 s=READY pgs=2924 cs=0 l=1 rev1=1 crypto rx=0x7f2a6802abe0 tx=0x7f2a68004a40 comp rx=0 tx=0).stop 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a6fe00640 1 -- 172.21.15.165:0/2799480568 reap_dead start 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 shutdown_connections 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 >> 172.21.15.165:0/2799480568 conn(0x7f2a70061cc0 msgr2=0x7f2a700ce210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 shutdown_connections 2024-04-25T10:22:17.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.770+0000 7f2a75000640 1 -- 172.21.15.165:0/2799480568 wait complete. 2024-04-25T10:22:17.789 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-25T10:22:17.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.970+0000 7f11b1a00640 1 Processor -- start 2024-04-25T10:22:17.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.970+0000 7f11b1a00640 1 -- start start 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11b1a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f11ac1446d0 0x7f11ac0b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11b1a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11ac144d50 0x7f11ac0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11b1a00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11ac0cccc0 con 0x7f11ac1452a0 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11b1a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac0b2b20 con 0x7f11ac144d50 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11b1a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11ac06d400 con 0x7f11ac1446d0 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11aa000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11ac144d50 0x7f11ac0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11aa000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f11ac144d50 0x7f11ac0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51594/0 (socket says 172.21.15.165:51594) 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11ab400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f11ac1452a0 0x7f11ac0cf640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52358/0 (socket says 172.21.15.165:52358) 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.972+0000 7f11ab400640 1 -- 172.21.15.165:0/3499193278 learned_addr learned my addr 172.21.15.165:0/3499193278 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:17.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.973+0000 7f11aaa00640 1 --2- 172.21.15.165:0/3499193278 >> v2:172.21.15.172:3300/0 conn(0x7f11ac1446d0 0x7f11ac0b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.973+0000 7f11aa000640 1 -- 172.21.15.165:0/3499193278 >> v2:172.21.15.172:3300/0 conn(0x7f11ac1446d0 msgr2=0x7f11ac0b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:17.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.973+0000 7f11aa000640 1 --2- 172.21.15.165:0/3499193278 >> v2:172.21.15.172:3300/0 conn(0x7f11ac1446d0 0x7f11ac0b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.973+0000 7f11aa000640 1 -- 172.21.15.165:0/3499193278 >> v1:172.21.15.165:6789/0 conn(0x7f11ac1452a0 legacy=0x7f11ac0cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.973+0000 7f11aa000640 1 -- 172.21.15.165:0/3499193278 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0cc140 con 0x7f11ac144d50 2024-04-25T10:22:17.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.974+0000 7f11aa000640 1 --2- 172.21.15.165:0/3499193278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac144d50 0x7f11ac0b4460 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cf560 tx=0x7f119402a680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e02d4b8ffae85b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.974+0000 7f11a9600640 1 -- 172.21.15.165:0/3499193278 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194007740 con 0x7f11ac144d50 2024-04-25T10:22:17.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.974+0000 7f11a9600640 1 -- 172.21.15.165:0/3499193278 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11940078e0 con 0x7f11ac144d50 2024-04-25T10:22:17.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.974+0000 7f11a9600640 1 -- 172.21.15.165:0/3499193278 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1194007cb0 con 0x7f11ac144d50 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 -- 172.21.15.165:0/3499193278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac144d50 msgr2=0x7f11ac0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 --2- 172.21.15.165:0/3499193278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac144d50 0x7f11ac0b4460 secure :-1 s=READY pgs=2926 cs=0 l=1 rev1=1 crypto rx=0x7f11ac0cf560 tx=0x7f119402a680 comp rx=0 tx=0).stop 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 -- 172.21.15.165:0/3499193278 shutdown_connections 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 --2- 172.21.15.165:0/3499193278 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac144d50 0x7f11ac0b4460 unknown :-1 s=CLOSED pgs=2926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 --2- 172.21.15.165:0/3499193278 >> v2:172.21.15.172:3300/0 conn(0x7f11ac1446d0 0x7f11ac0b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 -- 172.21.15.165:0/3499193278 >> 172.21.15.165:0/3499193278 conn(0x7f11ac06abe0 msgr2=0x7f11ac06afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 -- 172.21.15.165:0/3499193278 shutdown_connections 2024-04-25T10:22:17.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.975+0000 7f11b1a00640 1 -- 172.21.15.165:0/3499193278 wait complete. 2024-04-25T10:22:17.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.977+0000 7f11b1a00640 1 Processor -- start 2024-04-25T10:22:17.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.978+0000 7f11b1a00640 1 -- start start 2024-04-25T10:22:17.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11b1a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11ac1446d0 0x7f11ac13c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11b1a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11ac144d50 0x7f11ac143290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11b1a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 0x7f11ac13ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11b1a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0cc700 con 0x7f11ac1452a0 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11b1a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0b49a0 con 0x7f11ac1446d0 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11b1a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11ac0cc9e0 con 0x7f11ac144d50 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 0x7f11ac13ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11ab400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 0x7f11ac13ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51600/0 (socket says 172.21.15.165:51600) 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11ab400640 1 -- 172.21.15.165:0/253726094 learned_addr learned my addr 172.21.15.165:0/253726094 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11aaa00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11ac1446d0 0x7f11ac13c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.979+0000 7f11aa000640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11ac144d50 0x7f11ac143290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11ab400640 1 -- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11ac144d50 msgr2=0x7f11ac143290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11ab400640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11ac144d50 0x7f11ac143290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11ab400640 1 -- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11ac1446d0 msgr2=0x7f11ac13c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11ab400640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11ac1446d0 0x7f11ac13c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11ab400640 1 -- 172.21.15.165:0/253726094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11ac0b1f30 con 0x7f11ac1452a0 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11ab400640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 0x7f11ac13ae80 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c420 tx=0x7f11a400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a4017070 con 0x7f11ac1452a0 2024-04-25T10:22:17.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a4005ce0 con 0x7f11ac1452a0 2024-04-25T10:22:17.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a40072d0 con 0x7f11ac1452a0 2024-04-25T10:22:17.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11ac0af9c0 con 0x7f11ac1452a0 2024-04-25T10:22:17.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.980+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11ac0aff80 con 0x7f11ac1452a0 2024-04-25T10:22:17.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.982+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11ac073a00 con 0x7f11ac1452a0 2024-04-25T10:22:17.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.983+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f11a401a070 con 0x7f11ac1452a0 2024-04-25T10:22:17.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.984+0000 7f11a2a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118806b4d0 0x7f118806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:17.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.985+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f11a4089410 con 0x7f11ac1452a0 2024-04-25T10:22:17.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.985+0000 7f11aaa00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118806b4d0 0x7f118806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:17.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.985+0000 7f11aaa00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118806b4d0 0x7f118806d990 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f119c006fd0 tx=0x7f119c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:17.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:17.989+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f11a4057cc0 con 0x7f11ac1452a0 2024-04-25T10:22:18.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.177+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f11ac0b01c0 con 0x7f118806b4d0 2024-04-25T10:22:18.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.178+0000 7f11a2a00640 1 -- 172.21.15.165:0/253726094 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f119c00f860 con 0x7f118806b4d0 2024-04-25T10:22:18.181 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 3 to deep-scrub 2024-04-25T10:22:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.180+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118806b4d0 msgr2=0x7f118806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.180+0000 7f11b1a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118806b4d0 0x7f118806d990 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f119c006fd0 tx=0x7f119c008040 comp rx=0 tx=0).stop 2024-04-25T10:22:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 msgr2=0x7f11ac13ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 0x7f11ac13ae80 secure :-1 s=READY pgs=2927 cs=0 l=1 rev1=1 crypto rx=0x7f11a400c420 tx=0x7f11a400c8e0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 shutdown_connections 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f118806b4d0 0x7f118806d990 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f11ac1452a0 0x7f11ac13ae80 unknown :-1 s=CLOSED pgs=2927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f11ac144d50 0x7f11ac143290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 --2- 172.21.15.165:0/253726094 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f11ac1446d0 0x7f11ac13c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 >> 172.21.15.165:0/253726094 conn(0x7f11ac06abe0 msgr2=0x7f11ac06d790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 shutdown_connections 2024-04-25T10:22:18.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.181+0000 7f11b1a00640 1 -- 172.21.15.165:0/253726094 wait complete. 2024-04-25T10:22:18.199 INFO:tasks.ceph:Scrubbing osd.4 2024-04-25T10:22:18.199 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.396+0000 7fc7efe00640 1 Processor -- start 2024-04-25T10:22:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.397+0000 7fc7efe00640 1 -- start start 2024-04-25T10:22:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7efe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc7f015dd30 0x7fc7f00cebd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f006bff0 0x7fc7f00b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7efe00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7f015e110 con 0x7fc7f00b5970 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7efe00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f006d000 con 0x7fc7f015dd30 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7efe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b2fd0 con 0x7fc7f006bff0 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc7f015dd30 0x7fc7f00cebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7eea00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fc7f015dd30 0x7fc7f00cebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51608/0 (socket says 172.21.15.165:51608) 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7eea00640 1 -- 172.21.15.165:0/2863888858 learned_addr learned my addr 172.21.15.165:0/2863888858 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.398+0000 7fc7ee000640 1 --2- 172.21.15.165:0/2863888858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f006bff0 0x7fc7f00b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.399+0000 7fc7eea00640 1 -- 172.21.15.165:0/2863888858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f006bff0 msgr2=0x7fc7f00b5430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.399+0000 7fc7eea00640 1 --2- 172.21.15.165:0/2863888858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f006bff0 0x7fc7f00b5430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.399+0000 7fc7eea00640 1 -- 172.21.15.165:0/2863888858 >> v1:172.21.15.172:6789/0 conn(0x7fc7f00b5970 legacy=0x7fc7f0161820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.399+0000 7fc7eea00640 1 -- 172.21.15.165:0/2863888858 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f006c480 con 0x7fc7f015dd30 2024-04-25T10:22:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.400+0000 7fc7eea00640 1 --2- 172.21.15.165:0/2863888858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f015dd30 0x7fc7f00cebd0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009870 tx=0x7fc7e002a210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15221db2910f446d server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.400+0000 7fc7ed600640 1 -- 172.21.15.165:0/2863888858 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0037070 con 0x7fc7f015dd30 2024-04-25T10:22:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.400+0000 7fc7ed600640 1 -- 172.21.15.165:0/2863888858 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0032440 con 0x7fc7f015dd30 2024-04-25T10:22:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.400+0000 7fc7ed600640 1 -- 172.21.15.165:0/2863888858 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0032810 con 0x7fc7f015dd30 2024-04-25T10:22:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 -- 172.21.15.165:0/2863888858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f015dd30 msgr2=0x7fc7f00cebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 --2- 172.21.15.165:0/2863888858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f015dd30 0x7fc7f00cebd0 secure :-1 s=READY pgs=2928 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0009870 tx=0x7fc7e002a210 comp rx=0 tx=0).stop 2024-04-25T10:22:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 -- 172.21.15.165:0/2863888858 shutdown_connections 2024-04-25T10:22:18.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 --2- 172.21.15.165:0/2863888858 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f006bff0 0x7fc7f00b5430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 --2- 172.21.15.165:0/2863888858 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f015dd30 0x7fc7f00cebd0 unknown :-1 s=CLOSED pgs=2928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 -- 172.21.15.165:0/2863888858 >> 172.21.15.165:0/2863888858 conn(0x7fc7f00d0610 msgr2=0x7fc7f00d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 -- 172.21.15.165:0/2863888858 shutdown_connections 2024-04-25T10:22:18.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.401+0000 7fc7efe00640 1 -- 172.21.15.165:0/2863888858 wait complete. 2024-04-25T10:22:18.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.402+0000 7fc7efe00640 1 Processor -- start 2024-04-25T10:22:18.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.403+0000 7fc7efe00640 1 -- start start 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f00b5970 0x7fc7f0154070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7f015dd30 0x7fc7f00b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7efe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f00b8380 0x7fc7f015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7efe00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f006cd20 con 0x7fc7f00b5970 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7efe00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f006ca40 con 0x7fc7f015dd30 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7efe00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00cf110 con 0x7fc7f00b8380 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f00b5970 0x7fc7f0154070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f00b5970 0x7fc7f0154070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51614/0 (socket says 172.21.15.165:51614) 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 -- 172.21.15.165:0/1446138813 learned_addr learned my addr 172.21.15.165:0/1446138813 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 -- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f00b8380 msgr2=0x7fc7f015fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fc7f00b8380 0x7fc7f015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 -- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7f015dd30 msgr2=0x7fc7f00b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fc7f015dd30 0x7fc7f00b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.404+0000 7fc7eea00640 1 -- 172.21.15.165:0/1446138813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f015e2e0 con 0x7fc7f00b5970 2024-04-25T10:22:18.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.405+0000 7fc7eea00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f00b5970 0x7fc7f0154070 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0000c00 tx=0x7fc7e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.405+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0041020 con 0x7fc7f00b5970 2024-04-25T10:22:18.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.405+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e00306c0 con 0x7fc7f00b5970 2024-04-25T10:22:18.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.405+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0037040 con 0x7fc7f00b5970 2024-04-25T10:22:18.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.405+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f01545b0 con 0x7fc7f00b5970 2024-04-25T10:22:18.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.405+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f015e860 con 0x7fc7f00b5970 2024-04-25T10:22:18.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.406+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7f0073990 con 0x7fc7f00b5970 2024-04-25T10:22:18.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.407+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fc7e0044020 con 0x7fc7f00b5970 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.408+0000 7fc7e6a00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc7cc06b500 0x7fc7cc06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.408+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fc7e00aa870 con 0x7fc7f00b5970 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.408+0000 7fc7ee000640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc7cc06b500 0x7fc7cc06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.408+0000 7fc7e6a00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fc7cc071a40 0x7fc7cc073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.408+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 --> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc7e002be50 con 0x7fc7cc071a40 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.408+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7e00af2c0 con 0x7fc7f00b5970 2024-04-25T10:22:18.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.409+0000 7fc7ee000640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc7cc06b500 0x7fc7cc06d9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80070b0 tx=0x7fc7d8007040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.409+0000 7fc7ef400640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fc7cc071a40 0x7fc7cc073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.410+0000 7fc7ef400640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fc7cc071a40 0x7fc7cc073e80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.412+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== osd.4 v2:172.21.15.172:6817/928908638 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fc7e002be50 con 0x7fc7cc071a40 2024-04-25T10:22:18.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.427+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 --> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc7f00d16c0 con 0x7fc7cc071a40 2024-04-25T10:22:18.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7e6a00640 1 -- 172.21.15.165:0/1446138813 <== osd.4 v2:172.21.15.172:6817/928908638 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fc7f00d16c0 con 0x7fc7cc071a40 2024-04-25T10:22:18.431 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:18.431 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:18.431 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:18.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fc7cc071a40 msgr2=0x7fc7cc073e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7efe00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.172:6817/928908638,v1:172.21.15.172:6819/928908638] conn(0x7fc7cc071a40 0x7fc7cc073e80 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc7cc06b500 msgr2=0x7fc7cc06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7efe00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fc7cc06b500 0x7fc7cc06d9c0 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fc7d80070b0 tx=0x7fc7d8007040 comp rx=0 tx=0).stop 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f00b5970 msgr2=0x7fc7f0154070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7efe00640 1 --2- 172.21.15.165:0/1446138813 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fc7f00b5970 0x7fc7f0154070 secure :-1 s=READY pgs=2929 cs=0 l=1 rev1=1 crypto rx=0x7fc7e0000c00 tx=0x7fc7e00046c0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.429+0000 7fc7ef400640 1 -- 172.21.15.165:0/1446138813 reap_dead start 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.430+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 shutdown_connections 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.430+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 >> 172.21.15.165:0/1446138813 conn(0x7fc7f00d0610 msgr2=0x7fc7f00b6c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.430+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 shutdown_connections 2024-04-25T10:22:18.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.430+0000 7fc7efe00640 1 -- 172.21.15.165:0/1446138813 wait complete. 2024-04-25T10:22:18.448 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-25T10:22:18.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.650+0000 7f500a400640 1 Processor -- start 2024-04-25T10:22:18.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.650+0000 7f500a400640 1 -- start start 2024-04-25T10:22:18.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.652+0000 7f500a400640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5004145140 0x7f50040ce410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.652+0000 7f500a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040ce950 0x7f50040c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.652+0000 7f500a400640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5004145520 con 0x7f50040c5030 2024-04-25T10:22:18.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.652+0000 7f500a400640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50040ced90 con 0x7f5004145140 2024-04-25T10:22:18.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.652+0000 7f500a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5004071270 con 0x7f50040ce950 2024-04-25T10:22:18.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.653+0000 7f5009000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5004145140 0x7f50040ce410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.653+0000 7f4ffbe00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040ce950 0x7f50040c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.653+0000 7f5009000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f5004145140 0x7f50040ce410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:48892/0 (socket says 172.21.15.165:48892) 2024-04-25T10:22:18.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.653+0000 7f5009000640 1 -- 172.21.15.165:0/1784502485 learned_addr learned my addr 172.21.15.165:0/1784502485 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:18.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.654+0000 7f5009000640 1 -- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040ce950 msgr2=0x7f50040c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:18.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.654+0000 7f5009000640 1 --2- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040ce950 0x7f50040c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.654+0000 7f5009000640 1 -- 172.21.15.165:0/1784502485 >> v1:172.21.15.172:6789/0 conn(0x7f50040c5030 legacy=0x7f500406d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.654+0000 7f5009000640 1 -- 172.21.15.165:0/1784502485 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5004071550 con 0x7f5004145140 2024-04-25T10:22:18.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.655+0000 7f5009000640 1 --2- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 0x7f50040ce410 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009870 tx=0x7f4ff402a1c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5abf18e8052863b server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.656+0000 7f5003e00640 1 -- 172.21.15.165:0/1784502485 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4037070 con 0x7f5004145140 2024-04-25T10:22:18.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.656+0000 7f5003e00640 1 -- 172.21.15.165:0/1784502485 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff4032440 con 0x7f5004145140 2024-04-25T10:22:18.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.656+0000 7f5003e00640 1 -- 172.21.15.165:0/1784502485 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4032810 con 0x7f5004145140 2024-04-25T10:22:18.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 -- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 msgr2=0x7f50040ce410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 --2- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 0x7f50040ce410 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009870 tx=0x7f4ff402a1c0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 -- 172.21.15.165:0/1784502485 shutdown_connections 2024-04-25T10:22:18.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 --2- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040ce950 0x7f50040c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 --2- 172.21.15.165:0/1784502485 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 0x7f50040ce410 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 -- 172.21.15.165:0/1784502485 >> 172.21.15.165:0/1784502485 conn(0x7f5004061cc0 msgr2=0x7f500406abe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 -- 172.21.15.165:0/1784502485 shutdown_connections 2024-04-25T10:22:18.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.657+0000 7f500a400640 1 -- 172.21.15.165:0/1784502485 wait complete. 2024-04-25T10:22:18.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.658+0000 7f500a400640 1 Processor -- start 2024-04-25T10:22:18.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.659+0000 7f500a400640 1 -- start start 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.659+0000 7f500a400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040c5030 0x7f50040a8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.659+0000 7f500a400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 0x7f500415afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.659+0000 7f500a400640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 0x7f500415b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f500a400640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c54a0 con 0x7f50040ce950 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f500a400640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5004071b10 con 0x7f5004145140 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f500a400640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50040c5620 con 0x7f50040c5030 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 0x7f500415afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 0x7f500415afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51624/0 (socket says 172.21.15.165:51624) 2024-04-25T10:22:18.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 -- 172.21.15.165:0/1770823415 learned_addr learned my addr 172.21.15.165:0/1770823415 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f5009000640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040c5030 0x7f50040a8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 -- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040c5030 msgr2=0x7f50040a8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040c5030 0x7f50040a8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 -- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 msgr2=0x7f500415b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 0x7f500415b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.660+0000 7f4ffbe00640 1 -- 172.21.15.165:0/1770823415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500413d640 con 0x7f50040ce950 2024-04-25T10:22:18.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.661+0000 7f4ffbe00640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 0x7f500415afc0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00e9b0 tx=0x7f4fec00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.661+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec017070 con 0x7f50040ce950 2024-04-25T10:22:18.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.661+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fec00c9a0 con 0x7f50040ce950 2024-04-25T10:22:18.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.661+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f500413d920 con 0x7f50040ce950 2024-04-25T10:22:18.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.661+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fec00cd90 con 0x7f50040ce950 2024-04-25T10:22:18.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.661+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500413dee0 con 0x7f50040ce950 2024-04-25T10:22:18.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.662+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500413e320 con 0x7f50040ce950 2024-04-25T10:22:18.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.663+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f4fec01a070 con 0x7f50040ce950 2024-04-25T10:22:18.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.664+0000 7f5001600640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.664+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f4fec08a3e0 con 0x7f50040ce950 2024-04-25T10:22:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.664+0000 7f5009000640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:18.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.665+0000 7f5009000640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4fe006b4d0 0x7f4fe006d990 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4034000 tx=0x7f4ff402e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:18.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.668+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f4fec059410 con 0x7f50040ce950 2024-04-25T10:22:18.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.863+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5004073990 con 0x7f4fe006b4d0 2024-04-25T10:22:18.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.864+0000 7f5001600640 1 -- 172.21.15.165:0/1770823415 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f5004073990 con 0x7f4fe006b4d0 2024-04-25T10:22:18.867 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 4 to deep-scrub 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4fe006b4d0 msgr2=0x7f4fe006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4fe006b4d0 0x7f4fe006d990 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4034000 tx=0x7f4ff402e000 comp rx=0 tx=0).stop 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 msgr2=0x7f500415afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 0x7f500415afc0 secure :-1 s=READY pgs=2930 cs=0 l=1 rev1=1 crypto rx=0x7f4fec00e9b0 tx=0x7f4fec00ee70 comp rx=0 tx=0).stop 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 shutdown_connections 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f4fe006b4d0 0x7f4fe006d990 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f5004145140 0x7f500415b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f50040ce950 0x7f500415afc0 unknown :-1 s=CLOSED pgs=2930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 --2- 172.21.15.165:0/1770823415 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f50040c5030 0x7f50040a8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:18.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 >> 172.21.15.165:0/1770823415 conn(0x7f5004061cc0 msgr2=0x7f50040b2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:18.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 shutdown_connections 2024-04-25T10:22:18.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:18.867+0000 7f500a400640 1 -- 172.21.15.165:0/1770823415 wait complete. 2024-04-25T10:22:18.890 INFO:tasks.ceph:Scrubbing osd.5 2024-04-25T10:22:18.891 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.089+0000 7fd678e00640 1 Processor -- start 2024-04-25T10:22:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.090+0000 7fd678e00640 1 -- start start 2024-04-25T10:22:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.091+0000 7fd678e00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd6741446d0 0x7fd6740b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.091+0000 7fd678e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd67406e8a0 0x7fd6740b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.091+0000 7fd678e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6740cccc0 con 0x7fd674144d50 2024-04-25T10:22:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.091+0000 7fd678e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6740b2b20 con 0x7fd67406e8a0 2024-04-25T10:22:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.091+0000 7fd678e00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67406d400 con 0x7fd6741446d0 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd67406e8a0 0x7fd6740b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fd67406e8a0 0x7fd6740b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51630/0 (socket says 172.21.15.165:51630) 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 -- 172.21.15.165:0/4005494823 learned_addr learned my addr 172.21.15.165:0/4005494823 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd671e00640 1 --2- 172.21.15.165:0/4005494823 >> v2:172.21.15.172:3300/0 conn(0x7fd6741446d0 0x7fd6740b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 -- 172.21.15.165:0/4005494823 >> v2:172.21.15.172:3300/0 conn(0x7fd6741446d0 msgr2=0x7fd6740b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 --2- 172.21.15.165:0/4005494823 >> v2:172.21.15.172:3300/0 conn(0x7fd6741446d0 0x7fd6740b3f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 -- 172.21.15.165:0/4005494823 >> v1:172.21.15.172:6789/0 conn(0x7fd674144d50 legacy=0x7fd6740cf640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.092+0000 7fd672800640 1 -- 172.21.15.165:0/4005494823 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740cc140 con 0x7fd67406e8a0 2024-04-25T10:22:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.093+0000 7fd672800640 1 --2- 172.21.15.165:0/4005494823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd6740b4460 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fd6740cf560 tx=0x7fd66c02a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8a084372211bb2e server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.093+0000 7fd671400640 1 -- 172.21.15.165:0/4005494823 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c036070 con 0x7fd67406e8a0 2024-04-25T10:22:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.093+0000 7fd671400640 1 -- 172.21.15.165:0/4005494823 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66c031600 con 0x7fd67406e8a0 2024-04-25T10:22:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.093+0000 7fd671400640 1 -- 172.21.15.165:0/4005494823 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c0319d0 con 0x7fd67406e8a0 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 -- 172.21.15.165:0/4005494823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 msgr2=0x7fd6740b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 --2- 172.21.15.165:0/4005494823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd6740b4460 secure :-1 s=READY pgs=2931 cs=0 l=1 rev1=1 crypto rx=0x7fd6740cf560 tx=0x7fd66c02a6f0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 -- 172.21.15.165:0/4005494823 shutdown_connections 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 --2- 172.21.15.165:0/4005494823 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd6740b4460 unknown :-1 s=CLOSED pgs=2931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 --2- 172.21.15.165:0/4005494823 >> v2:172.21.15.172:3300/0 conn(0x7fd6741446d0 0x7fd6740b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 -- 172.21.15.165:0/4005494823 >> 172.21.15.165:0/4005494823 conn(0x7fd67406abe0 msgr2=0x7fd67406afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 -- 172.21.15.165:0/4005494823 shutdown_connections 2024-04-25T10:22:19.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.094+0000 7fd678e00640 1 -- 172.21.15.165:0/4005494823 wait complete. 2024-04-25T10:22:19.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.096+0000 7fd678e00640 1 Processor -- start 2024-04-25T10:22:19.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.096+0000 7fd678e00640 1 -- start start 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd678e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd67415b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd678e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6741446d0 0x7fd67413ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd678e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd674144d50 0x7fd67413ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd678e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740cc700 con 0x7fd67406e8a0 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd678e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740b49a0 con 0x7fd674144d50 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd678e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6740cc9e0 con 0x7fd6741446d0 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd671e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd67415b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd671e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd67415b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51632/0 (socket says 172.21.15.165:51632) 2024-04-25T10:22:19.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.097+0000 7fd671e00640 1 -- 172.21.15.165:0/3915223116 learned_addr learned my addr 172.21.15.165:0/3915223116 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:19.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd671e00640 1 -- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6741446d0 msgr2=0x7fd67413ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd671e00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd6741446d0 0x7fd67413ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd671e00640 1 -- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd674144d50 msgr2=0x7fd67413ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd671e00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd674144d50 0x7fd67413ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd671e00640 1 -- 172.21.15.165:0/3915223116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6740b1230 con 0x7fd67406e8a0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd671e00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd67415b100 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c820 tx=0x7fd66400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd664015070 con 0x7fd67406e8a0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd664004240 con 0x7fd67406e8a0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66400ce10 con 0x7fd67406e8a0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.098+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6740b1510 con 0x7fd67406e8a0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.099+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6740b1ad0 con 0x7fd67406e8a0 2024-04-25T10:22:19.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.100+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd67415b640 con 0x7fd67406e8a0 2024-04-25T10:22:19.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.101+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd664007830 con 0x7fd67406e8a0 2024-04-25T10:22:19.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.102+0000 7fd66a000640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd65006b4d0 0x7fd65006d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.102+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd66408a060 con 0x7fd67406e8a0 2024-04-25T10:22:19.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.102+0000 7fd66be00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd65006b4d0 0x7fd65006d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.102+0000 7fd66a000640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fd650071a10 0x7fd650073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.102+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 --> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd650074520 con 0x7fd650071a10 2024-04-25T10:22:19.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.102+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd664091050 con 0x7fd67406e8a0 2024-04-25T10:22:19.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.103+0000 7fd66be00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd65006b4d0 0x7fd65006d990 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009650 tx=0x7fd65c006ba0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.103+0000 7fd672800640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fd650071a10 0x7fd650073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.104+0000 7fd672800640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fd650071a10 0x7fd650073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.106+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== osd.5 v2:172.21.15.172:6800/4199573342 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd650074520 con 0x7fd650071a10 2024-04-25T10:22:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.121+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 --> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fd6740b1d10 con 0x7fd650071a10 2024-04-25T10:22:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.122+0000 7fd66a000640 1 -- 172.21.15.165:0/3915223116 <== osd.5 v2:172.21.15.172:6800/4199573342 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7fd66c037040 con 0x7fd650071a10 2024-04-25T10:22:19.125 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:19.125 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:19.125 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fd650071a10 msgr2=0x7fd650073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.172:6800/4199573342,v1:172.21.15.172:6801/4199573342] conn(0x7fd650071a10 0x7fd650073e50 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd65006b4d0 msgr2=0x7fd65006d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd65006b4d0 0x7fd65006d990 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7fd65c009650 tx=0x7fd65c006ba0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 msgr2=0x7fd67415b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 --2- 172.21.15.165:0/3915223116 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd67406e8a0 0x7fd67415b100 secure :-1 s=READY pgs=2932 cs=0 l=1 rev1=1 crypto rx=0x7fd66400c820 tx=0x7fd66400cce0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd672800640 1 -- 172.21.15.165:0/3915223116 reap_dead start 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 shutdown_connections 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 >> 172.21.15.165:0/3915223116 conn(0x7fd67406abe0 msgr2=0x7fd6740cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 shutdown_connections 2024-04-25T10:22:19.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.123+0000 7fd678e00640 1 -- 172.21.15.165:0/3915223116 wait complete. 2024-04-25T10:22:19.145 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-25T10:22:19.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.336+0000 7f0f2fc00640 1 Processor -- start 2024-04-25T10:22:19.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.337+0000 7f0f2fc00640 1 -- start start 2024-04-25T10:22:19.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.338+0000 7f0f2fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f28071480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.338+0000 7f0f2fc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0f2806ef10 0x7f0f2806d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.338+0000 7f0f2fc00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f28162ba0 con 0x7f0f28161fc0 2024-04-25T10:22:19.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.338+0000 7f0f2fc00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f28165be0 con 0x7f0f2806ef10 2024-04-25T10:22:19.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.338+0000 7f0f2fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28162e80 con 0x7f0f2806ea90 2024-04-25T10:22:19.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2dc00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f0f2806ef10 0x7f0f2806d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2d200640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f0f28161fc0 0x7f0f281623a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52360/0 (socket says 172.21.15.165:52360) 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2d200640 1 -- 172.21.15.165:0/2665353232 learned_addr learned my addr 172.21.15.165:0/2665353232 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f27e00640 1 --2- 172.21.15.165:0/2665353232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f28071480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2dc00640 1 -- 172.21.15.165:0/2665353232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 msgr2=0x7f0f28071480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2dc00640 1 --2- 172.21.15.165:0/2665353232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f28071480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2dc00640 1 -- 172.21.15.165:0/2665353232 >> v1:172.21.15.165:6789/0 conn(0x7f0f28161fc0 legacy=0x7f0f281623a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f2dc00640 1 -- 172.21.15.165:0/2665353232 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f28163160 con 0x7f0f2806ef10 2024-04-25T10:22:19.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.339+0000 7f0f27400640 1 -- 172.21.15.165:0/2665353232 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376487390 0 0) 0x7f0f28162ba0 con 0x7f0f28161fc0 2024-04-25T10:22:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.340+0000 7f0f2dc00640 1 --2- 172.21.15.165:0/2665353232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f2806ef10 0x7f0f2806d2d0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009d90 tx=0x7f0f2002a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847f40306aa37153 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.340+0000 7f0f27400640 1 -- 172.21.15.165:0/2665353232 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20031600 con 0x7f0f2806ef10 2024-04-25T10:22:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.340+0000 7f0f27400640 1 -- 172.21.15.165:0/2665353232 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f200317a0 con 0x7f0f2806ef10 2024-04-25T10:22:19.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.340+0000 7f0f27400640 1 -- 172.21.15.165:0/2665353232 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20031b70 con 0x7f0f2806ef10 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 -- 172.21.15.165:0/2665353232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f2806ef10 msgr2=0x7f0f2806d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/2665353232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f2806ef10 0x7f0f2806d2d0 secure :-1 s=READY pgs=2934 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009d90 tx=0x7f0f2002a5d0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 -- 172.21.15.165:0/2665353232 shutdown_connections 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/2665353232 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f2806ef10 0x7f0f2806d2d0 unknown :-1 s=CLOSED pgs=2934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/2665353232 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f28071480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 -- 172.21.15.165:0/2665353232 >> 172.21.15.165:0/2665353232 conn(0x7f0f2806abe0 msgr2=0x7f0f2806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 -- 172.21.15.165:0/2665353232 shutdown_connections 2024-04-25T10:22:19.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.341+0000 7f0f2fc00640 1 -- 172.21.15.165:0/2665353232 wait complete. 2024-04-25T10:22:19.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.343+0000 7f0f2fc00640 1 Processor -- start 2024-04-25T10:22:19.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.343+0000 7f0f2fc00640 1 -- start start 2024-04-25T10:22:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2fc00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f2814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2fc00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f2806ef10 0x7f0f280aba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2fc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 0x7f0f280abfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2fc00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f28071b40 con 0x7f0f28161fc0 2024-04-25T10:22:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2fc00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f280719c0 con 0x7f0f2806ef10 2024-04-25T10:22:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2fc00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f2806d810 con 0x7f0f2806ea90 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 0x7f0f280abfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2dc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 0x7f0f280abfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51656/0 (socket says 172.21.15.165:51656) 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.344+0000 7f0f2dc00640 1 -- 172.21.15.165:0/1163796207 learned_addr learned my addr 172.21.15.165:0/1163796207 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2d200640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f2814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2dc00640 1 -- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 msgr2=0x7f0f2814f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f27e00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f2806ef10 0x7f0f280aba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2dc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f2814f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2dc00640 1 -- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f2806ef10 msgr2=0x7f0f280aba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2dc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f2806ef10 0x7f0f280aba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2dc00640 1 -- 172.21.15.165:0/1163796207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f281fada0 con 0x7f0f28161fc0 2024-04-25T10:22:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2d200640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f2814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.345+0000 7f0f2dc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 0x7f0f280abfd0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009eb0 tx=0x7f0f20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.346+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20040070 con 0x7f0f28161fc0 2024-04-25T10:22:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.346+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f20030b60 con 0x7f0f28161fc0 2024-04-25T10:22:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.346+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f20009450 con 0x7f0f28161fc0 2024-04-25T10:22:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.346+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f281fb080 con 0x7f0f28161fc0 2024-04-25T10:22:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.346+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f281fb640 con 0x7f0f28161fc0 2024-04-25T10:22:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.347+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f28073a00 con 0x7f0f28161fc0 2024-04-25T10:22:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.348+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f0f20039040 con 0x7f0f28161fc0 2024-04-25T10:22:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.349+0000 7f0f24c00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.349+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f0f200ab7a0 con 0x7f0f28161fc0 2024-04-25T10:22:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.349+0000 7f0f2d200640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.350+0000 7f0f2d200640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b4d0 0x7f0f0806d990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0f18000f60 tx=0x7f0f18009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.354+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f0f2007a7d0 con 0x7f0f28161fc0 2024-04-25T10:22:19.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.551+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f281fba20 con 0x7f0f0806b4d0 2024-04-25T10:22:19.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.552+0000 7f0f24c00640 1 -- 172.21.15.165:0/1163796207 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0f281fba20 con 0x7f0f0806b4d0 2024-04-25T10:22:19.554 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 5 to deep-scrub 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b4d0 msgr2=0x7f0f0806d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b4d0 0x7f0f0806d990 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f0f18000f60 tx=0x7f0f18009040 comp rx=0 tx=0).stop 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 msgr2=0x7f0f280abfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 0x7f0f280abfd0 secure :-1 s=READY pgs=2935 cs=0 l=1 rev1=1 crypto rx=0x7f0f20009eb0 tx=0x7f0f20004a40 comp rx=0 tx=0).stop 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 shutdown_connections 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f0f0806b4d0 0x7f0f0806d990 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f0f28161fc0 0x7f0f280abfd0 unknown :-1 s=CLOSED pgs=2935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f0f2806ef10 0x7f0f280aba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 --2- 172.21.15.165:0/1163796207 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f0f2806ea90 0x7f0f2814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 >> 172.21.15.165:0/1163796207 conn(0x7f0f2806abe0 msgr2=0x7f0f280b04c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 shutdown_connections 2024-04-25T10:22:19.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.555+0000 7f0f2fc00640 1 -- 172.21.15.165:0/1163796207 wait complete. 2024-04-25T10:22:19.577 INFO:tasks.ceph:Scrubbing osd.6 2024-04-25T10:22:19.578 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.782+0000 7f8a5e600640 1 Processor -- start 2024-04-25T10:22:19.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.782+0000 7f8a5e600640 1 -- start start 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a5e600640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f8a581452a0 0x7f8a5806dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a5e600640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580cccc0 con 0x7f8a581446d0 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a5e600640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a580cc140 con 0x7f8a58144d50 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a5e600640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a580b2b20 con 0x7f8a581452a0 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a57400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f8a581446d0 0x7f8a580b3f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52376/0 (socket says 172.21.15.165:52376) 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a57400640 1 -- 172.21.15.165:0/2426242096 learned_addr learned my addr 172.21.15.165:0/2426242096 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:19.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.784+0000 7f8a57e00640 1 --2- 172.21.15.165:0/2426242096 >> v2:172.21.15.165:3300/0 conn(0x7f8a581452a0 0x7f8a5806dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.785+0000 7f8a56000640 1 -- 172.21.15.165:0/2426242096 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3738231402 0 0) 0x7f8a580cccc0 con 0x7f8a581446d0 2024-04-25T10:22:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.785+0000 7f8a56000640 1 -- 172.21.15.165:0/2426242096 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a3c003520 con 0x7f8a581446d0 2024-04-25T10:22:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.785+0000 7f8a57e00640 1 -- 172.21.15.165:0/2426242096 >> v1:172.21.15.172:6789/0 conn(0x7f8a58144d50 legacy=0x7f8a5806d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.785+0000 7f8a57e00640 1 -- 172.21.15.165:0/2426242096 >> v1:172.21.15.165:6789/0 conn(0x7f8a581446d0 legacy=0x7f8a580b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.785+0000 7f8a57e00640 1 -- 172.21.15.165:0/2426242096 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a580cc420 con 0x7f8a581452a0 2024-04-25T10:22:19.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.786+0000 7f8a57e00640 1 --2- 172.21.15.165:0/2426242096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a581452a0 0x7f8a5806dfa0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f8a500097a0 tx=0x7f8a5002a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b690abb8cdc60a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.786+0000 7f8a56000640 1 -- 172.21.15.165:0/2426242096 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50037070 con 0x7f8a581452a0 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.786+0000 7f8a56000640 1 -- 172.21.15.165:0/2426242096 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50032440 con 0x7f8a581452a0 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.786+0000 7f8a56000640 1 -- 172.21.15.165:0/2426242096 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a500327f0 con 0x7f8a581452a0 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 -- 172.21.15.165:0/2426242096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a581452a0 msgr2=0x7f8a5806dfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 --2- 172.21.15.165:0/2426242096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a581452a0 0x7f8a5806dfa0 secure :-1 s=READY pgs=2937 cs=0 l=1 rev1=1 crypto rx=0x7f8a500097a0 tx=0x7f8a5002a1e0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 -- 172.21.15.165:0/2426242096 shutdown_connections 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 --2- 172.21.15.165:0/2426242096 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a581452a0 0x7f8a5806dfa0 unknown :-1 s=CLOSED pgs=2937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 -- 172.21.15.165:0/2426242096 >> 172.21.15.165:0/2426242096 conn(0x7f8a5806abe0 msgr2=0x7f8a5806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 -- 172.21.15.165:0/2426242096 shutdown_connections 2024-04-25T10:22:19.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.787+0000 7f8a5e600640 1 -- 172.21.15.165:0/2426242096 wait complete. 2024-04-25T10:22:19.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.789+0000 7f8a5e600640 1 Processor -- start 2024-04-25T10:22:19.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.789+0000 7f8a5e600640 1 -- start start 2024-04-25T10:22:19.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a5e600640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a581446d0 0x7f8a581f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a5e600640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a58144d50 0x7f8a580b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a5e600640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a581452a0 0x7f8a580b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a5e600640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a580cc9e0 con 0x7f8a58144d50 2024-04-25T10:22:19.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a5e600640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a5806d400 con 0x7f8a581446d0 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a5e600640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a58144ab0 con 0x7f8a581452a0 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a56a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a58144d50 0x7f8a580b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a56a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a58144d50 0x7f8a580b0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51662/0 (socket says 172.21.15.165:51662) 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.790+0000 7f8a56a00640 1 -- 172.21.15.165:0/2346680433 learned_addr learned my addr 172.21.15.165:0/2346680433 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a56a00640 1 -- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a581452a0 msgr2=0x7f8a580b1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a56a00640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f8a581452a0 0x7f8a580b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a56a00640 1 -- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a581446d0 msgr2=0x7f8a581f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a56a00640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f8a581446d0 0x7f8a581f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a56a00640 1 -- 172.21.15.165:0/2346680433 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5813be60 con 0x7f8a58144d50 2024-04-25T10:22:19.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a56a00640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a58144d50 0x7f8a580b0cc0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800ba50 tx=0x7f8a4800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.791+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48019070 con 0x7f8a58144d50 2024-04-25T10:22:19.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.792+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4800cc20 con 0x7f8a58144d50 2024-04-25T10:22:19.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.792+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5813c140 con 0x7f8a58144d50 2024-04-25T10:22:19.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.792+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a48014680 con 0x7f8a58144d50 2024-04-25T10:22:19.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.792+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5813c700 con 0x7f8a58144d50 2024-04-25T10:22:19.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.793+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a5813bc00 con 0x7f8a58144d50 2024-04-25T10:22:19.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.794+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f8a4801c070 con 0x7f8a58144d50 2024-04-25T10:22:19.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.795+0000 7f8a4f400640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3406b4d0 0x7f8a3406d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.795+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f8a48089e50 con 0x7f8a58144d50 2024-04-25T10:22:19.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.795+0000 7f8a57400640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3406b4d0 0x7f8a3406d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.795+0000 7f8a4f400640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f8a34071a10 0x7f8a34073e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:19.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.795+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a34074520 con 0x7f8a34071a10 2024-04-25T10:22:19.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.795+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a4808c470 con 0x7f8a58144d50 2024-04-25T10:22:19.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.796+0000 7f8a57400640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3406b4d0 0x7f8a3406d990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000f60 tx=0x7f8a400078f0 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.796+0000 7f8a57e00640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f8a34071a10 0x7f8a34073e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:19.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.797+0000 7f8a57e00640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f8a34071a10 0x7f8a34073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:19.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.799+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== osd.6 v2:172.21.15.172:6808/4001543510 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8a34074520 con 0x7f8a34071a10 2024-04-25T10:22:19.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.813+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 --> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8a58073a00 con 0x7f8a34071a10 2024-04-25T10:22:19.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.814+0000 7f8a4f400640 1 -- 172.21.15.165:0/2346680433 <== osd.6 v2:172.21.15.172:6808/4001543510 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7f8a58073a00 con 0x7f8a34071a10 2024-04-25T10:22:19.817 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:19.817 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:19.817 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:19.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f8a34071a10 msgr2=0x7f8a34073e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.172:6808/4001543510,v1:172.21.15.172:6809/4001543510] conn(0x7f8a34071a10 0x7f8a34073e50 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3406b4d0 msgr2=0x7f8a3406d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f8a3406b4d0 0x7f8a3406d990 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f8a40000f60 tx=0x7f8a400078f0 comp rx=0 tx=0).stop 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a58144d50 msgr2=0x7f8a580b0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 --2- 172.21.15.165:0/2346680433 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f8a58144d50 0x7f8a580b0cc0 secure :-1 s=READY pgs=2938 cs=0 l=1 rev1=1 crypto rx=0x7f8a4800ba50 tx=0x7f8a4800bf10 comp rx=0 tx=0).stop 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a57e00640 1 -- 172.21.15.165:0/2346680433 reap_dead start 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 shutdown_connections 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 >> 172.21.15.165:0/2346680433 conn(0x7f8a5806abe0 msgr2=0x7f8a58143870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.815+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 shutdown_connections 2024-04-25T10:22:19.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:19.816+0000 7f8a5e600640 1 -- 172.21.15.165:0/2346680433 wait complete. 2024-04-25T10:22:19.836 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-25T10:22:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.032+0000 7f358e000640 1 Processor -- start 2024-04-25T10:22:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.032+0000 7f358e000640 1 -- start start 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.033+0000 7f358e000640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f358806bff0 0x7f35880b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.033+0000 7f358e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35880b5970 0x7f35880d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.033+0000 7f358e000640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f358815e110 con 0x7f358815dd30 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.033+0000 7f358e000640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f358806d000 con 0x7f358806bff0 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.033+0000 7f358e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880b2fd0 con 0x7f35880b5970 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f358cc00640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f358815dd30 0x7f35880cebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:52390/0 (socket says 172.21.15.165:52390) 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f358cc00640 1 -- 172.21.15.165:0/1531098548 learned_addr learned my addr 172.21.15.165:0/1531098548 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f357fe00640 1 --2- 172.21.15.165:0/1531098548 >> v2:172.21.15.165:3300/0 conn(0x7f358806bff0 0x7f35880b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f3587e00640 1 -- 172.21.15.165:0/1531098548 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048050792 0 0) 0x7f358815e110 con 0x7f358815dd30 2024-04-25T10:22:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f3587e00640 1 -- 172.21.15.165:0/1531098548 --> v1:172.21.15.165:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f356c003520 con 0x7f358815dd30 2024-04-25T10:22:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f358d600640 1 --2- 172.21.15.165:0/1531098548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35880b5970 0x7f35880d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f357fe00640 1 -- 172.21.15.165:0/1531098548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35880b5970 msgr2=0x7f35880d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f357fe00640 1 --2- 172.21.15.165:0/1531098548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35880b5970 0x7f35880d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f357fe00640 1 -- 172.21.15.165:0/1531098548 >> v1:172.21.15.165:6789/0 conn(0x7f358815dd30 legacy=0x7f35880cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.034+0000 7f357fe00640 1 -- 172.21.15.165:0/1531098548 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f358806c480 con 0x7f358806bff0 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.035+0000 7f357fe00640 1 --2- 172.21.15.165:0/1531098548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f358806bff0 0x7f35880b5430 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f35700097a0 tx=0x7f357002a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b46ff2bb71bb18a server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f3587e00640 1 -- 172.21.15.165:0/1531098548 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570037070 con 0x7f358806bff0 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f3587e00640 1 -- 172.21.15.165:0/1531098548 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3570032440 con 0x7f358806bff0 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f3587e00640 1 -- 172.21.15.165:0/1531098548 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35700327f0 con 0x7f358806bff0 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 -- 172.21.15.165:0/1531098548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f358806bff0 msgr2=0x7f35880b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 --2- 172.21.15.165:0/1531098548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f358806bff0 0x7f35880b5430 secure :-1 s=READY pgs=2940 cs=0 l=1 rev1=1 crypto rx=0x7f35700097a0 tx=0x7f357002a180 comp rx=0 tx=0).stop 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 -- 172.21.15.165:0/1531098548 shutdown_connections 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 --2- 172.21.15.165:0/1531098548 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f35880b5970 0x7f35880d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 --2- 172.21.15.165:0/1531098548 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f358806bff0 0x7f35880b5430 unknown :-1 s=CLOSED pgs=2940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 -- 172.21.15.165:0/1531098548 >> 172.21.15.165:0/1531098548 conn(0x7f35880d0610 msgr2=0x7f35880d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 -- 172.21.15.165:0/1531098548 shutdown_connections 2024-04-25T10:22:20.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.036+0000 7f358e000640 1 -- 172.21.15.165:0/1531098548 wait complete. 2024-04-25T10:22:20.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.038+0000 7f358e000640 1 Processor -- start 2024-04-25T10:22:20.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.038+0000 7f358e000640 1 -- start start 2024-04-25T10:22:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 0x7f35880cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358e000640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358815dd30 0x7f35881f2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358e000640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3588156140 0x7f35881565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358e000640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358806cd20 con 0x7f35880b5970 2024-04-25T10:22:20.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358e000640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358806ca40 con 0x7f358815dd30 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358e000640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35880d26a0 con 0x7f3588156140 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 0x7f35880cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.039+0000 7f358cc00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 0x7f35880cbad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51694/0 (socket says 172.21.15.165:51694) 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 -- 172.21.15.165:0/1875697368 learned_addr learned my addr 172.21.15.165:0/1875697368 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 -- 172.21.15.165:0/1875697368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3588156140 msgr2=0x7f35881565c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3588156140 0x7f35881565c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 -- 172.21.15.165:0/1875697368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358815dd30 msgr2=0x7f35881f2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358815dd30 0x7f35881f2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 -- 172.21.15.165:0/1875697368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880cc010 con 0x7f35880b5970 2024-04-25T10:22:20.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f358cc00640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 0x7f35880cbad0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f357800c390 tx=0x7f357800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.040+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3578017070 con 0x7f35880b5970 2024-04-25T10:22:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.041+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3578005ce0 con 0x7f35880b5970 2024-04-25T10:22:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.041+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35780072d0 con 0x7f35880b5970 2024-04-25T10:22:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.041+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35881f3290 con 0x7f35880b5970 2024-04-25T10:22:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.041+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f358815ea50 con 0x7f35880b5970 2024-04-25T10:22:20.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.042+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3588073990 con 0x7f35880b5970 2024-04-25T10:22:20.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.043+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f357801a070 con 0x7f35880b5970 2024-04-25T10:22:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.044+0000 7f3585600640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f356406b500 0x7f356406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.044+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f35780892d0 con 0x7f35880b5970 2024-04-25T10:22:20.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.044+0000 7f357fe00640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f356406b500 0x7f356406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.045+0000 7f357fe00640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f356406b500 0x7f356406d9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3570034000 tx=0x7f357002c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.048+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f3578057cb0 con 0x7f35880b5970 2024-04-25T10:22:20.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.244+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f35880d16c0 con 0x7f356406b500 2024-04-25T10:22:20.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.245+0000 7f3585600640 1 -- 172.21.15.165:0/1875697368 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f35880d16c0 con 0x7f356406b500 2024-04-25T10:22:20.247 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 6 to deep-scrub 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f356406b500 msgr2=0x7f356406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f356406b500 0x7f356406d9c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f3570034000 tx=0x7f357002c000 comp rx=0 tx=0).stop 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 msgr2=0x7f35880cbad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 0x7f35880cbad0 secure :-1 s=READY pgs=2941 cs=0 l=1 rev1=1 crypto rx=0x7f357800c390 tx=0x7f357800c850 comp rx=0 tx=0).stop 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 shutdown_connections 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f356406b500 0x7f356406d9c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f3588156140 0x7f35881565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f358815dd30 0x7f35881f2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 --2- 172.21.15.165:0/1875697368 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f35880b5970 0x7f35880cbad0 unknown :-1 s=CLOSED pgs=2941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 >> 172.21.15.165:0/1875697368 conn(0x7f35880d0610 msgr2=0x7f358815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 shutdown_connections 2024-04-25T10:22:20.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.248+0000 7f358e000640 1 -- 172.21.15.165:0/1875697368 wait complete. 2024-04-25T10:22:20.269 INFO:tasks.ceph:Scrubbing osd.7 2024-04-25T10:22:20.269 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-25T10:22:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.472+0000 7ff94f200640 1 Processor -- start 2024-04-25T10:22:20.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.473+0000 7ff94f200640 1 -- start start 2024-04-25T10:22:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.474+0000 7ff94f200640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff948144d50 0x7ff94806d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.474+0000 7ff94f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff94806ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.474+0000 7ff94f200640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9480cccc0 con 0x7ff9481446d0 2024-04-25T10:22:20.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.474+0000 7ff94f200640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9480b2b20 con 0x7ff948144d50 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.474+0000 7ff94f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94806d400 con 0x7ff9481452a0 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff948144d50 0x7ff94806d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7ff948144d50 0x7ff94806d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51704/0 (socket says 172.21.15.165:51704) 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 -- 172.21.15.165:0/2313174418 learned_addr learned my addr 172.21.15.165:0/2313174418 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff94d200640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff94806ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 -- 172.21.15.165:0/2313174418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 msgr2=0x7ff94806ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff94806ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 -- 172.21.15.165:0/2313174418 >> v1:172.21.15.172:6789/0 conn(0x7ff9481446d0 legacy=0x7ff9480b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff93f400640 1 -- 172.21.15.165:0/2313174418 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480cc140 con 0x7ff948144d50 2024-04-25T10:22:20.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.475+0000 7ff94d200640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff94806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:20.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.476+0000 7ff93f400640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948144d50 0x7ff94806d890 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7ff930009870 tx=0x7ff93002a200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d248a3f7f4ec3fd server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.476+0000 7ff947400640 1 -- 172.21.15.165:0/2313174418 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930037070 con 0x7ff948144d50 2024-04-25T10:22:20.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff947400640 1 -- 172.21.15.165:0/2313174418 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff930032440 con 0x7ff948144d50 2024-04-25T10:22:20.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff947400640 1 -- 172.21.15.165:0/2313174418 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930032810 con 0x7ff948144d50 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 -- 172.21.15.165:0/2313174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948144d50 msgr2=0x7ff94806d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948144d50 0x7ff94806d890 secure :-1 s=READY pgs=2942 cs=0 l=1 rev1=1 crypto rx=0x7ff930009870 tx=0x7ff93002a200 comp rx=0 tx=0).stop 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 -- 172.21.15.165:0/2313174418 shutdown_connections 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff94806ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 --2- 172.21.15.165:0/2313174418 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff948144d50 0x7ff94806d890 unknown :-1 s=CLOSED pgs=2942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 -- 172.21.15.165:0/2313174418 >> 172.21.15.165:0/2313174418 conn(0x7ff94806abe0 msgr2=0x7ff94806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.477+0000 7ff94f200640 1 -- 172.21.15.165:0/2313174418 shutdown_connections 2024-04-25T10:22:20.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.478+0000 7ff94f200640 1 -- 172.21.15.165:0/2313174418 wait complete. 2024-04-25T10:22:20.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.480+0000 7ff94f200640 1 Processor -- start 2024-04-25T10:22:20.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.480+0000 7ff94f200640 1 -- start start 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94f200640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9481446d0 0x7ff94813c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94f200640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948144d50 0x7ff94813e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94f200640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff948143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94f200640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480cc9e0 con 0x7ff9481446d0 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94f200640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480cc700 con 0x7ff948144d50 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94f200640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff948144ab0 con 0x7ff9481452a0 2024-04-25T10:22:20.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff947e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9481446d0 0x7ff94813c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff947e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9481446d0 0x7ff94813c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51720/0 (socket says 172.21.15.165:51720) 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff947e00640 1 -- 172.21.15.165:0/2293994582 learned_addr learned my addr 172.21.15.165:0/2293994582 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff94d200640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff948143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.481+0000 7ff93f400640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948144d50 0x7ff94813e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff947e00640 1 -- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 msgr2=0x7ff948143460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff947e00640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7ff9481452a0 0x7ff948143460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff947e00640 1 -- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948144d50 msgr2=0x7ff94813e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff947e00640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948144d50 0x7ff94813e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff947e00640 1 -- 172.21.15.165:0/2293994582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480cd650 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff93f400640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7ff948144d50 0x7ff94813e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff947e00640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9481446d0 0x7ff94813c100 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b3f0 tx=0x7ff93800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938015020 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff938011830 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9480cd930 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff938011c20 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.482+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9480cdef0 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.483+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff948073a00 con 0x7ff9481446d0 2024-04-25T10:22:20.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.484+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7ff938018020 con 0x7ff9481446d0 2024-04-25T10:22:20.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.485+0000 7ff944c00640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92406b500 0x7ff92406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.485+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7ff938089ce0 con 0x7ff9481446d0 2024-04-25T10:22:20.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.485+0000 7ff93f400640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92406b500 0x7ff92406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.486+0000 7ff944c00640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7ff924071a40 0x7ff924073e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.486+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 --> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff924074550 con 0x7ff924071a40 2024-04-25T10:22:20.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.486+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_get_version_reply(handle=1 version=201) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93808c3e0 con 0x7ff9481446d0 2024-04-25T10:22:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.486+0000 7ff94d200640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7ff924071a40 0x7ff924073e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.486+0000 7ff93f400640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92406b500 0x7ff92406d9c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff930034000 tx=0x7ff93002e000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.487+0000 7ff94d200640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7ff924071a40 0x7ff924073e80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.489+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== osd.7 v2:172.21.15.172:6816/3337160074 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7ff924074550 con 0x7ff924071a40 2024-04-25T10:22:20.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.504+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 --> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff9480b4130 con 0x7ff924071a40 2024-04-25T10:22:20.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff944c00640 1 -- 172.21.15.165:0/2293994582 <== osd.7 v2:172.21.15.172:6816/3337160074 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+97 (crc 0 0 0) 0x7ff9480b4130 con 0x7ff924071a40 2024-04-25T10:22:20.507 INFO:teuthology.orchestra.run.smithi165.stdout:{ 2024-04-25T10:22:20.507 INFO:teuthology.orchestra.run.smithi165.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-25T10:22:20.507 INFO:teuthology.orchestra.run.smithi165.stdout:} 2024-04-25T10:22:20.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7ff924071a40 msgr2=0x7ff924073e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.172:6816/3337160074,v1:172.21.15.172:6818/3337160074] conn(0x7ff924071a40 0x7ff924073e80 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92406b500 msgr2=0x7ff92406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7ff92406b500 0x7ff92406d9c0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7ff930034000 tx=0x7ff93002e000 comp rx=0 tx=0).stop 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9481446d0 msgr2=0x7ff94813c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 --2- 172.21.15.165:0/2293994582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7ff9481446d0 0x7ff94813c100 secure :-1 s=READY pgs=2943 cs=0 l=1 rev1=1 crypto rx=0x7ff93800b3f0 tx=0x7ff93800b8b0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94d200640 1 -- 172.21.15.165:0/2293994582 reap_dead start 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 shutdown_connections 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.505+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 >> 172.21.15.165:0/2293994582 conn(0x7ff94806abe0 msgr2=0x7ff9480af3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.506+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 shutdown_connections 2024-04-25T10:22:20.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.506+0000 7ff94f200640 1 -- 172.21.15.165:0/2293994582 wait complete. 2024-04-25T10:22:20.527 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-25T10:22:20.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.719+0000 7f3991c00640 1 Processor -- start 2024-04-25T10:22:20.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.720+0000 7f3991c00640 1 -- start start 2024-04-25T10:22:20.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f3991c00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f398c06bff0 0x7f398c0b5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f3991c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f398c0b5970 0x7f398c0d2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f3991c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c15e110 con 0x7f398c15dd30 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f3991c00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c06d000 con 0x7f398c06bff0 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f3991c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0b2fd0 con 0x7f398c0b5970 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f398b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f398c06bff0 0x7f398c0b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.721+0000 7f398b400640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f398c06bff0 0x7f398c0b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51732/0 (socket says 172.21.15.165:51732) 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f398b400640 1 -- 172.21.15.165:0/2790812040 learned_addr learned my addr 172.21.15.165:0/2790812040 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f3991200640 1 --2- 172.21.15.165:0/2790812040 >> v2:172.21.15.172:3300/0 conn(0x7f398c0b5970 0x7f398c0d2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f398b400640 1 -- 172.21.15.165:0/2790812040 >> v2:172.21.15.172:3300/0 conn(0x7f398c0b5970 msgr2=0x7f398c0d2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f398b400640 1 --2- 172.21.15.165:0/2790812040 >> v2:172.21.15.172:3300/0 conn(0x7f398c0b5970 0x7f398c0d2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f398b400640 1 -- 172.21.15.165:0/2790812040 >> v1:172.21.15.172:6789/0 conn(0x7f398c15dd30 legacy=0x7f398c0cebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f398b400640 1 -- 172.21.15.165:0/2790812040 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c06c480 con 0x7f398c06bff0 2024-04-25T10:22:20.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.722+0000 7f3991200640 1 --2- 172.21.15.165:0/2790812040 >> v2:172.21.15.172:3300/0 conn(0x7f398c0b5970 0x7f398c0d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.723+0000 7f398b400640 1 --2- 172.21.15.165:0/2790812040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c06bff0 0x7f398c0b5430 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f398c161760 tx=0x7f397402a6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5986c2c1d6a101c server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.723+0000 7f398aa00640 1 -- 172.21.15.165:0/2790812040 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974036070 con 0x7f398c06bff0 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.723+0000 7f398aa00640 1 -- 172.21.15.165:0/2790812040 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3974031600 con 0x7f398c06bff0 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.723+0000 7f398aa00640 1 -- 172.21.15.165:0/2790812040 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39740319d0 con 0x7f398c06bff0 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 -- 172.21.15.165:0/2790812040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c06bff0 msgr2=0x7f398c0b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 --2- 172.21.15.165:0/2790812040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c06bff0 0x7f398c0b5430 secure :-1 s=READY pgs=2944 cs=0 l=1 rev1=1 crypto rx=0x7f398c161760 tx=0x7f397402a6f0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 -- 172.21.15.165:0/2790812040 shutdown_connections 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 --2- 172.21.15.165:0/2790812040 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c06bff0 0x7f398c0b5430 unknown :-1 s=CLOSED pgs=2944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 --2- 172.21.15.165:0/2790812040 >> v2:172.21.15.172:3300/0 conn(0x7f398c0b5970 0x7f398c0d2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 -- 172.21.15.165:0/2790812040 >> 172.21.15.165:0/2790812040 conn(0x7f398c0d0610 msgr2=0x7f398c0d0a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 -- 172.21.15.165:0/2790812040 shutdown_connections 2024-04-25T10:22:20.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.724+0000 7f3991c00640 1 -- 172.21.15.165:0/2790812040 wait complete. 2024-04-25T10:22:20.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.726+0000 7f3991c00640 1 Processor -- start 2024-04-25T10:22:20.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.726+0000 7f3991c00640 1 -- start start 2024-04-25T10:22:20.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f3991c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f398c0b5970 0x7f398c1f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f3991c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 0x7f398c0cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f3991c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f398c0b1c80 0x7f398c160800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f3991c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c06cd20 con 0x7f398c15dd30 2024-04-25T10:22:20.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f3991c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c06ca40 con 0x7f398c0b5970 2024-04-25T10:22:20.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f3991c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0d26a0 con 0x7f398c0b1c80 2024-04-25T10:22:20.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 0x7f398c0cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398b400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 0x7f398c0cd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51748/0 (socket says 172.21.15.165:51748) 2024-04-25T10:22:20.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398b400640 1 -- 172.21.15.165:0/244651523 learned_addr learned my addr 172.21.15.165:0/244651523 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398b400640 1 -- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f398c0b1c80 msgr2=0x7f398c160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398b400640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f398c0b1c80 0x7f398c160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398b400640 1 -- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f398c0b5970 msgr2=0x7f398c1f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.727+0000 7f398be00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f398c0b5970 0x7f398c1f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f398b400640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f398c0b5970 0x7f398c1f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f398b400640 1 -- 172.21.15.165:0/244651523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0aa3e0 con 0x7f398c15dd30 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f398be00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f398c0b5970 0x7f398c1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:20.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f398b400640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 0x7f398c0cd8c0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f3974000f80 tx=0x7f3974004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974036050 con 0x7f398c15dd30 2024-04-25T10:22:20.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3974042070 con 0x7f398c15dd30 2024-04-25T10:22:20.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3974009510 con 0x7f398c15dd30 2024-04-25T10:22:20.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f398c0aa6c0 con 0x7f398c15dd30 2024-04-25T10:22:20.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.728+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c0aac80 con 0x7f398c15dd30 2024-04-25T10:22:20.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.729+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0b9120 con 0x7f398c15dd30 2024-04-25T10:22:20.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.730+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f3974007c70 con 0x7f398c15dd30 2024-04-25T10:22:20.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.731+0000 7f3983e00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f396806b500 0x7f396806d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:20.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.731+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f39740ab070 con 0x7f398c15dd30 2024-04-25T10:22:20.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.732+0000 7f398be00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f396806b500 0x7f396806d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:20.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.732+0000 7f398be00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f396806b500 0x7f396806d9c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f397c00a810 tx=0x7f397c008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:20.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.736+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f398c0b9120 con 0x7f398c15dd30 2024-04-25T10:22:20.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.933+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f398c0cde00 con 0x7f396806b500 2024-04-25T10:22:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.934+0000 7f3983e00640 1 -- 172.21.15.165:0/244651523 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f397c00ad90 con 0x7f396806b500 2024-04-25T10:22:20.936 INFO:teuthology.orchestra.run.smithi165.stderr:instructed osd(s) 7 to deep-scrub 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.936+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f396806b500 msgr2=0x7f396806d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.936+0000 7f3991c00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f396806b500 0x7f396806d9c0 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f397c00a810 tx=0x7f397c008040 comp rx=0 tx=0).stop 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.936+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 msgr2=0x7f398c0cd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.936+0000 7f3991c00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 0x7f398c0cd8c0 secure :-1 s=READY pgs=2945 cs=0 l=1 rev1=1 crypto rx=0x7f3974000f80 tx=0x7f3974004a40 comp rx=0 tx=0).stop 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 shutdown_connections 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f396806b500 0x7f396806d9c0 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f398c0b1c80 0x7f398c160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f398c15dd30 0x7f398c0cd8c0 unknown :-1 s=CLOSED pgs=2945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 --2- 172.21.15.165:0/244651523 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f398c0b5970 0x7f398c1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 >> 172.21.15.165:0/244651523 conn(0x7f398c0d0610 msgr2=0x7f398c0b6c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 shutdown_connections 2024-04-25T10:22:20.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:20.937+0000 7f3991c00640 1 -- 172.21.15.165:0/244651523 wait complete. 2024-04-25T10:22:20.958 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:21.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.151+0000 7fd875c00640 1 Processor -- start 2024-04-25T10:22:21.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.151+0000 7fd875c00640 1 -- start start 2024-04-25T10:22:21.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd875c00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8700ce920 0x7fd8700cedb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd875c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd87006d290 0x7fd87006d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd875c00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd870144b30 con 0x7fd870144750 2024-04-25T10:22:21.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd875c00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8700712d0 con 0x7fd8700ce920 2024-04-25T10:22:21.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd875c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8700715b0 con 0x7fd87006d290 2024-04-25T10:22:21.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd86e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8700ce920 0x7fd8700cedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.153+0000 7fd86f400640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd87006d290 0x7fd87006d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.154+0000 7fd86e000640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fd8700ce920 0x7fd8700cedb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.172:3300/0 says I am v2:172.21.15.165:48970/0 (socket says 172.21.15.165:48970) 2024-04-25T10:22:21.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.154+0000 7fd86e000640 1 -- 172.21.15.165:0/2094625268 learned_addr learned my addr 172.21.15.165:0/2094625268 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:21.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.154+0000 7fd86e000640 1 -- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd87006d290 msgr2=0x7fd87006d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:21.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.154+0000 7fd86e000640 1 --2- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd87006d290 0x7fd87006d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.154+0000 7fd86e000640 1 -- 172.21.15.165:0/2094625268 >> v1:172.21.15.172:6789/0 conn(0x7fd870144750 legacy=0x7fd87006e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.154+0000 7fd86e000640 1 -- 172.21.15.165:0/2094625268 --> v2:172.21.15.172:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd870071890 con 0x7fd8700ce920 2024-04-25T10:22:21.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.155+0000 7fd86e000640 1 --2- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8700ce920 0x7fd8700cedb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd858009870 tx=0x7fd85802a270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90d201e9d3b1abbc server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.156+0000 7fd86d600640 1 -- 172.21.15.165:0/2094625268 <== mon.1 v2:172.21.15.172:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858037070 con 0x7fd8700ce920 2024-04-25T10:22:21.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.156+0000 7fd86d600640 1 -- 172.21.15.165:0/2094625268 <== mon.1 v2:172.21.15.172:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858032440 con 0x7fd8700ce920 2024-04-25T10:22:21.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.156+0000 7fd86d600640 1 -- 172.21.15.165:0/2094625268 <== mon.1 v2:172.21.15.172:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858032810 con 0x7fd8700ce920 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 -- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8700ce920 msgr2=0x7fd8700cedb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 --2- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8700ce920 0x7fd8700cedb0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fd858009870 tx=0x7fd85802a270 comp rx=0 tx=0).stop 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 -- 172.21.15.165:0/2094625268 shutdown_connections 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 --2- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd87006d290 0x7fd87006d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 --2- 172.21.15.165:0/2094625268 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd8700ce920 0x7fd8700cedb0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 -- 172.21.15.165:0/2094625268 >> 172.21.15.165:0/2094625268 conn(0x7fd87006abe0 msgr2=0x7fd87006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 -- 172.21.15.165:0/2094625268 shutdown_connections 2024-04-25T10:22:21.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.157+0000 7fd875c00640 1 -- 172.21.15.165:0/2094625268 wait complete. 2024-04-25T10:22:21.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.159+0000 7fd875c00640 1 Processor -- start 2024-04-25T10:22:21.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.159+0000 7fd875c00640 1 -- start start 2024-04-25T10:22:21.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd875c00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd87006d290 0x7fd8701f43d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd875c00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 0x7fd8700b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd875c00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd870144750 0x7fd8700b3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd875c00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87006f020 con 0x7fd8700ce920 2024-04-25T10:22:21.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd875c00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87006ece0 con 0x7fd87006d290 2024-04-25T10:22:21.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd875c00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd870145450 con 0x7fd870144750 2024-04-25T10:22:21.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 0x7fd8700b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86e000640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 0x7fd8700b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:51760/0 (socket says 172.21.15.165:51760) 2024-04-25T10:22:21.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86e000640 1 -- 172.21.15.165:0/976194582 learned_addr learned my addr 172.21.15.165:0/976194582 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86e000640 1 -- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd870144750 msgr2=0x7fd8700b3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86e000640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd870144750 0x7fd8700b3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86e000640 1 -- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd87006d290 msgr2=0x7fd8701f43d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.161+0000 7fd86ea00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd87006d290 0x7fd8701f43d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd86e000640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd87006d290 0x7fd8701f43d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd86e000640 1 -- 172.21.15.165:0/976194582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8701f4910 con 0x7fd8700ce920 2024-04-25T10:22:21.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd86ea00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd87006d290 0x7fd8701f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd86e000640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 0x7fd8700b2f40 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fd858000c00 tx=0x7fd8580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858037040 con 0x7fd8700ce920 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858042070 con 0x7fd8700ce920 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858030880 con 0x7fd8700ce920 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8700ad530 con 0x7fd8700ce920 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.162+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8700adaf0 con 0x7fd8700ce920 2024-04-25T10:22:21.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.164+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd870073990 con 0x7fd8700ce920 2024-04-25T10:22:21.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.165+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fd858030a20 con 0x7fd8700ce920 2024-04-25T10:22:21.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.165+0000 7fd866a00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd84c06b4d0 0x7fd84c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:21.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.166+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fd8580aaa20 con 0x7fd8700ce920 2024-04-25T10:22:21.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.166+0000 7fd86ea00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd84c06b4d0 0x7fd84c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:21.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.167+0000 7fd86ea00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd84c06b4d0 0x7fd84c06d990 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd860001d70 tx=0x7fd860009040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:21.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.170+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fd8580798a0 con 0x7fd8700ce920 2024-04-25T10:22:21.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.360+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd87006bc00 con 0x7fd84c06b4d0 2024-04-25T10:22:21.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.361+0000 7fd866a00640 1 -- 172.21.15.165:0/976194582 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13377 (secure 0 0 0) 0x7fd87006bc00 con 0x7fd84c06b4d0 2024-04-25T10:22:21.364 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:21.364 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd84c06b4d0 msgr2=0x7fd84c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd84c06b4d0 0x7fd84c06d990 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7fd860001d70 tx=0x7fd860009040 comp rx=0 tx=0).stop 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 msgr2=0x7fd8700b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 0x7fd8700b2f40 secure :-1 s=READY pgs=2946 cs=0 l=1 rev1=1 crypto rx=0x7fd858000c00 tx=0x7fd8580046c0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 shutdown_connections 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fd84c06b4d0 0x7fd84c06d990 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fd870144750 0x7fd8700b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fd8700ce920 0x7fd8700b2f40 unknown :-1 s=CLOSED pgs=2946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 --2- 172.21.15.165:0/976194582 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fd87006d290 0x7fd8701f43d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 >> 172.21.15.165:0/976194582 conn(0x7fd87006abe0 msgr2=0x7fd87013cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 shutdown_connections 2024-04-25T10:22:21.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:21.364+0000 7fd875c00640 1 -- 172.21.15.165:0/976194582 wait complete. 2024-04-25T10:22:21.384 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":808,"stamp":"2024-04-25T10:22:20.546371+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":546556,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":525363,"kb_avail":754428164,"statfs":{"total":773094113280,"available":772534439936,"internally_reserved":0,"allocated":21569536,"data_stored":10778368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":537972274},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.098978"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":514,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:22:10.780373+0000","last_change":"2024-04-25T10:01:53.689062+0000","last_active":"2024-04-25T10:22:10.780373+0000","last_peered":"2024-04-25T10:22:10.780373+0000","last_clean":"2024-04-25T10:22:10.780373+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:22:10.780373+0000","last_undegraded":"2024-04-25T10:22:10.780373+0000","last_fullsized":"2024-04-25T10:22:10.780373+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-25T10:01:52.538527+0000","last_clean_scrub_stamp":"2024-04-25T10:01:52.538527+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T15:13:51.916119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575097,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64780,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307060,"statfs":{"total":96636764160,"available":96570429440,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575097,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60040,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575097,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55880,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575097,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87748,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284092,"statfs":{"total":96636764160,"available":96546910208,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575097,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87752,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284088,"statfs":{"total":96636764160,"available":96546906112,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575096,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64200,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307640,"statfs":{"total":96636764160,"available":96571023360,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575097,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60036,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575097,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66120,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305720,"statfs":{"total":96636764160,"available":96569057280,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:21.388 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-25T10:01:52.538527+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=1, tm_sec=52, tm_wday=3, tm_yday=116, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=25, tm_hour=10, tm_min=22, tm_sec=14, tm_wday=3, tm_yday=116, tm_isdst=0) 2024-04-25T10:22:21.388 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-25T10:22:32.228 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:32.236 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:41.390 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-25T10:22:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.599+0000 7f9a84a00640 1 Processor -- start 2024-04-25T10:22:41.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.599+0000 7f9a84a00640 1 -- start start 2024-04-25T10:22:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.600+0000 7f9a84a00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7f9a80144440 0x7f9a80070ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.600+0000 7f9a84a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a80071530 0x7f9a800cd320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.600+0000 7f9a84a00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9a800cd860 0x7f9a8006d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.600+0000 7f9a84a00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a8015c860 con 0x7f9a800cd860 2024-04-25T10:22:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.600+0000 7f9a84a00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a80071970 con 0x7f9a80144440 2024-04-25T10:22:41.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.600+0000 7f9a84a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8015bf30 con 0x7f9a80071530 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9a800cd860 0x7f9a8006d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f9a800cd860 0x7f9a8006d050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45020/0 (socket says 172.21.15.165:45020) 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 -- 172.21.15.165:0/2411553135 learned_addr learned my addr 172.21.15.165:0/2411553135 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a76a00640 1 --2- 172.21.15.165:0/2411553135 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a80071530 0x7f9a800cd320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7f400640 1 --2- 172.21.15.165:0/2411553135 >> v2:172.21.15.172:3300/0 conn(0x7f9a80144440 0x7f9a80070ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 -- 172.21.15.165:0/2411553135 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a80071530 msgr2=0x7f9a800cd320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 --2- 172.21.15.165:0/2411553135 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a80071530 0x7f9a800cd320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 -- 172.21.15.165:0/2411553135 >> v2:172.21.15.172:3300/0 conn(0x7f9a80144440 msgr2=0x7f9a80070ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:41.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 --2- 172.21.15.165:0/2411553135 >> v2:172.21.15.172:3300/0 conn(0x7f9a80144440 0x7f9a80070ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.601+0000 7f9a7fe00640 1 -- 172.21.15.165:0/2411553135 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a8015c210 con 0x7f9a800cd860 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.603+0000 7f9a7fe00640 1 --2- 172.21.15.165:0/2411553135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a800cd860 0x7f9a8006d050 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f9a8006f470 tx=0x7f9a7802a730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31a34420e50f4ac1 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.603+0000 7f9a7ea00640 1 -- 172.21.15.165:0/2411553135 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78036070 con 0x7f9a800cd860 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.603+0000 7f9a7ea00640 1 -- 172.21.15.165:0/2411553135 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78031600 con 0x7f9a800cd860 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.603+0000 7f9a7ea00640 1 -- 172.21.15.165:0/2411553135 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780319d0 con 0x7f9a800cd860 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.603+0000 7f9a84a00640 1 -- 172.21.15.165:0/2411553135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a800cd860 msgr2=0x7f9a8006d050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.603+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2411553135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a800cd860 0x7f9a8006d050 secure :-1 s=READY pgs=2947 cs=0 l=1 rev1=1 crypto rx=0x7f9a8006f470 tx=0x7f9a7802a730 comp rx=0 tx=0).stop 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 -- 172.21.15.165:0/2411553135 shutdown_connections 2024-04-25T10:22:41.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2411553135 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a800cd860 0x7f9a8006d050 unknown :-1 s=CLOSED pgs=2947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2411553135 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a80071530 0x7f9a800cd320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2411553135 >> v2:172.21.15.172:3300/0 conn(0x7f9a80144440 0x7f9a80070ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 -- 172.21.15.165:0/2411553135 >> 172.21.15.165:0/2411553135 conn(0x7f9a8006abe0 msgr2=0x7f9a8006afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 -- 172.21.15.165:0/2411553135 shutdown_connections 2024-04-25T10:22:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.604+0000 7f9a84a00640 1 -- 172.21.15.165:0/2411553135 wait complete. 2024-04-25T10:22:41.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.605+0000 7f9a84a00640 1 Processor -- start 2024-04-25T10:22:41.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.605+0000 7f9a84a00640 1 -- start start 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.606+0000 7f9a84a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 0x7f9a801f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.606+0000 7f9a84a00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a800cd860 0x7f9a8013d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.606+0000 7f9a84a00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a80144440 0x7f9a8013a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.606+0000 7f9a84a00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800cdcd0 con 0x7f9a80071530 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.606+0000 7f9a84a00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8015bc30 con 0x7f9a80144440 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.606+0000 7f9a84a00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a800cde50 con 0x7f9a800cd860 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 0x7f9a801f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 0x7f9a801f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45026/0 (socket says 172.21.15.165:45026) 2024-04-25T10:22:41.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 -- 172.21.15.165:0/2637190726 learned_addr learned my addr 172.21.15.165:0/2637190726 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 -- 172.21.15.165:0/2637190726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a800cd860 msgr2=0x7f9a8013d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a800cd860 0x7f9a8013d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 -- 172.21.15.165:0/2637190726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a80144440 msgr2=0x7f9a8013a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a80144440 0x7f9a8013a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 -- 172.21.15.165:0/2637190726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a800c7030 con 0x7f9a80071530 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.607+0000 7f9a7f400640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 0x7f9a801f51c0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000d5f0 tx=0x7f9a7000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.608+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a700125f0 con 0x7f9a80071530 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.608+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a70019070 con 0x7f9a80071530 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.608+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a800c7310 con 0x7f9a80071530 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.608+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a700129d0 con 0x7f9a80071530 2024-04-25T10:22:41.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.608+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a800c78d0 con 0x7f9a80071530 2024-04-25T10:22:41.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.609+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a80073990 con 0x7f9a80071530 2024-04-25T10:22:41.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.610+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f9a70012b70 con 0x7f9a80071530 2024-04-25T10:22:41.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.611+0000 7f9a77e00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:41.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.611+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f9a7008a590 con 0x7f9a80071530 2024-04-25T10:22:41.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.611+0000 7f9a76a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:41.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.612+0000 7f9a76a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9a800cbf90 tx=0x7f9a68008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:41.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.615+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f9a70059490 con 0x7f9a80071530 2024-04-25T10:22:41.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.809+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 --> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a80070fd0 con 0x7f9a5c06b4d0 2024-04-25T10:22:41.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.811+0000 7f9a77e00640 1 -- 172.21.15.165:0/2637190726 <== mgr.4107 v2:172.21.15.165:6832/3683600865 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+13387 (secure 0 0 0) 0x7f9a80070fd0 con 0x7f9a5c06b4d0 2024-04-25T10:22:41.814 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:41.814 INFO:teuthology.orchestra.run.smithi165.stderr:dumped all 2024-04-25T10:22:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5c06b4d0 msgr2=0x7f9a5c06d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f9a800cbf90 tx=0x7f9a68008040 comp rx=0 tx=0).stop 2024-04-25T10:22:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 msgr2=0x7f9a801f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 0x7f9a801f51c0 secure :-1 s=READY pgs=2948 cs=0 l=1 rev1=1 crypto rx=0x7f9a7000d5f0 tx=0x7f9a7000dab0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 shutdown_connections 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f9a5c06b4d0 0x7f9a5c06d990 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f9a80144440 0x7f9a8013a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f9a800cd860 0x7f9a8013d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 --2- 172.21.15.165:0/2637190726 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f9a80071530 0x7f9a801f51c0 unknown :-1 s=CLOSED pgs=2948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 >> 172.21.15.165:0/2637190726 conn(0x7f9a8006abe0 msgr2=0x7f9a800c6160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 shutdown_connections 2024-04-25T10:22:41.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:41.814+0000 7f9a84a00640 1 -- 172.21.15.165:0/2637190726 wait complete. 2024-04-25T10:22:41.832 INFO:teuthology.orchestra.run.smithi165.stdout:{"pg_ready":true,"pg_map":{"version":818,"stamp":"2024-04-25T10:22:40.552012+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":546544,"kb_used_data":21064,"kb_used_omap":12,"kb_used_meta":525363,"kb_avail":754428176,"statfs":{"total":773094113280,"available":772534452224,"internally_reserved":0,"allocated":21569536,"data_stored":10778368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12750,"internal_metadata":537972274},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003502"},"pg_stats":[{"pgid":"1.0","version":"201'119","reported_seq":525,"reported_epoch":201,"state":"active+clean","last_fresh":"2024-04-25T10:22:21.539077+0000","last_change":"2024-04-25T10:22:21.539158+0000","last_active":"2024-04-25T10:22:21.539077+0000","last_peered":"2024-04-25T10:22:21.539077+0000","last_clean":"2024-04-25T10:22:21.539077+0000","last_became_active":"2024-04-25T10:01:53.686815+0000","last_became_peered":"2024-04-25T10:01:53.686815+0000","last_unstale":"2024-04-25T10:22:21.539077+0000","last_undegraded":"2024-04-25T10:22:21.539077+0000","last_fullsized":"2024-04-25T10:22:21.539077+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"201'119","last_scrub_stamp":"2024-04-25T10:22:21.538906+0000","last_deep_scrub":"201'119","last_deep_scrub_stamp":"2024-04-25T10:22:21.538906+0000","last_clean_scrub_stamp":"2024-04-25T10:22:21.538906+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T13:27:21.031815+0000","scrub_duration":20,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575101,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":64772,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307068,"statfs":{"total":96636764160,"available":96570437632,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":63175115},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834575101,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60040,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311800,"statfs":{"total":96636764160,"available":96575283200,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58915270},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834575101,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":55880,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":53374,"kb_avail":94315960,"statfs":{"total":96636764160,"available":96579543040,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":54655430},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834575101,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87748,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284092,"statfs":{"total":96636764160,"available":96546910208,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":87292357},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":13,"seq":55834575101,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":87752,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":85246,"kb_avail":94284088,"statfs":{"total":96636764160,"available":96546906112,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":87292359},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834575100,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":64200,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":61694,"kb_avail":94307640,"statfs":{"total":96636764160,"available":96571023360,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":63175109},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834575101,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60036,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":57534,"kb_avail":94311804,"statfs":{"total":96636764160,"available":96575287296,"internally_reserved":0,"allocated":2547712,"data_stored":1199704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":58915269},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":13,"seq":55834575101,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":66116,"kb_used_data":3068,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94305724,"statfs":{"total":96636764160,"available":96569061376,"internally_reserved":0,"allocated":3141632,"data_stored":1790072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":64551365},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-25T10:22:41.833 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-25T10:22:42.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.018+0000 7fe03ca00640 1 Processor -- start 2024-04-25T10:22:42.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.018+0000 7fe03ca00640 1 -- start start 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe03ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381446d0 0x7fe0380b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe03ca00640 1 --2- >> v2:172.21.15.172:3300/0 conn(0x7fe038144d50 0x7fe0380b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe03ca00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0381452a0 0x7fe0380cf640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe03ca00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0380b2b20 con 0x7fe0381452a0 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe03ca00640 1 -- --> v2:172.21.15.172:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03806d400 con 0x7fe038144d50 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe03ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380b49a0 con 0x7fe0381446d0 2024-04-25T10:22:42.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe036800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0381452a0 0x7fe0380cf640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe036800640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7fe0381452a0 0x7fe0380cf640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45028/0 (socket says 172.21.15.165:45028) 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.020+0000 7fe036800640 1 -- 172.21.15.165:0/3168959950 learned_addr learned my addr 172.21.15.165:0/3168959950 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe02fe00640 1 --2- 172.21.15.165:0/3168959950 >> v2:172.21.15.172:3300/0 conn(0x7fe038144d50 0x7fe0380b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe035e00640 1 --2- 172.21.15.165:0/3168959950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381446d0 0x7fe0380b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe036800640 1 -- 172.21.15.165:0/3168959950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381446d0 msgr2=0x7fe0380b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe036800640 1 --2- 172.21.15.165:0/3168959950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381446d0 0x7fe0380b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe036800640 1 -- 172.21.15.165:0/3168959950 >> v2:172.21.15.172:3300/0 conn(0x7fe038144d50 msgr2=0x7fe0380b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe036800640 1 --2- 172.21.15.165:0/3168959950 >> v2:172.21.15.172:3300/0 conn(0x7fe038144d50 0x7fe0380b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.021+0000 7fe036800640 1 -- 172.21.15.165:0/3168959950 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0380cccc0 con 0x7fe0381452a0 2024-04-25T10:22:42.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.022+0000 7fe036800640 1 --2- 172.21.15.165:0/3168959950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0381452a0 0x7fe0380cf640 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7fe0380d07a0 tx=0x7fe03002a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c4f6deb94364b62 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.022+0000 7fe035400640 1 -- 172.21.15.165:0/3168959950 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe030036070 con 0x7fe0381452a0 2024-04-25T10:22:42.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.022+0000 7fe035400640 1 -- 172.21.15.165:0/3168959950 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe030031600 con 0x7fe0381452a0 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.022+0000 7fe035400640 1 -- 172.21.15.165:0/3168959950 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0300319d0 con 0x7fe0381452a0 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 -- 172.21.15.165:0/3168959950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0381452a0 msgr2=0x7fe0380cf640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 --2- 172.21.15.165:0/3168959950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0381452a0 0x7fe0380cf640 secure :-1 s=READY pgs=2949 cs=0 l=1 rev1=1 crypto rx=0x7fe0380d07a0 tx=0x7fe03002a6a0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 -- 172.21.15.165:0/3168959950 shutdown_connections 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 --2- 172.21.15.165:0/3168959950 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe0381452a0 0x7fe0380cf640 unknown :-1 s=CLOSED pgs=2949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 --2- 172.21.15.165:0/3168959950 >> v2:172.21.15.172:3300/0 conn(0x7fe038144d50 0x7fe0380b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 --2- 172.21.15.165:0/3168959950 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381446d0 0x7fe0380b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 -- 172.21.15.165:0/3168959950 >> 172.21.15.165:0/3168959950 conn(0x7fe03806abe0 msgr2=0x7fe03806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 -- 172.21.15.165:0/3168959950 shutdown_connections 2024-04-25T10:22:42.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.023+0000 7fe03ca00640 1 -- 172.21.15.165:0/3168959950 wait complete. 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.025+0000 7fe03ca00640 1 Processor -- start 2024-04-25T10:22:42.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.025+0000 7fe03ca00640 1 -- start start 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe03ca00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0381446d0 0x7fe03813e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe03ca00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 0x7fe03815b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe03ca00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381452a0 0x7fe03806d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe03ca00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380cc700 con 0x7fe038144d50 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe03ca00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380cc420 con 0x7fe0381446d0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe03ca00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380cc9e0 con 0x7fe0381452a0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe02fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 0x7fe03815b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe02fe00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 0x7fe03815b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45038/0 (socket says 172.21.15.165:45038) 2024-04-25T10:22:42.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe02fe00640 1 -- 172.21.15.165:0/1570342788 learned_addr learned my addr 172.21.15.165:0/1570342788 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.026+0000 7fe035e00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0381446d0 0x7fe03813e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe036800640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381452a0 0x7fe03806d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02fe00640 1 -- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381452a0 msgr2=0x7fe03806d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02fe00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381452a0 0x7fe03806d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02fe00640 1 -- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0381446d0 msgr2=0x7fe03813e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02fe00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0381446d0 0x7fe03813e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02fe00640 1 -- 172.21.15.165:0/1570342788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0380c5680 con 0x7fe038144d50 2024-04-25T10:22:42.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02fe00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 0x7fe03815b230 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7fe02000d830 tx=0x7fe02000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000b6c0 con 0x7fe038144d50 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe020019070 con 0x7fe038144d50 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02000ba80 con 0x7fe038144d50 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.027+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0380c5960 con 0x7fe038144d50 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.028+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0380cd2e0 con 0x7fe038144d50 2024-04-25T10:22:42.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.029+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe038073a00 con 0x7fe038144d50 2024-04-25T10:22:42.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.030+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7fe02001e460 con 0x7fe038144d50 2024-04-25T10:22:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.030+0000 7fe02e000640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe01406b500 0x7fe01406d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.031+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7fe02008a800 con 0x7fe038144d50 2024-04-25T10:22:42.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.031+0000 7fe035e00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe01406b500 0x7fe01406d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.031+0000 7fe035e00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe01406b500 0x7fe01406d9c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe028006fd0 tx=0x7fe028008040 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.035+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7fe020059740 con 0x7fe038144d50 2024-04-25T10:22:42.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.217+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fe0380aeb70 con 0x7fe038144d50 2024-04-25T10:22:42.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.224+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v38)=0 v38) v1 ==== 123+0+0 (secure 0 0 0) 0x7fe020059040 con 0x7fe038144d50 2024-04-25T10:22:42.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.226+0000 7fe02e000640 1 -- 172.21.15.165:0/1570342788 <== mon.0 v2:172.21.15.165:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fe02000be30 con 0x7fe038144d50 2024-04-25T10:22:42.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.228+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe01406b500 msgr2=0x7fe01406d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.228+0000 7fe03ca00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe01406b500 0x7fe01406d9c0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fe028006fd0 tx=0x7fe028008040 comp rx=0 tx=0).stop 2024-04-25T10:22:42.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.228+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 msgr2=0x7fe03815b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.228+0000 7fe03ca00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 0x7fe03815b230 secure :-1 s=READY pgs=2950 cs=0 l=1 rev1=1 crypto rx=0x7fe02000d830 tx=0x7fe02000dcf0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 shutdown_connections 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7fe01406b500 0x7fe01406d9c0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7fe0381452a0 0x7fe03806d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7fe038144d50 0x7fe03815b230 unknown :-1 s=CLOSED pgs=2950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 --2- 172.21.15.165:0/1570342788 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7fe0381446d0 0x7fe03813e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 >> 172.21.15.165:0/1570342788 conn(0x7fe03806abe0 msgr2=0x7fe0380b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 shutdown_connections 2024-04-25T10:22:42.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.229+0000 7fe03ca00640 1 -- 172.21.15.165:0/1570342788 wait complete. 2024-04-25T10:22:42.250 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-25T10:22:42.251 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-25T10:22:42.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.439+0000 7f6fc1e00640 1 Processor -- start 2024-04-25T10:22:42.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.439+0000 7f6fc1e00640 1 -- start start 2024-04-25T10:22:42.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fc1e00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6fbc0b93c0 0x7f6fbc0cffb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fc1e00640 1 -- --> v1:172.21.15.165:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc161df0 con 0x7f6fbc0d04f0 2024-04-25T10:22:42.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fc1e00640 1 -- --> v1:172.21.15.172:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fbc1620d0 con 0x7f6fbc15db90 2024-04-25T10:22:42.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fc1e00640 1 -- --> v2:172.21.15.165:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fbc0d1f50 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fbbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6fbc0b93c0 0x7f6fbc0cffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fc1400640 1 --1- >> v1:172.21.15.165:6789/0 conn(0x7f6fbc0d04f0 0x7f6fbc0d23e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.165:6789/0 says I am v1:172.21.15.165:56866/0 (socket says 172.21.15.165:56866) 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fc1400640 1 -- 172.21.15.165:0/3902527242 learned_addr learned my addr 172.21.15.165:0/3902527242 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.441+0000 7f6fbbe00640 1 --2- >> v2:172.21.15.165:3300/0 conn(0x7f6fbc0b93c0 0x7f6fbc0cffb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45054/0 (socket says 172.21.15.165:45054) 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.442+0000 7f6fbbe00640 1 -- 172.21.15.165:0/3902527242 >> v1:172.21.15.172:6789/0 conn(0x7f6fbc15db90 legacy=0x7f6fbc1615a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.442+0000 7f6fbbe00640 1 -- 172.21.15.165:0/3902527242 >> v1:172.21.15.165:6789/0 conn(0x7f6fbc0d04f0 legacy=0x7f6fbc0d23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.442+0000 7f6fbbe00640 1 -- 172.21.15.165:0/3902527242 --> v2:172.21.15.165:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc1623b0 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.442+0000 7f6fbb400640 1 -- 172.21.15.165:0/3902527242 <== mon.0 v1:172.21.15.165:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1869298137 0 0) 0x7f6fbc161df0 con 0x7f6fbc0d04f0 2024-04-25T10:22:42.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.443+0000 7f6fbbe00640 1 --2- 172.21.15.165:0/3902527242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0cffb0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80097a0 tx=0x7f6fa802a1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71363256756e8ce0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.443+0000 7f6fbb400640 1 -- 172.21.15.165:0/3902527242 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8037070 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.443+0000 7f6fbb400640 1 -- 172.21.15.165:0/3902527242 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6fa8032440 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.443+0000 7f6fbb400640 1 -- 172.21.15.165:0/3902527242 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fa8032830 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 -- 172.21.15.165:0/3902527242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 msgr2=0x7f6fbc0cffb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/3902527242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0cffb0 secure :-1 s=READY pgs=2952 cs=0 l=1 rev1=1 crypto rx=0x7f6fa80097a0 tx=0x7f6fa802a1e0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 -- 172.21.15.165:0/3902527242 shutdown_connections 2024-04-25T10:22:42.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/3902527242 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0cffb0 unknown :-1 s=CLOSED pgs=2952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 -- 172.21.15.165:0/3902527242 >> 172.21.15.165:0/3902527242 conn(0x7f6fbc0b52b0 msgr2=0x7f6fbc0b56c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 -- 172.21.15.165:0/3902527242 shutdown_connections 2024-04-25T10:22:42.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.444+0000 7f6fc1e00640 1 -- 172.21.15.165:0/3902527242 wait complete. 2024-04-25T10:22:42.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.445+0000 7f6fc1e00640 1 Processor -- start 2024-04-25T10:22:42.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.445+0000 7f6fc1e00640 1 -- start start 2024-04-25T10:22:42.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.446+0000 7f6fc1e00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.446+0000 7f6fc1e00640 1 --2- >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fbc0d04f0 0x7f6fbc15df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.446+0000 7f6fc1e00640 1 --2- >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fbc15db90 0x7f6fbc15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.446+0000 7f6fc1e00640 1 -- --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b68d0 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.446+0000 7f6fc1e00640 1 -- --> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0d0940 con 0x7f6fbc15db90 2024-04-25T10:22:42.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.446+0000 7f6fc1e00640 1 -- --> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc0b6a50 con 0x7f6fbc0d04f0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 --2- >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.165:3300/0 says I am v2:172.21.15.165:45062/0 (socket says 172.21.15.165:45062) 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 -- 172.21.15.165:0/1716888352 learned_addr learned my addr 172.21.15.165:0/1716888352 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 -- 172.21.15.165:0/1716888352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fbc0d04f0 msgr2=0x7f6fbc15df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fbc0d04f0 0x7f6fbc15df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 -- 172.21.15.165:0/1716888352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fbc15db90 msgr2=0x7f6fbc15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fbc15db90 0x7f6fbc15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 -- 172.21.15.165:0/1716888352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc15fff0 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.447+0000 7f6fc0a00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0ccac0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b3f0 tx=0x7f6fb000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.448+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0015020 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.448+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f6fb0011830 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.448+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fb0011c40 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.448+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fbc1602d0 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.448+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fbc14ffb0 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.449+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fbc072e40 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.450+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 4 ==== mgrmap(e 5) v1 ==== 89780+0+0 (secure 0 0 0) 0x7f6fb0018020 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.451+0000 7f6fb8c00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-25T10:22:42.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.451+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5421+0+0 (secure 0 0 0) 0x7f6fb0089e90 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.451+0000 7f6fbbe00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-25T10:22:42.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.452+0000 7f6fbbe00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8034000 tx=0x7f6fa802c000 comp rx=0 tx=0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-25T10:22:42.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.455+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+182440 (secure 0 0 0) 0x7f6fbc072e40 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.683+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 --> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6fbc073990 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.685+0000 7f6fb8c00640 1 -- 172.21.15.165:0/1716888352 <== mon.0 v2:172.21.15.165:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 148 v148) v1 ==== 95+0+3717 (secure 0 0 0) 0x7f6fb008b020 con 0x7f6fbc0b93c0 2024-04-25T10:22:42.687 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-25T10:22:42.687 INFO:teuthology.orchestra.run.smithi165.stdout:{"epoch":148,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6446,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6835/1602661690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6833","nonce":1602661690},{"type":"v1","addr":"172.21.15.172:6835","nonce":1602661690}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":7264,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6836/3097180751","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6834","nonce":3097180751},{"type":"v1","addr":"172.21.15.165:6836","nonce":3097180751}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":133},{"gid":7270,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.172:6834/1469392598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.172:6832","nonce":1469392598},{"type":"v1","addr":"172.21.15.172:6834","nonce":1469392598}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":134},{"gid":12093,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.165:6837/3386020531","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6835","nonce":3386020531},{"type":"v1","addr":"172.21.15.165:6837","nonce":3386020531}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":148}],"filesystems":[]} 2024-04-25T10:22:42.688 INFO:teuthology.orchestra.run.smithi165.stderr:dumped fsmap epoch 148 2024-04-25T10:22:42.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6f9c06b500 msgr2=0x7f6f9c06d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f6fa8034000 tx=0x7f6fa802c000 comp rx=0 tx=0).stop 2024-04-25T10:22:42.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 msgr2=0x7f6fbc0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-25T10:22:42.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0ccac0 secure :-1 s=READY pgs=2953 cs=0 l=1 rev1=1 crypto rx=0x7f6fb000b3f0 tx=0x7f6fb000b8b0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 shutdown_connections 2024-04-25T10:22:42.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:6832/3683600865,v1:172.21.15.165:6833/3683600865] conn(0x7f6f9c06b500 0x7f6f9c06d9c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.172:3300/0,v1:172.21.15.172:6789/0] conn(0x7f6fbc15db90 0x7f6fbc15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.172:3301/0,v1:172.21.15.172:6790/0] conn(0x7f6fbc0d04f0 0x7f6fbc15df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 --2- 172.21.15.165:0/1716888352 >> [v2:172.21.15.165:3300/0,v1:172.21.15.165:6789/0] conn(0x7f6fbc0b93c0 0x7f6fbc0ccac0 unknown :-1 s=CLOSED pgs=2953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-25T10:22:42.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.687+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 >> 172.21.15.165:0/1716888352 conn(0x7f6fbc0b52b0 msgr2=0x7f6fbc0c9860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-25T10:22:42.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.688+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 shutdown_connections 2024-04-25T10:22:42.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-25T10:22:42.688+0000 7f6fc1e00640 1 -- 172.21.15.165:0/1716888352 wait complete. 2024-04-25T10:22:42.707 DEBUG:tasks.cephfs.filesystem:already dead... 2024-04-25T10:22:42.707 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-25T10:22:42.707 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-04-25T10:22:42.707 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:42.730 INFO:tasks.ceph.mds.a:Stopped 2024-04-25T10:22:42.730 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-04-25T10:22:42.730 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:42.794 INFO:tasks.ceph.mds.c:Stopped 2024-04-25T10:22:42.795 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-04-25T10:22:42.795 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:42.815 INFO:tasks.ceph.mds.b:Stopped 2024-04-25T10:22:42.815 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-04-25T10:22:42.815 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:42.881 INFO:tasks.ceph.mds.d:Stopped 2024-04-25T10:22:42.882 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-25T10:22:42.882 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-25T10:22:42.882 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:42.940 INFO:tasks.ceph.osd.0:Stopped 2024-04-25T10:22:42.941 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-25T10:22:42.941 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:43.017 INFO:tasks.ceph.osd.1:Stopped 2024-04-25T10:22:43.018 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-25T10:22:43.018 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:43.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2024-04-25T10:22:43.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-04-25T10:22:43.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2024-04-25T10:22:43.083 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2024-04-25T10:22:43.096 INFO:tasks.ceph.osd.2:Stopped 2024-04-25T10:22:43.096 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-25T10:22:43.096 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:48.991 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2024-04-25T10:22:48.992 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2024-04-25T10:22:48.992 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-04-25T10:22:48.992 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2024-04-25T10:22:48.992 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2024-04-25T10:22:49.198 INFO:tasks.ceph.osd.3:Stopped 2024-04-25T10:22:49.199 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-25T10:22:49.199 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.268 INFO:tasks.ceph.osd.4:Stopped 2024-04-25T10:22:49.269 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-25T10:22:49.269 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.325 INFO:tasks.ceph.osd.5:Stopped 2024-04-25T10:22:49.325 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-25T10:22:49.325 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.395 INFO:tasks.ceph.osd.6:Stopped 2024-04-25T10:22:49.395 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-25T10:22:49.395 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.492 INFO:tasks.ceph.osd.7:Stopped 2024-04-25T10:22:49.492 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-25T10:22:49.492 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-25T10:22:49.492 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.567 INFO:tasks.ceph.mgr.y:Stopped 2024-04-25T10:22:49.568 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-25T10:22:49.568 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.655 INFO:tasks.ceph.mgr.x:Stopped 2024-04-25T10:22:49.655 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-25T10:22:49.655 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-25T10:22:49.655 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.712 INFO:tasks.ceph.mon.a:Stopped 2024-04-25T10:22:49.712 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-25T10:22:49.712 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.745 INFO:tasks.ceph.mon.b:Stopped 2024-04-25T10:22:49.746 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-25T10:22:49.746 INFO:teuthology.orchestra.run:waiting for 300 2024-04-25T10:22:49.810 INFO:tasks.ceph.mon.c:Stopped 2024-04-25T10:22:49.810 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-25T10:22:49.810 DEBUG:teuthology.orchestra.run.smithi165:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'Replacing daemon mds' | egrep -v 'Scrub error on inode' | egrep -v 'Behind on trimming' | egrep -v 'Metadata damage detected' | egrep -v 'bad backtrace on inode' | egrep -v 'overall HEALTH_' | egrep -v '\(MDS_TRIM\)' | head -n 1 2024-04-25T10:22:49.913 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:22:49.913 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-25T10:22:50.189 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:22:50.189 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-25T10:22:50.351 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:22:50.351 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-25T10:22:50.525 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:22:50.526 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-25T10:22:50.694 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:22:50.694 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-25T10:22:50.954 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:22:50.954 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-25T10:22:51.095 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:22:51.095 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-25T10:22:51.242 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:22:51.243 DEBUG:teuthology.orchestra.run.smithi172:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-25T10:22:51.414 INFO:tasks.ceph:Archiving mon data... 2024-04-25T10:22:51.415 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/data/mon.a.tgz 2024-04-25T10:22:51.415 DEBUG:teuthology.orchestra.run.smithi165:> mktemp 2024-04-25T10:22:51.474 INFO:teuthology.orchestra.run.smithi165.stdout:/tmp/tmp.5E51LyODF4 2024-04-25T10:22:51.474 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.5E51LyODF4 2024-04-25T10:22:51.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0666 /tmp/tmp.5E51LyODF4 2024-04-25T10:22:51.826 DEBUG:teuthology.orchestra.remote:smithi165:/tmp/tmp.5E51LyODF4 is 487KB 2024-04-25T10:22:51.936 DEBUG:teuthology.orchestra.run.smithi165:> rm -fr /tmp/tmp.5E51LyODF4 2024-04-25T10:22:51.993 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/data/mon.b.tgz 2024-04-25T10:22:51.993 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-25T10:22:52.051 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.ABgPvGNIEF 2024-04-25T10:22:52.051 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.ABgPvGNIEF 2024-04-25T10:22:52.260 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.ABgPvGNIEF 2024-04-25T10:22:52.397 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.ABgPvGNIEF is 496KB 2024-04-25T10:22:52.504 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.ABgPvGNIEF 2024-04-25T10:22:52.560 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/data/mon.c.tgz 2024-04-25T10:22:52.561 DEBUG:teuthology.orchestra.run.smithi172:> mktemp 2024-04-25T10:22:52.614 INFO:teuthology.orchestra.run.smithi172.stdout:/tmp/tmp.bvhWAzD7zm 2024-04-25T10:22:52.614 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.bvhWAzD7zm 2024-04-25T10:22:52.800 DEBUG:teuthology.orchestra.run.smithi172:> sudo chmod 0666 /tmp/tmp.bvhWAzD7zm 2024-04-25T10:22:52.943 DEBUG:teuthology.orchestra.remote:smithi172:/tmp/tmp.bvhWAzD7zm is 496KB 2024-04-25T10:22:53.052 DEBUG:teuthology.orchestra.run.smithi172:> rm -fr /tmp/tmp.bvhWAzD7zm 2024-04-25T10:22:53.104 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-25T10:22:53.104 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:22:53.112 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-25T10:22:53.244 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:22:53.301 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-25T10:22:53.302 INFO:tasks.ceph:Archiving crash dumps... 2024-04-25T10:22:53.303 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/remote/smithi165/crash 2024-04-25T10:22:53.304 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:22:53.391 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/remote/smithi172/crash 2024-04-25T10:22:53.392 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-25T10:22:53.478 INFO:tasks.ceph:Shutting down logrotate 2024-04-25T10:22:53.479 DEBUG:teuthology.orchestra.run.smithi165:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:53.486 DEBUG:teuthology.orchestra.run.smithi172:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:53.620 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:53.705 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-04-25T10:22:53.794 INFO:tasks.ceph:Compressing logs... 2024-04-25T10:22:53.794 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:22:53.802 DEBUG:teuthology.orchestra.run.smithi172:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-25T10:22:53.875 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84721.log 2024-04-25T10:22:53.875 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79901.log 2024-04-25T10:22:53.877 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40817.log 2024-04-25T10:22:53.877 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84721.log.gz 2024-04-25T10:22:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79901.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79901.log.gz 2024-04-25T10:22:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42405.log 2024-04-25T10:22:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97885.log 2024-04-25T10:22:53.879 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.40817.log.gz 2024-04-25T10:22:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42405.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93105.log 2024-04-25T10:22:53.880 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42405.log.gz 2024-04-25T10:22:53.881 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104847.log 2024-04-25T10:22:53.881 INFO:teuthology.orchestra.run.smithi165.stderr: 90.5% -- replaced with /var/log/ceph/ceph-client.admin.97885.log.gz 2024-04-25T10:22:53.882 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39668.log 2024-04-25T10:22:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55515.log 2024-04-25T10:22:53.889 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.93105.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61219.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.104847.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44814.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.55515.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78739.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61219.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84847.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44814.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84553.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78739.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84847.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84847.log.gz 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89893.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92350.log 2024-04-25T10:22:53.890 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.84553.log.gz 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33199.log 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.39668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39668.log.gz 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33370.log 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33199.log.gz 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33159.log 2024-04-25T10:22:53.893 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33370.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.33370.log.gz 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33159.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.33159.log.gz 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33290.log 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33410.log 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33290.log: 92.5% 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33290.log.gz -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33410.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.33410.log.gz 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-04-25T10:22:53.894 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-25T10:22:53.895 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-25T10:22:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61645.log 2024-04-25T10:22:53.898 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89893.log.gz 2024-04-25T10:22:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72911.log 2024-04-25T10:22:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61645.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.92350.log.gz 2024-04-25T10:22:53.898 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42945.log 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61645.log.gz 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.72911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61051.log 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.72911.log.gz 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78781.log 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.42945.log.gz 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41205.log 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61051.log.gz 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59279.log 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78781.log.gz 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61817.log 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.41205.log.gz 2024-04-25T10:22:53.899 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82441.log 2024-04-25T10:22:53.902 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39262.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi172.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.39262.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39262.log.gz 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.39672.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61817.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59279.log.gz 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101784.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61817.log.gz 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82441.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44898.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101784.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.82441.log.gz 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41247.log 2024-04-25T10:22:53.903 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101784.log.gz 2024-04-25T10:22:53.904 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77415.log 2024-04-25T10:22:53.904 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44898.log.gz 2024-04-25T10:22:53.905 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.3.39672.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.39672.log.gz 2024-04-25T10:22:53.905 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.39281.log 2024-04-25T10:22:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108598.log 2024-04-25T10:22:53.906 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41247.log.gz 2024-04-25T10:22:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77415.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101826.log 2024-04-25T10:22:53.906 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77415.log.gz 2024-04-25T10:22:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109086.log 2024-04-25T10:22:53.906 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108598.log.gz 2024-04-25T10:22:53.908 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.1.39281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.39281.log.gz 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33330.log 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45042.log 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101826.log.gz 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77481.log 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.109086.log.gz 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45042.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73586.log 2024-04-25T10:22:53.909 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45042.log.gz 2024-04-25T10:22:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42119.log 2024-04-25T10:22:53.911 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.77481.log.gz 2024-04-25T10:22:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44940.log 2024-04-25T10:22:53.911 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33249.log 2024-04-25T10:22:53.912 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.73586.log.gz 2024-04-25T10:22:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100616.log 2024-04-25T10:22:53.912 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42119.log.gz 2024-04-25T10:22:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74559.log 2024-04-25T10:22:53.913 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44940.log.gz 2024-04-25T10:22:53.914 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-client.admin.33249.log: 88.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.33330.log.gz --verbose -- /var/log/ceph/ceph-client.admin.33450.log 2024-04-25T10:22:53.914 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-25T10:22:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106929.log 2024-04-25T10:22:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74559.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.100616.log.gz 2024-04-25T10:22:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109850.log 2024-04-25T10:22:53.915 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74559.log.gz 2024-04-25T10:22:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97624.log 2024-04-25T10:22:53.917 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.106929.log.gz 2024-04-25T10:22:53.917 INFO:teuthology.orchestra.run.smithi172.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.33249.log.gz 2024-04-25T10:22:53.917 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-04-25T10:22:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75094.log 2024-04-25T10:22:53.918 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.109850.log.gz 2024-04-25T10:22:53.918 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55365.log 2024-04-25T10:22:53.919 INFO:teuthology.orchestra.run.smithi172.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-25T10:22:53.919 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.97624.log.gz 2024-04-25T10:22:53.919 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.75094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66315.log 2024-04-25T10:22:53.920 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.75094.log.gz 2024-04-25T10:22:53.920 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63370.log 2024-04-25T10:22:53.920 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mds.d.log: /var/log/ceph/ceph-client.admin.33450.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33450.log.gz 2024-04-25T10:22:53.921 INFO:teuthology.orchestra.run.smithi172.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-25T10:22:53.922 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.66315.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.55365.log.gz 0.0% 2024-04-25T10:22:53.922 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.0.66315.log.gz 2024-04-25T10:22:53.922 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60013.log 2024-04-25T10:22:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90313.log 2024-04-25T10:22:53.923 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63370.log.gz 2024-04-25T10:22:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59560.log 2024-04-25T10:22:53.924 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.60013.log.gz 2024-04-25T10:22:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41043.log 2024-04-25T10:22:53.925 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.90313.log.gz 2024-04-25T10:22:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92590.log 2024-04-25T10:22:53.926 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.59560.log.gz 2024-04-25T10:22:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100535.log 2024-04-25T10:22:53.927 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.41043.log.gz 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95283.log 2024-04-25T10:22:53.928 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.92590.log.gz 2024-04-25T10:22:53.929 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.100535.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.0.100535.log.gz --verbose 2024-04-25T10:22:53.929 INFO:teuthology.orchestra.run.smithi165.stderr: -- /var/log/ceph/ceph-client.admin.57772.log 2024-04-25T10:22:53.930 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94171.log 2024-04-25T10:22:53.930 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95283.log.gz 2024-04-25T10:22:53.931 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98369.log 2024-04-25T10:22:53.931 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.57772.log.gz 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42402.log 2024-04-25T10:22:53.932 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94171.log.gz 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40691.log 2024-04-25T10:22:53.933 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98369.log.gz 2024-04-25T10:22:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65424.log 2024-04-25T10:22:53.934 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42402.log.gz 2024-04-25T10:22:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104469.log 2024-04-25T10:22:53.935 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.40691.log.gz 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62310.log 2024-04-25T10:22:53.936 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65424.log.gz 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97755.log 2024-04-25T10:22:53.937 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.104469.log.gz 2024-04-25T10:22:53.938 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84338.log 2024-04-25T10:22:53.938 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62310.log.gz 2024-04-25T10:22:53.939 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107325.log 2024-04-25T10:22:53.940 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.97755.log.gz 2024-04-25T10:22:53.940 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84338.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.84338.log.gz 2024-04-25T10:22:53.940 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61093.log 2024-04-25T10:22:53.942 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107013.log 2024-04-25T10:22:53.942 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107325.log.gz 2024-04-25T10:22:53.943 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78528.log 2024-04-25T10:22:53.943 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61093.log.gz 2024-04-25T10:22:53.944 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89935.log 2024-04-25T10:22:53.944 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.107013.log.gz 2024-04-25T10:22:53.945 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78528.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42735.log 2024-04-25T10:22:53.945 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78528.log.gz 2024-04-25T10:22:53.946 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98953.log 2024-04-25T10:22:53.946 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89935.log.gz 2024-04-25T10:22:53.947 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46155.log 2024-04-25T10:22:53.947 INFO:teuthology.orchestra.run.smithi165.stderr: 89.7% -- replaced with /var/log/ceph/ceph-client.admin.42735.log.gz 2024-04-25T10:22:53.948 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55760.log 2024-04-25T10:22:53.948 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98953.log.gz 2024-04-25T10:22:53.949 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87271.log 2024-04-25T10:22:53.949 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46155.log.gz 2024-04-25T10:22:53.950 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55760.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87697.log 2024-04-25T10:22:53.951 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.55760.log.gz 2024-04-25T10:22:53.951 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93452.log 2024-04-25T10:22:53.951 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87271.log.gz 2024-04-25T10:22:53.952 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79058.log 2024-04-25T10:22:53.952 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.87697.log.gz 2024-04-25T10:22:53.953 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100054.log 2024-04-25T10:22:53.953 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93452.log.gz 2024-04-25T10:22:53.954 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39311.log 2024-04-25T10:22:53.954 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.79058.log.gz 2024-04-25T10:22:53.955 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45168.log 2024-04-25T10:22:53.955 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.100054.log.gz 2024-04-25T10:22:53.956 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43862.log 2024-04-25T10:22:53.956 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.39311.log.gz 2024-04-25T10:22:53.957 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73292.log 2024-04-25T10:22:53.957 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45168.log.gz 2024-04-25T10:22:53.958 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.43862.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43862.log.gz 2024-04-25T10:22:53.958 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105586.log 2024-04-25T10:22:53.959 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73114.log 2024-04-25T10:22:53.959 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.73292.log.gz 2024-04-25T10:22:53.960 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105586.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112355.log 2024-04-25T10:22:53.961 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.105586.log.gz 2024-04-25T10:22:53.961 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112313.log 2024-04-25T10:22:53.962 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73114.log.gz 2024-04-25T10:22:53.962 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112355.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107932.log 2024-04-25T10:22:53.963 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112355.log.gz 2024-04-25T10:22:53.964 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42455.log 2024-04-25T10:22:53.964 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112313.log.gz 2024-04-25T10:22:53.965 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107932.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83326.log 2024-04-25T10:22:53.965 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107932.log.gz 2024-04-25T10:22:53.966 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42455.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82231.log 2024-04-25T10:22:53.966 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42455.log.gz 2024-04-25T10:22:53.967 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100198.log 2024-04-25T10:22:53.967 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83326.log.gz 2024-04-25T10:22:53.968 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82231.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90271.log 2024-04-25T10:22:53.968 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.82231.log.gz 2024-04-25T10:22:53.969 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55233.log 2024-04-25T10:22:53.969 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100198.log.gz 2024-04-25T10:22:53.970 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57580.log 2024-04-25T10:22:53.970 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.90271.log.gz 2024-04-25T10:22:53.971 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105828.log 2024-04-25T10:22:53.972 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57580.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55233.log.gz 2024-04-25T10:22:53.972 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111365.log 2024-04-25T10:22:53.972 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57580.log.gz 2024-04-25T10:22:53.974 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106810.log 2024-04-25T10:22:53.974 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.105828.log.gz 2024-04-25T10:22:53.975 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38939.log 2024-04-25T10:22:53.975 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111365.log.gz 2024-04-25T10:22:53.975 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56316.log 2024-04-25T10:22:53.976 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.38939.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.106810.log.gz 2024-04-25T10:22:53.977 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57011.log 2024-04-25T10:22:53.977 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.38939.log.gz 2024-04-25T10:22:53.978 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56316.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56316.log.gz 2024-04-25T10:22:53.978 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77828.log 2024-04-25T10:22:53.979 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43113.log 2024-04-25T10:22:53.979 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.57011.log.gz 2024-04-25T10:22:53.980 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57706.log 2024-04-25T10:22:53.981 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.77828.log.gz 2024-04-25T10:22:53.981 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58685.log 2024-04-25T10:22:53.982 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43113.log.gz 2024-04-25T10:22:53.982 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98327.log 2024-04-25T10:22:53.983 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58685.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.57706.log.gz 2024-04-25T10:22:53.985 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95241.log 2024-04-25T10:22:53.985 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58685.log.gz 2024-04-25T10:22:53.985 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98327.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98327.log.gz 2024-04-25T10:22:53.985 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56022.log 2024-04-25T10:22:53.986 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84040.log 2024-04-25T10:22:53.986 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95241.log.gz 2024-04-25T10:22:53.987 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98663.log 2024-04-25T10:22:53.987 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56022.log.gz 2024-04-25T10:22:53.988 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107619.log 2024-04-25T10:22:53.988 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84040.log.gz 2024-04-25T10:22:53.989 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98663.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73030.log 2024-04-25T10:22:53.989 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98663.log.gz 2024-04-25T10:22:53.990 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64873.log 2024-04-25T10:22:53.990 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.107619.log.gz 2024-04-25T10:22:53.991 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73030.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41373.log 2024-04-25T10:22:53.991 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73030.log.gz 2024-04-25T10:22:53.992 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46670.log 2024-04-25T10:22:53.993 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64873.log.gz 2024-04-25T10:22:53.993 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2024-04-25T10:22:53.993 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75136.log 2024-04-25T10:22:53.994 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.41373.log.gz 2024-04-25T10:22:53.995 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112397.log 2024-04-25T10:22:53.995 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46670.log.gz 2024-04-25T10:22:53.996 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.75136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83872.log 2024-04-25T10:22:53.996 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75136.log.gz 2024-04-25T10:22:53.997 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41559.log 2024-04-25T10:22:53.997 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112397.log.gz 2024-04-25T10:22:53.998 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83872.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-25T10:22:53.998 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.83872.log.gz 2024-04-25T10:22:53.999 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62895.log 2024-04-25T10:22:53.999 INFO:teuthology.orchestra.run.smithi165.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.41559.log.gz 2024-04-25T10:22:54.000 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40013.log 2024-04-25T10:22:54.001 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64560.log 2024-04-25T10:22:54.001 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.62895.log.gz 2024-04-25T10:22:54.002 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60452.log 2024-04-25T10:22:54.003 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.40013.log.gz 2024-04-25T10:22:54.003 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107764.log 2024-04-25T10:22:54.003 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64560.log.gz 2024-04-25T10:22:54.004 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84987.log 2024-04-25T10:22:54.004 INFO:teuthology.orchestra.run.smithi165.stderr: 86.9% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-25T10:22:54.005 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60452.log.gz 2024-04-25T10:22:54.006 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90061.log 2024-04-25T10:22:54.006 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107764.log.gz 2024-04-25T10:22:54.006 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92416.log 2024-04-25T10:22:54.007 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84987.log.gz 2024-04-25T10:22:54.007 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2024-04-25T10:22:54.008 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.90061.log.gz 2024-04-25T10:22:54.008 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107806.log 2024-04-25T10:22:54.009 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.92416.log.gz 2024-04-25T10:22:54.009 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93588.log 2024-04-25T10:22:54.009 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2024-04-25T10:22:54.010 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42819.log 2024-04-25T10:22:54.011 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107806.log.gz 2024-04-25T10:22:54.011 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94894.log 2024-04-25T10:22:54.011 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93588.log.gz 2024-04-25T10:22:54.012 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108936.log 2024-04-25T10:22:54.012 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.42819.log.gz 2024-04-25T10:22:54.013 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105035.log 2024-04-25T10:22:54.013 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94894.log.gz 2024-04-25T10:22:54.014 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89110.log 2024-04-25T10:22:54.014 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108936.log.gz 2024-04-25T10:22:54.015 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105035.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93756.log 2024-04-25T10:22:54.015 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105035.log.gz 2024-04-25T10:22:54.016 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82273.log 2024-04-25T10:22:54.016 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.89110.log.gz 2024-04-25T10:22:54.017 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101102.log 2024-04-25T10:22:54.017 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93756.log.gz 2024-04-25T10:22:54.018 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82273.log: gzip -5 --verbose -- 89.1% /var/log/ceph/ceph-client.admin.56232.log -- replaced with /var/log/ceph/ceph-client.admin.82273.log.gz 2024-04-25T10:22:54.018 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-25T10:22:54.019 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101102.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42113.log 2024-04-25T10:22:54.019 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101102.log.gz 2024-04-25T10:22:54.020 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56232.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.56232.log.gz 2024-04-25T10:22:54.020 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87313.log 2024-04-25T10:22:54.021 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42113.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.42113.log.gz 2024-04-25T10:22:54.021 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.62609.log 2024-04-25T10:22:54.023 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47049.log 2024-04-25T10:22:54.023 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.87313.log.gz 2024-04-25T10:22:54.023 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.62609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62609.log.gz 2024-04-25T10:22:54.024 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63057.log 2024-04-25T10:22:54.025 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41767.log 2024-04-25T10:22:54.025 INFO:teuthology.orchestra.run.smithi165.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.47049.log.gz 2024-04-25T10:22:54.026 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59815.log 2024-04-25T10:22:54.026 INFO:teuthology.orchestra.run.smithi165.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.63057.log.gz 2024-04-25T10:22:54.027 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41767.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41767.log.gz 2024-04-25T10:22:54.027 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33276.log 2024-04-25T10:22:54.028 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39947.log 2024-04-25T10:22:54.028 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59815.log.gz 2024-04-25T10:22:54.029 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.33276.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-25T10:22:54.030 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.33276.log.gz 2024-04-25T10:22:54.030 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55896.log 2024-04-25T10:22:54.031 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.2.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.39947.log.gz 2024-04-25T10:22:54.031 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58921.log 2024-04-25T10:22:54.032 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55896.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64266.log 2024-04-25T10:22:54.033 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.55896.log.gz 2024-04-25T10:22:54.033 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103872.log 2024-04-25T10:22:54.034 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.58921.log.gz 2024-04-25T10:22:54.034 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64266.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101270.log 2024-04-25T10:22:54.035 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64266.log.gz 2024-04-25T10:22:54.035 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44303.log 2024-04-25T10:22:54.036 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.103872.log.gz 2024-04-25T10:22:54.037 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79397.log 2024-04-25T10:22:54.037 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101270.log.gz 2024-04-25T10:22:54.038 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.2.44303.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39567.log 2024-04-25T10:22:54.038 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44303.log.gz 2024-04-25T10:22:54.039 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79397.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92548.log 2024-04-25T10:22:54.039 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79397.log.gz 2024-04-25T10:22:54.040 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64046.log 2024-04-25T10:22:54.040 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.39567.log.gz 2024-04-25T10:22:54.041 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47007.log 2024-04-25T10:22:54.041 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.92548.log.gz 2024-04-25T10:22:54.042 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105119.log 2024-04-25T10:22:54.042 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64046.log.gz 2024-04-25T10:22:54.043 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47007.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41776.log 2024-04-25T10:22:54.043 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47007.log.gz 2024-04-25T10:22:54.044 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90187.log 2024-04-25T10:22:54.045 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105119.log.gz 2024-04-25T10:22:54.045 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95115.log 2024-04-25T10:22:54.045 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.41776.log.gz 2024-04-25T10:22:54.046 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56503.log 2024-04-25T10:22:54.047 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.90187.log.gz 2024-04-25T10:22:54.047 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64088.log 2024-04-25T10:22:54.048 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95115.log.gz 2024-04-25T10:22:54.049 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104637.log 2024-04-25T10:22:54.049 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56503.log.gz 2024-04-25T10:22:54.050 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92632.log 2024-04-25T10:22:54.050 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.64088.log.gz 2024-04-25T10:22:54.050 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39863.log 2024-04-25T10:22:54.051 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104637.log.gz 2024-04-25T10:22:54.052 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64602.log 2024-04-25T10:22:54.052 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.92632.log.gz 2024-04-25T10:22:54.053 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107097.log 2024-04-25T10:22:54.053 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.39863.log.gz 2024-04-25T10:22:54.054 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2024-04-25T10:22:54.054 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.64602.log.gz 2024-04-25T10:22:54.055 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107451.log 2024-04-25T10:22:54.055 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107097.log.gz 2024-04-25T10:22:54.056 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63564.log 2024-04-25T10:22:54.056 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2024-04-25T10:22:54.057 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75034.log 2024-04-25T10:22:54.057 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107451.log.gz 2024-04-25T10:22:54.058 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102914.log 2024-04-25T10:22:54.058 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.63564.log.gz 2024-04-25T10:22:54.059 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.75034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46755.log 2024-04-25T10:22:54.060 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.75034.log.gz 2024-04-25T10:22:54.060 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88816.log 2024-04-25T10:22:54.060 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102914.log.gz 2024-04-25T10:22:54.061 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46755.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89026.log 2024-04-25T10:22:54.062 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46755.log.gz 2024-04-25T10:22:54.062 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73922.log 2024-04-25T10:22:54.063 INFO:teuthology.orchestra.run.smithi165.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.88816.log.gz 2024-04-25T10:22:54.063 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42693.log 2024-04-25T10:22:54.064 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89026.log.gz 2024-04-25T10:22:54.065 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112719.log 2024-04-25T10:22:54.065 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.73922.log.gz 2024-04-25T10:22:54.066 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61135.log 2024-04-25T10:22:54.066 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42693.log.gz 2024-04-25T10:22:54.067 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41163.log 2024-04-25T10:22:54.067 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.112719.log.gz 2024-04-25T10:22:54.068 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108810.log 2024-04-25T10:22:54.068 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.61135.log.gz 2024-04-25T10:22:54.069 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104291.log 2024-04-25T10:22:54.069 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41163.log.gz 2024-04-25T10:22:54.070 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74515.log 2024-04-25T10:22:54.071 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108810.log.gz 2024-04-25T10:22:54.071 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59237.log 2024-04-25T10:22:54.071 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.104291.log.gz 2024-04-25T10:22:54.072 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77691.log 2024-04-25T10:22:54.072 INFO:teuthology.orchestra.run.smithi165.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74515.log.gz 2024-04-25T10:22:54.073 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59237.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107535.log 2024-04-25T10:22:54.073 INFO:teuthology.orchestra.run.smithi165.stderr: 89.6% -- replaced with /var/log/ceph/ceph-client.admin.59237.log.gz 2024-04-25T10:22:54.074 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104679.log 2024-04-25T10:22:54.075 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.77691.log.gz 2024-04-25T10:22:54.075 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107535.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100156.log 2024-04-25T10:22:54.076 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107535.log.gz 2024-04-25T10:22:54.077 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45867.log 2024-04-25T10:22:54.077 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104679.log.gz 2024-04-25T10:22:54.078 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100282.log 2024-04-25T10:22:54.078 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.100156.log.gz 2024-04-25T10:22:54.079 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60188.log 2024-04-25T10:22:54.079 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45867.log.gz 2024-04-25T10:22:54.079 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100282.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47629.log 2024-04-25T10:22:54.081 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60188.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.100282.log.gz 2024-04-25T10:22:54.081 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73072.log 2024-04-25T10:22:54.081 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60188.log.gz 2024-04-25T10:22:54.082 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77523.log 2024-04-25T10:22:54.082 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47629.log.gz 2024-04-25T10:22:54.082 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73072.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102162.log 2024-04-25T10:22:54.083 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.73072.log.gz 2024-04-25T10:22:54.084 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88039.log 2024-04-25T10:22:54.084 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77523.log.gz 2024-04-25T10:22:54.085 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104908.log 2024-04-25T10:22:54.085 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102162.log.gz 2024-04-25T10:22:54.086 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49006.log 2024-04-25T10:22:54.086 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88039.log.gz 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95199.log 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104908.log.gz 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.49006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45951.log 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.49006.log.gz 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95199.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.95199.log.gz 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42987.log 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57436.log 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45951.log.gz 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82057.log 2024-04-25T10:22:54.092 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42987.log.gz 2024-04-25T10:22:54.093 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57436.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57436.log.gz 2024-04-25T10:22:54.093 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100941.log 2024-04-25T10:22:54.094 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109542.log 2024-04-25T10:22:54.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100941.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.82057.log.gz 2024-04-25T10:22:54.095 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-25T10:22:54.096 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109542.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.100941.log.gz 2024-04-25T10:22:54.096 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.109542.log.gz 2024-04-25T10:22:54.096 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48069.log 2024-04-25T10:22:54.097 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102956.log 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48069.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48069.log.gz 2024-04-25T10:22:54.098 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109626.log 2024-04-25T10:22:54.099 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101532.log 2024-04-25T10:22:54.100 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109626.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102956.log.gz 2024-04-25T10:22:54.100 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109626.log.gz 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83746.log 2024-04-25T10:22:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101532.log: 89.7% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-25T10:22:54.102 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101532.log.gz 2024-04-25T10:22:54.102 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44436.log 2024-04-25T10:22:54.103 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110026.log 2024-04-25T10:22:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44436.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83746.log.gz 2024-04-25T10:22:54.104 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44436.log.gz 2024-04-25T10:22:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58144.log 2024-04-25T10:22:54.105 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74740.log 2024-04-25T10:22:54.106 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110026.log.gz 2024-04-25T10:22:54.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79353.log 2024-04-25T10:22:54.106 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.58144.log.gz 2024-04-25T10:22:54.108 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56358.log 2024-04-25T10:22:54.108 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.74740.log.gz 2024-04-25T10:22:54.109 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79353.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.79353.log.gz 2024-04-25T10:22:54.109 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108209.log 2024-04-25T10:22:54.110 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82657.log 2024-04-25T10:22:54.110 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56358.log.gz 2024-04-25T10:22:54.111 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95409.log 2024-04-25T10:22:54.111 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108209.log.gz 2024-04-25T10:22:54.112 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82657.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.82657.log.gz 2024-04-25T10:22:54.112 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101144.log 2024-04-25T10:22:54.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63755.log 2024-04-25T10:22:54.114 INFO:teuthology.orchestra.run.smithi165.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.95409.log.gz 2024-04-25T10:22:54.114 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101144.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42104.log 2024-04-25T10:22:54.114 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101144.log.gz 2024-04-25T10:22:54.115 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.63755.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63755.log.gz 2024-04-25T10:22:54.115 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90145.log 2024-04-25T10:22:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45504.log 2024-04-25T10:22:54.117 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42104.log.gz 2024-04-25T10:22:54.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90145.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.90145.log.gz 2024-04-25T10:22:54.117 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73502.log 2024-04-25T10:22:54.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74346.log 2024-04-25T10:22:54.118 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45504.log.gz 2024-04-25T10:22:54.119 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73502.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43239.log 2024-04-25T10:22:54.119 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.73502.log.gz 2024-04-25T10:22:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74346.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.74346.log.gz 2024-04-25T10:22:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40875.log 2024-04-25T10:22:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93714.log 2024-04-25T10:22:54.122 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40875.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43239.log.gz 2024-04-25T10:22:54.122 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.40875.log.gz 2024-04-25T10:22:54.122 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42115.log 2024-04-25T10:22:54.123 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99928.log 2024-04-25T10:22:54.124 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93714.log.gz 2024-04-25T10:22:54.124 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109020.log 2024-04-25T10:22:54.125 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42115.log.gz 2024-04-25T10:22:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44562.log 2024-04-25T10:22:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109020.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99928.log.gz 2024-04-25T10:22:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84135.log 2024-04-25T10:22:54.127 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.109020.log.gz 2024-04-25T10:22:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44562.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44562.log.gz 2024-04-25T10:22:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89433.log 2024-04-25T10:22:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42452.log 2024-04-25T10:22:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89433.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84135.log.gz 2024-04-25T10:22:54.130 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.89433.log.gz 2024-04-25T10:22:54.130 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109984.log 2024-04-25T10:22:54.131 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73838.log 2024-04-25T10:22:54.131 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42452.log.gz 2024-04-25T10:22:54.132 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109984.log: gzip 87.7% -- replaced with /var/log/ceph/ceph-client.admin.109984.log.gz 2024-04-25T10:22:54.132 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.112481.log 2024-04-25T10:22:54.133 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73838.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108894.log 2024-04-25T10:22:54.134 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112481.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.73838.log.gz 2024-04-25T10:22:54.134 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.112481.log.gz 2024-04-25T10:22:54.134 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65848.log 2024-04-25T10:22:54.135 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47259.log 2024-04-25T10:22:54.135 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108894.log.gz 2024-04-25T10:22:54.136 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65848.log: gzip 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65848.log.gz 2024-04-25T10:22:54.136 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27235.log 2024-04-25T10:22:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47259.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58551.log 2024-04-25T10:22:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph.tmp-client.admin.27235.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47259.log.gz 2024-04-25T10:22:54.138 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27235.log.gz 2024-04-25T10:22:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79309.log 2024-04-25T10:22:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58551.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110274.log 2024-04-25T10:22:54.139 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58551.log.gz 2024-04-25T10:22:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105161.log 2024-04-25T10:22:54.140 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79309.log.gz 2024-04-25T10:22:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73376.log 2024-04-25T10:22:54.141 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.110274.log.gz 2024-04-25T10:22:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105161.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82399.log 2024-04-25T10:22:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73376.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105161.log.gz 2024-04-25T10:22:54.143 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73376.log.gz 2024-04-25T10:22:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42399.log 2024-04-25T10:22:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111221.log 2024-04-25T10:22:54.144 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.82399.log.gz 2024-04-25T10:22:54.145 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105998.log 2024-04-25T10:22:54.145 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.42399.log.gz 2024-04-25T10:22:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111221.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111221.log.gz 2024-04-25T10:22:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103722.log 2024-04-25T10:22:54.147 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48738.log 2024-04-25T10:22:54.148 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103722.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.105998.log.gz 2024-04-25T10:22:54.149 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83662.log 2024-04-25T10:22:54.149 INFO:teuthology.orchestra.run.smithi165.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.103722.log.gz 2024-04-25T10:22:54.150 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65041.log 2024-04-25T10:22:54.150 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48738.log.gz 2024-04-25T10:22:54.151 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100682.log 2024-04-25T10:22:54.151 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.83662.log.gz 2024-04-25T10:22:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65041.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.65041.log.gz 2024-04-25T10:22:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57138.log 2024-04-25T10:22:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44646.log 2024-04-25T10:22:54.153 INFO:teuthology.orchestra.run.smithi165.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100682.log.gz 2024-04-25T10:22:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97529.log 2024-04-25T10:22:54.154 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.57138.log.gz 2024-04-25T10:22:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44646.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99886.log 2024-04-25T10:22:54.155 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44646.log.gz 2024-04-25T10:22:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97529.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97529.log.gz 2024-04-25T10:22:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72761.log 2024-04-25T10:22:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99886.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59635.log 2024-04-25T10:22:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.72761.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99886.log.gz 2024-04-25T10:22:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108852.log 2024-04-25T10:22:54.158 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.72761.log.gz 2024-04-25T10:22:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59635.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.59635.log.gz 2024-04-25T10:22:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56629.log 2024-04-25T10:22:54.160 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42777.log 2024-04-25T10:22:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56629.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108852.log.gz 2024-04-25T10:22:54.162 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56629.log.gz 2024-04-25T10:22:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-25T10:22:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108640.log 2024-04-25T10:22:54.163 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.42777.log.gz 2024-04-25T10:22:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74185.log 2024-04-25T10:22:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108640.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108640.log.gz 2024-04-25T10:22:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74388.log 2024-04-25T10:22:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83410.log 2024-04-25T10:22:54.166 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74388.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.74185.log.gz 2024-04-25T10:22:54.167 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.74388.log.gz 2024-04-25T10:22:54.167 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63169.log 2024-04-25T10:22:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83410.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.83410.log.gz 2024-04-25T10:22:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64518.log 2024-04-25T10:22:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42092.log 2024-04-25T10:22:54.170 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64518.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.63169.log.gz 2024-04-25T10:22:54.170 INFO:teuthology.orchestra.run.smithi165.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.64518.log.gz 2024-04-25T10:22:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89068.log 2024-04-25T10:22:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99419.log 2024-04-25T10:22:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89068.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42092.log.gz 2024-04-25T10:22:54.173 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89068.log.gz 2024-04-25T10:22:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44730.log 2024-04-25T10:22:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41782.log 2024-04-25T10:22:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44730.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99419.log.gz 2024-04-25T10:22:54.175 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44730.log.gz 2024-04-25T10:22:54.175 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104595.log 2024-04-25T10:22:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41773.log 2024-04-25T10:22:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104595.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.41782.log.gz 2024-04-25T10:22:54.177 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.104595.log.gz 2024-04-25T10:22:54.178 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78883.log 2024-04-25T10:22:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104207.log 2024-04-25T10:22:54.179 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.41773.log.gz 2024-04-25T10:22:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66016.log 2024-04-25T10:22:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104207.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78883.log.gz 2024-04-25T10:22:54.181 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.104207.log.gz 2024-04-25T10:22:54.181 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99970.log 2024-04-25T10:22:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.66016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110909.log 2024-04-25T10:22:54.183 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99970.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.66016.log.gz 2024-04-25T10:22:54.183 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99970.log.gz 2024-04-25T10:22:54.184 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42118.log 2024-04-25T10:22:54.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60673.log 2024-04-25T10:22:54.185 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42118.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110909.log.gz 2024-04-25T10:22:54.186 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42118.log.gz 2024-04-25T10:22:54.186 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78258.log 2024-04-25T10:22:54.187 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84509.log 2024-04-25T10:22:54.187 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78258.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.60673.log.gz 2024-04-25T10:22:54.188 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.78258.log.gz 2024-04-25T10:22:54.188 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61861.log 2024-04-25T10:22:54.189 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78300.log 2024-04-25T10:22:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61861.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84509.log.gz 2024-04-25T10:22:54.191 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61861.log.gz 2024-04-25T10:22:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82801.log 2024-04-25T10:22:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108726.log 2024-04-25T10:22:54.192 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78300.log.gz 2024-04-25T10:22:54.193 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110556.log 2024-04-25T10:22:54.193 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108726.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.82801.log.gz 2024-04-25T10:22:54.194 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.108726.log.gz 2024-04-25T10:22:54.194 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79439.log 2024-04-25T10:22:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94026.log 2024-04-25T10:22:54.196 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110556.log.gz 2024-04-25T10:22:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102419.log 2024-04-25T10:22:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94026.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79439.log.gz 2024-04-25T10:22:54.198 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.94026.log.gz 2024-04-25T10:22:54.198 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58252.log 2024-04-25T10:22:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47343.log 2024-04-25T10:22:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58252.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.102419.log.gz 2024-04-25T10:22:54.200 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45825.log 2024-04-25T10:22:54.200 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58252.log.gz 2024-04-25T10:22:54.201 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47343.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111053.log 2024-04-25T10:22:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45825.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47343.log.gz 2024-04-25T10:22:54.203 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45825.log.gz 2024-04-25T10:22:54.203 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105673.log 2024-04-25T10:22:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63412.log 2024-04-25T10:22:54.204 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111053.log.gz 2024-04-25T10:22:54.205 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111893.log 2024-04-25T10:22:54.205 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63412.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.105673.log.gz 2024-04-25T10:22:54.206 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.63412.log.gz 2024-04-25T10:22:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48929.log 2024-04-25T10:22:54.207 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88900.log 2024-04-25T10:22:54.207 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.111893.log.gz 2024-04-25T10:22:54.208 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.48929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60631.log 2024-04-25T10:22:54.208 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48929.log.gz 2024-04-25T10:22:54.210 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88900.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88900.log.gz 2024-04-25T10:22:54.210 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99546.log 2024-04-25T10:22:54.211 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105735.log 2024-04-25T10:22:54.211 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.60631.log.gz 2024-04-25T10:22:54.212 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57963.log 2024-04-25T10:22:54.212 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99546.log.gz 2024-04-25T10:22:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105735.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105735.log.gz 2024-04-25T10:22:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58318.log 2024-04-25T10:22:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.57963.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57963.log.gz 2024-04-25T10:22:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101910.log 2024-04-25T10:22:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58318.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.58318.log.gz 2024-04-25T10:22:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59947.log 2024-04-25T10:22:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43843.log 2024-04-25T10:22:54.216 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101910.log.gz 2024-04-25T10:22:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78655.log 2024-04-25T10:22:54.217 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59947.log.gz 2024-04-25T10:22:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.43843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.43843.log.gz 2024-04-25T10:22:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46113.log 2024-04-25T10:22:54.219 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82615.log 2024-04-25T10:22:54.219 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78655.log.gz 2024-04-25T10:22:54.220 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46113.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48546.log 2024-04-25T10:22:54.220 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.46113.log.gz 2024-04-25T10:22:54.221 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82615.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77954.log 2024-04-25T10:22:54.221 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82615.log.gz 2024-04-25T10:22:54.222 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90379.log 2024-04-25T10:22:54.222 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2024-04-25T10:22:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77954.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.77954.log.gz 2024-04-25T10:22:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61930.log 2024-04-25T10:22:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73198.log 2024-04-25T10:22:54.224 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.90379.log.gz 2024-04-25T10:22:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61930.log: gzip -5 88.6% --verbose -- replaced with /var/log/ceph/ceph-client.admin.61930.log.gz -- 2024-04-25T10:22:54.225 INFO:teuthology.orchestra.run.smithi165.stderr: /var/log/ceph/ceph-client.0.109277.log 2024-04-25T10:22:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90689.log 2024-04-25T10:22:54.226 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73198.log.gz 2024-04-25T10:22:54.227 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.109277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.109277.log.gz 2024-04-25T10:22:54.227 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-04-25T10:22:54.228 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84889.log 2024-04-25T10:22:54.228 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.90689.log.gz 2024-04-25T10:22:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104763.log 2024-04-25T10:22:54.230 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84889.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.84889.log.gz 2024-04-25T10:22:54.230 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100012.log 2024-04-25T10:22:54.231 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104763.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88081.log 2024-04-25T10:22:54.232 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104763.log.gz 2024-04-25T10:22:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99121.log 2024-04-25T10:22:54.233 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100012.log.gz 2024-04-25T10:22:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88081.log: gzip 89.5% -- replaced with /var/log/ceph/ceph-client.admin.88081.log.gz -5 2024-04-25T10:22:54.234 INFO:teuthology.orchestra.run.smithi165.stderr: --verbose -- /var/log/ceph/ceph-client.admin.82357.log 2024-04-25T10:22:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59712.log 2024-04-25T10:22:54.235 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.99121.log.gz 2024-04-25T10:22:54.236 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82357.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.82357.log.gz 2024-04-25T10:22:54.236 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110748.log 2024-04-25T10:22:54.238 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48292.log 2024-04-25T10:22:54.238 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59712.log.gz 2024-04-25T10:22:54.239 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61602.log 2024-04-25T10:22:54.240 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48292.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110748.log.gz 2024-04-25T10:22:54.240 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93966.log 2024-04-25T10:22:54.240 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.48292.log.gz 2024-04-25T10:22:54.241 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61602.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.61602.log.gz 2024-04-25T10:22:54.241 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99590.log 2024-04-25T10:22:54.242 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78834.log 2024-04-25T10:22:54.243 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0%/var/log/ceph/ceph-client.admin.99590.log: -- replaced with /var/log/ceph/ceph-client.admin.93966.log.gz 2024-04-25T10:22:54.243 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110951.log 2024-04-25T10:22:54.244 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99590.log.gz 2024-04-25T10:22:54.245 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78834.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78834.log.gz 2024-04-25T10:22:54.245 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83494.log 2024-04-25T10:22:54.246 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.110951.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.111263.log 2024-04-25T10:22:54.246 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110951.log.gz 2024-04-25T10:22:54.247 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83494.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83494.log.gz 2024-04-25T10:22:54.247 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2024-04-25T10:22:54.248 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94087.log 2024-04-25T10:22:54.248 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111263.log.gz 2024-04-25T10:22:54.249 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43281.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2024-04-25T10:22:54.249 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110640.log 2024-04-25T10:22:54.250 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94087.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59345.log 2024-04-25T10:22:54.251 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94087.log.gz 2024-04-25T10:22:54.251 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99760.log 2024-04-25T10:22:54.251 INFO:teuthology.orchestra.run.smithi165.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.110640.log.gz 2024-04-25T10:22:54.252 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75519.log 2024-04-25T10:22:54.253 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99760.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.59345.log.gz 2024-04-25T10:22:54.254 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.99760.log.gz 2024-04-25T10:22:54.254 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105459.log 2024-04-25T10:22:54.255 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.0.75519.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57394.log 2024-04-25T10:22:54.255 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75519.log.gz 2024-04-25T10:22:54.256 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2024-04-25T10:22:54.256 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.105459.log.gz 2024-04-25T10:22:54.257 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101952.log 2024-04-25T10:22:54.257 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57394.log.gz 2024-04-25T10:22:54.258 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46329.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2024-04-25T10:22:54.258 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103040.log 2024-04-25T10:22:54.259 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77283.log 2024-04-25T10:22:54.260 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.101952.log.gz 2024-04-25T10:22:54.260 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99462.log 2024-04-25T10:22:54.260 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103040.log.gz 2024-04-25T10:22:54.261 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47859.log 2024-04-25T10:22:54.262 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99462.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.77283.log.gz 2024-04-25T10:22:54.262 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.99462.log.gz 2024-04-25T10:22:54.263 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73670.log 2024-04-25T10:22:54.264 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88774.log 2024-04-25T10:22:54.264 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.47859.log.gz 2024-04-25T10:22:54.265 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42357.log 2024-04-25T10:22:54.265 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73670.log.gz 2024-04-25T10:22:54.266 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88774.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89212.log 2024-04-25T10:22:54.266 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.88774.log.gz 2024-04-25T10:22:54.267 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56671.log 2024-04-25T10:22:54.267 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.42357.log.gz 2024-04-25T10:22:54.268 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.89212.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39695.log 2024-04-25T10:22:54.268 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9%/var/log/ceph/ceph-client.admin.56671.log: -- replaced with /var/log/ceph/ceph-client.admin.89212.log.gz 2024-04-25T10:22:54.269 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.56671.log.gz 2024-04-25T10:22:54.269 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46455.log 2024-04-25T10:22:54.270 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80284.log 2024-04-25T10:22:54.271 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.39695.log.gz 2024-04-25T10:22:54.271 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46455.log: 88.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.46455.log.gz 2024-04-25T10:22:54.271 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.86989.log 2024-04-25T10:22:54.273 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.80284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87997.log 2024-04-25T10:22:54.273 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80284.log.gz 2024-04-25T10:22:54.273 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.86989.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-25T10:22:54.274 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.86989.log.gz 2024-04-25T10:22:54.275 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95325.log 2024-04-25T10:22:54.275 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.87997.log.gz 2024-04-25T10:22:54.276 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56713.log 2024-04-25T10:22:54.277 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95325.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-25T10:22:54.277 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.95325.log.gz 2024-04-25T10:22:54.278 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56713.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64476.log 2024-04-25T10:22:54.278 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.56713.log.gz 2024-04-25T10:22:54.280 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107241.log 2024-04-25T10:22:54.281 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48925.log 2024-04-25T10:22:54.282 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.64476.log.gz 2024-04-25T10:22:54.283 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60326.log 2024-04-25T10:22:54.283 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107241.log.gz 2024-04-25T10:22:54.283 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.48925.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.48925.log.gz 2024-04-25T10:22:54.284 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89170.log 2024-04-25T10:22:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60326.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79267.log 2024-04-25T10:22:54.285 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60326.log.gz 2024-04-25T10:22:54.287 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89170.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89170.log.gz 2024-04-25T10:22:54.287 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89851.log 2024-04-25T10:22:54.288 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79267.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79267.log.gz 2024-04-25T10:22:54.288 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60122.log 2024-04-25T10:22:54.290 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103666.log 2024-04-25T10:22:54.290 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.89851.log.gz 2024-04-25T10:22:54.291 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39905.log 2024-04-25T10:22:54.291 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.60122.log.gz 2024-04-25T10:22:54.292 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103666.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.103666.log.gz 2024-04-25T10:22:54.293 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39905.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39905.log.gz 2024-04-25T10:22:54.294 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93284.log 2024-04-25T10:22:54.295 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62226.log 2024-04-25T10:22:54.297 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73334.log 2024-04-25T10:22:54.297 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.93284.log.gz 2024-04-25T10:22:54.298 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62226.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.62226.log.gz 2024-04-25T10:22:54.299 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73334.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73334.log.gz 2024-04-25T10:22:54.300 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102631.log 2024-04-25T10:22:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43992.log 2024-04-25T10:22:54.302 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99718.log 2024-04-25T10:22:54.303 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.102631.log.gz 2024-04-25T10:22:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43992.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43992.log.gz 2024-04-25T10:22:54.305 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99718.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.99718.log.gz 2024-04-25T10:22:54.305 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77607.log 2024-04-25T10:22:54.307 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89807.log 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77607.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102337.log 2024-04-25T10:22:54.308 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77607.log.gz 2024-04-25T10:22:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89807.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.89807.log.gz 2024-04-25T10:22:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102337.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102337.log.gz 2024-04-25T10:22:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104553.log 2024-04-25T10:22:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74603.log 2024-04-25T10:22:54.314 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104553.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104553.log.gz 2024-04-25T10:22:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74603.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.74603.log.gz 2024-04-25T10:22:54.315 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-25T10:22:54.316 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64789.log 2024-04-25T10:22:54.318 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108290.log 2024-04-25T10:22:54.319 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64789.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.64789.log.gz 2024-04-25T10:22:54.320 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108290.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.108290.log.gz 2024-04-25T10:22:54.321 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80280.log 2024-04-25T10:22:54.322 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72695.log 2024-04-25T10:22:54.323 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.80280.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.80280.log.gz 2024-04-25T10:22:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.72695.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.72695.log.gz 2024-04-25T10:22:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87055.log 2024-04-25T10:22:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84679.log 2024-04-25T10:22:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87055.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87055.log.gz 2024-04-25T10:22:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84679.log.gz 2024-04-25T10:22:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46713.log 2024-04-25T10:22:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87739.log 2024-04-25T10:22:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46713.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46713.log.gz 2024-04-25T10:22:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87739.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.87739.log.gz 2024-04-25T10:22:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88260.log 2024-04-25T10:22:54.337 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111576.log 2024-04-25T10:22:54.339 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88260.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88260.log.gz 2024-04-25T10:22:54.340 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111576.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111576.log.gz 2024-04-25T10:22:54.340 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75220.log 2024-04-25T10:22:54.342 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94556.log 2024-04-25T10:22:54.343 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.75220.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75220.log.gz 2024-04-25T10:22:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94556.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.94556.log.gz 2024-04-25T10:22:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78090.log 2024-04-25T10:22:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56106.log 2024-04-25T10:22:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78090.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.78090.log.gz 2024-04-25T10:22:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56106.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.56106.log.gz 2024-04-25T10:22:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84423.log 2024-04-25T10:22:54.352 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84423.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.84423.log.gz 2024-04-25T10:22:54.352 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83106.log 2024-04-25T10:22:54.354 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77649.log 2024-04-25T10:22:54.355 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83106.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83106.log.gz 2024-04-25T10:22:54.356 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77649.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.77649.log.gz 2024-04-25T10:22:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41331.log 2024-04-25T10:22:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65974.log 2024-04-25T10:22:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41331.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41331.log.gz 2024-04-25T10:22:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65974.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65974.log.gz 2024-04-25T10:22:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88564.log 2024-04-25T10:22:54.363 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99296.log 2024-04-25T10:22:54.364 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88564.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88564.log.gz 2024-04-25T10:22:54.365 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99296.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99296.log.gz 2024-04-25T10:22:54.366 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59411.log 2024-04-25T10:22:54.367 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98284.log 2024-04-25T10:22:54.369 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59411.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59411.log.gz 2024-04-25T10:22:54.370 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98284.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.98284.log.gz 2024-04-25T10:22:54.370 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103998.log 2024-04-25T10:22:54.373 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103998.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.103998.log.gz 2024-04-25T10:22:54.373 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59759.log 2024-04-25T10:22:54.374 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60283.log 2024-04-25T10:22:54.376 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59759.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.59759.log.gz 2024-04-25T10:22:54.377 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47385.log 2024-04-25T10:22:54.377 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.60283.log.gz 2024-04-25T10:22:54.378 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46497.log 2024-04-25T10:22:54.380 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47385.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47385.log.gz 2024-04-25T10:22:54.380 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48378.log/var/log/ceph/ceph-client.admin.46497.log: 2024-04-25T10:22:54.381 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46497.log.gz 2024-04-25T10:22:54.382 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58102.log 2024-04-25T10:22:54.383 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48378.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48378.log.gz 2024-04-25T10:22:54.384 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111179.log 2024-04-25T10:22:54.385 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58102.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.58102.log.gz 2024-04-25T10:22:54.385 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41289.log 2024-04-25T10:22:54.387 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111179.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.111179.log.gz 2024-04-25T10:22:54.388 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102461.log/var/log/ceph/ceph-client.admin.41289.log: 2024-04-25T10:22:54.388 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41289.log.gz 2024-04-25T10:22:54.389 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111095.log 2024-04-25T10:22:54.390 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102461.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102461.log.gz 2024-04-25T10:22:54.392 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.111095.log: --verbose -- /var/log/ceph/ceph-client.admin.45084.log 2024-04-25T10:22:54.392 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111095.log.gz 2024-04-25T10:22:54.393 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84595.log 2024-04-25T10:22:54.394 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45084.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45084.log.gz 2024-04-25T10:22:54.395 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.84595.log: /var/log/ceph/ceph-client.admin.55676.log 2024-04-25T10:22:54.396 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.84595.log.gz 2024-04-25T10:22:54.396 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39525.log 2024-04-25T10:22:54.398 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55676.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55676.log.gz 2024-04-25T10:22:54.398 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.39525.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64644.log 2024-04-25T10:22:54.399 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.39525.log.gz 2024-04-25T10:22:54.400 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112565.log 2024-04-25T10:22:54.401 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64644.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64644.log.gz 2024-04-25T10:22:54.402 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.112565.log: /var/log/ceph/ceph-client.admin.97690.log 2024-04-25T10:22:54.403 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112565.log.gz 2024-04-25T10:22:54.404 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112145.log 2024-04-25T10:22:54.406 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97690.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.97690.log.gz 2024-04-25T10:22:54.406 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63498.log 2024-04-25T10:22:54.406 INFO:teuthology.orchestra.run.smithi165.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112145.log.gz 2024-04-25T10:22:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107890.log 2024-04-25T10:22:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63498.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63498.log.gz 2024-04-25T10:22:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107890.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59161.log 2024-04-25T10:22:54.411 INFO:teuthology.orchestra.run.smithi165.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107890.log.gz 2024-04-25T10:22:54.412 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103788.log 2024-04-25T10:22:54.413 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59161.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.59161.log.gz 2024-04-25T10:22:54.414 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79817.log 2024-04-25T10:22:54.415 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103788.log: 91.4% -- replaced with /var/log/ceph/ceph-client.admin.103788.log.gz 2024-04-25T10:22:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105628.log 2024-04-25T10:22:54.416 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79817.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.79817.log.gz 2024-04-25T10:22:54.417 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40607.log 2024-04-25T10:22:54.418 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105628.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.105628.log.gz 2024-04-25T10:22:54.419 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106293.log 2024-04-25T10:22:54.420 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40607.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40607.log.gz 2024-04-25T10:22:54.422 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106293.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106293.log.gz 2024-04-25T10:22:54.422 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65638.log 2024-04-25T10:22:54.424 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87805.log 2024-04-25T10:22:54.425 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65638.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.65638.log.gz 2024-04-25T10:22:54.426 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48027.log 2024-04-25T10:22:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87805.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.87805.log.gz 2024-04-25T10:22:54.429 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48027.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105914.log 2024-04-25T10:22:54.429 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.48027.log.gz 2024-04-25T10:22:54.430 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85290.log 2024-04-25T10:22:54.431 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105914.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105914.log.gz 2024-04-25T10:22:54.432 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.0.85290.log: /var/log/ceph/ceph-client.admin.79523.log 2024-04-25T10:22:54.432 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85290.log.gz 2024-04-25T10:22:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43529.log 2024-04-25T10:22:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79523.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.79523.log.gz 2024-04-25T10:22:54.436 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83788.log/var/log/ceph/ceph-client.admin.43529.log: 2024-04-25T10:22:54.436 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43529.log.gz 2024-04-25T10:22:54.437 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110122.log 2024-04-25T10:22:54.438 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83788.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.83788.log.gz 2024-04-25T10:22:54.439 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57268.log 2024-04-25T10:22:54.440 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110122.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.110122.log.gz 2024-04-25T10:22:54.440 INFO:teuthology.orchestra.run.smithi172.stderr:/var/log/ceph/ceph-mon.b.log: 92.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-25T10:22:54.441 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39145.log 2024-04-25T10:22:54.441 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57268.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57268.log.gz 2024-04-25T10:22:54.443 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39145.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39145.log.gz 2024-04-25T10:22:54.443 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45630.log 2024-04-25T10:22:54.445 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84381.log 2024-04-25T10:22:54.446 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45630.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45630.log.gz 2024-04-25T10:22:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58468.log 2024-04-25T10:22:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84381.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84381.log.gz 2024-04-25T10:22:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87121.log 2024-04-25T10:22:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58468.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58468.log.gz 2024-04-25T10:22:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87121.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.87121.log.gz 2024-04-25T10:22:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74992.log 2024-04-25T10:22:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98621.log 2024-04-25T10:22:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74992.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.74992.log.gz 2024-04-25T10:22:54.456 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89595.log 2024-04-25T10:22:54.456 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.98621.log.gz 2024-04-25T10:22:54.457 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111767.log 2024-04-25T10:22:54.458 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89595.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.89595.log.gz 2024-04-25T10:22:54.460 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111767.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111767.log.gz 2024-04-25T10:22:54.460 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44856.log 2024-04-25T10:22:54.462 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112271.log 2024-04-25T10:22:54.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44856.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44856.log.gz 2024-04-25T10:22:54.463 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62142.log 2024-04-25T10:22:54.464 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112271.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.112271.log.gz 2024-04-25T10:22:54.465 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45909.log 2024-04-25T10:22:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62142.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.62142.log.gz 2024-04-25T10:22:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100348.log 2024-04-25T10:22:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45909.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45909.log.gz 2024-04-25T10:22:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105870.log 2024-04-25T10:22:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100348.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100348.log.gz 2024-04-25T10:22:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84805.log 2024-04-25T10:22:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105870.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.105870.log.gz 2024-04-25T10:22:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78342.log 2024-04-25T10:22:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84805.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.84805.log.gz 2024-04-25T10:22:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78342.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.78342.log.gz 2024-04-25T10:22:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78978.log 2024-04-25T10:22:54.477 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112761.log 2024-04-25T10:22:54.478 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78978.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78978.log.gz 2024-04-25T10:22:54.479 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56064.log 2024-04-25T10:22:54.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112761.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.112761.log.gz 2024-04-25T10:22:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44520.log 2024-04-25T10:22:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56064.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.56064.log.gz 2024-04-25T10:22:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44520.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44520.log.gz 2024-04-25T10:22:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56969.log 2024-04-25T10:22:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84465.log 2024-04-25T10:22:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56969.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56969.log.gz 2024-04-25T10:22:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33545.log 2024-04-25T10:22:54.488 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84465.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.84465.log.gz 2024-04-25T10:22:54.489 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112061.log 2024-04-25T10:22:54.490 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.33545.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.33545.log.gz 2024-04-25T10:22:54.491 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104805.log 2024-04-25T10:22:54.492 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112061.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112061.log.gz 2024-04-25T10:22:54.492 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-04-25T10:22:54.493 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104805.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104805.log.gz 2024-04-25T10:22:54.494 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108129.log 2024-04-25T10:22:54.501 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39609.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-04-25T10:22:54.501 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101700.log 2024-04-25T10:22:54.501 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108129.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.108129.log.gz 2024-04-25T10:22:54.501 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95157.log 2024-04-25T10:22:54.502 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101700.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.101700.log.gz 2024-04-25T10:22:54.502 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103623.log 2024-04-25T10:22:54.502 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95157.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.95157.log.gz 2024-04-25T10:22:54.502 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106744.log 2024-04-25T10:22:54.503 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103623.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.103623.log.gz 2024-04-25T10:22:54.503 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94129.log 2024-04-25T10:22:54.504 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106744.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.106744.log.gz 2024-04-25T10:22:54.505 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94129.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94129.log.gz 2024-04-25T10:22:54.506 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104165.log 2024-04-25T10:22:54.507 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39226.log 2024-04-25T10:22:54.509 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104165.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.104165.log.gz 2024-04-25T10:22:54.509 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48672.log 2024-04-25T10:22:54.510 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39226.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39226.log.gz 2024-04-25T10:22:54.512 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48672.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.48672.log.gz 2024-04-25T10:22:54.513 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102768.log 2024-04-25T10:22:54.514 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73418.log 2024-04-25T10:22:54.516 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102768.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.102768.log.gz 2024-04-25T10:22:54.516 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58426.log 2024-04-25T10:22:54.517 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73418.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.73418.log.gz 2024-04-25T10:22:54.518 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47091.log 2024-04-25T10:22:54.519 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58426.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58426.log.gz 2024-04-25T10:22:54.520 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90647.log 2024-04-25T10:22:54.520 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47091.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.47091.log.gz 2024-04-25T10:22:54.522 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90647.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.90647.log.gz 2024-04-25T10:22:54.522 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79859.log 2024-04-25T10:22:54.524 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60547.log 2024-04-25T10:22:54.525 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79859.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.79859.log.gz 2024-04-25T10:22:54.526 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84931.log 2024-04-25T10:22:54.527 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60547.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.60547.log.gz 2024-04-25T10:22:54.528 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64172.log 2024-04-25T10:22:54.529 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84931.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.84931.log.gz 2024-04-25T10:22:54.529 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43029.log 2024-04-25T10:22:54.530 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64172.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.64172.log.gz 2024-04-25T10:22:54.532 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43029.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43029.log.gz 2024-04-25T10:22:54.532 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48199.log 2024-04-25T10:22:54.534 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94381.log 2024-04-25T10:22:54.535 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48199.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48199.log.gz 2024-04-25T10:22:54.536 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112019.log 2024-04-25T10:22:54.537 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94381.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94381.log.gz 2024-04-25T10:22:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99377.log 2024-04-25T10:22:54.538 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112019.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.112019.log.gz 2024-04-25T10:22:54.540 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99377.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99377.log.gz 2024-04-25T10:22:54.541 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106634.log 2024-04-25T10:22:54.542 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104123.log 2024-04-25T10:22:54.543 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.106634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106634.log.gz 2024-04-25T10:22:54.543 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87913.log 2024-04-25T10:22:54.546 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104123.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.104123.log.gz 2024-04-25T10:22:54.546 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87955.log 2024-04-25T10:22:54.546 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.87913.log.gz 2024-04-25T10:22:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-25T10:22:54.549 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87955.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87955.log.gz 2024-04-25T10:22:54.549 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66403.log 2024-04-25T10:22:54.552 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.66403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60410.log 2024-04-25T10:22:54.553 INFO:teuthology.orchestra.run.smithi165.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66403.log.gz 2024-04-25T10:22:54.554 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46071.log 2024-04-25T10:22:54.556 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104993.log 2024-04-25T10:22:54.557 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46071.log: 87.5% 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46071.log.gz 2024-04-25T10:22:54.557 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60410.log.gz 2024-04-25T10:22:54.559 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47427.log 2024-04-25T10:22:54.559 INFO:teuthology.orchestra.run.smithi165.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-25T10:22:54.561 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104993.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88648.log 2024-04-25T10:22:54.562 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47427.log: -- replaced with /var/log/ceph/ceph-client.admin.104993.log.gz 2024-04-25T10:22:54.562 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47427.log.gz 2024-04-25T10:22:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110208.log 2024-04-25T10:22:54.564 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88648.log.gz 2024-04-25T10:22:54.565 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95031.log 2024-04-25T10:22:54.567 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110208.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110208.log.gz 2024-04-25T10:22:54.568 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62605.log 2024-04-25T10:22:54.568 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95031.log.gz 2024-04-25T10:22:54.569 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93147.log 2024-04-25T10:22:54.570 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.62605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62605.log.gz 2024-04-25T10:22:54.571 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102503.log 2024-04-25T10:22:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93147.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.93147.log.gz 2024-04-25T10:22:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42454.log 2024-04-25T10:22:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102503.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102503.log.gz 2024-04-25T10:22:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.44307.log 2024-04-25T10:22:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42454.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.42454.log.gz 2024-04-25T10:22:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79181.log 2024-04-25T10:22:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.2.44307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.44307.log.gz 2024-04-25T10:22:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47217.log 2024-04-25T10:22:54.579 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79181.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.79181.log.gz 2024-04-25T10:22:54.580 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47217.log: -- /var/log/ceph/ceph-client.admin.73754.log 2024-04-25T10:22:54.580 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.47217.log.gz 2024-04-25T10:22:54.581 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73880.log 2024-04-25T10:22:54.583 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73754.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.73754.log.gz 2024-04-25T10:22:54.583 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.73880.log: --verbose -- /var/log/ceph/ceph-client.admin.88984.log 2024-04-25T10:22:54.584 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73880.log.gz 2024-04-25T10:22:54.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88984.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88984.log.gz 2024-04-25T10:22:54.586 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48334.log 2024-04-25T10:22:54.588 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62937.log 2024-04-25T10:22:54.589 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48334.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48334.log.gz 2024-04-25T10:22:54.590 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109928.log 2024-04-25T10:22:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62937.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.62937.log.gz 2024-04-25T10:22:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65381.log 2024-04-25T10:22:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109928.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.109928.log.gz 2024-04-25T10:22:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79139.log 2024-04-25T10:22:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65381.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65381.log.gz 2024-04-25T10:22:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62268.log 2024-04-25T10:22:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79139.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79139.log.gz 2024-04-25T10:22:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79607.log 2024-04-25T10:22:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62268.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.62268.log.gz 2024-04-25T10:22:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85033.log 2024-04-25T10:22:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79607.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.79607.log.gz 2024-04-25T10:22:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87439.log 2024-04-25T10:22:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.85033.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.85033.log.gz 2024-04-25T10:22:54.603 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87439.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.87439.log.gz 2024-04-25T10:22:54.603 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40917.log 2024-04-25T10:22:54.605 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46029.log 2024-04-25T10:22:54.606 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40917.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.40917.log.gz 2024-04-25T10:22:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111137.log 2024-04-25T10:22:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46029.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.46029.log.gz 2024-04-25T10:22:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47175.log 2024-04-25T10:22:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111137.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.111137.log.gz 2024-04-25T10:22:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82741.log/var/log/ceph/ceph-client.admin.47175.log: 2024-04-25T10:22:54.611 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.47175.log.gz 2024-04-25T10:22:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84637.log 2024-04-25T10:22:54.613 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82741.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.82741.log.gz 2024-04-25T10:22:54.614 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.84637.log: --verbose -- /var/log/ceph/ceph-client.admin.103082.log 2024-04-25T10:22:54.614 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.84637.log.gz 2024-04-25T10:22:54.615 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108016.log 2024-04-25T10:22:54.617 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103082.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.103082.log.gz 2024-04-25T10:22:54.618 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.108016.log: -- /var/log/ceph/ceph-client.admin.62016.log 2024-04-25T10:22:54.618 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108016.log.gz 2024-04-25T10:22:54.619 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106166.log 2024-04-25T10:22:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62016.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.62016.log.gz 2024-04-25T10:22:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105786.log 2024-04-25T10:22:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106166.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.106166.log.gz 2024-04-25T10:22:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63456.log 2024-04-25T10:22:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105786.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.105786.log.gz 2024-04-25T10:22:54.625 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79733.log 2024-04-25T10:22:54.626 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63456.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.63456.log.gz 2024-04-25T10:22:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88732.log 2024-04-25T10:22:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79733.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.79733.log.gz 2024-04-25T10:22:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40775.log 2024-04-25T10:22:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88732.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.88732.log.gz 2024-04-25T10:22:54.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40775.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.40775.log.gz 2024-04-25T10:22:54.631 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83578.log 2024-04-25T10:22:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85286.log 2024-04-25T10:22:54.634 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83578.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.83578.log.gz 2024-04-25T10:22:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87397.log 2024-04-25T10:22:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.85286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.85286.log.gz 2024-04-25T10:22:54.636 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98537.log 2024-04-25T10:22:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87397.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.87397.log.gz 2024-04-25T10:22:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.98537.log: /var/log/ceph/ceph-client.admin.64915.log 2024-04-25T10:22:54.639 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98537.log.gz 2024-04-25T10:22:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92800.log 2024-04-25T10:22:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64915.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64915.log.gz 2024-04-25T10:22:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82980.log/var/log/ceph/ceph-client.admin.92800.log: 2024-04-25T10:22:54.643 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.92800.log.gz 2024-04-25T10:22:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74654.log 2024-04-25T10:22:54.645 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82980.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.82980.log.gz 2024-04-25T10:22:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.74654.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.94987.log 2024-04-25T10:22:54.646 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74654.log.gz 2024-04-25T10:22:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101490.log 2024-04-25T10:22:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94987.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.94987.log.gz 2024-04-25T10:22:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83704.log/var/log/ceph/ceph-client.admin.101490.log: 2024-04-25T10:22:54.650 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101490.log.gz 2024-04-25T10:22:54.651 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103419.log 2024-04-25T10:22:54.652 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83704.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83704.log.gz 2024-04-25T10:22:54.653 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58732.log/var/log/ceph/ceph-client.0.103419.log: 2024-04-25T10:22:54.653 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103419.log.gz 2024-04-25T10:22:54.654 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89254.log 2024-04-25T10:22:54.656 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58732.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.58732.log.gz 2024-04-25T10:22:54.657 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.89254.log: -- /var/log/ceph/ceph-client.admin.62769.log 2024-04-25T10:22:54.657 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.89254.log.gz 2024-04-25T10:22:54.658 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65764.log 2024-04-25T10:22:54.659 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62769.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.62769.log.gz 2024-04-25T10:22:54.660 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42903.log 2024-04-25T10:22:54.661 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65764.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65764.log.gz 2024-04-25T10:22:54.662 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89679.log 2024-04-25T10:22:54.663 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42903.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42903.log.gz 2024-04-25T10:22:54.664 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65258.log 2024-04-25T10:22:54.664 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.89679.log.gz 2024-04-25T10:22:54.665 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95493.log 2024-04-25T10:22:54.666 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65258.log: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.65258.log.gz 2024-04-25T10:22:54.667 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102587.log 2024-04-25T10:22:54.668 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95493.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.95493.log.gz 2024-04-25T10:22:54.669 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66311.log 2024-04-25T10:22:54.670 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102587.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102587.log.gz 2024-04-25T10:22:54.671 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45462.log 2024-04-25T10:22:54.671 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.66311.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.66311.log.gz 2024-04-25T10:22:54.672 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41517.log 2024-04-25T10:22:54.673 INFO:teuthology.orchestra.run.smithi172.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-25T10:22:54.674 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45462.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45462.log.gz 2024-04-25T10:22:54.675 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101658.log/var/log/ceph/ceph-client.admin.41517.log: 2024-04-25T10:22:54.675 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.41517.log.gz 2024-04-25T10:22:54.676 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105501.log 2024-04-25T10:22:54.678 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101658.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.101658.log.gz 2024-04-25T10:22:54.679 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.105501.log: --verbose -- /var/log/ceph/ceph-client.admin.74824.log 2024-04-25T10:22:54.679 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.105501.log.gz 2024-04-25T10:22:54.680 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55802.log 2024-04-25T10:22:54.681 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74824.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.74824.log.gz 2024-04-25T10:22:54.682 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.55802.log: -- /var/log/ceph/ceph-client.admin.55634.log 2024-04-25T10:22:54.683 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.55802.log.gz 2024-04-25T10:22:54.684 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77349.log 2024-04-25T10:22:54.685 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55634.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.55634.log.gz 2024-04-25T10:22:54.685 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77349.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43365.log 2024-04-25T10:22:54.687 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.77349.log.gz 2024-04-25T10:22:54.688 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43197.log 2024-04-25T10:22:54.689 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43365.log: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.43365.log.gz 2024-04-25T10:22:54.690 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110166.log 2024-04-25T10:22:54.690 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43197.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43197.log.gz 2024-04-25T10:22:54.691 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65466.log 2024-04-25T10:22:54.692 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110166.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.110166.log.gz 2024-04-25T10:22:54.694 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65466.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.65466.log.gz 2024-04-25T10:22:54.694 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43071.log 2024-04-25T10:22:54.696 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92964.log 2024-04-25T10:22:54.697 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43071.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43071.log.gz 2024-04-25T10:22:54.698 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106040.log 2024-04-25T10:22:54.703 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92964.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92964.log.gz 2024-04-25T10:22:54.703 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46197.log 2024-04-25T10:22:54.703 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106040.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.106040.log.gz 2024-04-25T10:22:54.703 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46197.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46197.log.gz 2024-04-25T10:22:54.703 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84296.log 2024-04-25T10:22:54.704 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94476.log 2024-04-25T10:22:54.705 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84296.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.84296.log.gz 2024-04-25T10:22:54.706 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74782.log 2024-04-25T10:22:54.707 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94476.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.94476.log.gz 2024-04-25T10:22:54.708 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41415.log 2024-04-25T10:22:54.708 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74782.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74782.log.gz 2024-04-25T10:22:54.709 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60799.log 2024-04-25T10:22:54.711 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41415.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.41415.log.gz 2024-04-25T10:22:54.711 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106377.log 2024-04-25T10:22:54.712 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60799.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60799.log.gz 2024-04-25T10:22:54.713 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64130.log 2024-04-25T10:22:54.715 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106377.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.106377.log.gz 2024-04-25T10:22:54.715 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57182.log 2024-04-25T10:22:54.716 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64130.log.gz 2024-04-25T10:22:54.717 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44688.log 2024-04-25T10:22:54.719 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57182.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.57182.log.gz 2024-04-25T10:22:54.719 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.44688.log: -- /var/log/ceph/ceph-client.admin.48462.log 2024-04-25T10:22:54.720 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44688.log.gz 2024-04-25T10:22:54.721 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111851.log 2024-04-25T10:22:54.722 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48462.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.48462.log.gz 2024-04-25T10:22:54.723 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47691.log 2024-04-25T10:22:54.723 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111851.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111851.log.gz 2024-04-25T10:22:54.725 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47691.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.47691.log.gz 2024-04-25T10:22:54.726 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65722.log 2024-04-25T10:22:54.727 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60368.log 2024-04-25T10:22:54.728 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65722.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.65722.log.gz 2024-04-25T10:22:54.729 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48250.log 2024-04-25T10:22:54.730 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60368.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60368.log.gz 2024-04-25T10:22:54.731 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57226.log 2024-04-25T10:22:54.732 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48250.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.48250.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89763.log 2024-04-25T10:22:54.732 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-25T10:22:54.733 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57226.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.57226.log.gz 2024-04-25T10:22:54.735 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.89763.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.79985.log 2024-04-25T10:22:54.735 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.89763.log.gz 2024-04-25T10:22:54.735 INFO:teuthology.orchestra.run.smithi172.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-04-25T10:22:54.737 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79985.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.79985.log.gz 2024-04-25T10:22:54.737 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89977.log 2024-04-25T10:22:54.739 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61345.log 2024-04-25T10:22:54.740 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89977.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.89977.log.gz 2024-04-25T10:22:54.741 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99079.log 2024-04-25T10:22:54.742 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61345.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.61345.log.gz 2024-04-25T10:22:54.743 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65178.log/var/log/ceph/ceph-client.admin.99079.log: 2024-04-25T10:22:54.743 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99079.log.gz 2024-04-25T10:22:54.744 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111492.log 2024-04-25T10:22:54.745 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65178.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65178.log.gz 2024-04-25T10:22:54.747 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.111492.log: /var/log/ceph/ceph-client.admin.64831.log 2024-04-25T10:22:54.747 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111492.log.gz 2024-04-25T10:22:54.748 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82699.log 2024-04-25T10:22:54.749 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64831.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.64831.log.gz 2024-04-25T10:22:54.750 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42651.log/var/log/ceph/ceph-client.admin.82699.log: 2024-04-25T10:22:54.751 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.82699.log.gz 2024-04-25T10:22:54.751 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111809.log 2024-04-25T10:22:54.753 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42651.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42651.log.gz 2024-04-25T10:22:54.753 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66058.log 2024-04-25T10:22:54.754 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111809.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111809.log.gz 2024-04-25T10:22:54.755 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81991.log 2024-04-25T10:22:54.757 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.66058.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.66058.log.gz 2024-04-25T10:22:54.757 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.81991.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98995.log 2024-04-25T10:22:54.759 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.81991.log.gz 2024-04-25T10:22:54.759 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87655.log 2024-04-25T10:22:54.762 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98995.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94213.log/var/log/ceph/ceph-client.admin.87655.log: 2024-04-25T10:22:54.762 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98995.log.gz 2024-04-25T10:22:54.762 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.87655.log.gz 2024-04-25T10:22:54.762 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107493.log 2024-04-25T10:22:54.765 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94213.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.94213.log.gz 2024-04-25T10:22:54.766 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78571.log 2024-04-25T10:22:54.766 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107493.log.gz 2024-04-25T10:22:54.767 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78571.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.78571.log.gz 2024-04-25T10:22:54.768 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78697.log 2024-04-25T10:22:54.769 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56190.log 2024-04-25T10:22:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78697.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.78697.log.gz 2024-04-25T10:22:54.771 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112103.log 2024-04-25T10:22:54.772 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56190.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56190.log.gz 2024-04-25T10:22:54.773 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80027.log 2024-04-25T10:22:54.774 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112103.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.112103.log.gz 2024-04-25T10:22:54.775 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43323.log 2024-04-25T10:22:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.80027.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.80027.log.gz 2024-04-25T10:22:54.777 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99676.log 2024-04-25T10:22:54.778 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43323.log.gz 2024-04-25T10:22:54.779 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99676.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99676.log.gz 2024-04-25T10:22:54.780 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45420.log 2024-04-25T10:22:54.781 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99504.log 2024-04-25T10:22:54.783 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45420.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45420.log.gz 2024-04-25T10:22:54.783 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39395.log 2024-04-25T10:22:54.784 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99504.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.99504.log.gz 2024-04-25T10:22:54.785 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108682.log 2024-04-25T10:22:54.787 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39395.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.39395.log.gz 2024-04-25T10:22:54.787 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79224.log 2024-04-25T10:22:54.788 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108682.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108682.log.gz 2024-04-25T10:22:54.789 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105203.log 2024-04-25T10:22:54.790 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79224.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.79224.log.gz 2024-04-25T10:22:54.791 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83064.log 2024-04-25T10:22:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105203.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.105203.log.gz 2024-04-25T10:22:54.792 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78132.log 2024-04-25T10:22:54.795 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83064.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83064.log.gz 2024-04-25T10:22:54.795 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42453.log 2024-04-25T10:22:54.795 INFO:teuthology.orchestra.run.smithi165.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78132.log.gz 2024-04-25T10:22:54.796 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.84215.log 2024-04-25T10:22:54.797 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42453.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.42453.log.gz 2024-04-25T10:22:54.798 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104511.log 2024-04-25T10:22:54.799 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84215.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.84215.log.gz 2024-04-25T10:22:54.800 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43155.log 2024-04-25T10:22:54.801 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104511.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.104511.log.gz 2024-04-25T10:22:54.801 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63172.log 2024-04-25T10:22:54.803 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.43155.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43155.log.gz 2024-04-25T10:22:54.804 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83452.log 2024-04-25T10:22:54.804 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63172.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.63172.log.gz 2024-04-25T10:22:54.806 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83452.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.83452.log.gz 2024-04-25T10:22:54.806 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47301.log 2024-04-25T10:22:54.808 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65339.log 2024-04-25T10:22:54.809 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47301.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47301.log.gz 2024-04-25T10:22:54.810 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83242.log 2024-04-25T10:22:54.811 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65339.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.65339.log.gz 2024-04-25T10:22:54.812 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107848.log 2024-04-25T10:22:54.812 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83242.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83242.log.gz 2024-04-25T10:22:54.813 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55299.log 2024-04-25T10:22:54.814 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107848.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.107848.log.gz 2024-04-25T10:22:54.815 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60757.log 2024-04-25T10:22:54.817 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55299.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.55299.log.gz 2024-04-25T10:22:54.818 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60757.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.60757.log.gz 2024-04-25T10:22:54.818 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40649.log 2024-04-25T10:22:54.820 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82573.log 2024-04-25T10:22:54.821 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40649.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.40649.log.gz 2024-04-25T10:22:54.822 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61520.log 2024-04-25T10:22:54.822 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82573.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82573.log.gz 2024-04-25T10:22:54.823 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98783.log 2024-04-25T10:22:54.824 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61520.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61520.log.gz 2024-04-25T10:22:54.826 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98783.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98783.log.gz 2024-04-25T10:22:54.826 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94255.log 2024-04-25T10:22:54.828 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89338.log 2024-04-25T10:22:54.829 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94255.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94255.log.gz 2024-04-25T10:22:54.830 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105298.log 2024-04-25T10:22:54.831 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89338.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89338.log.gz 2024-04-25T10:22:54.831 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98147.log 2024-04-25T10:22:54.832 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105298.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.105298.log.gz 2024-04-25T10:22:54.834 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98147.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98147.log.gz 2024-04-25T10:22:54.834 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65680.log 2024-04-25T10:22:54.837 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65680.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62184.log 2024-04-25T10:22:54.837 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.65680.log.gz 2024-04-25T10:22:54.838 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88428.log 2024-04-25T10:22:54.839 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62184.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.62184.log.gz 2024-04-25T10:22:54.840 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88428.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88428.log.gz 2024-04-25T10:22:54.841 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46239.log 2024-04-25T10:22:54.843 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112229.log 2024-04-25T10:22:54.844 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46239.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46239.log.gz 2024-04-25T10:22:54.844 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98063.log 2024-04-25T10:22:54.845 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112229.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112229.log.gz 2024-04-25T10:22:54.846 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78384.log 2024-04-25T10:22:54.847 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98063.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.98063.log.gz 2024-04-25T10:22:54.848 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101868.log 2024-04-25T10:22:54.849 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78384.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.78384.log.gz 2024-04-25T10:22:54.850 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95451.log 2024-04-25T10:22:54.851 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101868.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.101868.log.gz 2024-04-25T10:22:54.852 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103232.log/var/log/ceph/ceph-client.admin.95451.log: 2024-04-25T10:22:54.852 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95451.log.gz 2024-04-25T10:22:54.853 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108332.log 2024-04-25T10:22:54.854 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103232.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.103232.log.gz 2024-04-25T10:22:54.856 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108332.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108332.log.gz 2024-04-25T10:22:54.856 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93798.log 2024-04-25T10:22:54.858 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108375.log 2024-04-25T10:22:54.859 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93798.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.93798.log.gz 2024-04-25T10:22:54.860 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94339.log 2024-04-25T10:22:54.861 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108375.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.108375.log.gz 2024-04-25T10:22:54.861 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64999.log 2024-04-25T10:22:54.862 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94339.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94339.log.gz 2024-04-25T10:22:54.864 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64999.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.64999.log.gz 2024-04-25T10:22:54.864 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57478.log 2024-04-25T10:22:54.866 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78468.log 2024-04-25T10:22:54.867 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57478.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.57478.log.gz 2024-04-25T10:22:54.868 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98579.log 2024-04-25T10:22:54.869 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78468.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78468.log.gz 2024-04-25T10:22:54.869 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99037.log 2024-04-25T10:22:54.870 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98579.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98579.log.gz 2024-04-25T10:22:54.872 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99037.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99037.log.gz 2024-04-25T10:22:54.872 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61689.log 2024-04-25T10:22:54.874 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108768.log 2024-04-25T10:22:54.876 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61689.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.61689.log.gz 2024-04-25T10:22:54.876 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41001.log 2024-04-25T10:22:54.876 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108768.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108768.log.gz 2024-04-25T10:22:54.877 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42116.log 2024-04-25T10:22:54.879 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41001.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.41001.log.gz 2024-04-25T10:22:54.879 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.42116.log: /var/log/ceph/ceph-client.admin.64434.log 2024-04-25T10:22:54.880 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.42116.log.gz 2024-04-25T10:22:54.881 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92842.log 2024-04-25T10:22:54.882 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64434.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.64434.log.gz 2024-04-25T10:22:54.883 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.92842.log: --verbose -- /var/log/ceph/ceph-client.admin.105544.log 2024-04-25T10:22:54.884 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.92842.log.gz 2024-04-25T10:22:54.885 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74006.log 2024-04-25T10:22:54.886 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105544.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.105544.log.gz 2024-04-25T10:22:54.887 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.74006.log: -- /var/log/ceph/ceph-client.admin.41783.log 2024-04-25T10:22:54.887 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74006.log.gz 2024-04-25T10:22:54.888 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56419.log 2024-04-25T10:22:54.890 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41783.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.41783.log.gz 2024-04-25T10:22:54.891 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56419.log: /var/log/ceph/ceph-client.admin.106335.log 2024-04-25T10:22:54.891 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.56419.log.gz 2024-04-25T10:22:54.892 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94638.log 2024-04-25T10:22:54.893 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106335.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.106335.log.gz 2024-04-25T10:22:54.894 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73964.log/var/log/ceph/ceph-client.admin.94638.log: 2024-04-25T10:22:54.895 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94638.log.gz 2024-04-25T10:22:54.896 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75515.log 2024-04-25T10:22:54.897 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73964.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.73964.log.gz 2024-04-25T10:22:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88522.log 2024-04-25T10:22:54.898 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.75515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.75515.log.gz 2024-04-25T10:22:54.899 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107974.log 2024-04-25T10:22:54.901 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88522.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.88522.log.gz 2024-04-25T10:22:54.901 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.107974.log: -- /var/log/ceph/ceph-client.admin.59095.log 2024-04-25T10:22:54.902 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107974.log.gz 2024-04-25T10:22:54.907 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64686.log 2024-04-25T10:22:54.908 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59095.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.59095.log.gz 2024-04-25T10:22:54.908 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64686.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58360.log 2024-04-25T10:22:54.908 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.64686.log.gz 2024-04-25T10:22:54.908 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83620.log 2024-04-25T10:22:54.909 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58360.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58360.log.gz 2024-04-25T10:22:54.910 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107409.log 88.1% 2024-04-25T10:22:54.910 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.83620.log.gz 2024-04-25T10:22:54.911 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77565.log 2024-04-25T10:22:54.913 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107409.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107409.log.gz 2024-04-25T10:22:54.914 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77565.log: /var/log/ceph/ceph-client.admin.77912.log 2024-04-25T10:22:54.914 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77565.log.gz 2024-04-25T10:22:54.915 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109873.log 2024-04-25T10:22:54.917 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77912.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77912.log.gz 2024-04-25T10:22:54.917 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93326.log 2024-04-25T10:22:54.918 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.109873.log.gz 2024-04-25T10:22:54.919 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83368.log 2024-04-25T10:22:54.920 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93326.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93326.log.gz 2024-04-25T10:22:54.921 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62853.log/var/log/ceph/ceph-client.admin.83368.log: 2024-04-25T10:22:54.921 INFO:teuthology.orchestra.run.smithi165.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.83368.log.gz 2024-04-25T10:22:54.922 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99844.log 2024-04-25T10:22:54.924 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62853.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.62853.log.gz 2024-04-25T10:22:54.925 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.99844.log: /var/log/ceph/ceph-client.admin.78174.log 2024-04-25T10:22:54.925 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99844.log.gz 2024-04-25T10:22:54.926 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87481.log 2024-04-25T10:22:54.927 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78174.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.78174.log.gz 2024-04-25T10:22:54.929 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87481.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.87481.log.gz 2024-04-25T10:22:54.929 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60715.log 2024-04-25T10:22:54.931 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100875.log 2024-04-25T10:22:54.932 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60715.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.60715.log.gz 2024-04-25T10:22:54.932 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102810.log 2024-04-25T10:22:54.934 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100875.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.100875.log.gz 2024-04-25T10:22:54.934 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.102810.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.88386.log 2024-04-25T10:22:54.935 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.102810.log.gz 2024-04-25T10:22:54.937 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88386.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.88386.log.gz 2024-04-25T10:22:54.937 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93630.log 2024-04-25T10:22:54.939 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.106630.log 2024-04-25T10:22:54.940 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93630.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.93630.log.gz 2024-04-25T10:22:54.941 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64746.log 2024-04-25T10:22:54.941 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.106630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.106630.log.gz 2024-04-25T10:22:54.942 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82885.log 2024-04-25T10:22:54.944 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64746.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64746.log.gz 2024-04-25T10:22:54.944 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82885.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104249.log 2024-04-25T10:22:54.945 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.82885.log.gz 2024-04-25T10:22:54.946 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106082.log 2024-04-25T10:22:54.947 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104249.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.104249.log.gz 2024-04-25T10:22:54.948 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56148.log/var/log/ceph/ceph-client.admin.106082.log: 2024-04-25T10:22:54.948 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.106082.log.gz 2024-04-25T10:22:54.949 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74950.log 2024-04-25T10:22:54.951 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56148.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.56148.log.gz 2024-04-25T10:22:54.951 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100724.log 2024-04-25T10:22:54.953 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74950.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47943.log/var/log/ceph/ceph-client.admin.100724.log: 2024-04-25T10:22:54.953 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.74950.log.gz 2024-04-25T10:22:54.954 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.100724.log.gz 2024-04-25T10:22:54.956 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47943.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47943.log.gz 2024-04-25T10:22:54.956 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73628.log 2024-04-25T10:22:54.958 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47133.log 2024-04-25T10:22:54.959 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73628.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73628.log.gz 2024-04-25T10:22:54.960 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97582.log 2024-04-25T10:22:54.961 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47133.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47133.log.gz 2024-04-25T10:22:54.961 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88344.log 2024-04-25T10:22:54.962 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97582.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97582.log.gz 2024-04-25T10:22:54.963 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101186.log/var/log/ceph/ceph-client.admin.88344.log: 2024-04-25T10:22:54.964 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.88344.log.gz 2024-04-25T10:22:54.965 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110424.log 2024-04-25T10:22:54.966 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101186.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.101186.log.gz 2024-04-25T10:22:54.967 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.110424.log: /var/log/ceph/ceph-client.admin.101228.log 2024-04-25T10:22:54.968 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.110424.log.gz 2024-04-25T10:22:54.969 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102078.log 2024-04-25T10:22:54.970 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101228.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101228.log.gz 2024-04-25T10:22:54.971 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87355.log 2024-04-25T10:22:54.972 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102078.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.102078.log.gz 2024-04-25T10:22:54.973 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82189.log 2024-04-25T10:22:54.974 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87355.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.87355.log.gz 2024-04-25T10:22:54.975 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48504.log 2024-04-25T10:22:54.975 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82189.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82189.log.gz 2024-04-25T10:22:54.976 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62058.log 2024-04-25T10:22:54.977 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48504.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.48504.log.gz 2024-04-25T10:22:54.979 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62058.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62058.log.gz 2024-04-25T10:22:54.979 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83536.log 2024-04-25T10:22:54.981 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93006.log 2024-04-25T10:22:54.982 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83536.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.83536.log.gz 2024-04-25T10:22:54.983 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82123.log 2024-04-25T10:22:54.983 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93006.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93006.log.gz 2024-04-25T10:22:54.984 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39353.log 2024-04-25T10:22:54.986 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82123.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.82123.log.gz 2024-04-25T10:22:54.987 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.39353.log: --verbose -- /var/log/ceph/ceph-client.admin.74696.log 2024-04-25T10:22:54.987 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39353.log.gz 2024-04-25T10:22:54.988 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62418.log 2024-04-25T10:22:54.989 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74696.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74696.log.gz 2024-04-25T10:22:54.991 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.62418.log: /var/log/ceph/ceph-client.admin.93189.log 2024-04-25T10:22:54.991 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.62418.log.gz 2024-04-25T10:22:54.992 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93368.log 2024-04-25T10:22:54.993 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93189.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93189.log.gz 2024-04-25T10:22:54.994 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.93368.log: /var/log/ceph/ceph-client.admin.103124.log 2024-04-25T10:22:54.995 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93368.log.gz 2024-04-25T10:22:54.995 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94764.log 2024-04-25T10:22:54.997 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103124.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103124.log.gz 2024-04-25T10:22:54.998 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.94764.log: -- /var/log/ceph/ceph-client.admin.101364.log 2024-04-25T10:22:54.998 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.94764.log.gz 2024-04-25T10:22:54.999 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112679.log 2024-04-25T10:22:55.001 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101364.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101364.log.gz 2024-04-25T10:22:55.001 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83998.log 2024-04-25T10:22:55.002 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112679.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112679.log.gz 2024-04-25T10:22:55.003 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108417.log 2024-04-25T10:22:55.004 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83998.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83998.log.gz 2024-04-25T10:22:55.005 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.108417.log: -- /var/log/ceph/ceph-client.admin.97820.log 2024-04-25T10:22:55.005 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108417.log.gz 2024-04-25T10:22:55.006 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40959.log 2024-04-25T10:22:55.008 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97820.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.97820.log.gz 2024-04-25T10:22:55.009 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92758.log 2024-04-25T10:22:55.009 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.40959.log.gz 2024-04-25T10:22:55.010 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41780.log 2024-04-25T10:22:55.012 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92758.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.92758.log.gz 2024-04-25T10:22:55.013 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.41780.log: -- /var/log/ceph/ceph-client.admin.39049.log 2024-04-25T10:22:55.013 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.41780.log.gz 2024-04-25T10:22:55.014 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46371.log 2024-04-25T10:22:55.016 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39049.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.39049.log.gz 2024-04-25T10:22:55.017 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.46371.log: --verbose -- /var/log/ceph/ceph-client.admin.73544.log 2024-04-25T10:22:55.017 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46371.log.gz 2024-04-25T10:22:55.018 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73156.log 2024-04-25T10:22:55.019 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73544.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.73544.log.gz 2024-04-25T10:22:55.020 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73156.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63850.log 2024-04-25T10:22:55.021 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.73156.log.gz 2024-04-25T10:22:55.022 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108503.log 2024-04-25T10:22:55.024 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63850.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.63850.log.gz 2024-04-25T10:22:55.024 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94806.log 2024-04-25T10:22:55.024 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.108503.log.gz 2024-04-25T10:22:55.025 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94297.log 2024-04-25T10:22:55.027 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94806.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94806.log.gz 2024-04-25T10:22:55.028 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.94297.log: -- /var/log/ceph/ceph-client.admin.84763.log 2024-04-25T10:22:55.028 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94297.log.gz 2024-04-25T10:22:55.029 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107722.log 2024-04-25T10:22:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.84763.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.84763.log.gz 2024-04-25T10:22:55.031 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.107722.log: /var/log/ceph/ceph-client.admin.94680.log 2024-04-25T10:22:55.032 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107722.log.gz 2024-04-25T10:22:55.033 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93672.log 2024-04-25T10:22:55.034 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94680.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94680.log.gz 2024-04-25T10:22:55.035 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90566.log 2024-04-25T10:22:55.036 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93672.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.93672.log.gz 2024-04-25T10:22:55.037 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46881.log 2024-04-25T10:22:55.037 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.90566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90566.log.gz 2024-04-25T10:22:55.038 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57054.log 2024-04-25T10:22:55.039 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46881.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46881.log.gz 2024-04-25T10:22:55.040 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74265.log 2024-04-25T10:22:55.041 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57054.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.57054.log.gz 2024-04-25T10:22:55.042 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74265.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74265.log.gz 2024-04-25T10:22:55.043 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73796.log 2024-04-25T10:22:55.045 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101448.log 2024-04-25T10:22:55.046 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73796.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.73796.log.gz 2024-04-25T10:22:55.046 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39779.log 2024-04-25T10:22:55.047 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101448.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101448.log.gz 2024-04-25T10:22:55.048 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78426.log 2024-04-25T10:22:55.049 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39779.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.39779.log.gz 2024-04-25T10:22:55.051 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78426.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78426.log.gz 2024-04-25T10:22:55.051 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79649.log 2024-04-25T10:22:55.053 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48155.log 2024-04-25T10:22:55.054 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79649.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.79649.log.gz 2024-04-25T10:22:55.055 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88123.log 2024-04-25T10:22:55.056 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48155.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48155.log.gz 2024-04-25T10:22:55.056 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59053.log 2024-04-25T10:22:55.057 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88123.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.88123.log.gz 2024-04-25T10:22:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59053.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.59053.log.gz 2024-04-25T10:22:55.059 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90019.log 2024-04-25T10:22:55.061 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105956.log 2024-04-25T10:22:55.062 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90019.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90019.log.gz 2024-04-25T10:22:55.063 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83830.log 2024-04-25T10:22:55.064 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105956.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105956.log.gz 2024-04-25T10:22:55.065 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46413.log 2024-04-25T10:22:55.065 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83830.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.83830.log.gz 2024-04-25T10:22:55.066 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99634.log 2024-04-25T10:22:55.067 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46413.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46413.log.gz 2024-04-25T10:22:55.068 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59881.log 2024-04-25T10:22:55.069 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99634.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.99634.log.gz 2024-04-25T10:22:55.071 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59881.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.59881.log.gz 2024-04-25T10:22:55.072 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103166.log 2024-04-25T10:22:55.073 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77217.log 2024-04-25T10:22:55.075 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.103166.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.103166.log.gz 2024-04-25T10:22:55.076 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39821.log 2024-04-25T10:22:55.076 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.77217.log.gz 2024-04-25T10:22:55.077 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77786.log 2024-04-25T10:22:55.078 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39821.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39821.log.gz 2024-04-25T10:22:55.079 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58854.log 2024-04-25T10:22:55.080 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77786.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.77786.log.gz 2024-04-25T10:22:55.081 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78216.log 2024-04-25T10:22:55.082 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58854.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58854.log.gz 2024-04-25T10:22:55.084 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.78216.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.83956.log 2024-04-25T10:22:55.084 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.78216.log.gz 2024-04-25T10:22:55.085 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78048.log 2024-04-25T10:22:55.086 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83956.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.83956.log.gz 2024-04-25T10:22:55.087 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101994.log/var/log/ceph/ceph-client.admin.78048.log: 2024-04-25T10:22:55.088 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.78048.log.gz 2024-04-25T10:22:55.088 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95601.log 2024-04-25T10:22:55.090 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101994.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.101994.log.gz 2024-04-25T10:22:55.091 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.95601.log: -- /var/log/ceph/ceph-client.admin.41779.log 2024-04-25T10:22:55.091 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95601.log.gz 2024-04-25T10:22:55.092 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39737.log 2024-04-25T10:22:55.094 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41779.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.41779.log.gz 2024-04-25T10:22:55.094 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92482.log 2024-04-25T10:22:55.095 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39737.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.39737.log.gz 2024-04-25T10:22:55.096 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79775.log 2024-04-25T10:22:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92482.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.92482.log.gz 2024-04-25T10:22:55.098 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89637.log 2024-04-25T10:22:55.099 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79775.log.gz 2024-04-25T10:22:55.100 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108459.log 2024-04-25T10:22:55.101 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89637.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.89637.log.gz 2024-04-25T10:22:55.102 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.108459.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102726.log 2024-04-25T10:22:55.102 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108459.log.gz 2024-04-25T10:22:55.103 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104385.log 2024-04-25T10:22:55.104 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102726.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.102726.log.gz 2024-04-25T10:22:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104385.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.104385.log.gz 2024-04-25T10:22:55.106 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.109273.log 2024-04-25T10:22:55.108 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45126.log 2024-04-25T10:22:55.109 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.109273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.109273.log.gz 2024-04-25T10:22:55.110 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.45126.log: -- /var/log/ceph/ceph-client.admin.110080.log 2024-04-25T10:22:55.111 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45126.log.gz 2024-04-25T10:22:55.112 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46923.log 2024-04-25T10:22:55.113 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110080.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110080.log.gz 2024-04-25T10:22:55.113 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.46923.log: /var/log/ceph/ceph-client.admin.58987.log 2024-04-25T10:22:55.114 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46923.log.gz 2024-04-25T10:22:55.115 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72629.log 2024-04-25T10:22:55.117 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58987.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58987.log.gz 2024-04-25T10:22:55.118 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.72629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88942.log 2024-04-25T10:22:55.118 INFO:teuthology.orchestra.run.smithi165.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.72629.log.gz 2024-04-25T10:22:55.119 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100096.log 2024-04-25T10:22:55.121 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88942.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88942.log.gz 2024-04-25T10:22:55.121 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.100096.log: /var/log/ceph/ceph-client.admin.83148.log 2024-04-25T10:22:55.122 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100096.log.gz 2024-04-25T10:22:55.123 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48111.log 2024-04-25T10:22:55.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.83148.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83148.log.gz 2024-04-25T10:22:55.125 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109668.log 2024-04-25T10:22:55.126 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.48111.log.gz 2024-04-25T10:22:55.126 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47511.log 2024-04-25T10:22:55.128 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109668.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.109668.log.gz 2024-04-25T10:22:55.129 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.47511.log: -- /var/log/ceph/ceph-client.admin.88858.log 2024-04-25T10:22:55.129 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47511.log.gz 2024-04-25T10:22:55.130 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102257.log 2024-04-25T10:22:55.132 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88858.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.88858.log.gz 2024-04-25T10:22:55.133 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.102257.log: -- /var/log/ceph/ceph-client.admin.60080.log 2024-04-25T10:22:55.133 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102257.log.gz 2024-04-25T10:22:55.134 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87871.log 2024-04-25T10:22:55.135 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60080.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.60080.log.gz 2024-04-25T10:22:55.136 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.87871.log: /var/log/ceph/ceph-client.admin.63171.log 2024-04-25T10:22:55.137 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.87871.log.gz 2024-04-25T10:22:55.138 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83284.log 2024-04-25T10:22:55.139 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63171.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.63171.log.gz 2024-04-25T10:22:55.140 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.83284.log: /var/log/ceph/ceph-client.admin.104721.log 2024-04-25T10:22:55.140 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83284.log.gz 2024-04-25T10:22:55.141 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33645.log 2024-04-25T10:22:55.143 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104721.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.104721.log.gz 2024-04-25T10:22:55.143 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82507.log 2024-04-25T10:22:55.144 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.33645.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.33645.log.gz 2024-04-25T10:22:55.145 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98003.log 2024-04-25T10:22:55.146 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82507.log: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.82507.log.gz 2024-04-25T10:22:55.147 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.98003.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.98411.log 2024-04-25T10:22:55.147 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.98003.log.gz 2024-04-25T10:22:55.148 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63751.log 2024-04-25T10:22:55.150 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98411.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98411.log.gz 2024-04-25T10:22:55.151 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.0.63751.log: --verbose -- /var/log/ceph/ceph-client.admin.65932.log 2024-04-25T10:22:55.151 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.63751.log.gz 2024-04-25T10:22:55.152 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55718.log 2024-04-25T10:22:55.153 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65932.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65932.log.gz 2024-04-25T10:22:55.154 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46540.log 2024-04-25T10:22:55.155 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55718.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.55718.log.gz 2024-04-25T10:22:55.156 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75262.log 2024-04-25T10:22:55.157 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46540.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46540.log.gz 2024-04-25T10:22:55.158 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57096.log 2024-04-25T10:22:55.159 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.75262.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.75262.log.gz 2024-04-25T10:22:55.160 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95535.log 2024-04-25T10:22:55.161 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57096.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.57096.log.gz 2024-04-25T10:22:55.162 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98869.log 2024-04-25T10:22:55.163 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95535.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.95535.log.gz 2024-04-25T10:22:55.164 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98869.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93882.log 2024-04-25T10:22:55.164 INFO:teuthology.orchestra.run.smithi165.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98869.log.gz 2024-04-25T10:22:55.165 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60967.log 2024-04-25T10:22:55.167 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93882.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.93882.log.gz 2024-04-25T10:22:55.167 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100790.log 2024-04-25T10:22:55.168 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60967.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.60967.log.gz 2024-04-25T10:22:55.169 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101742.log 2024-04-25T10:22:55.170 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100790.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.100790.log.gz 2024-04-25T10:22:55.171 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.101742.log: --verbose -- /var/log/ceph/ceph-client.admin.45210.log 2024-04-25T10:22:55.172 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.101742.log.gz 2024-04-25T10:22:55.174 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45210.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45210.log.gz 2024-04-25T10:22:55.174 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87523.log 2024-04-25T10:22:55.176 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93924.log 2024-04-25T10:22:55.178 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87523.log: 90.7% -- replaced with /var/log/ceph/ceph-client.admin.87523.log.gz 2024-04-25T10:22:55.178 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63173.log 2024-04-25T10:22:55.178 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.93924.log.gz 2024-04-25T10:22:55.179 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94850.log 2024-04-25T10:22:55.180 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63173.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.63173.log.gz 2024-04-25T10:22:55.181 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60883.log 2024-04-25T10:22:55.182 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94850.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94850.log.gz 2024-04-25T10:22:55.183 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98827.log 2024-04-25T10:22:55.184 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60883.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.60883.log.gz 2024-04-25T10:22:55.185 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47985.log 2024-04-25T10:22:55.186 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98827.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.98827.log.gz 2024-04-25T10:22:55.187 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90229.log 2024-04-25T10:22:55.187 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47985.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.47985.log.gz 2024-04-25T10:22:55.188 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56545.log 2024-04-25T10:22:55.189 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90229.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.90229.log.gz 2024-04-25T10:22:55.190 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47901.log 2024-04-25T10:22:55.191 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56545.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.56545.log.gz 2024-04-25T10:22:55.192 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.47901.log: --verbose -- /var/log/ceph/ceph-client.admin.104427.log 2024-04-25T10:22:55.193 INFO:teuthology.orchestra.run.smithi165.stderr: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.47901.log.gz 2024-04-25T10:22:55.194 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112523.log 2024-04-25T10:22:55.195 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104427.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.104427.log.gz 2024-04-25T10:22:55.196 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.112523.log: /var/log/ceph/ceph-client.admin.79565.log 2024-04-25T10:22:55.196 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112523.log.gz 2024-04-25T10:22:55.197 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75178.log 2024-04-25T10:22:55.198 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79565.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79565.log.gz 2024-04-25T10:22:55.200 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.75178.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.75178.log.gz 2024-04-25T10:22:55.200 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42110.log 2024-04-25T10:22:55.202 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93060.log 2024-04-25T10:22:55.203 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.42110.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.42110.log.gz 2024-04-25T10:22:55.204 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112439.log 2024-04-25T10:22:55.205 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93060.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.93060.log.gz 2024-04-25T10:22:55.206 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56274.log 2024-04-25T10:22:55.206 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112439.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.112439.log.gz 2024-04-25T10:22:55.208 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56274.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.56274.log.gz 2024-04-25T10:22:55.208 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79691.log 2024-04-25T10:22:55.210 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97487.log 2024-04-25T10:22:55.211 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79691.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.79691.log.gz 2024-04-25T10:22:55.212 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102036.log 2024-04-25T10:22:55.213 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.97487.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.97487.log.gz 2024-04-25T10:22:55.214 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65596.log 2024-04-25T10:22:55.214 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102036.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.102036.log.gz 2024-04-25T10:22:55.215 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44772.log 2024-04-25T10:22:55.216 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65596.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.65596.log.gz 2024-04-25T10:22:55.217 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45588.log 2024-04-25T10:22:55.218 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44772.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44772.log.gz 2024-04-25T10:22:55.220 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109788.log 2024-04-25T10:22:55.220 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45588.log.gz 2024-04-25T10:22:55.221 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83914.log 2024-04-25T10:22:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.109788.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.109788.log.gz 2024-04-25T10:22:55.223 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89296.log/var/log/ceph/ceph-client.admin.83914.log: 2024-04-25T10:22:55.224 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.83914.log.gz 2024-04-25T10:22:55.225 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61261.log 2024-04-25T10:22:55.226 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89296.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.89296.log.gz 2024-04-25T10:22:55.227 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74908.log 2024-04-25T10:22:55.228 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61261.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.61261.log.gz 2024-04-25T10:22:55.228 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107055.log 2024-04-25T10:22:55.231 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74908.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74908.log.gz 2024-04-25T10:22:55.231 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66124.log 2024-04-25T10:22:55.231 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107055.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.107055.log.gz 2024-04-25T10:22:55.232 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44478.log 2024-04-25T10:22:55.233 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.66124.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.66124.log.gz 2024-04-25T10:22:55.235 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.44478.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44478.log.gz 2024-04-25T10:22:55.235 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61773.log 2024-04-25T10:22:55.237 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42861.log 2024-04-25T10:22:55.239 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61773.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61773.log.gz 2024-04-25T10:22:55.239 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.42861.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.60589.log 2024-04-25T10:22:55.239 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.42861.log.gz 2024-04-25T10:22:55.241 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75328.log 2024-04-25T10:22:55.242 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60589.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.60589.log.gz 2024-04-25T10:22:55.243 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.75328.log: /var/log/ceph/ceph-client.admin.106971.log 2024-04-25T10:22:55.243 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.75328.log.gz 2024-04-25T10:22:55.244 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101406.log 2024-04-25T10:22:55.245 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106971.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.106971.log.gz 2024-04-25T10:22:55.246 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63014.log/var/log/ceph/ceph-client.admin.101406.log: 2024-04-25T10:22:55.247 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.101406.log.gz 2024-04-25T10:22:55.248 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94945.log 2024-04-25T10:22:55.250 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.63014.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.63014.log.gz 2024-04-25T10:22:55.250 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111977.log/var/log/ceph/ceph-client.admin.94945.log: 2024-04-25T10:22:55.250 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94945.log.gz 2024-04-25T10:22:55.251 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102998.log 2024-04-25T10:22:55.253 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111977.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111977.log.gz 2024-04-25T10:22:55.254 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.102998.log: --verbose -- /var/log/ceph/ceph-client.admin.60841.log 2024-04-25T10:22:55.254 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102998.log.gz 2024-04-25T10:22:55.255 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45252.log 2024-04-25T10:22:55.256 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60841.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60841.log.gz 2024-04-25T10:22:55.257 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101007.log 2024-04-25T10:22:55.258 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45252.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58788.log 2024-04-25T10:22:55.259 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45252.log.gz 2024-04-25T10:22:55.260 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101007.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.101007.log.gz 2024-04-25T10:22:55.262 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58788.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.58788.log.gz 2024-04-25T10:22:55.262 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46839.log 2024-04-25T10:22:55.264 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.83022.log 2024-04-25T10:22:55.265 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46839.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46839.log.gz 2024-04-25T10:22:55.266 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.83022.log: /var/log/ceph/ceph-client.admin.45783.log 2024-04-25T10:22:55.266 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.83022.log.gz 2024-04-25T10:22:55.268 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-04-25T10:22:55.269 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45783.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45783.log.gz 2024-04-25T10:22:55.270 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose --/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.102545.log 2024-04-25T10:22:55.272 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102545.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61009.log 2024-04-25T10:22:55.273 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.102545.log.gz 2024-04-25T10:22:55.275 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88302.log 2024-04-25T10:22:55.277 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56808.log 2024-04-25T10:22:55.277 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88302.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88302.log.gz 2024-04-25T10:22:55.278 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.61009.log.gz 2024-04-25T10:22:55.279 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41085.log 2024-04-25T10:22:55.280 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56808.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56808.log.gz 2024-04-25T10:22:55.283 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41085.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.41085.log.gz 2024-04-25T10:22:55.284 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58626.log 2024-04-25T10:22:55.286 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.58626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105077.log 2024-04-25T10:22:55.287 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.58626.log.gz 2024-04-25T10:22:55.289 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92904.log 2024-04-25T10:22:55.291 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105077.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77870.log 2024-04-25T10:22:55.292 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92904.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.105077.log.gz 2024-04-25T10:22:55.293 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.92904.log.gz 2024-04-25T10:22:55.295 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.77870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61440.log 2024-04-25T10:22:55.299 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61440.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.77870.log.gz 2024-04-25T10:22:55.299 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55938.log 2024-04-25T10:22:55.300 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.61440.log.gz 2024-04-25T10:22:55.302 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48420.log 2024-04-25T10:22:55.303 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47733.log 2024-04-25T10:22:55.305 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55938.log: /var/log/ceph/ceph-client.admin.48420.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.48420.log.gz 2024-04-25T10:22:55.307 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.55938.log.gz 2024-04-25T10:22:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47733.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47733.log.gz 2024-04-25T10:22:55.309 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39268.log 2024-04-25T10:22:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64004.log 2024-04-25T10:22:55.315 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.39268.log.gz 2024-04-25T10:22:55.315 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87589.log 2024-04-25T10:22:55.319 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64004.log: /var/log/ceph/ceph-client.admin.87589.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78613.log 2024-04-25T10:22:55.319 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64004.log.gz 2024-04-25T10:22:55.321 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100539.log 2024-04-25T10:22:55.321 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87589.log.gz 2024-04-25T10:22:55.323 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.100539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100539.log.gz 2024-04-25T10:22:55.325 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107283.log 2024-04-25T10:22:55.326 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.78613.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61731.log 2024-04-25T10:22:55.327 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.78613.log.gz 2024-04-25T10:22:55.329 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107283.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107283.log.gz 2024-04-25T10:22:55.329 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94722.log 2024-04-25T10:22:55.332 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56587.log 2024-04-25T10:22:55.333 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.94722.log: gzip -5 --verbose 89.5% -- replaced with /var/log/ceph/ceph-client.admin.94722.log.gz 2024-04-25T10:22:55.333 INFO:teuthology.orchestra.run.smithi165.stderr: -- /var/log/ceph/ceph-client.admin.57310.log 2024-04-25T10:22:55.335 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56587.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.56587.log.gz 2024-04-25T10:22:55.335 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98495.log 2024-04-25T10:22:55.336 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57310.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.57310.log.gz 2024-04-25T10:22:55.337 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61731.log.gz 2024-04-25T10:22:55.339 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108978.log 2024-04-25T10:22:55.342 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98495.log: /var/log/ceph/ceph-client.admin.108978.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108978.log.gz 2024-04-25T10:22:55.343 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110490.log 2024-04-25T10:22:55.343 INFO:teuthology.orchestra.run.smithi165.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.98495.log.gz 2024-04-25T10:22:55.344 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111534.log 2024-04-25T10:22:55.346 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110490.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45336.log 2024-04-25T10:22:55.348 INFO:teuthology.orchestra.run.smithi165.stderr: 90.8%/var/log/ceph/ceph-client.admin.111534.log: -- replaced with /var/log/ceph/ceph-client.admin.110490.log.gz 2024-04-25T10:22:55.348 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79943.log 2024-04-25T10:22:55.349 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45336.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111534.log.gz 2024-04-25T10:22:55.352 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79943.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48630.log 2024-04-25T10:22:55.353 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45336.log.gz 2024-04-25T10:22:55.356 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48630.log: 88.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61974.log 2024-04-25T10:22:55.356 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.79943.log.gz 2024-04-25T10:22:55.357 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.48630.log.gz 2024-04-25T10:22:55.359 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59477.log 2024-04-25T10:22:55.362 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.59477.log: /var/log/ceph/ceph-client.admin.61974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61303.log 2024-04-25T10:22:55.365 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74048.log 2024-04-25T10:22:55.365 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59477.log.gz 2024-04-25T10:22:55.365 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.61974.log.gz 2024-04-25T10:22:55.366 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61303.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.61303.log.gz 2024-04-25T10:22:55.367 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.103423.log 2024-04-25T10:22:55.368 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74048.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.74048.log.gz 2024-04-25T10:22:55.371 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.103423.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.103423.log.gz 2024-04-25T10:22:55.372 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99802.log 2024-04-25T10:22:55.374 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39653.log 2024-04-25T10:22:55.375 INFO:teuthology.orchestra.run.smithi165.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.99802.log.gz 2024-04-25T10:22:55.377 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95792.log 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39653.log: /var/log/ceph/ceph-client.0.95792.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40435.log 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.39653.log.gz 2024-04-25T10:22:55.382 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.95792.log.gz 2024-04-25T10:22:55.384 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95788.log 2024-04-25T10:22:55.386 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72845.log 2024-04-25T10:22:55.388 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40435.log: /var/log/ceph/ceph-client.0.95788.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.95788.log.gz 2024-04-25T10:22:55.389 INFO:teuthology.orchestra.run.smithi165.stderr: 92.5% -- replaced with /var/log/ceph/ceph-client.admin.40435.log.gz 2024-04-25T10:22:55.390 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62100.log 2024-04-25T10:22:55.395 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.72845.log: /var/log/ceph/ceph-client.admin.62100.log: 90.9% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.72845.log.gz 2024-04-25T10:22:55.395 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74090.log 2024-04-25T10:22:55.395 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62100.log.gz 2024-04-25T10:22:55.396 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90570.log 2024-04-25T10:22:55.398 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74090.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41475.log 2024-04-25T10:22:55.400 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.90570.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.74090.log.gz 2024-04-25T10:22:55.401 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88606.log 2024-04-25T10:22:55.401 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.90570.log.gz 2024-04-25T10:22:55.402 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108547.log 2024-04-25T10:22:55.403 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82315.log 2024-04-25T10:22:55.406 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6%/var/log/ceph/ceph-client.admin.108547.log: /var/log/ceph/ceph-client.admin.88606.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.88606.log.gz 2024-04-25T10:22:55.406 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41475.log.gz 2024-04-25T10:22:55.408 INFO:teuthology.orchestra.run.smithi165.stderr:gzip 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108547.log.gz 2024-04-25T10:22:55.408 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.82315.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.73712.log 2024-04-25T10:22:55.408 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.82315.log.gz 2024-04-25T10:22:55.410 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.73712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111408.log 2024-04-25T10:22:55.411 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.73712.log.gz 2024-04-25T10:22:55.413 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65806.log 2024-04-25T10:22:55.417 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111408.log: /var/log/ceph/ceph-client.admin.65806.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41764.log 2024-04-25T10:22:55.417 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.111408.log.gz 2024-04-25T10:22:55.418 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65806.log.gz 2024-04-25T10:22:55.420 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57959.log 2024-04-25T10:22:55.421 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101616.log 2024-04-25T10:22:55.423 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9%/var/log/ceph/ceph-client.0.57959.log: -- replaced with /var/log/ceph/ceph-client.admin.41764.log.gz 2024-04-25T10:22:55.424 INFO:teuthology.orchestra.run.smithi165.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.57959.log.gz 2024-04-25T10:22:55.426 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65083.log 2024-04-25T10:22:55.430 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101616.log: /var/log/ceph/ceph-client.admin.65083.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57664.log 2024-04-25T10:22:55.430 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101616.log.gz 2024-04-25T10:22:55.431 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.65083.log.gz 2024-04-25T10:22:55.433 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44604.log 2024-04-25T10:22:55.437 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57664.log: /var/log/ceph/ceph-client.admin.44604.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47775.log 2024-04-25T10:22:55.438 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57664.log.gz 2024-04-25T10:22:55.438 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44604.log.gz 2024-04-25T10:22:55.440 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74866.log 2024-04-25T10:22:55.444 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.47775.log: /var/log/ceph/ceph-client.admin.74866.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56461.log 2024-04-25T10:22:55.444 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47775.log.gz 2024-04-25T10:22:55.446 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.74866.log.gz 2024-04-25T10:22:55.448 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56461.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90103.log 2024-04-25T10:22:55.450 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.90103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106124.log 2024-04-25T10:22:55.451 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.56461.log.gz 2024-04-25T10:22:55.452 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64957.log 2024-04-25T10:22:55.453 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102120.log 2024-04-25T10:22:55.454 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106124.log: -- replaced with /var/log/ceph/ceph-client.admin.90103.log.gz 2024-04-25T10:22:55.455 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106124.log.gz 2024-04-25T10:22:55.459 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64957.log: /var/log/ceph/ceph-client.admin.102120.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102120.log.gz 2024-04-25T10:22:55.461 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.64957.log.gz 2024-04-25T10:22:55.461 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55980.log 2024-04-25T10:22:55.463 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95367.log 2024-04-25T10:22:55.467 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95367.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58210.log 2024-04-25T10:22:55.467 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.55980.log.gz 2024-04-25T10:22:55.468 INFO:teuthology.orchestra.run.smithi165.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95367.log.gz 2024-04-25T10:22:55.470 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106251.log 2024-04-25T10:22:55.474 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.106251.log: /var/log/ceph/ceph-client.admin.58210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61177.log 2024-04-25T10:22:55.475 INFO:teuthology.orchestra.run.smithi165.stderr: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111011.log 2024-04-25T10:22:55.475 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58210.log.gz 2024-04-25T10:22:55.477 INFO:teuthology.orchestra.run.smithi165.stderr: 87.8%/var/log/ceph/ceph-client.admin.111011.log: -- replaced with /var/log/ceph/ceph-client.admin.106251.log.gz 2024-04-25T10:22:55.478 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.111011.log.gz 2024-04-25T10:22:55.479 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93410.log 2024-04-25T10:22:55.480 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.61177.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.61177.log.gz 2024-04-25T10:22:55.482 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107577.log 2024-04-25T10:22:55.483 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93410.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.93410.log.gz 2024-04-25T10:22:55.485 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107577.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107577.log.gz 2024-04-25T10:22:55.486 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45294.log 2024-04-25T10:22:55.488 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64308.log 2024-04-25T10:22:55.489 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.45294.log.gz 2024-04-25T10:22:55.491 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103955.log 2024-04-25T10:22:55.496 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64308.log: /var/log/ceph/ceph-client.admin.103955.log: gzip 88.9% -5 --verbose -- /var/log/ceph/ceph-client.admin.46797.log 2024-04-25T10:22:55.496 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.64308.log.gz 2024-04-25T10:22:55.497 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.103955.log.gz 2024-04-25T10:22:55.499 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98911.log 2024-04-25T10:22:55.504 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98911.log: /var/log/ceph/ceph-client.admin.46797.log: 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62811.log 2024-04-25T10:22:55.505 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46797.log.gz 2024-04-25T10:22:55.506 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98911.log.gz 2024-04-25T10:22:55.506 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74473.log 2024-04-25T10:22:55.508 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74473.log: 89.5% -- replaced with /var/log/ceph/ceph-client.admin.74473.log.gz 2024-04-25T10:22:55.508 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98705.log 2024-04-25T10:22:55.510 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62811.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.62811.log.gz 2024-04-25T10:22:55.513 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98705.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.98705.log.gz 2024-04-25T10:22:55.514 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104951.log 2024-04-25T10:22:55.516 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.104951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46965.log 2024-04-25T10:22:55.518 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112187.log 2024-04-25T10:22:55.518 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.104951.log.gz/var/log/ceph/ceph-client.admin.46965.log: 2024-04-25T10:22:55.519 INFO:teuthology.orchestra.run.smithi165.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46965.log.gz 2024-04-25T10:22:55.522 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112187.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112187.log.gz 2024-04-25T10:22:55.522 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74431.log 2024-04-25T10:22:55.525 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.74431.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.74431.log.gz 2024-04-25T10:22:55.525 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98190.log 2024-04-25T10:22:55.527 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.98190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95073.log 2024-04-25T10:22:55.531 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.95073.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.98190.log.gz 2024-04-25T10:22:55.531 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.102675.log 2024-04-25T10:22:55.532 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95073.log.gz 2024-04-25T10:22:55.533 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89721.log 2024-04-25T10:22:55.536 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89721.log: /var/log/ceph/ceph-client.admin.102675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64392.log 2024-04-25T10:22:55.538 INFO:teuthology.orchestra.run.smithi165.stderr: 89.0% 88.4% -- replaced with /var/log/ceph/ceph-client.admin.102675.log.gz 2024-04-25T10:22:55.538 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89721.log.gz 2024-04-25T10:22:55.540 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45546.log 2024-04-25T10:22:55.543 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45546.log: /var/log/ceph/ceph-client.admin.64392.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111305.log 2024-04-25T10:22:55.545 INFO:teuthology.orchestra.run.smithi165.stderr: 88.5% 89.6% -- replaced with /var/log/ceph/ceph-client.admin.64392.log.gz 2024-04-25T10:22:55.546 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45546.log.gz 2024-04-25T10:22:55.547 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45000.log 2024-04-25T10:22:55.550 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45000.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110340.log 2024-04-25T10:22:55.551 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111305.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88690.log 2024-04-25T10:22:55.552 INFO:teuthology.orchestra.run.smithi165.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111305.log.gz 2024-04-25T10:22:55.552 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110340.log: -- replaced with /var/log/ceph/ceph-client.admin.45000.log.gz 2024-04-25T10:22:55.554 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.110340.log.gz 2024-04-25T10:22:55.555 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60925.log 2024-04-25T10:22:55.557 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.88690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109584.log 2024-04-25T10:22:55.559 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.88690.log.gz 2024-04-25T10:22:55.560 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.60925.log: /var/log/ceph/ceph-client.admin.109584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107199.log 2024-04-25T10:22:55.561 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% 88.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40733.log 2024-04-25T10:22:55.561 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60925.log.gz 2024-04-25T10:22:55.563 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107199.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107199.log.gz 2024-04-25T10:22:55.564 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.109584.log.gz 2024-04-25T10:22:55.567 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.40733.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.40733.log.gz 2024-04-25T10:22:55.568 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107679.log 2024-04-25T10:22:55.571 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80093.log 2024-04-25T10:22:55.574 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.80093.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.107679.log.gz 2024-04-25T10:22:55.574 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111935.log 2024-04-25T10:22:55.575 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92674.log 2024-04-25T10:22:55.578 INFO:teuthology.orchestra.run.smithi165.stderr: 89.2%/var/log/ceph/ceph-client.admin.111935.log: -- replaced with /var/log/ceph/ceph-client.admin.80093.log.gz 2024-04-25T10:22:55.579 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111935.log.gz 2024-04-25T10:22:55.582 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.92674.log: 89.1%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46626.log 2024-04-25T10:22:55.583 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.92674.log.gz 2024-04-25T10:22:55.584 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48588.log 2024-04-25T10:22:55.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.46626.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46626.log.gz 2024-04-25T10:22:55.588 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.48588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102872.log 2024-04-25T10:22:55.589 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.48588.log.gz 2024-04-25T10:22:55.592 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.102872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79481.log 2024-04-25T10:22:55.594 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.79481.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102872.log.gz 2024-04-25T10:22:55.595 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106443.log 2024-04-25T10:22:55.596 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.79481.log.gz 2024-04-25T10:22:55.598 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93546.log 2024-04-25T10:22:55.602 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.93546.log: /var/log/ceph/ceph-client.admin.106443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45378.log 2024-04-25T10:22:55.604 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106443.log.gz 2024-04-25T10:22:55.604 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.93546.log.gz 2024-04-25T10:22:55.605 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56888.log 2024-04-25T10:22:55.608 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87229.log 2024-04-25T10:22:55.611 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.56888.log: 88.6% 88.2% -- replaced with /var/log/ceph/ceph-client.admin.45378.log.gz 2024-04-25T10:22:55.611 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56888.log.gz 2024-04-25T10:22:55.613 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64350.log 2024-04-25T10:22:55.616 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.64350.log: /var/log/ceph/ceph-client.admin.87229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39437.log 2024-04-25T10:22:55.618 INFO:teuthology.orchestra.run.smithi165.stderr: 89.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100240.log 2024-04-25T10:22:55.618 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.87229.log.gz 2024-04-25T10:22:55.620 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.100240.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.64350.log.gz 2024-04-25T10:22:55.622 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.100240.log.gz 2024-04-25T10:22:55.622 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.87187.log 2024-04-25T10:22:55.623 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.39437.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.39437.log.gz 2024-04-25T10:22:55.625 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41601.log 2024-04-25T10:22:55.625 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73460.log 2024-04-25T10:22:55.627 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45741.log 2024-04-25T10:22:55.628 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.87187.log: /var/log/ceph/ceph-client.admin.73460.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.87187.log.gz 2024-04-25T10:22:55.629 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.41601.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.73460.log.gz 2024-04-25T10:22:55.629 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.45741.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.41601.log.gz 2024-04-25T10:22:55.630 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62352.log 2024-04-25T10:22:55.630 INFO:teuthology.orchestra.run.smithi165.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45741.log.gz 2024-04-25T10:22:55.632 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47817.log 2024-04-25T10:22:55.633 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.109710.log 2024-04-25T10:22:55.636 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.62352.log: /var/log/ceph/ceph-client.admin.109710.log: /var/log/ceph/ceph-client.admin.47817.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.109710.log.gz 2024-04-25T10:22:55.638 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47817.log.gz 2024-04-25T10:22:55.638 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62352.log.gz 2024-04-25T10:22:55.639 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65552.log 2024-04-25T10:22:55.641 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65552.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111450.log 2024-04-25T10:22:55.642 INFO:teuthology.orchestra.run.smithi165.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.65552.log.gz 2024-04-25T10:22:55.644 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55449.log 2024-04-25T10:22:55.648 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.111450.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89513.log 2024-04-25T10:22:55.649 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.55449.log: -- replaced with /var/log/ceph/ceph-client.admin.111450.log.gz 2024-04-25T10:22:55.650 INFO:teuthology.orchestra.run.smithi165.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.55449.log.gz 2024-04-25T10:22:55.652 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.88165.log 2024-04-25T10:22:55.656 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.89513.log: /var/log/ceph/ceph-client.admin.88165.log: 87.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98105.log 2024-04-25T10:22:55.657 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.89513.log.gz 2024-04-25T10:22:55.658 INFO:teuthology.orchestra.run.smithi165.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.88165.log.gz 2024-04-25T10:22:55.660 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.admin.98105.log: --verbose -- /var/log/ceph/ceph-client.admin.65890.log 2024-04-25T10:22:55.664 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.65890.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.85099.log 2024-04-25T10:22:55.664 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.98105.log.gz 2024-04-25T10:22:55.665 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.65890.log.gz 2024-04-25T10:22:55.666 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101574.log 2024-04-25T10:22:55.669 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.101574.log: /var/log/ceph/ceph-client.admin.85099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110867.log 2024-04-25T10:22:55.671 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3% 88.5% -- replaced with /var/log/ceph/ceph-client.admin.85099.log.gz 2024-04-25T10:22:55.671 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.101574.log.gz 2024-04-25T10:22:55.673 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99216.log 2024-04-25T10:22:55.676 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.99216.log: /var/log/ceph/ceph-client.admin.110867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65508.log 2024-04-25T10:22:55.678 INFO:teuthology.orchestra.run.smithi165.stderr: 88.4% 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110867.log.gz 2024-04-25T10:22:55.678 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.99216.log.gz 2024-04-25T10:22:55.680 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112607.log 2024-04-25T10:22:55.684 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.112607.log: /var/log/ceph/ceph-client.admin.65508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98453.log 2024-04-25T10:22:55.685 INFO:teuthology.orchestra.run.smithi165.stderr: 88.9% 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112607.log.gz 2024-04-25T10:22:55.686 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65508.log.gz 2024-04-25T10:22:55.688 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106208.log 2024-04-25T10:22:55.690 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.admin.98453.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.57352.log 2024-04-25T10:22:55.691 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.92716.log 2024-04-25T10:22:55.695 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3%/var/log/ceph/ceph-client.admin.106208.log: 88.3%/var/log/ceph/ceph-client.admin.92716.log: -- replaced with /var/log/ceph/ceph-client.admin.98453.log.gz 2024-04-25T10:22:55.695 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.106208.log.gz 2024-04-25T10:22:55.697 INFO:teuthology.orchestra.run.smithi165.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.92716.log.gz 2024-04-25T10:22:55.698 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57352.log: 88.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110706.log 2024-04-25T10:22:55.700 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.110706.log: -- replaced with /var/log/ceph/ceph-client.admin.57352.log.gz 2024-04-25T10:22:55.701 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105378.log 2024-04-25T10:22:55.702 INFO:teuthology.orchestra.run.smithi165.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110706.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57538.log 2024-04-25T10:22:55.702 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-25T10:22:55.703 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93840.log 2024-04-25T10:22:55.706 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57538.log: /var/log/ceph/ceph-client.admin.93840.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57538.log.gz 2024-04-25T10:22:55.708 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.105378.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.93840.log.gz 2024-04-25T10:22:55.709 INFO:teuthology.orchestra.run.smithi165.stderr: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57622.log 2024-04-25T10:22:55.712 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.105378.log.gz 2024-04-25T10:22:55.712 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.57622.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.57622.log.gz 2024-04-25T10:22:55.712 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107367.log 2024-04-25T10:22:55.714 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.82843.log 2024-04-25T10:22:55.719 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.107367.log: /var/log/ceph/ceph-client.admin.82843.log: 87.7% 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107367.log.gz 2024-04-25T10:22:55.719 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.82843.log.gz 2024-04-25T10:22:56.244 INFO:teuthology.orchestra.run.smithi165.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-25T10:22:58.010 INFO:teuthology.orchestra.run.smithi165.stderr: 94.7% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-04-25T10:22:59.059 INFO:teuthology.orchestra.run.smithi172.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-25T10:22:59.146 INFO:teuthology.orchestra.run.smithi172.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-25T10:23:00.336 INFO:teuthology.orchestra.run.smithi165.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-25T10:23:00.475 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-25T10:23:00.869 INFO:teuthology.orchestra.run.smithi172.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-25T10:23:01.178 INFO:teuthology.orchestra.run.smithi165.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-25T10:23:01.208 INFO:teuthology.orchestra.run.smithi172.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-25T10:23:01.257 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-25T10:23:16.932 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-04-25T10:23:16.935 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-25T10:23:16.936 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m23.090s 2024-04-25T10:23:16.936 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m50.486s 2024-04-25T10:23:16.936 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m6.438s 2024-04-25T10:23:18.022 INFO:teuthology.orchestra.run.smithi172.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-04-25T10:23:18.026 INFO:teuthology.orchestra.run.smithi172.stderr: 2024-04-25T10:23:18.027 INFO:teuthology.orchestra.run.smithi172.stderr:real 0m24.174s 2024-04-25T10:23:18.027 INFO:teuthology.orchestra.run.smithi172.stderr:user 0m47.436s 2024-04-25T10:23:18.027 INFO:teuthology.orchestra.run.smithi172.stderr:sys 0m3.633s 2024-04-25T10:23:18.028 INFO:tasks.ceph:Archiving logs... 2024-04-25T10:23:18.028 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/remote/smithi165/log 2024-04-25T10:23:18.029 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:23:25.826 DEBUG:teuthology.misc:Transferring archived files from smithi172:/var/log/ceph to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/remote/smithi172/log 2024-04-25T10:23:25.827 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-25T10:23:32.808 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-25T10:23:32.819 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-25T10:23:32.819 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:23:32.830 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-25T10:23:32.981 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-25T10:23:32.996 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-25T10:23:32.996 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:23:33.004 DEBUG:teuthology.orchestra.run.smithi172:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-25T10:23:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:23:33.059 INFO:teuthology.orchestra.run.smithi172.stderr:bash: line 1: ntpq: command not found 2024-04-25T10:23:33.060 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:23:33.060 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-25T10:23:33.061 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 7 377 1 +1151us[+1118us] +/- 50ms 2024-04-25T10:23:33.061 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 62 -978us[-1014us] +/- 55ms 2024-04-25T10:23:33.061 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 1 -663us[ -697us] +/- 70ms 2024-04-25T10:23:33.061 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:23:33.069 INFO:teuthology.orchestra.run.smithi172.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-25T10:23:33.069 INFO:teuthology.orchestra.run.smithi172.stdout:=============================================================================== 2024-04-25T10:23:33.069 INFO:teuthology.orchestra.run.smithi172.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +1177us[+1182us] +/- 50ms 2024-04-25T10:23:33.069 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 3 -946us[ -940us] +/- 56ms 2024-04-25T10:23:33.069 INFO:teuthology.orchestra.run.smithi172.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 63 -611us[ -604us] +/- 69ms 2024-04-25T10:23:33.069 INFO:teuthology.orchestra.run.smithi172.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-25T10:23:33.071 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-25T10:23:33.082 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-25T10:23:33.082 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-25T10:23:33.092 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:23:33.316 DEBUG:teuthology.orchestra.run.smithi172:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-25T10:23:33.493 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:23:33.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:23:33.590 DEBUG:teuthology.orchestra.run.smithi172:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-25T10:23:33.687 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:23:33.687 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-25T10:23:33.697 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-25T10:23:33.706 INFO:teuthology.task.internal:Duration was 1830.942072 seconds 2024-04-25T10:23:33.707 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-25T10:23:33.716 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-25T10:23:33.716 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:23:33.725 DEBUG:teuthology.orchestra.run.smithi172:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-25T10:23:33.861 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:23:33.873 INFO:teuthology.orchestra.run.smithi172.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-25T10:23:34.251 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-25T10:23:34.251 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-04-25T10:23:34.251 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:23:34.311 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi172.front.sepia.ceph.com 2024-04-25T10:23:34.312 DEBUG:teuthology.orchestra.run.smithi172:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-25T10:23:34.384 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-25T10:23:34.385 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:23:34.392 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-25T10:23:34.478 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-25T10:23:34.478 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:23:34.483 DEBUG:teuthology.orchestra.run.smithi172:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-25T10:23:34.703 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-25T10:23:34.713 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-25T10:23:34.713 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:23:34.797 DEBUG:teuthology.orchestra.run.smithi172:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-25T10:23:34.887 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-25T10:23:34.898 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:23:34.906 DEBUG:teuthology.orchestra.run.smithi172:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:23:35.001 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-04-25T10:23:35.004 INFO:teuthology.orchestra.run.smithi172.stdout:kernel.core_pattern = core 2024-04-25T10:23:35.047 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:23:35.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:23:35.094 DEBUG:teuthology.orchestra.run.smithi172:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-25T10:23:35.147 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-25T10:23:35.148 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-25T10:23:35.158 INFO:teuthology.task.internal:Transferring archived files... 2024-04-25T10:23:35.158 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/remote/smithi165 2024-04-25T10:23:35.159 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:23:35.257 DEBUG:teuthology.misc:Transferring archived files from smithi172:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2024-04-25_09:36:50-fs:functional-wip-neesingh-testing-240416-distro-default-smithi/7673043/remote/smithi172 2024-04-25T10:23:35.257 DEBUG:teuthology.orchestra.run.smithi172:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-25T10:23:35.350 INFO:teuthology.task.internal:Removing archive directory... 2024-04-25T10:23:35.350 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:23:35.358 DEBUG:teuthology.orchestra.run.smithi172:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-25T10:23:35.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-25T10:23:35.425 INFO:teuthology.task.internal:Not uploading archives. 2024-04-25T10:23:35.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-25T10:23:35.435 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-25T10:23:35.435 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:23:35.441 DEBUG:teuthology.orchestra.run.smithi172:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-25T10:23:35.488 INFO:teuthology.orchestra.run.smithi165.stdout: 393949 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:23 /home/ubuntu/cephtest 2024-04-25T10:23:35.494 INFO:teuthology.orchestra.run.smithi172.stdout: 393229 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 25 10:23 /home/ubuntu/cephtest 2024-04-25T10:23:35.498 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-25T10:23:35.508 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-25T10:23:35.531 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 client.1-kernel-sha1: 806ec36b21129f113ce356c007dd59694d340b37 description: fs:functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} subvol_versions/create_subvol_version_v1 tasks/scrub} duration: 1830.9420716762543 flavor: default owner: scheduled_neesingh@teuthology success: true 2024-04-25T10:23:35.531 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-25T10:23:35.616 INFO:teuthology.run:pass